From xen-devel-bounces@lists.xenproject.org Wed Sep 01 01:46:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 01:46:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176016.320410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLFK9-0004Hw-VL; Wed, 01 Sep 2021 01:45:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176016.320410; Wed, 01 Sep 2021 01:45:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLFK9-0004Hp-Rp; Wed, 01 Sep 2021 01:45:57 +0000
Received: by outflank-mailman (input) for mailman id 176016;
 Wed, 01 Sep 2021 01:45:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLFK7-0004He-Sa; Wed, 01 Sep 2021 01:45:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLFK6-0001Qh-4R; Wed, 01 Sep 2021 01:45:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLFK5-00038J-OF; Wed, 01 Sep 2021 01:45:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLFK5-00037H-Nk; Wed, 01 Sep 2021 01:45:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jTxFEIE/vbYKWqwn6R+xp/VWJmkfz42sx0L1t9O7P/w=; b=1R7fjffL51nYLsJJAzWqptgaZk
	mQ1W72i4DZPK/ciOK3fyE0HvUKnhLwZIji2SP0a+pfySkFVu1YZNUrKx9gvYVRVLyG/RBXEsSTp4M
	u4JSKniKLaJBEkoTxma+theWMriADxWw/ImWgypE49x1rqkPfYe3qnduQWqfzewhcyg0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164677-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164677: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=dea67b1de03dd0544b8d05322f4300abd44221bd
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 01:45:53 +0000

flight 164677 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164677/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              dea67b1de03dd0544b8d05322f4300abd44221bd
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  417 days
Failing since        151818  2020-07-11 04:18:52 Z  416 days  407 attempts
Testing same since   164677  2021-08-31 06:51:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71401 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 03:02:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 03:02:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176029.320424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLGVi-0004Ja-S0; Wed, 01 Sep 2021 03:01:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176029.320424; Wed, 01 Sep 2021 03:01:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLGVi-0004JT-P0; Wed, 01 Sep 2021 03:01:58 +0000
Received: by outflank-mailman (input) for mailman id 176029;
 Wed, 01 Sep 2021 03:01:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLGVh-0004JJ-4j; Wed, 01 Sep 2021 03:01:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLGVg-0007rS-Q4; Wed, 01 Sep 2021 03:01:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLGVg-0005wc-Ec; Wed, 01 Sep 2021 03:01:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLGVg-00054f-E8; Wed, 01 Sep 2021 03:01:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8XobyDTf2lcU9muUOeEgU4L9AZs5529FnQYkcyxysuM=; b=PYihxzdfiY+wI3VTce7ppsF7a9
	vv6SuHrCXIO6JoZE8kDDgDWccJXh0luobnspQWPRz465EuZPuqv586J8V1vcyoT84Z277CQ5XIv5P
	8XdScBPNdlEl4ABI+h859rhQd/AwCY1MoNE0G1DLBmfSsHwPv/yUsg5eGbK0jZZAUvLk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164674-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164674: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
X-Osstest-Versions-That:
    ovmf=77d5fa80246e8784f89e109ff9dadfeb7089ff85
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 03:01:56 +0000

flight 164674 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164674/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63
baseline version:
 ovmf                 77d5fa80246e8784f89e109ff9dadfeb7089ff85

Last test of basis   164630  2021-08-30 01:10:45 Z    2 days
Testing same since   164674  2021-08-31 02:56:52 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Cheng Zhou <zhoucheng@phytium.com.cn>
  Grzegorz Bernacki <gjb@semihalf.com>
  Hao A Wu <hao.a.wu@intel.com>
  Marvin H?user <mhaeuser@posteo.de>
  zhoucheng <zhoucheng@phytium.com.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   77d5fa8024..cae735f613  cae735f61328d64e2e8991036707b9e78c0f5f63 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 03:42:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 03:42:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176039.320438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLH8m-0000VV-0Z; Wed, 01 Sep 2021 03:42:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176039.320438; Wed, 01 Sep 2021 03:42:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLH8l-0000VO-Tc; Wed, 01 Sep 2021 03:42:19 +0000
Received: by outflank-mailman (input) for mailman id 176039;
 Wed, 01 Sep 2021 03:42:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLH8k-0000VB-Ay; Wed, 01 Sep 2021 03:42:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLH8k-0008V5-61; Wed, 01 Sep 2021 03:42:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLH8j-0007Fw-RX; Wed, 01 Sep 2021 03:42:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLH8j-0002zx-R5; Wed, 01 Sep 2021 03:42:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3cRJIUgaV9rvsdo7JA4CClURwVqpLPVUdy9w7ieAcig=; b=wtul7LEUaaGdu5/nX2p+REDfYG
	O41OgS0YWmA8LtbRvDOYAIGHoqq9BTLcgZkRZWkw/7ITQzeco35IlFVuJHECvf69Z6vEcpI/KKmk4
	cNOhbLgwhIVZwDN/D1w2SfMxaM8iZgqca+LeqJEdlKeLnjo9bJPNYZsdsrN3kBrjio/Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164667-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164667: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:xen-boot:fail:regression
    xen-unstable:test-amd64-amd64-dom0pvh-xl-intel:xen-boot:fail:regression
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
X-Osstest-Versions-That:
    xen=a931e8e64af07bd333a31f3b71a3f8f3e7910857
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 03:42:17 +0000

flight 164667 xen-unstable real [real]
flight 164685 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164667/
http://logs.test-lab.xenproject.org/osstest/logs/164685/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-dom0pvh-xl-amd  8 xen-boot              fail REGR. vs. 164477
 test-amd64-amd64-dom0pvh-xl-intel  8 xen-boot            fail REGR. vs. 164477

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine      4 memdisk-try-append  fail pass in 164685-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164477
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164477
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164477
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164477
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164477
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164477
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164477
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164477
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164477
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164477
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164477
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
baseline version:
 xen                  a931e8e64af07bd333a31f3b71a3f8f3e7910857

Last test of basis   164477  2021-08-25 09:09:48 Z    6 days
Failing since        164499  2021-08-26 10:21:51 Z    5 days    4 attempts
Testing same since   164667  2021-08-30 22:13:35 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Julien Grall <julien@xen.org>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 518 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 04:18:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 04:18:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176048.320452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLHhN-0004GI-PL; Wed, 01 Sep 2021 04:18:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176048.320452; Wed, 01 Sep 2021 04:18:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLHhN-0004GB-Lr; Wed, 01 Sep 2021 04:18:05 +0000
Received: by outflank-mailman (input) for mailman id 176048;
 Wed, 01 Sep 2021 04:18:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLHhN-0004G2-Cl; Wed, 01 Sep 2021 04:18:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLHhN-0000jw-0P; Wed, 01 Sep 2021 04:18:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLHhM-0000HV-MY; Wed, 01 Sep 2021 04:18:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLHhM-0001g2-Lu; Wed, 01 Sep 2021 04:18:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HlPBMqJT5Hy667227W42ZoYKc8DgyCKANZT0u7RdrhY=; b=FTOVzXIi8HGa7kY8q2vCArwiOD
	qu+bQHq/EvbWagU+dqzYG5t//ntA6CospY6wMzjtDSoCnb2U21fb6nPdO/wEFajPfaxgOtu554EOe
	HkadZFX4lNpFD+XQjDTrtpRv0pUOG9hVqRdY7/whopxFn22FvQ/07rfvANF50Gr5BZ/4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164673-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164673: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8596e589b787732c8346f0482919e83cc9362db1
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 04:18:04 +0000

flight 164673 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164673/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                8596e589b787732c8346f0482919e83cc9362db1
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  396 days
Failing since        152366  2020-08-01 20:49:34 Z  395 days  683 attempts
Testing same since   164673  2021-08-30 23:59:57 Z    1 days    1 attempts

------------------------------------------------------------
6859 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2036695 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 04:50:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 04:50:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176057.320466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLICr-0000wt-Ft; Wed, 01 Sep 2021 04:50:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176057.320466; Wed, 01 Sep 2021 04:50:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLICr-0000wm-CZ; Wed, 01 Sep 2021 04:50:37 +0000
Received: by outflank-mailman (input) for mailman id 176057;
 Wed, 01 Sep 2021 04:50:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ocLH=NX=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mLICp-0000wf-KL
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 04:50:35 +0000
Received: from mail-lj1-x22d.google.com (unknown [2a00:1450:4864:20::22d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7942cd84-54e6-4fb6-86b6-aad24475e815;
 Wed, 01 Sep 2021 04:50:34 +0000 (UTC)
Received: by mail-lj1-x22d.google.com with SMTP id s3so2618941ljp.11
 for <xen-devel@lists.xenproject.org>; Tue, 31 Aug 2021 21:50:34 -0700 (PDT)
Received: from [192.168.10.179] ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id e15sm2383177ljn.25.2021.08.31.21.50.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 31 Aug 2021 21:50:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7942cd84-54e6-4fb6-86b6-aad24475e815
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20161025;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=fp+l23ni8nD//u350996azMCPKCUo56T7jjZIzLPNDY=;
        b=LTZvKfPGCOt6z6sq51v4QCbm+YcdSuPPrXAbAWO9F0OjfFVFoXJOhtldbXUEgy/xMX
         ccC/gEbZPAr3+yTuyVGxI6bT16BGcGY22OPmd7ue3vWOZfA1ybc2/7kpcebA5gPn1bfA
         pAlKMp26Z6jeJ8JkwnV7oq7G0kplYcbb6e/BVn+CdXgx0y5jWT6La6uDAkjKpqycdgk2
         EZygunRC8ILxteP56YbI/Lt5m7VadKcj2HGnbkp/Pw/0wRSGJNhvNQUA8F/qfaCXEiaD
         sMUsEe6cFYQj5Ddz0dJ/D0gpc4e+uU9BmRSwlYvQvW2DLBFLp+4rw6+wnR+NaJQyrlA6
         w18w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=fp+l23ni8nD//u350996azMCPKCUo56T7jjZIzLPNDY=;
        b=CUhjZNBYH0K/9VZjTzOrzAI9/hRAGYDQ79GYmI2rO+IcoGuW+SP9Nbg4lD8bgnE4A5
         xmJ5ejTnGJStvJYjlOmT/YsW6Zi3TU4Nd3WVtGkOTx2yaOHI5ucNe52sxdz6XQe6XzXj
         NitLm+VUcVxTYF6ls6rrqDbLRDYCe7h2tvfqhHwyyCVOmgRwP1YpH8vfAmiC6guZd67R
         UsCMG7ms3xhe89zlcbz7WrH+H8CB63VZzZs7//6UhdUZBtSUZA1ejmFbGJFRy7Pzc1UZ
         xF1rLqbLpNsH/Bu+9x9K+27dXgPEM+lUKTzxhZjuX91CJ03VfeLxPJ99HD5Iv1KekLaR
         ziJg==
X-Gm-Message-State: AOAM531ZCJDEOKk7QugCyn7KpTvY5+lrhJ8RzVqcy5Aq/50U3mhHiILX
	1pNeIFZgweobp4aX70ZJ+m94B41Zodw4Xw==
X-Google-Smtp-Source: ABdhPJy+09EttPJHJkf6WstYZk8w+26229ZqHGUmytKZbcF+xhAAFrwoPcd32nxOFA0MKVR4+xFrDw==
X-Received: by 2002:a05:651c:200e:: with SMTP id s14mr28530367ljo.306.1630471832844;
        Tue, 31 Aug 2021 21:50:32 -0700 (PDT)
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <b2e8863d-217a-fc0e-3176-a20ef5ad0872@gmail.com>
 <e87ad2a2-7ee0-a2e7-7bab-0c332aca7aec@suse.com>
 <3a6ca3ac-8771-2a80-285e-701c5c1b8343@gmail.com>
 <956dd02c-553e-6aa2-eedf-28703a6ded32@suse.com>
 <13eb638e-c3c5-6794-c828-04c66bb2bd2f@gmail.com>
 <8295cd88-4e4e-4189-cf27-ce83823357df@suse.com>
 <1f571749-5d41-7c4d-8ca0-afc91d2f83fe@gmail.com>
 <2f81bc05-69e0-f3e7-4a50-67b40352efa6@suse.com>
From: Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <052efde6-bccd-b34c-ccbb-c2cc9f513f56@gmail.com>
Date: Wed, 1 Sep 2021 07:50:30 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <2f81bc05-69e0-f3e7-4a50-67b40352efa6@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 31.08.21 11:35, Jan Beulich wrote:
> On 31.08.2021 10:14, Oleksandr Andrushchenko wrote:
>> On 31.08.21 11:05, Jan Beulich wrote:
>>> On 31.08.2021 09:56, Oleksandr Andrushchenko wrote:
>>>> On 31.08.21 10:47, Jan Beulich wrote:
>>>>> On 31.08.2021 09:06, Oleksandr Andrushchenko wrote:
>>>>>> On 31.08.21 09:51, Jan Beulich wrote:
>>>>>>> On 31.08.2021 07:35, Oleksandr Andrushchenko wrote:
>>>>>>>> On 30.08.21 16:04, Jan Beulich wrote:
>>>>>>>>> @@ -265,7 +266,8 @@ static int modify_bars(const struct pci_
>>>>>>>>>            * Check for overlaps with other BARs. Note that only BARs that are
>>>>>>>>>            * currently mapped (enabled) are checked for overlaps.
>>>>>>>>>            */
>>>>>>>>> -    for_each_pdev ( pdev->domain, tmp )
>>>>>>>>> +for ( d = pdev->domain; ; d = dom_xen ) {//todo
>>>>>>>> I am not quite sure this will be correct for the cases where pdev->domain != dom0,
>>>>>>>> e.g. in the series for PCI passthrough for Arm this can be any guest. For such cases
>>>>>>>> we'll force running the loop for dom_xen which I am not sure is desirable.
>>>>>>> It is surely not desirable, but it also doesn't happen - see the
>>>>>>> is_hardware_domain() check further down (keeping context below).
>>>>>> Right
>>>>>>>> Another question is why such a hidden device has its pdev->domain not set correctly,
>>>>>>>> so we need to work this around?
>>>>>>> Please see _setup_hwdom_pci_devices() and commit e46ea4d44dc0
>>>>>>> ("PCI: don't allow guest assignment of devices used by Xen")
>>>>>>> introducing that temporary override. To permit limited
>>>>>>> visibility to Dom0, these devices still need setting up in the
>>>>>>> IOMMU for Dom0. Consequently BAR overlap detection also needs
>>>>>>> to take these into account (i.e. the goal here is not just to
>>>>>>> prevent triggering the ASSERT() in question).
>>>>>> So, why don't we set pdev->domain = dom_xen for such devices and call
>>>>>> modify_bars or something from pci_hide_device for instance (I didn't get too
>>>>>> much into implementation details though)? If pci_hide_device already handles
>>>>>> such exceptions, so it should also take care of the correct BAR overlaps etc.
>>>>> How would it? It runs long before Dom0 gets created, let alone when
>>>>> Dom0 may make adjustments to the BAR arrangement.
>>>> So, why don't we call "yet another hide function" while creating Dom0 for that
>>>> exactly reason, e.g. BAR overlap handling? E.g. make it 2-stage hide for special
>>>> devices such as console etc.
>>> This might be an option, but is imo going to result not only in more
>>> code churn, but also in redundant code. After all what modify_bars()
>>> needs is the union of BARs from Dom0's and DomXEN's devices.
>> To me DomXEN here is yet another workaround as strictly speaking
>> vpci code didn't need and doesn't(?) need it at the moment. Yes, at least on Arm.
>> So, I do understand why you want it there, but this then does need a very
>> good description of what is happening and why...
>>
>>>>> The temporary overriding of pdev->domain is because other IOMMU code
>>>>> takes the domain to act upon from that field.
>>>> So, you mean pdev->domain in that case is pointing to what?
>>> Did you look at the function I've pointed you at? DomXEN there gets
>>> temporarily overridden to Dom0.
>> This looks like yet another workaround to me which is not cute.
>> So, the overall solution is spread over multiple subsystems, each
>> introducing something which is hard to follow
> If you have any better suggestions, I'm all ears. Or feel free to send
> patches.

Unfortunately I don't have any. But, could you please at least document a bit

more what is happening here with DomXEN: either in the commit message or

in the code itself, so it is easier to understand why vpci has this code at all...

Thank you,

Oleksandr

>
> Jan
>


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 06:59:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 06:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176064.320476 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLKDB-0006QW-NW; Wed, 01 Sep 2021 06:59:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176064.320476; Wed, 01 Sep 2021 06:59:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLKDB-0006QP-KW; Wed, 01 Sep 2021 06:59:05 +0000
Received: by outflank-mailman (input) for mailman id 176064;
 Wed, 01 Sep 2021 06:59:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0rUu=NX=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLKD9-0006QJ-Kj
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 06:59:03 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c1e57438-37fa-409d-9e25-070c2e6561ae;
 Wed, 01 Sep 2021 06:59:02 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 97205224CB;
 Wed,  1 Sep 2021 06:59:01 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 62F5A13A1F;
 Wed,  1 Sep 2021 06:59:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id ZA3wFbUkL2HAcwAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 01 Sep 2021 06:59:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1e57438-37fa-409d-9e25-070c2e6561ae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630479541; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=dQEcCvQxlYcDyTgM3abdo+HELlzPwv5DmTjSLpwwCp8=;
	b=J5K5h0fWCcbU5znY17gvu3jIq9XGMSOEqgRfdujdPvWar8cF2r02uv3WQ1vXe5IOl4EUA1
	p5wfzv3NtLvuyjh6pd5OYFU6xJ1Gb8QaQB/TGIil4sYnLScb6E32Rd2R5CHxHElybZnc6u
	SpWUq8AicsFlKqBRKha+VysN9apDjzs=
To: Ian Jackson <iwj@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Wei Liu <wl@xen.org>
References: <20210730122643.2043-1-jgross@suse.com>
 <20210730122643.2043-3-jgross@suse.com>
 <24836.28.655841.510063@mariner.uk.xensource.com>
 <81a39ab5-5588-65ad-f1a1-7bfe0379b394@xen.org>
 <6ced9858-d425-887c-0045-1aad8521b826@suse.com>
 <c0a6bb5a-3bd8-b1cf-9973-7a95b347befc@xen.org>
 <9352f82a-1b43-4bf6-8b0a-5916627b7537@citrix.com>
 <24878.15168.981558.748531@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
Message-ID: <ea834f80-45ce-821a-bb2b-4a56dddff156@suse.com>
Date: Wed, 1 Sep 2021 08:59:00 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24878.15168.981558.748531@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="577crmY70HNGhRZXW3JEPSJJJKYN1HDBl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--577crmY70HNGhRZXW3JEPSJJJKYN1HDBl
Content-Type: multipart/mixed; boundary="ywNSWkD3TBAB3AsiWAojb1IzUR6xZNn5l";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Wei Liu <wl@xen.org>
Message-ID: <ea834f80-45ce-821a-bb2b-4a56dddff156@suse.com>
Subject: Re: [PATCH v3 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210730122643.2043-1-jgross@suse.com>
 <20210730122643.2043-3-jgross@suse.com>
 <24836.28.655841.510063@mariner.uk.xensource.com>
 <81a39ab5-5588-65ad-f1a1-7bfe0379b394@xen.org>
 <6ced9858-d425-887c-0045-1aad8521b826@suse.com>
 <c0a6bb5a-3bd8-b1cf-9973-7a95b347befc@xen.org>
 <9352f82a-1b43-4bf6-8b0a-5916627b7537@citrix.com>
 <24878.15168.981558.748531@mariner.uk.xensource.com>
In-Reply-To: <24878.15168.981558.748531@mariner.uk.xensource.com>

--ywNSWkD3TBAB3AsiWAojb1IzUR6xZNn5l
Content-Type: multipart/mixed;
 boundary="------------2FA5F4FC3F8FB8D34C158FAB"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2FA5F4FC3F8FB8D34C158FAB
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 31.08.21 16:22, Ian Jackson wrote:
> Andrew Cooper writes ("Re: [PATCH v3 2/2] tools/xenstore: set open file=
 descriptor limit for xenstored"):
>> xenstored is TCB.=C2=A0 It needs a large number of FDs, and can be tru=
sted
>> with unlimited.
>=20
> I baseically agree with this.
>=20
>> Also, like xenconsoled, we can calculate an upper bound, which is
>> derived from the ABI limit of 32k domids.
>=20
> IMO the default should support at leaat this much.
>=20
> However, I don't think you are right, because xenstored offers console
> connections to (possibly arbitrarily many) local socket connections.
>=20
>> All you're haggling over is the error semantics in the case of:
>> 1) the upper bound calculation is wrong, or
>> 2) there is an fd leak
>>
>> Personally, I think a fixed calculation is right, so fd leaks can be
>> spotted more obviously.
>>
>> An admin knob is not helpful - higher than the upper bound is just
>> wasteful, while lower will cause malfunctions.
>=20
> I don't agree.  Firstly, on a technical level, your statement is true
> only if the admin does not know they will be running a much smaller
> number of domains.  Secondly, we have had several people saying they
> want this to be configurable.  I think if several people say they want
> something to be configurable, we should respect that, even if we think
> the use cases for it are marginal.  If there are hazards in bad
> settings of such a know, that can be dealt with in the docs.
>=20
> Julien's point about not having the limit set by xenstored itself is
> very well taken.
>=20
> ISTM that the following scheme is in the intersection of everyone's
> requirements:
>=20
>   * The limit will be adjusted/imposed in the startup script.
>   * An /etc/{default,sysconfig} parameter will be provided to
>     adjust the setting.
>   * The default should be `unlimtied` since we cannot calculate
>     a safe upper bound for all configurations.
>   * Systems like Citrix Hypervisor (XenServer) which can calculate
>     a safe upper bound can do so, and adjust the default, enabling
>     them to spot fd leaks.

Makes sense for me.

So this would mean:
- the sysconfig parameter will no longer be "number of domains", but the
   fd limit of the Xenstore daemon
- default should be "unlimited"
- the comment should mention the current number of fds needed per domain
   (5 for HVM, 2 for PV/PVH) plus some headroom in dom0 (without any
   guest running on my test system 21 fds are active, so I guess a value
   of 50 seems appropriate)

Is this okay?


Juergen

--------------2FA5F4FC3F8FB8D34C158FAB
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2FA5F4FC3F8FB8D34C158FAB--

--ywNSWkD3TBAB3AsiWAojb1IzUR6xZNn5l--

--577crmY70HNGhRZXW3JEPSJJJKYN1HDBl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEvJLQFAwAAAAAACgkQsN6d1ii/Ey/m
LAf/augriOmlOWBkKyHqCAop9REFqosu6ecM2bTY+wMEDH1eViKcztGSHMRS5bMWh7B9vhJNrZGR
JAyQ4TZCDniurFuESay4A6up1MAgzomB9ha8LoubESK34kjoias1CO86XQz95+1+2KyRquf7qgaE
dIAuGCHhlobBhHyzFqZ3p5VYJZJGdrtluZpKHy94r/588EJQtSkZh2S9ewFoRtE4bdLB8C5FwmSB
YWCWvqV0fKUDF1wE+asz/zK0yyZckk3vIXwhj5LSWDrC7EC/Odtq1foUakyzxwEDNu0wX+rqiURy
KHOR6jqqXYS9d915P5B6BjZi7qgVJC/9+QjplC6/Bg==
=YC3z
-----END PGP SIGNATURE-----

--577crmY70HNGhRZXW3JEPSJJJKYN1HDBl--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:09:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:09:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176075.320488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLIk-0006SQ-BL; Wed, 01 Sep 2021 08:08:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176075.320488; Wed, 01 Sep 2021 08:08:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLIk-0006SJ-8P; Wed, 01 Sep 2021 08:08:54 +0000
Received: by outflank-mailman (input) for mailman id 176075;
 Wed, 01 Sep 2021 08:08:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLLIi-0006SD-Gd
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:08:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d6da4e3e-0afb-11ec-adb6-12813bfff9fa;
 Wed, 01 Sep 2021 08:08:50 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-1ZAPm3CeNW-nITZxRMk4-A-1; Wed, 01 Sep 2021 10:08:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7149.eurprd04.prod.outlook.com (2603:10a6:800:12e::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 1 Sep
 2021 08:08:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 08:08:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Wed, 1 Sep 2021 08:08:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6da4e3e-0afb-11ec-adb6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630483729;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Br8yhggrtkI50zpoT8ExDWcF1KSQhL1o7JlLr+hMIpo=;
	b=DGttOVKjp+4XjuuNSpPjOTAQxrvA+KmbR3umxmOdD1UuMPKuMzY6SSuYoqYVW/7WYojkVt
	iRM2EWJf4a//bUT7jyFfAA0nliIaomYoQSbY3IIgy/2N4KVjGTFEEIpRjKNbccUx11lPdG
	SO1SQp0wGbKN+PfXcFQvUoaWaHN0zRc=
X-MC-Unique: 1ZAPm3CeNW-nITZxRMk4-A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XK0kMNcnc8IIDbFp4NSqXfEuVXtBJbqr/l3V38eAN/6vjabGZfaA9x/ag1T1ffaYMp0oy6FZdsCU6tBFrURqL4pBAy/6AUimlcx8QGRpNCNwID2xgOLb4eDaExFETZsMRD6AzUa0wgbTeByrzmZAHlFGZrignhoYoXCjcKtYH8lOVkMpUkw9gEgxdfi4u3sCos84cJw3hc8N30ABhI3RhQso+t6X3Qg1eK2Ob1UrmymdPahs+vmvv+NweA13sz5Y/K7E/k8vwZIc/LqSG/OiukZiutaXRPt/n60HeNLevVwU7B3s0kvrfIqTFpTo5vPzZThRM68wnUtdu/wbpoIr/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Br8yhggrtkI50zpoT8ExDWcF1KSQhL1o7JlLr+hMIpo=;
 b=PmhZhWD3JJ7+0BKLmeRO0TmzNX6ZRSbgKvQWMhMoe39WeH93upKIRYG7BMdccfF6kLYEj822vqtAZ/5r/SLzgLUvWcLGaFpymfKcoiUf3Hj3K6MLmvfqfjTy3TJEK2XwKtqis3U1TNs3Mb4qNetzvvPuh5MgbdcbHNqoGj/hyM+mkvyeGDSK9lFMsfsotZLalkT0jvmurKg/pJWRglGLIVgcKlNG7p+OybW1QA9xqYX4VQrh9WXiAmXs030nCK5XhqPLD0yhqqZXlhnL6ODtASPSW8yqmY/7oR+ibRxjmSOl98i2TWcuRKuD706f2IDyImfvjxVTBd5828ti8j0oug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
Message-ID: <c48dd5c9-bd88-9ae8-856b-4049e26bc099@suse.com>
Date: Wed, 1 Sep 2021 10:08:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0P190CA0019.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:208:190::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 88028a76-1fa9-4d90-1d36-08d96d1fb945
X-MS-TrafficTypeDiagnostic: VI1PR04MB7149:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7149A284F0F1FD5E77EA6B85B3CD9@VI1PR04MB7149.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JlH//v+JTrd3af7F/8tMwKHhQacrJPN3kOQTbRdlylOu55HDqpqysai6Ybzaa6lf5Vv8hjFuW9XoSf26lVEv0O2Osrwt5QxyLuRRflMwR7GG2ZTxSo37cBHOjKTkaEbpfzsX94jzjt2+MeqkNgqYXRqHZuJBF+TpPrDAi7OaFrKvYTC3xjK2BeOJ+Qy4sPP3i7iDoOEJHIxzNIuk1plsrXypDHMwtJQNdAsqfOjsY2e1FVRFCksKbEdHMIbPl4tz5shnA58gwcUUSbH/I5dcp6HZBNTUfg2BMzuzmbdKmwx+w8l/jPaaQu0eah8gkbRriLs8qUWyvcjlaKJ1UnQcQKKP+d0hQbTAweSwJnbFxqjii/w1L4Bp9ZpAo16gPtH631/NtX5P2j/24Ll4QyrjTqjC8ExqhrfM/qh680s4OyMXsWkQ61PN2Aavfz3guhCKkIau3Uog5YASwuzREWLRj4nyPgJmlrA4gb8ntwSr8qC1FTGGbgp5CJkmDQ0VFGeYDYFGXnfOt2urC6SwBGGKMXNv5p1CtY1X7vactlE5L0nRsqRJ7WkaEjD8re+4Nb/3N9gP+uJqZjefbfhXy9MIWNjf13jEbl8MikzJSqlMbquU8mNh9qsUwPR+k9siA3vyH3J9P/PUxZP7+ylx+GRcyCXm/e+wbtB1QKniMFG3X10EvwjQoXneC7QbiM236uJTVPiabvnfPb88ujeFV5NtUZcdsYTfmup1XVCgkBdB5c8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66476007)(83380400001)(8936002)(66946007)(26005)(186003)(53546011)(316002)(16576012)(508600001)(6916009)(4326008)(2906002)(5660300002)(36756003)(2616005)(956004)(86362001)(6486002)(54906003)(31696002)(31686004)(8676002)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K0xIU0k3WDNUdU1ETi92bW40U2hJZFduKy9Zc2R6QVk1ajd4UmQyWFV0UEhK?=
 =?utf-8?B?MnZGVWRLblRjMUM3bGxuMUlYeU53SFllbnMwWFVsNjI2NjR1bmsveFp3b3Mv?=
 =?utf-8?B?bS9MYzlUS2krZzFuVlJGZlJybC81bHBTWHBtS01weVNOLzdvMDlJNWMramFW?=
 =?utf-8?B?OFFWR0FrZ0o0b1lqZWNWK1M4OVRCa0VzYWxFV0ZCQ0pNRjF4L1pPZGoxWnY0?=
 =?utf-8?B?VGNNSlFwYUJVTThKN0NtVzNDeUZZa2RRN3pFK3MwNis1aGovYnlFcjJVMGZS?=
 =?utf-8?B?YS96bHBqUlRGV0dnNlpFMm9vVEVmMDZhaWYxb0VadTFydmN6a0t6VVN5N1Vi?=
 =?utf-8?B?ZDRvaEM2bmt1RU9OclhMb2JTWkFsUGxLN0JtTittNTZ4WEF2dE5YWlBoQUpR?=
 =?utf-8?B?bTVzUWZBNVViWXZUTmQzb2xoMUN6b1IwNzVVeUR4aHkvMTd2N2RRVTZXYlhI?=
 =?utf-8?B?MzJjbmlINkd6Z2FhOWVEbkZSKzhxRW85YTBkM3ZTYzF1cDFteFd2aFlQaGxE?=
 =?utf-8?B?K1l0eGp2aEowMllVbVZzMDZjMkVKNkFnSjdrUGV1aEdaaTVNNFJHVGxZOE00?=
 =?utf-8?B?dWtQWDgxNzNMSjh2UTN6K2srdG9ZUFFEK1NRUU1zcWJ3TjNXa1gzeVlMV0g0?=
 =?utf-8?B?S3VRSm90RWlSU0swRGQxdTJBZGtzb01odjk1WTFKRWZETnUrNVBydUhhKzNH?=
 =?utf-8?B?VEZVMzk3NzV6ZktIQlBIWDhFMVdVV3BiOEhyNGV5WGpzbURTN0RJQU80Y3NF?=
 =?utf-8?B?MHRuMVhybEpNLzNZUlBPbzRYMmdSWnpOUmJ3UUZMdTMvK0NDZ013RW4wVHV6?=
 =?utf-8?B?SnUxSnZGeTZMQmtSVnhYS2pnTTkxaEg5SEJaRzU3b3hLR0RRampZTW5GUWZj?=
 =?utf-8?B?SXFwR2VrbDhRaERiazJuSGdMUjNrR1JDQmRjTG9PWFkwdVhuWnBSRmd0b0Ru?=
 =?utf-8?B?Qnh5akJNMEhLQTRBdkRla1RFL05IMUcwM0Y1ZEg1cUxnN0ZFVHQ3ZnMwMmhR?=
 =?utf-8?B?VEFRbHAyeUdHTzlERTVMMmJHaTJyZGE5Tk1YTmhlNTFQajlJNVpWaGhWUjJ4?=
 =?utf-8?B?QWxGbnd2Z292N0xVWG9qeEZ3Z0dicWNCaTZkZy9Wa01pcmw0NStoVERTSVlC?=
 =?utf-8?B?c3d6UGgvYmw2cjZ4MFZzWTkzc2V6clVuRlJZdmNPR2ovaGxwdzdlejZzQW4r?=
 =?utf-8?B?aDFEdXZUNWZvaitZZU5KcFNTRGFmZnNNU1hETTNnaTRjTmM5VHdqRFpJdnNY?=
 =?utf-8?B?WmNYZTQwY0dTL2Mzb1NSa2lHUTdwUUhJcFNFY3M1VkZTczcrTDllWlMzNTJF?=
 =?utf-8?B?Nko5SjJmYm51VVZPTm5mRUVuUkU5WnAxSFEra01MeGVYYkJsU0hkNnVQZ1FW?=
 =?utf-8?B?K05oc3pkZUkvMWRCK3RHNDZIOXQ5ZWh6cEhlalpXdTE2amJmOXBrbThDSTdw?=
 =?utf-8?B?aEVIaFdwZDB4SG9ZRm9LQkh5SUxRNWdaTWZVV3NsRVlqOVNFWW1hVnU5VE1I?=
 =?utf-8?B?YW9sM2hIakZBc2dMdnNEYmxiVkNCS25yUEc0cTR1cStMa2ZKbE1MNldxeXpG?=
 =?utf-8?B?a1lwODRnOVN6aUZTZEZSaCtXcXFTUm5IbXc2MnBlMExIYnd2NmdyT1A3cmlR?=
 =?utf-8?B?QXBPdVpXZmV4QSt5Sldxdm0yRzR1TTA2L3VVQWRQdDZQRWRiYURNSW9RYzJS?=
 =?utf-8?B?UzlBdXRyUlN3N24xNHRLQVNoaE1EU0Vhd1VTYXVUZWI0ME8yMTM0Y01wYzdr?=
 =?utf-8?Q?d6vt6pny/6sHE7GH6ytvCtHIofq/vrN9mPT2M4x?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88028a76-1fa9-4d90-1d36-08d96d1fb945
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:08:47.1462
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: H07kQ+VnfEDTR9YzU6avt6aMhjdjt8l2aBaze/323buka/D6ZHbk3v/s4sdqad2cXJih/Z32rQxVLZdyghRR5Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7149

On 31.08.2021 17:38, Jan Beulich wrote:
> On 31.08.2021 17:25, Andrew Cooper wrote:
>> On 31/08/2021 14:26, Jan Beulich wrote:
>>> On 31.08.2021 15:16, Andrew Cooper wrote:
>>>> On 30/08/2021 14:02, Jan Beulich wrote:
>>>>> Further permit "access" to differ in the "executable" attribute. While
>>>>> ideally only ROM regions would get mapped with X set, getting there is
>>>>> quite a bit of work. Therefore, as a temporary measure, permit X to
>>>>> vary. For Dom0 the more permissive of the types will be used, while for
>>>>> DomU it'll be the more restrictive one.
>>>> Split behaviour between dom0 and domU based on types alone cannot
>>>> possibly be correct.
>>> True, but what do you do.
>>>
>>>> DomU's need to execute ROMs too, and this looks like will malfunction if
>>>> a ROM ends up in the region that HVMLoader relocated RAM from.
>>>>
>>>> As this is a temporary bodge emergency bugfix, don't try to be clever -
>>>> just take the latest access.
>>> And how do we know that that's what is going to work?
>>
>> Because it's the pre-existing behaviour.
> 
> Valid point. But for the DomU case there simply has not been any
> pre-existing behavior. Hence my desire to be restrictive initially
> there.

Further to this: Using the last-value-set approach also puts us at
risk of running into a similar issue again when the ordering of
some operations changes elsewhere.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:20:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:20:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176082.320499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLTx-0000SK-IK; Wed, 01 Sep 2021 08:20:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176082.320499; Wed, 01 Sep 2021 08:20:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLTx-0000SD-EB; Wed, 01 Sep 2021 08:20:29 +0000
Received: by outflank-mailman (input) for mailman id 176082;
 Wed, 01 Sep 2021 08:20:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLLTw-0000S7-0h
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:20:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5b44084a-c6c9-4fc4-8545-5cb7d020362e;
 Wed, 01 Sep 2021 08:20:26 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2053.outbound.protection.outlook.com [104.47.0.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-qajYLHVMMY2npYd-QBHA5Q-1;
 Wed, 01 Sep 2021 10:20:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4943.eurprd04.prod.outlook.com (2603:10a6:803:51::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 08:20:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 08:20:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0060.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:53::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.20 via Frontend Transport; Wed, 1 Sep 2021 08:20:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b44084a-c6c9-4fc4-8545-5cb7d020362e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630484425;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nOlixGbmhdbdoOvOwStsrHan61/add9PscjBGtXzO5k=;
	b=fBOn3XossWF618Z/A3eI6iXX9GTIzaqsLg8Ld/xKY9DtfI/1zHnbgBHRmEpTPaTgdGPr3e
	LhD4dEIh2Hpqsx2SLC3QwFaBEId1v3MfSYhJpEFXavuYG9OUQHapMbkB1y41XAAPvBDsTG
	HE6LZs0e1iNCVWBf55ou28egVO3jYhk=
X-MC-Unique: qajYLHVMMY2npYd-QBHA5Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FugZK8BywBKbAYT5Q3bZ5Ri8iaQirK+8uQe3t8N3VNOeFOqtADjqoXBH+M9WiQTumFS1TIBX4aBKbe5FkqYtEC5E/qHQxIqC5eEjihPMqxxWKz3u1h5KncnbL7OJ2uEOplpqbJGCOkO7EimS4dl87E0Enpes9h0RUNtCotPfaNnREXsEPRvHDIZD5dCtaq0Sp/C/1Y5B/rPCq16sdhPzS7P2km1tehtemNYGLZnLQtH8lncvH0zWsSGf3shufHB8QAFbSIUJwgfRY21MwiFV0ZhHlLBhpey7gIvja60ar/hSunzW4ozyfaQ4aj5XTjNt3rYp2k51tVNevwqEta+XUg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nOlixGbmhdbdoOvOwStsrHan61/add9PscjBGtXzO5k=;
 b=dNTQ3HSHpNyXRfHWrOWk0pbjrUHC+fKOKsnSjYug/vvm3aHYxf/EmHhjT6jk5YWNAPWxLcqzTvos3XpDJ0qyjhpl94L1FyKH1d0y8D5Hqq9kQiF/jcGTtae/7ZV0efvXHm6ggT7e7AmMzrAFnT6fQYehZsSx2QHQNIE3KfgathEbyNmwpRygoUqFyp3CmF/pDRKwuYVVEImeEA2TA/cBZzsOBe4LVcNptZ14/WeAgnteiZaNV+VwZZ4XqEsb0YIeNjTSBK31oYg6uSoRKzRoDhyxROxHPP+dONLCozd736lxMnEfmZ2UdRBQAH8S+BWFS6VROshlSNWZEOvkr7OYqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <b2e8863d-217a-fc0e-3176-a20ef5ad0872@gmail.com>
 <e87ad2a2-7ee0-a2e7-7bab-0c332aca7aec@suse.com>
 <3a6ca3ac-8771-2a80-285e-701c5c1b8343@gmail.com>
 <956dd02c-553e-6aa2-eedf-28703a6ded32@suse.com>
 <13eb638e-c3c5-6794-c828-04c66bb2bd2f@gmail.com>
 <8295cd88-4e4e-4189-cf27-ce83823357df@suse.com>
 <1f571749-5d41-7c4d-8ca0-afc91d2f83fe@gmail.com>
 <2f81bc05-69e0-f3e7-4a50-67b40352efa6@suse.com>
 <052efde6-bccd-b34c-ccbb-c2cc9f513f56@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b71c0339-e795-94b0-9151-c6840ce847df@suse.com>
Date: Wed, 1 Sep 2021 10:20:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <052efde6-bccd-b34c-ccbb-c2cc9f513f56@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0060.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:53::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5eb9d599-0ed8-420a-5456-08d96d2157c1
X-MS-TrafficTypeDiagnostic: VI1PR04MB4943:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4943EE608E482ED1289B3573B3CD9@VI1PR04MB4943.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7hd4cg9hRXEaJJabae8cOR8zgdh5QaVO5snykEusDtvbnpqohPy9UVDWpPk7/AF+U6axjFAc7ix9gpy9xaxZHRL7hWKxVWGoVTfsAI456yLSnCSiEWjxxL5crTkLW6yZWlzj/yfgOmDz/gdF8//R7EDRFn0s5rfWSIxTf4UlKbXAAyHieTFihRT67QVahyOXlHUojCknkZu53rogwPNOWwvpexH5ue6Dhw/yLoJq1vB7Sk35MrHpYD+GC9zBTRWDtdiNsXbYrW2sXHoMhUwCwEeTFO2W8maHGoowMCQQVI/yAeQuVoCUItOiu8w770DBGu6zHpqXzuAd0v//Cs7XcFQSZNexdLU/WggtHgVMzV5Qtk4nH6EMQEmhwAF7j1FXoqBAR3fBR4/1QMRmOVqERU5YRC+tkd+mZFXXZY6hhTzuP/XgxqyqQl6aDdwKyKmtnwBcp/KpeKU7AHMpl9GI1JorGGXqloHuOsXiOBRvD6YQLEmJE+5gLr/5a7SazJcbhbIt4kt8hvr/errsZOdNZSYDdtp23ha1KQSFt5VhT+MLDVwJFBI1GEppk8gwuCsfDKSrnsUghPKyiz9WHkrwbNWvsG1PttHtWPxq7+7/ObRVss1cL8GrchcojHPj++p0m2ACdYGpmdQtFdo1CuLuU2J5MgO2G7rMSBD/uYIEJgg7gK2fVlIBk5acjogl+kM32WTwzDjppvtrUgzQYSHRodZ1qHvmjI8WQPKwXgqLEi4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(136003)(346002)(376002)(366004)(31686004)(53546011)(66556008)(8676002)(2906002)(16576012)(86362001)(66476007)(4326008)(2616005)(31696002)(36756003)(38100700002)(26005)(956004)(66946007)(5660300002)(8936002)(478600001)(186003)(316002)(15650500001)(54906003)(6916009)(6486002)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RUJ6SEhJcXhHK0VKOUFteHBLa3ZlSTNpZHh2Q05nVlZ4NkZ1VkplMkt6UWtu?=
 =?utf-8?B?bUQ2V3J0eklnSGxsMm55eFRBREVxZWQvdVpUbGYyMkVWd1lnMVZ0ZjVocnBV?=
 =?utf-8?B?Z0drcTkrcFZDbld4dlRScEQ0MXdhUHZRYTNyblI3S2FxR1d1VDFoU2RwT3Av?=
 =?utf-8?B?QjBvazlNMnVIUzc1TFhvd0hPZVlSZVdDTm1UUlFkU0JIbkZMcHQybHBZcHh1?=
 =?utf-8?B?OW12bE1xZWNlV21Yb1hYUVNvdG1XK0NPZGR2TnE5RCtsbHAyNHoyYi84M0o5?=
 =?utf-8?B?VU1HSTdVRFNJdjJQTUZ1Y1F1VE5wN1RJU2ZDc1IxblhWRTJidSthYllQcHB6?=
 =?utf-8?B?Nk9WNTZjbmxSTlBGS0RNc2JINzFjSUs5TGFQMk5vVDZmb3ZwVWRyMmkyNmNQ?=
 =?utf-8?B?TjNtbE11T3J2VEo0azlULzEvUC9QNjlVM0g2TTJ0V01qNGRNT20vUlBFd0JY?=
 =?utf-8?B?VkdJd1JhMzlpM1VRNnBhdGpkYVREcTNMQ21VOFBxTFZ5eitoRGFNWTZxSDJU?=
 =?utf-8?B?UkI5VTRBb3hiSkVXUldQS0hIOTEwZitWQVZPR0RWNFN2dVk2R0UvdENJRTR6?=
 =?utf-8?B?MkRVb0NqWEp1OVQrNE9LalJGWTFtRFVBOWJxMUcySG9IMHk2WWRpMk9EdUwv?=
 =?utf-8?B?eXlwU2FQR01TWU0ySjhYK3Fmd2tiNVd3YVBPSFlIN0VleklQdVZFWHlRa09O?=
 =?utf-8?B?VUgyZ3d1V2U4SzMycnVZM2U0N3dobW9kb3k3M2h4OElLTUMzQ2hudVF6Njdn?=
 =?utf-8?B?eWlUem5ZN05sNTBmdHBUcGdsYjRFMWxoeXpndXFueVlSQTJtMzBWR01sMmlF?=
 =?utf-8?B?OXNNSnVDSVdTYXBCaTJsdkwrNlJ2M2ZtUG1XVEpxbG4yQUdubi9HUS96RFIy?=
 =?utf-8?B?c3NoWXJPMDJjbGFiN2xIS2UxanpPc3k5NU84enR6dEZyRjBsSWw2Z25ySmY0?=
 =?utf-8?B?eWk3YVcrQWVaUVZ5T2ZwOXFnZnRhakdBd0xKcDhXbDdjcWJDT2VzT29qZkJE?=
 =?utf-8?B?cTdXcHBRVHhydGNZelNLb09zY0hYa09sQS9reWJRdWhkZ1BBWmZXcjgzZjhW?=
 =?utf-8?B?Q0NPZ3M1UVB0N2FLYWY5T3RRVjZqQ1J0bjhXVHZMYytOTm03ZXkrK0FRcmQr?=
 =?utf-8?B?NkF6YXJ4bE81aXhlQjBhOFJoS1ZVUTIxSENRZHEyT0FyQzFZVzlVVzZDMS80?=
 =?utf-8?B?MmphdTJYUkx5NW91N1cyTmg5M3VYeVJGMUY1MjZVQlZBcVQ3SzRsbFVsSzBE?=
 =?utf-8?B?d2FmSEd2QW9lRHF3Y3FGUGkzZDc0L2tnV2ZYSzZkajA5SlZMMk05MWFEVk91?=
 =?utf-8?B?YWVsYWZaMGRSVDg4UzdxeUxHN2lvUHhsSUduakhkSTNvYXdZL0oxYnVCdmhT?=
 =?utf-8?B?bUNYWm1ramlWMU1LcXJjZENoTFNvRWV1UVIyTmRXK2VSREZ6alJnT1IvRitS?=
 =?utf-8?B?MmludHJRV01wRS92bThHTElFSkZqUEhNYzBISnllUzgyb3NkYUpXbkYxbUM1?=
 =?utf-8?B?dzNxbE9paXlHa2Z3aG85b0U5dFdaMjFha21obGJaU2tCUHhiUVdGeFhrNm9R?=
 =?utf-8?B?dG9hSUV1Yzlta2FDTk45emtYbmtsVTNrOEZZM0hqc3FyaGFzT3ZSM0dUMHJ3?=
 =?utf-8?B?QnJseWNRdlpTZXlFV0hublczaWEvQWYxd09MWjNPRmtPVHVBRmJYQTJqalBI?=
 =?utf-8?B?TUxJOHlRR0dYSkV6eFRmWnRPSnZ2dEFkZTNXbmNja2d1NlBVN1N5S0loUkN6?=
 =?utf-8?Q?gzAGWXiqfGyiCz+Xj4xgPpXxgW/PpRt9tawvNc5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5eb9d599-0ed8-420a-5456-08d96d2157c1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:20:22.4111
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: r+/ghNczzfkfYIn82ugKg8uRStfxs23bm7Nc8WfHqbqYWD2lkQ5jwZJFoI6V5NgwP4mZP604IexPctRsDmyL/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4943

On 01.09.2021 06:50, Oleksandr Andrushchenko wrote:
> On 31.08.21 11:35, Jan Beulich wrote:
>> On 31.08.2021 10:14, Oleksandr Andrushchenko wrote:
>>> On 31.08.21 11:05, Jan Beulich wrote:
>>>> On 31.08.2021 09:56, Oleksandr Andrushchenko wrote:
>>>>> On 31.08.21 10:47, Jan Beulich wrote:
>>>>>> On 31.08.2021 09:06, Oleksandr Andrushchenko wrote:
>>>>>>> On 31.08.21 09:51, Jan Beulich wrote:
>>>>>>>> On 31.08.2021 07:35, Oleksandr Andrushchenko wrote:
>>>>>>>>> On 30.08.21 16:04, Jan Beulich wrote:
>>>>>>>>>> @@ -265,7 +266,8 @@ static int modify_bars(const struct pci_
>>>>>>>>>>            * Check for overlaps with other BARs. Note that only BARs that are
>>>>>>>>>>            * currently mapped (enabled) are checked for overlaps.
>>>>>>>>>>            */
>>>>>>>>>> -    for_each_pdev ( pdev->domain, tmp )
>>>>>>>>>> +for ( d = pdev->domain; ; d = dom_xen ) {//todo
>>>>>>>>> I am not quite sure this will be correct for the cases where pdev->domain != dom0,
>>>>>>>>> e.g. in the series for PCI passthrough for Arm this can be any guest. For such cases
>>>>>>>>> we'll force running the loop for dom_xen which I am not sure is desirable.
>>>>>>>> It is surely not desirable, but it also doesn't happen - see the
>>>>>>>> is_hardware_domain() check further down (keeping context below).
>>>>>>> Right
>>>>>>>>> Another question is why such a hidden device has its pdev->domain not set correctly,
>>>>>>>>> so we need to work this around?
>>>>>>>> Please see _setup_hwdom_pci_devices() and commit e46ea4d44dc0
>>>>>>>> ("PCI: don't allow guest assignment of devices used by Xen")
>>>>>>>> introducing that temporary override. To permit limited
>>>>>>>> visibility to Dom0, these devices still need setting up in the
>>>>>>>> IOMMU for Dom0. Consequently BAR overlap detection also needs
>>>>>>>> to take these into account (i.e. the goal here is not just to
>>>>>>>> prevent triggering the ASSERT() in question).
>>>>>>> So, why don't we set pdev->domain = dom_xen for such devices and call
>>>>>>> modify_bars or something from pci_hide_device for instance (I didn't get too
>>>>>>> much into implementation details though)? If pci_hide_device already handles
>>>>>>> such exceptions, so it should also take care of the correct BAR overlaps etc.
>>>>>> How would it? It runs long before Dom0 gets created, let alone when
>>>>>> Dom0 may make adjustments to the BAR arrangement.
>>>>> So, why don't we call "yet another hide function" while creating Dom0 for that
>>>>> exactly reason, e.g. BAR overlap handling? E.g. make it 2-stage hide for special
>>>>> devices such as console etc.
>>>> This might be an option, but is imo going to result not only in more
>>>> code churn, but also in redundant code. After all what modify_bars()
>>>> needs is the union of BARs from Dom0's and DomXEN's devices.
>>> To me DomXEN here is yet another workaround as strictly speaking
>>> vpci code didn't need and doesn't(?) need it at the moment. Yes, at least on Arm.
>>> So, I do understand why you want it there, but this then does need a very
>>> good description of what is happening and why...
>>>
>>>>>> The temporary overriding of pdev->domain is because other IOMMU code
>>>>>> takes the domain to act upon from that field.
>>>>> So, you mean pdev->domain in that case is pointing to what?
>>>> Did you look at the function I've pointed you at? DomXEN there gets
>>>> temporarily overridden to Dom0.
>>> This looks like yet another workaround to me which is not cute.
>>> So, the overall solution is spread over multiple subsystems, each
>>> introducing something which is hard to follow
>> If you have any better suggestions, I'm all ears. Or feel free to send
>> patches.
> 
> Unfortunately I don't have any. But, could you please at least document a bit
> more what is happening here with DomXEN: either in the commit message or
> in the code itself, so it is easier to understand why vpci has this code at all...

Well, the commit message imo already says what needs saying. I'll extend
the pre-existing code comment, though. But of course the primary purpose
of this RFC is to potentially have a better approach pointed out in the
first place, which I guess will mean to wait with v1 until Roger's return.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:41:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:41:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176090.320521 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLoZ-0003gf-IB; Wed, 01 Sep 2021 08:41:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176090.320521; Wed, 01 Sep 2021 08:41:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLoZ-0003gW-Ex; Wed, 01 Sep 2021 08:41:47 +0000
Received: by outflank-mailman (input) for mailman id 176090;
 Wed, 01 Sep 2021 08:41:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0rUu=NX=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLLoY-0003g8-Jl
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:41:46 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6bfb126c-8bcb-45d2-88e5-aa0d8f88f6bf;
 Wed, 01 Sep 2021 08:41:45 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 9331A224EE;
 Wed,  1 Sep 2021 08:41:44 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 3C9CE13A1F;
 Wed,  1 Sep 2021 08:41:44 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 1DRsDMg8L2E2DwAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 01 Sep 2021 08:41:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bfb126c-8bcb-45d2-88e5-aa0d8f88f6bf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630485704; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=lY6W7YdH9RogZrDmgUL66tDU5fn1ekzHXvABU8NVPCk=;
	b=FDvA7u95dunxZ++Z2dqjc3UgCCiUDbMy7tOEsomdNbINdaVwegn64JBqyHT11g+CFg3gVG
	oHujdck7oVzYsAogsO2QZ/jC15Qq0r5ThseVCcDRwPx8/ofTetsvSz2daz6p14JUC1VDk4
	2Fd8IAjVnClzuh5EMG/n71Ard5twfI8=
Subject: Re: [PATCH v2] xen/pcifront: Removed unnecessary __ref annotation
To: =?UTF-8?Q?Sergio_Migu=c3=a9ns_Iglesias?= <lonyelon@gmail.com>,
 konrad.wilk@oracle.com
Cc: boris.ostrovsky@oracle.com, sstabellini@kernel.org, bhelgaas@google.com,
 xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, =?UTF-8?Q?Sergio_Migu=c3=a9ns_Iglesias?=
 <sergio@lony.xyz>
References: <20210830175305.13370-1-sergio@lony.xyz>
From: Juergen Gross <jgross@suse.com>
Message-ID: <f954f07b-a973-76a6-5f0e-c2eccb3be92d@suse.com>
Date: Wed, 1 Sep 2021 10:41:43 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210830175305.13370-1-sergio@lony.xyz>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="EIBqyqtMK8vKu3EwJk8AW3qWpsEnLpuIY"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--EIBqyqtMK8vKu3EwJk8AW3qWpsEnLpuIY
Content-Type: multipart/mixed; boundary="q5YoPOE5vfxmkzIPMs0UKHG5VMG0bWMJ2";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Sergio_Migu=c3=a9ns_Iglesias?= <lonyelon@gmail.com>,
 konrad.wilk@oracle.com
Cc: boris.ostrovsky@oracle.com, sstabellini@kernel.org, bhelgaas@google.com,
 xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, =?UTF-8?Q?Sergio_Migu=c3=a9ns_Iglesias?=
 <sergio@lony.xyz>
Message-ID: <f954f07b-a973-76a6-5f0e-c2eccb3be92d@suse.com>
Subject: Re: [PATCH v2] xen/pcifront: Removed unnecessary __ref annotation
References: <20210830175305.13370-1-sergio@lony.xyz>
In-Reply-To: <20210830175305.13370-1-sergio@lony.xyz>

--q5YoPOE5vfxmkzIPMs0UKHG5VMG0bWMJ2
Content-Type: multipart/mixed;
 boundary="------------20098A477538115561BD46D1"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------20098A477538115561BD46D1
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 30.08.21 19:53, Sergio Migu=C3=A9ns Iglesias wrote:
> An unnecessary "__ref" annotation was removed from the
> "drivers/pci/xen_pcifront.c" file. The function where the annotation
> was used was "pcifront_backend_changed()", which does not call any
> functions annotated as "__*init" nor "__*exit". This makes "__ref"
> unnecessary since this annotation is used to make the compiler ignore
> section miss-matches when they are not happening here in the first
> place.
>=20
> In addition to the aforementioned change, some code style issues were
> fixed in the same file.
>=20
> Signed-off-by: Sergio Migu=C3=A9ns Iglesias <sergio@lony.xyz>

Pushed to xen/tip.git for-linus-5.15


Juergen

--------------20098A477538115561BD46D1
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------20098A477538115561BD46D1--

--q5YoPOE5vfxmkzIPMs0UKHG5VMG0bWMJ2--

--EIBqyqtMK8vKu3EwJk8AW3qWpsEnLpuIY
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEvPMcFAwAAAAAACgkQsN6d1ii/Ey9B
VQf9GD3+Dkl5gJyccur897+nZfeZsmoMHpvOmfVtaPU0CBegYsJwiGQvBxzxV7p1zx6dHvFPXltS
jD55+SnSAAaLUG8UvaHCERGaKlv+RQBjbJTFnxVbBad3nYanzLfGSyGbNxpJnBJeYWNIpY4Fnxv4
gRZUtpqO6zjYx6O0d3ma9ENVNfG8YOdcGBniQ2HebwrNLc73BZs9sdfD70R/ODxikHd/RpTxqgQA
80P/rein9eYq54aJzTyifGagTyznwZ/cbNr+BNYpAyKEZ28M3D1abgFqYayp6jndrG1JtVEan8Mc
cOU04sAiPy1ErjzFkSZppII+YOmvLdDwHxTeDAoNNA==
=3NDJ
-----END PGP SIGNATURE-----

--EIBqyqtMK8vKu3EwJk8AW3qWpsEnLpuIY--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:41:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:41:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176089.320509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLoK-0003Lz-92; Wed, 01 Sep 2021 08:41:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176089.320509; Wed, 01 Sep 2021 08:41:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLoK-0003Ls-64; Wed, 01 Sep 2021 08:41:32 +0000
Received: by outflank-mailman (input) for mailman id 176089;
 Wed, 01 Sep 2021 08:41:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2BIO=NX=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mLLoI-0003Lm-TE
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:41:31 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 66a61580-0b00-11ec-adb9-12813bfff9fa;
 Wed, 01 Sep 2021 08:41:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66a61580-0b00-11ec-adb9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630485688;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=4JfLTFLOD28S5qrWhN3Gp2Z5R1EyEGIHqAO0UFHt04k=;
  b=gwm9To9WUxzREtbV40yqZnSLS2kdxBfSncz7sM2U/p/d9Fr6fKBDmbIC
   izyvwcaUwivUwca+hMV2YqPq5GB0CynkkgQ0e3Sbb8X8IkwEcRBXhGKB0
   azNV/UGe1PxxNTM1H404Ygbh0Ux1CQeTMh2uYUHA2/erxO7z3jBazYnWT
   w=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: w8v8xFC/VqwLgk6a+nwW7EkKIIa4lKvOrNPcoipjKTx7NGZDsV7H4j/0aBjbpYX3JstoM8G0an
 ypX4iTeNKuwpu5MGQqzHjzV3kjW2m65//ahbrIOSJdB2950OkDq/3re3qX5sloO8rzKvFlwRby
 taZY2otOjjBYatG1q3CDWy6yUfLLflfLTuCzZ2GNoXcroJqIz64Zz2J/HZJ1zzqgGSCxRI20YM
 5RcSFq457qkUtv1nyAKrjMGwTqMaoCkC/C0DPdIL3LtrA1Ak8U2rxUrhbFzSMTqbzKYw4u0YiY
 dcQrtYL1L2n2xCSmm02lVZwX
X-SBRS: 5.1
X-MesageID: 51745175
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:ZpVxIamoCnYJap/jxnElZ0sjSmfpDfLW3DAbv31ZSRFFG/Fw9/
 rCoB3U73/JYVcqKRUdcLW7UpVoLkmyyXcY2+cs1NSZLWzbUQmTXeJfBOLZqlWNJ8SXzIVgPM
 xbAspD4bPLbGSTjazBkXSF+9RL+qj6zEh/792usEuETmtRGt9dBx8SMHf9LqXvLjM2fqbQEv
 Cnl6x6jgvlQ1s7ROKhCEIIWuDSzue77q4PMXY9dmcaABDlt0LR1ILH
X-IronPort-AV: E=Sophos;i="5.84,368,1620705600"; 
   d="scan'208";a="51745175"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NyRfmk6HC3OBebgm7qUPt6LtwhalpKQe73E2e1erW4oqmh9p+x8C3+Wg6b17ZIV5z7OgUs4vG9U/YBemKQgk5OSgDgK/P/+Pn+eCybF1gnUm3WLwEqQUM3J/VqIdA+yr6YJZ5YkR6AyhhPfmXKV3kWa80HX+TNjlW/ROnzTQ5x5Nf+D1ZsBRE0bVGErc30nTd4lYzo5iLhcx1CWWGP1H0Cy3ogwtYowt79VV2/etPaxSVILamVAu6HU0/NeGkafJ806ZxcGTwJppOVVDC+ulKw2T3TEk9B6cbxxqKLEs1Eb5ZDCbvhJi9RZPybZJK8lOpRIaFs8Bpga04CtJsfkgIQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HTMmjGYrRe680AdAYN8pAzhWSq3wfk6ltNwSmWH6CGc=;
 b=cBKl8tmsheOhZ6k90EizphfhYZKoIy96QmAgxnex2devo6Gi612xcbLzX5N+ePFDh3T/5YFdTxqBXPOSkolQLp1mFJCSXvUSy/6dNCkYWBJekjr24ln+/RMNWdGuMz8nScWPCnDR9J6VQO1w65B/XDCTOVlR9o+vZS4Z1hpyYnE5E8Nss7TzM5ezduI1/VuQkHTYse270w8XwAq8FLrvYoyKpXpSxz+ZQ2mtzP2vW+RhlwuQCjKCvqCc2qg1gYiNkHEXYVs4wQCxqBKMkwfb4NKAXgfnKrVJ6/tIBzCqfBDT43GZViDav4oQGdqpi6Ji0RN0ga5GjZ3hVt3XP5iLGw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HTMmjGYrRe680AdAYN8pAzhWSq3wfk6ltNwSmWH6CGc=;
 b=t1N7a0gnFrUqa1hKDEKk/iFh48P+yJ6424FdtOKzkmHdjxtS50sbcn2G88Guii5dn9ZbX9mNeyD3zEVahB9KkJQbqWCVoyZx1/3opWLThC9LORnXZe4bVhAh9tVg7ni2OhWlVGOgpBYf5LeSp4bUunaa36+/D0HipMl12K4TFuE=
Date: Wed, 1 Sep 2021 10:41:19 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/4] x86/PVH: de-duplicate mappings for first Mb of Dom0
 memory
Message-ID: <YS88r8O8WbIORlgT@Air-de-Roger>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <013c49f5-7a14-9d9e-4d25-75934bf6349f@suse.com>
 <9cfc122f-dc4e-9dc0-0b57-48abf941025a@citrix.com>
 <866a3677-1f1a-e0e5-7ab0-10c210e25610@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <866a3677-1f1a-e0e5-7ab0-10c210e25610@suse.com>
X-ClientProxiedBy: MR2P264CA0120.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:500:33::36) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 63312824-9005-437c-dcec-08d96d2448cc
X-MS-TrafficTypeDiagnostic: DM6PR03MB4140:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4140E1AC21A64AD5B6715B098FCD9@DM6PR03MB4140.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: oFfjqbKFZGyFWg8oofuPY4L3tYXGAa+cnyIkF0iQjTSe1ijqXDIWW5jwjaKd7KRkXP4Su3bJZIq9X7ngKz3kz4BOEt7BCQuejHsbVcFMpHK1Ubvrp8hC5oWRQ6yYfN+tb5R/8hD9m/sx8UgBwebDsT14rpWQ++BvSIAKJH8plpRInun9SLYM2SSaE92BiRts0heJMctMSnjvAWDqVGHjvUU3KmlFjWayAdSKCtXXLD8dxPg8njmztEbtj8e30G/0rH8HunMyvZZLHvF8OWyPrZdrMYQYLKAGQ9l3m+j80rHRABUJL14vmx54GEsxXZjRsmbMqlj8mb8Jz6jog+o8W1vr06w6RmqK0CJ2hhMCpEhUmtjBBSEepxsF9Yzsma8zPeP7ape0PL1fwYlvTJ/yr1iS0lgiJrBykQNkpZ+nm65KXlIGbf6EdKtRXy5HwYQT0yXqykKbMNXd6rgRandSAJtA932lNtuCCdKept4M2AJTQVR9cfnkx67jnpgCbbiD2oufG84om2B2kvyDgQyJ6VXqsvaB0ODFTzQGZaTwdhbVx6DaSfGjb8ZBY0jLKzHYmGO1wR6Shh5QtMWnI3mUz9qIlAImfDzUxQp0vR9svhfVopUf/sbElr890KnpJGI4fiAGkWUZIUdH4YXcocbPfg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(7916004)(346002)(396003)(376002)(136003)(366004)(39860400002)(54906003)(6496006)(53546011)(66946007)(956004)(86362001)(66476007)(5660300002)(316002)(2906002)(4326008)(8936002)(66556008)(8676002)(33716001)(478600001)(9686003)(85182001)(6666004)(6916009)(38100700002)(186003)(26005)(6486002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aEJMS0YvSWxhbUo2RVlvSytIZGd1U1d4d29LMzRFQ2xsNE1QNTFTdXI0MFhN?=
 =?utf-8?B?MUMvak10ZWhxdktGR004VlFuY2UvYWlvS04xb1loYW5wWFhaMkRyd3I0U3Z0?=
 =?utf-8?B?NW84VVZsZFdESGR2ZlpPZDZmWFdrLzBPK3ZqNDl3alJCbWVrblp2SHJvb0dV?=
 =?utf-8?B?ZmtNZUwrV2pDSzFvN04rU2oxTjJDUDNKb3FDLys4YkE4bExQcmwwRURONVBX?=
 =?utf-8?B?TlQ1aUI5NDVGcUU3UGNVT0g3ZjFiUTlLQ1gwR2FHR1BNQ2pqbVdaRVl3dEF5?=
 =?utf-8?B?UU9ETjljUWpwZTAwM2llUWVpbnBDRG9hVDhUVk5oUjA4RjRwRHorTHlRMVAy?=
 =?utf-8?B?MzVPUXg4SVNEMHBjNWEyNUpmT1lDckhBdEMxQzZLVGpWN1hwak9GZzZlSWRm?=
 =?utf-8?B?TzNVUXdJL0xhTUg2R2h3a1VWODFydHI0NGVYcnZSV1Z5ak5rbWpORFJUeHd5?=
 =?utf-8?B?L2lpa2NmMjFzb1dqc2kvVUtuT0V4QXNuYisrZVkrVG92dnlzZ09XOTdXMkxP?=
 =?utf-8?B?aVNYejY2eWk3UGpydXBYL2NMOG5tUzEvZlhsUE9jQS96OWRBRzhlNDB6OXkv?=
 =?utf-8?B?RE81Zm9HdVc5TDgxMVZGQnl1aXl0L09Cb0pRWUxYZFYxVkNyMDRCYUZ6MW9r?=
 =?utf-8?B?OWMrZmxSbDVSM1ZNVC9KaDRjNVBmL3hpUWtuWVFXdU9YWDJjT2JRTUtPY2g2?=
 =?utf-8?B?R1dVZ0o2MkhmZ2djRThKZzkxYTJCWWFiWlpkekRYUFFoYmNpKzRjLzZUdUdU?=
 =?utf-8?B?ZnAvcSttSUpaeHB0L2x0SGlOelpmK1RHTTZnOVNFVy94dm0rSnBjZHdwSjRz?=
 =?utf-8?B?eU85LzJJRzFyOGNNYWdtS2paTWxBVWNyaEZSZDF1M0FyZWtiK09LZmc2OXA5?=
 =?utf-8?B?eUdvZHZodXhXdlA2dmEyMUppU1F5dmo5TU9TT1d4TytDSy9mamhzVmtBek5C?=
 =?utf-8?B?bW1GZkZTNDduSHdRMjg4cnVnNnErR1dFbFpIdnJsOENZb090OEZvaXVEWWFv?=
 =?utf-8?B?c3BCaFNKUjNPL3czSDBZMWhyenhWRkdrcEYrbUlya2U1ZlhOdVJUOGFqMzJL?=
 =?utf-8?B?R0xYeVk0SE1xWk1QZEcyNjVZVit2NjhRNmxPUVVPUnJ2S2lYajlQazhBV01v?=
 =?utf-8?B?S3ZTdHc2MnlHL2NYeGQvTXF4RmR2SmhPSGdvMHBLbTc4M3ZtSGpQcEQyaVcw?=
 =?utf-8?B?RUxVdmJWbHFtR25UZVhjdFQ2bGplRXZmemFmV2ZiOGZ6UExZRXNuSStiQ0FH?=
 =?utf-8?B?Z1V6enR0ZWJ4M1MrN1g4ck8wRVN4Q3l3eXpHaDM4c3N4Q0FEOWRzTW50TnlT?=
 =?utf-8?B?ejB0YkFqaEc1M01idTBlb3VUS0pTdGN2WnIzNG93ZDh0a0ltR1JTRzYwd3NZ?=
 =?utf-8?B?ZjF2SHc3S1NUSk1leWRGaEoxSkErNEREWnpKZnI0VWZLUnArTW9vcmk5RFNp?=
 =?utf-8?B?SDl3MysySDRCblF4bENjTVdoYnh3cHc0eGVZempTRUZCOU5ENkxNMUQwNnVT?=
 =?utf-8?B?QnYvdTdkR2srd2lVOHZoQ29yOGFwTlRHVzhnTUhMc0NmczJOdXFiZk8vZllE?=
 =?utf-8?B?Z1hLN0QyTUJ5STJONGhBV2xIYmZuR1lQMDU0ODBsMFlJM3pqOUVwdGRMTlgx?=
 =?utf-8?B?SmVnUFF5ZXQ3R1Y3NjRJWStZRDkvc2pYbDRkcXFPNHBqb3VEbURxOVN4Q0FV?=
 =?utf-8?B?SURlVExHTGRmWXp2cE1iQzM4SkdCUDhiTjNTK2VtOHVISzlNZWVkUXJuVkdt?=
 =?utf-8?Q?OIgU4KQuYmmDgcQtFARgC+Gj0os0XuoYkk0OYtj?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 63312824-9005-437c-dcec-08d96d2448cc
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:41:25.8983
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oox+/mumLfWMOabDQxXyBsGi01HD9fSazdkDdhSAPgrZNKIpwTssb8c2SdFaMOnp5vFrw/nWMbs9X8aVf+wzpQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4140
X-OriginatorOrg: citrix.com

On Tue, Aug 31, 2021 at 03:14:06PM +0200, Jan Beulich wrote:

Sorry for the delay, and likely not being of much help right now.

> On 31.08.2021 15:02, Andrew Cooper wrote:
> > On 30/08/2021 14:02, Jan Beulich wrote:
> >> One of the changes comprising the fixes for XSA-378 disallows replacing
> >> MMIO mappings by unintended (for this purpose) code paths.
> > 
> > I'd drop the brackets.  All it does is confuse the sentence.
> > 
> >>  This means we
> >> need to be more careful about the mappings put in place in this range -
> >> mappings should be created exactly once:
> >> - iommu_hwdom_init() comes first; it should avoid the first Mb,
> >> - pvh_populate_p2m() should insert identity mappings only into ranges
> >>   not populated as RAM,
> >> - pvh_setup_acpi() should again avoid the first Mb, which was already
> >>   dealt with at that point.
> > 
> > This really is a mess.  It also seems very fragile.
> 
> So it seems to me.
> 
> > Why is iommu_hwdom_init() separate in the first place?  It only does
> > mappings up to 4G in the first place, and with this change, it is now
> > [1M-4G)
> 
> I guess we'll want to wait for Roger to return to shed some light on
> this.

iommu_hwdom_init should cover from [0, max_pdx], or 4G if max_pdx is
below that.

IIRC first PVH dom0 implementations used to have a behavior more
similar to PV in iommu_hwdom_init, as they would get almost everything
below 4GB that's not RAM identity mapped in the (IOMMU) page tables.

PVH dom0 has since diverged, and now iommu_hwdom_init just identity
maps reserved regions. We could likely move this somewhere else, but
given it's still shared with PV dom0 (by using the same command line
option, dom0-iommu=map-reserved) I think it would just make option
handling more confusing.

One way to simplify things would be to rely on the hardware provided
memory map to correctly have the regions in the low 1M marked as
reserved, so that iommu_hwdom_init would identity map them. Then we
would just need a bit of special handling to duplicate the data in RAM
regions for the low 1M but we could avoid a lot of complexity.  This
however requires trusting the hardware is not missing required regions
in the low 1M.

Another option might be to slightly modify hwdom_iommu_map so that for
PVH it returns true for all non-RAM regions in the low 1M. That would
avoid having to add another loop in pvh_populate_p2m to map those.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:45:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:45:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176102.320531 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLsc-0004gJ-90; Wed, 01 Sep 2021 08:45:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176102.320531; Wed, 01 Sep 2021 08:45:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLsc-0004gC-61; Wed, 01 Sep 2021 08:45:58 +0000
Received: by outflank-mailman (input) for mailman id 176102;
 Wed, 01 Sep 2021 08:45:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLLsa-0004g6-FQ
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:45:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a0caba4-5607-436f-8de1-f43c1278489b;
 Wed, 01 Sep 2021 08:45:55 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2053.outbound.protection.outlook.com [104.47.10.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-pVaYxaYpOIaZRfTz3Kv-Cw-1; Wed, 01 Sep 2021 10:45:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3119.eurprd04.prod.outlook.com (2603:10a6:802:10::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 08:45:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 08:45:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0032.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.9 via Frontend Transport; Wed, 1 Sep 2021 08:45:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a0caba4-5607-436f-8de1-f43c1278489b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630485954;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=QwTgqcw3zlA2T/dEROtSixpo30ORvGtc6P2RdjY0Gjg=;
	b=PzS2P6efoymwx83ilyV/+Epmw81JJwTLmMQTfspyeq4HmQiGVNlXYE5Js2YdYOUZ7dd9n1
	5l7PPWkwEAd2tFFUpaMwpY0Xc9QX9W3z0hKZQnxi+ORluMpRiUM47xJJgwnRy0xs+Ltfv+
	0RiTuhW+nyVL1rZPd0WjdUsoFQgliDY=
X-MC-Unique: pVaYxaYpOIaZRfTz3Kv-Cw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ggASe3nILsQkMY9u+7NaaCb+8aZEHa5QFf3rlrn7kH32vM90qDD0Y6p+2j1IJAc7IiwdfRmRInYqgRMJuPMzxba5/caNkQp3YuNQjpLsKLDPJ1rfN3VxrewYZ+21esPSoGLiK6Sbpy8YtgRw+fM2g3tsKmfPEQg1jp/c6Ni9l+iC/0w1VDrDoHmk4jSPshwcm1BvT91qfedp+PSB6OTqRtXHfUQmgrGj+C2NlDJF3/pan6IK2zxZdDUsyNrf5Ttnp9YSfDh1t2ZzxalCJMwmsHpyOtv0iZrRVR7RPaAiVj3fFuhdzDiKwew4/xnWy8xjtayd/RWREsgf4KblS26ujQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QwTgqcw3zlA2T/dEROtSixpo30ORvGtc6P2RdjY0Gjg=;
 b=GW7w5QtN03F9yx6157WuZMzBZSZlC8Z3lySQ3EIjkeG6X4zVF/dT8OSnAHZN05If8O+3Hpc0gw3unDYZ/tVqzMZhC3OxjdV+nUc1zbD+YrzPDzOlhCmaQEJ4mowLHVwJa2REVwZ7/Rq55SGEBY1hcSBX/hkMv7si49Uo/oRbjaJrmXAE989DMXZAPW4sddwjy505S5YLVDbopCkkrzOMHHiGUMDtwtRZ6Tf3UT3GsTbncweUogGbAHkn+12L6qWXyjKOTCEeallF2/+NmL4kZst7Ahc8A/d1xFP+ALqunECcGzvMdkFgaRQSLP2BQ9jbbob2/2iL9tA6Wc895tFsqQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
Message-ID: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
Date: Wed, 1 Sep 2021 10:45:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0032.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d2feab3b-1a44-42cf-c086-08d96d24e6c4
X-MS-TrafficTypeDiagnostic: VI1PR04MB3119:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB31191826650BFB051D471F33B3CD9@VI1PR04MB3119.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+rXZ7I3qL+OTk6CJBtJ6Cnw9BeFlt13zggOo+cMurxkLReMmhcoZkHzsaiMPN/knPx5g/ZogSCP8jgliDS/Rs+GcBP7/4QhJllUb/Bvo7MCoG7dLNcDFwCgHgTfZ4RoIE4or7Pk6+gCxQ6O/1LEvhoi1znuZHJAHS+VlxfkD8PJjBOHV9asgE1h5PmrJmSqj+anEBk8FdQp5F6UQXDrpsEK37Muge2bNt9B4WUehuBz52Hj1ujGUHEti98SllaH7URR5qu3ZQ8YQnUA95rr0Mkele25T8iMXmn7il6fvzsz4UF5gA4OxrdOSCxRRZjD7m+L/kOY41xHuK17/2eD015W3CER0iFf1JhBNyS6QG4V2YxyeKHis/slYk2sEFJAoZsHgcFFEGSunxZjsEUC2o2p5YEGAP4zioc2g2BayermqOv1JcBGDqUzYh1Rr8Fqk6DRMOrfIyqTotrAoXwHKmKgbdsYeTeLjGK2RyR0rJjHHEXuaV8l3GVzySQqIvr+nQ5IUfZObeiqEaC0LyBKSS62zp5ec5h0JI+E2lJ0ypp4Wm5CKUmp6+DRvUtM39Jh34k9EItVEPNafANwSYJnxlEbRBNCohCqr5f9Gc4hho7qLHXvmuCHuRc+sjWU785G862eR+2his+XWjnRaIGtdcfMCIJ105j3E/+L0jRWGxR+kBtDSFLIUrxAeXxRtC+xKh1eK/lCBKOCTASpeuctAd3tFE29pGFmLgFI14IyUWxQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(8936002)(6486002)(31686004)(31696002)(66556008)(508600001)(26005)(6916009)(66946007)(8676002)(54906003)(66476007)(16576012)(5660300002)(86362001)(956004)(2906002)(316002)(36756003)(4326008)(2616005)(186003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VHZiVUsvdXczUmVrdjRaOTJBSFhDck9ITDBOMnYvelA5ejlqVVRzdFgzYno5?=
 =?utf-8?B?d2JUeVA0cWhnekR0S3YvRys0SWxUQmZUSnMzYldzbFJTaFhlZzd1RDJaOHQr?=
 =?utf-8?B?MmxrMzdtL243N2dyeW1TNTduSm1jUERnVmRJYlI0QjhZTWN2V0pQNDJYTUoz?=
 =?utf-8?B?bGRQaTNRRE5kYVBrSHp5MHdUU1ZFcUU1UUVzVjFnMW1UeGQ0QzgzUGcweVZj?=
 =?utf-8?B?QzVTY1RKcFRrQVcvNFpMbHRHbm1rYjBpTjIxbGMzdU1RTFN1STA3N3p2NGZW?=
 =?utf-8?B?eG1lUjJVQWZvRkVsSllNRTRzbVF4SjlXQ1RxeVhmdW9pbmZ1VGlJZDVuWnhr?=
 =?utf-8?B?WDFxQzR5VDNObjRVTUJNb29UUVpNOXNsYTlZMnQyS21MR3dMMUtpUkpHc1Nz?=
 =?utf-8?B?UENBdHlVN3J2SWFNZjFmd0ZHdDdqUFNCM1pBamJSV1dmRkQ0TkZNYVJQcUxI?=
 =?utf-8?B?dTNLNzZnSGcyTTY1K1l5ekFEaTQ4RzFxV0RoaE9uN3VFMHdKNWJKRkU3b05R?=
 =?utf-8?B?U3hGWHdtV0xVc0cxOHhyc0R4RktIYWhnNnZjbmRGcHljdnhoNUFVL1BUbHZU?=
 =?utf-8?B?eWE1OHZhVnoxZjJLREdOaFpHWHVTdWVTSVpoLzVnazRjQVhLV1NRT2RpajlE?=
 =?utf-8?B?R2ZhcnpQV21KT2tVTGovSWt6THk1aG1peE9JUzZCUUFzcDhyMlBKck9TbU9j?=
 =?utf-8?B?K2VSMXRocU9ITVVXa0JuZXBWOWZxN0UwNnJpQXVRYjNMRTh4SFRTWjlEOStu?=
 =?utf-8?B?VHB5Mk5iaFVmaDRMR2ZPN2tOeXh2cmhieEVYOHRNMjFHR3piclJKb0c1MnBy?=
 =?utf-8?B?c2dVbXNpZ2o1a3M0MjZrVTM4RHk3dExtZVI3V1JhUFpxOWtRSkdpUldTWHEx?=
 =?utf-8?B?MzRNMlVXcExrTWFSZFNReld1SDJVVkRoa09EdllRb2FxZzh5cnIyR1JIUzhh?=
 =?utf-8?B?ODBET3FGb3NTc21Qc0x5SUVVcEtGeE10cVpTWTdKMU9TS3FyN2Y2ZElmRUZp?=
 =?utf-8?B?NlFPVFZEbjBGWlZXQUNwRWxIZEhpa0Juc1NMNVB5U0M1OWtQd2hadldPNzh6?=
 =?utf-8?B?OU9vOFlHcWJFT014WndUL05nTnp4WTNPWEdwdnZsKytJdWdReXRNcHl5VTc1?=
 =?utf-8?B?U0tBSWtnQ1lremNxeEd1d09CaFg4R0x4elBXMXgxRzd0MFVlVHFIK0N4TGdy?=
 =?utf-8?B?WlliWlhCcXpOQmRwWWphZ20zM050TDVkQUp3eXIxeWJpUUd4VGlURlM2TWVY?=
 =?utf-8?B?cGFiQ01wWDNYblhkTzRpTWN5WFdsUEZlb2dhY0JNZXJ6TEZERXAyTzluczJU?=
 =?utf-8?B?d1QzVEZOcklSbjZlWU1SdVFQL0cvdk9Uejh0WjN5VFhmYXkxS1J5a1RFdmxt?=
 =?utf-8?B?NmRBcjE0bTlxQ1RwYWh5YTNkTjZEOUV5UG04ZmltUXFuWXhqMzhLNldQYmtJ?=
 =?utf-8?B?T2xhQktnMG04bG5YZG1tdnpsZHlTUWwvZFhpSklqbEo4dDlaa1NQWnhpVjVC?=
 =?utf-8?B?NHAyMzJ3bjQyZDhObDhsMjZ5UXd4dFlWOUFQbEhJSkZSVmVCa2t1dzFLRFQ0?=
 =?utf-8?B?RXViRTFESDRNTjZaZm5kV20xbk11aWFBVUNnQnNVZ0JPS2hJdUlnUFlTeWo4?=
 =?utf-8?B?Smo5di9mWTdmYjJiT2kwOUlkdmpoTDlZMTRTYUZiZHlCL0lpRGR3THE1M1JM?=
 =?utf-8?B?MHhwWXhXakNmWjMxd2taQWFmNW9Mc3NmTTdKanpTOTZPQlFDQi8yV0phWjE5?=
 =?utf-8?Q?1C0X3G6QJ3P/9qCQooSNmyb+ysDCgfQvoG5elug?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d2feab3b-1a44-42cf-c086-08d96d24e6c4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:45:50.8159
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eALn/01KGz0exzbybadfJfYcoNwucimaFylaUHc6OYDshdrzUvnB933eze2yzSBIaZ/rNX0blXKT59Pp1oye5w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3119

Commit 806448806264 ("xen/domain: Fix label position in
domain_teardown()" has caused Coverity to report a _new_ supposedly
un-annotated fall-through in a switch(). I find this (once again)
puzzling; I'm having an increasingly hard time figuring what patterns
the tool is actually after. I would have expected that the tool would
either have spotted an issue also before this change, or not at all. Yet
if it had spotted one before, the statistics report should have included
an eliminated instance alongside the new one (because then the issue
would simply have moved by a few lines).

Hence the only thing I could guess is that the treatment of comments in
macro expansions might be subtly different. Therefore try whether
switching the comments to the still relatively new "fallthrough" pseudo
keyword actually helps.

Coverity-ID: 1490865
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
If this doesn't help, I'm afraid I'm lost as to what Coverity means us
to do to silence the reporting.

--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -401,13 +401,13 @@ static int domain_teardown(struct domain
          */
 #define PROGRESS(x)                             \
         d->teardown.val = PROG_ ## x;           \
-        /* Fallthrough */                       \
+        fallthrough;                            \
     case PROG_ ## x
 
 #define PROGRESS_VCPU(x)                        \
         d->teardown.val = PROG_vcpu_ ## x;      \
         d->teardown.vcpu = v;                   \
-        /* Fallthrough */                       \
+        fallthrough;                            \
     case PROG_vcpu_ ## x:                       \
         v = d->teardown.vcpu
 



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:50:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:50:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176108.320543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLwc-0005z6-Tf; Wed, 01 Sep 2021 08:50:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176108.320543; Wed, 01 Sep 2021 08:50:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLwc-0005yz-Nu; Wed, 01 Sep 2021 08:50:06 +0000
Received: by outflank-mailman (input) for mailman id 176108;
 Wed, 01 Sep 2021 08:50:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C3+8=NX=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mLLwb-0005uz-Uu
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:50:06 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.89]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae6bfd60-37f2-42fa-bcb7-403ad9f67df0;
 Wed, 01 Sep 2021 08:50:04 +0000 (UTC)
Received: from AM5PR0601CA0032.eurprd06.prod.outlook.com
 (2603:10a6:203:68::18) by HE1PR0801MB1771.eurprd08.prod.outlook.com
 (2603:10a6:3:88::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 08:49:54 +0000
Received: from VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:68:cafe::bd) by AM5PR0601CA0032.outlook.office365.com
 (2603:10a6:203:68::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Wed, 1 Sep 2021 08:49:54 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT026.mail.protection.outlook.com (10.152.18.148) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.17 via Frontend Transport; Wed, 1 Sep 2021 08:49:53 +0000
Received: ("Tessian outbound 1a0c40aa17d8:v103");
 Wed, 01 Sep 2021 08:49:53 +0000
Received: from b9d613c93ab0.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F7A21BAA-F4EF-4BC6-BE80-62232048294C.1; 
 Wed, 01 Sep 2021 08:49:47 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b9d613c93ab0.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 Sep 2021 08:49:47 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6511.eurprd08.prod.outlook.com (2603:10a6:102:12d::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 1 Sep
 2021 08:49:46 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4457.024; Wed, 1 Sep 2021
 08:49:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae6bfd60-37f2-42fa-bcb7-403ad9f67df0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KRa8tMlNf0zznQ/c905c3juUaUd51nNqBn86dcAqfd8=;
 b=DPP1P4YyBTHLVgVytNxvUkFov7hexMFHLg3TJBl8SMQOfKgPrKC1hegVm17uwKpeWKXohrQixPsIKlqXFuwSWHva1t5JgNf+nAAMTDCmTlT5TqpMUyxm/vDATB/vy5gvGgujSSPbJpuVfNmRYRxMm+a3z+vk9briRkb5U79wGlk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e90d8ecbde44be80
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UI71hRW21G8Kqg8yAMeV8uXNMAAjl1KN3uP68UVFSsFuvbuHHvDvvYlHFxRmkdMckTa0KD7DdLLoSErd+psF/HAulUtcEwkuPtChbWwS5rTrCUDidwQPXRf8qxdmAf4jL3IW1opwedCHRpnq5021KW7rRhkuYi0rkCcZzyBnGmJ0ma46AcOCQb+JFbZG0caP0Ipnu49NmWQByGjsuWswnRyvbvQjaaknAD9QqepEwXOi1K2TS1PSiMBvIIPQVUefjuXdc+tzc+yQRokccyrYbE/d8QAl0o2HpzCe4GNVLxpQGJtZd6eOJg8tfSZqyYVyvjuDS5cdAac0K+XQsGkz7g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KRa8tMlNf0zznQ/c905c3juUaUd51nNqBn86dcAqfd8=;
 b=LT+wNyMLWe5lqo1LDTP3O3LWLhmj0zbDh4HVUyxhNTrYWDIEapZ6GHW3CB9TGjbVPdPU5n9fglpUCb+XnQWFG3D0/5CgHj9mgxjA7TRwCTCfyTX+TkLXF6OfA4qx5R+hnXQ/Nf4lKBVseMGRWfAq81AZbeD9K29BURTOor/RKTvMaANRj+4bCoUdrde+kMXt0giOC2zSBptYWnfAFq+9/atdWLr138wy0K40eIq+iGu/7lwLP7sSN+oqJ9i8inTRinwhHiUsvS2kfFBJufMdV4ZAwCIsEFp+bHy42Fk/lJYbeh2RvjNbvDREu+S/NRrvOFaTZ3xfLLLiaBL+h6x0ZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KRa8tMlNf0zznQ/c905c3juUaUd51nNqBn86dcAqfd8=;
 b=DPP1P4YyBTHLVgVytNxvUkFov7hexMFHLg3TJBl8SMQOfKgPrKC1hegVm17uwKpeWKXohrQixPsIKlqXFuwSWHva1t5JgNf+nAAMTDCmTlT5TqpMUyxm/vDATB/vy5gvGgujSSPbJpuVfNmRYRxMm+a3z+vk9briRkb5U79wGlk=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
Thread-Topic: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
Thread-Index: AQHXnw3T6W43IPECMEKENcWB9l+Su6uO3oGA
Date: Wed, 1 Sep 2021 08:49:46 +0000
Message-ID: <9FDBB4B2-9025-4F75-BDFC-0BECBBBEB7E2@arm.com>
References: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
In-Reply-To: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c405c1a3-d6a5-4398-774e-08d96d2577ac
x-ms-traffictypediagnostic: PAXPR08MB6511:|HE1PR0801MB1771:
X-Microsoft-Antispam-PRVS:
	<HE1PR0801MB17714584A9E6B32937BE0B359DCD9@HE1PR0801MB1771.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 a4liixixZHgl5BOPISpQc2TNxHRoEJbnwKzmC5FsbOvYr8TiusfqlknFbWcphTCmTejuAgzYwkAx9gqPq8swO99EIJdzBJ89XzHFnGWn3OM3BBCk0/Ibu/epsCha3EPlm3du0drDQvpugyZMsq6GkBDOSezJmFXC+3Dyws9u7QvEFllhvROAhEMSp+d3743zkyBhKQfgqxGXBLOEm+ZwQMNcpa/b6bpqQFS3NZb7A3bsmS+NHInigQXrybjn119BuDzVMAByRcUh0QtVF/ynpXJD+Y7uGRKCNGfT3Wuk6lh8hyK7n7j7NN6ABFuihdDEc/ZvTkZciBxsMVQcOiLiW9Yv5UFQxxHOLXlUbJhl8+m+hrHnq75/YrJNFrPNvFHOJk1pd2kfc1Ubs/REJKwSexJc5dw62cv7bRvEOBgLJCtcxiFs2EE/1qVSDbxAhwladLQXEomvpyJ5DyF39KyBQi4fPFoNpj35MZLG6bt+ysOXY60Psd7jvFVnitKSQCnLDsf2hrm6GHitftoJ0ECaRZ6jn3wKOzqIMTYnlQSXKb5PK8ncZc8Y7ymlHGeuH3e+RGDgOwehxgilN/3WpKMoYd4A59WssObKfXJ3ZC8m+UudLGBr8FxaAJvvsM/tmPq42PuzgP6tH8pN1zkzA9TVJ4+B4q/VGXrJwqxe88GFbRMrvxRpr1eWiiCLHQXSdneeA5BIpW9go50LsBCSdKUpsslRIeoyMjjoQ5rMc5Gk/R8=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(346002)(366004)(376002)(38100700002)(6916009)(4326008)(38070700005)(8936002)(2616005)(91956017)(76116006)(66946007)(5660300002)(36756003)(54906003)(33656002)(6486002)(316002)(8676002)(66446008)(64756008)(66556008)(66476007)(71200400001)(122000001)(86362001)(2906002)(6506007)(478600001)(53546011)(26005)(186003)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?or1y6CDxGOMF4EGSwNPO0ruz04HAJVBSM9AoECLyeB7hLql9bPavIYzv6nN1?=
 =?us-ascii?Q?nQ4zohg2XAlFnwR9bPNdVb8HjAaKW3AamnATGi9NIB1MApU0IzQqpLtoXqiW?=
 =?us-ascii?Q?1Lq5jjooK5gt6MaXHtEX2QYysOOYVmTmMgERGzPQG95y8BtG5hw3WtNbfLsQ?=
 =?us-ascii?Q?MffdZM6nBgL1lT0VeTzKK09CPZ3QwFbLaqD/Cyu+XQgh0cLskrzjh8nS61me?=
 =?us-ascii?Q?69Qdvl8uj4+pNaaOTnrbFa8YEUSdRsyJCU0ar6TnBnf3Lwpm3Ek7U+Tu/UdV?=
 =?us-ascii?Q?4CxwShO0SWtIeoRIEok3V/ou9l26Tc9L2HvrtsdZfO6Da0/qxfeM6LtuheZK?=
 =?us-ascii?Q?OcaaWukik+g2/EmP/9pgNTgfx0fT0oTxBQuS98edLY+ySlS6nR5FILDxuKKs?=
 =?us-ascii?Q?cY2z0r9Ael0YJjLW4W3DECkYk+BTH9dcbApbFuyOC87zY6WxucnLZ30TprVP?=
 =?us-ascii?Q?hMsOTcWxe5oDKpAle6zE24SXf60LNyQc1avpvvBIB5e3Gko+TzBuJOcayLkg?=
 =?us-ascii?Q?xGJQon33d1tWcywQ4QGLB/lB29oM330IP1h1n7Xx9kT6vjhA8u+EL3+oOD+q?=
 =?us-ascii?Q?PZeGGMRImBMF2KtCN2IXiG7NYxpHLGP+1+MzpDpkoRUCWrOFtWymfb3ZOT+Z?=
 =?us-ascii?Q?vzmZZgohTuV94GcFxWfKsiinhfyKZ5ZzIucmBx9wRfbO1xadrP9whFZPjMeE?=
 =?us-ascii?Q?AVwg1bBR35pMw9wBcBqcIVbn12Mk8j79ihJKrKVYWYIVpRUf3iWvaBMG6YnS?=
 =?us-ascii?Q?6DeYUMKIZuMCviD4MeTRig78zX/oI8sfQFgIcBIj+Oldd2XkEWnf9GoROMzH?=
 =?us-ascii?Q?kvwc7XkX3Y7VTlwbc6lCCQGPvmjIco25OFN5Uu09mIEMbz2+Ejcr92HKR+/J?=
 =?us-ascii?Q?aKLkrm8qSxQ9O2yKdPUwae8PHQt4ckQOie2ECqUQrACANAUmhMPgsrljnWnv?=
 =?us-ascii?Q?IFPkRYoLuxQ2t8UoxAiFpcUjWI6UN3jWwwYIh+1j5DJZkZU+/W1wnGobALuT?=
 =?us-ascii?Q?8BYkJrwUg3alqg7Wjuqm70WeqE8S8ri3Uy0QK8q4/f/+sMJ3AWHkisikrvLS?=
 =?us-ascii?Q?yRa4rtA4Y/nPRtIqLuhufEJLiML9stljW3JYtGCjfqdyUBLSl/JyKypB9AMm?=
 =?us-ascii?Q?KRO+I+Gh5N9MS3l4lLqUxmIfEXN7hNnwYuj8ShREljeb/j9eNMDpBRTPRcSH?=
 =?us-ascii?Q?ox7wIiJMBWu+rrVIPkqZMgsaAwDiUzeskbV+5hJdiYnkGSehWxT9iogus/r5?=
 =?us-ascii?Q?DdNPB23UM48uMb8IF9RJji6ekdfl+FqHSitMgoQHIsjDbSWYkMLrnSjm90cL?=
 =?us-ascii?Q?ajqlHVsbg+OzQkd2S+P4uWcP?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <6569E7965F64A44099542E5F69489ABE@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6511
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7289598b-091f-4deb-48d0-08d96d257341
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VR2y68/b0zRLJYufOzKYD2vExvgRVyu9FRvHAHeENBMlKr6DdBjK0tltknXT+eJDwwJnRSnj3wOUsdhUqzzQWnmQi4a27OtzKSZ/5lquCgo0J2kkmdVby9bYGFd62SBOIxjJI7XU/7QjjovOOpvBtp+onozMUkUZN7HesyHTsAB51F59QLxzYR7JVkRBdmSILN8LA8CC4RdaHZ/4nTGcb9T06VmSkkTYInkYNEbGRZQXqxOAkcl/QGmhpi5x+0YSepvSndFhTDVxz01Xxx51tVQhaOLEzLuMZRfP/uKG9pgjNgiTYTxtOJcDc+kqN3w23+790E/ML/LNwXJMSBziBf7TzhIJNWnPzetYoWS6Btn6liUbUG5EpiCJFr9QkDLPc8BJAC8ieCqh2DMG6XvltuXOJvrRdfPAWgdUR2gAVL4DauKRB/kvpK6MikxBidjwJApyCIeci9vG70n/6u74BRyap0+gREW2nD/isT4gyGV9FCCc2uZv2oGMZ32zETAcohF0lbqNflqDPs4jbV2VjuqT2HdRY2tulEYM+IJ9zHItXYiUr4PC5PILC2EakPt3in97JIZJfjam+/QqKSrOrhzZvC9T3LXA40nV99cV9fakXdS4yMcVneEcVON/pkt/gup3xy5dGEZV/Sr6+DY4FbbmUnqb1dVPnhMXchFZsg0V3Szf0p93tn4AMHZuKz+B6UlFKgoM2s6N+wgf96abxA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(346002)(396003)(136003)(376002)(46966006)(36840700001)(8676002)(70586007)(36860700001)(2906002)(70206006)(6506007)(86362001)(8936002)(4326008)(6512007)(356005)(36756003)(26005)(2616005)(53546011)(336012)(6862004)(316002)(82310400003)(478600001)(186003)(5660300002)(33656002)(47076005)(54906003)(82740400003)(81166007)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:49:53.7601
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c405c1a3-d6a5-4398-774e-08d96d2577ac
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB1771

Hi Jan,

> On 1 Sep 2021, at 09:45, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> Commit 806448806264 ("xen/domain: Fix label position in
> domain_teardown()" has caused Coverity to report a _new_ supposedly
> un-annotated fall-through in a switch(). I find this (once again)
> puzzling; I'm having an increasingly hard time figuring what patterns
> the tool is actually after. I would have expected that the tool would
> either have spotted an issue also before this change, or not at all. Yet
> if it had spotted one before, the statistics report should have included
> an eliminated instance alongside the new one (because then the issue
> would simply have moved by a few lines).
>=20
> Hence the only thing I could guess is that the treatment of comments in
> macro expansions might be subtly different. Therefore try whether
> switching the comments to the still relatively new "fallthrough" pseudo
> keyword actually helps.
>=20
> Coverity-ID: 1490865
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

A grep inside Xen code show that we have occurence of:
/* fallthrough */
/* Fallthrough */
falltrough

Should we actually fix all of them ?

Anyway this can be in an other patch.

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> If this doesn't help, I'm afraid I'm lost as to what Coverity means us
> to do to silence the reporting.
>=20
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -401,13 +401,13 @@ static int domain_teardown(struct domain
>          */
> #define PROGRESS(x)                             \
>         d->teardown.val =3D PROG_ ## x;           \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>     case PROG_ ## x
>=20
> #define PROGRESS_VCPU(x)                        \
>         d->teardown.val =3D PROG_vcpu_ ## x;      \
>         d->teardown.vcpu =3D v;                   \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>     case PROG_vcpu_ ## x:                       \
>         v =3D d->teardown.vcpu
>=20
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:51:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:51:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176114.320554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLxb-0006n7-5X; Wed, 01 Sep 2021 08:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176114.320554; Wed, 01 Sep 2021 08:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLLxb-0006n0-2A; Wed, 01 Sep 2021 08:51:07 +0000
Received: by outflank-mailman (input) for mailman id 176114;
 Wed, 01 Sep 2021 08:51:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2BIO=NX=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mLLxa-0006ms-Ba
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:51:06 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bdaf1fd8-0b01-11ec-adba-12813bfff9fa;
 Wed, 01 Sep 2021 08:51:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdaf1fd8-0b01-11ec-adba-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630486265;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=c5BPbbgDzXeGsbVpJsVVbJNoVdDurBT7ZZTxaMMGzIc=;
  b=L7kO4sILxYSXL55X0OiPDgnBiFC1HRH7AORZgeg3Ldi5Tql+Y12QUQ2p
   ToKulxVnmVOzp7kzSDMV58Lv9usBvqeoB/cHfLAFio1Apuj6RQuIVb3rT
   Rbx/oVuOva6OzcUTME1eQAwFlGSZWakuBDmF4lPj9oc6R/eGsryCeB/Xz
   U=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: lgfB78udmYi6OtbUqX306EOhJxFENfODhTtu2W5RrgqhwIDVpIkwcZ5i0F9BhawhweG2BQ9ER5
 fTCoqURUoT9+APor9ZoeVdOS1hh+bzAUWIaomuleZgwbbQsj4Tx2Hy/rNafYGbRTVD3tOjN1n4
 /1FBLOOk2g439cJfPN55RHnz7kRgMt4VSazqR5/ZAA/0wVFu8VIKMdxrOHkmPsPYOLQnJRSFPK
 jYsiWniuTbOKGrFxr9KReIknXXbNMuqOVuT1ZvBCVhT1zOOI2ylZo7+7chAr7QR3cOwr0Vo4P4
 JE505lO4P5szXlesGlih4fMQ
X-SBRS: 5.1
X-MesageID: 51745793
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:08NuUKD+Jn1i6JflHemV55DYdb4zR+YMi2TDtnoBMCC9F/bzqy
 nApoV/6faZskdyZJhko6HiBEDiexLhHPxOkO0s1N6ZNWGMhILrFuFfBODZslrd8kPFh4hgPG
 RbH5SWyuecMbG3t6nHCcCDfeod/A==
X-IronPort-AV: E=Sophos;i="5.84,368,1620705600"; 
   d="scan'208";a="51745793"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QgXrF9bGjjjNkBdYvOBcL7jyF58lasdR/IR/CWctvywq8YtpkdNGUpJHRDqBaZ+fuc8u2PvEOgm6OoyLGdPGA6UALRGwWi3pdT329954L/weVkLqGelvGoFa2yqgsQz5IO+rqwpC/sq+wlHgN4u76SSeasjC17TegLo1utt8RmA7mQoXbXWwi4CE6V1uww+gGIp1qvfuIiLfhGJ6SOd0Rr3YJHdRb5znZa8H3oQBeCoxTL/s1hbM76GD8nlUpjCJbSwZ5rKm1xbuiKZzRLnESWXVT0PjgTULyLHHZv6hAPN3MTxMvmhxt9faXdP4jHY6PHd6zkXs6ee5ApLUxVYc2Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=e4RLCQgFsrNS291skH1nHoOEX+XScfAtKaNRaZxFwoU=;
 b=PyKHCnG0INawMV6wHy37ClZw6lei8xXJGZV/8Bt6HfUJIeeqjgPh+IGMIxBTnNfhsYicgXe6NgDXMRMvb0hcLvTButqlBYVSD6AbsJFAlJq4/4/zWCs5raEPEzsvFwrFXAGv9mWoKkWzrADqTDkG135OvmQ47p8JVcBNfYbmScv4BtjdTWu+TDwkhi36kv59GAqfAz2Q/05kWIKqrtUBNzYnBUwSzjLCt09E+QB8zG9ush0EhcGk04e/e8uB8s71VeaI+rsZbQcrEbYZcHFugK2vZYBSr723/Bxllx2TVtChUUfxqtL9hizFfm9l2iNo1EJdmG/N5iwswjAI6vLhxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e4RLCQgFsrNS291skH1nHoOEX+XScfAtKaNRaZxFwoU=;
 b=wUy4beXJniV/LrimrjE7RQfJMclVhlrN1zNMmT9ebFcqPcrorCcDhVXMdEAzDHOJv1RGIW7XX45d/IdC5IVUeAwvBnNquIRYs4dJuSVsfbMZOhrScjgdHeMWvlPjtYSRXYmjd30bc+eoNQLdRNeYVKgycdGNRDbzUks6th/09T4=
Date: Wed, 1 Sep 2021 10:50:55 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
Message-ID: <YS8+72EY4FrvSSIx@Air-de-Roger>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
X-ClientProxiedBy: MR1P264CA0004.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:501:2e::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0e507f84-7da4-4c92-8d50-08d96d259f47
X-MS-TrafficTypeDiagnostic: DM6PR03MB5340:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB53401376433CE66242268F668FCD9@DM6PR03MB5340.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1Xi1t4ey2SfWP5qDX/83sCaQBR0uWj5q+08rnJZLTDj9kRl1f+SZ4f/mytSD2kwvXGKbU35LM75c3MQHC+/a7OFdBrvl6uV5uZoCgY6LlMqvG58M4TbE49FwJfhh7sCnaR+C8X+V+iCFZWIABIBTRaWZldDytZ9YPuesYW2Wj6JhvAE6ofd0Y5JLHGqDsm/oeLot0+tdCXG/V7QvuCp+TMPDp6qWHVNueniwtVfJt1Zm2YaKaVZuGEeLOaT4MgooGiaq1RtVTGswikKqt9Zgn3lb3eoQVWWPmbKa/tfdYypfFP5jxK8KEMkvRU9+ItqqgGzi7WfHMyXe4Z2SDtEpj9loH7E0OyXc6H7TKwcm7jLWw8HTMYjn2A7gM/TXhMZsV3O1EWWiX3PwjPAYhDWHPDOZcCW+3Vf9rDpBcTnZZ8yODj9sjtpKVLvzxAzU444f90IR5bNom1xiSpE7i9n1hdbJVHcDoVr/tvuEImORqtjKa8T8H2hPdD8ffcjqYcRJlgWK/XWROM5ol6NbwTKtxVGgH4An7i3Ob06PVbHXzLvcmQbSxVCrKZd1sB0Z70E94JTyplf1TMhP4K4cQR+7X/pUuGYgFhkYZNwRmpi1tnAQIhivXqzy+iptHimkkmBWLlbagFC5mNEAuZ4R1MMgDt8KuRAdJde8QZx0NcmyWNrdV3ATW9lpJaKg+x+gN+NWEg5lc5yRMW0mtD8403ACmA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(7916004)(4636009)(366004)(136003)(376002)(39860400002)(346002)(396003)(26005)(54906003)(66476007)(66946007)(66556008)(85182001)(5660300002)(316002)(9686003)(186003)(83380400001)(6666004)(6496006)(38100700002)(53546011)(8676002)(33716001)(956004)(2906002)(478600001)(8936002)(6486002)(86362001)(4326008)(6916009)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SE81eTZYTSszcHozazRGcSt2cjhRbmRjVFloeUhEUjNZVUZ0TnlvbWlVcDVU?=
 =?utf-8?B?SG0xNXhIbGJtYzBzTmJyNmJld0E4YzNqeUdBeVg2ejJkZ2Z6TWpIdFY3Z2kr?=
 =?utf-8?B?bjlBclRWK1R3Tm1Jc1FOeEtzV05ySWptQWY2Mkd4U3g5c01PdUtKTEJVak9o?=
 =?utf-8?B?eVhYZmdLUllvaG9rZERaSlVSb3lpQWVMcGV5RUcvNG1ORGRCV29PTGpiU2ty?=
 =?utf-8?B?VitKTFZuUnVadHFoZWlubFRWV05tb0ZTVXBzQ0tETEVqQ3B5eTRlMWFoc2M0?=
 =?utf-8?B?R1pvQjJWOHlDWFlLSjVHcFFwOW40NEgrYS9aU1ZmZFBFUXEvSzg0ai9OeVZu?=
 =?utf-8?B?OWMwUEJBeEtGT2lheXR3RU5zYXVYYnlBSnBBYnk3WGlyb3l3T2xGY1ZNK1BX?=
 =?utf-8?B?YVFBTlY1L2V3dnJNWmVFazZ2bUlxUmRpeGlneCtlRFdVV2xWYXg5TnFyeEQ1?=
 =?utf-8?B?OURVRXpCbVo3aE5YREZ6blhoV3RXeWdQMXk1bm9GYUpoSW13VmRBMmt3b2RV?=
 =?utf-8?B?czA4d0xqUmlDdUFtMXoyYlJ0QWtZMkg3NWF3aFZLUFhreEUzaTdCZUtqUlhX?=
 =?utf-8?B?N01EOGxtamcvdzlEaENCaGxuK28wemJzOUhFMnJLYUNZVHFkZ0tYV1lKaFNp?=
 =?utf-8?B?bUFVdnpPN0M1Sk9vQ0ZJV3hzUnVBQkREeXRQUW4xQkJJeUU1MW5wdWxMS2xO?=
 =?utf-8?B?SG1jUWFhbWE3ZXFFYVN4a3hzNG0zYVVNalBDSlhiTHJ3cmJHbXhIU1lTZ2No?=
 =?utf-8?B?K2MvRjEvYWZTR0NrMXFGVFNmNEtSMk5QMnQ5eGtBRTV6Z2dsbUdLNUZDQVhH?=
 =?utf-8?B?NWdhUnpGZXMxRk12cFdFOUtYK25sR2JVZkVGOGkvWHJENG1FNU9qMDdMck5K?=
 =?utf-8?B?OVRKOSsrOVMySVNvS1BKODNZL0pzUWhLb0NObXhWNDNCK29QSGdjbGQ3dFp5?=
 =?utf-8?B?WEVmYUw0WFNlNnZzaWlGb3hHb3hyaDJldFNzcW5OckxsQkpFbzBrYmc3cDZw?=
 =?utf-8?B?a3Arb29lZGZiZG9KV2ZXSnJWRHdabnEyWTVQdHg0VGg0YWlXNWU4Zy9vWk8v?=
 =?utf-8?B?Y0IvbkdWVlVFYTcxU00wM2VXNGhGZWxmdmhRZmhxMERnMWtEQzBCTm1XYkFR?=
 =?utf-8?B?YjJFY2Z4VHF4OS9lNmdnRGhNTTZ1eU1LTmFHVlJyellGQ0pwdFBEa3NqRTdj?=
 =?utf-8?B?eUJ1V21uYThhdXJrME1CR0h4SFA0b21yTU0yNGFxWWE5b0ZtVG5WOTZ2RUVw?=
 =?utf-8?B?TzlDZG40cDdBU3llTzFlUkFFUzRpZWRaV3VDODFpQk9nZWg3TzhLQnVtQ1da?=
 =?utf-8?B?QmJXZi83c1p5ZllxVU9jQzd6NFpmZDFHRGM5Rjd0VVBhU3VKLzViUk11S3lB?=
 =?utf-8?B?bUpvUU5LalJNL3huN0cwNkVpbG9mQnoxbXcveEVVcjlSUG5iNFZqVXl5TVFv?=
 =?utf-8?B?bm9NNnA2TXhhc0NVeFBaZW9vY2JyRTBOUE8yQWsvVzd1V2tHQTVDVVRkY0hG?=
 =?utf-8?B?Ym5mNDBjSVFVZFBTR0NUaFA1V2ppWm5PbEM0alN0L1ZreVN0MHp4RmVZTUNo?=
 =?utf-8?B?QVNYWTJIVVFTVlRWTGxGdG80aWh1Uk91TVEwd3dtZ21pTTdQWmZ0RXN5VWZW?=
 =?utf-8?B?VDIwV3RwZEtkY3h1QVV6dmsxQzAvNGd5cGllUXpFQ1B4MUVDTllxb0JMM01J?=
 =?utf-8?B?SVFsTWpVSHVZK3JpbUUxYnJCWlgrRWUrOVVxVnVDZnczT3ByYm5MSGk5MWsw?=
 =?utf-8?Q?UOJDg1V5OnEB1MU5/ODUtFR+Lf3X5AYflEGqNA+?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e507f84-7da4-4c92-8d50-08d96d259f47
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:51:00.4752
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: T45VGRtlXO/btqlM1/amhcL/yWWc5kq5OpllMPih6nAjE7mKAq+HeQwJIkxXRuKW6bIl4AssSoo9pJQ10mqXWA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5340
X-OriginatorOrg: citrix.com

On Tue, Aug 31, 2021 at 05:38:49PM +0200, Jan Beulich wrote:
> On 31.08.2021 17:25, Andrew Cooper wrote:
> > On 31/08/2021 14:26, Jan Beulich wrote:
> >> On 31.08.2021 15:16, Andrew Cooper wrote:
> >>> On 30/08/2021 14:02, Jan Beulich wrote:
> >>>> Further permit "access" to differ in the "executable" attribute. While
> >>>> ideally only ROM regions would get mapped with X set, getting there is
> >>>> quite a bit of work. Therefore, as a temporary measure, permit X to
> >>>> vary. For Dom0 the more permissive of the types will be used, while for
> >>>> DomU it'll be the more restrictive one.
> >>> Split behaviour between dom0 and domU based on types alone cannot
> >>> possibly be correct.
> >> True, but what do you do.
> >>
> >>> DomU's need to execute ROMs too, and this looks like will malfunction if
> >>> a ROM ends up in the region that HVMLoader relocated RAM from.
> >>>
> >>> As this is a temporary bodge emergency bugfix, don't try to be clever -
> >>> just take the latest access.
> >> And how do we know that that's what is going to work?
> > 
> > Because it's the pre-existing behaviour.
> 
> Valid point. But for the DomU case there simply has not been any
> pre-existing behavior. Hence my desire to be restrictive initially
> there.
> 
> >>  We should
> >> strictly accumulate for Dom0. And what we do for DomU is moot for
> >> the moment, until PCI passthrough becomes a thing for PVH. Hence
> >> I've opted to be restrictive there - I'd rather see things break
> >> (and getting adjusted) when this future work actually gets carried
> >> out, than leave things permissive for no-one to notice that it's
> >> too permissive, leading to an XSA.
> > 
> > Restricting execute permissions is something unique to virt.  It doesn't
> > exist in a non-virtualised system, as I and D side reads are
> > indistinguishable outside of the core.
> > 
> > Furthermore, it is inexpressible on some systems/configurations.
> > 
> > Introspection is the only technology which should be restricting execute
> > permissions in the p2m, and only when it takes responsibility for
> > dealing with the fallout.
> 
> IOW are you saying that the calls to set_identity_p2m_entry()
> (pre-dating XSA-378) were wrong to use p2m_access_rw? Because that's
> what's getting the way here.

I did wonder this before, because I saw some messages on a couple of
systems about mappings override, and I'm not sure why we need to use
p2m_access_rw. My first thought was to suggest to switch to use the
default access type for the domain, like set_mmio_p2m_entry does.

I have to admit I'm not sure I see the point of preventing execution,
but it's possible I'm missing something.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:53:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:53:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176121.320565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM0E-0007XM-Mi; Wed, 01 Sep 2021 08:53:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176121.320565; Wed, 01 Sep 2021 08:53:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM0E-0007XF-JL; Wed, 01 Sep 2021 08:53:50 +0000
Received: by outflank-mailman (input) for mailman id 176121;
 Wed, 01 Sep 2021 08:53:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLM0D-0007X7-4U
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:53:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2dc88f39-720a-4c56-b075-d83fc9c3a439;
 Wed, 01 Sep 2021 08:53:48 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2058.outbound.protection.outlook.com [104.47.6.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-39mDbc8ZN2C-TWNKzoD8Lg-1;
 Wed, 01 Sep 2021 10:53:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4607.eurprd04.prod.outlook.com (2603:10a6:803:71::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 08:53:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 08:53:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0013.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 08:53:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2dc88f39-720a-4c56-b075-d83fc9c3a439
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630486427;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HQb3lb36j6E87dv80nGPY//1dizzbRcvuL1J6kgjH38=;
	b=ex3eMQDxbRNo8vHqSXxuYGJrsyLvj1Mmh/7rGu7y8Y4hoAvcu6tNUJtI1ygKG44F+IrwG8
	l0rG5kPxlLdse9sLk7XWeyT4fSiWsYqvcX0jNBJuWlrAsSRoCy4wsYGjgOrd/Y56lrlBoz
	yAbFb2kMx8sIxpSQF6b5e1OwiogCuKE=
X-MC-Unique: 39mDbc8ZN2C-TWNKzoD8Lg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b4dzlHzFyys8qyCTkFUK+XwBQUjgKRHIuqIzhW0K852SqRbf3m6ZSLyUMMzMyDmdfHTa29FKOAG+n9kcnuYfFp4Tqq6o9r9eABKducresqVyOQ29egu4tnzh2NmltVRY+FRhFlL+aFkYPhhBWC1Z17mdeLic7yBpeNXGVZJaAyckg/4aG/H5cLAcixNLtOe5vvgShxcqyzjSvIe2vSpEA5hdjKVBmCIrBvnrVkoxUAL6BlPDyMaSqh4gT/SKQYyYOXApRu22EaD6sg+0OYdVA9yjsOgxoxhcduUjC/jmRKxidFbq5rjPibfY2OqhiCIZhDRO5t4TSc8w3L4mH2YyDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HQb3lb36j6E87dv80nGPY//1dizzbRcvuL1J6kgjH38=;
 b=iLXVU6OnweTr+7K1JVNjH7p0ti4CoxZhkMYzaKVB175QQqURLT+/Vy7DQTsF3MKFSFMSgRHp6ApMkeJG6/M4/j33J5FGf8ykaDw8gwlbqO/wL5zUDmj4+5QXdF2ZUMvRKLVBlHrhTBEtobJ8Ralcu30OHWzDOIVy2V3hJeaGOqagZn2Vl7hIPdKLvopxISIvNMq7FQbjbBdGJitFlPWd3i9HBMiOjm+aT30tts5Ipq9Y0yn0+EBV3lxwGuPkHe94BjyiJBm5ECjZ2r1wHwSxJqHSAE/+X5phJoWFk/l59rFs3wvcInwHjT/oxRNMqAuzDzMCZ8l2isNy2Inx2ZujJw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
 <9FDBB4B2-9025-4F75-BDFC-0BECBBBEB7E2@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <175ce359-6ec5-6847-ed76-9aeb51705fe7@suse.com>
Date: Wed, 1 Sep 2021 10:53:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <9FDBB4B2-9025-4F75-BDFC-0BECBBBEB7E2@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0013.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: abe0983b-96a9-4a1f-b57e-08d96d260140
X-MS-TrafficTypeDiagnostic: VI1PR04MB4607:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB46070313F34A8270A2E202DAB3CD9@VI1PR04MB4607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iLrw0glNYC6MoVr7R7vA/7Qj7zjZmcMGQDOfqEZlsrrneW8Dzbm4eFqvndLAC9HDQKT1uSJEWtWqjFE78oOfEnicnhjxL10ZgFq/aJteUNhkfYLgqPXBP9jjbHKKIDFDMW7zIYzQ9npRXz1ByvmszOrnml4lPTErP0bPK3Kwkx6ojSzF6SYSnVcJBouSb3nLTF2Wiksdkd+kwJ378QqJBSAsqbGTa519rKup8V0ln+PyICLnCXkfJEr68TvM/VJ9asUU9DZLiCrr2HA4uqjVxnhqzOnkX5Q6sc8V3MUDKqDHm3AwUOcfgxw1KqAFGRnyRUVofeDaxmNvvP8KdBusMBYtdJ1m8ekLfiyN+IAbaBKPN1PYn+RkzR48V0TFHYGxvlYVv83Ihr0nSbjQXbVU2Uu5n8FszFYywM67mRTfeYgsakgf3h+jqX1/XTW9WmSVh4AiBtoDxKw0mFeJu6TqzbRrED7h/NdFdSUyiywVwd3ARpTQGuvKFeoFL9Jp9SUJOkMtUoCXAld1olY1KZLxx9e7Zf0fV14uDo7XupKjI6RzdFuMnYxF0JcTRwh/jQhXcFcItBCa4Q7ZZTEuS85zahvvYDXxlR6bWYfzMoyq6OSc9Vw3GKJsaWZ7S0ttGZgZ8eUxfolY8AHkZbJtz7/P1zhicHCV+JRV1+W4+xMaqDBe06FWFCNZ9lRJINbOr+r73U0pM9r8WDGx11gEqRCMukemZVDoO8cANs8t0XRnk8M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(5660300002)(4326008)(2616005)(478600001)(86362001)(38100700002)(6916009)(186003)(31696002)(26005)(53546011)(36756003)(16576012)(956004)(66556008)(8936002)(54906003)(6486002)(2906002)(8676002)(66476007)(31686004)(66946007)(83380400001)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aTRwNVVJOGdBSGp1Ykczd2NzZ2d5OFd2cG5TTzFVQ3Blbnlrc3NNbHY4dzM2?=
 =?utf-8?B?R2hwR2orWVk3ZWJzc3pkVFN1MWswU2hvc2ZXaGc3dEVPVXNxMS9GTVNuQ3ll?=
 =?utf-8?B?NVNlY2tONEduWFloWFBEcHprL1pFTUtQaFA4bTBSbFFYRjZ5UDJEM05nOWRN?=
 =?utf-8?B?SFJuNnI5WXlqek1jSXVkOXNBNS9ZYm5UMjNWZ1h1dTRKTVJQelFGMEhVT3RL?=
 =?utf-8?B?ZGZkN294RXhyQWtQTjJpc3EwRUZNVFNPSnNoSksxaFhoMFBrTm1XaUJYeGFh?=
 =?utf-8?B?MElBTnRYaFU5SEZNUml0ZUswZWNlTEwzVTZta1EzOTJkWXhxRDVHYjNxTUV3?=
 =?utf-8?B?UXE1NEFrYVdFQ3hGV2UxTzJSb3o3RllGeEl6MW0zSGliTjA1OEdOOEdmSVRU?=
 =?utf-8?B?eDZHYSt5SU5UVm5QTXZGZG56K2JQQ1ZWZEdWWGlYUUlKZzJGSDZ5MTlreEc3?=
 =?utf-8?B?eGZuWFR2ZXUwZytWMHFTL0Yxc1JiakNxOE9ubEZSN2lnMzVrRVFMOTQrK3Jk?=
 =?utf-8?B?WGc5Tm5hTGFmL24xMXQ3eHRmSWY3eGMzSVVhcDBZMXNpMmpBblNxUUE2clV5?=
 =?utf-8?B?QzNRRjZUZnJNSWZkeGZFVEZnRkxXa0RTTVhOMVBSQnNwNklNWjJGSnNHV3Vu?=
 =?utf-8?B?bnp1R1hXTkVwR1drV0ZTZzA1OW44enBSeGdTU0sxYXptbDBZeEplVjZ4L09x?=
 =?utf-8?B?M0hkRkNHUlV3TG93aGNCVnk2T1Ntdk1sd1ljS2N5eFZvNkFHN0hqZ1lVOUFC?=
 =?utf-8?B?OWd0MTBKYXJGMjFib05XY2ViUWkvbXFnbjJWK0VIeWlVVXJnYmJrN0RTQVRO?=
 =?utf-8?B?WXN2ZG1rVWxGODBlOGhzNXBJME5SZFIzY2FUMHdsS3h3LzNydWVhVjlpY3Jo?=
 =?utf-8?B?SFlOcm90ekFKVUl2Z25EYjNxUFlUaTIwM1ZnakF3WUFNbFNaUzJhNnBBV09R?=
 =?utf-8?B?cy9rVE05VVM4bmYrU2VZWUVWdFFiZk1kVGhvZ2RQb3FMNzJrdGhNejBqQUxs?=
 =?utf-8?B?OHRlT3BTaitNSm9IZGIzOHcrVGM3ais5Z0pWWVFjQXFtM09TTDU0cUsvMUlT?=
 =?utf-8?B?WEhwU1NtdWNCVXJnS05uaEsxOUZtRXZaa1ZWV1BLaU9na3g5dnJGMElwa2o0?=
 =?utf-8?B?S01Ucm5KKzNSNE9TQ0hBb2dWSUJpamt4UExSMlNWV2VDdHFhTERacnFRZ2Z3?=
 =?utf-8?B?ZEsvMmZaclljZVFyU3FsN0ZQREwwdEI2SHNVUXdVTXNmOUdwaGt4VmY3Mk1H?=
 =?utf-8?B?OFhpVzJENWxFTW04QU0rL2RCNXJraExyZVFsZmRPcmVua0t6REhsb1VPTjhC?=
 =?utf-8?B?aFZGZnZUL2VsS1BMTjRDTDlTcElYM01EdWd6N1FDV3FSQ3pzalc5NHBlSEJC?=
 =?utf-8?B?NGxDUFlyZGxMM3hPU2wvSTBsYXhZdE5zUGhLTEYxTmQzYlcwNU5kdnpqLzlQ?=
 =?utf-8?B?TXhteWRadVdsTDJlcVlrSytRRXlTWWd2aXdxdENRMTVVRDJTMmZhZ3JtZUZW?=
 =?utf-8?B?TldEU1VRVDU2bG1aUnBybXptVWlNMndBeFQyNnhZelhrR2Z4c3hkNWg0enVS?=
 =?utf-8?B?TC8yc2k2NFRUTk5UbVRZdnJkWkNCRXlyMEVOWURZZ3pySzh1WXZGREhraU50?=
 =?utf-8?B?QWJmclBVMW1FWFFWNFloa2RGTERER1JTUFdrVit5bFV5cFczME1vZ0pyQ2oz?=
 =?utf-8?B?dStobkRNY3JoRFFSQVprM2FPOVFoTXNiRHN2N0llTHNzUndCUXlDZExhMnRS?=
 =?utf-8?Q?LbqmOieTHixu83akT8+a1Cot7mk9OKB0BnxFZV0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: abe0983b-96a9-4a1f-b57e-08d96d260140
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:53:44.7858
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oeuqzVBo50/U2wtXHBykS4FMQYqwkm/6/2bd+2n9czGAu1XP1bFMp17HO4JxvzQctOHwcb4QbOePFuRNQEEPwQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4607

On 01.09.2021 10:49, Bertrand Marquis wrote:
>> On 1 Sep 2021, at 09:45, Jan Beulich <jbeulich@suse.com> wrote:
>> Commit 806448806264 ("xen/domain: Fix label position in
>> domain_teardown()" has caused Coverity to report a _new_ supposedly
>> un-annotated fall-through in a switch(). I find this (once again)
>> puzzling; I'm having an increasingly hard time figuring what patterns
>> the tool is actually after. I would have expected that the tool would
>> either have spotted an issue also before this change, or not at all. Yet
>> if it had spotted one before, the statistics report should have included
>> an eliminated instance alongside the new one (because then the issue
>> would simply have moved by a few lines).
>>
>> Hence the only thing I could guess is that the treatment of comments in
>> macro expansions might be subtly different. Therefore try whether
>> switching the comments to the still relatively new "fallthrough" pseudo
>> keyword actually helps.
>>
>> Coverity-ID: 1490865
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> A grep inside Xen code show that we have occurence of:
> /* fallthrough */
> /* Fallthrough */
> falltrough
> 
> Should we actually fix all of them ?

I don't think so, or at least don't view this as urgent. As described
what I'm suspecting here is an issue with such comments living inside
macros. I don't think we have too many instances of such. In case my
guess is right (and hence in case the patch actually helps), we may
want to change all in this much smaller set, but as you say ...

> Anyway this can be in an other patch.

... in a separate patch (not the least because we still need to see
whether the change helps in the first place).

> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Thanks.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:57:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:57:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176127.320576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM3S-0008C0-57; Wed, 01 Sep 2021 08:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176127.320576; Wed, 01 Sep 2021 08:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM3S-0008Bs-1x; Wed, 01 Sep 2021 08:57:10 +0000
Received: by outflank-mailman (input) for mailman id 176127;
 Wed, 01 Sep 2021 08:57:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLM3Q-0008B1-Qz
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:57:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLM3Q-0006Nv-JV; Wed, 01 Sep 2021 08:57:08 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLM3Q-0002oP-Dk; Wed, 01 Sep 2021 08:57:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=2woEMNGAqbi5v4Yljw72/vRg6uMvPADMrGXsyYLrLmw=; b=J9cM7gqPONS5y8YD0FxhRvcIPi
	g0BV1hy5ajvTmAV8hx3an+GHQCZ+6De3d2cnAvR3axjZeZwxGWyvMaXWvN9a7zpUs5VwXJscYUC63
	6S+nSyUe76tBDSXtlEgEVitWQnWZnK7rTRExmZrjtTlMZNQzxvusI/cKumZRDSlYHNns=;
Subject: Re: [PATCH v5 1/7] xen/arm: introduce new helper
 device_tree_get_meminfo
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-2-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <06395a17-24fb-35d5-8332-a6ca42dbc183@xen.org>
Date: Wed, 1 Sep 2021 09:57:06 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210824095045.2281500-2-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Penny,

On 24/08/2021 10:50, Penny Zheng wrote:
> A few functions iterate over the device tree property to get memory info,
> like "reg" or the later "xen,static-mem", so this commit creates a new helper
> device_tree_get_meminfo to extract the common codes.

The commit message needs to be updated as the patch has been reshuffled.

> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/bootfdt.c | 68 ++++++++++++++++++++++++------------------
>   1 file changed, 39 insertions(+), 29 deletions(-)
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 476e32e0f5..8c81be3379 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -63,6 +63,44 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
>       *size = dt_next_cell(size_cells, cell);
>   }
>   
> +static int __init device_tree_get_meminfo(const void *fdt, int node,
> +                                          const char *prop_name,
> +                                          u32 address_cells, u32 size_cells,
> +                                          void *data)
> +{
> +    const struct fdt_property *prop;
> +    unsigned int i, banks;
> +    const __be32 *cell;
> +    u32 reg_cells = address_cells + size_cells;
> +    paddr_t start, size;
> +    struct meminfo *mem = data;
> +
> +    prop = fdt_get_property(fdt, node, prop_name, NULL);
> +    if ( !prop )
> +        return -ENOENT;
> +
> +    cell = (const __be32 *)prop->data;
> +    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
> +
> +    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
> +    {
> +        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
> +        /* Some DT may describe empty bank, ignore them */
> +        if ( !size )
> +            continue;
> +        mem->bank[mem->nr_banks].start = start;
> +        mem->bank[mem->nr_banks].size = size;
> +        mem->nr_banks++;
> +    }
> +
> +    if ( i < banks )
> +    {
> +        printk("Warning: Max number of supported memory regions reached.\n");
> +        return -ENOSPC;
> +    }
> +    return 0;
> +}
> +
>   u32 __init device_tree_get_u32(const void *fdt, int node,
>                                  const char *prop_name, u32 dflt)
>   {
> @@ -139,14 +177,6 @@ static int __init process_memory_node(const void *fdt, int node,
>                                         u32 address_cells, u32 size_cells,
>                                         void *data)
>   {
> -    const struct fdt_property *prop;
> -    int i;
> -    int banks;
> -    const __be32 *cell;
> -    paddr_t start, size;
> -    u32 reg_cells = address_cells + size_cells;
> -    struct meminfo *mem = data;
> -
>       if ( address_cells < 1 || size_cells < 1 )

This check will be the same for "xen,static-mem". So can it be moved to 
device_tree_get_meminfo()?

>       {
>           printk("fdt: node `%s': invalid #address-cells or #size-cells",
> @@ -154,27 +184,7 @@ static int __init process_memory_node(const void *fdt, int node,
>           return -EINVAL;
>       }
>   
> -    prop = fdt_get_property(fdt, node, "reg", NULL);
> -    if ( !prop )
> -        return -ENOENT;
> -
> -    cell = (const __be32 *)prop->data;
> -    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
> -
> -    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
> -    {
> -        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
> -        /* Some DT may describe empty bank, ignore them */
> -        if ( !size )
> -            continue;
> -        mem->bank[mem->nr_banks].start = start;
> -        mem->bank[mem->nr_banks].size = size;
> -        mem->nr_banks++;
> -    }
> -
> -    if ( i < banks )
> -        return -ENOSPC;
> -    return 0;
> +    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
>   }
>   
>   static int __init process_reserved_memory_node(const void *fdt, int node,
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:59:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:59:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176134.320592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM5k-0000Ps-Qg; Wed, 01 Sep 2021 08:59:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176134.320592; Wed, 01 Sep 2021 08:59:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM5k-0000PM-MP; Wed, 01 Sep 2021 08:59:32 +0000
Received: by outflank-mailman (input) for mailman id 176134;
 Wed, 01 Sep 2021 08:59:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qyqW=NX=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mLM5j-0000NG-21
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 08:59:31 +0000
Received: from mail-wm1-x32c.google.com (unknown [2a00:1450:4864:20::32c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7b077ebb-3606-4097-8e86-7a9547631e3e;
 Wed, 01 Sep 2021 08:59:29 +0000 (UTC)
Received: by mail-wm1-x32c.google.com with SMTP id
 o39-20020a05600c512700b002e74638b567so1387676wms.2
 for <xen-devel@lists.xen.org>; Wed, 01 Sep 2021 01:59:29 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id q11sm4990687wmc.41.2021.09.01.01.59.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 Sep 2021 01:59:27 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id E26A91FF96;
 Wed,  1 Sep 2021 09:59:26 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b077ebb-3606-4097-8e86-7a9547631e3e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=4vx7Zbm8CuixWNdewxmOaS8u+QseNO7OTa6aQNrm0kI=;
        b=Dn56lMmrX0vlpkgbNgHhqRiO345nhCfgtmp3bXAdFY+1uszjRzJj3wC+21HEB3EUDi
         CbcMUWUe3+SjAPdtvjLenZP/vubL3FhqjPm5Dvcs01tds9PIkDgTal7dX5qrLBPnfoPA
         bh45l88MiLik/bdk8Fk6pEI2Xn1GqsV8X88IFUxQiiumqrwhDwpAO9fuBmPKmejsy8UU
         3i4UdlvkfACw/gz3uEvM8Ipw+vYnbMMkM3IM/oCTYU3YTtOijHfCgJe0LvunsEVeUchE
         cE2yp4J+6JAg4F6pN4F7APMOL7eBwaV5A/58njHTalQSj5MFZEdP5revZmdYmHRouKhk
         MPLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=4vx7Zbm8CuixWNdewxmOaS8u+QseNO7OTa6aQNrm0kI=;
        b=pNsKGl4JniibPg/b/waWhwLB99M8HFtFiCk9Lc6MANvsn14QpZFhcvVXykbc82av1l
         LRrhjvkKEmiYBZAeUIu9UACeyOOvXhJOoSJ0xDFX4snL1wyY3p2fC1k1+oVUBAXNRf0K
         AWAuPQ98nIRJJMB8my19bdjH/dRfdTZqpRBzGx8yHF2xAVqRBnEkZ745e5HilnxJglX4
         KGbT7vX1lJraSb7tqUyc2MTpkw56B/DNjBWiSC/g5pQtAlmfFho3ss0t/NHA8CEdrHmF
         FOd1nwl5Cw4RdQ7nRnB9JOPsptN2z2WllZ2NN58AOKVrFAhogl3r3M0nRkeUDmvBsU9/
         Fwug==
X-Gm-Message-State: AOAM533DxL1lWXtu0vLFN0NXx/sBVwhMm8iwhpI17ry54C5iyazSdWes
	28iBpIvwGvUqdA6cuKsMXjbTFw==
X-Google-Smtp-Source: ABdhPJy1JDP7j3J1vzC7astXeHHHohtbMvmxOm+XGvKbN37GI/QmPrNdycwQml/nktVwrMmQ0N4VcA==
X-Received: by 2002:a1c:f314:: with SMTP id q20mr8677397wmq.154.1630486768626;
        Wed, 01 Sep 2021 01:59:28 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <0100017b33e585a5-06d4248e-b1a7-485e-800c-7ead89e5f916-000000@email.amazonses.com>
 <CAHFG_=WKjJ1riKtaWC8jm13shc3RtVsNNqd3j9WD9Fq0NeRS2Q@mail.gmail.com>
 <20210813051038.GA77540@laputa>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Fran??ois Ozog <francois.ozog@linaro.org>, Stefano Stabellini
 <sstabellini@kernel.org>, paul@xen.org, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Jan
 Kiszka <jan.kiszka@siemens.com>, Arnd Bergmann <arnd.bergmann@linaro.org>,
 jgross@suse.com, julien@xen.org, Carl van Schaik
 <cvanscha@qti.qualcomm.com>, Bertrand.Marquis@arm.com,
 stefanha@redhat.com, Artem_Mygaiev@epam.com, xen-devel@lists.xen.org,
 olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
 xen-devel@lists.xenproject.org
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
Date: Wed, 01 Sep 2021 09:57:45 +0100
In-reply-to: <20210813051038.GA77540@laputa>
Message-ID: <87eea8hdsh.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


AKASHI Takahiro <takahiro.akashi@linaro.org> writes:

> Hi Fran=C3=A7ois,
>
> On Thu, Aug 12, 2021 at 09:55:52AM +0200, Fran??ois Ozog wrote:
>> I top post as I find it difficult to identify where to make the comments.
>
> Thank you for the posting.=20
> I think that we should first discuss more about the goal/requirements/
> practical use cases for the framework.
>
>> 1) BE acceleration
>> Network and storage backends may actually be executed in SmartNICs. As
>> virtio 1.1 is hardware friendly, there may be SmartNICs with virtio 1.1 =
PCI
>> VFs. Is it a valid use case for the generic BE framework to be used in t=
his
>> context?
>> DPDK is used in some BE to significantly accelerate switching. DPDK is a=
lso
>> used sometimes in guests. In that case, there are no event injection but
>> just high performance memory scheme. Is this considered as a use case?
>
> I'm not quite familiar with DPDK but it seems to be heavily reliant
> on not only virtqueues but also kvm/linux features/functionality, say,
> according to [1].
> I'm afraid that DPDK is not suitable for primary (at least, initial)
> target use.
> # In my proposal, virtio-proxy, I have in mind the assumption that we wou=
ld
> # create BE VM as a baremetal application on RTOS (and/or unikernel.)
>
> But as far as virtqueue is concerned, I think we can discuss in general
> technical details as Alex suggested, including:
> - sharing or mapping memory regions for data payload
> - efficient notification mechanism
>
> [1] https://www.redhat.com/en/blog/journey-vhost-users-realm
>
>> 2) Virtio as OS HAL
>> Panasonic CTO has been calling for a virtio based HAL and based on the
>> teachings of Google GKI, an internal HAL seem inevitable in the long ter=
m.
>> Virtio is then a contender to Google promoted Android HAL. Could the
>> framework be used in that context?
>
> In this case, where will the implementation of "HAL" reside?
> I don't think the portability of "HAL" code (as a set of virtio BEs)
> is a requirement here.

When I hear people referring to VirtIO HALs I'm thinking mainly of
VirtIO FE's living in a Linux kernel. There are certainly more devices
that can get added but the commonality on the guest side I think is
pretty much a solved problem (modulo Linux-ism's creeping into the
virtio spec).

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 08:59:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 08:59:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176133.320586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM5k-0000Nc-IA; Wed, 01 Sep 2021 08:59:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176133.320586; Wed, 01 Sep 2021 08:59:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM5k-0000NV-FD; Wed, 01 Sep 2021 08:59:32 +0000
Received: by outflank-mailman (input) for mailman id 176133;
 Wed, 01 Sep 2021 08:59:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qyqW=NX=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mLM5i-0000NG-KF
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 08:59:30 +0000
Received: from mail-wm1-x32c.google.com (unknown [2a00:1450:4864:20::32c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4befbfe0-02cf-4f58-ae92-cc2125c322fc;
 Wed, 01 Sep 2021 08:59:29 +0000 (UTC)
Received: by mail-wm1-x32c.google.com with SMTP id
 u15-20020a05600c19cf00b002f6445b8f55so1012152wmq.0
 for <xen-devel@lists.xenproject.org>; Wed, 01 Sep 2021 01:59:29 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id q11sm4990687wmc.41.2021.09.01.01.59.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 Sep 2021 01:59:27 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id E26A91FF96;
 Wed,  1 Sep 2021 09:59:26 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4befbfe0-02cf-4f58-ae92-cc2125c322fc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=4vx7Zbm8CuixWNdewxmOaS8u+QseNO7OTa6aQNrm0kI=;
        b=Dn56lMmrX0vlpkgbNgHhqRiO345nhCfgtmp3bXAdFY+1uszjRzJj3wC+21HEB3EUDi
         CbcMUWUe3+SjAPdtvjLenZP/vubL3FhqjPm5Dvcs01tds9PIkDgTal7dX5qrLBPnfoPA
         bh45l88MiLik/bdk8Fk6pEI2Xn1GqsV8X88IFUxQiiumqrwhDwpAO9fuBmPKmejsy8UU
         3i4UdlvkfACw/gz3uEvM8Ipw+vYnbMMkM3IM/oCTYU3YTtOijHfCgJe0LvunsEVeUchE
         cE2yp4J+6JAg4F6pN4F7APMOL7eBwaV5A/58njHTalQSj5MFZEdP5revZmdYmHRouKhk
         MPLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=4vx7Zbm8CuixWNdewxmOaS8u+QseNO7OTa6aQNrm0kI=;
        b=NPbvgMkRZmSNcVhbh6p/H/Wq+LvA6KjOWSu+8UGtp0tQM0beWpslObKmYG83SqZm3n
         /vHB4ZGgn3LS4RKp5isa2eIhgpcvXlgvlWsJqfHP4ppSyIMGqRzieSSxKPdqrcpCcWAX
         B3k0Xr1vGnZZOMhlN1jG8txj2V8KWfgzrqmPYy4TR62mb8+fuebzoCFmZ+jNHnyai59H
         G+4isKQPFGBPjBm7jyxLNugJAAniu2u1Ey+v7nIxGpWQLUNy4VC40bylgYN7q6+Z/0Pa
         NsnwUhpNmjsg+kEbWFJgCnydeE0rECikRxdSnLoKLa3XzcCMDKdT/nhUsDz81w6X7SOT
         pcZw==
X-Gm-Message-State: AOAM530MIKCoA1GJm5Af54q6AsB/n2pluN9zXrm6Febakv7v0+n1C4Y2
	0YeYrmXgTEQHrw/Y31VzpAhuiw==
X-Google-Smtp-Source: ABdhPJy1JDP7j3J1vzC7astXeHHHohtbMvmxOm+XGvKbN37GI/QmPrNdycwQml/nktVwrMmQ0N4VcA==
X-Received: by 2002:a1c:f314:: with SMTP id q20mr8677397wmq.154.1630486768626;
        Wed, 01 Sep 2021 01:59:28 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <0100017b33e585a5-06d4248e-b1a7-485e-800c-7ead89e5f916-000000@email.amazonses.com>
 <CAHFG_=WKjJ1riKtaWC8jm13shc3RtVsNNqd3j9WD9Fq0NeRS2Q@mail.gmail.com>
 <20210813051038.GA77540@laputa>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Fran??ois Ozog <francois.ozog@linaro.org>, Stefano Stabellini
 <sstabellini@kernel.org>, paul@xen.org, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Jan
 Kiszka <jan.kiszka@siemens.com>, Arnd Bergmann <arnd.bergmann@linaro.org>,
 jgross@suse.com, julien@xen.org, Carl van Schaik
 <cvanscha@qti.qualcomm.com>, Bertrand.Marquis@arm.com,
 stefanha@redhat.com, Artem_Mygaiev@epam.com, xen-devel@lists.xen.org,
 olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
 xen-devel@lists.xenproject.org
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
Date: Wed, 01 Sep 2021 09:57:45 +0100
In-reply-to: <20210813051038.GA77540@laputa>
Message-ID: <87eea8hdsh.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


AKASHI Takahiro <takahiro.akashi@linaro.org> writes:

> Hi Fran=C3=A7ois,
>
> On Thu, Aug 12, 2021 at 09:55:52AM +0200, Fran??ois Ozog wrote:
>> I top post as I find it difficult to identify where to make the comments.
>
> Thank you for the posting.=20
> I think that we should first discuss more about the goal/requirements/
> practical use cases for the framework.
>
>> 1) BE acceleration
>> Network and storage backends may actually be executed in SmartNICs. As
>> virtio 1.1 is hardware friendly, there may be SmartNICs with virtio 1.1 =
PCI
>> VFs. Is it a valid use case for the generic BE framework to be used in t=
his
>> context?
>> DPDK is used in some BE to significantly accelerate switching. DPDK is a=
lso
>> used sometimes in guests. In that case, there are no event injection but
>> just high performance memory scheme. Is this considered as a use case?
>
> I'm not quite familiar with DPDK but it seems to be heavily reliant
> on not only virtqueues but also kvm/linux features/functionality, say,
> according to [1].
> I'm afraid that DPDK is not suitable for primary (at least, initial)
> target use.
> # In my proposal, virtio-proxy, I have in mind the assumption that we wou=
ld
> # create BE VM as a baremetal application on RTOS (and/or unikernel.)
>
> But as far as virtqueue is concerned, I think we can discuss in general
> technical details as Alex suggested, including:
> - sharing or mapping memory regions for data payload
> - efficient notification mechanism
>
> [1] https://www.redhat.com/en/blog/journey-vhost-users-realm
>
>> 2) Virtio as OS HAL
>> Panasonic CTO has been calling for a virtio based HAL and based on the
>> teachings of Google GKI, an internal HAL seem inevitable in the long ter=
m.
>> Virtio is then a contender to Google promoted Android HAL. Could the
>> framework be used in that context?
>
> In this case, where will the implementation of "HAL" reside?
> I don't think the portability of "HAL" code (as a set of virtio BEs)
> is a requirement here.

When I hear people referring to VirtIO HALs I'm thinking mainly of
VirtIO FE's living in a Linux kernel. There are certainly more devices
that can get added but the commonality on the guest side I think is
pretty much a solved problem (modulo Linux-ism's creeping into the
virtio spec).

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:00:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:00:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176146.320609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM6M-0002Hi-6L; Wed, 01 Sep 2021 09:00:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176146.320609; Wed, 01 Sep 2021 09:00:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLM6M-0002Hb-2y; Wed, 01 Sep 2021 09:00:10 +0000
Received: by outflank-mailman (input) for mailman id 176146;
 Wed, 01 Sep 2021 09:00:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLM6K-0001Ea-KM
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 09:00:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a8bd7362-c5c4-44d9-b891-ff5aaeec671b;
 Wed, 01 Sep 2021 09:00:03 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-kl2wIvanN7-Qe8fHEPjJFw-1;
 Wed, 01 Sep 2021 11:00:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5166.eurprd04.prod.outlook.com (2603:10a6:803:53::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 08:59:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 08:59:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0011.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Wed, 1 Sep 2021 08:59:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8bd7362-c5c4-44d9-b891-ff5aaeec671b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630486802;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Hcz99EKLAguPUtj+MYXe/atzn888fi5RnKG/mjRKRuY=;
	b=XjtR/Kd4JgoOXi/Sz5c/oMKlQgNd/0Pzxdugf9rsQ3gjE7jZDFZdGQ7cPqmh4WIsbLsqbN
	Zwq44B8rXV4oXbUsOImcGKicpd9RA2LR+r2AHDNWnQkKPmRqk7COOJxKqLZp3S4AeMKxtb
	1gTsCOJ4DrQieH2nMz8p3DqSf18bWVA=
X-MC-Unique: kl2wIvanN7-Qe8fHEPjJFw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cPjrxb9HzE2Dsm5BmMFpNIRmxHkHsXYXlVeL8Snxo2jpbBMsSnMJdacIBEXROAcR/mjvYb2KPW1En7spXwuVOIm367uFo80JnPbvpOrZcMyTTSnXwhfFaTTSGTT2bZiUsj39uLacoFbAKPUk4yABl87hSxsE3u8S/NeyOw7dctrcCTlmMhmWa51d5ZaLgnTeQ1m72XAaT1Vyg1ffd8Ad+4KxL2hWiyLlcjP0lNVz+TwQ+EZvb6+T+Eka4govxznIudsMfu5lyODm2Tx9fjr3NueCfx8n0U3eow3HVW0EtBlTta4Xg42rAD7ABBXsN5B7bg6FQRkD25bSP1FKaqpwiw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Hcz99EKLAguPUtj+MYXe/atzn888fi5RnKG/mjRKRuY=;
 b=WD1fgt6DL936HyV3rO/vpO/9YRxGSGe4ACQaJm0qtQK3x/XtlR9a0fLl6dRMwC7MhU3BFKdcdcdCbOcrAtH+jLjDpLTc5JbUHKtX1Qn07eGXDY0FbOC7A7rL0+SJUoEVsJNmNzGTCPPrgDHQehYDGTE3MswuMKUc45gtCd1YTZn492moxHz2TNEtuKV7wBVcGeGvCBydI1MlfL+0RI2NHprsAKjY/S/EGK4jpvhnlHX7i9BeR0CZEmp/H6Mngtj92Q6bpIT3OZJnCqatg0+SueckndkAStzgMVkz62qoGnPu9krNhcv3/Ee5/eIgn5z6Atpz1RjVZ974mrs7IOe/JA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] Config: use Mini-OS master for xen-unstable
To: Juergen Gross <jgross@suse.com>, Wei Liu <wl@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <20210831073039.7101-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
Date: Wed, 1 Sep 2021 10:59:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210831073039.7101-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0011.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::23)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 52089834-26b8-43ea-c279-08d96d26e0bf
X-MS-TrafficTypeDiagnostic: VI1PR04MB5166:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5166821E4151009657B70B60B3CD9@VI1PR04MB5166.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	n5aI0wSi3lX2/9ONxciXPCNdhDDz5hbNZQtf6UjymPToUHsD0WtB1sZdN04jintrqhZUdTYrEz7+4ptFmVF9razZkyLRW/ed+GoJJAOTfdU44YtmWuSblsdCSIygKvDo9tnHVzZboKdXwdb2BVXXZqDveX5PfxFHOgTbGi5zxTXZcTLqjqXCWr3zOBfhXjG5FO2fnHfG/W84njpdX1esXurnl7/g7y6Iu07e2pfVcXpPWDzuv78yoCstVv2qmCiyB7e9P90JL8XMV7UtwI5q4eI84vlC2apjddXDQuG1LlDHu4Mxgp66ZlT8MWLWgk4ixdRhHZ5zEMp2taGOjJksD4SSVGeITYbpZtwcLRdxhfYv8iAj9As6URERpNvn0mylkL3ftCH8NrPMIpcSNS90ZvVhoKQKCNbwKyz+0ZA9xZzi/vl7RBvVUjWdO+Wua+9wgrkQZ5hYRuVL96muZuf2HtdYChph7gRHURHDQxE0bFq4+DUkpSt+VJH2SQjj03xWhvsGsnOwTO9+2K4iTaY5K4/qg9fBF+3rerchw6rxRMhvt7J+d8sQ3WxqCm4PHQsq2Ed01tsgRFbSeZTTgUxDZpnxnonlkxsML9vwfX5g3jSWdUi3mGkI7H7Te2bFYJ1KzEWTqjrR07G0qdU0Qruz/GDh2BzJTtmf9zX0bVYlREYnEBe17xM6MZNCcOlCVINfAPNIQmnVCjsUaQLMnl85MfRJOpVe9X3sqLHSNQAVYhQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(376002)(366004)(136003)(39860400002)(5660300002)(66946007)(54906003)(86362001)(2616005)(8936002)(66476007)(38100700002)(16576012)(31696002)(26005)(36756003)(4326008)(66556008)(53546011)(186003)(6486002)(956004)(31686004)(478600001)(83380400001)(2906002)(110136005)(316002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDB2em12OFFpdVNWYkVLek55MEFUWXBiY0phaHVOSUpjQXY0aEFJbXZuR3o3?=
 =?utf-8?B?dkptb2t5TDAzM2V4em9rWkQwY1dxN3FwZGVnNGV2VXZ4MFVxcDI3TG9NWmxj?=
 =?utf-8?B?eU1Ka2ZFcFZPMzlhUDJjSENDT09FQWRpNzRyR2E2MlVBWlk1cENBOEdyVW56?=
 =?utf-8?B?MkpkVzlkSE9ndHArQU45aDg5bmkwNkZvQmV6bXYyVWtaSDNDTEo0WDY2Ylln?=
 =?utf-8?B?UlE1dVpkSmFuejBDMytTK0pKclRwQmQ3R3V1MFF0U1ZCYWNHbVlLNGdNZDZJ?=
 =?utf-8?B?RFZ0T21KVUlhMjdRMER2cEVPUEN1a2JycWthMTY2bFdmRmhKWmJ3TG1MZkF2?=
 =?utf-8?B?WFIxaFhhWDU4cFVnUkFKaSt3dlZhUTVTVWFqR2ZLS1VxTFdSMkRzTjc2cHQ4?=
 =?utf-8?B?QnlBbHM2dkhVZHJIS3JhSzJ3MDBpVFZ2YzFETzFUZEwvQUJwaUZPMjlVaVo5?=
 =?utf-8?B?bFFSTGxMcWV4OGUva3R0Q0czVThSWkQ3QllWcVp5YVZJQmM5cUZiY3VzRVdq?=
 =?utf-8?B?dG9FeGNBMFJjY0FMTVhWVVpNaFNIYmlrc2NJZXNKRVNsQjJac2wzMTdDTnZV?=
 =?utf-8?B?Y1IreFpqcDU1UG5DR1FKTWZnZnZQQ2UyY1hTOXdwNlU4S0dXVXNyYTQ5TE44?=
 =?utf-8?B?N013OUwrUkN0eVNpVU1KNWsvVXRLYVYxY1VzcEhCNWQ2eHdTSGtIZHVTZk5N?=
 =?utf-8?B?S3hCcVpnUHFSNG9nY1VsOUx3ajlsS3l2WTRkQS9UUks1WG02ak1BODdrT01m?=
 =?utf-8?B?blFhQ3czRzhKdG1mK29LWWo4R2ZhdDhabmd6SW9HQlY3UHJvNjZNSU1PYy9S?=
 =?utf-8?B?UUU1cGdwZUU5MHJZYXJ2djJZdFIwZm5NWmV0ZGliL0hBZC9SangvTTJ4UU93?=
 =?utf-8?B?Q25FY29DMW1CV1FmVkZLeUozWlhYYjc3bnlhTlpkRGRKc0tGVy9WT3F6ekRC?=
 =?utf-8?B?T3p1Z1cvWGJEV1NRVEh6Zlg1VndLRXdFQTdOc0o1bVo0YXQ1Zm42THo1aUda?=
 =?utf-8?B?UTRqZW1uOTNLVGM5azJta3lTRTV5SkpoYm9iVnpqMUhETnhpYmErZVBFemFt?=
 =?utf-8?B?ZDZ4MmI5TTUxTHQ5NXFWcG9mVHdHYlo3aGcxVG54ZVppcEU4b00zSFRjOTVj?=
 =?utf-8?B?ejJoN0xOTTAwcVJoVG1iYTJLVG9xajRXWnFLQ1M3TWFwY3lXVkNaVWRnN3Ns?=
 =?utf-8?B?OFZkQW82ME5IL0lOUWVRVDFZTmdVUkI5bDUxMFBiL29ERW1kKzdkWUVvbXkw?=
 =?utf-8?B?aHl1UFFVK08yQTY0Z3duYzBmS2p0d1V5L2ZIZERwb29Ib3VsSG8yVjZoY05R?=
 =?utf-8?B?bVdrWm9tRXQzTmt1RUZxbUVwdWM4MkJsQm9qQVpGZTUyd3dSNnRFbk5NTHpa?=
 =?utf-8?B?OTlYZUpqb2h1d2VER2crTnorSFNGQzNYUnJzMXZ2RTdiQ0xMWjhRTVI4WVdB?=
 =?utf-8?B?TFo4WFpOOFZZbDhwdngwQisvakliK2ZoWm9iQzNnbEhsaDFDclR1MEE5TjAy?=
 =?utf-8?B?eWpXbEk4ektwd3ByWVBzdWdsRFpWbHdodVlXdWI4eXkxL1VYOElOK3pSZFBU?=
 =?utf-8?B?Y0pReDRXYi9DVTdzUFdvcHQvVXVEaVFjSWtzSUVTWmM1cnpPcHI2OUNYK0pV?=
 =?utf-8?B?NkhnT3BnQ0N5dVZ6OXZYYzQ5UDF0aktGOTFuN0hmQUlrTi94cmVTN3RkRlZp?=
 =?utf-8?B?TUp4SFRkS01Ydm82WnpaT0xHLzNScWhYVjNqdVR3QjgxMFJ2UHpRMnNRUVdC?=
 =?utf-8?Q?GS56lvII8Nm67IEzk+z1tIge/l3hgCgj4CLZdLf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 52089834-26b8-43ea-c279-08d96d26e0bf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 08:59:59.7743
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HtS+A2rPT0pqBFluKaKUyD3nK26IxnMeQvVNQCjfY7O8h0+b6Qtv6eQXss7wlV672M7qpihj/5sGpTxJPVsHhQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5166

On 31.08.2021 09:30, Juergen Gross wrote:
> In order to get Mini-OS master branch tested by OSStest, use it for
> xen-unstable.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> There are some Mini-OS patches pending, of which at least the Mini-OS
> netfront fix should be committed as it will be required for qemu-stubdom
> to work.
> ---
>  Config.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Config.mk b/Config.mk
> index 4d723eec1d..d4e8ced104 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?= git://xenbits.xen.org/mini-os.git
>  endif
>  OVMF_UPSTREAM_REVISION ?= b37cfdd2807181aed2fee1e17bd7ec1190db266a
>  QEMU_UPSTREAM_REVISION ?= master
> -MINIOS_UPSTREAM_REVISION ?= 051b87bb9c19609976fb038f386920e1ce5454c5
> +MINIOS_UPSTREAM_REVISION ?= master

I'm afraid this, effectively reverting 82c3d15c903a ("minios: Revert recent
change and revert to working minios"), would set us up for the same problem
again that was dealt with there. IOW you're re-proposing a change of yours
(e013e8514389 ["config: use mini-os master for unstable"]) which was
subsequently reverted, without spelling out what has since changed.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:31:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:31:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176160.320644 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMaN-0006eR-Vg; Wed, 01 Sep 2021 09:31:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176160.320644; Wed, 01 Sep 2021 09:31:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMaN-0006eK-QF; Wed, 01 Sep 2021 09:31:11 +0000
Received: by outflank-mailman (input) for mailman id 176160;
 Wed, 01 Sep 2021 09:31:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tzcz=NX=xenbits.xen.org=iwj@srs-us1.protection.inumbo.net>)
 id 1mLMaM-000673-Ho
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 09:31:10 +0000
Received: from mail.xenproject.org (unknown [104.130.215.37])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c4b943f9-c6d1-4fca-8114-98ddd83101fd;
 Wed, 01 Sep 2021 09:30:59 +0000 (UTC)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mLMa2-0006zy-LC; Wed, 01 Sep 2021 09:30:50 +0000
Received: from iwj by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mLMa2-0006VB-JU; Wed, 01 Sep 2021 09:30:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c4b943f9-c6d1-4fca-8114-98ddd83101fd
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=0ePqXSIKBX48B2cK5m/t9DhA8yei6wa4P1bd3JfsXGo=; b=kYO0tI9pMZm8aDxEs1YWieGE2x
	IRgOiwDuDSuka3bNS2aX7mgC2LjKHfzVooAmUJCF8c0RTL+3zRsGfZUYv+/EUZPNyGcQQvGkuoJWK
	Vnqo0rBsKy4nyKJpJYeXw/SlCjvsnzuZmxb38YS4TVuEw8rbWwPjERCs+JkHvInS2pW8=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 380 v3 (CVE-2021-28698) - long running
 loops in grant table handling
Message-Id: <E1mLMa2-0006VB-JU@xenbits.xenproject.org>
Date: Wed, 01 Sep 2021 09:30:50 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

            Xen Security Advisory CVE-2021-28698 / XSA-380
                               version 3

              long running loops in grant table handling

UPDATES IN VERSION 3
====================

New bugfix patch on top of the prior set.

ISSUE DESCRIPTION
=================

In order to properly monitor resource use, Xen maintains information on
the grant mappings a domain may create to map grants offered by other
domains.  In the process of carrying out certain actions, Xen would
iterate over all such entries, including ones which aren't in use
anymore and some which may have been created but never used.  If the
number of entries for a given domain is large enough, this iterating of
the entire table may tie up a CPU for too long, starving other domains
or causing issues in the hypervisor itself.

Note that a domain may map its own grants, i.e. there is no need for
multiple domains to be involved here.  A pair of "cooperating" guests
may, however, cause the effects to be more severe.

IMPACT
======

Malicious or buggy guest kernels may be able to mount a Denial of
Service (DoS) attack affecting the entire system.

VULNERABLE SYSTEMS
==================

All Xen versions from at least 3.2 onwards are vulnerable in principle.

Systems running with default settings are generally believed to not be
vulnerable.

MITIGATION
==========

The problem can be avoided by reducing the number of grant mappings Xen
would allow guests to establish.  For example, setting
"gnttab_max_maptrack_frames=256" on the Xen command line (available
from Xen 4.5 onwards) or "max_maptrack_frames=256" in the xl domain
configurations (available from Xen 4.10 onwards) may be low enough for
all hardware Xen is able to run on.  Note that except for driver
domains, guests don't normally have a need to establish grant mappings,
i.e. they may be okay to run with "max_maptrack_frames=0" in their xl
domain configurations.

- From Xen 4.14 onwards it is also possible to alter the system wide upper
bound of the number of grant mappings Xen would allow guests to
establish by writing to the /params/gnttab_max_maptrack_frames
hypervisor file system node.  Note however that changing the value this
way will only affect guests yet to be created on the respective host.

CREDITS
=======

This issue was discovered by Andrew Cooper of Citrix.

RESOLUTION
==========

Applying the appropriate pair of attached patches resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

To address an issue observed with some compiler versions, another patch
needs to be applied on top. This has been committed to the unstable
tree (commit hash b6da9d0414d69c2682214ee3ecf9816fcac500d0) only so far.
This patch is listed last below.

xsa380/xsa380-[12].patch        xen-unstable - Xen 4.15.x
xsa380/xsa380-4.14-?.patch      Xen 4.14.x
xsa380/xsa380-4.13-?.patch      Xen 4.13.x
xsa380/xsa380-4.12-?.patch      Xen 4.12.x
xsa380/xsa380-4.11-?.patch      Xen 4.11.x
xsa380/xsa380-3.patch           Xen 4.15.x - Xen 4.11.x

$ sha256sum xsa380* xsa380*/*
3b1938277665c195f6822a1170c50a853efa6bc3dcd6b2b0b9bbb0849a57bbf6  xsa380.meta
65411a0fd05d534ed2238b259aa2877331ac0e79f2dda80b424f34fffcce108a  xsa380/xsa380-1.patch
e6cd6d345abaad38e10d6f680fe881e874e35a7295199e5430bacd209f0b7304  xsa380/xsa380-2.patch
1ead53a28dedb0a502a700d9ea933e89e385c1582f4790f558a11d0d39e9d374  xsa380/xsa380-3.patch
f3b486aa99a75ab54f9e26e21a721a413f993b27dbc3e6f2fda976fe20ddbae3  xsa380/xsa380-4.11-1.patch
96a09c05ca87fe3590064ca6d269ca47b97c732401cb593ff1068ac91009f51a  xsa380/xsa380-4.11-2.patch
496063cd4641258e1854a77f626cdd86c866c3ed8603bdc2ff9ab709008c84a7  xsa380/xsa380-4.12-1.patch
d850e1263e89c7a718f2cddcfb639fe4a5095a1852fc35499ed16a4075d225e5  xsa380/xsa380-4.12-2.patch
c23d34527a2ec68015ad78cd90365e4d80bce842ce01eeaa8cd2246021a55693  xsa380/xsa380-4.13-1.patch
bd40ce749d02f343c79325488ac1348e1c9e88e698bbad351bdb0a0d3995f3e0  xsa380/xsa380-4.13-2.patch
9b06085f9a4b93c465563cd76fdc682ddb9dba968c214259425b234e7053a809  xsa380/xsa380-4.14-1.patch
bf6b53880abd53b56226080d1a32839c7c8c459867104697cd76eeafc2ea382a  xsa380/xsa380-4.14-2.patch
$

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.  HOWEVER, care has to be taken to avoid restricting
guests too much, as them suddenly being unable to map grants they used
to be able to map may lead to re-discovery of the issue.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmEvSDUMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZfboH/jtTLKbkb/K4SPliIDSRym1+wx17Ho30F9IRfd7E
ZdUBHfFLTEbZ/GvU+UkU914XeK49fCHZGylu2dcIRWgPmXFlqOtydFJny9a8sCjx
7/dLR1WXvhJ4eG8Hsma13uRzZuaF4JdXDp1M/QOr6aBHNTLBgMefbQNTAbFO0kzP
2xK3arGefi/eNa3GOEghrMlt2U/5r1h7PfLXBl69BETomOCOo5gyn6aQwqH/xKSY
W2cmqN0hYLyQSA+jx6QuTknsvSLS6NWWPEk/xfs31Op0E/P3tga0u6hCbRvWzOmE
AdISyyP8ogIBU4ekMn0H4yQoWKVgYctk3XAawBmogv+VJXI=
=7wyB
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa380.meta"
Content-Disposition: attachment; filename="xsa380.meta"
Content-Transfer-Encoding: base64

ewogICJYU0EiOiAzODAsCiAgIlN1cHBvcnRlZFZlcnNpb25zIjogWwogICAg
Im1hc3RlciIsCiAgICAiNC4xNSIsCiAgICAiNC4xNCIsCiAgICAiNC4xMyIs
CiAgICAiNC4xMiIsCiAgICAiNC4xMSIKICBdLAogICJUcmVlcyI6IFsKICAg
ICJ4ZW4iCiAgXSwKICAiUmVjaXBlcyI6IHsKICAgICI0LjExIjogewogICAg
ICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVuIjogewogICAgICAgICAgIlN0
YWJsZVJlZiI6ICJlZjMyYzdhZmEyNzMxYjc1ODIyNmQ2ZTEwYTFlNDg5YjFh
MTVmYzQxIiwKICAgICAgICAgICJQcmVyZXFzIjogWwogICAgICAgICAgICAz
NzgsCiAgICAgICAgICAgIDM3OQogICAgICAgICAgXSwKICAgICAgICAgICJQ
YXRjaGVzIjogWwogICAgICAgICAgICAieHNhMzgwL3hzYTM4MC00LjExLT8u
cGF0Y2giCiAgICAgICAgICBdCiAgICAgICAgfQogICAgICB9CiAgICB9LAog
ICAgIjQuMTIiOiB7CiAgICAgICJSZWNpcGVzIjogewogICAgICAgICJ4ZW4i
OiB7CiAgICAgICAgICAiU3RhYmxlUmVmIjogImVhMjBlZWU5N2U5ZTA4NjEx
MjdhODA3MGNjN2I5YWUzNTU3YjA5ZmIiLAogICAgICAgICAgIlByZXJlcXMi
OiBbCiAgICAgICAgICAgIDM3OCwKICAgICAgICAgICAgMzc5CiAgICAgICAg
ICBdLAogICAgICAgICAgIlBhdGNoZXMiOiBbCiAgICAgICAgICAgICJ4c2Ez
ODAveHNhMzgwLTQuMTItPy5wYXRjaCIKICAgICAgICAgIF0KICAgICAgICB9
CiAgICAgIH0KICAgIH0sCiAgICAiNC4xMyI6IHsKICAgICAgIlJlY2lwZXMi
OiB7CiAgICAgICAgInhlbiI6IHsKICAgICAgICAgICJTdGFibGVSZWYiOiAi
MzJkNTgwOTAyYjk1OTAwMGQ3OWQ1MWRmZjAzYTM1NjA2NTNjNGZjYiIsCiAg
ICAgICAgICAiUHJlcmVxcyI6IFsKICAgICAgICAgICAgMzc4LAogICAgICAg
ICAgICAzNzkKICAgICAgICAgIF0sCiAgICAgICAgICAiUGF0Y2hlcyI6IFsK
ICAgICAgICAgICAgInhzYTM4MC94c2EzODAtNC4xMy0/LnBhdGNoIgogICAg
ICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwKICAgICI0LjE0Ijog
ewogICAgICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVuIjogewogICAgICAg
ICAgIlN0YWJsZVJlZiI6ICI0OTI5OWM0ODEzYjc4NDdkMjlkZjA3YmY3OTBm
NTQ4OTA2MGYyYTljIiwKICAgICAgICAgICJQcmVyZXFzIjogWwogICAgICAg
ICAgICAzNzgsCiAgICAgICAgICAgIDM3OQogICAgICAgICAgXSwKICAgICAg
ICAgICJQYXRjaGVzIjogWwogICAgICAgICAgICAieHNhMzgwL3hzYTM4MC00
LjE0LT8ucGF0Y2giCiAgICAgICAgICBdCiAgICAgICAgfQogICAgICB9CiAg
ICB9LAogICAgIjQuMTUiOiB7CiAgICAgICJSZWNpcGVzIjogewogICAgICAg
ICJ4ZW4iOiB7CiAgICAgICAgICAiU3RhYmxlUmVmIjogImRiYTc3NDg5NmY3
ZGQ3NDc3M2MxNGQ1Mzc2NDNiN2Q3NDc3ZmVmY2QiLAogICAgICAgICAgIlBy
ZXJlcXMiOiBbCiAgICAgICAgICAgIDM3OCwKICAgICAgICAgICAgMzc5CiAg
ICAgICAgICBdLAogICAgICAgICAgIlBhdGNoZXMiOiBbCiAgICAgICAgICAg
ICJ4c2EzODAveHNhMzgwLT8ucGF0Y2giCiAgICAgICAgICBdCiAgICAgICAg
fQogICAgICB9CiAgICB9LAogICAgIm1hc3RlciI6IHsKICAgICAgIlJlY2lw
ZXMiOiB7CiAgICAgICAgInhlbiI6IHsKICAgICAgICAgICJTdGFibGVSZWYi
OiAiMjVkYTk0NTVmMWJiOGE2ZDMzMDM5NTc1YTdiMjhiZGZjNGUzZmNmZSIs
CiAgICAgICAgICAiUHJlcmVxcyI6IFsKICAgICAgICAgICAgMzc4LAogICAg
ICAgICAgICAzNzkKICAgICAgICAgIF0sCiAgICAgICAgICAiUGF0Y2hlcyI6
IFsKICAgICAgICAgICAgInhzYTM4MC94c2EzODAtPy5wYXRjaCIKICAgICAg
ICAgIF0KICAgICAgICB9CiAgICAgIH0KICAgIH0KICB9Cn0=

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-1.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGFkZCBwcmVlbXB0aW9uIGNoZWNrIHRvIGdudHRhYl9yZWxl
YXNlX21hcHBpbmdzKCkKCkEgZ3Vlc3QgbWF5IGRpZSB3aXRoIG1hbnkgZ3Jh
bnQgbWFwcGluZ3Mgc3RpbGwgaW4gcGxhY2UsIG9yIHNpbXBseSB3aXRoCmEg
bGFyZ2UgbWFwdHJhY2sgdGFibGUuIEl0ZXJhdGluZyB0aHJvdWdoIHRoaXMg
bWF5IHRha2UgbW9yZSB0aW1lIHRoYW4KaXMgcmVhc29uYWJsZSB3aXRob3V0
IGludGVybWVkaWF0ZSBwcmVlbXB0aW9uICh0byBydW4gc29mdGlycXMgYW5k
CnBlcmhhcHMgdGhlIHNjaGVkdWxlcikuCgpNb3ZlIHRoZSBpbnZvY2F0aW9u
IG9mIHRoZSBmdW5jdGlvbiB0byB0aGUgc2VjdGlvbiB3aGVyZSBvdGhlcgpy
ZXN0YXJ0YWJsZSBmdW5jdGlvbnMgZ2V0IGludm9rZWQsIGFuZCBoYXZlIHRo
ZSBmdW5jdGlvbiBpdHNlbGYgY2hlY2sKZm9yIHByZWVtcHRpb24gZXZlcnkg
b25jZSBpbiBhIHdoaWxlLiBIYXZlIGl0IGl0ZXJhdGUgdGhlIHRhYmxlCmJh
Y2t3YXJkcywgc3VjaCB0aGF0IGRlY3JlYXNpbmcgdGhlIG1hcHRyYWNrIGxp
bWl0IGlzIGFsbCBpdCB0YWtlcyB0bwpjb252ZXkgcmVzdGFydCBpbmZvcm1h
dGlvbi4KCkluIGRvbWFpbl90ZWFyZG93bigpIGludHJvZHVjZSBQUk9HX25v
bmUgc3VjaCB0aGF0IGluc2VydGluZyBhdCB0aGUKZnJvbnQgd2lsbCBiZSBl
YXNpZXIgZ29pbmcgZm9yd2FyZC4KClRoaXMgaXMgcGFydCBvZiBDVkUtMjAy
MS0yODY5OCAvIFhTQS0zODAuCgpSZXBvcnRlZC1ieTogQW5kcmV3IENvb3Bl
ciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
SnVsaWVuIEdyYWxsIDxqZ3JhbGxAYW1hem9uLmNvbT4KLS0tCldoaWxlIEkg
Y29uc2lkZXIgcmVtb3ZhbCBvZiB0aGUgZnJlZWluZyBvZiB0LT5tYXB0cmFj
a1tpXSBmcm9tCmdyYW50X3RhYmxlX2Rlc3Ryb3koKSBhbiBpbnRlZ3JhbCBw
YXJ0IG9mIHRoaXMgY2hhbmdlLCBhbHNvIGZyZWVpbmcKdC0+bWFwdHJhY2sg
cmlnaHQgaW4gZ250dGFiX3JlbGVhc2VfbWFwcGluZ3MoKSB3b3VsZCBzZWVt
IGxpa2UgYW4KdW5yZWxhdGVkIGNoYW5nZSB0byBtZSwgc28gSSdtIG5vdCBt
b3ZpbmcgdGhhdCBvbmUgZm9yIG5vdy4gSWYgb3RoZXJzCnRoaW5rIGl0IHdv
dWxkIGJldHRlciBiZSBtb3ZlZCwgSSdkIGJlIGhhcHB5IHRvIGRvIHNvLgoK
V2hpbGUgaW4gcHJpbmNpcGxlIGl0IHdvdWxkIGJlIG5pY2UgdG8gYWxzbyBl
bGltaW5hdGUgdGhlIG90aGVyIGxvb3BzCmZyb20gZ3JhbnRfdGFibGVfZGVz
dHJveSgpICh3aGljaCBjYW4gYWxsIHRha2UgbG9uZyBhcyB3ZWxsIHByb3Zp
ZGVkIGEKbGFyZ2UgZW5vdWdoIG1heF9ncmFudF9mcmFtZXMpLCAtPm1hcHRy
YWNrW10gcmVhbGx5IGlzIHNwZWNpYWwgaW4gdGhhdAppdCBvbmx5IGdldHMg
YWNjZXNzZWQgd2hlbiBwcm9jZXNzaW5nIHJlcXVlc3RzIGJ5IHRoZSBkb21h
aW4gaXRzZWxmLiBUaGUKb3RoZXIgYXJyYXlzIG1heSBhbGwgY29udGludWUg
dG8gYmUgYWNjZXNzZWQgYXMgcmVtb3RlIGRvbWFpbnMgZHJvcCB1c2VzCm9m
IGdyYW50cyBieSB0aGUgZHlpbmcgZG9tYWluLgotLS0KdjM6IEFkZCBjb21t
ZW50Lgp2MjogTW92ZSBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncygpIGludm9j
YXRpb24gaW50byBkb21haW5fdGVhcmRvd24oKS4KICAgIERvbid0IGNyYXNo
IHdoZW4gY2xlYW5pbmcgdXAgZG9tYWluIHdpdGhvdXQgbWFwdHJhY2sgdGFi
bGUuIEV4dGVuZAogICAgY29tbWVudCBuZXh0IHRvIG1hcHRyYWNrX2xpbWl0
LgoKLS0tIGEveGVuL2NvbW1vbi9kb21haW4uYworKysgYi94ZW4vY29tbW9u
L2RvbWFpbi5jCkBAIC00MTIsMTEgKzQxMiwxOCBAQCBzdGF0aWMgaW50IGRv
bWFpbl90ZWFyZG93bihzdHJ1Y3QgZG9tYWluCiAgICAgICAgIHYgPSBkLT50
ZWFyZG93bi52Y3B1CiAKICAgICAgICAgZW51bSB7Ci0gICAgICAgICAgICBQ
Uk9HX3ZjcHVfdGVhcmRvd24gPSAxLAorICAgICAgICAgICAgUFJPR19ub25l
LAorICAgICAgICAgICAgUFJPR19nbnR0YWJfbWFwcGluZ3MsCisgICAgICAg
ICAgICBQUk9HX3ZjcHVfdGVhcmRvd24sCiAgICAgICAgICAgICBQUk9HX2Rv
bmUsCiAgICAgICAgIH07CiAKLSAgICBjYXNlIDA6CisgICAgY2FzZSBQUk9H
X25vbmU6CisgICAgICAgIHJjID0gZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Mo
ZCk7CisgICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAgcmV0dXJuIHJj
OworCisgICAgUFJPR1JFU1MoZ250dGFiX21hcHBpbmdzKToKICAgICAgICAg
Zm9yX2VhY2hfdmNwdSAoIGQsIHYgKQogICAgICAgICB7CiAgICAgICAgICAg
ICBQUk9HUkVTU19WQ1BVKHRlYXJkb3duKTsKQEAgLTkwOCw3ICs5MTUsNiBA
QCBpbnQgZG9tYWluX2tpbGwoc3RydWN0IGRvbWFpbiAqZCkKICAgICAgICAg
ICAgIHJldHVybiBkb21haW5fa2lsbChkKTsKICAgICAgICAgZC0+aXNfZHlp
bmcgPSBET01EWUlOR19keWluZzsKICAgICAgICAgYXJnb19kZXN0cm95KGQp
OwotICAgICAgICBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhkKTsKICAgICAg
ICAgdm51bWFfZGVzdHJveShkLT52bnVtYSk7CiAgICAgICAgIGRvbWFpbl9z
ZXRfb3V0c3RhbmRpbmdfcGFnZXMoZCwgMCk7CiAgICAgICAgIC8qIGZhbGx0
aHJvdWdoICovCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRfdGFibGUuYworKysg
Yi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTY0LDcgKzY0LDEzIEBA
IHN0cnVjdCBncmFudF90YWJsZSB7CiAgICAgdW5zaWduZWQgaW50ICAgICAg
ICAgIG5yX2dyYW50X2ZyYW1lczsKICAgICAvKiBOdW1iZXIgb2YgZ3JhbnQg
c3RhdHVzIGZyYW1lcyBzaGFyZWQgd2l0aCBndWVzdCAoZm9yIHZlcnNpb24g
MikgKi8KICAgICB1bnNpZ25lZCBpbnQgICAgICAgICAgbnJfc3RhdHVzX2Zy
YW1lczsKLSAgICAvKiBOdW1iZXIgb2YgYXZhaWxhYmxlIG1hcHRyYWNrIGVu
dHJpZXMuICovCisgICAgLyoKKyAgICAgKiBOdW1iZXIgb2YgYXZhaWxhYmxl
IG1hcHRyYWNrIGVudHJpZXMuICBGb3IgY2xlYW51cCBwdXJwb3NlcyBpdCBp
cworICAgICAqIGltcG9ydGFudCB0byByZWFsaXplIHRoYXQgdGhpcyBmaWVs
ZCBhbmQgQG1hcHRyYWNrIGZ1cnRoZXIgZG93biB3aWxsCisgICAgICogb25s
eSBldmVyIGJlIGFjY2Vzc2VkIGJ5IHRoZSBsb2NhbCBkb21haW4uICBUaHVz
IGl0IGlzIG9rYXkgdG8gY2xlYW4KKyAgICAgKiB1cCBlYXJseSwgYW5kIHRv
IHNocmluayB0aGUgbGltaXQgZm9yIHRoZSBwdXJwb3NlIG9mIHRyYWNraW5n
IGNsZWFudXAKKyAgICAgKiBwcm9ncmVzcy4KKyAgICAgKi8KICAgICB1bnNp
Z25lZCBpbnQgICAgICAgICAgbWFwdHJhY2tfbGltaXQ7CiAgICAgLyogU2hh
cmVkIGdyYW50IHRhYmxlIChzZWUgaW5jbHVkZS9wdWJsaWMvZ3JhbnRfdGFi
bGUuaCkuICovCiAgICAgdW5pb24gewpAQCAtMzY3OSw5ICszNjg1LDcgQEAg
ZG9fZ3JhbnRfdGFibGVfb3AoCiAjaW5jbHVkZSAiY29tcGF0L2dyYW50X3Rh
YmxlLmMiCiAjZW5kaWYKIAotdm9pZAotZ250dGFiX3JlbGVhc2VfbWFwcGlu
Z3MoCi0gICAgc3RydWN0IGRvbWFpbiAqZCkKK2ludCBnbnR0YWJfcmVsZWFz
ZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkKQogewogICAgIHN0cnVjdCBn
cmFudF90YWJsZSAgICpndCA9IGQtPmdyYW50X3RhYmxlLCAqcmd0OwogICAg
IHN0cnVjdCBncmFudF9tYXBwaW5nICptYXA7CkBAIC0zNjk1LDggKzM2OTks
MzIgQEAgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3MoCiAKICAgICBCVUdfT04o
IWQtPmlzX2R5aW5nKTsKIAotICAgIGZvciAoIGhhbmRsZSA9IDA7IGhhbmRs
ZSA8IGd0LT5tYXB0cmFja19saW1pdDsgaGFuZGxlKysgKQorICAgIGlmICgg
IWd0IHx8ICFndC0+bWFwdHJhY2sgKQorICAgICAgICByZXR1cm4gMDsKKwor
ICAgIGZvciAoIGhhbmRsZSA9IGd0LT5tYXB0cmFja19saW1pdDsgaGFuZGxl
OyApCiAgICAgeworICAgICAgICAvKgorICAgICAgICAgKiBEZWFsIHdpdGgg
ZnVsbCBwYWdlcyBzdWNoIHRoYXQgdGhlaXIgZnJlZWluZyAoaW4gdGhlIGJv
ZHkgb2YgdGhlCisgICAgICAgICAqIGlmKCkpIHJlbWFpbnMgc2ltcGxlLgor
ICAgICAgICAgKi8KKyAgICAgICAgaWYgKCBoYW5kbGUgPCBndC0+bWFwdHJh
Y2tfbGltaXQgJiYgIShoYW5kbGUgJSBNQVBUUkFDS19QRVJfUEFHRSkgKQor
ICAgICAgICB7CisgICAgICAgICAgICAvKgorICAgICAgICAgICAgICogQ2hh
bmdpbmcgbWFwdHJhY2tfbGltaXQgYWx0ZXJzIG5yX21hcHRyYWNrX2ZyYW1l
cygpJ2VzIHJldHVybgorICAgICAgICAgICAgICogdmFsdWUuIEZyZWUgdGhl
IHRoZW4gZXhjZXNzIHRyYWlsaW5nIHBhZ2UgcmlnaHQgaGVyZSwgcmF0aGVy
CisgICAgICAgICAgICAgKiB0aGFuIGxlYXZpbmcgaXQgdG8gZ3JhbnRfdGFi
bGVfZGVzdHJveSgpIChhbmQgaW4gdHVybiByZXF1aXJpbmcKKyAgICAgICAg
ICAgICAqIHRvIGxlYXZlIGd0LT5tYXB0cmFja19saW1pdCB1bmFsdGVyZWQp
LgorICAgICAgICAgICAgICovCisgICAgICAgICAgICBndC0+bWFwdHJhY2tf
bGltaXQgPSBoYW5kbGU7CisgICAgICAgICAgICBGUkVFX1hFTkhFQVBfUEFH
RShndC0+bWFwdHJhY2tbbnJfbWFwdHJhY2tfZnJhbWVzKGd0KV0pOworCisg
ICAgICAgICAgICBpZiAoIGh5cGVyY2FsbF9wcmVlbXB0X2NoZWNrKCkgKQor
ICAgICAgICAgICAgICAgIHJldHVybiAtRVJFU1RBUlQ7CisgICAgICAgIH0K
KworICAgICAgICAtLWhhbmRsZTsKKwogICAgICAgICBtYXAgPSAmbWFwdHJh
Y2tfZW50cnkoZ3QsIGhhbmRsZSk7CiAgICAgICAgIGlmICggIShtYXAtPmZs
YWdzICYgKEdOVE1BUF9kZXZpY2VfbWFwfEdOVE1BUF9ob3N0X21hcCkpICkK
ICAgICAgICAgICAgIGNvbnRpbnVlOwpAQCAtMzc4MCw2ICszODA4LDExIEBA
IGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKAogCiAgICAgICAgIG1hcC0+Zmxh
Z3MgPSAwOwogICAgIH0KKworICAgIGd0LT5tYXB0cmFja19saW1pdCA9IDA7
CisgICAgRlJFRV9YRU5IRUFQX1BBR0UoZ3QtPm1hcHRyYWNrWzBdKTsKKwor
ICAgIHJldHVybiAwOwogfQogCiB2b2lkIGdyYW50X3RhYmxlX3dhcm5fYWN0
aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKQpAQCAtMzg0Myw4ICszODc2
LDcgQEAgZ3JhbnRfdGFibGVfZGVzdHJveSgKICAgICAgICAgZnJlZV94ZW5o
ZWFwX3BhZ2UodC0+c2hhcmVkX3Jhd1tpXSk7CiAgICAgeGZyZWUodC0+c2hh
cmVkX3Jhdyk7CiAKLSAgICBmb3IgKCBpID0gMDsgaSA8IG5yX21hcHRyYWNr
X2ZyYW1lcyh0KTsgaSsrICkKLSAgICAgICAgZnJlZV94ZW5oZWFwX3BhZ2Uo
dC0+bWFwdHJhY2tbaV0pOworICAgIEFTU0VSVCghdC0+bWFwdHJhY2tfbGlt
aXQpOwogICAgIHZmcmVlKHQtPm1hcHRyYWNrKTsKIAogICAgIGZvciAoIGkg
PSAwOyBpIDwgbnJfYWN0aXZlX2dyYW50X2ZyYW1lcyh0KTsgaSsrICkKLS0t
IGEveGVuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmgKKysrIGIveGVuL2lu
Y2x1ZGUveGVuL2dyYW50X3RhYmxlLmgKQEAgLTQ3LDkgKzQ3LDcgQEAgdm9p
ZCBncmFudF90YWJsZV9pbml0X3ZjcHUoc3RydWN0IHZjcHUgKgogdm9pZCBn
cmFudF90YWJsZV93YXJuX2FjdGl2ZV9ncmFudHMoc3RydWN0IGRvbWFpbiAq
ZCk7CiAKIC8qIERvbWFpbiBkZWF0aCByZWxlYXNlIG9mIGdyYW50ZWQgbWFw
cGluZ3Mgb2Ygb3RoZXIgZG9tYWlucycgbWVtb3J5LiAqLwotdm9pZAotZ250
dGFiX3JlbGVhc2VfbWFwcGluZ3MoCi0gICAgc3RydWN0IGRvbWFpbiAqZCk7
CitpbnQgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Moc3RydWN0IGRvbWFpbiAq
ZCk7CiAKIGludCBtZW1fc2hhcmluZ19ncmVmX3RvX2dmbihzdHJ1Y3QgZ3Jh
bnRfdGFibGUgKmd0LCBncmFudF9yZWZfdCByZWYsCiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgZ2ZuX3QgKmdmbiwgdWludDE2X3QgKnN0YXR1cyk7
CkBAIC04MCw3ICs3OCw3IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBncmFudF90
YWJsZV9pbml0X3ZjcHUKIAogc3RhdGljIGlubGluZSB2b2lkIGdyYW50X3Rh
YmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKSB7fQog
Ci1zdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Mo
c3RydWN0IGRvbWFpbiAqZCkge30KK3N0YXRpYyBpbmxpbmUgaW50IGdudHRh
Yl9yZWxlYXNlX21hcHBpbmdzKHN0cnVjdCBkb21haW4gKmQpIHsgcmV0dXJu
IDA7IH0KIAogc3RhdGljIGlubGluZSBpbnQgbWVtX3NoYXJpbmdfZ3JlZl90
b19nZm4oc3RydWN0IGdyYW50X3RhYmxlICpndCwKICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGdyYW50X3JlZl90IHJlZiwK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-2.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IHJlcGxhY2UgbWFwa2luZCgpCgptYXBraW5kKCkgZG9lc24n
dCBzY2FsZSB2ZXJ5IHdlbGwgd2l0aCBsYXJnZXIgbWFwdHJhY2sgZW50cnkg
Y291bnRzLAp1c2luZyBhIGJydXRlIGZvcmNlIGxpbmVhciBzZWFyY2ggdGhy
b3VnaCBhbGwgZW50cmllcywgd2l0aCB0aGUgb25seQpvcHRpb24gb2YgYW4g
ZWFybHkgbG9vcCBleGl0IGlmIGEgbWF0Y2hpbmcgd3JpdGFibGUgZW50cnkg
d2FzIGZvdW5kLgpJbnRyb2R1Y2UgYSByYWRpeCB0cmVlIGFsb25nc2lkZSB0
aGUgbWFpbiBtYXB0cmFjayB0YWJsZSwgdGh1cwphbGxvd2luZyBtdWNoIGZh
c3RlciBNRk4tYmFzZWQgbG9va3VwLiBUbyBhdm9pZCB0aGUgbmVlZCB0byBh
Y3R1YWxseQphbGxvY2F0ZSBzcGFjZSBmb3IgdGhlIGluZGl2aWR1YWwgbm9k
ZXMsIGVuY29kZSB0aGUgdHdvIGNvdW50ZXJzIGluIHRoZQpub2RlIHBvaW50
ZXJzIHRoZW1zZWx2ZXMsIHRodXMgbGltaXRpbmcgdGhlIG51bWJlciBvZiBw
ZXJtaXR0ZWQKc2ltdWx0YW5lb3VzIHIvbyBhbmQgci93IG1hcHBpbmdzIG9m
IHRoZSBzYW1lIE1GTiB0byAywrPCuS0xICg2NC1iaXQpIC8KMsK54oG1LTEg
KDMyLWJpdCkgZWFjaC4KClRvIGF2b2lkIGVuZm9yY2luZyBhbiB1bm5lY2Vz
c2FyaWx5IGxvdyBib3VuZCBvbiB0aGUgbnVtYmVyIG9mCnNpbXVsdGFuZW91
cyBtYXBwaW5ncyBvZiBhIHNpbmdsZSBNRk4sIGludHJvZHVjZQpyYWRpeF90
cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfSBwYXJhbGxlbGluZwpy
YWRpeF90cmVlX3tpbnRfdG9fcHRyLHB0cl90b19pbnR9LgoKQXMgYSBjb25z
ZXF1ZW5jZSBsb2NraW5nIGNoYW5nZXMgYXJlIGFsc28gYXBwbGljYWJsZTog
V2l0aCB0aGVyZSBubwpsb25nZXIgYmVpbmcgYW55IGluc3BlY3Rpb24gb2Yg
dGhlIHJlbW90ZSBkb21haW4ncyBhY3RpdmUgZW50cmllcywKdGhlcmUncyBh
bHNvIG5vIG5lZWQgYW55bW9yZSB0byBob2xkIHRoZSByZW1vdGUgZG9tYWlu
J3MgZ3JhbnQgdGFibGUKbG9jay4gQW5kIHNpbmNlIHdlJ3JlIG5vIGxvbmdl
ciBpdGVyYXRpbmcgb3ZlciB0aGUgbG9jYWwgZG9tYWluJ3MgbWFwCnRyYWNr
IHRhYmxlLCB0aGUgbG9jayBpbiBtYXBfZ3JhbnRfcmVmKCkgY2FuIGFsc28g
YmUgZHJvcHBlZCBiZWZvcmUgdGhlCm5ldyBtYXB0cmFjayBlbnRyeSBhY3R1
YWxseSBnZXRzIHBvcHVsYXRlZC4KCkFzIGEgbmljZSBzaWRlIGVmZmVjdCB0
aGlzIGFsc28gcmVkdWNlcyB0aGUgbnVtYmVyIG9mIElPTU1VIG9wZXJhdGlv
bnMKaW4gdW5tYXBfY29tbW9uKCk6IFByZXZpb3VzbHkgd2Ugd291bGQgaGF2
ZSAiZXN0YWJsaXNoZWQiIGEgcmVhZGFibGUKbWFwcGluZyB3aGVuZXZlciB3
ZSBkaWRuJ3QgZmluZCBhIHdyaXRhYmxlIGVudHJ5IGFueW1vcmUgKHlldCwg
b2YKY291cnNlLCBhdCBsZWFzdCBvbmUgcmVhZGFibGUgb25lKS4gQnV0IHdl
IG9ubHkgbmVlZCB0byBkbyB0aGlzIGlmIHdlCmFjdHVhbGx5IGRyb3BwZWQg
dGhlIGxhc3Qgd3JpdGFibGUgZW50cnksIG5vdCBpZiB0aGVyZSB3ZXJlIG5v
bmUgYWxyZWFkeQpiZWZvcmUuCgpUaGlzIGlzIHBhcnQgb2YgQ1ZFLTIwMjEt
Mjg2OTggLyBYU0EtMzgwLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTogSnVsaWVuIEdyYWxs
IDxqZ3JhbGxAYW1hem9uLmNvbT4KLS0tCkkgaG9wZSB0aGF0IGxpbWl0aW5n
IHRoZSBtYXAgY291bnQgdG8gMzJrIG9uIEFybTMyIGlzIGdvb2QgZW5vdWdo
LiBJCmFsc28gaG9wZSBpdCBpcyBvdXQgb2YgcXVlc3Rpb24gdGhhdCAyRyBv
ZiBtYXBwaW5ncyBhcmUgZW5vdWdoIG9uIDY0LWJpdAphcmNoaXRlY3R1cmVz
LgoKSSdtIHVzaW5nIHRoZSBncmFudCB0YWJsZSBsb2NrIGZvciBzeW5jaHJv
bml6YXRpb24gdG8gbGltaXQgZGlmZmVyZW5jZXMKaW4gYmVoYXZpb3IgdG8g
cHJpb3IgY29kZS4gSSB0aGluayBpbiBwcmluY2lwbGUgdGhlIG1hcHRyYWNr
IGxvY2sgY291bGQKYmUgdXNlZCBlcXVhbGx5IHdlbGwuCgpTaG91bGRuJ3Qg
SU9NTVUgaW5zZXJ0aW9ucyBiZSBsaW1pdGVkIGFueXdheSB0byBHTlRNQVBf
ZGV2aWNlX21hcApyZXF1ZXN0cz8gVGhpcyB3b3VsZCBmdXJ0aGVyIHNhdmUg
b24gdGhlIG51bWJlciBvZiByYWRpeCB0cmVlIG5vZGVzIGluCm5lZWQgb2Yg
bWFpbnRhaW5pbmcuCgpJJ20gaGVzaXRhbnQgdG8gaW50cm9kdWNlIEdOVFNU
XyogaW4gYSBzZWN1cml0eSBwYXRjaCwgYnV0IGJlaW5nIGFibGUgdG8KdGVs
bCBhbGxvY2F0aW9uIGZhaWx1cmUgb3IgY291bnRlciBvdmVyZmxvdyBmcm9t
IG90aGVyIGVycm9ycyBtaWdodCBiZQp3b3J0aHdoaWxlLgoKSSBkb24ndCB0
aGluayBhZGRpbmcgYW55dGhpbmcgdG8gZ250dGFiX3VzYWdlX3ByaW50KCkg
aXMgdXNlZnVsOgpyYWRpeF90cmVlX2dhbmdfbG9va3VwKCkgcmVxdWlyZXMg
bm9kZXMgdG8gcmVjb3JkIHRoZWlyIG93biBpbmRleGVzIGludG8KdGhlIHRy
ZWUsIHdoaWNoIHdlIGRvbid0IGRvIHRvIHNhdmUgc3BhY2UuIFlldCB3aXRo
b3V0IGluZGV4ZXMgcHJpbnRpbmcKbm9kZSBjb250ZW50cyBpc24ndCB2ZXJ5
IHVzZWZ1bC4gUGx1cyB0aGVyZSdzIGFsc28gbm8gcHJpbnRpbmcgb2YgdGhl
Cm1haW4gbWFwdHJhY2sgdGFibGUgY29udGVudHMuCi0tLQp2MzogQ2hlY2sg
Zm9yIHJhZGl4X3RyZWVfbG9va3VwX3Nsb3QoKSByZXR1cm5pbmcgTlVMTC4g
Q29udmVydCAtRUVYSVNUCiAgICB0byAtRUJVU1kuIEFkZCBjb21tZW50cy4g
UmUtYmFzZSBvdmVyIGNvbW1lbnQgYWRkaXRpb24gaW4gcGF0Y2ggMS4KdjI6
IE5ldy4KCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRfdGFibGUuYworKysgYi94
ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTM3LDYgKzM3LDcgQEAKICNp
bmNsdWRlIDx4ZW4vaW9tbXUuaD4KICNpbmNsdWRlIDx4ZW4vcGFnaW5nLmg+
CiAjaW5jbHVkZSA8eGVuL2tleWhhbmRsZXIuaD4KKyNpbmNsdWRlIDx4ZW4v
cmFkaXgtdHJlZS5oPgogI2luY2x1ZGUgPHhlbi92bWFwLmg+CiAjaW5jbHVk
ZSA8eGVuL25vc3BlYy5oPgogI2luY2x1ZGUgPHhzbS94c20uaD4KQEAgLTgy
LDggKzgzLDEzIEBAIHN0cnVjdCBncmFudF90YWJsZSB7CiAgICAgZ3JhbnRf
c3RhdHVzX3QgICAgICAgKipzdGF0dXM7CiAgICAgLyogQWN0aXZlIGdyYW50
IHRhYmxlLiAqLwogICAgIHN0cnVjdCBhY3RpdmVfZ3JhbnRfZW50cnkgKiph
Y3RpdmU7Ci0gICAgLyogTWFwcGluZyB0cmFja2luZyB0YWJsZSBwZXIgdmNw
dS4gKi8KKyAgICAvKiBIYW5kbGUtaW5kZXhlZCB0cmFja2luZyB0YWJsZSBv
ZiBtYXBwaW5ncy4gKi8KICAgICBzdHJ1Y3QgZ3JhbnRfbWFwcGluZyAqKm1h
cHRyYWNrOworICAgIC8qCisgICAgICogTUZOLWluZGV4ZWQgdHJhY2tpbmcg
dHJlZSBvZiBtYXBwaW5ncywgaWYgbmVlZGVkLiAgTm90ZSB0aGF0IHRoaXMg
aXMKKyAgICAgKiBwcm90ZWN0ZWQgYnkgQGxvY2ssIG5vdCBAbWFwdHJhY2tf
bG9jay4KKyAgICAgKi8KKyAgICBzdHJ1Y3QgcmFkaXhfdHJlZV9yb290IG1h
cHRyYWNrX3RyZWU7CiAKICAgICAvKiBEb21haW4gdG8gd2hpY2ggdGhpcyBz
dHJ1Y3QgZ3JhbnRfdGFibGUgYmVsb25ncy4gKi8KICAgICBjb25zdCBzdHJ1
Y3QgZG9tYWluICpkb21haW47CkBAIC01MTYsMzQgKzUyMiw2IEBAIHN0YXRp
YyBpbnQgZ2V0X3BhZ2VkX2ZyYW1lKHVuc2lnbmVkIGxvbmcKICAgICByZXR1
cm4gR05UU1Rfb2theTsKIH0KIAotc3RhdGljIGlubGluZSB2b2lkCi1kb3Vi
bGVfZ3RfbG9jayhzdHJ1Y3QgZ3JhbnRfdGFibGUgKmxndCwgc3RydWN0IGdy
YW50X3RhYmxlICpyZ3QpCi17Ci0gICAgLyoKLSAgICAgKiBTZWUgbWFwa2lu
ZCgpIGZvciB3aHkgdGhlIHdyaXRlIGxvY2sgaXMgYWxzbyByZXF1aXJlZCBm
b3IgdGhlCi0gICAgICogcmVtb3RlIGRvbWFpbi4KLSAgICAgKi8KLSAgICBp
ZiAoIGxndCA8IHJndCApCi0gICAgewotICAgICAgICBncmFudF93cml0ZV9s
b2NrKGxndCk7Ci0gICAgICAgIGdyYW50X3dyaXRlX2xvY2socmd0KTsKLSAg
ICB9Ci0gICAgZWxzZQotICAgIHsKLSAgICAgICAgaWYgKCBsZ3QgIT0gcmd0
ICkKLSAgICAgICAgICAgIGdyYW50X3dyaXRlX2xvY2socmd0KTsKLSAgICAg
ICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOwotICAgIH0KLX0KLQotc3RhdGlj
IGlubGluZSB2b2lkCi1kb3VibGVfZ3RfdW5sb2NrKHN0cnVjdCBncmFudF90
YWJsZSAqbGd0LCBzdHJ1Y3QgZ3JhbnRfdGFibGUgKnJndCkKLXsKLSAgICBn
cmFudF93cml0ZV91bmxvY2sobGd0KTsKLSAgICBpZiAoIGxndCAhPSByZ3Qg
KQotICAgICAgICBncmFudF93cml0ZV91bmxvY2socmd0KTsKLX0KLQogI2Rl
ZmluZSBJTlZBTElEX01BUFRSQUNLX0hBTkRMRSBVSU5UX01BWAogCiBzdGF0
aWMgaW5saW5lIGdyYW50X2hhbmRsZV90CkBAIC05NzAsNDEgKzk0OCwxNyBA
QCBzdGF0aWMgc3RydWN0IGFjdGl2ZV9ncmFudF9lbnRyeSAqZ3JhbnRfCiAg
ICAgcmV0dXJuIEVSUl9QVFIoLUVJTlZBTCk7CiB9CiAKLSNkZWZpbmUgTUFQ
S0lORF9SRUFEIDEKLSNkZWZpbmUgTUFQS0lORF9XUklURSAyCi1zdGF0aWMg
dW5zaWduZWQgaW50IG1hcGtpbmQoCi0gICAgc3RydWN0IGdyYW50X3RhYmxl
ICpsZ3QsIGNvbnN0IHN0cnVjdCBkb21haW4gKnJkLCBtZm5fdCBtZm4pCi17
Ci0gICAgc3RydWN0IGdyYW50X21hcHBpbmcgKm1hcDsKLSAgICBncmFudF9o
YW5kbGVfdCBoYW5kbGUsIGxpbWl0ID0gbGd0LT5tYXB0cmFja19saW1pdDsK
LSAgICB1bnNpZ25lZCBpbnQga2luZCA9IDA7Ci0KLSAgICAvKgotICAgICAq
IE11c3QgaGF2ZSB0aGUgbG9jYWwgZG9tYWluJ3MgZ3JhbnQgdGFibGUgd3Jp
dGUgbG9jayB3aGVuCi0gICAgICogaXRlcmF0aW5nIG92ZXIgaXRzIG1hcHRy
YWNrIGVudHJpZXMuCi0gICAgICovCi0gICAgQVNTRVJUKHBlcmNwdV9yd19p
c193cml0ZV9sb2NrZWQoJmxndC0+bG9jaykpOwotICAgIC8qCi0gICAgICog
TXVzdCBoYXZlIHRoZSByZW1vdGUgZG9tYWluJ3MgZ3JhbnQgdGFibGUgd3Jp
dGUgbG9jayB3aGlsZQotICAgICAqIGNvdW50aW5nIGl0cyBhY3RpdmUgZW50
cmllcy4KLSAgICAgKi8KLSAgICBBU1NFUlQocGVyY3B1X3J3X2lzX3dyaXRl
X2xvY2tlZCgmcmQtPmdyYW50X3RhYmxlLT5sb2NrKSk7Ci0KLSAgICBzbXBf
cm1iKCk7Ci0KLSAgICBmb3IgKCBoYW5kbGUgPSAwOyAhKGtpbmQgJiBNQVBL
SU5EX1dSSVRFKSAmJiBoYW5kbGUgPCBsaW1pdDsgaGFuZGxlKysgKQotICAg
IHsKLSAgICAgICAgbWFwID0gJm1hcHRyYWNrX2VudHJ5KGxndCwgaGFuZGxl
KTsKLSAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3MgJiAoR05UTUFQX2Rldmlj
ZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgfHwKLSAgICAgICAgICAgICBtYXAt
PmRvbWlkICE9IHJkLT5kb21haW5faWQgKQotICAgICAgICAgICAgY29udGlu
dWU7Ci0gICAgICAgIGlmICggbWZuX2VxKF9hY3RpdmVfZW50cnkocmQtPmdy
YW50X3RhYmxlLCBtYXAtPnJlZikubWZuLCBtZm4pICkKLSAgICAgICAgICAg
IGtpbmQgfD0gbWFwLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSA/Ci0gICAg
ICAgICAgICAgICAgICAgIE1BUEtJTkRfUkVBRCA6IE1BUEtJTkRfV1JJVEU7
Ci0gICAgfQotCi0gICAgcmV0dXJuIGtpbmQ7Ci19Cit1bmlvbiBtYXB0cmFj
a19ub2RlIHsKKyAgICBzdHJ1Y3QgeworICAgICAgICAvKiBSYWRpeCB0cmVl
IHNsb3QgcG9pbnRlcnMgdXNlIHR3byBvZiB0aGUgYml0cy4gKi8KKyNpZmRl
ZiBfX0JJR19FTkRJQU5fQklURklFTEQKKyAgICAgICAgdW5zaWduZWQgbG9u
ZyAgICA6IDI7CisjZW5kaWYKKyAgICAgICAgdW5zaWduZWQgbG9uZyByZCA6
IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsKKyAgICAgICAgdW5zaWduZWQgbG9u
ZyB3ciA6IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsKKyAgICB9IGNudDsKKyAg
ICB1bnNpZ25lZCBsb25nIHJhdzsKK307CiAKIHN0YXRpYyB2b2lkCiBtYXBf
Z3JhbnRfcmVmKApAQCAtMTAyMyw3ICs5NzcsNiBAQCBtYXBfZ3JhbnRfcmVm
KAogICAgIHN0cnVjdCBncmFudF9tYXBwaW5nICptdDsKICAgICBncmFudF9l
bnRyeV9oZWFkZXJfdCAqc2hhaDsKICAgICB1aW50MTZfdCAqc3RhdHVzOwot
ICAgIGJvb2xfdCBuZWVkX2lvbW11OwogCiAgICAgbGQgPSBjdXJyZW50LT5k
b21haW47CiAKQEAgLTEyNDQsMzEgKzExOTcsNzUgQEAgbWFwX2dyYW50X3Jl
ZigKICAgICAgKiBhcyBtZW0tc2hhcmluZyBhbmQgSU9NTVUgdXNlIGFyZSBp
bmNvbXBhdGlibGUpLiBUaGUgZG9tX2lvIGNhc2Ugd291bGQKICAgICAgKiBu
ZWVkIGNoZWNraW5nIHNlcGFyYXRlbHkgaWYgd2UgY29tcGFyZWQgYWdhaW5z
dCBvd25lciBoZXJlLgogICAgICAqLwotICAgIG5lZWRfaW9tbXUgPSBsZCAh
PSByZCAmJiBnbnR0YWJfbmVlZF9pb21tdV9tYXBwaW5nKGxkKTsKLSAgICBp
ZiAoIG5lZWRfaW9tbXUgKQorICAgIGlmICggbGQgIT0gcmQgJiYgZ250dGFi
X25lZWRfaW9tbXVfbWFwcGluZyhsZCkgKQogICAgIHsKKyAgICAgICAgdW5p
b24gbWFwdHJhY2tfbm9kZSBub2RlID0geworICAgICAgICAgICAgLmNudC5y
ZCA9ICEhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSksCisgICAgICAg
ICAgICAuY250LndyID0gIShvcC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkp
LAorICAgICAgICB9OworICAgICAgICBpbnQgZXJyOworICAgICAgICB2b2lk
ICoqc2xvdCA9IE5VTEw7CiAgICAgICAgIHVuc2lnbmVkIGludCBraW5kOwog
Ci0gICAgICAgIGRvdWJsZV9ndF9sb2NrKGxndCwgcmd0KTsKKyAgICAgICAg
Z3JhbnRfd3JpdGVfbG9jayhsZ3QpOworCisgICAgICAgIGVyciA9IHJhZGl4
X3RyZWVfaW5zZXJ0KCZsZ3QtPm1hcHRyYWNrX3RyZWUsIG1mbl94KG1mbiks
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVf
dWxvbmdfdG9fcHRyKG5vZGUucmF3KSk7CisgICAgICAgIGlmICggZXJyID09
IC1FRVhJU1QgKQorICAgICAgICB7CisgICAgICAgICAgICBzbG90ID0gcmFk
aXhfdHJlZV9sb29rdXBfc2xvdCgmbGd0LT5tYXB0cmFja190cmVlLCBtZm5f
eChtZm4pKTsKKyAgICAgICAgICAgIGlmICggbGlrZWx5KHNsb3QpICkKKyAg
ICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICBub2RlLnJhdyA9IHJhZGl4
X3RyZWVfcHRyX3RvX3Vsb25nKCpzbG90KTsKKyAgICAgICAgICAgICAgICBl
cnIgPSAtRUJVU1k7CisKKyAgICAgICAgICAgICAgICAvKiBVcGRhdGUgbm9k
ZSBvbmx5IHdoZW4gcmVmY291bnQgZG9lc24ndCBvdmVyZmxvdy4gKi8KKyAg
ICAgICAgICAgICAgICBpZiAoIG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25s
eSA/ICsrbm9kZS5jbnQucmQKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICA6ICsrbm9kZS5jbnQud3IgKQorICAg
ICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJl
ZV9yZXBsYWNlX3Nsb3Qoc2xvdCwKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJlZV91bG9uZ190b19wdHIo
bm9kZS5yYXcpKTsKKyAgICAgICAgICAgICAgICAgICAgZXJyID0gMDsKKyAg
ICAgICAgICAgICAgICB9CisgICAgICAgICAgICB9CisgICAgICAgICAgICBl
bHNlCisgICAgICAgICAgICAgICAgQVNTRVJUX1VOUkVBQ0hBQkxFKCk7Cisg
ICAgICAgIH0KIAogICAgICAgICAvKgogICAgICAgICAgKiBXZSdyZSBub3Qg
dHJhbnNsYXRlZCwgc28gd2Uga25vdyB0aGF0IGRmbnMgYW5kIG1mbnMgYXJl
CiAgICAgICAgICAqIHRoZSBzYW1lIHRoaW5ncywgc28gdGhlIElPTU1VIGVu
dHJ5IGlzIGFsd2F5cyAxLXRvLTEuCiAgICAgICAgICAqLwotICAgICAgICBr
aW5kID0gbWFwa2luZChsZ3QsIHJkLCBtZm4pOwotICAgICAgICBpZiAoICEo
b3AtPmZsYWdzICYgR05UTUFQX3JlYWRvbmx5KSAmJgotICAgICAgICAgICAg
ICEoa2luZCAmIE1BUEtJTkRfV1JJVEUpICkKKyAgICAgICAgaWYgKCAhKG9w
LT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSkgJiYgbm9kZS5jbnQud3IgPT0g
MSApCiAgICAgICAgICAgICBraW5kID0gSU9NTVVGX3JlYWRhYmxlIHwgSU9N
TVVGX3dyaXRhYmxlOwotICAgICAgICBlbHNlIGlmICggIWtpbmQgKQorICAg
ICAgICBlbHNlIGlmICggKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSkg
JiYKKyAgICAgICAgICAgICAgICAgIG5vZGUuY250LnJkID09IDEgJiYgIW5v
ZGUuY250LndyICkKICAgICAgICAgICAgIGtpbmQgPSBJT01NVUZfcmVhZGFi
bGU7CiAgICAgICAgIGVsc2UKICAgICAgICAgICAgIGtpbmQgPSAwOwotICAg
ICAgICBpZiAoIGtpbmQgJiYgaW9tbXVfbGVnYWN5X21hcChsZCwgX2Rmbiht
Zm5feChtZm4pKSwgbWZuLCAxLCBraW5kKSApCisgICAgICAgIGlmICggZXJy
IHx8CisgICAgICAgICAgICAgKGtpbmQgJiYgaW9tbXVfbGVnYWN5X21hcChs
ZCwgX2RmbihtZm5feChtZm4pKSwgbWZuLCAxLCBraW5kKSkgKQogICAgICAg
ICB7Ci0gICAgICAgICAgICBkb3VibGVfZ3RfdW5sb2NrKGxndCwgcmd0KTsK
KyAgICAgICAgICAgIGlmICggIWVyciApCisgICAgICAgICAgICB7CisgICAg
ICAgICAgICAgICAgaWYgKCBzbG90ICkKKyAgICAgICAgICAgICAgICB7Cisg
ICAgICAgICAgICAgICAgICAgIG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25s
eSA/IG5vZGUuY250LnJkLS0KKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIDogbm9kZS5jbnQud3ItLTsKKyAgICAg
ICAgICAgICAgICAgICAgcmFkaXhfdHJlZV9yZXBsYWNlX3Nsb3Qoc2xvdCwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
cmFkaXhfdHJlZV91bG9uZ190b19wdHIobm9kZS5yYXcpKTsKKyAgICAgICAg
ICAgICAgICB9CisgICAgICAgICAgICAgICAgZWxzZQorICAgICAgICAgICAg
ICAgICAgICByYWRpeF90cmVlX2RlbGV0ZSgmbGd0LT5tYXB0cmFja190cmVl
LCBtZm5feChtZm4pKTsKKyAgICAgICAgICAgIH0KKwogICAgICAgICAgICAg
cmMgPSBHTlRTVF9nZW5lcmFsX2Vycm9yOwotICAgICAgICAgICAgZ290byB1
bmRvX291dDsKICAgICAgICAgfQorCisgICAgICAgIGdyYW50X3dyaXRlX3Vu
bG9jayhsZ3QpOworCisgICAgICAgIGlmICggcmMgIT0gR05UU1Rfb2theSAp
CisgICAgICAgICAgICBnb3RvIHVuZG9fb3V0OwogICAgIH0KIAogICAgIFRS
QUNFXzFEKFRSQ19NRU1fUEFHRV9HUkFOVF9NQVAsIG9wLT5kb20pOwpAQCAt
MTI3NiwxMCArMTI3Myw2IEBAIG1hcF9ncmFudF9yZWYoCiAgICAgLyoKICAg
ICAgKiBBbGwgbWFwdHJhY2sgZW50cnkgdXNlcnMgY2hlY2sgbXQtPmZsYWdz
IGZpcnN0IGJlZm9yZSB1c2luZyB0aGUKICAgICAgKiBvdGhlciBmaWVsZHMg
c28ganVzdCBlbnN1cmUgdGhlIGZsYWdzIGZpZWxkIGlzIHN0b3JlZCBsYXN0
LgotICAgICAqCi0gICAgICogSG93ZXZlciwgaWYgZ250dGFiX25lZWRfaW9t
bXVfbWFwcGluZygpIHRoZW4gdGhpcyB3b3VsZCByYWNlCi0gICAgICogd2l0
aCBhIGNvbmN1cnJlbnQgbWFwa2luZCgpIGNhbGwgKG9uIGFuIHVubWFwLCBm
b3IgZXhhbXBsZSkKLSAgICAgKiBhbmQgYSBsb2NrIGlzIHJlcXVpcmVkLgog
ICAgICAqLwogICAgIG10ID0gJm1hcHRyYWNrX2VudHJ5KGxndCwgaGFuZGxl
KTsKICAgICBtdC0+ZG9taWQgPSBvcC0+ZG9tOwpAQCAtMTI4Nyw5ICsxMjgw
LDYgQEAgbWFwX2dyYW50X3JlZigKICAgICBzbXBfd21iKCk7CiAgICAgd3Jp
dGVfYXRvbWljKCZtdC0+ZmxhZ3MsIG9wLT5mbGFncyk7CiAKLSAgICBpZiAo
IG5lZWRfaW9tbXUgKQotICAgICAgICBkb3VibGVfZ3RfdW5sb2NrKGxndCwg
cmd0KTsKLQogICAgIG9wLT5kZXZfYnVzX2FkZHIgPSBtZm5fdG9fbWFkZHIo
bWZuKTsKICAgICBvcC0+aGFuZGxlICAgICAgID0gaGFuZGxlOwogICAgIG9w
LT5zdGF0dXMgICAgICAgPSBHTlRTVF9va2F5OwpAQCAtMTQ5NywxOSArMTQ4
NywzNCBAQCB1bm1hcF9jb21tb24oCiAgICAgLyogU2VlIHRoZSByZXNwZWN0
aXZlIGNvbW1lbnQgaW4gbWFwX2dyYW50X3JlZigpLiAqLwogICAgIGlmICgg
cmMgPT0gR05UU1Rfb2theSAmJiBsZCAhPSByZCAmJiBnbnR0YWJfbmVlZF9p
b21tdV9tYXBwaW5nKGxkKSApCiAgICAgewotICAgICAgICB1bnNpZ25lZCBp
bnQga2luZDsKKyAgICAgICAgdm9pZCAqKnNsb3Q7CisgICAgICAgIHVuaW9u
IG1hcHRyYWNrX25vZGUgbm9kZTsKICAgICAgICAgaW50IGVyciA9IDA7CiAK
LSAgICAgICAgZG91YmxlX2d0X2xvY2sobGd0LCByZ3QpOworICAgICAgICBn
cmFudF93cml0ZV9sb2NrKGxndCk7CisgICAgICAgIHNsb3QgPSByYWRpeF90
cmVlX2xvb2t1cF9zbG90KCZsZ3QtPm1hcHRyYWNrX3RyZWUsIG1mbl94KG9w
LT5tZm4pKTsKKyAgICAgICAgbm9kZS5yYXcgPSBsaWtlbHkoc2xvdCkgPyBy
YWRpeF90cmVlX3B0cl90b191bG9uZygqc2xvdCkgOiAwOworCisgICAgICAg
IC8qIFJlZmNvdW50IG11c3Qgbm90IHVuZGVyZmxvdy4gKi8KKyAgICAgICAg
aWYgKCAhKGZsYWdzICYgR05UTUFQX3JlYWRvbmx5ID8gbm9kZS5jbnQucmQt
LQorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiBu
b2RlLmNudC53ci0tKSApCisgICAgICAgICAgICBCVUcoKTsKIAotICAgICAg
ICBraW5kID0gbWFwa2luZChsZ3QsIHJkLCBvcC0+bWZuKTsKLSAgICAgICAg
aWYgKCAha2luZCApCisgICAgICAgIGlmICggIW5vZGUucmF3ICkKICAgICAg
ICAgICAgIGVyciA9IGlvbW11X2xlZ2FjeV91bm1hcChsZCwgX2RmbihtZm5f
eChvcC0+bWZuKSksIDEpOwotICAgICAgICBlbHNlIGlmICggIShraW5kICYg
TUFQS0lORF9XUklURSkgKQorICAgICAgICBlbHNlIGlmICggIShmbGFncyAm
IEdOVE1BUF9yZWFkb25seSkgJiYgIW5vZGUuY250LndyICkKICAgICAgICAg
ICAgIGVyciA9IGlvbW11X2xlZ2FjeV9tYXAobGQsIF9kZm4obWZuX3gob3At
Pm1mbikpLCBvcC0+bWZuLCAxLAogICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBJT01NVUZfcmVhZGFibGUpOwogCi0gICAgICAgIGRvdWJs
ZV9ndF91bmxvY2sobGd0LCByZ3QpOworICAgICAgICBpZiAoIGVyciApCisg
ICAgICAgICAgICA7CisgICAgICAgIGVsc2UgaWYgKCAhbm9kZS5yYXcgKQor
ICAgICAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUoJmxndC0+bWFwdHJhY2tf
dHJlZSwgbWZuX3gob3AtPm1mbikpOworICAgICAgICBlbHNlCisgICAgICAg
ICAgICByYWRpeF90cmVlX3JlcGxhY2Vfc2xvdChzbG90LAorICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJlZV91bG9uZ190
b19wdHIobm9kZS5yYXcpKTsKKworICAgICAgICBncmFudF93cml0ZV91bmxv
Y2sobGd0KTsKIAogICAgICAgICBpZiAoIGVyciApCiAgICAgICAgICAgICBy
YyA9IEdOVFNUX2dlbmVyYWxfZXJyb3I7CkBAIC0xOTU2LDYgKzE5NjEsOCBA
QCBpbnQgZ3JhbnRfdGFibGVfaW5pdChzdHJ1Y3QgZG9tYWluICpkLCBpCiAg
ICAgICAgIGd0LT5tYXB0cmFjayA9IHZ6YWxsb2MoZ3QtPm1heF9tYXB0cmFj
a19mcmFtZXMgKiBzaXplb2YoKmd0LT5tYXB0cmFjaykpOwogICAgICAgICBp
ZiAoIGd0LT5tYXB0cmFjayA9PSBOVUxMICkKICAgICAgICAgICAgIGdvdG8g
b3V0OworCisgICAgICAgIHJhZGl4X3RyZWVfaW5pdCgmZ3QtPm1hcHRyYWNr
X3RyZWUpOwogICAgIH0KIAogICAgIC8qIFNoYXJlZCBncmFudCB0YWJsZS4g
Ki8KQEAgLTM3MDQsNiArMzcxMSw4IEBAIGludCBnbnR0YWJfcmVsZWFzZV9t
YXBwaW5ncyhzdHJ1Y3QgZG9tYWkKIAogICAgIGZvciAoIGhhbmRsZSA9IGd0
LT5tYXB0cmFja19saW1pdDsgaGFuZGxlOyApCiAgICAgeworICAgICAgICBt
Zm5fdCBtZm47CisKICAgICAgICAgLyoKICAgICAgICAgICogRGVhbCB3aXRo
IGZ1bGwgcGFnZXMgc3VjaCB0aGF0IHRoZWlyIGZyZWVpbmcgKGluIHRoZSBi
b2R5IG9mIHRoZQogICAgICAgICAgKiBpZigpKSByZW1haW5zIHNpbXBsZS4K
QEAgLTM4MDEsMTcgKzM4MTAsMzEgQEAgaW50IGdudHRhYl9yZWxlYXNlX21h
cHBpbmdzKHN0cnVjdCBkb21haQogCiAgICAgICAgIHJlZHVjZV9zdGF0dXNf
Zm9yX3BpbihyZCwgYWN0LCBzdGF0dXMsIG1hcC0+ZmxhZ3MgJiBHTlRNQVBf
cmVhZG9ubHkpOwogCisgICAgICAgIG1mbiA9IGFjdC0+bWZuOworCiAgICAg
ICAgIGFjdGl2ZV9lbnRyeV9yZWxlYXNlKGFjdCk7CiAgICAgICAgIGdyYW50
X3JlYWRfdW5sb2NrKHJndCk7CiAKICAgICAgICAgcmN1X3VubG9ja19kb21h
aW4ocmQpOwogCiAgICAgICAgIG1hcC0+ZmxhZ3MgPSAwOworCisgICAgICAg
IC8qCisgICAgICAgICAqIFRoaXMgaXMgZXhjZXNzaXZlIGluIHRoYXQgYSBz
aW5nbGUgc3VjaCBjYWxsIHdvdWxkIHN1ZmZpY2UgcGVyCisgICAgICAgICAq
IG1hcHBlZCBNRk4gKG9yIG5vbmUgYXQgYWxsLCBpZiBubyBlbnRyeSB3YXMg
ZXZlciBpbnNlcnRlZCkuIEJ1dCBpdAorICAgICAgICAgKiBzaG91bGQgYmUg
dGhlIGNvbW1vbiBjYXNlIGZvciBhbiBNRk4gdG8gYmUgbWFwcGVkIGp1c3Qg
b25jZSwgYW5kCisgICAgICAgICAqIHRoaXMgd2F5IHdlIGRvbid0IG5lZWQg
dG8gZnVydGhlciBtYWludGFpbiB0aGUgY291bnRlcnMuIFdlIGFsc28KKyAg
ICAgICAgICogZG9uJ3Qgd2FudCB0byBsZWF2ZSBjbGVhbmluZyB1cCBvZiB0
aGUgdHJlZSBhcyBhIHdob2xlIHRvIHRoZSBlbmQKKyAgICAgICAgICogb2Yg
dGhlIGZ1bmN0aW9uLCBhcyB0aGlzIGNvdWxkIHRha2UgcXVpdGUgc29tZSB0
aW1lLgorICAgICAgICAgKi8KKyAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUo
Jmd0LT5tYXB0cmFja190cmVlLCBtZm5feChtZm4pKTsKICAgICB9CiAKICAg
ICBndC0+bWFwdHJhY2tfbGltaXQgPSAwOwogICAgIEZSRUVfWEVOSEVBUF9Q
QUdFKGd0LT5tYXB0cmFja1swXSk7CiAKKyAgICByYWRpeF90cmVlX2Rlc3Ry
b3koJmd0LT5tYXB0cmFja190cmVlLCBOVUxMKTsKKwogICAgIHJldHVybiAw
OwogfQogCi0tLSBhL3hlbi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmgKKysr
IGIveGVuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaApAQCAtMTkwLDYgKzE5
MCwyNSBAQCBzdGF0aWMgaW5saW5lIGludCByYWRpeF90cmVlX3B0cl90b19p
bnQoCiAgICAgcmV0dXJuIChpbnQpKChsb25nKXB0ciA+PiAyKTsKIH0KIAor
LyoqCisgKiByYWRpeF90cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25n
fToKKyAqCisgKiBTYW1lIGZvciB1bnNpZ25lZCBsb25nIHZhbHVlcy4gQmV3
YXJlIHRob3VnaCB0aGF0IG9ubHkgQklUU19QRVJfTE9ORy0yCisgKiBiaXRz
IGFyZSBhY3R1YWxseSB1c2FibGUgZm9yIHRoZSB2YWx1ZS4KKyAqLworc3Rh
dGljIGlubGluZSB2b2lkICpyYWRpeF90cmVlX3Vsb25nX3RvX3B0cih1bnNp
Z25lZCBsb25nIHZhbCkKK3sKKyAgICB1bnNpZ25lZCBsb25nIHB0ciA9ICh2
YWwgPDwgMikgfCAweDI7CisgICAgQVNTRVJUKChwdHIgPj4gMikgPT0gdmFs
KTsKKyAgICByZXR1cm4gKHZvaWQgKilwdHI7Cit9CisKK3N0YXRpYyBpbmxp
bmUgdW5zaWduZWQgbG9uZyByYWRpeF90cmVlX3B0cl90b191bG9uZyh2b2lk
ICpwdHIpCit7CisgICAgQVNTRVJUKCgodW5zaWduZWQgbG9uZylwdHIgJiAw
eDMpID09IDB4Mik7CisgICAgcmV0dXJuICh1bnNpZ25lZCBsb25nKXB0ciA+
PiAyOworfQorCiBpbnQgcmFkaXhfdHJlZV9pbnNlcnQoc3RydWN0IHJhZGl4
X3RyZWVfcm9vdCAqLCB1bnNpZ25lZCBsb25nLCB2b2lkICopOwogdm9pZCAq
cmFkaXhfdHJlZV9sb29rdXAoc3RydWN0IHJhZGl4X3RyZWVfcm9vdCAqLCB1
bnNpZ25lZCBsb25nKTsKIHZvaWQgKipyYWRpeF90cmVlX2xvb2t1cF9zbG90
KHN0cnVjdCByYWRpeF90cmVlX3Jvb3QgKiwgdW5zaWduZWQgbG9uZyk7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-3.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGF2b2lkIHRyaWdnZXJpbmcgYXNzZXJ0aW9uIGluIHJhZGl4
X3RyZWVfdWxvbmdfdG9fcHRyKCkKClJlbGV2YW50IHF1b3RlcyBmcm9tIHRo
ZSBDMTEgc3RhbmRhcmQ6CgoiRXhjZXB0IHdoZXJlIGV4cGxpY2l0bHkgc3Rh
dGVkIG90aGVyd2lzZSwgZm9yIHRoZSBwdXJwb3NlcyBvZiB0aGlzCiBzdWJj
bGF1c2UgdW5uYW1lZCBtZW1iZXJzIG9mIG9iamVjdHMgb2Ygc3RydWN0dXJl
IGFuZCB1bmlvbiB0eXBlIGRvIG5vdAogcGFydGljaXBhdGUgaW4gaW5pdGlh
bGl6YXRpb24uIFVubmFtZWQgbWVtYmVycyBvZiBzdHJ1Y3R1cmUgb2JqZWN0
cwogaGF2ZSBpbmRldGVybWluYXRlIHZhbHVlIGV2ZW4gYWZ0ZXIgaW5pdGlh
bGl6YXRpb24uIgoKIklmIHRoZXJlIGFyZSBmZXdlciBpbml0aWFsaXplcnMg
aW4gYSBicmFjZS1lbmNsb3NlZCBsaXN0IHRoYW4gdGhlcmUgYXJlCiBlbGVt
ZW50cyBvciBtZW1iZXJzIG9mIGFuIGFnZ3JlZ2F0ZSwgWy4uLl0sIHRoZSBy
ZW1haW5kZXIgb2YgdGhlCiBhZ2dyZWdhdGUgc2hhbGwgYmUgaW5pdGlhbGl6
ZWQgaW1wbGljaXRseSB0aGUgc2FtZSBhcyBvYmplY3RzIHRoYXQgaGF2ZQog
c3RhdGljIHN0b3JhZ2UgZHVyYXRpb24uIgoKIklmIGFuIG9iamVjdCB0aGF0
IGhhcyBzdGF0aWMgb3IgdGhyZWFkIHN0b3JhZ2UgZHVyYXRpb24gaXMgbm90
CiBpbml0aWFsaXplZCBleHBsaWNpdGx5LCB0aGVuOgogWy4uLl0KIOKAlCBp
ZiBpdCBpcyBhbiBhZ2dyZWdhdGUsIGV2ZXJ5IG1lbWJlciBpcyBpbml0aWFs
aXplZCAocmVjdXJzaXZlbHkpCiAgIGFjY29yZGluZyB0byB0aGVzZSBydWxl
cywgYW5kIGFueSBwYWRkaW5nIGlzIGluaXRpYWxpemVkIHRvIHplcm8KICAg
Yml0czsKIFsuLi5dIgoKIkEgYml0LWZpZWxkIGRlY2xhcmF0aW9uIHdpdGgg
bm8gZGVjbGFyYXRvciwgYnV0IG9ubHkgYSBjb2xvbiBhbmQgYQogd2lkdGgs
IGluZGljYXRlcyBhbiB1bm5hbWVkIGJpdC1maWVsZC4iIEZvb3Rub3RlOiAi
QW4gdW5uYW1lZCBiaXQtZmllbGQKIHN0cnVjdHVyZSBtZW1iZXIgaXMgdXNl
ZnVsIGZvciBwYWRkaW5nIHRvIGNvbmZvcm0gdG8gZXh0ZXJuYWxseSBpbXBv
c2VkCiBsYXlvdXRzLiIKCiJUaGVyZSBtYXkgYmUgdW5uYW1lZCBwYWRkaW5n
IHdpdGhpbiBhIHN0cnVjdHVyZSBvYmplY3QsIGJ1dCBub3QgYXQgaXRzCiBi
ZWdpbm5pbmcuIgoKV2hpY2ggbWFrZXMgbWUgY29uY2x1ZGU6Ci0gV2hldGhl
ciBhbiB1bm5hbWVkIGJpdC1maWVsZCBtZW1iZXIgaXMgYW4gdW5uYW1lZCBt
ZW1iZXIgb3IgcGFkZGluZyBpcwogIHVuY2xlYXIsIGFuZCBoZW5jZSBhbHNv
IHdoZXRoZXIgdGhlIGxhc3QgcXVvdGUgYWJvdmUgd291bGQgcmVuZGVyIHRo
ZQogIGJpZyBlbmRpYW4gY2FzZSBvZiB0aGUgc3RydWN0dXJlIGRlY2xhcmF0
aW9uIGludmFsaWQuCi0gV2hldGhlciB0aGUgbnVtYmVyIG9mIG1lbWJlcnMg
b2YgYW4gYWdncmVnYXRlIGluY2x1ZGVzIHVubmFtZWQgb25lcyBpcwogIGFs
c28gbm90IHJlYWxseSBjbGVhci4KLSBUaGUgaW5pdGlhbGl6ZXIgaW4gbWFw
X2dyYW50X3JlZigpIGluaXRpYWxpemVzIGFsbCBmaWVsZHMgb2YgdGhlICJj
bnQiCiAgc3ViLXN0cnVjdHVyZSBvZiB0aGUgdW5pb24sIHNvIGFzc3VtaW5n
IHRoZSBzZWNvbmQgcXVvdGUgYWJvdmUgYXBwbGllcwogIGhlcmUgKGluZGly
ZWN0bHkpLCB0aGUgY29tcGlsZXIgaXNuJ3QgcmVxdWlyZWQgdG8gaW1wbGlj
aXRseQogIGluaXRpYWxpemUgdGhlIHJlc3QgKGkuZS4gaW4gcGFydGljdWxh
ciBhbnkgcGFkZGluZykgbGlrZSB3b3VsZCBoYXBwZW4KICBmb3Igc3RhdGlj
IHN0b3JhZ2UgZHVyYXRpb24gb2JqZWN0cy4KCkdjYyA3LjQuMSBjYW4gYmUg
b2JzZXJ2ZWQgKGFwcGFyZW50bHkgaW4gZGVidWcgYnVpbGRzIG9ubHkpIHRv
IHRyYW5zbGF0ZQphZm9yZW1lbnRpb25lZCBpbml0aWFsaXplciB0byBhIHJl
YWQtbW9kaWZ5LXdyaXRlIG9wZXJhdGlvbiBvZiBhIHN0YWNrCnZhcmlhYmxl
LCBsZWF2aW5nIHVuY2hhbmdlZCB0aGUgdG9wIHR3byBiaXRzIG9mIHdoYXRl
dmVyIHdhcyBwcmV2aW91c2x5CmluIHRoYXQgc3RhY2sgc2xvdC4gQ2xlYXJs
eSBpZiBlaXRoZXIgb2YgdGhlIHR3byBiaXRzIHdlcmUgc2V0LApyYWRpeF90
cmVlX3Vsb25nX3RvX3B0cigpJ3MgYXNzZXJ0aW9uIHdvdWxkIHRyaWdnZXIu
CgpUaGVyZWZvcmUsIHRvIGJlIG9uIHRoZSBzYWZlIHNpZGUsIGFkZCBhbiBl
eHBsaWNpdCBwYWRkaW5nIGZpZWxkIGZvciB0aGUKbm9uLWJpZy1lbmRpYW4t
Yml0ZmllbGRzIGNhc2UgYW5kIGdpdmUgYSBkdW1teSBuYW1lIHRvIGJvdGgg
cGFkZGluZwpmaWVsZHMuCgpGaXhlczogOTc4MWI1MWVmZGUyICgiZ250dGFi
OiByZXBsYWNlIG1hcGtpbmQoKSIpClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KQWNrZWQtYnk6IEFuZHJldyBDb29w
ZXIgPGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb20+CgotLS0gYS94ZW4vY29t
bW9uL2dyYW50X3RhYmxlLmMKKysrIGIveGVuL2NvbW1vbi9ncmFudF90YWJs
ZS5jCkBAIC05NTIsMTAgKzk1MiwxMyBAQCB1bmlvbiBtYXB0cmFja19ub2Rl
IHsKICAgICBzdHJ1Y3QgewogICAgICAgICAvKiBSYWRpeCB0cmVlIHNsb3Qg
cG9pbnRlcnMgdXNlIHR3byBvZiB0aGUgYml0cy4gKi8KICNpZmRlZiBfX0JJ
R19FTkRJQU5fQklURklFTEQKLSAgICAgICAgdW5zaWduZWQgbG9uZyAgICA6
IDI7CisgICAgICAgIHVuc2lnbmVkIGxvbmcgXzAgOiAyOwogI2VuZGlmCiAg
ICAgICAgIHVuc2lnbmVkIGxvbmcgcmQgOiBCSVRTX1BFUl9MT05HIC8gMiAt
IDE7CiAgICAgICAgIHVuc2lnbmVkIGxvbmcgd3IgOiBCSVRTX1BFUl9MT05H
IC8gMiAtIDE7CisjaWZuZGVmIF9fQklHX0VORElBTl9CSVRGSUVMRAorICAg
ICAgICB1bnNpZ25lZCBsb25nIF8wIDogMjsKKyNlbmRpZgogICAgIH0gY250
OwogICAgIHVuc2lnbmVkIGxvbmcgcmF3OwogfTsK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.11-1.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.11-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGFkZCBwcmVlbXB0aW9uIGNoZWNrIHRvIGdudHRhYl9yZWxl
YXNlX21hcHBpbmdzKCkKCkEgZ3Vlc3QgbWF5IGRpZSB3aXRoIG1hbnkgZ3Jh
bnQgbWFwcGluZ3Mgc3RpbGwgaW4gcGxhY2UsIG9yIHNpbXBseSB3aXRoCmEg
bGFyZ2UgbWFwdHJhY2sgdGFibGUuIEl0ZXJhdGluZyB0aHJvdWdoIHRoaXMg
bWF5IHRha2UgbW9yZSB0aW1lIHRoYW4KaXMgcmVhc29uYWJsZSB3aXRob3V0
IGludGVybWVkaWF0ZSBwcmVlbXB0aW9uICh0byBydW4gc29mdGlycXMgYW5k
CnBlcmhhcHMgdGhlIHNjaGVkdWxlcikuCgpNb3ZlIHRoZSBpbnZvY2F0aW9u
IG9mIHRoZSBmdW5jdGlvbiB0byB0aGUgc2VjdGlvbiB3aGVyZSBvdGhlcgpy
ZXN0YXJ0YWJsZSBmdW5jdGlvbnMgZ2V0IGludm9rZWQsIGFuZCBoYXZlIHRo
ZSBmdW5jdGlvbiBpdHNlbGYgY2hlY2sKZm9yIHByZWVtcHRpb24gZXZlcnkg
b25jZSBpbiBhIHdoaWxlLiBIYXZlIGl0IGl0ZXJhdGUgdGhlIHRhYmxlCmJh
Y2t3YXJkcywgc3VjaCB0aGF0IGRlY3JlYXNpbmcgdGhlIG1hcHRyYWNrIGxp
bWl0IGlzIGFsbCBpdCB0YWtlcyB0bwpjb252ZXkgcmVzdGFydCBpbmZvcm1h
dGlvbi4KCkluIGRvbWFpbl90ZWFyZG93bigpIGludHJvZHVjZSBQUk9HX25v
bmUgc3VjaCB0aGF0IGluc2VydGluZyBhdCB0aGUKZnJvbnQgd2lsbCBiZSBl
YXNpZXIgZ29pbmcgZm9yd2FyZC4KClRoaXMgaXMgcGFydCBvZiBDVkUtMjAy
MS0yODY5OCAvIFhTQS0zODAuCgpSZXBvcnRlZC1ieTogQW5kcmV3IENvb3Bl
ciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
SnVsaWVuIEdyYWxsIDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9j
b21tb24vZG9tYWluLmMKKysrIGIveGVuL2NvbW1vbi9kb21haW4uYwpAQCAt
NjQ2LDEzICs2NDYsMTUgQEAgaW50IGRvbWFpbl9raWxsKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgICAgIGlmICggZC0+aXNfZHlpbmcgIT0gRE9NRFlJTkdf
YWxpdmUgKQogICAgICAgICAgICAgcmV0dXJuIGRvbWFpbl9raWxsKGQpOwog
ICAgICAgICBkLT5pc19keWluZyA9IERPTURZSU5HX2R5aW5nOwotICAgICAg
ICBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhkKTsKICAgICAgICAgdG1lbV9k
ZXN0cm95KGQtPnRtZW1fY2xpZW50KTsKICAgICAgICAgdm51bWFfZGVzdHJv
eShkLT52bnVtYSk7CiAgICAgICAgIGRvbWFpbl9zZXRfb3V0c3RhbmRpbmdf
cGFnZXMoZCwgMCk7CiAgICAgICAgIGQtPnRtZW1fY2xpZW50ID0gTlVMTDsK
ICAgICAgICAgLyogZmFsbHRocm91Z2ggKi8KICAgICBjYXNlIERPTURZSU5H
X2R5aW5nOgorICAgICAgICByYyA9IGdudHRhYl9yZWxlYXNlX21hcHBpbmdz
KGQpOworICAgICAgICBpZiAoIHJjICkKKyAgICAgICAgICAgIGJyZWFrOwog
ICAgICAgICByYyA9IGV2dGNobl9kZXN0cm95KGQpOwogICAgICAgICBpZiAo
IHJjICkKICAgICAgICAgICAgIGJyZWFrOwotLS0gYS94ZW4vY29tbW9uL2dy
YW50X3RhYmxlLmMKKysrIGIveGVuL2NvbW1vbi9ncmFudF90YWJsZS5jCkBA
IC02Miw3ICs2MiwxMyBAQCBzdHJ1Y3QgZ3JhbnRfdGFibGUgewogICAgIHVu
c2lnbmVkIGludCAgICAgICAgICBucl9ncmFudF9mcmFtZXM7CiAgICAgLyog
TnVtYmVyIG9mIGdyYW50IHN0YXR1cyBmcmFtZXMgc2hhcmVkIHdpdGggZ3Vl
c3QgKGZvciB2ZXJzaW9uIDIpICovCiAgICAgdW5zaWduZWQgaW50ICAgICAg
ICAgIG5yX3N0YXR1c19mcmFtZXM7Ci0gICAgLyogTnVtYmVyIG9mIGF2YWls
YWJsZSBtYXB0cmFjayBlbnRyaWVzLiAqLworICAgIC8qCisgICAgICogTnVt
YmVyIG9mIGF2YWlsYWJsZSBtYXB0cmFjayBlbnRyaWVzLiAgRm9yIGNsZWFu
dXAgcHVycG9zZXMgaXQgaXMKKyAgICAgKiBpbXBvcnRhbnQgdG8gcmVhbGl6
ZSB0aGF0IHRoaXMgZmllbGQgYW5kIEBtYXB0cmFjayBmdXJ0aGVyIGRvd24g
d2lsbAorICAgICAqIG9ubHkgZXZlciBiZSBhY2Nlc3NlZCBieSB0aGUgbG9j
YWwgZG9tYWluLiAgVGh1cyBpdCBpcyBva2F5IHRvIGNsZWFuCisgICAgICog
dXAgZWFybHksIGFuZCB0byBzaHJpbmsgdGhlIGxpbWl0IGZvciB0aGUgcHVy
cG9zZSBvZiB0cmFja2luZyBjbGVhbnVwCisgICAgICogcHJvZ3Jlc3MuCisg
ICAgICovCiAgICAgdW5zaWduZWQgaW50ICAgICAgICAgIG1hcHRyYWNrX2xp
bWl0OwogICAgIC8qIFNoYXJlZCBncmFudCB0YWJsZSAoc2VlIGluY2x1ZGUv
cHVibGljL2dyYW50X3RhYmxlLmgpLiAqLwogICAgIHVuaW9uIHsKQEAgLTM2
MTgsOSArMzYyNCw3IEBAIGdyYW50X3RhYmxlX2NyZWF0ZSgKICAgICByZXR1
cm4gcmV0OwogfQogCi12b2lkCi1nbnR0YWJfcmVsZWFzZV9tYXBwaW5ncygK
LSAgICBzdHJ1Y3QgZG9tYWluICpkKQoraW50IGdudHRhYl9yZWxlYXNlX21h
cHBpbmdzKHN0cnVjdCBkb21haW4gKmQpCiB7CiAgICAgc3RydWN0IGdyYW50
X3RhYmxlICAgKmd0ID0gZC0+Z3JhbnRfdGFibGUsICpyZ3Q7CiAgICAgc3Ry
dWN0IGdyYW50X21hcHBpbmcgKm1hcDsKQEAgLTM2MzQsOCArMzYzOCwzMiBA
QCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncygKIAogICAgIEJVR19PTighZC0+
aXNfZHlpbmcpOwogCi0gICAgZm9yICggaGFuZGxlID0gMDsgaGFuZGxlIDwg
Z3QtPm1hcHRyYWNrX2xpbWl0OyBoYW5kbGUrKyApCisgICAgaWYgKCAhZ3Qg
fHwgIWd0LT5tYXB0cmFjayApCisgICAgICAgIHJldHVybiAwOworCisgICAg
Zm9yICggaGFuZGxlID0gZ3QtPm1hcHRyYWNrX2xpbWl0OyBoYW5kbGU7ICkK
ICAgICB7CisgICAgICAgIC8qCisgICAgICAgICAqIERlYWwgd2l0aCBmdWxs
IHBhZ2VzIHN1Y2ggdGhhdCB0aGVpciBmcmVlaW5nIChpbiB0aGUgYm9keSBv
ZiB0aGUKKyAgICAgICAgICogaWYoKSkgcmVtYWlucyBzaW1wbGUuCisgICAg
ICAgICAqLworICAgICAgICBpZiAoIGhhbmRsZSA8IGd0LT5tYXB0cmFja19s
aW1pdCAmJiAhKGhhbmRsZSAlIE1BUFRSQUNLX1BFUl9QQUdFKSApCisgICAg
ICAgIHsKKyAgICAgICAgICAgIC8qCisgICAgICAgICAgICAgKiBDaGFuZ2lu
ZyBtYXB0cmFja19saW1pdCBhbHRlcnMgbnJfbWFwdHJhY2tfZnJhbWVzKCkn
ZXMgcmV0dXJuCisgICAgICAgICAgICAgKiB2YWx1ZS4gRnJlZSB0aGUgdGhl
biBleGNlc3MgdHJhaWxpbmcgcGFnZSByaWdodCBoZXJlLCByYXRoZXIKKyAg
ICAgICAgICAgICAqIHRoYW4gbGVhdmluZyBpdCB0byBncmFudF90YWJsZV9k
ZXN0cm95KCkgKGFuZCBpbiB0dXJuIHJlcXVpcmluZworICAgICAgICAgICAg
ICogdG8gbGVhdmUgZ3QtPm1hcHRyYWNrX2xpbWl0IHVuYWx0ZXJlZCkuCisg
ICAgICAgICAgICAgKi8KKyAgICAgICAgICAgIGd0LT5tYXB0cmFja19saW1p
dCA9IGhhbmRsZTsKKyAgICAgICAgICAgIEZSRUVfWEVOSEVBUF9QQUdFKGd0
LT5tYXB0cmFja1tucl9tYXB0cmFja19mcmFtZXMoZ3QpXSk7CisKKyAgICAg
ICAgICAgIGlmICggaHlwZXJjYWxsX3ByZWVtcHRfY2hlY2soKSApCisgICAg
ICAgICAgICAgICAgcmV0dXJuIC1FUkVTVEFSVDsKKyAgICAgICAgfQorCisg
ICAgICAgIC0taGFuZGxlOworCiAgICAgICAgIG1hcCA9ICZtYXB0cmFja19l
bnRyeShndCwgaGFuZGxlKTsKICAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3Mg
JiAoR05UTUFQX2RldmljZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgKQogICAg
ICAgICAgICAgY29udGludWU7CkBAIC0zNzIzLDYgKzM3NTEsMTEgQEAgZ250
dGFiX3JlbGVhc2VfbWFwcGluZ3MoCiAKICAgICAgICAgbWFwLT5mbGFncyA9
IDA7CiAgICAgfQorCisgICAgZ3QtPm1hcHRyYWNrX2xpbWl0ID0gMDsKKyAg
ICBGUkVFX1hFTkhFQVBfUEFHRShndC0+bWFwdHJhY2tbMF0pOworCisgICAg
cmV0dXJuIDA7CiB9CiAKIHZvaWQgZ3JhbnRfdGFibGVfd2Fybl9hY3RpdmVf
Z3JhbnRzKHN0cnVjdCBkb21haW4gKmQpCkBAIC0zNzg1LDggKzM4MTgsNyBA
QCBncmFudF90YWJsZV9kZXN0cm95KAogICAgICAgICBmcmVlX3hlbmhlYXBf
cGFnZSh0LT5zaGFyZWRfcmF3W2ldKTsKICAgICB4ZnJlZSh0LT5zaGFyZWRf
cmF3KTsKIAotICAgIGZvciAoIGkgPSAwOyBpIDwgbnJfbWFwdHJhY2tfZnJh
bWVzKHQpOyBpKysgKQotICAgICAgICBmcmVlX3hlbmhlYXBfcGFnZSh0LT5t
YXB0cmFja1tpXSk7CisgICAgQVNTRVJUKCF0LT5tYXB0cmFja19saW1pdCk7
CiAgICAgdmZyZWUodC0+bWFwdHJhY2spOwogCiAgICAgZm9yICggaSA9IDA7
IGkgPCBucl9hY3RpdmVfZ3JhbnRfZnJhbWVzKHQpOyBpKysgKQotLS0gYS94
ZW4vaW5jbHVkZS94ZW4vZ3JhbnRfdGFibGUuaAorKysgYi94ZW4vaW5jbHVk
ZS94ZW4vZ3JhbnRfdGFibGUuaApAQCAtNDYsOSArNDYsNyBAQCBpbnQgZ3Jh
bnRfdGFibGVfc2V0X2xpbWl0cyhzdHJ1Y3QgZG9tYWluCiB2b2lkIGdyYW50
X3RhYmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKTsK
IAogLyogRG9tYWluIGRlYXRoIHJlbGVhc2Ugb2YgZ3JhbnRlZCBtYXBwaW5n
cyBvZiBvdGhlciBkb21haW5zJyBtZW1vcnkuICovCi12b2lkCi1nbnR0YWJf
cmVsZWFzZV9tYXBwaW5ncygKLSAgICBzdHJ1Y3QgZG9tYWluICpkKTsKK2lu
dCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkKTsK
IAogaW50IG1lbV9zaGFyaW5nX2dyZWZfdG9fZ2ZuKHN0cnVjdCBncmFudF90
YWJsZSAqZ3QsIGdyYW50X3JlZl90IHJlZiwKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBnZm5fdCAqZ2ZuLCB1aW50MTZfdCAqc3RhdHVzKTsK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.11-2.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.11-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IHJlcGxhY2UgbWFwa2luZCgpCgptYXBraW5kKCkgZG9lc24n
dCBzY2FsZSB2ZXJ5IHdlbGwgd2l0aCBsYXJnZXIgbWFwdHJhY2sgZW50cnkg
Y291bnRzLAp1c2luZyBhIGJydXRlIGZvcmNlIGxpbmVhciBzZWFyY2ggdGhy
b3VnaCBhbGwgZW50cmllcywgd2l0aCB0aGUgb25seQpvcHRpb24gb2YgYW4g
ZWFybHkgbG9vcCBleGl0IGlmIGEgbWF0Y2hpbmcgd3JpdGFibGUgZW50cnkg
d2FzIGZvdW5kLgpJbnRyb2R1Y2UgYSByYWRpeCB0cmVlIGFsb25nc2lkZSB0
aGUgbWFpbiBtYXB0cmFjayB0YWJsZSwgdGh1cwphbGxvd2luZyBtdWNoIGZh
c3RlciBNRk4tYmFzZWQgbG9va3VwLiBUbyBhdm9pZCB0aGUgbmVlZCB0byBh
Y3R1YWxseQphbGxvY2F0ZSBzcGFjZSBmb3IgdGhlIGluZGl2aWR1YWwgbm9k
ZXMsIGVuY29kZSB0aGUgdHdvIGNvdW50ZXJzIGluIHRoZQpub2RlIHBvaW50
ZXJzIHRoZW1zZWx2ZXMsIHRodXMgbGltaXRpbmcgdGhlIG51bWJlciBvZiBw
ZXJtaXR0ZWQKc2ltdWx0YW5lb3VzIHIvbyBhbmQgci93IG1hcHBpbmdzIG9m
IHRoZSBzYW1lIE1GTiB0byAywrPCuS0xICg2NC1iaXQpIC8KMsK54oG1LTEg
KDMyLWJpdCkgZWFjaC4KClRvIGF2b2lkIGVuZm9yY2luZyBhbiB1bm5lY2Vz
c2FyaWx5IGxvdyBib3VuZCBvbiB0aGUgbnVtYmVyIG9mCnNpbXVsdGFuZW91
cyBtYXBwaW5ncyBvZiBhIHNpbmdsZSBNRk4sIGludHJvZHVjZQpyYWRpeF90
cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfSBwYXJhbGxlbGluZwpy
YWRpeF90cmVlX3tpbnRfdG9fcHRyLHB0cl90b19pbnR9LgoKQXMgYSBjb25z
ZXF1ZW5jZSBsb2NraW5nIGNoYW5nZXMgYXJlIGFsc28gYXBwbGljYWJsZTog
V2l0aCB0aGVyZSBubwpsb25nZXIgYmVpbmcgYW55IGluc3BlY3Rpb24gb2Yg
dGhlIHJlbW90ZSBkb21haW4ncyBhY3RpdmUgZW50cmllcywKdGhlcmUncyBh
bHNvIG5vIG5lZWQgYW55bW9yZSB0byBob2xkIHRoZSByZW1vdGUgZG9tYWlu
J3MgZ3JhbnQgdGFibGUKbG9jay4gQW5kIHNpbmNlIHdlJ3JlIG5vIGxvbmdl
ciBpdGVyYXRpbmcgb3ZlciB0aGUgbG9jYWwgZG9tYWluJ3MgbWFwCnRyYWNr
IHRhYmxlLCB0aGUgbG9jayBpbiBtYXBfZ3JhbnRfcmVmKCkgY2FuIGFsc28g
YmUgZHJvcHBlZCBiZWZvcmUgdGhlCm5ldyBtYXB0cmFjayBlbnRyeSBhY3R1
YWxseSBnZXRzIHBvcHVsYXRlZC4KCkFzIGEgbmljZSBzaWRlIGVmZmVjdCB0
aGlzIGFsc28gcmVkdWNlcyB0aGUgbnVtYmVyIG9mIElPTU1VIG9wZXJhdGlv
bnMKaW4gdW5tYXBfY29tbW9uKCk6IFByZXZpb3VzbHkgd2Ugd291bGQgaGF2
ZSAiZXN0YWJsaXNoZWQiIGEgcmVhZGFibGUKbWFwcGluZyB3aGVuZXZlciB3
ZSBkaWRuJ3QgZmluZCBhIHdyaXRhYmxlIGVudHJ5IGFueW1vcmUgKHlldCwg
b2YKY291cnNlLCBhdCBsZWFzdCBvbmUgcmVhZGFibGUgb25lKS4gQnV0IHdl
IG9ubHkgbmVlZCB0byBkbyB0aGlzIGlmIHdlCmFjdHVhbGx5IGRyb3BwZWQg
dGhlIGxhc3Qgd3JpdGFibGUgZW50cnksIG5vdCBpZiB0aGVyZSB3ZXJlIG5v
bmUgYWxyZWFkeQpiZWZvcmUuCgpUaGlzIGlzIHBhcnQgb2YgQ1ZFLTIwMjEt
Mjg2OTggLyBYU0EtMzgwLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTogSnVsaWVuIEdyYWxs
IDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRf
dGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTM2
LDYgKzM2LDcgQEAKICNpbmNsdWRlIDx4ZW4vaW9tbXUuaD4KICNpbmNsdWRl
IDx4ZW4vcGFnaW5nLmg+CiAjaW5jbHVkZSA8eGVuL2tleWhhbmRsZXIuaD4K
KyNpbmNsdWRlIDx4ZW4vcmFkaXgtdHJlZS5oPgogI2luY2x1ZGUgPHhlbi92
bWFwLmg+CiAjaW5jbHVkZSA8eHNtL3hzbS5oPgogI2luY2x1ZGUgPGFzbS9m
bHVzaHRsYi5oPgpAQCAtODAsOCArODEsMTMgQEAgc3RydWN0IGdyYW50X3Rh
YmxlIHsKICAgICBncmFudF9zdGF0dXNfdCAgICAgICAqKnN0YXR1czsKICAg
ICAvKiBBY3RpdmUgZ3JhbnQgdGFibGUuICovCiAgICAgc3RydWN0IGFjdGl2
ZV9ncmFudF9lbnRyeSAqKmFjdGl2ZTsKLSAgICAvKiBNYXBwaW5nIHRyYWNr
aW5nIHRhYmxlIHBlciB2Y3B1LiAqLworICAgIC8qIEhhbmRsZS1pbmRleGVk
IHRyYWNraW5nIHRhYmxlIG9mIG1hcHBpbmdzLiAqLwogICAgIHN0cnVjdCBn
cmFudF9tYXBwaW5nICoqbWFwdHJhY2s7CisgICAgLyoKKyAgICAgKiBNRk4t
aW5kZXhlZCB0cmFja2luZyB0cmVlIG9mIG1hcHBpbmdzLCBpZiBuZWVkZWQu
ICBOb3RlIHRoYXQgdGhpcyBpcworICAgICAqIHByb3RlY3RlZCBieSBAbG9j
aywgbm90IEBtYXB0cmFja19sb2NrLgorICAgICAqLworICAgIHN0cnVjdCBy
YWRpeF90cmVlX3Jvb3QgbWFwdHJhY2tfdHJlZTsKIAogICAgIC8qIERvbWFp
biB0byB3aGljaCB0aGlzIHN0cnVjdCBncmFudF90YWJsZSBiZWxvbmdzLiAq
LwogICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKmRvbWFpbjsKQEAgLTQyMSwz
NCArNDI3LDYgQEAgc3RhdGljIGludCBnZXRfcGFnZWRfZnJhbWUodW5zaWdu
ZWQgbG9uZwogICAgIHJldHVybiByYzsKIH0KIAotc3RhdGljIGlubGluZSB2
b2lkCi1kb3VibGVfZ3RfbG9jayhzdHJ1Y3QgZ3JhbnRfdGFibGUgKmxndCwg
c3RydWN0IGdyYW50X3RhYmxlICpyZ3QpCi17Ci0gICAgLyoKLSAgICAgKiBT
ZWUgbWFwa2luZCgpIGZvciB3aHkgdGhlIHdyaXRlIGxvY2sgaXMgYWxzbyBy
ZXF1aXJlZCBmb3IgdGhlCi0gICAgICogcmVtb3RlIGRvbWFpbi4KLSAgICAg
Ki8KLSAgICBpZiAoIGxndCA8IHJndCApCi0gICAgewotICAgICAgICBncmFu
dF93cml0ZV9sb2NrKGxndCk7Ci0gICAgICAgIGdyYW50X3dyaXRlX2xvY2so
cmd0KTsKLSAgICB9Ci0gICAgZWxzZQotICAgIHsKLSAgICAgICAgaWYgKCBs
Z3QgIT0gcmd0ICkKLSAgICAgICAgICAgIGdyYW50X3dyaXRlX2xvY2socmd0
KTsKLSAgICAgICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOwotICAgIH0KLX0K
LQotc3RhdGljIGlubGluZSB2b2lkCi1kb3VibGVfZ3RfdW5sb2NrKHN0cnVj
dCBncmFudF90YWJsZSAqbGd0LCBzdHJ1Y3QgZ3JhbnRfdGFibGUgKnJndCkK
LXsKLSAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKLSAgICBpZiAoIGxn
dCAhPSByZ3QgKQotICAgICAgICBncmFudF93cml0ZV91bmxvY2socmd0KTsK
LX0KLQogI2RlZmluZSBJTlZBTElEX01BUFRSQUNLX0hBTkRMRSBVSU5UX01B
WAogCiBzdGF0aWMgaW5saW5lIGdyYW50X2hhbmRsZV90CkBAIC04NzEsNDEg
Kzg0OSwxNyBAQCBzdGF0aWMgc3RydWN0IGFjdGl2ZV9ncmFudF9lbnRyeSAq
Z3JhbnRfCiAgICAgcmV0dXJuIEVSUl9QVFIoLUVJTlZBTCk7CiB9CiAKLSNk
ZWZpbmUgTUFQS0lORF9SRUFEIDEKLSNkZWZpbmUgTUFQS0lORF9XUklURSAy
Ci1zdGF0aWMgdW5zaWduZWQgaW50IG1hcGtpbmQoCi0gICAgc3RydWN0IGdy
YW50X3RhYmxlICpsZ3QsIGNvbnN0IHN0cnVjdCBkb21haW4gKnJkLCBtZm5f
dCBtZm4pCi17Ci0gICAgc3RydWN0IGdyYW50X21hcHBpbmcgKm1hcDsKLSAg
ICBncmFudF9oYW5kbGVfdCBoYW5kbGUsIGxpbWl0ID0gbGd0LT5tYXB0cmFj
a19saW1pdDsKLSAgICB1bnNpZ25lZCBpbnQga2luZCA9IDA7Ci0KLSAgICAv
KgotICAgICAqIE11c3QgaGF2ZSB0aGUgbG9jYWwgZG9tYWluJ3MgZ3JhbnQg
dGFibGUgd3JpdGUgbG9jayB3aGVuCi0gICAgICogaXRlcmF0aW5nIG92ZXIg
aXRzIG1hcHRyYWNrIGVudHJpZXMuCi0gICAgICovCi0gICAgQVNTRVJUKHBl
cmNwdV9yd19pc193cml0ZV9sb2NrZWQoJmxndC0+bG9jaykpOwotICAgIC8q
Ci0gICAgICogTXVzdCBoYXZlIHRoZSByZW1vdGUgZG9tYWluJ3MgZ3JhbnQg
dGFibGUgd3JpdGUgbG9jayB3aGlsZQotICAgICAqIGNvdW50aW5nIGl0cyBh
Y3RpdmUgZW50cmllcy4KLSAgICAgKi8KLSAgICBBU1NFUlQocGVyY3B1X3J3
X2lzX3dyaXRlX2xvY2tlZCgmcmQtPmdyYW50X3RhYmxlLT5sb2NrKSk7Ci0K
LSAgICBzbXBfcm1iKCk7Ci0KLSAgICBmb3IgKCBoYW5kbGUgPSAwOyAhKGtp
bmQgJiBNQVBLSU5EX1dSSVRFKSAmJiBoYW5kbGUgPCBsaW1pdDsgaGFuZGxl
KysgKQotICAgIHsKLSAgICAgICAgbWFwID0gJm1hcHRyYWNrX2VudHJ5KGxn
dCwgaGFuZGxlKTsKLSAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3MgJiAoR05U
TUFQX2RldmljZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgfHwKLSAgICAgICAg
ICAgICBtYXAtPmRvbWlkICE9IHJkLT5kb21haW5faWQgKQotICAgICAgICAg
ICAgY29udGludWU7Ci0gICAgICAgIGlmICggbWZuX2VxKF9hY3RpdmVfZW50
cnkocmQtPmdyYW50X3RhYmxlLCBtYXAtPnJlZikuZnJhbWUsIG1mbikgKQot
ICAgICAgICAgICAga2luZCB8PSBtYXAtPmZsYWdzICYgR05UTUFQX3JlYWRv
bmx5ID8KLSAgICAgICAgICAgICAgICAgICAgTUFQS0lORF9SRUFEIDogTUFQ
S0lORF9XUklURTsKLSAgICB9Ci0KLSAgICByZXR1cm4ga2luZDsKLX0KK3Vu
aW9uIG1hcHRyYWNrX25vZGUgeworICAgIHN0cnVjdCB7CisgICAgICAgIC8q
IFJhZGl4IHRyZWUgc2xvdCBwb2ludGVycyB1c2UgdHdvIG9mIHRoZSBiaXRz
LiAqLworI2lmZGVmIF9fQklHX0VORElBTl9CSVRGSUVMRAorICAgICAgICB1
bnNpZ25lZCBsb25nICAgIDogMjsKKyNlbmRpZgorICAgICAgICB1bnNpZ25l
ZCBsb25nIHJkIDogQklUU19QRVJfTE9ORyAvIDIgLSAxOworICAgICAgICB1
bnNpZ25lZCBsb25nIHdyIDogQklUU19QRVJfTE9ORyAvIDIgLSAxOworICAg
IH0gY250OworICAgIHVuc2lnbmVkIGxvbmcgcmF3OworfTsKIAogLyoKICAq
IFJldHVybnMgMCBpZiBUTEIgZmx1c2ggLyBpbnZhbGlkYXRlIHJlcXVpcmVk
IGJ5IGNhbGxlci4KQEAgLTkzMSw3ICs4ODUsNiBAQCBtYXBfZ3JhbnRfcmVm
KAogICAgIHN0cnVjdCBncmFudF9tYXBwaW5nICptdDsKICAgICBncmFudF9l
bnRyeV9oZWFkZXJfdCAqc2hhaDsKICAgICB1aW50MTZfdCAqc3RhdHVzOwot
ICAgIGJvb2xfdCBuZWVkX2lvbW11OwogCiAgICAgbGVkID0gY3VycmVudDsK
ICAgICBsZCA9IGxlZC0+ZG9tYWluOwpAQCAtMTEzOSwzMSArMTA5Miw3NSBA
QCBtYXBfZ3JhbnRfcmVmKAogICAgICAgICBnb3RvIHVuZG9fb3V0OwogICAg
IH0KIAotICAgIG5lZWRfaW9tbXUgPSBnbnR0YWJfbmVlZF9pb21tdV9tYXBw
aW5nKGxkKTsKLSAgICBpZiAoIG5lZWRfaW9tbXUgKQorICAgIGlmICggZ250
dGFiX25lZWRfaW9tbXVfbWFwcGluZyhsZCkgKQogICAgIHsKKyAgICAgICAg
dW5pb24gbWFwdHJhY2tfbm9kZSBub2RlID0geworICAgICAgICAgICAgLmNu
dC5yZCA9ICEhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSksCisgICAg
ICAgICAgICAuY250LndyID0gIShvcC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9u
bHkpLAorICAgICAgICB9OworICAgICAgICBpbnQgZXJyOworICAgICAgICB2
b2lkICoqc2xvdCA9IE5VTEw7CiAgICAgICAgIHVuc2lnbmVkIGludCBraW5k
OwogCi0gICAgICAgIGRvdWJsZV9ndF9sb2NrKGxndCwgcmd0KTsKKyAgICAg
ICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOworCisgICAgICAgIGVyciA9IHJh
ZGl4X3RyZWVfaW5zZXJ0KCZsZ3QtPm1hcHRyYWNrX3RyZWUsIG1mbl94KGZy
YW1lKSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmFkaXhf
dHJlZV91bG9uZ190b19wdHIobm9kZS5yYXcpKTsKKyAgICAgICAgaWYgKCBl
cnIgPT0gLUVFWElTVCApCisgICAgICAgIHsKKyAgICAgICAgICAgIHNsb3Qg
PSByYWRpeF90cmVlX2xvb2t1cF9zbG90KCZsZ3QtPm1hcHRyYWNrX3RyZWUs
IG1mbl94KGZyYW1lKSk7CisgICAgICAgICAgICBpZiAoIGxpa2VseShzbG90
KSApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgbm9kZS5yYXcg
PSByYWRpeF90cmVlX3B0cl90b191bG9uZygqc2xvdCk7CisgICAgICAgICAg
ICAgICAgZXJyID0gLUVCVVNZOworCisgICAgICAgICAgICAgICAgLyogVXBk
YXRlIG5vZGUgb25seSB3aGVuIHJlZmNvdW50IGRvZXNuJ3Qgb3ZlcmZsb3cu
ICovCisgICAgICAgICAgICAgICAgaWYgKCBvcC0+ZmxhZ3MgJiBHTlRNQVBf
cmVhZG9ubHkgPyArK25vZGUuY250LnJkCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiArK25vZGUuY250Lndy
ICkKKyAgICAgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgICAgIHJh
ZGl4X3RyZWVfcmVwbGFjZV9zbG90KHNsb3QsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVfdWxvbmdf
dG9fcHRyKG5vZGUucmF3KSk7CisgICAgICAgICAgICAgICAgICAgIGVyciA9
IDA7CisgICAgICAgICAgICAgICAgfQorICAgICAgICAgICAgfQorICAgICAg
ICAgICAgZWxzZQorICAgICAgICAgICAgICAgIEFTU0VSVF9VTlJFQUNIQUJM
RSgpOworICAgICAgICB9CiAKICAgICAgICAgLyoKICAgICAgICAgICogV2Un
cmUgbm90IHRyYW5zbGF0ZWQsIHNvIHdlIGtub3cgdGhhdCBkZm5zIGFuZCBt
Zm5zIGFyZQogICAgICAgICAgKiB0aGUgc2FtZSB0aGluZ3MsIHNvIHRoZSBJ
T01NVSBlbnRyeSBpcyBhbHdheXMgMS10by0xLgogICAgICAgICAgKi8KLSAg
ICAgICAga2luZCA9IG1hcGtpbmQobGd0LCByZCwgZnJhbWUpOwotICAgICAg
ICBpZiAoICEob3AtPmZsYWdzICYgR05UTUFQX3JlYWRvbmx5KSAmJgotICAg
ICAgICAgICAgICEoa2luZCAmIE1BUEtJTkRfV1JJVEUpICkKKyAgICAgICAg
aWYgKCAhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSkgJiYgbm9kZS5j
bnQud3IgPT0gMSApCiAgICAgICAgICAgICBraW5kID0gSU9NTVVGX3JlYWRh
YmxlIHwgSU9NTVVGX3dyaXRhYmxlOwotICAgICAgICBlbHNlIGlmICggIWtp
bmQgKQorICAgICAgICBlbHNlIGlmICggKG9wLT5mbGFncyAmIEdOVE1BUF9y
ZWFkb25seSkgJiYKKyAgICAgICAgICAgICAgICAgIG5vZGUuY250LnJkID09
IDEgJiYgIW5vZGUuY250LndyICkKICAgICAgICAgICAgIGtpbmQgPSBJT01N
VUZfcmVhZGFibGU7CiAgICAgICAgIGVsc2UKICAgICAgICAgICAgIGtpbmQg
PSAwOwotICAgICAgICBpZiAoIGtpbmQgJiYgaW9tbXVfbWFwX3BhZ2UobGQs
IG1mbl94KGZyYW1lKSwgbWZuX3goZnJhbWUpLCBraW5kKSApCisgICAgICAg
IGlmICggZXJyIHx8CisgICAgICAgICAgICAgKGtpbmQgJiYgaW9tbXVfbWFw
X3BhZ2UobGQsIG1mbl94KGZyYW1lKSwgbWZuX3goZnJhbWUpLCBraW5kKSkg
KQogICAgICAgICB7Ci0gICAgICAgICAgICBkb3VibGVfZ3RfdW5sb2NrKGxn
dCwgcmd0KTsKKyAgICAgICAgICAgIGlmICggIWVyciApCisgICAgICAgICAg
ICB7CisgICAgICAgICAgICAgICAgaWYgKCBzbG90ICkKKyAgICAgICAgICAg
ICAgICB7CisgICAgICAgICAgICAgICAgICAgIG9wLT5mbGFncyAmIEdOVE1B
UF9yZWFkb25seSA/IG5vZGUuY250LnJkLS0KKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDogbm9kZS5jbnQud3It
LTsKKyAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJlZV9yZXBsYWNlX3Ns
b3Qoc2xvdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgcmFkaXhfdHJlZV91bG9uZ190b19wdHIobm9kZS5yYXcpKTsK
KyAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICAgICAgZWxzZQorICAg
ICAgICAgICAgICAgICAgICByYWRpeF90cmVlX2RlbGV0ZSgmbGd0LT5tYXB0
cmFja190cmVlLCBtZm5feChmcmFtZSkpOworICAgICAgICAgICAgfQorCiAg
ICAgICAgICAgICByYyA9IEdOVFNUX2dlbmVyYWxfZXJyb3I7Ci0gICAgICAg
ICAgICBnb3RvIHVuZG9fb3V0OwogICAgICAgICB9CisKKyAgICAgICAgZ3Jh
bnRfd3JpdGVfdW5sb2NrKGxndCk7CisKKyAgICAgICAgaWYgKCByYyAhPSBH
TlRTVF9va2F5ICkKKyAgICAgICAgICAgIGdvdG8gdW5kb19vdXQ7CiAgICAg
fQogCiAgICAgVFJBQ0VfMUQoVFJDX01FTV9QQUdFX0dSQU5UX01BUCwgb3At
PmRvbSk7CkBAIC0xMTcxLDEwICsxMTY4LDYgQEAgbWFwX2dyYW50X3JlZigK
ICAgICAvKgogICAgICAqIEFsbCBtYXB0cmFjayBlbnRyeSB1c2VycyBjaGVj
ayBtdC0+ZmxhZ3MgZmlyc3QgYmVmb3JlIHVzaW5nIHRoZQogICAgICAqIG90
aGVyIGZpZWxkcyBzbyBqdXN0IGVuc3VyZSB0aGUgZmxhZ3MgZmllbGQgaXMg
c3RvcmVkIGxhc3QuCi0gICAgICoKLSAgICAgKiBIb3dldmVyLCBpZiBnbnR0
YWJfbmVlZF9pb21tdV9tYXBwaW5nKCkgdGhlbiB0aGlzIHdvdWxkIHJhY2UK
LSAgICAgKiB3aXRoIGEgY29uY3VycmVudCBtYXBraW5kKCkgY2FsbCAob24g
YW4gdW5tYXAsIGZvciBleGFtcGxlKQotICAgICAqIGFuZCBhIGxvY2sgaXMg
cmVxdWlyZWQuCiAgICAgICovCiAgICAgbXQgPSAmbWFwdHJhY2tfZW50cnko
bGd0LCBoYW5kbGUpOwogICAgIG10LT5kb21pZCA9IG9wLT5kb207CkBAIC0x
MTgyLDkgKzExNzUsNiBAQCBtYXBfZ3JhbnRfcmVmKAogICAgIHNtcF93bWIo
KTsKICAgICB3cml0ZV9hdG9taWMoJm10LT5mbGFncywgb3AtPmZsYWdzKTsK
IAotICAgIGlmICggbmVlZF9pb21tdSApCi0gICAgICAgIGRvdWJsZV9ndF91
bmxvY2sobGd0LCByZ3QpOwotCiAgICAgb3AtPmRldl9idXNfYWRkciA9IG1m
bl90b19tYWRkcihmcmFtZSk7CiAgICAgb3AtPmhhbmRsZSAgICAgICA9IGhh
bmRsZTsKICAgICBvcC0+c3RhdHVzICAgICAgID0gR05UU1Rfb2theTsKQEAg
LTE0MTEsMTkgKzE0MDEsMzQgQEAgdW5tYXBfY29tbW9uKAogCiAgICAgaWYg
KCByYyA9PSBHTlRTVF9va2F5ICYmIGdudHRhYl9uZWVkX2lvbW11X21hcHBp
bmcobGQpICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGludCBraW5kOwor
ICAgICAgICB2b2lkICoqc2xvdDsKKyAgICAgICAgdW5pb24gbWFwdHJhY2tf
bm9kZSBub2RlOwogICAgICAgICBpbnQgZXJyID0gMDsKIAotICAgICAgICBk
b3VibGVfZ3RfbG9jayhsZ3QsIHJndCk7CisgICAgICAgIGdyYW50X3dyaXRl
X2xvY2sobGd0KTsKKyAgICAgICAgc2xvdCA9IHJhZGl4X3RyZWVfbG9va3Vw
X3Nsb3QoJmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gob3AtPmZyYW1lKSk7
CisgICAgICAgIG5vZGUucmF3ID0gbGlrZWx5KHNsb3QpID8gcmFkaXhfdHJl
ZV9wdHJfdG9fdWxvbmcoKnNsb3QpIDogMDsKKworICAgICAgICAvKiBSZWZj
b3VudCBtdXN0IG5vdCB1bmRlcmZsb3cuICovCisgICAgICAgIGlmICggIShm
bGFncyAmIEdOVE1BUF9yZWFkb25seSA/IG5vZGUuY250LnJkLS0KKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDogbm9kZS5jbnQu
d3ItLSkgKQorICAgICAgICAgICAgQlVHKCk7CiAKLSAgICAgICAga2luZCA9
IG1hcGtpbmQobGd0LCByZCwgb3AtPmZyYW1lKTsKLSAgICAgICAgaWYgKCAh
a2luZCApCisgICAgICAgIGlmICggIW5vZGUucmF3ICkKICAgICAgICAgICAg
IGVyciA9IGlvbW11X3VubWFwX3BhZ2UobGQsIG1mbl94KG9wLT5mcmFtZSkp
OwotICAgICAgICBlbHNlIGlmICggIShraW5kICYgTUFQS0lORF9XUklURSkg
KQorICAgICAgICBlbHNlIGlmICggIShmbGFncyAmIEdOVE1BUF9yZWFkb25s
eSkgJiYgIW5vZGUuY250LndyICkKICAgICAgICAgICAgIGVyciA9IGlvbW11
X21hcF9wYWdlKGxkLCBtZm5feChvcC0+ZnJhbWUpLAogICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgbWZuX3gob3AtPmZyYW1lKSwgSU9NTVVG
X3JlYWRhYmxlKTsKIAotICAgICAgICBkb3VibGVfZ3RfdW5sb2NrKGxndCwg
cmd0KTsKKyAgICAgICAgaWYgKCBlcnIgKQorICAgICAgICAgICAgOworICAg
ICAgICBlbHNlIGlmICggIW5vZGUucmF3ICkKKyAgICAgICAgICAgIHJhZGl4
X3RyZWVfZGVsZXRlKCZsZ3QtPm1hcHRyYWNrX3RyZWUsIG1mbl94KG9wLT5m
cmFtZSkpOworICAgICAgICBlbHNlCisgICAgICAgICAgICByYWRpeF90cmVl
X3JlcGxhY2Vfc2xvdChzbG90LAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgcmFkaXhfdHJlZV91bG9uZ190b19wdHIobm9kZS5yYXcp
KTsKKworICAgICAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKIAogICAg
ICAgICBpZiAoIGVyciApCiAgICAgICAgICAgICByYyA9IEdOVFNUX2dlbmVy
YWxfZXJyb3I7CkBAIC0xODU0LDYgKzE4NTksOCBAQCBncmFudF90YWJsZV9p
bml0KHN0cnVjdCBkb21haW4gKmQsIHN0cnVjCiAgICAgICAgIGd0LT5tYXB0
cmFjayA9IHZ6YWxsb2MoZ3QtPm1heF9tYXB0cmFja19mcmFtZXMgKiBzaXpl
b2YoKmd0LT5tYXB0cmFjaykpOwogICAgICAgICBpZiAoIGd0LT5tYXB0cmFj
ayA9PSBOVUxMICkKICAgICAgICAgICAgIGdvdG8gb3V0OworCisgICAgICAg
IHJhZGl4X3RyZWVfaW5pdCgmZ3QtPm1hcHRyYWNrX3RyZWUpOwogICAgIH0K
IAogICAgIC8qIFNoYXJlZCBncmFudCB0YWJsZS4gKi8KQEAgLTM2NDMsNiAr
MzY1MCw4IEBAIGludCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3Qg
ZG9tYWkKIAogICAgIGZvciAoIGhhbmRsZSA9IGd0LT5tYXB0cmFja19saW1p
dDsgaGFuZGxlOyApCiAgICAgeworICAgICAgICBtZm5fdCBtZm47CisKICAg
ICAgICAgLyoKICAgICAgICAgICogRGVhbCB3aXRoIGZ1bGwgcGFnZXMgc3Vj
aCB0aGF0IHRoZWlyIGZyZWVpbmcgKGluIHRoZSBib2R5IG9mIHRoZQogICAg
ICAgICAgKiBpZigpKSByZW1haW5zIHNpbXBsZS4KQEAgLTM3NDQsMTcgKzM3
NTMsMzEgQEAgaW50IGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKHN0cnVjdCBk
b21haQogICAgICAgICBpZiAoIGFjdC0+cGluID09IDAgKQogICAgICAgICAg
ICAgZ250dGFiX2NsZWFyX2ZsYWcocmQsIF9HVEZfcmVhZGluZywgc3RhdHVz
KTsKIAorICAgICAgICBtZm4gPSBhY3QtPmZyYW1lOworCiAgICAgICAgIGFj
dGl2ZV9lbnRyeV9yZWxlYXNlKGFjdCk7CiAgICAgICAgIGdyYW50X3JlYWRf
dW5sb2NrKHJndCk7CiAKICAgICAgICAgcmN1X3VubG9ja19kb21haW4ocmQp
OwogCiAgICAgICAgIG1hcC0+ZmxhZ3MgPSAwOworCisgICAgICAgIC8qCisg
ICAgICAgICAqIFRoaXMgaXMgZXhjZXNzaXZlIGluIHRoYXQgYSBzaW5nbGUg
c3VjaCBjYWxsIHdvdWxkIHN1ZmZpY2UgcGVyCisgICAgICAgICAqIG1hcHBl
ZCBNRk4gKG9yIG5vbmUgYXQgYWxsLCBpZiBubyBlbnRyeSB3YXMgZXZlciBp
bnNlcnRlZCkuIEJ1dCBpdAorICAgICAgICAgKiBzaG91bGQgYmUgdGhlIGNv
bW1vbiBjYXNlIGZvciBhbiBNRk4gdG8gYmUgbWFwcGVkIGp1c3Qgb25jZSwg
YW5kCisgICAgICAgICAqIHRoaXMgd2F5IHdlIGRvbid0IG5lZWQgdG8gZnVy
dGhlciBtYWludGFpbiB0aGUgY291bnRlcnMuIFdlIGFsc28KKyAgICAgICAg
ICogZG9uJ3Qgd2FudCB0byBsZWF2ZSBjbGVhbmluZyB1cCBvZiB0aGUgdHJl
ZSBhcyBhIHdob2xlIHRvIHRoZSBlbmQKKyAgICAgICAgICogb2YgdGhlIGZ1
bmN0aW9uLCBhcyB0aGlzIGNvdWxkIHRha2UgcXVpdGUgc29tZSB0aW1lLgor
ICAgICAgICAgKi8KKyAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUoJmd0LT5t
YXB0cmFja190cmVlLCBtZm5feChtZm4pKTsKICAgICB9CiAKICAgICBndC0+
bWFwdHJhY2tfbGltaXQgPSAwOwogICAgIEZSRUVfWEVOSEVBUF9QQUdFKGd0
LT5tYXB0cmFja1swXSk7CiAKKyAgICByYWRpeF90cmVlX2Rlc3Ryb3koJmd0
LT5tYXB0cmFja190cmVlLCBOVUxMKTsKKwogICAgIHJldHVybiAwOwogfQog
Ci0tLSBhL3hlbi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmgKKysrIGIveGVu
L2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaApAQCAtMTkwLDYgKzE5MCwyNSBA
QCBzdGF0aWMgaW5saW5lIGludCByYWRpeF90cmVlX3B0cl90b19pbnQoCiAg
ICAgcmV0dXJuIChpbnQpKChsb25nKXB0ciA+PiAyKTsKIH0KIAorLyoqCisg
KiByYWRpeF90cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfToKKyAq
CisgKiBTYW1lIGZvciB1bnNpZ25lZCBsb25nIHZhbHVlcy4gQmV3YXJlIHRo
b3VnaCB0aGF0IG9ubHkgQklUU19QRVJfTE9ORy0yCisgKiBiaXRzIGFyZSBh
Y3R1YWxseSB1c2FibGUgZm9yIHRoZSB2YWx1ZS4KKyAqLworc3RhdGljIGlu
bGluZSB2b2lkICpyYWRpeF90cmVlX3Vsb25nX3RvX3B0cih1bnNpZ25lZCBs
b25nIHZhbCkKK3sKKyAgICB1bnNpZ25lZCBsb25nIHB0ciA9ICh2YWwgPDwg
MikgfCAweDI7CisgICAgQVNTRVJUKChwdHIgPj4gMikgPT0gdmFsKTsKKyAg
ICByZXR1cm4gKHZvaWQgKilwdHI7Cit9CisKK3N0YXRpYyBpbmxpbmUgdW5z
aWduZWQgbG9uZyByYWRpeF90cmVlX3B0cl90b191bG9uZyh2b2lkICpwdHIp
Cit7CisgICAgQVNTRVJUKCgodW5zaWduZWQgbG9uZylwdHIgJiAweDMpID09
IDB4Mik7CisgICAgcmV0dXJuICh1bnNpZ25lZCBsb25nKXB0ciA+PiAyOwor
fQorCiBpbnQgcmFkaXhfdHJlZV9pbnNlcnQoc3RydWN0IHJhZGl4X3RyZWVf
cm9vdCAqLCB1bnNpZ25lZCBsb25nLCB2b2lkICopOwogdm9pZCAqcmFkaXhf
dHJlZV9sb29rdXAoc3RydWN0IHJhZGl4X3RyZWVfcm9vdCAqLCB1bnNpZ25l
ZCBsb25nKTsKIHZvaWQgKipyYWRpeF90cmVlX2xvb2t1cF9zbG90KHN0cnVj
dCByYWRpeF90cmVlX3Jvb3QgKiwgdW5zaWduZWQgbG9uZyk7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.12-1.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.12-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGFkZCBwcmVlbXB0aW9uIGNoZWNrIHRvIGdudHRhYl9yZWxl
YXNlX21hcHBpbmdzKCkKCkEgZ3Vlc3QgbWF5IGRpZSB3aXRoIG1hbnkgZ3Jh
bnQgbWFwcGluZ3Mgc3RpbGwgaW4gcGxhY2UsIG9yIHNpbXBseSB3aXRoCmEg
bGFyZ2UgbWFwdHJhY2sgdGFibGUuIEl0ZXJhdGluZyB0aHJvdWdoIHRoaXMg
bWF5IHRha2UgbW9yZSB0aW1lIHRoYW4KaXMgcmVhc29uYWJsZSB3aXRob3V0
IGludGVybWVkaWF0ZSBwcmVlbXB0aW9uICh0byBydW4gc29mdGlycXMgYW5k
CnBlcmhhcHMgdGhlIHNjaGVkdWxlcikuCgpNb3ZlIHRoZSBpbnZvY2F0aW9u
IG9mIHRoZSBmdW5jdGlvbiB0byB0aGUgc2VjdGlvbiB3aGVyZSBvdGhlcgpy
ZXN0YXJ0YWJsZSBmdW5jdGlvbnMgZ2V0IGludm9rZWQsIGFuZCBoYXZlIHRo
ZSBmdW5jdGlvbiBpdHNlbGYgY2hlY2sKZm9yIHByZWVtcHRpb24gZXZlcnkg
b25jZSBpbiBhIHdoaWxlLiBIYXZlIGl0IGl0ZXJhdGUgdGhlIHRhYmxlCmJh
Y2t3YXJkcywgc3VjaCB0aGF0IGRlY3JlYXNpbmcgdGhlIG1hcHRyYWNrIGxp
bWl0IGlzIGFsbCBpdCB0YWtlcyB0bwpjb252ZXkgcmVzdGFydCBpbmZvcm1h
dGlvbi4KCkluIGRvbWFpbl90ZWFyZG93bigpIGludHJvZHVjZSBQUk9HX25v
bmUgc3VjaCB0aGF0IGluc2VydGluZyBhdCB0aGUKZnJvbnQgd2lsbCBiZSBl
YXNpZXIgZ29pbmcgZm9yd2FyZC4KClRoaXMgaXMgcGFydCBvZiBDVkUtMjAy
MS0yODY5OCAvIFhTQS0zODAuCgpSZXBvcnRlZC1ieTogQW5kcmV3IENvb3Bl
ciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
SnVsaWVuIEdyYWxsIDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9j
b21tb24vZG9tYWluLmMKKysrIGIveGVuL2NvbW1vbi9kb21haW4uYwpAQCAt
NzI0LDEzICs3MjQsMTUgQEAgaW50IGRvbWFpbl9raWxsKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgICAgICAgICByZXR1cm4gZG9tYWluX2tpbGwoZCk7CiAg
ICAgICAgIGQtPmlzX2R5aW5nID0gRE9NRFlJTkdfZHlpbmc7CiAgICAgICAg
IGFyZ29fZGVzdHJveShkKTsKLSAgICAgICAgZ250dGFiX3JlbGVhc2VfbWFw
cGluZ3MoZCk7CiAgICAgICAgIHRtZW1fZGVzdHJveShkLT50bWVtX2NsaWVu
dCk7CiAgICAgICAgIHZudW1hX2Rlc3Ryb3koZC0+dm51bWEpOwogICAgICAg
ICBkb21haW5fc2V0X291dHN0YW5kaW5nX3BhZ2VzKGQsIDApOwogICAgICAg
ICBkLT50bWVtX2NsaWVudCA9IE5VTEw7CiAgICAgICAgIC8qIGZhbGx0aHJv
dWdoICovCiAgICAgY2FzZSBET01EWUlOR19keWluZzoKKyAgICAgICAgcmMg
PSBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhkKTsKKyAgICAgICAgaWYgKCBy
YyApCisgICAgICAgICAgICBicmVhazsKICAgICAgICAgcmMgPSBldnRjaG5f
ZGVzdHJveShkKTsKICAgICAgICAgaWYgKCByYyApCiAgICAgICAgICAgICBi
cmVhazsKLS0tIGEveGVuL2NvbW1vbi9ncmFudF90YWJsZS5jCisrKyBiL3hl
bi9jb21tb24vZ3JhbnRfdGFibGUuYwpAQCAtNjIsNyArNjIsMTMgQEAgc3Ry
dWN0IGdyYW50X3RhYmxlIHsKICAgICB1bnNpZ25lZCBpbnQgICAgICAgICAg
bnJfZ3JhbnRfZnJhbWVzOwogICAgIC8qIE51bWJlciBvZiBncmFudCBzdGF0
dXMgZnJhbWVzIHNoYXJlZCB3aXRoIGd1ZXN0IChmb3IgdmVyc2lvbiAyKSAq
LwogICAgIHVuc2lnbmVkIGludCAgICAgICAgICBucl9zdGF0dXNfZnJhbWVz
OwotICAgIC8qIE51bWJlciBvZiBhdmFpbGFibGUgbWFwdHJhY2sgZW50cmll
cy4gKi8KKyAgICAvKgorICAgICAqIE51bWJlciBvZiBhdmFpbGFibGUgbWFw
dHJhY2sgZW50cmllcy4gIEZvciBjbGVhbnVwIHB1cnBvc2VzIGl0IGlzCisg
ICAgICogaW1wb3J0YW50IHRvIHJlYWxpemUgdGhhdCB0aGlzIGZpZWxkIGFu
ZCBAbWFwdHJhY2sgZnVydGhlciBkb3duIHdpbGwKKyAgICAgKiBvbmx5IGV2
ZXIgYmUgYWNjZXNzZWQgYnkgdGhlIGxvY2FsIGRvbWFpbi4gIFRodXMgaXQg
aXMgb2theSB0byBjbGVhbgorICAgICAqIHVwIGVhcmx5LCBhbmQgdG8gc2hy
aW5rIHRoZSBsaW1pdCBmb3IgdGhlIHB1cnBvc2Ugb2YgdHJhY2tpbmcgY2xl
YW51cAorICAgICAqIHByb2dyZXNzLgorICAgICAqLwogICAgIHVuc2lnbmVk
IGludCAgICAgICAgICBtYXB0cmFja19saW1pdDsKICAgICAvKiBTaGFyZWQg
Z3JhbnQgdGFibGUgKHNlZSBpbmNsdWRlL3B1YmxpYy9ncmFudF90YWJsZS5o
KS4gKi8KICAgICB1bmlvbiB7CkBAIC0zNjA0LDkgKzM2MTAsNyBAQCBkb19n
cmFudF90YWJsZV9vcCgKICNpbmNsdWRlICJjb21wYXQvZ3JhbnRfdGFibGUu
YyIKICNlbmRpZgogCi12b2lkCi1nbnR0YWJfcmVsZWFzZV9tYXBwaW5ncygK
LSAgICBzdHJ1Y3QgZG9tYWluICpkKQoraW50IGdudHRhYl9yZWxlYXNlX21h
cHBpbmdzKHN0cnVjdCBkb21haW4gKmQpCiB7CiAgICAgc3RydWN0IGdyYW50
X3RhYmxlICAgKmd0ID0gZC0+Z3JhbnRfdGFibGUsICpyZ3Q7CiAgICAgc3Ry
dWN0IGdyYW50X21hcHBpbmcgKm1hcDsKQEAgLTM2MjAsOCArMzYyNCwzMiBA
QCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncygKIAogICAgIEJVR19PTighZC0+
aXNfZHlpbmcpOwogCi0gICAgZm9yICggaGFuZGxlID0gMDsgaGFuZGxlIDwg
Z3QtPm1hcHRyYWNrX2xpbWl0OyBoYW5kbGUrKyApCisgICAgaWYgKCAhZ3Qg
fHwgIWd0LT5tYXB0cmFjayApCisgICAgICAgIHJldHVybiAwOworCisgICAg
Zm9yICggaGFuZGxlID0gZ3QtPm1hcHRyYWNrX2xpbWl0OyBoYW5kbGU7ICkK
ICAgICB7CisgICAgICAgIC8qCisgICAgICAgICAqIERlYWwgd2l0aCBmdWxs
IHBhZ2VzIHN1Y2ggdGhhdCB0aGVpciBmcmVlaW5nIChpbiB0aGUgYm9keSBv
ZiB0aGUKKyAgICAgICAgICogaWYoKSkgcmVtYWlucyBzaW1wbGUuCisgICAg
ICAgICAqLworICAgICAgICBpZiAoIGhhbmRsZSA8IGd0LT5tYXB0cmFja19s
aW1pdCAmJiAhKGhhbmRsZSAlIE1BUFRSQUNLX1BFUl9QQUdFKSApCisgICAg
ICAgIHsKKyAgICAgICAgICAgIC8qCisgICAgICAgICAgICAgKiBDaGFuZ2lu
ZyBtYXB0cmFja19saW1pdCBhbHRlcnMgbnJfbWFwdHJhY2tfZnJhbWVzKCkn
ZXMgcmV0dXJuCisgICAgICAgICAgICAgKiB2YWx1ZS4gRnJlZSB0aGUgdGhl
biBleGNlc3MgdHJhaWxpbmcgcGFnZSByaWdodCBoZXJlLCByYXRoZXIKKyAg
ICAgICAgICAgICAqIHRoYW4gbGVhdmluZyBpdCB0byBncmFudF90YWJsZV9k
ZXN0cm95KCkgKGFuZCBpbiB0dXJuIHJlcXVpcmluZworICAgICAgICAgICAg
ICogdG8gbGVhdmUgZ3QtPm1hcHRyYWNrX2xpbWl0IHVuYWx0ZXJlZCkuCisg
ICAgICAgICAgICAgKi8KKyAgICAgICAgICAgIGd0LT5tYXB0cmFja19saW1p
dCA9IGhhbmRsZTsKKyAgICAgICAgICAgIEZSRUVfWEVOSEVBUF9QQUdFKGd0
LT5tYXB0cmFja1tucl9tYXB0cmFja19mcmFtZXMoZ3QpXSk7CisKKyAgICAg
ICAgICAgIGlmICggaHlwZXJjYWxsX3ByZWVtcHRfY2hlY2soKSApCisgICAg
ICAgICAgICAgICAgcmV0dXJuIC1FUkVTVEFSVDsKKyAgICAgICAgfQorCisg
ICAgICAgIC0taGFuZGxlOworCiAgICAgICAgIG1hcCA9ICZtYXB0cmFja19l
bnRyeShndCwgaGFuZGxlKTsKICAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3Mg
JiAoR05UTUFQX2RldmljZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgKQogICAg
ICAgICAgICAgY29udGludWU7CkBAIC0zNzA5LDYgKzM3MzcsMTEgQEAgZ250
dGFiX3JlbGVhc2VfbWFwcGluZ3MoCiAKICAgICAgICAgbWFwLT5mbGFncyA9
IDA7CiAgICAgfQorCisgICAgZ3QtPm1hcHRyYWNrX2xpbWl0ID0gMDsKKyAg
ICBGUkVFX1hFTkhFQVBfUEFHRShndC0+bWFwdHJhY2tbMF0pOworCisgICAg
cmV0dXJuIDA7CiB9CiAKIHZvaWQgZ3JhbnRfdGFibGVfd2Fybl9hY3RpdmVf
Z3JhbnRzKHN0cnVjdCBkb21haW4gKmQpCkBAIC0zNzcxLDggKzM4MDQsNyBA
QCBncmFudF90YWJsZV9kZXN0cm95KAogICAgICAgICBmcmVlX3hlbmhlYXBf
cGFnZSh0LT5zaGFyZWRfcmF3W2ldKTsKICAgICB4ZnJlZSh0LT5zaGFyZWRf
cmF3KTsKIAotICAgIGZvciAoIGkgPSAwOyBpIDwgbnJfbWFwdHJhY2tfZnJh
bWVzKHQpOyBpKysgKQotICAgICAgICBmcmVlX3hlbmhlYXBfcGFnZSh0LT5t
YXB0cmFja1tpXSk7CisgICAgQVNTRVJUKCF0LT5tYXB0cmFja19saW1pdCk7
CiAgICAgdmZyZWUodC0+bWFwdHJhY2spOwogCiAgICAgZm9yICggaSA9IDA7
IGkgPCBucl9hY3RpdmVfZ3JhbnRfZnJhbWVzKHQpOyBpKysgKQotLS0gYS94
ZW4vaW5jbHVkZS94ZW4vZ3JhbnRfdGFibGUuaAorKysgYi94ZW4vaW5jbHVk
ZS94ZW4vZ3JhbnRfdGFibGUuaApAQCAtNDYsOSArNDYsNyBAQCB2b2lkIGdy
YW50X3RhYmxlX2luaXRfdmNwdShzdHJ1Y3QgdmNwdSAqCiB2b2lkIGdyYW50
X3RhYmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKTsK
IAogLyogRG9tYWluIGRlYXRoIHJlbGVhc2Ugb2YgZ3JhbnRlZCBtYXBwaW5n
cyBvZiBvdGhlciBkb21haW5zJyBtZW1vcnkuICovCi12b2lkCi1nbnR0YWJf
cmVsZWFzZV9tYXBwaW5ncygKLSAgICBzdHJ1Y3QgZG9tYWluICpkKTsKK2lu
dCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkKTsK
IAogaW50IG1lbV9zaGFyaW5nX2dyZWZfdG9fZ2ZuKHN0cnVjdCBncmFudF90
YWJsZSAqZ3QsIGdyYW50X3JlZl90IHJlZiwKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBnZm5fdCAqZ2ZuLCB1aW50MTZfdCAqc3RhdHVzKTsK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.12-2.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.12-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IHJlcGxhY2UgbWFwa2luZCgpCgptYXBraW5kKCkgZG9lc24n
dCBzY2FsZSB2ZXJ5IHdlbGwgd2l0aCBsYXJnZXIgbWFwdHJhY2sgZW50cnkg
Y291bnRzLAp1c2luZyBhIGJydXRlIGZvcmNlIGxpbmVhciBzZWFyY2ggdGhy
b3VnaCBhbGwgZW50cmllcywgd2l0aCB0aGUgb25seQpvcHRpb24gb2YgYW4g
ZWFybHkgbG9vcCBleGl0IGlmIGEgbWF0Y2hpbmcgd3JpdGFibGUgZW50cnkg
d2FzIGZvdW5kLgpJbnRyb2R1Y2UgYSByYWRpeCB0cmVlIGFsb25nc2lkZSB0
aGUgbWFpbiBtYXB0cmFjayB0YWJsZSwgdGh1cwphbGxvd2luZyBtdWNoIGZh
c3RlciBNRk4tYmFzZWQgbG9va3VwLiBUbyBhdm9pZCB0aGUgbmVlZCB0byBh
Y3R1YWxseQphbGxvY2F0ZSBzcGFjZSBmb3IgdGhlIGluZGl2aWR1YWwgbm9k
ZXMsIGVuY29kZSB0aGUgdHdvIGNvdW50ZXJzIGluIHRoZQpub2RlIHBvaW50
ZXJzIHRoZW1zZWx2ZXMsIHRodXMgbGltaXRpbmcgdGhlIG51bWJlciBvZiBw
ZXJtaXR0ZWQKc2ltdWx0YW5lb3VzIHIvbyBhbmQgci93IG1hcHBpbmdzIG9m
IHRoZSBzYW1lIE1GTiB0byAywrPCuS0xICg2NC1iaXQpIC8KMsK54oG1LTEg
KDMyLWJpdCkgZWFjaC4KClRvIGF2b2lkIGVuZm9yY2luZyBhbiB1bm5lY2Vz
c2FyaWx5IGxvdyBib3VuZCBvbiB0aGUgbnVtYmVyIG9mCnNpbXVsdGFuZW91
cyBtYXBwaW5ncyBvZiBhIHNpbmdsZSBNRk4sIGludHJvZHVjZQpyYWRpeF90
cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfSBwYXJhbGxlbGluZwpy
YWRpeF90cmVlX3tpbnRfdG9fcHRyLHB0cl90b19pbnR9LgoKQXMgYSBjb25z
ZXF1ZW5jZSBsb2NraW5nIGNoYW5nZXMgYXJlIGFsc28gYXBwbGljYWJsZTog
V2l0aCB0aGVyZSBubwpsb25nZXIgYmVpbmcgYW55IGluc3BlY3Rpb24gb2Yg
dGhlIHJlbW90ZSBkb21haW4ncyBhY3RpdmUgZW50cmllcywKdGhlcmUncyBh
bHNvIG5vIG5lZWQgYW55bW9yZSB0byBob2xkIHRoZSByZW1vdGUgZG9tYWlu
J3MgZ3JhbnQgdGFibGUKbG9jay4gQW5kIHNpbmNlIHdlJ3JlIG5vIGxvbmdl
ciBpdGVyYXRpbmcgb3ZlciB0aGUgbG9jYWwgZG9tYWluJ3MgbWFwCnRyYWNr
IHRhYmxlLCB0aGUgbG9jayBpbiBtYXBfZ3JhbnRfcmVmKCkgY2FuIGFsc28g
YmUgZHJvcHBlZCBiZWZvcmUgdGhlCm5ldyBtYXB0cmFjayBlbnRyeSBhY3R1
YWxseSBnZXRzIHBvcHVsYXRlZC4KCkFzIGEgbmljZSBzaWRlIGVmZmVjdCB0
aGlzIGFsc28gcmVkdWNlcyB0aGUgbnVtYmVyIG9mIElPTU1VIG9wZXJhdGlv
bnMKaW4gdW5tYXBfY29tbW9uKCk6IFByZXZpb3VzbHkgd2Ugd291bGQgaGF2
ZSAiZXN0YWJsaXNoZWQiIGEgcmVhZGFibGUKbWFwcGluZyB3aGVuZXZlciB3
ZSBkaWRuJ3QgZmluZCBhIHdyaXRhYmxlIGVudHJ5IGFueW1vcmUgKHlldCwg
b2YKY291cnNlLCBhdCBsZWFzdCBvbmUgcmVhZGFibGUgb25lKS4gQnV0IHdl
IG9ubHkgbmVlZCB0byBkbyB0aGlzIGlmIHdlCmFjdHVhbGx5IGRyb3BwZWQg
dGhlIGxhc3Qgd3JpdGFibGUgZW50cnksIG5vdCBpZiB0aGVyZSB3ZXJlIG5v
bmUgYWxyZWFkeQpiZWZvcmUuCgpUaGlzIGlzIHBhcnQgb2YgQ1ZFLTIwMjEt
Mjg2OTggLyBYU0EtMzgwLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTogSnVsaWVuIEdyYWxs
IDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRf
dGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTM2
LDYgKzM2LDcgQEAKICNpbmNsdWRlIDx4ZW4vaW9tbXUuaD4KICNpbmNsdWRl
IDx4ZW4vcGFnaW5nLmg+CiAjaW5jbHVkZSA8eGVuL2tleWhhbmRsZXIuaD4K
KyNpbmNsdWRlIDx4ZW4vcmFkaXgtdHJlZS5oPgogI2luY2x1ZGUgPHhlbi92
bWFwLmg+CiAjaW5jbHVkZSA8eHNtL3hzbS5oPgogI2luY2x1ZGUgPGFzbS9m
bHVzaHRsYi5oPgpAQCAtODAsOCArODEsMTMgQEAgc3RydWN0IGdyYW50X3Rh
YmxlIHsKICAgICBncmFudF9zdGF0dXNfdCAgICAgICAqKnN0YXR1czsKICAg
ICAvKiBBY3RpdmUgZ3JhbnQgdGFibGUuICovCiAgICAgc3RydWN0IGFjdGl2
ZV9ncmFudF9lbnRyeSAqKmFjdGl2ZTsKLSAgICAvKiBNYXBwaW5nIHRyYWNr
aW5nIHRhYmxlIHBlciB2Y3B1LiAqLworICAgIC8qIEhhbmRsZS1pbmRleGVk
IHRyYWNraW5nIHRhYmxlIG9mIG1hcHBpbmdzLiAqLwogICAgIHN0cnVjdCBn
cmFudF9tYXBwaW5nICoqbWFwdHJhY2s7CisgICAgLyoKKyAgICAgKiBNRk4t
aW5kZXhlZCB0cmFja2luZyB0cmVlIG9mIG1hcHBpbmdzLCBpZiBuZWVkZWQu
ICBOb3RlIHRoYXQgdGhpcyBpcworICAgICAqIHByb3RlY3RlZCBieSBAbG9j
aywgbm90IEBtYXB0cmFja19sb2NrLgorICAgICAqLworICAgIHN0cnVjdCBy
YWRpeF90cmVlX3Jvb3QgbWFwdHJhY2tfdHJlZTsKIAogICAgIC8qIERvbWFp
biB0byB3aGljaCB0aGlzIHN0cnVjdCBncmFudF90YWJsZSBiZWxvbmdzLiAq
LwogICAgIGNvbnN0IHN0cnVjdCBkb21haW4gKmRvbWFpbjsKQEAgLTQ0NSwz
NCArNDUxLDYgQEAgc3RhdGljIGludCBnZXRfcGFnZWRfZnJhbWUodW5zaWdu
ZWQgbG9uZwogICAgIHJldHVybiBHTlRTVF9va2F5OwogfQogCi1zdGF0aWMg
aW5saW5lIHZvaWQKLWRvdWJsZV9ndF9sb2NrKHN0cnVjdCBncmFudF90YWJs
ZSAqbGd0LCBzdHJ1Y3QgZ3JhbnRfdGFibGUgKnJndCkKLXsKLSAgICAvKgot
ICAgICAqIFNlZSBtYXBraW5kKCkgZm9yIHdoeSB0aGUgd3JpdGUgbG9jayBp
cyBhbHNvIHJlcXVpcmVkIGZvciB0aGUKLSAgICAgKiByZW1vdGUgZG9tYWlu
LgotICAgICAqLwotICAgIGlmICggbGd0IDwgcmd0ICkKLSAgICB7Ci0gICAg
ICAgIGdyYW50X3dyaXRlX2xvY2sobGd0KTsKLSAgICAgICAgZ3JhbnRfd3Jp
dGVfbG9jayhyZ3QpOwotICAgIH0KLSAgICBlbHNlCi0gICAgewotICAgICAg
ICBpZiAoIGxndCAhPSByZ3QgKQotICAgICAgICAgICAgZ3JhbnRfd3JpdGVf
bG9jayhyZ3QpOwotICAgICAgICBncmFudF93cml0ZV9sb2NrKGxndCk7Ci0g
ICAgfQotfQotCi1zdGF0aWMgaW5saW5lIHZvaWQKLWRvdWJsZV9ndF91bmxv
Y2soc3RydWN0IGdyYW50X3RhYmxlICpsZ3QsIHN0cnVjdCBncmFudF90YWJs
ZSAqcmd0KQotewotICAgIGdyYW50X3dyaXRlX3VubG9jayhsZ3QpOwotICAg
IGlmICggbGd0ICE9IHJndCApCi0gICAgICAgIGdyYW50X3dyaXRlX3VubG9j
ayhyZ3QpOwotfQotCiAjZGVmaW5lIElOVkFMSURfTUFQVFJBQ0tfSEFORExF
IFVJTlRfTUFYCiAKIHN0YXRpYyBpbmxpbmUgZ3JhbnRfaGFuZGxlX3QKQEAg
LTg5NSw0MSArODczLDE3IEBAIHN0YXRpYyBzdHJ1Y3QgYWN0aXZlX2dyYW50
X2VudHJ5ICpncmFudF8KICAgICByZXR1cm4gRVJSX1BUUigtRUlOVkFMKTsK
IH0KIAotI2RlZmluZSBNQVBLSU5EX1JFQUQgMQotI2RlZmluZSBNQVBLSU5E
X1dSSVRFIDIKLXN0YXRpYyB1bnNpZ25lZCBpbnQgbWFwa2luZCgKLSAgICBz
dHJ1Y3QgZ3JhbnRfdGFibGUgKmxndCwgY29uc3Qgc3RydWN0IGRvbWFpbiAq
cmQsIG1mbl90IG1mbikKLXsKLSAgICBzdHJ1Y3QgZ3JhbnRfbWFwcGluZyAq
bWFwOwotICAgIGdyYW50X2hhbmRsZV90IGhhbmRsZSwgbGltaXQgPSBsZ3Qt
Pm1hcHRyYWNrX2xpbWl0OwotICAgIHVuc2lnbmVkIGludCBraW5kID0gMDsK
LQotICAgIC8qCi0gICAgICogTXVzdCBoYXZlIHRoZSBsb2NhbCBkb21haW4n
cyBncmFudCB0YWJsZSB3cml0ZSBsb2NrIHdoZW4KLSAgICAgKiBpdGVyYXRp
bmcgb3ZlciBpdHMgbWFwdHJhY2sgZW50cmllcy4KLSAgICAgKi8KLSAgICBB
U1NFUlQocGVyY3B1X3J3X2lzX3dyaXRlX2xvY2tlZCgmbGd0LT5sb2NrKSk7
Ci0gICAgLyoKLSAgICAgKiBNdXN0IGhhdmUgdGhlIHJlbW90ZSBkb21haW4n
cyBncmFudCB0YWJsZSB3cml0ZSBsb2NrIHdoaWxlCi0gICAgICogY291bnRp
bmcgaXRzIGFjdGl2ZSBlbnRyaWVzLgotICAgICAqLwotICAgIEFTU0VSVChw
ZXJjcHVfcndfaXNfd3JpdGVfbG9ja2VkKCZyZC0+Z3JhbnRfdGFibGUtPmxv
Y2spKTsKLQotICAgIHNtcF9ybWIoKTsKLQotICAgIGZvciAoIGhhbmRsZSA9
IDA7ICEoa2luZCAmIE1BUEtJTkRfV1JJVEUpICYmIGhhbmRsZSA8IGxpbWl0
OyBoYW5kbGUrKyApCi0gICAgewotICAgICAgICBtYXAgPSAmbWFwdHJhY2tf
ZW50cnkobGd0LCBoYW5kbGUpOwotICAgICAgICBpZiAoICEobWFwLT5mbGFn
cyAmIChHTlRNQVBfZGV2aWNlX21hcHxHTlRNQVBfaG9zdF9tYXApKSB8fAot
ICAgICAgICAgICAgIG1hcC0+ZG9taWQgIT0gcmQtPmRvbWFpbl9pZCApCi0g
ICAgICAgICAgICBjb250aW51ZTsKLSAgICAgICAgaWYgKCBtZm5fZXEoX2Fj
dGl2ZV9lbnRyeShyZC0+Z3JhbnRfdGFibGUsIG1hcC0+cmVmKS5tZm4sIG1m
bikgKQotICAgICAgICAgICAga2luZCB8PSBtYXAtPmZsYWdzICYgR05UTUFQ
X3JlYWRvbmx5ID8KLSAgICAgICAgICAgICAgICAgICAgTUFQS0lORF9SRUFE
IDogTUFQS0lORF9XUklURTsKLSAgICB9Ci0KLSAgICByZXR1cm4ga2luZDsK
LX0KK3VuaW9uIG1hcHRyYWNrX25vZGUgeworICAgIHN0cnVjdCB7CisgICAg
ICAgIC8qIFJhZGl4IHRyZWUgc2xvdCBwb2ludGVycyB1c2UgdHdvIG9mIHRo
ZSBiaXRzLiAqLworI2lmZGVmIF9fQklHX0VORElBTl9CSVRGSUVMRAorICAg
ICAgICB1bnNpZ25lZCBsb25nICAgIDogMjsKKyNlbmRpZgorICAgICAgICB1
bnNpZ25lZCBsb25nIHJkIDogQklUU19QRVJfTE9ORyAvIDIgLSAxOworICAg
ICAgICB1bnNpZ25lZCBsb25nIHdyIDogQklUU19QRVJfTE9ORyAvIDIgLSAx
OworICAgIH0gY250OworICAgIHVuc2lnbmVkIGxvbmcgcmF3OworfTsKIAog
c3RhdGljIHZvaWQKIG1hcF9ncmFudF9yZWYoCkBAIC05NDgsNyArOTAyLDYg
QEAgbWFwX2dyYW50X3JlZigKICAgICBzdHJ1Y3QgZ3JhbnRfbWFwcGluZyAq
bXQ7CiAgICAgZ3JhbnRfZW50cnlfaGVhZGVyX3QgKnNoYWg7CiAgICAgdWlu
dDE2X3QgKnN0YXR1czsKLSAgICBib29sX3QgbmVlZF9pb21tdTsKIAogICAg
IGxlZCA9IGN1cnJlbnQ7CiAgICAgbGQgPSBsZWQtPmRvbWFpbjsKQEAgLTEx
NTYsMzEgKzExMDksNzUgQEAgbWFwX2dyYW50X3JlZigKICAgICAgICAgZ290
byB1bmRvX291dDsKICAgICB9CiAKLSAgICBuZWVkX2lvbW11ID0gZ250dGFi
X25lZWRfaW9tbXVfbWFwcGluZyhsZCk7Ci0gICAgaWYgKCBuZWVkX2lvbW11
ICkKKyAgICBpZiAoIGdudHRhYl9uZWVkX2lvbW11X21hcHBpbmcobGQpICkK
ICAgICB7CisgICAgICAgIHVuaW9uIG1hcHRyYWNrX25vZGUgbm9kZSA9IHsK
KyAgICAgICAgICAgIC5jbnQucmQgPSAhIShvcC0+ZmxhZ3MgJiBHTlRNQVBf
cmVhZG9ubHkpLAorICAgICAgICAgICAgLmNudC53ciA9ICEob3AtPmZsYWdz
ICYgR05UTUFQX3JlYWRvbmx5KSwKKyAgICAgICAgfTsKKyAgICAgICAgaW50
IGVycjsKKyAgICAgICAgdm9pZCAqKnNsb3QgPSBOVUxMOwogICAgICAgICB1
bnNpZ25lZCBpbnQga2luZDsKIAotICAgICAgICBkb3VibGVfZ3RfbG9jayhs
Z3QsIHJndCk7CisgICAgICAgIGdyYW50X3dyaXRlX2xvY2sobGd0KTsKKwor
ICAgICAgICBlcnIgPSByYWRpeF90cmVlX2luc2VydCgmbGd0LT5tYXB0cmFj
a190cmVlLCBtZm5feChtZm4pLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICByYWRpeF90cmVlX3Vsb25nX3RvX3B0cihub2RlLnJhdykpOwor
ICAgICAgICBpZiAoIGVyciA9PSAtRUVYSVNUICkKKyAgICAgICAgeworICAg
ICAgICAgICAgc2xvdCA9IHJhZGl4X3RyZWVfbG9va3VwX3Nsb3QoJmxndC0+
bWFwdHJhY2tfdHJlZSwgbWZuX3gobWZuKSk7CisgICAgICAgICAgICBpZiAo
IGxpa2VseShzbG90KSApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAg
ICAgbm9kZS5yYXcgPSByYWRpeF90cmVlX3B0cl90b191bG9uZygqc2xvdCk7
CisgICAgICAgICAgICAgICAgZXJyID0gLUVCVVNZOworCisgICAgICAgICAg
ICAgICAgLyogVXBkYXRlIG5vZGUgb25seSB3aGVuIHJlZmNvdW50IGRvZXNu
J3Qgb3ZlcmZsb3cuICovCisgICAgICAgICAgICAgICAgaWYgKCBvcC0+Zmxh
Z3MgJiBHTlRNQVBfcmVhZG9ubHkgPyArK25vZGUuY250LnJkCisgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiAr
K25vZGUuY250LndyICkKKyAgICAgICAgICAgICAgICB7CisgICAgICAgICAg
ICAgICAgICAgIHJhZGl4X3RyZWVfcmVwbGFjZV9zbG90KHNsb3QsCisgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJhZGl4
X3RyZWVfdWxvbmdfdG9fcHRyKG5vZGUucmF3KSk7CisgICAgICAgICAgICAg
ICAgICAgIGVyciA9IDA7CisgICAgICAgICAgICAgICAgfQorICAgICAgICAg
ICAgfQorICAgICAgICAgICAgZWxzZQorICAgICAgICAgICAgICAgIEFTU0VS
VF9VTlJFQUNIQUJMRSgpOworICAgICAgICB9CiAKICAgICAgICAgLyoKICAg
ICAgICAgICogV2UncmUgbm90IHRyYW5zbGF0ZWQsIHNvIHdlIGtub3cgdGhh
dCBkZm5zIGFuZCBtZm5zIGFyZQogICAgICAgICAgKiB0aGUgc2FtZSB0aGlu
Z3MsIHNvIHRoZSBJT01NVSBlbnRyeSBpcyBhbHdheXMgMS10by0xLgogICAg
ICAgICAgKi8KLSAgICAgICAga2luZCA9IG1hcGtpbmQobGd0LCByZCwgbWZu
KTsKLSAgICAgICAgaWYgKCAhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25s
eSkgJiYKLSAgICAgICAgICAgICAhKGtpbmQgJiBNQVBLSU5EX1dSSVRFKSAp
CisgICAgICAgIGlmICggIShvcC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkp
ICYmIG5vZGUuY250LndyID09IDEgKQogICAgICAgICAgICAga2luZCA9IElP
TU1VRl9yZWFkYWJsZSB8IElPTU1VRl93cml0YWJsZTsKLSAgICAgICAgZWxz
ZSBpZiAoICFraW5kICkKKyAgICAgICAgZWxzZSBpZiAoIChvcC0+ZmxhZ3Mg
JiBHTlRNQVBfcmVhZG9ubHkpICYmCisgICAgICAgICAgICAgICAgICBub2Rl
LmNudC5yZCA9PSAxICYmICFub2RlLmNudC53ciApCiAgICAgICAgICAgICBr
aW5kID0gSU9NTVVGX3JlYWRhYmxlOwogICAgICAgICBlbHNlCiAgICAgICAg
ICAgICBraW5kID0gMDsKLSAgICAgICAgaWYgKCBraW5kICYmIGlvbW11X2xl
Z2FjeV9tYXAobGQsIF9kZm4obWZuX3gobWZuKSksIG1mbiwgMCwga2luZCkg
KQorICAgICAgICBpZiAoIGVyciB8fAorICAgICAgICAgICAgIChraW5kICYm
IGlvbW11X2xlZ2FjeV9tYXAobGQsIF9kZm4obWZuX3gobWZuKSksIG1mbiwg
MCwga2luZCkpICkKICAgICAgICAgewotICAgICAgICAgICAgZG91YmxlX2d0
X3VubG9jayhsZ3QsIHJndCk7CisgICAgICAgICAgICBpZiAoICFlcnIgKQor
ICAgICAgICAgICAgeworICAgICAgICAgICAgICAgIGlmICggc2xvdCApCisg
ICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAgICBvcC0+Zmxh
Z3MgJiBHTlRNQVBfcmVhZG9ubHkgPyBub2RlLmNudC5yZC0tCisgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IG5v
ZGUuY250LndyLS07CisgICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVf
cmVwbGFjZV9zbG90KHNsb3QsCisgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVfdWxvbmdfdG9fcHRyKG5v
ZGUucmF3KSk7CisgICAgICAgICAgICAgICAgfQorICAgICAgICAgICAgICAg
IGVsc2UKKyAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUo
JmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gobWZuKSk7CisgICAgICAgICAg
ICB9CisKICAgICAgICAgICAgIHJjID0gR05UU1RfZ2VuZXJhbF9lcnJvcjsK
LSAgICAgICAgICAgIGdvdG8gdW5kb19vdXQ7CiAgICAgICAgIH0KKworICAg
ICAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKKworICAgICAgICBpZiAo
IHJjICE9IEdOVFNUX29rYXkgKQorICAgICAgICAgICAgZ290byB1bmRvX291
dDsKICAgICB9CiAKICAgICBUUkFDRV8xRChUUkNfTUVNX1BBR0VfR1JBTlRf
TUFQLCBvcC0+ZG9tKTsKQEAgLTExODgsMTAgKzExODUsNiBAQCBtYXBfZ3Jh
bnRfcmVmKAogICAgIC8qCiAgICAgICogQWxsIG1hcHRyYWNrIGVudHJ5IHVz
ZXJzIGNoZWNrIG10LT5mbGFncyBmaXJzdCBiZWZvcmUgdXNpbmcgdGhlCiAg
ICAgICogb3RoZXIgZmllbGRzIHNvIGp1c3QgZW5zdXJlIHRoZSBmbGFncyBm
aWVsZCBpcyBzdG9yZWQgbGFzdC4KLSAgICAgKgotICAgICAqIEhvd2V2ZXIs
IGlmIGdudHRhYl9uZWVkX2lvbW11X21hcHBpbmcoKSB0aGVuIHRoaXMgd291
bGQgcmFjZQotICAgICAqIHdpdGggYSBjb25jdXJyZW50IG1hcGtpbmQoKSBj
YWxsIChvbiBhbiB1bm1hcCwgZm9yIGV4YW1wbGUpCi0gICAgICogYW5kIGEg
bG9jayBpcyByZXF1aXJlZC4KICAgICAgKi8KICAgICBtdCA9ICZtYXB0cmFj
a19lbnRyeShsZ3QsIGhhbmRsZSk7CiAgICAgbXQtPmRvbWlkID0gb3AtPmRv
bTsKQEAgLTExOTksOSArMTE5Miw2IEBAIG1hcF9ncmFudF9yZWYoCiAgICAg
c21wX3dtYigpOwogICAgIHdyaXRlX2F0b21pYygmbXQtPmZsYWdzLCBvcC0+
ZmxhZ3MpOwogCi0gICAgaWYgKCBuZWVkX2lvbW11ICkKLSAgICAgICAgZG91
YmxlX2d0X3VubG9jayhsZ3QsIHJndCk7Ci0KICAgICBvcC0+ZGV2X2J1c19h
ZGRyID0gbWZuX3RvX21hZGRyKG1mbik7CiAgICAgb3AtPmhhbmRsZSAgICAg
ICA9IGhhbmRsZTsKICAgICBvcC0+c3RhdHVzICAgICAgID0gR05UU1Rfb2th
eTsKQEAgLTE0MTQsMTkgKzE0MDQsMzQgQEAgdW5tYXBfY29tbW9uKAogCiAg
ICAgaWYgKCByYyA9PSBHTlRTVF9va2F5ICYmIGdudHRhYl9uZWVkX2lvbW11
X21hcHBpbmcobGQpICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGludCBr
aW5kOworICAgICAgICB2b2lkICoqc2xvdDsKKyAgICAgICAgdW5pb24gbWFw
dHJhY2tfbm9kZSBub2RlOwogICAgICAgICBpbnQgZXJyID0gMDsKIAotICAg
ICAgICBkb3VibGVfZ3RfbG9jayhsZ3QsIHJndCk7CisgICAgICAgIGdyYW50
X3dyaXRlX2xvY2sobGd0KTsKKyAgICAgICAgc2xvdCA9IHJhZGl4X3RyZWVf
bG9va3VwX3Nsb3QoJmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gob3AtPm1m
bikpOworICAgICAgICBub2RlLnJhdyA9IGxpa2VseShzbG90KSA/IHJhZGl4
X3RyZWVfcHRyX3RvX3Vsb25nKCpzbG90KSA6IDA7CisKKyAgICAgICAgLyog
UmVmY291bnQgbXVzdCBub3QgdW5kZXJmbG93LiAqLworICAgICAgICBpZiAo
ICEoZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkgPyBub2RlLmNudC5yZC0tCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IG5vZGUu
Y250LndyLS0pICkKKyAgICAgICAgICAgIEJVRygpOwogCi0gICAgICAgIGtp
bmQgPSBtYXBraW5kKGxndCwgcmQsIG9wLT5tZm4pOwotICAgICAgICBpZiAo
ICFraW5kICkKKyAgICAgICAgaWYgKCAhbm9kZS5yYXcgKQogICAgICAgICAg
ICAgZXJyID0gaW9tbXVfbGVnYWN5X3VubWFwKGxkLCBfZGZuKG1mbl94KG9w
LT5tZm4pKSwgMCk7Ci0gICAgICAgIGVsc2UgaWYgKCAhKGtpbmQgJiBNQVBL
SU5EX1dSSVRFKSApCisgICAgICAgIGVsc2UgaWYgKCAhKGZsYWdzICYgR05U
TUFQX3JlYWRvbmx5KSAmJiAhbm9kZS5jbnQud3IgKQogICAgICAgICAgICAg
ZXJyID0gaW9tbXVfbGVnYWN5X21hcChsZCwgX2RmbihtZm5feChvcC0+bWZu
KSksIG9wLT5tZm4sIDAsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIElPTU1VRl9yZWFkYWJsZSk7CiAKLSAgICAgICAgZG91YmxlX2d0
X3VubG9jayhsZ3QsIHJndCk7CisgICAgICAgIGlmICggZXJyICkKKyAgICAg
ICAgICAgIDsKKyAgICAgICAgZWxzZSBpZiAoICFub2RlLnJhdyApCisgICAg
ICAgICAgICByYWRpeF90cmVlX2RlbGV0ZSgmbGd0LT5tYXB0cmFja190cmVl
LCBtZm5feChvcC0+bWZuKSk7CisgICAgICAgIGVsc2UKKyAgICAgICAgICAg
IHJhZGl4X3RyZWVfcmVwbGFjZV9zbG90KHNsb3QsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICByYWRpeF90cmVlX3Vsb25nX3RvX3B0
cihub2RlLnJhdykpOworCisgICAgICAgIGdyYW50X3dyaXRlX3VubG9jayhs
Z3QpOwogCiAgICAgICAgIGlmICggZXJyICkKICAgICAgICAgICAgIHJjID0g
R05UU1RfZ2VuZXJhbF9lcnJvcjsKQEAgLTE4NzQsNiArMTg3OSw4IEBAIGlu
dCBncmFudF90YWJsZV9pbml0KHN0cnVjdCBkb21haW4gKmQsIGkKICAgICAg
ICAgZ3QtPm1hcHRyYWNrID0gdnphbGxvYyhndC0+bWF4X21hcHRyYWNrX2Zy
YW1lcyAqIHNpemVvZigqZ3QtPm1hcHRyYWNrKSk7CiAgICAgICAgIGlmICgg
Z3QtPm1hcHRyYWNrID09IE5VTEwgKQogICAgICAgICAgICAgZ290byBvdXQ7
CisKKyAgICAgICAgcmFkaXhfdHJlZV9pbml0KCZndC0+bWFwdHJhY2tfdHJl
ZSk7CiAgICAgfQogCiAgICAgLyogU2hhcmVkIGdyYW50IHRhYmxlLiAqLwpA
QCAtMzYyOSw2ICszNjM2LDggQEAgaW50IGdudHRhYl9yZWxlYXNlX21hcHBp
bmdzKHN0cnVjdCBkb21haQogCiAgICAgZm9yICggaGFuZGxlID0gZ3QtPm1h
cHRyYWNrX2xpbWl0OyBoYW5kbGU7ICkKICAgICB7CisgICAgICAgIG1mbl90
IG1mbjsKKwogICAgICAgICAvKgogICAgICAgICAgKiBEZWFsIHdpdGggZnVs
bCBwYWdlcyBzdWNoIHRoYXQgdGhlaXIgZnJlZWluZyAoaW4gdGhlIGJvZHkg
b2YgdGhlCiAgICAgICAgICAqIGlmKCkpIHJlbWFpbnMgc2ltcGxlLgpAQCAt
MzczMCwxNyArMzczOSwzMSBAQCBpbnQgZ250dGFiX3JlbGVhc2VfbWFwcGlu
Z3Moc3RydWN0IGRvbWFpCiAgICAgICAgIGlmICggYWN0LT5waW4gPT0gMCAp
CiAgICAgICAgICAgICBnbnR0YWJfY2xlYXJfZmxhZyhyZCwgX0dURl9yZWFk
aW5nLCBzdGF0dXMpOwogCisgICAgICAgIG1mbiA9IGFjdC0+bWZuOworCiAg
ICAgICAgIGFjdGl2ZV9lbnRyeV9yZWxlYXNlKGFjdCk7CiAgICAgICAgIGdy
YW50X3JlYWRfdW5sb2NrKHJndCk7CiAKICAgICAgICAgcmN1X3VubG9ja19k
b21haW4ocmQpOwogCiAgICAgICAgIG1hcC0+ZmxhZ3MgPSAwOworCisgICAg
ICAgIC8qCisgICAgICAgICAqIFRoaXMgaXMgZXhjZXNzaXZlIGluIHRoYXQg
YSBzaW5nbGUgc3VjaCBjYWxsIHdvdWxkIHN1ZmZpY2UgcGVyCisgICAgICAg
ICAqIG1hcHBlZCBNRk4gKG9yIG5vbmUgYXQgYWxsLCBpZiBubyBlbnRyeSB3
YXMgZXZlciBpbnNlcnRlZCkuIEJ1dCBpdAorICAgICAgICAgKiBzaG91bGQg
YmUgdGhlIGNvbW1vbiBjYXNlIGZvciBhbiBNRk4gdG8gYmUgbWFwcGVkIGp1
c3Qgb25jZSwgYW5kCisgICAgICAgICAqIHRoaXMgd2F5IHdlIGRvbid0IG5l
ZWQgdG8gZnVydGhlciBtYWludGFpbiB0aGUgY291bnRlcnMuIFdlIGFsc28K
KyAgICAgICAgICogZG9uJ3Qgd2FudCB0byBsZWF2ZSBjbGVhbmluZyB1cCBv
ZiB0aGUgdHJlZSBhcyBhIHdob2xlIHRvIHRoZSBlbmQKKyAgICAgICAgICog
b2YgdGhlIGZ1bmN0aW9uLCBhcyB0aGlzIGNvdWxkIHRha2UgcXVpdGUgc29t
ZSB0aW1lLgorICAgICAgICAgKi8KKyAgICAgICAgcmFkaXhfdHJlZV9kZWxl
dGUoJmd0LT5tYXB0cmFja190cmVlLCBtZm5feChtZm4pKTsKICAgICB9CiAK
ICAgICBndC0+bWFwdHJhY2tfbGltaXQgPSAwOwogICAgIEZSRUVfWEVOSEVB
UF9QQUdFKGd0LT5tYXB0cmFja1swXSk7CiAKKyAgICByYWRpeF90cmVlX2Rl
c3Ryb3koJmd0LT5tYXB0cmFja190cmVlLCBOVUxMKTsKKwogICAgIHJldHVy
biAwOwogfQogCi0tLSBhL3hlbi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmgK
KysrIGIveGVuL2luY2x1ZGUveGVuL3JhZGl4LXRyZWUuaApAQCAtMTkwLDYg
KzE5MCwyNSBAQCBzdGF0aWMgaW5saW5lIGludCByYWRpeF90cmVlX3B0cl90
b19pbnQoCiAgICAgcmV0dXJuIChpbnQpKChsb25nKXB0ciA+PiAyKTsKIH0K
IAorLyoqCisgKiByYWRpeF90cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vs
b25nfToKKyAqCisgKiBTYW1lIGZvciB1bnNpZ25lZCBsb25nIHZhbHVlcy4g
QmV3YXJlIHRob3VnaCB0aGF0IG9ubHkgQklUU19QRVJfTE9ORy0yCisgKiBi
aXRzIGFyZSBhY3R1YWxseSB1c2FibGUgZm9yIHRoZSB2YWx1ZS4KKyAqLwor
c3RhdGljIGlubGluZSB2b2lkICpyYWRpeF90cmVlX3Vsb25nX3RvX3B0cih1
bnNpZ25lZCBsb25nIHZhbCkKK3sKKyAgICB1bnNpZ25lZCBsb25nIHB0ciA9
ICh2YWwgPDwgMikgfCAweDI7CisgICAgQVNTRVJUKChwdHIgPj4gMikgPT0g
dmFsKTsKKyAgICByZXR1cm4gKHZvaWQgKilwdHI7Cit9CisKK3N0YXRpYyBp
bmxpbmUgdW5zaWduZWQgbG9uZyByYWRpeF90cmVlX3B0cl90b191bG9uZyh2
b2lkICpwdHIpCit7CisgICAgQVNTRVJUKCgodW5zaWduZWQgbG9uZylwdHIg
JiAweDMpID09IDB4Mik7CisgICAgcmV0dXJuICh1bnNpZ25lZCBsb25nKXB0
ciA+PiAyOworfQorCiBpbnQgcmFkaXhfdHJlZV9pbnNlcnQoc3RydWN0IHJh
ZGl4X3RyZWVfcm9vdCAqLCB1bnNpZ25lZCBsb25nLCB2b2lkICopOwogdm9p
ZCAqcmFkaXhfdHJlZV9sb29rdXAoc3RydWN0IHJhZGl4X3RyZWVfcm9vdCAq
LCB1bnNpZ25lZCBsb25nKTsKIHZvaWQgKipyYWRpeF90cmVlX2xvb2t1cF9z
bG90KHN0cnVjdCByYWRpeF90cmVlX3Jvb3QgKiwgdW5zaWduZWQgbG9uZyk7
Cg==

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.13-1.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.13-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGFkZCBwcmVlbXB0aW9uIGNoZWNrIHRvIGdudHRhYl9yZWxl
YXNlX21hcHBpbmdzKCkKCkEgZ3Vlc3QgbWF5IGRpZSB3aXRoIG1hbnkgZ3Jh
bnQgbWFwcGluZ3Mgc3RpbGwgaW4gcGxhY2UsIG9yIHNpbXBseSB3aXRoCmEg
bGFyZ2UgbWFwdHJhY2sgdGFibGUuIEl0ZXJhdGluZyB0aHJvdWdoIHRoaXMg
bWF5IHRha2UgbW9yZSB0aW1lIHRoYW4KaXMgcmVhc29uYWJsZSB3aXRob3V0
IGludGVybWVkaWF0ZSBwcmVlbXB0aW9uICh0byBydW4gc29mdGlycXMgYW5k
CnBlcmhhcHMgdGhlIHNjaGVkdWxlcikuCgpNb3ZlIHRoZSBpbnZvY2F0aW9u
IG9mIHRoZSBmdW5jdGlvbiB0byB0aGUgc2VjdGlvbiB3aGVyZSBvdGhlcgpy
ZXN0YXJ0YWJsZSBmdW5jdGlvbnMgZ2V0IGludm9rZWQsIGFuZCBoYXZlIHRo
ZSBmdW5jdGlvbiBpdHNlbGYgY2hlY2sKZm9yIHByZWVtcHRpb24gZXZlcnkg
b25jZSBpbiBhIHdoaWxlLiBIYXZlIGl0IGl0ZXJhdGUgdGhlIHRhYmxlCmJh
Y2t3YXJkcywgc3VjaCB0aGF0IGRlY3JlYXNpbmcgdGhlIG1hcHRyYWNrIGxp
bWl0IGlzIGFsbCBpdCB0YWtlcyB0bwpjb252ZXkgcmVzdGFydCBpbmZvcm1h
dGlvbi4KCkluIGRvbWFpbl90ZWFyZG93bigpIGludHJvZHVjZSBQUk9HX25v
bmUgc3VjaCB0aGF0IGluc2VydGluZyBhdCB0aGUKZnJvbnQgd2lsbCBiZSBl
YXNpZXIgZ29pbmcgZm9yd2FyZC4KClRoaXMgaXMgcGFydCBvZiBDVkUtMjAy
MS0yODY5OCAvIFhTQS0zODAuCgpSZXBvcnRlZC1ieTogQW5kcmV3IENvb3Bl
ciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
SnVsaWVuIEdyYWxsIDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9j
b21tb24vZG9tYWluLmMKKysrIGIveGVuL2NvbW1vbi9kb21haW4uYwpAQCAt
Nzc2LDExICs3NzYsMTMgQEAgaW50IGRvbWFpbl9raWxsKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgICAgICAgICByZXR1cm4gZG9tYWluX2tpbGwoZCk7CiAg
ICAgICAgIGQtPmlzX2R5aW5nID0gRE9NRFlJTkdfZHlpbmc7CiAgICAgICAg
IGFyZ29fZGVzdHJveShkKTsKLSAgICAgICAgZ250dGFiX3JlbGVhc2VfbWFw
cGluZ3MoZCk7CiAgICAgICAgIHZudW1hX2Rlc3Ryb3koZC0+dm51bWEpOwog
ICAgICAgICBkb21haW5fc2V0X291dHN0YW5kaW5nX3BhZ2VzKGQsIDApOwog
ICAgICAgICAvKiBmYWxsdGhyb3VnaCAqLwogICAgIGNhc2UgRE9NRFlJTkdf
ZHlpbmc6CisgICAgICAgIHJjID0gZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Mo
ZCk7CisgICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAgYnJlYWs7CiAg
ICAgICAgIHJjID0gZXZ0Y2huX2Rlc3Ryb3koZCk7CiAgICAgICAgIGlmICgg
cmMgKQogICAgICAgICAgICAgYnJlYWs7Ci0tLSBhL3hlbi9jb21tb24vZ3Jh
bnRfdGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAg
LTYzLDcgKzYzLDEzIEBAIHN0cnVjdCBncmFudF90YWJsZSB7CiAgICAgdW5z
aWduZWQgaW50ICAgICAgICAgIG5yX2dyYW50X2ZyYW1lczsKICAgICAvKiBO
dW1iZXIgb2YgZ3JhbnQgc3RhdHVzIGZyYW1lcyBzaGFyZWQgd2l0aCBndWVz
dCAoZm9yIHZlcnNpb24gMikgKi8KICAgICB1bnNpZ25lZCBpbnQgICAgICAg
ICAgbnJfc3RhdHVzX2ZyYW1lczsKLSAgICAvKiBOdW1iZXIgb2YgYXZhaWxh
YmxlIG1hcHRyYWNrIGVudHJpZXMuICovCisgICAgLyoKKyAgICAgKiBOdW1i
ZXIgb2YgYXZhaWxhYmxlIG1hcHRyYWNrIGVudHJpZXMuICBGb3IgY2xlYW51
cCBwdXJwb3NlcyBpdCBpcworICAgICAqIGltcG9ydGFudCB0byByZWFsaXpl
IHRoYXQgdGhpcyBmaWVsZCBhbmQgQG1hcHRyYWNrIGZ1cnRoZXIgZG93biB3
aWxsCisgICAgICogb25seSBldmVyIGJlIGFjY2Vzc2VkIGJ5IHRoZSBsb2Nh
bCBkb21haW4uICBUaHVzIGl0IGlzIG9rYXkgdG8gY2xlYW4KKyAgICAgKiB1
cCBlYXJseSwgYW5kIHRvIHNocmluayB0aGUgbGltaXQgZm9yIHRoZSBwdXJw
b3NlIG9mIHRyYWNraW5nIGNsZWFudXAKKyAgICAgKiBwcm9ncmVzcy4KKyAg
ICAgKi8KICAgICB1bnNpZ25lZCBpbnQgICAgICAgICAgbWFwdHJhY2tfbGlt
aXQ7CiAgICAgLyogU2hhcmVkIGdyYW50IHRhYmxlIChzZWUgaW5jbHVkZS9w
dWJsaWMvZ3JhbnRfdGFibGUuaCkuICovCiAgICAgdW5pb24gewpAQCAtMzY3
NSw5ICszNjgxLDcgQEAgZG9fZ3JhbnRfdGFibGVfb3AoCiAjaW5jbHVkZSAi
Y29tcGF0L2dyYW50X3RhYmxlLmMiCiAjZW5kaWYKIAotdm9pZAotZ250dGFi
X3JlbGVhc2VfbWFwcGluZ3MoCi0gICAgc3RydWN0IGRvbWFpbiAqZCkKK2lu
dCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkKQog
ewogICAgIHN0cnVjdCBncmFudF90YWJsZSAgICpndCA9IGQtPmdyYW50X3Rh
YmxlLCAqcmd0OwogICAgIHN0cnVjdCBncmFudF9tYXBwaW5nICptYXA7CkBA
IC0zNjkxLDEwICszNjk1LDM0IEBAIGdudHRhYl9yZWxlYXNlX21hcHBpbmdz
KAogCiAgICAgQlVHX09OKCFkLT5pc19keWluZyk7CiAKLSAgICBmb3IgKCBo
YW5kbGUgPSAwOyBoYW5kbGUgPCBndC0+bWFwdHJhY2tfbGltaXQ7IGhhbmRs
ZSsrICkKKyAgICBpZiAoICFndCB8fCAhZ3QtPm1hcHRyYWNrICkKKyAgICAg
ICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCBoYW5kbGUgPSBndC0+bWFwdHJh
Y2tfbGltaXQ7IGhhbmRsZTsgKQogICAgIHsKICAgICAgICAgdW5zaWduZWQg
aW50IGNsZWFyX2ZsYWdzID0gMDsKIAorICAgICAgICAvKgorICAgICAgICAg
KiBEZWFsIHdpdGggZnVsbCBwYWdlcyBzdWNoIHRoYXQgdGhlaXIgZnJlZWlu
ZyAoaW4gdGhlIGJvZHkgb2YgdGhlCisgICAgICAgICAqIGlmKCkpIHJlbWFp
bnMgc2ltcGxlLgorICAgICAgICAgKi8KKyAgICAgICAgaWYgKCBoYW5kbGUg
PCBndC0+bWFwdHJhY2tfbGltaXQgJiYgIShoYW5kbGUgJSBNQVBUUkFDS19Q
RVJfUEFHRSkgKQorICAgICAgICB7CisgICAgICAgICAgICAvKgorICAgICAg
ICAgICAgICogQ2hhbmdpbmcgbWFwdHJhY2tfbGltaXQgYWx0ZXJzIG5yX21h
cHRyYWNrX2ZyYW1lcygpJ2VzIHJldHVybgorICAgICAgICAgICAgICogdmFs
dWUuIEZyZWUgdGhlIHRoZW4gZXhjZXNzIHRyYWlsaW5nIHBhZ2UgcmlnaHQg
aGVyZSwgcmF0aGVyCisgICAgICAgICAgICAgKiB0aGFuIGxlYXZpbmcgaXQg
dG8gZ3JhbnRfdGFibGVfZGVzdHJveSgpIChhbmQgaW4gdHVybiByZXF1aXJp
bmcKKyAgICAgICAgICAgICAqIHRvIGxlYXZlIGd0LT5tYXB0cmFja19saW1p
dCB1bmFsdGVyZWQpLgorICAgICAgICAgICAgICovCisgICAgICAgICAgICBn
dC0+bWFwdHJhY2tfbGltaXQgPSBoYW5kbGU7CisgICAgICAgICAgICBGUkVF
X1hFTkhFQVBfUEFHRShndC0+bWFwdHJhY2tbbnJfbWFwdHJhY2tfZnJhbWVz
KGd0KV0pOworCisgICAgICAgICAgICBpZiAoIGh5cGVyY2FsbF9wcmVlbXB0
X2NoZWNrKCkgKQorICAgICAgICAgICAgICAgIHJldHVybiAtRVJFU1RBUlQ7
CisgICAgICAgIH0KKworICAgICAgICAtLWhhbmRsZTsKKwogICAgICAgICBt
YXAgPSAmbWFwdHJhY2tfZW50cnkoZ3QsIGhhbmRsZSk7CiAgICAgICAgIGlm
ICggIShtYXAtPmZsYWdzICYgKEdOVE1BUF9kZXZpY2VfbWFwfEdOVE1BUF9o
b3N0X21hcCkpICkKICAgICAgICAgICAgIGNvbnRpbnVlOwpAQCAtMzc4NSw2
ICszODEzLDExIEBAIGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKAogCiAgICAg
ICAgIG1hcC0+ZmxhZ3MgPSAwOwogICAgIH0KKworICAgIGd0LT5tYXB0cmFj
a19saW1pdCA9IDA7CisgICAgRlJFRV9YRU5IRUFQX1BBR0UoZ3QtPm1hcHRy
YWNrWzBdKTsKKworICAgIHJldHVybiAwOwogfQogCiB2b2lkIGdyYW50X3Rh
YmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKQpAQCAt
Mzg0OCw4ICszODgxLDcgQEAgZ3JhbnRfdGFibGVfZGVzdHJveSgKICAgICAg
ICAgZnJlZV94ZW5oZWFwX3BhZ2UodC0+c2hhcmVkX3Jhd1tpXSk7CiAgICAg
eGZyZWUodC0+c2hhcmVkX3Jhdyk7CiAKLSAgICBmb3IgKCBpID0gMDsgaSA8
IG5yX21hcHRyYWNrX2ZyYW1lcyh0KTsgaSsrICkKLSAgICAgICAgZnJlZV94
ZW5oZWFwX3BhZ2UodC0+bWFwdHJhY2tbaV0pOworICAgIEFTU0VSVCghdC0+
bWFwdHJhY2tfbGltaXQpOwogICAgIHZmcmVlKHQtPm1hcHRyYWNrKTsKIAog
ICAgIGZvciAoIGkgPSAwOyBpIDwgbnJfYWN0aXZlX2dyYW50X2ZyYW1lcyh0
KTsgaSsrICkKLS0tIGEveGVuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmgK
KysrIGIveGVuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmgKQEAgLTQ3LDkg
KzQ3LDcgQEAgdm9pZCBncmFudF90YWJsZV9pbml0X3ZjcHUoc3RydWN0IHZj
cHUgKgogdm9pZCBncmFudF90YWJsZV93YXJuX2FjdGl2ZV9ncmFudHMoc3Ry
dWN0IGRvbWFpbiAqZCk7CiAKIC8qIERvbWFpbiBkZWF0aCByZWxlYXNlIG9m
IGdyYW50ZWQgbWFwcGluZ3Mgb2Ygb3RoZXIgZG9tYWlucycgbWVtb3J5LiAq
Lwotdm9pZAotZ250dGFiX3JlbGVhc2VfbWFwcGluZ3MoCi0gICAgc3RydWN0
IGRvbWFpbiAqZCk7CitpbnQgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Moc3Ry
dWN0IGRvbWFpbiAqZCk7CiAKIGludCBtZW1fc2hhcmluZ19ncmVmX3RvX2dm
bihzdHJ1Y3QgZ3JhbnRfdGFibGUgKmd0LCBncmFudF9yZWZfdCByZWYsCiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2ZuX3QgKmdmbiwgdWludDE2
X3QgKnN0YXR1cyk7CkBAIC03OCw3ICs3Niw3IEBAIHN0YXRpYyBpbmxpbmUg
dm9pZCBncmFudF90YWJsZV9pbml0X3ZjcHUKIAogc3RhdGljIGlubGluZSB2
b2lkIGdyYW50X3RhYmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9t
YWluICpkKSB7fQogCi1zdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX3JlbGVh
c2VfbWFwcGluZ3Moc3RydWN0IGRvbWFpbiAqZCkge30KK3N0YXRpYyBpbmxp
bmUgaW50IGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKHN0cnVjdCBkb21haW4g
KmQpIHsgcmV0dXJuIDA7IH0KIAogc3RhdGljIGlubGluZSBpbnQgbWVtX3No
YXJpbmdfZ3JlZl90b19nZm4oc3RydWN0IGdyYW50X3RhYmxlICpndCwKICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGdyYW50
X3JlZl90IHJlZiwK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.13-2.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.13-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IHJlcGxhY2UgbWFwa2luZCgpCgptYXBraW5kKCkgZG9lc24n
dCBzY2FsZSB2ZXJ5IHdlbGwgd2l0aCBsYXJnZXIgbWFwdHJhY2sgZW50cnkg
Y291bnRzLAp1c2luZyBhIGJydXRlIGZvcmNlIGxpbmVhciBzZWFyY2ggdGhy
b3VnaCBhbGwgZW50cmllcywgd2l0aCB0aGUgb25seQpvcHRpb24gb2YgYW4g
ZWFybHkgbG9vcCBleGl0IGlmIGEgbWF0Y2hpbmcgd3JpdGFibGUgZW50cnkg
d2FzIGZvdW5kLgpJbnRyb2R1Y2UgYSByYWRpeCB0cmVlIGFsb25nc2lkZSB0
aGUgbWFpbiBtYXB0cmFjayB0YWJsZSwgdGh1cwphbGxvd2luZyBtdWNoIGZh
c3RlciBNRk4tYmFzZWQgbG9va3VwLiBUbyBhdm9pZCB0aGUgbmVlZCB0byBh
Y3R1YWxseQphbGxvY2F0ZSBzcGFjZSBmb3IgdGhlIGluZGl2aWR1YWwgbm9k
ZXMsIGVuY29kZSB0aGUgdHdvIGNvdW50ZXJzIGluIHRoZQpub2RlIHBvaW50
ZXJzIHRoZW1zZWx2ZXMsIHRodXMgbGltaXRpbmcgdGhlIG51bWJlciBvZiBw
ZXJtaXR0ZWQKc2ltdWx0YW5lb3VzIHIvbyBhbmQgci93IG1hcHBpbmdzIG9m
IHRoZSBzYW1lIE1GTiB0byAywrPCuS0xICg2NC1iaXQpIC8KMsK54oG1LTEg
KDMyLWJpdCkgZWFjaC4KClRvIGF2b2lkIGVuZm9yY2luZyBhbiB1bm5lY2Vz
c2FyaWx5IGxvdyBib3VuZCBvbiB0aGUgbnVtYmVyIG9mCnNpbXVsdGFuZW91
cyBtYXBwaW5ncyBvZiBhIHNpbmdsZSBNRk4sIGludHJvZHVjZQpyYWRpeF90
cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfSBwYXJhbGxlbGluZwpy
YWRpeF90cmVlX3tpbnRfdG9fcHRyLHB0cl90b19pbnR9LgoKQXMgYSBjb25z
ZXF1ZW5jZSBsb2NraW5nIGNoYW5nZXMgYXJlIGFsc28gYXBwbGljYWJsZTog
V2l0aCB0aGVyZSBubwpsb25nZXIgYmVpbmcgYW55IGluc3BlY3Rpb24gb2Yg
dGhlIHJlbW90ZSBkb21haW4ncyBhY3RpdmUgZW50cmllcywKdGhlcmUncyBh
bHNvIG5vIG5lZWQgYW55bW9yZSB0byBob2xkIHRoZSByZW1vdGUgZG9tYWlu
J3MgZ3JhbnQgdGFibGUKbG9jay4gQW5kIHNpbmNlIHdlJ3JlIG5vIGxvbmdl
ciBpdGVyYXRpbmcgb3ZlciB0aGUgbG9jYWwgZG9tYWluJ3MgbWFwCnRyYWNr
IHRhYmxlLCB0aGUgbG9jayBpbiBtYXBfZ3JhbnRfcmVmKCkgY2FuIGFsc28g
YmUgZHJvcHBlZCBiZWZvcmUgdGhlCm5ldyBtYXB0cmFjayBlbnRyeSBhY3R1
YWxseSBnZXRzIHBvcHVsYXRlZC4KCkFzIGEgbmljZSBzaWRlIGVmZmVjdCB0
aGlzIGFsc28gcmVkdWNlcyB0aGUgbnVtYmVyIG9mIElPTU1VIG9wZXJhdGlv
bnMKaW4gdW5tYXBfY29tbW9uKCk6IFByZXZpb3VzbHkgd2Ugd291bGQgaGF2
ZSAiZXN0YWJsaXNoZWQiIGEgcmVhZGFibGUKbWFwcGluZyB3aGVuZXZlciB3
ZSBkaWRuJ3QgZmluZCBhIHdyaXRhYmxlIGVudHJ5IGFueW1vcmUgKHlldCwg
b2YKY291cnNlLCBhdCBsZWFzdCBvbmUgcmVhZGFibGUgb25lKS4gQnV0IHdl
IG9ubHkgbmVlZCB0byBkbyB0aGlzIGlmIHdlCmFjdHVhbGx5IGRyb3BwZWQg
dGhlIGxhc3Qgd3JpdGFibGUgZW50cnksIG5vdCBpZiB0aGVyZSB3ZXJlIG5v
bmUgYWxyZWFkeQpiZWZvcmUuCgpUaGlzIGlzIHBhcnQgb2YgQ1ZFLTIwMjEt
Mjg2OTggLyBYU0EtMzgwLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTogSnVsaWVuIEdyYWxs
IDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRf
dGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTM2
LDYgKzM2LDcgQEAKICNpbmNsdWRlIDx4ZW4vaW9tbXUuaD4KICNpbmNsdWRl
IDx4ZW4vcGFnaW5nLmg+CiAjaW5jbHVkZSA8eGVuL2tleWhhbmRsZXIuaD4K
KyNpbmNsdWRlIDx4ZW4vcmFkaXgtdHJlZS5oPgogI2luY2x1ZGUgPHhlbi92
bWFwLmg+CiAjaW5jbHVkZSA8eGVuL25vc3BlYy5oPgogI2luY2x1ZGUgPHhz
bS94c20uaD4KQEAgLTgxLDggKzgyLDEzIEBAIHN0cnVjdCBncmFudF90YWJs
ZSB7CiAgICAgZ3JhbnRfc3RhdHVzX3QgICAgICAgKipzdGF0dXM7CiAgICAg
LyogQWN0aXZlIGdyYW50IHRhYmxlLiAqLwogICAgIHN0cnVjdCBhY3RpdmVf
Z3JhbnRfZW50cnkgKiphY3RpdmU7Ci0gICAgLyogTWFwcGluZyB0cmFja2lu
ZyB0YWJsZSBwZXIgdmNwdS4gKi8KKyAgICAvKiBIYW5kbGUtaW5kZXhlZCB0
cmFja2luZyB0YWJsZSBvZiBtYXBwaW5ncy4gKi8KICAgICBzdHJ1Y3QgZ3Jh
bnRfbWFwcGluZyAqKm1hcHRyYWNrOworICAgIC8qCisgICAgICogTUZOLWlu
ZGV4ZWQgdHJhY2tpbmcgdHJlZSBvZiBtYXBwaW5ncywgaWYgbmVlZGVkLiAg
Tm90ZSB0aGF0IHRoaXMgaXMKKyAgICAgKiBwcm90ZWN0ZWQgYnkgQGxvY2ss
IG5vdCBAbWFwdHJhY2tfbG9jay4KKyAgICAgKi8KKyAgICBzdHJ1Y3QgcmFk
aXhfdHJlZV9yb290IG1hcHRyYWNrX3RyZWU7CiAKICAgICAvKiBEb21haW4g
dG8gd2hpY2ggdGhpcyBzdHJ1Y3QgZ3JhbnRfdGFibGUgYmVsb25ncy4gKi8K
ICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICpkb21haW47CkBAIC00NjAsMzQg
KzQ2Niw2IEBAIHN0YXRpYyBpbnQgZ2V0X3BhZ2VkX2ZyYW1lKHVuc2lnbmVk
IGxvbmcKICAgICByZXR1cm4gR05UU1Rfb2theTsKIH0KIAotc3RhdGljIGlu
bGluZSB2b2lkCi1kb3VibGVfZ3RfbG9jayhzdHJ1Y3QgZ3JhbnRfdGFibGUg
KmxndCwgc3RydWN0IGdyYW50X3RhYmxlICpyZ3QpCi17Ci0gICAgLyoKLSAg
ICAgKiBTZWUgbWFwa2luZCgpIGZvciB3aHkgdGhlIHdyaXRlIGxvY2sgaXMg
YWxzbyByZXF1aXJlZCBmb3IgdGhlCi0gICAgICogcmVtb3RlIGRvbWFpbi4K
LSAgICAgKi8KLSAgICBpZiAoIGxndCA8IHJndCApCi0gICAgewotICAgICAg
ICBncmFudF93cml0ZV9sb2NrKGxndCk7Ci0gICAgICAgIGdyYW50X3dyaXRl
X2xvY2socmd0KTsKLSAgICB9Ci0gICAgZWxzZQotICAgIHsKLSAgICAgICAg
aWYgKCBsZ3QgIT0gcmd0ICkKLSAgICAgICAgICAgIGdyYW50X3dyaXRlX2xv
Y2socmd0KTsKLSAgICAgICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOwotICAg
IH0KLX0KLQotc3RhdGljIGlubGluZSB2b2lkCi1kb3VibGVfZ3RfdW5sb2Nr
KHN0cnVjdCBncmFudF90YWJsZSAqbGd0LCBzdHJ1Y3QgZ3JhbnRfdGFibGUg
KnJndCkKLXsKLSAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKLSAgICBp
ZiAoIGxndCAhPSByZ3QgKQotICAgICAgICBncmFudF93cml0ZV91bmxvY2so
cmd0KTsKLX0KLQogI2RlZmluZSBJTlZBTElEX01BUFRSQUNLX0hBTkRMRSBV
SU5UX01BWAogCiBzdGF0aWMgaW5saW5lIGdyYW50X2hhbmRsZV90CkBAIC05
MDcsNDEgKzg4NSwxNyBAQCBzdGF0aWMgc3RydWN0IGFjdGl2ZV9ncmFudF9l
bnRyeSAqZ3JhbnRfCiAgICAgcmV0dXJuIEVSUl9QVFIoLUVJTlZBTCk7CiB9
CiAKLSNkZWZpbmUgTUFQS0lORF9SRUFEIDEKLSNkZWZpbmUgTUFQS0lORF9X
UklURSAyCi1zdGF0aWMgdW5zaWduZWQgaW50IG1hcGtpbmQoCi0gICAgc3Ry
dWN0IGdyYW50X3RhYmxlICpsZ3QsIGNvbnN0IHN0cnVjdCBkb21haW4gKnJk
LCBtZm5fdCBtZm4pCi17Ci0gICAgc3RydWN0IGdyYW50X21hcHBpbmcgKm1h
cDsKLSAgICBncmFudF9oYW5kbGVfdCBoYW5kbGUsIGxpbWl0ID0gbGd0LT5t
YXB0cmFja19saW1pdDsKLSAgICB1bnNpZ25lZCBpbnQga2luZCA9IDA7Ci0K
LSAgICAvKgotICAgICAqIE11c3QgaGF2ZSB0aGUgbG9jYWwgZG9tYWluJ3Mg
Z3JhbnQgdGFibGUgd3JpdGUgbG9jayB3aGVuCi0gICAgICogaXRlcmF0aW5n
IG92ZXIgaXRzIG1hcHRyYWNrIGVudHJpZXMuCi0gICAgICovCi0gICAgQVNT
RVJUKHBlcmNwdV9yd19pc193cml0ZV9sb2NrZWQoJmxndC0+bG9jaykpOwot
ICAgIC8qCi0gICAgICogTXVzdCBoYXZlIHRoZSByZW1vdGUgZG9tYWluJ3Mg
Z3JhbnQgdGFibGUgd3JpdGUgbG9jayB3aGlsZQotICAgICAqIGNvdW50aW5n
IGl0cyBhY3RpdmUgZW50cmllcy4KLSAgICAgKi8KLSAgICBBU1NFUlQocGVy
Y3B1X3J3X2lzX3dyaXRlX2xvY2tlZCgmcmQtPmdyYW50X3RhYmxlLT5sb2Nr
KSk7Ci0KLSAgICBzbXBfcm1iKCk7Ci0KLSAgICBmb3IgKCBoYW5kbGUgPSAw
OyAhKGtpbmQgJiBNQVBLSU5EX1dSSVRFKSAmJiBoYW5kbGUgPCBsaW1pdDsg
aGFuZGxlKysgKQotICAgIHsKLSAgICAgICAgbWFwID0gJm1hcHRyYWNrX2Vu
dHJ5KGxndCwgaGFuZGxlKTsKLSAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3Mg
JiAoR05UTUFQX2RldmljZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgfHwKLSAg
ICAgICAgICAgICBtYXAtPmRvbWlkICE9IHJkLT5kb21haW5faWQgKQotICAg
ICAgICAgICAgY29udGludWU7Ci0gICAgICAgIGlmICggbWZuX2VxKF9hY3Rp
dmVfZW50cnkocmQtPmdyYW50X3RhYmxlLCBtYXAtPnJlZikubWZuLCBtZm4p
ICkKLSAgICAgICAgICAgIGtpbmQgfD0gbWFwLT5mbGFncyAmIEdOVE1BUF9y
ZWFkb25seSA/Ci0gICAgICAgICAgICAgICAgICAgIE1BUEtJTkRfUkVBRCA6
IE1BUEtJTkRfV1JJVEU7Ci0gICAgfQotCi0gICAgcmV0dXJuIGtpbmQ7Ci19
Cit1bmlvbiBtYXB0cmFja19ub2RlIHsKKyAgICBzdHJ1Y3QgeworICAgICAg
ICAvKiBSYWRpeCB0cmVlIHNsb3QgcG9pbnRlcnMgdXNlIHR3byBvZiB0aGUg
Yml0cy4gKi8KKyNpZmRlZiBfX0JJR19FTkRJQU5fQklURklFTEQKKyAgICAg
ICAgdW5zaWduZWQgbG9uZyAgICA6IDI7CisjZW5kaWYKKyAgICAgICAgdW5z
aWduZWQgbG9uZyByZCA6IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsKKyAgICAg
ICAgdW5zaWduZWQgbG9uZyB3ciA6IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsK
KyAgICB9IGNudDsKKyAgICB1bnNpZ25lZCBsb25nIHJhdzsKK307CiAKIHN0
YXRpYyB2b2lkCiBtYXBfZ3JhbnRfcmVmKApAQCAtOTYxLDcgKzkxNSw2IEBA
IG1hcF9ncmFudF9yZWYoCiAgICAgc3RydWN0IGdyYW50X21hcHBpbmcgKm10
OwogICAgIGdyYW50X2VudHJ5X2hlYWRlcl90ICpzaGFoOwogICAgIHVpbnQx
Nl90ICpzdGF0dXM7Ci0gICAgYm9vbF90IG5lZWRfaW9tbXU7CiAKICAgICBs
ZWQgPSBjdXJyZW50OwogICAgIGxkID0gbGVkLT5kb21haW47CkBAIC0xMTgx
LDMxICsxMTM0LDc1IEBAIG1hcF9ncmFudF9yZWYoCiAgICAgICogYXMgbWVt
LXNoYXJpbmcgYW5kIElPTU1VIHVzZSBhcmUgaW5jb21wYXRpYmxlKS4gVGhl
IGRvbV9pbyBjYXNlIHdvdWxkCiAgICAgICogbmVlZCBjaGVja2luZyBzZXBh
cmF0ZWx5IGlmIHdlIGNvbXBhcmVkIGFnYWluc3Qgb3duZXIgaGVyZS4KICAg
ICAgKi8KLSAgICBuZWVkX2lvbW11ID0gbGQgIT0gcmQgJiYgZ250dGFiX25l
ZWRfaW9tbXVfbWFwcGluZyhsZCk7Ci0gICAgaWYgKCBuZWVkX2lvbW11ICkK
KyAgICBpZiAoIGxkICE9IHJkICYmIGdudHRhYl9uZWVkX2lvbW11X21hcHBp
bmcobGQpICkKICAgICB7CisgICAgICAgIHVuaW9uIG1hcHRyYWNrX25vZGUg
bm9kZSA9IHsKKyAgICAgICAgICAgIC5jbnQucmQgPSAhIShvcC0+ZmxhZ3Mg
JiBHTlRNQVBfcmVhZG9ubHkpLAorICAgICAgICAgICAgLmNudC53ciA9ICEo
b3AtPmZsYWdzICYgR05UTUFQX3JlYWRvbmx5KSwKKyAgICAgICAgfTsKKyAg
ICAgICAgaW50IGVycjsKKyAgICAgICAgdm9pZCAqKnNsb3QgPSBOVUxMOwog
ICAgICAgICB1bnNpZ25lZCBpbnQga2luZDsKIAotICAgICAgICBkb3VibGVf
Z3RfbG9jayhsZ3QsIHJndCk7CisgICAgICAgIGdyYW50X3dyaXRlX2xvY2so
bGd0KTsKKworICAgICAgICBlcnIgPSByYWRpeF90cmVlX2luc2VydCgmbGd0
LT5tYXB0cmFja190cmVlLCBtZm5feChtZm4pLAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICByYWRpeF90cmVlX3Vsb25nX3RvX3B0cihub2Rl
LnJhdykpOworICAgICAgICBpZiAoIGVyciA9PSAtRUVYSVNUICkKKyAgICAg
ICAgeworICAgICAgICAgICAgc2xvdCA9IHJhZGl4X3RyZWVfbG9va3VwX3Ns
b3QoJmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gobWZuKSk7CisgICAgICAg
ICAgICBpZiAoIGxpa2VseShzbG90KSApCisgICAgICAgICAgICB7CisgICAg
ICAgICAgICAgICAgbm9kZS5yYXcgPSByYWRpeF90cmVlX3B0cl90b191bG9u
Zygqc2xvdCk7CisgICAgICAgICAgICAgICAgZXJyID0gLUVCVVNZOworCisg
ICAgICAgICAgICAgICAgLyogVXBkYXRlIG5vZGUgb25seSB3aGVuIHJlZmNv
dW50IGRvZXNuJ3Qgb3ZlcmZsb3cuICovCisgICAgICAgICAgICAgICAgaWYg
KCBvcC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkgPyArK25vZGUuY250LnJk
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgOiArK25vZGUuY250LndyICkKKyAgICAgICAgICAgICAgICB7Cisg
ICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVfcmVwbGFjZV9zbG90KHNs
b3QsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHJhZGl4X3RyZWVfdWxvbmdfdG9fcHRyKG5vZGUucmF3KSk7CisgICAg
ICAgICAgICAgICAgICAgIGVyciA9IDA7CisgICAgICAgICAgICAgICAgfQor
ICAgICAgICAgICAgfQorICAgICAgICAgICAgZWxzZQorICAgICAgICAgICAg
ICAgIEFTU0VSVF9VTlJFQUNIQUJMRSgpOworICAgICAgICB9CiAKICAgICAg
ICAgLyoKICAgICAgICAgICogV2UncmUgbm90IHRyYW5zbGF0ZWQsIHNvIHdl
IGtub3cgdGhhdCBkZm5zIGFuZCBtZm5zIGFyZQogICAgICAgICAgKiB0aGUg
c2FtZSB0aGluZ3MsIHNvIHRoZSBJT01NVSBlbnRyeSBpcyBhbHdheXMgMS10
by0xLgogICAgICAgICAgKi8KLSAgICAgICAga2luZCA9IG1hcGtpbmQobGd0
LCByZCwgbWZuKTsKLSAgICAgICAgaWYgKCAhKG9wLT5mbGFncyAmIEdOVE1B
UF9yZWFkb25seSkgJiYKLSAgICAgICAgICAgICAhKGtpbmQgJiBNQVBLSU5E
X1dSSVRFKSApCisgICAgICAgIGlmICggIShvcC0+ZmxhZ3MgJiBHTlRNQVBf
cmVhZG9ubHkpICYmIG5vZGUuY250LndyID09IDEgKQogICAgICAgICAgICAg
a2luZCA9IElPTU1VRl9yZWFkYWJsZSB8IElPTU1VRl93cml0YWJsZTsKLSAg
ICAgICAgZWxzZSBpZiAoICFraW5kICkKKyAgICAgICAgZWxzZSBpZiAoIChv
cC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkpICYmCisgICAgICAgICAgICAg
ICAgICBub2RlLmNudC5yZCA9PSAxICYmICFub2RlLmNudC53ciApCiAgICAg
ICAgICAgICBraW5kID0gSU9NTVVGX3JlYWRhYmxlOwogICAgICAgICBlbHNl
CiAgICAgICAgICAgICBraW5kID0gMDsKLSAgICAgICAgaWYgKCBraW5kICYm
IGlvbW11X2xlZ2FjeV9tYXAobGQsIF9kZm4obWZuX3gobWZuKSksIG1mbiwg
MCwga2luZCkgKQorICAgICAgICBpZiAoIGVyciB8fAorICAgICAgICAgICAg
IChraW5kICYmIGlvbW11X2xlZ2FjeV9tYXAobGQsIF9kZm4obWZuX3gobWZu
KSksIG1mbiwgMCwga2luZCkpICkKICAgICAgICAgewotICAgICAgICAgICAg
ZG91YmxlX2d0X3VubG9jayhsZ3QsIHJndCk7CisgICAgICAgICAgICBpZiAo
ICFlcnIgKQorICAgICAgICAgICAgeworICAgICAgICAgICAgICAgIGlmICgg
c2xvdCApCisgICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAg
ICBvcC0+ZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkgPyBub2RlLmNudC5yZC0t
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICA6IG5vZGUuY250LndyLS07CisgICAgICAgICAgICAgICAgICAgIHJh
ZGl4X3RyZWVfcmVwbGFjZV9zbG90KHNsb3QsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJhZGl4X3RyZWVfdWxvbmdf
dG9fcHRyKG5vZGUucmF3KSk7CisgICAgICAgICAgICAgICAgfQorICAgICAg
ICAgICAgICAgIGVsc2UKKyAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJl
ZV9kZWxldGUoJmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gobWZuKSk7Cisg
ICAgICAgICAgICB9CisKICAgICAgICAgICAgIHJjID0gR05UU1RfZ2VuZXJh
bF9lcnJvcjsKLSAgICAgICAgICAgIGdvdG8gdW5kb19vdXQ7CiAgICAgICAg
IH0KKworICAgICAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKKworICAg
ICAgICBpZiAoIHJjICE9IEdOVFNUX29rYXkgKQorICAgICAgICAgICAgZ290
byB1bmRvX291dDsKICAgICB9CiAKICAgICBUUkFDRV8xRChUUkNfTUVNX1BB
R0VfR1JBTlRfTUFQLCBvcC0+ZG9tKTsKQEAgLTEyMTMsMTAgKzEyMTAsNiBA
QCBtYXBfZ3JhbnRfcmVmKAogICAgIC8qCiAgICAgICogQWxsIG1hcHRyYWNr
IGVudHJ5IHVzZXJzIGNoZWNrIG10LT5mbGFncyBmaXJzdCBiZWZvcmUgdXNp
bmcgdGhlCiAgICAgICogb3RoZXIgZmllbGRzIHNvIGp1c3QgZW5zdXJlIHRo
ZSBmbGFncyBmaWVsZCBpcyBzdG9yZWQgbGFzdC4KLSAgICAgKgotICAgICAq
IEhvd2V2ZXIsIGlmIGdudHRhYl9uZWVkX2lvbW11X21hcHBpbmcoKSB0aGVu
IHRoaXMgd291bGQgcmFjZQotICAgICAqIHdpdGggYSBjb25jdXJyZW50IG1h
cGtpbmQoKSBjYWxsIChvbiBhbiB1bm1hcCwgZm9yIGV4YW1wbGUpCi0gICAg
ICogYW5kIGEgbG9jayBpcyByZXF1aXJlZC4KICAgICAgKi8KICAgICBtdCA9
ICZtYXB0cmFja19lbnRyeShsZ3QsIGhhbmRsZSk7CiAgICAgbXQtPmRvbWlk
ID0gb3AtPmRvbTsKQEAgLTEyMjQsOSArMTIxNyw2IEBAIG1hcF9ncmFudF9y
ZWYoCiAgICAgc21wX3dtYigpOwogICAgIHdyaXRlX2F0b21pYygmbXQtPmZs
YWdzLCBvcC0+ZmxhZ3MpOwogCi0gICAgaWYgKCBuZWVkX2lvbW11ICkKLSAg
ICAgICAgZG91YmxlX2d0X3VubG9jayhsZ3QsIHJndCk7Ci0KICAgICBvcC0+
ZGV2X2J1c19hZGRyID0gbWZuX3RvX21hZGRyKG1mbik7CiAgICAgb3AtPmhh
bmRsZSAgICAgICA9IGhhbmRsZTsKICAgICBvcC0+c3RhdHVzICAgICAgID0g
R05UU1Rfb2theTsKQEAgLTE0NDgsMTkgKzE0MzgsMzQgQEAgdW5tYXBfY29t
bW9uKAogICAgIC8qIFNlZSB0aGUgcmVzcGVjdGl2ZSBjb21tZW50IGluIG1h
cF9ncmFudF9yZWYoKS4gKi8KICAgICBpZiAoIHJjID09IEdOVFNUX29rYXkg
JiYgbGQgIT0gcmQgJiYgZ250dGFiX25lZWRfaW9tbXVfbWFwcGluZyhsZCkg
KQogICAgIHsKLSAgICAgICAgdW5zaWduZWQgaW50IGtpbmQ7CisgICAgICAg
IHZvaWQgKipzbG90OworICAgICAgICB1bmlvbiBtYXB0cmFja19ub2RlIG5v
ZGU7CiAgICAgICAgIGludCBlcnIgPSAwOwogCi0gICAgICAgIGRvdWJsZV9n
dF9sb2NrKGxndCwgcmd0KTsKKyAgICAgICAgZ3JhbnRfd3JpdGVfbG9jayhs
Z3QpOworICAgICAgICBzbG90ID0gcmFkaXhfdHJlZV9sb29rdXBfc2xvdCgm
bGd0LT5tYXB0cmFja190cmVlLCBtZm5feChvcC0+bWZuKSk7CisgICAgICAg
IG5vZGUucmF3ID0gbGlrZWx5KHNsb3QpID8gcmFkaXhfdHJlZV9wdHJfdG9f
dWxvbmcoKnNsb3QpIDogMDsKKworICAgICAgICAvKiBSZWZjb3VudCBtdXN0
IG5vdCB1bmRlcmZsb3cuICovCisgICAgICAgIGlmICggIShmbGFncyAmIEdO
VE1BUF9yZWFkb25seSA/IG5vZGUuY250LnJkLS0KKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIDogbm9kZS5jbnQud3ItLSkgKQor
ICAgICAgICAgICAgQlVHKCk7CiAKLSAgICAgICAga2luZCA9IG1hcGtpbmQo
bGd0LCByZCwgb3AtPm1mbik7Ci0gICAgICAgIGlmICggIWtpbmQgKQorICAg
ICAgICBpZiAoICFub2RlLnJhdyApCiAgICAgICAgICAgICBlcnIgPSBpb21t
dV9sZWdhY3lfdW5tYXAobGQsIF9kZm4obWZuX3gob3AtPm1mbikpLCAwKTsK
LSAgICAgICAgZWxzZSBpZiAoICEoa2luZCAmIE1BUEtJTkRfV1JJVEUpICkK
KyAgICAgICAgZWxzZSBpZiAoICEoZmxhZ3MgJiBHTlRNQVBfcmVhZG9ubHkp
ICYmICFub2RlLmNudC53ciApCiAgICAgICAgICAgICBlcnIgPSBpb21tdV9s
ZWdhY3lfbWFwKGxkLCBfZGZuKG1mbl94KG9wLT5tZm4pKSwgb3AtPm1mbiwg
MCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSU9NTVVG
X3JlYWRhYmxlKTsKIAotICAgICAgICBkb3VibGVfZ3RfdW5sb2NrKGxndCwg
cmd0KTsKKyAgICAgICAgaWYgKCBlcnIgKQorICAgICAgICAgICAgOworICAg
ICAgICBlbHNlIGlmICggIW5vZGUucmF3ICkKKyAgICAgICAgICAgIHJhZGl4
X3RyZWVfZGVsZXRlKCZsZ3QtPm1hcHRyYWNrX3RyZWUsIG1mbl94KG9wLT5t
Zm4pKTsKKyAgICAgICAgZWxzZQorICAgICAgICAgICAgcmFkaXhfdHJlZV9y
ZXBsYWNlX3Nsb3Qoc2xvdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHJhZGl4X3RyZWVfdWxvbmdfdG9fcHRyKG5vZGUucmF3KSk7
CisKKyAgICAgICAgZ3JhbnRfd3JpdGVfdW5sb2NrKGxndCk7CiAKICAgICAg
ICAgaWYgKCBlcnIgKQogICAgICAgICAgICAgcmMgPSBHTlRTVF9nZW5lcmFs
X2Vycm9yOwpAQCAtMTkxOCw2ICsxOTIzLDggQEAgaW50IGdyYW50X3RhYmxl
X2luaXQoc3RydWN0IGRvbWFpbiAqZCwgaQogICAgICAgICBndC0+bWFwdHJh
Y2sgPSB2emFsbG9jKGd0LT5tYXhfbWFwdHJhY2tfZnJhbWVzICogc2l6ZW9m
KCpndC0+bWFwdHJhY2spKTsKICAgICAgICAgaWYgKCBndC0+bWFwdHJhY2sg
PT0gTlVMTCApCiAgICAgICAgICAgICBnb3RvIG91dDsKKworICAgICAgICBy
YWRpeF90cmVlX2luaXQoJmd0LT5tYXB0cmFja190cmVlKTsKICAgICB9CiAK
ICAgICAvKiBTaGFyZWQgZ3JhbnQgdGFibGUuICovCkBAIC0zNzAxLDYgKzM3
MDgsNyBAQCBpbnQgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Moc3RydWN0IGRv
bWFpCiAgICAgZm9yICggaGFuZGxlID0gZ3QtPm1hcHRyYWNrX2xpbWl0OyBo
YW5kbGU7ICkKICAgICB7CiAgICAgICAgIHVuc2lnbmVkIGludCBjbGVhcl9m
bGFncyA9IDA7CisgICAgICAgIG1mbl90IG1mbjsKIAogICAgICAgICAvKgog
ICAgICAgICAgKiBEZWFsIHdpdGggZnVsbCBwYWdlcyBzdWNoIHRoYXQgdGhl
aXIgZnJlZWluZyAoaW4gdGhlIGJvZHkgb2YgdGhlCkBAIC0zODA2LDE3ICsz
ODE0LDMxIEBAIGludCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3Qg
ZG9tYWkKICAgICAgICAgaWYgKCBjbGVhcl9mbGFncyApCiAgICAgICAgICAg
ICBnbnR0YWJfY2xlYXJfZmxhZ3MocmQsIGNsZWFyX2ZsYWdzLCBzdGF0dXMp
OwogCisgICAgICAgIG1mbiA9IGFjdC0+bWZuOworCiAgICAgICAgIGFjdGl2
ZV9lbnRyeV9yZWxlYXNlKGFjdCk7CiAgICAgICAgIGdyYW50X3JlYWRfdW5s
b2NrKHJndCk7CiAKICAgICAgICAgcmN1X3VubG9ja19kb21haW4ocmQpOwog
CiAgICAgICAgIG1hcC0+ZmxhZ3MgPSAwOworCisgICAgICAgIC8qCisgICAg
ICAgICAqIFRoaXMgaXMgZXhjZXNzaXZlIGluIHRoYXQgYSBzaW5nbGUgc3Vj
aCBjYWxsIHdvdWxkIHN1ZmZpY2UgcGVyCisgICAgICAgICAqIG1hcHBlZCBN
Rk4gKG9yIG5vbmUgYXQgYWxsLCBpZiBubyBlbnRyeSB3YXMgZXZlciBpbnNl
cnRlZCkuIEJ1dCBpdAorICAgICAgICAgKiBzaG91bGQgYmUgdGhlIGNvbW1v
biBjYXNlIGZvciBhbiBNRk4gdG8gYmUgbWFwcGVkIGp1c3Qgb25jZSwgYW5k
CisgICAgICAgICAqIHRoaXMgd2F5IHdlIGRvbid0IG5lZWQgdG8gZnVydGhl
ciBtYWludGFpbiB0aGUgY291bnRlcnMuIFdlIGFsc28KKyAgICAgICAgICog
ZG9uJ3Qgd2FudCB0byBsZWF2ZSBjbGVhbmluZyB1cCBvZiB0aGUgdHJlZSBh
cyBhIHdob2xlIHRvIHRoZSBlbmQKKyAgICAgICAgICogb2YgdGhlIGZ1bmN0
aW9uLCBhcyB0aGlzIGNvdWxkIHRha2UgcXVpdGUgc29tZSB0aW1lLgorICAg
ICAgICAgKi8KKyAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUoJmd0LT5tYXB0
cmFja190cmVlLCBtZm5feChtZm4pKTsKICAgICB9CiAKICAgICBndC0+bWFw
dHJhY2tfbGltaXQgPSAwOwogICAgIEZSRUVfWEVOSEVBUF9QQUdFKGd0LT5t
YXB0cmFja1swXSk7CiAKKyAgICByYWRpeF90cmVlX2Rlc3Ryb3koJmd0LT5t
YXB0cmFja190cmVlLCBOVUxMKTsKKwogICAgIHJldHVybiAwOwogfQogCi0t
LSBhL3hlbi9pbmNsdWRlL3hlbi9yYWRpeC10cmVlLmgKKysrIGIveGVuL2lu
Y2x1ZGUveGVuL3JhZGl4LXRyZWUuaApAQCAtMTkwLDYgKzE5MCwyNSBAQCBz
dGF0aWMgaW5saW5lIGludCByYWRpeF90cmVlX3B0cl90b19pbnQoCiAgICAg
cmV0dXJuIChpbnQpKChsb25nKXB0ciA+PiAyKTsKIH0KIAorLyoqCisgKiBy
YWRpeF90cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfToKKyAqCisg
KiBTYW1lIGZvciB1bnNpZ25lZCBsb25nIHZhbHVlcy4gQmV3YXJlIHRob3Vn
aCB0aGF0IG9ubHkgQklUU19QRVJfTE9ORy0yCisgKiBiaXRzIGFyZSBhY3R1
YWxseSB1c2FibGUgZm9yIHRoZSB2YWx1ZS4KKyAqLworc3RhdGljIGlubGlu
ZSB2b2lkICpyYWRpeF90cmVlX3Vsb25nX3RvX3B0cih1bnNpZ25lZCBsb25n
IHZhbCkKK3sKKyAgICB1bnNpZ25lZCBsb25nIHB0ciA9ICh2YWwgPDwgMikg
fCAweDI7CisgICAgQVNTRVJUKChwdHIgPj4gMikgPT0gdmFsKTsKKyAgICBy
ZXR1cm4gKHZvaWQgKilwdHI7Cit9CisKK3N0YXRpYyBpbmxpbmUgdW5zaWdu
ZWQgbG9uZyByYWRpeF90cmVlX3B0cl90b191bG9uZyh2b2lkICpwdHIpCit7
CisgICAgQVNTRVJUKCgodW5zaWduZWQgbG9uZylwdHIgJiAweDMpID09IDB4
Mik7CisgICAgcmV0dXJuICh1bnNpZ25lZCBsb25nKXB0ciA+PiAyOworfQor
CiBpbnQgcmFkaXhfdHJlZV9pbnNlcnQoc3RydWN0IHJhZGl4X3RyZWVfcm9v
dCAqLCB1bnNpZ25lZCBsb25nLCB2b2lkICopOwogdm9pZCAqcmFkaXhfdHJl
ZV9sb29rdXAoc3RydWN0IHJhZGl4X3RyZWVfcm9vdCAqLCB1bnNpZ25lZCBs
b25nKTsKIHZvaWQgKipyYWRpeF90cmVlX2xvb2t1cF9zbG90KHN0cnVjdCBy
YWRpeF90cmVlX3Jvb3QgKiwgdW5zaWduZWQgbG9uZyk7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.14-1.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.14-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGFkZCBwcmVlbXB0aW9uIGNoZWNrIHRvIGdudHRhYl9yZWxl
YXNlX21hcHBpbmdzKCkKCkEgZ3Vlc3QgbWF5IGRpZSB3aXRoIG1hbnkgZ3Jh
bnQgbWFwcGluZ3Mgc3RpbGwgaW4gcGxhY2UsIG9yIHNpbXBseSB3aXRoCmEg
bGFyZ2UgbWFwdHJhY2sgdGFibGUuIEl0ZXJhdGluZyB0aHJvdWdoIHRoaXMg
bWF5IHRha2UgbW9yZSB0aW1lIHRoYW4KaXMgcmVhc29uYWJsZSB3aXRob3V0
IGludGVybWVkaWF0ZSBwcmVlbXB0aW9uICh0byBydW4gc29mdGlycXMgYW5k
CnBlcmhhcHMgdGhlIHNjaGVkdWxlcikuCgpNb3ZlIHRoZSBpbnZvY2F0aW9u
IG9mIHRoZSBmdW5jdGlvbiB0byB0aGUgc2VjdGlvbiB3aGVyZSBvdGhlcgpy
ZXN0YXJ0YWJsZSBmdW5jdGlvbnMgZ2V0IGludm9rZWQsIGFuZCBoYXZlIHRo
ZSBmdW5jdGlvbiBpdHNlbGYgY2hlY2sKZm9yIHByZWVtcHRpb24gZXZlcnkg
b25jZSBpbiBhIHdoaWxlLiBIYXZlIGl0IGl0ZXJhdGUgdGhlIHRhYmxlCmJh
Y2t3YXJkcywgc3VjaCB0aGF0IGRlY3JlYXNpbmcgdGhlIG1hcHRyYWNrIGxp
bWl0IGlzIGFsbCBpdCB0YWtlcyB0bwpjb252ZXkgcmVzdGFydCBpbmZvcm1h
dGlvbi4KCkluIGRvbWFpbl90ZWFyZG93bigpIGludHJvZHVjZSBQUk9HX25v
bmUgc3VjaCB0aGF0IGluc2VydGluZyBhdCB0aGUKZnJvbnQgd2lsbCBiZSBl
YXNpZXIgZ29pbmcgZm9yd2FyZC4KClRoaXMgaXMgcGFydCBvZiBDVkUtMjAy
MS0yODY5OCAvIFhTQS0zODAuCgpSZXBvcnRlZC1ieTogQW5kcmV3IENvb3Bl
ciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
SnVsaWVuIEdyYWxsIDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9j
b21tb24vZG9tYWluLmMKKysrIGIveGVuL2NvbW1vbi9kb21haW4uYwpAQCAt
NzIxLDExICs3MjEsMTMgQEAgaW50IGRvbWFpbl9raWxsKHN0cnVjdCBkb21h
aW4gKmQpCiAgICAgICAgICAgICByZXR1cm4gZG9tYWluX2tpbGwoZCk7CiAg
ICAgICAgIGQtPmlzX2R5aW5nID0gRE9NRFlJTkdfZHlpbmc7CiAgICAgICAg
IGFyZ29fZGVzdHJveShkKTsKLSAgICAgICAgZ250dGFiX3JlbGVhc2VfbWFw
cGluZ3MoZCk7CiAgICAgICAgIHZudW1hX2Rlc3Ryb3koZC0+dm51bWEpOwog
ICAgICAgICBkb21haW5fc2V0X291dHN0YW5kaW5nX3BhZ2VzKGQsIDApOwog
ICAgICAgICAvKiBmYWxsdGhyb3VnaCAqLwogICAgIGNhc2UgRE9NRFlJTkdf
ZHlpbmc6CisgICAgICAgIHJjID0gZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Mo
ZCk7CisgICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAgYnJlYWs7CiAg
ICAgICAgIHJjID0gZXZ0Y2huX2Rlc3Ryb3koZCk7CiAgICAgICAgIGlmICgg
cmMgKQogICAgICAgICAgICAgYnJlYWs7Ci0tLSBhL3hlbi9jb21tb24vZ3Jh
bnRfdGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAg
LTY0LDcgKzY0LDEzIEBAIHN0cnVjdCBncmFudF90YWJsZSB7CiAgICAgdW5z
aWduZWQgaW50ICAgICAgICAgIG5yX2dyYW50X2ZyYW1lczsKICAgICAvKiBO
dW1iZXIgb2YgZ3JhbnQgc3RhdHVzIGZyYW1lcyBzaGFyZWQgd2l0aCBndWVz
dCAoZm9yIHZlcnNpb24gMikgKi8KICAgICB1bnNpZ25lZCBpbnQgICAgICAg
ICAgbnJfc3RhdHVzX2ZyYW1lczsKLSAgICAvKiBOdW1iZXIgb2YgYXZhaWxh
YmxlIG1hcHRyYWNrIGVudHJpZXMuICovCisgICAgLyoKKyAgICAgKiBOdW1i
ZXIgb2YgYXZhaWxhYmxlIG1hcHRyYWNrIGVudHJpZXMuICBGb3IgY2xlYW51
cCBwdXJwb3NlcyBpdCBpcworICAgICAqIGltcG9ydGFudCB0byByZWFsaXpl
IHRoYXQgdGhpcyBmaWVsZCBhbmQgQG1hcHRyYWNrIGZ1cnRoZXIgZG93biB3
aWxsCisgICAgICogb25seSBldmVyIGJlIGFjY2Vzc2VkIGJ5IHRoZSBsb2Nh
bCBkb21haW4uICBUaHVzIGl0IGlzIG9rYXkgdG8gY2xlYW4KKyAgICAgKiB1
cCBlYXJseSwgYW5kIHRvIHNocmluayB0aGUgbGltaXQgZm9yIHRoZSBwdXJw
b3NlIG9mIHRyYWNraW5nIGNsZWFudXAKKyAgICAgKiBwcm9ncmVzcy4KKyAg
ICAgKi8KICAgICB1bnNpZ25lZCBpbnQgICAgICAgICAgbWFwdHJhY2tfbGlt
aXQ7CiAgICAgLyogU2hhcmVkIGdyYW50IHRhYmxlIChzZWUgaW5jbHVkZS9w
dWJsaWMvZ3JhbnRfdGFibGUuaCkuICovCiAgICAgdW5pb24gewpAQCAtMzcw
OCw5ICszNzE0LDcgQEAgZG9fZ3JhbnRfdGFibGVfb3AoCiAjaW5jbHVkZSAi
Y29tcGF0L2dyYW50X3RhYmxlLmMiCiAjZW5kaWYKIAotdm9pZAotZ250dGFi
X3JlbGVhc2VfbWFwcGluZ3MoCi0gICAgc3RydWN0IGRvbWFpbiAqZCkKK2lu
dCBnbnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkKQog
ewogICAgIHN0cnVjdCBncmFudF90YWJsZSAgICpndCA9IGQtPmdyYW50X3Rh
YmxlLCAqcmd0OwogICAgIHN0cnVjdCBncmFudF9tYXBwaW5nICptYXA7CkBA
IC0zNzI0LDEwICszNzI4LDM0IEBAIGdudHRhYl9yZWxlYXNlX21hcHBpbmdz
KAogCiAgICAgQlVHX09OKCFkLT5pc19keWluZyk7CiAKLSAgICBmb3IgKCBo
YW5kbGUgPSAwOyBoYW5kbGUgPCBndC0+bWFwdHJhY2tfbGltaXQ7IGhhbmRs
ZSsrICkKKyAgICBpZiAoICFndCB8fCAhZ3QtPm1hcHRyYWNrICkKKyAgICAg
ICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCBoYW5kbGUgPSBndC0+bWFwdHJh
Y2tfbGltaXQ7IGhhbmRsZTsgKQogICAgIHsKICAgICAgICAgdW5zaWduZWQg
aW50IGNsZWFyX2ZsYWdzID0gMDsKIAorICAgICAgICAvKgorICAgICAgICAg
KiBEZWFsIHdpdGggZnVsbCBwYWdlcyBzdWNoIHRoYXQgdGhlaXIgZnJlZWlu
ZyAoaW4gdGhlIGJvZHkgb2YgdGhlCisgICAgICAgICAqIGlmKCkpIHJlbWFp
bnMgc2ltcGxlLgorICAgICAgICAgKi8KKyAgICAgICAgaWYgKCBoYW5kbGUg
PCBndC0+bWFwdHJhY2tfbGltaXQgJiYgIShoYW5kbGUgJSBNQVBUUkFDS19Q
RVJfUEFHRSkgKQorICAgICAgICB7CisgICAgICAgICAgICAvKgorICAgICAg
ICAgICAgICogQ2hhbmdpbmcgbWFwdHJhY2tfbGltaXQgYWx0ZXJzIG5yX21h
cHRyYWNrX2ZyYW1lcygpJ2VzIHJldHVybgorICAgICAgICAgICAgICogdmFs
dWUuIEZyZWUgdGhlIHRoZW4gZXhjZXNzIHRyYWlsaW5nIHBhZ2UgcmlnaHQg
aGVyZSwgcmF0aGVyCisgICAgICAgICAgICAgKiB0aGFuIGxlYXZpbmcgaXQg
dG8gZ3JhbnRfdGFibGVfZGVzdHJveSgpIChhbmQgaW4gdHVybiByZXF1aXJp
bmcKKyAgICAgICAgICAgICAqIHRvIGxlYXZlIGd0LT5tYXB0cmFja19saW1p
dCB1bmFsdGVyZWQpLgorICAgICAgICAgICAgICovCisgICAgICAgICAgICBn
dC0+bWFwdHJhY2tfbGltaXQgPSBoYW5kbGU7CisgICAgICAgICAgICBGUkVF
X1hFTkhFQVBfUEFHRShndC0+bWFwdHJhY2tbbnJfbWFwdHJhY2tfZnJhbWVz
KGd0KV0pOworCisgICAgICAgICAgICBpZiAoIGh5cGVyY2FsbF9wcmVlbXB0
X2NoZWNrKCkgKQorICAgICAgICAgICAgICAgIHJldHVybiAtRVJFU1RBUlQ7
CisgICAgICAgIH0KKworICAgICAgICAtLWhhbmRsZTsKKwogICAgICAgICBt
YXAgPSAmbWFwdHJhY2tfZW50cnkoZ3QsIGhhbmRsZSk7CiAgICAgICAgIGlm
ICggIShtYXAtPmZsYWdzICYgKEdOVE1BUF9kZXZpY2VfbWFwfEdOVE1BUF9o
b3N0X21hcCkpICkKICAgICAgICAgICAgIGNvbnRpbnVlOwpAQCAtMzgxOCw2
ICszODQ2LDExIEBAIGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKAogCiAgICAg
ICAgIG1hcC0+ZmxhZ3MgPSAwOwogICAgIH0KKworICAgIGd0LT5tYXB0cmFj
a19saW1pdCA9IDA7CisgICAgRlJFRV9YRU5IRUFQX1BBR0UoZ3QtPm1hcHRy
YWNrWzBdKTsKKworICAgIHJldHVybiAwOwogfQogCiB2b2lkIGdyYW50X3Rh
YmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9tYWluICpkKQpAQCAt
Mzg4MSw4ICszOTE0LDcgQEAgZ3JhbnRfdGFibGVfZGVzdHJveSgKICAgICAg
ICAgZnJlZV94ZW5oZWFwX3BhZ2UodC0+c2hhcmVkX3Jhd1tpXSk7CiAgICAg
eGZyZWUodC0+c2hhcmVkX3Jhdyk7CiAKLSAgICBmb3IgKCBpID0gMDsgaSA8
IG5yX21hcHRyYWNrX2ZyYW1lcyh0KTsgaSsrICkKLSAgICAgICAgZnJlZV94
ZW5oZWFwX3BhZ2UodC0+bWFwdHJhY2tbaV0pOworICAgIEFTU0VSVCghdC0+
bWFwdHJhY2tfbGltaXQpOwogICAgIHZmcmVlKHQtPm1hcHRyYWNrKTsKIAog
ICAgIGZvciAoIGkgPSAwOyBpIDwgbnJfYWN0aXZlX2dyYW50X2ZyYW1lcyh0
KTsgaSsrICkKLS0tIGEveGVuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmgK
KysrIGIveGVuL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmgKQEAgLTQ3LDkg
KzQ3LDcgQEAgdm9pZCBncmFudF90YWJsZV9pbml0X3ZjcHUoc3RydWN0IHZj
cHUgKgogdm9pZCBncmFudF90YWJsZV93YXJuX2FjdGl2ZV9ncmFudHMoc3Ry
dWN0IGRvbWFpbiAqZCk7CiAKIC8qIERvbWFpbiBkZWF0aCByZWxlYXNlIG9m
IGdyYW50ZWQgbWFwcGluZ3Mgb2Ygb3RoZXIgZG9tYWlucycgbWVtb3J5LiAq
Lwotdm9pZAotZ250dGFiX3JlbGVhc2VfbWFwcGluZ3MoCi0gICAgc3RydWN0
IGRvbWFpbiAqZCk7CitpbnQgZ250dGFiX3JlbGVhc2VfbWFwcGluZ3Moc3Ry
dWN0IGRvbWFpbiAqZCk7CiAKIGludCBtZW1fc2hhcmluZ19ncmVmX3RvX2dm
bihzdHJ1Y3QgZ3JhbnRfdGFibGUgKmd0LCBncmFudF9yZWZfdCByZWYsCiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2ZuX3QgKmdmbiwgdWludDE2
X3QgKnN0YXR1cyk7CkBAIC03OCw3ICs3Niw3IEBAIHN0YXRpYyBpbmxpbmUg
dm9pZCBncmFudF90YWJsZV9pbml0X3ZjcHUKIAogc3RhdGljIGlubGluZSB2
b2lkIGdyYW50X3RhYmxlX3dhcm5fYWN0aXZlX2dyYW50cyhzdHJ1Y3QgZG9t
YWluICpkKSB7fQogCi1zdGF0aWMgaW5saW5lIHZvaWQgZ250dGFiX3JlbGVh
c2VfbWFwcGluZ3Moc3RydWN0IGRvbWFpbiAqZCkge30KK3N0YXRpYyBpbmxp
bmUgaW50IGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKHN0cnVjdCBkb21haW4g
KmQpIHsgcmV0dXJuIDA7IH0KIAogc3RhdGljIGlubGluZSBpbnQgbWVtX3No
YXJpbmdfZ3JlZl90b19nZm4oc3RydWN0IGdyYW50X3RhYmxlICpndCwKICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGdyYW50
X3JlZl90IHJlZiwK

--=separator
Content-Type: application/octet-stream; name="xsa380/xsa380-4.14-2.patch"
Content-Disposition: attachment; filename="xsa380/xsa380-4.14-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IHJlcGxhY2UgbWFwa2luZCgpCgptYXBraW5kKCkgZG9lc24n
dCBzY2FsZSB2ZXJ5IHdlbGwgd2l0aCBsYXJnZXIgbWFwdHJhY2sgZW50cnkg
Y291bnRzLAp1c2luZyBhIGJydXRlIGZvcmNlIGxpbmVhciBzZWFyY2ggdGhy
b3VnaCBhbGwgZW50cmllcywgd2l0aCB0aGUgb25seQpvcHRpb24gb2YgYW4g
ZWFybHkgbG9vcCBleGl0IGlmIGEgbWF0Y2hpbmcgd3JpdGFibGUgZW50cnkg
d2FzIGZvdW5kLgpJbnRyb2R1Y2UgYSByYWRpeCB0cmVlIGFsb25nc2lkZSB0
aGUgbWFpbiBtYXB0cmFjayB0YWJsZSwgdGh1cwphbGxvd2luZyBtdWNoIGZh
c3RlciBNRk4tYmFzZWQgbG9va3VwLiBUbyBhdm9pZCB0aGUgbmVlZCB0byBh
Y3R1YWxseQphbGxvY2F0ZSBzcGFjZSBmb3IgdGhlIGluZGl2aWR1YWwgbm9k
ZXMsIGVuY29kZSB0aGUgdHdvIGNvdW50ZXJzIGluIHRoZQpub2RlIHBvaW50
ZXJzIHRoZW1zZWx2ZXMsIHRodXMgbGltaXRpbmcgdGhlIG51bWJlciBvZiBw
ZXJtaXR0ZWQKc2ltdWx0YW5lb3VzIHIvbyBhbmQgci93IG1hcHBpbmdzIG9m
IHRoZSBzYW1lIE1GTiB0byAywrPCuS0xICg2NC1iaXQpIC8KMsK54oG1LTEg
KDMyLWJpdCkgZWFjaC4KClRvIGF2b2lkIGVuZm9yY2luZyBhbiB1bm5lY2Vz
c2FyaWx5IGxvdyBib3VuZCBvbiB0aGUgbnVtYmVyIG9mCnNpbXVsdGFuZW91
cyBtYXBwaW5ncyBvZiBhIHNpbmdsZSBNRk4sIGludHJvZHVjZQpyYWRpeF90
cmVlX3t1bG9uZ190b19wdHIscHRyX3RvX3Vsb25nfSBwYXJhbGxlbGluZwpy
YWRpeF90cmVlX3tpbnRfdG9fcHRyLHB0cl90b19pbnR9LgoKQXMgYSBjb25z
ZXF1ZW5jZSBsb2NraW5nIGNoYW5nZXMgYXJlIGFsc28gYXBwbGljYWJsZTog
V2l0aCB0aGVyZSBubwpsb25nZXIgYmVpbmcgYW55IGluc3BlY3Rpb24gb2Yg
dGhlIHJlbW90ZSBkb21haW4ncyBhY3RpdmUgZW50cmllcywKdGhlcmUncyBh
bHNvIG5vIG5lZWQgYW55bW9yZSB0byBob2xkIHRoZSByZW1vdGUgZG9tYWlu
J3MgZ3JhbnQgdGFibGUKbG9jay4gQW5kIHNpbmNlIHdlJ3JlIG5vIGxvbmdl
ciBpdGVyYXRpbmcgb3ZlciB0aGUgbG9jYWwgZG9tYWluJ3MgbWFwCnRyYWNr
IHRhYmxlLCB0aGUgbG9jayBpbiBtYXBfZ3JhbnRfcmVmKCkgY2FuIGFsc28g
YmUgZHJvcHBlZCBiZWZvcmUgdGhlCm5ldyBtYXB0cmFjayBlbnRyeSBhY3R1
YWxseSBnZXRzIHBvcHVsYXRlZC4KCkFzIGEgbmljZSBzaWRlIGVmZmVjdCB0
aGlzIGFsc28gcmVkdWNlcyB0aGUgbnVtYmVyIG9mIElPTU1VIG9wZXJhdGlv
bnMKaW4gdW5tYXBfY29tbW9uKCk6IFByZXZpb3VzbHkgd2Ugd291bGQgaGF2
ZSAiZXN0YWJsaXNoZWQiIGEgcmVhZGFibGUKbWFwcGluZyB3aGVuZXZlciB3
ZSBkaWRuJ3QgZmluZCBhIHdyaXRhYmxlIGVudHJ5IGFueW1vcmUgKHlldCwg
b2YKY291cnNlLCBhdCBsZWFzdCBvbmUgcmVhZGFibGUgb25lKS4gQnV0IHdl
IG9ubHkgbmVlZCB0byBkbyB0aGlzIGlmIHdlCmFjdHVhbGx5IGRyb3BwZWQg
dGhlIGxhc3Qgd3JpdGFibGUgZW50cnksIG5vdCBpZiB0aGVyZSB3ZXJlIG5v
bmUgYWxyZWFkeQpiZWZvcmUuCgpUaGlzIGlzIHBhcnQgb2YgQ1ZFLTIwMjEt
Mjg2OTggLyBYU0EtMzgwLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTogSnVsaWVuIEdyYWxs
IDxqZ3JhbGxAYW1hem9uLmNvbT4KCi0tLSBhL3hlbi9jb21tb24vZ3JhbnRf
dGFibGUuYworKysgYi94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKQEAgLTM3
LDYgKzM3LDcgQEAKICNpbmNsdWRlIDx4ZW4vaW9tbXUuaD4KICNpbmNsdWRl
IDx4ZW4vcGFnaW5nLmg+CiAjaW5jbHVkZSA8eGVuL2tleWhhbmRsZXIuaD4K
KyNpbmNsdWRlIDx4ZW4vcmFkaXgtdHJlZS5oPgogI2luY2x1ZGUgPHhlbi92
bWFwLmg+CiAjaW5jbHVkZSA8eGVuL25vc3BlYy5oPgogI2luY2x1ZGUgPHhz
bS94c20uaD4KQEAgLTgyLDggKzgzLDEzIEBAIHN0cnVjdCBncmFudF90YWJs
ZSB7CiAgICAgZ3JhbnRfc3RhdHVzX3QgICAgICAgKipzdGF0dXM7CiAgICAg
LyogQWN0aXZlIGdyYW50IHRhYmxlLiAqLwogICAgIHN0cnVjdCBhY3RpdmVf
Z3JhbnRfZW50cnkgKiphY3RpdmU7Ci0gICAgLyogTWFwcGluZyB0cmFja2lu
ZyB0YWJsZSBwZXIgdmNwdS4gKi8KKyAgICAvKiBIYW5kbGUtaW5kZXhlZCB0
cmFja2luZyB0YWJsZSBvZiBtYXBwaW5ncy4gKi8KICAgICBzdHJ1Y3QgZ3Jh
bnRfbWFwcGluZyAqKm1hcHRyYWNrOworICAgIC8qCisgICAgICogTUZOLWlu
ZGV4ZWQgdHJhY2tpbmcgdHJlZSBvZiBtYXBwaW5ncywgaWYgbmVlZGVkLiAg
Tm90ZSB0aGF0IHRoaXMgaXMKKyAgICAgKiBwcm90ZWN0ZWQgYnkgQGxvY2ss
IG5vdCBAbWFwdHJhY2tfbG9jay4KKyAgICAgKi8KKyAgICBzdHJ1Y3QgcmFk
aXhfdHJlZV9yb290IG1hcHRyYWNrX3RyZWU7CiAKICAgICAvKiBEb21haW4g
dG8gd2hpY2ggdGhpcyBzdHJ1Y3QgZ3JhbnRfdGFibGUgYmVsb25ncy4gKi8K
ICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluICpkb21haW47CkBAIC01MDEsMzQg
KzUwNyw2IEBAIHN0YXRpYyBpbnQgZ2V0X3BhZ2VkX2ZyYW1lKHVuc2lnbmVk
IGxvbmcKICAgICByZXR1cm4gR05UU1Rfb2theTsKIH0KIAotc3RhdGljIGlu
bGluZSB2b2lkCi1kb3VibGVfZ3RfbG9jayhzdHJ1Y3QgZ3JhbnRfdGFibGUg
KmxndCwgc3RydWN0IGdyYW50X3RhYmxlICpyZ3QpCi17Ci0gICAgLyoKLSAg
ICAgKiBTZWUgbWFwa2luZCgpIGZvciB3aHkgdGhlIHdyaXRlIGxvY2sgaXMg
YWxzbyByZXF1aXJlZCBmb3IgdGhlCi0gICAgICogcmVtb3RlIGRvbWFpbi4K
LSAgICAgKi8KLSAgICBpZiAoIGxndCA8IHJndCApCi0gICAgewotICAgICAg
ICBncmFudF93cml0ZV9sb2NrKGxndCk7Ci0gICAgICAgIGdyYW50X3dyaXRl
X2xvY2socmd0KTsKLSAgICB9Ci0gICAgZWxzZQotICAgIHsKLSAgICAgICAg
aWYgKCBsZ3QgIT0gcmd0ICkKLSAgICAgICAgICAgIGdyYW50X3dyaXRlX2xv
Y2socmd0KTsKLSAgICAgICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOwotICAg
IH0KLX0KLQotc3RhdGljIGlubGluZSB2b2lkCi1kb3VibGVfZ3RfdW5sb2Nr
KHN0cnVjdCBncmFudF90YWJsZSAqbGd0LCBzdHJ1Y3QgZ3JhbnRfdGFibGUg
KnJndCkKLXsKLSAgICBncmFudF93cml0ZV91bmxvY2sobGd0KTsKLSAgICBp
ZiAoIGxndCAhPSByZ3QgKQotICAgICAgICBncmFudF93cml0ZV91bmxvY2so
cmd0KTsKLX0KLQogI2RlZmluZSBJTlZBTElEX01BUFRSQUNLX0hBTkRMRSBV
SU5UX01BWAogCiBzdGF0aWMgaW5saW5lIGdyYW50X2hhbmRsZV90CkBAIC05
NDgsNDEgKzkyNiwxNyBAQCBzdGF0aWMgc3RydWN0IGFjdGl2ZV9ncmFudF9l
bnRyeSAqZ3JhbnRfCiAgICAgcmV0dXJuIEVSUl9QVFIoLUVJTlZBTCk7CiB9
CiAKLSNkZWZpbmUgTUFQS0lORF9SRUFEIDEKLSNkZWZpbmUgTUFQS0lORF9X
UklURSAyCi1zdGF0aWMgdW5zaWduZWQgaW50IG1hcGtpbmQoCi0gICAgc3Ry
dWN0IGdyYW50X3RhYmxlICpsZ3QsIGNvbnN0IHN0cnVjdCBkb21haW4gKnJk
LCBtZm5fdCBtZm4pCi17Ci0gICAgc3RydWN0IGdyYW50X21hcHBpbmcgKm1h
cDsKLSAgICBncmFudF9oYW5kbGVfdCBoYW5kbGUsIGxpbWl0ID0gbGd0LT5t
YXB0cmFja19saW1pdDsKLSAgICB1bnNpZ25lZCBpbnQga2luZCA9IDA7Ci0K
LSAgICAvKgotICAgICAqIE11c3QgaGF2ZSB0aGUgbG9jYWwgZG9tYWluJ3Mg
Z3JhbnQgdGFibGUgd3JpdGUgbG9jayB3aGVuCi0gICAgICogaXRlcmF0aW5n
IG92ZXIgaXRzIG1hcHRyYWNrIGVudHJpZXMuCi0gICAgICovCi0gICAgQVNT
RVJUKHBlcmNwdV9yd19pc193cml0ZV9sb2NrZWQoJmxndC0+bG9jaykpOwot
ICAgIC8qCi0gICAgICogTXVzdCBoYXZlIHRoZSByZW1vdGUgZG9tYWluJ3Mg
Z3JhbnQgdGFibGUgd3JpdGUgbG9jayB3aGlsZQotICAgICAqIGNvdW50aW5n
IGl0cyBhY3RpdmUgZW50cmllcy4KLSAgICAgKi8KLSAgICBBU1NFUlQocGVy
Y3B1X3J3X2lzX3dyaXRlX2xvY2tlZCgmcmQtPmdyYW50X3RhYmxlLT5sb2Nr
KSk7Ci0KLSAgICBzbXBfcm1iKCk7Ci0KLSAgICBmb3IgKCBoYW5kbGUgPSAw
OyAhKGtpbmQgJiBNQVBLSU5EX1dSSVRFKSAmJiBoYW5kbGUgPCBsaW1pdDsg
aGFuZGxlKysgKQotICAgIHsKLSAgICAgICAgbWFwID0gJm1hcHRyYWNrX2Vu
dHJ5KGxndCwgaGFuZGxlKTsKLSAgICAgICAgaWYgKCAhKG1hcC0+ZmxhZ3Mg
JiAoR05UTUFQX2RldmljZV9tYXB8R05UTUFQX2hvc3RfbWFwKSkgfHwKLSAg
ICAgICAgICAgICBtYXAtPmRvbWlkICE9IHJkLT5kb21haW5faWQgKQotICAg
ICAgICAgICAgY29udGludWU7Ci0gICAgICAgIGlmICggbWZuX2VxKF9hY3Rp
dmVfZW50cnkocmQtPmdyYW50X3RhYmxlLCBtYXAtPnJlZikubWZuLCBtZm4p
ICkKLSAgICAgICAgICAgIGtpbmQgfD0gbWFwLT5mbGFncyAmIEdOVE1BUF9y
ZWFkb25seSA/Ci0gICAgICAgICAgICAgICAgICAgIE1BUEtJTkRfUkVBRCA6
IE1BUEtJTkRfV1JJVEU7Ci0gICAgfQotCi0gICAgcmV0dXJuIGtpbmQ7Ci19
Cit1bmlvbiBtYXB0cmFja19ub2RlIHsKKyAgICBzdHJ1Y3QgeworICAgICAg
ICAvKiBSYWRpeCB0cmVlIHNsb3QgcG9pbnRlcnMgdXNlIHR3byBvZiB0aGUg
Yml0cy4gKi8KKyNpZmRlZiBfX0JJR19FTkRJQU5fQklURklFTEQKKyAgICAg
ICAgdW5zaWduZWQgbG9uZyAgICA6IDI7CisjZW5kaWYKKyAgICAgICAgdW5z
aWduZWQgbG9uZyByZCA6IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsKKyAgICAg
ICAgdW5zaWduZWQgbG9uZyB3ciA6IEJJVFNfUEVSX0xPTkcgLyAyIC0gMTsK
KyAgICB9IGNudDsKKyAgICB1bnNpZ25lZCBsb25nIHJhdzsKK307CiAKIHN0
YXRpYyB2b2lkCiBtYXBfZ3JhbnRfcmVmKApAQCAtMTAwMSw3ICs5NTUsNiBA
QCBtYXBfZ3JhbnRfcmVmKAogICAgIHN0cnVjdCBncmFudF9tYXBwaW5nICpt
dDsKICAgICBncmFudF9lbnRyeV9oZWFkZXJfdCAqc2hhaDsKICAgICB1aW50
MTZfdCAqc3RhdHVzOwotICAgIGJvb2xfdCBuZWVkX2lvbW11OwogCiAgICAg
bGQgPSBjdXJyZW50LT5kb21haW47CiAKQEAgLTEyMjAsMzEgKzExNzMsNzUg
QEAgbWFwX2dyYW50X3JlZigKICAgICAgKiBhcyBtZW0tc2hhcmluZyBhbmQg
SU9NTVUgdXNlIGFyZSBpbmNvbXBhdGlibGUpLiBUaGUgZG9tX2lvIGNhc2Ug
d291bGQKICAgICAgKiBuZWVkIGNoZWNraW5nIHNlcGFyYXRlbHkgaWYgd2Ug
Y29tcGFyZWQgYWdhaW5zdCBvd25lciBoZXJlLgogICAgICAqLwotICAgIG5l
ZWRfaW9tbXUgPSBsZCAhPSByZCAmJiBnbnR0YWJfbmVlZF9pb21tdV9tYXBw
aW5nKGxkKTsKLSAgICBpZiAoIG5lZWRfaW9tbXUgKQorICAgIGlmICggbGQg
IT0gcmQgJiYgZ250dGFiX25lZWRfaW9tbXVfbWFwcGluZyhsZCkgKQogICAg
IHsKKyAgICAgICAgdW5pb24gbWFwdHJhY2tfbm9kZSBub2RlID0geworICAg
ICAgICAgICAgLmNudC5yZCA9ICEhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFk
b25seSksCisgICAgICAgICAgICAuY250LndyID0gIShvcC0+ZmxhZ3MgJiBH
TlRNQVBfcmVhZG9ubHkpLAorICAgICAgICB9OworICAgICAgICBpbnQgZXJy
OworICAgICAgICB2b2lkICoqc2xvdCA9IE5VTEw7CiAgICAgICAgIHVuc2ln
bmVkIGludCBraW5kOwogCi0gICAgICAgIGRvdWJsZV9ndF9sb2NrKGxndCwg
cmd0KTsKKyAgICAgICAgZ3JhbnRfd3JpdGVfbG9jayhsZ3QpOworCisgICAg
ICAgIGVyciA9IHJhZGl4X3RyZWVfaW5zZXJ0KCZsZ3QtPm1hcHRyYWNrX3Ry
ZWUsIG1mbl94KG1mbiksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHJhZGl4X3RyZWVfdWxvbmdfdG9fcHRyKG5vZGUucmF3KSk7CisgICAg
ICAgIGlmICggZXJyID09IC1FRVhJU1QgKQorICAgICAgICB7CisgICAgICAg
ICAgICBzbG90ID0gcmFkaXhfdHJlZV9sb29rdXBfc2xvdCgmbGd0LT5tYXB0
cmFja190cmVlLCBtZm5feChtZm4pKTsKKyAgICAgICAgICAgIGlmICggbGlr
ZWx5KHNsb3QpICkKKyAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICBu
b2RlLnJhdyA9IHJhZGl4X3RyZWVfcHRyX3RvX3Vsb25nKCpzbG90KTsKKyAg
ICAgICAgICAgICAgICBlcnIgPSAtRUJVU1k7CisKKyAgICAgICAgICAgICAg
ICAvKiBVcGRhdGUgbm9kZSBvbmx5IHdoZW4gcmVmY291bnQgZG9lc24ndCBv
dmVyZmxvdy4gKi8KKyAgICAgICAgICAgICAgICBpZiAoIG9wLT5mbGFncyAm
IEdOVE1BUF9yZWFkb25seSA/ICsrbm9kZS5jbnQucmQKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6ICsrbm9k
ZS5jbnQud3IgKQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAg
ICAgICAgcmFkaXhfdHJlZV9yZXBsYWNlX3Nsb3Qoc2xvdCwKKyAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJl
ZV91bG9uZ190b19wdHIobm9kZS5yYXcpKTsKKyAgICAgICAgICAgICAgICAg
ICAgZXJyID0gMDsKKyAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICB9
CisgICAgICAgICAgICBlbHNlCisgICAgICAgICAgICAgICAgQVNTRVJUX1VO
UkVBQ0hBQkxFKCk7CisgICAgICAgIH0KIAogICAgICAgICAvKgogICAgICAg
ICAgKiBXZSdyZSBub3QgdHJhbnNsYXRlZCwgc28gd2Uga25vdyB0aGF0IGRm
bnMgYW5kIG1mbnMgYXJlCiAgICAgICAgICAqIHRoZSBzYW1lIHRoaW5ncywg
c28gdGhlIElPTU1VIGVudHJ5IGlzIGFsd2F5cyAxLXRvLTEuCiAgICAgICAg
ICAqLwotICAgICAgICBraW5kID0gbWFwa2luZChsZ3QsIHJkLCBtZm4pOwot
ICAgICAgICBpZiAoICEob3AtPmZsYWdzICYgR05UTUFQX3JlYWRvbmx5KSAm
JgotICAgICAgICAgICAgICEoa2luZCAmIE1BUEtJTkRfV1JJVEUpICkKKyAg
ICAgICAgaWYgKCAhKG9wLT5mbGFncyAmIEdOVE1BUF9yZWFkb25seSkgJiYg
bm9kZS5jbnQud3IgPT0gMSApCiAgICAgICAgICAgICBraW5kID0gSU9NTVVG
X3JlYWRhYmxlIHwgSU9NTVVGX3dyaXRhYmxlOwotICAgICAgICBlbHNlIGlm
ICggIWtpbmQgKQorICAgICAgICBlbHNlIGlmICggKG9wLT5mbGFncyAmIEdO
VE1BUF9yZWFkb25seSkgJiYKKyAgICAgICAgICAgICAgICAgIG5vZGUuY250
LnJkID09IDEgJiYgIW5vZGUuY250LndyICkKICAgICAgICAgICAgIGtpbmQg
PSBJT01NVUZfcmVhZGFibGU7CiAgICAgICAgIGVsc2UKICAgICAgICAgICAg
IGtpbmQgPSAwOwotICAgICAgICBpZiAoIGtpbmQgJiYgaW9tbXVfbGVnYWN5
X21hcChsZCwgX2RmbihtZm5feChtZm4pKSwgbWZuLCAwLCBraW5kKSApCisg
ICAgICAgIGlmICggZXJyIHx8CisgICAgICAgICAgICAgKGtpbmQgJiYgaW9t
bXVfbGVnYWN5X21hcChsZCwgX2RmbihtZm5feChtZm4pKSwgbWZuLCAwLCBr
aW5kKSkgKQogICAgICAgICB7Ci0gICAgICAgICAgICBkb3VibGVfZ3RfdW5s
b2NrKGxndCwgcmd0KTsKKyAgICAgICAgICAgIGlmICggIWVyciApCisgICAg
ICAgICAgICB7CisgICAgICAgICAgICAgICAgaWYgKCBzbG90ICkKKyAgICAg
ICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgICAgIG9wLT5mbGFncyAm
IEdOVE1BUF9yZWFkb25seSA/IG5vZGUuY250LnJkLS0KKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDogbm9kZS5j
bnQud3ItLTsKKyAgICAgICAgICAgICAgICAgICAgcmFkaXhfdHJlZV9yZXBs
YWNlX3Nsb3Qoc2xvdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcmFkaXhfdHJlZV91bG9uZ190b19wdHIobm9kZS5y
YXcpKTsKKyAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICAgICAgZWxz
ZQorICAgICAgICAgICAgICAgICAgICByYWRpeF90cmVlX2RlbGV0ZSgmbGd0
LT5tYXB0cmFja190cmVlLCBtZm5feChtZm4pKTsKKyAgICAgICAgICAgIH0K
KwogICAgICAgICAgICAgcmMgPSBHTlRTVF9nZW5lcmFsX2Vycm9yOwotICAg
ICAgICAgICAgZ290byB1bmRvX291dDsKICAgICAgICAgfQorCisgICAgICAg
IGdyYW50X3dyaXRlX3VubG9jayhsZ3QpOworCisgICAgICAgIGlmICggcmMg
IT0gR05UU1Rfb2theSApCisgICAgICAgICAgICBnb3RvIHVuZG9fb3V0Owog
ICAgIH0KIAogICAgIFRSQUNFXzFEKFRSQ19NRU1fUEFHRV9HUkFOVF9NQVAs
IG9wLT5kb20pOwpAQCAtMTI1MiwxMCArMTI0OSw2IEBAIG1hcF9ncmFudF9y
ZWYoCiAgICAgLyoKICAgICAgKiBBbGwgbWFwdHJhY2sgZW50cnkgdXNlcnMg
Y2hlY2sgbXQtPmZsYWdzIGZpcnN0IGJlZm9yZSB1c2luZyB0aGUKICAgICAg
KiBvdGhlciBmaWVsZHMgc28ganVzdCBlbnN1cmUgdGhlIGZsYWdzIGZpZWxk
IGlzIHN0b3JlZCBsYXN0LgotICAgICAqCi0gICAgICogSG93ZXZlciwgaWYg
Z250dGFiX25lZWRfaW9tbXVfbWFwcGluZygpIHRoZW4gdGhpcyB3b3VsZCBy
YWNlCi0gICAgICogd2l0aCBhIGNvbmN1cnJlbnQgbWFwa2luZCgpIGNhbGwg
KG9uIGFuIHVubWFwLCBmb3IgZXhhbXBsZSkKLSAgICAgKiBhbmQgYSBsb2Nr
IGlzIHJlcXVpcmVkLgogICAgICAqLwogICAgIG10ID0gJm1hcHRyYWNrX2Vu
dHJ5KGxndCwgaGFuZGxlKTsKICAgICBtdC0+ZG9taWQgPSBvcC0+ZG9tOwpA
QCAtMTI2Myw5ICsxMjU2LDYgQEAgbWFwX2dyYW50X3JlZigKICAgICBzbXBf
d21iKCk7CiAgICAgd3JpdGVfYXRvbWljKCZtdC0+ZmxhZ3MsIG9wLT5mbGFn
cyk7CiAKLSAgICBpZiAoIG5lZWRfaW9tbXUgKQotICAgICAgICBkb3VibGVf
Z3RfdW5sb2NrKGxndCwgcmd0KTsKLQogICAgIG9wLT5kZXZfYnVzX2FkZHIg
PSBtZm5fdG9fbWFkZHIobWZuKTsKICAgICBvcC0+aGFuZGxlICAgICAgID0g
aGFuZGxlOwogICAgIG9wLT5zdGF0dXMgICAgICAgPSBHTlRTVF9va2F5OwpA
QCAtMTQ4NywxOSArMTQ3NywzNCBAQCB1bm1hcF9jb21tb24oCiAgICAgLyog
U2VlIHRoZSByZXNwZWN0aXZlIGNvbW1lbnQgaW4gbWFwX2dyYW50X3JlZigp
LiAqLwogICAgIGlmICggcmMgPT0gR05UU1Rfb2theSAmJiBsZCAhPSByZCAm
JiBnbnR0YWJfbmVlZF9pb21tdV9tYXBwaW5nKGxkKSApCiAgICAgewotICAg
ICAgICB1bnNpZ25lZCBpbnQga2luZDsKKyAgICAgICAgdm9pZCAqKnNsb3Q7
CisgICAgICAgIHVuaW9uIG1hcHRyYWNrX25vZGUgbm9kZTsKICAgICAgICAg
aW50IGVyciA9IDA7CiAKLSAgICAgICAgZG91YmxlX2d0X2xvY2sobGd0LCBy
Z3QpOworICAgICAgICBncmFudF93cml0ZV9sb2NrKGxndCk7CisgICAgICAg
IHNsb3QgPSByYWRpeF90cmVlX2xvb2t1cF9zbG90KCZsZ3QtPm1hcHRyYWNr
X3RyZWUsIG1mbl94KG9wLT5tZm4pKTsKKyAgICAgICAgbm9kZS5yYXcgPSBs
aWtlbHkoc2xvdCkgPyByYWRpeF90cmVlX3B0cl90b191bG9uZygqc2xvdCkg
OiAwOworCisgICAgICAgIC8qIFJlZmNvdW50IG11c3Qgbm90IHVuZGVyZmxv
dy4gKi8KKyAgICAgICAgaWYgKCAhKGZsYWdzICYgR05UTUFQX3JlYWRvbmx5
ID8gbm9kZS5jbnQucmQtLQorICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgOiBub2RlLmNudC53ci0tKSApCisgICAgICAgICAgICBC
VUcoKTsKIAotICAgICAgICBraW5kID0gbWFwa2luZChsZ3QsIHJkLCBvcC0+
bWZuKTsKLSAgICAgICAgaWYgKCAha2luZCApCisgICAgICAgIGlmICggIW5v
ZGUucmF3ICkKICAgICAgICAgICAgIGVyciA9IGlvbW11X2xlZ2FjeV91bm1h
cChsZCwgX2RmbihtZm5feChvcC0+bWZuKSksIDApOwotICAgICAgICBlbHNl
IGlmICggIShraW5kICYgTUFQS0lORF9XUklURSkgKQorICAgICAgICBlbHNl
IGlmICggIShmbGFncyAmIEdOVE1BUF9yZWFkb25seSkgJiYgIW5vZGUuY250
LndyICkKICAgICAgICAgICAgIGVyciA9IGlvbW11X2xlZ2FjeV9tYXAobGQs
IF9kZm4obWZuX3gob3AtPm1mbikpLCBvcC0+bWZuLCAwLAogICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBJT01NVUZfcmVhZGFibGUpOwog
Ci0gICAgICAgIGRvdWJsZV9ndF91bmxvY2sobGd0LCByZ3QpOworICAgICAg
ICBpZiAoIGVyciApCisgICAgICAgICAgICA7CisgICAgICAgIGVsc2UgaWYg
KCAhbm9kZS5yYXcgKQorICAgICAgICAgICAgcmFkaXhfdHJlZV9kZWxldGUo
JmxndC0+bWFwdHJhY2tfdHJlZSwgbWZuX3gob3AtPm1mbikpOworICAgICAg
ICBlbHNlCisgICAgICAgICAgICByYWRpeF90cmVlX3JlcGxhY2Vfc2xvdChz
bG90LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmFk
aXhfdHJlZV91bG9uZ190b19wdHIobm9kZS5yYXcpKTsKKworICAgICAgICBn
cmFudF93cml0ZV91bmxvY2sobGd0KTsKIAogICAgICAgICBpZiAoIGVyciAp
CiAgICAgICAgICAgICByYyA9IEdOVFNUX2dlbmVyYWxfZXJyb3I7CkBAIC0x
OTUxLDYgKzE5NTYsOCBAQCBpbnQgZ3JhbnRfdGFibGVfaW5pdChzdHJ1Y3Qg
ZG9tYWluICpkLCBpCiAgICAgICAgIGd0LT5tYXB0cmFjayA9IHZ6YWxsb2Mo
Z3QtPm1heF9tYXB0cmFja19mcmFtZXMgKiBzaXplb2YoKmd0LT5tYXB0cmFj
aykpOwogICAgICAgICBpZiAoIGd0LT5tYXB0cmFjayA9PSBOVUxMICkKICAg
ICAgICAgICAgIGdvdG8gb3V0OworCisgICAgICAgIHJhZGl4X3RyZWVfaW5p
dCgmZ3QtPm1hcHRyYWNrX3RyZWUpOwogICAgIH0KIAogICAgIC8qIFNoYXJl
ZCBncmFudCB0YWJsZS4gKi8KQEAgLTM3MzQsNiArMzc0MSw3IEBAIGludCBn
bnR0YWJfcmVsZWFzZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWkKICAgICBmb3Ig
KCBoYW5kbGUgPSBndC0+bWFwdHJhY2tfbGltaXQ7IGhhbmRsZTsgKQogICAg
IHsKICAgICAgICAgdW5zaWduZWQgaW50IGNsZWFyX2ZsYWdzID0gMDsKKyAg
ICAgICAgbWZuX3QgbWZuOwogCiAgICAgICAgIC8qCiAgICAgICAgICAqIERl
YWwgd2l0aCBmdWxsIHBhZ2VzIHN1Y2ggdGhhdCB0aGVpciBmcmVlaW5nIChp
biB0aGUgYm9keSBvZiB0aGUKQEAgLTM4MzksMTcgKzM4NDcsMzEgQEAgaW50
IGdudHRhYl9yZWxlYXNlX21hcHBpbmdzKHN0cnVjdCBkb21haQogICAgICAg
ICBpZiAoIGNsZWFyX2ZsYWdzICkKICAgICAgICAgICAgIGdudHRhYl9jbGVh
cl9mbGFncyhyZCwgY2xlYXJfZmxhZ3MsIHN0YXR1cyk7CiAKKyAgICAgICAg
bWZuID0gYWN0LT5tZm47CisKICAgICAgICAgYWN0aXZlX2VudHJ5X3JlbGVh
c2UoYWN0KTsKICAgICAgICAgZ3JhbnRfcmVhZF91bmxvY2socmd0KTsKIAog
ICAgICAgICByY3VfdW5sb2NrX2RvbWFpbihyZCk7CiAKICAgICAgICAgbWFw
LT5mbGFncyA9IDA7CisKKyAgICAgICAgLyoKKyAgICAgICAgICogVGhpcyBp
cyBleGNlc3NpdmUgaW4gdGhhdCBhIHNpbmdsZSBzdWNoIGNhbGwgd291bGQg
c3VmZmljZSBwZXIKKyAgICAgICAgICogbWFwcGVkIE1GTiAob3Igbm9uZSBh
dCBhbGwsIGlmIG5vIGVudHJ5IHdhcyBldmVyIGluc2VydGVkKS4gQnV0IGl0
CisgICAgICAgICAqIHNob3VsZCBiZSB0aGUgY29tbW9uIGNhc2UgZm9yIGFu
IE1GTiB0byBiZSBtYXBwZWQganVzdCBvbmNlLCBhbmQKKyAgICAgICAgICog
dGhpcyB3YXkgd2UgZG9uJ3QgbmVlZCB0byBmdXJ0aGVyIG1haW50YWluIHRo
ZSBjb3VudGVycy4gV2UgYWxzbworICAgICAgICAgKiBkb24ndCB3YW50IHRv
IGxlYXZlIGNsZWFuaW5nIHVwIG9mIHRoZSB0cmVlIGFzIGEgd2hvbGUgdG8g
dGhlIGVuZAorICAgICAgICAgKiBvZiB0aGUgZnVuY3Rpb24sIGFzIHRoaXMg
Y291bGQgdGFrZSBxdWl0ZSBzb21lIHRpbWUuCisgICAgICAgICAqLworICAg
ICAgICByYWRpeF90cmVlX2RlbGV0ZSgmZ3QtPm1hcHRyYWNrX3RyZWUsIG1m
bl94KG1mbikpOwogICAgIH0KIAogICAgIGd0LT5tYXB0cmFja19saW1pdCA9
IDA7CiAgICAgRlJFRV9YRU5IRUFQX1BBR0UoZ3QtPm1hcHRyYWNrWzBdKTsK
IAorICAgIHJhZGl4X3RyZWVfZGVzdHJveSgmZ3QtPm1hcHRyYWNrX3RyZWUs
IE5VTEwpOworCiAgICAgcmV0dXJuIDA7CiB9CiAKLS0tIGEveGVuL2luY2x1
ZGUveGVuL3JhZGl4LXRyZWUuaAorKysgYi94ZW4vaW5jbHVkZS94ZW4vcmFk
aXgtdHJlZS5oCkBAIC0xOTAsNiArMTkwLDI1IEBAIHN0YXRpYyBpbmxpbmUg
aW50IHJhZGl4X3RyZWVfcHRyX3RvX2ludCgKICAgICByZXR1cm4gKGludCko
KGxvbmcpcHRyID4+IDIpOwogfQogCisvKioKKyAqIHJhZGl4X3RyZWVfe3Vs
b25nX3RvX3B0cixwdHJfdG9fdWxvbmd9OgorICoKKyAqIFNhbWUgZm9yIHVu
c2lnbmVkIGxvbmcgdmFsdWVzLiBCZXdhcmUgdGhvdWdoIHRoYXQgb25seSBC
SVRTX1BFUl9MT05HLTIKKyAqIGJpdHMgYXJlIGFjdHVhbGx5IHVzYWJsZSBm
b3IgdGhlIHZhbHVlLgorICovCitzdGF0aWMgaW5saW5lIHZvaWQgKnJhZGl4
X3RyZWVfdWxvbmdfdG9fcHRyKHVuc2lnbmVkIGxvbmcgdmFsKQoreworICAg
IHVuc2lnbmVkIGxvbmcgcHRyID0gKHZhbCA8PCAyKSB8IDB4MjsKKyAgICBB
U1NFUlQoKHB0ciA+PiAyKSA9PSB2YWwpOworICAgIHJldHVybiAodm9pZCAq
KXB0cjsKK30KKworc3RhdGljIGlubGluZSB1bnNpZ25lZCBsb25nIHJhZGl4
X3RyZWVfcHRyX3RvX3Vsb25nKHZvaWQgKnB0cikKK3sKKyAgICBBU1NFUlQo
KCh1bnNpZ25lZCBsb25nKXB0ciAmIDB4MykgPT0gMHgyKTsKKyAgICByZXR1
cm4gKHVuc2lnbmVkIGxvbmcpcHRyID4+IDI7Cit9CisKIGludCByYWRpeF90
cmVlX2luc2VydChzdHJ1Y3QgcmFkaXhfdHJlZV9yb290ICosIHVuc2lnbmVk
IGxvbmcsIHZvaWQgKik7CiB2b2lkICpyYWRpeF90cmVlX2xvb2t1cChzdHJ1
Y3QgcmFkaXhfdHJlZV9yb290ICosIHVuc2lnbmVkIGxvbmcpOwogdm9pZCAq
KnJhZGl4X3RyZWVfbG9va3VwX3Nsb3Qoc3RydWN0IHJhZGl4X3RyZWVfcm9v
dCAqLCB1bnNpZ25lZCBsb25nKTsK

--=separator--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:32:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:32:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176195.320669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMbs-0000MC-G9; Wed, 01 Sep 2021 09:32:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176195.320669; Wed, 01 Sep 2021 09:32:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMbs-0000M5-D9; Wed, 01 Sep 2021 09:32:44 +0000
Received: by outflank-mailman (input) for mailman id 176195;
 Wed, 01 Sep 2021 09:32:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLMbq-0000LW-Ok; Wed, 01 Sep 2021 09:32:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLMbq-00073c-Hy; Wed, 01 Sep 2021 09:32:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLMbp-00014L-98; Wed, 01 Sep 2021 09:32:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLMbp-00014r-8b; Wed, 01 Sep 2021 09:32:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EdBIv1pfTVLmatmwiIhji2JZc4VYFRPftioRwfnDEnI=; b=1vLvU3zXy2fwphOWsZvpsqiMKz
	ZygFPPoMyiKsBcC4TW0CD49mNK5m/ayPDNdjaDQGNswb6mS12xpPLgGTl8lFerOTI3m2BgXm0PJCt
	MHU1ve3WZRAlcywhx3wUF0R/X6NxUVkpZffvo1HgqiflSrXOZ5Ti27jlz0gs69jlw0jI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164676-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.11-testing test] 164676: regressions - FAIL
X-Osstest-Failures:
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-ovmf-amd64:debian-hvm-install:fail:regression
    xen-4.11-testing:test-amd64-i386-xl-qemuu-ovmf-amd64:debian-hvm-install:fail:regression
    xen-4.11-testing:test-arm64-arm64-xl-credit2:xen-boot:fail:heisenbug
    xen-4.11-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:debian-hvm-install:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:debian-hvm-install:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ec4654ce64d87f14567008cfb42568fd434f1bed
X-Osstest-Versions-That:
    xen=ef32c7afa2731b758226d6e10a1e489b1a15fc41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 09:32:41 +0000

flight 164676 xen-4.11-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164676/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162548
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162548

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   8 xen-boot                   fail pass in 164625

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2 15 migrate-support-check fail in 164625 never pass
 test-arm64-arm64-xl-credit2 16 saverestore-support-check fail in 164625 never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 12 debian-hvm-install fail like 162548
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 12 debian-hvm-install fail like 162548
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 162548
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 162548
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 162548
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 162548
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 162548
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 162548
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 162548
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 162548
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 162548
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 162548
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 162548
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 xen                  ec4654ce64d87f14567008cfb42568fd434f1bed
baseline version:
 xen                  ef32c7afa2731b758226d6e10a1e489b1a15fc41

Last test of basis   162548  2021-06-08 18:37:00 Z   84 days
Testing same since   164486  2021-08-25 14:05:55 Z    6 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         fail    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        fail    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 450 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:33:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:33:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176159.320683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMcg-0001B2-0C; Wed, 01 Sep 2021 09:33:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176159.320683; Wed, 01 Sep 2021 09:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMcf-0001Av-Rl; Wed, 01 Sep 2021 09:33:33 +0000
Received: by outflank-mailman (input) for mailman id 176159;
 Wed, 01 Sep 2021 09:31:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tzcz=NX=xenbits.xen.org=iwj@srs-us1.protection.inumbo.net>)
 id 1mLMaL-00066x-L3
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 09:31:09 +0000
Received: from mail.xenproject.org (unknown [104.130.215.37])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 50e6d84a-0b07-11ec-adbd-12813bfff9fa;
 Wed, 01 Sep 2021 09:30:59 +0000 (UTC)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mLMZy-0006zN-V2; Wed, 01 Sep 2021 09:30:46 +0000
Received: from iwj by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mLMZy-0006TJ-Rw; Wed, 01 Sep 2021 09:30:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50e6d84a-0b07-11ec-adbd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=+rCVNnLKnAbp2uXbXH1bwsbzmEqZNGPyQGmZ8gq+7kE=; b=XVzq51vovCfFiKqUSm6Ks3HKws
	fa7WW4d9ebxm6BNl1z7XocAuZjsDUp3PGUxHPto1K+pgyM7YYj1Q/MivtuVo2QCBsyn41GUVrbVWt
	9krhcgY0GfWpIxXHH9ziFnXtd1cfQL0mmP77a+nu0AzyaXr+SA2LqYsaSjXtmqxgNmtc=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 378 v3 (CVE-2021-28694,CVE-2021-28695,CVE-2021-28696)
 - IOMMU page mapping issues on x86
Message-Id: <E1mLMZy-0006TJ-Rw@xenbits.xenproject.org>
Date: Wed, 01 Sep 2021 09:30:46 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 Xen Security Advisory CVE-2021-28694,CVE-2021-28695,CVE-2021-28696 / XSA-378
                                   version 3

                   IOMMU page mapping issues on x86

UPDATES IN VERSION 3
====================

Warn about dom0=pvh breakage in Resolution section.

ISSUE DESCRIPTION
=================

Both AMD and Intel allow ACPI tables to specify regions of memory
which should be left untranslated, which typically means these
addresses should pass the translation phase unaltered.  While these
are typically device specific ACPI properties, they can also be
specified to apply to a range of devices, or even all devices.

On all systems with such regions Xen failed to prevent guests from
undoing/replacing such mappings (CVE-2021-28694).

On AMD systems, where a discontinuous range is specified by firmware,
the supposedly-excluded middle range will also be identity-mapped
(CVE-2021-28695).

Further, on AMD systems, upon de-assigment of a physical device from a
guest, the identity mappings would be left in place, allowing a guest
continued access to ranges of memory which it shouldn't have access to
anymore (CVE-2021-28696).

IMPACT
======

The precise impact is system specific, but can - on affected systems -
be any or all of privilege escalation, denial of service, or information
leaks.

VULNERABLE SYSTEMS
==================

The vulnerability is only exploitable by guests granted access to
physical devices (ie, via PCI passthrough).

All versions of Xen are affected.

Only x86 systems with IOMMUs and with firmware specifying memory regions
to be identity mapped are affected.  Other x86 systems are not affected.

Whether a particular system whose ACPI tables declare such memory
region(s) is actually affected cannot be known without knowing when
and/or how these regions are used.  For example, if these regions were
used only during system boot, there would not be any vulnerability.
The necessary knowledge can only be obtained from, collectively, the
hardware and firmware manufacturers.

On Arm hardware IOMMU use is not security supported.  Accordingly, we
have not undertaken an analysis of these issues for Arm systems.

MITIGATION
==========

Not permitting untrusted guests access to phsyical devices will avoid
the vulnerability.

Likewise, limiting untrusted guest access to physical devices whose
firmware-provided ACPI tables declare identity mappings, will avoid
the vulnerability.  (Provided that there are no identity mapped
regions which are specified by the ACPI tables to apply globally.)

Note that a system is still vulnerable if a guest was trusted, while
it had such a device assigned, and then has the device removed in
anticipation of the guest becoming untrusted (because of, for example,
the insertion of an untrusted kernel module),

CREDITS
=======

This issue was discovered by Jan Beulich of SUSE.

RESOLUTION
==========

Applying the appropriate set of attached patches resolves this issue.

However, these patches are known to badly break PVH dom0 support.
Work is ongoing to resolve the problem, and will be committed to the
appropriate Xen trees (including backports) when ready.  PVH dom0 has
"Technical Preview" status so we do not intend to distribute the fixes
via an update to this advisory

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa378/xsa378-?.patch           xen-unstable
xsa378/xsa378-4.15-?.patch      Xen 4.15.x
xsa378/xsa378-4.14-?.patch      Xen 4.14.x
xsa378/xsa378-4.13-*.patch      Xen 4.13.x
xsa378/xsa378-4.12-*.patch      Xen 4.12.x
xsa378/xsa378-4.11-*.patch      Xen 4.11.x

$ sha256sum xsa378* xsa378*/*
b3b49681468bd2be4b95fc1d3861493bbea63bac1c115a12ece76d0313b6a81c  xsa378.meta
2560d4418e6b6d022b4e7fe1f84906ca13b3180537e746ee81cd3f97f0e86c35  xsa378/xsa378-1.patch
134b207c38f8d76bbc218220f1e61f82780e8166818ef51217716023cc27ce3c  xsa378/xsa378-2.patch
c166ef60e08a80440f79d92abe8488b123c8017e7633dbddacb95b20899f9b2b  xsa378/xsa378-3.patch
a2677e157724f57f65165f361a309c417d123ff009168f16d3a0e342b7a601eb  xsa378/xsa378-4.11-0a.patch
832f52f420bb47784bdefd75edd1ca521658327aafb94b4f3586da522d0d19c1  xsa378/xsa378-4.11-0b.patch
9eb7c53892f01424eec6b788f879b36dd75a6f89b1c8b9c973237603525d6546  xsa378/xsa378-4.11-0c.patch
48021fb9a2c52d7939fd11bbba80c67a5a5b6cae2039b4b2a2c6203eb5841260  xsa378/xsa378-4.11-1.patch
4f3fd39aa328c1e397aa2a10b92e98cc341908a8834302a4fc0f230876dd3570  xsa378/xsa378-4.11-2.patch
6e2eeecd2ccc4029f80888263f44b80acf078d69aeffda4d6accd8b2d0899182  xsa378/xsa378-4.11-3.patch
194b27510c8765f1cf8abee57624a075ab26f4f889aa8c201e126643f570f55a  xsa378/xsa378-4.11-4.patch
e8f4eab0984a7db2ca0c6c9ac0fb9107e45ddd572a36454db4a1bdb8b9a8b0c6  xsa378/xsa378-4.11-5.patch
943c7556be47b1a0f2273cf5691fe70dc65e6cb50ff0477d1d61eb3f0ba87a97  xsa378/xsa378-4.11-6.patch
2eb572e1a55caa4aca31f74a3844135a204c4e023a6776d7adc4f4043663fb99  xsa378/xsa378-4.11-7.patch
7c24ba33461d3adcb195a6611df3e9c0501e9dcacf9b8811456450b291d23edf  xsa378/xsa378-4.11-8.patch
fe14ea6699df673b787f93ff821838f2280407674df0c8c06d40efe8320e8748  xsa378/xsa378-4.12-0a.patch
1ba10aef9ea99c4455a34e61792ca65ea6b2ece56f05cd0b4adc14600a7ae346  xsa378/xsa378-4.12-0b.patch
2c2dbc0c18b695c1d2c93e4137228f82b6d21d895c3165a9e29ddea3db78e36a  xsa378/xsa378-4.12-0c.patch
48021fb9a2c52d7939fd11bbba80c67a5a5b6cae2039b4b2a2c6203eb5841260  xsa378/xsa378-4.12-1.patch
f0d62506ccdf081d0efbb553e2d33f17a084272d21f43bf95d11fda2dcc8a3fa  xsa378/xsa378-4.12-2.patch
e266512b18fc30e5bd4884cd720aa81644d3ca3323b38fe1f77d06fa98dd515d  xsa378/xsa378-4.12-3.patch
076f9955593a8ebdea5f24ed302b8a1004bbf50da4c8becf1f93764066b8981c  xsa378/xsa378-4.12-4.patch
9639bc35636ffae4e2b6dd026387347960c1fb986cb2924a18314a72e6b6ec0a  xsa378/xsa378-4.12-5.patch
32dd659aa365d9f8197d99b9334c22b33e9ce805e30376457df6e507f92282c8  xsa378/xsa378-4.12-6.patch
7eaae2fa968eb11e27277141456cc1bd657025aee738221c368e153535c7c0f4  xsa378/xsa378-4.12-7.patch
05982f43f35b580ff41f74b1280e469e0dd20176f184ec04a4874303c2aa3ad1  xsa378/xsa378-4.12-8.patch
c6f551ef9903a343b47692b34a63c70165a2dbf74878a6be6511cdffd55a7e8c  xsa378/xsa378-4.13-0a.patch
223cd63f7e1c39d862b8654da698455ded65ecb5abae0f57c330921522b7fdb4  xsa378/xsa378-4.13-0b.patch
5c33aa24f14e779dfe914e809cff11260083169a3adfb07a31ce11243d80b3ef  xsa378/xsa378-4.13-0c.patch
1d55426ff6a41f0ef4cbd2c943edafff394157703bb0b6ae751564abf93b5ee7  xsa378/xsa378-4.13-1.patch
4d5e7d5e65cd28d6bc7d1a9f2ab24f09dfaef295c4199d5f2db00915dcaa174f  xsa378/xsa378-4.13-2.patch
78df0bcb347f8bb45827f74b191aad36b6e907eb38c6d535035f2b2739645551  xsa378/xsa378-4.13-3.patch
237c33e0ae01a23db01721afb8e6a39101bfe081f8b75dbcff6b9fa9c9aaceda  xsa378/xsa378-4.13-4.patch
7d2f3ae3881d28073be54a6dfb35f13004e4efee742952788430201d86307ecf  xsa378/xsa378-4.13-5.patch
2ecd7580394667db0c41e4819025393e59ae24d6c97d54451c8e683585057367  xsa378/xsa378-4.13-6.patch
592a03d00e5d22d7a3c001681968dc469c70b3e57998b95877388e9528904ea2  xsa378/xsa378-4.13-7.patch
54e6a095f706c66dbbd74e39aa1d88031c9b537589e73ceb2925e2f0cc1854f0  xsa378/xsa378-4.13-8.patch
1d55426ff6a41f0ef4cbd2c943edafff394157703bb0b6ae751564abf93b5ee7  xsa378/xsa378-4.14-1.patch
86fbd88eb8a358575e42cf335c444b047ddb4d2f1c6a1bc6f9e57e6ac0041074  xsa378/xsa378-4.14-2.patch
0a23e5f93ff1bb55f003a56e0ef8c531384b164f0e840f5794acdb9ae3e91996  xsa378/xsa378-4.14-3.patch
e2226f7ddae1d24dbee8cf19efa8d67ebf312f3d10641cb9aec21d68a3c8f818  xsa378/xsa378-4.14-4.patch
f2cf8f7e4aa0460e606b5564adde366332ed323c4d5e3f957e64299ef1bc9baf  xsa378/xsa378-4.14-5.patch
7811426975757e3bb8c6ab3161ba2354e1780b55b9e6be7928229d5f23bf79b6  xsa378/xsa378-4.14-6.patch
58147bd6c0ea4e08e84a17afc796be4bbe53e6fbc1d393f9fe3c6191fd33eba5  xsa378/xsa378-4.14-7.patch
682a011d807a7c284faf0ec9d2cf0aabaddbc658979dea2b9ccbc007b660f9c5  xsa378/xsa378-4.14-8.patch
a00ada0cd673f0909cc7b462cb532dbd6fe17601e06bd84272f5ff1857ea4c73  xsa378/xsa378-4.15-1.patch
d74cc325be1e47d61ba3b1400837af35d044bf1d25806aa98926ec262f80bddd  xsa378/xsa378-4.15-2.patch
2ad685a04dbbdd2c81761b58146e70059b8a8a92b0c1176f36933510293ece5b  xsa378/xsa378-4.15-3.patch
18de9facccd70ce49dd839e219fe71667c43110e474e5d7b56a503a5786fc7e0  xsa378/xsa378-4.15-4.patch
296558b27ba82176f6d06b721102c8ba7c7e6e99d29b29392ea82244e88df0b9  xsa378/xsa378-4.15-5.patch
5d7cc84c66daf0aceab9407fa72f2827024c847f4ca10f8d123eee87b7451aba  xsa378/xsa378-4.15-6.patch
fd4aa4447562230a6684d17e6e4c55f1e48df4773247f66ab9d01181003bc9aa  xsa378/xsa378-4.15-7.patch
5305e3bc513bcd3e016ca3bbecc6cae38c8ed2b2eacb13e82f7b1f4401d3b67d  xsa378/xsa378-4.15-8.patch
78390bf59344ea5dcbfa5831db3634b3f1b3aabf029160c72cdbeeec2b46b2a9  xsa378/xsa378-4.patch
76fb4a1f2604b98d3a803744ad212b3984117ec8c8011ad6db3759f9337a9b5a  xsa378/xsa378-5.patch
a627f8c6b7d2ffac0b6de945189f96b718aeab8e0c8bb11476a40585d6411bd7  xsa378/xsa378-6.patch
28814b51fabd3c5cb3ca249ad291781f589cddd55fc8152fdd5668c5fcdc727c  xsa378/xsa378-7.patch
30e31749ade75fd5ab4c41fa27fe2124bdcc602ccc803a85e5eeec1b9c48a9c1  xsa378/xsa378-8.patch
$

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.


(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmEvSCUMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZrUYH/iYMkjMzUpv5ik/quZ+z34uXwq/mD8x8ROZKXDev
tgojXcFo0vNlcr71R0J5KrE0YeX+3MjbjqbF7A30cnw4/bgmvhzcdP/fwOfIKFYE
JqaGoDaxFRpsIMvHFXV6OxtjgBgleUukaikUrcwWUG+L90KMmjqZU4IAlNcBAALC
J6rDB8nHqD1s+ODIPdPE149jIE7LJfVqyuu9h4r/jmLEDRJBS2YEs6zgClqnIApl
3WBpZNhp8Tk3BFtbQcu4Uh9c0itymtp+8VsY0xzF2bWsE4aoe5DKHv0DyTSLe9Md
pMG0egH7ZrI3MV4nzHdbhkA2Hn3X0rsJiI7TXpmmQPmgi4g=
=A/oT
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa378.meta"
Content-Disposition: attachment; filename="xsa378.meta"
Content-Transfer-Encoding: base64

ewogICJYU0EiOiAzNzgsCiAgIlN1cHBvcnRlZFZlcnNpb25zIjogWwogICAg
Im1hc3RlciIsCiAgICAiNC4xNSIsCiAgICAiNC4xNCIsCiAgICAiNC4xMyIs
CiAgICAiNC4xMiIsCiAgICAiNC4xMSIKICBdLAogICJUcmVlcyI6IFsKICAg
ICJ4ZW4iCiAgXSwKICAiUmVjaXBlcyI6IHsKICAgICI0LjExIjogewogICAg
ICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVuIjogewogICAgICAgICAgIlN0
YWJsZVJlZiI6ICJlZjMyYzdhZmEyNzMxYjc1ODIyNmQ2ZTEwYTFlNDg5YjFh
MTVmYzQxIiwKICAgICAgICAgICJQcmVyZXFzIjogW10sCiAgICAgICAgICAi
UGF0Y2hlcyI6IFsKICAgICAgICAgICAgInhzYTM3OC94c2EzNzgtNC4xMS0q
LnBhdGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwK
ICAgICI0LjEyIjogewogICAgICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVu
IjogewogICAgICAgICAgIlN0YWJsZVJlZiI6ICJlYTIwZWVlOTdlOWUwODYx
MTI3YTgwNzBjYzdiOWFlMzU1N2IwOWZiIiwKICAgICAgICAgICJQcmVyZXFz
IjogWwogICAgICAgICAgICAzODMKICAgICAgICAgIF0sCiAgICAgICAgICAi
UGF0Y2hlcyI6IFsKICAgICAgICAgICAgInhzYTM3OC94c2EzNzgtNC4xMi0q
LnBhdGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwK
ICAgICI0LjEzIjogewogICAgICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVu
IjogewogICAgICAgICAgIlN0YWJsZVJlZiI6ICIzMmQ1ODA5MDJiOTU5MDAw
ZDc5ZDUxZGZmMDNhMzU2MDY1M2M0ZmNiIiwKICAgICAgICAgICJQcmVyZXFz
IjogWwogICAgICAgICAgICAzODMKICAgICAgICAgIF0sCiAgICAgICAgICAi
UGF0Y2hlcyI6IFsKICAgICAgICAgICAgInhzYTM3OC94c2EzNzgtNC4xMy0q
LnBhdGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwK
ICAgICI0LjE0IjogewogICAgICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVu
IjogewogICAgICAgICAgIlN0YWJsZVJlZiI6ICI0OTI5OWM0ODEzYjc4NDdk
MjlkZjA3YmY3OTBmNTQ4OTA2MGYyYTljIiwKICAgICAgICAgICJQcmVyZXFz
IjogWwogICAgICAgICAgICAzODMKICAgICAgICAgIF0sCiAgICAgICAgICAi
UGF0Y2hlcyI6IFsKICAgICAgICAgICAgInhzYTM3OC94c2EzNzgtNC4xNC0q
LnBhdGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwK
ICAgICI0LjE1IjogewogICAgICAiUmVjaXBlcyI6IHsKICAgICAgICAieGVu
IjogewogICAgICAgICAgIlN0YWJsZVJlZiI6ICJkYmE3NzQ4OTZmN2RkNzQ3
NzNjMTRkNTM3NjQzYjdkNzQ3N2ZlZmNkIiwKICAgICAgICAgICJQcmVyZXFz
IjogWwogICAgICAgICAgICAzODMKICAgICAgICAgIF0sCiAgICAgICAgICAi
UGF0Y2hlcyI6IFsKICAgICAgICAgICAgInhzYTM3OC94c2EzNzgtNC4xNS0q
LnBhdGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfSwK
ICAgICJtYXN0ZXIiOiB7CiAgICAgICJSZWNpcGVzIjogewogICAgICAgICJ4
ZW4iOiB7CiAgICAgICAgICAiU3RhYmxlUmVmIjogIjI1ZGE5NDU1ZjFiYjhh
NmQzMzAzOTU3NWE3YjI4YmRmYzRlM2ZjZmUiLAogICAgICAgICAgIlByZXJl
cXMiOiBbCiAgICAgICAgICAgIDM4MwogICAgICAgICAgXSwKICAgICAgICAg
ICJQYXRjaGVzIjogWwogICAgICAgICAgICAieHNhMzc4L3hzYTM3OC0/LnBh
dGNoIgogICAgICAgICAgXQogICAgICAgIH0KICAgICAgfQogICAgfQogIH0K
fQ==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+Ci0tLQp2ODogQWRkIGNvbW1lbnRzIHRvIHBsYWluIHRydWUvZmFs
c2UgZnVuY3Rpb24gYXJndW1lbnRzLgp2NDogRG9uJ3QgYWNjZXB0IHJlcXVl
c3RzIG5vdCBhbGxvd2luZyBmb3IgYm90aCByZWFkaW5nIGFuZCB3cml0aW5n
LgogICAgRG9uJ3QgYWNjZXB0IG1pc21hdGNoZXMgaW4gdGhlIGV4Y2x1c2lv
bl9hbGxvd19hbGwgc2V0dGluZy4gQ292ZXIKICAgIGZvciB0aGUgZHJvcHBp
bmcgb2YgImNvcnJlY3QgbGltaXQgY2FsY3VsYXRpb24gaW4gSVZNRCBwYXJz
aW5nIi4KdjI6IEFjY2VwdCBvdmVybGFwcGluZyByZWdpb25zIGFzIHdlbGwu
Ci0tLQpUQkQ6IFNob3VsZCB3ZSBhc3N1bWUgcmFuZG9tIG9yZGVyIG9mIGVu
dHJpZXMgcmlnaHQgYXdheT8KVEJEOiBBbmRyZXcgc3VnZ2VzdHMgdGhlcmUg
YXJlIGluZGljYXRpb25zIG9mIG9ubHkgb25lIGdsb2JhbCBleGNsdXNpb24K
ICAgICByYW5nZSBiZWluZyBhbGxvd2VkLCB3aGljaCB3b3VsZCB0aGVuIChw
cmVzdW1hYmx5KSBiZSB0aWVkIHRvCiAgICAgQUNQSV9JVk1EX0VYQ0xVU0lP
Tl9SQU5HRSAoYXMgcGVyIGFib3ZlKS4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bh
c3N0aHJvdWdoL2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVuL2RyaXZlcnMv
cGFzc3Rocm91Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtMTE2LDEyICsxMTYs
MjEgQEAgc3RhdGljIHN0cnVjdCBhbWRfaW9tbXUgKiBfX2luaXQgZmluZF9p
bwogICAgIHJldHVybiBOVUxMOwogfQogCi1zdGF0aWMgdm9pZCBfX2luaXQg
cmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoCi0gICAgc3RydWN0IGFt
ZF9pb21tdSAqaW9tbXUsIHVpbnQ2NF90IGJhc2UsIHVpbnQ2NF90IGxpbWl0
KQorc3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25f
cmFuZ2UoCisgICAgc3RydWN0IGFtZF9pb21tdSAqaW9tbXUsIHVpbnQ2NF90
IGJhc2UsIHVpbnQ2NF90IGxpbWl0LAorICAgIGJvb2wgYWxsLCBib29sIGl3
LCBib29sIGlyKQogeworICAgIGlmICggIWlyIHx8ICFpdyApCisgICAgICAg
IHJldHVybiAtRVBFUk07CisKICAgICAvKiBuZWVkIHRvIGV4dGVuZCBleGNs
dXNpb24gcmFuZ2U/ICovCiAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9uX2Vu
YWJsZSApCiAgICAgeworICAgICAgICBpZiAoIGlvbW11LT5leGNsdXNpb25f
bGltaXQgKyBQQUdFX1NJWkUgPCBiYXNlIHx8CisgICAgICAgICAgICAgbGlt
aXQgKyBQQUdFX1NJWkUgPCBpb21tdS0+ZXhjbHVzaW9uX2Jhc2UgfHwKKyAg
ICAgICAgICAgICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCAhPSBhbGwg
KQorICAgICAgICAgICAgcmV0dXJuIC1FQlVTWTsKKwogICAgICAgICBpZiAo
IGlvbW11LT5leGNsdXNpb25fYmFzZSA8IGJhc2UgKQogICAgICAgICAgICAg
YmFzZSA9IGlvbW11LT5leGNsdXNpb25fYmFzZTsKICAgICAgICAgaWYgKCBp
b21tdS0+ZXhjbHVzaW9uX2xpbWl0ID4gbGltaXQgKQpAQCAtMTI5LDE2ICsx
MzgsMTEgQEAgc3RhdGljIHZvaWQgX19pbml0IHJlc2VydmVfaW9tbXVfZXhj
bHVzaQogICAgIH0KIAogICAgIGlvbW11LT5leGNsdXNpb25fZW5hYmxlID0g
SU9NTVVfQ09OVFJPTF9FTkFCTEVEOworICAgIGlvbW11LT5leGNsdXNpb25f
YWxsb3dfYWxsID0gYWxsOwogICAgIGlvbW11LT5leGNsdXNpb25fYmFzZSA9
IGJhc2U7CiAgICAgaW9tbXUtPmV4Y2x1c2lvbl9saW1pdCA9IGxpbWl0Owot
fQogCi1zdGF0aWMgdm9pZCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2VfYWxsKAotICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LAot
ICAgIHVuc2lnbmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCkK
LXsKLSAgICByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwg
YmFzZSwgbGltaXQpOwotICAgIGlvbW11LT5leGNsdXNpb25fYWxsb3dfYWxs
ID0gSU9NTVVfQ09OVFJPTF9FTkFCTEVEOworICAgIHJldHVybiAwOwogfQog
CiBzdGF0aWMgdm9pZCBfX2luaXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2Rl
dmljZSgKQEAgLTE3Niw2ICsxODAsNyBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgdW5zaWduZWQgbG9uZyByYW5n
ZV90b3AsIGlvbW11X3RvcCwgbGVuZ3RoOwogICAgIHN0cnVjdCBhbWRfaW9t
bXUgKmlvbW11OwogICAgIHVuc2lnbmVkIGludCBiZGY7CisgICAgaW50IHJj
ID0gMDsKIAogICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJhbmdlIGlu
c2lkZSBvZiBJT01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICovCiAgICAg
Lyogbm90ZTogJ2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBw
YWdlLWFsaWduZWQgKi8KQEAgLTE5NywxMCArMjAyLDE1IEBAIHN0YXRpYyBp
bnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICBpZiAoIGxp
bWl0ID49IGlvbW11X3RvcCApCiAgICAgewogICAgICAgICBmb3JfZWFjaF9h
bWRfaW9tbXUoIGlvbW11ICkKLSAgICAgICAgICAgIHJlc2VydmVfaW9tbXVf
ZXhjbHVzaW9uX3JhbmdlX2FsbChpb21tdSwgYmFzZSwgbGltaXQpOworICAg
ICAgICB7CisgICAgICAgICAgICByYyA9IHJlc2VydmVfaW9tbXVfZXhjbHVz
aW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCwKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwg
Ki8sIGl3LCBpcik7CisgICAgICAgICAgICBpZiAoIHJjICkKKyAgICAgICAg
ICAgICAgICBicmVhazsKKyAgICAgICAgfQogICAgIH0KIAotICAgIHJldHVy
biAwOworICAgIHJldHVybiByYzsKIH0KIAogc3RhdGljIGludCBfX2luaXQg
cmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoCkBAIC0yMTEs
NiArMjIxLDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVz
aW9uX3JhbgogICAgIHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90
b3AsIGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAg
ICB1MTYgcmVxOworICAgIGludCByYyA9IDA7CiAKICAgICBpb21tdSA9IGZp
bmRfaW9tbXVfZm9yX2RldmljZShzZWcsIGJkZik7CiAgICAgaWYgKCAhaW9t
bXUgKQpAQCAtMjQwLDEyICsyNTEsMTMgQEAgc3RhdGljIGludCBfX2luaXQg
cmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgIC8qIHJlZ2lzdGVyIElPTU1V
IGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyBmb3IgZGV2aWNlICovCiAgICAg
aWYgKCBsaW1pdCA+PSBpb21tdV90b3AgICkKICAgICB7Ci0gICAgICAgIHJl
c2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1p
dCk7CisgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFu
Z2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIGZhbHNlIC8qIGFsbCAqLywgaXcsIGly
KTsKICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmR0ZV9hbGxvd19leGNs
dXNpb24gPSB0cnVlOwogICAgICAgICBpdnJzX21hcHBpbmdzW3JlcV0uZHRl
X2FsbG93X2V4Y2x1c2lvbiA9IHRydWU7CiAgICAgfQogCi0gICAgcmV0dXJu
IDA7CisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCkBA
IC0yNTUsNiArMjY3LDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJf
ZXhjbHVzaW9uX3JhbgogICAgIHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBp
b21tdV90b3AsIGxlbmd0aDsKICAgICB1bnNpZ25lZCBpbnQgYmRmOwogICAg
IHUxNiByZXE7CisgICAgaW50IHJjID0gMDsKIAogICAgIC8qIGlzIHBhcnQg
b2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJT01NVSB2aXJ0dWFsIGFk
ZHJlc3Mgc3BhY2U/ICovCiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJhbWV0
ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KQEAgLTI4NSw4
ICsyOTgsMTAgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVz
aW9uX3JhbgogCiAgICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJh
bmdlIHNldHRpbmdzICovCiAgICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3Ag
KQotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9hbGwo
aW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAgICByZXR1cm4gMDsKKyAgICAgICAg
cmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFz
ZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3LCBpcik7CisKKyAgICByZXR1
cm4gcmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfZGV2
aWNlX3NlbGVjdCgK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KLS0tClRCRDogT3ZlcmxhcHBpbmcgcmFuZ2VzIGNv
dWxkIGJlIGRlYWx0IHdpdGggKHdoZW4gdGhleSBzcGVjaWZ5IHRoZSBzYW1l
CiAgICAgcGVybWlzc2lvbnMpIGJ5IHNwbGl0dGluZyB0aGUgcmFuZ2VzIHRv
IG5vbi1vdmVybGFwcGluZyBwaWVjZXMuIFNlZQogICAgICJJT01NVTogZ2Vu
ZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVkIFJNUlIgcmVnaW9u
cyIgYW5kIGl0cwogICAgIHVzZXIgIkFNRC9JT01NVTogcmUtYXJyYW5nZS9j
b21wbGV0ZSByZS1hc3NpZ25tZW50IGhhbmRsaW5nIiBmb3IKICAgICB3aHkg
b3ZlcmxhcHMgY2FuJ3QgYmUgYWNjZXB0ZWQgd2l0aCB0aGUgY2hvc2VuIHRy
YWNraW5nIG1vZGVsLiBCdXQKICAgICB0aGUgYmFzZSBhc3N1bXB0aW9uIGlz
IGFueXdheSBmb3IgZmlybXdhcmUgdG8gbm90IHN1cmZhY2UKICAgICBvdmVy
bGFwcGluZyByYW5nZXMgaW4gdGhlIGZpcnN0IHBsYWNlLgotLS0Kdjg6IEZp
eCBidWlsZCAoZmFpbHVyZSByZXN1bHRlZCBmcm9tIGEgcGllY2Ugb2YgY29k
ZSBsaXZpbmcgaGVyZSB3aGljaAogICAgb25seSBhIGxhdGVyIHBhdGNoIHNo
b3VsZCBhZGQpLiBSZS1iYXNlIG92ZXIgYWRqdXN0bWVudHMgdG8gZWFybGll
cgogICAgcGF0Y2guCnY1OiBEb24ndCBhbGxvdyBvdmVybGFwcyAoZXhjZXB0
IGV4YWN0IG1hdGNoZXMpIGFuZCBkb24ndCBtZXJnZS4KICAgIFJlLWJhc2Uu
CnY0OiBBZGp1c3QgdGl0bGUuIFJlLWJhc2Ugb3ZlciBjaGFuZ2VzIGVhcmxp
ZXIgaW4gdGhlIHNlcmllcy4KdjI6IFJlLWJhc2Ugb3ZlciBYU0EtMzAyLgoK
LS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11LmgKKysr
IGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11LmgKQEAgLTEw
NywyMCArMTA3LDI0IEBAIHN0cnVjdCBhbWRfaW9tbXUgewogICAgIHN0cnVj
dCBsaXN0X2hlYWQgYXRzX2RldmljZXM7CiB9OwogCitzdHJ1Y3QgaXZyc191
bml0eV9tYXAgeworICAgIGJvb2wgcmVhZDoxOworICAgIGJvb2wgd3JpdGU6
MTsKKyAgICBwYWRkcl90IGFkZHI7CisgICAgdW5zaWduZWQgbG9uZyBsZW5n
dGg7CisgICAgc3RydWN0IGl2cnNfdW5pdHlfbWFwICpuZXh0OworfTsKKwog
c3RydWN0IGl2cnNfbWFwcGluZ3MgewogICAgIHVpbnQxNl90IGR0ZV9yZXF1
ZXN0b3JfaWQ7CiAgICAgYm9vbCB2YWxpZDoxOwogICAgIGJvb2wgZHRlX2Fs
bG93X2V4Y2x1c2lvbjoxOwotICAgIGJvb2wgdW5pdHlfbWFwX2VuYWJsZTox
OwotICAgIGJvb2wgd3JpdGVfcGVybWlzc2lvbjoxOwotICAgIGJvb2wgcmVh
ZF9wZXJtaXNzaW9uOjE7CiAKICAgICAvKiBpdmhkIGRldmljZSBkYXRhIHNl
dHRpbmdzICovCiAgICAgdWludDhfdCBkZXZpY2VfZmxhZ3M7CiAKLSAgICB1
bnNpZ25lZCBsb25nIGFkZHJfcmFuZ2Vfc3RhcnQ7Ci0gICAgdW5zaWduZWQg
bG9uZyBhZGRyX3JhbmdlX2xlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lvbW11
ICppb21tdTsKKyAgICBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKnVuaXR5X21h
cDsKIAogICAgIC8qIHBlciBkZXZpY2UgaW50ZXJydXB0IHJlbWFwcGluZyB0
YWJsZSAqLwogICAgIHZvaWQgKmludHJlbWFwX3RhYmxlOwotLS0gYS94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXVfYWNwaS5jCisrKyBiL3hl
bi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9hY3BpLmMKQEAgLTE0
NSwzMiArMTQ1LDQ4IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlc2VydmVfaW9t
bXVfZXhjbHVzaW8KICAgICByZXR1cm4gMDsKIH0KIAotc3RhdGljIHZvaWQg
X19pbml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoCi0gICAgdTE2
IHNlZywgdTE2IGJkZiwgdW5zaWduZWQgbG9uZyBiYXNlLAotICAgIHVuc2ln
bmVkIGxvbmcgbGVuZ3RoLCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9f
aW5pdCByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKAorICAgIHVpbnQx
Nl90IHNlZywgdWludDE2X3QgYmRmLCB1bnNpZ25lZCBsb25nIGJhc2UsCisg
ICAgdW5zaWduZWQgbG9uZyBsZW5ndGgsIGJvb2wgaXcsIGJvb2wgaXIpCiB7
CiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhzZWcpOwotICAgIHVuc2lnbmVkIGxvbmcgb2xk
X3RvcCwgbmV3X3RvcDsKKyAgICBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKnVu
aXR5X21hcCA9IGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXA7CiAKLSAg
ICAvKiBuZWVkIHRvIGV4dGVuZCB1bml0eS1tYXBwZWQgcmFuZ2U/ICovCi0g
ICAgaWYgKCBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwX2VuYWJsZSAp
CisgICAgLyogQ2hlY2sgZm9yIG92ZXJsYXBzLiAqLworICAgIGZvciAoIDsg
dW5pdHlfbWFwOyB1bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQogICAg
IHsKLSAgICAgICAgb2xkX3RvcCA9IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRy
X3JhbmdlX3N0YXJ0ICsKLSAgICAgICAgICAgIGl2cnNfbWFwcGluZ3NbYmRm
XS5hZGRyX3JhbmdlX2xlbmd0aDsKLSAgICAgICAgbmV3X3RvcCA9IGJhc2Ug
KyBsZW5ndGg7Ci0gICAgICAgIGlmICggb2xkX3RvcCA+IG5ld190b3AgKQot
ICAgICAgICAgICAgbmV3X3RvcCA9IG9sZF90b3A7Ci0gICAgICAgIGlmICgg
aXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQgPCBiYXNlICkK
LSAgICAgICAgICAgIGJhc2UgPSBpdnJzX21hcHBpbmdzW2JkZl0uYWRkcl9y
YW5nZV9zdGFydDsKLSAgICAgICAgbGVuZ3RoID0gbmV3X3RvcCAtIGJhc2U7
Ci0gICAgfQotCi0gICAgLyogZXh0ZW5kIHIvdyBwZXJtaXNzaW9tcyBhbmQg
a2VlcCBhZ2dyZWdhdGUgKi8KLSAgICBpdnJzX21hcHBpbmdzW2JkZl0ud3Jp
dGVfcGVybWlzc2lvbiA9IGl3OwotICAgIGl2cnNfbWFwcGluZ3NbYmRmXS5y
ZWFkX3Blcm1pc3Npb24gPSBpcjsKLSAgICBpdnJzX21hcHBpbmdzW2JkZl0u
dW5pdHlfbWFwX2VuYWJsZSA9IHRydWU7Ci0gICAgaXZyc19tYXBwaW5nc1ti
ZGZdLmFkZHJfcmFuZ2Vfc3RhcnQgPSBiYXNlOwotICAgIGl2cnNfbWFwcGlu
Z3NbYmRmXS5hZGRyX3JhbmdlX2xlbmd0aCA9IGxlbmd0aDsKKyAgICAgICAg
LyoKKyAgICAgICAgICogRXhhY3QgbWF0Y2hlcyBhcmUgb2theS4gVGhpcyBj
YW4gaW4gcGFydGljdWxhciBoYXBwZW4gd2hlbgorICAgICAgICAgKiByZWdp
c3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgpIGNhbGxzIGhlcmUg
dHdpY2UgZm9yIHRoZQorICAgICAgICAgKiBzYW1lIChzLGIsZCxmKS4KKyAg
ICAgICAgICovCisgICAgICAgIGlmICggYmFzZSA9PSB1bml0eV9tYXAtPmFk
ZHIgJiYgbGVuZ3RoID09IHVuaXR5X21hcC0+bGVuZ3RoICYmCisgICAgICAg
ICAgICAgaXIgPT0gdW5pdHlfbWFwLT5yZWFkICYmIGl3ID09IHVuaXR5X21h
cC0+d3JpdGUgKQorICAgICAgICAgICAgcmV0dXJuIDA7CisKKyAgICAgICAg
aWYgKCB1bml0eV9tYXAtPmFkZHIgKyB1bml0eV9tYXAtPmxlbmd0aCA+IGJh
c2UgJiYKKyAgICAgICAgICAgICBiYXNlICsgbGVuZ3RoID4gdW5pdHlfbWFw
LT5hZGRyICkKKyAgICAgICAgeworICAgICAgICAgICAgQU1EX0lPTU1VX0RF
QlVHKCJJVk1EIEVycm9yOiBvdmVybGFwIFslbHgsJWx4KSB2cyBbJWx4LCVs
eClcbiIsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmFzZSwgYmFz
ZSArIGxlbmd0aCwgdW5pdHlfbWFwLT5hZGRyLAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuaXR5X21hcC0+YWRkciArIHVuaXR5X21hcC0+bGVu
Z3RoKTsKKyAgICAgICAgICAgIHJldHVybiAtRVBFUk07CisgICAgICAgIH0K
KyAgICB9CisKKyAgICAvKiBQb3B1bGF0ZSBhbmQgaW5zZXJ0IGEgbmV3IHVu
aXR5IG1hcC4gKi8KKyAgICB1bml0eV9tYXAgPSB4bWFsbG9jKHN0cnVjdCBp
dnJzX3VuaXR5X21hcCk7CisgICAgaWYgKCAhdW5pdHlfbWFwICkKKyAgICAg
ICAgcmV0dXJuIC1FTk9NRU07CisKKyAgICB1bml0eV9tYXAtPnJlYWQgPSBp
cjsKKyAgICB1bml0eV9tYXAtPndyaXRlID0gaXc7CisgICAgdW5pdHlfbWFw
LT5hZGRyID0gYmFzZTsKKyAgICB1bml0eV9tYXAtPmxlbmd0aCA9IGxlbmd0
aDsKKyAgICB1bml0eV9tYXAtPm5leHQgPSBpdnJzX21hcHBpbmdzW2JkZl0u
dW5pdHlfbWFwOworICAgIGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXAg
PSB1bml0eV9tYXA7CisKKyAgICByZXR1cm4gMDsKIH0KIAogc3RhdGljIGlu
dCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9hbGxfZGV2
aWNlcygKQEAgLTE5MywxMyArMjA5LDEzIEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICAgICAgbGVuZ3RoID0gcmFu
Z2VfdG9wIC0gYmFzZTsKICAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHkt
bWFwcGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAqLwogICAgICAgICAv
KiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNp
b24gcmFuZ2UgKi8KLSAgICAgICAgZm9yICggYmRmID0gMDsgYmRmIDwgaXZy
c19iZGZfZW50cmllczsgYmRmKysgKQotICAgICAgICAgICAgcmVzZXJ2ZV91
bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBp
dywgaXIpOworICAgICAgICBmb3IgKCBiZGYgPSAwOyAhcmMgJiYgYmRmIDwg
aXZyc19iZGZfZW50cmllczsgYmRmKysgKQorICAgICAgICAgICAgcmMgPSBy
ZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywgYmRmLCBiYXNlLCBs
ZW5ndGgsIGl3LCBpcik7CiAgICAgICAgIC8qIHB1c2ggJ2Jhc2UnIGp1c3Qg
b3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgKi8KICAgICAgICAg
YmFzZSA9IGlvbW11X3RvcDsKICAgICB9CiAgICAgLyogcmVnaXN0ZXIgSU9N
TVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzICovCi0gICAgaWYgKCBsaW1p
dCA+PSBpb21tdV90b3AgKQorICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlv
bW11X3RvcCApCiAgICAgewogICAgICAgICBmb3JfZWFjaF9hbWRfaW9tbXUo
IGlvbW11ICkKICAgICAgICAgewpAQCAtMjQxLDE1ICsyNTcsMTUgQEAgc3Rh
dGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgICAg
ICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwogICAgICAgICAvKiByZXNl
cnZlIHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZSAqLwog
ICAgICAgICAvKiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRo
ZSBleGNsdXNpb24gcmFuZ2UgKi8KLSAgICAgICAgcmVzZXJ2ZV91bml0eV9t
YXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIp
OwotICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywg
cmVxLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7CisgICAgICAgIHJjID0gcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVu
Z3RoLCBpdywgaXIpID86CisgICAgICAgICAgICAgcmVzZXJ2ZV91bml0eV9t
YXBfZm9yX2RldmljZShzZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLCBpdywgaXIp
OwogCiAgICAgICAgIC8qIHB1c2ggJ2Jhc2UnIGp1c3Qgb3V0c2lkZSBvZiB2
aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgKi8KICAgICAgICAgYmFzZSA9IGlvbW11
X3RvcDsKICAgICB9CiAKICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNp
b24gcmFuZ2Ugc2V0dGluZ3MgZm9yIGRldmljZSAqLwotICAgIGlmICggbGlt
aXQgPj0gaW9tbXVfdG9wICApCisgICAgaWYgKCAhcmMgJiYgbGltaXQgPj0g
aW9tbXVfdG9wICApCiAgICAgewogICAgICAgICByYyA9IHJlc2VydmVfaW9t
bXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCwKICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWxzZSAv
KiBhbGwgKi8sIGl3LCBpcik7CkBAIC0yODAsMTUgKzI5NiwxNSBAQCBzdGF0
aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgICAg
IGxlbmd0aCA9IHJhbmdlX3RvcCAtIGJhc2U7CiAgICAgICAgIC8qIHJlc2Vy
dmUgci93IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMg
Ki8KICAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBv
ZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9
IDA7IGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKKyAgICAgICAg
Zm9yICggYmRmID0gMDsgIXJjICYmIGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7
IGJkZisrICkKICAgICAgICAgewogICAgICAgICAgICAgaWYgKCBpb21tdSA9
PSBmaW5kX2lvbW11X2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmKSApCiAg
ICAgICAgICAgICB7Ci0gICAgICAgICAgICAgICAgcmVzZXJ2ZV91bml0eV9t
YXBfZm9yX2RldmljZShpb21tdS0+c2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwK
LSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGl3LCBpcik7CiAgICAgICAgICAgICAgICAgcmVxID0gZ2V0X2l2cnNfbWFw
cGluZ3MoaW9tbXUtPnNlZylbYmRmXS5kdGVfcmVxdWVzdG9yX2lkOwotICAg
ICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIpOworICAgICAg
ICAgICAgICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShp
b21tdS0+c2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyKSA/
OgorICAgICAgICAgICAgICAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBfZm9y
X2RldmljZShpb21tdS0+c2VnLCByZXEsIGJhc2UsIGxlbmd0aCwKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
aXcsIGlyKTsKICAgICAgICAgICAgIH0KICAgICAgICAgfQogCkBAIC0yOTcs
NyArMzEzLDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVz
aW9uX3JhbgogICAgIH0KIAogICAgIC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1
c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICggbGltaXQgPj0gaW9t
bXVfdG9wICkKKyAgICBpZiAoICFyYyAmJiBsaW1pdCA+PSBpb21tdV90b3Ag
KQogICAgICAgICByYyA9IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3Jhbmdl
KGlvbW11LCBiYXNlLCBsaW1pdCwKICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB0cnVlIC8qIGFsbCAqLywgaXcsIGlyKTsK
IAotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9p
b21tdS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9wY2lf
YW1kX2lvbW11LmMKQEAgLTM4NCwxNSArMzg0LDE3IEBAIHN0YXRpYyBpbnQg
YW1kX2lvbW11X2Fzc2lnbl9kZXZpY2Uoc3RydWMKICAgICBzdHJ1Y3QgaXZy
c19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdz
KHBkZXYtPnNlZyk7CiAgICAgaW50IGJkZiA9IFBDSV9CREYyKHBkZXYtPmJ1
cywgZGV2Zm4pOwogICAgIGludCByZXFfaWQgPSBnZXRfZG1hX3JlcXVlc3Rv
cl9pZChwZGV2LT5zZWcsIGJkZik7CisgICAgY29uc3Qgc3RydWN0IGl2cnNf
dW5pdHlfbWFwICp1bml0eV9tYXA7CiAKLSAgICBpZiAoIGl2cnNfbWFwcGlu
Z3NbcmVxX2lkXS51bml0eV9tYXBfZW5hYmxlICkKKyAgICBmb3IgKCB1bml0
eV9tYXAgPSBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwOyB1bml0
eV9tYXA7CisgICAgICAgICAgdW5pdHlfbWFwID0gdW5pdHlfbWFwLT5uZXh0
ICkKICAgICB7Ci0gICAgICAgIGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91
bml0eV9tYXAoCi0gICAgICAgICAgICBkLAotICAgICAgICAgICAgaXZyc19t
YXBwaW5nc1tyZXFfaWRdLmFkZHJfcmFuZ2Vfc3RhcnQsCi0gICAgICAgICAg
ICBpdnJzX21hcHBpbmdzW3JlcV9pZF0uYWRkcl9yYW5nZV9sZW5ndGgsCi0g
ICAgICAgICAgICBpdnJzX21hcHBpbmdzW3JlcV9pZF0ud3JpdGVfcGVybWlz
c2lvbiwKLSAgICAgICAgICAgIGl2cnNfbWFwcGluZ3NbcmVxX2lkXS5yZWFk
X3Blcm1pc3Npb24pOworICAgICAgICBpbnQgcmMgPSBhbWRfaW9tbXVfcmVz
ZXJ2ZV9kb21haW5fdW5pdHlfbWFwKAorICAgICAgICAgICAgICAgICAgICAg
ZCwgdW5pdHlfbWFwLT5hZGRyLCB1bml0eV9tYXAtPmxlbmd0aCwKKyAgICAg
ICAgICAgICAgICAgICAgIHVuaXR5X21hcC0+d3JpdGUsIHVuaXR5X21hcC0+
cmVhZCk7CisKKyAgICAgICAgaWYgKCByYyApCisgICAgICAgICAgICByZXR1
cm4gcmM7CiAgICAgfQogCiAgICAgcmV0dXJuIHJlYXNzaWduX2RldmljZShw
ZGV2LT5kb21haW4sIGQsIGRldmZuLCBwZGV2KTsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+Ci0tLQp2NTogTmV3LgoKLS0t
IGEveGVuL2FyY2gveDg2L21tL3AybS1lcHQuYworKysgYi94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCkBAIC04MDgsNyArODA4LDcgQEAgZXB0X3NldF9l
bnRyeShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtLCBnZgogICAgIGJvb2xfdCBl
bnRyeV93cml0dGVuID0gMDsKICAgICBib29sX3QgbmVlZF9tb2RpZnlfdnRk
X3RhYmxlID0gMTsKICAgICBib29sX3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsK
LSAgICB1bnNpZ25lZCBpbnQgaW9tbXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11
X2ZsYWdzKHAybXQsIG1mbik7CisgICAgdW5zaWduZWQgaW50IGlvbW11X2Zs
YWdzID0gcDJtX2dldF9pb21tdV9mbGFncyhwMm10LCBwMm1hLCBtZm4pOwog
ICAgIGJvb2xfdCBuZWVkc19zeW5jID0gMTsKICAgICBlcHRfZW50cnlfdCBv
bGRfZW50cnkgPSB7IC5lcHRlID0gMCB9OwogICAgIGVwdF9lbnRyeV90IG5l
d19lbnRyeSA9IHsgLmVwdGUgPSAwIH07CkBAIC05MzgsOCArOTM4LDggQEAg
ZXB0X3NldF9lbnRyeShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtLCBnZgogCiAg
ICAgICAgIC8qIFNhZmUgdG8gcmVhZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2Ug
aG9sZCB0aGUgcDJtIGxvY2sgKi8KICAgICAgICAgaWYgKCBlcHRfZW50cnkt
Pm1mbiA9PSBuZXdfZW50cnkubWZuICYmCi0gICAgICAgICAgICAgcDJtX2dl
dF9pb21tdV9mbGFncyhlcHRfZW50cnktPnNhX3AybXQsIF9tZm4oZXB0X2Vu
dHJ5LT5tZm4pKSA9PQotICAgICAgICAgICAgIGlvbW11X2ZsYWdzICkKKyAg
ICAgICAgICAgICBwMm1fZ2V0X2lvbW11X2ZsYWdzKGVwdF9lbnRyeS0+c2Ff
cDJtdCwgZXB0X2VudHJ5LT5hY2Nlc3MsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBfbWZuKGVwdF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVf
ZmxhZ3MgKQogICAgICAgICAgICAgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0g
MDsKIAogICAgICAgICBlcHRfcDJtX3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3
X2VudHJ5KTsKLS0tIGEveGVuL2FyY2gveDg2L21tL3AybS1wdC5jCisrKyBi
L3hlbi9hcmNoL3g4Ni9tbS9wMm0tcHQuYwpAQCAtNTQ1LDYgKzU0NSwxNiBA
QCBpbnQgcDJtX3B0X2hhbmRsZV9kZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAg
ICAgcmV0dXJuIHJjOwogfQogCisvKiBSZWNvbnN0cnVjdCBhIGZha2UgcDJt
X2FjY2Vzc190IGZyb20gc3RvcmVkIFBURSBmbGFncy4gKi8KK3N0YXRpYyBw
Mm1fYWNjZXNzX3QgcDJtX2ZsYWdzX3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQg
ZmxhZ3MpCit7CisgICAgaWYgKCBmbGFncyAmIF9QQUdFX1BSRVNFTlQgKQor
ICAgICAgICByZXR1cm4gcDJtX2FjY2Vzc19uOworCisgICAgLyogTm8gbmVl
ZCB0byBsb29rIGF0IF9QQUdFX05YIGZvciBub3cuICovCisgICAgcmV0dXJu
IGZsYWdzICYgX1BBR0VfUlcgPyBwMm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vz
c19yOworfQorCiAvKiBDaGVja3Mgb25seSBhcHBsaWNhYmxlIHRvIGVudHJp
ZXMgd2l0aCBvcmRlciA+IFBBR0VfT1JERVJfNEsgKi8KIHN0YXRpYyB2b2lk
IGNoZWNrX2VudHJ5KG1mbl90IG1mbiwgcDJtX3R5cGVfdCBuZXcsIHAybV90
eXBlX3Qgb2xkLAogICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IG9yZGVyKQpAQCAtNTc5LDcgKzU4OSw3IEBAIHAybV9wdF9zZXRfZW50
cnkoc3RydWN0IHAybV9kb21haW4gKnAybSwKICAgICBsMl9wZ2VudHJ5X3Qg
bDJlX2NvbnRlbnQ7CiAgICAgbDNfcGdlbnRyeV90IGwzZV9jb250ZW50Owog
ICAgIGludCByYzsKLSAgICB1bnNpZ25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdz
ID0gcDJtX2dldF9pb21tdV9mbGFncyhwMm10LCBtZm4pOworICAgIHVuc2ln
bmVkIGludCBpb21tdV9wdGVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdz
KHAybXQsIHAybWEsIG1mbik7CiAgICAgLyoKICAgICAgKiBvbGRfbWZuIGFu
ZCBpb21tdV9vbGRfZmxhZ3MgY29udHJvbCBwb3NzaWJsZSBmbHVzaC91cGRh
dGUgbmVlZHMgb24gdGhlCiAgICAgICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1
c2ggd2hlbiBNRk4gb3IgZmxhZ3MgKGkuZS4gcGVybWlzc2lvbnMpIGNoYW5n
ZS4KQEAgLTY0Miw2ICs2NTIsNyBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVj
dCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgb2xkX21mbiA9
IGwxZV9nZXRfcGZuKCpwMm1fZW50cnkpOwogICAgICAgICAgICAgICAgIGlv
bW11X29sZF9mbGFncyA9CiAgICAgICAgICAgICAgICAgICAgIHAybV9nZXRf
aW9tbXVfZmxhZ3MocDJtX2ZsYWdzX3RvX3R5cGUoZmxhZ3MpLAorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9mbGFnc190
b19hY2Nlc3MoZmxhZ3MpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIF9tZm4ob2xkX21mbikpOwogICAgICAgICAgICAgfQog
ICAgICAgICAgICAgZWxzZQpAQCAtNjg0LDkgKzY5NSwxMCBAQCBwMm1fcHRf
c2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRS
SUVTKTsKICAgICAgICAgQVNTRVJUKHAybV9lbnRyeSk7CiAgICAgICAgIG9s
ZF9tZm4gPSBsMWVfZ2V0X3BmbigqcDJtX2VudHJ5KTsKKyAgICAgICAgZmxh
Z3MgPSBsMWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpOwogICAgICAgICBpb21t
dV9vbGRfZmxhZ3MgPQotICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFn
cyhwMm1fZmxhZ3NfdG9fdHlwZShsMWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkp
KSwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRf
bWZuKSk7CisgICAgICAgICAgICBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9m
bGFnc190b190eXBlKGZsYWdzKSwKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksIF9tZm4ob2xk
X21mbikpOwogCiAgICAgICAgIGlmICggbWZuX3ZhbGlkKG1mbikgfHwgcDJt
X2FsbG93c19pbnZhbGlkX21mbihwMm10KSApCiAgICAgICAgICAgICBlbnRy
eV9jb250ZW50ID0gcDJtX2wxZV9mcm9tX3BmbihtZm5feChtZm4pLApAQCAt
NzE0LDYgKzcyNiw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9k
b21haW4gKnAybSwKICAgICAgICAgICAgICAgICBvbGRfbWZuID0gbDFlX2dl
dF9wZm4oKnAybV9lbnRyeSk7CiAgICAgICAgICAgICAgICAgaW9tbXVfb2xk
X2ZsYWdzID0KICAgICAgICAgICAgICAgICAgICAgcDJtX2dldF9pb21tdV9m
bGFncyhwMm1fZmxhZ3NfdG9fdHlwZShmbGFncyksCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdzX3RvX2FjY2Vz
cyhmbGFncyksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgX21mbihvbGRfbWZuKSk7CiAgICAgICAgICAgICB9CiAgICAgICAg
ICAgICBlbHNlCi0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysr
IGIveGVuL2luY2x1ZGUvYXNtLXg4Ni9wMm0uaApAQCAtODkxLDcgKzg5MSw4
IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnUK
IC8qCiAgKiBwMm0gdHlwZSB0byBJT01NVSBmbGFncwogICovCi1zdGF0aWMg
aW5saW5lIHVuc2lnbmVkIGludCBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90
eXBlX3QgcDJtdCwgbWZuX3QgbWZuKQorc3RhdGljIGlubGluZSB1bnNpZ25l
ZCBpbnQgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fdHlwZV90IHAybXQsCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHAybV9hY2Nlc3NfdCBwMm1hLCBtZm5fdCBtZm4pCiB7CiAgICAgdW5zaWdu
ZWQgaW50IGZsYWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-0a.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-0a.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBmaXggUG9EIGFjY291bnRpbmcgaW4gZ3Vlc3RfcGh5c21h
cF9hZGRfZW50cnkoKQoKVGhlIGluaXRpYWwgb2JzZXJ2YXRpb24gd2FzIHRo
YXQgdGhlIG1mbl92YWxpZCgpIGNoZWNrIGNvbWVzIHRvbyBsYXRlOgpOZWl0
aGVyIG1mbl9hZGQoKSBub3IgbWZuX3RvX3BhZ2UoKSAobGV0IGFsb25lIGRl
LXJlZmVyZW5jaW5nIHRoZQpyZXN1bHQgb2YgdGhlIGxhdHRlcikgYXJlIHZh
bGlkIGZvciBNRk5zIGZhaWxpbmcgdGhpcyBjaGVjay4gTW92ZSBpdCB1cAph
bmQgLSBub3RpY2luZyB0aGF0IHRoZXJlJ3Mgbm8gY2FsbGVyIGRvaW5nIHNv
IC0gYWxzbyBhZGQgYW4gYXNzZXJ0aW9uCnRoYXQgdGhpcyBzaG91bGQgbmV2
ZXIgcHJvZHVjZSAiZmFsc2UiIGhlcmUuCgpJbiB0dXJuIHRoaXMgd291bGQg
aGF2ZSBtZWFudCB0aGF0IHRoZSAiZWxzZSIgdG8gdGhhdCBpZigpIGNvdWxk
IG5vdyBnbwphd2F5LCB3aGljaCBkaWRuJ3Qgc2VlbSByaWdodCBhdCBhbGwu
IEFuZCBpbmRlZWQsIGNvbnNpZGVyaW5nIGNhbGxlcnMKbGlrZSBtZW1vcnlf
ZXhjaGFuZ2UoKSBvciB2YXJpb3VzIGdyYW50IHRhYmxlIGZ1bmN0aW9ucywg
dGhlIFBvRAphY2NvdW50aW5nIHNob3VsZCBoYXZlIGJlZW4gb3V0c2lkZSBv
ZiB0aGF0IGlmKCkgZnJvbSB0aGUgdmVyeQpiZWdpbm5pbmcuCgpTaWduZWQt
b2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+CkFja2Vk
LWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29t
PgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3AybS5jCisrKyBiL3hlbi9hcmNo
L3g4Ni9tbS9wMm0uYwpAQCAtNzk0LDYgKzc5NCwxMiBAQCBndWVzdF9waHlz
bWFwX2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgaWYgKCBwMm1f
aXNfZm9yZWlnbih0KSApCiAgICAgICAgIHJldHVybiAtRUlOVkFMOwogCisg
ICAgaWYgKCAhbWZuX3ZhbGlkKG1mbikgKQorICAgIHsKKyAgICAgICAgQVNT
RVJUX1VOUkVBQ0hBQkxFKCk7CisgICAgICAgIHJldHVybiAtRUlOVkFMOwor
ICAgIH0KKwogICAgIHAybV9sb2NrKHAybSk7CiAKICAgICBQMk1fREVCVUco
ImFkZGluZyBnZm49JSNseCBtZm49JSNseFxuIiwgZ2ZuX3goZ2ZuKSwgbWZu
X3gobWZuKSk7CkBAIC04OTQsMTIgKzkwMCwxMyBAQCBndWVzdF9waHlzbWFw
X2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgfQogCiAgICAgLyog
Tm93LCBhY3R1YWxseSBkbyB0aGUgdHdvLXdheSBtYXBwaW5nICovCi0gICAg
aWYgKCBtZm5fdmFsaWQobWZuKSApCisgICAgcmMgPSBwMm1fc2V0X2VudHJ5
KHAybSwgZ2ZuLCBtZm4sIHBhZ2Vfb3JkZXIsIHQsIHAybS0+ZGVmYXVsdF9h
Y2Nlc3MpOworICAgIGlmICggcmMgPT0gMCApCiAgICAgewotICAgICAgICBy
YyA9IHAybV9zZXRfZW50cnkocDJtLCBnZm4sIG1mbiwgcGFnZV9vcmRlciwg
dCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybS0+ZGVmYXVsdF9h
Y2Nlc3MpOwotICAgICAgICBpZiAoIHJjICkKLSAgICAgICAgICAgIGdvdG8g
b3V0OyAvKiBGYWlsZWQgdG8gdXBkYXRlIHAybSwgYmFpbCB3aXRob3V0IHVw
ZGF0aW5nIG0ycC4gKi8KKyAgICAgICAgcG9kX2xvY2socDJtKTsKKyAgICAg
ICAgcDJtLT5wb2QuZW50cnlfY291bnQgLT0gcG9kX2NvdW50OworICAgICAg
ICBCVUdfT04ocDJtLT5wb2QuZW50cnlfY291bnQgPCAwKTsKKyAgICAgICAg
cG9kX3VubG9jayhwMm0pOwogCiAgICAgICAgIGlmICggIXAybV9pc19ncmFu
dCh0KSApCiAgICAgICAgIHsKQEAgLTkwOCwyMiArOTE1LDcgQEAgZ3Vlc3Rf
cGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFpbiAqZAogICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGdmbl94KGdmbl9hZGQoZ2ZuLCBp
KSkpOwogICAgICAgICB9CiAgICAgfQotICAgIGVsc2UKLSAgICB7Ci0gICAg
ICAgIGdkcHJpbnRrKFhFTkxPR19XQVJOSU5HLCAiQWRkaW5nIGJhZCBtZm4g
dG8gcDJtIG1hcCAoJSNseCAtPiAlI2x4KVxuIiwKLSAgICAgICAgICAgICAg
ICAgZ2ZuX3goZ2ZuKSwgbWZuX3gobWZuKSk7Ci0gICAgICAgIHJjID0gcDJt
X3NldF9lbnRyeShwMm0sIGdmbiwgSU5WQUxJRF9NRk4sIHBhZ2Vfb3JkZXIs
Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1faW52YWxpZCwgcDJt
LT5kZWZhdWx0X2FjY2Vzcyk7Ci0gICAgICAgIGlmICggcmMgPT0gMCApCi0g
ICAgICAgIHsKLSAgICAgICAgICAgIHBvZF9sb2NrKHAybSk7Ci0gICAgICAg
ICAgICBwMm0tPnBvZC5lbnRyeV9jb3VudCAtPSBwb2RfY291bnQ7Ci0gICAg
ICAgICAgICBCVUdfT04ocDJtLT5wb2QuZW50cnlfY291bnQgPCAwKTsKLSAg
ICAgICAgICAgIHBvZF91bmxvY2socDJtKTsKLSAgICAgICAgfQotICAgIH0K
IAotb3V0OgogICAgIHAybV91bmxvY2socDJtKTsKIAogICAgIHJldHVybiBy
YzsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-0b.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-0b.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBpZ25vcmUgcDJtX3JlbW92ZV9wYWdlKCkncyBy
ZXR1cm4gdmFsdWUKCkl0J3Mgbm90IHZlcnkgbmljZSB0byByZXR1cm4gZnJv
bSBndWVzdF9waHlzbWFwX2FkZF9lbnRyeSgpIGFmdGVyCnBlcmhhcHMgYWxy
ZWFkeSBoYXZpbmcgbWFkZSBzb21lIGNoYW5nZXMgdG8gdGhlIFAyTSwgYnV0
IHRoaXMgaXMgcHJlLQpleGlzdGluZyBwcmFjdGljZSBpbiB0aGUgZnVuY3Rp
b24sIGFuZCBpbW8gYmV0dGVyIHRoYW4gaWdub3JpbmcgZXJyb3JzLgoKVGFr
ZSB0aGUgbGliZXJ0eSBhbmQgcmVwbGFjZSBhbiBtZm5fYWRkKCkgaW5zdGFu
Y2Ugd2l0aCBhIGxvY2FsIHZhcmlhYmxlCmFscmVhZHkgaG9sZGluZyB0aGUg
cmVzdWx0IChhcyBwcm92ZW4gYnkgdGhlIGNoZWNrIGltbWVkaWF0ZWx5IGFo
ZWFkKS4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBz
dXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVycmFudCA8cGF1bC5kdXJy
YW50QGNpdHJpeC5jb20+CkFja2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgoKLS0tIGEveGVuL2FyY2gveDg2L21t
L3AybS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtNzAyLDgg
KzcwMiw3IEBAIHZvaWQgcDJtX2ZpbmFsX3RlYXJkb3duKHN0cnVjdCBkb21h
aW4gKmQKICAgICBwMm1fdGVhcmRvd25faG9zdHAybShkKTsKIH0KIAotCi1z
dGF0aWMgaW50CitzdGF0aWMgaW50IF9fbXVzdF9jaGVjawogcDJtX3JlbW92
ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpwMm0sIHVuc2lnbmVkIGxvbmcg
Z2ZuX2wsIHVuc2lnbmVkIGxvbmcgbWZuLAogICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBwYWdlX29yZGVyKQogewpAQCAtODkyLDkgKzg5MSw5IEBA
IGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAg
ICAgICAgICAgICAgICBBU1NFUlQobWZuX3ZhbGlkKG9tZm4pKTsKICAgICAg
ICAgICAgICAgICBQMk1fREVCVUcoIm9sZCBnZm49JSNseCAtPiBtZm4gJSNs
eFxuIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2ZuX3gob2dmbikg
LCBtZm5feChvbWZuKSk7Ci0gICAgICAgICAgICAgICAgaWYgKCBtZm5fZXEo
b21mbiwgbWZuX2FkZChtZm4sIGkpKSApCi0gICAgICAgICAgICAgICAgICAg
IHAybV9yZW1vdmVfcGFnZShwMm0sIGdmbl94KG9nZm4pLCBtZm5feChtZm5f
YWRkKG1mbiwgaSkpLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgMCk7CisgICAgICAgICAgICAgICAgaWYgKCBtZm5fZXEob21mbiwg
bWZuX2FkZChtZm4sIGkpKSAmJgorICAgICAgICAgICAgICAgICAgICAgKHJj
ID0gcDJtX3JlbW92ZV9wYWdlKHAybSwgZ2ZuX3gob2dmbiksIG1mbl94KG9t
Zm4pLCAwKSkgKQorICAgICAgICAgICAgICAgICAgICBnb3RvIG91dDsKICAg
ICAgICAgICAgIH0KICAgICAgICAgfQogICAgIH0KQEAgLTkxNiw2ICs5MTUs
NyBAQCBndWVzdF9waHlzbWFwX2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpk
CiAgICAgICAgIH0KICAgICB9CiAKKyBvdXQ6CiAgICAgcDJtX3VubG9jayhw
Mm0pOwogCiAgICAgcmV0dXJuIHJjOwpAQCAtMjM4NSw5ICsyMzg1LDkgQEAg
aW50IHAybV9jaGFuZ2VfYWx0cDJtX2dmbihzdHJ1Y3QgZG9tYWluCiAKICAg
ICBpZiAoIGdmbl9lcShuZXdfZ2ZuLCBJTlZBTElEX0dGTikgKQogICAgIHsK
LSAgICAgICAgaWYgKCBtZm5fdmFsaWQobWZuKSApCi0gICAgICAgICAgICBw
Mm1fcmVtb3ZlX3BhZ2UoYXAybSwgZ2ZuX3gob2xkX2dmbiksIG1mbl94KG1m
biksIFBBR0VfT1JERVJfNEspOwotICAgICAgICByYyA9IDA7CisgICAgICAg
IHJjID0gbWZuX3ZhbGlkKG1mbikKKyAgICAgICAgICAgICA/IHAybV9yZW1v
dmVfcGFnZShhcDJtLCBnZm5feChvbGRfZ2ZuKSwgbWZuX3gobWZuKSwgUEFH
RV9PUkRFUl80SykKKyAgICAgICAgICAgICA6IDA7CiAgICAgICAgIGdvdG8g
b3V0OwogICAgIH0KIAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-0c.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-0c.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBhc3NlcnQgdGhhdCB0aGUgcGFzc2VkIGluIE1G
TiBtYXRjaGVzIGZvciBhIHJlbW92ZQoKZ3Vlc3RfcGh5c21hcF9yZW1vdmVf
cGFnZSgpIGdldHMgaGFuZGVkIGFuIE1GTiBmcm9tIHRoZSBvdXRzaWRlLCB5
ZXQKdGFrZXMgdGhlIG5lY2Vzc2FyeSBsb2NrIHRvIHByZXZlbnQgZnVydGhl
ciBjaGFuZ2VzIHRvIHRoZSBHRk4gPC0+IE1GTgptYXBwaW5nIGl0c2VsZi4g
V2hpbGUgc29tZSBjYWxsZXJzLCBpbiBwYXJ0aWN1bGFyIGd1ZXN0X3JlbW92
ZV9wYWdlKCkKKGJ5IHdheSBvZiBoYXZpbmcgY2FsbGVkIGdldF9nZm5fcXVl
cnkoKSksIGhvbGQgdGhlIEdGTiBsb2NrIGFscmVhZHksCnZhcmlvdXMgb3Ro
ZXJzIChtb3N0IG5vdGFibHkgcGVyaGFwcyB0aGUgMm5kIGluc3RhbmNlIGlu
Cnhlbm1lbV9hZGRfdG9fcGh5c21hcF9vbmUoKSkgZG9uJ3QuIFdoaWxlIGl0
IGFsc28gaXMgYW4gb3B0aW9uIHRvIGZpeAphbGwgdGhlIGNhbGxlcnMsIGRl
YWwgd2l0aCB0aGUgaXNzdWUgaW4gcDJtX3JlbW92ZV9wYWdlKCkgaW5zdGVh
ZDoKUmVwbGFjZSB0aGUgQVNTRVJUKCkgYnkgYSBjb25kaXRpb25hbCBhbmQg
c3BsaXQgdGhlIGxvb3AgaW50byB0d28sIHN1Y2gKdGhhdCBhbGwgY2hlY2tp
bmcgZ2V0cyBkb25lIGJlZm9yZSBhbnkgbW9kaWZpY2F0aW9uIHdvdWxkIG9j
Y3VyLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1
c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsLmR1cnJh
bnRAY2l0cml4LmNvbT4KQWNrZWQtYnk6IEFuZHJldyBDb29wZXIgPGFuZHJl
dy5jb29wZXIzQGNpdHJpeC5jb20+CgotLS0gYS94ZW4vYXJjaC94ODYvbW0v
cDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC03MDgsNyAr
NzA4LDYgQEAgcDJtX3JlbW92ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpw
Mm0sCiB7CiAgICAgdW5zaWduZWQgbG9uZyBpOwogICAgIGdmbl90IGdmbiA9
IF9nZm4oZ2ZuX2wpOwotICAgIG1mbl90IG1mbl9yZXR1cm47CiAgICAgcDJt
X3R5cGVfdCB0OwogICAgIHAybV9hY2Nlc3NfdCBhOwogCkBAIC03MTksMTUg
KzcxOCwyNiBAQCBwMm1fcmVtb3ZlX3BhZ2Uoc3RydWN0IHAybV9kb21haW4g
KnAybSwKICAgICBBU1NFUlQoZ2ZuX2xvY2tlZF9ieV9tZShwMm0sIGdmbikp
OwogICAgIFAyTV9ERUJVRygicmVtb3ZpbmcgZ2ZuPSUjbHggbWZuPSUjbHhc
biIsIGdmbl9sLCBtZm4pOwogCisgICAgZm9yICggaSA9IDA7IGkgPCAoMVVM
IDw8IHBhZ2Vfb3JkZXIpOyApCisgICAgeworICAgICAgICB1bnNpZ25lZCBp
bnQgY3VyX29yZGVyOworICAgICAgICBtZm5fdCBtZm5fcmV0dXJuID0gcDJt
LT5nZXRfZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZ0LCAmYSwgMCwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICZj
dXJfb3JkZXIsIE5VTEwpOworCisgICAgICAgIGlmICggcDJtX2lzX3ZhbGlk
KHQpICYmCisgICAgICAgICAgICAgKCFtZm5fdmFsaWQoX21mbihtZm4pKSB8
fCBtZm4gKyBpICE9IG1mbl94KG1mbl9yZXR1cm4pKSApCisgICAgICAgICAg
ICByZXR1cm4gLUVJTFNFUTsKKworICAgICAgICBpICs9ICgxVUwgPDwgY3Vy
X29yZGVyKSAtICgoZ2ZuX2wgKyBpKSAmICgoMVVMIDw8IGN1cl9vcmRlcikg
LSAxKSk7CisgICAgfQorCiAgICAgaWYgKCBtZm5fdmFsaWQoX21mbihtZm4p
KSApCiAgICAgewogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwg
cGFnZV9vcmRlcik7IGkrKyApCiAgICAgICAgIHsKLSAgICAgICAgICAgIG1m
bl9yZXR1cm4gPSBwMm0tPmdldF9lbnRyeShwMm0sIGdmbl9hZGQoZ2ZuLCBp
KSwgJnQsICZhLCAwLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIE5VTEwsIE5VTEwpOworICAgICAgICAgICAgcDJtLT5nZXRf
ZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZ0LCAmYSwgMCwgTlVMTCwg
TlVMTCk7CiAgICAgICAgICAgICBpZiAoICFwMm1faXNfZ3JhbnQodCkgJiYg
IXAybV9pc19zaGFyZWQodCkgJiYgIXAybV9pc19mb3JlaWduKHQpICkKICAg
ICAgICAgICAgICAgICBzZXRfZ3Bmbl9mcm9tX21mbihtZm4raSwgSU5WQUxJ
RF9NMlBfRU5UUlkpOwotICAgICAgICAgICAgQVNTRVJUKCAhcDJtX2lzX3Zh
bGlkKHQpIHx8IG1mbiArIGkgPT0gbWZuX3gobWZuX3JldHVybikgKTsKICAg
ICAgICAgfQogICAgIH0KICAgICByZXR1cm4gcDJtX3NldF9lbnRyeShwMm0s
IGdmbiwgSU5WQUxJRF9NRk4sIHBhZ2Vfb3JkZXIsIHAybV9pbnZhbGlkLAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9p
b21tdV9hY3BpLmMKQEAgLTk4LDEyICs5OCwyMSBAQCBzdGF0aWMgc3RydWN0
IGFtZF9pb21tdSAqIF9faW5pdCBmaW5kX2lvCiAgICAgcmV0dXJuIE5VTEw7
CiB9CiAKLXN0YXRpYyB2b2lkIF9faW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZSgKLSAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwgdWlu
dDY0X3QgYmFzZSwgdWludDY0X3QgbGltaXQpCitzdGF0aWMgaW50IF9faW5p
dCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZSgKKyAgICBzdHJ1Y3Qg
YW1kX2lvbW11ICppb21tdSwgdWludDY0X3QgYmFzZSwgdWludDY0X3QgbGlt
aXQsCisgICAgYm9vbCBhbGwsIGJvb2wgaXcsIGJvb2wgaXIpCiB7CisgICAg
aWYgKCAhaXIgfHwgIWl3ICkKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKKwog
ICAgIC8qIG5lZWQgdG8gZXh0ZW5kIGV4Y2x1c2lvbiByYW5nZT8gKi8KICAg
ICBpZiAoIGlvbW11LT5leGNsdXNpb25fZW5hYmxlICkKICAgICB7CisgICAg
ICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9saW1pdCArIFBBR0VfU0laRSA8
IGJhc2UgfHwKKyAgICAgICAgICAgICBsaW1pdCArIFBBR0VfU0laRSA8IGlv
bW11LT5leGNsdXNpb25fYmFzZSB8fAorICAgICAgICAgICAgIGlvbW11LT5l
eGNsdXNpb25fYWxsb3dfYWxsICE9IGFsbCApCisgICAgICAgICAgICByZXR1
cm4gLUVCVVNZOworCiAgICAgICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9i
YXNlIDwgYmFzZSApCiAgICAgICAgICAgICBiYXNlID0gaW9tbXUtPmV4Y2x1
c2lvbl9iYXNlOwogICAgICAgICBpZiAoIGlvbW11LT5leGNsdXNpb25fbGlt
aXQgPiBsaW1pdCApCkBAIC0xMTEsMTYgKzEyMCwxMSBAQCBzdGF0aWMgdm9p
ZCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpCiAgICAgfQogCiAgICAg
aW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgPSBJT01NVV9DT05UUk9MX0VOQUJM
RUQ7CisgICAgaW9tbXUtPmV4Y2x1c2lvbl9hbGxvd19hbGwgPSBhbGw7CiAg
ICAgaW9tbXUtPmV4Y2x1c2lvbl9iYXNlID0gYmFzZTsKICAgICBpb21tdS0+
ZXhjbHVzaW9uX2xpbWl0ID0gbGltaXQ7Ci19CiAKLXN0YXRpYyB2b2lkIF9f
aW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9hbGwoCi0gICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXUsCi0gICAgdW5zaWduZWQgbG9uZyBi
YXNlLCB1bnNpZ25lZCBsb25nIGxpbWl0KQotewotICAgIHJlc2VydmVfaW9t
bXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCk7Ci0gICAg
aW9tbXUtPmV4Y2x1c2lvbl9hbGxvd19hbGwgPSBJT01NVV9DT05UUk9MX0VO
QUJMRUQ7CisgICAgcmV0dXJuIDA7CiB9CiAKIHN0YXRpYyB2b2lkIF9faW5p
dCByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKApAQCAtMTU4LDYgKzE2
Miw3IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW4KICAgICB1bnNpZ25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBs
ZW5ndGg7CiAgICAgc3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgdW5z
aWduZWQgaW50IGJkZjsKKyAgICBpbnQgcmMgPSAwOwogCiAgICAgLyogaXMg
cGFydCBvZiBleGNsdXNpb24gcmFuZ2UgaW5zaWRlIG9mIElPTU1VIHZpcnR1
YWwgYWRkcmVzcyBzcGFjZT8gKi8KICAgICAvKiBub3RlOiAnbGltaXQnIHBh
cmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwpAQCAt
MTc5LDEwICsxODQsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJf
ZXhjbHVzaW9uX3JhbgogICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkK
ICAgICB7CiAgICAgICAgIGZvcl9lYWNoX2FtZF9pb21tdSggaW9tbXUgKQot
ICAgICAgICAgICAgcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2VfYWxs
KGlvbW11LCBiYXNlLCBsaW1pdCk7CisgICAgICAgIHsKKyAgICAgICAgICAg
IHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJh
c2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB0cnVlIC8qIGFsbCAqLywgaXcsIGlyKTsKKyAgICAg
ICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAgICAgIGJyZWFrOworICAg
ICAgICB9CiAgICAgfQogCi0gICAgcmV0dXJuIDA7CisgICAgcmV0dXJuIHJj
OwogfQogCiBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25f
cmFuZ2VfZm9yX2RldmljZSgKQEAgLTE5Myw2ICsyMDMsNyBAQCBzdGF0aWMg
aW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgdW5zaWdu
ZWQgbG9uZyByYW5nZV90b3AsIGlvbW11X3RvcCwgbGVuZ3RoOwogICAgIHN0
cnVjdCBhbWRfaW9tbXUgKmlvbW11OwogICAgIHUxNiByZXE7CisgICAgaW50
IHJjID0gMDsKIAogICAgIGlvbW11ID0gZmluZF9pb21tdV9mb3JfZGV2aWNl
KHNlZywgYmRmKTsKICAgICBpZiAoICFpb21tdSApCkBAIC0yMjIsMTIgKzIz
MywxMyBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25f
cmFuCiAgICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNl
dHRpbmdzIGZvciBkZXZpY2UgKi8KICAgICBpZiAoIGxpbWl0ID49IGlvbW11
X3RvcCAgKQogICAgIHsKLSAgICAgICAgcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0KTsKKyAgICAgICAgcmMgPSBy
ZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGlt
aXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwogICAgICAgICBpdnJzX21h
cHBpbmdzW2JkZl0uZHRlX2FsbG93X2V4Y2x1c2lvbiA9IElPTU1VX0NPTlRS
T0xfRU5BQkxFRDsKICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFdLmR0ZV9h
bGxvd19leGNsdXNpb24gPSBJT01NVV9DT05UUk9MX0VOQUJMRUQ7CiAgICAg
fQogCi0gICAgcmV0dXJuIDA7CisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0
aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2lv
bW11X2RldmljZXMoCkBAIC0yMzcsNiArMjQ5LDcgQEAgc3RhdGljIGludCBf
X2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgIHVuc2lnbmVkIGxv
bmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAgICB1bnNpZ25l
ZCBpbnQgYmRmOwogICAgIHUxNiByZXE7CisgICAgaW50IHJjID0gMDsKIAog
ICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJ
T01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICovCiAgICAgLyogbm90ZTog
J2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWdu
ZWQgKi8KQEAgLTI2Nyw4ICsyODAsMTAgQEAgc3RhdGljIGludCBfX2luaXQg
cmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogCiAgICAgLyogcmVnaXN0ZXIgSU9N
TVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzICovCiAgICAgaWYgKCBsaW1p
dCA+PSBpb21tdV90b3AgKQotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZV9hbGwoaW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAgICByZXR1
cm4gMDsKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9y
YW5nZShpb21tdSwgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3LCBp
cik7CisKKyAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0
IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdCgK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdo
L2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtMTI3LDMyICsxMjcsNDggQEAgc3Rh
dGljIGludCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpbwogICAgIHJl
dHVybiAwOwogfQogCi1zdGF0aWMgdm9pZCBfX2luaXQgcmVzZXJ2ZV91bml0
eV9tYXBfZm9yX2RldmljZSgKLSAgICB1MTYgc2VnLCB1MTYgYmRmLCB1bnNp
Z25lZCBsb25nIGJhc2UsCi0gICAgdW5zaWduZWQgbG9uZyBsZW5ndGgsIHU4
IGl3LCB1OCBpcikKK3N0YXRpYyBpbnQgX19pbml0IHJlc2VydmVfdW5pdHlf
bWFwX2Zvcl9kZXZpY2UoCisgICAgdWludDE2X3Qgc2VnLCB1aW50MTZfdCBi
ZGYsIHVuc2lnbmVkIGxvbmcgYmFzZSwKKyAgICB1bnNpZ25lZCBsb25nIGxl
bmd0aCwgYm9vbCBpdywgYm9vbCBpcikKIHsKICAgICBzdHJ1Y3QgaXZyc19t
YXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHNl
Zyk7Ci0gICAgdW5zaWduZWQgbG9uZyBvbGRfdG9wLCBuZXdfdG9wOworICAg
IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwID0gaXZyc19tYXBw
aW5nc1tiZGZdLnVuaXR5X21hcDsKIAotICAgIC8qIG5lZWQgdG8gZXh0ZW5k
IHVuaXR5LW1hcHBlZCByYW5nZT8gKi8KLSAgICBpZiAoIGl2cnNfbWFwcGlu
Z3NbYmRmXS51bml0eV9tYXBfZW5hYmxlICkKKyAgICAvKiBDaGVjayBmb3Ig
b3ZlcmxhcHMuICovCisgICAgZm9yICggOyB1bml0eV9tYXA7IHVuaXR5X21h
cCA9IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewotICAgICAgICBvbGRfdG9w
ID0gaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQgKwotICAg
ICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2VfbGVuZ3Ro
OwotICAgICAgICBuZXdfdG9wID0gYmFzZSArIGxlbmd0aDsKLSAgICAgICAg
aWYgKCBvbGRfdG9wID4gbmV3X3RvcCApCi0gICAgICAgICAgICBuZXdfdG9w
ID0gb2xkX3RvcDsKLSAgICAgICAgaWYgKCBpdnJzX21hcHBpbmdzW2JkZl0u
YWRkcl9yYW5nZV9zdGFydCA8IGJhc2UgKQotICAgICAgICAgICAgYmFzZSA9
IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3JhbmdlX3N0YXJ0OwotICAgICAg
ICBsZW5ndGggPSBuZXdfdG9wIC0gYmFzZTsKLSAgICB9Ci0KLSAgICAvKiBl
eHRlbmQgci93IHBlcm1pc3Npb21zIGFuZCBrZWVwIGFnZ3JlZ2F0ZSAqLwot
ICAgIGl2cnNfbWFwcGluZ3NbYmRmXS53cml0ZV9wZXJtaXNzaW9uID0gaXc7
Ci0gICAgaXZyc19tYXBwaW5nc1tiZGZdLnJlYWRfcGVybWlzc2lvbiA9IGly
OwotICAgIGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXBfZW5hYmxlID0g
SU9NTVVfQ09OVFJPTF9FTkFCTEVEOwotICAgIGl2cnNfbWFwcGluZ3NbYmRm
XS5hZGRyX3JhbmdlX3N0YXJ0ID0gYmFzZTsKLSAgICBpdnJzX21hcHBpbmdz
W2JkZl0uYWRkcl9yYW5nZV9sZW5ndGggPSBsZW5ndGg7CisgICAgICAgIC8q
CisgICAgICAgICAqIEV4YWN0IG1hdGNoZXMgYXJlIG9rYXkuIFRoaXMgY2Fu
IGluIHBhcnRpY3VsYXIgaGFwcGVuIHdoZW4KKyAgICAgICAgICogcmVnaXN0
ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3
aWNlIGZvciB0aGUKKyAgICAgICAgICogc2FtZSAocyxiLGQsZikuCisgICAg
ICAgICAqLworICAgICAgICBpZiAoIGJhc2UgPT0gdW5pdHlfbWFwLT5hZGRy
ICYmIGxlbmd0aCA9PSB1bml0eV9tYXAtPmxlbmd0aCAmJgorICAgICAgICAg
ICAgIGlyID09IHVuaXR5X21hcC0+cmVhZCAmJiBpdyA9PSB1bml0eV9tYXAt
PndyaXRlICkKKyAgICAgICAgICAgIHJldHVybiAwOworCisgICAgICAgIGlm
ICggdW5pdHlfbWFwLT5hZGRyICsgdW5pdHlfbWFwLT5sZW5ndGggPiBiYXNl
ICYmCisgICAgICAgICAgICAgYmFzZSArIGxlbmd0aCA+IHVuaXR5X21hcC0+
YWRkciApCisgICAgICAgIHsKKyAgICAgICAgICAgIEFNRF9JT01NVV9ERUJV
RygiSVZNRCBFcnJvcjogb3ZlcmxhcCBbJWx4LCVseCkgdnMgWyVseCwlbHgp
XG4iLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJhc2UsIGJhc2Ug
KyBsZW5ndGgsIHVuaXR5X21hcC0+YWRkciwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bml0eV9tYXAtPmFkZHIgKyB1bml0eV9tYXAtPmxlbmd0
aCk7CisgICAgICAgICAgICByZXR1cm4gLUVQRVJNOworICAgICAgICB9Cisg
ICAgfQorCisgICAgLyogUG9wdWxhdGUgYW5kIGluc2VydCBhIG5ldyB1bml0
eSBtYXAuICovCisgICAgdW5pdHlfbWFwID0geG1hbGxvYyhzdHJ1Y3QgaXZy
c191bml0eV9tYXApOworICAgIGlmICggIXVuaXR5X21hcCApCisgICAgICAg
IHJldHVybiAtRU5PTUVNOworCisgICAgdW5pdHlfbWFwLT5yZWFkID0gaXI7
CisgICAgdW5pdHlfbWFwLT53cml0ZSA9IGl3OworICAgIHVuaXR5X21hcC0+
YWRkciA9IGJhc2U7CisgICAgdW5pdHlfbWFwLT5sZW5ndGggPSBsZW5ndGg7
CisgICAgdW5pdHlfbWFwLT5uZXh0ID0gaXZyc19tYXBwaW5nc1tiZGZdLnVu
aXR5X21hcDsKKyAgICBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwID0g
dW5pdHlfbWFwOworCisgICAgcmV0dXJuIDA7CiB9CiAKIHN0YXRpYyBpbnQg
X19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfYWxsX2Rldmlj
ZXMoCkBAIC0xNzUsMTMgKzE5MSwxMyBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgICAgIGxlbmd0aCA9IHJhbmdl
X3RvcCAtIGJhc2U7CiAgICAgICAgIC8qIHJlc2VydmUgci93IHVuaXR5LW1h
cHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMgKi8KICAgICAgICAgLyog
bm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9u
IHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9IDA7IGJkZiA8IGl2cnNf
YmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAgICAgICAgIHJlc2VydmVfdW5p
dHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcs
IGlyKTsKKyAgICAgICAgZm9yICggYmRmID0gMDsgIXJjICYmIGJkZiA8IGl2
cnNfYmRmX2VudHJpZXM7IGJkZisrICkKKyAgICAgICAgICAgIHJjID0gcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVu
Z3RoLCBpdywgaXIpOwogICAgICAgICAvKiBwdXNoICdiYXNlJyBqdXN0IG91
dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAgICAgIGJh
c2UgPSBpb21tdV90b3A7CiAgICAgfQogICAgIC8qIHJlZ2lzdGVyIElPTU1V
IGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICggbGltaXQg
Pj0gaW9tbXVfdG9wICkKKyAgICBpZiAoICFyYyAmJiBsaW1pdCA+PSBpb21t
dV90b3AgKQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBp
b21tdSApCiAgICAgICAgIHsKQEAgLTIyMywxNSArMjM5LDE1IEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICAgICAg
bGVuZ3RoID0gcmFuZ2VfdG9wIC0gYmFzZTsKICAgICAgICAgLyogcmVzZXJ2
ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2UgKi8KICAg
ICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUg
ZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsK
LSAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIHJl
cSwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOworICAgICAgICByYyA9IHJlc2Vy
dmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0
aCwgaXcsIGlyKSA/OgorICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsK
IAogICAgICAgICAvKiBwdXNoICdiYXNlJyBqdXN0IG91dHNpZGUgb2Ygdmly
dHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAgICAgIGJhc2UgPSBpb21tdV90
b3A7CiAgICAgfQogCiAgICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9u
IHJhbmdlIHNldHRpbmdzIGZvciBkZXZpY2UgKi8KLSAgICBpZiAoIGxpbWl0
ID49IGlvbW11X3RvcCAgKQorICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlv
bW11X3RvcCAgKQogICAgIHsKICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UgLyog
YWxsICovLCBpdywgaXIpOwpAQCAtMjYyLDE1ICsyNzgsMTUgQEAgc3RhdGlj
IGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgICAgICBs
ZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwogICAgICAgICAvKiByZXNlcnZl
IHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICov
CiAgICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJlIHBhcnQgb2Yg
dGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAgICBmb3IgKCBiZGYgPSAw
OyBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCisgICAgICAgIGZv
ciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBi
ZGYrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIGlmICggaW9tbXUgPT0g
ZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQogICAg
ICAgICAgICAgewotICAgICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBp
dywgaXIpOwogICAgICAgICAgICAgICAgIHJlcSA9IGdldF9pdnJzX21hcHBp
bmdzKGlvbW11LT5zZWcpW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZDsKLSAgICAg
ICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11
LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAotICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyKTsKKyAgICAgICAg
ICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcikgPzoK
KyAgICAgICAgICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9k
ZXZpY2UoaW9tbXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCisgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3
LCBpcik7CiAgICAgICAgICAgICB9CiAgICAgICAgIH0KIApAQCAtMjc5LDcg
KzI5NSw3IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW4KICAgICB9CiAKICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNp
b24gcmFuZ2Ugc2V0dGluZ3MgKi8KLSAgICBpZiAoIGxpbWl0ID49IGlvbW11
X3RvcCApCisgICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkK
ICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShp
b21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3LCBpcik7CiAK
LS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2luaXQu
YworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXVfaW5p
dC5jCkBAIC0xMTg3LDcgKzExODcsNiBAQCBzdGF0aWMgaW50IF9faW5pdCBh
bGxvY19pdnJzX21hcHBpbmdzKHUxCiAgICAgewogICAgICAgICBpdnJzX21h
cHBpbmdzW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZCA9IGJkZjsKICAgICAgICAg
aXZyc19tYXBwaW5nc1tiZGZdLmR0ZV9hbGxvd19leGNsdXNpb24gPSBJT01N
VV9DT05UUk9MX0RJU0FCTEVEOwotICAgICAgICBpdnJzX21hcHBpbmdzW2Jk
Zl0udW5pdHlfbWFwX2VuYWJsZSA9IElPTU1VX0NPTlRST0xfRElTQUJMRUQ7
CiAgICAgICAgIGl2cnNfbWFwcGluZ3NbYmRmXS5pb21tdSA9IE5VTEw7CiAK
ICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmludHJlbWFwX3RhYmxlID0g
TlVMTDsKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9h
bWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQv
cGNpX2FtZF9pb21tdS5jCkBAIC0zNzIsMTUgKzM3MiwxNyBAQCBzdGF0aWMg
aW50IGFtZF9pb21tdV9hc3NpZ25fZGV2aWNlKHN0cnVjCiAgICAgc3RydWN0
IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBw
aW5ncyhwZGV2LT5zZWcpOwogICAgIGludCBiZGYgPSBQQ0lfQkRGMihwZGV2
LT5idXMsIGRldmZuKTsKICAgICBpbnQgcmVxX2lkID0gZ2V0X2RtYV9yZXF1
ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYpOworICAgIGNvbnN0IHN0cnVjdCBp
dnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwOwogCi0gICAgaWYgKCBpdnJzX21h
cHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwX2VuYWJsZSApCisgICAgZm9yICgg
dW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcDsg
dW5pdHlfbWFwOworICAgICAgICAgIHVuaXR5X21hcCA9IHVuaXR5X21hcC0+
bmV4dCApCiAgICAgewotICAgICAgICBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21h
aW5fdW5pdHlfbWFwKAotICAgICAgICAgICAgZCwKLSAgICAgICAgICAgIGl2
cnNfbWFwcGluZ3NbcmVxX2lkXS5hZGRyX3JhbmdlX3N0YXJ0LAotICAgICAg
ICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLmFkZHJfcmFuZ2VfbGVuZ3Ro
LAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLndyaXRlX3Bl
cm1pc3Npb24sCi0gICAgICAgICAgICBpdnJzX21hcHBpbmdzW3JlcV9pZF0u
cmVhZF9wZXJtaXNzaW9uKTsKKyAgICAgICAgaW50IHJjID0gYW1kX2lvbW11
X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAgICAgICAgICAgICAg
ICAgIGQsIHVuaXR5X21hcC0+YWRkciwgdW5pdHlfbWFwLT5sZW5ndGgsCisg
ICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPndyaXRlLCB1bml0eV9t
YXAtPnJlYWQpOworCisgICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAg
cmV0dXJuIHJjOwogICAgIH0KIAogICAgIHJldHVybiByZWFzc2lnbl9kZXZp
Y2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwgcGRldik7Ci0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvYW1kLWlvbW11LmgKKysrIGIveGVuL2luY2x1ZGUv
YXNtLXg4Ni9hbWQtaW9tbXUuaApAQCAtMTA4LDE1ICsxMDgsMTkgQEAgc3Ry
dWN0IGFtZF9pb21tdSB7CiAgICAgc3RydWN0IGxpc3RfaGVhZCBhdHNfZGV2
aWNlczsKIH07CiAKK3N0cnVjdCBpdnJzX3VuaXR5X21hcCB7CisgICAgYm9v
bCByZWFkOjE7CisgICAgYm9vbCB3cml0ZToxOworICAgIHBhZGRyX3QgYWRk
cjsKKyAgICB1bnNpZ25lZCBsb25nIGxlbmd0aDsKKyAgICBzdHJ1Y3QgaXZy
c191bml0eV9tYXAgKm5leHQ7Cit9OworCiBzdHJ1Y3QgaXZyc19tYXBwaW5n
cyB7CiAgICAgdTE2IGR0ZV9yZXF1ZXN0b3JfaWQ7CiAgICAgdTggZHRlX2Fs
bG93X2V4Y2x1c2lvbjsKLSAgICB1OCB1bml0eV9tYXBfZW5hYmxlOwotICAg
IHU4IHdyaXRlX3Blcm1pc3Npb247Ci0gICAgdTggcmVhZF9wZXJtaXNzaW9u
OwotICAgIHVuc2lnbmVkIGxvbmcgYWRkcl9yYW5nZV9zdGFydDsKLSAgICB1
bnNpZ25lZCBsb25nIGFkZHJfcmFuZ2VfbGVuZ3RoOwogICAgIHN0cnVjdCBh
bWRfaW9tbXUgKmlvbW11OworICAgIHN0cnVjdCBpdnJzX3VuaXR5X21hcCAq
dW5pdHlfbWFwOwogCiAgICAgLyogcGVyIGRldmljZSBpbnRlcnJ1cHQgcmVt
YXBwaW5nIHRhYmxlICovCiAgICAgdm9pZCAqaW50cmVtYXBfdGFibGU7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0tZXB0
LmMKQEAgLTcxMSw3ICs3MTEsNyBAQCBlcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmCiAgICAgdWludDhfdCBpcGF0ID0gMDsKICAg
ICBib29sX3QgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMTsKICAgICBib29s
X3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsKLSAgICB1bnNpZ25lZCBpbnQgaW9t
bXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIG1mbik7Cisg
ICAgdW5zaWduZWQgaW50IGlvbW11X2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBwMm1hLCBtZm4pOwogICAgIGJvb2xfdCBuZWVkc19zeW5j
ID0gMTsKICAgICBlcHRfZW50cnlfdCBvbGRfZW50cnkgPSB7IC5lcHRlID0g
MCB9OwogICAgIGVwdF9lbnRyeV90IG5ld19lbnRyeSA9IHsgLmVwdGUgPSAw
IH07CkBAIC04MzcsOCArODM3LDggQEAgZXB0X3NldF9lbnRyeShzdHJ1Y3Qg
cDJtX2RvbWFpbiAqcDJtLCBnZgogCiAgICAgICAgIC8qIFNhZmUgdG8gcmVh
ZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2UgaG9sZCB0aGUgcDJtIGxvY2sgKi8K
ICAgICAgICAgaWYgKCBlcHRfZW50cnktPm1mbiA9PSBuZXdfZW50cnkubWZu
ICYmCi0gICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhlcHRfZW50
cnktPnNhX3AybXQsIF9tZm4oZXB0X2VudHJ5LT5tZm4pKSA9PQotICAgICAg
ICAgICAgIGlvbW11X2ZsYWdzICkKKyAgICAgICAgICAgICBwMm1fZ2V0X2lv
bW11X2ZsYWdzKGVwdF9lbnRyeS0+c2FfcDJtdCwgZXB0X2VudHJ5LT5hY2Nl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBfbWZuKGVw
dF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVfZmxhZ3MgKQogICAgICAgICAgICAg
bmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMDsKIAogICAgICAgICBlcHRfcDJt
X3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3X2VudHJ5LCBwMm10LCBwMm1hKTsK
LS0tIGEveGVuL2FyY2gveDg2L21tL3AybS1wdC5jCisrKyBiL3hlbi9hcmNo
L3g4Ni9tbS9wMm0tcHQuYwpAQCAtNDcxLDYgKzQ3MSwxNiBAQCBpbnQgcDJt
X3B0X2hhbmRsZV9kZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAgICAgcmV0dXJu
IHJjOwogfQogCisvKiBSZWNvbnN0cnVjdCBhIGZha2UgcDJtX2FjY2Vzc190
IGZyb20gc3RvcmVkIFBURSBmbGFncy4gKi8KK3N0YXRpYyBwMm1fYWNjZXNz
X3QgcDJtX2ZsYWdzX3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQgZmxhZ3MpCit7
CisgICAgaWYgKCBmbGFncyAmIF9QQUdFX1BSRVNFTlQgKQorICAgICAgICBy
ZXR1cm4gcDJtX2FjY2Vzc19uOworCisgICAgLyogTm8gbmVlZCB0byBsb29r
IGF0IF9QQUdFX05YIGZvciBub3cuICovCisgICAgcmV0dXJuIGZsYWdzICYg
X1BBR0VfUlcgPyBwMm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vzc19yOworfQor
CiAvKiBSZXR1cm5zOiAwIGZvciBzdWNjZXNzLCAtZXJybm8gZm9yIGZhaWx1
cmUgKi8KIHN0YXRpYyBpbnQKIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAy
bV9kb21haW4gKnAybSwgZ2ZuX3QgZ2ZuXywgbWZuX3QgbWZuLApAQCAtNDg3
LDcgKzQ5Nyw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21h
aW4gKnAybSwKICAgICBsMl9wZ2VudHJ5X3QgbDJlX2NvbnRlbnQ7CiAgICAg
bDNfcGdlbnRyeV90IGwzZV9jb250ZW50OwogICAgIGludCByYzsKLSAgICB1
bnNpZ25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBtZm4pOworICAgIHVuc2lnbmVkIGludCBpb21tdV9wdGVf
ZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIHAybWEsIG1mbik7
CiAgICAgLyoKICAgICAgKiBvbGRfbWZuIGFuZCBpb21tdV9vbGRfZmxhZ3Mg
Y29udHJvbCBwb3NzaWJsZSBmbHVzaC91cGRhdGUgbmVlZHMgb24gdGhlCiAg
ICAgICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1c2ggd2hlbiBNRk4gb3IgZmxh
Z3MgKGkuZS4gcGVybWlzc2lvbnMpIGNoYW5nZS4KQEAgLTU1Niw2ICs1NjYs
NyBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9tYWluICpwMm0s
CiAgICAgICAgICAgICAgICAgb2xkX21mbiA9IGwxZV9nZXRfcGZuKCpwMm1f
ZW50cnkpOwogICAgICAgICAgICAgICAgIGlvbW11X29sZF9mbGFncyA9CiAg
ICAgICAgICAgICAgICAgICAgIHAybV9nZXRfaW9tbXVfZmxhZ3MocDJtX2Zs
YWdzX3RvX3R5cGUoZmxhZ3MpLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHAybV9mbGFnc190b19hY2Nlc3MoZmxhZ3MpLAog
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF9tZm4o
b2xkX21mbikpOwogICAgICAgICAgICAgfQogICAgICAgICAgICAgZWxzZQpA
QCAtNjAyLDkgKzYxMywxMCBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRSSUVTKTsKICAgICAgICAgQVNT
RVJUKHAybV9lbnRyeSk7CiAgICAgICAgIG9sZF9tZm4gPSBsMWVfZ2V0X3Bm
bigqcDJtX2VudHJ5KTsKKyAgICAgICAgZmxhZ3MgPSBsMWVfZ2V0X2ZsYWdz
KCpwMm1fZW50cnkpOwogICAgICAgICBpb21tdV9vbGRfZmxhZ3MgPQotICAg
ICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9fdHlw
ZShsMWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpKSwKLSAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7CisgICAgICAgICAg
ICBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9mbGFnc190b190eXBlKGZsYWdz
KSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdz
X3RvX2FjY2VzcyhmbGFncyksIF9tZm4ob2xkX21mbikpOwogCiAgICAgICAg
IGlmICggbWZuX3ZhbGlkKG1mbikgfHwgcDJtX2FsbG93c19pbnZhbGlkX21m
bihwMm10KSApCiAgICAgICAgICAgICBlbnRyeV9jb250ZW50ID0gcDJtX2wx
ZV9mcm9tX3BmbihtZm5feChtZm4pLApAQCAtNjQ4LDYgKzY2MCw3IEBAIHAy
bV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAybSwKICAgICAg
ICAgICAgICAgICBvbGRfbWZuID0gbDFlX2dldF9wZm4oKnAybV9lbnRyeSk7
CiAgICAgICAgICAgICAgICAgaW9tbXVfb2xkX2ZsYWdzID0KICAgICAgICAg
ICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9f
dHlwZShmbGFncyksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZu
KSk7CiAgICAgICAgICAgICB9CiAgICAgICAgICAgICBlbHNlCi0tLSBhL3hl
bi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNt
LXg4Ni9wMm0uaApAQCAtODM5LDcgKzgzOSw4IEBAIGludCBwMm1fYWx0cDJt
X3Byb3BhZ2F0ZV9jaGFuZ2Uoc3RydWN0IGQKIC8qCiAgKiBwMm0gdHlwZSB0
byBJT01NVSBmbGFncwogICovCi1zdGF0aWMgaW5saW5lIHVuc2lnbmVkIGlu
dCBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90eXBlX3QgcDJtdCwgbWZuX3Qg
bWZuKQorc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJtX2dldF9pb21t
dV9mbGFncyhwMm1fdHlwZV90IHAybXQsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9hY2Nlc3NfdCBwMm1h
LCBtZm5fdCBtZm4pCiB7CiAgICAgdW5zaWduZWQgaW50IGZsYWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC0x
MTU3LDcgKzExNTcsOCBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShz
dHJ1Y3QgZG9tYWluCiAgICAgewogICAgICAgICBpZiAoICFuZWVkX2lvbW11
KGQpICkKICAgICAgICAgICAgIHJldHVybiAwOwotICAgICAgICByZXR1cm4g
aW9tbXVfbWFwX3BhZ2UoZCwgZ2ZuX2wsIGdmbl9sLCBJT01NVUZfcmVhZGFi
bGV8SU9NTVVGX3dyaXRhYmxlKTsKKyAgICAgICAgcmV0dXJuIGlvbW11X21h
cF9wYWdlKGQsIGdmbl9sLCBnZm5fbCwKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHAybV9hY2Nlc3NfdG9faW9tbXVfZmxhZ3MocDJtYSkpOwog
ICAgIH0KIAogICAgIGdmbl9sb2NrKHAybSwgZ2ZuLCAwKTsKLS0tIGEveGVu
L2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRkL2lvbW11LmMKKysrIGIveGVuL2Ry
aXZlcnMvcGFzc3Rocm91Z2gvdnRkL2lvbW11LmMKQEAgLTQyLDEyICs0Miw2
IEBACiAjaW5jbHVkZSAidnRkLmgiCiAjaW5jbHVkZSAiLi4vYXRzLmgiCiAK
LXN0cnVjdCBtYXBwZWRfcm1yciB7Ci0gICAgc3RydWN0IGxpc3RfaGVhZCBs
aXN0OwotICAgIHU2NCBiYXNlLCBlbmQ7Ci0gICAgdW5zaWduZWQgaW50IGNv
dW50OwotfTsKLQogLyogUG9zc2libGUgdW5maWx0ZXJlZCBMQVBJQy9NU0kg
bWVzc2FnZXMgZnJvbSB1bnRydXN0ZWQgc291cmNlcz8gKi8KIGJvb2wgX19y
ZWFkX21vc3RseSB1bnRydXN0ZWRfbXNpOwogCkBAIC0xNzg1LDE2ICsxNzc5
LDExIEBAIG91dDoKIHN0YXRpYyB2b2lkIGlvbW11X2RvbWFpbl90ZWFyZG93
bihzdHJ1Y3QgZG9tYWluICpkKQogewogICAgIHN0cnVjdCBkb21haW5faW9t
bXUgKmhkID0gZG9tX2lvbW11KGQpOwotICAgIHN0cnVjdCBtYXBwZWRfcm1y
ciAqbXJtcnIsICp0bXA7CiAKICAgICBpZiAoIGxpc3RfZW1wdHkoJmFjcGlf
ZHJoZF91bml0cykgKQogICAgICAgICByZXR1cm47CiAKLSAgICBsaXN0X2Zv
cl9lYWNoX2VudHJ5X3NhZmUgKCBtcm1yciwgdG1wLCAmaGQtPmFyY2gubWFw
cGVkX3JtcnJzLCBsaXN0ICkKLSAgICB7Ci0gICAgICAgIGxpc3RfZGVsKCZt
cm1yci0+bGlzdCk7Ci0gICAgICAgIHhmcmVlKG1ybXJyKTsKLSAgICB9Cisg
ICAgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKGQpOwogCiAgICAgaWYg
KCBpb21tdV91c2VfaGFwX3B0KGQpICkKICAgICAgICAgcmV0dXJuOwpAQCAt
MTkwMyw3NCArMTg5Miw2IEBAIHN0YXRpYyB2b2lkIGlvbW11X3NldF9wZ2Qo
c3RydWN0IGRvbWFpbgogICAgICAgICBwYWdldGFibGVfZ2V0X3BhZGRyKHBh
Z2V0YWJsZV9mcm9tX21mbihwZ2RfbWZuKSk7CiB9CiAKLXN0YXRpYyBpbnQg
cm1ycl9pZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsIGJvb2xf
dCBtYXAsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25z
dCBzdHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnIsCi0gICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB1MzIgZmxhZykKLXsKLSAgICB1bnNpZ25l
ZCBsb25nIGJhc2VfcGZuID0gcm1yci0+YmFzZV9hZGRyZXNzID4+IFBBR0Vf
U0hJRlRfNEs7Ci0gICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuID0gUEFHRV9B
TElHTl80SyhybXJyLT5lbmRfYWRkcmVzcykgPj4gUEFHRV9TSElGVF80SzsK
LSAgICBzdHJ1Y3QgbWFwcGVkX3JtcnIgKm1ybXJyOwotICAgIHN0cnVjdCBk
b21haW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOwotCi0gICAgQVNTRVJU
KHBjaWRldnNfbG9ja2VkKCkpOwotICAgIEFTU0VSVChybXJyLT5iYXNlX2Fk
ZHJlc3MgPCBybXJyLT5lbmRfYWRkcmVzcyk7Ci0KLSAgICAvKgotICAgICAq
IE5vIG5lZWQgdG8gYWNxdWlyZSBoZC0+YXJjaC5tYXBwaW5nX2xvY2s6IEJv
dGggaW5zZXJ0aW9uIGFuZCByZW1vdmFsCi0gICAgICogZ2V0IGRvbmUgd2hp
bGUgaG9sZGluZyBwY2lkZXZzX2xvY2suCi0gICAgICovCi0gICAgbGlzdF9m
b3JfZWFjaF9lbnRyeSggbXJtcnIsICZoZC0+YXJjaC5tYXBwZWRfcm1ycnMs
IGxpc3QgKQotICAgIHsKLSAgICAgICAgaWYgKCBtcm1yci0+YmFzZSA9PSBy
bXJyLT5iYXNlX2FkZHJlc3MgJiYKLSAgICAgICAgICAgICBtcm1yci0+ZW5k
ID09IHJtcnItPmVuZF9hZGRyZXNzICkKLSAgICAgICAgewotICAgICAgICAg
ICAgaW50IHJldCA9IDA7Ci0KLSAgICAgICAgICAgIGlmICggbWFwICkKLSAg
ICAgICAgICAgIHsKLSAgICAgICAgICAgICAgICArK21ybXJyLT5jb3VudDsK
LSAgICAgICAgICAgICAgICByZXR1cm4gMDsKLSAgICAgICAgICAgIH0KLQot
ICAgICAgICAgICAgaWYgKCAtLW1ybXJyLT5jb3VudCApCi0gICAgICAgICAg
ICAgICAgcmV0dXJuIDA7Ci0KLSAgICAgICAgICAgIHdoaWxlICggYmFzZV9w
Zm4gPCBlbmRfcGZuICkKLSAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAg
ICBpZiAoIGNsZWFyX2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3Bmbikg
KQotICAgICAgICAgICAgICAgICAgICByZXQgPSAtRU5YSU87Ci0gICAgICAg
ICAgICAgICAgYmFzZV9wZm4rKzsKLSAgICAgICAgICAgIH0KLQotICAgICAg
ICAgICAgbGlzdF9kZWwoJm1ybXJyLT5saXN0KTsKLSAgICAgICAgICAgIHhm
cmVlKG1ybXJyKTsKLSAgICAgICAgICAgIHJldHVybiByZXQ7Ci0gICAgICAg
IH0KLSAgICB9Ci0KLSAgICBpZiAoICFtYXAgKQotICAgICAgICByZXR1cm4g
LUVOT0VOVDsKLQotICAgIHdoaWxlICggYmFzZV9wZm4gPCBlbmRfcGZuICkK
LSAgICB7Ci0gICAgICAgIGludCBlcnIgPSBzZXRfaWRlbnRpdHlfcDJtX2Vu
dHJ5KGQsIGJhc2VfcGZuLCBwMm1fYWNjZXNzX3J3LCBmbGFnKTsKLQotICAg
ICAgICBpZiAoIGVyciApCi0gICAgICAgICAgICByZXR1cm4gZXJyOwotICAg
ICAgICBiYXNlX3BmbisrOwotICAgIH0KLQotICAgIG1ybXJyID0geG1hbGxv
YyhzdHJ1Y3QgbWFwcGVkX3JtcnIpOwotICAgIGlmICggIW1ybXJyICkKLSAg
ICAgICAgcmV0dXJuIC1FTk9NRU07Ci0gICAgbXJtcnItPmJhc2UgPSBybXJy
LT5iYXNlX2FkZHJlc3M7Ci0gICAgbXJtcnItPmVuZCA9IHJtcnItPmVuZF9h
ZGRyZXNzOwotICAgIG1ybXJyLT5jb3VudCA9IDE7Ci0gICAgbGlzdF9hZGRf
dGFpbCgmbXJtcnItPmxpc3QsICZoZC0+YXJjaC5tYXBwZWRfcm1ycnMpOwot
Ci0gICAgcmV0dXJuIDA7Ci19Ci0KIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVf
YWRkX2RldmljZSh1OCBkZXZmbiwgc3RydWN0IHBjaV9kZXYgKnBkZXYpCiB7
CiAgICAgc3RydWN0IGFjcGlfcm1ycl91bml0ICpybXJyOwpAQCAtMjAwMiw3
ICsxOTIzLDkgQEAgc3RhdGljIGludCBpbnRlbF9pb21tdV9hZGRfZGV2aWNl
KHU4IGRldgogICAgICAgICAgICAgICogU2luY2UgUk1SUnMgYXJlIGFsd2F5
cyByZXNlcnZlZCBpbiB0aGUgZTgyMCBtYXAgZm9yIHRoZSBoYXJkd2FyZQog
ICAgICAgICAgICAgICogZG9tYWluLCB0aGVyZSBzaG91bGRuJ3QgYmUgYSBj
b25mbGljdC4KICAgICAgICAgICAgICAqLwotICAgICAgICAgICAgcmV0ID0g
cm1ycl9pZGVudGl0eV9tYXBwaW5nKHBkZXYtPmRvbWFpbiwgMSwgcm1yciwg
MCk7CisgICAgICAgICAgICByZXQgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5n
KHBkZXYtPmRvbWFpbiwgcDJtX2FjY2Vzc19ydywKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcm1yci0+YmFzZV9hZGRyZXNz
LCBybXJyLT5lbmRfYWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgMCk7CiAgICAgICAgICAgICBpZiAoIHJldCAp
CiAgICAgICAgICAgICAgICAgZHByaW50ayhYRU5MT0dfRVJSIFZURFBSRUZJ
WCwgImQlZDogUk1SUiBtYXBwaW5nIGZhaWxlZFxuIiwKICAgICAgICAgICAg
ICAgICAgICAgICAgIHBkZXYtPmRvbWFpbi0+ZG9tYWluX2lkKTsKQEAgLTIw
NDcsNyArMTk3MCw4IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfcmVtb3Zl
X2RldmljZSh1OAogICAgICAgICAgKiBBbnkgZmxhZyBpcyBub3RoaW5nIHRv
IGNsZWFyIHRoZXNlIG1hcHBpbmdzIGJ1dCBoZXJlCiAgICAgICAgICAqIGl0
cyBhbHdheXMgc2FmZSBhbmQgc3RyaWN0IHRvIHNldCAwLgogICAgICAgICAg
Ki8KLSAgICAgICAgcm1ycl9pZGVudGl0eV9tYXBwaW5nKHBkZXYtPmRvbWFp
biwgMCwgcm1yciwgMCk7CisgICAgICAgIGlvbW11X2lkZW50aXR5X21hcHBp
bmcocGRldi0+ZG9tYWluLCBwMm1fYWNjZXNzX3gsIHJtcnItPmJhc2VfYWRk
cmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5l
bmRfYWRkcmVzcywgMCk7CiAgICAgfQogCiAgICAgcmV0dXJuIGRvbWFpbl9j
b250ZXh0X3VubWFwKHBkZXYtPmRvbWFpbiwgZGV2Zm4sIHBkZXYpOwpAQCAt
MjIxNCw3ICsyMTM4LDggQEAgc3RhdGljIHZvaWQgX19od2RvbV9pbml0IHNl
dHVwX2h3ZG9tX3JtcgogICAgICAgICAgKiBkb21haW4sIHRoZXJlIHNob3Vs
ZG4ndCBiZSBhIGNvbmZsaWN0LiBTbyBpdHMgYWx3YXlzIHNhZmUgYW5kCiAg
ICAgICAgICAqIHN0cmljdCB0byBzZXQgMC4KICAgICAgICAgICovCi0gICAg
ICAgIHJldCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhkLCAxLCBybXJyLCAw
KTsKKyAgICAgICAgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhkLCBw
Mm1fYWNjZXNzX3J3LCBybXJyLT5iYXNlX2FkZHJlc3MsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcm1yci0+ZW5kX2FkZHJlc3Ms
IDApOwogICAgICAgICBpZiAoIHJldCApCiAgICAgICAgICAgICBkcHJpbnRr
KFhFTkxPR19FUlIgVlREUFJFRklYLAogICAgICAgICAgICAgICAgICAgICAg
IklPTU1VOiBtYXBwaW5nIHJlc2VydmVkIHJlZ2lvbiBmYWlsZWRcbiIpOwpA
QCAtMjM3MSw3ICsyMjk2LDkgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZp
Y2Vfb3duZXJzaGlwKAogICAgICAgICAgICAgICAgICAqIEFueSBSTVJSIGZs
YWcgaXMgYWx3YXlzIGlnbm9yZWQgd2hlbiByZW1vdmUgYSBkZXZpY2UsCiAg
ICAgICAgICAgICAgICAgICogYnV0IGl0cyBhbHdheXMgc2FmZSBhbmQgc3Ry
aWN0IHRvIHNldCAwLgogICAgICAgICAgICAgICAgICAqLwotICAgICAgICAg
ICAgICAgIHJldCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhzb3VyY2UsIDAs
IHJtcnIsIDApOworICAgICAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50
aXR5X21hcHBpbmcoc291cmNlLCBwMm1fYWNjZXNzX3gsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5iYXNl
X2FkZHJlc3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBybXJyLT5lbmRfYWRkcmVzcywgMCk7CiAgICAgICAgICAg
ICAgICAgaWYgKCByZXQgIT0gLUVOT0VOVCApCiAgICAgICAgICAgICAgICAg
ICAgIHJldHVybiByZXQ7CiAgICAgICAgICAgICB9CkBAIC0yNDY4LDcgKzIz
OTUsOCBAQCBzdGF0aWMgaW50IGludGVsX2lvbW11X2Fzc2lnbl9kZXZpY2Uo
CiAgICAgICAgICAgICAgUENJX0JVUyhiZGYpID09IGJ1cyAmJgogICAgICAg
ICAgICAgIFBDSV9ERVZGTjIoYmRmKSA9PSBkZXZmbiApCiAgICAgICAgIHsK
LSAgICAgICAgICAgIHJldCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhkLCAx
LCBybXJyLCBmbGFnKTsKKyAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50
aXR5X21hcHBpbmcoZCwgcDJtX2FjY2Vzc19ydywgcm1yci0+YmFzZV9hZGRy
ZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBybXJyLT5lbmRfYWRkcmVzcywgZmxhZyk7CiAgICAgICAgICAgICBpZiAo
IHJldCApCiAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgICAgaW50IHJj
OwotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC94ODYvaW9tbXUuYwor
KysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC94ODYvaW9tbXUuYwpAQCAt
MTQ0LDcgKzE0NCw3IEBAIGludCBhcmNoX2lvbW11X2RvbWFpbl9pbml0KHN0
cnVjdCBkb21haW4KICAgICBzdHJ1Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRv
bV9pb21tdShkKTsKIAogICAgIHNwaW5fbG9ja19pbml0KCZoZC0+YXJjaC5t
YXBwaW5nX2xvY2spOwotICAgIElOSVRfTElTVF9IRUFEKCZoZC0+YXJjaC5t
YXBwZWRfcm1ycnMpOworICAgIElOSVRfTElTVF9IRUFEKCZoZC0+YXJjaC5p
ZGVudGl0eV9tYXBzKTsKIAogICAgIHJldHVybiAwOwogfQpAQCAtMTUzLDYg
KzE1Myw5OSBAQCB2b2lkIGFyY2hfaW9tbXVfZG9tYWluX2Rlc3Ryb3koc3Ry
dWN0IGRvCiB7CiB9CiAKK3N0cnVjdCBpZGVudGl0eV9tYXAgeworICAgIHN0
cnVjdCBsaXN0X2hlYWQgbGlzdDsKKyAgICBwYWRkcl90IGJhc2UsIGVuZDsK
KyAgICBwMm1fYWNjZXNzX3QgYWNjZXNzOworICAgIHVuc2lnbmVkIGludCBj
b3VudDsKK307CisKK2ludCBpb21tdV9pZGVudGl0eV9tYXBwaW5nKHN0cnVj
dCBkb21haW4gKmQsIHAybV9hY2Nlc3NfdCBwMm1hLAorICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGVuZCwKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKQor
eworICAgIHVuc2lnbmVkIGxvbmcgYmFzZV9wZm4gPSBiYXNlID4+IFBBR0Vf
U0hJRlRfNEs7CisgICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuID0gUEFHRV9B
TElHTl80SyhlbmQpID4+IFBBR0VfU0hJRlRfNEs7CisgICAgc3RydWN0IGlk
ZW50aXR5X21hcCAqbWFwOworICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhk
ID0gZG9tX2lvbW11KGQpOworCisgICAgQVNTRVJUKHBjaWRldnNfbG9ja2Vk
KCkpOworICAgIEFTU0VSVChiYXNlIDwgZW5kKTsKKworICAgIC8qCisgICAg
ICogTm8gbmVlZCB0byBhY3F1aXJlIGhkLT5hcmNoLm1hcHBpbmdfbG9jazog
Qm90aCBpbnNlcnRpb24gYW5kIHJlbW92YWwKKyAgICAgKiBnZXQgZG9uZSB3
aGlsZSBob2xkaW5nIHBjaWRldnNfbG9jay4KKyAgICAgKi8KKyAgICBsaXN0
X2Zvcl9lYWNoX2VudHJ5KCBtYXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBz
LCBsaXN0ICkKKyAgICB7CisgICAgICAgIGlmICggbWFwLT5iYXNlID09IGJh
c2UgJiYgbWFwLT5lbmQgPT0gZW5kICkKKyAgICAgICAgeworICAgICAgICAg
ICAgaW50IHJldCA9IDA7CisKKyAgICAgICAgICAgIGlmICggcDJtYSAhPSBw
Mm1fYWNjZXNzX3ggKQorICAgICAgICAgICAgeworICAgICAgICAgICAgICAg
IGlmICggbWFwLT5hY2Nlc3MgIT0gcDJtYSApCisgICAgICAgICAgICAgICAg
ICAgIHJldHVybiAtRUFERFJJTlVTRTsKKyAgICAgICAgICAgICAgICArK21h
cC0+Y291bnQ7CisgICAgICAgICAgICAgICAgcmV0dXJuIDA7CisgICAgICAg
ICAgICB9CisKKyAgICAgICAgICAgIGlmICggLS1tYXAtPmNvdW50ICkKKyAg
ICAgICAgICAgICAgICByZXR1cm4gMDsKKworICAgICAgICAgICAgd2hpbGUg
KCBiYXNlX3BmbiA8IGVuZF9wZm4gKQorICAgICAgICAgICAgeworICAgICAg
ICAgICAgICAgIGlmICggY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KGQsIGJh
c2VfcGZuKSApCisgICAgICAgICAgICAgICAgICAgIHJldCA9IC1FTlhJTzsK
KyAgICAgICAgICAgICAgICBiYXNlX3BmbisrOworICAgICAgICAgICAgfQor
CisgICAgICAgICAgICBsaXN0X2RlbCgmbWFwLT5saXN0KTsKKyAgICAgICAg
ICAgIHhmcmVlKG1hcCk7CisKKyAgICAgICAgICAgIHJldHVybiByZXQ7Cisg
ICAgICAgIH0KKworICAgICAgICBpZiAoIGVuZCA+PSBtYXAtPmJhc2UgJiYg
bWFwLT5lbmQgPj0gYmFzZSApCisgICAgICAgICAgICByZXR1cm4gLUVBRERS
SU5VU0U7CisgICAgfQorCisgICAgaWYgKCBwMm1hID09IHAybV9hY2Nlc3Nf
eCApCisgICAgICAgIHJldHVybiAtRU5PRU5UOworCisgICAgd2hpbGUgKCBi
YXNlX3BmbiA8IGVuZF9wZm4gKQorICAgIHsKKyAgICAgICAgaW50IGVyciA9
IHNldF9pZGVudGl0eV9wMm1fZW50cnkoZCwgYmFzZV9wZm4sIHAybWEsIGZs
YWcpOworCisgICAgICAgIGlmICggZXJyICkKKyAgICAgICAgICAgIHJldHVy
biBlcnI7CisgICAgICAgIGJhc2VfcGZuKys7CisgICAgfQorCisgICAgbWFw
ID0geG1hbGxvYyhzdHJ1Y3QgaWRlbnRpdHlfbWFwKTsKKyAgICBpZiAoICFt
YXAgKQorICAgICAgICByZXR1cm4gLUVOT01FTTsKKyAgICBtYXAtPmJhc2Ug
PSBiYXNlOworICAgIG1hcC0+ZW5kID0gZW5kOworICAgIG1hcC0+YWNjZXNz
ID0gcDJtYTsKKyAgICBtYXAtPmNvdW50ID0gMTsKKyAgICBsaXN0X2FkZF90
YWlsKCZtYXAtPmxpc3QsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzKTsKKwor
ICAgIHJldHVybiAwOworfQorCit2b2lkIGlvbW11X2lkZW50aXR5X21hcF90
ZWFyZG93bihzdHJ1Y3QgZG9tYWluICpkKQoreworICAgIHN0cnVjdCBkb21h
aW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOworICAgIHN0cnVjdCBpZGVu
dGl0eV9tYXAgKm1hcCwgKnRtcDsKKworICAgIGxpc3RfZm9yX2VhY2hfZW50
cnlfc2FmZSAoIG1hcCwgdG1wLCAmaGQtPmFyY2guaWRlbnRpdHlfbWFwcywg
bGlzdCApCisgICAgeworICAgICAgICBsaXN0X2RlbCgmbWFwLT5saXN0KTsK
KyAgICAgICAgeGZyZWUobWFwKTsKKyAgICB9Cit9CisKIC8qCiAgKiBMb2Nh
bCB2YXJpYWJsZXM6CiAgKiBtb2RlOiBDCi0tLSBhL3hlbi9pbmNsdWRlL2Fz
bS14ODYvaW9tbXUuaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L2lvbW11
LmgKQEAgLTE2LDYgKzE2LDcgQEAKIAogI2luY2x1ZGUgPHhlbi9lcnJuby5o
PgogI2luY2x1ZGUgPHhlbi9saXN0Lmg+CisjaW5jbHVkZSA8eGVuL21lbV9h
Y2Nlc3MuaD4KICNpbmNsdWRlIDx4ZW4vc3BpbmxvY2suaD4KICNpbmNsdWRl
IDxhc20vcHJvY2Vzc29yLmg+CiAjaW5jbHVkZSA8YXNtL2h2bS92bXgvdm1j
cy5oPgpAQCAtMzYsNyArMzcsNyBAQCBzdHJ1Y3QgYXJjaF9pb21tdQogICAg
IHNwaW5sb2NrX3QgbWFwcGluZ19sb2NrOyAgICAgICAgICAgIC8qIGlvIHBh
Z2UgdGFibGUgbG9jayAqLwogICAgIGludCBhZ2F3OyAgICAgLyogYWRqdXN0
ZWQgZ3Vlc3QgYWRkcmVzcyB3aWR0aCwgMCBpcyBsZXZlbCAyIDMwLWJpdCAq
LwogICAgIHU2NCBpb21tdV9iaXRtYXA7ICAgICAgICAgICAgICAvKiBiaXRt
YXAgb2YgaW9tbXUocykgdGhhdCB0aGUgZG9tYWluIHVzZXMgKi8KLSAgICBz
dHJ1Y3QgbGlzdF9oZWFkIG1hcHBlZF9ybXJyczsKKyAgICBzdHJ1Y3QgbGlz
dF9oZWFkIGlkZW50aXR5X21hcHM7CiAKICAgICAvKiBhbWQgaW9tbXUgc3Vw
cG9ydCAqLwogICAgIGludCBwYWdpbmdfbW9kZTsKQEAgLTk0LDYgKzk1LDEx
IEBAIGJvb2xfdCBpb21tdV9zdXBwb3J0c19laW0odm9pZCk7CiBpbnQgaW9t
bXVfZW5hYmxlX3gyYXBpY19JUih2b2lkKTsKIHZvaWQgaW9tbXVfZGlzYWJs
ZV94MmFwaWNfSVIodm9pZCk7CiAKK2ludCBpb21tdV9pZGVudGl0eV9tYXBw
aW5nKHN0cnVjdCBkb21haW4gKmQsIHAybV9hY2Nlc3NfdCBwMm1hLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGVuZCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBmbGFnKTsKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKHN0
cnVjdCBkb21haW4gKmQpOworCiBleHRlcm4gYm9vbCB1bnRydXN0ZWRfbXNp
OwogCiBpbnQgcGlfdXBkYXRlX2lydGUoY29uc3Qgc3RydWN0IHBpX2Rlc2Mg
KnBpX2Rlc2MsIGNvbnN0IHN0cnVjdCBwaXJxICpwaXJxLAotLS0gYS94ZW4v
aW5jbHVkZS9hc20teDg2L21lbV9hY2Nlc3MuaAorKysgYi94ZW4vaW5jbHVk
ZS9hc20teDg2L21lbV9hY2Nlc3MuaApAQCAtNDQsMTAgKzQ0LDggQEAgYm9v
bCBwMm1fbWVtX2FjY2Vzc19lbXVsYXRlX2NoZWNrKHN0cnVjdAogICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHZtX2V2ZW50X3Jl
c3BvbnNlX3QgKnJzcCk7CiAKIC8qIFNhbml0eSBjaGVjayBmb3IgbWVtX2Fj
Y2VzcyBoYXJkd2FyZSBzdXBwb3J0ICovCi1zdGF0aWMgaW5saW5lIGJvb2wg
cDJtX21lbV9hY2Nlc3Nfc2FuaXR5X2NoZWNrKHN0cnVjdCBkb21haW4gKmQp
Ci17Ci0gICAgcmV0dXJuIGlzX2h2bV9kb21haW4oZCkgJiYgY3B1X2hhc192
bXggJiYgaGFwX2VuYWJsZWQoZCk7Ci19CisjZGVmaW5lIHAybV9tZW1fYWNj
ZXNzX3Nhbml0eV9jaGVjayhkKSBcCisgICAgKGlzX2h2bV9kb21haW4oZCkg
JiYgY3B1X2hhc192bXggJiYgaGFwX2VuYWJsZWQoZCkpCiAKICNlbmRpZiAv
Kl9fQVNNX1g4Nl9NRU1fQUNDRVNTX0hfXyAqLwogCi0tLSBhL3hlbi9pbmNs
dWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4Ni9w
Mm0uaApAQCAtODM2LDYgKzgzNiwzNCBAQCBpbnQgcDJtX2FsdHAybV9wcm9w
YWdhdGVfY2hhbmdlKHN0cnVjdCBkCiAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIG1mbl90IG1mbiwgdW5zaWduZWQgaW50IHBhZ2Vfb3JkZXIs
CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV90eXBlX3Qg
cDJtdCwgcDJtX2FjY2Vzc190IHAybWEpOwogCisvKiBwMm0gYWNjZXNzIHRv
IElPTU1VIGZsYWdzICovCitzdGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBw
Mm1fYWNjZXNzX3RvX2lvbW11X2ZsYWdzKHAybV9hY2Nlc3NfdCBwMm1hKQor
eworICAgIHN3aXRjaCAoIHAybWEgKQorICAgIHsKKyAgICBjYXNlIHAybV9h
Y2Nlc3Nfcnc6CisgICAgY2FzZSBwMm1fYWNjZXNzX3J3eDoKKyAgICAgICAg
cmV0dXJuIElPTU1VRl9yZWFkYWJsZSB8IElPTU1VRl93cml0YWJsZTsKKwor
ICAgIGNhc2UgcDJtX2FjY2Vzc19yOgorICAgIGNhc2UgcDJtX2FjY2Vzc19y
eDoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcngycnc6CisgICAgICAgIHJldHVy
biBJT01NVUZfcmVhZGFibGU7CisKKyAgICBjYXNlIHAybV9hY2Nlc3NfdzoK
KyAgICBjYXNlIHAybV9hY2Nlc3Nfd3g6CisgICAgICAgIHJldHVybiBJT01N
VUZfd3JpdGFibGU7CisKKyAgICBjYXNlIHAybV9hY2Nlc3NfbjoKKyAgICBj
YXNlIHAybV9hY2Nlc3NfeDoKKyAgICBjYXNlIHAybV9hY2Nlc3NfbjJyd3g6
CisgICAgICAgIHJldHVybiAwOworICAgIH0KKworICAgIEFTU0VSVF9VTlJF
QUNIQUJMRSgpOworICAgIHJldHVybiAwOworfQorCiAvKgogICogcDJtIHR5
cGUgdG8gSU9NTVUgZmxhZ3MKICAqLwpAQCAtODU3LDkgKzg4NSwxMCBAQCBz
dGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBwMm1fZ2V0X2lvbW11CiAgICAg
ICAgIGZsYWdzID0gSU9NTVVGX3JlYWRhYmxlOwogICAgICAgICBicmVhazsK
ICAgICBjYXNlIHAybV9tbWlvX2RpcmVjdDoKLSAgICAgICAgZmxhZ3MgPSBJ
T01NVUZfcmVhZGFibGU7Ci0gICAgICAgIGlmICggIXJhbmdlc2V0X2NvbnRh
aW5zX3NpbmdsZXRvbihtbWlvX3JvX3JhbmdlcywgbWZuX3gobWZuKSkgKQot
ICAgICAgICAgICAgZmxhZ3MgfD0gSU9NTVVGX3dyaXRhYmxlOworICAgICAg
ICBmbGFncyA9IHAybV9hY2Nlc3NfdG9faW9tbXVfZmxhZ3MocDJtYSk7Cisg
ICAgICAgIGlmICggKGZsYWdzICYgSU9NTVVGX3dyaXRhYmxlKSAmJgorICAg
ICAgICAgICAgIHJhbmdlc2V0X2NvbnRhaW5zX3NpbmdsZXRvbihtbWlvX3Jv
X3JhbmdlcywgbWZuX3gobWZuKSkgKQorICAgICAgICAgICAgZmxhZ3MgJj0g
fklPTU1VRl93cml0YWJsZTsKICAgICAgICAgYnJlYWs7CiAgICAgZGVmYXVs
dDoKICAgICAgICAgZmxhZ3MgPSAwOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfbWFwLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lv
bW11X21hcC5jCkBAIC03MTYsMjcgKzcxNiw0OSBAQCBpbnQgYW1kX2lvbW11
X3VubWFwX3BhZ2Uoc3RydWN0IGRvbWFpbiAqCiAgICAgcmV0dXJuIDA7CiB9
CiAKLWludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKHN0
cnVjdCBkb21haW4gKmRvbWFpbiwKLSAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHU2NCBwaHlzX2FkZHIsCi0gICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIHNpemUs
IGludCBpdywgaW50IGlyKQoraW50IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFp
bl91bml0eV9tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBpdnJzX3Vu
aXR5X21hcCAqbWFwLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IGZsYWcpCiB7Ci0gICAgdW5zaWduZWQg
bG9uZyBucGFnZXMsIGk7Ci0gICAgdW5zaWduZWQgbG9uZyBnZm47Ci0gICAg
dW5zaWduZWQgaW50IGZsYWdzID0gISFpcjsKLSAgICBpbnQgcnQgPSAwOwot
Ci0gICAgaWYgKCBpdyApCi0gICAgICAgIGZsYWdzIHw9IElPTU1VRl93cml0
YWJsZTsKLQotICAgIG5wYWdlcyA9IHJlZ2lvbl90b19wYWdlcyhwaHlzX2Fk
ZHIsIHNpemUpOwotICAgIGdmbiA9IHBoeXNfYWRkciA+PiBQQUdFX1NISUZU
OwotICAgIGZvciAoIGkgPSAwOyBpIDwgbnBhZ2VzOyBpKysgKQorICAgIGlu
dCByYzsKKworICAgIGlmICggZCA9PSBkb21faW8gKQorICAgICAgICByZXR1
cm4gMDsKKworICAgIGZvciAoIHJjID0gMDsgIXJjICYmIG1hcDsgbWFwID0g
bWFwLT5uZXh0ICkKICAgICB7Ci0gICAgICAgIHJ0ID0gYW1kX2lvbW11X21h
cF9wYWdlKGRvbWFpbiwgZ2ZuICtpLCBnZm4gK2ksIGZsYWdzKTsKLSAgICAg
ICAgaWYgKCBydCAhPSAwICkKLSAgICAgICAgICAgIHJldHVybiBydDsKKyAg
ICAgICAgcDJtX2FjY2Vzc190IHAybWEgPSBwMm1fYWNjZXNzX247CisKKyAg
ICAgICAgaWYgKCBtYXAtPnJlYWQgKQorICAgICAgICAgICAgcDJtYSB8PSBw
Mm1fYWNjZXNzX3I7CisgICAgICAgIGlmICggbWFwLT53cml0ZSApCisgICAg
ICAgICAgICBwMm1hIHw9IHAybV9hY2Nlc3NfdzsKKworICAgICAgICByYyA9
IGlvbW11X2lkZW50aXR5X21hcHBpbmcoZCwgcDJtYSwgbWFwLT5hZGRyLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRy
ICsgbWFwLT5sZW5ndGggLSAxLCBmbGFnKTsKICAgICB9Ci0gICAgcmV0dXJu
IDA7CisKKyAgICByZXR1cm4gcmM7Cit9CisKK2ludCBhbWRfaW9tbXVfcmVz
ZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXApCit7CisgICAgaW50IHJjOwor
CisgICAgaWYgKCBkID09IGRvbV9pbyApCisgICAgICAgIHJldHVybiAwOwor
CisgICAgZm9yICggcmMgPSAwOyBtYXA7IG1hcCA9IG1hcC0+bmV4dCApCisg
ICAgeworICAgICAgICBpbnQgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGlu
ZyhkLCBwMm1fYWNjZXNzX3gsIG1hcC0+YWRkciwKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRyICsgbWFwLT5s
ZW5ndGggLSAxLCAwKTsKKworICAgICAgICBpZiAoIHJldCAmJiByZXQgIT0g
LUVOT0VOVCAmJiAhcmMgKQorICAgICAgICAgICAgcmMgPSByZXQ7CisgICAg
fQorCisgICAgcmV0dXJuIHJjOwogfQogCiAvKiBTaGFyZSBwMm0gdGFibGUg
d2l0aCBpb21tdS4gKi8KLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gv
YW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhy
b3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0zMzMsNiArMzMzLDcgQEAg
c3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbgogICAg
IHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwogICAgIGludCBiZGYsIHJjOwog
ICAgIHN0cnVjdCBkb21haW5faW9tbXUgKnQgPSBkb21faW9tbXUodGFyZ2V0
KTsKKyAgICBjb25zdCBzdHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBw
aW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHBkZXYtPnNlZyk7CiAKICAgICBi
ZGYgPSBQQ0lfQkRGMihwZGV2LT5idXMsIHBkZXYtPmRldmZuKTsKICAgICBp
b21tdSA9IGZpbmRfaW9tbXVfZm9yX2RldmljZShwZGV2LT5zZWcsIGJkZik7
CkBAIC0zNDcsMTAgKzM0OCwyNCBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2Rl
dmljZShzdHJ1Y3QgZG9tYWluCiAKICAgICBhbWRfaW9tbXVfZGlzYWJsZV9k
b21haW5fZGV2aWNlKHNvdXJjZSwgaW9tbXUsIGRldmZuLCBwZGV2KTsKIAot
ICAgIGlmICggZGV2Zm4gPT0gcGRldi0+ZGV2Zm4gKQorICAgIC8qCisgICAg
ICogSWYgdGhlIGRldmljZSBiZWxvbmdzIHRvIHRoZSBoYXJkd2FyZSBkb21h
aW4sIGFuZCBpdCBoYXMgYSB1bml0eSBtYXBwaW5nLAorICAgICAqIGRvbid0
IHJlbW92ZSBpdCBmcm9tIHRoZSBoYXJkd2FyZSBkb21haW4sIGJlY2F1c2Ug
QklPUyBtYXkgcmVmZXJlbmNlIHRoYXQKKyAgICAgKiBtYXBwaW5nLgorICAg
ICAqLworICAgIGlmICggIWlzX2hhcmR3YXJlX2RvbWFpbihzb3VyY2UpICkK
KyAgICB7CisgICAgICAgIHJjID0gYW1kX2lvbW11X3Jlc2VydmVfZG9tYWlu
X3VuaXR5X3VubWFwKAorICAgICAgICAgICAgICAgICBzb3VyY2UsCisgICAg
ICAgICAgICAgICAgIGl2cnNfbWFwcGluZ3NbZ2V0X2RtYV9yZXF1ZXN0b3Jf
aWQocGRldi0+c2VnLCBiZGYpXS51bml0eV9tYXApOworICAgICAgICBpZiAo
IHJjICkKKyAgICAgICAgICAgIHJldHVybiByYzsKKyAgICB9CisKKyAgICBp
ZiAoIGRldmZuID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRvbWFpbiAhPSBk
b21faW8gKQogICAgIHsKLSAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21h
aW5fbGlzdCwgJnRhcmdldC0+YXJjaC5wZGV2X2xpc3QpOwotICAgICAgICBw
ZGV2LT5kb21haW4gPSB0YXJnZXQ7CisgICAgICAgIGxpc3RfbW92ZSgmcGRl
di0+ZG9tYWluX2xpc3QsICZkb21faW8tPmFyY2gucGRldl9saXN0KTsKKyAg
ICAgICAgcGRldi0+ZG9tYWluID0gZG9tX2lvOwogICAgIH0KIAogICAgIHJj
ID0gYWxsb2NhdGVfZG9tYWluX3Jlc291cmNlcyh0KTsKQEAgLTM2Miw2ICsz
NzcsMTIgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRv
bWFpbgogICAgICAgICAgICAgICAgICAgICBwZGV2LT5zZWcsIHBkZXYtPmJ1
cywgUENJX1NMT1QoZGV2Zm4pLCBQQ0lfRlVOQyhkZXZmbiksCiAgICAgICAg
ICAgICAgICAgICAgIHNvdXJjZS0+ZG9tYWluX2lkLCB0YXJnZXQtPmRvbWFp
bl9pZCk7CiAKKyAgICBpZiAoIGRldmZuID09IHBkZXYtPmRldmZuICYmIHBk
ZXYtPmRvbWFpbiAhPSB0YXJnZXQgKQorICAgIHsKKyAgICAgICAgbGlzdF9t
b3ZlKCZwZGV2LT5kb21haW5fbGlzdCwgJnRhcmdldC0+YXJjaC5wZGV2X2xp
c3QpOworICAgICAgICBwZGV2LT5kb21haW4gPSB0YXJnZXQ7CisgICAgfQor
CiAgICAgcmV0dXJuIDA7CiB9CiAKQEAgLTM3MiwyMCArMzkzLDI4IEBAIHN0
YXRpYyBpbnQgYW1kX2lvbW11X2Fzc2lnbl9kZXZpY2Uoc3RydWMKICAgICBz
dHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJz
X21hcHBpbmdzKHBkZXYtPnNlZyk7CiAgICAgaW50IGJkZiA9IFBDSV9CREYy
KHBkZXYtPmJ1cywgZGV2Zm4pOwogICAgIGludCByZXFfaWQgPSBnZXRfZG1h
X3JlcXVlc3Rvcl9pZChwZGV2LT5zZWcsIGJkZik7Ci0gICAgY29uc3Qgc3Ry
dWN0IGl2cnNfdW5pdHlfbWFwICp1bml0eV9tYXA7CisgICAgaW50IHJjID0g
YW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAgICAg
ICAgICAgICAgZCwgaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcCwg
ZmxhZyk7CisKKyAgICBpZiAoICFyYyApCisgICAgICAgIHJjID0gcmVhc3Np
Z25fZGV2aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2Zm4sIHBkZXYpOwogCi0g
ICAgZm9yICggdW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVu
aXR5X21hcDsgdW5pdHlfbWFwOwotICAgICAgICAgIHVuaXR5X21hcCA9IHVu
aXR5X21hcC0+bmV4dCApCisgICAgaWYgKCByYyAmJiAhaXNfaGFyZHdhcmVf
ZG9tYWluKGQpICkKICAgICB7Ci0gICAgICAgIGludCByYyA9IGFtZF9pb21t
dV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoCi0gICAgICAgICAgICAgICAg
ICAgICBkLCB1bml0eV9tYXAtPmFkZHIsIHVuaXR5X21hcC0+bGVuZ3RoLAot
ICAgICAgICAgICAgICAgICAgICAgdW5pdHlfbWFwLT53cml0ZSwgdW5pdHlf
bWFwLT5yZWFkKTsKKyAgICAgICAgaW50IHJldCA9IGFtZF9pb21tdV9yZXNl
cnZlX2RvbWFpbl91bml0eV91bm1hcCgKKyAgICAgICAgICAgICAgICAgICAg
ICBkLCBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwKTsKIAotICAg
ICAgICBpZiAoIHJjICkKLSAgICAgICAgICAgIHJldHVybiByYzsKKyAgICAg
ICAgaWYgKCByZXQgKQorICAgICAgICB7CisgICAgICAgICAgICBwcmludGso
WEVOTE9HX0VSUiAiQU1ELVZpOiAiCisgICAgICAgICAgICAgICAgICAgInVu
aXR5LXVubWFwIGZvciBkJWQvJTA0eDolMDJ4OiUwMnguJXUgZmFpbGVkICgl
ZClcbiIsCisgICAgICAgICAgICAgICAgICAgZC0+ZG9tYWluX2lkLCBwZGV2
LT5zZWcsIHBkZXYtPmJ1cywKKyAgICAgICAgICAgICAgICAgICBQQ0lfU0xP
VChkZXZmbiksIFBDSV9GVU5DKGRldmZuKSwgcmV0KTsKKyAgICAgICAgICAg
IGRvbWFpbl9jcmFzaChkKTsKKyAgICAgICAgfQogICAgIH0KIAotICAgIHJl
dHVybiByZWFzc2lnbl9kZXZpY2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwg
cGRldik7CisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0aWMgdm9pZCBkZWFs
bG9jYXRlX25leHRfcGFnZV90YWJsZShzdHJ1Y3QgcGFnZV9pbmZvICpwZywg
aW50IGxldmVsKQpAQCAtNDUxLDYgKzQ4MCw3IEBAIHN0YXRpYyB2b2lkIGRl
YWxsb2NhdGVfaW9tbXVfcGFnZV90YWJsZXMKIAogc3RhdGljIHZvaWQgYW1k
X2lvbW11X2RvbWFpbl9kZXN0cm95KHN0cnVjdCBkb21haW4gKmQpCiB7Cisg
ICAgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKGQpOwogICAgIGRlYWxs
b2NhdGVfaW9tbXVfcGFnZV90YWJsZXMoZCk7CiAgICAgYW1kX2lvbW11X2Zs
dXNoX2FsbF9wYWdlcyhkKTsKIH0KLS0tIGEveGVuL2luY2x1ZGUvYXNtLXg4
Ni9odm0vc3ZtL2FtZC1pb21tdS1wcm90by5oCisrKyBiL3hlbi9pbmNsdWRl
L2FzbS14ODYvaHZtL3N2bS9hbWQtaW9tbXUtcHJvdG8uaApAQCAtNjAsOCAr
NjAsMTAgQEAgaW50IF9fbXVzdF9jaGVjayBhbWRfaW9tbXVfdW5tYXBfcGFn
ZShzdAogdTY0IGFtZF9pb21tdV9nZXRfbmV4dF90YWJsZV9mcm9tX3B0ZSh1
MzIgKmVudHJ5KTsKIGludCBfX211c3RfY2hlY2sgYW1kX2lvbW11X2FsbG9j
X3Jvb3Qoc3RydWN0IGRvbWFpbl9pb21tdSAqaGQpOwogaW50IGFtZF9pb21t
dV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoc3RydWN0IGRvbWFpbiAqZG9t
YWluLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dTY0IHBoeXNfYWRkciwgdW5zaWduZWQgbG9uZyBzaXplLAotICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50IGl3LCBpbnQgaXIp
OworICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29u
c3Qgc3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXAsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxhZyk7
CitpbnQgYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X3VubWFwKHN0
cnVjdCBkb21haW4gKmQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqbWFw
KTsKIAogLyogU2hhcmUgcDJtIHRhYmxlIHdpdGggaW9tbXUgKi8KIHZvaWQg
YW1kX2lvbW11X3NoYXJlX3AybShzdHJ1Y3QgZG9tYWluICpkKTsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVu
L2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtOTks
MTIgKzk5LDggQEAgc3RhdGljIHN0cnVjdCBhbWRfaW9tbXUgKiBfX2luaXQg
ZmluZF9pbwogfQogCiBzdGF0aWMgaW50IF9faW5pdCByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZSgKLSAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21t
dSwgdWludDY0X3QgYmFzZSwgdWludDY0X3QgbGltaXQsCi0gICAgYm9vbCBh
bGwsIGJvb2wgaXcsIGJvb2wgaXIpCisgICAgc3RydWN0IGFtZF9pb21tdSAq
aW9tbXUsIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwgYm9vbCBhbGwp
CiB7Ci0gICAgaWYgKCAhaXIgfHwgIWl3ICkKLSAgICAgICAgcmV0dXJuIC1F
UEVSTTsKLQogICAgIC8qIG5lZWQgdG8gZXh0ZW5kIGV4Y2x1c2lvbiByYW5n
ZT8gKi8KICAgICBpZiAoIGlvbW11LT5leGNsdXNpb25fZW5hYmxlICkKICAg
ICB7CkBAIC0xMzMsMTQgKzEyOSwxOCBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZXNlcnZlX3VuaXR5X21hcF9mb3JfCiB7CiAgICAgc3RydWN0IGl2cnNfbWFw
cGluZ3MgKml2cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhzZWcp
OwogICAgIHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwID0gaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcDsKKyAgICBpbnQgcGFnaW5nX21v
ZGUgPSBhbWRfaW9tbXVfZ2V0X3BhZ2luZ19tb2RlKFBGTl9VUChiYXNlICsg
bGVuZ3RoKSk7CisKKyAgICBpZiAoIHBhZ2luZ19tb2RlIDwgMCApCisgICAg
ICAgIHJldHVybiBwYWdpbmdfbW9kZTsKIAogICAgIC8qIENoZWNrIGZvciBv
dmVybGFwcy4gKi8KICAgICBmb3IgKCA7IHVuaXR5X21hcDsgdW5pdHlfbWFw
ID0gdW5pdHlfbWFwLT5uZXh0ICkKICAgICB7CiAgICAgICAgIC8qCiAgICAg
ICAgICAqIEV4YWN0IG1hdGNoZXMgYXJlIG9rYXkuIFRoaXMgY2FuIGluIHBh
cnRpY3VsYXIgaGFwcGVuIHdoZW4KLSAgICAgICAgICogcmVnaXN0ZXJfZXhj
bHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3aWNlIGZv
ciB0aGUKLSAgICAgICAgICogc2FtZSAocyxiLGQsZikuCisgICAgICAgICAq
IHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3aWNl
IGZvciB0aGUgc2FtZQorICAgICAgICAgKiAocyxiLGQsZikuCiAgICAgICAg
ICAqLwogICAgICAgICBpZiAoIGJhc2UgPT0gdW5pdHlfbWFwLT5hZGRyICYm
IGxlbmd0aCA9PSB1bml0eV9tYXAtPmxlbmd0aCAmJgogICAgICAgICAgICAg
IGlyID09IHVuaXR5X21hcC0+cmVhZCAmJiBpdyA9PSB1bml0eV9tYXAtPndy
aXRlICkKQEAgLTE2OCw1NSArMTY4LDUyIEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl8KICAgICB1bml0eV9tYXAtPm5leHQg
PSBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwOwogICAgIGl2cnNfbWFw
cGluZ3NbYmRmXS51bml0eV9tYXAgPSB1bml0eV9tYXA7CiAKKyAgICBpZiAo
IHBhZ2luZ19tb2RlID4gYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSApCisg
ICAgICAgIGFtZF9pb21tdV9taW5fcGFnaW5nX21vZGUgPSBwYWdpbmdfbW9k
ZTsKKwogICAgIHJldHVybiAwOwogfQogCi1zdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2FsbF9kZXZpY2VzKAotICAg
IHVuc2lnbmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTgg
aXcsIHU4IGlyKQorc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfcmFuZ2Vf
Zm9yX2FsbF9kZXZpY2VzKAorICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBs
aW1pdCwgYm9vbCBpdywgYm9vbCBpciwgYm9vbCBleGNsdXNpb24pCiB7CiAg
ICAgaW50IHNlZyA9IDA7IC8qIFhYWCAqLwotICAgIHVuc2lnbmVkIGxvbmcg
cmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1k
X2lvbW11ICppb21tdTsKLSAgICB1bnNpZ25lZCBpbnQgYmRmOwogICAgIGlu
dCByYyA9IDA7CiAKICAgICAvKiBpcyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5n
ZSBpbnNpZGUgb2YgSU9NTVUgdmlydHVhbCBhZGRyZXNzIHNwYWNlPyAqLwog
ICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8g
YmUgcGFnZS1hbGlnbmVkICovCi0gICAgcmFuZ2VfdG9wID0gbGltaXQgKyBQ
QUdFX1NJWkU7Ci0gICAgaW9tbXVfdG9wID0gbWF4X3BhZ2UgKiBQQUdFX1NJ
WkU7Ci0gICAgaWYgKCBiYXNlIDwgaW9tbXVfdG9wICkKLSAgICB7Ci0gICAg
ICAgIGlmICggcmFuZ2VfdG9wID4gaW9tbXVfdG9wICkKLSAgICAgICAgICAg
IHJhbmdlX3RvcCA9IGlvbW11X3RvcDsKLSAgICAgICAgbGVuZ3RoID0gcmFu
Z2VfdG9wIC0gYmFzZTsKLSAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHkt
bWFwcGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAqLwotICAgICAgICAv
KiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNp
b24gcmFuZ2UgKi8KLSAgICAgICAgZm9yICggYmRmID0gMDsgIXJjICYmIGJk
ZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAgICAgICAgIHJj
ID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFz
ZSwgbGVuZ3RoLCBpdywgaXIpOwotICAgICAgICAvKiBwdXNoICdiYXNlJyBq
dXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCi0gICAg
ICAgIGJhc2UgPSBpb21tdV90b3A7Ci0gICAgfQotICAgIC8qIHJlZ2lzdGVy
IElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICgg
IXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCApCisgICAgaWYgKCBleGNsdXNp
b24gKQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBpb21t
dSApCiAgICAgICAgIHsKLSAgICAgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21t
dV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAotICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cnVl
IC8qIGFsbCAqLywgaXcsIGlyKTsKLSAgICAgICAgICAgIGlmICggcmMgKQot
ICAgICAgICAgICAgICAgIGJyZWFrOworICAgICAgICAgICAgaW50IHJldCA9
IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBs
aW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB0cnVlIC8qIGFsbCAqLyk7CisKKyAgICAgICAgICAg
IGlmICggcmV0ICYmICFyYyApCisgICAgICAgICAgICAgICAgcmMgPSByZXQ7
CiAgICAgICAgIH0KICAgICB9CiAKKyAgICBpZiAoICFleGNsdXNpb24gfHwg
cmMgKQorICAgIHsKKyAgICAgICAgcGFkZHJfdCBsZW5ndGggPSBsaW1pdCAr
IFBBR0VfU0laRSAtIGJhc2U7CisgICAgICAgIHVuc2lnbmVkIGludCBiZGY7
CisKKyAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHktbWFwcGVkIHBhZ2Ug
ZW50cmllcyBmb3IgZGV2aWNlcyAqLworICAgICAgICBmb3IgKCBiZGYgPSBy
YyA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyAp
CisgICAgICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZp
Y2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsKKyAgICB9CisK
ICAgICByZXR1cm4gcmM7CiB9CiAKLXN0YXRpYyBpbnQgX19pbml0IHJlZ2lz
dGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfZGV2aWNlKAotICAgIHUxNiBiZGYs
IHVuc2lnbmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTgg
aXcsIHU4IGlyKQorc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfcmFuZ2Vf
Zm9yX2RldmljZSgKKyAgICB1bnNpZ25lZCBpbnQgYmRmLCBwYWRkcl90IGJh
c2UsIHBhZGRyX3QgbGltaXQsCisgICAgYm9vbCBpdywgYm9vbCBpciwgYm9v
bCBleGNsdXNpb24pCiB7CiAgICAgaW50IHNlZyA9IDA7IC8qIFhYWCAqLwog
ICAgIHN0cnVjdCBpdnJzX21hcHBpbmdzICppdnJzX21hcHBpbmdzID0gZ2V0
X2l2cnNfbWFwcGluZ3Moc2VnKTsKLSAgICB1bnNpZ25lZCBsb25nIHJhbmdl
X3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAgc3RydWN0IGFtZF9pb21t
dSAqaW9tbXU7CiAgICAgdTE2IHJlcTsKICAgICBpbnQgcmMgPSAwOwpAQCAt
MjMwLDI3ICsyMjcsMTkgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJf
ZXhjbHVzaW9uX3JhbgogICAgIHJlcSA9IGl2cnNfbWFwcGluZ3NbYmRmXS5k
dGVfcmVxdWVzdG9yX2lkOwogCiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJh
bWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICBy
YW5nZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsKLSAgICBpb21tdV90b3Ag
PSBtYXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21t
dV90b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21t
dV90b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwot
ICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOworICAgIGlmICgg
ZXhjbHVzaW9uICkKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UgLyogYWxsICov
KTsKKyAgICBpZiAoICFleGNsdXNpb24gfHwgcmMgKQorICAgIHsKKyAgICAg
ICAgcGFkZHJfdCBsZW5ndGggPSBsaW1pdCArIFBBR0VfU0laRSAtIGJhc2U7
CisKICAgICAgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2UgKi8KLSAgICAgICAgLyogbm90ZTogdGhlc2UgZW50
cmllcyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCiAgICAg
ICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJk
ZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpID86CiAgICAgICAgICAgICAgcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIHJlcSwgYmFzZSwgbGVu
Z3RoLCBpdywgaXIpOwotCi0gICAgICAgIC8qIHB1c2ggJ2Jhc2UnIGp1c3Qg
b3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgKi8KLSAgICAgICAg
YmFzZSA9IGlvbW11X3RvcDsKICAgICB9Ci0KLSAgICAvKiByZWdpc3RlciBJ
T01NVSBleGNsdXNpb24gcmFuZ2Ugc2V0dGluZ3MgZm9yIGRldmljZSAqLwot
ICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCAgKQorICAgIGVs
c2UKICAgICB7Ci0gICAgICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAotICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhbHNlIC8qIGFsbCAqLywg
aXcsIGlyKTsKICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmR0ZV9hbGxv
d19leGNsdXNpb24gPSBJT01NVV9DT05UUk9MX0VOQUJMRUQ7CiAgICAgICAg
IGl2cnNfbWFwcGluZ3NbcmVxXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gSU9N
TVVfQ09OVFJPTF9FTkFCTEVEOwogICAgIH0KQEAgLTI1OCw1MyArMjQ3LDQy
IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4K
ICAgICByZXR1cm4gcmM7CiB9CiAKLXN0YXRpYyBpbnQgX19pbml0IHJlZ2lz
dGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfaW9tbXVfZGV2aWNlcygKLSAgICBz
dHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25lZCBsb25nIGJh
c2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3LCB1OCBpcikKK3N0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21tdV9kZXZpY2Vz
KAorICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCBwYWRkcl90IGJhc2Us
IHBhZGRyX3QgbGltaXQsCisgICAgYm9vbCBpdywgYm9vbCBpciwgYm9vbCBl
eGNsdXNpb24pCiB7Ci0gICAgdW5zaWduZWQgbG9uZyByYW5nZV90b3AsIGlv
bW11X3RvcCwgbGVuZ3RoOworICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1l
dGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCisgICAgcGFk
ZHJfdCBsZW5ndGggPSBsaW1pdCArIFBBR0VfU0laRSAtIGJhc2U7CiAgICAg
dW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOwotICAgIGludCByYyA9
IDA7CisgICAgaW50IHJjOwogCi0gICAgLyogaXMgcGFydCBvZiBleGNsdXNp
b24gcmFuZ2UgaW5zaWRlIG9mIElPTU1VIHZpcnR1YWwgYWRkcmVzcyBzcGFj
ZT8gKi8KLSAgICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1
bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwotICAgIHJhbmdlX3RvcCA9IGxp
bWl0ICsgUEFHRV9TSVpFOwotICAgIGlvbW11X3RvcCA9IG1heF9wYWdlICog
UEFHRV9TSVpFOwotICAgIGlmICggYmFzZSA8IGlvbW11X3RvcCApCi0gICAg
ewotICAgICAgICBpZiAoIHJhbmdlX3RvcCA+IGlvbW11X3RvcCApCi0gICAg
ICAgICAgICByYW5nZV90b3AgPSBpb21tdV90b3A7Ci0gICAgICAgIGxlbmd0
aCA9IHJhbmdlX3RvcCAtIGJhc2U7Ci0gICAgICAgIC8qIHJlc2VydmUgci93
IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMgKi8KLSAg
ICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUg
ZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9IDA7ICFy
YyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCi0gICAgICAg
IHsKLSAgICAgICAgICAgIGlmICggaW9tbXUgPT0gZmluZF9pb21tdV9mb3Jf
ZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQotICAgICAgICAgICAgewotICAg
ICAgICAgICAgICAgIHJlcSA9IGdldF9pdnJzX21hcHBpbmdzKGlvbW11LT5z
ZWcpW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZDsKLSAgICAgICAgICAgICAgICBy
YyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywg
YmRmLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcikgPzoKLSAgICAgICAg
ICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcik7Ci0g
ICAgICAgICAgICB9Ci0gICAgICAgIH0KLQotICAgICAgICAvKiBwdXNoICdi
YXNlJyBqdXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICov
Ci0gICAgICAgIGJhc2UgPSBpb21tdV90b3A7CisgICAgaWYgKCBleGNsdXNp
b24gKQorICAgIHsKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsIHRydWUgLyogYWxsICov
KTsKKyAgICAgICAgaWYgKCAhcmMgKQorICAgICAgICAgICAgcmV0dXJuIDA7
CiAgICAgfQogCi0gICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJh
bmdlIHNldHRpbmdzICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9t
bXVfdG9wICkKLSAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lv
bl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3
LCBpcik7CisgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2VzICovCisgICAgZm9yICggYmRmID0gcmMgPSAwOyAh
cmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysgKQorICAgIHsK
KyAgICAgICAgaWYgKCBpb21tdSAhPSBmaW5kX2lvbW11X2Zvcl9kZXZpY2Uo
aW9tbXUtPnNlZywgYmRmKSApCisgICAgICAgICAgICBjb250aW51ZTsKKwor
ICAgICAgICByZXEgPSBnZXRfaXZyc19tYXBwaW5ncyhpb21tdS0+c2VnKVti
ZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7CisgICAgICAgIHJjID0gcmVzZXJ2ZV91
bml0eV9tYXBfZm9yX2RldmljZShpb21tdS0+c2VnLCBiZGYsIGJhc2UsIGxl
bmd0aCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGl3LCBpcikgPzoKKyAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21h
cF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcs
IGlyKTsKKyAgICB9CiAKICAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRpYyBp
bnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdCgKICAgICBjb25z
dCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaywKLSAgICB1
bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3
LCB1OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGltaXQsIGJv
b2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAgIHUxNiBi
ZGY7CiAKQEAgLTMxNSwxMiArMjkzLDEyIEBAIHN0YXRpYyBpbnQgX19pbml0
IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGUKICAgICAgICAgcmV0dXJuIC1FTk9E
RVY7CiAgICAgfQogCi0gICAgcmV0dXJuIHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW5nZV9mb3JfZGV2aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpcik7Cisg
ICAgcmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2UoYmRmLCBiYXNl
LCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQogCiBzdGF0aWMgaW50
IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9yYW5nZSgKICAgICBjb25zdCBz
dHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaywKLSAgICB1bnNp
Z25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3LCB1
OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGltaXQsIGJvb2wg
aXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAgIHVuc2lnbmVk
IGludCBmaXJzdF9iZGYsIGxhc3RfYmRmLCBiZGY7CiAgICAgaW50IGVycm9y
OwpAQCAtMzQyLDE1ICszMjAsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcGFy
c2VfaXZtZF9kZXZpY2VfcmFuZwogICAgIH0KIAogICAgIGZvciAoIGJkZiA9
IGZpcnN0X2JkZiwgZXJyb3IgPSAwOyAoYmRmIDw9IGxhc3RfYmRmKSAmJiAh
ZXJyb3I7IGJkZisrICkKLSAgICAgICAgZXJyb3IgPSByZWdpc3Rlcl9leGNs
dXNpb25fcmFuZ2VfZm9yX2RldmljZSgKLSAgICAgICAgICAgIGJkZiwgYmFz
ZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIGVycm9yID0gcmVnaXN0ZXJf
cmFuZ2VfZm9yX2RldmljZSgKKyAgICAgICAgICAgIGJkZiwgYmFzZSwgbGlt
aXQsIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIHJldHVybiBlcnJvcjsK
IH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9kZXZpY2VfaW9t
bXUoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19tZW1vcnkgKml2bWRf
YmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25n
IGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJfdCBiYXNlLCBwYWRk
cl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikK
IHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCiAgICAgc3RydWN0IGFt
ZF9pb21tdSAqaW9tbXU7CkBAIC0zNjUsMTQgKzM0MywxNCBAQCBzdGF0aWMg
aW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9pb21tCiAgICAgICAgIHJl
dHVybiAtRU5PREVWOwogICAgIH0KIAotICAgIHJldHVybiByZWdpc3Rlcl9l
eGNsdXNpb25fcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCi0gICAgICAgIGlv
bW11LCBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICByZXR1cm4gcmVnaXN0
ZXJfcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCisgICAgICAgIGlvbW11LCBi
YXNlLCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQogCiBzdGF0aWMg
aW50IF9faW5pdCBwYXJzZV9pdm1kX2Jsb2NrKGNvbnN0IHN0cnVjdCBhY3Bp
X2l2cnNfbWVtb3J5ICppdm1kX2Jsb2NrKQogewogICAgIHVuc2lnbmVkIGxv
bmcgc3RhcnRfYWRkciwgbWVtX2xlbmd0aCwgYmFzZSwgbGltaXQ7Ci0gICAg
dTggaXcsIGlyOworICAgIGJvb2wgaXcgPSB0cnVlLCBpciA9IHRydWUsIGV4
Y2x1c2lvbiA9IGZhbHNlOwogCiAgICAgaWYgKCBpdm1kX2Jsb2NrLT5oZWFk
ZXIubGVuZ3RoIDwgc2l6ZW9mKCppdm1kX2Jsb2NrKSApCiAgICAgewpAQCAt
Mzg5LDEzICszNjcsMTEgQEAgc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZt
ZF9ibG9jayhjb25zdAogICAgICAgICAgICAgICAgICAgICBpdm1kX2Jsb2Nr
LT5oZWFkZXIudHlwZSwgc3RhcnRfYWRkciwgbWVtX2xlbmd0aCk7CiAKICAg
ICBpZiAoIGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElfSVZNRF9F
WENMVVNJT05fUkFOR0UgKQotICAgICAgICBpdyA9IGlyID0gSU9NTVVfQ09O
VFJPTF9FTkFCTEVEOworICAgICAgICBleGNsdXNpb24gPSB0cnVlOwogICAg
IGVsc2UgaWYgKCBpdm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lW
TURfVU5JVFkgKQogICAgIHsKLSAgICAgICAgaXcgPSBpdm1kX2Jsb2NrLT5o
ZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfUkVBRCA/Ci0gICAgICAgICAgICBJ
T01NVV9DT05UUk9MX0VOQUJMRUQgOiBJT01NVV9DT05UUk9MX0RJU0FCTEVE
OwotICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFD
UElfSVZNRF9XUklURSA/Ci0gICAgICAgICAgICBJT01NVV9DT05UUk9MX0VO
QUJMRUQgOiBJT01NVV9DT05UUk9MX0RJU0FCTEVEOworICAgICAgICBpdyA9
IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElfSVZNRF9SRUFEOwor
ICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElf
SVZNRF9XUklURTsKICAgICB9CiAgICAgZWxzZQogICAgIHsKQEAgLTQwNiwy
MCArMzgyLDIwIEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfYmxv
Y2soY29uc3QKICAgICBzd2l0Y2goIGl2bWRfYmxvY2stPmhlYWRlci50eXBl
ICkKICAgICB7CiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9NRU1PUllfQUxM
OgotICAgICAgICByZXR1cm4gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zv
cl9hbGxfZGV2aWNlcygKLSAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywg
aXIpOworICAgICAgICByZXR1cm4gcmVnaXN0ZXJfcmFuZ2VfZm9yX2FsbF9k
ZXZpY2VzKAorICAgICAgICAgICAgYmFzZSwgbGltaXQsIGl3LCBpciwgZXhj
bHVzaW9uKTsKIAogICAgIGNhc2UgQUNQSV9JVlJTX1RZUEVfTUVNT1JZX09O
RToKLSAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdChp
dm1kX2Jsb2NrLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgICAgICByZXR1cm4g
cGFyc2VfaXZtZF9kZXZpY2Vfc2VsZWN0KGl2bWRfYmxvY2ssIGJhc2UsIGxp
bWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGNhc2UgQUNQSV9JVlJTX1RZ
UEVfTUVNT1JZX1JBTkdFOgotICAgICAgICByZXR1cm4gcGFyc2VfaXZtZF9k
ZXZpY2VfcmFuZ2UoaXZtZF9ibG9jaywKLSAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAg
ICAgICByZXR1cm4gcGFyc2VfaXZtZF9kZXZpY2VfcmFuZ2UoaXZtZF9ibG9j
aywgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBjYXNlIEFD
UElfSVZSU19UWVBFX01FTU9SWV9JT01NVToKLSAgICAgICAgcmV0dXJuIHBh
cnNlX2l2bWRfZGV2aWNlX2lvbW11KGl2bWRfYmxvY2ssCi0gICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1pdCwgaXcs
IGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX2lvbW11
KGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgaXcsIGlyLCBleGNsdXNpb24pOwogCiAg
ICAgZGVmYXVsdDoKICAgICAgICAgQU1EX0lPTU1VX0RFQlVHKCJJVk1EIEVy
cm9yOiBJbnZhbGlkIEJsb2NrIFR5cGUhXG4iKTsKLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0y
MTgsNiArMjE4LDggQEAgc3RhdGljIGludCBfX211c3RfY2hlY2sgYWxsb2Nh
dGVfZG9tYWluXwogICAgIHJldHVybiByYzsKIH0KIAoraW50IF9fcmVhZF9t
b3N0bHkgYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSA9IDE7CisKIHN0YXRp
YyBpbnQgYW1kX2lvbW11X2RvbWFpbl9pbml0KHN0cnVjdCBkb21haW4gKmQp
CiB7CiAgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUo
ZCk7CkBAIC0yMjksMTEgKzIzMSwxMyBAQCBzdGF0aWMgaW50IGFtZF9pb21t
dV9kb21haW5faW5pdChzdHJ1Y3QKICAgICAgKiAtIEhWTSBjb3VsZCBpbiBw
cmluY2lwbGUgdXNlIDMgb3IgNCBkZXBlbmRpbmcgb24gaG93IG11Y2ggZ3Vl
c3QKICAgICAgKiAgIHBoeXNpY2FsIGFkZHJlc3Mgc3BhY2Ugd2UgZ2l2ZSBp
dCwgYnV0IHRoaXMgaXNuJ3Qga25vd24geWV0IHNvIHVzZSA0CiAgICAgICog
ICB1bmlsYXRlcmFsbHkuCisgICAgICogLSBVbml0eSBtYXBzIG1heSByZXF1
aXJlIGFuIGV2ZW4gaGlnaGVyIG51bWJlci4KICAgICAgKi8KLSAgICBoZC0+
YXJjaC5wYWdpbmdfbW9kZSA9IGFtZF9pb21tdV9nZXRfcGFnaW5nX21vZGUo
Ci0gICAgICAgIGlzX2h2bV9kb21haW4oZCkKLSAgICAgICAgPyAxdWwgPDwg
KERFRkFVTFRfRE9NQUlOX0FERFJFU1NfV0lEVEggLSBQQUdFX1NISUZUKQot
ICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91bmQoKSArIDEpOworICAgIGhk
LT5hcmNoLnBhZ2luZ19tb2RlID0gbWF4KGFtZF9pb21tdV9nZXRfcGFnaW5n
X21vZGUoCisgICAgICAgICAgICBpc19odm1fZG9tYWluKGQpCisgICAgICAg
ICAgICA/IDF1bCA8PCAoREVGQVVMVF9ET01BSU5fQUREUkVTU19XSURUSCAt
IFBBR0VfU0hJRlQpCisgICAgICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91
bmQoKSArIDEpLAorICAgICAgICBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2Rl
KTsKIAogICAgIHJldHVybiAwOwogfQotLS0gYS94ZW4vaW5jbHVkZS9hc20t
eDg2L2h2bS9zdm0vYW1kLWlvbW11LXByb3RvLmgKKysrIGIveGVuL2luY2x1
ZGUvYXNtLXg4Ni9odm0vc3ZtL2FtZC1pb21tdS1wcm90by5oCkBAIC0xMjYs
NiArMTI2LDggQEAgZXh0ZXJuIHN0cnVjdCBocGV0X3NiZGYgewogICAgIH0g
aW5pdDsKIH0gaHBldF9zYmRmOwogCitleHRlcm4gaW50IGFtZF9pb21tdV9t
aW5fcGFnaW5nX21vZGU7CisKIGV4dGVybiB2b2lkICpzaGFyZWRfaW50cmVt
YXBfdGFibGU7CiBleHRlcm4gdW5zaWduZWQgbG9uZyAqc2hhcmVkX2ludHJl
bWFwX2ludXNlOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtNzM2LDcgKzcz
Niw3IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRl
cik7IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2Vu
dHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5V
TEwpOwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFw
Mm1faXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAg
ICAgICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3No
YXJlZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4o
bWZuK2ksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAgfQogICAgIH0K
QEAgLTg0OCwxMyArODQ4LDEzIEBAIGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5
KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAgICAgICAgICAg
IEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAgICB9Ci0gICAg
ICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNfZm9yZWlnbihv
dCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90KSApCiAgICAg
ICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRuJ3QgYmUgdW5t
YXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAqLworICAgICAg
ICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWdu
IHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwog
ICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAgICAgICAgCi0g
ICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAgICAgIHJldHVy
biAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBpZiAoIHAybV9p
c19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAgICAgICAgewpA
QCAtOTQ3LDggKzk0Nyw3IEBAIGludCBwMm1fY2hhbmdlX3R5cGVfb25lKHN0
cnVjdCBkb21haW4gKmQKICAgICBzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtID0g
cDJtX2dldF9ob3N0cDJtKGQpOwogICAgIGludCByYzsKIAotICAgIEJVR19P
TihwMm1faXNfZ3JhbnQob3QpIHx8IHAybV9pc19ncmFudChudCkpOwotICAg
IEJVR19PTihwMm1faXNfZm9yZWlnbihvdCkgfHwgcDJtX2lzX2ZvcmVpZ24o
bnQpKTsKKyAgICBCVUdfT04ocDJtX2lzX3NwZWNpYWwob3QpIHx8IHAybV9p
c19zcGVjaWFsKG50KSk7CiAKICAgICBnZm5fbG9jayhwMm0sIGdmbiwgMCk7
CiAKQEAgLTEwOTEsMTEgKzEwOTAsMTEgQEAgc3RhdGljIGludCBzZXRfdHlw
ZWRfcDJtX2VudHJ5KHN0cnVjdCBkbwogICAgICAgICBnZm5fdW5sb2NrKHAy
bSwgZ2ZuLCBvcmRlcik7CiAgICAgICAgIHJldHVybiBjdXJfb3JkZXIgKyAx
OwogICAgIH0KLSAgICBpZiAoIHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lz
X2ZvcmVpZ24ob3QpICkKKyAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90KSAp
CiAgICAgewogICAgICAgICBnZm5fdW5sb2NrKHAybSwgZ2ZuLCBvcmRlcik7
CiAgICAgICAgIGRvbWFpbl9jcmFzaChkKTsKLSAgICAgICAgcmV0dXJuIC1F
Tk9FTlQ7CisgICAgICAgIHJldHVybiAtRVBFUk07CiAgICAgfQogICAgIGVs
c2UgaWYgKCBwMm1faXNfcmFtKG90KSApCiAgICAgewotLS0gYS94ZW4vaW5j
bHVkZS9hc20teDg2L3AybS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYv
cDJtLmgKQEAgLTE0Miw2ICsxNDIsMTAgQEAgdHlwZWRlZiB1bnNpZ25lZCBp
bnQgcDJtX3F1ZXJ5X3Q7CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
fCBwMm1fdG9fbWFzayhwMm1fcmFtX2xvZ2RpcnR5KSApCiAjZGVmaW5lIFAy
TV9TSEFSRURfVFlQRVMgICAocDJtX3RvX21hc2socDJtX3JhbV9zaGFyZWQp
KQogCisvKiBUeXBlcyBlc3RhYmxpc2hlZC9jbGVhbmVkIHVwIHZpYSBzcGVj
aWFsIGFjY2Vzc29ycy4gKi8KKyNkZWZpbmUgUDJNX1NQRUNJQUxfVFlQRVMg
KFAyTV9HUkFOVF9UWVBFUyB8IFwKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHAybV90b19tYXNrKHAybV9tYXBfZm9yZWlnbikpCisKIC8qIFZhbGlk
IHR5cGVzIG5vdCBuZWNlc3NhcmlseSBhc3NvY2lhdGVkIHdpdGggYSAodmFs
aWQpIE1GTi4gKi8KICNkZWZpbmUgUDJNX0lOVkFMSURfTUZOX1RZUEVTIChQ
Mk1fUE9EX1RZUEVTICAgICAgICAgICAgICAgICAgXAogICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHwgcDJtX3RvX21hc2socDJtX21taW9fZGly
ZWN0KSBcCkBAIC0xNzAsNiArMTc0LDcgQEAgdHlwZWRlZiB1bnNpZ25lZCBp
bnQgcDJtX3F1ZXJ5X3Q7CiAjZGVmaW5lIHAybV9pc19wYWdlZChfdCkgICAg
KHAybV90b19tYXNrKF90KSAmIFAyTV9QQUdFRF9UWVBFUykKICNkZWZpbmUg
cDJtX2lzX3NoYXJhYmxlKF90KSAocDJtX3RvX21hc2soX3QpICYgUDJNX1NI
QVJBQkxFX1RZUEVTKQogI2RlZmluZSBwMm1faXNfc2hhcmVkKF90KSAgIChw
Mm1fdG9fbWFzayhfdCkgJiBQMk1fU0hBUkVEX1RZUEVTKQorI2RlZmluZSBw
Mm1faXNfc3BlY2lhbChfdCkgIChwMm1fdG9fbWFzayhfdCkgJiBQMk1fU1BF
Q0lBTF9UWVBFUykKICNkZWZpbmUgcDJtX2lzX2Jyb2tlbihfdCkgICAocDJt
X3RvX21hc2soX3QpICYgUDJNX0JST0tFTl9UWVBFUykKICNkZWZpbmUgcDJt
X2lzX2ZvcmVpZ24oX3QpICAocDJtX3RvX21hc2soX3QpICYgcDJtX3RvX21h
c2socDJtX21hcF9mb3JlaWduKSkKIAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.11-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.11-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tLmMK
KysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ3ODMsNyArNDc4Myw5IEBA
IGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogCiAgICAgLyogUmVt
b3ZlIHByZXZpb3VzbHkgbWFwcGVkIHBhZ2UgaWYgaXQgd2FzIHByZXNlbnQu
ICovCiAgICAgcHJldl9tZm4gPSBtZm5feChnZXRfZ2ZuKGQsIGdmbl94KGdw
Zm4pLCAmcDJtdCkpOwotICAgIGlmICggbWZuX3ZhbGlkKF9tZm4ocHJldl9t
Zm4pKSApCisgICAgaWYgKCBwMm10ID09IHAybV9tbWlvX2RpcmVjdCApCisg
ICAgICAgIHJjID0gLUVQRVJNOworICAgIGVsc2UgaWYgKCBtZm5fdmFsaWQo
X21mbihwcmV2X21mbikpICkKICAgICB7CiAgICAgICAgIGlmICggaXNfeGVu
X2hlYXBfbWZuKHByZXZfbWZuKSApCiAgICAgICAgICAgICAvKiBYZW4gaGVh
cCBmcmFtZXMgYXJlIHNpbXBseSB1bmhvb2tlZCBmcm9tIHRoaXMgcGh5cyBz
bG90LiAqLwotLS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLmMKKysrIGIveGVu
L2FyY2gveDg2L21tL3AybS5jCkBAIC03MjUsNyArNzI1LDggQEAgcDJtX3Jl
bW92ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmY3VyX29yZGVyLCBO
VUxMKTsKIAogICAgICAgICBpZiAoIHAybV9pc192YWxpZCh0KSAmJgotICAg
ICAgICAgICAgICghbWZuX3ZhbGlkKF9tZm4obWZuKSkgfHwgbWZuICsgaSAh
PSBtZm5feChtZm5fcmV0dXJuKSkgKQorICAgICAgICAgICAgICghbWZuX3Zh
bGlkKF9tZm4obWZuKSkgfHwgdCA9PSBwMm1fbW1pb19kaXJlY3QgfHwKKyAg
ICAgICAgICAgICAgbWZuICsgaSAhPSBtZm5feChtZm5fcmV0dXJuKSkgKQog
ICAgICAgICAgICAgcmV0dXJuIC1FSUxTRVE7CiAKICAgICAgICAgaSArPSAo
MVVMIDw8IGN1cl9vcmRlcikgLSAoKGdmbl9sICsgaSkgJiAoKDFVTCA8PCBj
dXJfb3JkZXIpIC0gMSkpOwpAQCAtODAzLDcgKzgwNCw3IEBAIGd1ZXN0X3Bo
eXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICBpZiAoIHAy
bV9pc19mb3JlaWduKHQpICkKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7CiAK
LSAgICBpZiAoICFtZm5fdmFsaWQobWZuKSApCisgICAgaWYgKCAhbWZuX3Zh
bGlkKG1mbikgfHwgdCA9PSBwMm1fbW1pb19kaXJlY3QgKQogICAgIHsKICAg
ICAgICAgQVNTRVJUX1VOUkVBQ0hBQkxFKCk7CiAgICAgICAgIHJldHVybiAt
RUlOVkFMOwpAQCAtODUwLDcgKzg1MSw3IEBAIGd1ZXN0X3BoeXNtYXBfYWRk
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgfQogICAgICAgICBp
ZiAoIHAybV9pc19zcGVjaWFsKG90KSApCiAgICAgICAgIHsKLSAgICAgICAg
ICAgIC8qIERvbid0IHBlcm1pdCB1bm1hcHBpbmcgZ3JhbnQvZm9yZWlnbiB0
aGlzIHdheS4gKi8KKyAgICAgICAgICAgIC8qIERvbid0IHBlcm1pdCB1bm1h
cHBpbmcgZ3JhbnQvZm9yZWlnbi9kaXJlY3QtTU1JTyB0aGlzIHdheS4gKi8K
ICAgICAgICAgICAgIGRvbWFpbl9jcmFzaChkKTsKICAgICAgICAgICAgIHAy
bV91bmxvY2socDJtKTsKICAgICAgICAgICAgIApAQCAtMTE5Miw4ICsxMTkz
LDggQEAgaW50IHNldF9pZGVudGl0eV9wMm1fZW50cnkoc3RydWN0IGRvbWFp
bgogICogICAgb3JkZXIrMSAgZm9yIGNhbGxlciB0byByZXRyeSB3aXRoIG9y
ZGVyIChndWFyYW50ZWVkIHNtYWxsZXIgdGhhbgogICogICAgICAgICAgICAg
dGhlIG9yZGVyIHZhbHVlIHBhc3NlZCBpbikKICAqLwotaW50IGNsZWFyX21t
aW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuX2wsIG1mbl90IG1mbiwKLSAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgb3JkZXIpCitzdGF0aWMgaW50IGNsZWFyX21taW9fcDJt
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuX2ws
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1mbl90IG1mbiwg
dW5zaWduZWQgaW50IG9yZGVyKQogewogICAgIGludCByYyA9IC1FSU5WQUw7
CiAgICAgZ2ZuX3QgZ2ZuID0gX2dmbihnZm5fbCk7Ci0tLSBhL3hlbi9hcmNo
L3g4Ni9tbS9wMm0tcG9kLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS1w
b2QuYwpAQCAtMTMwMiwxNyArMTMwMiwxNyBAQCBndWVzdF9waHlzbWFwX21h
cmtfcG9wdWxhdGVfb25fZGVtYW5kKHN0CiAKICAgICAgICAgcDJtLT5nZXRf
ZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZvdCwgJmEsIDAsICZjdXJf
b3JkZXIsIE5VTEwpOwogICAgICAgICBuID0gMVVMIDw8IG1pbihvcmRlciwg
Y3VyX29yZGVyKTsKLSAgICAgICAgaWYgKCBwMm1faXNfcmFtKG90KSApCisg
ICAgICAgIGlmICggb3QgPT0gcDJtX3BvcHVsYXRlX29uX2RlbWFuZCApCisg
ICAgICAgIHsKKyAgICAgICAgICAgIC8qIENvdW50IGhvdyBtYW55IFBvRCBl
bnRyaWVzIHdlJ2xsIGJlIHJlcGxhY2luZyBpZiBzdWNjZXNzZnVsICovCisg
ICAgICAgICAgICBwb2RfY291bnQgKz0gbjsKKyAgICAgICAgfQorICAgICAg
ICBlbHNlIGlmICggb3QgIT0gcDJtX2ludmFsaWQgJiYgb3QgIT0gcDJtX21t
aW9fZG0gKQogICAgICAgICB7CiAgICAgICAgICAgICBQMk1fREVCVUcoImdm
bl90b19tZm4gcmV0dXJuZWQgdHlwZSAlZCFcbiIsIG90KTsKICAgICAgICAg
ICAgIHJjID0gLUVCVVNZOwogICAgICAgICAgICAgZ290byBvdXQ7CiAgICAg
ICAgIH0KLSAgICAgICAgZWxzZSBpZiAoIG90ID09IHAybV9wb3B1bGF0ZV9v
bl9kZW1hbmQgKQotICAgICAgICB7Ci0gICAgICAgICAgICAvKiBDb3VudCBo
b3cgbWFuIFBvRCBlbnRyaWVzIHdlJ2xsIGJlIHJlcGxhY2luZyBpZiBzdWNj
ZXNzZnVsICovCi0gICAgICAgICAgICBwb2RfY291bnQgKz0gbjsKLSAgICAg
ICAgfQogICAgIH0KIAogICAgIC8qIE5vdywgYWN0dWFsbHkgZG8gdGhlIHR3
by13YXkgbWFwcGluZyAqLwotLS0gYS94ZW4vY29tbW9uL21lbW9yeS5jCisr
KyBiL3hlbi9jb21tb24vbWVtb3J5LmMKQEAgLTMzNSw3ICszMzUsNyBAQCBp
bnQgZ3Vlc3RfcmVtb3ZlX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwKICAgICB9
CiAgICAgaWYgKCBwMm10ID09IHAybV9tbWlvX2RpcmVjdCApCiAgICAgewot
ICAgICAgICByYyA9IGNsZWFyX21taW9fcDJtX2VudHJ5KGQsIGdtZm4sIG1m
biwgUEFHRV9PUkRFUl80Syk7CisgICAgICAgIHJjID0gLUVQRVJNOwogICAg
ICAgICBnb3RvIG91dF9wdXRfZ2ZuOwogICAgIH0KICNlbHNlCkBAIC0xNjUx
LDYgKzE2NTEsMTUgQEAgaW50IHByZXBhcmVfcmluZ19mb3JfaGVscGVyKAog
ICAgICAgICByZXR1cm4gLUVOT0VOVDsKICAgICB9CiAjZW5kaWYKKyNpZmRl
ZiBDT05GSUdfWDg2CisgICAgaWYgKCBwMm10ID09IHAybV9tbWlvX2RpcmVj
dCApCisgICAgeworICAgICAgICBpZiAoIHBhZ2UgKQorICAgICAgICAgICAg
cHV0X3BhZ2UocGFnZSk7CisKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKKyAg
ICB9CisjZW5kaWYKIAogICAgIGlmICggIXBhZ2UgKQogICAgICAgICByZXR1
cm4gLUVJTlZBTDsKLS0tIGEveGVuL2luY2x1ZGUvYXNtLXg4Ni9wMm0uaAor
KysgYi94ZW4vaW5jbHVkZS9hc20teDg2L3AybS5oCkBAIC0xNDQsNyArMTQ0
LDggQEAgdHlwZWRlZiB1bnNpZ25lZCBpbnQgcDJtX3F1ZXJ5X3Q7CiAKIC8q
IFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQgdXAgdmlhIHNwZWNpYWwgYWNj
ZXNzb3JzLiAqLwogI2RlZmluZSBQMk1fU1BFQ0lBTF9UWVBFUyAoUDJNX0dS
QU5UX1RZUEVTIHwgXAotICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJt
X3RvX21hc2socDJtX21hcF9mb3JlaWduKSkKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHAybV90b19tYXNrKHAybV9tYXBfZm9yZWlnbikgfCBcCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1fdG9fbWFzayhwMm1fbW1p
b19kaXJlY3QpKQogCiAvKiBWYWxpZCB0eXBlcyBub3QgbmVjZXNzYXJpbHkg
YXNzb2NpYXRlZCB3aXRoIGEgKHZhbGlkKSBNRk4uICovCiAjZGVmaW5lIFAy
TV9JTlZBTElEX01GTl9UWVBFUyAoUDJNX1BPRF9UWVBFUyAgICAgICAgICAg
ICAgICAgIFwKQEAgLTYyOSw4ICs2MzAsNiBAQCBpbnQgc2V0X2ZvcmVpZ25f
cDJtX2VudHJ5KHN0cnVjdCBkb21haW4KIC8qIFNldCBtbWlvIGFkZHJlc3Nl
cyBpbiB0aGUgcDJtIHRhYmxlIChmb3IgcGFzcy10aHJvdWdoKSAqLwogaW50
IHNldF9tbWlvX3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25l
ZCBsb25nIGdmbiwgbWZuX3QgbWZuLAogICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgb3JkZXIsIHAybV9hY2Nlc3NfdCBhY2Nlc3MpOwot
aW50IGNsZWFyX21taW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVu
c2lnbmVkIGxvbmcgZ2ZuLCBtZm5fdCBtZm4sCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVyKTsKIAogLyogU2V0IGlkZW50
aXR5IGFkZHJlc3NlcyBpbiB0aGUgcDJtIHRhYmxlIChmb3IgcGFzcy10aHJv
dWdoKSAqLwogaW50IHNldF9pZGVudGl0eV9wMm1fZW50cnkoc3RydWN0IGRv
bWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-0a.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-0a.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBmaXggUG9EIGFjY291bnRpbmcgaW4gZ3Vlc3RfcGh5c21h
cF9hZGRfZW50cnkoKQoKVGhlIGluaXRpYWwgb2JzZXJ2YXRpb24gd2FzIHRo
YXQgdGhlIG1mbl92YWxpZCgpIGNoZWNrIGNvbWVzIHRvbyBsYXRlOgpOZWl0
aGVyIG1mbl9hZGQoKSBub3IgbWZuX3RvX3BhZ2UoKSAobGV0IGFsb25lIGRl
LXJlZmVyZW5jaW5nIHRoZQpyZXN1bHQgb2YgdGhlIGxhdHRlcikgYXJlIHZh
bGlkIGZvciBNRk5zIGZhaWxpbmcgdGhpcyBjaGVjay4gTW92ZSBpdCB1cAph
bmQgLSBub3RpY2luZyB0aGF0IHRoZXJlJ3Mgbm8gY2FsbGVyIGRvaW5nIHNv
IC0gYWxzbyBhZGQgYW4gYXNzZXJ0aW9uCnRoYXQgdGhpcyBzaG91bGQgbmV2
ZXIgcHJvZHVjZSAiZmFsc2UiIGhlcmUuCgpJbiB0dXJuIHRoaXMgd291bGQg
aGF2ZSBtZWFudCB0aGF0IHRoZSAiZWxzZSIgdG8gdGhhdCBpZigpIGNvdWxk
IG5vdyBnbwphd2F5LCB3aGljaCBkaWRuJ3Qgc2VlbSByaWdodCBhdCBhbGwu
IEFuZCBpbmRlZWQsIGNvbnNpZGVyaW5nIGNhbGxlcnMKbGlrZSBtZW1vcnlf
ZXhjaGFuZ2UoKSBvciB2YXJpb3VzIGdyYW50IHRhYmxlIGZ1bmN0aW9ucywg
dGhlIFBvRAphY2NvdW50aW5nIHNob3VsZCBoYXZlIGJlZW4gb3V0c2lkZSBv
ZiB0aGF0IGlmKCkgZnJvbSB0aGUgdmVyeQpiZWdpbm5pbmcuCgpTaWduZWQt
b2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+CkFja2Vk
LWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29t
PgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3AybS5jCisrKyBiL3hlbi9hcmNo
L3g4Ni9tbS9wMm0uYwpAQCAtODY0LDYgKzg2NCwxMiBAQCBndWVzdF9waHlz
bWFwX2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgaWYgKCBwMm1f
aXNfZm9yZWlnbih0KSApCiAgICAgICAgIHJldHVybiAtRUlOVkFMOwogCisg
ICAgaWYgKCAhbWZuX3ZhbGlkKG1mbikgKQorICAgIHsKKyAgICAgICAgQVNT
RVJUX1VOUkVBQ0hBQkxFKCk7CisgICAgICAgIHJldHVybiAtRUlOVkFMOwor
ICAgIH0KKwogICAgIHAybV9sb2NrKHAybSk7CiAKICAgICBQMk1fREVCVUco
ImFkZGluZyBnZm49JSNseCBtZm49JSNseFxuIiwgZ2ZuX3goZ2ZuKSwgbWZu
X3gobWZuKSk7CkBAIC05NjMsMTIgKzk2OSwxNSBAQCBndWVzdF9waHlzbWFw
X2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgfQogCiAgICAgLyog
Tm93LCBhY3R1YWxseSBkbyB0aGUgdHdvLXdheSBtYXBwaW5nICovCi0gICAg
aWYgKCBtZm5fdmFsaWQobWZuKSApCisgICAgcmMgPSBwMm1fc2V0X2VudHJ5
KHAybSwgZ2ZuLCBtZm4sIHBhZ2Vfb3JkZXIsIHQsIHAybS0+ZGVmYXVsdF9h
Y2Nlc3MpOworICAgIGlmICggcmMgPT0gMCApCiAgICAgewotICAgICAgICBy
YyA9IHAybV9zZXRfZW50cnkocDJtLCBnZm4sIG1mbiwgcGFnZV9vcmRlciwg
dCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybS0+ZGVmYXVsdF9h
Y2Nlc3MpOwotICAgICAgICBpZiAoIHJjICkKLSAgICAgICAgICAgIGdvdG8g
b3V0OyAvKiBGYWlsZWQgdG8gdXBkYXRlIHAybSwgYmFpbCB3aXRob3V0IHVw
ZGF0aW5nIG0ycC4gKi8KKyNpZmRlZiBDT05GSUdfSFZNCisgICAgICAgIHBv
ZF9sb2NrKHAybSk7CisgICAgICAgIHAybS0+cG9kLmVudHJ5X2NvdW50IC09
IHBvZF9jb3VudDsKKyAgICAgICAgQlVHX09OKHAybS0+cG9kLmVudHJ5X2Nv
dW50IDwgMCk7CisgICAgICAgIHBvZF91bmxvY2socDJtKTsKKyNlbmRpZgog
CiAgICAgICAgIGlmICggIXAybV9pc19ncmFudCh0KSApCiAgICAgICAgIHsK
QEAgLTk3NywyNCArOTg2LDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnko
c3RydWN0IGRvbWFpbiAqZAogICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGdmbl94KGdmbl9hZGQoZ2ZuLCBpKSkpOwogICAgICAgICB9CiAg
ICAgfQotICAgIGVsc2UKLSAgICB7Ci0gICAgICAgIGdkcHJpbnRrKFhFTkxP
R19XQVJOSU5HLCAiQWRkaW5nIGJhZCBtZm4gdG8gcDJtIG1hcCAoJSNseCAt
PiAlI2x4KVxuIiwKLSAgICAgICAgICAgICAgICAgZ2ZuX3goZ2ZuKSwgbWZu
X3gobWZuKSk7Ci0gICAgICAgIHJjID0gcDJtX3NldF9lbnRyeShwMm0sIGdm
biwgSU5WQUxJRF9NRk4sIHBhZ2Vfb3JkZXIsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICBwMm1faW52YWxpZCwgcDJtLT5kZWZhdWx0X2FjY2Vzcyk7
Ci0jaWZkZWYgQ09ORklHX0hWTQotICAgICAgICBpZiAoIHJjID09IDAgKQot
ICAgICAgICB7Ci0gICAgICAgICAgICBwb2RfbG9jayhwMm0pOwotICAgICAg
ICAgICAgcDJtLT5wb2QuZW50cnlfY291bnQgLT0gcG9kX2NvdW50OwotICAg
ICAgICAgICAgQlVHX09OKHAybS0+cG9kLmVudHJ5X2NvdW50IDwgMCk7Ci0g
ICAgICAgICAgICBwb2RfdW5sb2NrKHAybSk7Ci0gICAgICAgIH0KLSNlbmRp
ZgotICAgIH0KIAotb3V0OgogICAgIHAybV91bmxvY2socDJtKTsKIAogICAg
IHJldHVybiByYzsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-0b.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-0b.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBpZ25vcmUgcDJtX3JlbW92ZV9wYWdlKCkncyBy
ZXR1cm4gdmFsdWUKCkl0J3Mgbm90IHZlcnkgbmljZSB0byByZXR1cm4gZnJv
bSBndWVzdF9waHlzbWFwX2FkZF9lbnRyeSgpIGFmdGVyCnBlcmhhcHMgYWxy
ZWFkeSBoYXZpbmcgbWFkZSBzb21lIGNoYW5nZXMgdG8gdGhlIFAyTSwgYnV0
IHRoaXMgaXMgcHJlLQpleGlzdGluZyBwcmFjdGljZSBpbiB0aGUgZnVuY3Rp
b24sIGFuZCBpbW8gYmV0dGVyIHRoYW4gaWdub3JpbmcgZXJyb3JzLgoKVGFr
ZSB0aGUgbGliZXJ0eSBhbmQgcmVwbGFjZSBhbiBtZm5fYWRkKCkgaW5zdGFu
Y2Ugd2l0aCBhIGxvY2FsIHZhcmlhYmxlCmFscmVhZHkgaG9sZGluZyB0aGUg
cmVzdWx0IChhcyBwcm92ZW4gYnkgdGhlIGNoZWNrIGltbWVkaWF0ZWx5IGFo
ZWFkKS4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBz
dXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVycmFudCA8cGF1bC5kdXJy
YW50QGNpdHJpeC5jb20+CkFja2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgoKLS0tIGEveGVuL2FyY2gveDg2L21t
L3AybS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtNzcyLDgg
Kzc3Miw3IEBAIHZvaWQgcDJtX2ZpbmFsX3RlYXJkb3duKHN0cnVjdCBkb21h
aW4gKmQKICAgICBwMm1fdGVhcmRvd25faG9zdHAybShkKTsKIH0KIAotCi1z
dGF0aWMgaW50CitzdGF0aWMgaW50IF9fbXVzdF9jaGVjawogcDJtX3JlbW92
ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpwMm0sIHVuc2lnbmVkIGxvbmcg
Z2ZuX2wsIHVuc2lnbmVkIGxvbmcgbWZuLAogICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBwYWdlX29yZGVyKQogewpAQCAtOTYxLDkgKzk2MCw5IEBA
IGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAg
ICAgICAgICAgICAgICBBU1NFUlQobWZuX3ZhbGlkKG9tZm4pKTsKICAgICAg
ICAgICAgICAgICBQMk1fREVCVUcoIm9sZCBnZm49JSNseCAtPiBtZm4gJSNs
eFxuIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2ZuX3gob2dmbikg
LCBtZm5feChvbWZuKSk7Ci0gICAgICAgICAgICAgICAgaWYgKCBtZm5fZXEo
b21mbiwgbWZuX2FkZChtZm4sIGkpKSApCi0gICAgICAgICAgICAgICAgICAg
IHAybV9yZW1vdmVfcGFnZShwMm0sIGdmbl94KG9nZm4pLCBtZm5feChtZm5f
YWRkKG1mbiwgaSkpLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgMCk7CisgICAgICAgICAgICAgICAgaWYgKCBtZm5fZXEob21mbiwg
bWZuX2FkZChtZm4sIGkpKSAmJgorICAgICAgICAgICAgICAgICAgICAgKHJj
ID0gcDJtX3JlbW92ZV9wYWdlKHAybSwgZ2ZuX3gob2dmbiksIG1mbl94KG9t
Zm4pLCAwKSkgKQorICAgICAgICAgICAgICAgICAgICBnb3RvIG91dDsKICAg
ICAgICAgICAgIH0KICAgICAgICAgfQogICAgIH0KQEAgLTk4Nyw2ICs5ODYs
NyBAQCBndWVzdF9waHlzbWFwX2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpk
CiAgICAgICAgIH0KICAgICB9CiAKKyBvdXQ6CiAgICAgcDJtX3VubG9jayhw
Mm0pOwogCiAgICAgcmV0dXJuIHJjOwpAQCAtMjY0Niw5ICsyNjQ2LDkgQEAg
aW50IHAybV9jaGFuZ2VfYWx0cDJtX2dmbihzdHJ1Y3QgZG9tYWluCiAKICAg
ICBpZiAoIGdmbl9lcShuZXdfZ2ZuLCBJTlZBTElEX0dGTikgKQogICAgIHsK
LSAgICAgICAgaWYgKCBtZm5fdmFsaWQobWZuKSApCi0gICAgICAgICAgICBw
Mm1fcmVtb3ZlX3BhZ2UoYXAybSwgZ2ZuX3gob2xkX2dmbiksIG1mbl94KG1m
biksIFBBR0VfT1JERVJfNEspOwotICAgICAgICByYyA9IDA7CisgICAgICAg
IHJjID0gbWZuX3ZhbGlkKG1mbikKKyAgICAgICAgICAgICA/IHAybV9yZW1v
dmVfcGFnZShhcDJtLCBnZm5feChvbGRfZ2ZuKSwgbWZuX3gobWZuKSwgUEFH
RV9PUkRFUl80SykKKyAgICAgICAgICAgICA6IDA7CiAgICAgICAgIGdvdG8g
b3V0OwogICAgIH0KIAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-0c.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-0c.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBhc3NlcnQgdGhhdCB0aGUgcGFzc2VkIGluIE1G
TiBtYXRjaGVzIGZvciBhIHJlbW92ZQoKZ3Vlc3RfcGh5c21hcF9yZW1vdmVf
cGFnZSgpIGdldHMgaGFuZGVkIGFuIE1GTiBmcm9tIHRoZSBvdXRzaWRlLCB5
ZXQKdGFrZXMgdGhlIG5lY2Vzc2FyeSBsb2NrIHRvIHByZXZlbnQgZnVydGhl
ciBjaGFuZ2VzIHRvIHRoZSBHRk4gPC0+IE1GTgptYXBwaW5nIGl0c2VsZi4g
V2hpbGUgc29tZSBjYWxsZXJzLCBpbiBwYXJ0aWN1bGFyIGd1ZXN0X3JlbW92
ZV9wYWdlKCkKKGJ5IHdheSBvZiBoYXZpbmcgY2FsbGVkIGdldF9nZm5fcXVl
cnkoKSksIGhvbGQgdGhlIEdGTiBsb2NrIGFscmVhZHksCnZhcmlvdXMgb3Ro
ZXJzIChtb3N0IG5vdGFibHkgcGVyaGFwcyB0aGUgMm5kIGluc3RhbmNlIGlu
Cnhlbm1lbV9hZGRfdG9fcGh5c21hcF9vbmUoKSkgZG9uJ3QuIFdoaWxlIGl0
IGFsc28gaXMgYW4gb3B0aW9uIHRvIGZpeAphbGwgdGhlIGNhbGxlcnMsIGRl
YWwgd2l0aCB0aGUgaXNzdWUgaW4gcDJtX3JlbW92ZV9wYWdlKCkgaW5zdGVh
ZDoKUmVwbGFjZSB0aGUgQVNTRVJUKCkgYnkgYSBjb25kaXRpb25hbCBhbmQg
c3BsaXQgdGhlIGxvb3AgaW50byB0d28sIHN1Y2gKdGhhdCBhbGwgY2hlY2tp
bmcgZ2V0cyBkb25lIGJlZm9yZSBhbnkgbW9kaWZpY2F0aW9uIHdvdWxkIG9j
Y3VyLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1
c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsLmR1cnJh
bnRAY2l0cml4LmNvbT4KQWNrZWQtYnk6IEFuZHJldyBDb29wZXIgPGFuZHJl
dy5jb29wZXIzQGNpdHJpeC5jb20+CgotLS0gYS94ZW4vYXJjaC94ODYvbW0v
cDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC03NzgsNyAr
Nzc4LDYgQEAgcDJtX3JlbW92ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpw
Mm0sCiB7CiAgICAgdW5zaWduZWQgbG9uZyBpOwogICAgIGdmbl90IGdmbiA9
IF9nZm4oZ2ZuX2wpOwotICAgIG1mbl90IG1mbl9yZXR1cm47CiAgICAgcDJt
X3R5cGVfdCB0OwogICAgIHAybV9hY2Nlc3NfdCBhOwogCkBAIC03ODksMTUg
Kzc4OCwyNiBAQCBwMm1fcmVtb3ZlX3BhZ2Uoc3RydWN0IHAybV9kb21haW4g
KnAybSwKICAgICBBU1NFUlQoZ2ZuX2xvY2tlZF9ieV9tZShwMm0sIGdmbikp
OwogICAgIFAyTV9ERUJVRygicmVtb3ZpbmcgZ2ZuPSUjbHggbWZuPSUjbHhc
biIsIGdmbl9sLCBtZm4pOwogCisgICAgZm9yICggaSA9IDA7IGkgPCAoMVVM
IDw8IHBhZ2Vfb3JkZXIpOyApCisgICAgeworICAgICAgICB1bnNpZ25lZCBp
bnQgY3VyX29yZGVyOworICAgICAgICBtZm5fdCBtZm5fcmV0dXJuID0gcDJt
LT5nZXRfZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZ0LCAmYSwgMCwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICZj
dXJfb3JkZXIsIE5VTEwpOworCisgICAgICAgIGlmICggcDJtX2lzX3ZhbGlk
KHQpICYmCisgICAgICAgICAgICAgKCFtZm5fdmFsaWQoX21mbihtZm4pKSB8
fCBtZm4gKyBpICE9IG1mbl94KG1mbl9yZXR1cm4pKSApCisgICAgICAgICAg
ICByZXR1cm4gLUVJTFNFUTsKKworICAgICAgICBpICs9ICgxVUwgPDwgY3Vy
X29yZGVyKSAtICgoZ2ZuX2wgKyBpKSAmICgoMVVMIDw8IGN1cl9vcmRlcikg
LSAxKSk7CisgICAgfQorCiAgICAgaWYgKCBtZm5fdmFsaWQoX21mbihtZm4p
KSApCiAgICAgewogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwg
cGFnZV9vcmRlcik7IGkrKyApCiAgICAgICAgIHsKLSAgICAgICAgICAgIG1m
bl9yZXR1cm4gPSBwMm0tPmdldF9lbnRyeShwMm0sIGdmbl9hZGQoZ2ZuLCBp
KSwgJnQsICZhLCAwLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIE5VTEwsIE5VTEwpOworICAgICAgICAgICAgcDJtLT5nZXRf
ZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZ0LCAmYSwgMCwgTlVMTCwg
TlVMTCk7CiAgICAgICAgICAgICBpZiAoICFwMm1faXNfZ3JhbnQodCkgJiYg
IXAybV9pc19zaGFyZWQodCkgJiYgIXAybV9pc19mb3JlaWduKHQpICkKICAg
ICAgICAgICAgICAgICBzZXRfZ3Bmbl9mcm9tX21mbihtZm4raSwgSU5WQUxJ
RF9NMlBfRU5UUlkpOwotICAgICAgICAgICAgQVNTRVJUKCAhcDJtX2lzX3Zh
bGlkKHQpIHx8IG1mbiArIGkgPT0gbWZuX3gobWZuX3JldHVybikgKTsKICAg
ICAgICAgfQogICAgIH0KICAgICByZXR1cm4gcDJtX3NldF9lbnRyeShwMm0s
IGdmbiwgSU5WQUxJRF9NRk4sIHBhZ2Vfb3JkZXIsIHAybV9pbnZhbGlkLAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9p
b21tdV9hY3BpLmMKQEAgLTk4LDEyICs5OCwyMSBAQCBzdGF0aWMgc3RydWN0
IGFtZF9pb21tdSAqIF9faW5pdCBmaW5kX2lvCiAgICAgcmV0dXJuIE5VTEw7
CiB9CiAKLXN0YXRpYyB2b2lkIF9faW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZSgKLSAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwgdWlu
dDY0X3QgYmFzZSwgdWludDY0X3QgbGltaXQpCitzdGF0aWMgaW50IF9faW5p
dCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZSgKKyAgICBzdHJ1Y3Qg
YW1kX2lvbW11ICppb21tdSwgdWludDY0X3QgYmFzZSwgdWludDY0X3QgbGlt
aXQsCisgICAgYm9vbCBhbGwsIGJvb2wgaXcsIGJvb2wgaXIpCiB7CisgICAg
aWYgKCAhaXIgfHwgIWl3ICkKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKKwog
ICAgIC8qIG5lZWQgdG8gZXh0ZW5kIGV4Y2x1c2lvbiByYW5nZT8gKi8KICAg
ICBpZiAoIGlvbW11LT5leGNsdXNpb25fZW5hYmxlICkKICAgICB7CisgICAg
ICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9saW1pdCArIFBBR0VfU0laRSA8
IGJhc2UgfHwKKyAgICAgICAgICAgICBsaW1pdCArIFBBR0VfU0laRSA8IGlv
bW11LT5leGNsdXNpb25fYmFzZSB8fAorICAgICAgICAgICAgIGlvbW11LT5l
eGNsdXNpb25fYWxsb3dfYWxsICE9IGFsbCApCisgICAgICAgICAgICByZXR1
cm4gLUVCVVNZOworCiAgICAgICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9i
YXNlIDwgYmFzZSApCiAgICAgICAgICAgICBiYXNlID0gaW9tbXUtPmV4Y2x1
c2lvbl9iYXNlOwogICAgICAgICBpZiAoIGlvbW11LT5leGNsdXNpb25fbGlt
aXQgPiBsaW1pdCApCkBAIC0xMTEsMTYgKzEyMCwxMSBAQCBzdGF0aWMgdm9p
ZCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpCiAgICAgfQogCiAgICAg
aW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgPSBJT01NVV9DT05UUk9MX0VOQUJM
RUQ7CisgICAgaW9tbXUtPmV4Y2x1c2lvbl9hbGxvd19hbGwgPSBhbGw7CiAg
ICAgaW9tbXUtPmV4Y2x1c2lvbl9iYXNlID0gYmFzZTsKICAgICBpb21tdS0+
ZXhjbHVzaW9uX2xpbWl0ID0gbGltaXQ7Ci19CiAKLXN0YXRpYyB2b2lkIF9f
aW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9hbGwoCi0gICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXUsCi0gICAgdW5zaWduZWQgbG9uZyBi
YXNlLCB1bnNpZ25lZCBsb25nIGxpbWl0KQotewotICAgIHJlc2VydmVfaW9t
bXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCk7Ci0gICAg
aW9tbXUtPmV4Y2x1c2lvbl9hbGxvd19hbGwgPSBJT01NVV9DT05UUk9MX0VO
QUJMRUQ7CisgICAgcmV0dXJuIDA7CiB9CiAKIHN0YXRpYyB2b2lkIF9faW5p
dCByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKApAQCAtMTU4LDYgKzE2
Miw3IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW4KICAgICB1bnNpZ25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBs
ZW5ndGg7CiAgICAgc3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgdW5z
aWduZWQgaW50IGJkZjsKKyAgICBpbnQgcmMgPSAwOwogCiAgICAgLyogaXMg
cGFydCBvZiBleGNsdXNpb24gcmFuZ2UgaW5zaWRlIG9mIElPTU1VIHZpcnR1
YWwgYWRkcmVzcyBzcGFjZT8gKi8KICAgICAvKiBub3RlOiAnbGltaXQnIHBh
cmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwpAQCAt
MTc5LDEwICsxODQsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJf
ZXhjbHVzaW9uX3JhbgogICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkK
ICAgICB7CiAgICAgICAgIGZvcl9lYWNoX2FtZF9pb21tdSggaW9tbXUgKQot
ICAgICAgICAgICAgcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2VfYWxs
KGlvbW11LCBiYXNlLCBsaW1pdCk7CisgICAgICAgIHsKKyAgICAgICAgICAg
IHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJh
c2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB0cnVlIC8qIGFsbCAqLywgaXcsIGlyKTsKKyAgICAg
ICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAgICAgIGJyZWFrOworICAg
ICAgICB9CiAgICAgfQogCi0gICAgcmV0dXJuIDA7CisgICAgcmV0dXJuIHJj
OwogfQogCiBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25f
cmFuZ2VfZm9yX2RldmljZSgKQEAgLTE5Myw2ICsyMDMsNyBAQCBzdGF0aWMg
aW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgdW5zaWdu
ZWQgbG9uZyByYW5nZV90b3AsIGlvbW11X3RvcCwgbGVuZ3RoOwogICAgIHN0
cnVjdCBhbWRfaW9tbXUgKmlvbW11OwogICAgIHUxNiByZXE7CisgICAgaW50
IHJjID0gMDsKIAogICAgIGlvbW11ID0gZmluZF9pb21tdV9mb3JfZGV2aWNl
KHNlZywgYmRmKTsKICAgICBpZiAoICFpb21tdSApCkBAIC0yMjIsMTIgKzIz
MywxMyBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25f
cmFuCiAgICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNl
dHRpbmdzIGZvciBkZXZpY2UgKi8KICAgICBpZiAoIGxpbWl0ID49IGlvbW11
X3RvcCAgKQogICAgIHsKLSAgICAgICAgcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0KTsKKyAgICAgICAgcmMgPSBy
ZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGlt
aXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwogICAgICAgICBpdnJzX21h
cHBpbmdzW2JkZl0uZHRlX2FsbG93X2V4Y2x1c2lvbiA9IElPTU1VX0NPTlRS
T0xfRU5BQkxFRDsKICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFdLmR0ZV9h
bGxvd19leGNsdXNpb24gPSBJT01NVV9DT05UUk9MX0VOQUJMRUQ7CiAgICAg
fQogCi0gICAgcmV0dXJuIDA7CisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0
aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2lv
bW11X2RldmljZXMoCkBAIC0yMzcsNiArMjQ5LDcgQEAgc3RhdGljIGludCBf
X2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgIHVuc2lnbmVkIGxv
bmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAgICB1bnNpZ25l
ZCBpbnQgYmRmOwogICAgIHUxNiByZXE7CisgICAgaW50IHJjID0gMDsKIAog
ICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJ
T01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICovCiAgICAgLyogbm90ZTog
J2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWdu
ZWQgKi8KQEAgLTI2Nyw4ICsyODAsMTAgQEAgc3RhdGljIGludCBfX2luaXQg
cmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogCiAgICAgLyogcmVnaXN0ZXIgSU9N
TVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzICovCiAgICAgaWYgKCBsaW1p
dCA+PSBpb21tdV90b3AgKQotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZV9hbGwoaW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAgICByZXR1
cm4gMDsKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9y
YW5nZShpb21tdSwgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3LCBp
cik7CisKKyAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0
IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdCgK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdo
L2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtMTI3LDMyICsxMjcsNDggQEAgc3Rh
dGljIGludCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpbwogICAgIHJl
dHVybiAwOwogfQogCi1zdGF0aWMgdm9pZCBfX2luaXQgcmVzZXJ2ZV91bml0
eV9tYXBfZm9yX2RldmljZSgKLSAgICB1MTYgc2VnLCB1MTYgYmRmLCB1bnNp
Z25lZCBsb25nIGJhc2UsCi0gICAgdW5zaWduZWQgbG9uZyBsZW5ndGgsIHU4
IGl3LCB1OCBpcikKK3N0YXRpYyBpbnQgX19pbml0IHJlc2VydmVfdW5pdHlf
bWFwX2Zvcl9kZXZpY2UoCisgICAgdWludDE2X3Qgc2VnLCB1aW50MTZfdCBi
ZGYsIHVuc2lnbmVkIGxvbmcgYmFzZSwKKyAgICB1bnNpZ25lZCBsb25nIGxl
bmd0aCwgYm9vbCBpdywgYm9vbCBpcikKIHsKICAgICBzdHJ1Y3QgaXZyc19t
YXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHNl
Zyk7Ci0gICAgdW5zaWduZWQgbG9uZyBvbGRfdG9wLCBuZXdfdG9wOworICAg
IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwID0gaXZyc19tYXBw
aW5nc1tiZGZdLnVuaXR5X21hcDsKIAotICAgIC8qIG5lZWQgdG8gZXh0ZW5k
IHVuaXR5LW1hcHBlZCByYW5nZT8gKi8KLSAgICBpZiAoIGl2cnNfbWFwcGlu
Z3NbYmRmXS51bml0eV9tYXBfZW5hYmxlICkKKyAgICAvKiBDaGVjayBmb3Ig
b3ZlcmxhcHMuICovCisgICAgZm9yICggOyB1bml0eV9tYXA7IHVuaXR5X21h
cCA9IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewotICAgICAgICBvbGRfdG9w
ID0gaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQgKwotICAg
ICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2VfbGVuZ3Ro
OwotICAgICAgICBuZXdfdG9wID0gYmFzZSArIGxlbmd0aDsKLSAgICAgICAg
aWYgKCBvbGRfdG9wID4gbmV3X3RvcCApCi0gICAgICAgICAgICBuZXdfdG9w
ID0gb2xkX3RvcDsKLSAgICAgICAgaWYgKCBpdnJzX21hcHBpbmdzW2JkZl0u
YWRkcl9yYW5nZV9zdGFydCA8IGJhc2UgKQotICAgICAgICAgICAgYmFzZSA9
IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3JhbmdlX3N0YXJ0OwotICAgICAg
ICBsZW5ndGggPSBuZXdfdG9wIC0gYmFzZTsKLSAgICB9Ci0KLSAgICAvKiBl
eHRlbmQgci93IHBlcm1pc3Npb21zIGFuZCBrZWVwIGFnZ3JlZ2F0ZSAqLwot
ICAgIGl2cnNfbWFwcGluZ3NbYmRmXS53cml0ZV9wZXJtaXNzaW9uID0gaXc7
Ci0gICAgaXZyc19tYXBwaW5nc1tiZGZdLnJlYWRfcGVybWlzc2lvbiA9IGly
OwotICAgIGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXBfZW5hYmxlID0g
SU9NTVVfQ09OVFJPTF9FTkFCTEVEOwotICAgIGl2cnNfbWFwcGluZ3NbYmRm
XS5hZGRyX3JhbmdlX3N0YXJ0ID0gYmFzZTsKLSAgICBpdnJzX21hcHBpbmdz
W2JkZl0uYWRkcl9yYW5nZV9sZW5ndGggPSBsZW5ndGg7CisgICAgICAgIC8q
CisgICAgICAgICAqIEV4YWN0IG1hdGNoZXMgYXJlIG9rYXkuIFRoaXMgY2Fu
IGluIHBhcnRpY3VsYXIgaGFwcGVuIHdoZW4KKyAgICAgICAgICogcmVnaXN0
ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3
aWNlIGZvciB0aGUKKyAgICAgICAgICogc2FtZSAocyxiLGQsZikuCisgICAg
ICAgICAqLworICAgICAgICBpZiAoIGJhc2UgPT0gdW5pdHlfbWFwLT5hZGRy
ICYmIGxlbmd0aCA9PSB1bml0eV9tYXAtPmxlbmd0aCAmJgorICAgICAgICAg
ICAgIGlyID09IHVuaXR5X21hcC0+cmVhZCAmJiBpdyA9PSB1bml0eV9tYXAt
PndyaXRlICkKKyAgICAgICAgICAgIHJldHVybiAwOworCisgICAgICAgIGlm
ICggdW5pdHlfbWFwLT5hZGRyICsgdW5pdHlfbWFwLT5sZW5ndGggPiBiYXNl
ICYmCisgICAgICAgICAgICAgYmFzZSArIGxlbmd0aCA+IHVuaXR5X21hcC0+
YWRkciApCisgICAgICAgIHsKKyAgICAgICAgICAgIEFNRF9JT01NVV9ERUJV
RygiSVZNRCBFcnJvcjogb3ZlcmxhcCBbJWx4LCVseCkgdnMgWyVseCwlbHgp
XG4iLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJhc2UsIGJhc2Ug
KyBsZW5ndGgsIHVuaXR5X21hcC0+YWRkciwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB1bml0eV9tYXAtPmFkZHIgKyB1bml0eV9tYXAtPmxlbmd0
aCk7CisgICAgICAgICAgICByZXR1cm4gLUVQRVJNOworICAgICAgICB9Cisg
ICAgfQorCisgICAgLyogUG9wdWxhdGUgYW5kIGluc2VydCBhIG5ldyB1bml0
eSBtYXAuICovCisgICAgdW5pdHlfbWFwID0geG1hbGxvYyhzdHJ1Y3QgaXZy
c191bml0eV9tYXApOworICAgIGlmICggIXVuaXR5X21hcCApCisgICAgICAg
IHJldHVybiAtRU5PTUVNOworCisgICAgdW5pdHlfbWFwLT5yZWFkID0gaXI7
CisgICAgdW5pdHlfbWFwLT53cml0ZSA9IGl3OworICAgIHVuaXR5X21hcC0+
YWRkciA9IGJhc2U7CisgICAgdW5pdHlfbWFwLT5sZW5ndGggPSBsZW5ndGg7
CisgICAgdW5pdHlfbWFwLT5uZXh0ID0gaXZyc19tYXBwaW5nc1tiZGZdLnVu
aXR5X21hcDsKKyAgICBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwID0g
dW5pdHlfbWFwOworCisgICAgcmV0dXJuIDA7CiB9CiAKIHN0YXRpYyBpbnQg
X19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfYWxsX2Rldmlj
ZXMoCkBAIC0xNzUsMTMgKzE5MSwxMyBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgICAgIGxlbmd0aCA9IHJhbmdl
X3RvcCAtIGJhc2U7CiAgICAgICAgIC8qIHJlc2VydmUgci93IHVuaXR5LW1h
cHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMgKi8KICAgICAgICAgLyog
bm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9u
IHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9IDA7IGJkZiA8IGl2cnNf
YmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAgICAgICAgIHJlc2VydmVfdW5p
dHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcs
IGlyKTsKKyAgICAgICAgZm9yICggYmRmID0gMDsgIXJjICYmIGJkZiA8IGl2
cnNfYmRmX2VudHJpZXM7IGJkZisrICkKKyAgICAgICAgICAgIHJjID0gcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVu
Z3RoLCBpdywgaXIpOwogICAgICAgICAvKiBwdXNoICdiYXNlJyBqdXN0IG91
dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAgICAgIGJh
c2UgPSBpb21tdV90b3A7CiAgICAgfQogICAgIC8qIHJlZ2lzdGVyIElPTU1V
IGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICggbGltaXQg
Pj0gaW9tbXVfdG9wICkKKyAgICBpZiAoICFyYyAmJiBsaW1pdCA+PSBpb21t
dV90b3AgKQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBp
b21tdSApCiAgICAgICAgIHsKQEAgLTIyMywxNSArMjM5LDE1IEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICAgICAg
bGVuZ3RoID0gcmFuZ2VfdG9wIC0gYmFzZTsKICAgICAgICAgLyogcmVzZXJ2
ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2UgKi8KICAg
ICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUg
ZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsK
LSAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIHJl
cSwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOworICAgICAgICByYyA9IHJlc2Vy
dmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0
aCwgaXcsIGlyKSA/OgorICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsK
IAogICAgICAgICAvKiBwdXNoICdiYXNlJyBqdXN0IG91dHNpZGUgb2Ygdmly
dHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAgICAgIGJhc2UgPSBpb21tdV90
b3A7CiAgICAgfQogCiAgICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9u
IHJhbmdlIHNldHRpbmdzIGZvciBkZXZpY2UgKi8KLSAgICBpZiAoIGxpbWl0
ID49IGlvbW11X3RvcCAgKQorICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlv
bW11X3RvcCAgKQogICAgIHsKICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UgLyog
YWxsICovLCBpdywgaXIpOwpAQCAtMjYyLDE1ICsyNzgsMTUgQEAgc3RhdGlj
IGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbgogICAgICAgICBs
ZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwogICAgICAgICAvKiByZXNlcnZl
IHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICov
CiAgICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJlIHBhcnQgb2Yg
dGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAgICBmb3IgKCBiZGYgPSAw
OyBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCisgICAgICAgIGZv
ciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBi
ZGYrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIGlmICggaW9tbXUgPT0g
ZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQogICAg
ICAgICAgICAgewotICAgICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFw
X2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBp
dywgaXIpOwogICAgICAgICAgICAgICAgIHJlcSA9IGdldF9pdnJzX21hcHBp
bmdzKGlvbW11LT5zZWcpW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZDsKLSAgICAg
ICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11
LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAotICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyKTsKKyAgICAgICAg
ICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcikgPzoK
KyAgICAgICAgICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9k
ZXZpY2UoaW9tbXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCisgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3
LCBpcik7CiAgICAgICAgICAgICB9CiAgICAgICAgIH0KIApAQCAtMjc5LDcg
KzI5NSw3IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW4KICAgICB9CiAKICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNp
b24gcmFuZ2Ugc2V0dGluZ3MgKi8KLSAgICBpZiAoIGxpbWl0ID49IGlvbW11
X3RvcCApCisgICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkK
ICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShp
b21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3LCBpcik7CiAK
LS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2luaXQu
YworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXVfaW5p
dC5jCkBAIC0xMTg5LDcgKzExODksNiBAQCBzdGF0aWMgaW50IF9faW5pdCBh
bGxvY19pdnJzX21hcHBpbmdzKHUxCiAgICAgewogICAgICAgICBpdnJzX21h
cHBpbmdzW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZCA9IGJkZjsKICAgICAgICAg
aXZyc19tYXBwaW5nc1tiZGZdLmR0ZV9hbGxvd19leGNsdXNpb24gPSBJT01N
VV9DT05UUk9MX0RJU0FCTEVEOwotICAgICAgICBpdnJzX21hcHBpbmdzW2Jk
Zl0udW5pdHlfbWFwX2VuYWJsZSA9IElPTU1VX0NPTlRST0xfRElTQUJMRUQ7
CiAgICAgICAgIGl2cnNfbWFwcGluZ3NbYmRmXS5pb21tdSA9IE5VTEw7CiAK
ICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmludHJlbWFwX3RhYmxlID0g
TlVMTDsKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9h
bWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQv
cGNpX2FtZF9pb21tdS5jCkBAIC0zNDYsMTUgKzM0NiwxNyBAQCBzdGF0aWMg
aW50IGFtZF9pb21tdV9hc3NpZ25fZGV2aWNlKHN0cnVjCiAgICAgc3RydWN0
IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBw
aW5ncyhwZGV2LT5zZWcpOwogICAgIGludCBiZGYgPSBQQ0lfQkRGMihwZGV2
LT5idXMsIGRldmZuKTsKICAgICBpbnQgcmVxX2lkID0gZ2V0X2RtYV9yZXF1
ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYpOworICAgIGNvbnN0IHN0cnVjdCBp
dnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwOwogCi0gICAgaWYgKCBpdnJzX21h
cHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwX2VuYWJsZSApCisgICAgZm9yICgg
dW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcDsg
dW5pdHlfbWFwOworICAgICAgICAgIHVuaXR5X21hcCA9IHVuaXR5X21hcC0+
bmV4dCApCiAgICAgewotICAgICAgICBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21h
aW5fdW5pdHlfbWFwKAotICAgICAgICAgICAgZCwKLSAgICAgICAgICAgIGl2
cnNfbWFwcGluZ3NbcmVxX2lkXS5hZGRyX3JhbmdlX3N0YXJ0LAotICAgICAg
ICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLmFkZHJfcmFuZ2VfbGVuZ3Ro
LAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLndyaXRlX3Bl
cm1pc3Npb24sCi0gICAgICAgICAgICBpdnJzX21hcHBpbmdzW3JlcV9pZF0u
cmVhZF9wZXJtaXNzaW9uKTsKKyAgICAgICAgaW50IHJjID0gYW1kX2lvbW11
X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAgICAgICAgICAgICAg
ICAgIGQsIHVuaXR5X21hcC0+YWRkciwgdW5pdHlfbWFwLT5sZW5ndGgsCisg
ICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPndyaXRlLCB1bml0eV9t
YXAtPnJlYWQpOworCisgICAgICAgIGlmICggcmMgKQorICAgICAgICAgICAg
cmV0dXJuIHJjOwogICAgIH0KIAogICAgIHJldHVybiByZWFzc2lnbl9kZXZp
Y2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwgcGRldik7Ci0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvYW1kLWlvbW11LmgKKysrIGIveGVuL2luY2x1ZGUv
YXNtLXg4Ni9hbWQtaW9tbXUuaApAQCAtMTA4LDE1ICsxMDgsMTkgQEAgc3Ry
dWN0IGFtZF9pb21tdSB7CiAgICAgc3RydWN0IGxpc3RfaGVhZCBhdHNfZGV2
aWNlczsKIH07CiAKK3N0cnVjdCBpdnJzX3VuaXR5X21hcCB7CisgICAgYm9v
bCByZWFkOjE7CisgICAgYm9vbCB3cml0ZToxOworICAgIHBhZGRyX3QgYWRk
cjsKKyAgICB1bnNpZ25lZCBsb25nIGxlbmd0aDsKKyAgICBzdHJ1Y3QgaXZy
c191bml0eV9tYXAgKm5leHQ7Cit9OworCiBzdHJ1Y3QgaXZyc19tYXBwaW5n
cyB7CiAgICAgdTE2IGR0ZV9yZXF1ZXN0b3JfaWQ7CiAgICAgdTggZHRlX2Fs
bG93X2V4Y2x1c2lvbjsKLSAgICB1OCB1bml0eV9tYXBfZW5hYmxlOwotICAg
IHU4IHdyaXRlX3Blcm1pc3Npb247Ci0gICAgdTggcmVhZF9wZXJtaXNzaW9u
OwotICAgIHVuc2lnbmVkIGxvbmcgYWRkcl9yYW5nZV9zdGFydDsKLSAgICB1
bnNpZ25lZCBsb25nIGFkZHJfcmFuZ2VfbGVuZ3RoOwogICAgIHN0cnVjdCBh
bWRfaW9tbXUgKmlvbW11OworICAgIHN0cnVjdCBpdnJzX3VuaXR5X21hcCAq
dW5pdHlfbWFwOwogCiAgICAgLyogcGVyIGRldmljZSBpbnRlcnJ1cHQgcmVt
YXBwaW5nIHRhYmxlICovCiAgICAgdm9pZCAqaW50cmVtYXBfdGFibGU7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0tZXB0
LmMKQEAgLTY3Miw3ICs2NzIsNyBAQCBlcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmCiAgICAgdWludDhfdCBpcGF0ID0gMDsKICAg
ICBib29sX3QgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMTsKICAgICBib29s
X3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsKLSAgICB1bnNpZ25lZCBpbnQgaW9t
bXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIG1mbik7Cisg
ICAgdW5zaWduZWQgaW50IGlvbW11X2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBwMm1hLCBtZm4pOwogICAgIGJvb2xfdCBuZWVkc19zeW5j
ID0gMTsKICAgICBlcHRfZW50cnlfdCBvbGRfZW50cnkgPSB7IC5lcHRlID0g
MCB9OwogICAgIGVwdF9lbnRyeV90IG5ld19lbnRyeSA9IHsgLmVwdGUgPSAw
IH07CkBAIC03OTksOCArNzk5LDggQEAgZXB0X3NldF9lbnRyeShzdHJ1Y3Qg
cDJtX2RvbWFpbiAqcDJtLCBnZgogCiAgICAgICAgIC8qIFNhZmUgdG8gcmVh
ZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2UgaG9sZCB0aGUgcDJtIGxvY2sgKi8K
ICAgICAgICAgaWYgKCBlcHRfZW50cnktPm1mbiA9PSBuZXdfZW50cnkubWZu
ICYmCi0gICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhlcHRfZW50
cnktPnNhX3AybXQsIF9tZm4oZXB0X2VudHJ5LT5tZm4pKSA9PQotICAgICAg
ICAgICAgIGlvbW11X2ZsYWdzICkKKyAgICAgICAgICAgICBwMm1fZ2V0X2lv
bW11X2ZsYWdzKGVwdF9lbnRyeS0+c2FfcDJtdCwgZXB0X2VudHJ5LT5hY2Nl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBfbWZuKGVw
dF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVfZmxhZ3MgKQogICAgICAgICAgICAg
bmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMDsKIAogICAgICAgICBlcHRfcDJt
X3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3X2VudHJ5LCBwMm10LCBwMm1hKTsK
LS0tIGEveGVuL2FyY2gveDg2L21tL3AybS1wdC5jCisrKyBiL3hlbi9hcmNo
L3g4Ni9tbS9wMm0tcHQuYwpAQCAtNTEyLDYgKzUxMiwxNiBAQCBpbnQgcDJt
X3B0X2hhbmRsZV9kZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAgICAgcmV0dXJu
IHJjOwogfQogCisvKiBSZWNvbnN0cnVjdCBhIGZha2UgcDJtX2FjY2Vzc190
IGZyb20gc3RvcmVkIFBURSBmbGFncy4gKi8KK3N0YXRpYyBwMm1fYWNjZXNz
X3QgcDJtX2ZsYWdzX3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQgZmxhZ3MpCit7
CisgICAgaWYgKCBmbGFncyAmIF9QQUdFX1BSRVNFTlQgKQorICAgICAgICBy
ZXR1cm4gcDJtX2FjY2Vzc19uOworCisgICAgLyogTm8gbmVlZCB0byBsb29r
IGF0IF9QQUdFX05YIGZvciBub3cuICovCisgICAgcmV0dXJuIGZsYWdzICYg
X1BBR0VfUlcgPyBwMm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vzc19yOworfQor
CiAvKiBDaGVja3Mgb25seSBhcHBsaWNhYmxlIHRvIGVudHJpZXMgd2l0aCBv
cmRlciA+IFBBR0VfT1JERVJfNEsgKi8KIHN0YXRpYyB2b2lkIGNoZWNrX2Vu
dHJ5KG1mbl90IG1mbiwgcDJtX3R5cGVfdCBuZXcsIHAybV90eXBlX3Qgb2xk
LAogICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVy
KQpAQCAtNTQ2LDcgKzU1Niw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0
IHAybV9kb21haW4gKnAybSwKICAgICBsMl9wZ2VudHJ5X3QgbDJlX2NvbnRl
bnQ7CiAgICAgbDNfcGdlbnRyeV90IGwzZV9jb250ZW50OwogICAgIGludCBy
YzsKLSAgICB1bnNpZ25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdzID0gcDJtX2dl
dF9pb21tdV9mbGFncyhwMm10LCBtZm4pOworICAgIHVuc2lnbmVkIGludCBp
b21tdV9wdGVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIHAy
bWEsIG1mbik7CiAgICAgLyoKICAgICAgKiBvbGRfbWZuIGFuZCBpb21tdV9v
bGRfZmxhZ3MgY29udHJvbCBwb3NzaWJsZSBmbHVzaC91cGRhdGUgbmVlZHMg
b24gdGhlCiAgICAgICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1c2ggd2hlbiBN
Rk4gb3IgZmxhZ3MgKGkuZS4gcGVybWlzc2lvbnMpIGNoYW5nZS4KQEAgLTYw
OSw2ICs2MTksNyBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9t
YWluICpwMm0sCiAgICAgICAgICAgICAgICAgb2xkX21mbiA9IGwxZV9nZXRf
cGZuKCpwMm1fZW50cnkpOwogICAgICAgICAgICAgICAgIGlvbW11X29sZF9m
bGFncyA9CiAgICAgICAgICAgICAgICAgICAgIHAybV9nZXRfaW9tbXVfZmxh
Z3MocDJtX2ZsYWdzX3RvX3R5cGUoZmxhZ3MpLAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9mbGFnc190b19hY2Nlc3Mo
ZmxhZ3MpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF9tZm4ob2xkX21mbikpOwogICAgICAgICAgICAgfQogICAgICAgICAg
ICAgZWxzZQpAQCAtNjU0LDkgKzY2NSwxMCBAQCBwMm1fcHRfc2V0X2VudHJ5
KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRSSUVTKTsKICAg
ICAgICAgQVNTRVJUKHAybV9lbnRyeSk7CiAgICAgICAgIG9sZF9tZm4gPSBs
MWVfZ2V0X3BmbigqcDJtX2VudHJ5KTsKKyAgICAgICAgZmxhZ3MgPSBsMWVf
Z2V0X2ZsYWdzKCpwMm1fZW50cnkpOwogICAgICAgICBpb21tdV9vbGRfZmxh
Z3MgPQotICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxh
Z3NfdG9fdHlwZShsMWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpKSwKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7Cisg
ICAgICAgICAgICBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9mbGFnc190b190
eXBlKGZsYWdzKSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
cDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksIF9tZm4ob2xkX21mbikpOwog
CiAgICAgICAgIGlmICggbWZuX3ZhbGlkKG1mbikgfHwgcDJtX2FsbG93c19p
bnZhbGlkX21mbihwMm10KSApCiAgICAgICAgICAgICBlbnRyeV9jb250ZW50
ID0gcDJtX2wxZV9mcm9tX3BmbihtZm5feChtZm4pLApAQCAtNjg3LDYgKzY5
OSw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAy
bSwKICAgICAgICAgICAgICAgICBvbGRfbWZuID0gbDFlX2dldF9wZm4oKnAy
bV9lbnRyeSk7CiAgICAgICAgICAgICAgICAgaW9tbXVfb2xkX2ZsYWdzID0K
ICAgICAgICAgICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1f
ZmxhZ3NfdG9fdHlwZShmbGFncyksCisgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyks
CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21m
bihvbGRfbWZuKSk7CiAgICAgICAgICAgICB9CiAgICAgICAgICAgICBlbHNl
Ci0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2lu
Y2x1ZGUvYXNtLXg4Ni9wMm0uaApAQCAtODk5LDcgKzg5OSw4IEBAIHN0YXRp
YyBpbmxpbmUgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnUKIC8qCiAgKiBw
Mm0gdHlwZSB0byBJT01NVSBmbGFncwogICovCi1zdGF0aWMgaW5saW5lIHVu
c2lnbmVkIGludCBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90eXBlX3QgcDJt
dCwgbWZuX3QgbWZuKQorc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJt
X2dldF9pb21tdV9mbGFncyhwMm1fdHlwZV90IHAybXQsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9hY2Nl
c3NfdCBwMm1hLCBtZm5fdCBtZm4pCiB7CiAgICAgdW5zaWduZWQgaW50IGZs
YWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC0x
MzQxLDcgKzEzNDEsNyBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShz
dHJ1Y3QgZG9tYWluCiAgICAgICAgIGlmICggIWhhc19pb21tdV9wdChkKSAp
CiAgICAgICAgICAgICByZXR1cm4gMDsKICAgICAgICAgcmV0dXJuIGlvbW11
X2xlZ2FjeV9tYXAoZCwgX2RmbihnZm5fbCksIF9tZm4oZ2ZuX2wpLCBQQUdF
X09SREVSXzRLLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBJ
T01NVUZfcmVhZGFibGUgfCBJT01NVUZfd3JpdGFibGUpOworICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBwMm1fYWNjZXNzX3RvX2lvbW11X2Zs
YWdzKHAybWEpKTsKICAgICB9CiAKICAgICBnZm5fbG9jayhwMm0sIGdmbiwg
MCk7Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5j
CisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jCkBA
IC00MiwxMiArNDIsNiBAQAogI2luY2x1ZGUgInZ0ZC5oIgogI2luY2x1ZGUg
Ii4uL2F0cy5oIgogCi1zdHJ1Y3QgbWFwcGVkX3JtcnIgewotICAgIHN0cnVj
dCBsaXN0X2hlYWQgbGlzdDsKLSAgICB1NjQgYmFzZSwgZW5kOwotICAgIHVu
c2lnbmVkIGludCBjb3VudDsKLX07Ci0KIC8qIFBvc3NpYmxlIHVuZmlsdGVy
ZWQgTEFQSUMvTVNJIG1lc3NhZ2VzIGZyb20gdW50cnVzdGVkIHNvdXJjZXM/
ICovCiBib29sIF9fcmVhZF9tb3N0bHkgdW50cnVzdGVkX21zaTsKIApAQCAt
MTc4NywxNiArMTc4MSwxMSBAQCBvdXQ6CiBzdGF0aWMgdm9pZCBpb21tdV9k
b21haW5fdGVhcmRvd24oc3RydWN0IGRvbWFpbiAqZCkKIHsKICAgICBzdHJ1
Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRvbV9pb21tdShkKTsKLSAgICBzdHJ1
Y3QgbWFwcGVkX3JtcnIgKm1ybXJyLCAqdG1wOwogCiAgICAgaWYgKCBsaXN0
X2VtcHR5KCZhY3BpX2RyaGRfdW5pdHMpICkKICAgICAgICAgcmV0dXJuOwog
Ci0gICAgbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlICggbXJtcnIsIHRtcCwg
JmhkLT5hcmNoLm1hcHBlZF9ybXJycywgbGlzdCApCi0gICAgewotICAgICAg
ICBsaXN0X2RlbCgmbXJtcnItPmxpc3QpOwotICAgICAgICB4ZnJlZShtcm1y
cik7Ci0gICAgfQorICAgIGlvbW11X2lkZW50aXR5X21hcF90ZWFyZG93bihk
KTsKIAogICAgIEFTU0VSVChpb21tdV9lbmFibGVkKTsKIApAQCAtMTk1NSw3
NCArMTk0NCw2IEBAIHN0YXRpYyB2b2lkIGlvbW11X3NldF9wZ2Qoc3RydWN0
IGRvbWFpbgogICAgICAgICBwYWdldGFibGVfZ2V0X3BhZGRyKHBhZ2V0YWJs
ZV9mcm9tX21mbihwZ2RfbWZuKSk7CiB9CiAKLXN0YXRpYyBpbnQgcm1ycl9p
ZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsIGJvb2xfdCBtYXAs
Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1
Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnIsCi0gICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1MzIgZmxhZykKLXsKLSAgICB1bnNpZ25lZCBsb25n
IGJhc2VfcGZuID0gcm1yci0+YmFzZV9hZGRyZXNzID4+IFBBR0VfU0hJRlRf
NEs7Ci0gICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuID0gUEFHRV9BTElHTl80
SyhybXJyLT5lbmRfYWRkcmVzcykgPj4gUEFHRV9TSElGVF80SzsKLSAgICBz
dHJ1Y3QgbWFwcGVkX3JtcnIgKm1ybXJyOwotICAgIHN0cnVjdCBkb21haW5f
aW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOwotCi0gICAgQVNTRVJUKHBjaWRl
dnNfbG9ja2VkKCkpOwotICAgIEFTU0VSVChybXJyLT5iYXNlX2FkZHJlc3Mg
PCBybXJyLT5lbmRfYWRkcmVzcyk7Ci0KLSAgICAvKgotICAgICAqIE5vIG5l
ZWQgdG8gYWNxdWlyZSBoZC0+YXJjaC5tYXBwaW5nX2xvY2s6IEJvdGggaW5z
ZXJ0aW9uIGFuZCByZW1vdmFsCi0gICAgICogZ2V0IGRvbmUgd2hpbGUgaG9s
ZGluZyBwY2lkZXZzX2xvY2suCi0gICAgICovCi0gICAgbGlzdF9mb3JfZWFj
aF9lbnRyeSggbXJtcnIsICZoZC0+YXJjaC5tYXBwZWRfcm1ycnMsIGxpc3Qg
KQotICAgIHsKLSAgICAgICAgaWYgKCBtcm1yci0+YmFzZSA9PSBybXJyLT5i
YXNlX2FkZHJlc3MgJiYKLSAgICAgICAgICAgICBtcm1yci0+ZW5kID09IHJt
cnItPmVuZF9hZGRyZXNzICkKLSAgICAgICAgewotICAgICAgICAgICAgaW50
IHJldCA9IDA7Ci0KLSAgICAgICAgICAgIGlmICggbWFwICkKLSAgICAgICAg
ICAgIHsKLSAgICAgICAgICAgICAgICArK21ybXJyLT5jb3VudDsKLSAgICAg
ICAgICAgICAgICByZXR1cm4gMDsKLSAgICAgICAgICAgIH0KLQotICAgICAg
ICAgICAgaWYgKCAtLW1ybXJyLT5jb3VudCApCi0gICAgICAgICAgICAgICAg
cmV0dXJuIDA7Ci0KLSAgICAgICAgICAgIHdoaWxlICggYmFzZV9wZm4gPCBl
bmRfcGZuICkKLSAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAgICBpZiAo
IGNsZWFyX2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3BmbikgKQotICAg
ICAgICAgICAgICAgICAgICByZXQgPSAtRU5YSU87Ci0gICAgICAgICAgICAg
ICAgYmFzZV9wZm4rKzsKLSAgICAgICAgICAgIH0KLQotICAgICAgICAgICAg
bGlzdF9kZWwoJm1ybXJyLT5saXN0KTsKLSAgICAgICAgICAgIHhmcmVlKG1y
bXJyKTsKLSAgICAgICAgICAgIHJldHVybiByZXQ7Ci0gICAgICAgIH0KLSAg
ICB9Ci0KLSAgICBpZiAoICFtYXAgKQotICAgICAgICByZXR1cm4gLUVOT0VO
VDsKLQotICAgIHdoaWxlICggYmFzZV9wZm4gPCBlbmRfcGZuICkKLSAgICB7
Ci0gICAgICAgIGludCBlcnIgPSBzZXRfaWRlbnRpdHlfcDJtX2VudHJ5KGQs
IGJhc2VfcGZuLCBwMm1fYWNjZXNzX3J3LCBmbGFnKTsKLQotICAgICAgICBp
ZiAoIGVyciApCi0gICAgICAgICAgICByZXR1cm4gZXJyOwotICAgICAgICBi
YXNlX3BmbisrOwotICAgIH0KLQotICAgIG1ybXJyID0geG1hbGxvYyhzdHJ1
Y3QgbWFwcGVkX3JtcnIpOwotICAgIGlmICggIW1ybXJyICkKLSAgICAgICAg
cmV0dXJuIC1FTk9NRU07Ci0gICAgbXJtcnItPmJhc2UgPSBybXJyLT5iYXNl
X2FkZHJlc3M7Ci0gICAgbXJtcnItPmVuZCA9IHJtcnItPmVuZF9hZGRyZXNz
OwotICAgIG1ybXJyLT5jb3VudCA9IDE7Ci0gICAgbGlzdF9hZGRfdGFpbCgm
bXJtcnItPmxpc3QsICZoZC0+YXJjaC5tYXBwZWRfcm1ycnMpOwotCi0gICAg
cmV0dXJuIDA7Ci19Ci0KIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfYWRkX2Rl
dmljZSh1OCBkZXZmbiwgc3RydWN0IHBjaV9kZXYgKnBkZXYpCiB7CiAgICAg
c3RydWN0IGFjcGlfcm1ycl91bml0ICpybXJyOwpAQCAtMjA1NCw3ICsxOTc1
LDkgQEAgc3RhdGljIGludCBpbnRlbF9pb21tdV9hZGRfZGV2aWNlKHU4IGRl
dgogICAgICAgICAgICAgICogU2luY2UgUk1SUnMgYXJlIGFsd2F5cyByZXNl
cnZlZCBpbiB0aGUgZTgyMCBtYXAgZm9yIHRoZSBoYXJkd2FyZQogICAgICAg
ICAgICAgICogZG9tYWluLCB0aGVyZSBzaG91bGRuJ3QgYmUgYSBjb25mbGlj
dC4KICAgICAgICAgICAgICAqLwotICAgICAgICAgICAgcmV0ID0gcm1ycl9p
ZGVudGl0eV9tYXBwaW5nKHBkZXYtPmRvbWFpbiwgMSwgcm1yciwgMCk7Cisg
ICAgICAgICAgICByZXQgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKHBkZXYt
PmRvbWFpbiwgcDJtX2FjY2Vzc19ydywKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgcm1yci0+YmFzZV9hZGRyZXNzLCBybXJy
LT5lbmRfYWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgMCk7CiAgICAgICAgICAgICBpZiAoIHJldCApCiAgICAg
ICAgICAgICAgICAgZHByaW50ayhYRU5MT0dfRVJSIFZURFBSRUZJWCwgImQl
ZDogUk1SUiBtYXBwaW5nIGZhaWxlZFxuIiwKICAgICAgICAgICAgICAgICAg
ICAgICAgIHBkZXYtPmRvbWFpbi0+ZG9tYWluX2lkKTsKQEAgLTIwOTksNyAr
MjAyMiw4IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfcmVtb3ZlX2Rldmlj
ZSh1OAogICAgICAgICAgKiBBbnkgZmxhZyBpcyBub3RoaW5nIHRvIGNsZWFy
IHRoZXNlIG1hcHBpbmdzIGJ1dCBoZXJlCiAgICAgICAgICAqIGl0cyBhbHdh
eXMgc2FmZSBhbmQgc3RyaWN0IHRvIHNldCAwLgogICAgICAgICAgKi8KLSAg
ICAgICAgcm1ycl9pZGVudGl0eV9tYXBwaW5nKHBkZXYtPmRvbWFpbiwgMCwg
cm1yciwgMCk7CisgICAgICAgIGlvbW11X2lkZW50aXR5X21hcHBpbmcocGRl
di0+ZG9tYWluLCBwMm1fYWNjZXNzX3gsIHJtcnItPmJhc2VfYWRkcmVzcywK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5lbmRfYWRk
cmVzcywgMCk7CiAgICAgfQogCiAgICAgcmV0dXJuIGRvbWFpbl9jb250ZXh0
X3VubWFwKHBkZXYtPmRvbWFpbiwgZGV2Zm4sIHBkZXYpOwpAQCAtMjI2Niw3
ICsyMTkwLDggQEAgc3RhdGljIHZvaWQgX19od2RvbV9pbml0IHNldHVwX2h3
ZG9tX3JtcgogICAgICAgICAgKiBkb21haW4sIHRoZXJlIHNob3VsZG4ndCBi
ZSBhIGNvbmZsaWN0LiBTbyBpdHMgYWx3YXlzIHNhZmUgYW5kCiAgICAgICAg
ICAqIHN0cmljdCB0byBzZXQgMC4KICAgICAgICAgICovCi0gICAgICAgIHJl
dCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhkLCAxLCBybXJyLCAwKTsKKyAg
ICAgICAgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhkLCBwMm1fYWNj
ZXNzX3J3LCBybXJyLT5iYXNlX2FkZHJlc3MsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgcm1yci0+ZW5kX2FkZHJlc3MsIDApOwog
ICAgICAgICBpZiAoIHJldCApCiAgICAgICAgICAgICBkcHJpbnRrKFhFTkxP
R19FUlIgVlREUFJFRklYLAogICAgICAgICAgICAgICAgICAgICAgIklPTU1V
OiBtYXBwaW5nIHJlc2VydmVkIHJlZ2lvbiBmYWlsZWRcbiIpOwpAQCAtMjQy
NSw3ICsyMzUwLDkgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Vfb3du
ZXJzaGlwKAogICAgICAgICAgICAgICAgICAqIEFueSBSTVJSIGZsYWcgaXMg
YWx3YXlzIGlnbm9yZWQgd2hlbiByZW1vdmUgYSBkZXZpY2UsCiAgICAgICAg
ICAgICAgICAgICogYnV0IGl0cyBhbHdheXMgc2FmZSBhbmQgc3RyaWN0IHRv
IHNldCAwLgogICAgICAgICAgICAgICAgICAqLwotICAgICAgICAgICAgICAg
IHJldCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhzb3VyY2UsIDAsIHJtcnIs
IDApOworICAgICAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50aXR5X21h
cHBpbmcoc291cmNlLCBwMm1fYWNjZXNzX3gsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5iYXNlX2FkZHJl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBybXJyLT5lbmRfYWRkcmVzcywgMCk7CiAgICAgICAgICAgICAgICAg
aWYgKCByZXQgIT0gLUVOT0VOVCApCiAgICAgICAgICAgICAgICAgICAgIHJl
dHVybiByZXQ7CiAgICAgICAgICAgICB9CkBAIC0yNTIyLDcgKzI0NDksOCBA
QCBzdGF0aWMgaW50IGludGVsX2lvbW11X2Fzc2lnbl9kZXZpY2UoCiAgICAg
ICAgICAgICAgUENJX0JVUyhiZGYpID09IGJ1cyAmJgogICAgICAgICAgICAg
IFBDSV9ERVZGTjIoYmRmKSA9PSBkZXZmbiApCiAgICAgICAgIHsKLSAgICAg
ICAgICAgIHJldCA9IHJtcnJfaWRlbnRpdHlfbWFwcGluZyhkLCAxLCBybXJy
LCBmbGFnKTsKKyAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50aXR5X21h
cHBpbmcoZCwgcDJtX2FjY2Vzc19ydywgcm1yci0+YmFzZV9hZGRyZXNzLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJy
LT5lbmRfYWRkcmVzcywgZmxhZyk7CiAgICAgICAgICAgICBpZiAoIHJldCAp
CiAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgICAgaW50IHJjOwotLS0g
YS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC94ODYvaW9tbXUuYworKysgYi94
ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC94ODYvaW9tbXUuYwpAQCAtMTUwLDcg
KzE1MCw3IEBAIGludCBhcmNoX2lvbW11X2RvbWFpbl9pbml0KHN0cnVjdCBk
b21haW4KICAgICBzdHJ1Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRvbV9pb21t
dShkKTsKIAogICAgIHNwaW5fbG9ja19pbml0KCZoZC0+YXJjaC5tYXBwaW5n
X2xvY2spOwotICAgIElOSVRfTElTVF9IRUFEKCZoZC0+YXJjaC5tYXBwZWRf
cm1ycnMpOworICAgIElOSVRfTElTVF9IRUFEKCZoZC0+YXJjaC5pZGVudGl0
eV9tYXBzKTsKIAogICAgIHJldHVybiAwOwogfQpAQCAtMTU5LDYgKzE1OSw5
OSBAQCB2b2lkIGFyY2hfaW9tbXVfZG9tYWluX2Rlc3Ryb3koc3RydWN0IGRv
CiB7CiB9CiAKK3N0cnVjdCBpZGVudGl0eV9tYXAgeworICAgIHN0cnVjdCBs
aXN0X2hlYWQgbGlzdDsKKyAgICBwYWRkcl90IGJhc2UsIGVuZDsKKyAgICBw
Mm1fYWNjZXNzX3QgYWNjZXNzOworICAgIHVuc2lnbmVkIGludCBjb3VudDsK
K307CisKK2ludCBpb21tdV9pZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBkb21h
aW4gKmQsIHAybV9hY2Nlc3NfdCBwMm1hLAorICAgICAgICAgICAgICAgICAg
ICAgICAgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGVuZCwKKyAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKQoreworICAg
IHVuc2lnbmVkIGxvbmcgYmFzZV9wZm4gPSBiYXNlID4+IFBBR0VfU0hJRlRf
NEs7CisgICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuID0gUEFHRV9BTElHTl80
SyhlbmQpID4+IFBBR0VfU0hJRlRfNEs7CisgICAgc3RydWN0IGlkZW50aXR5
X21hcCAqbWFwOworICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhkID0gZG9t
X2lvbW11KGQpOworCisgICAgQVNTRVJUKHBjaWRldnNfbG9ja2VkKCkpOwor
ICAgIEFTU0VSVChiYXNlIDwgZW5kKTsKKworICAgIC8qCisgICAgICogTm8g
bmVlZCB0byBhY3F1aXJlIGhkLT5hcmNoLm1hcHBpbmdfbG9jazogQm90aCBp
bnNlcnRpb24gYW5kIHJlbW92YWwKKyAgICAgKiBnZXQgZG9uZSB3aGlsZSBo
b2xkaW5nIHBjaWRldnNfbG9jay4KKyAgICAgKi8KKyAgICBsaXN0X2Zvcl9l
YWNoX2VudHJ5KCBtYXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzLCBsaXN0
ICkKKyAgICB7CisgICAgICAgIGlmICggbWFwLT5iYXNlID09IGJhc2UgJiYg
bWFwLT5lbmQgPT0gZW5kICkKKyAgICAgICAgeworICAgICAgICAgICAgaW50
IHJldCA9IDA7CisKKyAgICAgICAgICAgIGlmICggcDJtYSAhPSBwMm1fYWNj
ZXNzX3ggKQorICAgICAgICAgICAgeworICAgICAgICAgICAgICAgIGlmICgg
bWFwLT5hY2Nlc3MgIT0gcDJtYSApCisgICAgICAgICAgICAgICAgICAgIHJl
dHVybiAtRUFERFJJTlVTRTsKKyAgICAgICAgICAgICAgICArK21hcC0+Y291
bnQ7CisgICAgICAgICAgICAgICAgcmV0dXJuIDA7CisgICAgICAgICAgICB9
CisKKyAgICAgICAgICAgIGlmICggLS1tYXAtPmNvdW50ICkKKyAgICAgICAg
ICAgICAgICByZXR1cm4gMDsKKworICAgICAgICAgICAgd2hpbGUgKCBiYXNl
X3BmbiA8IGVuZF9wZm4gKQorICAgICAgICAgICAgeworICAgICAgICAgICAg
ICAgIGlmICggY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KGQsIGJhc2VfcGZu
KSApCisgICAgICAgICAgICAgICAgICAgIHJldCA9IC1FTlhJTzsKKyAgICAg
ICAgICAgICAgICBiYXNlX3BmbisrOworICAgICAgICAgICAgfQorCisgICAg
ICAgICAgICBsaXN0X2RlbCgmbWFwLT5saXN0KTsKKyAgICAgICAgICAgIHhm
cmVlKG1hcCk7CisKKyAgICAgICAgICAgIHJldHVybiByZXQ7CisgICAgICAg
IH0KKworICAgICAgICBpZiAoIGVuZCA+PSBtYXAtPmJhc2UgJiYgbWFwLT5l
bmQgPj0gYmFzZSApCisgICAgICAgICAgICByZXR1cm4gLUVBRERSSU5VU0U7
CisgICAgfQorCisgICAgaWYgKCBwMm1hID09IHAybV9hY2Nlc3NfeCApCisg
ICAgICAgIHJldHVybiAtRU5PRU5UOworCisgICAgd2hpbGUgKCBiYXNlX3Bm
biA8IGVuZF9wZm4gKQorICAgIHsKKyAgICAgICAgaW50IGVyciA9IHNldF9p
ZGVudGl0eV9wMm1fZW50cnkoZCwgYmFzZV9wZm4sIHAybWEsIGZsYWcpOwor
CisgICAgICAgIGlmICggZXJyICkKKyAgICAgICAgICAgIHJldHVybiBlcnI7
CisgICAgICAgIGJhc2VfcGZuKys7CisgICAgfQorCisgICAgbWFwID0geG1h
bGxvYyhzdHJ1Y3QgaWRlbnRpdHlfbWFwKTsKKyAgICBpZiAoICFtYXAgKQor
ICAgICAgICByZXR1cm4gLUVOT01FTTsKKyAgICBtYXAtPmJhc2UgPSBiYXNl
OworICAgIG1hcC0+ZW5kID0gZW5kOworICAgIG1hcC0+YWNjZXNzID0gcDJt
YTsKKyAgICBtYXAtPmNvdW50ID0gMTsKKyAgICBsaXN0X2FkZF90YWlsKCZt
YXAtPmxpc3QsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzKTsKKworICAgIHJl
dHVybiAwOworfQorCit2b2lkIGlvbW11X2lkZW50aXR5X21hcF90ZWFyZG93
bihzdHJ1Y3QgZG9tYWluICpkKQoreworICAgIHN0cnVjdCBkb21haW5faW9t
bXUgKmhkID0gZG9tX2lvbW11KGQpOworICAgIHN0cnVjdCBpZGVudGl0eV9t
YXAgKm1hcCwgKnRtcDsKKworICAgIGxpc3RfZm9yX2VhY2hfZW50cnlfc2Fm
ZSAoIG1hcCwgdG1wLCAmaGQtPmFyY2guaWRlbnRpdHlfbWFwcywgbGlzdCAp
CisgICAgeworICAgICAgICBsaXN0X2RlbCgmbWFwLT5saXN0KTsKKyAgICAg
ICAgeGZyZWUobWFwKTsKKyAgICB9Cit9CisKIHN0YXRpYyBib29sIF9faHdk
b21faW5pdCBod2RvbV9pb21tdV9tYXAoY29uc3Qgc3RydWN0IGRvbWFpbiAq
ZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgbG9uZyBwZm4sCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgbWF4X3BmbikKLS0tIGEv
eGVuL2luY2x1ZGUvYXNtLXg4Ni9pb21tdS5oCisrKyBiL3hlbi9pbmNsdWRl
L2FzbS14ODYvaW9tbXUuaApAQCAtMTYsNiArMTYsNyBAQAogCiAjaW5jbHVk
ZSA8eGVuL2Vycm5vLmg+CiAjaW5jbHVkZSA8eGVuL2xpc3QuaD4KKyNpbmNs
dWRlIDx4ZW4vbWVtX2FjY2Vzcy5oPgogI2luY2x1ZGUgPHhlbi9zcGlubG9j
ay5oPgogI2luY2x1ZGUgPGFzbS9wcm9jZXNzb3IuaD4KICNpbmNsdWRlIDxh
c20vaHZtL3ZteC92bWNzLmg+CkBAIC00OCw3ICs0OSw3IEBAIHN0cnVjdCBh
cmNoX2lvbW11CiAgICAgc3BpbmxvY2tfdCBtYXBwaW5nX2xvY2s7ICAgICAg
ICAgICAgLyogaW8gcGFnZSB0YWJsZSBsb2NrICovCiAgICAgaW50IGFnYXc7
ICAgICAvKiBhZGp1c3RlZCBndWVzdCBhZGRyZXNzIHdpZHRoLCAwIGlzIGxl
dmVsIDIgMzAtYml0ICovCiAgICAgdTY0IGlvbW11X2JpdG1hcDsgICAgICAg
ICAgICAgIC8qIGJpdG1hcCBvZiBpb21tdShzKSB0aGF0IHRoZSBkb21haW4g
dXNlcyAqLwotICAgIHN0cnVjdCBsaXN0X2hlYWQgbWFwcGVkX3JtcnJzOwor
ICAgIHN0cnVjdCBsaXN0X2hlYWQgaWRlbnRpdHlfbWFwczsKIAogICAgIC8q
IGFtZCBpb21tdSBzdXBwb3J0ICovCiAgICAgaW50IHBhZ2luZ19tb2RlOwpA
QCAtOTQsNiArOTUsMTEgQEAgYm9vbF90IGlvbW11X3N1cHBvcnRzX2VpbSh2
b2lkKTsKIGludCBpb21tdV9lbmFibGVfeDJhcGljX0lSKHZvaWQpOwogdm9p
ZCBpb21tdV9kaXNhYmxlX3gyYXBpY19JUih2b2lkKTsKIAoraW50IGlvbW11
X2lkZW50aXR5X21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwgcDJtX2FjY2Vz
c190IHAybWEsCisgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90
IGJhc2UsIHBhZGRyX3QgZW5kLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IGZsYWcpOwordm9pZCBpb21tdV9pZGVudGl0eV9t
YXBfdGVhcmRvd24oc3RydWN0IGRvbWFpbiAqZCk7CisKIGV4dGVybiBib29s
IHVudHJ1c3RlZF9tc2k7CiAKIGludCBwaV91cGRhdGVfaXJ0ZShjb25zdCBz
dHJ1Y3QgcGlfZGVzYyAqcGlfZGVzYywgY29uc3Qgc3RydWN0IHBpcnEgKnBp
cnEsCi0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVu
L2luY2x1ZGUvYXNtLXg4Ni9wMm0uaApAQCAtODk2LDYgKzg5NiwzNCBAQCBz
dHJ1Y3QgcDJtX2RvbWFpbiAqcDJtX2dldF9hbHRwMm0oc3RydWN0CiBzdGF0
aWMgaW5saW5lIHZvaWQgcDJtX2FsdHAybV9jaGVjayhzdHJ1Y3QgdmNwdSAq
diwgdWludDE2X3QgaWR4KSB7fQogI2VuZGlmCiAKKy8qIHAybSBhY2Nlc3Mg
dG8gSU9NTVUgZmxhZ3MgKi8KK3N0YXRpYyBpbmxpbmUgdW5zaWduZWQgaW50
IHAybV9hY2Nlc3NfdG9faW9tbXVfZmxhZ3MocDJtX2FjY2Vzc190IHAybWEp
Cit7CisgICAgc3dpdGNoICggcDJtYSApCisgICAgeworICAgIGNhc2UgcDJt
X2FjY2Vzc19ydzoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcnd4OgorICAgICAg
ICByZXR1cm4gSU9NTVVGX3JlYWRhYmxlIHwgSU9NTVVGX3dyaXRhYmxlOwor
CisgICAgY2FzZSBwMm1fYWNjZXNzX3I6CisgICAgY2FzZSBwMm1fYWNjZXNz
X3J4OgorICAgIGNhc2UgcDJtX2FjY2Vzc19yeDJydzoKKyAgICAgICAgcmV0
dXJuIElPTU1VRl9yZWFkYWJsZTsKKworICAgIGNhc2UgcDJtX2FjY2Vzc193
OgorICAgIGNhc2UgcDJtX2FjY2Vzc193eDoKKyAgICAgICAgcmV0dXJuIElP
TU1VRl93cml0YWJsZTsKKworICAgIGNhc2UgcDJtX2FjY2Vzc19uOgorICAg
IGNhc2UgcDJtX2FjY2Vzc194OgorICAgIGNhc2UgcDJtX2FjY2Vzc19uMnJ3
eDoKKyAgICAgICAgcmV0dXJuIDA7CisgICAgfQorCisgICAgQVNTRVJUX1VO
UkVBQ0hBQkxFKCk7CisgICAgcmV0dXJuIDA7Cit9CisKIC8qCiAgKiBwMm0g
dHlwZSB0byBJT01NVSBmbGFncwogICovCkBAIC05MTcsOSArOTQ1LDEwIEBA
IHN0YXRpYyBpbmxpbmUgdW5zaWduZWQgaW50IHAybV9nZXRfaW9tbXUKICAg
ICAgICAgZmxhZ3MgPSBJT01NVUZfcmVhZGFibGU7CiAgICAgICAgIGJyZWFr
OwogICAgIGNhc2UgcDJtX21taW9fZGlyZWN0OgotICAgICAgICBmbGFncyA9
IElPTU1VRl9yZWFkYWJsZTsKLSAgICAgICAgaWYgKCAhcmFuZ2VzZXRfY29u
dGFpbnNfc2luZ2xldG9uKG1taW9fcm9fcmFuZ2VzLCBtZm5feChtZm4pKSAp
Ci0gICAgICAgICAgICBmbGFncyB8PSBJT01NVUZfd3JpdGFibGU7CisgICAg
ICAgIGZsYWdzID0gcDJtX2FjY2Vzc190b19pb21tdV9mbGFncyhwMm1hKTsK
KyAgICAgICAgaWYgKCAoZmxhZ3MgJiBJT01NVUZfd3JpdGFibGUpICYmCisg
ICAgICAgICAgICAgcmFuZ2VzZXRfY29udGFpbnNfc2luZ2xldG9uKG1taW9f
cm9fcmFuZ2VzLCBtZm5feChtZm4pKSApCisgICAgICAgICAgICBmbGFncyAm
PSB+SU9NTVVGX3dyaXRhYmxlOwogICAgICAgICBicmVhazsKICAgICBkZWZh
dWx0OgogICAgICAgICBmbGFncyA9IDA7Cg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfbWFwLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lv
bW11X21hcC5jCkBAIC02MTMsMzggKzYxMyw0OSBAQCBpbnQgYW1kX2lvbW11
X2ZsdXNoX2lvdGxiX2FsbChzdHJ1Y3QgZG9tCiAgICAgcmV0dXJuIDA7CiB9
CiAKLWludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKHN0
cnVjdCBkb21haW4gKmRvbWFpbiwKLSAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHBhZGRyX3QgcGh5c19hZGRyLAotICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBz
aXplLCBpbnQgaXcsIGludCBpcikKK2ludCBhbWRfaW9tbXVfcmVzZXJ2ZV9k
b21haW5fdW5pdHlfbWFwKHN0cnVjdCBkb21haW4gKmQsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaXZy
c191bml0eV9tYXAgKm1hcCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKQogewotICAgIHVuc2ln
bmVkIGxvbmcgbnBhZ2VzLCBpOwotICAgIHVuc2lnbmVkIGxvbmcgZ2ZuOwot
ICAgIHVuc2lnbmVkIGludCBmbGFncyA9ICEhaXI7Ci0gICAgdW5zaWduZWQg
aW50IGZsdXNoX2ZsYWdzID0gMDsKLSAgICBpbnQgcnQgPSAwOwotCi0gICAg
aWYgKCBpdyApCi0gICAgICAgIGZsYWdzIHw9IElPTU1VRl93cml0YWJsZTsK
LQotICAgIG5wYWdlcyA9IHJlZ2lvbl90b19wYWdlcyhwaHlzX2FkZHIsIHNp
emUpOwotICAgIGdmbiA9IHBoeXNfYWRkciA+PiBQQUdFX1NISUZUOwotICAg
IGZvciAoIGkgPSAwOyBpIDwgbnBhZ2VzOyBpKysgKQorICAgIGludCByYzsK
KworICAgIGlmICggZCA9PSBkb21faW8gKQorICAgICAgICByZXR1cm4gMDsK
KworICAgIGZvciAoIHJjID0gMDsgIXJjICYmIG1hcDsgbWFwID0gbWFwLT5u
ZXh0ICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGxvbmcgZnJhbWUgPSBn
Zm4gKyBpOworICAgICAgICBwMm1fYWNjZXNzX3QgcDJtYSA9IHAybV9hY2Nl
c3NfbjsKKworICAgICAgICBpZiAoIG1hcC0+cmVhZCApCisgICAgICAgICAg
ICBwMm1hIHw9IHAybV9hY2Nlc3NfcjsKKyAgICAgICAgaWYgKCBtYXAtPndy
aXRlICkKKyAgICAgICAgICAgIHAybWEgfD0gcDJtX2FjY2Vzc193OwogCi0g
ICAgICAgIHJ0ID0gYW1kX2lvbW11X21hcF9wYWdlKGRvbWFpbiwgX2Rmbihm
cmFtZSksIF9tZm4oZnJhbWUpLCBmbGFncywKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgJmZsdXNoX2ZsYWdzKTsKLSAgICAgICAgaWYgKCBy
dCAhPSAwICkKLSAgICAgICAgICAgIGJyZWFrOworICAgICAgICByYyA9IGlv
bW11X2lkZW50aXR5X21hcHBpbmcoZCwgcDJtYSwgbWFwLT5hZGRyLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRyICsg
bWFwLT5sZW5ndGggLSAxLCBmbGFnKTsKICAgICB9CiAKLSAgICAvKiBVc2Ug
d2hpbGUtYnJlYWsgdG8gYXZvaWQgY29tcGlsZXIgd2FybmluZyAqLwotICAg
IHdoaWxlICggZmx1c2hfZmxhZ3MgJiYKLSAgICAgICAgICAgIGFtZF9pb21t
dV9mbHVzaF9pb3RsYl9wYWdlcyhkb21haW4sIF9kZm4oZ2ZuKSwKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBucGFnZXMsIGZs
dXNoX2ZsYWdzKSApCi0gICAgICAgIGJyZWFrOworICAgIHJldHVybiByYzsK
K30KKworaW50IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV91bm1h
cChzdHJ1Y3QgZG9tYWluICpkLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaXZyc191bml0eV9tYXAg
Km1hcCkKK3sKKyAgICBpbnQgcmM7CisKKyAgICBpZiAoIGQgPT0gZG9tX2lv
ICkKKyAgICAgICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCByYyA9IDA7IG1h
cDsgbWFwID0gbWFwLT5uZXh0ICkKKyAgICB7CisgICAgICAgIGludCByZXQg
PSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQsIHAybV9hY2Nlc3NfeCwgbWFw
LT5hZGRyLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBtYXAtPmFkZHIgKyBtYXAtPmxlbmd0aCAtIDEsIDApOworCisgICAg
ICAgIGlmICggcmV0ICYmIHJldCAhPSAtRU5PRU5UICYmICFyYyApCisgICAg
ICAgICAgICByYyA9IHJldDsKKyAgICB9CiAKLSAgICByZXR1cm4gcnQ7Cisg
ICAgcmV0dXJuIHJjOwogfQogCiAvKiBTaGFyZSBwMm0gdGFibGUgd2l0aCBp
b21tdS4gKi8KLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3Bj
aV9hbWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9h
bWQvcGNpX2FtZF9pb21tdS5jCkBAIC0zMDcsNiArMzA3LDcgQEAgc3RhdGlj
IGludCByZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbgogICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11OwogICAgIGludCBiZGYsIHJjOwogICAgIHN0
cnVjdCBkb21haW5faW9tbXUgKnQgPSBkb21faW9tbXUodGFyZ2V0KTsKKyAg
ICBjb25zdCBzdHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9
IGdldF9pdnJzX21hcHBpbmdzKHBkZXYtPnNlZyk7CiAKICAgICBiZGYgPSBQ
Q0lfQkRGMihwZGV2LT5idXMsIHBkZXYtPmRldmZuKTsKICAgICBpb21tdSA9
IGZpbmRfaW9tbXVfZm9yX2RldmljZShwZGV2LT5zZWcsIGJkZik7CkBAIC0z
MjEsMTAgKzMyMiwyNCBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2RldmljZShz
dHJ1Y3QgZG9tYWluCiAKICAgICBhbWRfaW9tbXVfZGlzYWJsZV9kb21haW5f
ZGV2aWNlKHNvdXJjZSwgaW9tbXUsIGRldmZuLCBwZGV2KTsKIAotICAgIGlm
ICggZGV2Zm4gPT0gcGRldi0+ZGV2Zm4gKQorICAgIC8qCisgICAgICogSWYg
dGhlIGRldmljZSBiZWxvbmdzIHRvIHRoZSBoYXJkd2FyZSBkb21haW4sIGFu
ZCBpdCBoYXMgYSB1bml0eSBtYXBwaW5nLAorICAgICAqIGRvbid0IHJlbW92
ZSBpdCBmcm9tIHRoZSBoYXJkd2FyZSBkb21haW4sIGJlY2F1c2UgQklPUyBt
YXkgcmVmZXJlbmNlIHRoYXQKKyAgICAgKiBtYXBwaW5nLgorICAgICAqLwor
ICAgIGlmICggIWlzX2hhcmR3YXJlX2RvbWFpbihzb3VyY2UpICkKKyAgICB7
CisgICAgICAgIHJjID0gYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5
X3VubWFwKAorICAgICAgICAgICAgICAgICBzb3VyY2UsCisgICAgICAgICAg
ICAgICAgIGl2cnNfbWFwcGluZ3NbZ2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRl
di0+c2VnLCBiZGYpXS51bml0eV9tYXApOworICAgICAgICBpZiAoIHJjICkK
KyAgICAgICAgICAgIHJldHVybiByYzsKKyAgICB9CisKKyAgICBpZiAoIGRl
dmZuID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRvbWFpbiAhPSBkb21faW8g
KQogICAgIHsKLSAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21haW5fbGlz
dCwgJnRhcmdldC0+YXJjaC5wZGV2X2xpc3QpOwotICAgICAgICBwZGV2LT5k
b21haW4gPSB0YXJnZXQ7CisgICAgICAgIGxpc3RfbW92ZSgmcGRldi0+ZG9t
YWluX2xpc3QsICZkb21faW8tPmFyY2gucGRldl9saXN0KTsKKyAgICAgICAg
cGRldi0+ZG9tYWluID0gZG9tX2lvOwogICAgIH0KIAogICAgIHJjID0gYWxs
b2NhdGVfZG9tYWluX3Jlc291cmNlcyh0KTsKQEAgLTMzNiw2ICszNTEsMTIg
QEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbgog
ICAgICAgICAgICAgICAgICAgICBwZGV2LT5zZWcsIHBkZXYtPmJ1cywgUENJ
X1NMT1QoZGV2Zm4pLCBQQ0lfRlVOQyhkZXZmbiksCiAgICAgICAgICAgICAg
ICAgICAgIHNvdXJjZS0+ZG9tYWluX2lkLCB0YXJnZXQtPmRvbWFpbl9pZCk7
CiAKKyAgICBpZiAoIGRldmZuID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRv
bWFpbiAhPSB0YXJnZXQgKQorICAgIHsKKyAgICAgICAgbGlzdF9tb3ZlKCZw
ZGV2LT5kb21haW5fbGlzdCwgJnRhcmdldC0+YXJjaC5wZGV2X2xpc3QpOwor
ICAgICAgICBwZGV2LT5kb21haW4gPSB0YXJnZXQ7CisgICAgfQorCiAgICAg
cmV0dXJuIDA7CiB9CiAKQEAgLTM0NiwyMCArMzY3LDI4IEBAIHN0YXRpYyBp
bnQgYW1kX2lvbW11X2Fzc2lnbl9kZXZpY2Uoc3RydWMKICAgICBzdHJ1Y3Qg
aXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBp
bmdzKHBkZXYtPnNlZyk7CiAgICAgaW50IGJkZiA9IFBDSV9CREYyKHBkZXYt
PmJ1cywgZGV2Zm4pOwogICAgIGludCByZXFfaWQgPSBnZXRfZG1hX3JlcXVl
c3Rvcl9pZChwZGV2LT5zZWcsIGJkZik7Ci0gICAgY29uc3Qgc3RydWN0IGl2
cnNfdW5pdHlfbWFwICp1bml0eV9tYXA7CisgICAgaW50IHJjID0gYW1kX2lv
bW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAgICAgICAgICAg
ICAgZCwgaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcCwgZmxhZyk7
CisKKyAgICBpZiAoICFyYyApCisgICAgICAgIHJjID0gcmVhc3NpZ25fZGV2
aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2Zm4sIHBkZXYpOwogCi0gICAgZm9y
ICggdW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21h
cDsgdW5pdHlfbWFwOwotICAgICAgICAgIHVuaXR5X21hcCA9IHVuaXR5X21h
cC0+bmV4dCApCisgICAgaWYgKCByYyAmJiAhaXNfaGFyZHdhcmVfZG9tYWlu
KGQpICkKICAgICB7Ci0gICAgICAgIGludCByYyA9IGFtZF9pb21tdV9yZXNl
cnZlX2RvbWFpbl91bml0eV9tYXAoCi0gICAgICAgICAgICAgICAgICAgICBk
LCB1bml0eV9tYXAtPmFkZHIsIHVuaXR5X21hcC0+bGVuZ3RoLAotICAgICAg
ICAgICAgICAgICAgICAgdW5pdHlfbWFwLT53cml0ZSwgdW5pdHlfbWFwLT5y
ZWFkKTsKKyAgICAgICAgaW50IHJldCA9IGFtZF9pb21tdV9yZXNlcnZlX2Rv
bWFpbl91bml0eV91bm1hcCgKKyAgICAgICAgICAgICAgICAgICAgICBkLCBp
dnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwKTsKIAotICAgICAgICBp
ZiAoIHJjICkKLSAgICAgICAgICAgIHJldHVybiByYzsKKyAgICAgICAgaWYg
KCByZXQgKQorICAgICAgICB7CisgICAgICAgICAgICBwcmludGsoWEVOTE9H
X0VSUiAiQU1ELVZpOiAiCisgICAgICAgICAgICAgICAgICAgInVuaXR5LXVu
bWFwIGZvciAlcGQvJTA0eDolMDJ4OiUwMnguJXUgZmFpbGVkICglZClcbiIs
CisgICAgICAgICAgICAgICAgICAgZCwgcGRldi0+c2VnLCBwZGV2LT5idXMs
CisgICAgICAgICAgICAgICAgICAgUENJX1NMT1QoZGV2Zm4pLCBQQ0lfRlVO
QyhkZXZmbiksIHJldCk7CisgICAgICAgICAgICBkb21haW5fY3Jhc2goZCk7
CisgICAgICAgIH0KICAgICB9CiAKLSAgICByZXR1cm4gcmVhc3NpZ25fZGV2
aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2Zm4sIHBkZXYpOworICAgIHJldHVy
biByYzsKIH0KIAogc3RhdGljIHZvaWQgZGVhbGxvY2F0ZV9uZXh0X3BhZ2Vf
dGFibGUoc3RydWN0IHBhZ2VfaW5mbyAqcGcsIGludCBsZXZlbCkKQEAgLTQy
NSw2ICs0NTQsNyBAQCBzdGF0aWMgdm9pZCBkZWFsbG9jYXRlX2lvbW11X3Bh
Z2VfdGFibGVzCiAKIHN0YXRpYyB2b2lkIGFtZF9pb21tdV9kb21haW5fZGVz
dHJveShzdHJ1Y3QgZG9tYWluICpkKQogeworICAgIGlvbW11X2lkZW50aXR5
X21hcF90ZWFyZG93bihkKTsKICAgICBkZWFsbG9jYXRlX2lvbW11X3BhZ2Vf
dGFibGVzKGQpOwogICAgIGFtZF9pb21tdV9mbHVzaF9hbGxfcGFnZXMoZCk7
CiB9Ci0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvaHZtL3N2bS9hbWQtaW9t
bXUtcHJvdG8uaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L2h2bS9zdm0v
YW1kLWlvbW11LXByb3RvLmgKQEAgLTYyLDggKzYyLDEwIEBAIGludCBfX211
c3RfY2hlY2sgYW1kX2lvbW11X3VubWFwX3BhZ2Uoc3QKIHVpbnQ2NF90IGFt
ZF9pb21tdV9nZXRfYWRkcmVzc19mcm9tX3B0ZSh2b2lkICplbnRyeSk7CiBp
bnQgX19tdXN0X2NoZWNrIGFtZF9pb21tdV9hbGxvY19yb290KHN0cnVjdCBk
b21haW5faW9tbXUgKmhkKTsKIGludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21h
aW5fdW5pdHlfbWFwKHN0cnVjdCBkb21haW4gKmRvbWFpbiwKLSAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhZGRyX3QgcGh5c19h
ZGRyLCB1bnNpZ25lZCBsb25nIHNpemUsCi0gICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBpbnQgaXcsIGludCBpcik7CisgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3Qg
aXZyc191bml0eV9tYXAgKm1hcCwKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKTsKK2ludCBhbWRf
aW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAoc3RydWN0IGRvbWFp
biAqZCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXApOwogaW50IF9f
bXVzdF9jaGVjayBhbWRfaW9tbXVfZmx1c2hfaW90bGJfcGFnZXMoc3RydWN0
IGRvbWFpbiAqZCwgZGZuX3QgZGZuLAogICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHBhZ2VfY291
bnQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgZmx1c2hfZmxhZ3MpOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVu
L2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtOTks
MTIgKzk5LDggQEAgc3RhdGljIHN0cnVjdCBhbWRfaW9tbXUgKiBfX2luaXQg
ZmluZF9pbwogfQogCiBzdGF0aWMgaW50IF9faW5pdCByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZSgKLSAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21t
dSwgdWludDY0X3QgYmFzZSwgdWludDY0X3QgbGltaXQsCi0gICAgYm9vbCBh
bGwsIGJvb2wgaXcsIGJvb2wgaXIpCisgICAgc3RydWN0IGFtZF9pb21tdSAq
aW9tbXUsIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwgYm9vbCBhbGwp
CiB7Ci0gICAgaWYgKCAhaXIgfHwgIWl3ICkKLSAgICAgICAgcmV0dXJuIC1F
UEVSTTsKLQogICAgIC8qIG5lZWQgdG8gZXh0ZW5kIGV4Y2x1c2lvbiByYW5n
ZT8gKi8KICAgICBpZiAoIGlvbW11LT5leGNsdXNpb25fZW5hYmxlICkKICAg
ICB7CkBAIC0xMzMsMTQgKzEyOSwxOCBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZXNlcnZlX3VuaXR5X21hcF9mb3JfCiB7CiAgICAgc3RydWN0IGl2cnNfbWFw
cGluZ3MgKml2cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhzZWcp
OwogICAgIHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwID0gaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcDsKKyAgICBpbnQgcGFnaW5nX21v
ZGUgPSBhbWRfaW9tbXVfZ2V0X3BhZ2luZ19tb2RlKFBGTl9VUChiYXNlICsg
bGVuZ3RoKSk7CisKKyAgICBpZiAoIHBhZ2luZ19tb2RlIDwgMCApCisgICAg
ICAgIHJldHVybiBwYWdpbmdfbW9kZTsKIAogICAgIC8qIENoZWNrIGZvciBv
dmVybGFwcy4gKi8KICAgICBmb3IgKCA7IHVuaXR5X21hcDsgdW5pdHlfbWFw
ID0gdW5pdHlfbWFwLT5uZXh0ICkKICAgICB7CiAgICAgICAgIC8qCiAgICAg
ICAgICAqIEV4YWN0IG1hdGNoZXMgYXJlIG9rYXkuIFRoaXMgY2FuIGluIHBh
cnRpY3VsYXIgaGFwcGVuIHdoZW4KLSAgICAgICAgICogcmVnaXN0ZXJfZXhj
bHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3aWNlIGZv
ciB0aGUKLSAgICAgICAgICogc2FtZSAocyxiLGQsZikuCisgICAgICAgICAq
IHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2UoKSBjYWxscyBoZXJlIHR3aWNl
IGZvciB0aGUgc2FtZQorICAgICAgICAgKiAocyxiLGQsZikuCiAgICAgICAg
ICAqLwogICAgICAgICBpZiAoIGJhc2UgPT0gdW5pdHlfbWFwLT5hZGRyICYm
IGxlbmd0aCA9PSB1bml0eV9tYXAtPmxlbmd0aCAmJgogICAgICAgICAgICAg
IGlyID09IHVuaXR5X21hcC0+cmVhZCAmJiBpdyA9PSB1bml0eV9tYXAtPndy
aXRlICkKQEAgLTE2OCw1NSArMTY4LDUyIEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl8KICAgICB1bml0eV9tYXAtPm5leHQg
PSBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwOwogICAgIGl2cnNfbWFw
cGluZ3NbYmRmXS51bml0eV9tYXAgPSB1bml0eV9tYXA7CiAKKyAgICBpZiAo
IHBhZ2luZ19tb2RlID4gYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSApCisg
ICAgICAgIGFtZF9pb21tdV9taW5fcGFnaW5nX21vZGUgPSBwYWdpbmdfbW9k
ZTsKKwogICAgIHJldHVybiAwOwogfQogCi1zdGF0aWMgaW50IF9faW5pdCBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2FsbF9kZXZpY2VzKAotICAg
IHVuc2lnbmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTgg
aXcsIHU4IGlyKQorc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfcmFuZ2Vf
Zm9yX2FsbF9kZXZpY2VzKAorICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBs
aW1pdCwgYm9vbCBpdywgYm9vbCBpciwgYm9vbCBleGNsdXNpb24pCiB7CiAg
ICAgaW50IHNlZyA9IDA7IC8qIFhYWCAqLwotICAgIHVuc2lnbmVkIGxvbmcg
cmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1k
X2lvbW11ICppb21tdTsKLSAgICB1bnNpZ25lZCBpbnQgYmRmOwogICAgIGlu
dCByYyA9IDA7CiAKICAgICAvKiBpcyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5n
ZSBpbnNpZGUgb2YgSU9NTVUgdmlydHVhbCBhZGRyZXNzIHNwYWNlPyAqLwog
ICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8g
YmUgcGFnZS1hbGlnbmVkICovCi0gICAgcmFuZ2VfdG9wID0gbGltaXQgKyBQ
QUdFX1NJWkU7Ci0gICAgaW9tbXVfdG9wID0gbWF4X3BhZ2UgKiBQQUdFX1NJ
WkU7Ci0gICAgaWYgKCBiYXNlIDwgaW9tbXVfdG9wICkKLSAgICB7Ci0gICAg
ICAgIGlmICggcmFuZ2VfdG9wID4gaW9tbXVfdG9wICkKLSAgICAgICAgICAg
IHJhbmdlX3RvcCA9IGlvbW11X3RvcDsKLSAgICAgICAgbGVuZ3RoID0gcmFu
Z2VfdG9wIC0gYmFzZTsKLSAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHkt
bWFwcGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAqLwotICAgICAgICAv
KiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNp
b24gcmFuZ2UgKi8KLSAgICAgICAgZm9yICggYmRmID0gMDsgIXJjICYmIGJk
ZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAgICAgICAgIHJj
ID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFz
ZSwgbGVuZ3RoLCBpdywgaXIpOwotICAgICAgICAvKiBwdXNoICdiYXNlJyBq
dXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCi0gICAg
ICAgIGJhc2UgPSBpb21tdV90b3A7Ci0gICAgfQotICAgIC8qIHJlZ2lzdGVy
IElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICgg
IXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCApCisgICAgaWYgKCBleGNsdXNp
b24gKQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBpb21t
dSApCiAgICAgICAgIHsKLSAgICAgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21t
dV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAotICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cnVl
IC8qIGFsbCAqLywgaXcsIGlyKTsKLSAgICAgICAgICAgIGlmICggcmMgKQot
ICAgICAgICAgICAgICAgIGJyZWFrOworICAgICAgICAgICAgaW50IHJldCA9
IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBs
aW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB0cnVlIC8qIGFsbCAqLyk7CisKKyAgICAgICAgICAg
IGlmICggcmV0ICYmICFyYyApCisgICAgICAgICAgICAgICAgcmMgPSByZXQ7
CiAgICAgICAgIH0KICAgICB9CiAKKyAgICBpZiAoICFleGNsdXNpb24gfHwg
cmMgKQorICAgIHsKKyAgICAgICAgcGFkZHJfdCBsZW5ndGggPSBsaW1pdCAr
IFBBR0VfU0laRSAtIGJhc2U7CisgICAgICAgIHVuc2lnbmVkIGludCBiZGY7
CisKKyAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHktbWFwcGVkIHBhZ2Ug
ZW50cmllcyBmb3IgZGV2aWNlcyAqLworICAgICAgICBmb3IgKCBiZGYgPSBy
YyA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyAp
CisgICAgICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZp
Y2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsKKyAgICB9CisK
ICAgICByZXR1cm4gcmM7CiB9CiAKLXN0YXRpYyBpbnQgX19pbml0IHJlZ2lz
dGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfZGV2aWNlKAotICAgIHUxNiBiZGYs
IHVuc2lnbmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTgg
aXcsIHU4IGlyKQorc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfcmFuZ2Vf
Zm9yX2RldmljZSgKKyAgICB1bnNpZ25lZCBpbnQgYmRmLCBwYWRkcl90IGJh
c2UsIHBhZGRyX3QgbGltaXQsCisgICAgYm9vbCBpdywgYm9vbCBpciwgYm9v
bCBleGNsdXNpb24pCiB7CiAgICAgaW50IHNlZyA9IDA7IC8qIFhYWCAqLwog
ICAgIHN0cnVjdCBpdnJzX21hcHBpbmdzICppdnJzX21hcHBpbmdzID0gZ2V0
X2l2cnNfbWFwcGluZ3Moc2VnKTsKLSAgICB1bnNpZ25lZCBsb25nIHJhbmdl
X3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAgc3RydWN0IGFtZF9pb21t
dSAqaW9tbXU7CiAgICAgdTE2IHJlcTsKICAgICBpbnQgcmMgPSAwOwpAQCAt
MjMwLDI3ICsyMjcsMTkgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJf
ZXhjbHVzaW9uX3JhbgogICAgIHJlcSA9IGl2cnNfbWFwcGluZ3NbYmRmXS5k
dGVfcmVxdWVzdG9yX2lkOwogCiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJh
bWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICBy
YW5nZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsKLSAgICBpb21tdV90b3Ag
PSBtYXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21t
dV90b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21t
dV90b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwot
ICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOworICAgIGlmICgg
ZXhjbHVzaW9uICkKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UgLyogYWxsICov
KTsKKyAgICBpZiAoICFleGNsdXNpb24gfHwgcmMgKQorICAgIHsKKyAgICAg
ICAgcGFkZHJfdCBsZW5ndGggPSBsaW1pdCArIFBBR0VfU0laRSAtIGJhc2U7
CisKICAgICAgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2UgKi8KLSAgICAgICAgLyogbm90ZTogdGhlc2UgZW50
cmllcyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCiAgICAg
ICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJk
ZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpID86CiAgICAgICAgICAgICAgcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShzZWcsIHJlcSwgYmFzZSwgbGVu
Z3RoLCBpdywgaXIpOwotCi0gICAgICAgIC8qIHB1c2ggJ2Jhc2UnIGp1c3Qg
b3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgKi8KLSAgICAgICAg
YmFzZSA9IGlvbW11X3RvcDsKICAgICB9Ci0KLSAgICAvKiByZWdpc3RlciBJ
T01NVSBleGNsdXNpb24gcmFuZ2Ugc2V0dGluZ3MgZm9yIGRldmljZSAqLwot
ICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCAgKQorICAgIGVs
c2UKICAgICB7Ci0gICAgICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAotICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhbHNlIC8qIGFsbCAqLywg
aXcsIGlyKTsKICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmR0ZV9hbGxv
d19leGNsdXNpb24gPSBJT01NVV9DT05UUk9MX0VOQUJMRUQ7CiAgICAgICAg
IGl2cnNfbWFwcGluZ3NbcmVxXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gSU9N
TVVfQ09OVFJPTF9FTkFCTEVEOwogICAgIH0KQEAgLTI1OCw1MyArMjQ3LDQy
IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4K
ICAgICByZXR1cm4gcmM7CiB9CiAKLXN0YXRpYyBpbnQgX19pbml0IHJlZ2lz
dGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfaW9tbXVfZGV2aWNlcygKLSAgICBz
dHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25lZCBsb25nIGJh
c2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3LCB1OCBpcikKK3N0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21tdV9kZXZpY2Vz
KAorICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCBwYWRkcl90IGJhc2Us
IHBhZGRyX3QgbGltaXQsCisgICAgYm9vbCBpdywgYm9vbCBpciwgYm9vbCBl
eGNsdXNpb24pCiB7Ci0gICAgdW5zaWduZWQgbG9uZyByYW5nZV90b3AsIGlv
bW11X3RvcCwgbGVuZ3RoOworICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1l
dGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCisgICAgcGFk
ZHJfdCBsZW5ndGggPSBsaW1pdCArIFBBR0VfU0laRSAtIGJhc2U7CiAgICAg
dW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOwotICAgIGludCByYyA9
IDA7CisgICAgaW50IHJjOwogCi0gICAgLyogaXMgcGFydCBvZiBleGNsdXNp
b24gcmFuZ2UgaW5zaWRlIG9mIElPTU1VIHZpcnR1YWwgYWRkcmVzcyBzcGFj
ZT8gKi8KLSAgICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1
bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwotICAgIHJhbmdlX3RvcCA9IGxp
bWl0ICsgUEFHRV9TSVpFOwotICAgIGlvbW11X3RvcCA9IG1heF9wYWdlICog
UEFHRV9TSVpFOwotICAgIGlmICggYmFzZSA8IGlvbW11X3RvcCApCi0gICAg
ewotICAgICAgICBpZiAoIHJhbmdlX3RvcCA+IGlvbW11X3RvcCApCi0gICAg
ICAgICAgICByYW5nZV90b3AgPSBpb21tdV90b3A7Ci0gICAgICAgIGxlbmd0
aCA9IHJhbmdlX3RvcCAtIGJhc2U7Ci0gICAgICAgIC8qIHJlc2VydmUgci93
IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMgKi8KLSAg
ICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUg
ZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9IDA7ICFy
YyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCi0gICAgICAg
IHsKLSAgICAgICAgICAgIGlmICggaW9tbXUgPT0gZmluZF9pb21tdV9mb3Jf
ZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQotICAgICAgICAgICAgewotICAg
ICAgICAgICAgICAgIHJlcSA9IGdldF9pdnJzX21hcHBpbmdzKGlvbW11LT5z
ZWcpW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZDsKLSAgICAgICAgICAgICAgICBy
YyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywg
YmRmLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcikgPzoKLSAgICAgICAg
ICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcik7Ci0g
ICAgICAgICAgICB9Ci0gICAgICAgIH0KLQotICAgICAgICAvKiBwdXNoICdi
YXNlJyBqdXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICov
Ci0gICAgICAgIGJhc2UgPSBpb21tdV90b3A7CisgICAgaWYgKCBleGNsdXNp
b24gKQorICAgIHsKKyAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsIHRydWUgLyogYWxsICov
KTsKKyAgICAgICAgaWYgKCAhcmMgKQorICAgICAgICAgICAgcmV0dXJuIDA7
CiAgICAgfQogCi0gICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJh
bmdlIHNldHRpbmdzICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9t
bXVfdG9wICkKLSAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lv
bl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8sIGl3
LCBpcik7CisgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2VzICovCisgICAgZm9yICggYmRmID0gcmMgPSAwOyAh
cmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysgKQorICAgIHsK
KyAgICAgICAgaWYgKCBpb21tdSAhPSBmaW5kX2lvbW11X2Zvcl9kZXZpY2Uo
aW9tbXUtPnNlZywgYmRmKSApCisgICAgICAgICAgICBjb250aW51ZTsKKwor
ICAgICAgICByZXEgPSBnZXRfaXZyc19tYXBwaW5ncyhpb21tdS0+c2VnKVti
ZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7CisgICAgICAgIHJjID0gcmVzZXJ2ZV91
bml0eV9tYXBfZm9yX2RldmljZShpb21tdS0+c2VnLCBiZGYsIGJhc2UsIGxl
bmd0aCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGl3LCBpcikgPzoKKyAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21h
cF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcs
IGlyKTsKKyAgICB9CiAKICAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRpYyBp
bnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdCgKICAgICBjb25z
dCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaywKLSAgICB1
bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3
LCB1OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGltaXQsIGJv
b2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAgIHUxNiBi
ZGY7CiAKQEAgLTMxNSwxMiArMjkzLDEyIEBAIHN0YXRpYyBpbnQgX19pbml0
IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGUKICAgICAgICAgcmV0dXJuIC1FTk9E
RVY7CiAgICAgfQogCi0gICAgcmV0dXJuIHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW5nZV9mb3JfZGV2aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpcik7Cisg
ICAgcmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2UoYmRmLCBiYXNl
LCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQogCiBzdGF0aWMgaW50
IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9yYW5nZSgKICAgICBjb25zdCBz
dHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaywKLSAgICB1bnNp
Z25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3LCB1
OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGltaXQsIGJvb2wg
aXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAgIHVuc2lnbmVk
IGludCBmaXJzdF9iZGYsIGxhc3RfYmRmLCBiZGY7CiAgICAgaW50IGVycm9y
OwpAQCAtMzQyLDE1ICszMjAsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcGFy
c2VfaXZtZF9kZXZpY2VfcmFuZwogICAgIH0KIAogICAgIGZvciAoIGJkZiA9
IGZpcnN0X2JkZiwgZXJyb3IgPSAwOyAoYmRmIDw9IGxhc3RfYmRmKSAmJiAh
ZXJyb3I7IGJkZisrICkKLSAgICAgICAgZXJyb3IgPSByZWdpc3Rlcl9leGNs
dXNpb25fcmFuZ2VfZm9yX2RldmljZSgKLSAgICAgICAgICAgIGJkZiwgYmFz
ZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIGVycm9yID0gcmVnaXN0ZXJf
cmFuZ2VfZm9yX2RldmljZSgKKyAgICAgICAgICAgIGJkZiwgYmFzZSwgbGlt
aXQsIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIHJldHVybiBlcnJvcjsK
IH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9kZXZpY2VfaW9t
bXUoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19tZW1vcnkgKml2bWRf
YmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25n
IGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJfdCBiYXNlLCBwYWRk
cl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikK
IHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCiAgICAgc3RydWN0IGFt
ZF9pb21tdSAqaW9tbXU7CkBAIC0zNjUsMTQgKzM0MywxNCBAQCBzdGF0aWMg
aW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9pb21tCiAgICAgICAgIHJl
dHVybiAtRU5PREVWOwogICAgIH0KIAotICAgIHJldHVybiByZWdpc3Rlcl9l
eGNsdXNpb25fcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCi0gICAgICAgIGlv
bW11LCBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICByZXR1cm4gcmVnaXN0
ZXJfcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCisgICAgICAgIGlvbW11LCBi
YXNlLCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQogCiBzdGF0aWMg
aW50IF9faW5pdCBwYXJzZV9pdm1kX2Jsb2NrKGNvbnN0IHN0cnVjdCBhY3Bp
X2l2cnNfbWVtb3J5ICppdm1kX2Jsb2NrKQogewogICAgIHVuc2lnbmVkIGxv
bmcgc3RhcnRfYWRkciwgbWVtX2xlbmd0aCwgYmFzZSwgbGltaXQ7Ci0gICAg
dTggaXcsIGlyOworICAgIGJvb2wgaXcgPSB0cnVlLCBpciA9IHRydWUsIGV4
Y2x1c2lvbiA9IGZhbHNlOwogCiAgICAgaWYgKCBpdm1kX2Jsb2NrLT5oZWFk
ZXIubGVuZ3RoIDwgc2l6ZW9mKCppdm1kX2Jsb2NrKSApCiAgICAgewpAQCAt
Mzg5LDEzICszNjcsMTEgQEAgc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZt
ZF9ibG9jayhjb25zdAogICAgICAgICAgICAgICAgICAgICBpdm1kX2Jsb2Nr
LT5oZWFkZXIudHlwZSwgc3RhcnRfYWRkciwgbWVtX2xlbmd0aCk7CiAKICAg
ICBpZiAoIGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElfSVZNRF9F
WENMVVNJT05fUkFOR0UgKQotICAgICAgICBpdyA9IGlyID0gSU9NTVVfQ09O
VFJPTF9FTkFCTEVEOworICAgICAgICBleGNsdXNpb24gPSB0cnVlOwogICAg
IGVsc2UgaWYgKCBpdm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lW
TURfVU5JVFkgKQogICAgIHsKLSAgICAgICAgaXcgPSBpdm1kX2Jsb2NrLT5o
ZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfUkVBRCA/Ci0gICAgICAgICAgICBJ
T01NVV9DT05UUk9MX0VOQUJMRUQgOiBJT01NVV9DT05UUk9MX0RJU0FCTEVE
OwotICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFD
UElfSVZNRF9XUklURSA/Ci0gICAgICAgICAgICBJT01NVV9DT05UUk9MX0VO
QUJMRUQgOiBJT01NVV9DT05UUk9MX0RJU0FCTEVEOworICAgICAgICBpdyA9
IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElfSVZNRF9SRUFEOwor
ICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElf
SVZNRF9XUklURTsKICAgICB9CiAgICAgZWxzZQogICAgIHsKQEAgLTQwNiwy
MCArMzgyLDIwIEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfYmxv
Y2soY29uc3QKICAgICBzd2l0Y2goIGl2bWRfYmxvY2stPmhlYWRlci50eXBl
ICkKICAgICB7CiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9NRU1PUllfQUxM
OgotICAgICAgICByZXR1cm4gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zv
cl9hbGxfZGV2aWNlcygKLSAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywg
aXIpOworICAgICAgICByZXR1cm4gcmVnaXN0ZXJfcmFuZ2VfZm9yX2FsbF9k
ZXZpY2VzKAorICAgICAgICAgICAgYmFzZSwgbGltaXQsIGl3LCBpciwgZXhj
bHVzaW9uKTsKIAogICAgIGNhc2UgQUNQSV9JVlJTX1RZUEVfTUVNT1JZX09O
RToKLSAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdChp
dm1kX2Jsb2NrLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgICAgICByZXR1cm4g
cGFyc2VfaXZtZF9kZXZpY2Vfc2VsZWN0KGl2bWRfYmxvY2ssIGJhc2UsIGxp
bWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGNhc2UgQUNQSV9JVlJTX1RZ
UEVfTUVNT1JZX1JBTkdFOgotICAgICAgICByZXR1cm4gcGFyc2VfaXZtZF9k
ZXZpY2VfcmFuZ2UoaXZtZF9ibG9jaywKLSAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAg
ICAgICByZXR1cm4gcGFyc2VfaXZtZF9kZXZpY2VfcmFuZ2UoaXZtZF9ibG9j
aywgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBjYXNlIEFD
UElfSVZSU19UWVBFX01FTU9SWV9JT01NVToKLSAgICAgICAgcmV0dXJuIHBh
cnNlX2l2bWRfZGV2aWNlX2lvbW11KGl2bWRfYmxvY2ssCi0gICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1pdCwgaXcs
IGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX2lvbW11
KGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgaXcsIGlyLCBleGNsdXNpb24pOwogCiAg
ICAgZGVmYXVsdDoKICAgICAgICAgQU1EX0lPTU1VX0RFQlVHKCJJVk1EIEVy
cm9yOiBJbnZhbGlkIEJsb2NrIFR5cGUhXG4iKTsKLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0y
MjIsNiArMjIyLDggQEAgc3RhdGljIGludCBfX211c3RfY2hlY2sgYWxsb2Nh
dGVfZG9tYWluXwogICAgIHJldHVybiByYzsKIH0KIAoraW50IF9fcmVhZF9t
b3N0bHkgYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSA9IDE7CisKIHN0YXRp
YyBpbnQgYW1kX2lvbW11X2RvbWFpbl9pbml0KHN0cnVjdCBkb21haW4gKmQp
CiB7CiAgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUo
ZCk7CkBAIC0yMzMsMTEgKzIzNSwxMyBAQCBzdGF0aWMgaW50IGFtZF9pb21t
dV9kb21haW5faW5pdChzdHJ1Y3QKICAgICAgKiAtIEhWTSBjb3VsZCBpbiBw
cmluY2lwbGUgdXNlIDMgb3IgNCBkZXBlbmRpbmcgb24gaG93IG11Y2ggZ3Vl
c3QKICAgICAgKiAgIHBoeXNpY2FsIGFkZHJlc3Mgc3BhY2Ugd2UgZ2l2ZSBp
dCwgYnV0IHRoaXMgaXNuJ3Qga25vd24geWV0IHNvIHVzZSA0CiAgICAgICog
ICB1bmlsYXRlcmFsbHkuCisgICAgICogLSBVbml0eSBtYXBzIG1heSByZXF1
aXJlIGFuIGV2ZW4gaGlnaGVyIG51bWJlci4KICAgICAgKi8KLSAgICBoZC0+
YXJjaC5wYWdpbmdfbW9kZSA9IGFtZF9pb21tdV9nZXRfcGFnaW5nX21vZGUo
Ci0gICAgICAgIGlzX2h2bV9kb21haW4oZCkKLSAgICAgICAgPyAxdWwgPDwg
KERFRkFVTFRfRE9NQUlOX0FERFJFU1NfV0lEVEggLSBQQUdFX1NISUZUKQot
ICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91bmQoKSArIDEpOworICAgIGhk
LT5hcmNoLnBhZ2luZ19tb2RlID0gbWF4KGFtZF9pb21tdV9nZXRfcGFnaW5n
X21vZGUoCisgICAgICAgICAgICBpc19odm1fZG9tYWluKGQpCisgICAgICAg
ICAgICA/IDF1bCA8PCAoREVGQVVMVF9ET01BSU5fQUREUkVTU19XSURUSCAt
IFBBR0VfU0hJRlQpCisgICAgICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91
bmQoKSArIDEpLAorICAgICAgICBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2Rl
KTsKIAogICAgIHJldHVybiAwOwogfQotLS0gYS94ZW4vaW5jbHVkZS9hc20t
eDg2L2h2bS9zdm0vYW1kLWlvbW11LXByb3RvLmgKKysrIGIveGVuL2luY2x1
ZGUvYXNtLXg4Ni9odm0vc3ZtL2FtZC1pb21tdS1wcm90by5oCkBAIC0xMzIs
NiArMTMyLDggQEAgZXh0ZXJuIHN0cnVjdCBocGV0X3NiZGYgewogICAgIH0g
aW5pdDsKIH0gaHBldF9zYmRmOwogCitleHRlcm4gaW50IGFtZF9pb21tdV9t
aW5fcGFnaW5nX21vZGU7CisKIGV4dGVybiB2b2lkICpzaGFyZWRfaW50cmVt
YXBfdGFibGU7CiBleHRlcm4gdW5zaWduZWQgbG9uZyAqc2hhcmVkX2ludHJl
bWFwX2ludXNlOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtODA2LDcgKzgw
Niw3IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRl
cik7IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2Vu
dHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5V
TEwpOwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFw
Mm1faXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAg
ICAgICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3No
YXJlZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4o
bWZuK2ksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAgfQogICAgIH0K
QEAgLTkxNywxMyArOTE3LDEzIEBAIGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5
KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAgICAgICAgICAg
IEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAgICB9Ci0gICAg
ICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNfZm9yZWlnbihv
dCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90KSApCiAgICAg
ICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRuJ3QgYmUgdW5t
YXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAqLworICAgICAg
ICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWdu
IHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwog
ICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAgICAgICAgCi0g
ICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAgICAgIHJldHVy
biAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBpZiAoIHAybV9p
c19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAgICAgICAgewpA
QCAtMTAxOCw4ICsxMDE4LDcgQEAgaW50IHAybV9jaGFuZ2VfdHlwZV9vbmUo
c3RydWN0IGRvbWFpbiAqZAogICAgIHN0cnVjdCBwMm1fZG9tYWluICpwMm0g
PSBwMm1fZ2V0X2hvc3RwMm0oZCk7CiAgICAgaW50IHJjOwogCi0gICAgQlVH
X09OKHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lzX2dyYW50KG50KSk7Ci0g
ICAgQlVHX09OKHAybV9pc19mb3JlaWduKG90KSB8fCBwMm1faXNfZm9yZWln
bihudCkpOworICAgIEJVR19PTihwMm1faXNfc3BlY2lhbChvdCkgfHwgcDJt
X2lzX3NwZWNpYWwobnQpKTsKIAogICAgIGdmbl9sb2NrKHAybSwgZ2ZuLCAw
KTsKIApAQCAtMTI3MiwxMSArMTI3MSwxMSBAQCBzdGF0aWMgaW50IHNldF90
eXBlZF9wMm1fZW50cnkoc3RydWN0IGRvCiAgICAgICAgIGdmbl91bmxvY2so
cDJtLCBnZm4sIG9yZGVyKTsKICAgICAgICAgcmV0dXJuIGN1cl9vcmRlciAr
IDE7CiAgICAgfQotICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1f
aXNfZm9yZWlnbihvdCkgKQorICAgIGlmICggcDJtX2lzX3NwZWNpYWwob3Qp
ICkKICAgICB7CiAgICAgICAgIGdmbl91bmxvY2socDJtLCBnZm4sIG9yZGVy
KTsKICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwotICAgICAgICByZXR1cm4g
LUVOT0VOVDsKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKICAgICB9CiAgICAg
ZWxzZSBpZiAoIHAybV9pc19yYW0ob3QpICkKICAgICB7Ci0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4
Ni9wMm0uaApAQCAtMTQxLDYgKzE0MSwxMCBAQCB0eXBlZGVmIHVuc2lnbmVk
IGludCBwMm1fcXVlcnlfdDsKICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB8IHAybV90b19tYXNrKHAybV9yYW1fbG9nZGlydHkpICkKICNkZWZpbmUg
UDJNX1NIQVJFRF9UWVBFUyAgIChwMm1fdG9fbWFzayhwMm1fcmFtX3NoYXJl
ZCkpCiAKKy8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQgdXAgdmlhIHNw
ZWNpYWwgYWNjZXNzb3JzLiAqLworI2RlZmluZSBQMk1fU1BFQ0lBTF9UWVBF
UyAoUDJNX0dSQU5UX1RZUEVTIHwgXAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkKKwogLyogVmFs
aWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQgd2l0aCBhICh2
YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9NRk5fVFlQRVMg
KFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfCBwMm1fdG9fbWFzayhwMm1fbW1pb19k
aXJlY3QpIFwKQEAgLTE2OSw2ICsxNzMsNyBAQCB0eXBlZGVmIHVuc2lnbmVk
IGludCBwMm1fcXVlcnlfdDsKICNkZWZpbmUgcDJtX2lzX3BhZ2VkKF90KSAg
ICAocDJtX3RvX21hc2soX3QpICYgUDJNX1BBR0VEX1RZUEVTKQogI2RlZmlu
ZSBwMm1faXNfc2hhcmFibGUoX3QpIChwMm1fdG9fbWFzayhfdCkgJiBQMk1f
U0hBUkFCTEVfVFlQRVMpCiAjZGVmaW5lIHAybV9pc19zaGFyZWQoX3QpICAg
KHAybV90b19tYXNrKF90KSAmIFAyTV9TSEFSRURfVFlQRVMpCisjZGVmaW5l
IHAybV9pc19zcGVjaWFsKF90KSAgKHAybV90b19tYXNrKF90KSAmIFAyTV9T
UEVDSUFMX1RZUEVTKQogI2RlZmluZSBwMm1faXNfYnJva2VuKF90KSAgIChw
Mm1fdG9fbWFzayhfdCkgJiBQMk1fQlJPS0VOX1RZUEVTKQogI2RlZmluZSBw
Mm1faXNfZm9yZWlnbihfdCkgIChwMm1fdG9fbWFzayhfdCkgJiBwMm1fdG9f
bWFzayhwMm1fbWFwX2ZvcmVpZ24pKQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.12-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.12-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tLmMK
KysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ4MDksNyArNDgwOSw5IEBA
IGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogCiAgICAgLyogUmVt
b3ZlIHByZXZpb3VzbHkgbWFwcGVkIHBhZ2UgaWYgaXQgd2FzIHByZXNlbnQu
ICovCiAgICAgcHJldl9tZm4gPSBtZm5feChnZXRfZ2ZuKGQsIGdmbl94KGdw
Zm4pLCAmcDJtdCkpOwotICAgIGlmICggbWZuX3ZhbGlkKF9tZm4ocHJldl9t
Zm4pKSApCisgICAgaWYgKCBwMm10ID09IHAybV9tbWlvX2RpcmVjdCApCisg
ICAgICAgIHJjID0gLUVQRVJNOworICAgIGVsc2UgaWYgKCBtZm5fdmFsaWQo
X21mbihwcmV2X21mbikpICkKICAgICB7CiAgICAgICAgIGlmICggaXNfeGVu
X2hlYXBfbWZuKHByZXZfbWZuKSApCiAgICAgICAgICAgICAvKiBYZW4gaGVh
cCBmcmFtZXMgYXJlIHNpbXBseSB1bmhvb2tlZCBmcm9tIHRoaXMgcGh5cyBz
bG90LiAqLwotLS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLmMKKysrIGIveGVu
L2FyY2gveDg2L21tL3AybS5jCkBAIC03OTUsNyArNzk1LDggQEAgcDJtX3Jl
bW92ZV9wYWdlKHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmY3VyX29yZGVyLCBO
VUxMKTsKIAogICAgICAgICBpZiAoIHAybV9pc192YWxpZCh0KSAmJgotICAg
ICAgICAgICAgICghbWZuX3ZhbGlkKF9tZm4obWZuKSkgfHwgbWZuICsgaSAh
PSBtZm5feChtZm5fcmV0dXJuKSkgKQorICAgICAgICAgICAgICghbWZuX3Zh
bGlkKF9tZm4obWZuKSkgfHwgdCA9PSBwMm1fbW1pb19kaXJlY3QgfHwKKyAg
ICAgICAgICAgICAgbWZuICsgaSAhPSBtZm5feChtZm5fcmV0dXJuKSkgKQog
ICAgICAgICAgICAgcmV0dXJuIC1FSUxTRVE7CiAKICAgICAgICAgaSArPSAo
MVVMIDw8IGN1cl9vcmRlcikgLSAoKGdmbl9sICsgaSkgJiAoKDFVTCA8PCBj
dXJfb3JkZXIpIC0gMSkpOwpAQCAtODczLDcgKzg3NCw3IEBAIGd1ZXN0X3Bo
eXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICBpZiAoIHAy
bV9pc19mb3JlaWduKHQpICkKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7CiAK
LSAgICBpZiAoICFtZm5fdmFsaWQobWZuKSApCisgICAgaWYgKCAhbWZuX3Zh
bGlkKG1mbikgfHwgdCA9PSBwMm1fbW1pb19kaXJlY3QgKQogICAgIHsKICAg
ICAgICAgQVNTRVJUX1VOUkVBQ0hBQkxFKCk7CiAgICAgICAgIHJldHVybiAt
RUlOVkFMOwpAQCAtOTE5LDcgKzkyMCw3IEBAIGd1ZXN0X3BoeXNtYXBfYWRk
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgfQogICAgICAgICBp
ZiAoIHAybV9pc19zcGVjaWFsKG90KSApCiAgICAgICAgIHsKLSAgICAgICAg
ICAgIC8qIERvbid0IHBlcm1pdCB1bm1hcHBpbmcgZ3JhbnQvZm9yZWlnbiB0
aGlzIHdheS4gKi8KKyAgICAgICAgICAgIC8qIERvbid0IHBlcm1pdCB1bm1h
cHBpbmcgZ3JhbnQvZm9yZWlnbi9kaXJlY3QtTU1JTyB0aGlzIHdheS4gKi8K
ICAgICAgICAgICAgIGRvbWFpbl9jcmFzaChkKTsKICAgICAgICAgICAgIHAy
bV91bmxvY2socDJtKTsKICAgICAgICAgICAgIApAQCAtMTM3NSw4ICsxMzc2
LDggQEAgaW50IHNldF9pZGVudGl0eV9wMm1fZW50cnkoc3RydWN0IGRvbWFp
bgogICogICAgb3JkZXIrMSAgZm9yIGNhbGxlciB0byByZXRyeSB3aXRoIG9y
ZGVyIChndWFyYW50ZWVkIHNtYWxsZXIgdGhhbgogICogICAgICAgICAgICAg
dGhlIG9yZGVyIHZhbHVlIHBhc3NlZCBpbikKICAqLwotaW50IGNsZWFyX21t
aW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuX2wsIG1mbl90IG1mbiwKLSAgICAgICAgICAgICAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgb3JkZXIpCitzdGF0aWMgaW50IGNsZWFyX21taW9fcDJt
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuX2ws
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1mbl90IG1mbiwg
dW5zaWduZWQgaW50IG9yZGVyKQogewogICAgIGludCByYyA9IC1FSU5WQUw7
CiAgICAgZ2ZuX3QgZ2ZuID0gX2dmbihnZm5fbCk7Ci0tLSBhL3hlbi9hcmNo
L3g4Ni9tbS9wMm0tcG9kLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS1w
b2QuYwpAQCAtMTI5NywxNyArMTI5NywxNyBAQCBndWVzdF9waHlzbWFwX21h
cmtfcG9wdWxhdGVfb25fZGVtYW5kKHN0CiAKICAgICAgICAgcDJtLT5nZXRf
ZW50cnkocDJtLCBnZm5fYWRkKGdmbiwgaSksICZvdCwgJmEsIDAsICZjdXJf
b3JkZXIsIE5VTEwpOwogICAgICAgICBuID0gMVVMIDw8IG1pbihvcmRlciwg
Y3VyX29yZGVyKTsKLSAgICAgICAgaWYgKCBwMm1faXNfcmFtKG90KSApCisg
ICAgICAgIGlmICggb3QgPT0gcDJtX3BvcHVsYXRlX29uX2RlbWFuZCApCisg
ICAgICAgIHsKKyAgICAgICAgICAgIC8qIENvdW50IGhvdyBtYW55IFBvRCBl
bnRyaWVzIHdlJ2xsIGJlIHJlcGxhY2luZyBpZiBzdWNjZXNzZnVsICovCisg
ICAgICAgICAgICBwb2RfY291bnQgKz0gbjsKKyAgICAgICAgfQorICAgICAg
ICBlbHNlIGlmICggb3QgIT0gcDJtX2ludmFsaWQgJiYgb3QgIT0gcDJtX21t
aW9fZG0gKQogICAgICAgICB7CiAgICAgICAgICAgICBQMk1fREVCVUcoImdm
bl90b19tZm4gcmV0dXJuZWQgdHlwZSAlZCFcbiIsIG90KTsKICAgICAgICAg
ICAgIHJjID0gLUVCVVNZOwogICAgICAgICAgICAgZ290byBvdXQ7CiAgICAg
ICAgIH0KLSAgICAgICAgZWxzZSBpZiAoIG90ID09IHAybV9wb3B1bGF0ZV9v
bl9kZW1hbmQgKQotICAgICAgICB7Ci0gICAgICAgICAgICAvKiBDb3VudCBo
b3cgbWFuIFBvRCBlbnRyaWVzIHdlJ2xsIGJlIHJlcGxhY2luZyBpZiBzdWNj
ZXNzZnVsICovCi0gICAgICAgICAgICBwb2RfY291bnQgKz0gbjsKLSAgICAg
ICAgfQogICAgIH0KIAogICAgIC8qIE5vdywgYWN0dWFsbHkgZG8gdGhlIHR3
by13YXkgbWFwcGluZyAqLwotLS0gYS94ZW4vY29tbW9uL21lbW9yeS5jCisr
KyBiL3hlbi9jb21tb24vbWVtb3J5LmMKQEAgLTMzNiw3ICszMzYsNyBAQCBp
bnQgZ3Vlc3RfcmVtb3ZlX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwKICAgICB9
CiAgICAgaWYgKCBwMm10ID09IHAybV9tbWlvX2RpcmVjdCApCiAgICAgewot
ICAgICAgICByYyA9IGNsZWFyX21taW9fcDJtX2VudHJ5KGQsIGdtZm4sIG1m
biwgUEFHRV9PUkRFUl80Syk7CisgICAgICAgIHJjID0gLUVQRVJNOwogICAg
ICAgICBnb3RvIG91dF9wdXRfZ2ZuOwogICAgIH0KICNlbHNlCkBAIC0xNzI0
LDYgKzE3MjQsMTUgQEAgaW50IGNoZWNrX2dldF9wYWdlX2Zyb21fZ2ZuKHN0
cnVjdCBkb21haQogICAgICAgICByZXR1cm4gLUVBR0FJTjsKICAgICB9CiAj
ZW5kaWYKKyNpZmRlZiBDT05GSUdfWDg2CisgICAgaWYgKCBwMm10ID09IHAy
bV9tbWlvX2RpcmVjdCApCisgICAgeworICAgICAgICBpZiAoIHBhZ2UgKQor
ICAgICAgICAgICAgcHV0X3BhZ2UocGFnZSk7CisKKyAgICAgICAgcmV0dXJu
IC1FUEVSTTsKKyAgICB9CisjZW5kaWYKIAogICAgIGlmICggIXBhZ2UgKQog
ICAgICAgICByZXR1cm4gLUVJTlZBTDsKLS0tIGEveGVuL2luY2x1ZGUvYXNt
LXg4Ni9wMm0uaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L3AybS5oCkBA
IC0xNDMsNyArMTQzLDggQEAgdHlwZWRlZiB1bnNpZ25lZCBpbnQgcDJtX3F1
ZXJ5X3Q7CiAKIC8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQgdXAgdmlh
IHNwZWNpYWwgYWNjZXNzb3JzLiAqLwogI2RlZmluZSBQMk1fU1BFQ0lBTF9U
WVBFUyAoUDJNX0dSQU5UX1RZUEVTIHwgXAotICAgICAgICAgICAgICAgICAg
ICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHAybV90b19tYXNrKHAybV9tYXBfZm9y
ZWlnbikgfCBcCisgICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1fdG9f
bWFzayhwMm1fbW1pb19kaXJlY3QpKQogCiAvKiBWYWxpZCB0eXBlcyBub3Qg
bmVjZXNzYXJpbHkgYXNzb2NpYXRlZCB3aXRoIGEgKHZhbGlkKSBNRk4uICov
CiAjZGVmaW5lIFAyTV9JTlZBTElEX01GTl9UWVBFUyAoUDJNX1BPRF9UWVBF
UyAgICAgICAgICAgICAgICAgIFwKQEAgLTY0MCw4ICs2NDEsNiBAQCBpbnQg
c2V0X2ZvcmVpZ25fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4KIC8qIFNldCBt
bWlvIGFkZHJlc3NlcyBpbiB0aGUgcDJtIHRhYmxlIChmb3IgcGFzcy10aHJv
dWdoKSAqLwogaW50IHNldF9tbWlvX3AybV9lbnRyeShzdHJ1Y3QgZG9tYWlu
ICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgbWZuX3QgbWZuLAogICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgb3JkZXIsIHAybV9hY2Nlc3Nf
dCBhY2Nlc3MpOwotaW50IGNsZWFyX21taW9fcDJtX2VudHJ5KHN0cnVjdCBk
b21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBtZm5fdCBtZm4sCi0gICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVyKTsKIAog
LyogU2V0IGlkZW50aXR5IGFkZHJlc3NlcyBpbiB0aGUgcDJtIHRhYmxlIChm
b3IgcGFzcy10aHJvdWdoKSAqLwogaW50IHNldF9pZGVudGl0eV9wMm1fZW50
cnkoc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm4sCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-0a.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-0a.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBmaXggUG9EIGFjY291bnRpbmcgaW4gZ3Vlc3RfcGh5c21h
cF9hZGRfZW50cnkoKQoKVGhlIGluaXRpYWwgb2JzZXJ2YXRpb24gd2FzIHRo
YXQgdGhlIG1mbl92YWxpZCgpIGNoZWNrIGNvbWVzIHRvbyBsYXRlOgpOZWl0
aGVyIG1mbl9hZGQoKSBub3IgbWZuX3RvX3BhZ2UoKSAobGV0IGFsb25lIGRl
LXJlZmVyZW5jaW5nIHRoZQpyZXN1bHQgb2YgdGhlIGxhdHRlcikgYXJlIHZh
bGlkIGZvciBNRk5zIGZhaWxpbmcgdGhpcyBjaGVjay4gTW92ZSBpdCB1cAph
bmQgLSBub3RpY2luZyB0aGF0IHRoZXJlJ3Mgbm8gY2FsbGVyIGRvaW5nIHNv
IC0gYWxzbyBhZGQgYW4gYXNzZXJ0aW9uCnRoYXQgdGhpcyBzaG91bGQgbmV2
ZXIgcHJvZHVjZSAiZmFsc2UiIGhlcmUuCgpJbiB0dXJuIHRoaXMgd291bGQg
aGF2ZSBtZWFudCB0aGF0IHRoZSAiZWxzZSIgdG8gdGhhdCBpZigpIGNvdWxk
IG5vdyBnbwphd2F5LCB3aGljaCBkaWRuJ3Qgc2VlbSByaWdodCBhdCBhbGwu
IEFuZCBpbmRlZWQsIGNvbnNpZGVyaW5nIGNhbGxlcnMKbGlrZSBtZW1vcnlf
ZXhjaGFuZ2UoKSBvciB2YXJpb3VzIGdyYW50IHRhYmxlIGZ1bmN0aW9ucywg
dGhlIFBvRAphY2NvdW50aW5nIHNob3VsZCBoYXZlIGJlZW4gb3V0c2lkZSBv
ZiB0aGF0IGlmKCkgZnJvbSB0aGUgdmVyeQpiZWdpbm5pbmcuCgpTaWduZWQt
b2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+CkFja2Vk
LWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29t
PgptYXN0ZXIgY29tbWl0OiBhZWEyNzBlM2Y3YzBkYjY5NmM4OGEwZTk0YjFl
Y2U3YWJkMzM5Yzg0Cm1hc3RlciBkYXRlOiAyMDIwLTAyLTIxIDE3OjE0OjM4
ICswMTAwCgotLS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLmMKKysrIGIveGVu
L2FyY2gveDg2L21tL3AybS5jCkBAIC04ODEsNiArODgxLDEyIEBAIGd1ZXN0
X3BoeXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICBpZiAo
IHAybV9pc19mb3JlaWduKHQpICkKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7
CiAKKyAgICBpZiAoICFtZm5fdmFsaWQobWZuKSApCisgICAgeworICAgICAg
ICBBU1NFUlRfVU5SRUFDSEFCTEUoKTsKKyAgICAgICAgcmV0dXJuIC1FSU5W
QUw7CisgICAgfQorCiAgICAgcDJtX2xvY2socDJtKTsKIAogICAgIFAyTV9E
RUJVRygiYWRkaW5nIGdmbj0lI2x4IG1mbj0lI2x4XG4iLCBnZm5feChnZm4p
LCBtZm5feChtZm4pKTsKQEAgLTk4MSwxMiArOTg3LDEzIEBAIGd1ZXN0X3Bo
eXNtYXBfYWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICB9CiAKICAg
ICAvKiBOb3csIGFjdHVhbGx5IGRvIHRoZSB0d28td2F5IG1hcHBpbmcgKi8K
LSAgICBpZiAoIG1mbl92YWxpZChtZm4pICkKKyAgICByYyA9IHAybV9zZXRf
ZW50cnkocDJtLCBnZm4sIG1mbiwgcGFnZV9vcmRlciwgdCwgcDJtLT5kZWZh
dWx0X2FjY2Vzcyk7CisgICAgaWYgKCByYyA9PSAwICkKICAgICB7Ci0gICAg
ICAgIHJjID0gcDJtX3NldF9lbnRyeShwMm0sIGdmbiwgbWZuLCBwYWdlX29y
ZGVyLCB0LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtLT5kZWZh
dWx0X2FjY2Vzcyk7Ci0gICAgICAgIGlmICggcmMgKQotICAgICAgICAgICAg
Z290byBvdXQ7IC8qIEZhaWxlZCB0byB1cGRhdGUgcDJtLCBiYWlsIHdpdGhv
dXQgdXBkYXRpbmcgbTJwLiAqLworICAgICAgICBwb2RfbG9jayhwMm0pOwor
ICAgICAgICBwMm0tPnBvZC5lbnRyeV9jb3VudCAtPSBwb2RfY291bnQ7Cisg
ICAgICAgIEJVR19PTihwMm0tPnBvZC5lbnRyeV9jb3VudCA8IDApOworICAg
ICAgICBwb2RfdW5sb2NrKHAybSk7CiAKICAgICAgICAgaWYgKCAhcDJtX2lz
X2dyYW50KHQpICkKICAgICAgICAgewpAQCAtOTk1LDIyICsxMDAyLDcgQEAg
Z3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFpbiAqZAogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGdmbl94KGdmbl9hZGQo
Z2ZuLCBpKSkpOwogICAgICAgICB9CiAgICAgfQotICAgIGVsc2UKLSAgICB7
Ci0gICAgICAgIGdkcHJpbnRrKFhFTkxPR19XQVJOSU5HLCAiQWRkaW5nIGJh
ZCBtZm4gdG8gcDJtIG1hcCAoJSNseCAtPiAlI2x4KVxuIiwKLSAgICAgICAg
ICAgICAgICAgZ2ZuX3goZ2ZuKSwgbWZuX3gobWZuKSk7Ci0gICAgICAgIHJj
ID0gcDJtX3NldF9lbnRyeShwMm0sIGdmbiwgSU5WQUxJRF9NRk4sIHBhZ2Vf
b3JkZXIsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1faW52YWxp
ZCwgcDJtLT5kZWZhdWx0X2FjY2Vzcyk7Ci0gICAgICAgIGlmICggcmMgPT0g
MCApCi0gICAgICAgIHsKLSAgICAgICAgICAgIHBvZF9sb2NrKHAybSk7Ci0g
ICAgICAgICAgICBwMm0tPnBvZC5lbnRyeV9jb3VudCAtPSBwb2RfY291bnQ7
Ci0gICAgICAgICAgICBCVUdfT04ocDJtLT5wb2QuZW50cnlfY291bnQgPCAw
KTsKLSAgICAgICAgICAgIHBvZF91bmxvY2socDJtKTsKLSAgICAgICAgfQot
ICAgIH0KIAotb3V0OgogICAgIHAybV91bmxvY2socDJtKTsKIAogICAgIHJl
dHVybiByYzsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-0b.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-0b.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBpZ25vcmUgcDJtX3JlbW92ZV9wYWdlKCkncyBy
ZXR1cm4gdmFsdWUKCkl0J3Mgbm90IHZlcnkgbmljZSB0byByZXR1cm4gZnJv
bSBndWVzdF9waHlzbWFwX2FkZF9lbnRyeSgpIGFmdGVyCnBlcmhhcHMgYWxy
ZWFkeSBoYXZpbmcgbWFkZSBzb21lIGNoYW5nZXMgdG8gdGhlIFAyTSwgYnV0
IHRoaXMgaXMgcHJlLQpleGlzdGluZyBwcmFjdGljZSBpbiB0aGUgZnVuY3Rp
b24sIGFuZCBpbW8gYmV0dGVyIHRoYW4gaWdub3JpbmcgZXJyb3JzLgoKVGFr
ZSB0aGUgbGliZXJ0eSBhbmQgcmVwbGFjZSBhbiBtZm5fYWRkKCkgaW5zdGFu
Y2Ugd2l0aCBhIGxvY2FsIHZhcmlhYmxlCmFscmVhZHkgaG9sZGluZyB0aGUg
cmVzdWx0IChhcyBwcm92ZW4gYnkgdGhlIGNoZWNrIGltbWVkaWF0ZWx5IGFo
ZWFkKS4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBz
dXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVycmFudCA8cGF1bC5kdXJy
YW50QGNpdHJpeC5jb20+CkFja2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgptYXN0ZXIgY29tbWl0OiBhNmIwNTFh
ODdhNTg2MzQ3OTY5YmZiYWE2OTI1YWMwZjBjODQ1NDEzCm1hc3RlciBkYXRl
OiAyMDIwLTA0LTAzIDEwOjU2OjEwICswMjAwCgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC03
NzMsOCArNzczLDcgQEAgdm9pZCBwMm1fZmluYWxfdGVhcmRvd24oc3RydWN0
IGRvbWFpbiAqZAogICAgIHAybV90ZWFyZG93bl9ob3N0cDJtKGQpOwogfQog
Ci0KLXN0YXRpYyBpbnQKK3N0YXRpYyBpbnQgX19tdXN0X2NoZWNrCiBwMm1f
cmVtb3ZlX3BhZ2Uoc3RydWN0IHAybV9kb21haW4gKnAybSwgdW5zaWduZWQg
bG9uZyBnZm5fbCwgdW5zaWduZWQgbG9uZyBtZm4sCiAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IHBhZ2Vfb3JkZXIpCiB7CkBAIC05NzksOSArOTc4
LDkgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFpbiAq
ZAogICAgICAgICAgICAgICAgIEFTU0VSVChtZm5fdmFsaWQob21mbikpOwog
ICAgICAgICAgICAgICAgIFAyTV9ERUJVRygib2xkIGdmbj0lI2x4IC0+IG1m
biAlI2x4XG4iLAogICAgICAgICAgICAgICAgICAgICAgICAgICBnZm5feChv
Z2ZuKSAsIG1mbl94KG9tZm4pKTsKLSAgICAgICAgICAgICAgICBpZiAoIG1m
bl9lcShvbWZuLCBtZm5fYWRkKG1mbiwgaSkpICkKLSAgICAgICAgICAgICAg
ICAgICAgcDJtX3JlbW92ZV9wYWdlKHAybSwgZ2ZuX3gob2dmbiksIG1mbl94
KG1mbl9hZGQobWZuLCBpKSksCi0gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAwKTsKKyAgICAgICAgICAgICAgICBpZiAoIG1mbl9lcShv
bWZuLCBtZm5fYWRkKG1mbiwgaSkpICYmCisgICAgICAgICAgICAgICAgICAg
ICAocmMgPSBwMm1fcmVtb3ZlX3BhZ2UocDJtLCBnZm5feChvZ2ZuKSwgbWZu
X3gob21mbiksIDApKSApCisgICAgICAgICAgICAgICAgICAgIGdvdG8gb3V0
OwogICAgICAgICAgICAgfQogICAgICAgICB9CiAgICAgfQpAQCAtMTAwMyw2
ICsxMDAyLDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRv
bWFpbiAqZAogICAgICAgICB9CiAgICAgfQogCisgb3V0OgogICAgIHAybV91
bmxvY2socDJtKTsKIAogICAgIHJldHVybiByYzsKQEAgLTI2OTAsOSArMjY5
MCw5IEBAIGludCBwMm1fY2hhbmdlX2FsdHAybV9nZm4oc3RydWN0IGRvbWFp
bgogICAgIGlmICggZ2ZuX2VxKG5ld19nZm4sIElOVkFMSURfR0ZOKSApCiAg
ICAgewogICAgICAgICBtZm4gPSBhcDJtLT5nZXRfZW50cnkoYXAybSwgb2xk
X2dmbiwgJnQsICZhLCAwLCBOVUxMLCBOVUxMKTsKLSAgICAgICAgaWYgKCBt
Zm5fdmFsaWQobWZuKSApCi0gICAgICAgICAgICBwMm1fcmVtb3ZlX3BhZ2Uo
YXAybSwgZ2ZuX3gob2xkX2dmbiksIG1mbl94KG1mbiksIFBBR0VfT1JERVJf
NEspOwotICAgICAgICByYyA9IDA7CisgICAgICAgIHJjID0gbWZuX3ZhbGlk
KG1mbikKKyAgICAgICAgICAgICA/IHAybV9yZW1vdmVfcGFnZShhcDJtLCBn
Zm5feChvbGRfZ2ZuKSwgbWZuX3gobWZuKSwgUEFHRV9PUkRFUl80SykKKyAg
ICAgICAgICAgICA6IDA7CiAgICAgICAgIGdvdG8gb3V0OwogICAgIH0KIAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-0c.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-0c.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBkb24ndCBhc3NlcnQgdGhhdCB0aGUgcGFzc2VkIGluIE1G
TiBtYXRjaGVzIGZvciBhIHJlbW92ZQoKZ3Vlc3RfcGh5c21hcF9yZW1vdmVf
cGFnZSgpIGdldHMgaGFuZGVkIGFuIE1GTiBmcm9tIHRoZSBvdXRzaWRlLCB5
ZXQKdGFrZXMgdGhlIG5lY2Vzc2FyeSBsb2NrIHRvIHByZXZlbnQgZnVydGhl
ciBjaGFuZ2VzIHRvIHRoZSBHRk4gPC0+IE1GTgptYXBwaW5nIGl0c2VsZi4g
V2hpbGUgc29tZSBjYWxsZXJzLCBpbiBwYXJ0aWN1bGFyIGd1ZXN0X3JlbW92
ZV9wYWdlKCkKKGJ5IHdheSBvZiBoYXZpbmcgY2FsbGVkIGdldF9nZm5fcXVl
cnkoKSksIGhvbGQgdGhlIEdGTiBsb2NrIGFscmVhZHksCnZhcmlvdXMgb3Ro
ZXJzIChtb3N0IG5vdGFibHkgcGVyaGFwcyB0aGUgMm5kIGluc3RhbmNlIGlu
Cnhlbm1lbV9hZGRfdG9fcGh5c21hcF9vbmUoKSkgZG9uJ3QuIFdoaWxlIGl0
IGFsc28gaXMgYW4gb3B0aW9uIHRvIGZpeAphbGwgdGhlIGNhbGxlcnMsIGRl
YWwgd2l0aCB0aGUgaXNzdWUgaW4gcDJtX3JlbW92ZV9wYWdlKCkgaW5zdGVh
ZDoKUmVwbGFjZSB0aGUgQVNTRVJUKCkgYnkgYSBjb25kaXRpb25hbCBhbmQg
c3BsaXQgdGhlIGxvb3AgaW50byB0d28sIHN1Y2gKdGhhdCBhbGwgY2hlY2tp
bmcgZ2V0cyBkb25lIGJlZm9yZSBhbnkgbW9kaWZpY2F0aW9uIHdvdWxkIG9j
Y3VyLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1
c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsLmR1cnJh
bnRAY2l0cml4LmNvbT4KQWNrZWQtYnk6IEFuZHJldyBDb29wZXIgPGFuZHJl
dy5jb29wZXIzQGNpdHJpeC5jb20+Cm1hc3RlciBjb21taXQ6IGM2NWVhMTZk
YmNhZmJlNGZlMjE2OTNiMThmOGMyYTNjNWQxNDYwMGUKbWFzdGVyIGRhdGU6
IDIwMjAtMDQtMDMgMTA6NTY6NTUgKzAyMDAKCi0tLSBhL3hlbi9hcmNoL3g4
Ni9tbS9wMm0uYworKysgYi94ZW4vYXJjaC94ODYvbW0vcDJtLmMKQEAgLTc3
OSw3ICs3NzksNiBAQCBwMm1fcmVtb3ZlX3BhZ2Uoc3RydWN0IHAybV9kb21h
aW4gKnAybSwKIHsKICAgICB1bnNpZ25lZCBsb25nIGk7CiAgICAgZ2ZuX3Qg
Z2ZuID0gX2dmbihnZm5fbCk7Ci0gICAgbWZuX3QgbWZuX3JldHVybjsKICAg
ICBwMm1fdHlwZV90IHQ7CiAgICAgcDJtX2FjY2Vzc190IGE7CiAKQEAgLTc5
MCwxNSArNzg5LDI2IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2Rv
bWFpbiAqcDJtLAogICAgIEFTU0VSVChnZm5fbG9ja2VkX2J5X21lKHAybSwg
Z2ZuKSk7CiAgICAgUDJNX0RFQlVHKCJyZW1vdmluZyBnZm49JSNseCBtZm49
JSNseFxuIiwgZ2ZuX2wsIG1mbik7CiAKKyAgICBmb3IgKCBpID0gMDsgaSA8
ICgxVUwgPDwgcGFnZV9vcmRlcik7ICkKKyAgICB7CisgICAgICAgIHVuc2ln
bmVkIGludCBjdXJfb3JkZXI7CisgICAgICAgIG1mbl90IG1mbl9yZXR1cm4g
PSBwMm0tPmdldF9lbnRyeShwMm0sIGdmbl9hZGQoZ2ZuLCBpKSwgJnQsICZh
LCAwLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgJmN1cl9vcmRlciwgTlVMTCk7CisKKyAgICAgICAgaWYgKCBwMm1faXNf
dmFsaWQodCkgJiYKKyAgICAgICAgICAgICAoIW1mbl92YWxpZChfbWZuKG1m
bikpIHx8IG1mbiArIGkgIT0gbWZuX3gobWZuX3JldHVybikpICkKKyAgICAg
ICAgICAgIHJldHVybiAtRUlMU0VROworCisgICAgICAgIGkgKz0gKDFVTCA8
PCBjdXJfb3JkZXIpIC0gKChnZm5fbCArIGkpICYgKCgxVUwgPDwgY3VyX29y
ZGVyKSAtIDEpKTsKKyAgICB9CisKICAgICBpZiAoIG1mbl92YWxpZChfbWZu
KG1mbikpICkKICAgICB7CiAgICAgICAgIGZvciAoIGkgPSAwOyBpIDwgKDFV
TCA8PCBwYWdlX29yZGVyKTsgaSsrICkKICAgICAgICAgewotICAgICAgICAg
ICAgbWZuX3JldHVybiA9IHAybS0+Z2V0X2VudHJ5KHAybSwgZ2ZuX2FkZChn
Zm4sIGkpLCAmdCwgJmEsIDAsCi0gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgTlVMTCwgTlVMTCk7CisgICAgICAgICAgICBwMm0t
PmdldF9lbnRyeShwMm0sIGdmbl9hZGQoZ2ZuLCBpKSwgJnQsICZhLCAwLCBO
VUxMLCBOVUxMKTsKICAgICAgICAgICAgIGlmICggIXAybV9pc19ncmFudCh0
KSAmJiAhcDJtX2lzX3NoYXJlZCh0KSAmJiAhcDJtX2lzX2ZvcmVpZ24odCkg
KQogICAgICAgICAgICAgICAgIHNldF9ncGZuX2Zyb21fbWZuKG1mbitpLCBJ
TlZBTElEX00yUF9FTlRSWSk7Ci0gICAgICAgICAgICBBU1NFUlQoICFwMm1f
aXNfdmFsaWQodCkgfHwgbWZuICsgaSA9PSBtZm5feChtZm5fcmV0dXJuKSAp
OwogICAgICAgICB9CiAgICAgfQogICAgIHJldHVybiBwMm1fc2V0X2VudHJ5
KHAybSwgZ2ZuLCBJTlZBTElEX01GTiwgcGFnZV9vcmRlciwgcDJtX2ludmFs
aWQsCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9p
b21tdV9hY3BpLmMKQEAgLTExNywxMiArMTE3LDIxIEBAIHN0YXRpYyBzdHJ1
Y3QgYW1kX2lvbW11ICogX19pbml0IGZpbmRfaW8KICAgICByZXR1cm4gTlVM
TDsKIH0KIAotc3RhdGljIHZvaWQgX19pbml0IHJlc2VydmVfaW9tbXVfZXhj
bHVzaW9uX3JhbmdlKAotICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCB1
aW50NjRfdCBiYXNlLCB1aW50NjRfdCBsaW1pdCkKK3N0YXRpYyBpbnQgX19p
bml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKAorICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBs
aW1pdCwKKyAgICBib29sIGFsbCwgYm9vbCBpdywgYm9vbCBpcikKIHsKKyAg
ICBpZiAoICFpciB8fCAhaXcgKQorICAgICAgICByZXR1cm4gLUVQRVJNOwor
CiAgICAgLyogbmVlZCB0byBleHRlbmQgZXhjbHVzaW9uIHJhbmdlPyAqLwog
ICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgKQogICAgIHsKKyAg
ICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9uX2xpbWl0ICsgUEFHRV9TSVpF
IDwgYmFzZSB8fAorICAgICAgICAgICAgIGxpbWl0ICsgUEFHRV9TSVpFIDwg
aW9tbXUtPmV4Y2x1c2lvbl9iYXNlIHx8CisgICAgICAgICAgICAgaW9tbXUt
PmV4Y2x1c2lvbl9hbGxvd19hbGwgIT0gYWxsICkKKyAgICAgICAgICAgIHJl
dHVybiAtRUJVU1k7CisKICAgICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9u
X2Jhc2UgPCBiYXNlICkKICAgICAgICAgICAgIGJhc2UgPSBpb21tdS0+ZXhj
bHVzaW9uX2Jhc2U7CiAgICAgICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9s
aW1pdCA+IGxpbWl0ICkKQEAgLTEzMCwxNiArMTM5LDExIEBAIHN0YXRpYyB2
b2lkIF9faW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2kKICAgICB9CiAKICAg
ICBpb21tdS0+ZXhjbHVzaW9uX2VuYWJsZSA9IElPTU1VX0NPTlRST0xfRU5B
QkxFRDsKKyAgICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IGFsbDsK
ICAgICBpb21tdS0+ZXhjbHVzaW9uX2Jhc2UgPSBiYXNlOwogICAgIGlvbW11
LT5leGNsdXNpb25fbGltaXQgPSBsaW1pdDsKLX0KIAotc3RhdGljIHZvaWQg
X19pbml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlX2FsbCgKLSAg
ICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25lZCBsb25n
IGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQpCi17Ci0gICAgcmVzZXJ2ZV9p
b21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAg
ICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IElPTU1VX0NPTlRST0xf
RU5BQkxFRDsKKyAgICByZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgX19p
bml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoCkBAIC0xNzcsNiAr
MTgxLDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbgogICAgIHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3As
IGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAgICB1
bnNpZ25lZCBpbnQgYmRmOworICAgIGludCByYyA9IDA7CiAKICAgICAvKiBp
cyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5nZSBpbnNpZGUgb2YgSU9NTVUgdmly
dHVhbCBhZGRyZXNzIHNwYWNlPyAqLwogICAgIC8qIG5vdGU6ICdsaW1pdCcg
cGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCkBA
IC0xOTgsMTAgKzIwMywxNSBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rl
cl9leGNsdXNpb25fcmFuCiAgICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3Ag
KQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBpb21tdSAp
Ci0gICAgICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9h
bGwoaW9tbXUsIGJhc2UsIGxpbWl0KTsKKyAgICAgICAgeworICAgICAgICAg
ICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwg
YmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOworICAg
ICAgICAgICAgaWYgKCByYyApCisgICAgICAgICAgICAgICAgYnJlYWs7Cisg
ICAgICAgIH0KICAgICB9CiAKLSAgICByZXR1cm4gMDsKKyAgICByZXR1cm4g
cmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW5nZV9mb3JfZGV2aWNlKApAQCAtMjEyLDYgKzIyMiw3IEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICB1bnNp
Z25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgdTE2IHJlcTsKKyAgICBp
bnQgcmMgPSAwOwogCiAgICAgaW9tbXUgPSBmaW5kX2lvbW11X2Zvcl9kZXZp
Y2Uoc2VnLCBiZGYpOwogICAgIGlmICggIWlvbW11ICkKQEAgLTI0MSwxMiAr
MjUyLDEzIEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW4KICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ug
c2V0dGluZ3MgZm9yIGRldmljZSAqLwogICAgIGlmICggbGltaXQgPj0gaW9t
bXVfdG9wICApCiAgICAgewotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQpOworICAgICAgICByYyA9
IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBs
aW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBmYWxzZSAvKiBhbGwgKi8sIGl3LCBpcik7CiAgICAgICAgIGl2cnNf
bWFwcGluZ3NbYmRmXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1ZTsKICAg
ICAgICAgaXZyc19tYXBwaW5nc1tyZXFdLmR0ZV9hbGxvd19leGNsdXNpb24g
PSB0cnVlOwogICAgIH0KIAotICAgIHJldHVybiAwOworICAgIHJldHVybiBy
YzsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbmdlX2Zvcl9pb21tdV9kZXZpY2VzKApAQCAtMjU2LDYgKzI2OCw3IEBA
IHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAg
ICB1bnNpZ25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7
CiAgICAgdW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOworICAgIGlu
dCByYyA9IDA7CiAKICAgICAvKiBpcyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5n
ZSBpbnNpZGUgb2YgSU9NTVUgdmlydHVhbCBhZGRyZXNzIHNwYWNlPyAqLwog
ICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8g
YmUgcGFnZS1hbGlnbmVkICovCkBAIC0yODYsOCArMjk5LDEwIEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KIAogICAgIC8q
IHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwog
ICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkKLSAgICAgICAgcmVzZXJ2
ZV9pb21tdV9leGNsdXNpb25fcmFuZ2VfYWxsKGlvbW11LCBiYXNlLCBsaW1p
dCk7Ci0gICAgcmV0dXJuIDA7CisgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21t
dV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUgLyog
YWxsICovLCBpdywgaXIpOworCisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9zZWxlY3QoCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdo
L2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtMTQ2LDMyICsxNDYsNDggQEAgc3Rh
dGljIGludCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9leGNsdXNpbwogICAgIHJl
dHVybiAwOwogfQogCi1zdGF0aWMgdm9pZCBfX2luaXQgcmVzZXJ2ZV91bml0
eV9tYXBfZm9yX2RldmljZSgKLSAgICB1MTYgc2VnLCB1MTYgYmRmLCB1bnNp
Z25lZCBsb25nIGJhc2UsCi0gICAgdW5zaWduZWQgbG9uZyBsZW5ndGgsIHU4
IGl3LCB1OCBpcikKK3N0YXRpYyBpbnQgX19pbml0IHJlc2VydmVfdW5pdHlf
bWFwX2Zvcl9kZXZpY2UoCisgICAgdWludDE2X3Qgc2VnLCB1aW50MTZfdCBi
ZGYsIHVuc2lnbmVkIGxvbmcgYmFzZSwKKyAgICB1bnNpZ25lZCBsb25nIGxl
bmd0aCwgYm9vbCBpdywgYm9vbCBpcikKIHsKICAgICBzdHJ1Y3QgaXZyc19t
YXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHNl
Zyk7Ci0gICAgdW5zaWduZWQgbG9uZyBvbGRfdG9wLCBuZXdfdG9wOworICAg
IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwID0gaXZyc19tYXBw
aW5nc1tiZGZdLnVuaXR5X21hcDsKIAotICAgIC8qIG5lZWQgdG8gZXh0ZW5k
IHVuaXR5LW1hcHBlZCByYW5nZT8gKi8KLSAgICBpZiAoIGl2cnNfbWFwcGlu
Z3NbYmRmXS51bml0eV9tYXBfZW5hYmxlICkKKyAgICAvKiBDaGVjayBmb3Ig
b3ZlcmxhcHMuICovCisgICAgZm9yICggOyB1bml0eV9tYXA7IHVuaXR5X21h
cCA9IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewotICAgICAgICBvbGRfdG9w
ID0gaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQgKwotICAg
ICAgICAgICAgaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2VfbGVuZ3Ro
OwotICAgICAgICBuZXdfdG9wID0gYmFzZSArIGxlbmd0aDsKLSAgICAgICAg
aWYgKCBvbGRfdG9wID4gbmV3X3RvcCApCi0gICAgICAgICAgICBuZXdfdG9w
ID0gb2xkX3RvcDsKLSAgICAgICAgaWYgKCBpdnJzX21hcHBpbmdzW2JkZl0u
YWRkcl9yYW5nZV9zdGFydCA8IGJhc2UgKQotICAgICAgICAgICAgYmFzZSA9
IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3JhbmdlX3N0YXJ0OwotICAgICAg
ICBsZW5ndGggPSBuZXdfdG9wIC0gYmFzZTsKLSAgICB9Ci0KLSAgICAvKiBl
eHRlbmQgci93IHBlcm1pc3Npb21zIGFuZCBrZWVwIGFnZ3JlZ2F0ZSAqLwot
ICAgIGl2cnNfbWFwcGluZ3NbYmRmXS53cml0ZV9wZXJtaXNzaW9uID0gaXc7
Ci0gICAgaXZyc19tYXBwaW5nc1tiZGZdLnJlYWRfcGVybWlzc2lvbiA9IGly
OwotICAgIGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXBfZW5hYmxlID0g
dHJ1ZTsKLSAgICBpdnJzX21hcHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9zdGFy
dCA9IGJhc2U7Ci0gICAgaXZyc19tYXBwaW5nc1tiZGZdLmFkZHJfcmFuZ2Vf
bGVuZ3RoID0gbGVuZ3RoOworICAgICAgICAvKgorICAgICAgICAgKiBFeGFj
dCBtYXRjaGVzIGFyZSBva2F5LiBUaGlzIGNhbiBpbiBwYXJ0aWN1bGFyIGhh
cHBlbiB3aGVuCisgICAgICAgICAqIHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5n
ZV9mb3JfZGV2aWNlKCkgY2FsbHMgaGVyZSB0d2ljZSBmb3IgdGhlCisgICAg
ICAgICAqIHNhbWUgKHMsYixkLGYpLgorICAgICAgICAgKi8KKyAgICAgICAg
aWYgKCBiYXNlID09IHVuaXR5X21hcC0+YWRkciAmJiBsZW5ndGggPT0gdW5p
dHlfbWFwLT5sZW5ndGggJiYKKyAgICAgICAgICAgICBpciA9PSB1bml0eV9t
YXAtPnJlYWQgJiYgaXcgPT0gdW5pdHlfbWFwLT53cml0ZSApCisgICAgICAg
ICAgICByZXR1cm4gMDsKKworICAgICAgICBpZiAoIHVuaXR5X21hcC0+YWRk
ciArIHVuaXR5X21hcC0+bGVuZ3RoID4gYmFzZSAmJgorICAgICAgICAgICAg
IGJhc2UgKyBsZW5ndGggPiB1bml0eV9tYXAtPmFkZHIgKQorICAgICAgICB7
CisgICAgICAgICAgICBBTURfSU9NTVVfREVCVUcoIklWTUQgRXJyb3I6IG92
ZXJsYXAgWyVseCwlbHgpIHZzIFslbHgsJWx4KVxuIiwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBiYXNlLCBiYXNlICsgbGVuZ3RoLCB1bml0eV9t
YXAtPmFkZHIsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5pdHlf
bWFwLT5hZGRyICsgdW5pdHlfbWFwLT5sZW5ndGgpOworICAgICAgICAgICAg
cmV0dXJuIC1FUEVSTTsKKyAgICAgICAgfQorICAgIH0KKworICAgIC8qIFBv
cHVsYXRlIGFuZCBpbnNlcnQgYSBuZXcgdW5pdHkgbWFwLiAqLworICAgIHVu
aXR5X21hcCA9IHhtYWxsb2Moc3RydWN0IGl2cnNfdW5pdHlfbWFwKTsKKyAg
ICBpZiAoICF1bml0eV9tYXAgKQorICAgICAgICByZXR1cm4gLUVOT01FTTsK
KworICAgIHVuaXR5X21hcC0+cmVhZCA9IGlyOworICAgIHVuaXR5X21hcC0+
d3JpdGUgPSBpdzsKKyAgICB1bml0eV9tYXAtPmFkZHIgPSBiYXNlOworICAg
IHVuaXR5X21hcC0+bGVuZ3RoID0gbGVuZ3RoOworICAgIHVuaXR5X21hcC0+
bmV4dCA9IGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXA7CisgICAgaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcCA9IHVuaXR5X21hcDsKKworICAg
IHJldHVybiAwOwogfQogCiBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9l
eGNsdXNpb25fcmFuZ2VfZm9yX2FsbF9kZXZpY2VzKApAQCAtMTk0LDEzICsy
MTAsMTMgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbgogICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwogICAg
ICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVz
IGZvciBkZXZpY2VzICovCiAgICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJp
ZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAg
ICBmb3IgKCBiZGYgPSAwOyBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYr
KyApCi0gICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNl
KHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7CisgICAgICAgIGZv
ciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBi
ZGYrKyApCisgICAgICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zv
cl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsKICAg
ICAgICAgLyogcHVzaCAnYmFzZScganVzdCBvdXRzaWRlIG9mIHZpcnR1YWwg
YWRkcmVzcyBzcGFjZSAqLwogICAgICAgICBiYXNlID0gaW9tbXVfdG9wOwog
ICAgIH0KICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ug
c2V0dGluZ3MgKi8KLSAgICBpZiAoIGxpbWl0ID49IGlvbW11X3RvcCApCisg
ICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkKICAgICB7CiAg
ICAgICAgIGZvcl9lYWNoX2FtZF9pb21tdSggaW9tbXUgKQogICAgICAgICB7
CkBAIC0yNDIsMTUgKzI1OCwxNSBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdp
c3Rlcl9leGNsdXNpb25fcmFuCiAgICAgICAgIGxlbmd0aCA9IHJhbmdlX3Rv
cCAtIGJhc2U7CiAgICAgICAgIC8qIHJlc2VydmUgdW5pdHktbWFwcGVkIHBh
Z2UgZW50cmllcyBmb3IgZGV2aWNlICovCiAgICAgICAgIC8qIG5vdGU6IHRo
ZXNlIGVudHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAq
LwotICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywg
YmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0gICAgICAgIHJlc2VydmVf
dW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2UsIGxlbmd0aCwg
aXcsIGlyKTsKKyAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3Jf
ZGV2aWNlKHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcikgPzoKKyAg
ICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywg
cmVxLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7CiAKICAgICAgICAgLyogcHVz
aCAnYmFzZScganVzdCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRkcmVzcyBzcGFj
ZSAqLwogICAgICAgICBiYXNlID0gaW9tbXVfdG9wOwogICAgIH0KIAogICAg
IC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyBm
b3IgZGV2aWNlICovCi0gICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3AgICkK
KyAgICBpZiAoICFyYyAmJiBsaW1pdCA+PSBpb21tdV90b3AgICkKICAgICB7
CiAgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2Uo
aW9tbXUsIGJhc2UsIGxpbWl0LAogICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGZhbHNlIC8qIGFsbCAqLywgaXcsIGlyKTsK
QEAgLTI4MSwxNSArMjk3LDE1IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lz
dGVyX2V4Y2x1c2lvbl9yYW4KICAgICAgICAgbGVuZ3RoID0gcmFuZ2VfdG9w
IC0gYmFzZTsKICAgICAgICAgLyogcmVzZXJ2ZSByL3cgdW5pdHktbWFwcGVk
IHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAqLwogICAgICAgICAvKiBub3Rl
OiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNpb24gcmFu
Z2UgKi8KLSAgICAgICAgZm9yICggYmRmID0gMDsgYmRmIDwgaXZyc19iZGZf
ZW50cmllczsgYmRmKysgKQorICAgICAgICBmb3IgKCBiZGYgPSAwOyAhcmMg
JiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysgKQogICAgICAgICB7
CiAgICAgICAgICAgICBpZiAoIGlvbW11ID09IGZpbmRfaW9tbXVfZm9yX2Rl
dmljZShpb21tdS0+c2VnLCBiZGYpICkKICAgICAgICAgICAgIHsKLSAgICAg
ICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11
LT5zZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLAotICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyKTsKICAgICAgICAg
ICAgICAgICByZXEgPSBnZXRfaXZyc19tYXBwaW5ncyhpb21tdS0+c2VnKVti
ZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7Ci0gICAgICAgICAgICAgICAgcmVzZXJ2
ZV91bml0eV9tYXBfZm9yX2RldmljZShpb21tdS0+c2VnLCByZXEsIGJhc2Us
IGxlbmd0aCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGl3LCBpcik7CisgICAgICAgICAgICAgICAgcmMgPSByZXNl
cnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZiwgYmFz
ZSwgbGVuZ3RoLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpID86CisgICAgICAgICAgICAgICAg
ICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcs
IHJlcSwgYmFzZSwgbGVuZ3RoLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIpOwogICAgICAgICAg
ICAgfQogICAgICAgICB9CiAKQEAgLTI5OCw3ICszMTQsNyBAQCBzdGF0aWMg
aW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgfQogCiAg
ICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdz
ICovCi0gICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3AgKQorICAgIGlmICgg
IXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCApCiAgICAgICAgIHJjID0gcmVz
ZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0
LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHRydWUgLyogYWxsICovLCBpdywgaXIpOwogCi0tLSBhL3hlbi9kcml2ZXJz
L3Bhc3N0aHJvdWdoL2FtZC9wY2lfYW1kX2lvbW11LmMKKysrIGIveGVuL2Ry
aXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYwpAQCAtMzY5
LDE1ICszNjksMTcgQEAgc3RhdGljIGludCBhbWRfaW9tbXVfYXNzaWduX2Rl
dmljZShzdHJ1YwogICAgIHN0cnVjdCBpdnJzX21hcHBpbmdzICppdnJzX21h
cHBpbmdzID0gZ2V0X2l2cnNfbWFwcGluZ3MocGRldi0+c2VnKTsKICAgICBp
bnQgYmRmID0gUENJX0JERjIocGRldi0+YnVzLCBkZXZmbik7CiAgICAgaW50
IHJlcV9pZCA9IGdldF9kbWFfcmVxdWVzdG9yX2lkKHBkZXYtPnNlZywgYmRm
KTsKKyAgICBjb25zdCBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKnVuaXR5X21h
cDsKIAotICAgIGlmICggaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21h
cF9lbmFibGUgKQorICAgIGZvciAoIHVuaXR5X21hcCA9IGl2cnNfbWFwcGlu
Z3NbcmVxX2lkXS51bml0eV9tYXA7IHVuaXR5X21hcDsKKyAgICAgICAgICB1
bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQogICAgIHsKLSAgICAgICAg
YW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKLSAgICAgICAg
ICAgIGQsCi0gICAgICAgICAgICBpdnJzX21hcHBpbmdzW3JlcV9pZF0uYWRk
cl9yYW5nZV9zdGFydCwKLSAgICAgICAgICAgIGl2cnNfbWFwcGluZ3NbcmVx
X2lkXS5hZGRyX3JhbmdlX2xlbmd0aCwKLSAgICAgICAgICAgIGl2cnNfbWFw
cGluZ3NbcmVxX2lkXS53cml0ZV9wZXJtaXNzaW9uLAotICAgICAgICAgICAg
aXZyc19tYXBwaW5nc1tyZXFfaWRdLnJlYWRfcGVybWlzc2lvbik7CisgICAg
ICAgIGludCByYyA9IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9t
YXAoCisgICAgICAgICAgICAgICAgICAgICBkLCB1bml0eV9tYXAtPmFkZHIs
IHVuaXR5X21hcC0+bGVuZ3RoLAorICAgICAgICAgICAgICAgICAgICAgdW5p
dHlfbWFwLT53cml0ZSwgdW5pdHlfbWFwLT5yZWFkKTsKKworICAgICAgICBp
ZiAoIHJjICkKKyAgICAgICAgICAgIHJldHVybiByYzsKICAgICB9CiAKICAg
ICByZXR1cm4gcmVhc3NpZ25fZGV2aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2
Zm4sIHBkZXYpOwotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L2FtZC1pb21t
dS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvYW1kLWlvbW11LmgKQEAg
LTEwNSwyMCArMTA1LDI0IEBAIHN0cnVjdCBhbWRfaW9tbXUgewogICAgIHN0
cnVjdCBsaXN0X2hlYWQgYXRzX2RldmljZXM7CiB9OwogCitzdHJ1Y3QgaXZy
c191bml0eV9tYXAgeworICAgIGJvb2wgcmVhZDoxOworICAgIGJvb2wgd3Jp
dGU6MTsKKyAgICBwYWRkcl90IGFkZHI7CisgICAgdW5zaWduZWQgbG9uZyBs
ZW5ndGg7CisgICAgc3RydWN0IGl2cnNfdW5pdHlfbWFwICpuZXh0OworfTsK
Kwogc3RydWN0IGl2cnNfbWFwcGluZ3MgewogICAgIHVpbnQxNl90IGR0ZV9y
ZXF1ZXN0b3JfaWQ7CiAgICAgYm9vbCB2YWxpZDoxOwogICAgIGJvb2wgZHRl
X2FsbG93X2V4Y2x1c2lvbjoxOwotICAgIGJvb2wgdW5pdHlfbWFwX2VuYWJs
ZToxOwotICAgIGJvb2wgd3JpdGVfcGVybWlzc2lvbjoxOwotICAgIGJvb2wg
cmVhZF9wZXJtaXNzaW9uOjE7CiAKICAgICAvKiBpdmhkIGRldmljZSBkYXRh
IHNldHRpbmdzICovCiAgICAgdWludDhfdCBkZXZpY2VfZmxhZ3M7CiAKLSAg
ICB1bnNpZ25lZCBsb25nIGFkZHJfcmFuZ2Vfc3RhcnQ7Ci0gICAgdW5zaWdu
ZWQgbG9uZyBhZGRyX3JhbmdlX2xlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lv
bW11ICppb21tdTsKKyAgICBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKnVuaXR5
X21hcDsKIAogICAgIC8qIHBlciBkZXZpY2UgaW50ZXJydXB0IHJlbWFwcGlu
ZyB0YWJsZSAqLwogICAgIHZvaWQgKmludHJlbWFwX3RhYmxlOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0tZXB0
LmMKQEAgLTY3OCw3ICs2NzgsNyBAQCBlcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmCiAgICAgdWludDhfdCBpcGF0ID0gMDsKICAg
ICBib29sX3QgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMTsKICAgICBib29s
X3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsKLSAgICB1bnNpZ25lZCBpbnQgaW9t
bXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIG1mbik7Cisg
ICAgdW5zaWduZWQgaW50IGlvbW11X2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBwMm1hLCBtZm4pOwogICAgIGJvb2xfdCBuZWVkc19zeW5j
ID0gMTsKICAgICBlcHRfZW50cnlfdCBvbGRfZW50cnkgPSB7IC5lcHRlID0g
MCB9OwogICAgIGVwdF9lbnRyeV90IG5ld19lbnRyeSA9IHsgLmVwdGUgPSAw
IH07CkBAIC04MDUsOCArODA1LDggQEAgZXB0X3NldF9lbnRyeShzdHJ1Y3Qg
cDJtX2RvbWFpbiAqcDJtLCBnZgogCiAgICAgICAgIC8qIFNhZmUgdG8gcmVh
ZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2UgaG9sZCB0aGUgcDJtIGxvY2sgKi8K
ICAgICAgICAgaWYgKCBlcHRfZW50cnktPm1mbiA9PSBuZXdfZW50cnkubWZu
ICYmCi0gICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhlcHRfZW50
cnktPnNhX3AybXQsIF9tZm4oZXB0X2VudHJ5LT5tZm4pKSA9PQotICAgICAg
ICAgICAgIGlvbW11X2ZsYWdzICkKKyAgICAgICAgICAgICBwMm1fZ2V0X2lv
bW11X2ZsYWdzKGVwdF9lbnRyeS0+c2FfcDJtdCwgZXB0X2VudHJ5LT5hY2Nl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBfbWZuKGVw
dF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVfZmxhZ3MgKQogICAgICAgICAgICAg
bmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMDsKIAogICAgICAgICBlcHRfcDJt
X3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3X2VudHJ5LCBwMm10LCBwMm1hKTsK
LS0tIGEveGVuL2FyY2gveDg2L21tL3AybS1wdC5jCisrKyBiL3hlbi9hcmNo
L3g4Ni9tbS9wMm0tcHQuYwpAQCAtNDgwLDYgKzQ4MCwxNiBAQCBpbnQgcDJt
X3B0X2hhbmRsZV9kZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAgICAgcmV0dXJu
IHJjOwogfQogCisvKiBSZWNvbnN0cnVjdCBhIGZha2UgcDJtX2FjY2Vzc190
IGZyb20gc3RvcmVkIFBURSBmbGFncy4gKi8KK3N0YXRpYyBwMm1fYWNjZXNz
X3QgcDJtX2ZsYWdzX3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQgZmxhZ3MpCit7
CisgICAgaWYgKCBmbGFncyAmIF9QQUdFX1BSRVNFTlQgKQorICAgICAgICBy
ZXR1cm4gcDJtX2FjY2Vzc19uOworCisgICAgLyogTm8gbmVlZCB0byBsb29r
IGF0IF9QQUdFX05YIGZvciBub3cuICovCisgICAgcmV0dXJuIGZsYWdzICYg
X1BBR0VfUlcgPyBwMm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vzc19yOworfQor
CiAvKiBDaGVja3Mgb25seSBhcHBsaWNhYmxlIHRvIGVudHJpZXMgd2l0aCBv
cmRlciA+IFBBR0VfT1JERVJfNEsgKi8KIHN0YXRpYyB2b2lkIGNoZWNrX2Vu
dHJ5KG1mbl90IG1mbiwgcDJtX3R5cGVfdCBuZXcsIHAybV90eXBlX3Qgb2xk
LAogICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVy
KQpAQCAtNTE0LDcgKzUyNCw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0
IHAybV9kb21haW4gKnAybSwKICAgICBsMl9wZ2VudHJ5X3QgbDJlX2NvbnRl
bnQ7CiAgICAgbDNfcGdlbnRyeV90IGwzZV9jb250ZW50OwogICAgIGludCBy
YzsKLSAgICB1bnNpZ25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdzID0gcDJtX2dl
dF9pb21tdV9mbGFncyhwMm10LCBtZm4pOworICAgIHVuc2lnbmVkIGludCBp
b21tdV9wdGVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIHAy
bWEsIG1mbik7CiAgICAgLyoKICAgICAgKiBvbGRfbWZuIGFuZCBpb21tdV9v
bGRfZmxhZ3MgY29udHJvbCBwb3NzaWJsZSBmbHVzaC91cGRhdGUgbmVlZHMg
b24gdGhlCiAgICAgICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1c2ggd2hlbiBN
Rk4gb3IgZmxhZ3MgKGkuZS4gcGVybWlzc2lvbnMpIGNoYW5nZS4KQEAgLTU3
Nyw2ICs1ODcsNyBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9t
YWluICpwMm0sCiAgICAgICAgICAgICAgICAgb2xkX21mbiA9IGwxZV9nZXRf
cGZuKCpwMm1fZW50cnkpOwogICAgICAgICAgICAgICAgIGlvbW11X29sZF9m
bGFncyA9CiAgICAgICAgICAgICAgICAgICAgIHAybV9nZXRfaW9tbXVfZmxh
Z3MocDJtX2ZsYWdzX3RvX3R5cGUoZmxhZ3MpLAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9mbGFnc190b19hY2Nlc3Mo
ZmxhZ3MpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIF9tZm4ob2xkX21mbikpOwogICAgICAgICAgICAgfQogICAgICAgICAg
ICAgZWxzZQpAQCAtNjE5LDkgKzYzMCwxMCBAQCBwMm1fcHRfc2V0X2VudHJ5
KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRSSUVTKTsKICAg
ICAgICAgQVNTRVJUKHAybV9lbnRyeSk7CiAgICAgICAgIG9sZF9tZm4gPSBs
MWVfZ2V0X3BmbigqcDJtX2VudHJ5KTsKKyAgICAgICAgZmxhZ3MgPSBsMWVf
Z2V0X2ZsYWdzKCpwMm1fZW50cnkpOwogICAgICAgICBpb21tdV9vbGRfZmxh
Z3MgPQotICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxh
Z3NfdG9fdHlwZShsMWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpKSwKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7Cisg
ICAgICAgICAgICBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9mbGFnc190b190
eXBlKGZsYWdzKSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
cDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksIF9tZm4ob2xkX21mbikpOwog
CiAgICAgICAgIGlmICggbWZuX3ZhbGlkKG1mbikgfHwgcDJtX2FsbG93c19p
bnZhbGlkX21mbihwMm10KSApCiAgICAgICAgICAgICBlbnRyeV9jb250ZW50
ID0gcDJtX2wxZV9mcm9tX3BmbihtZm5feChtZm4pLApAQCAtNjQ5LDYgKzY2
MSw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAy
bSwKICAgICAgICAgICAgICAgICBvbGRfbWZuID0gbDFlX2dldF9wZm4oKnAy
bV9lbnRyeSk7CiAgICAgICAgICAgICAgICAgaW9tbXVfb2xkX2ZsYWdzID0K
ICAgICAgICAgICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1f
ZmxhZ3NfdG9fdHlwZShmbGFncyksCisgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyks
CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21m
bihvbGRfbWZuKSk7CiAgICAgICAgICAgICB9CiAgICAgICAgICAgICBlbHNl
Ci0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2lu
Y2x1ZGUvYXNtLXg4Ni9wMm0uaApAQCAtOTA4LDcgKzkwOCw4IEBAIHN0YXRp
YyBpbmxpbmUgdm9pZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnUKIC8qCiAgKiBw
Mm0gdHlwZSB0byBJT01NVSBmbGFncwogICovCi1zdGF0aWMgaW5saW5lIHVu
c2lnbmVkIGludCBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90eXBlX3QgcDJt
dCwgbWZuX3QgbWZuKQorc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJt
X2dldF9pb21tdV9mbGFncyhwMm1fdHlwZV90IHAybXQsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9hY2Nl
c3NfdCBwMm1hLCBtZm5fdCBtZm4pCiB7CiAgICAgdW5zaWduZWQgaW50IGZs
YWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC0x
MzUxLDcgKzEzNTEsNyBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShz
dHJ1Y3QgZG9tYWluCiAgICAgICAgIGlmICggIWlzX2lvbW11X2VuYWJsZWQo
ZCkgKQogICAgICAgICAgICAgcmV0dXJuIDA7CiAgICAgICAgIHJldHVybiBp
b21tdV9sZWdhY3lfbWFwKGQsIF9kZm4oZ2ZuX2wpLCBfbWZuKGdmbl9sKSwg
UEFHRV9PUkRFUl80SywKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgSU9NTVVGX3JlYWRhYmxlIHwgSU9NTVVGX3dyaXRhYmxlKTsKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2FjY2Vzc190b19pb21t
dV9mbGFncyhwMm1hKSk7CiAgICAgfQogCiAgICAgZ2ZuX2xvY2socDJtLCBn
Zm4sIDApOwotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9t
bXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUu
YwpAQCAtNDIsMTIgKzQyLDYgQEAKICNpbmNsdWRlICJ2dGQuaCIKICNpbmNs
dWRlICIuLi9hdHMuaCIKIAotc3RydWN0IG1hcHBlZF9ybXJyIHsKLSAgICBz
dHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7Ci0gICAgdTY0IGJhc2UsIGVuZDsKLSAg
ICB1bnNpZ25lZCBpbnQgY291bnQ7Ci19OwotCiAvKiBQb3NzaWJsZSB1bmZp
bHRlcmVkIExBUElDL01TSSBtZXNzYWdlcyBmcm9tIHVudHJ1c3RlZCBzb3Vy
Y2VzPyAqLwogYm9vbCBfX3JlYWRfbW9zdGx5IHVudHJ1c3RlZF9tc2k7CiAK
QEAgLTE3OTksMTcgKzE3OTMsMTIgQEAgb3V0Ogogc3RhdGljIHZvaWQgaW9t
bXVfZG9tYWluX3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpCiB7CiAgICAg
c3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7Ci0gICAg
c3RydWN0IG1hcHBlZF9ybXJyICptcm1yciwgKnRtcDsKICAgICBjb25zdCBz
dHJ1Y3QgYWNwaV9kcmhkX3VuaXQgKmRyaGQ7CiAKICAgICBpZiAoIGxpc3Rf
ZW1wdHkoJmFjcGlfZHJoZF91bml0cykgKQogICAgICAgICByZXR1cm47CiAK
LSAgICBsaXN0X2Zvcl9lYWNoX2VudHJ5X3NhZmUgKCBtcm1yciwgdG1wLCAm
aGQtPmFyY2gubWFwcGVkX3JtcnJzLCBsaXN0ICkKLSAgICB7Ci0gICAgICAg
IGxpc3RfZGVsKCZtcm1yci0+bGlzdCk7Ci0gICAgICAgIHhmcmVlKG1ybXJy
KTsKLSAgICB9CisgICAgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKGQp
OwogCiAgICAgQVNTRVJUKGlzX2lvbW11X2VuYWJsZWQoZCkpOwogCkBAIC0x
OTYzLDc0ICsxOTUyLDYgQEAgc3RhdGljIHZvaWQgaW9tbXVfc2V0X3BnZChz
dHJ1Y3QgZG9tYWluCiAgICAgICAgIHBhZ2V0YWJsZV9nZXRfcGFkZHIocGFn
ZXRhYmxlX2Zyb21fbWZuKHBnZF9tZm4pKTsKIH0KIAotc3RhdGljIGludCBy
bXJyX2lkZW50aXR5X21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwgYm9vbF90
IG1hcCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBhY3BpX3JtcnJfdW5pdCAqcm1yciwKLSAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHUzMiBmbGFnKQotewotICAgIHVuc2lnbmVk
IGxvbmcgYmFzZV9wZm4gPSBybXJyLT5iYXNlX2FkZHJlc3MgPj4gUEFHRV9T
SElGVF80SzsKLSAgICB1bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdFX0FM
SUdOXzRLKHJtcnItPmVuZF9hZGRyZXNzKSA+PiBQQUdFX1NISUZUXzRLOwot
ICAgIHN0cnVjdCBtYXBwZWRfcm1yciAqbXJtcnI7Ci0gICAgc3RydWN0IGRv
bWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7Ci0KLSAgICBBU1NFUlQo
cGNpZGV2c19sb2NrZWQoKSk7Ci0gICAgQVNTRVJUKHJtcnItPmJhc2VfYWRk
cmVzcyA8IHJtcnItPmVuZF9hZGRyZXNzKTsKLQotICAgIC8qCi0gICAgICog
Tm8gbmVlZCB0byBhY3F1aXJlIGhkLT5hcmNoLm1hcHBpbmdfbG9jazogQm90
aCBpbnNlcnRpb24gYW5kIHJlbW92YWwKLSAgICAgKiBnZXQgZG9uZSB3aGls
ZSBob2xkaW5nIHBjaWRldnNfbG9jay4KLSAgICAgKi8KLSAgICBsaXN0X2Zv
cl9lYWNoX2VudHJ5KCBtcm1yciwgJmhkLT5hcmNoLm1hcHBlZF9ybXJycywg
bGlzdCApCi0gICAgewotICAgICAgICBpZiAoIG1ybXJyLT5iYXNlID09IHJt
cnItPmJhc2VfYWRkcmVzcyAmJgotICAgICAgICAgICAgIG1ybXJyLT5lbmQg
PT0gcm1yci0+ZW5kX2FkZHJlc3MgKQotICAgICAgICB7Ci0gICAgICAgICAg
ICBpbnQgcmV0ID0gMDsKLQotICAgICAgICAgICAgaWYgKCBtYXAgKQotICAg
ICAgICAgICAgewotICAgICAgICAgICAgICAgICsrbXJtcnItPmNvdW50Owot
ICAgICAgICAgICAgICAgIHJldHVybiAwOwotICAgICAgICAgICAgfQotCi0g
ICAgICAgICAgICBpZiAoIC0tbXJtcnItPmNvdW50ICkKLSAgICAgICAgICAg
ICAgICByZXR1cm4gMDsKLQotICAgICAgICAgICAgd2hpbGUgKCBiYXNlX3Bm
biA8IGVuZF9wZm4gKQotICAgICAgICAgICAgewotICAgICAgICAgICAgICAg
IGlmICggY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KGQsIGJhc2VfcGZuKSAp
Ci0gICAgICAgICAgICAgICAgICAgIHJldCA9IC1FTlhJTzsKLSAgICAgICAg
ICAgICAgICBiYXNlX3BmbisrOwotICAgICAgICAgICAgfQotCi0gICAgICAg
ICAgICBsaXN0X2RlbCgmbXJtcnItPmxpc3QpOwotICAgICAgICAgICAgeGZy
ZWUobXJtcnIpOwotICAgICAgICAgICAgcmV0dXJuIHJldDsKLSAgICAgICAg
fQotICAgIH0KLQotICAgIGlmICggIW1hcCApCi0gICAgICAgIHJldHVybiAt
RU5PRU5UOwotCi0gICAgd2hpbGUgKCBiYXNlX3BmbiA8IGVuZF9wZm4gKQot
ICAgIHsKLSAgICAgICAgaW50IGVyciA9IHNldF9pZGVudGl0eV9wMm1fZW50
cnkoZCwgYmFzZV9wZm4sIHAybV9hY2Nlc3NfcncsIGZsYWcpOwotCi0gICAg
ICAgIGlmICggZXJyICkKLSAgICAgICAgICAgIHJldHVybiBlcnI7Ci0gICAg
ICAgIGJhc2VfcGZuKys7Ci0gICAgfQotCi0gICAgbXJtcnIgPSB4bWFsbG9j
KHN0cnVjdCBtYXBwZWRfcm1ycik7Ci0gICAgaWYgKCAhbXJtcnIgKQotICAg
ICAgICByZXR1cm4gLUVOT01FTTsKLSAgICBtcm1yci0+YmFzZSA9IHJtcnIt
PmJhc2VfYWRkcmVzczsKLSAgICBtcm1yci0+ZW5kID0gcm1yci0+ZW5kX2Fk
ZHJlc3M7Ci0gICAgbXJtcnItPmNvdW50ID0gMTsKLSAgICBsaXN0X2FkZF90
YWlsKCZtcm1yci0+bGlzdCwgJmhkLT5hcmNoLm1hcHBlZF9ybXJycyk7Ci0K
LSAgICByZXR1cm4gMDsKLX0KLQogc3RhdGljIGludCBpbnRlbF9pb21tdV9h
ZGRfZGV2aWNlKHU4IGRldmZuLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikKIHsK
ICAgICBzdHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnI7CkBAIC0yMDYyLDcg
KzE5ODMsOSBAQCBzdGF0aWMgaW50IGludGVsX2lvbW11X2FkZF9kZXZpY2Uo
dTggZGV2CiAgICAgICAgICAgICAgKiBTaW5jZSBSTVJScyBhcmUgYWx3YXlz
IHJlc2VydmVkIGluIHRoZSBlODIwIG1hcCBmb3IgdGhlIGhhcmR3YXJlCiAg
ICAgICAgICAgICAgKiBkb21haW4sIHRoZXJlIHNob3VsZG4ndCBiZSBhIGNv
bmZsaWN0LgogICAgICAgICAgICAgICovCi0gICAgICAgICAgICByZXQgPSBy
bXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWluLCAxLCBybXJyLCAw
KTsKKyAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50aXR5X21hcHBpbmco
cGRldi0+ZG9tYWluLCBwMm1fYWNjZXNzX3J3LAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5iYXNlX2FkZHJlc3Ms
IHJtcnItPmVuZF9hZGRyZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAwKTsKICAgICAgICAgICAgIGlmICggcmV0ICkK
ICAgICAgICAgICAgICAgICBkcHJpbnRrKFhFTkxPR19FUlIgVlREUFJFRklY
LCAiZCVkOiBSTVJSIG1hcHBpbmcgZmFpbGVkXG4iLAogICAgICAgICAgICAg
ICAgICAgICAgICAgcGRldi0+ZG9tYWluLT5kb21haW5faWQpOwpAQCAtMjEw
Nyw3ICsyMDMwLDggQEAgc3RhdGljIGludCBpbnRlbF9pb21tdV9yZW1vdmVf
ZGV2aWNlKHU4CiAgICAgICAgICAqIEFueSBmbGFnIGlzIG5vdGhpbmcgdG8g
Y2xlYXIgdGhlc2UgbWFwcGluZ3MgYnV0IGhlcmUKICAgICAgICAgICogaXRz
IGFsd2F5cyBzYWZlIGFuZCBzdHJpY3QgdG8gc2V0IDAuCiAgICAgICAgICAq
LwotICAgICAgICBybXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWlu
LCAwLCBybXJyLCAwKTsKKyAgICAgICAgaW9tbXVfaWRlbnRpdHlfbWFwcGlu
ZyhwZGV2LT5kb21haW4sIHAybV9hY2Nlc3NfeCwgcm1yci0+YmFzZV9hZGRy
ZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmVu
ZF9hZGRyZXNzLCAwKTsKICAgICB9CiAKICAgICByZXR1cm4gZG9tYWluX2Nv
bnRleHRfdW5tYXAocGRldi0+ZG9tYWluLCBkZXZmbiwgcGRldik7CkBAIC0y
MzA2LDcgKzIyMzAsOCBAQCBzdGF0aWMgdm9pZCBfX2h3ZG9tX2luaXQgc2V0
dXBfaHdkb21fcm1yCiAgICAgICAgICAqIGRvbWFpbiwgdGhlcmUgc2hvdWxk
bid0IGJlIGEgY29uZmxpY3QuIFNvIGl0cyBhbHdheXMgc2FmZSBhbmQKICAg
ICAgICAgICogc3RyaWN0IHRvIHNldCAwLgogICAgICAgICAgKi8KLSAgICAg
ICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKGQsIDEsIHJtcnIsIDAp
OworICAgICAgICByZXQgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQsIHAy
bV9hY2Nlc3NfcncsIHJtcnItPmJhc2VfYWRkcmVzcywKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5lbmRfYWRkcmVzcywg
MCk7CiAgICAgICAgIGlmICggcmV0ICkKICAgICAgICAgICAgIGRwcmludGso
WEVOTE9HX0VSUiBWVERQUkVGSVgsCiAgICAgICAgICAgICAgICAgICAgICAi
SU9NTVU6IG1hcHBpbmcgcmVzZXJ2ZWQgcmVnaW9uIGZhaWxlZFxuIik7CkBA
IC0yNDY1LDcgKzIzOTAsOSBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2Rldmlj
ZV9vd25lcnNoaXAoCiAgICAgICAgICAgICAgICAgICogQW55IFJNUlIgZmxh
ZyBpcyBhbHdheXMgaWdub3JlZCB3aGVuIHJlbW92ZSBhIGRldmljZSwKICAg
ICAgICAgICAgICAgICAgKiBidXQgaXRzIGFsd2F5cyBzYWZlIGFuZCBzdHJp
Y3QgdG8gc2V0IDAuCiAgICAgICAgICAgICAgICAgICovCi0gICAgICAgICAg
ICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKHNvdXJjZSwgMCwg
cm1yciwgMCk7CisgICAgICAgICAgICAgICAgcmV0ID0gaW9tbXVfaWRlbnRp
dHlfbWFwcGluZyhzb3VyY2UsIHAybV9hY2Nlc3NfeCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmJhc2Vf
YWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCAwKTsKICAgICAgICAgICAg
ICAgICBpZiAoIHJldCAhPSAtRU5PRU5UICkKICAgICAgICAgICAgICAgICAg
ICAgcmV0dXJuIHJldDsKICAgICAgICAgICAgIH0KQEAgLTI1NjIsNyArMjQ4
OSw4IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfYXNzaWduX2RldmljZSgK
ICAgICAgICAgICAgICBQQ0lfQlVTKGJkZikgPT0gYnVzICYmCiAgICAgICAg
ICAgICAgUENJX0RFVkZOMihiZGYpID09IGRldmZuICkKICAgICAgICAgewot
ICAgICAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKGQsIDEs
IHJtcnIsIGZsYWcpOworICAgICAgICAgICAgcmV0ID0gaW9tbXVfaWRlbnRp
dHlfbWFwcGluZyhkLCBwMm1fYWNjZXNzX3J3LCBybXJyLT5iYXNlX2FkZHJl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHJtcnItPmVuZF9hZGRyZXNzLCBmbGFnKTsKICAgICAgICAgICAgIGlmICgg
cmV0ICkKICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICBpbnQgcmM7
Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21tdS5jCisr
KyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21tdS5jCkBAIC0x
MjcsNyArMTI3LDcgQEAgaW50IGFyY2hfaW9tbXVfZG9tYWluX2luaXQoc3Ry
dWN0IGRvbWFpbgogICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhkID0gZG9t
X2lvbW11KGQpOwogCiAgICAgc3Bpbl9sb2NrX2luaXQoJmhkLT5hcmNoLm1h
cHBpbmdfbG9jayk7Ci0gICAgSU5JVF9MSVNUX0hFQUQoJmhkLT5hcmNoLm1h
cHBlZF9ybXJycyk7CisgICAgSU5JVF9MSVNUX0hFQUQoJmhkLT5hcmNoLmlk
ZW50aXR5X21hcHMpOwogCiAgICAgcmV0dXJuIDA7CiB9CkBAIC0xMzYsNiAr
MTM2LDk5IEBAIHZvaWQgYXJjaF9pb21tdV9kb21haW5fZGVzdHJveShzdHJ1
Y3QgZG8KIHsKIH0KIAorc3RydWN0IGlkZW50aXR5X21hcCB7CisgICAgc3Ry
dWN0IGxpc3RfaGVhZCBsaXN0OworICAgIHBhZGRyX3QgYmFzZSwgZW5kOwor
ICAgIHAybV9hY2Nlc3NfdCBhY2Nlc3M7CisgICAgdW5zaWduZWQgaW50IGNv
dW50OworfTsKKworaW50IGlvbW11X2lkZW50aXR5X21hcHBpbmcoc3RydWN0
IGRvbWFpbiAqZCwgcDJtX2FjY2Vzc190IHAybWEsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgZW5kLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGZsYWcpCit7
CisgICAgdW5zaWduZWQgbG9uZyBiYXNlX3BmbiA9IGJhc2UgPj4gUEFHRV9T
SElGVF80SzsKKyAgICB1bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdFX0FM
SUdOXzRLKGVuZCkgPj4gUEFHRV9TSElGVF80SzsKKyAgICBzdHJ1Y3QgaWRl
bnRpdHlfbWFwICptYXA7CisgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQg
PSBkb21faW9tbXUoZCk7CisKKyAgICBBU1NFUlQocGNpZGV2c19sb2NrZWQo
KSk7CisgICAgQVNTRVJUKGJhc2UgPCBlbmQpOworCisgICAgLyoKKyAgICAg
KiBObyBuZWVkIHRvIGFjcXVpcmUgaGQtPmFyY2gubWFwcGluZ19sb2NrOiBC
b3RoIGluc2VydGlvbiBhbmQgcmVtb3ZhbAorICAgICAqIGdldCBkb25lIHdo
aWxlIGhvbGRpbmcgcGNpZGV2c19sb2NrLgorICAgICAqLworICAgIGxpc3Rf
Zm9yX2VhY2hfZW50cnkoIG1hcCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMs
IGxpc3QgKQorICAgIHsKKyAgICAgICAgaWYgKCBtYXAtPmJhc2UgPT0gYmFz
ZSAmJiBtYXAtPmVuZCA9PSBlbmQgKQorICAgICAgICB7CisgICAgICAgICAg
ICBpbnQgcmV0ID0gMDsKKworICAgICAgICAgICAgaWYgKCBwMm1hICE9IHAy
bV9hY2Nlc3NfeCApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAg
aWYgKCBtYXAtPmFjY2VzcyAhPSBwMm1hICkKKyAgICAgICAgICAgICAgICAg
ICAgcmV0dXJuIC1FQUREUklOVVNFOworICAgICAgICAgICAgICAgICsrbWFw
LT5jb3VudDsKKyAgICAgICAgICAgICAgICByZXR1cm4gMDsKKyAgICAgICAg
ICAgIH0KKworICAgICAgICAgICAgaWYgKCAtLW1hcC0+Y291bnQgKQorICAg
ICAgICAgICAgICAgIHJldHVybiAwOworCisgICAgICAgICAgICB3aGlsZSAo
IGJhc2VfcGZuIDwgZW5kX3BmbiApCisgICAgICAgICAgICB7CisgICAgICAg
ICAgICAgICAgaWYgKCBjbGVhcl9pZGVudGl0eV9wMm1fZW50cnkoZCwgYmFz
ZV9wZm4pICkKKyAgICAgICAgICAgICAgICAgICAgcmV0ID0gLUVOWElPOwor
ICAgICAgICAgICAgICAgIGJhc2VfcGZuKys7CisgICAgICAgICAgICB9CisK
KyAgICAgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOworICAgICAgICAg
ICAgeGZyZWUobWFwKTsKKworICAgICAgICAgICAgcmV0dXJuIHJldDsKKyAg
ICAgICAgfQorCisgICAgICAgIGlmICggZW5kID49IG1hcC0+YmFzZSAmJiBt
YXAtPmVuZCA+PSBiYXNlICkKKyAgICAgICAgICAgIHJldHVybiAtRUFERFJJ
TlVTRTsKKyAgICB9CisKKyAgICBpZiAoIHAybWEgPT0gcDJtX2FjY2Vzc194
ICkKKyAgICAgICAgcmV0dXJuIC1FTk9FTlQ7CisKKyAgICB3aGlsZSAoIGJh
c2VfcGZuIDwgZW5kX3BmbiApCisgICAgeworICAgICAgICBpbnQgZXJyID0g
c2V0X2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3BmbiwgcDJtYSwgZmxh
Zyk7CisKKyAgICAgICAgaWYgKCBlcnIgKQorICAgICAgICAgICAgcmV0dXJu
IGVycjsKKyAgICAgICAgYmFzZV9wZm4rKzsKKyAgICB9CisKKyAgICBtYXAg
PSB4bWFsbG9jKHN0cnVjdCBpZGVudGl0eV9tYXApOworICAgIGlmICggIW1h
cCApCisgICAgICAgIHJldHVybiAtRU5PTUVNOworICAgIG1hcC0+YmFzZSA9
IGJhc2U7CisgICAgbWFwLT5lbmQgPSBlbmQ7CisgICAgbWFwLT5hY2Nlc3Mg
PSBwMm1hOworICAgIG1hcC0+Y291bnQgPSAxOworICAgIGxpc3RfYWRkX3Rh
aWwoJm1hcC0+bGlzdCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMpOworCisg
ICAgcmV0dXJuIDA7Cit9CisKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFwX3Rl
YXJkb3duKHN0cnVjdCBkb21haW4gKmQpCit7CisgICAgc3RydWN0IGRvbWFp
bl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7CisgICAgc3RydWN0IGlkZW50
aXR5X21hcCAqbWFwLCAqdG1wOworCisgICAgbGlzdF9mb3JfZWFjaF9lbnRy
eV9zYWZlICggbWFwLCB0bXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzLCBs
aXN0ICkKKyAgICB7CisgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOwor
ICAgICAgICB4ZnJlZShtYXApOworICAgIH0KK30KKwogc3RhdGljIGJvb2wg
X19od2RvbV9pbml0IGh3ZG9tX2lvbW11X21hcChjb25zdCBzdHJ1Y3QgZG9t
YWluICpkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBsb25nIHBmbiwKICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBtYXhfcGZuKQot
LS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L2lvbW11LmgKKysrIGIveGVuL2lu
Y2x1ZGUvYXNtLXg4Ni9pb21tdS5oCkBAIC0xNiw2ICsxNiw3IEBACiAKICNp
bmNsdWRlIDx4ZW4vZXJybm8uaD4KICNpbmNsdWRlIDx4ZW4vbGlzdC5oPgor
I2luY2x1ZGUgPHhlbi9tZW1fYWNjZXNzLmg+CiAjaW5jbHVkZSA8eGVuL3Nw
aW5sb2NrLmg+CiAjaW5jbHVkZSA8YXNtL2FwaWNkZWYuaD4KICNpbmNsdWRl
IDxhc20vcHJvY2Vzc29yLmg+CkBAIC00OSw3ICs1MCw3IEBAIHN0cnVjdCBh
cmNoX2lvbW11CiAgICAgc3BpbmxvY2tfdCBtYXBwaW5nX2xvY2s7ICAgICAg
ICAgICAgLyogaW8gcGFnZSB0YWJsZSBsb2NrICovCiAgICAgaW50IGFnYXc7
ICAgICAvKiBhZGp1c3RlZCBndWVzdCBhZGRyZXNzIHdpZHRoLCAwIGlzIGxl
dmVsIDIgMzAtYml0ICovCiAgICAgdTY0IGlvbW11X2JpdG1hcDsgICAgICAg
ICAgICAgIC8qIGJpdG1hcCBvZiBpb21tdShzKSB0aGF0IHRoZSBkb21haW4g
dXNlcyAqLwotICAgIHN0cnVjdCBsaXN0X2hlYWQgbWFwcGVkX3JtcnJzOwor
ICAgIHN0cnVjdCBsaXN0X2hlYWQgaWRlbnRpdHlfbWFwczsKIAogICAgIC8q
IGFtZCBpb21tdSBzdXBwb3J0ICovCiAgICAgaW50IHBhZ2luZ19tb2RlOwpA
QCAtMTEyLDYgKzExMywxMSBAQCBzdGF0aWMgaW5saW5lIHZvaWQgaW9tbXVf
ZGlzYWJsZV94MmFwaWMoCiAgICAgICAgIGlvbW11X29wcy5kaXNhYmxlX3gy
YXBpYygpOwogfQogCitpbnQgaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhzdHJ1
Y3QgZG9tYWluICpkLCBwMm1fYWNjZXNzX3QgcDJtYSwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBlbmQsCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxhZyk7
Cit2b2lkIGlvbW11X2lkZW50aXR5X21hcF90ZWFyZG93bihzdHJ1Y3QgZG9t
YWluICpkKTsKKwogZXh0ZXJuIGJvb2wgdW50cnVzdGVkX21zaTsKIAogaW50
IHBpX3VwZGF0ZV9pcnRlKGNvbnN0IHN0cnVjdCBwaV9kZXNjICpwaV9kZXNj
LCBjb25zdCBzdHJ1Y3QgcGlycSAqcGlycSwKLS0tIGEveGVuL2luY2x1ZGUv
YXNtLXg4Ni9wMm0uaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L3AybS5o
CkBAIC05MDUsNiArOTA1LDM0IEBAIHN0cnVjdCBwMm1fZG9tYWluICpwMm1f
Z2V0X2FsdHAybShzdHJ1Y3QKIHN0YXRpYyBpbmxpbmUgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpIHt9CiAj
ZW5kaWYKIAorLyogcDJtIGFjY2VzcyB0byBJT01NVSBmbGFncyAqLworc3Rh
dGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJtX2FjY2Vzc190b19pb21tdV9m
bGFncyhwMm1fYWNjZXNzX3QgcDJtYSkKK3sKKyAgICBzd2l0Y2ggKCBwMm1h
ICkKKyAgICB7CisgICAgY2FzZSBwMm1fYWNjZXNzX3J3OgorICAgIGNhc2Ug
cDJtX2FjY2Vzc19yd3g6CisgICAgICAgIHJldHVybiBJT01NVUZfcmVhZGFi
bGUgfCBJT01NVUZfd3JpdGFibGU7CisKKyAgICBjYXNlIHAybV9hY2Nlc3Nf
cjoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcng6CisgICAgY2FzZSBwMm1fYWNj
ZXNzX3J4MnJ3OgorICAgICAgICByZXR1cm4gSU9NTVVGX3JlYWRhYmxlOwor
CisgICAgY2FzZSBwMm1fYWNjZXNzX3c6CisgICAgY2FzZSBwMm1fYWNjZXNz
X3d4OgorICAgICAgICByZXR1cm4gSU9NTVVGX3dyaXRhYmxlOworCisgICAg
Y2FzZSBwMm1fYWNjZXNzX246CisgICAgY2FzZSBwMm1fYWNjZXNzX3g6Cisg
ICAgY2FzZSBwMm1fYWNjZXNzX24ycnd4OgorICAgICAgICByZXR1cm4gMDsK
KyAgICB9CisKKyAgICBBU1NFUlRfVU5SRUFDSEFCTEUoKTsKKyAgICByZXR1
cm4gMDsKK30KKwogLyoKICAqIHAybSB0eXBlIHRvIElPTU1VIGZsYWdzCiAg
Ki8KQEAgLTkyNiw5ICs5NTQsMTAgQEAgc3RhdGljIGlubGluZSB1bnNpZ25l
ZCBpbnQgcDJtX2dldF9pb21tdQogICAgICAgICBmbGFncyA9IElPTU1VRl9y
ZWFkYWJsZTsKICAgICAgICAgYnJlYWs7CiAgICAgY2FzZSBwMm1fbW1pb19k
aXJlY3Q6Ci0gICAgICAgIGZsYWdzID0gSU9NTVVGX3JlYWRhYmxlOwotICAg
ICAgICBpZiAoICFyYW5nZXNldF9jb250YWluc19zaW5nbGV0b24obW1pb19y
b19yYW5nZXMsIG1mbl94KG1mbikpICkKLSAgICAgICAgICAgIGZsYWdzIHw9
IElPTU1VRl93cml0YWJsZTsKKyAgICAgICAgZmxhZ3MgPSBwMm1fYWNjZXNz
X3RvX2lvbW11X2ZsYWdzKHAybWEpOworICAgICAgICBpZiAoIChmbGFncyAm
IElPTU1VRl93cml0YWJsZSkgJiYKKyAgICAgICAgICAgICByYW5nZXNldF9j
b250YWluc19zaW5nbGV0b24obW1pb19yb19yYW5nZXMsIG1mbl94KG1mbikp
ICkKKyAgICAgICAgICAgIGZsYWdzICY9IH5JT01NVUZfd3JpdGFibGU7CiAg
ICAgICAgIGJyZWFrOwogICAgIGRlZmF1bHQ6CiAgICAgICAgIGZsYWdzID0g
MDsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfbWFwLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lv
bW11X21hcC5jCkBAIC00MzAsMzggKzQzMCw0OSBAQCBpbnQgYW1kX2lvbW11
X2ZsdXNoX2lvdGxiX2FsbChzdHJ1Y3QgZG9tCiAgICAgcmV0dXJuIDA7CiB9
CiAKLWludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKHN0
cnVjdCBkb21haW4gKmRvbWFpbiwKLSAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHBhZGRyX3QgcGh5c19hZGRyLAotICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBz
aXplLCBpbnQgaXcsIGludCBpcikKK2ludCBhbWRfaW9tbXVfcmVzZXJ2ZV9k
b21haW5fdW5pdHlfbWFwKHN0cnVjdCBkb21haW4gKmQsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaXZy
c191bml0eV9tYXAgKm1hcCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKQogewotICAgIHVuc2ln
bmVkIGxvbmcgbnBhZ2VzLCBpOwotICAgIHVuc2lnbmVkIGxvbmcgZ2ZuOwot
ICAgIHVuc2lnbmVkIGludCBmbGFncyA9ICEhaXI7Ci0gICAgdW5zaWduZWQg
aW50IGZsdXNoX2ZsYWdzID0gMDsKLSAgICBpbnQgcnQgPSAwOwotCi0gICAg
aWYgKCBpdyApCi0gICAgICAgIGZsYWdzIHw9IElPTU1VRl93cml0YWJsZTsK
LQotICAgIG5wYWdlcyA9IHJlZ2lvbl90b19wYWdlcyhwaHlzX2FkZHIsIHNp
emUpOwotICAgIGdmbiA9IHBoeXNfYWRkciA+PiBQQUdFX1NISUZUOwotICAg
IGZvciAoIGkgPSAwOyBpIDwgbnBhZ2VzOyBpKysgKQorICAgIGludCByYzsK
KworICAgIGlmICggZCA9PSBkb21faW8gKQorICAgICAgICByZXR1cm4gMDsK
KworICAgIGZvciAoIHJjID0gMDsgIXJjICYmIG1hcDsgbWFwID0gbWFwLT5u
ZXh0ICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGxvbmcgZnJhbWUgPSBn
Zm4gKyBpOworICAgICAgICBwMm1fYWNjZXNzX3QgcDJtYSA9IHAybV9hY2Nl
c3NfbjsKKworICAgICAgICBpZiAoIG1hcC0+cmVhZCApCisgICAgICAgICAg
ICBwMm1hIHw9IHAybV9hY2Nlc3NfcjsKKyAgICAgICAgaWYgKCBtYXAtPndy
aXRlICkKKyAgICAgICAgICAgIHAybWEgfD0gcDJtX2FjY2Vzc193OwogCi0g
ICAgICAgIHJ0ID0gYW1kX2lvbW11X21hcF9wYWdlKGRvbWFpbiwgX2Rmbihm
cmFtZSksIF9tZm4oZnJhbWUpLCBmbGFncywKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgJmZsdXNoX2ZsYWdzKTsKLSAgICAgICAgaWYgKCBy
dCAhPSAwICkKLSAgICAgICAgICAgIGJyZWFrOworICAgICAgICByYyA9IGlv
bW11X2lkZW50aXR5X21hcHBpbmcoZCwgcDJtYSwgbWFwLT5hZGRyLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRyICsg
bWFwLT5sZW5ndGggLSAxLCBmbGFnKTsKICAgICB9CiAKLSAgICAvKiBVc2Ug
d2hpbGUtYnJlYWsgdG8gYXZvaWQgY29tcGlsZXIgd2FybmluZyAqLwotICAg
IHdoaWxlICggZmx1c2hfZmxhZ3MgJiYKLSAgICAgICAgICAgIGFtZF9pb21t
dV9mbHVzaF9pb3RsYl9wYWdlcyhkb21haW4sIF9kZm4oZ2ZuKSwKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBucGFnZXMsIGZs
dXNoX2ZsYWdzKSApCi0gICAgICAgIGJyZWFrOworICAgIHJldHVybiByYzsK
K30KKworaW50IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV91bm1h
cChzdHJ1Y3QgZG9tYWluICpkLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaXZyc191bml0eV9tYXAg
Km1hcCkKK3sKKyAgICBpbnQgcmM7CisKKyAgICBpZiAoIGQgPT0gZG9tX2lv
ICkKKyAgICAgICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCByYyA9IDA7IG1h
cDsgbWFwID0gbWFwLT5uZXh0ICkKKyAgICB7CisgICAgICAgIGludCByZXQg
PSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQsIHAybV9hY2Nlc3NfeCwgbWFw
LT5hZGRyLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBtYXAtPmFkZHIgKyBtYXAtPmxlbmd0aCAtIDEsIDApOworCisgICAg
ICAgIGlmICggcmV0ICYmIHJldCAhPSAtRU5PRU5UICYmICFyYyApCisgICAg
ICAgICAgICByYyA9IHJldDsKKyAgICB9CiAKLSAgICByZXR1cm4gcnQ7Cisg
ICAgcmV0dXJuIHJjOwogfQogCiBpbnQgX19pbml0IGFtZF9pb21tdV9xdWFy
YW50aW5lX2luaXQoc3RydWN0IGRvbWFpbiAqZCkKLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0z
MzAsNiArMzMwLDcgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IGRvbWFpbgogICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwogICAg
IGludCBiZGYsIHJjOwogICAgIHN0cnVjdCBkb21haW5faW9tbXUgKnQgPSBk
b21faW9tbXUodGFyZ2V0KTsKKyAgICBjb25zdCBzdHJ1Y3QgaXZyc19tYXBw
aW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHBkZXYt
PnNlZyk7CiAKICAgICBiZGYgPSBQQ0lfQkRGMihwZGV2LT5idXMsIHBkZXYt
PmRldmZuKTsKICAgICBpb21tdSA9IGZpbmRfaW9tbXVfZm9yX2RldmljZShw
ZGV2LT5zZWcsIGJkZik7CkBAIC0zNDQsMTAgKzM0NSwyNCBAQCBzdGF0aWMg
aW50IHJlYXNzaWduX2RldmljZShzdHJ1Y3QgZG9tYWluCiAKICAgICBhbWRf
aW9tbXVfZGlzYWJsZV9kb21haW5fZGV2aWNlKHNvdXJjZSwgaW9tbXUsIGRl
dmZuLCBwZGV2KTsKIAotICAgIGlmICggZGV2Zm4gPT0gcGRldi0+ZGV2Zm4g
KQorICAgIC8qCisgICAgICogSWYgdGhlIGRldmljZSBiZWxvbmdzIHRvIHRo
ZSBoYXJkd2FyZSBkb21haW4sIGFuZCBpdCBoYXMgYSB1bml0eSBtYXBwaW5n
LAorICAgICAqIGRvbid0IHJlbW92ZSBpdCBmcm9tIHRoZSBoYXJkd2FyZSBk
b21haW4sIGJlY2F1c2UgQklPUyBtYXkgcmVmZXJlbmNlIHRoYXQKKyAgICAg
KiBtYXBwaW5nLgorICAgICAqLworICAgIGlmICggIWlzX2hhcmR3YXJlX2Rv
bWFpbihzb3VyY2UpICkKKyAgICB7CisgICAgICAgIHJjID0gYW1kX2lvbW11
X3Jlc2VydmVfZG9tYWluX3VuaXR5X3VubWFwKAorICAgICAgICAgICAgICAg
ICBzb3VyY2UsCisgICAgICAgICAgICAgICAgIGl2cnNfbWFwcGluZ3NbZ2V0
X2RtYV9yZXF1ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYpXS51bml0eV9tYXAp
OworICAgICAgICBpZiAoIHJjICkKKyAgICAgICAgICAgIHJldHVybiByYzsK
KyAgICB9CisKKyAgICBpZiAoIGRldmZuID09IHBkZXYtPmRldmZuICYmIHBk
ZXYtPmRvbWFpbiAhPSBkb21faW8gKQogICAgIHsKLSAgICAgICAgbGlzdF9t
b3ZlKCZwZGV2LT5kb21haW5fbGlzdCwgJnRhcmdldC0+cGRldl9saXN0KTsK
LSAgICAgICAgcGRldi0+ZG9tYWluID0gdGFyZ2V0OworICAgICAgICBsaXN0
X21vdmUoJnBkZXYtPmRvbWFpbl9saXN0LCAmZG9tX2lvLT5wZGV2X2xpc3Qp
OworICAgICAgICBwZGV2LT5kb21haW4gPSBkb21faW87CiAgICAgfQogCiAg
ICAgcmMgPSBhbGxvY2F0ZV9kb21haW5fcmVzb3VyY2VzKHQpOwpAQCAtMzU5
LDYgKzM3NCwxMiBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2RldmljZShzdHJ1
Y3QgZG9tYWluCiAgICAgICAgICAgICAgICAgICAgIHBkZXYtPnNlZywgcGRl
di0+YnVzLCBQQ0lfU0xPVChkZXZmbiksIFBDSV9GVU5DKGRldmZuKSwKICAg
ICAgICAgICAgICAgICAgICAgc291cmNlLT5kb21haW5faWQsIHRhcmdldC0+
ZG9tYWluX2lkKTsKIAorICAgIGlmICggZGV2Zm4gPT0gcGRldi0+ZGV2Zm4g
JiYgcGRldi0+ZG9tYWluICE9IHRhcmdldCApCisgICAgeworICAgICAgICBs
aXN0X21vdmUoJnBkZXYtPmRvbWFpbl9saXN0LCAmdGFyZ2V0LT5wZGV2X2xp
c3QpOworICAgICAgICBwZGV2LT5kb21haW4gPSB0YXJnZXQ7CisgICAgfQor
CiAgICAgcmV0dXJuIDA7CiB9CiAKQEAgLTM2OSwyMCArMzkwLDI4IEBAIHN0
YXRpYyBpbnQgYW1kX2lvbW11X2Fzc2lnbl9kZXZpY2Uoc3RydWMKICAgICBz
dHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJz
X21hcHBpbmdzKHBkZXYtPnNlZyk7CiAgICAgaW50IGJkZiA9IFBDSV9CREYy
KHBkZXYtPmJ1cywgZGV2Zm4pOwogICAgIGludCByZXFfaWQgPSBnZXRfZG1h
X3JlcXVlc3Rvcl9pZChwZGV2LT5zZWcsIGJkZik7Ci0gICAgY29uc3Qgc3Ry
dWN0IGl2cnNfdW5pdHlfbWFwICp1bml0eV9tYXA7CisgICAgaW50IHJjID0g
YW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAgICAg
ICAgICAgICAgZCwgaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcCwg
ZmxhZyk7CisKKyAgICBpZiAoICFyYyApCisgICAgICAgIHJjID0gcmVhc3Np
Z25fZGV2aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2Zm4sIHBkZXYpOwogCi0g
ICAgZm9yICggdW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVu
aXR5X21hcDsgdW5pdHlfbWFwOwotICAgICAgICAgIHVuaXR5X21hcCA9IHVu
aXR5X21hcC0+bmV4dCApCisgICAgaWYgKCByYyAmJiAhaXNfaGFyZHdhcmVf
ZG9tYWluKGQpICkKICAgICB7Ci0gICAgICAgIGludCByYyA9IGFtZF9pb21t
dV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoCi0gICAgICAgICAgICAgICAg
ICAgICBkLCB1bml0eV9tYXAtPmFkZHIsIHVuaXR5X21hcC0+bGVuZ3RoLAot
ICAgICAgICAgICAgICAgICAgICAgdW5pdHlfbWFwLT53cml0ZSwgdW5pdHlf
bWFwLT5yZWFkKTsKKyAgICAgICAgaW50IHJldCA9IGFtZF9pb21tdV9yZXNl
cnZlX2RvbWFpbl91bml0eV91bm1hcCgKKyAgICAgICAgICAgICAgICAgICAg
ICBkLCBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwKTsKIAotICAg
ICAgICBpZiAoIHJjICkKLSAgICAgICAgICAgIHJldHVybiByYzsKKyAgICAg
ICAgaWYgKCByZXQgKQorICAgICAgICB7CisgICAgICAgICAgICBwcmludGso
WEVOTE9HX0VSUiAiQU1ELVZpOiAiCisgICAgICAgICAgICAgICAgICAgInVu
aXR5LXVubWFwIGZvciAlcGQvJTA0eDolMDJ4OiUwMnguJXUgZmFpbGVkICgl
ZClcbiIsCisgICAgICAgICAgICAgICAgICAgZCwgcGRldi0+c2VnLCBwZGV2
LT5idXMsCisgICAgICAgICAgICAgICAgICAgUENJX1NMT1QoZGV2Zm4pLCBQ
Q0lfRlVOQyhkZXZmbiksIHJldCk7CisgICAgICAgICAgICBkb21haW5fY3Jh
c2goZCk7CisgICAgICAgIH0KICAgICB9CiAKLSAgICByZXR1cm4gcmVhc3Np
Z25fZGV2aWNlKHBkZXYtPmRvbWFpbiwgZCwgZGV2Zm4sIHBkZXYpOworICAg
IHJldHVybiByYzsKIH0KIAogc3RhdGljIHZvaWQgZGVhbGxvY2F0ZV9uZXh0
X3BhZ2VfdGFibGUoc3RydWN0IHBhZ2VfaW5mbyAqcGcsIGludCBsZXZlbCkK
QEAgLTQ0MSw2ICs0NzAsNyBAQCBzdGF0aWMgdm9pZCBkZWFsbG9jYXRlX2lv
bW11X3BhZ2VfdGFibGVzCiAKIHN0YXRpYyB2b2lkIGFtZF9pb21tdV9kb21h
aW5fZGVzdHJveShzdHJ1Y3QgZG9tYWluICpkKQogeworICAgIGlvbW11X2lk
ZW50aXR5X21hcF90ZWFyZG93bihkKTsKICAgICBkZWFsbG9jYXRlX2lvbW11
X3BhZ2VfdGFibGVzKGQpOwogICAgIGFtZF9pb21tdV9mbHVzaF9hbGxfcGFn
ZXMoZCk7CiB9Ci0tLSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvaHZtL3N2bS9h
bWQtaW9tbXUtcHJvdG8uaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L2h2
bS9zdm0vYW1kLWlvbW11LXByb3RvLmgKQEAgLTY0LDggKzY0LDEwIEBAIGlu
dCBfX211c3RfY2hlY2sgYW1kX2lvbW11X3VubWFwX3BhZ2Uoc3QKICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50
ICpmbHVzaF9mbGFncyk7CiBpbnQgX19tdXN0X2NoZWNrIGFtZF9pb21tdV9h
bGxvY19yb290KHN0cnVjdCBkb21haW5faW9tbXUgKmhkKTsKIGludCBhbWRf
aW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKHN0cnVjdCBkb21haW4g
KmRvbWFpbiwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHBhZGRyX3QgcGh5c19hZGRyLCB1bnNpZ25lZCBsb25nIHNpemUsCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnQgaXcs
IGludCBpcik7CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKm1hcCwKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBmbGFnKTsKK2ludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlf
dW5tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGl2cnNfdW5pdHlf
bWFwICptYXApOwogaW50IF9fbXVzdF9jaGVjayBhbWRfaW9tbXVfZmx1c2hf
aW90bGJfcGFnZXMoc3RydWN0IGRvbWFpbiAqZCwgZGZuX3QgZGZuLAogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IHBhZ2VfY291bnQsCiAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmx1c2hfZmxh
Z3MpOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9hY3BpLmMKKysrIGIveGVu
L2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2FjcGkuYwpAQCAtMTE4
LDEyICsxMTgsOCBAQCBzdGF0aWMgc3RydWN0IGFtZF9pb21tdSAqIF9faW5p
dCBmaW5kX2lvCiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlc2VydmVfaW9t
bXVfZXhjbHVzaW9uX3JhbmdlKAotICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlv
bW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBsaW1pdCwKLSAgICBib29s
IGFsbCwgYm9vbCBpdywgYm9vbCBpcikKKyAgICBzdHJ1Y3QgYW1kX2lvbW11
ICppb21tdSwgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGFs
bCkKIHsKLSAgICBpZiAoICFpciB8fCAhaXcgKQotICAgICAgICByZXR1cm4g
LUVQRVJNOwotCiAgICAgLyogbmVlZCB0byBleHRlbmQgZXhjbHVzaW9uIHJh
bmdlPyAqLwogICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgKQog
ICAgIHsKQEAgLTE1MiwxNCArMTQ4LDE4IEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl8KIHsKICAgICBzdHJ1Y3QgaXZyc19t
YXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJzX21hcHBpbmdzKHNl
Zyk7CiAgICAgc3RydWN0IGl2cnNfdW5pdHlfbWFwICp1bml0eV9tYXAgPSBp
dnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwOworICAgIGludCBwYWdpbmdf
bW9kZSA9IGFtZF9pb21tdV9nZXRfcGFnaW5nX21vZGUoUEZOX1VQKGJhc2Ug
KyBsZW5ndGgpKTsKKworICAgIGlmICggcGFnaW5nX21vZGUgPCAwICkKKyAg
ICAgICAgcmV0dXJuIHBhZ2luZ19tb2RlOwogCiAgICAgLyogQ2hlY2sgZm9y
IG92ZXJsYXBzLiAqLwogICAgIGZvciAoIDsgdW5pdHlfbWFwOyB1bml0eV9t
YXAgPSB1bml0eV9tYXAtPm5leHQgKQogICAgIHsKICAgICAgICAgLyoKICAg
ICAgICAgICogRXhhY3QgbWF0Y2hlcyBhcmUgb2theS4gVGhpcyBjYW4gaW4g
cGFydGljdWxhciBoYXBwZW4gd2hlbgotICAgICAgICAgKiByZWdpc3Rlcl9l
eGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgpIGNhbGxzIGhlcmUgdHdpY2Ug
Zm9yIHRoZQotICAgICAgICAgKiBzYW1lIChzLGIsZCxmKS4KKyAgICAgICAg
ICogcmVnaXN0ZXJfcmFuZ2VfZm9yX2RldmljZSgpIGNhbGxzIGhlcmUgdHdp
Y2UgZm9yIHRoZSBzYW1lCisgICAgICAgICAqIChzLGIsZCxmKS4KICAgICAg
ICAgICovCiAgICAgICAgIGlmICggYmFzZSA9PSB1bml0eV9tYXAtPmFkZHIg
JiYgbGVuZ3RoID09IHVuaXR5X21hcC0+bGVuZ3RoICYmCiAgICAgICAgICAg
ICAgaXIgPT0gdW5pdHlfbWFwLT5yZWFkICYmIGl3ID09IHVuaXR5X21hcC0+
d3JpdGUgKQpAQCAtMTg3LDU1ICsxODcsNTIgQEAgc3RhdGljIGludCBfX2lu
aXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yXwogICAgIHVuaXR5X21hcC0+bmV4
dCA9IGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9tYXA7CiAgICAgaXZyc19t
YXBwaW5nc1tiZGZdLnVuaXR5X21hcCA9IHVuaXR5X21hcDsKIAorICAgIGlm
ICggcGFnaW5nX21vZGUgPiBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2RlICkK
KyAgICAgICAgYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSA9IHBhZ2luZ19t
b2RlOworCiAgICAgcmV0dXJuIDA7CiB9CiAKLXN0YXRpYyBpbnQgX19pbml0
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCi0g
ICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25nIGxpbWl0LCB1
OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9yYW5n
ZV9mb3JfYWxsX2RldmljZXMoCisgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikKIHsK
ICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCi0gICAgdW5zaWduZWQgbG9u
ZyByYW5nZV90b3AsIGlvbW11X3RvcCwgbGVuZ3RoOwogICAgIHN0cnVjdCBh
bWRfaW9tbXUgKmlvbW11OwotICAgIHVuc2lnbmVkIGludCBiZGY7CiAgICAg
aW50IHJjID0gMDsKIAogICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJh
bmdlIGluc2lkZSBvZiBJT01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICov
CiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0
byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICByYW5nZV90b3AgPSBsaW1pdCAr
IFBBR0VfU0laRTsKLSAgICBpb21tdV90b3AgPSBtYXhfcGFnZSAqIFBBR0Vf
U0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21tdV90b3AgKQotICAgIHsKLSAg
ICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21tdV90b3AgKQotICAgICAgICAg
ICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwotICAgICAgICBsZW5ndGggPSBy
YW5nZV90b3AgLSBiYXNlOwotICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0
eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCi0gICAgICAg
IC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1
c2lvbiByYW5nZSAqLwotICAgICAgICBmb3IgKCBiZGYgPSAwOyAhcmMgJiYg
YmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysgKQotICAgICAgICAgICAg
cmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywgYmRmLCBi
YXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0gICAgICAgIC8qIHB1c2ggJ2Jhc2Un
IGp1c3Qgb3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgKi8KLSAg
ICAgICAgYmFzZSA9IGlvbW11X3RvcDsKLSAgICB9Ci0gICAgLyogcmVnaXN0
ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzICovCi0gICAgaWYg
KCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkKKyAgICBpZiAoIGV4Y2x1
c2lvbiApCiAgICAgewogICAgICAgICBmb3JfZWFjaF9hbWRfaW9tbXUoIGlv
bW11ICkKICAgICAgICAgewotICAgICAgICAgICAgcmMgPSByZXNlcnZlX2lv
bW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCi0gICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRy
dWUgLyogYWxsICovLCBpdywgaXIpOwotICAgICAgICAgICAgaWYgKCByYyAp
Ci0gICAgICAgICAgICAgICAgYnJlYWs7CisgICAgICAgICAgICBpbnQgcmV0
ID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2Us
IGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICovKTsKKworICAgICAgICAg
ICAgaWYgKCByZXQgJiYgIXJjICkKKyAgICAgICAgICAgICAgICByYyA9IHJl
dDsKICAgICAgICAgfQogICAgIH0KIAorICAgIGlmICggIWV4Y2x1c2lvbiB8
fCByYyApCisgICAgeworICAgICAgICBwYWRkcl90IGxlbmd0aCA9IGxpbWl0
ICsgUEFHRV9TSVpFIC0gYmFzZTsKKyAgICAgICAgdW5zaWduZWQgaW50IGJk
ZjsKKworICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFn
ZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCisgICAgICAgIGZvciAoIGJkZiA9
IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisr
ICkKKyAgICAgICAgICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2Rl
dmljZShzZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOworICAgIH0K
KwogICAgIHJldHVybiByYzsKIH0KIAotc3RhdGljIGludCBfX2luaXQgcmVn
aXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoCi0gICAgdTE2IGJk
ZiwgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25nIGxpbWl0LCB1
OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9yYW5n
ZV9mb3JfZGV2aWNlKAorICAgIHVuc2lnbmVkIGludCBiZGYsIHBhZGRyX3Qg
YmFzZSwgcGFkZHJfdCBsaW1pdCwKKyAgICBib29sIGl3LCBib29sIGlyLCBi
b29sIGV4Y2x1c2lvbikKIHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICov
CiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhzZWcpOwotICAgIHVuc2lnbmVkIGxvbmcgcmFu
Z2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lv
bW11ICppb21tdTsKICAgICB1MTYgcmVxOwogICAgIGludCByYyA9IDA7CkBA
IC0yNDksMjcgKzI0NiwxOSBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rl
cl9leGNsdXNpb25fcmFuCiAgICAgcmVxID0gaXZyc19tYXBwaW5nc1tiZGZd
LmR0ZV9yZXF1ZXN0b3JfaWQ7CiAKICAgICAvKiBub3RlOiAnbGltaXQnIHBh
cmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwotICAg
IHJhbmdlX3RvcCA9IGxpbWl0ICsgUEFHRV9TSVpFOwotICAgIGlvbW11X3Rv
cCA9IG1heF9wYWdlICogUEFHRV9TSVpFOwotICAgIGlmICggYmFzZSA8IGlv
bW11X3RvcCApCi0gICAgewotICAgICAgICBpZiAoIHJhbmdlX3RvcCA+IGlv
bW11X3RvcCApCi0gICAgICAgICAgICByYW5nZV90b3AgPSBpb21tdV90b3A7
Ci0gICAgICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAtIGJhc2U7CisgICAgaWYg
KCBleGNsdXNpb24gKQorICAgICAgICByYyA9IHJlc2VydmVfaW9tbXVfZXhj
bHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWxzZSAvKiBhbGwg
Ki8pOworICAgIGlmICggIWV4Y2x1c2lvbiB8fCByYyApCisgICAgeworICAg
ICAgICBwYWRkcl90IGxlbmd0aCA9IGxpbWl0ICsgUEFHRV9TSVpFIC0gYmFz
ZTsKKwogICAgICAgICAvKiByZXNlcnZlIHVuaXR5LW1hcHBlZCBwYWdlIGVu
dHJpZXMgZm9yIGRldmljZSAqLwotICAgICAgICAvKiBub3RlOiB0aGVzZSBl
bnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNpb24gcmFuZ2UgKi8KICAg
ICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywg
YmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcikgPzoKICAgICAgICAgICAgICBy
ZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNlZywgcmVxLCBiYXNlLCBs
ZW5ndGgsIGl3LCBpcik7Ci0KLSAgICAgICAgLyogcHVzaCAnYmFzZScganVz
dCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSAqLwotICAgICAg
ICBiYXNlID0gaW9tbXVfdG9wOwogICAgIH0KLQotICAgIC8qIHJlZ2lzdGVy
IElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyBmb3IgZGV2aWNlICov
Ci0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICApCisgICAg
ZWxzZQogICAgIHsKLSAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCi0gICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFsc2UgLyogYWxsICov
LCBpdywgaXIpOwogICAgICAgICBpdnJzX21hcHBpbmdzW2JkZl0uZHRlX2Fs
bG93X2V4Y2x1c2lvbiA9IHRydWU7CiAgICAgICAgIGl2cnNfbWFwcGluZ3Nb
cmVxXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1ZTsKICAgICB9CkBAIC0y
NzcsNTMgKzI2Niw0MiBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9l
eGNsdXNpb25fcmFuCiAgICAgcmV0dXJuIHJjOwogfQogCi1zdGF0aWMgaW50
IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2lvbW11X2Rl
dmljZXMoCi0gICAgc3RydWN0IGFtZF9pb21tdSAqaW9tbXUsCi0gICAgdW5z
aWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25nIGxpbWl0LCB1OCBpdywg
dTggaXIpCitzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9yYW5nZV9mb3Jf
aW9tbXVfZGV2aWNlcygKKyAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwg
cGFkZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LAorICAgIGJvb2wgaXcsIGJv
b2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewotICAgIHVuc2lnbmVkIGxvbmcg
cmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKKyAgICAvKiBub3RlOiAn
bGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2UtYWxpZ25l
ZCAqLworICAgIHBhZGRyX3QgbGVuZ3RoID0gbGltaXQgKyBQQUdFX1NJWkUg
LSBiYXNlOwogICAgIHVuc2lnbmVkIGludCBiZGY7CiAgICAgdTE2IHJlcTsK
LSAgICBpbnQgcmMgPSAwOworICAgIGludCByYzsKIAotICAgIC8qIGlzIHBh
cnQgb2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJT01NVSB2aXJ0dWFs
IGFkZHJlc3Mgc3BhY2U/ICovCi0gICAgLyogbm90ZTogJ2xpbWl0JyBwYXJh
bWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICBy
YW5nZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsKLSAgICBpb21tdV90b3Ag
PSBtYXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21t
dV90b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21t
dV90b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwot
ICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwotICAgICAgICAv
KiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBk
ZXZpY2VzICovCi0gICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJl
IHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAgICBmb3Ig
KCBiZGYgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRm
KysgKQotICAgICAgICB7Ci0gICAgICAgICAgICBpZiAoIGlvbW11ID09IGZp
bmRfaW9tbXVfZm9yX2RldmljZShpb21tdS0+c2VnLCBiZGYpICkKLSAgICAg
ICAgICAgIHsKLSAgICAgICAgICAgICAgICByZXEgPSBnZXRfaXZyc19tYXBw
aW5ncyhpb21tdS0+c2VnKVtiZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7Ci0gICAg
ICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNl
KGlvbW11LT5zZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLAotICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIp
ID86Ci0gICAgICAgICAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9m
b3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAotICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBpdywgaXIpOwotICAgICAgICAgICAgfQotICAgICAgICB9Ci0KLSAgICAg
ICAgLyogcHVzaCAnYmFzZScganVzdCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRk
cmVzcyBzcGFjZSAqLwotICAgICAgICBiYXNlID0gaW9tbXVfdG9wOworICAg
IGlmICggZXhjbHVzaW9uICkKKyAgICB7CisgICAgICAgIHJjID0gcmVzZXJ2
ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LCB0
cnVlIC8qIGFsbCAqLyk7CisgICAgICAgIGlmICggIXJjICkKKyAgICAgICAg
ICAgIHJldHVybiAwOwogICAgIH0KIAotICAgIC8qIHJlZ2lzdGVyIElPTU1V
IGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwotICAgIGlmICggIXJjICYm
IGxpbWl0ID49IGlvbW11X3RvcCApCi0gICAgICAgIHJjID0gcmVzZXJ2ZV9p
b21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAotICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUg
LyogYWxsICovLCBpdywgaXIpOworICAgIC8qIHJlc2VydmUgdW5pdHktbWFw
cGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAqLworICAgIGZvciAoIGJk
ZiA9IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJk
ZisrICkKKyAgICB7CisgICAgICAgIGlmICggaW9tbXUgIT0gZmluZF9pb21t
dV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQorICAgICAgICAgICAg
Y29udGludWU7CisKKyAgICAgICAgcmVxID0gZ2V0X2l2cnNfbWFwcGluZ3Mo
aW9tbXUtPnNlZylbYmRmXS5kdGVfcmVxdWVzdG9yX2lkOworICAgICAgICBy
YyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywg
YmRmLCBiYXNlLCBsZW5ndGgsCisgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpID86CisgICAgICAgICAgICAgcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShpb21tdS0+c2VnLCByZXEsIGJh
c2UsIGxlbmd0aCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGl3LCBpcik7CisgICAgfQogCiAgICAgcmV0dXJuIHJjOwog
fQogCiBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9zZWxl
Y3QoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19tZW1vcnkgKml2bWRf
YmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25n
IGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJfdCBiYXNlLCBwYWRk
cl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikK
IHsKICAgICB1MTYgYmRmOwogCkBAIC0zMzQsMTIgKzMxMiwxMiBAQCBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9zZWxlCiAgICAgICAg
IHJldHVybiAtRU5PREVWOwogICAgIH0KIAotICAgIHJldHVybiByZWdpc3Rl
cl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZShiZGYsIGJhc2UsIGxpbWl0
LCBpdywgaXIpOworICAgIHJldHVybiByZWdpc3Rlcl9yYW5nZV9mb3JfZGV2
aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpciwgZXhjbHVzaW9uKTsKIH0K
IAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9kZXZpY2VfcmFuZ2Uo
CiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19tZW1vcnkgKml2bWRfYmxv
Y2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25nIGxp
bWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikKIHsK
ICAgICB1bnNpZ25lZCBpbnQgZmlyc3RfYmRmLCBsYXN0X2JkZiwgYmRmOwog
ICAgIGludCBlcnJvcjsKQEAgLTM2MSwxNSArMzM5LDE1IEBAIHN0YXRpYyBp
bnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3JhbmcKICAgICB9CiAKICAg
ICBmb3IgKCBiZGYgPSBmaXJzdF9iZGYsIGVycm9yID0gMDsgKGJkZiA8PSBs
YXN0X2JkZikgJiYgIWVycm9yOyBiZGYrKyApCi0gICAgICAgIGVycm9yID0g
cmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZpY2UoCi0gICAgICAg
ICAgICBiZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgICAgICBlcnJv
ciA9IHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2UoCisgICAgICAgICAgICBi
ZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBy
ZXR1cm4gZXJyb3I7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2
bWRfZGV2aWNlX2lvbW11KAogICAgIGNvbnN0IHN0cnVjdCBhY3BpX2l2cnNf
bWVtb3J5ICppdm1kX2Jsb2NrLAotICAgIHVuc2lnbmVkIGxvbmcgYmFzZSwg
dW5zaWduZWQgbG9uZyBsaW1pdCwgdTggaXcsIHU4IGlyKQorICAgIHBhZGRy
X3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwgYm9vbCBpdywgYm9vbCBpciwgYm9v
bCBleGNsdXNpb24pCiB7CiAgICAgaW50IHNlZyA9IDA7IC8qIFhYWCAqLwog
ICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwpAQCAtMzg0LDE0ICszNjIs
MTQgQEAgc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9kZXZpY2VfaW9t
bQogICAgICAgICByZXR1cm4gLUVOT0RFVjsKICAgICB9CiAKLSAgICByZXR1
cm4gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9pb21tdV9kZXZpY2Vz
KAotICAgICAgICBpb21tdSwgYmFzZSwgbGltaXQsIGl3LCBpcik7CisgICAg
cmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21tdV9kZXZpY2VzKAorICAg
ICAgICBpb21tdSwgYmFzZSwgbGltaXQsIGl3LCBpciwgZXhjbHVzaW9uKTsK
IH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9ibG9jayhjb25z
dCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaykKIHsKICAg
ICB1bnNpZ25lZCBsb25nIHN0YXJ0X2FkZHIsIG1lbV9sZW5ndGgsIGJhc2Us
IGxpbWl0OwotICAgIHU4IGl3LCBpcjsKKyAgICBib29sIGl3ID0gdHJ1ZSwg
aXIgPSB0cnVlLCBleGNsdXNpb24gPSBmYWxzZTsKIAogICAgIGlmICggaXZt
ZF9ibG9jay0+aGVhZGVyLmxlbmd0aCA8IHNpemVvZigqaXZtZF9ibG9jaykg
KQogICAgIHsKQEAgLTQwOCwxMyArMzg2LDExIEBAIHN0YXRpYyBpbnQgX19p
bml0IHBhcnNlX2l2bWRfYmxvY2soY29uc3QKICAgICAgICAgICAgICAgICAg
ICAgaXZtZF9ibG9jay0+aGVhZGVyLnR5cGUsIHN0YXJ0X2FkZHIsIG1lbV9s
ZW5ndGgpOwogCiAgICAgaWYgKCBpdm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3Mg
JiBBQ1BJX0lWTURfRVhDTFVTSU9OX1JBTkdFICkKLSAgICAgICAgaXcgPSBp
ciA9IElPTU1VX0NPTlRST0xfRU5BQkxFRDsKKyAgICAgICAgZXhjbHVzaW9u
ID0gdHJ1ZTsKICAgICBlbHNlIGlmICggaXZtZF9ibG9jay0+aGVhZGVyLmZs
YWdzICYgQUNQSV9JVk1EX1VOSVRZICkKICAgICB7Ci0gICAgICAgIGl3ID0g
aXZtZF9ibG9jay0+aGVhZGVyLmZsYWdzICYgQUNQSV9JVk1EX1JFQUQgPwot
ICAgICAgICAgICAgSU9NTVVfQ09OVFJPTF9FTkFCTEVEIDogSU9NTVVfQ09O
VFJPTF9ESVNBQkxFRDsKLSAgICAgICAgaXIgPSBpdm1kX2Jsb2NrLT5oZWFk
ZXIuZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEUgPwotICAgICAgICAgICAgSU9N
TVVfQ09OVFJPTF9FTkFCTEVEIDogSU9NTVVfQ09OVFJPTF9ESVNBQkxFRDsK
KyAgICAgICAgaXcgPSBpdm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJ
X0lWTURfUkVBRDsKKyAgICAgICAgaXIgPSBpdm1kX2Jsb2NrLT5oZWFkZXIu
ZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEU7CiAgICAgfQogICAgIGVsc2UKICAg
ICB7CkBAIC00MjUsMjAgKzQwMSwyMCBAQCBzdGF0aWMgaW50IF9faW5pdCBw
YXJzZV9pdm1kX2Jsb2NrKGNvbnN0CiAgICAgc3dpdGNoKCBpdm1kX2Jsb2Nr
LT5oZWFkZXIudHlwZSApCiAgICAgewogICAgIGNhc2UgQUNQSV9JVlJTX1RZ
UEVfTUVNT1JZX0FMTDoKLSAgICAgICAgcmV0dXJuIHJlZ2lzdGVyX2V4Y2x1
c2lvbl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCi0gICAgICAgICAgICBiYXNl
LCBsaW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHJlZ2lzdGVyX3Jh
bmdlX2Zvcl9hbGxfZGV2aWNlcygKKyAgICAgICAgICAgIGJhc2UsIGxpbWl0
LCBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBjYXNlIEFDUElfSVZSU19U
WVBFX01FTU9SWV9PTkU6Ci0gICAgICAgIHJldHVybiBwYXJzZV9pdm1kX2Rl
dmljZV9zZWxlY3QoaXZtZF9ibG9jaywKLSAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAg
ICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdChpdm1kX2Js
b2NrLCBiYXNlLCBsaW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBjYXNl
IEFDUElfSVZSU19UWVBFX01FTU9SWV9SQU5HRToKLSAgICAgICAgcmV0dXJu
IHBhcnNlX2l2bWRfZGV2aWNlX3JhbmdlKGl2bWRfYmxvY2ssCi0gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1pdCwg
aXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3Jh
bmdlKGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyLCBleGNsdXNpb24pOwog
CiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9NRU1PUllfSU9NTVU6Ci0gICAg
ICAgIHJldHVybiBwYXJzZV9pdm1kX2RldmljZV9pb21tdShpdm1kX2Jsb2Nr
LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmFz
ZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIHJldHVybiBwYXJzZV9pdm1k
X2RldmljZV9pb21tdShpdm1kX2Jsb2NrLCBiYXNlLCBsaW1pdCwKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpciwgZXhj
bHVzaW9uKTsKIAogICAgIGRlZmF1bHQ6CiAgICAgICAgIEFNRF9JT01NVV9E
RUJVRygiSVZNRCBFcnJvcjogSW52YWxpZCBCbG9jayBUeXBlIVxuIik7Ci0t
LSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9wY2lfYW1kX2lvbW11
LmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRf
aW9tbXUuYwpAQCAtMjM0LDYgKzIzNCw4IEBAIHN0YXRpYyBpbnQgX19tdXN0
X2NoZWNrIGFsbG9jYXRlX2RvbWFpbl8KICAgICByZXR1cm4gcmM7CiB9CiAK
K2ludCBfX3JlYWRfbW9zdGx5IGFtZF9pb21tdV9taW5fcGFnaW5nX21vZGUg
PSAxOworCiBzdGF0aWMgaW50IGFtZF9pb21tdV9kb21haW5faW5pdChzdHJ1
Y3QgZG9tYWluICpkKQogewogICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhk
ID0gZG9tX2lvbW11KGQpOwpAQCAtMjQ1LDExICsyNDcsMTMgQEAgc3RhdGlj
IGludCBhbWRfaW9tbXVfZG9tYWluX2luaXQoc3RydWN0CiAgICAgICogLSBI
Vk0gY291bGQgaW4gcHJpbmNpcGxlIHVzZSAzIG9yIDQgZGVwZW5kaW5nIG9u
IGhvdyBtdWNoIGd1ZXN0CiAgICAgICogICBwaHlzaWNhbCBhZGRyZXNzIHNw
YWNlIHdlIGdpdmUgaXQsIGJ1dCB0aGlzIGlzbid0IGtub3duIHlldCBzbyB1
c2UgNAogICAgICAqICAgdW5pbGF0ZXJhbGx5LgorICAgICAqIC0gVW5pdHkg
bWFwcyBtYXkgcmVxdWlyZSBhbiBldmVuIGhpZ2hlciBudW1iZXIuCiAgICAg
ICovCi0gICAgaGQtPmFyY2gucGFnaW5nX21vZGUgPSBhbWRfaW9tbXVfZ2V0
X3BhZ2luZ19tb2RlKAotICAgICAgICBpc19odm1fZG9tYWluKGQpCi0gICAg
ICAgID8gMXVsIDw8IChERUZBVUxUX0RPTUFJTl9BRERSRVNTX1dJRFRIIC0g
UEFHRV9TSElGVCkKLSAgICAgICAgOiBnZXRfdXBwZXJfbWZuX2JvdW5kKCkg
KyAxKTsKKyAgICBoZC0+YXJjaC5wYWdpbmdfbW9kZSA9IG1heChhbWRfaW9t
bXVfZ2V0X3BhZ2luZ19tb2RlKAorICAgICAgICAgICAgaXNfaHZtX2RvbWFp
bihkKQorICAgICAgICAgICAgPyAxdWwgPDwgKERFRkFVTFRfRE9NQUlOX0FE
RFJFU1NfV0lEVEggLSBQQUdFX1NISUZUKQorICAgICAgICAgICAgOiBnZXRf
dXBwZXJfbWZuX2JvdW5kKCkgKyAxKSwKKyAgICAgICAgYW1kX2lvbW11X21p
bl9wYWdpbmdfbW9kZSk7CiAKICAgICByZXR1cm4gMDsKIH0KLS0tIGEveGVu
L2luY2x1ZGUvYXNtLXg4Ni9odm0vc3ZtL2FtZC1pb21tdS1wcm90by5oCisr
KyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvaHZtL3N2bS9hbWQtaW9tbXUtcHJv
dG8uaApAQCAtMTQwLDYgKzE0MCw4IEBAIGV4dGVybiBzdHJ1Y3QgaHBldF9z
YmRmIHsKICAgICB9IGluaXQ7CiB9IGhwZXRfc2JkZjsKIAorZXh0ZXJuIGlu
dCBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2RlOworCiBleHRlcm4gdm9pZCAq
c2hhcmVkX2ludHJlbWFwX3RhYmxlOwogZXh0ZXJuIHVuc2lnbmVkIGxvbmcg
KnNoYXJlZF9pbnRyZW1hcF9pbnVzZTsKIAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtODA3LDcgKzgw
Nyw3IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRl
cik7IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2Vu
dHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5V
TEwpOwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFw
Mm1faXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAg
ICAgICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3No
YXJlZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4o
bWZuK2ksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAgfQogICAgIH0K
QEAgLTkzNCwxMyArOTM0LDEzIEBAIGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5
KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAgICAgICAgICAg
IEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAgICB9Ci0gICAg
ICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNfZm9yZWlnbihv
dCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90KSApCiAgICAg
ICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRuJ3QgYmUgdW5t
YXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAqLworICAgICAg
ICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWdu
IHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwog
ICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAgICAgICAgCi0g
ICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAgICAgIHJldHVy
biAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBpZiAoIHAybV9p
c19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAgICAgICAgewpA
QCAtMTAzNCw4ICsxMDM0LDcgQEAgaW50IHAybV9jaGFuZ2VfdHlwZV9vbmUo
c3RydWN0IGRvbWFpbiAqZAogICAgIHN0cnVjdCBwMm1fZG9tYWluICpwMm0g
PSBwMm1fZ2V0X2hvc3RwMm0oZCk7CiAgICAgaW50IHJjOwogCi0gICAgQlVH
X09OKHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lzX2dyYW50KG50KSk7Ci0g
ICAgQlVHX09OKHAybV9pc19mb3JlaWduKG90KSB8fCBwMm1faXNfZm9yZWln
bihudCkpOworICAgIEJVR19PTihwMm1faXNfc3BlY2lhbChvdCkgfHwgcDJt
X2lzX3NwZWNpYWwobnQpKTsKIAogICAgIGdmbl9sb2NrKHAybSwgZ2ZuLCAw
KTsKIApAQCAtMTI4MiwxMSArMTI4MSwxMSBAQCBzdGF0aWMgaW50IHNldF90
eXBlZF9wMm1fZW50cnkoc3RydWN0IGRvCiAgICAgICAgIGdmbl91bmxvY2so
cDJtLCBnZm4sIG9yZGVyKTsKICAgICAgICAgcmV0dXJuIGN1cl9vcmRlciAr
IDE7CiAgICAgfQotICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1f
aXNfZm9yZWlnbihvdCkgKQorICAgIGlmICggcDJtX2lzX3NwZWNpYWwob3Qp
ICkKICAgICB7CiAgICAgICAgIGdmbl91bmxvY2socDJtLCBnZm4sIG9yZGVy
KTsKICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwotICAgICAgICByZXR1cm4g
LUVOT0VOVDsKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKICAgICB9CiAgICAg
ZWxzZSBpZiAoIHAybV9pc19yYW0ob3QpICkKICAgICB7Ci0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4
Ni9wMm0uaApAQCAtMTQxLDYgKzE0MSwxMCBAQCB0eXBlZGVmIHVuc2lnbmVk
IGludCBwMm1fcXVlcnlfdDsKICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB8IHAybV90b19tYXNrKHAybV9yYW1fbG9nZGlydHkpICkKICNkZWZpbmUg
UDJNX1NIQVJFRF9UWVBFUyAgIChwMm1fdG9fbWFzayhwMm1fcmFtX3NoYXJl
ZCkpCiAKKy8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQgdXAgdmlhIHNw
ZWNpYWwgYWNjZXNzb3JzLiAqLworI2RlZmluZSBQMk1fU1BFQ0lBTF9UWVBF
UyAoUDJNX0dSQU5UX1RZUEVTIHwgXAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkKKwogLyogVmFs
aWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQgd2l0aCBhICh2
YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9NRk5fVFlQRVMg
KFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgfCBwMm1fdG9fbWFzayhwMm1fbW1pb19k
aXJlY3QpIFwKQEAgLTE2OSw2ICsxNzMsNyBAQCB0eXBlZGVmIHVuc2lnbmVk
IGludCBwMm1fcXVlcnlfdDsKICNkZWZpbmUgcDJtX2lzX3BhZ2VkKF90KSAg
ICAocDJtX3RvX21hc2soX3QpICYgUDJNX1BBR0VEX1RZUEVTKQogI2RlZmlu
ZSBwMm1faXNfc2hhcmFibGUoX3QpIChwMm1fdG9fbWFzayhfdCkgJiBQMk1f
U0hBUkFCTEVfVFlQRVMpCiAjZGVmaW5lIHAybV9pc19zaGFyZWQoX3QpICAg
KHAybV90b19tYXNrKF90KSAmIFAyTV9TSEFSRURfVFlQRVMpCisjZGVmaW5l
IHAybV9pc19zcGVjaWFsKF90KSAgKHAybV90b19tYXNrKF90KSAmIFAyTV9T
UEVDSUFMX1RZUEVTKQogI2RlZmluZSBwMm1faXNfYnJva2VuKF90KSAgIChw
Mm1fdG9fbWFzayhfdCkgJiBQMk1fQlJPS0VOX1RZUEVTKQogI2RlZmluZSBw
Mm1faXNfZm9yZWlnbihfdCkgIChwMm1fdG9fbWFzayhfdCkgJiBwMm1fdG9f
bWFzayhwMm1fbWFwX2ZvcmVpZ24pKQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.13-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.13-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tLmMK
KysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ3NzAsNyArNDc3MCw5IEBA
IGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogCiAgICAgLyogUmVt
b3ZlIHByZXZpb3VzbHkgbWFwcGVkIHBhZ2UgaWYgaXQgd2FzIHByZXNlbnQu
ICovCiAgICAgcHJldl9tZm4gPSBnZXRfZ2ZuKGQsIGdmbl94KGdwZm4pLCAm
cDJtdCk7Ci0gICAgaWYgKCBtZm5fdmFsaWQocHJldl9tZm4pICkKKyAgICBp
ZiAoIHAybXQgPT0gcDJtX21taW9fZGlyZWN0ICkKKyAgICAgICAgcmMgPSAt
RVBFUk07CisgICAgZWxzZSBpZiAoIG1mbl92YWxpZChwcmV2X21mbikgKQog
ICAgIHsKICAgICAgICAgaWYgKCBpc194ZW5faGVhcF9tZm4ocHJldl9tZm4p
ICkKICAgICAgICAgICAgIC8qIFhlbiBoZWFwIGZyYW1lcyBhcmUgc2ltcGx5
IHVuaG9va2VkIGZyb20gdGhpcyBwaHlzIHNsb3QuICovCi0tLSBhL3hlbi9h
cmNoL3g4Ni9tbS9wMm0uYworKysgYi94ZW4vYXJjaC94ODYvbW0vcDJtLmMK
QEAgLTc5Niw3ICs3OTYsOCBAQCBwMm1fcmVtb3ZlX3BhZ2Uoc3RydWN0IHAy
bV9kb21haW4gKnAybSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICZjdXJfb3JkZXIsIE5VTEwpOwogCiAgICAgICAgIGlm
ICggcDJtX2lzX3ZhbGlkKHQpICYmCi0gICAgICAgICAgICAgKCFtZm5fdmFs
aWQoX21mbihtZm4pKSB8fCBtZm4gKyBpICE9IG1mbl94KG1mbl9yZXR1cm4p
KSApCisgICAgICAgICAgICAgKCFtZm5fdmFsaWQoX21mbihtZm4pKSB8fCB0
ID09IHAybV9tbWlvX2RpcmVjdCB8fAorICAgICAgICAgICAgICBtZm4gKyBp
ICE9IG1mbl94KG1mbl9yZXR1cm4pKSApCiAgICAgICAgICAgICByZXR1cm4g
LUVJTFNFUTsKIAogICAgICAgICBpICs9ICgxVUwgPDwgY3VyX29yZGVyKSAt
ICgoZ2ZuX2wgKyBpKSAmICgoMVVMIDw8IGN1cl9vcmRlcikgLSAxKSk7CkBA
IC04OTAsNyArODkxLDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3Ry
dWN0IGRvbWFpbiAqZAogICAgIGlmICggcDJtX2lzX2ZvcmVpZ24odCkgKQog
ICAgICAgICByZXR1cm4gLUVJTlZBTDsKIAotICAgIGlmICggIW1mbl92YWxp
ZChtZm4pICkKKyAgICBpZiAoICFtZm5fdmFsaWQobWZuKSB8fCB0ID09IHAy
bV9tbWlvX2RpcmVjdCApCiAgICAgewogICAgICAgICBBU1NFUlRfVU5SRUFD
SEFCTEUoKTsKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7CkBAIC05MzYsNyAr
OTM3LDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFp
biAqZAogICAgICAgICB9CiAgICAgICAgIGlmICggcDJtX2lzX3NwZWNpYWwo
b3QpICkKICAgICAgICAgewotICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0
IHVubWFwcGluZyBncmFudC9mb3JlaWduIHRoaXMgd2F5LiAqLworICAgICAg
ICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWdu
L2RpcmVjdC1NTUlPIHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWlu
X2NyYXNoKGQpOwogICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAg
ICAgICAgICAgCkBAIC0xMzg1LDggKzEzODYsOCBAQCBpbnQgc2V0X2lkZW50
aXR5X3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluCiAgKiAgICBvcmRlcisxICBm
b3IgY2FsbGVyIHRvIHJldHJ5IHdpdGggb3JkZXIgKGd1YXJhbnRlZWQgc21h
bGxlciB0aGFuCiAgKiAgICAgICAgICAgICB0aGUgb3JkZXIgdmFsdWUgcGFz
c2VkIGluKQogICovCi1pbnQgY2xlYXJfbW1pb19wMm1fZW50cnkoc3RydWN0
IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm5fbCwgbWZuX3QgbWZuLAot
ICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBvcmRlcikK
K3N0YXRpYyBpbnQgY2xlYXJfbW1pb19wMm1fZW50cnkoc3RydWN0IGRvbWFp
biAqZCwgdW5zaWduZWQgbG9uZyBnZm5fbCwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgbWZuX3QgbWZuLCB1bnNpZ25lZCBpbnQgb3JkZXIp
CiB7CiAgICAgaW50IHJjID0gLUVJTlZBTDsKICAgICBnZm5fdCBnZm4gPSBf
Z2ZuKGdmbl9sKTsKLS0tIGEveGVuL2FyY2gveDg2L21tL3AybS1wb2QuYwor
KysgYi94ZW4vYXJjaC94ODYvbW0vcDJtLXBvZC5jCkBAIC0xMjk1LDE3ICsx
Mjk1LDE3IEBAIGd1ZXN0X3BoeXNtYXBfbWFya19wb3B1bGF0ZV9vbl9kZW1h
bmQoc3QKIAogICAgICAgICBwMm0tPmdldF9lbnRyeShwMm0sIGdmbl9hZGQo
Z2ZuLCBpKSwgJm90LCAmYSwgMCwgJmN1cl9vcmRlciwgTlVMTCk7CiAgICAg
ICAgIG4gPSAxVUwgPDwgbWluKG9yZGVyLCBjdXJfb3JkZXIpOwotICAgICAg
ICBpZiAoIHAybV9pc19yYW0ob3QpICkKKyAgICAgICAgaWYgKCBvdCA9PSBw
Mm1fcG9wdWxhdGVfb25fZGVtYW5kICkKKyAgICAgICAgeworICAgICAgICAg
ICAgLyogQ291bnQgaG93IG1hbnkgUG9EIGVudHJpZXMgd2UnbGwgYmUgcmVw
bGFjaW5nIGlmIHN1Y2Nlc3NmdWwgKi8KKyAgICAgICAgICAgIHBvZF9jb3Vu
dCArPSBuOworICAgICAgICB9CisgICAgICAgIGVsc2UgaWYgKCBvdCAhPSBw
Mm1faW52YWxpZCAmJiBvdCAhPSBwMm1fbW1pb19kbSApCiAgICAgICAgIHsK
ICAgICAgICAgICAgIFAyTV9ERUJVRygiZ2ZuX3RvX21mbiByZXR1cm5lZCB0
eXBlICVkIVxuIiwgb3QpOwogICAgICAgICAgICAgcmMgPSAtRUJVU1k7CiAg
ICAgICAgICAgICBnb3RvIG91dDsKICAgICAgICAgfQotICAgICAgICBlbHNl
IGlmICggb3QgPT0gcDJtX3BvcHVsYXRlX29uX2RlbWFuZCApCi0gICAgICAg
IHsKLSAgICAgICAgICAgIC8qIENvdW50IGhvdyBtYW4gUG9EIGVudHJpZXMg
d2UnbGwgYmUgcmVwbGFjaW5nIGlmIHN1Y2Nlc3NmdWwgKi8KLSAgICAgICAg
ICAgIHBvZF9jb3VudCArPSBuOwotICAgICAgICB9CiAgICAgfQogCiAgICAg
LyogTm93LCBhY3R1YWxseSBkbyB0aGUgdHdvLXdheSBtYXBwaW5nICovCi0t
LSBhL3hlbi9jb21tb24vbWVtb3J5LmMKKysrIGIveGVuL2NvbW1vbi9tZW1v
cnkuYwpAQCAtMzI4LDcgKzMyOCw3IEBAIGludCBndWVzdF9yZW1vdmVfcGFn
ZShzdHJ1Y3QgZG9tYWluICpkLAogICAgIH0KICAgICBpZiAoIHAybXQgPT0g
cDJtX21taW9fZGlyZWN0ICkKICAgICB7Ci0gICAgICAgIHJjID0gY2xlYXJf
bW1pb19wMm1fZW50cnkoZCwgZ21mbiwgbWZuLCBQQUdFX09SREVSXzRLKTsK
KyAgICAgICAgcmMgPSAtRVBFUk07CiAgICAgICAgIGdvdG8gb3V0X3B1dF9n
Zm47CiAgICAgfQogI2Vsc2UKQEAgLTE3MjAsNiArMTcyMCwxNSBAQCBpbnQg
Y2hlY2tfZ2V0X3BhZ2VfZnJvbV9nZm4oc3RydWN0IGRvbWFpCiAgICAgICAg
IHJldHVybiAtRUFHQUlOOwogICAgIH0KICNlbmRpZgorI2lmZGVmIENPTkZJ
R19YODYKKyAgICBpZiAoIHAybXQgPT0gcDJtX21taW9fZGlyZWN0ICkKKyAg
ICB7CisgICAgICAgIGlmICggcGFnZSApCisgICAgICAgICAgICBwdXRfcGFn
ZShwYWdlKTsKKworICAgICAgICByZXR1cm4gLUVQRVJNOworICAgIH0KKyNl
bmRpZgogCiAgICAgaWYgKCAhcGFnZSApCiAgICAgICAgIHJldHVybiAtRUlO
VkFMOwotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L3AybS5oCisrKyBiL3hl
bi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKQEAgLTE0Myw3ICsxNDMsOCBAQCB0
eXBlZGVmIHVuc2lnbmVkIGludCBwMm1fcXVlcnlfdDsKIAogLyogVHlwZXMg
ZXN0YWJsaXNoZWQvY2xlYW5lZCB1cCB2aWEgc3BlY2lhbCBhY2Nlc3NvcnMu
ICovCiAjZGVmaW5lIFAyTV9TUEVDSUFMX1RZUEVTIChQMk1fR1JBTlRfVFlQ
RVMgfCBcCi0gICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1fdG9fbWFz
ayhwMm1fbWFwX2ZvcmVpZ24pKQorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSB8IFwKKyAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHAybV90b19tYXNrKHAybV9tbWlvX2RpcmVj
dCkpCiAKIC8qIFZhbGlkIHR5cGVzIG5vdCBuZWNlc3NhcmlseSBhc3NvY2lh
dGVkIHdpdGggYSAodmFsaWQpIE1GTi4gKi8KICNkZWZpbmUgUDJNX0lOVkFM
SURfTUZOX1RZUEVTIChQMk1fUE9EX1RZUEVTICAgICAgICAgICAgICAgICAg
XApAQCAtNjQ5LDggKzY1MCw2IEBAIGludCBzZXRfZm9yZWlnbl9wMm1fZW50
cnkoc3RydWN0IGRvbWFpbgogLyogU2V0IG1taW8gYWRkcmVzc2VzIGluIHRo
ZSBwMm0gdGFibGUgKGZvciBwYXNzLXRocm91Z2gpICovCiBpbnQgc2V0X21t
aW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuLCBtZm5fdCBtZm4sCiAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCBvcmRlciwgcDJtX2FjY2Vzc190IGFjY2Vzcyk7Ci1pbnQgY2xl
YXJfbW1pb19wMm1fZW50cnkoc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sIG1mbl90IG1mbiwKLSAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBpbnQgb3JkZXIpOwogCiAvKiBTZXQgaWRlbnRpdHkgYWRk
cmVzc2VzIGluIHRoZSBwMm0gdGFibGUgKGZvciBwYXNzLXRocm91Z2gpICov
CiBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluICpk
LCB1bnNpZ25lZCBsb25nIGdmbiwK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9p
b21tdV9hY3BpLmMKQEAgLTExNywxMiArMTE3LDIxIEBAIHN0YXRpYyBzdHJ1
Y3QgYW1kX2lvbW11ICogX19pbml0IGZpbmRfaW8KICAgICByZXR1cm4gTlVM
TDsKIH0KIAotc3RhdGljIHZvaWQgX19pbml0IHJlc2VydmVfaW9tbXVfZXhj
bHVzaW9uX3JhbmdlKAotICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCB1
aW50NjRfdCBiYXNlLCB1aW50NjRfdCBsaW1pdCkKK3N0YXRpYyBpbnQgX19p
bml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKAorICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBs
aW1pdCwKKyAgICBib29sIGFsbCwgYm9vbCBpdywgYm9vbCBpcikKIHsKKyAg
ICBpZiAoICFpciB8fCAhaXcgKQorICAgICAgICByZXR1cm4gLUVQRVJNOwor
CiAgICAgLyogbmVlZCB0byBleHRlbmQgZXhjbHVzaW9uIHJhbmdlPyAqLwog
ICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgKQogICAgIHsKKyAg
ICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9uX2xpbWl0ICsgUEFHRV9TSVpF
IDwgYmFzZSB8fAorICAgICAgICAgICAgIGxpbWl0ICsgUEFHRV9TSVpFIDwg
aW9tbXUtPmV4Y2x1c2lvbl9iYXNlIHx8CisgICAgICAgICAgICAgaW9tbXUt
PmV4Y2x1c2lvbl9hbGxvd19hbGwgIT0gYWxsICkKKyAgICAgICAgICAgIHJl
dHVybiAtRUJVU1k7CisKICAgICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9u
X2Jhc2UgPCBiYXNlICkKICAgICAgICAgICAgIGJhc2UgPSBpb21tdS0+ZXhj
bHVzaW9uX2Jhc2U7CiAgICAgICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9s
aW1pdCA+IGxpbWl0ICkKQEAgLTEzMCwxNiArMTM5LDExIEBAIHN0YXRpYyB2
b2lkIF9faW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2kKICAgICB9CiAKICAg
ICBpb21tdS0+ZXhjbHVzaW9uX2VuYWJsZSA9IElPTU1VX0NPTlRST0xfRU5B
QkxFRDsKKyAgICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IGFsbDsK
ICAgICBpb21tdS0+ZXhjbHVzaW9uX2Jhc2UgPSBiYXNlOwogICAgIGlvbW11
LT5leGNsdXNpb25fbGltaXQgPSBsaW1pdDsKLX0KIAotc3RhdGljIHZvaWQg
X19pbml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlX2FsbCgKLSAg
ICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25lZCBsb25n
IGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQpCi17Ci0gICAgcmVzZXJ2ZV9p
b21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAg
ICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IElPTU1VX0NPTlRST0xf
RU5BQkxFRDsKKyAgICByZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgX19p
bml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoCkBAIC0xNzcsNiAr
MTgxLDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbgogICAgIHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3As
IGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAgICB1
bnNpZ25lZCBpbnQgYmRmOworICAgIGludCByYyA9IDA7CiAKICAgICAvKiBp
cyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5nZSBpbnNpZGUgb2YgSU9NTVUgdmly
dHVhbCBhZGRyZXNzIHNwYWNlPyAqLwogICAgIC8qIG5vdGU6ICdsaW1pdCcg
cGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCkBA
IC0xOTgsMTAgKzIwMywxNSBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rl
cl9leGNsdXNpb25fcmFuCiAgICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3Ag
KQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBpb21tdSAp
Ci0gICAgICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9h
bGwoaW9tbXUsIGJhc2UsIGxpbWl0KTsKKyAgICAgICAgeworICAgICAgICAg
ICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwg
YmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOworICAg
ICAgICAgICAgaWYgKCByYyApCisgICAgICAgICAgICAgICAgYnJlYWs7Cisg
ICAgICAgIH0KICAgICB9CiAKLSAgICByZXR1cm4gMDsKKyAgICByZXR1cm4g
cmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW5nZV9mb3JfZGV2aWNlKApAQCAtMjEyLDYgKzIyMiw3IEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICB1bnNp
Z25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgdTE2IHJlcTsKKyAgICBp
bnQgcmMgPSAwOwogCiAgICAgaW9tbXUgPSBmaW5kX2lvbW11X2Zvcl9kZXZp
Y2Uoc2VnLCBiZGYpOwogICAgIGlmICggIWlvbW11ICkKQEAgLTI0MSwxMiAr
MjUyLDEzIEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW4KICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ug
c2V0dGluZ3MgZm9yIGRldmljZSAqLwogICAgIGlmICggbGltaXQgPj0gaW9t
bXVfdG9wICApCiAgICAgewotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQpOworICAgICAgICByYyA9
IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBs
aW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBmYWxzZSAvKiBhbGwgKi8sIGl3LCBpcik7CiAgICAgICAgIGl2cnNf
bWFwcGluZ3NbYmRmXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1ZTsKICAg
ICAgICAgaXZyc19tYXBwaW5nc1tyZXFdLmR0ZV9hbGxvd19leGNsdXNpb24g
PSB0cnVlOwogICAgIH0KIAotICAgIHJldHVybiAwOworICAgIHJldHVybiBy
YzsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbmdlX2Zvcl9pb21tdV9kZXZpY2VzKApAQCAtMjU2LDYgKzI2OCw3IEBA
IHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAg
ICB1bnNpZ25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7
CiAgICAgdW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOworICAgIGlu
dCByYyA9IDA7CiAKICAgICAvKiBpcyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5n
ZSBpbnNpZGUgb2YgSU9NTVUgdmlydHVhbCBhZGRyZXNzIHNwYWNlPyAqLwog
ICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8g
YmUgcGFnZS1hbGlnbmVkICovCkBAIC0yODYsOCArMjk5LDEwIEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KIAogICAgIC8q
IHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwog
ICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkKLSAgICAgICAgcmVzZXJ2
ZV9pb21tdV9leGNsdXNpb25fcmFuZ2VfYWxsKGlvbW11LCBiYXNlLCBsaW1p
dCk7Ci0gICAgcmV0dXJuIDA7CisgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21t
dV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUgLyog
YWxsICovLCBpdywgaXIpOworCisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9zZWxlY3QoCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdo
L2FtZC9pb21tdS5oCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Ft
ZC9pb21tdS5oCkBAIC0xMDcsMjAgKzEwNywyNCBAQCBzdHJ1Y3QgYW1kX2lv
bW11IHsKICAgICBzdHJ1Y3QgbGlzdF9oZWFkIGF0c19kZXZpY2VzOwogfTsK
IAorc3RydWN0IGl2cnNfdW5pdHlfbWFwIHsKKyAgICBib29sIHJlYWQ6MTsK
KyAgICBib29sIHdyaXRlOjE7CisgICAgcGFkZHJfdCBhZGRyOworICAgIHVu
c2lnbmVkIGxvbmcgbGVuZ3RoOworICAgIHN0cnVjdCBpdnJzX3VuaXR5X21h
cCAqbmV4dDsKK307CisKIHN0cnVjdCBpdnJzX21hcHBpbmdzIHsKICAgICB1
aW50MTZfdCBkdGVfcmVxdWVzdG9yX2lkOwogICAgIGJvb2wgdmFsaWQ6MTsK
ICAgICBib29sIGR0ZV9hbGxvd19leGNsdXNpb246MTsKLSAgICBib29sIHVu
aXR5X21hcF9lbmFibGU6MTsKLSAgICBib29sIHdyaXRlX3Blcm1pc3Npb246
MTsKLSAgICBib29sIHJlYWRfcGVybWlzc2lvbjoxOwogCiAgICAgLyogaXZo
ZCBkZXZpY2UgZGF0YSBzZXR0aW5ncyAqLwogICAgIHVpbnQ4X3QgZGV2aWNl
X2ZsYWdzOwogCi0gICAgdW5zaWduZWQgbG9uZyBhZGRyX3JhbmdlX3N0YXJ0
OwotICAgIHVuc2lnbmVkIGxvbmcgYWRkcl9yYW5nZV9sZW5ndGg7CiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CisgICAgc3RydWN0IGl2cnNfdW5p
dHlfbWFwICp1bml0eV9tYXA7CiAKICAgICAvKiBwZXIgZGV2aWNlIGludGVy
cnVwdCByZW1hcHBpbmcgdGFibGUgKi8KICAgICB2b2lkICppbnRyZW1hcF90
YWJsZTsKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11
X2FjcGkuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCkBAIC0xNDYsMzIgKzE0Niw0OCBAQCBzdGF0aWMgaW50IF9f
aW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvCiAgICAgcmV0dXJuIDA7CiB9
CiAKLXN0YXRpYyB2b2lkIF9faW5pdCByZXNlcnZlX3VuaXR5X21hcF9mb3Jf
ZGV2aWNlKAotICAgIHUxNiBzZWcsIHUxNiBiZGYsIHVuc2lnbmVkIGxvbmcg
YmFzZSwKLSAgICB1bnNpZ25lZCBsb25nIGxlbmd0aCwgdTggaXcsIHU4IGly
KQorc3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2Rl
dmljZSgKKyAgICB1aW50MTZfdCBzZWcsIHVpbnQxNl90IGJkZiwgdW5zaWdu
ZWQgbG9uZyBiYXNlLAorICAgIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLCBib29s
IGl3LCBib29sIGlyKQogewogICAgIHN0cnVjdCBpdnJzX21hcHBpbmdzICpp
dnJzX21hcHBpbmdzID0gZ2V0X2l2cnNfbWFwcGluZ3Moc2VnKTsKLSAgICB1
bnNpZ25lZCBsb25nIG9sZF90b3AsIG5ld190b3A7CisgICAgc3RydWN0IGl2
cnNfdW5pdHlfbWFwICp1bml0eV9tYXAgPSBpdnJzX21hcHBpbmdzW2JkZl0u
dW5pdHlfbWFwOwogCi0gICAgLyogbmVlZCB0byBleHRlbmQgdW5pdHktbWFw
cGVkIHJhbmdlPyAqLwotICAgIGlmICggaXZyc19tYXBwaW5nc1tiZGZdLnVu
aXR5X21hcF9lbmFibGUgKQorICAgIC8qIENoZWNrIGZvciBvdmVybGFwcy4g
Ki8KKyAgICBmb3IgKCA7IHVuaXR5X21hcDsgdW5pdHlfbWFwID0gdW5pdHlf
bWFwLT5uZXh0ICkKICAgICB7Ci0gICAgICAgIG9sZF90b3AgPSBpdnJzX21h
cHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9zdGFydCArCi0gICAgICAgICAgICBp
dnJzX21hcHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9sZW5ndGg7Ci0gICAgICAg
IG5ld190b3AgPSBiYXNlICsgbGVuZ3RoOwotICAgICAgICBpZiAoIG9sZF90
b3AgPiBuZXdfdG9wICkKLSAgICAgICAgICAgIG5ld190b3AgPSBvbGRfdG9w
OwotICAgICAgICBpZiAoIGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3Jhbmdl
X3N0YXJ0IDwgYmFzZSApCi0gICAgICAgICAgICBiYXNlID0gaXZyc19tYXBw
aW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQ7Ci0gICAgICAgIGxlbmd0aCA9
IG5ld190b3AgLSBiYXNlOwotICAgIH0KLQotICAgIC8qIGV4dGVuZCByL3cg
cGVybWlzc2lvbXMgYW5kIGtlZXAgYWdncmVnYXRlICovCi0gICAgaXZyc19t
YXBwaW5nc1tiZGZdLndyaXRlX3Blcm1pc3Npb24gPSBpdzsKLSAgICBpdnJz
X21hcHBpbmdzW2JkZl0ucmVhZF9wZXJtaXNzaW9uID0gaXI7Ci0gICAgaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcF9lbmFibGUgPSB0cnVlOwotICAg
IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3JhbmdlX3N0YXJ0ID0gYmFzZTsK
LSAgICBpdnJzX21hcHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9sZW5ndGggPSBs
ZW5ndGg7CisgICAgICAgIC8qCisgICAgICAgICAqIEV4YWN0IG1hdGNoZXMg
YXJlIG9rYXkuIFRoaXMgY2FuIGluIHBhcnRpY3VsYXIgaGFwcGVuIHdoZW4K
KyAgICAgICAgICogcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZp
Y2UoKSBjYWxscyBoZXJlIHR3aWNlIGZvciB0aGUKKyAgICAgICAgICogc2Ft
ZSAocyxiLGQsZikuCisgICAgICAgICAqLworICAgICAgICBpZiAoIGJhc2Ug
PT0gdW5pdHlfbWFwLT5hZGRyICYmIGxlbmd0aCA9PSB1bml0eV9tYXAtPmxl
bmd0aCAmJgorICAgICAgICAgICAgIGlyID09IHVuaXR5X21hcC0+cmVhZCAm
JiBpdyA9PSB1bml0eV9tYXAtPndyaXRlICkKKyAgICAgICAgICAgIHJldHVy
biAwOworCisgICAgICAgIGlmICggdW5pdHlfbWFwLT5hZGRyICsgdW5pdHlf
bWFwLT5sZW5ndGggPiBiYXNlICYmCisgICAgICAgICAgICAgYmFzZSArIGxl
bmd0aCA+IHVuaXR5X21hcC0+YWRkciApCisgICAgICAgIHsKKyAgICAgICAg
ICAgIEFNRF9JT01NVV9ERUJVRygiSVZNRCBFcnJvcjogb3ZlcmxhcCBbJWx4
LCVseCkgdnMgWyVseCwlbHgpXG4iLAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGJhc2UsIGJhc2UgKyBsZW5ndGgsIHVuaXR5X21hcC0+YWRkciwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPmFkZHIg
KyB1bml0eV9tYXAtPmxlbmd0aCk7CisgICAgICAgICAgICByZXR1cm4gLUVQ
RVJNOworICAgICAgICB9CisgICAgfQorCisgICAgLyogUG9wdWxhdGUgYW5k
IGluc2VydCBhIG5ldyB1bml0eSBtYXAuICovCisgICAgdW5pdHlfbWFwID0g
eG1hbGxvYyhzdHJ1Y3QgaXZyc191bml0eV9tYXApOworICAgIGlmICggIXVu
aXR5X21hcCApCisgICAgICAgIHJldHVybiAtRU5PTUVNOworCisgICAgdW5p
dHlfbWFwLT5yZWFkID0gaXI7CisgICAgdW5pdHlfbWFwLT53cml0ZSA9IGl3
OworICAgIHVuaXR5X21hcC0+YWRkciA9IGJhc2U7CisgICAgdW5pdHlfbWFw
LT5sZW5ndGggPSBsZW5ndGg7CisgICAgdW5pdHlfbWFwLT5uZXh0ID0gaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcDsKKyAgICBpdnJzX21hcHBpbmdz
W2JkZl0udW5pdHlfbWFwID0gdW5pdHlfbWFwOworCisgICAgcmV0dXJuIDA7
CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW5nZV9mb3JfYWxsX2RldmljZXMoCkBAIC0xOTQsMTMgKzIxMCwxMyBAQCBz
dGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAg
ICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAtIGJhc2U7CiAgICAgICAgIC8qIHJl
c2VydmUgci93IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmlj
ZXMgKi8KICAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFy
dCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJk
ZiA9IDA7IGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAg
ICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYs
IGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsKKyAgICAgICAgZm9yICggYmRmID0g
MDsgIXJjICYmIGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKKyAg
ICAgICAgICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShz
ZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOwogICAgICAgICAvKiBw
dXNoICdiYXNlJyBqdXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNw
YWNlICovCiAgICAgICAgIGJhc2UgPSBpb21tdV90b3A7CiAgICAgfQogICAg
IC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAq
LwotICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkKKyAgICBpZiAoICFy
YyAmJiBsaW1pdCA+PSBpb21tdV90b3AgKQogICAgIHsKICAgICAgICAgZm9y
X2VhY2hfYW1kX2lvbW11KCBpb21tdSApCiAgICAgICAgIHsKQEAgLTI0Miwx
NSArMjU4LDE1IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1
c2lvbl9yYW4KICAgICAgICAgbGVuZ3RoID0gcmFuZ2VfdG9wIC0gYmFzZTsK
ICAgICAgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVz
IGZvciBkZXZpY2UgKi8KICAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmll
cyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2Us
IGxlbmd0aCwgaXcsIGlyKTsKLSAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBf
Zm9yX2RldmljZShzZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOwor
ICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2Vn
LCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKSA/OgorICAgICAgICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2Us
IGxlbmd0aCwgaXcsIGlyKTsKIAogICAgICAgICAvKiBwdXNoICdiYXNlJyBq
dXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAg
ICAgIGJhc2UgPSBpb21tdV90b3A7CiAgICAgfQogCiAgICAgLyogcmVnaXN0
ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzIGZvciBkZXZpY2Ug
Ki8KLSAgICBpZiAoIGxpbWl0ID49IGlvbW11X3RvcCAgKQorICAgIGlmICgg
IXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCAgKQogICAgIHsKICAgICAgICAg
cmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFz
ZSwgbGltaXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwpAQCAtMjgxLDE1
ICsyOTcsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVz
aW9uX3JhbgogICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwog
ICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2VzICovCiAgICAgICAgIC8qIG5vdGU6IHRoZXNlIGVu
dHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAg
ICAgICBmb3IgKCBiZGYgPSAwOyBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBi
ZGYrKyApCisgICAgICAgIGZvciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYgPCBp
dnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCiAgICAgICAgIHsKICAgICAgICAg
ICAgIGlmICggaW9tbXUgPT0gZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11
LT5zZWcsIGJkZikgKQogICAgICAgICAgICAgewotICAgICAgICAgICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRm
LCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpOwogICAgICAgICAgICAgICAgIHJl
cSA9IGdldF9pdnJzX21hcHBpbmdzKGlvbW11LT5zZWcpW2JkZl0uZHRlX3Jl
cXVlc3Rvcl9pZDsKLSAgICAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21h
cF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAot
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
aXcsIGlyKTsKKyAgICAgICAgICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlf
bWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgs
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGl3LCBpcikgPzoKKyAgICAgICAgICAgICAgICAgICAgIHJlc2Vy
dmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgcmVxLCBiYXNl
LCBsZW5ndGgsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGl3LCBpcik7CiAgICAgICAgICAgICB9CiAgICAg
ICAgIH0KIApAQCAtMjk4LDcgKzMxNCw3IEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICB9CiAKICAgICAvKiByZWdp
c3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ugc2V0dGluZ3MgKi8KLSAgICBp
ZiAoIGxpbWl0ID49IGlvbW11X3RvcCApCisgICAgaWYgKCAhcmMgJiYgbGlt
aXQgPj0gaW9tbXVfdG9wICkKICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBh
bGwgKi8sIGl3LCBpcik7CiAKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNz
dGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0zNjYsMTUgKzM2Niwx
NyBAQCBzdGF0aWMgaW50IGFtZF9pb21tdV9hc3NpZ25fZGV2aWNlKHN0cnVj
CiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwogICAgIGludCBiZGYgPSBQ
Q0lfQkRGMihwZGV2LT5idXMsIGRldmZuKTsKICAgICBpbnQgcmVxX2lkID0g
Z2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYpOworICAgIGNv
bnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwOwogCi0gICAg
aWYgKCBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwX2VuYWJsZSAp
CisgICAgZm9yICggdW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRd
LnVuaXR5X21hcDsgdW5pdHlfbWFwOworICAgICAgICAgIHVuaXR5X21hcCA9
IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewotICAgICAgICBhbWRfaW9tbXVf
cmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKAotICAgICAgICAgICAgZCwKLSAg
ICAgICAgICAgIGl2cnNfbWFwcGluZ3NbcmVxX2lkXS5hZGRyX3JhbmdlX3N0
YXJ0LAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLmFkZHJf
cmFuZ2VfbGVuZ3RoLAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFf
aWRdLndyaXRlX3Blcm1pc3Npb24sCi0gICAgICAgICAgICBpdnJzX21hcHBp
bmdzW3JlcV9pZF0ucmVhZF9wZXJtaXNzaW9uKTsKKyAgICAgICAgaW50IHJj
ID0gYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAg
ICAgICAgICAgICAgICAgIGQsIHVuaXR5X21hcC0+YWRkciwgdW5pdHlfbWFw
LT5sZW5ndGgsCisgICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPndy
aXRlLCB1bml0eV9tYXAtPnJlYWQpOworCisgICAgICAgIGlmICggcmMgKQor
ICAgICAgICAgICAgcmV0dXJuIHJjOwogICAgIH0KIAogICAgIHJldHVybiBy
ZWFzc2lnbl9kZXZpY2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwgcGRldik7
Cg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0tZXB0
LmMKQEAgLTY4MCw3ICs2ODAsNyBAQCBlcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmCiAgICAgdWludDhfdCBpcGF0ID0gMDsKICAg
ICBib29sX3QgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMTsKICAgICBib29s
X3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsKLSAgICB1bnNpZ25lZCBpbnQgaW9t
bXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIG1mbik7Cisg
ICAgdW5zaWduZWQgaW50IGlvbW11X2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBwMm1hLCBtZm4pOwogICAgIGJvb2xfdCBuZWVkc19zeW5j
ID0gMTsKICAgICBlcHRfZW50cnlfdCBvbGRfZW50cnkgPSB7IC5lcHRlID0g
MCB9OwogICAgIGVwdF9lbnRyeV90IG5ld19lbnRyeSA9IHsgLmVwdGUgPSAw
IH07CkBAIC04MDgsOCArODA4LDggQEAgZXB0X3NldF9lbnRyeShzdHJ1Y3Qg
cDJtX2RvbWFpbiAqcDJtLCBnZgogCiAgICAgICAgIC8qIFNhZmUgdG8gcmVh
ZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2UgaG9sZCB0aGUgcDJtIGxvY2sgKi8K
ICAgICAgICAgaWYgKCBlcHRfZW50cnktPm1mbiA9PSBuZXdfZW50cnkubWZu
ICYmCi0gICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhlcHRfZW50
cnktPnNhX3AybXQsIF9tZm4oZXB0X2VudHJ5LT5tZm4pKSA9PQotICAgICAg
ICAgICAgIGlvbW11X2ZsYWdzICkKKyAgICAgICAgICAgICBwMm1fZ2V0X2lv
bW11X2ZsYWdzKGVwdF9lbnRyeS0+c2FfcDJtdCwgZXB0X2VudHJ5LT5hY2Nl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBfbWZuKGVw
dF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVfZmxhZ3MgKQogICAgICAgICAgICAg
bmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMDsKIAogICAgICAgICBlcHRfcDJt
X3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3X2VudHJ5KTsKLS0tIGEveGVuL2Fy
Y2gveDg2L21tL3AybS1wdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0t
cHQuYwpAQCAtNDgwLDYgKzQ4MCwxNiBAQCBpbnQgcDJtX3B0X2hhbmRsZV9k
ZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAgICAgcmV0dXJuIHJjOwogfQogCisv
KiBSZWNvbnN0cnVjdCBhIGZha2UgcDJtX2FjY2Vzc190IGZyb20gc3RvcmVk
IFBURSBmbGFncy4gKi8KK3N0YXRpYyBwMm1fYWNjZXNzX3QgcDJtX2ZsYWdz
X3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQgZmxhZ3MpCit7CisgICAgaWYgKCBm
bGFncyAmIF9QQUdFX1BSRVNFTlQgKQorICAgICAgICByZXR1cm4gcDJtX2Fj
Y2Vzc19uOworCisgICAgLyogTm8gbmVlZCB0byBsb29rIGF0IF9QQUdFX05Y
IGZvciBub3cuICovCisgICAgcmV0dXJuIGZsYWdzICYgX1BBR0VfUlcgPyBw
Mm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vzc19yOworfQorCiAvKiBDaGVja3Mg
b25seSBhcHBsaWNhYmxlIHRvIGVudHJpZXMgd2l0aCBvcmRlciA+IFBBR0Vf
T1JERVJfNEsgKi8KIHN0YXRpYyB2b2lkIGNoZWNrX2VudHJ5KG1mbl90IG1m
biwgcDJtX3R5cGVfdCBuZXcsIHAybV90eXBlX3Qgb2xkLAogICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVyKQpAQCAtNTE0LDcg
KzUyNCw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4g
KnAybSwKICAgICBsMl9wZ2VudHJ5X3QgbDJlX2NvbnRlbnQ7CiAgICAgbDNf
cGdlbnRyeV90IGwzZV9jb250ZW50OwogICAgIGludCByYzsKLSAgICB1bnNp
Z25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdzID0gcDJtX2dldF9pb21tdV9mbGFn
cyhwMm10LCBtZm4pOworICAgIHVuc2lnbmVkIGludCBpb21tdV9wdGVfZmxh
Z3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIHAybWEsIG1mbik7CiAg
ICAgLyoKICAgICAgKiBvbGRfbWZuIGFuZCBpb21tdV9vbGRfZmxhZ3MgY29u
dHJvbCBwb3NzaWJsZSBmbHVzaC91cGRhdGUgbmVlZHMgb24gdGhlCiAgICAg
ICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1c2ggd2hlbiBNRk4gb3IgZmxhZ3Mg
KGkuZS4gcGVybWlzc2lvbnMpIGNoYW5nZS4KQEAgLTU3Nyw2ICs1ODcsNyBA
QCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAg
ICAgICAgICAgICAgICAgb2xkX21mbiA9IGwxZV9nZXRfcGZuKCpwMm1fZW50
cnkpOwogICAgICAgICAgICAgICAgIGlvbW11X29sZF9mbGFncyA9CiAgICAg
ICAgICAgICAgICAgICAgIHAybV9nZXRfaW9tbXVfZmxhZ3MocDJtX2ZsYWdz
X3RvX3R5cGUoZmxhZ3MpLAorICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHAybV9mbGFnc190b19hY2Nlc3MoZmxhZ3MpLAogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF9tZm4ob2xk
X21mbikpOwogICAgICAgICAgICAgfQogICAgICAgICAgICAgZWxzZQpAQCAt
NjE5LDkgKzYzMCwxMCBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1f
ZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRSSUVTKTsKICAgICAgICAgQVNTRVJU
KHAybV9lbnRyeSk7CiAgICAgICAgIG9sZF9tZm4gPSBsMWVfZ2V0X3Bmbigq
cDJtX2VudHJ5KTsKKyAgICAgICAgZmxhZ3MgPSBsMWVfZ2V0X2ZsYWdzKCpw
Mm1fZW50cnkpOwogICAgICAgICBpb21tdV9vbGRfZmxhZ3MgPQotICAgICAg
ICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9fdHlwZShs
MWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpKSwKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7CisgICAgICAgICAgICBw
Mm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9mbGFnc190b190eXBlKGZsYWdzKSwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdzX3Rv
X2FjY2VzcyhmbGFncyksIF9tZm4ob2xkX21mbikpOwogCiAgICAgICAgIGlm
ICggbWZuX3ZhbGlkKG1mbikgfHwgcDJtX2FsbG93c19pbnZhbGlkX21mbihw
Mm10KSApCiAgICAgICAgICAgICBlbnRyeV9jb250ZW50ID0gcDJtX2wxZV9m
cm9tX3BmbihtZm5feChtZm4pLApAQCAtNjQ5LDYgKzY2MSw3IEBAIHAybV9w
dF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAybSwKICAgICAgICAg
ICAgICAgICBvbGRfbWZuID0gbDFlX2dldF9wZm4oKnAybV9lbnRyeSk7CiAg
ICAgICAgICAgICAgICAgaW9tbXVfb2xkX2ZsYWdzID0KICAgICAgICAgICAg
ICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9fdHlw
ZShmbGFncyksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7
CiAgICAgICAgICAgICB9CiAgICAgICAgICAgICBlbHNlCi0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4
Ni9wMm0uaApAQCAtOTAzLDcgKzkwMyw4IEBAIHN0YXRpYyBpbmxpbmUgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnUKIC8qCiAgKiBwMm0gdHlwZSB0byBJ
T01NVSBmbGFncwogICovCi1zdGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBw
Mm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90eXBlX3QgcDJtdCwgbWZuX3QgbWZu
KQorc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJtX2dldF9pb21tdV9m
bGFncyhwMm1fdHlwZV90IHAybXQsCisgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9hY2Nlc3NfdCBwMm1hLCBt
Zm5fdCBtZm4pCiB7CiAgICAgdW5zaWduZWQgaW50IGZsYWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC0x
MzUzLDcgKzEzNTMsNyBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShz
dHJ1Y3QgZG9tYWluCiAgICAgICAgIGlmICggIWlzX2lvbW11X2VuYWJsZWQo
ZCkgKQogICAgICAgICAgICAgcmV0dXJuIDA7CiAgICAgICAgIHJldHVybiBp
b21tdV9sZWdhY3lfbWFwKGQsIF9kZm4oZ2ZuX2wpLCBfbWZuKGdmbl9sKSwg
UEFHRV9PUkRFUl80SywKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgSU9NTVVGX3JlYWRhYmxlIHwgSU9NTVVGX3dyaXRhYmxlKTsKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2FjY2Vzc190b19pb21t
dV9mbGFncyhwMm1hKSk7CiAgICAgfQogCiAgICAgZ2ZuX2xvY2socDJtLCBn
Zm4sIDApOwotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9t
bXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUu
YwpAQCAtNDIsMTIgKzQyLDYgQEAKICNpbmNsdWRlICJ2dGQuaCIKICNpbmNs
dWRlICIuLi9hdHMuaCIKIAotc3RydWN0IG1hcHBlZF9ybXJyIHsKLSAgICBz
dHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7Ci0gICAgdTY0IGJhc2UsIGVuZDsKLSAg
ICB1bnNpZ25lZCBpbnQgY291bnQ7Ci19OwotCiAvKiBQb3NzaWJsZSB1bmZp
bHRlcmVkIExBUElDL01TSSBtZXNzYWdlcyBmcm9tIHVudHJ1c3RlZCBzb3Vy
Y2VzPyAqLwogYm9vbCBfX3JlYWRfbW9zdGx5IHVudHJ1c3RlZF9tc2k7CiAK
QEAgLTE4MDAsMTcgKzE3OTQsMTIgQEAgb3V0Ogogc3RhdGljIHZvaWQgaW9t
bXVfZG9tYWluX3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpCiB7CiAgICAg
c3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7Ci0gICAg
c3RydWN0IG1hcHBlZF9ybXJyICptcm1yciwgKnRtcDsKICAgICBjb25zdCBz
dHJ1Y3QgYWNwaV9kcmhkX3VuaXQgKmRyaGQ7CiAKICAgICBpZiAoIGxpc3Rf
ZW1wdHkoJmFjcGlfZHJoZF91bml0cykgKQogICAgICAgICByZXR1cm47CiAK
LSAgICBsaXN0X2Zvcl9lYWNoX2VudHJ5X3NhZmUgKCBtcm1yciwgdG1wLCAm
aGQtPmFyY2gubWFwcGVkX3JtcnJzLCBsaXN0ICkKLSAgICB7Ci0gICAgICAg
IGxpc3RfZGVsKCZtcm1yci0+bGlzdCk7Ci0gICAgICAgIHhmcmVlKG1ybXJy
KTsKLSAgICB9CisgICAgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKGQp
OwogCiAgICAgQVNTRVJUKGlzX2lvbW11X2VuYWJsZWQoZCkpOwogCkBAIC0x
OTY2LDc0ICsxOTU1LDYgQEAgc3RhdGljIHZvaWQgaW9tbXVfc2V0X3BnZChz
dHJ1Y3QgZG9tYWluCiAgICAgICAgIHBhZ2V0YWJsZV9nZXRfcGFkZHIocGFn
ZXRhYmxlX2Zyb21fbWZuKHBnZF9tZm4pKTsKIH0KIAotc3RhdGljIGludCBy
bXJyX2lkZW50aXR5X21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwgYm9vbF90
IG1hcCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBhY3BpX3JtcnJfdW5pdCAqcm1yciwKLSAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHUzMiBmbGFnKQotewotICAgIHVuc2lnbmVk
IGxvbmcgYmFzZV9wZm4gPSBybXJyLT5iYXNlX2FkZHJlc3MgPj4gUEFHRV9T
SElGVF80SzsKLSAgICB1bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdFX0FM
SUdOXzRLKHJtcnItPmVuZF9hZGRyZXNzKSA+PiBQQUdFX1NISUZUXzRLOwot
ICAgIHN0cnVjdCBtYXBwZWRfcm1yciAqbXJtcnI7Ci0gICAgc3RydWN0IGRv
bWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7Ci0KLSAgICBBU1NFUlQo
cGNpZGV2c19sb2NrZWQoKSk7Ci0gICAgQVNTRVJUKHJtcnItPmJhc2VfYWRk
cmVzcyA8IHJtcnItPmVuZF9hZGRyZXNzKTsKLQotICAgIC8qCi0gICAgICog
Tm8gbmVlZCB0byBhY3F1aXJlIGhkLT5hcmNoLm1hcHBpbmdfbG9jazogQm90
aCBpbnNlcnRpb24gYW5kIHJlbW92YWwKLSAgICAgKiBnZXQgZG9uZSB3aGls
ZSBob2xkaW5nIHBjaWRldnNfbG9jay4KLSAgICAgKi8KLSAgICBsaXN0X2Zv
cl9lYWNoX2VudHJ5KCBtcm1yciwgJmhkLT5hcmNoLm1hcHBlZF9ybXJycywg
bGlzdCApCi0gICAgewotICAgICAgICBpZiAoIG1ybXJyLT5iYXNlID09IHJt
cnItPmJhc2VfYWRkcmVzcyAmJgotICAgICAgICAgICAgIG1ybXJyLT5lbmQg
PT0gcm1yci0+ZW5kX2FkZHJlc3MgKQotICAgICAgICB7Ci0gICAgICAgICAg
ICBpbnQgcmV0ID0gMDsKLQotICAgICAgICAgICAgaWYgKCBtYXAgKQotICAg
ICAgICAgICAgewotICAgICAgICAgICAgICAgICsrbXJtcnItPmNvdW50Owot
ICAgICAgICAgICAgICAgIHJldHVybiAwOwotICAgICAgICAgICAgfQotCi0g
ICAgICAgICAgICBpZiAoIC0tbXJtcnItPmNvdW50ICkKLSAgICAgICAgICAg
ICAgICByZXR1cm4gMDsKLQotICAgICAgICAgICAgd2hpbGUgKCBiYXNlX3Bm
biA8IGVuZF9wZm4gKQotICAgICAgICAgICAgewotICAgICAgICAgICAgICAg
IGlmICggY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KGQsIGJhc2VfcGZuKSAp
Ci0gICAgICAgICAgICAgICAgICAgIHJldCA9IC1FTlhJTzsKLSAgICAgICAg
ICAgICAgICBiYXNlX3BmbisrOwotICAgICAgICAgICAgfQotCi0gICAgICAg
ICAgICBsaXN0X2RlbCgmbXJtcnItPmxpc3QpOwotICAgICAgICAgICAgeGZy
ZWUobXJtcnIpOwotICAgICAgICAgICAgcmV0dXJuIHJldDsKLSAgICAgICAg
fQotICAgIH0KLQotICAgIGlmICggIW1hcCApCi0gICAgICAgIHJldHVybiAt
RU5PRU5UOwotCi0gICAgd2hpbGUgKCBiYXNlX3BmbiA8IGVuZF9wZm4gKQot
ICAgIHsKLSAgICAgICAgaW50IGVyciA9IHNldF9pZGVudGl0eV9wMm1fZW50
cnkoZCwgYmFzZV9wZm4sIHAybV9hY2Nlc3NfcncsIGZsYWcpOwotCi0gICAg
ICAgIGlmICggZXJyICkKLSAgICAgICAgICAgIHJldHVybiBlcnI7Ci0gICAg
ICAgIGJhc2VfcGZuKys7Ci0gICAgfQotCi0gICAgbXJtcnIgPSB4bWFsbG9j
KHN0cnVjdCBtYXBwZWRfcm1ycik7Ci0gICAgaWYgKCAhbXJtcnIgKQotICAg
ICAgICByZXR1cm4gLUVOT01FTTsKLSAgICBtcm1yci0+YmFzZSA9IHJtcnIt
PmJhc2VfYWRkcmVzczsKLSAgICBtcm1yci0+ZW5kID0gcm1yci0+ZW5kX2Fk
ZHJlc3M7Ci0gICAgbXJtcnItPmNvdW50ID0gMTsKLSAgICBsaXN0X2FkZF90
YWlsKCZtcm1yci0+bGlzdCwgJmhkLT5hcmNoLm1hcHBlZF9ybXJycyk7Ci0K
LSAgICByZXR1cm4gMDsKLX0KLQogc3RhdGljIGludCBpbnRlbF9pb21tdV9h
ZGRfZGV2aWNlKHU4IGRldmZuLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikKIHsK
ICAgICBzdHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnI7CkBAIC0yMDY1LDcg
KzE5ODYsOSBAQCBzdGF0aWMgaW50IGludGVsX2lvbW11X2FkZF9kZXZpY2Uo
dTggZGV2CiAgICAgICAgICAgICAgKiBTaW5jZSBSTVJScyBhcmUgYWx3YXlz
IHJlc2VydmVkIGluIHRoZSBlODIwIG1hcCBmb3IgdGhlIGhhcmR3YXJlCiAg
ICAgICAgICAgICAgKiBkb21haW4sIHRoZXJlIHNob3VsZG4ndCBiZSBhIGNv
bmZsaWN0LgogICAgICAgICAgICAgICovCi0gICAgICAgICAgICByZXQgPSBy
bXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWluLCAxLCBybXJyLCAw
KTsKKyAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50aXR5X21hcHBpbmco
cGRldi0+ZG9tYWluLCBwMm1fYWNjZXNzX3J3LAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5iYXNlX2FkZHJlc3Ms
IHJtcnItPmVuZF9hZGRyZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAwKTsKICAgICAgICAgICAgIGlmICggcmV0ICkK
ICAgICAgICAgICAgICAgICBkcHJpbnRrKFhFTkxPR19FUlIgVlREUFJFRklY
LCAiZCVkOiBSTVJSIG1hcHBpbmcgZmFpbGVkXG4iLAogICAgICAgICAgICAg
ICAgICAgICAgICAgcGRldi0+ZG9tYWluLT5kb21haW5faWQpOwpAQCAtMjEx
MCw3ICsyMDMzLDggQEAgc3RhdGljIGludCBpbnRlbF9pb21tdV9yZW1vdmVf
ZGV2aWNlKHU4CiAgICAgICAgICAqIEFueSBmbGFnIGlzIG5vdGhpbmcgdG8g
Y2xlYXIgdGhlc2UgbWFwcGluZ3MgYnV0IGhlcmUKICAgICAgICAgICogaXRz
IGFsd2F5cyBzYWZlIGFuZCBzdHJpY3QgdG8gc2V0IDAuCiAgICAgICAgICAq
LwotICAgICAgICBybXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWlu
LCAwLCBybXJyLCAwKTsKKyAgICAgICAgaW9tbXVfaWRlbnRpdHlfbWFwcGlu
ZyhwZGV2LT5kb21haW4sIHAybV9hY2Nlc3NfeCwgcm1yci0+YmFzZV9hZGRy
ZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmVu
ZF9hZGRyZXNzLCAwKTsKICAgICB9CiAKICAgICByZXR1cm4gZG9tYWluX2Nv
bnRleHRfdW5tYXAocGRldi0+ZG9tYWluLCBkZXZmbiwgcGRldik7CkBAIC0y
MzA5LDcgKzIyMzMsOCBAQCBzdGF0aWMgdm9pZCBfX2h3ZG9tX2luaXQgc2V0
dXBfaHdkb21fcm1yCiAgICAgICAgICAqIGRvbWFpbiwgdGhlcmUgc2hvdWxk
bid0IGJlIGEgY29uZmxpY3QuIFNvIGl0cyBhbHdheXMgc2FmZSBhbmQKICAg
ICAgICAgICogc3RyaWN0IHRvIHNldCAwLgogICAgICAgICAgKi8KLSAgICAg
ICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKGQsIDEsIHJtcnIsIDAp
OworICAgICAgICByZXQgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQsIHAy
bV9hY2Nlc3NfcncsIHJtcnItPmJhc2VfYWRkcmVzcywKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5lbmRfYWRkcmVzcywg
MCk7CiAgICAgICAgIGlmICggcmV0ICkKICAgICAgICAgICAgIGRwcmludGso
WEVOTE9HX0VSUiBWVERQUkVGSVgsCiAgICAgICAgICAgICAgICAgICAgICAi
SU9NTVU6IG1hcHBpbmcgcmVzZXJ2ZWQgcmVnaW9uIGZhaWxlZFxuIik7CkBA
IC0yNDgwLDcgKzI0MDUsOSBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2Rldmlj
ZV9vd25lcnNoaXAoCiAgICAgICAgICAgICAgICAgICogQW55IFJNUlIgZmxh
ZyBpcyBhbHdheXMgaWdub3JlZCB3aGVuIHJlbW92ZSBhIGRldmljZSwKICAg
ICAgICAgICAgICAgICAgKiBidXQgaXRzIGFsd2F5cyBzYWZlIGFuZCBzdHJp
Y3QgdG8gc2V0IDAuCiAgICAgICAgICAgICAgICAgICovCi0gICAgICAgICAg
ICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKHNvdXJjZSwgMCwg
cm1yciwgMCk7CisgICAgICAgICAgICAgICAgcmV0ID0gaW9tbXVfaWRlbnRp
dHlfbWFwcGluZyhzb3VyY2UsIHAybV9hY2Nlc3NfeCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmJhc2Vf
YWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCAwKTsKICAgICAgICAgICAg
ICAgICBpZiAoIHJldCAhPSAtRU5PRU5UICkKICAgICAgICAgICAgICAgICAg
ICAgcmV0dXJuIHJldDsKICAgICAgICAgICAgIH0KQEAgLTI1NzcsNyArMjUw
NCw4IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfYXNzaWduX2RldmljZSgK
ICAgICAgICAgICAgICBQQ0lfQlVTKGJkZikgPT0gYnVzICYmCiAgICAgICAg
ICAgICAgUENJX0RFVkZOMihiZGYpID09IGRldmZuICkKICAgICAgICAgewot
ICAgICAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKGQsIDEs
IHJtcnIsIGZsYWcpOworICAgICAgICAgICAgcmV0ID0gaW9tbXVfaWRlbnRp
dHlfbWFwcGluZyhkLCBwMm1fYWNjZXNzX3J3LCBybXJyLT5iYXNlX2FkZHJl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHJtcnItPmVuZF9hZGRyZXNzLCBmbGFnKTsKICAgICAgICAgICAgIGlmICgg
cmV0ICkKICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICBpbnQgcmM7
Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21tdS5jCisr
KyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21tdS5jCkBAIC0x
MzksNyArMTM5LDcgQEAgaW50IGFyY2hfaW9tbXVfZG9tYWluX2luaXQoc3Ry
dWN0IGRvbWFpbgogICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhkID0gZG9t
X2lvbW11KGQpOwogCiAgICAgc3Bpbl9sb2NrX2luaXQoJmhkLT5hcmNoLm1h
cHBpbmdfbG9jayk7Ci0gICAgSU5JVF9MSVNUX0hFQUQoJmhkLT5hcmNoLm1h
cHBlZF9ybXJycyk7CisgICAgSU5JVF9MSVNUX0hFQUQoJmhkLT5hcmNoLmlk
ZW50aXR5X21hcHMpOwogCiAgICAgcmV0dXJuIDA7CiB9CkBAIC0xNDgsNiAr
MTQ4LDk5IEBAIHZvaWQgYXJjaF9pb21tdV9kb21haW5fZGVzdHJveShzdHJ1
Y3QgZG8KIHsKIH0KIAorc3RydWN0IGlkZW50aXR5X21hcCB7CisgICAgc3Ry
dWN0IGxpc3RfaGVhZCBsaXN0OworICAgIHBhZGRyX3QgYmFzZSwgZW5kOwor
ICAgIHAybV9hY2Nlc3NfdCBhY2Nlc3M7CisgICAgdW5zaWduZWQgaW50IGNv
dW50OworfTsKKworaW50IGlvbW11X2lkZW50aXR5X21hcHBpbmcoc3RydWN0
IGRvbWFpbiAqZCwgcDJtX2FjY2Vzc190IHAybWEsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgZW5kLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGZsYWcpCit7
CisgICAgdW5zaWduZWQgbG9uZyBiYXNlX3BmbiA9IGJhc2UgPj4gUEFHRV9T
SElGVF80SzsKKyAgICB1bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdFX0FM
SUdOXzRLKGVuZCkgPj4gUEFHRV9TSElGVF80SzsKKyAgICBzdHJ1Y3QgaWRl
bnRpdHlfbWFwICptYXA7CisgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQg
PSBkb21faW9tbXUoZCk7CisKKyAgICBBU1NFUlQocGNpZGV2c19sb2NrZWQo
KSk7CisgICAgQVNTRVJUKGJhc2UgPCBlbmQpOworCisgICAgLyoKKyAgICAg
KiBObyBuZWVkIHRvIGFjcXVpcmUgaGQtPmFyY2gubWFwcGluZ19sb2NrOiBC
b3RoIGluc2VydGlvbiBhbmQgcmVtb3ZhbAorICAgICAqIGdldCBkb25lIHdo
aWxlIGhvbGRpbmcgcGNpZGV2c19sb2NrLgorICAgICAqLworICAgIGxpc3Rf
Zm9yX2VhY2hfZW50cnkoIG1hcCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMs
IGxpc3QgKQorICAgIHsKKyAgICAgICAgaWYgKCBtYXAtPmJhc2UgPT0gYmFz
ZSAmJiBtYXAtPmVuZCA9PSBlbmQgKQorICAgICAgICB7CisgICAgICAgICAg
ICBpbnQgcmV0ID0gMDsKKworICAgICAgICAgICAgaWYgKCBwMm1hICE9IHAy
bV9hY2Nlc3NfeCApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAg
aWYgKCBtYXAtPmFjY2VzcyAhPSBwMm1hICkKKyAgICAgICAgICAgICAgICAg
ICAgcmV0dXJuIC1FQUREUklOVVNFOworICAgICAgICAgICAgICAgICsrbWFw
LT5jb3VudDsKKyAgICAgICAgICAgICAgICByZXR1cm4gMDsKKyAgICAgICAg
ICAgIH0KKworICAgICAgICAgICAgaWYgKCAtLW1hcC0+Y291bnQgKQorICAg
ICAgICAgICAgICAgIHJldHVybiAwOworCisgICAgICAgICAgICB3aGlsZSAo
IGJhc2VfcGZuIDwgZW5kX3BmbiApCisgICAgICAgICAgICB7CisgICAgICAg
ICAgICAgICAgaWYgKCBjbGVhcl9pZGVudGl0eV9wMm1fZW50cnkoZCwgYmFz
ZV9wZm4pICkKKyAgICAgICAgICAgICAgICAgICAgcmV0ID0gLUVOWElPOwor
ICAgICAgICAgICAgICAgIGJhc2VfcGZuKys7CisgICAgICAgICAgICB9CisK
KyAgICAgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOworICAgICAgICAg
ICAgeGZyZWUobWFwKTsKKworICAgICAgICAgICAgcmV0dXJuIHJldDsKKyAg
ICAgICAgfQorCisgICAgICAgIGlmICggZW5kID49IG1hcC0+YmFzZSAmJiBt
YXAtPmVuZCA+PSBiYXNlICkKKyAgICAgICAgICAgIHJldHVybiAtRUFERFJJ
TlVTRTsKKyAgICB9CisKKyAgICBpZiAoIHAybWEgPT0gcDJtX2FjY2Vzc194
ICkKKyAgICAgICAgcmV0dXJuIC1FTk9FTlQ7CisKKyAgICB3aGlsZSAoIGJh
c2VfcGZuIDwgZW5kX3BmbiApCisgICAgeworICAgICAgICBpbnQgZXJyID0g
c2V0X2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3BmbiwgcDJtYSwgZmxh
Zyk7CisKKyAgICAgICAgaWYgKCBlcnIgKQorICAgICAgICAgICAgcmV0dXJu
IGVycjsKKyAgICAgICAgYmFzZV9wZm4rKzsKKyAgICB9CisKKyAgICBtYXAg
PSB4bWFsbG9jKHN0cnVjdCBpZGVudGl0eV9tYXApOworICAgIGlmICggIW1h
cCApCisgICAgICAgIHJldHVybiAtRU5PTUVNOworICAgIG1hcC0+YmFzZSA9
IGJhc2U7CisgICAgbWFwLT5lbmQgPSBlbmQ7CisgICAgbWFwLT5hY2Nlc3Mg
PSBwMm1hOworICAgIG1hcC0+Y291bnQgPSAxOworICAgIGxpc3RfYWRkX3Rh
aWwoJm1hcC0+bGlzdCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMpOworCisg
ICAgcmV0dXJuIDA7Cit9CisKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFwX3Rl
YXJkb3duKHN0cnVjdCBkb21haW4gKmQpCit7CisgICAgc3RydWN0IGRvbWFp
bl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7CisgICAgc3RydWN0IGlkZW50
aXR5X21hcCAqbWFwLCAqdG1wOworCisgICAgbGlzdF9mb3JfZWFjaF9lbnRy
eV9zYWZlICggbWFwLCB0bXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzLCBs
aXN0ICkKKyAgICB7CisgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOwor
ICAgICAgICB4ZnJlZShtYXApOworICAgIH0KK30KKwogc3RhdGljIGJvb2wg
X19od2RvbV9pbml0IGh3ZG9tX2lvbW11X21hcChjb25zdCBzdHJ1Y3QgZG9t
YWluICpkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBsb25nIHBmbiwKICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBtYXhfcGZuKQot
LS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L2lvbW11LmgKKysrIGIveGVuL2lu
Y2x1ZGUvYXNtLXg4Ni9pb21tdS5oCkBAIC0xNiw2ICsxNiw3IEBACiAKICNp
bmNsdWRlIDx4ZW4vZXJybm8uaD4KICNpbmNsdWRlIDx4ZW4vbGlzdC5oPgor
I2luY2x1ZGUgPHhlbi9tZW1fYWNjZXNzLmg+CiAjaW5jbHVkZSA8eGVuL3Nw
aW5sb2NrLmg+CiAjaW5jbHVkZSA8YXNtL2FwaWNkZWYuaD4KICNpbmNsdWRl
IDxhc20vcHJvY2Vzc29yLmg+CkBAIC00OSw3ICs1MCw3IEBAIHN0cnVjdCBh
cmNoX2lvbW11CiAgICAgc3BpbmxvY2tfdCBtYXBwaW5nX2xvY2s7ICAgICAg
ICAgICAgLyogaW8gcGFnZSB0YWJsZSBsb2NrICovCiAgICAgaW50IGFnYXc7
ICAgICAvKiBhZGp1c3RlZCBndWVzdCBhZGRyZXNzIHdpZHRoLCAwIGlzIGxl
dmVsIDIgMzAtYml0ICovCiAgICAgdTY0IGlvbW11X2JpdG1hcDsgICAgICAg
ICAgICAgIC8qIGJpdG1hcCBvZiBpb21tdShzKSB0aGF0IHRoZSBkb21haW4g
dXNlcyAqLwotICAgIHN0cnVjdCBsaXN0X2hlYWQgbWFwcGVkX3JtcnJzOwor
ICAgIHN0cnVjdCBsaXN0X2hlYWQgaWRlbnRpdHlfbWFwczsKIAogICAgIC8q
IGFtZCBpb21tdSBzdXBwb3J0ICovCiAgICAgaW50IHBhZ2luZ19tb2RlOwpA
QCAtMTEyLDYgKzExMywxMSBAQCBzdGF0aWMgaW5saW5lIHZvaWQgaW9tbXVf
ZGlzYWJsZV94MmFwaWMoCiAgICAgICAgIGlvbW11X29wcy5kaXNhYmxlX3gy
YXBpYygpOwogfQogCitpbnQgaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhzdHJ1
Y3QgZG9tYWluICpkLCBwMm1fYWNjZXNzX3QgcDJtYSwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBlbmQsCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxhZyk7
Cit2b2lkIGlvbW11X2lkZW50aXR5X21hcF90ZWFyZG93bihzdHJ1Y3QgZG9t
YWluICpkKTsKKwogZXh0ZXJuIGJvb2wgdW50cnVzdGVkX21zaTsKIAogaW50
IHBpX3VwZGF0ZV9pcnRlKGNvbnN0IHN0cnVjdCBwaV9kZXNjICpwaV9kZXNj
LCBjb25zdCBzdHJ1Y3QgcGlycSAqcGlycSwKLS0tIGEveGVuL2luY2x1ZGUv
YXNtLXg4Ni9wMm0uaAorKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L3AybS5o
CkBAIC05MDAsNiArOTAwLDM0IEBAIHN0cnVjdCBwMm1fZG9tYWluICpwMm1f
Z2V0X2FsdHAybShzdHJ1Y3QKIHN0YXRpYyBpbmxpbmUgdm9pZCBwMm1fYWx0
cDJtX2NoZWNrKHN0cnVjdCB2Y3B1ICp2LCB1aW50MTZfdCBpZHgpIHt9CiAj
ZW5kaWYKIAorLyogcDJtIGFjY2VzcyB0byBJT01NVSBmbGFncyAqLworc3Rh
dGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJtX2FjY2Vzc190b19pb21tdV9m
bGFncyhwMm1fYWNjZXNzX3QgcDJtYSkKK3sKKyAgICBzd2l0Y2ggKCBwMm1h
ICkKKyAgICB7CisgICAgY2FzZSBwMm1fYWNjZXNzX3J3OgorICAgIGNhc2Ug
cDJtX2FjY2Vzc19yd3g6CisgICAgICAgIHJldHVybiBJT01NVUZfcmVhZGFi
bGUgfCBJT01NVUZfd3JpdGFibGU7CisKKyAgICBjYXNlIHAybV9hY2Nlc3Nf
cjoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcng6CisgICAgY2FzZSBwMm1fYWNj
ZXNzX3J4MnJ3OgorICAgICAgICByZXR1cm4gSU9NTVVGX3JlYWRhYmxlOwor
CisgICAgY2FzZSBwMm1fYWNjZXNzX3c6CisgICAgY2FzZSBwMm1fYWNjZXNz
X3d4OgorICAgICAgICByZXR1cm4gSU9NTVVGX3dyaXRhYmxlOworCisgICAg
Y2FzZSBwMm1fYWNjZXNzX246CisgICAgY2FzZSBwMm1fYWNjZXNzX3g6Cisg
ICAgY2FzZSBwMm1fYWNjZXNzX24ycnd4OgorICAgICAgICByZXR1cm4gMDsK
KyAgICB9CisKKyAgICBBU1NFUlRfVU5SRUFDSEFCTEUoKTsKKyAgICByZXR1
cm4gMDsKK30KKwogLyoKICAqIHAybSB0eXBlIHRvIElPTU1VIGZsYWdzCiAg
Ki8KQEAgLTkyMSw5ICs5NDksMTAgQEAgc3RhdGljIGlubGluZSB1bnNpZ25l
ZCBpbnQgcDJtX2dldF9pb21tdQogICAgICAgICBmbGFncyA9IElPTU1VRl9y
ZWFkYWJsZTsKICAgICAgICAgYnJlYWs7CiAgICAgY2FzZSBwMm1fbW1pb19k
aXJlY3Q6Ci0gICAgICAgIGZsYWdzID0gSU9NTVVGX3JlYWRhYmxlOwotICAg
ICAgICBpZiAoICFyYW5nZXNldF9jb250YWluc19zaW5nbGV0b24obW1pb19y
b19yYW5nZXMsIG1mbl94KG1mbikpICkKLSAgICAgICAgICAgIGZsYWdzIHw9
IElPTU1VRl93cml0YWJsZTsKKyAgICAgICAgZmxhZ3MgPSBwMm1fYWNjZXNz
X3RvX2lvbW11X2ZsYWdzKHAybWEpOworICAgICAgICBpZiAoIChmbGFncyAm
IElPTU1VRl93cml0YWJsZSkgJiYKKyAgICAgICAgICAgICByYW5nZXNldF9j
b250YWluc19zaW5nbGV0b24obW1pb19yb19yYW5nZXMsIG1mbl94KG1mbikp
ICkKKyAgICAgICAgICAgIGZsYWdzICY9IH5JT01NVUZfd3JpdGFibGU7CiAg
ICAgICAgIGJyZWFrOwogICAgIGRlZmF1bHQ6CiAgICAgICAgIGZsYWdzID0g
MDsK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXUuaAorKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXUu
aApAQCAtMjMyLDggKzIzMiwxMCBAQCBpbnQgX19tdXN0X2NoZWNrIGFtZF9p
b21tdV91bm1hcF9wYWdlKHN0CiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpOwogaW50
IF9fbXVzdF9jaGVjayBhbWRfaW9tbXVfYWxsb2Nfcm9vdChzdHJ1Y3QgZG9t
YWluX2lvbW11ICpoZCk7CiBpbnQgYW1kX2lvbW11X3Jlc2VydmVfZG9tYWlu
X3VuaXR5X21hcChzdHJ1Y3QgZG9tYWluICpkb21haW4sCi0gICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IHBoeXNfYWRk
ciwgdW5zaWduZWQgbG9uZyBzaXplLAotICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgaW50IGl3LCBpbnQgaXIpOworICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGl2
cnNfdW5pdHlfbWFwICptYXAsCisgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxhZyk7CitpbnQgYW1kX2lv
bW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X3VubWFwKHN0cnVjdCBkb21haW4g
KmQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqbWFwKTsKIGludCBfX211
c3RfY2hlY2sgYW1kX2lvbW11X2ZsdXNoX2lvdGxiX3BhZ2VzKHN0cnVjdCBk
b21haW4gKmQsIGRmbl90IGRmbiwKICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBwYWdlX2NvdW50
LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IGZsdXNoX2ZsYWdzKTsKLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X21hcC5jCisrKyBiL3hlbi9kcml2
ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9tYXAuYwpAQCAtNDIwLDM4ICs0
MjAsNDkgQEAgaW50IGFtZF9pb21tdV9mbHVzaF9pb3RsYl9hbGwoc3RydWN0
IGRvbQogICAgIHJldHVybiAwOwogfQogCi1pbnQgYW1kX2lvbW11X3Jlc2Vy
dmVfZG9tYWluX3VuaXR5X21hcChzdHJ1Y3QgZG9tYWluICpkb21haW4sCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90
IHBoeXNfYWRkciwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGxvbmcgc2l6ZSwgaW50IGl3LCBpbnQgaXIpCitp
bnQgYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcChzdHJ1Y3Qg
ZG9tYWluICpkLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgY29uc3Qgc3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXAsCisgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBp
bnQgZmxhZykKIHsKLSAgICB1bnNpZ25lZCBsb25nIG5wYWdlcywgaTsKLSAg
ICB1bnNpZ25lZCBsb25nIGdmbjsKLSAgICB1bnNpZ25lZCBpbnQgZmxhZ3Mg
PSAhIWlyOwotICAgIHVuc2lnbmVkIGludCBmbHVzaF9mbGFncyA9IDA7Ci0g
ICAgaW50IHJ0ID0gMDsKLQotICAgIGlmICggaXcgKQotICAgICAgICBmbGFn
cyB8PSBJT01NVUZfd3JpdGFibGU7Ci0KLSAgICBucGFnZXMgPSByZWdpb25f
dG9fcGFnZXMocGh5c19hZGRyLCBzaXplKTsKLSAgICBnZm4gPSBwaHlzX2Fk
ZHIgPj4gUEFHRV9TSElGVDsKLSAgICBmb3IgKCBpID0gMDsgaSA8IG5wYWdl
czsgaSsrICkKKyAgICBpbnQgcmM7CisKKyAgICBpZiAoIGQgPT0gZG9tX2lv
ICkKKyAgICAgICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCByYyA9IDA7ICFy
YyAmJiBtYXA7IG1hcCA9IG1hcC0+bmV4dCApCiAgICAgewotICAgICAgICB1
bnNpZ25lZCBsb25nIGZyYW1lID0gZ2ZuICsgaTsKKyAgICAgICAgcDJtX2Fj
Y2Vzc190IHAybWEgPSBwMm1fYWNjZXNzX247CisKKyAgICAgICAgaWYgKCBt
YXAtPnJlYWQgKQorICAgICAgICAgICAgcDJtYSB8PSBwMm1fYWNjZXNzX3I7
CisgICAgICAgIGlmICggbWFwLT53cml0ZSApCisgICAgICAgICAgICBwMm1h
IHw9IHAybV9hY2Nlc3NfdzsKIAotICAgICAgICBydCA9IGFtZF9pb21tdV9t
YXBfcGFnZShkb21haW4sIF9kZm4oZnJhbWUpLCBfbWZuKGZyYW1lKSwgZmxh
Z3MsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICZmbHVzaF9m
bGFncyk7Ci0gICAgICAgIGlmICggcnQgIT0gMCApCi0gICAgICAgICAgICBi
cmVhazsKKyAgICAgICAgcmMgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQs
IHAybWEsIG1hcC0+YWRkciwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIG1hcC0+YWRkciArIG1hcC0+bGVuZ3RoIC0gMSwgZmxhZyk7
CiAgICAgfQogCi0gICAgLyogVXNlIHdoaWxlLWJyZWFrIHRvIGF2b2lkIGNv
bXBpbGVyIHdhcm5pbmcgKi8KLSAgICB3aGlsZSAoIGZsdXNoX2ZsYWdzICYm
Ci0gICAgICAgICAgICBhbWRfaW9tbXVfZmx1c2hfaW90bGJfcGFnZXMoZG9t
YWluLCBfZGZuKGdmbiksCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgbnBhZ2VzLCBmbHVzaF9mbGFncykgKQotICAgICAgICBi
cmVhazsKKyAgICByZXR1cm4gcmM7Cit9CisKK2ludCBhbWRfaW9tbXVfcmVz
ZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXApCit7CisgICAgaW50IHJjOwor
CisgICAgaWYgKCBkID09IGRvbV9pbyApCisgICAgICAgIHJldHVybiAwOwor
CisgICAgZm9yICggcmMgPSAwOyBtYXA7IG1hcCA9IG1hcC0+bmV4dCApCisg
ICAgeworICAgICAgICBpbnQgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGlu
ZyhkLCBwMm1fYWNjZXNzX3gsIG1hcC0+YWRkciwKKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRyICsgbWFwLT5s
ZW5ndGggLSAxLCAwKTsKKworICAgICAgICBpZiAoIHJldCAmJiByZXQgIT0g
LUVOT0VOVCAmJiAhcmMgKQorICAgICAgICAgICAgcmMgPSByZXQ7CisgICAg
fQogCi0gICAgcmV0dXJuIHJ0OworICAgIHJldHVybiByYzsKIH0KIAogaW50
IF9faW5pdCBhbWRfaW9tbXVfcXVhcmFudGluZV9pbml0KHN0cnVjdCBkb21h
aW4gKmQpCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9wY2lf
YW1kX2lvbW11LmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1k
L3BjaV9hbWRfaW9tbXUuYwpAQCAtMzI3LDYgKzMyNyw3IEBAIHN0YXRpYyBp
bnQgcmVhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4KICAgICBzdHJ1Y3Qg
YW1kX2lvbW11ICppb21tdTsKICAgICBpbnQgYmRmLCByYzsKICAgICBzdHJ1
Y3QgZG9tYWluX2lvbW11ICp0ID0gZG9tX2lvbW11KHRhcmdldCk7CisgICAg
Y29uc3Qgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwogCiAgICAgYmRmID0gUENJ
X0JERjIocGRldi0+YnVzLCBwZGV2LT5kZXZmbik7CiAgICAgaW9tbXUgPSBm
aW5kX2lvbW11X2Zvcl9kZXZpY2UocGRldi0+c2VnLCBiZGYpOwpAQCAtMzQx
LDEwICszNDIsMjQgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IGRvbWFpbgogCiAgICAgYW1kX2lvbW11X2Rpc2FibGVfZG9tYWluX2Rl
dmljZShzb3VyY2UsIGlvbW11LCBkZXZmbiwgcGRldik7CiAKLSAgICBpZiAo
IGRldmZuID09IHBkZXYtPmRldmZuICkKKyAgICAvKgorICAgICAqIElmIHRo
ZSBkZXZpY2UgYmVsb25ncyB0byB0aGUgaGFyZHdhcmUgZG9tYWluLCBhbmQg
aXQgaGFzIGEgdW5pdHkgbWFwcGluZywKKyAgICAgKiBkb24ndCByZW1vdmUg
aXQgZnJvbSB0aGUgaGFyZHdhcmUgZG9tYWluLCBiZWNhdXNlIEJJT1MgbWF5
IHJlZmVyZW5jZSB0aGF0CisgICAgICogbWFwcGluZy4KKyAgICAgKi8KKyAg
ICBpZiAoICFpc19oYXJkd2FyZV9kb21haW4oc291cmNlKSApCisgICAgewor
ICAgICAgICByYyA9IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV91
bm1hcCgKKyAgICAgICAgICAgICAgICAgc291cmNlLAorICAgICAgICAgICAg
ICAgICBpdnJzX21hcHBpbmdzW2dldF9kbWFfcmVxdWVzdG9yX2lkKHBkZXYt
PnNlZywgYmRmKV0udW5pdHlfbWFwKTsKKyAgICAgICAgaWYgKCByYyApCisg
ICAgICAgICAgICByZXR1cm4gcmM7CisgICAgfQorCisgICAgaWYgKCBkZXZm
biA9PSBwZGV2LT5kZXZmbiAmJiBwZGV2LT5kb21haW4gIT0gZG9tX2lvICkK
ICAgICB7Ci0gICAgICAgIGxpc3RfbW92ZSgmcGRldi0+ZG9tYWluX2xpc3Qs
ICZ0YXJnZXQtPnBkZXZfbGlzdCk7Ci0gICAgICAgIHBkZXYtPmRvbWFpbiA9
IHRhcmdldDsKKyAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21haW5fbGlz
dCwgJmRvbV9pby0+cGRldl9saXN0KTsKKyAgICAgICAgcGRldi0+ZG9tYWlu
ID0gZG9tX2lvOwogICAgIH0KIAogICAgIHJjID0gYWxsb2NhdGVfZG9tYWlu
X3Jlc291cmNlcyh0KTsKQEAgLTM1Niw2ICszNzEsMTIgQEAgc3RhdGljIGlu
dCByZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbgogICAgICAgICAgICAg
ICAgICAgICBwZGV2LT5zZWcsIHBkZXYtPmJ1cywgUENJX1NMT1QoZGV2Zm4p
LCBQQ0lfRlVOQyhkZXZmbiksCiAgICAgICAgICAgICAgICAgICAgIHNvdXJj
ZS0+ZG9tYWluX2lkLCB0YXJnZXQtPmRvbWFpbl9pZCk7CiAKKyAgICBpZiAo
IGRldmZuID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRvbWFpbiAhPSB0YXJn
ZXQgKQorICAgIHsKKyAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21haW5f
bGlzdCwgJnRhcmdldC0+cGRldl9saXN0KTsKKyAgICAgICAgcGRldi0+ZG9t
YWluID0gdGFyZ2V0OworICAgIH0KKwogICAgIHJldHVybiAwOwogfQogCkBA
IC0zNjYsMjAgKzM4NywyOCBAQCBzdGF0aWMgaW50IGFtZF9pb21tdV9hc3Np
Z25fZGV2aWNlKHN0cnVjCiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2
cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwog
ICAgIGludCBiZGYgPSBQQ0lfQkRGMihwZGV2LT5idXMsIGRldmZuKTsKICAg
ICBpbnQgcmVxX2lkID0gZ2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRldi0+c2Vn
LCBiZGYpOwotICAgIGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5p
dHlfbWFwOworICAgIGludCByYyA9IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFp
bl91bml0eV9tYXAoCisgICAgICAgICAgICAgICAgIGQsIGl2cnNfbWFwcGlu
Z3NbcmVxX2lkXS51bml0eV9tYXAsIGZsYWcpOworCisgICAgaWYgKCAhcmMg
KQorICAgICAgICByYyA9IHJlYXNzaWduX2RldmljZShwZGV2LT5kb21haW4s
IGQsIGRldmZuLCBwZGV2KTsKIAotICAgIGZvciAoIHVuaXR5X21hcCA9IGl2
cnNfbWFwcGluZ3NbcmVxX2lkXS51bml0eV9tYXA7IHVuaXR5X21hcDsKLSAg
ICAgICAgICB1bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQorICAgIGlm
ICggcmMgJiYgIWlzX2hhcmR3YXJlX2RvbWFpbihkKSApCiAgICAgewotICAg
ICAgICBpbnQgcmMgPSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlf
bWFwKAotICAgICAgICAgICAgICAgICAgICAgZCwgdW5pdHlfbWFwLT5hZGRy
LCB1bml0eV9tYXAtPmxlbmd0aCwKLSAgICAgICAgICAgICAgICAgICAgIHVu
aXR5X21hcC0+d3JpdGUsIHVuaXR5X21hcC0+cmVhZCk7CisgICAgICAgIGlu
dCByZXQgPSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAo
CisgICAgICAgICAgICAgICAgICAgICAgZCwgaXZyc19tYXBwaW5nc1tyZXFf
aWRdLnVuaXR5X21hcCk7CiAKLSAgICAgICAgaWYgKCByYyApCi0gICAgICAg
ICAgICByZXR1cm4gcmM7CisgICAgICAgIGlmICggcmV0ICkKKyAgICAgICAg
eworICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIkFNRC1WaTogIgor
ICAgICAgICAgICAgICAgICAgICJ1bml0eS11bm1hcCBmb3IgJXBkLyUwNHg6
JTAyeDolMDJ4LiV1IGZhaWxlZCAoJWQpXG4iLAorICAgICAgICAgICAgICAg
ICAgIGQsIHBkZXYtPnNlZywgcGRldi0+YnVzLAorICAgICAgICAgICAgICAg
ICAgIFBDSV9TTE9UKGRldmZuKSwgUENJX0ZVTkMoZGV2Zm4pLCByZXQpOwor
ICAgICAgICAgICAgZG9tYWluX2NyYXNoKGQpOworICAgICAgICB9CiAgICAg
fQogCi0gICAgcmV0dXJuIHJlYXNzaWduX2RldmljZShwZGV2LT5kb21haW4s
IGQsIGRldmZuLCBwZGV2KTsKKyAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRp
YyB2b2lkIGRlYWxsb2NhdGVfbmV4dF9wYWdlX3RhYmxlKHN0cnVjdCBwYWdl
X2luZm8gKnBnLCBpbnQgbGV2ZWwpCkBAIC00MzgsNiArNDY3LDcgQEAgc3Rh
dGljIHZvaWQgZGVhbGxvY2F0ZV9pb21tdV9wYWdlX3RhYmxlcwogCiBzdGF0
aWMgdm9pZCBhbWRfaW9tbXVfZG9tYWluX2Rlc3Ryb3koc3RydWN0IGRvbWFp
biAqZCkKIHsKKyAgICBpb21tdV9pZGVudGl0eV9tYXBfdGVhcmRvd24oZCk7
CiAgICAgZGVhbGxvY2F0ZV9pb21tdV9wYWdlX3RhYmxlcyhkKTsKICAgICBh
bWRfaW9tbXVfZmx1c2hfYWxsX3BhZ2VzKGQpOwogfQo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdS5oCisrKyBiL3hlbi9kcml2
ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdS5oCkBAIC0zMDgsNiArMzA4LDgg
QEAgZXh0ZXJuIHN0cnVjdCBocGV0X3NiZGYgewogICAgIH0gaW5pdDsKIH0g
aHBldF9zYmRmOwogCitleHRlcm4gaW50IGFtZF9pb21tdV9taW5fcGFnaW5n
X21vZGU7CisKIGV4dGVybiB2b2lkICpzaGFyZWRfaW50cmVtYXBfdGFibGU7
CiBleHRlcm4gdW5zaWduZWQgbG9uZyAqc2hhcmVkX2ludHJlbWFwX2ludXNl
OwogCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9h
Y3BpLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11
X2FjcGkuYwpAQCAtMTE4LDEyICsxMTgsOCBAQCBzdGF0aWMgc3RydWN0IGFt
ZF9pb21tdSAqIF9faW5pdCBmaW5kX2lvCiB9CiAKIHN0YXRpYyBpbnQgX19p
bml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKAotICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBs
aW1pdCwKLSAgICBib29sIGFsbCwgYm9vbCBpdywgYm9vbCBpcikKKyAgICBz
dHJ1Y3QgYW1kX2lvbW11ICppb21tdSwgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGxpbWl0LCBib29sIGFsbCkKIHsKLSAgICBpZiAoICFpciB8fCAhaXcgKQot
ICAgICAgICByZXR1cm4gLUVQRVJNOwotCiAgICAgLyogbmVlZCB0byBleHRl
bmQgZXhjbHVzaW9uIHJhbmdlPyAqLwogICAgIGlmICggaW9tbXUtPmV4Y2x1
c2lvbl9lbmFibGUgKQogICAgIHsKQEAgLTE1MiwxNCArMTQ4LDE4IEBAIHN0
YXRpYyBpbnQgX19pbml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl8KIHsKICAg
ICBzdHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9p
dnJzX21hcHBpbmdzKHNlZyk7CiAgICAgc3RydWN0IGl2cnNfdW5pdHlfbWFw
ICp1bml0eV9tYXAgPSBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwOwor
ICAgIGludCBwYWdpbmdfbW9kZSA9IGFtZF9pb21tdV9nZXRfcGFnaW5nX21v
ZGUoUEZOX1VQKGJhc2UgKyBsZW5ndGgpKTsKKworICAgIGlmICggcGFnaW5n
X21vZGUgPCAwICkKKyAgICAgICAgcmV0dXJuIHBhZ2luZ19tb2RlOwogCiAg
ICAgLyogQ2hlY2sgZm9yIG92ZXJsYXBzLiAqLwogICAgIGZvciAoIDsgdW5p
dHlfbWFwOyB1bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQogICAgIHsK
ICAgICAgICAgLyoKICAgICAgICAgICogRXhhY3QgbWF0Y2hlcyBhcmUgb2th
eS4gVGhpcyBjYW4gaW4gcGFydGljdWxhciBoYXBwZW4gd2hlbgotICAgICAg
ICAgKiByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgpIGNh
bGxzIGhlcmUgdHdpY2UgZm9yIHRoZQotICAgICAgICAgKiBzYW1lIChzLGIs
ZCxmKS4KKyAgICAgICAgICogcmVnaXN0ZXJfcmFuZ2VfZm9yX2RldmljZSgp
IGNhbGxzIGhlcmUgdHdpY2UgZm9yIHRoZSBzYW1lCisgICAgICAgICAqIChz
LGIsZCxmKS4KICAgICAgICAgICovCiAgICAgICAgIGlmICggYmFzZSA9PSB1
bml0eV9tYXAtPmFkZHIgJiYgbGVuZ3RoID09IHVuaXR5X21hcC0+bGVuZ3Ro
ICYmCiAgICAgICAgICAgICAgaXIgPT0gdW5pdHlfbWFwLT5yZWFkICYmIGl3
ID09IHVuaXR5X21hcC0+d3JpdGUgKQpAQCAtMTg3LDU1ICsxODcsNTIgQEAg
c3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yXwogICAg
IHVuaXR5X21hcC0+bmV4dCA9IGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9t
YXA7CiAgICAgaXZyc19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcCA9IHVuaXR5
X21hcDsKIAorICAgIGlmICggcGFnaW5nX21vZGUgPiBhbWRfaW9tbXVfbWlu
X3BhZ2luZ19tb2RlICkKKyAgICAgICAgYW1kX2lvbW11X21pbl9wYWdpbmdf
bW9kZSA9IHBhZ2luZ19tb2RlOworCiAgICAgcmV0dXJuIDA7CiB9CiAKLXN0
YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3Jf
YWxsX2RldmljZXMoCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25l
ZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5p
dCByZWdpc3Rlcl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCisgICAgcGFkZHJf
dCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29s
IGV4Y2x1c2lvbikKIHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCi0g
ICAgdW5zaWduZWQgbG9uZyByYW5nZV90b3AsIGlvbW11X3RvcCwgbGVuZ3Ro
OwogICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwotICAgIHVuc2lnbmVk
IGludCBiZGY7CiAgICAgaW50IHJjID0gMDsKIAogICAgIC8qIGlzIHBhcnQg
b2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJT01NVSB2aXJ0dWFsIGFk
ZHJlc3Mgc3BhY2U/ICovCiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJhbWV0
ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICByYW5n
ZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsKLSAgICBpb21tdV90b3AgPSBt
YXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21tdV90
b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21tdV90
b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwotICAg
ICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwotICAgICAgICAvKiBy
ZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZp
Y2VzICovCi0gICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJlIHBh
cnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAgICBmb3IgKCBi
ZGYgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysg
KQotICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2
aWNlKHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0gICAgICAg
IC8qIHB1c2ggJ2Jhc2UnIGp1c3Qgb3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJl
c3Mgc3BhY2UgKi8KLSAgICAgICAgYmFzZSA9IGlvbW11X3RvcDsKLSAgICB9
Ci0gICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRp
bmdzICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkK
KyAgICBpZiAoIGV4Y2x1c2lvbiApCiAgICAgewogICAgICAgICBmb3JfZWFj
aF9hbWRfaW9tbXUoIGlvbW11ICkKICAgICAgICAgewotICAgICAgICAgICAg
cmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFz
ZSwgbGltaXQsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOwotICAgICAg
ICAgICAgaWYgKCByYyApCi0gICAgICAgICAgICAgICAgYnJlYWs7CisgICAg
ICAgICAgICBpbnQgcmV0ID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFu
Z2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICov
KTsKKworICAgICAgICAgICAgaWYgKCByZXQgJiYgIXJjICkKKyAgICAgICAg
ICAgICAgICByYyA9IHJldDsKICAgICAgICAgfQogICAgIH0KIAorICAgIGlm
ICggIWV4Y2x1c2lvbiB8fCByYyApCisgICAgeworICAgICAgICBwYWRkcl90
IGxlbmd0aCA9IGxpbWl0ICsgUEFHRV9TSVpFIC0gYmFzZTsKKyAgICAgICAg
dW5zaWduZWQgaW50IGJkZjsKKworICAgICAgICAvKiByZXNlcnZlIHIvdyB1
bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCisgICAg
ICAgIGZvciAoIGJkZiA9IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNfYmRm
X2VudHJpZXM7IGJkZisrICkKKyAgICAgICAgICAgIHJjID0gcmVzZXJ2ZV91
bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBp
dywgaXIpOworICAgIH0KKwogICAgIHJldHVybiByYzsKIH0KIAotc3RhdGlj
IGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZp
Y2UoCi0gICAgdTE2IGJkZiwgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25l
ZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5p
dCByZWdpc3Rlcl9yYW5nZV9mb3JfZGV2aWNlKAorICAgIHVuc2lnbmVkIGlu
dCBiZGYsIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwKKyAgICBib29s
IGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikKIHsKICAgICBpbnQgc2Vn
ID0gMDsgLyogWFhYICovCiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2
cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhzZWcpOwotICAgIHVu
c2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAg
ICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAgICB1MTYgcmVxOwogICAg
IGludCByYyA9IDA7CkBAIC0yNDksMjcgKzI0NiwxOSBAQCBzdGF0aWMgaW50
IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgcmVxID0gaXZy
c19tYXBwaW5nc1tiZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7CiAKICAgICAvKiBu
b3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2Ut
YWxpZ25lZCAqLwotICAgIHJhbmdlX3RvcCA9IGxpbWl0ICsgUEFHRV9TSVpF
OwotICAgIGlvbW11X3RvcCA9IG1heF9wYWdlICogUEFHRV9TSVpFOwotICAg
IGlmICggYmFzZSA8IGlvbW11X3RvcCApCi0gICAgewotICAgICAgICBpZiAo
IHJhbmdlX3RvcCA+IGlvbW11X3RvcCApCi0gICAgICAgICAgICByYW5nZV90
b3AgPSBpb21tdV90b3A7Ci0gICAgICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAt
IGJhc2U7CisgICAgaWYgKCBleGNsdXNpb24gKQorICAgICAgICByYyA9IHJl
c2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1p
dCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBmYWxzZSAvKiBhbGwgKi8pOworICAgIGlmICggIWV4Y2x1c2lvbiB8fCBy
YyApCisgICAgeworICAgICAgICBwYWRkcl90IGxlbmd0aCA9IGxpbWl0ICsg
UEFHRV9TSVpFIC0gYmFzZTsKKwogICAgICAgICAvKiByZXNlcnZlIHVuaXR5
LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZSAqLwotICAgICAgICAv
KiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNp
b24gcmFuZ2UgKi8KICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9m
b3JfZGV2aWNlKHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcikgPzoK
ICAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNl
ZywgcmVxLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0KLSAgICAgICAgLyog
cHVzaCAnYmFzZScganVzdCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRkcmVzcyBz
cGFjZSAqLwotICAgICAgICBiYXNlID0gaW9tbXVfdG9wOwogICAgIH0KLQot
ICAgIC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5n
cyBmb3IgZGV2aWNlICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9t
bXVfdG9wICApCisgICAgZWxzZQogICAgIHsKLSAgICAgICAgcmMgPSByZXNl
cnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQs
Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwogICAgICAgICBpdnJzX21hcHBp
bmdzW2JkZl0uZHRlX2FsbG93X2V4Y2x1c2lvbiA9IHRydWU7CiAgICAgICAg
IGl2cnNfbWFwcGluZ3NbcmVxXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1
ZTsKICAgICB9CkBAIC0yNzcsNTMgKzI2Niw0MiBAQCBzdGF0aWMgaW50IF9f
aW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgcmV0dXJuIHJjOwog
fQogCi1zdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFu
Z2VfZm9yX2lvbW11X2RldmljZXMoCi0gICAgc3RydWN0IGFtZF9pb21tdSAq
aW9tbXUsCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25n
IGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5pdCByZWdp
c3Rlcl9yYW5nZV9mb3JfaW9tbXVfZGV2aWNlcygKKyAgICBzdHJ1Y3QgYW1k
X2lvbW11ICppb21tdSwgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LAor
ICAgIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewotICAg
IHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsK
KyAgICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRv
IGJlIHBhZ2UtYWxpZ25lZCAqLworICAgIHBhZGRyX3QgbGVuZ3RoID0gbGlt
aXQgKyBQQUdFX1NJWkUgLSBiYXNlOwogICAgIHVuc2lnbmVkIGludCBiZGY7
CiAgICAgdTE2IHJlcTsKLSAgICBpbnQgcmMgPSAwOworICAgIGludCByYzsK
IAotICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBv
ZiBJT01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICovCi0gICAgLyogbm90
ZTogJ2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFs
aWduZWQgKi8KLSAgICByYW5nZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsK
LSAgICBpb21tdV90b3AgPSBtYXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBp
ZiAoIGJhc2UgPCBpb21tdV90b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCBy
YW5nZV90b3AgPiBpb21tdV90b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9w
ID0gaW9tbXVfdG9wOwotICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBi
YXNlOwotICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFn
ZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCi0gICAgICAgIC8qIG5vdGU6IHRo
ZXNlIGVudHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAq
LwotICAgICAgICBmb3IgKCBiZGYgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19i
ZGZfZW50cmllczsgYmRmKysgKQotICAgICAgICB7Ci0gICAgICAgICAgICBp
ZiAoIGlvbW11ID09IGZpbmRfaW9tbXVfZm9yX2RldmljZShpb21tdS0+c2Vn
LCBiZGYpICkKLSAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAgICByZXEg
PSBnZXRfaXZyc19tYXBwaW5ncyhpb21tdS0+c2VnKVtiZGZdLmR0ZV9yZXF1
ZXN0b3JfaWQ7Ci0gICAgICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5
X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZiwgYmFzZSwgbGVuZ3Ro
LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBpdywgaXIpID86Ci0gICAgICAgICAgICAgICAgICAgICByZXNl
cnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFz
ZSwgbGVuZ3RoLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpOwotICAgICAgICAgICAgfQotICAg
ICAgICB9Ci0KLSAgICAgICAgLyogcHVzaCAnYmFzZScganVzdCBvdXRzaWRl
IG9mIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSAqLwotICAgICAgICBiYXNlID0g
aW9tbXVfdG9wOworICAgIGlmICggZXhjbHVzaW9uICkKKyAgICB7CisgICAg
ICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUs
IGJhc2UsIGxpbWl0LCB0cnVlIC8qIGFsbCAqLyk7CisgICAgICAgIGlmICgg
IXJjICkKKyAgICAgICAgICAgIHJldHVybiAwOwogICAgIH0KIAotICAgIC8q
IHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwot
ICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCApCi0gICAgICAg
IHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJh
c2UsIGxpbWl0LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOworICAgIC8qIHJl
c2VydmUgdW5pdHktbWFwcGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAq
LworICAgIGZvciAoIGJkZiA9IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNf
YmRmX2VudHJpZXM7IGJkZisrICkKKyAgICB7CisgICAgICAgIGlmICggaW9t
bXUgIT0gZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikg
KQorICAgICAgICAgICAgY29udGludWU7CisKKyAgICAgICAgcmVxID0gZ2V0
X2l2cnNfbWFwcGluZ3MoaW9tbXUtPnNlZylbYmRmXS5kdGVfcmVxdWVzdG9y
X2lkOworICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZp
Y2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIpID86Cisg
ICAgICAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShpb21t
dS0+c2VnLCByZXEsIGJhc2UsIGxlbmd0aCwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcik7CisgICAgfQogCiAg
ICAgcmV0dXJuIHJjOwogfQogCiBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9p
dm1kX2RldmljZV9zZWxlY3QoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZy
c19tZW1vcnkgKml2bWRfYmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNl
LCB1bnNpZ25lZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFk
ZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBi
b29sIGV4Y2x1c2lvbikKIHsKICAgICB1MTYgYmRmOwogCkBAIC0zMzQsMTIg
KzMxMiwxMiBAQCBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2Rldmlj
ZV9zZWxlCiAgICAgICAgIHJldHVybiAtRU5PREVWOwogICAgIH0KIAotICAg
IHJldHVybiByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZShi
ZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgIHJldHVybiByZWdpc3Rl
cl9yYW5nZV9mb3JfZGV2aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpciwg
ZXhjbHVzaW9uKTsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZt
ZF9kZXZpY2VfcmFuZ2UoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19t
ZW1vcnkgKml2bWRfYmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1
bnNpZ25lZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJf
dCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29s
IGV4Y2x1c2lvbikKIHsKICAgICB1bnNpZ25lZCBpbnQgZmlyc3RfYmRmLCBs
YXN0X2JkZiwgYmRmOwogICAgIGludCBlcnJvcjsKQEAgLTM2MSwxNSArMzM5
LDE1IEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3Jh
bmcKICAgICB9CiAKICAgICBmb3IgKCBiZGYgPSBmaXJzdF9iZGYsIGVycm9y
ID0gMDsgKGJkZiA8PSBsYXN0X2JkZikgJiYgIWVycm9yOyBiZGYrKyApCi0g
ICAgICAgIGVycm9yID0gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9k
ZXZpY2UoCi0gICAgICAgICAgICBiZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIp
OworICAgICAgICBlcnJvciA9IHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2Uo
CisgICAgICAgICAgICBiZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIsIGV4Y2x1
c2lvbik7CiAKICAgICByZXR1cm4gZXJyb3I7CiB9CiAKIHN0YXRpYyBpbnQg
X19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX2lvbW11KAogICAgIGNvbnN0IHN0
cnVjdCBhY3BpX2l2cnNfbWVtb3J5ICppdm1kX2Jsb2NrLAotICAgIHVuc2ln
bmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTggaXcsIHU4
IGlyKQorICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwgYm9vbCBp
dywgYm9vbCBpciwgYm9vbCBleGNsdXNpb24pCiB7CiAgICAgaW50IHNlZyA9
IDA7IC8qIFhYWCAqLwogICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwpA
QCAtMzg0LDE0ICszNjIsMTQgQEAgc3RhdGljIGludCBfX2luaXQgcGFyc2Vf
aXZtZF9kZXZpY2VfaW9tbQogICAgICAgICByZXR1cm4gLUVOT0RFVjsKICAg
ICB9CiAKLSAgICByZXR1cm4gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zv
cl9pb21tdV9kZXZpY2VzKAotICAgICAgICBpb21tdSwgYmFzZSwgbGltaXQs
IGl3LCBpcik7CisgICAgcmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21t
dV9kZXZpY2VzKAorICAgICAgICBpb21tdSwgYmFzZSwgbGltaXQsIGl3LCBp
ciwgZXhjbHVzaW9uKTsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2Vf
aXZtZF9ibG9jayhjb25zdCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZt
ZF9ibG9jaykKIHsKICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0X2FkZHIsIG1l
bV9sZW5ndGgsIGJhc2UsIGxpbWl0OwotICAgIHU4IGl3LCBpcjsKKyAgICBi
b29sIGl3ID0gdHJ1ZSwgaXIgPSB0cnVlLCBleGNsdXNpb24gPSBmYWxzZTsK
IAogICAgIGlmICggaXZtZF9ibG9jay0+aGVhZGVyLmxlbmd0aCA8IHNpemVv
ZigqaXZtZF9ibG9jaykgKQogICAgIHsKQEAgLTQwOCwxMyArMzg2LDExIEBA
IHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfYmxvY2soY29uc3QKICAg
ICAgICAgICAgICAgICAgICAgaXZtZF9ibG9jay0+aGVhZGVyLnR5cGUsIHN0
YXJ0X2FkZHIsIG1lbV9sZW5ndGgpOwogCiAgICAgaWYgKCBpdm1kX2Jsb2Nr
LT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfRVhDTFVTSU9OX1JBTkdFICkK
LSAgICAgICAgaXcgPSBpciA9IElPTU1VX0NPTlRST0xfRU5BQkxFRDsKKyAg
ICAgICAgZXhjbHVzaW9uID0gdHJ1ZTsKICAgICBlbHNlIGlmICggaXZtZF9i
bG9jay0+aGVhZGVyLmZsYWdzICYgQUNQSV9JVk1EX1VOSVRZICkKICAgICB7
Ci0gICAgICAgIGl3ID0gaXZtZF9ibG9jay0+aGVhZGVyLmZsYWdzICYgQUNQ
SV9JVk1EX1JFQUQgPwotICAgICAgICAgICAgSU9NTVVfQ09OVFJPTF9FTkFC
TEVEIDogSU9NTVVfQ09OVFJPTF9ESVNBQkxFRDsKLSAgICAgICAgaXIgPSBp
dm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEUgPwot
ICAgICAgICAgICAgSU9NTVVfQ09OVFJPTF9FTkFCTEVEIDogSU9NTVVfQ09O
VFJPTF9ESVNBQkxFRDsKKyAgICAgICAgaXcgPSBpdm1kX2Jsb2NrLT5oZWFk
ZXIuZmxhZ3MgJiBBQ1BJX0lWTURfUkVBRDsKKyAgICAgICAgaXIgPSBpdm1k
X2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEU7CiAgICAg
fQogICAgIGVsc2UKICAgICB7CkBAIC00MjUsMjAgKzQwMSwyMCBAQCBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2Jsb2NrKGNvbnN0CiAgICAgc3dp
dGNoKCBpdm1kX2Jsb2NrLT5oZWFkZXIudHlwZSApCiAgICAgewogICAgIGNh
c2UgQUNQSV9JVlJTX1RZUEVfTUVNT1JZX0FMTDoKLSAgICAgICAgcmV0dXJu
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCi0g
ICAgICAgICAgICBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0
dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygKKyAgICAgICAg
ICAgIGJhc2UsIGxpbWl0LCBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBj
YXNlIEFDUElfSVZSU19UWVBFX01FTU9SWV9PTkU6Ci0gICAgICAgIHJldHVy
biBwYXJzZV9pdm1kX2RldmljZV9zZWxlY3QoaXZtZF9ibG9jaywKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1p
dCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNl
X3NlbGVjdChpdm1kX2Jsb2NrLCBiYXNlLCBsaW1pdCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lv
bik7CiAKICAgICBjYXNlIEFDUElfSVZSU19UWVBFX01FTU9SWV9SQU5HRToK
LSAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3JhbmdlKGl2bWRf
YmxvY2ssCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNl
X2l2bWRfZGV2aWNlX3JhbmdlKGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGly
LCBleGNsdXNpb24pOwogCiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9NRU1P
UllfSU9NTVU6Ci0gICAgICAgIHJldHVybiBwYXJzZV9pdm1kX2RldmljZV9p
b21tdShpdm1kX2Jsb2NrLAotICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgYmFzZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIHJl
dHVybiBwYXJzZV9pdm1kX2RldmljZV9pb21tdShpdm1kX2Jsb2NrLCBiYXNl
LCBsaW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGRlZmF1bHQ6CiAgICAg
ICAgIEFNRF9JT01NVV9ERUJVRygiSVZNRCBFcnJvcjogSW52YWxpZCBCbG9j
ayBUeXBlIVxuIik7Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Ft
ZC9wY2lfYW1kX2lvbW11LmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYwpAQCAtMjMxLDYgKzIzMSw4IEBAIHN0
YXRpYyBpbnQgX19tdXN0X2NoZWNrIGFsbG9jYXRlX2RvbWFpbl8KICAgICBy
ZXR1cm4gcmM7CiB9CiAKK2ludCBfX3JlYWRfbW9zdGx5IGFtZF9pb21tdV9t
aW5fcGFnaW5nX21vZGUgPSAxOworCiBzdGF0aWMgaW50IGFtZF9pb21tdV9k
b21haW5faW5pdChzdHJ1Y3QgZG9tYWluICpkKQogewogICAgIHN0cnVjdCBk
b21haW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOwpAQCAtMjQyLDExICsy
NDQsMTMgQEAgc3RhdGljIGludCBhbWRfaW9tbXVfZG9tYWluX2luaXQoc3Ry
dWN0CiAgICAgICogLSBIVk0gY291bGQgaW4gcHJpbmNpcGxlIHVzZSAzIG9y
IDQgZGVwZW5kaW5nIG9uIGhvdyBtdWNoIGd1ZXN0CiAgICAgICogICBwaHlz
aWNhbCBhZGRyZXNzIHNwYWNlIHdlIGdpdmUgaXQsIGJ1dCB0aGlzIGlzbid0
IGtub3duIHlldCBzbyB1c2UgNAogICAgICAqICAgdW5pbGF0ZXJhbGx5Lgor
ICAgICAqIC0gVW5pdHkgbWFwcyBtYXkgcmVxdWlyZSBhbiBldmVuIGhpZ2hl
ciBudW1iZXIuCiAgICAgICovCi0gICAgaGQtPmFyY2gucGFnaW5nX21vZGUg
PSBhbWRfaW9tbXVfZ2V0X3BhZ2luZ19tb2RlKAotICAgICAgICBpc19odm1f
ZG9tYWluKGQpCi0gICAgICAgID8gMXVsIDw8IChERUZBVUxUX0RPTUFJTl9B
RERSRVNTX1dJRFRIIC0gUEFHRV9TSElGVCkKLSAgICAgICAgOiBnZXRfdXBw
ZXJfbWZuX2JvdW5kKCkgKyAxKTsKKyAgICBoZC0+YXJjaC5wYWdpbmdfbW9k
ZSA9IG1heChhbWRfaW9tbXVfZ2V0X3BhZ2luZ19tb2RlKAorICAgICAgICAg
ICAgaXNfaHZtX2RvbWFpbihkKQorICAgICAgICAgICAgPyAxdWwgPDwgKERF
RkFVTFRfRE9NQUlOX0FERFJFU1NfV0lEVEggLSBQQUdFX1NISUZUKQorICAg
ICAgICAgICAgOiBnZXRfdXBwZXJfbWZuX2JvdW5kKCkgKyAxKSwKKyAgICAg
ICAgYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZSk7CiAKICAgICByZXR1cm4g
MDsKIH0K

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtODA3LDcgKzgw
Nyw3IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRl
cik7IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2Vu
dHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5V
TEwpOwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFw
Mm1faXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAg
ICAgICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3No
YXJlZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4o
bWZuX3gobWZuKSArIGksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAg
fQogICAgIH0KQEAgLTkzNSwxMyArOTM1LDEzIEBAIGd1ZXN0X3BoeXNtYXBf
YWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAg
ICAgICAgICAgIEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAg
ICB9Ci0gICAgICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNf
Zm9yZWlnbihvdCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90
KSApCiAgICAgICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRu
J3QgYmUgdW5tYXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAq
LworICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFu
dC9mb3JlaWduIHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2Ny
YXNoKGQpOwogICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAg
ICAgICAgCi0gICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAg
ICAgIHJldHVybiAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBp
ZiAoIHAybV9pc19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAg
ICAgICAgewpAQCAtMTAzNSw4ICsxMDM1LDcgQEAgaW50IHAybV9jaGFuZ2Vf
dHlwZV9vbmUoc3RydWN0IGRvbWFpbiAqZAogICAgIHN0cnVjdCBwMm1fZG9t
YWluICpwMm0gPSBwMm1fZ2V0X2hvc3RwMm0oZCk7CiAgICAgaW50IHJjOwog
Ci0gICAgQlVHX09OKHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lzX2dyYW50
KG50KSk7Ci0gICAgQlVHX09OKHAybV9pc19mb3JlaWduKG90KSB8fCBwMm1f
aXNfZm9yZWlnbihudCkpOworICAgIEJVR19PTihwMm1faXNfc3BlY2lhbChv
dCkgfHwgcDJtX2lzX3NwZWNpYWwobnQpKTsKIAogICAgIGdmbl9sb2NrKHAy
bSwgZ2ZuLCAwKTsKIApAQCAtMTI4MywxMSArMTI4MiwxMSBAQCBzdGF0aWMg
aW50IHNldF90eXBlZF9wMm1fZW50cnkoc3RydWN0IGRvCiAgICAgICAgIGdm
bl91bmxvY2socDJtLCBnZm4sIG9yZGVyKTsKICAgICAgICAgcmV0dXJuIGN1
cl9vcmRlciArIDE7CiAgICAgfQotICAgIGlmICggcDJtX2lzX2dyYW50KG90
KSB8fCBwMm1faXNfZm9yZWlnbihvdCkgKQorICAgIGlmICggcDJtX2lzX3Nw
ZWNpYWwob3QpICkKICAgICB7CiAgICAgICAgIGdmbl91bmxvY2socDJtLCBn
Zm4sIG9yZGVyKTsKICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwotICAgICAg
ICByZXR1cm4gLUVOT0VOVDsKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKICAg
ICB9CiAgICAgZWxzZSBpZiAoIHAybV9pc19yYW0ob3QpICkKICAgICB7Ci0t
LSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1
ZGUvYXNtLXg4Ni9wMm0uaApAQCAtMTQxLDYgKzE0MSwxMCBAQCB0eXBlZGVm
IHVuc2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB8IHAybV90b19tYXNrKHAybV9yYW1fbG9nZGlydHkpICkK
ICNkZWZpbmUgUDJNX1NIQVJFRF9UWVBFUyAgIChwMm1fdG9fbWFzayhwMm1f
cmFtX3NoYXJlZCkpCiAKKy8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQg
dXAgdmlhIHNwZWNpYWwgYWNjZXNzb3JzLiAqLworI2RlZmluZSBQMk1fU1BF
Q0lBTF9UWVBFUyAoUDJNX0dSQU5UX1RZUEVTIHwgXAorICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkK
KwogLyogVmFsaWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQg
d2l0aCBhICh2YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9N
Rk5fVFlQRVMgKFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCBwMm1fdG9fbWFzayhw
Mm1fbW1pb19kaXJlY3QpIFwKQEAgLTE2OSw2ICsxNzMsNyBAQCB0eXBlZGVm
IHVuc2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICNkZWZpbmUgcDJtX2lzX3Bh
Z2VkKF90KSAgICAocDJtX3RvX21hc2soX3QpICYgUDJNX1BBR0VEX1RZUEVT
KQogI2RlZmluZSBwMm1faXNfc2hhcmFibGUoX3QpIChwMm1fdG9fbWFzayhf
dCkgJiBQMk1fU0hBUkFCTEVfVFlQRVMpCiAjZGVmaW5lIHAybV9pc19zaGFy
ZWQoX3QpICAgKHAybV90b19tYXNrKF90KSAmIFAyTV9TSEFSRURfVFlQRVMp
CisjZGVmaW5lIHAybV9pc19zcGVjaWFsKF90KSAgKHAybV90b19tYXNrKF90
KSAmIFAyTV9TUEVDSUFMX1RZUEVTKQogI2RlZmluZSBwMm1faXNfYnJva2Vu
KF90KSAgIChwMm1fdG9fbWFzayhfdCkgJiBQMk1fQlJPS0VOX1RZUEVTKQog
I2RlZmluZSBwMm1faXNfZm9yZWlnbihfdCkgIChwMm1fdG9fbWFzayhfdCkg
JiBwMm1fdG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pKQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.14-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.14-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tLmMK
KysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ2NTIsNyArNDY1Miw5IEBA
IGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogCiAgICAgLyogUmVt
b3ZlIHByZXZpb3VzbHkgbWFwcGVkIHBhZ2UgaWYgaXQgd2FzIHByZXNlbnQu
ICovCiAgICAgcHJldl9tZm4gPSBnZXRfZ2ZuKGQsIGdmbl94KGdwZm4pLCAm
cDJtdCk7Ci0gICAgaWYgKCBtZm5fdmFsaWQocHJldl9tZm4pICkKKyAgICBp
ZiAoIHAybXQgPT0gcDJtX21taW9fZGlyZWN0ICkKKyAgICAgICAgcmMgPSAt
RVBFUk07CisgICAgZWxzZSBpZiAoIG1mbl92YWxpZChwcmV2X21mbikgKQog
ICAgIHsKICAgICAgICAgaWYgKCBpc19zcGVjaWFsX3BhZ2UobWZuX3RvX3Bh
Z2UocHJldl9tZm4pKSApCiAgICAgICAgICAgICAvKiBTcGVjaWFsIHBhZ2Vz
IGFyZSBzaW1wbHkgdW5ob29rZWQgZnJvbSB0aGlzIHBoeXMgc2xvdC4gKi8K
LS0tIGEveGVuL2FyY2gveDg2L21tL3AybS5jCisrKyBiL3hlbi9hcmNoL3g4
Ni9tbS9wMm0uYwpAQCAtNzk1LDcgKzc5NSw4IEBAIHAybV9yZW1vdmVfcGFn
ZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtLAogICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgJmN1cl9vcmRlciwgTlVMTCk7CiAK
ICAgICAgICAgaWYgKCBwMm1faXNfdmFsaWQodCkgJiYKLSAgICAgICAgICAg
ICAoIW1mbl92YWxpZChtZm4pIHx8ICFtZm5fZXEobWZuX2FkZChtZm4sIGkp
LCBtZm5fcmV0dXJuKSkgKQorICAgICAgICAgICAgICghbWZuX3ZhbGlkKG1m
bikgfHwgdCA9PSBwMm1fbW1pb19kaXJlY3QgfHwKKyAgICAgICAgICAgICAg
IW1mbl9lcShtZm5fYWRkKG1mbiwgaSksIG1mbl9yZXR1cm4pKSApCiAgICAg
ICAgICAgICByZXR1cm4gLUVJTFNFUTsKIAogICAgICAgICBpICs9ICgxVUwg
PDwgY3VyX29yZGVyKSAtCkBAIC04OTMsNyArODk0LDcgQEAgZ3Vlc3RfcGh5
c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFpbiAqZAogICAgIGlmICggcDJt
X2lzX2ZvcmVpZ24odCkgKQogICAgICAgICByZXR1cm4gLUVJTlZBTDsKIAot
ICAgIGlmICggIW1mbl92YWxpZChtZm4pICkKKyAgICBpZiAoICFtZm5fdmFs
aWQobWZuKSB8fCB0ID09IHAybV9tbWlvX2RpcmVjdCApCiAgICAgewogICAg
ICAgICBBU1NFUlRfVU5SRUFDSEFCTEUoKTsKICAgICAgICAgcmV0dXJuIC1F
SU5WQUw7CkBAIC05MzcsNyArOTM4LDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRf
ZW50cnkoc3RydWN0IGRvbWFpbiAqZAogICAgICAgICB9CiAgICAgICAgIGlm
ICggcDJtX2lzX3NwZWNpYWwob3QpICkKICAgICAgICAgewotICAgICAgICAg
ICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWduIHRo
aXMgd2F5LiAqLworICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFw
cGluZyBncmFudC9mb3JlaWduL2RpcmVjdC1NTUlPIHRoaXMgd2F5LiAqLwog
ICAgICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwogICAgICAgICAgICAgcDJt
X3VubG9jayhwMm0pOwogICAgICAgICAgICAgCkBAIC0xMzg3LDggKzEzODgs
OCBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShzdHJ1Y3QgZG9tYWlu
CiAgKiAgICBvcmRlcisxICBmb3IgY2FsbGVyIHRvIHJldHJ5IHdpdGggb3Jk
ZXIgKGd1YXJhbnRlZWQgc21hbGxlciB0aGFuCiAgKiAgICAgICAgICAgICB0
aGUgb3JkZXIgdmFsdWUgcGFzc2VkIGluKQogICovCi1pbnQgY2xlYXJfbW1p
b19wMm1fZW50cnkoc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBn
Zm5fbCwgbWZuX3QgbWZuLAotICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCBvcmRlcikKK3N0YXRpYyBpbnQgY2xlYXJfbW1pb19wMm1f
ZW50cnkoc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBnZm5fbCwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbWZuX3QgbWZuLCB1
bnNpZ25lZCBpbnQgb3JkZXIpCiB7CiAgICAgaW50IHJjID0gLUVJTlZBTDsK
ICAgICBnZm5fdCBnZm4gPSBfZ2ZuKGdmbl9sKTsKLS0tIGEveGVuL2FyY2gv
eDg2L21tL3AybS1wb2QuYworKysgYi94ZW4vYXJjaC94ODYvbW0vcDJtLXBv
ZC5jCkBAIC0xMjkyLDE3ICsxMjkyLDE3IEBAIGd1ZXN0X3BoeXNtYXBfbWFy
a19wb3B1bGF0ZV9vbl9kZW1hbmQoc3QKIAogICAgICAgICBwMm0tPmdldF9l
bnRyeShwMm0sIGdmbl9hZGQoZ2ZuLCBpKSwgJm90LCAmYSwgMCwgJmN1cl9v
cmRlciwgTlVMTCk7CiAgICAgICAgIG4gPSAxVUwgPDwgbWluKG9yZGVyLCBj
dXJfb3JkZXIpOwotICAgICAgICBpZiAoIHAybV9pc19yYW0ob3QpICkKKyAg
ICAgICAgaWYgKCBvdCA9PSBwMm1fcG9wdWxhdGVfb25fZGVtYW5kICkKKyAg
ICAgICAgeworICAgICAgICAgICAgLyogQ291bnQgaG93IG1hbnkgUG9EIGVu
dHJpZXMgd2UnbGwgYmUgcmVwbGFjaW5nIGlmIHN1Y2Nlc3NmdWwgKi8KKyAg
ICAgICAgICAgIHBvZF9jb3VudCArPSBuOworICAgICAgICB9CisgICAgICAg
IGVsc2UgaWYgKCBvdCAhPSBwMm1faW52YWxpZCAmJiBvdCAhPSBwMm1fbW1p
b19kbSApCiAgICAgICAgIHsKICAgICAgICAgICAgIFAyTV9ERUJVRygiZ2Zu
X3RvX21mbiByZXR1cm5lZCB0eXBlICVkIVxuIiwgb3QpOwogICAgICAgICAg
ICAgcmMgPSAtRUJVU1k7CiAgICAgICAgICAgICBnb3RvIG91dDsKICAgICAg
ICAgfQotICAgICAgICBlbHNlIGlmICggb3QgPT0gcDJtX3BvcHVsYXRlX29u
X2RlbWFuZCApCi0gICAgICAgIHsKLSAgICAgICAgICAgIC8qIENvdW50IGhv
dyBtYW4gUG9EIGVudHJpZXMgd2UnbGwgYmUgcmVwbGFjaW5nIGlmIHN1Y2Nl
c3NmdWwgKi8KLSAgICAgICAgICAgIHBvZF9jb3VudCArPSBuOwotICAgICAg
ICB9CiAgICAgfQogCiAgICAgLyogTm93LCBhY3R1YWxseSBkbyB0aGUgdHdv
LXdheSBtYXBwaW5nICovCi0tLSBhL3hlbi9jb21tb24vbWVtb3J5LmMKKysr
IGIveGVuL2NvbW1vbi9tZW1vcnkuYwpAQCAtMzI5LDcgKzMyOSw3IEBAIGlu
dCBndWVzdF9yZW1vdmVfcGFnZShzdHJ1Y3QgZG9tYWluICpkLAogICAgIH0K
ICAgICBpZiAoIHAybXQgPT0gcDJtX21taW9fZGlyZWN0ICkKICAgICB7Ci0g
ICAgICAgIHJjID0gY2xlYXJfbW1pb19wMm1fZW50cnkoZCwgZ21mbiwgbWZu
LCBQQUdFX09SREVSXzRLKTsKKyAgICAgICAgcmMgPSAtRVBFUk07CiAgICAg
ICAgIGdvdG8gb3V0X3B1dF9nZm47CiAgICAgfQogI2Vsc2UKQEAgLTE3MjEs
NiArMTcyMSwxNSBAQCBpbnQgY2hlY2tfZ2V0X3BhZ2VfZnJvbV9nZm4oc3Ry
dWN0IGRvbWFpCiAgICAgICAgIHJldHVybiAtRUFHQUlOOwogICAgIH0KICNl
bmRpZgorI2lmZGVmIENPTkZJR19YODYKKyAgICBpZiAoIHAybXQgPT0gcDJt
X21taW9fZGlyZWN0ICkKKyAgICB7CisgICAgICAgIGlmICggcGFnZSApCisg
ICAgICAgICAgICBwdXRfcGFnZShwYWdlKTsKKworICAgICAgICByZXR1cm4g
LUVQRVJNOworICAgIH0KKyNlbmRpZgogCiAgICAgaWYgKCAhcGFnZSApCiAg
ICAgICAgIHJldHVybiAtRUlOVkFMOwotLS0gYS94ZW4vaW5jbHVkZS9hc20t
eDg2L3AybS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKQEAg
LTE0Myw3ICsxNDMsOCBAQCB0eXBlZGVmIHVuc2lnbmVkIGludCBwMm1fcXVl
cnlfdDsKIAogLyogVHlwZXMgZXN0YWJsaXNoZWQvY2xlYW5lZCB1cCB2aWEg
c3BlY2lhbCBhY2Nlc3NvcnMuICovCiAjZGVmaW5lIFAyTV9TUEVDSUFMX1RZ
UEVTIChQMk1fR1JBTlRfVFlQRVMgfCBcCi0gICAgICAgICAgICAgICAgICAg
ICAgICAgICBwMm1fdG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pKQorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3Jl
aWduKSB8IFwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV90b19t
YXNrKHAybV9tbWlvX2RpcmVjdCkpCiAKIC8qIFZhbGlkIHR5cGVzIG5vdCBu
ZWNlc3NhcmlseSBhc3NvY2lhdGVkIHdpdGggYSAodmFsaWQpIE1GTi4gKi8K
ICNkZWZpbmUgUDJNX0lOVkFMSURfTUZOX1RZUEVTIChQMk1fUE9EX1RZUEVT
ICAgICAgICAgICAgICAgICAgXApAQCAtNjQ1LDggKzY0Niw2IEBAIGludCBz
ZXRfZm9yZWlnbl9wMm1fZW50cnkoc3RydWN0IGRvbWFpbgogLyogU2V0IG1t
aW8gYWRkcmVzc2VzIGluIHRoZSBwMm0gdGFibGUgKGZvciBwYXNzLXRocm91
Z2gpICovCiBpbnQgc2V0X21taW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4g
KmQsIGdmbl90IGdmbiwgbWZuX3QgbWZuLAogICAgICAgICAgICAgICAgICAg
ICAgICB1bnNpZ25lZCBpbnQgb3JkZXIpOwotaW50IGNsZWFyX21taW9fcDJt
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBt
Zm5fdCBtZm4sCi0gICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IG9yZGVyKTsKIAogLyogU2V0IGlkZW50aXR5IGFkZHJlc3NlcyBpbiB0
aGUgcDJtIHRhYmxlIChmb3IgcGFzcy10aHJvdWdoKSAqLwogaW50IHNldF9p
ZGVudGl0eV9wMm1fZW50cnkoc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQg
bG9uZyBnZm4sCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-1.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZ2xvYmFsIGV4Y2x1c2lvbiByYW5nZSBl
eHRlbmRpbmcKCkJlc2lkZXMgdW5pdHkgbWFwcGluZyByZWdpb25zLCB0aGUg
QU1EIElPTU1VIHNwZWMgYWxzbyBwcm92aWRlcyBmb3IKZXhjbHVzaW9uIHJh
bmdlcyAoYXJlYXMgb2YgbWVtb3J5IG5vdCB0byBiZSBzdWJqZWN0IHRvIERN
QSB0cmFuc2xhdGlvbikKdG8gYmUgc3BlY2lmaWVkIGJ5IGZpcm13YXJlIGlu
IHRoZSBBQ1BJIHRhYmxlcy4gVGhlIHNwZWMgZG9lcyBub3QgcHV0CmFueSBj
b25zdHJhaW50cyBvbiB0aGUgbnVtYmVyIG9mIHN1Y2ggcmVnaW9ucy4KCkJs
aW5kbHkgYXNzdW1pbmcgYWxsIGFkZHJlc3NlcyBiZXR3ZWVuIGFueSB0d28g
c3VjaCByYW5nZXMgc2hvdWxkIGFsc28KYmUgZXhjbHVkZWQgY2FuJ3QgYmUg
cmlnaHQuIFNpbmNlIGhhcmR3YXJlIGhhcyByb29tIGZvciBqdXN0IGEgc2lu
Z2xlCnN1Y2ggcmFuZ2UgKGNvbXByaXNlZCBvZiB0aGUgRXhjbHVzaW9uIEJh
c2UgUmVnaXN0ZXIgYW5kIHRoZSBFeGNsdXNpb24KUmFuZ2UgTGltaXQgUmVn
aXN0ZXIpLCBjb21iaW5lIG9ubHkgYWRqYWNlbnQgb3Igb3ZlcmxhcHBpbmcg
cmVnaW9ucyAoZm9yCm5vdzsgdGhpcyBtYXkgcmVxdWlyZSBmdXJ0aGVyIGFk
anVzdG1lbnQgaW4gY2FzZSB0YWJsZSBlbnRyaWVzIGFyZW4ndApzb3J0ZWQg
YnkgYWRkcmVzcykgd2l0aCBtYXRjaGluZyBleGNsdXNpb25fYWxsb3dfYWxs
IHNldHRpbmdzLiBUaGlzCnJlcXVpcmVzIGJ1YmJsaW5nIHVwIGVycm9yIGlu
ZGljYXRvcnMsIHN1Y2ggdGhhdCBJT01NVSBpbml0IGNhbiBiZQpmYWlsZWQg
d2hlbiBjb25jYXRlbmF0aW9uIHdhc24ndCBwb3NzaWJsZS4KCkZ1cnRoZXJt
b3JlLCBzaW5jZSB0aGUgZXhjbHVzaW9uIHJhbmdlIHNwZWNpZmllZCBpbiBJ
T01NVSByZWdpc3RlcnMKaW1wbGllcyBSL1cgYWNjZXNzLCByZWplY3QgcmVx
dWVzdHMgYXNraW5nIGZvciBsZXNzIHBlcm1pc3Npb25zICh0aGlzCndpbGwg
YmUgYnJvdWdodCBjbG9zZXIgdG8gdGhlIHNwZWMgYnkgYSBzdWJzZXF1ZW50
IGNoYW5nZSkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTM3OCAvIENWRS0yMDIx
LTI4Njk1LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9p
b21tdV9hY3BpLmMKQEAgLTExNiwxMiArMTE2LDIxIEBAIHN0YXRpYyBzdHJ1
Y3QgYW1kX2lvbW11ICogX19pbml0IGZpbmRfaW8KICAgICByZXR1cm4gTlVM
TDsKIH0KIAotc3RhdGljIHZvaWQgX19pbml0IHJlc2VydmVfaW9tbXVfZXhj
bHVzaW9uX3JhbmdlKAotICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCB1
aW50NjRfdCBiYXNlLCB1aW50NjRfdCBsaW1pdCkKK3N0YXRpYyBpbnQgX19p
bml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKAorICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBs
aW1pdCwKKyAgICBib29sIGFsbCwgYm9vbCBpdywgYm9vbCBpcikKIHsKKyAg
ICBpZiAoICFpciB8fCAhaXcgKQorICAgICAgICByZXR1cm4gLUVQRVJNOwor
CiAgICAgLyogbmVlZCB0byBleHRlbmQgZXhjbHVzaW9uIHJhbmdlPyAqLwog
ICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9lbmFibGUgKQogICAgIHsKKyAg
ICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9uX2xpbWl0ICsgUEFHRV9TSVpF
IDwgYmFzZSB8fAorICAgICAgICAgICAgIGxpbWl0ICsgUEFHRV9TSVpFIDwg
aW9tbXUtPmV4Y2x1c2lvbl9iYXNlIHx8CisgICAgICAgICAgICAgaW9tbXUt
PmV4Y2x1c2lvbl9hbGxvd19hbGwgIT0gYWxsICkKKyAgICAgICAgICAgIHJl
dHVybiAtRUJVU1k7CisKICAgICAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9u
X2Jhc2UgPCBiYXNlICkKICAgICAgICAgICAgIGJhc2UgPSBpb21tdS0+ZXhj
bHVzaW9uX2Jhc2U7CiAgICAgICAgIGlmICggaW9tbXUtPmV4Y2x1c2lvbl9s
aW1pdCA+IGxpbWl0ICkKQEAgLTEyOSwxNiArMTM4LDExIEBAIHN0YXRpYyB2
b2lkIF9faW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2kKICAgICB9CiAKICAg
ICBpb21tdS0+ZXhjbHVzaW9uX2VuYWJsZSA9IElPTU1VX0NPTlRST0xfRU5B
QkxFRDsKKyAgICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IGFsbDsK
ICAgICBpb21tdS0+ZXhjbHVzaW9uX2Jhc2UgPSBiYXNlOwogICAgIGlvbW11
LT5leGNsdXNpb25fbGltaXQgPSBsaW1pdDsKLX0KIAotc3RhdGljIHZvaWQg
X19pbml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlX2FsbCgKLSAg
ICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25lZCBsb25n
IGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQpCi17Ci0gICAgcmVzZXJ2ZV9p
b21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0KTsKLSAg
ICBpb21tdS0+ZXhjbHVzaW9uX2FsbG93X2FsbCA9IElPTU1VX0NPTlRST0xf
RU5BQkxFRDsKKyAgICByZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgX19p
bml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoCkBAIC0xNzYsNiAr
MTgwLDcgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbgogICAgIHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3As
IGxlbmd0aDsKICAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAgICB1
bnNpZ25lZCBpbnQgYmRmOworICAgIGludCByYyA9IDA7CiAKICAgICAvKiBp
cyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5nZSBpbnNpZGUgb2YgSU9NTVUgdmly
dHVhbCBhZGRyZXNzIHNwYWNlPyAqLwogICAgIC8qIG5vdGU6ICdsaW1pdCcg
cGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCkBA
IC0xOTcsMTAgKzIwMiwxNSBAQCBzdGF0aWMgaW50IF9faW5pdCByZWdpc3Rl
cl9leGNsdXNpb25fcmFuCiAgICAgaWYgKCBsaW1pdCA+PSBpb21tdV90b3Ag
KQogICAgIHsKICAgICAgICAgZm9yX2VhY2hfYW1kX2lvbW11KCBpb21tdSAp
Ci0gICAgICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZV9h
bGwoaW9tbXUsIGJhc2UsIGxpbWl0KTsKKyAgICAgICAgeworICAgICAgICAg
ICAgcmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwg
YmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOworICAg
ICAgICAgICAgaWYgKCByYyApCisgICAgICAgICAgICAgICAgYnJlYWs7Cisg
ICAgICAgIH0KICAgICB9CiAKLSAgICByZXR1cm4gMDsKKyAgICByZXR1cm4g
cmM7CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW5nZV9mb3JfZGV2aWNlKApAQCAtMjExLDYgKzIyMSw3IEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICB1bnNp
Z25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgdTE2IHJlcTsKKyAgICBp
bnQgcmMgPSAwOwogCiAgICAgaW9tbXUgPSBmaW5kX2lvbW11X2Zvcl9kZXZp
Y2Uoc2VnLCBiZGYpOwogICAgIGlmICggIWlvbW11ICkKQEAgLTI0MCwxMiAr
MjUxLDEzIEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lv
bl9yYW4KICAgICAvKiByZWdpc3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ug
c2V0dGluZ3MgZm9yIGRldmljZSAqLwogICAgIGlmICggbGltaXQgPj0gaW9t
bXVfdG9wICApCiAgICAgewotICAgICAgICByZXNlcnZlX2lvbW11X2V4Y2x1
c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQpOworICAgICAgICByYyA9
IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBs
aW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBmYWxzZSAvKiBhbGwgKi8sIGl3LCBpcik7CiAgICAgICAgIGl2cnNf
bWFwcGluZ3NbYmRmXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1ZTsKICAg
ICAgICAgaXZyc19tYXBwaW5nc1tyZXFdLmR0ZV9hbGxvd19leGNsdXNpb24g
PSB0cnVlOwogICAgIH0KIAotICAgIHJldHVybiAwOworICAgIHJldHVybiBy
YzsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbmdlX2Zvcl9pb21tdV9kZXZpY2VzKApAQCAtMjU1LDYgKzI2Nyw3IEBA
IHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAg
ICB1bnNpZ25lZCBsb25nIHJhbmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7
CiAgICAgdW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOworICAgIGlu
dCByYyA9IDA7CiAKICAgICAvKiBpcyBwYXJ0IG9mIGV4Y2x1c2lvbiByYW5n
ZSBpbnNpZGUgb2YgSU9NTVUgdmlydHVhbCBhZGRyZXNzIHNwYWNlPyAqLwog
ICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8g
YmUgcGFnZS1hbGlnbmVkICovCkBAIC0yODUsOCArMjk4LDEwIEBAIHN0YXRp
YyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KIAogICAgIC8q
IHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwog
ICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkKLSAgICAgICAgcmVzZXJ2
ZV9pb21tdV9leGNsdXNpb25fcmFuZ2VfYWxsKGlvbW11LCBiYXNlLCBsaW1p
dCk7Ci0gICAgcmV0dXJuIDA7CisgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21t
dV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUgLyog
YWxsICovLCBpdywgaXIpOworCisgICAgcmV0dXJuIHJjOwogfQogCiBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9zZWxlY3QoCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-2.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IGNvcnJlY3QgZGV2aWNlIHVuaXR5IG1hcCBoYW5kbGlu
ZwoKQmxpbmRseSBhc3N1bWluZyBhbGwgYWRkcmVzc2VzIGJldHdlZW4gYW55
IHR3byBzdWNoIHJhbmdlcywgc3BlY2lmaWVkIGJ5CmZpcm13YXJlIGluIHRo
ZSBBQ1BJIHRhYmxlcywgc2hvdWxkIGFsc28gYmUgdW5pdHktbWFwcGVkIGNh
bid0IGJlIHJpZ2h0LgpOb3IgY2FuIGl0IGJlIGNvcnJlY3QgdG8gbWVyZ2Ug
cmFuZ2VzIHdpdGggZGlmZmVyaW5nIHBlcm1pc3Npb25zLiBUcmFjawpyYW5n
ZXMgaW5kaXZpZHVhbGx5OyBkb24ndCBtZXJnZSBhdCBhbGwsIGJ1dCBjaGVj
ayBmb3Igb3ZlcmxhcHMgaW5zdGVhZC4KVGhpcyByZXF1aXJlcyBidWJibGlu
ZyB1cCBlcnJvciBpbmRpY2F0b3JzLCBzdWNoIHRoYXQgSU9NTVUgaW5pdCBj
YW4gYmUKZmFpbGVkIHdoZW4gYWxsb2NhdGlvbiBvZiBhIG5ldyB0cmFja2lu
ZyBzdHJ1Y3Qgd2Fzbid0IHBvc3NpYmxlLCBvciBhbgpvdmVybGFwIHdhcyBk
ZXRlY3RlZC4KCkF0IHRoaXMgb2NjYXNpb24gYWxzbyBzdG9wIGlnbm9yaW5n
CmFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoKSdzIHJldHVy
biB2YWx1ZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4IC8gQ1ZFLTIwMjEt
Mjg2OTUuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+ClJldmlld2VkLWJ5OiBHZW9yZ2UgRHVubGFwIDxnZW9yZ2Uu
ZHVubGFwQGNpdHJpeC5jb20+ClJldmlld2VkLWJ5OiBQYXVsIER1cnJhbnQg
PHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdo
L2FtZC9pb21tdS5oCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Ft
ZC9pb21tdS5oCkBAIC0xMDcsMjAgKzEwNywyNCBAQCBzdHJ1Y3QgYW1kX2lv
bW11IHsKICAgICBzdHJ1Y3QgbGlzdF9oZWFkIGF0c19kZXZpY2VzOwogfTsK
IAorc3RydWN0IGl2cnNfdW5pdHlfbWFwIHsKKyAgICBib29sIHJlYWQ6MTsK
KyAgICBib29sIHdyaXRlOjE7CisgICAgcGFkZHJfdCBhZGRyOworICAgIHVu
c2lnbmVkIGxvbmcgbGVuZ3RoOworICAgIHN0cnVjdCBpdnJzX3VuaXR5X21h
cCAqbmV4dDsKK307CisKIHN0cnVjdCBpdnJzX21hcHBpbmdzIHsKICAgICB1
aW50MTZfdCBkdGVfcmVxdWVzdG9yX2lkOwogICAgIGJvb2wgdmFsaWQ6MTsK
ICAgICBib29sIGR0ZV9hbGxvd19leGNsdXNpb246MTsKLSAgICBib29sIHVu
aXR5X21hcF9lbmFibGU6MTsKLSAgICBib29sIHdyaXRlX3Blcm1pc3Npb246
MTsKLSAgICBib29sIHJlYWRfcGVybWlzc2lvbjoxOwogCiAgICAgLyogaXZo
ZCBkZXZpY2UgZGF0YSBzZXR0aW5ncyAqLwogICAgIHVpbnQ4X3QgZGV2aWNl
X2ZsYWdzOwogCi0gICAgdW5zaWduZWQgbG9uZyBhZGRyX3JhbmdlX3N0YXJ0
OwotICAgIHVuc2lnbmVkIGxvbmcgYWRkcl9yYW5nZV9sZW5ndGg7CiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CisgICAgc3RydWN0IGl2cnNfdW5p
dHlfbWFwICp1bml0eV9tYXA7CiAKICAgICAvKiBwZXIgZGV2aWNlIGludGVy
cnVwdCByZW1hcHBpbmcgdGFibGUgKi8KICAgICB2b2lkICppbnRyZW1hcF90
YWJsZTsKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11
X2FjcGkuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXVfYWNwaS5jCkBAIC0xNDUsMzIgKzE0NSw0OCBAQCBzdGF0aWMgaW50IF9f
aW5pdCByZXNlcnZlX2lvbW11X2V4Y2x1c2lvCiAgICAgcmV0dXJuIDA7CiB9
CiAKLXN0YXRpYyB2b2lkIF9faW5pdCByZXNlcnZlX3VuaXR5X21hcF9mb3Jf
ZGV2aWNlKAotICAgIHUxNiBzZWcsIHUxNiBiZGYsIHVuc2lnbmVkIGxvbmcg
YmFzZSwKLSAgICB1bnNpZ25lZCBsb25nIGxlbmd0aCwgdTggaXcsIHU4IGly
KQorc3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2Rl
dmljZSgKKyAgICB1aW50MTZfdCBzZWcsIHVpbnQxNl90IGJkZiwgdW5zaWdu
ZWQgbG9uZyBiYXNlLAorICAgIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLCBib29s
IGl3LCBib29sIGlyKQogewogICAgIHN0cnVjdCBpdnJzX21hcHBpbmdzICpp
dnJzX21hcHBpbmdzID0gZ2V0X2l2cnNfbWFwcGluZ3Moc2VnKTsKLSAgICB1
bnNpZ25lZCBsb25nIG9sZF90b3AsIG5ld190b3A7CisgICAgc3RydWN0IGl2
cnNfdW5pdHlfbWFwICp1bml0eV9tYXAgPSBpdnJzX21hcHBpbmdzW2JkZl0u
dW5pdHlfbWFwOwogCi0gICAgLyogbmVlZCB0byBleHRlbmQgdW5pdHktbWFw
cGVkIHJhbmdlPyAqLwotICAgIGlmICggaXZyc19tYXBwaW5nc1tiZGZdLnVu
aXR5X21hcF9lbmFibGUgKQorICAgIC8qIENoZWNrIGZvciBvdmVybGFwcy4g
Ki8KKyAgICBmb3IgKCA7IHVuaXR5X21hcDsgdW5pdHlfbWFwID0gdW5pdHlf
bWFwLT5uZXh0ICkKICAgICB7Ci0gICAgICAgIG9sZF90b3AgPSBpdnJzX21h
cHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9zdGFydCArCi0gICAgICAgICAgICBp
dnJzX21hcHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9sZW5ndGg7Ci0gICAgICAg
IG5ld190b3AgPSBiYXNlICsgbGVuZ3RoOwotICAgICAgICBpZiAoIG9sZF90
b3AgPiBuZXdfdG9wICkKLSAgICAgICAgICAgIG5ld190b3AgPSBvbGRfdG9w
OwotICAgICAgICBpZiAoIGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3Jhbmdl
X3N0YXJ0IDwgYmFzZSApCi0gICAgICAgICAgICBiYXNlID0gaXZyc19tYXBw
aW5nc1tiZGZdLmFkZHJfcmFuZ2Vfc3RhcnQ7Ci0gICAgICAgIGxlbmd0aCA9
IG5ld190b3AgLSBiYXNlOwotICAgIH0KLQotICAgIC8qIGV4dGVuZCByL3cg
cGVybWlzc2lvbXMgYW5kIGtlZXAgYWdncmVnYXRlICovCi0gICAgaXZyc19t
YXBwaW5nc1tiZGZdLndyaXRlX3Blcm1pc3Npb24gPSBpdzsKLSAgICBpdnJz
X21hcHBpbmdzW2JkZl0ucmVhZF9wZXJtaXNzaW9uID0gaXI7Ci0gICAgaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcF9lbmFibGUgPSB0cnVlOwotICAg
IGl2cnNfbWFwcGluZ3NbYmRmXS5hZGRyX3JhbmdlX3N0YXJ0ID0gYmFzZTsK
LSAgICBpdnJzX21hcHBpbmdzW2JkZl0uYWRkcl9yYW5nZV9sZW5ndGggPSBs
ZW5ndGg7CisgICAgICAgIC8qCisgICAgICAgICAqIEV4YWN0IG1hdGNoZXMg
YXJlIG9rYXkuIFRoaXMgY2FuIGluIHBhcnRpY3VsYXIgaGFwcGVuIHdoZW4K
KyAgICAgICAgICogcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZp
Y2UoKSBjYWxscyBoZXJlIHR3aWNlIGZvciB0aGUKKyAgICAgICAgICogc2Ft
ZSAocyxiLGQsZikuCisgICAgICAgICAqLworICAgICAgICBpZiAoIGJhc2Ug
PT0gdW5pdHlfbWFwLT5hZGRyICYmIGxlbmd0aCA9PSB1bml0eV9tYXAtPmxl
bmd0aCAmJgorICAgICAgICAgICAgIGlyID09IHVuaXR5X21hcC0+cmVhZCAm
JiBpdyA9PSB1bml0eV9tYXAtPndyaXRlICkKKyAgICAgICAgICAgIHJldHVy
biAwOworCisgICAgICAgIGlmICggdW5pdHlfbWFwLT5hZGRyICsgdW5pdHlf
bWFwLT5sZW5ndGggPiBiYXNlICYmCisgICAgICAgICAgICAgYmFzZSArIGxl
bmd0aCA+IHVuaXR5X21hcC0+YWRkciApCisgICAgICAgIHsKKyAgICAgICAg
ICAgIEFNRF9JT01NVV9ERUJVRygiSVZNRCBFcnJvcjogb3ZlcmxhcCBbJWx4
LCVseCkgdnMgWyVseCwlbHgpXG4iLAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGJhc2UsIGJhc2UgKyBsZW5ndGgsIHVuaXR5X21hcC0+YWRkciwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPmFkZHIg
KyB1bml0eV9tYXAtPmxlbmd0aCk7CisgICAgICAgICAgICByZXR1cm4gLUVQ
RVJNOworICAgICAgICB9CisgICAgfQorCisgICAgLyogUG9wdWxhdGUgYW5k
IGluc2VydCBhIG5ldyB1bml0eSBtYXAuICovCisgICAgdW5pdHlfbWFwID0g
eG1hbGxvYyhzdHJ1Y3QgaXZyc191bml0eV9tYXApOworICAgIGlmICggIXVu
aXR5X21hcCApCisgICAgICAgIHJldHVybiAtRU5PTUVNOworCisgICAgdW5p
dHlfbWFwLT5yZWFkID0gaXI7CisgICAgdW5pdHlfbWFwLT53cml0ZSA9IGl3
OworICAgIHVuaXR5X21hcC0+YWRkciA9IGJhc2U7CisgICAgdW5pdHlfbWFw
LT5sZW5ndGggPSBsZW5ndGg7CisgICAgdW5pdHlfbWFwLT5uZXh0ID0gaXZy
c19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcDsKKyAgICBpdnJzX21hcHBpbmdz
W2JkZl0udW5pdHlfbWFwID0gdW5pdHlfbWFwOworCisgICAgcmV0dXJuIDA7
CiB9CiAKIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9y
YW5nZV9mb3JfYWxsX2RldmljZXMoCkBAIC0xOTMsMTMgKzIwOSwxMyBAQCBz
dGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAg
ICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAtIGJhc2U7CiAgICAgICAgIC8qIHJl
c2VydmUgci93IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmlj
ZXMgKi8KICAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFy
dCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJk
ZiA9IDA7IGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKLSAgICAg
ICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYs
IGJhc2UsIGxlbmd0aCwgaXcsIGlyKTsKKyAgICAgICAgZm9yICggYmRmID0g
MDsgIXJjICYmIGJkZiA8IGl2cnNfYmRmX2VudHJpZXM7IGJkZisrICkKKyAg
ICAgICAgICAgIHJjID0gcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShz
ZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOwogICAgICAgICAvKiBw
dXNoICdiYXNlJyBqdXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNw
YWNlICovCiAgICAgICAgIGJhc2UgPSBpb21tdV90b3A7CiAgICAgfQogICAg
IC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAq
LwotICAgIGlmICggbGltaXQgPj0gaW9tbXVfdG9wICkKKyAgICBpZiAoICFy
YyAmJiBsaW1pdCA+PSBpb21tdV90b3AgKQogICAgIHsKICAgICAgICAgZm9y
X2VhY2hfYW1kX2lvbW11KCBpb21tdSApCiAgICAgICAgIHsKQEAgLTI0MSwx
NSArMjU3LDE1IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1
c2lvbl9yYW4KICAgICAgICAgbGVuZ3RoID0gcmFuZ2VfdG9wIC0gYmFzZTsK
ICAgICAgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVz
IGZvciBkZXZpY2UgKi8KICAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmll
cyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2Us
IGxlbmd0aCwgaXcsIGlyKTsKLSAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBf
Zm9yX2RldmljZShzZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLCBpdywgaXIpOwor
ICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2Vn
LCBiZGYsIGJhc2UsIGxlbmd0aCwgaXcsIGlyKSA/OgorICAgICAgICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2Us
IGxlbmd0aCwgaXcsIGlyKTsKIAogICAgICAgICAvKiBwdXNoICdiYXNlJyBq
dXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCiAgICAg
ICAgIGJhc2UgPSBpb21tdV90b3A7CiAgICAgfQogCiAgICAgLyogcmVnaXN0
ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzIGZvciBkZXZpY2Ug
Ki8KLSAgICBpZiAoIGxpbWl0ID49IGlvbW11X3RvcCAgKQorICAgIGlmICgg
IXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCAgKQogICAgIHsKICAgICAgICAg
cmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFz
ZSwgbGltaXQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwpAQCAtMjgwLDE1
ICsyOTYsMTUgQEAgc3RhdGljIGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVz
aW9uX3JhbgogICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwog
ICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRy
aWVzIGZvciBkZXZpY2VzICovCiAgICAgICAgIC8qIG5vdGU6IHRoZXNlIGVu
dHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAg
ICAgICBmb3IgKCBiZGYgPSAwOyBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBi
ZGYrKyApCisgICAgICAgIGZvciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYgPCBp
dnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCiAgICAgICAgIHsKICAgICAgICAg
ICAgIGlmICggaW9tbXUgPT0gZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11
LT5zZWcsIGJkZikgKQogICAgICAgICAgICAgewotICAgICAgICAgICAgICAg
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRm
LCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpOwogICAgICAgICAgICAgICAgIHJl
cSA9IGdldF9pdnJzX21hcHBpbmdzKGlvbW11LT5zZWcpW2JkZl0uZHRlX3Jl
cXVlc3Rvcl9pZDsKLSAgICAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21h
cF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwgbGVuZ3RoLAot
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
aXcsIGlyKTsKKyAgICAgICAgICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlf
bWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgs
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGl3LCBpcikgPzoKKyAgICAgICAgICAgICAgICAgICAgIHJlc2Vy
dmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9tbXUtPnNlZywgcmVxLCBiYXNl
LCBsZW5ndGgsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGl3LCBpcik7CiAgICAgICAgICAgICB9CiAgICAg
ICAgIH0KIApAQCAtMjk3LDcgKzMxMyw3IEBAIHN0YXRpYyBpbnQgX19pbml0
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW4KICAgICB9CiAKICAgICAvKiByZWdp
c3RlciBJT01NVSBleGNsdXNpb24gcmFuZ2Ugc2V0dGluZ3MgKi8KLSAgICBp
ZiAoIGxpbWl0ID49IGlvbW11X3RvcCApCisgICAgaWYgKCAhcmMgJiYgbGlt
aXQgPj0gaW9tbXVfdG9wICkKICAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBh
bGwgKi8sIGl3LCBpcik7CiAKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNz
dGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jCkBAIC0zNjcsMTUgKzM2Nywx
NyBAQCBzdGF0aWMgaW50IGFtZF9pb21tdV9hc3NpZ25fZGV2aWNlKHN0cnVj
CiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwogICAgIGludCBiZGYgPSBQ
Q0lfQkRGMihwZGV2LT5idXMsIGRldmZuKTsKICAgICBpbnQgcmVxX2lkID0g
Z2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYpOworICAgIGNv
bnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFwOwogCi0gICAg
aWYgKCBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFwX2VuYWJsZSAp
CisgICAgZm9yICggdW5pdHlfbWFwID0gaXZyc19tYXBwaW5nc1tyZXFfaWRd
LnVuaXR5X21hcDsgdW5pdHlfbWFwOworICAgICAgICAgIHVuaXR5X21hcCA9
IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewotICAgICAgICBhbWRfaW9tbXVf
cmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKAotICAgICAgICAgICAgZCwKLSAg
ICAgICAgICAgIGl2cnNfbWFwcGluZ3NbcmVxX2lkXS5hZGRyX3JhbmdlX3N0
YXJ0LAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFfaWRdLmFkZHJf
cmFuZ2VfbGVuZ3RoLAotICAgICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFf
aWRdLndyaXRlX3Blcm1pc3Npb24sCi0gICAgICAgICAgICBpdnJzX21hcHBp
bmdzW3JlcV9pZF0ucmVhZF9wZXJtaXNzaW9uKTsKKyAgICAgICAgaW50IHJj
ID0gYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKKyAgICAg
ICAgICAgICAgICAgICAgIGQsIHVuaXR5X21hcC0+YWRkciwgdW5pdHlfbWFw
LT5sZW5ndGgsCisgICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPndy
aXRlLCB1bml0eV9tYXAtPnJlYWQpOworCisgICAgICAgIGlmICggcmMgKQor
ICAgICAgICAgICAgcmV0dXJuIHJjOwogICAgIH0KIAogICAgIHJldHVybiBy
ZWFzc2lnbl9kZXZpY2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwgcGRldik7
Cg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-3.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogYWxzbyBwYXNzIHAybV9hY2Nlc3NfdCB0byBwMm1fZ2V0X2lv
bW11X2ZsYWdzKCkKCkEgc3Vic2VxdWVudCBjaGFuZ2Ugd2lsbCB3YW50IHRv
IGN1c3RvbWl6ZSB0aGUgSU9NTVUgcGVybWlzc2lvbnMgYmFzZWQKb24gdGhp
cy4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLWVwdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0tZXB0
LmMKQEAgLTY4MSw3ICs2ODEsNyBAQCBlcHRfc2V0X2VudHJ5KHN0cnVjdCBw
Mm1fZG9tYWluICpwMm0sIGdmCiAgICAgdWludDhfdCBpcGF0ID0gMDsKICAg
ICBib29sX3QgbmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMTsKICAgICBib29s
X3QgdnRkX3B0ZV9wcmVzZW50ID0gMDsKLSAgICB1bnNpZ25lZCBpbnQgaW9t
bXVfZmxhZ3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIG1mbik7Cisg
ICAgdW5zaWduZWQgaW50IGlvbW11X2ZsYWdzID0gcDJtX2dldF9pb21tdV9m
bGFncyhwMm10LCBwMm1hLCBtZm4pOwogICAgIGJvb2xfdCBuZWVkc19zeW5j
ID0gMTsKICAgICBlcHRfZW50cnlfdCBvbGRfZW50cnkgPSB7IC5lcHRlID0g
MCB9OwogICAgIGVwdF9lbnRyeV90IG5ld19lbnRyeSA9IHsgLmVwdGUgPSAw
IH07CkBAIC04MDksOCArODA5LDggQEAgZXB0X3NldF9lbnRyeShzdHJ1Y3Qg
cDJtX2RvbWFpbiAqcDJtLCBnZgogCiAgICAgICAgIC8qIFNhZmUgdG8gcmVh
ZC10aGVuLXdyaXRlIGJlY2F1c2Ugd2UgaG9sZCB0aGUgcDJtIGxvY2sgKi8K
ICAgICAgICAgaWYgKCBlcHRfZW50cnktPm1mbiA9PSBuZXdfZW50cnkubWZu
ICYmCi0gICAgICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhlcHRfZW50
cnktPnNhX3AybXQsIF9tZm4oZXB0X2VudHJ5LT5tZm4pKSA9PQotICAgICAg
ICAgICAgIGlvbW11X2ZsYWdzICkKKyAgICAgICAgICAgICBwMm1fZ2V0X2lv
bW11X2ZsYWdzKGVwdF9lbnRyeS0+c2FfcDJtdCwgZXB0X2VudHJ5LT5hY2Nl
c3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBfbWZuKGVw
dF9lbnRyeS0+bWZuKSkgPT0gaW9tbXVfZmxhZ3MgKQogICAgICAgICAgICAg
bmVlZF9tb2RpZnlfdnRkX3RhYmxlID0gMDsKIAogICAgICAgICBlcHRfcDJt
X3R5cGVfdG9fZmxhZ3MocDJtLCAmbmV3X2VudHJ5KTsKLS0tIGEveGVuL2Fy
Y2gveDg2L21tL3AybS1wdC5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0t
cHQuYwpAQCAtNTQ1LDYgKzU0NSwxNiBAQCBpbnQgcDJtX3B0X2hhbmRsZV9k
ZWZlcnJlZF9jaGFuZ2VzKHVpbnQ2CiAgICAgcmV0dXJuIHJjOwogfQogCisv
KiBSZWNvbnN0cnVjdCBhIGZha2UgcDJtX2FjY2Vzc190IGZyb20gc3RvcmVk
IFBURSBmbGFncy4gKi8KK3N0YXRpYyBwMm1fYWNjZXNzX3QgcDJtX2ZsYWdz
X3RvX2FjY2Vzcyh1bnNpZ25lZCBpbnQgZmxhZ3MpCit7CisgICAgaWYgKCBm
bGFncyAmIF9QQUdFX1BSRVNFTlQgKQorICAgICAgICByZXR1cm4gcDJtX2Fj
Y2Vzc19uOworCisgICAgLyogTm8gbmVlZCB0byBsb29rIGF0IF9QQUdFX05Y
IGZvciBub3cuICovCisgICAgcmV0dXJuIGZsYWdzICYgX1BBR0VfUlcgPyBw
Mm1fYWNjZXNzX3J3IDogcDJtX2FjY2Vzc19yOworfQorCiAvKiBDaGVja3Mg
b25seSBhcHBsaWNhYmxlIHRvIGVudHJpZXMgd2l0aCBvcmRlciA+IFBBR0Vf
T1JERVJfNEsgKi8KIHN0YXRpYyB2b2lkIGNoZWNrX2VudHJ5KG1mbl90IG1m
biwgcDJtX3R5cGVfdCBuZXcsIHAybV90eXBlX3Qgb2xkLAogICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVyKQpAQCAtNTc5LDcg
KzU4OSw3IEBAIHAybV9wdF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4g
KnAybSwKICAgICBsMl9wZ2VudHJ5X3QgbDJlX2NvbnRlbnQ7CiAgICAgbDNf
cGdlbnRyeV90IGwzZV9jb250ZW50OwogICAgIGludCByYzsKLSAgICB1bnNp
Z25lZCBpbnQgaW9tbXVfcHRlX2ZsYWdzID0gcDJtX2dldF9pb21tdV9mbGFn
cyhwMm10LCBtZm4pOworICAgIHVuc2lnbmVkIGludCBpb21tdV9wdGVfZmxh
Z3MgPSBwMm1fZ2V0X2lvbW11X2ZsYWdzKHAybXQsIHAybWEsIG1mbik7CiAg
ICAgLyoKICAgICAgKiBvbGRfbWZuIGFuZCBpb21tdV9vbGRfZmxhZ3MgY29u
dHJvbCBwb3NzaWJsZSBmbHVzaC91cGRhdGUgbmVlZHMgb24gdGhlCiAgICAg
ICogSU9NTVU6IFdlIG5lZWQgdG8gZmx1c2ggd2hlbiBNRk4gb3IgZmxhZ3Mg
KGkuZS4gcGVybWlzc2lvbnMpIGNoYW5nZS4KQEAgLTY0Miw2ICs2NTIsNyBA
QCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAg
ICAgICAgICAgICAgICAgb2xkX21mbiA9IGwxZV9nZXRfcGZuKCpwMm1fZW50
cnkpOwogICAgICAgICAgICAgICAgIGlvbW11X29sZF9mbGFncyA9CiAgICAg
ICAgICAgICAgICAgICAgIHAybV9nZXRfaW9tbXVfZmxhZ3MocDJtX2ZsYWdz
X3RvX3R5cGUoZmxhZ3MpLAorICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHAybV9mbGFnc190b19hY2Nlc3MoZmxhZ3MpLAogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF9tZm4ob2xk
X21mbikpOwogICAgICAgICAgICAgfQogICAgICAgICAgICAgZWxzZQpAQCAt
Njg0LDkgKzY5NSwxMCBAQCBwMm1fcHRfc2V0X2VudHJ5KHN0cnVjdCBwMm1f
ZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIDAsIEwxX1BBR0VUQUJMRV9FTlRSSUVTKTsKICAgICAgICAgQVNTRVJU
KHAybV9lbnRyeSk7CiAgICAgICAgIG9sZF9tZm4gPSBsMWVfZ2V0X3Bmbigq
cDJtX2VudHJ5KTsKKyAgICAgICAgZmxhZ3MgPSBsMWVfZ2V0X2ZsYWdzKCpw
Mm1fZW50cnkpOwogICAgICAgICBpb21tdV9vbGRfZmxhZ3MgPQotICAgICAg
ICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9fdHlwZShs
MWVfZ2V0X2ZsYWdzKCpwMm1fZW50cnkpKSwKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7CisgICAgICAgICAgICBw
Mm1fZ2V0X2lvbW11X2ZsYWdzKHAybV9mbGFnc190b190eXBlKGZsYWdzKSwK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2ZsYWdzX3Rv
X2FjY2VzcyhmbGFncyksIF9tZm4ob2xkX21mbikpOwogCiAgICAgICAgIGlm
ICggbWZuX3ZhbGlkKG1mbikgfHwgcDJtX2FsbG93c19pbnZhbGlkX21mbihw
Mm10KSApCiAgICAgICAgICAgICBlbnRyeV9jb250ZW50ID0gcDJtX2wxZV9m
cm9tX3BmbihtZm5feChtZm4pLApAQCAtNzE0LDYgKzcyNiw3IEBAIHAybV9w
dF9zZXRfZW50cnkoc3RydWN0IHAybV9kb21haW4gKnAybSwKICAgICAgICAg
ICAgICAgICBvbGRfbWZuID0gbDFlX2dldF9wZm4oKnAybV9lbnRyeSk7CiAg
ICAgICAgICAgICAgICAgaW9tbXVfb2xkX2ZsYWdzID0KICAgICAgICAgICAg
ICAgICAgICAgcDJtX2dldF9pb21tdV9mbGFncyhwMm1fZmxhZ3NfdG9fdHlw
ZShmbGFncyksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcDJtX2ZsYWdzX3RvX2FjY2VzcyhmbGFncyksCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgX21mbihvbGRfbWZuKSk7
CiAgICAgICAgICAgICB9CiAgICAgICAgICAgICBlbHNlCi0tLSBhL3hlbi9p
bmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4
Ni9wMm0uaApAQCAtOTE1LDcgKzkxNSw4IEBAIHN0YXRpYyBpbmxpbmUgdm9p
ZCBwMm1fYWx0cDJtX2NoZWNrKHN0cnUKIC8qCiAgKiBwMm0gdHlwZSB0byBJ
T01NVSBmbGFncwogICovCi1zdGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBw
Mm1fZ2V0X2lvbW11X2ZsYWdzKHAybV90eXBlX3QgcDJtdCwgbWZuX3QgbWZu
KQorc3RhdGljIGlubGluZSB1bnNpZ25lZCBpbnQgcDJtX2dldF9pb21tdV9m
bGFncyhwMm1fdHlwZV90IHAybXQsCisgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHAybV9hY2Nlc3NfdCBwMm1hLCBt
Zm5fdCBtZm4pCiB7CiAgICAgdW5zaWduZWQgaW50IGZsYWdzOwogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+CgotLS0gYS94ZW4vYXJjaC94
ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2L21tL3AybS5jCkBAIC0x
MzY1LDcgKzEzNjUsNyBAQCBpbnQgc2V0X2lkZW50aXR5X3AybV9lbnRyeShz
dHJ1Y3QgZG9tYWluCiAgICAgICAgICAgICByZXR1cm4gMDsKICAgICAgICAg
cmV0dXJuIGlvbW11X2xlZ2FjeV9tYXAoZCwgX2RmbihnZm5fbCksIF9tZm4o
Z2ZuX2wpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAxdWwg
PDwgUEFHRV9PUkRFUl80SywKLSAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgSU9NTVVGX3JlYWRhYmxlIHwgSU9NTVVGX3dyaXRhYmxlKTsKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcDJtX2FjY2Vzc190b19p
b21tdV9mbGFncyhwMm1hKSk7CiAgICAgfQogCiAgICAgZ2ZuX2xvY2socDJt
LCBnZm4sIDApOwotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQv
aW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9t
bXUuYwpAQCAtNDIsMTIgKzQyLDYgQEAKICNpbmNsdWRlICJ2dGQuaCIKICNp
bmNsdWRlICIuLi9hdHMuaCIKIAotc3RydWN0IG1hcHBlZF9ybXJyIHsKLSAg
ICBzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7Ci0gICAgdTY0IGJhc2UsIGVuZDsK
LSAgICB1bnNpZ25lZCBpbnQgY291bnQ7Ci19OwotCiAvKiBQb3NzaWJsZSB1
bmZpbHRlcmVkIExBUElDL01TSSBtZXNzYWdlcyBmcm9tIHVudHJ1c3RlZCBz
b3VyY2VzPyAqLwogYm9vbCBfX3JlYWRfbW9zdGx5IHVudHJ1c3RlZF9tc2k7
CiAKQEAgLTEzMTEsNyArMTMwNSw2IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9t
bXVfZG9tYWluX2luaXQoc3RydWMKICAgICBzdHJ1Y3QgZG9tYWluX2lvbW11
ICpoZCA9IGRvbV9pb21tdShkKTsKIAogICAgIGhkLT5hcmNoLnZ0ZC5hZ2F3
ID0gd2lkdGhfdG9fYWdhdyhERUZBVUxUX0RPTUFJTl9BRERSRVNTX1dJRFRI
KTsKLSAgICBJTklUX0xJU1RfSEVBRCgmaGQtPmFyY2gudnRkLm1hcHBlZF9y
bXJycyk7CiAKICAgICByZXR1cm4gMDsKIH0KQEAgLTE3ODgsMTcgKzE3ODEs
MTIgQEAgc3RhdGljIHZvaWQgaW9tbXVfY2xlYXJfcm9vdF9wZ3RhYmxlKHN0
cgogc3RhdGljIHZvaWQgaW9tbXVfZG9tYWluX3RlYXJkb3duKHN0cnVjdCBk
b21haW4gKmQpCiB7CiAgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBk
b21faW9tbXUoZCk7Ci0gICAgc3RydWN0IG1hcHBlZF9ybXJyICptcm1yciwg
KnRtcDsKICAgICBjb25zdCBzdHJ1Y3QgYWNwaV9kcmhkX3VuaXQgKmRyaGQ7
CiAKICAgICBpZiAoIGxpc3RfZW1wdHkoJmFjcGlfZHJoZF91bml0cykgKQog
ICAgICAgICByZXR1cm47CiAKLSAgICBsaXN0X2Zvcl9lYWNoX2VudHJ5X3Nh
ZmUgKCBtcm1yciwgdG1wLCAmaGQtPmFyY2gudnRkLm1hcHBlZF9ybXJycywg
bGlzdCApCi0gICAgewotICAgICAgICBsaXN0X2RlbCgmbXJtcnItPmxpc3Qp
OwotICAgICAgICB4ZnJlZShtcm1ycik7Ci0gICAgfQorICAgIGlvbW11X2lk
ZW50aXR5X21hcF90ZWFyZG93bihkKTsKIAogICAgIEFTU0VSVCghaGQtPmFy
Y2gudnRkLnBnZF9tYWRkcik7CiAKQEAgLTE5NDYsNzQgKzE5MzQsNiBAQCBz
dGF0aWMgaW50IF9faW5pdCB2dGRfZXB0X3BhZ2VfY29tcGF0aWJsCiAgICAg
ICAgICAgIChlcHRfaGFzXzFnYihlcHRfY2FwKSAmJiBvcHRfaGFwXzFnYikg
PD0gY2FwX3Nwc18xZ2IodnRkX2NhcCk7CiB9CiAKLXN0YXRpYyBpbnQgcm1y
cl9pZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsIGJvb2xfdCBt
YXAsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnIsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1MzIgZmxhZykKLXsKLSAgICB1bnNpZ25lZCBs
b25nIGJhc2VfcGZuID0gcm1yci0+YmFzZV9hZGRyZXNzID4+IFBBR0VfU0hJ
RlRfNEs7Ci0gICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuID0gUEFHRV9BTElH
Tl80SyhybXJyLT5lbmRfYWRkcmVzcykgPj4gUEFHRV9TSElGVF80SzsKLSAg
ICBzdHJ1Y3QgbWFwcGVkX3JtcnIgKm1ybXJyOwotICAgIHN0cnVjdCBkb21h
aW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOwotCi0gICAgQVNTRVJUKHBj
aWRldnNfbG9ja2VkKCkpOwotICAgIEFTU0VSVChybXJyLT5iYXNlX2FkZHJl
c3MgPCBybXJyLT5lbmRfYWRkcmVzcyk7Ci0KLSAgICAvKgotICAgICAqIE5v
IG5lZWQgdG8gYWNxdWlyZSBoZC0+YXJjaC5tYXBwaW5nX2xvY2s6IEJvdGgg
aW5zZXJ0aW9uIGFuZCByZW1vdmFsCi0gICAgICogZ2V0IGRvbmUgd2hpbGUg
aG9sZGluZyBwY2lkZXZzX2xvY2suCi0gICAgICovCi0gICAgbGlzdF9mb3Jf
ZWFjaF9lbnRyeSggbXJtcnIsICZoZC0+YXJjaC52dGQubWFwcGVkX3JtcnJz
LCBsaXN0ICkKLSAgICB7Ci0gICAgICAgIGlmICggbXJtcnItPmJhc2UgPT0g
cm1yci0+YmFzZV9hZGRyZXNzICYmCi0gICAgICAgICAgICAgbXJtcnItPmVu
ZCA9PSBybXJyLT5lbmRfYWRkcmVzcyApCi0gICAgICAgIHsKLSAgICAgICAg
ICAgIGludCByZXQgPSAwOwotCi0gICAgICAgICAgICBpZiAoIG1hcCApCi0g
ICAgICAgICAgICB7Ci0gICAgICAgICAgICAgICAgKyttcm1yci0+Y291bnQ7
Ci0gICAgICAgICAgICAgICAgcmV0dXJuIDA7Ci0gICAgICAgICAgICB9Ci0K
LSAgICAgICAgICAgIGlmICggLS1tcm1yci0+Y291bnQgKQotICAgICAgICAg
ICAgICAgIHJldHVybiAwOwotCi0gICAgICAgICAgICB3aGlsZSAoIGJhc2Vf
cGZuIDwgZW5kX3BmbiApCi0gICAgICAgICAgICB7Ci0gICAgICAgICAgICAg
ICAgaWYgKCBjbGVhcl9pZGVudGl0eV9wMm1fZW50cnkoZCwgYmFzZV9wZm4p
ICkKLSAgICAgICAgICAgICAgICAgICAgcmV0ID0gLUVOWElPOwotICAgICAg
ICAgICAgICAgIGJhc2VfcGZuKys7Ci0gICAgICAgICAgICB9Ci0KLSAgICAg
ICAgICAgIGxpc3RfZGVsKCZtcm1yci0+bGlzdCk7Ci0gICAgICAgICAgICB4
ZnJlZShtcm1ycik7Ci0gICAgICAgICAgICByZXR1cm4gcmV0OwotICAgICAg
ICB9Ci0gICAgfQotCi0gICAgaWYgKCAhbWFwICkKLSAgICAgICAgcmV0dXJu
IC1FTk9FTlQ7Ci0KLSAgICB3aGlsZSAoIGJhc2VfcGZuIDwgZW5kX3BmbiAp
Ci0gICAgewotICAgICAgICBpbnQgZXJyID0gc2V0X2lkZW50aXR5X3AybV9l
bnRyeShkLCBiYXNlX3BmbiwgcDJtX2FjY2Vzc19ydywgZmxhZyk7Ci0KLSAg
ICAgICAgaWYgKCBlcnIgKQotICAgICAgICAgICAgcmV0dXJuIGVycjsKLSAg
ICAgICAgYmFzZV9wZm4rKzsKLSAgICB9Ci0KLSAgICBtcm1yciA9IHhtYWxs
b2Moc3RydWN0IG1hcHBlZF9ybXJyKTsKLSAgICBpZiAoICFtcm1yciApCi0g
ICAgICAgIHJldHVybiAtRU5PTUVNOwotICAgIG1ybXJyLT5iYXNlID0gcm1y
ci0+YmFzZV9hZGRyZXNzOwotICAgIG1ybXJyLT5lbmQgPSBybXJyLT5lbmRf
YWRkcmVzczsKLSAgICBtcm1yci0+Y291bnQgPSAxOwotICAgIGxpc3RfYWRk
X3RhaWwoJm1ybXJyLT5saXN0LCAmaGQtPmFyY2gudnRkLm1hcHBlZF9ybXJy
cyk7Ci0KLSAgICByZXR1cm4gMDsKLX0KLQogc3RhdGljIGludCBpbnRlbF9p
b21tdV9hZGRfZGV2aWNlKHU4IGRldmZuLCBzdHJ1Y3QgcGNpX2RldiAqcGRl
dikKIHsKICAgICBzdHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnI7CkBAIC0y
MDQ1LDcgKzE5NjUsOSBAQCBzdGF0aWMgaW50IGludGVsX2lvbW11X2FkZF9k
ZXZpY2UodTggZGV2CiAgICAgICAgICAgICAgKiBTaW5jZSBSTVJScyBhcmUg
YWx3YXlzIHJlc2VydmVkIGluIHRoZSBlODIwIG1hcCBmb3IgdGhlIGhhcmR3
YXJlCiAgICAgICAgICAgICAgKiBkb21haW4sIHRoZXJlIHNob3VsZG4ndCBi
ZSBhIGNvbmZsaWN0LgogICAgICAgICAgICAgICovCi0gICAgICAgICAgICBy
ZXQgPSBybXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWluLCAxLCBy
bXJyLCAwKTsKKyAgICAgICAgICAgIHJldCA9IGlvbW11X2lkZW50aXR5X21h
cHBpbmcocGRldi0+ZG9tYWluLCBwMm1fYWNjZXNzX3J3LAorICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5iYXNlX2Fk
ZHJlc3MsIHJtcnItPmVuZF9hZGRyZXNzLAorICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAwKTsKICAgICAgICAgICAgIGlmICgg
cmV0ICkKICAgICAgICAgICAgICAgICBkcHJpbnRrKFhFTkxPR19FUlIgVlRE
UFJFRklYLCAiZCVkOiBSTVJSIG1hcHBpbmcgZmFpbGVkXG4iLAogICAgICAg
ICAgICAgICAgICAgICAgICAgcGRldi0+ZG9tYWluLT5kb21haW5faWQpOwpA
QCAtMjA5MCw3ICsyMDEyLDggQEAgc3RhdGljIGludCBpbnRlbF9pb21tdV9y
ZW1vdmVfZGV2aWNlKHU4CiAgICAgICAgICAqIEFueSBmbGFnIGlzIG5vdGhp
bmcgdG8gY2xlYXIgdGhlc2UgbWFwcGluZ3MgYnV0IGhlcmUKICAgICAgICAg
ICogaXRzIGFsd2F5cyBzYWZlIGFuZCBzdHJpY3QgdG8gc2V0IDAuCiAgICAg
ICAgICAqLwotICAgICAgICBybXJyX2lkZW50aXR5X21hcHBpbmcocGRldi0+
ZG9tYWluLCAwLCBybXJyLCAwKTsKKyAgICAgICAgaW9tbXVfaWRlbnRpdHlf
bWFwcGluZyhwZGV2LT5kb21haW4sIHAybV9hY2Nlc3NfeCwgcm1yci0+YmFz
ZV9hZGRyZXNzLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJt
cnItPmVuZF9hZGRyZXNzLCAwKTsKICAgICB9CiAKICAgICByZXR1cm4gZG9t
YWluX2NvbnRleHRfdW5tYXAocGRldi0+ZG9tYWluLCBkZXZmbiwgcGRldik7
CkBAIC0yMjg5LDcgKzIyMTIsOCBAQCBzdGF0aWMgdm9pZCBfX2h3ZG9tX2lu
aXQgc2V0dXBfaHdkb21fcm1yCiAgICAgICAgICAqIGRvbWFpbiwgdGhlcmUg
c2hvdWxkbid0IGJlIGEgY29uZmxpY3QuIFNvIGl0cyBhbHdheXMgc2FmZSBh
bmQKICAgICAgICAgICogc3RyaWN0IHRvIHNldCAwLgogICAgICAgICAgKi8K
LSAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKGQsIDEsIHJt
cnIsIDApOworICAgICAgICByZXQgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5n
KGQsIHAybV9hY2Nlc3NfcncsIHJtcnItPmJhc2VfYWRkcmVzcywKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBybXJyLT5lbmRfYWRk
cmVzcywgMCk7CiAgICAgICAgIGlmICggcmV0ICkKICAgICAgICAgICAgIGRw
cmludGsoWEVOTE9HX0VSUiBWVERQUkVGSVgsCiAgICAgICAgICAgICAgICAg
ICAgICAiSU9NTVU6IG1hcHBpbmcgcmVzZXJ2ZWQgcmVnaW9uIGZhaWxlZFxu
Iik7CkBAIC0yNDYwLDcgKzIzODQsOSBAQCBzdGF0aWMgaW50IHJlYXNzaWdu
X2RldmljZV9vd25lcnNoaXAoCiAgICAgICAgICAgICAgICAgICogQW55IFJN
UlIgZmxhZyBpcyBhbHdheXMgaWdub3JlZCB3aGVuIHJlbW92ZSBhIGRldmlj
ZSwKICAgICAgICAgICAgICAgICAgKiBidXQgaXRzIGFsd2F5cyBzYWZlIGFu
ZCBzdHJpY3QgdG8gc2V0IDAuCiAgICAgICAgICAgICAgICAgICovCi0gICAg
ICAgICAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5nKHNvdXJj
ZSwgMCwgcm1yciwgMCk7CisgICAgICAgICAgICAgICAgcmV0ID0gaW9tbXVf
aWRlbnRpdHlfbWFwcGluZyhzb3VyY2UsIHAybV9hY2Nlc3NfeCwKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnIt
PmJhc2VfYWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCAwKTsKICAgICAg
ICAgICAgICAgICBpZiAoIHJldCAhPSAtRU5PRU5UICkKICAgICAgICAgICAg
ICAgICAgICAgcmV0dXJuIHJldDsKICAgICAgICAgICAgIH0KQEAgLTI1NTYs
NyArMjQ4Miw4IEBAIHN0YXRpYyBpbnQgaW50ZWxfaW9tbXVfYXNzaWduX2Rl
dmljZSgKICAgICAgICAgICAgICBQQ0lfQlVTKGJkZikgPT0gYnVzICYmCiAg
ICAgICAgICAgICAgUENJX0RFVkZOMihiZGYpID09IGRldmZuICkKICAgICAg
ICAgewotICAgICAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0eV9tYXBwaW5n
KGQsIDEsIHJtcnIsIGZsYWcpOworICAgICAgICAgICAgcmV0ID0gaW9tbXVf
aWRlbnRpdHlfbWFwcGluZyhkLCBwMm1fYWNjZXNzX3J3LCBybXJyLT5iYXNl
X2FkZHJlc3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCBmbGFnKTsKICAgICAgICAgICAg
IGlmICggcmV0ICkKICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICBp
bnQgcmM7Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21t
dS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3g4Ni9pb21tdS5j
CkBAIC0xNDMsNiArMTQzLDcgQEAgaW50IGFyY2hfaW9tbXVfZG9tYWluX2lu
aXQoc3RydWN0IGRvbWFpbgogCiAgICAgSU5JVF9QQUdFX0xJU1RfSEVBRCgm
aGQtPmFyY2gucGd0YWJsZXMubGlzdCk7CiAgICAgc3Bpbl9sb2NrX2luaXQo
JmhkLT5hcmNoLnBndGFibGVzLmxvY2spOworICAgIElOSVRfTElTVF9IRUFE
KCZoZC0+YXJjaC5pZGVudGl0eV9tYXBzKTsKIAogICAgIHJldHVybiAwOwog
fQpAQCAtMTU4LDYgKzE1OSw5OSBAQCB2b2lkIGFyY2hfaW9tbXVfZG9tYWlu
X2Rlc3Ryb3koc3RydWN0IGRvCiAgICAgICAgICAgIHBhZ2VfbGlzdF9lbXB0
eSgmZG9tX2lvbW11KGQpLT5hcmNoLnBndGFibGVzLmxpc3QpKTsKIH0KIAor
c3RydWN0IGlkZW50aXR5X21hcCB7CisgICAgc3RydWN0IGxpc3RfaGVhZCBs
aXN0OworICAgIHBhZGRyX3QgYmFzZSwgZW5kOworICAgIHAybV9hY2Nlc3Nf
dCBhY2Nlc3M7CisgICAgdW5zaWduZWQgaW50IGNvdW50OworfTsKKworaW50
IGlvbW11X2lkZW50aXR5X21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwgcDJt
X2FjY2Vzc190IHAybWEsCisgICAgICAgICAgICAgICAgICAgICAgICAgICBw
YWRkcl90IGJhc2UsIHBhZGRyX3QgZW5kLAorICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50IGZsYWcpCit7CisgICAgdW5zaWduZWQg
bG9uZyBiYXNlX3BmbiA9IGJhc2UgPj4gUEFHRV9TSElGVF80SzsKKyAgICB1
bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdFX0FMSUdOXzRLKGVuZCkgPj4g
UEFHRV9TSElGVF80SzsKKyAgICBzdHJ1Y3QgaWRlbnRpdHlfbWFwICptYXA7
CisgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7
CisKKyAgICBBU1NFUlQocGNpZGV2c19sb2NrZWQoKSk7CisgICAgQVNTRVJU
KGJhc2UgPCBlbmQpOworCisgICAgLyoKKyAgICAgKiBObyBuZWVkIHRvIGFj
cXVpcmUgaGQtPmFyY2gubWFwcGluZ19sb2NrOiBCb3RoIGluc2VydGlvbiBh
bmQgcmVtb3ZhbAorICAgICAqIGdldCBkb25lIHdoaWxlIGhvbGRpbmcgcGNp
ZGV2c19sb2NrLgorICAgICAqLworICAgIGxpc3RfZm9yX2VhY2hfZW50cnko
IG1hcCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMsIGxpc3QgKQorICAgIHsK
KyAgICAgICAgaWYgKCBtYXAtPmJhc2UgPT0gYmFzZSAmJiBtYXAtPmVuZCA9
PSBlbmQgKQorICAgICAgICB7CisgICAgICAgICAgICBpbnQgcmV0ID0gMDsK
KworICAgICAgICAgICAgaWYgKCBwMm1hICE9IHAybV9hY2Nlc3NfeCApCisg
ICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgaWYgKCBtYXAtPmFjY2Vz
cyAhPSBwMm1hICkKKyAgICAgICAgICAgICAgICAgICAgcmV0dXJuIC1FQURE
UklOVVNFOworICAgICAgICAgICAgICAgICsrbWFwLT5jb3VudDsKKyAgICAg
ICAgICAgICAgICByZXR1cm4gMDsKKyAgICAgICAgICAgIH0KKworICAgICAg
ICAgICAgaWYgKCAtLW1hcC0+Y291bnQgKQorICAgICAgICAgICAgICAgIHJl
dHVybiAwOworCisgICAgICAgICAgICB3aGlsZSAoIGJhc2VfcGZuIDwgZW5k
X3BmbiApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgaWYgKCBj
bGVhcl9pZGVudGl0eV9wMm1fZW50cnkoZCwgYmFzZV9wZm4pICkKKyAgICAg
ICAgICAgICAgICAgICAgcmV0ID0gLUVOWElPOworICAgICAgICAgICAgICAg
IGJhc2VfcGZuKys7CisgICAgICAgICAgICB9CisKKyAgICAgICAgICAgIGxp
c3RfZGVsKCZtYXAtPmxpc3QpOworICAgICAgICAgICAgeGZyZWUobWFwKTsK
KworICAgICAgICAgICAgcmV0dXJuIHJldDsKKyAgICAgICAgfQorCisgICAg
ICAgIGlmICggZW5kID49IG1hcC0+YmFzZSAmJiBtYXAtPmVuZCA+PSBiYXNl
ICkKKyAgICAgICAgICAgIHJldHVybiAtRUFERFJJTlVTRTsKKyAgICB9CisK
KyAgICBpZiAoIHAybWEgPT0gcDJtX2FjY2Vzc194ICkKKyAgICAgICAgcmV0
dXJuIC1FTk9FTlQ7CisKKyAgICB3aGlsZSAoIGJhc2VfcGZuIDwgZW5kX3Bm
biApCisgICAgeworICAgICAgICBpbnQgZXJyID0gc2V0X2lkZW50aXR5X3Ay
bV9lbnRyeShkLCBiYXNlX3BmbiwgcDJtYSwgZmxhZyk7CisKKyAgICAgICAg
aWYgKCBlcnIgKQorICAgICAgICAgICAgcmV0dXJuIGVycjsKKyAgICAgICAg
YmFzZV9wZm4rKzsKKyAgICB9CisKKyAgICBtYXAgPSB4bWFsbG9jKHN0cnVj
dCBpZGVudGl0eV9tYXApOworICAgIGlmICggIW1hcCApCisgICAgICAgIHJl
dHVybiAtRU5PTUVNOworICAgIG1hcC0+YmFzZSA9IGJhc2U7CisgICAgbWFw
LT5lbmQgPSBlbmQ7CisgICAgbWFwLT5hY2Nlc3MgPSBwMm1hOworICAgIG1h
cC0+Y291bnQgPSAxOworICAgIGxpc3RfYWRkX3RhaWwoJm1hcC0+bGlzdCwg
JmhkLT5hcmNoLmlkZW50aXR5X21hcHMpOworCisgICAgcmV0dXJuIDA7Cit9
CisKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKHN0cnVjdCBk
b21haW4gKmQpCit7CisgICAgc3RydWN0IGRvbWFpbl9pb21tdSAqaGQgPSBk
b21faW9tbXUoZCk7CisgICAgc3RydWN0IGlkZW50aXR5X21hcCAqbWFwLCAq
dG1wOworCisgICAgbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlICggbWFwLCB0
bXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBzLCBsaXN0ICkKKyAgICB7Cisg
ICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOworICAgICAgICB4ZnJlZSht
YXApOworICAgIH0KK30KKwogc3RhdGljIGJvb2wgX19od2RvbV9pbml0IGh3
ZG9tX2lvbW11X21hcChjb25zdCBzdHJ1Y3QgZG9tYWluICpkLAogICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBs
b25nIHBmbiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdW5zaWduZWQgbG9uZyBtYXhfcGZuKQotLS0gYS94ZW4vaW5jbHVk
ZS9hc20teDg2L2lvbW11LmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4Ni9p
b21tdS5oCkBAIC0xNiw2ICsxNiw3IEBACiAKICNpbmNsdWRlIDx4ZW4vZXJy
bm8uaD4KICNpbmNsdWRlIDx4ZW4vbGlzdC5oPgorI2luY2x1ZGUgPHhlbi9t
ZW1fYWNjZXNzLmg+CiAjaW5jbHVkZSA8eGVuL3NwaW5sb2NrLmg+CiAjaW5j
bHVkZSA8YXNtL2FwaWNkZWYuaD4KICNpbmNsdWRlIDxhc20vcHJvY2Vzc29y
Lmg+CkBAIC01MSwxMyArNTIsMTQgQEAgc3RydWN0IGFyY2hfaW9tbXUKICAg
ICAgICAgc3BpbmxvY2tfdCBsb2NrOwogICAgIH0gcGd0YWJsZXM7CiAKKyAg
ICBzdHJ1Y3QgbGlzdF9oZWFkIGlkZW50aXR5X21hcHM7CisKICAgICB1bmlv
biB7CiAgICAgICAgIC8qIEludGVsIFZULWQgKi8KICAgICAgICAgc3RydWN0
IHsKICAgICAgICAgICAgIHVpbnQ2NF90IHBnZF9tYWRkcjsgLyogaW8gcGFn
ZSBkaXJlY3RvcnkgbWFjaGluZSBhZGRyZXNzICovCiAgICAgICAgICAgICB1
bnNpZ25lZCBpbnQgYWdhdzsgLyogYWRqdXN0ZWQgZ3Vlc3QgYWRkcmVzcyB3
aWR0aCwgMCBpcyBsZXZlbCAyIDMwLWJpdCAqLwogICAgICAgICAgICAgdWlu
dDY0X3QgaW9tbXVfYml0bWFwOyAvKiBiaXRtYXAgb2YgaW9tbXUocykgdGhh
dCB0aGUgZG9tYWluIHVzZXMgKi8KLSAgICAgICAgICAgIHN0cnVjdCBsaXN0
X2hlYWQgbWFwcGVkX3JtcnJzOwogICAgICAgICB9IHZ0ZDsKICAgICAgICAg
LyogQU1EIElPTU1VICovCiAgICAgICAgIHN0cnVjdCB7CkBAIC0xMjMsNiAr
MTI1LDExIEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBpb21tdV9kaXNhYmxlX3gy
YXBpYygKICAgICAgICAgaW9tbXVfb3BzLmRpc2FibGVfeDJhcGljKCk7CiB9
CiAKK2ludCBpb21tdV9pZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBkb21haW4g
KmQsIHAybV9hY2Nlc3NfdCBwMm1hLAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGVuZCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFnKTsKK3ZvaWQgaW9t
bXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpOwor
CiBleHRlcm4gYm9vbCB1bnRydXN0ZWRfbXNpOwogCiBpbnQgcGlfdXBkYXRl
X2lydGUoY29uc3Qgc3RydWN0IHBpX2Rlc2MgKnBpX2Rlc2MsIGNvbnN0IHN0
cnVjdCBwaXJxICpwaXJxLAotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L3Ay
bS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKQEAgLTkxMiw2
ICs5MTIsMzQgQEAgc3RydWN0IHAybV9kb21haW4gKnAybV9nZXRfYWx0cDJt
KHN0cnVjdAogc3RhdGljIGlubGluZSB2b2lkIHAybV9hbHRwMm1fY2hlY2so
c3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlkeCkge30KICNlbmRpZgogCisv
KiBwMm0gYWNjZXNzIHRvIElPTU1VIGZsYWdzICovCitzdGF0aWMgaW5saW5l
IHVuc2lnbmVkIGludCBwMm1fYWNjZXNzX3RvX2lvbW11X2ZsYWdzKHAybV9h
Y2Nlc3NfdCBwMm1hKQoreworICAgIHN3aXRjaCAoIHAybWEgKQorICAgIHsK
KyAgICBjYXNlIHAybV9hY2Nlc3Nfcnc6CisgICAgY2FzZSBwMm1fYWNjZXNz
X3J3eDoKKyAgICAgICAgcmV0dXJuIElPTU1VRl9yZWFkYWJsZSB8IElPTU1V
Rl93cml0YWJsZTsKKworICAgIGNhc2UgcDJtX2FjY2Vzc19yOgorICAgIGNh
c2UgcDJtX2FjY2Vzc19yeDoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcngycnc6
CisgICAgICAgIHJldHVybiBJT01NVUZfcmVhZGFibGU7CisKKyAgICBjYXNl
IHAybV9hY2Nlc3NfdzoKKyAgICBjYXNlIHAybV9hY2Nlc3Nfd3g6CisgICAg
ICAgIHJldHVybiBJT01NVUZfd3JpdGFibGU7CisKKyAgICBjYXNlIHAybV9h
Y2Nlc3NfbjoKKyAgICBjYXNlIHAybV9hY2Nlc3NfeDoKKyAgICBjYXNlIHAy
bV9hY2Nlc3NfbjJyd3g6CisgICAgICAgIHJldHVybiAwOworICAgIH0KKwor
ICAgIEFTU0VSVF9VTlJFQUNIQUJMRSgpOworICAgIHJldHVybiAwOworfQor
CiAvKgogICogcDJtIHR5cGUgdG8gSU9NTVUgZmxhZ3MKICAqLwpAQCAtOTMz
LDkgKzk2MSwxMCBAQCBzdGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBwMm1f
Z2V0X2lvbW11CiAgICAgICAgIGZsYWdzID0gSU9NTVVGX3JlYWRhYmxlOwog
ICAgICAgICBicmVhazsKICAgICBjYXNlIHAybV9tbWlvX2RpcmVjdDoKLSAg
ICAgICAgZmxhZ3MgPSBJT01NVUZfcmVhZGFibGU7Ci0gICAgICAgIGlmICgg
IXJhbmdlc2V0X2NvbnRhaW5zX3NpbmdsZXRvbihtbWlvX3JvX3Jhbmdlcywg
bWZuX3gobWZuKSkgKQotICAgICAgICAgICAgZmxhZ3MgfD0gSU9NTVVGX3dy
aXRhYmxlOworICAgICAgICBmbGFncyA9IHAybV9hY2Nlc3NfdG9faW9tbXVf
ZmxhZ3MocDJtYSk7CisgICAgICAgIGlmICggKGZsYWdzICYgSU9NTVVGX3dy
aXRhYmxlKSAmJgorICAgICAgICAgICAgIHJhbmdlc2V0X2NvbnRhaW5zX3Np
bmdsZXRvbihtbWlvX3JvX3JhbmdlcywgbWZuX3gobWZuKSkgKQorICAgICAg
ICAgICAgZmxhZ3MgJj0gfklPTU1VRl93cml0YWJsZTsKICAgICAgICAgYnJl
YWs7CiAgICAgZGVmYXVsdDoKICAgICAgICAgZmxhZ3MgPSAwOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+CgotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9t
bXUuaAorKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXUu
aApAQCAtMjMyLDggKzIzMiwxMCBAQCBpbnQgX19tdXN0X2NoZWNrIGFtZF9p
b21tdV91bm1hcF9wYWdlKHN0CiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHVuc2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpOwogaW50
IF9fbXVzdF9jaGVjayBhbWRfaW9tbXVfYWxsb2Nfcm9vdChzdHJ1Y3QgZG9t
YWluICpkKTsKIGludCBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlf
bWFwKHN0cnVjdCBkb21haW4gKmRvbWFpbiwKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHBhZGRyX3QgcGh5c19hZGRyLCB1bnNp
Z25lZCBsb25nIHNpemUsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBpbnQgaXcsIGludCBpcik7CisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaXZyc191bml0
eV9tYXAgKm1hcCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHVuc2lnbmVkIGludCBmbGFnKTsKK2ludCBhbWRfaW9tbXVfcmVz
ZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qg
c3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXApOwogaW50IF9fbXVzdF9jaGVj
ayBhbWRfaW9tbXVfZmx1c2hfaW90bGJfcGFnZXMoc3RydWN0IGRvbWFpbiAq
ZCwgZGZuX3QgZGZuLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBwYWdlX2NvdW50LAogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IGZsdXNoX2ZsYWdzKTsKLS0tIGEveGVuL2RyaXZlcnMvcGFz
c3Rocm91Z2gvYW1kL2lvbW11X21hcC5jCisrKyBiL3hlbi9kcml2ZXJzL3Bh
c3N0aHJvdWdoL2FtZC9pb21tdV9tYXAuYwpAQCAtNDE5LDM4ICs0MTksNDkg
QEAgaW50IGFtZF9pb21tdV9mbHVzaF9pb3RsYl9hbGwoc3RydWN0IGRvbQog
ICAgIHJldHVybiAwOwogfQogCi1pbnQgYW1kX2lvbW11X3Jlc2VydmVfZG9t
YWluX3VuaXR5X21hcChzdHJ1Y3QgZG9tYWluICpkb21haW4sCi0gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IHBoeXNf
YWRkciwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGxvbmcgc2l6ZSwgaW50IGl3LCBpbnQgaXIpCitpbnQgYW1k
X2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcChzdHJ1Y3QgZG9tYWlu
ICpkLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
Y29uc3Qgc3RydWN0IGl2cnNfdW5pdHlfbWFwICptYXAsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxh
ZykKIHsKLSAgICB1bnNpZ25lZCBsb25nIG5wYWdlcywgaTsKLSAgICB1bnNp
Z25lZCBsb25nIGdmbjsKLSAgICB1bnNpZ25lZCBpbnQgZmxhZ3MgPSAhIWly
OwotICAgIHVuc2lnbmVkIGludCBmbHVzaF9mbGFncyA9IDA7Ci0gICAgaW50
IHJ0ID0gMDsKLQotICAgIGlmICggaXcgKQotICAgICAgICBmbGFncyB8PSBJ
T01NVUZfd3JpdGFibGU7Ci0KLSAgICBucGFnZXMgPSByZWdpb25fdG9fcGFn
ZXMocGh5c19hZGRyLCBzaXplKTsKLSAgICBnZm4gPSBwaHlzX2FkZHIgPj4g
UEFHRV9TSElGVDsKLSAgICBmb3IgKCBpID0gMDsgaSA8IG5wYWdlczsgaSsr
ICkKKyAgICBpbnQgcmM7CisKKyAgICBpZiAoIGQgPT0gZG9tX2lvICkKKyAg
ICAgICAgcmV0dXJuIDA7CisKKyAgICBmb3IgKCByYyA9IDA7ICFyYyAmJiBt
YXA7IG1hcCA9IG1hcC0+bmV4dCApCiAgICAgewotICAgICAgICB1bnNpZ25l
ZCBsb25nIGZyYW1lID0gZ2ZuICsgaTsKKyAgICAgICAgcDJtX2FjY2Vzc190
IHAybWEgPSBwMm1fYWNjZXNzX247CisKKyAgICAgICAgaWYgKCBtYXAtPnJl
YWQgKQorICAgICAgICAgICAgcDJtYSB8PSBwMm1fYWNjZXNzX3I7CisgICAg
ICAgIGlmICggbWFwLT53cml0ZSApCisgICAgICAgICAgICBwMm1hIHw9IHAy
bV9hY2Nlc3NfdzsKIAotICAgICAgICBydCA9IGFtZF9pb21tdV9tYXBfcGFn
ZShkb21haW4sIF9kZm4oZnJhbWUpLCBfbWZuKGZyYW1lKSwgZmxhZ3MsCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICZmbHVzaF9mbGFncyk7
Ci0gICAgICAgIGlmICggcnQgIT0gMCApCi0gICAgICAgICAgICBicmVhazsK
KyAgICAgICAgcmMgPSBpb21tdV9pZGVudGl0eV9tYXBwaW5nKGQsIHAybWEs
IG1hcC0+YWRkciwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIG1hcC0+YWRkciArIG1hcC0+bGVuZ3RoIC0gMSwgZmxhZyk7CiAgICAg
fQogCi0gICAgLyogVXNlIHdoaWxlLWJyZWFrIHRvIGF2b2lkIGNvbXBpbGVy
IHdhcm5pbmcgKi8KLSAgICB3aGlsZSAoIGZsdXNoX2ZsYWdzICYmCi0gICAg
ICAgICAgICBhbWRfaW9tbXVfZmx1c2hfaW90bGJfcGFnZXMoZG9tYWluLCBf
ZGZuKGdmbiksCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgbnBhZ2VzLCBmbHVzaF9mbGFncykgKQotICAgICAgICBicmVhazsK
KyAgICByZXR1cm4gcmM7Cit9CisKK2ludCBhbWRfaW9tbXVfcmVzZXJ2ZV9k
b21haW5fdW5pdHlfdW5tYXAoc3RydWN0IGRvbWFpbiAqZCwKKyAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0
IGl2cnNfdW5pdHlfbWFwICptYXApCit7CisgICAgaW50IHJjOworCisgICAg
aWYgKCBkID09IGRvbV9pbyApCisgICAgICAgIHJldHVybiAwOworCisgICAg
Zm9yICggcmMgPSAwOyBtYXA7IG1hcCA9IG1hcC0+bmV4dCApCisgICAgewor
ICAgICAgICBpbnQgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhkLCBw
Mm1fYWNjZXNzX3gsIG1hcC0+YWRkciwKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgbWFwLT5hZGRyICsgbWFwLT5sZW5ndGgg
LSAxLCAwKTsKKworICAgICAgICBpZiAoIHJldCAmJiByZXQgIT0gLUVOT0VO
VCAmJiAhcmMgKQorICAgICAgICAgICAgcmMgPSByZXQ7CisgICAgfQogCi0g
ICAgcmV0dXJuIHJ0OworICAgIHJldHVybiByYzsKIH0KIAogaW50IF9faW5p
dCBhbWRfaW9tbXVfcXVhcmFudGluZV9pbml0KHN0cnVjdCBkb21haW4gKmQp
Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9wY2lfYW1kX2lv
bW11LmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9h
bWRfaW9tbXUuYwpAQCAtMzI5LDYgKzMyOSw3IEBAIHN0YXRpYyBpbnQgcmVh
c3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4KIHsKICAgICBzdHJ1Y3QgYW1k
X2lvbW11ICppb21tdTsKICAgICBpbnQgYmRmLCByYzsKKyAgICBjb25zdCBz
dHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9pdnJz
X21hcHBpbmdzKHBkZXYtPnNlZyk7CiAKICAgICBiZGYgPSBQQ0lfQkRGMihw
ZGV2LT5idXMsIHBkZXYtPmRldmZuKTsKICAgICBpb21tdSA9IGZpbmRfaW9t
bXVfZm9yX2RldmljZShwZGV2LT5zZWcsIGJkZik7CkBAIC0zNDMsMTAgKzM0
NCwyNCBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2RldmljZShzdHJ1Y3QgZG9t
YWluCiAKICAgICBhbWRfaW9tbXVfZGlzYWJsZV9kb21haW5fZGV2aWNlKHNv
dXJjZSwgaW9tbXUsIGRldmZuLCBwZGV2KTsKIAotICAgIGlmICggZGV2Zm4g
PT0gcGRldi0+ZGV2Zm4gKQorICAgIC8qCisgICAgICogSWYgdGhlIGRldmlj
ZSBiZWxvbmdzIHRvIHRoZSBoYXJkd2FyZSBkb21haW4sIGFuZCBpdCBoYXMg
YSB1bml0eSBtYXBwaW5nLAorICAgICAqIGRvbid0IHJlbW92ZSBpdCBmcm9t
IHRoZSBoYXJkd2FyZSBkb21haW4sIGJlY2F1c2UgQklPUyBtYXkgcmVmZXJl
bmNlIHRoYXQKKyAgICAgKiBtYXBwaW5nLgorICAgICAqLworICAgIGlmICgg
IWlzX2hhcmR3YXJlX2RvbWFpbihzb3VyY2UpICkKICAgICB7Ci0gICAgICAg
IGxpc3RfbW92ZSgmcGRldi0+ZG9tYWluX2xpc3QsICZ0YXJnZXQtPnBkZXZf
bGlzdCk7Ci0gICAgICAgIHBkZXYtPmRvbWFpbiA9IHRhcmdldDsKKyAgICAg
ICAgcmMgPSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAo
CisgICAgICAgICAgICAgICAgIHNvdXJjZSwKKyAgICAgICAgICAgICAgICAg
aXZyc19tYXBwaW5nc1tnZXRfZG1hX3JlcXVlc3Rvcl9pZChwZGV2LT5zZWcs
IGJkZildLnVuaXR5X21hcCk7CisgICAgICAgIGlmICggcmMgKQorICAgICAg
ICAgICAgcmV0dXJuIHJjOworICAgIH0KKworICAgIGlmICggZGV2Zm4gPT0g
cGRldi0+ZGV2Zm4gJiYgcGRldi0+ZG9tYWluICE9IGRvbV9pbyApCisgICAg
eworICAgICAgICBsaXN0X21vdmUoJnBkZXYtPmRvbWFpbl9saXN0LCAmZG9t
X2lvLT5wZGV2X2xpc3QpOworICAgICAgICBwZGV2LT5kb21haW4gPSBkb21f
aW87CiAgICAgfQogCiAgICAgcmMgPSBhbGxvY2F0ZV9kb21haW5fcmVzb3Vy
Y2VzKHRhcmdldCk7CkBAIC0zNTcsNiArMzcyLDEyIEBAIHN0YXRpYyBpbnQg
cmVhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4KICAgICBBTURfSU9NTVVf
REVCVUcoIlJlLWFzc2lnbiAlcHAgZnJvbSBkb20lZCB0byBkb20lZFxuIiwK
ICAgICAgICAgICAgICAgICAgICAgJnBkZXYtPnNiZGYsIHNvdXJjZS0+ZG9t
YWluX2lkLCB0YXJnZXQtPmRvbWFpbl9pZCk7CiAKKyAgICBpZiAoIGRldmZu
ID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRvbWFpbiAhPSB0YXJnZXQgKQor
ICAgIHsKKyAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21haW5fbGlzdCwg
JnRhcmdldC0+cGRldl9saXN0KTsKKyAgICAgICAgcGRldi0+ZG9tYWluID0g
dGFyZ2V0OworICAgIH0KKwogICAgIHJldHVybiAwOwogfQogCkBAIC0zNjcs
MjAgKzM4OCwyOCBAQCBzdGF0aWMgaW50IGFtZF9pb21tdV9hc3NpZ25fZGV2
aWNlKHN0cnVjCiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFw
cGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwogICAgIGlu
dCBiZGYgPSBQQ0lfQkRGMihwZGV2LT5idXMsIGRldmZuKTsKICAgICBpbnQg
cmVxX2lkID0gZ2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRldi0+c2VnLCBiZGYp
OwotICAgIGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqdW5pdHlfbWFw
OworICAgIGludCByYyA9IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0
eV9tYXAoCisgICAgICAgICAgICAgICAgIGQsIGl2cnNfbWFwcGluZ3NbcmVx
X2lkXS51bml0eV9tYXAsIGZsYWcpOwogCi0gICAgZm9yICggdW5pdHlfbWFw
ID0gaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVuaXR5X21hcDsgdW5pdHlfbWFw
OwotICAgICAgICAgIHVuaXR5X21hcCA9IHVuaXR5X21hcC0+bmV4dCApCisg
ICAgaWYgKCAhcmMgKQorICAgICAgICByYyA9IHJlYXNzaWduX2RldmljZShw
ZGV2LT5kb21haW4sIGQsIGRldmZuLCBwZGV2KTsKKworICAgIGlmICggcmMg
JiYgIWlzX2hhcmR3YXJlX2RvbWFpbihkKSApCiAgICAgewotICAgICAgICBp
bnQgcmMgPSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKAot
ICAgICAgICAgICAgICAgICAgICAgZCwgdW5pdHlfbWFwLT5hZGRyLCB1bml0
eV9tYXAtPmxlbmd0aCwKLSAgICAgICAgICAgICAgICAgICAgIHVuaXR5X21h
cC0+d3JpdGUsIHVuaXR5X21hcC0+cmVhZCk7CisgICAgICAgIGludCByZXQg
PSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfdW5tYXAoCisgICAg
ICAgICAgICAgICAgICAgICAgZCwgaXZyc19tYXBwaW5nc1tyZXFfaWRdLnVu
aXR5X21hcCk7CiAKLSAgICAgICAgaWYgKCByYyApCi0gICAgICAgICAgICBy
ZXR1cm4gcmM7CisgICAgICAgIGlmICggcmV0ICkKKyAgICAgICAgeworICAg
ICAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIkFNRC1WaTogIgorICAgICAg
ICAgICAgICAgICAgICJ1bml0eS11bm1hcCBmb3IgJXBkLyUwNHg6JTAyeDol
MDJ4LiV1IGZhaWxlZCAoJWQpXG4iLAorICAgICAgICAgICAgICAgICAgIGQs
IHBkZXYtPnNlZywgcGRldi0+YnVzLAorICAgICAgICAgICAgICAgICAgIFBD
SV9TTE9UKGRldmZuKSwgUENJX0ZVTkMoZGV2Zm4pLCByZXQpOworICAgICAg
ICAgICAgZG9tYWluX2NyYXNoKGQpOworICAgICAgICB9CiAgICAgfQogCi0g
ICAgcmV0dXJuIHJlYXNzaWduX2RldmljZShwZGV2LT5kb21haW4sIGQsIGRl
dmZuLCBwZGV2KTsKKyAgICByZXR1cm4gcmM7CiB9CiAKIHN0YXRpYyB2b2lk
IGFtZF9pb21tdV9jbGVhcl9yb290X3BndGFibGUoc3RydWN0IGRvbWFpbiAq
ZCkKQEAgLTM5NCw2ICs0MjMsNyBAQCBzdGF0aWMgdm9pZCBhbWRfaW9tbXVf
Y2xlYXJfcm9vdF9wZ3RhYmxlCiAKIHN0YXRpYyB2b2lkIGFtZF9pb21tdV9k
b21haW5fZGVzdHJveShzdHJ1Y3QgZG9tYWluICpkKQogeworICAgIGlvbW11
X2lkZW50aXR5X21hcF90ZWFyZG93bihkKTsKICAgICBBU1NFUlQoIWRvbV9p
b21tdShkKS0+YXJjaC5hbWQucm9vdF90YWJsZSk7CiB9CiAK

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KCi0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdS5oCisrKyBiL3hlbi9kcml2
ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdS5oCkBAIC0zMDQsNiArMzA0LDgg
QEAgZXh0ZXJuIHN0cnVjdCBocGV0X3NiZGYgewogICAgIH0gaW5pdDsKIH0g
aHBldF9zYmRmOwogCitleHRlcm4gaW50IGFtZF9pb21tdV9taW5fcGFnaW5n
X21vZGU7CisKIGV4dGVybiB2b2lkICpzaGFyZWRfaW50cmVtYXBfdGFibGU7
CiBleHRlcm4gdW5zaWduZWQgbG9uZyAqc2hhcmVkX2ludHJlbWFwX2ludXNl
OwogCi0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9h
Y3BpLmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11
X2FjcGkuYwpAQCAtMTE3LDEyICsxMTcsOCBAQCBzdGF0aWMgc3RydWN0IGFt
ZF9pb21tdSAqIF9faW5pdCBmaW5kX2lvCiB9CiAKIHN0YXRpYyBpbnQgX19p
bml0IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKAotICAgIHN0cnVj
dCBhbWRfaW9tbXUgKmlvbW11LCB1aW50NjRfdCBiYXNlLCB1aW50NjRfdCBs
aW1pdCwKLSAgICBib29sIGFsbCwgYm9vbCBpdywgYm9vbCBpcikKKyAgICBz
dHJ1Y3QgYW1kX2lvbW11ICppb21tdSwgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGxpbWl0LCBib29sIGFsbCkKIHsKLSAgICBpZiAoICFpciB8fCAhaXcgKQot
ICAgICAgICByZXR1cm4gLUVQRVJNOwotCiAgICAgLyogbmVlZCB0byBleHRl
bmQgZXhjbHVzaW9uIHJhbmdlPyAqLwogICAgIGlmICggaW9tbXUtPmV4Y2x1
c2lvbl9lbmFibGUgKQogICAgIHsKQEAgLTE1MSwxNCArMTQ3LDE4IEBAIHN0
YXRpYyBpbnQgX19pbml0IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl8KIHsKICAg
ICBzdHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9p
dnJzX21hcHBpbmdzKHNlZyk7CiAgICAgc3RydWN0IGl2cnNfdW5pdHlfbWFw
ICp1bml0eV9tYXAgPSBpdnJzX21hcHBpbmdzW2JkZl0udW5pdHlfbWFwOwor
ICAgIGludCBwYWdpbmdfbW9kZSA9IGFtZF9pb21tdV9nZXRfcGFnaW5nX21v
ZGUoUEZOX1VQKGJhc2UgKyBsZW5ndGgpKTsKKworICAgIGlmICggcGFnaW5n
X21vZGUgPCAwICkKKyAgICAgICAgcmV0dXJuIHBhZ2luZ19tb2RlOwogCiAg
ICAgLyogQ2hlY2sgZm9yIG92ZXJsYXBzLiAqLwogICAgIGZvciAoIDsgdW5p
dHlfbWFwOyB1bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQogICAgIHsK
ICAgICAgICAgLyoKICAgICAgICAgICogRXhhY3QgbWF0Y2hlcyBhcmUgb2th
eS4gVGhpcyBjYW4gaW4gcGFydGljdWxhciBoYXBwZW4gd2hlbgotICAgICAg
ICAgKiByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgpIGNh
bGxzIGhlcmUgdHdpY2UgZm9yIHRoZQotICAgICAgICAgKiBzYW1lIChzLGIs
ZCxmKS4KKyAgICAgICAgICogcmVnaXN0ZXJfcmFuZ2VfZm9yX2RldmljZSgp
IGNhbGxzIGhlcmUgdHdpY2UgZm9yIHRoZSBzYW1lCisgICAgICAgICAqIChz
LGIsZCxmKS4KICAgICAgICAgICovCiAgICAgICAgIGlmICggYmFzZSA9PSB1
bml0eV9tYXAtPmFkZHIgJiYgbGVuZ3RoID09IHVuaXR5X21hcC0+bGVuZ3Ro
ICYmCiAgICAgICAgICAgICAgaXIgPT0gdW5pdHlfbWFwLT5yZWFkICYmIGl3
ID09IHVuaXR5X21hcC0+d3JpdGUgKQpAQCAtMTg2LDU1ICsxODYsNTIgQEAg
c3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV91bml0eV9tYXBfZm9yXwogICAg
IHVuaXR5X21hcC0+bmV4dCA9IGl2cnNfbWFwcGluZ3NbYmRmXS51bml0eV9t
YXA7CiAgICAgaXZyc19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcCA9IHVuaXR5
X21hcDsKIAorICAgIGlmICggcGFnaW5nX21vZGUgPiBhbWRfaW9tbXVfbWlu
X3BhZ2luZ19tb2RlICkKKyAgICAgICAgYW1kX2lvbW11X21pbl9wYWdpbmdf
bW9kZSA9IHBhZ2luZ19tb2RlOworCiAgICAgcmV0dXJuIDA7CiB9CiAKLXN0
YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3Jf
YWxsX2RldmljZXMoCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25l
ZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5p
dCByZWdpc3Rlcl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCisgICAgcGFkZHJf
dCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29s
IGV4Y2x1c2lvbikKIHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCi0g
ICAgdW5zaWduZWQgbG9uZyByYW5nZV90b3AsIGlvbW11X3RvcCwgbGVuZ3Ro
OwogICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwotICAgIHVuc2lnbmVk
IGludCBiZGY7CiAgICAgaW50IHJjID0gMDsKIAogICAgIC8qIGlzIHBhcnQg
b2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBvZiBJT01NVSB2aXJ0dWFsIGFk
ZHJlc3Mgc3BhY2U/ICovCiAgICAgLyogbm90ZTogJ2xpbWl0JyBwYXJhbWV0
ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFsaWduZWQgKi8KLSAgICByYW5n
ZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsKLSAgICBpb21tdV90b3AgPSBt
YXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBpZiAoIGJhc2UgPCBpb21tdV90
b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCByYW5nZV90b3AgPiBpb21tdV90
b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9wID0gaW9tbXVfdG9wOwotICAg
ICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBiYXNlOwotICAgICAgICAvKiBy
ZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZp
Y2VzICovCi0gICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJpZXMgYXJlIHBh
cnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwotICAgICAgICBmb3IgKCBi
ZGYgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysg
KQotICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2
aWNlKHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0gICAgICAg
IC8qIHB1c2ggJ2Jhc2UnIGp1c3Qgb3V0c2lkZSBvZiB2aXJ0dWFsIGFkZHJl
c3Mgc3BhY2UgKi8KLSAgICAgICAgYmFzZSA9IGlvbW11X3RvcDsKLSAgICB9
Ci0gICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhjbHVzaW9uIHJhbmdlIHNldHRp
bmdzICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9tbXVfdG9wICkK
KyAgICBpZiAoIGV4Y2x1c2lvbiApCiAgICAgewogICAgICAgICBmb3JfZWFj
aF9hbWRfaW9tbXUoIGlvbW11ICkKICAgICAgICAgewotICAgICAgICAgICAg
cmMgPSByZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFz
ZSwgbGltaXQsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOwotICAgICAg
ICAgICAgaWYgKCByYyApCi0gICAgICAgICAgICAgICAgYnJlYWs7CisgICAg
ICAgICAgICBpbnQgcmV0ID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFu
Z2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRydWUgLyogYWxsICov
KTsKKworICAgICAgICAgICAgaWYgKCByZXQgJiYgIXJjICkKKyAgICAgICAg
ICAgICAgICByYyA9IHJldDsKICAgICAgICAgfQogICAgIH0KIAorICAgIGlm
ICggIWV4Y2x1c2lvbiB8fCByYyApCisgICAgeworICAgICAgICBwYWRkcl90
IGxlbmd0aCA9IGxpbWl0ICsgUEFHRV9TSVpFIC0gYmFzZTsKKyAgICAgICAg
dW5zaWduZWQgaW50IGJkZjsKKworICAgICAgICAvKiByZXNlcnZlIHIvdyB1
bml0eS1tYXBwZWQgcGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCisgICAg
ICAgIGZvciAoIGJkZiA9IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNfYmRm
X2VudHJpZXM7IGJkZisrICkKKyAgICAgICAgICAgIHJjID0gcmVzZXJ2ZV91
bml0eV9tYXBfZm9yX2RldmljZShzZWcsIGJkZiwgYmFzZSwgbGVuZ3RoLCBp
dywgaXIpOworICAgIH0KKwogICAgIHJldHVybiByYzsKIH0KIAotc3RhdGlj
IGludCBfX2luaXQgcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9kZXZp
Y2UoCi0gICAgdTE2IGJkZiwgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25l
ZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5p
dCByZWdpc3Rlcl9yYW5nZV9mb3JfZGV2aWNlKAorICAgIHVuc2lnbmVkIGlu
dCBiZGYsIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwKKyAgICBib29s
IGl3LCBib29sIGlyLCBib29sIGV4Y2x1c2lvbikKIHsKICAgICBpbnQgc2Vn
ID0gMDsgLyogWFhYICovCiAgICAgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2
cnNfbWFwcGluZ3MgPSBnZXRfaXZyc19tYXBwaW5ncyhzZWcpOwotICAgIHVu
c2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsKICAg
ICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdTsKICAgICB1MTYgcmVxOwogICAg
IGludCByYyA9IDA7CkBAIC0yNDgsMjcgKzI0NSwxOSBAQCBzdGF0aWMgaW50
IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgcmVxID0gaXZy
c19tYXBwaW5nc1tiZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7CiAKICAgICAvKiBu
b3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2Ut
YWxpZ25lZCAqLwotICAgIHJhbmdlX3RvcCA9IGxpbWl0ICsgUEFHRV9TSVpF
OwotICAgIGlvbW11X3RvcCA9IG1heF9wYWdlICogUEFHRV9TSVpFOwotICAg
IGlmICggYmFzZSA8IGlvbW11X3RvcCApCi0gICAgewotICAgICAgICBpZiAo
IHJhbmdlX3RvcCA+IGlvbW11X3RvcCApCi0gICAgICAgICAgICByYW5nZV90
b3AgPSBpb21tdV90b3A7Ci0gICAgICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAt
IGJhc2U7CisgICAgaWYgKCBleGNsdXNpb24gKQorICAgICAgICByYyA9IHJl
c2VydmVfaW9tbXVfZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1p
dCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBmYWxzZSAvKiBhbGwgKi8pOworICAgIGlmICggIWV4Y2x1c2lvbiB8fCBy
YyApCisgICAgeworICAgICAgICBwYWRkcl90IGxlbmd0aCA9IGxpbWl0ICsg
UEFHRV9TSVpFIC0gYmFzZTsKKwogICAgICAgICAvKiByZXNlcnZlIHVuaXR5
LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZSAqLwotICAgICAgICAv
KiBub3RlOiB0aGVzZSBlbnRyaWVzIGFyZSBwYXJ0IG9mIHRoZSBleGNsdXNp
b24gcmFuZ2UgKi8KICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9m
b3JfZGV2aWNlKHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcikgPzoK
ICAgICAgICAgICAgICByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKHNl
ZywgcmVxLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7Ci0KLSAgICAgICAgLyog
cHVzaCAnYmFzZScganVzdCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRkcmVzcyBz
cGFjZSAqLwotICAgICAgICBiYXNlID0gaW9tbXVfdG9wOwogICAgIH0KLQot
ICAgIC8qIHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5n
cyBmb3IgZGV2aWNlICovCi0gICAgaWYgKCAhcmMgJiYgbGltaXQgPj0gaW9t
bXVfdG9wICApCisgICAgZWxzZQogICAgIHsKLSAgICAgICAgcmMgPSByZXNl
cnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQs
Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ZmFsc2UgLyogYWxsICovLCBpdywgaXIpOwogICAgICAgICBpdnJzX21hcHBp
bmdzW2JkZl0uZHRlX2FsbG93X2V4Y2x1c2lvbiA9IHRydWU7CiAgICAgICAg
IGl2cnNfbWFwcGluZ3NbcmVxXS5kdGVfYWxsb3dfZXhjbHVzaW9uID0gdHJ1
ZTsKICAgICB9CkBAIC0yNzYsNTMgKzI2NSw0MiBAQCBzdGF0aWMgaW50IF9f
aW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFuCiAgICAgcmV0dXJuIHJjOwog
fQogCi1zdGF0aWMgaW50IF9faW5pdCByZWdpc3Rlcl9leGNsdXNpb25fcmFu
Z2VfZm9yX2lvbW11X2RldmljZXMoCi0gICAgc3RydWN0IGFtZF9pb21tdSAq
aW9tbXUsCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNpZ25lZCBsb25n
IGxpbWl0LCB1OCBpdywgdTggaXIpCitzdGF0aWMgaW50IF9faW5pdCByZWdp
c3Rlcl9yYW5nZV9mb3JfaW9tbXVfZGV2aWNlcygKKyAgICBzdHJ1Y3QgYW1k
X2lvbW11ICppb21tdSwgcGFkZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LAor
ICAgIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewotICAg
IHVuc2lnbmVkIGxvbmcgcmFuZ2VfdG9wLCBpb21tdV90b3AsIGxlbmd0aDsK
KyAgICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRv
IGJlIHBhZ2UtYWxpZ25lZCAqLworICAgIHBhZGRyX3QgbGVuZ3RoID0gbGlt
aXQgKyBQQUdFX1NJWkUgLSBiYXNlOwogICAgIHVuc2lnbmVkIGludCBiZGY7
CiAgICAgdTE2IHJlcTsKLSAgICBpbnQgcmMgPSAwOworICAgIGludCByYzsK
IAotICAgIC8qIGlzIHBhcnQgb2YgZXhjbHVzaW9uIHJhbmdlIGluc2lkZSBv
ZiBJT01NVSB2aXJ0dWFsIGFkZHJlc3Mgc3BhY2U/ICovCi0gICAgLyogbm90
ZTogJ2xpbWl0JyBwYXJhbWV0ZXIgaXMgYXNzdW1lZCB0byBiZSBwYWdlLWFs
aWduZWQgKi8KLSAgICByYW5nZV90b3AgPSBsaW1pdCArIFBBR0VfU0laRTsK
LSAgICBpb21tdV90b3AgPSBtYXhfcGFnZSAqIFBBR0VfU0laRTsKLSAgICBp
ZiAoIGJhc2UgPCBpb21tdV90b3AgKQotICAgIHsKLSAgICAgICAgaWYgKCBy
YW5nZV90b3AgPiBpb21tdV90b3AgKQotICAgICAgICAgICAgcmFuZ2VfdG9w
ID0gaW9tbXVfdG9wOwotICAgICAgICBsZW5ndGggPSByYW5nZV90b3AgLSBi
YXNlOwotICAgICAgICAvKiByZXNlcnZlIHIvdyB1bml0eS1tYXBwZWQgcGFn
ZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCi0gICAgICAgIC8qIG5vdGU6IHRo
ZXNlIGVudHJpZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAq
LwotICAgICAgICBmb3IgKCBiZGYgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19i
ZGZfZW50cmllczsgYmRmKysgKQotICAgICAgICB7Ci0gICAgICAgICAgICBp
ZiAoIGlvbW11ID09IGZpbmRfaW9tbXVfZm9yX2RldmljZShpb21tdS0+c2Vn
LCBiZGYpICkKLSAgICAgICAgICAgIHsKLSAgICAgICAgICAgICAgICByZXEg
PSBnZXRfaXZyc19tYXBwaW5ncyhpb21tdS0+c2VnKVtiZGZdLmR0ZV9yZXF1
ZXN0b3JfaWQ7Ci0gICAgICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5
X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZiwgYmFzZSwgbGVuZ3Ro
LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBpdywgaXIpID86Ci0gICAgICAgICAgICAgICAgICAgICByZXNl
cnZlX3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFz
ZSwgbGVuZ3RoLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBpdywgaXIpOwotICAgICAgICAgICAgfQotICAg
ICAgICB9Ci0KLSAgICAgICAgLyogcHVzaCAnYmFzZScganVzdCBvdXRzaWRl
IG9mIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSAqLwotICAgICAgICBiYXNlID0g
aW9tbXVfdG9wOworICAgIGlmICggZXhjbHVzaW9uICkKKyAgICB7CisgICAg
ICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUs
IGJhc2UsIGxpbWl0LCB0cnVlIC8qIGFsbCAqLyk7CisgICAgICAgIGlmICgg
IXJjICkKKyAgICAgICAgICAgIHJldHVybiAwOwogICAgIH0KIAotICAgIC8q
IHJlZ2lzdGVyIElPTU1VIGV4Y2x1c2lvbiByYW5nZSBzZXR0aW5ncyAqLwot
ICAgIGlmICggIXJjICYmIGxpbWl0ID49IGlvbW11X3RvcCApCi0gICAgICAg
IHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNpb25fcmFuZ2UoaW9tbXUsIGJh
c2UsIGxpbWl0LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHRydWUgLyogYWxsICovLCBpdywgaXIpOworICAgIC8qIHJl
c2VydmUgdW5pdHktbWFwcGVkIHBhZ2UgZW50cmllcyBmb3IgZGV2aWNlcyAq
LworICAgIGZvciAoIGJkZiA9IHJjID0gMDsgIXJjICYmIGJkZiA8IGl2cnNf
YmRmX2VudHJpZXM7IGJkZisrICkKKyAgICB7CisgICAgICAgIGlmICggaW9t
bXUgIT0gZmluZF9pb21tdV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikg
KQorICAgICAgICAgICAgY29udGludWU7CisKKyAgICAgICAgcmVxID0gZ2V0
X2l2cnNfbWFwcGluZ3MoaW9tbXUtPnNlZylbYmRmXS5kdGVfcmVxdWVzdG9y
X2lkOworICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZp
Y2UoaW9tbXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIpID86Cisg
ICAgICAgICAgICAgcmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShpb21t
dS0+c2VnLCByZXEsIGJhc2UsIGxlbmd0aCwKKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcik7CisgICAgfQogCiAg
ICAgcmV0dXJuIHJjOwogfQogCiBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9p
dm1kX2RldmljZV9zZWxlY3QoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZy
c19tZW1vcnkgKml2bWRfYmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNl
LCB1bnNpZ25lZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFk
ZHJfdCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBi
b29sIGV4Y2x1c2lvbikKIHsKICAgICB1MTYgYmRmOwogCkBAIC0zMzMsMTIg
KzMxMSwxMiBAQCBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2Rldmlj
ZV9zZWxlCiAgICAgICAgIHJldHVybiAtRU5PREVWOwogICAgIH0KIAotICAg
IHJldHVybiByZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZShi
ZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgIHJldHVybiByZWdpc3Rl
cl9yYW5nZV9mb3JfZGV2aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpciwg
ZXhjbHVzaW9uKTsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZt
ZF9kZXZpY2VfcmFuZ2UoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19t
ZW1vcnkgKml2bWRfYmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1
bnNpZ25lZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJf
dCBiYXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29s
IGV4Y2x1c2lvbikKIHsKICAgICB1bnNpZ25lZCBpbnQgZmlyc3RfYmRmLCBs
YXN0X2JkZiwgYmRmOwogICAgIGludCBlcnJvcjsKQEAgLTM2MCwxNSArMzM4
LDE1IEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3Jh
bmcKICAgICB9CiAKICAgICBmb3IgKCBiZGYgPSBmaXJzdF9iZGYsIGVycm9y
ID0gMDsgKGJkZiA8PSBsYXN0X2JkZikgJiYgIWVycm9yOyBiZGYrKyApCi0g
ICAgICAgIGVycm9yID0gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9k
ZXZpY2UoCi0gICAgICAgICAgICBiZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIp
OworICAgICAgICBlcnJvciA9IHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2Uo
CisgICAgICAgICAgICBiZGYsIGJhc2UsIGxpbWl0LCBpdywgaXIsIGV4Y2x1
c2lvbik7CiAKICAgICByZXR1cm4gZXJyb3I7CiB9CiAKIHN0YXRpYyBpbnQg
X19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX2lvbW11KAogICAgIGNvbnN0IHN0
cnVjdCBhY3BpX2l2cnNfbWVtb3J5ICppdm1kX2Jsb2NrLAotICAgIHVuc2ln
bmVkIGxvbmcgYmFzZSwgdW5zaWduZWQgbG9uZyBsaW1pdCwgdTggaXcsIHU4
IGlyKQorICAgIHBhZGRyX3QgYmFzZSwgcGFkZHJfdCBsaW1pdCwgYm9vbCBp
dywgYm9vbCBpciwgYm9vbCBleGNsdXNpb24pCiB7CiAgICAgaW50IHNlZyA9
IDA7IC8qIFhYWCAqLwogICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11OwpA
QCAtMzgzLDE0ICszNjEsMTQgQEAgc3RhdGljIGludCBfX2luaXQgcGFyc2Vf
aXZtZF9kZXZpY2VfaW9tbQogICAgICAgICByZXR1cm4gLUVOT0RFVjsKICAg
ICB9CiAKLSAgICByZXR1cm4gcmVnaXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zv
cl9pb21tdV9kZXZpY2VzKAotICAgICAgICBpb21tdSwgYmFzZSwgbGltaXQs
IGl3LCBpcik7CisgICAgcmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21t
dV9kZXZpY2VzKAorICAgICAgICBpb21tdSwgYmFzZSwgbGltaXQsIGl3LCBp
ciwgZXhjbHVzaW9uKTsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2Vf
aXZtZF9ibG9jayhjb25zdCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZt
ZF9ibG9jaykKIHsKICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0X2FkZHIsIG1l
bV9sZW5ndGgsIGJhc2UsIGxpbWl0OwotICAgIHU4IGl3LCBpcjsKKyAgICBi
b29sIGl3ID0gdHJ1ZSwgaXIgPSB0cnVlLCBleGNsdXNpb24gPSBmYWxzZTsK
IAogICAgIGlmICggaXZtZF9ibG9jay0+aGVhZGVyLmxlbmd0aCA8IHNpemVv
ZigqaXZtZF9ibG9jaykgKQogICAgIHsKQEAgLTQwNywxMyArMzg1LDExIEBA
IHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfYmxvY2soY29uc3QKICAg
ICAgICAgICAgICAgICAgICAgaXZtZF9ibG9jay0+aGVhZGVyLnR5cGUsIHN0
YXJ0X2FkZHIsIG1lbV9sZW5ndGgpOwogCiAgICAgaWYgKCBpdm1kX2Jsb2Nr
LT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfRVhDTFVTSU9OX1JBTkdFICkK
LSAgICAgICAgaXcgPSBpciA9IElPTU1VX0NPTlRST0xfRU5BQkxFRDsKKyAg
ICAgICAgZXhjbHVzaW9uID0gdHJ1ZTsKICAgICBlbHNlIGlmICggaXZtZF9i
bG9jay0+aGVhZGVyLmZsYWdzICYgQUNQSV9JVk1EX1VOSVRZICkKICAgICB7
Ci0gICAgICAgIGl3ID0gaXZtZF9ibG9jay0+aGVhZGVyLmZsYWdzICYgQUNQ
SV9JVk1EX1JFQUQgPwotICAgICAgICAgICAgSU9NTVVfQ09OVFJPTF9FTkFC
TEVEIDogSU9NTVVfQ09OVFJPTF9ESVNBQkxFRDsKLSAgICAgICAgaXIgPSBp
dm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEUgPwot
ICAgICAgICAgICAgSU9NTVVfQ09OVFJPTF9FTkFCTEVEIDogSU9NTVVfQ09O
VFJPTF9ESVNBQkxFRDsKKyAgICAgICAgaXcgPSBpdm1kX2Jsb2NrLT5oZWFk
ZXIuZmxhZ3MgJiBBQ1BJX0lWTURfUkVBRDsKKyAgICAgICAgaXIgPSBpdm1k
X2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfV1JJVEU7CiAgICAg
fQogICAgIGVsc2UKICAgICB7CkBAIC00MjQsMjAgKzQwMCwyMCBAQCBzdGF0
aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2Jsb2NrKGNvbnN0CiAgICAgc3dp
dGNoKCBpdm1kX2Jsb2NrLT5oZWFkZXIudHlwZSApCiAgICAgewogICAgIGNh
c2UgQUNQSV9JVlJTX1RZUEVfTUVNT1JZX0FMTDoKLSAgICAgICAgcmV0dXJu
IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfYWxsX2RldmljZXMoCi0g
ICAgICAgICAgICBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0
dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygKKyAgICAgICAg
ICAgIGJhc2UsIGxpbWl0LCBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAgICBj
YXNlIEFDUElfSVZSU19UWVBFX01FTU9SWV9PTkU6Ci0gICAgICAgIHJldHVy
biBwYXJzZV9pdm1kX2RldmljZV9zZWxlY3QoaXZtZF9ibG9jaywKLSAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBsaW1p
dCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNl
X3NlbGVjdChpdm1kX2Jsb2NrLCBiYXNlLCBsaW1pdCwKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lv
bik7CiAKICAgICBjYXNlIEFDUElfSVZSU19UWVBFX01FTU9SWV9SQU5HRToK
LSAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX3JhbmdlKGl2bWRf
YmxvY2ssCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNl
X2l2bWRfZGV2aWNlX3JhbmdlKGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGly
LCBleGNsdXNpb24pOwogCiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9NRU1P
UllfSU9NTVU6Ci0gICAgICAgIHJldHVybiBwYXJzZV9pdm1kX2RldmljZV9p
b21tdShpdm1kX2Jsb2NrLAotICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgYmFzZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIHJl
dHVybiBwYXJzZV9pdm1kX2RldmljZV9pb21tdShpdm1kX2Jsb2NrLCBiYXNl
LCBsaW1pdCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGRlZmF1bHQ6CiAgICAg
ICAgIEFNRF9JT01NVV9ERUJVRygiSVZNRCBFcnJvcjogSW52YWxpZCBCbG9j
ayBUeXBlIVxuIik7Ci0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Ft
ZC9wY2lfYW1kX2lvbW11LmMKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYwpAQCAtMjM0LDYgKzIzNCw4IEBAIHN0
YXRpYyBpbnQgX19tdXN0X2NoZWNrIGFsbG9jYXRlX2RvbWFpbl8KICAgICBy
ZXR1cm4gcmM7CiB9CiAKK2ludCBfX3JlYWRfbW9zdGx5IGFtZF9pb21tdV9t
aW5fcGFnaW5nX21vZGUgPSAxOworCiBzdGF0aWMgaW50IGFtZF9pb21tdV9k
b21haW5faW5pdChzdHJ1Y3QgZG9tYWluICpkKQogewogICAgIHN0cnVjdCBk
b21haW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQpOwpAQCAtMjQ1LDExICsy
NDcsMTMgQEAgc3RhdGljIGludCBhbWRfaW9tbXVfZG9tYWluX2luaXQoc3Ry
dWN0CiAgICAgICogLSBIVk0gY291bGQgaW4gcHJpbmNpcGxlIHVzZSAzIG9y
IDQgZGVwZW5kaW5nIG9uIGhvdyBtdWNoIGd1ZXN0CiAgICAgICogICBwaHlz
aWNhbCBhZGRyZXNzIHNwYWNlIHdlIGdpdmUgaXQsIGJ1dCB0aGlzIGlzbid0
IGtub3duIHlldCBzbyB1c2UgNAogICAgICAqICAgdW5pbGF0ZXJhbGx5Lgor
ICAgICAqIC0gVW5pdHkgbWFwcyBtYXkgcmVxdWlyZSBhbiBldmVuIGhpZ2hl
ciBudW1iZXIuCiAgICAgICovCi0gICAgaGQtPmFyY2guYW1kLnBhZ2luZ19t
b2RlID0gYW1kX2lvbW11X2dldF9wYWdpbmdfbW9kZSgKLSAgICAgICAgaXNf
aHZtX2RvbWFpbihkKQotICAgICAgICA/IDF1bCA8PCAoREVGQVVMVF9ET01B
SU5fQUREUkVTU19XSURUSCAtIFBBR0VfU0hJRlQpCi0gICAgICAgIDogZ2V0
X3VwcGVyX21mbl9ib3VuZCgpICsgMSk7CisgICAgaGQtPmFyY2guYW1kLnBh
Z2luZ19tb2RlID0gbWF4KGFtZF9pb21tdV9nZXRfcGFnaW5nX21vZGUoCisg
ICAgICAgICAgICBpc19odm1fZG9tYWluKGQpCisgICAgICAgICAgICA/IDF1
bCA8PCAoREVGQVVMVF9ET01BSU5fQUREUkVTU19XSURUSCAtIFBBR0VfU0hJ
RlQpCisgICAgICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91bmQoKSArIDEp
LAorICAgICAgICBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2RlKTsKIAogICAg
IHJldHVybiAwOwogfQo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtODExLDcgKzgx
MSw3IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRl
cik7IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2Vu
dHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5V
TEwpOwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFw
Mm1faXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAg
ICAgICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3No
YXJlZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4o
bWZuX3gobWZuKSArIGksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAg
fQogICAgIH0KQEAgLTk0MSwxMyArOTQxLDEzIEBAIGd1ZXN0X3BoeXNtYXBf
YWRkX2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAg
ICAgICAgICAgIEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAg
ICB9Ci0gICAgICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNf
Zm9yZWlnbihvdCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90
KSApCiAgICAgICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRu
J3QgYmUgdW5tYXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAq
LworICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFu
dC9mb3JlaWduIHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2Ny
YXNoKGQpOwogICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAg
ICAgICAgCi0gICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAg
ICAgIHJldHVybiAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBp
ZiAoIHAybV9pc19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAg
ICAgICAgewpAQCAtMTA0MSw4ICsxMDQxLDcgQEAgaW50IHAybV9jaGFuZ2Vf
dHlwZV9vbmUoc3RydWN0IGRvbWFpbiAqZAogICAgIHN0cnVjdCBwMm1fZG9t
YWluICpwMm0gPSBwMm1fZ2V0X2hvc3RwMm0oZCk7CiAgICAgaW50IHJjOwog
Ci0gICAgQlVHX09OKHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lzX2dyYW50
KG50KSk7Ci0gICAgQlVHX09OKHAybV9pc19mb3JlaWduKG90KSB8fCBwMm1f
aXNfZm9yZWlnbihudCkpOworICAgIEJVR19PTihwMm1faXNfc3BlY2lhbChv
dCkgfHwgcDJtX2lzX3NwZWNpYWwobnQpKTsKIAogICAgIGdmbl9sb2NrKHAy
bSwgZ2ZuLCAwKTsKIApAQCAtMTI4OSwxMSArMTI4OCwxMSBAQCBzdGF0aWMg
aW50IHNldF90eXBlZF9wMm1fZW50cnkoc3RydWN0IGRvCiAgICAgICAgIGdm
bl91bmxvY2socDJtLCBnZm4sIG9yZGVyKTsKICAgICAgICAgcmV0dXJuIGN1
cl9vcmRlciArIDE7CiAgICAgfQotICAgIGlmICggcDJtX2lzX2dyYW50KG90
KSB8fCBwMm1faXNfZm9yZWlnbihvdCkgKQorICAgIGlmICggcDJtX2lzX3Nw
ZWNpYWwob3QpICkKICAgICB7CiAgICAgICAgIGdmbl91bmxvY2socDJtLCBn
Zm4sIG9yZGVyKTsKICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwotICAgICAg
ICByZXR1cm4gLUVOT0VOVDsKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKICAg
ICB9CiAgICAgZWxzZSBpZiAoIHAybV9pc19yYW0ob3QpICkKICAgICB7Ci0t
LSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1
ZGUvYXNtLXg4Ni9wMm0uaApAQCAtMTQ5LDYgKzE0OSwxMCBAQCB0eXBlZGVm
IHVuc2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICB8IHAybV90b19tYXNrKHAybV9yYW1fbG9nZGlydHkpICkK
ICNkZWZpbmUgUDJNX1NIQVJFRF9UWVBFUyAgIChwMm1fdG9fbWFzayhwMm1f
cmFtX3NoYXJlZCkpCiAKKy8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQg
dXAgdmlhIHNwZWNpYWwgYWNjZXNzb3JzLiAqLworI2RlZmluZSBQMk1fU1BF
Q0lBTF9UWVBFUyAoUDJNX0dSQU5UX1RZUEVTIHwgXAorICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkK
KwogLyogVmFsaWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQg
d2l0aCBhICh2YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9N
Rk5fVFlQRVMgKFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCBwMm1fdG9fbWFzayhw
Mm1fbW1pb19kaXJlY3QpIFwKQEAgLTE3Nyw2ICsxODEsNyBAQCB0eXBlZGVm
IHVuc2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICNkZWZpbmUgcDJtX2lzX3Bh
Z2VkKF90KSAgICAocDJtX3RvX21hc2soX3QpICYgUDJNX1BBR0VEX1RZUEVT
KQogI2RlZmluZSBwMm1faXNfc2hhcmFibGUoX3QpIChwMm1fdG9fbWFzayhf
dCkgJiBQMk1fU0hBUkFCTEVfVFlQRVMpCiAjZGVmaW5lIHAybV9pc19zaGFy
ZWQoX3QpICAgKHAybV90b19tYXNrKF90KSAmIFAyTV9TSEFSRURfVFlQRVMp
CisjZGVmaW5lIHAybV9pc19zcGVjaWFsKF90KSAgKHAybV90b19tYXNrKF90
KSAmIFAyTV9TUEVDSUFMX1RZUEVTKQogI2RlZmluZSBwMm1faXNfYnJva2Vu
KF90KSAgIChwMm1fdG9fbWFzayhfdCkgJiBQMk1fQlJPS0VOX1RZUEVTKQog
I2RlZmluZSBwMm1faXNfZm9yZWlnbihfdCkgIChwMm1fdG9fbWFzayhfdCkg
JiBwMm1fdG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pKQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.15-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.15-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3Ay
bS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtNzk5LDcgKzc5
OSw4IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJt
LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
JmN1cl9vcmRlciwgTlVMTCk7CiAKICAgICAgICAgaWYgKCBwMm1faXNfdmFs
aWQodCkgJiYKLSAgICAgICAgICAgICAoIW1mbl92YWxpZChtZm4pIHx8ICFt
Zm5fZXEobWZuX2FkZChtZm4sIGkpLCBtZm5fcmV0dXJuKSkgKQorICAgICAg
ICAgICAgICghbWZuX3ZhbGlkKG1mbikgfHwgdCA9PSBwMm1fbW1pb19kaXJl
Y3QgfHwKKyAgICAgICAgICAgICAgIW1mbl9lcShtZm5fYWRkKG1mbiwgaSks
IG1mbl9yZXR1cm4pKSApCiAgICAgICAgICAgICByZXR1cm4gLUVJTFNFUTsK
IAogICAgICAgICBpICs9ICgxVUwgPDwgY3VyX29yZGVyKSAtCkBAIC04OTks
NyArOTAwLDcgQEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRv
bWFpbiAqZAogICAgIGlmICggcDJtX2lzX2ZvcmVpZ24odCkgKQogICAgICAg
ICByZXR1cm4gLUVJTlZBTDsKIAotICAgIGlmICggIW1mbl92YWxpZChtZm4p
ICkKKyAgICBpZiAoICFtZm5fdmFsaWQobWZuKSB8fCB0ID09IHAybV9tbWlv
X2RpcmVjdCApCiAgICAgewogICAgICAgICBBU1NFUlRfVU5SRUFDSEFCTEUo
KTsKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7CkBAIC05NDMsNyArOTQ0LDcg
QEAgZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoc3RydWN0IGRvbWFpbiAqZAog
ICAgICAgICB9CiAgICAgICAgIGlmICggcDJtX2lzX3NwZWNpYWwob3QpICkK
ICAgICAgICAgewotICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFw
cGluZyBncmFudC9mb3JlaWduIHRoaXMgd2F5LiAqLworICAgICAgICAgICAg
LyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9mb3JlaWduL2RpcmVj
dC1NTUlPIHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2NyYXNo
KGQpOwogICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAgICAg
ICAgCkBAIC0xMzk5LDggKzE0MDAsOCBAQCBpbnQgc2V0X2lkZW50aXR5X3Ay
bV9lbnRyeShzdHJ1Y3QgZG9tYWluCiAgKiAgICBvcmRlcisxICBmb3IgY2Fs
bGVyIHRvIHJldHJ5IHdpdGggb3JkZXIgKGd1YXJhbnRlZWQgc21hbGxlciB0
aGFuCiAgKiAgICAgICAgICAgICB0aGUgb3JkZXIgdmFsdWUgcGFzc2VkIGlu
KQogICovCi1pbnQgY2xlYXJfbW1pb19wMm1fZW50cnkoc3RydWN0IGRvbWFp
biAqZCwgdW5zaWduZWQgbG9uZyBnZm5fbCwgbWZuX3QgbWZuLAotICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBvcmRlcikKK3N0YXRp
YyBpbnQgY2xlYXJfbW1pb19wMm1fZW50cnkoc3RydWN0IGRvbWFpbiAqZCwg
dW5zaWduZWQgbG9uZyBnZm5fbCwKKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgbWZuX3QgbWZuLCB1bnNpZ25lZCBpbnQgb3JkZXIpCiB7CiAg
ICAgaW50IHJjID0gLUVJTlZBTDsKICAgICBnZm5fdCBnZm4gPSBfZ2ZuKGdm
bl9sKTsKQEAgLTI3MzEsNyArMjczMiw5IEBAIGludCB4ZW5tZW1fYWRkX3Rv
X3BoeXNtYXBfb25lKAogCiAgICAgLyogUmVtb3ZlIHByZXZpb3VzbHkgbWFw
cGVkIHBhZ2UgaWYgaXQgd2FzIHByZXNlbnQuICovCiAgICAgcHJldl9tZm4g
PSBnZXRfZ2ZuKGQsIGdmbl94KGdwZm4pLCAmcDJtdCk7Ci0gICAgaWYgKCBt
Zm5fdmFsaWQocHJldl9tZm4pICkKKyAgICBpZiAoIHAybXQgPT0gcDJtX21t
aW9fZGlyZWN0ICkKKyAgICAgICAgcmMgPSAtRVBFUk07CisgICAgZWxzZSBp
ZiAoIG1mbl92YWxpZChwcmV2X21mbikgKQogICAgIHsKICAgICAgICAgaWYg
KCBpc19zcGVjaWFsX3BhZ2UobWZuX3RvX3BhZ2UocHJldl9tZm4pKSApCiAg
ICAgICAgICAgICAvKiBTcGVjaWFsIHBhZ2VzIGFyZSBzaW1wbHkgdW5ob29r
ZWQgZnJvbSB0aGlzIHBoeXMgc2xvdC4gKi8KLS0tIGEveGVuL2FyY2gveDg2
L21tL3AybS1wb2QuYworKysgYi94ZW4vYXJjaC94ODYvbW0vcDJtLXBvZC5j
CkBAIC0xMjk5LDE3ICsxMjk5LDE3IEBAIGd1ZXN0X3BoeXNtYXBfbWFya19w
b3B1bGF0ZV9vbl9kZW1hbmQoc3QKIAogICAgICAgICBwMm0tPmdldF9lbnRy
eShwMm0sIGdmbl9hZGQoZ2ZuLCBpKSwgJm90LCAmYSwgMCwgJmN1cl9vcmRl
ciwgTlVMTCk7CiAgICAgICAgIG4gPSAxVUwgPDwgbWluKG9yZGVyLCBjdXJf
b3JkZXIpOwotICAgICAgICBpZiAoIHAybV9pc19yYW0ob3QpICkKKyAgICAg
ICAgaWYgKCBvdCA9PSBwMm1fcG9wdWxhdGVfb25fZGVtYW5kICkKKyAgICAg
ICAgeworICAgICAgICAgICAgLyogQ291bnQgaG93IG1hbnkgUG9EIGVudHJp
ZXMgd2UnbGwgYmUgcmVwbGFjaW5nIGlmIHN1Y2Nlc3NmdWwgKi8KKyAgICAg
ICAgICAgIHBvZF9jb3VudCArPSBuOworICAgICAgICB9CisgICAgICAgIGVs
c2UgaWYgKCBvdCAhPSBwMm1faW52YWxpZCAmJiBvdCAhPSBwMm1fbW1pb19k
bSApCiAgICAgICAgIHsKICAgICAgICAgICAgIFAyTV9ERUJVRygiZ2ZuX3Rv
X21mbiByZXR1cm5lZCB0eXBlICVkIVxuIiwgb3QpOwogICAgICAgICAgICAg
cmMgPSAtRUJVU1k7CiAgICAgICAgICAgICBnb3RvIG91dDsKICAgICAgICAg
fQotICAgICAgICBlbHNlIGlmICggb3QgPT0gcDJtX3BvcHVsYXRlX29uX2Rl
bWFuZCApCi0gICAgICAgIHsKLSAgICAgICAgICAgIC8qIENvdW50IGhvdyBt
YW4gUG9EIGVudHJpZXMgd2UnbGwgYmUgcmVwbGFjaW5nIGlmIHN1Y2Nlc3Nm
dWwgKi8KLSAgICAgICAgICAgIHBvZF9jb3VudCArPSBuOwotICAgICAgICB9
CiAgICAgfQogCiAgICAgLyogTm93LCBhY3R1YWxseSBkbyB0aGUgdHdvLXdh
eSBtYXBwaW5nICovCi0tLSBhL3hlbi9jb21tb24vbWVtb3J5LmMKKysrIGIv
eGVuL2NvbW1vbi9tZW1vcnkuYwpAQCAtMzMwLDcgKzMzMCw3IEBAIGludCBn
dWVzdF9yZW1vdmVfcGFnZShzdHJ1Y3QgZG9tYWluICpkLAogICAgIH0KICAg
ICBpZiAoIHAybXQgPT0gcDJtX21taW9fZGlyZWN0ICkKICAgICB7Ci0gICAg
ICAgIHJjID0gY2xlYXJfbW1pb19wMm1fZW50cnkoZCwgZ21mbiwgbWZuLCBQ
QUdFX09SREVSXzRLKTsKKyAgICAgICAgcmMgPSAtRVBFUk07CiAgICAgICAg
IGdvdG8gb3V0X3B1dF9nZm47CiAgICAgfQogI2Vsc2UKQEAgLTE4NzUsNiAr
MTg3NSwxNSBAQCBpbnQgY2hlY2tfZ2V0X3BhZ2VfZnJvbV9nZm4oc3RydWN0
IGRvbWFpCiAgICAgICAgIHJldHVybiAtRUFHQUlOOwogICAgIH0KICNlbmRp
ZgorI2lmZGVmIENPTkZJR19YODYKKyAgICBpZiAoIHAybXQgPT0gcDJtX21t
aW9fZGlyZWN0ICkKKyAgICB7CisgICAgICAgIGlmICggcGFnZSApCisgICAg
ICAgICAgICBwdXRfcGFnZShwYWdlKTsKKworICAgICAgICByZXR1cm4gLUVQ
RVJNOworICAgIH0KKyNlbmRpZgogCiAgICAgaWYgKCAhcGFnZSApCiAgICAg
ICAgIHJldHVybiAtRUlOVkFMOwotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2
L3AybS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKQEAgLTE1
MSw3ICsxNTEsOCBAQCB0eXBlZGVmIHVuc2lnbmVkIGludCBwMm1fcXVlcnlf
dDsKIAogLyogVHlwZXMgZXN0YWJsaXNoZWQvY2xlYW5lZCB1cCB2aWEgc3Bl
Y2lhbCBhY2Nlc3NvcnMuICovCiAjZGVmaW5lIFAyTV9TUEVDSUFMX1RZUEVT
IChQMk1fR1JBTlRfVFlQRVMgfCBcCi0gICAgICAgICAgICAgICAgICAgICAg
ICAgICBwMm1fdG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pKQorICAgICAgICAg
ICAgICAgICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWdu
KSB8IFwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV90b19tYXNr
KHAybV9tbWlvX2RpcmVjdCkpCiAKIC8qIFZhbGlkIHR5cGVzIG5vdCBuZWNl
c3NhcmlseSBhc3NvY2lhdGVkIHdpdGggYSAodmFsaWQpIE1GTi4gKi8KICNk
ZWZpbmUgUDJNX0lOVkFMSURfTUZOX1RZUEVTIChQMk1fUE9EX1RZUEVTICAg
ICAgICAgICAgICAgICAgXApAQCAtNjY2LDggKzY2Nyw2IEBAIGludCBwMm1f
aXNfbG9nZGlydHlfcmFuZ2Uoc3RydWN0IHAybV9kb20KIC8qIFNldCBtbWlv
IGFkZHJlc3NlcyBpbiB0aGUgcDJtIHRhYmxlIChmb3IgcGFzcy10aHJvdWdo
KSAqLwogaW50IHNldF9tbWlvX3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluICpk
LCBnZm5fdCBnZm4sIG1mbl90IG1mbiwKICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IG9yZGVyKTsKLWludCBjbGVhcl9tbWlvX3AybV9l
bnRyeShzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwgbWZu
X3QgbWZuLAotICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBvcmRlcik7CiAKIC8qIFNldCBpZGVudGl0eSBhZGRyZXNzZXMgaW4gdGhl
IHAybSB0YWJsZSAoZm9yIHBhc3MtdGhyb3VnaCkgKi8KIGludCBzZXRfaWRl
bnRpdHlfcDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxv
bmcgZ2ZuLAo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-4.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-4.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBJT01NVTogZ2VuZXJhbGl6ZSBWVC1kJ3MgdHJhY2tpbmcgb2YgbWFwcGVk
IFJNUlIgcmVnaW9ucwoKSW4gb3JkZXIgdG8gcmUtdXNlIGl0IGVsc2V3aGVy
ZSwgbW92ZSB0aGUgbG9naWMgdG8gdmVuZG9yIGluZGVwZW5kZW50CmNvZGUg
YW5kIHN0cmlwIGl0IG9mIFJNUlIgc3BlY2lmaWNzLgoKTm90ZSB0aGF0IHRo
ZSBwcmlvciAibWFwIiBwYXJhbWV0ZXIgZ2V0cyBmb2xkZWQgaW50byB0aGUg
bmV3ICJwMm1hIiBvbmUKKHdoaWNoIEFNRCBJT01NVSBjb2RlIHdpbGwgd2Fu
dCB0byBtYWtlIHVzZSBvZiksIGFzc2lnbmluZyBhbHRlcm5hdGl2ZQptZWFu
aW5nICgidW5tYXAiKSB0byBwMm1fYWNjZXNzX3guIFByZXBhcmUgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpIGFuZApwMm1fZ2V0X2lvbW11X2ZsYWdzKCkg
Zm9yIGdldHRpbmcgcGFzc2VkIGFjY2VzcyB0eXBlcyBvdGhlciB0aGFuCnAy
bV9hY2Nlc3NfcncgKGluIHRoZSBsYXR0ZXIgY2FzZSBqdXN0IGZvciBwMm1f
bW1pb19kaXJlY3QgcmVxdWVzdHMpLgoKTm90ZSBhbHNvIHRoYXQsIHRvIGJl
IG9uIHRoZSBzYWZlIHNpZGUsIGFuIG92ZXJsYXAgY2hlY2sgZ2V0cyBhZGRl
ZCB0bwp0aGUgbWFpbiBsb29wIG9mIGlvbW11X2lkZW50aXR5X21hcHBpbmco
KS4KClRoaXMgaXMgcGFydCBvZiBYU0EtMzc4LgoKU2lnbmVkLW9mZi1ieTog
SmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpSZXZpZXdlZC1ieTog
UGF1bCBEdXJyYW50IDxwYXVsQHhlbi5vcmc+Ci0tLQpUQkQ6IEluIHRoaXMg
Y29udGV4dCAodGhlIGxhdGVzdCkgSSBxdWVzdGlvbiB3aGV0aGVyIGl0IGlz
IGEgZ29vZCBpZGVhCiAgICAgZm9yIGNsZWFyX2lkZW50aXR5X3AybV9lbnRy
eSgpIHRvIHJldHVybiBzdWNjZXNzIHdoZW4gaXQgZGlkbid0CiAgICAgY2xl
YXIgdGhlIGVudHJ5IGJlY2F1c2Ugb2YgbWlzbWF0Y2hpbmcgdHlwZSBvciBN
Rk4uClRCRDogUXVlc3Rpb24gaXMgd2hldGhlciBwMm1fZ2V0X2lvbW11X2Zs
YWdzKCkgc2hvdWxkIGFwcGx5IHRoZSBhY2Nlc3MKICAgICByZXN0cmljdGlv
biBpbiBhbGwgY2FzZXMsIG5vdCBqdXN0IGZvciBwMm1fbW1pb19kaXJlY3Qu
ClRCRDogTmVpdGhlciBwb3NzaWJsZSBwbGFjZW1lbnQgb2YgcDJtX2FjY2Vz
c19yeDJydyBhbmQgcDJtX2FjY2Vzc19uMnJ3eAogICAgIGluIHAybV9hY2Nl
c3NfdG9faW9tbXVfZmxhZ3MoKSBsb29rcyBlbnRpcmVseSBjb3JyZWN0L3Nl
bnNpYmxlLgotLS0Kdjk6IFJlLWJhc2UuCnY3OiBzL3VuaXR5X21hcC9pZGVu
dGl0eV9tYXAvZy4gUmUtYmFzZS4KdjY6IFJlLWJhc2UuCnY1OiBOZXcuCgot
LS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2
L21tL3AybS5jCkBAIC0xNDI3LDcgKzE0MjcsNyBAQCBpbnQgc2V0X2lkZW50
aXR5X3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluCiAgICAgICAgICAgICByZXR1
cm4gMDsKICAgICAgICAgcmV0dXJuIGlvbW11X2xlZ2FjeV9tYXAoZCwgX2Rm
bihnZm5fbCksIF9tZm4oZ2ZuX2wpLAogICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAxdWwgPDwgUEFHRV9PUkRFUl80SywKLSAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgSU9NTVVGX3JlYWRhYmxlIHwgSU9NTVVG
X3dyaXRhYmxlKTsKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
cDJtX2FjY2Vzc190b19pb21tdV9mbGFncyhwMm1hKSk7CiAjaWZkZWYgQ09O
RklHX0hWTQogICAgIH0KIAotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3Vn
aC92dGQvaW9tbXUuYworKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92
dGQvaW9tbXUuYwpAQCAtNDUsMTIgKzQ1LDYgQEAKIC8qIGRvbV9pbyBpcyB1
c2VkIGFzIGEgc2VudGluZWwgZm9yIHF1YXJhbnRpbmVkIGRldmljZXMgKi8K
ICNkZWZpbmUgUVVBUkFOVElORV9TS0lQKGQpICgoZCkgPT0gZG9tX2lvICYm
ICFkb21faW9tbXUoZCktPmFyY2gudnRkLnBnZF9tYWRkcikKIAotc3RydWN0
IG1hcHBlZF9ybXJyIHsKLSAgICBzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7Ci0g
ICAgdTY0IGJhc2UsIGVuZDsKLSAgICB1bnNpZ25lZCBpbnQgY291bnQ7Ci19
OwotCiAvKiBQb3NzaWJsZSB1bmZpbHRlcmVkIExBUElDL01TSSBtZXNzYWdl
cyBmcm9tIHVudHJ1c3RlZCBzb3VyY2VzPyAqLwogYm9vbCBfX3JlYWRfbW9z
dGx5IHVudHJ1c3RlZF9tc2k7CiAKQEAgLTEzMDYsNyArMTMwMCw2IEBAIHN0
YXRpYyBpbnQgaW50ZWxfaW9tbXVfZG9tYWluX2luaXQoc3RydWMKICAgICBz
dHJ1Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRvbV9pb21tdShkKTsKIAogICAg
IGhkLT5hcmNoLnZ0ZC5hZ2F3ID0gd2lkdGhfdG9fYWdhdyhERUZBVUxUX0RP
TUFJTl9BRERSRVNTX1dJRFRIKTsKLSAgICBJTklUX0xJU1RfSEVBRCgmaGQt
PmFyY2gudnRkLm1hcHBlZF9ybXJycyk7CiAKICAgICByZXR1cm4gMDsKIH0K
QEAgLTE3ODUsMTcgKzE3NzgsMTIgQEAgc3RhdGljIHZvaWQgaW9tbXVfY2xl
YXJfcm9vdF9wZ3RhYmxlKHN0cgogc3RhdGljIHZvaWQgaW9tbXVfZG9tYWlu
X3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpCiB7CiAgICAgc3RydWN0IGRv
bWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7Ci0gICAgc3RydWN0IG1h
cHBlZF9ybXJyICptcm1yciwgKnRtcDsKICAgICBjb25zdCBzdHJ1Y3QgYWNw
aV9kcmhkX3VuaXQgKmRyaGQ7CiAKICAgICBpZiAoIGxpc3RfZW1wdHkoJmFj
cGlfZHJoZF91bml0cykgKQogICAgICAgICByZXR1cm47CiAKLSAgICBsaXN0
X2Zvcl9lYWNoX2VudHJ5X3NhZmUgKCBtcm1yciwgdG1wLCAmaGQtPmFyY2gu
dnRkLm1hcHBlZF9ybXJycywgbGlzdCApCi0gICAgewotICAgICAgICBsaXN0
X2RlbCgmbXJtcnItPmxpc3QpOwotICAgICAgICB4ZnJlZShtcm1ycik7Ci0g
ICAgfQorICAgIGlvbW11X2lkZW50aXR5X21hcF90ZWFyZG93bihkKTsKIAog
ICAgIEFTU0VSVCghaGQtPmFyY2gudnRkLnBnZF9tYWRkcik7CiAKQEAgLTE5
NDMsNzQgKzE5MzEsNiBAQCBzdGF0aWMgaW50IF9faW5pdCB2dGRfZXB0X3Bh
Z2VfY29tcGF0aWJsCiAgICAgICAgICAgIChlcHRfaGFzXzFnYihlcHRfY2Fw
KSAmJiBvcHRfaGFwXzFnYikgPD0gY2FwX3Nwc18xZ2IodnRkX2NhcCk7CiB9
CiAKLXN0YXRpYyBpbnQgcm1ycl9pZGVudGl0eV9tYXBwaW5nKHN0cnVjdCBk
b21haW4gKmQsIGJvb2xfdCBtYXAsCi0gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgYWNwaV9ybXJyX3VuaXQgKnJtcnIs
Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1MzIgZmxhZykK
LXsKLSAgICB1bnNpZ25lZCBsb25nIGJhc2VfcGZuID0gcm1yci0+YmFzZV9h
ZGRyZXNzID4+IFBBR0VfU0hJRlRfNEs7Ci0gICAgdW5zaWduZWQgbG9uZyBl
bmRfcGZuID0gUEFHRV9BTElHTl80SyhybXJyLT5lbmRfYWRkcmVzcykgPj4g
UEFHRV9TSElGVF80SzsKLSAgICBzdHJ1Y3QgbWFwcGVkX3JtcnIgKm1ybXJy
OwotICAgIHN0cnVjdCBkb21haW5faW9tbXUgKmhkID0gZG9tX2lvbW11KGQp
OwotCi0gICAgQVNTRVJUKHBjaWRldnNfbG9ja2VkKCkpOwotICAgIEFTU0VS
VChybXJyLT5iYXNlX2FkZHJlc3MgPCBybXJyLT5lbmRfYWRkcmVzcyk7Ci0K
LSAgICAvKgotICAgICAqIE5vIG5lZWQgdG8gYWNxdWlyZSBoZC0+YXJjaC5t
YXBwaW5nX2xvY2s6IEJvdGggaW5zZXJ0aW9uIGFuZCByZW1vdmFsCi0gICAg
ICogZ2V0IGRvbmUgd2hpbGUgaG9sZGluZyBwY2lkZXZzX2xvY2suCi0gICAg
ICovCi0gICAgbGlzdF9mb3JfZWFjaF9lbnRyeSggbXJtcnIsICZoZC0+YXJj
aC52dGQubWFwcGVkX3JtcnJzLCBsaXN0ICkKLSAgICB7Ci0gICAgICAgIGlm
ICggbXJtcnItPmJhc2UgPT0gcm1yci0+YmFzZV9hZGRyZXNzICYmCi0gICAg
ICAgICAgICAgbXJtcnItPmVuZCA9PSBybXJyLT5lbmRfYWRkcmVzcyApCi0g
ICAgICAgIHsKLSAgICAgICAgICAgIGludCByZXQgPSAwOwotCi0gICAgICAg
ICAgICBpZiAoIG1hcCApCi0gICAgICAgICAgICB7Ci0gICAgICAgICAgICAg
ICAgKyttcm1yci0+Y291bnQ7Ci0gICAgICAgICAgICAgICAgcmV0dXJuIDA7
Ci0gICAgICAgICAgICB9Ci0KLSAgICAgICAgICAgIGlmICggLS1tcm1yci0+
Y291bnQgKQotICAgICAgICAgICAgICAgIHJldHVybiAwOwotCi0gICAgICAg
ICAgICB3aGlsZSAoIGJhc2VfcGZuIDwgZW5kX3BmbiApCi0gICAgICAgICAg
ICB7Ci0gICAgICAgICAgICAgICAgaWYgKCBjbGVhcl9pZGVudGl0eV9wMm1f
ZW50cnkoZCwgYmFzZV9wZm4pICkKLSAgICAgICAgICAgICAgICAgICAgcmV0
ID0gLUVOWElPOwotICAgICAgICAgICAgICAgIGJhc2VfcGZuKys7Ci0gICAg
ICAgICAgICB9Ci0KLSAgICAgICAgICAgIGxpc3RfZGVsKCZtcm1yci0+bGlz
dCk7Ci0gICAgICAgICAgICB4ZnJlZShtcm1ycik7Ci0gICAgICAgICAgICBy
ZXR1cm4gcmV0OwotICAgICAgICB9Ci0gICAgfQotCi0gICAgaWYgKCAhbWFw
ICkKLSAgICAgICAgcmV0dXJuIC1FTk9FTlQ7Ci0KLSAgICB3aGlsZSAoIGJh
c2VfcGZuIDwgZW5kX3BmbiApCi0gICAgewotICAgICAgICBpbnQgZXJyID0g
c2V0X2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3BmbiwgcDJtX2FjY2Vz
c19ydywgZmxhZyk7Ci0KLSAgICAgICAgaWYgKCBlcnIgKQotICAgICAgICAg
ICAgcmV0dXJuIGVycjsKLSAgICAgICAgYmFzZV9wZm4rKzsKLSAgICB9Ci0K
LSAgICBtcm1yciA9IHhtYWxsb2Moc3RydWN0IG1hcHBlZF9ybXJyKTsKLSAg
ICBpZiAoICFtcm1yciApCi0gICAgICAgIHJldHVybiAtRU5PTUVNOwotICAg
IG1ybXJyLT5iYXNlID0gcm1yci0+YmFzZV9hZGRyZXNzOwotICAgIG1ybXJy
LT5lbmQgPSBybXJyLT5lbmRfYWRkcmVzczsKLSAgICBtcm1yci0+Y291bnQg
PSAxOwotICAgIGxpc3RfYWRkX3RhaWwoJm1ybXJyLT5saXN0LCAmaGQtPmFy
Y2gudnRkLm1hcHBlZF9ybXJycyk7Ci0KLSAgICByZXR1cm4gMDsKLX0KLQog
c3RhdGljIGludCBpbnRlbF9pb21tdV9hZGRfZGV2aWNlKHU4IGRldmZuLCBz
dHJ1Y3QgcGNpX2RldiAqcGRldikKIHsKICAgICBzdHJ1Y3QgYWNwaV9ybXJy
X3VuaXQgKnJtcnI7CkBAIC0yMDQyLDcgKzE5NjIsOSBAQCBzdGF0aWMgaW50
IGludGVsX2lvbW11X2FkZF9kZXZpY2UodTggZGV2CiAgICAgICAgICAgICAg
KiBTaW5jZSBSTVJScyBhcmUgYWx3YXlzIHJlc2VydmVkIGluIHRoZSBlODIw
IG1hcCBmb3IgdGhlIGhhcmR3YXJlCiAgICAgICAgICAgICAgKiBkb21haW4s
IHRoZXJlIHNob3VsZG4ndCBiZSBhIGNvbmZsaWN0LgogICAgICAgICAgICAg
ICovCi0gICAgICAgICAgICByZXQgPSBybXJyX2lkZW50aXR5X21hcHBpbmco
cGRldi0+ZG9tYWluLCAxLCBybXJyLCAwKTsKKyAgICAgICAgICAgIHJldCA9
IGlvbW11X2lkZW50aXR5X21hcHBpbmcocGRldi0+ZG9tYWluLCBwMm1fYWNj
ZXNzX3J3LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBybXJyLT5iYXNlX2FkZHJlc3MsIHJtcnItPmVuZF9hZGRyZXNzLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwKTsK
ICAgICAgICAgICAgIGlmICggcmV0ICkKICAgICAgICAgICAgICAgICBkcHJp
bnRrKFhFTkxPR19FUlIgVlREUFJFRklYLCAiZCVkOiBSTVJSIG1hcHBpbmcg
ZmFpbGVkXG4iLAogICAgICAgICAgICAgICAgICAgICAgICAgcGRldi0+ZG9t
YWluLT5kb21haW5faWQpOwpAQCAtMjA4Nyw3ICsyMDA5LDggQEAgc3RhdGlj
IGludCBpbnRlbF9pb21tdV9yZW1vdmVfZGV2aWNlKHU4CiAgICAgICAgICAq
IEFueSBmbGFnIGlzIG5vdGhpbmcgdG8gY2xlYXIgdGhlc2UgbWFwcGluZ3Mg
YnV0IGhlcmUKICAgICAgICAgICogaXRzIGFsd2F5cyBzYWZlIGFuZCBzdHJp
Y3QgdG8gc2V0IDAuCiAgICAgICAgICAqLwotICAgICAgICBybXJyX2lkZW50
aXR5X21hcHBpbmcocGRldi0+ZG9tYWluLCAwLCBybXJyLCAwKTsKKyAgICAg
ICAgaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhwZGV2LT5kb21haW4sIHAybV9h
Y2Nlc3NfeCwgcm1yci0+YmFzZV9hZGRyZXNzLAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCAwKTsKICAgICB9
CiAKICAgICByZXR1cm4gZG9tYWluX2NvbnRleHRfdW5tYXAocGRldi0+ZG9t
YWluLCBkZXZmbiwgcGRldik7CkBAIC0yMjg2LDcgKzIyMDksOCBAQCBzdGF0
aWMgdm9pZCBfX2h3ZG9tX2luaXQgc2V0dXBfaHdkb21fcm1yCiAgICAgICAg
ICAqIGRvbWFpbiwgdGhlcmUgc2hvdWxkbid0IGJlIGEgY29uZmxpY3QuIFNv
IGl0cyBhbHdheXMgc2FmZSBhbmQKICAgICAgICAgICogc3RyaWN0IHRvIHNl
dCAwLgogICAgICAgICAgKi8KLSAgICAgICAgcmV0ID0gcm1ycl9pZGVudGl0
eV9tYXBwaW5nKGQsIDEsIHJtcnIsIDApOworICAgICAgICByZXQgPSBpb21t
dV9pZGVudGl0eV9tYXBwaW5nKGQsIHAybV9hY2Nlc3NfcncsIHJtcnItPmJh
c2VfYWRkcmVzcywKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBybXJyLT5lbmRfYWRkcmVzcywgMCk7CiAgICAgICAgIGlmICggcmV0
ICkKICAgICAgICAgICAgIGRwcmludGsoWEVOTE9HX0VSUiBWVERQUkVGSVgs
CiAgICAgICAgICAgICAgICAgICAgICAiSU9NTVU6IG1hcHBpbmcgcmVzZXJ2
ZWQgcmVnaW9uIGZhaWxlZFxuIik7CkBAIC0yNDY4LDcgKzIzOTIsOSBAQCBz
dGF0aWMgaW50IHJlYXNzaWduX2RldmljZV9vd25lcnNoaXAoCiAgICAgICAg
ICAgICAgICAgICogQW55IFJNUlIgZmxhZyBpcyBhbHdheXMgaWdub3JlZCB3
aGVuIHJlbW92ZSBhIGRldmljZSwKICAgICAgICAgICAgICAgICAgKiBidXQg
aXRzIGFsd2F5cyBzYWZlIGFuZCBzdHJpY3QgdG8gc2V0IDAuCiAgICAgICAg
ICAgICAgICAgICovCi0gICAgICAgICAgICAgICAgcmV0ID0gcm1ycl9pZGVu
dGl0eV9tYXBwaW5nKHNvdXJjZSwgMCwgcm1yciwgMCk7CisgICAgICAgICAg
ICAgICAgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhzb3VyY2UsIHAy
bV9hY2Nlc3NfeCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHJtcnItPmJhc2VfYWRkcmVzcywKKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmVuZF9h
ZGRyZXNzLCAwKTsKICAgICAgICAgICAgICAgICBpZiAoIHJldCAhPSAtRU5P
RU5UICkKICAgICAgICAgICAgICAgICAgICAgcmV0dXJuIHJldDsKICAgICAg
ICAgICAgIH0KQEAgLTI1NjQsNyArMjQ5MCw4IEBAIHN0YXRpYyBpbnQgaW50
ZWxfaW9tbXVfYXNzaWduX2RldmljZSgKICAgICAgICAgICAgICBQQ0lfQlVT
KGJkZikgPT0gYnVzICYmCiAgICAgICAgICAgICAgUENJX0RFVkZOMihiZGYp
ID09IGRldmZuICkKICAgICAgICAgewotICAgICAgICAgICAgcmV0ID0gcm1y
cl9pZGVudGl0eV9tYXBwaW5nKGQsIDEsIHJtcnIsIGZsYWcpOworICAgICAg
ICAgICAgcmV0ID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhkLCBwMm1fYWNj
ZXNzX3J3LCBybXJyLT5iYXNlX2FkZHJlc3MsCisgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHJtcnItPmVuZF9hZGRyZXNzLCBm
bGFnKTsKICAgICAgICAgICAgIGlmICggcmV0ICkKICAgICAgICAgICAgIHsK
ICAgICAgICAgICAgICAgICBpbnQgcmM7Ci0tLSBhL3hlbi9kcml2ZXJzL3Bh
c3N0aHJvdWdoL3g4Ni9pb21tdS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0
aHJvdWdoL3g4Ni9pb21tdS5jCkBAIC0xNDQsNiArMTQ0LDcgQEAgaW50IGFy
Y2hfaW9tbXVfZG9tYWluX2luaXQoc3RydWN0IGRvbWFpbgogCiAgICAgSU5J
VF9QQUdFX0xJU1RfSEVBRCgmaGQtPmFyY2gucGd0YWJsZXMubGlzdCk7CiAg
ICAgc3Bpbl9sb2NrX2luaXQoJmhkLT5hcmNoLnBndGFibGVzLmxvY2spOwor
ICAgIElOSVRfTElTVF9IRUFEKCZoZC0+YXJjaC5pZGVudGl0eV9tYXBzKTsK
IAogICAgIHJldHVybiAwOwogfQpAQCAtMTU5LDYgKzE2MCw5OSBAQCB2b2lk
IGFyY2hfaW9tbXVfZG9tYWluX2Rlc3Ryb3koc3RydWN0IGRvCiAgICAgICAg
ICAgIHBhZ2VfbGlzdF9lbXB0eSgmZG9tX2lvbW11KGQpLT5hcmNoLnBndGFi
bGVzLmxpc3QpKTsKIH0KIAorc3RydWN0IGlkZW50aXR5X21hcCB7CisgICAg
c3RydWN0IGxpc3RfaGVhZCBsaXN0OworICAgIHBhZGRyX3QgYmFzZSwgZW5k
OworICAgIHAybV9hY2Nlc3NfdCBhY2Nlc3M7CisgICAgdW5zaWduZWQgaW50
IGNvdW50OworfTsKKworaW50IGlvbW11X2lkZW50aXR5X21hcHBpbmcoc3Ry
dWN0IGRvbWFpbiAqZCwgcDJtX2FjY2Vzc190IHAybWEsCisgICAgICAgICAg
ICAgICAgICAgICAgICAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgZW5kLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGZsYWcp
Cit7CisgICAgdW5zaWduZWQgbG9uZyBiYXNlX3BmbiA9IGJhc2UgPj4gUEFH
RV9TSElGVF80SzsKKyAgICB1bnNpZ25lZCBsb25nIGVuZF9wZm4gPSBQQUdF
X0FMSUdOXzRLKGVuZCkgPj4gUEFHRV9TSElGVF80SzsKKyAgICBzdHJ1Y3Qg
aWRlbnRpdHlfbWFwICptYXA7CisgICAgc3RydWN0IGRvbWFpbl9pb21tdSAq
aGQgPSBkb21faW9tbXUoZCk7CisKKyAgICBBU1NFUlQocGNpZGV2c19sb2Nr
ZWQoKSk7CisgICAgQVNTRVJUKGJhc2UgPCBlbmQpOworCisgICAgLyoKKyAg
ICAgKiBObyBuZWVkIHRvIGFjcXVpcmUgaGQtPmFyY2gubWFwcGluZ19sb2Nr
OiBCb3RoIGluc2VydGlvbiBhbmQgcmVtb3ZhbAorICAgICAqIGdldCBkb25l
IHdoaWxlIGhvbGRpbmcgcGNpZGV2c19sb2NrLgorICAgICAqLworICAgIGxp
c3RfZm9yX2VhY2hfZW50cnkoIG1hcCwgJmhkLT5hcmNoLmlkZW50aXR5X21h
cHMsIGxpc3QgKQorICAgIHsKKyAgICAgICAgaWYgKCBtYXAtPmJhc2UgPT0g
YmFzZSAmJiBtYXAtPmVuZCA9PSBlbmQgKQorICAgICAgICB7CisgICAgICAg
ICAgICBpbnQgcmV0ID0gMDsKKworICAgICAgICAgICAgaWYgKCBwMm1hICE9
IHAybV9hY2Nlc3NfeCApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAg
ICAgaWYgKCBtYXAtPmFjY2VzcyAhPSBwMm1hICkKKyAgICAgICAgICAgICAg
ICAgICAgcmV0dXJuIC1FQUREUklOVVNFOworICAgICAgICAgICAgICAgICsr
bWFwLT5jb3VudDsKKyAgICAgICAgICAgICAgICByZXR1cm4gMDsKKyAgICAg
ICAgICAgIH0KKworICAgICAgICAgICAgaWYgKCAtLW1hcC0+Y291bnQgKQor
ICAgICAgICAgICAgICAgIHJldHVybiAwOworCisgICAgICAgICAgICB3aGls
ZSAoIGJhc2VfcGZuIDwgZW5kX3BmbiApCisgICAgICAgICAgICB7CisgICAg
ICAgICAgICAgICAgaWYgKCBjbGVhcl9pZGVudGl0eV9wMm1fZW50cnkoZCwg
YmFzZV9wZm4pICkKKyAgICAgICAgICAgICAgICAgICAgcmV0ID0gLUVOWElP
OworICAgICAgICAgICAgICAgIGJhc2VfcGZuKys7CisgICAgICAgICAgICB9
CisKKyAgICAgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3QpOworICAgICAg
ICAgICAgeGZyZWUobWFwKTsKKworICAgICAgICAgICAgcmV0dXJuIHJldDsK
KyAgICAgICAgfQorCisgICAgICAgIGlmICggZW5kID49IG1hcC0+YmFzZSAm
JiBtYXAtPmVuZCA+PSBiYXNlICkKKyAgICAgICAgICAgIHJldHVybiAtRUFE
RFJJTlVTRTsKKyAgICB9CisKKyAgICBpZiAoIHAybWEgPT0gcDJtX2FjY2Vz
c194ICkKKyAgICAgICAgcmV0dXJuIC1FTk9FTlQ7CisKKyAgICB3aGlsZSAo
IGJhc2VfcGZuIDwgZW5kX3BmbiApCisgICAgeworICAgICAgICBpbnQgZXJy
ID0gc2V0X2lkZW50aXR5X3AybV9lbnRyeShkLCBiYXNlX3BmbiwgcDJtYSwg
ZmxhZyk7CisKKyAgICAgICAgaWYgKCBlcnIgKQorICAgICAgICAgICAgcmV0
dXJuIGVycjsKKyAgICAgICAgYmFzZV9wZm4rKzsKKyAgICB9CisKKyAgICBt
YXAgPSB4bWFsbG9jKHN0cnVjdCBpZGVudGl0eV9tYXApOworICAgIGlmICgg
IW1hcCApCisgICAgICAgIHJldHVybiAtRU5PTUVNOworICAgIG1hcC0+YmFz
ZSA9IGJhc2U7CisgICAgbWFwLT5lbmQgPSBlbmQ7CisgICAgbWFwLT5hY2Nl
c3MgPSBwMm1hOworICAgIG1hcC0+Y291bnQgPSAxOworICAgIGxpc3RfYWRk
X3RhaWwoJm1hcC0+bGlzdCwgJmhkLT5hcmNoLmlkZW50aXR5X21hcHMpOwor
CisgICAgcmV0dXJuIDA7Cit9CisKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFw
X3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpCit7CisgICAgc3RydWN0IGRv
bWFpbl9pb21tdSAqaGQgPSBkb21faW9tbXUoZCk7CisgICAgc3RydWN0IGlk
ZW50aXR5X21hcCAqbWFwLCAqdG1wOworCisgICAgbGlzdF9mb3JfZWFjaF9l
bnRyeV9zYWZlICggbWFwLCB0bXAsICZoZC0+YXJjaC5pZGVudGl0eV9tYXBz
LCBsaXN0ICkKKyAgICB7CisgICAgICAgIGxpc3RfZGVsKCZtYXAtPmxpc3Qp
OworICAgICAgICB4ZnJlZShtYXApOworICAgIH0KK30KKwogc3RhdGljIGJv
b2wgX19od2RvbV9pbml0IGh3ZG9tX2lvbW11X21hcChjb25zdCBzdHJ1Y3Qg
ZG9tYWluICpkLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBsb25nIHBmbiwKICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBtYXhfcGZu
KQotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L2lvbW11LmgKKysrIGIveGVu
L2luY2x1ZGUvYXNtLXg4Ni9pb21tdS5oCkBAIC0xNiw2ICsxNiw3IEBACiAK
ICNpbmNsdWRlIDx4ZW4vZXJybm8uaD4KICNpbmNsdWRlIDx4ZW4vbGlzdC5o
PgorI2luY2x1ZGUgPHhlbi9tZW1fYWNjZXNzLmg+CiAjaW5jbHVkZSA8eGVu
L3NwaW5sb2NrLmg+CiAjaW5jbHVkZSA8YXNtL2FwaWNkZWYuaD4KICNpbmNs
dWRlIDxhc20vcHJvY2Vzc29yLmg+CkBAIC01MCwxMyArNTEsMTQgQEAgc3Ry
dWN0IGFyY2hfaW9tbXUKICAgICAgICAgc3BpbmxvY2tfdCBsb2NrOwogICAg
IH0gcGd0YWJsZXM7CiAKKyAgICBzdHJ1Y3QgbGlzdF9oZWFkIGlkZW50aXR5
X21hcHM7CisKICAgICB1bmlvbiB7CiAgICAgICAgIC8qIEludGVsIFZULWQg
Ki8KICAgICAgICAgc3RydWN0IHsKICAgICAgICAgICAgIHVpbnQ2NF90IHBn
ZF9tYWRkcjsgLyogaW8gcGFnZSBkaXJlY3RvcnkgbWFjaGluZSBhZGRyZXNz
ICovCiAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgYWdhdzsgLyogYWRqdXN0
ZWQgZ3Vlc3QgYWRkcmVzcyB3aWR0aCwgMCBpcyBsZXZlbCAyIDMwLWJpdCAq
LwogICAgICAgICAgICAgdWludDY0X3QgaW9tbXVfYml0bWFwOyAvKiBiaXRt
YXAgb2YgaW9tbXUocykgdGhhdCB0aGUgZG9tYWluIHVzZXMgKi8KLSAgICAg
ICAgICAgIHN0cnVjdCBsaXN0X2hlYWQgbWFwcGVkX3JtcnJzOwogICAgICAg
ICB9IHZ0ZDsKICAgICAgICAgLyogQU1EIElPTU1VICovCiAgICAgICAgIHN0
cnVjdCB7CkBAIC0xMjIsNiArMTI0LDExIEBAIHN0YXRpYyBpbmxpbmUgdm9p
ZCBpb21tdV9kaXNhYmxlX3gyYXBpYygKICAgICAgICAgaW9tbXVfb3BzLmRp
c2FibGVfeDJhcGljKCk7CiB9CiAKK2ludCBpb21tdV9pZGVudGl0eV9tYXBw
aW5nKHN0cnVjdCBkb21haW4gKmQsIHAybV9hY2Nlc3NfdCBwMm1hLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgcGFkZHJfdCBiYXNlLCBwYWRkcl90
IGVuZCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGlu
dCBmbGFnKTsKK3ZvaWQgaW9tbXVfaWRlbnRpdHlfbWFwX3RlYXJkb3duKHN0
cnVjdCBkb21haW4gKmQpOworCiBleHRlcm4gYm9vbCB1bnRydXN0ZWRfbXNp
OwogCiBpbnQgcGlfdXBkYXRlX2lydGUoY29uc3Qgc3RydWN0IHBpX2Rlc2Mg
KnBpX2Rlc2MsIGNvbnN0IHN0cnVjdCBwaXJxICpwaXJxLAotLS0gYS94ZW4v
aW5jbHVkZS9hc20teDg2L3AybS5oCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14
ODYvcDJtLmgKQEAgLTg4OCw2ICs4ODgsMzQgQEAgc3RydWN0IHAybV9kb21h
aW4gKnAybV9nZXRfYWx0cDJtKHN0cnVjdAogc3RhdGljIGlubGluZSB2b2lk
IHAybV9hbHRwMm1fY2hlY2soc3RydWN0IHZjcHUgKnYsIHVpbnQxNl90IGlk
eCkge30KICNlbmRpZgogCisvKiBwMm0gYWNjZXNzIHRvIElPTU1VIGZsYWdz
ICovCitzdGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBwMm1fYWNjZXNzX3Rv
X2lvbW11X2ZsYWdzKHAybV9hY2Nlc3NfdCBwMm1hKQoreworICAgIHN3aXRj
aCAoIHAybWEgKQorICAgIHsKKyAgICBjYXNlIHAybV9hY2Nlc3Nfcnc6Cisg
ICAgY2FzZSBwMm1fYWNjZXNzX3J3eDoKKyAgICAgICAgcmV0dXJuIElPTU1V
Rl9yZWFkYWJsZSB8IElPTU1VRl93cml0YWJsZTsKKworICAgIGNhc2UgcDJt
X2FjY2Vzc19yOgorICAgIGNhc2UgcDJtX2FjY2Vzc19yeDoKKyAgICBjYXNl
IHAybV9hY2Nlc3Nfcngycnc6CisgICAgICAgIHJldHVybiBJT01NVUZfcmVh
ZGFibGU7CisKKyAgICBjYXNlIHAybV9hY2Nlc3NfdzoKKyAgICBjYXNlIHAy
bV9hY2Nlc3Nfd3g6CisgICAgICAgIHJldHVybiBJT01NVUZfd3JpdGFibGU7
CisKKyAgICBjYXNlIHAybV9hY2Nlc3NfbjoKKyAgICBjYXNlIHAybV9hY2Nl
c3NfeDoKKyAgICBjYXNlIHAybV9hY2Nlc3NfbjJyd3g6CisgICAgICAgIHJl
dHVybiAwOworICAgIH0KKworICAgIEFTU0VSVF9VTlJFQUNIQUJMRSgpOwor
ICAgIHJldHVybiAwOworfQorCiAvKgogICogcDJtIHR5cGUgdG8gSU9NTVUg
ZmxhZ3MKICAqLwpAQCAtOTA5LDkgKzkzNywxMCBAQCBzdGF0aWMgaW5saW5l
IHVuc2lnbmVkIGludCBwMm1fZ2V0X2lvbW11CiAgICAgICAgIGZsYWdzID0g
SU9NTVVGX3JlYWRhYmxlOwogICAgICAgICBicmVhazsKICAgICBjYXNlIHAy
bV9tbWlvX2RpcmVjdDoKLSAgICAgICAgZmxhZ3MgPSBJT01NVUZfcmVhZGFi
bGU7Ci0gICAgICAgIGlmICggIXJhbmdlc2V0X2NvbnRhaW5zX3NpbmdsZXRv
bihtbWlvX3JvX3JhbmdlcywgbWZuX3gobWZuKSkgKQotICAgICAgICAgICAg
ZmxhZ3MgfD0gSU9NTVVGX3dyaXRhYmxlOworICAgICAgICBmbGFncyA9IHAy
bV9hY2Nlc3NfdG9faW9tbXVfZmxhZ3MocDJtYSk7CisgICAgICAgIGlmICgg
KGZsYWdzICYgSU9NTVVGX3dyaXRhYmxlKSAmJgorICAgICAgICAgICAgIHJh
bmdlc2V0X2NvbnRhaW5zX3NpbmdsZXRvbihtbWlvX3JvX3JhbmdlcywgbWZu
X3gobWZuKSkgKQorICAgICAgICAgICAgZmxhZ3MgJj0gfklPTU1VRl93cml0
YWJsZTsKICAgICAgICAgYnJlYWs7CiAgICAgZGVmYXVsdDoKICAgICAgICAg
ZmxhZ3MgPSAwOwo=

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-5.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-5.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UvY29tcGxldGUgcmUtYXNzaWdubWVu
dCBoYW5kbGluZwoKUHJpb3IgdG8gdGhlIGFzc2lnbm1lbnQgc3RlcCBoYXZp
bmcgY29tcGxldGVkIHN1Y2Nlc3NmdWxseSwgZGV2aWNlcwpzaG91bGQgbm90
IGdldCBhc3NvY2lhdGVkIHdpdGggdGhlaXIgbmV3IG93bmVyLiBIYW5kIHRo
ZSBkZXZpY2UgdG8gRG9tSU8KKHBlcmhhcHMgdGVtcG9yYXJpbHkpLCB1bnRp
bCBhZnRlciB0aGUgZGUtYXNzaWdubWVudCBzdGVwIGhhcyBjb21wbGV0ZWQu
CgpEZS1hc3NpZ25tZW50IG9mIGEgZGV2aWNlIChmcm9tIG90aGVyIHRoYW4g
RG9tMCkgYXMgd2VsbCBhcyBmYWlsdXJlIG9mCnJlYXNzaWduX2RldmljZSgp
IGR1cmluZyBhc3NpZ25tZW50IHNob3VsZCByZXN1bHQgaW4gdW5pdHkgbWFw
cGluZ3MKZ2V0dGluZyB0b3JuIGRvd24uIFRoaXMgaW4gdHVybiByZXF1aXJl
cyBzd2l0Y2hpbmcgdG8gYSByZWZjb3VudGVkCm1hcHBpbmcgYXBwcm9hY2gs
IGFzIHdhcyBhbHJlYWR5IHVzZWQgYnkgVlQtZCBmb3IgaXRzIFJNUlJzLCB0
byBwcmV2ZW50CnVubWFwcGluZyBhIHJlZ2lvbiB1c2VkIGJ5IG11bHRpcGxl
IGRldmljZXMuCgpUaGlzIGlzIENWRS0yMDIxLTI4Njk2IC8gcGFydCBvZiBY
U0EtMzc4LgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPgpSZXZpZXdlZC1ieTogUGF1bCBEdXJyYW50IDxwYXVsQHhl
bi5vcmc+Ci0tLQp2OTogUmUtYmFzZS4Kdjc6IFJlLWJhc2UuCnY2OiBSZS1i
YXNlLgp2NTogU3dpdGNoIHRvIHVzaW5nIGlvbW11X2lkZW50aXR5X21hcHBp
bmcoKS4gQWxzbyB1bm1hcCgpIHdoZW4gYW55CiAgICBtYXAoKSBmYWlscyBp
biBhbWRfaW9tbXVfYXNzaWduX2RldmljZSgpLCB1bmxlc3MgaXQncyBEb20w
LiBSZS1iYXNlLgp2NDogVXNlIGRhZGRyX3RvX2RmbigpLiBEcm9wIHN0cmF5
IGJsYW5rIGxpbmUuIERvbid0IGV2ZXIgdW5tYXAgZnJvbQogICAgRG9tMC4K
djM6IFVzZSBEb21JTyBpbnN0ZWFkIG9mIERvbVhFTi4KdjI6IFJlLWJhc2Ug
b3ZlciBYU0EtMzAyLgoKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gv
YW1kL2lvbW11LmgKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1k
L2lvbW11LmgKQEAgLTIzMiw4ICsyMzIsMTAgQEAgaW50IF9fbXVzdF9jaGVj
ayBhbWRfaW9tbXVfdW5tYXBfcGFnZShzdAogICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgKmZsdXNoX2ZsYWdz
KTsKIGludCBfX211c3RfY2hlY2sgYW1kX2lvbW11X2FsbG9jX3Jvb3Qoc3Ry
dWN0IGRvbWFpbiAqZCk7CiBpbnQgYW1kX2lvbW11X3Jlc2VydmVfZG9tYWlu
X3VuaXR5X21hcChzdHJ1Y3QgZG9tYWluICpkb21haW4sCi0gICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYWRkcl90IHBoeXNfYWRk
ciwgdW5zaWduZWQgbG9uZyBzaXplLAotICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgaW50IGl3LCBpbnQgaXIpOworICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGl2
cnNfdW5pdHlfbWFwICptYXAsCisgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgZmxhZyk7CitpbnQgYW1kX2lv
bW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X3VubWFwKHN0cnVjdCBkb21haW4g
KmQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqbWFwKTsKIGludCBfX211
c3RfY2hlY2sgYW1kX2lvbW11X2ZsdXNoX2lvdGxiX3BhZ2VzKHN0cnVjdCBk
b21haW4gKmQsIGRmbl90IGRmbiwKICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgcGFnZV9jb3Vu
dCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBmbHVzaF9mbGFncyk7Ci0tLSBhL3hlbi9kcml2
ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdV9tYXAuYworKysgYi94ZW4vZHJp
dmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXVfbWFwLmMKQEAgLTQxOSwzOCAr
NDE5LDQ5IEBAIGludCBhbWRfaW9tbXVfZmx1c2hfaW90bGJfYWxsKHN0cnVj
dCBkb20KICAgICByZXR1cm4gMDsKIH0KIAotaW50IGFtZF9pb21tdV9yZXNl
cnZlX2RvbWFpbl91bml0eV9tYXAoc3RydWN0IGRvbWFpbiAqZG9tYWluLAot
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGFkZHJf
dCBwaHlzX2FkZHIsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBsb25nIHNpemUsIGludCBpdywgaW50IGlyKQor
aW50IGFtZF9pb21tdV9yZXNlcnZlX2RvbWFpbl91bml0eV9tYXAoc3RydWN0
IGRvbWFpbiAqZCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIGNvbnN0IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqbWFwLAorICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
aW50IGZsYWcpCiB7Ci0gICAgdW5zaWduZWQgbG9uZyBucGFnZXMsIGk7Ci0g
ICAgdW5zaWduZWQgbG9uZyBnZm47Ci0gICAgdW5zaWduZWQgaW50IGZsYWdz
ID0gISFpcjsKLSAgICB1bnNpZ25lZCBpbnQgZmx1c2hfZmxhZ3MgPSAwOwot
ICAgIGludCBydCA9IDA7Ci0KLSAgICBpZiAoIGl3ICkKLSAgICAgICAgZmxh
Z3MgfD0gSU9NTVVGX3dyaXRhYmxlOwotCi0gICAgbnBhZ2VzID0gcmVnaW9u
X3RvX3BhZ2VzKHBoeXNfYWRkciwgc2l6ZSk7Ci0gICAgZ2ZuID0gcGh5c19h
ZGRyID4+IFBBR0VfU0hJRlQ7Ci0gICAgZm9yICggaSA9IDA7IGkgPCBucGFn
ZXM7IGkrKyApCisgICAgaW50IHJjOworCisgICAgaWYgKCBkID09IGRvbV9p
byApCisgICAgICAgIHJldHVybiAwOworCisgICAgZm9yICggcmMgPSAwOyAh
cmMgJiYgbWFwOyBtYXAgPSBtYXAtPm5leHQgKQogICAgIHsKLSAgICAgICAg
dW5zaWduZWQgbG9uZyBmcmFtZSA9IGdmbiArIGk7CisgICAgICAgIHAybV9h
Y2Nlc3NfdCBwMm1hID0gcDJtX2FjY2Vzc19uOworCisgICAgICAgIGlmICgg
bWFwLT5yZWFkICkKKyAgICAgICAgICAgIHAybWEgfD0gcDJtX2FjY2Vzc19y
OworICAgICAgICBpZiAoIG1hcC0+d3JpdGUgKQorICAgICAgICAgICAgcDJt
YSB8PSBwMm1fYWNjZXNzX3c7CiAKLSAgICAgICAgcnQgPSBhbWRfaW9tbXVf
bWFwX3BhZ2UoZG9tYWluLCBfZGZuKGZyYW1lKSwgX21mbihmcmFtZSksIGZs
YWdzLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmZmx1c2hf
ZmxhZ3MpOwotICAgICAgICBpZiAoIHJ0ICE9IDAgKQotICAgICAgICAgICAg
YnJlYWs7CisgICAgICAgIHJjID0gaW9tbXVfaWRlbnRpdHlfbWFwcGluZyhk
LCBwMm1hLCBtYXAtPmFkZHIsCisgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBtYXAtPmFkZHIgKyBtYXAtPmxlbmd0aCAtIDEsIGZsYWcp
OwogICAgIH0KIAotICAgIC8qIFVzZSB3aGlsZS1icmVhayB0byBhdm9pZCBj
b21waWxlciB3YXJuaW5nICovCi0gICAgd2hpbGUgKCBmbHVzaF9mbGFncyAm
JgotICAgICAgICAgICAgYW1kX2lvbW11X2ZsdXNoX2lvdGxiX3BhZ2VzKGRv
bWFpbiwgX2RmbihnZm4pLAotICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIG5wYWdlcywgZmx1c2hfZmxhZ3MpICkKLSAgICAgICAg
YnJlYWs7CisgICAgcmV0dXJuIHJjOworfQorCitpbnQgYW1kX2lvbW11X3Jl
c2VydmVfZG9tYWluX3VuaXR5X3VubWFwKHN0cnVjdCBkb21haW4gKmQsCisg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0
IHN0cnVjdCBpdnJzX3VuaXR5X21hcCAqbWFwKQoreworICAgIGludCByYzsK
KworICAgIGlmICggZCA9PSBkb21faW8gKQorICAgICAgICByZXR1cm4gMDsK
KworICAgIGZvciAoIHJjID0gMDsgbWFwOyBtYXAgPSBtYXAtPm5leHQgKQor
ICAgIHsKKyAgICAgICAgaW50IHJldCA9IGlvbW11X2lkZW50aXR5X21hcHBp
bmcoZCwgcDJtX2FjY2Vzc194LCBtYXAtPmFkZHIsCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1hcC0+YWRkciArIG1hcC0+
bGVuZ3RoIC0gMSwgMCk7CisKKyAgICAgICAgaWYgKCByZXQgJiYgcmV0ICE9
IC1FTk9FTlQgJiYgIXJjICkKKyAgICAgICAgICAgIHJjID0gcmV0OworICAg
IH0KIAotICAgIHJldHVybiBydDsKKyAgICByZXR1cm4gcmM7CiB9CiAKIGlu
dCBfX2luaXQgYW1kX2lvbW11X3F1YXJhbnRpbmVfaW5pdChzdHJ1Y3QgZG9t
YWluICpkKQotLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNp
X2FtZF9pb21tdS5jCisrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Ft
ZC9wY2lfYW1kX2lvbW11LmMKQEAgLTM0Nyw2ICszNDcsNyBAQCBzdGF0aWMg
aW50IHJlYXNzaWduX2RldmljZShzdHJ1Y3QgZG9tYWluCiB7CiAgICAgc3Ry
dWN0IGFtZF9pb21tdSAqaW9tbXU7CiAgICAgaW50IGJkZiwgcmM7CisgICAg
Y29uc3Qgc3RydWN0IGl2cnNfbWFwcGluZ3MgKml2cnNfbWFwcGluZ3MgPSBn
ZXRfaXZyc19tYXBwaW5ncyhwZGV2LT5zZWcpOwogCiAgICAgYmRmID0gUENJ
X0JERjIocGRldi0+YnVzLCBwZGV2LT5kZXZmbik7CiAgICAgaW9tbXUgPSBm
aW5kX2lvbW11X2Zvcl9kZXZpY2UocGRldi0+c2VnLCBiZGYpOwpAQCAtMzYx
LDEwICszNjIsMjQgQEAgc3RhdGljIGludCByZWFzc2lnbl9kZXZpY2Uoc3Ry
dWN0IGRvbWFpbgogCiAgICAgYW1kX2lvbW11X2Rpc2FibGVfZG9tYWluX2Rl
dmljZShzb3VyY2UsIGlvbW11LCBkZXZmbiwgcGRldik7CiAKLSAgICBpZiAo
IGRldmZuID09IHBkZXYtPmRldmZuICkKKyAgICAvKgorICAgICAqIElmIHRo
ZSBkZXZpY2UgYmVsb25ncyB0byB0aGUgaGFyZHdhcmUgZG9tYWluLCBhbmQg
aXQgaGFzIGEgdW5pdHkgbWFwcGluZywKKyAgICAgKiBkb24ndCByZW1vdmUg
aXQgZnJvbSB0aGUgaGFyZHdhcmUgZG9tYWluLCBiZWNhdXNlIEJJT1MgbWF5
IHJlZmVyZW5jZSB0aGF0CisgICAgICogbWFwcGluZy4KKyAgICAgKi8KKyAg
ICBpZiAoICFpc19oYXJkd2FyZV9kb21haW4oc291cmNlKSApCiAgICAgewot
ICAgICAgICBsaXN0X21vdmUoJnBkZXYtPmRvbWFpbl9saXN0LCAmdGFyZ2V0
LT5wZGV2X2xpc3QpOwotICAgICAgICBwZGV2LT5kb21haW4gPSB0YXJnZXQ7
CisgICAgICAgIHJjID0gYW1kX2lvbW11X3Jlc2VydmVfZG9tYWluX3VuaXR5
X3VubWFwKAorICAgICAgICAgICAgICAgICBzb3VyY2UsCisgICAgICAgICAg
ICAgICAgIGl2cnNfbWFwcGluZ3NbZ2V0X2RtYV9yZXF1ZXN0b3JfaWQocGRl
di0+c2VnLCBiZGYpXS51bml0eV9tYXApOworICAgICAgICBpZiAoIHJjICkK
KyAgICAgICAgICAgIHJldHVybiByYzsKKyAgICB9CisKKyAgICBpZiAoIGRl
dmZuID09IHBkZXYtPmRldmZuICYmIHBkZXYtPmRvbWFpbiAhPSBkb21faW8g
KQorICAgIHsKKyAgICAgICAgbGlzdF9tb3ZlKCZwZGV2LT5kb21haW5fbGlz
dCwgJmRvbV9pby0+cGRldl9saXN0KTsKKyAgICAgICAgcGRldi0+ZG9tYWlu
ID0gZG9tX2lvOwogICAgIH0KIAogICAgIHJjID0gYW1kX2lvbW11X3NldHVw
X2RvbWFpbl9kZXZpY2UodGFyZ2V0LCBpb21tdSwgZGV2Zm4sIHBkZXYpOwpA
QCAtMzc0LDYgKzM4OSwxMiBAQCBzdGF0aWMgaW50IHJlYXNzaWduX2Rldmlj
ZShzdHJ1Y3QgZG9tYWluCiAgICAgQU1EX0lPTU1VX0RFQlVHKCJSZS1hc3Np
Z24gJXBwIGZyb20gZG9tJWQgdG8gZG9tJWRcbiIsCiAgICAgICAgICAgICAg
ICAgICAgICZwZGV2LT5zYmRmLCBzb3VyY2UtPmRvbWFpbl9pZCwgdGFyZ2V0
LT5kb21haW5faWQpOwogCisgICAgaWYgKCBkZXZmbiA9PSBwZGV2LT5kZXZm
biAmJiBwZGV2LT5kb21haW4gIT0gdGFyZ2V0ICkKKyAgICB7CisgICAgICAg
IGxpc3RfbW92ZSgmcGRldi0+ZG9tYWluX2xpc3QsICZ0YXJnZXQtPnBkZXZf
bGlzdCk7CisgICAgICAgIHBkZXYtPmRvbWFpbiA9IHRhcmdldDsKKyAgICB9
CisKICAgICByZXR1cm4gMDsKIH0KIApAQCAtMzg0LDIwICs0MDUsMjggQEAg
c3RhdGljIGludCBhbWRfaW9tbXVfYXNzaWduX2RldmljZShzdHJ1YwogICAg
IHN0cnVjdCBpdnJzX21hcHBpbmdzICppdnJzX21hcHBpbmdzID0gZ2V0X2l2
cnNfbWFwcGluZ3MocGRldi0+c2VnKTsKICAgICBpbnQgYmRmID0gUENJX0JE
RjIocGRldi0+YnVzLCBkZXZmbik7CiAgICAgaW50IHJlcV9pZCA9IGdldF9k
bWFfcmVxdWVzdG9yX2lkKHBkZXYtPnNlZywgYmRmKTsKLSAgICBjb25zdCBz
dHJ1Y3QgaXZyc191bml0eV9tYXAgKnVuaXR5X21hcDsKKyAgICBpbnQgcmMg
PSBhbWRfaW9tbXVfcmVzZXJ2ZV9kb21haW5fdW5pdHlfbWFwKAorICAgICAg
ICAgICAgICAgICBkLCBpdnJzX21hcHBpbmdzW3JlcV9pZF0udW5pdHlfbWFw
LCBmbGFnKTsKIAotICAgIGZvciAoIHVuaXR5X21hcCA9IGl2cnNfbWFwcGlu
Z3NbcmVxX2lkXS51bml0eV9tYXA7IHVuaXR5X21hcDsKLSAgICAgICAgICB1
bml0eV9tYXAgPSB1bml0eV9tYXAtPm5leHQgKQorICAgIGlmICggIXJjICkK
KyAgICAgICAgcmMgPSByZWFzc2lnbl9kZXZpY2UocGRldi0+ZG9tYWluLCBk
LCBkZXZmbiwgcGRldik7CisKKyAgICBpZiAoIHJjICYmICFpc19oYXJkd2Fy
ZV9kb21haW4oZCkgKQogICAgIHsKLSAgICAgICAgaW50IHJjID0gYW1kX2lv
bW11X3Jlc2VydmVfZG9tYWluX3VuaXR5X21hcCgKLSAgICAgICAgICAgICAg
ICAgICAgIGQsIHVuaXR5X21hcC0+YWRkciwgdW5pdHlfbWFwLT5sZW5ndGgs
Ci0gICAgICAgICAgICAgICAgICAgICB1bml0eV9tYXAtPndyaXRlLCB1bml0
eV9tYXAtPnJlYWQpOworICAgICAgICBpbnQgcmV0ID0gYW1kX2lvbW11X3Jl
c2VydmVfZG9tYWluX3VuaXR5X3VubWFwKAorICAgICAgICAgICAgICAgICAg
ICAgIGQsIGl2cnNfbWFwcGluZ3NbcmVxX2lkXS51bml0eV9tYXApOwogCi0g
ICAgICAgIGlmICggcmMgKQotICAgICAgICAgICAgcmV0dXJuIHJjOworICAg
ICAgICBpZiAoIHJldCApCisgICAgICAgIHsKKyAgICAgICAgICAgIHByaW50
ayhYRU5MT0dfRVJSICJBTUQtVmk6ICIKKyAgICAgICAgICAgICAgICAgICAi
dW5pdHktdW5tYXAgZm9yICVwZC8lMDR4OiUwMng6JTAyeC4ldSBmYWlsZWQg
KCVkKVxuIiwKKyAgICAgICAgICAgICAgICAgICBkLCBwZGV2LT5zZWcsIHBk
ZXYtPmJ1cywKKyAgICAgICAgICAgICAgICAgICBQQ0lfU0xPVChkZXZmbiks
IFBDSV9GVU5DKGRldmZuKSwgcmV0KTsKKyAgICAgICAgICAgIGRvbWFpbl9j
cmFzaChkKTsKKyAgICAgICAgfQogICAgIH0KIAotICAgIHJldHVybiByZWFz
c2lnbl9kZXZpY2UocGRldi0+ZG9tYWluLCBkLCBkZXZmbiwgcGRldik7Cisg
ICAgcmV0dXJuIHJjOwogfQogCiBzdGF0aWMgdm9pZCBhbWRfaW9tbXVfY2xl
YXJfcm9vdF9wZ3RhYmxlKHN0cnVjdCBkb21haW4gKmQpCkBAIC00MTEsNiAr
NDQwLDcgQEAgc3RhdGljIHZvaWQgYW1kX2lvbW11X2NsZWFyX3Jvb3RfcGd0
YWJsZQogCiBzdGF0aWMgdm9pZCBhbWRfaW9tbXVfZG9tYWluX2Rlc3Ryb3ko
c3RydWN0IGRvbWFpbiAqZCkKIHsKKyAgICBpb21tdV9pZGVudGl0eV9tYXBf
dGVhcmRvd24oZCk7CiAgICAgQVNTRVJUKCFkb21faW9tbXUoZCktPmFyY2gu
YW1kLnJvb3RfdGFibGUpOwogfQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-6.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-6.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBBTUQvSU9NTVU6IHJlLWFycmFuZ2UgZXhjbHVzaW9uIHJhbmdlIGFuZCB1
bml0eSBtYXAgcmVjb3JkaW5nCgpUaGUgc3BlYyBtYWtlcyBubyBwcm92aXNp
b25zIGZvciBPUyBiZWhhdmlvciBoZXJlIHRvIGRlcGVuZCBvbiB0aGUKYW1v
dW50IG9mIFJBTSBmb3VuZCBvbiB0aGUgc3lzdGVtLiBXaGlsZSB0aGUgc3Bl
YyBtYXkgbm90IHN1ZmZpY2llbnRseQpjbGVhcmx5IGRpc3Rpbmd1aXNoIGJv
dGgga2luZHMgb2YgcmVnaW9ucywgdGhleSBhcmUgc3VyZWx5IG1lYW50IHRv
IGJlCnNlcGFyYXRlIHRoaW5nczogT25seSByZWdpb25zIHdpdGggQUNQSV9J
Vk1EX0VYQ0xVU0lPTl9SQU5HRSBzZXQgc2hvdWxkCmJlIGNhbmRpZGF0ZXMg
Zm9yIHB1dHRpbmcgaW4gdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3RlcnMu
IChBcyB0aGVyZSdzCm9ubHkgYSBzaW5nbGUgc3VjaCBwYWlyIG9mIHJlZ2lz
dGVycyBwZXIgSU9NTVUsIHNlY29uZGFyeSBub24tYWRqYWNlbnQKcmVnaW9u
cyB3aXRoIHRoZSBmbGFnIHNldCBhbHJlYWR5IGdldCBjb252ZXJ0ZWQgdG8g
dW5pdHkgbWFwcGVkCnJlZ2lvbnMuKQoKRmlyc3Qgb2YgYWxsLCBkcm9wIHRo
ZSBkZXBlbmRlbmN5IG9uIG1heF9wYWdlLiBXaXRoIGNvbW1pdCBiNGYwNDIy
MzZhZTAKKCJBTUQvSU9NTVU6IENlYXNlIHVzaW5nIGEgZHluYW1pYyBoZWln
aHQgZm9yIHRoZSBJT01NVSBwYWdldGFibGVzIikgdGhlCnVzZSBvZiBpdCBo
ZXJlIHdhcyBzdGFsZSBhbnl3YXk7IGl0IHdhcyBib2d1cyBhbHJlYWR5IGJl
Zm9yZSwgYXMgaXQKZGlkbid0IGFjY291bnQgZm9yIG1heF9wYWdlIGdldHRp
bmcgaW5jcmVhc2VkIGxhdGVyIG9uLiBTaW1wbHkgdHJ5IGFuCmV4Y2x1c2lv
biByYW5nZSByZWdpc3RyYXRpb24gZmlyc3QsIGFuZCBpZiBpdCBmYWlscyAo
Zm9yIGJlaW5nCnVuc3VpdGFibGUgb3Igbm9uLW1lcmdlYWJsZSksIHJlZ2lz
dGVyIGEgdW5pdHkgbWFwcGluZyByYW5nZS4KCldpdGggdGhpcyB2YXJpb3Vz
IGxvY2FsIHZhcmlhYmxlcyBiZWNvbWUgdW5uZWNlc3NhcnkgYW5kIGhlbmNl
IGdldApkcm9wcGVkIGF0IHRoZSBzYW1lIHRpbWUuCgpXaXRoIHRoZSBtYXhf
cGFnZSBib3VuZGFyeSBkcm9wcGVkIGZvciB1c2luZyB1bml0eSBtYXBzLCB0
aGUgbWluaW11bQpwYWdlIHRhYmxlIHRyZWUgaGVpZ2h0IG5vdyBuZWVkcyBi
b3RoIHJlY29yZGluZyBhbmQgZW5mb3JjaW5nIGluCmFtZF9pb21tdV9kb21h
aW5faW5pdCgpLiBTaW5jZSB3ZSBjYW4ndCBwcmVkaWN0IHdoaWNoIGRldmlj
ZXMgbWF5IGdldAphc3NpZ25lZCB0byBhIGRvbWFpbiwgb3VyIG9ubHkgb3B0
aW9uIGlzIHRvIHVuaWZvcm1seSBmb3JjZSBhdCBsZWFzdAp0aGF0IGhlaWdo
dCBmb3IgYWxsIGRvbWFpbnMsIG5vdyB0aGF0IHRoZSBoZWlnaHQgaXNuJ3Qg
ZHluYW1pYyBhbnltb3JlLgoKRnVydGhlciBkb24ndCBtYWtlIHVzZSBvZiB0
aGUgZXhjbHVzaW9uIHJhbmdlIHVubGVzcyBBQ1BJIGRhdGEgc2F5cyBzby4K
Ck5vdGUgdGhhdCBleGNsdXNpb24gcmFuZ2UgcmVnaXN0cmF0aW9uIGluCnJl
Z2lzdGVyX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygpIGlzIG9uIGEgYmVzdCBl
ZmZvcnQgYmFzaXMuIEhlbmNlIHVuaXR5Cm1hcCBlbnRyaWVzIGFsc28gcmVn
aXN0ZXJlZCBhcmUgcmVkdW5kYW50IHdoZW4gdGhlIGZvcm1lciBzdWNjZWVk
ZWQsIGJ1dAp0aGV5IGFsc28gZG8gbm8gaGFybS4gSW1wcm92ZW1lbnRzIGlu
IHRoaXMgYXJlYSBjYW4gYmUgZG9uZSBsYXRlciBpbW8uCgpBbHNvIGFkanVz
dCB0eXBlcyB3aGVyZSBzdWl0YWJsZSB3aXRob3V0IHRvdWNoaW5nIGV4dHJh
IGxpbmVzLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS0zNzguCgpTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2Vk
LWJ5OiBQYXVsIER1cnJhbnQgPHBhdWxAeGVuLm9yZz4KLS0tClRCRDogVGhl
IGFkanVzdG1lbnQgZG9uZSB0byBhbWRfaW9tbXVfZG9tYWluX2luaXQoKSB3
b3VsZCBzdWdnZXN0IHRoYXQKICAgICBpdCBtaWdodCBiZSB3b3J0aCBjb25z
aWRlcmluZyB0byB1c2UgdGhlIGV4Y2x1c2lvbiByYW5nZSByZWdpc3Rlcgog
ICAgIGZvciB0aGUgaGlnaGVzdCByZXBvcnRlZCAoYW5kIHN1aXRhYmxlKSBy
YW5nZSwgcmF0aGVyIHRoYW4gdGhlCiAgICAgZmlyc3Qgb25lIGZvdW5kICho
b3dldmVyIHVubGlrZWx5IGl0IG1pZ2h0IGJlIHRvIGZpbmQgbXVsdGlwbGUK
ICAgICBvbmVzIGluIHRoZSBmaXJzdCBwbGFjZSkuIFRoYXQgd2F5IHdlIG1p
Z2h0IGJlIGFibGUgdG8gbGltaXQgdGhlCiAgICAgcGFnZSB0YWJsZSB0cmVl
IGhlaWdodC4KLS0tCnY5OiBSZS1iYXNlLgp2ODogUmUtYmFzZSBvdmVyIGFk
anVzdG1lbnRzIHRvIGVhcmxpZXIgcGF0Y2guCnY3OiBEb24ndCB1c2UgZXhj
bHVzaW9uIHJhbmdlIHdoZW4gQUNQSSB0YWJsZSBlbnRyeSBkb2Vzbid0IHNh
eSBzby4KdjY6IFJlLWJhc2UuCnY1OiBOZXcuCi0tLQpCYWNrcG9ydGluZyBu
b3RlOiBUaGlzIGRlcGVuZHMgb24gYjc1YjNjNjJmZTRhICgiQU1EL0lPTU1V
OiBmaXgKb2ZmLWJ5LW9uZSBpbiBhbWRfaW9tbXVfZ2V0X3BhZ2luZ19tb2Rl
KCkgY2FsbGVycyIpLgoKLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gv
YW1kL2lvbW11LmgKKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1k
L2lvbW11LmgKQEAgLTMwNCw2ICszMDQsOCBAQCBleHRlcm4gc3RydWN0IGhw
ZXRfc2JkZiB7CiAgICAgfSBpbml0OwogfSBocGV0X3NiZGY7CiAKK2V4dGVy
biBpbnQgYW1kX2lvbW11X21pbl9wYWdpbmdfbW9kZTsKKwogZXh0ZXJuIHZv
aWQgKnNoYXJlZF9pbnRyZW1hcF90YWJsZTsKIGV4dGVybiB1bnNpZ25lZCBs
b25nICpzaGFyZWRfaW50cmVtYXBfaW51c2U7CiAKLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X2FjcGkuYworKysgYi94ZW4vZHJp
dmVycy9wYXNzdGhyb3VnaC9hbWQvaW9tbXVfYWNwaS5jCkBAIC0xMTcsMTIg
KzExNyw4IEBAIHN0YXRpYyBzdHJ1Y3QgYW1kX2lvbW11ICogX19pbml0IGZp
bmRfaW8KIH0KIAogc3RhdGljIGludCBfX2luaXQgcmVzZXJ2ZV9pb21tdV9l
eGNsdXNpb25fcmFuZ2UoCi0gICAgc3RydWN0IGFtZF9pb21tdSAqaW9tbXUs
IHVpbnQ2NF90IGJhc2UsIHVpbnQ2NF90IGxpbWl0LAotICAgIGJvb2wgYWxs
LCBib29sIGl3LCBib29sIGlyKQorICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlv
bW11LCBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGltaXQsIGJvb2wgYWxsKQog
ewotICAgIGlmICggIWlyIHx8ICFpdyApCi0gICAgICAgIHJldHVybiAtRVBF
Uk07Ci0KICAgICAvKiBuZWVkIHRvIGV4dGVuZCBleGNsdXNpb24gcmFuZ2U/
ICovCiAgICAgaWYgKCBpb21tdS0+ZXhjbHVzaW9uX2VuYWJsZSApCiAgICAg
ewpAQCAtMTUxLDE0ICsxNDcsMTggQEAgc3RhdGljIGludCBfX2luaXQgcmVz
ZXJ2ZV91bml0eV9tYXBfZm9yXwogewogICAgIHN0cnVjdCBpdnJzX21hcHBp
bmdzICppdnJzX21hcHBpbmdzID0gZ2V0X2l2cnNfbWFwcGluZ3Moc2VnKTsK
ICAgICBzdHJ1Y3QgaXZyc191bml0eV9tYXAgKnVuaXR5X21hcCA9IGl2cnNf
bWFwcGluZ3NbYmRmXS51bml0eV9tYXA7CisgICAgaW50IHBhZ2luZ19tb2Rl
ID0gYW1kX2lvbW11X2dldF9wYWdpbmdfbW9kZShQRk5fVVAoYmFzZSArIGxl
bmd0aCkpOworCisgICAgaWYgKCBwYWdpbmdfbW9kZSA8IDAgKQorICAgICAg
ICByZXR1cm4gcGFnaW5nX21vZGU7CiAKICAgICAvKiBDaGVjayBmb3Igb3Zl
cmxhcHMuICovCiAgICAgZm9yICggOyB1bml0eV9tYXA7IHVuaXR5X21hcCA9
IHVuaXR5X21hcC0+bmV4dCApCiAgICAgewogICAgICAgICAvKgogICAgICAg
ICAgKiBFeGFjdCBtYXRjaGVzIGFyZSBva2F5LiBUaGlzIGNhbiBpbiBwYXJ0
aWN1bGFyIGhhcHBlbiB3aGVuCi0gICAgICAgICAqIHJlZ2lzdGVyX2V4Y2x1
c2lvbl9yYW5nZV9mb3JfZGV2aWNlKCkgY2FsbHMgaGVyZSB0d2ljZSBmb3Ig
dGhlCi0gICAgICAgICAqIHNhbWUgKHMsYixkLGYpLgorICAgICAgICAgKiBy
ZWdpc3Rlcl9yYW5nZV9mb3JfZGV2aWNlKCkgY2FsbHMgaGVyZSB0d2ljZSBm
b3IgdGhlIHNhbWUKKyAgICAgICAgICogKHMsYixkLGYpLgogICAgICAgICAg
Ki8KICAgICAgICAgaWYgKCBiYXNlID09IHVuaXR5X21hcC0+YWRkciAmJiBs
ZW5ndGggPT0gdW5pdHlfbWFwLT5sZW5ndGggJiYKICAgICAgICAgICAgICBp
ciA9PSB1bml0eV9tYXAtPnJlYWQgJiYgaXcgPT0gdW5pdHlfbWFwLT53cml0
ZSApCkBAIC0xODYsNTUgKzE4Niw1MiBAQCBzdGF0aWMgaW50IF9faW5pdCBy
ZXNlcnZlX3VuaXR5X21hcF9mb3JfCiAgICAgdW5pdHlfbWFwLT5uZXh0ID0g
aXZyc19tYXBwaW5nc1tiZGZdLnVuaXR5X21hcDsKICAgICBpdnJzX21hcHBp
bmdzW2JkZl0udW5pdHlfbWFwID0gdW5pdHlfbWFwOwogCisgICAgaWYgKCBw
YWdpbmdfbW9kZSA+IGFtZF9pb21tdV9taW5fcGFnaW5nX21vZGUgKQorICAg
ICAgICBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2RlID0gcGFnaW5nX21vZGU7
CisKICAgICByZXR1cm4gMDsKIH0KIAotc3RhdGljIGludCBfX2luaXQgcmVn
aXN0ZXJfZXhjbHVzaW9uX3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygKLSAgICB1
bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3
LCB1OCBpcikKK3N0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX3JhbmdlX2Zv
cl9hbGxfZGV2aWNlcygKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGlt
aXQsIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAg
IGludCBzZWcgPSAwOyAvKiBYWFggKi8KLSAgICB1bnNpZ25lZCBsb25nIHJh
bmdlX3RvcCwgaW9tbXVfdG9wLCBsZW5ndGg7CiAgICAgc3RydWN0IGFtZF9p
b21tdSAqaW9tbXU7Ci0gICAgdW5zaWduZWQgaW50IGJkZjsKICAgICBpbnQg
cmMgPSAwOwogCiAgICAgLyogaXMgcGFydCBvZiBleGNsdXNpb24gcmFuZ2Ug
aW5zaWRlIG9mIElPTU1VIHZpcnR1YWwgYWRkcmVzcyBzcGFjZT8gKi8KICAg
ICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRlciBpcyBhc3N1bWVkIHRvIGJl
IHBhZ2UtYWxpZ25lZCAqLwotICAgIHJhbmdlX3RvcCA9IGxpbWl0ICsgUEFH
RV9TSVpFOwotICAgIGlvbW11X3RvcCA9IG1heF9wYWdlICogUEFHRV9TSVpF
OwotICAgIGlmICggYmFzZSA8IGlvbW11X3RvcCApCi0gICAgewotICAgICAg
ICBpZiAoIHJhbmdlX3RvcCA+IGlvbW11X3RvcCApCi0gICAgICAgICAgICBy
YW5nZV90b3AgPSBpb21tdV90b3A7Ci0gICAgICAgIGxlbmd0aCA9IHJhbmdl
X3RvcCAtIGJhc2U7Ci0gICAgICAgIC8qIHJlc2VydmUgci93IHVuaXR5LW1h
cHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmljZXMgKi8KLSAgICAgICAgLyog
bm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFydCBvZiB0aGUgZXhjbHVzaW9u
IHJhbmdlICovCi0gICAgICAgIGZvciAoIGJkZiA9IDA7ICFyYyAmJiBiZGYg
PCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyApCi0gICAgICAgICAgICByYyA9
IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYsIGJhc2Us
IGxlbmd0aCwgaXcsIGlyKTsKLSAgICAgICAgLyogcHVzaCAnYmFzZScganVz
dCBvdXRzaWRlIG9mIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSAqLwotICAgICAg
ICBiYXNlID0gaW9tbXVfdG9wOwotICAgIH0KLSAgICAvKiByZWdpc3RlciBJ
T01NVSBleGNsdXNpb24gcmFuZ2Ugc2V0dGluZ3MgKi8KLSAgICBpZiAoICFy
YyAmJiBsaW1pdCA+PSBpb21tdV90b3AgKQorICAgIGlmICggZXhjbHVzaW9u
ICkKICAgICB7CiAgICAgICAgIGZvcl9lYWNoX2FtZF9pb21tdSggaW9tbXUg
KQogICAgICAgICB7Ci0gICAgICAgICAgICByYyA9IHJlc2VydmVfaW9tbXVf
ZXhjbHVzaW9uX3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCwKLSAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAv
KiBhbGwgKi8sIGl3LCBpcik7Ci0gICAgICAgICAgICBpZiAoIHJjICkKLSAg
ICAgICAgICAgICAgICBicmVhazsKKyAgICAgICAgICAgIGludCByZXQgPSBy
ZXNlcnZlX2lvbW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGlt
aXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdHJ1ZSAvKiBhbGwgKi8pOworCisgICAgICAgICAgICBp
ZiAoIHJldCAmJiAhcmMgKQorICAgICAgICAgICAgICAgIHJjID0gcmV0Owog
ICAgICAgICB9CiAgICAgfQogCisgICAgaWYgKCAhZXhjbHVzaW9uIHx8IHJj
ICkKKyAgICB7CisgICAgICAgIHBhZGRyX3QgbGVuZ3RoID0gbGltaXQgKyBQ
QUdFX1NJWkUgLSBiYXNlOworICAgICAgICB1bnNpZ25lZCBpbnQgYmRmOwor
CisgICAgICAgIC8qIHJlc2VydmUgci93IHVuaXR5LW1hcHBlZCBwYWdlIGVu
dHJpZXMgZm9yIGRldmljZXMgKi8KKyAgICAgICAgZm9yICggYmRmID0gcmMg
PSAwOyAhcmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysgKQor
ICAgICAgICAgICAgcmMgPSByZXNlcnZlX3VuaXR5X21hcF9mb3JfZGV2aWNl
KHNlZywgYmRmLCBiYXNlLCBsZW5ndGgsIGl3LCBpcik7CisgICAgfQorCiAg
ICAgcmV0dXJuIHJjOwogfQogCi1zdGF0aWMgaW50IF9faW5pdCByZWdpc3Rl
cl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgKLSAgICB1MTYgYmRmLCB1
bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3
LCB1OCBpcikKK3N0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX3JhbmdlX2Zv
cl9kZXZpY2UoCisgICAgdW5zaWduZWQgaW50IGJkZiwgcGFkZHJfdCBiYXNl
LCBwYWRkcl90IGxpbWl0LAorICAgIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wg
ZXhjbHVzaW9uKQogewogICAgIGludCBzZWcgPSAwOyAvKiBYWFggKi8KICAg
ICBzdHJ1Y3QgaXZyc19tYXBwaW5ncyAqaXZyc19tYXBwaW5ncyA9IGdldF9p
dnJzX21hcHBpbmdzKHNlZyk7Ci0gICAgdW5zaWduZWQgbG9uZyByYW5nZV90
b3AsIGlvbW11X3RvcCwgbGVuZ3RoOwogICAgIHN0cnVjdCBhbWRfaW9tbXUg
KmlvbW11OwogICAgIHUxNiByZXE7CiAgICAgaW50IHJjID0gMDsKQEAgLTI0
OCwyNyArMjQ1LDE5IEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4
Y2x1c2lvbl9yYW4KICAgICByZXEgPSBpdnJzX21hcHBpbmdzW2JkZl0uZHRl
X3JlcXVlc3Rvcl9pZDsKIAogICAgIC8qIG5vdGU6ICdsaW1pdCcgcGFyYW1l
dGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICovCi0gICAgcmFu
Z2VfdG9wID0gbGltaXQgKyBQQUdFX1NJWkU7Ci0gICAgaW9tbXVfdG9wID0g
bWF4X3BhZ2UgKiBQQUdFX1NJWkU7Ci0gICAgaWYgKCBiYXNlIDwgaW9tbXVf
dG9wICkKLSAgICB7Ci0gICAgICAgIGlmICggcmFuZ2VfdG9wID4gaW9tbXVf
dG9wICkKLSAgICAgICAgICAgIHJhbmdlX3RvcCA9IGlvbW11X3RvcDsKLSAg
ICAgICAgbGVuZ3RoID0gcmFuZ2VfdG9wIC0gYmFzZTsKKyAgICBpZiAoIGV4
Y2x1c2lvbiApCisgICAgICAgIHJjID0gcmVzZXJ2ZV9pb21tdV9leGNsdXNp
b25fcmFuZ2UoaW9tbXUsIGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhbHNlIC8qIGFsbCAqLyk7
CisgICAgaWYgKCAhZXhjbHVzaW9uIHx8IHJjICkKKyAgICB7CisgICAgICAg
IHBhZGRyX3QgbGVuZ3RoID0gbGltaXQgKyBQQUdFX1NJWkUgLSBiYXNlOwor
CiAgICAgICAgIC8qIHJlc2VydmUgdW5pdHktbWFwcGVkIHBhZ2UgZW50cmll
cyBmb3IgZGV2aWNlICovCi0gICAgICAgIC8qIG5vdGU6IHRoZXNlIGVudHJp
ZXMgYXJlIHBhcnQgb2YgdGhlIGV4Y2x1c2lvbiByYW5nZSAqLwogICAgICAg
ICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCBiZGYs
IGJhc2UsIGxlbmd0aCwgaXcsIGlyKSA/OgogICAgICAgICAgICAgIHJlc2Vy
dmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2Uoc2VnLCByZXEsIGJhc2UsIGxlbmd0
aCwgaXcsIGlyKTsKLQotICAgICAgICAvKiBwdXNoICdiYXNlJyBqdXN0IG91
dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNzIHNwYWNlICovCi0gICAgICAgIGJh
c2UgPSBpb21tdV90b3A7CiAgICAgfQotCi0gICAgLyogcmVnaXN0ZXIgSU9N
TVUgZXhjbHVzaW9uIHJhbmdlIHNldHRpbmdzIGZvciBkZXZpY2UgKi8KLSAg
ICBpZiAoICFyYyAmJiBsaW1pdCA+PSBpb21tdV90b3AgICkKKyAgICBlbHNl
CiAgICAgewotICAgICAgICByYyA9IHJlc2VydmVfaW9tbXVfZXhjbHVzaW9u
X3JhbmdlKGlvbW11LCBiYXNlLCBsaW1pdCwKLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBmYWxzZSAvKiBhbGwgKi8sIGl3
LCBpcik7CiAgICAgICAgIGl2cnNfbWFwcGluZ3NbYmRmXS5kdGVfYWxsb3df
ZXhjbHVzaW9uID0gdHJ1ZTsKICAgICAgICAgaXZyc19tYXBwaW5nc1tyZXFd
LmR0ZV9hbGxvd19leGNsdXNpb24gPSB0cnVlOwogICAgIH0KQEAgLTI3Niw1
MyArMjY1LDQyIEBAIHN0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX2V4Y2x1
c2lvbl9yYW4KICAgICByZXR1cm4gcmM7CiB9CiAKLXN0YXRpYyBpbnQgX19p
bml0IHJlZ2lzdGVyX2V4Y2x1c2lvbl9yYW5nZV9mb3JfaW9tbXVfZGV2aWNl
cygKLSAgICBzdHJ1Y3QgYW1kX2lvbW11ICppb21tdSwKLSAgICB1bnNpZ25l
ZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQsIHU4IGl3LCB1OCBp
cikKK3N0YXRpYyBpbnQgX19pbml0IHJlZ2lzdGVyX3JhbmdlX2Zvcl9pb21t
dV9kZXZpY2VzKAorICAgIHN0cnVjdCBhbWRfaW9tbXUgKmlvbW11LCBwYWRk
cl90IGJhc2UsIHBhZGRyX3QgbGltaXQsCisgICAgYm9vbCBpdywgYm9vbCBp
ciwgYm9vbCBleGNsdXNpb24pCiB7Ci0gICAgdW5zaWduZWQgbG9uZyByYW5n
ZV90b3AsIGlvbW11X3RvcCwgbGVuZ3RoOworICAgIC8qIG5vdGU6ICdsaW1p
dCcgcGFyYW1ldGVyIGlzIGFzc3VtZWQgdG8gYmUgcGFnZS1hbGlnbmVkICov
CisgICAgcGFkZHJfdCBsZW5ndGggPSBsaW1pdCArIFBBR0VfU0laRSAtIGJh
c2U7CiAgICAgdW5zaWduZWQgaW50IGJkZjsKICAgICB1MTYgcmVxOwotICAg
IGludCByYyA9IDA7CisgICAgaW50IHJjOwogCi0gICAgLyogaXMgcGFydCBv
ZiBleGNsdXNpb24gcmFuZ2UgaW5zaWRlIG9mIElPTU1VIHZpcnR1YWwgYWRk
cmVzcyBzcGFjZT8gKi8KLSAgICAvKiBub3RlOiAnbGltaXQnIHBhcmFtZXRl
ciBpcyBhc3N1bWVkIHRvIGJlIHBhZ2UtYWxpZ25lZCAqLwotICAgIHJhbmdl
X3RvcCA9IGxpbWl0ICsgUEFHRV9TSVpFOwotICAgIGlvbW11X3RvcCA9IG1h
eF9wYWdlICogUEFHRV9TSVpFOwotICAgIGlmICggYmFzZSA8IGlvbW11X3Rv
cCApCi0gICAgewotICAgICAgICBpZiAoIHJhbmdlX3RvcCA+IGlvbW11X3Rv
cCApCi0gICAgICAgICAgICByYW5nZV90b3AgPSBpb21tdV90b3A7Ci0gICAg
ICAgIGxlbmd0aCA9IHJhbmdlX3RvcCAtIGJhc2U7Ci0gICAgICAgIC8qIHJl
c2VydmUgci93IHVuaXR5LW1hcHBlZCBwYWdlIGVudHJpZXMgZm9yIGRldmlj
ZXMgKi8KLSAgICAgICAgLyogbm90ZTogdGhlc2UgZW50cmllcyBhcmUgcGFy
dCBvZiB0aGUgZXhjbHVzaW9uIHJhbmdlICovCi0gICAgICAgIGZvciAoIGJk
ZiA9IDA7ICFyYyAmJiBiZGYgPCBpdnJzX2JkZl9lbnRyaWVzOyBiZGYrKyAp
Ci0gICAgICAgIHsKLSAgICAgICAgICAgIGlmICggaW9tbXUgPT0gZmluZF9p
b21tdV9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIGJkZikgKQotICAgICAgICAg
ICAgewotICAgICAgICAgICAgICAgIHJlcSA9IGdldF9pdnJzX21hcHBpbmdz
KGlvbW11LT5zZWcpW2JkZl0uZHRlX3JlcXVlc3Rvcl9pZDsKLSAgICAgICAg
ICAgICAgICByYyA9IHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9kZXZpY2UoaW9t
bXUtPnNlZywgYmRmLCBiYXNlLCBsZW5ndGgsCi0gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3LCBpcikgPzoK
LSAgICAgICAgICAgICAgICAgICAgIHJlc2VydmVfdW5pdHlfbWFwX2Zvcl9k
ZXZpY2UoaW9tbXUtPnNlZywgcmVxLCBiYXNlLCBsZW5ndGgsCi0gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGl3
LCBpcik7Ci0gICAgICAgICAgICB9Ci0gICAgICAgIH0KLQotICAgICAgICAv
KiBwdXNoICdiYXNlJyBqdXN0IG91dHNpZGUgb2YgdmlydHVhbCBhZGRyZXNz
IHNwYWNlICovCi0gICAgICAgIGJhc2UgPSBpb21tdV90b3A7CisgICAgaWYg
KCBleGNsdXNpb24gKQorICAgIHsKKyAgICAgICAgcmMgPSByZXNlcnZlX2lv
bW11X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsIHRydWUg
LyogYWxsICovKTsKKyAgICAgICAgaWYgKCAhcmMgKQorICAgICAgICAgICAg
cmV0dXJuIDA7CiAgICAgfQogCi0gICAgLyogcmVnaXN0ZXIgSU9NTVUgZXhj
bHVzaW9uIHJhbmdlIHNldHRpbmdzICovCi0gICAgaWYgKCAhcmMgJiYgbGlt
aXQgPj0gaW9tbXVfdG9wICkKLSAgICAgICAgcmMgPSByZXNlcnZlX2lvbW11
X2V4Y2x1c2lvbl9yYW5nZShpb21tdSwgYmFzZSwgbGltaXQsCi0gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdHJ1ZSAvKiBh
bGwgKi8sIGl3LCBpcik7CisgICAgLyogcmVzZXJ2ZSB1bml0eS1tYXBwZWQg
cGFnZSBlbnRyaWVzIGZvciBkZXZpY2VzICovCisgICAgZm9yICggYmRmID0g
cmMgPSAwOyAhcmMgJiYgYmRmIDwgaXZyc19iZGZfZW50cmllczsgYmRmKysg
KQorICAgIHsKKyAgICAgICAgaWYgKCBpb21tdSAhPSBmaW5kX2lvbW11X2Zv
cl9kZXZpY2UoaW9tbXUtPnNlZywgYmRmKSApCisgICAgICAgICAgICBjb250
aW51ZTsKKworICAgICAgICByZXEgPSBnZXRfaXZyc19tYXBwaW5ncyhpb21t
dS0+c2VnKVtiZGZdLmR0ZV9yZXF1ZXN0b3JfaWQ7CisgICAgICAgIHJjID0g
cmVzZXJ2ZV91bml0eV9tYXBfZm9yX2RldmljZShpb21tdS0+c2VnLCBiZGYs
IGJhc2UsIGxlbmd0aCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGl3LCBpcikgPzoKKyAgICAgICAgICAgICByZXNlcnZl
X3VuaXR5X21hcF9mb3JfZGV2aWNlKGlvbW11LT5zZWcsIHJlcSwgYmFzZSwg
bGVuZ3RoLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgaXcsIGlyKTsKKyAgICB9CiAKICAgICByZXR1cm4gcmM7CiB9CiAK
IHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGVjdCgK
ICAgICBjb25zdCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9j
aywKLSAgICB1bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGlt
aXQsIHU4IGl3LCB1OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3Qg
bGltaXQsIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewog
ICAgIHUxNiBiZGY7CiAKQEAgLTMzMywxMiArMzExLDEyIEBAIHN0YXRpYyBp
bnQgX19pbml0IHBhcnNlX2l2bWRfZGV2aWNlX3NlbGUKICAgICAgICAgcmV0
dXJuIC1FTk9ERVY7CiAgICAgfQogCi0gICAgcmV0dXJuIHJlZ2lzdGVyX2V4
Y2x1c2lvbl9yYW5nZV9mb3JfZGV2aWNlKGJkZiwgYmFzZSwgbGltaXQsIGl3
LCBpcik7CisgICAgcmV0dXJuIHJlZ2lzdGVyX3JhbmdlX2Zvcl9kZXZpY2Uo
YmRmLCBiYXNlLCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQogCiBz
dGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9yYW5nZSgKICAg
ICBjb25zdCBzdHJ1Y3QgYWNwaV9pdnJzX21lbW9yeSAqaXZtZF9ibG9jaywK
LSAgICB1bnNpZ25lZCBsb25nIGJhc2UsIHVuc2lnbmVkIGxvbmcgbGltaXQs
IHU4IGl3LCB1OCBpcikKKyAgICBwYWRkcl90IGJhc2UsIHBhZGRyX3QgbGlt
aXQsIGJvb2wgaXcsIGJvb2wgaXIsIGJvb2wgZXhjbHVzaW9uKQogewogICAg
IHVuc2lnbmVkIGludCBmaXJzdF9iZGYsIGxhc3RfYmRmLCBiZGY7CiAgICAg
aW50IGVycm9yOwpAQCAtMzYwLDE1ICszMzgsMTUgQEAgc3RhdGljIGludCBf
X2luaXQgcGFyc2VfaXZtZF9kZXZpY2VfcmFuZwogICAgIH0KIAogICAgIGZv
ciAoIGJkZiA9IGZpcnN0X2JkZiwgZXJyb3IgPSAwOyAoYmRmIDw9IGxhc3Rf
YmRmKSAmJiAhZXJyb3I7IGJkZisrICkKLSAgICAgICAgZXJyb3IgPSByZWdp
c3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2RldmljZSgKLSAgICAgICAgICAg
IGJkZiwgYmFzZSwgbGltaXQsIGl3LCBpcik7CisgICAgICAgIGVycm9yID0g
cmVnaXN0ZXJfcmFuZ2VfZm9yX2RldmljZSgKKyAgICAgICAgICAgIGJkZiwg
YmFzZSwgbGltaXQsIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIHJldHVy
biBlcnJvcjsKIH0KIAogc3RhdGljIGludCBfX2luaXQgcGFyc2VfaXZtZF9k
ZXZpY2VfaW9tbXUoCiAgICAgY29uc3Qgc3RydWN0IGFjcGlfaXZyc19tZW1v
cnkgKml2bWRfYmxvY2ssCi0gICAgdW5zaWduZWQgbG9uZyBiYXNlLCB1bnNp
Z25lZCBsb25nIGxpbWl0LCB1OCBpdywgdTggaXIpCisgICAgcGFkZHJfdCBi
YXNlLCBwYWRkcl90IGxpbWl0LCBib29sIGl3LCBib29sIGlyLCBib29sIGV4
Y2x1c2lvbikKIHsKICAgICBpbnQgc2VnID0gMDsgLyogWFhYICovCiAgICAg
c3RydWN0IGFtZF9pb21tdSAqaW9tbXU7CkBAIC0zODMsMTQgKzM2MSwxNCBA
QCBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2RldmljZV9pb21tCiAg
ICAgICAgIHJldHVybiAtRU5PREVWOwogICAgIH0KIAotICAgIHJldHVybiBy
ZWdpc3Rlcl9leGNsdXNpb25fcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCi0g
ICAgICAgIGlvbW11LCBiYXNlLCBsaW1pdCwgaXcsIGlyKTsKKyAgICByZXR1
cm4gcmVnaXN0ZXJfcmFuZ2VfZm9yX2lvbW11X2RldmljZXMoCisgICAgICAg
IGlvbW11LCBiYXNlLCBsaW1pdCwgaXcsIGlyLCBleGNsdXNpb24pOwogfQog
CiBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9pdm1kX2Jsb2NrKGNvbnN0IHN0
cnVjdCBhY3BpX2l2cnNfbWVtb3J5ICppdm1kX2Jsb2NrKQogewogICAgIHVu
c2lnbmVkIGxvbmcgc3RhcnRfYWRkciwgbWVtX2xlbmd0aCwgYmFzZSwgbGlt
aXQ7Ci0gICAgdTggaXcsIGlyOworICAgIGJvb2wgaXcgPSB0cnVlLCBpciA9
IHRydWUsIGV4Y2x1c2lvbiA9IGZhbHNlOwogCiAgICAgaWYgKCBpdm1kX2Js
b2NrLT5oZWFkZXIubGVuZ3RoIDwgc2l6ZW9mKCppdm1kX2Jsb2NrKSApCiAg
ICAgewpAQCAtNDA3LDEzICszODUsMTEgQEAgc3RhdGljIGludCBfX2luaXQg
cGFyc2VfaXZtZF9ibG9jayhjb25zdAogICAgICAgICAgICAgICAgICAgICBp
dm1kX2Jsb2NrLT5oZWFkZXIudHlwZSwgc3RhcnRfYWRkciwgbWVtX2xlbmd0
aCk7CiAKICAgICBpZiAoIGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFD
UElfSVZNRF9FWENMVVNJT05fUkFOR0UgKQotICAgICAgICBpdyA9IGlyID0g
SU9NTVVfQ09OVFJPTF9FTkFCTEVEOworICAgICAgICBleGNsdXNpb24gPSB0
cnVlOwogICAgIGVsc2UgaWYgKCBpdm1kX2Jsb2NrLT5oZWFkZXIuZmxhZ3Mg
JiBBQ1BJX0lWTURfVU5JVFkgKQogICAgIHsKLSAgICAgICAgaXcgPSBpdm1k
X2Jsb2NrLT5oZWFkZXIuZmxhZ3MgJiBBQ1BJX0lWTURfUkVBRCA/Ci0gICAg
ICAgICAgICBJT01NVV9DT05UUk9MX0VOQUJMRUQgOiBJT01NVV9DT05UUk9M
X0RJU0FCTEVEOwotICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5m
bGFncyAmIEFDUElfSVZNRF9XUklURSA/Ci0gICAgICAgICAgICBJT01NVV9D
T05UUk9MX0VOQUJMRUQgOiBJT01NVV9DT05UUk9MX0RJU0FCTEVEOworICAg
ICAgICBpdyA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFncyAmIEFDUElfSVZN
RF9SRUFEOworICAgICAgICBpciA9IGl2bWRfYmxvY2stPmhlYWRlci5mbGFn
cyAmIEFDUElfSVZNRF9XUklURTsKICAgICB9CiAgICAgZWxzZQogICAgIHsK
QEAgLTQyNCwyMCArNDAwLDIwIEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNl
X2l2bWRfYmxvY2soY29uc3QKICAgICBzd2l0Y2goIGl2bWRfYmxvY2stPmhl
YWRlci50eXBlICkKICAgICB7CiAgICAgY2FzZSBBQ1BJX0lWUlNfVFlQRV9N
RU1PUllfQUxMOgotICAgICAgICByZXR1cm4gcmVnaXN0ZXJfZXhjbHVzaW9u
X3JhbmdlX2Zvcl9hbGxfZGV2aWNlcygKLSAgICAgICAgICAgIGJhc2UsIGxp
bWl0LCBpdywgaXIpOworICAgICAgICByZXR1cm4gcmVnaXN0ZXJfcmFuZ2Vf
Zm9yX2FsbF9kZXZpY2VzKAorICAgICAgICAgICAgYmFzZSwgbGltaXQsIGl3
LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGNhc2UgQUNQSV9JVlJTX1RZUEVf
TUVNT1JZX09ORToKLSAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNl
X3NlbGVjdChpdm1kX2Jsb2NrLAotICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywgaXIpOworICAgICAg
ICByZXR1cm4gcGFyc2VfaXZtZF9kZXZpY2Vfc2VsZWN0KGl2bWRfYmxvY2ss
IGJhc2UsIGxpbWl0LAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGl3LCBpciwgZXhjbHVzaW9uKTsKIAogICAgIGNhc2UgQUNQ
SV9JVlJTX1RZUEVfTUVNT1JZX1JBTkdFOgotICAgICAgICByZXR1cm4gcGFy
c2VfaXZtZF9kZXZpY2VfcmFuZ2UoaXZtZF9ibG9jaywKLSAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJhc2UsIGxpbWl0LCBpdywg
aXIpOworICAgICAgICByZXR1cm4gcGFyc2VfaXZtZF9kZXZpY2VfcmFuZ2Uo
aXZtZF9ibG9jaywgYmFzZSwgbGltaXQsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBpdywgaXIsIGV4Y2x1c2lvbik7CiAKICAg
ICBjYXNlIEFDUElfSVZSU19UWVBFX01FTU9SWV9JT01NVToKLSAgICAgICAg
cmV0dXJuIHBhcnNlX2l2bWRfZGV2aWNlX2lvbW11KGl2bWRfYmxvY2ssCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBiYXNlLCBs
aW1pdCwgaXcsIGlyKTsKKyAgICAgICAgcmV0dXJuIHBhcnNlX2l2bWRfZGV2
aWNlX2lvbW11KGl2bWRfYmxvY2ssIGJhc2UsIGxpbWl0LAorICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaXcsIGlyLCBleGNsdXNp
b24pOwogCiAgICAgZGVmYXVsdDoKICAgICAgICAgQU1EX0lPTU1VX0RFQlVH
KCJJVk1EIEVycm9yOiBJbnZhbGlkIEJsb2NrIFR5cGUhXG4iKTsKLS0tIGEv
eGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYwor
KysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9pb21t
dS5jCkBAIC0yNDYsNiArMjQ2LDggQEAgaW50IGFtZF9pb21tdV9hbGxvY19y
b290KHN0cnVjdCBkb21haW4gKgogICAgIHJldHVybiAwOwogfQogCitpbnQg
X19yZWFkX21vc3RseSBhbWRfaW9tbXVfbWluX3BhZ2luZ19tb2RlID0gMTsK
Kwogc3RhdGljIGludCBhbWRfaW9tbXVfZG9tYWluX2luaXQoc3RydWN0IGRv
bWFpbiAqZCkKIHsKICAgICBzdHJ1Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRv
bV9pb21tdShkKTsKQEAgLTI1NywxMSArMjU5LDEzIEBAIHN0YXRpYyBpbnQg
YW1kX2lvbW11X2RvbWFpbl9pbml0KHN0cnVjdAogICAgICAqIC0gSFZNIGNv
dWxkIGluIHByaW5jaXBsZSB1c2UgMyBvciA0IGRlcGVuZGluZyBvbiBob3cg
bXVjaCBndWVzdAogICAgICAqICAgcGh5c2ljYWwgYWRkcmVzcyBzcGFjZSB3
ZSBnaXZlIGl0LCBidXQgdGhpcyBpc24ndCBrbm93biB5ZXQgc28gdXNlIDQK
ICAgICAgKiAgIHVuaWxhdGVyYWxseS4KKyAgICAgKiAtIFVuaXR5IG1hcHMg
bWF5IHJlcXVpcmUgYW4gZXZlbiBoaWdoZXIgbnVtYmVyLgogICAgICAqLwot
ICAgIGhkLT5hcmNoLmFtZC5wYWdpbmdfbW9kZSA9IGFtZF9pb21tdV9nZXRf
cGFnaW5nX21vZGUoCi0gICAgICAgIGlzX2h2bV9kb21haW4oZCkKLSAgICAg
ICAgPyAxdWwgPDwgKERFRkFVTFRfRE9NQUlOX0FERFJFU1NfV0lEVEggLSBQ
QUdFX1NISUZUKQotICAgICAgICA6IGdldF91cHBlcl9tZm5fYm91bmQoKSAr
IDEpOworICAgIGhkLT5hcmNoLmFtZC5wYWdpbmdfbW9kZSA9IG1heChhbWRf
aW9tbXVfZ2V0X3BhZ2luZ19tb2RlKAorICAgICAgICAgICAgaXNfaHZtX2Rv
bWFpbihkKQorICAgICAgICAgICAgPyAxdWwgPDwgKERFRkFVTFRfRE9NQUlO
X0FERFJFU1NfV0lEVEggLSBQQUdFX1NISUZUKQorICAgICAgICAgICAgOiBn
ZXRfdXBwZXJfbWZuX2JvdW5kKCkgKyAxKSwKKyAgICAgICAgYW1kX2lvbW11
X21pbl9wYWdpbmdfbW9kZSk7CiAKICAgICByZXR1cm4gMDsKIH0K

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-7.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-7.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBpbnRyb2R1Y2UgcDJtX2lzX3NwZWNpYWwoKQoKU2VlaW5n
IHRoZSBzaW1pbGFyaXR5IG9mIGdyYW50LCBmb3JlaWduLCBhbmQgKHN1YnNl
cXVlbnRseSkgZGlyZWN0LU1NSU8KaGFuZGxpbmcsIGludHJvZHVjZSBhIG5l
dyBQMk0gdHlwZSBncm91cCBuYW1lZCAic3BlY2lhbCIgKGFzIGluICJuZWVk
aW5nCnNwZWNpYWwgYWNjZXNzb3JzIHRvIGNyZWF0ZS9kZXN0cm95IikuCgpB
bHNvIHVzZSAtRVBFUk0gaW5zdGVhZCBvZiBvdGhlciBlcnJvciBjb2RlcyBv
biB0aGUgdHdvIGRvbWFpbl9jcmFzaCgpCnBhdGhzIHRvdWNoZWQuCgpUaGlz
IGlzIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgotLS0Kdjg6IE5ldywgc3BsaXQgZnJvbSBz
dWJzZXF1ZW50IHBhdGNoLgoKLS0tIGEveGVuL2FyY2gveDg2L21tL3AybS5j
CisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtODE3LDcgKzgxNyw3
IEBAIHAybV9yZW1vdmVfcGFnZShzdHJ1Y3QgcDJtX2RvbWFpbiAqcDJtLAog
ICAgICAgICBmb3IgKCBpID0gMDsgaSA8ICgxVUwgPDwgcGFnZV9vcmRlcik7
IGkrKyApCiAgICAgICAgIHsKICAgICAgICAgICAgIHAybS0+Z2V0X2VudHJ5
KHAybSwgZ2ZuX2FkZChnZm4sIGkpLCAmdCwgJmEsIDAsIE5VTEwsIE5VTEwp
OwotICAgICAgICAgICAgaWYgKCAhcDJtX2lzX2dyYW50KHQpICYmICFwMm1f
aXNfc2hhcmVkKHQpICYmICFwMm1faXNfZm9yZWlnbih0KSApCisgICAgICAg
ICAgICBpZiAoICFwMm1faXNfc3BlY2lhbCh0KSAmJiAhcDJtX2lzX3NoYXJl
ZCh0KSApCiAgICAgICAgICAgICAgICAgc2V0X2dwZm5fZnJvbV9tZm4obWZu
X3gobWZuKSArIGksIElOVkFMSURfTTJQX0VOVFJZKTsKICAgICAgICAgfQog
ICAgIH0KQEAgLTk1NCwxMyArOTU0LDEzIEBAIGd1ZXN0X3BoeXNtYXBfYWRk
X2VudHJ5KHN0cnVjdCBkb21haW4gKmQKICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAmb3QsICZhLCAwLCBOVUxMLCBOVUxMKTsKICAgICAg
ICAgICAgIEFTU0VSVCghcDJtX2lzX3NoYXJlZChvdCkpOwogICAgICAgICB9
Ci0gICAgICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8fCBwMm1faXNfZm9y
ZWlnbihvdCkgKQorICAgICAgICBpZiAoIHAybV9pc19zcGVjaWFsKG90KSAp
CiAgICAgICAgIHsKLSAgICAgICAgICAgIC8qIFJlYWxseSBzaG91bGRuJ3Qg
YmUgdW5tYXBwaW5nIGdyYW50L2ZvcmVpZ24gbWFwcyB0aGlzIHdheSAqLwor
ICAgICAgICAgICAgLyogRG9uJ3QgcGVybWl0IHVubWFwcGluZyBncmFudC9m
b3JlaWduIHRoaXMgd2F5LiAqLwogICAgICAgICAgICAgZG9tYWluX2NyYXNo
KGQpOwogICAgICAgICAgICAgcDJtX3VubG9jayhwMm0pOwogICAgICAgICAg
ICAgCi0gICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsKKyAgICAgICAgICAg
IHJldHVybiAtRVBFUk07CiAgICAgICAgIH0KICAgICAgICAgZWxzZSBpZiAo
IHAybV9pc19yYW0ob3QpICYmICFwMm1faXNfcGFnZWQob3QpICkKICAgICAg
ICAgewpAQCAtMTA1Myw4ICsxMDUzLDcgQEAgaW50IHAybV9jaGFuZ2VfdHlw
ZV9vbmUoc3RydWN0IGRvbWFpbiAqZAogICAgIHN0cnVjdCBwMm1fZG9tYWlu
ICpwMm0gPSBwMm1fZ2V0X2hvc3RwMm0oZCk7CiAgICAgaW50IHJjOwogCi0g
ICAgQlVHX09OKHAybV9pc19ncmFudChvdCkgfHwgcDJtX2lzX2dyYW50KG50
KSk7Ci0gICAgQlVHX09OKHAybV9pc19mb3JlaWduKG90KSB8fCBwMm1faXNf
Zm9yZWlnbihudCkpOworICAgIEJVR19PTihwMm1faXNfc3BlY2lhbChvdCkg
fHwgcDJtX2lzX3NwZWNpYWwobnQpKTsKIAogICAgIGdmbl9sb2NrKHAybSwg
Z2ZuLCAwKTsKIApAQCAtMTMwMCwxMSArMTI5OSwxMSBAQCBzdGF0aWMgaW50
IHNldF90eXBlZF9wMm1fZW50cnkoc3RydWN0IGRvCiAgICAgICAgIGdmbl91
bmxvY2socDJtLCBnZm4sIG9yZGVyKTsKICAgICAgICAgcmV0dXJuIGN1cl9v
cmRlciArIDE7CiAgICAgfQotICAgIGlmICggcDJtX2lzX2dyYW50KG90KSB8
fCBwMm1faXNfZm9yZWlnbihvdCkgKQorICAgIGlmICggcDJtX2lzX3NwZWNp
YWwob3QpICkKICAgICB7CiAgICAgICAgIGdmbl91bmxvY2socDJtLCBnZm4s
IG9yZGVyKTsKICAgICAgICAgZG9tYWluX2NyYXNoKGQpOwotICAgICAgICBy
ZXR1cm4gLUVOT0VOVDsKKyAgICAgICAgcmV0dXJuIC1FUEVSTTsKICAgICB9
CiAgICAgZWxzZSBpZiAoIHAybV9pc19yYW0ob3QpICkKICAgICB7Ci0tLSBh
L3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1ZGUv
YXNtLXg4Ni9wMm0uaApAQCAtMTU0LDYgKzE1NCwxMCBAQCB0eXBlZGVmIHVu
c2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB8IHAybV90b19tYXNrKHAybV9yYW1fbG9nZGlydHkpICkKICNk
ZWZpbmUgUDJNX1NIQVJFRF9UWVBFUyAgIChwMm1fdG9fbWFzayhwMm1fcmFt
X3NoYXJlZCkpCiAKKy8qIFR5cGVzIGVzdGFibGlzaGVkL2NsZWFuZWQgdXAg
dmlhIHNwZWNpYWwgYWNjZXNzb3JzLiAqLworI2RlZmluZSBQMk1fU1BFQ0lB
TF9UWVBFUyAoUDJNX0dSQU5UX1RZUEVTIHwgXAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21hcF9mb3JlaWduKSkKKwog
LyogVmFsaWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQgd2l0
aCBhICh2YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9NRk5f
VFlQRVMgKFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCiAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgfCBwMm1fdG9fbWFzayhwMm1f
bW1pb19kaXJlY3QpIFwKQEAgLTE4Miw2ICsxODYsNyBAQCB0eXBlZGVmIHVu
c2lnbmVkIGludCBwMm1fcXVlcnlfdDsKICNkZWZpbmUgcDJtX2lzX3BhZ2Vk
KF90KSAgICAocDJtX3RvX21hc2soX3QpICYgUDJNX1BBR0VEX1RZUEVTKQog
I2RlZmluZSBwMm1faXNfc2hhcmFibGUoX3QpIChwMm1fdG9fbWFzayhfdCkg
JiBQMk1fU0hBUkFCTEVfVFlQRVMpCiAjZGVmaW5lIHAybV9pc19zaGFyZWQo
X3QpICAgKHAybV90b19tYXNrKF90KSAmIFAyTV9TSEFSRURfVFlQRVMpCisj
ZGVmaW5lIHAybV9pc19zcGVjaWFsKF90KSAgKHAybV90b19tYXNrKF90KSAm
IFAyTV9TUEVDSUFMX1RZUEVTKQogI2RlZmluZSBwMm1faXNfYnJva2VuKF90
KSAgIChwMm1fdG9fbWFzayhfdCkgJiBQMk1fQlJPS0VOX1RZUEVTKQogI2Rl
ZmluZSBwMm1faXNfZm9yZWlnbihfdCkgIChwMm1fdG9fbWFzayhfdCkgJiBw
Mm1fdG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pKQogCg==

--=separator
Content-Type: application/octet-stream; name="xsa378/xsa378-8.patch"
Content-Disposition: attachment; filename="xsa378/xsa378-8.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODYvcDJtOiBndWFyZCAoaW4gcGFydGljdWxhcikgaWRlbnRpdHkgbWFw
cGluZyBlbnRyaWVzCgpTdWNoIGVudHJpZXMsIGNyZWF0ZWQgYnkgc2V0X2lk
ZW50aXR5X3AybV9lbnRyeSgpLCBzaG91bGQgb25seSBiZQpkZXN0cm95ZWQg
YnkgY2xlYXJfaWRlbnRpdHlfcDJtX2VudHJ5KCkuIEhvd2V2ZXIsIHNpbWls
YXJseSwgZW50cmllcwpjcmVhdGVkIGJ5IHNldF9tbWlvX3AybV9lbnRyeSgp
IHNob3VsZCBvbmx5IGJlIHRvcm4gZG93biBieQpjbGVhcl9tbWlvX3AybV9l
bnRyeSgpLCBzbyB0aGUgbG9naWMgZ2V0cyBiYXNlZCB1cG9uIHAybV9tbWlv
X2RpcmVjdCBhcwp0aGUgZW50cnkgdHlwZSAoc2VwYXJhdGlvbiBiZXR3ZWVu
ICJvcmRpbmFyeSIgYW5kIDE6MSBtYXBwaW5ncyB3b3VsZApyZXF1aXJlIGEg
ZnVydGhlciBpbmRpY2F0b3IgdG8gdGVsbCBhcGFydCB0aGUgdHdvKS4KCkFz
IHRvIHRoZSBndWVzdF9yZW1vdmVfcGFnZSgpIGNoYW5nZSwgY29tbWl0IDQ4
ZGZiMjk3YTIwYSAoIng4Ni9QVkg6CmFsbG93IGd1ZXN0X3JlbW92ZV9wYWdl
IHRvIHJlbW92ZSBwMm1fbW1pb19kaXJlY3QgcGFnZXMiKSwgd2hpY2gKaW50
cm9kdWNlZCB0aGUgY2FsbCB0byBjbGVhcl9tbWlvX3AybV9lbnRyeSgpLCBj
bGFpbWVkIHRoaXMgd2FzIGRvbmUgZm9yCmh3ZG9tIG9ubHkgd2l0aG91dCB0
aGlzIGFjdHVhbGx5IGhhdmluZyBiZWVuIHRoZSBjYXNlLiBIb3dldmVyLCB0
aGlzCmNvZGUgc2hvdWxkbid0IGJlIHRoZXJlIGluIHRoZSBmaXJzdCBwbGFj
ZSwgYXMgTU1JTyBlbnRyaWVzIHNob3VsZG4ndCBiZQpkcm9wcGVkIHRoaXMg
d2F5LiBBdm9pZCB0cmlnZ2VyaW5nIHRoZSB3YXJuaW5nIGFnYWluIHRoYXQg
NDhkZmIyOTdhMjBhCnNpbGVuY2VkIGJ5IGFuIGFkanVzdG1lbnQgdG8geGVu
bWVtX2FkZF90b19waHlzbWFwX29uZSgpIGluc3RlYWQuCgpOb3RlIHRoYXQg
Z3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZCgpIGdldHMg
dGlnaHRlbmVkIGJleW9uZAp0aGUgaW1tZWRpYXRlIHB1cnBvc2Ugb2YgdGhp
cyBjaGFuZ2UuCgpOb3RlIGFsc28gdGhhdCBJIGRpZG4ndCBpbnNwZWN0IGNv
ZGUgd2hpY2ggaXNuJ3Qgc2VjdXJpdHkgc3VwcG9ydGVkLAplLmcuIHNoYXJp
bmcsIHBhZ2luZywgb3IgYWx0cDJtLgoKVGhpcyBpcyBDVkUtMjAyMS0yODY5
NCAvIHBhcnQgb2YgWFNBLTM3OC4KClNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KUmV2aWV3ZWQtYnk6IFBhdWwgRHVy
cmFudCA8cGF1bEB4ZW4ub3JnPgotLS0Kdjg6IFNwbGl0IG9mZiBpbnRyb2R1
Y3Rpb24gb2YgcDJtX2lzX3NwZWNpYWwoKS4Kdjc6IFJlLWJhc2UuCnY2OiBB
bHNvIGFkanVzdCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKCkuIEZ1bGx5
IGRyb3AgcDJtX21taW9fZGlyZWN0CiAgICBzcGVjaWFsIGhhbmRsaW5nIGZy
b20gZ3Vlc3RfcmVtb3ZlX3BhZ2UoKTsganVzdCByZXR1cm4gYW4gZXJyb3Iu
CiAgICBNYWtlIGNsZWFyX21taW9fcDJtX2VudHJ5KCkgc3RhdGljLgp2NTog
TmV3LgotLS0KQmFja3BvcnRpbmcgbm90ZTogV2hpbGUgbGlrZWx5IG5vdGlj
YWJsZSBieSBwYXRjaCBjb25mbGljdHMgYW55d2F5LCB0aGlzCmRlcGVuZHMg
b24gb25lIG9yIGJvdGggb2YgYTZiMDUxYTg3YTU4ICgieDg2L3AybTogZG9u
J3QgaWdub3JlCnAybV9yZW1vdmVfcGFnZSgpJ3MgcmV0dXJuIHZhbHVlIikg
YW5kIGM2NWVhMTZkYmNhZiAoIng4Ni9wMm06IGRvbid0CmFzc2VydCB0aGF0
IHRoZSBwYXNzZWQgaW4gTUZOIG1hdGNoZXMgZm9yIGEgcmVtb3ZlIikuCgot
LS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLmMKKysrIGIveGVuL2FyY2gveDg2
L21tL3AybS5jCkBAIC04MDUsNyArODA1LDggQEAgcDJtX3JlbW92ZV9wYWdl
KHN0cnVjdCBwMm1fZG9tYWluICpwMm0sCiAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAmY3VyX29yZGVyLCBOVUxMKTsKIAog
ICAgICAgICBpZiAoIHAybV9pc192YWxpZCh0KSAmJgotICAgICAgICAgICAg
ICghbWZuX3ZhbGlkKG1mbikgfHwgIW1mbl9lcShtZm5fYWRkKG1mbiwgaSks
IG1mbl9yZXR1cm4pKSApCisgICAgICAgICAgICAgKCFtZm5fdmFsaWQobWZu
KSB8fCB0ID09IHAybV9tbWlvX2RpcmVjdCB8fAorICAgICAgICAgICAgICAh
bWZuX2VxKG1mbl9hZGQobWZuLCBpKSwgbWZuX3JldHVybikpICkKICAgICAg
ICAgICAgIHJldHVybiAtRUlMU0VROwogCiAgICAgICAgIGkgKz0gKDFVTCA8
PCBjdXJfb3JkZXIpIC0KQEAgLTkxMiw3ICs5MTMsNyBAQCBndWVzdF9waHlz
bWFwX2FkZF9lbnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgaWYgKCBwMm1f
aXNfZm9yZWlnbih0KSApCiAgICAgICAgIHJldHVybiAtRUlOVkFMOwogCi0g
ICAgaWYgKCAhbWZuX3ZhbGlkKG1mbikgKQorICAgIGlmICggIW1mbl92YWxp
ZChtZm4pIHx8IHQgPT0gcDJtX21taW9fZGlyZWN0ICkKICAgICB7CiAgICAg
ICAgIEFTU0VSVF9VTlJFQUNIQUJMRSgpOwogICAgICAgICByZXR1cm4gLUVJ
TlZBTDsKQEAgLTk1Niw3ICs5NTcsNyBAQCBndWVzdF9waHlzbWFwX2FkZF9l
bnRyeShzdHJ1Y3QgZG9tYWluICpkCiAgICAgICAgIH0KICAgICAgICAgaWYg
KCBwMm1faXNfc3BlY2lhbChvdCkgKQogICAgICAgICB7Ci0gICAgICAgICAg
ICAvKiBEb24ndCBwZXJtaXQgdW5tYXBwaW5nIGdyYW50L2ZvcmVpZ24gdGhp
cyB3YXkuICovCisgICAgICAgICAgICAvKiBEb24ndCBwZXJtaXQgdW5tYXBw
aW5nIGdyYW50L2ZvcmVpZ24vZGlyZWN0LU1NSU8gdGhpcyB3YXkuICovCiAg
ICAgICAgICAgICBkb21haW5fY3Jhc2goZCk7CiAgICAgICAgICAgICBwMm1f
dW5sb2NrKHAybSk7CiAgICAgICAgICAgICAKQEAgLTEzNjQsOCArMTM2NSw4
IEBAIGludCBzZXRfbW1pb19wMm1fZW50cnkoc3RydWN0IGRvbWFpbiAqZCwK
ICAqICAgIG9yZGVyKzEgIGZvciBjYWxsZXIgdG8gcmV0cnkgd2l0aCBvcmRl
ciAoZ3VhcmFudGVlZCBzbWFsbGVyIHRoYW4KICAqICAgICAgICAgICAgIHRo
ZSBvcmRlciB2YWx1ZSBwYXNzZWQgaW4pCiAgKi8KLWludCBjbGVhcl9tbWlv
X3AybV9lbnRyeShzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdm
bl9sLCBtZm5fdCBtZm4sCi0gICAgICAgICAgICAgICAgICAgICAgICAgdW5z
aWduZWQgaW50IG9yZGVyKQorc3RhdGljIGludCBjbGVhcl9tbWlvX3AybV9l
bnRyeShzdHJ1Y3QgZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbl9sLAor
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtZm5fdCBtZm4sIHVu
c2lnbmVkIGludCBvcmRlcikKIHsKICAgICBpbnQgcmMgPSAtRUlOVkFMOwog
ICAgIGdmbl90IGdmbiA9IF9nZm4oZ2ZuX2wpOwpAQCAtMjc2Niw3ICsyNzY3
LDkgQEAgaW50IHhlbm1lbV9hZGRfdG9fcGh5c21hcF9vbmUoCiAKICAgICAv
KiBSZW1vdmUgcHJldmlvdXNseSBtYXBwZWQgcGFnZSBpZiBpdCB3YXMgcHJl
c2VudC4gKi8KICAgICBwcmV2X21mbiA9IGdldF9nZm4oZCwgZ2ZuX3goZ3Bm
biksICZwMm10KTsKLSAgICBpZiAoIG1mbl92YWxpZChwcmV2X21mbikgKQor
ICAgIGlmICggcDJtdCA9PSBwMm1fbW1pb19kaXJlY3QgKQorICAgICAgICBy
YyA9IC1FUEVSTTsKKyAgICBlbHNlIGlmICggbWZuX3ZhbGlkKHByZXZfbWZu
KSApCiAgICAgewogICAgICAgICBpZiAoIGlzX3NwZWNpYWxfcGFnZShtZm5f
dG9fcGFnZShwcmV2X21mbikpICkKICAgICAgICAgICAgIC8qIFNwZWNpYWwg
cGFnZXMgYXJlIHNpbXBseSB1bmhvb2tlZCBmcm9tIHRoaXMgcGh5cyBzbG90
LiAqLwotLS0gYS94ZW4vYXJjaC94ODYvbW0vcDJtLXBvZC5jCisrKyBiL3hl
bi9hcmNoL3g4Ni9tbS9wMm0tcG9kLmMKQEAgLTEyOTksMTcgKzEyOTksMTcg
QEAgZ3Vlc3RfcGh5c21hcF9tYXJrX3BvcHVsYXRlX29uX2RlbWFuZChzdAog
CiAgICAgICAgIHAybS0+Z2V0X2VudHJ5KHAybSwgZ2ZuX2FkZChnZm4sIGkp
LCAmb3QsICZhLCAwLCAmY3VyX29yZGVyLCBOVUxMKTsKICAgICAgICAgbiA9
IDFVTCA8PCBtaW4ob3JkZXIsIGN1cl9vcmRlcik7Ci0gICAgICAgIGlmICgg
cDJtX2lzX3JhbShvdCkgKQorICAgICAgICBpZiAoIG90ID09IHAybV9wb3B1
bGF0ZV9vbl9kZW1hbmQgKQorICAgICAgICB7CisgICAgICAgICAgICAvKiBD
b3VudCBob3cgbWFueSBQb0QgZW50cmllcyB3ZSdsbCBiZSByZXBsYWNpbmcg
aWYgc3VjY2Vzc2Z1bCAqLworICAgICAgICAgICAgcG9kX2NvdW50ICs9IG47
CisgICAgICAgIH0KKyAgICAgICAgZWxzZSBpZiAoIG90ICE9IHAybV9pbnZh
bGlkICYmIG90ICE9IHAybV9tbWlvX2RtICkKICAgICAgICAgewogICAgICAg
ICAgICAgUDJNX0RFQlVHKCJnZm5fdG9fbWZuIHJldHVybmVkIHR5cGUgJWQh
XG4iLCBvdCk7CiAgICAgICAgICAgICByYyA9IC1FQlVTWTsKICAgICAgICAg
ICAgIGdvdG8gb3V0OwogICAgICAgICB9Ci0gICAgICAgIGVsc2UgaWYgKCBv
dCA9PSBwMm1fcG9wdWxhdGVfb25fZGVtYW5kICkKLSAgICAgICAgewotICAg
ICAgICAgICAgLyogQ291bnQgaG93IG1hbiBQb0QgZW50cmllcyB3ZSdsbCBi
ZSByZXBsYWNpbmcgaWYgc3VjY2Vzc2Z1bCAqLwotICAgICAgICAgICAgcG9k
X2NvdW50ICs9IG47Ci0gICAgICAgIH0KICAgICB9CiAKICAgICAvKiBOb3cs
IGFjdHVhbGx5IGRvIHRoZSB0d28td2F5IG1hcHBpbmcgKi8KLS0tIGEveGVu
L2NvbW1vbi9tZW1vcnkuYworKysgYi94ZW4vY29tbW9uL21lbW9yeS5jCkBA
IC0zMzIsNyArMzMyLDcgQEAgaW50IGd1ZXN0X3JlbW92ZV9wYWdlKHN0cnVj
dCBkb21haW4gKmQsCiAgICAgfQogICAgIGlmICggcDJtdCA9PSBwMm1fbW1p
b19kaXJlY3QgKQogICAgIHsKLSAgICAgICAgcmMgPSBjbGVhcl9tbWlvX3Ay
bV9lbnRyeShkLCBnbWZuLCBtZm4sIFBBR0VfT1JERVJfNEspOworICAgICAg
ICByYyA9IC1FUEVSTTsKICAgICAgICAgZ290byBvdXRfcHV0X2dmbjsKICAg
ICB9CiAjZWxzZQpAQCAtMTg4OCw2ICsxODg4LDE1IEBAIGludCBjaGVja19n
ZXRfcGFnZV9mcm9tX2dmbihzdHJ1Y3QgZG9tYWkKICAgICAgICAgcmV0dXJu
IC1FQUdBSU47CiAgICAgfQogI2VuZGlmCisjaWZkZWYgQ09ORklHX1g4Ngor
ICAgIGlmICggcDJtdCA9PSBwMm1fbW1pb19kaXJlY3QgKQorICAgIHsKKyAg
ICAgICAgaWYgKCBwYWdlICkKKyAgICAgICAgICAgIHB1dF9wYWdlKHBhZ2Up
OworCisgICAgICAgIHJldHVybiAtRVBFUk07CisgICAgfQorI2VuZGlmCiAK
ICAgICBpZiAoICFwYWdlICkKICAgICAgICAgcmV0dXJuIC1FSU5WQUw7Ci0t
LSBhL3hlbi9pbmNsdWRlL2FzbS14ODYvcDJtLmgKKysrIGIveGVuL2luY2x1
ZGUvYXNtLXg4Ni9wMm0uaApAQCAtMTU2LDcgKzE1Niw4IEBAIHR5cGVkZWYg
dW5zaWduZWQgaW50IHAybV9xdWVyeV90OwogCiAvKiBUeXBlcyBlc3RhYmxp
c2hlZC9jbGVhbmVkIHVwIHZpYSBzcGVjaWFsIGFjY2Vzc29ycy4gKi8KICNk
ZWZpbmUgUDJNX1NQRUNJQUxfVFlQRVMgKFAyTV9HUkFOVF9UWVBFUyB8IFwK
LSAgICAgICAgICAgICAgICAgICAgICAgICAgIHAybV90b19tYXNrKHAybV9t
YXBfZm9yZWlnbikpCisgICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1f
dG9fbWFzayhwMm1fbWFwX2ZvcmVpZ24pIHwgXAorICAgICAgICAgICAgICAg
ICAgICAgICAgICAgcDJtX3RvX21hc2socDJtX21taW9fZGlyZWN0KSkKIAog
LyogVmFsaWQgdHlwZXMgbm90IG5lY2Vzc2FyaWx5IGFzc29jaWF0ZWQgd2l0
aCBhICh2YWxpZCkgTUZOLiAqLwogI2RlZmluZSBQMk1fSU5WQUxJRF9NRk5f
VFlQRVMgKFAyTV9QT0RfVFlQRVMgICAgICAgICAgICAgICAgICBcCkBAIC02
MjcsMTkgKzYyOCw5IEBAIGludCBwMm1fZmluaXNoX3R5cGVfY2hhbmdlKHN0
cnVjdCBkb21haW4KIGludCBwMm1faXNfbG9nZGlydHlfcmFuZ2Uoc3RydWN0
IHAybV9kb21haW4gKiwgdW5zaWduZWQgbG9uZyBzdGFydCwKICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBlbmQpOwogCi0jaWZk
ZWYgQ09ORklHX0hWTQogLyogU2V0IG1taW8gYWRkcmVzc2VzIGluIHRoZSBw
Mm0gdGFibGUgKGZvciBwYXNzLXRocm91Z2gpICovCiBpbnQgc2V0X21taW9f
cDJtX2VudHJ5KHN0cnVjdCBkb21haW4gKmQsIGdmbl90IGdmbiwgbWZuX3Qg
bWZuLAogICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgb3Jk
ZXIpOwotaW50IGNsZWFyX21taW9fcDJtX2VudHJ5KHN0cnVjdCBkb21haW4g
KmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLCBtZm5fdCBtZm4sCi0gICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IG9yZGVyKTsKLSNlbHNlCi1z
dGF0aWMgaW5saW5lIGludCBjbGVhcl9tbWlvX3AybV9lbnRyeShzdHJ1Y3Qg
ZG9tYWluICpkLCB1bnNpZ25lZCBsb25nIGdmbiwKLSAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIG1mbl90IG1mbiwgdW5zaWduZWQg
aW50IG9yZGVyKQotewotICAgIHJldHVybiAtRUlPOwotfQotI2VuZGlmCiAK
IC8qIFNldCBpZGVudGl0eSBhZGRyZXNzZXMgaW4gdGhlIHAybSB0YWJsZSAo
Zm9yIHBhc3MtdGhyb3VnaCkgKi8KIGludCBzZXRfaWRlbnRpdHlfcDJtX2Vu
dHJ5KHN0cnVjdCBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcgZ2ZuLAo=

--=separator--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:38:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:38:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176234.320694 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMhd-0002Ib-Ta; Wed, 01 Sep 2021 09:38:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176234.320694; Wed, 01 Sep 2021 09:38:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMhd-0002IU-Pr; Wed, 01 Sep 2021 09:38:41 +0000
Received: by outflank-mailman (input) for mailman id 176234;
 Wed, 01 Sep 2021 09:38:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4/yp=NX=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mLMhc-0002IO-OD
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 09:38:40 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 62c9ef60-0b08-11ec-adbf-12813bfff9fa;
 Wed, 01 Sep 2021 09:38:38 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4A07B1FB;
 Wed,  1 Sep 2021 02:38:38 -0700 (PDT)
Received: from [10.57.13.24] (unknown [10.57.13.24])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0E67D3F766;
 Wed,  1 Sep 2021 02:38:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62c9ef60-0b08-11ec-adbf-12813bfff9fa
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Ash Wilding <ash.j.wilding@gmail.com>, Ian Jackson <iwj@xenproject.org>
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <4a5ac822-3078-de0e-8f84-6881d47ad3eb@xen.org>
 <dd9859ec-a3a9-6a74-bdae-fde092f26ac0@arm.com>
 <e8bdb065-9b46-e0d8-1fcf-7bba38e02f9e@xen.org>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <8974fc82-94c1-ef85-153c-0923f5faf331@arm.com>
Date: Wed, 1 Sep 2021 11:38:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <e8bdb065-9b46-e0d8-1fcf-7bba38e02f9e@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Julien,

On 06.08.2021 13:12, Julien Grall wrote:
> 
> 
> On 29/07/2021 12:47, Michal Orzel wrote:
>> Hi Julien,
> 
> Hi Michal,
> 
>> On 29.07.2021 13:20, Julien Grall wrote:
>>> Hi Michal,
>>>
>>> On 29/07/2021 11:42, Michal Orzel wrote:
>>>> According to ARMv8A architecture, AArch64 registers
>>>> are 64bit wide even though in many cases the upper
>>>> 32bit is reserved. Therefore there is no need for
>>>> function vreg_emulate_sysreg32 on arm64. This means
>>>> that we can have just one function vreg_emulate_sysreg
>>>> using new function pointer:
>>>> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>>>>                                 register_t *r, bool read);
>>>>
>>>> Modify vreg_emulate_cp32 to use the new function pointer
>>>> as well.
>>>>
>>>> This change allows to properly use 64bit registers in AArch64
>>>> state and in case of AArch32 the upper 32 bits of AArch64
>>>> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
>>>
>>> What you wrote only says that the bits are ignored. It doesn't say whether the bits will be 0.
>>>
>>> They are probably, but as I wrote yesterday, I couldn't confirm it.
>>>
>> Should I then remove this part of the commit or write below?:
>> "We can assume that those bits will be 0 but the architecture
>> reference manual does not clarify this."
> 
> There was some back and forth on security@xen.org about this. I will summarizing the discussion here as we considered this was a just a bug.
> 
> I wasn't looking at the correct section in the Arm Arm. There is a paragraph clearly describing the expected behavior in a different section (thanks Ash for the pointer!). Per section D1.19.2 in DDI 0487F.c:
> 
> "
> If the general-purpose register was accessible from AArch32 state
> 
> The upper 32 bits either become zero, or hold the value that the same architectural register held before any AArch32 execution. The choice between these two options is IMPLEMENTATION DEFINED, and might vary dynamically within an implementation. Correspondingly, software must
> regard the value as being a CONSTRAINED UNPREDICTABLE choice between these two values.
> 
> This behavior applies regardless of whether any execution occurred at the Exception level that was using AArch32. That is, this behavior applies even if AArch32 state was entered by an exception
> return from AArch64 state, and another exception was immediately taken to AArch64 state without any instruction execution in AArch32 state.
> "
> 
> So we can't assume the top 32-bits are zeroed unless the hypervisor ensured they were. Today, we don't have that guarantee in Xen.
> 
> This needs to be fixed. The two approachs we discussed are:
>    1) Update set_user_reg() to zero the top 32-bit. We have a couple of places using directly the fields xN. So we would need to switch them to use set_user_reg()
>    2) Only saving/restoring the bottom 32-bit when entering/leaving the hypervisor.
> 
> At the moment, my preference goes towards the latter because we don't risk to introduce new place where set_user_reg() is not used.
> 
> I have quickly hack the entry path. This would look like:
> 
> diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
> index fc3811ad0ad5..65e24c88b059 100644
> --- a/xen/arch/arm/arm64/entry.S
> +++ b/xen/arch/arm/arm64/entry.S
> @@ -111,6 +111,11 @@
>   */
>          .macro  entry, hyp, compat, save_x0_x1=1
>          sub     sp, sp, #(UREGS_SPSR_el1 - UREGS_LR) /* CPSR, PC, SP, LR */
> +        .if \compat == 1 /* AArch32 mode */
> +        /* Clobber the top 32-bit of the registers */
> +        mov    w0, w0
> +        mov    w1, w1
> +        .endif
>          push    x28, x29
>          push    x26, x27
>          push    x24, x25
> 
> I haven't looked whether this can be optimized or the exit path would be easier to modify.
> 
> Anyway, this is not a new bug so I would be fine to get this patch merged first. Although, I think this wants to be fixed for xen 4.16 (CCing Ian to track it).
> 
> I will try to find sometimes in the next couple of weeks to fix it and have another review of this patch.
> 

As the 4.16 release is getting closer I wanted to ask whether you need help with creating a pre-work patch so that this patch can be merged.
I believe this patch wants to be merged for 4.16 as the other sysreg related patches are merged already, so I'm offering a help.

> Cheers,
> 

Cheers,
Michal


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:53:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:53:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176242.320705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMvg-0004yM-6b; Wed, 01 Sep 2021 09:53:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176242.320705; Wed, 01 Sep 2021 09:53:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMvg-0004yF-2X; Wed, 01 Sep 2021 09:53:12 +0000
Received: by outflank-mailman (input) for mailman id 176242;
 Wed, 01 Sep 2021 09:53:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLMve-0004y9-Im
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 09:53:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 69716eea-0b0a-11ec-adc0-12813bfff9fa;
 Wed, 01 Sep 2021 09:53:09 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-dFfHWNfUN2y5W7OlfD3puw-1; Wed, 01 Sep 2021 11:53:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4941.eurprd04.prod.outlook.com (2603:10a6:803:5a::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 1 Sep
 2021 09:53:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 09:53:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0061.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.12 via Frontend Transport; Wed, 1 Sep 2021 09:53:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69716eea-0b0a-11ec-adc0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630489988;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z1EDBZu0WpUcHKjJqurIDGUHD3q1CvmzH/q+2dgnymM=;
	b=UNOuXtzX1WfS4Itl56Dniar6aziQvRhPOOwbMiGAejyfT7/wL/Ubbk25sJyRzmhSmE6PHv
	yQ4E2CGTUKEb5NVdthKesxgynyJf9aTliPgx9MVWgIwPhz2rWvYw0vqIAYI20QdgjtqdB+
	3PSmNuEiYrsNHvU3Q6CUg9XjQHDaWQ0=
X-MC-Unique: dFfHWNfUN2y5W7OlfD3puw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kOi5t8fkV7mE3oEPg24gICc669mhOlN2DGhKYKFInlEE5E5f3L7o2Yb+t+z7q3qvxomxmiDN6DtDXCux36Zd/3v/ErklmaWGch1hWV3GV47QBNYUu8uftl1noVPv3g+JyVx97pvYxIso4i5YxraU/lOdiCoHNXk/ZOiyqjXH7qWL5GdZJq1u+bZbLuXt/rJtM6tXz8Bb0pe+gd46r4CuUpznztHqEbfEXaVrnykhkhVmT25gM1xz5nsbLC/TzRFHrwkXMBTJJOwn4gxg0LFTo8P64BCO5hdYZm9JLA9jn6Y/n3xaUVjbysMiEeaKljmqYJHgeVeGkaYGhQr+Eaeixw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AT3M3+5f9uqBf3l+DjU+HB2tK3g3eHcn74+2vK7NM2E=;
 b=lQyb7C5MKqEA1yrFo9gCTwm/ewjumtyAPB9SBH0TStIruQaZ+OyYE9AqXFn/+X39URQG6FouLPWbVPQ1Sv2c21YVl6lNqOwXgQ1b+ZpCyTclEOv+VU7SLwRlGQ07hp+5LgzBwNUXFp5mJWUttIZcHIwk1Ufd9A1qdwGkjUCtlZ0DRLdlg5YjXhCKJbHCwAgacQTeaOTmiO4QJ8hd579Zjb/fXZuwHr7wYagmt4QLE8DoY7Vr5VO/dYLJ63LGz23NXIpmLDQG8hMEaiP7Fw6WGqLczLnNuf+CGBnH54OGxolpQ2UYIBxgfO1F8Ldouv0+289P7Ckstf3/kTen9rcQXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <YS8+72EY4FrvSSIx@Air-de-Roger>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ca3a9559-b95a-3b32-781d-2a2abfa50dae@suse.com>
Date: Wed, 1 Sep 2021 11:53:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YS8+72EY4FrvSSIx@Air-de-Roger>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0061.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2f4b8d0a-9d7b-4713-6a83-08d96d2e4bdb
X-MS-TrafficTypeDiagnostic: VI1PR04MB4941:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB494103A718F87669ED3DE40EB3CD9@VI1PR04MB4941.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eC9GN7Dwb2K0It89krZxNDB32XENRWHOYuEPmP3LUra9psOcaSi+8YDuX7gqOhyB7qvyPNoC7f1XCJ51iRN0xPlZatzgIme+WiYUgzX5GPEftKOmmF41AtyFF/RwqxGNDBhCOBmHhlKx4emEzqJ7sR1Rk/Z3fcJPT2mCCNlCQlh6FwSxe9VadpNRtTX+XNgOeBEiS3e8DtNTkJtHWony2+YNmHkbzybIisnV5AG/A1Uvg08yhGvxUXkHuZuxRh4zh1FYTlkneMUKTzlEHdMUI4Gb9a69wNyUcwyLbPMzqnoXbA1ycYNOL6NHlrweDUAMXNRYWOFP5XYoHqoIUD5qKSTO4GkOK2XrIcXaIluVSQIkVTOjPbdZEAwy1uj8Hipnfox0J5ul/6pUqxQz6aGSo7FUejnSTD6DyP6WONs2VGJNQrGGOjlpmpeG5DyqgLiKUxx5GWZluEkYF7jKwZbPCjmSSYkjLkDDakZr2ibAH96Y4fZg/aYy+RYTFK1xZI+HlfZtswGhQhxFQHXVUbtk1qxl4G3J9OrhLWIAycJotEfU3kIFSqK8yZCm3TTuz3b1xCr3iK4CqcOBe1BntJSsAzij3f7uneEJBr+x+9h/TcqHhLr+pF+G/FDXXiqpNK6mFnOddfJUM98FBpi3efJ1DO9LeKu0ShP+OM10aSX4//LT6dxnUhvrDrMeW35QcXid7HrDUzRyAdesf/T3w/QPI/EojMsbJCekYynY4QRX3VA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(956004)(31696002)(26005)(83380400001)(2616005)(186003)(53546011)(66556008)(2906002)(38100700002)(66476007)(86362001)(4326008)(5660300002)(8676002)(6916009)(8936002)(316002)(16576012)(36756003)(31686004)(6486002)(508600001)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?2OhPcru4d2grG4422h733Ol2f5dSOxofVAdgTIvYWv8Q1PBMQfOkaEls9j2g?=
 =?us-ascii?Q?u+GflmQVa+ejP0iqBD1/L/kLcBg7NnV02G5PYEdhQoP7pjNs62WpPebToxZe?=
 =?us-ascii?Q?lhlqs0DNxq15cxg1fIBR3xRMfcN/64kBrKyy2PLFcGjByi9vjL6tbclezImI?=
 =?us-ascii?Q?J5HjjBUAbfyVbUwWwDWLtcpx3tcFPgrs0DIXV+bIaKXZ1qPEWze8Zp8DcBiD?=
 =?us-ascii?Q?57tIAnu1CF32O9Qd/RrYBCpHUtKfVuLwAU0JxONIEYvT35Ru7dPI93Zbxpmd?=
 =?us-ascii?Q?q/SHOl7l8KXxQIyyR+btlVokmJypx6SQk73jC9fmFaDJK/Rukarqf1mstOAg?=
 =?us-ascii?Q?/yvJdPXk56IJ8KkGogYGosUTHAM9vAOAx4Q3w+MzUZLG+eAq97YNvDcqJGeo?=
 =?us-ascii?Q?T/zlntkC73EfCli3z3P3tqr0IW/he+CeKfm6sE24myOrGbNp4QtNXXCmxrkC?=
 =?us-ascii?Q?L6ZeQ4qEzl4HrxtfOJc05jxi4MsDnQFfLJN53arpwEIWRe66yIfruqxw3NiT?=
 =?us-ascii?Q?vdv3QD1aTaOrpRGNrb0msZL/9uDTjCCRzTUtsmkegGXsr7Ff+9ZpV78DotPC?=
 =?us-ascii?Q?WrAXUkGj4G5szt45ElWio3KQZ6aVs96t2HaRGNfwmxeKV+uMKg8MzAxtylSl?=
 =?us-ascii?Q?NM6y2wbNrYEgjLoUlNu3sE9E3xEnc8Gf5pFealmR9K8deXkzTbQSwLHz74ie?=
 =?us-ascii?Q?urh1rSa16AUKzXhRkU9hwo+85ef2L/BAGf0E8X0o5ShcJnPyKAkg53rDhiy9?=
 =?us-ascii?Q?qFrpkz22iz5RIg6YfYZgqjdxm1TO2VqpGGkjciSG0mRYmnQAUUiuWQNOuJjh?=
 =?us-ascii?Q?FwOVDZjjzO9eBkZWNw71SB7wm1kUOw9L6z7MlR1aF/S3dCoBZFYUPhKCHc+m?=
 =?us-ascii?Q?/7fbn+h2iwNJ88nYW1kJ7gO8v2T9621y37gGfS19r5jGH3luW2DK07NGFn0G?=
 =?us-ascii?Q?ceDf6WJ57tDxQFVglZ9ruiK/gnpbPB0Qy16+p8FMNYsoBgvVR2t4vh5R7aor?=
 =?us-ascii?Q?pbtkjuu7soFyHImm5IlfK93F8edjNTdwRKyG47Zli+obl00XOjTkxghJ9VUp?=
 =?us-ascii?Q?nfMHbBTB4n7wQQpil4aR9bdRU59qbmKasHw2qR92WEE4GCCYEobue6pTp8Gs?=
 =?us-ascii?Q?MEJoLQKuWtJv5AI3KQTWgEg6QS43256DCFt/kcc3nVRw2p+k+5vi50VZG/6L?=
 =?us-ascii?Q?6YI7RD8s+41lVOPILhRP4WO2st7wlmQ0KLjJQBObpG7IXfPKRcN+vCrzZwVq?=
 =?us-ascii?Q?GJrBuMXR/K1ch/4ZHpic+f/fwDRjCtuqrtkLgaFy6q78cnwxxRia0YD32TAR?=
 =?us-ascii?Q?OSYGGS4l9K8Uzk0FeMa0yZnI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f4b8d0a-9d7b-4713-6a83-08d96d2e4bdb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 09:53:05.9119
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aWyzBaQ+tWpHjtK26mhZaY13nCRp3OblQzNH0pRVS17/dKr513B+aBu2pMppvD5ymovagzHSvPZxjkBpgtaPxg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4941

On 01.09.2021 10:50, Roger Pau Monn=C3=A9 wrote:
> On Tue, Aug 31, 2021 at 05:38:49PM +0200, Jan Beulich wrote:
>> On 31.08.2021 17:25, Andrew Cooper wrote:
>>> On 31/08/2021 14:26, Jan Beulich wrote:
>>>> On 31.08.2021 15:16, Andrew Cooper wrote:
>>>>> On 30/08/2021 14:02, Jan Beulich wrote:
>>>>>> Further permit "access" to differ in the "executable" attribute. Whi=
le
>>>>>> ideally only ROM regions would get mapped with X set, getting there =
is
>>>>>> quite a bit of work. Therefore, as a temporary measure, permit X to
>>>>>> vary. For Dom0 the more permissive of the types will be used, while =
for
>>>>>> DomU it'll be the more restrictive one.
>>>>> Split behaviour between dom0 and domU based on types alone cannot
>>>>> possibly be correct.
>>>> True, but what do you do.
>>>>
>>>>> DomU's need to execute ROMs too, and this looks like will malfunction=
 if
>>>>> a ROM ends up in the region that HVMLoader relocated RAM from.
>>>>>
>>>>> As this is a temporary bodge emergency bugfix, don't try to be clever=
 -
>>>>> just take the latest access.
>>>> And how do we know that that's what is going to work?
>>>
>>> Because it's the pre-existing behaviour.
>>
>> Valid point. But for the DomU case there simply has not been any
>> pre-existing behavior. Hence my desire to be restrictive initially
>> there.
>>
>>>>  We should
>>>> strictly accumulate for Dom0. And what we do for DomU is moot for
>>>> the moment, until PCI passthrough becomes a thing for PVH. Hence
>>>> I've opted to be restrictive there - I'd rather see things break
>>>> (and getting adjusted) when this future work actually gets carried
>>>> out, than leave things permissive for no-one to notice that it's
>>>> too permissive, leading to an XSA.
>>>
>>> Restricting execute permissions is something unique to virt.=C2=A0 It d=
oesn't
>>> exist in a non-virtualised system, as I and D side reads are
>>> indistinguishable outside of the core.
>>>
>>> Furthermore, it is inexpressible on some systems/configurations.
>>>
>>> Introspection is the only technology which should be restricting execut=
e
>>> permissions in the p2m, and only when it takes responsibility for
>>> dealing with the fallout.
>>
>> IOW are you saying that the calls to set_identity_p2m_entry()
>> (pre-dating XSA-378) were wrong to use p2m_access_rw? Because that's
>> what's getting the way here.
>=20
> I did wonder this before, because I saw some messages on a couple of
> systems about mappings override, and I'm not sure why we need to use
> p2m_access_rw. My first thought was to suggest to switch to use the
> default access type for the domain, like set_mmio_p2m_entry does.
>=20
> I have to admit I'm not sure I see the point of preventing execution,
> but it's possible I'm missing something.

Well, what good can come from allowing execution from, say, the
IO-APIC or LAPIC pages? Or other MMIO-mapped register space? Insn
fetches might even trip bad hardware behavior in such a case by
being the wrong granularity. It's imo really only ROM space which
ought to have execution permitted.

The issue isn't just with execution, though, and as a result I may
need to change the logic here to also include at least W. As of
one of the XSA-378 changes we may now pass just p2m_access_r to
iommu_identity_mapping(), if the ACPI tables on an AMD system were
saying so. (We may also pass p2m_access_w, but I sincerely hope no
firmware would specify write but no read access.)

Similarly in "IOMMU/x86: restrict IO-APIC mappings for PV Dom0" I
now pass p2m_access_r to set_identity_p2m_entry().

I suppose an underlying issue is the mixed purpose of using
p2m_access_*, which possibly has been against the intentions in the
first place. We cannot, for example, express r/o access to an MMIO
page without using p2m_access_r (or p2m_access_rx), as there's no
suitable p2m type to express this via type alone. We may need to
split p2m_mmio_direct into multiple types (up to 7), I guess, if
we wanted to remove this (ab)use of p2m_access_*.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:53:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:53:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176246.320716 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMwC-0005TN-F2; Wed, 01 Sep 2021 09:53:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176246.320716; Wed, 01 Sep 2021 09:53:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMwC-0005TG-C0; Wed, 01 Sep 2021 09:53:44 +0000
Received: by outflank-mailman (input) for mailman id 176246;
 Wed, 01 Sep 2021 09:53:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0rUu=NX=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLMwA-0005Sz-LP
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 09:53:42 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 923e60a0-1d40-42e7-946d-0916f4d96220;
 Wed, 01 Sep 2021 09:53:41 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A1C98202A9;
 Wed,  1 Sep 2021 09:53:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 5360A13A45;
 Wed,  1 Sep 2021 09:53:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id sOO0EaRNL2G0IAAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 01 Sep 2021 09:53:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 923e60a0-1d40-42e7-946d-0916f4d96220
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630490020; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=aA6sPNMThuGUNYnDlunOAMyTa2yY+DaSnBdOYNd2FZY=;
	b=SeLuetNliD5GxhPHaMqcoEt2BmM/HT1trHCMQ5aac751U5VCy1NIjeb3saIDbKMUS1Apbb
	0IFRCp5PxCSZsE/VHKgZhRRCbW8XIHOLCtoKbT21kcW0d/P+sySIPMhx+PyrdMwYho6Mcl
	CRCbjCM/BZPrnVx7rtK12EyTPf1F+uE=
To: Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <20210831073039.7101-1-jgross@suse.com>
 <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] Config: use Mini-OS master for xen-unstable
Message-ID: <d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
Date: Wed, 1 Sep 2021 11:53:39 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="nffqn2zrwFK8ViexzHzIWpM3CQjmypOq3"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--nffqn2zrwFK8ViexzHzIWpM3CQjmypOq3
Content-Type: multipart/mixed; boundary="qDn4ONRi1pYHvLeWVtpyF4cKY3t3VuEvq";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
Message-ID: <d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
Subject: Re: [PATCH] Config: use Mini-OS master for xen-unstable
References: <20210831073039.7101-1-jgross@suse.com>
 <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
In-Reply-To: <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>

--qDn4ONRi1pYHvLeWVtpyF4cKY3t3VuEvq
Content-Type: multipart/mixed;
 boundary="------------909B2A3919721982D934BD8C"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------909B2A3919721982D934BD8C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 01.09.21 10:59, Jan Beulich wrote:
> On 31.08.2021 09:30, Juergen Gross wrote:
>> In order to get Mini-OS master branch tested by OSStest, use it for
>> xen-unstable.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> There are some Mini-OS patches pending, of which at least the Mini-OS
>> netfront fix should be committed as it will be required for qemu-stubd=
om
>> to work.
>> ---
>>   Config.mk | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/Config.mk b/Config.mk
>> index 4d723eec1d..d4e8ced104 100644
>> --- a/Config.mk
>> +++ b/Config.mk
>> @@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?=3D git://xenbits.xen.org/min=
i-os.git
>>   endif
>>   OVMF_UPSTREAM_REVISION ?=3D b37cfdd2807181aed2fee1e17bd7ec1190db266a=

>>   QEMU_UPSTREAM_REVISION ?=3D master
>> -MINIOS_UPSTREAM_REVISION ?=3D 051b87bb9c19609976fb038f386920e1ce5454c=
5
>> +MINIOS_UPSTREAM_REVISION ?=3D master
>=20
> I'm afraid this, effectively reverting 82c3d15c903a ("minios: Revert re=
cent
> change and revert to working minios"), would set us up for the same pro=
blem
> again that was dealt with there. IOW you're re-proposing a change of yo=
urs
> (e013e8514389 ["config: use mini-os master for unstable"]) which was
> subsequently reverted, without spelling out what has since changed.

How do you suggest to proceed then?

Will every Mini-OS commit require to be followed by a Xen commit then,
which will either result in a test success, or in a revert of that Xen
commit?

It should be noted that the presumed interface change in Mini-OS causing
the failure in OSStest last year has been reverted by me.


Juergen

--------------909B2A3919721982D934BD8C
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------909B2A3919721982D934BD8C--

--qDn4ONRi1pYHvLeWVtpyF4cKY3t3VuEvq--

--nffqn2zrwFK8ViexzHzIWpM3CQjmypOq3
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEvTaMFAwAAAAAACgkQsN6d1ii/Ey8n
BAf/SHfDRN/F/Dz+lUFMTTxNvOB8w2/qMorYOPAKT/kAKDmyC/UGUeNjKALUdGEt3kdd6PrQVczs
9JiaeeL/4OJLv2pnccrfXu/wwABZatvgy1VPyr+siI6jV1HxGGhEgDXsZ5bgyIIGxBfCVQaEyE57
M4PZG1bOZ5oPmtj5Bz5UA2it8xs1yMwqRcr11k7XbnCA0P4oHOC+bZew70P/U6+/Qp1Ku05aeBtz
FUjoOVrrm3u6X8gIKcUu6StmB2rBPQbpWAXNNnu6ay8XsYoVnBp59OW2jQ474bTOcv+X11L//bdJ
/KxJH+rmgXGe5Mfw+B3WmMqIBZhEFxShqmZrbDAS8w==
=zCn0
-----END PGP SIGNATURE-----

--nffqn2zrwFK8ViexzHzIWpM3CQjmypOq3--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 09:56:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 09:56:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176255.320726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMyQ-0006G9-1Z; Wed, 01 Sep 2021 09:56:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176255.320726; Wed, 01 Sep 2021 09:56:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLMyP-0006G2-Uk; Wed, 01 Sep 2021 09:56:01 +0000
Received: by outflank-mailman (input) for mailman id 176255;
 Wed, 01 Sep 2021 09:56:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLMyP-0006Fq-03
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 09:56:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6655b2a7-e3b7-4fb0-9e6d-2cc730b15527;
 Wed, 01 Sep 2021 09:56:00 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-3lGGkDqjPbiQOR7DLdaJ2g-1; Wed, 01 Sep 2021 11:55:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4845.eurprd04.prod.outlook.com (2603:10a6:803:51::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.21; Wed, 1 Sep
 2021 09:55:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 09:55:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Wed, 1 Sep 2021 09:55:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6655b2a7-e3b7-4fb0-9e6d-2cc730b15527
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630490159;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FGtjZiIFf5R/oYLpBhLuFfUgjNszBMgjxxQb52D/ECA=;
	b=IpegrnIB96Y939kQ3nJiC+2A/bQu/mkswGUcgxOPHnsHmUscZQrOC9n2owxl10Jh1wMixJ
	5sJCpTfGiuQB/ch+OAMB5oO26q3fenY6LHcC5kNTSpJIsmAnNo1C6hZjT8/mLKNs54CHef
	GqkjPFbhj8vy4lIM0S9qi2fYFcIGZns=
X-MC-Unique: 3lGGkDqjPbiQOR7DLdaJ2g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MuJGUsdzmBgRgSs3Fd2EEmjjX8tc9VOzjEnkyyCrs7zkix2mQvERZAzDG8gAWBqqbBJWKNjoShK0VD2JOfwyrdO7J0YNnrG5lu+gd+tOXeqsrQOzWUYu5cMrClSo+1dOXOMsJZdLuvxuF8POCjRQckMY01zX1fkAZje5kDkB0ci++JNtqecwspC77FLjWHSi6LPdc5Nw2Cv/YHZrYiu6/tK+bZrwuuu8Nrd+Mh2ksTBBvzFx7p+JNbmDRzyPuLlxBaKPUhqA5G03gAESkcjqrDqly5HebwibbdvEzngmx8R974Hx3UbG0a6bJ0ODqOmnWYOlK3dmoMnvIbecBS0pzw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FGtjZiIFf5R/oYLpBhLuFfUgjNszBMgjxxQb52D/ECA=;
 b=JoPOkj2JXoCd4iFw3tjp7oSW6+6H4mcV18h4tXgMhHA3Qxp0OJJPD42xCFgViCQYArj+fQn2NQxfarUmeBTcm/54BthwJwZ5JKlbBF1qyQXD+nyaMLiWQHWPOSDWhEOZKBDu0VVpA5/SlUH/6ir+FmR0jCxJ6Khel8nxp7YhTk1r7K9H52zKg+rbaQ24QlmpMSyW0Oql/R30MUiD1zdI09/O9+qpDWPhEGTWmxTKdRkUVFPU6LQb9XfmZf7Yx6zbu22u1+ERVxfWA74rvZdBnNxBk2zI2YOOSgDDk4RJTxArr4WJlvZo0byZhGLc+npaUti3/YYvzp5NM69EeNRLnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] Config: use Mini-OS master for xen-unstable
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20210831073039.7101-1-jgross@suse.com>
 <6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
 <d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3a662b17-bde7-df0f-3d96-d275c70b43b4@suse.com>
Date: Wed, 1 Sep 2021 11:55:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 94446512-6cd1-4f10-0bbf-08d96d2eb192
X-MS-TrafficTypeDiagnostic: VI1PR04MB4845:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4845734A8300A9D894A21A38B3CD9@VI1PR04MB4845.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	v9l2HxlLY7d2v56MxpKDbJxZ3nqM7uw6Gcx65ojkoco8to87Kz/qls+56GkNvnanQXquuNCqeSjbAtxHwpgDq9mMQL1NIPqPu8toXmJhxM0FPw3vElThbNMCSbEu5RMOqyf30mDrQWr8Zu+fJyasOT7vThYQVXtJ2bSXaiOYyVuEJRyc1Y1FE3Q1XSgEiP/hE75HmDtBPfTKujgJKlXGPJa0HglFm/czZHMcXrDOeXh+Nz5r/sIp1m4T2vMxpMN7GBnCDL6ScRuXZWYrT2x8vFYcZgad2ocRdZ0ArHurHkS5+QyOxUTXRbjsz6W0wIE0LbHWoVPO/lFCo9uQSEF8P27oRZaHnhmiPnmAwPLmwD7MthGPtsgWi5YqWEauR+r+dwu8i/r/fH5O/nL3eNWi8NvHvG+c3txooF5kAYdZA1p64a7W+JsDaihdmm+VExfdtG87aNhUCrSOag0WXWfqCmSU7qDdhtB58CoahQMrObEnQ860KOzcxu4aqDIPIpt7K+BNGlugm5lTj1riH6qlQc96QGLVz0wwzu9ClELoXrdBp1LtppLte7/SlBEUOzqzmqMJFTBlwWrFweXx2gwp9Sj8KGHWdsMS/hiqBdjuELos339tp+f3sIow6b1BOSjsEHH+metoOxOqfrYlnXbYjye1HL4q65mwhCFXBzqTs3AfQe7VS6c+9t+1UpoZrgxhFPbpXMSlH3rw3MkhTLXQcfCIQmMgfpXskkIaYQ0rjwplb0npM5w/0ofjOz+nrPD8
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(6486002)(8936002)(508600001)(6636002)(66476007)(2906002)(6862004)(54906003)(66556008)(86362001)(53546011)(4326008)(316002)(16576012)(83380400001)(38100700002)(31696002)(5660300002)(26005)(8676002)(31686004)(2616005)(36756003)(956004)(186003)(37006003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NjZxSmRNKzBLeUswTHYvRzZFcXlJUlZUYjlQaDdDS0tob2FOcUVnL1VGYUdH?=
 =?utf-8?B?cXRTQXZqdmR4QlhOa3VXcFc2cFU2clRaMlRHaVZLVW16dG0wQ3A2TzROSWR3?=
 =?utf-8?B?d0lmLzJnTU9GVExTWWFRQWFJekFHekp4a09zOUNqYnVaZE54dERhdHVWOWZN?=
 =?utf-8?B?cTNXRGh2K0FHK21pQWtmZ1pwYUIvNW0ybWxScEpGMSsxUzNxRlFPaGk0UjRK?=
 =?utf-8?B?Y0hGdFZXSEZFQ25ZOTdKNFlmaGRMeWZDTE1IVjQrQVQzL3lCZ3dneTFVY2k4?=
 =?utf-8?B?NVlnTE8xQnBib0ExcC9LcnpmcWZteTBLVXhpUnVkQzRGTnkwMkxsWHYzY2Ry?=
 =?utf-8?B?RkpSajJVb2xFN2cyQUJUVStqVGNFdHhOSWdVUWROV2FiUWl1TGhneGFCLzZP?=
 =?utf-8?B?Ui9BczNCdForMlRTU1pUQkhXRTdkcnRZWENVMW5KZmVSdlRQK3lTaFozdmVp?=
 =?utf-8?B?VlQrNVgwV1IvMzA2bmtiN2xpdm9LTmNick1zc3l3VHA3dnRGNnJsUVpmNnhL?=
 =?utf-8?B?TVQ1YzdnbUcxeTRsdDhoZGwrMHdvdnZOSnA4MVczWmxOYjFQeDFRYlNJeXZr?=
 =?utf-8?B?N3lLZ0hoUmJjTTUrWUc5VFJLeTNrcXFnM1BHaG96N0ZXb1JQd0o1UWdBQWFO?=
 =?utf-8?B?TlFvRVhjYmhzeEZiUXNGYktwQlRmUUxPN05veVNXL0R0Yno2d2p6TmJJdWtT?=
 =?utf-8?B?Z0Q1ME9jajhWZ1lYcWQwelRXdEhLRVJwSW9oTjlBcjVpMzBOYlB1TDhlWUl6?=
 =?utf-8?B?czJSb1JOenR3WDhwOG1GZjRuSVF6aTUzenVLWllZUXJ4bXFNQzRRdXJUOWJx?=
 =?utf-8?B?cDVzSzNEaFZkN0VkZDVMQWhGQWlMUmxVS2RKcENVMHVmWVJZMkRmUGlFRmxa?=
 =?utf-8?B?YVphK0FVQVNOZ0w0TEx0QUxVU2FHRTdkMi9LenB0RmtOaEdtT1VtcHJoZk5S?=
 =?utf-8?B?S2MvWDlJS0RuZUZuRzJYeGRHTW11Z0oyRW8rOE5NNmlFTCt3aVRFcWtMeE5o?=
 =?utf-8?B?SDlIU1MvK2NiU1FjdXJob0Z6dTJHdGdHSE56ZDlNVXpja0dJamR3SlRBb0J1?=
 =?utf-8?B?akx3WWpnREUwb2ZUMzg0UnNIbUFTbEh5OWE2UTJuUzcwNWhuL3FGMmY4WHRu?=
 =?utf-8?B?dVcvTTgyZ3dHRnVJRERPOTFnYmVRYjZTbU5SdTlkMTV4Mk1zMXByWVp0N2RQ?=
 =?utf-8?B?Q1BDSFpHTHJZVnhzRGdsRVFDdlpCYkU3bVRROURUaVBhVUkvOXZjZ0FwWGZV?=
 =?utf-8?B?bzFjTnFaeXZnamhVMEkya211a2hSTStqZnBySkJLdS9XcWZlTDBralk0N2cv?=
 =?utf-8?B?MmozRUI0ZmV4dzh5TWNiVmM2R0FGZVJ4dnRrWkhyUkRPSzQ5alNrREpWVnpR?=
 =?utf-8?B?NnVOVzFlek9SSHg2enlXZmx6WVNKbXk3MGh2SzNJRjMyTEY5Q1dlaHBRbVJj?=
 =?utf-8?B?NCtMdC9JQzRHR0JvbVMzeU16MzZxbXJNZjVGQ1FadFNibDg0TUVVRmFYWWRZ?=
 =?utf-8?B?WHhwK21ZTCtWNUhVTURyaURSVFpuSnhNcHJ1RFE3dnFrR0xocTZoa3NkZ3dE?=
 =?utf-8?B?bWwwcGg3R0s2OTZEc2N1OTI1K2liWmh1L1A1dEYzU1h6N0xyREk5Zm9DSGFT?=
 =?utf-8?B?T21xbmM3bjFNKzgrTmlwZWhMNUxKd2lmZVBWeFFDOFNXK0U3eGZ2aGFkODFm?=
 =?utf-8?B?eDZWaEZ6YTZTclBCc1hXUnRiZGk2US9BUEtPUVd0SkdydTM2STZBWE1GdjZO?=
 =?utf-8?Q?U8QOSkl+vjfs/4aTDiUfWTUMB1I3Dy7q3nHKovx?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 94446512-6cd1-4f10-0bbf-08d96d2eb192
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 09:55:56.5397
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: B3gaREVvKF3qEyVu8Gy+UedJq80sZKo4xOrnZB5h6rLrHFG029f9RI5mcjao/wga9yRBaOUcT+F7KcMCq/Dp3g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4845

On 01.09.2021 11:53, Juergen Gross wrote:
> On 01.09.21 10:59, Jan Beulich wrote:
>> On 31.08.2021 09:30, Juergen Gross wrote:
>>> In order to get Mini-OS master branch tested by OSStest, use it for
>>> xen-unstable.
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ---
>>> There are some Mini-OS patches pending, of which at least the Mini-OS
>>> netfront fix should be committed as it will be required for qemu-stubdom
>>> to work.
>>> ---
>>>   Config.mk | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/Config.mk b/Config.mk
>>> index 4d723eec1d..d4e8ced104 100644
>>> --- a/Config.mk
>>> +++ b/Config.mk
>>> @@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?= git://xenbits.xen.org/mini-os.git
>>>   endif
>>>   OVMF_UPSTREAM_REVISION ?= b37cfdd2807181aed2fee1e17bd7ec1190db266a
>>>   QEMU_UPSTREAM_REVISION ?= master
>>> -MINIOS_UPSTREAM_REVISION ?= 051b87bb9c19609976fb038f386920e1ce5454c5
>>> +MINIOS_UPSTREAM_REVISION ?= master
>>
>> I'm afraid this, effectively reverting 82c3d15c903a ("minios: Revert recent
>> change and revert to working minios"), would set us up for the same problem
>> again that was dealt with there. IOW you're re-proposing a change of yours
>> (e013e8514389 ["config: use mini-os master for unstable"]) which was
>> subsequently reverted, without spelling out what has since changed.
> 
> How do you suggest to proceed then?
> 
> Will every Mini-OS commit require to be followed by a Xen commit then,
> which will either result in a test success, or in a revert of that Xen
> commit?

Once the two trees are assumed to be fully decoupled, and once a push
gate has been put in place for mini-os.git, we can do what you suggest.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 10:00:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 10:00:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176261.320737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLN2H-0006vU-JQ; Wed, 01 Sep 2021 10:00:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176261.320737; Wed, 01 Sep 2021 10:00:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLN2H-0006v5-G5; Wed, 01 Sep 2021 10:00:01 +0000
Received: by outflank-mailman (input) for mailman id 176261;
 Wed, 01 Sep 2021 10:00:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2KbH=NX=epam.com=prvs=987800c254=roman_skakun@srs-us1.protection.inumbo.net>)
 id 1mLN2F-0006uz-P6
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 10:00:00 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5d42ca6e-0b0b-11ec-adc1-12813bfff9fa;
 Wed, 01 Sep 2021 09:59:58 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1818W2aY029947; 
 Wed, 1 Sep 2021 09:59:55 GMT
Received: from eur02-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3at5yr8h4j-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 01 Sep 2021 09:59:55 +0000
Received: from AM7PR03MB6593.eurprd03.prod.outlook.com (2603:10a6:20b:1b4::10)
 by AS8PR03MB6757.eurprd03.prod.outlook.com (2603:10a6:20b:292::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 09:59:52 +0000
Received: from AM7PR03MB6593.eurprd03.prod.outlook.com
 ([fe80::7cb2:a2bc:d477:683d]) by AM7PR03MB6593.eurprd03.prod.outlook.com
 ([fe80::7cb2:a2bc:d477:683d%9]) with mapi id 15.20.4457.024; Wed, 1 Sep 2021
 09:59:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d42ca6e-0b0b-11ec-adc1-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oQL0FSUpOR6ZZw4dzZ/WLDoYJRtr+dnA/6nKFP+FJdK3t7mQDaLQmiv/4yzs2ddRWiuquNuEWv3lGIRiirB+fSyN/nylMaMo3p2g9vsv3KuikQANoY8PVRlJdaAeXB2nQDXsKaKRGbqSXtQ4Lw41VsrJ+8U7dAyEVzsWryJSuefKp/cEZfc9gkV3mJlYVLvF32f5pcTk8h/7W/mORBBzsJ9k8cbBmfmNNn8Cd+NQlBkK2jxU9Pq0Fy8MxntIX71NF+aSeKHoV1RHwpDYd3oa8fDy1TlWGN6ghWQ6KUbHKC2U39EPLXHk68sGit7tjboKBV89C0nmxKXzoSX4qKU1HA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=La8gXQMXfHGw4BBQ/lbRYKOiAPc1dG59oOZFJuu/0R4=;
 b=atiOGJeaTfQD/X7a0Y5DxpMn/1zW8kvmxi+a0vyi117D0eUM5CIeTxrjskfuGc13BpsURhpQ+nSMbsxcllTBoFPk6r+GEsL5Fdp1DHOfJiJx6/HMB+QH/Rv5CECRl/6xh7N9qyle5VZPhw5jqy11vcXe0cgRH8SJmnJ/lRi6344yDtFfJFDxemywR/DLLDOdiY8j6d7Qh9KJGHA2krlmJnrhwzoCiP9Xf2nUyVVDAqjXQU5zc8pbwbeh/UTj3zq3eV0QzmcifqBBya8ow80o9pGQFNd9/vBSwr1GFB6ikB2aHvJPOd3p/8YfskKNJ9bIVrU1mPo6qPK83gxO1CR4gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=La8gXQMXfHGw4BBQ/lbRYKOiAPc1dG59oOZFJuu/0R4=;
 b=bzgZGNp7TSEPPhuFfoDeIX+LYW42sv438/+BrxDIlI2dQfHWYB+gBDxPQy5NjOesZTnCTpfabMjXtIXBx7xW2A8wRbMmeElQW4ixWD/SRX1I0gJdT+gjNUh/rUq33i9KRsqrrQHDBsYF8ep0piMuYhVAXJ9f/x8Dll3dxhSEekPU2uMZv96ngmr77IQS/4adLQh8YfwGLtfxrLc7PbC2ewBVh+28tJdmjoYRhI79GTJodghkuos7QX8GOriEAztEVINyMsC1rAqy+it25hH6inp41Qj0KfaoclfYwL3eeaKgTN/9/j95JWBqHgBmb/dZcyIIbCcI6G7oCuFYHBXzvA==
From: Roman Skakun <Roman_Skakun@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Andrii Anisov <Andrii_Anisov@epam.com>,
        Roman
 Skakun <rm.skakun@gmail.com>,
        Roman Skakun <Roman_Skakun@epam.com>
Subject: Re: Disable IOMMU in Dom0
Thread-Topic: Disable IOMMU in Dom0
Thread-Index: AQHXnnd7IVjUu+L6S0mPq9naNxJ1FauOJ3YAgADDGos=
Date: Wed, 1 Sep 2021 09:59:52 +0000
Message-ID: 
 <AM7PR03MB659358C0EBA9D71AEEF7A60885CD9@AM7PR03MB6593.eurprd03.prod.outlook.com>
References: 
 <AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com>
 <alpine.DEB.2.21.2108311444310.18862@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2108311444310.18862@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e480f3ce-6a3e-44ea-c1c1-08d96d2f3e73
x-ms-traffictypediagnostic: AS8PR03MB6757:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB67575F4385E18435A2A5E92985CD9@AS8PR03MB6757.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 0J2GrC+3U+dvjt4VfBDewh7ZaRnVTgwszZGiIN0mOXsMZzgeDF77YwUx8Pm5AUhzbQlGUCbi9+v6gL0+1oJsSxAUrwLj+Xj/6e5xybjMaohm7o+PR7IyxWzOh8AhOpOAlZ5Ca9HBCaNlVno7qmv30UCRvVukBnK/GLHgf5NTiT1tTXmS8VKIFIm9N6RWyq5yVJkHqqVWnxPWQ0Bhl4ejxTUZpnzeDa4DxrUyenP/JvJgSsF5NuKRXdZ1SC/dA00TE7ljuyhKggETqugoKEtkweuKQDGorQG2ndDyAKSI1ERuk4DCNfRY2wDWR8xJHVguTE9o87jg8cKRUTelZeKkDjTsmsK9iFfS1VU0ko/ielKUf+YX2h4VUqLNruOX8B0qzK0//8z5fm9vUlwvGAcDgVuGC7EFOMRrHAps3aIlZSEQpWcECaV0XntF7y6OQGdwTjtl5aLT6A4Qz3bQF1/HuBjj9SIg9Di/V8DGKe6P8nCQ/jsW8abmoaW3IdioKxHz/hepValPgsn3NGJ8N4F0wEINhKhm8zR7Xks7v4skV5ckSa5knP6BbRxL7T82AqiWWL9Qw6CiaaeWB0lm4/2ZXjazO/AFR7Pr50vMeuKZj0WOQ9m1VE8vMXdXzbxqIMJprD3ky20QVNWba9GBKaxpp8oBQKXDckG4Wd3qUq8yziTAZakugn7aE7/4JYnMYONgVoLAw/iurTy8Fu0nDjJS7Luru5IOFFlDWHeQ9gBsBebrvLKBHk5QG5dpYsXNxXYpkHig5AB22lSz05A3KZ6z5AgOY+uRLpkserliFVRZilM=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6593.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(966005)(54906003)(38070700005)(166002)(508600001)(66946007)(6916009)(52536014)(5660300002)(33656002)(19627235002)(316002)(38100700002)(26005)(64756008)(8676002)(66556008)(4326008)(66476007)(122000001)(86362001)(6506007)(8936002)(76116006)(186003)(2906002)(107886003)(55016002)(9686003)(19627405001)(53546011)(7696005)(66446008)(71200400001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?us-ascii?Q?dXdAFwMOel6Y+LgrmmBEe1vCjW3DC5BUNlcOIlLOyh2n0EWZbpck3S2c/l8s?=
 =?us-ascii?Q?T0QumJZ3ENjw0IrYSxWQnzh3jKnHPPenoHkzzpAsImy7txk9nzJK2n/hF6Gs?=
 =?us-ascii?Q?2hgaoem6aqfDy1ZGzVgsCHnHNYYJjK2/60WW9Ugpw6uMm3MxFHvM72eag6HU?=
 =?us-ascii?Q?bPBuBwe0nbe4xBP0+sR85COktzASZQ6+onMsEh1Yolvpo2DShj1Bw2rJ4MlE?=
 =?us-ascii?Q?62enXAV3f0s1/9w+4k3cGFNj832PjXgmUMiW+Vd9e9YYbCtjbaiMwAi39wgK?=
 =?us-ascii?Q?Eneh+IBI4hLd9iXjmvXiJz8CLGoTUiv51ehpdYF6daOLntbyUMvs9y3ZSC3l?=
 =?us-ascii?Q?AjaBmw2cjbcYBvOuOeDPBhFMbTdDdxeXRL7OuSwk58bW8keAZiItYUwItDZ2?=
 =?us-ascii?Q?iAYDjOVQMMSfh7ioTko3MjORqUMrAkWlQEQ9782JY3z2oh6A/6gh5kAO6wYp?=
 =?us-ascii?Q?QdWbphIxTYknFHfjgut4tcuEp3N3uDcfr4cXagegyGnEor8o3qvsRs3bRMwa?=
 =?us-ascii?Q?1movT+uemGV+P6QAoiRSFQrGFh5IVG8dElY1BrmjGQS0leG5ULBr2mdg7tc4?=
 =?us-ascii?Q?8BsdxqO4g5CrfXildw3BfkFtoRJcdA1g/xmhW6ql+VbGW2GFu73I1wtPsmhR?=
 =?us-ascii?Q?vdAsQ9H8WQRFdnwAAL+m9z3N3nGOqlN3HaJf5iLo7trIuwce19ap115gNcVp?=
 =?us-ascii?Q?bvFodvtttggo376TWvV+V9LEoyH8qyDvIQ3Qy4tLI2jBWH9ASI3lm7JhhWh2?=
 =?us-ascii?Q?acDJN4P7w99/Nryw9KIMOfl+H7sVK5WWs49+I7BS3XBG8wD7xLXetigURkoB?=
 =?us-ascii?Q?LDpeUTij61reziSVhgqMIXF5XSkHTflsWuqeQn2t8/rG7JBj/0RRI/mNMBwi?=
 =?us-ascii?Q?jRflXKkknibU7FZ50qLxMrXhu/1FjBnJvrm+iUWJbA933yVLr7nJZY8JIgN9?=
 =?us-ascii?Q?B/49zypIpMAvKaLe0MROOpQoZ2IBl/L1F1T7k3nidbo/QfDWVXRZaaHtQoAg?=
 =?us-ascii?Q?uOFSep8RqUjOT340v+367anbCl9Jw0pk8nnweXZSHMrbO2pUVT5jTyQCTp0L?=
 =?us-ascii?Q?I9+RgFGBGAgMDcjPN1Vvf5eScJ4Ez/xqq1eU9DMNPdFAXeDOrT4WYA8N1jYA?=
 =?us-ascii?Q?MJHyPKRilmNc26Ki70J/+tue3lFqk45dJovIuXOgvjwTTL0D8O66DEzvQkEU?=
 =?us-ascii?Q?coyRF7X9hxDME5w9TPhkorolqUbmveKZckNrrxgeOhTEk29tjJdPy1P9FSj+?=
 =?us-ascii?Q?uPQS8LKJdc8yLNdqtbjhNKOIm8KqS1PQXC4hAPcRX3DF1zAao7YPNO9iX6L1?=
 =?us-ascii?Q?+Mc=3D?=
Content-Type: multipart/alternative;
	boundary="_000_AM7PR03MB659358C0EBA9D71AEEF7A60885CD9AM7PR03MB6593eurp_"
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6593.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e480f3ce-6a3e-44ea-c1c1-08d96d2f3e73
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Sep 2021 09:59:52.7354
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: xvyKAp3fP1C2MdxzGwuduA5+107HnnQ6ba5q06+VmopgXhShct6Ii4VMyBE16KIXZcyN8LmR6OaXs3Z2T7RAvw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB6757
X-Proofpoint-GUID: YE7uaaEk6YAdQTzKXmC9hKelH24PX-Ys
X-Proofpoint-ORIG-GUID: YE7uaaEk6YAdQTzKXmC9hKelH24PX-Ys
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-01_03,2021-08-31_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 bulkscore=0
 phishscore=0 lowpriorityscore=0 mlxscore=0 clxscore=1015 malwarescore=0
 mlxlogscore=999 impostorscore=0 priorityscore=1501 suspectscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2107140000 definitions=main-2109010058

--_000_AM7PR03MB659358C0EBA9D71AEEF7A60885CD9AM7PR03MB6593eurp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi, Stefano!

Thanks for responding!

> If you have a setup where Dom0 is not 1:1 mapped (which is not currently
> possible with upstream Xen but it is possible with cache coloring) and
> uses the IOMMU to make device DMA work like regular DomUs, then passing
> XENFEAT_not_direct_mapped to Linux would make it work. Without
> XENFEAT_not_direct_mapped, Linux would try to use swiotlb-xen which has
> code that relies on Linux being 1:1 mapped to work properly.

I'm using 1:1 Dom0.
According to your patch series, xen-swiotlb fops will be applied for all de=
vices
because XENFEAT_direct_mapped is active, as shown here:
https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c#L=
56

I agreed, that xen-swiotlb should work correctly, but in my case, I retriev=
ed MFN here:
https://elixir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c#L36=
6
which is greater than 32bit and xen-swiotlb tries to use bounce buffer as e=
xpected.
It can lead to decrease a performance because I have a long buffer ~4MB.

I thought, that we can disable swiotlb fops for devices which are controlle=
d by IOMMU.

Cheers,
________________________________
From: Stefano Stabellini <sstabellini@kernel.org>
Sent: Wednesday, September 1, 2021 12:50 AM
To: Roman Skakun <Roman_Skakun@epam.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>; Julien Grall <julien@xen.o=
rg>; xen-devel@lists.xenproject.org <xen-devel@lists.xenproject.org>; Bertr=
and Marquis <bertrand.marquis@arm.com>; Andrii Anisov <Andrii_Anisov@epam.c=
om>; Roman Skakun <rm.skakun@gmail.com>
Subject: Re: Disable IOMMU in Dom0

On Tue, 31 Aug 2021, Roman Skakun wrote:
> Hi, Stefano!
>
> I have seen your negotiation of disabling xen-swiotlb for devices which a=
re controlled by IOMMU in Dom0:
> https://urldefense.com/v3/__https://patchwork.kernel.org/project/xen-deve=
l/patch/alpine.DEB.2.21.2102161333090.3234@sstabellini-ThinkPad-T480s/__;!!=
GF_29dbcQIUBPA!hl4_2pWlI3NQmAkwh4uvJFu7NhcOay9O9bIJRr7eOOMKD_I2z8MPbwnSXpNm=
qU6F$ [patchwork[.]kernel[.]org]
>
> As I was thinking to create a common implementation because I have the is=
sue
> when device controlled by IOMMU using xen-swiotlb fops and bounce buffer =
as the result.
> https://urldefense.com/v3/__https://lore.kernel.org/xen-devel/060b5741-92=
2c-115c-7e8c-97d8aa5f46f4@xen.org/T/__;!!GF_29dbcQIUBPA!hl4_2pWlI3NQmAkwh4u=
vJFu7NhcOay9O9bIJRr7eOOMKD_I2z8MPbwnSXp1TdIa7$ [lore[.]kernel[.]org]
>
> Do you have any future plans to finish implementation for upstream?

Hi Roman,

The feature is already upstream in Linux as f5079a9a2, and the new
feature flags are XENFEAT_direct_mapped and XENFEAT_not_direct_mapped.

If you have a setup where Dom0 is not 1:1 mapped (which is not currently
possible with upstream Xen but it is possible with cache coloring) and
uses the IOMMU to make device DMA work like regular DomUs, then passing
XENFEAT_not_direct_mapped to Linux would make it work. Without
XENFEAT_not_direct_mapped, Linux would try to use swiotlb-xen which has
code that relies on Linux being 1:1 mapped to work properly.

Is that the same problem that you have, or is dom0 1:1 mapped in your
case? If dom0 is 1:1 mapped then swiotlb-xen should work regardless of
whether the IOMMU is enabled or disabled.

I hope this helps.

Cheers,

Stefano

--_000_AM7PR03MB659358C0EBA9D71AEEF7A60885CD9AM7PR03MB6593eurp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size=
: 12pt; color: rgb(0, 0, 0);">
<span style=3D"font-size: 12pt;">Hi, Stefano!</span><br>
<br>
<span style=3D"font-size: 12pt;">Thanks for responding!</span><br>
<br>
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; If you have a set=
up
 where Dom0 is not 1:1 mapped (which is not currently</span><br>
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; possible with ups=
tream
 Xen but it is possible with cache coloring) and</span><br style=3D"font-fa=
mily:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot;=
Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica =
Neue&quot;, sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 2=
55)">
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; uses the IOMMU to=
 make
 device DMA work like regular DomUs, then passing</span><br style=3D"font-f=
amily:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot=
;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica=
 Neue&quot;, sans-serif;font-size:14.6667px;background-color:rgb(255, 255, =
255)">
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; XENFEAT_not_direc=
t_mapped
 to Linux would make it work. Without</span><br style=3D"font-family:&quot;=
Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot;Segoe UI&qu=
ot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica Neue&quot;,=
 sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255)">
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; XENFEAT_not_direc=
t_mapped,
 Linux would try to use swiotlb-xen which has</span><br style=3D"font-famil=
y:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot;Seg=
oe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica Neu=
e&quot;, sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255)=
">
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important">&gt; code that relies =
on
 Linux being 1:1 mapped to work properly.<br>
</span><br>
<span style=3D"font-size: 12pt;">I'm using 1:1 Dom0.</span><span style=3D"f=
ont-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, =
&quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helv=
etica Neue&quot;, sans-serif;font-size:14.6667px;background-color:rgb(255, =
255, 255);display:inline !important"><br>
<div class=3D"_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_=
1"></div>
</span><span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web=
 (West European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystem=
Font, Roboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; back=
ground-color: rgb(255, 255, 255); display: inline !important;">According to
<span style=3D"background-color:rgb(255, 255, 255);display:inline !importan=
t">your patch series,&nbsp;</span>xen-swiotlb fops will be applied for all =
devices&nbsp;</span></div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size=
: 12pt; color: rgb(0, 0, 0);">
<span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web (West =
European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, R=
oboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; background-=
color: rgb(255, 255, 255); display: inline !important;">because&nbsp;</span=
><span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web (West=
 European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, =
Roboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; background=
-color: rgb(255, 255, 255); display: inline !important;"><span style=3D"bac=
kground-color: rgb(255, 255, 255); display: inline !important; font-size: 1=
2pt;">XENFEAT_direct_mapped
 is active, as shown here:</span></span><span style=3D"font-family:&quot;Se=
goe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot;Segoe UI&quot=
;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica Neue&quot;, s=
ans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:i=
nline !important"><br>
<a href=3D"https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-=
mapping.c#L56" id=3D"LPlnk"><span style=3D"font-size: 12pt;">https://elixir=
.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c#L56</span></a><=
br>
<div class=3D"_Entity _EType_OWALinkPreview _EId_OWALinkPreview_1 _EReadonl=
y_1"></div>
<br>
</span><span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web=
 (West European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystem=
Font, Roboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; back=
ground-color: rgb(255, 255, 255); display: inline !important;">I agreed, th=
at
 xen-swiotlb should work correctly, but&nbsp;</span><span style=3D"font-fam=
ily: &quot;Segoe UI&quot;, &quot;Segoe UI Web (West European)&quot;, &quot;=
Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &quot;Helvetica =
Neue&quot;, sans-serif; font-size: 12pt;">in my case, I retrieved MFN here:=
</span></div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size=
: 12pt; color: rgb(0, 0, 0);">
<span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West E=
uropean)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Ro=
boto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background=
-color:rgb(255, 255, 255);display:inline !important"><a href=3D"https://eli=
xir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c#L366" style=3D=
"margin:0px"><span style=3D"font-size: 12pt;">https://elixir.bootlin.com/li=
nux/v5.14/source/drivers/xen/swiotlb-xen.c#L366</span></a></span></div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size=
: 12pt; color: rgb(0, 0, 0);">
<span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web (West =
European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, R=
oboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; background-=
color: rgb(255, 255, 255); display: inline !important;">which is greater th=
an
 32bit and xen-swiotlb tries to use bounce buffer as expected.</span><span =
style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West Europea=
n)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, =
&quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background-color=
:rgb(255, 255, 255);display:inline !important"><br>
</span><span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web=
 (West European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystem=
Font, Roboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; back=
ground-color: rgb(255, 255, 255); display: inline !important;">It can lead
 to decrease a performance because I have a long buffer ~4MB.</span><span s=
tyle=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web (West European=
)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystemFont, Roboto, &=
quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;background-color:=
rgb(255, 255, 255);display:inline !important"><br>
<br>
</span><span style=3D"font-family: &quot;Segoe UI&quot;, &quot;Segoe UI Web=
 (West European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystem=
Font, Roboto, &quot;Helvetica Neue&quot;, sans-serif; font-size: 12pt; back=
ground-color: rgb(255, 255, 255); display: inline !important;">I thought,
 that we can disable swiotlb fops for devices which are controlled by IOMMU=
.</span><span style=3D"font-family:&quot;Segoe UI&quot;, &quot;Segoe UI Web=
 (West European)&quot;, &quot;Segoe UI&quot;, -apple-system, BlinkMacSystem=
Font, Roboto, &quot;Helvetica Neue&quot;, sans-serif;font-size:14.6667px;ba=
ckground-color:rgb(255, 255, 255);display:inline !important"><br>
<br>
Cheers,</span></div>
<div id=3D"appendonsend"></div>
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Stefano Stabellini &l=
t;sstabellini@kernel.org&gt;<br>
<b>Sent:</b> Wednesday, September 1, 2021 12:50 AM<br>
<b>To:</b> Roman Skakun &lt;Roman_Skakun@epam.com&gt;<br>
<b>Cc:</b> Stefano Stabellini &lt;sstabellini@kernel.org&gt;; Julien Grall =
&lt;julien@xen.org&gt;; xen-devel@lists.xenproject.org &lt;xen-devel@lists.=
xenproject.org&gt;; Bertrand Marquis &lt;bertrand.marquis@arm.com&gt;; Andr=
ii Anisov &lt;Andrii_Anisov@epam.com&gt;; Roman Skakun &lt;rm.skakun@gmail.=
com&gt;<br>
<b>Subject:</b> Re: Disable IOMMU in Dom0</font>
<div>&nbsp;</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">On Tue, 31 Aug 2021, Roman Skakun wrote:<br>
&gt; Hi, Stefano!<br>
&gt; <br>
&gt; I have seen your negotiation of disabling xen-swiotlb for devices whic=
h are controlled by IOMMU in Dom0:<br>
&gt; <a href=3D"https://urldefense.com/v3/__https://patchwork.kernel.org/pr=
oject/xen-devel/patch/alpine.DEB.2.21.2102161333090.3234@sstabellini-ThinkP=
ad-T480s/__;!!GF_29dbcQIUBPA!hl4_2pWlI3NQmAkwh4uvJFu7NhcOay9O9bIJRr7eOOMKD_=
I2z8MPbwnSXpNmqU6F$">
https://urldefense.com/v3/__https://patchwork.kernel.org/project/xen-devel/=
patch/alpine.DEB.2.21.2102161333090.3234@sstabellini-ThinkPad-T480s/__;!!GF=
_29dbcQIUBPA!hl4_2pWlI3NQmAkwh4uvJFu7NhcOay9O9bIJRr7eOOMKD_I2z8MPbwnSXpNmqU=
6F$</a> [patchwork[.]kernel[.]org]<br>
&gt; <br>
&gt; As I was thinking to create a common implementation because I have the=
 issue<br>
&gt; when device controlled by IOMMU using xen-swiotlb fops and bounce buff=
er as the result.<br>
&gt; <a href=3D"https://urldefense.com/v3/__https://lore.kernel.org/xen-dev=
el/060b5741-922c-115c-7e8c-97d8aa5f46f4@xen.org/T/__;!!GF_29dbcQIUBPA!hl4_2=
pWlI3NQmAkwh4uvJFu7NhcOay9O9bIJRr7eOOMKD_I2z8MPbwnSXp1TdIa7$">
https://urldefense.com/v3/__https://lore.kernel.org/xen-devel/060b5741-922c=
-115c-7e8c-97d8aa5f46f4@xen.org/T/__;!!GF_29dbcQIUBPA!hl4_2pWlI3NQmAkwh4uvJ=
Fu7NhcOay9O9bIJRr7eOOMKD_I2z8MPbwnSXp1TdIa7$</a> [lore[.]kernel[.]org]<br>
&gt; <br>
&gt; Do you have any future plans to finish implementation for upstream?<br=
>
<br>
Hi Roman,<br>
<br>
The feature is already upstream in Linux as f5079a9a2, and the new<br>
feature flags are XENFEAT_direct_mapped and XENFEAT_not_direct_mapped.<br>
<br>
If you have a setup where Dom0 is not 1:1 mapped (which is not currently<br=
>
possible with upstream Xen but it is possible with cache coloring) and<br>
uses the IOMMU to make device DMA work like regular DomUs, then passing<br>
XENFEAT_not_direct_mapped to Linux would make it work. Without<br>
XENFEAT_not_direct_mapped, Linux would try to use swiotlb-xen which has<br>
code that relies on Linux being 1:1 mapped to work properly.<br>
<br>
Is that the same problem that you have, or is dom0 1:1 mapped in your<br>
case? If dom0 is 1:1 mapped then swiotlb-xen should work regardless of<br>
whether the IOMMU is enabled or disabled.<br>
<br>
I hope this helps.<br>
<br>
Cheers,<br>
<br>
Stefano<br>
</div>
</span></font></div>
</body>
</html>

--_000_AM7PR03MB659358C0EBA9D71AEEF7A60885CD9AM7PR03MB6593eurp_--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 10:05:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 10:05:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176268.320748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLN7w-0000Ee-Cc; Wed, 01 Sep 2021 10:05:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176268.320748; Wed, 01 Sep 2021 10:05:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLN7w-0000EX-9g; Wed, 01 Sep 2021 10:05:52 +0000
Received: by outflank-mailman (input) for mailman id 176268;
 Wed, 01 Sep 2021 10:05:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLN7u-0000EN-Uy; Wed, 01 Sep 2021 10:05:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLN7u-0007jP-Py; Wed, 01 Sep 2021 10:05:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLN7u-0003Qd-I2; Wed, 01 Sep 2021 10:05:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLN7u-0004H0-Ha; Wed, 01 Sep 2021 10:05:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=j24VJPcwVdYQa9uXurFdKSllutLiT6F+ExzgVYTfIp4=; b=5ThjvdKqEVYCOp9dDeFj3Gamh6
	acyoEzYMhEZ5sUiiCkLYI/y5AKu6StXQ/cxEkMuatfIBD90ESXPTBzeiBNmE0d4z++H0NXeetWFfu
	e5FiZZ4Uxv8FTHJpRMZacknwWJ+ETqOQ2Q8gmFFrmzh3Pdby16lzv353IKPyBTqgDyzE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164692-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 164692: all pass - PUSHED
X-Osstest-Versions-This:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
X-Osstest-Versions-That:
    xen=8064488062641ae505b2a7369611c38057a7788b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 10:05:50 +0000

flight 164692 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164692/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618
baseline version:
 xen                  8064488062641ae505b2a7369611c38057a7788b

Last test of basis   164595  2021-08-29 09:18:29 Z    3 days
Testing same since   164692  2021-09-01 09:18:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Fabrice Fontaine <fontaine.fabrice@gmail.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <julien@xen.org>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   8064488062..96607a8e68  96607a8e680e7f965ca868d11f8b0636317d2618 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 10:22:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 10:22:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176276.320762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNNg-0002ri-QU; Wed, 01 Sep 2021 10:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176276.320762; Wed, 01 Sep 2021 10:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNNg-0002rG-NE; Wed, 01 Sep 2021 10:22:08 +0000
Received: by outflank-mailman (input) for mailman id 176276;
 Wed, 01 Sep 2021 10:22:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLNNf-0002r8-NB
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 10:22:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLNNd-0007zX-Co; Wed, 01 Sep 2021 10:22:05 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLNNd-00061t-6E; Wed, 01 Sep 2021 10:22:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=J5xaLmys0DZR5B87YG9hJdaXxyVbpOBZsGEg/stJC60=; b=aWti9DinL7kqLdmR6/NApgKp90
	akIy4rqjEAA6OrkcRX7JHprBmlesV4muG2yJ6hbRBQ4aPKMbNZ7AKxSpd2UwAApU1egBJd8Y62MF7
	J7BDU2zgRpPJrppbGcUIofyclo98wzOG/T+zB5Wu6LiJIMeU67wEaW1SoLPXX/UkgFj0=;
Subject: Re: Disable IOMMU in Dom0
To: Roman Skakun <Roman_Skakun@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Andrii Anisov <Andrii_Anisov@epam.com>, Roman Skakun <rm.skakun@gmail.com>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
References: <AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com>
 <alpine.DEB.2.21.2108311444310.18862@sstabellini-ThinkPad-T480s>
 <AM7PR03MB659358C0EBA9D71AEEF7A60885CD9@AM7PR03MB6593.eurprd03.prod.outlook.com>
From: Julien Grall <julien@xen.org>
Message-ID: <179e4b7f-38d9-991d-3f99-64a74559986d@xen.org>
Date: Wed, 1 Sep 2021 11:22:03 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <AM7PR03MB659358C0EBA9D71AEEF7A60885CD9@AM7PR03MB6593.eurprd03.prod.outlook.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Roman

On 01/09/2021 10:59, Roman Skakun wrote:
>> If you have a setup  where Dom0 is not 1:1 mapped (which is not currently
>> possible with upstream  Xen but it is possible with cache coloring) and
>> uses the IOMMU to make  device DMA work like regular DomUs, then passing
>> XENFEAT_not_direct_mapped  to Linux would make it work. Without
>> XENFEAT_not_direct_mapped,  Linux would try to use swiotlb-xen which has
>> code that relies on  Linux being 1:1 mapped to work properly.
> 
> I'm using 1:1 Dom0.
> According to your patch series, xen-swiotlb fops will be applied for all 
> devices
> because XENFEAT_direct_mapped is active, as shown here:
> https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c#L56 
> <https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c#L56>
> 
> I agreed, that xen-swiotlb should work correctly, but in my case, I 
> retrieved MFN here:
> https://elixir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c#L366 
> <https://elixir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c#L366>
> which is greater than 32bit and xen-swiotlb tries to use bounce buffer 
> as expected.
> It can lead to decrease a performance because I have a long buffer ~4MB.
> 
> I thought, that we can disable swiotlb fops for devices which are 
> controlled by IOMMU.

Yes you can disable swiotlb for devices which are controlled by the 
IOMMU. But this will not make your problem disappear, it simply hides 
until it bites you in a more subttle way.

 From what you wrote, you have a 32-bit DMA capable. So you always need 
to have an address below 4GB. For foreign mapping, there is no guarantee 
the Guest Physical Address will actually be below 4GB.

Today, the ballooning code only ask Linux to steal *a* RAM page for 
mapping the foreign page. This may or may not be below 4GB depending on 
how you assigned the RAM to dom0 (IIRC you had some RAM above 4GB).

But that's the current behavior. One of your work colleague is looking 
at avoid to steal RAM page to avoid exhausting the memory. So foreign 
mapping may end up to be a lot higher in memory.

IOW, you will need to be able to bounce the DMA buffer for your device. 
If you want to avoid bouncing, the proper way would be to rework the 
ballonning code so pages are allocated below 4GB.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 10:24:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 10:24:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176282.320774 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNPd-0003UM-7f; Wed, 01 Sep 2021 10:24:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176282.320774; Wed, 01 Sep 2021 10:24:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNPd-0003UF-49; Wed, 01 Sep 2021 10:24:09 +0000
Received: by outflank-mailman (input) for mailman id 176282;
 Wed, 01 Sep 2021 10:24:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLNPb-0003U5-LE; Wed, 01 Sep 2021 10:24:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLNPb-00081A-F4; Wed, 01 Sep 2021 10:24:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLNPb-0004fT-88; Wed, 01 Sep 2021 10:24:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLNPb-0003IW-7d; Wed, 01 Sep 2021 10:24:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9CfheEgIl+2VO5giw5dGD+foRsQnOqOT4h60Tyc1jsE=; b=fkMNCsLl0TICMYlpPZR3fzesxK
	HYuAaLSXT5b+2m0zLT/Qkqcl8YSdWY/dVcldW6l8mc8HaOoiAC0XF0YUxmemw6vAczUKHzudX3yir
	sE5HwK4/TGIEKn/xkIJDOAnaQ3ptzAKUEjT7PEhrPatJyKQPifLlI8Sk/vj0EnzuMbdM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164686-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164686: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=9f3eda177a4b2d4a33ff1b0307cad42906396562
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 10:24:07 +0000

flight 164686 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164686/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 9f3eda177a4b2d4a33ff1b0307cad42906396562
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Testing same since   164686  2021-09-01 03:03:43 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 10:33:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 10:33:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176290.320790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNY2-00059S-4V; Wed, 01 Sep 2021 10:32:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176290.320790; Wed, 01 Sep 2021 10:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLNY2-00059L-1N; Wed, 01 Sep 2021 10:32:50 +0000
Received: by outflank-mailman (input) for mailman id 176290;
 Wed, 01 Sep 2021 10:32:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLNY0-00059F-D7
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 10:32:48 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLNY0-0008BJ-BW
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 10:32:48 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLNY0-00079q-AT
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 10:32:48 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLNXn-00088q-W7; Wed, 01 Sep 2021 11:32:36 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=cwyy9inWMU2OFz4WdY/TbmnH7ieKYhzdnQ4z+pCsQBM=; b=fywikXv1TWe3rr4D8XCHI+zxMy
	BXOIy+KbegKgUFnGfLRq14O8RvOsuqm317+hru5SmnPvh9bcvwNtWHoVHleooPTQo72kqdRBswcpI
	8Vh5MrOy02knzvP218+jLlR4UDadge/A/68oMp/P99OXiGPpUnzQcB0IBMaWxd4Axfkw=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24879.22211.552820.121035@mariner.uk.xensource.com>
Date: Wed, 1 Sep 2021 11:32:35 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    Wei Liu <wl@xen.org>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH] Config: use Mini-OS master for xen-unstable
In-Reply-To: <d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
References: <20210831073039.7101-1-jgross@suse.com>
	<6837d551-6473-e34d-c0ae-043fcaa9482b@suse.com>
	<d610c89b-fb46-338f-e4b6-5e1d5a0b5040@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH] Config: use Mini-OS master for xen-unstable"):
> How do you suggest to proceed then?
> 
> Will every Mini-OS commit require to be followed by a Xen commit then,
> which will either result in a test success, or in a revert of that Xen
> commit?

That is how the current arrangement works.  That's strict coupling.

If you want to decouple it, that's possible.  Ie, run a separate
osstest "branch" (ie, series of flights) for mini-os; have a separate
osstest-tested mini-os output branch, etc.

Wei, if we were to do that, should we have a "staging" and "main" for
mini-os in the main mini-os tree, or what ?

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 11:05:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 11:05:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176300.320808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLO3W-0000Sr-Nm; Wed, 01 Sep 2021 11:05:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176300.320808; Wed, 01 Sep 2021 11:05:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLO3W-0000Sk-KG; Wed, 01 Sep 2021 11:05:22 +0000
Received: by outflank-mailman (input) for mailman id 176300;
 Wed, 01 Sep 2021 11:05:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1ICC=NX=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLO3V-0000Se-2E
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 11:05:21 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.59]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7d43c135-0b14-11ec-adc6-12813bfff9fa;
 Wed, 01 Sep 2021 11:05:19 +0000 (UTC)
Received: from AM5P194CA0016.EURP194.PROD.OUTLOOK.COM (2603:10a6:203:8f::26)
 by PAXPR08MB6365.eurprd08.prod.outlook.com (2603:10a6:102:156::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Wed, 1 Sep
 2021 11:05:17 +0000
Received: from AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:8f:cafe::e) by AM5P194CA0016.outlook.office365.com
 (2603:10a6:203:8f::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Wed, 1 Sep 2021 11:05:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT039.mail.protection.outlook.com (10.152.17.185) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.17 via Frontend Transport; Wed, 1 Sep 2021 11:05:16 +0000
Received: ("Tessian outbound f11f34576ce3:v103");
 Wed, 01 Sep 2021 11:05:16 +0000
Received: from a6d1ef8b4f9a.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B27C819D-887B-4E18-942E-769D873BED20.1; 
 Wed, 01 Sep 2021 11:05:10 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a6d1ef8b4f9a.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 Sep 2021 11:05:10 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6507.eurprd08.prod.outlook.com (2603:10a6:10:25a::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Wed, 1 Sep
 2021 11:04:54 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.024; Wed, 1 Sep 2021
 11:04:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d43c135-0b14-11ec-adc6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=taURv7rLin99uMRFD4DUiQt0w7d5GpQcs6q42zdEV4c=;
 b=dOe2ZzxLd7Fnf0BEO/ImzgRIN1Zhfc44eJvv0Tye8Sbt7Bg+yB2rJP/BJRPZtzTv0vw0YTrrYQWiu/qM8mOgQsqONRO3utuJS0r/8yS1gtLq3twqOC0A+1UX4mO52nu66UIqFGUzg2Z+Hq4sdGuI0O2JMkmtMu//mS+v9srcWaI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H3sUvAp3Y+T5ffH83q2K15+0jiHu8+Lf3BBw09sqYAdgCbVE1c7oRxQRB0OnmS0YxDbvd1IAfgPgXxHO6NuXvjSuKvm3f/54RdHnJzwoHA2I+6bx4J2Rra20wL+0V7iUCo2hgidioN0M+pPVEGXqdcgWPtdp2wVWo2hZy/3+AYTG5rEQvgwjLaOZ7SYwjlxVSRvPpR3HHMoUGdFaCqu24lXegqTMNWXDHYfcWovaeuHLcjddvVSiDhBCa7DvCMtvglqIwSQxRIuP/nCta/bH4ogvgS3HJeEizVEYyBfs0xnQhmVqM1pN7HBDWBQJwjWktp9l0Xe06QwUbU8XEJ7Q/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=taURv7rLin99uMRFD4DUiQt0w7d5GpQcs6q42zdEV4c=;
 b=cZyJ6HksLM2uPO40guGsveb9qwgNeB05QeoWkSs2wG0y/U7f4LFFVJjzRc1al0P4lUwtnaXkaEAL2gwi0iz7X9LfmL9urQKoYJLxFVGODRIKfGiamegemftnWP3EphdJNREuVLJWB2pmr7nf9QNRHtB/n0dQvHvG/Q8ryzAWrqwAj3xyYwnsyHCAEqe+fzxoOToJGBDxLRnyTkWErasbNcD3MLQHGl8wcn3LWr222Ew8ONqbBvRHxGd0njGk9bWEXvumIM7hO5e/ETVztaQh00JbswfIBX2QJjqOp7CyCEOuvKWHSnc+c4XwF12Uxuoz+hszSWWzOUx4lbkCA2MiEA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=taURv7rLin99uMRFD4DUiQt0w7d5GpQcs6q42zdEV4c=;
 b=dOe2ZzxLd7Fnf0BEO/ImzgRIN1Zhfc44eJvv0Tye8Sbt7Bg+yB2rJP/BJRPZtzTv0vw0YTrrYQWiu/qM8mOgQsqONRO3utuJS0r/8yS1gtLq3twqOC0A+1UX4mO52nu66UIqFGUzg2Z+Hq4sdGuI0O2JMkmtMu//mS+v9srcWaI=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
Thread-Topic: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
 device tree NUMA distance map
Thread-Index: AQHXjps+iLbrOQu8n06b5z+EtCRt06uM5puAgABnI4CAAPWOAIAA3gOA
Date: Wed, 1 Sep 2021 11:04:53 +0000
Message-ID:
 <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-25-wei.chen@arm.com>
 <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: C3D79CBE99666740BDBA0FEEF858F8C1.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 96c79db5-a1c1-4b46-aa1f-08d96d386131
x-ms-traffictypediagnostic: DB9PR08MB6507:|PAXPR08MB6365:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6365CD6324D3933E3A4090429ECD9@PAXPR08MB6365.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 OQhFkHAc/Vg6Pu292zrG1qOYz4jj8UnazzNez37Mwv7mV7ufB8Y2F7m1ZU51UsBKzzaWEiSTFp/Inufd8XpV11sBiUSoRjWR2998cn4P3hXlRbpH66bkPDMyc3jqQcp7WwW+BTyEjZJJiLYcrGi8Lb8wNdNGxEgPRsR/eDLtWScpcDK9x7Hx3UHkMO7fa6d5OnpKrOtBj4gjjNXcwcaRS2E38IvUAq+m6Zplpg94r1Isc7iMbhcTVfwHXeS+bQbgawvcY+DGEA5CixQgz5mp0IoZRM9C2EvMVBuQ4z74vY91iT5eu+tmGKYumQJlLgH11mBWv9kJVtrdZ2LV3hzxsxXo9+G6CZZpeUWYCXVHOZzuruDVi7cdHH4ZZfR8S6T1VxkHh3WuaqGhygP1Fvhee6IUz/SOBWJ6OkOpzTQAHv1wryD/Md0AA2jQ8fNygam++xC+cZAhaqzNpD5o81XIWEt+p6G/cYANV8pj8XBFJ7Znxp4CyY+gHuTLx8lI9s4BqaQBHKPWKX3se0pFO+vcf1g2kNib4frxA98V8KGgtrhTF/L38j8XFikQeDIQqODPMghrnJKa3gDUdea8Zd2pHx5v2PskjW92LapkH64d/s1aH37EakC2seioICHUzTKZLnkIYU2y4VWSTEQdFthGSFmpdgukLtrS+A8YVUw2wemJhex9okHZR/HAEREJekaoSkRrLEWmOx/OiMiffWDs2g==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(396003)(39860400002)(346002)(366004)(186003)(53546011)(66946007)(76116006)(55016002)(6506007)(7696005)(64756008)(52536014)(66446008)(66556008)(6916009)(8676002)(66476007)(478600001)(9686003)(4326008)(83380400001)(26005)(38070700005)(8936002)(33656002)(316002)(71200400001)(2906002)(122000001)(86362001)(38100700002)(5660300002)(54906003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?d0lteWo5c3pMdUQ4RlppK2kza0RENjNML1piR21Mc0VPdzhFaFU5Y2ZYb0lr?=
 =?utf-8?B?Y1FMUWROTmN2T24rcElQaTZKL0tieVpnMEFqUmpSODZ1NFQ2K3NWMEloMU13?=
 =?utf-8?B?TTBiTHNoeVJtY0MzRW9MN0VuaGpVZmw1UXZrZGx5bm5talc2ZmdwNDh1Zjl0?=
 =?utf-8?B?bXU3czFteEZBMmdhU0QzbDdKV1cwMmVLYkZWVGIxRDhvSituRHlsU250L0ZK?=
 =?utf-8?B?VGtyRFlUZmJzVHZOQ0UrUTIxMUZqUjFOMTBQV0REL3ZJZWFNeGtabGEydGdM?=
 =?utf-8?B?NG83Y21LQitMSUxpc3NrM1V6V2duekNpc3RMc0J0MmMxZDdDRU92R3phY2JL?=
 =?utf-8?B?SDM0YVNHanQzM0ZTa2JJalhOYUhnc21KSy96ZFhCR2NMb2ZDZHZBbTR0RlQ0?=
 =?utf-8?B?dkx4WjR6QkJmL25aK3NkajBtWUpsc1orbmthbnRZQXY3ZkNtNVVmZ3FhWVlR?=
 =?utf-8?B?d1FKaUk3a20wR2RIaUoybERqNlJyZmE2K1dnOHVYTTdsV21nU1lHTFdsUmUv?=
 =?utf-8?B?NXI0WCt6a011bXpKNU5rWGk3eWV0ZldsbXR5NlZ6ZU5OQkcwY3BINU9VRGtP?=
 =?utf-8?B?R3VXV1I3R0RLMHlkaFRUQUdnUVRtNjJQUHZYNkR5Sjh2bUNBaHBJU3lRYmdG?=
 =?utf-8?B?My8za09DZVZFV2VHWHgvVXdBSTdtY3J3SE03RHc3dVFKa3ozV3VKdXNtS1lE?=
 =?utf-8?B?VlRsM3ZKUU5NOGlyK0RLRDlrVFczUDRNcTM0L1YyRmxYQzdqaFMwRzA5UUdy?=
 =?utf-8?B?MzVRVURnVlhSdnMzcXZNVURnYlZUZUE1NTRlNExucnpha3dkWEdPSGlwd3Ns?=
 =?utf-8?B?TEVKSC9vbmcrL1MyN1ovdmx6RzYzYWdmdGZPanBLZVpXYkhZMXlFaHlvQ3hF?=
 =?utf-8?B?clNNMytwWDZmVHNUTm1wRlNjTUY1Nnc1SXRQaEVqNlB2aGlkNE96WmxYSTNS?=
 =?utf-8?B?bnhpR0d0TGMzUzl5MW1UOHpoNnZ6U2hxakFkOXFXajJyQWN6bkoxejlpbnlC?=
 =?utf-8?B?U3NMTldoNG5jU0FrdWlIT2s5T0doSUlVK1pEdUpXVFloNUlSMGZoYnJJay83?=
 =?utf-8?B?ZmRlVzhRMkFNamRjejMwaGdiVG1lK2NkSVQ2TlFFaWg4VnZUbFVDOCtWeDBE?=
 =?utf-8?B?UVAxdUZFb0F5d2VJOSt0bnl0OWF2dDRHRzNmdkFMZjBMbnFEVTVRKzdFdE9x?=
 =?utf-8?B?dkQrL25STFczcHAxMVBLQlYyd3pHWjZqUlRWbVUvcDgwUFExSkJKZ1RMdzBa?=
 =?utf-8?B?YjQ3OWZpTDczVFEwNmtacmphUHdMazJnNjhQYWh1bkE3UFMyV3NGb1V6LzZi?=
 =?utf-8?B?c3dRMFc5a1lCTUtZcktTUUQyTmlSaUZ0Qnlqb1hQWVdLOXl6SlU2S1RSRHBY?=
 =?utf-8?B?b3hoQ3p4R0d4cENDNnFVQ2NyRUpZTHlBM2duRW9YdXcvWWk1VXFiN3JLVDdK?=
 =?utf-8?B?cVV0dTVGSjNWejk1VDExSEVPV1p0RU84dXFjR2RGcGx5VG80U3lWcGZOMzFS?=
 =?utf-8?B?YmpGelhBeHJldHRzMStpYzF2UkhISHVJSm1HcW5CUmRrcDQ4eVNVbUtTWUU1?=
 =?utf-8?B?S0Jpd0dtTE1wbWF2RFkwa3BnMndQQU95WlAvNmprMHRybkdDT0ZJcEFka0Zp?=
 =?utf-8?B?SkV4djZRQVdlNFQ1U1BFamtGanZXbWxFR0pUV28veXVOc0dqOThlNWh4UTZz?=
 =?utf-8?B?cTNFL0xGTFZkMlc1ankxdW1xUFpPN1ZVOWRVU2RZNjlDYmhlSGFVUHFCMjU5?=
 =?utf-8?Q?59IMbKIbpR5+Cxou3R/O68xRyYuUo/dN9BaQYrh?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6507
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	4f0d553c-e065-4eea-5992-08d96d3853d0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JwArvXzP4kESMjXKXPw+MBfSiUEhjyMlWVR9wZtDAReobnoTMP71bp3aBYiHkF/Xwq1LUOe6U6ZAwrZMT3S9SEhILo4WNhGQCjlSALCSVOT8uvhSzjncoK/glw6s209EkOJO6jku3ahbZJ/2KIqnQzykIRchoUBaBsJWSoHMGeplXYsBRwFgILEkDQuNXH8wCnsGmzrq1suuTTd9hp4lZ2XJRjV/6Tdr0oCaH6vU6rwknmmlZsU4ZBRNYqC0d+RzaF4RKATlpNF0M5a6TkM4bzYXi+U6G5g5jDsHlkpnnGbU8YQJ9OYBuIQJ1FG4XBv0OANHZ1zuqhMFrMa588lZrCW2lm2Gbmtpun4wzhtLJMqAS/3JMC7w3EIVotPX371Tl07/1RgwllWSvx0/xExsyyaASAGDIFyzPZdc6egDtsiWGNLz9HkpKHdPEz5UdHsexZOdEv5m1rKFiB0CsBTNF7On9IJ85bsTMjcB4EY/12WxwI64qRcvgd93Defcp7odW97P4iK0W4sQpbL7dtTy0PS9Oxu3yxl/OAcM9okbfHdPVYuDdGd2BUmotAuk1+A7Z7C9vtbF5Fc5ODJYgBAfK4h87WoedNTgL34dNFBtXBkZPo1QzrQOg05KZ2a8Cv+ttxdcQ6KUPL/1XRz9Cji1BNog/gR4p15T/QN6Gtf3uldzkjGp4zQqCZ1FAyoLYcZmXNvvaP8ynUwdBRdbRYJ+pQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(336012)(53546011)(47076005)(8676002)(316002)(86362001)(7696005)(2906002)(5660300002)(356005)(70586007)(70206006)(83380400001)(81166007)(82310400003)(9686003)(508600001)(6506007)(8936002)(33656002)(4326008)(36860700001)(26005)(55016002)(52536014)(6862004)(186003)(54906003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 11:05:16.5177
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 96c79db5-a1c1-4b46-aa1f-08d96d386131
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6365

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBYZW4t
ZGV2ZWwgPHhlbi1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPiBPbiBCZWhhbGYg
T2YNCj4gU3RlZmFubyBTdGFiZWxsaW5pDQo+IFNlbnQ6IDIwMjHlubQ55pyIMeaXpSA1OjM2DQo+
IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0NCj4gZGV2ZWxAbGlzdHMueGVucHJvamVj
dC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+IDxCZXJ0cmFuZC5NYXJx
dWlzQGFybS5jb20+DQo+IFN1YmplY3Q6IFJFOiBbWEVOIFJGQyBQQVRDSCAyNC80MF0geGVuL2Fy
bTogaW50cm9kdWNlIGEgaGVscGVyIHRvIHBhcnNlDQo+IGRldmljZSB0cmVlIE5VTUEgZGlzdGFu
Y2UgbWFwDQo+IA0KPiBPbiBUdWUsIDMxIEF1ZyAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiBI
aSBTdGVmYW5vLA0KPiA+DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4g
RnJvbTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4g
U2VudDogMjAyMeW5tDjmnIgzMeaXpSA4OjQ4DQo+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVu
QGFybS5jb20+DQo+ID4gPiBDYzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3Rh
YmVsbGluaUBrZXJuZWwub3JnOw0KPiBqdWxpZW5AeGVuLm9yZzsNCj4gPiA+IGpiZXVsaWNoQHN1
c2UuY29tOyBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+ID4g
PiBTdWJqZWN0OiBSZTogW1hFTiBSRkMgUEFUQ0ggMjQvNDBdIHhlbi9hcm06IGludHJvZHVjZSBh
IGhlbHBlciB0bw0KPiBwYXJzZQ0KPiA+ID4gZGV2aWNlIHRyZWUgTlVNQSBkaXN0YW5jZSBtYXAN
Cj4gPiA+DQo+ID4gPiBPbiBXZWQsIDExIEF1ZyAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+
ID4gQSBOVU1BIGF3YXJlIGRldmljZSB0cmVlIHdpbGwgcHJvdmlkZSBhICJkaXN0YW5jZS1tYXAi
IG5vZGUgdG8NCj4gPiA+ID4gZGVzY3JpYmUgZGlzdGFuY2UgYmV0d2VlbiBhbnkgdHdvIG5vZGVz
LiBUaGlzIHBhdGNoIGludHJvZHVjZSBhDQo+ID4gPiA+IG5ldyBoZWxwZXIgdG8gcGFyc2UgdGhp
cyBkaXN0YW5jZSBtYXAuDQo+ID4gPiA+DQo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IFdlaSBDaGVu
IDx3ZWkuY2hlbkBhcm0uY29tPg0KPiA+ID4gPiAtLS0NCj4gPiA+ID4gIHhlbi9hcmNoL2FybS9u
dW1hX2RldmljZV90cmVlLmMgfCA2Nw0KPiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysNCj4gPiA+ID4gIDEgZmlsZSBjaGFuZ2VkLCA2NyBpbnNlcnRpb25zKCspDQo+ID4gPiA+DQo+
ID4gPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+ID4g
PiBiL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+ID4gaW5kZXggYmJlMDgx
ZGNkMS4uNmUwZDFkM2Q5ZiAxMDA2NDQNCj4gPiA+ID4gLS0tIGEveGVuL2FyY2gvYXJtL251bWFf
ZGV2aWNlX3RyZWUuYw0KPiA+ID4gPiArKysgYi94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJl
ZS5jDQo+ID4gPiA+IEBAIC0yMDAsMyArMjAwLDcwIEBAIGRldmljZV90cmVlX3BhcnNlX251bWFf
bWVtb3J5X25vZGUoY29uc3Qgdm9pZA0KPiAqZmR0LA0KPiA+ID4gaW50IG5vZGUsDQo+ID4gPiA+
DQo+ID4gPiA+ICAgICAgcmV0dXJuIDA7DQo+ID4gPiA+ICB9DQo+ID4gPiA+ICsNCj4gPiA+ID4g
Ky8qIFBhcnNlIE5VTUEgZGlzdGFuY2UgbWFwIHYxICovDQo+ID4gPiA+ICtpbnQgX19pbml0DQo+
ID4gPiA+ICtkZXZpY2VfdHJlZV9wYXJzZV9udW1hX2Rpc3RhbmNlX21hcF92MShjb25zdCB2b2lk
ICpmZHQsIGludCBub2RlKQ0KPiA+ID4gPiArew0KPiA+ID4gPiArICAgIGNvbnN0IHN0cnVjdCBm
ZHRfcHJvcGVydHkgKnByb3A7DQo+ID4gPiA+ICsgICAgY29uc3QgX19iZTMyICptYXRyaXg7DQo+
ID4gPiA+ICsgICAgaW50IGVudHJ5X2NvdW50LCBsZW4sIGk7DQo+ID4gPiA+ICsNCj4gPiA+ID4g
KyAgICBwcmludGsoWEVOTE9HX0lORk8gIk5VTUE6IHBhcnNpbmcgbnVtYS1kaXN0YW5jZS1tYXBc
biIpOw0KPiA+ID4gPiArDQo+ID4gPiA+ICsgICAgcHJvcCA9IGZkdF9nZXRfcHJvcGVydHkoZmR0
LCBub2RlLCAiZGlzdGFuY2UtbWF0cml4IiwgJmxlbik7DQo+ID4gPiA+ICsgICAgaWYgKCAhcHJv
cCApDQo+ID4gPiA+ICsgICAgew0KPiA+ID4gPiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5J
TkcNCj4gPiA+ID4gKyAgICAgICAgICAgICAgICJOVU1BOiBObyBkaXN0YW5jZS1tYXRyaXggcHJv
cGVydHkgaW4gZGlzdGFuY2UtDQo+IG1hcFxuIik7DQo+ID4gPiA+ICsNCj4gPiA+ID4gKyAgICAg
ICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gPiA+ICsgICAgfQ0KPiA+ID4gPiArDQo+ID4gPiA+ICsg
ICAgaWYgKCBsZW4gJSBzaXplb2YodWludDMyX3QpICE9IDAgKQ0KPiA+ID4gPiArICAgIHsNCj4g
PiA+ID4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19XQVJOSU5HDQo+ID4gPiA+ICsgICAgICAgICAg
ICAgICAiZGlzdGFuY2UtbWF0cml4IGluIG5vZGUgaXMgbm90IGEgbXVsdGlwbGUgb2YNCj4gdTMy
XG4iKTsNCj4gPiA+ID4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gPiA+ICsgICAgfQ0K
PiA+ID4gPiArDQo+ID4gPiA+ICsgICAgZW50cnlfY291bnQgPSBsZW4gLyBzaXplb2YodWludDMy
X3QpOw0KPiA+ID4gPiArICAgIGlmICggZW50cnlfY291bnQgPD0gMCApDQo+ID4gPiA+ICsgICAg
ew0KPiA+ID4gPiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcgIk5VTUE6IEludmFsaWQg
ZGlzdGFuY2UtbWF0cml4XG4iKTsNCj4gPiA+ID4gKw0KPiA+ID4gPiArICAgICAgICByZXR1cm4g
LUVJTlZBTDsNCj4gPiA+ID4gKyAgICB9DQo+ID4gPiA+ICsNCj4gPiA+ID4gKyAgICBtYXRyaXgg
PSAoY29uc3QgX19iZTMyICopcHJvcC0+ZGF0YTsNCj4gPiA+ID4gKyAgICBmb3IgKCBpID0gMDsg
aSArIDIgPCBlbnRyeV9jb3VudDsgaSArPSAzICkNCj4gPiA+ID4gKyAgICB7DQo+ID4gPiA+ICsg
ICAgICAgIHVpbnQzMl90IGZyb20sIHRvLCBkaXN0YW5jZTsNCj4gPiA+ID4gKw0KPiA+ID4gPiAr
ICAgICAgICBmcm9tID0gZHRfcmVhZF9udW1iZXIobWF0cml4LCAxKTsNCj4gPiA+ID4gKyAgICAg
ICAgbWF0cml4Kys7DQo+ID4gPiA+ICsgICAgICAgIHRvID0gZHRfcmVhZF9udW1iZXIobWF0cml4
LCAxKTsNCj4gPiA+ID4gKyAgICAgICAgbWF0cml4Kys7DQo+ID4gPiA+ICsgICAgICAgIGRpc3Rh
bmNlID0gZHRfcmVhZF9udW1iZXIobWF0cml4LCAxKTsNCj4gPiA+ID4gKyAgICAgICAgbWF0cml4
Kys7DQo+ID4gPiA+ICsNCj4gPiA+ID4gKyAgICAgICAgaWYgKCAoZnJvbSA9PSB0byAmJiBkaXN0
YW5jZSAhPSBOVU1BX0xPQ0FMX0RJU1RBTkNFKSB8fA0KPiA+ID4gPiArICAgICAgICAgICAgKGZy
b20gIT0gdG8gJiYgZGlzdGFuY2UgPD0gTlVNQV9MT0NBTF9ESVNUQU5DRSkgKQ0KPiA+ID4gPiAr
ICAgICAgICB7DQo+ID4gPiA+ICsgICAgICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcNCj4g
PiA+ID4gKyAgICAgICAgICAgICAgICAgICAiSW52YWxpZCBub2RlcycgZGlzdGFuY2UgZnJvbSBu
b2RlIyVkIHRvIG5vZGUjJWQNCj4gPiA+ID0gJWRcbiIsDQo+ID4gPiA+ICsgICAgICAgICAgICAg
ICAgICAgZnJvbSwgdG8sIGRpc3RhbmNlKTsNCj4gPiA+ID4gKyAgICAgICAgICAgIHJldHVybiAt
RUlOVkFMOw0KPiA+ID4gPiArICAgICAgICB9DQo+ID4gPiA+ICsNCj4gPiA+ID4gKyAgICAgICAg
cHJpbnRrKFhFTkxPR19JTkZPICJOVU1BOiBkaXN0YW5jZSBmcm9tIG5vZGUjJWQgdG8gbm9kZSMl
ZA0KPiA+ID4gPSAlZFxuIiwNCj4gPiA+ID4gKyAgICAgICAgICAgICAgIGZyb20sIHRvLCBkaXN0
YW5jZSk7DQo+ID4gPiA+ICsgICAgICAgIG51bWFfc2V0X2Rpc3RhbmNlKGZyb20sIHRvLCBkaXN0
YW5jZSk7DQo+ID4gPiA+ICsNCj4gPiA+ID4gKyAgICAgICAgLyogU2V0IGRlZmF1bHQgZGlzdGFu
Y2Ugb2Ygbm9kZSBCLT5BIHNhbWUgYXMgQS0+QiAqLw0KPiA+ID4gPiArICAgICAgICBpZiAodG8g
PiBmcm9tKQ0KPiA+ID4gPiArICAgICAgICAgICAgIG51bWFfc2V0X2Rpc3RhbmNlKHRvLCBmcm9t
LCBkaXN0YW5jZSk7DQo+ID4gPg0KPiA+ID4gSSBhbSBhIGJpdCB1bnN1cmUgYWJvdXQgdGhpcyBs
YXN0IDIgbGluZXM6IHdoeSBjYWxsaW5nDQo+IG51bWFfc2V0X2Rpc3RhbmNlDQo+ID4gPiBpbiB0
aGUgb3Bwb3NpdGUgZGlyZWN0aW9uIG9ubHkgd2hlbiB0byA+IGZyb20/IFdvdWxkbid0IGl0IGJl
IE9LIHRvDQo+ID4gPiBhbHdheXMgZG8gYm90aDoNCj4gPiA+DQo+ID4gPiBudW1hX3NldF9kaXN0
YW5jZShmcm9tLCB0bywgZGlzdGFuY2UpOw0KPiA+ID4gbnVtYV9zZXRfZGlzdGFuY2UodG8sIGZy
b20sIGRpc3RhbmNlKTsNCj4gPiA+DQo+ID4gPiA/DQo+ID4gPg0KPiA+IEkgYm9ycm93ZWQgdGhp
cyBjb2RlIGZyb20gTGludXgsIGJ1dCBoZXJlIGlzIG15IHVuZGVyc3RhbmRpbmc6DQo+ID4NCj4g
PiBGaXJzdCwgSSByZWFkIHNvbWUgbm90ZXMgaW4gRG9jdW1lbnRhdGlvbi9kZXZpY2V0cmVlL2Jp
bmRpbmdzL251bWEudHh0DQo+ID4gMS4gRWFjaCBlbnRyeSByZXByZXNlbnRzIGRpc3RhbmNlIGZy
b20gZmlyc3Qgbm9kZSB0byBzZWNvbmQgbm9kZS4NCj4gPiBUaGUgZGlzdGFuY2VzIGFyZSBlcXVh
bCBpbiBlaXRoZXIgZGlyZWN0aW9uLg0KPiA+IDIuIGRpc3RhbmNlLW1hdHJpeCBzaG91bGQgaGF2
ZSBlbnRyaWVzIGluIGxleGljb2dyYXBoaWNhbCBhc2NlbmRpbmcNCj4gPiBvcmRlciBvZiBub2Rl
cy4NCj4gPg0KPiA+IEhlcmUgaXMgYW4gZXhhbXBsZSBvZiBkaXN0YW5jZS1tYXAgbm9kZSBpbiBE
VEI6DQo+ID4gU2FtcGxlIzEsIGZ1bGwgbGlzdDoNCj4gPiAJCWRpc3RhbmNlLW1hcCB7DQo+ID4g
CQkJIGNvbXBhdGlibGUgPSAibnVtYS1kaXN0YW5jZS1tYXAtdjEiOw0KPiA+IAkJCSBkaXN0YW5j
ZS1tYXRyaXggPSA8MCAwICAxMD4sDQo+ID4gCQkJCQkgICA8MCAxICAyMD4sDQo+ID4gCQkJCQkg
ICA8MCAyICA0MD4sDQo+ID4gCQkJCQkgICA8MCAzICAyMD4sDQo+ID4gCQkJCQkgICA8MSAwICAy
MD4sDQo+ID4gCQkJCQkgICA8MSAxICAxMD4sDQo+ID4gCQkJCQkgICA8MSAyICAyMD4sDQo+ID4g
CQkJCQkgICA8MSAzICA0MD4sDQo+ID4gCQkJCQkgICA8MiAwICA0MD4sDQo+ID4gCQkJCQkgICA8
MiAxICAyMD4sDQo+ID4gCQkJCQkgICA8MiAyICAxMD4sDQo+ID4gCQkJCQkgICA8MiAzICAyMD4s
DQo+ID4gCQkJCQkgICA8MyAwICAyMD4sDQo+ID4gCQkJCQkgICA8MyAxICA0MD4sDQo+ID4gCQkJ
CQkgICA8MyAyICAyMD4sDQo+ID4gCQkJCQkgICA8MyAzICAxMD47DQo+ID4gCQl9Ow0KPiA+DQo+
ID4gQ2FsbCBudW1hX3NldF9kaXN0YW5jZSB3aGVuICJ0byA+IGZyb20iIHdpbGwgcHJldmVudCBY
ZW4gdG8gY2FsbA0KPiA+IG51bWFfc2V0X2Rpc3RhbmNlKDAsIDEsIDIwKSBhZ2FpbiB3aGVuIGl0
J3Mgc2V0dGluZyBkaXN0YW5jZSBmb3IgPDEgMA0KPiAyMD4uDQo+ID4gQnV0LCBudW1hX3NldF9k
aXN0YW5jZSgxLCAwLCAyMCkgd2lsbCBiZSBjYWxsIHR3aWNlLg0KPiA+DQo+ID4gTm9ybWFsbHks
IGRpc3RhbmNlLW1hcCBub2RlIHdpbGwgYmUgb3B0aW1pemVkIGluIGZvbGxvd2luZyBzYW1wbGUj
MiwNCj4gPiBhbGwgcmVkdW5kYW50IGVudHJpZXMgYXJlIHJlbW92ZWQ6DQo+ID4gU2FtcGxlIzIs
IHBhcnRpYWwgbGlzdDoNCj4gPiAJCWRpc3RhbmNlLW1hcCB7DQo+ID4gCQkJIGNvbXBhdGlibGUg
PSAibnVtYS1kaXN0YW5jZS1tYXAtdjEiOw0KPiA+IAkJCSBkaXN0YW5jZS1tYXRyaXggPSA8MCAw
ICAxMD4sDQo+ID4gCQkJCQkgICA8MCAxICAyMD4sDQo+ID4gCQkJCQkgICA8MCAyICA0MD4sDQo+
ID4gCQkJCQkgICA8MCAzICAyMD4sDQo+ID4gCQkJCQkgICA8MSAxICAxMD4sDQo+ID4gCQkJCQkg
ICA8MSAyICAyMD4sDQo+ID4gCQkJCQkgICA8MSAzICA0MD4sDQo+ID4gCQkJCQkgICA8MiAyICAx
MD4sDQo+ID4gCQkJCQkgICA8MiAzICAyMD4sDQo+ID4gCQkJCQkgICA8MyAzICAxMD47DQo+ID4g
CQl9Ow0KPiA+DQo+ID4gVGhlcmUgaXMgbm90IGFueSAiZnJvbSA+IHRvIiBlbnRyeSBpbiB0aGUg
bWFwLiBCdXQgdXNpbmcgdGhpcyBwYXJ0aWFsDQo+IG1hcA0KPiA+IHN0aWxsIGNhbiBzZXQgYWxs
IGRpc3RhbmNlcyBmb3IgYWxsIHBhaXJzLiBBbmQgbnVtYV9zZXRfZGlzdGFuY2UoMSwgMCwNCj4g
MjApDQo+ID4gd2lsbCBiZSBvbmx5IG9uY2UuDQo+IA0KPiBJIHNlZS4gSSBjYW4ndCBmaW5kIGlu
IERvY3VtZW50YXRpb24vZGV2aWNldHJlZS9iaW5kaW5ncy9udW1hLnR4dCB3aGVyZQ0KPiBpdCBz
YXlzIHRoYXQgImZyb20gPiB0byIgbm9kZXMgY2FuIGJlIG9taXR0ZWQuIElmIGl0IGlzIG5vdCB3
cml0dGVuDQo+IGRvd24sIHRoZW4gc29tZWJvZHkgY291bGQgZWFzaWx5IG9wdGltaXplIGl0IHRo
ZSBvcHBvc2l0ZSB3YXk6DQo+IA0KPiAgCQkJIGRpc3RhbmNlLW1hdHJpeCA9IDwwIDAgIDEwPiwN
Cj4gIAkJCQkJICAgPDEgMCAgMjA+LA0KPiAgCQkJCQkgICA8MiAwICA0MD4sDQo+ICAJCQkJCSAg
IDwzIDAgIDIwPiwNCj4gIAkJCQkJICAgPDEgMSAgMTA+LA0KPiAgCQkJCQkgICA8MiAxICAyMD4s
DQo+ICAJCQkJCSAgIDwzIDEgIDQwPiwNCj4gIAkJCQkJICAgPDIgMiAgMTA+LA0KPiAgCQkJCQkg
ICA8MyAyICAyMD4sDQo+ICAJCQkJCSAgIDwzIDMgIDEwPjsNCj4gDQoNClllcywgeW91J3JlIHJp
Z2h0LiBTcGVjIGRvZXNuJ3Qgc2F5IG9wcG9zaXRlIHdheSBpcyB1bmFsbG93ZWQuDQoNCj4gSSB0
aGluayB0aGUgY29kZSBpbiBYZW4gc2hvdWxkIGJlIHJlc2lsaWVudCBhbmQgYWJsZSB0byBjb3Bl
IHdpdGggYQ0KPiBkZXZpY2UgdHJlZSBsaWtlIHRoZSBvbmUgeW91IHdyb3RlIG9yIHRoZSBvbmUg
SSB3cm90ZS4gRnJvbSBhIGNvZGUNCj4gcGVyc3BlY3RpdmUsIGl0IHNob3VsZCBiZSB2ZXJ5IGVh
c3kgdG8gZG8uIElmIG5vdGhpbmcgZWxzZSBpdCB3b3VsZCBtYWtlDQo+IFhlbiBtb3JlIHJlc2ls
aWVudCBhZ2FpbnN0IGJ1Z2d5IGZpcm13YXJlLg0KPiANCj4gDQoNCkkgZG9uJ3QgZGlzYWdyZWUg
d2l0aCB0aGF0Lg0KDQo+ID4gPiBCdXQgaW4gYW55IGNhc2UsIEkgaGF2ZSBhIGRpZmZlcmVudCBz
dWdnZXN0aW9uLiBUaGUgYmluZGluZyBzdGF0ZXMNCj4gdGhhdA0KPiA+ID4gImRpc3RhbmNlcyBh
cmUgZXF1YWwgaW4gZWl0aGVyIGRpcmVjdGlvbiIuIEFsc28gaXQgaGFzIGFuIGV4YW1wbGUNCj4g
d2hlcmUNCj4gPiA+IG9ubHkgb25lIGRpcmVjdGlvbiBpcyBleHByZXNzZWQgdW5mb3J0dW5hdGVs
eSAoYXQgdGhlIGVuZCBvZiB0aGUNCj4gPiA+IGRvY3VtZW50KS4NCj4gPiA+DQo+ID4NCj4gPiBP
aCwgSSBzaG91bGQgc2VlIHRoaXMgY29tbWVudCBmaXJzdCwgdGhlbiBJIHdpbGwgbm90IHBvc3Qg
YWJvdmUNCj4gPiBjb21tZW50IDogKQ0KPiA+DQo+ID4gPiBTbyBteSBzdWdnZXN0aW9uIGlzIHRv
IHBhcnNlIGl0IGFzIGZvbGxvd3M6DQo+ID4gPg0KPiA+ID4gLSBjYWxsIG51bWFfc2V0X2Rpc3Rh
bmNlIGp1c3Qgb25jZSBmcm9tDQo+ID4gPiAgIGRldmljZV90cmVlX3BhcnNlX251bWFfZGlzdGFu
Y2VfbWFwX3YxDQo+ID4gPg0KPiA+ID4gLSBpbiBudW1hX3NldF9kaXN0YW5jZToNCj4gPiA+ICAg
ICAtIHNldCBub2RlX2Rpc3RhbmNlX21hcFtmcm9tXVt0b10gPSBkaXN0YW5jZTsNCj4gPiA+ICAg
ICAtIGNoZWNrIG5vZGVfZGlzdGFuY2VfbWFwW3RvXVtmcm9tXQ0KPiA+ID4gICAgICAgICAgIC0g
aWYgdW5zZXQsIG5vZGVfZGlzdGFuY2VfbWFwW3RvXVtmcm9tXSA9IGRpc3RhbmNlOw0KPiA+ID4g
ICAgICAgICAgIC0gaWYgYWxyZWFkeSBzZXQgdG8gdGhlIHNhbWUgdmFsdWUsIHJldHVybiBzdWNj
ZXNzOw0KPiA+ID4gICAgICAgICAgIC0gaWYgYWxyZWFkeSBzZXQgdG8gYSBkaWZmZXJlbnQgdmFs
dWUsIHJldHVybiBlcnJvcjsNCj4gPg0KPiA+IEkgZG9uJ3QgcmVhbGx5IGxpa2UgdGhpcyBpbXBs
ZW1lbnRhdGlvbi4gSSB3YW50IHRoZSBiZWhhdmlvciBvZg0KPiA+IG51bWFfc2V0X2Rpc3RhbmNl
IGp1c3QgbGlrZSB0aGUgZnVuY3Rpb24gbmFtZSwgZG8gbm90IGluY2x1ZGUNCj4gPiBpbXBsaWNp
dCBvcGVyYXRpb25zLiBPdGhlcndpc2UsIGV4Y2VwdCB0aGUgdXNlciByZWFkIHRoaXMgZnVuY3Rp
b24NCj4gPiBpbXBsZW1lbnRhdGlvbiBiZWZvcmUgaGUgdXNlIGl0LCBoZSBwcm9iYWJseSBkb2Vz
bid0IGtub3cgdGhpcw0KPiA+IGZ1bmN0aW9uIGhhcyBkb25lIHNvIG1hbnkgdGhpbmdzLg0KPiAN
Cj4gWW91IGNhbiBsZWF2ZSBudW1hX3NldF9kaXN0YW5jZSBhcy1pcyB3aXRob3V0IGFueSBpbXBs
aWNpdCBvcGVyYXRpb25zLg0KPiANCj4gSW4gdGhhdCBjYXNlLCBqdXN0IGNhbGwgbnVtYV9zZXRf
ZGlzdGFuY2UgdHdpY2UgZnJvbSBudW1hX3NldF9kaXN0YW5jZQ0KPiBmb3IgYm90aCBmcm9tL3Rv
IGFuZCB0by9mcm9tLiBudW1hX3NldF9kaXN0YW5jZSBjb3VsZCByZXR1cm4gZXJyb3IgaXMNCg0K
SSBhbSBPSyBmb3IgdGhlIGZpcnN0IHNlbnRlbmNlLiBCdXQuLi4NCg0KPiB0aGUgZW50cnkgd2Fz
IGFscmVhZHkgc2V0IHRvIGEgZGlmZmVyZW50IHZhbHVlIG9yIHN1Y2Nlc3Mgb3RoZXJ3aXNlDQo+
IChhbHNvIGluIHRoZSBjYXNlIGl0IHdhcyBhbHJlYWR5IHNldCB0byB0aGUgc2FtZSB2YWx1ZSku
IFRoaXMgd291bGQNCg0KLi4uIEkgcHJlZmVyIG5vdCB0byBjaGVjayB0aGUgcHJldmlvdXMgdmFs
dWUuIFN1YnNlcXVlbnQgbnVtYV9zZXRfZGlzdGFuY2UNCmNhbGwgd2lsbCBvdmVycmlkZSBwcmV2
aW91cyBjYWxscy4gS2VlcCBudW1hX3NldF9kaXN0YW5jZSBhcyBzaW1wbGUgYXMNCml0IGNhbi4g
QW5kIHdoZW4geW91IHBhc3MgbmV3IGRhdGEgdG8gbnVtYV9zZXRfZGlzdGFuY2UsIGl0IGRvZXNu
J3QNCmtub3cgd2hldGhlciB0aGUgcHJldmlvdXMgZGF0YSB3YXMgY29ycmVjdCBvciB0aGUgbmV3
IGRhdGEgaXMgY29ycmVjdC4NCk9ubHkgY2FsbGVyIG1heSBoYXZlIGtub3duLiAgDQoNCj4gYWxs
b3cgWGVuIHRvIGNvcGUgd2l0aCBib3RoIHNjZW5hcmlvcyBhYm92ZS4NCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 11:13:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 11:13:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176307.320819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOBH-00029A-NC; Wed, 01 Sep 2021 11:13:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176307.320819; Wed, 01 Sep 2021 11:13:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOBH-000293-Jq; Wed, 01 Sep 2021 11:13:23 +0000
Received: by outflank-mailman (input) for mailman id 176307;
 Wed, 01 Sep 2021 11:13:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1ICC=NX=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLOBG-00028x-Sr
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 11:13:22 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.76]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9bb5ca6c-0b15-11ec-adc6-12813bfff9fa;
 Wed, 01 Sep 2021 11:13:18 +0000 (UTC)
Received: from DU2PR04CA0325.eurprd04.prod.outlook.com (2603:10a6:10:2b5::30)
 by AM0PR08MB4194.eurprd08.prod.outlook.com (2603:10a6:208:130::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 11:13:13 +0000
Received: from DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::24) by DU2PR04CA0325.outlook.office365.com
 (2603:10a6:10:2b5::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Wed, 1 Sep 2021 11:13:13 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT036.mail.protection.outlook.com (10.152.20.185) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.17 via Frontend Transport; Wed, 1 Sep 2021 11:13:13 +0000
Received: ("Tessian outbound 32695b2df2f8:v103");
 Wed, 01 Sep 2021 11:13:12 +0000
Received: from bd6ec8162f79.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BA8665A1-F513-4FEB-8A42-7A1934CDEE01.1; 
 Wed, 01 Sep 2021 11:13:03 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bd6ec8162f79.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 Sep 2021 11:13:03 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4887.eurprd08.prod.outlook.com (2603:10a6:10:d7::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 11:12:59 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.024; Wed, 1 Sep 2021
 11:12:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9bb5ca6c-0b15-11ec-adc6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qymYKIcXOO0dP0B4z6PRI6eakNqJsD30akV7Mw1HHHg=;
 b=rX+Yy4gsYmATFacMupKndvzmQVMDbH1PkzeRfORDwAhFm4WU/i1cueDAK9HTmfZuWfehm3GxZzrrvk/KBBYd0IyzrLMlznRLC3QoCnyKHi4YbFCevrQqW6U23bhwyuhM5UDtcjLC5jz71ck12G5FCZeno1z90AksAbj/BTDL5us=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xen.org; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;lists.xen.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: b2e5d06b9e648783
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iJm09T/LgAhZ6lxIoeNR5kpvzsWp2QAtwZW6Anlrh63hWGCrIgs5heMYE2umvMh0m5pI4N6RsVWh8UQiLZsTHEOOkbabtsHbmbcTRqaVBqTbdYdlPKyP0j4/OfBnJnM7N09sNpUrAioXnyuHSxTb46X1kcidPnFwdbsqKBEkpXQmseZCWhJkalQ0Tfr+Dsvv/V4YLPhVwLHHVjqqN8UOA29LQQKoeJ1H/5ia1Dx87bU/EYGtSOUwhfS6ovyf8b4G9IZXiLSek/NYWURpm+GgJAM+SSfPDGz8NdhFaVnYN0bWRKE1UUZzX4pCfjbfWq/iphOhxpFQnFizGm0c4UyBFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qymYKIcXOO0dP0B4z6PRI6eakNqJsD30akV7Mw1HHHg=;
 b=hyDev9GVkOiJbP650AyNIlaMdCLxvwIZ9zf46NixmDHTXlCnTHbsmWuSML6aue9AbPIXb1pMYKKCiME+niZIFztf3iXwbPIX3Kk4GaW4rqLWQ7hGIcS81MHy9AT0lWOEsXIQdfvpz23nzVZ+zjPTkaTE22eaaEmEW11G6UE+agAOhVN67AER7nfa/lqvWa4Zi7nMzGgICsSz6nWaQt6EEhWbaSHbTZtmC79lC7P7iLqr1z2bsG06+D+2MQFN+Z4u0l3P7RuRisHUTKq+lF/QVhBcgddxWfxdAlnibFvETSy5Qwt6I8OAtxyGhEogMhQGofQ0OQKK3s8MMGCa+bl3DA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qymYKIcXOO0dP0B4z6PRI6eakNqJsD30akV7Mw1HHHg=;
 b=rX+Yy4gsYmATFacMupKndvzmQVMDbH1PkzeRfORDwAhFm4WU/i1cueDAK9HTmfZuWfehm3GxZzrrvk/KBBYd0IyzrLMlznRLC3QoCnyKHi4YbFCevrQqW6U23bhwyuhM5UDtcjLC5jz71ck12G5FCZeno1z90AksAbj/BTDL5us=
From: Wei Chen <Wei.Chen@arm.com>
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
CC: Oleksandr Tyshchenko <olekstysh@gmail.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin
	<Kaly.Xin@arm.com>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd
 Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com"
	<stefanha@redhat.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik
	<cvanscha@qti.qualcomm.com>, "pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker
	<jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien
 Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, Paul Durrant
	<paul@xen.org>, nd <nd@arm.com>, Xen Devel <xen-devel@lists.xen.org>
Subject: RE: Enabling hypervisor agnosticism for VirtIO backends
Thread-Topic: Enabling hypervisor agnosticism for VirtIO backends
Thread-Index:
 AQHXiWXOlXnRp9GT30mY/CfP5cid16tt4TUAgAVQpQCAAsKLQIABdsmAgAADLTCAAWVyAIAAMB+ggAMGMQCACZ/8gIAAKWUQgAd5oYCAAeKfwA==
Date: Wed, 1 Sep 2021 11:12:58 +0000
Message-ID:
 <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811062748.GB54169@laputa>
 <CAPD2p-mMeY=MDbAdLGrmmioSkJo147aMDrK=Qzr=PCa4jztGGg@mail.gmail.com>
 <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa>
 <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210831061805.GB69817@laputa>
In-Reply-To: <20210831061805.GB69817@laputa>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: D25FD981A493B04A9A287D69C0E5CFEB.0
Authentication-Results-Original: linaro.org; dkim=none (message not signed)
 header.d=none;linaro.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c29a6f61-2d92-4bc3-74e8-08d96d397d37
x-ms-traffictypediagnostic: DBBPR08MB4887:|AM0PR08MB4194:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB41949700828443EE634824229ECD9@AM0PR08MB4194.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 3D+9LuLEUPjDhKQM7OQC2G2wWIqfA1xYzIQ/Ue2vZumRNJBbfelqjG7Lj6NhUjUgbp2yoqQX6FZbIRJ15XJyyeHnoO4vXCQkTphkwON0U3hPoXPr6xqnslxTG+gv9E8ATlYPXXW3SW3cjsdjPn3DN4MGMPAJPJV02kL3RXU/t+z9WLOwERuhG5QWFrWmIQEtF+20cGZEHjHkvya01NQ7PWI39fU5CqUl/VcwkZFITtq3aUwJMm+cFiyZhJrszuxG8QzW7rjQPoDKzyU8qtSFn60Vg0OEZZZqShpxbgRU+JkPLCk0qK6UBv3/5tYCENbXtFwlqweIWijQVQhnZ9CLsT02aCfCmCWhNYVt0w1o5RZ5jT0ZW2gNV9Rd+FNI78/CjUhf/b9MZ487oV4iIwG+UO4NoFa68ClA9UiGhhyfGOKHAJhHCCvJCivumhwKN2JYz2PSX6sCo9+nN6JUTW+d0CjQZfqlka+CxK8kIs3VR+hcw+sUFr2rmBW+NMb1fGBLrlU75426tJmWCSFFVqzUhoprwDATcLB9D/NRouM50Z+TtN9cNplE71YLy+1Hnp7vTF9lsi0sHvuS4zhjw1ZxAn6k2810UMlh5X9c+INcgRmvfTJhXykUry+YFbNqNFGgztNLrTGgclmSgSYmkHICKYsrJYdXh6BZotz1O2xevlD98eT2qR0v6005UVUmFVZEwPzliasK/b62sKFeESSxkAEVeOJ4AWoP3diPt1C/ilYoL0tH4G3v4+21bQJUg+d4BFJ/PZfTlRnXlzJmiUh+SBahCQbKjPyncsEwxLEwHKfFJ/Sd4qnuUIg5ZlcfcXMuawbn+cpves41PIAx2sA4Bw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(71200400001)(8936002)(66476007)(33656002)(6506007)(478600001)(52536014)(122000001)(7696005)(53546011)(38100700002)(55016002)(966005)(9686003)(54906003)(30864003)(38070700005)(8676002)(186003)(26005)(76116006)(2906002)(7416002)(86362001)(316002)(66446008)(66556008)(66946007)(83380400001)(5660300002)(64756008)(66574015)(6916009)(4326008)(559001)(579004);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?OEZqL1dTREV1SE9UK012VkVZaUZCOEpWR2sxbnhzNGNWd3RzSDRuUTNHVmEy?=
 =?utf-8?B?U3dPRTZCY2JtYmZCckRzOGJVVXk3UXMrcW5xZHNrLzFrVHlPdFB3YTg5Wmpq?=
 =?utf-8?B?U0xSWStOMVRtM0F1ZFpDUndJdkYvVHBlMEhwRVFXWDBsOHQvQ20xbFlSMHp3?=
 =?utf-8?B?QTdMZUZFbXpET01CMUoxWFR6RWRjZlBnazEzL1lRTUI5MVlEUmRzQXhNQXpR?=
 =?utf-8?B?TzUxUFJ6b2xFaU03T1RkUEpodERkWXVNbG5jYk80QkxabWJyTFUwOHkwYk51?=
 =?utf-8?B?ekgyaXJLNVpoUm1Za0xicVV2SG1KODRwekRSbXYwdTlJZThvR2V3bmErYlhi?=
 =?utf-8?B?OFB4TW1Xek0zbEhQNzVIMjNsM1loYktMU0ZkVkwyWG9SVkZ5M2U3YkdaUWN5?=
 =?utf-8?B?TUo0T2tpTDBFUGRJdXNxY3pJSXAzZTZIMk9yL2xiZGFmRTFKYmREVjgzdG1w?=
 =?utf-8?B?b21rWmZOQXVDWm9nTDl6b3p4bitPdnV0TVY3Y016b1RWNkY2WCs5ZUpjZGx0?=
 =?utf-8?B?aGhueWpLc010OWRZeXpQL0VJRmo1bmFyc04xOHJPNC82NThDQzFoZjZ6K2hD?=
 =?utf-8?B?WDJYcUI2TmlwNThTbWZZeEIwVXM1R0RJR0NlSWI1WnhKUDFGQkFLdi9YNVdB?=
 =?utf-8?B?b2ZDYWVtWm53WUJLeHVzN0R2NWQxcitzaExaV3NnUnJ3WjI5UE9zR2RBU3NJ?=
 =?utf-8?B?Rlh2N1NOdkVEUVFSMXB0RHJMVUJYajlRckhQbXZrNnJldEd3enJWT1lUckVC?=
 =?utf-8?B?L2FSYWNVTEkyVXVENGFCeVV5SDZNWVFVam9yNkpWSVhUNU9IS1Y3Sk10L2Qz?=
 =?utf-8?B?LzQ3OXRZUVUrSzJqZjBGeHRTSmtqajUrQTJHelV3K002NW1wUWJZNktScjk0?=
 =?utf-8?B?RWE2eXlVNk5yNEFJenY3WFV0bVVWOGh4RVI1MXB1cUl5ZVRyeTZSTUxxMXJV?=
 =?utf-8?B?Y21sQUxqS1g2NFVObW1XQmdUazRrMGRmdHZRcFJzaG9jOHo1N2lGVnpYTUlp?=
 =?utf-8?B?MU5kTnJWa0hxU2crcHpBL0Z3NEg5TmZjcGIza0pzdnZLejFaNlZKSzNYWkFR?=
 =?utf-8?B?OVRzdXlqN0dIRmlrOWxSd2NwQjhmaThkSWdqVldUaEE3TUkyV3YzZUFuV0Rq?=
 =?utf-8?B?d3BTTFlQT0xWajNWemZDY3BaZjJBZXZFSE54c3JIelJyUTRhVFo0NHRmSCtM?=
 =?utf-8?B?NEN2NTBSK2s1N0xYVklUVGQvWlkvemluOTlLMTBtTWFOOFZKYXNVNXVZUDZP?=
 =?utf-8?B?U3pTb1pLbERndkNvMmRWdHZzZWtQaVpCbUlNK1N6ek5SQVVpYnpTaGZXNCts?=
 =?utf-8?B?bkRCS1hFaEFDVWVFU2loTDQzd0taQ2FPL0srYTVyKzdJTEJMZTlDV09PL052?=
 =?utf-8?B?OHNXTTJCa21RMjFUZ0g3N3lqaDJiNzdrUi80M1Y3ejJsb1RMQWoyT1dKUFR4?=
 =?utf-8?B?eWltQVVyOTBidEJ6Zm1ITGdmRkNCNkpna1VDcm03cjBDcVF0TnF2bm9sSTR1?=
 =?utf-8?B?bGM5bnI3YW9wTHk3d0dEZVJJM1lIeGR4dUpIZTJTNDFYQWpLTm9FZnRvMnE5?=
 =?utf-8?B?R0U1SG5NMy9hNDhtR0NyU1k3dE5xYmFvaVBXNmYrc3JOSGNnVWRqMHRhYXNl?=
 =?utf-8?B?K2R2RjY2b0g3TnFkU3Fya04vdUlPejNJTXRiU09kSWpteUthY29jYW5vRU5k?=
 =?utf-8?B?dE4yOTZTUjBKekFWaUJBUy9xZWN6RTI3VFYxcSszK3YxNWlFeXFtVHRvS3ox?=
 =?utf-8?Q?PFUtBbLwHxHzoidzA3sVo8UOQuRBXiDFGe3eVMG?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4887
Original-Authentication-Results: linaro.org; dkim=none (message not signed)
 header.d=none;linaro.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	cf1ce180-4049-448c-436d-08d96d3974f3
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DuU9i8WxAogp7VDvU7Ka+zJZHBcrQEFCj/Yu3sfj60fX9/ZvSM4Gx4VS5ERSEHQHLBlSzN0zjyByez4NTX7CWbkx5iNqFa+PubhhP1s06X77fomdZ3Ea5Du5g9js1GXzzZCzo+O431AO1lti8Ly/TQoSdycM2toZX0hqDS5R9ID9yBfV0NtJAlw+Cs2f1TWEe9wVcKEYUGPGCP6mqls+Et5stFfM4cj1vxQeEqwP+lGUoY12KcuvJflJKUNGV6fn5KAQGvYQOv9eMoOEicyFB5LpWjxIwDzOyqlyw0J5xFmIlyO/JBfINHV3PXUZ462f7dhUhAwPExJ39hGRBeR0i++c6dthZ0Mh1p9xrE4naXKgiOvBDVrMztinCtYolutbIJXdxe6oFTMscdkZFYBNmzGvpC6Ft+EB7qmCN3ZtG29LGH4EX1lb41haOUXAsGB13FJhBQmwvSr3eZsEL5MicwvK7VeWB1+TnEh+Zcf1zd0HGdkdLIQhW4jzYK3zNK6IrRAli10jpQiStiRP3HJwOSw667KdJMnYhKOkXbo6WkKylxedfwDbfpF5UV6PUTzwV9Ph/BpKboUKzH9C7YKRFWh4uhn+1yiaQf5bxs+xKnND6PoaUAkj+gvQm9TJ5j4v7GGNU3TAJwoJuJalF2VBHlAqHb396GwBnAgUlinJRg4DDN+tUowjhAamwQee6OAAIbg4ZJ4xvZ0Dzxcc3Zp5J5YfNV9UYl+u9PQXlg408UMF3Cw60JvZ/Trxt6xSYFx0B/XQhr6OOaMUb0jBJ/TbEUKQmsx55GiIMGoOIWTIue55KdcSfsuqCm1qq+o9R7cNH+j4aXJ5FMaRMPZLORA70w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(316002)(5660300002)(966005)(508600001)(36860700001)(6506007)(9686003)(30864003)(53546011)(55016002)(8936002)(82310400003)(70206006)(26005)(70586007)(4326008)(6862004)(54906003)(86362001)(8676002)(7696005)(2906002)(33656002)(83380400001)(52536014)(356005)(81166007)(47076005)(186003)(336012)(66574015)(579004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 11:13:13.0900
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c29a6f61-2d92-4bc3-74e8-08d96d397d37
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4194

SGkgQWthc2hpLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEFLQVNI
SSBUYWthaGlybyA8dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ4
5pyIMzHml6UgMTQ6MTgNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzog
T2xla3NhbmRyIFR5c2hjaGVua28gPG9sZWtzdHlzaEBnbWFpbC5jb20+OyBTdGVmYW5vIFN0YWJl
bGxpbmkNCj4gPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyBBbGV4IEJlbm4/P2UgPGFsZXguYmVu
bmVlQGxpbmFyby5vcmc+OyBLYWx5IFhpbg0KPiA8S2FseS5YaW5AYXJtLmNvbT47IFN0cmF0b3Mg
TWFpbGluZyBMaXN0IDxzdHJhdG9zLWRldkBvcC1saXN0cy5saW5hcm8ub3JnPjsNCj4gdmlydGlv
LWRldkBsaXN0cy5vYXNpcy1vcGVuLm9yZzsgQXJuZCBCZXJnbWFubiA8YXJuZC5iZXJnbWFubkBs
aW5hcm8ub3JnPjsNCj4gVmlyZXNoIEt1bWFyIDx2aXJlc2gua3VtYXJAbGluYXJvLm9yZz47IFN0
ZWZhbm8gU3RhYmVsbGluaQ0KPiA8c3RlZmFuby5zdGFiZWxsaW5pQHhpbGlueC5jb20+OyBzdGVm
YW5oYUByZWRoYXQuY29tOyBKYW4gS2lzemthDQo+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsg
Q2FybCB2YW4gU2NoYWlrIDxjdmFuc2NoYUBxdGkucXVhbGNvbW0uY29tPjsNCj4gcHJhdGlrcEBx
dWljaW5jLmNvbTsgU3JpdmF0c2EgVmFkZGFnaXJpIDx2YXRzYUBjb2RlYXVyb3JhLm9yZz47IEpl
YW4tDQo+IFBoaWxpcHBlIEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhp
ZXUgUG9pcmllcg0KPiA8bWF0aGlldS5wb2lyaWVyQGxpbmFyby5vcmc+OyBPbGVrc2FuZHIgVHlz
aGNoZW5rbw0KPiA8T2xla3NhbmRyX1R5c2hjaGVua29AZXBhbS5jb20+OyBCZXJ0cmFuZCBNYXJx
dWlzDQo+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2IDxBcnRlbV9N
eWdhaWV2QGVwYW0uY29tPjsgSnVsaWVuDQo+IEdyYWxsIDxqdWxpZW5AeGVuLm9yZz47IEp1ZXJn
ZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT47IFBhdWwgRHVycmFudA0KPiA8cGF1bEB4ZW4ub3Jn
PjsgWGVuIERldmVsIDx4ZW4tZGV2ZWxAbGlzdHMueGVuLm9yZz4NCj4gU3ViamVjdDogUmU6IEVu
YWJsaW5nIGh5cGVydmlzb3IgYWdub3N0aWNpc20gZm9yIFZpcnRJTyBiYWNrZW5kcw0KPiANCj4g
V2VpLA0KPiANCj4gT24gVGh1LCBBdWcgMjYsIDIwMjEgYXQgMTI6MTA6MTlQTSArMDAwMCwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gSGkgQWthc2hpLA0KPiA+DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1l
c3NhZ2UtLS0tLQ0KPiA+ID4gRnJvbTogQUtBU0hJIFRha2FoaXJvIDx0YWthaGlyby5ha2FzaGlA
bGluYXJvLm9yZz4NCj4gPiA+IFNlbnQ6IDIwMjHlubQ45pyIMjbml6UgMTc6NDENCj4gPiA+IFRv
OiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiA+IENjOiBPbGVrc2FuZHIgVHlzaGNo
ZW5rbyA8b2xla3N0eXNoQGdtYWlsLmNvbT47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+ID4gPHNz
dGFiZWxsaW5pQGtlcm5lbC5vcmc+OyBBbGV4IEJlbm4/P2UgPGFsZXguYmVubmVlQGxpbmFyby5v
cmc+OyBLYWx5DQo+IFhpbg0KPiA+ID4gPEthbHkuWGluQGFybS5jb20+OyBTdHJhdG9zIE1haWxp
bmcgTGlzdCA8c3RyYXRvcy1kZXZAb3AtDQo+IGxpc3RzLmxpbmFyby5vcmc+Ow0KPiA+ID4gdmly
dGlvLWRldkBsaXN0cy5vYXNpcy1vcGVuLm9yZzsgQXJuZCBCZXJnbWFubg0KPiA8YXJuZC5iZXJn
bWFubkBsaW5hcm8ub3JnPjsNCj4gPiA+IFZpcmVzaCBLdW1hciA8dmlyZXNoLmt1bWFyQGxpbmFy
by5vcmc+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPiA+IDxzdGVmYW5vLnN0YWJlbGxpbmlAeGls
aW54LmNvbT47IHN0ZWZhbmhhQHJlZGhhdC5jb207IEphbiBLaXN6a2ENCj4gPiA+IDxqYW4ua2lz
emthQHNpZW1lbnMuY29tPjsgQ2FybCB2YW4gU2NoYWlrIDxjdmFuc2NoYUBxdGkucXVhbGNvbW0u
Y29tPjsNCj4gPiA+IHByYXRpa3BAcXVpY2luYy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaSA8dmF0
c2FAY29kZWF1cm9yYS5vcmc+OyBKZWFuLQ0KPiA+ID4gUGhpbGlwcGUgQnJ1Y2tlciA8amVhbi1w
aGlsaXBwZUBsaW5hcm8ub3JnPjsgTWF0aGlldSBQb2lyaWVyDQo+ID4gPiA8bWF0aGlldS5wb2ly
aWVyQGxpbmFyby5vcmc+OyBPbGVrc2FuZHIgVHlzaGNoZW5rbw0KPiA+ID4gPE9sZWtzYW5kcl9U
eXNoY2hlbmtvQGVwYW0uY29tPjsgQmVydHJhbmQgTWFycXVpcw0KPiA+ID4gPEJlcnRyYW5kLk1h
cnF1aXNAYXJtLmNvbT47IEFydGVtIE15Z2FpZXYgPEFydGVtX015Z2FpZXZAZXBhbS5jb20+Ow0K
PiBKdWxpZW4NCj4gPiA+IEdyYWxsIDxqdWxpZW5AeGVuLm9yZz47IEp1ZXJnZW4gR3Jvc3MgPGpn
cm9zc0BzdXNlLmNvbT47IFBhdWwgRHVycmFudA0KPiA+ID4gPHBhdWxAeGVuLm9yZz47IFhlbiBE
ZXZlbCA8eGVuLWRldmVsQGxpc3RzLnhlbi5vcmc+DQo+ID4gPiBTdWJqZWN0OiBSZTogRW5hYmxp
bmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lzbSBmb3IgVmlydElPIGJhY2tlbmRzDQo+ID4gPg0KPiA+
ID4gSGkgV2VpLA0KPiA+ID4NCj4gPiA+IE9uIEZyaSwgQXVnIDIwLCAyMDIxIGF0IDAzOjQxOjUw
UE0gKzA5MDAsIEFLQVNISSBUYWthaGlybyB3cm90ZToNCj4gPiA+ID4gT24gV2VkLCBBdWcgMTgs
IDIwMjEgYXQgMDg6MzU6NTFBTSArMDAwMCwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiA+ID4gSGkg
QWthc2hp77yMDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t
LS0tDQo+ID4gPiA+ID4gPiBGcm9tOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFrYXNoaUBs
aW5hcm8ub3JnPg0KPiA+ID4gPiA+ID4gU2VudDogMjAyMeW5tDjmnIgxOOaXpSAxMzozOQ0KPiA+
ID4gPiA+ID4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gQ2M6
IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hAZ21haWwuY29tPjsgU3RlZmFubw0KPiBT
dGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IEFsZXggQmVu
bj8/ZSA8YWxleC5iZW5uZWVAbGluYXJvLm9yZz47DQo+ID4gPiBTdHJhdG9zDQo+ID4gPiA+ID4g
PiBNYWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLWxpc3RzLmxpbmFyby5vcmc+OyB2aXJ0aW8t
DQo+ID4gPiBkZXZAbGlzdHMub2FzaXMtDQo+ID4gPiA+ID4gPiBvcGVuLm9yZzsgQXJuZCBCZXJn
bWFubiA8YXJuZC5iZXJnbWFubkBsaW5hcm8ub3JnPjsgVmlyZXNoIEt1bWFyDQo+ID4gPiA+ID4g
PiA8dmlyZXNoLmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPiA+ID4g
PiA+IDxzdGVmYW5vLnN0YWJlbGxpbmlAeGlsaW54LmNvbT47IHN0ZWZhbmhhQHJlZGhhdC5jb207
IEphbiBLaXN6a2ENCj4gPiA+ID4gPiA+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsgQ2FybCB2
YW4gU2NoYWlrDQo+ID4gPiA8Y3ZhbnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+ID4g
PiBwcmF0aWtwQHF1aWNpbmMuY29tOyBTcml2YXRzYSBWYWRkYWdpcmkgPHZhdHNhQGNvZGVhdXJv
cmEub3JnPjsNCj4gPiA+IEplYW4tDQo+ID4gPiA+ID4gPiBQaGlsaXBwZSBCcnVja2VyIDxqZWFu
LXBoaWxpcHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1IFBvaXJpZXINCj4gPiA+ID4gPiA+IDxtYXRo
aWV1LnBvaXJpZXJAbGluYXJvLm9yZz47IE9sZWtzYW5kciBUeXNoY2hlbmtvDQo+ID4gPiA+ID4g
PiA8T2xla3NhbmRyX1R5c2hjaGVua29AZXBhbS5jb20+OyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4g
PiA+ID4gPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgQXJ0ZW0gTXlnYWlldg0KPiA8QXJ0
ZW1fTXlnYWlldkBlcGFtLmNvbT47DQo+ID4gPiBKdWxpZW4NCj4gPiA+ID4gPiA+IEdyYWxsIDxq
dWxpZW5AeGVuLm9yZz47IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT47IFBhdWwNCj4g
PiA+IER1cnJhbnQNCj4gPiA+ID4gPiA+IDxwYXVsQHhlbi5vcmc+OyBYZW4gRGV2ZWwgPHhlbi1k
ZXZlbEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4gPiA+ID4gU3ViamVjdDogUmU6IEVuYWJsaW5nIGh5
cGVydmlzb3IgYWdub3N0aWNpc20gZm9yIFZpcnRJTyBiYWNrZW5kcw0KPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+IE9uIFR1ZSwgQXVnIDE3LCAyMDIxIGF0IDA4OjM5OjA5QU0gKzAwMDAsIFdlaSBD
aGVuIHdyb3RlOg0KPiA+ID4gPiA+ID4gPiBIaSBBa2FzaGksDQo+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+ID4gPiA+ID4gPiBG
cm9tOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFrYXNoaUBsaW5hcm8ub3JnPg0KPiA+ID4g
PiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ45pyIMTfml6UgMTY6MDgNCj4gPiA+ID4gPiA+ID4gPiBU
bzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiA+ID4gPiA+ID4gQ2M6IE9sZWtz
YW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hAZ21haWwuY29tPjsgU3RlZmFubw0KPiA+ID4gU3Rh
YmVsbGluaQ0KPiA+ID4gPiA+ID4gPiA+IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBC
ZW5uPz9lDQo+IDxhbGV4LmJlbm5lZUBsaW5hcm8ub3JnPjsNCj4gPiA+ID4gPiA+IFN0cmF0b3MN
Cj4gPiA+ID4gPiA+ID4gPiBNYWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLWxpc3RzLmxpbmFy
by5vcmc+OyB2aXJ0aW8tDQo+ID4gPiA+ID4gPiBkZXZAbGlzdHMub2FzaXMtDQo+ID4gPiA+ID4g
PiA+ID4gb3Blbi5vcmc7IEFybmQgQmVyZ21hbm4gPGFybmQuYmVyZ21hbm5AbGluYXJvLm9yZz47
IFZpcmVzaA0KPiBLdW1hcg0KPiA+ID4gPiA+ID4gPiA+IDx2aXJlc2gua3VtYXJAbGluYXJvLm9y
Zz47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+ID4gPiA+ID4gPiA+IDxzdGVmYW5vLnN0YWJlbGxp
bmlAeGlsaW54LmNvbT47IHN0ZWZhbmhhQHJlZGhhdC5jb207IEphbg0KPiBLaXN6a2ENCj4gPiA+
ID4gPiA+ID4gPiA8amFuLmtpc3prYUBzaWVtZW5zLmNvbT47IENhcmwgdmFuIFNjaGFpaw0KPiA+
ID4gPGN2YW5zY2hhQHF0aS5xdWFsY29tbS5jb20+Ow0KPiA+ID4gPiA+ID4gPiA+IHByYXRpa3BA
cXVpY2luYy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaQ0KPiA8dmF0c2FAY29kZWF1cm9yYS5vcmc+
Ow0KPiA+ID4gSmVhbi0NCj4gPiA+ID4gPiA+ID4gPiBQaGlsaXBwZSBCcnVja2VyIDxqZWFuLXBo
aWxpcHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1IFBvaXJpZXINCj4gPiA+ID4gPiA+ID4gPiA8bWF0
aGlldS5wb2lyaWVyQGxpbmFyby5vcmc+OyBPbGVrc2FuZHIgVHlzaGNoZW5rbw0KPiA+ID4gPiA+
ID4gPiA+IDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5kIE1hcnF1aXMN
Cj4gPiA+ID4gPiA+ID4gPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgQXJ0ZW0gTXlnYWll
dg0KPiA+ID4gPEFydGVtX015Z2FpZXZAZXBhbS5jb20+Ow0KPiA+ID4gPiA+ID4gSnVsaWVuDQo+
ID4gPiA+ID4gPiA+ID4gR3JhbGwgPGp1bGllbkB4ZW4ub3JnPjsgSnVlcmdlbiBHcm9zcyA8amdy
b3NzQHN1c2UuY29tPjsNCj4gUGF1bA0KPiA+ID4gRHVycmFudA0KPiA+ID4gPiA+ID4gPiA+IDxw
YXVsQHhlbi5vcmc+OyBYZW4gRGV2ZWwgPHhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4g
PiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFibGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIGZv
ciBWaXJ0SU8NCj4gYmFja2VuZHMNCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+IEhp
IFdlaSwgT2xla3NhbmRyLA0KPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gT24gTW9u
LCBBdWcgMTYsIDIwMjEgYXQgMTA6MDQ6MDNBTSArMDAwMCwgV2VpIENoZW4gd3JvdGU6DQo+ID4g
PiA+ID4gPiA+ID4gPiBIaSBBbGwsDQo+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gVGhhbmtzIGZvciBTdGVmYW5vIHRvIGxpbmsgbXkga3ZtdG9vbCBmb3IgWGVuIHByb3Bvc2Fs
DQo+IGhlcmUuDQo+ID4gPiA+ID4gPiA+ID4gPiBUaGlzIHByb3Bvc2FsIGlzIHN0aWxsIGRpc2N1
c3NpbmcgaW4gWGVuIGFuZCBLVk0NCj4gY29tbXVuaXRpZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiBU
aGUgbWFpbiB3b3JrIGlzIHRvIGRlY291cGxlIHRoZSBrdm10b29sIGZyb20gS1ZNIGFuZCBtYWtl
DQo+ID4gPiA+ID4gPiA+ID4gPiBvdGhlciBoeXBlcnZpc29ycyBjYW4gcmV1c2UgdGhlIHZpcnR1
YWwgZGV2aWNlDQo+IGltcGxlbWVudGF0aW9ucy4NCj4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiBJbiB0aGlzIGNhc2UsIHdlIG5lZWQgdG8gaW50cm9kdWNlIGFuIGludGVybWVk
aWF0ZQ0KPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gPiBsYXllciBmb3IgVk1NIGFic3Ry
YWN0aW9uLCBXaGljaCBpcywgSSB0aGluayBpdCdzIHZlcnkNCj4gY2xvc2UNCj4gPiA+ID4gPiA+
ID4gPiA+IHRvIHN0cmF0b3MnIHZpcnRpbyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIHdvcmsuDQo+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiAjIE15IHByb3Bvc2FsWzFdIGNvbWVzIGZy
b20gbXkgb3duIGlkZWEgYW5kIGRvZXNuJ3QgYWx3YXlzDQo+ID4gPiByZXByZXNlbnQNCj4gPiA+
ID4gPiA+ID4gPiAjIExpbmFybydzIHZpZXcgb24gdGhpcyBzdWJqZWN0IG5vciByZWZsZWN0IEFs
ZXgncyBjb25jZXJucy4NCj4gPiA+ID4gPiA+IE5ldmVydGhlbGVzcywNCj4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+IFlvdXIgaWRlYSBhbmQgbXkgcHJvcG9zYWwgc2VlbSB0byBzaGFy
ZSB0aGUgc2FtZSBiYWNrZ3JvdW5kLg0KPiA+ID4gPiA+ID4gPiA+IEJvdGggaGF2ZSB0aGUgc2lt
aWxhciBnb2FsIGFuZCBjdXJyZW50bHkgc3RhcnQgd2l0aCwgYXQNCj4gZmlyc3QsDQo+ID4gPiBY
ZW4NCj4gPiA+ID4gPiA+ID4gPiBhbmQgYXJlIGJhc2VkIG9uIGt2bS10b29sLiAoQWN0dWFsbHks
IG15IHdvcmsgaXMgZGVyaXZlZA0KPiBmcm9tDQo+ID4gPiA+ID4gPiA+ID4gRVBBTSdzIHZpcnRp
by1kaXNrLCB3aGljaCBpcyBhbHNvIGJhc2VkIG9uIGt2bS10b29sLikNCj4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+IEluIHBhcnRpY3VsYXIsIHRoZSBhYnN0cmFjdGlvbiBvZiBoeXBl
cnZpc29yIGludGVyZmFjZXMgaGFzDQo+IGENCj4gPiA+IHNhbWUNCj4gPiA+ID4gPiA+ID4gPiBz
ZXQgb2YgaW50ZXJmYWNlcyAoZm9yIHlvdXIgInN0cnVjdCB2bW1faW1wbCIgYW5kIG15ICJSUEMN
Cj4gPiA+IGludGVyZmFjZXMiKS4NCj4gPiA+ID4gPiA+ID4gPiBUaGlzIGlzIG5vdCBjby1pbmNp
ZGVudCBhcyB3ZSBib3RoIHNoYXJlIHRoZSBzYW1lIG9yaWdpbiBhcw0KPiBJDQo+ID4gPiBzYWlk
DQo+ID4gPiA+ID4gPiBhYm92ZS4NCj4gPiA+ID4gPiA+ID4gPiBBbmQgc28gd2Ugd2lsbCBhbHNv
IHNoYXJlIHRoZSBzYW1lIGlzc3Vlcy4gT25lIG9mIHRoZW0gaXMgYQ0KPiB3YXkNCj4gPiA+IG9m
DQo+ID4gPiA+ID4gPiA+ID4gInNoYXJpbmcvbWFwcGluZyBGRSdzIG1lbW9yeSIuIFRoZXJlIGlz
IHNvbWUgdHJhZGUtb2ZmDQo+IGJldHdlZW4NCj4gPiA+ID4gPiA+ID4gPiB0aGUgcG9ydGFiaWxp
dHkgYW5kIHRoZSBwZXJmb3JtYW5jZSBpbXBhY3QuDQo+ID4gPiA+ID4gPiA+ID4gU28gd2UgY2Fu
IGRpc2N1c3MgdGhlIHRvcGljIGhlcmUgaW4gdGhpcyBNTCwgdG9vLg0KPiA+ID4gPiA+ID4gPiA+
IChTZWUgQWxleCdzIG9yaWdpbmFsIGVtYWlsLCB0b28pLg0KPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+IFllcywgSSBhZ3JlZS4NCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiBP
biB0aGUgb3RoZXIgaGFuZCwgbXkgYXBwcm9hY2ggYWltcyB0byBjcmVhdGUgYSAic2luZ2xlLQ0K
PiBiaW5hcnkiDQo+ID4gPiA+ID4gPiBzb2x1dGlvbg0KPiA+ID4gPiA+ID4gPiA+IGluIHdoaWNo
IHRoZSBzYW1lIGJpbmFyeSBvZiBCRSB2bSBjb3VsZCBydW4gb24gYW55DQo+IGh5cGVydmlzb3Jz
Lg0KPiA+ID4gPiA+ID4gPiA+IFNvbWVob3cgc2ltaWxhciB0byB5b3VyICJwcm9wb3NhbC0jMiIg
aW4gWzJdLCBidXQgaW4gbXkNCj4gc29sdXRpb24sDQo+ID4gPiBhbGwNCj4gPiA+ID4gPiA+ID4g
PiB0aGUgaHlwZXJ2aXNvci1zcGVjaWZpYyBjb2RlIHdvdWxkIGJlIHB1dCBpbnRvIGFub3RoZXIN
Cj4gZW50aXR5DQo+ID4gPiAoVk0pLA0KPiA+ID4gPiA+ID4gPiA+IG5hbWVkICJ2aXJ0aW8tcHJv
eHkiIGFuZCB0aGUgYWJzdHJhY3RlZCBvcGVyYXRpb25zIGFyZQ0KPiBzZXJ2ZWQNCj4gPiA+IHZp
YSBSUEMuDQo+ID4gPiA+ID4gPiA+ID4gKEluIHRoaXMgc2Vuc2UsIEJFIGlzIGh5cGVydmlzb3It
YWdub3N0aWMgYnV0IG1pZ2h0IGhhdmUgT1MNCj4gPiA+ID4gPiA+IGRlcGVuZGVuY3kuKQ0KPiA+
ID4gPiA+ID4gPiA+IEJ1dCBJIGtub3cgdGhhdCB3ZSBuZWVkIGRpc2N1c3MgaWYgdGhpcyBpcyBh
IHJlcXVpcmVtZW50DQo+IGV2ZW4NCj4gPiA+ID4gPiA+ID4gPiBpbiBTdHJhdG9zIHByb2plY3Qg
b3Igbm90LiAoTWF5YmUgbm90KQ0KPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+IFNvcnJ5LCBJIGhhdmVuJ3QgaGFkIHRpbWUgdG8gZmluaXNoIHJlYWRpbmcgeW91
ciB2aXJ0aW8tcHJveHkNCj4gPiA+IGNvbXBsZXRlbHkNCj4gPiA+ID4gPiA+ID4gKEkgd2lsbCBk
byBpdCBBU0FQKS4gQnV0IGZyb20geW91ciBkZXNjcmlwdGlvbiwgaXQgc2VlbXMgd2UNCj4gbmVl
ZCBhDQo+ID4gPiA+ID4gPiA+IDNyZCBWTSBiZXR3ZWVuIEZFIGFuZCBCRT8gTXkgY29uY2VybiBp
cyB0aGF0LCBpZiBteSBhc3N1bXB0aW9uDQo+IGlzDQo+ID4gPiByaWdodCwNCj4gPiA+ID4gPiA+
ID4gd2lsbCBpdCBpbmNyZWFzZSB0aGUgbGF0ZW5jeSBpbiBkYXRhIHRyYW5zcG9ydCBwYXRoPyBF
dmVuIGlmDQo+IHdlJ3JlDQo+ID4gPiA+ID4gPiA+IHVzaW5nIHNvbWUgbGlnaHR3ZWlnaHQgZ3Vl
c3QgbGlrZSBSVE9TIG9yIFVuaWtlcm5lbCwNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBZZXMs
IHlvdSdyZSByaWdodC4gQnV0IEknbSBhZnJhaWQgdGhhdCBpdCBpcyBhIG1hdHRlciBvZiBkZWdy
ZWUuDQo+ID4gPiA+ID4gPiBBcyBmYXIgYXMgd2UgZXhlY3V0ZSAnbWFwcGluZycgb3BlcmF0aW9u
cyBhdCBldmVyeSBmZXRjaCBvZg0KPiBwYXlsb2FkLA0KPiA+ID4gPiA+ID4gd2Ugd2lsbCBzZWUg
bGF0ZW5jeSBpc3N1ZSAoZXZlbiBpbiB5b3VyIGNhc2UpIGFuZCBpZiB3ZSBoYXZlDQo+IHNvbWUN
Cj4gPiA+IHNvbHV0aW9uDQo+ID4gPiA+ID4gPiBmb3IgaXQsIHdlIHdvbid0IHNlZSBpdCBuZWl0
aGVyIGluIG15IHByb3Bvc2FsIDopDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+DQo+ID4gPiA+ID4g
T2xla3NhbmRyIGhhcyBzZW50IGEgcHJvcG9zYWwgdG8gWGVuIG1haWxpbmcgbGlzdCB0byByZWR1
Y2UgdGhpcw0KPiBraW5kDQo+ID4gPiA+ID4gb2YgIm1hcHBpbmcvdW5tYXBwaW5nIiBvcGVyYXRp
b25zLiBTbyB0aGUgbGF0ZW5jeSBjYXVzZWQgYnkgdGhpcw0KPiA+ID4gYmVoYXZpb3INCj4gPiA+
ID4gPiBvbiBYZW4gbWF5IGV2ZW50dWFsbHkgYmUgZWxpbWluYXRlZCwgYW5kIExpbnV4LUtWTSBk
b2Vzbid0IGhhdmUNCj4gdGhhdA0KPiA+ID4gcHJvYmxlbS4NCj4gPiA+ID4NCj4gPiA+ID4gT2J2
aW91c2x5LCBJIGhhdmUgbm90IHlldCBjYXVnaHQgdXAgdGhlcmUgaW4gdGhlIGRpc2N1c3Npb24u
DQo+ID4gPiA+IFdoaWNoIHBhdGNoIHNwZWNpZmljYWxseT8NCj4gPiA+DQo+ID4gPiBDYW4geW91
IGdpdmUgbWUgdGhlIGxpbmsgdG8gdGhlIGRpc2N1c3Npb24gb3IgcGF0Y2gsIHBsZWFzZT8NCj4g
PiA+DQo+ID4NCj4gPiBJdCdzIGEgUkZDIGRpc2N1c3Npb24uIFdlIGhhdmUgdGVzdGVkIHRoaXMg
UkZDIHBhdGNoIGludGVybmFsbHkuDQo+ID4gaHR0cHM6Ly9saXN0cy54ZW5wcm9qZWN0Lm9yZy9h
cmNoaXZlcy9odG1sL3hlbi1kZXZlbC8yMDIxLQ0KPiAwNy9tc2cwMTUzMi5odG1sDQo+IA0KPiBJ
J20gYWZyYWlkIHRoYXQgSSBtaXNzIHNvbWV0aGluZyBoZXJlLCBidXQgSSBkb24ndCBrbm93DQo+
IHdoeSB0aGlzIHByb3Bvc2VkIEFQSSB3aWxsIGxlYWQgdG8gZWxpbWluYXRpbmcgJ21tYXAnIGlu
IGFjY2Vzc2luZw0KPiB0aGUgcXVldWVkIHBheWxvYWQgYXQgZXZlcnkgcmVxdWVzdD8NCj4gDQoN
ClRoaXMgQVBJIGdpdmUgWGVuIGRldmljZSBtb2RlbCAoUUVNVSBvciBrdm10b29sKSB0aGUgYWJp
bGl0eSB0byBtYXANCndob2xlIGd1ZXN0IFJBTSBpbiBkZXZpY2UgbW9kZWwncyBhZGRyZXNzIHNw
YWNlLiBJbiB0aGlzIGNhc2UsIGRldmljZQ0KbW9kZWwgZG9lc24ndCBuZWVkIGR5bmFtaWMgaHlw
ZXJjYWxsIHRvIG1hcC91bm1hcCBwYXlsb2FkIG1lbW9yeS4NCkl0IGNhbiB1c2UgYSBmbGF0IG9m
ZnNldCB0byBhY2Nlc3MgcGF5bG9hZCBtZW1vcnkgaW4gaXRzIGFkZHJlc3MNCnNwYWNlIGRpcmVj
dGx5LiBKdXN0IExpa2UgS1ZNIGRldmljZSBtb2RlbCBkb2VzIG5vdy4NCg0KQmVmb3JlIHRoaXMg
QVBJLCBXaGVuIGRldmljZSBtb2RlbCB0byBtYXAgd2hvbGUgZ3Vlc3QgbWVtb3J5LCB3aWxsDQpz
ZXZlcmVseSBjb25zdW1lIHRoZSBwaHlzaWNhbCBwYWdlcyBvZiBEb20tMC9Eb20tRC4NCg0KPiAt
VGFrYWhpcm8gQWthc2hpDQo+IA0KPiANCj4gPiA+IFRoYW5rcywNCj4gPiA+IC1UYWthaGlybyBB
a2FzaGkNCj4gPiA+DQo+ID4gPiA+IC1UYWthaGlybyBBa2FzaGkNCj4gPiA+ID4NCj4gPiA+ID4g
PiA+ID4gPiBTcGVjaWZpY2FsbHkgc3BlYWtpbmcgYWJvdXQga3ZtLXRvb2wsIEkgaGF2ZSBhIGNv
bmNlcm4gYWJvdXQNCj4gaXRzDQo+ID4gPiA+ID4gPiA+ID4gbGljZW5zZSB0ZXJtOyBUYXJnZXRp
bmcgZGlmZmVyZW50IGh5cGVydmlzb3JzIGFuZCBkaWZmZXJlbnQNCj4gT1NzDQo+ID4gPiA+ID4g
PiA+ID4gKHdoaWNoIEkgYXNzdW1lIGluY2x1ZGVzIFJUT1MncyksIHRoZSByZXN1bHRhbnQgbGli
cmFyeQ0KPiBzaG91bGQNCj4gPiA+IGJlDQo+ID4gPiA+ID4gPiA+ID4gbGljZW5zZSBwZXJtaXNz
aXZlIGFuZCBHUEwgZm9yIGt2bS10b29sIG1pZ2h0IGJlIGFuIGlzc3VlLg0KPiA+ID4gPiA+ID4g
PiA+IEFueSB0aG91Z2h0cz8NCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiBZZXMuIElmIHVzZXIgd2FudCB0byBpbXBsZW1lbnQgYSBGcmVlQlNEIGRldmljZSBt
b2RlbCwgYnV0IHRoZQ0KPiA+ID4gdmlydGlvDQo+ID4gPiA+ID4gPiA+IGxpYnJhcnkgaXMgR1BM
LiBUaGVuIEdQTCB3b3VsZCBiZSBhIHByb2JsZW0uIElmIHdlIGhhdmUNCj4gYW5vdGhlcg0KPiA+
ID4gZ29vZA0KPiA+ID4gPiA+ID4gPiBjYW5kaWRhdGUsIEkgYW0gb3BlbiB0byBpdC4NCj4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiBJIGhhdmUgc29tZSBjYW5kaWRhdGVzLCBwYXJ0aWN1bGFybHkg
Zm9yIHZxL3ZyaW5nLCBpbiBteSBtaW5kOg0KPiA+ID4gPiA+ID4gKiBPcGVuLUFNUCwgb3INCj4g
PiA+ID4gPiA+ICogY29ycmVzcG9uZGluZyBGcmVlLUJTRCBjb2RlDQo+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+DQo+ID4gPiA+ID4gSW50ZXJlc3RpbmcsIEkgd2lsbCBsb29rIGludG8gdGhlbSA6ICkN
Cj4gPiA+ID4gPg0KPiA+ID4gPiA+IENoZWVycywNCj4gPiA+ID4gPiBXZWkgQ2hlbg0KPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+IFsxXSBodHRwczovL29wLWxpc3RzLmxpbmFy
by5vcmcvcGlwZXJtYWlsL3N0cmF0b3MtZGV2LzIwMjEtDQo+ID4gPiA+ID4gPiA+ID4gQXVndXN0
LzAwMDU0OC5odG1sDQo+ID4gPiA+ID4gPiA+ID4gWzJdIGh0dHBzOi8vbWFyYy5pbmZvLz9sPXhl
bi1kZXZlbCZtPTE2MjM3Mzc1NDcwNTIzMyZ3PTINCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gRnJvbTogT2xla3NhbmRyIFR5c2hjaGVua28g
PG9sZWtzdHlzaEBnbWFpbC5jb20+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ4
5pyIMTTml6UgMjM6MzgNCj4gPiA+ID4gPiA+ID4gPiA+ID4gVG86IEFLQVNISSBUYWthaGlybyA8
dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+Ow0KPiBTdGVmYW5vDQo+ID4gPiA+ID4gPiBTdGFi
ZWxsaW5pDQo+ID4gPiA+ID4gPiA+ID4gPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+IENjOiBBbGV4IEJlbm4/P2UgPGFsZXguYmVubmVlQGxpbmFyby5vcmc+OyBT
dHJhdG9zDQo+IE1haWxpbmcNCj4gPiA+IExpc3QNCj4gPiA+ID4gPiA+ID4gPiA8c3RyYXRvcy1k
ZXZAb3AtbGlzdHMubGluYXJvLm9yZz47IHZpcnRpby1kZXZAbGlzdHMub2FzaXMtDQo+ID4gPiBv
cGVuLm9yZzsNCj4gPiA+ID4gPiA+IEFybmQNCj4gPiA+ID4gPiA+ID4gPiBCZXJnbWFubiA8YXJu
ZC5iZXJnbWFubkBsaW5hcm8ub3JnPjsgVmlyZXNoIEt1bWFyDQo+ID4gPiA+ID4gPiA+ID4gPHZp
cmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBTdGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA+
ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFAcmVkaGF0LmNvbTsg
SmFuDQo+IEtpc3prYQ0KPiA+ID4gPiA+ID4gPiA+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsg
Q2FybCB2YW4gU2NoYWlrDQo+ID4gPiA8Y3ZhbnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4g
PiA+ID4gPiA+ID4gcHJhdGlrcEBxdWljaW5jLmNvbTsgU3JpdmF0c2EgVmFkZGFnaXJpDQo+IDx2
YXRzYUBjb2RlYXVyb3JhLm9yZz47DQo+ID4gPiBKZWFuLQ0KPiA+ID4gPiA+ID4gPiA+IFBoaWxp
cHBlIEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUgUG9pcmllcg0K
PiA+ID4gPiA+ID4gPiA+IDxtYXRoaWV1LnBvaXJpZXJAbGluYXJvLm9yZz47IFdlaSBDaGVuIDxX
ZWkuQ2hlbkBhcm0uY29tPjsNCj4gPiA+IE9sZWtzYW5kcg0KPiA+ID4gPiA+ID4gPiA+IFR5c2hj
aGVua28gPE9sZWtzYW5kcl9UeXNoY2hlbmtvQGVwYW0uY29tPjsgQmVydHJhbmQgTWFycXVpcw0K
PiA+ID4gPiA+ID4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2
DQo+ID4gPiA8QXJ0ZW1fTXlnYWlldkBlcGFtLmNvbT47DQo+ID4gPiA+ID4gPiBKdWxpZW4NCj4g
PiA+ID4gPiA+ID4gPiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+OyBKdWVyZ2VuIEdyb3NzIDxqZ3Jv
c3NAc3VzZS5jb20+Ow0KPiBQYXVsDQo+ID4gPiBEdXJyYW50DQo+ID4gPiA+ID4gPiA+ID4gPHBh
dWxAeGVuLm9yZz47IFhlbiBEZXZlbCA8eGVuLWRldmVsQGxpc3RzLnhlbi5vcmc+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFibGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNt
IGZvciBWaXJ0SU8NCj4gPiA+IGJhY2tlbmRzDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+IEhlbGxvLCBhbGwuDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+IFBsZWFzZSBzZWUgc29tZSBjb21tZW50cyBiZWxvdy4gQW5kIHNvcnJ5IGZvciB0
aGUNCj4gcG9zc2libGUNCj4gPiA+IGZvcm1hdA0KPiA+ID4gPiA+ID4gPiA+IGlzc3Vlcy4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbiBXZWQsIEF1ZyAxMSwg
MjAyMSBhdCA5OjI3IEFNIEFLQVNISSBUYWthaGlybw0KPiA+ID4gPiA+ID4gPiA+IDxtYWlsdG86
dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+IHdyb3RlOg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IE9uIFdlZCwgQXVnIDA0LCAyMDIxIGF0IDEyOjIwOjAxUE0gLTA3MDAsIFN0ZWZhbm8NCj4gPiA+
IFN0YWJlbGxpbmkNCj4gPiA+ID4gPiA+IHdyb3RlOg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Q0NpbmcgcGVvcGxlIHdvcmtpbmcgb24gWGVuK1ZpcnRJTyBhbmQgSU9SRVFzLiBOb3QNCj4gPiA+
IHRyaW1taW5nDQo+ID4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiBvcmlnaW5hbA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gZW1haWwgdG8gbGV0IHRoZW0gcmVhZCB0aGUgZnVsbCBjb250
ZXh0Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE15
IGNvbW1lbnRzIGJlbG93IGFyZSByZWxhdGVkIHRvIGEgcG90ZW50aWFsIFhlbg0KPiA+ID4gPiA+
ID4gaW1wbGVtZW50YXRpb24sDQo+ID4gPiA+ID4gPiA+ID4gbm90DQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBiZWNhdXNlIGl0IGlzIHRoZSBvbmx5IGltcGxlbWVudGF0aW9uIHRoYXQgbWF0dGVy
cywNCj4gYnV0DQo+ID4gPiA+ID4gPiBiZWNhdXNlIGl0DQo+ID4gPiA+ID4gPiA+ID4gaXMNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBvbmUgSSBrbm93IGJlc3QuDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBQbGVhc2Ugbm90ZSB0aGF0IG15IHByb3Bv
c2FsIChhbmQgaGVuY2UgdGhlIHdvcmtpbmcNCj4gPiA+IHByb3RvdHlwZSlbMV0NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBpcyBiYXNlZCBvbiBYZW4ncyB2aXJ0aW8gaW1wbGVtZW50YXRpb24gKGku
ZS4gSU9SRVEpDQo+IGFuZA0KPiA+ID4gPiA+ID4gPiA+IHBhcnRpY3VsYXJseQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IEVQQU0ncyB2aXJ0aW8tZGlzayBhcHBsaWNhdGlvbiAoYmFja2VuZCBzZXJ2
ZXIpLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEl0IGhhcyBiZWVuLCBJIGJlbGlldmUsIHdlbGwg
Z2VuZXJhbGl6ZWQgYnV0IGlzIHN0aWxsDQo+IGENCj4gPiA+IGJpdA0KPiA+ID4gPiA+ID4gYmlh
c2VkDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG93YXJkIHRoaXMgb3JpZ2luYWwgZGVzaWduLg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gU28gSSBob3BlIHlv
dSBsaWtlIG15IGFwcHJvYWNoIDopDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBbMV0gaHR0cHM6Ly9vcC1saXN0cy5saW5hcm8ub3JnL3BpcGVybWFpbC9zdHJh
dG9zLQ0KPiA+ID4gZGV2LzIwMjEtDQo+ID4gPiA+ID4gPiA+ID4gQXVndXN0LzAwMDU0Ni5odG1s
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBMZXQgbWUgdGFr
ZSB0aGlzIG9wcG9ydHVuaXR5IHRvIGV4cGxhaW4gYSBiaXQgbW9yZQ0KPiBhYm91dA0KPiA+ID4g
bXkNCj4gPiA+ID4gPiA+IGFwcHJvYWNoDQo+ID4gPiA+ID4gPiA+ID4gYmVsb3cuDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEFsc28sIHBsZWFzZSBzZWUg
dGhpcyByZWxldmFudCBlbWFpbCB0aHJlYWQ6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRw
czovL21hcmMuaW5mby8/bD14ZW4tZGV2ZWwmbT0xNjIzNzM3NTQ3MDUyMzMmdz0yDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IE9uIFdlZCwgNCBBdWcgMjAyMSwgQWxleCBCZW5uw6llIHdyb3RlOg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBIaSwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gT25lIG9mIHRoZSBnb2FscyBvZiBQcm9qZWN0IFN0cmF0b3Mg
aXMgdG8gZW5hYmxlDQo+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gYWdub3N0aWMN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYmFja2VuZHMgc28gd2UgY2FuIGVuYWJsZSBhcyBt
dWNoIHJlLXVzZSBvZiBjb2RlDQo+IGFzDQo+ID4gPiBwb3NzaWJsZQ0KPiA+ID4gPiA+ID4gYW5k
DQo+ID4gPiA+ID4gPiA+ID4gYXZvaWQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gcmVwZWF0
aW5nIG91cnNlbHZlcy4gVGhpcyBpcyB0aGUgZmxpcCBzaWRlIG9mIHRoZQ0KPiA+ID4gZnJvbnQg
ZW5kDQo+ID4gPiA+ID4gPiA+ID4gd2hlcmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbXVs
dGlwbGUgZnJvbnQtZW5kIGltcGxlbWVudGF0aW9ucyBhcmUgcmVxdWlyZWQgLQ0KPiBvbmUNCj4g
PiA+IHBlciBPUywNCj4gPiA+ID4gPiA+ID4gPiBhc3N1bWluZw0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiB5b3UgZG9uJ3QganVzdCB3YW50IExpbnV4IGd1ZXN0cy4gVGhlIHJlc3VsdGFudA0K
PiBndWVzdHMNCj4gPiA+IGFyZQ0KPiA+ID4gPiA+ID4gPiA+IHRyaXZpYWxseQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBtb3ZhYmxlIGJldHdlZW4gaHlwZXJ2aXNvcnMgbW9kdWxvIGFueSBh
YnN0cmFjdGVkDQo+ID4gPiBwYXJhdmlydA0KPiA+ID4gPiA+ID4gdHlwZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBpbnRlcmZhY2VzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJbiBteSBvcmlnaW5hbCB0aHVtYiBuYWlsIHNrZXRjaCBv
ZiBhIHNvbHV0aW9uIEkNCj4gPiA+IGVudmlzaW9uZWQNCj4gPiA+ID4gPiA+ID4gPiB2aG9zdC11
c2VyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGRhZW1vbnMgcnVubmluZyBpbiBhIGJyb2Fk
bHkgUE9TSVggbGlrZQ0KPiBlbnZpcm9ubWVudC4NCj4gPiA+IFRoZQ0KPiA+ID4gPiA+ID4gPiA+
IGludGVyZmFjZSB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgZGFlbW9uIGlzIGZh
aXJseSBzaW1wbGUgcmVxdWlyaW5nIG9ubHkgc29tZQ0KPiBtYXBwZWQNCj4gPiA+ID4gPiA+IG1l
bW9yeQ0KPiA+ID4gPiA+ID4gPiA+IGFuZCBzb21lDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IHNvcnQgb2Ygc2lnbmFsbGluZyBmb3IgZXZlbnRzIChvbiBMaW51eCB0aGlzIGlzDQo+ID4gPiBl
dmVudGZkKS4NCj4gPiA+ID4gPiA+IFRoZQ0KPiA+ID4gPiA+ID4gPiA+IGlkZWEgd2FzIGENCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc3R1YiBiaW5hcnkgd291bGQgYmUgcmVzcG9uc2libGUg
Zm9yIGFueQ0KPiBoeXBlcnZpc29yDQo+ID4gPiBzcGVjaWZpYw0KPiA+ID4gPiA+ID4gPiA+IHNl
dHVwIGFuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGVuIGxhdW5jaCBhIGNvbW1vbiBi
aW5hcnkgdG8gZGVhbCB3aXRoIHRoZQ0KPiBhY3R1YWwNCj4gPiA+ID4gPiA+IHZpcnRxdWV1ZQ0K
PiA+ID4gPiA+ID4gPiA+IHJlcXVlc3RzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZW1z
ZWx2ZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IFNpbmNlIHRoYXQgb3JpZ2luYWwgc2tldGNoIHdlJ3ZlIHNlZW4gYW4gZXhwYW5zaW9uDQo+
IGluDQo+ID4gPiB0aGUNCj4gPiA+ID4gPiA+IHNvcnQNCj4gPiA+ID4gPiA+ID4gPiBvZiB3YXlz
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGJhY2tlbmRzIGNvdWxkIGJlIGNyZWF0ZWQuIFRo
ZXJlIGlzIGludGVyZXN0IGluDQo+ID4gPiA+ID4gPiBlbmNhcHN1bGF0aW5nDQo+ID4gPiA+ID4g
PiA+ID4gYmFja2VuZHMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW4gUlRPU2VzIG9yIHVu
aWtlcm5lbHMgZm9yIHNvbHV0aW9ucyBsaWtlIFNDTUkuDQo+IFRoZXJlDQo+ID4gPiA+ID4gPiBp
bnRlcmVzdA0KPiA+ID4gPiA+ID4gPiA+IGluIFJ1c3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gaGFzIHByb21wdGVkIGlkZWFzIG9mIHVzaW5nIHRoZSB0cmFpdCBpbnRlcmZhY2UgdG8NCj4g
PiA+IGFic3RyYWN0DQo+ID4gPiA+ID4gPiA+ID4gZGlmZmVyZW5jZXMNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gYXdheSBhcyB3ZWxsIGFzIHRoZSBpZGVhIG9mIGJhcmUtbWV0YWwgUnVzdA0K
PiBiYWNrZW5kcy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gV2UgaGF2ZSBhIGNhcmQgKFNUUi0xMikgY2FsbGVkICJIeXBlcmNhbGwNCj4gPiA+
IFN0YW5kYXJkaXNhdGlvbiINCj4gPiA+ID4gPiA+IHdoaWNoDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IGNhbGxzIGZvciBhIGRlc2NyaXB0aW9uIG9mIHRoZSBBUElzIG5lZWRlZCBmcm9tDQo+
IHRoZQ0KPiA+ID4gPiA+ID4gaHlwZXJ2aXNvcg0KPiA+ID4gPiA+ID4gPiA+IHNpZGUgdG8NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc3VwcG9ydCBWaXJ0SU8gZ3Vlc3RzIGFuZCB0aGVpciBi
YWNrZW5kcy4gSG93ZXZlcg0KPiB3ZQ0KPiA+ID4gYXJlDQo+ID4gPiA+ID4gPiBzb21lDQo+ID4g
PiA+ID4gPiA+ID4gd2F5IG9mZg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBmcm9tIHRoYXQg
YXQgdGhlIG1vbWVudCBhcyBJIHRoaW5rIHdlIG5lZWQgdG8gYXQNCj4gbGVhc3QNCj4gPiA+ID4g
PiA+ID4gPiBkZW1vbnN0cmF0ZSBvbmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gcG9ydGFi
bGUgYmFja2VuZCBiZWZvcmUgd2Ugc3RhcnQgY29kaWZ5aW5nDQo+ID4gPiByZXF1aXJlbWVudHMu
IFRvDQo+ID4gPiA+ID4gPiB0aGF0DQo+ID4gPiA+ID4gPiA+ID4gZW5kIEkNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gd2FudCB0byB0aGluayBhYm91dCB3aGF0IHdlIG5lZWQgZm9yIGEgYmFj
a2VuZCB0bw0KPiA+ID4gZnVuY3Rpb24uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IENvbmZpZ3VyYXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPT09PT09PT09PT09PQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBJbiB0aGUgdHlwZS0yIHNldHVwIHRoaXMgaXMgdHlwaWNhbGx5IGZh
aXJseQ0KPiBzaW1wbGUNCj4gPiA+IGJlY2F1c2UNCj4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+
ID4gPiA+IGhvc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc3lzdGVtIGNhbiBvcmNoZXN0
cmF0ZSB0aGUgdmFyaW91cyBtb2R1bGVzIHRoYXQNCj4gbWFrZQ0KPiA+ID4gdXAgdGhlDQo+ID4g
PiA+ID4gPiA+ID4gY29tcGxldGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc3lzdGVtLiBJ
biB0aGUgdHlwZS0xIGNhc2UgKG9yIGV2ZW4gdHlwZS0yIHdpdGgNCj4gPiA+IGRlbGVnYXRlZA0K
PiA+ID4gPiA+ID4gPiA+IHNlcnZpY2UgVk1zKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB3
ZSBuZWVkIHNvbWUgc29ydCBvZiBtZWNoYW5pc20gdG8gaW5mb3JtIHRoZQ0KPiBiYWNrZW5kDQo+
ID4gPiBWTQ0KPiA+ID4gPiA+ID4gYWJvdXQNCj4gPiA+ID4gPiA+ID4gPiBrZXkNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gZGV0YWlscyBhYm91dCB0aGUgc3lzdGVtOg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gd2hlcmUgdmlydCBx
dWV1ZSBtZW1vcnkgaXMgaW4gaXQncyBhZGRyZXNzDQo+IHNwYWNlDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICAgLSBob3cgaXQncyBnb2luZyB0byByZWNlaXZlIChpbnRlcnJ1cHQpIGFuZA0K
PiB0cmlnZ2VyDQo+ID4gPiAoa2ljaykNCj4gPiA+ID4gPiA+ID4gPiBldmVudHMNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gICAtIHdoYXQgKGlmIGFueSkgcmVzb3VyY2VzIHRoZSBiYWNrZW5k
IG5lZWRzIHRvDQo+ID4gPiBjb25uZWN0IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9idmlvdXNseSB5b3UgY2FuIGVsaWRlIG92ZXIgY29u
ZmlndXJhdGlvbiBpc3N1ZXMNCj4gYnkNCj4gPiA+IGhhdmluZw0KPiA+ID4gPiA+ID4gPiA+IHN0
YXRpYw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBjb25maWd1cmF0aW9ucyBhbmQgYmFraW5n
IHRoZSBhc3N1bXB0aW9ucyBpbnRvDQo+IHlvdXINCj4gPiA+IGd1ZXN0DQo+ID4gPiA+ID4gPiBp
bWFnZXMNCj4gPiA+ID4gPiA+ID4gPiBob3dldmVyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IHRoaXMgaXNuJ3Qgc2NhbGFibGUgaW4gdGhlIGxvbmcgdGVybS4gVGhlIG9idmlvdXMNCj4gPiA+
IHNvbHV0aW9uDQo+ID4gPiA+ID4gPiBzZWVtcw0KPiA+ID4gPiA+ID4gPiA+IHRvIGJlDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV4dGVuZGluZyBhIHN1YnNldCBvZiBEZXZpY2UgVHJlZSBk
YXRhIHRvIHVzZXINCj4gc3BhY2UNCj4gPiA+IGJ1dA0KPiA+ID4gPiA+ID4gcGVyaGFwcw0KPiA+
ID4gPiA+ID4gPiA+IHRoZXJlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFyZSBvdGhlciBh
cHByb2FjaGVzPw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBCZWZvcmUgYW55IHZpcnRpbyB0cmFuc2FjdGlvbnMgY2FuIHRha2UgcGxhY2UgdGhl
DQo+ID4gPiA+ID4gPiBhcHByb3ByaWF0ZQ0KPiA+ID4gPiA+ID4gPiA+IG1lbW9yeQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBtYXBwaW5ncyBuZWVkIHRvIGJlIG1hZGUgYmV0d2VlbiB0aGUg
RkUgZ3Vlc3QgYW5kDQo+IHRoZQ0KPiA+ID4gQkUNCj4gPiA+ID4gPiA+IGd1ZXN0Lg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQ3VycmVudGx5IHRo
ZSB3aG9sZSBvZiB0aGUgRkUgZ3Vlc3RzIGFkZHJlc3Mgc3BhY2UNCj4gPiA+IG5lZWRzIHRvDQo+
ID4gPiA+ID4gPiBiZQ0KPiA+ID4gPiA+ID4gPiA+IHZpc2libGUNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gdG8gd2hhdGV2ZXIgaXMgc2VydmluZyB0aGUgdmlydGlvIHJlcXVlc3RzLiBJIGNh
bg0KPiA+ID4gZW52aXNpb24gMw0KPiA+ID4gPiA+ID4gPiA+IGFwcHJvYWNoZXM6DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAqIEJFIGd1ZXN0
IGJvb3RzIHdpdGggbWVtb3J5IGFscmVhZHkgbWFwcGVkDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBUaGlzIHdvdWxkIGVudGFpbCB0aGUgZ3Vl
c3QgT1Mga25vd2luZyB3aGVyZSBpbg0KPiBpdCdzDQo+ID4gPiBHdWVzdA0KPiA+ID4gPiA+ID4g
PiA+IFBoeXNpY2FsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBBZGRyZXNzIHNwYWNlIGlz
IGFscmVhZHkgdGFrZW4gdXAgYW5kIGF2b2lkaW5nDQo+ID4gPiBjbGFzaGluZy4gSQ0KPiA+ID4g
PiA+ID4gPiA+IHdvdWxkIGFzc3VtZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgaW4gdGhp
cyBjYXNlIHlvdSB3b3VsZCB3YW50IGEgc3RhbmRhcmQgaW50ZXJmYWNlDQo+IHRvDQo+ID4gPiA+
ID4gPiB1c2Vyc3BhY2UNCj4gPiA+ID4gPiA+ID4gPiB0byB0aGVuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICBtYWtlIHRoYXQgYWRkcmVzcyBzcGFjZSB2aXNpYmxlIHRvIHRoZSBiYWNrZW5k
DQo+IGRhZW1vbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IFlldCBhbm90aGVyIHdheSBoZXJlIGlzIHRoYXQgd2Ugd291bGQgaGF2ZSB3ZWxsIGtub3duDQo+
ID4gPiAic2hhcmVkDQo+ID4gPiA+ID4gPiA+ID4gbWVtb3J5IiBiZXR3ZWVuDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gVk1zLiBJIHRoaW5rIHRoYXQgSmFpbGhvdXNlJ3MgaXZzaG1lbSBnaXZlcyB1
cyBnb29kDQo+ID4gPiBpbnNpZ2h0cyBvbg0KPiA+ID4gPiA+ID4gdGhpcw0KPiA+ID4gPiA+ID4g
PiA+IG1hdHRlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCB0aGF0IGl0IGNhbiBldmVuIGJl
IGFuIGFsdGVybmF0aXZlIGZvciBoeXBlcnZpc29yLQ0KPiA+ID4gYWdub3N0aWMNCj4gPiA+ID4g
PiA+ID4gPiBzb2x1dGlvbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IChQbGVhc2Ugbm90ZSBtZW1vcnkgcmVnaW9ucyBpbiBpdnNobWVtIGFwcGVhciBhcyBh
IFBDSQ0KPiA+ID4gZGV2aWNlDQo+ID4gPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiBjYW4g
YmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBtYXBwZWQgbG9jYWxseS4pDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJIHdhbnQgdG8gYWRkIHRoaXMgc2hhcmVk
IG1lbW9yeSBhc3BlY3QgdG8gbXkgdmlydGlvLQ0KPiBwcm94eSwNCj4gPiA+IGJ1dA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHRoZSByZXN1bHRhbnQgc29sdXRpb24gd291bGQgZXZlbnR1YWxseSBs
b29rIHNpbWlsYXINCj4gdG8NCj4gPiA+IGl2c2htZW0uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICogQkUgZ3Vlc3RzIGJvb3RzIHdpdGggYSBoeXBl
cnZpc29yIGhhbmRsZSB0bw0KPiBtZW1vcnkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIFRoZSBCRSBndWVzdCBpcyB0aGVuIGZyZWUgdG8gbWFw
IHRoZSBGRSdzIG1lbW9yeQ0KPiB0bw0KPiA+ID4gd2hlcmUNCj4gPiA+ID4gPiA+IGl0DQo+ID4g
PiA+ID4gPiA+ID4gd2FudHMgaW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIHRoZSBCRSdz
IGd1ZXN0IHBoeXNpY2FsIGFkZHJlc3Mgc3BhY2UuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSSBjYW5ub3Qgc2VlIGhvdyB0aGlzIGNvdWxkIHdvcmsg
Zm9yIFhlbi4gVGhlcmUgaXMNCj4gbm8NCj4gPiA+ICJoYW5kbGUiDQo+ID4gPiA+ID4gPiB0bw0K
PiA+ID4gPiA+ID4gPiA+IGdpdmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHRoZSBiYWNr
ZW5kIGlmIHRoZSBiYWNrZW5kIGlzIG5vdCBydW5uaW5nIGluIGRvbTAuDQo+IFNvDQo+ID4gPiBm
b3INCj4gPiA+ID4gPiA+IFhlbiBJDQo+ID4gPiA+ID4gPiA+ID4gdGhpbmsNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IHRoZSBtZW1vcnkgaGFzIHRvIGJlIGFscmVhZHkgbWFwcGVkDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJbiBYZW4ncyBJT1JFUSBzb2x1
dGlvbiAodmlydGlvLWJsayksIHRoZSBmb2xsb3dpbmcNCj4gPiA+IGluZm9ybWF0aW9uDQo+ID4g
PiA+ID4gPiBpcw0KPiA+ID4gPiA+ID4gPiA+IGV4cGVjdGVkDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gdG8gYmUgZXhwb3NlZCB0byBCRSB2aWEgWGVuc3RvcmU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gKEkga25vdyB0aGF0IHRoaXMgaXMgYSB0ZW50YXRpdmUgYXBwcm9hY2ggdGhvdWdoLikNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiAgICAtIHRoZSBzdGFydCBhZGRyZXNzIG9mIGNvbmZpZ3VyYXRp
b24gc3BhY2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAtIGludGVycnVwdCBudW1iZXINCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiAgICAtIGZpbGUgcGF0aCBmb3IgYmFja2luZyBzdG9yYWdlDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgLSByZWFkLW9ubHkgZmxhZw0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IEFuZCB0aGUgQkUgc2VydmVyIGhhdmUgdG8gY2FsbCBhIHBhcnRpY3VsYXIgaHlwZXJ2
aXNvcg0KPiA+ID4gaW50ZXJmYWNlDQo+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IG1hcCB0aGUgY29uZmlndXJhdGlvbiBzcGFjZS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gWWVzLCBYZW5zdG9yZSB3YXMgY2hvc2VuIGFzIGEgc2ltcGxlIHdh
eSB0byBwYXNzDQo+ID4gPiBjb25maWd1cmF0aW9uDQo+ID4gPiA+ID4gPiBpbmZvIHRvDQo+ID4g
PiA+ID4gPiA+ID4gdGhlIGJhY2tlbmQgcnVubmluZyBpbiBhIG5vbi10b29sc3RhY2sgZG9tYWlu
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBJIHJlbWVtYmVyLCB0aGVyZSB3YXMgYSB3aXNoIHRvIGF2
b2lkIHVzaW5nIFhlbnN0b3JlIGluDQo+ID4gPiBWaXJ0aW8NCj4gPiA+ID4gPiA+IGJhY2tlbmQN
Cj4gPiA+ID4gPiA+ID4gPiBpdHNlbGYgaWYgcG9zc2libGUsIHNvIGZvciBub24tdG9vbHN0YWNr
IGRvbWFpbiwgdGhpcyBjb3VsZA0KPiBkb25lDQo+ID4gPiB3aXRoDQo+ID4gPiA+ID4gPiA+ID4g
YWRqdXN0aW5nIGRldmQgKGRhZW1vbiB0aGF0IGxpc3RlbnMgZm9yIGRldmljZXMgYW5kIGxhdW5j
aGVzDQo+ID4gPiBiYWNrZW5kcykNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gcmVhZCBiYWNrZW5k
IGNvbmZpZ3VyYXRpb24gZnJvbSB0aGUgWGVuc3RvcmUgYW55d2F5DQo+IGFuZA0KPiA+ID4gcGFz
cyBpdA0KPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiB0aGUgYmFja2VuZCB2aWEgY29t
bWFuZCBsaW5lIGFyZ3VtZW50cy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiBZZXMsIGluIGN1cnJlbnQgUG9DIGNvZGUgd2UncmUgdXNp
bmcgeGVuc3RvcmUgdG8gcGFzcw0KPiBkZXZpY2UNCj4gPiA+ID4gPiA+ID4gPiBjb25maWd1cmF0
aW9uLg0KPiA+ID4gPiA+ID4gPiA+ID4gV2UgYWxzbyBkZXNpZ25lZCBhIHN0YXRpYyBkZXZpY2Ug
Y29uZmlndXJhdGlvbiBwYXJzZQ0KPiBtZXRob2QNCj4gPiA+IGZvcg0KPiA+ID4gPiA+ID4gRG9t
MGxlc3MNCj4gPiA+ID4gPiA+ID4gPiBvcg0KPiA+ID4gPiA+ID4gPiA+ID4gb3RoZXIgc2NlbmFy
aW9zIGRvbid0IGhhdmUgeGVudG9vbC4geWVzLCBpdCdzIGZyb20gZGV2aWNlDQo+ID4gPiBtb2Rl
bA0KPiA+ID4gPiA+ID4gY29tbWFuZA0KPiA+ID4gPiA+ID4gPiA+IGxpbmUNCj4gPiA+ID4gPiA+
ID4gPiA+IG9yIGEgY29uZmlnIGZpbGUuDQo+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiBCdXQsIGlmIC4uLg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSW4gbXkgYXBwcm9hY2ggKHZpcnRpby1wcm94
eSksIGFsbCB0aG9zZSBYZW4gKG9yDQo+ID4gPiBoeXBlcnZpc29yKS0NCj4gPiA+ID4gPiA+ID4g
PiBzcGVjaWZpYw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHN0dWZmcyBhcmUgY29udGFpbmVkIGlu
IHZpcnRpby1wcm94eSwgeWV0IGFub3RoZXIgVk0sDQo+IHRvDQo+ID4gPiBoaWRlDQo+ID4gPiA+
ID4gPiBhbGwNCj4gPiA+ID4gPiA+ID4gPiBkZXRhaWxzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiAuLi4gdGhlIHNvbHV0aW9uIGhvdyB0byBvdmVyY29tZSB0aGF0
IGlzIGFscmVhZHkgZm91bmQNCj4gYW5kDQo+ID4gPiBwcm92ZW4NCj4gPiA+ID4gPiA+IHRvDQo+
ID4gPiA+ID4gPiA+ID4gd29yayB0aGVuIGV2ZW4gYmV0dGVyLg0KPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICMgTXkgcG9pbnQgaXMgdGhhdCBhICJoYW5kbGUiIGlzIG5vdCBtYW5kYXRvcnkg
Zm9yDQo+ID4gPiBleGVjdXRpbmcNCj4gPiA+ID4gPiA+IG1hcHBpbmcuDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCB0aGUgbWFwcGluZyBwcm9iYWJs
eSBkb25lIGJ5IHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG9vbHN0YWNrIChhbHNvIHNl
ZSBiZWxvdy4pIE9yIHdlIHdvdWxkIGhhdmUgdG8NCj4gaW52ZW50IGENCj4gPiA+IG5ldw0KPiA+
ID4gPiA+ID4gWGVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBoeXBlcnZpc29yIGludGVyZmFj
ZSBhbmQgWGVuIHZpcnR1YWwgbWFjaGluZQ0KPiBwcml2aWxlZ2VzDQo+ID4gPiB0bw0KPiA+ID4g
PiA+ID4gYWxsb3cNCj4gPiA+ID4gPiA+ID4gPiB0aGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBraW5kIG9mIG1hcHBpbmcuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IElmIHdlIHJ1biB0aGUgYmFja2VuZCBpbiBEb20wIHRoYXQgd2UgaGF2ZSBubw0K
PiBwcm9ibGVtcw0KPiA+ID4gb2YNCj4gPiA+ID4gPiA+IGNvdXJzZS4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uZSBvZiBkaWZmaWN1bHRpZXMgb24gWGVu
IHRoYXQgSSBmb3VuZCBpbiBteSBhcHByb2FjaA0KPiBpcw0KPiA+ID4gdGhhdA0KPiA+ID4gPiA+
ID4gPiA+IGNhbGxpbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzdWNoIGh5cGVydmlzb3IgaW50
ZWZhY2VzIChyZWdpc3RlcmluZyBJT1JFUSwgbWFwcGluZw0KPiA+ID4gbWVtb3J5KSBpcw0KPiA+
ID4gPiA+ID4gPiA+IG9ubHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhbGxvd2VkIG9uIEJFIHNl
cnZlcnMgdGhlbXNlbHZpZXMgYW5kIHNvIHdlIHdpbGwgaGF2ZQ0KPiB0bw0KPiA+ID4gZXh0ZW5k
DQo+ID4gPiA+ID4gPiA+ID4gdGhvc2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2Vz
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoaXMsIGhvd2V2ZXIsIHdpbGwgcmFpc2Ugc29tZSBj
b25jZXJuIG9uIHNlY3VyaXR5IGFuZA0KPiA+ID4gcHJpdmlsZWdlDQo+ID4gPiA+ID4gPiA+ID4g
ZGlzdHJpYnV0aW9uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYXMgU3RlZmFuIHN1Z2dlc3RlZC4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gV2UgYWxzbyBmYWNlZCBw
b2xpY3kgcmVsYXRlZCBpc3N1ZXMgd2l0aCBWaXJ0aW8gYmFja2VuZA0KPiA+ID4gcnVubmluZyBp
bg0KPiA+ID4gPiA+ID4gPiA+IG90aGVyIHRoYW4gRG9tMCBkb21haW4gaW4gYSAiZHVtbXkiIHhz
bSBtb2RlLiBJbiBvdXIgdGFyZ2V0DQo+ID4gPiBzeXN0ZW0gd2UNCj4gPiA+ID4gPiA+IHJ1bg0K
PiA+ID4gPiA+ID4gPiA+IHRoZSBiYWNrZW5kIGluIGEgZHJpdmVyDQo+ID4gPiA+ID4gPiA+ID4g
PiA+IGRvbWFpbiAod2UgY2FsbCBpdCBEb21EKSB3aGVyZSB0aGUgdW5kZXJseWluZyBIL1cNCj4g
cmVzaWRlcy4NCj4gPiA+IFdlDQo+ID4gPiA+ID4gPiB0cnVzdCBpdCwNCj4gPiA+ID4gPiA+ID4g
PiBzbyB3ZSB3cm90ZSBwb2xpY3kgcnVsZXMgKHRvIGJlIHVzZWQgaW4gImZsYXNrIiB4c20gbW9k
ZSkgdG8NCj4gPiA+IHByb3ZpZGUNCj4gPiA+ID4gPiA+IGl0DQo+ID4gPiA+ID4gPiA+ID4gd2l0
aCBhIGxpdHRsZSBiaXQgbW9yZSBwcml2aWxlZ2VzIHRoYW4gYSBzaW1wbGUgRG9tVSBoYWQuDQo+
ID4gPiA+ID4gPiA+ID4gPiA+IE5vdyBpdCBpcyBwZXJtaXR0ZWQgdG8gaXNzdWUgZGV2aWNlLW1v
ZGVsLCByZXNvdXJjZSBhbmQNCj4gPiA+IG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiA+IG1hcHBpbmdz
LCBldGMgY2FsbHMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVG8g
YWN0aXZhdGUgdGhlIG1hcHBpbmcgd2lsbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgcmVx
dWlyZSBzb21lIHNvcnQgb2YgaHlwZXJjYWxsIHRvIHRoZSBoeXBlcnZpc29yLg0KPiBJDQo+ID4g
PiBjYW4gc2VlDQo+ID4gPiA+ID4gPiB0d28NCj4gPiA+ID4gPiA+ID4gPiBvcHRpb25zDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBhdCB0aGlzIHBvaW50Og0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gZXhwb3NlIHRoZSBoYW5kbGUg
dG8gdXNlcnNwYWNlIGZvcg0KPiBkYWVtb24vaGVscGVyDQo+ID4gPiB0bw0KPiA+ID4gPiA+ID4g
dHJpZ2dlcg0KPiA+ID4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
ICAgbWFwcGluZyB2aWEgZXhpc3RpbmcgaHlwZXJjYWxsIGludGVyZmFjZXMuIElmDQo+ID4gPiB1
c2luZyBhDQo+ID4gPiA+ID4gPiBoZWxwZXINCj4gPiA+ID4gPiA+ID4gPiB5b3UNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gICAgIHdvdWxkIGhhdmUgYSBoeXBlcnZpc29yIHNwZWNpZmljIG9u
ZSB0byBhdm9pZA0KPiB0aGUNCj4gPiA+IGRhZW1vbg0KPiA+ID4gPiA+ID4gPiA+IGhhdmluZyB0
bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgY2FyZSB0b28gbXVjaCBhYm91dCB0aGUg
ZGV0YWlscyBvciBwdXNoIHRoYXQNCj4gPiA+IGNvbXBsZXhpdHkNCj4gPiA+ID4gPiA+IGludG8N
Cj4gPiA+ID4gPiA+ID4gPiBhDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICBjb21waWxl
IHRpbWUgb3B0aW9uIGZvciB0aGUgZGFlbW9uIHdoaWNoIHdvdWxkDQo+ID4gPiByZXN1bHQgaW4N
Cj4gPiA+ID4gPiA+ID4gPiBkaWZmZXJlbnQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAg
IGJpbmFyaWVzIGFsdGhvdWdoIGEgY29tbW9uIHNvdXJjZSBiYXNlLg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gZXhwb3NlIGEgbmV3IGtl
cm5lbCBBQkkgdG8gYWJzdHJhY3QgdGhlDQo+IGh5cGVyY2FsbA0KPiA+ID4gPiA+ID4gPiA+IGRp
ZmZlcmVuY2VzIGF3YXkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGluIHRoZSBndWVz
dCBrZXJuZWwuIEluIHRoaXMgY2FzZSB0aGUNCj4gdXNlcnNwYWNlDQo+ID4gPiB3b3VsZA0KPiA+
ID4gPiA+ID4gPiA+IGVzc2VudGlhbGx5DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICBh
c2sgZm9yIGFuIGFic3RyYWN0ICJtYXAgZ3Vlc3QgTiBtZW1vcnkgdG8NCj4gPiA+IHVzZXJzcGFj
ZQ0KPiA+ID4gPiA+ID4gcHRyIg0KPiA+ID4gPiA+ID4gPiA+IGFuZCBsZXQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gICAgIHRoZSBrZXJuZWwgZGVhbCB3aXRoIHRoZSBkaWZmZXJlbnQgaHlw
ZXJjYWxsDQo+ID4gPiBpbnRlcmZhY2VzLg0KPiA+ID4gPiA+ID4gPiA+IFRoaXMgb2YNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGNvdXJzZSBhc3N1bWVzIHRoZSBtYWpvcml0eSBvZiBC
RSBndWVzdHMgd291bGQNCj4gYmUNCj4gPiA+IExpbnV4DQo+ID4gPiA+ID4gPiA+ID4ga2VybmVs
cyBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGxlYXZlcyB0aGUgYmFyZS1tZXRh
bC91bmlrZXJuZWwgYXBwcm9hY2hlcyB0bw0KPiA+ID4gdGhlaXIgb3duDQo+ID4gPiA+ID4gPiA+
ID4gZGV2aWNlcy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gT3BlcmF0aW9uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID09PT09PT09PQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGUg
Y29yZSBvZiB0aGUgb3BlcmF0aW9uIG9mIFZpcnRJTyBpcyBmYWlybHkNCj4gc2ltcGxlLg0KPiA+
ID4gT25jZQ0KPiA+ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHZob3N0
LXVzZXIgZmVhdHVyZSBuZWdvdGlhdGlvbiBpcyBkb25lIGl0J3MgYSBjYXNlDQo+IG9mDQo+ID4g
PiA+ID4gPiByZWNlaXZpbmcNCj4gPiA+ID4gPiA+ID4gPiB1cGRhdGUNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gZXZlbnRzIGFuZCBwYXJzaW5nIHRoZSByZXN1bHRhbnQgdmlydCBxdWV1ZSBm
b3INCj4gZGF0YS4NCj4gPiA+IFRoZQ0KPiA+ID4gPiA+ID4gdmhvc3QtDQo+ID4gPiA+ID4gPiA+
ID4gdXNlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzcGVjaWZpY2F0aW9uIGhhbmRsZXMg
YSBidW5jaCBvZiBzZXR1cCBiZWZvcmUgdGhhdA0KPiA+ID4gcG9pbnQsDQo+ID4gPiA+ID4gPiBt
b3N0bHkNCj4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkZXRh
aWwgd2hlcmUgdGhlIHZpcnQgcXVldWVzIGFyZSBzZXQgdXAgRkQncyBmb3INCj4gPiA+IG1lbW9y
eSBhbmQNCj4gPiA+ID4gPiA+ID4gPiBldmVudA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBj
b21tdW5pY2F0aW9uLiBUaGlzIGlzIHdoZXJlIHRoZSBlbnZpc2lvbmVkIHN0dWINCj4gPiA+IHBy
b2Nlc3MNCj4gPiA+ID4gPiA+IHdvdWxkDQo+ID4gPiA+ID4gPiA+ID4gYmUNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gcmVzcG9uc2libGUgZm9yIGdldHRpbmcgdGhlIGRhZW1vbiB1cCBhbmQg
cmVhZHkgdG8NCj4gcnVuLg0KPiA+ID4gVGhpcw0KPiA+ID4gPiA+ID4gaXMNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gY3VycmVudGx5IGRvbmUgaW5zaWRlIGEgYmlnIFZNTSBsaWtlIFFFTVUg
YnV0IEkNCj4gPiA+IHN1c3BlY3QgYQ0KPiA+ID4gPiA+ID4gbW9kZXJuDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IGFwcHJvYWNoIHdvdWxkIGJlIHRvIHVzZSB0aGUgcnVzdC12bW0gdmhvc3Qg
Y3JhdGUuDQo+IEl0DQo+ID4gPiB3b3VsZA0KPiA+ID4gPiA+ID4gdGhlbg0KPiA+ID4gPiA+ID4g
PiA+IGVpdGhlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBjb21tdW5pY2F0ZSB3aXRoIHRo
ZSBrZXJuZWwncyBhYnN0cmFjdGVkIEFCSSBvciBiZQ0KPiByZS0NCj4gPiA+ID4gPiA+IHRhcmdl
dGVkDQo+ID4gPiA+ID4gPiA+ID4gYXMgYQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBidWls
ZCBvcHRpb24gZm9yIHRoZSB2YXJpb3VzIGh5cGVydmlzb3JzLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uZSB0aGluZyBJIG1lbnRpb25lZCBiZWZv
cmUgdG8gQWxleCBpcyB0aGF0IFhlbg0KPiBkb2Vzbid0DQo+ID4gPiBoYXZlDQo+ID4gPiA+ID4g
PiBWTU1zDQo+ID4gPiA+ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB3YXkg
dGhleSBhcmUgdHlwaWNhbGx5IGVudmlzaW9uZWQgYW5kIGRlc2NyaWJlZCBpbg0KPiBvdGhlcg0K
PiA+ID4gPiA+ID4gPiA+IGVudmlyb25tZW50cy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IElu
c3RlYWQsIFhlbiBoYXMgSU9SRVEgc2VydmVycy4gRWFjaCBvZiB0aGVtDQo+IGNvbm5lY3RzDQo+
ID4gPiA+ID4gPiA+ID4gaW5kZXBlbmRlbnRseSB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
WGVuIHZpYSB0aGUgSU9SRVEgaW50ZXJmYWNlLiBFLmcuIHRvZGF5IG11bHRpcGxlDQo+IFFFTVVz
DQo+ID4gPiBjb3VsZA0KPiA+ID4gPiA+ID4gYmUNCj4gPiA+ID4gPiA+ID4gPiB1c2VkIGFzDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBlbXVsYXRvcnMgZm9yIGEgc2luZ2xlIFhlbiBWTSwgZWFj
aCBvZiB0aGVtDQo+IGNvbm5lY3RpbmcNCj4gPiA+IHRvIFhlbg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gaW5kZXBlbmRlbnRseSB2aWEgdGhlIElPUkVRIGludGVyZmFjZS4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGUgY29tcG9uZW50IHJlc3Bv
bnNpYmxlIGZvciBzdGFydGluZyBhIGRhZW1vbg0KPiBhbmQvb3INCj4gPiA+IHNldHRpbmcNCj4g
PiA+ID4gPiA+IHVwDQo+ID4gPiA+ID4gPiA+ID4gc2hhcmVkDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBpbnRlcmZhY2VzIGlzIHRoZSB0b29sc3RhY2s6IHRoZSB4bCBjb21tYW5kIGFuZCB0aGUN
Cj4gPiA+ID4gPiA+IGxpYnhsL2xpYnhjDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBsaWJyYXJp
ZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJIHRoaW5r
IHRoYXQgVk0gY29uZmlndXJhdGlvbiBtYW5hZ2VtZW50IChvcg0KPiBvcmNoZXN0cmF0aW9uDQo+
ID4gPiBpbg0KPiA+ID4gPiA+ID4gPiA+IFN0YXJ0b3MNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBq
YXJnb24/KSBpcyBhIHN1YmplY3QgdG8gZGViYXRlIGluIHBhcmFsbGVsLg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IE90aGVyd2lzZSwgaXMgdGhlcmUgYW55IGdvb2QgYXNzdW1wdGlvbiB0byBhdm9p
ZCBpdA0KPiByaWdodA0KPiA+ID4gbm93Pw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBPbGVrc2FuZHIgYW5kIG90aGVycyBJIENDZWQgaGF2ZSBiZWVuIHdv
cmtpbmcgb24NCj4gd2F5cw0KPiA+ID4gZm9yIHRoZQ0KPiA+ID4gPiA+ID4gPiA+IHRvb2xzdGFj
aw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gY3JlYXRlIHZpcnRpbyBiYWNrZW5kcyBhbmQg
c2V0dXAgbWVtb3J5IG1hcHBpbmdzLg0KPiA+ID4gVGhleQ0KPiA+ID4gPiA+ID4gbWlnaHQgYmUN
Cj4gPiA+ID4gPiA+ID4gPiBhYmxlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0byBwcm92aWRl
IG1vcmUgaW5mbyBvbiB0aGUgc3ViamVjdC4gSSBkbyB0aGluayB3ZQ0KPiBtaXNzDQo+ID4gPiBh
IHdheQ0KPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiBwcm92aWRlDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiB0aGUgY29uZmlndXJhdGlvbiB0byB0aGUgYmFja2VuZCBhbmQgYW55dGhp
bmcgZWxzZQ0KPiB0aGF0DQo+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBtaWdodCByZXF1aXJlIHRvIHN0YXJ0IGRvaW5nIGl0cyBqb2Iu
DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IFllcywgc29tZSB3b3Jr
IGhhcyBiZWVuIGRvbmUgZm9yIHRoZSB0b29sc3RhY2sgdG8gaGFuZGxlDQo+ID4gPiBWaXJ0aW8N
Cj4gPiA+ID4gPiA+IE1NSU8NCj4gPiA+ID4gPiA+ID4gPiBkZXZpY2VzIGluDQo+ID4gPiA+ID4g
PiA+ID4gPiA+IGdlbmVyYWwgYW5kIFZpcnRpbyBibG9jayBkZXZpY2VzIGluIHBhcnRpY3VsYXIu
IEhvd2V2ZXIsDQo+IGl0DQo+ID4gPiBoYXMNCj4gPiA+ID4gPiA+IG5vdA0KPiA+ID4gPiA+ID4g
PiA+IGJlZW4gdXBzdHJlYW5lZCB5ZXQuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IFVwZGF0ZWQgcGF0
Y2hlcyBvbiByZXZpZXcgbm93Og0KPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRwczovL2xvcmUua2Vy
bmVsLm9yZy94ZW4tZGV2ZWwvMTYyMTYyNjM2MS0yOTA3Ni0xLQ0KPiBnaXQtDQo+ID4gPiBzZW5k
LQ0KPiA+ID4gPiA+ID4gZW1haWwtDQo+ID4gPiA+ID4gPiA+ID4gb2xla3N0eXNoQGdtYWlsLmNv
bS8NCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gVGhlcmUgaXMgYW4g
YWRkaXRpb25hbCAoYWxzbyBpbXBvcnRhbnQpIGFjdGl2aXR5IHRvDQo+ID4gPiBpbXByb3ZlL2Zp
eA0KPiA+ID4gPiA+ID4gPiA+IGZvcmVpZ24gbWVtb3J5IG1hcHBpbmcgb24gQXJtIHdoaWNoIEkg
YW0gYWxzbyBpbnZvbHZlZCBpbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gVGhlIGZvcmVpZ24gbWVt
b3J5IG1hcHBpbmcgaXMgcHJvcG9zZWQgdG8gYmUgdXNlZCBmb3INCj4gVmlydGlvDQo+ID4gPiA+
ID4gPiBiYWNrZW5kcw0KPiA+ID4gPiA+ID4gPiA+IChkZXZpY2UgZW11bGF0b3JzKSBpZiB0aGVy
ZSBpcyBhIG5lZWQgdG8gcnVuIGd1ZXN0IE9TDQo+IGNvbXBsZXRlbHkNCj4gPiA+ID4gPiA+ID4g
PiB1bm1vZGlmaWVkLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBPZiBjb3Vyc2UsIHRoZSBtb3JlIHNl
Y3VyZSB3YXkgd291bGQgYmUgdG8gdXNlIGdyYW50DQo+IG1lbW9yeQ0KPiA+ID4gPiA+ID4gbWFw
cGluZy4NCj4gPiA+ID4gPiA+ID4gPiBCcmlldGx5LCB0aGUgbWFpbiBkaWZmZXJlbmNlIGJldHdl
ZW4gdGhlbSBpcyB0aGF0IHdpdGgNCj4gZm9yZWlnbg0KPiA+ID4gbWFwcGluZw0KPiA+ID4gPiA+
ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gYmFja2VuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBjYW4g
bWFwIGFueSBndWVzdCBtZW1vcnkgaXQgd2FudHMgdG8gbWFwLCBidXQgd2l0aCBncmFudA0KPiA+
ID4gbWFwcGluZw0KPiA+ID4gPiA+ID4gaXQgaXMNCj4gPiA+ID4gPiA+ID4gPiBhbGxvd2VkIHRv
IG1hcCBvbmx5IHdoYXQgd2FzIHByZXZpb3VzbHkgZ3JhbnRlZCBieSB0aGUNCj4gZnJvbnRlbmQu
DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IFNvLCB0aGVyZSBtaWdo
dCBiZSBhIHByb2JsZW0gaWYgd2Ugd2FudCB0byBwcmUtbWFwIHNvbWUNCj4gPiA+IGd1ZXN0DQo+
ID4gPiA+ID4gPiBtZW1vcnkNCj4gPiA+ID4gPiA+ID4gPiBpbiBhZHZhbmNlIG9yIHRvIGNhY2hl
IG1hcHBpbmdzIGluIHRoZSBiYWNrZW5kIGluIG9yZGVyIHRvDQo+ID4gPiBpbXByb3ZlDQo+ID4g
PiA+ID4gPiA+ID4gcGVyZm9ybWFuY2UgKGJlY2F1c2UgdGhlIG1hcHBpbmcvdW5tYXBwaW5nIGd1
ZXN0IHBhZ2VzIGV2ZXJ5DQo+ID4gPiByZXF1ZXN0DQo+ID4gPiA+ID4gPiA+ID4gcmVxdWlyZXMg
YSBsb3Qgb2YgYmFjayBhbmQgZm9ydGggdG8gWGVuICsgUDJNIHVwZGF0ZXMpLiBJbiBhDQo+ID4g
PiBudXRzaGVsbCwNCj4gPiA+ID4gPiA+ID4gPiBjdXJyZW50bHksIGluIG9yZGVyIHRvIG1hcCBh
IGd1ZXN0IHBhZ2UgaW50byB0aGUgYmFja2VuZA0KPiBhZGRyZXNzDQo+ID4gPiBzcGFjZQ0KPiA+
ID4gPiA+ID4gd2UNCj4gPiA+ID4gPiA+ID4gPiBuZWVkIHRvIHN0ZWFsIGEgcmVhbCBwaHlzaWNh
bCBwYWdlIGZyb20gdGhlIGJhY2tlbmQgZG9tYWluLg0KPiBTbywNCj4gPiA+IHdpdGgNCj4gPiA+
ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+IHNhaWQgb3B0aW1pemF0aW9ucyB3ZSBtaWdodCBl
bmQgdXAgd2l0aCBubyBmcmVlIG1lbW9yeSBpbg0KPiB0aGUNCj4gPiA+IGJhY2tlbmQNCj4gPiA+
ID4gPiA+ID4gPiBkb21haW4gKHNlZSBYU0EtMzAwKS4gQW5kIHdoYXQgd2UgdHJ5IHRvIGFjaGll
dmUgaXMgdG8gbm90DQo+IHdhc3RlDQo+ID4gPiBhDQo+ID4gPiA+ID4gPiByZWFsDQo+ID4gPiA+
ID4gPiA+ID4gZG9tYWluIG1lbW9yeSBhdCBhbGwgYnkgcHJvdmlkaW5nIHNhZmUgbm9uLWFsbG9j
YXRlZC15ZXQgKHNvDQo+ID4gPiB1bnVzZWQpDQo+ID4gPiA+ID4gPiA+ID4gYWRkcmVzcyBzcGFj
ZSBmb3IgdGhlIGZvcmVpZ24gKGFuZCBncmFudCkgcGFnZXMgdG8gYmUgbWFwcGVkDQo+ID4gPiBp
bnRvLA0KPiA+ID4gPiA+ID4gdGhpcw0KPiA+ID4gPiA+ID4gPiA+IGVuYWJsaW5nIHdvcmsgaW1w
bGllcyBYZW4gYW5kIExpbnV4IChhbmQgbGlrZWx5IERUQiBiaW5kaW5ncykNCj4gPiA+IGNoYW5n
ZXMuDQo+ID4gPiA+ID4gPiA+ID4gSG93ZXZlciwgYXMgaXQgdHVybmVkIG91dCwgZm9yIHRoaXMg
dG8gd29yayBpbiBhIHByb3BlciBhbmQNCj4gc2FmZQ0KPiA+ID4gd2F5DQo+ID4gPiA+ID4gPiBz
b21lDQo+ID4gPiA+ID4gPiA+ID4gcHJlcmVxIHdvcmsgbmVlZHMgdG8gYmUgZG9uZS4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gWW91IGNhbiBmaW5kIHRoZSByZWxhdGVkIFhlbiBkaXNjdXNzaW9uIGF0
Og0KPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRwczovL2xvcmUua2VybmVsLm9yZy94ZW4tZGV2ZWwv
MTYyNzQ4OTExMC0yNTYzMy0xLQ0KPiBnaXQtDQo+ID4gPiBzZW5kLQ0KPiA+ID4gPiA+ID4gZW1h
aWwtDQo+ID4gPiA+ID4gPiA+ID4gb2xla3N0eXNoQGdtYWlsLmNvbS8NCj4gPiA+ID4gPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbmUgcXVlc3Rp
b24gaXMgaG93IHRvIGJlc3QgaGFuZGxlIG5vdGlmaWNhdGlvbg0KPiBhbmQNCj4gPiA+IGtpY2tz
Lg0KPiA+ID4gPiA+ID4gVGhlDQo+ID4gPiA+ID4gPiA+ID4gZXhpc3RpbmcNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gdmhvc3QtdXNlciBmcmFtZXdvcmsgdXNlcyBldmVudGZkIHRvIHNpZ25h
bCB0aGUNCj4gZGFlbW9uDQo+ID4gPiA+ID4gPiAoYWx0aG91Z2gNCj4gPiA+ID4gPiA+ID4gPiBR
RU1VDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGlzIHF1aXRlIGNhcGFibGUgb2Ygc2ltdWxh
dGluZyB0aGVtIHdoZW4geW91IHVzZQ0KPiBUQ0cpLg0KPiA+ID4gWGVuDQo+ID4gPiA+ID4gPiBo
YXMNCj4gPiA+ID4gPiA+ID4gPiBpdCdzIG93bg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJ
T1JFUSBtZWNoYW5pc20uIEhvd2V2ZXIgbGF0ZW5jeSBpcyBhbiBpbXBvcnRhbnQNCj4gPiA+IGZh
Y3RvciBhbmQNCj4gPiA+ID4gPiA+ID4gPiBoYXZpbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gZXZlbnRzIGdvIHRocm91Z2ggdGhlIHN0dWIgd291bGQgYWRkIHF1aXRlIGEgbG90Lg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFllYWggSSB0aGlu
aywgcmVnYXJkbGVzcyBvZiBhbnl0aGluZyBlbHNlLCB3ZSB3YW50DQo+IHRoZQ0KPiA+ID4gPiA+
ID4gYmFja2VuZHMgdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNvbm5lY3QgZGlyZWN0bHkg
dG8gdGhlIFhlbiBoeXBlcnZpc29yLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gSW4gbXkgYXBwcm9hY2gsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIGEpIEJF
IC0+IEZFOiBpbnRlcnJ1cHRzIHRyaWdnZXJlZCBieSBCRSBjYWxsaW5nIGENCj4gPiA+IGh5cGVy
dmlzb3INCj4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
ICAgICAgICAgICAgIHZpYSB2aXJ0aW8tcHJveHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgYikg
RkUgLT4gQkU6IE1NSU8gdG8gY29uZmlnIHJhaXNlcyBldmVudHMgKGluIGV2ZW50DQo+ID4gPiBj
aGFubmVscyksDQo+ID4gPiA+ID4gPiA+ID4gd2hpY2ggaXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiAgICAgICAgICAgICAgIGNvbnZlcnRlZCB0byBhIGNhbGxiYWNrIHRvIEJFIHZpYQ0KPiB2aXJ0
aW8tDQo+ID4gPiBwcm94eQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgKFhl
bidzIGV2ZW50IGNoYW5uZWwgaXMgaW50ZXJubmFsbHkNCj4gPiA+IGltcGxlbWVudGVkIGJ5DQo+
ID4gPiA+ID4gPiA+ID4gaW50ZXJydXB0cy4pDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBJIGRvbid0IGtub3cgd2hhdCAiY29ubmVjdCBkaXJlY3RseSIgbWVh
bnMgaGVyZSwgYnV0DQo+ID4gPiBzZW5kaW5nDQo+ID4gPiA+ID4gPiA+ID4gaW50ZXJydXB0cw0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHRoZSBvcHBvc2l0ZSBzaWRlIHdvdWxkIGJlIGJlc3Qg
ZWZmaWNpZW50Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEl2c2htZW0sIEkgc3VwcG9zZSwgdGFr
ZXMgdGhpcyBhcHByb2FjaCBieSB1dGlsaXppbmcNCj4gUENJJ3MNCj4gPiA+IG1zaS14DQo+ID4g
PiA+ID4gPiA+ID4gbWVjaGFuaXNtLg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiBBZ3JlZSB0aGF0IE1TSSB3b3VsZCBiZSBtb3JlIGVmZmljaWVudCB0aGFuIFNQSS4u
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBBdCB0aGUgbW9tZW50LCBpbiBvcmRlciB0byBub3RpZnkg
dGhlIGZyb250ZW5kLCB0aGUNCj4gYmFja2VuZA0KPiA+ID4gaXNzdWVzDQo+ID4gPiA+ID4gPiBh
DQo+ID4gPiA+ID4gPiA+ID4gc3BlY2lmaWMgZGV2aWNlLW1vZGVsIGNhbGwgdG8gcXVlcnkgWGVu
IHRvIGluamVjdCBhDQo+ID4gPiBjb3JyZXNwb25kaW5nIFNQSQ0KPiA+ID4gPiA+ID4gdG8NCj4g
PiA+ID4gPiA+ID4gPiB0aGUgZ3Vlc3QuDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBDb3VsZCB3ZSBjb25zaWRlciB0aGUga2VybmVsIGlu
dGVybmFsbHkgY29udmVydGluZw0KPiA+ID4gSU9SRVENCj4gPiA+ID4gPiA+ID4gPiBtZXNzYWdl
cyBmcm9tDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBYZW4gaHlwZXJ2aXNvciB0byBl
dmVudGZkIGV2ZW50cz8gV291bGQgdGhpcw0KPiBzY2FsZQ0KPiA+ID4gd2l0aA0KPiA+ID4gPiA+
ID4gPiA+IG90aGVyIGtlcm5lbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBoeXBlcmNhbGwg
aW50ZXJmYWNlcz8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gU28gYW55IHRob3VnaHRzIG9uIHdoYXQgZGlyZWN0aW9ucyBhcmUgd29ydGgNCj4g
PiA+IGV4cGVyaW1lbnRpbmcNCj4gPiA+ID4gPiA+IHdpdGg/DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT25lIG9wdGlvbiB3ZSBzaG91bGQgY29uc2lk
ZXIgaXMgZm9yIGVhY2ggYmFja2VuZCB0bw0KPiA+ID4gY29ubmVjdA0KPiA+ID4gPiA+ID4gdG8N
Cj4gPiA+ID4gPiA+ID4gPiBYZW4gdmlhDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgSU9S
RVEgaW50ZXJmYWNlLiBXZSBjb3VsZCBnZW5lcmFsaXplIHRoZSBJT1JFUQ0KPiA+ID4gaW50ZXJm
YWNlDQo+ID4gPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiBtYWtlIGl0DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBoeXBlcnZpc29yIGFnbm9zdGljLiBUaGUgaW50ZXJmYWNlIGlzIHJlYWxs
eSB0cml2aWFsDQo+IGFuZA0KPiA+ID4gZWFzeQ0KPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+
ID4gPiBhZGQuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBB
cyBJIHNhaWQgYWJvdmUsIG15IHByb3Bvc2FsIGRvZXMgdGhlIHNhbWUgdGhpbmcgdGhhdA0KPiB5
b3UNCj4gPiA+ID4gPiA+IG1lbnRpb25lZA0KPiA+ID4gPiA+ID4gPiA+IGhlcmUgOikNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBUaGUgZGlmZmVyZW5jZSBpcyB0aGF0IEkgZG8gY2FsbCBoeXBlcnZp
c29yIGludGVyZmFjZXMNCj4gdmlhDQo+ID4gPiA+ID4gPiB2aXJ0aW8tDQo+ID4gPiA+ID4gPiA+
ID4gcHJveHkuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IFRoZSBvbmx5IFhlbi1zcGVjaWZpYyBwYXJ0IGlzIHRoZSBub3RpZmljYXRpb24NCj4gbWVjaGFu
aXNtLA0KPiA+ID4gPiA+ID4gd2hpY2ggaXMNCj4gPiA+ID4gPiA+ID4gPiBhbg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gZXZlbnQgY2hhbm5lbC4gSWYgd2UgcmVwbGFjZWQgdGhlIGV2ZW50IGNo
YW5uZWwgd2l0aA0KPiA+ID4gc29tZXRoaW5nDQo+ID4gPiA+ID4gPiA+ID4gZWxzZSB0aGUNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZSB3b3VsZCBiZSBnZW5lcmljLiBTZWU6DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRwczovL2dpdGxhYi5jb20veGVuLXByb2plY3QveGVu
Ly0NCj4gPiA+ID4gPiA+ID4gPiAvYmxvYi9zdGFnaW5nL3hlbi9pbmNsdWRlL3B1YmxpYy9odm0v
aW9yZXEuaCNMNTINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBJIGRvbid0IHRoaW5rIHRoYXQgdHJhbnNsYXRpbmcgSU9SRVFzIHRvIGV2ZW50ZmQgaW4N
Cj4gdGhlDQo+ID4gPiBrZXJuZWwNCj4gPiA+ID4gPiA+IGlzDQo+ID4gPiA+ID4gPiA+ID4gYQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZ29vZCBpZGVhOiBpZiBmZWVscyBsaWtlIGl0IHdvdWxk
IGJlIGV4dHJhDQo+IGNvbXBsZXhpdHkNCj4gPiA+IGFuZCB0aGF0DQo+ID4gPiA+ID4gPiB0aGUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGtlcm5lbCBzaG91bGRuJ3QgYmUgaW52b2x2ZWQgYXMg
dGhpcyBpcyBhIGJhY2tlbmQtDQo+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gaW50
ZXJmYWNlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gR2l2
ZW4gdGhhdCB3ZSBtYXkgd2FudCB0byBpbXBsZW1lbnQgQkUgYXMgYSBiYXJlLW1ldGFsDQo+ID4g
PiA+ID4gPiBhcHBsaWNhdGlvbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFzIEkgZGlkIG9uIFpl
cGh5ciwgSSBkb24ndCB0aGluayB0aGF0IHRoZSB0cmFuc2xhdGlvbg0KPiA+ID4gd291bGQgbm90
DQo+ID4gPiA+ID4gPiBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGEgYmlnIGlzc3VlLCBlc3Bl
Y2lhbGx5IG9uIFJUT1Mncy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJdCB3aWxsIGJlIHNvbWUg
a2luZCBvZiBhYnN0cmFjdGlvbiBsYXllciBvZiBpbnRlcnJ1cHQNCj4gPiA+IGhhbmRsaW5nDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gKG9yIG5vdGhpbmcgYnV0IGEgY2FsbGJhY2sgbWVjaGFuaXNt
KS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQWxzbywg
ZXZlbnRmZCBpcyB2ZXJ5IExpbnV4LWNlbnRyaWMgYW5kIHdlIGFyZQ0KPiB0cnlpbmcgdG8NCj4g
PiA+ID4gPiA+IGRlc2lnbiBhbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlIHRo
YXQgY291bGQgd29yayB3ZWxsIGZvciBSVE9TZXMgdG9vLiBJZiB3ZQ0KPiA+ID4gd2FudCB0bw0K
PiA+ID4gPiA+ID4gZG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHNvbWV0aGluZyBkaWZmZXJl
bnQsIGJvdGggT1MtYWdub3N0aWMgYW5kDQo+IGh5cGVydmlzb3ItDQo+ID4gPiBhZ25vc3RpYywN
Cj4gPiA+ID4gPiA+ID4gPiBwZXJoYXBzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB3ZSBjb3Vs
ZCBkZXNpZ24gYSBuZXcgaW50ZXJmYWNlLiBPbmUgdGhhdCBjb3VsZCBiZQ0KPiA+ID4gPiA+ID4g
aW1wbGVtZW50YWJsZQ0KPiA+ID4gPiA+ID4gPiA+IGluIHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gWGVuIGh5cGVydmlzb3IgaXRzZWxmIChsaWtlIElPUkVRKSBhbmQgb2YgY291cnNlIGFu
eQ0KPiA+ID4gb3RoZXINCj4gPiA+ID4gPiA+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiB0b28uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoZXJlIGlzIGFsc28gYW5vdGhlciBw
cm9ibGVtLiBJT1JFUSBpcyBwcm9iYWJseSBub3QNCj4gYmUNCj4gPiA+IHRoZQ0KPiA+ID4gPiA+
ID4gb25seQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlIG5lZWRlZC4gSGF2ZSBh
IGxvb2sgYXQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGh0dHBzOi8vbWFyYy5pbmZvLz9sPXhl
bi1kZXZlbCZtPTE2MjM3Mzc1NDcwNTIzMyZ3PTIuDQo+ID4gPiBEb24ndCB3ZQ0KPiA+ID4gPiA+
ID4gPiA+IGFsc28gbmVlZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYW4gaW50ZXJmYWNlIGZv
ciB0aGUgYmFja2VuZCB0byBpbmplY3QgaW50ZXJydXB0cw0KPiBpbnRvDQo+ID4gPiB0aGUNCj4g
PiA+ID4gPiA+ID4gPiBmcm9udGVuZD8gQW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpZiB0
aGUgYmFja2VuZCByZXF1aXJlcyBkeW5hbWljIG1lbW9yeSBtYXBwaW5ncyBvZg0KPiA+ID4gZnJv
bnRlbmQNCj4gPiA+ID4gPiA+IHBhZ2VzLA0KPiA+ID4gPiA+ID4gPiA+IHRoZW4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHdlIHdvdWxkIGFsc28gbmVlZCBhbiBpbnRlcmZhY2UgdG8gbWFwL3Vu
bWFwIGRvbVUNCj4gcGFnZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBNeSBwcm9wb3NhbCBkb2N1bWVudCBtaWdodCBoZWxwIGhlcmU7IEFsbCB0aGUNCj4g
aW50ZXJmYWNlcw0KPiA+ID4gPiA+ID4gcmVxdWlyZWQNCj4gPiA+ID4gPiA+ID4gPiBmb3INCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiB2aXJ0aW8tcHJveHkgKG9yIGh5cGVydmlzb3ItcmVsYXRlZCBp
bnRlcmZhY2VzKSBhcmUNCj4gbGlzdGVkDQo+ID4gPiBhcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IFJQQyBwcm90b2NvbHMgOikNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gVGhlc2UgaW50ZXJmYWNlcyBhcmUgYSBsb3QgbW9yZSBwcm9ibGVtYXRpYyB0aGFu
DQo+IElPUkVROg0KPiA+ID4gSU9SRVENCj4gPiA+ID4gPiA+IGlzDQo+ID4gPiA+ID4gPiA+ID4g
dGlueQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYW5kIHNlbGYtY29udGFpbmVkLiBJdCBpcyBl
YXN5IHRvIGFkZCBhbnl3aGVyZS4gQQ0KPiBuZXcNCj4gPiA+ID4gPiA+IGludGVyZmFjZSB0bw0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW5qZWN0IGludGVycnVwdHMgb3IgbWFwIHBhZ2VzIGlz
IG1vcmUgZGlmZmljdWx0IHRvDQo+ID4gPiBtYW5hZ2UNCj4gPiA+ID4gPiA+IGJlY2F1c2UNCj4g
PiA+ID4gPiA+ID4gPiBpdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd291bGQgcmVxdWlyZSBj
aGFuZ2VzIHNjYXR0ZXJlZCBhY3Jvc3MgdGhlIHZhcmlvdXMNCj4gPiA+IGVtdWxhdG9ycy4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEV4YWN0bHkuIEkgaGF2
ZSBubyBjb25maWRlbnQgeWV0IHRoYXQgbXkgYXBwcm9hY2ggd2lsbA0KPiA+ID4gYWxzbw0KPiA+
ID4gPiA+ID4gYXBwbHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0byBvdGhlciBoeXBlcnZpc29y
cyB0aGFuIFhlbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUZWNobmljYWxseSwgeWVzLCBidXQg
d2hldGhlciBwZW9wbGUgY2FuIGFjY2VwdCBpdCBvcg0KPiBub3QNCj4gPiA+IGlzIGENCj4gPiA+
ID4gPiA+ID4gPiBkaWZmZXJlbnQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBtYXR0ZXIuDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGFua3MsDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gLVRha2FoaXJvIEFrYXNoaQ0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiAtLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBSZWdhcmRzLA0KPiA+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiBPbGVrc2FuZHIgVHlzaGNoZW5rbw0KPiA+ID4gPiA+ID4gPiA+
ID4gSU1QT1JUQU5UIE5PVElDRTogVGhlIGNvbnRlbnRzIG9mIHRoaXMgZW1haWwgYW5kIGFueQ0K
PiA+ID4gYXR0YWNobWVudHMgYXJlDQo+ID4gPiA+ID4gPiA+ID4gY29uZmlkZW50aWFsIGFuZCBt
YXkgYWxzbyBiZSBwcml2aWxlZ2VkLiBJZiB5b3UgYXJlIG5vdCB0aGUNCj4gPiA+IGludGVuZGVk
DQo+ID4gPiA+ID4gPiA+ID4gcmVjaXBpZW50LCBwbGVhc2Ugbm90aWZ5IHRoZSBzZW5kZXIgaW1t
ZWRpYXRlbHkgYW5kIGRvIG5vdA0KPiA+ID4gZGlzY2xvc2UNCj4gPiA+ID4gPiA+IHRoZQ0KPiA+
ID4gPiA+ID4gPiA+IGNvbnRlbnRzIHRvIGFueSBvdGhlciBwZXJzb24sIHVzZSBpdCBmb3IgYW55
IHB1cnBvc2UsIG9yDQo+IHN0b3JlDQo+ID4gPiBvciBjb3B5DQo+ID4gPiA+ID4gPiB0aGUNCj4g
PiA+ID4gPiA+ID4gPiBpbmZvcm1hdGlvbiBpbiBhbnkgbWVkaXVtLiBUaGFuayB5b3UuDQo+ID4g
PiA+ID4gPiA+IElNUE9SVEFOVCBOT1RJQ0U6IFRoZSBjb250ZW50cyBvZiB0aGlzIGVtYWlsIGFu
ZCBhbnkNCj4gYXR0YWNobWVudHMNCj4gPiA+IGFyZQ0KPiA+ID4gPiA+ID4gY29uZmlkZW50aWFs
IGFuZCBtYXkgYWxzbyBiZSBwcml2aWxlZ2VkLiBJZiB5b3UgYXJlIG5vdCB0aGUNCj4gaW50ZW5k
ZWQNCj4gPiA+ID4gPiA+IHJlY2lwaWVudCwgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVyIGltbWVk
aWF0ZWx5IGFuZCBkbyBub3QNCj4gZGlzY2xvc2UNCj4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gY29u
dGVudHMgdG8gYW55IG90aGVyIHBlcnNvbiwgdXNlIGl0IGZvciBhbnkgcHVycG9zZSwgb3Igc3Rv
cmUNCj4gb3INCj4gPiA+IGNvcHkgdGhlDQo+ID4gPiA+ID4gPiBpbmZvcm1hdGlvbiBpbiBhbnkg
bWVkaXVtLiBUaGFuayB5b3UuDQo+ID4gPiA+ID4gSU1QT1JUQU5UIE5PVElDRTogVGhlIGNvbnRl
bnRzIG9mIHRoaXMgZW1haWwgYW5kIGFueSBhdHRhY2htZW50cw0KPiBhcmUNCj4gPiA+IGNvbmZp
ZGVudGlhbCBhbmQgbWF5IGFsc28gYmUgcHJpdmlsZWdlZC4gSWYgeW91IGFyZSBub3QgdGhlIGlu
dGVuZGVkDQo+ID4gPiByZWNpcGllbnQsIHBsZWFzZSBub3RpZnkgdGhlIHNlbmRlciBpbW1lZGlh
dGVseSBhbmQgZG8gbm90IGRpc2Nsb3NlDQo+IHRoZQ0KPiA+ID4gY29udGVudHMgdG8gYW55IG90
aGVyIHBlcnNvbiwgdXNlIGl0IGZvciBhbnkgcHVycG9zZSwgb3Igc3RvcmUgb3IgY29weQ0KPiB0
aGUNCj4gPiA+IGluZm9ybWF0aW9uIGluIGFueSBtZWRpdW0uIFRoYW5rIHlvdS4NCj4gPiBJTVBP
UlRBTlQgTk9USUNFOiBUaGUgY29udGVudHMgb2YgdGhpcyBlbWFpbCBhbmQgYW55IGF0dGFjaG1l
bnRzIGFyZQ0KPiBjb25maWRlbnRpYWwgYW5kIG1heSBhbHNvIGJlIHByaXZpbGVnZWQuIElmIHlv
dSBhcmUgbm90IHRoZSBpbnRlbmRlZA0KPiByZWNpcGllbnQsIHBsZWFzZSBub3RpZnkgdGhlIHNl
bmRlciBpbW1lZGlhdGVseSBhbmQgZG8gbm90IGRpc2Nsb3NlIHRoZQ0KPiBjb250ZW50cyB0byBh
bnkgb3RoZXIgcGVyc29uLCB1c2UgaXQgZm9yIGFueSBwdXJwb3NlLCBvciBzdG9yZSBvciBjb3B5
IHRoZQ0KPiBpbmZvcm1hdGlvbiBpbiBhbnkgbWVkaXVtLiBUaGFuayB5b3UuDQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 11:48:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 11:48:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176318.320841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOiv-00061F-Kd; Wed, 01 Sep 2021 11:48:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176318.320841; Wed, 01 Sep 2021 11:48:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOiv-000618-Hd; Wed, 01 Sep 2021 11:48:09 +0000
Received: by outflank-mailman (input) for mailman id 176318;
 Wed, 01 Sep 2021 11:48:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLOit-00060y-Ue; Wed, 01 Sep 2021 11:48:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLOit-00011v-Q6; Wed, 01 Sep 2021 11:48:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLOit-0000tK-J8; Wed, 01 Sep 2021 11:48:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLOit-0005bE-If; Wed, 01 Sep 2021 11:48:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3Rth7U/q/9bvQyAWjxieMb40Jd1LEppkXg3cKsRDZKE=; b=GBEXuN4lTf+PAfqrqB9EZfD9aw
	z15VSt3PWfUQMhFb4MCtWuNjhl1unGaf2QA0C7JGudjddR4f3y8XRT71fUDfPZ1q8J1aByZMgWKPA
	06CNla7KiNgyV3lTcNOakzZ0wP+tcyEmrttVxLRd7rDg8mY+l4VNan0r8S4o56nrwlQU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164688-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164688: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=a4785a1ea58a59c6ee14619b9bbeb4856eaf8175
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 11:48:07 +0000

flight 164688 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164688/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              a4785a1ea58a59c6ee14619b9bbeb4856eaf8175
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  418 days
Failing since        151818  2020-07-11 04:18:52 Z  417 days  408 attempts
Testing same since   164688  2021-09-01 04:20:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71454 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 11:49:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 11:49:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176325.320855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOkb-0006as-0o; Wed, 01 Sep 2021 11:49:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176325.320855; Wed, 01 Sep 2021 11:49:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLOka-0006al-U5; Wed, 01 Sep 2021 11:49:52 +0000
Received: by outflank-mailman (input) for mailman id 176325;
 Wed, 01 Sep 2021 11:49:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rqR/=NX=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mLOkY-0006ab-LS
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 11:49:50 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5789470-0b1a-11ec-adcd-12813bfff9fa;
 Wed, 01 Sep 2021 11:49:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5789470-0b1a-11ec-adcd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630496989;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=35SwvRjjqZ2KnuTWfd6fU16T0czSr0GGQW4+FbwuguA=;
  b=D/rmxfz8wodF7lzhzF55kz0snDFbLU9ZywLVWwCLEkKo9S7141kVq894
   DjGbkPoHx1V4f90freWZsUCdfBdLU771TZP55Z8RwMEUrIdPDbA11LPB0
   Bk+abGn6Co1sp4EVKLlXwLeYoAfayCwiXFWxgEQKPVd8+iX0njGaZv0F+
   M=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: Tt9abo8Jh6bH658YlIkLPZtE83hkPRUjbi68J1FQIjXHyvupdzQ97k/6MOLc/WCgito3LGKiIG
 1NsRBVcjwaNxXNbAZDhSfPy6Iw0swWH4mb6C6nXyI49fieD09Jzk/LGfKGhkUBTSLAzmtJO9KK
 gmrBQ0FI6gynknTMTWvPUPXSZq8Lpf87XXQ6/CaE7DjI5+5YrQ2dM2+salV1AxkaICYgTVqix8
 L27fhgILjctj+POvhFWi8BhhsCvfBlxMwRCkw6D+ACesd93Jh9lqRPWy35p363l1uhXpdokpW0
 qyF2EbTliryKgiPVC/pYcaQf
X-SBRS: 5.1
X-MesageID: 53523089
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:8i7vcK0NYLSmBIeU/mjBLgqjBTJyeYIsimQD101hICG9Lfb3qy
 n+ppsmPEHP5Ar5OEtBpTiBUJPwJU80hqQFn7X5Wo3SIzUO2VHYUL2KiLGC/9SOIVyEygcw79
 YHT0E6MqyMMbEYt7eI3ODbKadZ/DDvysnB7o2yvhQdL3AeV0gj1XYfNu/yKDwHeOAsP+tBKH
 Pz3Lsjm9PtQwVsUiztbUN1LtQr6ue72q7OUFojPVoK+QOOhTSn5PrTFAWZ5A4XV3dqza05+W
 bIvgTl7uH72svLiiP05iv21dB7idHhwtxMCIiljdUUECzljkKNaJ56U7OPkTgpqKWE6Uoskv
 PLvxA8Vv4Dqk/5TyWQm1/AygPg2DEh5zvLzkKZu2LqpYjDSDczG6N69M9kWyqcz3BlkMB30a
 pN0W7cnYFQFwn8kCP04MWNfw12l2KvyEBS1NI7vjh6a88zebVRpYsQ8Ad+C5EbBh/374ghDa
 1HENzc3vBLalmXBkqp/FWH+ObcGUjbIy32BHTr4qeuomFrdTFCvgglLfUk7zM9HMlXcegd2w
 ysWZ4Y5o2nTactHOhA7ak6MJCK4sGke2OEDIuoGyWRKEgwAQOHl3fG2sRA2AiUQu1/8HITou
 WMbLoKjx98R6rRYff+lKGjtCq9GlmAYQ==
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="53523089"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PPZbS1jOrRpOst6n+6nM+/o7uOL1PHvVU96Xn1xsOp7V6/xuFQWZLgU1xg1cFjEniKOOedkju+UIUNNJMh5vtdRH56wUU3x2mCkfuor/mgvx3kLvcs9xjlPSpJ/l653pm55ww272tFRlzfaoKmcgIPyxmtO0WUVoILsg9sZ2wHs0ljklugi1JddWC6ogo0ZgRcXDpTlyAiT3Tx3sNhE8ByXoL+pG/l8vMHZPI5OE+fndV4xZqjozjGLUgZV1AUJouvY8DWYBfHhVwUGZjaAfcHmoQMagbGrupJA/dVVC/ngjbxYrz3ABxu62deUnyqPvvQ9av9n0SkCgcBnE64SuRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cKjO7PQGRNxGgIvaAymBw9Wc66auu2P0CgYv89Iupg4=;
 b=QBwBcrcwv/kk/NQ+wrBHuDoE7zNr/28C5XmKa/55gMcYTPcAnjUxCNaoG2nG+Pvx5sUBp1k4SRa8YasxjUCrAo78hrkFfdNPDBaoscE0k9/2HPs/K/4k4y9xyG9SbVrbKFZ/4E4mPIL0ftTtq3KsS2HiIdL8bdOVvZ87YFylEfnmiSCFOaSoYIlChOS9w1k7SBq/MVh7+fYGn3Zs/oObaLMCwAJ2r8xu8Oky/AZj6NkmYQqCO7dHcbyPSSsVq0ItOprT4OetvT9jSsVxO2YTD16Hdw3DTl0tAPycMfVyGryigTC6F3QjseMMH/zBL+7I388N4XMjuaRdNfArYyE8Mg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cKjO7PQGRNxGgIvaAymBw9Wc66auu2P0CgYv89Iupg4=;
 b=SLHwyq6PSzue0yxO+Chmeh6OVKBD6amNIqCv6VwOZYb6yx9gn0QMMcwvG4LXX8ceHv7W2/S9OEFmccj59h6qm/TejXSxw/P3LQ8F0EMM0+Rx8w40W/OKWDZ7M0t6amlo76EC2IBZu3w4BTrnNeSADnaPuZgDeLocoqxhPXUJVHA=
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <013c49f5-7a14-9d9e-4d25-75934bf6349f@suse.com>
 <9cfc122f-dc4e-9dc0-0b57-48abf941025a@citrix.com>
 <866a3677-1f1a-e0e5-7ab0-10c210e25610@suse.com>
 <2265ca67-2a30-a858-38ba-a6343782929e@suse.com>
 <2ef761cf-e609-e0df-e360-9e904f5ad9a2@citrix.com>
 <a64e4a10-6792-2900-128c-ece7ba6d6919@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/4] x86/PVH: de-duplicate mappings for first Mb of Dom0
 memory
Message-ID: <96563cf3-4c99-520e-99d4-68bf98ef46a2@citrix.com>
Date: Wed, 1 Sep 2021 12:49:35 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <a64e4a10-6792-2900-128c-ece7ba6d6919@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0339.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4fa0e4f4-e39b-442c-f827-08d96d3e9588
X-MS-TrafficTypeDiagnostic: BY5PR03MB5045:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB5045AD594FF6FA10F8939EFEBACD9@BY5PR03MB5045.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: VdDZsNlVRE/GOg6HbW5K4gdbEeLHaCClLtf7HYeElVkHygl1MzxpTUU2eMsVOAOorQR+NlJUJrnC6iTG5RHdMaqRUb3Pju/rr1Gaj/Ezte2ORlCnh4LHV7yldAEHA/siwrXu/j3A1/SaItvM5VR/RDvweaa8FCJNwy9brSbp/NeVMs/O7ExCyTi4DOIzCE7M4FbKtJPV7LGkDRPLZqTuvyl41yVHwBfi70zbFioka8bPK88kf7Ow3t/QjryAGHO9/zz30AZbGHnI/4CPev165uQRgaQ3prQDxuAqh8ak3ntWaQtXtIW1GF4mucVKprQOvYFNXgLa18Kkd64Lxh2xTtW4kQbl39i7B1X4PkhqFcxDtJFgPF1olrKDtsahkykES7kfbK3NFk9tpDca7NNhz8DlDWZPxKKfmjNauVUwq1ETLGGwfpceoexhefLGcYn5c2fP+aH1dLpzM3wjJk1OzmBNx76/+L4EDAe/TxuFKrkqT+nLYz0zgw0P39PJ7tMq3PxiI2JPGscg1XHMuZ/Vjaci9L0sfBPOn4z7V0Zmzu5OstCKOlj6li1URdSYpirbQIVxSS+6fFtaJF3OCrC+Vkm90W0l45kbJQMBocaNqSv5r0L6gTfgdHsgibJsq/zDgnD6jMuOStZMMyTDUD+DbmbDLdDWKdQS/0DGwaCHdYU/OKIBU2eWW4jogFe6uEBf/7s0/XqBDVFHeGWwzq+5U0ab6xDQLq1PjPL6OpX3z3g=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(956004)(38100700002)(54906003)(2616005)(16576012)(6916009)(66476007)(86362001)(31686004)(31696002)(508600001)(6666004)(8936002)(36756003)(8676002)(2906002)(53546011)(55236004)(66946007)(186003)(26005)(316002)(4326008)(66556008)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WVJYSjl3WjFiYmpOUzJodTM2R21TeVVlQWdrOHc3M3Z4d2duOHpQZ0RWMmFY?=
 =?utf-8?B?a1NYbkRVWWVvNkdBMnJVZllNUjIzS0laU0FxWVJBSXdrbVhlRFNnM0ZFQi9a?=
 =?utf-8?B?cit2NmRKTXA5UWc3SHV5NDlPV1pHYmxtejlOWFZpak5YN0tta2FuSmprc1RK?=
 =?utf-8?B?Ty9IY05iTlZtYmJUSjFNRGJCU2RObVVXUnlWZGsrNm1oYnkyVndVZzFtbHgx?=
 =?utf-8?B?Vk1vMTdnd2psb0NEd1pBQVBxRC90RTZtSTB5L2ZWUU9McjVqWEtwSDRJNFBu?=
 =?utf-8?B?SWREV20vN2RMVkNLZWUxemNRTDVaYU9wOHlOd1M4R2JuQjdCam5oL0pRQXlo?=
 =?utf-8?B?OU1CYnZmYmdwOWNmU2h6NXgrUkZTeDBHZE9WQlJSQVRoQTdxdFdSc1BqWFY3?=
 =?utf-8?B?T0FpSnQreDhoQ1hDTlhhUnJndXBJWTEvSkxkRzJlaE5DWVgyUDZJWjlBQUNq?=
 =?utf-8?B?MTFEQjc0RU5tODloRUNXeFlZZ1JZZ0syamtIbDBlNHU0ZWxWOTM5TkpOT2ZX?=
 =?utf-8?B?YkErTjZQQWZzYUV6SkhOMnRwcSs2U1d1R1dhTk8wY2VOd05NbndvSm5ESzQy?=
 =?utf-8?B?VXZqZHBzQ3NndzRXbTQ1bVRsZDJtNXZYS0JuWGNCdERDbytOWUVvVzFZK08v?=
 =?utf-8?B?VTZwaDRteGhBOHY1UFVseE43eDZWbzVKcXJjUUQvSUpNZ0YrSWhaWDR4ZVpT?=
 =?utf-8?B?RkY3RlFCUjlVdjdLUGFFNWVKRDhsR0w1UFJGcTNJZzk4N2FOWGV1Q1ZzL2JC?=
 =?utf-8?B?ckllS25ROGdqTDdsWkZqdzN6TTd1MmNnN0RDc04wWU5KVUpPT01DUFhybitG?=
 =?utf-8?B?TWR6bURET3ZzSlN6cFBLYTJmQWM3bitPZ2FkT2tOTEJnTklLT2U4WWdNNmV0?=
 =?utf-8?B?b3p4a1Z5STNkNlc3YUNuUTVMaE1YWGI4YjhQRDUrazNQeHJDS1g4bDl3dklk?=
 =?utf-8?B?SkJ0bUc0Wktta1Fad2ZRVkRNV1JVN2JYZVl2d0NLbmFQVTNMNUtBZXlWaUs2?=
 =?utf-8?B?Z1VtVG95QlBLMkQ1MVpqUWxCdXdtR2syS0dVdnF0MjdjY2dJTUJCT0l0Q3ZJ?=
 =?utf-8?B?bzZOUnJnTWRqenlpVzdZSXY4OWVQRW9xeFhiUHlqVlBwaFBrUGMwd1owYWU4?=
 =?utf-8?B?MEVnSmxyVCtLcTBVL25pRHcxUTRMdmM1RFA4MHExdGpQSVFURmhpc0xOY3Rh?=
 =?utf-8?B?WmJ0NHNSTlpuZGdnWm1SVGEzbGRiV3Z4SU8vN1VHb1FER21kZ2x5bkV2YnVl?=
 =?utf-8?B?TmtvSm5XOHFZc1d6a1phV2NKTmczWCtrNWRBT0FQd0tQYXdMUXNDWlV0b2Vi?=
 =?utf-8?B?YmRUY3NLRk5MQXdKV2tFWXZBU2JXZWs0bHA2QSswVXNmYkZVL0ZpcFBEbTZl?=
 =?utf-8?B?NUs4VTdsenhRVncxSTNhMkhVa1c4dTdRblhHUEhCUTN1Mk9GTGtST2phOUlP?=
 =?utf-8?B?SlVzbGtBTWwzMW9HdUFJdzIrUmJ5K1Ezd3U0Rjc1aDUxYi91eERBMCs5SGta?=
 =?utf-8?B?YVZ5WDloOEs5SFdIKzNiMnFFSG9sTUFyVE1Ud1RjOVc1WXRZMFkyRmFCd2tt?=
 =?utf-8?B?dUd2YVE0M2ZqUXhhRy9obkdOZ2hSbXZhMFZNRVhKZ3lQaDQ1RUk1Y1I4WGV3?=
 =?utf-8?B?eXI0QU4wUkRLZXMxeXhRWW1YTmZFS0dRM25NMUpWa2FTdjRDNXRTeThwUVpw?=
 =?utf-8?B?dnlwbGx4MHNEczhtNHZrbTJRZnFPcjlBMXlSakE1cHFqTGZwTDJQSWRTcUtt?=
 =?utf-8?Q?fOW478VM6qY5eO1FIm04AOT2dYkgv0+/rAg1fNE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4fa0e4f4-e39b-442c-f827-08d96d3e9588
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 11:49:41.5226
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lHMNK6NdFOd2YA85XwjNKIBtMBOILlhob2fYU5wmHUxbIsblrzafj2ySPylL/k1FtwPEwaRV1dufV+uUl/y/ApknV9uhfHEch3gi9sUNuIQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5045
X-OriginatorOrg: citrix.com

On 31/08/2021 14:36, Jan Beulich wrote:
> On 31.08.2021 15:27, Andrew Cooper wrote:
>> On 31/08/2021 14:19, Jan Beulich wrote:
>>>>>> @@ -1095,6 +1101,17 @@ static int __init pvh_setup_acpi(struct
>>>>>>          nr_pages =3D PFN_UP((d->arch.e820[i].addr & ~PAGE_MASK) +
>>>>>>                            d->arch.e820[i].size);
>>>>>> =20
>>>>>> +        /* Memory below 1MB has been dealt with by pvh_populate_p2m=
(). */
>>>>>> +        if ( pfn < PFN_DOWN(MB(1)) )
>>>>>> +        {
>>>>>> +            if ( pfn + nr_pages <=3D PFN_DOWN(MB(1)) )
>>>>>> +                continue;
>>>>>> +
>>>>>> +            /* This shouldn't happen, but is easy to deal with. */
>>>>> I'm not sure this comment is helpful.
>>>>>
>>>>> Under PVH, there is nothing special about the 1M boundary, and we can
>>>>> reasonably have something else here or crossing the boundary.
>>>> As long as we have this special treatment of the low Mb, the boundary
>>>> is meaningful imo. I'd see the comment go away when the handling in
>>>> general gets streamlined.
>>> I should have added: And as long as Dom0's E820 map gets cloned from
>>> the host's, which will necessarily consider the 1Mb boundary special.
>> Not when you're booting virtualised in the first place.
> You mean when Xen itself runs in PVH (not HVM) mode, and then in turn
> has a PVH Dom0? And if the underlying Xen has not in turn cloned
> the E820 from the host's? That's surely too exotic a case to warrant
> removing this comment. If you insist, I can mention that case as a
> possible exception.

It's a long way from exotic.

Also the magic surrounding the 1M boundary is disappearing on real
hardware with legacy BIOS support being dropped from platforms.


The comment is misleading and should be dropped.=C2=A0 The logic is still
perfectly clear given the outer comment.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 12:27:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 12:27:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176337.320878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPKZ-00031d-7L; Wed, 01 Sep 2021 12:27:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176337.320878; Wed, 01 Sep 2021 12:27:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPKZ-00031W-44; Wed, 01 Sep 2021 12:27:03 +0000
Received: by outflank-mailman (input) for mailman id 176337;
 Wed, 01 Sep 2021 12:27:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLPKX-00031M-Q0; Wed, 01 Sep 2021 12:27:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLPKX-0001j2-Le; Wed, 01 Sep 2021 12:27:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLPKX-0002HD-A4; Wed, 01 Sep 2021 12:27:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLPKX-0004SF-9W; Wed, 01 Sep 2021 12:27:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cehNK+/L30sbac4CFO5mJJ2ywt8vH27tObIs9axzprU=; b=NyMQjXNtnYgKgzDCAdnFH7cYPK
	agFkt85cURMY1ZKorjCaKMSH6GgY19xlQ/QgpgQr10YDdFQrZ1xD0tvMqbakRtFGOhDgex8YQKSCv
	k5PFfQ+qOf5YIzKbHYtP0EKr7Z04/ybjV51rfwBenZdIgrN7JOEX4RW5TbRdpdVuu/dk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164695-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164695: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b170806518c1e414939c8b085866544814e1ce8e
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 12:27:01 +0000

flight 164695 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164695/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b170806518c1e414939c8b085866544814e1ce8e
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    2 attempts
Testing same since   164695  2021-09-01 10:40:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 12:29:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 12:29:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176344.320893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPMw-0003fy-NF; Wed, 01 Sep 2021 12:29:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176344.320893; Wed, 01 Sep 2021 12:29:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPMw-0003fr-Jd; Wed, 01 Sep 2021 12:29:30 +0000
Received: by outflank-mailman (input) for mailman id 176344;
 Wed, 01 Sep 2021 12:29:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ymi3=NX=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mLPMv-0003fj-0W
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 12:29:29 +0000
Received: from mail-pg1-x52f.google.com (unknown [2607:f8b0:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 19227e72-2afa-4c5a-983b-279d7332980d;
 Wed, 01 Sep 2021 12:29:20 +0000 (UTC)
Received: by mail-pg1-x52f.google.com with SMTP id s11so2574023pgr.11
 for <xen-devel@lists.xen.org>; Wed, 01 Sep 2021 05:29:20 -0700 (PDT)
Received: from laputa (p784a44f4.tkyea130.ap.so-net.ne.jp. [120.74.68.244])
 by smtp.gmail.com with ESMTPSA id 11sm24800891pgh.52.2021.09.01.05.29.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 Sep 2021 05:29:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19227e72-2afa-4c5a-983b-279d7332980d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:content-transfer-encoding:in-reply-to;
        bh=07U2ss57vpxKYbvMqehbTSa/mQqWhg8J3EKt6sEzu1Q=;
        b=tGEGEJQN/y8wkh6wE9Tpje/mLcwS1Cpy3s5pLdntikNHnVh1IemOBAjnAl85CfDjFa
         vlDDXh8UrmfY3Yla0qa5wbTMrogk5Qz/GyWKc0Rs7GSer2zm7XPzP7SDEpKONpSqiIn4
         YNcsHJYcn73N59BmxcDuI+LK1SWBy4/F/5A9trw2x3zr5IYxbHP6vIv5KuSDw8gnUpmt
         C9ev6p1PS/WW3/B7WlqzFMgVn5tLXsJOpyc6Sdatl38qu8Gdzkc9pphWii6WARPgBkt3
         N+Wuy47sNSORxdFKe/H/AZ3wu2PCmxIGpozp0tB2kQJdprr9hEEkpSLgVTSdcKk06T0W
         Ps2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:content-transfer-encoding
         :in-reply-to;
        bh=07U2ss57vpxKYbvMqehbTSa/mQqWhg8J3EKt6sEzu1Q=;
        b=GJNahvGXOQaslOFZ/EaUdT3rEvxtJk8a1uw/GJ+0+qH3Doe0qOWAsJedSFMRwGE+h3
         m5keSVcPY125/m7kR0ubVrZts723q7irnFuSOZ1i7v7g0lGYPnxZpGczO/j8Pwl03e2I
         tPvNscI/50RDMbRdaCS9wJCoQYVS28UMSzYO0jbdYeeoiMM6XY1bNAJLZFjld13TICPJ
         1tBbGvmyzb8P8GHL9nMrV3ykEv2cSoEOsczmccqilosGi5H4+pZP7qJSYH1Zxcwmfgt6
         ELhzpvWqiU5wgi/oz/wvkcqJZ2pDSYRsbh1c8TsTg9zOQ6nh24JHssCH09tq9zrV61ox
         LZkw==
X-Gm-Message-State: AOAM533cBkfOaAc31PqFqiqrVYwpARliBtRKn/hWFXLL5+YN9sZJ0S92
	oPFKL/qUpYMq+lCjYgwrJTttlQ==
X-Google-Smtp-Source: ABdhPJx7LVfGA76Fa52njG3Bl8xTL8wryQTg0FB02cmdtaD+7SpKet6xkUpwxHIe7GgFKBfahnN+0g==
X-Received: by 2002:a63:e04a:: with SMTP id n10mr32324146pgj.381.1630499359569;
        Wed, 01 Sep 2021 05:29:19 -0700 (PDT)
Date: Wed, 1 Sep 2021 21:29:10 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Wei Chen <Wei.Chen@arm.com>
Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin <Kaly.Xin@arm.com>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	"stefanha@redhat.com" <stefanha@redhat.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>,
	"pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
	Paul Durrant <paul@xen.org>, nd <nd@arm.com>,
	Xen Devel <xen-devel@lists.xen.org>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210901122910.GA100270@laputa>
References: <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa>
 <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa>
 <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210831061805.GB69817@laputa>
 <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>

Hi Wei,

On Wed, Sep 01, 2021 at 11:12:58AM +0000, Wei Chen wrote:
> Hi Akashi,
> 
> > -----Original Message-----
> > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > Sent: 2021年8月31日 14:18
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano Stabellini
> > <sstabellini@kernel.org>; Alex Benn??e <alex.bennee@linaro.org>; Kaly Xin
> > <Kaly.Xin@arm.com>; Stratos Mailing List <stratos-dev@op-lists.linaro.org>;
> > virtio-dev@lists.oasis-open.org; Arnd Bergmann <arnd.bergmann@linaro.org>;
> > Viresh Kumar <viresh.kumar@linaro.org>; Stefano Stabellini
> > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan Kiszka
> > <jan.kiszka@siemens.com>; Carl van Schaik <cvanscha@qti.qualcomm.com>;
> > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org>; Jean-
> > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>; Artem Mygaiev <Artem_Mygaiev@epam.com>; Julien
> > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul Durrant
> > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> > Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
> > 
> > Wei,
> > 
> > On Thu, Aug 26, 2021 at 12:10:19PM +0000, Wei Chen wrote:
> > > Hi Akashi,
> > >
> > > > -----Original Message-----
> > > > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > > Sent: 2021年8月26日 17:41
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano Stabellini
> > > > <sstabellini@kernel.org>; Alex Benn??e <alex.bennee@linaro.org>; Kaly
> > Xin
> > > > <Kaly.Xin@arm.com>; Stratos Mailing List <stratos-dev@op-
> > lists.linaro.org>;
> > > > virtio-dev@lists.oasis-open.org; Arnd Bergmann
> > <arnd.bergmann@linaro.org>;
> > > > Viresh Kumar <viresh.kumar@linaro.org>; Stefano Stabellini
> > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan Kiszka
> > > > <jan.kiszka@siemens.com>; Carl van Schaik <cvanscha@qti.qualcomm.com>;
> > > > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org>; Jean-
> > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> > > > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> > > > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev <Artem_Mygaiev@epam.com>;
> > Julien
> > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul Durrant
> > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
> > > >
> > > > Hi Wei,
> > > >
> > > > On Fri, Aug 20, 2021 at 03:41:50PM +0900, AKASHI Takahiro wrote:
> > > > > On Wed, Aug 18, 2021 at 08:35:51AM +0000, Wei Chen wrote:
> > > > > > Hi Akashi，
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > > > > > Sent: 2021年8月18日 13:39
> > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano
> > Stabellini
> > > > > > > <sstabellini@kernel.org>; Alex Benn??e <alex.bennee@linaro.org>;
> > > > Stratos
> > > > > > > Mailing List <stratos-dev@op-lists.linaro.org>; virtio-
> > > > dev@lists.oasis-
> > > > > > > open.org; Arnd Bergmann <arnd.bergmann@linaro.org>; Viresh Kumar
> > > > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> > > > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan Kiszka
> > > > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> > > > <cvanscha@qti.qualcomm.com>;
> > > > > > > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org>;
> > > > Jean-
> > > > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> > > > > > > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> > > > > > > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> > > > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev
> > <Artem_Mygaiev@epam.com>;
> > > > Julien
> > > > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul
> > > > Durrant
> > > > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> > > > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
> > > > > > >
> > > > > > > On Tue, Aug 17, 2021 at 08:39:09AM +0000, Wei Chen wrote:
> > > > > > > > Hi Akashi,
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > > > > > > > Sent: 2021年8月17日 16:08
> > > > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > > > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano
> > > > Stabellini
> > > > > > > > > <sstabellini@kernel.org>; Alex Benn??e
> > <alex.bennee@linaro.org>;
> > > > > > > Stratos
> > > > > > > > > Mailing List <stratos-dev@op-lists.linaro.org>; virtio-
> > > > > > > dev@lists.oasis-
> > > > > > > > > open.org; Arnd Bergmann <arnd.bergmann@linaro.org>; Viresh
> > Kumar
> > > > > > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> > > > > > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan
> > Kiszka
> > > > > > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> > > > <cvanscha@qti.qualcomm.com>;
> > > > > > > > > pratikp@quicinc.com; Srivatsa Vaddagiri
> > <vatsa@codeaurora.org>;
> > > > Jean-
> > > > > > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> > > > > > > > > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> > > > > > > > > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> > > > > > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev
> > > > <Artem_Mygaiev@epam.com>;
> > > > > > > Julien
> > > > > > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>;
> > Paul
> > > > Durrant
> > > > > > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> > > > > > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO
> > backends
> > > > > > > > >
> > > > > > > > > Hi Wei, Oleksandr,
> > > > > > > > >
> > > > > > > > > On Mon, Aug 16, 2021 at 10:04:03AM +0000, Wei Chen wrote:
> > > > > > > > > > Hi All,
> > > > > > > > > >
> > > > > > > > > > Thanks for Stefano to link my kvmtool for Xen proposal
> > here.
> > > > > > > > > > This proposal is still discussing in Xen and KVM
> > communities.
> > > > > > > > > > The main work is to decouple the kvmtool from KVM and make
> > > > > > > > > > other hypervisors can reuse the virtual device
> > implementations.
> > > > > > > > > >
> > > > > > > > > > In this case, we need to introduce an intermediate
> > hypervisor
> > > > > > > > > > layer for VMM abstraction, Which is, I think it's very
> > close
> > > > > > > > > > to stratos' virtio hypervisor agnosticism work.
> > > > > > > > >
> > > > > > > > > # My proposal[1] comes from my own idea and doesn't always
> > > > represent
> > > > > > > > > # Linaro's view on this subject nor reflect Alex's concerns.
> > > > > > > Nevertheless,
> > > > > > > > >
> > > > > > > > > Your idea and my proposal seem to share the same background.
> > > > > > > > > Both have the similar goal and currently start with, at
> > first,
> > > > Xen
> > > > > > > > > and are based on kvm-tool. (Actually, my work is derived
> > from
> > > > > > > > > EPAM's virtio-disk, which is also based on kvm-tool.)
> > > > > > > > >
> > > > > > > > > In particular, the abstraction of hypervisor interfaces has
> > a
> > > > same
> > > > > > > > > set of interfaces (for your "struct vmm_impl" and my "RPC
> > > > interfaces").
> > > > > > > > > This is not co-incident as we both share the same origin as
> > I
> > > > said
> > > > > > > above.
> > > > > > > > > And so we will also share the same issues. One of them is a
> > way
> > > > of
> > > > > > > > > "sharing/mapping FE's memory". There is some trade-off
> > between
> > > > > > > > > the portability and the performance impact.
> > > > > > > > > So we can discuss the topic here in this ML, too.
> > > > > > > > > (See Alex's original email, too).
> > > > > > > > >
> > > > > > > > Yes, I agree.
> > > > > > > >
> > > > > > > > > On the other hand, my approach aims to create a "single-
> > binary"
> > > > > > > solution
> > > > > > > > > in which the same binary of BE vm could run on any
> > hypervisors.
> > > > > > > > > Somehow similar to your "proposal-#2" in [2], but in my
> > solution,
> > > > all
> > > > > > > > > the hypervisor-specific code would be put into another
> > entity
> > > > (VM),
> > > > > > > > > named "virtio-proxy" and the abstracted operations are
> > served
> > > > via RPC.
> > > > > > > > > (In this sense, BE is hypervisor-agnostic but might have OS
> > > > > > > dependency.)
> > > > > > > > > But I know that we need discuss if this is a requirement
> > even
> > > > > > > > > in Stratos project or not. (Maybe not)
> > > > > > > > >
> > > > > > > >
> > > > > > > > Sorry, I haven't had time to finish reading your virtio-proxy
> > > > completely
> > > > > > > > (I will do it ASAP). But from your description, it seems we
> > need a
> > > > > > > > 3rd VM between FE and BE? My concern is that, if my assumption
> > is
> > > > right,
> > > > > > > > will it increase the latency in data transport path? Even if
> > we're
> > > > > > > > using some lightweight guest like RTOS or Unikernel,
> > > > > > >
> > > > > > > Yes, you're right. But I'm afraid that it is a matter of degree.
> > > > > > > As far as we execute 'mapping' operations at every fetch of
> > payload,
> > > > > > > we will see latency issue (even in your case) and if we have
> > some
> > > > solution
> > > > > > > for it, we won't see it neither in my proposal :)
> > > > > > >
> > > > > >
> > > > > > Oleksandr has sent a proposal to Xen mailing list to reduce this
> > kind
> > > > > > of "mapping/unmapping" operations. So the latency caused by this
> > > > behavior
> > > > > > on Xen may eventually be eliminated, and Linux-KVM doesn't have
> > that
> > > > problem.
> > > > >
> > > > > Obviously, I have not yet caught up there in the discussion.
> > > > > Which patch specifically?
> > > >
> > > > Can you give me the link to the discussion or patch, please?
> > > >
> > >
> > > It's a RFC discussion. We have tested this RFC patch internally.
> > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > 07/msg01532.html
> > 
> > I'm afraid that I miss something here, but I don't know
> > why this proposed API will lead to eliminating 'mmap' in accessing
> > the queued payload at every request?
> > 
> 
> This API give Xen device model (QEMU or kvmtool) the ability to map
> whole guest RAM in device model's address space. In this case, device
> model doesn't need dynamic hypercall to map/unmap payload memory.
> It can use a flat offset to access payload memory in its address
> space directly. Just Like KVM device model does now.

Thank you. Quickly, let me make sure one thing:
This API itself doesn't do any mapping operations, right?
So I suppose that virtio BE guest is responsible to
1) fetch the information about all the memory regions in FE,
2) call this API to allocate a big chunk of unused space in BE,
3) create grant/foreign mappings for FE onto this region(S)
in the initialization/configuration of emulated virtio devices.

Is this the way this API is expected to be used?
Does Xen already has an interface for (1)?

-Takahiro Akashi

> Before this API, When device model to map whole guest memory, will
> severely consume the physical pages of Dom-0/Dom-D.
> 
> > -Takahiro Akashi
> > 
> > 
> > > > Thanks,
> > > > -Takahiro Akashi
> > > >
> > > > > -Takahiro Akashi
> > > > >
> > > > > > > > > Specifically speaking about kvm-tool, I have a concern about
> > its
> > > > > > > > > license term; Targeting different hypervisors and different
> > OSs
> > > > > > > > > (which I assume includes RTOS's), the resultant library
> > should
> > > > be
> > > > > > > > > license permissive and GPL for kvm-tool might be an issue.
> > > > > > > > > Any thoughts?
> > > > > > > > >
> > > > > > > >
> > > > > > > > Yes. If user want to implement a FreeBSD device model, but the
> > > > virtio
> > > > > > > > library is GPL. Then GPL would be a problem. If we have
> > another
> > > > good
> > > > > > > > candidate, I am open to it.
> > > > > > >
> > > > > > > I have some candidates, particularly for vq/vring, in my mind:
> > > > > > > * Open-AMP, or
> > > > > > > * corresponding Free-BSD code
> > > > > > >
> > > > > >
> > > > > > Interesting, I will look into them : )
> > > > > >
> > > > > > Cheers,
> > > > > > Wei Chen
> > > > > >
> > > > > > > -Takahiro Akashi
> > > > > > >
> > > > > > >
> > > > > > > > > -Takahiro Akashi
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > [1] https://op-lists.linaro.org/pipermail/stratos-dev/2021-
> > > > > > > > > August/000548.html
> > > > > > > > > [2] https://marc.info/?l=xen-devel&m=162373754705233&w=2
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> > > > > > > > > > > Sent: 2021年8月14日 23:38
> > > > > > > > > > > To: AKASHI Takahiro <takahiro.akashi@linaro.org>;
> > Stefano
> > > > > > > Stabellini
> > > > > > > > > <sstabellini@kernel.org>
> > > > > > > > > > > Cc: Alex Benn??e <alex.bennee@linaro.org>; Stratos
> > Mailing
> > > > List
> > > > > > > > > <stratos-dev@op-lists.linaro.org>; virtio-dev@lists.oasis-
> > > > open.org;
> > > > > > > Arnd
> > > > > > > > > Bergmann <arnd.bergmann@linaro.org>; Viresh Kumar
> > > > > > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> > > > > > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan
> > Kiszka
> > > > > > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> > > > <cvanscha@qti.qualcomm.com>;
> > > > > > > > > pratikp@quicinc.com; Srivatsa Vaddagiri
> > <vatsa@codeaurora.org>;
> > > > Jean-
> > > > > > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> > > > > > > > > <mathieu.poirier@linaro.org>; Wei Chen <Wei.Chen@arm.com>;
> > > > Oleksandr
> > > > > > > > > Tyshchenko <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> > > > > > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev
> > > > <Artem_Mygaiev@epam.com>;
> > > > > > > Julien
> > > > > > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>;
> > Paul
> > > > Durrant
> > > > > > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> > > > > > > > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO
> > > > backends
> > > > > > > > > > >
> > > > > > > > > > > Hello, all.
> > > > > > > > > > >
> > > > > > > > > > > Please see some comments below. And sorry for the
> > possible
> > > > format
> > > > > > > > > issues.
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, Aug 11, 2021 at 9:27 AM AKASHI Takahiro
> > > > > > > > > <mailto:takahiro.akashi@linaro.org> wrote:
> > > > > > > > > > > > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano
> > > > Stabellini
> > > > > > > wrote:
> > > > > > > > > > > > > CCing people working on Xen+VirtIO and IOREQs. Not
> > > > trimming
> > > > > > > the
> > > > > > > > > original
> > > > > > > > > > > > > email to let them read the full context.
> > > > > > > > > > > > >
> > > > > > > > > > > > > My comments below are related to a potential Xen
> > > > > > > implementation,
> > > > > > > > > not
> > > > > > > > > > > > > because it is the only implementation that matters,
> > but
> > > > > > > because it
> > > > > > > > > is
> > > > > > > > > > > > > the one I know best.
> > > > > > > > > > > >
> > > > > > > > > > > > Please note that my proposal (and hence the working
> > > > prototype)[1]
> > > > > > > > > > > > is based on Xen's virtio implementation (i.e. IOREQ)
> > and
> > > > > > > > > particularly
> > > > > > > > > > > > EPAM's virtio-disk application (backend server).
> > > > > > > > > > > > It has been, I believe, well generalized but is still
> > a
> > > > bit
> > > > > > > biased
> > > > > > > > > > > > toward this original design.
> > > > > > > > > > > >
> > > > > > > > > > > > So I hope you like my approach :)
> > > > > > > > > > > >
> > > > > > > > > > > > [1] https://op-lists.linaro.org/pipermail/stratos-
> > > > dev/2021-
> > > > > > > > > August/000546.html
> > > > > > > > > > > >
> > > > > > > > > > > > Let me take this opportunity to explain a bit more
> > about
> > > > my
> > > > > > > approach
> > > > > > > > > below.
> > > > > > > > > > > >
> > > > > > > > > > > > > Also, please see this relevant email thread:
> > > > > > > > > > > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 4 Aug 2021, Alex Bennée wrote:
> > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > One of the goals of Project Stratos is to enable
> > > > hypervisor
> > > > > > > > > agnostic
> > > > > > > > > > > > > > backends so we can enable as much re-use of code
> > as
> > > > possible
> > > > > > > and
> > > > > > > > > avoid
> > > > > > > > > > > > > > repeating ourselves. This is the flip side of the
> > > > front end
> > > > > > > > > where
> > > > > > > > > > > > > > multiple front-end implementations are required -
> > one
> > > > per OS,
> > > > > > > > > assuming
> > > > > > > > > > > > > > you don't just want Linux guests. The resultant
> > guests
> > > > are
> > > > > > > > > trivially
> > > > > > > > > > > > > > movable between hypervisors modulo any abstracted
> > > > paravirt
> > > > > > > type
> > > > > > > > > > > > > > interfaces.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In my original thumb nail sketch of a solution I
> > > > envisioned
> > > > > > > > > vhost-user
> > > > > > > > > > > > > > daemons running in a broadly POSIX like
> > environment.
> > > > The
> > > > > > > > > interface to
> > > > > > > > > > > > > > the daemon is fairly simple requiring only some
> > mapped
> > > > > > > memory
> > > > > > > > > and some
> > > > > > > > > > > > > > sort of signalling for events (on Linux this is
> > > > eventfd).
> > > > > > > The
> > > > > > > > > idea was a
> > > > > > > > > > > > > > stub binary would be responsible for any
> > hypervisor
> > > > specific
> > > > > > > > > setup and
> > > > > > > > > > > > > > then launch a common binary to deal with the
> > actual
> > > > > > > virtqueue
> > > > > > > > > requests
> > > > > > > > > > > > > > themselves.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Since that original sketch we've seen an expansion
> > in
> > > > the
> > > > > > > sort
> > > > > > > > > of ways
> > > > > > > > > > > > > > backends could be created. There is interest in
> > > > > > > encapsulating
> > > > > > > > > backends
> > > > > > > > > > > > > > in RTOSes or unikernels for solutions like SCMI.
> > There
> > > > > > > interest
> > > > > > > > > in Rust
> > > > > > > > > > > > > > has prompted ideas of using the trait interface to
> > > > abstract
> > > > > > > > > differences
> > > > > > > > > > > > > > away as well as the idea of bare-metal Rust
> > backends.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We have a card (STR-12) called "Hypercall
> > > > Standardisation"
> > > > > > > which
> > > > > > > > > > > > > > calls for a description of the APIs needed from
> > the
> > > > > > > hypervisor
> > > > > > > > > side to
> > > > > > > > > > > > > > support VirtIO guests and their backends. However
> > we
> > > > are
> > > > > > > some
> > > > > > > > > way off
> > > > > > > > > > > > > > from that at the moment as I think we need to at
> > least
> > > > > > > > > demonstrate one
> > > > > > > > > > > > > > portable backend before we start codifying
> > > > requirements. To
> > > > > > > that
> > > > > > > > > end I
> > > > > > > > > > > > > > want to think about what we need for a backend to
> > > > function.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Configuration
> > > > > > > > > > > > > > =============
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In the type-2 setup this is typically fairly
> > simple
> > > > because
> > > > > > > the
> > > > > > > > > host
> > > > > > > > > > > > > > system can orchestrate the various modules that
> > make
> > > > up the
> > > > > > > > > complete
> > > > > > > > > > > > > > system. In the type-1 case (or even type-2 with
> > > > delegated
> > > > > > > > > service VMs)
> > > > > > > > > > > > > > we need some sort of mechanism to inform the
> > backend
> > > > VM
> > > > > > > about
> > > > > > > > > key
> > > > > > > > > > > > > > details about the system:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >   - where virt queue memory is in it's address
> > space
> > > > > > > > > > > > > >   - how it's going to receive (interrupt) and
> > trigger
> > > > (kick)
> > > > > > > > > events
> > > > > > > > > > > > > >   - what (if any) resources the backend needs to
> > > > connect to
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Obviously you can elide over configuration issues
> > by
> > > > having
> > > > > > > > > static
> > > > > > > > > > > > > > configurations and baking the assumptions into
> > your
> > > > guest
> > > > > > > images
> > > > > > > > > however
> > > > > > > > > > > > > > this isn't scalable in the long term. The obvious
> > > > solution
> > > > > > > seems
> > > > > > > > > to be
> > > > > > > > > > > > > > extending a subset of Device Tree data to user
> > space
> > > > but
> > > > > > > perhaps
> > > > > > > > > there
> > > > > > > > > > > > > > are other approaches?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Before any virtio transactions can take place the
> > > > > > > appropriate
> > > > > > > > > memory
> > > > > > > > > > > > > > mappings need to be made between the FE guest and
> > the
> > > > BE
> > > > > > > guest.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Currently the whole of the FE guests address space
> > > > needs to
> > > > > > > be
> > > > > > > > > visible
> > > > > > > > > > > > > > to whatever is serving the virtio requests. I can
> > > > envision 3
> > > > > > > > > approaches:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >  * BE guest boots with memory already mapped
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >  This would entail the guest OS knowing where in
> > it's
> > > > Guest
> > > > > > > > > Physical
> > > > > > > > > > > > > >  Address space is already taken up and avoiding
> > > > clashing. I
> > > > > > > > > would assume
> > > > > > > > > > > > > >  in this case you would want a standard interface
> > to
> > > > > > > userspace
> > > > > > > > > to then
> > > > > > > > > > > > > >  make that address space visible to the backend
> > daemon.
> > > > > > > > > > > >
> > > > > > > > > > > > Yet another way here is that we would have well known
> > > > "shared
> > > > > > > > > memory" between
> > > > > > > > > > > > VMs. I think that Jailhouse's ivshmem gives us good
> > > > insights on
> > > > > > > this
> > > > > > > > > matter
> > > > > > > > > > > > and that it can even be an alternative for hypervisor-
> > > > agnostic
> > > > > > > > > solution.
> > > > > > > > > > > >
> > > > > > > > > > > > (Please note memory regions in ivshmem appear as a PCI
> > > > device
> > > > > > > and
> > > > > > > > > can be
> > > > > > > > > > > > mapped locally.)
> > > > > > > > > > > >
> > > > > > > > > > > > I want to add this shared memory aspect to my virtio-
> > proxy,
> > > > but
> > > > > > > > > > > > the resultant solution would eventually look similar
> > to
> > > > ivshmem.
> > > > > > > > > > > >
> > > > > > > > > > > > > >  * BE guests boots with a hypervisor handle to
> > memory
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >  The BE guest is then free to map the FE's memory
> > to
> > > > where
> > > > > > > it
> > > > > > > > > wants in
> > > > > > > > > > > > > >  the BE's guest physical address space.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I cannot see how this could work for Xen. There is
> > no
> > > > "handle"
> > > > > > > to
> > > > > > > > > give
> > > > > > > > > > > > > to the backend if the backend is not running in dom0.
> > So
> > > > for
> > > > > > > Xen I
> > > > > > > > > think
> > > > > > > > > > > > > the memory has to be already mapped
> > > > > > > > > > > >
> > > > > > > > > > > > In Xen's IOREQ solution (virtio-blk), the following
> > > > information
> > > > > > > is
> > > > > > > > > expected
> > > > > > > > > > > > to be exposed to BE via Xenstore:
> > > > > > > > > > > > (I know that this is a tentative approach though.)
> > > > > > > > > > > >    - the start address of configuration space
> > > > > > > > > > > >    - interrupt number
> > > > > > > > > > > >    - file path for backing storage
> > > > > > > > > > > >    - read-only flag
> > > > > > > > > > > > And the BE server have to call a particular hypervisor
> > > > interface
> > > > > > > to
> > > > > > > > > > > > map the configuration space.
> > > > > > > > > > >
> > > > > > > > > > > Yes, Xenstore was chosen as a simple way to pass
> > > > configuration
> > > > > > > info to
> > > > > > > > > the backend running in a non-toolstack domain.
> > > > > > > > > > > I remember, there was a wish to avoid using Xenstore in
> > > > Virtio
> > > > > > > backend
> > > > > > > > > itself if possible, so for non-toolstack domain, this could
> > done
> > > > with
> > > > > > > > > adjusting devd (daemon that listens for devices and launches
> > > > backends)
> > > > > > > > > > > to read backend configuration from the Xenstore anyway
> > and
> > > > pass it
> > > > > > > to
> > > > > > > > > the backend via command line arguments.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Yes, in current PoC code we're using xenstore to pass
> > device
> > > > > > > > > configuration.
> > > > > > > > > > We also designed a static device configuration parse
> > method
> > > > for
> > > > > > > Dom0less
> > > > > > > > > or
> > > > > > > > > > other scenarios don't have xentool. yes, it's from device
> > > > model
> > > > > > > command
> > > > > > > > > line
> > > > > > > > > > or a config file.
> > > > > > > > > >
> > > > > > > > > > > But, if ...
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > In my approach (virtio-proxy), all those Xen (or
> > > > hypervisor)-
> > > > > > > > > specific
> > > > > > > > > > > > stuffs are contained in virtio-proxy, yet another VM,
> > to
> > > > hide
> > > > > > > all
> > > > > > > > > details.
> > > > > > > > > > >
> > > > > > > > > > > ... the solution how to overcome that is already found
> > and
> > > > proven
> > > > > > > to
> > > > > > > > > work then even better.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > # My point is that a "handle" is not mandatory for
> > > > executing
> > > > > > > mapping.
> > > > > > > > > > > >
> > > > > > > > > > > > > and the mapping probably done by the
> > > > > > > > > > > > > toolstack (also see below.) Or we would have to
> > invent a
> > > > new
> > > > > > > Xen
> > > > > > > > > > > > > hypervisor interface and Xen virtual machine
> > privileges
> > > > to
> > > > > > > allow
> > > > > > > > > this
> > > > > > > > > > > > > kind of mapping.
> > > > > > > > > > > >
> > > > > > > > > > > > > If we run the backend in Dom0 that we have no
> > problems
> > > > of
> > > > > > > course.
> > > > > > > > > > > >
> > > > > > > > > > > > One of difficulties on Xen that I found in my approach
> > is
> > > > that
> > > > > > > > > calling
> > > > > > > > > > > > such hypervisor intefaces (registering IOREQ, mapping
> > > > memory) is
> > > > > > > > > only
> > > > > > > > > > > > allowed on BE servers themselvies and so we will have
> > to
> > > > extend
> > > > > > > > > those
> > > > > > > > > > > > interfaces.
> > > > > > > > > > > > This, however, will raise some concern on security and
> > > > privilege
> > > > > > > > > distribution
> > > > > > > > > > > > as Stefan suggested.
> > > > > > > > > > >
> > > > > > > > > > > We also faced policy related issues with Virtio backend
> > > > running in
> > > > > > > > > other than Dom0 domain in a "dummy" xsm mode. In our target
> > > > system we
> > > > > > > run
> > > > > > > > > the backend in a driver
> > > > > > > > > > > domain (we call it DomD) where the underlying H/W
> > resides.
> > > > We
> > > > > > > trust it,
> > > > > > > > > so we wrote policy rules (to be used in "flask" xsm mode) to
> > > > provide
> > > > > > > it
> > > > > > > > > with a little bit more privileges than a simple DomU had.
> > > > > > > > > > > Now it is permitted to issue device-model, resource and
> > > > memory
> > > > > > > > > mappings, etc calls.
> > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > To activate the mapping will
> > > > > > > > > > > > > >  require some sort of hypercall to the hypervisor.
> > I
> > > > can see
> > > > > > > two
> > > > > > > > > options
> > > > > > > > > > > > > >  at this point:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >   - expose the handle to userspace for
> > daemon/helper
> > > > to
> > > > > > > trigger
> > > > > > > > > the
> > > > > > > > > > > > > >     mapping via existing hypercall interfaces. If
> > > > using a
> > > > > > > helper
> > > > > > > > > you
> > > > > > > > > > > > > >     would have a hypervisor specific one to avoid
> > the
> > > > daemon
> > > > > > > > > having to
> > > > > > > > > > > > > >     care too much about the details or push that
> > > > complexity
> > > > > > > into
> > > > > > > > > a
> > > > > > > > > > > > > >     compile time option for the daemon which would
> > > > result in
> > > > > > > > > different
> > > > > > > > > > > > > >     binaries although a common source base.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >   - expose a new kernel ABI to abstract the
> > hypercall
> > > > > > > > > differences away
> > > > > > > > > > > > > >     in the guest kernel. In this case the
> > userspace
> > > > would
> > > > > > > > > essentially
> > > > > > > > > > > > > >     ask for an abstract "map guest N memory to
> > > > userspace
> > > > > > > ptr"
> > > > > > > > > and let
> > > > > > > > > > > > > >     the kernel deal with the different hypercall
> > > > interfaces.
> > > > > > > > > This of
> > > > > > > > > > > > > >     course assumes the majority of BE guests would
> > be
> > > > Linux
> > > > > > > > > kernels and
> > > > > > > > > > > > > >     leaves the bare-metal/unikernel approaches to
> > > > their own
> > > > > > > > > devices.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Operation
> > > > > > > > > > > > > > =========
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The core of the operation of VirtIO is fairly
> > simple.
> > > > Once
> > > > > > > the
> > > > > > > > > > > > > > vhost-user feature negotiation is done it's a case
> > of
> > > > > > > receiving
> > > > > > > > > update
> > > > > > > > > > > > > > events and parsing the resultant virt queue for
> > data.
> > > > The
> > > > > > > vhost-
> > > > > > > > > user
> > > > > > > > > > > > > > specification handles a bunch of setup before that
> > > > point,
> > > > > > > mostly
> > > > > > > > > to
> > > > > > > > > > > > > > detail where the virt queues are set up FD's for
> > > > memory and
> > > > > > > > > event
> > > > > > > > > > > > > > communication. This is where the envisioned stub
> > > > process
> > > > > > > would
> > > > > > > > > be
> > > > > > > > > > > > > > responsible for getting the daemon up and ready to
> > run.
> > > > This
> > > > > > > is
> > > > > > > > > > > > > > currently done inside a big VMM like QEMU but I
> > > > suspect a
> > > > > > > modern
> > > > > > > > > > > > > > approach would be to use the rust-vmm vhost crate.
> > It
> > > > would
> > > > > > > then
> > > > > > > > > either
> > > > > > > > > > > > > > communicate with the kernel's abstracted ABI or be
> > re-
> > > > > > > targeted
> > > > > > > > > as a
> > > > > > > > > > > > > > build option for the various hypervisors.
> > > > > > > > > > > > >
> > > > > > > > > > > > > One thing I mentioned before to Alex is that Xen
> > doesn't
> > > > have
> > > > > > > VMMs
> > > > > > > > > the
> > > > > > > > > > > > > way they are typically envisioned and described in
> > other
> > > > > > > > > environments.
> > > > > > > > > > > > > Instead, Xen has IOREQ servers. Each of them
> > connects
> > > > > > > > > independently to
> > > > > > > > > > > > > Xen via the IOREQ interface. E.g. today multiple
> > QEMUs
> > > > could
> > > > > > > be
> > > > > > > > > used as
> > > > > > > > > > > > > emulators for a single Xen VM, each of them
> > connecting
> > > > to Xen
> > > > > > > > > > > > > independently via the IOREQ interface.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The component responsible for starting a daemon
> > and/or
> > > > setting
> > > > > > > up
> > > > > > > > > shared
> > > > > > > > > > > > > interfaces is the toolstack: the xl command and the
> > > > > > > libxl/libxc
> > > > > > > > > > > > > libraries.
> > > > > > > > > > > >
> > > > > > > > > > > > I think that VM configuration management (or
> > orchestration
> > > > in
> > > > > > > > > Startos
> > > > > > > > > > > > jargon?) is a subject to debate in parallel.
> > > > > > > > > > > > Otherwise, is there any good assumption to avoid it
> > right
> > > > now?
> > > > > > > > > > > >
> > > > > > > > > > > > > Oleksandr and others I CCed have been working on
> > ways
> > > > for the
> > > > > > > > > toolstack
> > > > > > > > > > > > > to create virtio backends and setup memory mappings.
> > > > They
> > > > > > > might be
> > > > > > > > > able
> > > > > > > > > > > > > to provide more info on the subject. I do think we
> > miss
> > > > a way
> > > > > > > to
> > > > > > > > > provide
> > > > > > > > > > > > > the configuration to the backend and anything else
> > that
> > > > the
> > > > > > > > > backend
> > > > > > > > > > > > > might require to start doing its job.
> > > > > > > > > > >
> > > > > > > > > > > Yes, some work has been done for the toolstack to handle
> > > > Virtio
> > > > > > > MMIO
> > > > > > > > > devices in
> > > > > > > > > > > general and Virtio block devices in particular. However,
> > it
> > > > has
> > > > > > > not
> > > > > > > > > been upstreaned yet.
> > > > > > > > > > > Updated patches on review now:
> > > > > > > > > > > https://lore.kernel.org/xen-devel/1621626361-29076-1-
> > git-
> > > > send-
> > > > > > > email-
> > > > > > > > > olekstysh@gmail.com/
> > > > > > > > > > >
> > > > > > > > > > > There is an additional (also important) activity to
> > > > improve/fix
> > > > > > > > > foreign memory mapping on Arm which I am also involved in.
> > > > > > > > > > > The foreign memory mapping is proposed to be used for
> > Virtio
> > > > > > > backends
> > > > > > > > > (device emulators) if there is a need to run guest OS
> > completely
> > > > > > > > > unmodified.
> > > > > > > > > > > Of course, the more secure way would be to use grant
> > memory
> > > > > > > mapping.
> > > > > > > > > Brietly, the main difference between them is that with
> > foreign
> > > > mapping
> > > > > > > the
> > > > > > > > > backend
> > > > > > > > > > > can map any guest memory it wants to map, but with grant
> > > > mapping
> > > > > > > it is
> > > > > > > > > allowed to map only what was previously granted by the
> > frontend.
> > > > > > > > > > >
> > > > > > > > > > > So, there might be a problem if we want to pre-map some
> > > > guest
> > > > > > > memory
> > > > > > > > > in advance or to cache mappings in the backend in order to
> > > > improve
> > > > > > > > > performance (because the mapping/unmapping guest pages every
> > > > request
> > > > > > > > > requires a lot of back and forth to Xen + P2M updates). In a
> > > > nutshell,
> > > > > > > > > currently, in order to map a guest page into the backend
> > address
> > > > space
> > > > > > > we
> > > > > > > > > need to steal a real physical page from the backend domain.
> > So,
> > > > with
> > > > > > > the
> > > > > > > > > said optimizations we might end up with no free memory in
> > the
> > > > backend
> > > > > > > > > domain (see XSA-300). And what we try to achieve is to not
> > waste
> > > > a
> > > > > > > real
> > > > > > > > > domain memory at all by providing safe non-allocated-yet (so
> > > > unused)
> > > > > > > > > address space for the foreign (and grant) pages to be mapped
> > > > into,
> > > > > > > this
> > > > > > > > > enabling work implies Xen and Linux (and likely DTB bindings)
> > > > changes.
> > > > > > > > > However, as it turned out, for this to work in a proper and
> > safe
> > > > way
> > > > > > > some
> > > > > > > > > prereq work needs to be done.
> > > > > > > > > > > You can find the related Xen discussion at:
> > > > > > > > > > > https://lore.kernel.org/xen-devel/1627489110-25633-1-
> > git-
> > > > send-
> > > > > > > email-
> > > > > > > > > olekstysh@gmail.com/
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > One question is how to best handle notification
> > and
> > > > kicks.
> > > > > > > The
> > > > > > > > > existing
> > > > > > > > > > > > > > vhost-user framework uses eventfd to signal the
> > daemon
> > > > > > > (although
> > > > > > > > > QEMU
> > > > > > > > > > > > > > is quite capable of simulating them when you use
> > TCG).
> > > > Xen
> > > > > > > has
> > > > > > > > > it's own
> > > > > > > > > > > > > > IOREQ mechanism. However latency is an important
> > > > factor and
> > > > > > > > > having
> > > > > > > > > > > > > > events go through the stub would add quite a lot.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yeah I think, regardless of anything else, we want
> > the
> > > > > > > backends to
> > > > > > > > > > > > > connect directly to the Xen hypervisor.
> > > > > > > > > > > >
> > > > > > > > > > > > In my approach,
> > > > > > > > > > > >  a) BE -> FE: interrupts triggered by BE calling a
> > > > hypervisor
> > > > > > > > > interface
> > > > > > > > > > > >               via virtio-proxy
> > > > > > > > > > > >  b) FE -> BE: MMIO to config raises events (in event
> > > > channels),
> > > > > > > > > which is
> > > > > > > > > > > >               converted to a callback to BE via
> > virtio-
> > > > proxy
> > > > > > > > > > > >               (Xen's event channel is internnally
> > > > implemented by
> > > > > > > > > interrupts.)
> > > > > > > > > > > >
> > > > > > > > > > > > I don't know what "connect directly" means here, but
> > > > sending
> > > > > > > > > interrupts
> > > > > > > > > > > > to the opposite side would be best efficient.
> > > > > > > > > > > > Ivshmem, I suppose, takes this approach by utilizing
> > PCI's
> > > > msi-x
> > > > > > > > > mechanism.
> > > > > > > > > > >
> > > > > > > > > > > Agree that MSI would be more efficient than SPI...
> > > > > > > > > > > At the moment, in order to notify the frontend, the
> > backend
> > > > issues
> > > > > > > a
> > > > > > > > > specific device-model call to query Xen to inject a
> > > > corresponding SPI
> > > > > > > to
> > > > > > > > > the guest.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Could we consider the kernel internally converting
> > > > IOREQ
> > > > > > > > > messages from
> > > > > > > > > > > > > > the Xen hypervisor to eventfd events? Would this
> > scale
> > > > with
> > > > > > > > > other kernel
> > > > > > > > > > > > > > hypercall interfaces?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So any thoughts on what directions are worth
> > > > experimenting
> > > > > > > with?
> > > > > > > > > > > > >
> > > > > > > > > > > > > One option we should consider is for each backend to
> > > > connect
> > > > > > > to
> > > > > > > > > Xen via
> > > > > > > > > > > > > the IOREQ interface. We could generalize the IOREQ
> > > > interface
> > > > > > > and
> > > > > > > > > make it
> > > > > > > > > > > > > hypervisor agnostic. The interface is really trivial
> > and
> > > > easy
> > > > > > > to
> > > > > > > > > add.
> > > > > > > > > > > >
> > > > > > > > > > > > As I said above, my proposal does the same thing that
> > you
> > > > > > > mentioned
> > > > > > > > > here :)
> > > > > > > > > > > > The difference is that I do call hypervisor interfaces
> > via
> > > > > > > virtio-
> > > > > > > > > proxy.
> > > > > > > > > > > >
> > > > > > > > > > > > > The only Xen-specific part is the notification
> > mechanism,
> > > > > > > which is
> > > > > > > > > an
> > > > > > > > > > > > > event channel. If we replaced the event channel with
> > > > something
> > > > > > > > > else the
> > > > > > > > > > > > > interface would be generic. See:
> > > > > > > > > > > > > https://gitlab.com/xen-project/xen/-
> > > > > > > > > /blob/staging/xen/include/public/hvm/ioreq.h#L52
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't think that translating IOREQs to eventfd in
> > the
> > > > kernel
> > > > > > > is
> > > > > > > > > a
> > > > > > > > > > > > > good idea: if feels like it would be extra
> > complexity
> > > > and that
> > > > > > > the
> > > > > > > > > > > > > kernel shouldn't be involved as this is a backend-
> > > > hypervisor
> > > > > > > > > interface.
> > > > > > > > > > > >
> > > > > > > > > > > > Given that we may want to implement BE as a bare-metal
> > > > > > > application
> > > > > > > > > > > > as I did on Zephyr, I don't think that the translation
> > > > would not
> > > > > > > be
> > > > > > > > > > > > a big issue, especially on RTOS's.
> > > > > > > > > > > > It will be some kind of abstraction layer of interrupt
> > > > handling
> > > > > > > > > > > > (or nothing but a callback mechanism).
> > > > > > > > > > > >
> > > > > > > > > > > > > Also, eventfd is very Linux-centric and we are
> > trying to
> > > > > > > design an
> > > > > > > > > > > > > interface that could work well for RTOSes too. If we
> > > > want to
> > > > > > > do
> > > > > > > > > > > > > something different, both OS-agnostic and
> > hypervisor-
> > > > agnostic,
> > > > > > > > > perhaps
> > > > > > > > > > > > > we could design a new interface. One that could be
> > > > > > > implementable
> > > > > > > > > in the
> > > > > > > > > > > > > Xen hypervisor itself (like IOREQ) and of course any
> > > > other
> > > > > > > > > hypervisor
> > > > > > > > > > > > > too.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > There is also another problem. IOREQ is probably not
> > be
> > > > the
> > > > > > > only
> > > > > > > > > > > > > interface needed. Have a look at
> > > > > > > > > > > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2.
> > > > Don't we
> > > > > > > > > also need
> > > > > > > > > > > > > an interface for the backend to inject interrupts
> > into
> > > > the
> > > > > > > > > frontend? And
> > > > > > > > > > > > > if the backend requires dynamic memory mappings of
> > > > frontend
> > > > > > > pages,
> > > > > > > > > then
> > > > > > > > > > > > > we would also need an interface to map/unmap domU
> > pages.
> > > > > > > > > > > >
> > > > > > > > > > > > My proposal document might help here; All the
> > interfaces
> > > > > > > required
> > > > > > > > > for
> > > > > > > > > > > > virtio-proxy (or hypervisor-related interfaces) are
> > listed
> > > > as
> > > > > > > > > > > > RPC protocols :)
> > > > > > > > > > > >
> > > > > > > > > > > > > These interfaces are a lot more problematic than
> > IOREQ:
> > > > IOREQ
> > > > > > > is
> > > > > > > > > tiny
> > > > > > > > > > > > > and self-contained. It is easy to add anywhere. A
> > new
> > > > > > > interface to
> > > > > > > > > > > > > inject interrupts or map pages is more difficult to
> > > > manage
> > > > > > > because
> > > > > > > > > it
> > > > > > > > > > > > > would require changes scattered across the various
> > > > emulators.
> > > > > > > > > > > >
> > > > > > > > > > > > Exactly. I have no confident yet that my approach will
> > > > also
> > > > > > > apply
> > > > > > > > > > > > to other hypervisors than Xen.
> > > > > > > > > > > > Technically, yes, but whether people can accept it or
> > not
> > > > is a
> > > > > > > > > different
> > > > > > > > > > > > matter.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > -Takahiro Akashi
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Oleksandr Tyshchenko
> > > > > > > > > > IMPORTANT NOTICE: The contents of this email and any
> > > > attachments are
> > > > > > > > > confidential and may also be privileged. If you are not the
> > > > intended
> > > > > > > > > recipient, please notify the sender immediately and do not
> > > > disclose
> > > > > > > the
> > > > > > > > > contents to any other person, use it for any purpose, or
> > store
> > > > or copy
> > > > > > > the
> > > > > > > > > information in any medium. Thank you.
> > > > > > > > IMPORTANT NOTICE: The contents of this email and any
> > attachments
> > > > are
> > > > > > > confidential and may also be privileged. If you are not the
> > intended
> > > > > > > recipient, please notify the sender immediately and do not
> > disclose
> > > > the
> > > > > > > contents to any other person, use it for any purpose, or store
> > or
> > > > copy the
> > > > > > > information in any medium. Thank you.
> > > > > > IMPORTANT NOTICE: The contents of this email and any attachments
> > are
> > > > confidential and may also be privileged. If you are not the intended
> > > > recipient, please notify the sender immediately and do not disclose
> > the
> > > > contents to any other person, use it for any purpose, or store or copy
> > the
> > > > information in any medium. Thank you.
> > > IMPORTANT NOTICE: The contents of this email and any attachments are
> > confidential and may also be privileged. If you are not the intended
> > recipient, please notify the sender immediately and do not disclose the
> > contents to any other person, use it for any purpose, or store or copy the
> > information in any medium. Thank you.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 12:43:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 12:43:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176353.320910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPaO-0006NL-5w; Wed, 01 Sep 2021 12:43:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176353.320910; Wed, 01 Sep 2021 12:43:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPaO-0006NE-1x; Wed, 01 Sep 2021 12:43:24 +0000
Received: by outflank-mailman (input) for mailman id 176353;
 Wed, 01 Sep 2021 12:43:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4/yp=NX=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mLPaN-0006N8-3v
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 12:43:23 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 2f8e18dd-0b22-11ec-add5-12813bfff9fa;
 Wed, 01 Sep 2021 12:43:20 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A138F1042;
 Wed,  1 Sep 2021 05:43:19 -0700 (PDT)
Received: from e123311-lin.arm.com (unknown [10.57.13.24])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B66CF3F766;
 Wed,  1 Sep 2021 05:43:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f8e18dd-0b22-11ec-add5-12813bfff9fa
From: Michal Orzel <michal.orzel@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	bertrand.marquis@arm.com
Subject: [PATCH] xen/arm: Introduce pmu_access parameter
Date: Wed,  1 Sep 2021 14:43:08 +0200
Message-Id: <20210901124308.31573-1-michal.orzel@arm.com>
X-Mailer: git-send-email 2.29.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Introduce new Xen command line parameter called "pmu_access".
The default value is "trap": Xen traps PMU accesses.
In case of setting pmu_access to "native", Xen does not trap
PMU accesses allowing all the guests to access PMU registers.
However, guests cannot make use of PMU overflow interrupts as
PMU uses PPI which Xen cannot route to guests.

This option is only intended for development and testing purposes.
Do not use this in production system.

Signed-off-by: Michal Orzel <michal.orzel@arm.com>
---
 docs/misc/xen-command-line.pandoc | 18 +++++++++++++++++
 xen/arch/arm/traps.c              | 33 ++++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index b175645fde..03637a9f6d 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1813,6 +1813,24 @@ paging controls access to usermode addresses.
 ### ple_window (Intel)
 > `= <integer>`
 
+### pmu_access (arm)
+> `= trap | native`
+
+> Default: `trap`
+
+Controls for accessing Performance Monitor Unit (PMU).
+
+By default Xen traps Performance Monitor accesses.
+When setting pmu_access to `native`, Xen does not trap PMU accesses allowing
+the guests to access PMU registers. This option is intended to aid monitoring
+and measuring the performance. Setting pmu_access to `native` allows
+all the guests to access PMU, however, there is no mechanism for forwarding
+PMU overflow interrupt requests.
+
+*Warning*
+This option is only intended for development and testing purposes.
+Do not use this in production system.
+
 ### psr (Intel)
 > `= List of ( cmt:<boolean> | rmid_max:<integer> | cat:<boolean> | cos_max:<integer> | cdp:<boolean> )`
 
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 219ab3c3fb..d30e78b4d6 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -34,6 +34,7 @@
 #include <xen/symbols.h>
 #include <xen/version.h>
 #include <xen/virtual_region.h>
+#include <xen/warning.h>
 
 #include <public/sched.h>
 #include <public/xen.h>
@@ -77,12 +78,19 @@ static int debug_stack_lines = 40;
 #define stack_words_per_line 4
 #endif
 
+static const char __initconst warning_pmu_access[] =
+    "WARNING: PMU ACCESSES ARE NOW ENABLED\n"
+    "This option is intended to aid monitoring and measuring\n"
+    "the performance by allowing the guests to access PMU registers.\n"
+    "It has implications on the security of the system.\n"
+    "Please *DO NOT* use this in production.\n";
+
 integer_param("debug_stack_lines", debug_stack_lines);
 
 static enum {
 	TRAP,
 	NATIVE,
-} vwfi;
+} vwfi, pmu_access;
 
 static int __init parse_vwfi(const char *s)
 {
@@ -95,6 +103,29 @@ static int __init parse_vwfi(const char *s)
 }
 custom_param("vwfi", parse_vwfi);
 
+static int __init parse_pmu_access(const char *s)
+{
+    if ( !strcmp(s, "native") )
+        pmu_access = NATIVE;
+    else
+        pmu_access = TRAP;
+
+    return 0;
+}
+custom_param("pmu_access", parse_pmu_access);
+
+static int __init update_pmu_access(void)
+{
+    if ( pmu_access == NATIVE )
+    {
+        WRITE_SYSREG(READ_SYSREG(MDCR_EL2) &~ (HDCR_TPM|HDCR_TPMCR), MDCR_EL2);
+        warning_add(warning_pmu_access);
+    }
+
+    return 0;
+}
+__initcall(update_pmu_access);
+
 register_t get_default_hcr_flags(void)
 {
     return  (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
-- 
2.29.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 12:47:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 12:47:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176359.320921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPei-00071s-OJ; Wed, 01 Sep 2021 12:47:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176359.320921; Wed, 01 Sep 2021 12:47:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPei-00071l-KP; Wed, 01 Sep 2021 12:47:52 +0000
Received: by outflank-mailman (input) for mailman id 176359;
 Wed, 01 Sep 2021 12:47:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rqR/=NX=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mLPeh-00071f-Cq
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 12:47:51 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d08a981e-0b22-11ec-add6-12813bfff9fa;
 Wed, 01 Sep 2021 12:47:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d08a981e-0b22-11ec-add6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630500470;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=h6mJzdTcFw3IRJ6Tvbd87qhOMqz7nqvNglic+QwxbPI=;
  b=HhFzni5QH6luLYn1LAsYA3fNdt0OUyoJbJTThbazXXe3kHMhCcVd8HOU
   kSQrQ7iKSZC691bAXz+foqvG1F136S3rGvciTB1SSXSPk5e1lURmzfU2S
   dvd98sLQSeNhUTT7QivUBRh94j8qJC2v/GD85as5kgmJ2t1odseqpJtG9
   A=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: yt6rC4h0IVg9WGAgQqBw+QI6Vyb4RHsSc6ZlVB+K0T/55JP6s6qj35S9ggo30I++PdE4fBkTFr
 UD2cYkH3DbnCECHRq9hRLxSqhqw+WvMyxBId3EmoXf/QlWSfIQkbJJtt2CefiqLayQ01zWdrDS
 zFst2jTkHAdQslKjesrQKNk2uN2UOdPy9OTTpBTkWHGX5BKcVDD8p5dlO5nr6X5mi/6OZ4Hpzb
 shOmxOSp/r0AIyJffE4nRU/dzjSRC59fxXjoSzZliKPubAs1ivOp3/cjr3Ggb0M+vIzTDqgg1r
 7sHXvMP44Xpog+lgvMoVYiW3
X-SBRS: 5.1
X-MesageID: 52179159
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:O+qdP6yIvAOamiwdUOZ5KrPxouskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjkfZquz+8L3WB3B8bfYOCGghrUEGgG1+XfKlLbalXDH4JmpM
 Bdmu1FeafN5DtB/LbHCWuDYq8dKbC8mcjC74eurAYecegpUdAF0+4QMHfrLqQcfnghOXNWLu
 v/2iMKnUvaRZxBBf7LeEXtEtKz6OHjpdbDW1orFhQn4A6BgXeB76P7KQGR2lM7XylUybkv3G
 DZm0ihj5/T8s2T+1v57Sv+/p5WkNzuxp9qA9GNsNEcLnHJhhyzbIpsdrWetHQeof2p6nwtjN
 7Qyi1Qcfhb2jf0RCWYsBHt0w7v3HIH7GLj80aRhT/ZrcnwVFsBeoB8rLMcViGcx1srvdl63q
 4O9XmerYBrARTJmzm4z8TUVjlx/3DE40YKoKo2tThyQIEeYLheocg050VOCqoNGyr89cQODP
 RuNsfB//xbGGnqLEwxhlMfhOBEY05DWStvGiM5y4qoOnlt7TBEJnIjtYkidixqzuNld3EsjN
 60QZiBl9l1P4QrhOxGdb88qWbeMB26ffv2ChPnHb3QLtBOB5v8ke+D3FwL3pDcRHUp9up+pH
 2TaiIViYYNE3ieQPFmmqc7qSzwfA==
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="52179159"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VehwfQI3Yr0WBQFibVvQEaG1LPfYEzpwEhoCcelziCNpSFcZGwoKLwYteeQkacAKGDWgtfOK5zV9bLWt2hmH4pYbuYcAVycf0a3j2EmD0OuBzrORMcUZkgx/pISnC2J+UwR8FUnB0AuGMNU7T6Srknurfflt6SX3E/6ZXE9teW7eZAVI8h4G9jO4RJHZeYwtUAlDHf5YDgoXJIRFC1D52CMJ8lkHc0w5GjD4ftcy9k0yO10w+ySnYaVnVN+ZUjAIXjh5gigw0q0tcZH2TeNpmVlvdy9rJ+jUoGYDav2nJZ5F+7Y/jthRBBUXsjrBpPmJo+cT2r8hEbc6bIAQODioQg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tjDlw24xjL9N6LfcvdiORpfx0u28EjiV6lJoNjLfxq8=;
 b=W6o4kmppL91QG6I9Dz2uK6zVgiKL6PoXwcTmI8ViPCXT8oqW7ZoRwbIszzKSyCJj3o5ahfZCYDW+KkogUb9yWBRl1MXl0U6LVDQ4WonTHpz0GELWlCZgW30Pkk4A09ZhYZIflrLCgt2VTT7eSUfUGqJY0Z82qGqKS1KOctzpFPkhRAU9t/M/TUtPzIJRKTTE5qpNtkNgDmS7iD4RZKvWKaaxEl/lnBCV3vET1yKdWoo0aPqelgfuLKcmF29IETbYwEiAcbjL3k5t1yLaB5lMuekqc6RGGTOg4AH1mDBLrCoTUcr/SM55v2LKxlnwwZK3+Y9YbkqyWwpoKrl1rbkflw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tjDlw24xjL9N6LfcvdiORpfx0u28EjiV6lJoNjLfxq8=;
 b=d2uV98ZJ6Z8VgmoEuRoyExqVWaap36SQRaTIjQ9tF0Gkrob6LU4Hu5wQEijsKAMpyyYUh6C9D8SGe9I64bXxNl+JAhz9O1pjUHYCNOPLCvOTyYRqB8Ej6ofng8kUryW3ldVOMp/MjukUhZjFQNtMl7htzT+IYPwJfMV4kvUwIEc=
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
Message-ID: <62cb78c9-f4fd-76e7-225d-bc575f10850f@citrix.com>
Date: Wed, 1 Sep 2021 13:47:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0357.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18d::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5405d40e-c282-4aea-83a2-08d96d46b2f6
X-MS-TrafficTypeDiagnostic: BY5PR03MB4997:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB4997C5380E8A31D76A8BC46EBACD9@BY5PR03MB4997.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: X8wG6yt5GYJNe95xxdetZ5DX+ZM7eoFiskJ9wiN7pW9/XFXmy9OxE6Uv3SjDET2d3AgfzzrjFzvsPpu7scxilLctrBjixUbclWOxlb9NKnXuOfRrMmZncSTewoa1h8xZadXhTjzOqvs/q21x/QdFUnpstanfV4aRhtIDHX0IYS1tn+hErZMY4s+UEFOdTJ9qlI68TJWKcwV/dWHFIXZpQZrmBkxSvIswK2FrF1k+IYjGgVkgyCCpy8VOMIMRTy+3wL/MeatLgQSPftlxZ9rNaTIzHB8qcA1DixE6u+OHXgei6TTyPQ671dKzNNZ2lLKEU4sxCmI5g01oGSDqJiyrwLtUhX9msdT+eBP1hmUvhYrgBNjRGgsGkPrNvjjoKvKOWdg4GsEyuajifesNcZIIafR09lB9dUoWKdA4KOTGuSwC4utqz8hypczeotCj47U1xsr2M3AuhGCn8nPgHPHuZpxc3mD3+nQAJuc2btJ1qAtlahCjgk7l4pTzt2t49st2htuj4GfWYW8mZ+u0eSc7Jbn8Pnya+TITdmNVTJm1ub3lqR4dQUU8mBuN+Na2LhW5ddgousfyyJZoQ7Qv24XzPsBLL3OQD8oD1IaO7QWR19Mqrrx+DC4y5gSlW94QfD4tz2PN38gSD8OOT0BirhA2kJJDwkjiQZuu+MbPYDQwKmPDoOqNdysOHvTRLUodaEVh5XcwfrNJCDa4W1tGmJurhaAs6PQ2D4/rt/wnREwmYko=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(39860400002)(366004)(346002)(136003)(956004)(86362001)(66476007)(66556008)(2616005)(16576012)(31696002)(66946007)(26005)(5660300002)(54906003)(36756003)(2906002)(478600001)(6666004)(316002)(8676002)(186003)(55236004)(4326008)(31686004)(53546011)(6486002)(6916009)(38100700002)(83380400001)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RkxJbGNGUUFYYzcyTUZnSDNGb092WXNva3pYR1FlYXo4RVQ4V3dKSVJRaklv?=
 =?utf-8?B?VDJrVmlsOEo0WDhVamYzNXAwOFoxUmY4TytWdlpGQzEvYkIvUnZCaHBWeXRi?=
 =?utf-8?B?aFdLT3o1ck5yWjBDeE1xZEZ5WU5iN2sxcGVPVzNHZmpDZGFLQUpFMkhQQ2hx?=
 =?utf-8?B?OUNKTWNnK2RPdzV2VDJkc2piQitvNXBOdWFKTWtaSzdidll2VkgyZXFyZVQx?=
 =?utf-8?B?VkVXZ1RXTHpqdS9ZL2E5MTdiWndkRkJaRlBadm1ITGtsbzhVQXBQd3pCcGRP?=
 =?utf-8?B?SjcxcFVGUHd2am9Sek5TZmU3M3NBM3VKZ3lUd05ONy9IUkdWVlV4dlFoZnQr?=
 =?utf-8?B?aWJLTFdtK0JENWR6VG8rdGJmRHQ1ZkxWcFlEdzFPSC9FYjdoMENXMk9uZFl2?=
 =?utf-8?B?akppQ2VVYWYzdUVndDZ3elBlSkZpOFhoSUE0Y25SbFVtUEUvVDd6R0NzUDhV?=
 =?utf-8?B?d3haUGJ0QkRob2RYaVNqQmhuYXM2ZkRXbktUbmt0Ry9PcEgwem9ZUTlqRGNu?=
 =?utf-8?B?WlB0a3dpelkvclRVMTlsSUtNOUhUYzg0ZGRDNzRsUGdPSnIwMkdNMVR4VGdq?=
 =?utf-8?B?TjJiRGlpRytzV2RmVTFWdVFJU0FXYmtWUUs2bTdtbGF3QVBSU1JDTGVJdVNq?=
 =?utf-8?B?R2ZSYXM3VVE2QjNkU2lTNFVyZy8yUG9ONlZpaVNxb3JLaU1rY0NxZXNzb09j?=
 =?utf-8?B?eCs0TTFiUzRKSWh4N09vNTJTVmRUUU9FNS9HNlNoK2wzamNQNWF4N0dydm13?=
 =?utf-8?B?ajE0K0o4SzBxRWVFRFhCY2xsd0xiVElaWVF0VlRIY0RPektLaHVoV09VWkJO?=
 =?utf-8?B?WFJ1eUFQSzRaZG9ZZzVZbFB1QllFNUpPMlgwcXRvUWE4TGdWMkRoejFpVHlr?=
 =?utf-8?B?QlB2cjZwdnJkbHlPMW0zMWxibnBRc0xhdnU1YXo4VFVWMVNGL1E5eTBuOHY4?=
 =?utf-8?B?WC9ZOFNsbkJxM2lld2MxSHJudEZ4cGRmcGVrOTYxNjBwUzBVaUFLMEh4cnR4?=
 =?utf-8?B?RW5RTDZ3OWVBMlBQNkpyaHlnS1UyK29pUUlGNnNrNThIUUhRYk5TdUpDUFc5?=
 =?utf-8?B?YWV5VW1GeU9MaWo2b3VtM3FKcWxjZ29zSFpESG9JejZ5cGUxZlQ3U2JsM2tW?=
 =?utf-8?B?eVdOZzNRM1JINDdjd3l3cEJMYzI4b3plcmJnYXpvQ0hKRUwzL1JmbjUwcVBE?=
 =?utf-8?B?WkowRHNRY0lCNnVNeUVMR3RPV3QxeldWakxwYW11ZC9lbWFhTGRPazlFS0R0?=
 =?utf-8?B?L3llZ3I4MGhENjhkVWRFOUlZK3N5MjZ0VVhRb0c1TXE2dmpvdTJPWStLcFdH?=
 =?utf-8?B?RTJKOHRHYTZSOVBtK0w5cVhsY2x0dnJWSmR3QUFEcFBZcExObWpMRkpsS0dD?=
 =?utf-8?B?UW1zUTJadzVrcUROOWFlWE1LTElqRy84MjZiVi9udzM4eHAyamVoMXdRVDBz?=
 =?utf-8?B?SklCcGowMnBLWVZ1TkcvNW5qejNENUEzTWFSRHRsZzhZblJ1QTBuK216cm5q?=
 =?utf-8?B?TTNiR1NRR1gzQ2k2VlRocGc0OUVEaGhGSUorSVQ5M2pITU9wUm5mTkFsdHdj?=
 =?utf-8?B?eXhQdys0cVRVMEdtdlMrdEtPemY5bE1iZ0phS3VadElERXhJcGNqT2RwQ1Rj?=
 =?utf-8?B?eGU3YW12blNHSXNDVHVacUZHNm54NkdDUFR0WUQzZTBHQU5VcWQ2SWxMOWtW?=
 =?utf-8?B?ekgwOUorT1R2MDdKMGsydzlzempPZmQ2NmYxb3plNE9UUmJYMEdiZVJJTjZj?=
 =?utf-8?Q?92sogwZlfbxijYlchVOFquK31v7toqtdoKHlx0G?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5405d40e-c282-4aea-83a2-08d96d46b2f6
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 12:47:46.8553
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CF3Rl+KfsU6yvVA4Spcl/1DnmOGEn9Fw3gEwZWwv0KhXHL2xnfYG9N7ddWfRcZapweD8K3eO23tlwMUr6EUoJDPhEFMeeaB1pO+jvM3gwDI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4997
X-OriginatorOrg: citrix.com

On 31/08/2021 16:38, Jan Beulich wrote:
> On 31.08.2021 17:25, Andrew Cooper wrote:
>> On 31/08/2021 14:26, Jan Beulich wrote:
>>> On 31.08.2021 15:16, Andrew Cooper wrote:
>>>> On 30/08/2021 14:02, Jan Beulich wrote:
>>>>> Further permit "access" to differ in the "executable" attribute. Whil=
e
>>>>> ideally only ROM regions would get mapped with X set, getting there i=
s
>>>>> quite a bit of work. Therefore, as a temporary measure, permit X to
>>>>> vary. For Dom0 the more permissive of the types will be used, while f=
or
>>>>> DomU it'll be the more restrictive one.
>>>> Split behaviour between dom0 and domU based on types alone cannot
>>>> possibly be correct.
>>> True, but what do you do.
>>>
>>>> DomU's need to execute ROMs too, and this looks like will malfunction =
if
>>>> a ROM ends up in the region that HVMLoader relocated RAM from.
>>>>
>>>> As this is a temporary bodge emergency bugfix, don't try to be clever =
-
>>>> just take the latest access.
>>> And how do we know that that's what is going to work?
>> Because it's the pre-existing behaviour.
> Valid point. But for the DomU case there simply has not been any
> pre-existing behavior. Hence my desire to be restrictive initially
> there.

But you're conflating a feature (under question anyway, because I gave
you an example where I expect this will collide in a regular domU
already), with an emergency bugfix to unbreak staging caused by an
unexpected interaction in a security hotfix.

At an absolute minimum, this patch needs splitting in two to separate
the bugfix from the proposed feature.

>>>  We should
>>> strictly accumulate for Dom0. And what we do for DomU is moot for
>>> the moment, until PCI passthrough becomes a thing for PVH. Hence
>>> I've opted to be restrictive there - I'd rather see things break
>>> (and getting adjusted) when this future work actually gets carried
>>> out, than leave things permissive for no-one to notice that it's
>>> too permissive, leading to an XSA.
>> Restricting execute permissions is something unique to virt.=C2=A0 It do=
esn't
>> exist in a non-virtualised system, as I and D side reads are
>> indistinguishable outside of the core.
>>
>> Furthermore, it is inexpressible on some systems/configurations.
>>
>> Introspection is the only technology which should be restricting execute
>> permissions in the p2m, and only when it takes responsibility for
>> dealing with the fallout.
> IOW are you saying that the calls to set_identity_p2m_entry()
> (pre-dating XSA-378) were wrong to use p2m_access_rw?

Yes.

>  Because that's
> what's getting the way here.

On a real machine, you really can write some executable code into an
E820 reserved region and jump to it.=C2=A0 You can also execute code from
real BARs is you happen to know that they are prefetchable (or you're a
glutton for UC reads...)

And there is the WPBT ACPI table which exists specifically to let
firmware inject drivers/applications into a windows environment, and may
come out of the SPI ROM in the first place.


Is it sensible to execute an E820 reserved region, or unmarked BAR?=C2=A0
Probably not.

Should it work, because that's how real hardware behaves?=C2=A0 Absolutely.

Any restrictions beyond that want handling by some kind of introspection
agent which has a policy of what to do with legal-but-dodgy-looking actions=
.

> Plus, as a side note, then we don't even have e.g. IOMMUF_executable.

Just as I vs D side reads are indistinguishable outside of the CPU core,
the same is in principle true for PCI devices which execute code (e.g.
GPU shaders).=C2=A0 Reads on the bus are just reads.

That said, the latest SIOV spec does appear to include the ER (Execution
Request) bit for use with PASSID/Shared-Virtual-Memory, which interacts
with the EPT-X/ia32-NX bits, and goes as far as having SMEP/SMAP bits in
the IOMMU configuration.=C2=A0 I'm not sure if this is in released hardware
yet, but it's clearly on the horizon.=C2=A0 I can't spot any execute relate=
d
controls in the AMD IOMMU spec, although it does have user/supervisor
for PASSID/SVM.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 12:55:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 12:55:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176367.320938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPmB-0000Ca-Jt; Wed, 01 Sep 2021 12:55:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176367.320938; Wed, 01 Sep 2021 12:55:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPmB-0000CT-G3; Wed, 01 Sep 2021 12:55:35 +0000
Received: by outflank-mailman (input) for mailman id 176367;
 Wed, 01 Sep 2021 12:55:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLPmA-0000CN-I3
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 12:55:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLPm6-0002Dm-PT; Wed, 01 Sep 2021 12:55:30 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLPm6-0007By-Hy; Wed, 01 Sep 2021 12:55:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=2jdGvkMeJGwBMQ7U7T6XK/oCun6VmqB1B0FRR7GMfzI=; b=dtAyaE/e4bARK5hKVTA1wVYfVO
	l3n2kZkw2WaHB57QZzLrXIrWn+uwBcorCRlr6Xb6sR7Av3cLGZIWSGpiAsXiqTpDiXv9aPHoWMkwE
	wzL5m0nuyYYXTHozaFytCXDtHsJ9lcKQ8jYKmgJiSO0ds/E0LgDGM6djJmak31SKh+w4=;
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
To: Michal Orzel <michal.orzel@arm.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com
References: <20210901124308.31573-1-michal.orzel@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
Date: Wed, 1 Sep 2021 13:55:28 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210901124308.31573-1-michal.orzel@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 01/09/2021 13:43, Michal Orzel wrote:
> Introduce new Xen command line parameter called "pmu_access".
> The default value is "trap": Xen traps PMU accesses.
> In case of setting pmu_access to "native", Xen does not trap
> PMU accesses allowing all the guests to access PMU registers.
> However, guests cannot make use of PMU overflow interrupts as
> PMU uses PPI which Xen cannot route to guests.
> 
> This option is only intended for development and testing purposes.
> Do not use this in production system.
I am afraid your option is not safe even in development system as a vCPU 
may move between pCPUs.

However, even if we restricted the use to pinned vCPU *and* dedicated 
pCPU, I am not convinced that exposing an half backed PMU (the overflow 
interrupt would not work) to the guest is the right solution. This 
likely means the guest OS would need to be modified and therefore the 
usage of this option is fairly limited.

So I think the first steps are:
   1) Make the PPI work. There was some attempt in the past for it on 
xen-devel. You could have a look.
   2) Provide PMU bindings

With that in place, we can discuss how to expose the PMU even if it is 
unsafe in some conditions.

> 
> Signed-off-by: Michal Orzel <michal.orzel@arm.com> > ---
>   docs/misc/xen-command-line.pandoc | 18 +++++++++++++++++
>   xen/arch/arm/traps.c              | 33 ++++++++++++++++++++++++++++++-
>   2 files changed, 50 insertions(+), 1 deletion(-)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index b175645fde..03637a9f6d 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1813,6 +1813,24 @@ paging controls access to usermode addresses.
>   ### ple_window (Intel)
>   > `= <integer>`
>   
> +### pmu_access (arm)
> +> `= trap | native`
> +
> +> Default: `trap`
> +
> +Controls for accessing Performance Monitor Unit (PMU).
> +
> +By default Xen traps Performance Monitor accesses.
> +When setting pmu_access to `native`, Xen does not trap PMU accesses allowing
> +the guests to access PMU registers. This option is intended to aid monitoring
> +and measuring the performance. Setting pmu_access to `native` allows
> +all the guests to access PMU, however, there is no mechanism for forwarding
> +PMU overflow interrupt requests.
> +
> +*Warning*
> +This option is only intended for development and testing purposes.
> +Do not use this in production system.
> +
>   ### psr (Intel)
>   > `= List of ( cmt:<boolean> | rmid_max:<integer> | cat:<boolean> | cos_max:<integer> | cdp:<boolean> )`
>   
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 219ab3c3fb..d30e78b4d6 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -34,6 +34,7 @@
>   #include <xen/symbols.h>
>   #include <xen/version.h>
>   #include <xen/virtual_region.h>
> +#include <xen/warning.h>
>   
>   #include <public/sched.h>
>   #include <public/xen.h>
> @@ -77,12 +78,19 @@ static int debug_stack_lines = 40;
>   #define stack_words_per_line 4
>   #endif
>   
> +static const char __initconst warning_pmu_access[] =
> +    "WARNING: PMU ACCESSES ARE NOW ENABLED\n"
> +    "This option is intended to aid monitoring and measuring\n"
> +    "the performance by allowing the guests to access PMU registers.\n"
> +    "It has implications on the security of the system.\n"
> +    "Please *DO NOT* use this in production.\n";
> +
>   integer_param("debug_stack_lines", debug_stack_lines);
>   
>   static enum {
>   	TRAP,
>   	NATIVE,
> -} vwfi;
> +} vwfi, pmu_access;
>   
>   static int __init parse_vwfi(const char *s)
>   {
> @@ -95,6 +103,29 @@ static int __init parse_vwfi(const char *s)
>   }
>   custom_param("vwfi", parse_vwfi);
>   
> +static int __init parse_pmu_access(const char *s)
> +{
> +    if ( !strcmp(s, "native") )
> +        pmu_access = NATIVE;
> +    else
> +        pmu_access = TRAP;
> +
> +    return 0;
> +}
> +custom_param("pmu_access", parse_pmu_access);
> +
> +static int __init update_pmu_access(void)
> +{
> +    if ( pmu_access == NATIVE )
> +    {
> +        WRITE_SYSREG(READ_SYSREG(MDCR_EL2) &~ (HDCR_TPM|HDCR_TPMCR), MDCR_EL2);
> +        warning_add(warning_pmu_access);
> +    }
> +
> +    return 0;
> +}
> +__initcall(update_pmu_access);
> +
>   register_t get_default_hcr_flags(void)
>   {
>       return  (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:08:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:08:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176373.320948 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPyN-0001up-NP; Wed, 01 Sep 2021 13:08:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176373.320948; Wed, 01 Sep 2021 13:08:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLPyN-0001ui-KQ; Wed, 01 Sep 2021 13:08:11 +0000
Received: by outflank-mailman (input) for mailman id 176373;
 Wed, 01 Sep 2021 13:08:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLPyM-0001uc-1j
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 13:08:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a906c2b8-1a14-4c4a-bd88-808982da25ca;
 Wed, 01 Sep 2021 13:08:08 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-LBB-0BR0M1afRj73yfeIcw-1; Wed, 01 Sep 2021 15:08:06 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3934.eurprd04.prod.outlook.com (2603:10a6:803:1b::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 13:08:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 13:08:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0077.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Wed, 1 Sep 2021 13:08:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a906c2b8-1a14-4c4a-bd88-808982da25ca
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630501687;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kKOJUpvcTFM8hyAhhxDP25PXxQ/V9ZW9KDvbc9qzjtc=;
	b=W3SGPKuwsFly/nRvsu+u7cKqF0qou50X72FHXDJ7LZ+4HvCgWfvhYhxzhacFnAk4h7ElyK
	rePzS95B1Q0B8n6KEbiMduapheNoEoI9NE4esCvAFnrM7O5PNsqVAlyWMheWX3UfU3YuA4
	ZrxhtW6VQ3lONXnHa3Nxt7iXKqwkKRA=
X-MC-Unique: LBB-0BR0M1afRj73yfeIcw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SDDSdJLfPuegCUeDLRclJQFt4Tt4PDzQ9+wMnf9KuHABi1KthxETQzUrDvSbDAIrpWuean7z1HtUYAHbSOsKG7tP+grvxbqvChoQc935X92UXv/jRbZLg38VvBsW037bcayDBAiKyRagRLwqZcHcFEL3Lv40r0ytdv5qrWVDFPqYZtGE9G4f7j4iozq5vhwLIm4hWSaP5bBWhof0yNuCtUhXl4pmPmsyfdmJPrqNI/8mNRx5GH5sT99jBDAdSqYoqhQkAibdTvopdqFdYsCl3yl4WyvA+nfsfO0JcmKjY9Y1Qf4nSvGTQFOovi7xvefYT7kQsnd9gS1vk/6O2BJX/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Zj3a51YS0sJYi/JgKZoY15y3HvCrl4xmlsN+RSvg7T4=;
 b=KaRX+6TnU6ZH7U7+J7j0O8V5NYN/X67Q5Kb3gLKgTtQ7T5gbuzl/MD9h5OUQ9tGn/U2t/g7wI1EqhK8ue6RFm46PHxtCgUF8az8/kfTkjOmlA59mh+R2hVRK6AohMZiuY2MDEwppvyWWHuYXrb42WKoAnxHM3xESEzO6LRju4ns3t3+3onTqfDOUU3MdnKDT0gQuO3TTP+B3rPwbOxSIGGiPd/muM5gOWH0ncX/w0tcLYv7EW3OCwKhyA6bmCjMRZA1R8OMnr85ylEuY5yk5HNTNsh1IUfhkXYPdSNJdynNofbBMXprqKpS5tq3GkQHjLjy2Cz/pWdshP88uv0tyBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <62cb78c9-f4fd-76e7-225d-bc575f10850f@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <818cc08e-8372-35c4-935f-86a87cceec75@suse.com>
Date: Wed, 1 Sep 2021 15:08:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <62cb78c9-f4fd-76e7-225d-bc575f10850f@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P189CA0077.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 14109199-61f2-43d9-12a6-08d96d498897
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3934:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3934AF505D929EFDC4E1217BB3CD9@VI1PR0402MB3934.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ehrCuRY1QnwLcCPSjdSepu4DdseauyPFCyw+m/luUE1ynD1v5dFPWyosTDBwUH1UsZKpJhXu1ywDxI0ab4XPf6BY5ySC8j0c/TED8cizNgaCdbeWSvKSbkj+AZoJA8fgimVO4IR6UGx42RtCddjf6h/JWU59zY0+Qs+xQbch6Nj4YgzFiao+wqMc3eHPUlpsh7CCLqbHDTpwS9OCCuVNtczqagOrBn6C4k1poW6noLX0jFJ0QJhfh5xG7i+T9p3M9gxknVevPvN6a9IYuBDt5Mx0wOMg+lychpBInvDVUqpVMFz7BkWUAwbSPbRtZju9sChsgjQrphP1ZDkAica+ufjmOqRMj4qDU/ryrrVFxsTImr/purj3OCzkHpKKf9ew9IvdNfXmFySdWF7nVPFh7IOwOzVTuzQNQV1abVgPcdFUazLXp6eleIjJgUUnkrDBSRsHbNI2J8pnY9Q5Pag1G4w8dINQ2ZFLdpTM3RkCbJ5gvmic200wG9inFC2tiotnqmn+uJOKztIq9SfRv1z4ZNwSP796GwYfm16I1aEBBxWc/ajlX3/F03wmImHO+fdHGWLEC8eUtdgEEWUS+PmuxEOG95o09r2yp+KRU0OQ+e/HPcnHo1EgIEYGtAeyNENM2LM+v4sHMFUnsgbS8TZ2BeA98hX2ip8tbiE01nbpoNGXxG9saZL4YPmTyL27cvFjHvtnXZv8LIMfW3d5eNeubGOzkUHqq3aGuaECUXQ+TEQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(376002)(39860400002)(396003)(136003)(366004)(54906003)(6916009)(36756003)(38100700002)(4326008)(31696002)(5660300002)(31686004)(83380400001)(26005)(316002)(66946007)(53546011)(2906002)(66556008)(8676002)(66476007)(6486002)(16576012)(956004)(8936002)(478600001)(86362001)(2616005)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?5tq/BWyhVfmd1Z3Bt6/LnQkS8XkSR8ALSkf5dBRb2DlAoLK8+jqiLyh4d3nW?=
 =?us-ascii?Q?f3r9wgOkyGoiFlzLmOUv4gJkRks/s+Oi6BnJyuYtxH41vzIpu91EuIrd9rkw?=
 =?us-ascii?Q?ruUtuVKtJFF5qibtirW+SLJ4CnoCDlmFhODMkYz0SYVpnuTB2bRrlGN5h0Bn?=
 =?us-ascii?Q?Ve+MXURaMXLeH7tLZN3pCSVdwyemAmL9wdrXytBJV2G0xcKzlo/MP2M9b3Fl?=
 =?us-ascii?Q?IxkHuaoFHnqa7I7f+oBC/fwtX5biNK/3qot+x5tfvfU80Vd8CiIaEx0Nhmmp?=
 =?us-ascii?Q?4zNFjI7b0Bi/xBnxYz8Om+1bargQ6+yb7c5A9pANSFOIsyI80yTPOPBy1+8n?=
 =?us-ascii?Q?YWwvp91L85gjhrGJWAW441xdrT4/Ry+nsI+colizqJB0wKY3DFBNgkA+YtAc?=
 =?us-ascii?Q?bF+oyTnHm68SkFTjgIzEPVGfFdgI7jyIDLd2jtzIy8rITdq8LgDqlslXTNbw?=
 =?us-ascii?Q?Ym2pqn2iqbq9E661AuMZYHiA8hGL330tOiRhm7a+/rv29tcwBSUAnCEkM8yL?=
 =?us-ascii?Q?qBA7P71piVAbtfCzQtXZYM495iVoPYuJRmpHbYLtiEojewLzAaDoMzhv5AAy?=
 =?us-ascii?Q?iCouckIyECw/3AqY9fw1QC+Km+0EheDetXDispyDJqCq6VKnKh/r2C9hplj0?=
 =?us-ascii?Q?8OfVER3QKwUgRDSEo4W5Tx3arjIg1bJpUEGEHkcXPQBZk2LpoK4ULXXpK7b7?=
 =?us-ascii?Q?v9BEzaRQ6l9GAV2vG3aYLETcvQlF0uzmxdcwWX6u4Atvaa6i+WAsNO48/tTa?=
 =?us-ascii?Q?Nnv6+zO2266HaqJAO5d1PzKEjVnFp6Dadol9XAeSCdo7EzKNa9SEfoYKaS+n?=
 =?us-ascii?Q?ZvYDd+q1MBbxBOYDpgu0arD9zNwMka0zf0MjTr8PXxeJ4RuSwIlE1oVPcpW9?=
 =?us-ascii?Q?1gj5QYt8wwPYx6FmJiYs1YhMEFMYrn+IqY3HbCh9PxrT4gnUkYzbj4KcYk0s?=
 =?us-ascii?Q?SSJGLOusQ08m1MVHsuanbm+zbkGpbWs0THsw6zoJ3t6PK986veczVHj2erwY?=
 =?us-ascii?Q?+Hh016a5NN7JKbADRlNkXFuekexECRwT4ZnDpmJ8AlPFQ+ZtC/ncU3Z7Q51j?=
 =?us-ascii?Q?Anh0+/b8JmIUjoaFUk2JGbJbNUlSdHboja4NdmHv7iHntNxxGww+NwWweA6q?=
 =?us-ascii?Q?yDpyjFFKl9CvgBmAhwv3ELm8FrSAGWszQPf4zV4HcRxzmWUoch7A9uu6szjy?=
 =?us-ascii?Q?AfzQFbOyReoorxqrgDXhUAHA4FA6uan+hR8qmxiFSjGrC7ZHnyxAlNK33w/9?=
 =?us-ascii?Q?7HMUU72fkCzrnYnmDCaDUK4wsbgr/3ysS1NCs3kjZ9dvFTjns/tWvS69zI8g?=
 =?us-ascii?Q?iJ0sjqkVWZWQiTeHdY3vDjCZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 14109199-61f2-43d9-12a6-08d96d498897
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:08:04.5705
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rn+5t6xCfAYqUfx/0xwYygrvSZA139ZK7GgMLpEEWtvUgBVcr8w46iWrUN18FHA3slZiiH3xMEpEtndDLD691g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3934

On 01.09.2021 14:47, Andrew Cooper wrote:
> On 31/08/2021 16:38, Jan Beulich wrote:
>> On 31.08.2021 17:25, Andrew Cooper wrote:
>>> On 31/08/2021 14:26, Jan Beulich wrote:
>>>> On 31.08.2021 15:16, Andrew Cooper wrote:
>>>>> On 30/08/2021 14:02, Jan Beulich wrote:
>>>>>> Further permit "access" to differ in the "executable" attribute. Whi=
le
>>>>>> ideally only ROM regions would get mapped with X set, getting there =
is
>>>>>> quite a bit of work. Therefore, as a temporary measure, permit X to
>>>>>> vary. For Dom0 the more permissive of the types will be used, while =
for
>>>>>> DomU it'll be the more restrictive one.
>>>>> Split behaviour between dom0 and domU based on types alone cannot
>>>>> possibly be correct.
>>>> True, but what do you do.
>>>>
>>>>> DomU's need to execute ROMs too, and this looks like will malfunction=
 if
>>>>> a ROM ends up in the region that HVMLoader relocated RAM from.
>>>>>
>>>>> As this is a temporary bodge emergency bugfix, don't try to be clever=
 -
>>>>> just take the latest access.
>>>> And how do we know that that's what is going to work?
>>> Because it's the pre-existing behaviour.
>> Valid point. But for the DomU case there simply has not been any
>> pre-existing behavior. Hence my desire to be restrictive initially
>> there.
>=20
> But you're conflating a feature (under question anyway, because I gave
> you an example where I expect this will collide in a regular domU
> already),

I don't think your example fits: hvmloader moving RAM will first
convert the p2m slot to non-present. Then a ROM page can get mapped
there quite fine. A direct transition (without going through n/p)
would not work independent of the change here: The MFNs would
differ, as would the p2m types.

> with an emergency bugfix to unbreak staging caused by an
> unexpected interaction in a security hotfix.
>=20
> At an absolute minimum, this patch needs splitting in two to separate
> the bugfix from the proposed feature.

Well, okay, I will split the patch, despite not being convinced this
will do us any good - we'd backport just the part you consider a bug
fix, but not the part you deem a feature (and which I consider part
of the bug fix).

>>>>  We should
>>>> strictly accumulate for Dom0. And what we do for DomU is moot for
>>>> the moment, until PCI passthrough becomes a thing for PVH. Hence
>>>> I've opted to be restrictive there - I'd rather see things break
>>>> (and getting adjusted) when this future work actually gets carried
>>>> out, than leave things permissive for no-one to notice that it's
>>>> too permissive, leading to an XSA.

Actually I think I was missing an important aspect here: The code in
question gets used not only for PVH, but also for HVM, where pass-
through is a thing. Hence I'll restrict the "feature" part to Dom0
for now.

>>> Restricting execute permissions is something unique to virt.=C2=A0 It d=
oesn't
>>> exist in a non-virtualised system, as I and D side reads are
>>> indistinguishable outside of the core.
>>>
>>> Furthermore, it is inexpressible on some systems/configurations.
>>>
>>> Introspection is the only technology which should be restricting execut=
e
>>> permissions in the p2m, and only when it takes responsibility for
>>> dealing with the fallout.
>> IOW are you saying that the calls to set_identity_p2m_entry()
>> (pre-dating XSA-378) were wrong to use p2m_access_rw?
>=20
> Yes.
>=20
>>  Because that's
>> what's getting the way here.
>=20
> On a real machine, you really can write some executable code into an
> E820 reserved region and jump to it.=C2=A0 You can also execute code from
> real BARs is you happen to know that they are prefetchable (or you're a
> glutton for UC reads...)
>=20
> And there is the WPBT ACPI table which exists specifically to let
> firmware inject drivers/applications into a windows environment, and may
> come out of the SPI ROM in the first place.
>=20
>=20
> Is it sensible to execute an E820 reserved region, or unmarked BAR?=C2=A0
> Probably not.
>=20
> Should it work, because that's how real hardware behaves?=C2=A0 Absolutel=
y.
>=20
> Any restrictions beyond that want handling by some kind of introspection
> agent which has a policy of what to do with legal-but-dodgy-looking actio=
ns.

IOW you suggest we remove p2m_access_t parameters from various functions,
going with just default access? Of course, as pointed out in another
reply, we'll need to split p2m_mmio_direct into multiple types then, at
the very least to honor the potential r/o restriction of AMD IOMMU unity
mapped regions. (FAOD all of this isn't a short term plan anyway, at least
afaic.)

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:10:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:10:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176380.320960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQ0w-0003Iq-AN; Wed, 01 Sep 2021 13:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176380.320960; Wed, 01 Sep 2021 13:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQ0w-0003Ii-7L; Wed, 01 Sep 2021 13:10:50 +0000
Received: by outflank-mailman (input) for mailman id 176380;
 Wed, 01 Sep 2021 13:10:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C3+8=NX=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mLQ0v-0003Ic-N5
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 13:10:49 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.59]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da6234c4-7ba4-4d9a-871e-c0fb449e7526;
 Wed, 01 Sep 2021 13:10:44 +0000 (UTC)
Received: from DU2PR04CA0296.eurprd04.prod.outlook.com (2603:10a6:10:28c::31)
 by AM6PR08MB4262.eurprd08.prod.outlook.com (2603:10a6:20b:b4::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 13:10:43 +0000
Received: from DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:28c:cafe::9b) by DU2PR04CA0296.outlook.office365.com
 (2603:10a6:10:28c::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend
 Transport; Wed, 1 Sep 2021 13:10:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT062.mail.protection.outlook.com (10.152.20.197) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.17 via Frontend Transport; Wed, 1 Sep 2021 13:10:42 +0000
Received: ("Tessian outbound 5918cb94a4b5:v103");
 Wed, 01 Sep 2021 13:10:42 +0000
Received: from 4568ce9c777a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D4AAC822-4C85-48D1-BCF6-987B4E5F6BD3.1; 
 Wed, 01 Sep 2021 13:10:35 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4568ce9c777a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 01 Sep 2021 13:10:35 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6080.eurprd08.prod.outlook.com (2603:10a6:102:ec::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 13:10:34 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4457.024; Wed, 1 Sep 2021
 13:10:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da6234c4-7ba4-4d9a-871e-c0fb449e7526
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qwJnAE7Gxld7BbtZEYeSNKlissKtld1AdmcWjzKLjfU=;
 b=4HYkgxs7k5oT0SuUzKVcOVLnZSFL4ddFc/UiJbnriMkAK7uRzJV5G6V6h0rg+BIOSzH2L264a5QnHlPhhfSrD83A8Q4tyODGj9fts4bC5TAE19fi296CZ3SWS9PmDm3Owt2u+WQiPJ/kikLfvIOPbnLOjXKoPzFOl+Guonuz8lE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f1b80aaeefa67559
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ox+UeuvfD6mMTfI1bDlPDOFzHXCrGqVSTRjP+7fOOWey9O/jR8C1h8KgnkP71zABZULuNFO90v5RRKxbFrIMwxzvBXnqfEruA0UGgakvkCO20HIJJKX5Tl3e2tVK3DuZJZHqbMKf49JdPp4cpBGseTzdP4BH9iLVvP+Hc1GdX7OSGRJqIMtxcLCMjYqn6EPXlh8RJM+GUNwdwZV7BIF6wPK8DWlWmD2LRSSKW+hIg+DKcDEn5ojZvLyVY/TSwOREzqGt54Baoqq9gHqf1MDgPrtTldKqz7BFAiRLat3tMQMOGkJxxCFIgICo7NFIn84+Ql/PksuULkmDPlY1beSlxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qwJnAE7Gxld7BbtZEYeSNKlissKtld1AdmcWjzKLjfU=;
 b=I5JGzPpq/q6Ln+qwQ1lnz7T9YYjrJV0UADUCbIerrXuWccKa8lvgKCt8gZkDyeRrHayO3TD69v5T4mKeLsXRz5ABUaNyZJn2FJWrGANJ0cHnDZXCGAmPvN9QZVe58NU/hjMlrg7o3tFUatSGe/2IaAuLQPmGMpUhskA31ULOKSAl5PscCHrlcbGFqrHKOs6QyVDRGph3FFaY0Vyuxs4335MQ3XXwn2CzGIRhutZv8eaUkuZnxqmaF3gCDCf3e8w3WHHkSOnVg1tTkJQZgsU6GajdVNNedKJ1GQmTKRCOP4kNFSIBR6au0VSnS/2M4YM5evQf4jUBGqrS2L2V+FJr/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qwJnAE7Gxld7BbtZEYeSNKlissKtld1AdmcWjzKLjfU=;
 b=4HYkgxs7k5oT0SuUzKVcOVLnZSFL4ddFc/UiJbnriMkAK7uRzJV5G6V6h0rg+BIOSzH2L264a5QnHlPhhfSrD83A8Q4tyODGj9fts4bC5TAE19fi296CZ3SWS9PmDm3Owt2u+WQiPJ/kikLfvIOPbnLOjXKoPzFOl+Guonuz8lE=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Michal Orzel <Michal.Orzel@arm.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Topic: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Index: AQHXny72/ET98Hho/USYtlrYv98fUauPIugAgAAENgA=
Date: Wed, 1 Sep 2021 13:10:33 +0000
Message-ID: <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
In-Reply-To: <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 9672f376-092a-468a-7bb7-08d96d49e72b
x-ms-traffictypediagnostic: PA4PR08MB6080:|AM6PR08MB4262:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4262E6E956BFFDCD7B0064749DCD9@AM6PR08MB4262.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:5797;OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 tynD5neOP/ZjryYfOQbXwLD3c2WawOOVSrQPmDAW4Gf609g1WBTrQ8J4of/1cCJGYFJyggnDqtZrZ03vpogd0vSejekcNrZD5uTioq3jhs1TWVT42NY0wAZ+YYQoR8lR6oZxAe+lUDVR7zSNg13RTdQguIpYQ8ojA/0cjPtQUtR3dRXyk+Ust5L3d+315r444aGmQZyiTmUaYsE10zEfXNVsw4aKvalJUaRegfSs2dEkjqM/uvrtaUgX+fXcSRSSNVbWLFroyXFmaYKGcxouSr1OZjqX+KDJdPnVM6s6EViKVy7IHo9aHtFaLjh1iQarJWHRyIENn6t52jkfINEgHmwJOOqxXFxduqiRYHOcD461t3nUBd3IKpPoo3v6/5QcFVd/t3dlAAjOI1vrDkkFAPMLjc1UBaZOpm1nZyhTcN3GgeyBuCqExf/B0hNB+c3Dc9U4aCVelslYAxtUSH/qYdsT82Lnbute+HD7ZQdAHlGK4JMx8H5oTkBnml8p4bJQt2QOTwR2QW7Znsh0fd5xiz5si3w0xyAmBWUgJO7OCggUsF5QRGuupTYroXl+wyQqfiRSrpwSCxA2V17/cqxHTi7+kE2/KMDN1M/n04zOdwifBNLiDv93vjy/A2PL6hOtPEfPxHeSnJLTHhMfJ9NgTeZftdgwVBv2jdNLDPVzzF5kIjyVjrDyxjDoBHVf5By39+cLofGVIJ8wuJSU/t8lPmk81kdEZOx7tgSYYbibeTRGVJ658qg0JbwP7g7wxHh0
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(346002)(376002)(366004)(478600001)(36756003)(4326008)(6486002)(8676002)(71200400001)(38070700005)(6506007)(38100700002)(53546011)(8936002)(54906003)(66946007)(6512007)(122000001)(186003)(66446008)(64756008)(66556008)(66476007)(6916009)(91956017)(76116006)(86362001)(2906002)(2616005)(316002)(26005)(83380400001)(33656002)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?WkVpQ3p0dW9hTlBaUE5iaEVFQkd0bEpRdnh4VVNVN1R1RG54WUxrWldmMVhX?=
 =?utf-8?B?UW1oYjJBYy9OdXcwRjg1eGVkQ1RiNjRkcG1wbitIQjJOTkdOM1E5R2pyOXpQ?=
 =?utf-8?B?TU95ajRyY29CdVY2cVV6SC9lQTdheTdDa2VMQndXRjN5d0JoeWRnbG13TUlr?=
 =?utf-8?B?SktuK2pwck14UTFhZ01yeWJHcDZ5eWdXdHNWMHdWcC92TktzaTgyOU1TdlVr?=
 =?utf-8?B?UngwWDJsN1YrQlVLc1RYdWVKQlNscGQyem9OOWxDWTRyZFg5Rk5BWjVBQXBh?=
 =?utf-8?B?Y2lFT2xaQTFHR1ppOC9KSWlaY2ZzMXVuSWZ5U0g2dW9heFN5ekRaWkx5b29p?=
 =?utf-8?B?c01YRGliWkdsWEFSUHhTakdDZEVVVExraW9jbkZKeGxsOVZjdjQ1Tk9Rb3oy?=
 =?utf-8?B?UzlOVGduY1ZhVWFVVVZtdExZYWRrQzdWSktFUHo5S3pzenQ2czN1bnFCdk4w?=
 =?utf-8?B?Q3YxWFYvcDBLSmRWZGpKTlJ4UHpJT2R3eVg5WTRhK25odUVUcFo2eUNBQTZQ?=
 =?utf-8?B?WDNlZllhcjNOZG0xL0NSM04xd3RXbGFwWGdJYnZITisza3Y5VWVBaU5CVTUr?=
 =?utf-8?B?Rjk4ZkhhQjArZTF2dDRGNldOS0pzNUs0S3djU1A0b0cxZm1ZQnBUM0J0U0Zn?=
 =?utf-8?B?WG0wMUkvdFNON1JXQTA5b1ltMjRWaFZiNitTUVFLSXhWcDNoT1BJZTl4dWp4?=
 =?utf-8?B?Qm5lZ3VnVjh2SVlQZnVBOHJZZzhnMEQzVExzS1RkNHdlSUhRY1VKMFhBV2VJ?=
 =?utf-8?B?by8rd3UxLzRLblVVTm9mbms2WkZRSW1kQ1FQWGdxZUxQdVVybkpCVGh3M2VU?=
 =?utf-8?B?OWk1c0RuTnBlMWFDL1drOEx6d2IybklpT3QyWngzT09DYWEvaEFyR2lQNm9q?=
 =?utf-8?B?aDN3ZnYvSzBtaDdiOVdZaU1zb3lQbXJZc094anVFVW4rTmVNVnpuU25ZRUQx?=
 =?utf-8?B?RkRPNjlGNlp0eDVTdVhROXV4ZzU5ZVRYL3VpZ09KUmpjcnRGZ2hLTWRIOXIw?=
 =?utf-8?B?V3NESkwrWXhPakc0TnZHbkJFOFI2eXlaMUgyc3hkVHYrUjNFa3A3VXAzUEk3?=
 =?utf-8?B?TGNmZGg0RFBMMURHdERGaVZaL24wZm5zWHozQll3WU8vOFl2b1UzYzVqYkVx?=
 =?utf-8?B?VGpPWlZnbUF5TVJFR2R3azl2SkFJd2dzdFE3SXJLUnNxRlFKckhHLzFKSlVQ?=
 =?utf-8?B?ZFBTditPMSs5QmdmaUtHc1hLMzQ5WEdXbU85aU9hMkhFRWh6S0FZTHhnOVpS?=
 =?utf-8?B?QWhWdWtsMWVrVmVsc0phd1UxTjhIc1JtY3l2R1NHNXVQOXhCeDRpbm9aY3J1?=
 =?utf-8?B?TFFZUU02dkdlY3F5WitJWWVGb1NGelFMZEF4b0w3NWp3NlJwcDExQXQrV0Ji?=
 =?utf-8?B?aDByVWlKL24rYWpvWTBwQUFnNFNTbklsbHFzdHZtTGRNd01xVzZnRi9lK2dK?=
 =?utf-8?B?dHZRaVI4K1dSWEhjdXozNzBLRnFaTWVheHdLZk12b05XUllTazR5b3ZlcGNX?=
 =?utf-8?B?a1doWFpWbitWVXRhWjFsQVN1Y011OU04Sm1FdVJrWVJObll4OGV4MDhnMVpP?=
 =?utf-8?B?MGs1WFN0b3lwNFZwd20wRFVJdnM5R2tsOENyOEtmZFplci91RU1BTGNDYXBD?=
 =?utf-8?B?NnZaTzYwandzYXN1ekVnSis5d3lBLzB1Vmkyd3Q2ZjZJL29QU2c1TDBvaHJz?=
 =?utf-8?B?TWJkWmRFNDlIWEZvc1BiNmV1S2J4NUxuTU5yRlQ0eHBKL01YV0Q2TElaNlFM?=
 =?utf-8?Q?GsUvCscjYDEr+A1Qg6OhZyxxXXTYfCGSFrZCZpB?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <2DB9E7BA48157D4A8E6F07C397D7B3B3@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6080
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a3a685c1-8879-4f51-b411-08d96d49e1f8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PhFURO3WDi0vQjxjCtBiNW/wvfU5f8JUBDOTlkwzEvr8ZyoQXDjtJ0OD7mLVsLMt/MQ/nUzw+UpohGvEkujjgRfeCNxampQauPYB7uohqa6dWlJD1sRs+qn6s0PAbCKtC3fSTislQCSd3MatDSEH3uYKdRY5aqCHx3UO1hQsvbH6mF/I6NiohU8Mz8b6nUPda7qk1y6uSqlZloQacpVl+mplu2jQd/z6GakMQNzqurEFeqXR2q0POe+Qpev3Wv/T3CnrUBP2Ti2x7+9LQDSZ2SnjBzNRC6gBwt6lU4+2ct0zcSsE3WLP1cWQhyVA59AbnCLWKaFupsLToCB8yHXXMH1S884vgaphm+yJEom2e+/lExWeH8sAjwIXWrXTxHhsOzqVVxfnr9M13iUAgmuylz7T1JIO71Iunc68UdqAIpRnRkmtJwcZuWRhsH3K5aRjP7c4R6+HKv1v7EIMeYAjFfo8INfHSFbyMKP0mRAlUpC1elcx+cIA3XqCDydhbi5NeCcRvXY3iibVZevCR4X041rq3i3HV1B4ecT8Kx1zu6voq8eWkhgT7d5wJYz6XY36P96z88vjMKeCOQZkivY4+qK0jFocvXJ+oL315ivFoe04ft0QUMa3m2nYeMPYJo8OgUXoJC/PIfJwA/bZIQ+kflAgBUURRBWRs50mlMUxnEvVyjsDc6au5lE6sIrc6klaMiP0m58xWWYcSYuq1M0PcA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(376002)(346002)(39860400002)(396003)(36840700001)(46966006)(6862004)(336012)(36860700001)(356005)(478600001)(6512007)(316002)(86362001)(8936002)(54906003)(4326008)(5660300002)(53546011)(36756003)(186003)(2616005)(8676002)(107886003)(70586007)(81166007)(26005)(70206006)(82310400003)(47076005)(82740400003)(83380400001)(6486002)(2906002)(6506007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:10:42.7962
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9672f376-092a-468a-7bb7-08d96d49e72b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4262

SGkgSnVsaWVuLA0KDQo+IE9uIDEgU2VwIDIwMjEsIGF0IDEzOjU1LCBKdWxpZW4gR3JhbGwgPGp1
bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4gDQo+IEhpLA0KPiANCj4gT24gMDEvMDkvMjAyMSAxMzo0
MywgTWljaGFsIE9yemVsIHdyb3RlOg0KPj4gSW50cm9kdWNlIG5ldyBYZW4gY29tbWFuZCBsaW5l
IHBhcmFtZXRlciBjYWxsZWQgInBtdV9hY2Nlc3MiLg0KPj4gVGhlIGRlZmF1bHQgdmFsdWUgaXMg
InRyYXAiOiBYZW4gdHJhcHMgUE1VIGFjY2Vzc2VzLg0KPj4gSW4gY2FzZSBvZiBzZXR0aW5nIHBt
dV9hY2Nlc3MgdG8gIm5hdGl2ZSIsIFhlbiBkb2VzIG5vdCB0cmFwDQo+PiBQTVUgYWNjZXNzZXMg
YWxsb3dpbmcgYWxsIHRoZSBndWVzdHMgdG8gYWNjZXNzIFBNVSByZWdpc3RlcnMuDQo+PiBIb3dl
dmVyLCBndWVzdHMgY2Fubm90IG1ha2UgdXNlIG9mIFBNVSBvdmVyZmxvdyBpbnRlcnJ1cHRzIGFz
DQo+PiBQTVUgdXNlcyBQUEkgd2hpY2ggWGVuIGNhbm5vdCByb3V0ZSB0byBndWVzdHMuDQo+PiBU
aGlzIG9wdGlvbiBpcyBvbmx5IGludGVuZGVkIGZvciBkZXZlbG9wbWVudCBhbmQgdGVzdGluZyBw
dXJwb3Nlcy4NCj4+IERvIG5vdCB1c2UgdGhpcyBpbiBwcm9kdWN0aW9uIHN5c3RlbS4NCj4gSSBh
bSBhZnJhaWQgeW91ciBvcHRpb24gaXMgbm90IHNhZmUgZXZlbiBpbiBkZXZlbG9wbWVudCBzeXN0
ZW0gYXMgYSB2Q1BVIG1heSBtb3ZlIGJldHdlZW4gcENQVXMuDQo+IA0KPiBIb3dldmVyLCBldmVu
IGlmIHdlIHJlc3RyaWN0ZWQgdGhlIHVzZSB0byBwaW5uZWQgdkNQVSAqYW5kKiBkZWRpY2F0ZWQg
cENQVSwgSSBhbSBub3QgY29udmluY2VkIHRoYXQgZXhwb3NpbmcgYW4gaGFsZiBiYWNrZWQgUE1V
ICh0aGUgb3ZlcmZsb3cgaW50ZXJydXB0IHdvdWxkIG5vdCB3b3JrKSB0byB0aGUgZ3Vlc3QgaXMg
dGhlIHJpZ2h0IHNvbHV0aW9uLiBUaGlzIGxpa2VseSBtZWFucyB0aGUgZ3Vlc3QgT1Mgd291bGQg
bmVlZCB0byBiZSBtb2RpZmllZCBhbmQgdGhlcmVmb3JlIHRoZSB1c2FnZSBvZiB0aGlzIG9wdGlv
biBpcyBmYWlybHkgbGltaXRlZC4NCj4gDQo+IFNvIEkgdGhpbmsgdGhlIGZpcnN0IHN0ZXBzIGFy
ZToNCj4gIDEpIE1ha2UgdGhlIFBQSSB3b3JrLiBUaGVyZSB3YXMgc29tZSBhdHRlbXB0IGluIHRo
ZSBwYXN0IGZvciBpdCBvbiB4ZW4tZGV2ZWwuIFlvdSBjb3VsZCBoYXZlIGEgbG9vay4NCj4gIDIp
IFByb3ZpZGUgUE1VIGJpbmRpbmdzDQo+IA0KPiBXaXRoIHRoYXQgaW4gcGxhY2UsIHdlIGNhbiBk
aXNjdXNzIGhvdyB0byBleHBvc2UgdGhlIFBNVSBldmVuIGlmIGl0IGlzIHVuc2FmZSBpbiBzb21l
IGNvbmRpdGlvbnMuDQoNCldpdGggdGhvc2UgbGltaXRhdGlvbnMsIHVzaW5nIHRoZSBQTVUgdG8g
bW9uaXRvciB0aGUgc3lzdGVtIHBlcmZvcm1hbmNlcyBvciBvbiBzb21lIHNwZWNpZmljIHVzZSBj
YXNlcyBpcyBzdGlsbCByZWFsbHkgdXNlZnVsLg0KV2UgYXJlIHVzaW5nIHRoYXQgdG8gZG8gc29t
ZSBiZW5jaG1hcmtzIG9mIFhlbiBvciBvZiBzb21lIGFwcGxpY2F0aW9ucyB0byBjb21wYXJlIHRo
ZSBiZWhhdmlvdXIgdG8gYSBuYXRpdmUgc3lzdGVtIG9yDQphbmFseXNlIHRoZSBwZXJmb3JtYW5j
ZXMgb2YgWGVuIGl0c2VsZiAoaHlwZXJjYWxscyxjb250ZXh0IHN3aXRjaCDigKZldGMpDQoNClRo
ZSBzdGVwcyB5b3UgYXJlIG1lbnRpb25pbmcgZG8gbWFrZSBzZW5zZSBidXQgdXNpbmcgdGhlIFBN
VSB3aXRob3V0IHRob3NlIGtub3dpbmcgdGhlIGxpbWl0YXRpb25zIGlzIGFsc28gdmVyeSB1c2Vm
dWwuDQoNCkNoZWVycw0KQmVydHJhbmQNCg0KDQo+IA0KPj4gU2lnbmVkLW9mZi1ieTogTWljaGFs
IE9yemVsIDxtaWNoYWwub3J6ZWxAYXJtLmNvbT4gPiAtLS0NCj4+ICBkb2NzL21pc2MveGVuLWNv
bW1hbmQtbGluZS5wYW5kb2MgfCAxOCArKysrKysrKysrKysrKysrKw0KPj4gIHhlbi9hcmNoL2Fy
bS90cmFwcy5jICAgICAgICAgICAgICB8IDMzICsrKysrKysrKysrKysrKysrKysrKysrKysrKysr
Ky0NCj4+ICAyIGZpbGVzIGNoYW5nZWQsIDUwIGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkN
Cj4+IGRpZmYgLS1naXQgYS9kb2NzL21pc2MveGVuLWNvbW1hbmQtbGluZS5wYW5kb2MgYi9kb2Nz
L21pc2MveGVuLWNvbW1hbmQtbGluZS5wYW5kb2MNCj4+IGluZGV4IGIxNzU2NDVmZGUuLjAzNjM3
YTlmNmQgMTAwNjQ0DQo+PiAtLS0gYS9kb2NzL21pc2MveGVuLWNvbW1hbmQtbGluZS5wYW5kb2MN
Cj4+ICsrKyBiL2RvY3MvbWlzYy94ZW4tY29tbWFuZC1saW5lLnBhbmRvYw0KPj4gQEAgLTE4MTMs
NiArMTgxMywyNCBAQCBwYWdpbmcgY29udHJvbHMgYWNjZXNzIHRvIHVzZXJtb2RlIGFkZHJlc3Nl
cy4NCj4+ICAjIyMgcGxlX3dpbmRvdyAoSW50ZWwpDQo+PiAgPiBgPSA8aW50ZWdlcj5gDQo+PiAg
KyMjIyBwbXVfYWNjZXNzIChhcm0pDQo+PiArPiBgPSB0cmFwIHwgbmF0aXZlYA0KPj4gKw0KPj4g
Kz4gRGVmYXVsdDogYHRyYXBgDQo+PiArDQo+PiArQ29udHJvbHMgZm9yIGFjY2Vzc2luZyBQZXJm
b3JtYW5jZSBNb25pdG9yIFVuaXQgKFBNVSkuDQo+PiArDQo+PiArQnkgZGVmYXVsdCBYZW4gdHJh
cHMgUGVyZm9ybWFuY2UgTW9uaXRvciBhY2Nlc3Nlcy4NCj4+ICtXaGVuIHNldHRpbmcgcG11X2Fj
Y2VzcyB0byBgbmF0aXZlYCwgWGVuIGRvZXMgbm90IHRyYXAgUE1VIGFjY2Vzc2VzIGFsbG93aW5n
DQo+PiArdGhlIGd1ZXN0cyB0byBhY2Nlc3MgUE1VIHJlZ2lzdGVycy4gVGhpcyBvcHRpb24gaXMg
aW50ZW5kZWQgdG8gYWlkIG1vbml0b3JpbmcNCj4+ICthbmQgbWVhc3VyaW5nIHRoZSBwZXJmb3Jt
YW5jZS4gU2V0dGluZyBwbXVfYWNjZXNzIHRvIGBuYXRpdmVgIGFsbG93cw0KPj4gK2FsbCB0aGUg
Z3Vlc3RzIHRvIGFjY2VzcyBQTVUsIGhvd2V2ZXIsIHRoZXJlIGlzIG5vIG1lY2hhbmlzbSBmb3Ig
Zm9yd2FyZGluZw0KPj4gK1BNVSBvdmVyZmxvdyBpbnRlcnJ1cHQgcmVxdWVzdHMuDQo+PiArDQo+
PiArKldhcm5pbmcqDQo+PiArVGhpcyBvcHRpb24gaXMgb25seSBpbnRlbmRlZCBmb3IgZGV2ZWxv
cG1lbnQgYW5kIHRlc3RpbmcgcHVycG9zZXMuDQo+PiArRG8gbm90IHVzZSB0aGlzIGluIHByb2R1
Y3Rpb24gc3lzdGVtLg0KPj4gKw0KPj4gICMjIyBwc3IgKEludGVsKQ0KPj4gID4gYD0gTGlzdCBv
ZiAoIGNtdDo8Ym9vbGVhbj4gfCBybWlkX21heDo8aW50ZWdlcj4gfCBjYXQ6PGJvb2xlYW4+IHwg
Y29zX21heDo8aW50ZWdlcj4gfCBjZHA6PGJvb2xlYW4+IClgDQo+PiAgZGlmZiAtLWdpdCBhL3hl
bi9hcmNoL2FybS90cmFwcy5jIGIveGVuL2FyY2gvYXJtL3RyYXBzLmMNCj4+IGluZGV4IDIxOWFi
M2MzZmIuLmQzMGU3OGI0ZDYgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdHJhcHMuYw0K
Pj4gKysrIGIveGVuL2FyY2gvYXJtL3RyYXBzLmMNCj4+IEBAIC0zNCw2ICszNCw3IEBADQo+PiAg
I2luY2x1ZGUgPHhlbi9zeW1ib2xzLmg+DQo+PiAgI2luY2x1ZGUgPHhlbi92ZXJzaW9uLmg+DQo+
PiAgI2luY2x1ZGUgPHhlbi92aXJ0dWFsX3JlZ2lvbi5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vd2Fy
bmluZy5oPg0KPj4gICAgI2luY2x1ZGUgPHB1YmxpYy9zY2hlZC5oPg0KPj4gICNpbmNsdWRlIDxw
dWJsaWMveGVuLmg+DQo+PiBAQCAtNzcsMTIgKzc4LDE5IEBAIHN0YXRpYyBpbnQgZGVidWdfc3Rh
Y2tfbGluZXMgPSA0MDsNCj4+ICAjZGVmaW5lIHN0YWNrX3dvcmRzX3Blcl9saW5lIDQNCj4+ICAj
ZW5kaWYNCj4+ICArc3RhdGljIGNvbnN0IGNoYXIgX19pbml0Y29uc3Qgd2FybmluZ19wbXVfYWNj
ZXNzW10gPQ0KPj4gKyAgICAiV0FSTklORzogUE1VIEFDQ0VTU0VTIEFSRSBOT1cgRU5BQkxFRFxu
Ig0KPj4gKyAgICAiVGhpcyBvcHRpb24gaXMgaW50ZW5kZWQgdG8gYWlkIG1vbml0b3JpbmcgYW5k
IG1lYXN1cmluZ1xuIg0KPj4gKyAgICAidGhlIHBlcmZvcm1hbmNlIGJ5IGFsbG93aW5nIHRoZSBn
dWVzdHMgdG8gYWNjZXNzIFBNVSByZWdpc3RlcnMuXG4iDQo+PiArICAgICJJdCBoYXMgaW1wbGlj
YXRpb25zIG9uIHRoZSBzZWN1cml0eSBvZiB0aGUgc3lzdGVtLlxuIg0KPj4gKyAgICAiUGxlYXNl
ICpETyBOT1QqIHVzZSB0aGlzIGluIHByb2R1Y3Rpb24uXG4iOw0KPj4gKw0KPj4gIGludGVnZXJf
cGFyYW0oImRlYnVnX3N0YWNrX2xpbmVzIiwgZGVidWdfc3RhY2tfbGluZXMpOw0KPj4gICAgc3Rh
dGljIGVudW0gew0KPj4gIAlUUkFQLA0KPj4gIAlOQVRJVkUsDQo+PiAtfSB2d2ZpOw0KPj4gK30g
dndmaSwgcG11X2FjY2VzczsNCj4+ICAgIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX3Z3ZmkoY29u
c3QgY2hhciAqcykNCj4+ICB7DQo+PiBAQCAtOTUsNiArMTAzLDI5IEBAIHN0YXRpYyBpbnQgX19p
bml0IHBhcnNlX3Z3ZmkoY29uc3QgY2hhciAqcykNCj4+ICB9DQo+PiAgY3VzdG9tX3BhcmFtKCJ2
d2ZpIiwgcGFyc2VfdndmaSk7DQo+PiAgK3N0YXRpYyBpbnQgX19pbml0IHBhcnNlX3BtdV9hY2Nl
c3MoY29uc3QgY2hhciAqcykNCj4+ICt7DQo+PiArICAgIGlmICggIXN0cmNtcChzLCAibmF0aXZl
IikgKQ0KPj4gKyAgICAgICAgcG11X2FjY2VzcyA9IE5BVElWRTsNCj4+ICsgICAgZWxzZQ0KPj4g
KyAgICAgICAgcG11X2FjY2VzcyA9IFRSQVA7DQo+PiArDQo+PiArICAgIHJldHVybiAwOw0KPj4g
K30NCj4+ICtjdXN0b21fcGFyYW0oInBtdV9hY2Nlc3MiLCBwYXJzZV9wbXVfYWNjZXNzKTsNCj4+
ICsNCj4+ICtzdGF0aWMgaW50IF9faW5pdCB1cGRhdGVfcG11X2FjY2Vzcyh2b2lkKQ0KPj4gK3sN
Cj4+ICsgICAgaWYgKCBwbXVfYWNjZXNzID09IE5BVElWRSApDQo+PiArICAgIHsNCj4+ICsgICAg
ICAgIFdSSVRFX1NZU1JFRyhSRUFEX1NZU1JFRyhNRENSX0VMMikgJn4gKEhEQ1JfVFBNfEhEQ1Jf
VFBNQ1IpLCBNRENSX0VMMik7DQo+PiArICAgICAgICB3YXJuaW5nX2FkZCh3YXJuaW5nX3BtdV9h
Y2Nlc3MpOw0KPj4gKyAgICB9DQo+PiArDQo+PiArICAgIHJldHVybiAwOw0KPj4gK30NCj4+ICtf
X2luaXRjYWxsKHVwZGF0ZV9wbXVfYWNjZXNzKTsNCj4+ICsNCj4+ICByZWdpc3Rlcl90IGdldF9k
ZWZhdWx0X2hjcl9mbGFncyh2b2lkKQ0KPj4gIHsNCj4+ICAgICAgcmV0dXJuICAoSENSX1BUV3xI
Q1JfQlNVX0lOTkVSfEhDUl9BTU98SENSX0lNT3xIQ1JfRk1PfEhDUl9WTXwNCj4gDQo+IENoZWVy
cywNCj4gDQo+IC0tIA0KPiBKdWxpZW4gR3JhbGwNCg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:18:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:18:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176386.320971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQ89-0004F6-21; Wed, 01 Sep 2021 13:18:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176386.320971; Wed, 01 Sep 2021 13:18:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQ88-0004Ez-UN; Wed, 01 Sep 2021 13:18:16 +0000
Received: by outflank-mailman (input) for mailman id 176386;
 Wed, 01 Sep 2021 13:18:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQ87-0004EZ-MM; Wed, 01 Sep 2021 13:18:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQ87-0002d1-GT; Wed, 01 Sep 2021 13:18:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQ87-0004zh-7I; Wed, 01 Sep 2021 13:18:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQ87-0004kd-6o; Wed, 01 Sep 2021 13:18:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pO8xrE9MD3M/RlePB0hCndllLyMICn5OEKnGgj0YIoI=; b=ZEiJdBBsogdWFJed7sqYZCiCBU
	/PyHaEwuQvVmkeoca/I3bCEmC13rRInFzNygSz9fUmdjI4PEU1D8gwccXK/5QI20UFQNq04OkW6cB
	UtJmbEvb7zUgBQMXMj247We7IjCMcwYv2+zKqpyS0iSCKhGxXYv2SSKHiwy+48muUtmg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164679-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-upstream-unstable test] 164679: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-upstream-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:guest-start/debianhvm.repeat:fail:heisenbug
    qemu-upstream-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-upstream-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=b6e539830bf45e2d7a6bd86ddfdf003088b173b0
X-Osstest-Versions-That:
    qemuu=136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 13:18:15 +0000

flight 164679 qemu-upstream-unstable real [real]
flight 164696 qemu-upstream-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164679/
http://logs.test-lab.xenproject.org/osstest/logs/164696/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 18 guest-start/debianhvm.repeat fail pass in 164696-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 163309
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 163309
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163309
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 163309
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 163309
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 163309
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 163309
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                b6e539830bf45e2d7a6bd86ddfdf003088b173b0
baseline version:
 qemuu                136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823

Last test of basis   163309  2021-07-05 10:10:06 Z   58 days
Testing same since   164679  2021-08-31 11:08:25 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    <ruixin.bao@ibm.com>
  Aaron Lindsay <aaron@os.amperecomputing.com>
  Ahmed Abouzied <email@aabouzied.com>
  Akihiko Odaki <akihiko.odaki@gmail.com>
  Alberto Garcia <berto@igalia.com>
  Aleksandar Rikalo <Aleksandar.Rikalo@syrmia.com>
  Alessandro Di Federico <ale@rev.ng>
  Alex Bennée <alex.bennee@linaro.org>
  Alex Williamson <alex.williamson@redhat.com>
  Alexander Bulekov <alxndr@bu.edu>
  Alexander Graf <agraf@csgraf.de>
  Alexander Wagner <alexander.wagner@ulal.de>
  Alexandre Iooss <erdnaxe@crans.org>
  Alexey Kardashevskiy <aik@ozlabs.ru>
  Alexey Neyman <stilor@att.net>
  Ali Shirvani <alishir@routerhosting.com>
  Alistair Francis <alistair.francis@wdc.com>
  Alyssa Ross <hi@alyssa.is>
  Andreas Schwab <schwab@suse.de>
  Andrew Jones <drjones@redhat.com>
  Andrew Melnychenko <andrew@daynix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arseny Krasnov <arseny.krasnov@kaspersky.com>
  Atish Patra <atish.patra@wdc.com>
  Axel Heider <axelheider@gmx.de>
  BALATON Zoltan <balaton@eik.bme.hu>
  Basil Salman <basil@daynix.com>
  Basil Salman <basil@redhat.com>
  Basil Salman <bsalman@daynix.com>
  Basil Salman <bsalman@redhat.com>
  Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
  Benjamin Herrenschmidt <benh@kernel.crashing.org>
  Bharata B Rao <bharata@linux.ibm.com>
  Bibo Mao <maobibo@loongson.cn>
  Bin Meng <bin.meng@windriver.com>
  Bin Meng <bmeng.cn@gmail.com>
  Brad Smith <brad@comstyle.com>
  Brijesh Singh <brijesh.singh@amd.com>
  Bruno Larsen (billionai) <bruno.larsen@eldorado.org.br>
  Cai Huoqing <caihuoqing@baidu.com>
  Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
  Changbin Du <changbin.du@gmail.com>
  Chenyi Qiang <chenyi.qiang@intel.com>
  Chetan Pant <chetan4windows@gmail.com>
  Cho, Yu-Chen <acho@suse.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Christina Wang <christina.wang@windriver.com>
  Claudio Fontana <cfontana@suse.de>
  Cleber Rosa <crosa@redhat.com>
  Coiby Xu <coiby.xu@gmail.com>
  Cole Robinson <crobinso@redhat.com>
  Connor Kuehl <ckuehl@redhat.com>
  Corey Minyard <cminyard@mvista.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Le Goater <clg@kaod.org>
  Damien Goutte-Gattat <dgouttegattat@incenp.org>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Edmondson <david.edmondson@oracle.com>
  David Gibson <david@gibson.dropbear.id.au>
  David Hildenbrand <david@redhat.com>
  David Michael <fedora.dm0@gmail.com>
  Dennis Wölfing <denniswoelfing@gmx.de>
  Dmitry Voronetskiy <davoronetskiy@gmail.com>
  Dmitry Voronetskiy <vda1999@yandex.ru>
  Dov Murik <dovmurik@linux.ibm.com>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Dylan Jhong <dylan@andestech.com>
  Edgar E. Iglesias <edgar.iglesias@xilinx.com>
  Eduardo Habkost <ehabkost@redhat.com>
  Eduardo Habkost <ehabkost@redhat.com> for memory backend and machine core
  Eduardo Otubo <otubo@redhat.com>
  Emanuele Giuseppe Esposito <eesposit@redhat.com>
  Emmanuel Blot <emmanuel.blot@sifive.com>
  Eric Auger <eric.auger@redhat.com>
  Eric Blake <eblake@redhat.com>
  Eric Farman <farman@linux.ibm.com>
  Fabian Ebner <f.ebner@proxmox.com>
  Fabiano Rosas <farosas@linux.ibm.com>
  Fabrice Fontaine <fontaine.fabrice@gmail.com>
  Feng Lin <linfeng23@huawei.com>
  Filip Vidojevic <filip.vidojevic@syrmia.com>
  Finn Thain <fthain@linux-m68k.org>
  Frank Chang <frank.chang@sifive.com>
  Frederic Konrad <frederic.konrad@adacore.com>
  Frederic Konrad <konrad@adacore.com>
  Gan Qixin <ganqixin@huawei.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Giuseppe Musacchio <thatlemon@gmail.com>
  Gollu Appalanaidu <anaidu.gollu@samsung.com>
  Greg Kurz <groug@kaod.org>
  Guenter Roeck <linux@roeck-us.net>
  Han Han <hhan@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Heinrich Schuchardt <xypron.glpk@gmx.de>
  hnick@vmware.com <hnick@vmware.com>
  Hongzheng-Li <Ethan.Lee.QNL@gmail.com>
  Hou Weiying <weiying_hou@outlook.com>
  Hubert Jasudowicz <hubert.jasudowicz@gmail.com>
  Hyman <huangy81@chinatelecom.cn>
  Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
  Ian Jackson <ian.jackson@eu.citrix.com>
  Igor Druzhinin <igor.druzhinin@citrix.com>
  Igor Mammedov &lt;<a href="mailto:imammedo@redhat.com" target="_blank">imammedo@redhat.com</a>&gt;<br>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Dryomov <idryomov@gmail.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jade Fink <qemu@jade.fyi>
  Jagannathan Raman <jag.raman@oracle.com>
  James Bottomley <jejb@linux.ibm.com>
  Jamie Iles <jamie@nuviainc.com>
  Jason Andryuk <jandryuk@gmail.com>
  Jason Thorpe <thorpej@me.com>
  Jason Wang <jasowang@redhat.com>
  Jean-Philippe Brucker <jean-philippe@linaro.org>
  Jinhua Cao <caojinhua1@huawei.com>
  Joe Komlodi <joe.komlodi@xilinx.com>
  Joel Stanley <joel@jms.id.au>
  Joelle van Dyne <j@getutm.app>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  John Sucaet <john.sucaet@ekinops.com>
  Jonathan Albrecht <jonathan.albrecht@linux.vnet.ibm.com>
  jonathan.albrecht <jonathan.albrecht@linux.vnet.ibm.com>
  Jose Martins <josemartins90@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Juergen Lock <nox@FreeBSD.org>
  Julia Suvorova <jusual@redhat.com>
  Kamil Rytarowski <kamil@NetBSD.org>
  Katsuhiro Ueno <uenobk@gmail.com>
  Keith Busch <kbusch@kernel.org>
  Keqian Zhu <zhukeqian1@huawei.com>
  Kevin Wolf <kwolf@redhat.com>
  Khem Raj <raj.khem@gmail.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Kirti Wankhede <kwankhede@nvidia.com>
  Kit Westneat <kit.westneat@gmail.com>
  Kito Cheng <kito.cheng@sifive.com>
  Klaus Jensen <k.jensen@samsung.com>
  KONRAD Frederic <frederic.konrad@adacore.com>
  Konstantin Nazarov <mail@knazarov.com>
  Kostiantyn Kostiuk <konstantin@daynix.com>
  Kunkun Jiang <jiangkunkun@huawei.com>
  Lara Lazier <laramglazier@gmail.com>
  Laszlo Ersek <lersek@redhat.com>
  Laurent Vivier <laurent@vivier.eu>
  Laurent Vivier <lvivier@redhat.com>
  Lei Rao <lei.rao@intel.com>
  Leonardo Bras <leobras.c@gmail.com>
  Li Qiang <liq3ea@163.com>
  Li Qiang <liq3ea@gmail.com>
  Li Zhijian <lizhijian@cn.fujitsu.com>
  Liren Wei <lrwei@bupt.edu.cn>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
  Luis Fernando Fujita Pires <luis.pires@eldorado.org.br>
  Luis Pires <luis.pires@eldorado.org.br>
  Lukas Jünger <lukas.juenger@greensocs.com>
  Lukas Straub <lukasstraub2@web.de>
  Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
  Mahesh Salgaonkar <mahesh@linux.ibm.com>
  Mahmoud Mandour <ma.mandourr@gmail.com>
  Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
  maobibo <maobibo@loongson.cn>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
  Marcel Apfelbaum <marcel@redhat.com>
  Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
  Mark Johnston <markj@FreeBSD.org>
  Mark Nelson <mdnelson8@gmail.com>
  Markus Armbruster <armbru@redhat.com>
  Matheus Ferst <matheus.ferst@eldorado.org.br>
  Mathieu Poirier <mathieu.poirier@linaro.org>
  Matthew Rosato <mjrosato@linux.ibm.com>
  Matus Kysel <mkysel@tachyum.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Max Filippov <jcmvbkbc@gmail.com>
  Max Reitz <mreitz@redhat.com>
  Maxim Levitsky <mlevitsk@redhat.com>
  Maxim Uvarov <maxim.uvarov@linaro.org>
  Michael Forney <mforney@mforney.org>
  Michael Rolnik <mrolnik@gmail.com>
  Michael Roth <michael.roth@amd.com>
  Michael S. Tsirkin <mst@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michael Walle <michael@walle.cc>
  Michal Privoznik <mprivozn@redhat.com>
  Miroslav Rezanina <mrezanin@redhat.com>
  Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
  Myriad-Dreamin <camiyoru@gmail.com>
  Nicholas Piggin <npiggin@gmail.com>
  Nick Hudson <hnick@vmware.com>
  Niklas Cassel <niklas.cassel@wdc.com>
  Nolan Leake <nolan@sigbus.net>
  Olaf Hering <olaf@aepfle.de>
  Or Ozeri <oro@il.ibm.com>
  Owen Anderson <oanderso@google.com>
  Padmakar Kalghatgi <p.kalghatgi@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Paolo Montesel <babush@rev.ng>
  Patrick Venture <venture@google.com>
  Paul A. Clarke <pc@us.ibm.com>
  Paul Durrant <paul@xen.org>
  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
  Pavel Pisa <pisa@cmp.felk.cvut.cz>
  Peng Liang <liangpeng10@huawei.com>
  Peter Collingbourne <pcc@google.com>
  Peter Krempa <pkrempa@redhat.com>
  Peter Lieven <pl@kamp.de>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Qiang Liu <cyruscyliu@gmail.com>
  Ram Pai <linuxram@us.ibm.com>
  Rao, Lei <lei.rao@intel.com>
  Raphael Norwitz <raphael.norwitz@nutanix.com>
  Ravi Bangoria <ravi.bangoria@linux.ibm.com>
  Rebecca Cran <rebecca@nuviainc.com>
  Reinoud Zandijk <reinoud@NetBSD.org>
  Ricardo Koller <ricarkol@google.com>
  Ricgard Henderson <richard.henderson@linaro.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Richard Zak <richard.j.zak@gmail.com>
  Robert Hoo <robert.hu@linux.intel.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Kagan <rvkagan@yandex-team.ru>
  Roman Kapl <rka@sysgo.com>
  Ruimei Yan <ruimei.yan@windriver.com>
  Ruinland ChuanTzu Tsai <ruinland@andestech.com>
  Samuel Thibault <samuel.thibault@ens-lyon.org>
  Serge Guelton <sguelton@redhat.com>
  Sergio Lopez <slp@redhat.com>
  Shivaprasad G Bhat <sbhat@linux.ibm.com>
  Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Stefan Reiter <s.reiter@proxmox.com>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Stephen Long <steplong@quicinc.com>
  Tarun Gupta <targupta@nvidia.com>
  Taylor Simpson <tsimpson@quicinc.com>
  Thomas Huth <thuth@redhat.com>
  Titus Rwantare <titusr@google.com>
  Tom Lendacky <thomas.lendacky@amd.com>
  Ulrich Weigand <ulrich.weigand@de.ibm.com>
  Vaibhav Jain <vaibhav@linux.ibm.com>
  VictorV (Kunlun Lab) <vv474172261@gmail.com>
  Vijai Kumar K <vijai@behindbytes.com>
  Viktor Ashirov <vashirov@redhat.com>
  Vincent Bernat <vincent@bernat.ch>
  Viresh Kumar <viresh.kumar@linaro.org>
  Vitaly Kuznetsov <vkuznets@redhat.com>
  Vivek Goyal <vgoyal@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Volker Rümelin <vr_qemu@t-online.de>
  Wainer dos Santos Moschetta <wainersm@redhat.com>
  Warner Losh <imp@bsdimp.com>
  Wei Liu <wei.liu2@citrix.com>
  Wei Wang <wei.w.wang@intel.com>
  Willian Rampazzo <willianr@redhat.com>
  Xie Yongji <xieyongji@bytedance.com>
  Xingang Wang <wangxingang5@huawei.com>
  Xuzhou Cheng <xuzhou.cheng@windriver.com>
  YAMAMOTO Takashi <yamamoto@midokura.com>
  Yang Zhong <yang.zhong@intel.com>
  Yasuo Kuwahara <kwhr00@gmail.com>
  Yuri Benditovich <yuri.benditovich@daynix.com>
  Yuval Shaia <yuval.shaia.ml@gmail.com>
  Zenghui Yu <yuzenghui@huawei.com>
  Zhang Chen <chen.zhang@intel.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>
  Ziqiao Kong <ziqiaokong@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   136c34c9bc..b6e539830b  b6e539830bf45e2d7a6bd86ddfdf003088b173b0 -> master


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:20:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:20:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176394.320985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQAZ-0005dc-JT; Wed, 01 Sep 2021 13:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176394.320985; Wed, 01 Sep 2021 13:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQAZ-0005dV-Ga; Wed, 01 Sep 2021 13:20:47 +0000
Received: by outflank-mailman (input) for mailman id 176394;
 Wed, 01 Sep 2021 13:20:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qyqW=NX=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mLQAX-0005dN-Rx
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 13:20:45 +0000
Received: from mail-wr1-x429.google.com (unknown [2a00:1450:4864:20::429])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b5b09572-24bc-452d-87bf-2c25f69caa2f;
 Wed, 01 Sep 2021 13:20:44 +0000 (UTC)
Received: by mail-wr1-x429.google.com with SMTP id u9so4458220wrg.8
 for <xen-devel@lists.xen.org>; Wed, 01 Sep 2021 06:20:44 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id h11sm25481339wrx.9.2021.09.01.06.20.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 01 Sep 2021 06:20:43 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 6C4D31FF96;
 Wed,  1 Sep 2021 14:20:42 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5b09572-24bc-452d-87bf-2c25f69caa2f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=y/0tGsa7zxs1Lge6n3O6ldiXuhSjzQntOAObjxG3wD8=;
        b=gAnfhwBM6t2sfmWr0LXDnlOXCmfqqkhQvg8PKhZg0mwsVmibDpiyA0ycZRCaPZM/o9
         jWOSV+puCL7VGYR9cJuMUCQlMqfLPQlGiUk4SK9A0wFvpOSsKIHMhHZqu/KF8pArEMJ6
         /K2zuth1vsa6Noddrq4zuOPYXKJkZ2Q9hgUfk2iqtVSf3BP0pVK0BmuZpJghk56qVwZA
         KVVzErNnIv1DqO8RRtIGGKsQZL1U0Fo/GZjHHXlX/1c8vF2xbQQ2E4gI1nVjtDGHg3K0
         gW/ehRFVFcBd1onW2djYp7qR7jy11EXMYIKqt/C/T3/bAZglF4+LSDewaVtD3Rnk96w+
         C7Jg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=y/0tGsa7zxs1Lge6n3O6ldiXuhSjzQntOAObjxG3wD8=;
        b=Nb5mqR5TsAz+SC1PjkXWUrHYg7+RYIk2PofavxzZZ1W9z7k7DyBIdtI9aTlSci0TB7
         w8t1qSFHcHfg0Wd9Tq2jcnOy2Xz7IfFH9r3JetnYwMqHRDb66kjuPrhcnXtx/UfZhD7h
         8gAyvV9hCMoX2x67efoVVNilx6cTOonqBF1BdCC8cCRUhScHRYES2nZ9crHt9aSwxhFP
         91HZB7plrUKY4oQH24/CRhOQ7Rl8aOF+DQGtd/kO+kup8bxHyUMNKwiCkjyu7+7ChprY
         QvlrizXqekLy6BCTjxcC8uGmcJoSt+1MftQlVXdLmf92RYVE+iN/U1OrrayTL7T0gh3c
         tMrA==
X-Gm-Message-State: AOAM5316lkUVkwVO/rO9hvknS0Y5pkVZv6uNUiUh6Yw9tgWnf+gODKmJ
	pTvSvzZ9GexVS8CvljiORcLmVA==
X-Google-Smtp-Source: ABdhPJxCevrGYCx7XFVJhI0D2nblAkpnQCi5PGoMrSDHbafJqS3oT1ctZ1duD7JVNhkkNRwTAvhNgg==
X-Received: by 2002:adf:b7cd:: with SMTP id t13mr17597181wre.63.1630502444042;
        Wed, 01 Sep 2021 06:20:44 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Arnd
 Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar
 <viresh.kumar@linaro.org>, AKASHI Takahiro <takahiro.akashi@linaro.org>,
 Stefano Stabellini <stefano.stabellini@xilinx.com>, Jan Kiszka
 <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>,
 pratikp@quicinc.com, Srivatsa Vaddagiri <vatsa@codeaurora.org>,
 Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier
 <mathieu.poirier@linaro.org>, Wei.Chen@arm.com, olekstysh@gmail.com,
 Oleksandr_Tyshchenko@epam.com, Bertrand.Marquis@arm.com,
 Artem_Mygaiev@epam.com, julien@xen.org, jgross@suse.com, paul@xen.org,
 xen-devel@lists.xen.org, Elena Afanasova <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Date: Wed, 01 Sep 2021 13:53:34 +0100
In-reply-to: <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
Message-ID: <875yvkh1p1.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Stefan Hajnoczi <stefanha@redhat.com> writes:

> [[PGP Signed Part:Undecided]]
> On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
>> > Could we consider the kernel internally converting IOREQ messages from
>> > the Xen hypervisor to eventfd events? Would this scale with other kern=
el
>> > hypercall interfaces?
>> >=20
>> > So any thoughts on what directions are worth experimenting with?
>>=20=20
>> One option we should consider is for each backend to connect to Xen via
>> the IOREQ interface. We could generalize the IOREQ interface and make it
>> hypervisor agnostic. The interface is really trivial and easy to add.
>> The only Xen-specific part is the notification mechanism, which is an
>> event channel. If we replaced the event channel with something else the
>> interface would be generic. See:
>> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm=
/ioreq.h#L52
>
> There have been experiments with something kind of similar in KVM
> recently (see struct ioregionfd_cmd):
> https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613=
828727.git.eafanasova@gmail.com/

Reading the cover letter was very useful in showing how this provides a
separate channel for signalling IO events to userspace instead of using
the normal type-2 vmexit type event. I wonder how deeply tied the
userspace facing side of this is to KVM? Could it provide a common FD
type interface to IOREQ?

As I understand IOREQ this is currently a direct communication between
userspace and the hypervisor using the existing Xen message bus. My
worry would be that by adding knowledge of what the underlying
hypervisor is we'd end up with excess complexity in the kernel. For one
thing we certainly wouldn't want an API version dependency on the kernel
to understand which version of the Xen hypervisor it was running on.

>> There is also another problem. IOREQ is probably not be the only
>> interface needed. Have a look at
>> https://marc.info/?l=3Dxen-devel&m=3D162373754705233&w=3D2. Don't we als=
o need
>> an interface for the backend to inject interrupts into the frontend? And
>> if the backend requires dynamic memory mappings of frontend pages, then
>> we would also need an interface to map/unmap domU pages.
>>=20
>> These interfaces are a lot more problematic than IOREQ: IOREQ is tiny
>> and self-contained. It is easy to add anywhere. A new interface to
>> inject interrupts or map pages is more difficult to manage because it
>> would require changes scattered across the various emulators.
>
> Something like ioreq is indeed necessary to implement arbitrary devices,
> but if you are willing to restrict yourself to VIRTIO then other
> interfaces are possible too because the VIRTIO device model is different
> from the general purpose x86 PIO/MMIO that Xen's ioreq seems to
> support.

It's true our focus is just VirtIO which does support alternative
transport options however most implementations seem to be targeting
virtio-mmio for it's relative simplicity and understood semantics
(modulo a desire for MSI to reduce round trip latency handling
signalling).

>
> Stefan
>
> [[End of PGP Signed Part]]


--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:22:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:22:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176400.320995 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQCE-0006Tl-VB; Wed, 01 Sep 2021 13:22:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176400.320995; Wed, 01 Sep 2021 13:22:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQCE-0006Te-SE; Wed, 01 Sep 2021 13:22:30 +0000
Received: by outflank-mailman (input) for mailman id 176400;
 Wed, 01 Sep 2021 13:22:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gczT=NX=gmail.com=jandryuk@srs-us1.protection.inumbo.net>)
 id 1mLQCD-0006TY-LR
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 13:22:29 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cf2850d5-6dd5-40f8-bffe-1da694fcf464;
 Wed, 01 Sep 2021 13:22:28 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id t19so4487310lfe.13;
 Wed, 01 Sep 2021 06:22:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf2850d5-6dd5-40f8-bffe-1da694fcf464
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=jaFQ1dJz/0jVDnUJDux1CudavpmIQDH+qRgNmOHy2kw=;
        b=V08N3wY0tEFQVk3cnI/b65MC6y/NleF6UKnkKSF4FljmBjrUCVrMazguZLA7LdjAmT
         +ukjleA/aXsaghg8GdsMYuAgm5c+MPEAreIaSQmdzDGpoYxtukbwGOq6AUzSQwEpTY7+
         AeRpjO4diPCC57slSCKz6Up8XUfN5G55R75Z0ghi/XJHYQzGmDGp0k5rosewss7I0J/f
         I3lzZ16I+iwkRH8e8zHzRSxcJ5rmOUhqIQSEZfM6y6/Rh5VlbXhlnuTgeV0C1Ud4mU1N
         SF3wfplIkh8EJ7NydDk2HhzRMfGvJ4zuckdXpFbwpiK3mBrr//Kfse12BEWAhzVvi6fQ
         5kXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=jaFQ1dJz/0jVDnUJDux1CudavpmIQDH+qRgNmOHy2kw=;
        b=JrpFxVaBMe8DrcL/LAJ8shCaXJ4Ue23W3hoyGVRIfapRy/ZvbNVLLlSjeZSgktxEDL
         Vh2Sp7eI/S1m6J+aOzTPhK/OjKSQv0DUsS8VLE/ER2OXJb5e7DrDEUAfcWFKgCUWBIJ9
         s/Kla6Em+S3TDU16XXV5I2ZSffiv70n7ZR6GPpIqH7YWe4QeKI1cBHkGYPUgPG9WhT98
         qUpcueZVT0jFl0KmEqCN6AjuV272lFqDw+t4pBkERk2Z1dpyDJJF6Upf9ouXhwuLQxhT
         NxWj42tEw9w3UQ1iaCguh0upNstAu2iIOjf088UvzbfhRVoDPW5yNR+VLsjcSeD6swWs
         81Xg==
X-Gm-Message-State: AOAM531FlJMmjORy+uaUxkviFED/ekdPu42SfqE9CgrRoMpouTj6ERDZ
	+BbtjZgMJBUCUdkKkArhpvZNhTj6cG5Ld3/jo3Y=
X-Google-Smtp-Source: ABdhPJzjhjRenLVlNS1HJ0Z1WF422vUK6cSHDYkcons243OdgRRX3wOE4F63Tq0W2CKrBteyFQ6T9OAR2SBTJbX1W8Y=
X-Received: by 2002:ac2:5d4a:: with SMTP id w10mr26376733lfd.529.1630502546778;
 Wed, 01 Sep 2021 06:22:26 -0700 (PDT)
MIME-Version: 1.0
References: <E1mLMZy-0006TJ-Rw@xenbits.xenproject.org>
In-Reply-To: <E1mLMZy-0006TJ-Rw@xenbits.xenproject.org>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 1 Sep 2021 09:22:14 -0400
Message-ID: <CAKf6xpsAz7oPX+PdZokm+BAqreqL8rnrAd9UYvdz=F0M-bcMUQ@mail.gmail.com>
Subject: Re: Xen Security Advisory 378 v3 (CVE-2021-28694,CVE-2021-28695,CVE-2021-28696)
 - IOMMU page mapping issues on x86
To: "Xen.org security team" <security@xen.org>
Cc: xen-announce@lists.xen.org, xen-devel@lists.xen.org, 
	xen-users@lists.xen.org, oss-security@lists.openwall.com, 
	"Xen.org security team" <security-team-members@xen.org>
Content-Type: text/plain; charset="UTF-8"

On Wed, Sep 1, 2021 at 5:34 AM Xen.org security team <security@xen.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>  Xen Security Advisory CVE-2021-28694,CVE-2021-28695,CVE-2021-28696 / XSA-378
>                                    version 3
>
>                    IOMMU page mapping issues on x86
>
> UPDATES IN VERSION 3
> ====================
>
> Warn about dom0=pvh breakage in Resolution section.
>
> ISSUE DESCRIPTION
> =================
>
> Both AMD and Intel allow ACPI tables to specify regions of memory
> which should be left untranslated, which typically means these
> addresses should pass the translation phase unaltered.  While these
> are typically device specific ACPI properties, they can also be
> specified to apply to a range of devices, or even all devices.
>
> On all systems with such regions Xen failed to prevent guests from
> undoing/replacing such mappings (CVE-2021-28694).

Hi,

Is there a way to identify if a system's ACPI tables have untranslated
regions?  Does it show up in xen or linux dmesg or can it be
identified in sysfs?

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:39:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:39:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176414.321021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQSo-0008Kt-Ph; Wed, 01 Sep 2021 13:39:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176414.321021; Wed, 01 Sep 2021 13:39:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQSo-0008Km-MA; Wed, 01 Sep 2021 13:39:38 +0000
Received: by outflank-mailman (input) for mailman id 176414;
 Wed, 01 Sep 2021 13:39:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLQSo-0008Kg-1K
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 13:39:38 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0c2ac658-0b2a-11ec-ade2-12813bfff9fa;
 Wed, 01 Sep 2021 13:39:37 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2056.outbound.protection.outlook.com [104.47.4.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-fbMU7R2YNQioOZLaDSkUtQ-1; Wed, 01 Sep 2021 15:39:35 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 1 Sep
 2021 13:39:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 13:39:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0279.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 13:39:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c2ac658-0b2a-11ec-ade2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630503576;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=6gOIlwfeDw151rTxi3HU+tBg3DOPTUR73ZuMPiIZHoc=;
	b=Z482f9E7EEmo6MVfw58J/4W1+m1CYfp6Nqr/DXVVLrcKisF7NavRafDuwn9T+trw6AOtvd
	L6XSiLCr85ZBNOfiYGxCyUgB5thTvk0wPluICpB5SemewSXsl8qyPOq4HluupM+5GJStkO
	JZ+N+TCmE/U5XgaD1fl9hGnw/9VDY6U=
X-MC-Unique: fbMU7R2YNQioOZLaDSkUtQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JBhcJGpW5huymKKurh1Y480OXWQYKrpZ0ZPCfYX/2LQcj/OTZxGIWy0BhYkNF26PqQQxLraxurcPWSFrsyE3MOHV/x4Hrbd7K5L1qa7Bl7Xf1b/I7F8+aUuxoe9FmvNly07BKavMIdm9QzuLzcu8WY4ly+KLzvI5NawY4lBjdP2htQC8XCd9fA+bOMbu9yRUbvSYziTHM8zQxQsP0dn42k0PBsgGzjUieeSdvNWaWV1U7iWfCyhIF/iVau1EgDwKL2oG23hcmfRCXaJ47TSfhhhs91QL7srqbRPrfvPre/q5uTU65SPLl1feU9tN/+uxXvwKolew9rgfkaV8gq61RA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6gOIlwfeDw151rTxi3HU+tBg3DOPTUR73ZuMPiIZHoc=;
 b=CRp3Ye82S0aOXdlC4tp4jCzjyS9ZmS92mF73ZVYdbNG6BqTgbMRzU2ylYQFyCcflU1qpQTUky9sQNpJBdzjesEtAjWjOQTIIxVqUKKEv2rd/8EOgm/9C2uKF7USILMQN69fso0fz7qKWyVtbP4DytqfdEkWV03gKBgtafki0piCDK7KyS8bDlpW3D7QVnYBAM5Mj/K7KfCmdOVtnaZnAmt16D8wdQtCcFdh0R3bii38FmlImXRzUDsT6UDO249NL+8uoVjZ3kh/5kBOZdWDJT3ONSJ9Ehfke8o57BHUuFwcQe1dRAg/1KdpS/aD8kQDHnT0DxlTMTK4bz4Gd9L5E0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Andrew Cooper <andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: vcpu_show_execution_state() difference between Arm and x86
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
Date: Wed, 1 Sep 2021 15:39:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0279.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e96cb151-17a3-4cd7-69e4-08d96d4dee7f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23350E89D1744C4B0A104C1CB3CD9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+++OrTw4EWygWwWI14Rtt7qgQeSSx4f6jECIqLjA/sGU65yRp+VV+o/k8wy66ExXG1SHJKDy5O30ULidjBv0cdgGQlcHUCi90mC2HsQELaD3uEaVB2zNI4zqzK3twtfWKMk6zroiSqwMeOYtqkTzpvemuksW1/VKAVUP6VuiBksUv2WL0DgNus+1nenMx/nsvS3FCNssgSZX9I2TlDr0bF3YJehyRukPUFhyR9oSq5+S6tqq4yOKeZ7RTrcXpWBuWFXgAXTaTLt5sZXZ04bvZ+8diHxSUhSVZUlUDZqK4BBLsdt1eI6b5ogiGKUdW9292vcQjczdcoLXHQihowKdCWEmZR8ocRI1CWX4BLEfO17e0lfULpmhBmQn/Q3v/GsXcd6jsjlHiVOfdrTVNjYkDU+kFwkkzC7fe1MAGARRcmObUFwQQEzOnNiE2XxhvYj94GmP98qpcbgAFgvWpYwDoe2mJcUoS2Ft2+NyOKAg4DLlacmYrKabLhYBB3D2q6d9qXxZT+6ITMQ3Y/wUeFFeTwdN0MUNov3BbmaA07kFjpp4eEonE6gzz8603d/l/N/PuQ4pH6+A4Y70pKwZIZog7TUTLHWRrnLboQr2cOfUqRCyzLrxIQc6i4Q1TnSVvY/gE3nAVy2cXC+ROEYnzy+jki4Pue4r3TxXE4n+SwW2FMLh22t/jp3aFiqpYr4LF3o3n/tGBGf0F7YvtU4nPb5oPTX219IJrNoPANekTfyjqTM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(396003)(366004)(346002)(39860400002)(4326008)(36756003)(5660300002)(478600001)(2906002)(31696002)(31686004)(8676002)(38100700002)(956004)(86362001)(2616005)(4744005)(6486002)(66946007)(83380400001)(66556008)(66476007)(8936002)(316002)(16576012)(110136005)(26005)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cVhjT20wOWl0bEFvSHl1eFNXYUk0L3MzWVlNODJ5VTRNbUNTcGJ0QzB6bjZE?=
 =?utf-8?B?SFpUL0JTOU1WYUNzN1NUS05jUE1UNXpLUmE0UFRhRVJtOEpUVXM1TXUyVUV4?=
 =?utf-8?B?NTFVSDNkc1hyTVVWNjlqeDZBRmNaZnhvaC95cVdJcklzRTZKdkJnQXg3akNn?=
 =?utf-8?B?ZjVwNkt6MCtWRnQ4NUNlUFRCVk1yV3J2b2VDeHBxcVBzQjdiQ2V0SGhRaFVS?=
 =?utf-8?B?T2M4azlMOUlQVmFLbkV1Q3ZGcXcxeFVHN21OalVhSUZ4UCsza0w2WndqL2JW?=
 =?utf-8?B?NDAxNlhpRU5XYy8vcWN2RnhEeFR0Z3NXYlhHeHJuV2hNaXZVcUY0N0tSa2dU?=
 =?utf-8?B?WmRQRjZ0WkJYMitlOVdDZnUxaEdITjQ1YjkyOGZ5ZGFSUUcycy9qUkFQQkQ0?=
 =?utf-8?B?ZEtGYkhtRWZZbDlVNkZzMHRxVEZZM1BETWpIK25Od21JazdwY2ZGMVBKMU9J?=
 =?utf-8?B?cWl2a2F1REZFazV3VzQ0OVBIR0JWODFjNHIwYngxTk00Z2VZdU1jSmk2eHpj?=
 =?utf-8?B?eEZTLy90dkNyQ0FUU0xuYktscCtIazkxZ3YzcnczN2w3cmVSZmpEdzVWSGRN?=
 =?utf-8?B?V3NsSmk2ZmJ4TDlRU0F5YXd0UThRYW0xb2xJdU00aGRSaG5Nd3FJeTRoV2lj?=
 =?utf-8?B?VmFxT1c3QkZqRTYxeHRkekFSeis0LzUxQVpmL3FYZHlVSmE5SnYxMFBxdEhK?=
 =?utf-8?B?VXVNMDZnWGxEVjhhSGdITnhCWlZmdDdtTEEzYUV2ZkZ2ZmhRZEh4QXZZMEF5?=
 =?utf-8?B?eUsyeTl0bWtFck1MMzJSY2R1UXlaOEZ0Q1RGQTBmenhDdGFMblRGbHBvV25z?=
 =?utf-8?B?Q25hR3lyQjZTOHF6R2xjVFdzZzBWYkE3QjRVZU1rb25lMStyMjNHdFFYK3FX?=
 =?utf-8?B?anBZbzliRms5ZTJub3ZlaTQ4a2FJYytSQThiamdnUllicjZNMmsyR2hYb01y?=
 =?utf-8?B?Nk5BMXRiTUFLbEt3blk3eUsvWkpzK2xlbXhvMlhQcDdVaEw4TGxmWVUyUnRp?=
 =?utf-8?B?STlJSkpGVWpLTDNXVjFtOE10dkRVRUhDbTVwSGp6Q1hPamdCaXZhZHkzMmdZ?=
 =?utf-8?B?SWFwSXhVTjI0Z0x1STNMNUUxZlpWaXJIZmZkcFUyQTBySDRvTFNaVnd1NUFR?=
 =?utf-8?B?VFBrSDRXaFpzZWZHalA2ZGtBQjZRVFNMc1FyWmxtcURWbk9nT1UrcWNxY2dN?=
 =?utf-8?B?RTdsK1R1UlNQa0svbnBZUlhZNGE3dENDblkrT0E3NU5uWFUrV0RxcEVJN0pw?=
 =?utf-8?B?VXdlMzM5ZzRFZDA3VGtTWkZIa1VlSkk3VnNmRXJ0VWY0c3N6L3pySEtKbVBi?=
 =?utf-8?B?WHBZNzJZRGx6ZjhpR2ZPVVN1UWlsZGk4dzNZazdmSmJ4YUYwMkl5SzdoVjFs?=
 =?utf-8?B?RXhsSVhibTN0UzRVV2tuOXMvTVlldG5ReWdNSnBTYXJ5ZTlQRW5kKzRWZDkv?=
 =?utf-8?B?RXMweVRrcnZ1MnREdnNZcUVyRGZYRHlYalQ3T1M4OWg2OUQ4VUNOcjE4RFRI?=
 =?utf-8?B?a0w0Rm94TW5EelhVWWhyUGRTWnpCbXNiV3d0d0RJYW1SSnNrMG90MG5CenBz?=
 =?utf-8?B?UFhwNmhwa0hjOGRZZjZkVEdBd2VpSG5ENnBMRThUalIyaGNIMU5xNkxWazhv?=
 =?utf-8?B?bnN4Zm9JSEJvdGxmb3B5UmlvOHJSVXhmOVQwaGZpSlpSVGpUZEthaWdYdDNi?=
 =?utf-8?B?Z2tKenczRk5qVGtoRlFsQkdOdCtLY1VtVFI3KzVTMkxxdlU2VHo0VW1LbEpB?=
 =?utf-8?Q?bu68Kq05zmHGd2uBKDkowWwCgQqlLLKIizgLpu7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e96cb151-17a3-4cd7-69e4-08d96d4dee7f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:39:33.1932
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M+DLaaVeiAmqLYWFog5pgxerKDbudS7thTvwuaPC+/CYtxuuwAQwhF2M4MZUlein1YesZpdQtJltB6ESARnjyw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

All,

back in 2016 Andrew added code to x86'es variant to avoid interleaving
of output. The same issue ought to exist on Arm. The lock acquired,
or more importantly the turning off of IRQs while doing so, is now
getting in the way of having PVH Dom0's state dumped the 2nd time. For
register state I did find a sufficiently simple (yet not pretty)
workaround. For the stack, where I can't reasonably avoid using p2m
functions, this is going to be more difficult.

Since I expect Arm to want to also have interleave protection at some
point, and since Arm also acquires the p2m lock while accessing Dom0's
stacks, I wonder whether anyone has any clever idea on how to avoid
the (valid) triggering of check_lock()'s assertion without intrusive
changes. (As to intrusive changes - acquiring the p2m lock up front in
recursive mode, plus silencing check_lock() for nested acquires of a
lock that's already being held by a CPU was my initial idea.)

Thanks, Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:45:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:45:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176422.321038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQYg-0001W3-Iu; Wed, 01 Sep 2021 13:45:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176422.321038; Wed, 01 Sep 2021 13:45:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQYg-0001Vw-DT; Wed, 01 Sep 2021 13:45:42 +0000
Received: by outflank-mailman (input) for mailman id 176422;
 Wed, 01 Sep 2021 13:45:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLQYf-0001Vq-B5
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 13:45:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e4bd0fda-0b2a-11ec-ade8-12813bfff9fa;
 Wed, 01 Sep 2021 13:45:40 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-7b8lZMMcPm6vtlABb0kWiw-1; Wed, 01 Sep 2021 15:45:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 13:45:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 13:45:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR05CA0124.eurprd05.prod.outlook.com (2603:10a6:207:2::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 13:45:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4bd0fda-0b2a-11ec-ade8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630503939;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h++qf6KL033IrhpD4G9A6T6GzqjOOh03iX8crHOUczI=;
	b=gkCU88ShhK9ERjWmul6HOvfOC+Gqj0MyoxfQqpDYrS13bi1SzLhEXzMv9oRTXMWP85qhuA
	Qlf9GEgda3jyo+kU5fCnH9jH/tE271AB4yyK07+p4TTpVMH+trKSlQBJ16lh0dKD0oOMe8
	C72qUL1TzjLQez44b8Qld6r4Un4MO9w=
X-MC-Unique: 7b8lZMMcPm6vtlABb0kWiw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GuXJfWY0amJ91RgzcHI470nvMF64yGQ3cnRte9BYhY/5ix/pOgT3/l0WrDbceGnUmo91wGrHuAz2ESz8JkC1Sh8jrSRmqDh4KYDpZ1eIRMfLWOOf99ySvgQw1dYfvX0gF8qmJNs6Z/0Uj0tuoymOjQA8K4uGWbRY1riL+t0/mz7SppE4JRxe4DlaYP84TdcFq5VYXG2jZKPfAmLS+8S6i8mcfKyU9/7bOwNidn702QFEghLyJ7FJ5vA0FCh7wpiO7PRwEuYnAuXVUVnWU2xpHs+x4YAEs0yicOzxhiWXy/0pTtJockM7kiwuBITPfn5DDIDHOg7WgfY9PwSi+2DQbg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=h++qf6KL033IrhpD4G9A6T6GzqjOOh03iX8crHOUczI=;
 b=Yg5C7l/JK4Ol2KDRKt/3rW7rwD0yAREQK9pb7+YNteb49CiX3W7dgtCkSoZXxPXl/CG43pmRZa2bDhM/9rtG2H5NZLNWCCe7bzauDkavAf/I7jlu0fM8RvwNJjntpjwiUq6I18jryTq3tR8UYPtqlGv6YhyyK1w6OWqHb2NhGd76NxxcnmCdn6MuWZdB3Ja+1PXVkyqMUSKPZH44rD/6yRYJ99P3UyB6HcNO7c8irErHf/ZFNCpIKToebI6L/MQfAsGQ2tU+3d/lFqF8XD1ymo5vwwqgHpxImgo/vg5+8WFdDDjTE5Xh5j4JlJTbkViITH3Usmu60yc6X93gCsD8rQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xen.org; dkim=none (message not signed)
 header.d=none;lists.xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: Xen Security Advisory 378 v3
 (CVE-2021-28694,CVE-2021-28695,CVE-2021-28696) - IOMMU page mapping issues on
 x86
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xen.org
References: <E1mLMZy-0006TJ-Rw@xenbits.xenproject.org>
 <CAKf6xpsAz7oPX+PdZokm+BAqreqL8rnrAd9UYvdz=F0M-bcMUQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <367a756d-9a14-6970-3820-d863a998583e@suse.com>
Date: Wed, 1 Sep 2021 15:45:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <CAKf6xpsAz7oPX+PdZokm+BAqreqL8rnrAd9UYvdz=F0M-bcMUQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR05CA0124.eurprd05.prod.outlook.com
 (2603:10a6:207:2::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 60b81f1d-a3fd-4b7d-ba5c-08d96d4ec6d7
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23348D16AE16713AB275AF82B3CD9@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Lm9E47T5e3pNBd3xexBskqnLAhkNqfNUfcAVkxdv1rgj+8PIvqVOEWIPYuU7Fl+b9/k99CnB5SbWShUlH8THGGhJG76BHpPuSrk9qP4EwlQ2R9zUVDP8W0gA6g6S/jis+ex+HrTqmQNvY4b3U2KPrET/jKVXRk8TYJal/SiVoAkWBv+x/PHgNvcYU23rrUZlczfcFhwN1suGbAYsFBxbHi9L601qgx2irA7qTN2RMvfCupv0SKyko8bxMRq3eM7+GIxGqnWNHtxjG9bEEGYLkMyhMT0tOlu51cFh5XJLg170o0zhcGormi/tSFCPhe1lNS8AT0x3BY+LK/Uovi9a78O1lf5kq2gthBCWHw5chw0gXvyNjN5e2oAwKQWcQF3iC1ERfcbuQttqnuSThAVO1OG/vzL2gBeTNhiMvseOCVJEai0cz8OwvTlDk+50lg9OeyfLsJTobZ0YKiFdRjiwzXX6H8F0UM6pryJmQgsPPM61Ak2s+4Aj4xKaNmTRmHdBxp/FLb/31Ws3LPDLvkvB3DbTnopy9KqMh2eBIBcOzrekaKz+54i6aDudZvWQdcmm7mjrZmzYnSl7sJofZ+u3ZT2w0d4w881QIPzCPfQrWQzA3kVYBZObwdXe57aR/H6bl7QEmwmIVxjAohBODGndHfu2tcaMdgMlC6R3dEC7mpYVDvrGYfYTtkfPG4XpaqJIx/wed2riSh2tadnjTaBxVDWo5EHVQrUHX66VKdnz+QJ9B2T01ssQNq/j2j7uJfWJ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(376002)(346002)(39860400002)(66946007)(15650500001)(36756003)(83380400001)(2616005)(956004)(6916009)(4326008)(16576012)(316002)(2906002)(31686004)(66556008)(66476007)(86362001)(31696002)(53546011)(5660300002)(8936002)(26005)(6486002)(478600001)(8676002)(38100700002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RHg1Q3BVeTJ2VTlaa3lING9DSW5oblhnaXh4QTVQbHAxVlpPWDhuRWNWbVc5?=
 =?utf-8?B?aU12SGZ4bzdpQ2dXM3JRYnhSNW95am1aVjlzWjRCNmduaUJ4eUR6MzVNb3Bu?=
 =?utf-8?B?QkZrY1BkTkxmVS82aHd0cERyMlJrSEcrMlhiYTBVamZHUXZzalJramhFVHdP?=
 =?utf-8?B?WTJCZFZoOFZncCtCWkFUL3FuNXgwWFR5bGNVUVgrS3ZvL2RSSHRMYVVyVE1o?=
 =?utf-8?B?cW5SWEE1VjdiOTNWYU95cWxndDRnVTI3OG5aYldsbzBLb3dNN2xBSWxRc3Zo?=
 =?utf-8?B?YnJVbGpJWDJUaS9sVENyL1IzaEpxK011SmVNQThnZGNEWU0xalhiUEplcFBO?=
 =?utf-8?B?QlYxTXk4c0dMdWJhZTZJYVVrODNvbUpvcGUrR29nWWhYZVU1WEhrUU85Sk9I?=
 =?utf-8?B?emVwbCtrWFJleUFuVVgrUCt2eVZGYmk4QWQxTmk1aXFtTElpTG5YUm44R29a?=
 =?utf-8?B?UWdrRFdLL2JnRUw5SmhBdzhKR2szY3FsMlQ0b2wvcHdzalNHR1N1Njdad0x1?=
 =?utf-8?B?UVZiYjBueHZTRmEyektWK0FhUXpYL3RIV09ldGdzRzY1cEN2eTVVVEgzREVQ?=
 =?utf-8?B?aHdnZ3IxbUZDb2RrS051ZDdTRTY0RmR1cFZIemMrNXZneFQ2aWRyaGVOL0dp?=
 =?utf-8?B?cHd5NFo4M09OdVlCWjBKOG54MVgzOXdMcU5RWVR3TnF4RTF4K2p0Wmsvb2Jx?=
 =?utf-8?B?dE8xbEpXRmgrbUljc2Q2SlNRVW5LSkFiZHNyRkpwQXV0ZTd5cm9QT2hKelIr?=
 =?utf-8?B?SUVVY3c4cXFqUE5QclA0YTluVnNtUVozT25sVXQ4bzZzcGVXdFNNQ1pGcjBR?=
 =?utf-8?B?RksrQTBiN0x0ME14Z3N5dTRlTXIyOCtYSzdLUWNuQ1B0M1FhUzBXQ2c1Qm8y?=
 =?utf-8?B?dlNUa2ttMGxMVHR5bWVjZkU1V2NHU1puYVFpaStQUVBFQ0huZ24wWVpmVUNk?=
 =?utf-8?B?a1kwaVltaVUxU1crV1ZrUkxNN2t6SWFTUTcvKzhxcXRCRm1COUNiY2pyRVNj?=
 =?utf-8?B?RW04ZGZyQnFvU2lXdGMyV3lJZFRCUUZyQ2xvMysxVXVIdDNYTlhCRGY4S1Jm?=
 =?utf-8?B?SFpNR0IvMHV0Uzl3WnVkWnRWWVFKSE5ZOXdvY2dOL2NyZFhZUzZQd1p6SWQv?=
 =?utf-8?B?THJERDRpUEl0N3VZWUJJVmdFVFlJVnhSMkVCc0FuczJHUDMyT2oyeW10bnp1?=
 =?utf-8?B?US82N0U4Mkx5UjkrbVlGemY1WnNEajl1OFdSbi9kdU1CdnZDOVU3MzBCSEFF?=
 =?utf-8?B?KzBrQXJNWFRHa2c1L1FKSU5YY1lqVzVETksvTlVtcE9pditHNGl4bEdKTzdH?=
 =?utf-8?B?cGRnT3JRUjVRTXR1b2F3SFhqcis5WHZubGlxdWZSSnRKMldCRnFGaXZhZnBO?=
 =?utf-8?B?K3dwR2J6UWJuWmU1SU5YTk95dkU3SmVQMGxRRkR2N3FwdDI3ZFlFcGdMa2oz?=
 =?utf-8?B?QlhnV3BmSGlOWHRXUzVPSGVBYlRFNjQyT01iYkNDRnU4OUc2UUxocm95NnlF?=
 =?utf-8?B?TkU5WFJMWlRlTW94cVR6enFPVjhyeUQxYXV4S3U5S1FZKzI4QXYvWENSSjhj?=
 =?utf-8?B?YUx5TGVDdWtQUjVGNlpPZzlRUElidmVNK1M1Njd5cytnOUNrQjRabVFJMTZW?=
 =?utf-8?B?NHdUVUMyeUZOSkYxSXltU0RsanJsZWNCbGtIazdZeVI0UEJGcHFSckVueU1p?=
 =?utf-8?B?TDJzUU9GdmlVT08rVmJqWitaSVhUblo4VEcvVjdDT1N5MGhkdjZIS0ppTERL?=
 =?utf-8?Q?aMJeP/a0hD4XvkiwIDftT0bIdPnpli1Mv+p09Xl?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 60b81f1d-a3fd-4b7d-ba5c-08d96d4ec6d7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:45:36.1518
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: E3V2wUtrqZrPA87VtkBWBub158OXgC4bKAn+P+aDMPOpp6VtRv+bnWOiX3Tnyrc6nnwRR3n2F84jUmTssJ+mAQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

(removing all lists inappropriate for a question like this one)

On 01.09.2021 15:22, Jason Andryuk wrote:
> On Wed, Sep 1, 2021 at 5:34 AM Xen.org security team <security@xen.org> wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>>  Xen Security Advisory CVE-2021-28694,CVE-2021-28695,CVE-2021-28696 / XSA-378
>>                                    version 3
>>
>>                    IOMMU page mapping issues on x86
>>
>> UPDATES IN VERSION 3
>> ====================
>>
>> Warn about dom0=pvh breakage in Resolution section.
>>
>> ISSUE DESCRIPTION
>> =================
>>
>> Both AMD and Intel allow ACPI tables to specify regions of memory
>> which should be left untranslated, which typically means these
>> addresses should pass the translation phase unaltered.  While these
>> are typically device specific ACPI properties, they can also be
>> specified to apply to a range of devices, or even all devices.
>>
>> On all systems with such regions Xen failed to prevent guests from
>> undoing/replacing such mappings (CVE-2021-28694).
> 
> Is there a way to identify if a system's ACPI tables have untranslated
> regions?  Does it show up in xen or linux dmesg or can it be
> identified in sysfs?

For VT-d, "iommu=verbose" will cause ACPI table contents to get logged.
For AMD you need to go one step further and set "iommu=debug". Obviously
you'll want to be careful with enabling anything like this on production
systems.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:48:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:48:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176429.321049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQbo-0002E3-35; Wed, 01 Sep 2021 13:48:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176429.321049; Wed, 01 Sep 2021 13:48:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQbn-0002Dw-Vw; Wed, 01 Sep 2021 13:48:55 +0000
Received: by outflank-mailman (input) for mailman id 176429;
 Wed, 01 Sep 2021 13:48:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2BIO=NX=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mLQbm-0002Dq-Kg
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 13:48:54 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 580d6a66-0b2b-11ec-ade8-12813bfff9fa;
 Wed, 01 Sep 2021 13:48:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 580d6a66-0b2b-11ec-ade8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630504133;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=Qh3XfuC09YdLPSqmBa5G4bMkC4fwUMe2iEY/h46nkQ8=;
  b=CW6sGQCzJwCCS7ZbuDN4Oq+OHUTRId554N65EV7BwcoG1hQ1tgvDywrP
   +sLwsQj9/dOpZh8mlR2xT1Yuw525jRW/3mvrtG0sVDdXK78WxsDvR9sJ/
   9Yy1yiqv8PvKJzn9dm2cwNvYdshgt/OK9TM4df6updgEbupt6pRLIsTUZ
   o=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: jPi5RsrDq8vGxVA0R5jGxt5cenwvNBfmBZGDQMKiE1AFYKURsTUmSTbwQra4o5dyL59hBeY8ZK
 4idzOWHFaBF7g8iREyL1Yjhi9Pq/4Et2hc3Ae04KWUWZc+jd14hUvsxjO1y9Ai0mtOvwf7MtyG
 +d/olGD96J2WM3+a3UCASXQz4xcTZ6Wb9XCs5f2emQn5Vozut7L8Cu1k643/A8easgzRZnWRNb
 ElncjzE7/9IpFudkEtNW9GkpgvOffpKe6SHHq+gtpTOkQPmiLt7tJUwmRnK6XSyJ+ht6Q3pJB0
 mMQOBniVUtDseSIDuJ68S+49
X-SBRS: 5.1
X-MesageID: 51396146
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:+WB0G6P5PJ+DH8BcT1L155DYdb4zR+YMi2TDiHoedfUFSKOlfp
 6V8MjztSWVtN4QMEtQ/+xoHJPwPE80kqQFnbX5XI3SJjUO3VHIEGgM1/qG/9SNIVybygcZ79
 YeT0EcMqyBMbEZt7eD3ODQKb9Jq7PrgcPY55as854ud3AQV0gJ1XYJNu/xKDwOeOApP+tfKH
 LKjfA32QZINE5nIviTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1Sv1
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfp2oCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8DDeiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6NpOTgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeTP003MwmNG9yUkqp+lWGmLeXLzMO91a9Mwk/U/WuonprdCsT9Tpf+CQd9k1wvK7VBaM0vt
 gtn8xT5cZzp/QtHNdA7dE6MIKK41z2MGDx2V2pUCDa/YE8SjjwQs3MkfgIDN/DQu1/8HJ1ou
 WYbG9l
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="51396146"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Hni2p7i6o8Fc8nMMgBLTmo2PuGG3ZzI2f1YKH8ioqFl5Y0hawo8p/+r3zXsx25d7brFFws0ha+mm6IjE7Z3vaADbFh2tg+sl6ZEIb4Z5ggWsJSvdUZjj6tYR/FCczpC233kYkuXL8coYz3rKgqc+fuZPu4VYF8ycxodSUaDeaZRo3P0oRZmx/jgWilBJssKI4uwmKVWzpmDtbUdK79ra1BJIdzi6mtr34rlQNGUG+CmlDyUCmELKndXSFxqQdmw26aofilFlHCweTiHaB+wQZNBgc9Aa6u1e1JXtfpB9ljK5s8w1wS4FtWwAdeNA5j9VFhAZffFrjCaYi9/3Iw6R7w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0uko84/08g7wVwgcKSsgJA9ORW/fkmuAIxaRx9UBpIw=;
 b=Uj8PqFGqRk7J8OifrZ9KfBd0j+3oqnQS7sgKSgUA4CMt3t2NQrlaQm5Eybjnl/ht1J42QWZkvMrOGlYLX7cH9F7Y61gwl4jVdSlUNHalGrZZXJ2yrPWul5Yt9RxVz+tZtoQKNIWo7lzroLgKU1iIU43mUdRcfGdpneF7CLVCnhHLULDzRgWWjxdjyBuJJeivpKxagzI7sxXMTEtdK4WayYg+grgbTgl+pOofu4mKHkvrMrnUpYtYPctpv+E8rRWk0t4GASMQ0NubXf8SskkNy8swP0VHbg7YZn9VSbfnBU895VUBlwKpUNF6WQ2vxiK+WwJe8llMNyTgXSwMzb69yw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0uko84/08g7wVwgcKSsgJA9ORW/fkmuAIxaRx9UBpIw=;
 b=e6xzsuQNFBn1dawlLa4DZOmsiO70TtY6lujay/4cOzQoR+dffOafwYTNYRnD7oHK3T6suTR/WeSj3lWYkY0VDvj+nauJROAKUATldr9aKdxiFcViJYKGJh4U7GRYUzEKHJRr6uHYP74Sr1jOIWiRIj+NW2TsWnONJb6NR7CQAzU=
Date: Wed, 1 Sep 2021 15:48:24 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
Message-ID: <YS+EqCZUAsT/WTBX@Air-de-Roger>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <YS8+72EY4FrvSSIx@Air-de-Roger>
 <ca3a9559-b95a-3b32-781d-2a2abfa50dae@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ca3a9559-b95a-3b32-781d-2a2abfa50dae@suse.com>
X-ClientProxiedBy: PR1PR01CA0035.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::48) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bc2a53e1-0d91-4a32-74f5-08d96d4f2e4d
X-MS-TrafficTypeDiagnostic: DM6PR03MB4763:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4763BDD5F958E101FE584B268FCD9@DM6PR03MB4763.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: n+IbpYeOeTMzhjFzLAJz9/6ouWC2LTS4xhC/baz3NqHVWeIvvcX2Q9rnSZE5J5h6+GKUzSC5c2mWz2LMewH+2kAXB+g1Wm6W08An21ZdTCGMFgf2cdhFmj6+Dv0b3M4hZVFdWpjFrWr2Vp5CqXUbigpXVrbNoEj1SxQJDFD/SBnqQmkIy+xNmWCX2uJEpf4sKR6gQqTsHpibXAgxpdrQCY+/KjCd68XHlkU9GZC82sFkdhh/Xogu6xyNVhRaAitsyrdobEDmTLkM5Pwf4Il9Qsbyb+JN14aPg8Xo9qRsP9+mhUsmSrUMVE2y2RoXrWN8HbOYK4n8ogpfcLaJoWTA2czG8nKMCk7PvbFCOKOO0C1bAMciy+uORDYDh+7r8pe8gjvFcPi1sYObgZ0AzZ9pZ8aLSa/0hYioK98undPO0MeW9BWIm/1tSy+y7oUB09RNW3b4OgLyBVzCM0OeIzfAz4PnBoiq38RcDN2hA54p1Pp9vU4Eb520yRQyOD6zH+59k89ubMF96TdZG7yACLwPJrWjOHzsvza1qsE2RyIhmiAdDhuA3EWrnAyhGJxo1pI5M+ydq3EHGN9dMYk3oJUagfe9KqvX5Lh6WLd1zFtNSdTxlyGIW3biuLkaAwWqMheBA9y1OTYqfTwJk6vIPzDkaQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(7916004)(4636009)(396003)(136003)(366004)(346002)(376002)(39860400002)(38100700002)(86362001)(4326008)(53546011)(9686003)(66556008)(33716001)(956004)(5660300002)(66946007)(83380400001)(6496006)(6666004)(66476007)(8676002)(6486002)(26005)(186003)(2906002)(478600001)(316002)(8936002)(54906003)(85182001)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Mi83anFXVytGeXNndU5RSFFzRXpSa3FGMDJxL0ZiQWUvQThQekhacW4yV21W?=
 =?utf-8?B?d2dtVzlkTmtlNmVQVEptZzVtNjZ2bTJzNHU5TVkrSE1BTEJ5cXRkOGZxTnpa?=
 =?utf-8?B?bU1tOXRNb25wMHNrbTB3Z1VWMUc5aVUydXhoNGNoTnYrRXA4SmpjWkUwVFpE?=
 =?utf-8?B?Y1BEWTNSY0JrdDhxelRRQUMvRnRsVnNvSkV3T01zMjFYMnV1WkQ0c2k5aHRR?=
 =?utf-8?B?ZDJSVkRnTlZLejhSdlA3NDBkRlUzTzMweG1KcEVXL1RoeFNmdkR4emdTUVZx?=
 =?utf-8?B?dE5WVG4yMFZsNllZTHA5SGZHcm9CK0RTL3lvVkl0cHNkNXFyK1ZpeXdneHAz?=
 =?utf-8?B?b2FCOGhBQ0VTL1BscG9xdk5kdklrUjdmaEpyZ3A1b05LUWF3bUZxUG9zRFNn?=
 =?utf-8?B?cXIyUmc4RWtRckVYRys0MlhBeTdjVzhuV1FmYUNMa0hHSVVPU0d6Wm5XbEpF?=
 =?utf-8?B?SHdJanBrSFFBbFN5UFNYd2FBQ2FGWXV3amNlVVhma1ovcWdReHZ3WmoxTTBK?=
 =?utf-8?B?UWJsc21UTk5RaUhFa3R5SGdTMXA5RmpPcjBwVmJEQ2F4M1czaVV3MGR2RlBn?=
 =?utf-8?B?QmYvYWJ3UHQ1Yk5Gd2IvUi9EVitWUlhTalBhdjZ5clFoWGlKQmMwUlpCU3dI?=
 =?utf-8?B?Z3hYYnR5TVMzZmVHQUNEMXYvTjhVdGVoWE9NNVhwUlRoRUROTWNZRzNkcU5C?=
 =?utf-8?B?TG5Kc0g1OXkvOXRNay95UittclN0TUIrZTY0a2grelRwMVdrSExjQjdncnBH?=
 =?utf-8?B?SEhoZzdjbWwzTHhjSkYzcVU2bFh6WnJ5ZU1WQ2c4Y212L3d1Q25NVXE1RmpD?=
 =?utf-8?B?dlBudUVsRHpzMUZVWGVuTUVOREJzZWlBeURteDluMmJsSDl2WHBFaVJMcmVT?=
 =?utf-8?B?R2RneHlyNW5hdHpWMmREZlBSRDB2dEJTa2JwZk1JeUZBZ3cvVHFxdEZ1ZC9Q?=
 =?utf-8?B?b01uZDI2UzZWcmRkYytvSDNwLzhBd2o0S21xL2FBZExjNHlqWTRzV1YzTHhV?=
 =?utf-8?B?c3IwS2VvSnpTNjJXY2o1RUxpdTViM1l3R0ZBdGFTdmx3dlJGcWVVNlpDWFo5?=
 =?utf-8?B?aFlSeWFkT1Q2VTdldW55MUdhYUVPUjFmRVBxZE5sMXpNcWI3YWJBOU5lZkFP?=
 =?utf-8?B?T20wcjd5WWVEc1hIV01FRjN3bUJTeXF1NVdKc1FsL0dOQ05YejBNZWVBZWQv?=
 =?utf-8?B?eVE0ZG5jWnZKTjA3VTUyQlhIeEFnbGlWUlZ6c09PK0w3c0xDVTlsTnhlTVZh?=
 =?utf-8?B?dUlSeU4weXZmQ0dCOWRCQ1NrY2xQUzc3OWZXUWM2TUsvOVZUQ3ZubDNGUWNy?=
 =?utf-8?B?ckowU216YWc5N01vOGN6Lzh0ZkpyUzczMWpWQTNSUVFyNzQwMS8zaDhNNnBU?=
 =?utf-8?B?b2RKWHVyS1Q5dFRpd2hHdTQrL2h1dUUvNHZEYVBsT0N5c1Q5bzdWZ0N4Titw?=
 =?utf-8?B?WDVreGE2TjBRU201OWlHMUlpRjkxN3hhY2ljdE9BSFZoOXBlMjdWenpSSHFa?=
 =?utf-8?B?UGorK1VCeEFxYjZNUzB1QXo2VGhlci9qRjh4WjYwakxhY3lZQTg1c1RQYVF1?=
 =?utf-8?B?elZ5TExHVnh1SHV2bG15VG1sMWVzNFhMZ25uOENaU1EzZHpGamxIbFVIWGJz?=
 =?utf-8?B?YjNrYkpHc216b1Z4dUZlSG9ERnl6QmRwVlpWTlR5cTZ6T3NpWjN6NHNOTTFh?=
 =?utf-8?B?elN1d3BKTm1JSVpzYk9aZGNsbmlGdWorcFc4WDJHbm1VU1o2emV6MlBRM3hq?=
 =?utf-8?Q?CUtUxC3BQK2i+ycnuqxYZl9ibNylvKXqZ1ZmuUA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: bc2a53e1-0d91-4a32-74f5-08d96d4f2e4d
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:48:29.7442
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Tb2SOovK+89YBZvcjbFbxzluKwpHXZ/FYuQG6mY86cqbfyk5J3cwO+xYib4uw1KPjSwiGTg9BlrIY+jdob5K6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4763
X-OriginatorOrg: citrix.com

On Wed, Sep 01, 2021 at 11:53:03AM +0200, Jan Beulich wrote:
> On 01.09.2021 10:50, Roger Pau Monné wrote:
> > On Tue, Aug 31, 2021 at 05:38:49PM +0200, Jan Beulich wrote:
> >> On 31.08.2021 17:25, Andrew Cooper wrote:
> >>> On 31/08/2021 14:26, Jan Beulich wrote:
> >>>> On 31.08.2021 15:16, Andrew Cooper wrote:
> >>>>> On 30/08/2021 14:02, Jan Beulich wrote:
> >>>>>> Further permit "access" to differ in the "executable" attribute. While
> >>>>>> ideally only ROM regions would get mapped with X set, getting there is
> >>>>>> quite a bit of work. Therefore, as a temporary measure, permit X to
> >>>>>> vary. For Dom0 the more permissive of the types will be used, while for
> >>>>>> DomU it'll be the more restrictive one.
> >>>>> Split behaviour between dom0 and domU based on types alone cannot
> >>>>> possibly be correct.
> >>>> True, but what do you do.
> >>>>
> >>>>> DomU's need to execute ROMs too, and this looks like will malfunction if
> >>>>> a ROM ends up in the region that HVMLoader relocated RAM from.
> >>>>>
> >>>>> As this is a temporary bodge emergency bugfix, don't try to be clever -
> >>>>> just take the latest access.
> >>>> And how do we know that that's what is going to work?
> >>>
> >>> Because it's the pre-existing behaviour.
> >>
> >> Valid point. But for the DomU case there simply has not been any
> >> pre-existing behavior. Hence my desire to be restrictive initially
> >> there.
> >>
> >>>>  We should
> >>>> strictly accumulate for Dom0. And what we do for DomU is moot for
> >>>> the moment, until PCI passthrough becomes a thing for PVH. Hence
> >>>> I've opted to be restrictive there - I'd rather see things break
> >>>> (and getting adjusted) when this future work actually gets carried
> >>>> out, than leave things permissive for no-one to notice that it's
> >>>> too permissive, leading to an XSA.
> >>>
> >>> Restricting execute permissions is something unique to virt.  It doesn't
> >>> exist in a non-virtualised system, as I and D side reads are
> >>> indistinguishable outside of the core.
> >>>
> >>> Furthermore, it is inexpressible on some systems/configurations.
> >>>
> >>> Introspection is the only technology which should be restricting execute
> >>> permissions in the p2m, and only when it takes responsibility for
> >>> dealing with the fallout.
> >>
> >> IOW are you saying that the calls to set_identity_p2m_entry()
> >> (pre-dating XSA-378) were wrong to use p2m_access_rw? Because that's
> >> what's getting the way here.
> > 
> > I did wonder this before, because I saw some messages on a couple of
> > systems about mappings override, and I'm not sure why we need to use
> > p2m_access_rw. My first thought was to suggest to switch to use the
> > default access type for the domain, like set_mmio_p2m_entry does.
> > 
> > I have to admit I'm not sure I see the point of preventing execution,
> > but it's possible I'm missing something.
> 
> Well, what good can come from allowing execution from, say, the
> IO-APIC or LAPIC pages? Or other MMIO-mapped register space?

map_mmio_regions does already map BARs with execute permissions, so
it's just some MMIO regions that get mapped without execution
permissions, which makes all this confusing.

> Insn
> fetches might even trip bad hardware behavior in such a case by
> being the wrong granularity.

Normal reads could also trigger such bad hardware behavior, so I'm not
sure preventing execution provides Xen any more safety.

> It's imo really only ROM space which
> ought to have execution permitted.
> 
> The issue isn't just with execution, though, and as a result I may
> need to change the logic here to also include at least W. As of
> one of the XSA-378 changes we may now pass just p2m_access_r to
> iommu_identity_mapping(), if the ACPI tables on an AMD system were
> saying so. (We may also pass p2m_access_w, but I sincerely hope no
> firmware would specify write but no read access.)
> 
> Similarly in "IOMMU/x86: restrict IO-APIC mappings for PV Dom0" I
> now pass p2m_access_r to set_identity_p2m_entry().

Not really I think, as PVH dom0 is the only user of the
set_identity_p2m_entry call in arch_iommu_hwdom_init, and we should
never identity map the IO-APIC range in that case because a set of
emulated IO-APIC replacements are provided and those require ranges to
be unmapped so that accesses can be trapped.

> 
> I suppose an underlying issue is the mixed purpose of using
> p2m_access_*, which possibly has been against the intentions in the
> first place. We cannot, for example, express r/o access to an MMIO
> page without using p2m_access_r (or p2m_access_rx), as there's no
> suitable p2m type to express this via type alone. We may need to
> split p2m_mmio_direct into multiple types (up to 7), I guess, if
> we wanted to remove this (ab)use of p2m_access_*.

My main complaint is mostly with the fact that some MMIO ranges are
mapped without execute permissions when mapped by
set_identity_p2m_entry vs map_mmio_regions that will map them with the
default permissions and that has execution set.

If the mappings from arch_iommu_hwdom_init for example would use the
default permissions that could solve quite a lot of the issues there
AFAIC.

Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:52:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:52:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176436.321060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQei-0003my-HF; Wed, 01 Sep 2021 13:51:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176436.321060; Wed, 01 Sep 2021 13:51:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQei-0003mr-E8; Wed, 01 Sep 2021 13:51:56 +0000
Received: by outflank-mailman (input) for mailman id 176436;
 Wed, 01 Sep 2021 13:51:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQeh-0003mh-Al; Wed, 01 Sep 2021 13:51:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQeh-0003DP-4O; Wed, 01 Sep 2021 13:51:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQeg-0006LF-Rl; Wed, 01 Sep 2021 13:51:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLQeg-0000xx-R5; Wed, 01 Sep 2021 13:51:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zxDmiBiuazEHqG8wBcaJfsNq8jtHVu9JVDVRK4LgWnY=; b=6/TIkGyJ2cBv4a8H1b+aBKR7Ai
	YX/kivcLlQZ9kScwBIz6KNd8df8XIn9YA2nwVlpMFCnJX2wUlGxeSLtJfXbTnEr8l2H2cIBDUKCyI
	fVkvjm5G+g11E2QLObRBI7MgCbFQGlFGhm9c47ekxjZfOYDeCIT9y+cM/HrwzYahqu4E=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164689-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164689: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-amd64-pvops:kernel-build:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:build-i386-pvops:kernel-build:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:guest-start:fail:allowable
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-freebsd10-i386:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-freebsd10-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-i386-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=9e9fb7655ed585da8f468e29221f0ba194a5f613
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 13:51:54 +0000

flight 164689 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164689/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-pvops             6 kernel-build             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 build-i386-pvops              6 kernel-build             fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     14 guest-start              fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-raw        1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-shadow     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-xsm        1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)               blocked  n/a
 test-amd64-i386-examine       1 build-check(1)               blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-amd64-i386-pair          1 build-check(1)               blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-xl            1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-pvshim     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)              blocked n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                9e9fb7655ed585da8f468e29221f0ba194a5f613
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  396 days
Failing since        152366  2020-08-01 20:49:34 Z  395 days  684 attempts
Testing same since   164689  2021-09-01 04:20:58 Z    0 days    1 attempts

------------------------------------------------------------
6967 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            fail    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             fail    
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           blocked 
 test-amd64-coresched-i386-xl                                 blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       blocked 
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-i386-qemut-rhel6hvm-amd                           blocked 
 test-amd64-i386-qemuu-rhel6hvm-amd                           blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemut-debianhvm-amd64                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     blocked 
 test-amd64-i386-freebsd10-amd64                              blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-i386-xl-qemut-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-i386-xl-qemuu-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemut-ws16-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ws16-amd64                          blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         blocked 
 test-amd64-amd64-examine                                     blocked 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      blocked 
 test-amd64-i386-freebsd10-i386                               blocked 
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-i386-qemut-rhel6hvm-intel                         blocked 
 test-amd64-i386-qemuu-rhel6hvm-intel                         blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-i386-pair                                         blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-i386-xl-pvshim                                    blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       blocked 
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-amd64-i386-xl-shadow                                    blocked 
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2103251 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 13:57:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 13:57:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176444.321074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQjT-0004WK-8g; Wed, 01 Sep 2021 13:56:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176444.321074; Wed, 01 Sep 2021 13:56:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQjT-0004WD-5B; Wed, 01 Sep 2021 13:56:51 +0000
Received: by outflank-mailman (input) for mailman id 176444;
 Wed, 01 Sep 2021 13:56:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2BIO=NX=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mLQjS-0004W7-2y
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 13:56:50 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7348bdf3-0b2c-11ec-adeb-12813bfff9fa;
 Wed, 01 Sep 2021 13:56:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7348bdf3-0b2c-11ec-adeb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630504608;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=eYJQJArDZC7fN6Yxqf5+EVylu10Lw5x4htbEdY96X90=;
  b=VV24t14fAhFeVNcZ9wvrsX9H+GAYF8411kYEBD+MlbWguRoDFGq3ZpUL
   7U2D+pRK2ZC7HICfDS6M4d8EDy3V0mGFYZQP6ykexT2egtY5MKvuXJoCr
   iGfj8gNzV04RgkWYVxG0bmZ4HsIhouqKGMIP28H7bVAM2aH4eBgodTY+D
   c=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 6dvqWVnEwM6SEJ8MmxvtZ2WID4OZ2nsjPKjJUSq2pDMgyl0soQm0RP7D6OU0KX+XzKpSRo6njK
 B0+UH/rUMjfJX796/X1VbmCjgmntnd0V5+lnmeKRUPNT+Pvj1Wq8AJt11Do5M+BUYbUCXpIUUA
 /2R5+6vI8V5hrj2tqTy04G4Zg04njaBD/uuiqfgKSOduATk3X4rL2znPh3QsGw0LPQ2cNRKlSW
 LBqJOibNCd9doebXBAQuA/Agg8Y5YwTZ+sdl6z1/yTjjNG8cnwd5glk/059Y27XKwdvJmMFGEQ
 GM3vN6vOn5vpBDsUsVjflcpX
X-SBRS: 5.1
X-MesageID: 53535801
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Ggx2N6le8q/+24tuJdDpYqcf+p7pDfLp3DAbv31ZSRFFG/FxXq
 iV7Y0mPHjP+VAssRAb6Le90ca7LU80maQb3WAxB8bBYOCEghrKEGgB1+vfKlTbckWUnINgPM
 9bAtBD4bbLY2SS4/ya3CCIV/MH5uDvytHZuQ6n9QYIcegQUdAE0++yYjzraXGfh2F9dOAE/O
 D33Ls6m9L6E05nEfhSwxE+LpX+TxmiruOZXfbAbyRXmTWzsQ==
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="53535801"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CRyJDE8gATnRhc7ntydSeH8JaS4KoPbAfz7HkINJF/pPALKw4nWmErY3cZdAkRv9mvWlGsfxdCLD6LUoiRGKpqK2OR+ezcW8tlFAZr5SgcU87MnbOlwgpBv5Csqkra9sckot5fnRvetaXhoP+cW+bmgowWwNWnL5vD2jy3Y0ajs4kKoG66gP6ZvmlJJsofigwyosvE7LTfb6syrjJJJklvs4NYvG2CEeX1JMLBzafzwQFaPyB7O8IwzUk914IySQd0ubkRHg5oFkNcwVk/WWUumOC9GTSEScCm4VYOBzAAm7a9FKsX8Tlrkbw3+l6JnefTufMjit9UZxsrX29gqHOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=O/Rwm/zZbFnYdOqWPNkLOtEmXucxfXteYGvFJw99IeA=;
 b=Ucu3do1Mucy4Jv9goTV9dX6eAaZfJ+FtSC3Cy5TlS9xzpPsS/S6jsjCQHB6+TvNH38qRbYpO4V/mzGyru0uO1sseYuSSzBor1983TPb5y27KZ4NfeXrubc05zxwwUvqWngjQn8aTZLSP43i+5umcWhl//DP3bt+UPv95fX4BNxIsspD33JkPMhwwaZC8nqUtJ3Gu4oe1NQpxfBXfksMCQJZ9MHedLLpJ1/09C9szBgZ0piIhDp9pj56s/xFDBy6dGg0rDb/5WlxM5M1eRrpxK6Q3TS/zcYYG7egjWu0XvOkTrLFRs9vwDTNbO+88NhnRAAW2MQxG5CMbznpyZnvjaQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O/Rwm/zZbFnYdOqWPNkLOtEmXucxfXteYGvFJw99IeA=;
 b=CQ3lRVrAZq+cuaEPD3haeU715gSouzAAqlLTH+LuEgiwvmmMBDQpU5c2h9vE6rx/mNBuYUwu8VUUdOzh1i4AkYIfWKv6s4n+7Ui3n757gk1JWNi0lBL1iGTlviLG/1IK7dNtQh3EJddqGS6EUXHmJpeaRdiTmzRI/LbLLJJPJqc=
Date: Wed, 1 Sep 2021 15:56:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
Message-ID: <YS+GmC7IjxoODTu+@Air-de-Roger>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
X-ClientProxiedBy: MR2P264CA0155.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:501:1::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fcac1e0d-6b7d-437b-861d-08d96d5055e6
X-MS-TrafficTypeDiagnostic: DM6PR03MB4844:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB484481ACB6FAE9C25A3013028FCD9@DM6PR03MB4844.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PpZEmP5tvtGotuYUm/EmbLj9mo9rU4abSl544HqEnGKU1zyW5tMM1fu0loSlGgH/TvAgnZHU6iyzs2lmRrEIZN49q7bER/3mvbkBMSkPTzRIhH+BFZU8G7ERM3EaDv7deOVSrrTi7cLBL9SRyPDF3/qTjbi4zQl8Yr6qavh8Wuf1A7a3n0IEY6nY+ITxJ59RAaap5QVZmDD1ntXOmjVN5DiAvIZfj0qvSf9i403CHDxhK8QVaYEN97Iwr/+R7cOxkCXa+VFznUoXH2IQ37Da80OKHgOfnlwFMVc10n8muNbjpRrROrJZ8VdVp+c1OZV2CBvnIJzzwdZ/kffVtbDCRHP8S6T5eB8bzCUjGW9CqTPOfVeYUQDxpaaCZ1Aepscx/BYo2s5jxabCiYnqioZLuTrurYgUWZcIooWBHPfsGJA7Y+/06tJ42chGo+wd2/cjlmCOqaBJ1ai0E5+4m4Gi6L64W7SYp5vvcyw8YCw6H0a7bZknz3TrKMkDd2VX6nCfTik7erVrNT9vi+fp0OzR4ZBrCbGpZfGeUcVTg4JQGGlstaJT0clPme5teDCEVC+cIjhbooGIxTOsNkYx/zjo5JUZRThmU5eNNSMVWPw0EsUWfIHaOZoca69K4IXD/WMbotHgZBKSGOIjr2uH9ayCUA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(7916004)(346002)(136003)(396003)(376002)(366004)(39860400002)(6496006)(956004)(86362001)(5660300002)(33716001)(66476007)(66556008)(53546011)(54906003)(26005)(6666004)(316002)(186003)(66946007)(4326008)(9686003)(478600001)(85182001)(6486002)(83380400001)(8936002)(38100700002)(2906002)(8676002)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bVFJRm5IdGQ1dGxnSWpEVkcwUXZvRGM4QXZYa2RrZG1CNUpkL3p5Tkw1V0Z6?=
 =?utf-8?B?cGRzNGRDWjBDU3NITXh3WWRtSnYxWW5DNUgrcWY2dmZXSFRQTi9ZMk12M04z?=
 =?utf-8?B?ZGcvUjYrSGU0SjIydnpjclcrdlgvUmtnWFNMTE9USFUyTnJiQXhPMW1TSzV0?=
 =?utf-8?B?VGR0SnFHbWhJa0VxUDd5RUxFUzh6eUo1VHYzY3R3U3NXT1loRnp3VThqVGlq?=
 =?utf-8?B?NE5VNzhOUXBWOXhzcXVWRjBJY3MrdWpIWGphVkZqYnJBTnMrODlLK0d1dHZC?=
 =?utf-8?B?UGZGRklwWWorZmQ5K3pRSDVUVTdZSHlMVmhFankrc1dWWFkvODN0VVUrUy9z?=
 =?utf-8?B?dUh2VCtHN0hjclFDYjd6UTBNbEIvS2ZwTElSTFZNZVFzVlAxTWx2SC9pYzU2?=
 =?utf-8?B?U3p3bmZMd1AxeHFsTTZBQWQ5MnNOUXdUamFDYzNDRjhSeHBwYkhHQ3ZGdmsx?=
 =?utf-8?B?eVNrOEFWMUMvaE5EakgvMWRkcmVST2xWWHZsbDlGZlZlM1diM2ltV3c2RXdH?=
 =?utf-8?B?R0RVa0ViQ0JMcm14VWMyL0Q3eDFmNVZHQXN5QVVTb1VDYU43YlY5UnZ0cUgr?=
 =?utf-8?B?TUgxc0lJZi9odXpLVFVLVFh3UHd0MjJwcTA1NlZqN3ExS2k4UEowTEhFRjlG?=
 =?utf-8?B?TnNUZFlOd2tmeHl5VndqRnE2NDBVd0tIdG4yT0ZCdWxyT3Z2WHpkNWlRenNa?=
 =?utf-8?B?K0tyQTlsS25qdE5FWjlybEJKMjJxakZvbVZKb0xvSkNRTkNOdC9DVmVwTkZI?=
 =?utf-8?B?T3RPZ3F0bkFUWkFwNWN4bWpXWUgxakIvaHJnL1YxYTNYYVk5czY2bnFoSlpy?=
 =?utf-8?B?YTJHeStmV3k3QVlXMjl1VWZTcnFNdHUzUVJRdFlOYjNnZ2lRYmloZkxtMjA2?=
 =?utf-8?B?WEc4MHlaSDM5SHpuOVluWEcvbUIxOEtXc2l2dUcrRXliWmNMeEFENDQ1WHdl?=
 =?utf-8?B?TjVQZXhmSWwvdWVVR0k1UEMvU1ZKU3hxS3FlMGE2WUJFQWwvMnFmY3ZTeTY4?=
 =?utf-8?B?ajkxdU9SZHpZTFVkeGpRa2lmTy9raXRiVjFjRG9SaHV6K3pSOG40a0wzckhK?=
 =?utf-8?B?aGQ0dC9xMDF6OGFzb1VVU0o4TXlobldXRUpIVDdVT254V3BXZnJ5ejU1SjZV?=
 =?utf-8?B?djFLazZBKzB3ZVVuak4vZ0t5QUtzbWRIdnhYWlRCUU15MHJEZ1JyOU45NUlv?=
 =?utf-8?B?dURLeHNWV1duSWdqcGFpMy95K0ZXUTVKYVZMQkxRczB6cEptcHh3clR5cVR5?=
 =?utf-8?B?UGdEN1ZGWkJXYTRkMmx2RFlNdThrT1liSU5CaEJqSHdqNXdtNUJ2Yy9XU3pN?=
 =?utf-8?B?bWVnamcvNEZ3TVdrVWxLNDhIUzBlNWd1c1ZiVVY1TVhITEppMnpWNGhTNng2?=
 =?utf-8?B?VzJHL09uRW1xb3B6OUdDQUF2ZlNHNDhXZGhEd0xWMTFvS3J6cEdVbTB4eUhG?=
 =?utf-8?B?cmRTWVBlTTdINVY5T1lGZ3FxNjJDZ2xreHhwUVEzQmNaeTlyZlBZZlUreWtQ?=
 =?utf-8?B?ZXdWeXBnQzA3M0VGOE12STJhODRpeDJELzJoRWlMUDlIcElQUGV6VEFQMkpT?=
 =?utf-8?B?bWF3NkUrSWFGRi9wS25xUVBCcEtxQ3h5VzhnV2hidjZjNGpXaEQrV1dYbUxn?=
 =?utf-8?B?RWtwZ2E3WGpxRTNMODBvZjMyMW0wTHQvelI3R2VJVGg0ZVlJODNvWDlhMkZt?=
 =?utf-8?B?d0ZOSXFFOTd3TkRnOHZBMnJrZnpwbFV2MHhKY0xpOGRCWEZIVHVqR3FORWVH?=
 =?utf-8?Q?4PFcvr6i9JsOsRFMtdpdUwOlDVPkXKHmrnjUWWH?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fcac1e0d-6b7d-437b-861d-08d96d5055e6
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 13:56:45.7509
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oPO7si4FIV8lmBFhi1Y1yeWZajC5qqH55yPZK6SvM9ha9wfc28rsouH3vxE4w/lfSrPe1NWTxdlBVr2ejGRP4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4844
X-OriginatorOrg: citrix.com

On Tue, Aug 31, 2021 at 10:53:59AM +0200, Jan Beulich wrote:
> On 30.08.2021 15:01, Jan Beulich wrote:
> > The code building PVH Dom0 made use of sequences of P2M changes
> > which are disallowed as of XSA-378. First of all population of the
> > first Mb of memory needs to be redone. Then, largely as a
> > workaround, checking introduced by XSA-378 needs to be slightly
> > relaxed.
> > 
> > Note that with these adjustments I get Dom0 to start booting on my
> > development system, but the Dom0 kernel then gets stuck. Since it
> > was the first time for me to try PVH Dom0 in this context (see
> > below for why I was hesitant), I cannot tell yet whether this is
> > due further fallout from the XSA, or some further unrelated
> > problem.

Iff you have some time could you check without the XSA applied? I have
to admit I haven't been testing staging, so it's possible some
breakage as slipped in (however osstest seemed fine with it).

> > Dom0's BSP is in VPF_blocked state while all APs are
> > still in VPF_down. The 'd' debug key, unhelpfully, doesn't produce
> > any output, so it's non-trivial to check whether (like PV likes to
> > do) Dom0 has panic()ed without leaving any (visible) output.

Not sure it would help much, but maybe you can post the Xen+Linux
output?

Do you have iommu debug/verbose enabled to catch iommu faults?

> Correction: I did mean '0' here, producing merely
> 
> (XEN) '0' pressed -> dumping Dom0's registers
> (XEN) *** Dumping Dom0 vcpu#0 state: ***
> (XEN) *** Dumping Dom0 vcpu#1 state: ***
> (XEN) *** Dumping Dom0 vcpu#2 state: ***
> (XEN) *** Dumping Dom0 vcpu#3 state: ***
> 
> 'd' output supports the "system is idle" that was also visible from
> 'q' output.

Can you dump the state of the VMCS and see where the IP points to in
Linux?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:06:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:06:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176453.321090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQsO-0006GX-7L; Wed, 01 Sep 2021 14:06:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176453.321090; Wed, 01 Sep 2021 14:06:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLQsO-0006GQ-3j; Wed, 01 Sep 2021 14:06:04 +0000
Received: by outflank-mailman (input) for mailman id 176453;
 Wed, 01 Sep 2021 14:06:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLQsN-0006GK-7g
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 14:06:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d39c2ae-c4d0-4bcc-9315-8e38495a13b0;
 Wed, 01 Sep 2021 14:06:02 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2059.outbound.protection.outlook.com [104.47.12.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-JttB8YdOMda894Ng-kwk_Q-1; Wed, 01 Sep 2021 16:06:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5598.eurprd04.prod.outlook.com (2603:10a6:803:e9::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 14:05:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 14:05:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0014.eurprd09.prod.outlook.com (2603:10a6:101:16::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 14:05:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d39c2ae-c4d0-4bcc-9315-8e38495a13b0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630505161;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=dqWHal1kSmpABLmnO9tkjLbFQHGNQcZ3wuTeZ1Bs8FE=;
	b=Lcj9sV3pBRA0zGtf4W0Ew+Em84AJDGPr9Yycxb4rvQgEOP1wc97tZg5SM6EMSWjUdovBfh
	BagboJcsrCMVLwh6XgC2oDlfMhCWETGXBCSlFONoO9Frpw1Hotei1XHzdGcrm2syRw7TAU
	nzT1mxQB8uw0t/ZwIk6JRSPWksGxkko=
X-MC-Unique: JttB8YdOMda894Ng-kwk_Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DjUHqOWzRUNL9QlCkm1cLKLVBY7B/lLKPtbKv98snqghPwianmnh9oZmPTzAT/4cCWzz/nYacZBUTkl8t9x3onMCII5hLJWwf1Mrvh8BsosvImXZ6wies1l4tDV3DrTj9ZDv3UqzHPe0YJOUWlqwltfhxmwCjRFyM71ao8pHaUrjNAKX2/k1DUIEWnbiOOgjMbaRawkgIHNvBJVPHURmv5EBN66ziU0zM6oO1wEpPBdZ/yd2Dt9U+zM96Qlxill5E8iISlPtwI6ghGBK+MIMurbhHzfjqGQV6ZklQ23SwOO4n34tEaIl8eTfN7Fi7JTYtPipui9f++7qNWPl/ccjJQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=paypqYQBXso7iTGQUkcCymonj0cg77qB8rkP+sbL1jE=;
 b=XJaJWGwmZ0Fra57Se5qp+oxp+ioxrkkfUGwGcLj/SKE2Y1hjDRGEMiiOHPcOsv67Eyxrfr08r3vNvF6ly+eLWruQsnNlkHX/DmnzQNADLKbdMwVrCOB1A0hTwDXpd1qrHsOF/wEBspivdoZleEfl2vsIesTrBM/bV483lNbZ+cRa956YJC6v1VIrHIcKVrb4E3HK469E6tBgdQvTTLuZcr/bSwqTdwjSDn+IElonWjYHf9HVnZDxVuwqAv2WTYLoGAbgdyzN/6ryj70iBhOIz+LCL4Gb/XLHO0ASpWAWI1MQRDFK/jvVJON71ORBYUQHvelFMlSS60YsgXDeFkXH6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <YS8+72EY4FrvSSIx@Air-de-Roger>
 <ca3a9559-b95a-3b32-781d-2a2abfa50dae@suse.com>
 <YS+EqCZUAsT/WTBX@Air-de-Roger>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9682f7a9-7c67-a9c3-0345-11c110193f52@suse.com>
Date: Wed, 1 Sep 2021 16:05:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YS+EqCZUAsT/WTBX@Air-de-Roger>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2PR09CA0014.eurprd09.prod.outlook.com
 (2603:10a6:101:16::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ced71f10-58d2-4911-befa-08d96d519f46
X-MS-TrafficTypeDiagnostic: VI1PR04MB5598:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5598C5AFF1BAAEB8795061C0B3CD9@VI1PR04MB5598.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EZLZdJpqAVRbmzg9G42NNm1xuYil3/cWALCsrqLJQUGHrRoL6lXSdGtf+bUFDqULyGCGi1dypTHUItVMWIvLOjgWWr3zOitJhpaexgf/QsddFXK8RztFBrwbHvhu4xuZth4aR+PTKEIkaqedxYKACmUVUI8svOsusbfYCAkA6FofKcszQH2Mlceh6lnINn7TadtpEW8+BCbXxEZYtTrmRDxXKvgxJq0M/R5KLHYDOD43fZava8uWQszCQ8g5ZHtNZtv5X2S6n2XJx2Z7Ixj4oQVWtkQCj76gJiXSNgi0P5AvJTJv3JNfffyaiD7oWHEU/X76C3gR3qaHdOQsnabDAb1NiRufW4pLgrsN0i1lfxuFTe2V1EFOMZAkPmiqH0SadT5sWp7sYBCXagriuvNM8lW+fz7+a5oRvW9ALqsf8IzbYqcrBQngKYSD8d03/5poo0aMBLdY8q5KuBXHu//2NLRtiQwUsPaz1R8BEUSKKE45QcaInit+D0M5eCT1OGtVexmcNH1ejHkgzcqEHeQH5EJJgEzHJXAD/lME7ifk7LYNP0CnTc5ulHHD7RdAZGfvoq0G3p1/7oX0ewCUIY4D8RDdtD+kzLFThmlpF0ZIExfqt1kG1g3kcYs91wjhw4OBI6Sokd109T71DWe3WoBhJA4rZqGh8m/vYC55SLS1EyH3q269+gwnPoExwgdV31GSqtez5sIgEf/v+T/ujGrwhzdbHzJNaAM/I3u/qETxa/k=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(346002)(396003)(376002)(39860400002)(53546011)(38100700002)(36756003)(8676002)(6916009)(6486002)(2906002)(31696002)(186003)(5660300002)(8936002)(26005)(478600001)(956004)(83380400001)(16576012)(54906003)(4326008)(66946007)(66476007)(66556008)(31686004)(86362001)(316002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?uCsrbEH13C/c88d3uq4WMugelqd/tgqx8vOTlnitcyd7XT7sis5sXqXkjz6X?=
 =?us-ascii?Q?uz+gsdX8kwYNQSk1hIdpI3B6X8+lbIkSn5zBwNt83DmTwaIYRjmDFR32zBZ+?=
 =?us-ascii?Q?n5aKlvf9SEfVlD+2wl9sv4XrSEjeWTEspHfbKETP+xOc7X8lwek3ffO4zQk1?=
 =?us-ascii?Q?kgoHjWZbdyu1DmghlOsfkM9xfqLkP1F8KLqkdmbtT3gjuvP+b8vwOcRlO3Zf?=
 =?us-ascii?Q?MWtehunU2QFzfE2BAPEPgAg7wh07CQlD3RgxNqm3dj0ARpY1BTwhepwN5Xzk?=
 =?us-ascii?Q?ZOO9FzMxk2svd4nZG7XG+GO8O61IKNNaCfhudRjCq+RagiNBNIVWGd5cNolP?=
 =?us-ascii?Q?iz4l1L1YJ89p9nZl7djTAOeEyNeOKByPydq51UuRAj5X/uVYFHu10vSK8R02?=
 =?us-ascii?Q?bJHsEu2CIKREJlD+UW1BDZeKyMlDJLBLUJc/U2eOqS/e4idvAFXajFMXU/rX?=
 =?us-ascii?Q?aqAkPCSszkzZ22gAEsdWCCb1iuZV+s9XlI+wre7f61H8tCUAGQHYQy3Y/Y8Z?=
 =?us-ascii?Q?Biqk7JHuayLbyKE1iuqyQrEosUkFuIvJ5IUYHKYw+1BsxSXdbSPy1hrRahqI?=
 =?us-ascii?Q?N2/4/35NIzZsmQyIqT5cN5gTl6H1WR0R2SSZ9rYkfQ1HUE2LYa3+71JWlSgr?=
 =?us-ascii?Q?FIQl9j1vlzpOOvz/YXHaw7oGPABe9Bf1L+HxTQwEpjIf3Poa2JNMO9+5HVDz?=
 =?us-ascii?Q?S7sEPEofm123O1Yvf+4kixlnCRydsDmlQUuYK0CAK0fs7RN0z+wRyq5mUH7P?=
 =?us-ascii?Q?l+Uv6bosyXY5OBl7hokjtzTVIB2OCzhDf8g/3/ZriP0gKJ0KJ3tiMNkGSqAz?=
 =?us-ascii?Q?6tN/zw0nAnXHFigmUmCQhYgDQbYD4nD9j3RXkhPehO255fgPjXrsw179GTmT?=
 =?us-ascii?Q?dlb0mI9pssDUjmuAiNTgdBN3AqDNcYbL5X61NLTHqsipiA+5nUn9BVScDNaW?=
 =?us-ascii?Q?sjb10rOAXaW4cFfPzaXjYiZwOOVeSBELIX+dRBly4tLo9e98Wy5U1vn0RJrH?=
 =?us-ascii?Q?5FsdMk64ZxnYU0yHAhZyfheAOLSTV74+LfglzpPdQjuNqJaMoA7NacAh9dlq?=
 =?us-ascii?Q?t4tBDOeW4ZOSVta5U0dQeLxslFWPpXUuchUyv7+pltKdhLtHn1ZEVwyRm4EZ?=
 =?us-ascii?Q?6Jl1es+4Lg61QtZ65GRi1JbYeyD16ToWgXmE8R260tLvo7fU/e8ULPDW0WAh?=
 =?us-ascii?Q?pruFU0YjmzFegj5zQuHzVv4SmKWRSoGx/XeH4geHOZi8iIbHAmO+bKfHgcM4?=
 =?us-ascii?Q?44ojUC2TA2Yn6gwP761sjmLk8kHk7TqlZYZTVsjdV0F6fQK2LUJnSD9mfGQn?=
 =?us-ascii?Q?v61AZqsqqJb6VSO0rNG3TxUp?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ced71f10-58d2-4911-befa-08d96d519f46
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 14:05:58.2667
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8JS7TxIV/o/XVTPfISXZ/b1gdduJfVPNhbJDlX52dp6npQ45YqbLIvt5nVtcNVH0sMLQKLyAXNPxNUbsizvOkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5598

On 01.09.2021 15:48, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 01, 2021 at 11:53:03AM +0200, Jan Beulich wrote:
>> The issue isn't just with execution, though, and as a result I may
>> need to change the logic here to also include at least W. As of
>> one of the XSA-378 changes we may now pass just p2m_access_r to
>> iommu_identity_mapping(), if the ACPI tables on an AMD system were
>> saying so. (We may also pass p2m_access_w, but I sincerely hope no
>> firmware would specify write but no read access.)
>>
>> Similarly in "IOMMU/x86: restrict IO-APIC mappings for PV Dom0" I
>> now pass p2m_access_r to set_identity_p2m_entry().
>=20
> Not really I think, as PVH dom0 is the only user of the
> set_identity_p2m_entry call in arch_iommu_hwdom_init, and we should
> never identity map the IO-APIC range in that case because a set of
> emulated IO-APIC replacements are provided and those require ranges to
> be unmapped so that accesses can be trapped.

Correct, this would be only a latent issue for now. The code passes
that value, but that path is not going to be taken (until, perhaps,
a future change).

>> I suppose an underlying issue is the mixed purpose of using
>> p2m_access_*, which possibly has been against the intentions in the
>> first place. We cannot, for example, express r/o access to an MMIO
>> page without using p2m_access_r (or p2m_access_rx), as there's no
>> suitable p2m type to express this via type alone. We may need to
>> split p2m_mmio_direct into multiple types (up to 7), I guess, if
>> we wanted to remove this (ab)use of p2m_access_*.
>=20
> My main complaint is mostly with the fact that some MMIO ranges are
> mapped without execute permissions when mapped by
> set_identity_p2m_entry vs map_mmio_regions that will map them with the
> default permissions and that has execution set.

Right - as said in a reply to Andrew, we may need to drop some
p2m_access_t parameters in favor of using default permissions. But
at least for AMD's IOMMU-unity-mapped ranges we will need to have
a way to express at least r/o.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:16:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:16:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176461.321120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR21-0008L8-HD; Wed, 01 Sep 2021 14:16:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176461.321120; Wed, 01 Sep 2021 14:16:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR21-0008Kz-Dg; Wed, 01 Sep 2021 14:16:01 +0000
Received: by outflank-mailman (input) for mailman id 176461;
 Wed, 01 Sep 2021 14:16:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rqR/=NX=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mLR20-0007uB-9r
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 14:16:00 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5d5064de-aaff-4d0b-b2e1-04820891abd9;
 Wed, 01 Sep 2021 14:15:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d5064de-aaff-4d0b-b2e1-04820891abd9
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630505753;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=9G1ixSwpVVNGO1qRAX6VZalDZxApZgkQDoR8Qn+vkJ8=;
  b=TsbSjKru0uqxfW7Z2gI3J5b+LdmwoiasA1xw0FDNYqTt+Mv1YCedGyHI
   QKUeJt26GR8aLlhAUCgjIfb0hO+6mIjjyEDbbtveSot1usJPJuvWmQyNf
   2NsLl/xNqgMNizd+38WZ/AqU4tlGLuPUtfuw3TB/y2nBpx0hEqFmTYbmu
   4=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: jReWgYl4LhKRC49tJO1X6kLz9XPWilzcEjx7uo+RU7C/pmTAF/49A7VNUznIYMmgmPzJhGwAMD
 b7kjXB2GgNsP2V+8FiD78hKAufV6WrT9K94ggJ7ZWWoJjjlDFbokr0R7KWtgmedrDCW0bed6es
 qtNVefiGvOCBv8n2wshBZEUN+JvxMpAfIIrfov99781KqSeY29eSxyGaWpbzj4wt9+TIH8qU3K
 pa0QvEdNb/uIfoc3SPuysCX0LiUVbjT2ZQ77VsiGjUEbgyq0dpvIzE0VnxPc40UJhL0k665V0v
 USu7frJixBq6jcAUVLjaSYOc
X-SBRS: 5.1
X-MesageID: 53539810
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:z7uTmqhJkhqUKspSXCnSq/OhrnBQXgoji2hC6mlwRA09TyX5ra
 qTdZUgpH3JYVkqNE3I9eruBEDiewK+yXcW2+gs1N6ZNWGKhILBFupfBODZowEIdReRygcn79
 YET0B/YueAdmSTj6zBkXOF+wBL+qj7zEiq792us0uEVWtRGsVdB58SMHfiLqVxLjM2YqbQRP
 Knl6x6TyPLQwVvUviG
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="53539810"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fYbYmCJH+WJklh4o8rDBPVj/33lbhgYmnWm0ogsc4QS2Jm4UVOtgaK8C3ylx3hoXIq+KNLqqCowcF7OdNKllOV+Mw9wob6TVf1Bpn2QZxzHi4Hb/WQikuS4+Zo8S3vZl4e2GZqUAg9jWi4L9OXnYD+l/WjxTMQK2AxAMcm+JcO7q5v4QJ9pq4hEXD66g+sYbwg7uFmOWnt8U2j+Rm+bN0+8KFjLPsxnKlDv2tVwYt3+y2V9jLsWspy/xtAaYn+QvIsL8ZwDwpABJcput94pmiGPlE5yMW64NMY9C3QILsGvRbkLGNGYL9JT8EGmcDB0goJjY01O1TpMSXO20CXMwLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CiBzRMexSlFVfa1By3ARHlf7/NosDQULtDXwjFeft14=;
 b=Wa3zR+FNV8Z1gbEQFgdsf/oFNVGQyg8yediOTYI1UeP+0BySo1ZIQ8zdcLi+2gJnCjgXUek026qJiBBSjDl2Q+Uqk0dPY85EzqKtrCTFX5Y2J6KpqATdp4tbnms/pP6XfvOMMtCB8nq3FzC6UaHisIOGjyemi18EijjDZWp7KfGTUFyAEQncCfUoDSlDw8UZDy6Q23RkyHBtNAViSf+La6lcHLp+ljyYob+o1pQ3WPDOW1znNeSq67W3GU2AaworQvC+OmMFOXf5nEST0Xs764pwcMOWDgOLbSAD43D2l6o5v0/4y8OoCbKUUB0+AH2W158VM1c1AeOM13JUKNXifw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CiBzRMexSlFVfa1By3ARHlf7/NosDQULtDXwjFeft14=;
 b=VdL0Rp8yod+mjLG0Tisy0+BaIegQ1GRk76S/9mYeOKVX+cklOFIFD0yQ9Q4Xj/tqRh3EMPbLlqLtdmwbQpFP7R4tNTio46zyujxWcByKY6SYvBgONu4Zvzcz4Unr5CBzageoXlayIjkmExIFI5MWNCnPfJv7rQA4O1X402Gs25M=
To: Jason Andryuk <jandryuk@gmail.com>, Xen.org security team
	<security@xen.org>
CC: <xen-announce@lists.xen.org>, <xen-devel@lists.xen.org>,
	<xen-users@lists.xen.org>, <oss-security@lists.openwall.com>, "Xen.org
 security team" <security-team-members@xen.org>
References: <E1mLMZy-0006TJ-Rw@xenbits.xenproject.org>
 <CAKf6xpsAz7oPX+PdZokm+BAqreqL8rnrAd9UYvdz=F0M-bcMUQ@mail.gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Xen Security Advisory 378 v3
 (CVE-2021-28694,CVE-2021-28695,CVE-2021-28696) - IOMMU page mapping issues on
 x86
Message-ID: <43ba10b9-4b20-81d4-6bdc-18f9c2e2e0bc@citrix.com>
Date: Wed, 1 Sep 2021 15:15:42 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <CAKf6xpsAz7oPX+PdZokm+BAqreqL8rnrAd9UYvdz=F0M-bcMUQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0073.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::6) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 233b439b-3171-444e-d1fe-08d96d53000b
X-MS-TrafficTypeDiagnostic: BYAPR03MB3861:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3861272ADC8D5D63FA919C73BACD9@BYAPR03MB3861.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: YlrVzEdOwyd4p4YyT1+xDKvUIYH2vTA4KKrgtaoryH3DoBaZ46y8fKkPSzQ9uzSu9oysNpyIceLKBiVg/7kT7fGthqRZr3977LbToSj9wmXxb+8uHdgBOIbjnSOI5UV4jJ2Sfa6tDvZjl9SHipB1aAsEFfzijD/eVN3zDalPBq1SKD0v8yufF2tH8YOtR0r+rd7109tZNVbTY2TbU+QMYNhnHccYRG4OujO2Nc5Wm5N5E6E0eML9Uji+BWZvaX+RW4ENC042hiOg8Z+VVDRzQzJMXmTvcFwsSX3uFFmrCzq6H/3E6I6i8oowAJoyRbKcAfxjRJHjAqmhuNnCAOjGtMBwnsT7yZiRdFktmioSOEILIluQM9hA8t7SRKxEOZ86YUataMsPnUPgS/ZxL4nky1JjCmINEQjrLI9fAoSrPO5uJ5czYxUN8+xY6xxOxZ+hjZK34xx1aGJ85XvncNKNNoSwrDfxMsPIxqtr0wBHhfWwnFw5m2kmVTpOj+nCCiM+cHphpM0nxDUnEcV2AoajXtD1aEC/4Y6A8zZn1CBJvuCxUyhNJkN7BhZFCdU0jPqddc4BxmJiyYqSv1bASq8MhT+ZA406dxX28jwZoMHAqONhEPVRnWLTwMzYLsHYeuBgCatBg5hpof7yldRsy91f1GPraIgZZce35lotcc8QDvj+b+2ha/lUUmgxWnKf5oUYXa2rQMSJCBT38c/VY0HzbH1PzkiqFWrS0MqiJNYBjAE=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(346002)(376002)(39860400002)(396003)(366004)(38100700002)(31686004)(86362001)(6486002)(5660300002)(8676002)(66946007)(8936002)(83380400001)(6666004)(31696002)(66556008)(66476007)(15650500001)(4326008)(316002)(110136005)(16576012)(26005)(36756003)(186003)(478600001)(956004)(55236004)(53546011)(2906002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bG9HcXcrVkIxOHNFSXZBWjVwTUQ1d0NPWDVZSUh5cHptQ2V5K3RRQ3RDSVJH?=
 =?utf-8?B?TjU0UGV2UWt3SFYwNlVkdlllbTFtMVRIVjdNc2diYmVGSWtRTjJLMHpnaU9u?=
 =?utf-8?B?RGxDVm13Rm5PdGoxR2ZWVXRBRVV5bWVWZlRadDdMSyszcHhPRXdLM082NG4y?=
 =?utf-8?B?U20zM1dVRXEwbU04TWlRMk5ZeUFHdkJpRHN0UEJlc1dlV0c0WWRYUDRCa3pv?=
 =?utf-8?B?NVBBSG5HeHlPbS9KcjBhMFQySU83YVVYUlppbzlsYm9hN0JyYkNsb3Y2TDMw?=
 =?utf-8?B?dmNDK25KdDhlVDVjTHpsRG9WWkVkQWhEaG9CMng3MUF4L0ZuWWtKc1VpVGwx?=
 =?utf-8?B?VkhveGlJZGxZbVBkNXU1VlBSSGo0SXNyOHNOZ1BRa1l3cGpUcE1YY2ttZFlE?=
 =?utf-8?B?c21SNllQVncveDJSN1lIazlnZ3NGdFA1TklnMDUzNVozZnVDMjBIakEvcVMv?=
 =?utf-8?B?RmM3NWcyV2llcnBCSWN5RXVLMTBNZExSR0xHRFlMMHNsMms3V3ltTXJnQWZT?=
 =?utf-8?B?U3Z3ZWIzT2hzRks4MlNNVC82eFlUUy9KRm1qd3RndnMzLzc0aS9JYVZ4UUYv?=
 =?utf-8?B?UWh1YUUveW1EQytyMkFsQnNuYkJTcUhGUWN6MDkxZE1JTzlRRWQ1THIrRDJp?=
 =?utf-8?B?LzVnSUFpNVB2Q210bE9nVitKMXl1ckgrN1Jtd3A2VHhOV3VtcG9GSG56Q2xk?=
 =?utf-8?B?NVZmWmFydFV1aytUZDRtbW8vOG1yYlkyMlVVYUduMXMzQ01Tc09CSzhQNkVD?=
 =?utf-8?B?UGZVamtFaW41NzljOHJFNW5nWHJ3N1dyM2o0aHVua1ZEVHFjbEVOaUc4RVJW?=
 =?utf-8?B?cjdwb3dURGthM1FOSVRGU1M5bUwzNlFXY0NBWjk3Wkt0UCtqYlNzV0ltZnRZ?=
 =?utf-8?B?bGRycXZhano5dEswYzY1TUlPV3NzS2grQXJIbjZTZnk5RWN0dVY5K3MxVGtN?=
 =?utf-8?B?V2VwdEw2cWZOWTNjOUtaTXRCQ2R4UUdFbzZBVWE0NktSdlZzQXFFcHdkRGkr?=
 =?utf-8?B?NWdXc1hoQmJRS0dTSE9mZ3gwbWgzaEczSjQxTWlNYTFxNUxFV1U0TlpPWStE?=
 =?utf-8?B?alJoZUttcXBFMkVzN2JMM0lDWVpSTDJ1THNaMVNjdk5QOXlSSW5OQ0FrTjJJ?=
 =?utf-8?B?VGs1bWwrS0xhZW0xNnlIU2pzL3NNRkVCbXdaTXJwWnJnTzBKbVJpcHJGdzN3?=
 =?utf-8?B?Vy9HYlljTExCYlh4VFNTZDVvc3h5U29HNTdSV1k2cXZ2L3lQRnBmZFFEcXFx?=
 =?utf-8?B?b3h5NUwvZm40YzlOeG5lbmlORnNsa3luSVdpMnBUaG1rVFdDQXFtQmdmQUtC?=
 =?utf-8?B?UWpJMENqRHdqbHZ5c1I1UmN4dFc3NTJzRy9lblVPMXpnUURJbHFTUmtvdU1M?=
 =?utf-8?B?bVlabGxiVU13SUVLUGh2T3hFbFlCTWZ5akFwUVNjRU9ycTRqR2Z1VkhBRTVW?=
 =?utf-8?B?aFpmY3pDcHl3ZjBaYWU0eC80SDRlMDEweWtvVXo0SjBUeFhrS1hMenljU1dF?=
 =?utf-8?B?V290azY4clBRTHFEYXRoMUxJeVdYVzNHTHdNUEdHU0dNSkI1SCtBemdkb1JU?=
 =?utf-8?B?c1Z5dlpuVlBBeG54dkdIOHNkcXBlakw3TmpYTkhSbGNUa3QrT053eSt6a3Nm?=
 =?utf-8?B?RW1Ta3VLNitPbkY4eGlzK3drMDVIM1VqTWtOUDY3VWhVQ2xlTUFOS3JGSzA0?=
 =?utf-8?B?M2VIelhGZ0VPYzVvSDc3UEs2R3NiVjM5Wld4SllzeERqOHQyTTQ3NXNza0x4?=
 =?utf-8?Q?0DErMNqBZ9c7rzkGe8H8u5vpqTcCFSAn3gpVEJm?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 233b439b-3171-444e-d1fe-08d96d53000b
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 14:15:50.1472
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Js9mm6UeiYw54KeNyeuL1/3dX+wMRzRqyXCYzraVNrPAlbz+LVflrRnXipHl6auEZ3JCC9BqUqRaJl448oUYifNePqRHFx20mzllBhakcgI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3861
X-OriginatorOrg: citrix.com

On 01/09/2021 14:22, Jason Andryuk wrote:
> On Wed, Sep 1, 2021 at 5:34 AM Xen.org security team <security@xen.org> w=
rote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>>  Xen Security Advisory CVE-2021-28694,CVE-2021-28695,CVE-2021-28696 / XS=
A-378
>>                                    version 3
>>
>>                    IOMMU page mapping issues on x86
>>
>> UPDATES IN VERSION 3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>
>> Warn about dom0=3Dpvh breakage in Resolution section.
>>
>> ISSUE DESCRIPTION
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>
>> Both AMD and Intel allow ACPI tables to specify regions of memory
>> which should be left untranslated, which typically means these
>> addresses should pass the translation phase unaltered.  While these
>> are typically device specific ACPI properties, they can also be
>> specified to apply to a range of devices, or even all devices.
>>
>> On all systems with such regions Xen failed to prevent guests from
>> undoing/replacing such mappings (CVE-2021-28694).
> Hi,
>
> Is there a way to identify if a system's ACPI tables have untranslated
> regions?  Does it show up in xen or linux dmesg or can it be
> identified in sysfs?

It's possible, but a little convoluted to do.=C2=A0 In dom0 (and in an empt=
y
directory) you want:

acpidump > acpi.dmp
acpixtract -a acpi.dmp

On Intel, open up rmad.dat and hexedit the first 4 bytes from RMAD to
DMAR (yes - really - this is how we stop the dom0 kernel from trying to
poke the IOMMU directly.)

Then disassemble (iasl -d) either rmad.dat or ivrs.dat depending on
whether you're on Intel or AMD.

On Intel, you're looking for Reserved Memory Regions, while on AMD
you're looking for IVMD ranges (specifically, types 20 thru 22)

These, if present, describe a range of memory needing identity mapping,
and a scope of the PCI device(s) the range applies to.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:19:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:19:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176488.321136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR5g-0001Fl-8a; Wed, 01 Sep 2021 14:19:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176488.321136; Wed, 01 Sep 2021 14:19:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR5g-0001Fe-5g; Wed, 01 Sep 2021 14:19:48 +0000
Received: by outflank-mailman (input) for mailman id 176488;
 Wed, 01 Sep 2021 14:19:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLR5e-0001FR-Le
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 14:19:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a6bf3524-59be-40ec-a7af-5a65a7662838;
 Wed, 01 Sep 2021 14:19:45 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-pO-0Ug3aNSKlsbxObN8-zQ-1; Wed, 01 Sep 2021 16:19:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Wed, 1 Sep
 2021 14:19:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 14:19:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0008.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.23 via Frontend Transport; Wed, 1 Sep 2021 14:19:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6bf3524-59be-40ec-a7af-5a65a7662838
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630505984;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=tQvyW9DZnTbNz1g08eDHxiiv3P155ZpR0UZdx8FIPGY=;
	b=XcHdqP7S2a4Ka3OzIdYDDFCELdnzpDskxnc+IuFxRO4f+eSejIVDR19t73C6cp8tqWsSa6
	kRGKTrz7/ZbOe2SEFIxgd+jrZ+7bX9TyjEvlwiCHlQQvOGT7gfcYSlEufs0DRyGoPRYKNJ
	KCLos0K7CJh7GLH2FdHwdL9uIQYkfrs=
X-MC-Unique: pO-0Ug3aNSKlsbxObN8-zQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d1JwmWHmVkxC17IAWtWsXZ6e/7Yn/j2Mfg1mvPh++ppSeFFMXhdR2nwaoYuWFPDERhs8WCsRdB3vuPiR8AenBRtoTcyDbMbHWpPE+RrD40MSbAEaTQhst89fhJH4Bl1xfrvLjmwDl9hlf9SXNw9J3uCa9la7yWX8JxHA0XB0yrRaz2xZVD5+3tIfrgteji1HjWDLV708RUOmas3KlSyxxAXooHKjSut8kiCns4EtKoDyuSQ/e5Q+ZC+9J4O4iVtVa1sF5gM8pRIqBNz71GvbCYu1i2wK9ZA9iYuQ3gA93vJpba8dihS91H/FxgBgsbAiS/br3pMWFvR4txK3FkKEEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PmraVwts6NZHguDo6kFeRqkMKWV84aKR4N5NB7fnG4Y=;
 b=Y+pNbrhgjZ2lzkIpKUA27VJVoraviRC/tTOVuKwxa6GKd1GIUBAmPy9bxgjSLyis/afO3aXzdMw6A6yGX3TLAvf5Y9ppEWwRbLdtU/Cbxn8zc3lxsSLPe3HmHfTMLsZuiHd11EKfTdvplk/rqdjVLqzTTswrFpQIrydmgrjxp1RFurkQRPwRFLoV5HzBWP3uVHD6TbQX4TlpxsjvPqqhJUME/y1Iofv92g6EXt8zp7Tq03q9b2LP9yLojtspmW9QaAEYxjQJ9Q7wMPpYurEHanO0/lELezJNu4+g+5SQ3XvkCxA9BuL0wKjnvHTlIp2rWeX1pbubVT21+Dp4htvy2A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
 <YS+GmC7IjxoODTu+@Air-de-Roger>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
Date: Wed, 1 Sep 2021 16:19:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YS+GmC7IjxoODTu+@Air-de-Roger>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR1P264CA0008.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 18118962-76fd-4960-2a47-08d96d538a41
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3933AB7E3994D820FD3BE7C4B3CD9@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	99wmS3++441NMUm23N7P34rGaiR7JqlOaYSJDF18MlTHLNJRYdM+HHYhZG4PgZ35g9zGXnfC3DWduPfD62Er14lQEt47IkC94aF3enKm0nnxZKMMp9v3hfOR5KwsYpI+/wXHlmVPHAYZou6Hkf0KMJXPkpOTEqDmktOrOZAt4Ts2s7lTlBxalVhrcegr3/yMLDLPjEgLhfhReugiMkzV5GfqbX3hpd62TEhgIP+Ai9X1ct8knDODHnO1vz6JW+PUAuDJzrBzMZ85DQKaQopnBcvVBGJJektmODFb/q96T+grNhRw68oCEJXrhsJWzVPKQ29RPiYu2rIsULzTeqlJ0k87d/4gaNF83GpoiZMXYsorRKuGy6zCAaJUPqrG3epstNK51XLVweS/2wa7QHUWQgQcviOVDibDgLgSZ+Vqs+SmqPDjNoMk9fI3gvwV2Kv7YaYQwXlAr7ELUnW4fdKvwyDMnJ2T5LaIhPAHDbd1MORTAhZ9Xmfz7u8waugXA7j7IuQex51nbdKHXmnjFbqVg30sNPMg4WqgmuQPP8SJWz6/y13hOh8hIyoW9H3nSGRsULc3r3rfaN1c6WPqo7/6Nw8Susn4iJB9m/Y4BKT4kdWcIHp2AXpFmbFcXjjolEa7xkXJAJsixct4POrYiC2LlGYKznjy80ImQ418V83Ik6ZCb79y3WBAGOGwnVbS2CHiI2X59ZSHWkw2pJgxix7cCAFSGDzMJ+nGAMCoYrgI3Aw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(8936002)(31686004)(4326008)(36756003)(26005)(186003)(6916009)(83380400001)(956004)(2616005)(66476007)(2906002)(66556008)(8676002)(53546011)(38100700002)(31696002)(316002)(478600001)(6486002)(16576012)(66946007)(86362001)(54906003)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?XFEJSOHv4THPSg81W2K6anC7cOpTYyckTO+8CMB95Tiuqa1M8wi6APyVfXQd?=
 =?us-ascii?Q?CcvfRCCtl1jAwDstEUVcxm8kKFDhVkyVSnd+53n5vmSfT2XmjcqH+owkc/zh?=
 =?us-ascii?Q?cwO/M6BtWTngj47xS77Rt1X0fDQwSdjWsigsM9Hyoy/MPS9TtjkSe9GB1OPI?=
 =?us-ascii?Q?Hg/XP5YtKT7o0Ax8XrtymCfuiLBu4qOlinpRMqrmNmRUrUqUmWHQaXBShSsm?=
 =?us-ascii?Q?lsiTqjKS3BYDs5pcjMM4HHnBpBc3+vcN/6bzDF3RpkvotU7MqarubPG9ZcVv?=
 =?us-ascii?Q?DmnQH7uTzBM374MBP0asBVPkg6fJqU4tvDDtxNQgrh2X689tHHd1iyLIB5D7?=
 =?us-ascii?Q?gpUoO+T9hFK8BO5QOPIv8sHQlu0A5f0xG7hV9ovK/d/oFMrx0+qpJOdSUMpd?=
 =?us-ascii?Q?/kw2wpHB+ZB8sS7WzVfynngOjSzMWEIcBdnAjiK8tOwzzm70Nv0nHDP0AG6g?=
 =?us-ascii?Q?gO4pgUuywimOjvIlwm9tz49B+tfksw1I9urGoDiTRrRqebt6dYtr3pox2FyE?=
 =?us-ascii?Q?gI5hqRmeLwEWdDDfi/OMP+aTEdSASUmAShbni9t4kog5nDjT1LBS+5cOXt2P?=
 =?us-ascii?Q?/JhoRhGtzrH1Ij3eI0+pXA57R8wYLbLvyFOh2ONNPqVy3hCecVdV/jzROyZx?=
 =?us-ascii?Q?RHqzhoa7+uggMObF56ZjjgU/noYJjlrIZmPRtQ1IlQAPt+H6xjgy4BNbHz5L?=
 =?us-ascii?Q?SkTxT/ppAt4cqnsRhENN3/BEVzqUm95ctp9ickmN9W2zMaeBLw1h9bix2z6d?=
 =?us-ascii?Q?L1GXKPJ/iBMlRa1WNm+rfSaP2wrr9gpYqG/nGxz6MW0deg3ajCjWoymTDPFl?=
 =?us-ascii?Q?3L1BsE8WZ7ZPNl9BJ2AN2pMefihpMGxfqVLX93aKpVJQHyvyYCiI8RvLkSz+?=
 =?us-ascii?Q?NpmM3nNRjBq4E0JdzQXWI2Y83qNCfd3KD+IJqKaXk4YTgUnoZq/29iLLnMvl?=
 =?us-ascii?Q?LNN7X8NrbMyv/FVycHhFloGFzqScCYVPXA6sF8OOg2Jj7SYwGRNpxORG+c3f?=
 =?us-ascii?Q?r7/C6NIk07jNturre0twO/usWrmXR5RQ5x/UWuIdx5pd6pJOKJrxk38ahMvA?=
 =?us-ascii?Q?QnZ4Wf6uNutDfEjGNhsZ8k2khLc4E6psor04ixVwjUNZPmVWhabdSW/fjGQE?=
 =?us-ascii?Q?mba5oYP2uZJZit8pRLasmdfPLvsBJ/YCf++Mm2pgP6Zaa/rbEaPFkyjsQRmx?=
 =?us-ascii?Q?ZDcgqLX0rzZZbGpnZzYvRkuw5dGXg/eDPIU0NbldQSBWh+cwvMs7UFFZSwfG?=
 =?us-ascii?Q?cpSnFNmRfxyDfinpQiOSd6xShtPbhlLYUEym/Dawl4nBDUZ86Hm7OKqtqBUh?=
 =?us-ascii?Q?eKv/Sj5zODiAuB62GNbUuYjq?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 18118962-76fd-4960-2a47-08d96d538a41
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 14:19:42.0074
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MNN04Ce+d1dSEimCtB8IPK+zoD/+YoB14W17VtEmOnDmweG18petr3AoDuFzcpFe3VSkV5fJhHEcnDtXttgdhA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 01.09.2021 15:56, Roger Pau Monn=C3=A9 wrote:
> On Tue, Aug 31, 2021 at 10:53:59AM +0200, Jan Beulich wrote:
>> On 30.08.2021 15:01, Jan Beulich wrote:
>>> The code building PVH Dom0 made use of sequences of P2M changes
>>> which are disallowed as of XSA-378. First of all population of the
>>> first Mb of memory needs to be redone. Then, largely as a
>>> workaround, checking introduced by XSA-378 needs to be slightly
>>> relaxed.
>>>
>>> Note that with these adjustments I get Dom0 to start booting on my
>>> development system, but the Dom0 kernel then gets stuck. Since it
>>> was the first time for me to try PVH Dom0 in this context (see
>>> below for why I was hesitant), I cannot tell yet whether this is
>>> due further fallout from the XSA, or some further unrelated
>>> problem.
>=20
> Iff you have some time could you check without the XSA applied? I have
> to admit I haven't been testing staging, so it's possible some
> breakage as slipped in (however osstest seemed fine with it).

Well, I'd rather try to use the time to find the actual issue. From
osstest being fine I'm kind of inferring this might be machine
specific, or this might be due to yet some other of the overly many
patches I'm carrying. So if I can't infer anything from the stack
once I can actually dump that, I may indeed need to bisect my pile,
which would then also include the XSA-378 patches (as I didn't have
time to re-base so far).

>>> Dom0's BSP is in VPF_blocked state while all APs are
>>> still in VPF_down. The 'd' debug key, unhelpfully, doesn't produce
>>> any output, so it's non-trivial to check whether (like PV likes to
>>> do) Dom0 has panic()ed without leaving any (visible) output.
>=20
> Not sure it would help much, but maybe you can post the Xen+Linux
> output?

There's no Linux output yet by that point (and either
"earlyprintk=3Dxen" doesn't work in PVH mode, or it's even too early
for that). All Xen has to say is

(XEN) Dom0 callback via changed to Direct Vector 0xf3
(XEN) vmx.c:3265:d0v0 RDMSR 0x0000064e unimplemented
(XEN) vmx.c:3265:d0v0 RDMSR 0x00000034 unimplemented

> Do you have iommu debug/verbose enabled to catch iommu faults?

I'll try to remember to check that, but since Linux hasn't
brought up APs yet I don't think there's any device activity
just yet.

>> Correction: I did mean '0' here, producing merely
>>
>> (XEN) '0' pressed -> dumping Dom0's registers
>> (XEN) *** Dumping Dom0 vcpu#0 state: ***
>> (XEN) *** Dumping Dom0 vcpu#1 state: ***
>> (XEN) *** Dumping Dom0 vcpu#2 state: ***
>> (XEN) *** Dumping Dom0 vcpu#3 state: ***
>>
>> 'd' output supports the "system is idle" that was also visible from
>> 'q' output.
>=20
> Can you dump the state of the VMCS and see where the IP points to in
> Linux?

Both that and the register dumping I have meanwhile working tell
me that it's the HLT in default_idle(). IOW Dom0 gives the impression
of also being idle, at the first glance. The stack pointer, however,
is farther away from the stack top than I would have expected, so it
may still have entered default_idle() for other reasons.

The VMCS also told me that the last VM entry was to deliver an
interrupt at vector 0xf3 (i.e. the "callback" one).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:24:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:24:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176497.321148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR9t-0002rr-S8; Wed, 01 Sep 2021 14:24:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176497.321148; Wed, 01 Sep 2021 14:24:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLR9t-0002rk-Mu; Wed, 01 Sep 2021 14:24:09 +0000
Received: by outflank-mailman (input) for mailman id 176497;
 Wed, 01 Sep 2021 14:24:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLR9s-0002re-FG
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 14:24:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLR9o-0003rO-8c; Wed, 01 Sep 2021 14:24:04 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLR9o-0000BT-1a; Wed, 01 Sep 2021 14:24:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=1BPnIhtExWspVUUxouiBfqJrd9IwX6xVp7qTG38Djw4=; b=dbi/pPqONio3mP7UO1zDHTrSNc
	puc1wVTvvd7TmQ9EPXVvHkwRMFwAmQeYrMBbhy1T/XQAm/7CAyewfOmheM8bCMrhlxZgqUXVb+pWU
	UvD5uRdFGZpGK0c7ppS4FmfIXggeI6+c2N52iopJLQjlenmeIcgdQSzlMWgz5hJjcj0o=;
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: Michal Orzel <Michal.Orzel@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
Date: Wed, 1 Sep 2021 15:24:01 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 01/09/2021 14:10, Bertrand Marquis wrote:
> Hi Julien,

Hi Bertrand,

> 
>> On 1 Sep 2021, at 13:55, Julien Grall <julien@xen.org> wrote:
>>
>> Hi,
>>
>> On 01/09/2021 13:43, Michal Orzel wrote:
>>> Introduce new Xen command line parameter called "pmu_access".
>>> The default value is "trap": Xen traps PMU accesses.
>>> In case of setting pmu_access to "native", Xen does not trap
>>> PMU accesses allowing all the guests to access PMU registers.
>>> However, guests cannot make use of PMU overflow interrupts as
>>> PMU uses PPI which Xen cannot route to guests.
>>> This option is only intended for development and testing purposes.
>>> Do not use this in production system.
>> I am afraid your option is not safe even in development system as a vCPU may move between pCPUs.
>>
>> However, even if we restricted the use to pinned vCPU *and* dedicated pCPU, I am not convinced that exposing an half backed PMU (the overflow interrupt would not work) to the guest is the right solution. This likely means the guest OS would need to be modified and therefore the usage of this option is fairly limited.
>>
>> So I think the first steps are:
>>   1) Make the PPI work. There was some attempt in the past for it on xen-devel. You could have a look.
>>   2) Provide PMU bindings
>>
>> With that in place, we can discuss how to expose the PMU even if it is unsafe in some conditions.
> 
> With those limitations, using the PMU to monitor the system performances or on some specific use cases is still really useful.
> We are using that to do some benchmarks of Xen or of some applications to compare the behaviour to a native system or
> analyse the performances of Xen itself (hypercalls,context switch …etc)

I understand this is useful for some setup and I am not trying to say we 
should not have a way to expose the PMU (even unsafely) in upstream. 
However, I think the option as it stands is too wide (this should be a 
per domain knob) and we should properly expose the PMU (interrupts, 
bindings...).

> 
> The steps you are mentioning do make sense but using the PMU without those knowing the limitations is also very useful.

Right, this would require a lot more work than what I wrote before.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:25:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:25:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176503.321159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRB7-0003SQ-6k; Wed, 01 Sep 2021 14:25:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176503.321159; Wed, 01 Sep 2021 14:25:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRB7-0003SJ-1w; Wed, 01 Sep 2021 14:25:25 +0000
Received: by outflank-mailman (input) for mailman id 176503;
 Wed, 01 Sep 2021 14:25:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLRB4-0003S6-SA
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 14:25:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7058c3e0-0b30-11ec-adf6-12813bfff9fa;
 Wed, 01 Sep 2021 14:25:22 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-x6Wda70zPZ-kkgxYV1AHfg-1; Wed, 01 Sep 2021 16:25:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (20.178.127.208) by
 VI1PR0401MB2333.eurprd04.prod.outlook.com (10.169.133.24) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19; Wed, 1 Sep 2021 14:25:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 14:25:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0P190CA0023.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 14:25:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7058c3e0-0b30-11ec-adf6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630506321;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7HPcJqXemlh/M+q3GUqr/9ybdIci7HrM7vUhZHsFzCs=;
	b=jXSpoqUVYZqiwSXr/VF1HIvKiupZnhD50emelEkkorfyOcrCzB7U7508AM5r1VkDscr2Sn
	Fja6Y9EhtfbgTpnUCE6tV+kqn2HyEZBuTssJ+MQPRfNeUefrYwowh7Ll1NkhHrnyB7zDNf
	IDUc3kNJzD10hrs+aHMrF9CJzwpbOJU=
X-MC-Unique: x6Wda70zPZ-kkgxYV1AHfg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dks4BD7DFn9qEVxSaOoiPrPtnl9MG6K1lZnD4vLbPzKFVTHsJ/9HghzsSkwApTNu0Z3nEyNtytcYBlfDMm7785xzzVg9ZRvjga673CwOqSiepFJPzbzwkEa7JkKsinoGaolHlb6MMjElfDbnR4COTIMK25mo4M7ohtOfwcN00iuDGuuRu1qKz1UbWLhD+XC0oi8yGyVye3XGmYyHnyQuuLs3dXcOY/ObpSzPp+TO9ppSfk0EfZzISdPK7mZnETimbDwdMzu5gey+e9D4xdFABjhPIrMqCZ+5v4isJ+KoTaB+m+C2sGN0w5NB817Au/NYDQeYgafhSYR5GCaN97zWqQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3Fevk7k48OM6SAmkM3l2HgTjJcLe/LKPJf4QIPKH2os=;
 b=mnp9xYwr5kj2IKbgCDJsqM5b96f0/McoZqrsfZkfyDSnuF3zvxYhOYx2bjjWTVxLVwc1/gRf8yGyt2fRSLwRRLVsNSl1r9n/P+kJtrIia8htaPApFmDO2pwnuyIT8FUdq4ksnCVbc9eqc9pkDkz41pFpMBIxOgRr46Ef9O26B2plmoPd/tJCZ671t8PqnS5H45Oy4TRuH3ukTbIvYaHfke5Bs5r97D8qHhVH6glrq8tFmDLaOsx21X5SFKA1EAKZsu/EVsba3c2jvhj86C1W53MgsKcdmRnnIY64WpwoijXasQb1HGQpjzaadB/CnmvzgyKB4G4puDZ/XGAddrcnIw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
From: Jan Beulich <jbeulich@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
 <YS+GmC7IjxoODTu+@Air-de-Roger>
 <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
Message-ID: <e40194de-f490-d917-0626-cf13d0f3c976@suse.com>
Date: Wed, 1 Sep 2021 16:25:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0P190CA0023.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:208:190::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 44a8f791-a8cd-41c7-8231-08d96d54533f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23339A93FBEB6EA4E82E20F8B3CD9@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tsB6AwgsbuRyQ8bEJW2Z/TwBDf/3hhixoo7t3335gPUAmaLzFC/ph0xT5bVdjRdd8UlEZHua+m/KNhnxb8AlxxqjwrJfTJR/KoTBFIqnkfxfGFkhWvxkCx4svY1HVzsQK5497htAnt9kBmOPiWIZwf+K0Fw/NEYST9DZt7o+x34jLXHrD6Puw6avMAezf5QX7+5E0YxlcRszR7mX78182OUISnCLxPnUHlRrCxT+paYhPtsLXUZPKFupA9jAQxdspGFhey/3GLhuNk2T6/GcNSXx1sLFrTFpiJyX9vc4TntlCxA6AwPwnyZ2TmuN3rZ1niCkw+SFCZ8eOT1NJzqHzl+zo8iX1yUBMj7XHdfhH4hUKxrKb/+uIHR6P7RNK6nvP91E4X3xFxStD+GX3Q1QF6PCpAMwFoQReVFbFcrmhZkSk90ZOyS1iAA8jwUke/YIggw1dTKEaq743cn7KKLxzgNjH6F8oh3/xGn9Z4smr5VhX7HIPMKVILWU67Odi/cFyo/Q+kHazl8suqbWX1oEQjpftQp5W6SeTdCzYsW9bj0agcs+zhd/tFLQ9egscFG6AgbgNfHXiLFDpp39f3Wq2IrxcnAMIJI5FnuJZAkScWJxOw0qCTC2Lc8iytMxrAo9Z+njW/XU1M0w0HJlSlkkl5TW2/d+tM3VxUOaoOn4TJZaDFV+0m6o4wqWtxuuuJEdUsk/kIlbFEE3GnQEiZuRYoAD7p8htz/vNiXyrZRL1PF5x0/MGtqA4WjGwbjbwJr5
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(396003)(366004)(346002)(376002)(54906003)(53546011)(31686004)(956004)(2616005)(16576012)(86362001)(316002)(186003)(31696002)(36756003)(38100700002)(8936002)(8676002)(5660300002)(6486002)(2906002)(83380400001)(4744005)(66946007)(66556008)(66476007)(6916009)(478600001)(4326008)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?fh2bxSI7YJ1P41qfjRRzA7H5A0v6/E7j73CdJvOr1Us0BR/HttGvxElM/KRN?=
 =?us-ascii?Q?u86BrmXDeOAOdN8ols/w9nZInOvAxiWOJJr7C/AsUOiwXAwkG0TyTHR1LC5k?=
 =?us-ascii?Q?ErrOSoGBD5EonwYvyiNBCd3HMO0cxmQDrB8ryAO3zYEwVwCq6wk+HVHGy/xb?=
 =?us-ascii?Q?bDZ5/98LgaaoioRaAPEg4kFEOUuI5q2LvhXHdKwzhob2UtnBk6OMZgmkHn8C?=
 =?us-ascii?Q?sx7EliUZvp6olGtucIgy68DAqXZNVE8YOEjV/ayJhfdbDLgGeS8LDQbKir8y?=
 =?us-ascii?Q?DdTYmwaTpb29OD4vsaea35Qg2iSfXB4ainD4iMX5AHhiLjGiXuTjOyUhyM/4?=
 =?us-ascii?Q?sdEBhjdkl61d3xNlE4Ab51Qy/qdF6PvZOHL0f8Y/6cfIiMQJf1KUm9ZcnYit?=
 =?us-ascii?Q?kHDWQ0EUcLkv3Hp+inbhR4zGbJKpsEnauQS0X6Qucoc94PW4pKzarxJbqj9E?=
 =?us-ascii?Q?BhM45tkM1DRqw0NYlygf35htmMLi6TXEXXA2ntBrX5R7BOppL23VufJqGfoX?=
 =?us-ascii?Q?gjlIQr3RMH2Y2zr0mTiJETL4mFnjEgTPkEZZ2mcx9fKCuXFgyDQCzl9KXiZg?=
 =?us-ascii?Q?LKOG8AfHLnNRLoJ5KtPZmsgOjVheUnmfckYLe/xyPrmyBoBXo6TlCpq1mOx5?=
 =?us-ascii?Q?DQ61U/2GvXKq+bFQwZBXvRMQyEWBbJF/klh6MH03AW0/IuD1lebcTk5Cpa0n?=
 =?us-ascii?Q?yxHAZ/Srwzme5uFZh5W46JDfVn+9N6lshTOgeK6PSH8IzBDsyP0zSLFiRCWr?=
 =?us-ascii?Q?nk9HukzGoyFuW3y3So/HeerFeVXG3cOcHoeEUwOKCTnfQq1bxKmNms7j5Awy?=
 =?us-ascii?Q?h3Fz8wVHUEYKjSCmuKCh8URB8s/8z42JTiJfQLRZZNV3FKWUPksqlE/jGIkl?=
 =?us-ascii?Q?hwiUHcpo7C9eO1JvjYMsji3nO+266AJUD+MKYBHdiF+OIvRj4ZjmAKDFNdxI?=
 =?us-ascii?Q?u0Bp+xNcALJDz8+RA+mLSJCejRjfrSaKWu/hlgbw7U6JK4bY56lcs+CMqgaI?=
 =?us-ascii?Q?1AiBu/7KwDRdH3YgRtADWZArQePyI1IYFj4vq4FIpB9Z+Tx6CYi/cdSP72CT?=
 =?us-ascii?Q?jPrrwhH05fleqm4g9ajnsx+d479tgLGoN5sxAGoPdkEo4KD+XJAPPsfZs3a8?=
 =?us-ascii?Q?olWsijWqJqAii/pLJ6+xvkfiJ2XMQCk+omRAVE2YhlWnkiSbkcs/gV+yUUL8?=
 =?us-ascii?Q?ew8uqwl2zS2APcfFS3ty18JgrblzBV2tSCUKP5NAdMv184UviIgNZqyCJgf3?=
 =?us-ascii?Q?TGAzKTP8j2QzT6v8pieCj8WMjuQfvf8yyqG2tJEg+GyZrg8s/4/3pBA+/za9?=
 =?us-ascii?Q?RhWU/nlVCgEbCk5XKULfE5Ue?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 44a8f791-a8cd-41c7-8231-08d96d54533f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 14:25:19.1896
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hk+dvaco5348to9K5jyYM2xGZhSzqoa3IxKpAhk1eo3cvUCrYHtTDFJ4BUGgG8RvnTfVNejh+KGc1fLOl2l2kQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

On 01.09.2021 16:19, Jan Beulich wrote:
> On 01.09.2021 15:56, Roger Pau Monn=C3=A9 wrote:
>> Do you have iommu debug/verbose enabled to catch iommu faults?
>=20
> I'll try to remember to check that, but since Linux hasn't
> brought up APs yet I don't think there's any device activity
> just yet.

No IOMMU faults, as expected.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 14:34:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 14:34:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176509.321169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRJm-00059I-1z; Wed, 01 Sep 2021 14:34:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176509.321169; Wed, 01 Sep 2021 14:34:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRJl-00059B-Uy; Wed, 01 Sep 2021 14:34:21 +0000
Received: by outflank-mailman (input) for mailman id 176509;
 Wed, 01 Sep 2021 14:34:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLRJl-000591-6m; Wed, 01 Sep 2021 14:34:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLRJk-00041t-WA; Wed, 01 Sep 2021 14:34:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLRJk-0008Ry-PU; Wed, 01 Sep 2021 14:34:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLRJk-0002Fk-P1; Wed, 01 Sep 2021 14:34:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=EJjEFxAXfiM72/PLrvPX3UAZ6GtaE4crts9y9ZHbeqY=; b=cBtgGRiRAAfyYYjwqWHuYbqWk2
	Vcy7kNDiCCTLTDSMFwkAK4lY+uMUv7wvca1QS3tX6bXpRccAXe8llCMFHoIpnZfIsO/WrgwHsTAJA
	4TQ11We5TETV/VBzsIFHrphZKbrPf04Z2ATOwf3I883slf+jB0CXfI2jiy3Fnrih36zQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Subject: [ovmf bisection] complete build-amd64
Message-Id: <E1mLRJk-0002Fk-P1@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 14:34:20 +0000

branch xen-unstable
xenbranch xen-unstable
job build-amd64
testid xen-build

Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164710/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/ovmf/build-amd64.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/ovmf/build-amd64.xen-build --summary-out=tmp/164710.bisection-summary --basis-template=164674 --blessings=real,real-bisect,real-retry ovmf build-amd64 xen-build
Searching for failure / basis pass:
 164695 fail [host=himrod1] / 164674 [host=himrod2] 164630 [host=himrod2] 164556 ok.
Failure / basis pass flights: 164695 / 164556
(tree with no url: minios)
Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Basis pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Generating revisions with ./adhoc-revtuple-generator  https://github.com/tianocore/edk2.git#a82bad9730178a1e3a67c9bfc83412b87a8ad734-b170806518c1e414939c8b085866544814e1ce8e git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823-136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 git://xenbits.xen.org/osstest/seabios.git#54082c81d96028ba8c76fbe6784085cf1df76b\
 20-54082c81d96028ba8c76fbe6784085cf1df76b20 git://xenbits.xen.org/xen.git#a931e8e64af07bd333a31f3b71a3f8f3e7910857-a931e8e64af07bd333a31f3b71a3f8f3e7910857
Loaded 5001 nodes in revision graph
Searching for test results:
 164556 pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164630 [host=himrod2]
 164674 [host=himrod2]
 164686 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164694 pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164697 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164699 pass b04453d36bd87735aadd29adbefce8d147f18a35 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164700 pass cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164701 pass 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164695 fail b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164702 pass 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164703 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164705 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164706 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164707 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164708 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164710 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Searching for interesting versions
 Result found: flight 164556 (pass), for basis pass
 For basis failure, parent search stopping at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27d4058) HASH(0x561ed27e6cf0) HASH(0x561ed2848548) For basis failure, parent search stopping at 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d\
 96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27dc0a0) For basis failure, parent search stopping at 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27db5f8) For basis failure, parent search stopping at cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee\
 84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27d95f0) For basis failure, parent search stopping at b04453d36bd87735aadd29adbefce8d147f18a35 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27cf120) For basis failure, parent search stopping at a82bad973017\
 8a1e3a67c9bfc83412b87a8ad734 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561ed27cd298) HASH(0x561ed27c4440) Result found: flight 164686 (fail), for basis failure (at ancestor ~5000)
 Repro found: flight 164694 (pass), for basis pass
 Repro found: flight 164695 (fail), for basis failure
 0 revisions at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
No revisions left to test, checking graph state.
 Result found: flight 164703 (pass), for last pass
 Result found: flight 164705 (fail), for first failure
 Repro found: flight 164706 (pass), for last pass
 Repro found: flight 164707 (fail), for first failure
 Repro found: flight 164708 (pass), for last pass
 Repro found: flight 164710 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164710/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>

Revision graph left in /home/logs/results/bisect/ovmf/build-amd64.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
164710: tolerable ALL FAIL

flight 164710 ovmf real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/164710/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-amd64                   6 xen-build               fail baseline untested


jobs:
 build-amd64                                                  fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 15:06:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 15:06:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176520.321193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRob-0000cB-Qq; Wed, 01 Sep 2021 15:06:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176520.321193; Wed, 01 Sep 2021 15:06:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLRob-0000c4-No; Wed, 01 Sep 2021 15:06:13 +0000
Received: by outflank-mailman (input) for mailman id 176520;
 Wed, 01 Sep 2021 15:06:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLRoa-0000bt-9C
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 15:06:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 262e4026-ba4f-4d09-9b19-2a6765503b85;
 Wed, 01 Sep 2021 15:06:10 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-nk--lYA9MjSxirvyqLSAYA-1; Wed, 01 Sep 2021 17:06:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6864.eurprd04.prod.outlook.com (2603:10a6:803:138::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 1 Sep
 2021 15:06:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 15:06:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0021.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.23 via Frontend Transport; Wed, 1 Sep 2021 15:06:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 262e4026-ba4f-4d09-9b19-2a6765503b85
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630508769;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lIEEVx3pKZhCJQYNuhuiguKiqn5e+HYCDbQkL7NIP7I=;
	b=f2TUGBMhWp5o1bkG8YjVQQMGOLV4R5wK1HPb4h1/2FtWriMEVctaEuka12XaVthvAc+qfz
	4k4vVjbUaXh0Zv/UspIiAOwn5ZpWyGiHMK9zOCecQXh0AYIa4qdilQXr/grrJzkqOGwaIH
	Mihr6SOwi0PQeva8ktoECo2FT+T7siY=
X-MC-Unique: nk--lYA9MjSxirvyqLSAYA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lsVp0pnb5svi4vgRWnekEUj0FYNzoQMJwN75M/1z6MpcacJcx7kL4pRWezLC4CKAnWU9HnRhpNQWDTd3u86ubd6ZSf+I/n/GF6x6/Wdobg5ihJXdXyA4AumO2jINxDgYRAdr+ELRFxxLY734u8XLvsVTKjL8nIz5lFe+90TYUWaW6YdP4ZP05LNe7iikRMl33vLqH/Pv1/8S1zr0FhrkNGI/j5sWyqCW724a68IxuL5aY4GPppOGC7yxMp+DfRBlp3YAUDhCSwZHWMWQJ3r3FlzdUX6HtFe+8ICoHVAbiLlSL2AegBgxXHzsxFesoR0YIgRl42J99blIiwVWu5sneg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lIEEVx3pKZhCJQYNuhuiguKiqn5e+HYCDbQkL7NIP7I=;
 b=EOAXveL+MZ9jEWbSSGxZlMdQeEGg7xTsr/lkhokGk3jGmYC8srUbwLdRj5/m6vYPV5tXcgCyY3z4Y/xmSIlXrgb9V7wSaq9sml0uCXmLift2Z/CRuAWT/9yr3uDCBwPfHl6q8ePsiCDpSWGcUbgMziSm8UiG7WcCZPwPRotoI8hKVjtnUPnFIPZoOlsQxJwWIgIaVDrxsa/3/oMSwthT+uPfMyosgn4eKn7WqvLzQF0bFSHHKI7560CPnCVW9FFv0vm9CdpX3B3Dpy2W+7z8/9/brDT9+g9f2NnXSR90CNPUOzzcmOpMM6iAQgHOQSSoCCE35bw2EBD/NokJ+CaW/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
Message-ID: <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>
Date: Wed, 1 Sep 2021 17:06:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0021.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40e84511-4ea7-4a6d-76af-08d96d5a056f
X-MS-TrafficTypeDiagnostic: VI1PR04MB6864:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68648DC50D749EFCF6FC42EEB3CD9@VI1PR04MB6864.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	K5D4Y5B1MkK36YlI2zLhx1Pv7URAjtQSDEx+DoruwOBOdwRXnbIJ4UMDsrpn1eWvZXouSy4rRN7H9t5ATr58kGRNm/nivRrbUnThmavzvG0KiOu8X6pUe+xLIV0keyTwKI+IiIh0WPlP7bv8z+U9HDODl2KpPYy//f9WUFX8bcvVFXc+Ju8IiFkOVkKc9V1ttNv/730rY0fck0RvCiNMnLrf022278zXxMtnxs0/Vkd3vXiZcG9sdhm2M2Pg5pQUsdJTnGrrEQoSpyXN24L1CPzk7YJhoW8AKphJExyyk6n3EXhpc1302NcN5ZOdd9hzgUPyfJ21QG/+kN9DRFswo25rw5kXTmukyaUVEjSMkOs68kP//9MzGizzN66G9bOa3mDpZ+yCo3P701OiF2yFFG+kn5tfo8wkhHMMniCKwFsqJsnpDUxRdckANFaAHmKJVG119Igz7WV5MzXAw+pylOkssEJxBIAzSakB1cJ7g4+ecrfj9H1D2Fr5KyzPhz4p+ZyNeXcMGVoeENrXH+dOKXFqwDnSu9vXsYt9aq6KdLOsVb/zcJfKF2VLYeYx6bT0RicCCwq+aj3/7RGmmokNXt78V/dIW3VAasW+WACmfitME6lfHRGJPmwqxT3hm/5QlkJOnjNoJIyNrWsa8gFL/pr4ixvHCv0/Hsri+66KzCgqNCqZcc+tRHZv2XK9Z0sMiuvTXnYr3v+la6A+wZDHG+SraUsu4XUmGLGVVLttEs0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(53546011)(31686004)(16576012)(2616005)(956004)(86362001)(316002)(8936002)(186003)(31696002)(36756003)(38100700002)(8676002)(5660300002)(6486002)(2906002)(83380400001)(66946007)(66556008)(66476007)(6916009)(4326008)(26005)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NHhtNmRTTmtQcGhBMTZ3MTRndWQvcjBQQ3doWWhIWmZ2bzM4cUg2ODBMQ0t1?=
 =?utf-8?B?OXQzaEVvOG5uc0syMXNjY1Fzb2xxOHNpdml2cm1jdWw2UUNYcTdGcmZiWngr?=
 =?utf-8?B?UU9mckRNcnQwVHVRMFhHNHgxb2lCOVZKZlBYanRWL1FMVDlvUHhGZ0VzRFV5?=
 =?utf-8?B?T25jMjd4Rm5sdStSSGV4d2lSbEhMYjlRMlZoTTJKdFRmSzFPTnNoTWpIcEEx?=
 =?utf-8?B?bXI0RGZYYVU1YWF2eEhQN3dKUkQ2T2VPN0lUYkMxYWs5T2gvK1BrelNOM1dl?=
 =?utf-8?B?SU5lQmxIOHhjekhCZXdFL0lwby8wTDJhbVEwUDZIaDR2QWVnVkpVbGZvQkxF?=
 =?utf-8?B?LzNlN1VVNTJpK01WcUJIQ1dSWnVyVG9MNkdKYzhGUTdNKzVVN1dDV1lUaTh3?=
 =?utf-8?B?Rzl3My9Ycnp6RlRGcFZXazlxYzBMVEV4ei9MaG1BamIyUFB1WjM5a2JtSldE?=
 =?utf-8?B?ZlhrQVRBS013UG1WYTVEWUNnWjQ4VmNUSGFaa2xvN3Y1MGhySlNwUXZZaTJx?=
 =?utf-8?B?cVhBbEc1OG5tdzN5Mmptd00yN0ZBYzlCWU1ZM2JxcFRqcXRucWZXTno2VDMx?=
 =?utf-8?B?cllhZzcwVzcvZ3lhZUVQQTIzTm5UUktWMjIyd3pkMWQ2MjZpZ3pMejNLT3N6?=
 =?utf-8?B?YVRrb2VRcWZoSkZWcVBtbExhVVc0aE52WVN2ZEhTdk1qbDFkdXR2VUY1b1Rw?=
 =?utf-8?B?clpRY3pTUTZPVG5sWmNNdTY1TW5wTmFlNDJJWi8xQks3Q01Id29mTGh4ZHUy?=
 =?utf-8?B?K3ZFUkxuYnlSNzArQUdzazNVRStCWFRpZzdjbWJIcGxPTUk5QS9EdmJCeUJY?=
 =?utf-8?B?YndXTjAwS0FLQzMwOFNienpLV29hWThKSk8wNmlCM0pQUDNQS3RKbnVzV1U5?=
 =?utf-8?B?bXNRTm16T3hxKzU3OGV5ditHaE1BSDVUb251N0RYUHNSQ1dKSXUrU04zdVVE?=
 =?utf-8?B?MUMzalNUNVkvaDNQbHk3WDJ1Uk95UWhTSzVjcncvNUp4QVRTYzhzWmJ6QU9V?=
 =?utf-8?B?OE0ybnNrSGRKSE8yTzJ5Tm1YTHIyOVBIZFNhakNnMks2ZWpnV2pySDE1VVUx?=
 =?utf-8?B?WFlvd0VCZ0JxV3RUS2VrYk1yV0VUNDNObGFoRDlZV3I0L2dRTmRkQWdQSEp5?=
 =?utf-8?B?clo3TXZpbTdjc08vRmhvTzNMR0ZNUzlzUmFRY0FYUDFpeVp2TEFUc0ZIYys0?=
 =?utf-8?B?V0pFbGpQRnQwSVVkaEdTZkRhUERmaTZlR01waVVLYkRZN1lkM05RTGcwWjBB?=
 =?utf-8?B?U1ZBa1BYTDRvNG0wV2hHNkxZYXRoenpaQWE2QWR2RW1QSGpoNmptekJhbGNx?=
 =?utf-8?B?NnJueVdHam1lTW9TNnVYVFljT2tPTUJhemJKQWJuRFlTajVIekVyU1JnczFT?=
 =?utf-8?B?YmQxOWtWVnBSZlNiRGJSUGtGRjhUZnJBeHVKUmlVOTdiZk9QRlRtTVJNREly?=
 =?utf-8?B?TXdrQTlxNWxIc2pVaUo2ZXMxcG04NEthMlJMbmhBcThxZWREbDhTRDAza280?=
 =?utf-8?B?QzNUVzlZVVl6cUI3bGpLcEFZanhWZ1VILytYMDhlaXRtQjVYb2I3QzJFai94?=
 =?utf-8?B?K3ZtQk1zTVo1ZGxKb2E5RnQwWEtrWERrdmFZRmpxdXgrbWY5a1JUQy8rTmhu?=
 =?utf-8?B?NFR4OG1ucVJNNnRQUVVuRVd4eGNqck1GbmtVcXhPa0JiQmFVVWtBckE1bTM4?=
 =?utf-8?B?TmgxbHhxSE4xM3NaNXQ5UEd5ZDlqbVdaMS9zdjZQa3Q3bURTVUVEd1IvZi9Z?=
 =?utf-8?Q?czvYt1qGIbQtBQSiKbcoixduH+oFEPuT6XCuFTC?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40e84511-4ea7-4a6d-76af-08d96d5a056f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 15:06:05.7060
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FaEUpYWZx21VHKcv6/9vgIzW1oM5VHgGaLwYMyf0P17JPP5I9RIlq0rziLDz5sb1LRrAlVXl+Ea5/VO9RaeuAg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6864

On 30.08.2021 15:01, Jan Beulich wrote:
> The code building PVH Dom0 made use of sequences of P2M changes
> which are disallowed as of XSA-378. First of all population of the
> first Mb of memory needs to be redone. Then, largely as a
> workaround, checking introduced by XSA-378 needs to be slightly
> relaxed.
> 
> Note that with these adjustments I get Dom0 to start booting on my
> development system, but the Dom0 kernel then gets stuck. Since it
> was the first time for me to try PVH Dom0 in this context (see
> below for why I was hesitant), I cannot tell yet whether this is
> due further fallout from the XSA, or some further unrelated
> problem. Dom0's BSP is in VPF_blocked state while all APs are
> still in VPF_down. The '0' debug key, unhelpfully, doesn't produce
> any output, so it's non-trivial to check whether (like PV likes to
> do) Dom0 has panic()ed without leaving any (visible) output.

Having made '0' work at least partly, I can now see that Dom0's
vCPU0 enters its idle loop after having gone through all normal
initialization. Clearly certain things must not have worked as
intended (no APs booted, no drivers loaded afaict), but I'm
having a hard time seeing how to find out what that might be
when there's no output at all. PV Dom0 does not require any
special command line option to do output to both the VGA console
and through hvc_xen (making its output also go to the serial
log) - is this perhaps different for PVH? I couldn't find
anything under docs/ ...

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 15:24:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 15:24:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176530.321216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLS6H-0003M7-EH; Wed, 01 Sep 2021 15:24:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176530.321216; Wed, 01 Sep 2021 15:24:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLS6H-0003M0-BC; Wed, 01 Sep 2021 15:24:29 +0000
Received: by outflank-mailman (input) for mailman id 176530;
 Wed, 01 Sep 2021 15:24:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0rUu=NX=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLS6F-0003Lu-E9
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 15:24:27 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b053ab93-0b38-11ec-adfd-12813bfff9fa;
 Wed, 01 Sep 2021 15:24:26 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 998982258A;
 Wed,  1 Sep 2021 15:24:25 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 6C02B13AD8;
 Wed,  1 Sep 2021 15:24:25 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id icDXGCmbL2E9fgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 01 Sep 2021 15:24:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b053ab93-0b38-11ec-adfd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630509865; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=UDNb358/nQzDG8jtYlhrBrmnU7HIB/SQi6GDbLNHHRM=;
	b=N+CdnPNdkGF2+88rmtd5N9EvrgQi+3UcsW1BWTu7wpNt7Yi66kVZceRekPvc7iZnVC1GA0
	OVXfUNpIGYs599vTyrN3ucXATmUeBRIOLVjJCcZ5CXFXBxCiSi5RFxI9amFaOyY+3hQE3N
	R6tDOJ0C2lMQhPXm2YLrSdjj0Gff31A=
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <02ebe247-6891-dbdf-82f7-5b33b5a08dd5@suse.com>
Date: Wed, 1 Sep 2021 17:24:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Iqbl49u9DNzZ0n75ACpqzJ0tOW9ot3wUF"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Iqbl49u9DNzZ0n75ACpqzJ0tOW9ot3wUF
Content-Type: multipart/mixed; boundary="KfmP4bymJzv4KKJycYhWi0mz7a5vFdZ3t";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <02ebe247-6891-dbdf-82f7-5b33b5a08dd5@suse.com>
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>
In-Reply-To: <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>

--KfmP4bymJzv4KKJycYhWi0mz7a5vFdZ3t
Content-Type: multipart/mixed;
 boundary="------------A9357959A655AB520B15A2F3"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A9357959A655AB520B15A2F3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 01.09.21 17:06, Jan Beulich wrote:
> On 30.08.2021 15:01, Jan Beulich wrote:
>> The code building PVH Dom0 made use of sequences of P2M changes
>> which are disallowed as of XSA-378. First of all population of the
>> first Mb of memory needs to be redone. Then, largely as a
>> workaround, checking introduced by XSA-378 needs to be slightly
>> relaxed.
>>
>> Note that with these adjustments I get Dom0 to start booting on my
>> development system, but the Dom0 kernel then gets stuck. Since it
>> was the first time for me to try PVH Dom0 in this context (see
>> below for why I was hesitant), I cannot tell yet whether this is
>> due further fallout from the XSA, or some further unrelated
>> problem. Dom0's BSP is in VPF_blocked state while all APs are
>> still in VPF_down. The '0' debug key, unhelpfully, doesn't produce
>> any output, so it's non-trivial to check whether (like PV likes to
>> do) Dom0 has panic()ed without leaving any (visible) output.
>=20
> Having made '0' work at least partly, I can now see that Dom0's
> vCPU0 enters its idle loop after having gone through all normal
> initialization. Clearly certain things must not have worked as
> intended (no APs booted, no drivers loaded afaict), but I'm
> having a hard time seeing how to find out what that might be
> when there's no output at all. PV Dom0 does not require any
> special command line option to do output to both the VGA console
> and through hvc_xen (making its output also go to the serial
> log) - is this perhaps different for PVH? I couldn't find
> anything under docs/ ...

Did you add earlyprintk=3Dxen to the dom0 boot parameters?


Juergen

--------------A9357959A655AB520B15A2F3
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A9357959A655AB520B15A2F3--

--KfmP4bymJzv4KKJycYhWi0mz7a5vFdZ3t--

--Iqbl49u9DNzZ0n75ACpqzJ0tOW9ot3wUF
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEvmygFAwAAAAAACgkQsN6d1ii/Ey+O
eQf+ImG1ACU1xjQmOG9RvSEfIFi9B0Dbp26QVsEYmvhmAKTV9xgLXHzrAj/vno+jidq92Lujyr7x
wOXZRL1mxIuNnlK+ESrf54eAAG4r9BP8/w8DB+JDcbBNsHAsvzl5Xz2Pc5CImjCWoccoWShEAykD
nqjpwYNraR75Z1m6RRbZ1F8G11UR7lunu2RCCZZPfmDCfwtFAWvMkiU2ijrk1vEYa6tyGQ8UWxFu
x6KU0F032xtJeGfNbWh5M8YkYjGxn+RaP4k2kHags0KS7oH9T0vQBBNelRzzEgC69JkEpq4LsHUm
Lw4wgTZ0Ig288JVWDHk2fY3Rz6Jy9f0pFX1GuGWU2w==
=2MfS
-----END PGP SIGNATURE-----

--Iqbl49u9DNzZ0n75ACpqzJ0tOW9ot3wUF--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 15:52:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 15:52:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176537.321227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSWx-00075i-Md; Wed, 01 Sep 2021 15:52:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176537.321227; Wed, 01 Sep 2021 15:52:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSWx-00075b-JX; Wed, 01 Sep 2021 15:52:03 +0000
Received: by outflank-mailman (input) for mailman id 176537;
 Wed, 01 Sep 2021 15:52:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLSWx-00075V-4K
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 15:52:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1642fdd1-e56f-4bcf-ae0a-0e39a73f0556;
 Wed, 01 Sep 2021 15:52:01 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2058.outbound.protection.outlook.com [104.47.1.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-5N_FybicO5GK4fwW33LkwA-2; Wed, 01 Sep 2021 17:51:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2960.eurprd04.prod.outlook.com (2603:10a6:802:9::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep
 2021 15:51:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 15:51:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0017.eurprd09.prod.outlook.com (2603:10a6:102:b7::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 1 Sep 2021 15:51:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1642fdd1-e56f-4bcf-ae0a-0e39a73f0556
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630511520;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=yDUxKs17CBRyEk1FI7eJvE3umkbbk3O/fEeF6QKW0cg=;
	b=EHjzOGsFJpi3XeMDuHI20cTqO3YUZtuBXbcMePnicEdbL3LV3F3TnG8FKblI6V9GU25sVQ
	dMhAn1ufylATYLbhnaAFNuuSt6+jjppRBJMU7gsH3ON01J/WPHdjF77S2+1LW3PcW5S9+j
	E3HPHQyKLpsX+3ybeb8dGiTaOhCRC9E=
X-MC-Unique: 5N_FybicO5GK4fwW33LkwA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gk8N2VnkqjKffSgjxCLbiPTQs40owmPlLJsXp4oEjqm6t6KOtI0hNgdNerZV25Y0hzGetzG9WmqNhtzg7zzGJPMSjJkKc0TLKCGaTK6EY+zuTbA4/s5H3kwpLDkKwA9XvrJe3+37UGszrkDMZ5SezPaT6OukqhKTr3PvToA/E9HXE13iXeqVdnejRGQY0007jKjKX7FqW8SyMOaXcO7Al4oMF5q9P94bgvdsJRQS8HMs1fgjsjv86oS1zOYiNfKqiOdqKiwUETVIfpO8ECLoPGqxgGOB5py6xWtnrDTAAwPSy6GhT5l3GwVXgQd1N6jdIWo51iiW3NMb6CuFixCFhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yDUxKs17CBRyEk1FI7eJvE3umkbbk3O/fEeF6QKW0cg=;
 b=H0gxnNBCyS13nzEsQDbay45/XVyIxd2pcUBQF2pl6UzgbIYFE9mOD2RyISD6H69siqDZ6Am0957f8AkdTpUV757DCOnb7jyad4I4nYs9Ae+0KcX97DhG9re6a1ijJ+5aAFvAqWyjIzXuCp1j14vRbXfUV7/dze1JQ4YToziES1bri6cGizGQJsP5a1ftSlNuHaQC+rZ37h9J92VgKVwg1rd081zOLtiQGwZBdc30a6K3rc7z9+PfW0uBgNYwSGiZ4sB9VTHtqJIgoMCRZqYQVyS/IOxpLHO8BgojhThsfUX9H+oHdFMJYXvO0CwFMWzdQArj5al8gTG2nUYU1fX52Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <1d600035-851a-190a-f744-8cd7b2d887fd@suse.com>
 <02ebe247-6891-dbdf-82f7-5b33b5a08dd5@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <32638483-c44f-d2f7-a741-c51dd98461c8@suse.com>
Date: Wed, 1 Sep 2021 17:51:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <02ebe247-6891-dbdf-82f7-5b33b5a08dd5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0017.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d62c22a7-a491-42dd-81a3-08d96d606d25
X-MS-TrafficTypeDiagnostic: VI1PR04MB2960:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2960B098BB72B3D28C741454B3CD9@VI1PR04MB2960.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z/BhyqfxnNuO/PKuyx0HTtvAAWSdjpB0asnAgOtB1GaqSMT2+ryrZIjBguJtFCzZgiHCdvzJKbLY2VhezkLHn2y2bfIY9CACJ4vVv8oPYc5B1LA1epJ566z15ul8OAIfBniZe6iaoYfN9KMQHY5vKmzfo/X5npTpFzGFWpHRtfVMkuoQApIxPcKjMny3M01DYmHYodw+BFl44drIPvbMu5Dyo9Xo1MnYAFMtqSabFVYvceKoJ3jIW+tOKSbO0KiLxNPTYWljs29wmqH/1OTFDpSbX4IFieeYgPePRXj7I4JM5BcVBA9vh9ui6PsFdvtfcrXLRr8LOfWq91HhctF87VpJlXjBcXu3yo2Cy+oaTPOxCAYjNdwZ4M90Zkfj7pFdwULGPXOVz2uUg4ZXijVIHLSGQcswMlw3fjxF61vIo4y9PpstdgGNUDPwQS6rnf4Q75hYVY8yeRbSZUbGxA/Fvim3vTIlwgJMwamU4kbUX4a1z5ktXo3zjvPckJnQzuFWgp+4TGWHUw/gPF5y8q/xDVdbXmeTEi+qe4KJSBG/5OULCVAlPFhTi+8SjrmR6KAVo1Rdk4pkK4e9clqDste80cnVyHyIAb8B5wSLCzuLvXRIgbZEV1NTWL9Tslnfkal5E575lKF/k9sr/UDlqdn7IKCxZMm4QkSKDGsFRqZPAJyBuuGYhEBGbLJDN5qh/JOCgDAvnPcBUwigUSaYILG7wZE1IYEHaMTf0VQQFem4Lvk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(366004)(346002)(376002)(39860400002)(396003)(8936002)(5660300002)(53546011)(66946007)(36756003)(37006003)(83380400001)(66476007)(66556008)(6486002)(31686004)(6862004)(478600001)(956004)(86362001)(31696002)(8676002)(6636002)(38100700002)(2616005)(16576012)(4326008)(2906002)(316002)(26005)(186003)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TmFzVHVzRFdDY1UvRTJkSysvdlRMY21HTEE1Q1RJcnBPUExva2E4MzQ2L0JW?=
 =?utf-8?B?N2VTTVNrb2Z2bGQ1SFJqbFFIaGQzWm5OVWR2dXFOTWdFck0vWFZGK1FHeGl2?=
 =?utf-8?B?Q1dKaTE2Ti9meWJZaDVZM2Nvdjk1Mi93cDNNaXM1UHJYTGEwK2dka01Yd0VS?=
 =?utf-8?B?VlVnK0tuRE1UbHEzSS9VZVhyZHdsN25oNlMwaTNqbll6VnZFNmkxK2ZOeGJW?=
 =?utf-8?B?cnVXTXcycjViWjI0aEN5aGFDaStHM1pvc0FuRlNyZmNQSlRvK1dxcENLWHhB?=
 =?utf-8?B?eEEwNjBOS0c1K2puMHg5cThCcnhIRWU3dXd6NHJNcUxWT0N5a0RxbXB5TDFS?=
 =?utf-8?B?dXZsS2VqL2RzSVYyUGlaRDFjL0hURHhITmVQQmZ4alhaOXllbFNhWW9kQzI3?=
 =?utf-8?B?S3pxMkJCSEZyM0RVRzRsNW1PdDJuek9zcjR3aDFrMTkrZ3E3aURlSXo5bWJU?=
 =?utf-8?B?cXpmdXFlajhydGRqazgxa2ZzZ0FVT3hsRm5iZXdWalAvNVBVbS9xaUhPQjk0?=
 =?utf-8?B?OFI2NzZ6MS9VdDFyZ092bUUzc1lyUVhUSitrWGJiRGVjZVF6RnlEZnV5dWlS?=
 =?utf-8?B?TTRBNkN6WU96ek1iYVZQbnoyanNncVFDU3FWcWowTXA1NUdJcDRSdXRGZXBR?=
 =?utf-8?B?WEM5dWxUVk9rVjF4cTZjTWNHTGxteDFRdEhaNXEzM1hjd0dFT0VIQ1BLQTJV?=
 =?utf-8?B?SGI4ZEhESHA4ZEI5d2g4VkVEdVVyTmZHRGpnYTZMeEhKL3NpZnYxaUtBN0Iy?=
 =?utf-8?B?RlZzeW9Rc0dnZ1U5OHNYVnFiTng3V3pmVTRCTUx0eVh2TGplMGFaRVBmRG5I?=
 =?utf-8?B?ZjhSdzlzenBLWHhuNFBWWVRYMm1YWmpPakFrS3FrYjloa0ppczBqQVZvR0d6?=
 =?utf-8?B?bHkyRjZTeUxSVXR4eiswQWRlbi8wRlE0OEhPSEN1RHNqblR0THJZcUVmQ2Fp?=
 =?utf-8?B?Yk1aNUFLYnh6K2F1SEdpd3hNYkQ3dm9MYVFqcmJ1TXY3cWIxMWErUldKeGt0?=
 =?utf-8?B?cENkaXFUMkU0RTR3MkpSdndBY3VyYnJaZ2g2U2l4YWpWTUdaNWE0NmNLaTJO?=
 =?utf-8?B?djZtQTZsQXozTDBxM1ZZd0lPWEtvcjVsOEhlL1V3Um5Gb0FZT2FXNnJZNDBr?=
 =?utf-8?B?RkprK2VkbUN0TERCdHpWaG5GZ0VkUVZwdzYwSjZVckJlUGQ4TEtuQ1JxQVFz?=
 =?utf-8?B?NURtSlZZRjVQOHdqUWxRajdNckk5dHRoNkExVnRwSUljanBTV0wzTzltSExQ?=
 =?utf-8?B?R2xSdUQrYVRKaG1jYnhVSTFzckhVaVJKTmMySFBJYlFLT2pLMXVhK1FEUFJB?=
 =?utf-8?B?b1FiemhIOWhlSWFNSDRwN2YzM1YvUmpaTlY2ZDQ5UHE4VnpSM0N4YlNJUFl0?=
 =?utf-8?B?emZZK1psbFRXdFI1QW5BQ2hsSEJGMUh4RjlIYzlqcFJ1MjBGT0NkYkczL0x4?=
 =?utf-8?B?N21PMit2WWZ3TkRFQXdqVEk1YllBOHhyOTNuRW4weTNucm1xYy9ScXJLSHJN?=
 =?utf-8?B?TWE1b2pjKzRMbTJ5aUNReFdzeERFSDNkU1Z1Y216T2w1bnE4cTlUUFBDdkRk?=
 =?utf-8?B?VG9rcDBWWlZyblN4ZE9uUUhOWGFhRTYxL0hPRkpOQjNYVE1HQUFGczVTMTc3?=
 =?utf-8?B?Z0hSQ20rdytiaXlwWVpjOWtMbTQ4YTlFZlNEaEVacG5uRENBN0IrTTZveWZt?=
 =?utf-8?B?dllyamJ3NjNBV1VNR3BEYnQ2enRRWHk2SFNXYmNpM2tnR2pSdGFZcFRuMndY?=
 =?utf-8?Q?PZ6xtpSK0V/fP1C1DZowbOeOkCvq0bwADw5B4YK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d62c22a7-a491-42dd-81a3-08d96d606d25
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 15:51:56.6477
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nVOOVHIT9fBOhPnyV7BE3owEi0dcsSlOXtaNutaeWvOCtTzuEXd4z/SYd+eKYY9JylT/DXQnACs2BT0pqsHJ/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2960

On 01.09.2021 17:24, Juergen Gross wrote:
> On 01.09.21 17:06, Jan Beulich wrote:
>> On 30.08.2021 15:01, Jan Beulich wrote:
>>> The code building PVH Dom0 made use of sequences of P2M changes
>>> which are disallowed as of XSA-378. First of all population of the
>>> first Mb of memory needs to be redone. Then, largely as a
>>> workaround, checking introduced by XSA-378 needs to be slightly
>>> relaxed.
>>>
>>> Note that with these adjustments I get Dom0 to start booting on my
>>> development system, but the Dom0 kernel then gets stuck. Since it
>>> was the first time for me to try PVH Dom0 in this context (see
>>> below for why I was hesitant), I cannot tell yet whether this is
>>> due further fallout from the XSA, or some further unrelated
>>> problem. Dom0's BSP is in VPF_blocked state while all APs are
>>> still in VPF_down. The '0' debug key, unhelpfully, doesn't produce
>>> any output, so it's non-trivial to check whether (like PV likes to
>>> do) Dom0 has panic()ed without leaving any (visible) output.
>>
>> Having made '0' work at least partly, I can now see that Dom0's
>> vCPU0 enters its idle loop after having gone through all normal
>> initialization. Clearly certain things must not have worked as
>> intended (no APs booted, no drivers loaded afaict), but I'm
>> having a hard time seeing how to find out what that might be
>> when there's no output at all. PV Dom0 does not require any
>> special command line option to do output to both the VGA console
>> and through hvc_xen (making its output also go to the serial
>> log) - is this perhaps different for PVH? I couldn't find
>> anything under docs/ ...
> 
> Did you add earlyprintk=xen to the dom0 boot parameters?

Yes (I did mention this before) - no difference at all. I guess I'll
try again, just in case I made a stupid mistake.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 16:07:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 16:07:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176546.321244 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSlD-00010l-4D; Wed, 01 Sep 2021 16:06:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176546.321244; Wed, 01 Sep 2021 16:06:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSlD-00010e-1C; Wed, 01 Sep 2021 16:06:47 +0000
Received: by outflank-mailman (input) for mailman id 176546;
 Wed, 01 Sep 2021 16:06:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=18Fz=NX=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLSlC-00010Y-4n
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 16:06:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 99e5fc56-0b3e-11ec-ae00-12813bfff9fa;
 Wed, 01 Sep 2021 16:06:44 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-sbgjM-IgPLW50Sk0nrDtbw-1; Wed, 01 Sep 2021 18:06:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6174.eurprd04.prod.outlook.com (2603:10a6:803:fa::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.20; Wed, 1 Sep
 2021 16:06:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.019; Wed, 1 Sep 2021
 16:06:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR05CA0079.eurprd05.prod.outlook.com (2603:10a6:208:136::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Wed, 1 Sep 2021 16:06:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99e5fc56-0b3e-11ec-ae00-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630512403;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=tt+7kx5BdjY1U94SmVlxWaRlQbb57pNXoUVSt8g/sWM=;
	b=NX9025zHV5SrMf2eaV8FstQb0O2zfrMmlcJbljXRo1RVbYTQkpI8t5wfrQJkel4Sp/JSea
	wz2pzQ9vhNC8SjUuZ8fKAC8NL70edTopac/X6ck/dLva6VsuIVhVhHDiMpyhNzTobL5YdZ
	7Lc0FSgELa1yAYULrz+DV+n8qPOm1fg=
X-MC-Unique: sbgjM-IgPLW50Sk0nrDtbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CSNlk/SaIxvLDw3VzVNotapcqgLkSLmQup7XbyEuz45fLQe3O1kccLOccSDxcjYhXHh/+t6ZeXayoVJZ7m1Nje8nBTC1Mry4qvIOmJNNAskafJQ6BMphCZ1H+5Amrc4zSdAjtSO7wGs14v6bgjRQUWo0HiHsZSLtHPmmvQOtaczMkoGuWXkWtL8OakZqUn1swTpwAmCsdPFX1HFPCIdhwJKyq5LqM/SGVTrb2y7U8J3S48WUjo78PkwQ/OjGYf1vDr0hQ4/WsG0BD3l+FvfyCapBjzXmzWDESGv/dZhQBLY7pNxSs1xx+m4JAnFbQ9odSqA2xnqeqVNwSl1UZKf/RQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tt+7kx5BdjY1U94SmVlxWaRlQbb57pNXoUVSt8g/sWM=;
 b=mGU749/3xxMTAqGM5Pzu5WxGOfQ2gqs/IkRoRU6RHWtcYDpw9slq4ex6Sz/ef145SAZxPZSjfV1ZozUyYI04EEgvXmWbbmdsx2MDWpFjRPcLfAVbVoiHel4O+wRFtixUWx0l6TqmadkLcXDOV8iyF9zSDAD6O6Fx7orFcjQeQtktbSL3TcEDPiFYB7xGf3nF6FVbZWSam6xE4Y6luUwA4RI/NU3ZsUdu/m497nSDMVT/UBfH1O9z8yhRZgU6uPkMN927MDy+ybwlPdXbSoAIyhK5oSUoLQJ61yhNVgp1L/BGult73hrbP0usjvAJj1efveSC9A07craV7B4R6j7zqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
Message-ID: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
Date: Wed, 1 Sep 2021 18:06:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR05CA0079.eurprd05.prod.outlook.com
 (2603:10a6:208:136::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2ce9d0ae-d029-4ac7-589d-08d96d627b84
X-MS-TrafficTypeDiagnostic: VI1PR04MB6174:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB61749CD031FBA4EC131502D0B3CD9@VI1PR04MB6174.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vpjI26fkDv6QD2Kl1Klj+xnjoRqyHuKDNzgP1lJuQyBTZlL5FrVKhBgLCdz7ry+dqczTQx74iict04exc61SA3SunMzd3hmxFPvhmy//XZkrMwzZ+7NflA3gHSP8H5qr/fs5s9adxyVS72wPwOqIGxQHrb9skSB+ApDi7RBAkaPcQISTNRVGfJqWizHeqDY0gHkrsfbNMIqD7cZFo3FJ88vF/8oOHTLb+xh/U9K5H7KgfeRVenWIJre+7YYCQPFwknsDIo9kyluOarAcMJC4Bx6RTc+UIT+E8uNseFc795pXuQIJvf+eOY3ecCnfA+w2y9PoOuExo593kxHB6IIlNpeohsJO+ogNtLeFni7w8vZZJHthF0t2HHabZaJC5sDMVwYNHXF9lHnv+ESp5z8y5MKawPU31u1pTXQ4j5LN+0/xm+QPF3/qez3j5mad7efsUvfOtUB0R6bVhHy+5a6oV08wQUHDOrB/GSJ5ucFtX+UX1NU+bffLtOvvxX96HiVRjmm63V7ZqzEFdptjAvWqfFY03HpT/DD58w6lwwznYwKzrnPMl92vir+CCO7jBdtBNYviR1IE1+42SDfk5mQKs3AkhEziK2tqIrz7wSEOlLCnVQMTsi6hy4lAhpIaTOmgQs+W8+peIzXj+e/N8fgrQFHmepqUw1dr6n43hCJJopnJQtVBrD7bkkCt0Py5Ah5nQzGyXcBiymuxqeIV9Z4BYpNeUUw+kQzoIEUmNofDA/s=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(396003)(366004)(39860400002)(346002)(8936002)(5660300002)(2906002)(31696002)(478600001)(6916009)(16576012)(6486002)(2616005)(83380400001)(38100700002)(316002)(54906003)(66556008)(186003)(86362001)(8676002)(26005)(31686004)(66476007)(4326008)(956004)(36756003)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cFhYTWtwZnFkTEMvdlQxSEM3T3dYWXZ0cTFGczlRSzFNY0ZyUE05TElGeDdN?=
 =?utf-8?B?dXUwWUhVbUI4cEFFemVVaFdtTHBEYi9TTkZGQUNXbzFTOE9IRmhjR2orYm9r?=
 =?utf-8?B?bFVSM1JYL3FCOEU1ek5hdDlKUFdQU0tMalAwdDBhNUZnbVhURkNxNWlnZ1NO?=
 =?utf-8?B?YThuN0dWdC8ydk9JSTIvaysva0dtTjA4aTMwcE9Zbm1zMXl3VCtZY2o2UVpN?=
 =?utf-8?B?T25pUzYxUG5NNFlBWUVGMW9aVW5xNjQ2Y2ZMV3FtRVpYOUgyMXFVbnNFUzdp?=
 =?utf-8?B?TUViREV6SjBXUmJTZUxvK0dUU1B0T2pXak1JZGhueUR1bzhuWjlUMDBJT3BY?=
 =?utf-8?B?cGFQYUhORXFRTllHdjRRSkNMRW1BWUg4RE51d3dycWRhT3B6OXA3TDBtcFhF?=
 =?utf-8?B?N0Njd1h2UGsrR2o3ditkZSsra09TRXZ4ekZOeVk1WWREOEZuM1crS1NFTkQ0?=
 =?utf-8?B?WCs5YkpQV2FmWTBqOTc4RzVJc0F5dnlEMHVCbzRrNFBkVk4zTE1VZzYwc3pt?=
 =?utf-8?B?NzRydGdLbHFqS1JwaExTSGIwMFJ3dEFOSk10S1hlQ3pFK1pZdzdWKzRBYkxn?=
 =?utf-8?B?MWp3VVdlc25weGFiam1hU3I1Yy9lRlV4aXNxRDdDbWNMaGJMRVdFRmNwMVJ1?=
 =?utf-8?B?SFc2TTBTWTcxMXNFQk9JSXVJMFJxVnVRVmtNVlVQRE1oZS9yRExyRkp2cUo1?=
 =?utf-8?B?bm9abS9teWdPc1IvNDRVbzFIVHpYTXl6SDg3Y3JHcGZ0VmZxV0l0UnNKMmdU?=
 =?utf-8?B?MEU3c2JJNnhEaFhVYjJsUUlhckNPUWJGVXR4OFpuY3FZN3dOc2ZiMGJwWFJl?=
 =?utf-8?B?OWpLZGs0KzdkbnkxS1BHUFl4QjVtNXVBekdPaXVvblhlbzZUaER0U2phUG9l?=
 =?utf-8?B?NlBpY0szeS9qQWROUUkzd3lxb3VwNzFtSi9CakxSU25EaWo5WE5nZVc4K0l0?=
 =?utf-8?B?T0F0SWN1WkUxWnhtM3FJVzNBTFY3RjBxSjMzQjBMaHk0TUYySk1uZDU2YVZq?=
 =?utf-8?B?WFJxdWx4a3NIVmxlYWx5R09xVG5qMXJrSnZYd1luK0ZTM3hzTzVGSjJnMitm?=
 =?utf-8?B?bGkvY0VON0U3b3JlK2lKTFlHdU1IbzVmTGFVUk4yemxVOXFSRW9FYzAxR25h?=
 =?utf-8?B?MFZ1b1huSlNFWlY2WmZrWUxvYVJJZFZvWWhBaDZGajNobGUxemtOZlBEZVJV?=
 =?utf-8?B?cU5aVXdKR2JyMUprRkw3bk5iYWhTY2JiMElYaUVaNlUrVFcrUkFpOUF6dDlR?=
 =?utf-8?B?TVVJT3RYbEM2d28zOWs4c0IyTC9lZEZxbVBaV1lieGw1UUFCc2p6S3ZuWkla?=
 =?utf-8?B?MS9EVlhDUFk5eDA2eTExTXk5N1c5dDZoWjUxcnZSYjZoNFVEN0tKbWR3UHBM?=
 =?utf-8?B?SlhBOEtJYkh3YUdLZVJFb0sxWkpUQi8xQ1U4MFBQdGY5dFhjbzR0YWlrOTlZ?=
 =?utf-8?B?L3NHSEo1c21zVFVpNE43MGpLYndxcXFEU0JCS2srN2F4bEkrTXVyV2JmcHNl?=
 =?utf-8?B?M1F6ZXorcnUvYXpUTWwxUlcvWVZ0MUZFRUhsZWhJaDNOeHRYeWlpUlgwU1hC?=
 =?utf-8?B?cEhnNnNGNG05VHFLWnM0WFEzVVY2dTBicVNKWU4zTm1mSWZZOXl5bDJ0MG1t?=
 =?utf-8?B?QVUyVjNXb2VzN0NXVkVGWmRaM3o1b0p6Z0xXMkJtZGtBSk1mVVdzd1YrRTMv?=
 =?utf-8?B?dFB3SjQ0SHQxd3Z2SktlY3JFMW1teC9oQ1dFM3UyM21BckRlTGx2WC8zV1l5?=
 =?utf-8?Q?hcqgHkN0xv2Xq30twLcvvesezGzC1nC8Xd1eMOo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ce9d0ae-d029-4ac7-589d-08d96d627b84
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 16:06:39.7258
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vlqGfXZuf3At/oNe3DAFp+/0CHHXS5N1ZEeQrV17B/bJGboOAtFgJZ+AQLTzlo+uKa9yoLrrIprhP/NkkalD2A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6174

The function may fail; it is not correct to indicate "success" in this
case up the call stack. Mark the function must-check to prove all
cases have been caught (and no new ones will get introduced).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
In the grant-transfer case it is not really clear to me whether we can
stick to setting GTF_transfer_completed in the error case. Since a guest
may spin-wait for the flag to become set, simply not setting the flag is
not an option either. I was wondering whether we may want to slightly
alter (extend) the ABI and allow for a GTF_transfer_committed ->
GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
at the same time as setting GTF_transfer_completed).

--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -2394,7 +2394,7 @@ gnttab_transfer(
         {
             grant_entry_v1_t *sha = &shared_entry_v1(e->grant_table, gop.ref);
 
-            guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
+            rc = guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
             if ( !paging_mode_translate(e) )
                 sha->frame = mfn_x(mfn);
         }
@@ -2402,7 +2402,7 @@ gnttab_transfer(
         {
             grant_entry_v2_t *sha = &shared_entry_v2(e->grant_table, gop.ref);
 
-            guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
+            rc = guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
             if ( !paging_mode_translate(e) )
                 sha->full_page.frame = mfn_x(mfn);
         }
@@ -2415,7 +2415,7 @@ gnttab_transfer(
 
         rcu_unlock_domain(e);
 
-        gop.status = GNTST_okay;
+        gop.status = rc ? GNTST_general_error : GNTST_okay;
 
     copyback:
         if ( unlikely(__copy_field_to_guest(uop, &gop, status)) )
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -268,7 +268,8 @@ static void populate_physmap(struct memo
                 mfn = page_to_mfn(page);
             }
 
-            guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order);
+            if ( guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order) )
+                goto out;
 
             if ( !paging_mode_translate(d) &&
                  /* Inform the domain of the new page's machine address. */
@@ -765,8 +766,8 @@ static long memory_exchange(XEN_GUEST_HA
             }
 
             mfn = page_to_mfn(page);
-            guest_physmap_add_page(d, _gfn(gpfn), mfn,
-                                   exch.out.extent_order);
+            rc = guest_physmap_add_page(d, _gfn(gpfn), mfn,
+                                        exch.out.extent_order) ?: rc;
 
             if ( !paging_mode_translate(d) &&
                  __copy_mfn_to_guest_offset(exch.out.extent_start,
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -307,10 +307,9 @@ int guest_physmap_add_entry(struct domai
                             p2m_type_t t);
 
 /* Untyped version for RAM only, for compatibility */
-static inline int guest_physmap_add_page(struct domain *d,
-                                         gfn_t gfn,
-                                         mfn_t mfn,
-                                         unsigned int page_order)
+static inline int __must_check
+guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
+                       unsigned int page_order)
 {
     return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
 }
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -583,8 +583,8 @@ int guest_physmap_add_entry(struct domain *d, gfn_t gfn,
                             p2m_type_t t);
 
 /* Untyped version for RAM only, for compatibility and PV. */
-int guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
-                           unsigned int page_order);
+int __must_check guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
+                                        unsigned int page_order);
 
 /* Set a p2m range as populate-on-demand */
 int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 16:13:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 16:13:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176552.321255 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSrT-0002di-Rg; Wed, 01 Sep 2021 16:13:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176552.321255; Wed, 01 Sep 2021 16:13:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSrT-0002db-OI; Wed, 01 Sep 2021 16:13:15 +0000
Received: by outflank-mailman (input) for mailman id 176552;
 Wed, 01 Sep 2021 16:13:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2BIO=NX=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mLSrR-0002dV-Kx
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 16:13:13 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81102a66-0b3f-11ec-ae01-12813bfff9fa;
 Wed, 01 Sep 2021 16:13:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81102a66-0b3f-11ec-ae01-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630512792;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=HvvlN/n0WnyfSJuxCS3vjmgoOHALxUs8+4ZZAxp0Ed8=;
  b=WaHU6LM8OQbM/MGhAUFoBhQITBqYf52tPMckkuU4tyZ4HRIQ666Ijgek
   ezSOEHLUlafatuLcSFq+3gZJ0PqxiAJt3L9naHNq+/US2s7S9uD81JPwZ
   eDKg76jgpHH7DcrTkItFPzM6f+NdejKWXpzJuRpbibu4Ls1pgEpqMwi7L
   c=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: gjr09FkSY08HDDV29MdiUKT/rneNt0X7U9fFtSFhq1QEe++zSzlICW/mIUFtakzC7vxvoe8ajh
 67T8B9xtSr+LGaPLqlRXyJlcgGVZ6Ciumx1fdvbnPBgtjZ7lFqnbu47pG5lEycjTllAHZx94bH
 Vxl3NGZxwOqSMqn+SmMRZCX0Q+4m2UhthqodgOkRK7LZ/nrOuCPntC6kRD5A1VfOuMzN5G6pN0
 kn6ycrgTSiNUDG6eMtSf9f1iPRMPINEAMRxf0xkQB2wiELjJyTPHz31J4AAwVFL9QNlRPwVYpO
 SXgXxn/1eMiz8KjRoEN9S0H+
X-SBRS: 5.1
X-MesageID: 52206275
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:9huq46hoVw77GhpT+UxzAz1XoXBQXhEji2hC6mlwRA09TyXBrb
 HXoBwavSWVtN9jYgBapTngAtjwfZqyz+8Q3WB8B9mftUzdyRKVxeJZnPvfKl/bakrDH4dmvM
 8LHsQQNDSzNykdsS+Q2njaLz9U+qjjzEnev5a9854Cd2BXgqdbhDuRQTzraHGeDDM2eKYRBd
 6Y4MxfzgDQB0j+eKyAdwU4tsb41qL2qK4=
X-IronPort-AV: E=Sophos;i="5.84,369,1620705600"; 
   d="scan'208";a="52206275"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PEd3Rn+azAXta2wzmUGSDW4o+AADo4swardXmAKHmIuiy4joq8yOmiViEfUVhIq19MO2NZ0tuXrCpO0ni5/awd5OPmp4qYplWiYL26Ui5FC1/i9M4qoShUk/mO74d/Yo2nNP+2L7sL2azabKrkReNKGqE3sfDtsOfSSR/bbG17G/vhWwEdMW9w+GW7sakfCpn/ksV0TNVTlo/56Z9xUn7k/i8duOMf9gkc/yF/uqPFo5yo96mmpOHpU5CXhuRlQOlkMHfggdAGGQ+rkigAHtEegKufFEJenWiBgOdDiZRmXg0rSsBz8mGasr2bhqwIeMb6qZtlbl4shrZzSvexSDdA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=w3gAsDWXO8sHjqoatlOYh71lPbgIcsOnbCK8nxc/AU8=;
 b=IgFaIPFwhhJiYJS0zw7CxwqvXc6aEedghCJz0kJUprOhCoDrpxnSYaGO+NFccbBMY10r7Tc6eiTI3jYLpKiP09mK09t/OveUEMsH3TihLnSc5MhITb9P2kU8DGd0a3WOQNL8cCHSkvaEAtVYVIXApVVzm66EEveMc8AO5/KIcsqAd7fHKfLtzRl+hGqSwP0LHeyeuP0ZiKpyyxHiHQ4FHlc8veRwmT+rspemna9LrmlyhhqdaNCGp3BUcPcU5TbRZ/9naGcYJi/zAAq53O7zroQZ8ualvTOsb9t+5vX+ey+y38sN+9AKi9BKTYn9K00OxSQ4eBWhQm4eZz4Dc118/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=w3gAsDWXO8sHjqoatlOYh71lPbgIcsOnbCK8nxc/AU8=;
 b=owP3rKZPw4jaVsu+JwbkGzp1YieXzlpvuMW1fct51/emHFMHuBr+XqrFxjleRLHZo04gEy0ckV/SlNUb3dEIb8jk+B5Rp2Za2CyQsRZvsG0ZYPRfNhEH3cH8TzxkN9ji08RCGPVtKtcZIPVgKEC03c678W5rrvyrgqQUAJEX08w=
Date: Wed, 1 Sep 2021 18:13:03 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
Message-ID: <YS+mj/mxSJCIso6r@Air-de-Roger>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
 <YS+GmC7IjxoODTu+@Air-de-Roger>
 <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
X-ClientProxiedBy: MR2P264CA0171.FRAP264.PROD.OUTLOOK.COM (2603:10a6:501::10)
 To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2fef1be4-0fca-4577-0fbd-08d96d636379
X-MS-TrafficTypeDiagnostic: DS7PR03MB5590:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5590E145E4C8C9F44E6171138FCD9@DS7PR03MB5590.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: bxJew0XlIm2AISGmCQh+bEzt+9A0LDVSwyBB9dk/uw+R15Z/jkNr7RVFkLEm67V8SShZqNRD8UsQaQVaPuqQeUmZQhk7voeSg2C+BkrX9y5CfTIjIp54/Roenp/WbP3GFtoAnt4YeoEKt1mFl4su1wuZ1JtvYwSniBbba/ombyvPRkW2FlD88kFelvGSXVl9TPHIYgygFgftPOeIGoRphRkGwgVJAxOkGWOCfUkScgNIbQjdA1pT0Bfde1dUeSbkbE7mjVDrF9cMcerEkBuyNpCK5BWnuBFGAGVTSEHQF3cC4gYs9eiK5hC2JRJASHP9KfCxX0okF3w0TLW83p8tCrRAW4lgjckabCy/LMSyGiS5bAakiveai9dM9dUNbYyI3BuUbZFOvLO8KZWPRFbjxAO0hc85taZe9+JxcY6IAHJHknlQKco3yNx2D1yMDdojAL/NTDCMCbCVxYok+4Kq7O91JFhP/ZNZKv/FLoBwXCZ1ob9VAsLRss67LEno6mhQjHopSV/3PHBsxZprzYn+GAtlrQQXc2TFOP6gnqSjZLsin5z17COlxzaK8f4QDEO6dgLT34xt1ubHBOF7I0Jab6IVbD8kmKUh5qGWFplBgy+6ghewhHMteysOFmlYenHivrs+yPSZ8diSNTyDOBoZSA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(7916004)(4636009)(376002)(346002)(136003)(396003)(39840400004)(366004)(86362001)(85182001)(316002)(4326008)(6916009)(8676002)(6486002)(66476007)(9686003)(54906003)(83380400001)(2906002)(6666004)(66946007)(26005)(53546011)(956004)(38100700002)(33716001)(186003)(5660300002)(8936002)(478600001)(66556008)(6496006);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dkNwZ3RHYjdFRlljOFlZRUdEVWE5U05PTmRIUUtEQUhCSmFpbmU0ODg1WEs2?=
 =?utf-8?B?MTJ0MmVnZU9QUG5MSVZhamwyWVdiT3p2U1g5eUxBUFF1MnpuTGtxdkl2YUdM?=
 =?utf-8?B?bm5JTUpJOHpTakhEN1V3bUgxMVZLSEJPaENSZzRtQnNRRVp1VHVJeGJrcVNY?=
 =?utf-8?B?L0FEUXJUamszU1cwZk9URGlQVzhEang5Q0ZYNlVhaEhkZVRZenNDRXY5akd4?=
 =?utf-8?B?aFR4bGs3VDZRRFBiYTBMeEhhRTFJRVUzSFczei9xZkRJTW5OYzczdVNESkxw?=
 =?utf-8?B?dkRhQkRoY01wbnBNRkJSUjdFMFVpKzM3OFdtY053cDFyWmo1MDIxSit6SEcy?=
 =?utf-8?B?L29sek1Ubkp2OHZsTGRQTXF2OEt1R3UrY3BNWnRISVpPdVhTeTE4OTM2a3JI?=
 =?utf-8?B?US9TL09jOVdRREUyODA2UG1LZ291N2wzV1NocnhYRjJlYllPQ2F5WGdXWmNH?=
 =?utf-8?B?S0FjQis1UHEvMUtkZWV5d3ptWXhkTXRYc254UWUwbHk1dlBvdElWVHVtYWJ2?=
 =?utf-8?B?Y0ZxL3JmVVdNcTJEWmUyak43LzNtUmhlak1zaTkycDg4SzVkVndjQ0ZmV1cy?=
 =?utf-8?B?MjBRT0xVOFZkRGJyK2RmTjVKQUJkZmtuVUVzQk5aR3RsMXFrRFIyTUsySmsv?=
 =?utf-8?B?UWdmUmdPbU10K0wzNWdHbU5MS2xPckVkbU9jOERNQWpES1lIRUZ1TVgxeWNJ?=
 =?utf-8?B?NE00aExzRm5URk9kSnkxWXpxUFdXQzlKYk40ZnFkb2JJajBiMEVQSm5tTFVE?=
 =?utf-8?B?dTdsYWpWcS9ZUG5McFN4V0h6Y0JRU1JnTWFkZ1NUWE5yRkExR3oxQ1BvYVpM?=
 =?utf-8?B?Z0VBQW8yVjNQVEd5RVcxckVWWXJvekJmOGdrMVIwNk04YjVoaGhGajZhczVO?=
 =?utf-8?B?QlNaVHphNmYrRytrQWxXZU1BVlpRNEo2UXRGdytraUpWL1A2RGRiR29XMTlN?=
 =?utf-8?B?U3dGTHZtZlAweG4ySUJDZG1EdG9SSWxhSkEwYWtJNmgydlBBKzNsek5EYXFL?=
 =?utf-8?B?L2JVZ2IvRVZ2Ny9sVFFoVkJzYkRDL3VHVTVBM3RsU1d1Mlh1cTVBRTdIRUp1?=
 =?utf-8?B?TlZMVXpoeXFHTWZGcTF5U3diVVI1NFRib3JjQzRRejJKUHVQR1ord0xtNXNa?=
 =?utf-8?B?YWNBNEkvN1A2czdNdDk1bTlyV1NxYmFXbkF4T0lHemRjcCt4cGhXR05yYnRn?=
 =?utf-8?B?aGNreUVVeERtSWdSS1NxL2hIcFlScFpnOFFDZTZlN2xVcTg2dFJoTHc2dXNa?=
 =?utf-8?B?YWhVWnZXQU16M3NOczFZSlpLNmhxQ2FMQ2FpSlBQRW5Mak9VdkNDWTBhN21o?=
 =?utf-8?B?QVc0b2xhdU4zS0JLd0RJakY4MlNKNTFxWnlmb005enBHQWc0a1QxQkx5ejZv?=
 =?utf-8?B?cjdldU9QSE5CcTE1UE03dWFUSnltcHNNdWhQYk1RN3hRYW5ZRVpEWWtaTTd2?=
 =?utf-8?B?akx5c1lYOVpHaWtBV3VNanExNjduUHFDUE5MSE1QalJBc2Q5c1BxWGgxZXF4?=
 =?utf-8?B?MHRKUVdtczA4aWJaWDF4cUFIaDNHOFJWamF2YmpoUXg2YUo1WUg5T3BrbnNp?=
 =?utf-8?B?STNpUXNialdTcE5nSFprLzZKTUNQS2pkV2U1YUg4VTY1bWVuQ3VjNDVXY1Qy?=
 =?utf-8?B?Ujh0WDNFNVNZM202OWkxaEhyRDdibG0wYWZPajAxYm5mcGV4Z3JLWnR3Wkkz?=
 =?utf-8?B?NlJxMG1vTTlJd3pmazJYMmpvODkwRWkrdWI1b3BoSXV5RFZ2N2tSYndRMnhI?=
 =?utf-8?Q?G7DBNlaD/lAp42WXI/PE6C+vOZvk+ghIgVMrcZc?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2fef1be4-0fca-4577-0fbd-08d96d636379
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 16:13:08.9854
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oUKrbj29RCl5yKvLkjHwtpYVs1JGH5mOMotbej5gOofsiAfDWB5WzytGyzl6Qok0rf6qDsVdSmAN/yYsSmglKQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5590
X-OriginatorOrg: citrix.com

On Wed, Sep 01, 2021 at 04:19:40PM +0200, Jan Beulich wrote:
> On 01.09.2021 15:56, Roger Pau Monné wrote:
> > On Tue, Aug 31, 2021 at 10:53:59AM +0200, Jan Beulich wrote:
> >> On 30.08.2021 15:01, Jan Beulich wrote:
> >>> The code building PVH Dom0 made use of sequences of P2M changes
> >>> which are disallowed as of XSA-378. First of all population of the
> >>> first Mb of memory needs to be redone. Then, largely as a
> >>> workaround, checking introduced by XSA-378 needs to be slightly
> >>> relaxed.
> >>>
> >>> Note that with these adjustments I get Dom0 to start booting on my
> >>> development system, but the Dom0 kernel then gets stuck. Since it
> >>> was the first time for me to try PVH Dom0 in this context (see
> >>> below for why I was hesitant), I cannot tell yet whether this is
> >>> due further fallout from the XSA, or some further unrelated
> >>> problem.
> > 
> > Iff you have some time could you check without the XSA applied? I have
> > to admit I haven't been testing staging, so it's possible some
> > breakage as slipped in (however osstest seemed fine with it).
> 
> Well, I'd rather try to use the time to find the actual issue. From
> osstest being fine I'm kind of inferring this might be machine
> specific, or this might be due to yet some other of the overly many
> patches I'm carrying. So if I can't infer anything from the stack
> once I can actually dump that, I may indeed need to bisect my pile,
> which would then also include the XSA-378 patches (as I didn't have
> time to re-base so far).
> 
> >>> Dom0's BSP is in VPF_blocked state while all APs are
> >>> still in VPF_down. The 'd' debug key, unhelpfully, doesn't produce
> >>> any output, so it's non-trivial to check whether (like PV likes to
> >>> do) Dom0 has panic()ed without leaving any (visible) output.
> > 
> > Not sure it would help much, but maybe you can post the Xen+Linux
> > output?
> 
> There's no Linux output yet by that point (and either
> "earlyprintk=xen" doesn't work in PVH mode, or it's even too early
> for that). All Xen has to say is
> 
> (XEN) Dom0 callback via changed to Direct Vector 0xf3
> (XEN) vmx.c:3265:d0v0 RDMSR 0x0000064e unimplemented
> (XEN) vmx.c:3265:d0v0 RDMSR 0x00000034 unimplemented

Weird, I don't see why earlyprintk=xen shouldn't work in PVH mode,
unless it's not properly wired up. Certainly needs checking and
fixing, or else we won't be able to make much progress I think.

> > Do you have iommu debug/verbose enabled to catch iommu faults?
> 
> I'll try to remember to check that, but since Linux hasn't
> brought up APs yet I don't think there's any device activity
> just yet.
> 
> >> Correction: I did mean '0' here, producing merely
> >>
> >> (XEN) '0' pressed -> dumping Dom0's registers
> >> (XEN) *** Dumping Dom0 vcpu#0 state: ***
> >> (XEN) *** Dumping Dom0 vcpu#1 state: ***
> >> (XEN) *** Dumping Dom0 vcpu#2 state: ***
> >> (XEN) *** Dumping Dom0 vcpu#3 state: ***
> >>
> >> 'd' output supports the "system is idle" that was also visible from
> >> 'q' output.
> > 
> > Can you dump the state of the VMCS and see where the IP points to in
> > Linux?
> 
> Both that and the register dumping I have meanwhile working tell
> me that it's the HLT in default_idle(). IOW Dom0 gives the impression
> of also being idle, at the first glance. The stack pointer, however,
> is farther away from the stack top than I would have expected, so it
> may still have entered default_idle() for other reasons.
> 
> The VMCS also told me that the last VM entry was to deliver an
> interrupt at vector 0xf3 (i.e. the "callback" one).

That's all quite weird. Did dom0 setup the vCPU timer? What version of
Linux are you using?

It seems to get stuck very early (or either fail to output anything
while booting), which seems unlikely to be related to your specific
hardware.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 16:22:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 16:22:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176560.321272 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSzt-0004LF-PJ; Wed, 01 Sep 2021 16:21:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176560.321272; Wed, 01 Sep 2021 16:21:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLSzt-0004L8-M3; Wed, 01 Sep 2021 16:21:57 +0000
Received: by outflank-mailman (input) for mailman id 176560;
 Wed, 01 Sep 2021 16:21:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B+dc=NX=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLSzs-0004Kz-6k
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 16:21:56 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b85db17c-0b40-11ec-ae01-12813bfff9fa;
 Wed, 01 Sep 2021 16:21:54 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 740B56069E;
 Wed,  1 Sep 2021 16:21:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b85db17c-0b40-11ec-ae01-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630513313;
	bh=vRwjV5s4lKaeJNiv3puLhTfXA2shIEq8W7TQ2apezrQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GNJG9s+oQomjzMi7Ot9T/6d3kPwB5n24zx2sr0/cIe3y+JlHPZTIyy4EayFEPChI9
	 nY6FdxHdbvKkfUvXKdkQJ8yFPRwIglYeHSRKCkh9Z/GJ2TVd6Lgnyr28fz+YFavDzC
	 sTkIpvZSW9EMHqjl83oxOJhdgOwZtEx9LZu6MX47lVaS3cGE9fqiMblOmkyqT2aUZ/
	 UiV01PZ+V2hDNCHuVguyY5C62p1VpJNurWZ/Fip8U0MiBlItTjpSN7hU77sqp5sUuY
	 k8AJ/EDTecw3krj6HCRSsrGBBNCHZbtGf8/PdT4eK067dXFPIBEnszi4pW1pk8sOoX
	 NlS5SCe5098Ng==
Date: Wed, 1 Sep 2021 09:21:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
In-Reply-To: <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
References: <20210811102423.28908-1-wei.chen@arm.com> <20210811102423.28908-25-wei.chen@arm.com> <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s> <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s> <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-968995931-1630513313=:17925"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-968995931-1630513313=:17925
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 1 Sep 2021, Wei Chen wrote:
> Hi Stefano,
> 
> > -----Original Message-----
> > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > Stefano Stabellini
> > Sent: 2021年9月1日 5:36
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>
> > Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
> > device tree NUMA distance map
> > 
> > On Tue, 31 Aug 2021, Wei Chen wrote:
> > > Hi Stefano,
> > >
> > > > -----Original Message-----
> > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > Sent: 2021年8月31日 8:48
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > julien@xen.org;
> > > > jbeulich@suse.com; Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > Subject: Re: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to
> > parse
> > > > device tree NUMA distance map
> > > >
> > > > On Wed, 11 Aug 2021, Wei Chen wrote:
> > > > > A NUMA aware device tree will provide a "distance-map" node to
> > > > > describe distance between any two nodes. This patch introduce a
> > > > > new helper to parse this distance map.
> > > > >
> > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > ---
> > > > >  xen/arch/arm/numa_device_tree.c | 67
> > +++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 67 insertions(+)
> > > > >
> > > > > diff --git a/xen/arch/arm/numa_device_tree.c
> > > > b/xen/arch/arm/numa_device_tree.c
> > > > > index bbe081dcd1..6e0d1d3d9f 100644
> > > > > --- a/xen/arch/arm/numa_device_tree.c
> > > > > +++ b/xen/arch/arm/numa_device_tree.c
> > > > > @@ -200,3 +200,70 @@ device_tree_parse_numa_memory_node(const void
> > *fdt,
> > > > int node,
> > > > >
> > > > >      return 0;
> > > > >  }
> > > > > +
> > > > > +/* Parse NUMA distance map v1 */
> > > > > +int __init
> > > > > +device_tree_parse_numa_distance_map_v1(const void *fdt, int node)
> > > > > +{
> > > > > +    const struct fdt_property *prop;
> > > > > +    const __be32 *matrix;
> > > > > +    int entry_count, len, i;
> > > > > +
> > > > > +    printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n");
> > > > > +
> > > > > +    prop = fdt_get_property(fdt, node, "distance-matrix", &len);
> > > > > +    if ( !prop )
> > > > > +    {
> > > > > +        printk(XENLOG_WARNING
> > > > > +               "NUMA: No distance-matrix property in distance-
> > map\n");
> > > > > +
> > > > > +        return -EINVAL;
> > > > > +    }
> > > > > +
> > > > > +    if ( len % sizeof(uint32_t) != 0 )
> > > > > +    {
> > > > > +        printk(XENLOG_WARNING
> > > > > +               "distance-matrix in node is not a multiple of
> > u32\n");
> > > > > +        return -EINVAL;
> > > > > +    }
> > > > > +
> > > > > +    entry_count = len / sizeof(uint32_t);
> > > > > +    if ( entry_count <= 0 )
> > > > > +    {
> > > > > +        printk(XENLOG_WARNING "NUMA: Invalid distance-matrix\n");
> > > > > +
> > > > > +        return -EINVAL;
> > > > > +    }
> > > > > +
> > > > > +    matrix = (const __be32 *)prop->data;
> > > > > +    for ( i = 0; i + 2 < entry_count; i += 3 )
> > > > > +    {
> > > > > +        uint32_t from, to, distance;
> > > > > +
> > > > > +        from = dt_read_number(matrix, 1);
> > > > > +        matrix++;
> > > > > +        to = dt_read_number(matrix, 1);
> > > > > +        matrix++;
> > > > > +        distance = dt_read_number(matrix, 1);
> > > > > +        matrix++;
> > > > > +
> > > > > +        if ( (from == to && distance != NUMA_LOCAL_DISTANCE) ||
> > > > > +            (from != to && distance <= NUMA_LOCAL_DISTANCE) )
> > > > > +        {
> > > > > +            printk(XENLOG_WARNING
> > > > > +                   "Invalid nodes' distance from node#%d to node#%d
> > > > = %d\n",
> > > > > +                   from, to, distance);
> > > > > +            return -EINVAL;
> > > > > +        }
> > > > > +
> > > > > +        printk(XENLOG_INFO "NUMA: distance from node#%d to node#%d
> > > > = %d\n",
> > > > > +               from, to, distance);
> > > > > +        numa_set_distance(from, to, distance);
> > > > > +
> > > > > +        /* Set default distance of node B->A same as A->B */
> > > > > +        if (to > from)
> > > > > +             numa_set_distance(to, from, distance);
> > > >
> > > > I am a bit unsure about this last 2 lines: why calling
> > numa_set_distance
> > > > in the opposite direction only when to > from? Wouldn't it be OK to
> > > > always do both:
> > > >
> > > > numa_set_distance(from, to, distance);
> > > > numa_set_distance(to, from, distance);
> > > >
> > > > ?
> > > >
> > > I borrowed this code from Linux, but here is my understanding:
> > >
> > > First, I read some notes in Documentation/devicetree/bindings/numa.txt
> > > 1. Each entry represents distance from first node to second node.
> > > The distances are equal in either direction.
> > > 2. distance-matrix should have entries in lexicographical ascending
> > > order of nodes.
> > >
> > > Here is an example of distance-map node in DTB:
> > > Sample#1, full list:
> > > 		distance-map {
> > > 			 compatible = "numa-distance-map-v1";
> > > 			 distance-matrix = <0 0  10>,
> > > 					   <0 1  20>,
> > > 					   <0 2  40>,
> > > 					   <0 3  20>,
> > > 					   <1 0  20>,
> > > 					   <1 1  10>,
> > > 					   <1 2  20>,
> > > 					   <1 3  40>,
> > > 					   <2 0  40>,
> > > 					   <2 1  20>,
> > > 					   <2 2  10>,
> > > 					   <2 3  20>,
> > > 					   <3 0  20>,
> > > 					   <3 1  40>,
> > > 					   <3 2  20>,
> > > 					   <3 3  10>;
> > > 		};
> > >
> > > Call numa_set_distance when "to > from" will prevent Xen to call
> > > numa_set_distance(0, 1, 20) again when it's setting distance for <1 0
> > 20>.
> > > But, numa_set_distance(1, 0, 20) will be call twice.
> > >
> > > Normally, distance-map node will be optimized in following sample#2,
> > > all redundant entries are removed:
> > > Sample#2, partial list:
> > > 		distance-map {
> > > 			 compatible = "numa-distance-map-v1";
> > > 			 distance-matrix = <0 0  10>,
> > > 					   <0 1  20>,
> > > 					   <0 2  40>,
> > > 					   <0 3  20>,
> > > 					   <1 1  10>,
> > > 					   <1 2  20>,
> > > 					   <1 3  40>,
> > > 					   <2 2  10>,
> > > 					   <2 3  20>,
> > > 					   <3 3  10>;
> > > 		};
> > >
> > > There is not any "from > to" entry in the map. But using this partial
> > map
> > > still can set all distances for all pairs. And numa_set_distance(1, 0,
> > 20)
> > > will be only once.
> > 
> > I see. I can't find in Documentation/devicetree/bindings/numa.txt where
> > it says that "from > to" nodes can be omitted. If it is not written
> > down, then somebody could easily optimize it the opposite way:
> > 
> >  			 distance-matrix = <0 0  10>,
> >  					   <1 0  20>,
> >  					   <2 0  40>,
> >  					   <3 0  20>,
> >  					   <1 1  10>,
> >  					   <2 1  20>,
> >  					   <3 1  40>,
> >  					   <2 2  10>,
> >  					   <3 2  20>,
> >  					   <3 3  10>;
> > 
> 
> Yes, you're right. Spec doesn't say opposite way is unallowed.
> 
> > I think the code in Xen should be resilient and able to cope with a
> > device tree like the one you wrote or the one I wrote. From a code
> > perspective, it should be very easy to do. If nothing else it would make
> > Xen more resilient against buggy firmware.
> > 
> > 
> 
> I don't disagree with that.
> 
> > > > But in any case, I have a different suggestion. The binding states
> > that
> > > > "distances are equal in either direction". Also it has an example
> > where
> > > > only one direction is expressed unfortunately (at the end of the
> > > > document).
> > > >
> > >
> > > Oh, I should see this comment first, then I will not post above
> > > comment : )
> > >
> > > > So my suggestion is to parse it as follows:
> > > >
> > > > - call numa_set_distance just once from
> > > >   device_tree_parse_numa_distance_map_v1
> > > >
> > > > - in numa_set_distance:
> > > >     - set node_distance_map[from][to] = distance;
> > > >     - check node_distance_map[to][from]
> > > >           - if unset, node_distance_map[to][from] = distance;
> > > >           - if already set to the same value, return success;
> > > >           - if already set to a different value, return error;
> > >
> > > I don't really like this implementation. I want the behavior of
> > > numa_set_distance just like the function name, do not include
> > > implicit operations. Otherwise, except the user read this function
> > > implementation before he use it, he probably doesn't know this
> > > function has done so many things.
> > 
> > You can leave numa_set_distance as-is without any implicit operations.
> > 
> > In that case, just call numa_set_distance twice from numa_set_distance
> > for both from/to and to/from. numa_set_distance could return error is
> 
> I am OK for the first sentence. But...
> 
> > the entry was already set to a different value or success otherwise
> > (also in the case it was already set to the same value). This would
> 
> ... I prefer not to check the previous value. Subsequent numa_set_distance
> call will override previous calls. Keep numa_set_distance as simple as
> it can. And when you pass new data to numa_set_distance, it doesn't
> know whether the previous data was correct or the new data is correct.
> Only caller may have known.  

That might be OK but if not numa_set_distance then somebody else needs
to check against overwriting previous values. That is to be able to spot
bad device tree cases like:

  0 1 20
  1 0 40
--8323329-968995931-1630513313=:17925--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 16:25:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 16:25:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176567.321283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLT2u-00055Y-Bn; Wed, 01 Sep 2021 16:25:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176567.321283; Wed, 01 Sep 2021 16:25:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLT2u-00055R-8e; Wed, 01 Sep 2021 16:25:04 +0000
Received: by outflank-mailman (input) for mailman id 176567;
 Wed, 01 Sep 2021 16:25:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLT2t-000559-NM; Wed, 01 Sep 2021 16:25:03 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLT2t-0006Nk-Dj; Wed, 01 Sep 2021 16:25:03 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLT2t-0004xd-7M; Wed, 01 Sep 2021 16:25:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLT2t-0007F0-6u; Wed, 01 Sep 2021 16:25:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3GXjwKk1jtgZb8r2OIwTJemQ5cT4UDoIsUDY52UewyE=; b=Rno4H2CnRUzfODlEzp8OQg3DNj
	UxZWVrEED9Pjp+3mIy4TSOLmhdmHIeorPcIPnArB8uatvOl6HnYzyb73MydS72kUXD5tnu4HgdgPj
	KZLr9m34juD/vxo1lOk/MjnAGWaM5uvYTOeen9DFRGB+dOb/gLdLGT65GuptsKsPeoFk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164704-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164704: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b170806518c1e414939c8b085866544814e1ce8e
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 16:25:03 +0000

flight 164704 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164704/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b170806518c1e414939c8b085866544814e1ce8e
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    3 attempts
Testing same since   164695  2021-09-01 10:40:04 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 16:26:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 16:26:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176574.321297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLT4J-0005hH-Os; Wed, 01 Sep 2021 16:26:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176574.321297; Wed, 01 Sep 2021 16:26:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLT4J-0005hA-Lp; Wed, 01 Sep 2021 16:26:31 +0000
Received: by outflank-mailman (input) for mailman id 176574;
 Wed, 01 Sep 2021 16:26:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=viXG=NX=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mLT4I-0005gw-AR
 for xen-devel@lists.xen.org; Wed, 01 Sep 2021 16:26:30 +0000
Received: from mail-wm1-x329.google.com (unknown [2a00:1450:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0add8b29-e91e-4907-9b94-66544d7c7a2f;
 Wed, 01 Sep 2021 16:26:28 +0000 (UTC)
Received: by mail-wm1-x329.google.com with SMTP id
 k20-20020a05600c0b5400b002e87ad6956eso84542wmr.1
 for <xen-devel@lists.xen.org>; Wed, 01 Sep 2021 09:26:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0add8b29-e91e-4907-9b94-66544d7c7a2f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=hD6i+Tmn6yQISidhCVyfY7yvCP2yOSJZBebtAHZRXp8=;
        b=CwvDbyjqE/5U4XrUdyWNPgwHyB8fguYfy6xEzFi3IH08QqKYsTprTFs24eXjfPCF8a
         ShehDh1Qm/bis30H4x8En2ucFmy//wQL6v8omiNkShASFVlaa2o/31esQtfnNGwKMWID
         +IINvGanj9JDaUIM4ww8f5noBQvL2l0PakkYoT2VuViWy1V9zmMBA4jy6qw8W63G/TrP
         0bXzpe42VGXPnbah1UHS6yiZNNyzT1w0UKqAa9SE2wHTouHAjz+pPxz4I/EFBCZJOnkc
         0zf/6X8gwHuAFja90RBn+fFOMr3IiNSFB+p24nrVd/lUFuVS1gx035Qjv5VfEBgssWAq
         Ls7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=hD6i+Tmn6yQISidhCVyfY7yvCP2yOSJZBebtAHZRXp8=;
        b=qj6HsWGv4ojU4LfqMsSCAeVaFuyme0mqlrlv3+eIHwfS3fTdT5i+m1K7wgJLPtvuMf
         Y7gOuLlCakjBiJmYviuRXUdPfgahFCFZWmy6DVl1yXp/NATVgMMgrK8SArOn7mO59mxL
         39u+o+a1H4gQWS+kcH93rWiElh15FgwsRQOGRae4yiuz07RpmOF+cRl2wQvNdXNn+Tgy
         VbU+hcrFI0zGYNaYlqt1ncZXsFRwmEf6ByxK9L40kykU06x33Gusovar+4veIgvuZtYi
         6EBuadbYLJsVzCGFv2kMbEy7ls5ERKohLypBjpgvsItnSnChXXLmR+GRJadficQRjvyB
         82WA==
X-Gm-Message-State: AOAM5327NqANsbPjWpcx1Dk2OWcHaR5J6ftsSXOoAYapVCYsDADsxLPb
	Fy5XuowL2V3+38vFFqZhcRn1wqdBDK4BidCzvUM=
X-Google-Smtp-Source: ABdhPJxysw6v03n5aApJo8ZcR8hFITSnVlLB+2zC4NsLIOHNVJn4JcLUuEUjcZXwFCHikApOxhJ1pTdAJ66vKauC56c=
X-Received: by 2002:a1c:a181:: with SMTP id k123mr378799wme.90.1630513587810;
 Wed, 01 Sep 2021 09:26:27 -0700 (PDT)
MIME-Version: 1.0
References: <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa> <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa> <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210831061805.GB69817@laputa> <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210901122910.GA100270@laputa>
In-Reply-To: <20210901122910.GA100270@laputa>
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
Date: Wed, 1 Sep 2021 19:26:16 +0300
Message-ID: <CAPD2p-n1on9H5aegznmMwZHHYvMAN8QR7kxmfj0UCBgyrcyWsw@mail.gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Wei Chen <Wei.Chen@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	"Alex Benn??e" <alex.bennee@linaro.org>, Kaly Xin <Kaly.Xin@arm.com>, 
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	Viresh Kumar <viresh.kumar@linaro.org>, 
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com" <stefanha@redhat.com>, 
	Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>, 
	"pratikp@quicinc.com" <pratikp@quicinc.com>, Srivatsa Vaddagiri <vatsa@codeaurora.org>, 
	Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>, 
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, 
	Paul Durrant <paul@xen.org>, nd <nd@arm.com>, Xen Devel <xen-devel@lists.xen.org>
Content-Type: multipart/alternative; boundary="00000000000033f5d805caf1878f"

--00000000000033f5d805caf1878f
Content-Type: text/plain; charset="UTF-8"

Hi Akashi,

I am sorry for the possible format issues.


>
> > > >
> > > > It's a RFC discussion. We have tested this RFC patch internally.
> > > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > > 07/msg01532.html
> > >
> > > I'm afraid that I miss something here, but I don't know
> > > why this proposed API will lead to eliminating 'mmap' in accessing
> > > the queued payload at every request?
> > >
> >
> > This API give Xen device model (QEMU or kvmtool) the ability to map
> > whole guest RAM in device model's address space. In this case, device
> > model doesn't need dynamic hypercall to map/unmap payload memory.
> > It can use a flat offset to access payload memory in its address
> > space directly. Just Like KVM device model does now.
>
Yes!


>
> Thank you. Quickly, let me make sure one thing:
> This API itself doesn't do any mapping operations, right?


Right. The only purpose of that "API" is to guery hypervisor to find
unallocated address space ranges to map the foreign pages into (instead of
stealing real RAM pages),
In a nutshell, if you try to map the whole guest memory in the backend
address space on Arm (or even cache some mappings) you might end up with
memory exhaustion in the backend domain (XSA-300),
and the possibility to hit XSA-300 is higher if your backend needs to serve
several Guests. Of course, this depends on the memory assigned to the
backend domain and to the Guest(s) it serves...
We believe that with the proposed solution the backend will be able to
handle Guest(s) without wasting it's real RAM. However, please note that
proposed Xen + Linux changes which are on review now [1] are far from the
final solution and require rework and some prereq work to operate in a
proper and safe way.


>
> So I suppose that virtio BE guest is responsible to
> 1) fetch the information about all the memory regions in FE,
> 2) call this API to allocate a big chunk of unused space in BE,
> 3) create grant/foreign mappings for FE onto this region(S)
> in the initialization/configuration of emulated virtio devices.
>
> Is this the way this API is expected to be used?
>

No really, the userspace backend doesn't need to call this API at all, all
what backend calls is still
xenforeignmemory_map()/xenforeignmemory_unmap(), so let's say "magic" is
done by Linux and Xen internally.
You can take a look at the virtio-disk PoC [2] (last 4 patches) to better
understand what Wei and I are talking about. There we map the Guest memory
at the beginning and just calculate a pointer at runtime. Again, the code
is not in good shape, but it is enough to demonstrate the feasibility of
the improvement.



> Does Xen already has an interface for (1)?
>

I am not aware of anything existing. For the PoC I guessed the Guest memory
layout in a really hackish way (I got total Guest memory size, so having
GUEST_RAMX_BASE/GUEST_RAMX_SIZE in hand just performed calculation).
Definitely, it is a no-go, so 1) deserves additional discussion/design.

[1]
https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
https://lore.kernel.org/lkml/1627490656-1267-1-git-send-email-olekstysh@gmail.com/
https://lore.kernel.org/lkml/1627490656-1267-2-git-send-email-olekstysh@gmail.com/
[2]
https://github.com/otyshchenko1/virtio-disk/commits/map_opt_next
-- 
Regards,

Oleksandr Tyshchenko

--00000000000033f5d805caf1878f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><div>Hi Akashi,</div><div><br><=
/div><div>I am sorry for the possible format issues.</div><div><br></div><d=
iv class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:=
0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">=
<br><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; It&#39;s a RFC discussion. We have tested this RFC patch int=
ernally.<br>
&gt; &gt; &gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-de=
vel/2021-" rel=3D"noreferrer" target=3D"_blank">https://lists.xenproject.or=
g/archives/html/xen-devel/2021-</a><br>
&gt; &gt; 07/msg01532.html<br>
&gt; &gt; <br>
&gt; &gt; I&#39;m afraid that I miss something here, but I don&#39;t know<b=
r>
&gt; &gt; why this proposed API will lead to eliminating &#39;mmap&#39; in =
accessing<br>
&gt; &gt; the queued payload at every request?<br>
&gt; &gt; <br>
&gt; <br>
&gt; This API give Xen device model (QEMU or kvmtool) the ability to map<br=
>
&gt; whole guest RAM in device model&#39;s address space. In this case, dev=
ice<br>
&gt; model doesn&#39;t need dynamic hypercall to map/unmap payload memory.<=
br>
&gt; It can use a flat offset to access payload memory in its address<br>
&gt; space directly. Just Like KVM device model does now.<br></blockquote><=
div>Yes!</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex">
<br>
Thank you. Quickly, let me make sure one thing:<br>
This API itself doesn&#39;t do any mapping operations, right?</blockquote><=
div>=C2=A0</div><div>Right. The only purpose of that &quot;API&quot; is to =
guery hypervisor to find unallocated address space ranges to map the foreig=
n pages into (instead of stealing real RAM pages),</div><div>In a nutshell,=
 if you try to map the whole guest memory in the backend address space on A=
rm (or even cache some mappings) you might end up with memory exhaustion in=
 the backend domain (XSA-300),</div><div>and the possibility to hit XSA-300=
 is higher if your backend needs to serve several Guests. Of course, this d=
epends on the memory assigned to the backend domain and to the Guest(s) it =
serves...</div><div>We believe that with the proposed solution the backend =
will be able to handle Guest(s) without wasting it&#39;s real RAM. However,=
 please note that proposed Xen=C2=A0+ Linux changes which are on review now=
 [1] are far from the final solution and require rework and some prereq wor=
k to operate in a proper and safe way.</div><div>=C2=A0</div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex"><br>
So I suppose that virtio BE guest is responsible to<br>
1) fetch the information about all the memory regions in FE,<br>
2) call this API to allocate a big chunk of unused space in BE,<br>
3) create grant/foreign mappings for FE onto this region(S)<br>
in the initialization/configuration of emulated virtio devices.<br>
<br>
Is this the way this API is expected to be used?<br></blockquote><div>=C2=
=A0</div><div>No really, the userspace backend doesn&#39;t need to call thi=
s API at all, all what backend calls is still xenforeignmemory_map()/xenfor=
eignmemory_unmap(), so let&#39;s say &quot;magic&quot; is done by Linux and=
 Xen internally.</div><div>You can take a look at the virtio-disk PoC [2] (=
last 4 patches) to better understand what Wei and I are talking about. Ther=
e we map the Guest memory at the beginning and just calculate a pointer at =
runtime. Again, the code</div><div>is not in good shape, but it is enough t=
o demonstrate the feasibility of the improvement.</div><div><br></div><div>=
=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Does Xen already has an interface for (1)?<br></blockquote><div>=C2=A0</div=
><div>I am not aware of anything existing. For the PoC I guessed the Guest =
memory layout in a really hackish way (I got total Guest memory size, so ha=
ving GUEST_RAMX_BASE/GUEST_RAMX_SIZE in hand just performed calculation). D=
efinitely, it is a no-go, so 1) deserves additional discussion/design.</div=
></div><div><br></div><div>[1] <br><a href=3D"https://lore.kernel.org/xen-d=
evel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/">https://lore.k=
ernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/<=
/a><br><a href=3D"https://lore.kernel.org/lkml/1627490656-1267-1-git-send-e=
mail-olekstysh@gmail.com/">https://lore.kernel.org/lkml/1627490656-1267-1-g=
it-send-email-olekstysh@gmail.com/</a><br><a href=3D"https://lore.kernel.or=
g/lkml/1627490656-1267-2-git-send-email-olekstysh@gmail.com/">https://lore.=
kernel.org/lkml/1627490656-1267-2-git-send-email-olekstysh@gmail.com/</a><b=
r></div><div>[2]=C2=A0</div><div><a href=3D"https://github.com/otyshchenko1=
/virtio-disk/commits/map_opt_next">https://github.com/otyshchenko1/virtio-d=
isk/commits/map_opt_next</a></div>-- <br><div dir=3D"ltr" class=3D"gmail_si=
gnature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr"><span=
 style=3D"background-color:rgb(255,255,255)"><font size=3D"2"><span style=
=3D"color:rgb(51,51,51);font-family:Arial,sans-serif">Regards,</span></font=
></span></div><div dir=3D"ltr"><br></div><div dir=3D"ltr"><div><span style=
=3D"background-color:rgb(255,255,255)"><font size=3D"2">Oleksandr Tyshchenk=
o</font></span></div></div></div></div></div></div></div></div>

--00000000000033f5d805caf1878f--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 17:52:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 17:52:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176591.321337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLUOa-0008C8-6W; Wed, 01 Sep 2021 17:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176591.321337; Wed, 01 Sep 2021 17:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLUOa-0008C1-3f; Wed, 01 Sep 2021 17:51:32 +0000
Received: by outflank-mailman (input) for mailman id 176591;
 Wed, 01 Sep 2021 17:51:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLUOY-0008Bo-G6; Wed, 01 Sep 2021 17:51:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLUOY-0007q7-AT; Wed, 01 Sep 2021 17:51:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLUOY-0001lK-39; Wed, 01 Sep 2021 17:51:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLUOY-0004yd-2f; Wed, 01 Sep 2021 17:51:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=gVoODzvwTLgF6rGhJdmnqDDgXPX/Bg3PyRE+sZLd31s=; b=V2mJHWxwXFV48lJvuqa0bB2/Kg
	8/1tM9Lxr/zNaUBCluYVUCrcAlJkq+8oWf1mu4BntOMWxgPQ9X7l6i/T4v0F0YjJE1u1UL2Qt9xSv
	Okw0N6StTWhi3T5StKBHk+Hc6p57Hx6sqte9ac0Pj0f9D3vTA7bJbs8DEe8pwgmLXaiY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Subject: [ovmf bisection] complete build-amd64-xsm
Message-Id: <E1mLUOY-0004yd-2f@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 17:51:30 +0000

branch xen-unstable
xenbranch xen-unstable
job build-amd64-xsm
testid xen-build

Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164722/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/ovmf/build-amd64-xsm.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/ovmf/build-amd64-xsm.xen-build --summary-out=tmp/164722.bisection-summary --basis-template=164674 --blessings=real,real-bisect,real-retry ovmf build-amd64-xsm xen-build
Searching for failure / basis pass:
 164704 fail [host=himrod2] / 164674 ok.
Failure / basis pass flights: 164704 / 164674
(tree with no url: minios)
Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Basis pass cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Generating revisions with ./adhoc-revtuple-generator  https://github.com/tianocore/edk2.git#cae735f61328d64e2e8991036707b9e78c0f5f63-b170806518c1e414939c8b085866544814e1ce8e git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823-136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 git://xenbits.xen.org/osstest/seabios.git#54082c81d96028ba8c76fbe6784085cf1df76b\
 20-54082c81d96028ba8c76fbe6784085cf1df76b20 git://xenbits.xen.org/xen.git#a931e8e64af07bd333a31f3b71a3f8f3e7910857-a931e8e64af07bd333a31f3b71a3f8f3e7910857
Loaded 5001 nodes in revision graph
Searching for test results:
 164674 pass cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164686 [host=himrod1]
 164695 fail b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164711 pass cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164712 fail b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164713 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164714 pass 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164715 pass 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164704 fail b170806518c1e414939c8b085866544814e1ce8e 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164716 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164717 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164719 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164720 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164721 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
 164722 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
Searching for interesting versions
 Result found: flight 164674 (pass), for basis pass
 For basis failure, parent search stopping at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561910c2bd50) HASH(0x561910c2f580) HASH(0x561910c33110) For basis failure, parent search stopping at 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d\
 96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561910c27ec0) For basis failure, parent search stopping at 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561910c26c98) For basis failure, parent search stopping at cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee\
 84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857, results HASH(0x561910c19838) HASH(0x561910c20178) Result found: flight 164695 (fail), for basis failure (at ancestor ~5000)
 Repro found: flight 164711 (pass), for basis pass
 Repro found: flight 164712 (fail), for basis failure
 0 revisions at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 a931e8e64af07bd333a31f3b71a3f8f3e7910857
No revisions left to test, checking graph state.
 Result found: flight 164716 (pass), for last pass
 Result found: flight 164717 (fail), for first failure
 Repro found: flight 164719 (pass), for last pass
 Repro found: flight 164720 (fail), for first failure
 Repro found: flight 164721 (pass), for last pass
 Repro found: flight 164722 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164722/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>

Revision graph left in /home/logs/results/bisect/ovmf/build-amd64-xsm.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
164722: tolerable ALL FAIL

flight 164722 ovmf real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/164722/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-amd64-xsm               6 xen-build               fail baseline untested


jobs:
 build-amd64-xsm                                              fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 17:54:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 17:54:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176600.321355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLURZ-0000VJ-Tb; Wed, 01 Sep 2021 17:54:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176600.321355; Wed, 01 Sep 2021 17:54:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLURZ-0000VC-QS; Wed, 01 Sep 2021 17:54:37 +0000
Received: by outflank-mailman (input) for mailman id 176600;
 Wed, 01 Sep 2021 17:54:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B+dc=NX=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLURY-0000V6-Mt
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 17:54:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ab1c3152-0b4d-11ec-ae10-12813bfff9fa;
 Wed, 01 Sep 2021 17:54:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8F05D61075;
 Wed,  1 Sep 2021 17:54:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab1c3152-0b4d-11ec-ae10-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630518875;
	bh=DUYclV0TShbbiOecFDX12GrrUJ+2Px1HUu7Ti14aY7A=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=E/2Vg2VmQYMX33ke1fDrCFkwKZAxJiN07iIg7F/Lw3A5A97UTDBL8QL0IWyGAAaad
	 Qj0R5EgohIYcg8WgHWVmxum+RofseXOscq8E/5hbyH54C+R2AaoOwumNen4Ub0kAJ8
	 5YjZnIzHoSlg/8HWICwcCGkJwLWYXK7zA6oJyCk1+pFwLbFvEwglkX2FgiWDZWuALM
	 YPQGNCgjokyYQMI8MRw8uhV37qch7VwfBEqPavXymiSO28qEc9O/YPx04aF4JW0tLv
	 9+YTAvxhdRcSQTgEOXkAHSFv0o3pIf/z0/9pF/OF/aTgss9pq4iuYZ9G/wOa4nOzr6
	 bYD2V6CfSvYRQ==
Date: Wed, 1 Sep 2021 10:54:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Michal Orzel <Michal.Orzel@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
In-Reply-To: <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
Message-ID: <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
References: <20210901124308.31573-1-michal.orzel@arm.com> <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org> <44B48162-EB44-4295-9629-2D95A40D7232@arm.com> <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-93430960-1630518639=:17925"
Content-ID: <alpine.DEB.2.21.2109011050420.17925@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-93430960-1630518639=:17925
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109011050421.17925@sstabellini-ThinkPad-T480s>

On Wed, 1 Sep 2021, Julien Grall wrote:
> On 01/09/2021 14:10, Bertrand Marquis wrote:
> > > On 1 Sep 2021, at 13:55, Julien Grall <julien@xen.org> wrote:
> > > 
> > > Hi,
> > > 
> > > On 01/09/2021 13:43, Michal Orzel wrote:
> > > > Introduce new Xen command line parameter called "pmu_access".
> > > > The default value is "trap": Xen traps PMU accesses.
> > > > In case of setting pmu_access to "native", Xen does not trap
> > > > PMU accesses allowing all the guests to access PMU registers.
> > > > However, guests cannot make use of PMU overflow interrupts as
> > > > PMU uses PPI which Xen cannot route to guests.
> > > > This option is only intended for development and testing purposes.
> > > > Do not use this in production system.
> > > I am afraid your option is not safe even in development system as a vCPU
> > > may move between pCPUs.
> > > 
> > > However, even if we restricted the use to pinned vCPU *and* dedicated
> > > pCPU, I am not convinced that exposing an half backed PMU (the overflow
> > > interrupt would not work) to the guest is the right solution. This likely
> > > means the guest OS would need to be modified and therefore the usage of
> > > this option is fairly limited.
> > > 
> > > So I think the first steps are:
> > >   1) Make the PPI work. There was some attempt in the past for it on
> > > xen-devel. You could have a look.
> > >   2) Provide PMU bindings
> > > 
> > > With that in place, we can discuss how to expose the PMU even if it is
> > > unsafe in some conditions.
> > 
> > With those limitations, using the PMU to monitor the system performances or
> > on some specific use cases is still really useful.
> > We are using that to do some benchmarks of Xen or of some applications to
> > compare the behaviour to a native system or
> > analyse the performances of Xen itself (hypercalls,context switch …etc)

I also already had to write a patch almost exactly like this one to
provide to customers a few months back.


> I understand this is useful for some setup and I am not trying to say we
> should not have a way to expose the PMU (even unsafely) in upstream. However,
> I think the option as it stands is too wide (this should be a per domain knob)
> and we should properly expose the PMU (interrupts, bindings...).

I have never used PMU directly myself, only provided a patch similar to
this one.  But as far as I could tell the users were fully satisfied
with it and it had no interrupts support either. Could it be that
interrupts are not actually needed to read the perf counters, which is
probably what users care about?

In regards to "this should be a per domain knob", in reality if you are
doing PMU experiments you don't care if only one or all domains have
access: you are working in a controlled environment trying to figure out
if your setup meets the timing requirements. There are no security or
safety concerns (those are different experiments) and there is no
interference problems in the sense of multiple domains trying to access
PMU at the same time -- you control the domains you decide which one is
accessing them.

So I am in favor of a patch like this one because it actually satisfy
our requirements. Even if we had per-domain support and interrupts
support, I don't think they would end up being used.
--8323329-93430960-1630518639=:17925--


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 18:11:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 18:11:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176607.321366 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLUhk-0003B3-Bx; Wed, 01 Sep 2021 18:11:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176607.321366; Wed, 01 Sep 2021 18:11:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLUhk-0003Aw-8o; Wed, 01 Sep 2021 18:11:20 +0000
Received: by outflank-mailman (input) for mailman id 176607;
 Wed, 01 Sep 2021 18:11:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLUhi-0003AX-W8
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 18:11:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLUhi-0008Fy-NV; Wed, 01 Sep 2021 18:11:18 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLUhi-0000v1-HK; Wed, 01 Sep 2021 18:11:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=icnWcTlQp/oE8mUZ05u+b5+ihbagLiHOfp5+wRHA+Gc=; b=ZXdspewTVPJE5qslYe9D6qFmL2
	lv3B7YG6jLU5MkHS2IRcnpiiL6EVEEvykaBV6Mu+ns+7kg2AsTdaLsbalT5KF8OdiT/dVcJchCVBy
	LrA48cP23cRl/urp4BWc6GQOOrYUTq6bifGG/k6xxKNq3kJ9nXABAqL9SH6dNMTp9SDo=;
Subject: Re: vcpu_show_execution_state() difference between Arm and x86
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <8f295eeb-e8f2-a115-3f83-60922d0fe5f9@xen.org>
Date: Wed, 1 Sep 2021 19:11:15 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 01/09/2021 14:39, Jan Beulich wrote:
> All,
> 
> back in 2016 Andrew added code to x86'es variant to avoid interleaving
> of output. The same issue ought to exist on Arm.

Agree. I guess we got away so far because it is pretty rare to have two 
CPUs running at the same.

> The lock acquired,
> or more importantly the turning off of IRQs while doing so, is now
> getting in the way of having PVH Dom0's state dumped the 2nd time. 

I am not quite too sure to understand the problem with PVH dom0. Do you 
have a pointer to the issue?

> For
> register state I did find a sufficiently simple (yet not pretty)
> workaround. For the stack, where I can't reasonably avoid using p2m
> functions, this is going to be more difficult. >
> Since I expect Arm to want to also have interleave protection at some
> point, and since Arm also acquires the p2m lock while accessing Dom0's
> stacks, I wonder whether anyone has any clever idea on how to avoid
> the (valid) triggering of check_lock()'s assertion without intrusive
> changes. (As to intrusive changes - acquiring the p2m lock up front in
> recursive mode, plus silencing check_lock() for nested acquires of a
> lock that's already being held by a CPU was my initial idea.)

At least one Arm, the P2M lock is a rwlock which is not yet recursive. 
But then it feels to me that this solution is only going to cause us 
more trouble in the future.

I looked at the original commit to find out the reason to use the 
console lock. AFAICT, this was to allow console_force_unlock() to work 
properly. But it is not entirely clear why we couldn't get a new lock 
(with IRQ enabled) that could be forced unlocked in that function.

Can either you or Andrew clarify it?

The other solution I can think off is buffering the output for 
show_registers and only print it once at the end. The downside is we may 
not get any output if there is an issue in the middle of the dump.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 18:31:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 18:31:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176616.321377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLV0j-00060G-8T; Wed, 01 Sep 2021 18:30:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176616.321377; Wed, 01 Sep 2021 18:30:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLV0j-000609-5Y; Wed, 01 Sep 2021 18:30:57 +0000
Received: by outflank-mailman (input) for mailman id 176616;
 Wed, 01 Sep 2021 18:30:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B+dc=NX=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLV0h-000603-Ve
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 18:30:55 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d05f4a3-0a22-45c4-9358-3b17bc4240e3;
 Wed, 01 Sep 2021 18:30:54 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0DE5461090;
 Wed,  1 Sep 2021 18:30:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d05f4a3-0a22-45c4-9358-3b17bc4240e3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630521054;
	bh=PI67GhDQWWeYIukosO0exq6kkx/XCvPBL+4kl7aUjaQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=j+oevX6Pi+xFWQOjT69dJfBnwrbu/z0PbpQySvHDFCre2oH+paWL4Fe7TLqvN6UuF
	 f1Od+l5IlLHWQMkRfeg/M2/Lti0gNJAMEcsjQkplFAy2kKkjhHrJj6AIAdUx1fGkLa
	 p7gCUYYALULKL0cFZH0GP2Ax/nAvtxSNsVLPTYRrQXPJj8gLHiT4IfIMMl1IZZiBz+
	 VxYqdYrmSu61YiwZajWJEE5wxt3RdAB3uVcT1Bme9O8Fp8O3RQZsp6/w9rYOT+CoDq
	 t2neVELFFfn4QgVyl64mu7W2Pbq68qFp19Z5su2+cL56cDlcovkhuJy4c3+cwK1buN
	 m7LEcmGY0mADA==
Date: Wed, 1 Sep 2021 11:30:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <wei.chen@arm.com>, 
    xen-devel@lists.xenproject.org, jbeulich@suse.com, 
    Bertrand.Marquis@arm.com
Subject: Re: [XEN RFC PATCH 25/40] xen/arm: unified entry to parse all NUMA
 data from device tree
In-Reply-To: <088e4b16-afea-8248-55f1-a081bba7f80c@xen.org>
Message-ID: <alpine.DEB.2.21.2109011129050.17925@sstabellini-ThinkPad-T480s>
References: <20210811102423.28908-1-wei.chen@arm.com> <20210811102423.28908-26-wei.chen@arm.com> <alpine.DEB.2.21.2108301752501.17851@sstabellini-ThinkPad-T480s> <088e4b16-afea-8248-55f1-a081bba7f80c@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 31 Aug 2021, Julien Grall wrote:
> Hi Stefano,
> 
> On 31/08/2021 01:54, Stefano Stabellini wrote:
> > On Wed, 11 Aug 2021, Wei Chen wrote:
> > > In this API, we scan whole device tree to parse CPU node id, memory
> > > node id and distance-map. Though early_scan_node will invoke has a
> > > handler to process memory nodes. If we want to parse memory node id
> > > in this handler, we have to embeded NUMA parse code in this handler.
> > > But we still need to scan whole device tree to find CPU NUMA id and
> > > distance-map. In this case, we include memory NUMA id parse in this
> > > API too. Another benefit is that we have a unique entry for device
> > > tree NUMA data parse.
> > > 
> > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > ---
> > >   xen/arch/arm/numa_device_tree.c | 31 ++++++++++++++++++++++++++++---
> > >   xen/include/asm-arm/numa.h      |  1 +
> > >   2 files changed, 29 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/xen/arch/arm/numa_device_tree.c
> > > b/xen/arch/arm/numa_device_tree.c
> > > index 6e0d1d3d9f..27ffb72f7b 100644
> > > --- a/xen/arch/arm/numa_device_tree.c
> > > +++ b/xen/arch/arm/numa_device_tree.c
> > > @@ -131,7 +131,8 @@ save_memblk:
> > >   }
> > >     /* Parse CPU NUMA node info */
> > > -int __init device_tree_parse_numa_cpu_node(const void *fdt, int node)
> > > +static int __init
> > > +device_tree_parse_numa_cpu_node(const void *fdt, int node)
> > >   {
> > >       uint32_t nid;
> > >   @@ -147,7 +148,7 @@ int __init device_tree_parse_numa_cpu_node(const
> > > void *fdt, int node)
> > >   }
> > >     /* Parse memory node NUMA info */
> > > -int __init
> > > +static int __init
> > >   device_tree_parse_numa_memory_node(const void *fdt, int node,
> > >       const char *name, uint32_t addr_cells, uint32_t size_cells)
> > >   {
> > > @@ -202,7 +203,7 @@ device_tree_parse_numa_memory_node(const void *fdt,
> > > int node,
> > >   }
> > >     /* Parse NUMA distance map v1 */
> > > -int __init
> > > +static int __init
> > >   device_tree_parse_numa_distance_map_v1(const void *fdt, int node)
> > >   {
> > >       const struct fdt_property *prop;
> > > @@ -267,3 +268,27 @@ device_tree_parse_numa_distance_map_v1(const void
> > > *fdt, int node)
> > >         return 0;
> > >   }
> > > +
> > > +static int __init fdt_scan_numa_nodes(const void *fdt,
> > > +                int node, const char *uname, int depth,
> > > +                u32 address_cells, u32 size_cells, void *data)
> > > +{
> > > +    int ret = 0;
> > > +
> > > +    if ( fdt_node_check_type(fdt, node, "cpu") == 0 )
> > > +        ret = device_tree_parse_numa_cpu_node(fdt, node);
> > > +    else if ( fdt_node_check_type(fdt, node, "memory") == 0 )
> > > +        ret = device_tree_parse_numa_memory_node(fdt, node, uname,
> > > +                                address_cells, size_cells);
> > > +    else if ( fdt_node_check_compatible(fdt, node,
> > > +                                "numa-distance-map-v1") == 0 )
> > > +        ret = device_tree_parse_numa_distance_map_v1(fdt, node);
> > > +
> > > +    return ret;
> > > +}
> > 
> > Julien, do you have an opinion on whether it might be worth reusing the
> > existing early_scan_node function for this to avoiding another full FDT
> > scan (to avoid another call to device_tree_for_each_node)?
> 
> I don't like the full FDT scan and actually drafted an e-mail in reply-to [1]
> to suggest parse all the NUMA information from early_scan_node().
> 
> However, we don't know whether ACPI or DT will be used at the time
> early_scan_node() is called. So we will need to revert any change which can
> make the code a little more awkward.
> 
> So I decided to drop my e-mail because I prefer the full DT scan for now. We
> can look at optimizing later if this becomes a pain point.

Uhm, yes you are right.

We would have to move some of the logic to detect ACPI earlier to
early_scan_node (e.g. xen/arch/arm/acpi/boot.c:dt_scan_depth1_nodes).
That could actually be a good idea, but it is true that could be done
with a separate patch later.


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 18:33:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 18:33:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176622.321388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLV3F-0006ry-Nm; Wed, 01 Sep 2021 18:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176622.321388; Wed, 01 Sep 2021 18:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLV3F-0006rr-Ka; Wed, 01 Sep 2021 18:33:33 +0000
Received: by outflank-mailman (input) for mailman id 176622;
 Wed, 01 Sep 2021 18:33:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLV3F-0006rl-Ay
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 18:33:33 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLV3B-00009l-De; Wed, 01 Sep 2021 18:33:29 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLV3B-0003Hv-6u; Wed, 01 Sep 2021 18:33:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=9PGx/b8tr+VJ14PQ7FQDRJ8vRtnsMyRlQBn9zoS8bA4=; b=ZzVDPPSiIxDnYRWqoLlpuisL1m
	0Hhm4oREb5mZoJY/QZ8yfX7XSKUi2rarC3kiuCy0wkrs5uoL/TqofHmiDg1IY0mQtubIDX/7r3kDg
	GQTpBUnFNkarj0KA2Z1jrWwj68Cdv/BJ0VyYZmglcxANZU9pKxAQI5aab4lRFV6jp9LE=;
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Michal Orzel <Michal.Orzel@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
 <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
 <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <0e72fb05-a477-ef79-c26f-bc011393a403@xen.org>
Date: Wed, 1 Sep 2021 19:33:26 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Stefano,

On 01/09/2021 18:54, Stefano Stabellini wrote:
> On Wed, 1 Sep 2021, Julien Grall wrote:
>> On 01/09/2021 14:10, Bertrand Marquis wrote:
>>>> On 1 Sep 2021, at 13:55, Julien Grall <julien@xen.org> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 01/09/2021 13:43, Michal Orzel wrote:
>>>>> Introduce new Xen command line parameter called "pmu_access".
>>>>> The default value is "trap": Xen traps PMU accesses.
>>>>> In case of setting pmu_access to "native", Xen does not trap
>>>>> PMU accesses allowing all the guests to access PMU registers.
>>>>> However, guests cannot make use of PMU overflow interrupts as
>>>>> PMU uses PPI which Xen cannot route to guests.
>>>>> This option is only intended for development and testing purposes.
>>>>> Do not use this in production system.
>>>> I am afraid your option is not safe even in development system as a vCPU
>>>> may move between pCPUs.
>>>>
>>>> However, even if we restricted the use to pinned vCPU *and* dedicated
>>>> pCPU, I am not convinced that exposing an half backed PMU (the overflow
>>>> interrupt would not work) to the guest is the right solution. This likely
>>>> means the guest OS would need to be modified and therefore the usage of
>>>> this option is fairly limited.
>>>>
>>>> So I think the first steps are:
>>>>    1) Make the PPI work. There was some attempt in the past for it on
>>>> xen-devel. You could have a look.
>>>>    2) Provide PMU bindings
>>>>
>>>> With that in place, we can discuss how to expose the PMU even if it is
>>>> unsafe in some conditions.
>>>
>>> With those limitations, using the PMU to monitor the system performances or
>>> on some specific use cases is still really useful.
>>> We are using that to do some benchmarks of Xen or of some applications to
>>> compare the behaviour to a native system or
>>> analyse the performances of Xen itself (hypercalls,context switch …etc)
> 
> I also already had to write a patch almost exactly like this one to
> provide to customers a few months back.
> 
> 
>> I understand this is useful for some setup and I am not trying to say we
>> should not have a way to expose the PMU (even unsafely) in upstream. However,
>> I think the option as it stands is too wide (this should be a per domain knob)
>> and we should properly expose the PMU (interrupts, bindings...).
> 
> I have never used PMU directly myself, only provided a patch similar to
> this one.  But as far as I could tell the users were fully satisfied
> with it and it had no interrupts support either. Could it be that
> interrupts are not actually needed to read the perf counters, which is
> probably what users care about?

You don't need the interrupts to read the perf counters. But AFAIU, you 
would have to poll at a regular interval yourself. There is also the 
question on how to catch the overflow?

> 
> In regards to "this should be a per domain knob", in reality if you are
> doing PMU experiments you don't care if only one or all domains have
> access: you are working in a controlled environment trying to figure out
> if your setup meets the timing requirements. There are no security or
> safety concerns (those are different experiments) and there is no
> interference problems in the sense of multiple domains trying to access
> PMU at the same time -- you control the domains you decide which one is
> accessing them. >
> So I am in favor of a patch like this one because it actually satisfy
> our requirements. Even if we had per-domain support and interrupts
> support, I don't think they would end up being used.

I have to disagree with that. There are valid use-cases where you may 
want to expose the PMU for using perf to a single domain because you 
know it is safe to do so. I appreciate this is may not be the use case 
for your users (yet), but I have seen (and used) it on x86. So to me 
this approach is short-sighed.

TBH, this is not the first time I have seen patch for "let's expose the 
same feature to everyone because this is easy to do" and really dislike 
it. Exposing a new feature from the toolstack is easier than you think 
(baring the lack of reviews), this is a matter of creating a new flag a 
new option. This would make Xen a lot more flexible and enable more users...

So as it stands:

NAcked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 19:18:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 19:18:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176628.321399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLVk0-0003bF-5R; Wed, 01 Sep 2021 19:17:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176628.321399; Wed, 01 Sep 2021 19:17:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLVk0-0003b8-25; Wed, 01 Sep 2021 19:17:44 +0000
Received: by outflank-mailman (input) for mailman id 176628;
 Wed, 01 Sep 2021 19:17:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLVjy-0003ay-R8; Wed, 01 Sep 2021 19:17:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLVjy-0000td-KC; Wed, 01 Sep 2021 19:17:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLVjy-0004yL-8e; Wed, 01 Sep 2021 19:17:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLVjy-0000Pe-87; Wed, 01 Sep 2021 19:17:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2kijOnpQXrHSVAla8qAxcva3/OA/I7JNm8i02Ah8h5U=; b=gSc+Epg67JtWdtzwTWQRch/hPQ
	pc1bNZZZkGuL42qVL9jCo/chm2krBXZy75IJDMfEHIWXkts7AhWu8iRiqikj3hROWFYY/j+6vYB1U
	MbyyExvuHvUL35NJWu0AK58yKbHT8W+0mUnIHk9z6BLBtzh9asROX6+FdGocJi6c6z3U=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164687-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164687: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
X-Osstest-Versions-That:
    xen=a931e8e64af07bd333a31f3b71a3f8f3e7910857
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 19:17:42 +0000

flight 164687 xen-unstable real [real]
flight 164724 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164687/
http://logs.test-lab.xenproject.org/osstest/logs/164724/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   8 xen-boot            fail pass in 164724-retest

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2 15 migrate-support-check fail in 164724 never pass
 test-arm64-arm64-xl-credit2 16 saverestore-support-check fail in 164724 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164477
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164477
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164477
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164477
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164477
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164477
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164477
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164477
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164477
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164477
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164477
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618
baseline version:
 xen                  a931e8e64af07bd333a31f3b71a3f8f3e7910857

Last test of basis   164477  2021-08-25 09:09:48 Z    7 days
Failing since        164499  2021-08-26 10:21:51 Z    6 days    5 attempts
Testing same since   164687  2021-09-01 03:44:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Fabrice Fontaine <fontaine.fabrice@gmail.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Julien Grall <julien@xen.org>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a931e8e64a..96607a8e68  96607a8e680e7f965ca868d11f8b0636317d2618 -> master


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 20:02:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 20:02:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176663.321504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLWRK-0002LJ-T2; Wed, 01 Sep 2021 20:02:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176663.321504; Wed, 01 Sep 2021 20:02:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLWRK-0002LC-Ps; Wed, 01 Sep 2021 20:02:30 +0000
Received: by outflank-mailman (input) for mailman id 176663;
 Wed, 01 Sep 2021 20:02:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rqR/=NX=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mLWRK-0002L6-3F
 for xen-devel@lists.xenproject.org; Wed, 01 Sep 2021 20:02:30 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 60c7c1e6-1de6-4860-9bd0-4cb4b60bdd2b;
 Wed, 01 Sep 2021 20:02:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60c7c1e6-1de6-4860-9bd0-4cb4b60bdd2b
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630526548;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=EmmyNtx9gJnqTuQ1T0C4F0cNpIZQvuM75YDFAWagy74=;
  b=Si3uCYNbLsJIGNACwi/8aOVJ29ocU+luRPuncqpSYgdwdzucxmdZj/sK
   mYfpIVhIKTWiKe4S8wwx3u4FGYmWnapbvgSUglJLFHuQfkB63J6RgiB2t
   syHTm0MYIjxuA4eo0d5JN9uBSuzmXCpPhCJnpkoyvlqDfVzp1kep5vKSH
   4=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: YrbDmm78MqY8HkOXyNnxTm+XUC472m1zU8p0jm4KvCFkX1zPUjZeThCktDYOs/C+ml7jLgQR1A
 7yQpHbEhU5Yp9kF7fRJ87eQT3Xph+ClSDmH4Tz9kiZLf3eG4mSzIoOoWgc4w2YYyzpMJ3VPIkw
 bHki+XWuna1xmI6SkNUINki52x1lTnlHoCO/OS2xl1AVh0PSLdPXp9fOb3CYwhUPlGJUtil/wL
 Q2RrilOEq4EwW20IP7FwxC5tB0fVCBV60mGGhP4B4/NzrCX9CViUp5QjOeYG4zR5FFVkgE7cRc
 FxoG8OhLoShTmGIwmvPRQ3wM
X-SBRS: 5.1
X-MesageID: 53580309
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:BTEvT6Ah5Vrc5grlHegisceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LW90dq7MAzhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSuwEIdxeOkdK1kJ
 0QCZSWa+eAfWSS7/yKmTVQeuxIqLLskNHK9JXjJjVWPGVXgslbnnZE422gYytLrWd9dPgE/d
 anl7F6T23KQwVoUi33PAhLY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX252oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iDnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDg4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWApqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocUTbqjVQGcgoBT+q3qYpxqdS32BHTq+/blkwS+pUoJinfxn6ck7y49HPtXceg22w
 zGWp4Y342mAPVmNZ6UqY86ML2K41f2MGbx2VSpUBza/ZE8SgfwQqHMkcIIDcGRCdE1JcgJ6d
 j8uG0xjx96R6upM7zU4KF2
X-IronPort-AV: E=Sophos;i="5.84,370,1620705600"; 
   d="scan'208";a="53580309"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GjRT/FjPZS6CbnMueP2+d7fUFHy52bNwYgpVFML6OfLwlQkXDKtgJRNdq7KD9esG41lTSrySb87zFwgkPV1NhBvnKMssh4cZtBAgvUJnl1/DKyjOq+w/p/ubwU6tjVleRuWROxpaxYaxWwfvGsNliWhFfYS8vhEYyjEDcFDZ59ycasUEw3lVeuZ3bTtGSqCsWR+Upm1dGz62uukCwxSs9tnYphrkzRENyS0ufqzY+L3aRde0vIpkdzE7kSD08kMShi0zxoT11qIPWEoqWKHS3todBuJddZDtbztDezCsvFyMoubRy8n3u5RKxMz9NVU+HBwtX7DOuH4rGayZDhP01g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EmmyNtx9gJnqTuQ1T0C4F0cNpIZQvuM75YDFAWagy74=;
 b=aD2Sg7++GJQTZwTsM4iC9gsTOzwApxlxHZbZ1Yl5I0yq/11bGYSBr2M44A2QtAlAvX5AZLO5REcSK6PL+f/lC5CEGiiwuElc3i2elI2Bg0qXVkN/OrujecZJlxMSvksqxnenhyxeTDxNruDahkkXfKsWW9mTrpDKqCfkf+D80Z0DOVcjb96XaKylRnQ1Ko+hvyvRXeMzo0xebY0y6VBvZBlCLL0JxteU3nlL3CWienjDj8Mo8hhonQLUAQUUQgCrFq1KsXZjTq8ohKp7Ed+X2LjVtTRFZf6IK/D9wr7as3Lff2vWrI3Oo6tLydH5bWecjPPhuNpvLag7wZJd732mig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EmmyNtx9gJnqTuQ1T0C4F0cNpIZQvuM75YDFAWagy74=;
 b=Bn8Tk0RD/41282DH7wnfZcMQ+GxVzjHOUet/4u8X9GpiABoYNE8MvQ0uS16L58cUTICfJ9F38BIPWVwlwf78pD3Y0OpGmGJ1ZBsZuK7WBKY6IElC+qD2bogyLZvDSbr1BI0F5ek7wcdkh7jkA0H0QB1frXRc8OaZn/jNEEryCG4=
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
Message-ID: <601d46e2-5c08-3948-85bc-1e027358700b@citrix.com>
Date: Wed, 1 Sep 2021 21:02:16 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0083.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::16) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2fd9dd0f-a1c7-4afd-5bb3-08d96d836989
X-MS-TrafficTypeDiagnostic: BYAPR03MB3560:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3560E9A541B255AF7FDAC1E1BACD9@BYAPR03MB3560.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: MexvoXncCsIbznvqnfacMJ7V6q0akAtzYRoZ0TBJpARbps1x31yYA4hVOrP69lktsPTp5UiVj8SjAoRN2zzf1pfVXyybObhjFLiueKYaQiFKYjlWo8kSTfTC0ItNxizV2P7N+SBc++x2IDc2pkrdX2mVwPK15bmL9SOuaD5eHXUZCYT1r4YWDZ18SX2vamR4wbm36cUJWpZEgTOe5QxYQGZFXHsxIDVTD9l2xDABHQVmu58ZRNLBt/QlI0tyq3WVrm9U6RTskGSj+qQrHsPErLc+dK6zCyPQzt+I+8PkFO3ty8pTI+DL9EuKMx0ajGWQYRX80Q2YDH41boobE/Z9cWsbAAwsYcXemMhcfxB7LRDMomP4KyMwq7CMeWnQ0ddXf9ZSf1L0EExxJVpkiGmfq/s7EabyQsXhoGKGoLiCC4Ze4Sp5Qf7OT/xJyi1pFLnDqsN2yLOqBydc+YW44EzVHnE4ZHqDBI1Sro7tUYImZkbAFRPrcuFNctmtc/xAWsSoaRyqRn853zjfazBuTxtR/OKhZvmz0c+f90Yv4VrNv07ahlLFXjpvoVpGZJx7Qihn/KVgYz3Ge8K4ZKFCq54TqYM8nf0el/S6kNIXs5mqCzZKfhK5OZO5WVnzmipucuNmewC3BcfwSF9MxyCQms+Gz4fJayJ2XX9xnJwo+eohgODrTJ0fr3up3aXeve3Qys8tqD6TvKlaEd0ulzyn4Auuw9YZDy23sJUrTCti7wtTWfg=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(4326008)(55236004)(38100700002)(16576012)(54906003)(53546011)(316002)(8676002)(186003)(6666004)(86362001)(6486002)(2616005)(36756003)(2906002)(26005)(110136005)(107886003)(66946007)(31686004)(8936002)(31696002)(5660300002)(66476007)(66556008)(956004)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RElSajFKZHF6cERuWDAyTlF5QndWbFRKNWdoV2VoaXdOa0pWOGRJTmpEZGRD?=
 =?utf-8?B?MUNBVitpTWdHeXplM3c5TmhiRjVQNUU4ak04STFaMDRhN0pGa2JqaERYUlRB?=
 =?utf-8?B?MnN6UmxwVzhKU3BXdzZhRStsSzZOZmdORmQvaVl6TmFHTWVaanFWM2tWOWx1?=
 =?utf-8?B?em9kWTc1akpKYndrNjE2NkNwN1hsQjhXM0hJdGg0d2dZSWRMWUtadzBjWUNK?=
 =?utf-8?B?cVc5d3FETldOeVo5U2JFc3NYYTVOdW16a3U0T1Jzc0pHOURsVEtsOHJJeXRv?=
 =?utf-8?B?aUpLUEJlaERMa204L0NxWXdaeWs5WUdCRDhXbHRzSGsxeWhGZHRpMWV4VmJx?=
 =?utf-8?B?QXdLejBBZEhmOHZDR0s5czR0SjY1ZGRSOXVjamUzZXptdzhTZlFOblpOU3c2?=
 =?utf-8?B?RzdWYSs5TlBReWNEWGZscEt4eTFSN1JtRXNHQlo0SCtrQmdIQ0JscDQ2VmpJ?=
 =?utf-8?B?Z2dxYTdoRUVNaGxwenMrbUpia2pzN2pWOGtHRElkWStXWmpXMmNPbU5ZSnNa?=
 =?utf-8?B?THNGcHlJSVlMdFlJVFNLU3FSZDJPbDdrQXFXZWgwUmpEdGJOTEtNRTJiWjRq?=
 =?utf-8?B?UGl6a3FmbE5raEppZFEvTzMzZ0MweDZiOVdQWjgzUVhORUsybVFiazVwcTJK?=
 =?utf-8?B?bzM0WTBCNitYNkFWdDlQZ0pmNk9rVWxxajBrdlNlNWRnb2xrTjJpeVdkbUFV?=
 =?utf-8?B?eWp3bFNRdnp5RVFZVWJQeWZuMUhqWlM4cEh5VDJ3SDRxcnd2M2FIUG91bnF1?=
 =?utf-8?B?TkxENlM3TEt5SmRVTWRHdGZFcSs0cHhNd1B0aFc3ZmJCR01Bb3lEenYzYXNj?=
 =?utf-8?B?bk5tR2lxYy9IVUdUTFRBS0pQbkZ6MlRLMlBIT0hZa2xzUnVDdGo3UkkrWGdm?=
 =?utf-8?B?UFk5aFFHMkJMNmg2ZUZvSitDWnVjNUN6OWV1d3phSExaK3JtUmlVY3JVUEhl?=
 =?utf-8?B?YTRZRnQzRjhpc0FuNGtyZXFGemF5U1RiUVV3OTVSYm9GRmhCOUpJb2owT1FR?=
 =?utf-8?B?NS9xS2lqQkljVk9ZVS91NFNjTjR6VFRiblpSNUdwSEpSVVppTjhiaktDZksw?=
 =?utf-8?B?ckN3RFFscXpwMjFSSjltOWhTL2pqS3NVdGo1V0pXSHRBMzFGakxWcnpjWG1n?=
 =?utf-8?B?dHpUWGY4Ujh2SWFlWm1heVloL0VLY2JYSGtJbFAvaThQTnA5SUhMT1l2RHFX?=
 =?utf-8?B?SDlnZXpwemJseXV0bjNhcWVXUGM4VkJoVTZUeHJ5YnhhZUtNSnhjQTUxM0lS?=
 =?utf-8?B?amQ2cy9KUkFVbmR1dmJ3djVBS09xZVhIa0FQSm1sTWFvNlM0b2NHanZVUFRS?=
 =?utf-8?B?NGdlRVA2MVg2cTBMVG9ZWGNueU8razM0bE1yK2VaQ1c3NXhPL2hBdXB4NXdv?=
 =?utf-8?B?WDRpbTBkUDZVbTVuUllUQS82Z2NTTWJTOFduSXVlakN0THN5Q3VKeHRPOFhm?=
 =?utf-8?B?SjdPQUxHb29vWjhOOExyTDE2QWFsUUJqaldOdjVTazRVLzlTSDltbnFoL3dK?=
 =?utf-8?B?SFVJR0dxUlVzT00zYkJObWsxbXIzR1Bqb05BZCtYUGNxSHdxbzlOb01zTUkx?=
 =?utf-8?B?Y0UydjNZbjRuNlY0TnAzOWJ0eVBMMDU5MlVCWlJlU1lGVlQ4L3VBWFFLY21E?=
 =?utf-8?B?ZU5wTE5zYUNWaFFsRHdidkpnZGhWTnVGL2hPNWFWQ21HREg2WWcxLzZvRmV2?=
 =?utf-8?B?MmFtalhNZjNFMkRZUnJVUktYSTdQMzBicFpPOXA0dHh5VGlYM1hkM0xiQWhS?=
 =?utf-8?Q?QpHJh98xseG0YFX84jRkBuuEue3pW2CBFyrHzuE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2fd9dd0f-a1c7-4afd-5bb3-08d96d836989
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 20:02:23.1017
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Tm+4ouc25JYwTS+wF78Pk4NeRlK+veLwaKi2VH+tTQXoS6vJ4KR1nvYbssoElopkLryFSnf8CWq5zVjJBquw7DhbEVlkrSewzgjZzgy/qhM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3560
X-OriginatorOrg: citrix.com

On 01/09/2021 17:06, Jan Beulich wrote:
> The function may fail; it is not correct to indicate "success" in this
> case up the call stack. Mark the function must-check to prove all
> cases have been caught (and no new ones will get introduced).
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> In the grant-transfer case it is not really clear to me whether we can
> stick to setting GTF_transfer_completed in the error case. Since a guest
> may spin-wait for the flag to become set, simply not setting the flag is
> not an option either. I was wondering whether we may want to slightly
> alter (extend) the ABI and allow for a GTF_transfer_committed ->
> GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
> at the same time as setting GTF_transfer_completed).

Considering there are no production users of gnttab_transfer(), we can
do what we want.=C2=A0 It was introduced for (IIRC) netlink2 and never got
into production, and then we clobbered it almost entirely in an XSA
several years ago by restricting steal_page() to PV guests only.

As a consequence, we can do anything which seems sensible, and does not
necessarily need to be bound by a guest spinning on the bit.

The concept of gnttab_transfer() alone is crazy from an in-guest memory
management point of view.=C2=A0 We could alternatively save our future selv=
es
more trouble by just Kconfig'ing it out now, deleting it in several
releases time, and fogetting about the problem as nothing will break in
practice.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 01 21:19:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 21:19:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176672.321516 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLXd5-00026V-Ed; Wed, 01 Sep 2021 21:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176672.321516; Wed, 01 Sep 2021 21:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLXd5-00026O-BA; Wed, 01 Sep 2021 21:18:43 +0000
Received: by outflank-mailman (input) for mailman id 176672;
 Wed, 01 Sep 2021 21:18:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLXd4-00026E-Pj; Wed, 01 Sep 2021 21:18:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLXd4-0003CS-B3; Wed, 01 Sep 2021 21:18:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLXd3-0002LW-Uc; Wed, 01 Sep 2021 21:18:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLXd3-0004CE-U9; Wed, 01 Sep 2021 21:18:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pkrosh2zFTxkIRnLKSgp7UDnqR7KcXQp9OY/znICQw4=; b=ZCvhCg09ivZi9hTkd+Wh67YhAu
	o3uXpMGLHwq7CesgFezbk2qnJv02VpTSclHT0BGWnfz3yZfVAsVFeWcAygUVfosMl8T6x2CXs/jHP
	6sCMrO0BT+j/ZKyIcEGdUpHUtE2suMaOg/aO5OPqQueepJD+3P6vyigtzS26InJ25ljA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164690-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164690: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=d52dff5d8048d4982437db9606c27bb4127cf9d0
X-Osstest-Versions-That:
    qemuu=ad22d0583300df420819e6c89b1c022b998fac8a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 21:18:41 +0000

flight 164690 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164690/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164601
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164601
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164601
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164601
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164601
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164601
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164601
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                d52dff5d8048d4982437db9606c27bb4127cf9d0
baseline version:
 qemuu                ad22d0583300df420819e6c89b1c022b998fac8a

Last test of basis   164601  2021-08-29 11:06:48 Z    3 days
Testing same since   164690  2021-09-01 08:37:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   ad22d05833..d52dff5d80  d52dff5d8048d4982437db9606c27bb4127cf9d0 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 23:08:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 23:08:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176684.321542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLZL0-0006Hb-Lg; Wed, 01 Sep 2021 23:08:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176684.321542; Wed, 01 Sep 2021 23:08:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLZL0-0006HU-Hs; Wed, 01 Sep 2021 23:08:10 +0000
Received: by outflank-mailman (input) for mailman id 176684;
 Wed, 01 Sep 2021 23:08:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLZKy-0006HK-Jm; Wed, 01 Sep 2021 23:08:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLZKy-0005Bt-DU; Wed, 01 Sep 2021 23:08:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLZKy-0002ZE-4t; Wed, 01 Sep 2021 23:08:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLZKy-0007au-4L; Wed, 01 Sep 2021 23:08:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=I753QPrczn9QkuyxqPxdWPBn6WhmIc/0JnZFdYDjdVo=; b=UTIX8gcZwPEBKG/eZbxXSYQQVa
	G1EEcSal6ZpqQJ3o5rYst7JJqSwlE3GMmF8NT/5h8/N2hkKrYbqLDnVR8rnhWDRqLW28ihK53VQ/I
	inFjYf5LQALUUjV2UmA3m7ufCq9hYRAD4AcDMTW5srAXAJ5qX5JFraDhA4AJnB3F6/Ek=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164718-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164718: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=12e33dca4c0612a0975265e5ba641c6261a26455
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 23:08:08 +0000

flight 164718 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164718/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 12e33dca4c0612a0975265e5ba641c6261a26455
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    4 attempts
Testing same since   164718  2021-09-01 16:40:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 01 23:56:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 01 Sep 2021 23:56:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176695.321562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLa5a-0003rS-Jc; Wed, 01 Sep 2021 23:56:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176695.321562; Wed, 01 Sep 2021 23:56:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLa5a-0003rL-GV; Wed, 01 Sep 2021 23:56:18 +0000
Received: by outflank-mailman (input) for mailman id 176695;
 Wed, 01 Sep 2021 23:56:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLa5Z-0003rB-Gy; Wed, 01 Sep 2021 23:56:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLa5Z-0005zH-4g; Wed, 01 Sep 2021 23:56:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLa5Y-0004Na-RF; Wed, 01 Sep 2021 23:56:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLa5Y-0005qk-Qi; Wed, 01 Sep 2021 23:56:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KhrTjozmEmC9DKBQdIbpOY0sMTdu47t2eezytA/Jc78=; b=fP5xV+faZgfes4WV95aVIdIwgR
	aCRugFIIp/KOy5Qb8OQdto41t9bh/BpHhRXcJkChpHyjzQJsoKGqYKMQcixILadk4CcXWijRi3DGl
	1OkP+xzreGLLbSR2cSVO1yczstfaLIRaOyY7zWS8e53tiiN/ziGcbXHqbClTqiYW12h0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164709-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164709: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-amd64-pvops:kernel-build:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start:fail:regression
    linux-linus:build-i386-pvops:kernel-build:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-raw:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-freebsd10-i386:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-freebsd10-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-i386-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-pair:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-pvshim:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=9e9fb7655ed585da8f468e29221f0ba194a5f613
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 01 Sep 2021 23:56:16 +0000

flight 164709 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164709/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-pvops             6 kernel-build             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 14 guest-start              fail REGR. vs. 152332
 build-i386-pvops              6 kernel-build             fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-raw        1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-shadow     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-xsm        1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)               blocked  n/a
 test-amd64-i386-examine       1 build-check(1)               blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-examine      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-amd64-i386-pair          1 build-check(1)               blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-xl            1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-pvshim     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)              blocked n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                9e9fb7655ed585da8f468e29221f0ba194a5f613
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  397 days
Failing since        152366  2020-08-01 20:49:34 Z  396 days  685 attempts
Testing same since   164689  2021-09-01 04:20:58 Z    0 days    2 attempts

------------------------------------------------------------
6967 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            fail    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             fail    
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           blocked 
 test-amd64-coresched-i386-xl                                 blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       blocked 
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-i386-qemut-rhel6hvm-amd                           blocked 
 test-amd64-i386-qemuu-rhel6hvm-amd                           blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemut-debianhvm-amd64                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     blocked 
 test-amd64-i386-freebsd10-amd64                              blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-i386-xl-qemut-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-i386-xl-qemuu-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemut-ws16-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ws16-amd64                          blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         blocked 
 test-amd64-amd64-examine                                     blocked 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      blocked 
 test-amd64-i386-freebsd10-i386                               blocked 
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-i386-qemut-rhel6hvm-intel                         blocked 
 test-amd64-i386-qemuu-rhel6hvm-intel                         blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-i386-pair                                         blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-i386-xl-pvshim                                    blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       blocked 
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-amd64-i386-xl-shadow                                    blocked 
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2103251 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:14:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:14:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176704.321582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaMm-0007Da-Ix; Thu, 02 Sep 2021 00:14:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176704.321582; Thu, 02 Sep 2021 00:14:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaMm-0007DT-FM; Thu, 02 Sep 2021 00:14:04 +0000
Received: by outflank-mailman (input) for mailman id 176704;
 Thu, 02 Sep 2021 00:14:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLaMk-0007DJ-De; Thu, 02 Sep 2021 00:14:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLaMk-0006tL-8u; Thu, 02 Sep 2021 00:14:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLaMk-00051T-0d; Thu, 02 Sep 2021 00:14:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLaMk-0005T3-0A; Thu, 02 Sep 2021 00:14:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LqEoeYbL2qClYzMcBg6XPcebzs9wV6/lc4/p47SYp34=; b=VtpGTC7eLxWD7ueRFMVmJIZMHS
	KC80bqX1SpEuodAyFJ9pRLTOIjzIs5OzBkxxYV23xVja4qv11s+C/UPq0RxhQbLKXFkk3RY2MnC/J
	fHZlRaqpXbOzqELydJ3jpGcNn5/kzBSGSpJu/eJ+egktXPVEzonydFRW1lQ3Jz2znO2c=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164728-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164728: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=12e33dca4c0612a0975265e5ba641c6261a26455
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 00:14:02 +0000

flight 164728 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164728/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 12e33dca4c0612a0975265e5ba641c6261a26455
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    5 attempts
Testing same since   164718  2021-09-01 16:40:02 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:15:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:15:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176712.321596 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaNs-0007qQ-6D; Thu, 02 Sep 2021 00:15:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176712.321596; Thu, 02 Sep 2021 00:15:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaNs-0007qJ-2y; Thu, 02 Sep 2021 00:15:12 +0000
Received: by outflank-mailman (input) for mailman id 176712;
 Thu, 02 Sep 2021 00:15:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaNq-0007qB-9S
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:15:10 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.0])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0838a62-e773-4bfa-8b8e-f7f674f62b80;
 Thu, 02 Sep 2021 00:15:06 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0797.namprd21.prod.outlook.com (2603:10b6:300:11b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.1; Thu, 2 Sep
 2021 00:15:03 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:15:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0838a62-e773-4bfa-8b8e-f7f674f62b80
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=h8BvVlJXCwbhMTEelowaN9kx+nSDBGru5s7InceIX2Po4KWVpJ9roHMd/HCgBjctY+yfPUfYaGNLdP0C7C3/k3ibkdI3yS8myRWQ9hSAdwm3RmSgF4ic3PwzTh+Ohfs+bJc0QJ3CJy2p/VsqBqit7kPKrafI+XuMeCOkvhEidSOTySuNX/5lq7eUVBuymJX//cN1tcVRrsuR/E7FrJDR7mrAww93YjW5uljePBfs9Yzw8nXxhffKaIH97oJ68UrXfRd20GiPMro2luozaSykmfstmHPOP+V/5hqh67T4Bxu/qVIxO3B3mOaE01uFDrbW4avWGix5Zu/9cYBxrFUmLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2wJZwyg52cCLIvGROM9MWlEN9cOqwTvnQY12hCyFx/M=;
 b=PBg5e0gbR6i6m1kfUdBM0yttzgG6W3oki6YYMb+OpIhotgMUyPhbW/zmQqSfdenV5yHItttnIcqtZjyZtxHGNhRiM8A2XYeRSl/Cj92gJUdhaWkUgov87B+UkKqmp6oVPKIIEw6ETznyygch+fSUdwnIY4mkFFgzryPgGHDeC+5a2AVdLSONFKAkf+p7Gm+l/PVtc/MOanHkQRSKxFnRaQwT7PxVT8ZlM0PV5eca63f/J7m24zMpHzY9eptYV0Kivpl/xlbRRBI6Po8SGvg4uctJU8fSdJGIJ4njXkQaehViwJQPV9x6bYIHXnI8lEGRt1I4xEPkQKP/+90DvR7wnQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2wJZwyg52cCLIvGROM9MWlEN9cOqwTvnQY12hCyFx/M=;
 b=aIfunch/uFPwfCEApTqxLCAAZxJPvXnaRb7Pj8i7HYwjQ8P8SbWpTRgpUiYkQLLmQVAKBJZ3ttPxukfts2EM+obZus2xdhUWv56/DaGyXjYJFT/fM0RvR7hCvbagI3R+8jullzbTd+QlKODpqdoy3y4R+/5uzJJAHAHD0vo+GI8=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 01/13] x86/hyperv: Initialize GHCB page in Isolation VM
Thread-Topic: [PATCH V4 01/13] x86/hyperv: Initialize GHCB page in Isolation
 VM
Thread-Index: AQHXm2f8V9hgDb15oEeIZM6sY/3E8auH7dAA
Date: Thu, 2 Sep 2021 00:15:03 +0000
Message-ID:
 <MWHPR21MB1593366997ACAFCC79EBC0B0D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-2-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-2-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=11e436c2-915c-4d7c-84ac-3bd04a4dfdb3;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-27T22:24:34Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a93f77f8-5e8d-4146-86b8-08d96da6b5ec
x-ms-traffictypediagnostic: MWHPR21MB0797:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB07976EFF864A059B6428E017D7CE9@MWHPR21MB0797.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3044;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 epIsuYLDpNuvDcYJI1k0v+H+bBdkVs7GflPwJ89C7scLQt3PA8qBjBhhCjLcJ3STPnN7wrJytjcvU4hl0MYDSA/M207W9ZMTZ37Wd6OugAl6aezKycuTmFk77kpY88OfUh09NeI0vZ4l6URGi1BH3jkOgO+sBcjXVlellQ8kYwwGTZgl6scPuWO8up0RHHvR4XObjCS7HjHaI42LIbnIFzqWLzxJXgHp8+oIix3fh+zOf65qNMd4iPQv2BVuuaBLCRWplWUr6H6vAJsDFInSEDqIyrmje5XPV1EtXzoD4nLSWfOU/9a2gtMZBFaMWlOl7UAN59dHTNCldAD3s3wEI9/wOLRjXaRJzoBwjuSFBNjl6RzLrbZ89J8F4nI1fw1zrMoMilaQVJ70R0JBJ3ZwpI5Ofnc9WCTeH5wWTQ6Sb+McqQLhvO5+cie/iUTFJD5dls2fMYR6e8/pYxNXIN6EMcUR2KVAg6ys9nRDPECW3zkcnpIqDVHkPioYmSY7FgpGUFN3fA4//FfoMO0FLRP+K7Ce7o48GQ8mGf09t+UBtngfnTv0eBbrJTOu+Sa0H/sM4I1zEIBewjnOW0yLv+eQ4zPILbVDpuUHDPmP6sX8KJNwCtY+VtTxvts+r5CeSerSZ4SqHQKDausoCx4hUmtY+NW/I839xu+E8yjEnKCrz7eHixQWz5DX0oYg0LQdZzN9/TRgoXgK78QrvIGhVO7kFhR6FAN4+CDioU83HC74qgE=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(38070700005)(8936002)(316002)(8676002)(7696005)(71200400001)(5660300002)(52536014)(10290500003)(82960400001)(508600001)(8990500004)(82950400001)(7366002)(7406005)(66446008)(83380400001)(186003)(66476007)(66556008)(122000001)(76116006)(86362001)(921005)(33656002)(66946007)(9686003)(54906003)(55016002)(110136005)(64756008)(6506007)(4326008)(7416002)(38100700002)(26005);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?+SGvGo5LkyV3z+SW3AVjl3lmleGqyjpZ/boSbjjpuWW5SKx1ApAWYE1hlCTO?=
 =?us-ascii?Q?M7dMdQq8LUMvGbrXI86PorsvIaGjOuHQgc8ceSyzq6p2IEw6aj9e7RoNsLzh?=
 =?us-ascii?Q?+2RamO64+yd2hERjxFouOqyFd7ereJcl5HrAbIk/6CgQp6X3igVVNtA+OIXb?=
 =?us-ascii?Q?/V8Jr8+Swayl2Zkek8ng1r0qv6bnyZ4e2yq6vGz3XJIDzVWJQnsPtXRZ4n3Y?=
 =?us-ascii?Q?hToD/HPrFnHlMOK7KCeeuPruWaF4duxcruMBuYlSYsdvjiX/NYtevhjo4mPk?=
 =?us-ascii?Q?Pp1BPdyc54XNnenMlC5Rcx4zBjW2lPehSKRIYjtSy3u8A2hT3i/fIZ+rLS1W?=
 =?us-ascii?Q?skCK43EqJq9ZRA/0ayPuCPeUln685c3+YEU4DDVj/V2Awtfnlmw3Spy978Cl?=
 =?us-ascii?Q?G4mqEOPp0hFGPKHcY7Ppkz/S6ZkUYqZzKGH9g0lIBlbQVZKSrnVXaT/xpxuM?=
 =?us-ascii?Q?eO24L6OEKTYxWDhBCkaXqzRgVYLmMXdDCy0+JDoLdIEVBKvUUtzqX0pThptA?=
 =?us-ascii?Q?pExZRrp8K19Sf2lPHyjS6LtJrHvGRL8qwNHNdxVJCqtyZGpkYERdxwHXNt0K?=
 =?us-ascii?Q?dzF3dwCHdaOFdglJHBoihyelfe3V1o5d8A2lfM0GWICQOHEcZLQPqeOaOYFt?=
 =?us-ascii?Q?n4rabWOuGaNTLALS5eIwgjX548KNnEOuoP+77UvnBS1uNcnijusVJEiGF09J?=
 =?us-ascii?Q?JcgHidD1tXjJTsSNdnFGr+h8DtyEx7eop+uJtdq0dcoqM+UHqXBDG7wIygjG?=
 =?us-ascii?Q?qzn6p4DT21ArJV8QRDEewJCEdMw6BCtIPj1BXXqIH7dQD62DjHU7VvmUrUhC?=
 =?us-ascii?Q?JCSGgUn1pxSZDm+E1ydRtgM8UnQzzvSDJLMIyFx0YT3HM3huVz5TFRXqUmJ9?=
 =?us-ascii?Q?UxGP7a1EUwoftA0kAEj2zLBcTjqblpcnT27wzhwEzMfkRpaqSg7GSF/64xvy?=
 =?us-ascii?Q?4UB/j/0IBxnyPxY/hwJgVr4zpFAsIKyaYhE8SswCoPSQC1Va3yraTzHVJhJB?=
 =?us-ascii?Q?GWhOODZX3rIKIUGD0IiXw0IQASfRTS/4+Us/SKfYFOnvhD5PqRIy95JkjsIG?=
 =?us-ascii?Q?lDPpsidDnekVVuLsMKzaGYmmLSlqdUWCA9gRttqi8MJUyKf5pWY8dFxmGCqv?=
 =?us-ascii?Q?2y5AmFXa1jJRXarV4vIoPvevlzJqmtbgToTktN+Y1CxX8F4Fvl3wtHsfmZB3?=
 =?us-ascii?Q?A1ZWW4MKulOUpPCCsBOsVgROJuS9wvCloK1LZKwWqqMbqguRY5EDJ395PM2D?=
 =?us-ascii?Q?A2bcYFF+ZlueEd2dVCTkguUibyFLiqHbMdXYdvk6Q32xl2Mn8EEvqlAioWsz?=
 =?us-ascii?Q?H+rz7J+co1poMS4EpHmARJCo?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a93f77f8-5e8d-4146-86b8-08d96da6b5ec
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:15:03.0983
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: p8dISAuhX9IHX9/2j7Bsfwp5edDHyaJ+23+T2AH2eOY3r/aR7IqvRmfOlB8qS0BdABm/bMD8pSHZu6ldb6ZT/WmE/DgBmGyuWoge9svJhZM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0797

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> Hyperv exposes GHCB page via SEV ES GHCB MSR for SNP guest
> to communicate with hypervisor. Map GHCB page for all
> cpus to read/write MSR register and submit hvcall request
> via ghcb page.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Chagne since v3:
>         * Rename ghcb_base to hv_ghcb_pg and move it out of
> 	  struct ms_hyperv_info.
> 	* Allocate hv_ghcb_pg before cpuhp_setup_state() and leverage
> 	  hv_cpu_init() to initialize ghcb page.
> ---
>  arch/x86/hyperv/hv_init.c       | 68 +++++++++++++++++++++++++++++----
>  arch/x86/include/asm/mshyperv.h |  4 ++
>  arch/x86/kernel/cpu/mshyperv.c  |  3 ++
>  include/asm-generic/mshyperv.h  |  1 +
>  4 files changed, 69 insertions(+), 7 deletions(-)
>=20
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index 708a2712a516..eba10ed4f73e 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -20,6 +20,7 @@
>  #include <linux/kexec.h>
>  #include <linux/version.h>
>  #include <linux/vmalloc.h>
> +#include <linux/io.h>
>  #include <linux/mm.h>
>  #include <linux/hyperv.h>
>  #include <linux/slab.h>
> @@ -36,12 +37,42 @@ EXPORT_SYMBOL_GPL(hv_current_partition_id);
>  void *hv_hypercall_pg;
>  EXPORT_SYMBOL_GPL(hv_hypercall_pg);
>=20
> +void __percpu **hv_ghcb_pg;
> +
>  /* Storage to save the hypercall page temporarily for hibernation */
>  static void *hv_hypercall_pg_saved;
>=20
>  struct hv_vp_assist_page **hv_vp_assist_page;
>  EXPORT_SYMBOL_GPL(hv_vp_assist_page);
>=20
> +static int hyperv_init_ghcb(void)
> +{
> +	u64 ghcb_gpa;
> +	void *ghcb_va;
> +	void **ghcb_base;
> +
> +	if (!hv_isolation_type_snp())
> +		return 0;
> +
> +	if (!hv_ghcb_pg)
> +		return -EINVAL;
> +
> +	/*
> +	 * GHCB page is allocated by paravisor. The address
> +	 * returned by MSR_AMD64_SEV_ES_GHCB is above shared
> +	 * ghcb boundary and map it here.

I'm not sure what the "shared ghcb boundary" is.  Did you
mean "shared_gpa_boundary"?

> +	 */
> +	rdmsrl(MSR_AMD64_SEV_ES_GHCB, ghcb_gpa);
> +	ghcb_va =3D memremap(ghcb_gpa, HV_HYP_PAGE_SIZE, MEMREMAP_WB);
> +	if (!ghcb_va)
> +		return -ENOMEM;
> +
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	*ghcb_base =3D ghcb_va;
> +
> +	return 0;
> +}
> +
>  static int hv_cpu_init(unsigned int cpu)
>  {
>  	union hv_vp_assist_msr_contents msr =3D { 0 };
> @@ -85,7 +116,7 @@ static int hv_cpu_init(unsigned int cpu)
>  		}
>  	}
>=20
> -	return 0;
> +	return hyperv_init_ghcb();
>  }
>=20
>  static void (*hv_reenlightenment_cb)(void);
> @@ -177,6 +208,14 @@ static int hv_cpu_die(unsigned int cpu)
>  {
>  	struct hv_reenlightenment_control re_ctrl;
>  	unsigned int new_cpu;
> +	void **ghcb_va;
> +
> +	if (hv_ghcb_pg) {
> +		ghcb_va =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +		if (*ghcb_va)
> +			memunmap(*ghcb_va);
> +		*ghcb_va =3D NULL;
> +	}
>=20
>  	hv_common_cpu_die(cpu);
>=20
> @@ -366,10 +405,16 @@ void __init hyperv_init(void)
>  		goto common_free;
>  	}
>=20
> +	if (hv_isolation_type_snp()) {
> +		hv_ghcb_pg =3D alloc_percpu(void *);
> +		if (!hv_ghcb_pg)
> +			goto free_vp_assist_page;
> +	}
> +
>  	cpuhp =3D cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:onlin=
e",
>  				  hv_cpu_init, hv_cpu_die);
>  	if (cpuhp < 0)
> -		goto free_vp_assist_page;
> +		goto free_ghcb_page;
>=20
>  	/*
>  	 * Setup the hypercall page and enable hypercalls.
> @@ -383,10 +428,8 @@ void __init hyperv_init(void)
>  			VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX,
>  			VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
>  			__builtin_return_address(0));
> -	if (hv_hypercall_pg =3D=3D NULL) {
> -		wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> -		goto remove_cpuhp_state;
> -	}
> +	if (hv_hypercall_pg =3D=3D NULL)
> +		goto clean_guest_os_id;
>=20
>  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>  	hypercall_msr.enable =3D 1;
> @@ -456,8 +499,11 @@ void __init hyperv_init(void)
>  	hv_query_ext_cap(0);
>  	return;
>=20
> -remove_cpuhp_state:
> +clean_guest_os_id:
> +	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
>  	cpuhp_remove_state(cpuhp);
> +free_ghcb_page:
> +	free_percpu(hv_ghcb_pg);
>  free_vp_assist_page:
>  	kfree(hv_vp_assist_page);
>  	hv_vp_assist_page =3D NULL;
> @@ -559,3 +605,11 @@ bool hv_is_isolation_supported(void)
>  {
>  	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
>  }
> +
> +DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
> +
> +bool hv_isolation_type_snp(void)
> +{
> +	return static_branch_unlikely(&isolation_type_snp);
> +}
> +EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index adccbc209169..37739a277ac6 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -11,6 +11,8 @@
>  #include <asm/paravirt.h>
>  #include <asm/mshyperv.h>
>=20
> +DECLARE_STATIC_KEY_FALSE(isolation_type_snp);
> +
>  typedef int (*hyperv_fill_flush_list_func)(
>  		struct hv_guest_mapping_flush_list *flush,
>  		void *data);
> @@ -39,6 +41,8 @@ extern void *hv_hypercall_pg;
>=20
>  extern u64 hv_current_partition_id;
>=20
> +extern void __percpu **hv_ghcb_pg;
> +
>  int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
>  int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
>  int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flag=
s);
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyper=
v.c
> index 6b5835a087a3..20557a9d6e25 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -316,6 +316,9 @@ static void __init ms_hyperv_init_platform(void)
>=20
>  		pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n",
>  			ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
> +
> +		if (hv_get_isolation_type() =3D=3D HV_ISOLATION_TYPE_SNP)
> +			static_branch_enable(&isolation_type_snp);
>  	}
>=20
>  	if (hv_max_functions_eax >=3D HYPERV_CPUID_NESTED_FEATURES) {
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index c1ab6a6e72b5..0924bbd8458e 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -237,6 +237,7 @@ bool hv_is_hyperv_initialized(void);
>  bool hv_is_hibernation_supported(void);
>  enum hv_isolation_type hv_get_isolation_type(void);
>  bool hv_is_isolation_supported(void);
> +bool hv_isolation_type_snp(void);
>  void hyperv_cleanup(void);
>  bool hv_query_ext_cap(u64 cap_query);
>  #else /* CONFIG_HYPERV */
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:15:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:15:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176715.321607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaOF-0008ND-Eh; Thu, 02 Sep 2021 00:15:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176715.321607; Thu, 02 Sep 2021 00:15:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaOF-0008N6-Bi; Thu, 02 Sep 2021 00:15:35 +0000
Received: by outflank-mailman (input) for mailman id 176715;
 Thu, 02 Sep 2021 00:15:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaOE-0008IR-Ay
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:15:34 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.0])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 27f8eb29-ef3f-4de2-a97e-6b7e1cfc01c6;
 Thu, 02 Sep 2021 00:15:30 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0797.namprd21.prod.outlook.com (2603:10b6:300:11b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.1; Thu, 2 Sep
 2021 00:15:28 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:15:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27f8eb29-ef3f-4de2-a97e-6b7e1cfc01c6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E4b3tJqAMNcxiBzzEYA+v8JE1ZchrBMzycvlyjyyEaC0jibjIvp+NMHjfnl2zmnDwsHyahDsfktZS8PTxXLG+popmuIiBWOXz62JNRCg9cA3EIm3Fl2xpFj6Zb2E/Ag12U7AtWeOq4NSjPWf3SV5ApwYFF+YKVYCId1L+R6KTZgx4cfIU/K5XPnwYiY1o/g4hR/u1cb1vUVxSKhGMtzSSJHjHZb5BPRHV3RnDzTsYRaTSA+xSq2F8Taod9dNaV71f1ckAKaLxZ5Qrqk+CmtCaJL1DO4F+SyEp1a8zv0sXuUqZuBN1RaDEnrXuzNTwu4E/W1rIOKs+UF6tV8gxM7bGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6tVUDVw6S4dvfWR9PY8PHDrUdy5zfJaPLLhUIx49AUM=;
 b=JZjZjgEg7RToqs/5LVea8kPluWOU/O61jvpFHollPfj+oJSi4JR6eH8JW9kRxmyA4RWMfTsnQqQ9vk17ASZSn0SBndGQ80RprXLJvPnw2hHOZFV2OUh4LdbhMb5jVIhtJs3mWBUI3PNFrzD7ylBwuFNgnu/mIEsIDCM9MW/6rDWCEHCQCSKU085CFaUyU+tpEjMWtatYhqNFOx7W3vy9K9wJIswRA4qOifGPTLZstnDYGQmBn7ysip7alJ6+IzLOcVCvmoyD21LhMQseCHCxpNVNstFrjO4eGcsMRqRscHSddmU7n0N199RZiy80BVwWLHDvY6pb+7o+me7ayHlk7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6tVUDVw6S4dvfWR9PY8PHDrUdy5zfJaPLLhUIx49AUM=;
 b=XumoSZuZrB97O7Abt8lXvJ9jNJfQv5OV+xW/t/sl9NOQ/+OdR4zwVipb0ca/QetLiiid3b5PZeIMVJ443lNTePri9lg8ZMU+k+6BuJWmz0nL0oKwaKRHiUIC7602NP1teRCekzNyw17VX8V+3GXFOslyI6mSMYd+0qzlfcZegFg=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 02/13] x86/hyperv: Initialize shared memory boundary in
 the Isolation VM.
Thread-Topic: [PATCH V4 02/13] x86/hyperv: Initialize shared memory boundary
 in the Isolation VM.
Thread-Index: AQHXm2f5YhiZigcKo0WndvjSqX/I9KuH761w
Date: Thu, 2 Sep 2021 00:15:27 +0000
Message-ID:
 <MWHPR21MB1593EF63423A2422DA839793D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-3-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-3-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=0a271260-2add-4c88-99c4-2e5b54a0e418;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-27T22:31:14Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0bb98b2c-b332-43be-b6bc-08d96da6c4b7
x-ms-traffictypediagnostic: MWHPR21MB0797:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB079715FAF7178E238D3B3B27D7CE9@MWHPR21MB0797.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:4714;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 Zzhi7VwaXM35ruQkmXyNvFX42K8MnHgcxR/I3JJ+p15J4AndM8A8FmXFNPW0Pw8Zmj5+C/dQkGgoAPemXaY0hIgueRmcLQgPytlXLZYv1msC0kBb0zg2LC1O/bANppFMlGFQstY1/XNvxZsH9spLv3aAl9A4dNcvZStMcPOVrKOLVuv+nDhIgq4fK3KXfsO4GdfwA1kEyDV25i4RCCxvMewMCc9uEP87gxhsd9Pl8+hCRnPMjvEcttz6uV0CP2h477ANsQg/YwF3fdGcCnJownxpZwVAUvFDf+hKjXN6ksIpzSW1U4gOHq2f0QIKx0i7mBp0Oma1Tyi4sQvsJ6YSHs2vTKVW4WOXUT1xl6ro+ymF7OzJk9Dysu2CKhEo8+1tCVg8bM3E1rnbYTrCWmAl7vRXivkk6XMwpUojkA1+rMvn3XEaQZwMM6GPP4WPuAIhp7+vByu4U8NP5lj2ffCybg6h6xUrShI4JkrArYq1qzNufWyC8aVh4p0k28nzgoVtL1cvWYZ7CT46DmYDP/gVhwwHKhgBtNVmcI+9dfUgW62vHeD77Qlo9uXoSxIjiVa2hLen6ZE+hCVPK4au4pauws+DPDiU64wJOURLa3BzaOGckOZ+MFRc5stLX/0DsR0UPlNCGbgqVCC4J3JNW4Tac2vCad2Ul8E8GLFt66H7NKEJf9IELvrkPYiHpEf8II1jmaYpQav5ec5kN7poLkKbHd/VfqCb79Ba6ivjZvF72Xg=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(38070700005)(8936002)(316002)(8676002)(7696005)(71200400001)(5660300002)(52536014)(10290500003)(82960400001)(508600001)(8990500004)(82950400001)(7366002)(7406005)(66446008)(83380400001)(186003)(66476007)(66556008)(122000001)(76116006)(86362001)(921005)(33656002)(66946007)(9686003)(54906003)(55016002)(110136005)(64756008)(6506007)(4326008)(7416002)(38100700002)(26005);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?a1LzKABlpY2yhIlj+iKhGJUcJvC5jESOKLs2Un6hyPqUyS6IMhSh+/lr4UEg?=
 =?us-ascii?Q?w4bnjgOkDQSxvfqIbFWEZAzGVSuLVIBM4RQ/I86Nn1ARK/FcNqUjRlr9TmSj?=
 =?us-ascii?Q?9qyHRcd495zNkey5DZL/fabj2qoNgeBp82a97j5ZQVGLE9vVLtvoggyGeUgp?=
 =?us-ascii?Q?2xCOv/jeJmCqda4zvsT3tgkxXvBWZVb3sISBZYsCRBDEmPamgwcUeBzVvbtX?=
 =?us-ascii?Q?VGIjkwOjmm10orfXz7Cgchunxbb1VuEFU/wMBozMap5TT6IiWP5+BQFfy/Iz?=
 =?us-ascii?Q?gupQltgZqYsx35Evb86AUkTXsp4fG3dYkzZ0P8u+uk5+BLqENGX5ssRaIoaS?=
 =?us-ascii?Q?DFIcuFZmoXPM7Of2oQkcWs5EACSU0XgrWQi61j4qki8Qssm8lT9Ia6XVa0Ji?=
 =?us-ascii?Q?os3x4qfNzqFQcZLqqUziRbomGbLaFNf/sB5gAcIkh+RGt9A2ozj+rmSvcKqV?=
 =?us-ascii?Q?2lLtSkDRdJA1KH8VdBc+s5qQsrtpaPn7WkL1Z6231nhUoOUaJmSsMm1wSi7t?=
 =?us-ascii?Q?ETzPaZHbWMN3NmCGr3hq07iKmE06QOLZclGMxy6ZO4vEnXY2iknm4oAMCbhv?=
 =?us-ascii?Q?/u1tex/FmTFBJyhfl02YFRCT2w/MiIcpK8utaFED4nNUN+vhdlo5IwmZ17QX?=
 =?us-ascii?Q?QP7r5xsNZVxjVoi+OCGRBHWpo/2uRpZQFojUCEKH4MssdzvR6BgC5vSh7LCk?=
 =?us-ascii?Q?e4lnYHSXS0lEkFsEkfTU0LVRer4M2HHgg/V9vbGUWU8iXrUK+l3gdvlSdJ8F?=
 =?us-ascii?Q?nXh0iWR7Pwk9p6pU1va7cOebYDKGsG2S8cAvT+O1RNYvU9aaFYsSXOF+InoU?=
 =?us-ascii?Q?tswUliudNqGh8ijCBXjReAIeb+XzWFrpBuQxBTrKMEj4adRifDNKRrisvXwu?=
 =?us-ascii?Q?1sopz+gy1XxgBpb2wbV3povJXd4rDFWXSjxbshyzGBGqSh9fsQEqax6hcE2p?=
 =?us-ascii?Q?G5AaQc5l3LvDxC0WSaqtHmyU0M1GBp1fu86fKP6CyhHWz0TbHekuiW/rHV6j?=
 =?us-ascii?Q?TfgIpj+iOZBMHdTdtBlAmLmA5v9JEs+26IrHhbO48+b2PaV9ImIixo370IWc?=
 =?us-ascii?Q?+4pdlJaIzduBwQn1/pzyIZ4NuHNoxJt01ab4/WTT+nBUlAa6sl4x9gs0KThQ?=
 =?us-ascii?Q?kmwqeNT+VXoO0T31VLZRNtJcz6rAZ0rWDrhj7rVfEYe5CQc/XEjRjoLS4+d4?=
 =?us-ascii?Q?Tm09Us06khNRTuhmGc+gGt8QG9UO+Sau7cnxUX6vm3h7U4HD3SpcI3WPwvR8?=
 =?us-ascii?Q?sxx+sFAULKiGz5tCKQ17Z8bJp9Myyb1bM+LObHFwNrntegCzX+UXMLAzXRyE?=
 =?us-ascii?Q?Ece15J/8FKpNHmK4Omwm5laf?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0bb98b2c-b332-43be-b6bc-08d96da6c4b7
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:15:27.9650
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kqyCC46tRZ6Ol7tcUTWt/2lSA8O5oztS7qxpD8ekw1tgtmu+y1jejDT1KAvl57lUrUlPeLUjEzGIJsjVZ1O4C5AKq10LHwiGv5P17CeeGlQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0797

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> Hyper-V exposes shared memory boundary via cpuid
> HYPERV_CPUID_ISOLATION_CONFIG and store it in the
> shared_gpa_boundary of ms_hyperv struct. This prepares
> to share memory with host for SNP guest.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* user BIT_ULL to get shared_gpa_boundary
> 	* Rename field Reserved* to reserved
> ---
>  arch/x86/kernel/cpu/mshyperv.c |  2 ++
>  include/asm-generic/mshyperv.h | 12 +++++++++++-
>  2 files changed, 13 insertions(+), 1 deletion(-)
>=20
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyper=
v.c
> index 20557a9d6e25..8bb001198316 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -313,6 +313,8 @@ static void __init ms_hyperv_init_platform(void)
>  	if (ms_hyperv.priv_high & HV_ISOLATION) {
>  		ms_hyperv.isolation_config_a =3D cpuid_eax(HYPERV_CPUID_ISOLATION_CONF=
IG);
>  		ms_hyperv.isolation_config_b =3D cpuid_ebx(HYPERV_CPUID_ISOLATION_CONF=
IG);
> +		ms_hyperv.shared_gpa_boundary =3D
> +			BIT_ULL(ms_hyperv.shared_gpa_boundary_bits);
>=20
>  		pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n",
>  			ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index 0924bbd8458e..7537ae1db828 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -35,7 +35,17 @@ struct ms_hyperv_info {
>  	u32 max_vp_index;
>  	u32 max_lp_index;
>  	u32 isolation_config_a;
> -	u32 isolation_config_b;
> +	union {
> +		u32 isolation_config_b;
> +		struct {
> +			u32 cvm_type : 4;
> +			u32 reserved11 : 1;
> +			u32 shared_gpa_boundary_active : 1;
> +			u32 shared_gpa_boundary_bits : 6;
> +			u32 reserved12 : 20;

I'm still curious about the "11" and "12" in the reserved
field names.  Why not just "reserved1" and "reserved2"?
Having the "11" and "12" isn't wrong, but it makes one
wonder why since it's not usual. :-)

> +		};
> +	};
> +	u64 shared_gpa_boundary;
>  };
>  extern struct ms_hyperv_info ms_hyperv;
>=20
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:16:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:16:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176725.321618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaPM-0000hN-S5; Thu, 02 Sep 2021 00:16:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176725.321618; Thu, 02 Sep 2021 00:16:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaPM-0000hG-Oj; Thu, 02 Sep 2021 00:16:44 +0000
Received: by outflank-mailman (input) for mailman id 176725;
 Thu, 02 Sep 2021 00:16:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaPL-0000h4-8U
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:16:43 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.14])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0a38c2e2-0b83-11ec-ae2c-12813bfff9fa;
 Thu, 02 Sep 2021 00:16:38 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by CO1PR21MB1314.namprd21.prod.outlook.com (2603:10b6:303:151::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4; Thu, 2 Sep
 2021 00:16:34 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:16:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a38c2e2-0b83-11ec-ae2c-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y4w4QNo088iiDF7WpKo5d2HaaMS7RSYImGHIUr5/Tl//9csWEPwMnnX2KMUrNybliUOgQ1Sloxs+2j4rmwjbJmbbCIBFvjtlz85hZLLBGZ9TgOZToBXaRJNMNWSh91AKGSmyqtSiJLtdqOIfO9CUqDD/lyATiZqihVd1+b7QXC3XEfL9OooQ9O2zKto4VlElV7nm1X1EUtzlAIPyEeANEtDH+Pkt0VaPyroFgHjcCpROgDjTGRX2SX6BgaB1KNjxaxkB+8+IuhBBux5xF5RypFjlKhA2/LPxDLEXvGBl0Jc79ULx4pZSkdMiUKyGTCmQteA7HoztderafBWQw0665w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rGjVQNUXVDO4QwZnYpGSTcEEWMpwB5D9QDdioJ1qzYM=;
 b=PPe4WGH5WOMJJijs85AS7NtP8dE8jdNfodo5az0kecFnWGgjTwrXGbc9Bh0EAjFfakMIMPH6l6CtKKOoWWuvjKFH3+4d+9ljimrIrYwSN+i2dH62SPQRPb6tqiDK05RKcFCzyKhPqTaIRSAgGUm7yi0qk7FuOQKeHOYi/wjFy+dE29M1+V8+mapMfVrNwrUmKBz6xeqX/m/1sawbUL2LAmoXos6PljQVPjoxb6Wms0ANUoGpPw//MmbKYH7PsPRWz/Qkyx17lxkvFx//MTcFLbDp9k8pSdAKPg8Dq6KnCnlXyrg5J5xVAGGzUVFcR4RtyHIQz+4/u0u0o+y75NwpiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rGjVQNUXVDO4QwZnYpGSTcEEWMpwB5D9QDdioJ1qzYM=;
 b=MIeMV26IwzWTaPJPirf8QDvR17EiHNKPWCapnyJH/eF0HmSxebOyZBlG7hPUktF8//ikMwtRGa7AFnHi3W1X2Qbzi7HAo2tLcka+rlM3qE2lfM6KqNdmMWAAHw9zQHACwHW1ZBU1ZZxBMqYE+geJk41lwvqvn/qXYcZDChSn0pc=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 03/13] x86/hyperv: Add new hvcall guest address host
 visibility support
Thread-Topic: [PATCH V4 03/13] x86/hyperv: Add new hvcall guest address host
 visibility support
Thread-Index: AQHXm2f+zKaCHZTB30K5pZMyKctp/auH8Tww
Date: Thu, 2 Sep 2021 00:16:34 +0000
Message-ID:
 <MWHPR21MB159348E5A5BE84C36E84800AD7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-4-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-4-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=a96f9a08-110c-4822-a43b-0d2d08ef1746;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-27T22:36:48Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a47f00d8-8849-4fbb-ac96-08d96da6ec39
x-ms-traffictypediagnostic: CO1PR21MB1314:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <CO1PR21MB1314DBA2810799F5F62DE9C5D7CE9@CO1PR21MB1314.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 bTLYupGy/LbS0xQhhUnKkQfH9+pV+XrKMU1UnFme7fWER5H9Lr3jDBjpUkxY0EkbkHZcSGWKDKclo+u+QKdSeGEisecMjpytRwf9xNfDlXpv+V8ENWd8PKHKIw3kARh0Nh4V/da9ORN23Kh/DeSFEBtzpFBrP6xMANv7yrPfXAvaaI58RXcIP1t1jhNgAFHHfj4UnnmnWR0o7fZSjReGnURTZCdxeb8/swyFn581WqpbFyTD8bcn9OfkaRMwmG6Sgbj+5qifE3gIXkITOw/zFaiCeWNoxi2jn8R3kZiYnYwIOLUMVhy9qXs3wSgiLiQsvrHnIlf+UZWaEmvilnyR3usEack32K63cAqL/B3NrrzuOM8L6nxRO9m4uMVT/nHl/t9QHlqolHaVENQOrlh2jSLkkluiH+Nr0XB9w70sesqPJF7aE/okVzfkm+6+yTbL9VvrEllLpMtBpRj5wQWzoAij3Puc0gM168glxxBQj18MAockWNwPgLDseDANo9f+p54S71AL4p5jowoDOUySP/FxCZz6RKJX1bay9EtELMdVRumApAMrrF4rvD/7i3KH8VloBbAn6uhuo8WlMCoZFayEMRJ6diiBg9dYV+1pHd+1g4dIHfN6oiPz/jbEGea9VRy3gCGeopeTZGJqO5SbZOlGkmD+BWXv+FK8gt5I1gQ1XzdtpmBLne4S2tacAqqJi3kKqW/xFvMXia3b7ipzAcq4V8is9c4ng75n34p5f74=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(10290500003)(7696005)(186003)(66946007)(6506007)(76116006)(38100700002)(8676002)(110136005)(9686003)(316002)(54906003)(71200400001)(38070700005)(8990500004)(86362001)(55016002)(2906002)(122000001)(508600001)(8936002)(4326008)(7406005)(5660300002)(7416002)(7366002)(33656002)(52536014)(83380400001)(82950400001)(30864003)(66556008)(82960400001)(921005)(66476007)(66446008)(64756008);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?Oyj20tWEpY4HT/oJ0UQqteE/BHXeBnXQUewR3f2TeAjvT1uuMkia78HGPYvo?=
 =?us-ascii?Q?/uacGbRrpw/7VcTrGhpFMstHzc0fC8fO7Uw5Bb0kUKI1fOQdpN83fNfQxbXW?=
 =?us-ascii?Q?D5OCEJMIVHVv6Ncg9Cxv2+sUly5BVW4t9Yk+IVGKbuVnNhDV6uKotO5ve6lg?=
 =?us-ascii?Q?uwBlEL0ci6e3XakoPhb7mbCI7RYJZjcwyAc7uST8rLK6w7K0ne/NzfRKFnh6?=
 =?us-ascii?Q?rRA8hpkvJjEktQiB61q1wFPWr8mRdW0/CmTBfD33kAy7SGey2SPSuCm/BG8s?=
 =?us-ascii?Q?Kv95F7VSo9HxxJSjYIq5ibFONL1FXWgPjofkqblmd2ovd2ad/mLcvV4ISWcE?=
 =?us-ascii?Q?WwGaFkpatO2zq9l1RaWGgsxeX03ZP3W+cDtE2qByEZlVJvpRZie9XaiIzwFc?=
 =?us-ascii?Q?QvRtnO0wTQiAFv/Vo5Vx40kiTtmbKCXnbQG+dzlZlDts2z0/IbnRaxU6d6P9?=
 =?us-ascii?Q?yhoMNLAYM58UQs119ru35xWRyLJ/nN5vDMTwH4pmttQUmaHj6ZMBvJ41kOFC?=
 =?us-ascii?Q?Hwc24zW7BlOi9Y68VlvcZ/2G1w3FTSgO/ArRJ/1IKeJ7e24Yt4wdA+d15FiP?=
 =?us-ascii?Q?qzX0q2jA0GHD/fvrYz9dkmx8yf/vNMSCv2xzAoK4QO6owDL5uuxSIwlJNVcj?=
 =?us-ascii?Q?Uedr/m9USGqqXg9RwWw2kuwQX39q0U1Wbmug0D4NGxbnHxYynPZBs/7rL3l2?=
 =?us-ascii?Q?SDWbHl0oj4wADgD/bpFhaz91PMv68rklAYXj/5GdNzMzWYWUONKQsznKKGwH?=
 =?us-ascii?Q?KIXB4VWEO+5wW0vuO5ydrPUgXzk86IcWBdhq36tmpZI360haF53Ts8U4Vc/6?=
 =?us-ascii?Q?82+mvgYqeVGfKB1EHNTv/w1CVUnKMnbcKtXSWLbp4gNxS2htKc7NscF5IJ5u?=
 =?us-ascii?Q?qiLG9VBT6F9dMweSXpR50DgSmgQrT1Aag8QgXb6RJeA+BIOZ+3vT4ujLf+CI?=
 =?us-ascii?Q?azag7e+UT03PqqBKtrxhnrelfjta8AUUBEfgursanYZG4WMgydj9exK9w5q0?=
 =?us-ascii?Q?YlG3gguR2WncMBb4UXV0ZWp45zoCpuL9GmF+hN7M7H4g7IN5cycZaDFuq3dR?=
 =?us-ascii?Q?Kcp/8nHaiVltDvUC3mh9BA5gpfiir8JnPRtXqu72nV1hz0An0vF5MOPo1rJu?=
 =?us-ascii?Q?fGpkAWMOw7QKfLD0jEumZO5cONj4c4Ic9LkJX6KlrjE41CTd0j4WURg46EBF?=
 =?us-ascii?Q?hkZ/vJXFuBr+pOFkbACLizdbLqpoxyRi4P8k6cHFqmIcHePqcG8drUEIhv+w?=
 =?us-ascii?Q?hoBcAFW1u09sn1dvP47QsLMkAku0hCZmlmjtd074t982tKScsc5dNyi8rnBH?=
 =?us-ascii?Q?btNv23jiKRhT+7da4+VEJf4Z?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a47f00d8-8849-4fbb-ac96-08d96da6ec39
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:16:34.1759
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: iy7Bm09R2ybhwyJZEZLyvD9iuXUgQWan7i+o0pFTehAQFo+nV250s0Vhubtgu9mRkpslfJasLFpR6yX495cvj7CTXKrGKBt/EevN0WlWfzE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR21MB1314

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> Add new hvcall guest address host visibility support to mark
> memory visible to host. Call it inside set_memory_decrypted
> /encrypted(). Add HYPERVISOR feature check in the
> hv_is_isolation_supported() to optimize in non-virtualization
> environment.
>=20
> Acked-by: Dave Hansen <dave.hansen@intel.com>
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Fix error code handle in the __hv_set_mem_host_visibility().
> 	* Move HvCallModifySparseGpaPageHostVisibility near to enum
> 	  hv_mem_host_visibility.
>=20
> Change since v2:
>        * Rework __set_memory_enc_dec() and call Hyper-V and AMD function
>          according to platform check.
>=20
> Change since v1:
>        * Use new staic call x86_set_memory_enc to avoid add Hyper-V
>          specific check in the set_memory code.
> ---
>  arch/x86/hyperv/Makefile           |   2 +-
>  arch/x86/hyperv/hv_init.c          |   6 ++
>  arch/x86/hyperv/ivm.c              | 113 +++++++++++++++++++++++++++++
>  arch/x86/include/asm/hyperv-tlfs.h |  17 +++++
>  arch/x86/include/asm/mshyperv.h    |   4 +-
>  arch/x86/mm/pat/set_memory.c       |  19 +++--
>  include/asm-generic/hyperv-tlfs.h  |   1 +
>  include/asm-generic/mshyperv.h     |   1 +
>  8 files changed, 156 insertions(+), 7 deletions(-)
>  create mode 100644 arch/x86/hyperv/ivm.c
>=20
> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> index 48e2c51464e8..5d2de10809ae 100644
> --- a/arch/x86/hyperv/Makefile
> +++ b/arch/x86/hyperv/Makefile
> @@ -1,5 +1,5 @@
>  # SPDX-License-Identifier: GPL-2.0-only
> -obj-y			:=3D hv_init.o mmu.o nested.o irqdomain.o
> +obj-y			:=3D hv_init.o mmu.o nested.o irqdomain.o ivm.o
>  obj-$(CONFIG_X86_64)	+=3D hv_apic.o hv_proc.o
>=20
>  ifdef CONFIG_X86_64
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index eba10ed4f73e..b1aa42f60faa 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -603,6 +603,12 @@ EXPORT_SYMBOL_GPL(hv_get_isolation_type);
>=20
>  bool hv_is_isolation_supported(void)
>  {
> +	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
> +		return 0;

Use "return false" per previous comment from Wei Liu.

> +
> +	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
> +		return 0;

Use "return false".

> +
>  	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
>  }
>=20
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> new file mode 100644
> index 000000000000..a069c788ce3c
> --- /dev/null
> +++ b/arch/x86/hyperv/ivm.c
> @@ -0,0 +1,113 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Hyper-V Isolation VM interface with paravisor and hypervisor
> + *
> + * Author:
> + *  Tianyu Lan <Tianyu.Lan@microsoft.com>
> + */
> +
> +#include <linux/hyperv.h>
> +#include <linux/types.h>
> +#include <linux/bitfield.h>
> +#include <linux/slab.h>
> +#include <asm/io.h>
> +#include <asm/mshyperv.h>
> +
> +/*
> + * hv_mark_gpa_visibility - Set pages visible to host via hvcall.
> + *
> + * In Isolation VM, all guest memory is encripted from host and guest

s/encripted/encrypted/

> + * needs to set memory visible to host via hvcall before sharing memory
> + * with host.
> + */
> +int hv_mark_gpa_visibility(u16 count, const u64 pfn[],
> +			   enum hv_mem_host_visibility visibility)
> +{
> +	struct hv_gpa_range_for_visibility **input_pcpu, *input;
> +	u16 pages_processed;
> +	u64 hv_status;
> +	unsigned long flags;
> +
> +	/* no-op if partition isolation is not enabled */
> +	if (!hv_is_isolation_supported())
> +		return 0;
> +
> +	if (count > HV_MAX_MODIFY_GPA_REP_COUNT) {
> +		pr_err("Hyper-V: GPA count:%d exceeds supported:%lu\n", count,
> +			HV_MAX_MODIFY_GPA_REP_COUNT);
> +		return -EINVAL;
> +	}
> +
> +	local_irq_save(flags);
> +	input_pcpu =3D (struct hv_gpa_range_for_visibility **)
> +			this_cpu_ptr(hyperv_pcpu_input_arg);
> +	input =3D *input_pcpu;
> +	if (unlikely(!input)) {
> +		local_irq_restore(flags);
> +		return -EINVAL;
> +	}
> +
> +	input->partition_id =3D HV_PARTITION_ID_SELF;
> +	input->host_visibility =3D visibility;
> +	input->reserved0 =3D 0;
> +	input->reserved1 =3D 0;
> +	memcpy((void *)input->gpa_page_list, pfn, count * sizeof(*pfn));
> +	hv_status =3D hv_do_rep_hypercall(
> +			HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY, count,
> +			0, input, &pages_processed);
> +	local_irq_restore(flags);
> +
> +	if (hv_result_success(hv_status))
> +		return 0;
> +	else
> +		return -EFAULT;
> +}
> +EXPORT_SYMBOL(hv_mark_gpa_visibility);

In later comments on Patch 7 of this series, I have suggested that
code in that patch should not call hv_mark_gpa_visibility() directly,
but instead should call set_memory_encrypted() and
set_memory_decrypted().  I'm thinking that those functions should
be the standard way to change the visibility of pages in the Isolated
VM case.  Then hv_mark_gpa_visibility() could be static and it would
not need a stub version for ARM64.  It would only be called by
__hv_set_mem_host_visibility below, and in turn by
set_memory_encrypted()/decrypted(). =20

> +
> +static int __hv_set_mem_host_visibility(void *kbuffer, int pagecount,
> +				      enum hv_mem_host_visibility visibility)
> +{
> +	u64 *pfn_array;
> +	int ret =3D 0;
> +	int i, pfn;
> +
> +	if (!hv_is_isolation_supported() || !hv_hypercall_pg)
> +		return 0;
> +
> +	pfn_array =3D kmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
> +	if (!pfn_array)
> +		return -ENOMEM;
> +
> +	for (i =3D 0, pfn =3D 0; i < pagecount; i++) {
> +		pfn_array[pfn] =3D virt_to_hvpfn(kbuffer + i * HV_HYP_PAGE_SIZE);
> +		pfn++;
> +
> +		if (pfn =3D=3D HV_MAX_MODIFY_GPA_REP_COUNT || i =3D=3D pagecount - 1) =
{
> +			ret =3D hv_mark_gpa_visibility(pfn, pfn_array,
> +						     visibility);
> +			if (ret)
> +				goto err_free_pfn_array;
> +			pfn =3D 0;
> +		}
> +	}
> +
> + err_free_pfn_array:
> +	kfree(pfn_array);
> +	return ret;
> +}
> +
> +/*
> + * hv_set_mem_host_visibility - Set specified memory visible to host.
> + *
> + * In Isolation VM, all guest memory is encrypted from host and guest
> + * needs to set memory visible to host via hvcall before sharing memory
> + * with host. This function works as wrap of hv_mark_gpa_visibility()
> + * with memory base and size.
> + */
> +int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible)
> +{
> +	enum hv_mem_host_visibility visibility =3D visible ?
> +			VMBUS_PAGE_VISIBLE_READ_WRITE : VMBUS_PAGE_NOT_VISIBLE;
> +
> +	return __hv_set_mem_host_visibility((void *)addr, numpages, visibility)=
;
> +}

Is there a need for this wrapper function?  Couldn't the handling of the ho=
st
visibility enum be folded into __hv_set_mem_host_visibility() and the initi=
al
double underscore dropped?  Maybe I missed it, but I don't see that
__hv_set_mem_host_visibility() is called anyplace else.   Just trying to av=
oid
complexity if it isn't really needed.

> diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hy=
perv-tlfs.h
> index 2322d6bd5883..381e88122a5f 100644
> --- a/arch/x86/include/asm/hyperv-tlfs.h
> +++ b/arch/x86/include/asm/hyperv-tlfs.h
> @@ -276,6 +276,23 @@ enum hv_isolation_type {
>  #define HV_X64_MSR_TIME_REF_COUNT	HV_REGISTER_TIME_REF_COUNT
>  #define HV_X64_MSR_REFERENCE_TSC	HV_REGISTER_REFERENCE_TSC
>=20
> +/* Hyper-V memory host visibility */
> +enum hv_mem_host_visibility {
> +	VMBUS_PAGE_NOT_VISIBLE		=3D 0,
> +	VMBUS_PAGE_VISIBLE_READ_ONLY	=3D 1,
> +	VMBUS_PAGE_VISIBLE_READ_WRITE	=3D 3
> +};
> +
> +/* HvCallModifySparseGpaPageHostVisibility hypercall */
> +#define HV_MAX_MODIFY_GPA_REP_COUNT	((PAGE_SIZE / sizeof(u64)) - 2)
> +struct hv_gpa_range_for_visibility {
> +	u64 partition_id;
> +	u32 host_visibility:2;
> +	u32 reserved0:30;
> +	u32 reserved1;
> +	u64 gpa_page_list[HV_MAX_MODIFY_GPA_REP_COUNT];
> +} __packed;
> +
>  /*
>   * Declare the MSR used to setup pages used to communicate with the hype=
rvisor.
>   */
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index 37739a277ac6..ffb2af079c6b 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -192,7 +192,9 @@ struct irq_domain *hv_create_pci_msi_domain(void);
>  int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vec=
tor,
>  		struct hv_interrupt_entry *entry);
>  int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *=
entry);
> -
> +int hv_mark_gpa_visibility(u16 count, const u64 pfn[],
> +			   enum hv_mem_host_visibility visibility);
> +int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible);
>  #else /* CONFIG_HYPERV */
>  static inline void hyperv_init(void) {}
>  static inline void hyperv_setup_mmu_ops(void) {}
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index ad8a5c586a35..1e4a0882820a 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -29,6 +29,8 @@
>  #include <asm/proto.h>
>  #include <asm/memtype.h>
>  #include <asm/set_memory.h>
> +#include <asm/hyperv-tlfs.h>
> +#include <asm/mshyperv.h>
>=20
>  #include "../mm_internal.h"
>=20
> @@ -1980,15 +1982,11 @@ int set_memory_global(unsigned long addr, int num=
pages)
>  				    __pgprot(_PAGE_GLOBAL), 0);
>  }
>=20
> -static int __set_memory_enc_dec(unsigned long addr, int numpages, bool e=
nc)
> +static int __set_memory_enc_pgtable(unsigned long addr, int numpages, bo=
ol enc)
>  {
>  	struct cpa_data cpa;
>  	int ret;
>=20
> -	/* Nothing to do if memory encryption is not active */
> -	if (!mem_encrypt_active())
> -		return 0;
> -
>  	/* Should not be working on unaligned addresses */
>  	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
>  		addr &=3D PAGE_MASK;
> @@ -2023,6 +2021,17 @@ static int __set_memory_enc_dec(unsigned long addr=
, int numpages, bool enc)
>  	return ret;
>  }
>=20
> +static int __set_memory_enc_dec(unsigned long addr, int numpages, bool e=
nc)
> +{
> +	if (hv_is_isolation_supported())
> +		return hv_set_mem_host_visibility(addr, numpages, !enc);
> +
> +	if (mem_encrypt_active())
> +		return __set_memory_enc_pgtable(addr, numpages, enc);
> +
> +	return 0;
> +}
> +
>  int set_memory_encrypted(unsigned long addr, int numpages)
>  {
>  	return __set_memory_enc_dec(addr, numpages, true);
> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hype=
rv-tlfs.h
> index 56348a541c50..8ed6733d5146 100644
> --- a/include/asm-generic/hyperv-tlfs.h
> +++ b/include/asm-generic/hyperv-tlfs.h
> @@ -158,6 +158,7 @@ struct ms_hyperv_tsc_page {
>  #define HVCALL_RETARGET_INTERRUPT		0x007e
>  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
>  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
> +#define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db
>=20
>  /* Extended hypercalls */
>  #define HV_EXT_CALL_QUERY_CAPABILITIES		0x8001
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index 7537ae1db828..aa55447b9700 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -254,6 +254,7 @@ bool hv_query_ext_cap(u64 cap_query);
>  static inline bool hv_is_hyperv_initialized(void) { return false; }
>  static inline bool hv_is_hibernation_supported(void) { return false; }
>  static inline void hyperv_cleanup(void) {}
> +static inline hv_is_isolation_supported(void);
>  #endif /* CONFIG_HYPERV */
>=20
>  #endif
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:17:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:17:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176731.321629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaQG-0001Kd-77; Thu, 02 Sep 2021 00:17:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176731.321629; Thu, 02 Sep 2021 00:17:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaQG-0001KW-39; Thu, 02 Sep 2021 00:17:40 +0000
Received: by outflank-mailman (input) for mailman id 176731;
 Thu, 02 Sep 2021 00:17:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaQF-0001KJ-7m
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:17:39 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.13])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 998183fd-87cc-4b4e-8529-d0b98a59f69d;
 Thu, 02 Sep 2021 00:17:36 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by CO1PR21MB1314.namprd21.prod.outlook.com (2603:10b6:303:151::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4; Thu, 2 Sep
 2021 00:17:22 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:17:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 998183fd-87cc-4b4e-8529-d0b98a59f69d
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LfmIQHlIMHkqsDEAx8MQ1NAfCm8Ro4Ky/wpN3QRZZqC+gl24I3fBOBpaAoJ276Zn4/+W7dy3A4lBJqQ62Whv2d3K48CbNH7Rhos//THmKEOs2CH7ppX0wYO9pTwByCirxd7peIwJwAfWE5182t9MkY34906BI5iNMD6h2WrRZV194+8iA4ckgtIAgH9afbS7KEp6JUx2KXZVhvFMGdYFEU6DIaDUYaSI0WTmQzR57bo04IgRksMlZdnCmswT659u5mbwWNnKf3/v3UZtFlUGgqUemB42CjLGGy8qWMVHgPynoYB1PZ598QH60CxFJTc9m3vcNrrxayo8G3MO3gcnCQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=owhgNgpehRNG4rXYBOXS5bq3vm/tUORmSSFhaZC5C1c=;
 b=J2j5p1zJj6tGbpheajl6tqdPVzbC73oG0tZiZKx1v3FPAHaRsjaUVrpBRw9iwA1npsTobm9MyZgTr9x3WIrOSosPbbar6hb8buls0dvgQlzGachFJosDiQMMwSIF+MCh/CY5Oi9SmDCM+8cbfmRSDTbMb7mHuYS0AELmkGpZX90DB6Mt4O/SZLCe/hr105Yl0c1mRR4V4riLQbDTB3TjyYMCMYMsVgmFAw9IKs6dpWnzV8vyMWyc+lqawHiGXawRyYftzdZ9JCC2/CdR3I+Pgl5RKSB/2uqti0dLVjHFpQr6yOZKPzdBh8JLBbRcVoZbga1ULR64YAqPePd42YNDXg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=owhgNgpehRNG4rXYBOXS5bq3vm/tUORmSSFhaZC5C1c=;
 b=ZskhXqZeDDAqP6hth/AVqP2hJ6V/pcbzmclUYaXy0ADUh1saL12nGxB4dWFO/MWBYPZYCeVYgOheGEye6uf+eTrppAbd31RRqUORI3N6aLT3qWwiCOWk0qMULnCsK9RhX41zy7/2vLb7BADf+XqyPZaE+Iv/otMaNSzzRljSsKg=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 04/13] hyperv: Mark vmbus ring buffer visible to host
 in Isolation VM
Thread-Topic: [PATCH V4 04/13] hyperv: Mark vmbus ring buffer visible to host
 in Isolation VM
Thread-Index: AQHXm2gHOMlvw7enDEWSDoli9Xi0FquJEt7Q
Date: Thu, 2 Sep 2021 00:17:21 +0000
Message-ID:
 <MWHPR21MB1593907C65C249D00F1A5717D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-5-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-5-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=8b920a56-6992-44f3-a51e-d6ed365110c4;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-28T15:53:27Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4cf4ec66-38f2-483e-19b0-08d96da708a2
x-ms-traffictypediagnostic: CO1PR21MB1314:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <CO1PR21MB131475BC7B28CED23E9D2903D7CE9@CO1PR21MB1314.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:56;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 JS2keamfM0jx37N7WlgsOSbQHuA0Wn2B+meaecDh668t0uFHu1b6F9vaDvTBLx2VKFqIXQux4S7RdVg5kLwZAzhkddRHN09fdW0rHMs/zwus14Ri4c2uJZAjZNiJJxon53OVVyCnwptUWdC3djGkubA5Iufw+jkPnEqoh2bcjvVZwaJ3a/Spocbf0Z/3W+RTTnAph5VpUoFZW7YcPIUrJe/SdC+L54R1e+bf+r1PkjfCmpcyRDbOZWqSiQSuxWVcbOjJCizQpivEiGiZn3Z0/wpe/hEqtsUKigJRtrud00EoMtoOe7Rew3IEzgVp3SFOzaJaS2L8FQG64zrGBx0ZjbkSn7g5GZYkw4onVe2qtl7nIpkwZC5NAanU2BykM9Zw9nOUzo6RYgFKKtHaEJRJigK8FZccbFPztem2qGOaY+1MJTbSTAragATuoKZ+sz0OhzdupvCIZSb2WBMVG2MbmBGBjhZIjrq8oFj7vNxrPq+8Ad9DzLBivoyGQwKcybt7VHCSw8WwnfKpIc4kczVKYt2owMPsBvIxAoRS9dCpAbMxBVb3GZ3+NmD3iTXbDH8fptRTaCRYgnxB7Hgl4YNWId9Zc77Gcz9Oxk15pklkdCcwpAHlccCF8LsWK4fCFdEN98mOTflZE9UBbLCfDdBWEQuIa/E1jXCYSHt5xDO0T5gHlrFIQcoqLGXCmogNYVv5l0fDox/iMA7wNJEnBou53cOPq2lw5w8WxLGfarsrFWY=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(10290500003)(7696005)(186003)(66946007)(6506007)(76116006)(38100700002)(8676002)(110136005)(9686003)(316002)(54906003)(71200400001)(38070700005)(8990500004)(86362001)(55016002)(2906002)(122000001)(508600001)(8936002)(4326008)(7406005)(5660300002)(7416002)(7366002)(33656002)(52536014)(83380400001)(82950400001)(30864003)(66556008)(82960400001)(921005)(66476007)(66446008)(64756008)(579004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?myQ8QsXlfAfVc23WymszcgcfUlIfJShxhHjbJ2Aib3aynOpa5jp1m09TdBOP?=
 =?us-ascii?Q?44/cyx/EUsQn02ziq8stvkYyA2AH219MYTbHgkZ0PQO4uphUjo+EoK/6jX93?=
 =?us-ascii?Q?/8+oVNcd0ZzfI8oDPDi4KFFKM+NFIB79Ft6rQpuGGERSF2M9GJajNTIRyu+L?=
 =?us-ascii?Q?1uRGT6XJsA9+5xtqvTSffvm6KHvwD/L0Fk+XLjyzzqBJITEn01Ut/oB8ragL?=
 =?us-ascii?Q?Pe+gfGJlxHYB2GRd1yhJIESJhcX9jO9rIHeeNHguyzgosvuXiY5Kk98mKfqD?=
 =?us-ascii?Q?jnnrbNrvyu1KAQSh3O43imVdxYZVhcmTXw36md5kdZJr0BOJI5J2a+Bnzutj?=
 =?us-ascii?Q?x8Bcqs2PRM2nLaRVLxbU4VHAwd1Oc/pZLE1NimqV1Hrk+G18FlP7b6Hcg1iT?=
 =?us-ascii?Q?B41PuTjVN0HgdNDwzLbtAMjdpYkTYqKRX9Wu0NZnPgAboV8jKSZ8y55xPb9J?=
 =?us-ascii?Q?jYifbwG3d/3ObjcplK7DZIks7DZHsJmRKD7ZTD0L3ESwUMPRS5x9q1jSKUvu?=
 =?us-ascii?Q?IxPHYsR3u2mUH83T4g3F9l47fKXzRGyGB0kHvXiAEU+osfPJh4qGiKiKKLoB?=
 =?us-ascii?Q?u8PO4EDKyBxifNj43NynqQrRaivVnclmt1pvmGeMOOa6v/2c9zVQv4FSXWtK?=
 =?us-ascii?Q?96kaxaam2QlwCfG0kkO51XU7gyKY75KQWnCcntCuK1ONzTJY/8xBXxjw6LDG?=
 =?us-ascii?Q?R9mp2ZsMDUfoJ6pZuMGsczXz5TO1MqYX+ctwvqAMehPtgzigJ4mlXnfkdl+q?=
 =?us-ascii?Q?fG7LSyRXQtKq+gv5Gt2z+BmRgKLPMySLDihIpPqGVdsgh9hNuiwkv9C3bEbI?=
 =?us-ascii?Q?bhNdVeGjZVjKE0408/GwvocD2kymDt0j/+0DIfZu70ZcNoewYNPugEh9fNDG?=
 =?us-ascii?Q?Pi5QQZ8MRjqAlYBk/P2E6/pkwjIxxqLcbK2Oe6c1RqUaK0n2ix/F3e8OydKm?=
 =?us-ascii?Q?i/8FgDCGm+x2AIodelpK2kJlusSYbi9Sg4Y1Ti66LnG1si8mM3wf4V5R5U7w?=
 =?us-ascii?Q?AfqyAEhVwXJgTlZP7ckgQREkdKmpqp4qdott4zlHCxVYQAieh28g+Q17pEfC?=
 =?us-ascii?Q?lYm8rmVfpgxTLCJfM/8I1pOCLh0Nxj/Z6K9ThcY4wmqlKE3pbayzJ6+rwXrN?=
 =?us-ascii?Q?iiY+Ke6HBCWlHS4ZiYUsdqGGDFmZcqRBYEggP4RvI2NfsHaX8jBpzEY5XJOV?=
 =?us-ascii?Q?ZN7D50242CoX0WGg5ksLuGhXcTQo3GUnn1qwBft1qVmXW7hQVqUDMfG7Tufz?=
 =?us-ascii?Q?K+KjAfUJpRQucjdmLeoeEv3zKMTw1x5vYTkgAmexLc43Sko03RTL7tY3S3Z1?=
 =?us-ascii?Q?nvWdQTsTDtPen+Cxg/Z9ym3o?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4cf4ec66-38f2-483e-19b0-08d96da708a2
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:17:21.8744
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: RvDiBlbiojDhzzfcbdOFBQye/Pw0hBMnWGLgEzVinucglU+0phNLl1XfUwN/S83lBQSENW2MEid4+ZQm1OH2Wai4GCLnrqZil8MQ9Y0w9dU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR21MB1314

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> Mark vmbus ring buffer visible with set_memory_decrypted() when
> establish gpadl handle.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
>        * Change vmbus_teardown_gpadl() parameter and put gpadl handle,
>        buffer and buffer size in the struct vmbus_gpadl.
> ---
>  drivers/hv/channel.c            | 36 ++++++++++++++++++++++++++++-----
>  drivers/net/hyperv/hyperv_net.h |  1 +
>  drivers/net/hyperv/netvsc.c     | 16 +++++++++++----
>  drivers/uio/uio_hv_generic.c    | 14 +++++++++++--
>  include/linux/hyperv.h          |  8 +++++++-
>  5 files changed, 63 insertions(+), 12 deletions(-)
>=20
> diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
> index f3761c73b074..82650beb3af0 100644
> --- a/drivers/hv/channel.c
> +++ b/drivers/hv/channel.c
> @@ -17,6 +17,7 @@
>  #include <linux/hyperv.h>
>  #include <linux/uio.h>
>  #include <linux/interrupt.h>
> +#include <linux/set_memory.h>
>  #include <asm/page.h>
>  #include <asm/mshyperv.h>
>=20
> @@ -474,6 +475,13 @@ static int __vmbus_establish_gpadl(struct vmbus_chan=
nel *channel,
>  	if (ret)
>  		return ret;
>=20
> +	ret =3D set_memory_decrypted((unsigned long)kbuffer,
> +				   HVPFN_UP(size));
> +	if (ret) {
> +		pr_warn("Failed to set host visibility for new GPADL %d.\n", ret);
> +		return ret;
> +	}
> +
>  	init_completion(&msginfo->waitevent);
>  	msginfo->waiting_channel =3D channel;
>=20
> @@ -549,6 +557,11 @@ static int __vmbus_establish_gpadl(struct vmbus_chan=
nel *channel,
>  	}
>=20
>  	kfree(msginfo);
> +
> +	if (ret)
> +		set_memory_encrypted((unsigned long)kbuffer,
> +				     HVPFN_UP(size));
> +
>  	return ret;
>  }
>=20
> @@ -639,6 +652,7 @@ static int __vmbus_open(struct vmbus_channel *newchan=
nel,
>  	struct vmbus_channel_open_channel *open_msg;
>  	struct vmbus_channel_msginfo *open_info =3D NULL;
>  	struct page *page =3D newchannel->ringbuffer_page;
> +	struct vmbus_gpadl gpadl;
>  	u32 send_pages, recv_pages;
>  	unsigned long flags;
>  	int err;
> @@ -759,7 +773,10 @@ static int __vmbus_open(struct vmbus_channel *newcha=
nnel,
>  error_free_info:
>  	kfree(open_info);
>  error_free_gpadl:
> -	vmbus_teardown_gpadl(newchannel, newchannel->ringbuffer_gpadlhandle);
> +	gpadl.gpadl_handle =3D newchannel->ringbuffer_gpadlhandle;
> +	gpadl.buffer =3D page_address(newchannel->ringbuffer_page);
> +	gpadl.size =3D (send_pages + recv_pages) << PAGE_SHIFT;
> +	vmbus_teardown_gpadl(newchannel, &gpadl);
>  	newchannel->ringbuffer_gpadlhandle =3D 0;
>  error_clean_ring:
>  	hv_ringbuffer_cleanup(&newchannel->outbound);
> @@ -806,7 +823,7 @@ EXPORT_SYMBOL_GPL(vmbus_open);
>  /*
>   * vmbus_teardown_gpadl -Teardown the specified GPADL handle
>   */
> -int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle=
)
> +int vmbus_teardown_gpadl(struct vmbus_channel *channel, struct vmbus_gpa=
dl *gpadl)
>  {
>  	struct vmbus_channel_gpadl_teardown *msg;
>  	struct vmbus_channel_msginfo *info;
> @@ -825,7 +842,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channe=
l, u32 gpadl_handle)
>=20
>  	msg->header.msgtype =3D CHANNELMSG_GPADL_TEARDOWN;
>  	msg->child_relid =3D channel->offermsg.child_relid;
> -	msg->gpadl =3D gpadl_handle;
> +	msg->gpadl =3D gpadl->gpadl_handle;
>=20
>  	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
>  	list_add_tail(&info->msglistentry,
> @@ -859,6 +876,12 @@ int vmbus_teardown_gpadl(struct vmbus_channel *chann=
el, u32 gpadl_handle)
>  	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
>=20
>  	kfree(info);
> +
> +	ret =3D set_memory_encrypted((unsigned long)gpadl->buffer,
> +				   HVPFN_UP(gpadl->size));
> +	if (ret)
> +		pr_warn("Fail to set mem host visibility in GPADL teardown %d.\n", ret=
);
> +
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vmbus_teardown_gpadl);
> @@ -896,6 +919,7 @@ void vmbus_reset_channel_cb(struct vmbus_channel *cha=
nnel)
>  static int vmbus_close_internal(struct vmbus_channel *channel)
>  {
>  	struct vmbus_channel_close_channel *msg;
> +	struct vmbus_gpadl gpadl;
>  	int ret;
>=20
>  	vmbus_reset_channel_cb(channel);
> @@ -934,8 +958,10 @@ static int vmbus_close_internal(struct vmbus_channel=
 *channel)
>=20
>  	/* Tear down the gpadl for the channel's ring buffer */
>  	else if (channel->ringbuffer_gpadlhandle) {
> -		ret =3D vmbus_teardown_gpadl(channel,
> -					   channel->ringbuffer_gpadlhandle);
> +		gpadl.gpadl_handle =3D channel->ringbuffer_gpadlhandle;
> +		gpadl.buffer =3D page_address(channel->ringbuffer_page);
> +		gpadl.size =3D channel->ringbuffer_pagecount;
> +		ret =3D vmbus_teardown_gpadl(channel, &gpadl);
>  		if (ret) {
>  			pr_err("Close failed: teardown gpadl return %d\n", ret);
>  			/*
> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_=
net.h
> index bc48855dff10..aa7c9962dbd8 100644
> --- a/drivers/net/hyperv/hyperv_net.h
> +++ b/drivers/net/hyperv/hyperv_net.h
> @@ -1082,6 +1082,7 @@ struct netvsc_device {
>=20
>  	/* Send buffer allocated by us */
>  	void *send_buf;
> +	u32 send_buf_size;
>  	u32 send_buf_gpadl_handle;
>  	u32 send_section_cnt;
>  	u32 send_section_size;
> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
> index 7bd935412853..f19bffff6a63 100644
> --- a/drivers/net/hyperv/netvsc.c
> +++ b/drivers/net/hyperv/netvsc.c
> @@ -276,11 +276,14 @@ static void netvsc_teardown_recv_gpadl(struct hv_de=
vice *device,
>  				       struct netvsc_device *net_device,
>  				       struct net_device *ndev)
>  {
> +	struct vmbus_gpadl gpadl;
>  	int ret;
>=20
>  	if (net_device->recv_buf_gpadl_handle) {
> -		ret =3D vmbus_teardown_gpadl(device->channel,
> -					   net_device->recv_buf_gpadl_handle);
> +		gpadl.gpadl_handle =3D net_device->recv_buf_gpadl_handle;
> +		gpadl.buffer =3D net_device->recv_buf;
> +		gpadl.size =3D net_device->recv_buf_size;
> +		ret =3D vmbus_teardown_gpadl(device->channel, &gpadl);
>=20
>  		/* If we failed here, we might as well return and have a leak
>  		 * rather than continue and a bugchk
> @@ -298,11 +301,15 @@ static void netvsc_teardown_send_gpadl(struct hv_de=
vice *device,
>  				       struct netvsc_device *net_device,
>  				       struct net_device *ndev)
>  {
> +	struct vmbus_gpadl gpadl;
>  	int ret;
>=20
>  	if (net_device->send_buf_gpadl_handle) {
> -		ret =3D vmbus_teardown_gpadl(device->channel,
> -					   net_device->send_buf_gpadl_handle);
> +		gpadl.gpadl_handle =3D net_device->send_buf_gpadl_handle;
> +		gpadl.buffer =3D net_device->send_buf;
> +		gpadl.size =3D net_device->send_buf_size;
> +
> +		ret =3D vmbus_teardown_gpadl(device->channel, &gpadl);
>=20
>  		/* If we failed here, we might as well return and have a leak
>  		 * rather than continue and a bugchk
> @@ -463,6 +470,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  		ret =3D -ENOMEM;
>  		goto cleanup;
>  	}
> +	net_device->send_buf_size =3D buf_size;
>=20
>  	/* Establish the gpadl handle for this buffer on this
>  	 * channel.  Note: This call uses the vmbus connection rather
> diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c
> index 652fe2547587..13c5df8dd11d 100644
> --- a/drivers/uio/uio_hv_generic.c
> +++ b/drivers/uio/uio_hv_generic.c
> @@ -179,14 +179,24 @@ hv_uio_new_channel(struct vmbus_channel *new_sc)
>  static void
>  hv_uio_cleanup(struct hv_device *dev, struct hv_uio_private_data *pdata)
>  {
> +	struct vmbus_gpadl gpadl;
> +
>  	if (pdata->send_gpadl) {
> -		vmbus_teardown_gpadl(dev->channel, pdata->send_gpadl);
> +		gpadl.gpadl_handle =3D pdata->send_gpadl;
> +		gpadl.buffer =3D pdata->send_buf;
> +		gpadl.size =3D SEND_BUFFER_SIZE;
> +
> +		vmbus_teardown_gpadl(dev->channel, &gpadl);
>  		pdata->send_gpadl =3D 0;
>  		vfree(pdata->send_buf);
>  	}
>=20
>  	if (pdata->recv_gpadl) {
> -		vmbus_teardown_gpadl(dev->channel, pdata->recv_gpadl);
> +		gpadl.gpadl_handle =3D pdata->recv_gpadl;
> +		gpadl.buffer =3D pdata->recv_buf;
> +		gpadl.size =3D RECV_BUFFER_SIZE;
> +
> +		vmbus_teardown_gpadl(dev->channel, &gpadl);
>  		pdata->recv_gpadl =3D 0;
>  		vfree(pdata->recv_buf);
>  	}
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index ddc8713ce57b..757e09606fd3 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -803,6 +803,12 @@ struct vmbus_device {
>=20
>  #define VMBUS_DEFAULT_MAX_PKT_SIZE 4096
>=20
> +struct vmbus_gpadl {
> +	u32 gpadl_handle;
> +	u32 size;
> +	void *buffer;
> +};
> +
>  struct vmbus_channel {
>  	struct list_head listentry;
>=20
> @@ -1195,7 +1201,7 @@ extern int vmbus_establish_gpadl(struct vmbus_chann=
el *channel,
>  				      u32 *gpadl_handle);
>=20
>  extern int vmbus_teardown_gpadl(struct vmbus_channel *channel,
> -				     u32 gpadl_handle);
> +				     struct vmbus_gpadl *gpadl);
>=20
>  void vmbus_reset_channel_cb(struct vmbus_channel *channel);
>=20
> --
> 2.25.1

This isn't quite what I had in mind in my comments on v3 of this
patch series.  My idea is to store the full struct vmbus_gpadl
data structure in places where previously just the
u32 gpadl_handle was stored.  Then pass around a pointer to the
struct vmbus_gpadl where previously just the gpadl_handle (or a
pointer to it) was passed. This lets __vmbus_establish_gpadl()
fill in the actual handle value as well the other info (buffer pointer
and size) that vmbus_teardown_gpadl() needs.  Callers of the
gpadl functions don't need to worry about saving or finding
the right info.  Most of the changes are just tweaking the references
to what is now a struct instead of a u32. =20

Here's a diff of what I had in mind.  My version also has
vmbus_teardown_gpadl() set the handle field to zero, rather than
each caller having to do it.  The code compiles, but I
have not done a runtime test.  This diff is a net +21 lines of code,
whereas your v3 and v4 patches were both +51 lines of code.

diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index f3761c7..fc041ae 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -17,6 +17,7 @@
 #include <linux/hyperv.h>
 #include <linux/uio.h>
 #include <linux/interrupt.h>
+#include <linux/set_memory.h>
 #include <asm/page.h>
 #include <asm/mshyperv.h>
=20
@@ -456,7 +457,7 @@ static int create_gpadl_header(enum hv_gpadl_type type,=
 void *kbuffer,
 static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
 				   enum hv_gpadl_type type, void *kbuffer,
 				   u32 size, u32 send_offset,
-				   u32 *gpadl_handle)
+				   struct vmbus_gpadl *gpadl_handle)
 {
 	struct vmbus_channel_gpadl_header *gpadlmsg;
 	struct vmbus_channel_gpadl_body *gpadl_body;
@@ -474,6 +475,13 @@ static int __vmbus_establish_gpadl(struct vmbus_channe=
l *channel,
 	if (ret)
 		return ret;
=20
+	ret =3D set_memory_decrypted((unsigned long)kbuffer,
+				   HVPFN_UP(size));
+	if (ret) {
+		pr_warn("Failed to set host visibility for new GPADL %d.\n", ret);
+		return ret;
+	}
+
 	init_completion(&msginfo->waitevent);
 	msginfo->waiting_channel =3D channel;
=20
@@ -537,7 +545,9 @@ static int __vmbus_establish_gpadl(struct vmbus_channel=
 *channel,
 	}
=20
 	/* At this point, we received the gpadl created msg */
-	*gpadl_handle =3D gpadlmsg->gpadl;
+	gpadl_handle->handle =3D gpadlmsg->gpadl;
+	gpadl_handle->buffer =3D kbuffer;
+	gpadl_handle->size =3D size;
=20
 cleanup:
 	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
@@ -549,6 +559,11 @@ static int __vmbus_establish_gpadl(struct vmbus_channe=
l *channel,
 	}
=20
 	kfree(msginfo);
+
+	if (ret)
+		set_memory_encrypted((unsigned long)kbuffer,
+				     HVPFN_UP(size));
+
 	return ret;
 }
=20
@@ -561,7 +576,7 @@ static int __vmbus_establish_gpadl(struct vmbus_channel=
 *channel,
  * @gpadl_handle: some funky thing
  */
 int vmbus_establish_gpadl(struct vmbus_channel *channel, void *kbuffer,
-			  u32 size, u32 *gpadl_handle)
+			  u32 size, struct vmbus_gpadl *gpadl_handle)
 {
 	return __vmbus_establish_gpadl(channel, HV_GPADL_BUFFER, kbuffer, size,
 				       0U, gpadl_handle);
@@ -675,7 +690,7 @@ static int __vmbus_open(struct vmbus_channel *newchanne=
l,
 		goto error_clean_ring;
=20
 	/* Establish the gpadl for the ring buffer */
-	newchannel->ringbuffer_gpadlhandle =3D 0;
+	newchannel->ringbuffer_gpadlhandle.handle =3D 0;
=20
 	err =3D __vmbus_establish_gpadl(newchannel, HV_GPADL_RING,
 				      page_address(newchannel->ringbuffer_page),
@@ -701,7 +716,7 @@ static int __vmbus_open(struct vmbus_channel *newchanne=
l,
 	open_msg->header.msgtype =3D CHANNELMSG_OPENCHANNEL;
 	open_msg->openid =3D newchannel->offermsg.child_relid;
 	open_msg->child_relid =3D newchannel->offermsg.child_relid;
-	open_msg->ringbuffer_gpadlhandle =3D newchannel->ringbuffer_gpadlhandle;
+	open_msg->ringbuffer_gpadlhandle =3D newchannel->ringbuffer_gpadlhandle.h=
andle;
 	/*
 	 * The unit of ->downstream_ringbuffer_pageoffset is HV_HYP_PAGE and
 	 * the unit of ->ringbuffer_send_offset (i.e. send_pages) is PAGE, so
@@ -759,8 +774,7 @@ static int __vmbus_open(struct vmbus_channel *newchanne=
l,
 error_free_info:
 	kfree(open_info);
 error_free_gpadl:
-	vmbus_teardown_gpadl(newchannel, newchannel->ringbuffer_gpadlhandle);
-	newchannel->ringbuffer_gpadlhandle =3D 0;
+	vmbus_teardown_gpadl(newchannel, &newchannel->ringbuffer_gpadlhandle);
 error_clean_ring:
 	hv_ringbuffer_cleanup(&newchannel->outbound);
 	hv_ringbuffer_cleanup(&newchannel->inbound);
@@ -806,7 +820,7 @@ int vmbus_open(struct vmbus_channel *newchannel,
 /*
  * vmbus_teardown_gpadl -Teardown the specified GPADL handle
  */
-int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)
+int vmbus_teardown_gpadl(struct vmbus_channel *channel, struct vmbus_gpadl=
 *gpadl)
 {
 	struct vmbus_channel_gpadl_teardown *msg;
 	struct vmbus_channel_msginfo *info;
@@ -825,7 +839,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel,=
 u32 gpadl_handle)
=20
 	msg->header.msgtype =3D CHANNELMSG_GPADL_TEARDOWN;
 	msg->child_relid =3D channel->offermsg.child_relid;
-	msg->gpadl =3D gpadl_handle;
+	msg->gpadl =3D gpadl->handle;
=20
 	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
 	list_add_tail(&info->msglistentry,
@@ -844,6 +858,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel,=
 u32 gpadl_handle)
 		goto post_msg_err;
=20
 	wait_for_completion(&info->waitevent);
+	gpadl->handle =3D 0;
=20
 post_msg_err:
 	/*
@@ -859,6 +874,12 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel=
, u32 gpadl_handle)
 	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
=20
 	kfree(info);
+
+	ret =3D set_memory_encrypted((unsigned long)gpadl->buffer,
+				   HVPFN_UP(gpadl->size));
+	if (ret)
+		pr_warn("Fail to set mem host visibility in GPADL teardown %d.\n", ret);
+
 	return ret;
 }
 EXPORT_SYMBOL_GPL(vmbus_teardown_gpadl);
@@ -933,9 +954,9 @@ static int vmbus_close_internal(struct vmbus_channel *c=
hannel)
 	}
=20
 	/* Tear down the gpadl for the channel's ring buffer */
-	else if (channel->ringbuffer_gpadlhandle) {
+	else if (channel->ringbuffer_gpadlhandle.handle) {
 		ret =3D vmbus_teardown_gpadl(channel,
-					   channel->ringbuffer_gpadlhandle);
+					   &channel->ringbuffer_gpadlhandle);
 		if (ret) {
 			pr_err("Close failed: teardown gpadl return %d\n", ret);
 			/*
@@ -943,8 +964,6 @@ static int vmbus_close_internal(struct vmbus_channel *c=
hannel)
 			 * it is perhaps better to leak memory.
 			 */
 		}
-
-		channel->ringbuffer_gpadlhandle =3D 0;
 	}
=20
 	if (!ret)
diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_ne=
t.h
index bc48855..54cbce1 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -1075,14 +1075,14 @@ struct netvsc_device {
 	/* Receive buffer allocated by us but manages by NetVSP */
 	void *recv_buf;
 	u32 recv_buf_size; /* allocated bytes */
-	u32 recv_buf_gpadl_handle;
+	struct vmbus_gpadl recv_buf_gpadl_handle;
 	u32 recv_section_cnt;
 	u32 recv_section_size;
 	u32 recv_completion_cnt;
=20
 	/* Send buffer allocated by us */
 	void *send_buf;
-	u32 send_buf_gpadl_handle;
+	struct vmbus_gpadl send_buf_gpadl_handle;
 	u32 send_section_cnt;
 	u32 send_section_size;
 	unsigned long *send_section_map;
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index 7bd9354..585974c 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -278,9 +278,9 @@ static void netvsc_teardown_recv_gpadl(struct hv_device=
 *device,
 {
 	int ret;
=20
-	if (net_device->recv_buf_gpadl_handle) {
+	if (net_device->recv_buf_gpadl_handle.handle) {
 		ret =3D vmbus_teardown_gpadl(device->channel,
-					   net_device->recv_buf_gpadl_handle);
+					   &net_device->recv_buf_gpadl_handle);
=20
 		/* If we failed here, we might as well return and have a leak
 		 * rather than continue and a bugchk
@@ -290,7 +290,6 @@ static void netvsc_teardown_recv_gpadl(struct hv_device=
 *device,
 				   "unable to teardown receive buffer's gpadl\n");
 			return;
 		}
-		net_device->recv_buf_gpadl_handle =3D 0;
 	}
 }
=20
@@ -300,9 +299,9 @@ static void netvsc_teardown_send_gpadl(struct hv_device=
 *device,
 {
 	int ret;
=20
-	if (net_device->send_buf_gpadl_handle) {
+	if (net_device->send_buf_gpadl_handle.handle) {
 		ret =3D vmbus_teardown_gpadl(device->channel,
-					   net_device->send_buf_gpadl_handle);
+					   &net_device->send_buf_gpadl_handle);
=20
 		/* If we failed here, we might as well return and have a leak
 		 * rather than continue and a bugchk
@@ -312,7 +311,6 @@ static void netvsc_teardown_send_gpadl(struct hv_device=
 *device,
 				   "unable to teardown send buffer's gpadl\n");
 			return;
 		}
-		net_device->send_buf_gpadl_handle =3D 0;
 	}
 }
=20
@@ -380,7 +378,7 @@ static int netvsc_init_buf(struct hv_device *device,
 	memset(init_packet, 0, sizeof(struct nvsp_message));
 	init_packet->hdr.msg_type =3D NVSP_MSG1_TYPE_SEND_RECV_BUF;
 	init_packet->msg.v1_msg.send_recv_buf.
-		gpadl_handle =3D net_device->recv_buf_gpadl_handle;
+		gpadl_handle =3D net_device->recv_buf_gpadl_handle.handle;
 	init_packet->msg.v1_msg.
 		send_recv_buf.id =3D NETVSC_RECEIVE_BUFFER_ID;
=20
@@ -482,7 +480,7 @@ static int netvsc_init_buf(struct hv_device *device,
 	memset(init_packet, 0, sizeof(struct nvsp_message));
 	init_packet->hdr.msg_type =3D NVSP_MSG1_TYPE_SEND_SEND_BUF;
 	init_packet->msg.v1_msg.send_send_buf.gpadl_handle =3D
-		net_device->send_buf_gpadl_handle;
+		net_device->send_buf_gpadl_handle.handle;
 	init_packet->msg.v1_msg.send_send_buf.id =3D NETVSC_SEND_BUFFER_ID;
=20
 	trace_nvsp_send(ndev, init_packet);
diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c
index 652fe25..97e08e7 100644
--- a/drivers/uio/uio_hv_generic.c
+++ b/drivers/uio/uio_hv_generic.c
@@ -58,11 +58,11 @@ struct hv_uio_private_data {
 	atomic_t refcnt;
=20
 	void	*recv_buf;
-	u32	recv_gpadl;
+	struct vmbus_gpadl recv_gpadl;
 	char	recv_name[32];	/* "recv_4294967295" */
=20
 	void	*send_buf;
-	u32	send_gpadl;
+	struct vmbus_gpadl send_gpadl;
 	char	send_name[32];
 };
=20
@@ -179,15 +179,13 @@ static int hv_uio_ring_mmap(struct file *filp, struct=
 kobject *kobj,
 static void
 hv_uio_cleanup(struct hv_device *dev, struct hv_uio_private_data *pdata)
 {
-	if (pdata->send_gpadl) {
-		vmbus_teardown_gpadl(dev->channel, pdata->send_gpadl);
-		pdata->send_gpadl =3D 0;
+	if (pdata->send_gpadl.handle) {
+		vmbus_teardown_gpadl(dev->channel, &pdata->send_gpadl);
 		vfree(pdata->send_buf);
 	}
=20
-	if (pdata->recv_gpadl) {
-		vmbus_teardown_gpadl(dev->channel, pdata->recv_gpadl);
-		pdata->recv_gpadl =3D 0;
+	if (pdata->recv_gpadl.handle) {
+		vmbus_teardown_gpadl(dev->channel, &pdata->recv_gpadl);
 		vfree(pdata->recv_buf);
 	}
 }
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 2e859d2..a0d64c3 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -809,6 +809,12 @@ struct vmbus_device {
=20
 #define VMBUS_DEFAULT_MAX_PKT_SIZE 4096
=20
+struct vmbus_gpadl {
+	u32 handle;
+	u32 size;
+	void *buffer;
+};
+
 struct vmbus_channel {
 	struct list_head listentry;
=20
@@ -828,7 +834,7 @@ struct vmbus_channel {
 	bool rescind_ref; /* got rescind msg, got channel reference */
 	struct completion rescind_event;
=20
-	u32 ringbuffer_gpadlhandle;
+	struct vmbus_gpadl ringbuffer_gpadlhandle;
=20
 	/* Allocated memory for ring buffer */
 	struct page *ringbuffer_page;
@@ -1208,10 +1214,10 @@ extern int vmbus_sendpacket_mpb_desc(struct vmbus_c=
hannel *channel,
 extern int vmbus_establish_gpadl(struct vmbus_channel *channel,
 				      void *kbuffer,
 				      u32 size,
-				      u32 *gpadl_handle);
+				      struct vmbus_gpadl *gpadl_handle);
=20
 extern int vmbus_teardown_gpadl(struct vmbus_channel *channel,
-				     u32 gpadl_handle);
+				     struct vmbus_gpadl *gpadl);
=20
 void vmbus_reset_channel_cb(struct vmbus_channel *channel);



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:20:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:20:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176738.321639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaT7-0002mB-OT; Thu, 02 Sep 2021 00:20:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176738.321639; Thu, 02 Sep 2021 00:20:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaT7-0002m4-LX; Thu, 02 Sep 2021 00:20:37 +0000
Received: by outflank-mailman (input) for mailman id 176738;
 Thu, 02 Sep 2021 00:20:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaT6-0002ly-6w
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:20:36 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.12])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 95582b6a-0b83-11ec-ae2d-12813bfff9fa;
 Thu, 02 Sep 2021 00:20:31 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1019.namprd21.prod.outlook.com (2603:10b6:302:5::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6; Thu, 2 Sep
 2021 00:20:28 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:20:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95582b6a-0b83-11ec-ae2d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k83v89YVVc0G1XqbkkLj6dfEwTSojS2fSelG4HtWtU1ErvNnbw40jyjzblBtrkNATS2YsG9I2fHM94o5F7gTPrg0ny+D0wMYqlzjt9gVEKHym5j1sMKXGyDhWi/kDeEsFYQbR/32jRqxhltZnFoUBzpHd3jBt9Crv7PeSVZY2sDfC0+PsFzh8Knca7Ecm+Jh6jMQoydr9piR4OHDALSIRw5N+M4xoD5Z6Pj9lMBuzD+8Fpw2wiqUDg86OnmLiQmTaTS1Jfnck5dyum8oSlIv0Ey9pB0g7PTizpg23ILWZn+MIE/Ukoou77lOrbwWbCR3cZKPJWaupYzyYZdWOCQE0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uSAcrrAsaLphArkTow8aPf5W91AfPrV8zL6lQf1PILA=;
 b=HFZXSuxknTm3S2CREWA+2VhMCRMANvuHegWkDgbrBCoSrJkX0DoGM08rOXtpsNkqZKNTyziWu2AZ4gOz5OoohBrZMEVUXilTEHQTtJIpWySnKI/El6N/xYBNjGEiTXx9dyxVp3XvV4Vd8DP/IJxSEoiIrFfc2g1TfoTGUrJr+xVKXB41y33Bv25nQSVK0pWohTmadHb4HKhcaSd+ugHZb0GnVDS07y1noam21lnvriqCH8BnSK57ba8UXmxJ3OhFWV/5B9IlEUxDu1lFJMO+Efub9AkIyrK09eiMi43MgK4gMjvsZmmQyrJYmvfxQoDSWGbCB5xa+lwoWeCsfFllJg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uSAcrrAsaLphArkTow8aPf5W91AfPrV8zL6lQf1PILA=;
 b=AeuGe5LwAEiMrZh5Cu8Mwsy+8XYsLsfMr0CCy6lIyi296lgyc3vps0FxLTvaSFjbS0uYUHIsqc5BI3nSN+2wuVWKzi1ZpmnrMHwQVxvK0do9f0aeRpi+bkoDcm9WY79tnsKMyYxGI5B8AFd3lf9ni2tgjTkIdDkxeN/6sG9Uy+c=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 06/13] hyperv: Add ghcb hvcall support for SNP VM
Thread-Topic: [PATCH V4 06/13] hyperv: Add ghcb hvcall support for SNP VM
Thread-Index: AQHXm2f/0mSKj2OCNkCC+r4DaEQmJquKug9A
Date: Thu, 2 Sep 2021 00:20:28 +0000
Message-ID:
 <MWHPR21MB1593B1F15B489617F39E69F3D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-7-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-7-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=6d580330-91d6-4af5-96cd-a8d1305f2392;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-29T17:08:06Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1ef6e0ca-5957-4140-2dba-08d96da777c1
x-ms-traffictypediagnostic: MW2PR2101MB1019:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB1019C1F8AA667275B576094CD7CE9@MW2PR2101MB1019.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5236;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 hh48EIj+igbz1dMXOksKbSyyFrHFlI5LJftl0O2XFLfLUgWQoUQY+o7dG3cKu31K+5Qw055KqyXURz92W1+2KfZnMKi6fBkhlT1YtjLqke2U2qwjHGTTrk2C8Di8BFqsxE340VRpaRxpLqUa0HNwfZIwB8grGSABzGjQKajnFZfCP6ZMyiDwHyuoJXwLLwhci6qoUTePZTEC7YOsUKGRPT0k02KTDRddNI3ZcRZKo5RAcSVaFb1f9kxbHMJZorxewBJxTO37cJnIHbEYmwP+s9zgRaDY1g/lrLcTaM0ueULKPwYbXsX0bfvc28PRA41gwYNCex3cv1tTrVpfZ6gfA21kBWEuTItaZH832AGHqPTd4HXYVUly7MSKFND45t7piGGv3YvrBaU9vif0f2T88Ld04AXSqXU9+KDHvlS+VoKSSVghPyW+Pnrt3jRZqDIb/dr5PA5B9e97lKPfJn24CKl7vakZYKRSa/W2sLAMXpWskUt9Qb+sWlQxArdxzP5flFgPzL510Wsss17jYSOKbWFIB8nUP5AdgAj+QarEa2vVQ+dCMT8DvNfRJv8Tuo1pBkRMPvw5LgGRpvcMdKdePbIFdBhx5GngE+LZYl8FpWO+lBAr1L6SH88ZwgzZ5enyjAibAaIwOga3Q9w3l5bDgHcXQKiajMIuaUvn8aMch6AiXAKzOVsX51nRZ+ptWoBzOu28V9waKu8Mj6VkdejbA9LwFRIx3Bn1V64evmkF2Gs=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(66556008)(66476007)(64756008)(921005)(66446008)(10290500003)(26005)(55016002)(66946007)(2906002)(7416002)(82950400001)(82960400001)(52536014)(7366002)(5660300002)(7406005)(76116006)(6506007)(316002)(508600001)(8676002)(8990500004)(71200400001)(122000001)(9686003)(86362001)(7696005)(38100700002)(83380400001)(38070700005)(33656002)(186003)(8936002)(110136005)(54906003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?m6hUatX3Wt0L18oS1Yu9aEWLCF2uuZMJfEdIY7AHScXfuZuS9Vn1i5zr0kMX?=
 =?us-ascii?Q?UyDb74sjV+Y/mJVP7pmDTaxal7lh4mwvOZiQ+VHsLiSIrG2YLVgJhWrr2r+a?=
 =?us-ascii?Q?Scr7JKHoBNz3+yZuxgbPZ1zFJxLGdDtxv6Y0RtMQS367vHuFXdFz04f3D+ww?=
 =?us-ascii?Q?jKgV2pNzpRaFyzQrQbeEfuTDmzNs655TEp/TFFA9YGf4cnyjYzTDOg5m+a5Y?=
 =?us-ascii?Q?S/IH++PysmFJsXjboHB8rN/UXWNR0lpaEoqE3h8UNWvp3lltNcsrcEhykJcJ?=
 =?us-ascii?Q?bAnyMXKIGdQzfobT7jCFyyrPoCFGlMCktNWsBM7OsNqQZT3Wg+TgS1aqpXDK?=
 =?us-ascii?Q?oLXIjMSAx7Q0pu3Ehjk0dAKJM/maCVFKFoY3OVmgmllXgi/WVqms0+m1tU+f?=
 =?us-ascii?Q?J+z1W0iHOMDQub++FmeOuRPgaXHMIRr57ktzcK1awLP9FoBruaVk1YLFbC7A?=
 =?us-ascii?Q?0RFLu73/0BzK/7z+pgSrAoCV2+SVWYyZa2hzyWZ0Ba0P7hLCubCJtoKA6ZRx?=
 =?us-ascii?Q?ltkynVtjd3Jd/gTU+cpguMpE+0eyX66vM8VubA+dBhpLnoE42G+LszgFRAMh?=
 =?us-ascii?Q?+uQjnXszP5rFOGJhddSIK2RlWeSPcwUXl6KMXsjBWR919E0N2yb0fGhgGQ70?=
 =?us-ascii?Q?nLNLetSBMA/oyydd6Sojj/aNxR9lgqya8cVLCNR53YwqO/9LakQeyvX6vj3I?=
 =?us-ascii?Q?qEjf2bmrS/F/P5+fc/RfzTKmQXveg/PgV+FRtYOhXclhfT+4BNeS28UvRXNa?=
 =?us-ascii?Q?NmT0vvuVDsZoLBbpZCJ63MrwsiP2GoS97FxfijKyvCcNJYEMMSx2RYvVc/xT?=
 =?us-ascii?Q?hRWHnNrVDJAjIlzLOeq4B+TTxqxq4N3ZqUEAhuNKu6q/Zf9eJpVB97LL022l?=
 =?us-ascii?Q?xJ1wO91ryKPf8U14K92Yyle6LGG4663Z7+/3zQiDFcD2eOH329G6aWwHuoyW?=
 =?us-ascii?Q?1QOynDuZ9fk6nfDE4uM/CFH9Pl7/ugNM46BidxO1RrsyRTN4gPPj6zdkGLgE?=
 =?us-ascii?Q?IBzAV4EzkqKKO9j6h+M2n+IE23ECcRiPMTyxlrjFEgwn19RwPRxdV4jBqdVX?=
 =?us-ascii?Q?jgRjeyEgAicEeBdwk/iOs17bgx3ovSZxgMmNGixKevcERWyjtnS4EdIH+iR3?=
 =?us-ascii?Q?Zt9iO7TL8VoNLbH8F1Uqdd1nzK3Kup7b+7Li7tdNirudyfm+aTRQbh4dG0ir?=
 =?us-ascii?Q?zl0VAIGENIVsisC2yxxoLrn3pidcN0VOl+BCd8ISSux3/3kKRccNT0QliUHz?=
 =?us-ascii?Q?37EN7y/zvGP+yOt37yjUsOqcpuB0at/dOrdrZ40fn2+e0tIPCJ5XC2ANZa92?=
 =?us-ascii?Q?dI4XwIEaregrSP4uNqlmiQPU?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ef6e0ca-5957-4140-2dba-08d96da777c1
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:20:28.3182
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3eA+qo79/uH9Dbp62BmlOSNm51w8AwVRPXoN6jwWsBsxnooEZz+t3zwtKNtVVlv0ClKDK/DmKF0WXrAsb3MspbKkw+PcOVzmV2MNKBDKP2I=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1019

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20

Subject line tag should probably be "x86/hyperv:" since the majority
of the code added is under arch/x86.

> hyperv provides ghcb hvcall to handle VMBus
> HVCALL_SIGNAL_EVENT and HVCALL_POST_MESSAGE
> msg in SNP Isolation VM. Add such support.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Add hv_ghcb_hypercall() stub function to avoid
> 	  compile error for ARM.
> ---
>  arch/x86/hyperv/ivm.c          | 71 ++++++++++++++++++++++++++++++++++
>  drivers/hv/connection.c        |  6 ++-
>  drivers/hv/hv.c                |  8 +++-
>  drivers/hv/hv_common.c         |  6 +++
>  include/asm-generic/mshyperv.h |  1 +
>  5 files changed, 90 insertions(+), 2 deletions(-)
>=20
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> index f56fe4f73000..e761c67e2218 100644
> --- a/arch/x86/hyperv/ivm.c
> +++ b/arch/x86/hyperv/ivm.c
> @@ -17,10 +17,81 @@
>  #include <asm/io.h>
>  #include <asm/mshyperv.h>
>=20
> +#define GHCB_USAGE_HYPERV_CALL	1
> +
>  union hv_ghcb {
>  	struct ghcb ghcb;
> +	struct {
> +		u64 hypercalldata[509];
> +		u64 outputgpa;
> +		union {
> +			union {
> +				struct {
> +					u32 callcode        : 16;
> +					u32 isfast          : 1;
> +					u32 reserved1       : 14;
> +					u32 isnested        : 1;
> +					u32 countofelements : 12;
> +					u32 reserved2       : 4;
> +					u32 repstartindex   : 12;
> +					u32 reserved3       : 4;
> +				};
> +				u64 asuint64;
> +			} hypercallinput;
> +			union {
> +				struct {
> +					u16 callstatus;
> +					u16 reserved1;
> +					u32 elementsprocessed : 12;
> +					u32 reserved2         : 20;
> +				};
> +				u64 asunit64;
> +			} hypercalloutput;
> +		};
> +		u64 reserved2;
> +	} hypercall;
>  } __packed __aligned(HV_HYP_PAGE_SIZE);
>=20
> +u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_=
size)
> +{
> +	union hv_ghcb *hv_ghcb;
> +	void **ghcb_base;
> +	unsigned long flags;
> +
> +	if (!hv_ghcb_pg)
> +		return -EFAULT;
> +
> +	WARN_ON(in_nmi());
> +
> +	local_irq_save(flags);
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	hv_ghcb =3D (union hv_ghcb *)*ghcb_base;
> +	if (!hv_ghcb) {
> +		local_irq_restore(flags);
> +		return -EFAULT;
> +	}
> +
> +	hv_ghcb->ghcb.protocol_version =3D GHCB_PROTOCOL_MAX;
> +	hv_ghcb->ghcb.ghcb_usage =3D GHCB_USAGE_HYPERV_CALL;
> +
> +	hv_ghcb->hypercall.outputgpa =3D (u64)output;
> +	hv_ghcb->hypercall.hypercallinput.asuint64 =3D 0;
> +	hv_ghcb->hypercall.hypercallinput.callcode =3D control;
> +
> +	if (input_size)
> +		memcpy(hv_ghcb->hypercall.hypercalldata, input, input_size);
> +
> +	VMGEXIT();
> +
> +	hv_ghcb->ghcb.ghcb_usage =3D 0xffffffff;
> +	memset(hv_ghcb->ghcb.save.valid_bitmap, 0,
> +	       sizeof(hv_ghcb->ghcb.save.valid_bitmap));
> +
> +	local_irq_restore(flags);
> +
> +	return hv_ghcb->hypercall.hypercalloutput.callstatus;

The hypercall.hypercalloutput.callstatus value must be saved
in a local variable *before* the call to local_irq_restore().  Then
the local variable is the return value.  Once local_irq_restore()
is called, the GHCB page could get reused.

> +}
> +
>  void hv_ghcb_msr_write(u64 msr, u64 value)
>  {
>  	union hv_ghcb *hv_ghcb;
> diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
> index 5e479d54918c..6d315c1465e0 100644
> --- a/drivers/hv/connection.c
> +++ b/drivers/hv/connection.c
> @@ -447,6 +447,10 @@ void vmbus_set_event(struct vmbus_channel *channel)
>=20
>  	++channel->sig_events;
>=20
> -	hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT, channel->sig_event);
> +	if (hv_isolation_type_snp())
> +		hv_ghcb_hypercall(HVCALL_SIGNAL_EVENT, &channel->sig_event,
> +				NULL, sizeof(u64));

Better to use "sizeof(channel->sig_event)" instead of explicitly coding
the type.

> +	else
> +		hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT, channel->sig_event);
>  }
>  EXPORT_SYMBOL_GPL(vmbus_set_event);
> diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
> index 97b21256a9db..d4531c64d9d3 100644
> --- a/drivers/hv/hv.c
> +++ b/drivers/hv/hv.c
> @@ -98,7 +98,13 @@ int hv_post_message(union hv_connection_id connection_=
id,
>  	aligned_msg->payload_size =3D payload_size;
>  	memcpy((void *)aligned_msg->payload, payload, payload_size);
>=20
> -	status =3D hv_do_hypercall(HVCALL_POST_MESSAGE, aligned_msg, NULL);
> +	if (hv_isolation_type_snp())
> +		status =3D hv_ghcb_hypercall(HVCALL_POST_MESSAGE,
> +				(void *)aligned_msg, NULL,
> +				sizeof(struct hv_input_post_message));

As above, use "sizeof(*aligned_msg)".

> +	else
> +		status =3D hv_do_hypercall(HVCALL_POST_MESSAGE,
> +				aligned_msg, NULL);
>=20
>  	/* Preemption must remain disabled until after the hypercall
>  	 * so some other thread can't get scheduled onto this cpu and
> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> index 1fc82d237161..7be173a99f27 100644
> --- a/drivers/hv/hv_common.c
> +++ b/drivers/hv/hv_common.c
> @@ -289,3 +289,9 @@ void __weak hyperv_cleanup(void)
>  {
>  }
>  EXPORT_SYMBOL_GPL(hyperv_cleanup);
> +
> +u64 __weak hv_ghcb_hypercall(u64 control, void *input, void *output, u32=
 input_size)
> +{
> +	return HV_STATUS_INVALID_PARAMETER;
> +}
> +EXPORT_SYMBOL_GPL(hv_ghcb_hypercall);
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index 04a687d95eac..0da45807c36a 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -250,6 +250,7 @@ bool hv_is_hibernation_supported(void);
>  enum hv_isolation_type hv_get_isolation_type(void);
>  bool hv_is_isolation_supported(void);
>  bool hv_isolation_type_snp(void);
> +u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_=
size);
>  void hyperv_cleanup(void);
>  bool hv_query_ext_cap(u64 cap_query);
>  #else /* CONFIG_HYPERV */
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:21:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:21:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176744.321650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaU1-0003aO-2f; Thu, 02 Sep 2021 00:21:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176744.321650; Thu, 02 Sep 2021 00:21:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaU0-0003aH-Vu; Thu, 02 Sep 2021 00:21:32 +0000
Received: by outflank-mailman (input) for mailman id 176744;
 Thu, 02 Sep 2021 00:21:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaTz-0003aB-Gi
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:21:31 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b67d74d0-0b83-11ec-ae2d-12813bfff9fa;
 Thu, 02 Sep 2021 00:21:27 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1019.namprd21.prod.outlook.com (2603:10b6:302:5::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6; Thu, 2 Sep
 2021 00:21:24 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:21:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b67d74d0-0b83-11ec-ae2d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DpFAEAVFtRJX1cTnnpRrX3R9Pm8gSHt0iXZHDYb88Ud8MzsVIKt0rQdFMcNLZlm0LHgQhOL99Xo1yH6JTnxDpqXM3nbtPc/lyPIkSLqLHLZTwFputX5NZuSiXZ1Vm/5dUyps0EGsiBAC+Zkmu8mGkINU3TAuGnAWNdNV+FTLQ47olsWuhp5cMEPn0NF8LFs6bt5cmSfhj64fBOim9HjOaa/K1lmSXDoPoPV7pi/bhcPz4r2koxMX1HcOi7j9DPIdLIOFmJBNWkig/PAr3XKHfCvBHkdMV9SIB5U6ph8xEpLJt9wnd9LMRcCevwtC4mcG8j0RVdO71mw22MbqfUCXLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Mqf14wQtw+7JBy9v/vfQD5FN7Z7H44udWTOYeVPnD5A=;
 b=T7SuaRWd0/1riTDQu1TvUyEICtpJthhBkGQpZ9xd/R46UTMCDHn7592zKpzg8fGjXDrcUweNiNcOdwCpn255g4f5rrR+bRghzPBIutc5+TOt9j+bwpOL8QLrrtoE2nB59dRQii0R1PHyNnk+6leXMPfxnMQ2YBYjVaDdxK5zsx0LMi0FsMdgsrhthqSnZIpWKoK926sRFWFNDYuN+XB1H2EMIvkiP1x8rEF/kHJ/j160HRV/dzpdoBsVDqVI4j0/Taai6mlk+SL7f7MItqooweZ8SA9L/PgE6Go2J+hpnzd+1MUHUo4jY0pGfxSqXzMZvI0Wn/Ya7egdLwPvhWyb7w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Mqf14wQtw+7JBy9v/vfQD5FN7Z7H44udWTOYeVPnD5A=;
 b=WRS/XTElGOOmSfSxr8RARTjq6kei7DdGfXZSQeOIEASbO2LFrNR5iACVCa0FNfGdT8Ech8QPYtCIoRfUZyPbtGjgS24mc1NdB7q/Cz3nj6yOsvQENJfvSdJTcbz7ecGD4mXQUmgJd1rPOpCrQWUjQ0pstyJglbDhtysSDP3u2iM=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 07/13] hyperv/Vmbus: Add SNP support for VMbus channel
 initiate  message
Thread-Topic: [PATCH V4 07/13] hyperv/Vmbus: Add SNP support for VMbus channel
 initiate  message
Thread-Index: AQHXm2gBdvpa+X8MqESuEZ+wA8gCQ6uKsPEw
Date: Thu, 2 Sep 2021 00:21:24 +0000
Message-ID:
 <MWHPR21MB1593CF062ACBE0707D0F46AFD7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-8-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-8-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=18641334-deb6-4e85-959f-298ebfb11fed;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-29T16:35:29Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 24e4b7bc-b0c1-4b1d-8cec-08d96da7992c
x-ms-traffictypediagnostic: MW2PR2101MB1019:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB101914B265525342CCD17035D7CE9@MW2PR2101MB1019.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:4714;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 DXQW1zzjyK4YVkF1ykwSGWrws5AuKCpcL60tLoEFQfNz4OE7v47/9aWyCXZgxHs2OfyoZcU1s/7pzwcK52fMuoSWu1PLT7xQg5x+NXGlvDAteXE+bRTTu5C7JmM9cf/2hxdBi7dxz+byW25l9TH6FPHQXtzXk762upIUw5ry0c3Pr1eiLPXlq/LIDh6QE2o63LX3Xy5LT6qQb4tyEFDAb/YwY0XqAf8zRJefLyWPS9JzzHyCDX1iil4W9scABHTddjwPwm/bHHxSNCnQQqABixngwYIs9ohyjVegZPDSaPJGZR11lzkMUKWm9IJ2QBckpNcIhloJ0Uzuxf9XMUMrebWHwrLtz03RrlukRWZ5vntjeGGVTW5K12t1UWLRQF6FAnLTS4BV1QZA8MKVWOGBScX4xwOkwfWPWDXBtAQwmczPxhcnWyqNTiSwlYrh/TFYU/mYXrE3E8kjrrcnfGlUdGatu3caVm2uRDlE4gePdzoHkLtaNav+qKFK6bhDbVEBt3IhWL07HhCHC6hYM79cAnZT+2flvTVG1Gf0VFxMo3wkgFdULXl0lQ3HWqqm12h+lpyqnD0LKTlgnMm90tpc28EqQCqS7u4qQEvrAKXhQ4tAOI/Rkr5vOrYnODFp62MOFDvITTOuj0zK+Y3EzRUZ/n81hEdVy75UbEOqPIswBIX6jtaPxpAJv50Qn0XgAJtZ7yq9Z7TM8nothzzVnV6+LM8gBzUt6RFI+fEsu7PyjeM=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(66556008)(66476007)(64756008)(921005)(66446008)(10290500003)(26005)(55016002)(66946007)(2906002)(7416002)(82950400001)(82960400001)(52536014)(7366002)(5660300002)(7406005)(76116006)(6506007)(316002)(508600001)(8676002)(8990500004)(15650500001)(71200400001)(122000001)(9686003)(86362001)(7696005)(38100700002)(83380400001)(38070700005)(33656002)(186003)(8936002)(110136005)(54906003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?80/e3gUgachnycbVd5r3uUqOemHZxHUVPdbVExd1VVw7E5+cFX8fzMliAkjq?=
 =?us-ascii?Q?NVp0Sl9gVYXLNgOrb/zhkTWrjhRVaF4CeExuSuBg+P/fILrA6B1oNqOXKgWf?=
 =?us-ascii?Q?3vhi5VovoPdDncZotT3ktFkm2RSpHFhUD5vfJ630psAWUbJsRlDna9xwSodE?=
 =?us-ascii?Q?v44mdrCLv13r36yBxZFbz5wXSaBDoyOdu0HKdn1/9/D2Y+HpKbVIlsdMYrfs?=
 =?us-ascii?Q?pFMsG25nU0W0T1GIfXfKp91MrnPhjkKLLrEJl49FO99T0y8lXidw1Eb0Slfa?=
 =?us-ascii?Q?qK615ya9jCH3C7YW89JJ2EdOi1b0lJZawwdw9tHVPzCPmc33B8/I9WghhPwn?=
 =?us-ascii?Q?1jj41MowK5iMTQ0J90rzxFWVy7nVZYrDQu+rHFYz3itwRl4wZAqziTdCz0dW?=
 =?us-ascii?Q?B5tIN4u0im/teed4Q5sLszoUKYkqGfzAN3HAe+k2AeFcslkf3pZCpEof4zog?=
 =?us-ascii?Q?Gp8Cq1ZJQkfyXTLpoh3fdYTieVKi6HuF3WHxr/N04ljtCt5bYwItS1byfczJ?=
 =?us-ascii?Q?tXoqwN1rWfYxEu5SenG0R3AIfpfOojO93WXgJUpmiPPxjQjJywulNeeAHU+B?=
 =?us-ascii?Q?ZvWpmlm5oF5HRBOjv+dkEFE0Iq1+io4hssD9Vx8TOqWapbOOTu2WaxE00h22?=
 =?us-ascii?Q?GUXsJlUi6HoOKvIgCwjbFRC4RATmrvlNLaX+8mnEGGtSuomDjAJwQOG46aNo?=
 =?us-ascii?Q?O7QNGTwIjIMKhnqfZ3SyJPX2ZuuCDd3+mudmIEJtpd1VuQgYxaAkfBZdHJgG?=
 =?us-ascii?Q?gG54nU29LkLgPVHoRroTR0EQwIhxpKtWun0RrrBBdoq8ZVObxZOeMGw1UObX?=
 =?us-ascii?Q?YTGu8kXnPSyyUUr9kKktZpU7fbjMxzNCvOcKFimKeMZ6YORnY2/UBPxsZenO?=
 =?us-ascii?Q?Gkl+AjHsucnAuZ43jVsBFH8nkBY37RZWOAj0LSlNrKUf5877ZgxEFO4l3jb9?=
 =?us-ascii?Q?nE9xfNOadBsiLxzbjj5u7zYqQ2ud6BkPBEQ6higZl2OCdJ2TL3f6CF7BZqqp?=
 =?us-ascii?Q?oLFRTqxVMmdyLMsUgCmb+aJtI3bOySO/wHX1w+wo1Iz2W/pJBOKc7kgBAjnw?=
 =?us-ascii?Q?BIxVDY2w9LkRchVClpu19mvsNJeu2kRf+3EyqXg+IiBSc0LukXRB7166NWXQ?=
 =?us-ascii?Q?yiEgcTn/sxrD4jAaiSlF1mdzt/OUAMZ8PBIkYXhVIMIMQ4irna/kHGmsfV3W?=
 =?us-ascii?Q?rngpaCeAJKB+7HPsP3UiOBC5PygUjaNt/iAL+UL9SrJ2g8UFuASFydBPjW2Z?=
 =?us-ascii?Q?PdDBZw0RxX+Y8zc7IHfy7E+Xwfv+EKL+EnDF3UMAjf0+wiBA0BULLPAkRYXU?=
 =?us-ascii?Q?6CHGz2V6NC6lwXkHoZh9vGKD?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 24e4b7bc-b0c1-4b1d-8cec-08d96da7992c
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:21:24.4076
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: r07DIs7vHswtxB9LhpsP5x44bMSZ3HFWS6aSop8iZ2SVU44k5Q6//diDx1yPWIqR6SKynf4MyuvRP4ypBuzp1/Qi0qvz5lF9wjYTwpI2mQY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1019

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20

Subject line tag should be "Drivers: hv: vmbus:"

> The monitor pages in the CHANNELMSG_INITIATE_CONTACT msg are shared
> with host in Isolation VM and so it's necessary to use hvcall to set
> them visible to host. In Isolation VM with AMD SEV SNP, the access
> address should be in the extra space which is above shared gpa
> boundary. So remap these pages into the extra address(pa +
> shared_gpa_boundary).
>=20
> Introduce monitor_pages_original[] in the struct vmbus_connection
> to store monitor page virtual address returned by hv_alloc_hyperv_
> zeroed_page() and free monitor page via monitor_pages_original in
> the vmbus_disconnect(). The monitor_pages[] is to used to access
> monitor page and it is initialized to be equal with monitor_pages_
> original. The monitor_pages[] will be overridden in the isolation VM
> with va of extra address.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Rename monitor_pages_va with monitor_pages_original
> 	* free monitor page via monitor_pages_original and
> 	  monitor_pages is used to access monitor page.
>=20
> Change since v1:
>         * Not remap monitor pages in the non-SNP isolation VM.
> ---
>  drivers/hv/connection.c   | 75 ++++++++++++++++++++++++++++++++++++---
>  drivers/hv/hyperv_vmbus.h |  1 +
>  2 files changed, 72 insertions(+), 4 deletions(-)
>=20
> diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
> index 6d315c1465e0..9a48d8115c87 100644
> --- a/drivers/hv/connection.c
> +++ b/drivers/hv/connection.c
> @@ -19,6 +19,7 @@
>  #include <linux/vmalloc.h>
>  #include <linux/hyperv.h>
>  #include <linux/export.h>
> +#include <linux/io.h>
>  #include <asm/mshyperv.h>
>=20
>  #include "hyperv_vmbus.h"
> @@ -104,6 +105,12 @@ int vmbus_negotiate_version(struct vmbus_channel_msg=
info *msginfo, u32 version)
>=20
>  	msg->monitor_page1 =3D virt_to_phys(vmbus_connection.monitor_pages[0]);
>  	msg->monitor_page2 =3D virt_to_phys(vmbus_connection.monitor_pages[1]);
> +
> +	if (hv_isolation_type_snp()) {
> +		msg->monitor_page1 +=3D ms_hyperv.shared_gpa_boundary;
> +		msg->monitor_page2 +=3D ms_hyperv.shared_gpa_boundary;
> +	}
> +
>  	msg->target_vcpu =3D hv_cpu_number_to_vp_number(VMBUS_CONNECT_CPU);
>=20
>  	/*
> @@ -148,6 +155,35 @@ int vmbus_negotiate_version(struct vmbus_channel_msg=
info *msginfo, u32 version)
>  		return -ECONNREFUSED;
>  	}
>=20
> +
> +	if (hv_is_isolation_supported()) {
> +		if (hv_isolation_type_snp()) {
> +			vmbus_connection.monitor_pages[0]
> +				=3D memremap(msg->monitor_page1, HV_HYP_PAGE_SIZE,
> +					   MEMREMAP_WB);
> +			if (!vmbus_connection.monitor_pages[0])
> +				return -ENOMEM;
> +
> +			vmbus_connection.monitor_pages[1]
> +				=3D memremap(msg->monitor_page2, HV_HYP_PAGE_SIZE,
> +					   MEMREMAP_WB);
> +			if (!vmbus_connection.monitor_pages[1]) {
> +				memunmap(vmbus_connection.monitor_pages[0]);
> +				return -ENOMEM;
> +			}
> +		}
> +
> +		/*
> +		 * Set memory host visibility hvcall smears memory
> +		 * and so zero monitor pages here.
> +		 */
> +		memset(vmbus_connection.monitor_pages[0], 0x00,
> +		       HV_HYP_PAGE_SIZE);
> +		memset(vmbus_connection.monitor_pages[1], 0x00,
> +		       HV_HYP_PAGE_SIZE);
> +
> +	}

I still find it somewhat confusing to have the handling of the
shared_gpa_boundary and memory mapping in the function for
negotiating the VMbus version.  I think the code works as written,
but it would seem cleaner and easier to understand to precompute
the physical addresses and do all the mapping and memory zero'ing
in a single place in vmbus_connect().  Then the negotiate version
function can focus on doing only the version negotiation.

> +
>  	return ret;
>  }
>=20
> @@ -159,6 +195,7 @@ int vmbus_connect(void)
>  	struct vmbus_channel_msginfo *msginfo =3D NULL;
>  	int i, ret =3D 0;
>  	__u32 version;
> +	u64 pfn[2];
>=20
>  	/* Initialize the vmbus connection */
>  	vmbus_connection.conn_state =3D CONNECTING;
> @@ -216,6 +253,21 @@ int vmbus_connect(void)
>  		goto cleanup;
>  	}
>=20
> +	vmbus_connection.monitor_pages_original[0]
> +		=3D vmbus_connection.monitor_pages[0];
> +	vmbus_connection.monitor_pages_original[1]
> +		=3D vmbus_connection.monitor_pages[1];
> +
> +	if (hv_is_isolation_supported()) {
> +		pfn[0] =3D virt_to_hvpfn(vmbus_connection.monitor_pages[0]);
> +		pfn[1] =3D virt_to_hvpfn(vmbus_connection.monitor_pages[1]);
> +		if (hv_mark_gpa_visibility(2, pfn,
> +				VMBUS_PAGE_VISIBLE_READ_WRITE)) {

In Patch 4 of this series, host visibility for the specified buffer is done
by calling set_memory_decrypted()/set_memory_encrypted().  Could
the same be done here?   The code would be more consistent overall
with better encapsulation.  hv_mark_gpa_visibility() would not need to
be exported or need an ARM64 stub.

set_memory_decrypted()/encrypted() seem to be the primary functions
that should be used for this purpose, and they have already have the
appropriate stubs for architectures that don't support memory encryption.

> +			ret =3D -EFAULT;
> +			goto cleanup;
> +		}
> +	}
> +
>  	msginfo =3D kzalloc(sizeof(*msginfo) +
>  			  sizeof(struct vmbus_channel_initiate_contact),
>  			  GFP_KERNEL);
> @@ -284,6 +336,8 @@ int vmbus_connect(void)
>=20
>  void vmbus_disconnect(void)
>  {
> +	u64 pfn[2];
> +
>  	/*
>  	 * First send the unload request to the host.
>  	 */
> @@ -303,10 +357,23 @@ void vmbus_disconnect(void)
>  		vmbus_connection.int_page =3D NULL;
>  	}
>=20
> -	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[0]);
> -	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[1]);
> -	vmbus_connection.monitor_pages[0] =3D NULL;
> -	vmbus_connection.monitor_pages[1] =3D NULL;
> +	if (hv_is_isolation_supported()) {
> +		memunmap(vmbus_connection.monitor_pages[0]);
> +		memunmap(vmbus_connection.monitor_pages[1]);
> +
> +		pfn[0] =3D virt_to_hvpfn(vmbus_connection.monitor_pages[0]);
> +		pfn[1] =3D virt_to_hvpfn(vmbus_connection.monitor_pages[1]);
> +		hv_mark_gpa_visibility(2, pfn, VMBUS_PAGE_NOT_VISIBLE);

Same comment about using set_memory_encrypted() instead.

> +	}
> +
> +	hv_free_hyperv_page((unsigned long)
> +		vmbus_connection.monitor_pages_original[0]);
> +	hv_free_hyperv_page((unsigned long)
> +		vmbus_connection.monitor_pages_original[1]);
> +	vmbus_connection.monitor_pages_original[0] =3D
> +		vmbus_connection.monitor_pages[0] =3D NULL;
> +	vmbus_connection.monitor_pages_original[1] =3D
> +		vmbus_connection.monitor_pages[1] =3D NULL;
>  }
>=20
>  /*
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 42f3d9d123a1..7cb11ef694da 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -240,6 +240,7 @@ struct vmbus_connection {
>  	 * is child->parent notification
>  	 */
>  	struct hv_monitor_page *monitor_pages[2];
> +	void *monitor_pages_original[2];
>  	struct list_head chn_msg_list;
>  	spinlock_t channelmsg_lock;
>=20
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 00:23:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 00:23:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176751.321661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaVq-0004Jl-Lb; Thu, 02 Sep 2021 00:23:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176751.321661; Thu, 02 Sep 2021 00:23:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLaVq-0004Jd-IV; Thu, 02 Sep 2021 00:23:26 +0000
Received: by outflank-mailman (input) for mailman id 176751;
 Thu, 02 Sep 2021 00:23:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLaVp-0004JU-Lv
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 00:23:25 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fa2cf48b-0b83-11ec-ae2d-12813bfff9fa;
 Thu, 02 Sep 2021 00:23:21 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1019.namprd21.prod.outlook.com (2603:10b6:302:5::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6; Thu, 2 Sep
 2021 00:23:08 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 00:23:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa2cf48b-0b83-11ec-ae2d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=S8hY9iJsRcCnx9oyRQ/hspUHIv5ff5374Rs6oJjEBnrTbvDk5OVpo0Dytg9YAtPPhZr2l5bsIK+3ilDvmgjxGyhDo5DazVJGf5g3p9s+a7ifYE/OqB+IVNp6W/b1GDIf14H4eQPlzE5fHzkG9QcOTAaGkRXI0CheiguOKNcKQBwnDgTmC20tuonkdBTxywuHT+oa3OFcYQE4s+NacqVM7320m1M2DWs3PTUtXvlQlWH4xqX/50jUCo0tmFahSIIj/Oq7ny4p7lSYujeyDChp/0Ze+ZUy5UBVLgnyj/inOOjQp+xFjvmb3XKi38+uCARx9Yx0y7ezjshT6Ergk0biRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qzrjg2Ld5ycVv0rSLliu9cUFTVcVw5lvAUeicQF2lCU=;
 b=TD5YCrNUvn78vMeILZY/K1kN7MDYqpxRmyv1nyfnfetEvkiiGIWOviUgmGTxUOT3tTz8DT6AcojE/n4UXu+xZujVVXiORo734A80/XxTJrls2GzULEfoo5gU20wm6ZO9C8LxbRPQHJt8ZPIubtjEJdOnrWFCsmEcCuk5mEFvdbGkkO11QK7njiEmOONbnL2RYCgWzLX+yJL8B7gQlzVffowcYscJWFwefoK/8KWpqtwihCvDRisT3CBgLgf1QciWejLJmWELbTH+YzPtVfqhhRSuy3TmZh07wf6x+5vBZzen2El8KBTx5ivWGaJ8+JpsJhC0VRrSnVJ9Qsl5D8CYHA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qzrjg2Ld5ycVv0rSLliu9cUFTVcVw5lvAUeicQF2lCU=;
 b=U5bibdZiNOinwuDMK081bP3N0/p7F9BHnGIsAEpvJQuA6Ihj7otxojR0zUqUnUiD0Yh4MRNEIdDf402n6pckNC3TDDyvLAY8dufxHi/UK6uoVZLuRrk7ZJ6pH91S32HeTbmhMDfTzCVd+wpm6DZK8k33MsTtVd3/LunuK+n82Qk=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 08/13] hyperv/vmbus: Initialize VMbus ring buffer for
 Isolation VM
Thread-Topic: [PATCH V4 08/13] hyperv/vmbus: Initialize VMbus ring buffer for
 Isolation VM
Thread-Index: AQHXm2gFIWSYU02+UUyReBHQOl5/ZauK4YnA
Date: Thu, 2 Sep 2021 00:23:08 +0000
Message-ID:
 <MWHPR21MB1593B416ED91CD454FC036E5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-9-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-9-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=bc1bd6dd-73be-4423-bfec-d24cf5650634;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-29T19:29:24Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: fdd00e19-74c7-465e-488a-08d96da7d743
x-ms-traffictypediagnostic: MW2PR2101MB1019:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB10194E3D49874ACDD0F694DFD7CE9@MW2PR2101MB1019.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:1360;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 a/1uWWyCm6KRWpOZ3PLSe+hwH5KkYNdAfvjSkhJa6lFuCklklUG/XcyZBFKgN2AmmBkr2TBUHNINXZNh0dNw4ocmONb7WXoudBjauOPc3GnHW4Wg/qf5gmFold/8RyJO7lTfbQrenuk7lt8iraXKKwG1PTU24Fi5pm3jjxvzwoErjxm0Rk3pGEuIhCb2yt/jHdUohFwAJLrtIvyy/49dwrizA4T+wUah8Ydz/Zj4XnKnbNdJZSCobmq1u2VKgoxxZCODLTqyUjE3Mynw9xmlOYY8MMMaw1yZNSMDK45zhYSEjigcd6DBOcTRRVniT1qsRFMpwNpmtA+mIM+a0MUW86c0SFDSZoS+NbOp8J5RP9iFmaRdRmiMYYGYJ14vVmkrtqLCd0AD+jRbTPgTMWN2a3jem1lPpfnOSuZtLLjEI1s2CxJT2f+l3tIzzMp+U+vRsAwEYhRvlHT6+v47USlJufwuSsjmfG2BFgoRWAo8+LGnfd4cp/+f5KQDfPLigB7oTEGNKVc2sJR+T7Z5J1pQduSXsskar11wKYUD8XHYCaM2Qd4lsEPFR9Kmb92XK/YIsQrOiaGtJw21wMlwgUNUorsDwOXEkhWymepocLVf7Q94IDyyMEUBcLH/V5//3uUc2/Wdt0+z9PYdVcNrOKn9tLXZrqSsuMnIoDizHEX72nYhCg2KyUSvRmna68APZlkt4iw8SwP5mEH4CiXI407t1fR3sOBR6YA7uF5M0Xye0y0=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(66556008)(66476007)(64756008)(921005)(66446008)(10290500003)(26005)(55016002)(66946007)(2906002)(7416002)(82950400001)(82960400001)(52536014)(7366002)(5660300002)(7406005)(76116006)(6506007)(316002)(508600001)(8676002)(8990500004)(71200400001)(122000001)(9686003)(86362001)(7696005)(38100700002)(83380400001)(38070700005)(33656002)(186003)(8936002)(110136005)(54906003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?6XoYftWOuBZ8DMMdcUow8qMAsztK90BM4zwgFHqXCTT0/q3RP3X2SwCIN0AN?=
 =?us-ascii?Q?1CO7TSM7aUgUQLy7/CS7apcmkWvaltP9dLrgtfUDUC21iu4mseZZaxiPcowW?=
 =?us-ascii?Q?FMzfYbCUD/MFcaRSpeOttttCfjw3BU+ayYxvspuBvq37pb7GREN1+j54ihRI?=
 =?us-ascii?Q?q+YKDNIRsQ8e5q9Mm49uRK3NMEPv1wDKWzDX4Dr9DfKAC3P/szxR4vsOu5ji?=
 =?us-ascii?Q?an3hnegJR3hQpW5lgodD7eGTGEl+yACooBujO62NgBtEvX2F3cKyTtYGg7Dv?=
 =?us-ascii?Q?qxlWTBCFJ372GbfMENy6ySZbmimtyzpCb9XIr2BsANfKh7KQSIyNryOjHC+S?=
 =?us-ascii?Q?lei2n1cWW4Zkbp59Uow5NopFsoKMwh8GpQmwLoXvfg6tex7uXWqDWO0q6/2/?=
 =?us-ascii?Q?d1DZ9BhOXns3eLlElLCP432cZOsypM9iC9J7ZqLpPF9U9wSJQpqcP9ssyrRW?=
 =?us-ascii?Q?zWbSM3DX8sO/rXmASE5w4QQVm007eC3GdWNg19sRSWdWAQe6TufvoVdHG97T?=
 =?us-ascii?Q?z6aUCZyPbPwv9Hmv3voLunZ5wMGOrsjrDqeeEEBu0ra3tb7zQ33xsJrELABw?=
 =?us-ascii?Q?stXUSAkVZg/1YtJbdRvCmIYsghJ7CdGKEo8LWxfaV+/pzXGuRrYLVEKyzccg?=
 =?us-ascii?Q?uHS0Ac94KveJFfPz8X9ggjJWrkzeNt9GJ+9PA8Q3nXrtYgyBL+vk27ZAsNvV?=
 =?us-ascii?Q?Xo+wRjsh7oIVMxQOAwDmROPephgD8IaokpRhpnr3JWa0077FA9EpgtSnqOSg?=
 =?us-ascii?Q?148fkt6WhUETRuE9dcbdbDda9c4b0hJ5F1dMTS1I2UPgSpUo9LXXUh0LIDt8?=
 =?us-ascii?Q?8/k2FJkot+Nc82CvZvBob6T4ntN9duPJtHpnV0fZAEc0KZ5eyohvIJNyZlU1?=
 =?us-ascii?Q?kWzW0qNRF7ofsWr1haiGZfqkplyq+PShvF/9C80NnqZSWC2vLfgqGTjxMgDj?=
 =?us-ascii?Q?NUhxmBOvxs+284vS99i5RTaL26s28x/V3OdB5u38+M6+sblMeeYIARkqWaS/?=
 =?us-ascii?Q?v2Bpm45vyAYk7NPFqOpiuxPLKsbyFqdZgNeQUDMLoKmf58EBDi9qdbV2QbSh?=
 =?us-ascii?Q?rq1BiCTg6y0U9W1rBsL3quQ1AJFe8yqXkuELcZObY+COu/BQlUVZys9+mJiy?=
 =?us-ascii?Q?MZ7HJL41DJJmC/9yrNJn8rCb5Y3K+DR8K+S1IsrUflLh3r+MeuVTXs28LwwO?=
 =?us-ascii?Q?JNztev9TSSmPYmRm9g2k3TTuad7FbXxy8kfAQaX3/pbrSkVqTy+wpJSgR8yL?=
 =?us-ascii?Q?xyPkiLcnYOOQjVYm8k/3s/ZYp5fjpLDafvtV9xpv25hsgcalc+ravYcCs5BA?=
 =?us-ascii?Q?8MmkSCuy0Xlhbl8Au+1+r8+q?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fdd00e19-74c7-465e-488a-08d96da7d743
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 00:23:08.5814
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: bDiVg39aGq1FaXP2zo95/QN78I8IgtTSDrwy0TrxEwRiCpJ/PsqE6NddNnIMEFHdz5LmrkDwBlr26DPvAdo1fd6EHWSKXQK+l7enS4fOYHQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1019

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20

Subject tag should be "Drivers: hv: vmbus: "

> VMbus ring buffer are shared with host and it's need to
> be accessed via extra address space of Isolation VM with
> AMD SNP support. This patch is to map the ring buffer
> address in extra address space via vmap_pfn(). Hyperv set
> memory host visibility hvcall smears data in the ring buffer
> and so reset the ring buffer memory to zero after mapping.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Remove hv_ringbuffer_post_init(), merge map
> 	operation for Isolation VM into hv_ringbuffer_init()
> 	* Call hv_ringbuffer_init() after __vmbus_establish_gpadl().
> ---
>  drivers/hv/Kconfig       |  1 +
>  drivers/hv/channel.c     | 19 +++++++-------
>  drivers/hv/ring_buffer.c | 56 ++++++++++++++++++++++++++++++----------
>  3 files changed, 54 insertions(+), 22 deletions(-)
>=20
> diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
> index d1123ceb38f3..dd12af20e467 100644
> --- a/drivers/hv/Kconfig
> +++ b/drivers/hv/Kconfig
> @@ -8,6 +8,7 @@ config HYPERV
>  		|| (ARM64 && !CPU_BIG_ENDIAN))
>  	select PARAVIRT
>  	select X86_HV_CALLBACK_VECTOR if X86
> +	select VMAP_PFN
>  	help
>  	  Select this option to run Linux as a Hyper-V client operating
>  	  system.
> diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
> index 82650beb3af0..81f8629e4491 100644
> --- a/drivers/hv/channel.c
> +++ b/drivers/hv/channel.c
> @@ -679,15 +679,6 @@ static int __vmbus_open(struct vmbus_channel *newcha=
nnel,
>  	if (!newchannel->max_pkt_size)
>  		newchannel->max_pkt_size =3D VMBUS_DEFAULT_MAX_PKT_SIZE;
>=20
> -	err =3D hv_ringbuffer_init(&newchannel->outbound, page, send_pages, 0);
> -	if (err)
> -		goto error_clean_ring;
> -
> -	err =3D hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
> -				 recv_pages, newchannel->max_pkt_size);
> -	if (err)
> -		goto error_clean_ring;
> -
>  	/* Establish the gpadl for the ring buffer */
>  	newchannel->ringbuffer_gpadlhandle =3D 0;
>=20
> @@ -699,6 +690,16 @@ static int __vmbus_open(struct vmbus_channel *newcha=
nnel,
>  	if (err)
>  		goto error_clean_ring;
>=20
> +	err =3D hv_ringbuffer_init(&newchannel->outbound,
> +				 page, send_pages, 0);
> +	if (err)
> +		goto error_free_gpadl;
> +
> +	err =3D hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
> +				 recv_pages, newchannel->max_pkt_size);
> +	if (err)
> +		goto error_free_gpadl;
> +
>  	/* Create and init the channel open message */
>  	open_info =3D kzalloc(sizeof(*open_info) +
>  			   sizeof(struct vmbus_channel_open_channel),
> diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
> index 2aee356840a2..24d64d18eb65 100644
> --- a/drivers/hv/ring_buffer.c
> +++ b/drivers/hv/ring_buffer.c
> @@ -17,6 +17,8 @@
>  #include <linux/vmalloc.h>
>  #include <linux/slab.h>
>  #include <linux/prefetch.h>
> +#include <linux/io.h>
> +#include <asm/mshyperv.h>
>=20
>  #include "hyperv_vmbus.h"
>=20
> @@ -183,8 +185,10 @@ void hv_ringbuffer_pre_init(struct vmbus_channel *ch=
annel)
>  int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
>  		       struct page *pages, u32 page_cnt, u32 max_pkt_size)
>  {
> -	int i;
>  	struct page **pages_wraparound;
> +	unsigned long *pfns_wraparound;
> +	u64 pfn;
> +	int i;
>=20
>  	BUILD_BUG_ON((sizeof(struct hv_ring_buffer) !=3D PAGE_SIZE));
>=20
> @@ -192,23 +196,49 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *=
ring_info,
>  	 * First page holds struct hv_ring_buffer, do wraparound mapping for
>  	 * the rest.
>  	 */
> -	pages_wraparound =3D kcalloc(page_cnt * 2 - 1, sizeof(struct page *),
> -				   GFP_KERNEL);
> -	if (!pages_wraparound)
> -		return -ENOMEM;
> +	if (hv_isolation_type_snp()) {
> +		pfn =3D page_to_pfn(pages) +
> +			HVPFN_DOWN(ms_hyperv.shared_gpa_boundary);

Use PFN_DOWN, not HVPFN_DOWN.  This is all done in units of guest page
size, not Hyper-V page size.

>=20
> -	pages_wraparound[0] =3D pages;
> -	for (i =3D 0; i < 2 * (page_cnt - 1); i++)
> -		pages_wraparound[i + 1] =3D &pages[i % (page_cnt - 1) + 1];
> +		pfns_wraparound =3D kcalloc(page_cnt * 2 - 1,
> +			sizeof(unsigned long), GFP_KERNEL);
> +		if (!pfns_wraparound)
> +			return -ENOMEM;
>=20
> -	ring_info->ring_buffer =3D (struct hv_ring_buffer *)
> -		vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, PAGE_KERNEL);
> +		pfns_wraparound[0] =3D pfn;
> +		for (i =3D 0; i < 2 * (page_cnt - 1); i++)
> +			pfns_wraparound[i + 1] =3D pfn + i % (page_cnt - 1) + 1;
>=20
> -	kfree(pages_wraparound);
> +		ring_info->ring_buffer =3D (struct hv_ring_buffer *)
> +			vmap_pfn(pfns_wraparound, page_cnt * 2 - 1,
> +				 PAGE_KERNEL);
> +		kfree(pfns_wraparound);
>=20
> +		if (!ring_info->ring_buffer)
> +			return -ENOMEM;
> +
> +		/* Zero ring buffer after setting memory host visibility. */
> +		memset(ring_info->ring_buffer, 0x00,
> +			HV_HYP_PAGE_SIZE * page_cnt);

The page_cnt parameter is in units of the guest page size.  So this
should use PAGE_SIZE, not HV_HYP_PAGE_SIZE.

> +	} else {
> +		pages_wraparound =3D kcalloc(page_cnt * 2 - 1,
> +					   sizeof(struct page *),
> +					   GFP_KERNEL);
> +
> +		pages_wraparound[0] =3D pages;
> +		for (i =3D 0; i < 2 * (page_cnt - 1); i++)
> +			pages_wraparound[i + 1] =3D
> +				&pages[i % (page_cnt - 1) + 1];
> +
> +		ring_info->ring_buffer =3D (struct hv_ring_buffer *)
> +			vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP,
> +				PAGE_KERNEL);
> +
> +		kfree(pages_wraparound);
> +		if (!ring_info->ring_buffer)
> +			return -ENOMEM;
> +	}

With this patch, the code is a big "if" statement with two halves -- one
when SNP isolation is in effect, and the other when not.  The SNP isolation
case does the work using PFNs with the shared_gpa_boundary added,
while the other case does the same work but using struct page.  Perhaps
I'm missing something, but can both halves be combined and always
do the work using PFNs?  The only difference is whether to add the
shared_gpa_boundary, and whether to zero the memory when done.
So get the starting PFN, then have an "if" statement for whether to
add the shared_gpa_boundary.  Then everything else is the same.
At the end, use an "if" statement to decide whether to zero the
memory.  It would really be better to have the logic in this algorithm
coded only once.

>=20
> -	if (!ring_info->ring_buffer)
> -		return -ENOMEM;
>=20
>  	ring_info->ring_buffer->read_index =3D
>  		ring_info->ring_buffer->write_index =3D 0;
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 01:27:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 01:27:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176761.321685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbVu-0003zp-B2; Thu, 02 Sep 2021 01:27:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176761.321685; Thu, 02 Sep 2021 01:27:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbVu-0003zi-6r; Thu, 02 Sep 2021 01:27:34 +0000
Received: by outflank-mailman (input) for mailman id 176761;
 Thu, 02 Sep 2021 01:27:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLbVt-0003zb-K0
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 01:27:33 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f217440e-29c0-4c22-a23c-3a3dc3d95c61;
 Thu, 02 Sep 2021 01:27:31 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0767.namprd21.prod.outlook.com (2603:10b6:300:76::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.0; Thu, 2 Sep
 2021 01:27:14 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 01:27:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f217440e-29c0-4c22-a23c-3a3dc3d95c61
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PqH6zbdXDocj/kohzFNuFSZBk5b/2UyJU4cNNkSedGUI/Bt9AxktZr3L9inKZ89BLNZWRm8pjRFCVSdZmWh/D7zCDBnsK08tTqyTw8xnPM9T8qD795zce97qnANfvL4DnnQWxwCb2N4JEwDw0UEvFAm8otRci6Tysb/aS5cHwHhOfjMh3cXZAj1eIBlay6EwhFekB2EYr8FJCDpHO5MVGXV2/2iIorc6FfGOf9YZax4NWdmFeVRYUsLHy0SiExMp8GxVdCILOrEVCk4FOUhnoaVD2FX2c9FWXysJSUXRZcQi06bPgUZu/8NUdXzsATcF5LAx2D2WuDHfJe7lhyKDHA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=BZ2kwHMnelplbCchdbwZ+ia6PwO7lsSSD7ssrmF5dHQ=;
 b=dp8QKS4leVE/cV8OgvsZK9JLaqJ7eGztPQq3Qf2qrspWwGLK2Ys0PMv8qGHsPkGBKmaiFNJTmyJ27T6w1rxsJvMl9L83mH/xVWiVI8vUNkIC7JexfSgzx8/Vfl2ISUmJltmBEjp7JSzWVHyZxHuNBprA5P5NHl3VMYE1zmE6ACs3LpDfNWq1FUjoNp6/EIAsZsvm/505P1UXIw2TKXH/GyQnyK8OB/FF5og9GMqmW87u5UvwctGCN7/F/sadvMj454yKssyDGfjXlwLOtm3AX9vao2CUITlniy2ov2giGs+dfJfYtvU2SyznIOQjF1qqwxTb1HWM5AvGNsY4v6VoPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BZ2kwHMnelplbCchdbwZ+ia6PwO7lsSSD7ssrmF5dHQ=;
 b=YHOFezUF1yqpjllueuzWf+I0rHCtn83zrPTxn6ZddbZkFWosRcY7pGRkctpTaFd5I318INsrp+RQMmpsEpNjrO1bXE1mmqkIZDv0nFyFWs2LtR75VcpuZZTLMsJP0SuJTo3dkxZyNT5Z8InyQJ1QrId29JvNHTHxv7r7b4eyu1A=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 11/13] hyperv/IOMMU: Enable swiotlb bounce buffer for
 Isolation VM
Thread-Topic: [PATCH V4 11/13] hyperv/IOMMU: Enable swiotlb bounce buffer for
 Isolation VM
Thread-Index: AQHXm2gQQ4+yOBIZA0OE+jNjJUIyPauJR4sw
Date: Thu, 2 Sep 2021 01:27:14 +0000
Message-ID:
 <MWHPR21MB159386DFD2EA14A585288C0FD7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-12-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-12-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=38142b78-d6f5-49b9-b305-836e5d853fb5;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-28T19:01:59Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6b34df77-4512-4c58-304d-08d96db0cbb2
x-ms-traffictypediagnostic: MWHPR21MB0767:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB0767F12FD371AC9DD10BFF9CD7CE9@MWHPR21MB0767.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:50;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 dNq0v0YJw7pRWijNhH6+mIFGGl3G+Qy79F9GPxBTEW6zvQTzNqClqowJIrhXZYp9GMtvZLsJp+tfuP2Tq9Qzulp93q84HKxOehoOdf6j5szdLfh8p6NKk4iBe0baCWKjhzr/XVFCpi2tzdKK7mhtiQXQdsKXcc+fAKRo/nIVdw1dBzrRoiMOsLwHrPNJl33LKNf/mTN2SCqHWsKWNnKgs70cJ27xsUSSU9jgWcxgraVidVFIrG+lJL0jVO7cL+tGq8qLvO0/E2f2dApLKfSVWxpmBzjXhM9KrqBRioxe2hXhZApwWIpsjYaGp37/uQZjdvb3HrwX4crtF5ITsReC2o5kK3wwjo37h/aJLeedmXT1rVes9TIc8Y6cUWGdejCHMoGfNYeisGNCby/L7jGOxFU4CXJgRRgKt+VRvayQwPh+fP+FsOEj9loTggZaNzdFNxYYj+QASN+y6ba9wth9UbMhI82frDFbwjruTQ1KnVrA7wIBMd4ZkpbfdkqkRbGIRGW11qQrY6a06A9mA4mnbSP/WIOH/9adlHFdVVA6F5WfiQWe7hUqMuqYC8DXeSz120oAzCMrDApGxQ5zZQ8MGTdwIfibzSrUc4ysz7CPc2G9TBXSbhSG+R2Aqxkuh7PuwyMx2hx3fuLZ2svCeqDC6n1ovPoJYs3UQU37UGg0XjVuR/ltirl+LZk8Sm+t5QRDajCrPxWnjESMJSZ46C9Cil2Ckv2R+mNO5DdNtc7RKUA=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(7366002)(71200400001)(83380400001)(66476007)(66946007)(82950400001)(7416002)(316002)(38100700002)(7406005)(86362001)(9686003)(76116006)(921005)(66446008)(4326008)(64756008)(33656002)(10290500003)(2906002)(508600001)(186003)(7696005)(6506007)(55016002)(38070700005)(82960400001)(8676002)(122000001)(8936002)(52536014)(8990500004)(54906003)(26005)(5660300002)(110136005)(66556008);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?49jzUXvgf7ZLIZvLg6Q1/1eUHzzwc3TfIDOhgzDpWNqfBbBgjatdlOGSnacX?=
 =?us-ascii?Q?ENqJRp+Z0muZsNhTVYljQwAe/VGj2sStR4WnwzZc79blYZvjVMk+GG0MweJe?=
 =?us-ascii?Q?2JqqGqlvmjOnf4EXWk+S1brb/7E1P9hA5Oiawk0Zfw+v4bTMJIi/V6FXez2D?=
 =?us-ascii?Q?ci8LPNuzdPxLnUkIwX/5AfMsb+4g5Tm33b7LPpZMtiLGrhDTlCH9hRF6dz28?=
 =?us-ascii?Q?XQ+9iLPB0lid5Rd+ouoAatq4QEGqryt/JqQrDB64Arr+YXHMpGTYN5K81L9Y?=
 =?us-ascii?Q?wJF628NgGQSRV9cgx64AmqJ4pdV7onoO9aT3xMzSZ7uuxdkjtNm7cFyuKz7d?=
 =?us-ascii?Q?JdP4kC1d33iP9/Fy/9xiKq9fxhkEgySgN2uEJcwA62mHZoShL5HfMPw17oL1?=
 =?us-ascii?Q?2o5NLnScTG89+ja+xQQIH9+ZFXAmtNDxDLjQXexz4s3d7oZHGsRHDpkor/3z?=
 =?us-ascii?Q?Xz7K1yGKF1+d3geiaFVInqOs1B1PRX/ISoy66V8EJG83zv/tng4lN62g0VU6?=
 =?us-ascii?Q?EHFWVPQwulkliliYaj83EyaMBfE0tKy6lSA9i/e7aqiK/GVz+KdwRe/GM83C?=
 =?us-ascii?Q?3IqSpRfKdZHHrQfXW0b9Tz9m5791BaKicfzQLqopfkn+Bm9fAzOyLsDYPId/?=
 =?us-ascii?Q?rRPI/mORNTvsSZ43cQThwfR//IzO27dIfeRi4EqtEd4TmpkdxE6hNj7xIF2E?=
 =?us-ascii?Q?DY0hfGunCRPEpkvCxRGUVvRHxkzQ7Xi7lCTVYFCNkZRLEyPnk2DfzTfphJ5v?=
 =?us-ascii?Q?f1FhT8/5vTYzmdsBX7mDQuxMQb1pNuSND18uQ6wVzHML9exonLJWPKldospq?=
 =?us-ascii?Q?PfKItemm+7ZwQVg+Dfn+jlBmMuB0uiaFKZXWzB3xyMvwvy2H0gE5wETBA7pM?=
 =?us-ascii?Q?pYlH/htXd34h/GFdDYo8TqMI5lbfI1zQ7Xmpa01bOMRzT3yr7TvvLFx3ozXT?=
 =?us-ascii?Q?59AoBSq0rNeJfOXXJmMBTJHZbtZVg2lw9u0S788POdPmqLonKm/LA64/QtiU?=
 =?us-ascii?Q?nYnglQMC4AF/hH8P9+CyOOFokvkJeUrKsyyF7Hy9Wgy5QC1rSCS0xM7hOTLf?=
 =?us-ascii?Q?EF+5WVbpx6UKzzsgThY0B/RSWIk5wXoSLEr/aDXhWAhyPVUznQqaLoI1o7tg?=
 =?us-ascii?Q?DxYNRg48YTnP7nuF8p2UheDY/7jtOJGIz+cBnGn0brq9gya+pKXcNZU6fusb?=
 =?us-ascii?Q?lQghAss56HBsmXFDmJQVOp/5XUoBjjMle/4yh6rwurNoEWHZOaAGPgx9l3UA?=
 =?us-ascii?Q?ODSn8cZc52xyUtnnI4yHzECBTay9uqWw5IVAdqEZuTZHLr7+fX3XTOArA0q7?=
 =?us-ascii?Q?YItS/HFI811TBI46JLo079y1?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6b34df77-4512-4c58-304d-08d96db0cbb2
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 01:27:14.5072
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2KGO2qEIsLkOxALKzCEuY2PwU2DdLRgZF+IJZLEV/P7dB4OctEWxjII6iy63CCsAmyNnusPkM8cmgSdfqC69NrrNVbuVYcoMq/KOXZBRD5g=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0767

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> hyperv Isolation VM requires bounce buffer support to copy
> data from/to encrypted memory and so enable swiotlb force
> mode to use swiotlb bounce buffer for DMA transaction.
>=20
> In Isolation VM with AMD SEV, the bounce buffer needs to be
> accessed via extra address space which is above shared_gpa_boundary
> (E.G 39 bit address line) reported by Hyper-V CPUID ISOLATION_CONFIG.
> The access physical address will be original physical address +
> shared_gpa_boundary. The shared_gpa_boundary in the AMD SEV SNP
> spec is called virtual top of memory(vTOM). Memory addresses below
> vTOM are automatically treated as private while memory above
> vTOM is treated as shared.
>=20
> Swiotlb bounce buffer code calls dma_map_decrypted()
> to mark bounce buffer visible to host and map it in extra
> address space. Populate dma memory decrypted ops with hv
> map/unmap function.
>=20
> Hyper-V initalizes swiotlb bounce buffer and default swiotlb
> needs to be disabled. pci_swiotlb_detect_override() and
> pci_swiotlb_detect_4gb() enable the default one. To override
> the setting, hyperv_swiotlb_detect() needs to run before
> these detect functions which depends on the pci_xen_swiotlb_
> init(). Make pci_xen_swiotlb_init() depends on the hyperv_swiotlb
> _detect() to keep the order.
>=20
> The map function vmap_pfn() can't work in the early place
> hyperv_iommu_swiotlb_init() and so initialize swiotlb bounce
> buffer in the hyperv_iommu_swiotlb_later_init().
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
>        * Get hyperv bounce bufffer size via default swiotlb
>        bounce buffer size function and keep default size as
>        same as the one in the AMD SEV VM.
> ---
>  arch/x86/hyperv/ivm.c           | 28 +++++++++++++++
>  arch/x86/include/asm/mshyperv.h |  2 ++
>  arch/x86/mm/mem_encrypt.c       |  3 +-
>  arch/x86/xen/pci-swiotlb-xen.c  |  3 +-
>  drivers/hv/vmbus_drv.c          |  3 ++
>  drivers/iommu/hyperv-iommu.c    | 61 +++++++++++++++++++++++++++++++++
>  include/linux/hyperv.h          |  1 +
>  7 files changed, 99 insertions(+), 2 deletions(-)
>=20
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> index e761c67e2218..84563b3c9f3a 100644
> --- a/arch/x86/hyperv/ivm.c
> +++ b/arch/x86/hyperv/ivm.c
> @@ -294,3 +294,31 @@ int hv_set_mem_host_visibility(unsigned long addr, i=
nt numpages, bool visible)
>=20
>  	return __hv_set_mem_host_visibility((void *)addr, numpages, visibility)=
;
>  }
> +
> +/*
> + * hv_map_memory - map memory to extra space in the AMD SEV-SNP Isolatio=
n VM.
> + */
> +void *hv_map_memory(void *addr, unsigned long size)
> +{
> +	unsigned long *pfns =3D kcalloc(size / HV_HYP_PAGE_SIZE,
> +				      sizeof(unsigned long), GFP_KERNEL);

Should be PAGE_SIZE, not HV_HYP_PAGE_SIZE, since this code
only manipulates guest page tables.  There's no communication with
Hyper-V that requires HV_HYP_PAGE_SIZE.

> +	void *vaddr;
> +	int i;
> +
> +	if (!pfns)
> +		return NULL;
> +
> +	for (i =3D 0; i < size / PAGE_SIZE; i++)
> +		pfns[i] =3D virt_to_hvpfn(addr + i * PAGE_SIZE) +

Use virt_to_pfn(), not virt_to_hvpfn(), for the same reason.

> +			(ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
> +
> +	vaddr =3D vmap_pfn(pfns, size / PAGE_SIZE, PAGE_KERNEL_IO);
> +	kfree(pfns);
> +
> +	return vaddr;
> +}
> +
> +void hv_unmap_memory(void *addr)
> +{
> +	vunmap(addr);
> +}
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index b77f4caee3ee..627fcf8d443c 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -252,6 +252,8 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct h=
v_interrupt_entry *entry);
>  int hv_mark_gpa_visibility(u16 count, const u64 pfn[],
>  			   enum hv_mem_host_visibility visibility);
>  int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible);
> +void *hv_map_memory(void *addr, unsigned long size);
> +void hv_unmap_memory(void *addr);
>  void hv_sint_wrmsrl_ghcb(u64 msr, u64 value);
>  void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value);
>  void hv_signal_eom_ghcb(void);
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index ff08dc463634..e2db0b8ed938 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -30,6 +30,7 @@
>  #include <asm/processor-flags.h>
>  #include <asm/msr.h>
>  #include <asm/cmdline.h>
> +#include <asm/mshyperv.h>
>=20
>  #include "mm_internal.h"
>=20
> @@ -202,7 +203,7 @@ void __init sev_setup_arch(void)
>  	phys_addr_t total_mem =3D memblock_phys_mem_size();
>  	unsigned long size;
>=20
> -	if (!sev_active())
> +	if (!sev_active() && !hv_is_isolation_supported())
>  		return;
>=20
>  	/*
> diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xe=
n.c
> index 54f9aa7e8457..43bd031aa332 100644
> --- a/arch/x86/xen/pci-swiotlb-xen.c
> +++ b/arch/x86/xen/pci-swiotlb-xen.c
> @@ -4,6 +4,7 @@
>=20
>  #include <linux/dma-map-ops.h>
>  #include <linux/pci.h>
> +#include <linux/hyperv.h>
>  #include <xen/swiotlb-xen.h>
>=20
>  #include <asm/xen/hypervisor.h>
> @@ -91,6 +92,6 @@ int pci_xen_swiotlb_init_late(void)
>  EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
>=20
>  IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
> -		  NULL,
> +		  hyperv_swiotlb_detect,
>  		  pci_xen_swiotlb_init,
>  		  NULL);
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 57bbbaa4e8f7..f068e22a5636 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -23,6 +23,7 @@
>  #include <linux/cpu.h>
>  #include <linux/sched/task_stack.h>
>=20
> +#include <linux/dma-map-ops.h>
>  #include <linux/delay.h>
>  #include <linux/notifier.h>
>  #include <linux/panic_notifier.h>
> @@ -2081,6 +2082,7 @@ struct hv_device *vmbus_device_create(const guid_t =
*type,
>  	return child_device_obj;
>  }
>=20
> +static u64 vmbus_dma_mask =3D DMA_BIT_MASK(64);
>  /*
>   * vmbus_device_register - Register the child device
>   */
> @@ -2121,6 +2123,7 @@ int vmbus_device_register(struct hv_device *child_d=
evice_obj)
>  	}
>  	hv_debug_add_dev_dir(child_device_obj);
>=20
> +	child_device_obj->device.dma_mask =3D &vmbus_dma_mask;
>  	return 0;
>=20
>  err_kset_unregister:
> diff --git a/drivers/iommu/hyperv-iommu.c b/drivers/iommu/hyperv-iommu.c
> index e285a220c913..899563551574 100644
> --- a/drivers/iommu/hyperv-iommu.c
> +++ b/drivers/iommu/hyperv-iommu.c
> @@ -13,14 +13,22 @@
>  #include <linux/irq.h>
>  #include <linux/iommu.h>
>  #include <linux/module.h>
> +#include <linux/hyperv.h>
> +#include <linux/io.h>
>=20
>  #include <asm/apic.h>
>  #include <asm/cpu.h>
>  #include <asm/hw_irq.h>
>  #include <asm/io_apic.h>
> +#include <asm/iommu.h>
> +#include <asm/iommu_table.h>
>  #include <asm/irq_remapping.h>
>  #include <asm/hypervisor.h>
>  #include <asm/mshyperv.h>
> +#include <asm/swiotlb.h>
> +#include <linux/dma-map-ops.h>
> +#include <linux/dma-direct.h>
> +#include <linux/set_memory.h>
>=20
>  #include "irq_remapping.h"
>=20
> @@ -36,6 +44,9 @@
>  static cpumask_t ioapic_max_cpumask =3D { CPU_BITS_NONE };
>  static struct irq_domain *ioapic_ir_domain;
>=20
> +static unsigned long hyperv_io_tlb_size;
> +static void *hyperv_io_tlb_start;
> +
>  static int hyperv_ir_set_affinity(struct irq_data *data,
>  		const struct cpumask *mask, bool force)
>  {
> @@ -337,4 +348,54 @@ static const struct irq_domain_ops hyperv_root_ir_do=
main_ops =3D {
>  	.free =3D hyperv_root_irq_remapping_free,
>  };
>=20
> +void __init hyperv_iommu_swiotlb_init(void)
> +{
> +	/*
> +	 * Allocate Hyper-V swiotlb bounce buffer at early place
> +	 * to reserve large contiguous memory.
> +	 */
> +	hyperv_io_tlb_size =3D swiotlb_size_or_default();
> +	hyperv_io_tlb_start =3D memblock_alloc(
> +		hyperv_io_tlb_size, HV_HYP_PAGE_SIZE);

Could the alignment be specified as just PAGE_SIZE?  I don't
see any particular relationship here to the Hyper-V page size.

> +
> +	if (!hyperv_io_tlb_start) {
> +		pr_warn("Fail to allocate Hyper-V swiotlb buffer.\n");
> +		return;
> +	}
> +}
> +
> +int __init hyperv_swiotlb_detect(void)
> +{
> +	if (hypervisor_is_type(X86_HYPER_MS_HYPERV)
> +	    && hv_is_isolation_supported()) {
> +		/*
> +		 * Enable swiotlb force mode in Isolation VM to
> +		 * use swiotlb bounce buffer for dma transaction.
> +		 */
> +		swiotlb_force =3D SWIOTLB_FORCE;
> +
> +		dma_memory_generic_decrypted_ops.map =3D hv_map_memory;
> +		dma_memory_generic_decrypted_ops.unmap =3D hv_unmap_memory;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +void __init hyperv_iommu_swiotlb_later_init(void)
> +{
> +	/*
> +	 * Swiotlb bounce buffer needs to be mapped in extra address
> +	 * space. Map function doesn't work in the early place and so
> +	 * call swiotlb_late_init_with_tbl() here.
> +	 */
> +	if (swiotlb_late_init_with_tbl(hyperv_io_tlb_start,
> +				       hyperv_io_tlb_size >> IO_TLB_SHIFT))
> +		panic("Fail to initialize hyperv swiotlb.\n");
> +}
> +
> +IOMMU_INIT_FINISH(hyperv_swiotlb_detect,
> +		  NULL, hyperv_iommu_swiotlb_init,
> +		  hyperv_iommu_swiotlb_later_init);
> +
>  #endif
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index 757e09606fd3..724a735d722a 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1739,6 +1739,7 @@ int hyperv_write_cfg_blk(struct pci_dev *dev, void =
*buf, unsigned int len,
>  int hyperv_reg_block_invalidate(struct pci_dev *dev, void *context,
>  				void (*block_invalidate)(void *context,
>  							 u64 block_mask));
> +int __init hyperv_swiotlb_detect(void);
>=20
>  struct hyperv_pci_block_ops {
>  	int (*read_block)(struct pci_dev *dev, void *buf, unsigned int buf_len,
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 01:31:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 01:31:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176770.321702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbZa-0005ge-3c; Thu, 02 Sep 2021 01:31:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176770.321702; Thu, 02 Sep 2021 01:31:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbZZ-0005gX-U7; Thu, 02 Sep 2021 01:31:21 +0000
Received: by outflank-mailman (input) for mailman id 176770;
 Thu, 02 Sep 2021 01:31:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do2V=NY=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLbZY-0005gR-6d
 for xen-devel@lists.xen.org; Thu, 02 Sep 2021 01:31:20 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.73]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae9e7f78-35e3-404d-aa11-9ba934235093;
 Thu, 02 Sep 2021 01:31:12 +0000 (UTC)
Received: from AM4PR0302CA0009.eurprd03.prod.outlook.com (2603:10a6:205:2::22)
 by AS8PR08MB7142.eurprd08.prod.outlook.com (2603:10a6:20b:402::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 01:30:45 +0000
Received: from VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:205:2:cafe::d9) by AM4PR0302CA0009.outlook.office365.com
 (2603:10a6:205:2::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 01:30:45 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT064.mail.protection.outlook.com (10.152.19.210) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.17 via Frontend Transport; Thu, 2 Sep 2021 01:30:44 +0000
Received: ("Tessian outbound 6ba9ec11d42b:v103");
 Thu, 02 Sep 2021 01:30:43 +0000
Received: from c82113e1ef35.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4A84C218-891F-4D71-9360-D6E99C6F380C.1; 
 Thu, 02 Sep 2021 01:30:42 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c82113e1ef35.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 01:30:42 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6890.eurprd08.prod.outlook.com (2603:10a6:10:2a8::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 01:30:36 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.026; Thu, 2 Sep 2021
 01:30:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae9e7f78-35e3-404d-aa11-9ba934235093
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AQMxAsYY4/hP7OgFAreOwfcANt90hvUXFToNMh6+Q7g=;
 b=JqPWYmrmfSLIwU5SPpKSHUtAKwQaPILY5Eam00Lr4bqt6xKqITBCnqPQGL+rEvv9stV9TWTuvmwhAjTXfrLQcJFrHQTjbZRWf+NFOs1ljkZE+HH2GzthwOLNFybs0vYr3rbCs8GLzh7b7imnumjVpT0/xcmmtd6H8SkGPxk1t6w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xen.org; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;lists.xen.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SvRXmSb0duufpIrkmzkzF4DaOlSsSTpVEgPlUSWETocWkgn7zUkis24RazJZYMu+lYbfbSgTFOyHmkls7mM4bvcZ4/dYyg/HrCOlqI90efZ/unM8l7EK+19vnuO7PjmP1CFys5+kmrn//4QwpeT0j5g0VDigONk4FhJWmDqoiQMYuzsZd9GLEnU0HDWrVgKGIlfGDaDf3OuRoArtQmKddpH7G/84JWR6S971KqmKMXeJPb+/xqbTFcxkoBHWUUMAco3TvAS13kWXTFfGmxRfno4zfaNKeHnnFBsq6Anc+P2Ee/u6jtvpZGeUoN+Jc+o40pJpcAABqSGnTuTIjJF/QQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AQMxAsYY4/hP7OgFAreOwfcANt90hvUXFToNMh6+Q7g=;
 b=h00dH6vLO7SJXiYiJIEFxkxkcXk6/yfcrSZJLNlTA53aKU+vZKoVYqXgcvHc9Wf9vhom2A7SbUPVx0RYtOTD6MzilzSDeYk9M4xKq2rL6CRwqadfFNDM1R6fl1JYWV8xq7CkCmahbk/W3PjGlAK2+H5gz1DS5VkoxMET7Rb0sG5hQfaWGqqKRlODOAHttLvxnp/ZTtNLJd1g1/lYf6Qoz9eCFylqUbxx04nFZNpvaU1Q+uYWda1s8uymLgmDz5TP6gmLvhQa0777gBFJIFlysHKTWLM+p7lN3N1vYXrpjedlA+t/mAS7j0fcq6i7eMFk9AtUFt3ll/Xuzq1P5QdLOA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AQMxAsYY4/hP7OgFAreOwfcANt90hvUXFToNMh6+Q7g=;
 b=JqPWYmrmfSLIwU5SPpKSHUtAKwQaPILY5Eam00Lr4bqt6xKqITBCnqPQGL+rEvv9stV9TWTuvmwhAjTXfrLQcJFrHQTjbZRWf+NFOs1ljkZE+HH2GzthwOLNFybs0vYr3rbCs8GLzh7b7imnumjVpT0/xcmmtd6H8SkGPxk1t6w=
From: Wei Chen <Wei.Chen@arm.com>
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
CC: Oleksandr Tyshchenko <olekstysh@gmail.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin
	<Kaly.Xin@arm.com>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd
 Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com"
	<stefanha@redhat.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik
	<cvanscha@qti.qualcomm.com>, "pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker
	<jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien
 Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, Paul Durrant
	<paul@xen.org>, nd <nd@arm.com>, Xen Devel <xen-devel@lists.xen.org>
Subject: RE: Enabling hypervisor agnosticism for VirtIO backends
Thread-Topic: Enabling hypervisor agnosticism for VirtIO backends
Thread-Index:
 AQHXiWXOlXnRp9GT30mY/CfP5cid16tt4TUAgAVQpQCAAsKLQIABdsmAgAADLTCAAWVyAIAAMB+ggAMGMQCACZ/8gIAAKWUQgAd5oYCAAeKfwIAAF2QAgADY4nA=
Date: Thu, 2 Sep 2021 01:30:34 +0000
Message-ID:
 <DB9PR08MB6857256B9927EDB9772636139ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References:
 <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa>
 <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210831061805.GB69817@laputa>
 <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210901122910.GA100270@laputa>
In-Reply-To: <20210901122910.GA100270@laputa>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 31FB94E798E61943B3D0F32FDA2C2E35.0
x-checkrecipientchecked: true
Authentication-Results-Original: linaro.org; dkim=none (message not signed)
 header.d=none;linaro.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 9c1bad50-8fe3-4233-9823-08d96db148d5
x-ms-traffictypediagnostic: DB9PR08MB6890:|AS8PR08MB7142:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB7142B0A9267101BF7A1DC5049ECE9@AS8PR08MB7142.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 7c9Q0dU+GVK7k8AaKwPi0cpGAIo5AYqArgzjy4DN3zlofthc8I/0x9XfpDPXn08NY/M+d5ppel6x8kqPIZULDklE55MfHFciGTCbU9XhbwhyMtlV16/hwKJkuyc5cdH5MY8gcVuJ25HEHndlJJXzwZmPQ77752N0KOYUETQ5Cge/8uDuhGEORFaFFaaT6i4OYOyfm9s28Il2vXX+4T0yjz+KRZC5kd2aeqQmjAiAYvx/7RbWs8T9AqZ7HNhtsGE8Zax2GgnSpU6GiaUqzyejLVIWxrt05XLliWyQyVmWrLBs1aMqoRmT/pLm2lR6nt9pT5HEk0LFE1YXPp7K32RjFUqpupH8DVm3ZhtN71NipAGlWYw77VJTqX0qUIl2drmwocLCbdsMM8R8kX5NogbO3IVhe4NUjAiHKjc7ow1KRN+hRrjsni71s541TLNS7P600IIL8gDeUNcg8Ua2KqAeG5fum+vrrKDhy/ZyJHyNJJiVmqEcRX4zRxYFhqoubn1VRyInMaA8TR/v0KGAnZ6mQTf+nYVUkMbYkRwZH58ec6MIw/OPc2cIUL5jofTX0y9RihXT3zt1a0YYgQx6APyPn2Q3YWJBuSkPnFTnzvJmu2b0HoE7rU2lYbB7mPq9CNMeREDd7c+Vst27AUC8dzESdoksyVEb1EvFgg7A7EcGbPmmqvS1u7k1z/nFtU5krEta+YW1Mvxmo5t9/9rWvRIuRQJ/cdH0RqcFoZI69aIB4zJAEELvL9h6vQTfuc4U4cWyy/zKTX7NLmtzsmKjkyIZzvYyZx+BLeHCd+BBoA9WdAA7q73bK/jQL3l70VbOn9okPhVuTvawabpZeRzX4l8e5w==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(39860400002)(366004)(396003)(376002)(30864003)(33656002)(186003)(54906003)(86362001)(71200400001)(83380400001)(4326008)(66574015)(8676002)(478600001)(52536014)(76116006)(7416002)(38100700002)(5660300002)(6916009)(66946007)(53546011)(66556008)(64756008)(966005)(55016002)(9686003)(8936002)(26005)(122000001)(316002)(38070700005)(6506007)(66446008)(7696005)(66476007)(2906002)(579004)(559001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?MjRvTUUwWHRwRXFwMmRFNERFZkJZYk4wWXIrUHpGbUxIV1N0SWo3NTFTYkFn?=
 =?utf-8?B?T0wxeHZnRWdINS80amtrY09TR1VYNU51djI1SGZhNUNWVTdkRmIyZCtIYUxF?=
 =?utf-8?B?VlB2MkYvb3NVQ1BQMC9kYi9vMlQ0VENoSzJ6b1NQbWFtbFlaYTlVM0p2MDNw?=
 =?utf-8?B?STd0WUtkSjJrQUJtdDdZdHVTWDFiWWRNNHROSTBKVHFXa2l2TVVBZzNGM2hz?=
 =?utf-8?B?RUo5bzZ0L3lBSVVBK2ppczJWS1gzWHBOUWFCeGR1U1FDWTA5QVEydTdaanhY?=
 =?utf-8?B?NE1nQlhYUFpOTFVMT3lXczhmNERpb0ZWRy9nWFdYdHlZTzNDVVpyRjNISVRo?=
 =?utf-8?B?bHdpeXg2SGV0U09GMVlTMk02UXFZYmV6OGRJSk1aMGFrNGh1ZllYdldmYjBQ?=
 =?utf-8?B?UzM4aVhJY2VNUyt5dHZEeDhDa2dmTjAxU3N2MTBvR0J3V1lvNE1WNzFNVHZR?=
 =?utf-8?B?Z3J4TGtvcnFRUlR5SzEwU0lQRnNxZU96NzB2UGltVFd3M1l5T2U1Uk0xNHl2?=
 =?utf-8?B?RW4vVy9OZmt4ZGV4LzczOTJKcUgzWHlYc0xqcEJzaEhGaFZHS1NwUzF5bkJq?=
 =?utf-8?B?ZHhSY2pWNUltdkZjTmpUS2t2aGRXOS9JZnZYYTk0NnlBY3Y4c1o2QmRidmZm?=
 =?utf-8?B?V2hZTUpTMGxtM1liMy9GK3dkeGRSUFhDM3VESjBkQzhFQkxuRVJqS2pIbllu?=
 =?utf-8?B?VCthaXppVXY1dWo4Ym5HSkc2MCt0VlRia0Z3Z0tOL1dPWlc4RUFmUVN3eDZw?=
 =?utf-8?B?NmVBNy83MXpXTVBGVUI2elF1TjdOSzVjQllCMGRPUG1MQXlNbWp0U0VvT0w1?=
 =?utf-8?B?MHRnT2NpZmFKRmxER3Biekg2MERqWm9QbHJ3eEJOYVNvMWFlTFczZEVsWU1S?=
 =?utf-8?B?dnkrTXpkTnlMTElhMW01TjFsV1FqUUo5aWVoUmJUZkYvalhyVWF3ZGE2cDli?=
 =?utf-8?B?Ry85R3ZJMXZJWXM4Wmx5YkNicnFVVkpVSWdiNkVnNmZycWZOanhBSnBlVk1N?=
 =?utf-8?B?NStWM1NpUHZVWU1nSFI0ZVhCQVdBUnpZWC9oTUVFNGFIUWFacDZXQ2hlVkpR?=
 =?utf-8?B?S01vOVZlSC9Ob29vN0ZEbU9jblFkRjkwZzRCaVc1cGFESUh5QWgwSytab1VO?=
 =?utf-8?B?dUdnMTZvaUUzUGljNm11dWMzRHM3alpOcnovYUEzdUUvSGN3RkFJelNRbmp5?=
 =?utf-8?B?QUZEeDhwREkwcWxqMW40KzFSUXAzTnpFRXc1UE1EL0xXQm45UzZhOWZLamtk?=
 =?utf-8?B?S2xVK2dqeVlodmVrcUlheU5oUzk3V1ljR01kc1VDcHRHSGdFTHR6aXVPZkcx?=
 =?utf-8?B?YnVCRjJ6TDhKV0JqN3hFUmFpd2tjaTJKS0lTenBpbnR2M3ErOEdLUnJWdk1j?=
 =?utf-8?B?Ym5tYmExRHpDeTFPcnlyZ1JUZFBxejFjdERSRE9hYUg5ckpjZlo3Mmk1SG5j?=
 =?utf-8?B?Y3daaXJhcDgrK1RXMyticitPbFJRdEZUVDVtWjRhUmJCbDcxU3k3N2RJeUpm?=
 =?utf-8?B?UG0zY29lU1VVbkpIT3RMUjdENEg5RkZ0dnZxcUVLSGRkNno4SGVma2dHL0N4?=
 =?utf-8?B?RXkvaURseTJrQ0NTRWdEeFk3ZFlSUDNHVlJJcGRnei9GZ0Z1dHZPU042WWlu?=
 =?utf-8?B?QTN0bjlHMnlIR3VUZUlyMUU2Tkl1MjFobWVpRHF0aWJSbVE1R25CUC9jWHM1?=
 =?utf-8?B?cEVrSkZZYVdLYURjYURjbEVhejcxMi9SeUxqWVJ5dGRweTd1dEhMRmg5Tys5?=
 =?utf-8?Q?hwc7OuVDqmgj2l39Ns6zwnIr1g9XFXveepCL5MX?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6890
Original-Authentication-Results: linaro.org; dkim=none (message not signed)
 header.d=none;linaro.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	4ef72bad-92f2-4db1-ca58-08d96db1431a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cER8HBOGBs2bSvSTR4asddcv2Xjhmkjb54PyUd4kmJWGOczx3X8plfrzNmPHhR5KwU9/0w+n7rF6l8fQSIXTZAP5lYTEjFfhIKFZRU2nBlJW1HJI8+mIALdAi0h39TbGnhG9pFzphv2egVBguOeqIBYEG1Lj3JexoKxPwv80hOotKr9O6Q1KDgFPyJA7ZwHMBrH2bkJk8sGyuyQxqi0C5izPb+z8YwSs9G2rsakko4TJ6O4m628Lgio+UV77SI7XJckOG0OAv+2QWISN5kZHxJlbjQn3xl/LkE84K8nsSZTqt3yFyomlmsIqPms1sEwig0Ud5vI7SICgYsQoC0a+AkxDipDEA1FBuA9Yapo2sGsyQlYZq/iAWvj7rfXAGfflzDuT7EUS67Tbxoz/MrWdAFc54L8Bes7grUloqOPuJK1Oer443lJ4MfZA5UAJyfmBysNNVHqbptjTIbPNLlZSjk6nj3JEpz9hW1IuEo0btY7igH3x7UN9GQYKIm7Y5lYOO0AqTHDnHs6AnWbZIIHIPPs67KDvT7VAvKcsUzlmfcBmgiy7mMRq41oeAO/+CsZTTFyjQlPKZVmeU0FrqtTDL6gb1JWaXY0fWqyEitMMNHOaCu0TUu0t4Hr43JbpztDqlm9M+rP19eBjZJnGyttkdR8uKSjFpGat6DUWWZJMoAQT+Fs+FldWhRMGx2GDdYMj/+Q/DRmFaMcnSE2xDgVscFIMcWBgwaIobL/Tuva9ZqP0fCRfD2RW06AIiHvjOfzyn9tmY1ovuYVM7mOYrVomRfKsoZliWy/IUr6xXEOzPQ+6hoCQP0tisOMYD4V4vgxelD6FZumQP+7Im6T+vczcKg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(136003)(346002)(36840700001)(46966006)(186003)(8676002)(53546011)(70586007)(2906002)(70206006)(82310400003)(86362001)(356005)(26005)(81166007)(336012)(54906003)(7696005)(5660300002)(6506007)(6862004)(8936002)(82740400003)(316002)(33656002)(9686003)(47076005)(52536014)(66574015)(478600001)(83380400001)(55016002)(30864003)(36860700001)(966005)(4326008)(579004)(559001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 01:30:44.7099
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c1bad50-8fe3-4233-9823-08d96db148d5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7142

SGkgQWthc2hpLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEFLQVNI
SSBUYWthaGlybyA8dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMeaXpSAyMDoyOQ0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBP
bGVrc2FuZHIgVHlzaGNoZW5rbyA8b2xla3N0eXNoQGdtYWlsLmNvbT47IFN0ZWZhbm8gU3RhYmVs
bGluaQ0KPiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IEFsZXggQmVubj8/ZSA8YWxleC5iZW5u
ZWVAbGluYXJvLm9yZz47IEthbHkgWGluDQo+IDxLYWx5LlhpbkBhcm0uY29tPjsgU3RyYXRvcyBN
YWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLWxpc3RzLmxpbmFyby5vcmc+Ow0KPiB2aXJ0aW8t
ZGV2QGxpc3RzLm9hc2lzLW9wZW4ub3JnOyBBcm5kIEJlcmdtYW5uIDxhcm5kLmJlcmdtYW5uQGxp
bmFyby5vcmc+Ow0KPiBWaXJlc2ggS3VtYXIgPHZpcmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3Rl
ZmFubyBTdGFiZWxsaW5pDQo+IDxzdGVmYW5vLnN0YWJlbGxpbmlAeGlsaW54LmNvbT47IHN0ZWZh
bmhhQHJlZGhhdC5jb207IEphbiBLaXN6a2ENCj4gPGphbi5raXN6a2FAc2llbWVucy5jb20+OyBD
YXJsIHZhbiBTY2hhaWsgPGN2YW5zY2hhQHF0aS5xdWFsY29tbS5jb20+Ow0KPiBwcmF0aWtwQHF1
aWNpbmMuY29tOyBTcml2YXRzYSBWYWRkYWdpcmkgPHZhdHNhQGNvZGVhdXJvcmEub3JnPjsgSmVh
bi0NCj4gUGhpbGlwcGUgQnJ1Y2tlciA8amVhbi1waGlsaXBwZUBsaW5hcm8ub3JnPjsgTWF0aGll
dSBQb2lyaWVyDQo+IDxtYXRoaWV1LnBvaXJpZXJAbGluYXJvLm9yZz47IE9sZWtzYW5kciBUeXNo
Y2hlbmtvDQo+IDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5kIE1hcnF1
aXMNCj4gPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IEFydGVtIE15Z2FpZXYgPEFydGVtX015
Z2FpZXZAZXBhbS5jb20+OyBKdWxpZW4NCj4gR3JhbGwgPGp1bGllbkB4ZW4ub3JnPjsgSnVlcmdl
biBHcm9zcyA8amdyb3NzQHN1c2UuY29tPjsgUGF1bCBEdXJyYW50DQo+IDxwYXVsQHhlbi5vcmc+
OyBuZCA8bmRAYXJtLmNvbT47IFhlbiBEZXZlbCA8eGVuLWRldmVsQGxpc3RzLnhlbi5vcmc+DQo+
IFN1YmplY3Q6IFJlOiBFbmFibGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIGZvciBWaXJ0SU8g
YmFja2VuZHMNCj4gDQo+IEhpIFdlaSwNCj4gDQo+IE9uIFdlZCwgU2VwIDAxLCAyMDIxIGF0IDEx
OjEyOjU4QU0gKzAwMDAsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IEhpIEFrYXNoaSwNCj4gPg0KPiA+
ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+IEZyb206IEFLQVNISSBUYWthaGly
byA8dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+DQo+ID4gPiBTZW50OiAyMDIx5bm0OOaciDMx
5pelIDE0OjE4DQo+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiBD
YzogT2xla3NhbmRyIFR5c2hjaGVua28gPG9sZWtzdHlzaEBnbWFpbC5jb20+OyBTdGVmYW5vIFN0
YWJlbGxpbmkNCj4gPiA+IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5uPz9lIDxh
bGV4LmJlbm5lZUBsaW5hcm8ub3JnPjsgS2FseQ0KPiBYaW4NCj4gPiA+IDxLYWx5LlhpbkBhcm0u
Y29tPjsgU3RyYXRvcyBNYWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLQ0KPiBsaXN0cy5saW5h
cm8ub3JnPjsNCj4gPiA+IHZpcnRpby1kZXZAbGlzdHMub2FzaXMtb3Blbi5vcmc7IEFybmQgQmVy
Z21hbm4NCj4gPGFybmQuYmVyZ21hbm5AbGluYXJvLm9yZz47DQo+ID4gPiBWaXJlc2ggS3VtYXIg
PHZpcmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBTdGFiZWxsaW5pDQo+ID4gPiA8c3Rl
ZmFuby5zdGFiZWxsaW5pQHhpbGlueC5jb20+OyBzdGVmYW5oYUByZWRoYXQuY29tOyBKYW4gS2lz
emthDQo+ID4gPiA8amFuLmtpc3prYUBzaWVtZW5zLmNvbT47IENhcmwgdmFuIFNjaGFpayA8Y3Zh
bnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiBwcmF0aWtwQHF1aWNpbmMuY29tOyBTcml2
YXRzYSBWYWRkYWdpcmkgPHZhdHNhQGNvZGVhdXJvcmEub3JnPjsgSmVhbi0NCj4gPiA+IFBoaWxp
cHBlIEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUgUG9pcmllcg0K
PiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xla3NhbmRyIFR5c2hjaGVua28N
Cj4gPiA+IDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5kIE1hcnF1aXMN
Cj4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2IDxBcnRlbV9N
eWdhaWV2QGVwYW0uY29tPjsNCj4gSnVsaWVuDQo+ID4gPiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+
OyBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+OyBQYXVsIER1cnJhbnQNCj4gPiA+IDxw
YXVsQHhlbi5vcmc+OyBYZW4gRGV2ZWwgPHhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4g
U3ViamVjdDogUmU6IEVuYWJsaW5nIGh5cGVydmlzb3IgYWdub3N0aWNpc20gZm9yIFZpcnRJTyBi
YWNrZW5kcw0KPiA+ID4NCj4gPiA+IFdlaSwNCj4gPiA+DQo+ID4gPiBPbiBUaHUsIEF1ZyAyNiwg
MjAyMSBhdCAxMjoxMDoxOVBNICswMDAwLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gSGkgQWth
c2hpLA0KPiA+ID4gPg0KPiA+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4g
PiA+ID4gRnJvbTogQUtBU0hJIFRha2FoaXJvIDx0YWthaGlyby5ha2FzaGlAbGluYXJvLm9yZz4N
Cj4gPiA+ID4gPiBTZW50OiAyMDIx5bm0OOaciDI25pelIDE3OjQxDQo+ID4gPiA+ID4gVG86IFdl
aSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+IENjOiBPbGVrc2FuZHIgVHlzaGNo
ZW5rbyA8b2xla3N0eXNoQGdtYWlsLmNvbT47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+ID4gPiA+
IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5uPz9lIDxhbGV4LmJlbm5lZUBsaW5h
cm8ub3JnPjsNCj4gS2FseQ0KPiA+ID4gWGluDQo+ID4gPiA+ID4gPEthbHkuWGluQGFybS5jb20+
OyBTdHJhdG9zIE1haWxpbmcgTGlzdCA8c3RyYXRvcy1kZXZAb3AtDQo+ID4gPiBsaXN0cy5saW5h
cm8ub3JnPjsNCj4gPiA+ID4gPiB2aXJ0aW8tZGV2QGxpc3RzLm9hc2lzLW9wZW4ub3JnOyBBcm5k
IEJlcmdtYW5uDQo+ID4gPiA8YXJuZC5iZXJnbWFubkBsaW5hcm8ub3JnPjsNCj4gPiA+ID4gPiBW
aXJlc2ggS3VtYXIgPHZpcmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBTdGFiZWxsaW5p
DQo+ID4gPiA+ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFAcmVk
aGF0LmNvbTsgSmFuIEtpc3prYQ0KPiA+ID4gPiA+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsg
Q2FybCB2YW4gU2NoYWlrDQo+IDxjdmFuc2NoYUBxdGkucXVhbGNvbW0uY29tPjsNCj4gPiA+ID4g
PiBwcmF0aWtwQHF1aWNpbmMuY29tOyBTcml2YXRzYSBWYWRkYWdpcmkgPHZhdHNhQGNvZGVhdXJv
cmEub3JnPjsNCj4gSmVhbi0NCj4gPiA+ID4gPiBQaGlsaXBwZSBCcnVja2VyIDxqZWFuLXBoaWxp
cHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1IFBvaXJpZXINCj4gPiA+ID4gPiA8bWF0aGlldS5wb2ly
aWVyQGxpbmFyby5vcmc+OyBPbGVrc2FuZHIgVHlzaGNoZW5rbw0KPiA+ID4gPiA+IDxPbGVrc2Fu
ZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiA+ID4gPiA8QmVy
dHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgQXJ0ZW0gTXlnYWlldiA8QXJ0ZW1fTXlnYWlldkBlcGFt
LmNvbT47DQo+ID4gPiBKdWxpZW4NCj4gPiA+ID4gPiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+OyBK
dWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+OyBQYXVsDQo+IER1cnJhbnQNCj4gPiA+ID4g
PiA8cGF1bEB4ZW4ub3JnPjsgWGVuIERldmVsIDx4ZW4tZGV2ZWxAbGlzdHMueGVuLm9yZz4NCj4g
PiA+ID4gPiBTdWJqZWN0OiBSZTogRW5hYmxpbmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lzbSBmb3Ig
VmlydElPIGJhY2tlbmRzDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiBIaSBXZWksDQo+ID4gPiA+ID4N
Cj4gPiA+ID4gPiBPbiBGcmksIEF1ZyAyMCwgMjAyMSBhdCAwMzo0MTo1MFBNICswOTAwLCBBS0FT
SEkgVGFrYWhpcm8gd3JvdGU6DQo+ID4gPiA+ID4gPiBPbiBXZWQsIEF1ZyAxOCwgMjAyMSBhdCAw
ODozNTo1MUFNICswMDAwLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPiA+ID4gSGkgQWthc2hp
77yMDQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdl
LS0tLS0NCj4gPiA+ID4gPiA+ID4gPiBGcm9tOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFr
YXNoaUBsaW5hcm8ub3JnPg0KPiA+ID4gPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ45pyIMTjml6Ug
MTM6MzkNCj4gPiA+ID4gPiA+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+
ID4gPiA+ID4gPiA+ID4gQ2M6IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hAZ21haWwu
Y29tPjsgU3RlZmFubw0KPiA+ID4gU3RhYmVsbGluaQ0KPiA+ID4gPiA+ID4gPiA+IDxzc3RhYmVs
bGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5uPz9lDQo+IDxhbGV4LmJlbm5lZUBsaW5hcm8ub3Jn
PjsNCj4gPiA+ID4gPiBTdHJhdG9zDQo+ID4gPiA+ID4gPiA+ID4gTWFpbGluZyBMaXN0IDxzdHJh
dG9zLWRldkBvcC1saXN0cy5saW5hcm8ub3JnPjsgdmlydGlvLQ0KPiA+ID4gPiA+IGRldkBsaXN0
cy5vYXNpcy0NCj4gPiA+ID4gPiA+ID4gPiBvcGVuLm9yZzsgQXJuZCBCZXJnbWFubiA8YXJuZC5i
ZXJnbWFubkBsaW5hcm8ub3JnPjsgVmlyZXNoDQo+IEt1bWFyDQo+ID4gPiA+ID4gPiA+ID4gPHZp
cmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBTdGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA+
ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFAcmVkaGF0LmNvbTsg
SmFuDQo+IEtpc3prYQ0KPiA+ID4gPiA+ID4gPiA+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsg
Q2FybCB2YW4gU2NoYWlrDQo+ID4gPiA+ID4gPGN2YW5zY2hhQHF0aS5xdWFsY29tbS5jb20+Ow0K
PiA+ID4gPiA+ID4gPiA+IHByYXRpa3BAcXVpY2luYy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaQ0K
PiA8dmF0c2FAY29kZWF1cm9yYS5vcmc+Ow0KPiA+ID4gPiA+IEplYW4tDQo+ID4gPiA+ID4gPiA+
ID4gUGhpbGlwcGUgQnJ1Y2tlciA8amVhbi1waGlsaXBwZUBsaW5hcm8ub3JnPjsgTWF0aGlldSBQ
b2lyaWVyDQo+ID4gPiA+ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xl
a3NhbmRyIFR5c2hjaGVua28NCj4gPiA+ID4gPiA+ID4gPiA8T2xla3NhbmRyX1R5c2hjaGVua29A
ZXBhbS5jb20+OyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4gPiA+ID4gPiA+ID4gPEJlcnRyYW5kLk1h
cnF1aXNAYXJtLmNvbT47IEFydGVtIE15Z2FpZXYNCj4gPiA+IDxBcnRlbV9NeWdhaWV2QGVwYW0u
Y29tPjsNCj4gPiA+ID4gPiBKdWxpZW4NCj4gPiA+ID4gPiA+ID4gPiBHcmFsbCA8anVsaWVuQHhl
bi5vcmc+OyBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+Ow0KPiBQYXVsDQo+ID4gPiA+
ID4gRHVycmFudA0KPiA+ID4gPiA+ID4gPiA+IDxwYXVsQHhlbi5vcmc+OyBYZW4gRGV2ZWwgPHhl
bi1kZXZlbEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFi
bGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIGZvciBWaXJ0SU8NCj4gYmFja2VuZHMNCj4gPiA+
ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+IE9uIFR1ZSwgQXVnIDE3LCAyMDIxIGF0IDA4OjM5
OjA5QU0gKzAwMDAsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiA+ID4gPiA+ID4gSGkgQWthc2hp
LA0KPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gLS0tLS1PcmlnaW5hbCBN
ZXNzYWdlLS0tLS0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gRnJvbTogQUtBU0hJIFRha2FoaXJvIDx0
YWthaGlyby5ha2FzaGlAbGluYXJvLm9yZz4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gU2VudDogMjAy
MeW5tDjmnIgxN+aXpSAxNjowOA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBUbzogV2VpIENoZW4gPFdl
aS5DaGVuQGFybS5jb20+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IENjOiBPbGVrc2FuZHIgVHlzaGNo
ZW5rbyA8b2xla3N0eXNoQGdtYWlsLmNvbT47IFN0ZWZhbm8NCj4gPiA+ID4gPiBTdGFiZWxsaW5p
DQo+ID4gPiA+ID4gPiA+ID4gPiA+IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5u
Pz9lDQo+ID4gPiA8YWxleC5iZW5uZWVAbGluYXJvLm9yZz47DQo+ID4gPiA+ID4gPiA+ID4gU3Ry
YXRvcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBNYWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLWxp
c3RzLmxpbmFyby5vcmc+OyB2aXJ0aW8tDQo+ID4gPiA+ID4gPiA+ID4gZGV2QGxpc3RzLm9hc2lz
LQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBvcGVuLm9yZzsgQXJuZCBCZXJnbWFubiA8YXJuZC5iZXJn
bWFubkBsaW5hcm8ub3JnPjsNCj4gVmlyZXNoDQo+ID4gPiBLdW1hcg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA8dmlyZXNoLmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFA
cmVkaGF0LmNvbTsNCj4gSmFuDQo+ID4gPiBLaXN6a2ENCj4gPiA+ID4gPiA+ID4gPiA+ID4gPGph
bi5raXN6a2FAc2llbWVucy5jb20+OyBDYXJsIHZhbiBTY2hhaWsNCj4gPiA+ID4gPiA8Y3ZhbnNj
aGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4gPiA+IHByYXRpa3BAcXVpY2lu
Yy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaQ0KPiA+ID4gPHZhdHNhQGNvZGVhdXJvcmEub3JnPjsN
Cj4gPiA+ID4gPiBKZWFuLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBQaGlsaXBwZSBCcnVja2VyIDxq
ZWFuLXBoaWxpcHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1DQo+IFBvaXJpZXINCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xla3NhbmRyIFR5c2hjaGVu
a28NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPE9sZWtzYW5kcl9UeXNoY2hlbmtvQGVwYW0uY29tPjsg
QmVydHJhbmQgTWFycXVpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA8QmVydHJhbmQuTWFycXVpc0Bh
cm0uY29tPjsgQXJ0ZW0gTXlnYWlldg0KPiA+ID4gPiA+IDxBcnRlbV9NeWdhaWV2QGVwYW0uY29t
PjsNCj4gPiA+ID4gPiA+ID4gPiBKdWxpZW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gR3JhbGwgPGp1
bGllbkB4ZW4ub3JnPjsgSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPjsNCj4gPiA+IFBh
dWwNCj4gPiA+ID4gPiBEdXJyYW50DQo+ID4gPiA+ID4gPiA+ID4gPiA+IDxwYXVsQHhlbi5vcmc+
OyBYZW4gRGV2ZWwgPHhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiBTdWJqZWN0OiBSZTogRW5hYmxpbmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lzbSBmb3IgVmlydElP
DQo+ID4gPiBiYWNrZW5kcw0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiBIaSBXZWksIE9sZWtzYW5kciwNCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gT24gTW9uLCBBdWcgMTYsIDIwMjEgYXQgMTA6MDQ6MDNBTSArMDAwMCwgV2VpIENoZW4g
d3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSGkgQWxsLA0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVGhhbmtzIGZvciBTdGVmYW5vIHRvIGxpbmsgbXkg
a3ZtdG9vbCBmb3IgWGVuIHByb3Bvc2FsDQo+ID4gPiBoZXJlLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IFRoaXMgcHJvcG9zYWwgaXMgc3RpbGwgZGlzY3Vzc2luZyBpbiBYZW4gYW5kIEtWTQ0KPiA+
ID4gY29tbXVuaXRpZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVGhlIG1haW4gd29yayBpcyB0
byBkZWNvdXBsZSB0aGUga3ZtdG9vbCBmcm9tIEtWTSBhbmQNCj4gbWFrZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IG90aGVyIGh5cGVydmlzb3JzIGNhbiByZXVzZSB0aGUgdmlydHVhbCBkZXZpY2UN
Cj4gPiA+IGltcGxlbWVudGF0aW9ucy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IEluIHRoaXMgY2FzZSwgd2UgbmVlZCB0byBpbnRyb2R1Y2UgYW4gaW50ZXJt
ZWRpYXRlDQo+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbGF5ZXIgZm9y
IFZNTSBhYnN0cmFjdGlvbiwgV2hpY2ggaXMsIEkgdGhpbmsgaXQncyB2ZXJ5DQo+ID4gPiBjbG9z
ZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHN0cmF0b3MnIHZpcnRpbyBoeXBlcnZpc29yIGFn
bm9zdGljaXNtIHdvcmsuDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ICMgTXkgcHJvcG9zYWxbMV0gY29tZXMgZnJvbSBteSBvd24gaWRlYSBhbmQgZG9lc24ndA0KPiBh
bHdheXMNCj4gPiA+ID4gPiByZXByZXNlbnQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gIyBMaW5hcm8n
cyB2aWV3IG9uIHRoaXMgc3ViamVjdCBub3IgcmVmbGVjdCBBbGV4J3MNCj4gY29uY2VybnMuDQo+
ID4gPiA+ID4gPiA+ID4gTmV2ZXJ0aGVsZXNzLA0KPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiBZb3VyIGlkZWEgYW5kIG15IHByb3Bvc2FsIHNlZW0gdG8gc2hhcmUgdGhl
IHNhbWUNCj4gYmFja2dyb3VuZC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gQm90aCBoYXZlIHRoZSBz
aW1pbGFyIGdvYWwgYW5kIGN1cnJlbnRseSBzdGFydCB3aXRoLCBhdA0KPiA+ID4gZmlyc3QsDQo+
ID4gPiA+ID4gWGVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCBhcmUgYmFzZWQgb24ga3ZtLXRv
b2wuIChBY3R1YWxseSwgbXkgd29yayBpcyBkZXJpdmVkDQo+ID4gPiBmcm9tDQo+ID4gPiA+ID4g
PiA+ID4gPiA+IEVQQU0ncyB2aXJ0aW8tZGlzaywgd2hpY2ggaXMgYWxzbyBiYXNlZCBvbiBrdm0t
dG9vbC4pDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IEluIHBhcnRp
Y3VsYXIsIHRoZSBhYnN0cmFjdGlvbiBvZiBoeXBlcnZpc29yIGludGVyZmFjZXMNCj4gaGFzDQo+
ID4gPiBhDQo+ID4gPiA+ID4gc2FtZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBzZXQgb2YgaW50ZXJm
YWNlcyAoZm9yIHlvdXIgInN0cnVjdCB2bW1faW1wbCIgYW5kIG15DQo+ICJSUEMNCj4gPiA+ID4g
PiBpbnRlcmZhY2VzIikuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IFRoaXMgaXMgbm90IGNvLWluY2lk
ZW50IGFzIHdlIGJvdGggc2hhcmUgdGhlIHNhbWUgb3JpZ2luDQo+IGFzDQo+ID4gPiBJDQo+ID4g
PiA+ID4gc2FpZA0KPiA+ID4gPiA+ID4gPiA+IGFib3ZlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBB
bmQgc28gd2Ugd2lsbCBhbHNvIHNoYXJlIHRoZSBzYW1lIGlzc3Vlcy4gT25lIG9mIHRoZW0NCj4g
aXMgYQ0KPiA+ID4gd2F5DQo+ID4gPiA+ID4gb2YNCj4gPiA+ID4gPiA+ID4gPiA+ID4gInNoYXJp
bmcvbWFwcGluZyBGRSdzIG1lbW9yeSIuIFRoZXJlIGlzIHNvbWUgdHJhZGUtb2ZmDQo+ID4gPiBi
ZXR3ZWVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBwb3J0YWJpbGl0eSBhbmQgdGhlIHBlcmZv
cm1hbmNlIGltcGFjdC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gU28gd2UgY2FuIGRpc2N1c3MgdGhl
IHRvcGljIGhlcmUgaW4gdGhpcyBNTCwgdG9vLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiAoU2VlIEFs
ZXgncyBvcmlnaW5hbCBlbWFpbCwgdG9vKS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4g
PiA+ID4gPiA+IFllcywgSSBhZ3JlZS4NCj4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+
ID4gPiA+IE9uIHRoZSBvdGhlciBoYW5kLCBteSBhcHByb2FjaCBhaW1zIHRvIGNyZWF0ZSBhICJz
aW5nbGUtDQo+ID4gPiBiaW5hcnkiDQo+ID4gPiA+ID4gPiA+ID4gc29sdXRpb24NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gaW4gd2hpY2ggdGhlIHNhbWUgYmluYXJ5IG9mIEJFIHZtIGNvdWxkIHJ1biBv
biBhbnkNCj4gPiA+IGh5cGVydmlzb3JzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBTb21laG93IHNp
bWlsYXIgdG8geW91ciAicHJvcG9zYWwtIzIiIGluIFsyXSwgYnV0IGluIG15DQo+ID4gPiBzb2x1
dGlvbiwNCj4gPiA+ID4gPiBhbGwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIGh5cGVydmlzb3It
c3BlY2lmaWMgY29kZSB3b3VsZCBiZSBwdXQgaW50byBhbm90aGVyDQo+ID4gPiBlbnRpdHkNCj4g
PiA+ID4gPiAoVk0pLA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBuYW1lZCAidmlydGlvLXByb3h5IiBh
bmQgdGhlIGFic3RyYWN0ZWQgb3BlcmF0aW9ucyBhcmUNCj4gPiA+IHNlcnZlZA0KPiA+ID4gPiA+
IHZpYSBSUEMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IChJbiB0aGlzIHNlbnNlLCBCRSBpcyBoeXBl
cnZpc29yLWFnbm9zdGljIGJ1dCBtaWdodCBoYXZlDQo+IE9TDQo+ID4gPiA+ID4gPiA+ID4gZGVw
ZW5kZW5jeS4pDQo+ID4gPiA+ID4gPiA+ID4gPiA+IEJ1dCBJIGtub3cgdGhhdCB3ZSBuZWVkIGRp
c2N1c3MgaWYgdGhpcyBpcyBhIHJlcXVpcmVtZW50DQo+ID4gPiBldmVuDQo+ID4gPiA+ID4gPiA+
ID4gPiA+IGluIFN0cmF0b3MgcHJvamVjdCBvciBub3QuIChNYXliZSBub3QpDQo+ID4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gU29ycnksIEkg
aGF2ZW4ndCBoYWQgdGltZSB0byBmaW5pc2ggcmVhZGluZyB5b3VyIHZpcnRpby0NCj4gcHJveHkN
Cj4gPiA+ID4gPiBjb21wbGV0ZWx5DQo+ID4gPiA+ID4gPiA+ID4gPiAoSSB3aWxsIGRvIGl0IEFT
QVApLiBCdXQgZnJvbSB5b3VyIGRlc2NyaXB0aW9uLCBpdCBzZWVtcw0KPiB3ZQ0KPiA+ID4gbmVl
ZCBhDQo+ID4gPiA+ID4gPiA+ID4gPiAzcmQgVk0gYmV0d2VlbiBGRSBhbmQgQkU/IE15IGNvbmNl
cm4gaXMgdGhhdCwgaWYgbXkNCj4gYXNzdW1wdGlvbg0KPiA+ID4gaXMNCj4gPiA+ID4gPiByaWdo
dCwNCj4gPiA+ID4gPiA+ID4gPiA+IHdpbGwgaXQgaW5jcmVhc2UgdGhlIGxhdGVuY3kgaW4gZGF0
YSB0cmFuc3BvcnQgcGF0aD8gRXZlbg0KPiBpZg0KPiA+ID4gd2UncmUNCj4gPiA+ID4gPiA+ID4g
PiA+IHVzaW5nIHNvbWUgbGlnaHR3ZWlnaHQgZ3Vlc3QgbGlrZSBSVE9TIG9yIFVuaWtlcm5lbCwN
Cj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+IFllcywgeW91J3JlIHJpZ2h0LiBCdXQg
SSdtIGFmcmFpZCB0aGF0IGl0IGlzIGEgbWF0dGVyIG9mDQo+IGRlZ3JlZS4NCj4gPiA+ID4gPiA+
ID4gPiBBcyBmYXIgYXMgd2UgZXhlY3V0ZSAnbWFwcGluZycgb3BlcmF0aW9ucyBhdCBldmVyeSBm
ZXRjaCBvZg0KPiA+ID4gcGF5bG9hZCwNCj4gPiA+ID4gPiA+ID4gPiB3ZSB3aWxsIHNlZSBsYXRl
bmN5IGlzc3VlIChldmVuIGluIHlvdXIgY2FzZSkgYW5kIGlmIHdlIGhhdmUNCj4gPiA+IHNvbWUN
Cj4gPiA+ID4gPiBzb2x1dGlvbg0KPiA+ID4gPiA+ID4gPiA+IGZvciBpdCwgd2Ugd29uJ3Qgc2Vl
IGl0IG5laXRoZXIgaW4gbXkgcHJvcG9zYWwgOikNCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gPiBPbGVrc2FuZHIgaGFzIHNlbnQgYSBwcm9wb3NhbCB0byBYZW4g
bWFpbGluZyBsaXN0IHRvIHJlZHVjZQ0KPiB0aGlzDQo+ID4gPiBraW5kDQo+ID4gPiA+ID4gPiA+
IG9mICJtYXBwaW5nL3VubWFwcGluZyIgb3BlcmF0aW9ucy4gU28gdGhlIGxhdGVuY3kgY2F1c2Vk
IGJ5DQo+IHRoaXMNCj4gPiA+ID4gPiBiZWhhdmlvcg0KPiA+ID4gPiA+ID4gPiBvbiBYZW4gbWF5
IGV2ZW50dWFsbHkgYmUgZWxpbWluYXRlZCwgYW5kIExpbnV4LUtWTSBkb2Vzbid0DQo+IGhhdmUN
Cj4gPiA+IHRoYXQNCj4gPiA+ID4gPiBwcm9ibGVtLg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
IE9idmlvdXNseSwgSSBoYXZlIG5vdCB5ZXQgY2F1Z2h0IHVwIHRoZXJlIGluIHRoZSBkaXNjdXNz
aW9uLg0KPiA+ID4gPiA+ID4gV2hpY2ggcGF0Y2ggc3BlY2lmaWNhbGx5Pw0KPiA+ID4gPiA+DQo+
ID4gPiA+ID4gQ2FuIHlvdSBnaXZlIG1lIHRoZSBsaW5rIHRvIHRoZSBkaXNjdXNzaW9uIG9yIHBh
dGNoLCBwbGVhc2U/DQo+ID4gPiA+ID4NCj4gPiA+ID4NCj4gPiA+ID4gSXQncyBhIFJGQyBkaXNj
dXNzaW9uLiBXZSBoYXZlIHRlc3RlZCB0aGlzIFJGQyBwYXRjaCBpbnRlcm5hbGx5Lg0KPiA+ID4g
PiBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hpdmVzL2h0bWwveGVuLWRldmVsLzIw
MjEtDQo+ID4gPiAwNy9tc2cwMTUzMi5odG1sDQo+ID4gPg0KPiA+ID4gSSdtIGFmcmFpZCB0aGF0
IEkgbWlzcyBzb21ldGhpbmcgaGVyZSwgYnV0IEkgZG9uJ3Qga25vdw0KPiA+ID4gd2h5IHRoaXMg
cHJvcG9zZWQgQVBJIHdpbGwgbGVhZCB0byBlbGltaW5hdGluZyAnbW1hcCcgaW4gYWNjZXNzaW5n
DQo+ID4gPiB0aGUgcXVldWVkIHBheWxvYWQgYXQgZXZlcnkgcmVxdWVzdD8NCj4gPiA+DQo+ID4N
Cj4gPiBUaGlzIEFQSSBnaXZlIFhlbiBkZXZpY2UgbW9kZWwgKFFFTVUgb3Iga3ZtdG9vbCkgdGhl
IGFiaWxpdHkgdG8gbWFwDQo+ID4gd2hvbGUgZ3Vlc3QgUkFNIGluIGRldmljZSBtb2RlbCdzIGFk
ZHJlc3Mgc3BhY2UuIEluIHRoaXMgY2FzZSwgZGV2aWNlDQo+ID4gbW9kZWwgZG9lc24ndCBuZWVk
IGR5bmFtaWMgaHlwZXJjYWxsIHRvIG1hcC91bm1hcCBwYXlsb2FkIG1lbW9yeS4NCj4gPiBJdCBj
YW4gdXNlIGEgZmxhdCBvZmZzZXQgdG8gYWNjZXNzIHBheWxvYWQgbWVtb3J5IGluIGl0cyBhZGRy
ZXNzDQo+ID4gc3BhY2UgZGlyZWN0bHkuIEp1c3QgTGlrZSBLVk0gZGV2aWNlIG1vZGVsIGRvZXMg
bm93Lg0KPiANCj4gVGhhbmsgeW91LiBRdWlja2x5LCBsZXQgbWUgbWFrZSBzdXJlIG9uZSB0aGlu
ZzoNCj4gVGhpcyBBUEkgaXRzZWxmIGRvZXNuJ3QgZG8gYW55IG1hcHBpbmcgb3BlcmF0aW9ucywg
cmlnaHQ/DQo+IFNvIEkgc3VwcG9zZSB0aGF0IHZpcnRpbyBCRSBndWVzdCBpcyByZXNwb25zaWJs
ZSB0bw0KPiAxKSBmZXRjaCB0aGUgaW5mb3JtYXRpb24gYWJvdXQgYWxsIHRoZSBtZW1vcnkgcmVn
aW9ucyBpbiBGRSwNCj4gMikgY2FsbCB0aGlzIEFQSSB0byBhbGxvY2F0ZSBhIGJpZyBjaHVuayBv
ZiB1bnVzZWQgc3BhY2UgaW4gQkUsDQo+IDMpIGNyZWF0ZSBncmFudC9mb3JlaWduIG1hcHBpbmdz
IGZvciBGRSBvbnRvIHRoaXMgcmVnaW9uKFMpDQo+IGluIHRoZSBpbml0aWFsaXphdGlvbi9jb25m
aWd1cmF0aW9uIG9mIGVtdWxhdGVkIHZpcnRpbyBkZXZpY2VzLg0KPiANCj4gSXMgdGhpcyB0aGUg
d2F5IHRoaXMgQVBJIGlzIGV4cGVjdGVkIHRvIGJlIHVzZWQ/DQo+IERvZXMgWGVuIGFscmVhZHkg
aGFzIGFuIGludGVyZmFjZSBmb3IgKDEpPw0KPiANCg0KVGhleSBhcmUgZGlzY3Vzc2luZyBpbiB0
aGF0IHRocmVhZCB0byBmaW5kIGEgcHJvcGVyIHdheSB0byBkbyBpdC4NCkJlY2F1c2UgdGhpcyBB
UEkgaXMgY29tbW9uLCBib3RoIHg4NiBhbmQgQXJtIHNob3VsZCBiZSBjb25zaWRlcmVkLg0KDQo+
IC1UYWthaGlybyBBa2FzaGkNCj4gDQo+ID4gQmVmb3JlIHRoaXMgQVBJLCBXaGVuIGRldmljZSBt
b2RlbCB0byBtYXAgd2hvbGUgZ3Vlc3QgbWVtb3J5LCB3aWxsDQo+ID4gc2V2ZXJlbHkgY29uc3Vt
ZSB0aGUgcGh5c2ljYWwgcGFnZXMgb2YgRG9tLTAvRG9tLUQuDQo+ID4NCj4gPiA+IC1UYWthaGly
byBBa2FzaGkNCj4gPiA+DQo+ID4gPg0KPiA+ID4gPiA+IFRoYW5rcywNCj4gPiA+ID4gPiAtVGFr
YWhpcm8gQWthc2hpDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IC1UYWthaGlybyBBa2FzaGkNCj4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IFNwZWNpZmljYWxseSBzcGVha2luZyBhYm91
dCBrdm0tdG9vbCwgSSBoYXZlIGEgY29uY2Vybg0KPiBhYm91dA0KPiA+ID4gaXRzDQo+ID4gPiA+
ID4gPiA+ID4gPiA+IGxpY2Vuc2UgdGVybTsgVGFyZ2V0aW5nIGRpZmZlcmVudCBoeXBlcnZpc29y
cyBhbmQNCj4gZGlmZmVyZW50DQo+ID4gPiBPU3MNCj4gPiA+ID4gPiA+ID4gPiA+ID4gKHdoaWNo
IEkgYXNzdW1lIGluY2x1ZGVzIFJUT1MncyksIHRoZSByZXN1bHRhbnQgbGlicmFyeQ0KPiA+ID4g
c2hvdWxkDQo+ID4gPiA+ID4gYmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gbGljZW5zZSBwZXJtaXNz
aXZlIGFuZCBHUEwgZm9yIGt2bS10b29sIG1pZ2h0IGJlIGFuDQo+IGlzc3VlLg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiBBbnkgdGhvdWdodHM/DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gWWVzLiBJZiB1c2VyIHdhbnQgdG8gaW1wbGVtZW50
IGEgRnJlZUJTRCBkZXZpY2UgbW9kZWwsIGJ1dA0KPiB0aGUNCj4gPiA+ID4gPiB2aXJ0aW8NCj4g
PiA+ID4gPiA+ID4gPiA+IGxpYnJhcnkgaXMgR1BMLiBUaGVuIEdQTCB3b3VsZCBiZSBhIHByb2Js
ZW0uIElmIHdlIGhhdmUNCj4gPiA+IGFub3RoZXINCj4gPiA+ID4gPiBnb29kDQo+ID4gPiA+ID4g
PiA+ID4gPiBjYW5kaWRhdGUsIEkgYW0gb3BlbiB0byBpdC4NCj4gPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiA+IEkgaGF2ZSBzb21lIGNhbmRpZGF0ZXMsIHBhcnRpY3VsYXJseSBmb3IgdnEv
dnJpbmcsIGluIG15DQo+IG1pbmQ6DQo+ID4gPiA+ID4gPiA+ID4gKiBPcGVuLUFNUCwgb3INCj4g
PiA+ID4gPiA+ID4gPiAqIGNvcnJlc3BvbmRpbmcgRnJlZS1CU0QgY29kZQ0KPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+IEludGVyZXN0aW5nLCBJIHdpbGwgbG9v
ayBpbnRvIHRoZW0gOiApDQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+IENoZWVycywNCj4g
PiA+ID4gPiA+ID4gV2VpIENoZW4NCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiAtVGFr
YWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+IFsxXSBodHRwczovL29wLWxpc3RzLmxp
bmFyby5vcmcvcGlwZXJtYWlsL3N0cmF0b3MtDQo+IGRldi8yMDIxLQ0KPiA+ID4gPiA+ID4gPiA+
ID4gPiBBdWd1c3QvMDAwNTQ4Lmh0bWwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gWzJdIGh0dHBzOi8v
bWFyYy5pbmZvLz9sPXhlbi1kZXZlbCZtPTE2MjM3Mzc1NDcwNTIzMyZ3PTINCj4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
RnJvbTogT2xla3NhbmRyIFR5c2hjaGVua28gPG9sZWtzdHlzaEBnbWFpbC5jb20+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBTZW50OiAyMDIx5bm0OOaciDE05pelIDIzOjM4DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBUbzogQUtBU0hJIFRha2FoaXJvIDx0YWthaGlyby5ha2FzaGlAbGluYXJv
Lm9yZz47DQo+ID4gPiBTdGVmYW5vDQo+ID4gPiA+ID4gPiA+ID4gU3RhYmVsbGluaQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IENjOiBBbGV4IEJlbm4/P2UgPGFsZXguYmVubmVlQGxpbmFyby5vcmc+OyBTdHJhdG9z
DQo+ID4gPiBNYWlsaW5nDQo+ID4gPiA+ID4gTGlzdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA8c3Ry
YXRvcy1kZXZAb3AtbGlzdHMubGluYXJvLm9yZz47IHZpcnRpby0NCj4gZGV2QGxpc3RzLm9hc2lz
LQ0KPiA+ID4gPiA+IG9wZW4ub3JnOw0KPiA+ID4gPiA+ID4gPiA+IEFybmQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gQmVyZ21hbm4gPGFybmQuYmVyZ21hbm5AbGluYXJvLm9yZz47IFZpcmVzaCBLdW1h
cg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA8dmlyZXNoLmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5v
IFN0YWJlbGxpbmkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxp
bnguY29tPjsgc3RlZmFuaGFAcmVkaGF0LmNvbTsNCj4gSmFuDQo+ID4gPiBLaXN6a2ENCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPGphbi5raXN6a2FAc2llbWVucy5jb20+OyBDYXJsIHZhbiBTY2hhaWsN
Cj4gPiA+ID4gPiA8Y3ZhbnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4g
PiA+IHByYXRpa3BAcXVpY2luYy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaQ0KPiA+ID4gPHZhdHNh
QGNvZGVhdXJvcmEub3JnPjsNCj4gPiA+ID4gPiBKZWFuLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBQ
aGlsaXBwZSBCcnVja2VyIDxqZWFuLXBoaWxpcHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1DQo+IFBv
aXJpZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsg
V2VpIENoZW4NCj4gPFdlaS5DaGVuQGFybS5jb20+Ow0KPiA+ID4gPiA+IE9sZWtzYW5kcg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiBUeXNoY2hlbmtvIDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNv
bT47IEJlcnRyYW5kDQo+IE1hcnF1aXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPEJlcnRyYW5kLk1h
cnF1aXNAYXJtLmNvbT47IEFydGVtIE15Z2FpZXYNCj4gPiA+ID4gPiA8QXJ0ZW1fTXlnYWlldkBl
cGFtLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4gSnVsaWVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IEdy
YWxsIDxqdWxpZW5AeGVuLm9yZz47IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT47DQo+
ID4gPiBQYXVsDQo+ID4gPiA+ID4gRHVycmFudA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA8cGF1bEB4
ZW4ub3JnPjsgWGVuIERldmVsIDx4ZW4tZGV2ZWxAbGlzdHMueGVuLm9yZz4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFibGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNt
IGZvcg0KPiBWaXJ0SU8NCj4gPiA+ID4gPiBiYWNrZW5kcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEhlbGxvLCBhbGwuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gUGxlYXNlIHNlZSBzb21lIGNvbW1lbnRz
IGJlbG93LiBBbmQgc29ycnkgZm9yIHRoZQ0KPiA+ID4gcG9zc2libGUNCj4gPiA+ID4gPiBmb3Jt
YXQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaXNzdWVzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT24gV2VkLCBBdWcgMTEsIDIwMjEgYXQgOToyNyBB
TSBBS0FTSEkgVGFrYWhpcm8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPG1haWx0bzp0YWthaGlyby5h
a2FzaGlAbGluYXJvLm9yZz4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uIFdl
ZCwgQXVnIDA0LCAyMDIxIGF0IDEyOjIwOjAxUE0gLTA3MDAsIFN0ZWZhbm8NCj4gPiA+ID4gPiBT
dGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA+ID4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gQ0NpbmcgcGVvcGxlIHdvcmtpbmcgb24gWGVuK1ZpcnRJTyBhbmQgSU9SRVFzLg0KPiBO
b3QNCj4gPiA+ID4gPiB0cmltbWluZw0KPiA+ID4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiBvcmlnaW5hbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGVtYWlsIHRvIGxl
dCB0aGVtIHJlYWQgdGhlIGZ1bGwgY29udGV4dC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE15IGNvbW1lbnRzIGJlbG93IGFyZSByZWxh
dGVkIHRvIGEgcG90ZW50aWFsIFhlbg0KPiA+ID4gPiA+ID4gPiA+IGltcGxlbWVudGF0aW9uLA0K
PiA+ID4gPiA+ID4gPiA+ID4gPiBub3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBiZWNh
dXNlIGl0IGlzIHRoZSBvbmx5IGltcGxlbWVudGF0aW9uIHRoYXQNCj4gbWF0dGVycywNCj4gPiA+
IGJ1dA0KPiA+ID4gPiA+ID4gPiA+IGJlY2F1c2UgaXQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaXMN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgb25lIEkga25vdyBiZXN0Lg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBQbGVhc2Ugbm90
ZSB0aGF0IG15IHByb3Bvc2FsIChhbmQgaGVuY2UgdGhlDQo+IHdvcmtpbmcNCj4gPiA+ID4gPiBw
cm90b3R5cGUpWzFdDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGlzIGJhc2VkIG9uIFhlbidz
IHZpcnRpbyBpbXBsZW1lbnRhdGlvbiAoaS5lLg0KPiBJT1JFUSkNCj4gPiA+IGFuZA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiBwYXJ0aWN1bGFybHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gRVBB
TSdzIHZpcnRpby1kaXNrIGFwcGxpY2F0aW9uIChiYWNrZW5kIHNlcnZlcikuDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IEl0IGhhcyBiZWVuLCBJIGJlbGlldmUsIHdlbGwgZ2VuZXJhbGl6ZWQg
YnV0IGlzDQo+IHN0aWxsDQo+ID4gPiBhDQo+ID4gPiA+ID4gYml0DQo+ID4gPiA+ID4gPiA+ID4g
Ymlhc2VkDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvd2FyZCB0aGlzIG9yaWdpbmFsIGRl
c2lnbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gU28gSSBob3BlIHlvdSBsaWtlIG15IGFwcHJvYWNoIDopDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFsxXSBodHRwczovL29wLWxpc3RzLmxp
bmFyby5vcmcvcGlwZXJtYWlsL3N0cmF0b3MtDQo+ID4gPiA+ID4gZGV2LzIwMjEtDQo+ID4gPiA+
ID4gPiA+ID4gPiA+IEF1Z3VzdC8wMDA1NDYuaHRtbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBMZXQgbWUgdGFrZSB0aGlzIG9wcG9ydHVuaXR5
IHRvIGV4cGxhaW4gYSBiaXQgbW9yZQ0KPiA+ID4gYWJvdXQNCj4gPiA+ID4gPiBteQ0KPiA+ID4g
PiA+ID4gPiA+IGFwcHJvYWNoDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGJlbG93Lg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEFsc28sIHBsZWFz
ZSBzZWUgdGhpcyByZWxldmFudCBlbWFpbCB0aHJlYWQ6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gaHR0cHM6Ly9tYXJjLmluZm8vP2w9eGVuLQ0KPiBkZXZlbCZtPTE2MjM3Mzc1NDcwNTIz
MyZ3PTINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT24gV2VkLCA0IEF1ZyAyMDIxLCBB
bGV4IEJlbm7DqWUgd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBIaSwNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBPbmUgb2YgdGhlIGdvYWxzIG9mIFByb2plY3QgU3RyYXRvcyBpcyB0bw0KPiBlbmFibGUNCj4g
PiA+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFnbm9zdGljDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kcyBzbyB3ZSBjYW4gZW5hYmxlIGFzIG11Y2gg
cmUtdXNlIG9mDQo+IGNvZGUNCj4gPiA+IGFzDQo+ID4gPiA+ID4gcG9zc2libGUNCj4gPiA+ID4g
PiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYXZvaWQNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IHJlcGVhdGluZyBvdXJzZWx2ZXMuIFRoaXMgaXMgdGhlIGZsaXAgc2lkZSBv
Zg0KPiB0aGUNCj4gPiA+ID4gPiBmcm9udCBlbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gd2hlcmUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG11bHRpcGxlIGZyb250LWVuZCBpbXBsZW1l
bnRhdGlvbnMgYXJlDQo+IHJlcXVpcmVkIC0NCj4gPiA+IG9uZQ0KPiA+ID4gPiA+IHBlciBPUywN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gYXNzdW1pbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IHlvdSBkb24ndCBqdXN0IHdhbnQgTGludXggZ3Vlc3RzLiBUaGUNCj4gcmVzdWx0YW50DQo+
ID4gPiBndWVzdHMNCj4gPiA+ID4gPiBhcmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdHJpdmlhbGx5
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBtb3ZhYmxlIGJldHdlZW4gaHlwZXJ2aXNv
cnMgbW9kdWxvIGFueQ0KPiBhYnN0cmFjdGVkDQo+ID4gPiA+ID4gcGFyYXZpcnQNCj4gPiA+ID4g
PiA+ID4gPiB0eXBlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2VzLg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IEluIG15IG9yaWdpbmFsIHRodW1iIG5haWwgc2tldGNoIG9mIGEgc29sdXRpb24NCj4gSQ0K
PiA+ID4gPiA+IGVudmlzaW9uZWQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdmhvc3QtdXNlcg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZGFlbW9ucyBydW5uaW5nIGluIGEgYnJvYWRseSBQ
T1NJWCBsaWtlDQo+ID4gPiBlbnZpcm9ubWVudC4NCj4gPiA+ID4gPiBUaGUNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gaW50ZXJmYWNlIHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUg
ZGFlbW9uIGlzIGZhaXJseSBzaW1wbGUgcmVxdWlyaW5nIG9ubHkNCj4gc29tZQ0KPiA+ID4gbWFw
cGVkDQo+ID4gPiA+ID4gPiA+ID4gbWVtb3J5DQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCBzb21l
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzb3J0IG9mIHNpZ25hbGxpbmcgZm9yIGV2
ZW50cyAob24gTGludXggdGhpcw0KPiBpcw0KPiA+ID4gPiA+IGV2ZW50ZmQpLg0KPiA+ID4gPiA+
ID4gPiA+IFRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBpZGVhIHdhcyBhDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBzdHViIGJpbmFyeSB3b3VsZCBiZSByZXNwb25zaWJsZSBmb3IgYW55
DQo+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gc3BlY2lmaWMNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gc2V0dXAgYW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGVuIGxhdW5jaCBh
IGNvbW1vbiBiaW5hcnkgdG8gZGVhbCB3aXRoIHRoZQ0KPiA+ID4gYWN0dWFsDQo+ID4gPiA+ID4g
PiA+ID4gdmlydHF1ZXVlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHJlcXVlc3RzDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiB0aGVtc2VsdmVzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFNpbmNlIHRoYXQgb3JpZ2luYWwg
c2tldGNoIHdlJ3ZlIHNlZW4gYW4NCj4gZXhwYW5zaW9uDQo+ID4gPiBpbg0KPiA+ID4gPiA+IHRo
ZQ0KPiA+ID4gPiA+ID4gPiA+IHNvcnQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gb2Ygd2F5cw0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYmFja2VuZHMgY291bGQgYmUgY3JlYXRlZC4gVGhl
cmUgaXMgaW50ZXJlc3QNCj4gaW4NCj4gPiA+ID4gPiA+ID4gPiBlbmNhcHN1bGF0aW5nDQo+ID4g
PiA+ID4gPiA+ID4gPiA+IGJhY2tlbmRzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBp
biBSVE9TZXMgb3IgdW5pa2VybmVscyBmb3Igc29sdXRpb25zIGxpa2UNCj4gU0NNSS4NCj4gPiA+
IFRoZXJlDQo+ID4gPiA+ID4gPiA+ID4gaW50ZXJlc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaW4g
UnVzdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaGFzIHByb21wdGVkIGlkZWFzIG9m
IHVzaW5nIHRoZSB0cmFpdA0KPiBpbnRlcmZhY2UgdG8NCj4gPiA+ID4gPiBhYnN0cmFjdA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiBkaWZmZXJlbmNlcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gYXdheSBhcyB3ZWxsIGFzIHRoZSBpZGVhIG9mIGJhcmUtbWV0YWwgUnVzdA0KPiA+ID4gYmFj
a2VuZHMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gV2UgaGF2ZSBhIGNhcmQgKFNUUi0xMikgY2FsbGVkICJIeXBlcmNhbGwNCj4g
PiA+ID4gPiBTdGFuZGFyZGlzYXRpb24iDQo+ID4gPiA+ID4gPiA+ID4gd2hpY2gNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNhbGxzIGZvciBhIGRlc2NyaXB0aW9uIG9mIHRoZSBBUElz
IG5lZWRlZA0KPiBmcm9tDQo+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiBoeXBlcnZpc29yDQo+
ID4gPiA+ID4gPiA+ID4gPiA+IHNpZGUgdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IHN1cHBvcnQgVmlydElPIGd1ZXN0cyBhbmQgdGhlaXIgYmFja2VuZHMuDQo+IEhvd2V2ZXINCj4g
PiA+IHdlDQo+ID4gPiA+ID4gYXJlDQo+ID4gPiA+ID4gPiA+ID4gc29tZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiB3YXkgb2ZmDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBmcm9tIHRoYXQg
YXQgdGhlIG1vbWVudCBhcyBJIHRoaW5rIHdlIG5lZWQgdG8NCj4gYXQNCj4gPiA+IGxlYXN0DQo+
ID4gPiA+ID4gPiA+ID4gPiA+IGRlbW9uc3RyYXRlIG9uZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gcG9ydGFibGUgYmFja2VuZCBiZWZvcmUgd2Ugc3RhcnQgY29kaWZ5aW5nDQo+ID4g
PiA+ID4gcmVxdWlyZW1lbnRzLiBUbw0KPiA+ID4gPiA+ID4gPiA+IHRoYXQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gZW5kIEkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHdhbnQgdG8gdGhp
bmsgYWJvdXQgd2hhdCB3ZSBuZWVkIGZvciBhIGJhY2tlbmQNCj4gdG8NCj4gPiA+ID4gPiBmdW5j
dGlvbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBDb25maWd1cmF0aW9uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA9
PT09PT09PT09PT09DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gSW4gdGhlIHR5cGUtMiBzZXR1cCB0aGlzIGlzIHR5cGljYWxseSBm
YWlybHkNCj4gPiA+IHNpbXBsZQ0KPiA+ID4gPiA+IGJlY2F1c2UNCj4gPiA+ID4gPiA+ID4gPiB0
aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaG9zdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gc3lzdGVtIGNhbiBvcmNoZXN0cmF0ZSB0aGUgdmFyaW91cyBtb2R1bGVzDQo+IHRoYXQNCj4g
PiA+IG1ha2UNCj4gPiA+ID4gPiB1cCB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gY29tcGxldGUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHN5c3RlbS4gSW4gdGhlIHR5cGUtMSBjYXNl
IChvciBldmVuIHR5cGUtMg0KPiB3aXRoDQo+ID4gPiA+ID4gZGVsZWdhdGVkDQo+ID4gPiA+ID4g
PiA+ID4gPiA+IHNlcnZpY2UgVk1zKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd2Ug
bmVlZCBzb21lIHNvcnQgb2YgbWVjaGFuaXNtIHRvIGluZm9ybSB0aGUNCj4gPiA+IGJhY2tlbmQN
Cj4gPiA+ID4gPiBWTQ0KPiA+ID4gPiA+ID4gPiA+IGFib3V0DQo+ID4gPiA+ID4gPiA+ID4gPiA+
IGtleQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZGV0YWlscyBhYm91dCB0aGUgc3lz
dGVtOg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICAgLSB3aGVyZSB2aXJ0IHF1ZXVlIG1lbW9yeSBpcyBpbiBpdCdzIGFkZHJlc3MN
Cj4gPiA+IHNwYWNlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gaG93IGl0J3Mg
Z29pbmcgdG8gcmVjZWl2ZSAoaW50ZXJydXB0KSBhbmQNCj4gPiA+IHRyaWdnZXINCj4gPiA+ID4g
PiAoa2ljaykNCj4gPiA+ID4gPiA+ID4gPiA+ID4gZXZlbnRzDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiAgIC0gd2hhdCAoaWYgYW55KSByZXNvdXJjZXMgdGhlIGJhY2tlbmQgbmVlZHMN
Cj4gdG8NCj4gPiA+ID4gPiBjb25uZWN0IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT2J2aW91c2x5IHlvdSBjYW4gZWxpZGUg
b3ZlciBjb25maWd1cmF0aW9uDQo+IGlzc3Vlcw0KPiA+ID4gYnkNCj4gPiA+ID4gPiBoYXZpbmcN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gc3RhdGljDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBjb25maWd1cmF0aW9ucyBhbmQgYmFraW5nIHRoZSBhc3N1bXB0aW9ucyBpbnRvDQo+ID4gPiB5
b3VyDQo+ID4gPiA+ID4gZ3Vlc3QNCj4gPiA+ID4gPiA+ID4gPiBpbWFnZXMNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gaG93ZXZlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhpcyBpc24n
dCBzY2FsYWJsZSBpbiB0aGUgbG9uZyB0ZXJtLiBUaGUNCj4gb2J2aW91cw0KPiA+ID4gPiA+IHNv
bHV0aW9uDQo+ID4gPiA+ID4gPiA+ID4gc2VlbXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gYmUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV4dGVuZGluZyBhIHN1YnNldCBvZiBEZXZp
Y2UgVHJlZSBkYXRhIHRvIHVzZXINCj4gPiA+IHNwYWNlDQo+ID4gPiA+ID4gYnV0DQo+ID4gPiA+
ID4gPiA+ID4gcGVyaGFwcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGVyZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gYXJlIG90aGVyIGFwcHJvYWNoZXM/DQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQmVmb3JlIGFueSB2
aXJ0aW8gdHJhbnNhY3Rpb25zIGNhbiB0YWtlIHBsYWNlDQo+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+
IGFwcHJvcHJpYXRlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gbWFwcGluZ3MgbmVlZCB0byBiZSBtYWRlIGJldHdlZW4gdGhlIEZFIGd1
ZXN0DQo+IGFuZA0KPiA+ID4gdGhlDQo+ID4gPiA+ID4gQkUNCj4gPiA+ID4gPiA+ID4gPiBndWVz
dC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gQ3VycmVudGx5IHRoZSB3aG9sZSBvZiB0aGUgRkUgZ3Vlc3RzIGFkZHJlc3MNCj4gc3Bh
Y2UNCj4gPiA+ID4gPiBuZWVkcyB0bw0KPiA+ID4gPiA+ID4gPiA+IGJlDQo+ID4gPiA+ID4gPiA+
ID4gPiA+IHZpc2libGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHdoYXRldmVy
IGlzIHNlcnZpbmcgdGhlIHZpcnRpbyByZXF1ZXN0cy4gSQ0KPiBjYW4NCj4gPiA+ID4gPiBlbnZp
c2lvbiAzDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFwcHJvYWNoZXM6DQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICogQkUgZ3Vlc3Qg
Ym9vdHMgd2l0aCBtZW1vcnkgYWxyZWFkeSBtYXBwZWQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgVGhpcyB3b3VsZCBlbnRhaWwg
dGhlIGd1ZXN0IE9TIGtub3dpbmcgd2hlcmUNCj4gaW4NCj4gPiA+IGl0J3MNCj4gPiA+ID4gPiBH
dWVzdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBQaHlzaWNhbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gIEFkZHJlc3Mgc3BhY2UgaXMgYWxyZWFkeSB0YWtlbiB1cCBhbmQNCj4gYXZvaWRp
bmcNCj4gPiA+ID4gPiBjbGFzaGluZy4gSQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiB3b3VsZCBhc3N1
bWUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBpbiB0aGlzIGNhc2UgeW91IHdvdWxk
IHdhbnQgYSBzdGFuZGFyZA0KPiBpbnRlcmZhY2UNCj4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+ID4g
dXNlcnNwYWNlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHRoZW4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ICBtYWtlIHRoYXQgYWRkcmVzcyBzcGFjZSB2aXNpYmxlIHRvIHRoZQ0KPiBi
YWNrZW5kDQo+ID4gPiBkYWVtb24uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IFlldCBhbm90aGVyIHdheSBoZXJlIGlzIHRoYXQgd2Ugd291bGQg
aGF2ZSB3ZWxsDQo+IGtub3duDQo+ID4gPiA+ID4gInNoYXJlZA0KPiA+ID4gPiA+ID4gPiA+ID4g
PiBtZW1vcnkiIGJldHdlZW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVk1zLiBJIHRoaW5r
IHRoYXQgSmFpbGhvdXNlJ3MgaXZzaG1lbSBnaXZlcyB1cw0KPiBnb29kDQo+ID4gPiA+ID4gaW5z
aWdodHMgb24NCj4gPiA+ID4gPiA+ID4gPiB0aGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+IG1hdHRl
cg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhbmQgdGhhdCBpdCBjYW4gZXZlbiBiZSBhbiBh
bHRlcm5hdGl2ZSBmb3INCj4gaHlwZXJ2aXNvci0NCj4gPiA+ID4gPiBhZ25vc3RpYw0KPiA+ID4g
PiA+ID4gPiA+ID4gPiBzb2x1dGlvbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gKFBsZWFzZSBub3RlIG1lbW9yeSByZWdpb25zIGluIGl2c2ht
ZW0gYXBwZWFyIGFzIGENCj4gUENJDQo+ID4gPiA+ID4gZGV2aWNlDQo+ID4gPiA+ID4gPiA+ID4g
YW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGNhbiBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBtYXBwZWQgbG9jYWxseS4pDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IEkgd2FudCB0byBhZGQgdGhpcyBzaGFyZWQgbWVtb3J5IGFzcGVjdCB0
byBteQ0KPiB2aXJ0aW8tDQo+ID4gPiBwcm94eSwNCj4gPiA+ID4gPiBidXQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gdGhlIHJlc3VsdGFudCBzb2x1dGlvbiB3b3VsZCBldmVudHVhbGx5IGxv
b2sNCj4gc2ltaWxhcg0KPiA+ID4gdG8NCj4gPiA+ID4gPiBpdnNobWVtLg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICogQkUgZ3Vlc3Rz
IGJvb3RzIHdpdGggYSBoeXBlcnZpc29yIGhhbmRsZSB0bw0KPiA+ID4gbWVtb3J5DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIFRo
ZSBCRSBndWVzdCBpcyB0aGVuIGZyZWUgdG8gbWFwIHRoZSBGRSdzDQo+IG1lbW9yeQ0KPiA+ID4g
dG8NCj4gPiA+ID4gPiB3aGVyZQ0KPiA+ID4gPiA+ID4gPiA+IGl0DQo+ID4gPiA+ID4gPiA+ID4g
PiA+IHdhbnRzIGluDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgdGhlIEJFJ3MgZ3Vl
c3QgcGh5c2ljYWwgYWRkcmVzcyBzcGFjZS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEkgY2Fubm90IHNlZSBob3cgdGhpcyBjb3VsZCB3
b3JrIGZvciBYZW4uIFRoZXJlDQo+IGlzDQo+ID4gPiBubw0KPiA+ID4gPiA+ICJoYW5kbGUiDQo+
ID4gPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gZ2l2ZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHRvIHRoZSBiYWNrZW5kIGlmIHRoZSBiYWNrZW5kIGlzIG5vdCBydW5u
aW5nIGluDQo+IGRvbTAuDQo+ID4gPiBTbw0KPiA+ID4gPiA+IGZvcg0KPiA+ID4gPiA+ID4gPiA+
IFhlbiBJDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRoaW5rDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gdGhlIG1lbW9yeSBoYXMgdG8gYmUgYWxyZWFkeSBtYXBwZWQNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSW4gWGVuJ3MgSU9SRVEgc29s
dXRpb24gKHZpcnRpby1ibGspLCB0aGUNCj4gZm9sbG93aW5nDQo+ID4gPiA+ID4gaW5mb3JtYXRp
b24NCj4gPiA+ID4gPiA+ID4gPiBpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBleHBlY3RlZA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0byBiZSBleHBvc2VkIHRvIEJFIHZpYSBYZW5zdG9yZToN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gKEkga25vdyB0aGF0IHRoaXMgaXMgYSB0ZW50YXRp
dmUgYXBwcm9hY2ggdGhvdWdoLikNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgLSB0aGUg
c3RhcnQgYWRkcmVzcyBvZiBjb25maWd1cmF0aW9uIHNwYWNlDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ICAgIC0gaW50ZXJydXB0IG51bWJlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
ICAtIGZpbGUgcGF0aCBmb3IgYmFja2luZyBzdG9yYWdlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ICAgIC0gcmVhZC1vbmx5IGZsYWcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQW5kIHRo
ZSBCRSBzZXJ2ZXIgaGF2ZSB0byBjYWxsIGEgcGFydGljdWxhcg0KPiBoeXBlcnZpc29yDQo+ID4g
PiA+ID4gaW50ZXJmYWNlDQo+ID4gPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gbWFwIHRoZSBjb25maWd1cmF0aW9uIHNwYWNlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFllcywgWGVuc3RvcmUgd2FzIGNob3NlbiBhcyBh
IHNpbXBsZSB3YXkgdG8gcGFzcw0KPiA+ID4gPiA+IGNvbmZpZ3VyYXRpb24NCj4gPiA+ID4gPiA+
ID4gPiBpbmZvIHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBiYWNrZW5kIHJ1bm5pbmcgaW4g
YSBub24tdG9vbHN0YWNrIGRvbWFpbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEkgcmVtZW1i
ZXIsIHRoZXJlIHdhcyBhIHdpc2ggdG8gYXZvaWQgdXNpbmcgWGVuc3RvcmUNCj4gaW4NCj4gPiA+
ID4gPiBWaXJ0aW8NCj4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+
IGl0c2VsZiBpZiBwb3NzaWJsZSwgc28gZm9yIG5vbi10b29sc3RhY2sgZG9tYWluLCB0aGlzDQo+
IGNvdWxkDQo+ID4gPiBkb25lDQo+ID4gPiA+ID4gd2l0aA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBh
ZGp1c3RpbmcgZGV2ZCAoZGFlbW9uIHRoYXQgbGlzdGVucyBmb3IgZGV2aWNlcyBhbmQNCj4gbGF1
bmNoZXMNCj4gPiA+ID4gPiBiYWNrZW5kcykNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHJl
YWQgYmFja2VuZCBjb25maWd1cmF0aW9uIGZyb20gdGhlIFhlbnN0b3JlDQo+IGFueXdheQ0KPiA+
ID4gYW5kDQo+ID4gPiA+ID4gcGFzcyBpdA0KPiA+ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4g
PiA+ID4gPiA+IHRoZSBiYWNrZW5kIHZpYSBjb21tYW5kIGxpbmUgYXJndW1lbnRzLg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IFllcywgaW4gY3VycmVudCBQb0MgY29kZSB3ZSdyZSB1c2luZyB4ZW5zdG9yZSB0byBw
YXNzDQo+ID4gPiBkZXZpY2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gY29uZmlndXJhdGlvbi4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBXZSBhbHNvIGRlc2lnbmVkIGEgc3RhdGljIGRldmljZSBjb25m
aWd1cmF0aW9uIHBhcnNlDQo+ID4gPiBtZXRob2QNCj4gPiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+
ID4gPiBEb20wbGVzcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBvcg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IG90aGVyIHNjZW5hcmlvcyBkb24ndCBoYXZlIHhlbnRvb2wuIHllcywgaXQncyBmcm9tDQo+
IGRldmljZQ0KPiA+ID4gPiA+IG1vZGVsDQo+ID4gPiA+ID4gPiA+ID4gY29tbWFuZA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiBsaW5lDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gb3IgYSBjb25maWcgZmls
ZS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQnV0LCBp
ZiAuLi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEluIG15IGFwcHJvYWNoICh2aXJ0aW8tcHJveHkp
LCBhbGwgdGhvc2UgWGVuIChvcg0KPiA+ID4gPiA+IGh5cGVydmlzb3IpLQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiBzcGVjaWZpYw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzdHVmZnMgYXJlIGNv
bnRhaW5lZCBpbiB2aXJ0aW8tcHJveHksIHlldCBhbm90aGVyDQo+IFZNLA0KPiA+ID4gdG8NCj4g
PiA+ID4gPiBoaWRlDQo+ID4gPiA+ID4gPiA+ID4gYWxsDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGRl
dGFpbHMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Li4uIHRoZSBzb2x1dGlvbiBob3cgdG8gb3ZlcmNvbWUgdGhhdCBpcyBhbHJlYWR5DQo+IGZvdW5k
DQo+ID4gPiBhbmQNCj4gPiA+ID4gPiBwcm92ZW4NCj4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4g
PiA+ID4gPiA+ID4gPiB3b3JrIHRoZW4gZXZlbiBiZXR0ZXIuDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICMgTXkgcG9pbnQgaXMgdGhhdCBhICJoYW5kbGUiIGlz
IG5vdCBtYW5kYXRvcnkgZm9yDQo+ID4gPiA+ID4gZXhlY3V0aW5nDQo+ID4gPiA+ID4gPiA+ID4g
bWFwcGluZy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBhbmQgdGhlIG1hcHBpbmcgcHJvYmFibHkgZG9uZSBieSB0aGUNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiB0b29sc3RhY2sgKGFsc28gc2VlIGJlbG93LikgT3Igd2Ugd291bGQg
aGF2ZSB0bw0KPiA+ID4gaW52ZW50IGENCj4gPiA+ID4gPiBuZXcNCj4gPiA+ID4gPiA+ID4gPiBY
ZW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBoeXBlcnZpc29yIGludGVyZmFjZSBhbmQg
WGVuIHZpcnR1YWwgbWFjaGluZQ0KPiA+ID4gcHJpdmlsZWdlcw0KPiA+ID4gPiA+IHRvDQo+ID4g
PiA+ID4gPiA+ID4gYWxsb3cNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdGhpcw0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IGtpbmQgb2YgbWFwcGluZy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJZiB3ZSBydW4gdGhlIGJhY2tlbmQgaW4g
RG9tMCB0aGF0IHdlIGhhdmUgbm8NCj4gPiA+IHByb2JsZW1zDQo+ID4gPiA+ID4gb2YNCj4gPiA+
ID4gPiA+ID4gPiBjb3Vyc2UuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IE9uZSBvZiBkaWZmaWN1bHRpZXMgb24gWGVuIHRoYXQgSSBmb3VuZCBp
biBteQ0KPiBhcHByb2FjaA0KPiA+ID4gaXMNCj4gPiA+ID4gPiB0aGF0DQo+ID4gPiA+ID4gPiA+
ID4gPiA+IGNhbGxpbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc3VjaCBoeXBlcnZpc29y
IGludGVmYWNlcyAocmVnaXN0ZXJpbmcgSU9SRVEsDQo+IG1hcHBpbmcNCj4gPiA+ID4gPiBtZW1v
cnkpIGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+IG9ubHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gYWxsb3dlZCBvbiBCRSBzZXJ2ZXJzIHRoZW1zZWx2aWVzIGFuZCBzbyB3ZSB3aWxsDQo+IGhh
dmUNCj4gPiA+IHRvDQo+ID4gPiA+ID4gZXh0ZW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRob3Nl
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZXMuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IFRoaXMsIGhvd2V2ZXIsIHdpbGwgcmFpc2Ugc29tZSBjb25jZXJuIG9uIHNlY3Vy
aXR5DQo+IGFuZA0KPiA+ID4gPiA+IHByaXZpbGVnZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBkaXN0
cmlidXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYXMgU3RlZmFuIHN1Z2dlc3RlZC4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBXZSBhbHNv
IGZhY2VkIHBvbGljeSByZWxhdGVkIGlzc3VlcyB3aXRoIFZpcnRpbw0KPiBiYWNrZW5kDQo+ID4g
PiA+ID4gcnVubmluZyBpbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBvdGhlciB0aGFuIERvbTAgZG9t
YWluIGluIGEgImR1bW15IiB4c20gbW9kZS4gSW4gb3VyDQo+IHRhcmdldA0KPiA+ID4gPiA+IHN5
c3RlbSB3ZQ0KPiA+ID4gPiA+ID4gPiA+IHJ1bg0KPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgYmFj
a2VuZCBpbiBhIGRyaXZlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZG9tYWluICh3ZSBjYWxs
IGl0IERvbUQpIHdoZXJlIHRoZSB1bmRlcmx5aW5nIEgvVw0KPiA+ID4gcmVzaWRlcy4NCj4gPiA+
ID4gPiBXZQ0KPiA+ID4gPiA+ID4gPiA+IHRydXN0IGl0LA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBz
byB3ZSB3cm90ZSBwb2xpY3kgcnVsZXMgKHRvIGJlIHVzZWQgaW4gImZsYXNrIiB4c20gbW9kZSkN
Cj4gdG8NCj4gPiA+ID4gPiBwcm92aWRlDQo+ID4gPiA+ID4gPiA+ID4gaXQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gd2l0aCBhIGxpdHRsZSBiaXQgbW9yZSBwcml2aWxlZ2VzIHRoYW4gYSBzaW1wbGUg
RG9tVSBoYWQuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBOb3cgaXQgaXMgcGVybWl0dGVkIHRv
IGlzc3VlIGRldmljZS1tb2RlbCwgcmVzb3VyY2UNCj4gYW5kDQo+ID4gPiA+ID4gbWVtb3J5DQo+
ID4gPiA+ID4gPiA+ID4gPiA+IG1hcHBpbmdzLCBldGMgY2FsbHMuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRvIGFjdGl2YXRlIHRoZSBt
YXBwaW5nIHdpbGwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICByZXF1aXJlIHNvbWUg
c29ydCBvZiBoeXBlcmNhbGwgdG8gdGhlDQo+IGh5cGVydmlzb3IuDQo+ID4gPiBJDQo+ID4gPiA+
ID4gY2FuIHNlZQ0KPiA+ID4gPiA+ID4gPiA+IHR3bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBvcHRp
b25zDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgYXQgdGhpcyBwb2ludDoNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
IC0gZXhwb3NlIHRoZSBoYW5kbGUgdG8gdXNlcnNwYWNlIGZvcg0KPiA+ID4gZGFlbW9uL2hlbHBl
cg0KPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+ID4gdHJpZ2dlcg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICBtYXBwaW5nIHZpYSBl
eGlzdGluZyBoeXBlcmNhbGwgaW50ZXJmYWNlcy4NCj4gSWYNCj4gPiA+ID4gPiB1c2luZyBhDQo+
ID4gPiA+ID4gPiA+ID4gaGVscGVyDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHlvdQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIHdvdWxkIGhhdmUgYSBoeXBlcnZpc29yIHNwZWNpZmlj
IG9uZSB0bw0KPiBhdm9pZA0KPiA+ID4gdGhlDQo+ID4gPiA+ID4gZGFlbW9uDQo+ID4gPiA+ID4g
PiA+ID4gPiA+IGhhdmluZyB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGNh
cmUgdG9vIG11Y2ggYWJvdXQgdGhlIGRldGFpbHMgb3IgcHVzaA0KPiB0aGF0DQo+ID4gPiA+ID4g
Y29tcGxleGl0eQ0KPiA+ID4gPiA+ID4gPiA+IGludG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gYQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGNvbXBpbGUgdGltZSBvcHRpb24gZm9y
IHRoZSBkYWVtb24gd2hpY2gNCj4gd291bGQNCj4gPiA+ID4gPiByZXN1bHQgaW4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gZGlmZmVyZW50DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAg
YmluYXJpZXMgYWx0aG91Z2ggYSBjb21tb24gc291cmNlIGJhc2UuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAtIGV4cG9zZSBh
IG5ldyBrZXJuZWwgQUJJIHRvIGFic3RyYWN0IHRoZQ0KPiA+ID4gaHlwZXJjYWxsDQo+ID4gPiA+
ID4gPiA+ID4gPiA+IGRpZmZlcmVuY2VzIGF3YXkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ICAgICBpbiB0aGUgZ3Vlc3Qga2VybmVsLiBJbiB0aGlzIGNhc2UgdGhlDQo+ID4gPiB1c2Vy
c3BhY2UNCj4gPiA+ID4gPiB3b3VsZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBlc3NlbnRpYWxseQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGFzayBmb3IgYW4gYWJzdHJhY3QgIm1h
cCBndWVzdCBOIG1lbW9yeSB0bw0KPiA+ID4gPiA+IHVzZXJzcGFjZQ0KPiA+ID4gPiA+ID4gPiA+
IHB0ciINCj4gPiA+ID4gPiA+ID4gPiA+ID4gYW5kIGxldA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gICAgIHRoZSBrZXJuZWwgZGVhbCB3aXRoIHRoZSBkaWZmZXJlbnQNCj4gaHlwZXJj
YWxsDQo+ID4gPiA+ID4gaW50ZXJmYWNlcy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gVGhpcyBvZg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGNvdXJzZSBhc3N1bWVzIHRoZSBtYWpv
cml0eSBvZiBCRSBndWVzdHMNCj4gd291bGQNCj4gPiA+IGJlDQo+ID4gPiA+ID4gTGludXgNCj4g
PiA+ID4gPiA+ID4gPiA+ID4ga2VybmVscyBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ICAgICBsZWF2ZXMgdGhlIGJhcmUtbWV0YWwvdW5pa2VybmVsIGFwcHJvYWNoZXMNCj4gdG8N
Cj4gPiA+ID4gPiB0aGVpciBvd24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gZGV2aWNlcy4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBP
cGVyYXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID09PT09PT09PQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRo
ZSBjb3JlIG9mIHRoZSBvcGVyYXRpb24gb2YgVmlydElPIGlzIGZhaXJseQ0KPiA+ID4gc2ltcGxl
Lg0KPiA+ID4gPiA+IE9uY2UNCj4gPiA+ID4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IHZob3N0LXVzZXIgZmVhdHVyZSBuZWdvdGlhdGlvbiBpcyBkb25lIGl0J3Mg
YQ0KPiBjYXNlDQo+ID4gPiBvZg0KPiA+ID4gPiA+ID4gPiA+IHJlY2VpdmluZw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiB1cGRhdGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV2ZW50cyBh
bmQgcGFyc2luZyB0aGUgcmVzdWx0YW50IHZpcnQgcXVldWUNCj4gZm9yDQo+ID4gPiBkYXRhLg0K
PiA+ID4gPiA+IFRoZQ0KPiA+ID4gPiA+ID4gPiA+IHZob3N0LQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiB1c2VyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzcGVjaWZpY2F0aW9uIGhhbmRs
ZXMgYSBidW5jaCBvZiBzZXR1cCBiZWZvcmUNCj4gdGhhdA0KPiA+ID4gPiA+IHBvaW50LA0KPiA+
ID4gPiA+ID4gPiA+IG1vc3RseQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gZGV0YWlsIHdoZXJlIHRoZSB2aXJ0IHF1ZXVlcyBhcmUgc2V0IHVw
IEZEJ3MNCj4gZm9yDQo+ID4gPiA+ID4gbWVtb3J5IGFuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBl
dmVudA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY29tbXVuaWNhdGlvbi4gVGhpcyBp
cyB3aGVyZSB0aGUgZW52aXNpb25lZA0KPiBzdHViDQo+ID4gPiA+ID4gcHJvY2Vzcw0KPiA+ID4g
PiA+ID4gPiA+IHdvdWxkDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGJlDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiByZXNwb25zaWJsZSBmb3IgZ2V0dGluZyB0aGUgZGFlbW9uIHVwIGFuZA0K
PiByZWFkeSB0bw0KPiA+ID4gcnVuLg0KPiA+ID4gPiA+IFRoaXMNCj4gPiA+ID4gPiA+ID4gPiBp
cw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY3VycmVudGx5IGRvbmUgaW5zaWRlIGEg
YmlnIFZNTSBsaWtlIFFFTVUgYnV0DQo+IEkNCj4gPiA+ID4gPiBzdXNwZWN0IGENCj4gPiA+ID4g
PiA+ID4gPiBtb2Rlcm4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFwcHJvYWNoIHdv
dWxkIGJlIHRvIHVzZSB0aGUgcnVzdC12bW0gdmhvc3QNCj4gY3JhdGUuDQo+ID4gPiBJdA0KPiA+
ID4gPiA+IHdvdWxkDQo+ID4gPiA+ID4gPiA+ID4gdGhlbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBl
aXRoZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNvbW11bmljYXRlIHdpdGggdGhl
IGtlcm5lbCdzIGFic3RyYWN0ZWQgQUJJDQo+IG9yIGJlDQo+ID4gPiByZS0NCj4gPiA+ID4gPiA+
ID4gPiB0YXJnZXRlZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBhcyBhDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBidWlsZCBvcHRpb24gZm9yIHRoZSB2YXJpb3VzIGh5cGVydmlzb3JzLg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
T25lIHRoaW5nIEkgbWVudGlvbmVkIGJlZm9yZSB0byBBbGV4IGlzIHRoYXQgWGVuDQo+ID4gPiBk
b2Vzbid0DQo+ID4gPiA+ID4gaGF2ZQ0KPiA+ID4gPiA+ID4gPiA+IFZNTXMNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd2F5IHRoZXkgYXJlIHR5
cGljYWxseSBlbnZpc2lvbmVkIGFuZCBkZXNjcmliZWQNCj4gaW4NCj4gPiA+IG90aGVyDQo+ID4g
PiA+ID4gPiA+ID4gPiA+IGVudmlyb25tZW50cy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBJbnN0ZWFkLCBYZW4gaGFzIElPUkVRIHNlcnZlcnMuIEVhY2ggb2YgdGhlbQ0KPiA+ID4gY29u
bmVjdHMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaW5kZXBlbmRlbnRseSB0bw0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFhlbiB2aWEgdGhlIElPUkVRIGludGVyZmFjZS4gRS5nLiB0b2RheSBt
dWx0aXBsZQ0KPiA+ID4gUUVNVXMNCj4gPiA+ID4gPiBjb3VsZA0KPiA+ID4gPiA+ID4gPiA+IGJl
DQo+ID4gPiA+ID4gPiA+ID4gPiA+IHVzZWQgYXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBlbXVsYXRvcnMgZm9yIGEgc2luZ2xlIFhlbiBWTSwgZWFjaCBvZiB0aGVtDQo+ID4gPiBjb25u
ZWN0aW5nDQo+ID4gPiA+ID4gdG8gWGVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW5k
ZXBlbmRlbnRseSB2aWEgdGhlIElPUkVRIGludGVyZmFjZS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoZSBjb21wb25lbnQgcmVzcG9u
c2libGUgZm9yIHN0YXJ0aW5nIGEgZGFlbW9uDQo+ID4gPiBhbmQvb3INCj4gPiA+ID4gPiBzZXR0
aW5nDQo+ID4gPiA+ID4gPiA+ID4gdXANCj4gPiA+ID4gPiA+ID4gPiA+ID4gc2hhcmVkDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlcyBpcyB0aGUgdG9vbHN0YWNrOiB0aGUg
eGwgY29tbWFuZCBhbmQNCj4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gbGlieGwvbGlieGMNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBsaWJyYXJpZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEkgdGhpbmsgdGhhdCBWTSBjb25maWd1cmF0
aW9uIG1hbmFnZW1lbnQgKG9yDQo+ID4gPiBvcmNoZXN0cmF0aW9uDQo+ID4gPiA+ID4gaW4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gU3RhcnRvcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBqYXJn
b24/KSBpcyBhIHN1YmplY3QgdG8gZGViYXRlIGluIHBhcmFsbGVsLg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBPdGhlcndpc2UsIGlzIHRoZXJlIGFueSBnb29kIGFzc3VtcHRpb24gdG8gYXZv
aWQNCj4gaXQNCj4gPiA+IHJpZ2h0DQo+ID4gPiA+ID4gbm93Pw0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9sZWtzYW5kciBhbmQgb3RoZXJz
IEkgQ0NlZCBoYXZlIGJlZW4gd29ya2luZyBvbg0KPiA+ID4gd2F5cw0KPiA+ID4gPiA+IGZvciB0
aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdG9vbHN0YWNrDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gdG8gY3JlYXRlIHZpcnRpbyBiYWNrZW5kcyBhbmQgc2V0dXAgbWVtb3J5DQo+IG1hcHBp
bmdzLg0KPiA+ID4gPiA+IFRoZXkNCj4gPiA+ID4gPiA+ID4gPiBtaWdodCBiZQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiBhYmxlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gcHJvdmlkZSBt
b3JlIGluZm8gb24gdGhlIHN1YmplY3QuIEkgZG8gdGhpbmsNCj4gd2UNCj4gPiA+IG1pc3MNCj4g
PiA+ID4gPiBhIHdheQ0KPiA+ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHBy
b3ZpZGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgY29uZmlndXJhdGlvbiB0byB0
aGUgYmFja2VuZCBhbmQgYW55dGhpbmcNCj4gZWxzZQ0KPiA+ID4gdGhhdA0KPiA+ID4gPiA+IHRo
ZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gbWlnaHQgcmVxdWlyZSB0byBzdGFydCBkb2luZyBpdHMgam9iLg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFllcywgc29tZSB3b3JrIGhhcyBiZWVu
IGRvbmUgZm9yIHRoZSB0b29sc3RhY2sgdG8NCj4gaGFuZGxlDQo+ID4gPiA+ID4gVmlydGlvDQo+
ID4gPiA+ID4gPiA+ID4gTU1JTw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBkZXZpY2VzIGluDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBnZW5lcmFsIGFuZCBWaXJ0aW8gYmxvY2sgZGV2aWNlcyBpbiBw
YXJ0aWN1bGFyLg0KPiBIb3dldmVyLA0KPiA+ID4gaXQNCj4gPiA+ID4gPiBoYXMNCj4gPiA+ID4g
PiA+ID4gPiBub3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYmVlbiB1cHN0cmVhbmVkIHlldC4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IFVwZGF0ZWQgcGF0Y2hlcyBvbiByZXZpZXcgbm93Og0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcveGVuLWRldmVsLzE2
MjE2MjYzNjEtMjkwNzYtDQo+IDEtDQo+ID4gPiBnaXQtDQo+ID4gPiA+ID4gc2VuZC0NCj4gPiA+
ID4gPiA+ID4gPiBlbWFpbC0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gb2xla3N0eXNoQGdtYWlsLmNv
bS8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGVy
ZSBpcyBhbiBhZGRpdGlvbmFsIChhbHNvIGltcG9ydGFudCkgYWN0aXZpdHkgdG8NCj4gPiA+ID4g
PiBpbXByb3ZlL2ZpeA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBmb3JlaWduIG1lbW9yeSBtYXBwaW5n
IG9uIEFybSB3aGljaCBJIGFtIGFsc28gaW52b2x2ZWQNCj4gaW4uDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBUaGUgZm9yZWlnbiBtZW1vcnkgbWFwcGluZyBpcyBwcm9wb3NlZCB0byBiZSB1c2Vk
DQo+IGZvcg0KPiA+ID4gVmlydGlvDQo+ID4gPiA+ID4gPiA+ID4gYmFja2VuZHMNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gKGRldmljZSBlbXVsYXRvcnMpIGlmIHRoZXJlIGlzIGEgbmVlZCB0byBydW4g
Z3Vlc3QgT1MNCj4gPiA+IGNvbXBsZXRlbHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdW5tb2RpZmll
ZC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9mIGNvdXJzZSwgdGhlIG1vcmUgc2VjdXJlIHdh
eSB3b3VsZCBiZSB0byB1c2UgZ3JhbnQNCj4gPiA+IG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiA+IG1h
cHBpbmcuDQo+ID4gPiA+ID4gPiA+ID4gPiA+IEJyaWV0bHksIHRoZSBtYWluIGRpZmZlcmVuY2Ug
YmV0d2VlbiB0aGVtIGlzIHRoYXQgd2l0aA0KPiA+ID4gZm9yZWlnbg0KPiA+ID4gPiA+IG1hcHBp
bmcNCj4gPiA+ID4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYmFja2VuZA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gY2FuIG1hcCBhbnkgZ3Vlc3QgbWVtb3J5IGl0IHdhbnRzIHRv
IG1hcCwgYnV0IHdpdGgNCj4gZ3JhbnQNCj4gPiA+ID4gPiBtYXBwaW5nDQo+ID4gPiA+ID4gPiA+
ID4gaXQgaXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYWxsb3dlZCB0byBtYXAgb25seSB3aGF0IHdh
cyBwcmV2aW91c2x5IGdyYW50ZWQgYnkgdGhlDQo+ID4gPiBmcm9udGVuZC4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBTbywgdGhlcmUgbWlnaHQgYmUg
YSBwcm9ibGVtIGlmIHdlIHdhbnQgdG8gcHJlLW1hcA0KPiBzb21lDQo+ID4gPiA+ID4gZ3Vlc3QN
Cj4gPiA+ID4gPiA+ID4gPiBtZW1vcnkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaW4gYWR2YW5jZSBv
ciB0byBjYWNoZSBtYXBwaW5ncyBpbiB0aGUgYmFja2VuZCBpbiBvcmRlcg0KPiB0bw0KPiA+ID4g
PiA+IGltcHJvdmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gcGVyZm9ybWFuY2UgKGJlY2F1c2UgdGhl
IG1hcHBpbmcvdW5tYXBwaW5nIGd1ZXN0IHBhZ2VzDQo+IGV2ZXJ5DQo+ID4gPiA+ID4gcmVxdWVz
dA0KPiA+ID4gPiA+ID4gPiA+ID4gPiByZXF1aXJlcyBhIGxvdCBvZiBiYWNrIGFuZCBmb3J0aCB0
byBYZW4gKyBQMk0gdXBkYXRlcykuDQo+IEluIGENCj4gPiA+ID4gPiBudXRzaGVsbCwNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gY3VycmVudGx5LCBpbiBvcmRlciB0byBtYXAgYSBndWVzdCBwYWdlIGlu
dG8gdGhlIGJhY2tlbmQNCj4gPiA+IGFkZHJlc3MNCj4gPiA+ID4gPiBzcGFjZQ0KPiA+ID4gPiA+
ID4gPiA+IHdlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IG5lZWQgdG8gc3RlYWwgYSByZWFsIHBoeXNp
Y2FsIHBhZ2UgZnJvbSB0aGUgYmFja2VuZA0KPiBkb21haW4uDQo+ID4gPiBTbywNCj4gPiA+ID4g
PiB3aXRoDQo+ID4gPiA+ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHNhaWQgb3B0
aW1pemF0aW9ucyB3ZSBtaWdodCBlbmQgdXAgd2l0aCBubyBmcmVlIG1lbW9yeQ0KPiBpbg0KPiA+
ID4gdGhlDQo+ID4gPiA+ID4gYmFja2VuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBkb21haW4gKHNl
ZSBYU0EtMzAwKS4gQW5kIHdoYXQgd2UgdHJ5IHRvIGFjaGlldmUgaXMgdG8NCj4gbm90DQo+ID4g
PiB3YXN0ZQ0KPiA+ID4gPiA+IGENCj4gPiA+ID4gPiA+ID4gPiByZWFsDQo+ID4gPiA+ID4gPiA+
ID4gPiA+IGRvbWFpbiBtZW1vcnkgYXQgYWxsIGJ5IHByb3ZpZGluZyBzYWZlIG5vbi1hbGxvY2F0
ZWQteWV0DQo+IChzbw0KPiA+ID4gPiA+IHVudXNlZCkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYWRk
cmVzcyBzcGFjZSBmb3IgdGhlIGZvcmVpZ24gKGFuZCBncmFudCkgcGFnZXMgdG8gYmUNCj4gbWFw
cGVkDQo+ID4gPiA+ID4gaW50bywNCj4gPiA+ID4gPiA+ID4gPiB0aGlzDQo+ID4gPiA+ID4gPiA+
ID4gPiA+IGVuYWJsaW5nIHdvcmsgaW1wbGllcyBYZW4gYW5kIExpbnV4IChhbmQgbGlrZWx5IERU
Qg0KPiBiaW5kaW5ncykNCj4gPiA+ID4gPiBjaGFuZ2VzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBI
b3dldmVyLCBhcyBpdCB0dXJuZWQgb3V0LCBmb3IgdGhpcyB0byB3b3JrIGluIGEgcHJvcGVyDQo+
IGFuZA0KPiA+ID4gc2FmZQ0KPiA+ID4gPiA+IHdheQ0KPiA+ID4gPiA+ID4gPiA+IHNvbWUNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gcHJlcmVxIHdvcmsgbmVlZHMgdG8gYmUgZG9uZS4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFlvdSBjYW4gZmluZCB0aGUgcmVsYXRlZCBYZW4gZGlzY3Vzc2lvbiBh
dDoNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGh0dHBzOi8vbG9yZS5rZXJuZWwub3JnL3hlbi1k
ZXZlbC8xNjI3NDg5MTEwLTI1NjMzLQ0KPiAxLQ0KPiA+ID4gZ2l0LQ0KPiA+ID4gPiA+IHNlbmQt
DQo+ID4gPiA+ID4gPiA+ID4gZW1haWwtDQo+ID4gPiA+ID4gPiA+ID4gPiA+IG9sZWtzdHlzaEBn
bWFpbC5jb20vDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbmUgcXVlc3Rpb24gaXMgaG93IHRv
IGJlc3QgaGFuZGxlDQo+IG5vdGlmaWNhdGlvbg0KPiA+ID4gYW5kDQo+ID4gPiA+ID4ga2lja3Mu
DQo+ID4gPiA+ID4gPiA+ID4gVGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGV4aXN0aW5nDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB2aG9zdC11c2VyIGZyYW1ld29yayB1c2VzIGV2ZW50
ZmQgdG8gc2lnbmFsDQo+IHRoZQ0KPiA+ID4gZGFlbW9uDQo+ID4gPiA+ID4gPiA+ID4gKGFsdGhv
dWdoDQo+ID4gPiA+ID4gPiA+ID4gPiA+IFFFTVUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IGlzIHF1aXRlIGNhcGFibGUgb2Ygc2ltdWxhdGluZyB0aGVtIHdoZW4geW91DQo+IHVzZQ0K
PiA+ID4gVENHKS4NCj4gPiA+ID4gPiBYZW4NCj4gPiA+ID4gPiA+ID4gPiBoYXMNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gaXQncyBvd24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IElPUkVR
IG1lY2hhbmlzbS4gSG93ZXZlciBsYXRlbmN5IGlzIGFuDQo+IGltcG9ydGFudA0KPiA+ID4gPiA+
IGZhY3RvciBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gaGF2aW5nDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBldmVudHMgZ28gdGhyb3VnaCB0aGUgc3R1YiB3b3VsZCBhZGQgcXVpdGUg
YQ0KPiBsb3QuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBZZWFoIEkgdGhpbmssIHJlZ2FyZGxlc3Mgb2YgYW55dGhpbmcgZWxzZSwgd2UN
Cj4gd2FudA0KPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gYmFja2VuZHMgdG8NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBjb25uZWN0IGRpcmVjdGx5IHRvIHRoZSBYZW4gaHlwZXJ2aXNv
ci4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
SW4gbXkgYXBwcm9hY2gsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBhKSBCRSAtPiBGRTog
aW50ZXJydXB0cyB0cmlnZ2VyZWQgYnkgQkUgY2FsbGluZyBhDQo+ID4gPiA+ID4gaHlwZXJ2aXNv
cg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gICAgICAgICAgICAgICB2aWEgdmlydGlvLXByb3h5DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ICBiKSBGRSAtPiBCRTogTU1JTyB0byBjb25maWcgcmFpc2VzIGV2ZW50cyAoaW4NCj4gZXZl
bnQNCj4gPiA+ID4gPiBjaGFubmVscyksDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHdoaWNoIGlzDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgY29udmVydGVkIHRvIGEgY2Fs
bGJhY2sgdG8gQkUgdmlhDQo+ID4gPiB2aXJ0aW8tDQo+ID4gPiA+ID4gcHJveHkNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgICAgICAoWGVuJ3MgZXZlbnQgY2hhbm5lbCBpcyBp
bnRlcm5uYWxseQ0KPiA+ID4gPiA+IGltcGxlbWVudGVkIGJ5DQo+ID4gPiA+ID4gPiA+ID4gPiA+
IGludGVycnVwdHMuKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBJIGRvbid0IGtub3cgd2hhdCAiY29ubmVjdCBkaXJlY3RseSIgbWVhbnMgaGVy
ZSwNCj4gYnV0DQo+ID4gPiA+ID4gc2VuZGluZw0KPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcnJ1
cHRzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHRoZSBvcHBvc2l0ZSBzaWRlIHdvdWxk
IGJlIGJlc3QgZWZmaWNpZW50Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJdnNobWVtLCBJ
IHN1cHBvc2UsIHRha2VzIHRoaXMgYXBwcm9hY2ggYnkNCj4gdXRpbGl6aW5nDQo+ID4gPiBQQ0kn
cw0KPiA+ID4gPiA+IG1zaS14DQo+ID4gPiA+ID4gPiA+ID4gPiA+IG1lY2hhbmlzbS4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBBZ3JlZSB0aGF0IE1T
SSB3b3VsZCBiZSBtb3JlIGVmZmljaWVudCB0aGFuIFNQSS4uLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gQXQgdGhlIG1vbWVudCwgaW4gb3JkZXIgdG8gbm90aWZ5IHRoZSBmcm9udGVuZCwgdGhl
DQo+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gaXNzdWVzDQo+ID4gPiA+ID4gPiA+ID4gYQ0KPiA+
ID4gPiA+ID4gPiA+ID4gPiBzcGVjaWZpYyBkZXZpY2UtbW9kZWwgY2FsbCB0byBxdWVyeSBYZW4g
dG8gaW5qZWN0IGENCj4gPiA+ID4gPiBjb3JyZXNwb25kaW5nIFNQSQ0KPiA+ID4gPiA+ID4gPiA+
IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBndWVzdC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Q291bGQgd2UgY29uc2lkZXIgdGhlIGtlcm5lbCBpbnRlcm5hbGx5DQo+IGNvbnZlcnRpbmcNCj4g
PiA+ID4gPiBJT1JFUQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiBtZXNzYWdlcyBmcm9tDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgWGVuIGh5cGVydmlzb3IgdG8gZXZlbnRmZCBldmVu
dHM/IFdvdWxkDQo+IHRoaXMNCj4gPiA+IHNjYWxlDQo+ID4gPiA+ID4gd2l0aA0KPiA+ID4gPiA+
ID4gPiA+ID4gPiBvdGhlciBrZXJuZWwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGh5
cGVyY2FsbCBpbnRlcmZhY2VzPw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFNvIGFueSB0aG91Z2h0cyBvbiB3aGF0IGRpcmVjdGlv
bnMgYXJlIHdvcnRoDQo+ID4gPiA+ID4gZXhwZXJpbWVudGluZw0KPiA+ID4gPiA+ID4gPiA+IHdp
dGg/DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBPbmUgb3B0aW9uIHdlIHNob3VsZCBjb25zaWRlciBpcyBmb3IgZWFjaA0KPiBiYWNrZW5k
IHRvDQo+ID4gPiA+ID4gY29ubmVjdA0KPiA+ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+
ID4gPiA+IFhlbiB2aWENCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgSU9SRVEgaW50
ZXJmYWNlLiBXZSBjb3VsZCBnZW5lcmFsaXplIHRoZQ0KPiBJT1JFUQ0KPiA+ID4gPiA+IGludGVy
ZmFjZQ0KPiA+ID4gPiA+ID4gPiA+IGFuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBtYWtlIGl0DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHlwZXJ2aXNvciBhZ25vc3RpYy4gVGhlIGludGVy
ZmFjZSBpcyByZWFsbHkNCj4gdHJpdmlhbA0KPiA+ID4gYW5kDQo+ID4gPiA+ID4gZWFzeQ0KPiA+
ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFkZC4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQXMgSSBzYWlkIGFib3ZlLCBt
eSBwcm9wb3NhbCBkb2VzIHRoZSBzYW1lIHRoaW5nDQo+IHRoYXQNCj4gPiA+IHlvdQ0KPiA+ID4g
PiA+ID4gPiA+IG1lbnRpb25lZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiBoZXJlIDopDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IFRoZSBkaWZmZXJlbmNlIGlzIHRoYXQgSSBkbyBjYWxsIGh5cGVy
dmlzb3INCj4gaW50ZXJmYWNlcw0KPiA+ID4gdmlhDQo+ID4gPiA+ID4gPiA+ID4gdmlydGlvLQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiBwcm94eS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGUgb25seSBYZW4tc3BlY2lmaWMgcGFydCBpcyB0
aGUgbm90aWZpY2F0aW9uDQo+ID4gPiBtZWNoYW5pc20sDQo+ID4gPiA+ID4gPiA+ID4gd2hpY2gg
aXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gYW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBl
dmVudCBjaGFubmVsLiBJZiB3ZSByZXBsYWNlZCB0aGUgZXZlbnQgY2hhbm5lbA0KPiB3aXRoDQo+
ID4gPiA+ID4gc29tZXRoaW5nDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGVsc2UgdGhlDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlIHdvdWxkIGJlIGdlbmVyaWMuIFNlZToNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRwczovL2dpdGxhYi5jb20veGVuLXByb2plY3Qv
eGVuLy0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gL2Jsb2Ivc3RhZ2luZy94ZW4vaW5jbHVkZS9wdWJs
aWMvaHZtL2lvcmVxLmgjTDUyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBJIGRvbid0IHRoaW5rIHRoYXQgdHJhbnNsYXRpbmcgSU9SRVFz
IHRvIGV2ZW50ZmQNCj4gaW4NCj4gPiA+IHRoZQ0KPiA+ID4gPiA+IGtlcm5lbA0KPiA+ID4gPiA+
ID4gPiA+IGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGENCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBnb29kIGlkZWE6IGlmIGZlZWxzIGxpa2UgaXQgd291bGQgYmUgZXh0cmENCj4gPiA+IGNv
bXBsZXhpdHkNCj4gPiA+ID4gPiBhbmQgdGhhdA0KPiA+ID4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGtlcm5lbCBzaG91bGRuJ3QgYmUgaW52b2x2ZWQgYXMgdGhp
cyBpcyBhDQo+IGJhY2tlbmQtDQo+ID4gPiA+ID4gaHlwZXJ2aXNvcg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiBpbnRlcmZhY2UuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IEdpdmVuIHRoYXQgd2UgbWF5IHdhbnQgdG8gaW1wbGVtZW50IEJFIGFzIGEg
YmFyZS0NCj4gbWV0YWwNCj4gPiA+ID4gPiA+ID4gPiBhcHBsaWNhdGlvbg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBhcyBJIGRpZCBvbiBaZXBoeXIsIEkgZG9uJ3QgdGhpbmsgdGhhdCB0aGUN
Cj4gdHJhbnNsYXRpb24NCj4gPiA+ID4gPiB3b3VsZCBub3QNCj4gPiA+ID4gPiA+ID4gPiBiZQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhIGJpZyBpc3N1ZSwgZXNwZWNpYWxseSBvbiBSVE9T
J3MuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEl0IHdpbGwgYmUgc29tZSBraW5kIG9mIGFi
c3RyYWN0aW9uIGxheWVyIG9mDQo+IGludGVycnVwdA0KPiA+ID4gPiA+IGhhbmRsaW5nDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IChvciBub3RoaW5nIGJ1dCBhIGNhbGxiYWNrIG1lY2hhbmlz
bSkuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gQWxzbywgZXZlbnRmZCBpcyB2ZXJ5IExpbnV4LWNlbnRyaWMgYW5kIHdlIGFyZQ0KPiA+ID4g
dHJ5aW5nIHRvDQo+ID4gPiA+ID4gPiA+ID4gZGVzaWduIGFuDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gaW50ZXJmYWNlIHRoYXQgY291bGQgd29yayB3ZWxsIGZvciBSVE9TZXMgdG9vLg0K
PiBJZiB3ZQ0KPiA+ID4gPiA+IHdhbnQgdG8NCj4gPiA+ID4gPiA+ID4gPiBkbw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IHNvbWV0aGluZyBkaWZmZXJlbnQsIGJvdGggT1MtYWdub3N0aWMg
YW5kDQo+ID4gPiBoeXBlcnZpc29yLQ0KPiA+ID4gPiA+IGFnbm9zdGljLA0KPiA+ID4gPiA+ID4g
PiA+ID4gPiBwZXJoYXBzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd2UgY291bGQgZGVz
aWduIGEgbmV3IGludGVyZmFjZS4gT25lIHRoYXQgY291bGQNCj4gYmUNCj4gPiA+ID4gPiA+ID4g
PiBpbXBsZW1lbnRhYmxlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IGluIHRoZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFhlbiBoeXBlcnZpc29yIGl0c2VsZiAobGlrZSBJT1JFUSkgYW5kIG9m
IGNvdXJzZQ0KPiBhbnkNCj4gPiA+ID4gPiBvdGhlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBoeXBl
cnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG9vLg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBUaGVyZSBpcyBhbHNvIGFub3RoZXIgcHJvYmxlbS4gSU9SRVEgaXMgcHJv
YmFibHkNCj4gbm90DQo+ID4gPiBiZQ0KPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+IG9u
bHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2UgbmVlZGVkLiBIYXZlIGEg
bG9vayBhdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGh0dHBzOi8vbWFyYy5pbmZvLz9s
PXhlbi0NCj4gZGV2ZWwmbT0xNjIzNzM3NTQ3MDUyMzMmdz0yLg0KPiA+ID4gPiA+IERvbid0IHdl
DQo+ID4gPiA+ID4gPiA+ID4gPiA+IGFsc28gbmVlZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IGFuIGludGVyZmFjZSBmb3IgdGhlIGJhY2tlbmQgdG8gaW5qZWN0DQo+IGludGVycnVwdHMN
Cj4gPiA+IGludG8NCj4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gZnJvbnRlbmQ/
IEFuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGlmIHRoZSBiYWNrZW5kIHJlcXVpcmVz
IGR5bmFtaWMgbWVtb3J5IG1hcHBpbmdzDQo+IG9mDQo+ID4gPiA+ID4gZnJvbnRlbmQNCj4gPiA+
ID4gPiA+ID4gPiBwYWdlcywNCj4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlbg0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHdlIHdvdWxkIGFsc28gbmVlZCBhbiBpbnRlcmZhY2UgdG8gbWFwL3Vu
bWFwDQo+IGRvbVUNCj4gPiA+IHBhZ2VzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBNeSBwcm9wb3NhbCBkb2N1bWVudCBtaWdodCBoZWxwIGhl
cmU7IEFsbCB0aGUNCj4gPiA+IGludGVyZmFjZXMNCj4gPiA+ID4gPiA+ID4gPiByZXF1aXJlZA0K
PiA+ID4gPiA+ID4gPiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdmlydGlv
LXByb3h5IChvciBoeXBlcnZpc29yLXJlbGF0ZWQgaW50ZXJmYWNlcykNCj4gYXJlDQo+ID4gPiBs
aXN0ZWQNCj4gPiA+ID4gPiBhcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBSUEMgcHJvdG9j
b2xzIDopDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gVGhlc2UgaW50ZXJmYWNlcyBhcmUgYSBsb3QgbW9yZSBwcm9ibGVtYXRpYyB0aGFuDQo+
ID4gPiBJT1JFUToNCj4gPiA+ID4gPiBJT1JFUQ0KPiA+ID4gPiA+ID4gPiA+IGlzDQo+ID4gPiA+
ID4gPiA+ID4gPiA+IHRpbnkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhbmQgc2VsZi1j
b250YWluZWQuIEl0IGlzIGVhc3kgdG8gYWRkIGFueXdoZXJlLg0KPiBBDQo+ID4gPiBuZXcNCj4g
PiA+ID4gPiA+ID4gPiBpbnRlcmZhY2UgdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBp
bmplY3QgaW50ZXJydXB0cyBvciBtYXAgcGFnZXMgaXMgbW9yZSBkaWZmaWN1bHQNCj4gdG8NCj4g
PiA+ID4gPiBtYW5hZ2UNCj4gPiA+ID4gPiA+ID4gPiBiZWNhdXNlDQo+ID4gPiA+ID4gPiA+ID4g
PiA+IGl0DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd291bGQgcmVxdWlyZSBjaGFuZ2Vz
IHNjYXR0ZXJlZCBhY3Jvc3MgdGhlDQo+IHZhcmlvdXMNCj4gPiA+ID4gPiBlbXVsYXRvcnMuDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEV4YWN0
bHkuIEkgaGF2ZSBubyBjb25maWRlbnQgeWV0IHRoYXQgbXkgYXBwcm9hY2gNCj4gd2lsbA0KPiA+
ID4gPiA+IGFsc28NCj4gPiA+ID4gPiA+ID4gPiBhcHBseQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiB0byBvdGhlciBoeXBlcnZpc29ycyB0aGFuIFhlbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gVGVjaG5pY2FsbHksIHllcywgYnV0IHdoZXRoZXIgcGVvcGxlIGNhbiBhY2NlcHQgaXQN
Cj4gb3INCj4gPiA+IG5vdA0KPiA+ID4gPiA+IGlzIGENCj4gPiA+ID4gPiA+ID4gPiA+ID4gZGlm
ZmVyZW50DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG1hdHRlci4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVGhhbmtzLA0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiAtLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gUmVnYXJkcywN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbGVrc2Fu
ZHIgVHlzaGNoZW5rbw0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 01:33:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 01:33:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176777.321713 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbbr-0006Oy-MU; Thu, 02 Sep 2021 01:33:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176777.321713; Thu, 02 Sep 2021 01:33:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbbr-0006Or-Hp; Thu, 02 Sep 2021 01:33:43 +0000
Received: by outflank-mailman (input) for mailman id 176777;
 Thu, 02 Sep 2021 01:33:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLbbq-0006Oh-DQ; Thu, 02 Sep 2021 01:33:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLbbq-0008J5-9X; Thu, 02 Sep 2021 01:33:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLbbp-0008A7-TL; Thu, 02 Sep 2021 01:33:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLbbp-0004nd-Sp; Thu, 02 Sep 2021 01:33:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qsEPoFgN8+i4iFXQU0KZjS9JdrEJXiSkpTu5VLLbcsA=; b=x/57GB9eEhXdqdsV0sxl379Lf1
	/ZL0ZKQi5hrqNBmUFLVyk7R2lQiChkBnbnVO+KDyxsg8QYE7ElxOFp3Dd3DkHedvkzxNriTzoJan3
	PNMnb4SvdahWgc7QJzoNwPCNHKzUNcDaJXC0swQ1Yqq5tFVCG3amrPJeOJ0JPKNT/n/c=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164733-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164733: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=12e33dca4c0612a0975265e5ba641c6261a26455
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 01:33:41 +0000

flight 164733 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164733/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 12e33dca4c0612a0975265e5ba641c6261a26455
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    6 attempts
Testing same since   164718  2021-09-01 16:40:02 Z    0 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 01:51:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 01:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176786.321732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbt7-0000mf-5S; Thu, 02 Sep 2021 01:51:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176786.321732; Thu, 02 Sep 2021 01:51:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLbt7-0000mY-1l; Thu, 02 Sep 2021 01:51:33 +0000
Received: by outflank-mailman (input) for mailman id 176786;
 Thu, 02 Sep 2021 01:51:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do2V=NY=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLbt5-0000mS-5k
 for xen-devel@lists.xen.org; Thu, 02 Sep 2021 01:51:31 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.87]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 46f9b54e-0b90-11ec-ae34-12813bfff9fa;
 Thu, 02 Sep 2021 01:51:24 +0000 (UTC)
Received: from DB8PR06CA0065.eurprd06.prod.outlook.com (2603:10a6:10:120::39)
 by AM6PR08MB3430.eurprd08.prod.outlook.com (2603:10a6:20b:43::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 01:51:21 +0000
Received: from DB5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:120:cafe::d4) by DB8PR06CA0065.outlook.office365.com
 (2603:10a6:10:120::39) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 01:51:21 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT024.mail.protection.outlook.com (10.152.20.67) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 01:51:20 +0000
Received: ("Tessian outbound 56612e04f172:v103");
 Thu, 02 Sep 2021 01:51:20 +0000
Received: from c4cfdaf40bab.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8D5327BB-6238-4626-A4E2-77B47D878D02.1; 
 Thu, 02 Sep 2021 01:51:19 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c4cfdaf40bab.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 01:51:19 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4886.eurprd08.prod.outlook.com (2603:10a6:10:f4::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 01:50:50 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.026; Thu, 2 Sep 2021
 01:50:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46f9b54e-0b90-11ec-ae34-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=67jr1OBRC0o1ayuQ1as8Gfa0TcOJljslrUMNhB8dyuw=;
 b=yeq4pwiYliZmoYmNVQEtijnvgsGSPvTbvzkLDHC+++SskgSCEHW2UVGJXquubZ71M5QZXTVsO8GyHvShuj4rtNYlk1KyUbXBbhGyXugAF5uk06lqMP9FCmoGeikkPr0iGPZnO8SJR5ckuW22hyRe0WkGS41reBY+7IbPLjtxy/8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xen.org; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;lists.xen.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eXwtsRSyGfTEDiDlnjAJxjNNuHTuhptzq8ORA2BpAisR9jQg0ZehjmlvZVRoUXllkFPem7dI34gkAK+QKtAJIGgVk5IlCfRkQxH1NnvyqX1kYGnFgr3n6/OorMtxDmArwNVAwTXIDPoJo0an8VLdmx0YA8/tVjR31M1s9fG3JtugMqvtvfPGjaeyOEpun7xDD5DSEqwMXNunM23oftVmyYNMtwN26kY+ZFFPN+b2FAWmar6WVy0LK2fJoptp7SJJuQMxS9V46z9ixmno8bvxckmDAi6kt9Po7IoN+MSfZneZIJRvR58MRAuDNt1V3g0BMHWh84E6lY/TrpmfpXIhhw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=67jr1OBRC0o1ayuQ1as8Gfa0TcOJljslrUMNhB8dyuw=;
 b=ci9Q5NWjne4CtEavERpV0i33Uh+Ikx+MxmnXR4XEaKBt5f1J6sGzOaFNA73tJTKa1CWzoHalTfco3+9FLj67jWVOKlca3d2AH3ssjwVenE89PrwFJnfgjRF92p0UlqlwIs0bDgpMznFk6MboEmO3xyHzrVtGydzo+A6yBGi0hQJLeueb0JqKM/B+n61zEf6qpvEX/1wDZH0eaQpFyC+dOfyzieWbqTY/BpNfxJo6oYV+fT+xTRlxfvNJr37dyFeoM0en+GsyQJ56NoVdNyaScCEyv90Mln6odS4FXK/1RMez94AxK+wvzilKVt75Qd+2ZJyc6Em+3T+xq6a48sNZng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=67jr1OBRC0o1ayuQ1as8Gfa0TcOJljslrUMNhB8dyuw=;
 b=yeq4pwiYliZmoYmNVQEtijnvgsGSPvTbvzkLDHC+++SskgSCEHW2UVGJXquubZ71M5QZXTVsO8GyHvShuj4rtNYlk1KyUbXBbhGyXugAF5uk06lqMP9FCmoGeikkPr0iGPZnO8SJR5ckuW22hyRe0WkGS41reBY+7IbPLjtxy/8=
From: Wei Chen <Wei.Chen@arm.com>
To: Wei Chen <Wei.Chen@arm.com>, AKASHI Takahiro <takahiro.akashi@linaro.org>
CC: Oleksandr Tyshchenko <olekstysh@gmail.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin
	<Kaly.Xin@arm.com>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd
 Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com"
	<stefanha@redhat.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik
	<cvanscha@qti.qualcomm.com>, "pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker
	<jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien
 Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, Paul Durrant
	<paul@xen.org>, nd <nd@arm.com>, Xen Devel <xen-devel@lists.xen.org>
Subject: RE: Enabling hypervisor agnosticism for VirtIO backends
Thread-Topic: Enabling hypervisor agnosticism for VirtIO backends
Thread-Index:
 AQHXiWXOlXnRp9GT30mY/CfP5cid16tt4TUAgAVQpQCAAsKLQIABdsmAgAADLTCAAWVyAIAAMB+ggAMGMQCACZ/8gIAAKWUQgAd5oYCAAeKfwIAAF2QAgADY4nCAAAatsA==
Date: Thu, 2 Sep 2021 01:50:49 +0000
Message-ID:
 <DB9PR08MB6857E89F3072B459D374D0719ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References:
 <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa>
 <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210831061805.GB69817@laputa>
 <DB9PR08MB6857A38DAF12053DCA502E039ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210901122910.GA100270@laputa>
 <DB9PR08MB6857256B9927EDB9772636139ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
In-Reply-To:
 <DB9PR08MB6857256B9927EDB9772636139ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: EF3DCAE37B0AA847B2A4DC19FCA89E88.0
x-checkrecipientchecked: true
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 8e4df78d-ec5c-4f87-74b9-08d96db42991
x-ms-traffictypediagnostic: DBBPR08MB4886:|AM6PR08MB3430:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB3430710E5DDA75B4B29ED9FC9ECE9@AM6PR08MB3430.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GWrzqVot/KvGSsKO8j6OVdjTTvxiuNDausV2e1r8elS5+yl/R/uVMC2OLlqx72RoIUL0JTt+XydrQRmQRo811RhwG0RFMAkDvSQcHJon05j18BwKo5vsis3Q1ZIdviyZu0m7FOxbbr4XtGKdhtMRzQmpJOwDC4TQHYLcd7R51ca5CTaRJKKRashInEU2lTrsq7mLce4exo2edmlU/wmJ0ySqYmARWvWFGYC/3gPneM0X084wAIfKkjY+3N4awuWFn8O6lEctu66YAZgGcFqt1klezKGgb3FwmxkBGj+6inZ1cCK0MW5qpOTyxDM4zvdbsVtENeMIzXAc/TF8yap0nNuZVcpRVpqIKytaZR96uRfDEL6SlYP31LeBPgr5T7bXnGoOKef673yS9VzoYF7Dt74s4pTfRYxxT1LUNkiXBHLY9kxyWA4jq4nUwHNrx13qTAhYbkRH7ZvLCWAwIg3k59WathRHHLElFtx+gsr+GorMsVrMsIhkvH4Wh1gvSJolPTDtrQxgmGEm6bp2+7jZTSiBG+E6FUNaZu9q5bkBRsOeYxa/1VQQP1zgwWFzPb9/5ABU2ZAFez7vmspcClmWnvK+qh+qEHHV6wza1yX9g7lBSNjh3DKeqtDx4+go3U/Z7ydeX9W2H9OR28SMw+uQbtINzsioi2JN68uN45ZMJ2ZBx8G4W3ytxdiMJycfC0xOlaBAr61BqW9NANVx0aou3itKUzKGX8xlQEWXAsP0Yy/7oau+nP3gffxczwLZAj1pNnvqQDHBClJeHqNBGeuS/ZqumNWLsPU+C84GmhrliqeOU1Zi4JrJ8sz9d+8W4JHtFhP4BVFSrHXWCNKiSIGy3Q==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(366004)(396003)(346002)(39860400002)(86362001)(38070700005)(30864003)(8936002)(8676002)(2906002)(110136005)(64756008)(66446008)(66556008)(54906003)(316002)(5660300002)(7696005)(122000001)(2940100002)(83380400001)(9686003)(966005)(55016002)(66574015)(38100700002)(478600001)(4326008)(71200400001)(186003)(6506007)(53546011)(26005)(33656002)(76116006)(66476007)(66946007)(7416002)(52536014)(559001)(579004);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?SVg1OFRSTWhPOTErRk9Wb1ZnU2lKR0JzUm40MEU4czF3SVZKL29sSkVnaHQy?=
 =?utf-8?B?K2ZjcUlJWFoxZnFrTEM5MktjNGRnVXBMU2hpMER0MFZYUjBFLzBZcW42VEw5?=
 =?utf-8?B?eUtjTkFOV24waXNOd1JCNW9Wa3VMM3ZLNmZBMlB6Q3lpc1dtNEwvUXAwVzBW?=
 =?utf-8?B?ZUZnVlZ3cUx3RlZ4Z08yTEcyVEFUN0czWU9jYkVJbnp2VHk2eXhuYldKWkFo?=
 =?utf-8?B?Rm9SMU01VnkwZ3BIK2hBemtRMlpSUDVxU1JYeXd0K1RVdHF4aGR4cGVxRG03?=
 =?utf-8?B?LzIwSjVEeDZwQUdUSTA0MnNpS0x2UVFEVUd3Z2VlQmtMTlA4cGJJblZncW9t?=
 =?utf-8?B?Nm02VWswU3RHZVQ4ODlxM0FCSmk1cjE4dTFHUEltNTRVaytzUEFxRkdZNG15?=
 =?utf-8?B?Yk5mTmVEUUdEdW1tWEJIZ3liY29vMUo3S2tjdnoxRDdhbFdXTCswQVoyY2wx?=
 =?utf-8?B?TFFFUCtnTFF0QjdNMENhUy9kbVFGOFZmQzNQZW9zZWlNdEtqbjhLWEVoN0Zn?=
 =?utf-8?B?ak5YM0YxdHhXSFdOOFNpYzFtYnhlcnQzY3V1YjVwZ203THZvZGpsWHBCei91?=
 =?utf-8?B?SW1Oa0RmSStFN25Vd0cycW1UcTFYTmFSL3FOWEJCSE5hS1JKYitkS1NjU2wy?=
 =?utf-8?B?ZlBNZ3ZuaDFxbXNMVEQrVGFmNWl3NHNwS1pDaUFiVWRhczRpNHZ4cXdjSkdF?=
 =?utf-8?B?YVdlTmN0YjZiOHdpdDkvU1JuRUhKOHphclI5QjF4OXMwUkN5L1QySG8vbUxU?=
 =?utf-8?B?elBHc2wzbUF2dzVvQS85K0JyMFNUVmFlR2I2Z0R1Mi8zSy9JTk0wZHFWQ3Rr?=
 =?utf-8?B?UGNpdVUrTkloVis5cld0Wm5GWDlaSnZYa3czamxFNHAyVHVuQk5hdThKazZX?=
 =?utf-8?B?VHROQUpaRkJPQnZNcUtRQnRGSUkxWGVJNE9BY3V5cWZzeWlidEFlQ20xQnpN?=
 =?utf-8?B?WldXcnFWU2JNZUxVWDBrYlJ5YVdWWVZGMFBjdXJhMC9FYzZFbUlkMFJsUTZx?=
 =?utf-8?B?aXJ5Yk9WUGc0eTM2Kzg1TUMzSjVpdks1c1VOOUF4eUVuck9MZUordTJ2WWJq?=
 =?utf-8?B?Q1pkS0tWQjdXR1YyT0lmRm44RDREY2YvZXRkUmo1ZTh5T0V3TVFXczdBVWZm?=
 =?utf-8?B?NWRJRjFZbUxHU2VOcFphWG1wM1lhakx2dHdXWDd2bEt1UHdsTDN1OHkwclU2?=
 =?utf-8?B?SHMrQWlFOFhBaFpBaUh0ODdUY1NtMnVOK2dmUWdZSXNuN3VDTTIydWxDUS9r?=
 =?utf-8?B?eUVoc2w5Tml4TDhzWXo0cUFiOXpHSGlYVVM5ZmxLMlMxRzVscEZodWRaKzdL?=
 =?utf-8?B?M3ZkanBrUGI2ZCszSnd6S2xHK3A2WCsvME9TTjFYcXN0UWJQdVZERmNtZkdJ?=
 =?utf-8?B?eXh2aDc4bW5CNWNHd3Z4VEJpMUUxRzBCVlpDcVR3S0dkcllIamNnaWRCVWZ2?=
 =?utf-8?B?VHNvSERlUjhGeUFVWmVockJoWWl2MjZqZ2krYy9RcUR5MFp0NWJJUjRLK1hn?=
 =?utf-8?B?MXY3RUZORVNSQTFKVU9TRzR6cHF0RUVVbVFMeGFEVUdwQ3l3VVJSTGcrQjVB?=
 =?utf-8?B?N3VlVDFrbmlyeUpwRFNUQW0yR0ZyUlEvNm9neGp0YUdxOHQrZWZNM05PUnJh?=
 =?utf-8?B?QkJPVloxSjZmcEJtSHpwaU5jcWRBcnRVQzN0djdYWkRVR29MZFVQMmhPT1NO?=
 =?utf-8?B?L2NNQ1hreG5GZTVVdGFzRVRqVDd3aHdtMjFNdUNVNVVaNkhvVC9xRFVlY1N4?=
 =?utf-8?Q?8Qy8T7/h2jTUsISheFkzxkOdeUNS3semOO2e4ws?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4886
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT024.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8b97cdaf-8e5f-42d2-7004-08d96db41743
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	E0aigNk6vmBuKMIiwjlQZC8f4cbcPQELC/0xx0aWofHWWh/aCnBErSEP2a8qEeE92331ccS+Y4E2aDq70OckR7eU8yua1PR7Lm4ZW7GkU73si1T1SMRoRqdSAMssKVY/icfYBnPXzeeByNN2UGEKzVX8Bu0PC8I4ct7jbiWxKPXq/W41DtMM9ShHVRCKYfkJDxxal3Is5pe8Q5ikoiuobCh99eq47ex6jMlxxoZ/4Dwa7Jes/TdxYkZl6rHHgxOHF1aRunT61Xjx9otsqcCgRj322DDUXZXHHt59Dp1o4lzOcTl8WpW2NBW7TF2t2i2WRTdYrPi1QdD/BxVlkaKaz1d+HQNE5mmg1QzlvYwnVhjzMDrnpIdSyKh+r6FDNPvu4vkCmVdFCcIq2RzfuvOY5UMzTn60Rhl5Vxukv+x/Wegxe0PHh7UsNhFbYOJT4JV8RkIyEUrX9UdBIW+oWsg4aTeukUKda8/tsj/P1dBoN/CGySxcEB4PRjwakexMkrxwLtjnE9M5NiTy1JmgRD2UfnSuoqvg0hKdbTLlQqyA9voPtDHIuilIIysLYpyK3IxZn1VWe9Qh63BWdSLX+kgyiCC3S22TbYPgz0p6LjVFAIk1IqICLEhEclttUFPA/T3gsbQUUP9LdtWlLBT5PYJ/hDG8Drpim6jhUWIsEL/zIcDMk9uAgDp3yJoIieUJhVH54doYu9OGfUoKCvUOPn9AdO6m4NkOYZGvoV9CcU4CE75RQa0yg9cyh+EBt3tYnh3ybSqPMKOalDR2HZ1if/UfkAq4CMo1GAk8H+bvQY8MHIAqnX6pEaNictCHbqyocmFl7ROBF59rQWevIiEAYxg0ow==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(136003)(396003)(346002)(39860400002)(46966006)(36840700001)(30864003)(336012)(478600001)(966005)(110136005)(7696005)(2906002)(5660300002)(26005)(52536014)(2940100002)(66574015)(6506007)(316002)(8676002)(53546011)(82310400003)(9686003)(4326008)(47076005)(81166007)(83380400001)(86362001)(356005)(186003)(8936002)(36860700001)(33656002)(55016002)(54906003)(70206006)(70586007)(82740400003)(579004)(559001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 01:51:20.8495
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8e4df78d-ec5c-4f87-74b9-08d96db42991
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT024.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3430

SGkgQWthc2hpLCBPbGVrc2FuZHIsDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4g
RnJvbTogWGVuLWRldmVsIDx4ZW4tZGV2ZWwtYm91bmNlc0BsaXN0cy54ZW5wcm9qZWN0Lm9yZz4g
T24gQmVoYWxmIE9mIFdlaQ0KPiBDaGVuDQo+IFNlbnQ6IDIwMjHlubQ55pyIMuaXpSA5OjMxDQo+
IFRvOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFrYXNoaUBsaW5hcm8ub3JnPg0KPiBDYzog
T2xla3NhbmRyIFR5c2hjaGVua28gPG9sZWtzdHlzaEBnbWFpbC5jb20+OyBTdGVmYW5vIFN0YWJl
bGxpbmkNCj4gPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyBBbGV4IEJlbm4/P2UgPGFsZXguYmVu
bmVlQGxpbmFyby5vcmc+OyBLYWx5IFhpbg0KPiA8S2FseS5YaW5AYXJtLmNvbT47IFN0cmF0b3Mg
TWFpbGluZyBMaXN0IDxzdHJhdG9zLWRldkBvcC1saXN0cy5saW5hcm8ub3JnPjsNCj4gdmlydGlv
LWRldkBsaXN0cy5vYXNpcy1vcGVuLm9yZzsgQXJuZCBCZXJnbWFubiA8YXJuZC5iZXJnbWFubkBs
aW5hcm8ub3JnPjsNCj4gVmlyZXNoIEt1bWFyIDx2aXJlc2gua3VtYXJAbGluYXJvLm9yZz47IFN0
ZWZhbm8gU3RhYmVsbGluaQ0KPiA8c3RlZmFuby5zdGFiZWxsaW5pQHhpbGlueC5jb20+OyBzdGVm
YW5oYUByZWRoYXQuY29tOyBKYW4gS2lzemthDQo+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsg
Q2FybCB2YW4gU2NoYWlrIDxjdmFuc2NoYUBxdGkucXVhbGNvbW0uY29tPjsNCj4gcHJhdGlrcEBx
dWljaW5jLmNvbTsgU3JpdmF0c2EgVmFkZGFnaXJpIDx2YXRzYUBjb2RlYXVyb3JhLm9yZz47IEpl
YW4tDQo+IFBoaWxpcHBlIEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhp
ZXUgUG9pcmllcg0KPiA8bWF0aGlldS5wb2lyaWVyQGxpbmFyby5vcmc+OyBPbGVrc2FuZHIgVHlz
aGNoZW5rbw0KPiA8T2xla3NhbmRyX1R5c2hjaGVua29AZXBhbS5jb20+OyBCZXJ0cmFuZCBNYXJx
dWlzDQo+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2IDxBcnRlbV9N
eWdhaWV2QGVwYW0uY29tPjsgSnVsaWVuDQo+IEdyYWxsIDxqdWxpZW5AeGVuLm9yZz47IEp1ZXJn
ZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT47IFBhdWwgRHVycmFudA0KPiA8cGF1bEB4ZW4ub3Jn
PjsgbmQgPG5kQGFybS5jb20+OyBYZW4gRGV2ZWwgPHhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnPg0K
PiBTdWJqZWN0OiBSRTogRW5hYmxpbmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lzbSBmb3IgVmlydElP
IGJhY2tlbmRzDQo+IA0KPiBIaSBBa2FzaGksDQo+IA0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn
ZS0tLS0tDQo+ID4gRnJvbTogQUtBU0hJIFRha2FoaXJvIDx0YWthaGlyby5ha2FzaGlAbGluYXJv
Lm9yZz4NCj4gPiBTZW50OiAyMDIx5bm0OeaciDHml6UgMjA6MjkNCj4gPiBUbzogV2VpIENoZW4g
PFdlaS5DaGVuQGFybS5jb20+DQo+ID4gQ2M6IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5
c2hAZ21haWwuY29tPjsgU3RlZmFubyBTdGFiZWxsaW5pDQo+ID4gPHNzdGFiZWxsaW5pQGtlcm5l
bC5vcmc+OyBBbGV4IEJlbm4/P2UgPGFsZXguYmVubmVlQGxpbmFyby5vcmc+OyBLYWx5DQo+IFhp
bg0KPiA+IDxLYWx5LlhpbkBhcm0uY29tPjsgU3RyYXRvcyBNYWlsaW5nIExpc3QgPHN0cmF0b3Mt
ZGV2QG9wLQ0KPiBsaXN0cy5saW5hcm8ub3JnPjsNCj4gPiB2aXJ0aW8tZGV2QGxpc3RzLm9hc2lz
LW9wZW4ub3JnOyBBcm5kIEJlcmdtYW5uDQo+IDxhcm5kLmJlcmdtYW5uQGxpbmFyby5vcmc+Ow0K
PiA+IFZpcmVzaCBLdW1hciA8dmlyZXNoLmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5vIFN0YWJl
bGxpbmkNCj4gPiA8c3RlZmFuby5zdGFiZWxsaW5pQHhpbGlueC5jb20+OyBzdGVmYW5oYUByZWRo
YXQuY29tOyBKYW4gS2lzemthDQo+ID4gPGphbi5raXN6a2FAc2llbWVucy5jb20+OyBDYXJsIHZh
biBTY2hhaWsgPGN2YW5zY2hhQHF0aS5xdWFsY29tbS5jb20+Ow0KPiA+IHByYXRpa3BAcXVpY2lu
Yy5jb207IFNyaXZhdHNhIFZhZGRhZ2lyaSA8dmF0c2FAY29kZWF1cm9yYS5vcmc+OyBKZWFuLQ0K
PiA+IFBoaWxpcHBlIEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUg
UG9pcmllcg0KPiA+IDxtYXRoaWV1LnBvaXJpZXJAbGluYXJvLm9yZz47IE9sZWtzYW5kciBUeXNo
Y2hlbmtvDQo+ID4gPE9sZWtzYW5kcl9UeXNoY2hlbmtvQGVwYW0uY29tPjsgQmVydHJhbmQgTWFy
cXVpcw0KPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2IDxBcnRl
bV9NeWdhaWV2QGVwYW0uY29tPjsNCj4gSnVsaWVuDQo+ID4gR3JhbGwgPGp1bGllbkB4ZW4ub3Jn
PjsgSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPjsgUGF1bCBEdXJyYW50DQo+ID4gPHBh
dWxAeGVuLm9yZz47IG5kIDxuZEBhcm0uY29tPjsgWGVuIERldmVsIDx4ZW4tZGV2ZWxAbGlzdHMu
eGVuLm9yZz4NCj4gPiBTdWJqZWN0OiBSZTogRW5hYmxpbmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lz
bSBmb3IgVmlydElPIGJhY2tlbmRzDQo+ID4NCj4gPiBIaSBXZWksDQo+ID4NCj4gPiBPbiBXZWQs
IFNlcCAwMSwgMjAyMSBhdCAxMToxMjo1OEFNICswMDAwLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+
IEhpIEFrYXNoaSwNCj4gPiA+DQo+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+
ID4gPiA+IEZyb206IEFLQVNISSBUYWthaGlybyA8dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+
DQo+ID4gPiA+IFNlbnQ6IDIwMjHlubQ45pyIMzHml6UgMTQ6MTgNCj4gPiA+ID4gVG86IFdlaSBD
aGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiBDYzogT2xla3NhbmRyIFR5c2hjaGVua28g
PG9sZWtzdHlzaEBnbWFpbC5jb20+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPiA+ID4gPHNzdGFi
ZWxsaW5pQGtlcm5lbC5vcmc+OyBBbGV4IEJlbm4/P2UgPGFsZXguYmVubmVlQGxpbmFyby5vcmc+
Ow0KPiBLYWx5DQo+ID4gWGluDQo+ID4gPiA+IDxLYWx5LlhpbkBhcm0uY29tPjsgU3RyYXRvcyBN
YWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLQ0KPiA+IGxpc3RzLmxpbmFyby5vcmc+Ow0KPiA+
ID4gPiB2aXJ0aW8tZGV2QGxpc3RzLm9hc2lzLW9wZW4ub3JnOyBBcm5kIEJlcmdtYW5uDQo+ID4g
PGFybmQuYmVyZ21hbm5AbGluYXJvLm9yZz47DQo+ID4gPiA+IFZpcmVzaCBLdW1hciA8dmlyZXNo
Lmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPiA+ID4gPHN0ZWZhbm8u
c3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFAcmVkaGF0LmNvbTsgSmFuIEtpc3prYQ0K
PiA+ID4gPiA8amFuLmtpc3prYUBzaWVtZW5zLmNvbT47IENhcmwgdmFuIFNjaGFpaw0KPiA8Y3Zh
bnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+IHByYXRpa3BAcXVpY2luYy5jb207IFNy
aXZhdHNhIFZhZGRhZ2lyaSA8dmF0c2FAY29kZWF1cm9yYS5vcmc+Ow0KPiBKZWFuLQ0KPiA+ID4g
PiBQaGlsaXBwZSBCcnVja2VyIDxqZWFuLXBoaWxpcHBlQGxpbmFyby5vcmc+OyBNYXRoaWV1IFBv
aXJpZXINCj4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xla3NhbmRyIFR5
c2hjaGVua28NCj4gPiA+ID4gPE9sZWtzYW5kcl9UeXNoY2hlbmtvQGVwYW0uY29tPjsgQmVydHJh
bmQgTWFycXVpcw0KPiA+ID4gPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgQXJ0ZW0gTXln
YWlldiA8QXJ0ZW1fTXlnYWlldkBlcGFtLmNvbT47DQo+ID4gSnVsaWVuDQo+ID4gPiA+IEdyYWxs
IDxqdWxpZW5AeGVuLm9yZz47IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT47IFBhdWwN
Cj4gRHVycmFudA0KPiA+ID4gPiA8cGF1bEB4ZW4ub3JnPjsgWGVuIERldmVsIDx4ZW4tZGV2ZWxA
bGlzdHMueGVuLm9yZz4NCj4gPiA+ID4gU3ViamVjdDogUmU6IEVuYWJsaW5nIGh5cGVydmlzb3Ig
YWdub3N0aWNpc20gZm9yIFZpcnRJTyBiYWNrZW5kcw0KPiA+ID4gPg0KPiA+ID4gPiBXZWksDQo+
ID4gPiA+DQo+ID4gPiA+IE9uIFRodSwgQXVnIDI2LCAyMDIxIGF0IDEyOjEwOjE5UE0gKzAwMDAs
IFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiA+IEhpIEFrYXNoaSwNCj4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+ID4gPiA+IEZyb206IEFLQVNI
SSBUYWthaGlybyA8dGFrYWhpcm8uYWthc2hpQGxpbmFyby5vcmc+DQo+ID4gPiA+ID4gPiBTZW50
OiAyMDIx5bm0OOaciDI25pelIDE3OjQxDQo+ID4gPiA+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5D
aGVuQGFybS5jb20+DQo+ID4gPiA+ID4gPiBDYzogT2xla3NhbmRyIFR5c2hjaGVua28gPG9sZWtz
dHlzaEBnbWFpbC5jb20+OyBTdGVmYW5vDQo+IFN0YWJlbGxpbmkNCj4gPiA+ID4gPiA+IDxzc3Rh
YmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5uPz9lIDxhbGV4LmJlbm5lZUBsaW5hcm8ub3Jn
PjsNCj4gPiBLYWx5DQo+ID4gPiA+IFhpbg0KPiA+ID4gPiA+ID4gPEthbHkuWGluQGFybS5jb20+
OyBTdHJhdG9zIE1haWxpbmcgTGlzdCA8c3RyYXRvcy1kZXZAb3AtDQo+ID4gPiA+IGxpc3RzLmxp
bmFyby5vcmc+Ow0KPiA+ID4gPiA+ID4gdmlydGlvLWRldkBsaXN0cy5vYXNpcy1vcGVuLm9yZzsg
QXJuZCBCZXJnbWFubg0KPiA+ID4gPiA8YXJuZC5iZXJnbWFubkBsaW5hcm8ub3JnPjsNCj4gPiA+
ID4gPiA+IFZpcmVzaCBLdW1hciA8dmlyZXNoLmt1bWFyQGxpbmFyby5vcmc+OyBTdGVmYW5vIFN0
YWJlbGxpbmkNCj4gPiA+ID4gPiA+IDxzdGVmYW5vLnN0YWJlbGxpbmlAeGlsaW54LmNvbT47IHN0
ZWZhbmhhQHJlZGhhdC5jb207IEphbiBLaXN6a2ENCj4gPiA+ID4gPiA+IDxqYW4ua2lzemthQHNp
ZW1lbnMuY29tPjsgQ2FybCB2YW4gU2NoYWlrDQo+ID4gPGN2YW5zY2hhQHF0aS5xdWFsY29tbS5j
b20+Ow0KPiA+ID4gPiA+ID4gcHJhdGlrcEBxdWljaW5jLmNvbTsgU3JpdmF0c2EgVmFkZGFnaXJp
IDx2YXRzYUBjb2RlYXVyb3JhLm9yZz47DQo+ID4gSmVhbi0NCj4gPiA+ID4gPiA+IFBoaWxpcHBl
IEJydWNrZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUgUG9pcmllcg0KPiA+
ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xla3NhbmRyIFR5c2hjaGVu
a28NCj4gPiA+ID4gPiA+IDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5k
IE1hcnF1aXMNCj4gPiA+ID4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBBcnRlbSBN
eWdhaWV2DQo+IDxBcnRlbV9NeWdhaWV2QGVwYW0uY29tPjsNCj4gPiA+ID4gSnVsaWVuDQo+ID4g
PiA+ID4gPiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+OyBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3Vz
ZS5jb20+OyBQYXVsDQo+ID4gRHVycmFudA0KPiA+ID4gPiA+ID4gPHBhdWxAeGVuLm9yZz47IFhl
biBEZXZlbCA8eGVuLWRldmVsQGxpc3RzLnhlbi5vcmc+DQo+ID4gPiA+ID4gPiBTdWJqZWN0OiBS
ZTogRW5hYmxpbmcgaHlwZXJ2aXNvciBhZ25vc3RpY2lzbSBmb3IgVmlydElPIGJhY2tlbmRzDQo+
ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gSGkgV2VpLA0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
IE9uIEZyaSwgQXVnIDIwLCAyMDIxIGF0IDAzOjQxOjUwUE0gKzA5MDAsIEFLQVNISSBUYWthaGly
byB3cm90ZToNCj4gPiA+ID4gPiA+ID4gT24gV2VkLCBBdWcgMTgsIDIwMjEgYXQgMDg6MzU6NTFB
TSArMDAwMCwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gSGkgQWthc2hp77yMDQo+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t
LS0tDQo+ID4gPiA+ID4gPiA+ID4gPiBGcm9tOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFr
YXNoaUBsaW5hcm8ub3JnPg0KPiA+ID4gPiA+ID4gPiA+ID4gU2VudDogMjAyMeW5tDjmnIgxOOaX
pSAxMzozOQ0KPiA+ID4gPiA+ID4gPiA+ID4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29t
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gQ2M6IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hA
Z21haWwuY29tPjsgU3RlZmFubw0KPiA+ID4gPiBTdGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA+ID4g
PiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IEFsZXggQmVubj8/ZQ0KPiA+IDxhbGV4LmJlbm5l
ZUBsaW5hcm8ub3JnPjsNCj4gPiA+ID4gPiA+IFN0cmF0b3MNCj4gPiA+ID4gPiA+ID4gPiA+IE1h
aWxpbmcgTGlzdCA8c3RyYXRvcy1kZXZAb3AtbGlzdHMubGluYXJvLm9yZz47IHZpcnRpby0NCj4g
PiA+ID4gPiA+IGRldkBsaXN0cy5vYXNpcy0NCj4gPiA+ID4gPiA+ID4gPiA+IG9wZW4ub3JnOyBB
cm5kIEJlcmdtYW5uIDxhcm5kLmJlcmdtYW5uQGxpbmFyby5vcmc+OyBWaXJlc2gNCj4gPiBLdW1h
cg0KPiA+ID4gPiA+ID4gPiA+ID4gPHZpcmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBT
dGFiZWxsaW5pDQo+ID4gPiA+ID4gPiA+ID4gPiA8c3RlZmFuby5zdGFiZWxsaW5pQHhpbGlueC5j
b20+OyBzdGVmYW5oYUByZWRoYXQuY29tOyBKYW4NCj4gPiBLaXN6a2ENCj4gPiA+ID4gPiA+ID4g
PiA+IDxqYW4ua2lzemthQHNpZW1lbnMuY29tPjsgQ2FybCB2YW4gU2NoYWlrDQo+ID4gPiA+ID4g
PiA8Y3ZhbnNjaGFAcXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4gPiBwcmF0aWtw
QHF1aWNpbmMuY29tOyBTcml2YXRzYSBWYWRkYWdpcmkNCj4gPiA8dmF0c2FAY29kZWF1cm9yYS5v
cmc+Ow0KPiA+ID4gPiA+ID4gSmVhbi0NCj4gPiA+ID4gPiA+ID4gPiA+IFBoaWxpcHBlIEJydWNr
ZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUNCj4gUG9pcmllcg0KPiA+ID4g
PiA+ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgT2xla3NhbmRyIFR5c2hj
aGVua28NCj4gPiA+ID4gPiA+ID4gPiA+IDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47
IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiA+ID4gPiA+ID4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFy
bS5jb20+OyBBcnRlbSBNeWdhaWV2DQo+ID4gPiA+IDxBcnRlbV9NeWdhaWV2QGVwYW0uY29tPjsN
Cj4gPiA+ID4gPiA+IEp1bGllbg0KPiA+ID4gPiA+ID4gPiA+ID4gR3JhbGwgPGp1bGllbkB4ZW4u
b3JnPjsgSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPjsNCj4gPiBQYXVsDQo+ID4gPiA+
ID4gPiBEdXJyYW50DQo+ID4gPiA+ID4gPiA+ID4gPiA8cGF1bEB4ZW4ub3JnPjsgWGVuIERldmVs
IDx4ZW4tZGV2ZWxAbGlzdHMueGVuLm9yZz4NCj4gPiA+ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJl
OiBFbmFibGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIGZvciBWaXJ0SU8NCj4gPiBiYWNrZW5k
cw0KPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+IE9uIFR1ZSwgQXVnIDE3LCAy
MDIxIGF0IDA4OjM5OjA5QU0gKzAwMDAsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiBIaSBBa2FzaGksDQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBGcm9t
OiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFrYXNoaUBsaW5hcm8ub3JnPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ45pyIMTfml6UgMTY6MDgNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gQ2M6IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hAZ21haWwuY29tPjsNCj4g
U3RlZmFubw0KPiA+ID4gPiA+ID4gU3RhYmVsbGluaQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxz
c3RhYmVsbGluaUBrZXJuZWwub3JnPjsgQWxleCBCZW5uPz9lDQo+ID4gPiA+IDxhbGV4LmJlbm5l
ZUBsaW5hcm8ub3JnPjsNCj4gPiA+ID4gPiA+ID4gPiA+IFN0cmF0b3MNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBNYWlsaW5nIExpc3QgPHN0cmF0b3MtZGV2QG9wLWxpc3RzLmxpbmFyby5vcmc+Ow0K
PiB2aXJ0aW8tDQo+ID4gPiA+ID4gPiA+ID4gPiBkZXZAbGlzdHMub2FzaXMtDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gb3Blbi5vcmc7IEFybmQgQmVyZ21hbm4gPGFybmQuYmVyZ21hbm5AbGluYXJv
Lm9yZz47DQo+ID4gVmlyZXNoDQo+ID4gPiA+IEt1bWFyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PHZpcmVzaC5rdW1hckBsaW5hcm8ub3JnPjsgU3RlZmFubyBTdGFiZWxsaW5pDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29tPjsgc3RlZmFuaGFAcmVk
aGF0LmNvbTsNCj4gPiBKYW4NCj4gPiA+ID4gS2lzemthDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PGphbi5raXN6a2FAc2llbWVucy5jb20+OyBDYXJsIHZhbiBTY2hhaWsNCj4gPiA+ID4gPiA+IDxj
dmFuc2NoYUBxdGkucXVhbGNvbW0uY29tPjsNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBwcmF0aWtw
QHF1aWNpbmMuY29tOyBTcml2YXRzYSBWYWRkYWdpcmkNCj4gPiA+ID4gPHZhdHNhQGNvZGVhdXJv
cmEub3JnPjsNCj4gPiA+ID4gPiA+IEplYW4tDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gUGhpbGlw
cGUgQnJ1Y2tlciA8amVhbi1waGlsaXBwZUBsaW5hcm8ub3JnPjsgTWF0aGlldQ0KPiA+IFBvaXJp
ZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA8bWF0aGlldS5wb2lyaWVyQGxpbmFyby5vcmc+OyBP
bGVrc2FuZHIgVHlzaGNoZW5rbw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxPbGVrc2FuZHJfVHlz
aGNoZW5rb0BlcGFtLmNvbT47IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgQXJ0ZW0gTXlnYWlldg0KPiA+ID4gPiA+ID4g
PEFydGVtX015Z2FpZXZAZXBhbS5jb20+Ow0KPiA+ID4gPiA+ID4gPiA+ID4gSnVsaWVuDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gR3JhbGwgPGp1bGllbkB4ZW4ub3JnPjsgSnVlcmdlbiBHcm9zcw0K
PiA8amdyb3NzQHN1c2UuY29tPjsNCj4gPiA+ID4gUGF1bA0KPiA+ID4gPiA+ID4gRHVycmFudA0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxwYXVsQHhlbi5vcmc+OyBYZW4gRGV2ZWwgPHhlbi1kZXZl
bEBsaXN0cy54ZW4ub3JnPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFi
bGluZyBoeXBlcnZpc29yIGFnbm9zdGljaXNtIGZvcg0KPiBWaXJ0SU8NCj4gPiA+ID4gYmFja2Vu
ZHMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEhpIFdlaSwg
T2xla3NhbmRyLA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
T24gTW9uLCBBdWcgMTYsIDIwMjEgYXQgMTA6MDQ6MDNBTSArMDAwMCwgV2VpIENoZW4NCj4gd3Jv
dGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBIaSBBbGwsDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVGhhbmtzIGZvciBTdGVmYW5vIHRvIGxpbmsg
bXkga3ZtdG9vbCBmb3IgWGVuDQo+IHByb3Bvc2FsDQo+ID4gPiA+IGhlcmUuDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBUaGlzIHByb3Bvc2FsIGlzIHN0aWxsIGRpc2N1c3NpbmcgaW4gWGVuIGFu
ZCBLVk0NCj4gPiA+ID4gY29tbXVuaXRpZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGUg
bWFpbiB3b3JrIGlzIHRvIGRlY291cGxlIHRoZSBrdm10b29sIGZyb20gS1ZNDQo+IGFuZA0KPiA+
IG1ha2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG90aGVyIGh5cGVydmlzb3JzIGNhbiByZXVz
ZSB0aGUgdmlydHVhbCBkZXZpY2UNCj4gPiA+ID4gaW1wbGVtZW50YXRpb25zLg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEluIHRoaXMgY2FzZSwgd2Ug
bmVlZCB0byBpbnRyb2R1Y2UgYW4gaW50ZXJtZWRpYXRlDQo+ID4gPiA+IGh5cGVydmlzb3INCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGxheWVyIGZvciBWTU0gYWJzdHJhY3Rpb24sIFdoaWNoIGlz
LCBJIHRoaW5rIGl0J3MNCj4gdmVyeQ0KPiA+ID4gPiBjbG9zZQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gdG8gc3RyYXRvcycgdmlydGlvIGh5cGVydmlzb3IgYWdub3N0aWNpc20gd29yay4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICMgTXkgcHJvcG9zYWxb
MV0gY29tZXMgZnJvbSBteSBvd24gaWRlYSBhbmQgZG9lc24ndA0KPiA+IGFsd2F5cw0KPiA+ID4g
PiA+ID4gcmVwcmVzZW50DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIyBMaW5hcm8ncyB2aWV3IG9u
IHRoaXMgc3ViamVjdCBub3IgcmVmbGVjdCBBbGV4J3MNCj4gPiBjb25jZXJucy4NCj4gPiA+ID4g
PiA+ID4gPiA+IE5ldmVydGhlbGVzcywNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IFlvdXIgaWRlYSBhbmQgbXkgcHJvcG9zYWwgc2VlbSB0byBzaGFyZSB0aGUg
c2FtZQ0KPiA+IGJhY2tncm91bmQuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQm90aCBoYXZlIHRo
ZSBzaW1pbGFyIGdvYWwgYW5kIGN1cnJlbnRseSBzdGFydCB3aXRoLA0KPiBhdA0KPiA+ID4gPiBm
aXJzdCwNCj4gPiA+ID4gPiA+IFhlbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCBhcmUgYmFz
ZWQgb24ga3ZtLXRvb2wuIChBY3R1YWxseSwgbXkgd29yayBpcw0KPiBkZXJpdmVkDQo+ID4gPiA+
IGZyb20NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBFUEFNJ3MgdmlydGlvLWRpc2ssIHdoaWNoIGlz
IGFsc28gYmFzZWQgb24ga3ZtLXRvb2wuKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gSW4gcGFydGljdWxhciwgdGhlIGFic3RyYWN0aW9uIG9mIGh5cGVydmlz
b3INCj4gaW50ZXJmYWNlcw0KPiA+IGhhcw0KPiA+ID4gPiBhDQo+ID4gPiA+ID4gPiBzYW1lDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gc2V0IG9mIGludGVyZmFjZXMgKGZvciB5b3VyICJzdHJ1Y3Qg
dm1tX2ltcGwiIGFuZCBteQ0KPiA+ICJSUEMNCj4gPiA+ID4gPiA+IGludGVyZmFjZXMiKS4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBUaGlzIGlzIG5vdCBjby1pbmNpZGVudCBhcyB3ZSBib3RoIHNo
YXJlIHRoZSBzYW1lDQo+IG9yaWdpbg0KPiA+IGFzDQo+ID4gPiA+IEkNCj4gPiA+ID4gPiA+IHNh
aWQNCj4gPiA+ID4gPiA+ID4gPiA+IGFib3ZlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEFuZCBz
byB3ZSB3aWxsIGFsc28gc2hhcmUgdGhlIHNhbWUgaXNzdWVzLiBPbmUgb2YgdGhlbQ0KPiA+IGlz
IGENCj4gPiA+ID4gd2F5DQo+ID4gPiA+ID4gPiBvZg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICJz
aGFyaW5nL21hcHBpbmcgRkUncyBtZW1vcnkiLiBUaGVyZSBpcyBzb21lIHRyYWRlLW9mZg0KPiA+
ID4gPiBiZXR3ZWVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIHBvcnRhYmlsaXR5IGFuZCB0
aGUgcGVyZm9ybWFuY2UgaW1wYWN0Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFNvIHdlIGNhbiBk
aXNjdXNzIHRoZSB0b3BpYyBoZXJlIGluIHRoaXMgTUwsIHRvby4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiAoU2VlIEFsZXgncyBvcmlnaW5hbCBlbWFpbCwgdG9vKS4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiBZZXMsIEkgYWdyZWUuDQo+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT24gdGhlIG90aGVyIGhhbmQsIG15IGFwcHJvYWNo
IGFpbXMgdG8gY3JlYXRlIGENCj4gInNpbmdsZS0NCj4gPiA+ID4gYmluYXJ5Ig0KPiA+ID4gPiA+
ID4gPiA+ID4gc29sdXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbiB3aGljaCB0aGUgc2Ft
ZSBiaW5hcnkgb2YgQkUgdm0gY291bGQgcnVuIG9uIGFueQ0KPiA+ID4gPiBoeXBlcnZpc29ycy4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBTb21laG93IHNpbWlsYXIgdG8geW91ciAicHJvcG9zYWwt
IzIiIGluIFsyXSwgYnV0IGluDQo+IG15DQo+ID4gPiA+IHNvbHV0aW9uLA0KPiA+ID4gPiA+ID4g
YWxsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIGh5cGVydmlzb3Itc3BlY2lmaWMgY29kZSB3
b3VsZCBiZSBwdXQgaW50byBhbm90aGVyDQo+ID4gPiA+IGVudGl0eQ0KPiA+ID4gPiA+ID4gKFZN
KSwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBuYW1lZCAidmlydGlvLXByb3h5IiBhbmQgdGhlIGFi
c3RyYWN0ZWQgb3BlcmF0aW9ucyBhcmUNCj4gPiA+ID4gc2VydmVkDQo+ID4gPiA+ID4gPiB2aWEg
UlBDLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IChJbiB0aGlzIHNlbnNlLCBCRSBpcyBoeXBlcnZp
c29yLWFnbm9zdGljIGJ1dCBtaWdodA0KPiBoYXZlDQo+ID4gT1MNCj4gPiA+ID4gPiA+ID4gPiA+
IGRlcGVuZGVuY3kuKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEJ1dCBJIGtub3cgdGhhdCB3ZSBu
ZWVkIGRpc2N1c3MgaWYgdGhpcyBpcyBhDQo+IHJlcXVpcmVtZW50DQo+ID4gPiA+IGV2ZW4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBpbiBTdHJhdG9zIHByb2plY3Qgb3Igbm90LiAoTWF5YmUgbm90
KQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+ID4gPiA+IFNvcnJ5LCBJIGhhdmVuJ3QgaGFkIHRpbWUgdG8gZmluaXNoIHJlYWRpbmcgeW91
ciB2aXJ0aW8tDQo+ID4gcHJveHkNCj4gPiA+ID4gPiA+IGNvbXBsZXRlbHkNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gKEkgd2lsbCBkbyBpdCBBU0FQKS4gQnV0IGZyb20geW91ciBkZXNjcmlwdGlvbiwg
aXQgc2VlbXMNCj4gPiB3ZQ0KPiA+ID4gPiBuZWVkIGENCj4gPiA+ID4gPiA+ID4gPiA+ID4gM3Jk
IFZNIGJldHdlZW4gRkUgYW5kIEJFPyBNeSBjb25jZXJuIGlzIHRoYXQsIGlmIG15DQo+ID4gYXNz
dW1wdGlvbg0KPiA+ID4gPiBpcw0KPiA+ID4gPiA+ID4gcmlnaHQsDQo+ID4gPiA+ID4gPiA+ID4g
PiA+IHdpbGwgaXQgaW5jcmVhc2UgdGhlIGxhdGVuY3kgaW4gZGF0YSB0cmFuc3BvcnQgcGF0aD8N
Cj4gRXZlbg0KPiA+IGlmDQo+ID4gPiA+IHdlJ3JlDQo+ID4gPiA+ID4gPiA+ID4gPiA+IHVzaW5n
IHNvbWUgbGlnaHR3ZWlnaHQgZ3Vlc3QgbGlrZSBSVE9TIG9yIFVuaWtlcm5lbCwNCj4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiBZZXMsIHlvdSdyZSByaWdodC4gQnV0IEknbSBh
ZnJhaWQgdGhhdCBpdCBpcyBhIG1hdHRlciBvZg0KPiA+IGRlZ3JlZS4NCj4gPiA+ID4gPiA+ID4g
PiA+IEFzIGZhciBhcyB3ZSBleGVjdXRlICdtYXBwaW5nJyBvcGVyYXRpb25zIGF0IGV2ZXJ5IGZl
dGNoDQo+IG9mDQo+ID4gPiA+IHBheWxvYWQsDQo+ID4gPiA+ID4gPiA+ID4gPiB3ZSB3aWxsIHNl
ZSBsYXRlbmN5IGlzc3VlIChldmVuIGluIHlvdXIgY2FzZSkgYW5kIGlmIHdlDQo+IGhhdmUNCj4g
PiA+ID4gc29tZQ0KPiA+ID4gPiA+ID4gc29sdXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+IGZvciBp
dCwgd2Ugd29uJ3Qgc2VlIGl0IG5laXRoZXIgaW4gbXkgcHJvcG9zYWwgOikNCj4gPiA+ID4gPiA+
ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiBPbGVrc2FuZHIgaGFzIHNl
bnQgYSBwcm9wb3NhbCB0byBYZW4gbWFpbGluZyBsaXN0IHRvIHJlZHVjZQ0KPiA+IHRoaXMNCj4g
PiA+ID4ga2luZA0KPiA+ID4gPiA+ID4gPiA+IG9mICJtYXBwaW5nL3VubWFwcGluZyIgb3BlcmF0
aW9ucy4gU28gdGhlIGxhdGVuY3kgY2F1c2VkIGJ5DQo+ID4gdGhpcw0KPiA+ID4gPiA+ID4gYmVo
YXZpb3INCj4gPiA+ID4gPiA+ID4gPiBvbiBYZW4gbWF5IGV2ZW50dWFsbHkgYmUgZWxpbWluYXRl
ZCwgYW5kIExpbnV4LUtWTSBkb2Vzbid0DQo+ID4gaGF2ZQ0KPiA+ID4gPiB0aGF0DQo+ID4gPiA+
ID4gPiBwcm9ibGVtLg0KPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiBPYnZpb3VzbHksIEkg
aGF2ZSBub3QgeWV0IGNhdWdodCB1cCB0aGVyZSBpbiB0aGUgZGlzY3Vzc2lvbi4NCj4gPiA+ID4g
PiA+ID4gV2hpY2ggcGF0Y2ggc3BlY2lmaWNhbGx5Pw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
IENhbiB5b3UgZ2l2ZSBtZSB0aGUgbGluayB0byB0aGUgZGlzY3Vzc2lvbiBvciBwYXRjaCwgcGxl
YXNlPw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IEl0J3MgYSBSRkMgZGlzY3Vz
c2lvbi4gV2UgaGF2ZSB0ZXN0ZWQgdGhpcyBSRkMgcGF0Y2ggaW50ZXJuYWxseS4NCj4gPiA+ID4g
PiBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hpdmVzL2h0bWwveGVuLWRldmVsLzIw
MjEtDQo+ID4gPiA+IDA3L21zZzAxNTMyLmh0bWwNCj4gPiA+ID4NCj4gPiA+ID4gSSdtIGFmcmFp
ZCB0aGF0IEkgbWlzcyBzb21ldGhpbmcgaGVyZSwgYnV0IEkgZG9uJ3Qga25vdw0KPiA+ID4gPiB3
aHkgdGhpcyBwcm9wb3NlZCBBUEkgd2lsbCBsZWFkIHRvIGVsaW1pbmF0aW5nICdtbWFwJyBpbiBh
Y2Nlc3NpbmcNCj4gPiA+ID4gdGhlIHF1ZXVlZCBwYXlsb2FkIGF0IGV2ZXJ5IHJlcXVlc3Q/DQo+
ID4gPiA+DQo+ID4gPg0KPiA+ID4gVGhpcyBBUEkgZ2l2ZSBYZW4gZGV2aWNlIG1vZGVsIChRRU1V
IG9yIGt2bXRvb2wpIHRoZSBhYmlsaXR5IHRvIG1hcA0KPiA+ID4gd2hvbGUgZ3Vlc3QgUkFNIGlu
IGRldmljZSBtb2RlbCdzIGFkZHJlc3Mgc3BhY2UuIEluIHRoaXMgY2FzZSwgZGV2aWNlDQo+ID4g
PiBtb2RlbCBkb2Vzbid0IG5lZWQgZHluYW1pYyBoeXBlcmNhbGwgdG8gbWFwL3VubWFwIHBheWxv
YWQgbWVtb3J5Lg0KPiA+ID4gSXQgY2FuIHVzZSBhIGZsYXQgb2Zmc2V0IHRvIGFjY2VzcyBwYXls
b2FkIG1lbW9yeSBpbiBpdHMgYWRkcmVzcw0KPiA+ID4gc3BhY2UgZGlyZWN0bHkuIEp1c3QgTGlr
ZSBLVk0gZGV2aWNlIG1vZGVsIGRvZXMgbm93Lg0KPiA+DQo+ID4gVGhhbmsgeW91LiBRdWlja2x5
LCBsZXQgbWUgbWFrZSBzdXJlIG9uZSB0aGluZzoNCj4gPiBUaGlzIEFQSSBpdHNlbGYgZG9lc24n
dCBkbyBhbnkgbWFwcGluZyBvcGVyYXRpb25zLCByaWdodD8NCj4gPiBTbyBJIHN1cHBvc2UgdGhh
dCB2aXJ0aW8gQkUgZ3Vlc3QgaXMgcmVzcG9uc2libGUgdG8NCj4gPiAxKSBmZXRjaCB0aGUgaW5m
b3JtYXRpb24gYWJvdXQgYWxsIHRoZSBtZW1vcnkgcmVnaW9ucyBpbiBGRSwNCj4gPiAyKSBjYWxs
IHRoaXMgQVBJIHRvIGFsbG9jYXRlIGEgYmlnIGNodW5rIG9mIHVudXNlZCBzcGFjZSBpbiBCRSwN
Cj4gPiAzKSBjcmVhdGUgZ3JhbnQvZm9yZWlnbiBtYXBwaW5ncyBmb3IgRkUgb250byB0aGlzIHJl
Z2lvbihTKQ0KPiA+IGluIHRoZSBpbml0aWFsaXphdGlvbi9jb25maWd1cmF0aW9uIG9mIGVtdWxh
dGVkIHZpcnRpbyBkZXZpY2VzLg0KPiA+DQo+ID4gSXMgdGhpcyB0aGUgd2F5IHRoaXMgQVBJIGlz
IGV4cGVjdGVkIHRvIGJlIHVzZWQ/DQo+ID4gRG9lcyBYZW4gYWxyZWFkeSBoYXMgYW4gaW50ZXJm
YWNlIGZvciAoMSk/DQo+ID4NCj4gDQo+IFRoZXkgYXJlIGRpc2N1c3NpbmcgaW4gdGhhdCB0aHJl
YWQgdG8gZmluZCBhIHByb3BlciB3YXkgdG8gZG8gaXQuDQo+IEJlY2F1c2UgdGhpcyBBUEkgaXMg
Y29tbW9uLCBib3RoIHg4NiBhbmQgQXJtIHNob3VsZCBiZSBjb25zaWRlcmVkLg0KPiANCg0KUGxl
YXNlIGlnbm9yZSBteSBhYm92ZSByZXBseS4gSSBoYWRuJ3Qgc2VlbiBPbGVrc2FuZHIgaGFkIHJl
cGxpZWQNCnRoaXMgcXVlc3Rpb24uIFNvcnJ5IGFib3V0IGl0IQ0KDQo+ID4gLVRha2FoaXJvIEFr
YXNoaQ0KPiA+DQo+ID4gPiBCZWZvcmUgdGhpcyBBUEksIFdoZW4gZGV2aWNlIG1vZGVsIHRvIG1h
cCB3aG9sZSBndWVzdCBtZW1vcnksIHdpbGwNCj4gPiA+IHNldmVyZWx5IGNvbnN1bWUgdGhlIHBo
eXNpY2FsIHBhZ2VzIG9mIERvbS0wL0RvbS1ELg0KPiA+ID4NCj4gPiA+ID4gLVRha2FoaXJvIEFr
YXNoaQ0KPiA+ID4gPg0KPiA+ID4gPg0KPiA+ID4gPiA+ID4gVGhhbmtzLA0KPiA+ID4gPiA+ID4g
LVRha2FoaXJvIEFrYXNoaQ0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gLVRha2FoaXJvIEFr
YXNoaQ0KPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFNwZWNpZmljYWxseSBz
cGVha2luZyBhYm91dCBrdm0tdG9vbCwgSSBoYXZlIGEgY29uY2Vybg0KPiA+IGFib3V0DQo+ID4g
PiA+IGl0cw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGxpY2Vuc2UgdGVybTsgVGFyZ2V0aW5nIGRp
ZmZlcmVudCBoeXBlcnZpc29ycyBhbmQNCj4gPiBkaWZmZXJlbnQNCj4gPiA+ID4gT1NzDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gKHdoaWNoIEkgYXNzdW1lIGluY2x1ZGVzIFJUT1MncyksIHRoZSBy
ZXN1bHRhbnQNCj4gbGlicmFyeQ0KPiA+ID4gPiBzaG91bGQNCj4gPiA+ID4gPiA+IGJlDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gbGljZW5zZSBwZXJtaXNzaXZlIGFuZCBHUEwgZm9yIGt2bS10b29s
IG1pZ2h0IGJlIGFuDQo+ID4gaXNzdWUuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQW55IHRob3Vn
aHRzPw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+IFllcy4gSWYgdXNlciB3YW50IHRvIGltcGxlbWVudCBhIEZyZWVCU0QgZGV2
aWNlIG1vZGVsLA0KPiBidXQNCj4gPiB0aGUNCj4gPiA+ID4gPiA+IHZpcnRpbw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiBsaWJyYXJ5IGlzIEdQTC4gVGhlbiBHUEwgd291bGQgYmUgYSBwcm9ibGVtLiBJ
ZiB3ZSBoYXZlDQo+ID4gPiA+IGFub3RoZXINCj4gPiA+ID4gPiA+IGdvb2QNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gY2FuZGlkYXRlLCBJIGFtIG9wZW4gdG8gaXQuDQo+ID4gPiA+ID4gPiA+ID4gPg0K
PiA+ID4gPiA+ID4gPiA+ID4gSSBoYXZlIHNvbWUgY2FuZGlkYXRlcywgcGFydGljdWxhcmx5IGZv
ciB2cS92cmluZywgaW4gbXkNCj4gPiBtaW5kOg0KPiA+ID4gPiA+ID4gPiA+ID4gKiBPcGVuLUFN
UCwgb3INCj4gPiA+ID4gPiA+ID4gPiA+ICogY29ycmVzcG9uZGluZyBGcmVlLUJTRCBjb2RlDQo+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gSW50ZXJl
c3RpbmcsIEkgd2lsbCBsb29rIGludG8gdGhlbSA6ICkNCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiA+IENoZWVycywNCj4gPiA+ID4gPiA+ID4gPiBXZWkgQ2hlbg0KPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWthc2hpDQo+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiAtVGFrYWhpcm8gQWth
c2hpDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFsxXSBodHRwczovL29wLWxpc3RzLmxpbmFyby5vcmcvcGlwZXJtYWls
L3N0cmF0b3MtDQo+ID4gZGV2LzIwMjEtDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQXVndXN0LzAw
MDU0OC5odG1sDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gWzJdIGh0dHBzOi8vbWFyYy5pbmZvLz9s
PXhlbi0NCj4gZGV2ZWwmbT0xNjIzNzM3NTQ3MDUyMzMmdz0yDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEZy
b206IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc3R5c2hAZ21haWwuY29tPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBTZW50OiAyMDIx5bm0OOaciDE05pelIDIzOjM4DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFRvOiBBS0FTSEkgVGFrYWhpcm8gPHRha2FoaXJvLmFrYXNoaUBsaW5h
cm8ub3JnPjsNCj4gPiA+ID4gU3RlZmFubw0KPiA+ID4gPiA+ID4gPiA+ID4gU3RhYmVsbGluaQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBDYzogQWxleCBCZW5uPz9lIDxhbGV4LmJlbm5lZUBsaW5hcm8ub3Jn
PjsgU3RyYXRvcw0KPiA+ID4gPiBNYWlsaW5nDQo+ID4gPiA+ID4gPiBMaXN0DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPHN0cmF0b3MtZGV2QG9wLWxpc3RzLmxpbmFyby5vcmc+OyB2aXJ0aW8tDQo+
ID4gZGV2QGxpc3RzLm9hc2lzLQ0KPiA+ID4gPiA+ID4gb3Blbi5vcmc7DQo+ID4gPiA+ID4gPiA+
ID4gPiBBcm5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQmVyZ21hbm4gPGFybmQuYmVyZ21hbm5A
bGluYXJvLm9yZz47IFZpcmVzaCBLdW1hcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IDx2aXJlc2gu
a3VtYXJAbGluYXJvLm9yZz47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IDxzdGVmYW5vLnN0YWJlbGxpbmlAeGlsaW54LmNvbT47IHN0ZWZhbmhhQHJlZGhhdC5jb207
DQo+ID4gSmFuDQo+ID4gPiA+IEtpc3prYQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxqYW4ua2lz
emthQHNpZW1lbnMuY29tPjsgQ2FybCB2YW4gU2NoYWlrDQo+ID4gPiA+ID4gPiA8Y3ZhbnNjaGFA
cXRpLnF1YWxjb21tLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gcHJhdGlrcEBxdWljaW5j
LmNvbTsgU3JpdmF0c2EgVmFkZGFnaXJpDQo+ID4gPiA+IDx2YXRzYUBjb2RlYXVyb3JhLm9yZz47
DQo+ID4gPiA+ID4gPiBKZWFuLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFBoaWxpcHBlIEJydWNr
ZXIgPGplYW4tcGhpbGlwcGVAbGluYXJvLm9yZz47IE1hdGhpZXUNCj4gPiBQb2lyaWVyDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPG1hdGhpZXUucG9pcmllckBsaW5hcm8ub3JnPjsgV2VpIENoZW4N
Cj4gPiA8V2VpLkNoZW5AYXJtLmNvbT47DQo+ID4gPiA+ID4gPiBPbGVrc2FuZHINCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBUeXNoY2hlbmtvIDxPbGVrc2FuZHJfVHlzaGNoZW5rb0BlcGFtLmNvbT47
IEJlcnRyYW5kDQo+ID4gTWFycXVpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IDxCZXJ0cmFuZC5N
YXJxdWlzQGFybS5jb20+OyBBcnRlbSBNeWdhaWV2DQo+ID4gPiA+ID4gPiA8QXJ0ZW1fTXlnYWll
dkBlcGFtLmNvbT47DQo+ID4gPiA+ID4gPiA+ID4gPiBKdWxpZW4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+OyBKdWVyZ2VuIEdyb3NzDQo+IDxqZ3Jvc3NAc3Vz
ZS5jb20+Ow0KPiA+ID4gPiBQYXVsDQo+ID4gPiA+ID4gPiBEdXJyYW50DQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPHBhdWxAeGVuLm9yZz47IFhlbiBEZXZlbCA8eGVuLWRldmVsQGxpc3RzLnhlbi5v
cmc+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBFbmFibGluZyBoeXBl
cnZpc29yIGFnbm9zdGljaXNtIGZvcg0KPiA+IFZpcnRJTw0KPiA+ID4gPiA+ID4gYmFja2VuZHMN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSGVs
bG8sIGFsbC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gUGxlYXNlIHNlZSBzb21lIGNvbW1lbnRzIGJlbG93LiBBbmQgc29ycnkgZm9yIHRoZQ0K
PiA+ID4gPiBwb3NzaWJsZQ0KPiA+ID4gPiA+ID4gZm9ybWF0DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gaXNzdWVzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IE9uIFdlZCwgQXVnIDExLCAyMDIxIGF0IDk6MjcgQU0gQUtBU0hJIFRha2FoaXJv
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPG1haWx0bzp0YWthaGlyby5ha2FzaGlAbGluYXJvLm9y
Zz4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT24gV2VkLCBBdWcgMDQsIDIw
MjEgYXQgMTI6MjA6MDFQTSAtMDcwMCwNCj4gU3RlZmFubw0KPiA+ID4gPiA+ID4gU3RhYmVsbGlu
aQ0KPiA+ID4gPiA+ID4gPiA+ID4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBDQ2luZyBwZW9wbGUgd29ya2luZyBvbiBYZW4rVmlydElPIGFuZCBJT1JFUXMuDQo+ID4gTm90
DQo+ID4gPiA+ID4gPiB0cmltbWluZw0KPiA+ID4gPiA+ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gb3JpZ2luYWwNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGVtYWls
IHRvIGxldCB0aGVtIHJlYWQgdGhlIGZ1bGwgY29udGV4dC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBNeSBjb21tZW50cyBiZWxv
dyBhcmUgcmVsYXRlZCB0byBhIHBvdGVudGlhbA0KPiBYZW4NCj4gPiA+ID4gPiA+ID4gPiA+IGlt
cGxlbWVudGF0aW9uLA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG5vdA0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gYmVjYXVzZSBpdCBpcyB0aGUgb25seSBpbXBsZW1lbnRhdGlvbiB0aGF0
DQo+ID4gbWF0dGVycywNCj4gPiA+ID4gYnV0DQo+ID4gPiA+ID4gPiA+ID4gPiBiZWNhdXNlIGl0
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IHRoZSBvbmUgSSBrbm93IGJlc3QuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBQbGVhc2Ugbm90ZSB0aGF0IG15IHByb3Bvc2FsIChhbmQg
aGVuY2UgdGhlDQo+ID4gd29ya2luZw0KPiA+ID4gPiA+ID4gcHJvdG90eXBlKVsxXQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGlzIGJhc2VkIG9uIFhlbidzIHZpcnRpbyBpbXBsZW1lbnRh
dGlvbiAoaS5lLg0KPiA+IElPUkVRKQ0KPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBwYXJ0aWN1bGFybHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBFUEFNJ3MgdmlydGlv
LWRpc2sgYXBwbGljYXRpb24gKGJhY2tlbmQgc2VydmVyKS4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBJdCBoYXMgYmVlbiwgSSBiZWxpZXZlLCB3ZWxsIGdlbmVyYWxpemVkIGJ1dCBpcw0K
PiA+IHN0aWxsDQo+ID4gPiA+IGENCj4gPiA+ID4gPiA+IGJpdA0KPiA+ID4gPiA+ID4gPiA+ID4g
Ymlhc2VkDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG93YXJkIHRoaXMgb3JpZ2luYWwg
ZGVzaWduLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gU28gSSBob3BlIHlvdSBsaWtlIG15IGFwcHJvYWNoIDopDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBbMV0gaHR0cHM6Ly9v
cC0NCj4gbGlzdHMubGluYXJvLm9yZy9waXBlcm1haWwvc3RyYXRvcy0NCj4gPiA+ID4gPiA+IGRl
di8yMDIxLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEF1Z3VzdC8wMDA1NDYuaHRtbA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gTGV0IG1l
IHRha2UgdGhpcyBvcHBvcnR1bml0eSB0byBleHBsYWluIGEgYml0DQo+IG1vcmUNCj4gPiA+ID4g
YWJvdXQNCj4gPiA+ID4gPiA+IG15DQo+ID4gPiA+ID4gPiA+ID4gPiBhcHByb2FjaA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IGJlbG93Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBBbHNvLCBwbGVhc2Ugc2VlIHRoaXMgcmVsZXZhbnQg
ZW1haWwgdGhyZWFkOg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHR0cHM6Ly9tYXJj
LmluZm8vP2w9eGVuLQ0KPiA+IGRldmVsJm09MTYyMzczNzU0NzA1MjMzJnc9Mg0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbiBXZWQsIDQgQXVnIDIwMjEsIEFsZXggQmVubsOp
ZSB3cm90ZToNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSGksDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9u
ZSBvZiB0aGUgZ29hbHMgb2YgUHJvamVjdCBTdHJhdG9zIGlzIHRvDQo+ID4gZW5hYmxlDQo+ID4g
PiA+ID4gPiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYWdub3N0aWMNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYmFja2VuZHMgc28gd2UgY2FuIGVuYWJsZSBhcyBt
dWNoIHJlLXVzZSBvZg0KPiA+IGNvZGUNCj4gPiA+ID4gYXMNCj4gPiA+ID4gPiA+IHBvc3NpYmxl
DQo+ID4gPiA+ID4gPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhdm9pZA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiByZXBlYXRpbmcgb3Vyc2VsdmVzLiBUaGlzIGlz
IHRoZSBmbGlwIHNpZGUNCj4gb2YNCj4gPiB0aGUNCj4gPiA+ID4gPiA+IGZyb250IGVuZA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IHdoZXJlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IG11bHRpcGxlIGZyb250LWVuZCBpbXBsZW1lbnRhdGlvbnMgYXJlDQo+ID4gcmVxdWlyZWQgLQ0K
PiA+ID4gPiBvbmUNCj4gPiA+ID4gPiA+IHBlciBPUywNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBh
c3N1bWluZw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB5b3UgZG9uJ3QganVzdCB3
YW50IExpbnV4IGd1ZXN0cy4gVGhlDQo+ID4gcmVzdWx0YW50DQo+ID4gPiA+IGd1ZXN0cw0KPiA+
ID4gPiA+ID4gYXJlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdHJpdmlhbGx5DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IG1vdmFibGUgYmV0d2VlbiBoeXBlcnZpc29ycyBtb2R1bG8g
YW55DQo+ID4gYWJzdHJhY3RlZA0KPiA+ID4gPiA+ID4gcGFyYXZpcnQNCj4gPiA+ID4gPiA+ID4g
PiA+IHR5cGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlcy4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gSW4gbXkgb3JpZ2luYWwgdGh1bWIgbmFpbCBza2V0Y2ggb2YgYQ0KPiBzb2x1dGlvbg0K
PiA+IEkNCj4gPiA+ID4gPiA+IGVudmlzaW9uZWQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiB2aG9z
dC11c2VyDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGRhZW1vbnMgcnVubmluZyBp
biBhIGJyb2FkbHkgUE9TSVggbGlrZQ0KPiA+ID4gPiBlbnZpcm9ubWVudC4NCj4gPiA+ID4gPiA+
IFRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZSB0bw0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiB0aGUgZGFlbW9uIGlzIGZhaXJseSBzaW1wbGUgcmVxdWlyaW5nIG9u
bHkNCj4gPiBzb21lDQo+ID4gPiA+IG1hcHBlZA0KPiA+ID4gPiA+ID4gPiA+ID4gbWVtb3J5DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gYW5kIHNvbWUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gc29ydCBvZiBzaWduYWxsaW5nIGZvciBldmVudHMgKG9uIExpbnV4IHRoaXMNCj4gPiBp
cw0KPiA+ID4gPiA+ID4gZXZlbnRmZCkuDQo+ID4gPiA+ID4gPiA+ID4gPiBUaGUNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBpZGVhIHdhcyBhDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IHN0dWIgYmluYXJ5IHdvdWxkIGJlIHJlc3BvbnNpYmxlIGZvciBhbnkNCj4gPiA+ID4gaHlwZXJ2
aXNvcg0KPiA+ID4gPiA+ID4gc3BlY2lmaWMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzZXR1cCBh
bmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlbiBsYXVuY2ggYSBjb21tb24g
YmluYXJ5IHRvIGRlYWwgd2l0aCB0aGUNCj4gPiA+ID4gYWN0dWFsDQo+ID4gPiA+ID4gPiA+ID4g
PiB2aXJ0cXVldWUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiByZXF1ZXN0cw0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiB0aGVtc2VsdmVzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBTaW5jZSB0aGF0IG9yaWdp
bmFsIHNrZXRjaCB3ZSd2ZSBzZWVuIGFuDQo+ID4gZXhwYW5zaW9uDQo+ID4gPiA+IGluDQo+ID4g
PiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+IHNvcnQNCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBvZiB3YXlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGJhY2tlbmRzIGNvdWxk
IGJlIGNyZWF0ZWQuIFRoZXJlIGlzIGludGVyZXN0DQo+ID4gaW4NCj4gPiA+ID4gPiA+ID4gPiA+
IGVuY2Fwc3VsYXRpbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kcw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbiBSVE9TZXMgb3IgdW5pa2VybmVscyBmb3Igc29sdXRp
b25zIGxpa2UNCj4gPiBTQ01JLg0KPiA+ID4gPiBUaGVyZQ0KPiA+ID4gPiA+ID4gPiA+ID4gaW50
ZXJlc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbiBSdXN0DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IGhhcyBwcm9tcHRlZCBpZGVhcyBvZiB1c2luZyB0aGUgdHJhaXQNCj4gPiBp
bnRlcmZhY2UgdG8NCj4gPiA+ID4gPiA+IGFic3RyYWN0DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
ZGlmZmVyZW5jZXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYXdheSBhcyB3ZWxs
IGFzIHRoZSBpZGVhIG9mIGJhcmUtbWV0YWwgUnVzdA0KPiA+ID4gPiBiYWNrZW5kcy4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gV2UgaGF2ZSBhIGNhcmQgKFNUUi0xMikgY2FsbGVkICJIeXBlcmNhbGwNCj4gPiA+ID4gPiA+
IFN0YW5kYXJkaXNhdGlvbiINCj4gPiA+ID4gPiA+ID4gPiA+IHdoaWNoDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGNhbGxzIGZvciBhIGRlc2NyaXB0aW9uIG9mIHRoZSBBUElzIG5l
ZWRlZA0KPiA+IGZyb20NCj4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiBoeXBlcnZpc29y
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc2lkZSB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBzdXBwb3J0IFZpcnRJTyBndWVzdHMgYW5kIHRoZWlyIGJhY2tlbmRzLg0KPiA+IEhv
d2V2ZXINCj4gPiA+ID4gd2UNCj4gPiA+ID4gPiA+IGFyZQ0KPiA+ID4gPiA+ID4gPiA+ID4gc29t
ZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHdheSBvZmYNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gZnJvbSB0aGF0IGF0IHRoZSBtb21lbnQgYXMgSSB0aGluayB3ZSBuZWVkDQo+IHRv
DQo+ID4gYXQNCj4gPiA+ID4gbGVhc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkZW1vbnN0cmF0
ZSBvbmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gcG9ydGFibGUgYmFja2VuZCBi
ZWZvcmUgd2Ugc3RhcnQgY29kaWZ5aW5nDQo+ID4gPiA+ID4gPiByZXF1aXJlbWVudHMuIFRvDQo+
ID4gPiA+ID4gPiA+ID4gPiB0aGF0DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZW5kIEkNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gd2FudCB0byB0aGluayBhYm91dCB3aGF0IHdlIG5l
ZWQgZm9yIGENCj4gYmFja2VuZA0KPiA+IHRvDQo+ID4gPiA+ID4gPiBmdW5jdGlvbi4NCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gQ29uZmlndXJhdGlvbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA9PT09PT09
PT09PT09DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IEluIHRoZSB0eXBlLTIgc2V0dXAgdGhpcyBpcyB0eXBpY2FsbHkgZmFp
cmx5DQo+ID4gPiA+IHNpbXBsZQ0KPiA+ID4gPiA+ID4gYmVjYXVzZQ0KPiA+ID4gPiA+ID4gPiA+
ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaG9zdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBzeXN0ZW0gY2FuIG9yY2hlc3RyYXRlIHRoZSB2YXJpb3VzIG1vZHVsZXMNCj4g
PiB0aGF0DQo+ID4gPiA+IG1ha2UNCj4gPiA+ID4gPiA+IHVwIHRoZQ0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IGNvbXBsZXRlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHN5c3RlbS4g
SW4gdGhlIHR5cGUtMSBjYXNlIChvciBldmVuIHR5cGUtMg0KPiA+IHdpdGgNCj4gPiA+ID4gPiA+
IGRlbGVnYXRlZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHNlcnZpY2UgVk1zKQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB3ZSBuZWVkIHNvbWUgc29ydCBvZiBtZWNoYW5pc20gdG8g
aW5mb3JtIHRoZQ0KPiA+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gPiBWTQ0KPiA+ID4gPiA+ID4g
PiA+ID4gYWJvdXQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBrZXkNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gZGV0YWlscyBhYm91dCB0aGUgc3lzdGVtOg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gd2hl
cmUgdmlydCBxdWV1ZSBtZW1vcnkgaXMgaW4gaXQncw0KPiBhZGRyZXNzDQo+ID4gPiA+IHNwYWNl
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgLSBob3cgaXQncyBnb2luZyB0byBy
ZWNlaXZlIChpbnRlcnJ1cHQpDQo+IGFuZA0KPiA+ID4gPiB0cmlnZ2VyDQo+ID4gPiA+ID4gPiAo
a2ljaykNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBldmVudHMNCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gICAtIHdoYXQgKGlmIGFueSkgcmVzb3VyY2VzIHRoZSBiYWNrZW5kDQo+IG5l
ZWRzDQo+ID4gdG8NCj4gPiA+ID4gPiA+IGNvbm5lY3QgdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT2J2aW91c2x5IHlv
dSBjYW4gZWxpZGUgb3ZlciBjb25maWd1cmF0aW9uDQo+ID4gaXNzdWVzDQo+ID4gPiA+IGJ5DQo+
ID4gPiA+ID4gPiBoYXZpbmcNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzdGF0aWMNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY29uZmlndXJhdGlvbnMgYW5kIGJha2luZyB0aGUgYXNz
dW1wdGlvbnMNCj4gaW50bw0KPiA+ID4gPiB5b3VyDQo+ID4gPiA+ID4gPiBndWVzdA0KPiA+ID4g
PiA+ID4gPiA+ID4gaW1hZ2VzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaG93ZXZlcg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGlzIGlzbid0IHNjYWxhYmxlIGluIHRoZSBsb25n
IHRlcm0uIFRoZQ0KPiA+IG9idmlvdXMNCj4gPiA+ID4gPiA+IHNvbHV0aW9uDQo+ID4gPiA+ID4g
PiA+ID4gPiBzZWVtcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIGJlDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGV4dGVuZGluZyBhIHN1YnNldCBvZiBEZXZpY2UgVHJlZSBkYXRh
IHRvDQo+IHVzZXINCj4gPiA+ID4gc3BhY2UNCj4gPiA+ID4gPiA+IGJ1dA0KPiA+ID4gPiA+ID4g
PiA+ID4gcGVyaGFwcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZXJlDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGFyZSBvdGhlciBhcHByb2FjaGVzPw0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBCZWZvcmUg
YW55IHZpcnRpbyB0cmFuc2FjdGlvbnMgY2FuIHRha2UNCj4gcGxhY2UNCj4gPiB0aGUNCj4gPiA+
ID4gPiA+ID4gPiA+IGFwcHJvcHJpYXRlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWVtb3J5DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG1hcHBpbmdzIG5lZWQgdG8gYmUgbWFkZSBi
ZXR3ZWVuIHRoZSBGRQ0KPiBndWVzdA0KPiA+IGFuZA0KPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+
IEJFDQo+ID4gPiA+ID4gPiA+ID4gPiBndWVzdC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEN1cnJlbnRseSB0aGUgd2hvbGUg
b2YgdGhlIEZFIGd1ZXN0cyBhZGRyZXNzDQo+ID4gc3BhY2UNCj4gPiA+ID4gPiA+IG5lZWRzIHRv
DQo+ID4gPiA+ID4gPiA+ID4gPiBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHZpc2libGUNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gd2hhdGV2ZXIgaXMgc2VydmluZyB0aGUg
dmlydGlvIHJlcXVlc3RzLg0KPiBJDQo+ID4gY2FuDQo+ID4gPiA+ID4gPiBlbnZpc2lvbiAzDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gYXBwcm9hY2hlczoNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICogQkUgZ3Vlc3QgYm9v
dHMgd2l0aCBtZW1vcnkgYWxyZWFkeSBtYXBwZWQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIFRoaXMgd291bGQgZW50YWls
IHRoZSBndWVzdCBPUyBrbm93aW5nDQo+IHdoZXJlDQo+ID4gaW4NCj4gPiA+ID4gaXQncw0KPiA+
ID4gPiA+ID4gR3Vlc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBQaHlzaWNhbA0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgQWRkcmVzcyBzcGFjZSBpcyBhbHJlYWR5IHRha2VuIHVw
IGFuZA0KPiA+IGF2b2lkaW5nDQo+ID4gPiA+ID4gPiBjbGFzaGluZy4gSQ0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IHdvdWxkIGFzc3VtZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
aW4gdGhpcyBjYXNlIHlvdSB3b3VsZCB3YW50IGEgc3RhbmRhcmQNCj4gPiBpbnRlcmZhY2UNCj4g
PiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiA+IHVzZXJzcGFjZQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IHRvIHRoZW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIG1ha2UgdGhhdCBh
ZGRyZXNzIHNwYWNlIHZpc2libGUgdG8gdGhlDQo+ID4gYmFja2VuZA0KPiA+ID4gPiBkYWVtb24u
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBZZXQgYW5vdGhlciB3YXkgaGVyZSBpcyB0aGF0IHdlIHdvdWxkIGhhdmUgd2VsbA0KPiA+IGtu
b3duDQo+ID4gPiA+ID4gPiAic2hhcmVkDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWVtb3J5IiBi
ZXR3ZWVuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVk1zLiBJIHRoaW5rIHRoYXQgSmFp
bGhvdXNlJ3MgaXZzaG1lbSBnaXZlcyB1cw0KPiA+IGdvb2QNCj4gPiA+ID4gPiA+IGluc2lnaHRz
IG9uDQo+ID4gPiA+ID4gPiA+ID4gPiB0aGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWF0dGVy
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYW5kIHRoYXQgaXQgY2FuIGV2ZW4gYmUgYW4g
YWx0ZXJuYXRpdmUgZm9yDQo+ID4gaHlwZXJ2aXNvci0NCj4gPiA+ID4gPiA+IGFnbm9zdGljDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gc29sdXRpb24uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAoUGxlYXNlIG5vdGUgbWVtb3J5IHJlZ2lv
bnMgaW4gaXZzaG1lbSBhcHBlYXIgYXMNCj4gYQ0KPiA+IFBDSQ0KPiA+ID4gPiA+ID4gZGV2aWNl
DQo+ID4gPiA+ID4gPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBjYW4gYmUNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBtYXBwZWQgbG9jYWxseS4pDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJIHdhbnQgdG8gYWRk
IHRoaXMgc2hhcmVkIG1lbW9yeSBhc3BlY3QgdG8gbXkNCj4gPiB2aXJ0aW8tDQo+ID4gPiA+IHBy
b3h5LA0KPiA+ID4gPiA+ID4gYnV0DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIHJl
c3VsdGFudCBzb2x1dGlvbiB3b3VsZCBldmVudHVhbGx5IGxvb2sNCj4gPiBzaW1pbGFyDQo+ID4g
PiA+IHRvDQo+ID4gPiA+ID4gPiBpdnNobWVtLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAqIEJFIGd1ZXN0cyBib290cyB3aXRo
IGEgaHlwZXJ2aXNvciBoYW5kbGUNCj4gdG8NCj4gPiA+ID4gbWVtb3J5DQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBUaGUg
QkUgZ3Vlc3QgaXMgdGhlbiBmcmVlIHRvIG1hcCB0aGUgRkUncw0KPiA+IG1lbW9yeQ0KPiA+ID4g
PiB0bw0KPiA+ID4gPiA+ID4gd2hlcmUNCj4gPiA+ID4gPiA+ID4gPiA+IGl0DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gd2FudHMgaW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gIHRo
ZSBCRSdzIGd1ZXN0IHBoeXNpY2FsIGFkZHJlc3Mgc3BhY2UuDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSSBjYW5ub3Qgc2VlIGhv
dyB0aGlzIGNvdWxkIHdvcmsgZm9yIFhlbi4NCj4gVGhlcmUNCj4gPiBpcw0KPiA+ID4gPiBubw0K
PiA+ID4gPiA+ID4gImhhbmRsZSINCj4gPiA+ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gZ2l2ZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gdGhlIGJhY2tl
bmQgaWYgdGhlIGJhY2tlbmQgaXMgbm90IHJ1bm5pbmcNCj4gaW4NCj4gPiBkb20wLg0KPiA+ID4g
PiBTbw0KPiA+ID4gPiA+ID4gZm9yDQo+ID4gPiA+ID4gPiA+ID4gPiBYZW4gSQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IHRoaW5rDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgbWVt
b3J5IGhhcyB0byBiZSBhbHJlYWR5IG1hcHBlZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSW4gWGVuJ3MgSU9SRVEgc29sdXRpb24gKHZp
cnRpby1ibGspLCB0aGUNCj4gPiBmb2xsb3dpbmcNCj4gPiA+ID4gPiA+IGluZm9ybWF0aW9uDQo+
ID4gPiA+ID4gPiA+ID4gPiBpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV4cGVjdGVkDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gYmUgZXhwb3NlZCB0byBCRSB2aWEgWGVuc3RvcmU6
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gKEkga25vdyB0aGF0IHRoaXMgaXMgYSB0ZW50
YXRpdmUgYXBwcm9hY2gNCj4gdGhvdWdoLikNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
ICAtIHRoZSBzdGFydCBhZGRyZXNzIG9mIGNvbmZpZ3VyYXRpb24gc3BhY2UNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiAgICAtIGludGVycnVwdCBudW1iZXINCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiAgICAtIGZpbGUgcGF0aCBmb3IgYmFja2luZyBzdG9yYWdlDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gICAgLSByZWFkLW9ubHkgZmxhZw0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IEFuZCB0aGUgQkUgc2VydmVyIGhhdmUgdG8gY2FsbCBhIHBhcnRpY3VsYXINCj4g
PiBoeXBlcnZpc29yDQo+ID4gPiA+ID4gPiBpbnRlcmZhY2UNCj4gPiA+ID4gPiA+ID4gPiA+IHRv
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWFwIHRoZSBjb25maWd1cmF0aW9uIHNwYWNl
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBZ
ZXMsIFhlbnN0b3JlIHdhcyBjaG9zZW4gYXMgYSBzaW1wbGUgd2F5IHRvIHBhc3MNCj4gPiA+ID4g
PiA+IGNvbmZpZ3VyYXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+IGluZm8gdG8NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiB0aGUgYmFja2VuZCBydW5uaW5nIGluIGEgbm9uLXRvb2xzdGFjayBkb21haW4u
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEkgcmVtZW1iZXIsIHRoZXJlIHdhcyBhIHdpc2gg
dG8gYXZvaWQgdXNpbmcNCj4gWGVuc3RvcmUNCj4gPiBpbg0KPiA+ID4gPiA+ID4gVmlydGlvDQo+
ID4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaXRzZWxmIGlm
IHBvc3NpYmxlLCBzbyBmb3Igbm9uLXRvb2xzdGFjayBkb21haW4sIHRoaXMNCj4gPiBjb3VsZA0K
PiA+ID4gPiBkb25lDQo+ID4gPiA+ID4gPiB3aXRoDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYWRq
dXN0aW5nIGRldmQgKGRhZW1vbiB0aGF0IGxpc3RlbnMgZm9yIGRldmljZXMgYW5kDQo+ID4gbGF1
bmNoZXMNCj4gPiA+ID4gPiA+IGJhY2tlbmRzKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0
byByZWFkIGJhY2tlbmQgY29uZmlndXJhdGlvbiBmcm9tIHRoZSBYZW5zdG9yZQ0KPiA+IGFueXdh
eQ0KPiA+ID4gPiBhbmQNCj4gPiA+ID4gPiA+IHBhc3MgaXQNCj4gPiA+ID4gPiA+ID4gPiA+IHRv
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIGJhY2tlbmQgdmlhIGNvbW1hbmQgbGluZSBhcmd1
bWVudHMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
Pg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gWWVzLCBpbiBjdXJyZW50IFBvQyBjb2RlIHdlJ3Jl
IHVzaW5nIHhlbnN0b3JlIHRvDQo+IHBhc3MNCj4gPiA+ID4gZGV2aWNlDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gY29uZmlndXJhdGlvbi4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFdlIGFsc28g
ZGVzaWduZWQgYSBzdGF0aWMgZGV2aWNlIGNvbmZpZ3VyYXRpb24gcGFyc2UNCj4gPiA+ID4gbWV0
aG9kDQo+ID4gPiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+ID4gPiA+IERvbTBsZXNzDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gb3INCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG90aGVyIHNjZW5hcmlv
cyBkb24ndCBoYXZlIHhlbnRvb2wuIHllcywgaXQncyBmcm9tDQo+ID4gZGV2aWNlDQo+ID4gPiA+
ID4gPiBtb2RlbA0KPiA+ID4gPiA+ID4gPiA+ID4gY29tbWFuZA0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IGxpbmUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG9yIGEgY29uZmlnIGZpbGUuDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBCdXQsIGlmIC4u
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSW4gbXkgYXBwcm9hY2ggKHZpcnRpby1wcm94
eSksIGFsbCB0aG9zZSBYZW4gKG9yDQo+ID4gPiA+ID4gPiBoeXBlcnZpc29yKS0NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBzcGVjaWZpYw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHN0dWZm
cyBhcmUgY29udGFpbmVkIGluIHZpcnRpby1wcm94eSwgeWV0DQo+IGFub3RoZXINCj4gPiBWTSwN
Cj4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+IGhpZGUNCj4gPiA+ID4gPiA+ID4gPiA+IGFsbA0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGRldGFpbHMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IC4uLiB0aGUgc29sdXRpb24gaG93IHRvIG92ZXJjb21l
IHRoYXQgaXMgYWxyZWFkeQ0KPiA+IGZvdW5kDQo+ID4gPiA+IGFuZA0KPiA+ID4gPiA+ID4gcHJv
dmVuDQo+ID4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHdvcmsgdGhl
biBldmVuIGJldHRlci4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiAjIE15IHBvaW50IGlzIHRoYXQgYSAiaGFuZGxlIiBpcyBub3QgbWFuZGF0b3J5
DQo+IGZvcg0KPiA+ID4gPiA+ID4gZXhlY3V0aW5nDQo+ID4gPiA+ID4gPiA+ID4gPiBtYXBwaW5n
Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBhbmQgdGhlIG1hcHBpbmcgcHJvYmFibHkgZG9uZSBieSB0aGUNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHRvb2xzdGFjayAoYWxzbyBzZWUgYmVsb3cuKSBPciB3ZSB3b3VsZCBo
YXZlDQo+IHRvDQo+ID4gPiA+IGludmVudCBhDQo+ID4gPiA+ID4gPiBuZXcNCj4gPiA+ID4gPiA+
ID4gPiA+IFhlbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHlwZXJ2aXNvciBpbnRl
cmZhY2UgYW5kIFhlbiB2aXJ0dWFsIG1hY2hpbmUNCj4gPiA+ID4gcHJpdmlsZWdlcw0KPiA+ID4g
PiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiA+IGFsbG93DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
dGhpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4ga2luZCBvZiBtYXBwaW5nLg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJ
ZiB3ZSBydW4gdGhlIGJhY2tlbmQgaW4gRG9tMCB0aGF0IHdlIGhhdmUgbm8NCj4gPiA+ID4gcHJv
YmxlbXMNCj4gPiA+ID4gPiA+IG9mDQo+ID4gPiA+ID4gPiA+ID4gPiBjb3Vyc2UuDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPbmUgb2Yg
ZGlmZmljdWx0aWVzIG9uIFhlbiB0aGF0IEkgZm91bmQgaW4gbXkNCj4gPiBhcHByb2FjaA0KPiA+
ID4gPiBpcw0KPiA+ID4gPiA+ID4gdGhhdA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNhbGxpbmcN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzdWNoIGh5cGVydmlzb3IgaW50ZWZhY2VzIChy
ZWdpc3RlcmluZyBJT1JFUSwNCj4gPiBtYXBwaW5nDQo+ID4gPiA+ID4gPiBtZW1vcnkpIGlzDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gb25seQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFs
bG93ZWQgb24gQkUgc2VydmVycyB0aGVtc2VsdmllcyBhbmQgc28gd2Ugd2lsbA0KPiA+IGhhdmUN
Cj4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+IGV4dGVuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRo
b3NlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJmYWNlcy4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBUaGlzLCBob3dldmVyLCB3aWxsIHJhaXNlIHNvbWUgY29uY2VybiBv
bg0KPiBzZWN1cml0eQ0KPiA+IGFuZA0KPiA+ID4gPiA+ID4gcHJpdmlsZWdlDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gZGlzdHJpYnV0aW9uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYXMg
U3RlZmFuIHN1Z2dlc3RlZC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gV2UgYWxzbyBmYWNlZCBwb2xpY3kgcmVsYXRlZCBpc3N1ZXMgd2l0aCBW
aXJ0aW8NCj4gPiBiYWNrZW5kDQo+ID4gPiA+ID4gPiBydW5uaW5nIGluDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gb3RoZXIgdGhhbiBEb20wIGRvbWFpbiBpbiBhICJkdW1teSIgeHNtIG1vZGUuIElu
IG91cg0KPiA+IHRhcmdldA0KPiA+ID4gPiA+ID4gc3lzdGVtIHdlDQo+ID4gPiA+ID4gPiA+ID4g
PiBydW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgYmFja2VuZCBpbiBhIGRyaXZlcg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkb21haW4gKHdlIGNhbGwgaXQgRG9tRCkgd2hlcmUgdGhl
IHVuZGVybHlpbmcgSC9XDQo+ID4gPiA+IHJlc2lkZXMuDQo+ID4gPiA+ID4gPiBXZQ0KPiA+ID4g
PiA+ID4gPiA+ID4gdHJ1c3QgaXQsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gc28gd2Ugd3JvdGUg
cG9saWN5IHJ1bGVzICh0byBiZSB1c2VkIGluICJmbGFzayIgeHNtDQo+IG1vZGUpDQo+ID4gdG8N
Cj4gPiA+ID4gPiA+IHByb3ZpZGUNCj4gPiA+ID4gPiA+ID4gPiA+IGl0DQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gd2l0aCBhIGxpdHRsZSBiaXQgbW9yZSBwcml2aWxlZ2VzIHRoYW4gYSBzaW1wbGUg
RG9tVQ0KPiBoYWQuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE5vdyBpdCBpcyBwZXJtaXR0
ZWQgdG8gaXNzdWUgZGV2aWNlLW1vZGVsLA0KPiByZXNvdXJjZQ0KPiA+IGFuZA0KPiA+ID4gPiA+
ID4gbWVtb3J5DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWFwcGluZ3MsIGV0YyBjYWxscy4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBUbyBhY3RpdmF0ZSB0aGUgbWFwcGluZyB3aWxsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICByZXF1aXJlIHNvbWUgc29ydCBvZiBoeXBlcmNhbGwgdG8gdGhlDQo+ID4gaHlw
ZXJ2aXNvci4NCj4gPiA+ID4gSQ0KPiA+ID4gPiA+ID4gY2FuIHNlZQ0KPiA+ID4gPiA+ID4gPiA+
ID4gdHdvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gb3B0aW9ucw0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiAgYXQgdGhpcyBwb2ludDoNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAtIGV4cG9zZSB0aGUgaGFu
ZGxlIHRvIHVzZXJzcGFjZSBmb3INCj4gPiA+ID4gZGFlbW9uL2hlbHBlcg0KPiA+ID4gPiA+ID4g
dG8NCj4gPiA+ID4gPiA+ID4gPiA+IHRyaWdnZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIG1hcHBpbmcgdmlhIGV4aXN0aW5n
IGh5cGVyY2FsbA0KPiBpbnRlcmZhY2VzLg0KPiA+IElmDQo+ID4gPiA+ID4gPiB1c2luZyBhDQo+
ID4gPiA+ID4gPiA+ID4gPiBoZWxwZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiB5b3UNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIHdvdWxkIGhhdmUgYSBoeXBlcnZpc29yIHNw
ZWNpZmljIG9uZSB0bw0KPiA+IGF2b2lkDQo+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gZGFlbW9u
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaGF2aW5nIHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ICAgICBjYXJlIHRvbyBtdWNoIGFib3V0IHRoZSBkZXRhaWxzIG9yIHB1c2gNCj4g
PiB0aGF0DQo+ID4gPiA+ID4gPiBjb21wbGV4aXR5DQo+ID4gPiA+ID4gPiA+ID4gPiBpbnRvDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gYQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAg
ICAgY29tcGlsZSB0aW1lIG9wdGlvbiBmb3IgdGhlIGRhZW1vbiB3aGljaA0KPiA+IHdvdWxkDQo+
ID4gPiA+ID4gPiByZXN1bHQgaW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkaWZmZXJlbnQNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGJpbmFyaWVzIGFsdGhvdWdoIGEgY29t
bW9uIHNvdXJjZSBiYXNlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgIC0gZXhwb3NlIGEgbmV3IGtlcm5lbCBBQkkgdG8g
YWJzdHJhY3QgdGhlDQo+ID4gPiA+IGh5cGVyY2FsbA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGRp
ZmZlcmVuY2VzIGF3YXkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gICAgIGluIHRo
ZSBndWVzdCBrZXJuZWwuIEluIHRoaXMgY2FzZSB0aGUNCj4gPiA+ID4gdXNlcnNwYWNlDQo+ID4g
PiA+ID4gPiB3b3VsZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGVzc2VudGlhbGx5DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICBhc2sgZm9yIGFuIGFic3RyYWN0ICJtYXAgZ3Vl
c3QgTiBtZW1vcnkNCj4gdG8NCj4gPiA+ID4gPiA+IHVzZXJzcGFjZQ0KPiA+ID4gPiA+ID4gPiA+
ID4gcHRyIg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFuZCBsZXQNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gICAgIHRoZSBrZXJuZWwgZGVhbCB3aXRoIHRoZSBkaWZmZXJlbnQNCj4g
PiBoeXBlcmNhbGwNCj4gPiA+ID4gPiA+IGludGVyZmFjZXMuDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gVGhpcyBvZg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgY291cnNlIGFz
c3VtZXMgdGhlIG1ham9yaXR5IG9mIEJFIGd1ZXN0cw0KPiA+IHdvdWxkDQo+ID4gPiA+IGJlDQo+
ID4gPiA+ID4gPiBMaW51eA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGtlcm5lbHMgYW5kDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICBsZWF2ZXMgdGhlIGJhcmUtbWV0YWwvdW5p
a2VybmVsDQo+IGFwcHJvYWNoZXMNCj4gPiB0bw0KPiA+ID4gPiA+ID4gdGhlaXIgb3duDQo+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gZGV2aWNlcy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT3BlcmF0aW9uDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID09PT09PT09PQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGUgY29yZSBvZiB0aGUg
b3BlcmF0aW9uIG9mIFZpcnRJTyBpcw0KPiBmYWlybHkNCj4gPiA+ID4gc2ltcGxlLg0KPiA+ID4g
PiA+ID4gT25jZQ0KPiA+ID4gPiA+ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IHZob3N0LXVzZXIgZmVhdHVyZSBuZWdvdGlhdGlvbiBpcyBkb25lIGl0J3MNCj4g
YQ0KPiA+IGNhc2UNCj4gPiA+ID4gb2YNCj4gPiA+ID4gPiA+ID4gPiA+IHJlY2VpdmluZw0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IHVwZGF0ZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBldmVudHMgYW5kIHBhcnNpbmcgdGhlIHJlc3VsdGFudCB2aXJ0IHF1ZXVlDQo+ID4gZm9yDQo+
ID4gPiA+IGRhdGEuDQo+ID4gPiA+ID4gPiBUaGUNCj4gPiA+ID4gPiA+ID4gPiA+IHZob3N0LQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IHVzZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gc3BlY2lmaWNhdGlvbiBoYW5kbGVzIGEgYnVuY2ggb2Ygc2V0dXANCj4gYmVmb3JlDQo+ID4g
dGhhdA0KPiA+ID4gPiA+ID4gcG9pbnQsDQo+ID4gPiA+ID4gPiA+ID4gPiBtb3N0bHkNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkZXRh
aWwgd2hlcmUgdGhlIHZpcnQgcXVldWVzIGFyZSBzZXQgdXAgRkQncw0KPiA+IGZvcg0KPiA+ID4g
PiA+ID4gbWVtb3J5IGFuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV2ZW50DQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNvbW11bmljYXRpb24uIFRoaXMgaXMgd2hlcmUgdGhlIGVu
dmlzaW9uZWQNCj4gPiBzdHViDQo+ID4gPiA+ID4gPiBwcm9jZXNzDQo+ID4gPiA+ID4gPiA+ID4g
PiB3b3VsZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGJlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IHJlc3BvbnNpYmxlIGZvciBnZXR0aW5nIHRoZSBkYWVtb24gdXAgYW5kDQo+ID4g
cmVhZHkgdG8NCj4gPiA+ID4gcnVuLg0KPiA+ID4gPiA+ID4gVGhpcw0KPiA+ID4gPiA+ID4gPiA+
ID4gaXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY3VycmVudGx5IGRvbmUgaW5z
aWRlIGEgYmlnIFZNTSBsaWtlIFFFTVUNCj4gYnV0DQo+ID4gSQ0KPiA+ID4gPiA+ID4gc3VzcGVj
dCBhDQo+ID4gPiA+ID4gPiA+ID4gPiBtb2Rlcm4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gYXBwcm9hY2ggd291bGQgYmUgdG8gdXNlIHRoZSBydXN0LXZtbSB2aG9zdA0KPiA+IGNy
YXRlLg0KPiA+ID4gPiBJdA0KPiA+ID4gPiA+ID4gd291bGQNCj4gPiA+ID4gPiA+ID4gPiA+IHRo
ZW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBlaXRoZXINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gY29tbXVuaWNhdGUgd2l0aCB0aGUga2VybmVsJ3MgYWJzdHJhY3RlZCBBQkkNCj4g
PiBvciBiZQ0KPiA+ID4gPiByZS0NCj4gPiA+ID4gPiA+ID4gPiA+IHRhcmdldGVkDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gYXMgYQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBidWls
ZCBvcHRpb24gZm9yIHRoZSB2YXJpb3VzIGh5cGVydmlzb3JzLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uZSB0aGluZyBJIG1l
bnRpb25lZCBiZWZvcmUgdG8gQWxleCBpcyB0aGF0DQo+IFhlbg0KPiA+ID4gPiBkb2Vzbid0DQo+
ID4gPiA+ID4gPiBoYXZlDQo+ID4gPiA+ID4gPiA+ID4gPiBWTU1zDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB3YXkgdGhleSBhcmUgdHlw
aWNhbGx5IGVudmlzaW9uZWQgYW5kDQo+IGRlc2NyaWJlZA0KPiA+IGluDQo+ID4gPiA+IG90aGVy
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZW52aXJvbm1lbnRzLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gSW5zdGVhZCwgWGVuIGhhcyBJT1JFUSBzZXJ2ZXJzLiBFYWNoIG9mIHRoZW0N
Cj4gPiA+ID4gY29ubmVjdHMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbmRlcGVuZGVudGx5IHRv
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBYZW4gdmlhIHRoZSBJT1JFUSBpbnRlcmZh
Y2UuIEUuZy4gdG9kYXkNCj4gbXVsdGlwbGUNCj4gPiA+ID4gUUVNVXMNCj4gPiA+ID4gPiA+IGNv
dWxkDQo+ID4gPiA+ID4gPiA+ID4gPiBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHVzZWQgYXMN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGVtdWxhdG9ycyBmb3IgYSBzaW5nbGUgWGVu
IFZNLCBlYWNoIG9mIHRoZW0NCj4gPiA+ID4gY29ubmVjdGluZw0KPiA+ID4gPiA+ID4gdG8gWGVu
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbmRlcGVuZGVudGx5IHZpYSB0aGUgSU9S
RVEgaW50ZXJmYWNlLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IFRoZSBjb21wb25lbnQgcmVzcG9uc2libGUgZm9yIHN0YXJ0aW5n
IGENCj4gZGFlbW9uDQo+ID4gPiA+IGFuZC9vcg0KPiA+ID4gPiA+ID4gc2V0dGluZw0KPiA+ID4g
PiA+ID4gPiA+ID4gdXANCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzaGFyZWQNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZXMgaXMgdGhlIHRvb2xzdGFjazogdGhlIHhsIGNv
bW1hbmQNCj4gYW5kDQo+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiBsaWJ4bC9saWJ4Yw0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbGlicmFyaWVzLg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSSB0aGluayB0aGF0IFZNIGNv
bmZpZ3VyYXRpb24gbWFuYWdlbWVudCAob3INCj4gPiA+ID4gb3JjaGVzdHJhdGlvbg0KPiA+ID4g
PiA+ID4gaW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBTdGFydG9zDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gamFyZ29uPykgaXMgYSBzdWJqZWN0IHRvIGRlYmF0ZSBpbiBwYXJhbGxlbC4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBPdGhlcndpc2UsIGlzIHRoZXJlIGFueSBnb29k
IGFzc3VtcHRpb24gdG8gYXZvaWQNCj4gPiBpdA0KPiA+ID4gPiByaWdodA0KPiA+ID4gPiA+ID4g
bm93Pw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBPbGVrc2FuZHIgYW5kIG90aGVycyBJIENDZWQgaGF2ZSBiZWVuIHdvcmtpbmcNCj4g
b24NCj4gPiA+ID4gd2F5cw0KPiA+ID4gPiA+ID4gZm9yIHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IHRvb2xzdGFjaw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdG8gY3JlYXRlIHZp
cnRpbyBiYWNrZW5kcyBhbmQgc2V0dXAgbWVtb3J5DQo+ID4gbWFwcGluZ3MuDQo+ID4gPiA+ID4g
PiBUaGV5DQo+ID4gPiA+ID4gPiA+ID4gPiBtaWdodCBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+
IGFibGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHByb3ZpZGUgbW9yZSBpbmZv
IG9uIHRoZSBzdWJqZWN0LiBJIGRvDQo+IHRoaW5rDQo+ID4gd2UNCj4gPiA+ID4gbWlzcw0KPiA+
ID4gPiA+ID4gYSB3YXkNCj4gPiA+ID4gPiA+ID4gPiA+IHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gcHJvdmlkZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlIGNvbmZpZ3VyYXRp
b24gdG8gdGhlIGJhY2tlbmQgYW5kIGFueXRoaW5nDQo+ID4gZWxzZQ0KPiA+ID4gPiB0aGF0DQo+
ID4gPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBiYWNrZW5kDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBtaWdodCByZXF1aXJlIHRvIHN0YXJ0IGRvaW5nIGl0cyBqb2Iu
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFll
cywgc29tZSB3b3JrIGhhcyBiZWVuIGRvbmUgZm9yIHRoZSB0b29sc3RhY2sgdG8NCj4gPiBoYW5k
bGUNCj4gPiA+ID4gPiA+IFZpcnRpbw0KPiA+ID4gPiA+ID4gPiA+ID4gTU1JTw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IGRldmljZXMgaW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZ2VuZXJh
bCBhbmQgVmlydGlvIGJsb2NrIGRldmljZXMgaW4gcGFydGljdWxhci4NCj4gPiBIb3dldmVyLA0K
PiA+ID4gPiBpdA0KPiA+ID4gPiA+ID4gaGFzDQo+ID4gPiA+ID4gPiA+ID4gPiBub3QNCj4gPiA+
ID4gPiA+ID4gPiA+ID4gPiBiZWVuIHVwc3RyZWFuZWQgeWV0Lg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiBVcGRhdGVkIHBhdGNoZXMgb24gcmV2aWV3IG5vdzoNCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcveGVuLWRldmVsLzE2MjE2MjYzNjEtDQo+
IDI5MDc2LQ0KPiA+IDEtDQo+ID4gPiA+IGdpdC0NCj4gPiA+ID4gPiA+IHNlbmQtDQo+ID4gPiA+
ID4gPiA+ID4gPiBlbWFpbC0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBvbGVrc3R5c2hAZ21haWwu
Y29tLw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBUaGVyZSBpcyBhbiBhZGRpdGlvbmFsIChhbHNvIGltcG9ydGFudCkgYWN0aXZpdHkNCj4gdG8N
Cj4gPiA+ID4gPiA+IGltcHJvdmUvZml4DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZm9yZWlnbiBt
ZW1vcnkgbWFwcGluZyBvbiBBcm0gd2hpY2ggSSBhbSBhbHNvIGludm9sdmVkDQo+ID4gaW4uDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoZSBmb3JlaWduIG1lbW9yeSBtYXBwaW5nIGlzIHBy
b3Bvc2VkIHRvIGJlIHVzZWQNCj4gPiBmb3INCj4gPiA+ID4gVmlydGlvDQo+ID4gPiA+ID4gPiA+
ID4gPiBiYWNrZW5kcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IChkZXZpY2UgZW11bGF0b3JzKSBp
ZiB0aGVyZSBpcyBhIG5lZWQgdG8gcnVuIGd1ZXN0IE9TDQo+ID4gPiA+IGNvbXBsZXRlbHkNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiB1bm1vZGlmaWVkLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBPZiBjb3Vyc2UsIHRoZSBtb3JlIHNlY3VyZSB3YXkgd291bGQgYmUgdG8gdXNlDQo+IGdyYW50
DQo+ID4gPiA+IG1lbW9yeQ0KPiA+ID4gPiA+ID4gPiA+ID4gbWFwcGluZy4NCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBCcmlldGx5LCB0aGUgbWFpbiBkaWZmZXJlbmNlIGJldHdlZW4gdGhlbSBpcyB0
aGF0IHdpdGgNCj4gPiA+ID4gZm9yZWlnbg0KPiA+ID4gPiA+ID4gbWFwcGluZw0KPiA+ID4gPiA+
ID4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gYmFja2VuZA0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBjYW4gbWFwIGFueSBndWVzdCBtZW1vcnkgaXQgd2FudHMgdG8gbWFwLCBi
dXQgd2l0aA0KPiA+IGdyYW50DQo+ID4gPiA+ID4gPiBtYXBwaW5nDQo+ID4gPiA+ID4gPiA+ID4g
PiBpdCBpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFsbG93ZWQgdG8gbWFwIG9ubHkgd2hhdCB3
YXMgcHJldmlvdXNseSBncmFudGVkIGJ5IHRoZQ0KPiA+ID4gPiBmcm9udGVuZC4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gU28sIHRoZXJlIG1p
Z2h0IGJlIGEgcHJvYmxlbSBpZiB3ZSB3YW50IHRvIHByZS1tYXANCj4gPiBzb21lDQo+ID4gPiA+
ID4gPiBndWVzdA0KPiA+ID4gPiA+ID4gPiA+ID4gbWVtb3J5DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gaW4gYWR2YW5jZSBvciB0byBjYWNoZSBtYXBwaW5ncyBpbiB0aGUgYmFja2VuZCBpbg0KPiBv
cmRlcg0KPiA+IHRvDQo+ID4gPiA+ID4gPiBpbXByb3ZlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
cGVyZm9ybWFuY2UgKGJlY2F1c2UgdGhlIG1hcHBpbmcvdW5tYXBwaW5nIGd1ZXN0IHBhZ2VzDQo+
ID4gZXZlcnkNCj4gPiA+ID4gPiA+IHJlcXVlc3QNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiByZXF1
aXJlcyBhIGxvdCBvZiBiYWNrIGFuZCBmb3J0aCB0byBYZW4gKyBQMk0gdXBkYXRlcykuDQo+ID4g
SW4gYQ0KPiA+ID4gPiA+ID4gbnV0c2hlbGwsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY3VycmVu
dGx5LCBpbiBvcmRlciB0byBtYXAgYSBndWVzdCBwYWdlIGludG8gdGhlDQo+IGJhY2tlbmQNCj4g
PiA+ID4gYWRkcmVzcw0KPiA+ID4gPiA+ID4gc3BhY2UNCj4gPiA+ID4gPiA+ID4gPiA+IHdlDQo+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gbmVlZCB0byBzdGVhbCBhIHJlYWwgcGh5c2ljYWwgcGFnZSBm
cm9tIHRoZSBiYWNrZW5kDQo+ID4gZG9tYWluLg0KPiA+ID4gPiBTbywNCj4gPiA+ID4gPiA+IHdp
dGgNCj4gPiA+ID4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHNhaWQgb3B0
aW1pemF0aW9ucyB3ZSBtaWdodCBlbmQgdXAgd2l0aCBubyBmcmVlIG1lbW9yeQ0KPiA+IGluDQo+
ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gYmFja2VuZA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGRv
bWFpbiAoc2VlIFhTQS0zMDApLiBBbmQgd2hhdCB3ZSB0cnkgdG8gYWNoaWV2ZSBpcyB0bw0KPiA+
IG5vdA0KPiA+ID4gPiB3YXN0ZQ0KPiA+ID4gPiA+ID4gYQ0KPiA+ID4gPiA+ID4gPiA+ID4gcmVh
bA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGRvbWFpbiBtZW1vcnkgYXQgYWxsIGJ5IHByb3ZpZGlu
ZyBzYWZlIG5vbi1hbGxvY2F0ZWQtDQo+IHlldA0KPiA+IChzbw0KPiA+ID4gPiA+ID4gdW51c2Vk
KQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFkZHJlc3Mgc3BhY2UgZm9yIHRoZSBmb3JlaWduIChh
bmQgZ3JhbnQpIHBhZ2VzIHRvIGJlDQo+ID4gbWFwcGVkDQo+ID4gPiA+ID4gPiBpbnRvLA0KPiA+
ID4gPiA+ID4gPiA+ID4gdGhpcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGVuYWJsaW5nIHdvcmsg
aW1wbGllcyBYZW4gYW5kIExpbnV4IChhbmQgbGlrZWx5IERUQg0KPiA+IGJpbmRpbmdzKQ0KPiA+
ID4gPiA+ID4gY2hhbmdlcy4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBIb3dldmVyLCBhcyBpdCB0
dXJuZWQgb3V0LCBmb3IgdGhpcyB0byB3b3JrIGluIGENCj4gcHJvcGVyDQo+ID4gYW5kDQo+ID4g
PiA+IHNhZmUNCj4gPiA+ID4gPiA+IHdheQ0KPiA+ID4gPiA+ID4gPiA+ID4gc29tZQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IHByZXJlcSB3b3JrIG5lZWRzIHRvIGJlIGRvbmUuDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IFlvdSBjYW4gZmluZCB0aGUgcmVsYXRlZCBYZW4gZGlzY3Vzc2lvbiBh
dDoNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcveGVu
LWRldmVsLzE2Mjc0ODkxMTAtDQo+IDI1NjMzLQ0KPiA+IDEtDQo+ID4gPiA+IGdpdC0NCj4gPiA+
ID4gPiA+IHNlbmQtDQo+ID4gPiA+ID4gPiA+ID4gPiBlbWFpbC0NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBvbGVrc3R5c2hAZ21haWwuY29tLw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IE9uZSBxdWVzdGlvbiBpcyBob3cgdG8gYmVzdCBoYW5kbGUNCj4gPiBub3RpZmljYXRpb24N
Cj4gPiA+ID4gYW5kDQo+ID4gPiA+ID4gPiBraWNrcy4NCj4gPiA+ID4gPiA+ID4gPiA+IFRoZQ0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV4aXN0aW5nDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IHZob3N0LXVzZXIgZnJhbWV3b3JrIHVzZXMgZXZlbnRmZCB0byBzaWduYWwNCj4gPiB0
aGUNCj4gPiA+ID4gZGFlbW9uDQo+ID4gPiA+ID4gPiA+ID4gPiAoYWx0aG91Z2gNCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiBRRU1VDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGlzIHF1
aXRlIGNhcGFibGUgb2Ygc2ltdWxhdGluZyB0aGVtIHdoZW4geW91DQo+ID4gdXNlDQo+ID4gPiA+
IFRDRykuDQo+ID4gPiA+ID4gPiBYZW4NCj4gPiA+ID4gPiA+ID4gPiA+IGhhcw0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+IGl0J3Mgb3duDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IElP
UkVRIG1lY2hhbmlzbS4gSG93ZXZlciBsYXRlbmN5IGlzIGFuDQo+ID4gaW1wb3J0YW50DQo+ID4g
PiA+ID4gPiBmYWN0b3IgYW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaGF2aW5nDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV2ZW50cyBnbyB0aHJvdWdoIHRoZSBzdHViIHdvdWxk
IGFkZCBxdWl0ZSBhDQo+ID4gbG90Lg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFllYWggSSB0aGluaywgcmVnYXJkbGVzcyBvZiBh
bnl0aGluZyBlbHNlLCB3ZQ0KPiA+IHdhbnQNCj4gPiA+ID4gdGhlDQo+ID4gPiA+ID4gPiA+ID4g
PiBiYWNrZW5kcyB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gY29ubmVjdCBkaXJl
Y3RseSB0byB0aGUgWGVuIGh5cGVydmlzb3IuDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJbiBteSBhcHByb2FjaCwNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiAgYSkgQkUgLT4gRkU6IGludGVycnVwdHMgdHJpZ2dlcmVkIGJ5IEJF
IGNhbGxpbmcNCj4gYQ0KPiA+ID4gPiA+ID4gaHlwZXJ2aXNvcg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IGludGVyZmFjZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAg
dmlhIHZpcnRpby1wcm94eQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICBiKSBGRSAtPiBC
RTogTU1JTyB0byBjb25maWcgcmFpc2VzIGV2ZW50cyAoaW4NCj4gPiBldmVudA0KPiA+ID4gPiA+
ID4gY2hhbm5lbHMpLA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHdoaWNoIGlzDQo+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgICAgICBjb252ZXJ0ZWQgdG8gYSBjYWxsYmFjayB0
byBCRSB2aWENCj4gPiA+ID4gdmlydGlvLQ0KPiA+ID4gPiA+ID4gcHJveHkNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgICAgIChYZW4ncyBldmVudCBjaGFubmVsIGlzDQo+
IGludGVybm5hbGx5DQo+ID4gPiA+ID4gPiBpbXBsZW1lbnRlZCBieQ0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+IGludGVycnVwdHMuKQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gSSBkb24ndCBrbm93IHdoYXQgImNvbm5lY3QgZGlyZWN0bHki
IG1lYW5zIGhlcmUsDQo+ID4gYnV0DQo+ID4gPiA+ID4gPiBzZW5kaW5nDQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gaW50ZXJydXB0cw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvIHRoZSBv
cHBvc2l0ZSBzaWRlIHdvdWxkIGJlIGJlc3QgZWZmaWNpZW50Lg0KPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+IEl2c2htZW0sIEkgc3VwcG9zZSwgdGFrZXMgdGhpcyBhcHByb2FjaCBieQ0KPiA+
IHV0aWxpemluZw0KPiA+ID4gPiBQQ0kncw0KPiA+ID4gPiA+ID4gbXNpLXgNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBtZWNoYW5pc20uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IEFncmVlIHRoYXQgTVNJIHdvdWxkIGJlIG1vcmUgZWZmaWNpZW50
IHRoYW4gU1BJLi4uDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEF0IHRoZSBtb21lbnQsIGlu
IG9yZGVyIHRvIG5vdGlmeSB0aGUgZnJvbnRlbmQsDQo+IHRoZQ0KPiA+ID4gPiBiYWNrZW5kDQo+
ID4gPiA+ID4gPiBpc3N1ZXMNCj4gPiA+ID4gPiA+ID4gPiA+IGENCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiBzcGVjaWZpYyBkZXZpY2UtbW9kZWwgY2FsbCB0byBxdWVyeSBYZW4gdG8gaW5qZWN0IGEN
Cj4gPiA+ID4gPiA+IGNvcnJlc3BvbmRpbmcgU1BJDQo+ID4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBndWVzdC4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IENvdWxkIHdlIGNvbnNpZGVyIHRoZSBrZXJuZWwgaW50ZXJuYWxseQ0KPiA+IGNvbnZlcnRp
bmcNCj4gPiA+ID4gPiA+IElPUkVRDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWVzc2FnZXMgZnJv
bQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB0aGUgWGVuIGh5cGVydmlzb3IgdG8g
ZXZlbnRmZCBldmVudHM/IFdvdWxkDQo+ID4gdGhpcw0KPiA+ID4gPiBzY2FsZQ0KPiA+ID4gPiA+
ID4gd2l0aA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IG90aGVyIGtlcm5lbA0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiBoeXBlcmNhbGwgaW50ZXJmYWNlcz8NCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gU28gYW55
IHRob3VnaHRzIG9uIHdoYXQgZGlyZWN0aW9ucyBhcmUgd29ydGgNCj4gPiA+ID4gPiA+IGV4cGVy
aW1lbnRpbmcNCj4gPiA+ID4gPiA+ID4gPiA+IHdpdGg/DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT25lIG9wdGlvbiB3ZSBzaG91
bGQgY29uc2lkZXIgaXMgZm9yIGVhY2gNCj4gPiBiYWNrZW5kIHRvDQo+ID4gPiA+ID4gPiBjb25u
ZWN0DQo+ID4gPiA+ID4gPiA+ID4gPiB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFhlbiB2aWEN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBJT1JFUSBpbnRlcmZhY2UuIFdlIGNv
dWxkIGdlbmVyYWxpemUgdGhlDQo+ID4gSU9SRVENCj4gPiA+ID4gPiA+IGludGVyZmFjZQ0KPiA+
ID4gPiA+ID4gPiA+ID4gYW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWFrZSBpdA0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaHlwZXJ2aXNvciBhZ25vc3RpYy4gVGhlIGludGVyZmFj
ZSBpcyByZWFsbHkNCj4gPiB0cml2aWFsDQo+ID4gPiA+IGFuZA0KPiA+ID4gPiA+ID4gZWFzeQ0K
PiA+ID4gPiA+ID4gPiA+ID4gdG8NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhZGQuDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBBcyBJIHNh
aWQgYWJvdmUsIG15IHByb3Bvc2FsIGRvZXMgdGhlIHNhbWUgdGhpbmcNCj4gPiB0aGF0DQo+ID4g
PiA+IHlvdQ0KPiA+ID4gPiA+ID4gPiA+ID4gbWVudGlvbmVkDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gaGVyZSA6KQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoZSBkaWZmZXJlbmNlIGlz
IHRoYXQgSSBkbyBjYWxsIGh5cGVydmlzb3INCj4gPiBpbnRlcmZhY2VzDQo+ID4gPiA+IHZpYQ0K
PiA+ID4gPiA+ID4gPiA+ID4gdmlydGlvLQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHByb3h5Lg0K
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBUaGUgb25seSBYZW4tc3BlY2lmaWMgcGFydCBpcyB0aGUgbm90aWZpY2F0aW9uDQo+ID4gPiA+
IG1lY2hhbmlzbSwNCj4gPiA+ID4gPiA+ID4gPiA+IHdoaWNoIGlzDQo+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gYW4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGV2ZW50IGNoYW5uZWwuIElm
IHdlIHJlcGxhY2VkIHRoZSBldmVudA0KPiBjaGFubmVsDQo+ID4gd2l0aA0KPiA+ID4gPiA+ID4g
c29tZXRoaW5nDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZWxzZSB0aGUNCj4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZSB3b3VsZCBiZSBnZW5lcmljLiBTZWU6DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRwczovL2dpdGxhYi5jb20veGVuLXByb2plY3QveGVu
Ly0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiAvYmxvYi9zdGFnaW5nL3hlbi9pbmNsdWRlL3B1Ymxp
Yy9odm0vaW9yZXEuaCNMNTINCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJIGRvbid0IHRoaW5rIHRoYXQgdHJhbnNsYXRpbmcgSU9S
RVFzIHRvDQo+IGV2ZW50ZmQNCj4gPiBpbg0KPiA+ID4gPiB0aGUNCj4gPiA+ID4gPiA+IGtlcm5l
bA0KPiA+ID4gPiA+ID4gPiA+ID4gaXMNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBnb29kIGlkZWE6IGlmIGZlZWxzIGxpa2UgaXQgd291bGQg
YmUgZXh0cmENCj4gPiA+ID4gY29tcGxleGl0eQ0KPiA+ID4gPiA+ID4gYW5kIHRoYXQNCj4gPiA+
ID4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4ga2VybmVsIHNo
b3VsZG4ndCBiZSBpbnZvbHZlZCBhcyB0aGlzIGlzIGENCj4gPiBiYWNrZW5kLQ0KPiA+ID4gPiA+
ID4gaHlwZXJ2aXNvcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZS4NCj4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEdpdmVuIHRo
YXQgd2UgbWF5IHdhbnQgdG8gaW1wbGVtZW50IEJFIGFzIGENCj4gYmFyZS0NCj4gPiBtZXRhbA0K
PiA+ID4gPiA+ID4gPiA+ID4gYXBwbGljYXRpb24NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiBhcyBJIGRpZCBvbiBaZXBoeXIsIEkgZG9uJ3QgdGhpbmsgdGhhdCB0aGUNCj4gPiB0cmFuc2xh
dGlvbg0KPiA+ID4gPiA+ID4gd291bGQgbm90DQo+ID4gPiA+ID4gPiA+ID4gPiBiZQ0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IGEgYmlnIGlzc3VlLCBlc3BlY2lhbGx5IG9uIFJUT1Mncy4N
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJdCB3aWxsIGJlIHNvbWUga2luZCBvZiBhYnN0
cmFjdGlvbiBsYXllciBvZg0KPiA+IGludGVycnVwdA0KPiA+ID4gPiA+ID4gaGFuZGxpbmcNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiAob3Igbm90aGluZyBidXQgYSBjYWxsYmFjayBtZWNo
YW5pc20pLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBBbHNvLCBldmVudGZkIGlzIHZlcnkgTGludXgtY2VudHJpYyBhbmQgd2UgYXJl
DQo+ID4gPiA+IHRyeWluZyB0bw0KPiA+ID4gPiA+ID4gPiA+ID4gZGVzaWduIGFuDQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbnRlcmZhY2UgdGhhdCBjb3VsZCB3b3JrIHdlbGwgZm9y
IFJUT1NlcyB0b28uDQo+ID4gSWYgd2UNCj4gPiA+ID4gPiA+IHdhbnQgdG8NCj4gPiA+ID4gPiA+
ID4gPiA+IGRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzb21ldGhpbmcgZGlmZmVy
ZW50LCBib3RoIE9TLWFnbm9zdGljIGFuZA0KPiA+ID4gPiBoeXBlcnZpc29yLQ0KPiA+ID4gPiA+
ID4gYWdub3N0aWMsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gcGVyaGFwcw0KPiA+ID4gPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gd2UgY291bGQgZGVzaWduIGEgbmV3IGludGVyZmFjZS4gT25lIHRo
YXQNCj4gY291bGQNCj4gPiBiZQ0KPiA+ID4gPiA+ID4gPiA+ID4gaW1wbGVtZW50YWJsZQ0KPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IGluIHRoZQ0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
WGVuIGh5cGVydmlzb3IgaXRzZWxmIChsaWtlIElPUkVRKSBhbmQgb2YNCj4gY291cnNlDQo+ID4g
YW55DQo+ID4gPiA+ID4gPiBvdGhlcg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGh5cGVydmlzb3IN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRvby4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gVGhlcmUgaXMgYWxzbyBhbm90aGVyIHByb2JsZW0uIElPUkVRIGlzDQo+
IHByb2JhYmx5DQo+ID4gbm90DQo+ID4gPiA+IGJlDQo+ID4gPiA+ID4gPiB0aGUNCj4gPiA+ID4g
PiA+ID4gPiA+IG9ubHkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGludGVyZmFjZSBu
ZWVkZWQuIEhhdmUgYSBsb29rIGF0DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBodHRw
czovL21hcmMuaW5mby8/bD14ZW4tDQo+ID4gZGV2ZWwmbT0xNjIzNzM3NTQ3MDUyMzMmdz0yLg0K
PiA+ID4gPiA+ID4gRG9uJ3Qgd2UNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBhbHNvIG5lZWQNCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGFuIGludGVyZmFjZSBmb3IgdGhlIGJhY2tlbmQg
dG8gaW5qZWN0DQo+ID4gaW50ZXJydXB0cw0KPiA+ID4gPiBpbnRvDQo+ID4gPiA+ID4gPiB0aGUN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBmcm9udGVuZD8gQW5kDQo+ID4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiBpZiB0aGUgYmFja2VuZCByZXF1aXJlcyBkeW5hbWljIG1lbW9yeQ0KPiBtYXBw
aW5ncw0KPiA+IG9mDQo+ID4gPiA+ID4gPiBmcm9udGVuZA0KPiA+ID4gPiA+ID4gPiA+ID4gcGFn
ZXMsDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhlbg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gd2Ugd291bGQgYWxzbyBuZWVkIGFuIGludGVyZmFjZSB0byBtYXAvdW5tYXANCj4gPiBk
b21VDQo+ID4gPiA+IHBhZ2VzLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gTXkgcHJvcG9zYWwgZG9jdW1lbnQgbWlnaHQgaGVscCBoZXJl
OyBBbGwgdGhlDQo+ID4gPiA+IGludGVyZmFjZXMNCj4gPiA+ID4gPiA+ID4gPiA+IHJlcXVpcmVk
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZm9yDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
dmlydGlvLXByb3h5IChvciBoeXBlcnZpc29yLXJlbGF0ZWQgaW50ZXJmYWNlcykNCj4gPiBhcmUN
Cj4gPiA+ID4gbGlzdGVkDQo+ID4gPiA+ID4gPiBhcw0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+IFJQQyBwcm90b2NvbHMgOikNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gVGhlc2UgaW50ZXJmYWNlcyBhcmUgYSBsb3QgbW9yZSBw
cm9ibGVtYXRpYw0KPiB0aGFuDQo+ID4gPiA+IElPUkVROg0KPiA+ID4gPiA+ID4gSU9SRVENCj4g
PiA+ID4gPiA+ID4gPiA+IGlzDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGlueQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+ID4gYW5kIHNlbGYtY29udGFpbmVkLiBJdCBpcyBlYXN5IHRvIGFk
ZCBhbnl3aGVyZS4NCj4gPiBBDQo+ID4gPiA+IG5ldw0KPiA+ID4gPiA+ID4gPiA+ID4gaW50ZXJm
YWNlIHRvDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBpbmplY3QgaW50ZXJydXB0cyBv
ciBtYXAgcGFnZXMgaXMgbW9yZQ0KPiBkaWZmaWN1bHQNCj4gPiB0bw0KPiA+ID4gPiA+ID4gbWFu
YWdlDQo+ID4gPiA+ID4gPiA+ID4gPiBiZWNhdXNlDQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gaXQN
Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IHdvdWxkIHJlcXVpcmUgY2hhbmdlcyBzY2F0
dGVyZWQgYWNyb3NzIHRoZQ0KPiA+IHZhcmlvdXMNCj4gPiA+ID4gPiA+IGVtdWxhdG9ycy4NCj4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEV4
YWN0bHkuIEkgaGF2ZSBubyBjb25maWRlbnQgeWV0IHRoYXQgbXkNCj4gYXBwcm9hY2gNCj4gPiB3
aWxsDQo+ID4gPiA+ID4gPiBhbHNvDQo+ID4gPiA+ID4gPiA+ID4gPiBhcHBseQ0KPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gPiA+IHRvIG90aGVyIGh5cGVydmlzb3JzIHRoYW4gWGVuLg0KPiA+ID4g
PiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRlY2huaWNhbGx5LCB5ZXMsIGJ1dCB3aGV0aGVyIHBlb3Bs
ZSBjYW4gYWNjZXB0DQo+IGl0DQo+ID4gb3INCj4gPiA+ID4gbm90DQo+ID4gPiA+ID4gPiBpcyBh
DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gZGlmZmVyZW50DQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gbWF0dGVyLg0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4gVGhhbmtzLA0KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IC1UYWth
aGlybyBBa2FzaGkNCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4g
PiA+ID4gLS0NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gUmVnYXJkcywNCj4gPiA+ID4gPiA+
ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gT2xla3NhbmRyIFR5c2hj
aGVua28NCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:08:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:08:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176793.321744 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLc9m-0002zQ-SJ; Thu, 02 Sep 2021 02:08:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176793.321744; Thu, 02 Sep 2021 02:08:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLc9m-0002zJ-Mq; Thu, 02 Sep 2021 02:08:46 +0000
Received: by outflank-mailman (input) for mailman id 176793;
 Thu, 02 Sep 2021 02:08:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLc9l-0002zD-Sm
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 02:08:45 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.7])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8b12e6f6-cf9a-4990-a0f6-441daf1a528f;
 Thu, 02 Sep 2021 02:08:42 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by CO2PR21MB3009.namprd21.prod.outlook.com (2603:10b6:102:18::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4; Thu, 2 Sep
 2021 02:08:38 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 02:08:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b12e6f6-cf9a-4990-a0f6-441daf1a528f
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IUpSChkKBWMqyiukbEC8IxJNip992xfSXG5TmpQZhkTvCM/aDZMzscsPteLJxO9Z+5nnAwfCHVRJ2W0YK/xFtjDjwHurm+0Uoqtr6zVSPfMdiMi6cSkgOZkktBCMP8rEmFLNUaWJ2Dhp7THgrZYlneAqfo9iUKClaU0aHvP1AmlX3TXvz93tpYlHU/qOF6/hlZFIwPYBV7ek4LskPSoO3CPFQvILuX9vbB3Zbp+hxpChe0+AufmBZ5o6B+FBxq9NEGICdotV83xIIWBC64c6PrUYihGmPpQEwMmJhcJtqBnnwvbmXJY6CGbXFBAY1rdoHC5K+nZ+RhLtZij0t2NT7Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=m8Z4zZgDo/ng30TsOukCORL2mWs0iQjFTdzvLSYEXk8=;
 b=j3uJ/vgN2Svy1/0rRfkdPjkX3676/Ve/pCQkW24hZX2cQlqfjygR2sWm6x9pTpyd/XcmeLGdWLTF+9XdRWXSrHmrmRKXNRiIpyX8sH81E5g2zkeOIQqEanzEwbrslp+dKshg/BmJ/bhCENBW4Vz9E1KcigFSaLX90cGresqjzboZ4X1nTRmGBYJQnQDEfcQbAdE6tGxM1l7KMRhs43sQkkktASolHwV/KUNZ4JC07adFBSl020syB2TBtyEkp/uLGjdjYd5Ds9bgq9aDxTDkXVrAReL86MfBx9lPiT8heE/mPlRzU24k276JY4geouUHknOK91hgN4/lTs/Zk4rzWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=m8Z4zZgDo/ng30TsOukCORL2mWs0iQjFTdzvLSYEXk8=;
 b=bd/JOGRzT95HRhvOUNcHOfelq5cct9xFQv9lwJCNx7oSiw34v9MsYGLbyVgS39nixvvJ8lcvmPh920F5WnTGKtg3zCvRIW/DKiesmXiXZ80gITl9JyjYcJHTfA5qIhgnqZS859ZT4dw1V+ArSVbyybPU+uOdhKiAKbZdBI1rOf8=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 13/13] hv_storvsc: Add Isolation VM support for storvsc
 driver
Thread-Topic: [PATCH V4 13/13] hv_storvsc: Add Isolation VM support for
 storvsc driver
Thread-Index: AQHXm2gLgD2LnlrK1Ui6MWEQREt+OKuIA2QQ
Date: Thu, 2 Sep 2021 02:08:38 +0000
Message-ID:
 <MWHPR21MB1593CB6AD9521190CA0AB0DED7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-14-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-14-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=f3fb0eb4-3fe0-4fc3-9520-c93e315d7a42;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-27T23:41:48Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6ea2dd65-3c0f-43cb-0521-08d96db693fe
x-ms-traffictypediagnostic: CO2PR21MB3009:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <CO2PR21MB30092A47D1401DE9FFD4B0BDD7CE9@CO2PR21MB3009.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2449;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 ndTLqZdHuCSzY96whS6o3zQp5pWBl/+LPOxouGx7MvdNaYKuKvaFNQwUMEMZfwmHr4jOi6oWBYiz3uNN1AIyg7MhnWomwEL0ihAtK7DvmJFI1pwN23oEItvoA9pfELcTzjHqzto7bCclgTX/tL8m2FVSWS9DznLZ6SL5bED36RZkXCgWoZ61O2xsYrmNK2BYGQJZqTbGKL8JnYl6a1VckOlhVdTcP0kloEzheeMPbCao6ltZge2w3RTnxLv22Ap1BXZKd5XelSzST1sQ2afXyEbqeK+gHzbQeItQFBGpcdjBoy4bU1Ou2Bzw2Ywm9ZmzAgYWhsQCsjwaUgtTmad6fEEO8kOVFKE3mNsXM/zB9yvCObJ1w3dqn7fesYfaweLGMdsWkx80wNt87AjQS+VuaML7txbgbnsj9Sh1pTWE8BQru7dwe3i+e4HyJQSi9RzvoqBvVNbd+n2Wn64j2hrkR7K0QiRW25kzFfeWGwxqIrxhyQ00KRYF8fVPVWDn1zrJUc2GjpQdJuUvR6pQ0ibrmPBeMJgLnb2KDAC0KCTMtEsB1uXDSF1BfkvUnPbYlQv3iq1KsDsPYIHFz7oKMnTKE2KvuGVD8tYcxf/YQdJAN3jNDLQWWp1ORhABKivtbremVstJAU2ihezQYo77y3CPHdlF81y59pgSoKUtaskzqrOzL8EWJtkpNf0UmCBSDzCuYTmwjdSI/nxE35dzdARuaGYoiP+gfuPjhNpq6jMqsI8=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(82950400001)(82960400001)(4326008)(186003)(76116006)(316002)(8990500004)(110136005)(66476007)(83380400001)(7406005)(7416002)(7366002)(66946007)(66556008)(64756008)(66446008)(54906003)(6506007)(9686003)(55016002)(122000001)(26005)(2906002)(52536014)(921005)(86362001)(71200400001)(5660300002)(38070700005)(7696005)(8676002)(8936002)(38100700002)(508600001)(33656002)(10290500003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?DolnhatuDAmIAaAz1/YX4Nu77VLxLJi5Z5u3+A6UHOQkEZWqwmLzr6hkf5h6?=
 =?us-ascii?Q?mVAM7sfILM0sb62t/FIdI2R41We7PR+TF1q7yPx6JbgXnRJQjOGD7e6VAlL0?=
 =?us-ascii?Q?VYlx3oYPMhMWiZoMNmNhSz2nfZTsTT/s8eRHAg5yZtQHCyQJGnBRXvLo2ZT0?=
 =?us-ascii?Q?zllaDQdkxQxV7ukn+Bt6fJgbkj8n2gr1b/uO4SQXC7NISFP1EyxCgQ6JQ7Fc?=
 =?us-ascii?Q?jV4VmCJP8+JQNgbqAyardqPmpig/GBcw/1UqOeybTfbfoSVxOsatsZERTnP5?=
 =?us-ascii?Q?UxSKKL6MuvTM5sTNREAIy2bYLBDu4U0OhiPvTlUJpnYMfyXYNKH267zIFzCM?=
 =?us-ascii?Q?PldhdohAzJfsYI0h6w2kOSo7q8IBiuWm+zf+aaRpA9PB7MlLaixaNGZcU4lE?=
 =?us-ascii?Q?gjYWnJJoud5x0cHcwyYxq8sUexK1uweGy3v6UEKjsTVj/m4FyZ2vXqirFf4H?=
 =?us-ascii?Q?JkbMuFrXswddWR1K8OpAK1204L0MPy5x9yuzN7+maWBfL+enrzX3W+tzdxs4?=
 =?us-ascii?Q?f/1SdBIJEDEM3OYuR9P/7zhpNhHDIL++WxG8hrWBN7w3IkOCRGlCO0rUW/hX?=
 =?us-ascii?Q?E1kEuquD99AvKjJK+AGHbABnEfuVNY3viHqFns4Yx0N3O29ES8uVUvwVSY/q?=
 =?us-ascii?Q?+l3hpQlfW1lBAVS/zySdayy9k4GLK6ubKQfh9vHhnx7ccUqzzuw5w18Pgs3W?=
 =?us-ascii?Q?VRTWWM2CMj5YA1zbaCKGqvuXdOyY0XYRog8BcGfUdaczLPsyPQCLGNqwQGU2?=
 =?us-ascii?Q?AqEnxtXliDQDWBFJSrNj/c1k911nSNH4BGyPKYlT4zHgcDL+97I4sJ2OoBp9?=
 =?us-ascii?Q?KyBj4U1Du4CbFvWCafngd6ltxm1/GKtYyAEZrP7Qya5gxRfabpq2pmyMcAmf?=
 =?us-ascii?Q?lbXmuOj9wMIO2kVGMbHqdvd2octK5nt3V+7yNs2wI876Xq7lDMRF2Gee7USb?=
 =?us-ascii?Q?ZpRrKueEWAIPyLJKQ26Kj/f8msXqtX6JMPvdt0iunoCn5X0YJwFET5D9eDn8?=
 =?us-ascii?Q?GWtR1tpnTxey2vk4daEsB2DBReKe8Hj20zwcaezjVCEwUo7/evMJ4lxdeSv2?=
 =?us-ascii?Q?9pmAy+NcK9mp0y78xdFmLKyXxk82K2vRtiOA7SxTlAFsleQE4DHwPMte4tfC?=
 =?us-ascii?Q?kN+9LxsXMpZ8Z1qM7k5Pf/nedS/xTbiRCsjqmXAwwMjgPtQKTgPVY3xLtLjT?=
 =?us-ascii?Q?bybW5G1aOMa+Vf7yWoRXHxa/Ha4Sans5gAcg2zsEFAIyoTy4qGFChbOT0mGz?=
 =?us-ascii?Q?CiPjKJ3SsfvxpRmVUTLDus/nOnN6kdsW79nDuLhkSL3No277Wu/5FMMUS18+?=
 =?us-ascii?Q?+15h35tJ4UXVtGV1HnHg+BNj?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ea2dd65-3c0f-43cb-0521-08d96db693fe
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 02:08:38.0541
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tXT0bjcyirvjVG/opzOIdPJBrRMiRaNV9Y9VlcCS+iP2p9aRelS7uwRI97sbYAJCtJayrUXtfu+t7qPz3kutGOq/HYyGfA8POko7JlWeOqw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR21MB3009

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20

Per previous comment, the Subject line tag should be "scsi: storvsc: "

> In Isolation VM, all shared memory with host needs to mark visible
> to host via hvcall. vmbus_establish_gpadl() has already done it for
> storvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
> mpb_desc() still needs to be handled. Use DMA API(dma_map_sg) to map
> these memory during sending/receiving packet and return swiotlb bounce
> buffer dma address. In Isolation VM, swiotlb  bounce buffer is marked
> to be visible to host and the swiotlb force mode is enabled.
>=20
> Set device's dma min align mask to HV_HYP_PAGE_SIZE - 1 in order to
> keep the original data offset in the bounce buffer.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Rplace dma_map_page with dma_map_sg()
> 	* Use for_each_sg() to populate payload->range.pfn_array.
> 	* Remove storvsc_dma_map macro
> ---
>  drivers/hv/vmbus_drv.c     |  1 +
>  drivers/scsi/storvsc_drv.c | 41 +++++++++++++++-----------------------
>  include/linux/hyperv.h     |  1 +
>  3 files changed, 18 insertions(+), 25 deletions(-)
>=20
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index f068e22a5636..270d526fd9de 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -2124,6 +2124,7 @@ int vmbus_device_register(struct hv_device *child_d=
evice_obj)
>  	hv_debug_add_dev_dir(child_device_obj);
>=20
>  	child_device_obj->device.dma_mask =3D &vmbus_dma_mask;
> +	child_device_obj->device.dma_parms =3D &child_device_obj->dma_parms;
>  	return 0;
>=20
>  err_kset_unregister:
> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> index 328bb961c281..4f1793be1fdc 100644
> --- a/drivers/scsi/storvsc_drv.c
> +++ b/drivers/scsi/storvsc_drv.c
> @@ -21,6 +21,8 @@
>  #include <linux/device.h>
>  #include <linux/hyperv.h>
>  #include <linux/blkdev.h>
> +#include <linux/dma-mapping.h>
> +
>  #include <scsi/scsi.h>
>  #include <scsi/scsi_cmnd.h>
>  #include <scsi/scsi_host.h>
> @@ -1312,6 +1314,9 @@ static void storvsc_on_channel_callback(void *conte=
xt)
>  					continue;
>  				}
>  				request =3D (struct storvsc_cmd_request *)scsi_cmd_priv(scmnd);
> +				if (scsi_sg_count(scmnd))
> +					dma_unmap_sg(&device->device, scsi_sglist(scmnd),
> +						     scsi_sg_count(scmnd), scmnd->sc_data_direction);

Use scsi_dma_unmap(), which does exactly what you have written
above. :-)

>  			}
>=20
>  			storvsc_on_receive(stor_device, packet, request);
> @@ -1725,7 +1730,6 @@ static int storvsc_queuecommand(struct Scsi_Host *h=
ost, struct scsi_cmnd *scmnd)
>  	struct hv_host_device *host_dev =3D shost_priv(host);
>  	struct hv_device *dev =3D host_dev->dev;
>  	struct storvsc_cmd_request *cmd_request =3D scsi_cmd_priv(scmnd);
> -	int i;
>  	struct scatterlist *sgl;
>  	unsigned int sg_count;
>  	struct vmscsi_request *vm_srb;
> @@ -1807,10 +1811,11 @@ static int storvsc_queuecommand(struct Scsi_Host =
*host, struct scsi_cmnd *scmnd)
>  	payload_sz =3D sizeof(cmd_request->mpb);
>=20
>  	if (sg_count) {
> -		unsigned int hvpgoff, hvpfns_to_add;
>  		unsigned long offset_in_hvpg =3D offset_in_hvpage(sgl->offset);
>  		unsigned int hvpg_count =3D HVPFN_UP(offset_in_hvpg + length);
> -		u64 hvpfn;
> +		struct scatterlist *sg;
> +		unsigned long hvpfn, hvpfns_to_add;
> +		int j, i =3D 0;
>=20
>  		if (hvpg_count > MAX_PAGE_BUFFER_COUNT) {
>=20
> @@ -1824,31 +1829,16 @@ static int storvsc_queuecommand(struct Scsi_Host =
*host, struct scsi_cmnd *scmnd)
>  		payload->range.len =3D length;
>  		payload->range.offset =3D offset_in_hvpg;
>=20
> +		if (dma_map_sg(&dev->device, sgl, sg_count,
> +		    scmnd->sc_data_direction) =3D=3D 0)
> +			return SCSI_MLQUEUE_DEVICE_BUSY;
>=20
> -		for (i =3D 0; sgl !=3D NULL; sgl =3D sg_next(sgl)) {
> -			/*
> -			 * Init values for the current sgl entry. hvpgoff
> -			 * and hvpfns_to_add are in units of Hyper-V size
> -			 * pages. Handling the PAGE_SIZE !=3D HV_HYP_PAGE_SIZE
> -			 * case also handles values of sgl->offset that are
> -			 * larger than PAGE_SIZE. Such offsets are handled
> -			 * even on other than the first sgl entry, provided
> -			 * they are a multiple of PAGE_SIZE.
> -			 */

Any reason not to keep this comment?  It's still correct and
mentions important cases that must be handled.

> -			hvpgoff =3D HVPFN_DOWN(sgl->offset);
> -			hvpfn =3D page_to_hvpfn(sg_page(sgl)) + hvpgoff;
> -			hvpfns_to_add =3D	HVPFN_UP(sgl->offset + sgl->length) -
> -						hvpgoff;
> +		for_each_sg(sgl, sg, sg_count, j) {

There's a subtle issue here in that the number of entries in the
mapped sgl might not be the same as the number of entries prior
to the mapping.  A change in the count probably never happens for
the direct DMA mapping being done here, but let's code to be
correct in the general case.  Either need to refetch the value of
sg_count, or arrange to use something like for_each_sgtable_dma_sg().

> +			hvpfns_to_add =3D HVPFN_UP(sg_dma_len(sg));

This simplification in calculating hvpnfs_to_add is not correct.  Consider
the case of one sgl entry specifying a buffer of 3 Kbytes that starts at a
2K offset in the first page and runs over into the second page.  This case
can happen when the physical memory for the two pages is contiguous
due to random happenstance, due to huge pages, or due to being on an
architecture like ARM64 where the guest page size may be larger than
the Hyper-V page size.

In this case, we need two Hyper-V PFNs because the buffer crosses a
Hyper-V page boundary.   But the above will calculate only one PFN.
The original algorithm handles this case correctly.

> +			hvpfn =3D HVPFN_DOWN(sg_dma_address(sg));
>=20
> -			/*
> -			 * Fill the next portion of the PFN array with
> -			 * sequential Hyper-V PFNs for the continguous physical
> -			 * memory described by the sgl entry. The end of the
> -			 * last sgl should be reached at the same time that
> -			 * the PFN array is filled.
> -			 */

Any reason not to keep this comment?  It's still correct.

>  			while (hvpfns_to_add--)
> -				payload->range.pfn_array[i++] =3D	hvpfn++;
> +				payload->range.pfn_array[i++] =3D hvpfn++;
>  		}
>  	}
>=20
> @@ -1992,6 +1982,7 @@ static int storvsc_probe(struct hv_device *device,
>  	stor_device->vmscsi_size_delta =3D sizeof(struct vmscsi_win8_extension)=
;
>  	spin_lock_init(&stor_device->lock);
>  	hv_set_drvdata(device, stor_device);
> +	dma_set_min_align_mask(&device->device, HV_HYP_PAGE_SIZE - 1);
>=20
>  	stor_device->port_number =3D host->host_no;
>  	ret =3D storvsc_connect_to_vsp(device, storvsc_ringbuffer_size, is_fc);
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index 139a43ad65a1..8f39893f8ccf 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1274,6 +1274,7 @@ struct hv_device {
>=20
>  	struct vmbus_channel *channel;
>  	struct kset	     *channels_kset;
> +	struct device_dma_parameters dma_parms;
>=20
>  	/* place holder to keep track of the dir for hv device in debugfs */
>  	struct dentry *debug_dir;
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:30:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:30:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176799.321754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcUG-0005lY-Jo; Thu, 02 Sep 2021 02:29:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176799.321754; Thu, 02 Sep 2021 02:29:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcUG-0005lR-Ge; Thu, 02 Sep 2021 02:29:56 +0000
Received: by outflank-mailman (input) for mailman id 176799;
 Thu, 02 Sep 2021 02:29:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLcUE-0005lD-Hq; Thu, 02 Sep 2021 02:29:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLcUE-0001Ey-BX; Thu, 02 Sep 2021 02:29:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLcUE-0002ki-0J; Thu, 02 Sep 2021 02:29:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLcUD-0005cY-WA; Thu, 02 Sep 2021 02:29:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/eIgvflmIhZhQI4ZIS1OfpAezLcfBg23PY7XRzN7lbw=; b=65x2/tIHhqEoUrTxpWpiJANyJ+
	tZoy3bWR0VBs/PMg6TFj7stpspIaOzcaf24+M7njE4wNqDCxaKc0zFox7KXm2x7e6AfaAU8/hmqAO
	AuZs9/DG/RhRdk8u8qk3FJZq6Eqsk9YARZ6FHSYlNkqTNYdG/cjqQVFDmGrWDaXnbhwU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164693-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.11-testing test] 164693: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.11-testing:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-4.11-testing:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:debian-hvm-install:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:debian-hvm-install:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.11-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e9bee143fd4248b004d1235c320bb0180a4c757f
X-Osstest-Versions-That:
    xen=ef32c7afa2731b758226d6e10a1e489b1a15fc41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 02:29:53 +0000

flight 164693 xen-4.11-testing real [real]
flight 164735 xen-4.11-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164693/
http://logs.test-lab.xenproject.org/osstest/logs/164735/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164735-retest

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 12 debian-hvm-install fail like 162548
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 12 debian-hvm-install fail like 162548
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 162548
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 162548
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 162548
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 162548
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 162548
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 162548
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 162548
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 162548
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 162548
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 162548
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 162548
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 xen                  e9bee143fd4248b004d1235c320bb0180a4c757f
baseline version:
 xen                  ef32c7afa2731b758226d6e10a1e489b1a15fc41

Last test of basis   162548  2021-06-08 18:37:00 Z   85 days
Failing since        164486  2021-08-25 14:05:55 Z    7 days    6 attempts
Testing same since   164693  2021-09-01 09:36:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        fail    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ef32c7afa2..e9bee143fd  e9bee143fd4248b004d1235c320bb0180a4c757f -> stable-4.11


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:30:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:30:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176805.321769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcUo-00071Y-4d; Thu, 02 Sep 2021 02:30:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176805.321769; Thu, 02 Sep 2021 02:30:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcUo-00071R-0r; Thu, 02 Sep 2021 02:30:30 +0000
Received: by outflank-mailman (input) for mailman id 176805;
 Thu, 02 Sep 2021 02:30:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do2V=NY=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLcUm-0006w4-IU
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 02:30:28 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.77]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2acc9a32-9e0f-4a2c-b085-76b2b2909665;
 Thu, 02 Sep 2021 02:30:20 +0000 (UTC)
Received: from DB6PR07CA0112.eurprd07.prod.outlook.com (2603:10a6:6:2c::26) by
 AS8PR08MB6790.eurprd08.prod.outlook.com (2603:10a6:20b:397::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 02:30:18 +0000
Received: from DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2c:cafe::39) by DB6PR07CA0112.outlook.office365.com
 (2603:10a6:6:2c::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4 via Frontend
 Transport; Thu, 2 Sep 2021 02:30:18 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT061.mail.protection.outlook.com (10.152.21.234) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 02:30:17 +0000
Received: ("Tessian outbound 6ba9ec11d42b:v103");
 Thu, 02 Sep 2021 02:30:17 +0000
Received: from 5b88d9df9254.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 337490BF-0A08-4339-9F83-9BC61E732793.1; 
 Thu, 02 Sep 2021 02:30:11 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5b88d9df9254.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 02:30:11 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4315.eurprd08.prod.outlook.com (2603:10a6:10:cd::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 02:30:09 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.026; Thu, 2 Sep 2021
 02:30:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2acc9a32-9e0f-4a2c-b085-76b2b2909665
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lHGjjLyUt7AJP/HqcDCD570L6Zw4mKyCG8YQ/5AkkyY=;
 b=FfJ2qPrwiwQ3+qtq7NRaSqak9YBMeFWO5J1KkMmT6RpcQ3L6AW3iXTMZSFcAanmcVwkESbhbT2cL1cs7T1Z/slJajaJAKXTcX4KnyHPAgI8Qx4fh9m2d1lJ5sukro/Q0jOsmXpreyVuM0sKF4ur4tSTN3b6wfBnkiwPymMyE6HU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ENhPZCgrIZ3NH+4nUwGrgFS4mqKizgTFjS5GrmfodRmtQmp/lhVrFEfkW/MMxQDuhZxZsriS1yuLqC4c0T0PESlLwqOdqJ1TZOelZt8C4xjKzLnXjhKcLZv+wZMa23Xx3nVyuGYixuYGpuP1zJuxTeXseE4PhV/p0YaZgoIBCxWlHQTsCPtukPRP8D+/vwekSCRvNAYFaN43IbAtQxgRxTHz0JtwllqKYDxuM0ytr6PP8eRIRgRPb9T+JXm6Iay/3fkFAHkxiS0AgsiuixF/GlE48o3TPDxMeiQn78QqDRJGHCvFwJPtAEU/C2jX0wPD5QNnxhQqXsHxdz1QN5B2+A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lHGjjLyUt7AJP/HqcDCD570L6Zw4mKyCG8YQ/5AkkyY=;
 b=VfFjACFhRe1a8A65VkL0AZVQ7Su8gBGFyz70RYHLwdlp/T2cIXcaYGipyG6X8UQD1yTf5Cja9WHIZc7MjazHjkXfNl6W9PPJQelIxFG1Yhw95fsQLKuR/agtqqeIP+4I1+kxrTK0TFdFlsT3QFaJlsgnFjNix6L/EDQJgjUgnZfkyq5kF6NTLtlmgBZ5E467DHkLgRGKJj5Ox+um5atVxo8j67zXs/G6xhqynFkK8JUkoURIHlJD5jKHn3844+TsLsAES8DK5oI4pQxvD4t7Wcd80uy/r6gPuKN3UXTGV/ZI3Jva+A0gaet2YcbLkF4392/WxXLzxzKQltkt83C1bw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lHGjjLyUt7AJP/HqcDCD570L6Zw4mKyCG8YQ/5AkkyY=;
 b=FfJ2qPrwiwQ3+qtq7NRaSqak9YBMeFWO5J1KkMmT6RpcQ3L6AW3iXTMZSFcAanmcVwkESbhbT2cL1cs7T1Z/slJajaJAKXTcX4KnyHPAgI8Qx4fh9m2d1lJ5sukro/Q0jOsmXpreyVuM0sKF4ur4tSTN3b6wfBnkiwPymMyE6HU=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
Thread-Topic: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
 device tree NUMA distance map
Thread-Index:
 AQHXjps+iLbrOQu8n06b5z+EtCRt06uM5puAgABnI4CAAPWOAIAA3gOAgABcYACAAKNX0A==
Date: Thu, 2 Sep 2021 02:30:08 +0000
Message-ID:
 <DB9PR08MB68572F556DB52ADCA4DA07349ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-25-wei.chen@arm.com>
 <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: D2C81D7069A0E447BB451A1CE27AB9AA.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4e223013-a22e-4343-ffa8-08d96db99a97
x-ms-traffictypediagnostic: DBBPR08MB4315:|AS8PR08MB6790:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB6790A789D3802385312FFC569ECE9@AS8PR08MB6790.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 XCbzMCNrezVhusBXHN/aXujP4S0huck0rx+kfxx6PhCiN/ReB1DppPdNWH9Tvey51cgl+r6SfKNEPAd5RoAqi6v3DeWZnhUAvOVc0034KiJzztWkir2KYUmlhZTOE1Nd2SKCP1OSWdrsVJYm0yJPqkc8kdbUsHaX9t7lZUgzjFACzVA9jwFDNxSOGu2wokOcQ/41BdI3uBj/4JVqSBHHk1wGY43gGQM1Na7lF9ZY4tJe+ihHhcX7RJ3T/Frn0VU3zXqii/pm5V8GhCLqjs7fk0CHJdfl+K4L4aXUaeQMw+zz0AT3Y1JzFC1Q1vaoe5NFkHULDl6/iyeb1vPmcd+KuBm02LqLFIPgaI88UuiQHQe5j3EuGpIK/Yxc73iJutmJ3h1WWMMajPiFpfhFsBCXXW58oipbb8W65I7JL1ej4xBl2PjSFpL5bgA2B0MWRxGh3bTsB9WYBe6Y679ihOIvUh/O/j1TU0MFnpeB7uPo2hUMO3IJvHdgEBUW4TUOTBPSXMJmo2uTD5i1hhA63sXvl1biHZkKn2o/1Fb4xqH39HVLUJrHNxL7Ukbr03rPR6z7Muxxeun4MPwUTXbPqF5KP8TiBHr2pOpUs436318WvWgjJk1H/MseY1iZbzM+FTmQkDjJnnRUybRC/h+9nUjtH3iTFSV3hbxYNx7sbToyf0HlZzf0vmfqCHrgRfmqL7Jj/5jwSpaFIeukhYhY0TRbjA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(366004)(346002)(136003)(39860400002)(376002)(2906002)(33656002)(6916009)(38100700002)(478600001)(9686003)(55016002)(26005)(122000001)(83380400001)(8676002)(186003)(53546011)(66476007)(6506007)(66446008)(64756008)(66556008)(66946007)(38070700005)(30864003)(86362001)(5660300002)(316002)(71200400001)(8936002)(76116006)(52536014)(7696005)(54906003)(4326008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4315
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e5aa2546-5672-45b8-bd13-08d96db9954d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	E70hnrVePJUVmKlLXD3zLVd5OzIxhBHC9o4YrQvS3qQ06gqRBfY34MyVfQuMQeF0EkipDNf8s65xNYrC95SNjVs5/O31QPoMFXRfP5Y/qg0i9Zv9NljFTHM0utVkIrpc11CKeuaeZzgffPfHcXSZ0+chioJK7CFxDDZVas6gG69C1bzV0SjcmEPJry0S/jdzRnBmgRFb9CVYKZz7z7YOvDo6Ev0Ncej/wI3NUAUP5jXgbqyi2qCaYO+Exf0J0yhckevh1r5P9SuJHaU4RitPprQSy2gMQupQGgZ8KJrvKaWXojJPiwr1hxrFnuMB20hZe9i4iOvpEOAzTVCKMJEvx/GuJhEAIiQqr+m4dkqZXGZVLrfYml+Jcs+OYNz2q43oMzrAz9I/B4Z61y2DSIlZaSxe5N15sIEOA33ghSKgm8lFRyA856+8lGCSv1qvXbCr+P2L5p3luzGzTY5UM/5Y/T2n59G1pHA3jBxcDdiiz6KemBvhXIFVsOqcYeWRqwF89TuVKsDUz64IEfcMEOeH2YTbsoa/az6dNkRRg0vh/7OwbNUp4lI6LaghmVsaLvl0Rb21GWEYhsEPyPhnXySESPoSRLhGQ7UwrYqGacfy343pP+Hh+0KW0L9Q4YIyiLUrYmdDGVtXEojm0O7M7l6sVWXFUCKHbAWl7Jg8AXXOgLGPi1G0ysb2Vm5fEgtd+ZLYOea3JDDD8kq9uHUX8r1vXg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(136003)(346002)(376002)(46966006)(36840700001)(70586007)(4326008)(5660300002)(36860700001)(6862004)(7696005)(70206006)(9686003)(8936002)(55016002)(52536014)(82310400003)(86362001)(53546011)(2906002)(33656002)(47076005)(82740400003)(8676002)(478600001)(356005)(186003)(336012)(316002)(30864003)(81166007)(26005)(83380400001)(54906003)(6506007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 02:30:17.9585
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e223013-a22e-4343-ffa8-08d96db99a97
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6790

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMuaXpSAwOjIyDQo+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IFN0
ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0NCj4gZGV2ZWxA
bGlzdHMueGVucHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+
IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+IFN1YmplY3Q6IFJFOiBbWEVOIFJGQyBQQVRD
SCAyNC80MF0geGVuL2FybTogaW50cm9kdWNlIGEgaGVscGVyIHRvIHBhcnNlDQo+IGRldmljZSB0
cmVlIE5VTUEgZGlzdGFuY2UgbWFwDQo+IA0KPiBPbiBXZWQsIDEgU2VwIDIwMjEsIFdlaSBDaGVu
IHdyb3RlOg0KPiA+IEhpIFN0ZWZhbm8sDQo+ID4NCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn
ZS0tLS0tDQo+ID4gPiBGcm9tOiBYZW4tZGV2ZWwgPHhlbi1kZXZlbC1ib3VuY2VzQGxpc3RzLnhl
bnByb2plY3Qub3JnPiBPbiBCZWhhbGYgT2YNCj4gPiA+IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+
ID4gU2VudDogMjAyMeW5tDnmnIgx5pelIDU6MzYNCj4gPiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNo
ZW5AYXJtLmNvbT4NCj4gPiA+IENjOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtl
cm5lbC5vcmc+OyB4ZW4tDQo+ID4gPiBkZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsganVsaWVu
QHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5j
b20+DQo+ID4gPiBTdWJqZWN0OiBSRTogW1hFTiBSRkMgUEFUQ0ggMjQvNDBdIHhlbi9hcm06IGlu
dHJvZHVjZSBhIGhlbHBlciB0bw0KPiBwYXJzZQ0KPiA+ID4gZGV2aWNlIHRyZWUgTlVNQSBkaXN0
YW5jZSBtYXANCj4gPiA+DQo+ID4gPiBPbiBUdWUsIDMxIEF1ZyAyMDIxLCBXZWkgQ2hlbiB3cm90
ZToNCj4gPiA+ID4gSGkgU3RlZmFubywNCj4gPiA+ID4NCj4gPiA+ID4gPiAtLS0tLU9yaWdpbmFs
IE1lc3NhZ2UtLS0tLQ0KPiA+ID4gPiA+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJl
bGxpbmlAa2VybmVsLm9yZz4NCj4gPiA+ID4gPiBTZW50OiAyMDIx5bm0OOaciDMx5pelIDg6NDgN
Cj4gPiA+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiA+ID4gQ2M6
IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsgc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsN
Cj4gPiA+IGp1bGllbkB4ZW4ub3JnOw0KPiA+ID4gPiA+IGpiZXVsaWNoQHN1c2UuY29tOyBCZXJ0
cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+ID4gPiA+ID4gU3ViamVj
dDogUmU6IFtYRU4gUkZDIFBBVENIIDI0LzQwXSB4ZW4vYXJtOiBpbnRyb2R1Y2UgYSBoZWxwZXIg
dG8NCj4gPiA+IHBhcnNlDQo+ID4gPiA+ID4gZGV2aWNlIHRyZWUgTlVNQSBkaXN0YW5jZSBtYXAN
Cj4gPiA+ID4gPg0KPiA+ID4gPiA+IE9uIFdlZCwgMTEgQXVnIDIwMjEsIFdlaSBDaGVuIHdyb3Rl
Og0KPiA+ID4gPiA+ID4gQSBOVU1BIGF3YXJlIGRldmljZSB0cmVlIHdpbGwgcHJvdmlkZSBhICJk
aXN0YW5jZS1tYXAiIG5vZGUgdG8NCj4gPiA+ID4gPiA+IGRlc2NyaWJlIGRpc3RhbmNlIGJldHdl
ZW4gYW55IHR3byBub2Rlcy4gVGhpcyBwYXRjaCBpbnRyb2R1Y2UgYQ0KPiA+ID4gPiA+ID4gbmV3
IGhlbHBlciB0byBwYXJzZSB0aGlzIGRpc3RhbmNlIG1hcC4NCj4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiA+ID4g
PiA+IC0tLQ0KPiA+ID4gPiA+ID4gIHhlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMgfCA2
Nw0KPiA+ID4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQo+ID4gPiA+ID4gPiAg
MSBmaWxlIGNoYW5nZWQsIDY3IGluc2VydGlvbnMoKykNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+ID4gPiA+
IGIveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+ID4gPiA+ID4gaW5kZXggYmJl
MDgxZGNkMS4uNmUwZDFkM2Q5ZiAxMDA2NDQNCj4gPiA+ID4gPiA+IC0tLSBhL3hlbi9hcmNoL2Fy
bS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+ID4gPiA+ICsrKyBiL3hlbi9hcmNoL2FybS9udW1h
X2RldmljZV90cmVlLmMNCj4gPiA+ID4gPiA+IEBAIC0yMDAsMyArMjAwLDcwIEBAIGRldmljZV90
cmVlX3BhcnNlX251bWFfbWVtb3J5X25vZGUoY29uc3QNCj4gdm9pZA0KPiA+ID4gKmZkdCwNCj4g
PiA+ID4gPiBpbnQgbm9kZSwNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiAgICAgIHJldHVybiAw
Ow0KPiA+ID4gPiA+ID4gIH0NCj4gPiA+ID4gPiA+ICsNCj4gPiA+ID4gPiA+ICsvKiBQYXJzZSBO
VU1BIGRpc3RhbmNlIG1hcCB2MSAqLw0KPiA+ID4gPiA+ID4gK2ludCBfX2luaXQNCj4gPiA+ID4g
PiA+ICtkZXZpY2VfdHJlZV9wYXJzZV9udW1hX2Rpc3RhbmNlX21hcF92MShjb25zdCB2b2lkICpm
ZHQsIGludA0KPiBub2RlKQ0KPiA+ID4gPiA+ID4gK3sNCj4gPiA+ID4gPiA+ICsgICAgY29uc3Qg
c3RydWN0IGZkdF9wcm9wZXJ0eSAqcHJvcDsNCj4gPiA+ID4gPiA+ICsgICAgY29uc3QgX19iZTMy
ICptYXRyaXg7DQo+ID4gPiA+ID4gPiArICAgIGludCBlbnRyeV9jb3VudCwgbGVuLCBpOw0KPiA+
ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gKyAgICBwcmludGsoWEVOTE9HX0lORk8gIk5VTUE6IHBh
cnNpbmcgbnVtYS1kaXN0YW5jZS1tYXBcbiIpOw0KPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4g
KyAgICBwcm9wID0gZmR0X2dldF9wcm9wZXJ0eShmZHQsIG5vZGUsICJkaXN0YW5jZS1tYXRyaXgi
LCAmbGVuKTsNCj4gPiA+ID4gPiA+ICsgICAgaWYgKCAhcHJvcCApDQo+ID4gPiA+ID4gPiArICAg
IHsNCj4gPiA+ID4gPiA+ICsgICAgICAgIHByaW50ayhYRU5MT0dfV0FSTklORw0KPiA+ID4gPiA+
ID4gKyAgICAgICAgICAgICAgICJOVU1BOiBObyBkaXN0YW5jZS1tYXRyaXggcHJvcGVydHkgaW4g
ZGlzdGFuY2UtDQo+ID4gPiBtYXBcbiIpOw0KPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gKyAg
ICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gPiA+ID4gPiArICAgIH0NCj4gPiA+ID4gPiA+ICsN
Cj4gPiA+ID4gPiA+ICsgICAgaWYgKCBsZW4gJSBzaXplb2YodWludDMyX3QpICE9IDAgKQ0KPiA+
ID4gPiA+ID4gKyAgICB7DQo+ID4gPiA+ID4gPiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5J
TkcNCj4gPiA+ID4gPiA+ICsgICAgICAgICAgICAgICAiZGlzdGFuY2UtbWF0cml4IGluIG5vZGUg
aXMgbm90IGEgbXVsdGlwbGUgb2YNCj4gPiA+IHUzMlxuIik7DQo+ID4gPiA+ID4gPiArICAgICAg
ICByZXR1cm4gLUVJTlZBTDsNCj4gPiA+ID4gPiA+ICsgICAgfQ0KPiA+ID4gPiA+ID4gKw0KPiA+
ID4gPiA+ID4gKyAgICBlbnRyeV9jb3VudCA9IGxlbiAvIHNpemVvZih1aW50MzJfdCk7DQo+ID4g
PiA+ID4gPiArICAgIGlmICggZW50cnlfY291bnQgPD0gMCApDQo+ID4gPiA+ID4gPiArICAgIHsN
Cj4gPiA+ID4gPiA+ICsgICAgICAgIHByaW50ayhYRU5MT0dfV0FSTklORyAiTlVNQTogSW52YWxp
ZCBkaXN0YW5jZS0NCj4gbWF0cml4XG4iKTsNCj4gPiA+ID4gPiA+ICsNCj4gPiA+ID4gPiA+ICsg
ICAgICAgIHJldHVybiAtRUlOVkFMOw0KPiA+ID4gPiA+ID4gKyAgICB9DQo+ID4gPiA+ID4gPiAr
DQo+ID4gPiA+ID4gPiArICAgIG1hdHJpeCA9IChjb25zdCBfX2JlMzIgKilwcm9wLT5kYXRhOw0K
PiA+ID4gPiA+ID4gKyAgICBmb3IgKCBpID0gMDsgaSArIDIgPCBlbnRyeV9jb3VudDsgaSArPSAz
ICkNCj4gPiA+ID4gPiA+ICsgICAgew0KPiA+ID4gPiA+ID4gKyAgICAgICAgdWludDMyX3QgZnJv
bSwgdG8sIGRpc3RhbmNlOw0KPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gKyAgICAgICAgZnJv
bSA9IGR0X3JlYWRfbnVtYmVyKG1hdHJpeCwgMSk7DQo+ID4gPiA+ID4gPiArICAgICAgICBtYXRy
aXgrKzsNCj4gPiA+ID4gPiA+ICsgICAgICAgIHRvID0gZHRfcmVhZF9udW1iZXIobWF0cml4LCAx
KTsNCj4gPiA+ID4gPiA+ICsgICAgICAgIG1hdHJpeCsrOw0KPiA+ID4gPiA+ID4gKyAgICAgICAg
ZGlzdGFuY2UgPSBkdF9yZWFkX251bWJlcihtYXRyaXgsIDEpOw0KPiA+ID4gPiA+ID4gKyAgICAg
ICAgbWF0cml4Kys7DQo+ID4gPiA+ID4gPiArDQo+ID4gPiA+ID4gPiArICAgICAgICBpZiAoIChm
cm9tID09IHRvICYmIGRpc3RhbmNlICE9IE5VTUFfTE9DQUxfRElTVEFOQ0UpIHx8DQo+ID4gPiA+
ID4gPiArICAgICAgICAgICAgKGZyb20gIT0gdG8gJiYgZGlzdGFuY2UgPD0gTlVNQV9MT0NBTF9E
SVNUQU5DRSkgKQ0KPiA+ID4gPiA+ID4gKyAgICAgICAgew0KPiA+ID4gPiA+ID4gKyAgICAgICAg
ICAgIHByaW50ayhYRU5MT0dfV0FSTklORw0KPiA+ID4gPiA+ID4gKyAgICAgICAgICAgICAgICAg
ICAiSW52YWxpZCBub2RlcycgZGlzdGFuY2UgZnJvbSBub2RlIyVkIHRvDQo+IG5vZGUjJWQNCj4g
PiA+ID4gPiA9ICVkXG4iLA0KPiA+ID4gPiA+ID4gKyAgICAgICAgICAgICAgICAgICBmcm9tLCB0
bywgZGlzdGFuY2UpOw0KPiA+ID4gPiA+ID4gKyAgICAgICAgICAgIHJldHVybiAtRUlOVkFMOw0K
PiA+ID4gPiA+ID4gKyAgICAgICAgfQ0KPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gKyAgICAg
ICAgcHJpbnRrKFhFTkxPR19JTkZPICJOVU1BOiBkaXN0YW5jZSBmcm9tIG5vZGUjJWQgdG8NCj4g
bm9kZSMlZA0KPiA+ID4gPiA+ID0gJWRcbiIsDQo+ID4gPiA+ID4gPiArICAgICAgICAgICAgICAg
ZnJvbSwgdG8sIGRpc3RhbmNlKTsNCj4gPiA+ID4gPiA+ICsgICAgICAgIG51bWFfc2V0X2Rpc3Rh
bmNlKGZyb20sIHRvLCBkaXN0YW5jZSk7DQo+ID4gPiA+ID4gPiArDQo+ID4gPiA+ID4gPiArICAg
ICAgICAvKiBTZXQgZGVmYXVsdCBkaXN0YW5jZSBvZiBub2RlIEItPkEgc2FtZSBhcyBBLT5CICov
DQo+ID4gPiA+ID4gPiArICAgICAgICBpZiAodG8gPiBmcm9tKQ0KPiA+ID4gPiA+ID4gKyAgICAg
ICAgICAgICBudW1hX3NldF9kaXN0YW5jZSh0bywgZnJvbSwgZGlzdGFuY2UpOw0KPiA+ID4gPiA+
DQo+ID4gPiA+ID4gSSBhbSBhIGJpdCB1bnN1cmUgYWJvdXQgdGhpcyBsYXN0IDIgbGluZXM6IHdo
eSBjYWxsaW5nDQo+ID4gPiBudW1hX3NldF9kaXN0YW5jZQ0KPiA+ID4gPiA+IGluIHRoZSBvcHBv
c2l0ZSBkaXJlY3Rpb24gb25seSB3aGVuIHRvID4gZnJvbT8gV291bGRuJ3QgaXQgYmUgT0sNCj4g
dG8NCj4gPiA+ID4gPiBhbHdheXMgZG8gYm90aDoNCj4gPiA+ID4gPg0KPiA+ID4gPiA+IG51bWFf
c2V0X2Rpc3RhbmNlKGZyb20sIHRvLCBkaXN0YW5jZSk7DQo+ID4gPiA+ID4gbnVtYV9zZXRfZGlz
dGFuY2UodG8sIGZyb20sIGRpc3RhbmNlKTsNCj4gPiA+ID4gPg0KPiA+ID4gPiA+ID8NCj4gPiA+
ID4gPg0KPiA+ID4gPiBJIGJvcnJvd2VkIHRoaXMgY29kZSBmcm9tIExpbnV4LCBidXQgaGVyZSBp
cyBteSB1bmRlcnN0YW5kaW5nOg0KPiA+ID4gPg0KPiA+ID4gPiBGaXJzdCwgSSByZWFkIHNvbWUg
bm90ZXMgaW4NCj4gRG9jdW1lbnRhdGlvbi9kZXZpY2V0cmVlL2JpbmRpbmdzL251bWEudHh0DQo+
ID4gPiA+IDEuIEVhY2ggZW50cnkgcmVwcmVzZW50cyBkaXN0YW5jZSBmcm9tIGZpcnN0IG5vZGUg
dG8gc2Vjb25kIG5vZGUuDQo+ID4gPiA+IFRoZSBkaXN0YW5jZXMgYXJlIGVxdWFsIGluIGVpdGhl
ciBkaXJlY3Rpb24uDQo+ID4gPiA+IDIuIGRpc3RhbmNlLW1hdHJpeCBzaG91bGQgaGF2ZSBlbnRy
aWVzIGluIGxleGljb2dyYXBoaWNhbCBhc2NlbmRpbmcNCj4gPiA+ID4gb3JkZXIgb2Ygbm9kZXMu
DQo+ID4gPiA+DQo+ID4gPiA+IEhlcmUgaXMgYW4gZXhhbXBsZSBvZiBkaXN0YW5jZS1tYXAgbm9k
ZSBpbiBEVEI6DQo+ID4gPiA+IFNhbXBsZSMxLCBmdWxsIGxpc3Q6DQo+ID4gPiA+IAkJZGlzdGFu
Y2UtbWFwIHsNCj4gPiA+ID4gCQkJIGNvbXBhdGlibGUgPSAibnVtYS1kaXN0YW5jZS1tYXAtdjEi
Ow0KPiA+ID4gPiAJCQkgZGlzdGFuY2UtbWF0cml4ID0gPDAgMCAgMTA+LA0KPiA+ID4gPiAJCQkJ
CSAgIDwwIDEgIDIwPiwNCj4gPiA+ID4gCQkJCQkgICA8MCAyICA0MD4sDQo+ID4gPiA+IAkJCQkJ
ICAgPDAgMyAgMjA+LA0KPiA+ID4gPiAJCQkJCSAgIDwxIDAgIDIwPiwNCj4gPiA+ID4gCQkJCQkg
ICA8MSAxICAxMD4sDQo+ID4gPiA+IAkJCQkJICAgPDEgMiAgMjA+LA0KPiA+ID4gPiAJCQkJCSAg
IDwxIDMgIDQwPiwNCj4gPiA+ID4gCQkJCQkgICA8MiAwICA0MD4sDQo+ID4gPiA+IAkJCQkJICAg
PDIgMSAgMjA+LA0KPiA+ID4gPiAJCQkJCSAgIDwyIDIgIDEwPiwNCj4gPiA+ID4gCQkJCQkgICA8
MiAzICAyMD4sDQo+ID4gPiA+IAkJCQkJICAgPDMgMCAgMjA+LA0KPiA+ID4gPiAJCQkJCSAgIDwz
IDEgIDQwPiwNCj4gPiA+ID4gCQkJCQkgICA8MyAyICAyMD4sDQo+ID4gPiA+IAkJCQkJICAgPDMg
MyAgMTA+Ow0KPiA+ID4gPiAJCX07DQo+ID4gPiA+DQo+ID4gPiA+IENhbGwgbnVtYV9zZXRfZGlz
dGFuY2Ugd2hlbiAidG8gPiBmcm9tIiB3aWxsIHByZXZlbnQgWGVuIHRvIGNhbGwNCj4gPiA+ID4g
bnVtYV9zZXRfZGlzdGFuY2UoMCwgMSwgMjApIGFnYWluIHdoZW4gaXQncyBzZXR0aW5nIGRpc3Rh
bmNlIGZvciA8MQ0KPiAwDQo+ID4gPiAyMD4uDQo+ID4gPiA+IEJ1dCwgbnVtYV9zZXRfZGlzdGFu
Y2UoMSwgMCwgMjApIHdpbGwgYmUgY2FsbCB0d2ljZS4NCj4gPiA+ID4NCj4gPiA+ID4gTm9ybWFs
bHksIGRpc3RhbmNlLW1hcCBub2RlIHdpbGwgYmUgb3B0aW1pemVkIGluIGZvbGxvd2luZyBzYW1w
bGUjMiwNCj4gPiA+ID4gYWxsIHJlZHVuZGFudCBlbnRyaWVzIGFyZSByZW1vdmVkOg0KPiA+ID4g
PiBTYW1wbGUjMiwgcGFydGlhbCBsaXN0Og0KPiA+ID4gPiAJCWRpc3RhbmNlLW1hcCB7DQo+ID4g
PiA+IAkJCSBjb21wYXRpYmxlID0gIm51bWEtZGlzdGFuY2UtbWFwLXYxIjsNCj4gPiA+ID4gCQkJ
IGRpc3RhbmNlLW1hdHJpeCA9IDwwIDAgIDEwPiwNCj4gPiA+ID4gCQkJCQkgICA8MCAxICAyMD4s
DQo+ID4gPiA+IAkJCQkJICAgPDAgMiAgNDA+LA0KPiA+ID4gPiAJCQkJCSAgIDwwIDMgIDIwPiwN
Cj4gPiA+ID4gCQkJCQkgICA8MSAxICAxMD4sDQo+ID4gPiA+IAkJCQkJICAgPDEgMiAgMjA+LA0K
PiA+ID4gPiAJCQkJCSAgIDwxIDMgIDQwPiwNCj4gPiA+ID4gCQkJCQkgICA8MiAyICAxMD4sDQo+
ID4gPiA+IAkJCQkJICAgPDIgMyAgMjA+LA0KPiA+ID4gPiAJCQkJCSAgIDwzIDMgIDEwPjsNCj4g
PiA+ID4gCQl9Ow0KPiA+ID4gPg0KPiA+ID4gPiBUaGVyZSBpcyBub3QgYW55ICJmcm9tID4gdG8i
IGVudHJ5IGluIHRoZSBtYXAuIEJ1dCB1c2luZyB0aGlzDQo+IHBhcnRpYWwNCj4gPiA+IG1hcA0K
PiA+ID4gPiBzdGlsbCBjYW4gc2V0IGFsbCBkaXN0YW5jZXMgZm9yIGFsbCBwYWlycy4gQW5kIG51
bWFfc2V0X2Rpc3RhbmNlKDEsDQo+IDAsDQo+ID4gPiAyMCkNCj4gPiA+ID4gd2lsbCBiZSBvbmx5
IG9uY2UuDQo+ID4gPg0KPiA+ID4gSSBzZWUuIEkgY2FuJ3QgZmluZCBpbiBEb2N1bWVudGF0aW9u
L2RldmljZXRyZWUvYmluZGluZ3MvbnVtYS50eHQNCj4gd2hlcmUNCj4gPiA+IGl0IHNheXMgdGhh
dCAiZnJvbSA+IHRvIiBub2RlcyBjYW4gYmUgb21pdHRlZC4gSWYgaXQgaXMgbm90IHdyaXR0ZW4N
Cj4gPiA+IGRvd24sIHRoZW4gc29tZWJvZHkgY291bGQgZWFzaWx5IG9wdGltaXplIGl0IHRoZSBv
cHBvc2l0ZSB3YXk6DQo+ID4gPg0KPiA+ID4gIAkJCSBkaXN0YW5jZS1tYXRyaXggPSA8MCAwICAx
MD4sDQo+ID4gPiAgCQkJCQkgICA8MSAwICAyMD4sDQo+ID4gPiAgCQkJCQkgICA8MiAwICA0MD4s
DQo+ID4gPiAgCQkJCQkgICA8MyAwICAyMD4sDQo+ID4gPiAgCQkJCQkgICA8MSAxICAxMD4sDQo+
ID4gPiAgCQkJCQkgICA8MiAxICAyMD4sDQo+ID4gPiAgCQkJCQkgICA8MyAxICA0MD4sDQo+ID4g
PiAgCQkJCQkgICA8MiAyICAxMD4sDQo+ID4gPiAgCQkJCQkgICA8MyAyICAyMD4sDQo+ID4gPiAg
CQkJCQkgICA8MyAzICAxMD47DQo+ID4gPg0KPiA+DQo+ID4gWWVzLCB5b3UncmUgcmlnaHQuIFNw
ZWMgZG9lc24ndCBzYXkgb3Bwb3NpdGUgd2F5IGlzIHVuYWxsb3dlZC4NCj4gPg0KPiA+ID4gSSB0
aGluayB0aGUgY29kZSBpbiBYZW4gc2hvdWxkIGJlIHJlc2lsaWVudCBhbmQgYWJsZSB0byBjb3Bl
IHdpdGggYQ0KPiA+ID4gZGV2aWNlIHRyZWUgbGlrZSB0aGUgb25lIHlvdSB3cm90ZSBvciB0aGUg
b25lIEkgd3JvdGUuIEZyb20gYSBjb2RlDQo+ID4gPiBwZXJzcGVjdGl2ZSwgaXQgc2hvdWxkIGJl
IHZlcnkgZWFzeSB0byBkby4gSWYgbm90aGluZyBlbHNlIGl0IHdvdWxkDQo+IG1ha2UNCj4gPiA+
IFhlbiBtb3JlIHJlc2lsaWVudCBhZ2FpbnN0IGJ1Z2d5IGZpcm13YXJlLg0KPiA+ID4NCj4gPiA+
DQo+ID4NCj4gPiBJIGRvbid0IGRpc2FncmVlIHdpdGggdGhhdC4NCj4gPg0KPiA+ID4gPiA+IEJ1
dCBpbiBhbnkgY2FzZSwgSSBoYXZlIGEgZGlmZmVyZW50IHN1Z2dlc3Rpb24uIFRoZSBiaW5kaW5n
IHN0YXRlcw0KPiA+ID4gdGhhdA0KPiA+ID4gPiA+ICJkaXN0YW5jZXMgYXJlIGVxdWFsIGluIGVp
dGhlciBkaXJlY3Rpb24iLiBBbHNvIGl0IGhhcyBhbiBleGFtcGxlDQo+ID4gPiB3aGVyZQ0KPiA+
ID4gPiA+IG9ubHkgb25lIGRpcmVjdGlvbiBpcyBleHByZXNzZWQgdW5mb3J0dW5hdGVseSAoYXQg
dGhlIGVuZCBvZiB0aGUNCj4gPiA+ID4gPiBkb2N1bWVudCkuDQo+ID4gPiA+ID4NCj4gPiA+ID4N
Cj4gPiA+ID4gT2gsIEkgc2hvdWxkIHNlZSB0aGlzIGNvbW1lbnQgZmlyc3QsIHRoZW4gSSB3aWxs
IG5vdCBwb3N0IGFib3ZlDQo+ID4gPiA+IGNvbW1lbnQgOiApDQo+ID4gPiA+DQo+ID4gPiA+ID4g
U28gbXkgc3VnZ2VzdGlvbiBpcyB0byBwYXJzZSBpdCBhcyBmb2xsb3dzOg0KPiA+ID4gPiA+DQo+
ID4gPiA+ID4gLSBjYWxsIG51bWFfc2V0X2Rpc3RhbmNlIGp1c3Qgb25jZSBmcm9tDQo+ID4gPiA+
ID4gICBkZXZpY2VfdHJlZV9wYXJzZV9udW1hX2Rpc3RhbmNlX21hcF92MQ0KPiA+ID4gPiA+DQo+
ID4gPiA+ID4gLSBpbiBudW1hX3NldF9kaXN0YW5jZToNCj4gPiA+ID4gPiAgICAgLSBzZXQgbm9k
ZV9kaXN0YW5jZV9tYXBbZnJvbV1bdG9dID0gZGlzdGFuY2U7DQo+ID4gPiA+ID4gICAgIC0gY2hl
Y2sgbm9kZV9kaXN0YW5jZV9tYXBbdG9dW2Zyb21dDQo+ID4gPiA+ID4gICAgICAgICAgIC0gaWYg
dW5zZXQsIG5vZGVfZGlzdGFuY2VfbWFwW3RvXVtmcm9tXSA9IGRpc3RhbmNlOw0KPiA+ID4gPiA+
ICAgICAgICAgICAtIGlmIGFscmVhZHkgc2V0IHRvIHRoZSBzYW1lIHZhbHVlLCByZXR1cm4gc3Vj
Y2VzczsNCj4gPiA+ID4gPiAgICAgICAgICAgLSBpZiBhbHJlYWR5IHNldCB0byBhIGRpZmZlcmVu
dCB2YWx1ZSwgcmV0dXJuIGVycm9yOw0KPiA+ID4gPg0KPiA+ID4gPiBJIGRvbid0IHJlYWxseSBs
aWtlIHRoaXMgaW1wbGVtZW50YXRpb24uIEkgd2FudCB0aGUgYmVoYXZpb3Igb2YNCj4gPiA+ID4g
bnVtYV9zZXRfZGlzdGFuY2UganVzdCBsaWtlIHRoZSBmdW5jdGlvbiBuYW1lLCBkbyBub3QgaW5j
bHVkZQ0KPiA+ID4gPiBpbXBsaWNpdCBvcGVyYXRpb25zLiBPdGhlcndpc2UsIGV4Y2VwdCB0aGUg
dXNlciByZWFkIHRoaXMgZnVuY3Rpb24NCj4gPiA+ID4gaW1wbGVtZW50YXRpb24gYmVmb3JlIGhl
IHVzZSBpdCwgaGUgcHJvYmFibHkgZG9lc24ndCBrbm93IHRoaXMNCj4gPiA+ID4gZnVuY3Rpb24g
aGFzIGRvbmUgc28gbWFueSB0aGluZ3MuDQo+ID4gPg0KPiA+ID4gWW91IGNhbiBsZWF2ZSBudW1h
X3NldF9kaXN0YW5jZSBhcy1pcyB3aXRob3V0IGFueSBpbXBsaWNpdCBvcGVyYXRpb25zLg0KPiA+
ID4NCj4gPiA+IEluIHRoYXQgY2FzZSwganVzdCBjYWxsIG51bWFfc2V0X2Rpc3RhbmNlIHR3aWNl
IGZyb20gbnVtYV9zZXRfZGlzdGFuY2UNCj4gPiA+IGZvciBib3RoIGZyb20vdG8gYW5kIHRvL2Zy
b20uIG51bWFfc2V0X2Rpc3RhbmNlIGNvdWxkIHJldHVybiBlcnJvciBpcw0KPiA+DQo+ID4gSSBh
bSBPSyBmb3IgdGhlIGZpcnN0IHNlbnRlbmNlLiBCdXQuLi4NCj4gPg0KPiA+ID4gdGhlIGVudHJ5
IHdhcyBhbHJlYWR5IHNldCB0byBhIGRpZmZlcmVudCB2YWx1ZSBvciBzdWNjZXNzIG90aGVyd2lz
ZQ0KPiA+ID4gKGFsc28gaW4gdGhlIGNhc2UgaXQgd2FzIGFscmVhZHkgc2V0IHRvIHRoZSBzYW1l
IHZhbHVlKS4gVGhpcyB3b3VsZA0KPiA+DQo+ID4gLi4uIEkgcHJlZmVyIG5vdCB0byBjaGVjayB0
aGUgcHJldmlvdXMgdmFsdWUuIFN1YnNlcXVlbnQNCj4gbnVtYV9zZXRfZGlzdGFuY2UNCj4gPiBj
YWxsIHdpbGwgb3ZlcnJpZGUgcHJldmlvdXMgY2FsbHMuIEtlZXAgbnVtYV9zZXRfZGlzdGFuY2Ug
YXMgc2ltcGxlIGFzDQo+ID4gaXQgY2FuLiBBbmQgd2hlbiB5b3UgcGFzcyBuZXcgZGF0YSB0byBu
dW1hX3NldF9kaXN0YW5jZSwgaXQgZG9lc24ndA0KPiA+IGtub3cgd2hldGhlciB0aGUgcHJldmlv
dXMgZGF0YSB3YXMgY29ycmVjdCBvciB0aGUgbmV3IGRhdGEgaXMgY29ycmVjdC4NCj4gPiBPbmx5
IGNhbGxlciBtYXkgaGF2ZSBrbm93bi4NCj4gDQo+IFRoYXQgbWlnaHQgYmUgT0sgYnV0IGlmIG5v
dCBudW1hX3NldF9kaXN0YW5jZSB0aGVuIHNvbWVib2R5IGVsc2UgbmVlZHMNCj4gdG8gY2hlY2sg
YWdhaW5zdCBvdmVyd3JpdGluZyBwcmV2aW91cyB2YWx1ZXMuIFRoYXQgaXMgdG8gYmUgYWJsZSB0
byBzcG90DQo+IGJhZCBkZXZpY2UgdHJlZSBjYXNlcyBsaWtlOg0KPiANCj4gICAwIDEgMjANCj4g
ICAxIDAgNDANCg0KDQpIb3cgYWJvdXQgd2UgY2hlY2sgaXQgc3RpbGwgaW4gTlVNQSBkaXN0YW5j
ZSBwYXJzZSBmdW5jdGlvbj8NCkJlZm9yZSBzZXR0aW5nIHRoZSBudW1hX3NldF9kaXN0YW5jZSBm
b3Igb25lIHBhaXIgbm9kZXMgKGUuZy4gYSAtPiBiKSwNCndlIGNhbiBnZXQgaXRzIG9wcG9zaXRl
IHdheSBkaXN0YW5jZSBmaXJzdC4NCg0KZGlzdGFuY2VfYl9hID0gX19ub2RlX2Rpc3RhbmNlKGIs
IGEpOyA9PT4gZ2V0IG9wcG9zaXRlIHdheSBkaXN0YW5jZS4NCmlmIChkaXN0YW5jZV9iX2EgPT0g
MCkgPT0+IG9wcG9zaXRlIHdheSBkaXN0YW5jZSBoYXMgbm90IGJlZW4gc2V0DQp7DQogICAgbnVt
YV9zZXRfZGlzdGFuY2UoYSwgYiwgMjApOyA9PT4gc2V0IGJvdGgNCiAgICBudW1hX3NldF9kaXN0
YW5jZShiLCBhLCAyMCkNCn0gZWxzZSB7DQogICAgaWYgKGRpc3RhbmNlX2JfYSA9PSAyMCkgPT0+
IG9wcG9zaXRlIHdheSBkaXN0YW5jZSBoYXMgYmVlbiBzZXQNCiAgICAgICBudW1hX3NldF9kaXN0
YW5jZShhLCBiLCAyMCk7ID09PiBzZXQgdGhpcyB3YXkgb25seQ0KICAgIGVsc2UgPT09PiBvcHBv
c2l0ZSB3YXkgZGlzdGFuY2UgaGFzIGJlZW4gc2V0LCBidXQgaXMgdW5tYXRjaGVkDQogICAgICAg
Ly8gV2hhdCBjYW4gd2UgZG8gaGVyZT8NCiAgICAgICBQYW5pYyB0aGUgc3lzdGVtPyBvciBKdXN0
IHdhcm5pbmcgdXNlcnM/IE9yIGNob29zZSB0aGUgYmlnZ2VyDQogICAgICAgZGlzdGFuY2UgZm9y
IGJvdGggd2F5cz8NCiAgICAgICANCiAgICAgICBBbmQgZGlzdGFuY2VfYl9hID09IE5VTUFfTk9f
RElTVEFOQ0Ugd291bGQgYmUgYSBzcGVjaWFsIGNhc2UNCiAgICAgICBoZXJlLg0KfQ0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:34:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:34:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176823.321816 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcYj-0008Te-53; Thu, 02 Sep 2021 02:34:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176823.321816; Thu, 02 Sep 2021 02:34:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcYj-0008TX-1D; Thu, 02 Sep 2021 02:34:33 +0000
Received: by outflank-mailman (input) for mailman id 176823;
 Thu, 02 Sep 2021 02:34:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLcYg-0008T6-Sd
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 02:34:31 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.3])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id de8bd298-0d55-4c9e-b611-760d29c6a1b8;
 Thu, 02 Sep 2021 02:34:28 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by CO1PR21MB1313.namprd21.prod.outlook.com (2603:10b6:303:150::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.1; Thu, 2 Sep
 2021 02:34:21 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 02:34:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de8bd298-0d55-4c9e-b611-760d29c6a1b8
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jCx04av9Xh77Ekv4nMdxj0EWuugkjJrPzsF+MpmbIaRcQvM+RxnE3gxDGupHQtUqDAwH69IrhLnYRqRtsE0dyrAMuhQbdUUSTCf8jLuL8Y/ajPuKCgslBJYAaOhyjsJ9aAb0woCKZt8knhURMmLjSAgct2Pherw8Q+qZ5AwQ4XRyNOMOU6ODl4lZhnAlBjvNMLGMWnF89eQIt/TcWCU6iRRvm8BpiSy1zsKr/ei4C6rkMxjTJpoqxj2ik0Y5vuPX64oRCvbKgmYgU+y4LGqpA9mu6ASDNNdDh86jFkGm9q+k3Qpy0qXgEB2oztT1+iFxNGuYbKxaQ9du/ld91anWWw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QKy56hNGj4/CPd4BTTIQaQKfTgUrKi7AodsQzErWTDY=;
 b=iIgSUeLqi9iUloQ1Cc5dm0nvKICmqky/SvABYtH+RfK9BRa8NCOM1G1bcKlpL5O39rmq9UG6FTPeLJWthPU/Yu3V2RXHJBdltqOxmW+RkIur9uNtWcXxLP325q++G7IM1IGslVgaA6ajgXsNlSXYesLJeme/UqaQTgkYmnCxTgp1NRWU5vuGG1m4rdFsjugqQRVOf+6blEBNe/vHPbyFrHE/Gd00tTuCl7CPuN1Bun+hya4bTeD9Gx2Tgv8xySjHvNho2+JzL1C5xisdBEOv6bCNzjFwLv0Q5D3dNaWugkI9uXwjg8SLXNlv/VWVMiCc0eIDUelM3ddYuwFY3Vi16w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QKy56hNGj4/CPd4BTTIQaQKfTgUrKi7AodsQzErWTDY=;
 b=IpmkJTL5nBImCGGLSxRDJrrQor9sZgWdUpecAJfxBth+eYiOc/0sRoJ+IYAN0DbwJjuStPeTlMIvhX8QXnwhdKMBhr//AjVAPIMo9LeKKWrjwAKUhkp+U7ZPavc14xU/3VaEJ3eGEMjkYoUB9BUAxvdziaUyBK9w7C9Ni1z6AfI=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 12/13] hv_netvsc: Add Isolation VM support for netvsc
 driver
Thread-Topic: [PATCH V4 12/13] hv_netvsc: Add Isolation VM support for netvsc
 driver
Thread-Index: AQHXm2gP6jy1GRuc+0K7S1qphPLgHKuQCC+w
Date: Thu, 2 Sep 2021 02:34:21 +0000
Message-ID:
 <MWHPR21MB1593CD9E7B545EF5A268B745D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-13-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-13-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=24c2ad3d-af1e-4524-bef7-f93352ae7e8f;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-02T02:09:02Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 689d41b7-625c-4abb-a483-08d96dba2bc1
x-ms-traffictypediagnostic: CO1PR21MB1313:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <CO1PR21MB1313A834F78F6B030DD374F0D7CE9@CO1PR21MB1313.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2089;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 DAvZwmqwIU8z/VumI40sOIsAL8kY1QtKrD+eDOHx9Sz3XiadoCQcQobAFINlK3ApTacKo8PgaJfemchgf+GLNMTzHG08fpXiUrfUe4MJhuvMfdnT1kVSw5KOvrIgS3Enl8VZRwxQZlc5eqGhZRrc91vSx4ZB1ndPjIbuTF5x1+aDRwIIpA3GIRUSG8aiY1FFeDoVtrLRXMFBR/btQtBEY0xtLlHSGYC0nrfFdB4IWF+KYdBrF6pNpTZ3X+pY46fkK5md+aPJH6u6vM2TedUR+POIxPEbakPeeEj1/11JEYFI1FsNs87RKncViF2p4tSDV+P1sJ+7kjtVyaWCDS++zFv3/nfIYeay1/CT7rQ1fb9Vf8DZZZ4zBksxwXXtIC4jyWB/waxNHcN0mtI1DW393SpXA4a68MkCFUFk80JrGNfxEpTaC3FDP7hp14VoRMrSBP9sLWrC/hBDa0x8dF0kz3VCtUUlG8RS1ScK7vBzRfVSCPf7NC5niNrBGW09242FijzSE1spD5qKJ/Cod1nM74Rd55ec3fewa5tk2e1rHForKqOY9R6KucRXhvWBSVfkaf1BFKJXBXR/UO6ZMAXgQd7PfZn63JE2JEhrSRa5xjcclApHQTb+dJLtofa4MxMxZFkAS0A1QWKtgGUay2+aPJtmedGp8qa9f8FjG84Nt2Mo3B8CRCJAaSO5tLH92WwiMRVgXAeNrx7eVrHnbv0CiEt1tZGqCXCl6GS6IX3dGoI=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(86362001)(76116006)(5660300002)(66946007)(186003)(54906003)(30864003)(921005)(66476007)(66556008)(64756008)(66446008)(7416002)(7366002)(7406005)(52536014)(2906002)(82960400001)(6506007)(316002)(82950400001)(9686003)(55016002)(8936002)(8990500004)(83380400001)(38100700002)(10290500003)(38070700005)(122000001)(8676002)(4326008)(508600001)(33656002)(7696005)(110136005)(71200400001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?xQlDpLVHR5QkNC1X7tRB4TxveZLJaQS1hw8CRJaKaAeeVaMhzp7LecYMifJh?=
 =?us-ascii?Q?s5wS2OsCPzuMHkmVRyjamNhZzwzDFRCeqRMAR3jtBg35DVJj1poV5cOfp0rd?=
 =?us-ascii?Q?OaN1z64LV+DT7xaTzo6vwp1wM7tIMQDoQ/YrK4+qjE4kXlRb62jCDIaaMlp9?=
 =?us-ascii?Q?8Ua0ik61reSOxctXQRuhxfeg2YidtdLKlzQZ+t6pSjN1S+GYNj6pJTcHr28x?=
 =?us-ascii?Q?9neCEGMaxMdhzndRPEvsqXou36pCHJKKzsGZnqaBxyIZ33ogDCQPqTyQaR0p?=
 =?us-ascii?Q?PJ/x+dRxTeSEWn8pJG/Hw8ygZVgc/YbqQLnkqBqBx8MjI9rJQMH9i5bcBSKv?=
 =?us-ascii?Q?AVh5p0g4KtSU54zj5Y4MjEoejzQfRvrdnqc9dCPWoPSCQr+DaKvIlV0Eusqm?=
 =?us-ascii?Q?VCipJ6alCUyB0ZpoB1AuH/fdqRrDEPoqAg2L/LZS199pA3MKCpjawD0UvIwj?=
 =?us-ascii?Q?MDwDxXvdfP6fkYJkpOs/xp4juA1fjtK9/Xr00TtC/TCULrcdCZ+a8fgK5DHp?=
 =?us-ascii?Q?K01HOwCJv608Ei39PuKX7T4v9A66WRGVcPoKDmCFDIqY0B0pVow7DKHCyHcp?=
 =?us-ascii?Q?61coUmemBmab6FzEIqyl5NACtoTvNJ9EyJP87A25QbCGTQibvQqk2rQM72NV?=
 =?us-ascii?Q?hDyGrG36RGqhhwoz2x0+pnu9biZOXlPjdnR+vLKN1NnfQl3yndjmPNRPsWyJ?=
 =?us-ascii?Q?z2yTjCswKMEME9zgqj72S79oxR8MAriVHWa21DSgkEJf5mulLWmK+3VoOhzT?=
 =?us-ascii?Q?+81GIePqODB1bMvXwzZjcd9Mw9WC5e0nnN9Gdt0g+Cqxz5HhTgCa/QdT8jb6?=
 =?us-ascii?Q?oMsXTvJ0zSAdhIVaHNMSsQ3r96c70O/EdlfunILaVpe9qV4UiUCnLXqQyqh6?=
 =?us-ascii?Q?8edXaMRqY6yOVkO6cjqi56xS3yD/wknEDIgPsoGLaJwlkf/lbRYx9M6iEUaH?=
 =?us-ascii?Q?cPWqL/u5F2i7eTpFSUUr3SfjGml4N/kFgfitpqe+ao0yOICH/K7iBqmg1CR+?=
 =?us-ascii?Q?Bj31b1hZtq4+AT+CCoq7seB3RTGKm7mKeV2OsOiRIHosYa/YN8adSNOZCNHg?=
 =?us-ascii?Q?DjiWfF2ofh+FvH0LArOcrWLTEIzJb7m68dvsF+1DeZsC4wTbxvbAFTu55oXy?=
 =?us-ascii?Q?v0KEB91O0Y8nsNzG9sFdzXT4fhghv6KcBHdWKq3dCbmRKwFqA+VvclfLKu8+?=
 =?us-ascii?Q?1UPzihMFvB19heOQB3gd20WEVh8tjULPLwtcN0QiejgDUz9oWj6Z+K8cOtyk?=
 =?us-ascii?Q?aXwtJHgUnv9id+DTfgkwUQ4F8T7W75VIFSTBxSpfPEetmFIFAY+EWrIgzc6c?=
 =?us-ascii?Q?6l+C3cUDAdMMToQg9hUG9EFd?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 689d41b7-625c-4abb-a483-08d96dba2bc1
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 02:34:21.2342
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: g08ziwEqEgDWZKHJyOVnRnVS6cDO5fPg8rfoY4T/qM5/GwGsBOZ+r3nX6j9fGc5D+JukmU5NjK7Zmkh8voCMtGf2u1QtxeKzRuqBo/ZDvrk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR21MB1313

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> In Isolation VM, all shared memory with host needs to mark visible
> to host via hvcall. vmbus_establish_gpadl() has already done it for
> netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
> pagebuffer() stills need to be handled. Use DMA API to map/umap
> these memory during sending/receiving packet and Hyper-V swiotlb
> bounce buffer dma adress will be returned. The swiotlb bounce buffer
> has been masked to be visible to host during boot up.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v3:
> 	* Add comment to explain why not to use dma_map_sg()
> 	* Fix some error handle.
> ---
>  arch/x86/hyperv/ivm.c             |   1 +
>  drivers/net/hyperv/hyperv_net.h   |   5 ++
>  drivers/net/hyperv/netvsc.c       | 135 +++++++++++++++++++++++++++++-
>  drivers/net/hyperv/rndis_filter.c |   2 +
>  include/linux/hyperv.h            |   5 ++
>  5 files changed, 145 insertions(+), 3 deletions(-)
>=20
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> index 84563b3c9f3a..08d8e01de017 100644
> --- a/arch/x86/hyperv/ivm.c
> +++ b/arch/x86/hyperv/ivm.c
> @@ -317,6 +317,7 @@ void *hv_map_memory(void *addr, unsigned long size)
>=20
>  	return vaddr;
>  }
> +EXPORT_SYMBOL_GPL(hv_map_memory);
>=20
>  void hv_unmap_memory(void *addr)
>  {
> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_=
net.h
> index aa7c9962dbd8..862419912bfb 100644
> --- a/drivers/net/hyperv/hyperv_net.h
> +++ b/drivers/net/hyperv/hyperv_net.h
> @@ -164,6 +164,7 @@ struct hv_netvsc_packet {
>  	u32 total_bytes;
>  	u32 send_buf_index;
>  	u32 total_data_buflen;
> +	struct hv_dma_range *dma_range;
>  };
>=20
>  #define NETVSC_HASH_KEYLEN 40
> @@ -1074,6 +1075,7 @@ struct netvsc_device {
>=20
>  	/* Receive buffer allocated by us but manages by NetVSP */
>  	void *recv_buf;
> +	void *recv_original_buf;
>  	u32 recv_buf_size; /* allocated bytes */
>  	u32 recv_buf_gpadl_handle;
>  	u32 recv_section_cnt;
> @@ -1082,6 +1084,7 @@ struct netvsc_device {
>=20
>  	/* Send buffer allocated by us */
>  	void *send_buf;
> +	void *send_original_buf;
>  	u32 send_buf_size;
>  	u32 send_buf_gpadl_handle;
>  	u32 send_section_cnt;
> @@ -1731,4 +1734,6 @@ struct rndis_message {
>  #define RETRY_US_HI	10000
>  #define RETRY_MAX	2000	/* >10 sec */
>=20
> +void netvsc_dma_unmap(struct hv_device *hv_dev,
> +		      struct hv_netvsc_packet *packet);
>  #endif /* _HYPERV_NET_H */
> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
> index f19bffff6a63..edd336b08c2c 100644
> --- a/drivers/net/hyperv/netvsc.c
> +++ b/drivers/net/hyperv/netvsc.c
> @@ -153,8 +153,21 @@ static void free_netvsc_device(struct rcu_head *head=
)
>  	int i;
>=20
>  	kfree(nvdev->extension);
> -	vfree(nvdev->recv_buf);
> -	vfree(nvdev->send_buf);
> +
> +	if (nvdev->recv_original_buf) {
> +		vunmap(nvdev->recv_buf);

In patch 11, you have added a hv_unmap_memory()
function as the inverse of hv_map_memory().  Since this
buffer was mapped with hv_map_memory() and you have
added that function, the cleanup should use
hv_unmap_memory() rather than calling vunmap() directly.

> +		vfree(nvdev->recv_original_buf);
> +	} else {
> +		vfree(nvdev->recv_buf);
> +	}
> +
> +	if (nvdev->send_original_buf) {
> +		vunmap(nvdev->send_buf);

Same here.

> +		vfree(nvdev->send_original_buf);
> +	} else {
> +		vfree(nvdev->send_buf);
> +	}
> +
>  	kfree(nvdev->send_section_map);
>=20
>  	for (i =3D 0; i < VRSS_CHANNEL_MAX; i++) {
> @@ -347,6 +360,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  	unsigned int buf_size;
>  	size_t map_words;
>  	int i, ret =3D 0;
> +	void *vaddr;
>=20
>  	/* Get receive buffer area. */
>  	buf_size =3D device_info->recv_sections * device_info->recv_section_siz=
e;
> @@ -382,6 +396,17 @@ static int netvsc_init_buf(struct hv_device *device,
>  		goto cleanup;
>  	}
>=20
> +	if (hv_isolation_type_snp()) {
> +		vaddr =3D hv_map_memory(net_device->recv_buf, buf_size);

Since the netvsc driver is architecture neutral, this code also needs
to compile for ARM64.  A stub will be needed for hv_map_memory()
on the ARM64 side.  Same for hv_unmap_memory() as suggested
above.  Or better, move hv_map_memory() and hv_unmap_memory()
to an architecture neutral module such as hv_common.c.

Or if Christop's approach of creating the vmap_phys_addr() helper
comes to fruition, that's an even better approach since it will already
handle multiple architectures.

> +		if (!vaddr) {
> +			ret =3D -ENOMEM;
> +			goto cleanup;
> +		}
> +
> +		net_device->recv_original_buf =3D net_device->recv_buf;
> +		net_device->recv_buf =3D vaddr;
> +	}
> +
>  	/* Notify the NetVsp of the gpadl handle */
>  	init_packet =3D &net_device->channel_init_pkt;
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
> @@ -485,6 +510,17 @@ static int netvsc_init_buf(struct hv_device *device,
>  		goto cleanup;
>  	}
>=20
> +	if (hv_isolation_type_snp()) {
> +		vaddr =3D hv_map_memory(net_device->send_buf, buf_size);
> +		if (!vaddr) {
> +			ret =3D -ENOMEM;
> +			goto cleanup;
> +		}
> +
> +		net_device->send_original_buf =3D net_device->send_buf;
> +		net_device->send_buf =3D vaddr;
> +	}
> +
>  	/* Notify the NetVsp of the gpadl handle */
>  	init_packet =3D &net_device->channel_init_pkt;
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
> @@ -775,7 +811,7 @@ static void netvsc_send_tx_complete(struct net_device=
 *ndev,
>=20
>  	/* Notify the layer above us */
>  	if (likely(skb)) {
> -		const struct hv_netvsc_packet *packet
> +		struct hv_netvsc_packet *packet
>  			=3D (struct hv_netvsc_packet *)skb->cb;
>  		u32 send_index =3D packet->send_buf_index;
>  		struct netvsc_stats *tx_stats;
> @@ -791,6 +827,7 @@ static void netvsc_send_tx_complete(struct net_device=
 *ndev,
>  		tx_stats->bytes +=3D packet->total_bytes;
>  		u64_stats_update_end(&tx_stats->syncp);
>=20
> +		netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>  		napi_consume_skb(skb, budget);
>  	}
>=20
> @@ -955,6 +992,87 @@ static void netvsc_copy_to_send_buf(struct netvsc_de=
vice *net_device,
>  		memset(dest, 0, padding);
>  }
>=20
> +void netvsc_dma_unmap(struct hv_device *hv_dev,
> +		      struct hv_netvsc_packet *packet)
> +{
> +	u32 page_count =3D packet->cp_partial ?
> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
> +		packet->page_buf_cnt;
> +	int i;
> +
> +	if (!hv_is_isolation_supported())
> +		return;
> +
> +	if (!packet->dma_range)
> +		return;
> +
> +	for (i =3D 0; i < page_count; i++)
> +		dma_unmap_single(&hv_dev->device, packet->dma_range[i].dma,
> +				 packet->dma_range[i].mapping_size,
> +				 DMA_TO_DEVICE);
> +
> +	kfree(packet->dma_range);
> +}
> +
> +/* netvsc_dma_map - Map swiotlb bounce buffer with data page of
> + * packet sent by vmbus_sendpacket_pagebuffer() in the Isolation
> + * VM.
> + *
> + * In isolation VM, netvsc send buffer has been marked visible to
> + * host and so the data copied to send buffer doesn't need to use
> + * bounce buffer. The data pages handled by vmbus_sendpacket_pagebuffer(=
)
> + * may not be copied to send buffer and so these pages need to be
> + * mapped with swiotlb bounce buffer. netvsc_dma_map() is to do
> + * that. The pfns in the struct hv_page_buffer need to be converted
> + * to bounce buffer's pfn. The loop here is necessary becuase the

s/becuase/because/

> + * entries in the page buffer array are not necessarily full
> + * pages of data.  Each entry in the array has a separate offset and
> + * len that may be non-zero, even for entries in the middle of the
> + * array.  And the entries are not physically contiguous.  So each
> + * entry must be individually mapped rather than as a contiguous unit.
> + * So not use dma_map_sg() here.
> + */
> +int netvsc_dma_map(struct hv_device *hv_dev,
> +		   struct hv_netvsc_packet *packet,
> +		   struct hv_page_buffer *pb)
> +{
> +	u32 page_count =3D  packet->cp_partial ?
> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
> +		packet->page_buf_cnt;
> +	dma_addr_t dma;
> +	int i;
> +
> +	if (!hv_is_isolation_supported())
> +		return 0;
> +
> +	packet->dma_range =3D kcalloc(page_count,
> +				    sizeof(*packet->dma_range),
> +				    GFP_KERNEL);
> +	if (!packet->dma_range)
> +		return -ENOMEM;
> +
> +	for (i =3D 0; i < page_count; i++) {
> +		char *src =3D phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
> +					 + pb[i].offset);
> +		u32 len =3D pb[i].len;
> +
> +		dma =3D dma_map_single(&hv_dev->device, src, len,
> +				     DMA_TO_DEVICE);
> +		if (dma_mapping_error(&hv_dev->device, dma)) {
> +			kfree(packet->dma_range);
> +			return -ENOMEM;
> +		}
> +
> +		packet->dma_range[i].dma =3D dma;
> +		packet->dma_range[i].mapping_size =3D len;
> +		pb[i].pfn =3D dma >> HV_HYP_PAGE_SHIFT;
> +		pb[i].offset =3D offset_in_hvpage(dma);
> +		pb[i].len =3D len;
> +	}

Just to confirm, this driver does *not* set the DMA min_align_mask
like storvsc does.  So after the call to dma_map_single(), the offset
in the page could be different.  That's why you are updating
the pb[i].offset value.  Alternatively, you could set the DMA
min_align_mask, which would ensure the offset is unchanged.
I'm OK with either approach, though perhaps a comment is
warranted to explain, as this is a subtle issue.

> +
> +	return 0;
> +}
> +
>  static inline int netvsc_send_pkt(
>  	struct hv_device *device,
>  	struct hv_netvsc_packet *packet,
> @@ -995,14 +1113,24 @@ static inline int netvsc_send_pkt(
>=20
>  	trace_nvsp_send_pkt(ndev, out_channel, rpkt);
>=20
> +	packet->dma_range =3D NULL;
>  	if (packet->page_buf_cnt) {
>  		if (packet->cp_partial)
>  			pb +=3D packet->rmsg_pgcnt;
>=20
> +		ret =3D netvsc_dma_map(ndev_ctx->device_ctx, packet, pb);
> +		if (ret) {
> +			ret =3D -EAGAIN;
> +			goto exit;
> +		}
> +
>  		ret =3D vmbus_sendpacket_pagebuffer(out_channel,
>  						  pb, packet->page_buf_cnt,
>  						  &nvmsg, sizeof(nvmsg),
>  						  req_id);
> +
> +		if (ret)
> +			netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>  	} else {
>  		ret =3D vmbus_sendpacket(out_channel,
>  				       &nvmsg, sizeof(nvmsg),
> @@ -1010,6 +1138,7 @@ static inline int netvsc_send_pkt(
>  				       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
>  	}
>=20
> +exit:
>  	if (ret =3D=3D 0) {
>  		atomic_inc_return(&nvchan->queue_sends);
>=20
> diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis=
_filter.c
> index f6c9c2a670f9..448fcc325ed7 100644
> --- a/drivers/net/hyperv/rndis_filter.c
> +++ b/drivers/net/hyperv/rndis_filter.c
> @@ -361,6 +361,8 @@ static void rndis_filter_receive_response(struct net_=
device *ndev,
>  			}
>  		}
>=20
> +		netvsc_dma_unmap(((struct net_device_context *)
> +			netdev_priv(ndev))->device_ctx, &request->pkt);
>  		complete(&request->wait_event);
>  	} else {
>  		netdev_err(ndev,
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index 724a735d722a..139a43ad65a1 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1596,6 +1596,11 @@ struct hyperv_service_callback {
>  	void (*callback)(void *context);
>  };
>=20
> +struct hv_dma_range {
> +	dma_addr_t dma;
> +	u32 mapping_size;
> +};
> +
>  #define MAX_SRV_VER	0x7ffffff
>  extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *icmsghdrp, u8 *b=
uf, u32 buflen,
>  				const int *fw_version, int fw_vercnt,
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:38:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176839.321863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLccH-0001CP-5Y; Thu, 02 Sep 2021 02:38:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176839.321863; Thu, 02 Sep 2021 02:38:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLccH-0001CI-2P; Thu, 02 Sep 2021 02:38:13 +0000
Received: by outflank-mailman (input) for mailman id 176839;
 Thu, 02 Sep 2021 02:38:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLccG-0001C8-7m; Thu, 02 Sep 2021 02:38:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLccG-0001S4-1a; Thu, 02 Sep 2021 02:38:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLccF-00031Y-R5; Thu, 02 Sep 2021 02:38:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLccF-0005HY-Qc; Thu, 02 Sep 2021 02:38:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=w6hoIB42nMN4YO4CMmTamcddJ9aKSe8e0uneQZYQVqk=; b=pjJWi8F9cNynfOSCLpHhQOvX1X
	zlxZsywv0XACX/Q5mAqb5NUBWnqJnPMMnrSOLPFNzDJsukYUl2JxonC8Qr/EelaCsSuD0moeLFYTt
	gYFCv6mzN+G3/Aw8yDeZzWVFzKVpKu6x1t+RElKWSZSz9YIgamY6J2k8deUGHgfqibEs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164737-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164737: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=12e33dca4c0612a0975265e5ba641c6261a26455
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 02:38:11 +0000

flight 164737 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164737/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 12e33dca4c0612a0975265e5ba641c6261a26455
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    1 days
Failing since        164686  2021-09-01 03:03:43 Z    0 days    7 attempts
Testing same since   164718  2021-09-01 16:40:02 Z    0 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 02:49:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 02:49:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176848.321882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcmx-0002uh-AZ; Thu, 02 Sep 2021 02:49:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176848.321882; Thu, 02 Sep 2021 02:49:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLcmx-0002ua-7S; Thu, 02 Sep 2021 02:49:15 +0000
Received: by outflank-mailman (input) for mailman id 176848;
 Thu, 02 Sep 2021 02:49:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do2V=NY=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLcmv-0002uU-W6
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 02:49:14 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0c::625])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4ea214a7-0b2b-42ba-a15c-c2ba08ef346f;
 Thu, 02 Sep 2021 02:49:11 +0000 (UTC)
Received: from AM6P191CA0012.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8b::25)
 by VE1PR08MB5741.eurprd08.prod.outlook.com (2603:10a6:800:1b3::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 02:49:07 +0000
Received: from AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8b:cafe::5a) by AM6P191CA0012.outlook.office365.com
 (2603:10a6:209:8b::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 02:49:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT030.mail.protection.outlook.com (10.152.16.117) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 02:49:05 +0000
Received: ("Tessian outbound 6ba9ec11d42b:v103");
 Thu, 02 Sep 2021 02:49:05 +0000
Received: from fe5c61d1ba8d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D0BD3C8B-8302-442D-A88F-722B8C8FC3DE.1; 
 Thu, 02 Sep 2021 02:48:58 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id fe5c61d1ba8d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 02:48:58 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB7PR08MB3532.eurprd08.prod.outlook.com (2603:10a6:10:4f::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 02:48:46 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%7]) with mapi id 15.20.4457.026; Thu, 2 Sep 2021
 02:48:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ea214a7-0b2b-42ba-a15c-c2ba08ef346f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=51bC7OwzeC89ccTfqq5DnigU2oiyXRd0TM+PRBRudxM=;
 b=wSLtzGuLZNWvxM58rp7NJDBWUCx6PPDfax8JLMBi7tI/iP3ffWxeWmzUGVqQqO3R0WFxFJCf/oa4tJ0KrEizsRibY/gxL2foCTQRiTeKfN25lMHaNryOpTD+zqrxtj6LgUBC2W1yGhNQAfCdNA9KVnAKaabE34pfmN/gPVFTKOQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gCB8cAkCmJaEICifainkhzoQRYdCKf5IrnwazooulxHDurnvncQLvs53O0knu7Ew5SrLRSuV7QlrR/Elyg+oO/Ieh4FUysB2RPnMOw1FIhGzrbBaVHwlBJWdj+hNSfBMzfToEUSnraTWGsMYsuPdx63eriaph8lM6Y4Cq1DVqbazU1jMASUoYHtkQw4No6RaAjfufldtFiSXH3B9e359kUgUo2CvgeA5uOa3X8Wwj2srecVrszOy31RhlQfYa5bSKHovD0XcmXWXKn97haYphRVdwhV7nfmctPVxgqwncuxYhC1rc1107e0j3FYoq+T2nbPNBO3ca/bPSbMsFlIs6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=51bC7OwzeC89ccTfqq5DnigU2oiyXRd0TM+PRBRudxM=;
 b=bPsXPs0mq4II6mza68Knz2YUPD7WgX6g0mAZRvQeJ2PlZ9ihxVT3ZfBAM83gaxnOBqEa8CAyP5UCXbIYyHyXVqWpsH3htDv1kkWQsJR4jV1kR8m4zfUI+02ksqc485hEAd2l0grXpxgPqhw25qNJWiwouApbzAL10Dxy4OUFYrIJgAU7GyFaVbaZnfGEBOXJgk9Dax3wxdHUeXP8OOxVo1idgG6UyopXp6danQxKQCsTVn8yMx4NAXgtVk0N3S1N+fGJNpS5Kvu//i/hRfJa2Z8GKtmJTAAh/dxvGWj6EuZqXhiiXCJfaUU5ITTpWux239B9r9ccYVpXRoBQVQyxUQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=51bC7OwzeC89ccTfqq5DnigU2oiyXRd0TM+PRBRudxM=;
 b=wSLtzGuLZNWvxM58rp7NJDBWUCx6PPDfax8JLMBi7tI/iP3ffWxeWmzUGVqQqO3R0WFxFJCf/oa4tJ0KrEizsRibY/gxL2foCTQRiTeKfN25lMHaNryOpTD+zqrxtj6LgUBC2W1yGhNQAfCdNA9KVnAKaabE34pfmN/gPVFTKOQ=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 25/40] xen/arm: unified entry to parse all NUMA
 data from device tree
Thread-Topic: [XEN RFC PATCH 25/40] xen/arm: unified entry to parse all NUMA
 data from device tree
Thread-Index: AQHXjptEfh486q3QYEyqJH3oIGMDv6uM6C8AgAEbO4CAAZ5dgIAAiwHA
Date: Thu, 2 Sep 2021 02:48:45 +0000
Message-ID:
 <DB9PR08MB68579F8A62FB53FBE8256FE49ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-26-wei.chen@arm.com>
 <alpine.DEB.2.21.2108301752501.17851@sstabellini-ThinkPad-T480s>
 <088e4b16-afea-8248-55f1-a081bba7f80c@xen.org>
 <alpine.DEB.2.21.2109011129050.17925@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109011129050.17925@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 83BC028FF5AB0541B4E6B989F2FBE67B.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b2a6cd65-eeec-49dc-65ed-08d96dbc3af3
x-ms-traffictypediagnostic: DB7PR08MB3532:|VE1PR08MB5741:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB5741A65EE950C464481BC0D09ECE9@VE1PR08MB5741.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 /YhbuRzgEnTa+Ydv3QJkCVdcVamAx0LTRUxPttkUMP+tD+Ic4sx94zgD5pqPkwnIF67CCE89sPdarWXGO1m8PxB0Qf2NIEFsRxOar9g9MvY0t9SVfXoM4Y+84LW4JO0HZXETd8g6TkezKiw+TtQM7fWdrozToNYXWlcdm8s0myHb3ZRKVRIsrX2ED23bHkzZanF1nVg429NMiwiEiAkUZzyfGCEApETN16eVvUq5OMTRp4NkFteEh/OLaRDGn6kTf2g7+sUkP+DmeO9Y6L6FEQORwt/+E8wrTf/8DNa77Ysnlv4jeeQEZxZIxrSNTQiVBQWejsD3+Tz6xRIX63YaAJrMxOEuPvxmQY0SSTwZU5pqYhO8TdFjret8jD9YEjsvL/PdxfCT40iDPkngTAJDnPM8wcVcpvq+hlLDWLPKzuNKzCISbkxnlrBa6/8XG5O2PaprQQVrEN9GMHdKzg7slnMWP4l/QApU/IuD2vIeoey/mM9ljsJWcnj2Wjiz8E54FCv2sU48DNUWqj5Rsr6UOuedPY6bNrYo1YNWfgwnsvkc8nc2Mbjtu89Kbob8MCzNkutMf+hVjtkNBCGgxDc4A50tJMcBggQxtL7S0/jH+sV8dY8crrxADrOg4U98F9uhhntCoRsGqyT07cbEDmr9yKH8o6Q1Qs7UBirkGq0gLxsfNGLSmm+DDtHDlS9HwVDXR23Nvtgo5hPGNjTEUlZ8Rw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(39860400002)(346002)(136003)(376002)(8936002)(83380400001)(71200400001)(38100700002)(6506007)(53546011)(5660300002)(122000001)(64756008)(26005)(66946007)(38070700005)(76116006)(66476007)(66446008)(66556008)(52536014)(478600001)(4326008)(186003)(2906002)(9686003)(316002)(55016002)(7696005)(54906003)(8676002)(33656002)(86362001)(110136005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3532
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b239dab7-fef3-4b94-bf6b-08d96dbc2f26
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DJHCDxfW0ZJjUtOVIHPJ818s7g+bG4cIP4a6KTrQxNpaj2OgeSqyJ7k8SnM9F0CWJHjgzRq6mswUOdiSu6VVwNY3p06KQUKy/iihtRfNWocxZzQzcNm2DLuh1Xfh1ZjOtHS+sQXQwNYqtnR9hEzwgM7VdRfhiDxk7KLOH1iN+1JARgHi92uYgKay/dXfKuCZ3wUgWV6rrkVq0Nk+UGabibkOh2z9OKFe6a5m11uv+kAwXZkZA9oP7wA9VgGeTIaW3I0S6ruCpT7t3SKtQCyzAvVGQpWDGlbOsNtREjCQUmHydt3MGEFDpQ3NI5Xr1dCSqljW5XGZkASBsIp9oGJLJich712d2BFycH66MvMP9Br8LrKK7IvWozKRhUObtPdB9psaik48jFk3/FYOK4K37bEeaMyMC7nZIWLRD1Sd1smBLYSxssHiBBUBdy10JrucqCEsHABGuWkYyoVHAkiRMFNYlZH9o0PE3IFTOQS8mBkjyn/c49t0hiTCl62VMYzvk2pMVrFsZXv5Tcbp3tNQyW101duXgyi5+aiVO+CzsA7gcL2aC7EuZMfSofejIEzag4eMGUy2kY8WdESOWnyNFouKcM5gwiifjVETy9XIwE0DaHAYJHOpa0sdC2JaW35g7sSfA5cbhpA8Plw20ToTszQuvLO/qxk6pA2NGY/VTrt+3jgLg460Ukw0hyYXp7GSPzL/Rfy5faJsCWy2r3KLgg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(46966006)(36840700001)(356005)(9686003)(8676002)(55016002)(8936002)(5660300002)(81166007)(316002)(52536014)(4326008)(33656002)(82740400003)(110136005)(54906003)(478600001)(7696005)(26005)(53546011)(83380400001)(186003)(70206006)(70586007)(336012)(47076005)(86362001)(82310400003)(36860700001)(2906002)(6506007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 02:49:05.9195
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b2a6cd65-eeec-49dc-65ed-08d96dbc3af3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5741

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiBTZW50OiAyMDIxxOo51MIyyNUgMjozMQ0K
PiBUbzogSnVsaWVuIEdyYWxsIDxqdWxpZW5AeGVuLm9yZz4NCj4gQ2M6IFN0ZWZhbm8gU3RhYmVs
bGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IFdlaSBDaGVuDQo+IDxXZWkuQ2hlbkBhcm0u
Y29tPjsgeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBqYmV1bGljaEBzdXNlLmNvbTsN
Cj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPg0KPiBTdWJqZWN0
OiBSZTogW1hFTiBSRkMgUEFUQ0ggMjUvNDBdIHhlbi9hcm06IHVuaWZpZWQgZW50cnkgdG8gcGFy
c2UgYWxsDQo+IE5VTUEgZGF0YSBmcm9tIGRldmljZSB0cmVlDQo+IA0KPiBPbiBUdWUsIDMxIEF1
ZyAyMDIxLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+ID4gSGkgU3RlZmFubywNCj4gPg0KPiA+IE9u
IDMxLzA4LzIwMjEgMDE6NTQsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4gPiA+IE9uIFdl
ZCwgMTEgQXVnIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiBJbiB0aGlzIEFQSSwgd2Ug
c2NhbiB3aG9sZSBkZXZpY2UgdHJlZSB0byBwYXJzZSBDUFUgbm9kZSBpZCwgbWVtb3J5DQo+ID4g
PiA+IG5vZGUgaWQgYW5kIGRpc3RhbmNlLW1hcC4gVGhvdWdoIGVhcmx5X3NjYW5fbm9kZSB3aWxs
IGludm9rZSBoYXMgYQ0KPiA+ID4gPiBoYW5kbGVyIHRvIHByb2Nlc3MgbWVtb3J5IG5vZGVzLiBJ
ZiB3ZSB3YW50IHRvIHBhcnNlIG1lbW9yeSBub2RlIGlkDQo+ID4gPiA+IGluIHRoaXMgaGFuZGxl
ciwgd2UgaGF2ZSB0byBlbWJlZGVkIE5VTUEgcGFyc2UgY29kZSBpbiB0aGlzIGhhbmRsZXIuDQo+
ID4gPiA+IEJ1dCB3ZSBzdGlsbCBuZWVkIHRvIHNjYW4gd2hvbGUgZGV2aWNlIHRyZWUgdG8gZmlu
ZCBDUFUgTlVNQSBpZCBhbmQNCj4gPiA+ID4gZGlzdGFuY2UtbWFwLiBJbiB0aGlzIGNhc2UsIHdl
IGluY2x1ZGUgbWVtb3J5IE5VTUEgaWQgcGFyc2UgaW4gdGhpcw0KPiA+ID4gPiBBUEkgdG9vLiBB
bm90aGVyIGJlbmVmaXQgaXMgdGhhdCB3ZSBoYXZlIGEgdW5pcXVlIGVudHJ5IGZvciBkZXZpY2UN
Cj4gPiA+ID4gdHJlZSBOVU1BIGRhdGEgcGFyc2UuDQo+ID4gPiA+DQo+ID4gPiA+IFNpZ25lZC1v
ZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29tPg0KPiA+ID4gPiAtLS0NCj4gPiA+ID4g
ICB4ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jIHwgMzEgKysrKysrKysrKysrKysrKysr
KysrKysrKysrKy0NCj4gLS0NCj4gPiA+ID4gICB4ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaCAg
ICAgIHwgIDEgKw0KPiA+ID4gPiAgIDIgZmlsZXMgY2hhbmdlZCwgMjkgaW5zZXJ0aW9ucygrKSwg
MyBkZWxldGlvbnMoLSkNCj4gPiA+ID4NCj4gPiA+ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2Fy
bS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+ID4gYi94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2Vf
dHJlZS5jDQo+ID4gPiA+IGluZGV4IDZlMGQxZDNkOWYuLjI3ZmZiNzJmN2IgMTAwNjQ0DQo+ID4g
PiA+IC0tLSBhL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+ID4gKysrIGIv
eGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+ID4gPiBAQCAtMTMxLDcgKzEzMSw4
IEBAIHNhdmVfbWVtYmxrOg0KPiA+ID4gPiAgIH0NCj4gPiA+ID4gICAgIC8qIFBhcnNlIENQVSBO
VU1BIG5vZGUgaW5mbyAqLw0KPiA+ID4gPiAtaW50IF9faW5pdCBkZXZpY2VfdHJlZV9wYXJzZV9u
dW1hX2NwdV9ub2RlKGNvbnN0IHZvaWQgKmZkdCwgaW50DQo+IG5vZGUpDQo+ID4gPiA+ICtzdGF0
aWMgaW50IF9faW5pdA0KPiA+ID4gPiArZGV2aWNlX3RyZWVfcGFyc2VfbnVtYV9jcHVfbm9kZShj
b25zdCB2b2lkICpmZHQsIGludCBub2RlKQ0KPiA+ID4gPiAgIHsNCj4gPiA+ID4gICAgICAgdWlu
dDMyX3QgbmlkOw0KPiA+ID4gPiAgIEBAIC0xNDcsNyArMTQ4LDcgQEAgaW50IF9faW5pdA0KPiBk
ZXZpY2VfdHJlZV9wYXJzZV9udW1hX2NwdV9ub2RlKGNvbnN0DQo+ID4gPiA+IHZvaWQgKmZkdCwg
aW50IG5vZGUpDQo+ID4gPiA+ICAgfQ0KPiA+ID4gPiAgICAgLyogUGFyc2UgbWVtb3J5IG5vZGUg
TlVNQSBpbmZvICovDQo+ID4gPiA+IC1pbnQgX19pbml0DQo+ID4gPiA+ICtzdGF0aWMgaW50IF9f
aW5pdA0KPiA+ID4gPiAgIGRldmljZV90cmVlX3BhcnNlX251bWFfbWVtb3J5X25vZGUoY29uc3Qg
dm9pZCAqZmR0LCBpbnQgbm9kZSwNCj4gPiA+ID4gICAgICAgY29uc3QgY2hhciAqbmFtZSwgdWlu
dDMyX3QgYWRkcl9jZWxscywgdWludDMyX3Qgc2l6ZV9jZWxscykNCj4gPiA+ID4gICB7DQo+ID4g
PiA+IEBAIC0yMDIsNyArMjAzLDcgQEAgZGV2aWNlX3RyZWVfcGFyc2VfbnVtYV9tZW1vcnlfbm9k
ZShjb25zdCB2b2lkDQo+ICpmZHQsDQo+ID4gPiA+IGludCBub2RlLA0KPiA+ID4gPiAgIH0NCj4g
PiA+ID4gICAgIC8qIFBhcnNlIE5VTUEgZGlzdGFuY2UgbWFwIHYxICovDQo+ID4gPiA+IC1pbnQg
X19pbml0DQo+ID4gPiA+ICtzdGF0aWMgaW50IF9faW5pdA0KPiA+ID4gPiAgIGRldmljZV90cmVl
X3BhcnNlX251bWFfZGlzdGFuY2VfbWFwX3YxKGNvbnN0IHZvaWQgKmZkdCwgaW50IG5vZGUpDQo+
ID4gPiA+ICAgew0KPiA+ID4gPiAgICAgICBjb25zdCBzdHJ1Y3QgZmR0X3Byb3BlcnR5ICpwcm9w
Ow0KPiA+ID4gPiBAQCAtMjY3LDMgKzI2OCwyNyBAQCBkZXZpY2VfdHJlZV9wYXJzZV9udW1hX2Rp
c3RhbmNlX21hcF92MShjb25zdA0KPiB2b2lkDQo+ID4gPiA+ICpmZHQsIGludCBub2RlKQ0KPiA+
ID4gPiAgICAgICAgIHJldHVybiAwOw0KPiA+ID4gPiAgIH0NCj4gPiA+ID4gKw0KPiA+ID4gPiAr
c3RhdGljIGludCBfX2luaXQgZmR0X3NjYW5fbnVtYV9ub2Rlcyhjb25zdCB2b2lkICpmZHQsDQo+
ID4gPiA+ICsgICAgICAgICAgICAgICAgaW50IG5vZGUsIGNvbnN0IGNoYXIgKnVuYW1lLCBpbnQg
ZGVwdGgsDQo+ID4gPiA+ICsgICAgICAgICAgICAgICAgdTMyIGFkZHJlc3NfY2VsbHMsIHUzMiBz
aXplX2NlbGxzLCB2b2lkICpkYXRhKQ0KPiA+ID4gPiArew0KPiA+ID4gPiArICAgIGludCByZXQg
PSAwOw0KPiA+ID4gPiArDQo+ID4gPiA+ICsgICAgaWYgKCBmZHRfbm9kZV9jaGVja190eXBlKGZk
dCwgbm9kZSwgImNwdSIpID09IDAgKQ0KPiA+ID4gPiArICAgICAgICByZXQgPSBkZXZpY2VfdHJl
ZV9wYXJzZV9udW1hX2NwdV9ub2RlKGZkdCwgbm9kZSk7DQo+ID4gPiA+ICsgICAgZWxzZSBpZiAo
IGZkdF9ub2RlX2NoZWNrX3R5cGUoZmR0LCBub2RlLCAibWVtb3J5IikgPT0gMCApDQo+ID4gPiA+
ICsgICAgICAgIHJldCA9IGRldmljZV90cmVlX3BhcnNlX251bWFfbWVtb3J5X25vZGUoZmR0LCBu
b2RlLCB1bmFtZSwNCj4gPiA+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWRk
cmVzc19jZWxscywgc2l6ZV9jZWxscyk7DQo+ID4gPiA+ICsgICAgZWxzZSBpZiAoIGZkdF9ub2Rl
X2NoZWNrX2NvbXBhdGlibGUoZmR0LCBub2RlLA0KPiA+ID4gPiArICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAibnVtYS1kaXN0YW5jZS1tYXAtdjEiKSA9PSAwICkNCj4gPiA+ID4gKyAg
ICAgICAgcmV0ID0gZGV2aWNlX3RyZWVfcGFyc2VfbnVtYV9kaXN0YW5jZV9tYXBfdjEoZmR0LCBu
b2RlKTsNCj4gPiA+ID4gKw0KPiA+ID4gPiArICAgIHJldHVybiByZXQ7DQo+ID4gPiA+ICt9DQo+
ID4gPg0KPiA+ID4gSnVsaWVuLCBkbyB5b3UgaGF2ZSBhbiBvcGluaW9uIG9uIHdoZXRoZXIgaXQg
bWlnaHQgYmUgd29ydGggcmV1c2luZw0KPiB0aGUNCj4gPiA+IGV4aXN0aW5nIGVhcmx5X3NjYW5f
bm9kZSBmdW5jdGlvbiBmb3IgdGhpcyB0byBhdm9pZGluZyBhbm90aGVyIGZ1bGwNCj4gRkRUDQo+
ID4gPiBzY2FuICh0byBhdm9pZCBhbm90aGVyIGNhbGwgdG8gZGV2aWNlX3RyZWVfZm9yX2VhY2hf
bm9kZSk/DQo+ID4NCj4gPiBJIGRvbid0IGxpa2UgdGhlIGZ1bGwgRkRUIHNjYW4gYW5kIGFjdHVh
bGx5IGRyYWZ0ZWQgYW4gZS1tYWlsIGluIHJlcGx5LQ0KPiB0byBbMV0NCj4gPiB0byBzdWdnZXN0
IHBhcnNlIGFsbCB0aGUgTlVNQSBpbmZvcm1hdGlvbiBmcm9tIGVhcmx5X3NjYW5fbm9kZSgpLg0K
PiA+DQo+ID4gSG93ZXZlciwgd2UgZG9uJ3Qga25vdyB3aGV0aGVyIEFDUEkgb3IgRFQgd2lsbCBi
ZSB1c2VkIGF0IHRoZSB0aW1lDQo+ID4gZWFybHlfc2Nhbl9ub2RlKCkgaXMgY2FsbGVkLiBTbyB3
ZSB3aWxsIG5lZWQgdG8gcmV2ZXJ0IGFueSBjaGFuZ2Ugd2hpY2gNCj4gY2FuDQo+ID4gbWFrZSB0
aGUgY29kZSBhIGxpdHRsZSBtb3JlIGF3a3dhcmQuDQo+ID4NCj4gPiBTbyBJIGRlY2lkZWQgdG8g
ZHJvcCBteSBlLW1haWwgYmVjYXVzZSBJIHByZWZlciB0aGUgZnVsbCBEVCBzY2FuIGZvciBub3cu
DQo+IFdlDQo+ID4gY2FuIGxvb2sgYXQgb3B0aW1pemluZyBsYXRlciBpZiB0aGlzIGJlY29tZXMg
YSBwYWluIHBvaW50Lg0KPiANCj4gVWhtLCB5ZXMgeW91IGFyZSByaWdodC4NCj4gDQo+IFdlIHdv
dWxkIGhhdmUgdG8gbW92ZSBzb21lIG9mIHRoZSBsb2dpYyB0byBkZXRlY3QgQUNQSSBlYXJsaWVy
IHRvDQo+IGVhcmx5X3NjYW5fbm9kZSAoZS5nLiB4ZW4vYXJjaC9hcm0vYWNwaS9ib290LmM6ZHRf
c2Nhbl9kZXB0aDFfbm9kZXMpLg0KPiBUaGF0IGNvdWxkIGFjdHVhbGx5IGJlIGEgZ29vZCBpZGVh
LCBidXQgaXQgaXMgdHJ1ZSB0aGF0IGNvdWxkIGJlIGRvbmUNCj4gd2l0aCBhIHNlcGFyYXRlIHBh
dGNoIGxhdGVyLg0KDQpTb3VuZHMgZ29vZCB0byBtZS4NCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 03:23:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 03:23:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176857.321900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLdJk-0007tE-84; Thu, 02 Sep 2021 03:23:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176857.321900; Thu, 02 Sep 2021 03:23:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLdJk-0007t7-4b; Thu, 02 Sep 2021 03:23:08 +0000
Received: by outflank-mailman (input) for mailman id 176857;
 Thu, 02 Sep 2021 03:23:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLdJi-0007sx-Vo; Thu, 02 Sep 2021 03:23:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLdJi-0002Ds-Mq; Thu, 02 Sep 2021 03:23:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLdJi-0004gT-Fz; Thu, 02 Sep 2021 03:23:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLdJi-0003tq-FX; Thu, 02 Sep 2021 03:23:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=kmxWLjMTj8eyU39JWMio7wg43TDTEDDg/N+n27aWTf0=; b=cQoJk3XJaCDOOouQD14B+1VgEa
	AZdcgisHfSPiXv/FwnV9TPb1wPa+Rsh5o/TK3aYpO11gImEWiVxIhb9yakwpN/p7heDMXv5ZOGgYs
	d5AWBmbEucTPNW96IGlOpu7jjJKi5yxu1dCac6fDrcwQmZO5I455LDv24DhvUVYSEu8A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164740-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164740: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 03:23:06 +0000

flight 164740 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164740/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days    8 attempts
Testing same since   164740  2021-09-02 02:41:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 03:32:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 03:32:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176864.321914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLdSd-00019H-8L; Thu, 02 Sep 2021 03:32:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176864.321914; Thu, 02 Sep 2021 03:32:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLdSd-00019A-3c; Thu, 02 Sep 2021 03:32:19 +0000
Received: by outflank-mailman (input) for mailman id 176864;
 Thu, 02 Sep 2021 03:32:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLdSc-000191-Ap
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 03:32:18 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5c1d418a-0b9e-11ec-ae3a-12813bfff9fa;
 Thu, 02 Sep 2021 03:32:12 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1034.namprd21.prod.outlook.com (2603:10b6:302:a::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.5; Thu, 2 Sep
 2021 03:32:03 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 03:32:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c1d418a-0b9e-11ec-ae3a-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JQbXSpgvwBAqzskd+XYzXa9qd1ZwoapNxtxzqozS6nQxvzYigLh+evBweV2kRwNBwz7ekzLu2XVsXXi2v7i5ItqEu7BuwK5CdgKyPLq1DcbYN7pVpwDgkTf/UG6qUz7Z9HjWehMaJ3ETymj3b1I+CrfC9x22KDGThi/fUW8hMT/V7cXb3J2EFCGQaXeDqUv/1y0iSIZfwv76cYWZNfg1aCrnC+RGZ1cQ7NrR4ckbQgG9StnsAb+X1FqStW11xRTsVvTttdjrisGXI9dBqy0x/ikW91fE2YbLvlb48ANt13T3TdYHRpB3YtQy/4GCnpBnGVurTLZw71gBiVGIOUNInw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jphdFbQcqCygNrEGqinmRvb9upDBy+g/qvA+rRnOhP8=;
 b=SYYxwG0OsrpLcH4Q6CDa18r/qgihczGLjh7H3l0GDkZ+w53MzhVPPYaI5n3cKFjlWv4lOOLOM0uXIjJDG3KriE3w+ot7G4cNolUNacC3bEOdyxLaATCgROOyzrDkpv3/Z77QRHreailtiyGDQENGEwq50ixH/7jz1aF2VNUPinI8FcpcWnckIygsfHfrABHt4INlXI6o9X4MAB3ONKUVaBOkz1FCR7jaz00S7f+edoPhdqPB8ZybkE1Lr+v4Kbe/30fslCM3DNf+jXHGKilq7St6Mzdh6sCvIiWtOC8m/q9axraXxVMu86hH6/HQCvcOYivaQz+rj9e6GWYhI9lY5g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jphdFbQcqCygNrEGqinmRvb9upDBy+g/qvA+rRnOhP8=;
 b=NYdFsYXX4yDXiFL7Tfpi03UbXXyFUwBxfcu1Ah8CAumhgn+kuBMN3OmR6XtBqEgcVgzUfQ6PZwBKhP0pmzP3DEM9JrHQibw3t2Q7JYVCdeZgNnyZ3uWYG0XFQxR7C6H0Rjm7ulI+IStOpsC8s0xiVm5z/vgnpcAWWAIUA9NoBcE=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 05/13] hyperv: Add Write/Read MSR registers via ghcb
 page
Thread-Topic: [PATCH V4 05/13] hyperv: Add Write/Read MSR registers via ghcb
 page
Thread-Index: AQHXm2gCMSN7/9Nrs0WqDjRR9LMyqquJGfKw
Date: Thu, 2 Sep 2021 03:32:03 +0000
Message-ID:
 <MWHPR21MB15937EFF970147CEC61DE227D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-6-ltykernel@gmail.com>
In-Reply-To: <20210827172114.414281-6-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=2fbe6f01-ab97-4a86-a26b-7e5e56e5a33e;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-08-28T16:18:47Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c1470254-253b-409a-9a29-08d96dc23b3a
x-ms-traffictypediagnostic: MW2PR2101MB1034:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB1034343B123E288F391ABD5BD7CE9@MW2PR2101MB1034.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:127;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 qLKUGIr7FLuydpSAU/8sGZ+xE0tOXbhOYJ7JeyuXCj9gMJI28oJ3xLcB4MCazN7HHAvjeS/Ud+l753dD0ZgVT32um1Mx/ZD4d3JGXji6s1KYAk2q/Ky7l1e6RpE4luX4rVOd2gioJu02xeJ1Aykf0gm9CBC1ls0Y7GoMY/fXKZLqb/YDAYTop61raq2HF1DmaR6PWhwR8UBxSKLG3LQIlJRGNyEs/mP+NJ7hLnto3iWe9VOSI2EL4R5MSSvzKnR2sSsTHZiZWAQXwHiYyuANUakgaeMB0W0rg+4e59ukpIrxXPr7LgL5SEJJN51HxfeCeipaQgvmY6AWlaejF0wEqT1AG0D6yBxBgDAYqBH/bXPgz3AmPiDiklOXI3NRciHQB78G480xQhGsxw8dGMOV8M/jIj2ZzM68PRe/LQmFOBvJeXkeYGOcZ84915Gc6x0qWvbabCrcCbTnS/4iPN0/IJJ5QVqMAWDQsdaCVr99YfIEB0caHRVaFroyNOxyl482Pa+ubJDVZh78bkE7u6z4j4OBcAi+RKLbA7AmHVqIFUB4wt4IAJJSu+x/bgltGK/drCUWhD6/7a6NJyOIZ5lwhqPeZDaaXWRgkyrr8JIdkxsAXg1Y/SRnBFF4K/knyXAURiDQQRuJr2+K0HE0bgLFRdZq/RD9voETWt9mSxaikD8NUlZss/nJG6Pxidly8w19YOIChxXKJgnHvYf8YWwekGWryuTPyi7juTWMYD9k9Oc=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8990500004)(55016002)(8676002)(186003)(30864003)(26005)(9686003)(82960400001)(82950400001)(71200400001)(83380400001)(38070700005)(8936002)(7406005)(38100700002)(33656002)(52536014)(54906003)(66946007)(4326008)(66446008)(66476007)(64756008)(122000001)(66556008)(76116006)(10290500003)(2906002)(921005)(6506007)(5660300002)(110136005)(7416002)(7696005)(7366002)(508600001)(316002)(86362001)(559001)(579004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?B5IQac8XXoI6m6n27Qn50J6iupQuKS7E221ZRdCyye0Le6JfVsFtHXomwvMo?=
 =?us-ascii?Q?XfbrEc7tBAsCxOGO1hqQqFa4AlZEP8weaZ3czpp1mh02Z8NUkto3s+Xb29Q4?=
 =?us-ascii?Q?f24GPCTtJl1AObkdHjme+sYyrQ4ZylFSkzf3Bowsjh5GWXuzZSh8JCxDTHby?=
 =?us-ascii?Q?LULMRPrEK+Tr9s3EGYI7C+aYElVfg60PIUOFq2Rl95c95QD4P/1DyTMnL18+?=
 =?us-ascii?Q?J04O8qCtd3Ka2gIQrVUrXKxzlAPY3yyaIeP96cjbqq1kX5HKRsUfD/PV4BGw?=
 =?us-ascii?Q?XzQdDuXxmoQ5pVuA56Q52Yat9ut3sXYnBuGQttTrvcQrLZTLKJX7LiuaKXvR?=
 =?us-ascii?Q?ePMuIBTrjig7U0RXgO7C0Ui5eDHX9BF/PJ7XuopGyQkJ+/8uaz3UcNdu8Gev?=
 =?us-ascii?Q?aCYwKzLS0+c9lb6CRTZ6x+H2QviqPLPFjdDxKAsEMmBc8z7+Oagif1znE/Ui?=
 =?us-ascii?Q?VaZ0BezrilepyKsTV6cG4PYN5orWGEmPtq4asC1dT8r9LVcnuuj2xm1e5UTy?=
 =?us-ascii?Q?ka5Sv5xrAJLHFLMYD7XMWEch4S/yT1nsyySt8p6vq5awiBnaonh3gahObBnl?=
 =?us-ascii?Q?3jLwlVS4j28w3VfIWQ++wagwye0kXVlGoRdMrm8jDXOiO834heMF8JzTGi7L?=
 =?us-ascii?Q?RhKApIg8+zs713eMKa7IZWbdQEL41BVLrfj+FsfB9b4HQULQ/fbNwI0ssvAf?=
 =?us-ascii?Q?DYLP4rZwTkOoy21QOkbNqbEcQzgxC/b7tmQO6QRG7I0FTwLdLwQEWa3x7QgI?=
 =?us-ascii?Q?7QKeLVa5J19B0vHzKNpJpopoTzgQfDdnUS74fmx56RxYe3hHjZyhruIizOpu?=
 =?us-ascii?Q?p4DJ5iKvDcs+PC/E7+7cy5Jq3Xl70nR6ahz5k+VI1nx+G5qYg+l6lmuyH7Ey?=
 =?us-ascii?Q?x00pUuMmo74nxSLcFNm6w5UJ4q+MErjqYtzr7WvsUZX9HJnpAmt9r6rRbVwP?=
 =?us-ascii?Q?ngtc6ZPwe+mrEVL8ZXGdOEpKEnv/WvbGTD7HQ9eCUjEYLf7nrzjz7tKd5Wcs?=
 =?us-ascii?Q?gQEUp1h/X7YLNN3rLNozyBSOY0QzQMqXZ/cKb4vn4MKAa+vdcKwDOram6ZED?=
 =?us-ascii?Q?VBLnoWrEW8zHbwU98vQlAFri+Edn6VozfE79m1W1STEOYqEl5cD9RA/NjE4i?=
 =?us-ascii?Q?ASPXqecl8/hRFAWwUipQMqEr1fMI/myLPDwR0LAgduOVjsN0WRQTQPMwSCi7?=
 =?us-ascii?Q?Q7PvBfx5Gp39rfUvbQFxh7Hq8o+wp0JeK+UGuAPFIxjrCwIDKMDvQdmrI//9?=
 =?us-ascii?Q?Fh1YTE56V03umqHp/K2tB3TQwL+xGPZqqIdb0Aees/ni0+ha6YrW0H7cCq3S?=
 =?us-ascii?Q?qkVPssI+4XCiw8lC5sntpFL4?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c1470254-253b-409a-9a29-08d96dc23b3a
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 03:32:03.1186
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vPzNSqamS7p5aXRDNKV/5JzW1XO7DWUJFW0QQGcNNAlVGQK0/oQWaD5sx5rOxa0wqmrqVlRCyigO1wAzb/PRDCUXClSEn+7Xor8QiiWC55I=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1034

From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 =
AM
>=20
> Hyperv provides GHCB protocol to write Synthetic Interrupt
> Controller MSR registers in Isolation VM with AMD SEV SNP
> and these registers are emulated by hypervisor directly.
> Hyperv requires to write SINTx MSR registers twice. First
> writes MSR via GHCB page to communicate with hypervisor
> and then writes wrmsr instruction to talk with paravisor
> which runs in VMPL0. Guest OS ID MSR also needs to be set
> via GHCB page.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v1:
>          * Introduce sev_es_ghcb_hv_call_simple() and share code
>            between SEV and Hyper-V code.
> Change since v3:
>          * Pass old_msg_type to hv_signal_eom() as parameter.
> 	 * Use HV_REGISTER_* marcro instead of HV_X64_MSR_*
> 	 * Add hv_isolation_type_snp() weak function.
> 	 * Add maros to set syinc register in ARM code.
> ---
>  arch/arm64/include/asm/mshyperv.h |  23 ++++++
>  arch/x86/hyperv/hv_init.c         |  36 ++--------
>  arch/x86/hyperv/ivm.c             | 112 ++++++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h   |  80 ++++++++++++++++++++-
>  arch/x86/include/asm/sev.h        |   3 +
>  arch/x86/kernel/sev-shared.c      |  63 ++++++++++-------
>  drivers/hv/hv.c                   | 112 ++++++++++++++++++++----------
>  drivers/hv/hv_common.c            |   6 ++
>  include/asm-generic/mshyperv.h    |   4 +-
>  9 files changed, 345 insertions(+), 94 deletions(-)
>=20
> diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/m=
shyperv.h
> index 20070a847304..ced83297e009 100644
> --- a/arch/arm64/include/asm/mshyperv.h
> +++ b/arch/arm64/include/asm/mshyperv.h
> @@ -41,6 +41,29 @@ static inline u64 hv_get_register(unsigned int reg)
>  	return hv_get_vpreg(reg);
>  }
>=20
> +#define hv_get_simp(val)	{ val =3D hv_get_register(HV_REGISTER_SIMP); }
> +#define hv_set_simp(val)	hv_set_register(HV_REGISTER_SIMP, val)
> +
> +#define hv_get_siefp(val)	{ val =3D hv_get_register(HV_REGISTER_SIEFP); =
}
> +#define hv_set_siefp(val)	hv_set_register(HV_REGISTER_SIEFP, val)
> +
> +#define hv_get_synint_state(int_num, val) {			\
> +	val =3D hv_get_register(HV_REGISTER_SINT0 + int_num);	\
> +	}
> +
> +#define hv_set_synint_state(int_num, val)			\
> +	hv_set_register(HV_REGISTER_SINT0 + int_num, val)
> +
> +#define hv_get_synic_state(val) {			\
> +	val =3D hv_get_register(HV_REGISTER_SCONTROL);	\
> +	}
> +
> +#define hv_set_synic_state(val)			\
> +	hv_set_register(HV_REGISTER_SCONTROL, val)
> +
> +#define hv_signal_eom(old_msg_type)		 \
> +	hv_set_register(HV_REGISTER_EOM, 0)
> +
>  /* SMCCC hypercall parameters */
>  #define HV_SMCCC_FUNC_NUMBER	1
>  #define HV_FUNC_ID	ARM_SMCCC_CALL_VAL(			\
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index b1aa42f60faa..be6210a3fd2f 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -37,7 +37,7 @@ EXPORT_SYMBOL_GPL(hv_current_partition_id);
>  void *hv_hypercall_pg;
>  EXPORT_SYMBOL_GPL(hv_hypercall_pg);
>=20
> -void __percpu **hv_ghcb_pg;
> +union hv_ghcb __percpu **hv_ghcb_pg;
>=20
>  /* Storage to save the hypercall page temporarily for hibernation */
>  static void *hv_hypercall_pg_saved;
> @@ -406,7 +406,7 @@ void __init hyperv_init(void)
>  	}
>=20
>  	if (hv_isolation_type_snp()) {
> -		hv_ghcb_pg =3D alloc_percpu(void *);
> +		hv_ghcb_pg =3D alloc_percpu(union hv_ghcb *);
>  		if (!hv_ghcb_pg)
>  			goto free_vp_assist_page;
>  	}
> @@ -424,6 +424,9 @@ void __init hyperv_init(void)
>  	guest_id =3D generate_guest_id(0, LINUX_VERSION_CODE, 0);
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, guest_id);
>=20
> +	/* Hyper-V requires to write guest os id via ghcb in SNP IVM. */
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, guest_id);
> +
>  	hv_hypercall_pg =3D __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START,
>  			VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX,
>  			VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
> @@ -501,6 +504,7 @@ void __init hyperv_init(void)
>=20
>  clean_guest_os_id:
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
>  	cpuhp_remove_state(cpuhp);
>  free_ghcb_page:
>  	free_percpu(hv_ghcb_pg);
> @@ -522,6 +526,7 @@ void hyperv_cleanup(void)
>=20
>  	/* Reset our OS id */
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
>=20
>  	/*
>  	 * Reset hypercall page reference before reset the page,
> @@ -592,30 +597,3 @@ bool hv_is_hyperv_initialized(void)
>  	return hypercall_msr.enable;
>  }
>  EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
> -
> -enum hv_isolation_type hv_get_isolation_type(void)
> -{
> -	if (!(ms_hyperv.priv_high & HV_ISOLATION))
> -		return HV_ISOLATION_TYPE_NONE;
> -	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
> -}
> -EXPORT_SYMBOL_GPL(hv_get_isolation_type);
> -
> -bool hv_is_isolation_supported(void)
> -{
> -	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
> -		return 0;
> -
> -	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
> -		return 0;
> -
> -	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
> -}
> -
> -DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
> -
> -bool hv_isolation_type_snp(void)
> -{
> -	return static_branch_unlikely(&isolation_type_snp);
> -}
> -EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> index a069c788ce3c..f56fe4f73000 100644
> --- a/arch/x86/hyperv/ivm.c
> +++ b/arch/x86/hyperv/ivm.c
> @@ -6,13 +6,125 @@
>   *  Tianyu Lan <Tianyu.Lan@microsoft.com>
>   */
>=20
> +#include <linux/types.h>
> +#include <linux/bitfield.h>
>  #include <linux/hyperv.h>
>  #include <linux/types.h>
>  #include <linux/bitfield.h>
>  #include <linux/slab.h>
> +#include <asm/svm.h>
> +#include <asm/sev.h>
>  #include <asm/io.h>
>  #include <asm/mshyperv.h>
>=20
> +union hv_ghcb {
> +	struct ghcb ghcb;
> +} __packed __aligned(HV_HYP_PAGE_SIZE);
> +
> +void hv_ghcb_msr_write(u64 msr, u64 value)
> +{
> +	union hv_ghcb *hv_ghcb;
> +	void **ghcb_base;
> +	unsigned long flags;
> +
> +	if (!hv_ghcb_pg)
> +		return;
> +
> +	WARN_ON(in_nmi());
> +
> +	local_irq_save(flags);
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	hv_ghcb =3D (union hv_ghcb *)*ghcb_base;
> +	if (!hv_ghcb) {
> +		local_irq_restore(flags);
> +		return;
> +	}
> +
> +	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
> +	ghcb_set_rax(&hv_ghcb->ghcb, lower_32_bits(value));
> +	ghcb_set_rdx(&hv_ghcb->ghcb, upper_32_bits(value));
> +
> +	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 1, 0))
> +		pr_warn("Fail to write msr via ghcb %llx.\n", msr);
> +
> +	local_irq_restore(flags);
> +}
> +
> +void hv_ghcb_msr_read(u64 msr, u64 *value)
> +{
> +	union hv_ghcb *hv_ghcb;
> +	void **ghcb_base;
> +	unsigned long flags;
> +
> +	/* Check size of union hv_ghcb here. */
> +	BUILD_BUG_ON(sizeof(union hv_ghcb) !=3D HV_HYP_PAGE_SIZE);
> +
> +	if (!hv_ghcb_pg)
> +		return;
> +
> +	WARN_ON(in_nmi());
> +
> +	local_irq_save(flags);
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	hv_ghcb =3D (union hv_ghcb *)*ghcb_base;
> +	if (!hv_ghcb) {
> +		local_irq_restore(flags);
> +		return;
> +	}
> +
> +	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
> +	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 0, 0))
> +		pr_warn("Fail to read msr via ghcb %llx.\n", msr);
> +	else
> +		*value =3D (u64)lower_32_bits(hv_ghcb->ghcb.save.rax)
> +			| ((u64)lower_32_bits(hv_ghcb->ghcb.save.rdx) << 32);
> +	local_irq_restore(flags);
> +}
> +
> +void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value)
> +{
> +	hv_ghcb_msr_read(msr, value);
> +}
> +EXPORT_SYMBOL_GPL(hv_sint_rdmsrl_ghcb);
> +
> +void hv_sint_wrmsrl_ghcb(u64 msr, u64 value)
> +{
> +	hv_ghcb_msr_write(msr, value);
> +
> +	/* Write proxy bit vua wrmsrl instruction. */

s/vua/via/

> +	if (msr >=3D HV_X64_MSR_SINT0 && msr <=3D HV_X64_MSR_SINT15)
> +		wrmsrl(msr, value | 1 << 20);
> +}
> +EXPORT_SYMBOL_GPL(hv_sint_wrmsrl_ghcb);
> +
> +enum hv_isolation_type hv_get_isolation_type(void)
> +{
> +	if (!(ms_hyperv.priv_high & HV_ISOLATION))
> +		return HV_ISOLATION_TYPE_NONE;
> +	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
> +}
> +EXPORT_SYMBOL_GPL(hv_get_isolation_type);
> +
> +/*
> + * hv_is_isolation_supported - Check system runs in the Hyper-V
> + * isolation VM.
> + */
> +bool hv_is_isolation_supported(void)
> +{
> +	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
> +}
> +
> +DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
> +
> +/*
> + * hv_isolation_type_snp - Check system runs in the AMD SEV-SNP based
> + * isolation VM.
> + */
> +bool hv_isolation_type_snp(void)
> +{
> +	return static_branch_unlikely(&isolation_type_snp);
> +}
> +
>  /*
>   * hv_mark_gpa_visibility - Set pages visible to host via hvcall.
>   *
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index ffb2af079c6b..b77f4caee3ee 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -11,6 +11,8 @@
>  #include <asm/paravirt.h>
>  #include <asm/mshyperv.h>
>=20
> +union hv_ghcb;
> +
>  DECLARE_STATIC_KEY_FALSE(isolation_type_snp);
>=20
>  typedef int (*hyperv_fill_flush_list_func)(
> @@ -30,6 +32,61 @@ static inline u64 hv_get_register(unsigned int reg)
>  	return value;
>  }
>=20
> +#define hv_get_sint_reg(val, reg) {		\
> +	if (hv_isolation_type_snp())		\
> +		hv_get_##reg##_ghcb(&val);	\
> +	else					\
> +		rdmsrl(HV_REGISTER_##reg, val);	\
> +	}
> +
> +#define hv_set_sint_reg(val, reg) {		\
> +	if (hv_isolation_type_snp())		\
> +		hv_set_##reg##_ghcb(val);	\
> +	else					\
> +		wrmsrl(HV_REGISTER_##reg, val);	\
> +	}
> +
> +
> +#define hv_get_simp(val) hv_get_sint_reg(val, SIMP)
> +#define hv_get_siefp(val) hv_get_sint_reg(val, SIEFP)
> +
> +#define hv_set_simp(val) hv_set_sint_reg(val, SIMP)
> +#define hv_set_siefp(val) hv_set_sint_reg(val, SIEFP)
> +
> +#define hv_get_synic_state(val) {			\
> +	if (hv_isolation_type_snp())			\
> +		hv_get_synic_state_ghcb(&val);		\
> +	else						\
> +		rdmsrl(HV_REGISTER_SCONTROL, val);	\
> +	}
> +#define hv_set_synic_state(val) {			\
> +	if (hv_isolation_type_snp())			\
> +		hv_set_synic_state_ghcb(val);		\
> +	else						\
> +		wrmsrl(HV_REGISTER_SCONTROL, val);	\
> +	}
> +
> +#define hv_signal_eom(old_msg_type) {		 \
> +	if (hv_isolation_type_snp() &&		 \
> +	    old_msg_type !=3D HVMSG_TIMER_EXPIRED) \
> +		hv_sint_wrmsrl_ghcb(HV_REGISTER_EOM, 0); \
> +	else						\
> +		wrmsrl(HV_REGISTER_EOM, 0);		\
> +	}
> +
> +#define hv_get_synint_state(int_num, val) {		\
> +	if (hv_isolation_type_snp())			\
> +		hv_get_synint_state_ghcb(int_num, &val);\
> +	else						\
> +		rdmsrl(HV_REGISTER_SINT0 + int_num, val);\
> +	}
> +#define hv_set_synint_state(int_num, val) {		\
> +	if (hv_isolation_type_snp())			\
> +		hv_set_synint_state_ghcb(int_num, val);	\
> +	else						\
> +		wrmsrl(HV_REGISTER_SINT0 + int_num, val);\
> +	}
> +
>  #define hv_get_raw_timer() rdtsc_ordered()
>=20
>  void hyperv_vector_handler(struct pt_regs *regs);
> @@ -41,7 +98,7 @@ extern void *hv_hypercall_pg;
>=20
>  extern u64 hv_current_partition_id;
>=20
> -extern void __percpu **hv_ghcb_pg;
> +extern union hv_ghcb  __percpu **hv_ghcb_pg;
>=20
>  int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
>  int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
> @@ -195,6 +252,25 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct =
hv_interrupt_entry *entry);
>  int hv_mark_gpa_visibility(u16 count, const u64 pfn[],
>  			   enum hv_mem_host_visibility visibility);
>  int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible);
> +void hv_sint_wrmsrl_ghcb(u64 msr, u64 value);
> +void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value);
> +void hv_signal_eom_ghcb(void);
> +void hv_ghcb_msr_write(u64 msr, u64 value);
> +void hv_ghcb_msr_read(u64 msr, u64 *value);
> +
> +#define hv_get_synint_state_ghcb(int_num, val)			\
> +	hv_sint_rdmsrl_ghcb(HV_X64_MSR_SINT0 + int_num, val)
> +#define hv_set_synint_state_ghcb(int_num, val) \
> +	hv_sint_wrmsrl_ghcb(HV_X64_MSR_SINT0 + int_num, val)
> +
> +#define hv_get_SIMP_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SIMP, val)
> +#define hv_set_SIMP_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SIMP, val)
> +
> +#define hv_get_SIEFP_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SIEFP, val=
)
> +#define hv_set_SIEFP_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SIEFP, val=
)
> +
> +#define hv_get_synic_state_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SCON=
TROL, val)
> +#define hv_set_synic_state_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SCON=
TROL, val)


I'm not seeing the value in the multiple layers of #define to get and set t=
he
various syinc registers.  My thought was a completely different approach, w=
hich is
to simply implement the hv_get_register() and hv_set_register() functions w=
ith=20
a little bit more logic.  Here's my proposal.  This code is not even compil=
e tested,
but you get the idea:

static bool hv_is_synic_reg(unsigned int reg)
{
	if ((reg >=3D HV_REGISTER_SCONTROL) &&
	    (reg <=3D HV_REGISTER_SINT15))
		return true;
	return false;
}

u64 hv_get_register(unsigned int reg)
{
	u64 value;

	if (hv_is_synic_reg(reg) && hv_isolation_type_snp())
		hv_ghcb_msr_read(reg, &value);
	else
		rdmsrl(reg, value);
	return value;
}

void hv_set_register(unsigned int reg, u64 value)
{
	if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) {
		hv_ghcb_msr_write(reg, value);

		/* Write proxy bit via wrmsl instruction */
		if (reg >=3D HV_REGISTER_SINT0 &&
		    reg <=3D HV_REGISTER_SINT15)
			wrmsrl(reg, value | 1 << 20);
	} else {
		wrmsrl(reg, value);
	}
}

If the above code is implemented in one of the modules under arch/x86
to replace the existing implementations in arch/x86/include/asm/mshyper.h,
then it will only be built for x86/x64, and the existing code will just wor=
k
for ARM64.   Architecture neutral code in hv_synic_enable_regs() and
hv_synic_disable_regs() will still need check for hv_isolation_type_snp()
and take some special actions, but the calls to hv_get_register() and
hv_set_register() can remain unchanged.

This approach seems a lot simpler to me, but maybe I'm missing
something that your current patch is doing.

Your code does have a special case for HV_REGISTER_EOM.  Is
there a reason it needs to do an additional check of the old_msg_type?
I'm just not understanding why an SNP isolated VM requires
special treatment of this register.

A key point:  Getting/setting any of the synthetic MSRs requires
a trap to the hypervisor.  So they already not super-fast.  The
only code path in Linux that is performance sensitive is setting
HV_REGISTER_STIMER0_COUNT in hv_ce_set_next_event().
That's not a synic register, so the only additional burden with the
above implementation is checking the MSR value to see if it is
in the synic range.  The cost of that check is reasonable for
something that has to trap to the hypervisor anyway.


>  #else /* CONFIG_HYPERV */
>  static inline void hyperv_init(void) {}
>  static inline void hyperv_setup_mmu_ops(void) {}
> @@ -211,9 +287,9 @@ static inline int hyperv_flush_guest_mapping_range(u6=
4 as,
>  {
>  	return -1;
>  }
> +static inline void hv_signal_eom_ghcb(void) { };
>  #endif /* CONFIG_HYPERV */
>=20
> -
>  #include <asm-generic/mshyperv.h>
>=20
>  #endif
> diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
> index fa5cd05d3b5b..81beb2a8031b 100644
> --- a/arch/x86/include/asm/sev.h
> +++ b/arch/x86/include/asm/sev.h
> @@ -81,6 +81,9 @@ static __always_inline void sev_es_nmi_complete(void)
>  		__sev_es_nmi_complete();
>  }
>  extern int __init sev_es_efi_map_ghcbs(pgd_t *pgd);
> +extern enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2);
>  #else
>  static inline void sev_es_ist_enter(struct pt_regs *regs) { }
>  static inline void sev_es_ist_exit(void) { }
> diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
> index 9f90f460a28c..dd7f37de640b 100644
> --- a/arch/x86/kernel/sev-shared.c
> +++ b/arch/x86/kernel/sev-shared.c
> @@ -94,10 +94,9 @@ static void vc_finish_insn(struct es_em_ctxt *ctxt)
>  	ctxt->regs->ip +=3D ctxt->insn.length;
>  }
>=20
> -static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
> -					  struct es_em_ctxt *ctxt,
> -					  u64 exit_code, u64 exit_info_1,
> -					  u64 exit_info_2)
> +enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2)
>  {
>  	enum es_result ret;
>=20
> @@ -109,29 +108,45 @@ static enum es_result sev_es_ghcb_hv_call(struct gh=
cb *ghcb,
>  	ghcb_set_sw_exit_info_1(ghcb, exit_info_1);
>  	ghcb_set_sw_exit_info_2(ghcb, exit_info_2);
>=20
> -	sev_es_wr_ghcb_msr(__pa(ghcb));
>  	VMGEXIT();
>=20
> -	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) =3D=3D 1) {
> -		u64 info =3D ghcb->save.sw_exit_info_2;
> -		unsigned long v;
> -
> -		info =3D ghcb->save.sw_exit_info_2;
> -		v =3D info & SVM_EVTINJ_VEC_MASK;
> -
> -		/* Check if exception information from hypervisor is sane. */
> -		if ((info & SVM_EVTINJ_VALID) &&
> -		    ((v =3D=3D X86_TRAP_GP) || (v =3D=3D X86_TRAP_UD)) &&
> -		    ((info & SVM_EVTINJ_TYPE_MASK) =3D=3D SVM_EVTINJ_TYPE_EXEPT)) {
> -			ctxt->fi.vector =3D v;
> -			if (info & SVM_EVTINJ_VALID_ERR)
> -				ctxt->fi.error_code =3D info >> 32;
> -			ret =3D ES_EXCEPTION;
> -		} else {
> -			ret =3D ES_VMM_ERROR;
> -		}
> -	} else {
> +	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) =3D=3D 1)
> +		ret =3D ES_VMM_ERROR;
> +	else
>  		ret =3D ES_OK;
> +
> +	return ret;
> +}
> +
> +static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
> +				   struct es_em_ctxt *ctxt,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2)
> +{
> +	unsigned long v;
> +	enum es_result ret;
> +	u64 info;
> +
> +	sev_es_wr_ghcb_msr(__pa(ghcb));
> +
> +	ret =3D sev_es_ghcb_hv_call_simple(ghcb, exit_code, exit_info_1,
> +					 exit_info_2);
> +	if (ret =3D=3D ES_OK)
> +		return ret;
> +
> +	info =3D ghcb->save.sw_exit_info_2;
> +	v =3D info & SVM_EVTINJ_VEC_MASK;
> +
> +	/* Check if exception information from hypervisor is sane. */
> +	if ((info & SVM_EVTINJ_VALID) &&
> +	    ((v =3D=3D X86_TRAP_GP) || (v =3D=3D X86_TRAP_UD)) &&
> +	    ((info & SVM_EVTINJ_TYPE_MASK) =3D=3D SVM_EVTINJ_TYPE_EXEPT)) {
> +		ctxt->fi.vector =3D v;
> +		if (info & SVM_EVTINJ_VALID_ERR)
> +			ctxt->fi.error_code =3D info >> 32;
> +		ret =3D ES_EXCEPTION;
> +	} else {
> +		ret =3D ES_VMM_ERROR;
>  	}
>=20
>  	return ret;
> diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
> index e83507f49676..97b21256a9db 100644
> --- a/drivers/hv/hv.c
> +++ b/drivers/hv/hv.c
> @@ -8,6 +8,7 @@
>   */
>  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>=20
> +#include <linux/io.h>
>  #include <linux/kernel.h>
>  #include <linux/mm.h>
>  #include <linux/slab.h>
> @@ -136,17 +137,24 @@ int hv_synic_alloc(void)
>  		tasklet_init(&hv_cpu->msg_dpc,
>  			     vmbus_on_msg_dpc, (unsigned long) hv_cpu);
>=20
> -		hv_cpu->synic_message_page =3D
> -			(void *)get_zeroed_page(GFP_ATOMIC);
> -		if (hv_cpu->synic_message_page =3D=3D NULL) {
> -			pr_err("Unable to allocate SYNIC message page\n");
> -			goto err;
> -		}
> +		/*
> +		 * Synic message and event pages are allocated by paravisor.
> +		 * Skip these pages allocation here.
> +		 */
> +		if (!hv_isolation_type_snp()) {
> +			hv_cpu->synic_message_page =3D
> +				(void *)get_zeroed_page(GFP_ATOMIC);
> +			if (hv_cpu->synic_message_page =3D=3D NULL) {
> +				pr_err("Unable to allocate SYNIC message page\n");
> +				goto err;
> +			}
>=20
> -		hv_cpu->synic_event_page =3D (void *)get_zeroed_page(GFP_ATOMIC);
> -		if (hv_cpu->synic_event_page =3D=3D NULL) {
> -			pr_err("Unable to allocate SYNIC event page\n");
> -			goto err;
> +			hv_cpu->synic_event_page =3D
> +				(void *)get_zeroed_page(GFP_ATOMIC);
> +			if (hv_cpu->synic_event_page =3D=3D NULL) {
> +				pr_err("Unable to allocate SYNIC event page\n");
> +				goto err;
> +			}
>  		}
>=20
>  		hv_cpu->post_msg_page =3D (void *)get_zeroed_page(GFP_ATOMIC);
> @@ -199,26 +207,43 @@ void hv_synic_enable_regs(unsigned int cpu)
>  	union hv_synic_scontrol sctrl;
>=20
>  	/* Setup the Synic's message page */
> -	simp.as_uint64 =3D hv_get_register(HV_REGISTER_SIMP);
> +	hv_get_simp(simp.as_uint64);
>  	simp.simp_enabled =3D 1;
> -	simp.base_simp_gpa =3D virt_to_phys(hv_cpu->synic_message_page)
> -		>> HV_HYP_PAGE_SHIFT;
>=20
> -	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
> +	if (hv_isolation_type_snp()) {
> +		hv_cpu->synic_message_page
> +			=3D memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT,
> +				   HV_HYP_PAGE_SIZE, MEMREMAP_WB);
> +		if (!hv_cpu->synic_message_page)
> +			pr_err("Fail to map syinc message page.\n");
> +	} else {
> +		simp.base_simp_gpa =3D virt_to_phys(hv_cpu->synic_message_page)
> +			>> HV_HYP_PAGE_SHIFT;
> +	}
> +
> +	hv_set_simp(simp.as_uint64);
>=20
>  	/* Setup the Synic's event page */
> -	siefp.as_uint64 =3D hv_get_register(HV_REGISTER_SIEFP);
> +	hv_get_siefp(siefp.as_uint64);
>  	siefp.siefp_enabled =3D 1;
> -	siefp.base_siefp_gpa =3D virt_to_phys(hv_cpu->synic_event_page)
> -		>> HV_HYP_PAGE_SHIFT;
>=20
> -	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
> +	if (hv_isolation_type_snp()) {
> +		hv_cpu->synic_event_page =3D
> +			memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT,
> +				 HV_HYP_PAGE_SIZE, MEMREMAP_WB);
> +
> +		if (!hv_cpu->synic_event_page)
> +			pr_err("Fail to map syinc event page.\n");
> +	} else {
> +		siefp.base_siefp_gpa =3D virt_to_phys(hv_cpu->synic_event_page)
> +			>> HV_HYP_PAGE_SHIFT;
> +	}
> +	hv_set_siefp(siefp.as_uint64);
>=20
>  	/* Setup the shared SINT. */
>  	if (vmbus_irq !=3D -1)
>  		enable_percpu_irq(vmbus_irq, 0);
> -	shared_sint.as_uint64 =3D hv_get_register(HV_REGISTER_SINT0 +
> -					VMBUS_MESSAGE_SINT);
> +	hv_get_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
>=20
>  	shared_sint.vector =3D vmbus_interrupt;
>  	shared_sint.masked =3D false;
> @@ -233,14 +258,12 @@ void hv_synic_enable_regs(unsigned int cpu)
>  #else
>  	shared_sint.auto_eoi =3D 0;
>  #endif
> -	hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT,
> -				shared_sint.as_uint64);
> +	hv_set_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
>=20
>  	/* Enable the global synic bit */
> -	sctrl.as_uint64 =3D hv_get_register(HV_REGISTER_SCONTROL);
> +	hv_get_synic_state(sctrl.as_uint64);
>  	sctrl.enable =3D 1;
> -
> -	hv_set_register(HV_REGISTER_SCONTROL, sctrl.as_uint64);
> +	hv_set_synic_state(sctrl.as_uint64);
>  }
>=20
>  int hv_synic_init(unsigned int cpu)
> @@ -257,37 +280,50 @@ int hv_synic_init(unsigned int cpu)
>   */
>  void hv_synic_disable_regs(unsigned int cpu)
>  {
> +	struct hv_per_cpu_context *hv_cpu
> +		=3D per_cpu_ptr(hv_context.cpu_context, cpu);
>  	union hv_synic_sint shared_sint;
>  	union hv_synic_simp simp;
>  	union hv_synic_siefp siefp;
>  	union hv_synic_scontrol sctrl;
>=20
> -	shared_sint.as_uint64 =3D hv_get_register(HV_REGISTER_SINT0 +
> -					VMBUS_MESSAGE_SINT);
> -
> +	hv_get_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
>  	shared_sint.masked =3D 1;
> +	hv_set_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
> +
>=20
>  	/* Need to correctly cleanup in the case of SMP!!! */
>  	/* Disable the interrupt */
> -	hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT,
> -				shared_sint.as_uint64);
> +	hv_get_simp(simp.as_uint64);
>=20
> -	simp.as_uint64 =3D hv_get_register(HV_REGISTER_SIMP);
> +	/*
> +	 * In Isolation VM, sim and sief pages are allocated by
> +	 * paravisor. These pages also will be used by kdump
> +	 * kernel. So just reset enable bit here and keep page
> +	 * addresses.
> +	 */
>  	simp.simp_enabled =3D 0;
> -	simp.base_simp_gpa =3D 0;
> +	if (hv_isolation_type_snp())
> +		memunmap(hv_cpu->synic_message_page);
> +	else
> +		simp.base_simp_gpa =3D 0;
>=20
> -	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
> +	hv_set_simp(simp.as_uint64);
>=20
> -	siefp.as_uint64 =3D hv_get_register(HV_REGISTER_SIEFP);
> +	hv_get_siefp(siefp.as_uint64);
>  	siefp.siefp_enabled =3D 0;
> -	siefp.base_siefp_gpa =3D 0;
>=20
> -	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
> +	if (hv_isolation_type_snp())
> +		memunmap(hv_cpu->synic_event_page);
> +	else
> +		siefp.base_siefp_gpa =3D 0;
> +
> +	hv_set_siefp(siefp.as_uint64);
>=20
>  	/* Disable the global synic bit */
> -	sctrl.as_uint64 =3D hv_get_register(HV_REGISTER_SCONTROL);
> +	hv_get_synic_state(sctrl.as_uint64);
>  	sctrl.enable =3D 0;
> -	hv_set_register(HV_REGISTER_SCONTROL, sctrl.as_uint64);
> +	hv_set_synic_state(sctrl.as_uint64);
>=20
>  	if (vmbus_irq !=3D -1)
>  		disable_percpu_irq(vmbus_irq);
> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> index c0d9048a4112..1fc82d237161 100644
> --- a/drivers/hv/hv_common.c
> +++ b/drivers/hv/hv_common.c
> @@ -249,6 +249,12 @@ bool __weak hv_is_isolation_supported(void)
>  }
>  EXPORT_SYMBOL_GPL(hv_is_isolation_supported);
>=20
> +bool __weak hv_isolation_type_snp(void)
> +{
> +	return false;
> +}
> +EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
> +
>  void __weak hv_setup_vmbus_handler(void (*handler)(void))
>  {
>  }
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index aa55447b9700..04a687d95eac 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -24,6 +24,7 @@
>  #include <linux/cpumask.h>
>  #include <linux/nmi.h>
>  #include <asm/ptrace.h>
> +#include <asm/mshyperv.h>
>  #include <asm/hyperv-tlfs.h>
>=20
>  struct ms_hyperv_info {
> @@ -54,6 +55,7 @@ extern void  __percpu  **hyperv_pcpu_output_arg;
>=20
>  extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputadd=
r);
>  extern u64 hv_do_fast_hypercall8(u16 control, u64 input8);
> +extern bool hv_isolation_type_snp(void);
>=20
>  /* Helper functions that provide a consistent pattern for checking Hyper=
-V hypercall status. */
>  static inline int hv_result(u64 status)
> @@ -148,7 +150,7 @@ static inline void vmbus_signal_eom(struct hv_message=
 *msg, u32 old_msg_type)
>  		 * possibly deliver another msg from the
>  		 * hypervisor
>  		 */
> -		hv_set_register(HV_REGISTER_EOM, 0);
> +		hv_signal_eom(old_msg_type);
>  	}
>  }
>=20
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 04:29:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 04:29:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176873.321931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLeLz-0007Sm-Qj; Thu, 02 Sep 2021 04:29:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176873.321931; Thu, 02 Sep 2021 04:29:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLeLz-0007Sf-NE; Thu, 02 Sep 2021 04:29:31 +0000
Received: by outflank-mailman (input) for mailman id 176873;
 Thu, 02 Sep 2021 04:29:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLeLx-0007ST-Ui; Thu, 02 Sep 2021 04:29:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLeLx-0003VR-Nd; Thu, 02 Sep 2021 04:29:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLeLx-000875-FP; Thu, 02 Sep 2021 04:29:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLeLx-0002ur-Ev; Thu, 02 Sep 2021 04:29:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pNHNu+7/0kygYYHdfwSaykdtG98avYjjp7p186XenJo=; b=o6vza1WNPHjKgLUrgiQBr2N5OI
	CRTCJDlErAW14Ux2cSoKz+NEDo1yTqcLofehQLxdsYZpqrnNoEW61b8qma2OHWIxVQZmpIHQYLTRj
	ToWGOwLXnCvTery80ssEE5oPTg4wUkTH7k7TdqaQOGffjruyMkzkb7gdOuLFB3mvOodk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164743-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164743: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 04:29:29 +0000

flight 164743 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164743/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days    9 attempts
Testing same since   164740  2021-09-02 02:41:13 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 04:57:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 04:57:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176883.321951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLemP-00031x-4y; Thu, 02 Sep 2021 04:56:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176883.321951; Thu, 02 Sep 2021 04:56:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLemP-00031q-1k; Thu, 02 Sep 2021 04:56:49 +0000
Received: by outflank-mailman (input) for mailman id 176883;
 Thu, 02 Sep 2021 04:56:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLemN-00031k-9F
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 04:56:47 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.8.11])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ab2f5f11-c20f-41e1-8f83-106d04bbc49e;
 Thu, 02 Sep 2021 04:56:44 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0704.namprd21.prod.outlook.com (2603:10b6:300:128::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.1; Thu, 2 Sep
 2021 04:56:41 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 04:56:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab2f5f11-c20f-41e1-8f83-106d04bbc49e
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=diHzAfLzvHkLnGDBA77CxjILWQ0f4yi16eeG/tktyrNTahSkFhlY4zM/KqPreJW5GvOG5lnHcmdLoGHxu/GnypUBLdOuwXZrB5X5iaY5ZWEAeYUkQ24CYeU4XYbHPPdY087rZ66Koj/CtdHhpbAoLdiv29fYui7rEWaz/E27FLQq4rbC/Hllk84wAkRx508cuEPKqwMTKZ9ljCmxYaY2YO6mP13CeUCYq73ntFtINifFfzfQQOEVAET6RG99JNrSjos94yTmg/FFPHJilJRaADS4sL0ziwKi/JCEUn4vnRovq4pAbFLothJaUzEMLrmrFw6BTPrd4QxTnwBYH+m9hg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/g+2tfLa4TWsUYnWx7YAfFpKGu7PCDx8XRRyHbecAJY=;
 b=ISc15lW0YFYaIzGaYZcf7eSaag/OIA6d2PDby2AN03uuws9YkkDRS+BeaCF7hDi7FXbNdZOTM0OfreC8gKqKzWfX72n7NrfVJkN+FO+NSX+QZpPgRCRrzp/FwD12TK9KxuxOlMpaTsdTwoHKRTgr2FH8j8Y7zzTqzjQe6nVTtLjFrs8+Wm+27I5fNZzWXR8yPUJSbfJ1edY2u8maoJ1GatLGyKvvc9eNya/tFT8wmytGJc2cQdEffaAcKbISwnCgxSxNy3F/um/brEfwunRTOYzzOUAta2qrVU9LIMuRmx1daNCem7Pv/NVK7BGArSXRwZFRNbAvhSKwV7BBI4gW0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/g+2tfLa4TWsUYnWx7YAfFpKGu7PCDx8XRRyHbecAJY=;
 b=GmQbcuvDU5mYwcbUJ2PC5qkKcAeHuBop+hvo5IULzB7MP/WHRrNX3ze0EQRNBGFccqbV1wj7garo8nxApCLffgPlbZhUM1lgqoD4lOwClaPJmtlvW4qaERy1vXfh3eAXrbLY/sLKD9tYs/o8RhHB94SvVCQcDPRLUSRNJ5CQP9M=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 12/13] hv_netvsc: Add Isolation VM support for netvsc
 driver
Thread-Topic: [PATCH V4 12/13] hv_netvsc: Add Isolation VM support for netvsc
 driver
Thread-Index: AQHXm2gP6jy1GRuc+0K7S1qphPLgHKuQCC+wgAAtP2A=
Date: Thu, 2 Sep 2021 04:56:40 +0000
Message-ID:
 <MWHPR21MB1593A3909B3DB49EABB78CF5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-13-ltykernel@gmail.com>
 <MWHPR21MB1593CD9E7B545EF5A268B745D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
In-Reply-To:
 <MWHPR21MB1593CD9E7B545EF5A268B745D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=24c2ad3d-af1e-4524-bef7-f93352ae7e8f;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-02T02:09:02Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7f4c007b-0072-45e0-73ee-08d96dce0d9d
x-ms-traffictypediagnostic: MWHPR21MB0704:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB07040B6B69FEF6567783EBCAD7CE9@MWHPR21MB0704.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 64n9ZHQNFsz+NZtjmS32VhXFDsqb8zSWHHkpOgEAVkAOEYbK/lXOyJqt83v3+p/qJhmo1s4vtVRa5OFw+A80IEGG56thQ86l8sHgNWJJZzk+y80rlDn3zM74xtFvm0zFR9PXp4daK9ZEV+5x4R12LF0mVAAAh3i5Jx/ncYh1K73KkVgtBS7G0aAgZ33DJ0VLYZixVKxyZgiqIrr4GKstKHTQf7oSCByGiv8ykKVEeRYQ2zzX3FKEJ46WvygnaZWc2/i27ylRxUqCsE3FVujK4swpkfIEBbT3s9hFiufZ/agR0+EuqeuPdnKwDYPZVFxdqm+4yElIMyN4W9TsEXbaHohZHwwJJY/nHZhrJSTEWqDD5Dn4Tk01ukOe4E5rhnh1tQXionIjcAm13RTUZSWBP3ceRArg3r74e/XGA5YEmeZBWoDg79JJJY/GBwB5DtR8C/7RRvG4463rpwlG4oEoEv4WfOeTY77ikav98zTrcOXnUoUbLksknqpkwvAinPOTTEjyJFVNhdtZIZVD1ekXcbgKONHpVy6VLyiRqONqcq9w81ZRlRtw15+42oi4QhLpgK6ve5dokzz7q0iGJRtEWq6ELDLbn5OCOJH7SQEBBvP0iNNOJO0uRpAoYPaYvQTS8fdYfyvLbuoAFRWyyBAK+jOA/k+AyczCL5J1FGCb1Dk7wjVYuBSEnCwmvmwStG7YOfHdthwm2QC9hE9j7GdtYa2fO0PngUipTC7szx/U3Q8=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(9686003)(66476007)(66556008)(10290500003)(76116006)(64756008)(508600001)(186003)(7696005)(66446008)(122000001)(2940100002)(5660300002)(8936002)(316002)(921005)(6506007)(8676002)(33656002)(83380400001)(110136005)(8990500004)(2906002)(38070700005)(26005)(82950400001)(38100700002)(71200400001)(82960400001)(86362001)(52536014)(7416002)(7366002)(7406005)(55016002)(4326008)(54906003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?6EXkemG7X07lxyyEVGMWnZY1RJrvvTxJhOuuA3ahPWNi/QB6p2Fe1JSTWrhL?=
 =?us-ascii?Q?zNvbAB2OIg6osr5Wcg38XUQKN95LY8icqsTsmNQQMyPhQ4WqC7Pr6I/YnHIY?=
 =?us-ascii?Q?kAehk0612De9pWwHx2JufPylsK+6aLdZxTS0kOFlVuYHJMfyTZA4QrSdFTRb?=
 =?us-ascii?Q?l941Wi26kmgYfHho1pRjLJdBU0+3HBfO9MAQ54nGFq+cViyI5WFZO14ywN4P?=
 =?us-ascii?Q?dkf05ljqyoJeEqKb5ODxUl3MaE1NDDXlwsPTSqPi+YlKhtU3nw7LqFRuPv77?=
 =?us-ascii?Q?fva9x9vDPH9wRGCJMZExTTl3ffVGILZJtmp48Eowql8wJoTBxp0L6d+ibw7l?=
 =?us-ascii?Q?yLI5Hicoqgvp1GGDioypge+6qDJMQW3p+06HR1ethl5meh8SS+549sQzSOTh?=
 =?us-ascii?Q?t7XFV1DVuKl2ZTt6+u3mm3gcCb2vuwWsbSEltgn4Pjgs3zS+U8EPGnNNMckG?=
 =?us-ascii?Q?dpFxBEq7P3O4xwgINCk80PSIhLFeHsmh55wxSyArjayop2e5MY9fQ0T/7YGn?=
 =?us-ascii?Q?3o229iwTW5zpedYah+f4zMkX9MQfQ8P9CGN7t4pqXnb/JTSmjwdbPy0L4+A5?=
 =?us-ascii?Q?aiyR0LAlWMcZhk24zw+TYSBVEbQ36LBikmKzICRL+5TyimSGEXmuVDOxsGyu?=
 =?us-ascii?Q?zOeylCJ70LTnZPR+0xx+nxTBxhyAnfTpImGYrPiye7uhk11aiVwWbGaxaspe?=
 =?us-ascii?Q?Dci8Tu62eUS8EUGCGQo/G3x3DqcR7fStDnQ1Fl5pvV9NCC/+ou91D4q4U8yC?=
 =?us-ascii?Q?GYHb9sAC9MIMG99g7i8RvZ5XArm25QKCu5+tz9pcp3C3so0F5/TPlkgePyf8?=
 =?us-ascii?Q?bnNPvwFJQrA4h7857rad4qC1K//+ntDXp3mso0Gf4k1NoCs30CW7ha2+MObm?=
 =?us-ascii?Q?VT+LZay4wDOgoQDEfWO+LRcd0wWHB9ozah3a2RRt7HRdrCbZSFY5Dyu6+hN5?=
 =?us-ascii?Q?SE4Uo7p0YZyONhi6oLj2T1c8+/ou7hxoNYwTfW/QER5yN6ats56gLjLR9F8e?=
 =?us-ascii?Q?ycfJhEcfMewHC/123Moiy6lcUUHPmXehUFwNhAaGrnyXPTOOE9LqLAlL8GaC?=
 =?us-ascii?Q?GpHY00fVHl0FhFa9Zw3mMJ1rBFGi7ysY9atT4KZGRwSt4OaacwO5CJyTvSVu?=
 =?us-ascii?Q?NXsRm7QwfkenbM9NfewkB8NT8LPagUyTMthGedIdo+kyHF9e4UCwbElo+A3t?=
 =?us-ascii?Q?lDGdJ07HBcp76BT7Fz2zHC0T/mlvuwEBTdapFc3ts6fOURINtJ5pir67PgHa?=
 =?us-ascii?Q?oOPkvGdqPu8l5XwHFt1lY0aemWmDCUyFohffPqTqvMLEX0jgNifasxn5Jt3Q?=
 =?us-ascii?Q?XjnOeVB+bdznBYzrnkKcTU01?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f4c007b-0072-45e0-73ee-08d96dce0d9d
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 04:56:40.6250
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: HQFrIKEO1/7vXPGsi6Dj8gkGAUk2kVTDMG1CDEjqxBoLGRSW2nnJEeXs7PCEgMxxQYhcC5DNiGsb5npNJkcRdevFW1BZIHUZ93h3srjzPa0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0704

From: Michael Kelley <mikelley@microsoft.com> Sent: Wednesday, September 1,=
 2021 7:34 PM

[snip]

> > +int netvsc_dma_map(struct hv_device *hv_dev,
> > +		   struct hv_netvsc_packet *packet,
> > +		   struct hv_page_buffer *pb)
> > +{
> > +	u32 page_count =3D  packet->cp_partial ?
> > +		packet->page_buf_cnt - packet->rmsg_pgcnt :
> > +		packet->page_buf_cnt;
> > +	dma_addr_t dma;
> > +	int i;
> > +
> > +	if (!hv_is_isolation_supported())
> > +		return 0;
> > +
> > +	packet->dma_range =3D kcalloc(page_count,
> > +				    sizeof(*packet->dma_range),
> > +				    GFP_KERNEL);
> > +	if (!packet->dma_range)
> > +		return -ENOMEM;
> > +
> > +	for (i =3D 0; i < page_count; i++) {
> > +		char *src =3D phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
> > +					 + pb[i].offset);
> > +		u32 len =3D pb[i].len;
> > +
> > +		dma =3D dma_map_single(&hv_dev->device, src, len,
> > +				     DMA_TO_DEVICE);
> > +		if (dma_mapping_error(&hv_dev->device, dma)) {
> > +			kfree(packet->dma_range);
> > +			return -ENOMEM;
> > +		}
> > +
> > +		packet->dma_range[i].dma =3D dma;
> > +		packet->dma_range[i].mapping_size =3D len;
> > +		pb[i].pfn =3D dma >> HV_HYP_PAGE_SHIFT;
> > +		pb[i].offset =3D offset_in_hvpage(dma);
> > +		pb[i].len =3D len;
> > +	}
>=20
> Just to confirm, this driver does *not* set the DMA min_align_mask
> like storvsc does.  So after the call to dma_map_single(), the offset
> in the page could be different.  That's why you are updating
> the pb[i].offset value.  Alternatively, you could set the DMA
> min_align_mask, which would ensure the offset is unchanged.
> I'm OK with either approach, though perhaps a comment is
> warranted to explain, as this is a subtle issue.
>=20

On second thought, I don't think either approach is OK.  The default
alignment in the swiotlb is 2K, and if the length of the data in the
buffer was 3K, the data could cross a page boundary in the bounce
buffer when it originally did not.  This would break the above code
which can only deal with one page at a time.  So I think the netvsc
driver also must set the DMA min_align_mask to 4K, which will
preserve the offset.

Michael


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:00:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:00:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176893.321973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLfm5-0002xr-VE; Thu, 02 Sep 2021 06:00:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176893.321973; Thu, 02 Sep 2021 06:00:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLfm5-0002xk-S5; Thu, 02 Sep 2021 06:00:33 +0000
Received: by outflank-mailman (input) for mailman id 176893;
 Thu, 02 Sep 2021 06:00:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLfm4-0002xe-31
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:00:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1391809c-0bb3-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:00:30 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-_ZlLMGi5PW2ujws4aoQMqA-1; Thu, 02 Sep 2021 08:00:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6173.eurprd04.prod.outlook.com (2603:10a6:803:ff::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:00:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 06:00:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0030.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:00:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1391809c-0bb3-11ec-ae3e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630562429;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nC+9HmiyN8tVvvPJOvEBYdkLxGcD03BuhBqwPRC12mQ=;
	b=Fvkhcpn5ROPnx59kqiGL7ot1nXCPqsYuIH3SmgyXsw+RwA8QPAPR/2DdrrEicu5ggwV4Xh
	5u4NDPgZiliSPcNCGLTdEuPKGQp6nLjjRUtTnwUzmYOLNMSK2lAOoiABqM2PiuMhlPoFfR
	a54t4Zoocz1RV/AriDcv+iTSsXRj44Q=
X-MC-Unique: _ZlLMGi5PW2ujws4aoQMqA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=h/Myo87nMPuu3OvTsQwALPeycrrQFH4n8VvJ5umuLgQzm/sqKIWQq9VKavtwwRJFEJPTo1++E4QSL8KBikHt0eCdXZEUualqniHI2vD7xAzUh/x5RqMTq4PFg2SqbA135NarC3D91c8BENprXecU+ez1C8dlMz6EeLlYK65GcaI1vzdenUywnSypIJcmOXQyQSGlCVuoblPHjc1pnAnz4w+qsky9QYFx2NrCFaDhQ/EyIJdek1leBl4I6ly2ttdWOyIT2kTNPKMmRT39YQn4KceC7EmLijJesNvg6Bv4GXJmvBVEpFDAKfiSSPY2uDGJxH0i++cOptkN/Lq/ql9Zjw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TbBGseStlkeOTJ5Gdk2wq00BX9kdOoWEm2+cqRGWZDA=;
 b=OzTQufgGHLa22yzQDwIw69LRQoouqhuTV2NzaraOWKoLQ49Uhp6Lm8pqRSkUub2RAAE5qibpT/bSWB4aVF6HxvTm4Yz524VNQAiJecOb/vCZ+5pkfquGb3W3Z6WoJ7bwoGt8FR+HurevEvJVR5dpYhGxbpmN4T+tge/BRWECAIBHCxD4xOl6IFXfsprwPw0jLrzkKHIQiMkCEbUIObHefvdDGQEI8eCppIb3Xzmwo/ngoSohNYb7x83pGpB8UwAWsXz5BC4dh1MZ201Fv+530D4ZbHNdr0XhTyDP39QLGjmyn8Cf2NaSyLWW5C1OAsLfJJAK4hQrMeXwbIsG1B597A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
To: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "julien@xen.org" <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-25-wei.chen@arm.com>
 <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e28f6230-56cc-e716-0dbb-8e1dacec7df2@suse.com>
Date: Thu, 2 Sep 2021 08:00:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P193CA0030.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0f228516-5502-423e-f163-08d96dd6f673
X-MS-TrafficTypeDiagnostic: VI1PR04MB6173:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6173A83A0CFC0F9D10F6D254B3CE9@VI1PR04MB6173.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	O2x7S07dpmshUk8wFmgnSyV4xuUJ5+sD8h978JYQAP4mD5kqnVB7DL2VCIUwxISvMX3kfP2hVXtvW/2Yr8uR9/kCwcSEMZiFNabEVI1/C+Kun/lD2Mh0ZpDfTeOv/GRJC65LcCldgJ2F5AelADQ5ugdQhHnnYzEhpTClo377FGhVWQJq3NiLSOUmrEGxnBs/g4lD9wQ/ozjpeBoqY7p2ISyM+gYgEdBgjWjNmmFkeqgyT5k2R/FmXuhRKJC+K4hRH468L3PMR9Z5a1Xi4yTI/6N4RLhUJkUhWNg57GInFeqgbJTJ3ejCoP9w2ZPKh6gV4TQ3mm/jkZ1EZWgZiz9qOgt976bSnO9cMGdquZB5PysYrggmB1SEYqU2LypS+7zEt0doPm+L9Ol17vnEAJpufqbsrsqK2hlyFvY3Y0zSjQN3t3AjUz8hz6IglnFt+S2II4DrALPaFOjLaUm6cceillROoAo6CQ9hMS0LsvgPsNZBWTCGNgJ0/Z6+oTX3mHYhaiDXbC0kxC4vXUFOpo1fokRkq9HBETuCFiYMvperAqDiK8CCY1votnPrs8rIFXpXZ13mN7cWqlTPQzN/8xEr8x7Pl/68dPXmTw0E8fQTQqslocR3rWWfXYFBSRH2R9b1zwv1IsyUg79TkxpjNCin0IKJZww1hFAvK6ZkX2+Sr462fqoyRrZuiPohkkgo9cGAvoeX93vR3CZCSxE90dn8ZdPpJdxXSgjDV+5owlHjqXw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(396003)(376002)(346002)(136003)(36756003)(8676002)(2616005)(8936002)(6486002)(956004)(86362001)(26005)(110136005)(66556008)(66476007)(53546011)(54906003)(16576012)(316002)(31696002)(66946007)(478600001)(4326008)(38100700002)(31686004)(186003)(2906002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?aY/riTP+rSmQ6CrbF/9tBbS0He25OlEingMszT5jU8d7dBShRYlVbNVpP381?=
 =?us-ascii?Q?96e5Erg5treQfk0U8ky7rs6krtp5v0ucrV83a3A9MVbEZW3q6e7DdrFqvdRd?=
 =?us-ascii?Q?ViR/z1W1Tb7xCBsC5ebrNrN/mPYv/njFBPgpLi3jPaF7IW0WmkGY9+Oij7V/?=
 =?us-ascii?Q?hFiM3NZI3VJMa2FNXf+0pYYYNpJPSpjlwSTx+CY/k2fFEs0MqcUKMjTTOsCw?=
 =?us-ascii?Q?yTD2n57sbhrPFW4myHGQ06uYKruNIju+vlnf4+HP82zbHhd4qvkSaBx94O2J?=
 =?us-ascii?Q?jIxl0kNIFRzOaw9wqP4leEBpg12nS23Ox0m9eWhfW/Zuj71X4cxJMORaNGod?=
 =?us-ascii?Q?Qo9x7FTQzUDfP0G2FhIaog2ntAW3yM1wygXf9EO0Y62S4+UPB1a7JIGhj0lw?=
 =?us-ascii?Q?LG0Xw+mh3+RLWtyutEX5ZQzwIZ/Ed7dAuZjd5sLJ4PZ83ni4wCoXzb3QTOV9?=
 =?us-ascii?Q?rqPupBOOIjPuyvjp4M4WvCMEiu9kz+WxBDojwQ2ev02YSEG2f7nDKmzI5XRY?=
 =?us-ascii?Q?1LZ6xtZv46gYRXdwjcwdRl0zSwXdSw5q77xkrECNk81nVM+5xcsN9YLWAMLd?=
 =?us-ascii?Q?GuiGJtspya9MF0UGZGlzOVE7MUslno2LlO4kffoziNiFrC5hdnv8XJwr7O9o?=
 =?us-ascii?Q?3q3ZRKbMTM3C7qmTE7IT3GOap/47MnklCEjH1/3MOnkilvGSZre4b2stkKle?=
 =?us-ascii?Q?wqG/T0RJfJsg0g/B2/tH/NlwlQtQf69quY7ILwpQgvXcjCLq5qYMf/bCgAA+?=
 =?us-ascii?Q?mJ7Zkt5CXhsaicm9VGCpASm7YvbiHqGAUYTROW7nLMTvH4jKknkV6pd8BMwb?=
 =?us-ascii?Q?2RopmJMXERhN/6fVyId3ZQ7tbtrAot1n+SsNH4Tkgsahmv8RuZBHHBNgct9U?=
 =?us-ascii?Q?VNpqpAdw8UIsAFjdDHWUt1LrsBQ+FnZRVbxUWMcjXT1jkYs81y1enrmErtok?=
 =?us-ascii?Q?MbdJJ0wQAZCFYaO48OsjnCvj7M+/0iJm0csfpHf08F1NibqypQaZvfsE8Hix?=
 =?us-ascii?Q?EQlVOdOLZQqMnVyhN7lUgdi/N6X9G8kMu6QGsqBzajmwb0hjs3g1rpwZdHVj?=
 =?us-ascii?Q?vEXhbFlAn5UJ3Wz9BgDuF9/L5FY/HTv+zm2+8RzChtykvBcwNSr3EnZtg+9h?=
 =?us-ascii?Q?jnlyugydMwCLuQ6L0UsTxUSI5Eu7zuYZvMIQMBKa7L+LXY36Hefac0z43A4z?=
 =?us-ascii?Q?DvdHVopqQGaRDj0XxJxXpZAIe+ZyGxtIWi6eMq06c+/782/Tj565WRLxmTZa?=
 =?us-ascii?Q?sPLEFTZXA+EJUm31TSUMKI1Q03eF/eL1BK0dkxstmWkDgD/p8q09SRnTZ5V2?=
 =?us-ascii?Q?SmAaZqTOnFDgteEfoiVvedqt?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f228516-5502-423e-f163-08d96dd6f673
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:00:27.6917
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ojAZtHKtXPbvmBV18kYQwwWXzK5+2ZhSpSLMDwdvfKajnYIXRj76IaWkTEWPvatnKOANlthkfo18EKIXFcAlAA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6173

On 01.09.2021 18:21, Stefano Stabellini wrote:
> On Wed, 1 Sep 2021, Wei Chen wrote:
>>> Stefano Stabellini
>>> Sent: 2021=E5=B9=B49=E6=9C=881=E6=97=A5 5:36
>>>
>>> On Tue, 31 Aug 2021, Wei Chen wrote:
>>>> I don't really like this implementation. I want the behavior of
>>>> numa_set_distance just like the function name, do not include
>>>> implicit operations. Otherwise, except the user read this function
>>>> implementation before he use it, he probably doesn't know this
>>>> function has done so many things.
>>>
>>> You can leave numa_set_distance as-is without any implicit operations.
>>>
>>> In that case, just call numa_set_distance twice from numa_set_distance
>>> for both from/to and to/from. numa_set_distance could return error is
>>
>> I am OK for the first sentence. But...
>>
>>> the entry was already set to a different value or success otherwise
>>> (also in the case it was already set to the same value). This would
>>
>> ... I prefer not to check the previous value. Subsequent numa_set_distan=
ce
>> call will override previous calls. Keep numa_set_distance as simple as
>> it can. And when you pass new data to numa_set_distance, it doesn't
>> know whether the previous data was correct or the new data is correct.
>> Only caller may have known. =20
>=20
> That might be OK but if not numa_set_distance then somebody else needs
> to check against overwriting previous values. That is to be able to spot
> bad device tree cases like:
>=20
>   0 1 20
>   1 0 40

What's wrong with this? At least the ACPI spec cares to specifically
permit this case:

"Except for the relative distance from a System Locality to itself,
 each relative distance is stored twice in the matrix. This provides
 the capability to describe the scenario where the relative distances
 for the two directions between System Localities is different."

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:04:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:04:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176901.321985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLfpa-0003vt-L5; Thu, 02 Sep 2021 06:04:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176901.321985; Thu, 02 Sep 2021 06:04:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLfpa-0003vm-I8; Thu, 02 Sep 2021 06:04:10 +0000
Received: by outflank-mailman (input) for mailman id 176901;
 Thu, 02 Sep 2021 06:04:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLfpZ-0003vc-2d; Thu, 02 Sep 2021 06:04:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLfpY-0005U5-SP; Thu, 02 Sep 2021 06:04:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLfpY-0004xW-Jz; Thu, 02 Sep 2021 06:04:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLfpY-0001YZ-JV; Thu, 02 Sep 2021 06:04:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6PHwo3l0jfuI9G8Xy/DtSlE4Hi0bM+BslyfBpaVetrM=; b=WEp1zu7/D9EF8gUevd+sKYRoTX
	V7MTbvYJQzUN5pCqo4wcyct2qs7sM7iS5W603VN8tLjp5YEyxdLe7wLQR7pvivichthb+bvGLQWNn
	95PmCLhDyDe82sgPx1tl/Cj/lMIrvGsfZAfexAeqqJ/arjCde4oH+848eo2nwJr9hXSA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164746-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164746: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 06:04:08 +0000

flight 164746 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164746/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   10 attempts
Testing same since   164740  2021-09-02 02:41:13 Z    0 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:20:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:20:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176938.322005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLg4w-00060H-3C; Thu, 02 Sep 2021 06:20:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176938.322005; Thu, 02 Sep 2021 06:20:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLg4v-0005zU-VR; Thu, 02 Sep 2021 06:20:01 +0000
Received: by outflank-mailman (input) for mailman id 176938;
 Thu, 02 Sep 2021 06:20:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4i5Y=NY=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mLg4u-0005t9-4L
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:20:00 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id cc3910cd-0bb5-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:19:58 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2F8BB1FB;
 Wed,  1 Sep 2021 23:19:58 -0700 (PDT)
Received: from [10.57.19.246] (unknown [10.57.19.246])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6CFB03F766;
 Wed,  1 Sep 2021 23:19:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc3910cd-0bb5-11ec-ae3e-12813bfff9fa
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
 <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
 <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <0c527a57-44f2-d888-3210-644ea3aa940e@arm.com>
Date: Thu, 2 Sep 2021 08:19:51 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Stefano,

On 01.09.2021 19:54, Stefano Stabellini wrote:
> On Wed, 1 Sep 2021, Julien Grall wrote:
>> On 01/09/2021 14:10, Bertrand Marquis wrote:
>>>> On 1 Sep 2021, at 13:55, Julien Grall <julien@xen.org> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 01/09/2021 13:43, Michal Orzel wrote:
>>>>> Introduce new Xen command line parameter called "pmu_access".
>>>>> The default value is "trap": Xen traps PMU accesses.
>>>>> In case of setting pmu_access to "native", Xen does not trap
>>>>> PMU accesses allowing all the guests to access PMU registers.
>>>>> However, guests cannot make use of PMU overflow interrupts as
>>>>> PMU uses PPI which Xen cannot route to guests.
>>>>> This option is only intended for development and testing purposes.
>>>>> Do not use this in production system.
>>>> I am afraid your option is not safe even in development system as a vCPU
>>>> may move between pCPUs.
>>>>
>>>> However, even if we restricted the use to pinned vCPU *and* dedicated
>>>> pCPU, I am not convinced that exposing an half backed PMU (the overflow
>>>> interrupt would not work) to the guest is the right solution. This likely
>>>> means the guest OS would need to be modified and therefore the usage of
>>>> this option is fairly limited.
>>>>
>>>> So I think the first steps are:
>>>>   1) Make the PPI work. There was some attempt in the past for it on
>>>> xen-devel. You could have a look.
>>>>   2) Provide PMU bindings
>>>>
>>>> With that in place, we can discuss how to expose the PMU even if it is
>>>> unsafe in some conditions.
>>>
>>> With those limitations, using the PMU to monitor the system performances or
>>> on some specific use cases is still really useful.
>>> We are using that to do some benchmarks of Xen or of some applications to
>>> compare the behaviour to a native system or
>>> analyse the performances of Xen itself (hypercalls,context switch …etc)
> 
> I also already had to write a patch almost exactly like this one to
> provide to customers a few months back.
> 
> 
>> I understand this is useful for some setup and I am not trying to say we
>> should not have a way to expose the PMU (even unsafely) in upstream. However,
>> I think the option as it stands is too wide (this should be a per domain knob)
>> and we should properly expose the PMU (interrupts, bindings...).
> 
> I have never used PMU directly myself, only provided a patch similar to
> this one.  But as far as I could tell the users were fully satisfied
> with it and it had no interrupts support either. Could it be that
> interrupts are not actually needed to read the perf counters, which is
> probably what users care about?
> 
Most of the people using PMU do not need interrupt as PMU can only be programmed
to generate overflow interrupt request when a counter overflows.
If we want to catch counter overflow without using interrupt, we can check
if overflow status bit is set to 1 in PMOVSCLR register.

> In regards to "this should be a per domain knob", in reality if you are
> doing PMU experiments you don't care if only one or all domains have
> access: you are working in a controlled environment trying to figure out
> if your setup meets the timing requirements. There are no security or
> safety concerns (those are different experiments) and there is no
> interference problems in the sense of multiple domains trying to access
> PMU at the same time -- you control the domains you decide which one is
> accessing them.
> 
> So I am in favor of a patch like this one because it actually satisfy
> our requirements. Even if we had per-domain support and interrupts
> support, I don't think they would end up being used.
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:27:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:27:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176944.322016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgC2-0007Yp-SC; Thu, 02 Sep 2021 06:27:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176944.322016; Thu, 02 Sep 2021 06:27:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgC2-0007Yi-Ob; Thu, 02 Sep 2021 06:27:22 +0000
Received: by outflank-mailman (input) for mailman id 176944;
 Thu, 02 Sep 2021 06:27:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLgC0-0007YY-MX; Thu, 02 Sep 2021 06:27:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLgC0-0005ud-Hh; Thu, 02 Sep 2021 06:27:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLgC0-0005qj-9X; Thu, 02 Sep 2021 06:27:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLgC0-0007Gy-91; Thu, 02 Sep 2021 06:27:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FT+xdppRgBkZokyT92a/gOctpaZDhag/iLyFbntrcT4=; b=SSPKY0k/xiyN/vdoVbKNpSmjGF
	nDNQR++YzAcvUEWNtAe6Bo4CvsCScJOPbAbWrIm4DmHB8bbmGkQSKUEGR6hGdFpltoaUx2N7q8uV1
	RQWY2hfPv2+iPPih3KXF7twzHvmBUN+fXBQFg2fLmTuJfpKuzt3plyM8WbkTD+La58pA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164725-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164725: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=4c41a1c595e1ce3fe29f3b7bb22ff7402be9c77d
X-Osstest-Versions-That:
    qemuu=d52dff5d8048d4982437db9606c27bb4127cf9d0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 06:27:20 +0000

flight 164725 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164725/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164690
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164690
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164690
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164690
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164690
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164690
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164690
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                4c41a1c595e1ce3fe29f3b7bb22ff7402be9c77d
baseline version:
 qemuu                d52dff5d8048d4982437db9606c27bb4127cf9d0

Last test of basis   164690  2021-09-01 08:37:55 Z    0 days
Testing same since   164725  2021-09-01 21:39:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Bulekov <alxndr@bu.edu>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  David Hoppenbrouwers <david@salt-inc.org>
  Dongwon Kim <dongwon.kim@intel.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Helge Deller <deller@gmx.de>
  Joe Komlodi <joe.komlodi@xilinx.com>
  Jose R. Ziviani <jziviani@suse.de>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Qiang Liu <cyruscyliu@gmail.com>
  Richard Henderson <richard.henderson@linaro.org>
  Thomas Huth <thuth@redhat.com>
  Vijai Kumar K <vijai@behindbytes.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   d52dff5d80..4c41a1c595  4c41a1c595e1ce3fe29f3b7bb22ff7402be9c77d -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:31:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:31:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176952.322029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgFZ-0000YN-Ij; Thu, 02 Sep 2021 06:31:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176952.322029; Thu, 02 Sep 2021 06:31:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgFZ-0000YG-Fo; Thu, 02 Sep 2021 06:31:01 +0000
Received: by outflank-mailman (input) for mailman id 176952;
 Thu, 02 Sep 2021 06:31:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLgFY-0000YA-Fr
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:31:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 55adbe10-0bb7-11ec-ae40-12813bfff9fa;
 Thu, 02 Sep 2021 06:30:59 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2055.outbound.protection.outlook.com [104.47.0.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-om1gsGNxM8KXJl3CXTu7yw-1; Thu, 02 Sep 2021 08:30:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 06:30:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 06:30:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR07CA0025.eurprd07.prod.outlook.com (2603:10a6:208:ac::38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.16 via Frontend Transport; Thu, 2 Sep 2021 06:30:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55adbe10-0bb7-11ec-ae40-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630564258;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NHsd+b8WBTxsiYSoThy12w4xRZWH/jbrxlJXllqH2rw=;
	b=G05TDj1ZSgpnH/o0ZbW1MbIEzbmGigctWqJPv8ne5cmgLYMolicnmDBJ9LVIuRixYlkxZa
	E9bTxC/0BujPsIcWS2npw4HBPg0yCNkVQVISriYcIq4W5GLRKf9KjsDbB8zLTHWK+Bh/ai
	qpR2HAuAg76FwMUYXU2yqFJBWcpHReM=
X-MC-Unique: om1gsGNxM8KXJl3CXTu7yw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d1IuVKr6MBQlKnf9DNzaw8sZttvukCHtaG6Uzk6vDFvQJ2vyBCiiUCkpBljERlLo29TpzrcPlL4XusU5cmLetH/DtRQ48rYO6mgGhvKpLfP29qfNUXsfIPFzMeqmimFxG7CV9u3eeWsZkSLuu8efFY2qbrBvMijBGb097er8U77MGbCVojaow0ttJvEFvrb4tBneAzB1EbwhX9u91UMJ3VyapFY1qGefNynZLFH/WkFnRJAhTzOWeWROQ3mxyxQkk0hrYUtZPAwM1QRVg5mB7sQS1guEuO2b5rO06r/dOpMjHSoxlz6lFzNgqa8GMZACHka9WL8xvZpq1Qi8cZ6mUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6yovaqJcbacXnyqAkIzRPSSq5Hm0cSAJGlG1VSsXglM=;
 b=ISI/b1bm9BmDvdmzNmGaLNnG1uDe5E2QP6swui8siJcOqjKv1Mcrmh8OUquM3CB7XnuQ8PFc4480lOqqyXw3x6NQ8/Cy63h9ErpjopAmKOcr8lx+93DX+WzcI2dGMq+xSWkY3wq+n4rYO5SYOPeNALPDMxK83ctWfbCx9oNWc/t8pbHIFLpazIUJ/44OrJY7DdbxOTPaI2gDEMq9r8Dx9U+N/eHLbX0UEk7hmuEmMen/3IDJhyhmh0qQSeZnhRv0iLQ+JfVryRrTEvbXA1LPi/BeyKvoDrswITyVczr5N1L4arCMhr0/Doq1tySiprSWsbWhCnb7GrkxIG4L61a3cA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 0/4] x86/PVH: Dom0 building adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <a2b8421d-e608-2abe-b78c-924ab0bf529b@suse.com>
 <YS+GmC7IjxoODTu+@Air-de-Roger>
 <17c5ce48-11e6-ac97-fa67-294cbb8b79b7@suse.com>
 <YS+mj/mxSJCIso6r@Air-de-Roger>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1e37d6ef-b96f-cf09-277d-403abd9eecbb@suse.com>
Date: Thu, 2 Sep 2021 08:30:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YS+mj/mxSJCIso6r@Air-de-Roger>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR07CA0025.eurprd07.prod.outlook.com
 (2603:10a6:208:ac::38) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a8dac602-16aa-48d1-26ab-08d96ddb3817
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB244595748E0D55FA8499219FB3CE9@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wBXNTn+KClSpf+pJ9VV4QRaKujiY9BasFYv84VSLNLxrcxHqOc+SpfCzVe+1EphoGaxr/9imm8CSJ7lcnW9xYDmEggZLiJV7MwqWfEj0acaqgjc/NcJlzm4LTsrf/XQaP3AGl99gAJ1TwST3Ii7DTJhQdjAReJlx6Y3b851AQQo2gD1J9ywlLNVvIFtq6QydVQnjCs/qibJ00pVYw0RUdgLKOO6Ib3helfdgw3mBZdOV2iV66I6bgDoCogmSWWXF2sSCWFrlTdSPVTMdyPRd0o2YIga66xSBybaSg/oGI+/3lt0OfIxX+7TyiGwDs26kR7Wqmt6z0cx7BtBco58iBIkiTh6R1cxxwmhjB0m/Wgg/O4kOAXNDA2zfeR74dJKa3snHszdKbK0+R7AdS+AtzA90Raw6lYhdfIEtCXJkUYklbE/3e1q7c8WYSHP3wRckhoRb6GyxNVETN1EV47qIUYE8pMrA8sON3f9rq0y6PkscC7GigtngIBPW8m64i540CG0/z83927Ew3RNcfT3czxSKQMyV/ex7caUAuLA/i5k3Nn2HgfziNZ6SetGzgFBKJx93Z7zUbvEHimDFM+ojvP0bBXTAyTGC5qEXwXLSJfFh3PDMbu1wnEHytiw/a2MR5ZFIvfH+GsJvwdnVmjDVPyaJ1apMLBZlKKNnWCFqgmSWodUBqNUjDvUFx1z002WyfL7ZOaVXAQM8hgA5YuQ1eUiXaYzbdymunxBhtbYkNHw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(376002)(136003)(366004)(396003)(38100700002)(66476007)(66946007)(316002)(8936002)(31696002)(36756003)(6916009)(66556008)(8676002)(5660300002)(54906003)(16576012)(31686004)(478600001)(86362001)(2906002)(53546011)(26005)(186003)(6486002)(4326008)(956004)(83380400001)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?y7guap8YjqQXZsDHt7FzG+pd3pJB/bIkHLb+R0Hrh7f86rg+03WVwicywtWM?=
 =?us-ascii?Q?VYcXFTidtsZHuTVCVv6MrQt/MyxvZhjkK4aNuzbQiSJSGkI4003z6cuNjcji?=
 =?us-ascii?Q?N+nxHMW6wUxmrNsIxodlR7ysK1p1On2hrH2gCK3Mlizjwjwin+f8I2c69fIS?=
 =?us-ascii?Q?D6gt/SkWy7O1gGtQb++FaC6YcjjKSPwYXDGIJir1R+QqJvbpyWXHTxs91pkm?=
 =?us-ascii?Q?CCqinxKRzfoqOq99DjOJNhwi2SFgV8YcRqCk+n6p6/caT2DWYHalNza7+xrN?=
 =?us-ascii?Q?aD/5huqEqSjiaA5VN9pcNZQXI4PzSmqdOQEeginUiwMeXO33Fy1wkdR2ZLpU?=
 =?us-ascii?Q?TlKY05NzMIWvyA1Hf5rxzxGmDQsWtpefAQZ6LtvGGO65hp3tZ+kWs5BbBhuk?=
 =?us-ascii?Q?DFt60DaM54qvijVoPECquPsdQch3CWDw2w6Vhh30KHAZDrKuHL+e/ti9o82I?=
 =?us-ascii?Q?a6UDGWh8xQcMU8hd7WL2XRec4/Hizt7b9VvJJPTm1QCgaIi8cMY7wfb0OKFw?=
 =?us-ascii?Q?IQKahR/ylRu9v7NP2gzUeQEmMHSwrdAQGR7Wf/ewLBS50POrJ3NEl7OVa4qr?=
 =?us-ascii?Q?vga6QP6ReQBGLFmaiDRzmhAoZfHSelfArUlpIB7bYaaFKEqXejT3F2VmANtt?=
 =?us-ascii?Q?zXWMb6Js9vYNhASMI2EuhMnG1NcpcWP8SJWq7HeNhunddgi9mWrwpfyiZPcd?=
 =?us-ascii?Q?m6Xwx88bhsIDghZPSmEx9KJN8bFuYaqBlHwOiWXlyFvNzb/ZAslB1SGNoKxk?=
 =?us-ascii?Q?ZYfVsfZNZoMU376MMGyJdc/tK4CzY17UV88nwm0aBZDi5vtnr1LEjpX3XK7t?=
 =?us-ascii?Q?cC5DhFdVwScl2rMeY0KgJDOK4baeYe7Bt0Cg6ye48mj8DMRcYPfhotKOqMvc?=
 =?us-ascii?Q?GLoKzNP/Zd0DU5M1Q14t43kPEWN6D8D5C7pKhgAKQCHCu/HltyvxHzRI1m4t?=
 =?us-ascii?Q?j60gXEcs+DkqaKMmx6mbdF9LF6J6P4Xh2uM6Acf2smr5eKrNbNql4lunhDEr?=
 =?us-ascii?Q?LJuRqQBkQisDw1H83Bcc5DK0exUsYY1qPV0XFmQaz+cahoLtik3tmJKsbg0m?=
 =?us-ascii?Q?xfP8g5EkNQyE+VNtoyL9QoiMkgJISbB5YQe5nmiGrcz87EOFjNgKVvmvQlv2?=
 =?us-ascii?Q?o0h3+ZZL41Wt3p6LoCToTqw32MRNALdBxAioDStnJ8p35gdqeG4S/SFzP/n6?=
 =?us-ascii?Q?acSZFQ0+TZkx4hHYJDAUgXVsdxad8aloEf7Ls7vm4hRpPhTxgrTRUhhcySHh?=
 =?us-ascii?Q?N2ktEwz/DI1dioOMk6+FKIZP+ZhrwlK/jj+Fy+ekjX442jRWyfPZnBI+mrYE?=
 =?us-ascii?Q?ciXtS3ltCqH1Bwz8hsKjLwZ7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a8dac602-16aa-48d1-26ab-08d96ddb3817
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:30:55.7351
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +SVbiIU2qnU9407pjy+3OtFTPIRZSO9Hbbf9tuCpzJ1MD/+lWvgJMG6uazJdtA5WcxX3w7ShMtfFjLxtjYxKEQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

On 01.09.2021 18:13, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 01, 2021 at 04:19:40PM +0200, Jan Beulich wrote:
>> On 01.09.2021 15:56, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Aug 31, 2021 at 10:53:59AM +0200, Jan Beulich wrote:
>>>> On 30.08.2021 15:01, Jan Beulich wrote:
>>>>> The code building PVH Dom0 made use of sequences of P2M changes
>>>>> which are disallowed as of XSA-378. First of all population of the
>>>>> first Mb of memory needs to be redone. Then, largely as a
>>>>> workaround, checking introduced by XSA-378 needs to be slightly
>>>>> relaxed.
>>>>>
>>>>> Note that with these adjustments I get Dom0 to start booting on my
>>>>> development system, but the Dom0 kernel then gets stuck. Since it
>>>>> was the first time for me to try PVH Dom0 in this context (see
>>>>> below for why I was hesitant), I cannot tell yet whether this is
>>>>> due further fallout from the XSA, or some further unrelated
>>>>> problem.
>>>
>>> Iff you have some time could you check without the XSA applied? I have
>>> to admit I haven't been testing staging, so it's possible some
>>> breakage as slipped in (however osstest seemed fine with it).
>>
>> Well, I'd rather try to use the time to find the actual issue. From
>> osstest being fine I'm kind of inferring this might be machine
>> specific, or this might be due to yet some other of the overly many
>> patches I'm carrying. So if I can't infer anything from the stack
>> once I can actually dump that, I may indeed need to bisect my pile,
>> which would then also include the XSA-378 patches (as I didn't have
>> time to re-base so far).
>>
>>>>> Dom0's BSP is in VPF_blocked state while all APs are
>>>>> still in VPF_down. The 'd' debug key, unhelpfully, doesn't produce
>>>>> any output, so it's non-trivial to check whether (like PV likes to
>>>>> do) Dom0 has panic()ed without leaving any (visible) output.
>>>
>>> Not sure it would help much, but maybe you can post the Xen+Linux
>>> output?
>>
>> There's no Linux output yet by that point (and either
>> "earlyprintk=3Dxen" doesn't work in PVH mode, or it's even too early
>> for that). All Xen has to say is
>>
>> (XEN) Dom0 callback via changed to Direct Vector 0xf3
>> (XEN) vmx.c:3265:d0v0 RDMSR 0x0000064e unimplemented
>> (XEN) vmx.c:3265:d0v0 RDMSR 0x00000034 unimplemented
>=20
> Weird, I don't see why earlyprintk=3Dxen shouldn't work in PVH mode,
> unless it's not properly wired up. Certainly needs checking and
> fixing, or else we won't be able to make much progress I think.

Right - I'm intending to check this, including whether at least
xen_raw_console_write() would work.

>>>> Correction: I did mean '0' here, producing merely
>>>>
>>>> (XEN) '0' pressed -> dumping Dom0's registers
>>>> (XEN) *** Dumping Dom0 vcpu#0 state: ***
>>>> (XEN) *** Dumping Dom0 vcpu#1 state: ***
>>>> (XEN) *** Dumping Dom0 vcpu#2 state: ***
>>>> (XEN) *** Dumping Dom0 vcpu#3 state: ***
>>>>
>>>> 'd' output supports the "system is idle" that was also visible from
>>>> 'q' output.
>>>
>>> Can you dump the state of the VMCS and see where the IP points to in
>>> Linux?
>>
>> Both that and the register dumping I have meanwhile working tell
>> me that it's the HLT in default_idle(). IOW Dom0 gives the impression
>> of also being idle, at the first glance. The stack pointer, however,
>> is farther away from the stack top than I would have expected, so it
>> may still have entered default_idle() for other reasons.
>>
>> The VMCS also told me that the last VM entry was to deliver an
>> interrupt at vector 0xf3 (i.e. the "callback" one).
>=20
> That's all quite weird. Did dom0 setup the vCPU timer?

Ah - I had meant to check active timers, but then forgot. Otoh I
thought I could observe vCPU0 waking up from HLT, as RIP in the
registers dumped has been pointing either at it or right past it.
Now that I write this I'm wondering though whether that's an
artifact rather than reflection of something that's really
happening, in particular because of this

(XEN) RSP =3D 0xffffffff81c03eb8 (0xffffffff81c03eb8)  RIP =3D 0xffffffff81=
4be422 (0xffffffff814be423)

in the VMCS dump.

> What version of Linux are you using?

5.13.2; didn't get around to switching to 5.14 yet, but I also don't
expect this to make a difference.

> It seems to get stuck very early (or either fail to output anything
> while booting), which seems unlikely to be related to your specific
> hardware.

Well, it can't be extremely early - I see the ACPI IRQ getting set
up (from "iommu=3Ddebug" output mentioning GSI 9), and I see PCI
device BARs being played with (from debug messages I had added to
vPCI to monitor what P2M adjustments are being requested). As said
on another sub-thread, I get all the way through start_kernel()
and rest_init(), just that apparently some of the steps don't do
what they're supposed to do.

I'm meanwhile wondering whether I'm using a badly configured
kernel, i.e. whether there are any Kconfig settings which I ought
to enable, but which aren't "select"-ed nor have proper
"depends on". What I did is simply take my XEN_PV=3Dy config,
replacing that by XEN_PVH=3Dy. I did observe that this let XEN_DOM0
go off, but according to my checking (at the time) nothing this
crucial should have been affected by that.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:35:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:35:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176958.322041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgJw-0001Uh-5C; Thu, 02 Sep 2021 06:35:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176958.322041; Thu, 02 Sep 2021 06:35:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgJw-0001Ua-1o; Thu, 02 Sep 2021 06:35:32 +0000
Received: by outflank-mailman (input) for mailman id 176958;
 Thu, 02 Sep 2021 06:35:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEJZ=NY=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mLgJu-0001UU-9E
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:35:30 +0000
Received: from mail-pl1-x636.google.com (unknown [2607:f8b0:4864:20::636])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 80600cbb-2ca5-4052-96ee-071c6917b238;
 Thu, 02 Sep 2021 06:35:29 +0000 (UTC)
Received: by mail-pl1-x636.google.com with SMTP id u1so553660plq.5
 for <xen-devel@lists.xenproject.org>; Wed, 01 Sep 2021 23:35:29 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 g4sm1018401pjt.56.2021.09.01.23.35.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 01 Sep 2021 23:35:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80600cbb-2ca5-4052-96ee-071c6917b238
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=p4zDh7Zmvhg146z4qnwrQjuDGyGfYeNoi9EXmtV3vCk=;
        b=D1rnGa8kIjiZ7syMAy++5cDhLy8JHAOLoRVSufAq+d7fBo+t78Ke3s5Yz1uPgVHodR
         U6y3CPKzBmSzyPUZn2Il5d6T3nJ0NLYayesHqiQlQ4AEq/1OrsdnVq5iF5qtETbh4p6f
         YgZRdRXpDzg75R4rrFTDxF1Zz1nXCMjG9r/QcQXP8xK1ZctdFRxf6VSwgwv/ubG846BU
         K739MDCHAYaN+1tdvPFg/QEZ+f5A+dgNahgxMiNjqV+fKo13+NpkXyburB09UGN30GsF
         XrTtEDpmj/CBGDgU+QPL5YGf1gME4ItHnSLzoIa1FCF+YWcVe+sOEyV7VJxulilpk/4U
         stQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=p4zDh7Zmvhg146z4qnwrQjuDGyGfYeNoi9EXmtV3vCk=;
        b=HnOntxY7PPHffI/brxhFU1A2oI6rtw7Vt76QCfjql/e17KC2vdpkAePsfH8Y4JCLy2
         +TF5UyZYJg1S0VV8BLvT+NgGRKrPf+EoVOTb91KB06tZrCgqymI2cGK5xFYrHSJ50leE
         o8jjAWEfUK5KaSVpX5NHwUGh4Jx8OB4hQa3QK7RmusXKr4c4UEcLr4K6Jmi/eVjtwgng
         FSG5mI9tUMN4pz2P3FVzlUawsDxDIOLCmcPMW/M8SWZKABTbfs/ghPa9wPxIYwQkaRSh
         S/9qubGj/fmVdgi3ebS3uMLNxHJCb/ROPyDqpqX/Bt9I/Ub4dEudTszqdCDKrfOyYBJw
         7cxg==
X-Gm-Message-State: AOAM532g7TG6KUCBbbvi/w+iZTF0eWoAm2vMemgzbK4fykFHlWOucCOV
	BG4S+polr96PH9gmB1311xc=
X-Google-Smtp-Source: ABdhPJz4pUoRMJIRNU1Ww7Cq95vazM9v9pAhLPt1toZi1VH9XpEJzie9uEpEa09aj/QIF3t5VvMn8Q==
X-Received: by 2002:a17:90a:509:: with SMTP id h9mr2194087pjh.71.1630564528531;
        Wed, 01 Sep 2021 23:35:28 -0700 (PDT)
Subject: Re: [PATCH V4 02/13] x86/hyperv: Initialize shared memory boundary in
 the Isolation VM.
To: Michael Kelley <mikelley@microsoft.com>, KY Srinivasan
 <kys@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
 "will@kernel.org" <will@kernel.org>, "tglx@linutronix.de"
 <tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>,
 "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
 "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
 <pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
 <hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
 <aneesh.kumar@linux.ibm.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "linux-arm-kernel@lists.infradead.org"
 <linux-arm-kernel@lists.infradead.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "rientjes@google.com" <rientjes@google.com>,
 "ardb@kernel.org" <ardb@kernel.org>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-3-ltykernel@gmail.com>
 <MWHPR21MB1593EF63423A2422DA839793D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <3162bb35-f047-ddbb-3836-6b048b7358e7@gmail.com>
Date: Thu, 2 Sep 2021 14:35:12 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB1593EF63423A2422DA839793D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit


Hi Michael:
       Thanks for your review.

On 9/2/2021 8:15 AM, Michael Kelley wrote:
> From: Tianyu Lan <ltykernel@gmail.com> Sent: Friday, August 27, 2021 10:21 AM
>>
>> Hyper-V exposes shared memory boundary via cpuid
>> HYPERV_CPUID_ISOLATION_CONFIG and store it in the
>> shared_gpa_boundary of ms_hyperv struct. This prepares
>> to share memory with host for SNP guest.
>>
>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>> ---
>> Change since v3:
>> 	* user BIT_ULL to get shared_gpa_boundary
>> 	* Rename field Reserved* to reserved
>> ---
>>   arch/x86/kernel/cpu/mshyperv.c |  2 ++
>>   include/asm-generic/mshyperv.h | 12 +++++++++++-
>>   2 files changed, 13 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
>> index 20557a9d6e25..8bb001198316 100644
>> --- a/arch/x86/kernel/cpu/mshyperv.c
>> +++ b/arch/x86/kernel/cpu/mshyperv.c
>> @@ -313,6 +313,8 @@ static void __init ms_hyperv_init_platform(void)
>>   	if (ms_hyperv.priv_high & HV_ISOLATION) {
>>   		ms_hyperv.isolation_config_a = cpuid_eax(HYPERV_CPUID_ISOLATION_CONFIG);
>>   		ms_hyperv.isolation_config_b = cpuid_ebx(HYPERV_CPUID_ISOLATION_CONFIG);
>> +		ms_hyperv.shared_gpa_boundary =
>> +			BIT_ULL(ms_hyperv.shared_gpa_boundary_bits);
>>
>>   		pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n",
>>   			ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
>> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
>> index 0924bbd8458e..7537ae1db828 100644
>> --- a/include/asm-generic/mshyperv.h
>> +++ b/include/asm-generic/mshyperv.h
>> @@ -35,7 +35,17 @@ struct ms_hyperv_info {
>>   	u32 max_vp_index;
>>   	u32 max_lp_index;
>>   	u32 isolation_config_a;
>> -	u32 isolation_config_b;
>> +	union {
>> +		u32 isolation_config_b;
>> +		struct {
>> +			u32 cvm_type : 4;
>> +			u32 reserved11 : 1;
>> +			u32 shared_gpa_boundary_active : 1;
>> +			u32 shared_gpa_boundary_bits : 6;
>> +			u32 reserved12 : 20;
> 
> I'm still curious about the "11" and "12" in the reserved
> field names.  Why not just "reserved1" and "reserved2"?
> Having the "11" and "12" isn't wrong, but it makes one
> wonder why since it's not usual. :-)
> 

Yes, will update. Thanks.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:45:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:45:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176964.322052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgTj-0003BG-3h; Thu, 02 Sep 2021 06:45:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176964.322052; Thu, 02 Sep 2021 06:45:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgTj-0003B9-0c; Thu, 02 Sep 2021 06:45:39 +0000
Received: by outflank-mailman (input) for mailman id 176964;
 Thu, 02 Sep 2021 06:45:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLgTh-0003B3-OG
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:45:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 605c21ce-0bb9-11ec-ae41-12813bfff9fa;
 Thu, 02 Sep 2021 06:45:36 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2058.outbound.protection.outlook.com [104.47.5.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-p2y-zJ8XNKKXF2aMxxZt1A-1;
 Thu, 02 Sep 2021 08:45:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2605.eurprd04.prod.outlook.com (2603:10a6:800:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.20; Thu, 2 Sep
 2021 06:45:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 06:45:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0008.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Thu, 2 Sep 2021 06:45:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 605c21ce-0bb9-11ec-ae41-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630565135;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2CCz0N6kirv4Y+bKxgrtsTMCCf7h9FVJy0MMlSYiGu8=;
	b=ZOpoVuvfvqggUGmfVmiOmmn+lpSJfOY4TG4SBzBF+SJdJzPymQ9DCVfTCdwlQWuvfkV2vr
	/NR2PONqyBBiL9Y3r1JLSNr2TeBP8aEZtiq730FUE1DRi8DDVyfTv86cqsy6zYDg2nOmEm
	qI0noTOkkmldhLLMyjFXZx1+OLubiB8=
X-MC-Unique: p2y-zJ8XNKKXF2aMxxZt1A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H98I8P6tAF8SV0fMbBr7XkMZcpT8SHAdmIPkz+Ze0mQ8iieyAUSEAsWrspYe9MZ8qu2kVEhD0UO8TMXu+RHOWvR2o9iesqrranmdOHMOB6VYl4E40RFQXbJwhgTekIgpV+777kSrT4hTVTT89pV6iSdqab/L7/wDFirM4g/GFy+AU2SFylqjsVGW1VNnCP1DMN+pXLdiVI+006NTKO9xgmyowSgPq58AzdXd3t70vWniTAN1q0xgqtqDsxCROmtH5XFfDFioEMbPdV7KXRXi2bebdOdCwazH3MZh0QR99jjdODRWFbGrNq4Eog2n/9puvNDCAjmM0NINecg/H3lJZw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2CCz0N6kirv4Y+bKxgrtsTMCCf7h9FVJy0MMlSYiGu8=;
 b=V2ssBBQLYaczegaAcnkwCLAqnWV1Zecrq6x12M4GdjjPlh70kcy6kLcHQ0ysEDceE9tVMNoE/Gl91niyR5vvQYaSm4mN2tVRK5ZzBx7NERqKzQDr4gyo+cwoz4WykjGkzYIVRiuRouEQ4kZY40O9mo/CItEXna3O2DAlvMs4tRScaQxOAhZhGTsltxBbrcF2BpIu/pyGUDMX9KPQfeNb2+B7EoYyGCz285nhGQmjVzlY+q/OBzhRxDKQmNjQoyU3WgOqXlXLsQPzm+U7pyzXRBvYn2G/EhM1wN2ZvnVE0/ebQ+QcgDfQcTQI/YgsWjpLNbqyLGqwWyZkZCt+2O+9yA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: vcpu_show_execution_state() difference between Arm and x86
To: Julien Grall <julien@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
 <8f295eeb-e8f2-a115-3f83-60922d0fe5f9@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <00457189-6ab5-24e4-b2ec-1050ebec67eb@suse.com>
Date: Thu, 2 Sep 2021 08:45:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <8f295eeb-e8f2-a115-3f83-60922d0fe5f9@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0008.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: beabf8b4-90eb-4380-3141-08d96ddd42ee
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2605189E4E8D654B3C6675ABB3CE9@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5kw6HmKoGN55MLGtfyJSgIArVfzD0Dkhyz0+32QQq1+wm2S4qH7jOP6QUeY3oNdUNV+ksmO5ME0HARfG2UfixAypqfAdWpaDFShCr1ubs6SFjMX6DMLXmar+xtRyMkg2agggZTNFFxPAhDz/MjygNJsYTDs8tlc6PCgn6A/V/PVRTWSQQffOCKt0iDFc+UEgW8jL1CImUGVDtFT6GTJP90Gd/0gzvrbRgTqkZC2Zs/tIzb0xyObd8K8vuY0OaAkdA9A3DsFekNmoBF+U9wuDwS4504cnGONDZ+5lCEuNM5vDx9C0bu3jKD6gG68EoMwTdJxGVB1CYYEE7QxI4sCttlybEZ9ISJYTLwLTCrM+Tg8N9vLmerM989y/UIKxyrJ/rPKHamU8wqnE6KsUDlsoqbrrA+3R/eQu81LhzvofTVSTnIDTCeLgwqSFEF9SiBoWCP7TNQtjzzRXmMTlejJXyzATMYO7/h9pgXp1N7SU3LyNF8euTzzK69hvIuaLuqHpif6T0CcL/QCM65ZoffMCLl1ucnMdWS3e8xLs+FygMWxUDVHE0U1vjxDeX8o5bPmcOW6iW25AVtPxld4AOI0vRDQJUSQzHvzpph75B4wSPD7wmyHqmI4jwYvGcYpiJOy/1v0m4Voa/7hD+Y3+A0/vdIBHCNjfvNqjQWzeLHjtX2eg4sQLd4rpldTjPrBgsU2+jiD/6hv3fCMR7GiAZqEGSqqH7gzmy0paeiiYfk6vJ5M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(376002)(346002)(396003)(366004)(38100700002)(2616005)(2906002)(956004)(16576012)(8936002)(8676002)(316002)(86362001)(6486002)(66946007)(478600001)(26005)(54906003)(6916009)(31686004)(5660300002)(4326008)(31696002)(36756003)(83380400001)(66556008)(53546011)(66476007)(186003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VHcvOElJczAwNUloQnIwQkZqZXBLaGVMMUJnTDVVeVBia2FBME1wM3dtVENM?=
 =?utf-8?B?VU9XZG1WajdKMU1wUlB2eEI1VWN6T3h3Y05DeG5EckUzejBZaUlMc3ZYRkNY?=
 =?utf-8?B?Y2hNWlhoOWxVeHJ1NjcxVzRnSGVjUW1zajhSbUd4UFJyNm1pRWVBTjUwUXFt?=
 =?utf-8?B?azdJK3lQQTdqL25vb28rSVVRTzFWM2laMDl1MWRkVGFjQWlmSE1jaXVocFhr?=
 =?utf-8?B?c09ua3QvUTIxNUdKK3J4djFNNDl1V05aSWZnTDJTTExTSjIxSU50elp6czlM?=
 =?utf-8?B?UUhiUXNpZlBzRkFLYjUzekxkZzRkbWxKSE1OL21NTjRYR054bXJ4b1kwQVoz?=
 =?utf-8?B?VE5ESXl5RTlaclNUN2I4cGpjZGk1UE9laTkvQWdTWlQ4WW5PMjdUUTdnd20y?=
 =?utf-8?B?NnNUMzcxNGxlSnpicFo5a2x2T3ZYTTRsWEQzMklyREJldGdGVWtXSXB2bDM3?=
 =?utf-8?B?RjNGUmg5MTlrczJiRUh2VzE5RGlrWllLRzhjYkpiZ3FuWVdKdHNTMVl0dk94?=
 =?utf-8?B?TzFpNElKdDdTNXBudGF3UmZGOFhiZVpJaXhSUjI3Z2xMQUNUcnFlZkN2czNL?=
 =?utf-8?B?d2JLaFA4WHhGL0JqMEIvVzh6ZkpwZlB1Y2hzbWpma1VRam1Pc292dDZGODJG?=
 =?utf-8?B?bE1uNVp0Vm56eStoVzZPZHkzWHR1d0tjRDFyaDB4NlRoT2xWWUY3ZG5EQkI2?=
 =?utf-8?B?YXRFY2lRUWVGQ2VnaVJheVZ0VXY1em9KaWV1VVZhSkhQWnJaeXAwNXo3R2lz?=
 =?utf-8?B?WXYxT3FoTUFsNHM5cWl6dTk1dlNlczVMMVFzTWE4ZTRPT3kxdGc1MzllRjk3?=
 =?utf-8?B?Wm9uTmt5VERZZisxMzlVWWRINXN6Yi8rd3pjMFVlSUdZVVVXUURYTk4rL2Nw?=
 =?utf-8?B?bEpSeHpJbGlqei9yZEhNTkovUm8vUXcvcWJvOGVtTFAwN2xhd2lDRWdKVHBD?=
 =?utf-8?B?QWEyY3Vzc0lVbTN2WDVzdFRpQ01sakNjeWVvNmZuU0ROWVlleEZLMDZINWNZ?=
 =?utf-8?B?ZUp1eFlzMDM3ZEZXYk1YdVB1bGpON2hSMzVMUzlQd1VJT3N3QnFKaTljNGMw?=
 =?utf-8?B?Z29ZVkVqZ2M0ekpKTXYxOEEwamErS3ZjS01hQVRkcFZ6eFR0QXVKTWthalpU?=
 =?utf-8?B?V0lDVjNJbUxuZEM5YmxLTGFldDQzQlBYOFVWU1lJcHQ1dEk3U0FYOUxob0lW?=
 =?utf-8?B?aWRCWHVGL0xFTGEyUmJ1STdjRlU0azlickZPRHNTRldqWElHcktjcTZ6WW1v?=
 =?utf-8?B?bldDNzRadENZdjZhenNPVUxQVkIwelJkZi9WeGpldHpUZnI1eG1uODdud0pL?=
 =?utf-8?B?dEMvWHpsTm8wRVRJNit5bzZSSm9UdEFDTkxpWnZJVWprdmI1b240dEEzenJt?=
 =?utf-8?B?a2JldVViUzdhNStlb0ZURUcxUFdCb3IrTTNsOU5HRTBjdjJXNWYzZ3h2V1BX?=
 =?utf-8?B?Um41Y082d2Z2UUo5alg3VklvTTArSjlaZTZoTmNQa2c2THk5UUtlTnIzZU1y?=
 =?utf-8?B?WTdCOGdueWZ5QTFkRUZha3NlNmh0cHBGUlVSZjREdEM3QzNBcW1ZMXF3ZFRn?=
 =?utf-8?B?d2dZcG1hbW16R2V5UzBBTUxZZVNCT3l6NXBkSWlYQ0puTGpLb2RWZVEvZWMv?=
 =?utf-8?B?MnVlc2JyQ3J5a1pvc29CWjhMalRxZTh6YjBpRHFabFpJM1k0cWdMSjBIR3BE?=
 =?utf-8?B?b1RHL04rZ1pHL2thUHloaiswZzl2U0Q5dFZvdVdaRlU2YTYyTDhXUjBVK3pa?=
 =?utf-8?Q?yp1WMIhrspLz9ctZusv0t5t3JRjoJaxlbmSjMW7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: beabf8b4-90eb-4380-3141-08d96ddd42ee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:45:32.8741
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Gl4/yAgtASFgtNl/BHKi+5WXBNQJdo/OoSMDPMiceH7TsA+mLTlBsaod3HREugOFphk8+QlUhBRUasraB3p9Lg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 01.09.2021 20:11, Julien Grall wrote:
> On 01/09/2021 14:39, Jan Beulich wrote:
>> back in 2016 Andrew added code to x86'es variant to avoid interleaving
>> of output. The same issue ought to exist on Arm.
> 
> Agree. I guess we got away so far because it is pretty rare to have two 
> CPUs running at the same.

I guess you've meant "crashing", not "running"?

>> The lock acquired,
>> or more importantly the turning off of IRQs while doing so, is now
>> getting in the way of having PVH Dom0's state dumped the 2nd time. 
> 
> I am not quite too sure to understand the problem with PVH dom0. Do you 
> have a pointer to the issue?

Locking in both cases: For registers it was VMX'es vmx_vmcs_enter()
acquiring a (non-IRQ-safe) lock, and for the stack it is the P2M lock.
Neither can / should sensibly be made IRQ-safe.

>> For
>> register state I did find a sufficiently simple (yet not pretty)
>> workaround. For the stack, where I can't reasonably avoid using p2m
>> functions, this is going to be more difficult. >
>> Since I expect Arm to want to also have interleave protection at some
>> point, and since Arm also acquires the p2m lock while accessing Dom0's
>> stacks, I wonder whether anyone has any clever idea on how to avoid
>> the (valid) triggering of check_lock()'s assertion without intrusive
>> changes. (As to intrusive changes - acquiring the p2m lock up front in
>> recursive mode, plus silencing check_lock() for nested acquires of a
>> lock that's already being held by a CPU was my initial idea.)
> 
> At least one Arm, the P2M lock is a rwlock which is not yet recursive. 

Right; the same is tru on x86. Hence the expected intrusiveness.

> But then it feels to me that this solution is only going to cause us 
> more trouble in the future.

Same here - we'd need to be very careful not only when making such
a change, but also going forward. Hence my desire to come up with a
better approach.

> I looked at the original commit to find out the reason to use the 
> console lock. AFAICT, this was to allow console_force_unlock() to work 
> properly. But it is not entirely clear why we couldn't get a new lock 
> (with IRQ enabled) that could be forced unlocked in that function.
> 
> Can either you or Andrew clarify it?

AIUI any new lock would need to be IRQ-safe as well, as the lock
would be on paths taken to output stuff when the system crashed.
Hence it would be pointless to introduce yet another lock when the
one we have is already good enough. But I may be missing aspects,
in which case I'd have to defer to Andrew.

> The other solution I can think off is buffering the output for 
> show_registers and only print it once at the end. The downside is we may 
> not get any output if there is an issue in the middle of the dump.

Indeed; I'd prefer to avoid that, the more that it may be hard to
predict how much output there's going to be.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:50:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176921.322112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYs-0005Po-JV; Thu, 02 Sep 2021 06:50:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176921.322112; Thu, 02 Sep 2021 06:50:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYs-0005LP-69; Thu, 02 Sep 2021 06:50:58 +0000
Received: by outflank-mailman (input) for mailman id 176921;
 Thu, 02 Sep 2021 06:07:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsa-0004fO-NW
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:16 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com (unknown
 [40.107.101.78]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 68a4c72f-d06d-41dc-b586-46d6db34e47a;
 Thu, 02 Sep 2021 06:07:13 +0000 (UTC)
Received: from SA0PR11CA0205.namprd11.prod.outlook.com (2603:10b6:806:1bc::30)
 by DM6PR02MB4955.namprd02.prod.outlook.com (2603:10b6:5:11::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:07:11 +0000
Received: from SN1NAM02FT0051.eop-nam02.prod.protection.outlook.com
 (2603:10b6:806:1bc:cafe::14) by SA0PR11CA0205.outlook.office365.com
 (2603:10b6:806:1bc::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:11 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0051.mail.protection.outlook.com (10.97.5.34) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:11 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:06:58 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:06:58 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsI-000F6q-7C; Wed, 01 Sep 2021 23:06:58 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68a4c72f-d06d-41dc-b586-46d6db34e47a
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dTzketYN7Sz9QYVI5TRbaykYekb6lQdCGNL14wDwjJbJvkqKNVRJCetHXBYkG+hsmyd2Ec8byYIh61Gtd2EfhNv6hb1OZQmxvAJFlVSjrJZkajCzIt1gklA0SW5eD820aD3fY4CFtvmf0mHa94SgyWefsoiIGE2omU+sX7X4WRAdIDIj37C6YiSZNFG4wRBfqpdnLOqg8LqpIaElCJoPa4kc5jt69+btV+7MVARN50Zx3Vu9rB8/QzHdYkNWb1GzlmiZqCF5xKw4iScINXD4X6ROBD6N+iyo4cEObCCVMxtuUlz2qPvNKTSiqusuDCEyPbJEKHNJd6j+Xn2+B36tjQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=odPD3Pqm1GqjPKTu4SCQEahFowuxrRf5d2NDi//Xqhc=;
 b=b7t7Q08EDV9jbIAi9YacfuBNdYe5UT9DpmdR6zksyt/pmhLaGS2kRte8K2x4uJaEm1NWO9Op5HQKZ91pulo2xbQKc2MWcAy0iyGt0rNlJOvfWNZ1gfqN+kcnNhhAQx8niMQQNIUPx8MelvpDp/Fd3bftwrhxpk4quWX3He2RBQ5nXzbYfFQFv9e0fJitZRMvba7N4Y3o7tz2NiL25fwOOuTTo05iHtxSl4xZmQlB3eZTdGSssp9UAawVn9G00bkhmYYy9U/CihHVRADRV0qxkhHQEixrShWiwoU8Yqm8S+8kOvxPM8/FNRzvu7pMOFbq6E8hdCZ68Zl+zjWVxdZn7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=odPD3Pqm1GqjPKTu4SCQEahFowuxrRf5d2NDi//Xqhc=;
 b=ptPq88z/q1iJFHOMrdxAUhpYrrVwcYiB+PwZY66JsKYHwoqXy9IQ133PTfzXon9J80oEQU/sriailXFqa2EXkeJeejxPwYvTGTU1IemAuXExyfpRgWL+DL4VVKHZNNKqgRhkx+94la5pgFnFq/6i2nBjTrj/mRXo0cPoqY1nW4w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 04/13] libfdt: Copy required libfdt functions from Linux
Date: Wed, 1 Sep 2021 23:05:54 -0700
Message-ID: <1630562763-390068-5-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8327c211-e114-4ee9-c4a0-08d96dd7e711
X-MS-TrafficTypeDiagnostic: DM6PR02MB4955:
X-Microsoft-Antispam-PRVS:
	<DM6PR02MB495574433775A4EC89D28ECDBCCE9@DM6PR02MB4955.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cnMudUiEYwg/DRZQQkVkoNMDqoPlKtpzwpPFJo7zR8JLQKmfQrcYbRsNCvbg5qnJkg+uaCcQwgsriP26Jxf2LXmVRuAYzfbj4Z/29wGNs0lLrQyqoJ+5zzUa693okFFhTgRSmPx3BmfeU14ssAhxNHLtwAzZyG5vLj5xVkiT3SyLaH5Zdhy5V6u+uni9bxXwxgR2RxqhgooqPAjciVrZGBSemg0bOlZyncSpCBfICq5SaIGN9WMJI+Z/GlWCQAgjK4Az3DcemyaqG9+vmIaQdWj4+crR+/P62B8QW58x9pCy2yuyFR647M43LHE6xXMwJDuf306VPsCITNoSYfD1UPK4Tv8wifiCrwDMkIgpVtR6IDJ5W4MOviwPdbQrjRwXOCmm4yWKqRVaZwjt2xGhaqRx2fL1IPLggcodKjr1nUN4IlsvdXPRtHXWubnPufa2NBth9VoYgZfKV/ccIqfqL1xhCePueVMrFBlc3G/EERsFKIqjS0JKygrwHqDCu17zmug8CRjf1/28MPDeO91F9bbAfPgQNxdiXH4yrCUveMJRy9KRCDuwEDd8bh1z+TJfrBsFm5i35botvPmsX3jLVxv7a4EjPNr1YQU1/5TkRbj7HmOOpgh6Ivmu6xQam2X7fAIBmCSUrTmj8S5YbyvBH1ZE5jYHC1gmojKwzCKjy4MB7jkNKU2xVGSKnYnpNIGk551FBZi2fpc28QysuVyu1qmgVEiKnhr11zBwZMf4euM=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(346002)(396003)(376002)(36840700001)(46966006)(47076005)(426003)(186003)(36756003)(107886003)(6666004)(8676002)(2616005)(7636003)(2906002)(70586007)(6916009)(70206006)(82310400003)(83380400001)(82740400003)(26005)(36906005)(336012)(356005)(316002)(478600001)(5660300002)(7696005)(4326008)(9786002)(54906003)(36860700001)(30864003)(8936002)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:11.0913
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8327c211-e114-4ee9-c4a0-08d96dd7e711
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0051.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB4955

fdt_overlay.c uses a number of functions. Below is the list of functions copied
from Linux tree(commid: 6e9c9686d826564f44c93cdd6f111b1c0a9dc224) to compile
the fdt_overlay.c():

libfdt_internal.h: FDT_RO_PROBE() and can_assume().

libfdt.h: fdt_for_each_subnode(), fdt_get_max_phandle(),
    fdt_for_each_property_offset() and fdt_getprop_namelen_w().

fdt.c: fdt_ro_probe_().

fdt_ro.c: fdt_find_max_phandle(), fdt_path_offset_namelen() and
   fdt_path_offset().

fdt_rw.c: fdt_rw_probe_(), FDT_RW_PROBE(), fdt_del_last_string_(),
     fdt_setprop_placeholder() and fdt_setprop().

fdt_wip.c: fdt_setprop_inplace_namelen_partial().

Updated fdt_rw.c: _fdt_find_add_string() with required changes from Linux
    fdt_rw.c:fdt_find_add_string_().
Updated fdt_rw.c: _fdt_add_property() with required changes from Linux
    fdt_rw.c:fdt_find_add_string_().

Replaced strtoul() with simple_strtoull().

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/libfdt/Makefile.libfdt   |   2 +-
 xen/common/libfdt/fdt.c             |  35 ++++++
 xen/common/libfdt/fdt_overlay.c     |   6 +-
 xen/common/libfdt/fdt_ro.c          |  52 ++++++--
 xen/common/libfdt/fdt_rw.c          |  81 +++++++++++--
 xen/common/libfdt/fdt_wip.c         |  20 ++++
 xen/common/libfdt/libfdt_internal.h | 130 ++++++++++++++++++++
 xen/include/xen/libfdt/libfdt.h     | 230 +++++++++++++++++++++++++++++++++++-
 8 files changed, 536 insertions(+), 20 deletions(-)

diff --git a/xen/common/libfdt/Makefile.libfdt b/xen/common/libfdt/Makefile.libfdt
index 91126c0..aea9d9b 100644
--- a/xen/common/libfdt/Makefile.libfdt
+++ b/xen/common/libfdt/Makefile.libfdt
@@ -6,5 +6,5 @@
 LIBFDT_soname = libfdt.$(SHAREDLIB_EXT).1
 LIBFDT_INCLUDES = fdt.h libfdt.h libfdt_env.h
 LIBFDT_VERSION = version.lds
-LIBFDT_SRCS = fdt.c fdt_ro.c fdt_wip.c fdt_sw.c fdt_rw.c fdt_strerror.c fdt_empty_tree.c
+LIBFDT_SRCS = fdt.c fdt_ro.c fdt_wip.c fdt_sw.c fdt_rw.c fdt_strerror.c fdt_empty_tree.c fdt_overlay.c
 LIBFDT_OBJS = $(LIBFDT_SRCS:%.c=%.o)
diff --git a/xen/common/libfdt/fdt.c b/xen/common/libfdt/fdt.c
index bbc7717..55a9de6 100644
--- a/xen/common/libfdt/fdt.c
+++ b/xen/common/libfdt/fdt.c
@@ -53,6 +53,41 @@
 
 #include "libfdt_internal.h"
 
+/*
+ * Minimal sanity check for a read-only tree. fdt_ro_probe_() checks
+ * that the given buffer contains what appears to be a flattened
+ * device tree with sane information in its header.
+ */
+int32_t fdt_ro_probe_(const void *fdt)
+{
+	uint32_t totalsize = fdt_totalsize(fdt);
+
+	if (can_assume(VALID_DTB))
+		return totalsize;
+
+	if (fdt_magic(fdt) == FDT_MAGIC) {
+		/* Complete tree */
+		if (!can_assume(LATEST)) {
+			if (fdt_version(fdt) < FDT_FIRST_SUPPORTED_VERSION)
+				return -FDT_ERR_BADVERSION;
+			if (fdt_last_comp_version(fdt) >
+					FDT_LAST_SUPPORTED_VERSION)
+				return -FDT_ERR_BADVERSION;
+		}
+	} else if (fdt_magic(fdt) == FDT_SW_MAGIC) {
+		/* Unfinished sequential-write blob */
+		if (!can_assume(VALID_INPUT) && fdt_size_dt_struct(fdt) == 0)
+			return -FDT_ERR_BADSTATE;
+	} else {
+		return -FDT_ERR_BADMAGIC;
+	}
+
+	if (totalsize < INT32_MAX)
+		return totalsize;
+	else
+		return -FDT_ERR_TRUNCATED;
+}
+
 int fdt_check_header(const void *fdt)
 {
 	if (fdt_magic(fdt) == FDT_MAGIC) {
diff --git a/xen/common/libfdt/fdt_overlay.c b/xen/common/libfdt/fdt_overlay.c
index d217e79..15a8cdb 100644
--- a/xen/common/libfdt/fdt_overlay.c
+++ b/xen/common/libfdt/fdt_overlay.c
@@ -9,6 +9,7 @@
 #include <fdt.h>
 #include <libfdt.h>
 
+#include <xen/lib.h>
 #include "libfdt_internal.h"
 
 /**
@@ -446,7 +447,8 @@ static int overlay_fixup_phandle(void *fdt, void *fdto, int symbols_off,
 		const char *fixup_str = value;
 		uint32_t path_len, name_len;
 		uint32_t fixup_len;
-		char *sep, *endptr;
+		char *sep;
+		const char *endptr;
 		int poffset, ret;
 
 		fixup_end = memchr(value, '\0', len);
@@ -476,7 +478,7 @@ static int overlay_fixup_phandle(void *fdt, void *fdto, int symbols_off,
 		if (!name_len)
 			return -FDT_ERR_BADOVERLAY;
 
-		poffset = strtoul(sep + 1, &endptr, 10);
+		poffset = simple_strtoull(sep + 1, &endptr, 10);
 		if ((*endptr != '\0') || (endptr <= (sep + 1)))
 			return -FDT_ERR_BADOVERLAY;
 
diff --git a/xen/common/libfdt/fdt_ro.c b/xen/common/libfdt/fdt_ro.c
index 36f9b48..383791d 100644
--- a/xen/common/libfdt/fdt_ro.c
+++ b/xen/common/libfdt/fdt_ro.c
@@ -86,6 +86,34 @@ static int _fdt_string_eq(const void *fdt, int stroffset,
 	return (strlen(p) == len) && (memcmp(p, s, len) == 0);
 }
 
+int fdt_find_max_phandle(const void *fdt, uint32_t *phandle)
+{
+	uint32_t max = 0;
+	int offset = -1;
+
+	while (true) {
+		uint32_t value;
+
+		offset = fdt_next_node(fdt, offset, NULL);
+		if (offset < 0) {
+			if (offset == -FDT_ERR_NOTFOUND)
+				break;
+
+			return offset;
+		}
+
+		value = fdt_get_phandle(fdt, offset);
+
+		if (value > max)
+			max = value;
+	}
+
+	if (phandle)
+		*phandle = max;
+
+	return 0;
+}
+
 int fdt_get_mem_rsv(const void *fdt, int n, uint64_t *address, uint64_t *size)
 {
 	FDT_CHECK_HEADER(fdt);
@@ -152,17 +180,17 @@ int fdt_subnode_offset(const void *fdt, int parentoffset,
 	return fdt_subnode_offset_namelen(fdt, parentoffset, name, strlen(name));
 }
 
-int fdt_path_offset(const void *fdt, const char *path)
+int fdt_path_offset_namelen(const void *fdt, const char *path, int namelen)
 {
-	const char *end = path + strlen(path);
+	const char *end = path + namelen;
 	const char *p = path;
 	int offset = 0;
 
-	FDT_CHECK_HEADER(fdt);
+	FDT_RO_PROBE(fdt);
 
 	/* see if we have an alias */
 	if (*path != '/') {
-		const char *q = strchr(path, '/');
+		const char *q = memchr(path, '/', end - p);
 
 		if (!q)
 			q = end;
@@ -175,14 +203,15 @@ int fdt_path_offset(const void *fdt, const char *path)
 		p = q;
 	}
 
-	while (*p) {
+	while (p < end) {
 		const char *q;
 
-		while (*p == '/')
+		while (*p == '/') {
 			p++;
-		if (! *p)
-			return offset;
-		q = strchr(p, '/');
+			if (p == end)
+				return offset;
+		}
+		q = memchr(p, '/', end - p);
 		if (! q)
 			q = end;
 
@@ -196,6 +225,11 @@ int fdt_path_offset(const void *fdt, const char *path)
 	return offset;
 }
 
+int fdt_path_offset(const void *fdt, const char *path)
+{
+	return fdt_path_offset_namelen(fdt, path, strlen(path));
+}
+
 const char *fdt_get_name(const void *fdt, int nodeoffset, int *len)
 {
 	const struct fdt_node_header *nh = _fdt_offset_ptr(fdt, nodeoffset);
diff --git a/xen/common/libfdt/fdt_rw.c b/xen/common/libfdt/fdt_rw.c
index 8b8cd25..fc53644 100644
--- a/xen/common/libfdt/fdt_rw.c
+++ b/xen/common/libfdt/fdt_rw.c
@@ -65,6 +65,30 @@ static int _fdt_blocks_misordered(const void *fdt,
 		    (fdt_off_dt_strings(fdt) + fdt_size_dt_strings(fdt)));
 }
 
+static int fdt_rw_probe_(void *fdt)
+{
+	if (can_assume(VALID_DTB))
+		return 0;
+	FDT_RO_PROBE(fdt);
+
+	if (!can_assume(LATEST) && fdt_version(fdt) < 17)
+		return -FDT_ERR_BADVERSION;
+	if (_fdt_blocks_misordered(fdt, sizeof(struct fdt_reserve_entry),
+				   fdt_size_dt_struct(fdt)))
+		return -FDT_ERR_BADLAYOUT;
+	if (!can_assume(LATEST) && fdt_version(fdt) > 17)
+		fdt_set_version(fdt, 17);
+
+	return 0;
+}
+
+#define FDT_RW_PROBE(fdt) \
+	{ \
+		int err_; \
+		if ((err_ = fdt_rw_probe_(fdt)) != 0) \
+			return err_; \
+	}
+
 static int _fdt_rw_check_header(void *fdt)
 {
 	FDT_CHECK_HEADER(fdt);
@@ -133,6 +157,14 @@ static int _fdt_splice_struct(void *fdt, void *p,
 	return 0;
 }
 
+/* Must only be used to roll back in case of error */
+static void fdt_del_last_string_(void *fdt, const char *s)
+{
+	int newlen = strlen(s) + 1;
+
+	fdt_set_size_dt_strings(fdt, fdt_size_dt_strings(fdt) - newlen);
+}
+
 static int _fdt_splice_string(void *fdt, int newlen)
 {
 	void *p = (char *)fdt
@@ -146,7 +178,16 @@ static int _fdt_splice_string(void *fdt, int newlen)
 	return 0;
 }
 
-static int _fdt_find_add_string(void *fdt, const char *s)
+/**
+ * New _fdt_find_add_string() - Find or allocate a string
+ *
+ * @fdt: pointer to the device tree to check/adjust
+ * @s: string to find/add
+ * @allocated: Set to 0 if the string was found, 1 if not found and so
+ *  allocated. Ignored if can_assume(NO_ROLLBACK)
+ * @return offset of string in the string table (whether found or added)
+ */
+static int _fdt_find_add_string(void *fdt, const char *s, int *allocated)
 {
 	char *strtab = (char *)fdt + fdt_off_dt_strings(fdt);
 	const char *p;
@@ -154,6 +195,9 @@ static int _fdt_find_add_string(void *fdt, const char *s)
 	int len = strlen(s) + 1;
 	int err;
 
+	if (!can_assume(NO_ROLLBACK))
+		*allocated = 0;
+
 	p = _fdt_find_string(strtab, fdt_size_dt_strings(fdt), s);
 	if (p)
 		/* found it */
@@ -164,6 +208,9 @@ static int _fdt_find_add_string(void *fdt, const char *s)
 	if (err)
 		return err;
 
+	if (!can_assume(NO_ROLLBACK))
+		*allocated = 1;
+
 	memcpy(new, s, len);
 	return (new - strtab);
 }
@@ -226,11 +273,12 @@ static int _fdt_add_property(void *fdt, int nodeoffset, const char *name,
 	int nextoffset;
 	int namestroff;
 	int err;
+	int allocated;
 
 	if ((nextoffset = _fdt_check_node_offset(fdt, nodeoffset)) < 0)
 		return nextoffset;
 
-	namestroff = _fdt_find_add_string(fdt, name);
+	namestroff = _fdt_find_add_string(fdt, name, &allocated);
 	if (namestroff < 0)
 		return namestroff;
 
@@ -238,8 +286,12 @@ static int _fdt_add_property(void *fdt, int nodeoffset, const char *name,
 	proplen = sizeof(**prop) + FDT_TAGALIGN(len);
 
 	err = _fdt_splice_struct(fdt, *prop, 0, proplen);
-	if (err)
+	if (err) {
+		/* Delete the string if we failed to add it */
+		if (!can_assume(NO_ROLLBACK) && allocated)
+			fdt_del_last_string_(fdt, name);
 		return err;
+	}
 
 	(*prop)->tag = cpu_to_fdt32(FDT_PROP);
 	(*prop)->nameoff = cpu_to_fdt32(namestroff);
@@ -270,13 +322,13 @@ int fdt_set_name(void *fdt, int nodeoffset, const char *name)
 	return 0;
 }
 
-int fdt_setprop(void *fdt, int nodeoffset, const char *name,
-		const void *val, int len)
+int fdt_setprop_placeholder(void *fdt, int nodeoffset, const char *name,
+				int len, void **prop_data)
 {
 	struct fdt_property *prop;
 	int err;
 
-	FDT_RW_CHECK_HEADER(fdt);
+	FDT_RW_PROBE(fdt);
 
 	err = _fdt_resize_property(fdt, nodeoffset, name, len, &prop);
 	if (err == -FDT_ERR_NOTFOUND)
@@ -284,7 +336,22 @@ int fdt_setprop(void *fdt, int nodeoffset, const char *name,
 	if (err)
 		return err;
 
-	memcpy(prop->data, val, len);
+	*prop_data = prop->data;
+	return 0;
+}
+
+int fdt_setprop(void *fdt, int nodeoffset, const char *name,
+		const void *val, int len)
+{
+	void *prop_data;
+	int err;
+
+	err = fdt_setprop_placeholder(fdt, nodeoffset, name, len, &prop_data);
+	if (err)
+		return err;
+
+	if (len)
+		memcpy(prop_data, val, len);
 	return 0;
 }
 
diff --git a/xen/common/libfdt/fdt_wip.c b/xen/common/libfdt/fdt_wip.c
index 2d1cac0..3f61085 100644
--- a/xen/common/libfdt/fdt_wip.c
+++ b/xen/common/libfdt/fdt_wip.c
@@ -53,6 +53,26 @@
 
 #include "libfdt_internal.h"
 
+int fdt_setprop_inplace_namelen_partial(void *fdt, int nodeoffset,
+					const char *name, int namelen,
+					uint32_t idx, const void *val,
+					int len)
+{
+	void *propval;
+	int proplen;
+
+	propval = fdt_getprop_namelen_w(fdt, nodeoffset, name, namelen,
+					&proplen);
+	if (!propval)
+		return proplen;
+
+	if ((unsigned)proplen < (len + idx))
+		return -FDT_ERR_NOSPACE;
+
+	memcpy((char *)propval + idx, val, len);
+	return 0;
+}
+
 int fdt_setprop_inplace(void *fdt, int nodeoffset, const char *name,
 			const void *val, int len)
 {
diff --git a/xen/common/libfdt/libfdt_internal.h b/xen/common/libfdt/libfdt_internal.h
index d50c4e1..523bcee 100644
--- a/xen/common/libfdt/libfdt_internal.h
+++ b/xen/common/libfdt/libfdt_internal.h
@@ -60,6 +60,14 @@
 			return err; \
 	}
 
+int32_t fdt_ro_probe_(const void *fdt);
+#define FDT_RO_PROBE(fdt)					\
+	{							\
+		int32_t totalsize_;				\
+		if ((totalsize_ = fdt_ro_probe_(fdt)) < 0)	\
+			return totalsize_;			\
+	}
+
 int _fdt_check_node_offset(const void *fdt, int offset);
 int _fdt_check_prop_offset(const void *fdt, int offset);
 const char *_fdt_find_string(const char *strtab, int tabsize, const char *s);
@@ -90,4 +98,126 @@ static inline struct fdt_reserve_entry *_fdt_mem_rsv_w(void *fdt, int n)
 
 #define FDT_SW_MAGIC		(~FDT_MAGIC)
 
+/**********************************************************************/
+/* Checking controls                                                  */
+/**********************************************************************/
+
+#ifndef FDT_ASSUME_MASK
+#define FDT_ASSUME_MASK 0
+#endif
+
+/*
+ * Defines assumptions which can be enabled. Each of these can be enabled
+ * individually. For maximum safety, don't enable any assumptions!
+ *
+ * For minimal code size and no safety, use ASSUME_PERFECT at your own risk.
+ * You should have another method of validating the device tree, such as a
+ * signature or hash check before using libfdt.
+ *
+ * For situations where security is not a concern it may be safe to enable
+ * ASSUME_SANE.
+ */
+enum {
+	/*
+	 * This does essentially no checks. Only the latest device-tree
+	 * version is correctly handled. Inconsistencies or errors in the device
+	 * tree may cause undefined behaviour or crashes. Invalid parameters
+	 * passed to libfdt may do the same.
+	 *
+	 * If an error occurs when modifying the tree it may leave the tree in
+	 * an intermediate (but valid) state. As an example, adding a property
+	 * where there is insufficient space may result in the property name
+	 * being added to the string table even though the property itself is
+	 * not added to the struct section.
+	 *
+	 * Only use this if you have a fully validated device tree with
+	 * the latest supported version and wish to minimise code size.
+	 */
+	ASSUME_PERFECT		= 0xff,
+
+	/*
+	 * This assumes that the device tree is sane. i.e. header metadata
+	 * and basic hierarchy are correct.
+	 *
+	 * With this assumption enabled, normal device trees produced by libfdt
+	 * and the compiler should be handled safely. Malicious device trees and
+	 * complete garbage may cause libfdt to behave badly or crash. Truncated
+	 * device trees (e.g. those only partially loaded) can also cause
+	 * problems.
+	 *
+	 * Note: Only checks that relate exclusively to the device tree itself
+	 * (not the parameters passed to libfdt) are disabled by this
+	 * assumption. This includes checking headers, tags and the like.
+	 */
+	ASSUME_VALID_DTB	= 1 << 0,
+
+	/*
+	 * This builds on ASSUME_VALID_DTB and further assumes that libfdt
+	 * functions are called with valid parameters, i.e. not trigger
+	 * FDT_ERR_BADOFFSET or offsets that are out of bounds. It disables any
+	 * extensive checking of parameters and the device tree, making various
+	 * assumptions about correctness.
+	 *
+	 * It doesn't make sense to enable this assumption unless
+	 * ASSUME_VALID_DTB is also enabled.
+	 */
+	ASSUME_VALID_INPUT	= 1 << 1,
+
+	/*
+	 * This disables checks for device-tree version and removes all code
+	 * which handles older versions.
+	 *
+	 * Only enable this if you know you have a device tree with the latest
+	 * version.
+	 */
+	ASSUME_LATEST		= 1 << 2,
+
+	/*
+	 * This assumes that it is OK for a failed addition to the device tree,
+	 * due to lack of space or some other problem, to skip any rollback
+	 * steps (such as dropping the property name from the string table).
+	 * This is safe to enable in most circumstances, even though it may
+	 * leave the tree in a sub-optimal state.
+	 */
+	ASSUME_NO_ROLLBACK	= 1 << 3,
+
+	/*
+	 * This assumes that the device tree components appear in a 'convenient'
+	 * order, i.e. the memory reservation block first, then the structure
+	 * block and finally the string block.
+	 *
+	 * This order is not specified by the device-tree specification,
+	 * but is expected by libfdt. The device-tree compiler always created
+	 * device trees with this order.
+	 *
+	 * This assumption disables a check in fdt_open_into() and removes the
+	 * ability to fix the problem there. This is safe if you know that the
+	 * device tree is correctly ordered. See fdt_blocks_misordered_().
+	 */
+	ASSUME_LIBFDT_ORDER	= 1 << 4,
+
+	/*
+	 * This assumes that libfdt itself does not have any internal bugs. It
+	 * drops certain checks that should never be needed unless libfdt has an
+	 * undiscovered bug.
+	 *
+	 * This can generally be considered safe to enable.
+	 */
+	ASSUME_LIBFDT_FLAWLESS	= 1 << 5,
+};
+
+/**
+ * can_assume_() - check if a particular assumption is enabled
+ *
+ * @mask: Mask to check (ASSUME_...)
+ * @return true if that assumption is enabled, else false
+ */
+static inline bool can_assume_(int mask)
+{
+	return FDT_ASSUME_MASK & mask;
+}
+
+/** helper macros for checking assumptions */
+#define can_assume(_assume)	can_assume_(ASSUME_ ## _assume)
+
 #endif /* _LIBFDT_INTERNAL_H */
diff --git a/xen/include/xen/libfdt/libfdt.h b/xen/include/xen/libfdt/libfdt.h
index 7c75688..b6c8b67 100644
--- a/xen/include/xen/libfdt/libfdt.h
+++ b/xen/include/xen/libfdt/libfdt.h
@@ -114,7 +114,30 @@
 	 * Should never be returned, if it is, it indicates a bug in
 	 * libfdt itself. */
 
-#define FDT_ERR_MAX		13
+/* Errors in device tree content */
+#define FDT_ERR_BADNCELLS	14
+	/* FDT_ERR_BADNCELLS: Device tree has a #address-cells, #size-cells
+	 * or similar property with a bad format or value */
+
+#define FDT_ERR_BADVALUE	15
+	/* FDT_ERR_BADVALUE: Device tree has a property with an unexpected
+	 * value. For example: a property expected to contain a string list
+	 * is not NUL-terminated within the length of its value. */
+
+#define FDT_ERR_BADOVERLAY	16
+	/* FDT_ERR_BADOVERLAY: The device tree overlay, while
+	 * correctly structured, cannot be applied due to some
+	 * unexpected or missing value, property or node. */
+
+#define FDT_ERR_NOPHANDLES	17
+	/* FDT_ERR_NOPHANDLES: The device tree doesn't have any
+	 * phandle available anymore without causing an overflow */
+
+#define FDT_ERR_BADFLAGS	18
+	/* FDT_ERR_BADFLAGS: The function was passed a flags field that
+	 * contains invalid flags or an invalid combination of flags. */
+
+#define FDT_ERR_MAX		18
 
 /**********************************************************************/
 /* Low-level functions (you probably don't need these)                */
@@ -156,6 +179,33 @@ int fdt_first_subnode(const void *fdt, int offset);
  */
 int fdt_next_subnode(const void *fdt, int offset);
 
+/**
+ * fdt_for_each_subnode - iterate over all subnodes of a parent
+ *
+ * @node:	child node (int, lvalue)
+ * @fdt:	FDT blob (const void *)
+ * @parent:	parent node (int)
+ *
+ * This is actually a wrapper around a for loop and would be used like so:
+ *
+ *	fdt_for_each_subnode(node, fdt, parent) {
+ *		Use node
+ *		...
+ *	}
+ *
+ *	if ((node < 0) && (node != -FDT_ERR_NOTFOUND)) {
+ *		Error handling
+ *	}
+ *
+ * Note that this is implemented as a macro and @node is used as
+ * iterator in the loop. The parent variable be constant or even a
+ * literal.
+ */
+#define fdt_for_each_subnode(node, fdt, parent)		\
+	for (node = fdt_first_subnode(fdt, parent);	\
+	     node >= 0;					\
+	     node = fdt_next_subnode(fdt, node))
+
 /**********************************************************************/
 /* General functions                                                  */
 /**********************************************************************/
@@ -247,6 +297,47 @@ int fdt_move(const void *fdt, void *buf, int bufsize);
 const char *fdt_string(const void *fdt, int stroffset);
 
 /**
+ * fdt_find_max_phandle - find and return the highest phandle in a tree
+ * @fdt: pointer to the device tree blob
+ * @phandle: return location for the highest phandle value found in the tree
+ *
+ * fdt_find_max_phandle() finds the highest phandle value in the given device
+ * tree. The value returned in @phandle is only valid if the function returns
+ * success.
+ *
+ * returns:
+ *     0 on success or a negative error code on failure
+ */
+int fdt_find_max_phandle(const void *fdt, uint32_t *phandle);
+
+/**
+ * fdt_get_max_phandle - retrieves the highest phandle in a tree
+ * @fdt: pointer to the device tree blob
+ *
+ * fdt_get_max_phandle retrieves the highest phandle in the given
+ * device tree. This will ignore badly formatted phandles, or phandles
+ * with a value of 0 or -1.
+ *
+ * This function is deprecated in favour of fdt_find_max_phandle().
+ *
+ * returns:
+ *      the highest phandle on success
+ *      0, if no phandle was found in the device tree
+ *      -1, if an error occurred
+ */
+static inline uint32_t fdt_get_max_phandle(const void *fdt)
+{
+	uint32_t phandle;
+	int err;
+
+	err = fdt_find_max_phandle(fdt, &phandle);
+	if (err < 0)
+		return (uint32_t)-1;
+
+	return phandle;
+}
+
+/**
  * fdt_num_mem_rsv - retrieve the number of memory reserve map entries
  * @fdt: pointer to the device tree blob
  *
@@ -316,6 +407,21 @@ int fdt_subnode_offset_namelen(const void *fdt, int parentoffset,
 int fdt_subnode_offset(const void *fdt, int parentoffset, const char *name);
 
 /**
+ * fdt_path_offset_namelen - find a tree node by its full path
+ * @fdt: pointer to the device tree blob
+ * @path: full path of the node to locate
+ * @namelen: number of characters of path to consider
+ *
+ * Identical to fdt_path_offset(), but only consider the first namelen
+ * characters of path as the path name.
+ *
+ * Return: offset of the node or negative libfdt error value otherwise
+ */
+#ifndef SWIG /* Not available in Python */
+int fdt_path_offset_namelen(const void *fdt, const char *path, int namelen);
+#endif
+
+/**
  * fdt_path_offset - find a tree node by its full path
  * @fdt: pointer to the device tree blob
  * @path: full path of the node to locate
@@ -404,6 +510,33 @@ int fdt_first_property_offset(const void *fdt, int nodeoffset);
 int fdt_next_property_offset(const void *fdt, int offset);
 
 /**
+ * fdt_for_each_property_offset - iterate over all properties of a node
+ *
+ * @property:	property offset (int, lvalue)
+ * @fdt:	FDT blob (const void *)
+ * @node:	node offset (int)
+ *
+ * This is actually a wrapper around a for loop and would be used like so:
+ *
+ *	fdt_for_each_property_offset(property, fdt, node) {
+ *		Use property
+ *		...
+ *	}
+ *
+ *	if ((property < 0) && (property != -FDT_ERR_NOTFOUND)) {
+ *		Error handling
+ *	}
+ *
+ * Note that this is implemented as a macro and property is used as
+ * iterator in the loop. The node variable can be constant or even a
+ * literal.
+ */
+#define fdt_for_each_property_offset(property, fdt, node)	\
+	for (property = fdt_first_property_offset(fdt, node);	\
+	     property >= 0;					\
+	     property = fdt_next_property_offset(fdt, property))
+
+/**
  * fdt_get_property_by_offset - retrieve the property at a given offset
  * @fdt: pointer to the device tree blob
  * @offset: offset of the property to retrieve
@@ -532,6 +665,14 @@ const void *fdt_getprop_by_offset(const void *fdt, int offset,
 const void *fdt_getprop_namelen(const void *fdt, int nodeoffset,
 				const char *name, int namelen, int *lenp);
 
+static inline void *fdt_getprop_namelen_w(void *fdt, int nodeoffset,
+					  const char *name, int namelen,
+					  int *lenp)
+{
+	return (void *)(uintptr_t)fdt_getprop_namelen(fdt, nodeoffset, name,
+							  namelen, lenp);
+}
+
 /**
  * fdt_getprop - retrieve the value of a given property
  * @fdt: pointer to the device tree blob
@@ -993,6 +1134,31 @@ static inline int fdt_setprop_inplace_cell(void *fdt, int nodeoffset,
 int fdt_nop_property(void *fdt, int nodeoffset, const char *name);
 
 /**
+ * fdt_setprop_inplace_namelen_partial - change a property's value,
+ *                                       but not its size
+ * @fdt: pointer to the device tree blob
+ * @nodeoffset: offset of the node whose property to change
+ * @name: name of the property to change
+ * @namelen: number of characters of name to consider
+ * @idx: index of the property to change in the array
+ * @val: pointer to data to replace the property value with
+ * @len: length of the property value
+ *
+ * Identical to fdt_setprop_inplace(), but modifies the given property
+ * starting from the given index, and using only the first characters
+ * of the name. It is useful when you want to manipulate only one value of
+ * an array and you have a string that doesn't end with \0.
+ *
+ * Return: 0 on success, negative libfdt error value otherwise
+ */
+#ifndef SWIG /* Not available in Python */
+int fdt_setprop_inplace_namelen_partial(void *fdt, int nodeoffset,
+					const char *name, int namelen,
+					uint32_t idx, const void *val,
+					int len);
+#endif
+
+/**
  * fdt_nop_node - replace a node (subtree) with nop tags
  * @fdt: pointer to the device tree blob
  * @nodeoffset: offset of the node to nop
@@ -1158,6 +1324,37 @@ int fdt_setprop(void *fdt, int nodeoffset, const char *name,
 		const void *val, int len);
 
 /**
+ * fdt_setprop_placeholder - allocate space for a property
+ * @fdt: pointer to the device tree blob
+ * @nodeoffset: offset of the node whose property to change
+ * @name: name of the property to change
+ * @len: length of the property value
+ * @prop_data: return pointer to property data
+ *
+ * fdt_setprop_placeholer() allocates the named property in the given node.
+ * If the property exists it is resized. In either case a pointer to the
+ * property data is returned.
+ *
+ * This function may insert or delete data from the blob, and will
+ * therefore change the offsets of some existing nodes.
+ *
+ * returns:
+ *	0, on success
+ *	-FDT_ERR_NOSPACE, there is insufficient free space in the blob to
+ *		contain the new property value
+ *	-FDT_ERR_BADOFFSET, nodeoffset did not point to FDT_BEGIN_NODE tag
+ *	-FDT_ERR_BADLAYOUT,
+ *	-FDT_ERR_BADMAGIC,
+ *	-FDT_ERR_BADVERSION,
+ *	-FDT_ERR_BADSTATE,
+ *	-FDT_ERR_BADSTRUCTURE,
+ *	-FDT_ERR_BADLAYOUT,
+ *	-FDT_ERR_TRUNCATED, standard meanings
+ */
+int fdt_setprop_placeholder(void *fdt, int nodeoffset, const char *name,
+			    int len, void **prop_data);
+
+/**
  * fdt_setprop_u32 - set a property to a 32-bit integer
  * @fdt: pointer to the device tree blob
  * @nodeoffset: offset of the node whose property to change
@@ -1503,6 +1700,37 @@ int fdt_add_subnode(void *fdt, int parentoffset, const char *name);
  */
 int fdt_del_node(void *fdt, int nodeoffset);
 
+/**
+ * fdt_overlay_apply - Applies a DT overlay on a base DT
+ * @fdt: pointer to the base device tree blob
+ * @fdto: pointer to the device tree overlay blob
+ *
+ * fdt_overlay_apply() will apply the given device tree overlay on the
+ * given base device tree.
+ *
+ * Expect the base device tree to be modified, even if the function
+ * returns an error.
+ *
+ * returns:
+ *	0, on success
+ *	-FDT_ERR_NOSPACE, there's not enough space in the base device tree
+ *	-FDT_ERR_NOTFOUND, the overlay points to some inexistant nodes or
+ *		properties in the base DT
+ *	-FDT_ERR_BADPHANDLE,
+ *	-FDT_ERR_BADOVERLAY,
+ *	-FDT_ERR_NOPHANDLES,
+ *	-FDT_ERR_INTERNAL,
+ *	-FDT_ERR_BADLAYOUT,
+ *	-FDT_ERR_BADMAGIC,
+ *	-FDT_ERR_BADOFFSET,
+ *	-FDT_ERR_BADPATH,
+ *	-FDT_ERR_BADVERSION,
+ *	-FDT_ERR_BADSTRUCTURE,
+ *	-FDT_ERR_BADSTATE,
+ *	-FDT_ERR_TRUNCATED, standard meanings
+ */
+int fdt_overlay_apply(void *fdt, void *fdto);
+
 /**********************************************************************/
 /* Debugging / informational functions                                */
 /**********************************************************************/
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:50:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176912.322080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYq-0004la-JG; Thu, 02 Sep 2021 06:50:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176912.322080; Thu, 02 Sep 2021 06:50:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYq-0004l5-DG; Thu, 02 Sep 2021 06:50:56 +0000
Received: by outflank-mailman (input) for mailman id 176912;
 Thu, 02 Sep 2021 06:07:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsO-0004aE-Nu
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:04 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.236.63]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fd80c276-0bb3-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:02 +0000 (UTC)
Received: from DM3PR12CA0089.namprd12.prod.outlook.com (2603:10b6:0:57::33) by
 BN7PR02MB4211.namprd02.prod.outlook.com (2603:10b6:406:fc::23) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.23; Thu, 2 Sep 2021 06:06:59 +0000
Received: from DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
 (2603:10b6:0:57:cafe::58) by DM3PR12CA0089.outlook.office365.com
 (2603:10b6:0:57::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 06:06:59 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 DM3NAM02FT042.mail.protection.outlook.com (10.13.4.213) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:06:58 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:06:56 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:06:56 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsG-000F6q-6V; Wed, 01 Sep 2021 23:06:56 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd80c276-0bb3-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R6Vc4VFGnNuCCjRMR92oBl1bAZff2gHkQsaMvWJXlyY7ssCfoaZvdezkRw+D+7ISyeq/qMDSjl7kbYmvkJzBkz9SdCRIJwyPEe/NjbMCZ9veKiS1a5mJYM2wkeS/RdoA+Hapri/rlW/dwjOYfbA7Q1aNIsJi82YWAflviuIxk2K4r0XAaR/ECDp/MxCXtPNHwOphzdcG8lj60/Ir862yHzu+nj3KRU+LT8AsbDpS3dBRcV/2gE0Vw3mg3Lms/odAE2WpZBOpPpjfpFIjRY6HbXHOLoF2ry7/Nl6Eolf/EC3+iEwf1FXuoc5oqxHiO+NAtj/9lRmp2l0nRbp7Y+CH0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FBIDlGqTHEriiyE7H+BVFfcsrCknmNO0E31FG6ZulT4=;
 b=Sun1v8s7rdBMypKl7wq30b8gHJPwCCBjAd+LZIlTy5RKSVCuQ+JFQUIrpkfuO3iPgIKo06YrM7jJcPj6d+htlJ5SBc8kZ4l9awvjC5pJy7PWXsQzPDiwIJS3CPH3h7BRc7FnaIDyA/ioFmpXjRIDCL68LDNSE/xzeq1fwhyxLf1q5rLqT/8DR2U57jvH2ZBl+RkXaFPK6vxPToe/YimCwGOBmc/hQYvwFQgc+BO+/LViin+qP14BwoDINaaBOfeUOuwlDJddA/WK+bNzaD5IGBpuXUCu9+VFwADGytv7ARragyJwe0gb7GxbyyCT0aDt1geqC2C/NwMo+70THM5xPQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FBIDlGqTHEriiyE7H+BVFfcsrCknmNO0E31FG6ZulT4=;
 b=FLUVpOfTD/W41h3mRIDt6XCQPHQ/wBHdw8Bikj2+6WL57PbHWKTUgLJ0gE0kEgvb3ankE03JYrvY2bsnHXLb5TTv0dYxcGvRPbOa/v1jI/zZneHcpAM+rFK5HCWkhfPQX54SPftiqiT0jO6BbNS3PpGYAC6o71QPgXc6+kOZuMU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 03/13] libfdt: import fdt_overlay from Linux
Date: Wed, 1 Sep 2021 23:05:53 -0700
Message-ID: <1630562763-390068-4-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 89b30686-b128-471c-dc4a-08d96dd7df9f
X-MS-TrafficTypeDiagnostic: BN7PR02MB4211:
X-Microsoft-Antispam-PRVS:
	<BN7PR02MB4211B8938F72DEDCCA422090BCCE9@BN7PR02MB4211.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:130;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yjqvN7lSYivxzQcNipLCYLIXc4ae1BOoYUmYeHnmYIX8DTF6TSc1F/EjcJYyBj0RClsoXswFmP1pYPOntxMIa/Hf38w7L2LRjfd2YgMKITmooPwKbBANrmeAuC0fkkRDfcfVwwbQvDcXxUTsj965VFj/Q2uqV81ebBusq/wlnH9iL0WjRpyGzDBhwriBTlENWG0K5/o5C3p/q6OKvCSn/euwvLHrcuTeO8WN8zruf2HbHEUqAEJvVmerEGAJ5Qv0z/TbdVRWZk/fZ9aybpC7SoXTUeZo4+VpqC79c14An8H1XdVyphz3f0uhjdBezhfsuQ+nyqd1YiftQQfhtaf3N9BIWJVoZfnwid0IshabO6fhIhhPDdr5dcnF8rr7DB13AJz3fasHb3l2+msrHuU1piCpLfe+Z3uEhCK0TfEmAuucqXUDPet+OUf19iAlx2lgDlF4oAtThVTbj4k8V/yDUOUmxsA88S+WMwxTqKJEyJcJDF4x/6SRG/gdHPNIr/PTmSDoeubHZw2q6jbIz4Qo/ADLGEFxyMyJVgMw6cv42042qrUvMuMNaKZhVYR3ElLt/FPUX7lhiiFYMXkjuOjfpGtp/6/GvTIpbNCBo7zfeDx98cB2lLvbSDu9z57yFzfWocbGA5/sP2Pjxw+lradx5EOpTiXWEZabxIYujNTRt2mKwALw27u1neemjh8bKWA9E62myncgQcz75B9GOfuoc1mpn+DgoKdKIZrBYp6LA84=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(83380400001)(2906002)(426003)(186003)(82310400003)(54906003)(508600001)(7636003)(6916009)(7696005)(30864003)(70206006)(5660300002)(316002)(47076005)(36906005)(36860700001)(9786002)(8676002)(2616005)(70586007)(336012)(4326008)(8936002)(356005)(6666004)(26005)(36756003)(107886003)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:06:58.5918
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 89b30686-b128-471c-dc4a-08d96dd7df9f
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR02MB4211

Xen is missing fdt overlay functionalities. FDT overlay is required for changing
the device tree nodes during run-time.

fdt_overlay.c file is copied from Linux tree's following patch:
commit: 6e9c9686d826564f44c93cdd6f111b1c0a9dc224
scripts/dtc: Update to upstream version v1.6.0-31-gcbca977ea121

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/libfdt/fdt_overlay.c | 882 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 882 insertions(+)
 create mode 100644 xen/common/libfdt/fdt_overlay.c

diff --git a/xen/common/libfdt/fdt_overlay.c b/xen/common/libfdt/fdt_overlay.c
new file mode 100644
index 0000000..d217e79
--- /dev/null
+++ b/xen/common/libfdt/fdt_overlay.c
@@ -0,0 +1,882 @@
+// SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause)
+/*
+ * libfdt - Flat Device Tree manipulation
+ * Copyright (C) 2016 Free Electrons
+ * Copyright (C) 2016 NextThing Co.
+ */
+#include "libfdt_env.h"
+
+#include <fdt.h>
+#include <libfdt.h>
+
+#include "libfdt_internal.h"
+
+/**
+ * overlay_get_target_phandle - retrieves the target phandle of a fragment
+ * @fdto: pointer to the device tree overlay blob
+ * @fragment: node offset of the fragment in the overlay
+ *
+ * overlay_get_target_phandle() retrieves the target phandle of an
+ * overlay fragment when that fragment uses a phandle (target
+ * property) instead of a path (target-path property).
+ *
+ * returns:
+ *      the phandle pointed by the target property
+ *      0, if the phandle was not found
+ *	-1, if the phandle was malformed
+ */
+static uint32_t overlay_get_target_phandle(const void *fdto, int fragment)
+{
+	const fdt32_t *val;
+	int len;
+
+	val = fdt_getprop(fdto, fragment, "target", &len);
+	if (!val)
+		return 0;
+
+	if ((len != sizeof(*val)) || (fdt32_to_cpu(*val) == (uint32_t)-1))
+		return (uint32_t)-1;
+
+	return fdt32_to_cpu(*val);
+}
+
+/**
+ * overlay_get_target - retrieves the offset of a fragment's target
+ * @fdt: Base device tree blob
+ * @fdto: Device tree overlay blob
+ * @fragment: node offset of the fragment in the overlay
+ * @pathp: pointer which receives the path of the target (or NULL)
+ *
+ * overlay_get_target() retrieves the target offset in the base
+ * device tree of a fragment, no matter how the actual targeting is
+ * done (through a phandle or a path)
+ *
+ * returns:
+ *      the targeted node offset in the base device tree
+ *      Negative error code on error
+ */
+static int overlay_get_target(const void *fdt, const void *fdto,
+			      int fragment, char const **pathp)
+{
+	uint32_t phandle;
+	const char *path = NULL;
+	int path_len = 0, ret;
+
+	/* Try first to do a phandle based lookup */
+	phandle = overlay_get_target_phandle(fdto, fragment);
+	if (phandle == (uint32_t)-1)
+		return -FDT_ERR_BADPHANDLE;
+
+	/* no phandle, try path */
+	if (!phandle) {
+		/* And then a path based lookup */
+		path = fdt_getprop(fdto, fragment, "target-path", &path_len);
+		if (path)
+			ret = fdt_path_offset(fdt, path);
+		else
+			ret = path_len;
+	} else
+		ret = fdt_node_offset_by_phandle(fdt, phandle);
+
+	/*
+	* If we haven't found either a target or a
+	* target-path property in a node that contains a
+	* __overlay__ subnode (we wouldn't be called
+	* otherwise), consider it a improperly written
+	* overlay
+	*/
+	if (ret < 0 && path_len == -FDT_ERR_NOTFOUND)
+		ret = -FDT_ERR_BADOVERLAY;
+
+	/* return on error */
+	if (ret < 0)
+		return ret;
+
+	/* return pointer to path (if available) */
+	if (pathp)
+		*pathp = path ? path : NULL;
+
+	return ret;
+}
+
+/**
+ * overlay_phandle_add_offset - Increases a phandle by an offset
+ * @fdt: Base device tree blob
+ * @node: Device tree overlay blob
+ * @name: Name of the property to modify (phandle or linux,phandle)
+ * @delta: offset to apply
+ *
+ * overlay_phandle_add_offset() increments a node phandle by a given
+ * offset.
+ *
+ * returns:
+ *      0 on success.
+ *      Negative error code on error
+ */
+static int overlay_phandle_add_offset(void *fdt, int node,
+				      const char *name, uint32_t delta)
+{
+	const fdt32_t *val;
+	uint32_t adj_val;
+	int len;
+
+	val = fdt_getprop(fdt, node, name, &len);
+	if (!val)
+		return len;
+
+	if (len != sizeof(*val))
+		return -FDT_ERR_BADPHANDLE;
+
+	adj_val = fdt32_to_cpu(*val);
+	if ((adj_val + delta) < adj_val)
+		return -FDT_ERR_NOPHANDLES;
+
+	adj_val += delta;
+	if (adj_val == (uint32_t)-1)
+		return -FDT_ERR_NOPHANDLES;
+
+	return fdt_setprop_inplace_u32(fdt, node, name, adj_val);
+}
+
+/**
+ * overlay_adjust_node_phandles - Offsets the phandles of a node
+ * @fdto: Device tree overlay blob
+ * @node: Offset of the node we want to adjust
+ * @delta: Offset to shift the phandles of
+ *
+ * overlay_adjust_node_phandles() adds a constant to all the phandles
+ * of a given node. This is mainly use as part of the overlay
+ * application process, when we want to update all the overlay
+ * phandles to not conflict with the overlays of the base device tree.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_adjust_node_phandles(void *fdto, int node,
+					uint32_t delta)
+{
+	int child;
+	int ret;
+
+	ret = overlay_phandle_add_offset(fdto, node, "phandle", delta);
+	if (ret && ret != -FDT_ERR_NOTFOUND)
+		return ret;
+
+	ret = overlay_phandle_add_offset(fdto, node, "linux,phandle", delta);
+	if (ret && ret != -FDT_ERR_NOTFOUND)
+		return ret;
+
+	fdt_for_each_subnode(child, fdto, node) {
+		ret = overlay_adjust_node_phandles(fdto, child, delta);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+/**
+ * overlay_adjust_local_phandles - Adjust the phandles of a whole overlay
+ * @fdto: Device tree overlay blob
+ * @delta: Offset to shift the phandles of
+ *
+ * overlay_adjust_local_phandles() adds a constant to all the
+ * phandles of an overlay. This is mainly use as part of the overlay
+ * application process, when we want to update all the overlay
+ * phandles to not conflict with the overlays of the base device tree.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_adjust_local_phandles(void *fdto, uint32_t delta)
+{
+	/*
+	 * Start adjusting the phandles from the overlay root
+	 */
+	return overlay_adjust_node_phandles(fdto, 0, delta);
+}
+
+/**
+ * overlay_update_local_node_references - Adjust the overlay references
+ * @fdto: Device tree overlay blob
+ * @tree_node: Node offset of the node to operate on
+ * @fixup_node: Node offset of the matching local fixups node
+ * @delta: Offset to shift the phandles of
+ *
+ * overlay_update_local_nodes_references() update the phandles
+ * pointing to a node within the device tree overlay by adding a
+ * constant delta.
+ *
+ * This is mainly used as part of a device tree application process,
+ * where you want the device tree overlays phandles to not conflict
+ * with the ones from the base device tree before merging them.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_update_local_node_references(void *fdto,
+						int tree_node,
+						int fixup_node,
+						uint32_t delta)
+{
+	int fixup_prop;
+	int fixup_child;
+	int ret;
+
+	fdt_for_each_property_offset(fixup_prop, fdto, fixup_node) {
+		const fdt32_t *fixup_val;
+		const char *tree_val;
+		const char *name;
+		int fixup_len;
+		int tree_len;
+		int i;
+
+		fixup_val = fdt_getprop_by_offset(fdto, fixup_prop,
+						  &name, &fixup_len);
+		if (!fixup_val)
+			return fixup_len;
+
+		if (fixup_len % sizeof(uint32_t))
+			return -FDT_ERR_BADOVERLAY;
+		fixup_len /= sizeof(uint32_t);
+
+		tree_val = fdt_getprop(fdto, tree_node, name, &tree_len);
+		if (!tree_val) {
+			if (tree_len == -FDT_ERR_NOTFOUND)
+				return -FDT_ERR_BADOVERLAY;
+
+			return tree_len;
+		}
+
+		for (i = 0; i < fixup_len; i++) {
+			fdt32_t adj_val;
+			uint32_t poffset;
+
+			poffset = fdt32_to_cpu(fixup_val[i]);
+
+			/*
+			 * phandles to fixup can be unaligned.
+			 *
+			 * Use a memcpy for the architectures that do
+			 * not support unaligned accesses.
+			 */
+			memcpy(&adj_val, tree_val + poffset, sizeof(adj_val));
+
+			adj_val = cpu_to_fdt32(fdt32_to_cpu(adj_val) + delta);
+
+			ret = fdt_setprop_inplace_namelen_partial(fdto,
+								  tree_node,
+								  name,
+								  strlen(name),
+								  poffset,
+								  &adj_val,
+								  sizeof(adj_val));
+			if (ret == -FDT_ERR_NOSPACE)
+				return -FDT_ERR_BADOVERLAY;
+
+			if (ret)
+				return ret;
+		}
+	}
+
+	fdt_for_each_subnode(fixup_child, fdto, fixup_node) {
+		const char *fixup_child_name = fdt_get_name(fdto, fixup_child,
+							    NULL);
+		int tree_child;
+
+		tree_child = fdt_subnode_offset(fdto, tree_node,
+						fixup_child_name);
+		if (tree_child == -FDT_ERR_NOTFOUND)
+			return -FDT_ERR_BADOVERLAY;
+		if (tree_child < 0)
+			return tree_child;
+
+		ret = overlay_update_local_node_references(fdto,
+							   tree_child,
+							   fixup_child,
+							   delta);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+/**
+ * overlay_update_local_references - Adjust the overlay references
+ * @fdto: Device tree overlay blob
+ * @delta: Offset to shift the phandles of
+ *
+ * overlay_update_local_references() update all the phandles pointing
+ * to a node within the device tree overlay by adding a constant
+ * delta to not conflict with the base overlay.
+ *
+ * This is mainly used as part of a device tree application process,
+ * where you want the device tree overlays phandles to not conflict
+ * with the ones from the base device tree before merging them.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_update_local_references(void *fdto, uint32_t delta)
+{
+	int fixups;
+
+	fixups = fdt_path_offset(fdto, "/__local_fixups__");
+	if (fixups < 0) {
+		/* There's no local phandles to adjust, bail out */
+		if (fixups == -FDT_ERR_NOTFOUND)
+			return 0;
+
+		return fixups;
+	}
+
+	/*
+	 * Update our local references from the root of the tree
+	 */
+	return overlay_update_local_node_references(fdto, 0, fixups,
+						    delta);
+}
+
+/**
+ * overlay_fixup_one_phandle - Set an overlay phandle to the base one
+ * @fdt: Base Device Tree blob
+ * @fdto: Device tree overlay blob
+ * @symbols_off: Node offset of the symbols node in the base device tree
+ * @path: Path to a node holding a phandle in the overlay
+ * @path_len: number of path characters to consider
+ * @name: Name of the property holding the phandle reference in the overlay
+ * @name_len: number of name characters to consider
+ * @poffset: Offset within the overlay property where the phandle is stored
+ * @label: Label of the node referenced by the phandle
+ *
+ * overlay_fixup_one_phandle() resolves an overlay phandle pointing to
+ * a node in the base device tree.
+ *
+ * This is part of the device tree overlay application process, when
+ * you want all the phandles in the overlay to point to the actual
+ * base dt nodes.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_fixup_one_phandle(void *fdt, void *fdto,
+				     int symbols_off,
+				     const char *path, uint32_t path_len,
+				     const char *name, uint32_t name_len,
+				     int poffset, const char *label)
+{
+	const char *symbol_path;
+	uint32_t phandle;
+	fdt32_t phandle_prop;
+	int symbol_off, fixup_off;
+	int prop_len;
+
+	if (symbols_off < 0)
+		return symbols_off;
+
+	symbol_path = fdt_getprop(fdt, symbols_off, label,
+				  &prop_len);
+	if (!symbol_path)
+		return prop_len;
+
+	symbol_off = fdt_path_offset(fdt, symbol_path);
+	if (symbol_off < 0)
+		return symbol_off;
+
+	phandle = fdt_get_phandle(fdt, symbol_off);
+	if (!phandle)
+		return -FDT_ERR_NOTFOUND;
+
+	fixup_off = fdt_path_offset_namelen(fdto, path, path_len);
+	if (fixup_off == -FDT_ERR_NOTFOUND)
+		return -FDT_ERR_BADOVERLAY;
+	if (fixup_off < 0)
+		return fixup_off;
+
+	phandle_prop = cpu_to_fdt32(phandle);
+	return fdt_setprop_inplace_namelen_partial(fdto, fixup_off,
+						   name, name_len, poffset,
+						   &phandle_prop,
+						   sizeof(phandle_prop));
+};
+
+/**
+ * overlay_fixup_phandle - Set an overlay phandle to the base one
+ * @fdt: Base Device Tree blob
+ * @fdto: Device tree overlay blob
+ * @symbols_off: Node offset of the symbols node in the base device tree
+ * @property: Property offset in the overlay holding the list of fixups
+ *
+ * overlay_fixup_phandle() resolves all the overlay phandles pointed
+ * to in a __fixups__ property, and updates them to match the phandles
+ * in use in the base device tree.
+ *
+ * This is part of the device tree overlay application process, when
+ * you want all the phandles in the overlay to point to the actual
+ * base dt nodes.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_fixup_phandle(void *fdt, void *fdto, int symbols_off,
+				 int property)
+{
+	const char *value;
+	const char *label;
+	int len;
+
+	value = fdt_getprop_by_offset(fdto, property,
+				      &label, &len);
+	if (!value) {
+		if (len == -FDT_ERR_NOTFOUND)
+			return -FDT_ERR_INTERNAL;
+
+		return len;
+	}
+
+	do {
+		const char *path, *name, *fixup_end;
+		const char *fixup_str = value;
+		uint32_t path_len, name_len;
+		uint32_t fixup_len;
+		char *sep, *endptr;
+		int poffset, ret;
+
+		fixup_end = memchr(value, '\0', len);
+		if (!fixup_end)
+			return -FDT_ERR_BADOVERLAY;
+		fixup_len = fixup_end - fixup_str;
+
+		len -= fixup_len + 1;
+		value += fixup_len + 1;
+
+		path = fixup_str;
+		sep = memchr(fixup_str, ':', fixup_len);
+		if (!sep || *sep != ':')
+			return -FDT_ERR_BADOVERLAY;
+
+		path_len = sep - path;
+		if (path_len == (fixup_len - 1))
+			return -FDT_ERR_BADOVERLAY;
+
+		fixup_len -= path_len + 1;
+		name = sep + 1;
+		sep = memchr(name, ':', fixup_len);
+		if (!sep || *sep != ':')
+			return -FDT_ERR_BADOVERLAY;
+
+		name_len = sep - name;
+		if (!name_len)
+			return -FDT_ERR_BADOVERLAY;
+
+		poffset = strtoul(sep + 1, &endptr, 10);
+		if ((*endptr != '\0') || (endptr <= (sep + 1)))
+			return -FDT_ERR_BADOVERLAY;
+
+		ret = overlay_fixup_one_phandle(fdt, fdto, symbols_off,
+						path, path_len, name, name_len,
+						poffset, label);
+		if (ret)
+			return ret;
+	} while (len > 0);
+
+	return 0;
+}
+
+/**
+ * overlay_fixup_phandles - Resolve the overlay phandles to the base
+ *                          device tree
+ * @fdt: Base Device Tree blob
+ * @fdto: Device tree overlay blob
+ *
+ * overlay_fixup_phandles() resolves all the overlay phandles pointing
+ * to nodes in the base device tree.
+ *
+ * This is one of the steps of the device tree overlay application
+ * process, when you want all the phandles in the overlay to point to
+ * the actual base dt nodes.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_fixup_phandles(void *fdt, void *fdto)
+{
+	int fixups_off, symbols_off;
+	int property;
+
+	/* We can have overlays without any fixups */
+	fixups_off = fdt_path_offset(fdto, "/__fixups__");
+	if (fixups_off == -FDT_ERR_NOTFOUND)
+		return 0; /* nothing to do */
+	if (fixups_off < 0)
+		return fixups_off;
+
+	/* And base DTs without symbols */
+	symbols_off = fdt_path_offset(fdt, "/__symbols__");
+	if ((symbols_off < 0 && (symbols_off != -FDT_ERR_NOTFOUND)))
+		return symbols_off;
+
+	fdt_for_each_property_offset(property, fdto, fixups_off) {
+		int ret;
+
+		ret = overlay_fixup_phandle(fdt, fdto, symbols_off, property);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+/**
+ * overlay_apply_node - Merges a node into the base device tree
+ * @fdt: Base Device Tree blob
+ * @target: Node offset in the base device tree to apply the fragment to
+ * @fdto: Device tree overlay blob
+ * @node: Node offset in the overlay holding the changes to merge
+ *
+ * overlay_apply_node() merges a node into a target base device tree
+ * node pointed.
+ *
+ * This is part of the final step in the device tree overlay
+ * application process, when all the phandles have been adjusted and
+ * resolved and you just have to merge overlay into the base device
+ * tree.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_apply_node(void *fdt, int target,
+			      void *fdto, int node)
+{
+	int property;
+	int subnode;
+
+	fdt_for_each_property_offset(property, fdto, node) {
+		const char *name;
+		const void *prop;
+		int prop_len;
+		int ret;
+
+		prop = fdt_getprop_by_offset(fdto, property, &name,
+					     &prop_len);
+		if (prop_len == -FDT_ERR_NOTFOUND)
+			return -FDT_ERR_INTERNAL;
+		if (prop_len < 0)
+			return prop_len;
+
+		ret = fdt_setprop(fdt, target, name, prop, prop_len);
+		if (ret)
+			return ret;
+	}
+
+	fdt_for_each_subnode(subnode, fdto, node) {
+		const char *name = fdt_get_name(fdto, subnode, NULL);
+		int nnode;
+		int ret;
+
+		nnode = fdt_add_subnode(fdt, target, name);
+		if (nnode == -FDT_ERR_EXISTS) {
+			nnode = fdt_subnode_offset(fdt, target, name);
+			if (nnode == -FDT_ERR_NOTFOUND)
+				return -FDT_ERR_INTERNAL;
+		}
+
+		if (nnode < 0)
+			return nnode;
+
+		ret = overlay_apply_node(fdt, nnode, fdto, subnode);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+/**
+ * overlay_merge - Merge an overlay into its base device tree
+ * @fdt: Base Device Tree blob
+ * @fdto: Device tree overlay blob
+ *
+ * overlay_merge() merges an overlay into its base device tree.
+ *
+ * This is the next to last step in the device tree overlay application
+ * process, when all the phandles have been adjusted and resolved and
+ * you just have to merge overlay into the base device tree.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_merge(void *fdt, void *fdto)
+{
+	int fragment;
+
+	fdt_for_each_subnode(fragment, fdto, 0) {
+		int overlay;
+		int target;
+		int ret;
+
+		/*
+		 * Each fragments will have an __overlay__ node. If
+		 * they don't, it's not supposed to be merged
+		 */
+		overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+		if (overlay == -FDT_ERR_NOTFOUND)
+			continue;
+
+		if (overlay < 0)
+			return overlay;
+
+		target = overlay_get_target(fdt, fdto, fragment, NULL);
+		if (target < 0)
+			return target;
+
+		ret = overlay_apply_node(fdt, target, fdto, overlay);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int get_path_len(const void *fdt, int nodeoffset)
+{
+	int len = 0, namelen;
+	const char *name;
+
+	FDT_RO_PROBE(fdt);
+
+	for (;;) {
+		name = fdt_get_name(fdt, nodeoffset, &namelen);
+		if (!name)
+			return namelen;
+
+		/* root? we're done */
+		if (namelen == 0)
+			break;
+
+		nodeoffset = fdt_parent_offset(fdt, nodeoffset);
+		if (nodeoffset < 0)
+			return nodeoffset;
+		len += namelen + 1;
+	}
+
+	/* in case of root pretend it's "/" */
+	if (len == 0)
+		len++;
+	return len;
+}
+
+/**
+ * overlay_symbol_update - Update the symbols of base tree after a merge
+ * @fdt: Base Device Tree blob
+ * @fdto: Device tree overlay blob
+ *
+ * overlay_symbol_update() updates the symbols of the base tree with the
+ * symbols of the applied overlay
+ *
+ * This is the last step in the device tree overlay application
+ * process, allowing the reference of overlay symbols by subsequent
+ * overlay operations.
+ *
+ * returns:
+ *      0 on success
+ *      Negative error code on failure
+ */
+static int overlay_symbol_update(void *fdt, void *fdto)
+{
+	int root_sym, ov_sym, prop, path_len, fragment, target;
+	int len, frag_name_len, ret, rel_path_len;
+	const char *s, *e;
+	const char *path;
+	const char *name;
+	const char *frag_name;
+	const char *rel_path;
+	const char *target_path;
+	char *buf;
+	void *p;
+
+	ov_sym = fdt_subnode_offset(fdto, 0, "__symbols__");
+
+	/* if no overlay symbols exist no problem */
+	if (ov_sym < 0)
+		return 0;
+
+	root_sym = fdt_subnode_offset(fdt, 0, "__symbols__");
+
+	/* it no root symbols exist we should create them */
+	if (root_sym == -FDT_ERR_NOTFOUND)
+		root_sym = fdt_add_subnode(fdt, 0, "__symbols__");
+
+	/* any error is fatal now */
+	if (root_sym < 0)
+		return root_sym;
+
+	/* iterate over each overlay symbol */
+	fdt_for_each_property_offset(prop, fdto, ov_sym) {
+		path = fdt_getprop_by_offset(fdto, prop, &name, &path_len);
+		if (!path)
+			return path_len;
+
+		/* verify it's a string property (terminated by a single \0) */
+		if (path_len < 1 || memchr(path, '\0', path_len) != &path[path_len - 1])
+			return -FDT_ERR_BADVALUE;
+
+		/* keep end marker to avoid strlen() */
+		e = path + path_len;
+
+		if (*path != '/')
+			return -FDT_ERR_BADVALUE;
+
+		/* get fragment name first */
+		s = strchr(path + 1, '/');
+		if (!s) {
+			/* Symbol refers to something that won't end
+			 * up in the target tree */
+			continue;
+		}
+
+		frag_name = path + 1;
+		frag_name_len = s - path - 1;
+
+		/* verify format; safe since "s" lies in \0 terminated prop */
+		len = sizeof("/__overlay__/") - 1;
+		if ((e - s) > len && (memcmp(s, "/__overlay__/", len) == 0)) {
+			/* /<fragment-name>/__overlay__/<relative-subnode-path> */
+			rel_path = s + len;
+			rel_path_len = e - rel_path - 1;
+		} else if ((e - s) == len
+			   && (memcmp(s, "/__overlay__", len - 1) == 0)) {
+			/* /<fragment-name>/__overlay__ */
+			rel_path = "";
+			rel_path_len = 0;
+		} else {
+			/* Symbol refers to something that won't end
+			 * up in the target tree */
+			continue;
+		}
+
+		/* find the fragment index in which the symbol lies */
+		ret = fdt_subnode_offset_namelen(fdto, 0, frag_name,
+					       frag_name_len);
+		/* not found? */
+		if (ret < 0)
+			return -FDT_ERR_BADOVERLAY;
+		fragment = ret;
+
+		/* an __overlay__ subnode must exist */
+		ret = fdt_subnode_offset(fdto, fragment, "__overlay__");
+		if (ret < 0)
+			return -FDT_ERR_BADOVERLAY;
+
+		/* get the target of the fragment */
+		ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+		if (ret < 0)
+			return ret;
+		target = ret;
+
+		/* if we have a target path use */
+		if (!target_path) {
+			ret = get_path_len(fdt, target);
+			if (ret < 0)
+				return ret;
+			len = ret;
+		} else {
+			len = strlen(target_path);
+		}
+
+		ret = fdt_setprop_placeholder(fdt, root_sym, name,
+				len + (len > 1) + rel_path_len + 1, &p);
+		if (ret < 0)
+			return ret;
+
+		if (!target_path) {
+			/* again in case setprop_placeholder changed it */
+			ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+			if (ret < 0)
+				return ret;
+			target = ret;
+		}
+
+		buf = p;
+		if (len > 1) { /* target is not root */
+			if (!target_path) {
+				ret = fdt_get_path(fdt, target, buf, len + 1);
+				if (ret < 0)
+					return ret;
+			} else
+				memcpy(buf, target_path, len + 1);
+
+		} else
+			len--;
+
+		buf[len] = '/';
+		memcpy(buf + len + 1, rel_path, rel_path_len);
+		buf[len + 1 + rel_path_len] = '\0';
+	}
+
+	return 0;
+}
+
+int fdt_overlay_apply(void *fdt, void *fdto)
+{
+	uint32_t delta;
+	int ret;
+
+	FDT_RO_PROBE(fdt);
+	FDT_RO_PROBE(fdto);
+
+	ret = fdt_find_max_phandle(fdt, &delta);
+	if (ret)
+		goto err;
+
+	ret = overlay_adjust_local_phandles(fdto, delta);
+	if (ret)
+		goto err;
+
+	ret = overlay_update_local_references(fdto, delta);
+	if (ret)
+		goto err;
+
+	ret = overlay_fixup_phandles(fdt, fdto);
+	if (ret)
+		goto err;
+
+	ret = overlay_merge(fdt, fdto);
+	if (ret)
+		goto err;
+
+	ret = overlay_symbol_update(fdt, fdto);
+	if (ret)
+		goto err;
+
+	/*
+	 * The overlay has been damaged, erase its magic.
+	 */
+	fdt_set_magic(fdto, ~0);
+
+	return 0;
+
+err:
+	/*
+	 * The overlay might have been damaged, erase its magic.
+	 */
+	fdt_set_magic(fdto, ~0);
+
+	/*
+	 * The base device tree might have been damaged, erase its
+	 * magic.
+	 */
+	fdt_set_magic(fdt, ~0);
+
+	return ret;
+}
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:50:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176917.322095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYr-00051E-Bb; Thu, 02 Sep 2021 06:50:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176917.322095; Thu, 02 Sep 2021 06:50:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYr-0004z1-2e; Thu, 02 Sep 2021 06:50:57 +0000
Received: by outflank-mailman (input) for mailman id 176917;
 Thu, 02 Sep 2021 06:07:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsU-0004aK-4w
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:10 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (unknown
 [40.107.223.42]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6b740e9e-d45e-48aa-a9e2-50634644e3d4;
 Thu, 02 Sep 2021 06:07:07 +0000 (UTC)
Received: from DM3PR12CA0077.namprd12.prod.outlook.com (2603:10b6:0:57::21) by
 SN1PR02MB3678.namprd02.prod.outlook.com (2603:10b6:802:24::15) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17; Thu, 2 Sep 2021 06:07:05 +0000
Received: from DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
 (2603:10b6:0:57:cafe::6f) by DM3PR12CA0077.outlook.office365.com
 (2603:10b6:0:57::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:05 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 DM3NAM02FT042.mail.protection.outlook.com (10.13.4.213) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:04 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:03 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:03 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsN-000F6q-2G; Wed, 01 Sep 2021 23:07:03 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b740e9e-d45e-48aa-a9e2-50634644e3d4
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rl4T92K7Fut2IIfUEKMpAEcvU69Hlgw4vqQXXLOgTc3QCZhwAtKznHM+TT77SjKpL+8iKTZV2if0Bq/ogNoqrBwsnX7T+oOrGxG1rbCro0PrhpjawwTpzdeBpYxyF8pFoSA8bhG6RNxLmNgcHcJqhwUOHtpXYTE5d7BEcp/IB/yk86gbUorlTmdYZOlojDET28UlZT690kpdDD3VfYUij/LnPcb8jzDYxm0DRKGaLQ+7rAt4fbjbDLdxSer+q2udMdkxk2XMxpKZ3MBC96zMq02TYTMLr+lB8uCgdSpxM/OMIzc6Gr/KGn2/sYQdAt/r7Cx334uckbIt9vi7ng5kpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=98AtMQVzaAHOwKKFvylID8GyFmwvLo9S59QnYCd+rzM=;
 b=G5j+giD11VwxA/+WjkeuVgz1IKhyT5QrsNZcBBg+HfzHOPGvK/TD4n7zzgbwieSfiw4kuHNN9SSUShmjvL3fFvF/XTkyFEh77acor6VGTtm+4yrkk9YtndmWIsmziVD+9N0h2HJBuY+blFa+BaC+pRWxKsXIqLopX95qgKk18I30pa0cVtKvhdm9TJRtFqr0KO3pWnGFshhN+8GaNrCrygYYZ3vzjyJVHUu/+i3ToFA5NZCr2QPw75PVVdPZpmBrGCZbFdh3Cxr6vb7zaM4pFOlh67bWMwgl9hjzGVGvELRG7XKcoh2R3/F0r7t6A26MW0VFDRWh6yIborcIqxESXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=98AtMQVzaAHOwKKFvylID8GyFmwvLo9S59QnYCd+rzM=;
 b=fEWmNfRVw1Iw8SEg1ZJOVObVMv/9iTTh6owNYcdnonI8ZYPCJGVNVTb8SUJJmGVD8VH+egahPgwdubShCsrwZwyHUKeYy87EnMpegIxRJW4PJxce9p3oQh06ta2cXK5iJUM+w4SRa5AFroCrtqfEIv2zBOn67BqTh1F8aoAkQkE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 05/13] libfdt: Change overlay_get_target() type
Date: Wed, 1 Sep 2021 23:05:55 -0700
Message-ID: <1630562763-390068-6-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0f9a09cc-64e5-4749-23cf-08d96dd7e370
X-MS-TrafficTypeDiagnostic: SN1PR02MB3678:
X-Microsoft-Antispam-PRVS:
	<SN1PR02MB3678E7AD48A1587026BC4B29BCCE9@SN1PR02MB3678.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ckx58VpUs6M3WyHg9+f3TqPbipbP2X8Vt1GQIVqSNAERsgzKnshwp3cxb9Nns37n+s5ns0VbNIB544kPvP5jy0tLHvh7SMNMggeLtvftBKBZBqEEL5SJ1yi/CKgXDuQjuzekuB5+fr+LKGPVnlNhcBmRU+bl7V69dfXqFUkzjXp6ic0Znws3VNTV3NYqcXatPYo4s472WDu/hQXjofqak6d22e/tgeFGX4el0eIRH9Z8+pADwfxVXpkrXQpCQfWN47nPcFQrIKTN+ItqgIHv7VSpvr7cNvns4K7tPEXVAVZbIlvLO+6QEv+4qsSZdQ0IWrW53dOZuxPBTNGOulf/EoITBmfrO7n7nC42oqCU8HCFwCiqEjI8r25ATcrhSvAG69J6Ro9sOfLlueIHT+8BoFWHaHR4yjbC+F+8DzwQefiGtc1bhE9a9X0ysfRYHDxzY2qNzyUyf+PBvut6oMDmfQZndm5O5UaijpHnCzB0lXwW0fMJpUCQzedaNBHmHypWBwhYibwKLQbVbfdYS93wbc+iNOfWJeTmX8533vx+SRAz1bLSlbLCMiQmYjyZCAydXqcrK+ckVgopGap6iDzd7kwuBW3Mlu8RWa1cnsXeYmBXsiJKDdl+/gdKBYoqT9cHy3NYUUAUBsZfSJgcTCFg5JCtdgKL2f1PBl6vrsJmSnrvH9/Yrhwjxke23jXA6uUY/evD5iFC2yvEpoT/aHEF4c1sSn7IAYVKKzyvtNZsb0A=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(376002)(396003)(346002)(136003)(39860400002)(46966006)(36840700001)(82310400003)(47076005)(36756003)(70586007)(2616005)(336012)(6666004)(426003)(186003)(26005)(478600001)(2906002)(83380400001)(8936002)(316002)(70206006)(9786002)(8676002)(7696005)(5660300002)(36906005)(82740400003)(7636003)(356005)(54906003)(36860700001)(4326008)(107886003)(6916009)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:04.9759
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f9a09cc-64e5-4749-23cf-08d96dd7e370
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR02MB3678

Remove static function type from overlay_get_target().

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/libfdt/fdt_overlay.c | 2 +-
 xen/include/xen/libfdt/libfdt.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/common/libfdt/fdt_overlay.c b/xen/common/libfdt/fdt_overlay.c
index 15a8cdb..1ffb669 100644
--- a/xen/common/libfdt/fdt_overlay.c
+++ b/xen/common/libfdt/fdt_overlay.c
@@ -56,7 +56,7 @@ static uint32_t overlay_get_target_phandle(const void *fdto, int fragment)
  *      the targeted node offset in the base device tree
  *      Negative error code on error
  */
-static int overlay_get_target(const void *fdt, const void *fdto,
+int overlay_get_target(const void *fdt, const void *fdto,
 			      int fragment, char const **pathp)
 {
 	uint32_t phandle;
diff --git a/xen/include/xen/libfdt/libfdt.h b/xen/include/xen/libfdt/libfdt.h
index b6c8b67..e1cc6f2 100644
--- a/xen/include/xen/libfdt/libfdt.h
+++ b/xen/include/xen/libfdt/libfdt.h
@@ -1737,4 +1737,6 @@ int fdt_overlay_apply(void *fdt, void *fdto);
 
 const char *fdt_strerror(int errval);
 
+int overlay_get_target(const void *fdt, const void *fdto, int fragment,
+                       char const **pathp);
 #endif /* _LIBFDT_H */
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:50:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176924.322120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYt-0005dZ-AL; Thu, 02 Sep 2021 06:50:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176924.322120; Thu, 02 Sep 2021 06:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYs-0005aj-Rl; Thu, 02 Sep 2021 06:50:58 +0000
Received: by outflank-mailman (input) for mailman id 176924;
 Thu, 02 Sep 2021 06:07:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsu-0004hL-Fg
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:36 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (unknown
 [40.107.220.65]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 10ba58f3-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:35 +0000 (UTC)
Received: from BN8PR12CA0025.namprd12.prod.outlook.com (2603:10b6:408:60::38)
 by MWHPR02MB3245.namprd02.prod.outlook.com (2603:10b6:301:6b::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Thu, 2 Sep
 2021 06:07:31 +0000
Received: from BN1NAM02FT008.eop-nam02.prod.protection.outlook.com
 (2603:10b6:408:60:cafe::92) by BN8PR12CA0025.outlook.office365.com
 (2603:10b6:408:60::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:31 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 BN1NAM02FT008.mail.protection.outlook.com (10.13.2.126) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:30 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:08 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:08 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsS-000F6q-2P; Wed, 01 Sep 2021 23:07:08 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10ba58f3-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SYzw5n/WzHl182B+tAIvwfDyBmQvwX19sQvk72oOM9gOO9pzTW5DcD4zRUZysYkRyMrGBsT0kUqeVsB3T0HQmB/lC8Qnb/7fQNEaeGSQp/Vu5fow0E46fi/E/RWVcs0Us3ArSspm0y5n3+upKgeo4of4OUsGXkJi3tuUfLlaQGwK664iUDUMrdw/71qdgOqYXufRwXESfV2ez7gwnABVT6OmXmucUdzOTYBh54RwWW7lOv2yRXdDhYYz6sA60QIoFna9SS5JmplJ/25wLpAVyAUDsvMNpj0s9B/zH1DlUFpdJyBBV05H1+ipjYqSoz2S9yw6xGkk93Ovg+4P/Ky87Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=02lvqIFeq5ueMXlcG0MgTjXHnFIspvYKtsnJlaQoi8I=;
 b=VmAPSp6bs7xh91+LR0Ait+pq78OmwpFJ1L1fdgJW39/e4zIba+z+k8HdNdsJ9GCBUOnKnog+A0eu8d0PBf+j9rVpSHpHm/0Orh/PhZkfYSjZf23sLtyW7nRtCuDJmh7UO0P0FHSNVQQyOlnNc3T7MtOXwfrGCjhdoV/MVFSAeyXMlwnqrPRUk4Kse69kEJGk2hETXFH15/JoKZUyhsRcsCjPsiGsM84GgWCVpnADsa7HijwP6nGHmHL8HoJ7OB/LlPGqghRc+b3KOHE9LYQKyqKI63Ldl6j5dObSwLkDmS7ne8nvu3RAgF1E2Pud6Ig1k704DWPyTzc6yfvrirfplg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=02lvqIFeq5ueMXlcG0MgTjXHnFIspvYKtsnJlaQoi8I=;
 b=lLuKWj6CBelA+oFhxzhOFkaU5QhzYs/lLiXHL53JnI+IJuh6JBqKImZkJ4rUunsLnPKwyxCufvVDnnujlGHT296ZVtj29/KUiy2fwsg/Pn8IrwKZWVjFgLrcl4mH1A3JpSleBEDJbbLB5z5wi7sIEAU2RIHZXy10nlCyE/hz5bs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 07/13] device tree: Add _dt_find_node_by_path() to find nodes in device tree
Date: Wed, 1 Sep 2021 23:05:57 -0700
Message-ID: <1630562763-390068-8-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d13ac05-bd98-4b27-6ce7-08d96dd7f2d7
X-MS-TrafficTypeDiagnostic: MWHPR02MB3245:
X-Microsoft-Antispam-PRVS:
	<MWHPR02MB3245AFF35E16EA33879237AABCCE9@MWHPR02MB3245.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tIH8MefcwcRpvihGrdXSfRE9v6Hh/IeHlbfK0/zW9GVbZG02tTx7mx2WfKTl1kS27XrYGS++2+AbkpfEUxznalv9QcGNa9Xw4ATLF+8X8eWrrJ6OjM1PzkTMc55rJ2mspgTgI8Abc4Lxf2Z1aba7EgnEzQsF1TzasW/+uW32vq2jDAwJGtOJu2NrVA5TjoehMMc8+xjln7mQKtICIzp/vJLCjNAxpzAOlDcRQDJBLrmUtA6T6IufcGRc0acDgammV7QwrNoK3ZbLRqtUVQJue3Q24ZPpDiB/JrvXLASyaa+tZdK8zBpjLF3Rzf332cZE+5QRPt3uNbnioYymlGJ9iVUExJfYzbBGlCVPsVnwHK5FwHm9h8TSTB9ysGBUYj1VjHfeMI9ARGpgUT/TkVzqqffUf1Xa6MqaX43uEyD6HARRLOiv5/7KzBsxIXZdUJc5OqWWWCJwTxD1JHwh9J+uzsQ0sz3nD32KS8pivxqlIH6Y+oXgsJORcXLmLTYhSelNKaWSgHgPKrnDWFCbi58qM3obS9rrG+apG8PMtYf6147KYreB9Y86VyekTnm875mKseajHfXIaxMHXR4A7j+ZVtnFRYivHOhx0X2HUyT5nTX+hNJuvc34XHaPbD8C2ADR7Sn5urTeU0KSsvb4FKapaph+Tyunhlwu+906YXgP05fzz41bOY8ahI26oXBnVH8l7ZEoV+IKV6IrTDPVEc2DvvK0AcJPXpnbTKWd8V6GguhNBWG9P7kHyZVb9ZmNUJRO
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(346002)(136003)(376002)(46966006)(36840700001)(26005)(9786002)(426003)(36860700001)(47076005)(478600001)(82310400003)(2616005)(6916009)(70206006)(7696005)(6666004)(7636003)(8676002)(70586007)(186003)(107886003)(8936002)(2906002)(4326008)(54906003)(5660300002)(36756003)(336012)(356005)(316002)(36906005)(82740400003)(83380400001)(102446001)(37363002);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:30.7817
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d13ac05-bd98-4b27-6ce7-08d96dd7f2d7
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1NAM02FT008.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR02MB3245

Add _dt_find_by_path() to find a matching node with path for a dt_device_node.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/device_tree.c      | 10 ++++++++--
 xen/include/xen/device_tree.h |  9 +++++++++
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index bfe3191..4946e83 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -297,17 +297,23 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
     return np;
 }
 
-struct dt_device_node *dt_find_node_by_path(const char *path)
+struct dt_device_node *_dt_find_node_by_path(struct dt_device_node *dt,
+                                             const char *path)
 {
     struct dt_device_node *np;
 
-    dt_for_each_device_node(dt_host, np)
+    dt_for_each_device_node(dt, np)
         if ( np->full_name && (dt_node_cmp(np->full_name, path) == 0) )
             break;
 
     return np;
 }
 
+struct dt_device_node *dt_find_node_by_path(const char *path)
+{
+    return _dt_find_node_by_path(dt_host, path);
+}
+
 void dt_print_node_names(struct dt_device_node *dt)
 {
     struct dt_device_node *np;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index dcd96b4..7cc6093 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -469,6 +469,15 @@ struct dt_device_node *dt_find_node_by_alias(const char *alias);
  */
 struct dt_device_node *dt_find_node_by_path(const char *path);
 
+/**
+ * _dt_find_node_by_path - Find a node matching a full DT path
+ * @dt_node: The device tree to search
+ * @path: The full path to match
+ *
+ * Returns a node pointer.
+ */
+struct dt_device_node *_dt_find_node_by_path(struct dt_device_node *dt,
+                                             const char *path);
 
 /**
  * dt_find_node_by_gpath - Same as dt_find_node_by_path but retrieve the
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176918.322102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYr-0005C1-T7; Thu, 02 Sep 2021 06:50:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176918.322102; Thu, 02 Sep 2021 06:50:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYr-00058P-Ij; Thu, 02 Sep 2021 06:50:57 +0000
Received: by outflank-mailman (input) for mailman id 176918;
 Thu, 02 Sep 2021 06:07:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsU-0004ds-5y
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:10 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.100.73]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 01ae88ce-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:09 +0000 (UTC)
Received: from DM3PR12CA0074.namprd12.prod.outlook.com (2603:10b6:0:57::18) by
 PH0PR02MB8501.namprd02.prod.outlook.com (2603:10b6:510:100::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:07:08 +0000
Received: from DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
 (2603:10b6:0:57:cafe::8) by DM3PR12CA0074.outlook.office365.com
 (2603:10b6:0:57::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:08 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 DM3NAM02FT042.mail.protection.outlook.com (10.13.4.213) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:07 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:05 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:04 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsO-000F6q-Ug; Wed, 01 Sep 2021 23:07:04 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01ae88ce-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gQLsUwitGlfNT7El7H4B/XS7vcpOlCuP/nodYDXRqQkXDeLL04P0W7cYSfDMKGPtH0AbUkmZFOX5nWIdLeZy6vWnUNESCqlg6nQzfS4ht43yxzw2uTaYy9KN1e88ohU+8r0KfXBxfqfxo97N1fcCiJAsl1nvZOzEiFxz1nZyzg6WLGQPOZrg35nUqc84G3fsTGCMoagz1SDvySFGmu875Brmw6ZgRgooquKEXkZUdDUaW737wPQjEzy5QO5Jpluxd6IPhCoJAeIhUVn5nQH81XdrXf3T5jtxMe0WszjetHKnjU+cuqGb5d5rJPNN1y6gaDkZeCNIewxuaMq3I346kA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qQqiWML3nRBgAmoYFeaXNgx+nXTK27DqgxYrbIIxS+4=;
 b=Fw8d6SeJXhmuguBjdLmY99Wwotpio0YOuUCFzzVbWSn+WsIQaw4+4/CeKH2+ruwASa4t4P2CBfm1eWis+A9Uy1jITMljZaBBxM0on3nZYre/UuU2+fHY/s3FJouDH7ZhcAfVxHURiBjTSmwlqfJsEzuuCSrEwe5Sb8RwckxRINNhzAo217+b8dQrxL8N4chvAg7CYjGXuJT97QB8GHRd3D9mAudWhgDyeJuEMfHAqcwkFiH2TTGsIicLmuKHlHKZqoBa/YP2b3l54n27o6ORY8ZHItHEsS/LB0NAxaMsXrldiyT0spy8UxYr0LrxHGt81H3kEHWLZtul+7voG8uHCw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qQqiWML3nRBgAmoYFeaXNgx+nXTK27DqgxYrbIIxS+4=;
 b=bYD62rRraU3PKWxLybe0b2QGpjErYChmHJwCmVGwIkmBM1E5XlYWcgzaj4e6pWT0oyrKQvWRm5ulz7DQcSUDue0+HedEjxHDd2px7TTkm05+bMLBB/Gg/DRY5pc/2PvDEAHU4ILDqEEqgV8WBUOV9ljDcTwJMYLi3ESjdZxo5T8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 06/13] device tree: Add dt_print_node_names()
Date: Wed, 1 Sep 2021 23:05:56 -0700
Message-ID: <1630562763-390068-7-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3613ec78-5dcd-4519-f133-08d96dd7e514
X-MS-TrafficTypeDiagnostic: PH0PR02MB8501:
X-Microsoft-Antispam-PRVS:
	<PH0PR02MB8501F13CF415DE8BBD61E09CBCCE9@PH0PR02MB8501.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:154;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zthyOMV3KedVnyLP35/3+eY38V0CwfqIzFmRHxK9tLY7wM6M9jxmRgrxqR8NzYhhIMkWg//XpjuzbLt1AepqGZi73OonoWXkDmTiXW27iFtgSZJVIeukVxIZlqHA/uynne6XkDwNOgniVQwpicQmOJDJwF3UjPsihvA3wXR+bZpR3cyoOgX5HKMkC14zVquC8H+xLO1OTJlBEuGZHBrtTMaH/+kzXLJYVXfCwIyf/LZswVXemlEA+zcCboGZKoFm54BAGYXe7APBk0K1L6nokdbBjNXfaE2VRYgt9fD0ddcpNqEOLvAejQdjP11hjPgdPR2LB6Z2IpjSQ1SqmryNOv4xvmvuxCWwjFzvupbrTwNJGXOyVG2J5mWeVaz3vKTOOFUnb6wGtfe4SRNC+9eHM36uUamS/k6KP+VJtbbKdixrW2ZZ0BLU72nvNINHeiFmYY7kspSOjwKPmmdqq3saD1jPQC2V67BPzILD+Lzqt1AtOlwJOax1nPF6eNFAq99Kk5wocNO2CImmhvhxuAshcJYeWTj1zhpYjL3uQa4wxlGOoCBT+mnaWE1xFsVtMlZVbbHzFbqmHiwc2iTXXugYkH9ZJm2l1R9cZNdm6eG493rQzXm5r0Y6KQRLzTJff9wknn8ON58geKCJSTQJAJ0Faf9AfLrc8/cA1BTAmqa6rR15lHk85dQdlE2P7WEHcHMY1Sah9XkS5lFRCm9E3fBBuvbMLeGfHwPMwwYd5KgErs0=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(426003)(186003)(2616005)(6666004)(8676002)(107886003)(36756003)(7636003)(2906002)(26005)(6916009)(82310400003)(70206006)(70586007)(336012)(47076005)(316002)(36906005)(36860700001)(508600001)(7696005)(5660300002)(9786002)(4326008)(8936002)(356005)(54906003)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:07.7477
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3613ec78-5dcd-4519-f133-08d96dd7e514
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DM3NAM02FT042.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8501

Add dt_print_node_names() to print all nodes under a dt_device_node.
dt_print_node_names() takes a dt_device_node type input and prints the node name
of all the subsequent nodes. This is added for debugging purpose for device tree
overlays.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/device_tree.c      | 10 ++++++++++
 xen/include/xen/device_tree.h |  5 +++++
 2 files changed, 15 insertions(+)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index cda21be..bfe3191 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -308,6 +308,16 @@ struct dt_device_node *dt_find_node_by_path(const char *path)
     return np;
 }
 
+void dt_print_node_names(struct dt_device_node *dt)
+{
+    struct dt_device_node *np;
+
+    dt_for_each_device_node(dt, np)
+        dt_dprintk("Node name: %s Full name %s\n", np->name, np->full_name);
+
+    return;
+}
+
 int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
                           struct dt_device_node **node)
 {
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index a4e98a7..dcd96b4 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -483,6 +483,11 @@ struct dt_device_node *dt_find_node_by_path(const char *path);
 int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
                           struct dt_device_node **node);
 
+/*
+ * Prints all node names.
+ */
+void dt_print_node_names(struct dt_device_node *dt);
+
 /**
  * dt_get_parent - Get a node's parent if any
  * @node: Node to get parent
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176908.322068 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYp-0004dP-Tp; Thu, 02 Sep 2021 06:50:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176908.322068; Thu, 02 Sep 2021 06:50:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYp-0004dI-Qn; Thu, 02 Sep 2021 06:50:55 +0000
Received: by outflank-mailman (input) for mailman id 176908;
 Thu, 02 Sep 2021 06:06:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfro-0004YX-UW
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:06:29 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com (unknown
 [40.107.95.82]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 30620bd6-43c4-461f-960f-858df19fd544;
 Thu, 02 Sep 2021 06:06:27 +0000 (UTC)
Received: from SN1PR12CA0061.namprd12.prod.outlook.com (2603:10b6:802:20::32)
 by MWHPR02MB2302.namprd02.prod.outlook.com (2603:10b6:300:5b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17; Thu, 2 Sep
 2021 06:06:25 +0000
Received: from SN1NAM02FT0022.eop-nam02.prod.protection.outlook.com
 (2603:10b6:802:20:cafe::27) by SN1PR12CA0061.outlook.office365.com
 (2603:10b6:802:20::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:06:25 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0022.mail.protection.outlook.com (10.97.5.0) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:06:25 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:06:24 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:06:24 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfrk-000F6q-6e; Wed, 01 Sep 2021 23:06:24 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30620bd6-43c4-461f-960f-858df19fd544
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WdCUJjGX/U02no8M7aGRhNE8QGABZTwSibiGs5cryOzgLs/kXrwATPn3lxBnVgWbHy4YnkhNmJ9vi/Gw3iu1IsoRYZsyeYb88ks0uKzBAt1lo4InLlE3PJeyS2EoVae1ANmls8Zw0c8vlLwW21sCyoAjk5QpKoPbJ7NpssUqWSeT+BNHCIYOnkutkulJmrGnlcLle5auY29SvpQryqRjEYhbwOgARKEKhcE8xG9NVZ/sjY+TvlgqaOLmPjK305sXho5T3N9JJ1SCKGn+biqwgrlph9Gad4tiTM+0yi7X8whS1V2tRbZXfBEod8YrgMbjlzP1QN3WXH1wHDXh5RrymQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ubn0KOlNn2bGNNG4KibjKy56O3QZHfvA/XdfMZxo38c=;
 b=eHfV1Jun6/Y9BR7EOwH5D65neWPrYr2/plZLf6dfkn8VqY9EqaoI7UpYieURbUNWeKPnBfKWYuOwZY4t+wwKPEvWXy6lhh/xI8Wl17RjSoe4fEBR5FM56EhY7EvBfuufa9FfbqzYFlSbtZNMxir3XTXNjC4nkynMFnbfhgK2hIGB9LwFIVB5Bzo+Ufwukdy4vZIZaTrWvtd4DdaLkUJXFEbfVzwr8dHbaqS2w+aNJaxglqldaTOy6LsFEZYFhv4w+Hoh8Hg3v56hsn+gDX8Gtpp/zO30yh4uYPV0RwrK8K5Un5P4eO6etcL/8PT9tIwGdWyhgvMgb7pwqeJ2e2KOOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ubn0KOlNn2bGNNG4KibjKy56O3QZHfvA/XdfMZxo38c=;
 b=cGnM3y0zqxRiK5jg0p2ffdAPvgKnhADdF8GbGtARgl8WKKjA5b1Nvccpg3yY0Ty8kTzwMcKLzL54YEKWMFGUg0jstqk18gDp+9VhG4s9Bd3qDSC3+XfB08KqT5yXIbY/R16nXbSi73MjN8sZpcIFaVYWtW9Rqsh2yqwU+5X7A1I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan
 Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Juergen Gross
	<jgross@suse.com>, Anthony PERARD <anthony.perard@citrix.com>
Subject: [XEN][RFC PATCH 00/13] Add Support for dynamic Programming
Date: Wed, 1 Sep 2021 23:05:50 -0700
Message-ID: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c9207777-2281-4bf2-ad36-08d96dd7cba0
X-MS-TrafficTypeDiagnostic: MWHPR02MB2302:
X-Microsoft-Antispam-PRVS:
	<MWHPR02MB23023F8AF9FFCE0FECC07EF9BCCE9@MWHPR02MB2302.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	w5KHpqXz4mYyBcpsuTSzM1/Vs0D8q4a4ugCmQ4Zt5HjOK7r74D79y6at2F9MD4tBfQgGX5ah0FsXMIfbtFaQf2g75IyayA3OF4DQNNikPdermzCZqejLTtU1UOrQzGksKF7yxZNfw4d1+xH0YgZ77lVwm7CxgJ8hpL8qQDVHQye58WByKUYwueqlUUBKcWieni8oGElTYoqFD5vxl3F/5FX4vhHm4DtrJ/WQc0rgp5HSCmTxWBqLE4qGH0gU4OZxqIIi8RIxSRHOeuL7mtJtVFD+d7mV4RkmFpCvIS99PIu5z8I2Es/kbkbBz59vf/ie8JP1KH+C2XYsSJinyPcUjQR/JiHqPunqWQatusp9ytnuhwXEixL4VmrUndUsZE60LQRZ5PgBv0rihWIEyc1bzZ3ZXcR+yA69aGJ2kuzOIdkJgN/pSZv5PhpshX9HL9TNDEJYq++8iTkHijXnbKlsXMSZ7cUtyXsr6H8T+5Bg4VQSNdc4yW2wnBLyT2rSDrx1GvHEVKKg81b4g3NGSGYergvwLsWJAPkk1w813t8xFRuqIenU7CEJfI0f0zMs4sxgP6GAuy2OY3YRIkvUMVFHc3APFX4qVdO/wTCOUSn7yDMPBpeGETqRIT1kcGAv5qyF5Ygigf7nci+HRCbg9+2dE8I/fHmgmw6oZjBHHSlvoxBWHNXiTqNJTSsoTlWlmqA6Vkn7PyCrR7ZF25zXvQCgdcPnZ+4RJ+MRtbKO+ankrak=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(136003)(376002)(39860400002)(396003)(346002)(46966006)(36840700001)(82310400003)(4326008)(2616005)(426003)(2906002)(356005)(186003)(82740400003)(8676002)(36860700001)(6666004)(36756003)(70586007)(70206006)(83380400001)(7636003)(478600001)(336012)(5660300002)(7696005)(47076005)(36906005)(54906003)(7416002)(316002)(9786002)(26005)(8936002)(6916009)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:06:25.0547
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c9207777-2281-4bf2-ad36-08d96dd7cba0
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0022.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR02MB2302

Hi,
This RFC patch series is for introducing dynamic programming i.e. add/remove the
devices during run time. Using "xl fpga-add" a device can be added with dtbo.

fdt_overlay.c file is taken from Linux and other existing fdt files are modified
to accommodate the fdt_overlay. The changes w.r.t. existing fdt are kept minimal
i.e. only the required library file/functions from Linux fdt are pulled in.

For adding a node using dynamic programming:
    1. flatten device tree overlay node will be added to a fdt
    2. Updated fdt will be unflattened to a new dt_host_new
    3. Extract the newly added node information from dt_host_new
    4. Add the added node under correct parent in original dt_host.
    3. Map interrupt and iomem region as required.

For removing a node:
    1. Find the node with given path.
    2. Check if the node is used by any of dom0 or domus. Removes the node
only when it's not used by any domain.
    3. Removes IRQ permissions and MMIO access.
    5. Find the node in dt_host and delete the device node entry from dt_host.
    6. Free the overlay_tracker entry which means free dt_host_new also(created
in adding node step).

fpga-add and fpga-del are probably not the best names for these commands. This
was done initially for xilinx FPGA programmable logic block dynamic programming.
I am okay with replacing fpga-add and fpga-del with better names if there are
any suggestions.

Regards,
Vikram

Vikram Garhwal (13):
  device tree: Remove __init from function type
  libfdt: Keep fdt functions after init.
  libfdt: import fdt_overlay from Linux
  libfdt: Copy required libfdt functions from Linux
  libfdt: Change overlay_get_target() type
  device tree: Add dt_print_node_names()
  device tree: Add _dt_find_node_by_path() to find nodes in device tree
  xen/iommu: Introduce iommu_remove_dt_devices function
  xen/arm: Implement device tree node removal functionalities
  xen/arm: Implement device tree node addition functionalities
  tools/libs/ctrl: Implement new xc interfaces for fpga-add and fpga-del
  tools/libs/light: Implement new libxl functions for fpga-add and
    fpga-del
  tools/xl: Add new xl commands fpga-add and fpga-del

 tools/include/libxl.h                 |   5 +
 tools/include/xenctrl.h               |   4 +
 tools/libs/ctrl/Makefile              |   1 +
 tools/libs/ctrl/xc_fpga.c             |  82 ++++
 tools/libs/light/Makefile             |   1 +
 tools/libs/light/libxl_fpga.c         |  73 +++
 tools/xl/xl.h                         |   2 +
 tools/xl/xl_cmdtable.c                |  12 +
 tools/xl/xl_vmcontrol.c               |  51 ++
 xen/arch/arm/Makefile                 |   2 +-
 xen/arch/arm/domain_build.c           |  15 +-
 xen/arch/arm/domctl.c                 | 445 +++++++++++++++++
 xen/common/device_tree.c              | 143 +++++-
 xen/common/libfdt/Makefile            |   1 -
 xen/common/libfdt/Makefile.libfdt     |   2 +-
 xen/common/libfdt/fdt.c               |  35 ++
 xen/common/libfdt/fdt_overlay.c       | 884 ++++++++++++++++++++++++++++++++++
 xen/common/libfdt/fdt_ro.c            |  52 +-
 xen/common/libfdt/fdt_rw.c            |  81 +++-
 xen/common/libfdt/fdt_wip.c           |  20 +
 xen/common/libfdt/libfdt_internal.h   | 130 +++++
 xen/drivers/passthrough/arm/smmu.c    |  53 ++
 xen/drivers/passthrough/device_tree.c |  30 ++
 xen/include/asm-arm/domain_build.h    |  10 +
 xen/include/public/domctl.h           |  16 +
 xen/include/xen/device_tree.h         |  21 +
 xen/include/xen/iommu.h               |   2 +
 xen/include/xen/libfdt/libfdt.h       | 232 ++++++++-
 28 files changed, 2363 insertions(+), 42 deletions(-)
 create mode 100644 tools/libs/ctrl/xc_fpga.c
 create mode 100644 tools/libs/light/libxl_fpga.c
 create mode 100644 xen/common/libfdt/fdt_overlay.c

-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176913.322087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYr-0004t7-0L; Thu, 02 Sep 2021 06:50:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176913.322087; Thu, 02 Sep 2021 06:50:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYq-0004qA-Mm; Thu, 02 Sep 2021 06:50:56 +0000
Received: by outflank-mailman (input) for mailman id 176913;
 Thu, 02 Sep 2021 06:07:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsP-0004aK-62
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:05 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7ea9::620])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8dbdcee9-9b09-417a-9938-75d68b68fcde;
 Thu, 02 Sep 2021 06:07:04 +0000 (UTC)
Received: from SN6PR08CA0015.namprd08.prod.outlook.com (2603:10b6:805:66::28)
 by DM6PR02MB5963.namprd02.prod.outlook.com (2603:10b6:5:150::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 06:07:01 +0000
Received: from SN1NAM02FT0033.eop-nam02.prod.protection.outlook.com
 (2603:10b6:805:66:cafe::27) by SN6PR08CA0015.outlook.office365.com
 (2603:10b6:805:66::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:01 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0033.mail.protection.outlook.com (10.97.5.40) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:00 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:06:53 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:06:53 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsD-000F6q-LT; Wed, 01 Sep 2021 23:06:53 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8dbdcee9-9b09-417a-9938-75d68b68fcde
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kjYN5gRN41ONS/9py890LcpPZatY8XZoRIXzL4G1r2Fv+MDCZ6TA1BJpViPe9TMa95LQq249H/uWXqzYGBca1ndVn8vRQslgXx68qicKNWOFYiB9gclZjFvJu+2s+eOkpDIKc8cZfRjaiz3KGHNZmHag77JSio/RziVaNqnUrxpVUgCiGVdbvjBvFdT9iIyDPEhdfCzZtmWNj8otE0U/t3MEHxHLHjuyGo/lLTvAFZ6vTDAOp1+nfWbzCUUeo7jzEgqAX0C15cEec9fEUtmmJobhXxkDaJVI+U7jqv7tIgC99xpi+fW5dBEZtVZkT/jIJRSbw+HqxBuXCvxU6QbglA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZeeTflOXr2fLy1zXz1tSECVgWqlU5geTYJsntNlox3s=;
 b=OF+IgvEhO5Y5P+PZkSDmLjnOguHEuvAr7js1bLekz+GPSoYqZS4ZvstfYo+XHHrB2pSiX9z9pgRmuw3UBl8w9n5tZ9rINAG1Md+0QpyQBh64B+rdbFxcmD3j1Ata26vC9AOzbNI7XeBEGcQeliqMrX6qYB/FCgzv3luuvIbfLy4kfnC1jAUcwtw/H5w9jcnbeIllWoQDN0PPiHWt+afr/AWdfz5el+im+AEc0ZraXq9vafa6zLhGWOmysMMFRFHsZkvMl6XMZqFqJiAMFp7bfpLAJwLSZ/1+qIMJCNZnEPQEJNJVGZ/KSfkfO9g7Y1gMWyXNYu+Z6DKy74qMI0QoQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZeeTflOXr2fLy1zXz1tSECVgWqlU5geTYJsntNlox3s=;
 b=eo4dZRj8Dg/W/uPTz4yYReeQwF2MlpnkcUwpSlWoSbdvtIdY2FQoM9GT3vHowOSGar5hcJNf8m1nxWchrO/Ih7Pf1LJr5rlO6WyVJEwxpox5N5MDDGe5B54r9T7YSKdrE3dHCwNwZS3CbXh0REORFjjw7Uy2jflkVIeDcko72PU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>
Subject: [XEN][RFC PATCH 02/13] libfdt: Keep fdt functions after init.
Date: Wed, 1 Sep 2021 23:05:52 -0700
Message-ID: <1630562763-390068-3-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2b4bb6bb-5988-4c66-fc7d-08d96dd7e0bc
X-MS-TrafficTypeDiagnostic: DM6PR02MB5963:
X-Microsoft-Antispam-PRVS:
	<DM6PR02MB59637A67F32DDF00E76BD589BCCE9@DM6PR02MB5963.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:162;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	BMAij+jdIAVuLQQq++CAx7Fjpjid4kCxlOJKB2jkx8H22EVbZIYETpXZY/UCpQf/T2eT7XXOTWRRjj8OF2mjIeVhcDSjAXf2TXVcDU2nWjOBSl4tDRrZQfm6q7mAIxfBzj4zFCy4zmpKJfisMkJPsQPL4MIAs8c04MIszjuOfyeCALzoSLlpUy5gGw88m6lLR0wgiuFFNGj42Cpfe8kB98CAheKbvbTFYNIrTE84nFqeftyoDOzR02CYdRR+ddYaf5EfOoOMRHrsgOSSBoORg2sF67apzaLYJ1tpCb/Z8xO3Bnomk3rNqcVkcqCvV78T0bKwskjGuJutelYrTPIdUohIe1FdNCeGsKrHwCeEDTM9fveL5mYqHR+eur7B0iqImCEreEFDz/uS9/x74uzGUeAZ3dgTnCNq0XPc5WiBEYXSQeIIBNFxZxuD9uy/yvp+bLCeVvjsTcZUvWRq9BO4CTj2yISC+0KAdAa5Xun7PlWMHnbRje3B2iLunyjgLwI+4CuVXa4lSAArDotueBp0wsy4sUia65pDcBvPZkeqQP1jA5ymA5daNrjsvW7Npq61RMRGUWVeDyX5hjatss6JDfroF2Kp4jXwhlI3xmMB0FJxCwulIZWvQo/dx8oEssY92/wNy1h8FraBUd7oO9qDIRGsoxCrhJ6x84O5jAlATBkOmsXWEOqwihkB1J+aJLRSAdVZISWikNhQ5mj+tO7dhxHgvVbObyCnivnsMsMeIYo=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(346002)(396003)(39860400002)(376002)(136003)(46966006)(36840700001)(4326008)(356005)(70586007)(336012)(107886003)(426003)(478600001)(82310400003)(70206006)(36860700001)(6916009)(2616005)(47076005)(186003)(7636003)(26005)(83380400001)(82740400003)(5660300002)(9786002)(54906003)(7696005)(8936002)(36756003)(2906002)(8676002)(36906005)(316002)(4744005)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:00.4714
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b4bb6bb-5988-4c66-fc7d-08d96dd7e0bc
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0033.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB5963

Keep libfdt library functionalities after boot of hw_domain. This is done to
access fdt library function which are required for adding device tree overlay
nodes for dynamic programming of nodes.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/common/libfdt/Makefile | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 6bd207c..8002f8c 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -1,7 +1,6 @@
 include Makefile.libfdt
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
-OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 obj-y += libfdt.o
 nocov-y += libfdt.o
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176910.322072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYq-0004ff-7B; Thu, 02 Sep 2021 06:50:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176910.322072; Thu, 02 Sep 2021 06:50:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYq-0004f4-3V; Thu, 02 Sep 2021 06:50:56 +0000
Received: by outflank-mailman (input) for mailman id 176910;
 Thu, 02 Sep 2021 06:06:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfs8-0004ZP-Ju
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:06:48 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com (unknown
 [40.107.92.53]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f479392f-0bb3-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:06:47 +0000 (UTC)
Received: from DM5PR07CA0167.namprd07.prod.outlook.com (2603:10b6:3:ee::33) by
 DM6PR02MB4218.namprd02.prod.outlook.com (2603:10b6:5:a0::28) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4436.22; Thu, 2 Sep 2021 06:06:45 +0000
Received: from DM3NAM02FT050.eop-nam02.prod.protection.outlook.com
 (2603:10b6:3:ee:cafe::29) by DM5PR07CA0167.outlook.office365.com
 (2603:10b6:3:ee::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:06:45 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 DM3NAM02FT050.mail.protection.outlook.com (10.13.5.53) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:06:45 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:06:44 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:06:44 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfs4-000F6q-1L; Wed, 01 Sep 2021 23:06:44 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f479392f-0bb3-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W9cc6jUikAlp+4tIMYtHViI3oxKr/q1xm+LoyryDZHotnMiUXsZgpfvUIHty6EejDMZNwtXqhJ4BzJnBD6K5nNAmAqrk9sTOElDdhpOYjgcohynI6CCOkXXzBYScsJAuXa2CVB3U6mQD5XbQql66mkorGckbhNeOU3mjNbNIeNQke9JP1vUyJSbQVZEctoLMoAqeumBjLgBNgYB63xhURgjNno9VkV5gTJRbfg6vfXG1qFbG4vrL50If1CPtKfkIitJj0JHTgqOO6Yh+aB0P8m2vBB0CyS3LtGyNnheKWa3Uxhpm+8hbCbCsruwx/aHDLp7k4ms0dh1xcjiMwnsZEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p0sULzvu56yUkRR8j2QyEg/IiWAf0h/83PzvUjdhg/o=;
 b=mPdDmT3Baq+YiuR8uzEKBZhOsWrbumyFJsNnoIyMcvi0YQbxAdwVPtm00y9H40s7hz0zmfqtFyw7W/uyr6se3CD0xoafNwty2IuJ5rhCn9oe1kce/lI+kqxLGhvnROdoaVygFtThXelxQ/72qCSSM/R8ZmkHsZdeb/q3mFZ3YwMcPc83A64ReGEjREV5o+U3BmqrA3KRAXFDWlyLzJ7PL6nfGEmXCks8KtbINCmOwG2SC5JVbb1mw5pVNk1Z9OBSpQH+mtPE+zhsIX5fP60GE8mrxD+Ct6/mT+1GlOMIkVdlMG4RAFp2duNGaJnYxYKpw3x/UlRzsOXCGabQJU4C8A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p0sULzvu56yUkRR8j2QyEg/IiWAf0h/83PzvUjdhg/o=;
 b=TiuU8dH4jgG+X2FlI0nGaDbqiNe+3eB17iIgiAudaWv47XmqislETrrLTz1wKXNThHGL0Jc4aCKMr+LI+L4lpEy4DEwj/TFVQmJCkQax8Dz0rhWqm9aEM/KVzmnn3D1dx5pz+rLxqrZLLwhBfN0oU3pt5UWQgndjjWPmbHhdUaQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN][RFC PATCH 01/13] device tree: Remove __init from function type
Date: Wed, 1 Sep 2021 23:05:51 -0700
Message-ID: <1630562763-390068-2-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2b402bf5-966b-4cbd-aa4a-08d96dd7d797
X-MS-TrafficTypeDiagnostic: DM6PR02MB4218:
X-Microsoft-Antispam-PRVS:
	<DM6PR02MB4218712926570DB3C137CB34BCCE9@DM6PR02MB4218.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0HMU2JX5WpfuP5g1RAWmdCLTpQ2YBrd+IW8uYGBCTUstQQxgphy0rUMYUt6hGkyZyazn4d/7zTjiT6Qs9aBLxgrUa1osuklJGTortSVQXmNa2Fk1PSvTS5UZ7XopWxT9ACqkIQ3QmUrWlONzxiztbYVJyAUzffYwkSfPxq9ylwe7eebs5mpWjHQ2md5gsBGbmuYeuZ6UXogEiNyruqJvRmJ9B9wW61t5hkmjSZZac8CBSicVuKJxF81n3gO73wrmUMgpDwrqah9ZOoUGPj12ng8p9163AHDUA18F3OzYihHUMDpOxjbzQ035AUO8q8SDDmer1bzZOR94FLB2IeZ9VxOx47Vb+bH+/7g5SJRpKlK2vl9XMeaOVSvkp3vLoegbmPx2ECrc3JNdsd1nCYyso95LMSI4+nqXCjRdoeJs3U28xgo1xxfy0mBJwgF1seqyGds8TOOp1NIZKQymAFlQ3cO3oSollbCjTlUJtwRgqRRZljP1yvcPFckjTJoih728lyyHXZkjd3GiDAh+WE338kkxxmJNZAUGmjrI57OGqXLgosygNdf+ILmpSHOnT0L4HWczSOZ0843VoMjjJVzpmS4M7QXJriz5w/fIyG6xhZG9y92GXLLzYoRgbhspZpV8gFj3jPhEoBOTSVTvY8Nv9RplCal5xwO844n62nAyeOX3+p/Gy6IHbfhKGK2x7pzUYBrOYaq9M2DM1DnXcP2pITiYQM4+Ef5fVAQuiqVuJk8=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6666004)(2906002)(54906003)(508600001)(8936002)(8676002)(2616005)(70206006)(426003)(6916009)(36756003)(5660300002)(7696005)(36860700001)(83380400001)(36906005)(82310400003)(26005)(336012)(316002)(4326008)(70586007)(356005)(9786002)(186003)(47076005)(7636003)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:06:45.1163
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b402bf5-966b-4cbd-aa4a-08d96dd7d797
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DM3NAM02FT050.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB4218

Change function type of following function to access during runtime:
    1. handle_device_interrupt()
    2. map_irq_to_domain()
    3. map_range_to_domain()
    4. unflatten_dt_node()
    5. unflatten_device_tree()

Remove .init from domain_build.o and move map_range_data declaration to
domain_build.h.

These changes are done to support the dynamic programming of a nodes where an
overlay node will be added to fdt and unflattened node will be added to dt_host.
Furthermore, irq and mmio mapping will be done for the added node.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/arch/arm/Makefile              |  2 +-
 xen/arch/arm/domain_build.c        | 15 ++++-----------
 xen/common/device_tree.c           | 18 +++++++++---------
 xen/include/asm-arm/domain_build.h | 10 ++++++++++
 xen/include/xen/device_tree.h      |  5 +++++
 5 files changed, 29 insertions(+), 21 deletions(-)

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 3d3b97b..bef4517 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -15,7 +15,7 @@ obj-y += decode.o
 obj-y += device.o
 obj-$(CONFIG_IOREQ_SERVER) += dm.o
 obj-y += domain.o
-obj-y += domain_build.init.o
+obj-y += domain_build.o
 obj-y += domctl.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-y += gic.o
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d..3a457d3 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -47,12 +47,6 @@ static int __init parse_dom0_mem(const char *s)
 }
 custom_param("dom0_mem", parse_dom0_mem);
 
-struct map_range_data
-{
-    struct domain *d;
-    p2m_type_t p2mt;
-};
-
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
@@ -1144,7 +1138,7 @@ int __init make_chosen_node(const struct kernel_info *kinfo)
     return res;
 }
 
-int __init map_irq_to_domain(struct domain *d, unsigned int irq,
+int map_irq_to_domain(struct domain *d, unsigned int irq,
                              bool need_mapping, const char *devname)
 {
     int res;
@@ -1210,7 +1204,7 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
     return 0;
 }
 
-static int __init map_range_to_domain(const struct dt_device_node *dev,
+int map_range_to_domain(const struct dt_device_node *dev,
                                       u64 addr, u64 len,
                                       void *data)
 {
@@ -1300,9 +1294,8 @@ static int __init map_device_children(struct domain *d,
  *   < 0 error
  *   0   success
  */
-static int __init handle_device_interrupts(struct domain *d,
-                                           struct dt_device_node *dev,
-                                           bool need_mapping)
+int handle_device_interrupts(struct domain *d, struct dt_device_node *dev,
+                             bool need_mapping)
 {
     unsigned int i, nirq;
     int res;
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 03d25a8..cda21be 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1750,12 +1750,12 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
  * @allnextpp: pointer to ->allnext from last allocated device_node
  * @fpsize: Size of the node path up at the current depth.
  */
-static unsigned long __init unflatten_dt_node(const void *fdt,
-                                              unsigned long mem,
-                                              unsigned long *p,
-                                              struct dt_device_node *dad,
-                                              struct dt_device_node ***allnextpp,
-                                              unsigned long fpsize)
+static unsigned long unflatten_dt_node(const void *fdt,
+                                unsigned long mem,
+                                unsigned long *p,
+                                struct dt_device_node *dad,
+                                struct dt_device_node ***allnextpp,
+                                unsigned long fpsize)
 {
     struct dt_device_node *np;
     struct dt_property *pp, **prev_pp = NULL;
@@ -1986,7 +1986,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
 }
 
 /**
- * __unflatten_device_tree - create tree of device_nodes from flat blob
+ * unflatten_device_tree - create tree of device_nodes from flat blob
  *
  * unflattens a device-tree, creating the
  * tree of struct device_node. It also fills the "name" and "type"
@@ -1995,7 +1995,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
  * @fdt: The fdt to expand
  * @mynodes: The device_node tree created by the call
  */
-static void __init __unflatten_device_tree(const void *fdt,
+void unflatten_device_tree(const void *fdt,
                                            struct dt_device_node **mynodes)
 {
     unsigned long start, mem, size;
@@ -2118,7 +2118,7 @@ dt_find_interrupt_controller(const struct dt_device_match *matches)
 
 void __init dt_unflatten_host_device_tree(void)
 {
-    __unflatten_device_tree(device_tree_flattened, &dt_host);
+    unflatten_device_tree(device_tree_flattened, &dt_host);
     dt_alias_scan();
 }
 
diff --git a/xen/include/asm-arm/domain_build.h b/xen/include/asm-arm/domain_build.h
index 34ceddc..17449b1 100644
--- a/xen/include/asm-arm/domain_build.h
+++ b/xen/include/asm-arm/domain_build.h
@@ -4,10 +4,20 @@
 #include <xen/sched.h>
 #include <asm/kernel.h>
 
+struct map_range_data
+{
+    struct domain *d;
+    p2m_type_t p2mt;
+};
+
 int map_irq_to_domain(struct domain *d, unsigned int irq,
                       bool need_mapping, const char *devname);
 int make_chosen_node(const struct kernel_info *kinfo);
 void evtchn_allocate(struct domain *d);
+int handle_device_interrupts(struct domain *d, struct dt_device_node *dev,
+                             bool need_mapping);
+int map_range_to_domain(const struct dt_device_node *dev, u64 addr, u64 len,
+                        void *data);
 
 #ifndef CONFIG_ACPI
 static inline int prepare_acpi(struct domain *d, struct kernel_info *kinfo)
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index b02696b..a4e98a7 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -177,6 +177,11 @@ int device_tree_for_each_node(const void *fdt, int node,
  */
 void dt_unflatten_host_device_tree(void);
 
+/*
+ * unflatten any device tree.
+ */
+void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes);
+
 /**
  * IRQ translation callback
  * TODO: For the moment we assume that we only have ONE
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176926.322132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYu-0005n8-5L; Thu, 02 Sep 2021 06:51:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176926.322132; Thu, 02 Sep 2021 06:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYt-0005kp-Co; Thu, 02 Sep 2021 06:50:59 +0000
Received: by outflank-mailman (input) for mailman id 176926;
 Thu, 02 Sep 2021 06:07:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsv-0004hX-Ty
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:37 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com (unknown
 [40.107.243.58]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d71d7a5a-f0b0-458d-8457-d905d920e6b9;
 Thu, 02 Sep 2021 06:07:36 +0000 (UTC)
Received: from SA0PR11CA0187.namprd11.prod.outlook.com (2603:10b6:806:1bc::12)
 by MWHPR02MB2606.namprd02.prod.outlook.com (2603:10b6:300:44::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 06:07:35 +0000
Received: from SN1NAM02FT0051.eop-nam02.prod.protection.outlook.com
 (2603:10b6:806:1bc:cafe::47) by SA0PR11CA0187.outlook.office365.com
 (2603:10b6:806:1bc::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:35 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0051.mail.protection.outlook.com (10.97.5.34) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:34 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:13 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:13 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsX-000F6q-F6; Wed, 01 Sep 2021 23:07:13 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d71d7a5a-f0b0-458d-8457-d905d920e6b9
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PwWMyGPlBKtFHj1zM/gtcQiLwDgH+yShnsoTOJSsSbOnxUVjkYii203fAKF2tfN3J0iRBuAX9wx2as5wSlKlDt7V+FZrd5iC0gVGQmYWWG6l1u2+ald2Ldc28FAhqxaxOqHB95/lLOX0PQ6w9Uf+eziftZfbnhPOZPC4nlsOK8vd6eHFz2IGUuP2CYDoC7lqInZ7kIp9FnGC+b2SqJCkMEoYcQjlpokiCL4h7ppC7IrStudyGvXuVfRJ1OuYls8OmMwPwoS/unmsvEVPgF+rhyFN00spTN9ZDa1lN8WsIyt7iDsLSrm/KNbWnGNkaRbrpvr2JtRAxwcQcOOwNzZZpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EVas2kXYqY8J1i69hD1pnLTQnzKqbS4jKsdjLz6x97c=;
 b=baGfgKv6wfpslNLcyv4rKCbR3vhTEl/V+OdFy+IgivH9xZIn3/bMExf23cMh7y5/jpnd9xauUSY+AQopw3w2txn1uXKketaIOsJ9OE96cbKG8aOyLa+/RvbFZ33Uoa7wfbcWnx3buXcylsfn7L8sElihGqZIUKiUn+imUsnMxQcuvuTBjh1wWN6YaOXhpaCkf2vvWBC0PXRpMOd5b1OwiUMgmRyVdYf2DpvDx325qbPUsd2ZieSnwbQp6m/NYoDF90spKygJvB4ITpEYVyGH8hwE4+3SlbWGxcCjRbVdCNbrnCuFGFQk50cJvNXa6LFZxpPRXK7uYgoB2G1ntl4guA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EVas2kXYqY8J1i69hD1pnLTQnzKqbS4jKsdjLz6x97c=;
 b=OhDJ97JWnZlQmoCvp1jgfKBkXfc7CTrXRjx9a1itEgWqm+nvR8rkYqPd8FEVw2kYPd3cEUQhMrGgpdT85MROte2W9HmBJLrRE6QRIrg35hC2a3+dMECG4dqdsnbXG30TvYW1g4f8/V9tmF/jQQc2SBdGJdPfTKe99GV9/AcniQ8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan
 Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>
Subject: [XEN][RFC PATCH 08/13] xen/iommu: Introduce iommu_remove_dt_devices function
Date: Wed, 1 Sep 2021 23:05:58 -0700
Message-ID: <1630562763-390068-9-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e9508a7b-7601-4ec8-4fbb-08d96dd7f529
X-MS-TrafficTypeDiagnostic: MWHPR02MB2606:
X-Microsoft-Antispam-PRVS:
	<MWHPR02MB2606479ED549E6A3A76D6865BCCE9@MWHPR02MB2606.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TMQQTZIpkWWv10Ikt+KPGzuRQXvsobuI7WlcFc7zHglAU7EPq34TwwLc8REM6x9UCzfA+OIWUlUkYBJPchPItU4BduToxXXwNtLikwy+i7xfVo7EBHiKkws/1TqNsdPhOjOeeIU4/db7oS1GreqCl1UXP0bSnlLzYoKVeaQJcWRxIWK3OO8dN0xWFGzZDiNNDOZavcS6X4K6iaKtE7loInF61Q2w+ETeuV2N5+GfUytah1iE3bB5GlQE+0G5k4eYfJpVHh1lj/GMfiKHvfJOEgpPSVRIslaa6SaVTOlidGQGcRd/E59DAmBJYDB+dOvMh0IMoxSIYMMge3QPEq9XwmUFiQMcB0gowseVr41IVhhO74seru+ixoSQZAiF4vQmR83rQ2Z66odrEQPai81dJBE9LMr6stnG5nya1WpvLFLMUVjzXWAgcl2p73NIeUjaz6xVHoDqgkJvayixjJz9K6oKBsvaimIBnHgBmo9EZuMf/fKiZMhLfCdAxzKp856xouT9IuQDMHBRiAp9LC8+wLaFFt0QgvW4seVVdPaTqKshfyFW0HghGsv2ChgU0zWonNdP/KXYv14FP+4LKIwm+HnpUKyxycO2gGqCnKjeqyMVTRrxRUKuv5DRkb+wDyVapCTNoWkgaIo5OGoUchKQhTvO2tKEC8thALJM8DI8adV44qAHjuUdRh4SwMc9JbRWtA4N4i9tD9nUU1S3CwiHfSUsqE3SfXHG7yvIsw6KMJU=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(376002)(39860400002)(346002)(136003)(396003)(46966006)(36840700001)(36860700001)(186003)(336012)(36906005)(2906002)(9786002)(26005)(82740400003)(316002)(4326008)(426003)(54906003)(70206006)(70586007)(7636003)(47076005)(5660300002)(7696005)(8936002)(356005)(36756003)(82310400003)(6916009)(2616005)(478600001)(8676002)(6666004)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:34.7337
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9508a7b-7601-4ec8-4fbb-08d96dd7f529
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0051.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR02MB2606

iommu_remove_dt_device function is introduced for supporting dynamic programming
i.e. adding and removing a node during runtime. When user removes the device
node, iommu_remove_dt_device() removes the device entry from smmu-masters too
using following steps:
    1. Find if SMMU master exists for the device node.
    2. Remove the SMMU master.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/drivers/passthrough/arm/smmu.c    | 53 +++++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/device_tree.c | 30 ++++++++++++++++++++
 xen/include/xen/iommu.h               |  2 ++
 3 files changed, 85 insertions(+)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index c9dfc4c..7b615bc 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -816,6 +816,17 @@ static int insert_smmu_master(struct arm_smmu_device *smmu,
 	return 0;
 }
 
+static int remove_smmu_master(struct arm_smmu_device *smmu,
+			      struct arm_smmu_master *master)
+{
+	if (!(smmu->masters.rb_node))
+		return -ENOENT;
+
+	rb_erase(&master->node, &smmu->masters);
+
+	return 0;
+}
+
 static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
 					 struct device *dev,
 					 struct iommu_fwspec *fwspec)
@@ -853,6 +864,31 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
 	return insert_smmu_master(smmu, master);
 }
 
+static int arm_smmu_dt_remove_device_legacy(struct arm_smmu_device *smmu,
+					 struct device *dev)
+{
+	struct arm_smmu_master *master;
+	struct device_node *dev_node = dev_get_dev_node(dev);
+	int ret;
+
+	master = find_smmu_master(smmu, dev_node);
+	if (master == NULL) {
+		dev_err(dev,
+			"No registrations found for master device %s\n",
+			dev_node->name);
+		return -EINVAL;
+	}
+
+	ret = remove_smmu_master(smmu, master);
+
+	if (ret)
+		return ret;
+
+	master->of_node = NULL;
+	kfree(master);
+	return 0;
+}
+
 static int register_smmu_master(struct arm_smmu_device *smmu,
 				struct device *dev,
 				struct of_phandle_args *masterspec)
@@ -876,6 +912,22 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
 					     fwspec);
 }
 
+static int arm_smmu_dt_remove_device_generic(u8 devfn, struct device *dev)
+{
+	struct arm_smmu_device *smmu;
+	struct iommu_fwspec *fwspec;
+
+	fwspec = dev_iommu_fwspec_get(dev);
+	if (fwspec == NULL)
+		return -ENXIO;
+
+	smmu = find_smmu(fwspec->iommu_dev);
+	if (smmu == NULL)
+		return -ENXIO;
+
+	return arm_smmu_dt_remove_device_legacy(smmu, dev);
+}
+
 static int arm_smmu_dt_add_device_generic(u8 devfn, struct device *dev)
 {
 	struct arm_smmu_device *smmu;
@@ -2876,6 +2928,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
     .init = arm_smmu_iommu_domain_init,
     .hwdom_init = arm_smmu_iommu_hwdom_init,
     .add_device = arm_smmu_dt_add_device_generic,
+    .remove_device = arm_smmu_dt_remove_device_generic,
     .teardown = arm_smmu_iommu_domain_teardown,
     .iotlb_flush = arm_smmu_iotlb_flush,
     .iotlb_flush_all = arm_smmu_iotlb_flush_all,
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 98f2aa0..37f4945 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -127,6 +127,36 @@ int iommu_release_dt_devices(struct domain *d)
     return 0;
 }
 
+int iommu_remove_dt_device(struct dt_device_node *np)
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    struct device *dev = dt_to_dev(np);
+    int rc = 1;
+
+    if ( !ops )
+        return -EINVAL;
+
+    if ( iommu_dt_device_is_assigned(np) )
+        return -EPERM;
+
+    /*
+     * The driver which supports generic IOMMU DT bindings must have
+     * these callback implemented.
+     */
+    if ( !ops->remove_device )
+        return -EINVAL;
+
+    /*
+     * Remove master device from the IOMMU if latter is present and available.
+     */
+    rc = ops->remove_device(0, dev);
+
+    if ( rc == 0 )
+        iommu_fwspec_free(dev);
+
+    return rc;
+}
+
 int iommu_add_dt_device(struct dt_device_node *np)
 {
     const struct iommu_ops *ops = iommu_get_ops();
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 6b2cdff..c4d5d12 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -215,6 +215,8 @@ int iommu_release_dt_devices(struct domain *d);
  */
 int iommu_add_dt_device(struct dt_device_node *np);
 
+int iommu_remove_dt_device(struct dt_device_node *np);
+
 int iommu_do_dt_domctl(struct xen_domctl *, struct domain *,
                        XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176928.322136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYu-0005wr-NY; Thu, 02 Sep 2021 06:51:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176928.322136; Thu, 02 Sep 2021 06:51:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYt-0005sp-VH; Thu, 02 Sep 2021 06:50:59 +0000
Received: by outflank-mailman (input) for mailman id 176928;
 Thu, 02 Sep 2021 06:07:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLfsz-0004hL-EG
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:41 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.236.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 13554a0e-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:38 +0000 (UTC)
Received: from SN4PR0501CA0080.namprd05.prod.outlook.com
 (2603:10b6:803:22::18) by CY4PR02MB2181.namprd02.prod.outlook.com
 (2603:10b6:903:e::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:07:35 +0000
Received: from SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
 (2603:10b6:803:22:cafe::39) by SN4PR0501CA0080.outlook.office365.com
 (2603:10b6:803:22::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:35 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0029.mail.protection.outlook.com (10.97.4.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:34 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:14 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:14 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsY-000F6q-R6; Wed, 01 Sep 2021 23:07:14 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13554a0e-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DqkxgLDre73op25XKcR7jwvLLzEuiKnjoIesQcZcYYRcePW/Wczl6JfiC9Uir3gQK8nM3zNoTtErqKygYvEy1dkmH4H9s4dfeAcxub7lv7x/8yGC94r71oG++PKDB7197eywaey815bCUl1WDPiL8AKZgQForTwgwTew6DltJlL/GvTk59TdUROA7meHHXL0d3lvAienNWYmBxeeNvKJEwzNCesJMfJkCd0wPd22bYrW0rpilKaP9Dp0YrOpwS27MpaU4WA3ZWmtN2KLkmXp00ruvPiNHODynhRfxIUs5VHSzlRewe7EzZa2eajXfdcANqW+hcYQsQEiO2yS8zjMAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bk58T2PdmP0MpUXqDTQL+WHaYdm6dJQvCtJfKGAwTtY=;
 b=Gr3IkqVkZ3nbcF2g6AmzHCrwhD7TpaUL/J623YInRRQQwNuMt0oEvfHv5RSjQtxp3YhARmZ3vE0TTPOP7MKObZb4/07WxGKA915Kle7tovwA9WvaGcIFyFQvFp9h1oKBBd6PbzfsfeI5r+XzZjGW8aPlErba6h9U94tvttjLgmLyrV3+26LieVWrzMbj6GESn/dMMgvM3MLMeCxsanApdfTAgNQVUrVt8r8O+4U0He9qiA7a0XfqalW14nCi+lmvZ/Bf53OgWkHqnfDfsAGh5qEsg6TmhfOoJaiV9vHZzNYkARYxCAWTRmSj2JRtpbVSVKNX/B5QhCDefVSLvqeucg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bk58T2PdmP0MpUXqDTQL+WHaYdm6dJQvCtJfKGAwTtY=;
 b=mUirYEvbfTT77RlG81HQUzD6LadetmXz6xk4ERMiEy5FuyPHTdnz0h7q3CYy64U9pEAsjH9zCIHJfLSN+Tf/FuyXJKY8tprwYgxI8skLWFuHgbteNYu+lx/BMF59auKMGQqsEbFfG7K76I6T6OL/axyftPGhkQgq2eKeJDyjlQY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: [XEN][RFC PATCH 09/13] xen/arm: Implement device tree node removal functionalities
Date: Wed, 1 Sep 2021 23:05:59 -0700
Message-ID: <1630562763-390068-10-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dce2cf17-de68-461f-9cf7-08d96dd7f531
X-MS-TrafficTypeDiagnostic: CY4PR02MB2181:
X-Microsoft-Antispam-PRVS:
	<CY4PR02MB2181A55F46C21721B0AB6A5DBCCE9@CY4PR02MB2181.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:199;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IO8yxi2Y5dHAysq/2g414+Sg8E//eFoQU44RCwtQOiZRJ45iK2Z11EfoBqD38MwB5c6tln/ETy+B1062kNw8CAcMX+uISjGSF9J6aqN8Q/cchBpHfbvaiMQe+2zh8wDXcMXvcZDymRr3GnCkTFw1RcMAr0kR6mVqN9hz14BV3TVHAI+FJzjJTKlgWUYcVxBGecl7a+V/yWwBFEr9nJOqmRQpsPDbQGPhekageHEsJnBOKmMP181lmd1EPdHnk4E9L4ewggaOV2RbyoAkdh7Ke0nGgc6Ry+Fdle8lS2aAI1jzykBKyt0lctPLi/rRbw9VawYyEPmcLZ6/CnAEYbOTSZMboVV/lyGXSInFazTxOO4TV5O6PABU5BSj77jglhKQ7y+xZWgUk/qDeb2ReM/fEsUPW14IqA54VxYG0y+Ur+5phrSPCwu5bZEi997EHmThvGM9xWqK3lYZ/PCO4tqWn3dZaFobHt0jjwLGAEV3X0fkLc0thkUcT0YLmdw1fD2GWrMGdKSWxwH34Mziy2AlrtAba1hCoK5PJJ/sTwprTIld2ergybfXEc7IG+jdTrQiTpLrG51Ubu+yT54ov+p3J0K6RtBGfIN48j1wEXmCgtegRnQvPMf3b0i//posj6G+nNCMIVLT9wLAbDnITeaR+Z2AeRiE4sgemxCKg/m4LF0AVj8Jc5ecC/2IUqfyZXVjzYjGnk2fPxxEvohGqPshaojBjpLFjTKVIorgzprAnXjDGnZ/GSCPLlnDmOssbvQO4Q2+d2Uc5oZOPOHV6l5Ntw==
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(396003)(136003)(376002)(36840700001)(46966006)(82740400003)(36860700001)(36756003)(4326008)(6916009)(316002)(2616005)(36906005)(83380400001)(7636003)(8676002)(54906003)(47076005)(2906002)(70206006)(70586007)(6666004)(9786002)(5660300002)(7696005)(186003)(82310400003)(356005)(8936002)(26005)(478600001)(426003)(336012)(102446001)(403724002);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:34.7861
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dce2cf17-de68-461f-9cf7-08d96dd7f531
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR02MB2181

Introduce domctl XEN_DOMCTL_delfpga to remove a device-tree node added through
device tree overlay. Currently, this supports removing one node at a time.

DT node removal works with the following steps:
    1. finds a node with given path.
    2. Check if the node is used by any of dom0 or domus. It removes the node
only when it's not used by any domain.
    3. Removes IRQ permissions.
    4. Remove MMIO access.
    5. Find the node in dt_host and delete the device node entry from dt_host.
    6. Free the overlay_tracker node.

Also, added overlay_track struct to keep the track of added node through device
tree overlay. overlay_track has dt_host_new which is unflattened form of updated
fdt and name of overlay node.

When a node is removed, we also free the memory used by overlay_track for the
particular overlay node.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/arch/arm/domctl.c         | 183 ++++++++++++++++++++++++++++++++++++++++++
 xen/common/device_tree.c      |  51 ++++++++++++
 xen/include/public/domctl.h   |   9 +++
 xen/include/xen/device_tree.h |   1 +
 4 files changed, 244 insertions(+)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index b7d27f3..5986934 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -9,11 +9,34 @@
 #include <xen/hypercall.h>
 #include <xen/iocap.h>
 #include <xen/lib.h>
+#include <xen/list.h>
 #include <xen/mm.h>
 #include <xen/sched.h>
 #include <xen/types.h>
 #include <xsm/xsm.h>
 #include <public/domctl.h>
+#include <xen/xmalloc.h>
+#include <xen/device_tree.h>
+#include <asm/domain_build.h>
+
+/*
+ * overlay_node_track describes information about added nodes through dtbo.
+ * @dt_host_new: Pointer to the updated dt_host_new unflattened 'updated fdt'.
+ * @node_fullname: Store the name of nodes.
+ * @entry: List pointer.
+ */
+struct overlay_track {
+    struct list_head entry;
+    struct dt_device_node *dt_host_new;
+    /*
+     * TODO: We keep max nodes to 10 in an overlay. But for now we will be
+     * adding one node only.
+     */
+    char *node_fullname;
+};
+
+static LIST_HEAD(overlay_tracker);
+static DEFINE_SPINLOCK(overlay_lock);
 
 void arch_get_domain_info(const struct domain *d,
                           struct xen_domctl_getdomaininfo *info)
@@ -45,6 +68,132 @@ static int handle_vuart_init(struct domain *d,
     return rc;
 }
 
+/*
+ * First finds the device node to remove. Check if the device is being used by
+ * any dom and finally remove it from dt_host. IOMMU is already being taken care
+ * while destroying the domain.
+ */
+static long handle_del_fpga_nodes(char *full_dt_node_path)
+{
+    struct domain *d = hardware_domain;
+    int rc = 0;
+    uint32_t ret = 0;
+    struct dt_device_node *fpga_device;
+    struct overlay_track *entry, *temp;
+    unsigned int naddr;
+    unsigned int i, nirq;
+    struct dt_raw_irq rirq;
+    u64 addr, size;
+
+    fpga_device = dt_find_node_by_path(full_dt_node_path);
+
+    if ( fpga_device == NULL )
+    {
+        printk(XENLOG_G_ERR "Device %s is not present in the tree\n",
+               full_dt_node_path);
+        return -EINVAL;
+    }
+
+    ret = dt_device_used_by(fpga_device);
+
+    if ( ret != 0 && ret != DOMID_IO )
+    {
+        printk(XENLOG_G_ERR "Cannot remove the device as it is being used by"
+               "domain %d\n", ret);
+        return -EPERM;
+    }
+
+    spin_lock(&overlay_lock);
+
+    nirq = dt_number_of_irq(fpga_device);
+
+    /* Remove IRQ permission */
+    for ( i = 0; i < nirq; i++ )
+    {
+        rc = dt_device_get_raw_irq(fpga_device, i, &rirq);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
+                   i, dt_node_full_name(fpga_device));
+            goto out;
+        }
+
+        rc = platform_get_irq(fpga_device, i);
+        if ( rc < 0 )
+        {
+            printk(XENLOG_ERR "Unable to get irq %u for %s\n",
+                   i, dt_node_full_name(fpga_device));
+            goto out;
+        }
+
+        rc = irq_deny_access(d, rc);
+
+        if ( rc )
+        {
+            printk(XENLOG_ERR "unable to revoke access for irq %u for %s\n",
+                   i, dt_node_full_name(fpga_device));
+            goto out;
+        }
+    }
+
+    rc = iommu_remove_dt_device(fpga_device);
+
+    if ( rc )
+        goto out;
+
+    naddr = dt_number_of_address(fpga_device);
+
+    /* Remove mmio access. */
+    for ( i = 0; i < naddr; i++ )
+    {
+        rc = dt_device_get_address(fpga_device, i, &addr, &size);
+
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                   i, dt_node_full_name(fpga_device));
+            goto out;
+        }
+
+        rc = iomem_deny_access(d, paddr_to_pfn(addr),
+                               paddr_to_pfn(PAGE_ALIGN(addr + size - 1)));
+
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Unable to remove dom%d access to"
+                    " 0x%"PRIx64" - 0x%"PRIx64"\n",
+                    d->domain_id,
+                    addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+            goto out;
+        }
+    }
+
+    rc = fpga_del_node(fpga_device);
+
+    if ( rc )
+        goto out;
+
+    list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+    {
+        if ( (strcmp(full_dt_node_path, entry->node_fullname) == 0) )
+        {
+            list_del(&entry->entry);
+            xfree(entry->node_fullname);
+            xfree(entry->dt_host_new);
+            xfree(entry);
+            goto out;
+        }
+    }
+
+    printk(XENLOG_G_ERR "Cannot find the node in tracker. Memory will not"
+           "be freed\n");
+    rc = -ENOENT;
+
+out:
+    spin_unlock(&overlay_lock);
+    return rc;
+}
+
 long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
                     XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
 {
@@ -173,6 +322,40 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
 
         return rc;
     }
+
+    case XEN_DOMCTL_delfpga:
+    {
+        char *full_dt_node_path;
+        int rc;
+
+        if ( domctl->u.fpga_del_dt.size > 0 )
+            full_dt_node_path = xmalloc_bytes(domctl->u.fpga_del_dt.size);
+        else
+            return -EINVAL;
+
+        if ( full_dt_node_path == NULL )
+            return -ENOMEM;
+
+        rc = copy_from_guest(full_dt_node_path,
+                             domctl->u.fpga_del_dt.full_dt_node_path,
+                             domctl->u.fpga_del_dt.size);
+        if ( rc )
+        {
+            gprintk(XENLOG_ERR, "copy from guest failed\n");
+            xfree(full_dt_node_path);
+
+            return -EFAULT;
+        }
+
+        full_dt_node_path[domctl->u.fpga_del_dt.size - 1] = '\0';
+
+        rc = handle_del_fpga_nodes(full_dt_node_path);
+
+        xfree(full_dt_node_path);
+
+        return rc;
+    }
+
     default:
     {
         int rc;
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 4946e83..04f2578 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -324,6 +324,57 @@ void dt_print_node_names(struct dt_device_node *dt)
     return;
 }
 
+int fpga_del_node(struct dt_device_node *device_node)
+{
+    struct dt_device_node *np;
+    struct dt_device_node *parent_node;
+    struct dt_device_node *current_node;
+
+    parent_node = device_node->parent;
+
+    current_node = parent_node;
+
+    if ( parent_node == NULL )
+    {
+        dt_dprintk("%s's parent node not found\n", device_node->name);
+        return -EFAULT;
+    }
+
+    np = parent_node->child;
+
+    if ( np == NULL )
+    {
+        dt_dprintk("parent node %s's not found\n", parent_node->name);
+        return -EFAULT;
+    }
+
+    /* If node to be removed is only child node or first child. */
+    if ( np->name == device_node->name )
+    {
+        current_node->allnext = np->next;
+        return 0;
+    }
+
+    for ( np = parent_node->child; np->sibling != NULL; np = np->sibling )
+    {
+        current_node = np;
+        if ( np->sibling->name == device_node->name )
+        {
+            /* Found the node. Now we remove it. */
+            current_node->allnext = np->allnext->allnext;
+
+            if ( np->sibling->sibling )
+                current_node->sibling = np->sibling->sibling;
+            else
+                current_node->sibling = NULL;
+
+            break;
+        }
+    }
+
+    return 0;
+}
+
 int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
                           struct dt_device_node **node)
 {
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3..b1b8efd 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1169,6 +1169,13 @@ struct xen_domctl_vmtrace_op {
 typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
 
+/* XEN_DOMCTL_fpga_del. */
+struct xen_domctl_fpga_del_dt {
+    XEN_GUEST_HANDLE_64(char) full_dt_node_path;
+    uint32_t size;
+};
+
+
 struct xen_domctl {
     uint32_t cmd;
 #define XEN_DOMCTL_createdomain                   1
@@ -1254,6 +1261,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_get_cpu_policy                82
 #define XEN_DOMCTL_set_cpu_policy                83
 #define XEN_DOMCTL_vmtrace_op                    84
+#define XEN_DOMCTL_delfpga                      86
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1315,6 +1323,7 @@ struct xen_domctl {
         struct xen_domctl_psr_alloc         psr_alloc;
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
+        struct xen_domctl_fpga_del_dt       fpga_del_dt;
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 7cc6093..eb7f645 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -496,6 +496,7 @@ int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
  * Prints all node names.
  */
 void dt_print_node_names(struct dt_device_node *dt);
+int fpga_del_node(struct dt_device_node *device_node);
 
 /**
  * dt_get_parent - Get a node's parent if any
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176930.322145 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYv-0006BB-CL; Thu, 02 Sep 2021 06:51:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176930.322145; Thu, 02 Sep 2021 06:51:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYu-000669-Pc; Thu, 02 Sep 2021 06:51:00 +0000
Received: by outflank-mailman (input) for mailman id 176930;
 Thu, 02 Sep 2021 06:07:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLft4-0004hL-EK
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:46 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com (unknown
 [40.107.93.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1425e966-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:40 +0000 (UTC)
Received: from SN4PR0501CA0073.namprd05.prod.outlook.com
 (2603:10b6:803:22::11) by BL0PR02MB4721.namprd02.prod.outlook.com
 (2603:10b6:208:59::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:07:38 +0000
Received: from SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
 (2603:10b6:803:22:cafe::e9) by SN4PR0501CA0073.outlook.office365.com
 (2603:10b6:803:22::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.11 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:38 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0029.mail.protection.outlook.com (10.97.4.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:38 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:17 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:17 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsb-000F6q-4A; Wed, 01 Sep 2021 23:07:17 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1425e966-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Te3CLIQwSSYtGGBlbzgNjB+UBGrhjIq38k9KB7AOSVEw/gPnlTGnHTH852/N/X7gaX5SG46VL9qkBWskLRdlMgk22CMaN3nSRlw7vY7agl7h69FaIdZx0l3BNIy6IDFC7AycymdJKo/B9SA8AS2xj45WLx7C4bCRQdXWGvwPezaeHxgIzQDBA7cz+YOENgQTuF4ywnBRySDg5jdP3feB4i44GkxbpiJAsJEQ8tYdLNorCfzutV6G9Lxp4GsdMimRQu41Bvj2iaIwRmmiY08u1OYaY1puN/yBaQzUrSbHA0hRtvja/Pp7xR0LPKiUWlm/FKB+OZAhGqP6bxsEPmIR/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PuN2b2H12aJwTeXjwq0M4sE74WKCTLZyi/JgSK4cHCM=;
 b=EaqN0tOBrpnsA6ddF4D8tgIRkFZgE0U6CGwWzSSAdleOzxyTKEsVTzNTcDIhC+k/atll/tzLaeAOEkZErvNO1kfaT70ONMdPGCCTzAeyZuHl7+z8RLUDZaC8MWURa7hMdelENdcF1cC7RxJukQUDIsBW8xJNWD41ERi7i2KNgeOTGOeeIaBpoeD4W+/wD941V2PVoWwRgIK/sBqxnUOOjAz0DPmGEL+cqnnaDAAzYbO8hJw2g7A8NMdio8Ud/Xmq2VmlgR72RvAgg9DMNxmaofCvLrJBU71ZYARIAffj+ngE5VQ3bGZuKz6dmYj8DMb+mDHbHEO2KsK2HxSigj26UA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PuN2b2H12aJwTeXjwq0M4sE74WKCTLZyi/JgSK4cHCM=;
 b=hkwddA76I8tqtEvqpBsYNlYYwVHbCraLiYVXg6Z7D79GDPw64Q1cKYMKkaN+vov4YExUUlIsMTb3JA9kZuSJ6Soz3OaM1TN+M8NpV6ns5XBLEsYtJe02khQd+bV4ywA8luHBgFVNInkX8sLDh6Xpv5pFTFer8bnXSA07NolVq3U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: [XEN][RFC PATCH 10/13] xen/arm: Implement device tree node addition functionalities
Date: Wed, 1 Sep 2021 23:06:00 -0700
Message-ID: <1630562763-390068-11-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 291affca-15c1-4473-a81f-08d96dd7f72b
X-MS-TrafficTypeDiagnostic: BL0PR02MB4721:
X-Microsoft-Antispam-PRVS:
	<BL0PR02MB47217FD248B5BDB39FB95FF7BCCE9@BL0PR02MB4721.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:366;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0h6eZwWpeRoOmH8M/+P3RMble3LIBuH/jz0oYTE0UXq7ZqgCDd2SyiFG/EBxNy4duillUNfdkMFi4VLSZFB8BjkRTMvfQFkHifKB0UnEqgLnlFOwz7x5FFSzIyyL5kBxDIcAeQ8Efx8bfzJa9FqTQO1b3tFqyP0JCk0OJjzFnAJzqYI0CIHVxmnIcfyRhqisSunIOo1BFSc6Ys3A9Fo/XgL6cf1LvpGjCsDHiGxp8QKwIPCdIbrjinno3sxVvrDFjoEOhDV62ng9IZffiZib1QnUICSNoLB+mMSpBxY/pGyxKs0OLxc8oD6kEoybh8z4QED8bo8jXJ+SGQ8pFPCV/M1DsHAcdltf7qAMxL5tBfZYFnHG+o+LgVsn+PDhHYjD/GcfExjDka/u3wq7TQIUQPvZ2u0C9bgMYfyInrJKTk7ng4G0/BcG5Qj1yYNILcV/z33/AdwNowahz2Zd0rsPw+kwAAFhT+Y87j9pqr+EU8bfSFCGiuBtfCVe2Cpdy4GP43vN2AoArGDhIiA+JzcuCOmRYH3loNGHXK8bm50xae7BqOb2/2pNpyacj7tjPbFQxvemvT2ebzM1x0aGmqip0PqDHGHwSVsEfwpd5htY/yTm9w9ka66IEvVxgVVv17mLUKMfmUGKoPQ//JtWUIZFVwDa0XL0WCkrGdE7TBrHp9lK8zJC+sVYCtJhY6ZXfLaQFmyoUG8qyQExTC1VjK5M/WCeGCH9qzdW1/nu621QgmM89h19U6ug4UQruy5SeKSieeBOITb5Iu3VJgHh3WcI3w==
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(39860400002)(376002)(136003)(346002)(396003)(46966006)(36840700001)(4326008)(7696005)(2616005)(186003)(478600001)(36860700001)(356005)(83380400001)(7636003)(47076005)(30864003)(5660300002)(6666004)(36906005)(6916009)(54906003)(8676002)(2906002)(36756003)(8936002)(426003)(70586007)(82740400003)(70206006)(9786002)(316002)(336012)(82310400003)(26005)(102446001)(403724002);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:38.1075
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 291affca-15c1-4473-a81f-08d96dd7f72b
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR02MB4721

Introduce domctl XEN_DOMCTL_addfpga to add a device-tree node through device
tree overlay. This works with a device tree overlay(.dtbo) as input.

Add check_pfdt() to do sanity check on the dtbo.

Also, added overlay_get_node_info() to get the node's full name with path. This
comes handy when checking node for duplication.

Each time a overlay node is added, a new fdt(memcpy of device_tree_flattened) is
created and updated with overlay node. This updated fdt is further unflattened
to a dt_host_new. Next, it checks if overlay node already exists in the dt_host.
If overlay node doesn't exist then find the overlay node in dt_host_new, find
the overlay node's parent in dt_host and add the node as child under parent in
the dt_host. The node is attached as the last node under target parent.

Finally, add IRQs, add device to IOMMUs, set permissions and map MMIO for the
overlay node.

When a node is added using overlay, a new entry is allocated in the
overlay_track to keep the track of memory allocation due to addition of overlay
node. This is helpful for freeing the memory allocated when a device tree node
is removed with domctl XEN_DOMCTL_delfpga domctl.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 xen/arch/arm/domctl.c         | 262 ++++++++++++++++++++++++++++++++++++++++++
 xen/common/device_tree.c      |  54 +++++++++
 xen/include/public/domctl.h   |   7 ++
 xen/include/xen/device_tree.h |   1 +
 4 files changed, 324 insertions(+)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index 5986934..0ac635f 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -15,6 +15,8 @@
 #include <xen/types.h>
 #include <xsm/xsm.h>
 #include <public/domctl.h>
+/* Included for FPGA dt add. */
+#include <xen/libfdt/libfdt.h>
 #include <xen/xmalloc.h>
 #include <xen/device_tree.h>
 #include <asm/domain_build.h>
@@ -68,6 +70,61 @@ static int handle_vuart_init(struct domain *d,
     return rc;
 }
 
+static int check_pfdt(void *pfdt, uint32_t pfdt_size)
+{
+    if ( fdt_totalsize(pfdt) != pfdt_size )
+    {
+        printk(XENLOG_ERR "Partial FDT is not a valid Flat Device Tree\n");
+        return -EFAULT;
+    }
+
+    if ( fdt_check_header(pfdt) )
+    {
+        printk(XENLOG_ERR "Partial FDT is not a valid Flat Device Tree\n");
+        return -EFAULT;
+    }
+
+    return 0;
+}
+
+static void overlay_get_node_info(void *fdto, char *node_full_path)
+{
+    int fragment;
+
+    /*
+     * Handle overlay nodes. But for now we are just handling one node.
+     */
+    fdt_for_each_subnode(fragment, fdto, 0)
+    {
+        int target;
+        int overlay;
+        int subnode;
+        const char *target_path;
+
+        target = overlay_get_target(device_tree_flattened, fdto, fragment,
+                                    &target_path);
+        overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+
+        fdt_for_each_subnode(subnode, fdto, overlay)
+        {
+            const char *node_name = fdt_get_name(fdto, subnode, NULL);
+            int node_name_len = strlen(node_name);
+            int target_path_len = strlen(target_path);
+
+            memcpy(node_full_path, target_path, target_path_len);
+
+            node_full_path[target_path_len] = '/';
+
+            memcpy(node_full_path + target_path_len + 1, node_name,
+                   node_name_len);
+
+            node_full_path[target_path_len + 1 + node_name_len] = '\0';
+
+            return;
+        }
+    }
+}
+
 /*
  * First finds the device node to remove. Check if the device is being used by
  * any dom and finally remove it from dt_host. IOMMU is already being taken care
@@ -194,6 +251,181 @@ out:
     return rc;
 }
 
+/*
+ * Adds only one device node at a time under target node.
+ * We use dt_host_new to unflatten the updated device_tree_flattened. This is
+ * done to avoid the removal of device_tree generation, iomem regions mapping to
+ * DOM0 done by handle_node().
+ */
+static long handle_add_fpga_overlay(void *pfdt, uint32_t pfdt_size)
+{
+    int rc = 0;
+    struct dt_device_node *fpga_node;
+    char node_full_path[128];
+    void *fdt = xmalloc_bytes(fdt_totalsize(device_tree_flattened));
+    struct dt_device_node *dt_host_new;
+    struct domain *d = hardware_domain;
+    struct overlay_track *tr = NULL;
+    int node_full_path_namelen;
+    unsigned int naddr;
+    unsigned int i;
+    u64 addr, size;
+
+    if ( fdt == NULL )
+        return ENOMEM;
+
+    spin_lock(&overlay_lock);
+
+    memcpy(fdt, device_tree_flattened, fdt_totalsize(device_tree_flattened));
+
+    rc = check_pfdt(pfdt, pfdt_size);
+
+    if ( rc )
+        goto err;
+
+    overlay_get_node_info(pfdt, node_full_path);
+
+    rc = fdt_overlay_apply(fdt, pfdt);
+
+    if ( rc )
+    {
+        printk(XENLOG_ERR "Adding overlay node %s failed with error %d\n",
+               node_full_path, rc);
+        goto err;
+    }
+
+    /* Check if node already exists in dt_host. */
+    fpga_node = dt_find_node_by_path(node_full_path);
+
+    if ( fpga_node != NULL )
+    {
+        printk(XENLOG_ERR "node %s exists in device tree\n", node_full_path);
+        rc = -EINVAL;
+        goto err;
+    }
+
+    /* Unflatten the fdt into a new dt_host. */
+    unflatten_device_tree(fdt, &dt_host_new);
+
+    /* Find the newly added node in dt_host_new by it's full path. */
+    fpga_node = _dt_find_node_by_path(dt_host_new, node_full_path);
+
+    if ( fpga_node == NULL )
+    {
+        dt_dprintk("%s node not found\n", node_full_path);
+        rc = -EFAULT;
+        xfree(dt_host_new);
+        goto err;
+    }
+
+    /* Just keep the node we intend to add. Remove every other node in list. */
+    fpga_node->allnext = NULL;
+    fpga_node->sibling = NULL;
+
+    /* Add the node to dt_host. */
+    rc = fpga_add_node(fpga_node, fpga_node->parent->full_name);
+
+    if ( rc )
+    {
+        /* Node not added in dt_host. Safe to free dt_host_new. */
+        xfree(dt_host_new);
+        goto err;
+    }
+
+    /* Get the node from dt_host and add interrupt and IOMMUs. */
+    fpga_node = dt_find_node_by_path(fpga_node->full_name);
+
+    if ( fpga_node == NULL )
+    {
+        /* Sanity check. But code will never come in this loop. */
+        printk(XENLOG_ERR "Cannot find %s node under updated dt_host\n",
+               fpga_node->name);
+        goto remove_node;
+    }
+
+    /* First let's handle the interrupts. */
+    rc = handle_device_interrupts(d, fpga_node, false);
+
+    if ( rc )
+    {
+        printk(XENLOG_G_ERR "Interrupt failed\n");
+        goto remove_node;
+    }
+
+    /* Add device to IOMMUs */
+    rc = iommu_add_dt_device(fpga_node);
+
+    if ( rc < 0 )
+    {
+        printk(XENLOG_G_ERR "Failed to add %s to the IOMMU\n",
+               dt_node_full_name(fpga_node));
+        goto remove_node;
+    }
+
+    /* Set permissions. */
+    naddr = dt_number_of_address(fpga_node);
+
+    dt_dprintk("%s passthrough = %d naddr = %u\n",
+               dt_node_full_name(fpga_node), false, naddr);
+
+    /* Give permission and map MMIOs */
+    for ( i = 0; i < naddr; i++ )
+    {
+        struct map_range_data mr_data = { .d = d, .p2mt = p2m_mmio_direct_c };
+        rc = dt_device_get_address(fpga_node, i, &addr, &size);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                   i, dt_node_full_name(fpga_node));
+            goto remove_node;
+        }
+
+        rc = map_range_to_domain(fpga_node, addr, size, &mr_data);
+        if ( rc )
+            goto remove_node;
+    }
+
+    /* This will happen if everything above goes right. */
+    tr = xzalloc(struct overlay_track);
+    tr->dt_host_new = dt_host_new;
+    node_full_path_namelen = strlen(node_full_path);
+    tr->node_fullname = xmalloc_bytes(node_full_path_namelen + 1);
+
+    if ( tr->node_fullname == NULL )
+    {
+        rc = -ENOMEM;
+        goto remove_node;
+    }
+
+    memcpy(tr->node_fullname, node_full_path, node_full_path_namelen);
+    tr->node_fullname[node_full_path_namelen] = '\0';
+
+    INIT_LIST_HEAD(&tr->entry);
+    list_add_tail(&tr->entry, &overlay_tracker);
+
+err:
+    spin_unlock(&overlay_lock);
+    xfree(fdt);
+    return rc;
+
+/*
+ * Failure case. We need to remove the node, free tracker(if tr exists) and
+ * dt_host_new. As the tracker is not in list yet so it doesn't get freed in
+ * handle_del_fpga_nodes() and due to that dt_host_new will not get freed so we
+ * we free tracker and dt_host_new here.
+ */
+remove_node:
+    spin_unlock(&overlay_lock);
+    handle_del_fpga_nodes(node_full_path);
+    xfree(dt_host_new);
+
+    if ( tr )
+        xfree(tr);
+
+    xfree(fdt);
+    return rc;
+}
+
 long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
                     XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
 {
@@ -323,6 +555,36 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
         return rc;
     }
 
+    case XEN_DOMCTL_addfpga:
+    {
+        void *pfdt;
+        int rc;
+
+        if ( domctl->u.fpga_add_dt.pfdt_size > 0 )
+            pfdt = xmalloc_bytes(domctl->u.fpga_add_dt.pfdt_size);
+        else
+            return -EINVAL;
+
+        if ( pfdt == NULL )
+            return -ENOMEM;
+
+        rc = copy_from_guest(pfdt, domctl->u.fpga_add_dt.pfdt,
+                             domctl->u.fpga_add_dt.pfdt_size);
+        if ( rc )
+        {
+            gprintk(XENLOG_ERR, "copy from guest failed\n");
+            xfree(pfdt);
+
+            return -EFAULT;
+        }
+
+        rc = handle_add_fpga_overlay(pfdt, domctl->u.fpga_add_dt.pfdt_size);
+
+        xfree(pfdt);
+
+        return rc;
+    }
+
     case XEN_DOMCTL_delfpga:
     {
         char *full_dt_node_path;
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 04f2578..d062c17 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -324,6 +324,60 @@ void dt_print_node_names(struct dt_device_node *dt)
     return;
 }
 
+int fpga_add_node(struct dt_device_node *fpga_node,
+                  const char *parent_node_path)
+{
+    struct dt_device_node *parent_node;
+    struct dt_device_node *np;
+    struct dt_device_node *next_node;
+    struct dt_device_node *new_node;
+
+    parent_node = dt_find_node_by_path(parent_node_path);
+
+    new_node = fpga_node;
+
+    if ( new_node == NULL )
+        return -EINVAL;
+
+    if ( parent_node == NULL )
+    {
+        dt_dprintk("Node not found. Partial dtb will not be added");
+        return -EINVAL;
+    }
+
+    /*
+     * If node is found. We can attach the fpga_node as a child of the
+     * parent node.
+     */
+
+    for ( np = parent_node->child; np->sibling != NULL; np = np->sibling )
+    {
+    }
+
+    /*
+     * Before attaching also check if the parent node of fpga_node is also
+     * same named as parent.
+     */
+    next_node = np->allnext;
+
+    new_node->parent = parent_node;
+    np->sibling = new_node;
+    np->allnext = new_node;
+
+    /*
+     * Reach at the end of fpga_node.
+     * TODO: Remove this loop as we are just adding one node for now.
+     */
+    for ( np = new_node; np->allnext != NULL; np = np->allnext )
+    {
+    }
+
+    /* Now plug next_node at the end of fpga_node. */
+    np->allnext = next_node;
+
+    return 0;
+}
+
 int fpga_del_node(struct dt_device_node *device_node)
 {
     struct dt_device_node *np;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index b1b8efd..ce4667e 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1175,6 +1175,11 @@ struct xen_domctl_fpga_del_dt {
     uint32_t size;
 };
 
+/* XEN_DOMCTL_fpga_add. */
+struct xen_domctl_fpga_add_dt {
+    XEN_GUEST_HANDLE_64(void) pfdt;
+    uint32_t pfdt_size;  /* Partial dtb size. */
+};
 
 struct xen_domctl {
     uint32_t cmd;
@@ -1261,6 +1266,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_get_cpu_policy                82
 #define XEN_DOMCTL_set_cpu_policy                83
 #define XEN_DOMCTL_vmtrace_op                    84
+#define XEN_DOMCTL_addfpga                      85
 #define XEN_DOMCTL_delfpga                      86
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
@@ -1323,6 +1329,7 @@ struct xen_domctl {
         struct xen_domctl_psr_alloc         psr_alloc;
         struct xen_domctl_vuart_op          vuart_op;
         struct xen_domctl_vmtrace_op        vmtrace_op;
+        struct xen_domctl_fpga_add_dt       fpga_add_dt;
         struct xen_domctl_fpga_del_dt       fpga_del_dt;
         uint8_t                             pad[128];
     } u;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index eb7f645..4c8dec6 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -496,6 +496,7 @@ int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
  * Prints all node names.
  */
 void dt_print_node_names(struct dt_device_node *dt);
+int fpga_add_node(struct dt_device_node *fpga_node, const char *parent_node);
 int fpga_del_node(struct dt_device_node *device_node);
 
 /**
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176932.322158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYw-0006WX-IT; Thu, 02 Sep 2021 06:51:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176932.322158; Thu, 02 Sep 2021 06:51:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYv-0006R6-Rs; Thu, 02 Sep 2021 06:51:01 +0000
Received: by outflank-mailman (input) for mailman id 176932;
 Thu, 02 Sep 2021 06:07:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLft9-0004hL-Ea
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:07:51 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (unknown
 [40.107.96.81]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 17559226-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:07:45 +0000 (UTC)
Received: from SN4PR0501CA0083.namprd05.prod.outlook.com
 (2603:10b6:803:22::21) by DM6PR02MB6716.namprd02.prod.outlook.com
 (2603:10b6:5:218::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Thu, 2 Sep
 2021 06:07:43 +0000
Received: from SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
 (2603:10b6:803:22:cafe::de) by SN4PR0501CA0083.outlook.office365.com
 (2603:10b6:803:22::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.12 via Frontend
 Transport; Thu, 2 Sep 2021 06:07:42 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 SN1NAM02FT0029.mail.protection.outlook.com (10.97.4.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:42 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:22 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:22 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsg-000F6q-Dd; Wed, 01 Sep 2021 23:07:22 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17559226-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LnC/zNap3NmjyBziOpMvDWQXaPdOpD89oEQX+ubKVltND/+dxXMhbCEef2RIk5fspdbgiS9gDPtZQq3zCITQVgOVF1uZiVBRGPBF7wpnaJTpPeHrFM6OkggsLyVJXDrhtYW2hrXmePDB4S66IzUgNe1UyNA2WdXvOQQLI4APdUegOtGSTURqS7M+l4rabkrgmWs2KasRCUHP8uTwnNkhmUMw8levOdkR+5/ef3tXxyRGteEuIiiPxpHccPEQ0mrrxSu5Lf1S0P9rmY1nIWJnpDRWxnOupSWTCQf0yuuFfW6LcczPQwRCjIjtYlwpFaunZxeeisESEJ6n96k94KLBag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=U7bgcXdJTXoKYKNKTamQfYEjcYOoItL0nadg/TeJWjc=;
 b=mCyKJMgU9KJR2RtMojV/Nq0IeDvxGu8cciiDV6vgXxXvjrWj2YRRx7I2AzY38N1UdzdvOgh4xi6nk9LpDA+IMz7IU+v6u2krT0D5+nlafEs9Mlc/thIRB9/f+HueuSPg/MzfghNoCoH0BQXn+HvbqziYrlX7dBED9WI6WNok43sUhExApe4rGUy5ruW0DJcToXlawTIBdQDNJOZ6pNpuml+IciSWp+0ITRqq2I6lQBShh0erdZVReobp6e9oRS8JH0PCdrz9jxTXADdWK/DuvCe9dt2eO2pZm51SeD2FwORtIOlMoj07k1gPrdl9ZkfVklibdDbwKiS/FzLqT9VnPQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=U7bgcXdJTXoKYKNKTamQfYEjcYOoItL0nadg/TeJWjc=;
 b=ZG9hLXS8uar1ahAzaZbhU+0vOk/tGHggI3e0eGYbEHfIYivRIYEWMUhYiJ9hbCFMovIqQCluPtHSXTRMAzhns7Vc8PT+WQtJ+jrN1lll1hYQl2ifrGQIHF+Z3W+pE3/s0ztIMnMN5dMMaoDB8xO7KnPWrDnWmzMjC2PiSCYk87c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>
Subject: [XEN][RFC PATCH 12/13] tools/libs/light: Implement new libxl functions for fpga-add and fpga-del
Date: Wed, 1 Sep 2021 23:06:02 -0700
Message-ID: <1630562763-390068-13-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 486282e8-4a79-441f-0264-08d96dd7f9cd
X-MS-TrafficTypeDiagnostic: DM6PR02MB6716:
X-Microsoft-Antispam-PRVS:
	<DM6PR02MB6716778FE694B490CB32CA89BCCE9@DM6PR02MB6716.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7da38gZlNn3e0k98mCCgPkePjD0Y6Mmg0n0YzpnVhA0UlfuB5PtLftDi2EmxPx5t2ro6pNFEXqnNkkCuwKgzDXPCcxuP1NQyaLEa8ErqZ9iRkiuY3Ru/vGcOe9xBKTySCWC2jPD8fVxCnIejAHeTXFhe5Zu7LhTJPAslvMCEOMpCuiqF02LrUaQFBAc06+i5E8dtctC7wPCXS2VMh0HOcpkloSw7Zov9gZFEYKc5ZtjpC8YJ5OhFVHPu/n0fkee9oel19yvc/bWYJQ8bOJwaJRRmrkDu5aQTERE+/0ujwxzwhs/OO97SWl4ryH5dgZfiDB6/iwxsDtMi2BtK1mzxH/Ri91eBsA1UUY7NC6ZiBySz30zgJJznLRtqvX0CPlWKLfE/vl8NqfWnoJUioVY9Ya6PSfz8VhsRByWdoF6B9PtbyDzEGL5eeMNaPyGC7LZwq/PKJkM1+DPNyUInV1dWmmteoA6KmE1g8Cv0DQSgjd0iAstUEPIgJQIH6K8JJCb74AlYyaP4QTpJgXq1j0F9e/vyJbavJaZBGsYQbzMpSn9cSvIeao19phWmnSuthBGWkPTRWHbjMuhFJldRoo/hjmn2xPT1bHJ5TbmTVFyINAGBDRa+cLHf6wE7gJL/8QbP+q2edzuWDNuHBamFm2RKM9eTGDaWXqJMImXayVfl8p6iYd/UZDAo/GPZw0F4IFtifKL+xW3y56Ldghk2SNF8Q62oJmzgOArPmE1rQTWg0D2NpE3qqGoclTPp1VXZ6V2X
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(39860400002)(346002)(396003)(376002)(136003)(36840700001)(46966006)(82310400003)(70206006)(70586007)(5660300002)(36756003)(8676002)(9786002)(36860700001)(8936002)(47076005)(478600001)(4326008)(54906003)(82740400003)(316002)(2906002)(36906005)(356005)(7636003)(336012)(186003)(2616005)(426003)(6666004)(7696005)(26005)(6916009)(102446001)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:42.5194
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 486282e8-4a79-441f-0264-08d96dd7f9cd
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1NAM02FT0029.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB6716

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 tools/include/libxl.h         |  5 +++
 tools/libs/light/Makefile     |  1 +
 tools/libs/light/libxl_fpga.c | 73 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 79 insertions(+)
 create mode 100644 tools/libs/light/libxl_fpga.c

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d..896cbcf 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -2386,6 +2386,11 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
                                         int *num);
 void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
+/* FPGA device add. */
+int libxl_add_fpga_node(libxl_ctx *ctx, void *pfdt, int pfdt_size);
+/* FPGA device remove. */
+int libxl_del_fpga_node(libxl_ctx *ctx, char *full_dt_node_path);
+
 /*
  * Turns the current process into a backend device service daemon
  * for a driver domain.
diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 7d8c51d..b17d4a6 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -115,6 +115,7 @@ SRCS-y += libxl_genid.c
 SRCS-y += _libxl_types.c
 SRCS-y += libxl_flask.c
 SRCS-y += _libxl_types_internal.c
+SRCS-y += libxl_fpga.o
 
 ifeq ($(CONFIG_LIBNL),y)
 CFLAGS_LIBXL += $(LIBNL3_CFLAGS)
diff --git a/tools/libs/light/libxl_fpga.c b/tools/libs/light/libxl_fpga.c
new file mode 100644
index 0000000..a33d00f
--- /dev/null
+++ b/tools/libs/light/libxl_fpga.c
@@ -0,0 +1,73 @@
+/*
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+#include "libxl_internal.h"
+#include <libfdt.h>
+#include <xenguest.h>
+#include <xenctrl.h>
+
+static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
+{
+    int r;
+
+    if (fdt_magic(fdt) != FDT_MAGIC) {
+        LOG(ERROR, "Partial FDT is not a valid Flat Device Tree");
+        return ERROR_FAIL;
+    }
+
+    r = fdt_check_header(fdt);
+    if (r) {
+        LOG(ERROR, "Failed to check the partial FDT (%d)", r);
+        return ERROR_FAIL;
+    }
+
+    if (fdt_totalsize(fdt) > size) {
+        LOG(ERROR, "Partial FDT totalsize is too big");
+        return ERROR_FAIL;
+    }
+
+    return 0;
+}
+
+int libxl_add_fpga_node(libxl_ctx *ctx, void *pfdt, int pfdt_size)
+{
+    int rc = 0;
+    GC_INIT(ctx);
+
+    if (check_partial_fdt(gc, pfdt, pfdt_size)) {
+        LOG(ERROR, "Partial DTB check failed\n");
+        return ERROR_FAIL;
+    } else
+        LOG(DEBUG, "Partial DTB check passed\n");
+
+    /* We don't need to do  xc_interface_open here. */
+    rc = xc_domain_add_fpga(ctx->xch, pfdt, pfdt_size);
+
+    if (rc)
+        LOG(ERROR, "%s: Adding partial dtb failed.\n", __func__);
+
+    return rc;
+}
+
+int libxl_del_fpga_node(libxl_ctx *ctx, char *device_path)
+{
+    int rc = 0;
+
+    /* We don't need to do  xc_interface_open here. */
+    rc = xc_domain_del_fpga(ctx->xch, device_path);
+
+    return rc;
+}
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176934.322169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYx-0006nJ-Pj; Thu, 02 Sep 2021 06:51:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176934.322169; Thu, 02 Sep 2021 06:51:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYw-0006gN-R6; Thu, 02 Sep 2021 06:51:02 +0000
Received: by outflank-mailman (input) for mailman id 176934;
 Thu, 02 Sep 2021 06:08:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLftL-0004km-3p
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:08:03 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.236.50]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 21108f96-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:08:01 +0000 (UTC)
Received: from BN1PR14CA0017.namprd14.prod.outlook.com (2603:10b6:408:e3::22)
 by DM6PR02MB4043.namprd02.prod.outlook.com (2603:10b6:5:9f::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 06:08:00 +0000
Received: from BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
 (2603:10b6:408:e3:cafe::64) by BN1PR14CA0017.outlook.office365.com
 (2603:10b6:408:e3::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 06:08:00 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 BN1NAM02FT024.mail.protection.outlook.com (10.13.2.138) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:07:59 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:21 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:21 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsf-000F6q-32; Wed, 01 Sep 2021 23:07:21 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21108f96-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wha1qfH0ZPATby9Q4mskRJHLQuUFsgYuFwayM2JLL9kFOEi6UTuLqfGD7h0DHAwEvM25P+v7FwjXuyf8LxrWZm+lLPC/JyxnMaLUs45BrdupAe0jyBpmKlBjoSuJWsik/UbiO+vbYwJ0hek69GpbGuFMtormyMIFSRCXRsMJhGspWj5k+2+KICf2Kx5bW+PxLfEjejhWYExkrM/y+r5acvQ/WI+lVWYzKSlwypNrCqMsJ9euHDgfCUbxaQwpTwrHqqdBhaaZcfaJAJmDJGz3SuJqxM+JSuQUYvKvQwdfL+6RRmaIMp7/hSd5tJqjJRd4FkSXYzwT5AHjn6L7OefJxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6vRNUT8x84jkmg4e61wPSC8IbP4rdyDEfRG2ff/C/Jw=;
 b=JBGHCjEfJN7w/dKGMVnu9wvudHVio860healWWmpFiTxuP0vQ8rclTzp3VqJrrIluzImVYGDwVm8+ENZCeQtJ9LJktWjDSmJ65qSD/3tJ6ETgjEpIoyDUuqv3zPOglf2fpSWz7N7bNe8boXWH98nR1bVLSg1ZdRyyTm7qoGiWAdzNEyJxOnS1upCl3WuhptAsGsxq+jxW/skkfjLog2CCuPY1YyN7OqVEXca5FUULzEUr3lsCOYJRKCbSoqq4xM/1XPYPajy/Ba4OTYwyINvv7KsIxkVuv9QOL9FtJ8lnz9ZzWzYZzYCsKaRn8rB60+kIrld0xEhbpIAmP9CRa0lgQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6vRNUT8x84jkmg4e61wPSC8IbP4rdyDEfRG2ff/C/Jw=;
 b=D7zaaaYViPL0cynA6mP7xFHD5UMNWK19BWJoTPyo4QVUQovhGIpQmJT0Ckg1aJwVQxS2U7kPrTTZRbHpgVM7eCNnBe1hytLGalfNRyYYkfp+HXu/qBirtZznZuDsQyhglo6zq1pBjvcCPgxocRHUrQXOFMIw56X2OQ+z0sO2v50=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: [XEN][RFC PATCH 11/13] tools/libs/ctrl: Implement new xc interfaces for fpga-add and fpga-del
Date: Wed, 1 Sep 2021 23:06:01 -0700
Message-ID: <1630562763-390068-12-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: becd6285-e0dd-4a35-7bf2-08d96dd80433
X-MS-TrafficTypeDiagnostic: DM6PR02MB4043:
X-Microsoft-Antispam-PRVS:
	<DM6PR02MB4043D97CCF955D32A752F967BCCE9@DM6PR02MB4043.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:1468;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	u+EdAWF8Cc9fzd1XrneN/sHZ+iXJv8tDOsLWUoefmZO87ab09jnLPwNvpTIrdRuyfJ8yiVlSwU387RfV53S3f8AnPgHqed9e0SxF3idUHXTl2LnuLB9AS73NSST9YDr8mriSWDu/+lGUB83RTkfB2ZafObA1jk8Z9ucKCyfMmlmkJFNPF/fuOyykc4t2uC1+TQJim2D+QP+UYH6kONTgaKelrjNAmGFlJXDbiaEMraCQQqAAZOdBhd/61P/mf1i8UbWktvzArgodv9Bae6HNw5/h5TK6WkgbMvksl2MAEst/KgzzW4AtNPxo0ZhHAXsLFk6tEIckc3qVCaXpMEVBL/8F/ca0Kaqyd+6gOUC9hTe1QLZfpe19vNQSjuE8R280yO6J2ZWShVYeHpVU2ZL4FVI9p0mk0f1SStGX8z98SQaNkLVqG9Lqw7y6HeimywkEApQIvwYG6vvWNHF/e/+gQiwbatYrcoIC5pY4u3m20RI2ARhBqLWTHp2THuGUacTDXFW+G3BHoHb6tmMyXQRk9v4uRHPkKKMaqmb0+B3WkXGUaYTM0ansGwfup+m33drd5WU+Y+75XtoSV0bO/6K5YeBLmIsqEidFRcb5tTa+PT9uUEC4vv9CMQlE4LXJ+9W1oPK8IqUAqSZoIti85EZlzN/q/skqWfkEUAi0x/vup2QenryjPDemBN5PA0N9ofLyrtZn8ItZSZzclp2yoEogRDYHRmF1QjbUs9A7o6kwNrmVtEcmdFAAwQpK/ZzIynuKUdAnWMymH2Tu1hZbHzOWsFeB659FH7F6bIDo+R798gxude+KJ1GxwNjRlCLZdH/mnt8R3LA1JM6zGBXNTADcdg==
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(36840700001)(46966006)(478600001)(82740400003)(47076005)(36756003)(8676002)(70206006)(82310400003)(6666004)(36906005)(26005)(8936002)(36860700001)(2616005)(186003)(7636003)(336012)(426003)(2906002)(4326008)(7696005)(5660300002)(316002)(70586007)(9786002)(356005)(6916009)(54906003)(102446001)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:07:59.9152
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: becd6285-e0dd-4a35-7bf2-08d96dd80433
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB4043

xc_domain_add_fpga() sends the device tree binary overlay and size of .dtbo to
xen.
xc_domain_del_fpga() sends full path for the node to be removed.

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 tools/include/xenctrl.h   |  4 +++
 tools/libs/ctrl/Makefile  |  1 +
 tools/libs/ctrl/xc_fpga.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 87 insertions(+)
 create mode 100644 tools/libs/ctrl/xc_fpga.c

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index b77726e..d14b3df 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2679,6 +2679,10 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32
 int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
                          xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
 
+int xc_domain_add_fpga(xc_interface *xch, void *pfdt, int pdft_size);
+int xc_domain_del_fpga(xc_interface *xch, char *full_dt_node_path);
+
+
 /* Compat shims */
 #include "xenctrl_compat.h"
 
diff --git a/tools/libs/ctrl/Makefile b/tools/libs/ctrl/Makefile
index 519246b..95021b9 100644
--- a/tools/libs/ctrl/Makefile
+++ b/tools/libs/ctrl/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 SRCS-y       += xc_altp2m.c
 SRCS-y       += xc_cpupool.c
+SRCS-$(CONFIG_ARM) += xc_fpga.c
 SRCS-y       += xc_domain.c
 SRCS-y       += xc_evtchn.c
 SRCS-y       += xc_gnttab.c
diff --git a/tools/libs/ctrl/xc_fpga.c b/tools/libs/ctrl/xc_fpga.c
new file mode 100644
index 0000000..41c37b5
--- /dev/null
+++ b/tools/libs/ctrl/xc_fpga.c
@@ -0,0 +1,82 @@
+/*
+ *
+ * FPGA control functions.
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "xc_bitops.h"
+#include "xc_private.h"
+#include <xen/hvm/hvm_op.h>
+#include <libfdt.h>
+
+int xc_domain_add_fpga(xc_interface *xch, void *pfdt, int pfdt_size)
+{
+    int err;
+    DECLARE_DOMCTL;
+
+    DECLARE_HYPERCALL_BOUNCE(pfdt, pfdt_size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, pfdt)) )
+        goto err;
+
+    domctl.cmd = XEN_DOMCTL_addfpga;
+    /* Adding the device to hardware domain by default. */
+    domctl.domain = 0;
+    domctl.u.fpga_add_dt.pfdt_size = pfdt_size;
+
+    set_xen_guest_handle(domctl.u.fpga_add_dt.pfdt, pfdt);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed\n", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, pfdt);
+
+    return err;
+}
+
+int xc_domain_del_fpga(xc_interface *xch, char *full_dt_node_path)
+{
+    int err;
+    DECLARE_DOMCTL;
+    size_t size = strlen(full_dt_node_path) + 1;
+
+    DECLARE_HYPERCALL_BOUNCE(full_dt_node_path, size,
+                             XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+    if ( (err = xc_hypercall_bounce_pre(xch, full_dt_node_path)) )
+        goto err;
+
+    domctl.cmd = XEN_DOMCTL_delfpga;
+    /*
+     * Remove the device from the dt_host, setting hardware domain by
+     * default.
+     */
+    domctl.domain = 0;
+    domctl.u.fpga_del_dt.size = size;
+
+    set_xen_guest_handle(domctl.u.fpga_del_dt.full_dt_node_path,
+                         full_dt_node_path);
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        PERROR("%s failed\n", __func__);
+
+err:
+    xc_hypercall_bounce_post(xch, full_dt_node_path);
+
+    return err;
+}
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 06:51:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 06:51:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.176935.322181 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYz-00078B-Ia; Thu, 02 Sep 2021 06:51:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 176935.322181; Thu, 02 Sep 2021 06:51:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgYy-00071s-7R; Thu, 02 Sep 2021 06:51:04 +0000
Received: by outflank-mailman (input) for mailman id 176935;
 Thu, 02 Sep 2021 06:08:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=393g=NY=xilinx.com=fnuv@srs-us1.protection.inumbo.net>)
 id 1mLftM-0004km-9P
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 06:08:04 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.236.89]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 22069f30-0bb4-11ec-ae3e-12813bfff9fa;
 Thu, 02 Sep 2021 06:08:03 +0000 (UTC)
Received: from BN1PR14CA0026.namprd14.prod.outlook.com (2603:10b6:408:e3::31)
 by SA0PR02MB7500.namprd02.prod.outlook.com (2603:10b6:806:e9::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 06:08:02 +0000
Received: from BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
 (2603:10b6:408:e3:cafe::fd) by BN1PR14CA0026.outlook.office365.com
 (2603:10b6:408:e3::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Thu, 2 Sep 2021 06:08:02 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 BN1NAM02FT024.mail.protection.outlook.com (10.13.2.138) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 06:08:01 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 1 Sep 2021 23:07:23 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 1 Sep 2021 23:07:23 -0700
Received: from [172.19.2.115] (port=56596 helo=xsjfnuv50.xilinx.com)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <fnu.vikram@xilinx.com>)
 id 1mLfsh-000F6q-AB; Wed, 01 Sep 2021 23:07:23 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22069f30-0bb4-11ec-ae3e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PqXc9nZHuBVpovZhtd88HvzHVB8dah0yQfDIGTWKMs+Vv+T5UAMeNAT5E/D6uYfZvEm2uWuuW82J0pjvKifUUNzrwm8/+Skpdi4AgJYGFABnVQlR1QgqiMShDXODKmneMn0YfQ7HenSe8ZdVbg0X9CbRUUoQ58IXoSIVM4kcEwfMGBD8ZHvQGiV9a1oRl/njSUh56tAgKvZ0Z9VmjL7B63ywm2zS46nBMW61TPi9YYsSgCIOKhuJfFrtbgoFyEMWiDRv8iDnNrPZtzBQDaDYMbyJtGdWXnpsWvarRn7AgaWu1S4r7vT+UjOGUWcAE0JFsM8Uc/4twb5JwWjyCZYPUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ykNwRHO6V8HwcABuxYp6Pws7yb5pYX82KhqW7gTX5nc=;
 b=CgY1gDga71hBm89KMbDdD2WJPWR2lnb/Xx0NUtEGdx/Aw+gXSQYIMeiDGyLL7+PNgMeIfowbOOSaH7E5JSBSbypdS0/VXeSh3BEKcwaHOYfuDe/st6iirOvHFrBBabzMUaKQ0Jnuvfy6lRliYhPAgAO+LgwPb05j7ix8uy4Sg+u63jMr8JgSS4eXOkeM54EVt75XTLeTsRPwhnqKsZwEwqct5IDv7ZZWoWvRQ1rk36RitMlitsv4YO81ODiL8BBmXC33L7k8CRYvUXpzkiy6QmiiPJuJvbXM3uSM/Si5wJuctlwhhH5H5QvOYTfR6QvCOIWw7JlroSUjWv05IEXO2A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=lists.xenproject.org
 smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none
 header.from=xilinx.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ykNwRHO6V8HwcABuxYp6Pws7yb5pYX82KhqW7gTX5nc=;
 b=G6+822z/VzQ5VXb3CZJGI+vF84PpPixiKHazT8usadnJUJV3GNk1pVcE09e/4/lWTYLrRrqJANikEdGS8Q7HDuOaeJf0OAsaoKBMNTKKIdKqsz910P17k8I65eu3sITtESFZsvSLWPYJWbHL6S82yLDzUN3R2VCtNwLPBI9cy6c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
From: Vikram Garhwal <fnu.vikram@xilinx.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <julien@xen.org>, Vikram Garhwal
	<fnu.vikram@xilinx.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Subject: [XEN][RFC PATCH 13/13] tools/xl: Add new xl commands fpga-add and fpga-del
Date: Wed, 1 Sep 2021 23:06:03 -0700
Message-ID: <1630562763-390068-14-git-send-email-fnu.vikram@xilinx.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
References: <1630562763-390068-1-git-send-email-fnu.vikram@xilinx.com>
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c76b6324-d7df-4ebd-1549-08d96dd8056b
X-MS-TrafficTypeDiagnostic: SA0PR02MB7500:
X-Microsoft-Antispam-PRVS:
	<SA0PR02MB75008BEFC4A4BEC256638290BCCE9@SA0PR02MB7500.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:580;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	X4ERjSdA+RWK/hy0b42uEAoSAyrO4UpkaYB7WUQ0Tf+YwDLtoSbqoAuTzatAkj6FhC8rhHhQGKykX/auKQywlj/5IU8rd8cbCIY5D35NXs671zs+GEb6yMwqBcw3jqL2CiAmal8ElmIacC1yg4j2ZBgTZeI5Pan+W85+dlZrw96rE3WFvLZVmy40X+U5EjfL2gvYzbN4XTJXyxSs4X/D9dhZM9Vcp0n62NU+cdvbrCs8zq/h20wNOHmjvXdi2X7UPyNx0/HOLcaRX0FuU3CBAaY38n4XRK2xUh8jCShiMoKV3Fux49PWP9fJOFviHMCTxOWGfxSqLm2JcQ3rz36dWUWmLRBjij8P3NFW7XTyvy+1vm7QP4AlMOIqZVEOzmS9ovfOroBuVdPvb47lY945NumWVBscmwakiIbHieNIfHd33rI5tB6VokevY0nh4DBQ1yuVvIi2utIE5hE4oQcm5kw89FKpZ2rJL2TyisKW21MwN+fJi3JWEsXluiWYrpw7RBPQTstRe3aGMISukcWxV2jpynYMMRoDT/kgQyWB+8YSekOQKARDNKYGzhhZ4qcdKY+jrL5Sehx/poY9pIE6sn65lrLGy9B25mj3p7EG2PFMazDrzFMJjG4eUvtOZnY7ly712XLFUptEP+WlAo4K+E/FvFBmIRW1kqRvkSfqnzfHVhoIM+vlQjHMUHLdzZ8RUHKKyan5BSFtOyKMhIkbucxxenwgKbY5F3olGZtBoQI=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(39860400002)(136003)(36840700001)(46966006)(316002)(7696005)(36906005)(426003)(83380400001)(478600001)(36756003)(54906003)(26005)(2906002)(36860700001)(9786002)(8936002)(336012)(82310400003)(5660300002)(4326008)(2616005)(82740400003)(356005)(7636003)(8676002)(6916009)(70586007)(6666004)(70206006)(186003)(47076005)(102446001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 06:08:01.9563
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c76b6324-d7df-4ebd-1549-08d96dd8056b
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7500

Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
---
 tools/xl/xl.h           |  2 ++
 tools/xl/xl_cmdtable.c  | 12 ++++++++++++
 tools/xl/xl_vmcontrol.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 65 insertions(+)

diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 7e23f30..63be31e 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -140,6 +140,8 @@ int main_shutdown(int argc, char **argv);
 int main_reboot(int argc, char **argv);
 int main_list(int argc, char **argv);
 int main_vm_list(int argc, char **argv);
+int main_fpga_add(int argc, char **argv);
+int main_fpga_del(int argc, char **argv);
 int main_create(int argc, char **argv);
 int main_config_update(int argc, char **argv);
 int main_button_press(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 661323d..135fe6a 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -20,6 +20,18 @@
 #include "xl.h"
 
 const struct cmd_spec cmd_table[] = {
+    { "fpga-add",
+      &main_fpga_add, 1, 1,
+      "Add a PL block",
+      "<.dtbo>"
+      "-h print this help\n"
+    },
+    { "fpga-del",
+      &main_fpga_del, 1, 1,
+      "Remove a PL block",
+      "<full_node_path>"
+      "-h print this help\n"
+    },
     { "create",
       &main_create, 1, 1,
       "Create a domain from config file <filename>",
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 435155a..f5bfdbc 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1262,6 +1262,57 @@ int main_create(int argc, char **argv)
     return 0;
 }
 
+int main_fpga_add(int argc, char **argv)
+{
+    const char *fpga_config_file = argv[1];
+    void *pfdt = NULL;
+    int rc;
+    int pfdt_size = 0;
+
+    if (fpga_config_file) {
+        rc = libxl_read_file_contents(ctx, fpga_config_file,
+                                      &pfdt, &pfdt_size);
+
+        if (rc) {
+            fprintf(stderr, "failed to read the fpga-partial device file %s\n",
+                    fpga_config_file);
+            free(pfdt);
+            return ERROR_FAIL;
+        }
+    } else {
+        fprintf(stderr, "FPGA config file is not provided\n");
+        return ERROR_FAIL;
+    }
+
+    rc = libxl_add_fpga_node(ctx, pfdt, pfdt_size);
+    if (rc)
+        fprintf(stderr, "Adding FPGA node failed\n");
+
+    free(pfdt);
+    return rc;
+}
+
+int main_fpga_del(int argc, char **argv)
+{
+    char *full_dt_node_path = argv[1];
+    int rc = 0;
+
+    if (full_dt_node_path) {
+        rc = libxl_del_fpga_node(ctx, full_dt_node_path);
+
+        fprintf(stdout, "fpga-del called for device = %s\n", full_dt_node_path);
+
+        if (rc)
+            fprintf(stderr, "Removing FPGA node failed\n");
+
+    } else {
+        fprintf(stderr, "No device node path provided\n");
+        return ERROR_FAIL;
+    }
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:00:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:00:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177039.322229 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgi3-00063H-7a; Thu, 02 Sep 2021 07:00:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177039.322229; Thu, 02 Sep 2021 07:00:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgi3-00063A-3e; Thu, 02 Sep 2021 07:00:27 +0000
Received: by outflank-mailman (input) for mailman id 177039;
 Thu, 02 Sep 2021 07:00:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLgi2-000634-GK
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:00:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 72228770-0bbb-11ec-ae41-12813bfff9fa;
 Thu, 02 Sep 2021 07:00:25 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2050.outbound.protection.outlook.com [104.47.5.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-VH_ELK56Nlm_85o7wd7EDw-1; Thu, 02 Sep 2021 09:00:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Thu, 2 Sep
 2021 07:00:20 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 07:00:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0008.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Thu, 2 Sep 2021 07:00:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72228770-0bbb-11ec-ae41-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630566024;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kHyfyd4gEvYSCBmF+GcFJVNB691mgc2xHZaGZWzVAvU=;
	b=PJ7bBsaAUQth3+ZaFzcZIhmrlbbUE++4/LBKOWk/usOuvxuXYUZo7hj8WuJPzlChX1TpU2
	XkhCV/SS70kYdPZ8opvLmmNZoyDFe9mA44Q5P64NyQ4D/p8hfPEG0F0u3h6LBICmrHYgtX
	rawW/sEd//clM5C2H9K88p5efGsbV9E=
X-MC-Unique: VH_ELK56Nlm_85o7wd7EDw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UlP1RISFv3UfrAUtTSxlXQ4IYloSQULwD6SZE/1E+Im4fflCi6MjkjLMXY10ia1jUxi1+q0k3265Cl4LxNpCXbwI+C+HDRUjMnGjFoKRjEsvMq+30IK1myP7c1LOdwmVpkSGmb4jDwu9pDKboyo4gwqH66Y2qSwJ2p+4ijDF6kbJaOkpI/WjZb6mAcK42xugWllIg5sTYXATQLrQtxo9ffWr7EBMOi4ve9X6Pn8UIPRqR6GwIjVwSmK3UkShYFUlwuZ8TQL89y3czIrpHLDKI0avaS2CB7DgZ+aEdYcZhS/oQ8uksyiPkc0rNy0JQKd/CV09KlHpmugdigTOB6kP5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Du1n8tHvw2rKWfhNlde3mS2ais71qjhr//pscVaPcMY=;
 b=C/fWVH+ZsZzQaLVwa1JdOvT5CmkAbiYGUGbt9f8V1qlKSyNGCGrvlOnYznquIj8+Vg2CLTxPNpotNjS8cL62EXWsEkRrN+qiZ7JsleI9Y5V4ydv9CkO+XNm++lMtOvr8nBiJObIbEbytueZ1eDce8gpucl8FJ+pXdDAGra+XwEs0xBdkoFk4u/wOJ0J9yXTZrMPt7OQbt7pbyxZ+oduyzilKUjUeGmvkZeFSMLu/gDp1zF4JJnmcD144hBLzEtpGiYvpQEKhQwE3x3gk9t29+B29rPlwjOYfGHJza+4kU//0mn/SWCMuMOGz/mYeuDDd+uCQ2nrReMk+6Sf2bsxVyw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
 <601d46e2-5c08-3948-85bc-1e027358700b@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f5c0f417-c271-011d-4c7a-fb210b5efde0@suse.com>
Date: Thu, 2 Sep 2021 09:00:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <601d46e2-5c08-3948-85bc-1e027358700b@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P192CA0008.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 50850b0e-bddc-4423-4431-08d96ddf5400
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB29590EEA0CA77D07860B9707B3CE9@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ajOC5YbxyrAjr3artB7TyVqDX0NBsY04+jrVNIyT8YSPyd4j8gbNE9znkT/+gSweUNMfg4fagRkwJgV9DJlOHhC8BXfqUANMf1Mj6arnPHEbZW68quhBMSg/WPNl0TP3DDc6QLGJWyENbyRfJcI/CnPDGFiLfPRKrCIEwoxJObPFKknQflyuRmu4tT25LV1u4jWc0GrULi3E58ZAYYuutQZnMPi7mm8WGp9/jl/N+DC4OaqiJCayNpZyAhNwnkSwsV3Ag8JINMnQizcxry/bI0Te+09ZnNtAXEsN/zWskTdIGXK6fVNyrZ/IfYLFgb8rRbN3FMxaoDk+gfycID60MuB4b6o82Flh+Pe3jtSBw4tkJd/365EY38j8HU9Q8U7Px0TrK2TU0zQ2uJL37QRr0RqBQecswvgdQmChTjTfYt2Pop5o0G6mb7OI6ppfEsFVzIj1ZUAupqKK4UrXI6I3UNjXUIKJk0iA5tr7t90wOm+YctpwPuAmEl/WvhYesvzaidn2tnMo2j1rt26QzPkiWU3zAJaF1WIxQr1pZyLqGEI8wBpE+ohklzghJgTP6Q/yuh/XYW+APpr1ZM/nnBclqP8KUTxCwYpvEDLXpm7BLMKMjV7yHkcgsadu9qE0ZYCRijE40If3X35DJgf1qt7N6rhK77lTOROVCSXMG/42SzRhVUB3Fi6r2emfciqG4XpWsD7DdHuFKci+ZokCjgfdX5fFhwhb8AbtgZgm1Ymj46Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(956004)(2616005)(86362001)(83380400001)(66556008)(66476007)(66946007)(6916009)(6486002)(38100700002)(54906003)(4326008)(26005)(53546011)(16576012)(186003)(31696002)(8936002)(2906002)(5660300002)(508600001)(36756003)(31686004)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?VcQFa+Smcz0/DkvbGOXNA2y96oLOBmYt8eyDjdxHNDnKOd4NTu8OuvHkwdMc?=
 =?us-ascii?Q?dKlIGSvjqxgtAj0Nwk/H/NKztLs1tRokpw1aiHvKtA+OC6swi/T/P6+qHbfb?=
 =?us-ascii?Q?gPqW8P2GC3D9JElgTPIi45OI24t6PeWPLNSek93k5LXrW5iBhI1uMyDDskKW?=
 =?us-ascii?Q?W6+cy+k9LQwR5VCL9J+q3bL+98X4inzR5A79vw+NrZfOyEWUoeuUQsrQuZOD?=
 =?us-ascii?Q?4Mgu5pjiTVkh1uMxq1ceiQu3LgSVFumdvpVZO+Mm3WCdyEfGW+XbV1IozK3K?=
 =?us-ascii?Q?KPyVcRd/KbcxFfghlvAZCKRAO7ILS3W15xGZn11UDHIghF9LqMBPquq8Wc/J?=
 =?us-ascii?Q?JmYSd75Fz4T6S1FXOoV8K0WHdDwyV+/KrXcCPqOpJDWNzMPg2I3JQfzhWMsT?=
 =?us-ascii?Q?OORgGACJybHZ7YYPtSrIRW6aE45TXDp0TUZEPr46iInT3ujBcGbeg2fU41Sv?=
 =?us-ascii?Q?Y1etoQvahtrtOKZo90GnXHBjaWoItH+1se9VOm5gjm45FCximU6mii5b/A+M?=
 =?us-ascii?Q?2TxpZkWfTsZdXId9kDRnCCC5ymBvcSHPwEtXcmyxm2VFVxBRfMLSo2ZCAbFf?=
 =?us-ascii?Q?8zZkPaHFCjlosIgPDvMpN3z1pYf919vE/nv4Y/kDx8NTDQ/IzLKC3AHIaQCw?=
 =?us-ascii?Q?KL8LrKmNhmmL2Qamef4MnJ6jWcj6WuCX2KZmwfi7Z46qzPFDTD44aoJ4kJiS?=
 =?us-ascii?Q?HBJPzSwXjMglKsfAK/rt2g3WffYDuHAFIFTouLXRU9D+6QF3H0ZtkW4L1Vcx?=
 =?us-ascii?Q?aDeMEoK3/Q32mU/TRlATdwZoYGKtoDRgBJ7VP1tNM4Q5Pyd5V5F7S4cPxIb3?=
 =?us-ascii?Q?6bLcJ6iDcmwXgI3hLzxtGxCqV3aRiK7Ie5ezoi/MFQXHV8dQSioRmknjZZNa?=
 =?us-ascii?Q?QkQnCds/kXogIZ4TZaLYXf5P9t9QlNs6wmFv68Y1tqRpLNoqK967ww2O+apP?=
 =?us-ascii?Q?aaLV/teVrhxAlxsBeC4vOZ/a3JfP7UGY5DF3dECuewAMEeMeUohTPc7YXXC1?=
 =?us-ascii?Q?eU3Gsfs/7H+tpdlo3IRh9SHMw4Kbc7an2n8MTHlcoH3MNaDer7pSUuM6MvM6?=
 =?us-ascii?Q?8tUJHhGf/DwCYiHdnIymTjoXT7FKJOCroJlHx/hlaXWMeau5lE6E+FtqDgqP?=
 =?us-ascii?Q?pWJndGv/tG7yt4nPiDuHms6VuWIs2raty50do2QCQupBwwx9xKxzr1aE205t?=
 =?us-ascii?Q?4i/NTxo3c9fSJPwnGjSZeCK6o1b6PBqjOv4GwV2TnT5qsO8BE9rRLQkTCSU7?=
 =?us-ascii?Q?LyEh08nM6U444bQx/7WY82euh3pDG5cmePFzzSRSECw1j89PoVQOjkK2YgYx?=
 =?us-ascii?Q?viNgC0Q7REVrg16NV4s3a9Qu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 50850b0e-bddc-4423-4431-08d96ddf5400
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 07:00:20.5313
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DxvZLK+3wbulOW9nm26/HNyejGgufW5bh0DK93kwApz798waMVFtvqQdftR3VES4GtnTZrbpXL5v/Bi55rNdFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

On 01.09.2021 22:02, Andrew Cooper wrote:
> On 01/09/2021 17:06, Jan Beulich wrote:
>> The function may fail; it is not correct to indicate "success" in this
>> case up the call stack. Mark the function must-check to prove all
>> cases have been caught (and no new ones will get introduced).
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> In the grant-transfer case it is not really clear to me whether we can
>> stick to setting GTF_transfer_completed in the error case. Since a guest
>> may spin-wait for the flag to become set, simply not setting the flag is
>> not an option either. I was wondering whether we may want to slightly
>> alter (extend) the ABI and allow for a GTF_transfer_committed ->
>> GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
>> at the same time as setting GTF_transfer_completed).
>=20
> Considering there are no production users of gnttab_transfer(), we can
> do what we want.=C2=A0 It was introduced for (IIRC) netlink2 and never go=
t
> into production, and then we clobbered it almost entirely in an XSA
> several years ago by restricting steal_page() to PV guests only.
>=20
> As a consequence, we can do anything which seems sensible, and does not
> necessarily need to be bound by a guest spinning on the bit.

Is this a "yes, let's go that route" then? Or rather leaving it to me,
i.e. translating "we can do anything which seems sensible" to "feel free
to do anything which seems sensible"? Which might as well be what is
there right now, and hence there could be the implied question of
whether your reply could be translated to an ack.

> The concept of gnttab_transfer() alone is crazy from an in-guest memory
> management point of view.=C2=A0 We could alternatively save our future se=
lves
> more trouble by just Kconfig'ing it out now, deleting it in several
> releases time, and fogetting about the problem as nothing will break in
> practice.

I might ack such an initial patch. I might even consider making one
myself as long as it's agreed that the option will need to default to
y. I might also ack such a subsequent patch. But I would not want to
be the one to propose a patch removing functionality. I think I did say
more than once in the past that I don't think we can validly remove
anything from the public interface, as we will never be able to _prove_
there's no user anywhere. An exception might only be in cases where we
can prove certain functionality could never have been used successfully
for its intended or any other purpose. (For example, recently I've
[again] been considering to fully disable XENMEM_increase_reservation
for translated guests, rather than just leaving it useless by not
reporting back the allocated MFNs.)

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:01:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:01:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177056.322240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgjM-00075L-M1; Thu, 02 Sep 2021 07:01:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177056.322240; Thu, 02 Sep 2021 07:01:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLgjM-00075E-IB; Thu, 02 Sep 2021 07:01:48 +0000
Received: by outflank-mailman (input) for mailman id 177056;
 Thu, 02 Sep 2021 07:01:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLgjL-000756-GI
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:01:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a2d0c698-0bbb-11ec-ae41-12813bfff9fa;
 Thu, 02 Sep 2021 07:01:46 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-OLV-JV4COwOG_7GeAIMqYQ-1; Thu, 02 Sep 2021 09:01:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7151.eurprd04.prod.outlook.com (2603:10a6:800:129::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 07:01:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 07:01:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0012.eurprd03.prod.outlook.com (2603:10a6:208:14::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Thu, 2 Sep 2021 07:01:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2d0c698-0bbb-11ec-ae41-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630566105;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=27WMxhXQJYDK8zBJxq3wqZWYbPIQe9ZgUpLtLbNSdZo=;
	b=A3qlVHgvffdkiGTZ/azdUis9J0mJNJul+Pong685ScUO69AC/9rzl48r2jFiy7uUrgGzGk
	J2fDAv3oFvGU/aCiNr5SXMp1JZE25HP2W68lJC689UdnrZOGvgvWM4DDMHTed7NlSDD7kW
	0gkD/ka9Jgf4KQlFGFpLzOF/QXtbloQ=
X-MC-Unique: OLV-JV4COwOG_7GeAIMqYQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nHEnISrcHdZrmeEhjciMwCW7JIRW3f0n2KvAxSRbtZII+QfpNvzbT/1mnXRYPve6+raQpdkpr5DsdYF79HjCGSs5U0d9Y8/gcSBquK8t+m5VJEy26R3Aoguh1M+zF8rP+CmQQXWev6sv2CH02F+YsXc7Y7Z5KTxgt8EzGJm3wdp5Qaz5KcPbwfnXu7VuaBfzf1onlo3A20NuhzJEusn/H1nVz87uksPG9YD4oefzeP2MMAp7vb0YJFLk+KIfENJNdbTgijsDYN/FVBh6/cC5LYRQHGOSXYDilO8Y1dNFKipObTrFYvvqarIUGWrvqAf9Z5TvpScN/RxxFQhO1oE6PA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=27WMxhXQJYDK8zBJxq3wqZWYbPIQe9ZgUpLtLbNSdZo=;
 b=k+vYz7XkeCgxv0UA8VqD72xKzK2oYkw5izLtTAyYeAuEf4SMSrSOfTL0ZngxdzFOtnw6Ftbi6n8K8i/20YgUJqqBWmhGRSE837k+MMvrZc1P32elvt67ptuyaOoUyYhCAVsv2o7eaq8BGJudYL6BkIEXOUdq0JGi/jyyE4GQ01iSr9qyJyLyj1oFVjl1F+Ah8+hmYYVZ4mFTmdcBqyocW7+jaOXnQSSdQIqxz/V7PcbweZf78dSp9yApQxwBC7TwvOiB4z8/cTATGZHeU8dasTRmS259JsMZ5pmFWL89RFs8BhwKVMZBZYPrvv01SlcRaT0+qFjXvcP4pmTeRcSzoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/p2m-pt: fix p2m_flags_to_access()
Message-ID: <2ec3fb93-5b91-15d0-4769-adf414af86b9@suse.com>
Date: Thu, 2 Sep 2021 09:01:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0012.eurprd03.prod.outlook.com
 (2603:10a6:208:14::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cd5acd9f-2fb4-4565-b91c-08d96ddf85a0
X-MS-TrafficTypeDiagnostic: VI1PR04MB7151:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71513AE6B5666A7FED7FD17EB3CE9@VI1PR04MB7151.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	N9Rmf4qgpAwGvEOWd0pbMrv85qMp/QOUoLk0i/IsWjLIY7Mu14hvufIYYsIff0HYZ3+wW7UL9rM8HNd0hzHiakjz1KINuP9uJktc21yOR4Eve5riUQff4r5aBEhesAeNzGH5SReLNTvwahb1YPEhbW9hx5qQczrko4wKAMKqQRUWWaZuprLKfTgT5wol+0mAwbptBPU91Da/KdU+kj5KmykWx8ORIzECAQOUgy4qxO1kSpS8pnIZrJyCXVqiDbmZe4k4jEyJwmWFDLejeDDAL/y3ZGyED2U6fyXFKtb68AfhnUrDzpr20x+byMGbiGLq4DOqOBwPTiOLgCzPkFSpJMRK+zPv+BVgi21FPmffndYOvIFOusaUjXbtzl0V6g/kzq6mDFV6Ck7r1lygOkVOWcVGSoOg7Bxtty2z83R11DFHUcRKOf5DyF3U+0Gh3VhClJTFFGQfMx58TS6YvfuxbU8+mXv+fyW3CaiPhmZLktCQYuSEP5U46DTMWZDzOo6xwrMc9/MiTiE0snTmkl2YTLdOzrrSkXGZfxbmM3prOj/aRppRpKF8EDwMgOarWiHSw+J/DRDZziK8ZsHTPDRPcTrTyIgbiqkprcC1uQgFI6HJMQY7fVInF+c1PKeDKmu2DeoaMr8vAUy7eCyBNXLNm1EaWxm8ohwxEZXJSnXyYC9lj0QyQLCBat0Bsvs7LLgxEOpbXWiQjwv5Qjzp1X2MjNIshOW4/Ep40eIo3iEJGxg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(136003)(396003)(346002)(376002)(6486002)(5660300002)(83380400001)(86362001)(956004)(2616005)(54906003)(31696002)(31686004)(6916009)(36756003)(478600001)(316002)(16576012)(66556008)(66946007)(66476007)(38100700002)(186003)(26005)(4326008)(8936002)(2906002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?azB0YlpoQm91WThOQ2hPRHZmVTh1VE5uMFJjempNOGpwL3BpbUxUOUlaMmcv?=
 =?utf-8?B?LzI5NlFPSTBEejZqemR3bDB4d1BKa1dQME9TdVA5dTFZRXNuWEMrR21tSVFH?=
 =?utf-8?B?anNQZkJPMXRja01TeHNEWktmTjgyQnRGQ3VEN3hSZ2VPWm9NZmc0NCtscUVx?=
 =?utf-8?B?VEV4ZmpuNDRqb1dWUEp0c0FCcTNuTy9iMDNBb01DcHJQWmpEVFZvTi9sN0Fj?=
 =?utf-8?B?UmVtRTdZa0tSdE45L3lQdGxqSDRpUUNBSDVFbTl4c0ZBb0UwRkZoaThSVmtv?=
 =?utf-8?B?MFcwRHFUQllCcmZlN2g0c0diNXFadjJlM2kxMEJvb0RFdjZteW81L0s3dUVE?=
 =?utf-8?B?K0t5M0lhbllFNmhHZlNwenc1NnQ5UWJwWG9zZkJkSVBMSzJZQlJ0UW5RbXVS?=
 =?utf-8?B?RngwZnhDUHRCVVBzTlVOSDJ0elE5UW0vVm5MN3Y4U0Q3dkI3UmZHb0VyUVd6?=
 =?utf-8?B?S3UxVDRXZEt5RHhpWTlQRkVrV3dpWURmbzk5R2dxNTNSc2lROGtDbXNJRHVY?=
 =?utf-8?B?bEVrU2FmZ3lWMjd0UzRTRGxMd0IvOVlCN1JFWDJNckFTZm1FV0pMT1ZCVHQr?=
 =?utf-8?B?Zzh6aWtuM3I5cktsdHZzandQT21pV2ZkU2duL1QzTWhnUERaZExaTzl4NnNQ?=
 =?utf-8?B?QlpPVHowcUNjbndyT1BwZzFyV0ppTjhZUnUvdjVjb0p6bGxUN2ZENGZwaG9r?=
 =?utf-8?B?TUZ1Z2tlM0pRTEFYQ3JlVkpHTWhRdEtFNWdmVjE5TS9STTRiMnRTcjBqaTJ2?=
 =?utf-8?B?bUFZbXJOVlRIanFXZ1l3VmYrSituejlLbWFJOUErM1BlODlmVzFNc1liUWpu?=
 =?utf-8?B?QmVMQkxvQ3AwTEV5UUw0VXp6L0FEVDJ0RzJsOUFKNm1zS3NYVHIxYmF0VzR5?=
 =?utf-8?B?SUEwMjNlRXVoVzF5NGtrdkNJM0hocUNXTzVlMVp1aHBNVnk0cmJOMDB5MXJV?=
 =?utf-8?B?OU9QTGVsVW5VQkx1YnMyUElJYzYvbkxBaTVwQWpia0dWam0wZGcya3hwM2Qy?=
 =?utf-8?B?K1NoRnB1VGtyVmRBV3JzOEVGWmtQRnJ6ekdTSlF0b2N3eWc2TVJSOCt6c0hy?=
 =?utf-8?B?MU5pdWtQbENpdG1YZzBYWXgraGhIVEh4NXdFOE85QWJreGl4eC9XbkhzU3Yv?=
 =?utf-8?B?blAzWkFLSlBOV2FkL2p6a2RmR1orYU9pR205QVgxbTNBL1hGNXB3V2padGFM?=
 =?utf-8?B?Q2JZUzdCd2s1ZTRmbmcwL3p0cm5Yd2FFUjFGTU1WQVVZQ3JxLzZlNzdGV3Jx?=
 =?utf-8?B?S0ZYVkdFdnk0TWorbDVnR1BMMUFTK1REeXAvdFpPa3lrUDdMSFJsUFFKME1y?=
 =?utf-8?B?OFZsbzNaSE4xVEhma1ZZWkVQTk9mQ2pQR3cxLy9aWW9tM2NZVkp1bTUvQUJO?=
 =?utf-8?B?T2ZTN2VRVEh2T2xxYUxxZnp4aHJDTFkvL1ZET1R6SEE3eklkRXBkVkQzVnN2?=
 =?utf-8?B?amxXWmUxMUcrYXd4SG1OeXNyZHV2c0lqUVFTSi9QdDd3Tjd2M0o2cEpmbkVJ?=
 =?utf-8?B?QWpyNC9Ca3NNaWNCOXNaZXg4MlowdTBTWVpXNWszY0MwNHhOYUlVTWlyUVM0?=
 =?utf-8?B?dXlZK0I4clhQSGl0WmpYYjlpbmtXQW1maGxmMHJzNlNKMGlHUENBQ05NMzhm?=
 =?utf-8?B?eHZpaVpPNFA0Smc0dHhkU0JBOWNSTGs1TDd3aVBHdWFpQytTQlJlMnNhcmY4?=
 =?utf-8?B?dFFLdVZpTkJ4ZHFEZjRMVVlaUStSUWZRT3ZjUlRBa2FmMmM0Q0tNMUhhenB2?=
 =?utf-8?Q?4S3ob/DlcQOh1LqZRns70TEGq+qG13/O8WY5rdI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cd5acd9f-2fb4-4565-b91c-08d96ddf85a0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 07:01:43.7642
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9RIxMwgIb1xSBrHgKfR2il+Ua6kz7tlEfwDWWrxlQNY9xBjo97EMDWwzH+z60sdiJHzlysMMm5OdVt1R4Gpfow==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7151

The initial if() was inverted, invalidating all output from this
function. Which in turn means the mirroring of P2M mappings into the
IOMMU didn't always work as intended: Mappings may have got updated when
there was no need to. There would not have been too few (un)mappings;
what saves us is that alongside the flags comparison MFNs also get
compared, with non-present entries always having an MFN of 0 or
INVALID_MFN while present entries always have MFNs different from these
two (0 in the table also meant to cover INVALID_MFN):

OLD					NEW
P W	access	MFN			P W	access	MFN
0 0	r	0			0 0	n	0
0 1	rw	0			0 1	n	0
1 0	n	non-0			1 0	r	non-0
1 1	n	non-0			1 1	rw	non-0

present <-> non-present transitions are fine because the MFNs differ.
present -> present transitions as well as non-present -> non-present
ones are potentially causing too many map/unmap operations, but never
too few, because in that case old (bogus) and new access differ.

Fixes: d1bb6c97c31e ("IOMMU: also pass p2m_access_t to p2m_get_iommu_flags())
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -548,7 +548,7 @@ int p2m_pt_handle_deferred_changes(uint6
 /* Reconstruct a fake p2m_access_t from stored PTE flags. */
 static p2m_access_t p2m_flags_to_access(unsigned int flags)
 {
-    if ( flags & _PAGE_PRESENT )
+    if ( !(flags & _PAGE_PRESENT) )
         return p2m_access_n;
 
     /* No need to look at _PAGE_NX for now. */



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:19:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:19:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177062.322251 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLh0M-0000WO-Vi; Thu, 02 Sep 2021 07:19:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177062.322251; Thu, 02 Sep 2021 07:19:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLh0M-0000WH-Rz; Thu, 02 Sep 2021 07:19:22 +0000
Received: by outflank-mailman (input) for mailman id 177062;
 Thu, 02 Sep 2021 07:19:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+3xD=NY=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mLh0L-0000WB-Tc
 for xen-devel@lists.xen.org; Thu, 02 Sep 2021 07:19:21 +0000
Received: from mail-pg1-x52f.google.com (unknown [2607:f8b0:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b1a972a6-c42a-43dd-8b2e-5bf38f908459;
 Thu, 02 Sep 2021 07:19:13 +0000 (UTC)
Received: by mail-pg1-x52f.google.com with SMTP id f129so1004812pgc.1
 for <xen-devel@lists.xen.org>; Thu, 02 Sep 2021 00:19:13 -0700 (PDT)
Received: from laputa (p784a44f4.tkyea130.ap.so-net.ne.jp. [120.74.68.244])
 by smtp.gmail.com with ESMTPSA id a1sm1355173pgt.47.2021.09.02.00.19.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 00:19:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1a972a6-c42a-43dd-8b2e-5bf38f908459
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:content-transfer-encoding:in-reply-to;
        bh=2lOwlOf92xRQZ27XUHLa6fDJUvFnDxY8HtJzSHF0QdM=;
        b=AoRoz7WdNEuSMcu3bkrghr0jyxkFc0ZM7XiodMFBjs5shG6pEI5HFWEHU0Lp/eTJ/d
         Np89iwjpE4kGLpa2BFxahcbIbpKs8aZN/ag5M35NO/j9CtJP6vltBV3Il/L/0Bp/8wHL
         ErqdtsIKDAzs+NAqrMZLgC4MccXy9gRg0SVuJ0UCvwmrxjk/i4C+g8E49Qz0usShYdWv
         CNK+DGPQJ9ddd67R2iJzJwUEBIae5XhXt2UadZ6D2zHR4CoM7UbJiX0UklV0ADyjVxqU
         L0RrD/yErltWETAUIBATXWLgVQkLhQbQpHEWEicfRgq1me9DDNNriInz/qTy3q7ISCJy
         e9uQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:content-transfer-encoding
         :in-reply-to;
        bh=2lOwlOf92xRQZ27XUHLa6fDJUvFnDxY8HtJzSHF0QdM=;
        b=UMxHXQLftBEJvRVJSLV6tsL06VElwlc8bdvJMPaHs9209JtTxLda6rQBHaxnZXwag9
         P/kcPqr5X4lS4holHlIauEy7AZjBoTd0sOHcFL6Y/gLak+XjqgmqsDyZwu7/wmQ8f70M
         gJpKvHibW6ueRgNGomRG2gzNhU8lXs2anOlNrhqG8S1ilQC/kFim30NlW5qPeLHN3rPF
         cl9Dz/uTuecVWcOWUQWCa0aWVwMrWZum1hnQYnBlptuKk/XLQ5pu6aIBa7QsHbsYXCxM
         yPi1T5OmUg79XwxfHBZ//2WPJfVZ7/dohpQLIeEiRXWGtkFaN0tU14kRbRD45aJdyBSM
         sseg==
X-Gm-Message-State: AOAM532amTXTPVqwGwnmLBTKbAdIxwXx0aFipfO7lW9zAVU2INilBqPq
	4NTEuiA2Kin6oMAdpMnNeuqY2w==
X-Google-Smtp-Source: ABdhPJy4a+nNN6RWBSYKYShbk20muC5dtgGesXq8wgcPokrFdgcwMQx+4CpQChDoPxIV+vfk9ilkTA==
X-Received: by 2002:a62:1453:0:b0:3fd:ffd5:35cf with SMTP id 80-20020a621453000000b003fdffd535cfmr1977907pfu.34.1630567152430;
        Thu, 02 Sep 2021 00:19:12 -0700 (PDT)
Date: Thu, 2 Sep 2021 16:19:02 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: Wei Chen <Wei.Chen@arm.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin <Kaly.Xin@arm.com>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	"stefanha@redhat.com" <stefanha@redhat.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>,
	"pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
	Paul Durrant <paul@xen.org>, Xen Devel <xen-devel@lists.xen.org>,
	Rich Persaud <persaur@gmail.com>,
	Daniel Smith <dpsmith@apertussolutions.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210902071902.GC71098@laputa>
References: <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa>
 <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa>
 <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CACMJ4GbmNgbB5ponYt3NGEk3j6YCksot+kDy2qs8HMdFXWnQbw@mail.gmail.com>
 <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com>

Hi Christopher,

Thank you for your feedback.

On Mon, Aug 30, 2021 at 12:53:00PM -0700, Christopher Clark wrote:
> [ resending message to ensure delivery to the CCd mailing lists
> post-subscription ]
> 
> Apologies for being late to this thread, but I hope to be able to
> contribute to
> this discussion in a meaningful way. I am grateful for the level of
> interest in
> this topic. I would like to draw your attention to Argo as a suitable
> technology for development of VirtIO's hypervisor-agnostic interfaces.
> 
> * Argo is an interdomain communication mechanism in Xen (on x86 and Arm)
> that
>   can send and receive hypervisor-mediated notifications and messages
> between
>   domains (VMs). [1] The hypervisor can enforce Mandatory Access Control
> over
>   all communication between domains. It is derived from the earlier v4v,
> which
>   has been deployed on millions of machines with the HP/Bromium uXen
> hypervisor
>   and with OpenXT.
> 
> * Argo has a simple interface with a small number of operations that was
>   designed for ease of integration into OS primitives on both Linux
> (sockets)
>   and Windows (ReadFile/WriteFile) [2].
>     - A unikernel example of using it has also been developed for XTF. [3]
> 
> * There has been recent discussion and support in the Xen community for
> making
>   revisions to the Argo interface to make it hypervisor-agnostic, and
> support
>   implementations of Argo on other hypervisors. This will enable a single
>   interface for an OS kernel binary to use for inter-VM communication that
> will
>   work on multiple hypervisors -- this applies equally to both backends and
>   frontend implementations. [4]

Regarding virtio-over-Argo, let me ask a few questions:
(In figure "Virtual device buffer access:Virtio+Argo" in [4])
1) How the configuration is managed?
   On either virtio-mmio or virtio-pci, there always takes place
   some negotiation between the FE and BE through the "configuration"
   space. How can this be done in virtio-over-Argo?
2) Do there physically exist virtio's available/used vrings as well as
   descriptors, or are they virtually emulated over Argo (rings)?
3) The payload in a request will be copied into the receiver's Argo ring.
   What does the address in a descriptor mean?
   Address/offset in a ring buffer?
4) Estimate of performance or latency?
   It appears that, on FE side, at least three hypervisor calls (and data
   copying) need to be invoked at every request, right?

Thanks,
-Takahiro Akashi


> * Here are the design documents for building VirtIO-over-Argo, to support a
>   hypervisor-agnostic frontend VirtIO transport driver using Argo.
> 
> The Development Plan to build VirtIO virtual device support over Argo
> transport:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> 
> A design for using VirtIO over Argo, describing how VirtIO data structures
> and communication is handled over the Argo transport:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo
> 
> Diagram (from the above document) showing how VirtIO rings are synchronized
> between domains without using shared memory:
> https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-blob-url=true&id=01f7d0e1-7686-4f0b-88e1-457c1d30df40&collection=contentId-1348763698&contextId=1348763698&mimeType=image%2Fpng&name=device-buffer-access-virtio-argo.png&size=243175&width=1106&height=1241
> 
> Please note that the above design documents show that the existing VirtIO
> device drivers, and both vring and virtqueue data structures can be
> preserved
> while interdomain communication can be performed with no shared memory
> required
> for most drivers; (the exceptions where further design is required are those
> such as virtual framebuffer devices where shared memory regions are
> intentionally
> added to the communication structure beyond the vrings and virtqueues).
> 
> An analysis of VirtIO and Argo, informing the design:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO
> 
> * Argo can be used for a communication path for configuration between the
> backend
>   and the toolstack, avoiding the need for a dependency on XenStore, which
> is an
>   advantage for any hypervisor-agnostic design. It is also amenable to a
> notification
>   mechanism that is not based on Xen event channels.
> 
> * Argo does not use or require shared memory between VMs and provides an
> alternative
>   to the use of foreign shared memory mappings. It avoids some of the
> complexities
>   involved with using grants (eg. XSA-300).
> 
> * Argo supports Mandatory Access Control by the hypervisor, satisfying a
> common
>   certification requirement.
> 
> * The Argo headers are BSD-licensed and the Xen hypervisor implementation
> is GPLv2 but
>   accessible via the hypercall interface. The licensing should not present
> an obstacle
>   to adoption of Argo in guest software or implementation by other
> hypervisors.
> 
> * Since the interface that Argo presents to a guest VM is similar to DMA, a
> VirtIO-Argo
>   frontend transport driver should be able to operate with a physical
> VirtIO-enabled
>   smart-NIC if the toolstack and an Argo-aware backend provide support.
> 
> The next Xen Community Call is next week and I would be happy to answer
> questions
> about Argo and on this topic. I will also be following this thread.
> 
> Christopher
> (Argo maintainer, Xen Community)
> 
> --------------------------------------------------------------------------------
> [1]
> An introduction to Argo:
> https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> https://www.youtube.com/watch?v=cnC0Tg3jqJQ
> Xen Wiki page for Argo:
> https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen
> 
> [2]
> OpenXT Linux Argo driver and userspace library:
> https://github.com/openxt/linux-xen-argo
> 
> Windows V4V at OpenXT wiki:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> Windows v4v driver source:
> https://github.com/OpenXT/xc-windows/tree/master/xenv4v
> 
> HP/Bromium uXen V4V driver:
> https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
> 
> [3]
> v2 of the Argo test unikernel for XTF:
> https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.html
> 
> [4]
> Argo HMX Transport for VirtIO meeting minutes:
> https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html
> 
> VirtIO-Argo Development wiki page:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> 
> 
> > On Thu, Aug 26, 2021 at 5:11 AM Wei Chen <Wei.Chen@arm.com> wrote:
> >
> >> Hi Akashi,
> >>
> >> > -----Original Message-----
> >> > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> >> > Sent: 2021年8月26日 17:41
> >> > To: Wei Chen <Wei.Chen@arm.com>
> >> > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano Stabellini
> >> > <sstabellini@kernel.org>; Alex Benn??e <alex.bennee@linaro.org>; Kaly
> >> Xin
> >> > <Kaly.Xin@arm.com>; Stratos Mailing List <
> >> stratos-dev@op-lists.linaro.org>;
> >> > virtio-dev@lists.oasis-open.org; Arnd Bergmann <
> >> arnd.bergmann@linaro.org>;
> >> > Viresh Kumar <viresh.kumar@linaro.org>; Stefano Stabellini
> >> > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan Kiszka
> >> > <jan.kiszka@siemens.com>; Carl van Schaik <cvanscha@qti.qualcomm.com>;
> >> > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org>; Jean-
> >> > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> >> > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> >> > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> >> > <Bertrand.Marquis@arm.com>; Artem Mygaiev <Artem_Mygaiev@epam.com>;
> >> Julien
> >> > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul Durrant
> >> > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> >> > Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
> >> >
> >> > Hi Wei,
> >> >
> >> > On Fri, Aug 20, 2021 at 03:41:50PM +0900, AKASHI Takahiro wrote:
> >> > > On Wed, Aug 18, 2021 at 08:35:51AM +0000, Wei Chen wrote:
> >> > > > Hi Akashi，
> >> > > >
> >> > > > > -----Original Message-----
> >> > > > > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> >> > > > > Sent: 2021年8月18日 13:39
> >> > > > > To: Wei Chen <Wei.Chen@arm.com>
> >> > > > > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano
> >> Stabellini
> >> > > > > <sstabellini@kernel.org>; Alex Benn??e <alex.bennee@linaro.org>;
> >> > Stratos
> >> > > > > Mailing List <stratos-dev@op-lists.linaro.org>; virtio-
> >> > dev@lists.oasis-
> >> > > > > open.org; Arnd Bergmann <arnd.bergmann@linaro.org>; Viresh Kumar
> >> > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> >> > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan Kiszka
> >> > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> >> > <cvanscha@qti.qualcomm.com>;
> >> > > > > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org>;
> >> > Jean-
> >> > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> >> > > > > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> >> > > > > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> >> > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev <Artem_Mygaiev@epam.com
> >> >;
> >> > Julien
> >> > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul
> >> > Durrant
> >> > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> >> > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
> >> > > > >
> >> > > > > On Tue, Aug 17, 2021 at 08:39:09AM +0000, Wei Chen wrote:
> >> > > > > > Hi Akashi,
> >> > > > > >
> >> > > > > > > -----Original Message-----
> >> > > > > > > From: AKASHI Takahiro <takahiro.akashi@linaro.org>
> >> > > > > > > Sent: 2021年8月17日 16:08
> >> > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> >> > > > > > > Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>; Stefano
> >> > Stabellini
> >> > > > > > > <sstabellini@kernel.org>; Alex Benn??e <
> >> alex.bennee@linaro.org>;
> >> > > > > Stratos
> >> > > > > > > Mailing List <stratos-dev@op-lists.linaro.org>; virtio-
> >> > > > > dev@lists.oasis-
> >> > > > > > > open.org; Arnd Bergmann <arnd.bergmann@linaro.org>; Viresh
> >> Kumar
> >> > > > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> >> > > > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan
> >> Kiszka
> >> > > > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> >> > <cvanscha@qti.qualcomm.com>;
> >> > > > > > > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org
> >> >;
> >> > Jean-
> >> > > > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> >> > > > > > > <mathieu.poirier@linaro.org>; Oleksandr Tyshchenko
> >> > > > > > > <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> >> > > > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev
> >> > <Artem_Mygaiev@epam.com>;
> >> > > > > Julien
> >> > > > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul
> >> > Durrant
> >> > > > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> >> > > > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO
> >> backends
> >> > > > > > >
> >> > > > > > > Hi Wei, Oleksandr,
> >> > > > > > >
> >> > > > > > > On Mon, Aug 16, 2021 at 10:04:03AM +0000, Wei Chen wrote:
> >> > > > > > > > Hi All,
> >> > > > > > > >
> >> > > > > > > > Thanks for Stefano to link my kvmtool for Xen proposal here.
> >> > > > > > > > This proposal is still discussing in Xen and KVM
> >> communities.
> >> > > > > > > > The main work is to decouple the kvmtool from KVM and make
> >> > > > > > > > other hypervisors can reuse the virtual device
> >> implementations.
> >> > > > > > > >
> >> > > > > > > > In this case, we need to introduce an intermediate
> >> hypervisor
> >> > > > > > > > layer for VMM abstraction, Which is, I think it's very close
> >> > > > > > > > to stratos' virtio hypervisor agnosticism work.
> >> > > > > > >
> >> > > > > > > # My proposal[1] comes from my own idea and doesn't always
> >> > represent
> >> > > > > > > # Linaro's view on this subject nor reflect Alex's concerns.
> >> > > > > Nevertheless,
> >> > > > > > >
> >> > > > > > > Your idea and my proposal seem to share the same background.
> >> > > > > > > Both have the similar goal and currently start with, at first,
> >> > Xen
> >> > > > > > > and are based on kvm-tool. (Actually, my work is derived from
> >> > > > > > > EPAM's virtio-disk, which is also based on kvm-tool.)
> >> > > > > > >
> >> > > > > > > In particular, the abstraction of hypervisor interfaces has a
> >> > same
> >> > > > > > > set of interfaces (for your "struct vmm_impl" and my "RPC
> >> > interfaces").
> >> > > > > > > This is not co-incident as we both share the same origin as I
> >> > said
> >> > > > > above.
> >> > > > > > > And so we will also share the same issues. One of them is a
> >> way
> >> > of
> >> > > > > > > "sharing/mapping FE's memory". There is some trade-off between
> >> > > > > > > the portability and the performance impact.
> >> > > > > > > So we can discuss the topic here in this ML, too.
> >> > > > > > > (See Alex's original email, too).
> >> > > > > > >
> >> > > > > > Yes, I agree.
> >> > > > > >
> >> > > > > > > On the other hand, my approach aims to create a
> >> "single-binary"
> >> > > > > solution
> >> > > > > > > in which the same binary of BE vm could run on any
> >> hypervisors.
> >> > > > > > > Somehow similar to your "proposal-#2" in [2], but in my
> >> solution,
> >> > all
> >> > > > > > > the hypervisor-specific code would be put into another entity
> >> > (VM),
> >> > > > > > > named "virtio-proxy" and the abstracted operations are served
> >> > via RPC.
> >> > > > > > > (In this sense, BE is hypervisor-agnostic but might have OS
> >> > > > > dependency.)
> >> > > > > > > But I know that we need discuss if this is a requirement even
> >> > > > > > > in Stratos project or not. (Maybe not)
> >> > > > > > >
> >> > > > > >
> >> > > > > > Sorry, I haven't had time to finish reading your virtio-proxy
> >> > completely
> >> > > > > > (I will do it ASAP). But from your description, it seems we
> >> need a
> >> > > > > > 3rd VM between FE and BE? My concern is that, if my assumption
> >> is
> >> > right,
> >> > > > > > will it increase the latency in data transport path? Even if
> >> we're
> >> > > > > > using some lightweight guest like RTOS or Unikernel,
> >> > > > >
> >> > > > > Yes, you're right. But I'm afraid that it is a matter of degree.
> >> > > > > As far as we execute 'mapping' operations at every fetch of
> >> payload,
> >> > > > > we will see latency issue (even in your case) and if we have some
> >> > solution
> >> > > > > for it, we won't see it neither in my proposal :)
> >> > > > >
> >> > > >
> >> > > > Oleksandr has sent a proposal to Xen mailing list to reduce this
> >> kind
> >> > > > of "mapping/unmapping" operations. So the latency caused by this
> >> > behavior
> >> > > > on Xen may eventually be eliminated, and Linux-KVM doesn't have that
> >> > problem.
> >> > >
> >> > > Obviously, I have not yet caught up there in the discussion.
> >> > > Which patch specifically?
> >> >
> >> > Can you give me the link to the discussion or patch, please?
> >> >
> >>
> >> It's a RFC discussion. We have tested this RFC patch internally.
> >> https://lists.xenproject.org/archives/html/xen-devel/2021-07/msg01532.html
> >>
> >> > Thanks,
> >> > -Takahiro Akashi
> >> >
> >> > > -Takahiro Akashi
> >> > >
> >> > > > > > > Specifically speaking about kvm-tool, I have a concern about
> >> its
> >> > > > > > > license term; Targeting different hypervisors and different
> >> OSs
> >> > > > > > > (which I assume includes RTOS's), the resultant library should
> >> > be
> >> > > > > > > license permissive and GPL for kvm-tool might be an issue.
> >> > > > > > > Any thoughts?
> >> > > > > > >
> >> > > > > >
> >> > > > > > Yes. If user want to implement a FreeBSD device model, but the
> >> > virtio
> >> > > > > > library is GPL. Then GPL would be a problem. If we have another
> >> > good
> >> > > > > > candidate, I am open to it.
> >> > > > >
> >> > > > > I have some candidates, particularly for vq/vring, in my mind:
> >> > > > > * Open-AMP, or
> >> > > > > * corresponding Free-BSD code
> >> > > > >
> >> > > >
> >> > > > Interesting, I will look into them : )
> >> > > >
> >> > > > Cheers,
> >> > > > Wei Chen
> >> > > >
> >> > > > > -Takahiro Akashi
> >> > > > >
> >> > > > >
> >> > > > > > > -Takahiro Akashi
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > [1] https://op-lists.linaro.org/pipermail/stratos-dev/2021-
> >> > > > > > > August/000548.html
> >> > > > > > > [2] https://marc.info/?l=xen-devel&m=162373754705233&w=2
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > > From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> >> > > > > > > > > Sent: 2021年8月14日 23:38
> >> > > > > > > > > To: AKASHI Takahiro <takahiro.akashi@linaro.org>; Stefano
> >> > > > > Stabellini
> >> > > > > > > <sstabellini@kernel.org>
> >> > > > > > > > > Cc: Alex Benn??e <alex.bennee@linaro.org>; Stratos
> >> Mailing
> >> > List
> >> > > > > > > <stratos-dev@op-lists.linaro.org>; virtio-dev@lists.oasis-
> >> > open.org;
> >> > > > > Arnd
> >> > > > > > > Bergmann <arnd.bergmann@linaro.org>; Viresh Kumar
> >> > > > > > > <viresh.kumar@linaro.org>; Stefano Stabellini
> >> > > > > > > <stefano.stabellini@xilinx.com>; stefanha@redhat.com; Jan
> >> Kiszka
> >> > > > > > > <jan.kiszka@siemens.com>; Carl van Schaik
> >> > <cvanscha@qti.qualcomm.com>;
> >> > > > > > > pratikp@quicinc.com; Srivatsa Vaddagiri <vatsa@codeaurora.org
> >> >;
> >> > Jean-
> >> > > > > > > Philippe Brucker <jean-philippe@linaro.org>; Mathieu Poirier
> >> > > > > > > <mathieu.poirier@linaro.org>; Wei Chen <Wei.Chen@arm.com>;
> >> > Oleksandr
> >> > > > > > > Tyshchenko <Oleksandr_Tyshchenko@epam.com>; Bertrand Marquis
> >> > > > > > > <Bertrand.Marquis@arm.com>; Artem Mygaiev
> >> > <Artem_Mygaiev@epam.com>;
> >> > > > > Julien
> >> > > > > > > Grall <julien@xen.org>; Juergen Gross <jgross@suse.com>; Paul
> >> > Durrant
> >> > > > > > > <paul@xen.org>; Xen Devel <xen-devel@lists.xen.org>
> >> > > > > > > > > Subject: Re: Enabling hypervisor agnosticism for VirtIO
> >> > backends
> >> > > > > > > > >
> >> > > > > > > > > Hello, all.
> >> > > > > > > > >
> >> > > > > > > > > Please see some comments below. And sorry for the possible
> >> > format
> >> > > > > > > issues.
> >> > > > > > > > >
> >> > > > > > > > > > On Wed, Aug 11, 2021 at 9:27 AM AKASHI Takahiro
> >> > > > > > > <mailto:takahiro.akashi@linaro.org> wrote:
> >> > > > > > > > > > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano
> >> > Stabellini
> >> > > > > wrote:
> >> > > > > > > > > > > CCing people working on Xen+VirtIO and IOREQs. Not
> >> > trimming
> >> > > > > the
> >> > > > > > > original
> >> > > > > > > > > > > email to let them read the full context.
> >> > > > > > > > > > >
> >> > > > > > > > > > > My comments below are related to a potential Xen
> >> > > > > implementation,
> >> > > > > > > not
> >> > > > > > > > > > > because it is the only implementation that matters,
> >> but
> >> > > > > because it
> >> > > > > > > is
> >> > > > > > > > > > > the one I know best.
> >> > > > > > > > > >
> >> > > > > > > > > > Please note that my proposal (and hence the working
> >> > prototype)[1]
> >> > > > > > > > > > is based on Xen's virtio implementation (i.e. IOREQ) and
> >> > > > > > > particularly
> >> > > > > > > > > > EPAM's virtio-disk application (backend server).
> >> > > > > > > > > > It has been, I believe, well generalized but is still a
> >> > bit
> >> > > > > biased
> >> > > > > > > > > > toward this original design.
> >> > > > > > > > > >
> >> > > > > > > > > > So I hope you like my approach :)
> >> > > > > > > > > >
> >> > > > > > > > > > [1] https://op-lists.linaro.org/pipermail/stratos-
> >> > dev/2021-
> >> > > > > > > August/000546.html
> >> > > > > > > > > >
> >> > > > > > > > > > Let me take this opportunity to explain a bit more about
> >> > my
> >> > > > > approach
> >> > > > > > > below.
> >> > > > > > > > > >
> >> > > > > > > > > > > Also, please see this relevant email thread:
> >> > > > > > > > > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Wed, 4 Aug 2021, Alex Bennée wrote:
> >> > > > > > > > > > > > Hi,
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > One of the goals of Project Stratos is to enable
> >> > hypervisor
> >> > > > > > > agnostic
> >> > > > > > > > > > > > backends so we can enable as much re-use of code as
> >> > possible
> >> > > > > and
> >> > > > > > > avoid
> >> > > > > > > > > > > > repeating ourselves. This is the flip side of the
> >> > front end
> >> > > > > > > where
> >> > > > > > > > > > > > multiple front-end implementations are required -
> >> one
> >> > per OS,
> >> > > > > > > assuming
> >> > > > > > > > > > > > you don't just want Linux guests. The resultant
> >> guests
> >> > are
> >> > > > > > > trivially
> >> > > > > > > > > > > > movable between hypervisors modulo any abstracted
> >> > paravirt
> >> > > > > type
> >> > > > > > > > > > > > interfaces.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > In my original thumb nail sketch of a solution I
> >> > envisioned
> >> > > > > > > vhost-user
> >> > > > > > > > > > > > daemons running in a broadly POSIX like environment.
> >> > The
> >> > > > > > > interface to
> >> > > > > > > > > > > > the daemon is fairly simple requiring only some
> >> mapped
> >> > > > > memory
> >> > > > > > > and some
> >> > > > > > > > > > > > sort of signalling for events (on Linux this is
> >> > eventfd).
> >> > > > > The
> >> > > > > > > idea was a
> >> > > > > > > > > > > > stub binary would be responsible for any hypervisor
> >> > specific
> >> > > > > > > setup and
> >> > > > > > > > > > > > then launch a common binary to deal with the actual
> >> > > > > virtqueue
> >> > > > > > > requests
> >> > > > > > > > > > > > themselves.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Since that original sketch we've seen an expansion
> >> in
> >> > the
> >> > > > > sort
> >> > > > > > > of ways
> >> > > > > > > > > > > > backends could be created. There is interest in
> >> > > > > encapsulating
> >> > > > > > > backends
> >> > > > > > > > > > > > in RTOSes or unikernels for solutions like SCMI.
> >> There
> >> > > > > interest
> >> > > > > > > in Rust
> >> > > > > > > > > > > > has prompted ideas of using the trait interface to
> >> > abstract
> >> > > > > > > differences
> >> > > > > > > > > > > > away as well as the idea of bare-metal Rust
> >> backends.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > We have a card (STR-12) called "Hypercall
> >> > Standardisation"
> >> > > > > which
> >> > > > > > > > > > > > calls for a description of the APIs needed from the
> >> > > > > hypervisor
> >> > > > > > > side to
> >> > > > > > > > > > > > support VirtIO guests and their backends. However we
> >> > are
> >> > > > > some
> >> > > > > > > way off
> >> > > > > > > > > > > > from that at the moment as I think we need to at
> >> least
> >> > > > > > > demonstrate one
> >> > > > > > > > > > > > portable backend before we start codifying
> >> > requirements. To
> >> > > > > that
> >> > > > > > > end I
> >> > > > > > > > > > > > want to think about what we need for a backend to
> >> > function.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Configuration
> >> > > > > > > > > > > > =============
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > In the type-2 setup this is typically fairly simple
> >> > because
> >> > > > > the
> >> > > > > > > host
> >> > > > > > > > > > > > system can orchestrate the various modules that make
> >> > up the
> >> > > > > > > complete
> >> > > > > > > > > > > > system. In the type-1 case (or even type-2 with
> >> > delegated
> >> > > > > > > service VMs)
> >> > > > > > > > > > > > we need some sort of mechanism to inform the backend
> >> > VM
> >> > > > > about
> >> > > > > > > key
> >> > > > > > > > > > > > details about the system:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >   - where virt queue memory is in it's address space
> >> > > > > > > > > > > >   - how it's going to receive (interrupt) and
> >> trigger
> >> > (kick)
> >> > > > > > > events
> >> > > > > > > > > > > >   - what (if any) resources the backend needs to
> >> > connect to
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Obviously you can elide over configuration issues by
> >> > having
> >> > > > > > > static
> >> > > > > > > > > > > > configurations and baking the assumptions into your
> >> > guest
> >> > > > > images
> >> > > > > > > however
> >> > > > > > > > > > > > this isn't scalable in the long term. The obvious
> >> > solution
> >> > > > > seems
> >> > > > > > > to be
> >> > > > > > > > > > > > extending a subset of Device Tree data to user space
> >> > but
> >> > > > > perhaps
> >> > > > > > > there
> >> > > > > > > > > > > > are other approaches?
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Before any virtio transactions can take place the
> >> > > > > appropriate
> >> > > > > > > memory
> >> > > > > > > > > > > > mappings need to be made between the FE guest and
> >> the
> >> > BE
> >> > > > > guest.
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Currently the whole of the FE guests address space
> >> > needs to
> >> > > > > be
> >> > > > > > > visible
> >> > > > > > > > > > > > to whatever is serving the virtio requests. I can
> >> > envision 3
> >> > > > > > > approaches:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >  * BE guest boots with memory already mapped
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >  This would entail the guest OS knowing where in
> >> it's
> >> > Guest
> >> > > > > > > Physical
> >> > > > > > > > > > > >  Address space is already taken up and avoiding
> >> > clashing. I
> >> > > > > > > would assume
> >> > > > > > > > > > > >  in this case you would want a standard interface to
> >> > > > > userspace
> >> > > > > > > to then
> >> > > > > > > > > > > >  make that address space visible to the backend
> >> daemon.
> >> > > > > > > > > >
> >> > > > > > > > > > Yet another way here is that we would have well known
> >> > "shared
> >> > > > > > > memory" between
> >> > > > > > > > > > VMs. I think that Jailhouse's ivshmem gives us good
> >> > insights on
> >> > > > > this
> >> > > > > > > matter
> >> > > > > > > > > > and that it can even be an alternative for hypervisor-
> >> > agnostic
> >> > > > > > > solution.
> >> > > > > > > > > >
> >> > > > > > > > > > (Please note memory regions in ivshmem appear as a PCI
> >> > device
> >> > > > > and
> >> > > > > > > can be
> >> > > > > > > > > > mapped locally.)
> >> > > > > > > > > >
> >> > > > > > > > > > I want to add this shared memory aspect to my
> >> virtio-proxy,
> >> > but
> >> > > > > > > > > > the resultant solution would eventually look similar to
> >> > ivshmem.
> >> > > > > > > > > >
> >> > > > > > > > > > > >  * BE guests boots with a hypervisor handle to
> >> memory
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >  The BE guest is then free to map the FE's memory to
> >> > where
> >> > > > > it
> >> > > > > > > wants in
> >> > > > > > > > > > > >  the BE's guest physical address space.
> >> > > > > > > > > > >
> >> > > > > > > > > > > I cannot see how this could work for Xen. There is no
> >> > "handle"
> >> > > > > to
> >> > > > > > > give
> >> > > > > > > > > > > to the backend if the backend is not running in dom0.
> >> So
> >> > for
> >> > > > > Xen I
> >> > > > > > > think
> >> > > > > > > > > > > the memory has to be already mapped
> >> > > > > > > > > >
> >> > > > > > > > > > In Xen's IOREQ solution (virtio-blk), the following
> >> > information
> >> > > > > is
> >> > > > > > > expected
> >> > > > > > > > > > to be exposed to BE via Xenstore:
> >> > > > > > > > > > (I know that this is a tentative approach though.)
> >> > > > > > > > > >    - the start address of configuration space
> >> > > > > > > > > >    - interrupt number
> >> > > > > > > > > >    - file path for backing storage
> >> > > > > > > > > >    - read-only flag
> >> > > > > > > > > > And the BE server have to call a particular hypervisor
> >> > interface
> >> > > > > to
> >> > > > > > > > > > map the configuration space.
> >> > > > > > > > >
> >> > > > > > > > > Yes, Xenstore was chosen as a simple way to pass
> >> > configuration
> >> > > > > info to
> >> > > > > > > the backend running in a non-toolstack domain.
> >> > > > > > > > > I remember, there was a wish to avoid using Xenstore in
> >> > Virtio
> >> > > > > backend
> >> > > > > > > itself if possible, so for non-toolstack domain, this could
> >> done
> >> > with
> >> > > > > > > adjusting devd (daemon that listens for devices and launches
> >> > backends)
> >> > > > > > > > > to read backend configuration from the Xenstore anyway and
> >> > pass it
> >> > > > > to
> >> > > > > > > the backend via command line arguments.
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Yes, in current PoC code we're using xenstore to pass device
> >> > > > > > > configuration.
> >> > > > > > > > We also designed a static device configuration parse method
> >> > for
> >> > > > > Dom0less
> >> > > > > > > or
> >> > > > > > > > other scenarios don't have xentool. yes, it's from device
> >> > model
> >> > > > > command
> >> > > > > > > line
> >> > > > > > > > or a config file.
> >> > > > > > > >
> >> > > > > > > > > But, if ...
> >> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > In my approach (virtio-proxy), all those Xen (or
> >> > hypervisor)-
> >> > > > > > > specific
> >> > > > > > > > > > stuffs are contained in virtio-proxy, yet another VM, to
> >> > hide
> >> > > > > all
> >> > > > > > > details.
> >> > > > > > > > >
> >> > > > > > > > > ... the solution how to overcome that is already found and
> >> > proven
> >> > > > > to
> >> > > > > > > work then even better.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > > # My point is that a "handle" is not mandatory for
> >> > executing
> >> > > > > mapping.
> >> > > > > > > > > >
> >> > > > > > > > > > > and the mapping probably done by the
> >> > > > > > > > > > > toolstack (also see below.) Or we would have to
> >> invent a
> >> > new
> >> > > > > Xen
> >> > > > > > > > > > > hypervisor interface and Xen virtual machine
> >> privileges
> >> > to
> >> > > > > allow
> >> > > > > > > this
> >> > > > > > > > > > > kind of mapping.
> >> > > > > > > > > >
> >> > > > > > > > > > > If we run the backend in Dom0 that we have no problems
> >> > of
> >> > > > > course.
> >> > > > > > > > > >
> >> > > > > > > > > > One of difficulties on Xen that I found in my approach
> >> is
> >> > that
> >> > > > > > > calling
> >> > > > > > > > > > such hypervisor intefaces (registering IOREQ, mapping
> >> > memory) is
> >> > > > > > > only
> >> > > > > > > > > > allowed on BE servers themselvies and so we will have to
> >> > extend
> >> > > > > > > those
> >> > > > > > > > > > interfaces.
> >> > > > > > > > > > This, however, will raise some concern on security and
> >> > privilege
> >> > > > > > > distribution
> >> > > > > > > > > > as Stefan suggested.
> >> > > > > > > > >
> >> > > > > > > > > We also faced policy related issues with Virtio backend
> >> > running in
> >> > > > > > > other than Dom0 domain in a "dummy" xsm mode. In our target
> >> > system we
> >> > > > > run
> >> > > > > > > the backend in a driver
> >> > > > > > > > > domain (we call it DomD) where the underlying H/W resides.
> >> > We
> >> > > > > trust it,
> >> > > > > > > so we wrote policy rules (to be used in "flask" xsm mode) to
> >> > provide
> >> > > > > it
> >> > > > > > > with a little bit more privileges than a simple DomU had.
> >> > > > > > > > > Now it is permitted to issue device-model, resource and
> >> > memory
> >> > > > > > > mappings, etc calls.
> >> > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > > To activate the mapping will
> >> > > > > > > > > > > >  require some sort of hypercall to the hypervisor. I
> >> > can see
> >> > > > > two
> >> > > > > > > options
> >> > > > > > > > > > > >  at this point:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >   - expose the handle to userspace for daemon/helper
> >> > to
> >> > > > > trigger
> >> > > > > > > the
> >> > > > > > > > > > > >     mapping via existing hypercall interfaces. If
> >> > using a
> >> > > > > helper
> >> > > > > > > you
> >> > > > > > > > > > > >     would have a hypervisor specific one to avoid
> >> the
> >> > daemon
> >> > > > > > > having to
> >> > > > > > > > > > > >     care too much about the details or push that
> >> > complexity
> >> > > > > into
> >> > > > > > > a
> >> > > > > > > > > > > >     compile time option for the daemon which would
> >> > result in
> >> > > > > > > different
> >> > > > > > > > > > > >     binaries although a common source base.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >   - expose a new kernel ABI to abstract the
> >> hypercall
> >> > > > > > > differences away
> >> > > > > > > > > > > >     in the guest kernel. In this case the userspace
> >> > would
> >> > > > > > > essentially
> >> > > > > > > > > > > >     ask for an abstract "map guest N memory to
> >> > userspace
> >> > > > > ptr"
> >> > > > > > > and let
> >> > > > > > > > > > > >     the kernel deal with the different hypercall
> >> > interfaces.
> >> > > > > > > This of
> >> > > > > > > > > > > >     course assumes the majority of BE guests would
> >> be
> >> > Linux
> >> > > > > > > kernels and
> >> > > > > > > > > > > >     leaves the bare-metal/unikernel approaches to
> >> > their own
> >> > > > > > > devices.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Operation
> >> > > > > > > > > > > > =========
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > The core of the operation of VirtIO is fairly
> >> simple.
> >> > Once
> >> > > > > the
> >> > > > > > > > > > > > vhost-user feature negotiation is done it's a case
> >> of
> >> > > > > receiving
> >> > > > > > > update
> >> > > > > > > > > > > > events and parsing the resultant virt queue for
> >> data.
> >> > The
> >> > > > > vhost-
> >> > > > > > > user
> >> > > > > > > > > > > > specification handles a bunch of setup before that
> >> > point,
> >> > > > > mostly
> >> > > > > > > to
> >> > > > > > > > > > > > detail where the virt queues are set up FD's for
> >> > memory and
> >> > > > > > > event
> >> > > > > > > > > > > > communication. This is where the envisioned stub
> >> > process
> >> > > > > would
> >> > > > > > > be
> >> > > > > > > > > > > > responsible for getting the daemon up and ready to
> >> run.
> >> > This
> >> > > > > is
> >> > > > > > > > > > > > currently done inside a big VMM like QEMU but I
> >> > suspect a
> >> > > > > modern
> >> > > > > > > > > > > > approach would be to use the rust-vmm vhost crate.
> >> It
> >> > would
> >> > > > > then
> >> > > > > > > either
> >> > > > > > > > > > > > communicate with the kernel's abstracted ABI or be
> >> re-
> >> > > > > targeted
> >> > > > > > > as a
> >> > > > > > > > > > > > build option for the various hypervisors.
> >> > > > > > > > > > >
> >> > > > > > > > > > > One thing I mentioned before to Alex is that Xen
> >> doesn't
> >> > have
> >> > > > > VMMs
> >> > > > > > > the
> >> > > > > > > > > > > way they are typically envisioned and described in
> >> other
> >> > > > > > > environments.
> >> > > > > > > > > > > Instead, Xen has IOREQ servers. Each of them connects
> >> > > > > > > independently to
> >> > > > > > > > > > > Xen via the IOREQ interface. E.g. today multiple QEMUs
> >> > could
> >> > > > > be
> >> > > > > > > used as
> >> > > > > > > > > > > emulators for a single Xen VM, each of them connecting
> >> > to Xen
> >> > > > > > > > > > > independently via the IOREQ interface.
> >> > > > > > > > > > >
> >> > > > > > > > > > > The component responsible for starting a daemon and/or
> >> > setting
> >> > > > > up
> >> > > > > > > shared
> >> > > > > > > > > > > interfaces is the toolstack: the xl command and the
> >> > > > > libxl/libxc
> >> > > > > > > > > > > libraries.
> >> > > > > > > > > >
> >> > > > > > > > > > I think that VM configuration management (or
> >> orchestration
> >> > in
> >> > > > > > > Startos
> >> > > > > > > > > > jargon?) is a subject to debate in parallel.
> >> > > > > > > > > > Otherwise, is there any good assumption to avoid it
> >> right
> >> > now?
> >> > > > > > > > > >
> >> > > > > > > > > > > Oleksandr and others I CCed have been working on ways
> >> > for the
> >> > > > > > > toolstack
> >> > > > > > > > > > > to create virtio backends and setup memory mappings.
> >> > They
> >> > > > > might be
> >> > > > > > > able
> >> > > > > > > > > > > to provide more info on the subject. I do think we
> >> miss
> >> > a way
> >> > > > > to
> >> > > > > > > provide
> >> > > > > > > > > > > the configuration to the backend and anything else
> >> that
> >> > the
> >> > > > > > > backend
> >> > > > > > > > > > > might require to start doing its job.
> >> > > > > > > > >
> >> > > > > > > > > Yes, some work has been done for the toolstack to handle
> >> > Virtio
> >> > > > > MMIO
> >> > > > > > > devices in
> >> > > > > > > > > general and Virtio block devices in particular. However,
> >> it
> >> > has
> >> > > > > not
> >> > > > > > > been upstreaned yet.
> >> > > > > > > > > Updated patches on review now:
> >> > > > > > > > > https://lore.kernel.org/xen-devel/1621626361-29076-1-git-
> >> > send-
> >> > > > > email-
> >> > > > > > > olekstysh@gmail.com/
> >> > > > > > > > >
> >> > > > > > > > > There is an additional (also important) activity to
> >> > improve/fix
> >> > > > > > > foreign memory mapping on Arm which I am also involved in.
> >> > > > > > > > > The foreign memory mapping is proposed to be used for
> >> Virtio
> >> > > > > backends
> >> > > > > > > (device emulators) if there is a need to run guest OS
> >> completely
> >> > > > > > > unmodified.
> >> > > > > > > > > Of course, the more secure way would be to use grant
> >> memory
> >> > > > > mapping.
> >> > > > > > > Brietly, the main difference between them is that with foreign
> >> > mapping
> >> > > > > the
> >> > > > > > > backend
> >> > > > > > > > > can map any guest memory it wants to map, but with grant
> >> > mapping
> >> > > > > it is
> >> > > > > > > allowed to map only what was previously granted by the
> >> frontend.
> >> > > > > > > > >
> >> > > > > > > > > So, there might be a problem if we want to pre-map some
> >> > guest
> >> > > > > memory
> >> > > > > > > in advance or to cache mappings in the backend in order to
> >> > improve
> >> > > > > > > performance (because the mapping/unmapping guest pages every
> >> > request
> >> > > > > > > requires a lot of back and forth to Xen + P2M updates). In a
> >> > nutshell,
> >> > > > > > > currently, in order to map a guest page into the backend
> >> address
> >> > space
> >> > > > > we
> >> > > > > > > need to steal a real physical page from the backend domain.
> >> So,
> >> > with
> >> > > > > the
> >> > > > > > > said optimizations we might end up with no free memory in the
> >> > backend
> >> > > > > > > domain (see XSA-300). And what we try to achieve is to not
> >> waste
> >> > a
> >> > > > > real
> >> > > > > > > domain memory at all by providing safe non-allocated-yet (so
> >> > unused)
> >> > > > > > > address space for the foreign (and grant) pages to be mapped
> >> > into,
> >> > > > > this
> >> > > > > > > enabling work implies Xen and Linux (and likely DTB bindings)
> >> > changes.
> >> > > > > > > However, as it turned out, for this to work in a proper and
> >> safe
> >> > way
> >> > > > > some
> >> > > > > > > prereq work needs to be done.
> >> > > > > > > > > You can find the related Xen discussion at:
> >> > > > > > > > > https://lore.kernel.org/xen-devel/1627489110-25633-1-git-
> >> > send-
> >> > > > > email-
> >> > > > > > > olekstysh@gmail.com/
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > > One question is how to best handle notification and
> >> > kicks.
> >> > > > > The
> >> > > > > > > existing
> >> > > > > > > > > > > > vhost-user framework uses eventfd to signal the
> >> daemon
> >> > > > > (although
> >> > > > > > > QEMU
> >> > > > > > > > > > > > is quite capable of simulating them when you use
> >> TCG).
> >> > Xen
> >> > > > > has
> >> > > > > > > it's own
> >> > > > > > > > > > > > IOREQ mechanism. However latency is an important
> >> > factor and
> >> > > > > > > having
> >> > > > > > > > > > > > events go through the stub would add quite a lot.
> >> > > > > > > > > > >
> >> > > > > > > > > > > Yeah I think, regardless of anything else, we want the
> >> > > > > backends to
> >> > > > > > > > > > > connect directly to the Xen hypervisor.
> >> > > > > > > > > >
> >> > > > > > > > > > In my approach,
> >> > > > > > > > > >  a) BE -> FE: interrupts triggered by BE calling a
> >> > hypervisor
> >> > > > > > > interface
> >> > > > > > > > > >               via virtio-proxy
> >> > > > > > > > > >  b) FE -> BE: MMIO to config raises events (in event
> >> > channels),
> >> > > > > > > which is
> >> > > > > > > > > >               converted to a callback to BE via virtio-
> >> > proxy
> >> > > > > > > > > >               (Xen's event channel is internnally
> >> > implemented by
> >> > > > > > > interrupts.)
> >> > > > > > > > > >
> >> > > > > > > > > > I don't know what "connect directly" means here, but
> >> > sending
> >> > > > > > > interrupts
> >> > > > > > > > > > to the opposite side would be best efficient.
> >> > > > > > > > > > Ivshmem, I suppose, takes this approach by utilizing
> >> PCI's
> >> > msi-x
> >> > > > > > > mechanism.
> >> > > > > > > > >
> >> > > > > > > > > Agree that MSI would be more efficient than SPI...
> >> > > > > > > > > At the moment, in order to notify the frontend, the
> >> backend
> >> > issues
> >> > > > > a
> >> > > > > > > specific device-model call to query Xen to inject a
> >> > corresponding SPI
> >> > > > > to
> >> > > > > > > the guest.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Could we consider the kernel internally converting
> >> > IOREQ
> >> > > > > > > messages from
> >> > > > > > > > > > > > the Xen hypervisor to eventfd events? Would this
> >> scale
> >> > with
> >> > > > > > > other kernel
> >> > > > > > > > > > > > hypercall interfaces?
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > So any thoughts on what directions are worth
> >> > experimenting
> >> > > > > with?
> >> > > > > > > > > > >
> >> > > > > > > > > > > One option we should consider is for each backend to
> >> > connect
> >> > > > > to
> >> > > > > > > Xen via
> >> > > > > > > > > > > the IOREQ interface. We could generalize the IOREQ
> >> > interface
> >> > > > > and
> >> > > > > > > make it
> >> > > > > > > > > > > hypervisor agnostic. The interface is really trivial
> >> and
> >> > easy
> >> > > > > to
> >> > > > > > > add.
> >> > > > > > > > > >
> >> > > > > > > > > > As I said above, my proposal does the same thing that
> >> you
> >> > > > > mentioned
> >> > > > > > > here :)
> >> > > > > > > > > > The difference is that I do call hypervisor interfaces
> >> via
> >> > > > > virtio-
> >> > > > > > > proxy.
> >> > > > > > > > > >
> >> > > > > > > > > > > The only Xen-specific part is the notification
> >> mechanism,
> >> > > > > which is
> >> > > > > > > an
> >> > > > > > > > > > > event channel. If we replaced the event channel with
> >> > something
> >> > > > > > > else the
> >> > > > > > > > > > > interface would be generic. See:
> >> > > > > > > > > > > https://gitlab.com/xen-project/xen/-
> >> > > > > > > /blob/staging/xen/include/public/hvm/ioreq.h#L52
> >> > > > > > > > > > >
> >> > > > > > > > > > > I don't think that translating IOREQs to eventfd in
> >> the
> >> > kernel
> >> > > > > is
> >> > > > > > > a
> >> > > > > > > > > > > good idea: if feels like it would be extra complexity
> >> > and that
> >> > > > > the
> >> > > > > > > > > > > kernel shouldn't be involved as this is a backend-
> >> > hypervisor
> >> > > > > > > interface.
> >> > > > > > > > > >
> >> > > > > > > > > > Given that we may want to implement BE as a bare-metal
> >> > > > > application
> >> > > > > > > > > > as I did on Zephyr, I don't think that the translation
> >> > would not
> >> > > > > be
> >> > > > > > > > > > a big issue, especially on RTOS's.
> >> > > > > > > > > > It will be some kind of abstraction layer of interrupt
> >> > handling
> >> > > > > > > > > > (or nothing but a callback mechanism).
> >> > > > > > > > > >
> >> > > > > > > > > > > Also, eventfd is very Linux-centric and we are trying
> >> to
> >> > > > > design an
> >> > > > > > > > > > > interface that could work well for RTOSes too. If we
> >> > want to
> >> > > > > do
> >> > > > > > > > > > > something different, both OS-agnostic and hypervisor-
> >> > agnostic,
> >> > > > > > > perhaps
> >> > > > > > > > > > > we could design a new interface. One that could be
> >> > > > > implementable
> >> > > > > > > in the
> >> > > > > > > > > > > Xen hypervisor itself (like IOREQ) and of course any
> >> > other
> >> > > > > > > hypervisor
> >> > > > > > > > > > > too.
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > There is also another problem. IOREQ is probably not
> >> be
> >> > the
> >> > > > > only
> >> > > > > > > > > > > interface needed. Have a look at
> >> > > > > > > > > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2.
> >> > Don't we
> >> > > > > > > also need
> >> > > > > > > > > > > an interface for the backend to inject interrupts into
> >> > the
> >> > > > > > > frontend? And
> >> > > > > > > > > > > if the backend requires dynamic memory mappings of
> >> > frontend
> >> > > > > pages,
> >> > > > > > > then
> >> > > > > > > > > > > we would also need an interface to map/unmap domU
> >> pages.
> >> > > > > > > > > >
> >> > > > > > > > > > My proposal document might help here; All the interfaces
> >> > > > > required
> >> > > > > > > for
> >> > > > > > > > > > virtio-proxy (or hypervisor-related interfaces) are
> >> listed
> >> > as
> >> > > > > > > > > > RPC protocols :)
> >> > > > > > > > > >
> >> > > > > > > > > > > These interfaces are a lot more problematic than
> >> IOREQ:
> >> > IOREQ
> >> > > > > is
> >> > > > > > > tiny
> >> > > > > > > > > > > and self-contained. It is easy to add anywhere. A new
> >> > > > > interface to
> >> > > > > > > > > > > inject interrupts or map pages is more difficult to
> >> > manage
> >> > > > > because
> >> > > > > > > it
> >> > > > > > > > > > > would require changes scattered across the various
> >> > emulators.
> >> > > > > > > > > >
> >> > > > > > > > > > Exactly. I have no confident yet that my approach will
> >> > also
> >> > > > > apply
> >> > > > > > > > > > to other hypervisors than Xen.
> >> > > > > > > > > > Technically, yes, but whether people can accept it or
> >> not
> >> > is a
> >> > > > > > > different
> >> > > > > > > > > > matter.
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > -Takahiro Akashi
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Regards,
> >> > > > > > > > >
> >> > > > > > > > > Oleksandr Tyshchenko
> >> > > > > > > > IMPORTANT NOTICE: The contents of this email and any
> >> > attachments are
> >> > > > > > > confidential and may also be privileged. If you are not the
> >> > intended
> >> > > > > > > recipient, please notify the sender immediately and do not
> >> > disclose
> >> > > > > the
> >> > > > > > > contents to any other person, use it for any purpose, or store
> >> > or copy
> >> > > > > the
> >> > > > > > > information in any medium. Thank you.
> >> > > > > > IMPORTANT NOTICE: The contents of this email and any attachments
> >> > are
> >> > > > > confidential and may also be privileged. If you are not the
> >> intended
> >> > > > > recipient, please notify the sender immediately and do not
> >> disclose
> >> > the
> >> > > > > contents to any other person, use it for any purpose, or store or
> >> > copy the
> >> > > > > information in any medium. Thank you.
> >> > > > IMPORTANT NOTICE: The contents of this email and any attachments are
> >> > confidential and may also be privileged. If you are not the intended
> >> > recipient, please notify the sender immediately and do not disclose the
> >> > contents to any other person, use it for any purpose, or store or copy
> >> the
> >> > information in any medium. Thank you.
> >> IMPORTANT NOTICE: The contents of this email and any attachments are
> >> confidential and may also be privileged. If you are not the intended
> >> recipient, please notify the sender immediately and do not disclose the
> >> contents to any other person, use it for any purpose, or store or copy the
> >> information in any medium. Thank you.
> >>
> >


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:20:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177071.322261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLh1W-0001sI-Dq; Thu, 02 Sep 2021 07:20:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177071.322261; Thu, 02 Sep 2021 07:20:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLh1W-0001sB-Aj; Thu, 02 Sep 2021 07:20:34 +0000
Received: by outflank-mailman (input) for mailman id 177071;
 Thu, 02 Sep 2021 07:20:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLh1U-0001rz-Qf; Thu, 02 Sep 2021 07:20:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLh1U-0006vP-I7; Thu, 02 Sep 2021 07:20:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLh1U-0007uA-AK; Thu, 02 Sep 2021 07:20:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLh1U-0006L6-9o; Thu, 02 Sep 2021 07:20:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YSNsVfVLlDuPuXFPyaS0xSlkqQuUylt5HxzY4lfpp8I=; b=m3FBUxCpj1fPD6kv20wZC2ODVe
	XASfeswHH5O94ajXGVoaYFtdFOJ3YgEtv4aZgOo1YTFnkaoT3wtbSwfkEcQhCgosnGifx3UNZJn5f
	AuBgbM+h4Z5ldknkRBaltKtWkttd+FNTeCh2f7326kLSMa8qXMUKWKA1rJOg/ZWqBkz4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164745-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164745: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=345996c6208b281233074362a8d81295e2e711d4
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 07:20:32 +0000

flight 164745 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164745/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              345996c6208b281233074362a8d81295e2e711d4
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  419 days
Failing since        151818  2020-07-11 04:18:52 Z  418 days  409 attempts
Testing same since   164745  2021-09-02 04:18:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71488 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:29:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:29:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177079.322275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhA5-0002wN-8E; Thu, 02 Sep 2021 07:29:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177079.322275; Thu, 02 Sep 2021 07:29:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhA5-0002wG-5M; Thu, 02 Sep 2021 07:29:25 +0000
Received: by outflank-mailman (input) for mailman id 177079;
 Thu, 02 Sep 2021 07:29:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fIFR=NY=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLhA4-0002wA-0E
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:29:24 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7e147abc-0bbf-11ec-ae41-12813bfff9fa;
 Thu, 02 Sep 2021 07:29:23 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 18F7720312;
 Thu,  2 Sep 2021 07:29:22 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id DE1D813887;
 Thu,  2 Sep 2021 07:29:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id am2SNFF9MGH1OQAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 02 Sep 2021 07:29:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e147abc-0bbf-11ec-ae41-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630567762; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=IhcUjy8QGji2IABo6zcQqKi+1URPttsD+rMluhzr+U0=;
	b=gmK9k6utB4XDKQvE7yRgd7hk0/UUoy/oSRCveEWBBA12IccGUhOvjxNCVAqE7Hv9LWIwzf
	RfboEvRXeWHC5aZvw5Ej+gSD+W065MgGPNiCDNnnWXjNWBjs38EK8644ge/S3VHdKXlmfT
	f8HsXXPLRLgwX4qF6ZXJoXm+7xmJI1I=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	boris.ostrovsky@oracle.com
Subject: [GIT PULL] xen: branch for v5.15-rc1
Date: Thu,  2 Sep 2021 09:29:21 +0200
Message-Id: <20210902072921.27763-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15-rc1-tag

xen: branch for v5.15-rc1

It contains the following changes:

- some small cleanups
- a fix for a bug when running as Xen PV guest which could result in
  not all memory being transferred in case of a migration of the guest
- a small series for getting rid of code for supporting very old Xen
  hypervisor versions nobody should be using since many years now
- a series for hardening the Xen block frontend driver
- a fix for Xen PV boot code issuing warning messages due to a stray
  preempt_disable() on the non-boot processors

Thanks.

Juergen

 arch/x86/xen/enlighten_pv.c        |  12 +---
 arch/x86/xen/mmu_pv.c              |   4 +-
 arch/x86/xen/p2m.c                 |   4 +-
 arch/x86/xen/platform-pci-unplug.c |  16 ++---
 arch/x86/xen/smp_pv.c              |   1 -
 drivers/block/xen-blkfront.c       | 126 ++++++++++++++++++++++++-------------
 drivers/pci/xen-pcifront.c         |  30 +++++----
 drivers/xen/features.c             |  18 ++++++
 drivers/xen/gntdev.c               |  36 +----------
 drivers/xen/xenbus/xenbus_client.c |   9 +--
 10 files changed, 141 insertions(+), 115 deletions(-)

Jing Yangyang (1):
      drivers/xen/xenbus/xenbus_client.c: fix bugon.cocci warnings

Juergen Gross (9):
      xen: fix setting of max_pfn in shared_info
      xen: check required Xen features
      xen: assume XENFEAT_mmu_pt_update_preserve_ad being set for pv guests
      xen: assume XENFEAT_gnttab_map_avail_bits being set for pv guests
      xen/blkfront: read response from backend only once
      xen/blkfront: don't take local copy of a request from the ring page
      xen/blkfront: don't trust the backend response data blindly
      xen: remove stray preempt_disable() from PV AP startup code

Sergio Miguéns Iglesias (1):
      xen/pcifront: Removed unnecessary __ref annotation

zhaoxiao (1):
      x86: xen: platform-pci-unplug: use pr_err() and pr_warn() instead of raw printk()


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:36:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:36:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177085.322287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhGn-0004eG-Vc; Thu, 02 Sep 2021 07:36:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177085.322287; Thu, 02 Sep 2021 07:36:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhGn-0004e9-SZ; Thu, 02 Sep 2021 07:36:21 +0000
Received: by outflank-mailman (input) for mailman id 177085;
 Thu, 02 Sep 2021 07:36:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhGm-0004dx-F3; Thu, 02 Sep 2021 07:36:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhGm-0007B8-9S; Thu, 02 Sep 2021 07:36:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhGm-0008LC-13; Thu, 02 Sep 2021 07:36:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhGm-0007oK-0W; Thu, 02 Sep 2021 07:36:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=t3gwX6ivxTbp46ew6HRNNt4ACR+5Slw7ptLrr4N2toM=; b=tXV+dnD0TSMasm62gfFZ1ar6qs
	uq/fLlUa+q+96IryyDmezZ7iDk1FuIqA1a20pXPf23WgM98W38yKKrCg5AqMggTTTeFeqled1sOFu
	I267Ij/Ol2PcHIpK1x09V0d0j25RESymlv+1GKtxCzikDKtmVPbta+hVDNZsoKmIh0EQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Subject: [ovmf bisection] complete build-i386-xsm
Message-Id: <E1mLhGm-0007oK-0W@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 07:36:20 +0000

branch xen-unstable
xenbranch xen-unstable
job build-i386-xsm
testid xen-build

Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164753/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/ovmf/build-i386-xsm.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/ovmf/build-i386-xsm.xen-build --summary-out=tmp/164753.bisection-summary --basis-template=164674 --blessings=real,real-bisect,real-retry ovmf build-i386-xsm xen-build
Searching for failure / basis pass:
 164746 fail [host=huxelrebe1] / 164674 [host=fiano1] 164630 [host=albana0] 164556 [host=elbling1] 164513 [host=albana1] 164461 [host=huxelrebe0] 164406 [host=albana0] 164229 [host=huxelrebe0] 164173 [host=albana0] 164166 [host=albana1] 164160 ok.
Failure / basis pass flights: 164746 / 164160
(tree with no url: minios)
Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
Basis pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
Generating revisions with ./adhoc-revtuple-generator  https://github.com/tianocore/edk2.git#a776bbabd9e8f370feb089cdc016324b1d6a37c7-e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823-b6e539830bf45e2d7a6bd86ddfdf003088b173b0 git://xenbits.xen.org/osstest/seabios.git#54082c81d96028ba8c76fbe6784085cf1df76b\
 20-54082c81d96028ba8c76fbe6784085cf1df76b20 git://xenbits.xen.org/xen.git#25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe-96607a8e680e7f965ca868d11f8b0636317d2618
Loaded 76357 nodes in revision graph
Searching for test results:
 164160 pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
 164166 [host=albana1]
 164173 [host=albana0]
 164229 [host=huxelrebe0]
 164406 [host=albana0]
 164461 [host=huxelrebe0]
 164513 [host=albana1]
 164556 [host=elbling1]
 164630 [host=albana0]
 164674 [host=fiano1]
 164686 [host=huxelrebe0]
 164695 fail irrelevant
 164704 fail irrelevant
 164723 pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
 164726 fail irrelevant
 164718 fail irrelevant
 164727 pass 8dd4fc5be6189666b37e5b00131a4173c6a2b085 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 93713f444b3f29d6848527506db69cf78976b32d
 164729 fail irrelevant
 164728 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164730 pass irrelevant
 164732 pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
 164734 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164733 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164736 pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 8064488062641ae505b2a7369611c38057a7788b
 164737 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164738 pass cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 0e95e1df345bce95ac4d6b15c6ed1bf07659fac1 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164741 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 305c2c3abf8228989f7a6f6d8e698ca088c8dc42
 164740 [host=huxelrebe0]
 164742 pass 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164743 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164744 [host=huxelrebe0]
 164747 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164748 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164746 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164749 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164750 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164752 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164753 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
Searching for interesting versions
 Result found: flight 164160 (pass), for basis pass
 For basis failure, parent search stopping at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356, results HASH(0x5635eef165c0) HASH(0x5635f42f0720) HASH(0x5635eef0f138) For basis failure, parent search stopping at 28152333bccb778b62e6e97446b28bfa0e92ef82 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d\
 96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356, results HASH(0x5635eef0f228) For basis failure, parent search stopping at cae735f61328d64e2e8991036707b9e78c0f5f63 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 0e95e1df345bce95ac4d6b15c6ed1bf07659fac1 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356, results HASH(0x5635f42f1c88) Result found: flight 164728 (fail), for basis failure (at ancestor ~17)
 Repro found: flight 164732 (pass), for basis pass
 Repro found: flight 164743 (fail), for basis failure
 0 revisions at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
No revisions left to test, checking graph state.
 Result found: flight 164747 (pass), for last pass
 Result found: flight 164748 (fail), for first failure
 Repro found: flight 164749 (pass), for last pass
 Repro found: flight 164750 (fail), for first failure
 Repro found: flight 164752 (pass), for last pass
 Repro found: flight 164753 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164753/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>

Revision graph left in /home/logs/results/bisect/ovmf/build-i386-xsm.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
164753: tolerable ALL FAIL

flight 164753 ovmf real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/164753/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-i386-xsm                6 xen-build               fail baseline untested


jobs:
 build-i386-xsm                                               fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:51:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:51:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177095.322304 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhVc-0007bd-Fa; Thu, 02 Sep 2021 07:51:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177095.322304; Thu, 02 Sep 2021 07:51:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhVc-0007bW-Ca; Thu, 02 Sep 2021 07:51:40 +0000
Received: by outflank-mailman (input) for mailman id 177095;
 Thu, 02 Sep 2021 07:51:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=C28T=NY=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mLhVb-0007bQ-39
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:51:39 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 999aa22c-0bc2-11ec-ae45-12813bfff9fa;
 Thu, 02 Sep 2021 07:51:37 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id A78796736F; Thu,  2 Sep 2021 09:51:31 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 999aa22c-0bc2-11ec-ae45-12813bfff9fa
Date: Thu, 2 Sep 2021 09:51:31 +0200
From: Christoph Hellwig <hch@lst.de>
To: Tianyu Lan <ltykernel@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>, kys@microsoft.com,
	haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org,
	decui@microsoft.com, catalin.marinas@arm.com, will@kernel.org,
	tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org,
	hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org,
	peterz@infradead.org, konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com, jgross@suse.com, sstabellini@kernel.org,
	joro@8bytes.org, davem@davemloft.net, kuba@kernel.org,
	jejb@linux.ibm.com, martin.petersen@oracle.com,
	gregkh@linuxfoundation.org, arnd@arndb.de, m.szyprowski@samsung.com,
	robin.murphy@arm.com, brijesh.singh@amd.com,
	thomas.lendacky@amd.com, Tianyu.Lan@microsoft.com,
	pgonda@google.com, martin.b.radev@gmail.com,
	akpm@linux-foundation.org, kirill.shutemov@linux.intel.com,
	rppt@kernel.org, hannes@cmpxchg.org, aneesh.kumar@linux.ibm.com,
	krish.sadhukhan@oracle.com, saravanand@fb.com,
	linux-arm-kernel@lists.infradead.org,
	xen-devel@lists.xenproject.org, rientjes@google.com,
	ardb@kernel.org, michael.h.kelley@microsoft.com,
	iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org, netdev@vger.kernel.org,
	vkuznets@redhat.com, parri.andrea@gmail.com, dave.hansen@intel.com
Subject: Re: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
Message-ID: <20210902075131.GA14986@lst.de>
References: <20210827172114.414281-1-ltykernel@gmail.com> <20210830120036.GA22005@lst.de> <91b5e997-8d44-77f0-6519-f574b541ba9f@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <91b5e997-8d44-77f0-6519-f574b541ba9f@gmail.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Aug 31, 2021 at 11:20:06PM +0800, Tianyu Lan wrote:
>> If so I suspect the best way to allocate them is by not using vmalloc
>> but just discontiguous pages, and then use kmap_local_pfn where the
>> PFN includes the share_gpa offset when actually copying from/to the
>> skbs.
>>
> When netvsc needs to copy packet data to send buffer, it needs to caculate 
> position with section_index and send_section_size.
> Please seee netvsc_copy_to_send_buf() detail. So the contiguous virtual 
> address of send buffer is necessary to copy data and batch packets.

Actually that makes the kmap approach much easier.  The phys_to_virt
can just be replaced with a kmap_local_pfn and the unmap needs to
be added.  I've been mostly focussing on the receive path, which
would need a similar treatment.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:57:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:57:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177104.322320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhbH-0008Ll-5p; Thu, 02 Sep 2021 07:57:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177104.322320; Thu, 02 Sep 2021 07:57:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhbH-0008Le-2r; Thu, 02 Sep 2021 07:57:31 +0000
Received: by outflank-mailman (input) for mailman id 177104;
 Thu, 02 Sep 2021 07:57:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+khH=NY=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mLhbF-0008LW-Ha
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:57:29 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.88]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0b8d669d-1468-457d-96e2-40290352b25a;
 Thu, 02 Sep 2021 07:57:28 +0000 (UTC)
Received: from AM5PR1001CA0041.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::18) by PAXPR08MB6861.eurprd08.prod.outlook.com
 (2603:10a6:102:138::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 07:57:24 +0000
Received: from AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:15:cafe::cb) by AM5PR1001CA0041.outlook.office365.com
 (2603:10a6:206:15::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 07:57:24 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT063.mail.protection.outlook.com (10.152.16.226) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 07:57:24 +0000
Received: ("Tessian outbound 6ba9ec11d42b:v103");
 Thu, 02 Sep 2021 07:57:23 +0000
Received: from 510daa24c306.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3ECF3DD2-E246-461E-B55B-2672DFBCCA53.1; 
 Thu, 02 Sep 2021 07:57:17 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 510daa24c306.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 07:57:17 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6477.eurprd08.prod.outlook.com (2603:10a6:102:12d::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 2 Sep
 2021 07:57:16 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4457.024; Thu, 2 Sep 2021
 07:57:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b8d669d-1468-457d-96e2-40290352b25a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ntSFR6cfOCTWWDlTYNXDpalfdTw+y4oUqcYj51r9iZM=;
 b=KVgBj6vFozjH9e0m+YA5LxmqvI+6fu/bXc8cpcpJULzGePhqDvotMk1us0B1xmM77NGQJ0mtzraXTNMSVvtI+Oh2DEDCDQ6EocO5ClNlOpPbbaRallj0N/CP+I1DtWxFlTfIXik0cSstAzU/4By+cf1gfI2GBAFZ8kSDNEt9wl8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8e29bfe8e8ce2551
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GeeGDSltJszWaKXDmZwH5KbhHf3NaUrpTjIb/Mkpmm6zv1k7y3WUCh1ElJ4d6i52903iWiKpKY385/iVUnB97+KmWEUtBQbg9fUtl2XgNztpa7G+bfML5Ira3eNO361K8qbuGVXryR03/FI44DKWUWH/O5M55CNNA68xdSEJj0vjwGqnJOQj9+JWC8+N7Eey+wAqWtLIxUws4Kq8WbtBXvxHTr08dVsTmrLexYgV+1jMK02ZIabhCBduRCJjq1YvDVaAU2k4bgSyOf7mS6zh6JjEjP9uL7n/2DsS7Y0oDgOE45IUNOUzyVT8CQjMjXBnl1pK/3NYRbdAg5GZ6jD04w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ntSFR6cfOCTWWDlTYNXDpalfdTw+y4oUqcYj51r9iZM=;
 b=PPeSkPxbHwHnyQLOdZdvZrPejRcX20+zhf9lxLISbXz/YCtVbd1BSSh+QJ97rNIFrP0zqE7PxF8xxWieQOpseC/dtAu8gr/d4SnupPC7W3PkZqZBT9mfQYrTHONetW/KUe1MhO0YXcDs8HKzjkKlXzjle2VGYk2uFUO2ZDQNF4dQMDUywzo8RPlTn7bpabmclYLrbNVN8Vv5woE35u0FUxZ5o9oPQ+MDVsDx8az3a8XYo9ZkR1f88iuG97KudZUIxIq5XDyqEecYE0u+zwmO3x+BxxRe6Zo4mLZwbF0SmM/5xVDwB9OYTutcM/jBm9gHkZhqP/XZChBi253jZS+BCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ntSFR6cfOCTWWDlTYNXDpalfdTw+y4oUqcYj51r9iZM=;
 b=KVgBj6vFozjH9e0m+YA5LxmqvI+6fu/bXc8cpcpJULzGePhqDvotMk1us0B1xmM77NGQJ0mtzraXTNMSVvtI+Oh2DEDCDQ6EocO5ClNlOpPbbaRallj0N/CP+I1DtWxFlTfIXik0cSstAzU/4By+cf1gfI2GBAFZ8kSDNEt9wl8=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Michal Orzel
	<Michal.Orzel@arm.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Topic: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Index:
 AQHXny72/ET98Hho/USYtlrYv98fUauPIugAgAAENgCAABSIgIAAOtOAgAAK3QCAAOCVgA==
Date: Thu, 2 Sep 2021 07:57:16 +0000
Message-ID: <5B7B9C01-FF1D-44F9-8815-A422E51916A0@arm.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
 <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
 <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
 <0e72fb05-a477-ef79-c26f-bc011393a403@xen.org>
In-Reply-To: <0e72fb05-a477-ef79-c26f-bc011393a403@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3fa7285f-14a1-42b9-96b6-08d96de74cc7
x-ms-traffictypediagnostic: PAXPR08MB6477:|PAXPR08MB6861:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6861FEC68A13662FAFF8E3C39DCE9@PAXPR08MB6861.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 JxM2BR8EKGesiIpWcr2WI4Gzz97N2CMbXBa0g66NCj1EGjajJnY1VkV/ulWDs20FL45eoonRgOD7T2PuAAH63MUqfOGS9iAI2J9ztelOAjSCNDCmkps7hULL8ZA+eemPFSvWlTwQDkPHVKcXbjmx8uIYjiS4ygEr0WU9g0jA0gVmqNJchUQ29rLAXtr7X34b9dXxTdaT9bNbAmTBNjVhfFifs/zDeRUAx7TBsZ9rCfrInxL4Qq8NqRFGIV+A66iNVjOLKDmTszbHc15C0OVR5SAmWKsjUPb0WECeZ6WWbWikmekNNUjjCGP3gUG9qhua78uUiyCx6TZT0MaRrWTlsMZB0uQ2AipsX7mg9QMzgi49w1Nu7dhvalPcj4FELda+e79JX1qFvjUeQfxsm9+72r0v4yem64qAHBtRgRb3a10BlWDtllWRzIk4hCn4OgD0MB9WmF1WmMO50CDJPDQqNDKMlcoNY0xA1TFZq8zInx4guqKUpNxfbkzUn+3pCvBkLK2tXrqf871lGE/s1H0Cv4SOF4Y6DF7+0BVG5jhXFi0ZDusXUsMDE4rDaoUMbXMSAe3dqWXJNsH4SUAzQXxKVeA04xiGI/baITsC54DHrk//wtS8rbdCp2R/AV1WLZtRLXc7YRvma1/5EnLrh/99S9ealuVa7/GMN4KWp8jVpnFuid1T3TFNb1VWO6SctASBfuFu78Kuj8pSJUPHbMxi6uUb6Tky2dK6NLkox2c6P3AAT6E4BxqknNDos35QWsKz
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(366004)(136003)(396003)(376002)(346002)(26005)(6506007)(86362001)(478600001)(53546011)(38070700005)(6512007)(33656002)(6916009)(2616005)(36756003)(66476007)(66556008)(64756008)(66446008)(6486002)(2906002)(186003)(91956017)(76116006)(8676002)(8936002)(71200400001)(4326008)(38100700002)(54906003)(316002)(5660300002)(83380400001)(122000001)(66946007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <867EF41F77E3234C88AFCA1BDAE35D46@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6477
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c1e4dcca-8f65-4b19-7459-08d96de747f7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	656ATI06FYh8EQeTYkgRV2lC1edRwuxgQic7RMeeBTVVz0gklXqYnLmCwjVmutYLM3ta6kzY8m4Fe0yMpMY7mHNXVbI0BvGFmZ8RhNETV10go8Zq0F7CrsiIYuOcy1oqyv0nUZm+x1TPP+ntiFlaycVugBxUBxK7tYO5Ny5n6DqVjad9NXR659GUAEcoFgeoflf+RmZiuQ9mPKTDaOEx4GekBy51E7wL1rr1ouTlNzRf7rAzbAZezbJU/kaLp9VyxjMGyi3C4L8sd3dAeGRGnOYIn0RQRieqsiOsJ+kXm8aoGgZ26oLjxui3Loil92QPPG51i28ubEpo2aipKfMf9ehU7znSnRxZTS3rszlmw8wCMRhME2YEWLb9Xk7CCKVswMZ0eQYhV4lLVzhCJhP2P1K6sAMgSV7VNxMS4Bb0VqWqw8dWv5QwDX/CdIiWsm9r7odrAHHcE4LO67eaeRcL2n+1PH46WLF5yHe0RMXm0h0dRJTRtTsSbOr6wXp1s6KW46kEKcAayjI5qshb1MVix0B3ScLgHCJ3Q00kXX8Xave5E3DNW8jxKfEzO7wv4Pwtu/N2k8VaEzYDGa5+c31pPrVMWA3ytKUzw7PuUh5P7IyUVaqM1zucWcRzrTFodXxjfFETJD2Uz8TppGLkEXW3hv55JIWok5Q+kVnEdex9TtiShvvu/y8JFnTkMQOj104w6+4Q2fJJW9amm++bKXFDyg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(396003)(346002)(39860400002)(136003)(36840700001)(46966006)(83380400001)(8676002)(478600001)(6512007)(81166007)(6486002)(4326008)(6862004)(2616005)(107886003)(54906003)(86362001)(356005)(316002)(36756003)(47076005)(36860700001)(53546011)(6506007)(70206006)(26005)(8936002)(186003)(70586007)(336012)(33656002)(5660300002)(82740400003)(82310400003)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 07:57:24.1927
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3fa7285f-14a1-42b9-96b6-08d96de74cc7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6861

SGkgSnVsaWVuLA0KDQo+IE9uIDEgU2VwIDIwMjEsIGF0IDE5OjMzLCBKdWxpZW4gR3JhbGwgPGp1
bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4gDQo+IEhpIFN0ZWZhbm8sDQo+IA0KPiBPbiAwMS8wOS8y
MDIxIDE4OjU0LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+PiBPbiBXZWQsIDEgU2VwIDIw
MjEsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+PiBPbiAwMS8wOS8yMDIxIDE0OjEwLCBCZXJ0cmFu
ZCBNYXJxdWlzIHdyb3RlOg0KPj4+Pj4gT24gMSBTZXAgMjAyMSwgYXQgMTM6NTUsIEp1bGllbiBH
cmFsbCA8anVsaWVuQHhlbi5vcmc+IHdyb3RlOg0KPj4+Pj4gDQo+Pj4+PiBIaSwNCj4+Pj4+IA0K
Pj4+Pj4gT24gMDEvMDkvMjAyMSAxMzo0MywgTWljaGFsIE9yemVsIHdyb3RlOg0KPj4+Pj4+IElu
dHJvZHVjZSBuZXcgWGVuIGNvbW1hbmQgbGluZSBwYXJhbWV0ZXIgY2FsbGVkICJwbXVfYWNjZXNz
Ii4NCj4+Pj4+PiBUaGUgZGVmYXVsdCB2YWx1ZSBpcyAidHJhcCI6IFhlbiB0cmFwcyBQTVUgYWNj
ZXNzZXMuDQo+Pj4+Pj4gSW4gY2FzZSBvZiBzZXR0aW5nIHBtdV9hY2Nlc3MgdG8gIm5hdGl2ZSIs
IFhlbiBkb2VzIG5vdCB0cmFwDQo+Pj4+Pj4gUE1VIGFjY2Vzc2VzIGFsbG93aW5nIGFsbCB0aGUg
Z3Vlc3RzIHRvIGFjY2VzcyBQTVUgcmVnaXN0ZXJzLg0KPj4+Pj4+IEhvd2V2ZXIsIGd1ZXN0cyBj
YW5ub3QgbWFrZSB1c2Ugb2YgUE1VIG92ZXJmbG93IGludGVycnVwdHMgYXMNCj4+Pj4+PiBQTVUg
dXNlcyBQUEkgd2hpY2ggWGVuIGNhbm5vdCByb3V0ZSB0byBndWVzdHMuDQo+Pj4+Pj4gVGhpcyBv
cHRpb24gaXMgb25seSBpbnRlbmRlZCBmb3IgZGV2ZWxvcG1lbnQgYW5kIHRlc3RpbmcgcHVycG9z
ZXMuDQo+Pj4+Pj4gRG8gbm90IHVzZSB0aGlzIGluIHByb2R1Y3Rpb24gc3lzdGVtLg0KPj4+Pj4g
SSBhbSBhZnJhaWQgeW91ciBvcHRpb24gaXMgbm90IHNhZmUgZXZlbiBpbiBkZXZlbG9wbWVudCBz
eXN0ZW0gYXMgYSB2Q1BVDQo+Pj4+PiBtYXkgbW92ZSBiZXR3ZWVuIHBDUFVzLg0KPj4+Pj4gDQo+
Pj4+PiBIb3dldmVyLCBldmVuIGlmIHdlIHJlc3RyaWN0ZWQgdGhlIHVzZSB0byBwaW5uZWQgdkNQ
VSAqYW5kKiBkZWRpY2F0ZWQNCj4+Pj4+IHBDUFUsIEkgYW0gbm90IGNvbnZpbmNlZCB0aGF0IGV4
cG9zaW5nIGFuIGhhbGYgYmFja2VkIFBNVSAodGhlIG92ZXJmbG93DQo+Pj4+PiBpbnRlcnJ1cHQg
d291bGQgbm90IHdvcmspIHRvIHRoZSBndWVzdCBpcyB0aGUgcmlnaHQgc29sdXRpb24uIFRoaXMg
bGlrZWx5DQo+Pj4+PiBtZWFucyB0aGUgZ3Vlc3QgT1Mgd291bGQgbmVlZCB0byBiZSBtb2RpZmll
ZCBhbmQgdGhlcmVmb3JlIHRoZSB1c2FnZSBvZg0KPj4+Pj4gdGhpcyBvcHRpb24gaXMgZmFpcmx5
IGxpbWl0ZWQuDQo+Pj4+PiANCj4+Pj4+IFNvIEkgdGhpbmsgdGhlIGZpcnN0IHN0ZXBzIGFyZToN
Cj4+Pj4+ICAgMSkgTWFrZSB0aGUgUFBJIHdvcmsuIFRoZXJlIHdhcyBzb21lIGF0dGVtcHQgaW4g
dGhlIHBhc3QgZm9yIGl0IG9uDQo+Pj4+PiB4ZW4tZGV2ZWwuIFlvdSBjb3VsZCBoYXZlIGEgbG9v
ay4NCj4+Pj4+ICAgMikgUHJvdmlkZSBQTVUgYmluZGluZ3MNCj4+Pj4+IA0KPj4+Pj4gV2l0aCB0
aGF0IGluIHBsYWNlLCB3ZSBjYW4gZGlzY3VzcyBob3cgdG8gZXhwb3NlIHRoZSBQTVUgZXZlbiBp
ZiBpdCBpcw0KPj4+Pj4gdW5zYWZlIGluIHNvbWUgY29uZGl0aW9ucy4NCj4+Pj4gDQo+Pj4+IFdp
dGggdGhvc2UgbGltaXRhdGlvbnMsIHVzaW5nIHRoZSBQTVUgdG8gbW9uaXRvciB0aGUgc3lzdGVt
IHBlcmZvcm1hbmNlcyBvcg0KPj4+PiBvbiBzb21lIHNwZWNpZmljIHVzZSBjYXNlcyBpcyBzdGls
bCByZWFsbHkgdXNlZnVsLg0KPj4+PiBXZSBhcmUgdXNpbmcgdGhhdCB0byBkbyBzb21lIGJlbmNo
bWFya3Mgb2YgWGVuIG9yIG9mIHNvbWUgYXBwbGljYXRpb25zIHRvDQo+Pj4+IGNvbXBhcmUgdGhl
IGJlaGF2aW91ciB0byBhIG5hdGl2ZSBzeXN0ZW0gb3INCj4+Pj4gYW5hbHlzZSB0aGUgcGVyZm9y
bWFuY2VzIG9mIFhlbiBpdHNlbGYgKGh5cGVyY2FsbHMsY29udGV4dCBzd2l0Y2gg4oCmZXRjKQ0K
Pj4gSSBhbHNvIGFscmVhZHkgaGFkIHRvIHdyaXRlIGEgcGF0Y2ggYWxtb3N0IGV4YWN0bHkgbGlr
ZSB0aGlzIG9uZSB0bw0KPj4gcHJvdmlkZSB0byBjdXN0b21lcnMgYSBmZXcgbW9udGhzIGJhY2su
DQo+Pj4gSSB1bmRlcnN0YW5kIHRoaXMgaXMgdXNlZnVsIGZvciBzb21lIHNldHVwIGFuZCBJIGFt
IG5vdCB0cnlpbmcgdG8gc2F5IHdlDQo+Pj4gc2hvdWxkIG5vdCBoYXZlIGEgd2F5IHRvIGV4cG9z
ZSB0aGUgUE1VIChldmVuIHVuc2FmZWx5KSBpbiB1cHN0cmVhbS4gSG93ZXZlciwNCj4+PiBJIHRo
aW5rIHRoZSBvcHRpb24gYXMgaXQgc3RhbmRzIGlzIHRvbyB3aWRlICh0aGlzIHNob3VsZCBiZSBh
IHBlciBkb21haW4ga25vYikNCj4+PiBhbmQgd2Ugc2hvdWxkIHByb3Blcmx5IGV4cG9zZSB0aGUg
UE1VIChpbnRlcnJ1cHRzLCBiaW5kaW5ncy4uLikuDQo+PiBJIGhhdmUgbmV2ZXIgdXNlZCBQTVUg
ZGlyZWN0bHkgbXlzZWxmLCBvbmx5IHByb3ZpZGVkIGEgcGF0Y2ggc2ltaWxhciB0bw0KPj4gdGhp
cyBvbmUuICBCdXQgYXMgZmFyIGFzIEkgY291bGQgdGVsbCB0aGUgdXNlcnMgd2VyZSBmdWxseSBz
YXRpc2ZpZWQNCj4+IHdpdGggaXQgYW5kIGl0IGhhZCBubyBpbnRlcnJ1cHRzIHN1cHBvcnQgZWl0
aGVyLiBDb3VsZCBpdCBiZSB0aGF0DQo+PiBpbnRlcnJ1cHRzIGFyZSBub3QgYWN0dWFsbHkgbmVl
ZGVkIHRvIHJlYWQgdGhlIHBlcmYgY291bnRlcnMsIHdoaWNoIGlzDQo+PiBwcm9iYWJseSB3aGF0
IHVzZXJzIGNhcmUgYWJvdXQ/DQo+IA0KPiBZb3UgZG9uJ3QgbmVlZCB0aGUgaW50ZXJydXB0cyB0
byByZWFkIHRoZSBwZXJmIGNvdW50ZXJzLiBCdXQgQUZBSVUsIHlvdSB3b3VsZCBoYXZlIHRvIHBv
bGwgYXQgYSByZWd1bGFyIGludGVydmFsIHlvdXJzZWxmLiBUaGVyZSBpcyBhbHNvIHRoZSBxdWVz
dGlvbiBvbiBob3cgdG8gY2F0Y2ggdGhlIG92ZXJmbG93Pw0KPiANCj4+IEluIHJlZ2FyZHMgdG8g
InRoaXMgc2hvdWxkIGJlIGEgcGVyIGRvbWFpbiBrbm9iIiwgaW4gcmVhbGl0eSBpZiB5b3UgYXJl
DQo+PiBkb2luZyBQTVUgZXhwZXJpbWVudHMgeW91IGRvbid0IGNhcmUgaWYgb25seSBvbmUgb3Ig
YWxsIGRvbWFpbnMgaGF2ZQ0KPj4gYWNjZXNzOiB5b3UgYXJlIHdvcmtpbmcgaW4gYSBjb250cm9s
bGVkIGVudmlyb25tZW50IHRyeWluZyB0byBmaWd1cmUgb3V0DQo+PiBpZiB5b3VyIHNldHVwIG1l
ZXRzIHRoZSB0aW1pbmcgcmVxdWlyZW1lbnRzLiBUaGVyZSBhcmUgbm8gc2VjdXJpdHkgb3INCj4+
IHNhZmV0eSBjb25jZXJucyAodGhvc2UgYXJlIGRpZmZlcmVudCBleHBlcmltZW50cykgYW5kIHRo
ZXJlIGlzIG5vDQo+PiBpbnRlcmZlcmVuY2UgcHJvYmxlbXMgaW4gdGhlIHNlbnNlIG9mIG11bHRp
cGxlIGRvbWFpbnMgdHJ5aW5nIHRvIGFjY2Vzcw0KPj4gUE1VIGF0IHRoZSBzYW1lIHRpbWUgLS0g
eW91IGNvbnRyb2wgdGhlIGRvbWFpbnMgeW91IGRlY2lkZSB3aGljaCBvbmUgaXMNCj4+IGFjY2Vz
c2luZyB0aGVtLiA+DQo+PiBTbyBJIGFtIGluIGZhdm9yIG9mIGEgcGF0Y2ggbGlrZSB0aGlzIG9u
ZSBiZWNhdXNlIGl0IGFjdHVhbGx5IHNhdGlzZnkNCj4+IG91ciByZXF1aXJlbWVudHMuIEV2ZW4g
aWYgd2UgaGFkIHBlci1kb21haW4gc3VwcG9ydCBhbmQgaW50ZXJydXB0cw0KPj4gc3VwcG9ydCwg
SSBkb24ndCB0aGluayB0aGV5IHdvdWxkIGVuZCB1cCBiZWluZyB1c2VkLg0KPiANCj4gSSBoYXZl
IHRvIGRpc2FncmVlIHdpdGggdGhhdC4gVGhlcmUgYXJlIHZhbGlkIHVzZS1jYXNlcyB3aGVyZSB5
b3UgbWF5IHdhbnQgdG8gZXhwb3NlIHRoZSBQTVUgZm9yIHVzaW5nIHBlcmYgdG8gYSBzaW5nbGUg
ZG9tYWluIGJlY2F1c2UgeW91IGtub3cgaXQgaXMgc2FmZSB0byBkbyBzby4gSSBhcHByZWNpYXRl
IHRoaXMgaXMgbWF5IG5vdCBiZSB0aGUgdXNlIGNhc2UgZm9yIHlvdXIgdXNlcnMgKHlldCksIGJ1
dCBJIGhhdmUgc2VlbiAoYW5kIHVzZWQpIGl0IG9uIHg4Ni4gU28gdG8gbWUgdGhpcyBhcHByb2Fj
aCBpcyBzaG9ydC1zaWdoZWQuDQo+IA0KPiBUQkgsIHRoaXMgaXMgbm90IHRoZSBmaXJzdCB0aW1l
IEkgaGF2ZSBzZWVuIHBhdGNoIGZvciAibGV0J3MgZXhwb3NlIHRoZSBzYW1lIGZlYXR1cmUgdG8g
ZXZlcnlvbmUgYmVjYXVzZSB0aGlzIGlzIGVhc3kgdG8gZG8iIGFuZCByZWFsbHkgZGlzbGlrZSBp
dC4gRXhwb3NpbmcgYSBuZXcgZmVhdHVyZSBmcm9tIHRoZSB0b29sc3RhY2sgaXMgZWFzaWVyIHRo
YW4geW91IHRoaW5rIChiYXJpbmcgdGhlIGxhY2sgb2YgcmV2aWV3cyksIHRoaXMgaXMgYSBtYXR0
ZXIgb2YgY3JlYXRpbmcgYSBuZXcgZmxhZyBhIG5ldyBvcHRpb24uIFRoaXMgd291bGQgbWFrZSBY
ZW4gYSBsb3QgbW9yZSBmbGV4aWJsZSBhbmQgZW5hYmxlIG1vcmUgdXNlcnMuLi4NCg0KSWYgSSB1
bmRlcnN0YW5kIGl0IHJpZ2h0LCB5b3Ugd2FudCBhIHBlciBndWVzdCBwYXJhbWV0ZXIgdG8gYmUg
YWJsZSB0byBhbGxvdyBQTVUgYWNjZXNzZXMuDQpUaGlzIHdvdWxkIHJlcXVpcmU6DQotIHRvIHNh
dmUvcmVzdG9yZSBNRENSIG9uIGNvbnRleHQgc3dpdGNoDQotIGludHJvZHVjZSBhIG5ldyBjb25m
aWcgcGFyYW1ldGVyIGZvciBndWVzdHMgKG1pZ2h0IG9yIG1pZ2h0IG5vdCBuZWVkIGEgdG9vbCBj
aGFuZ2UpDQotIGhhdmUgYSB4ZW4gY29tbWFuZCBsaW5lIHBhcmFtZXRlciB0byBoYXZlIGEgc29s
dXRpb24gdG8gQWxsb3cgUE1VIGZvciBkb20wIChvciBtYXliZSBhIERUQiBvbmUpDQoNCkJ1dCB0
aGlzIHdvdWxkIE5PVCBpbmNsdWRlOg0KLSBpbnRlcnJ1cHQgc3VwcG9ydCAob25seSBuZWVkZWQg
dG8gZ2V0IGluZm9ybWVkIG9mIG92ZXJmbG93KQ0KLSBwcm92aWRlIFBNVSB2aXJ0dWFsaXphdGlv
biAobm90IGV2ZW4gc3VyZSBzb21ldGhpbmcgbGlrZSB0aGF0IGNvdWxkIG1ha2UgbXVjaCBzZW5z
ZSkNCg0KSSBhbSBub3Qgc2F5aW5nIHRoYXQgd2Ugd2lsbCBkbyB0aGF0IG5vdyBidXQgYXMgSSBu
ZWVkIHRvIHVuYmxvY2sgdGhpcyBJIGNvdWxkIGV2YWx1YXRlIHRoZSBlZmZvcnQgYW5kIHNlZSBp
ZiBpdCBjb3VsZCBiZSBwb3NzaWJsZSB0byBkbyB0aGlzIGluIHRoZSBmdXR1cmUuDQpJbiB0aGUg
bWVhbnRpbWUgd2Ugd2lsbCBzdGFydCBtYWludGFpbmluZyBhbiBpbnRlcm5hbCBicmFuY2ggd2l0
aCBwYXRjaGVzIHJlZnVzZWQgdXBzdHJlYW0gYXMgdGhpcyBpcyBibG9ja2luZyB1cy4NCg0KS2lu
ZCByZWdhcmRzDQpCZXJ0cmFuZA0KDQo+IA0KPiBTbyBhcyBpdCBzdGFuZHM6DQo+IA0KPiBOQWNr
ZWQtYnk6IEp1bGllbiBHcmFsbCA8amdyYWxsQGFtYXpvbi5jb20+DQo+IA0KPiBDaGVlcnMsDQo+
IA0KPiAtLSANCj4gSnVsaWVuIEdyYWxsDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 07:59:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 07:59:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177110.322332 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhdT-0000Wi-JZ; Thu, 02 Sep 2021 07:59:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177110.322332; Thu, 02 Sep 2021 07:59:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhdT-0000Wb-Gf; Thu, 02 Sep 2021 07:59:47 +0000
Received: by outflank-mailman (input) for mailman id 177110;
 Thu, 02 Sep 2021 07:59:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=C28T=NY=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mLhdS-0000WV-HP
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 07:59:46 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c2a8d726-d574-48be-9489-4c37a4850615;
 Thu, 02 Sep 2021 07:59:44 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 4E9586736F; Thu,  2 Sep 2021 09:59:39 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2a8d726-d574-48be-9489-4c37a4850615
Date: Thu, 2 Sep 2021 09:59:39 +0200
From: Christoph Hellwig <hch@lst.de>
To: Michael Kelley <mikelley@microsoft.com>
Cc: Christoph Hellwig <hch@lst.de>, Tianyu Lan <ltykernel@gmail.com>,
	KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	"wei.liu@kernel.org" <wei.liu@kernel.org>,
	Dexuan Cui <decui@microsoft.com>,
	"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
	"will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
	"jgross@suse.com" <jgross@suse.com>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>,
	"thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>,
	"pgonda@google.com" <pgonda@google.com>,
	"martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>,
	"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
	"aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"rientjes@google.com" <rientjes@google.com>,
	"ardb@kernel.org" <ardb@kernel.org>,
	"iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	vkuznets <vkuznets@redhat.com>,
	"parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: Re: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
Message-ID: <20210902075939.GB14986@lst.de>
References: <20210827172114.414281-1-ltykernel@gmail.com> <20210830120036.GA22005@lst.de> <MWHPR21MB15933503E7C324167CB4132CD7CC9@MWHPR21MB1593.namprd21.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <MWHPR21MB15933503E7C324167CB4132CD7CC9@MWHPR21MB1593.namprd21.prod.outlook.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Aug 31, 2021 at 05:16:19PM +0000, Michael Kelley wrote:
> As a quick overview, I think there are four places where the
> shared_gpa_boundary must be applied to adjust the guest physical
> address that is used.  Each requires mapping a corresponding
> virtual address range.  Here are the four places:
> 
> 1)  The so-called "monitor pages" that are a core communication
> mechanism between the guest and Hyper-V.  These are two single
> pages, and the mapping is handled by calling memremap() for
> each of the two pages.  See Patch 7 of Tianyu's series.

Ah, interesting.

> 3)  The network driver send and receive buffers.  vmap_phys_range()
> should work here.

Actually it won't.  The problem with these buffers is that they are
physically non-contiguous allocations.  We really have two sensible
options:

 1) use vmap_pfn as in the current series.  But in that case I think
    we should get rid of the other mapping created by vmalloc.  I
    though a bit about finding a way to apply the offset in vmalloc
    itself, but I think it would be too invasive to the normal fast
    path.  So the other sub-option would be to allocate the pages
    manually (maybe even using high order allocations to reduce TLB
    pressure) and then remap them
 2) do away with the contiguous kernel mapping entirely.  This means
    the simple memcpy calls become loops over kmap_local_pfn.  As
    I just found out for the send side that would be pretty easy,
    but the receive side would be more work.  We'd also need to check
    the performance implications.

> 4) The swiotlb memory used for bounce buffers.  vmap_phys_range()
> should work here as well.

Or memremap if it works for 1.

> Case #2 above does unusual mapping.  The ring buffer consists of a ring
> buffer header page, followed by one or more pages that are the actual
> ring buffer.  The pages making up the actual ring buffer are mapped
> twice in succession.  For example, if the ring buffer has 4 pages
> (one header page and three ring buffer pages), the contiguous
> virtual mapping must cover these seven pages:  0, 1, 2, 3, 1, 2, 3.
> The duplicate contiguous mapping allows the code that is reading
> or writing the actual ring buffer to not be concerned about wrap-around
> because writing off the end of the ring buffer is automatically
> wrapped-around by the mapping.  The amount of data read or
> written in one batch never exceeds the size of the ring buffer, and
> after a batch is read or written, the read or write indices are adjusted
> to put them back into the range of the first mapping of the actual
> ring buffer pages.  So there's method to the madness, and the
> technique works pretty well.  But this kind of mapping is not
> amenable to using vmap_phys_range().

Hmm.  Can you point me to where this is mapped?  Especially for the
classic non-isolated case where no vmap/vmalloc mapping is involved
at all?


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:06:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:06:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177125.322348 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhjy-0002s7-Tf; Thu, 02 Sep 2021 08:06:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177125.322348; Thu, 02 Sep 2021 08:06:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLhjy-0002s0-Qj; Thu, 02 Sep 2021 08:06:30 +0000
Received: by outflank-mailman (input) for mailman id 177125;
 Thu, 02 Sep 2021 08:06:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhjx-0002rn-0H; Thu, 02 Sep 2021 08:06:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhjw-0008E9-SZ; Thu, 02 Sep 2021 08:06:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhjw-0000tX-Ij; Thu, 02 Sep 2021 08:06:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLhjw-0005JT-ID; Thu, 02 Sep 2021 08:06:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HATam6E7o9YxcybleTMWpz3Qj92MPFuUIzS59hO2zaA=; b=qMLyFr+qOHYk0chz+LJK//Onqg
	J6sNXaXXCmcpGXGapVwvgJHT7k7G0Ur2leWY9QqV/WUrW3qc+Koy0AZ8GnwzEihwK0a6lQCfh/+PO
	s7iWEOVwhuXn/ig6U6M30IY83LIetk5scFSbV91DubgDDEJIeVjMR+pob/EKHMu20CfM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164751-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164751: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 08:06:28 +0000

flight 164751 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164751/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   11 attempts
Testing same since   164740  2021-09-02 02:41:13 Z    0 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:31:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:31:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177132.322363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi7l-0006Va-Ul; Thu, 02 Sep 2021 08:31:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177132.322363; Thu, 02 Sep 2021 08:31:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi7l-0006VT-R2; Thu, 02 Sep 2021 08:31:05 +0000
Received: by outflank-mailman (input) for mailman id 177132;
 Thu, 02 Sep 2021 08:31:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLi7k-0006VN-D1
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:31:04 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7f51fa1d-6613-41a3-9899-677809f43e6d;
 Thu, 02 Sep 2021 08:31:01 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-nYqOpu4tOfSFDLmcRm-NOQ-1; Thu, 02 Sep 2021 10:31:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4189.eurprd04.prod.outlook.com (2603:10a6:803:3d::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 08:30:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:30:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0023.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 08:30:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f51fa1d-6613-41a3-9899-677809f43e6d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571460;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=k4ggceMNvf4IRPKEMkkUaFmoZHmRsMaw7LT5S9Zt76I=;
	b=mLQHJxW/8r4d8ZJyMdCJONtumbTxG8k9KCGMt8810MPppKDctjRBIVSE3DlQRiwSZTLXLK
	LyhsskIwk2iMP6vsevNw+ucox4YO1ruszMpwmFFfgy9/ZEnmZJ5vyBwpigSiQf4jN6NpLP
	h4xXoiSXSwLqx5YzTwc+1B0clv7tCgk=
X-MC-Unique: nYqOpu4tOfSFDLmcRm-NOQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hSZ/CGTR841tgVVIFefnw7W1STvHB8Y87QlEZlFUrN8b/VY2Joq+qHPMM6OsnLGTa0pgaJvHatxUYNdLs7ar7S+232d9+pmJ02owC62nT+Xrep0QopFG7cmgtBX/MNCXpKxUot6I1HK6zswfJ2W2tOXrqKeSyFrtdx6sGjKkNF0bgqjqS+6/l6/CB5i34Y80BnOBYcoctX3lz9880439HgbYpOplHIuY5p1+ovkWl6tWjVpO2TCy9fh+Uumw+OgSY33gdRu1gPbnzDJVh+pX5ihJUEfB5muaq30ZPFAUiVuTJw00IvynV10wYC/HS4BHgfXhVP4s3Lw06LQ6hfQ+Dw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=k4ggceMNvf4IRPKEMkkUaFmoZHmRsMaw7LT5S9Zt76I=;
 b=V9kYmAZo3wOL90yX3vpomujPPRJ8OpE9lwj3O9TihoZCU//Vfn+ZQRpFtfhEv6Dp1DVjuTUNvsDIdjgJX1Ysj2bn4SXEQQuvV/wvsFS1IiMpDZuYfMpUXXRLiymz+n4Os/a4pc5Zoz4Mo3H9gJi5BobmloNQbfC5+riM0ZKmKrnAnEe4PimJJnkxgh1udZwF8wCeaSLXplMfA2hFwiTvxxd89GgHDJ0Vva2N/aJb+FKBNdU3moCzoqjjNGIc7LsPedukr9Pbi/Qk26sXWfl3amMwxJXh4a0vyFaHH6f7nxJpqssWJw5PFlj5iDfM6mVNIU/n1HCHG4ehI1KVdxy+ew==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/6] x86/PVH: Dom0 building adjustments
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Date: Thu, 2 Sep 2021 10:30:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0023.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::35)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1ce5d787-71a9-4a44-7abb-08d96debfbb0
X-MS-TrafficTypeDiagnostic: VI1PR04MB4189:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4189D37111B49187237AA39CB3CE9@VI1PR04MB4189.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0uSdjd5vYEr33F7X5LJOgQobq3EsVDtEBtUCq0tj2aJSoBKjmfmSB9B6u0M0i5rvkWYiaTHGOnM+8o/rFdqCnxHJRkGSliJPt/ZrxEcZwzO9YBF1IU89A4LJAdljbHLY+QqbZCMt9B4gnR84CrYndTEwO+XOQuRAPPA1rXg1dAiFI6HtsqoSmqsaxMw7juAU/a5j1k2TCzOaPert9ILCTb36wv+S7kq3iqjtEyz9HZGukwVhTiIJVhpuH7LIOSd4i9xHso3JqCf+OpdN3eWqqAKeqaLaTfons+H2DDvrAKQaCVHyOI9O1Y0hT3XHnBWcS2EMBelUpzD/EysGo0RmUpLJML4t2BNOFuofVWtxDhULPr3p/S2jkfFY8+RZwd7AFseyFdVwp5cfJlZnhpfro+dmuZYeVNig5Grr3rncLxstvEPdTeS8agcveO+Fd561rVmlFUtVLEovtEfW8eKdE05Lp9GqW8d4XXGLgwESynwLNOXDaUPdBm6aJZ34nFZpotD4rjkqM5/0i3snRw8z/j+FGneVW0Ws/rdALfJJPs1V/RA11ISTzqLG/P0Vah7dB/bj1e70wsQgjHKXv8RTcleiRpM3ilAXC/15Rl6EIfKxeQABGRrXrnJ7hY/l9UYajTFVWTxWmN+j8Kqdu+mKy9rMJ9JC8CfOYCV6eY8nFua4ojVjT5XaKOpSNKquiQr97H6hcWY6nC+voBTtRbbyu6S3o03zeBCY/VEuwkCLKY0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(2906002)(4326008)(508600001)(36756003)(5660300002)(31696002)(31686004)(8676002)(54906003)(38100700002)(956004)(86362001)(2616005)(6486002)(8936002)(66946007)(83380400001)(66556008)(66476007)(316002)(16576012)(186003)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MGo0dUhJZHFqZ0tZTGs5bnExbUFOVFFMUzJCakx0czk1bWhQTENOQWdDSjVL?=
 =?utf-8?B?STlNdWt2elNWYnFQUnVDYnZKWmMzNHUrRzVKZ3pzc0lTcmQ4ZGNMOG1DNWk0?=
 =?utf-8?B?b04xUG5HdHdVOSt3N2xDbGlWWWF1empyTGcySEh3Y0xNTVM0d1VOb2ppdUVP?=
 =?utf-8?B?dG9HRFFRbWRhZEg2T0ZMSExKN0NYQTdudTJlUWFWQU9rSnZzTm5ZaDZGZVBr?=
 =?utf-8?B?cktOYlpZckc3Q0Q3cUFYamllQUtCUHFFSWQ1UWZYc2xhSThHQXN0aEZVRnpz?=
 =?utf-8?B?Q0NKVmVkd0xzZmpwa3FRcHVBUHdnYUMwc3dBcGtJM1JhVnBYV1g1YkpLZ3Zn?=
 =?utf-8?B?V2ZLYnB3UlZNa0V3SmhIY0owalAzRTJPVStFTkhKd2VsU0I5WFBYTnhOTGFS?=
 =?utf-8?B?cGgveEp0akJJSVR0STBaV0ViR3hMQ1p5R2F5VHk1MUhjUTI4cE1GSUcxUVp0?=
 =?utf-8?B?NHM2STBMd0E4QWZvd1VWSTdVazA2QmI0R2U3d3VKazFyV0VKM0ZyT0F1anBx?=
 =?utf-8?B?Rm1NNDRKU2tIeDluUXVMcUwyWjF2bDZrY0NsUFZQcHozTzJPRm5naTFRNGE3?=
 =?utf-8?B?ZlQyeVdKVFloR0p6T2RWdEIyNU55aUw4dHdXUGxHNE56SENTczM1S0krVnhU?=
 =?utf-8?B?Y09XbGNSbWxZbUZyOE5seEVQNzNQdFlNc21kNkV2T241YytHR0JDa3krbDMz?=
 =?utf-8?B?TUVTeVFKcTBIR0RGZEE0RDg3cXpiNmpUNXEvTXhMSjkzWmtCSExraUhlQUEw?=
 =?utf-8?B?dkpNMkRVdmZzTGZvZWRmMUdYajBHNCt6Qzlud1pVK3ArOUZLSlZKZUMvK3NG?=
 =?utf-8?B?Q1dFTUVIeHRLK0NEMVhFdXMrWUFnelVOL2pnNUtHSHIvRGppVXlaM1N5RUZx?=
 =?utf-8?B?ZU84VXJ3V3RTM01Dd01BcTBTRkE4RnlXQkpJbTNmVmtWQStzek14ZE1hU2hB?=
 =?utf-8?B?VjNTblladlhDMkxXUTUwN2V4VDhaN2VYaGpHeW4wN0FMc2ZPWFIzdHU0dkJn?=
 =?utf-8?B?V1RvcGF5Mkl0YTZxcUJqUkNxa0VlZTl1Uk5zU2pvZ3R5dWhMdlV0bk8rSUhk?=
 =?utf-8?B?aUU0MTVMNzlWKzlrS000YVZjL2F1Z1hiU3h3VDQrSW45NWhaRG0rbmVmTk11?=
 =?utf-8?B?RXdhR241SUdHRjhWNnVXd1FudlBJaGZJdFNHOFpyV3h6S3hWMGord2VaVVlq?=
 =?utf-8?B?ekxZUFpGRFYwZis1NWk0ZS9SOFpDYzlndEJKTHRQVFdNdExaNTJJd2lwYm5j?=
 =?utf-8?B?UTZxYWlZNFg2ZkFIcjA4bi9MdFFQQjFMbUhjM3I5czg4VGxJK1VVMmxPVHVP?=
 =?utf-8?B?V1NnK1Jjc0Ntak5iWm9OVlAxMDV1TUZPdmlLMjl4UE44RkhjaEFYWUQrbnVS?=
 =?utf-8?B?OGlwdDFQQjZyNU5uUmpWczM5M0p3Mi9QbmthdnUyalI0alRCbW4vVmZ6ZEdT?=
 =?utf-8?B?K0hnM2NSam9tci9zbjFFOUlLaDB1SzljcHZhVG94UGhGbDBwVEwvQnM1a05L?=
 =?utf-8?B?b0UrYXNDVU9pVWxnSGNHQUQwZTlRTzdRQnpvTEVMM0J3RDBqa3h6ZVhTbmRr?=
 =?utf-8?B?NXZ1K2dNbzVETldGQWk2QnpFZG10STMxRmFtYWxxd2hLOE1VODJ1aUdKT1pO?=
 =?utf-8?B?OVhEbUJvOThBZXFCaVd2SDVvSHMxeEhoL1QwN1FMbE84TldKOUVHQkxFN0RW?=
 =?utf-8?B?azRxaVFPSU1JRFBRWU1kdmxoWWtTajRUYm96U3RxdXBDeEI5VThSTFlzSGND?=
 =?utf-8?Q?Q5tzl4rNQ71g5RGrPnION4UxC73S5sgLWC2tQvf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ce5d787-71a9-4a44-7abb-08d96debfbb0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:30:55.8544
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: u7AZYy9lWD23se5Xc7Bkhdkb2YayuXgUtI3BGPbHnnjjIaie4psptz7UenHTFi4jjlS8pHCQM4zMK0JBc/QAKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4189

The code building PVH Dom0 made use of sequences of P2M changes
which are disallowed as of XSA-378. First of all population of the
first Mb of memory needs to be redone. Then, largely as a
workaround, checking introduced by XSA-378 needs to be slightly
relaxed.

Note that with these adjustments I get Dom0 to start booting on my
development system, but the Dom0 kernel then gets stuck. Since it
was the first time for me to try PVH Dom0 in this context (see
below for why I was hesitant), I cannot tell yet whether this is
due further fallout from the XSA, or some further unrelated
problem. Dom0's BSP makes it all the way through to entering the
idle loop while all APs are still in VPF_down.

[And there was another rather basic issue to fight first (patch
 was submitted separately as RFC): vPCI wasn't aware of hidden PCI
 devices, hitting an ASSERT(). Obviously I couldn't afford not
 having a functioning serial console.]

In the course I ran into an oom condition while populating Dom0's
RAM. Hence next some re-work of dom0_compute_nr_pages(). In turn
in the course of putting that together I did notice that PV Dom0,
when run in shadow mode, wouldn't have its shadow allocation
properly set.

Finally make debug key '0' work at least partially for PVH Dom0
and limit the amount of log output for offline (down) vCPU-s.

1: P2M: relax guarding of MMIO entries
2: P2M: relax permissions of PVH Dom0's MMIO entries
3: PVH: improve Dom0 memory size calculation
4: PV: properly set shadow allocation for Dom0
5: PVH: actually show Dom0's register state from debug key '0'
6: HVM: skip offline vCPU-s when dumping VMCBs/VMCSes

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:32:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:32:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177138.322373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi8y-0007OC-8K; Thu, 02 Sep 2021 08:32:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177138.322373; Thu, 02 Sep 2021 08:32:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi8y-0007O5-5F; Thu, 02 Sep 2021 08:32:20 +0000
Received: by outflank-mailman (input) for mailman id 177138;
 Thu, 02 Sep 2021 08:32:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLi8x-0007Nz-MB
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:32:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b54a0351-d668-400f-9754-8cb4883e52f5;
 Thu, 02 Sep 2021 08:32:18 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-8rI5qsy5Ot-aOx5yvDc6Lg-1; Thu, 02 Sep 2021 10:32:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4189.eurprd04.prod.outlook.com (2603:10a6:803:3d::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 08:32:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:32:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0084.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 08:32:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b54a0351-d668-400f-9754-8cb4883e52f5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571537;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FuErrn7dPOKk18p14Eqqw0Ug5qQHJVZRhPNfvjkAXCc=;
	b=L94RNv+6FlEjlgqvSwvSNstDsWDSyMyRbG69/F/2G3hwkGiz7CL6wz4XOAQFJtexf0gLSI
	JMjLtEE9DrYwRRD9aKcwT+yig234ilgbDK05hmv4kKHIS6bWn9nG0EKBLRwa9Ux82KylSp
	892ol2c0NK2OfCKXuICW8QIFODNWZn4=
X-MC-Unique: 8rI5qsy5Ot-aOx5yvDc6Lg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=esop9D2vOOSfKjxkqZ2QTy/dXVlj+3dCmsXtOrdkgM9Xd04dHZP1kfcLV5luhYvDUSA5pF8SG+oBfVk/uAmFTICChATsNPEZKNpCCALNnzQYiGYIOyWeMAYGoWFE77PGivScaWBk2vEil2SkqeE/wKZh+7T732+gtWPwfT/zkQwWVmxAP/CYHtsN/br42smHMGCieMO85HTfiReerk8mqA0rDKxSpDeO0khv8QJC0OcUk6K4gJtr+iQboeXhNsRb4ua2vggiPVpcmnx4vMnkw1CwRGbuQy+Mf1kWvp9r5Nvbp8Orom/+IDXPWSg3I5ySyqza17cEG5VAW3EQxqFlDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FuErrn7dPOKk18p14Eqqw0Ug5qQHJVZRhPNfvjkAXCc=;
 b=QqHhhd6RNunR/gm9IEcERf6hsRrZ91u49seIlwSyduy/tgvlyY9L98qBJVFOaKS2JIEkvEVJiHfmWBnqArM4yK+Er4Ub9eTn/0mjgKStW7AHEhberdHR3XraM6cmUPuPDDY5LA60qLszeodZqMhtpmwxNc3jxslCrWbMBOcthES5KwTbsp0FasEx46tvfPRR98w98OITl+F8utSupwpVPSU0F16swNPILpDJANAg4OWbzi3nyGUDGTmnTRj6f/4DYl/OHhXBaZ9cX9F8MEePiYiaDxRbw7anKMMu+X2BXdGq2b+RyJpR65p9rzVg551Kinho6UJrzSn8V71iiDGL6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 1/6] x86/P2M: relax guarding of MMIO entries
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <9ab471d8-877c-b569-96c3-c3145800adcf@suse.com>
Date: Thu, 2 Sep 2021 10:32:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0084.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 45c4cfa0-5137-4660-cee9-08d96dec296a
X-MS-TrafficTypeDiagnostic: VI1PR04MB4189:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB418960DA18FD87F9884E86F6B3CE9@VI1PR04MB4189.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SkzESaDIGIt8nj7pKgXEjA05Kg+86BAjcW2RYiEawFWSTTDVrX/jSD9X1x1ktOkTveqImF2eOieqMJ4Bpu413IdjI6y6vsMGfAah7BnHMXSlIkV/Zemr1f776d1Pcy5SyD2/plKhIwOnMRL75DUCuuu9cE54ce3NS0swsnmy3BxYnRNHXTdbeneu0LgdF3v6EoivRCJptvWjY39wPmfHwRLdR23vn8y7opqLYe5m+vLCC/HFM6yy0Tl6Hatl4o6mgM//ObQNO4PgnujK+03/wqmCWsn7iGP99ETPASC2mBhLCUNvmA8tFsciRjJIXhxBalfwGLmVzF/x1u2OPnTw9KTxXe/M2vsqwtjH0tkN+JCn/1USL/Sxr8TRB+eZy7ZbCFkA8wcSPWdTQUvrMu4VLHQBoPQjkR/13KihYCxe3FYhp9fkdwRdhRM+PfEHdWtZH+Tylr9ydY4qvaxx0iUrSs6T7H94/SMfpQb9a8qarLtN8l6nY29YUeTJrrRR2ipeT8YaBckvJiwTOKaTTfotM8rlvFhEFGsoREU/cSv5P/Rey9PYaxzfwpNV1EiBO9XA/VkGEeTv2W6G5MKNEExRFJvIGbszNtIvcMfR/4xIqNzoUBSpeL4xnpda7Xq+qHg4w5R9WqUuSk9WxsgCstxhoRqmTF5htOYXvzn7FwdcMcXuyY1MTda73NrLPgkO0UfO9cSHbrqaRDYk6Z9r3d6iJqRRdzBycGozfHux/Yggbvg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(2906002)(4326008)(508600001)(36756003)(5660300002)(31696002)(31686004)(8676002)(54906003)(38100700002)(956004)(86362001)(2616005)(6486002)(8936002)(66946007)(83380400001)(66556008)(66476007)(316002)(16576012)(186003)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cGJ1ZVNZSytLZ1VLSnFWdXM2WWxsbWdpcmNoZ2lRT3g3Nit5c2Y0V3FUTy9P?=
 =?utf-8?B?ZmtUNlR3NGhzK01wajE0c0tJWEt2MWp6WUhVb2pWVmswR3NQRllrb2pVVWpD?=
 =?utf-8?B?dFVoSXB1WXhkY083cHg5Z3FtMkg2SmFhVDJzeS9PN1ZkQ2VUSzYwRnlOTFQy?=
 =?utf-8?B?T2MwNGlHd09HS3djNG52enl5RjF4UkRYT0puYTFtRlN6enJxTFFnUjQ1eVB1?=
 =?utf-8?B?WVUrZVNsek52MXZ6ZFFYUm5aU3RsbG5QUnBVNzF2UlNJazNwRXdTQ2xyQ05O?=
 =?utf-8?B?YjUyZUZ3UFRSYVFQdG1lMXhJZWJtcDVZb2ZWQTMvZFhUYjQyODRnS0dmSmh4?=
 =?utf-8?B?b1ErcTdTOEVNMmNZeTZicjFpNFMwTERXZFdBZjBXekRqNi9Malo0cUxiQ3Vr?=
 =?utf-8?B?dXNBT2pDbFJIa3VXbmpHN1lIaFRkQlplZjNEaTFMZDQ5NUJVNXA3aWpQMFhp?=
 =?utf-8?B?Vk11dEh3UVVJbHRJdmJId1JjSVBpdFFwSXU1OEk1WGMrOG1zOW1aZC9pOUhh?=
 =?utf-8?B?ZXJVRjBFT2RsUWdzY1pSbzRoUHh6RHQ4YVQxRVo5UngvWUxSWmFubEg0TXRW?=
 =?utf-8?B?TU9acWg2cDRFejVUMTBqV3VPOHUwSHZaOW4wcThRNUp4SFY5ZG9hNUdCWkF0?=
 =?utf-8?B?MEt4SzNtaVc0ZHpiYXJERGRaQTlSTXk2RlVHcFFabGYwL29XaUpXSmlqa3NC?=
 =?utf-8?B?OE5TanZNSEdFRUhUenFBRjZkNmJ2UUdNaWQrWXpldjRJbVc2REk2VDBCdW0x?=
 =?utf-8?B?RzZPYUR0RDhmRGR2b0l2MzFVTHNzT3lVQ2lwWFRnU3JTbUtzT00xNGVZbTd6?=
 =?utf-8?B?Z21KZTd1VUo4UFA2WVN3ZGpnS1FvTXVzKzQ0YURNT0ZVUFA5Y3pDRWl6ak9R?=
 =?utf-8?B?OC9pOUdzdE8xTHpiZGRHYXBqT0ZWZ1kvU0dsUzlOV3dwcnNtNU90ZUZOZTdR?=
 =?utf-8?B?Q2F5TnV4cncwOFU1dlp0V1pRbGkyYlNCbW1HWkNNNDcvT2lZQWtYNUo2dDNt?=
 =?utf-8?B?UEhJREdaQUJINjBiZkFHei9MOHU5dmhkVFBUWXBCbXBVTU5kcU1GdU92Q0NB?=
 =?utf-8?B?eE5aNXFnZmhRNElpQTRqazR5dE9xSEc0TEdzdy9sQ1VCeVdNZk5KTzdKWjZx?=
 =?utf-8?B?VnB1a1EwajMyd1dIRWtQRWl4ajk1VmwzQ0ZZOFlnKy9IblpBU3FscnVDTHZt?=
 =?utf-8?B?dkxFY0VNZlRuOWJwaWdUTWFoUzB6S3VEMkV6eXN0RDdSOTJJVENua1p3NFBB?=
 =?utf-8?B?VUdaVnJDZ1h2bmpKVERuZHRObVdNeCtxZzF3QU1iU2lRZHdJQmdJNTAzempj?=
 =?utf-8?B?UnBwcGlwdFVXbWtNbytqTm41ZU90UFpjYkFxUkJLb3NKQnhmNmxCYUdYODJr?=
 =?utf-8?B?Q0JmVnBtTUJkdXN1SDVlcXljVXBSaExRNENqWVI3QThFQmRPVUJmNG9iZ0Rp?=
 =?utf-8?B?eDhlZVBHaEtuS21JbDIrTHAySjFOSU1jOHVQc2hOcGpPT2I2RW5mdDJWUVRB?=
 =?utf-8?B?dXpiQkJRdmd6UitxM2s0QmFib0VHanBoZGs5bTQ0Z1VlZTNyeUlZeWFOSzhw?=
 =?utf-8?B?T25sMXB2SEdJRU9pMkc5MEhxU1JCNDRkY2hrMW5XSitrSHozaXBLdENkOWZV?=
 =?utf-8?B?cWVmcVlWaGEwWjBweThOOXo0dlNrbVNPVzdaeVdyZ05iZlpPMmxZSERuLzQ2?=
 =?utf-8?B?ZWVMaE92eGorblVJMzZTNHA5eUFScXNXMDc4NnBiRkpTR3Nsd3c0R2tnR3VB?=
 =?utf-8?Q?LOoB/m6LOyENjS5F6O1LWLoKb5oy6g5nVMc+Mrh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 45c4cfa0-5137-4660-cee9-08d96dec296a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:32:12.5660
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iE6TS2myTByzrqwPsYAy2o/vaqD1rHTAnFsHPyhCBGF25NAY+v5+1iD6JG/zBJizAnixdGooglWxiGFbxHTLvg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4189

One of the changes comprising the fixes for XSA-378 disallows replacing
MMIO mappings by code paths not intended for this purpose. At least in
the case of PVH Dom0 hitting an RMRR covered by an E820 ACPI region,
this is too strict. Generally short-circuit requests establishing the
same kind of mapping that's already in place, while otherwise adjusting
permissions without - as before - allowing MFN or type to change.

While there, also add a log message to the other domain_crash()
invocation that did prevent PVH Dom0 from coming up after the XSA-378
changes.

Fixes: 753cb68e6530 ("x86/p2m: guard (in particular) identity mapping entries")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I may have gone too far by allowing "access" to change for all special
types now.
---
v2: Format string and comment adjustments. Split off access
    accumulation.

--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -958,9 +958,13 @@ guest_physmap_add_entry(struct domain *d
         if ( p2m_is_special(ot) )
         {
             /* Don't permit unmapping grant/foreign/direct-MMIO this way. */
-            domain_crash(d);
             p2m_unlock(p2m);
-            
+            printk(XENLOG_G_ERR
+                   "%pd: GFN %#lx (%#lx,%u,%u) -> (%#lx,%u,%u) not permitted\n",
+                   d, gfn_x(gfn) + i,
+                   mfn_x(omfn), ot, a,
+                   mfn_x(mfn) + i, t, p2m->default_access);
+            domain_crash(d);
             return -EPERM;
         }
         else if ( p2m_is_ram(ot) && !p2m_is_paged(ot) )
@@ -1302,9 +1306,24 @@ static int set_typed_p2m_entry(struct do
     }
     if ( p2m_is_special(ot) )
     {
-        gfn_unlock(p2m, gfn, order);
-        domain_crash(d);
-        return -EPERM;
+        /* Special-case (almost) identical mappings. */
+        if ( !mfn_eq(mfn, omfn) || gfn_p2mt != ot )
+        {
+            gfn_unlock(p2m, gfn, order);
+            printk(XENLOG_G_ERR
+                   "%pd: GFN %#lx (%#lx,%u,%u,%u) -> (%#lx,%u,%u,%u) not permitted\n",
+                   d, gfn_l,
+                   mfn_x(omfn), cur_order, ot, a,
+                   mfn_x(mfn), order, gfn_p2mt, access);
+            domain_crash(d);
+            return -EPERM;
+        }
+
+        if ( access == a )
+        {
+            gfn_unlock(p2m, gfn, order);
+            return 0;
+        }
     }
     else if ( p2m_is_ram(ot) )
     {



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:33:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:33:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177145.322385 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi9n-00082X-N9; Thu, 02 Sep 2021 08:33:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177145.322385; Thu, 02 Sep 2021 08:33:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLi9n-00082Q-Jn; Thu, 02 Sep 2021 08:33:11 +0000
Received: by outflank-mailman (input) for mailman id 177145;
 Thu, 02 Sep 2021 08:33:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLi9m-00082K-JB
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:33:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 66e249ba-0bc8-11ec-ae46-12813bfff9fa;
 Thu, 02 Sep 2021 08:33:09 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-HQDKnutlP9Km-SFC1oMmNw-1; Thu, 02 Sep 2021 10:33:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Thu, 2 Sep
 2021 08:33:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:33:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0075.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 08:33:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66e249ba-0bc8-11ec-ae46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571588;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=QGrOOS+VJ3eASvPOqSojyeVdMYBWoCT8pMRR8J+S6bQ=;
	b=SVBPpEgrtnX1KUYJqZY2JwSJUnF4mUAbNzM6nsOt0d10pmYgHKRTm35RIenERIk9HMpP9j
	a98zUH/+5ZzqVrrmJHY9/Zs+EAzyyseFYEW0IM5986p2xlJms1kZTU0YmlWy+osVq2AuBy
	D2A13sM7On93SO9yyCkTD+Jhwheg3qg=
X-MC-Unique: HQDKnutlP9Km-SFC1oMmNw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rh+lzgizB+Oaq/xwxm81ndeEY5ORQ+As9I0bSFEGtmWiwqTD7aW04BhDWaDWCaEm6Vsm7DvzaKDWtJEXBKfWn8tG8PP6MJobsDPyvZ+oi5idAarrhvsb0YXL47BIsVDvJSjp7H0nOpuvWP7yc4k2IsjCrxLS4icROuBzBtdFzNKjQmkw7xOJwExf2M25ruCDpQZEZ5iQSH7b6HOq/O9rKowuy6r7Kt5ui61Is1GIRgF4WGd6wz0eoP2XlPw9Z7PwDjxhUUVfsLtkDazVD0PLuasQ1zwj7EjbM2gOijfR6rtQkSSAYQr7VB6Ii/BP/gJxi0wFD6C8TrUuTT1b2eQsLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QGrOOS+VJ3eASvPOqSojyeVdMYBWoCT8pMRR8J+S6bQ=;
 b=UVGP2pIZ9hKKbLUDr8rzSB9KEcyrOzfHAHCDn1SbpjIX/hhqQ6rw5sz6Tfkq7umAVGlUkXXryl0NgCFWFfkwCD5RA1xCciaPGNgmJlhLyalDZ/jSgnsi6DEA9SmH/mqkNZHj7AijtHgA0to8DyO+WssVG7f7NWN+TYIWB+nT8DSqHMmyEoRGpOG2z+03c1UpyKzLJr2uTBbehqHovGTul+mMvGDd+kdbc4sRe8RhZv0hh6YrCoA9OvOM02LhdoXE1CLzbE6eEdtOzo562DLh7y17nwCD/cS+JhNhCnhIT3ro/wOiOy9hbv84PKl2lI1B8RuFbQ7qWR5BM4JAEagrKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 2/6] x86/P2M: relax permissions of PVH Dom0's MMIO entries
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <457637e2-d35c-18d0-ead8-0b513b257927@suse.com>
Date: Thu, 2 Sep 2021 10:33:04 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0075.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e20ac382-f2fa-4d49-97f6-08d96dec4958
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68621B6F6E7C3A6FABFDBD12B3CE9@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lFdn9aPOLdawBoXCVdE4Nz9eCCGK+Q5syenMA6zgtSACw4CgOMZ/YufsaCugC9GOgE2yxUHiwvmrRLjJIpk7SrJfRxEjeDYuMize4YXW3cr7buL3Iop6z5OORouZtj5vN5SDgzMpvhwdkM1cFhXAkVdp3a5yF9OZfAQYkwk5Iy/PKjsdbKxq3Vz/kDgaJftsgdaGFfbQvaUAOPMqGXFGijiYa7CYxqmEANfMq+IxiC88ZIya9tVp4RsXDLL/XDUwG23NcBJXFXssQJAxZ9AynQnv1MZJ5tmDG5zMW0s0qEIKZHWnn8E2N1sjBTIozZ0cYeJjXmEzTVAtDruTVXhnnesUSpbpwNHm7SFXt2dAzd8EirLE+wvdWQH42Sr5FpPzUTrEU+btt2GhXMC7nXekxP7f1SlfLK21nlsPzLtphBTjzxaaZaIeAhFQ1sAIdGUnI8A/Duh2rhzKscRjSms9IGsNOM9Tb+EUwbQpCKSvllI/18ZYSzikFa1w+VuVfgtH6hWalvV/heF34sZVoDaFYsuCwcPzNh35vsOlqDyFfVk2ZDe/+yLoHI4J2fyGYp6QcU+9poP6zFLQlWLhpGAH0pMjXPXLApdgmSXM49WjFZcYxdeAfzq/oGr5yiW1MvcEjxOLYNUXBLAqRebxyuc7pV8Zs86Q4YRrhCla4j07UdHckR6+sWpQ9mXZ/wFqWa9aNmVX4GMdlXPkRkxOmdC5kuR9q6z97KYTyw1GTqOhcJc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(366004)(396003)(346002)(2906002)(8676002)(5660300002)(2616005)(83380400001)(8936002)(478600001)(54906003)(316002)(36756003)(16576012)(38100700002)(956004)(31696002)(66946007)(186003)(31686004)(66556008)(6916009)(66476007)(4326008)(6486002)(26005)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bVZmZGxYOVBhZUpKZ00zbDhpS0dTakNNbjNYL3hXdis0T3BZZDhnWEhXN01w?=
 =?utf-8?B?aVIxS25pbGdLUlFyTHdieEhkSDlvcHBjV3JyVmdQQXh2ZVQvVGdOTDV2V3hI?=
 =?utf-8?B?Sy9zTmpDbjRBbEVIdVVRc1QrUGlWU1k5c2huQnFucWt0S2xKS1ZPbmZEM0Uv?=
 =?utf-8?B?cmpoSlgyNUJwV1RMaHcxL0RYV0ViOHNuT0NHR0IzUHZpUUZLZ21HOWQwTjVl?=
 =?utf-8?B?ZWtCRWRZZnkva3J0M1Q1Y1lqMmw4eW05WHBUckZwdHU5VklISUMxcFhEeWRr?=
 =?utf-8?B?dXFwWkZGUDFyUDM1Z2VkeVE1a1NMdVNHNW8vWWhFTncwcTdFN3FoQndjUk5l?=
 =?utf-8?B?V2UyMVAyMnVGZnp5Y3NkaFBIeFhGTFdCRXV6SjRKWHNhOU14VVBiR3dHc3ZQ?=
 =?utf-8?B?TVJRWXdtWlgxVVdCSUZ5NmJTdnFKMHErK2lZbjZLSGthN3ZIQStWZi9kcDBQ?=
 =?utf-8?B?NlBuZVVaaTBwSHRHVWdMVXI4dGVzelA1OEhuazE3bkM3TXhwNi9YK2JHK3dU?=
 =?utf-8?B?enFhNWNaS0F0eGhFUUw4VmJnWmR1YXZmYXpEY3NLTTRPY1BvRGs4S0ZSN3hP?=
 =?utf-8?B?V0dVV0ozTVZ3c0R6eEVnRG55WitiZFZZV2xRMnVZVWo2V2R2L0kwTXl4SEtz?=
 =?utf-8?B?ZVVFT0E0VXhUemp1UWQ4NjgyVFo4NW1RQkR2QUkrb2FGVUVsRWJnWGVZVDFH?=
 =?utf-8?B?dm1DWG9xa1B4akpUVTlIREJCN2FhVjZib29mM09jK09NcFFobmtXTm9zZnJ3?=
 =?utf-8?B?TllaTm5TdWFSY1o5aUdNckpPdnZBalFGa2xENnZaa1l5QzQyQjFqVEVhbTBP?=
 =?utf-8?B?SzlwblM3ZFQ1aWNLMDNVa1ArUWFTaTZCQ29qYS9JU01UTjVmN1pxd0s1YkJy?=
 =?utf-8?B?Qk5HaEtCWkhQb1F2ZkRJNWU0K2h4Z1NwcllJcWxMaUg3NHpURGZIUWp0RE54?=
 =?utf-8?B?WDBLalQ1QTlCcnJOMld6SFBDNzN0QVpxVVhKczI0YWxwdWo5Q0tyV2JwWElp?=
 =?utf-8?B?UnRhS2MxODVNUGk5M0NGUjV3MEVRVGFDdGlwY1NwQ2lnWFJkZWg1MEQ4QzVM?=
 =?utf-8?B?aWpENGF3MmpoVERLaGdidndMTCtsNlhGa1F0OEFaUWtFb2lJcy83aTZkakl1?=
 =?utf-8?B?ZHd5elp3UGhHZXIxMVlZaWhXTjNsWmxia3VIZkN3b01SbTN2V1p3Z2tuWkFp?=
 =?utf-8?B?eXlrSC9uZTdwVFZVU2g4WjREL3lQclZwWkd2NElCbUphMDIvUUh4TTE4OXl3?=
 =?utf-8?B?YTRwb05pS2t2WGRGcENZQ0tQaUdTU2hOeUVDVUlqZWFrMENvWkdsTUIvdGxi?=
 =?utf-8?B?d2xTcEZ6NVRnOU5Pakt1U1k0ZGF6Tkp0ZmpJMHYzQzJYNGhyVGxzWXZDeExs?=
 =?utf-8?B?V2JscGtja2k5YzZrajJCTGRkcDBoOTdESWR6eVFKZGoxQ3VFWW1NMUlMVDhj?=
 =?utf-8?B?ODdpNmVZMHNSZGowL1hVMkYxU2Rob0cvUDNlV3hHRE1Ld1RJOUdOOVBkb09S?=
 =?utf-8?B?TldkNGZRN0MxL0hxenNCeFNSMnZuREZ1QkRzbnYyVEx6b3BuaGtPS2JZZkQ4?=
 =?utf-8?B?YUIvK0doZ1JoU0pUVWFWc25jV1lsVWZIVzJpTVZTUHV0QlBWdEh0YXZQdm1W?=
 =?utf-8?B?ak9kYU5uS25QaWNjclJwNW42RStTL0hiWllNTEhpVkRoSGp3eWlDYmZuaUR6?=
 =?utf-8?B?WTBtbEl4aFpjRGZQbjE2M3pseElBYWEyaEJyNElqV0pCQWZpbDhmMWJVaHd1?=
 =?utf-8?Q?xPb68Wz7OitJcxLikf7f3Pb2bAjFz5e+VFtj2a6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e20ac382-f2fa-4d49-97f6-08d96dec4958
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:33:06.1307
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CdqcVrJ/MlU3FZzJXiIeI4nU2pwKHFML8YKsvPkMmdLQ7NWSPpTIA/IW5Iv1WT+ch2RGs5a8DF4tOaleKOlJRQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

To become independent of the sequence of mapping operations, permit
"access" to accumulate for Dom0, noting that there's not going to be an
introspection agent for it which this might interfere with. While e.g.
ideally only ROM regions would get mapped with X set, getting there is
quite a bit of work. Plus the use of p2m_access_* here is abusive in the
first place.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Split off from original patch.

--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
             return -EPERM;
         }
 
+        /*
+         * Gross bodge, to go away again rather sooner than later:
+         *
+         * For MMIO allow access permissions to accumulate, but only for Dom0.
+         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
+         * the way they specify "access", this will allow the ultimate result
+         * be independent of the sequence of operations.
+         */
+        if ( is_hardware_domain(d) && gfn_p2mt == p2m_mmio_direct &&
+             access <= p2m_access_rwx && a <= p2m_access_rwx )
+            access |= a;
+
         if ( access == a )
         {
             gfn_unlock(p2m, gfn, order);



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:33:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:33:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177151.322395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiAR-0000Bk-0h; Thu, 02 Sep 2021 08:33:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177151.322395; Thu, 02 Sep 2021 08:33:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiAQ-0000BU-Tx; Thu, 02 Sep 2021 08:33:50 +0000
Received: by outflank-mailman (input) for mailman id 177151;
 Thu, 02 Sep 2021 08:33:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLiAP-0000BK-Kj
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:33:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7e10e197-0bc8-11ec-ae46-12813bfff9fa;
 Thu, 02 Sep 2021 08:33:48 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2057.outbound.protection.outlook.com [104.47.0.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-U9UFGGm3PHCgDY-C_PZclA-1; Thu, 02 Sep 2021 10:33:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4382.eurprd04.prod.outlook.com (2603:10a6:803:73::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17; Thu, 2 Sep
 2021 08:33:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:33:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0213.eurprd02.prod.outlook.com (2603:10a6:20b:28f::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 08:33:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e10e197-0bc8-11ec-ae46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571627;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4Vo4Dkwt5tDPO7OINdISKIs97Cf1OIMtewg4ko0cXdk=;
	b=nN9SvvOIay6vC30EvNHJtrgaqAJUmMe2KpBr8+d+DbjQ80RtRAeBbokhnvLMK7omu6yIJZ
	gH2rBnJ/owZXsdJC5rmFx/R5jzJ0BZAMnEeWSMDJ/5D/RoxM+LgTutZF2dxhYDKMqE1+F3
	4Ky64GnFDn7C4LBQ+JvyUlZnxNBByWE=
X-MC-Unique: U9UFGGm3PHCgDY-C_PZclA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cJCmyBG9TXdvHXI8t+c3lNnbSqceaqc/oOMmaeofSybpK6E/5MjKHD2pdmGbWfOXAtECd2yY6lh0x1XnnrIVjqpuVaZa6W3iiI+sjf35WBRrxRFTO8ZmqHb9D9YMngnamdvJgXIxgoyEauGVpuvcK+eqgNUX2mFQb/JZO33gKuXlGvQqy0Lyx50iBjHiKenyXSnBd8QKR60sbV4Uv/SgimXVvCyj/SiIKnbAC59sL/Zmjo1X0BgRJp6LGyo/GD9o0AeAx61hkMXo9hHvQDmikiTd/oUlXNfTH6jjRAbzl2MAtcudNTH8jrid/0hWKqjaBdg0kRCKSOx6lKkkO/Oecw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4Vo4Dkwt5tDPO7OINdISKIs97Cf1OIMtewg4ko0cXdk=;
 b=YkvDIDmpb4098GeRj2v/7Ht7iK4JAghUIsJipMxpDiFxVpPhc4z0XEhYJq3Le/JSsQ9ojRYweSH7ymUa6MT2UZkFeXKdxeky/6g2PEjwjJUanhUvIUBROrM9/ORs3mIh4da0Db28x85gM66mUOODF0vG/lBbE1iRaHiSbUa2as9pAVPe+1gOUgkJknp4tLDKqIxf9ZFl+7vksZluI0Q1SgblexiJNwlMkgeF64bopcbzNCSsH5blMXZ16O8s7lA0Txh+QNEtj5jP0AQ2t5SIG+X4g5yg+2vDhC3gLI24dX0mh1rKA60HtjHOavHvgLYVwMie116I3O91Vn74M/MQgQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 3/6] x86/PVH: improve Dom0 memory size calculation
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <98d4f692-5287-fe40-c8b7-4d8bea933d50@suse.com>
Date: Thu, 2 Sep 2021 10:33:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0213.eurprd02.prod.outlook.com
 (2603:10a6:20b:28f::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 526246b0-1ff0-4e2c-3d5e-08d96dec60d9
X-MS-TrafficTypeDiagnostic: VI1PR04MB4382:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4382D361F0ECF521DB20CB9CB3CE9@VI1PR04MB4382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PLhfQ33scBgxDMfhItDtpr+1CdlEEg74uAtdMIiLxDFiO5dqJGnF29DzuukX/pXEqhjKPVuAQ6Ec94oFLHBUFFINr30hGL6dNDpmJYsr5XQB4rykjrLAWv13uwivMXcqsJo2pWPIT3QYySqGTKQahuTetOGCADZwokMc4qGpMtAh2Nf1NtzSQVlJ5V0M1zwvzmQz2ZGUWrCYbn3Y3+mPnyMy4GiWv6/CKnqo60GdjaptkSKfcv/V8Sgel/ZPUzxsjnSxUT4uw+guVg78mHaT8+PcYdi4Gm9EWEs4WWFjVKg+L/iRfQ24O9pEdP3MeHy3F47rPHGHU9pJUxmK9AV3rlukeHPkwW789r+8koQHMJZhbqHtfaiObLJs//B3HyD2vzIQONy2xjt3ni45KDug4ayqsCn9YeRRdVFKDluPJoK8x5qrzFiPUMqpcxFdpdG0CtPmn3RUd1c6n8unYHRGDFsNrnkgEBcuesw7jZr2JEXp9rIDlDvil8xdW0D71gGekLdvu2D/d/1dlSvtIe35YXucwMxMpkXpF7mIVmh5lkkGazMNQ2CTwZ835IxQKp5Of3z90YA/jrqCUgktgk9kpVAWdp5nQsC22WtD0PPX+wcd4Gp9vWDXo3HymetIswAd1+eyQsgdujNew0B8VnBnbf78ZSsSORtbASPJv/5f8hr+ucpfjcqBLHhuK/Uqj0hNmJhtQYU3WiRjvXJUQ14N/qZRzylNG14n/zLfRPw6rBpXOw61rNO/hlMKoVZ99JvQ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(396003)(376002)(346002)(31696002)(66476007)(2906002)(186003)(66946007)(38100700002)(26005)(66556008)(5660300002)(478600001)(4326008)(31686004)(36756003)(8676002)(6486002)(316002)(16576012)(6916009)(8936002)(54906003)(956004)(2616005)(83380400001)(86362001)(25903002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WG1rVUdOZHRHYlc4TTBiZHBNSkIzVGsxWk9ONlVwdUpneGttSi9HemJKbVY2?=
 =?utf-8?B?OXVqSVRsVGpzTHQ4aElpR1Rhbnp0QUtTa0FJK0xlN0UvK3JvQUh4eCtCb2gy?=
 =?utf-8?B?SkJFdmxia2Z6R3lvMlR0dy96VFdRaVBVbUpLZ21uYmNVM29DTUNZRmZoWDBK?=
 =?utf-8?B?MStleWhaTFFpTUhOY2duUmllUWlyd1orQzdYQm9LblRaZW9vLzQ5WVJ1dStQ?=
 =?utf-8?B?MWNkSTlIUERIbGswblBTSFNGeFpzbnhnQUt1VGV2T1VpYzR6VkVITGkrejdY?=
 =?utf-8?B?d3I2STJyMS95Y0lZUFVmZENqVVlrUkI5TXh0TmZkaTRmdkJYMEoxRkRSUEVw?=
 =?utf-8?B?aGl3UndrUnJ4NTZtQWVKTjA4YWlJcFBrMnZKYXBDVGF5QXlpMVF1ZlloemZI?=
 =?utf-8?B?Tlgxa3Y5SzVVVmhzb08xMzZIY3Z2ZDVERDU5VmRWSU1Gd1ArZTRQTWthejlp?=
 =?utf-8?B?ZzM0TVhkYnNSMnl2bzZuMWFyM0pER3EwSzJwY2NyT1RTUEczaUpUVUs1VytQ?=
 =?utf-8?B?aitadmJKNWZ3cStjTjBEalFJVDg0clIyV0RDcVVRUWlpdVEraGJEaHUrb3Zq?=
 =?utf-8?B?NlBMcGxJZysyMkJMelJoZVQwM2pVSkVvSUI5V0o3b0hnbGZLWlZ2anpmaHdr?=
 =?utf-8?B?UWpqaitmNGJNTHZBbkMxbFRDUUh4M2tyeVY2S3l3aEZ0TXJnZzZjNUc1elRw?=
 =?utf-8?B?YiswSWFvbGpMekRqUUNNeVllR0l6RXpaakYyNTZvc0F2QVIzWEdKQ20yeWs0?=
 =?utf-8?B?MEFqMS9YZWJreU1uS001L0NMMFZ4THpsRHFOTnRaMjV0MEwwU3FyTGFNalZN?=
 =?utf-8?B?a2ZZV3BPUUtMeUlZbVdrc3dJREdYM0pVNDRKZ09HTWQyTDlhWFFFUHVsNnEv?=
 =?utf-8?B?ckEvbWtobjB2ZE81U3Q4M1BVZjgxVHNDSk9UWFpteTVUVkpPSk9QZ0lqTk9W?=
 =?utf-8?B?QzZwcXUxU1RJSWR5OG1MSU9LNnhyeDFDL21ZT3NncHl2d0RGaVB2VFQzOFhv?=
 =?utf-8?B?blNVZTV1a0RzaG9RUjhnK1MrZkRoZ2FiUkg5a0craktHWUdLaGhUY0tFekJ0?=
 =?utf-8?B?eFdrZmlZOXdIYVQxL0krUUZnREtySUdPNnNobi9tSEYwbUZvazkyMVJzYUhr?=
 =?utf-8?B?V00zek9SbDRQc0p5aWdkaUtpaUppWmpVN3BBdjBhdS9lUktDUms5SjVONXFS?=
 =?utf-8?B?VXZoYjR6NWhwSGhSUGNVUFBRdVkzU2w2RjRPa1Y5blQ1Ujk2Y0tkSk9FT00z?=
 =?utf-8?B?czUyMit4eUdNQll0M3Vjb1A4NDBxdzVYMnNJUEpCRk9lUTdmWHEwOTRNdlpm?=
 =?utf-8?B?dDRiMnpERjRkb2QzY3lrYUNkYWdpcUxvNkVSV0JXcXFONHVFUkUzYXIxRHJY?=
 =?utf-8?B?b1VsaXYxdkFCOXNqQjRzdGdGV0VYbDUrUDJjK2ZRcU5yczR1VWRvWDA0QWdP?=
 =?utf-8?B?NEo3VmxSTW00ckFkbkRkVjZLMy93YVRPMFJtYVo1QWUzK1pCZHlSZWtKMVkz?=
 =?utf-8?B?VUtpUXF4c3VIUzZ2S1lNd25abkc3TTdlMU84SS9EU3ZWS2tJYzg0VGlXWTA3?=
 =?utf-8?B?RER6TTd5K292NTkvcWhnYkRHNzdqYmtTdTFnMkkxWkdsdkt4N0NjYlltaDFE?=
 =?utf-8?B?WTJQVnJGQUpUZ1IwWGFXb2dLTFZ3RVVNUExKenozRGJuT04veHV0aURvQ3Nw?=
 =?utf-8?B?d0laK0tEam5qN3g1b3kzc2xBQXVOakNxaWREQlBQYzZZSGJKdTRaSlgxcEhm?=
 =?utf-8?Q?MKIvYa6UBm20vqyImMoTq8mRrwaWX2ePzIu1fP3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 526246b0-1ff0-4e2c-3d5e-08d96dec60d9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:33:45.5974
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: F1EGcA3cNjCnedJ3cetZGVH+d9QpKSas/7Uwo/r0EmOXbFG3P20VITadmjfLNd8Luy8EWmK1W3h7ayY3YYxRdA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4382

Assuming that the accounting for IOMMU page tables will also take care
of the P2M needs was wrong: dom0_paging_pages() can determine a far
higher value, high enough for the system to run out of memory while
setting up Dom0. Hence in the case of shared page tables the larger of
the two values needs to be used (without shared page tables the sum of
both continues to be applicable).

While there also account for two further aspects in the PV case: With
"iommu=dom0-passthrough" no IOMMU page tables would get allocated, so
none need accounting for. And if shadow mode is to be enabled, setting
aside a suitable amount for the P2M pool to get populated is also
necessary (i.e. similar to the non-shared-page-tables case of PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -318,7 +318,7 @@ unsigned long __init dom0_compute_nr_pag
     struct domain *d, struct elf_dom_parms *parms, unsigned long initrd_len)
 {
     nodeid_t node;
-    unsigned long avail = 0, nr_pages, min_pages, max_pages;
+    unsigned long avail = 0, nr_pages, min_pages, max_pages, iommu_pages = 0;
     bool need_paging;
 
     /* The ordering of operands is to work around a clang5 issue. */
@@ -337,18 +337,23 @@ unsigned long __init dom0_compute_nr_pag
         avail -= d->max_vcpus - 1;
 
     /* Reserve memory for iommu_dom0_init() (rough estimate). */
-    if ( is_iommu_enabled(d) )
+    if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
     {
         unsigned int s;
 
         for ( s = 9; s < BITS_PER_LONG; s += 9 )
-            avail -= max_pdx >> s;
+            iommu_pages += max_pdx >> s;
+
+        avail -= iommu_pages;
     }
 
-    need_paging = is_hvm_domain(d) &&
-        (!iommu_use_hap_pt(d) || !paging_mode_hap(d));
+    need_paging = is_hvm_domain(d)
+                  ? !iommu_use_hap_pt(d) || !paging_mode_hap(d)
+                  : opt_dom0_shadow;
     for ( ; ; need_paging = false )
     {
+        unsigned long paging_pages;
+
         nr_pages = get_memsize(&dom0_size, avail);
         min_pages = get_memsize(&dom0_min_size, avail);
         max_pages = get_memsize(&dom0_max_size, avail);
@@ -377,11 +382,20 @@ unsigned long __init dom0_compute_nr_pag
         nr_pages = min(nr_pages, max_pages);
         nr_pages = min(nr_pages, avail);
 
-        if ( !need_paging )
-            break;
+        paging_pages = paging_mode_enabled(d) || need_paging
+                       ? dom0_paging_pages(d, nr_pages) : 0;
 
         /* Reserve memory for shadow or HAP. */
-        avail -= dom0_paging_pages(d, nr_pages);
+        if ( !need_paging )
+        {
+            if ( paging_pages <= iommu_pages )
+                break;
+
+            avail -= paging_pages - iommu_pages;
+        }
+        else
+            avail -= paging_pages;
+        iommu_pages = paging_pages;
     }
 
     if ( is_pv_domain(d) &&



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:34:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:34:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177157.322407 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiAy-0000lU-Ad; Thu, 02 Sep 2021 08:34:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177157.322407; Thu, 02 Sep 2021 08:34:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiAy-0000lN-7R; Thu, 02 Sep 2021 08:34:24 +0000
Received: by outflank-mailman (input) for mailman id 177157;
 Thu, 02 Sep 2021 08:34:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLiAw-0000jF-Gv
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:34:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 91cebf8c-0bc8-11ec-ae46-12813bfff9fa;
 Thu, 02 Sep 2021 08:34:21 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2051.outbound.protection.outlook.com [104.47.0.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-mV4NjS7ZPO6kVHAaWD_-qA-1; Thu, 02 Sep 2021 10:34:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4382.eurprd04.prod.outlook.com (2603:10a6:803:73::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17; Thu, 2 Sep
 2021 08:34:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:34:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0054.eurprd03.prod.outlook.com (2603:10a6:208::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 08:34:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91cebf8c-0bc8-11ec-ae46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571660;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MFNiMXoiKa06wTzVJFs8SMvmLCcjsyrCF6I15nLpdgY=;
	b=Q8mz+ZAeCT+gw/PXiBqkk2rX86/MVDlLmBcq/5ennSLm0lNsVVIXwEnj6jYOqPIk/wcYBG
	BqMhYH/HLvG7zPpUMJUtYneipSzkXQE0wHxH1MODQQRZO3iBLT4K6kcAUZRIn2C4eisbwb
	n+8wncth7wpaaOMpxZiW2TuP8Ucqi+0=
X-MC-Unique: mV4NjS7ZPO6kVHAaWD_-qA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BRlP2XFEfmByhwI/5Jf+T8woCrmkt8PDBZQUnFGG1RT+wj9ZnW2niRp5wFisVcrdH6rOdX9pGqJNIjCvztOFUEQtiqCyIJ2crF9k9Iu6s0o90FfXVDwJDY2IjewBuS05SMGeHPOTRATkC0cTINUxHkY7LfUM5NuRel1AU8mboDs+FgMn+xLmzJJOTMSrbw3vrYYkznPQfQ6Zy2RGPeQb5fzk7prlGyZA2KKT77uaTojfquY0joDihX8NA189LHmBcEGdufjUqKgymI+IPrhGjrAhfHU3NdbETz0UemsKUVkS0KPTCTHOaW97xbRTy9LJJ3WTWs6P6U1PdQjGUzn24w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MFNiMXoiKa06wTzVJFs8SMvmLCcjsyrCF6I15nLpdgY=;
 b=Nd3+D0GZyx9wHfGziRufm/aqTed9mhjaXx5K/e1jT0P4o5R6Qi8Zkm3fu17mQ1D75l3mRx7Xu0nAPcUI3TqCF06j/rXkWEO8r9nBSiCcb8s+EqoUrMPuaR0KO2BZRL+QjgtQzE8GmFZseEwmYe4b1FdJC99PWhMCKUA4E0hrnD5frM/k8mWG3U6EDtuvXv6RpdP2M9oxsTbADTru0ndZBj5ZHKhVhH4q7GUpbIU9NQPTQIeFm6kGae63oNDMSE9nO+hW0q4y3LE2Ta+VRwmdKDEbwkJTOUBKqWCOzTmEF3M2TOU3Cg03N2MpwWaVY31OFXNJ3Cs5j7wpk7vXS/L9pw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 4/6] x86/PV: properly set shadow allocation for Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Tim Deegan <tim@xen.org>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <1a916bf0-cf23-c2cf-eb18-07f8cea27850@suse.com>
Date: Thu, 2 Sep 2021 10:34:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0054.eurprd03.prod.outlook.com (2603:10a6:208::31)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2200fa26-af8c-4f75-d3c6-08d96dec7440
X-MS-TrafficTypeDiagnostic: VI1PR04MB4382:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4382AA0DDAB0A991F057B95FB3CE9@VI1PR04MB4382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MthWaVx89Qa+jwjD9+9vfDYthol0POTKtKJPBdsoOIKmYxGjEJKKrgOIg8z5Y4Owf2zYJj0/fdc9+hgh1SnlMvCu2g3COA2YLPy1PH8U2wJXzplLUqXT4JsnEIEGy/nDc+CCoY3U+saSo1oBYj0X9U1TVnjzRQi3WMEKvWazEedhxl5M7oALKsG0mC3bKXH2ra0u8FomBmzkm608HHcBXbZMNwCC1oD+m8kwu5W+jC0u7N/g+Ns/BvEb2uhRhQPpWlrlglO01x7+Icxa0X9pNYM4oxJfZUuxsGxYa1p0uN1rPrriljUbWZlTe3ABacjVzZrCJSlkkHpSnTclSgmLbNIdjOkPZzmwd43PXVQkeD2j1wRxM1ExVoYWaq/jjjaScgVXhKXBx5m4fA+tjf9V87sJjEHTlX1Tm3uEPFAUIo68ri7RsYoLxSCfwb3+h440S0dEXD2hm+tGYMUJUrFnYpouGItJ7VNgxwDlksfCHsRtm9dDKnrocU+3WuZwBFz/x3U+FYkX3TdW1nlsBRGpti6cIhqXL012+QYTm0QZpEpySAb/XrGpO6sQg+N1UmiADU1o5EeGtuQyupSYYb9rZ6zSqDfGgO8CtPkgZK++hGmIDZZAV5quUe7k0E3eGnkXhxKgd482TNg9GYlibLJEQgLdPD4sGClevToFw0cmv3Q7St3aZ9IRS2kc0/JphHbI//6vrEyJmCIi6+Xb22X23wFJ3+bQEr5W8In2InL72Lo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(396003)(376002)(346002)(31696002)(66476007)(2906002)(186003)(66946007)(38100700002)(26005)(66556008)(5660300002)(478600001)(4326008)(31686004)(36756003)(8676002)(6486002)(316002)(16576012)(6916009)(8936002)(54906003)(956004)(2616005)(83380400001)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R3djZk0vczdoZmZZTWxqL04vSFVxTHNyYzFRLy9MTllidThKWFgxQTl0cElP?=
 =?utf-8?B?OENKckRaRk5Jem1BSDQ5cjVDZ2QxYlVLS2hwUkFQSDhGYTY0Rkw5dGtwNkxE?=
 =?utf-8?B?Y3dwWWdpSWZFeUM1UW00ZWJUU3BpQ2JMSGgraUpVUmJRWGdPcm9CeTY5T3c5?=
 =?utf-8?B?WklDRFQ2VXA1a1dsVUJrbVdLZjNqek80M3lVeUVrdWhmTGp2VzZmaFZZOHAy?=
 =?utf-8?B?SjloUnlEWDQ5aWk1cVduU3VOVVpDZVI5K0RJVjJJR2VnTVFrbmtab2dvbXdB?=
 =?utf-8?B?MzVqeVRLNjRHZWlvdCs0WXp4dEowdk5ZcGJtdlIxOURjTUw5VXV5VXVycy9v?=
 =?utf-8?B?R0ZlcVRMcDAwUmY3d3lFbE13QWxQV3M0Y3lDQkNJd2NGdkV1N2ZJS0Q0bXIw?=
 =?utf-8?B?cGp4T2dsL2FBd203b3RQVjlML1dMem5FMWx3anJpNTZkeVhORk81QVFRTHlR?=
 =?utf-8?B?MTNsUUhmODRVSlRtUThHQjhVZDAwSXRnTjllTzZwUGgxZXB6NU8vZ3p1TG1u?=
 =?utf-8?B?WGNwVVR3NHpSNTduR0dIeUVBYnF3UERrTXpGQnJQcVF5dFAyY05wY1lrdmYx?=
 =?utf-8?B?Zi9ES1p2NEFPZTUzanNpQ1RMTDVyQzdWdDRFYnVYcTVyZHkyQ0p0WHZCWHlm?=
 =?utf-8?B?WDBUcVFmMzFUUFNRRUZNQUhkZmFFTFVXOGJKcXpEYmlGaFVyTWZuekdJWjl5?=
 =?utf-8?B?V1dLUVcwamt3aDROQUwvS2RDYlRuOGMxYURzSElWRmpvV1FUNmIycFdCQURO?=
 =?utf-8?B?MXlrNWowMDJ6dCtWK1BOR3pGdUFVNi8xWFErSURmdWF4SE1ydXJEOU5uNVdT?=
 =?utf-8?B?RXR0dll0dGkzc3kvZGlkdkxDRVphU05HSjlVYm8xMFpSUTJPeEFSejlQQWpQ?=
 =?utf-8?B?RU1sMjB5ZUxnM2Jhbk9veG5NUnh3R1BienBNUXFpZ0R3ZXc4N0hOdFhWSVk2?=
 =?utf-8?B?UlBPT2c4R3ZYZjFzVUFtOUg3elkvY2Z1Z2dJamt1MTlIdXR2cDVNN0RHRU5K?=
 =?utf-8?B?ZHJEUkFhVE9pZm9DTnpEYjFjMmJYeUhGUHJFODFrSER4QTBOUVA3SWJTZXFm?=
 =?utf-8?B?TjFESHNNVFJHVXNWRFJibEpaUkNsVjFxbno4bHJwNkFmU09kZkpxZUxXVFNX?=
 =?utf-8?B?eU1DTkRzdk5KNjVPUUVlZUNEaWk2LzNtQmdIZTVLUmZhYjRWaFpjY3V6ZTNy?=
 =?utf-8?B?RTNVZTRCZFdMRWczbERKTG9TbHVKdklvdDBhaXQyZ0luTHhWM2c0WkJ4ZE5I?=
 =?utf-8?B?VzhrOUpjdVVHeGtGcnQxckRneWlrWVBoczJENWF0QkNMRVU4NGRCN1dYd2dK?=
 =?utf-8?B?dTFvVGtQN1hxUGppUDhzem1GSWFRdy9ZM3dPMm9XSlQrUVVKRXhaNkQvd0Fx?=
 =?utf-8?B?KzdUdU1rMkhqa2p1Z0c2KzFHcXdJeEpXUnloUTMyeFY5cW80dG1vK3ZjTGtt?=
 =?utf-8?B?RzNNOXhZcTRneTlINFZmVlZNaU5Cc1B2ei9nbjIzL1R3bzN3SVlmdTJ5bm5H?=
 =?utf-8?B?QlkvcmZtTHZMZC9HWWJSblFCWHlNSzFXMHdCWFQ2Z3ZjK1ZGRUNBOVNDdWlS?=
 =?utf-8?B?OGZybksrY3ZRemloQ1JLWmNNbE01dmdWUWdSMXh5VkQ0UkpBQmIvZ1gzbSsy?=
 =?utf-8?B?UmRFUzdKU1dkQ011UmY3eEJIdUlEZzl6NEsvM3pxODljM2tLQndYL3hwUDda?=
 =?utf-8?B?RjJadFcvNHI3aHJ2Q1Bzenl0NS9ieU54eEdZLzdEUWpCMDhZQmkvemR3Vnow?=
 =?utf-8?Q?Q/Os+aEgRb74BhFKUqdpNBIpNQNL/1KM6gbts70?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2200fa26-af8c-4f75-d3c6-08d96dec7440
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:34:18.0900
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mcEMG2vgLKJ+r/fnn2mjT4ZguMdZtK43BX69hBl8YK0RRCh12jcUMsSGqfjfC4oEERJsSna2vVhba8UV+c0GPw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4382

Leaving shadow setup just to the L1TF tasklet means running Dom0 on a
minimally acceptable shadow memory pool, rather than what normally
would be used (also, for example, for PVH). Populate the pool before
triggering the tasklet, on a best effort basis (again like done for
PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
---
v2: Latch dom0_paging_pages() result.

--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1298,7 +1298,7 @@ int shadow_set_allocation(struct domain
 {
     struct page_info *sp;
 
-    ASSERT(paging_locked_by_me(d));
+    ASSERT(paging_locked_by_me(d) || system_state < SYS_STATE_active);
 
     if ( pages > 0 )
     {
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -21,6 +21,7 @@
 #include <asm/page.h>
 #include <asm/pv/mm.h>
 #include <asm/setup.h>
+#include <asm/shadow.h>
 
 /* Allow ring-3 access in long mode as guest cannot use ring 1 ... */
 #define BASE_PROT (_PAGE_PRESENT|_PAGE_RW|_PAGE_ACCESSED|_PAGE_USER)
@@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
 #ifdef CONFIG_SHADOW_PAGING
     if ( opt_dom0_shadow )
     {
+        bool preempted;
+
         printk("Switching dom0 to using shadow paging\n");
+
+        nr_pt_pages = dom0_paging_pages(d, nr_pages);
+
+        do {
+            preempted = false;
+            shadow_set_allocation(d, nr_pt_pages, &preempted);
+            process_pending_softirqs();
+        } while ( preempted );
+
         tasklet_schedule(&d->arch.paging.shadow.pv_l1tf_tasklet);
     }
 #endif



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:35:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:35:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177164.322418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiCC-0001Vi-QS; Thu, 02 Sep 2021 08:35:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177164.322418; Thu, 02 Sep 2021 08:35:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiCC-0001Vb-NB; Thu, 02 Sep 2021 08:35:40 +0000
Received: by outflank-mailman (input) for mailman id 177164;
 Thu, 02 Sep 2021 08:35:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLiCB-0001VV-MD
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:35:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 61649c31-d848-4c36-a8cb-50d2e2728f10;
 Thu, 02 Sep 2021 08:35:38 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-WLfgfpoDObGdH1pRwezw8Q-1; Thu, 02 Sep 2021 10:35:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4382.eurprd04.prod.outlook.com (2603:10a6:803:73::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17; Thu, 2 Sep
 2021 08:35:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:35:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0113.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 08:35:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61649c31-d848-4c36-a8cb-50d2e2728f10
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571737;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WA1KAMRckXvuUXmqZbyvq0j4l0CKS5BjF16pVyrhqnE=;
	b=l0KzgmUbnDjAa4c0c92ZIOG9cY8Js3XZN98MiqVOQXq4ylEH3DG1UtjjBbDltGZbAUAzUO
	a2thv3prkpIb6bSg5Ovkz1SGdvXsbfdhznMZ7XHkStQR1jLWwY4dcd8c3AMfU3yCvU+pKQ
	RrrLl+leruAHEGbmMdkKNNESUQDti8Y=
X-MC-Unique: WLfgfpoDObGdH1pRwezw8Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FgeD+kR1iFq0fUzJG7/zgifj1bz42Y7p2u26lBTkipvySMyy5MAhDAlCFtnvxt/asK6FwWehwRf8Grv4pDJc6ncyO8acE8UxX7LPyvJbL8C/KkQ2bRlW/lldNU5x20k3CYrSQtaxtCHm5eGyt9xwv5bdJpiNxqUAfMhsVmVhhzEd/RjnsHnxEvkdGYZeaQkGwb4acj5vwPaWtbWYDyjDtr4PrZgxwVDrNQi775M9li80jbYBqYM2/Z1zoY+tl4Dheea4n/je4nMxHGtMWneMqmbtMjA2AT7aZxfDFz0Iui5mZMROfVkXg6Si3JXrXAOFallipkqWaGfrTWh7QxTKAw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WA1KAMRckXvuUXmqZbyvq0j4l0CKS5BjF16pVyrhqnE=;
 b=LlDIJJzGwrLpIejPSQto09A+luKjSqvO85sFz+h9Ae5jsLWPqQXx8xdRj+oHCA4dOXD7W+Yedx0oQ74T+x5LFEYfRPiJZ7suc+gfRraWS0jr8E3GXODBAHpqrQpQlbhG4yNO34j8Pc+MHGh/NYaRAd4MlJfsqtOdIXPDXVDktUWDhPD79lRka8DYsWZ4vxunmS+onSQJiWIlpkwxGgJDNQRwXOurIQWtv4yguUS1AKyXQ0rBXrBcaNdDB4JTCyFJghe842o9FfAjv+DoZAqUeFJfoSXRQ13vbYTORfuODqfUs5gOxz4uzoPMcY8VNp5PRyLBXu5P3/ga1Ch4NFHsaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 5/6] x86/PVH: actually show Dom0's register state from
 debug key '0'
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <f8e599d1-e6eb-19f9-aa9d-c217376eeab0@suse.com>
Date: Thu, 2 Sep 2021 10:35:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0113.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f1203fff-04e3-4fb5-3320-08d96dec9f9c
X-MS-TrafficTypeDiagnostic: VI1PR04MB4382:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB438207D39A0F34DA379A7988B3CE9@VI1PR04MB4382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1kujJhewmYX+Uw+noRoRi+x+Ilvv0mAPF2HIKZBEKac22XkThkOwvpReFnTUKEqZPO/15SCA5Y3Pui0aG7O7Lr5+jCkAxL+IvDZmUAuGB5CrHXUYcM99Il3kKpfEZW97x92aAz5mcnAUtE3BYWQbHSOaGhdTpaFBV5yWJN+7NScuHTgRQS/th/qbBWKXmU0NP1F7pGf4jbInKUOLCegKl6/1CgOTqBRV3wy39vVbp/JpN3dtWv5eQzueLKkoHQJZkbP7yiKwoV/x7SFPp1lY3chMepyjd28Z1Fn1zFLSPNDavx9T1tu/2dvD/YemhntkLV7uiyzvzFKZLSCRMYsIoUcWM063GhaFycO5IKzTT5hd+dNWhGGqM3V57UIRlQQh0OfVypemmB7s6REen1tpILPNOPDehPTsMsVWdXOR+lJzuv4fGEd8cLJV0v8uvrNoZY2JG2VuEsF/1/VEZ5QPYx4Ofj9gPqMB0VdfoOPTVMpTUf0cyRGEemm7G3iuxlsCaLWZ+efjpuvRo3ix/G1cA7baPq20/NV6Wsgo4wAuy/cKuB8zj4ShIaYKz8OTzBlLqN1ot9CYOlYyo7o8B6ZG1/JP+ghhPvSqqjY+uZGXwhxqkh2eY/xAXPzur1hwj53yGyy7BsaLhL+URaYMjVjxqLoqF7XWrMgcUSYUV7OiH2HglqdyDOP5jnSfhk0sDg7eNHJU+CwtxhW4mymai2OUZg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(396003)(376002)(346002)(31696002)(66476007)(2906002)(186003)(66946007)(38100700002)(26005)(66556008)(5660300002)(478600001)(4326008)(31686004)(36756003)(8676002)(6486002)(316002)(16576012)(6916009)(8936002)(54906003)(956004)(2616005)(83380400001)(86362001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YW1xb01rVExQNTZqVk12TjlKaURVTlRYUVVlNTFTeEdha2M0QXlydlFHaFdr?=
 =?utf-8?B?ZmcrRFZLam1mTGNNdkllZlppQlZvTXBQRzdOTUJVNmZmVzArdXllUnA4K2RH?=
 =?utf-8?B?Z2wvUUp5UmlXT1Zod0JWZmhyemV1L3hhZTJlaHFQU0UyVHlNUnZHL3ErR0VS?=
 =?utf-8?B?L0RsRkdaK2xEdDBNS29qeVdIWlZjKzNpbjA5Qk5ZTHh4SzZDdEdQZWlJLzk4?=
 =?utf-8?B?aWxJdnJteUFYaHhTeFlOOUxPZHJyeWJ1T05MWFRxemRORlpCSHByaWZkSmpY?=
 =?utf-8?B?SGhQMEFzRGYvQmdrOWswdmRYcEppWE9QalAzM281ZGJodEFMSHUzYzJ4a3FF?=
 =?utf-8?B?Mm1YOTd4REJxWFdsMzFheEVMNFNnMjU4NUgxRHZNRWNRYUNPNG5jTENaNlVO?=
 =?utf-8?B?dk04UEZDTHo3ZmxmREdGYjJ4TFRDVGYva3VlWkczdzdGVW9hblFTRmVTYnNY?=
 =?utf-8?B?dC9PTjlyaDNoNjZwdlJScmExeHI4WEppQ2s0MHBvdmJ6SDFzR2hIS3lIak8w?=
 =?utf-8?B?MVpBWnJ3REo0bFFPNFlEZ2s2UTgwK2FOTGRDbTcvUU9rNVVLTmZqNGF1WFlD?=
 =?utf-8?B?NnFxejBZb2owU1MvaEJuNCtSUWZDQ3Q5Z3NXUHFpYUZ4cm8rYWt1RTNNUUx6?=
 =?utf-8?B?cncyZGdXRnlRZU5hazNVdUFBMzVzMVBlUExpUzllMW9VL1d5OUl0UVVDZ1p5?=
 =?utf-8?B?VjgveWw0WlhLbmIrbTFsb0RTU2dCZmNLcld1VHE1UENKeXlGR0FWQldENGsy?=
 =?utf-8?B?RFBmTW9HQWs2QS9wSzA0c3NvU3RyekFnQkN4L0NFZmtjZ0xiOWJHbjlDWkdL?=
 =?utf-8?B?Rjdrc1ZHTGJOazRiL3kvWXhGRm9TTFcyTEcxekt4MGE0UjhKRElDc09kaUlP?=
 =?utf-8?B?UzJKU1FWTllSdW4zU1hVVEpOUUxoVEpIek5mck4yNHA5cFFQVWJidFplT0Vw?=
 =?utf-8?B?T2tuVmZoTVBIUmY3RkczNDRPT0xWZ3hCei9tTVJKb1I1eFVjY2lOVW81a2FX?=
 =?utf-8?B?VUZ0elhqL3M0NERyTlFIUExwY3d3Y1pZc0VPK0F2NTR1SS9Qdm5YWFgvKzdB?=
 =?utf-8?B?aE1ldjFKTVY5bEYyYTlpc21HL0pjVjNOVDJOYnFqZGREcjRoaHhoQjh4Q250?=
 =?utf-8?B?aCswM25PM2l1TDlPbDNzeDl1ZkVVUjFyUXdNMi9ZcnNPellucVJmWFRkUTFo?=
 =?utf-8?B?Q2dScVdubGxJWVU1eDB0eTZuRzRQaE1ubTcwWEhjNkNoek9Lbk9nMGxReExI?=
 =?utf-8?B?NkhrMGg2U1VqMHZNVzRvbEFDLzNpck5vSlRnWnNwS3NuRndlaWVsWHFXMmpV?=
 =?utf-8?B?dXJqWG1vaWFVdmc4c2dFSkVUUDFMcmdNMnFMbFEvc2hXMmk2eDRIUmUycmFm?=
 =?utf-8?B?Tys0R01seHB5OFoxQkQzaHQ5RTFkT2F3YzdvMnBKS0dBcEhvenB5bG52UzZq?=
 =?utf-8?B?K01sSURVUGRVMS9Kd2Qvb1FpNFdEZjhpUnI5SUZtN3RpN3dmM09IWXY3ODk4?=
 =?utf-8?B?V0dTV0Zac20vN3k0U0l6dW16c1plWmsvelllK3FHV05oa1FDY0V4NzZ1NXY1?=
 =?utf-8?B?cXlCZytJT1FzQkl4T1o1ZFVaTDZieXpzdEdZRmNkT3cvMFdRaGxsclY2Z1NS?=
 =?utf-8?B?RDFSK2g0OXFidlIva01PZVRMR2RxZXlsL0ozK3A1WEFVRDZ5N3RDN1hrNHVL?=
 =?utf-8?B?ckprL1gyaWdJT3U5WnFjOTdUMWYwSmlOTTEwWVg5MTNuL3FLWVAzTGtGZndN?=
 =?utf-8?Q?JrH28oUtXR9viFbSvsubGlI8eLp5WgE+okcIgK9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f1203fff-04e3-4fb5-3320-08d96dec9f9c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:35:30.8818
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gcQ/uPgMGbWDlFY+ZF5tk7X9RMxD/7loktpn1QcPD/55qFIiyM6jj3egHPLRy7v5hyUtWM2r5B3O3CL/x5GbqQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4382

vcpu_show_registers() didn't do anything for HVM so far. Note though
that some extra hackery is needed for VMX - see the code comment.

Note further that the show_guest_stack() invocation is left alone here:
While strictly speaking guest_kernel_mode() should be predicated by a
PV / !HVM check, show_guest_stack() itself will bail immediately for
HVM.

While there and despite not being PVH-specific, take the opportunity and
filter offline vCPU-s: There's not really any register state associated
with them, so avoid spamming the log with useless information while
still leaving an indication of the fact.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I was pondering whether to also have the VMCS/VMCB dumped for every
vCPU, to present full state. The downside is that for larger systems
this would be a lot of output.
---
v2: New.

--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -49,6 +49,39 @@ static void read_registers(struct cpu_us
     crs[7] = read_gs_shadow();
 }
 
+static void get_hvm_registers(struct vcpu *v, struct cpu_user_regs *regs,
+                              unsigned long crs[8])
+{
+    struct segment_register sreg;
+
+    crs[0] = v->arch.hvm.guest_cr[0];
+    crs[2] = v->arch.hvm.guest_cr[2];
+    crs[3] = v->arch.hvm.guest_cr[3];
+    crs[4] = v->arch.hvm.guest_cr[4];
+
+    hvm_get_segment_register(v, x86_seg_cs, &sreg);
+    regs->cs = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_ds, &sreg);
+    regs->ds = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_es, &sreg);
+    regs->es = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_fs, &sreg);
+    regs->fs = sreg.sel;
+    crs[5] = sreg.base;
+
+    hvm_get_segment_register(v, x86_seg_gs, &sreg);
+    regs->gs = sreg.sel;
+    crs[6] = sreg.base;
+
+    hvm_get_segment_register(v, x86_seg_ss, &sreg);
+    regs->ss = sreg.sel;
+
+    crs[7] = hvm_get_shadow_gs_base(v);
+}
+
 static void _show_registers(
     const struct cpu_user_regs *regs, unsigned long crs[8],
     enum context context, const struct vcpu *v)
@@ -99,27 +132,8 @@ void show_registers(const struct cpu_use
 
     if ( guest_mode(regs) && is_hvm_vcpu(v) )
     {
-        struct segment_register sreg;
+        get_hvm_registers(v, &fault_regs, fault_crs);
         context = CTXT_hvm_guest;
-        fault_crs[0] = v->arch.hvm.guest_cr[0];
-        fault_crs[2] = v->arch.hvm.guest_cr[2];
-        fault_crs[3] = v->arch.hvm.guest_cr[3];
-        fault_crs[4] = v->arch.hvm.guest_cr[4];
-        hvm_get_segment_register(v, x86_seg_cs, &sreg);
-        fault_regs.cs = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_ds, &sreg);
-        fault_regs.ds = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_es, &sreg);
-        fault_regs.es = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_fs, &sreg);
-        fault_regs.fs = sreg.sel;
-        fault_crs[5] = sreg.base;
-        hvm_get_segment_register(v, x86_seg_gs, &sreg);
-        fault_regs.gs = sreg.sel;
-        fault_crs[6] = sreg.base;
-        hvm_get_segment_register(v, x86_seg_ss, &sreg);
-        fault_regs.ss = sreg.sel;
-        fault_crs[7] = hvm_get_shadow_gs_base(v);
     }
     else
     {
@@ -159,24 +173,35 @@ void show_registers(const struct cpu_use
 void vcpu_show_registers(const struct vcpu *v)
 {
     const struct cpu_user_regs *regs = &v->arch.user_regs;
-    bool kernel = guest_kernel_mode(v, regs);
+    struct cpu_user_regs aux_regs;
+    enum context context;
     unsigned long crs[8];
 
-    /* Only handle PV guests for now */
-    if ( !is_pv_vcpu(v) )
-        return;
-
-    crs[0] = v->arch.pv.ctrlreg[0];
-    crs[2] = arch_get_cr2(v);
-    crs[3] = pagetable_get_paddr(kernel ?
-                                 v->arch.guest_table :
-                                 v->arch.guest_table_user);
-    crs[4] = v->arch.pv.ctrlreg[4];
-    crs[5] = v->arch.pv.fs_base;
-    crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
-    crs[7 - !kernel] = v->arch.pv.gs_base_user;
+    if ( is_hvm_vcpu(v) )
+    {
+        aux_regs = *regs;
+        get_hvm_registers(v->domain->vcpu[v->vcpu_id], &aux_regs, crs);
+        regs = &aux_regs;
+        context = CTXT_hvm_guest;
+    }
+    else
+    {
+        bool kernel = guest_kernel_mode(v, regs);
+
+        crs[0] = v->arch.pv.ctrlreg[0];
+        crs[2] = arch_get_cr2(v);
+        crs[3] = pagetable_get_paddr(kernel ?
+                                     v->arch.guest_table :
+                                     v->arch.guest_table_user);
+        crs[4] = v->arch.pv.ctrlreg[4];
+        crs[5] = v->arch.pv.fs_base;
+        crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
+        crs[7 - !kernel] = v->arch.pv.gs_base_user;
+
+        context = CTXT_pv_guest;
+    }
 
-    _show_registers(regs, crs, CTXT_pv_guest, v);
+    _show_registers(regs, crs, context, v);
 }
 
 void show_page_walk(unsigned long addr)
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -631,6 +631,12 @@ void vcpu_show_execution_state(struct vc
 {
     unsigned long flags;
 
+    if ( test_bit(_VPF_down, &v->pause_flags) )
+    {
+        printk("*** %pv is offline ***\n", v);
+        return;
+    }
+
     printk("*** Dumping Dom%d vcpu#%d state: ***\n",
            v->domain->domain_id, v->vcpu_id);
 
@@ -642,6 +648,21 @@ void vcpu_show_execution_state(struct vc
 
     vcpu_pause(v); /* acceptably dangerous */
 
+#ifdef CONFIG_HVM
+    /*
+     * For VMX special care is needed: Reading some of the register state will
+     * require VMCS accesses. Engaging foreign VMCSes involves acquiring of a
+     * lock, which check_lock() would object to when done from an IRQs-disabled
+     * region. Despite this being a layering violation, engage the VMCS right
+     * here. This then also avoids doing so several times in close succession.
+     */
+    if ( cpu_has_vmx && is_hvm_vcpu(v) )
+    {
+        ASSERT(!in_irq());
+        vmx_vmcs_enter(v);
+    }
+#endif
+
     /* Prevent interleaving of output. */
     flags = console_lock_recursive_irqsave();
 
@@ -651,6 +672,11 @@ void vcpu_show_execution_state(struct vc
 
     console_unlock_recursive_irqrestore(flags);
 
+#ifdef CONFIG_HVM
+    if ( cpu_has_vmx && is_hvm_vcpu(v) )
+        vmx_vmcs_exit(v);
+#endif
+
     vcpu_unpause(v);
 }
 



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:36:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:36:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177170.322429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiCw-00026I-4y; Thu, 02 Sep 2021 08:36:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177170.322429; Thu, 02 Sep 2021 08:36:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiCw-00026B-1O; Thu, 02 Sep 2021 08:36:26 +0000
Received: by outflank-mailman (input) for mailman id 177170;
 Thu, 02 Sep 2021 08:36:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLiCu-000261-El
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:36:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id da8f42fa-0bc8-11ec-ae48-12813bfff9fa;
 Thu, 02 Sep 2021 08:36:23 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-h7Kl6CexN6K9du2Ae_-NoA-2; Thu, 02 Sep 2021 10:36:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3774.eurprd04.prod.outlook.com (2603:10a6:803:1f::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 08:36:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 08:36:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0142.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::47) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 08:36:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da8f42fa-0bc8-11ec-ae48-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630571782;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3HFchKadupoYKtpNV3JYrysgf0xXI5YrxKsF7FnzFzY=;
	b=cER4fy4euTrxYJt6cNuZ4nrq7Fj7i0suLo17RUdAUdCV9ImGEH2zJUBcEM90+FiEBwC7fT
	8BnIl1mrtb9eVDdXE/hs9PUyf8aMUO6BLPsYBjO1HKVUwIf/HI2xIOyCMYrNufz6q82thK
	FDoWmWFfgNrcjHka2EySYVtsrTOo5T8=
X-MC-Unique: h7Kl6CexN6K9du2Ae_-NoA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ENAuNW26oOVoUlaUGTBVtchjUH19yEzgBRSRLdxfPOPmQvDWcd0xq3ixkf3UHAZ3aDGnJmJ/stubckZ0KW7kes2Kbr1t2/A1mrLu2WBmAJIBEiFrvh5q5TQdnI8rzBn31UHP3tlbA412BHUc9c3euIFIoYU2A4ebtkh31J01+mYw1Alb9VwXGpa/QvRsjp0WBtgs6V8/3eKzRXTUDuhUtwseJSCUQ15ndw2OqEBtpa1m1qS+iaXaANK/4H70ZBOeK2+ysQs/DM+s+q+EeSDQjgnbEDPQvnzpdA9LaohrwHUp3pQHm7bSarx3ZIVzm/7uWjEtWUlMIO2rz/DNx3moFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3HFchKadupoYKtpNV3JYrysgf0xXI5YrxKsF7FnzFzY=;
 b=HnwuMz2pbxBhJcat1k1iXydYwGElHcWwFXQ14FJnWWm601UzLOYzOEWI0Rx6fFj83kKn4wyfkdLYjE/R+Nie3d4wufAIyV2+wnoaY4WhrT0CHBbs0cyxVQm7SqTa//e1VHygXC7Fv66mTz0rWFctK578yXqPXcaQ/p0ZFxTLdoisjDGcw74KlYDiuTxQqub1St2fz3aJDV8Oc9ESN7abL2TRzu6bDReeWtDYxQH/EbWHrkhPgqHVf5pmLxOKDFJSdVo9ahubKYTGKkoHbOJ9f9QEHKM6eABCn8FHlRm8i48YIuLjZUhQ8KNdpTghrmQ4hSgBtDYetEFbCV0BzzsbzA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 6/6] x86/HVM: skip offline vCPU-s when dumping VMCBs/VMCSes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <332739d2-be0d-fe8f-9ae7-a34916f4eab6@suse.com>
Date: Thu, 2 Sep 2021 10:36:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0142.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::47) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0668c48c-5c43-402a-9a7e-08d96decbda0
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3774:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3774BA4DC6393856380E7D89B3CE9@VI1PR0402MB3774.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:457;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5rBBEvOhR7faCZjTzNU50BXFN68nDasGZ3jlg0KO6SjhlL5TykJJbwZB/CyohboKMnUO/nslnlgub068PBCWIWWfI40fKCO+T+Hr52k6a+wjRni+zPWhUhLIN3zP83pOCGUYLBpYM6OCKTXYfSxLPSVaMgcZlX47JKARUJ6hpgEnXq5soW+PcBKtKEA9I6J9QbgW/ZJd8RIoYUOEIdPWu1yG/qF7hpRCPK8mXJFi3FXBUTZ2rAn0ZP8ju0bJfWS/4+4bbhhzEtk2/QzbjrpRBIfPeHBb0OXTZ1yCJxPTUKfQwIkAav2ZoglzjfSgDoji4pvxcT9vc4sBl/3u8Rizq8go2OSyHS8tIs4TklTnZwkkFFoplyM8336FYhoTDsaZSpyYYssi/Bl7fMBNzxKM6GYGxVNufb8qFPD1HWd+m/IzOCkFtsezeKE54oIHod2A0kna6EpLJAv2VZEEtz8TCKmdcvPxauwc5RSHXwAU1jiFB9LISvyeUN4aYBS/GI4aCbmRXpsfu6zVZGZPnRGuVsXe+rPX/fkZhxs+Ywx/u8zHcsWzxSRd2CvdqyYToZswGxZqUWlAO6t0wS4w5+3RMGF1wPC863rcd/3RzDEfqzNEcXM4t5RL+kE8Z20IA0EYYsCIY1M7kBbelcH/jGRhDfEdpgtgpO1MUCxX000bUFWilely20YKIIo5t7buSxaixk5n/VDHUFsSUYEEH4NdLA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(366004)(396003)(39860400002)(376002)(36756003)(956004)(54906003)(6486002)(6916009)(478600001)(16576012)(38100700002)(66946007)(4326008)(316002)(66556008)(2906002)(26005)(66476007)(86362001)(31696002)(186003)(8676002)(8936002)(31686004)(2616005)(5660300002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?djZHK21UVmVpaEFnQUFyeWxLMDVaVG84RDRkOThWUFhpdGdaZGhWaWFFWjFj?=
 =?utf-8?B?Qi9NaWpOdysvTkVuVE1VTnBJWDNxdEErNnowSnlycnZWZTBUSEdwUWxjcFg4?=
 =?utf-8?B?NWJJTTBiRnFzdmh3OU82VzZHeXpoQ3BKK3ZWMk80cTJWcGdaeEdyUWZXTnhp?=
 =?utf-8?B?bTAxNGtsa3dVbUg4a3dTWExzUVJuT29xVk5lK0wyUnQ3TThJWEhWUFhnTldp?=
 =?utf-8?B?emtOWnRBaDRDOHpvK0VSN1I2VjhYVC9HN1lzUE1FMU1XU3lQdDNwNzBDcldr?=
 =?utf-8?B?WDRKZDFqSG1TZk9rdmxqSmY3UXo2VHhKRG9VY0MwMXJraWhqbGdMOGhJNnFn?=
 =?utf-8?B?SCtJYVVmYm1tYStUSzJMQWp4NFNDc2dtUkRjTEVYb01YOVJ5dmdwYXFoVGVD?=
 =?utf-8?B?YnkybnZaZnUweUdTaWxHMSt2MEo0OHhsSmF6dkpONlpzTWlFTmoxd09IeVRt?=
 =?utf-8?B?R21aV29jT3VuMUYyOHhPMVdqL3RqTWF0MlVLcjd4WDE3LzR3Wm5kK1RVd1RP?=
 =?utf-8?B?aFJkUkorU1kxWEpFTkxUOXVaZ2ZGekd3amZhWlpZZGRpVk5ObGdBSUZFbEdK?=
 =?utf-8?B?NG52SW5CeEZ4MFAyRjBNVlhFWXUrcTN6OGlNYzZyVnZCcjVnWENDUk95VjlX?=
 =?utf-8?B?MTJwejQwN0gyY0dEMUF2S3o0dmtGZDZkQVJKRmF2MEU5WkErZUpwd0JiUmRX?=
 =?utf-8?B?b2JEVUNKSW1tMy91NE16VTVvQ1RYdXhWRFVrZ05CVE5NT0RYOXpaSGV0YWVs?=
 =?utf-8?B?eGJnMjkvcmRWTEpkaHdNM2tUVkdIcE54RHlMV2hwV3pjU2x6NXFsSDhnMnZ6?=
 =?utf-8?B?S0pZc3BMNm85L2xPdWgwMi9ERlVjbjFteUtHaVFERkNiSDJXUXVXbzBla1BR?=
 =?utf-8?B?V2xzQmJyRVlhbEkvdEtuVXpaQlo3YmZCV2RmWEpERllzKzFNTDZNT0lNSFJL?=
 =?utf-8?B?Y1VDcmYvdml6cmpmenVxbk9CdytCcVo4QmRxNDZ5aEhDWVRsTS9Ed0M4TENH?=
 =?utf-8?B?MXBvSXU5R2VURkliaEVXYmFoaStrbnRzY2VqcWNiV1FRS2lwM3VUTlFlT0dn?=
 =?utf-8?B?T3cxSjN1YUY3RFRrZGJqcGVEeVozdGpObUFybVFpWDBqcWFjeThrT1Nqb3U0?=
 =?utf-8?B?YmxZTWZWV2JWWmVLUTB6c1VQQWIrdkpMOHVjajdPUHcxa3RaRFVTdFkxcnpF?=
 =?utf-8?B?Qk9tSEdhNWtuNXFiQWJlWmF1c0ZVWXlIZmVLa2E5bkFMajRNNHVGSys1Tzd1?=
 =?utf-8?B?M2xzdUFsWnV4Wk5MbGx3SktBbEFtK3lDZGUzcldmSXlic3JKeEl2NUZqMWgy?=
 =?utf-8?B?TkJvaEswZ285bUgzSUdNZTlkVHVVbGpSazZTampXbU1VeHF4N0ZFaTFnek1h?=
 =?utf-8?B?MzY4WWNiQm80LzlqUlh0OFk5LzVUN0E3bWF3WTFyTGFWeGszcE9TWVlkL2Za?=
 =?utf-8?B?akR2d1B2VXkxL2dld3EvUUJTbnBOL25VN2JRUW0rd3FtcXRWZW1BMnN2dzJU?=
 =?utf-8?B?c054YnBZRHdpN2g4dVZ5SFN1dDM2YjdMUFFWQ05mNXVqMU9YYVVaMFhhVi9o?=
 =?utf-8?B?QzFTSm9OODBsYUdnUnUwNXdvZW5ZTVlLTC9RWlBtdDkxNjJmTVQwcjRrOVhi?=
 =?utf-8?B?N1A4YkVNQ01aMGhvNE9tYjZxWk5tWjA4dXdmRUlETmowM3ArVWx3NTBRL0Er?=
 =?utf-8?B?UTQwQ29nZS9sVEYrd0JWa0NoWmZZTlg3S3BFYVpSTEFQRk80Rm9ybVdoMEJw?=
 =?utf-8?Q?ZtMhKvRL5RZit2oyUbt/l6ZZuqpE+JCOUVyH10q?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0668c48c-5c43-402a-9a7e-08d96decbda0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 08:36:21.1863
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Z3HeDrcO2NcmV0MsIdjnSmiC5l8AtPoOEBrEOyMQ3ojFJN/4VbgudW6OEh1AzWG99uZsTyJYiJ/ri9Ylqqyy/A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3774

There's not really any register state associated with offline vCPU-s, so
avoid spamming the log with largely useless information while still
leaving an indication of the fact.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -241,6 +241,11 @@ static void vmcb_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( test_bit(_VPF_down, &v->pause_flags) )
+            {
+                printk("\tVCPU %u: offline\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             svm_vmcb_dump("key_handler", v->arch.hvm.svm.vmcb);
         }
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -2133,6 +2133,11 @@ static void vmcs_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( test_bit(_VPF_down, &v->pause_flags) )
+            {
+                printk("\tVCPU %u: offline\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             vmcs_dump_vcpu(v);
         }



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 08:59:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 08:59:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177178.322445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiZY-00054u-4F; Thu, 02 Sep 2021 08:59:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177178.322445; Thu, 02 Sep 2021 08:59:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLiZY-00054n-1I; Thu, 02 Sep 2021 08:59:48 +0000
Received: by outflank-mailman (input) for mailman id 177178;
 Thu, 02 Sep 2021 08:59:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLiZW-00054h-2F
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 08:59:46 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLiZS-0000gz-9G; Thu, 02 Sep 2021 08:59:42 +0000
Received: from 54-240-197-224.amazon.com ([54.240.197.224]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLiZS-0001SW-1B; Thu, 02 Sep 2021 08:59:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=AZFZoEmGVV9n+CJDtxX6UCmu2LESlsmMlXLemu/qy0c=; b=faZWoWpkJwZTY5H72pvskTMuuC
	lMzemWpbgKHYSfD4HcyWFy3LLmGqfNKx92KSTLkqie+UAU8e7gwzAdn8dtgyDTCHIK3bgLlOf+X+u
	IVi8zkEGayJIwEaUa2yKTkS6XbCuKsTZeahcLblGfEoj0KNpeSASX43BHTaX1ji9Rsao=;
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Michal Orzel <Michal.Orzel@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
 <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
 <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
 <0e72fb05-a477-ef79-c26f-bc011393a403@xen.org>
 <5B7B9C01-FF1D-44F9-8815-A422E51916A0@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <a7920989-f0e3-3eb1-5317-09e6bcd67211@xen.org>
Date: Thu, 2 Sep 2021 09:59:39 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <5B7B9C01-FF1D-44F9-8815-A422E51916A0@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 02/09/2021 08:57, Bertrand Marquis wrote:
> Hi Julien,

Hi Bertrand,

> If I understand it right, you want a per guest parameter to be able to allow PMU accesses.
> This would require:
> - to save/restore MDCR on context switch
> - introduce a new config parameter for guests (might or might not need a tool change)
> - have a xen command line parameter to have a solution to Allow PMU for dom0 (or maybe a DTB one)
Yes.

> 
> But this would NOT include:
> - interrupt support (only needed to get informed of overflow)
> - provide PMU virtualization (not even sure something like that could make much sense)

I am guessing the following is also included here:

- provide a PMU node in the DTB for the domain.

Without those 3, I feel we are exposing an half backed PMU to the guest. 
But this would still be a good first step, so I would be OK if they are 
not implemented in the first shot.

> 
> I am not saying that we will do that now but as I need to unblock this I could evaluate the effort and see if it could be possible to do this in the future.

Well... Below the patch I wrote during my breakfast this morning. This 
has not been tested and miss some documentation.

 From 690a92cffed82451dcbd8b966e8dee31c1dce5fc Mon Sep 17 00:00:00 2001
From: Julien Grall <jgrall@amazon.com>
Date: Thu, 2 Sep 2021 08:46:12 +0000
Subject: [PATCH] xen/arm: Expose the PMU to the guest

There are requests to expose the PMU (even in a hackish/non-secure way)
to the guests. This is taking the first steps by adding a per-domain
flag to disable the PMU traps.

Long term, we will want to at least expose the PMU interrupts, device-tree
binding. For more use cases, we will also need to save/restore the
PMU context.

Signed-off-by: Julien Grall <jgrall@amazon.com>
---
  tools/include/libxl.h            |  2 ++
  tools/libs/light/libxl_arm.c     |  3 +++
  tools/libs/light/libxl_types.idl |  2 ++
  tools/xl/xl_parse.c              |  3 +++
  xen/arch/arm/domain.c            | 10 ++++++++--
  xen/include/asm-arm/domain.h     |  1 +
  xen/include/public/domctl.h      |  4 ++++
  7 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d69869..d3e795a38670 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -502,6 +502,8 @@
   */
  #define LIBXL_HAVE_X86_MSR_RELAXED 1

+#define LIBXL_HAVE_ARM_VPMU 1
+
  /*
   * libxl ABI compatibility
   *
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6e0039..89865a90dd3e 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -29,6 +29,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
      uint32_t vuart_irq;
      bool vuart_enabled = false;

+    if (d_config->b_info.arch.vpmu)
+        config->flags |= XEN_DOMCTL_CDF_PMU;
+
      /*
       * If pl011 vuart is enabled then increment the nr_spis to allow 
allocation
       * of SPI VIRQ for pl011.
diff --git a/tools/libs/light/libxl_types.idl 
b/tools/libs/light/libxl_types.idl
index 3f9fff653a4a..daf768cba568 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -644,6 +644,8 @@ libxl_domain_build_info = Struct("domain_build_info",[

      ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                 ("vuart", libxl_vuart_type),
+                              # XXX: Can this be common?
+                               ("vpmu", boolean)
                                ])),
      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                                ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd534..6e497cc0b67e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2729,6 +2729,9 @@ skip_usbdev:
          }
      }

+    /* XXX: This probably want to be common or #ifdef-ed */
+    xlu_cfg_get_defbool(config, "vpmu", &b_info->arch_arm.vpmu, 0);
+
      if (!xlu_cfg_get_string (config, "tee", &buf, 1)) {
          e = libxl_tee_type_from_string(buf, &b_info->tee);
          if (e) {
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d46..a0e2321008ab 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -276,6 +276,9 @@ static void ctxt_switch_to(struct vcpu *n)
       * timer. The interrupt needs to be injected into the guest. */
      WRITE_SYSREG(n->arch.cntkctl, CNTKCTL_EL1);
      virt_timer_restore(n);
+
+    /* XXX: Check the position and synchronization requirement */
+    WRITE_SYSREG(n->arch.mdcr_el2, MDCR_EL2);
  }

  /* Update per-VCPU guest runstate shared memory area (if registered). */
@@ -585,6 +588,9 @@ int arch_vcpu_create(struct vcpu *v)
      v->arch.vmpidr = MPIDR_SMP | vcpuid_to_vaffinity(v->vcpu_id);

      v->arch.hcr_el2 = get_default_hcr_flags();
+    v->arch.mdcr_el2 = HDCR_TDRA|HDCR_TDOSA|HDCR_TDA;
+    if ( !(v->domain->options & XEN_DOMCTL_CDF_PMU) )
+        v->arch.mdcr_el2 |= HDCR_TPM|HDCR_TPMCR;

      if ( (rc = vcpu_vgic_init(v)) != 0 )
          goto fail;
@@ -622,8 +628,8 @@ int arch_sanitise_domain_config(struct 
xen_domctl_createdomain *config)
  {
      unsigned int max_vcpus;

-    /* HVM and HAP must be set. IOMMU may or may not be */
-    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
+    /* HVM and HAP must be set. IOMMU and PMU may or may not be */
+    if ( (config->flags & ~(XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_pmu)) !=
           (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
      {
          dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c9277b5c6d94..14e575288f77 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -166,6 +166,7 @@ struct arch_vcpu

      /* HYP configuration */
      register_t hcr_el2;
+    register_t mdcr_el2;

      uint32_t teecr, teehbr; /* ThumbEE, 32-bit guests only */
  #ifdef CONFIG_ARM_32
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842da..db9539ddd579 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -71,6 +71,10 @@ struct xen_domctl_createdomain {
  #define _XEN_DOMCTL_CDF_nested_virt   6
  #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)

+/* Should we expose the vPMU to the guest? */
+#define _XEN_DOMCTL_CDF_pmu           6
+#define XEN_DOMCTL_CDF_pmu            (1U<<_XEN_DOMCTL_CDF_pmu)
+
  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
  #define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt

> In the meantime we will start maintaining an internal branch with patches refused upstream as this is blocking us.

For the future, please consider a per-domain option from the beginning. 
This is not much extra effort (see the patch above) and would make the 
acceptance of a patch more likely.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 09:06:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 09:06:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177185.322456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLigA-0006rt-0U; Thu, 02 Sep 2021 09:06:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177185.322456; Thu, 02 Sep 2021 09:06:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLig9-0006rm-To; Thu, 02 Sep 2021 09:06:37 +0000
Received: by outflank-mailman (input) for mailman id 177185;
 Thu, 02 Sep 2021 09:06:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLig8-0006rc-OZ; Thu, 02 Sep 2021 09:06:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLig8-0000rX-Dt; Thu, 02 Sep 2021 09:06:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLig8-0002g2-3j; Thu, 02 Sep 2021 09:06:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLig8-0000Iy-3D; Thu, 02 Sep 2021 09:06:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pXBTzorr5Q2J+UyjMbNLPDqtGFYOWLYYZGBHxKHAM+8=; b=audqGzQxEbyXfgwgwUVKGlreAC
	XzPr3tuyeftb4dAXnKupEQ4CKZhWfUejS1ALpyTfxgQ1eRWjNbKwuMw40F4Z6GH0Mu5HTqYSgOBsR
	ocnb5ftzvb44Rwly6ZOqWpkc8uQ5e+Kza9J38LaiRTgDBiShnWl23V+eccyUnzq0KNDI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164731-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164731: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7c636d4d20f8c5acfbfbc60f326fddb0e1cf5daa
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 09:06:36 +0000

flight 164731 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164731/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-xl         22 guest-start/debian.repeat fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7c636d4d20f8c5acfbfbc60f326fddb0e1cf5daa
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  397 days
Failing since        152366  2020-08-01 20:49:34 Z  396 days  686 attempts
Testing same since   164731  2021-09-02 00:11:48 Z    0 days    1 attempts

------------------------------------------------------------
7128 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2174137 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 09:12:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 09:12:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177192.322471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLilY-000083-My; Thu, 02 Sep 2021 09:12:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177192.322471; Thu, 02 Sep 2021 09:12:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLilY-00007w-Jd; Thu, 02 Sep 2021 09:12:12 +0000
Received: by outflank-mailman (input) for mailman id 177192;
 Thu, 02 Sep 2021 09:12:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ch7x=NY=redhat.com=stefanha@srs-us1.protection.inumbo.net>)
 id 1mLilX-00007p-L9
 for xen-devel@lists.xen.org; Thu, 02 Sep 2021 09:12:11 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 2a1271a1-7667-42ff-ad86-b09dcbcb361b;
 Thu, 02 Sep 2021 09:12:10 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-483-b6o_pRY-Nx-21DG1pghVNQ-1; Thu, 02 Sep 2021 05:12:06 -0400
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
 [10.5.11.16])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AE1DD1005D70;
 Thu,  2 Sep 2021 09:12:02 +0000 (UTC)
Received: from localhost (unknown [10.39.194.169])
 by smtp.corp.redhat.com (Postfix) with ESMTP id C51215C3DF;
 Thu,  2 Sep 2021 09:12:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a1271a1-7667-42ff-ad86-b09dcbcb361b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1630573929;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=kF1K6NnmCiy6sVyUFnUfFMOScUNqa4i7qai+pdkjbOE=;
	b=PCm2Ip3UzPjOf90TT1Uy6L1zPDrWkozzkKvlArryuXu3+uUNq5Ounk7TOtnJT3ZLqkWOY2
	98+LYPfqhvdDK2XV/Dwam+SwbUVAdHvjhDbIMt3Le7zJGz1Ss3ozNyGIvoEBLIqSTShZQ9
	0iX1IjWiAQsPanhjSh1heS0NBWqKQCE=
X-MC-Unique: b6o_pRY-Nx-21DG1pghVNQ-1
Date: Thu, 2 Sep 2021 10:12:00 +0100
From: Stefan Hajnoczi <stefanha@redhat.com>
To: Alex =?iso-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	virtio-dev@lists.oasis-open.org,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	AKASHI Takahiro <takahiro.akashi@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>, Wei.Chen@arm.com,
	olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
	Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org,
	jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org,
	Elena Afanasova <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <YTCVYHyjhdaAPAni@stefanha-x1.localdomain>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
 <875yvkh1p1.fsf@linaro.org>
MIME-Version: 1.0
In-Reply-To: <875yvkh1p1.fsf@linaro.org>
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="wqq+Rojkn7jctvdi"
Content-Disposition: inline

--wqq+Rojkn7jctvdi
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn=E9e wrote:
>=20
> Stefan Hajnoczi <stefanha@redhat.com> writes:
>=20
> > [[PGP Signed Part:Undecided]]
> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
> >> > Could we consider the kernel internally converting IOREQ messages fr=
om
> >> > the Xen hypervisor to eventfd events? Would this scale with other ke=
rnel
> >> > hypercall interfaces?
> >> >=20
> >> > So any thoughts on what directions are worth experimenting with?
> >> =20
> >> One option we should consider is for each backend to connect to Xen vi=
a
> >> the IOREQ interface. We could generalize the IOREQ interface and make =
it
> >> hypervisor agnostic. The interface is really trivial and easy to add.
> >> The only Xen-specific part is the notification mechanism, which is an
> >> event channel. If we replaced the event channel with something else th=
e
> >> interface would be generic. See:
> >> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/h=
vm/ioreq.h#L52
> >
> > There have been experiments with something kind of similar in KVM
> > recently (see struct ioregionfd_cmd):
> > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.16=
13828727.git.eafanasova@gmail.com/
>=20
> Reading the cover letter was very useful in showing how this provides a
> separate channel for signalling IO events to userspace instead of using
> the normal type-2 vmexit type event. I wonder how deeply tied the
> userspace facing side of this is to KVM? Could it provide a common FD
> type interface to IOREQ?

I wondered this too after reading Stefano's link to Xen's ioreq. They
seem to be quite similar. ioregionfd is closer to have PIO/MMIO vmexits
are handled in KVM while I guess ioreq is closer to how Xen handles
them, but those are small details.

It may be possible to use the ioreq struct instead of ioregionfd in KVM,
but I haven't checked each field.

> As I understand IOREQ this is currently a direct communication between
> userspace and the hypervisor using the existing Xen message bus. My
> worry would be that by adding knowledge of what the underlying
> hypervisor is we'd end up with excess complexity in the kernel. For one
> thing we certainly wouldn't want an API version dependency on the kernel
> to understand which version of the Xen hypervisor it was running on.
>=20
> >> There is also another problem. IOREQ is probably not be the only
> >> interface needed. Have a look at
> >> https://marc.info/?l=3Dxen-devel&m=3D162373754705233&w=3D2. Don't we a=
lso need
> >> an interface for the backend to inject interrupts into the frontend? A=
nd
> >> if the backend requires dynamic memory mappings of frontend pages, the=
n
> >> we would also need an interface to map/unmap domU pages.
> >>=20
> >> These interfaces are a lot more problematic than IOREQ: IOREQ is tiny
> >> and self-contained. It is easy to add anywhere. A new interface to
> >> inject interrupts or map pages is more difficult to manage because it
> >> would require changes scattered across the various emulators.
> >
> > Something like ioreq is indeed necessary to implement arbitrary devices=
,
> > but if you are willing to restrict yourself to VIRTIO then other
> > interfaces are possible too because the VIRTIO device model is differen=
t
> > from the general purpose x86 PIO/MMIO that Xen's ioreq seems to
> > support.
>=20
> It's true our focus is just VirtIO which does support alternative
> transport options however most implementations seem to be targeting
> virtio-mmio for it's relative simplicity and understood semantics
> (modulo a desire for MSI to reduce round trip latency handling
> signalling).

Okay.

Stefan

--wqq+Rojkn7jctvdi
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmEwlWAACgkQnKSrs4Gr
c8iywwf/UbbZqAAwRx5aV+yxn3NfXRbvDky1X190RHTAps3WhO8yK2JbKJD8iKbQ
rshIvCjdE8gtVEBLo1T+73NCbrLFablTajILJ6OZXpKvUb2Q4xp2v2fowVz49ggv
07MKYC7J+AkIXEIxZ5vocByUaaF150FWIyU8Anb5GDLnF7BVFXF1mM0iqtgELgS7
0/UOVvjXSs63QNWRqIkGNuCsb9E7+cD9Rnn805nrwxi3A1Mr2qJXprgDHOIdc8DZ
GfH+4OCe2sfLjXDg8UD586JC81vR1MR/8XDS7Ojv1/WOwZFvcpboOHp/WBxaxOLU
1VbgmtZyWDa51CFJZBkzq+DRQe8aEw==
=LPa3
-----END PGP SIGNATURE-----

--wqq+Rojkn7jctvdi--



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 09:57:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 09:57:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177202.322488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjSk-0005X1-AL; Thu, 02 Sep 2021 09:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177202.322488; Thu, 02 Sep 2021 09:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjSk-0005Wu-6q; Thu, 02 Sep 2021 09:56:50 +0000
Received: by outflank-mailman (input) for mailman id 177202;
 Thu, 02 Sep 2021 09:56:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLjSi-0005Wl-F0
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 09:56:48 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e37a3955-8d92-44b4-85b9-c9d90503737a;
 Thu, 02 Sep 2021 09:56:46 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-5S3dmyduOmedLylCb4vH8w-1; Thu, 02 Sep 2021 11:56:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6480.eurprd04.prod.outlook.com (2603:10a6:803:11d::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 09:56:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 09:56:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0192.eurprd02.prod.outlook.com (2603:10a6:20b:28e::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 09:56:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e37a3955-8d92-44b4-85b9-c9d90503737a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630576605;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vgtEOi+Tk+N1Qhq1ZjjXcboumpOXjoDLgESCBpFZp68=;
	b=eC9Lau+U+rDtM+tkCWN/Rqh3kSuqC14DFGAo2R2zVOPsMHzIQAgXeAECHZQvhK/AjFfdKr
	lsQRpWqeFQ3EZJgf1ZOlq7OrIknfOkAFIbVSmeVIem3jc8xYjT0GbyBgr6aL8yrnC5/NSO
	L2kDKGMXcNIu4gtWPyiR4/NCQiKYP8o=
X-MC-Unique: 5S3dmyduOmedLylCb4vH8w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DphjXKh/tuw+Tw+JeXFFe+06FkWazd/Hb6rKuH8qQZY7gObzhmlvNB8oJkzs6wx+xpq5/6f6dIYiAM6ZBv5QrL4GcAvTguGWMKxIkez4FEizSLoykWy1EJDqZsbHKRbmXdbBVK8BYid+J4WIKzMcb7TkRj3KK3+MjZPUJXORURdsCTF1RAY2zOMo1eb/3O0kBhA5yFvJGclwkhGAgSlmE3SZRIi3EId3jiXo5tkXyh/rVufLaHglK7Hw+ImqI/3i8zSQCCOmGCgPk2sV2VXT3q9rEXAjoFOLeQFnD+ze9Wa6xKXneTOREMtwDWLz9lN/6scGR+eALsaoKyOGk+Ycig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vgtEOi+Tk+N1Qhq1ZjjXcboumpOXjoDLgESCBpFZp68=;
 b=bjqUCXNF+C5eHhPK7u5tG0SbwsPcL4z5LJ7ozNCnIqL7VgpW9TBON1HL4+hQSxf2t7k9mr7vQZYEXKbysuHkdTJH65pWuXqrer7C2ZoowJAJzCpI5OgYsOzzwLZ69YT+DsFD2Hle2nbtsFgofIIZNTNyHWBmb8GLxOAGbGxfbJQi654GB0Wr/Qt9unJQlzTGDy1YUWyH9K1RowkanTK47b3ix4Ic83ryMAJAn4B44bD08WT3ej/BwS/qYq+9B7/Nf1uYSfQ78BsrcGmCzZH6eeSlcEh6lSYMySHiAvssx73QOehzl9KLF7cn5Id2qMpzKRn8aOlZUtdryeD5DLUK1A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] Grab the EFI System Reference Table and check it
To: Demi Marie Obenour <demi@invisiblethingslab.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <YSt7P0a63GCxygAa@itl-email>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <eec7828f-ed75-2aab-982a-ad98352c5728@suse.com>
Date: Thu, 2 Sep 2021 11:56:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YSt7P0a63GCxygAa@itl-email>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0192.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 73c167e8-17b4-4a2a-1237-08d96df7f5a2
X-MS-TrafficTypeDiagnostic: VE1PR04MB6480:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB648061CA10813B7BBE06FA3CB3CE9@VE1PR04MB6480.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2733;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PwSnTndDfRlEZSTHreSGL1vE/F5g7LaFIDdN1yXDC+fHdV+TCWuz0z4xlu9sGIqlFwWuGwb/astUy0TtKPWgsmlYw8nHSu78teev3fvsIfcKrpS7ZUCRB3uvjN1mO24wASf4X9JKt0MiC+QICpjn4g5jv6K1Mza2YFZDXZbGfV+8whdeimNhWdYfM5594lJ2BU6Ou8AQ09987Rfv/tgAAGFip+OtqpAyiK9xgIW6pbC3GBvAMbGwdqMKGPK0D9obYcHOJD0qMaPULgZ9OIQNcjU3thIduV6kQo0J4+hd6B4dDIw4rDPoEZXmB2nl6Tz2NEA4ia6JeqebldPNl2gAufgb2q9WfzPnCtGgDBFZHZ1pvBtHT9xZX/EyfbGAu7yFm4w+P4VdkiBGWdmvKye3fW74+7Nh1laj/lWYKZBwza2jT497Ay/7yCG7c8ZGslPBFCAkSmWcqJT4SuTpJ2QQDDFMvh/IHUbR2S+QQnHiqHx1QP7+tjqLt5S1fuyDSatD+vg4kQTNfbgCf6/83YNcxNk3wYlNaxx6mJX0wcWE/dN5FcRVeLPlOObtOSa7wnrffIBnxjH4OUuu4QK5cUWUr86vG1qefyf8u+8OFOyp5YGbMPM3iA4M6kJwdfLhVGl+7/3fIX50/GD/avkPiui56Ud0zhCEmrlVhAsNMLycgy8f+eiAMPxWmMusdV/sa1SD8swf901HTqb6G1tOK7dwDLFkXwP6vXTLa+aF2mA5pxg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(136003)(396003)(366004)(376002)(54906003)(53546011)(2616005)(16576012)(31686004)(316002)(956004)(86362001)(8936002)(186003)(31696002)(36756003)(38100700002)(8676002)(2906002)(83380400001)(6486002)(5660300002)(66946007)(66556008)(66476007)(6916009)(7416002)(478600001)(4326008)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?Windows-1252?Q?Vg5sufvTDInRUQKy3fm5myKiOVueIk5seUpjp+vk7zthd9aYSTfCl4K8?=
 =?Windows-1252?Q?EUD3civvNXl7PjEQCj7LQXigP+8qKzrz8ansQUqcDYoKl8fpx9MbF8Je?=
 =?Windows-1252?Q?S8dTe0ziPHBJRzGkYsjkSLvEwBTZdhzYNMlC33eOGgFOwavektjNugXy?=
 =?Windows-1252?Q?DX3l9F2sh1G/MH4MhQUojZHy1E1g26EgIe644PjcaJrFg14XYE/dDEnC?=
 =?Windows-1252?Q?LqdS2zHJVrMC2ynPQm1hDLzVCITIJp4XjS0bcrFnD6fBCmFLYJcKB7/a?=
 =?Windows-1252?Q?5lvKoTtI8GHPQfqjhRhNRycG44Up501ulCJuvhK7kd8OtMTuuzEgU5gp?=
 =?Windows-1252?Q?V2UtQm9TShHfWAEFDpb6+xG19XCuOOctnxNe5nrMv0PIV5ihUDOFXDVw?=
 =?Windows-1252?Q?R68Bww0Sx8qBPdxD0TqzbZGVH3haOm6/jqkWzZrhKZyBaHoYzOPVoJA5?=
 =?Windows-1252?Q?VQ1z9iRDoThEx/ghPERq8zW4ExuttbAzLSRz6NphoYpLjRwUj4Zq2inQ?=
 =?Windows-1252?Q?bZsZarP3bdXqhkuoUQKTW7cfZ78Q3qokW6+QdS5ZBUkuoGsSKVM5B7Ai?=
 =?Windows-1252?Q?P2m5e8ldgdbQ+oHZcszFeMddAqFHo6RlPZRsBQqDSzpaJZ3u3rG+lQAk?=
 =?Windows-1252?Q?+p1JQ81Nt3cJ0W8DvQbkLSVlSGNXBOeTHimOWYV2al5iDeV6u7oVhtea?=
 =?Windows-1252?Q?1sxgoDF4cMYAwJP9zBlpIffnYC4GDa/uof1g03riroyvHnyW/6nA6uHr?=
 =?Windows-1252?Q?9x9cmzri9DbDMLHjES2B2cRjhW0PWuACP5jlr/unSHxnrwxJu6BRyxAX?=
 =?Windows-1252?Q?Oy7mBxvQEEScJO0/nJZVQH9z6OGho8jH47O0einJMul+YNz+kzJ1R2DI?=
 =?Windows-1252?Q?roJgF0rIMn+rWy9Tq2SL8H2hhQWqth4DrVAyZF7DMt6ZAFtuLFxVJC7T?=
 =?Windows-1252?Q?T5ueHSkMkHby3lWztFN9G91rPRUvONuo5H4pWXXMihfHYPr0c+71LKRS?=
 =?Windows-1252?Q?CNUU15EeOsY4CTna++lLh+p9Ho6bKgQcfteJBRyYYUCOxtuUoQ5gifhK?=
 =?Windows-1252?Q?8hdQJRWpvRJDa7N5fmKVUflUny89k6rgwtCg7In5gZSDP0lDmyMD4Zn6?=
 =?Windows-1252?Q?TYeugOGyI334vnwJmKyvaHRoMgCDhG6RJePIcKYyz9UG73cMmDhn+SUC?=
 =?Windows-1252?Q?BAZZ9eGvnauLVm24NNw4Sf3W74SBWPEKypjOBwcFAdqvFwbwWZqztHfq?=
 =?Windows-1252?Q?yO16S97MkgCK+armTkgeG89PEGsDcDXfsbHdyYA8I09vWMXCeMVqLSOq?=
 =?Windows-1252?Q?ywFpivYCKn2GAsWgVjXM/nWrWCzvIOXHwGt8ylCv5t6uRRb3ocGv8Esb?=
 =?Windows-1252?Q?Q55+dt6V43QbSOaQBGBFhWlKhEQfLc/NMliOI+xtQ6f6QYCZhkqdWakG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73c167e8-17b4-4a2a-1237-08d96df7f5a2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 09:56:39.6444
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 76jU/swXVLyo8tN/JEWV/8Kguexxaxw/HhshhM54VOsXl380TX+Z8P664cD4ROIqGyszd/Skv9LugAMB36JXTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6480

On 29.08.2021 14:18, Demi Marie Obenour wrote:
> The EFI System Reference Table (ESRT) is necessary for fwupd to identify

To properly match the UEFI spec, both here and in the subject I suppose
you mean "Resource" instead of "Reference"?

> @@ -171,7 +172,7 @@ static void __init efi_arch_process_memory_map(EFI_SYSTEM_TABLE *SystemTable,
>          {
>          case EfiBootServicesCode:
>          case EfiBootServicesData:
> -            if ( map_bs )
> +            if ( map_bs || desc == esrt_desc )
>              {
>          default:
>                  type = E820_RESERVED;

Elsewhere you also permit ESRT to live in EfiRuntimeServicesData. The
code addition here looks to contradict that.

Furthermore you're potentially making a large chunk of data unavailable
just because it contains a small piece of data which needs to be
preserved.

> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -93,6 +93,23 @@ typedef struct _EFI_LOAD_OPTION {
>      CHAR16 Description[];
>  } EFI_LOAD_OPTION;
>  
> +struct esrt_entry {
> +    EFI_GUID fw_class;
> +    uint32_t fw_type;
> +    uint32_t fw_version;
> +    uint32_t fw_lowest_supported_version;
> +    uint32_t fw_capsule_flags;
> +    uint32_t fw_last_attempt_version;
> +    uint32_t fw_last_attempt_status;
> +};
> +
> +struct esrt {
> +    uint32_t esrt_count;
> +    uint32_t esrt_max;
> +    uint64_t esrt_version;
> +    struct esrt_entry esrt_entries[];
> +};

These don't match the gnu-efi style (where we're generally taking
pieces from), but since I can't spot that package supporting ESRT at
all, we have to roll our own, so the style inconsistency is perhaps
okay. But at least - alongside EFI_GUID - I think you want to use EFI
types (UINT<nn>).

> @@ -166,6 +183,41 @@ static void __init PrintErr(const CHAR16 *s)
>      StdErr->OutputString(StdErr, (CHAR16 *)s );
>  }
>  
> +enum esrt_status {
> +    ESRT_NOTFOUND = 0,
> +    ESRT_GOOD,
> +    ESRT_BAD,
> +};

I think we generally try to limit all-caps identifiers to macros.

> +static enum esrt_status __init is_esrt_valid(
> +    const EFI_MEMORY_DESCRIPTOR *const desc) {

Style nit: Brace goes on it own line.

> +    size_t available_len, esrt_len, len;
> +    const uintptr_t physical_start = desc->PhysicalStart;
> +    const uintptr_t esrt = efi.esrt;

Like said above and to match other code in this file, EFI types
(UINTN in particular) should be preferred.

> +    const struct esrt *esrt_ptr;
> +
> +    len = desc->NumberOfPages << EFI_PAGE_SHIFT;
> +    if ( esrt == EFI_INVALID_TABLE_ADDR )
> +        return ESRT_BAD;

I think this is rather ESRT_NOTFOUND, too? But see also below.

> +    if ( physical_start > esrt || esrt - physical_start >= len )
> +        return ESRT_NOTFOUND;
> +    if ( ! ( desc->Attribute & EFI_MEMORY_RUNTIME ) &&
> +           ( desc->Type != EfiRuntimeServicesData ) &&
> +           ( desc->Type != EfiBootServicesData ) )

As you say in the description, the spec mandates the latter. I can
accept you also wanting to cover other plausible types, but then
please add a brief comment mentioning the intentional diverging
from what the spec says. This would then hopefully also clarify
the curious redundancy between checking EFI_MEMORY_RUNTIME and
EfiRuntimeServicesData.

As to style - there are quite a few too many blanks here.

> +        return ESRT_BAD;
> +    available_len = len - (esrt - physical_start);
> +    if ( available_len < offsetof(struct esrt, esrt_entries) )

Iirc both gcc and clang allow sizeof(*estr_ptr) despite the
flexible array member - please prefer that form (also again further
down).

> +        return ESRT_BAD;
> +    esrt_ptr = (const struct esrt *) esrt;

For this and the sizeof() below to be applicable, you need to check
the version to be 1. Also, as to style, there's not supposed to be
any blank after the closing parenthesis.

> +    if ( esrt_ptr->esrt_count <= 0 ||
> +         __builtin_mul_overflow(esrt_ptr->esrt_count,
> +                                sizeof(struct esrt_entry),

sizeof(esrt_ptr->esrt_entries[0]) or alike please.

Also - is __builtin_mul_overflow() supported by all gcc versions we
support building this code with? And is this overflow check needed
in the first place? After all we build EFI code only for 64-bit
hypervisors, and a 32-bit quantity times the sizeof() can't overflow
in 64 bits.

> +                                &esrt_len) ||
> +         esrt_len > available_len - offsetof(struct esrt, esrt_entries) )
> +        return ESRT_BAD;
> +    return ESRT_GOOD;
> +}

Style nit: Please put a blank line ahead of the function's main
"return" statement.

> @@ -835,6 +887,10 @@ static void __init efi_tables(void)
>  {
>      unsigned int i;
>  
> +    BUILD_BUG_ON(sizeof(struct esrt_entry) != 40);
> +    BUILD_BUG_ON(_Alignof(struct esrt_entry) != 4);
> +    BUILD_BUG_ON(offsetof(struct esrt, esrt_entries) != 16);

Why? We're not doing this for other types declared locally in this
source file afaics. Plus - did you check that all gcc versions we
support for building this code can actually deal with the C99
_Alignof()? I would generally have expected you to use __alignof(),
as we do elsewhere.

> @@ -1042,9 +1101,7 @@ static void __init efi_exit_boot(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *Syste
>      EFI_STATUS status;
>      UINTN info_size = 0, map_key;
>      bool retry;
> -#ifdef CONFIG_EFI_SET_VIRTUAL_ADDRESS_MAP
>      unsigned int i;
> -#endif
>  
>      efi_bs->GetMemoryMap(&info_size, NULL, &map_key,
>                           &efi_mdesc_size, &mdesc_ver);
> @@ -1055,6 +1112,10 @@ static void __init efi_exit_boot(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *Syste
>  
>      for ( retry = false; ; retry = true )
>      {
> +        enum esrt_status esrt_status = ESRT_NOTFOUND;
> +        const EFI_MEMORY_DESCRIPTOR *esrt_desc =
> +            (const EFI_MEMORY_DESCRIPTOR*) EFI_INVALID_TABLE_ADDR;

Style: Misplaced blank.

> @@ -1063,8 +1124,31 @@ static void __init efi_exit_boot(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *Syste
>          if ( EFI_ERROR(status) )
>              PrintErrMesg(L"Cannot obtain memory map", status);
>  
> +        for ( i = 0; i < efi_memmap_size; i += efi_mdesc_size )
> +        {
> +            EFI_MEMORY_DESCRIPTOR *desc = efi_memmap + i;
> +            switch ( is_esrt_valid(desc) )

Style: Blank line please between declaration(s) and statement(s).
I question the need for the local variable though - you only need
it ...

> +            {
> +            case ESRT_NOTFOUND:
> +                break;
> +            case ESRT_GOOD:
> +                esrt_status = ESRT_GOOD;
> +                esrt_desc = desc;

... here.

> +                break;
> +            case ESRT_BAD:
> +                esrt_status = ESRT_BAD;
> +                break;
> +            }
> +        }
> +        if ( esrt_status != ESRT_GOOD )
> +            efi.esrt = EFI_INVALID_TABLE_ADDR;

In the end you don't distinguish ESRT_BAD from ESRT_NOTFOUND, so I
don't see the need for the enum. The function can simply return
bool then.

> +        /*
> +         * We cannot pass efi.esrt because we need to explicitly compare the
> +         * descriptor pointers for equality.
> +         */
>          efi_arch_process_memory_map(SystemTable, efi_memmap, efi_memmap_size,
> -                                    efi_mdesc_size, mdesc_ver);
> +                                    efi_mdesc_size, mdesc_ver, esrt_desc);

I'm not convinced this is "need" (as the comment says), but at best
"want", for being an implementation choice. But the model itself is
suspect to me in the first place, as said above.

> --- a/xen/include/xen/efi.h
> +++ b/xen/include/xen/efi.h
> @@ -19,6 +19,7 @@ struct efi {
>      unsigned long acpi20;       /* ACPI table (ACPI 2.0) */
>      unsigned long smbios;       /* SM BIOS table */
>      unsigned long smbios3;      /* SMBIOS v3 table */
> +    unsigned long esrt;         /* EFI System Reference Table */
>  };

I have to admit that I'm having difficulty spotting why this field
needs adding here: No other component in Xen wants to use it. Yet
that's what the struct is for.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 10:09:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 10:09:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177209.322499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjec-0007XW-CB; Thu, 02 Sep 2021 10:09:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177209.322499; Thu, 02 Sep 2021 10:09:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjec-0007XP-8Y; Thu, 02 Sep 2021 10:09:06 +0000
Received: by outflank-mailman (input) for mailman id 177209;
 Thu, 02 Sep 2021 10:09:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLjeb-0007XJ-KY
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 10:09:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b460bf3-d7f4-4a85-8e74-5a068b0940e0;
 Thu, 02 Sep 2021 10:09:04 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2052.outbound.protection.outlook.com [104.47.6.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-1w7TI-zcNQWOKAlYaIRzgQ-1; Thu, 02 Sep 2021 12:09:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2605.eurprd04.prod.outlook.com (2603:10a6:800:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.20; Thu, 2 Sep
 2021 10:09:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 10:09:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0008.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Thu, 2 Sep 2021 10:09:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b460bf3-d7f4-4a85-8e74-5a068b0940e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630577343;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9yxvLHW7DrlOt3aGmZBGtc6h5E1gB+AHZW+4J/6Z/sQ=;
	b=Zgl9xpmYjbqXHET/JW1W6JRCm9RpG9smaBaLXKfoCX2ubxjnJ1fgFI2omVTSxMSSJC/xIT
	Svgmt7S1PuPe54FfjaImFp9ZuVbD5hWWeh0D0wW8ZpoYN/FtuG7MJk3FjVLuI0X4+F4J2B
	oVwUJxhoR4aBm4tl8ZBW1nv06s4ZIEc=
X-MC-Unique: 1w7TI-zcNQWOKAlYaIRzgQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YE3kKwatrUCoCXOglCwjuP8j/DwoQdsN5AQpaHp6x5JNm0bVrvI+8umQIsPmCs9Rw9AQjZ7EGifkEohiZCeX02SD+iYM70WFMyHfSoevDPrKpIAq9a0oiyhhDElTclNNgcbf54UbO+lGehCheW2IRDLYOaaCwV7EVpcaRHYx2BAAGrrRZviCYkjOnUJFPxHPLKqMHYklUpAhzsWY3cn1u+X7ViTTNwr0lbduK7A+wzLic05ZKBJzPQ/mjZkCh97SUHpCfgQkwmhDOEut0rSYgGO33RdVWeGlvg78XOsGZ/m6k5G04NKHJAg19AaHnXfZocaCSGpLwIT2CMd+1Ltb1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9yxvLHW7DrlOt3aGmZBGtc6h5E1gB+AHZW+4J/6Z/sQ=;
 b=XYc3Fza8SrSazbCb5uIgrFDMvUSw6QODoR0TQQulR1fvY4XJ2x1fN1xD0nTwD8adzlAm4kblSLKmhi5sxoLxj8TNPHdmshsSOdEw2ly64V4mOlxO3gPcQThi1vglUqac+QyT+6XIduiKHytlL44Do+ewW3tAToPWP770MNAwONJiCyxqgF/LAE9BFCEAiRJl6FY4tv/c1XyoMrFn3mGsfA0FMmQ2ViOHLSwwDknnBO3yPib3Z1FrUssf1Sa17+keYSAM1Wuj7A6r3KEctVxl2908VeiOE0hfSCQNyyhCbIrecOPNkw8tv+zqWc88VGMPU/s5/1WteEjoxHJykHg9sw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 01/51] build: introduce cpp_flags macro
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-2-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ba3cfb06-1d89-ff62-1012-6edc2a9c5259@suse.com>
Date: Thu, 2 Sep 2021 12:08:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210824105038.1257926-2-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0008.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 108c2e8b-c526-470e-b675-08d96df9afa5
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2605D98DFD577F6E824DDE3EB3CE9@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2043;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	d+30HUtBcErhi5zzCKeaK66+xLt/mMlwF92UDt1wXWKN07pCI8WqWRBs39k/yeWjYo8wpaadHJ84zlwmX35ioUvaSNMwa5DKVZAjJIKT7fxvvyVIJKaM/nHyO0sDAtAMaG1Kz4IPiwVa2sXVKxwyMqcuoD7qDpcZid9gcoNF75O6c0Max72Gopda9bVVaHJOeGdstA5DqKe2yLPP0+p+Z9hL7s0QzyFM/m8ru9k56+mt/KhLV8BEjJyYFoEYlFuZDdbwzchKjXePa8QG++7ONDuaw33he4lYYC5yECdcjaWDot2Qydph5TJW0Z4U+wP5vx+o179HfyL+OqxU3h2kwYX8ql1oZ8brwwdnZ+G3Slo/p13b439V45H0Kni59tooy19mJWCi3MxTvNExGloah4Py2dtoWYyB73BWn7dHqC0mVNgHUpMGd7zPJcaffulZMLDjr8yRnjs1WiqfOxGW8GxdO+voifx13D6NG5TxQMlcJXPNLQO+7GMQTcXN1bgszYEAbj8UA13/aESwbKK8dI4ziCMya7FMWItmMa9QhpoLQiZgLfseVxdcrLb+vxv/4K4a2AfoKkZHpNPIANGiACwmUWriRqLeSJkWfJUENGw983gJhcCiDecA2aJcasDjkIAEWRuQHuZOto+rs1Q4868jbjsI0pDIbltvvpVa1erZkjbtA0WlNheQcA5DeYYmpq12HrJpj1Tb5DCCD9jqNDjpxuQVJYx2HDUA8ItPiEg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(396003)(366004)(376002)(39860400002)(136003)(6916009)(31686004)(54906003)(86362001)(26005)(478600001)(6486002)(66946007)(66556008)(53546011)(186003)(66476007)(31696002)(5660300002)(4326008)(7416002)(36756003)(8676002)(956004)(38100700002)(2616005)(2906002)(16576012)(8936002)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?N3lTNEF6SHprZXMrOW9vL0xiWFI1TXVWVGY3VEZlODE3UlhQUTdCQStvRTVU?=
 =?utf-8?B?bjB4U0hZMmRFY2ZZb3dBbHdHUXRtdHpLSDJxajZ0YnlTdDRiZStjSGJpeGFu?=
 =?utf-8?B?bGs5MVBDZ2RYUVB6WGpSWWcrQkZldU1KdysvcXlwbjcvK25teVB5VE5paU12?=
 =?utf-8?B?NXBoT2R3YUxQY25PUzVralA5a3gyNW1oRWV1NUxqcGxERFJKclpiVWVNWG5p?=
 =?utf-8?B?TUtyS09SMkN0OExZeU5HeEIxMTVKeUFJeVluQ2JyeVRTdFdtdFJ1OXRRZEdU?=
 =?utf-8?B?RThZSTVDeEMvWXFhNDJDNCt6RmNrckg1VFhGRzJkazB3Wk53NXZnQjkya1ZG?=
 =?utf-8?B?WElITWZkZ25YRTJBaWNkY2N1emJqUS9FN0s5OWtMeGVvWHh2Qmh3WVEyNExn?=
 =?utf-8?B?TmVsbHFxb3FIcmxCNmtSVUowYm9NVFQrRDF5ZTkxdStaOWdPdVJkSEp0THJR?=
 =?utf-8?B?YjhIdmVCL0dwN0ZpeGR3US80a1NpVXpESFRSdzRYMU1JT2NTSjcxL3BzSnZ5?=
 =?utf-8?B?Qk1tbC9iSE5TWVlwMEpXUEd4dHZoTWNYb0VWOWk5b1VKblhIRDIvR3JBUXJ6?=
 =?utf-8?B?ZXNPQ1hmR0huQjFkb0YvZ21UVmtIZlV1NUpoek15eGtBMWYxaWgrbk9WTHhN?=
 =?utf-8?B?U3VnWjFESEpLUGE0VUdkMFFDUWlEQkNySXBNVEZNR2kxZ0dXelh1Rm5admhi?=
 =?utf-8?B?N2Nuak9iWWRsR095TmVQRE9yOWh2RkFLaXU4dHEzQzRJWFpFbnFERUd3NVR6?=
 =?utf-8?B?b0MzMkFyOG1YTlNmcWtqWVhYb2RyeGxHOS9PSDR1QiszNTYyS2w4S3lDWCtC?=
 =?utf-8?B?dVhlV2pZWnV2M01qSW0vUi9ncy9UQTE0U0U0WHc5T2dCMndXZ1AyYjZqQk5M?=
 =?utf-8?B?WUV5MzQzV01pTVI4RCt0cTkwZG9LU3dYamNYalFsVjNkV05sVFdUUEpXTlF2?=
 =?utf-8?B?OXpFNWFxd3hGTkErMlNuZUJMUDIxa3Q0ZUEyVksyMlFvdGRoTnpXQkFRWTNQ?=
 =?utf-8?B?NUQrcGtGdE9BbXliUVY5WHFBUEI5WXJFaFdVaFJUT0tiWjdEN0ptTlJkV0lu?=
 =?utf-8?B?YUVYM1JnMXNHdXNxcjAzVk01ak9NU0hHQnJZR0U5TWN1YjBBSWlFcDZ6ZlJy?=
 =?utf-8?B?ZTNkZTdpNGdWRVFweG4wOXo1UmdxT0RyY0hZTjBZMnY3RVovOWtlTlRuOGo4?=
 =?utf-8?B?cFZsWlVpR1Z5cmxIUUZlUE15YysyN3FWSFlwQzZkTVpoMDBEMkRLVzFUbEt6?=
 =?utf-8?B?RFJDdWpUbWpOVlRRNThqc3NOUms2VnZGbjZlMFpnR1R0RlZGSXNaeVlXc3lK?=
 =?utf-8?B?MitvOThCSjBONmNHMW5zMUhVZktEZHRXUDhETGJUM2hURmNZU08yRXRLdkl5?=
 =?utf-8?B?cUlFT2Q1dUhoOHoyOG1JQ1FkL0pMQXF2SHJnMzk2WDRIY0M2WGI1ZWU0Y1BW?=
 =?utf-8?B?YzZCclQyUDVPcGt1N21ZaFF4bCtOZGRWOVYwN29sZzYraDFGMDdteTZiZ3R4?=
 =?utf-8?B?eDVNSlNOTFovRElBU1VQb2ZHOXdOWStneHhuWjV5T0lZRWh1WDFmYWhwMmNP?=
 =?utf-8?B?YmExUlk1RFZGRzZOYzZMSTYrTUF3MElPZEpDWUgyYTZoaEJQSXNyVUhSV3k2?=
 =?utf-8?B?TmZ6U0dCN0hUbGxqcENDaWZYTTRvWHJLcmhYbkxDZ3lHVURmL3NteGpkeDZT?=
 =?utf-8?B?Q1ljUWtLbStFRzBQWGVsVm9RNXZaS3pjc0wrVjc5YnU4ai9SbmZoaU5NMHEr?=
 =?utf-8?Q?mTo6uhe/UDj4Xk6gYX3AzhYXDqf7yVXBwztvRsv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 108c2e8b-c526-470e-b675-08d96df9afa5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 10:09:01.1902
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IRcsMZ+U8vy+B2jwfLlQl9282gxGOwKu5OHovyjj0eXQ+zr9NWp74UJe8d3DZZB51mxfNS8TD6CaBjAUkRAAPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 24.08.2021 12:49, Anthony PERARD wrote:
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit with a remark:

> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -133,6 +133,9 @@ endif
>  # Always build obj-bin files as binary even if they come from C source. 
>  $(obj-bin-y): XEN_CFLAGS := $(filter-out -flto,$(XEN_CFLAGS))
>  
> +# To be use with $(a_flags) or $(c_flags) to produce CPP flags
> +cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))

Afaics this has nothing to do with Linux'es cpp_flags, so what we do here
is entirely up to us. If this is strictly intended to be used the another
macro, wouldn't it make sense to have

cpp_flags = $(filter-out -Wa$(comma)% -flto,$($(1)))

here and then e.g. ...

> @@ -222,13 +225,13 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
>  	$(call if_changed,obj_init_o)
>  
>  quiet_cmd_cpp_i_c = CPP     $@
> -cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -MQ $@ -o $@ $<
> +cmd_cpp_i_c = $(CPP) $(call cpp_flags,$(c_flags)) -MQ $@ -o $@ $<

... the slightly simpler / easier to read

cmd_cpp_i_c = $(CPP) $(call cpp_flags,c_flags) -MQ $@ -o $@ $<

here?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 10:17:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 10:17:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177215.322509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjmm-0000gl-6f; Thu, 02 Sep 2021 10:17:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177215.322509; Thu, 02 Sep 2021 10:17:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjmm-0000ge-3k; Thu, 02 Sep 2021 10:17:32 +0000
Received: by outflank-mailman (input) for mailman id 177215;
 Thu, 02 Sep 2021 10:17:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLjmk-0000gY-O2
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 10:17:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8d40e283-a67f-4ec0-a9ee-6d6ef7aba0df;
 Thu, 02 Sep 2021 10:17:29 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-nJlk4btgPE6_MAMJj49RPg-1;
 Thu, 02 Sep 2021 12:17:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 10:17:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 10:17:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0041.eurprd04.prod.outlook.com (2603:10a6:208:1::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Thu, 2 Sep 2021 10:17:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d40e283-a67f-4ec0-a9ee-6d6ef7aba0df
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630577848;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lwOFmKTMA/e7SRGI2xHrMSVxtHkhekdm0cD8fSrNIQA=;
	b=iIY6eQghJnCL8z6dVx5wJKVWHxOXZqxv6dZkCBGhXuku/4z0eiKOF0+4drdJbzZ1a4edY6
	c9AHdNvGMqtM+skEHpNg/LnFJsDHM9PT4lMJSXM+kHNo46sjxK1JDmpuZLsuJ4NhjqCq9W
	170jhOMSaT5J5/R//gRt/SZRYa2Sysk=
X-MC-Unique: nJlk4btgPE6_MAMJj49RPg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V7RlemlW1Qeb+inu6gcupbATM4QtP+OXD6SNIa5hSQnlaw5hlVSjSo45OYQHNuMQSqRVjhgwfRgAZVVQIqnf+sxd2dYCp/RSEVqIkw4AjDQmjpdDgvRlWfC9s7SVG80cP+wN9NmFA9OFyoR6NlHiPVCxZR91uT6dtmrUfQN6nJa4EGe6ckEYxsaprsv2i+2BeXjpCGwa3QaV0O//Dhoatki1wqefOygoqO9Lau2sB/ULxGG3Sl13YasXQQQqeKU/+I+nD6rl3leuEJVPLYyr9/KKUU++T+S23BM2fXC/SK9GV1E2Xw/HCxVva4VQhbTuQQ7t9x3wdZ547osrfPuSRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lwOFmKTMA/e7SRGI2xHrMSVxtHkhekdm0cD8fSrNIQA=;
 b=TY+aLBGKiS2Fnry1kuMHmbimQGrxqBXLfVF26Wb/ZCftu/BjkijJdqv26iYZkW69m+7bMB8IF6rWPJUFtpY1ewZPGjIsB51ZM+dd1cfHgLVcojAO99om9KI1OVZ+p620lQztP7uyAbeg1SIVYdVxw3l7kO3V58Jrk+J8vyL8HYwsHcYv1fgEueamGH4HdTLjss7sXa74CDUFyn8eoSNVvUX6AesnWmlMgoJi6288j5NKdAKI9zw/uvvjIDlUgs7wkR2e/Ja6LHjrHTSV7DZrLqoWiXt2pbPh4LdYVP0DuECVAxJNBqM0lkqiLQvqLnRl2VqNeQA4N39dreoZrmVcqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 09/51] build: use subdir-y in test/Makefile
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-10-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ea4d48a9-0aff-edf0-71cd-6d87387ebb95@suse.com>
Date: Thu, 2 Sep 2021 12:17:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210824105038.1257926-10-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0041.eurprd04.prod.outlook.com
 (2603:10a6:208:1::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 41f397d4-c271-4e20-67df-08d96dfadc85
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB244708248CB5FD6E599CD518B3CE9@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:506;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4d/4YNm4x7CiabuWhktC1I7t+CpJaWtNHjEsypw2yX68rU3ATMX1CBo4gi52lG+ZT+GDwCacthUDAW4joDOHnSHbrdsSlt2veCZK+RGSQPWwEnqOxh+MrYaeCUMQFq0q+sUMsn1tUs3Xx9wr/6t92v5KnO9AfpEr3qnK0PtoXjapxJBLCZZPHWzriZKcNbm7tHSWFHC3FgFDMbXDKMpbjfJ0T4w4tTKFjZXOd1RA6wA4xbT4P9o7hejfp/Em8lqDko7THYvJAvKtsS+1T/kM9wNVSB6eyM2CTuFMj39BHKs7Nh0XYElV41K7lKptCfeubEQnFTEzIoC62Dwf5e5yL0+kNEIAsusoZYjxtV5D9VhopW2zYZ2ubqwAopswzjjKaW3lTW5IWificx7lAEUZUEti2VL5fWYGRXhA1u2dhTsqSccq81gqF2uBAjRwl8C/1rqtZ/xehaxwiaADnNFdKE6iYJkzi5WN+eoT86zqKaOGFj6XQEWhusl9y3+5wAp/PaWuOsIiCPf5oPKAdqWYFm3oAcTFx9XlmdpdrJNUNMwpj8rdISQSkhxidMBCUUgbd2Fnvfbz5YmjJb2ygltvL5RJQUYwalIkID8OCBcUn2UMfq8n7b2z51ZQLQQQHQTlekwmxz0LQPvzswX+R2E/eNNug4xZ8Py4tv8sbyAkz9KcABccpV5KGFX8WMCdlFicyLEcR8tQTzTACczEP23kPfumDGmJe2Ivfg5Aesn8JUc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(366004)(39860400002)(396003)(346002)(6916009)(2906002)(4326008)(5660300002)(478600001)(36756003)(31696002)(31686004)(8676002)(38100700002)(54906003)(956004)(86362001)(2616005)(6486002)(8936002)(66946007)(66556008)(66476007)(316002)(16576012)(186003)(26005)(558084003)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?S0owYXhtbDFvWGJIcTMwdmZwbFpwSFJmY2xUa0U5MjNQRTkrcUZrTDhEbkor?=
 =?utf-8?B?TXh0bUxvWnZiVjFkTk9lNWRZdVM3M0xrOGFySjQwbmVqUFpLSDJkZm9oN1Ny?=
 =?utf-8?B?YmNsTHRmZmEvUDB4Q013VXAyNWVpWjRCM2tTWGdSbGt2bVF1Z3AxYkErTjNr?=
 =?utf-8?B?cjFodE9XeXJ2N2hoelU4dkE1dWxwb0cyam1vOUpEWjJFODlUaVpUaG5PYWZw?=
 =?utf-8?B?M0Fad1hMa2FibWtCbHo1UnRqQlF6aEJzSjkwbjdmaDFCVXdDOTh6a3NnRlpL?=
 =?utf-8?B?NXRrU0M5NFg3YzNJT09CWUJudWhSLyt1MWRwM1VRUXd4bnVwditRYkk5cGZ1?=
 =?utf-8?B?K1ltZHo2ajFoMVN6ajdxUWx5VGxGQngzcG9RYmpzY1RUTlBaTlJVVFRjV1ho?=
 =?utf-8?B?aTJGcnNYMUVXMTJSM0dILy84WnFQTFpSaStiU3NxdGtXWTV3NEk4bVZid1J1?=
 =?utf-8?B?YW9TeHA0N3dqTEdMTGR2c1hSQnF4L0dFVDZ6STBEQTNRNzFGVHMraGxsdERq?=
 =?utf-8?B?dVoyWlpBdjF5d3JZa2dYUzlmMXBzQk02NjdXcDhTWHZpMnRmdk1DQmxZVWpv?=
 =?utf-8?B?QnJaNVNNRzV1SkYzS0ZoQkRNWFdQdm5tWVVRSmxEYjk0ek9HMlRWK1dXbC95?=
 =?utf-8?B?cURXd1dSSnE2K1RYS1JJWTRBWFlMMy9DRi81dGd3M2pJdTA4dG5OcStsQ1px?=
 =?utf-8?B?dFp0RjkwdnpqWWF6VnlHeFY0SWlhQWZUd2RISnlJSDA3eHJoaEFvN1JRQVFy?=
 =?utf-8?B?WUMwS0tHK1J2NzkrejZzY3lnOXVuS0RsL1NjSVVMNG1scjNyclRhUExBT3Qv?=
 =?utf-8?B?M2pVMDFCblhvNDFjVzV3N1NuOVR5cXhZK25WTjJkSUFMV2ZabWVBSFZNbkN4?=
 =?utf-8?B?d2QvNTE1ZmFPMXlpbDJYem5ZaW4zYWh0QUViTmJnNXV1eDByNnVlcUhqZXlS?=
 =?utf-8?B?VkxocFVQQTM0cXJLMjFObVgrYkl1NlQvRDBMSmNnTGlhTUsrQTlmRlpraTRl?=
 =?utf-8?B?VkxweTBhWVlqMVFhMDgzcHd2M1BTNXFvdXAxb0hORTk4QlBRekpNWDl0SnhZ?=
 =?utf-8?B?K044aVBKUEtwbkxhZjltTWlsUkwzQk8vMDdIaVJWbWFEMExobmxPWTdJQUM0?=
 =?utf-8?B?WFpsVlduNUdWL1lxaEVQa2lSOUE5WnBDZ1NETXJSK20zTnN5VUpJV0I2NWpH?=
 =?utf-8?B?dzBDdG55M0tqZzN2ZWVBZ3YvNG9TalRBRERSYWZHVW9xcHZDUmdJbUt5cXRy?=
 =?utf-8?B?TkZBaW9VTnlLOFR5ZjNvb1JEU01UclRyZ2lFcG5vUllIK1E2L3NHeVgxWG5x?=
 =?utf-8?B?V1RqWTFiMDc0OXl1NFM4eVdPRG9wVC90Y2VkMVBIdjRVc0JXWG1KSXJhNWt4?=
 =?utf-8?B?SVl2eGU4d2Fxdk9uN3NaWXRKczNOS2Z0UDBDbUYxWmU3a2NBUzNzd0NNZHZN?=
 =?utf-8?B?SzBFVFJpRktuR3RDR2RpUDFYQzJGTzR0di9SNG1wQVMySityTHB5ZkZmUGFw?=
 =?utf-8?B?NXhjSzJJallLVlNPQ21oTm0vU2VJRGpiNlhCNUdzdUJYYlBxWXpScTZlMk9s?=
 =?utf-8?B?ZmlrN1pkZnFiNDhJTWIwYnFGb3hVRmV4cW5YS2pmRmIzMW5vb2pXK0tSV0Q2?=
 =?utf-8?B?dUhPdjZ3NDBoTXhibmwycmZ0THFWTjB0ZVByLzJBdVVrYmkyTVJwbTI0ZWNz?=
 =?utf-8?B?UU5xL1pBZ28yWWhacUdzVzByVlZpcWk3QUhMQVdSRlBqWjZpVk44QnVlQXZZ?=
 =?utf-8?Q?tdr1yx6IoUbBn2OzSsstbOTEHXY6KkzIAfi0Hg3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 41f397d4-c271-4e20-67df-08d96dfadc85
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 10:17:26.0007
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OBnDfa0ge7derCedYuGkDbNGhwrCcr5dXwEg9NuPKOB3nJBZiswfGdfpjkucGt8T9vl0oj1uDMGvzj0oH+f45A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

On 24.08.2021 12:49, Anthony PERARD wrote:
> This allows Makefile.clean to recurse into livepatch without help.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 10:19:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 10:19:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177222.322520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjoR-0001LF-Nr; Thu, 02 Sep 2021 10:19:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177222.322520; Thu, 02 Sep 2021 10:19:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjoR-0001L8-Kq; Thu, 02 Sep 2021 10:19:15 +0000
Received: by outflank-mailman (input) for mailman id 177222;
 Thu, 02 Sep 2021 10:19:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+khH=NY=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mLjoQ-0001L0-6E
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 10:19:14 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.61]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3817619c-0bd7-11ec-ae51-12813bfff9fa;
 Thu, 02 Sep 2021 10:19:13 +0000 (UTC)
Received: from AM5PR0301CA0029.eurprd03.prod.outlook.com
 (2603:10a6:206:14::42) by AM0PR08MB5362.eurprd08.prod.outlook.com
 (2603:10a6:208:180::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 10:19:01 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:14:cafe::80) by AM5PR0301CA0029.outlook.office365.com
 (2603:10a6:206:14::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 10:19:01 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 10:19:00 +0000
Received: ("Tessian outbound f11f34576ce3:v103");
 Thu, 02 Sep 2021 10:18:59 +0000
Received: from d07b1656b146.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D95FE19C-E716-48ED-95B2-C1F361DBC423.1; 
 Thu, 02 Sep 2021 10:18:53 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d07b1656b146.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 10:18:53 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5609.eurprd08.prod.outlook.com (2603:10a6:102:81::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 10:18:52 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4457.024; Thu, 2 Sep 2021
 10:18:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3817619c-0bd7-11ec-ae51-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tw0iUjtKkLEHPRfdJHIx0kRvdubuEIfM3/VmTTJDH0E=;
 b=pIxoocTFccO3R5idc6ai9owoztQn/16d+Jzrx3AZaG0vlt5pudj3Avkjro6+JCVJl/XQQq1ZOQzRRbraqsqCqpV6eWPVT7dTc3JckYZPEZp5Z0NioeaKSrsWM28T1g0o/JzAPY/R3TwVahdBntrbG2uGAKMYIzD3Ud2Ph7TuCUE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: dc87c5284b92ed52
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ndCVH+jAGAx+eRStRkwsQR+B/x0NyUKClrH5eqMnlKMPxGDZdbsl7yuWdVG2Kz/sHwKZlkLKvgqVV4FFcmc0I+PNdY1cEVeL5Z56vlIvOnoksuKYzkcgFufZANnfUboKtMxglh+RFff1sGNEe3AxHCGeMV73F+nikE+uyI/i91kqbe3m77L0yZN9LWUFzqbqBJNohX1KT9FMJ4jHqNipj5OwFs09hYuu/yu/idnXCxxXn5AuX2jgxF/KTowOOqPRZkkYcdFVFbdXh4oFBdlLZH+9oXNQSbzo2guT+v9HGYPlvPPR1Vy03/MrLgF9Guc7+T3kTXrRdC1/5ksNgujjKw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tw0iUjtKkLEHPRfdJHIx0kRvdubuEIfM3/VmTTJDH0E=;
 b=D9ZA2XDmD0KPb0VrJdkBm6igUQGAHhF4oqY+CeVvynWliKuuV+gY5sKVE/WimiwYzXUPyoPr2M4v8pylxElmR6DyjSy17rfpNRctp1WbrynpJH9w1ZwFhNhtPgy5TlBADlVol4nF+DhHSwNW/p2dNs46URXba8wJVPdPVJXd27t1B+voTL12yPqFCe5mTkqEZM/ptt4L/ixqHmN6AMtt4VgBOh/n9t9/okoJPXpS3/vbqqykY6bD0D8wd3moPHvnpGAgGELsGkTJjdrMOQGJfb932OJtm4xQtIURGmqQY/GaV1F90pBG7piMPZswr+dizlF0feEXI2i3ShziPt9fpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tw0iUjtKkLEHPRfdJHIx0kRvdubuEIfM3/VmTTJDH0E=;
 b=pIxoocTFccO3R5idc6ai9owoztQn/16d+Jzrx3AZaG0vlt5pudj3Avkjro6+JCVJl/XQQq1ZOQzRRbraqsqCqpV6eWPVT7dTc3JckYZPEZp5Z0NioeaKSrsWM28T1g0o/JzAPY/R3TwVahdBntrbG2uGAKMYIzD3Ud2Ph7TuCUE=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Michal Orzel
	<Michal.Orzel@arm.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Topic: [PATCH] xen/arm: Introduce pmu_access parameter
Thread-Index:
 AQHXny72/ET98Hho/USYtlrYv98fUauPIugAgAAENgCAABSIgIAAOtOAgAAK3QCAAOCVgIAAEW+AgAAWIYA=
Date: Thu, 2 Sep 2021 10:18:51 +0000
Message-ID: <8B5CFAFE-FB9E-40EE-B162-35554B71EE68@arm.com>
References: <20210901124308.31573-1-michal.orzel@arm.com>
 <599515f0-4b8a-3a71-caea-b17ec27c9123@xen.org>
 <44B48162-EB44-4295-9629-2D95A40D7232@arm.com>
 <46e36ba5-8b3d-2d43-6297-b92d2654ab72@xen.org>
 <alpine.DEB.2.21.2109011045440.17925@sstabellini-ThinkPad-T480s>
 <0e72fb05-a477-ef79-c26f-bc011393a403@xen.org>
 <5B7B9C01-FF1D-44F9-8815-A422E51916A0@arm.com>
 <a7920989-f0e3-3eb1-5317-09e6bcd67211@xen.org>
In-Reply-To: <a7920989-f0e3-3eb1-5317-09e6bcd67211@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a756ad49-82f9-45a8-9fd0-08d96dfb14fa
x-ms-traffictypediagnostic: PR3PR08MB5609:|AM0PR08MB5362:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB53629577A4D7BF78AEB4365F9DCE9@AM0PR08MB5362.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 UYW4hFDjfb3a7zWLI/JLtDJNG25czlp28CopXaYChrWMKGJnoBLyNPvG2hMbEoTi7vflQrqrvp/1dWH5+G5rbaQIb0amIA/qUCLCLlsgawtq92ODTa6tcEI2973Ivh+CNVYqTMywIh8iZGZ/P48TGSSVYX/n5ftZ82U1lIc/qENGWU9DLZRr8MVUSPX7PgQV9FkkocPLlAB3MTi9o3BwZiUAtXJTBrVio18kWZnd1naT07RmdDxKA+0+zJZViPdMTiptApCukK9Snk0h0AXht/Go3NIUrmQy9L241csQsitZVUZCGcYnbFSlFIEAHj1rPHtLmdhZANqEDJbhlUVkUGlLW8hfjLAtZtAv9z2UNwa+tHsTEjnhaGKLDKSwGtl/5NL7ss+/t5RzrA2scp55ePw7GCjrOVcChT2u6n1VT5xb+53g3yY7TzItBe/BH9T1F/U25k3y90Tk8H+qYVqvmKFusB8RPAt+A+nWJatAZM/ADuSbjGffyjJesB2L/mMjWPen4PE5I0zbsClem034toacp8Syai5NGeFnqEJEGX4ZeFmGRzZTP3MQBgjXl9caVjeYWX043eMl1vhKIfg9iGx4aiTuV2ySCI7bWsKqmvtyoNZWuC5cgWUQA6Qv52xl7PPQlqxB/Qb7nzJH5oKwk7qq16gkahNpLSTyjcwBrgh6iQ+1N2gsbTfUkldhXapcUSHr/tRk9d5xJUBQqDJ7WQUyzxR5X/CL8biZA+4rnIkkXwe2bIlBu+aqe7hWlGH1IMBjIzeWuMbAcDpAyuk/ZA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(376002)(39860400002)(396003)(136003)(2906002)(83380400001)(6486002)(5660300002)(8676002)(76116006)(91956017)(38070700005)(4326008)(26005)(6512007)(478600001)(64756008)(66556008)(66476007)(6916009)(66446008)(33656002)(66946007)(316002)(6506007)(54906003)(53546011)(2616005)(36756003)(186003)(71200400001)(38100700002)(122000001)(8936002)(86362001)(21314003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D444033A09F3D147883758DE0D76CB92@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5609
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f2252119-6265-4c78-bf74-08d96dfb0fd8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dQ2DjelHeRHZVrg7zwS+YIdP4sY2qy8UQ+J+jrq66N4WJvzjyBOhvxiV4HbKuP4qAQHhfxlC3q8/lJOItk27f1ySscJL+bz5kLPu9KD9DeOpXrFWsAjEqABmxEvAhOOwTcvN3PXt5kvSAZdsTeb/uIZVT5o4o6HruheQ5YATxMcQfsI0SGVF0LlUBY3x/aZgjM/iLuIMrZkM7g+2A76iCuH4h8qP7orsvDDYtKStD4MR1pJwEdn8qO7RteCMHH4lB0WtgqllIjBCBvzv4s4tek8sj/IoifURTq2xAXLnPmsOh2vxrLLW8X9phB3OzNmor65ZhEPXyTNkDg8wmK0UARUf4dviICqaqth273yqHnOYr3AruituQBwLM+MYQZEwZ7xHGhGg1KEj7Ok6mBI26HPDySJ0h+Nds9nb24LnKpnxbB9WelFsO/pFwf4laWmsxhY/3PK9bCOVP9mB6thnpjDJZwDIRdiR8BfRXMRRCWPgW21y19UjUm5Xn8C52BtRU01U6g68Gnhe/aiZODLMbRRNlNzCMVSEuz1FVY/68qgRyfKIuhTAdnXaUFBsEOIfRhI3EGlsalTsE07vwwpT1b2yFBzFw+w8tF6TzYjM3nTPXXTY0HIKdSfyfPmwflks5D7tZi3XdmB8nHvLplPFPEe+z42ipHEPl4nY7SpTecPr5yqQcJncB90t0CGaNRA8shHMd2mzj7iL00VPQLQ5+Emku9l7yBAXYe1XtMhwxjE=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(39860400002)(346002)(376002)(396003)(36840700001)(46966006)(6486002)(5660300002)(8676002)(33656002)(70586007)(8936002)(6512007)(82310400003)(81166007)(82740400003)(356005)(70206006)(36860700001)(47076005)(6862004)(83380400001)(4326008)(86362001)(107886003)(478600001)(36756003)(2906002)(54906003)(2616005)(336012)(6506007)(316002)(26005)(186003)(53546011)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 10:19:00.4954
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a756ad49-82f9-45a8-9fd0-08d96dfb14fa
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5362

Hi,

> On 2 Sep 2021, at 09:59, Julien Grall <julien@xen.org> wrote:
>=20
>=20
>=20
> On 02/09/2021 08:57, Bertrand Marquis wrote:
>> Hi Julien,
>=20
> Hi Bertrand,
>=20
>> If I understand it right, you want a per guest parameter to be able to a=
llow PMU accesses.
>> This would require:
>> - to save/restore MDCR on context switch
>> - introduce a new config parameter for guests (might or might not need a=
 tool change)
>> - have a xen command line parameter to have a solution to Allow PMU for =
dom0 (or maybe a DTB one)
> Yes.
>=20
>> But this would NOT include:
>> - interrupt support (only needed to get informed of overflow)
>> - provide PMU virtualization (not even sure something like that could ma=
ke much sense)
>=20
> I am guessing the following is also included here:
>=20
> - provide a PMU node in the DTB for the domain.
>=20
> Without those 3, I feel we are exposing an half backed PMU to the guest. =
But this would still be a good first step, so I would be OK if they are not=
 implemented in the first shot.
>=20
>> I am not saying that we will do that now but as I need to unblock this I=
 could evaluate the effort and see if it could be possible to do this in th=
e future.
>=20
> Well... Below the patch I wrote during my breakfast this morning. This ha=
s not been tested and miss some documentation.

Impressive but be careful not to put jam on your keyboard :-)

We are clearly not at your level of expertise and this would have taken us =
a lot more time, even if we tried without eating in parallel.

>=20
> From 690a92cffed82451dcbd8b966e8dee31c1dce5fc Mon Sep 17 00:00:00 2001
> From: Julien Grall <jgrall@amazon.com>
> Date: Thu, 2 Sep 2021 08:46:12 +0000
> Subject: [PATCH] xen/arm: Expose the PMU to the guest
>=20
> There are requests to expose the PMU (even in a hackish/non-secure way)
> to the guests. This is taking the first steps by adding a per-domain
> flag to disable the PMU traps.
>=20
> Long term, we will want to at least expose the PMU interrupts, device-tre=
e
> binding. For more use cases, we will also need to save/restore the
> PMU context.
>=20
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> ---
> tools/include/libxl.h            |  2 ++
> tools/libs/light/libxl_arm.c     |  3 +++
> tools/libs/light/libxl_types.idl |  2 ++
> tools/xl/xl_parse.c              |  3 +++
> xen/arch/arm/domain.c            | 10 ++++++++--
> xen/include/asm-arm/domain.h     |  1 +
> xen/include/public/domctl.h      |  4 ++++
> 7 files changed, 23 insertions(+), 2 deletions(-)
>=20
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d69869..d3e795a38670 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -502,6 +502,8 @@
>  */
> #define LIBXL_HAVE_X86_MSR_RELAXED 1
>=20
> +#define LIBXL_HAVE_ARM_VPMU 1
> +
> /*
>  * libxl ABI compatibility
>  *
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e0039..89865a90dd3e 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -29,6 +29,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>     uint32_t vuart_irq;
>     bool vuart_enabled =3D false;
>=20
> +    if (d_config->b_info.arch.vpmu)
> +        config->flags |=3D XEN_DOMCTL_CDF_PMU;
> +
>     /*
>      * If pl011 vuart is enabled then increment the nr_spis to allow allo=
cation
>      * of SPI VIRQ for pl011.
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_ty=
pes.idl
> index 3f9fff653a4a..daf768cba568 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,8 @@ libxl_domain_build_info =3D Struct("domain_build_info=
",[
>=20
>     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                ("vuart", libxl_vuart_type),
> +                              # XXX: Can this be common?
> +                               ("vpmu", boolean)
>                               ])),
>     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                               ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd534..6e497cc0b67e 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -2729,6 +2729,9 @@ skip_usbdev:
>         }
>     }
>=20
> +    /* XXX: This probably want to be common or #ifdef-ed */
> +    xlu_cfg_get_defbool(config, "vpmu", &b_info->arch_arm.vpmu, 0);
> +
>     if (!xlu_cfg_get_string (config, "tee", &buf, 1)) {
>         e =3D libxl_tee_type_from_string(buf, &b_info->tee);
>         if (e) {
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d46..a0e2321008ab 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -276,6 +276,9 @@ static void ctxt_switch_to(struct vcpu *n)
>      * timer. The interrupt needs to be injected into the guest. */
>     WRITE_SYSREG(n->arch.cntkctl, CNTKCTL_EL1);
>     virt_timer_restore(n);
> +
> +    /* XXX: Check the position and synchronization requirement */
> +    WRITE_SYSREG(n->arch.mdcr_el2, MDCR_EL2);
> }
>=20
> /* Update per-VCPU guest runstate shared memory area (if registered). */
> @@ -585,6 +588,9 @@ int arch_vcpu_create(struct vcpu *v)
>     v->arch.vmpidr =3D MPIDR_SMP | vcpuid_to_vaffinity(v->vcpu_id);
>=20
>     v->arch.hcr_el2 =3D get_default_hcr_flags();
> +    v->arch.mdcr_el2 =3D HDCR_TDRA|HDCR_TDOSA|HDCR_TDA;
> +    if ( !(v->domain->options & XEN_DOMCTL_CDF_PMU) )
> +        v->arch.mdcr_el2 |=3D HDCR_TPM|HDCR_TPMCR;
>=20
>     if ( (rc =3D vcpu_vgic_init(v)) !=3D 0 )
>         goto fail;
> @@ -622,8 +628,8 @@ int arch_sanitise_domain_config(struct xen_domctl_cre=
atedomain *config)
> {
>     unsigned int max_vcpus;
>=20
> -    /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=3D
> +    /* HVM and HAP must be set. IOMMU and PMU may or may not be */
> +    if ( (config->flags & ~(XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_pmu)) =
!=3D
>          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>     {
>         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d94..14e575288f77 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -166,6 +166,7 @@ struct arch_vcpu
>=20
>     /* HYP configuration */
>     register_t hcr_el2;
> +    register_t mdcr_el2;
>=20
>     uint32_t teecr, teehbr; /* ThumbEE, 32-bit guests only */
> #ifdef CONFIG_ARM_32
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842da..db9539ddd579 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -71,6 +71,10 @@ struct xen_domctl_createdomain {
> #define _XEN_DOMCTL_CDF_nested_virt   6
> #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
>=20
> +/* Should we expose the vPMU to the guest? */
> +#define _XEN_DOMCTL_CDF_pmu           6
> +#define XEN_DOMCTL_CDF_pmu            (1U<<_XEN_DOMCTL_CDF_pmu)
> +
> /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> #define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
>=20
>> In the meantime we will start maintaining an internal branch with patche=
s refused upstream as this is blocking us.
>=20
> For the future, please consider a per-domain option from the beginning. T=
his is not much extra effort (see the patch above) and would make the accep=
tance of a patch more likely.

We wanted to share something we did internally which we thought could be us=
eful for others.
We will be more careful in the future.

Cheers
Bertrand

>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 10:22:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 10:22:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177230.322537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjrc-0002tz-8c; Thu, 02 Sep 2021 10:22:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177230.322537; Thu, 02 Sep 2021 10:22:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLjrc-0002ts-5i; Thu, 02 Sep 2021 10:22:32 +0000
Received: by outflank-mailman (input) for mailman id 177230;
 Thu, 02 Sep 2021 10:22:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLjrb-0002tm-6A
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 10:22:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 581489b4-254d-40ab-baaa-f40e8832e8ed;
 Thu, 02 Sep 2021 10:22:30 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-O7RvBVukNHCZhA4CO9IjYA-1; Thu, 02 Sep 2021 12:22:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5599.eurprd04.prod.outlook.com (2603:10a6:803:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20; Thu, 2 Sep
 2021 10:22:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 10:22:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0041.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Thu, 2 Sep 2021 10:22:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 581489b4-254d-40ab-baaa-f40e8832e8ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630578149;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jBvMBrv4oxvPPQ9EqN0vykb5XJjI42SfMRO3gMpCVf0=;
	b=UkmyJ8Ao5+zN8J1NZgK3F6n/sYhSVmoFqJj+pBsHqx+YtnlGKoLJKULoJlWmtDgVyQvxek
	HWbTBupWDmGgYQoEGlbfjZq9u84iih3KeL2e3KgEg/j/UoclmqW9uZ/BIH10QDK6+BOJ76
	QYTGcin+gSSQ0RHioa9eJxFsT+dZfnk=
X-MC-Unique: O7RvBVukNHCZhA4CO9IjYA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MrBGUGX1ckyFBe9DdcIzX3NPXIBIw4JmrF8NFZ3aW2PPLu2rw557PfQTjfSUV90XGHWZl966YI4GX8t4PLzFkx2kJRJx98rIsC72kW26ioVMDH+MYnLnsgG0i0NYtEC7741+pvB/L9gf7NEm8jboOR2DsECSxliM8uGPaJB+ggTu8icqj2GY+9z7Xlebky9HjG8cEkliBe6TWwU5ESLKyRiC3w4L+soRAZk57GxWismzr2XJvJRrgd1CxBUj87WOPjZHyKg/EJ85trn1WTFLjPxBjwiBbcFH8w5BOKIXOP36Idvz8Qogg0IRras0hSu5krmurBO5YNsayifZJSQ6XA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jBvMBrv4oxvPPQ9EqN0vykb5XJjI42SfMRO3gMpCVf0=;
 b=HzF1B2dtd5VX/tZmm8JXo11T1DORvuGHI+k1j8ckqKdviuW+LmZoZH8y1F95jQ5Ck69Qh0X0ygdg+0+rhbKynMAYvV8mlKryN7v7aBGt4gTkBDlHrcYEZcMfYRFlQwN3tU62OFHELeABYyo+EM7UCDrckDBJHea2JIC3ljSofi34HABCkuffpdIDljYMcTahQF4moJhK6o2jW9abSw81KYWrmxX+6RnEaBmXbfOAaN91XUUTIRIPUWxoYpytg2IeST1OzYPkKs7FO2TpVAVNKIclaxO7FxXHl/uqo43HkeRvz+/xtwNad5L58oIebISSTG2co9XLgkPO+Fgreg1K+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 00/51] xen: Build system improvements, now with
 out-of-tree build!
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ff609b7b-0174-8161-c48b-353387dc78cd@suse.com>
Date: Thu, 2 Sep 2021 12:22:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210824105038.1257926-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0041.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 73adc729-ea34-4779-e5ee-08d96dfb8f7c
X-MS-TrafficTypeDiagnostic: VI1PR04MB5599:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB55995B5AA83D645C11C6AF26B3CE9@VI1PR04MB5599.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9uT1Wpnj60xmtKnpwLM4Y/jDGk4CYf9UpxsHsdPgHuPKelDTeGF+FetqGeDY+INlMPsB/4Q6foGCJSes3B39/doNqrY5Bn8PDjyvv6UFLCZB8dk4B/1kSfm5m1AvgBoPsiWwaxVeyPwqFHtNQE1pq4K7SJvX9bktmkB2uZa5nGtblzZaFmGX0VrB5yVLVrgfzJtQoEDVUaE+/KyzMkufw8b9G4wXOATFymmQxIOr7Yay/zgzC7eRW4nPmV92WoguyHNKXi4hNgKF4q3Gby/3aYFJGv0+9QPTaR74+7vaPL4rliNYloDIaH7Bmax4YnvqQfTXTy73WFJ3KgZUcTjRktbMhWi80/MRAaJg24DmaUJOddcrwOzt4zOWMA9VD/5BCjyo0ah+oB++2a+ANUbmPXIPEOcNTVfR0CqIAiaou/Gp8PcuJtQMcxs/kQGynTVGuyBLJGPky/XGispaoVCZ3bWznd70G3CgzUbmZ+aSpvV/WigOxRjKJXRd1GMfDvUKANhc1lyd9QLoFAuWUkakKS7BmXOEImuJgHWN5hvBBlQvbC1cjSnXAw+tGMcu9espftBUE8UUDNJejSYKIhjrLjJJadmHFqYuXTusErzZYuMC3k2EMaEZwg0GbWpzKFFFUbVIxMLqjaTFGp4qGiRpFUMt1zjr2TkIStJrwbR/X2PStwkdPSbd3MS8ppkRmXRlgAxerpyzCvWgXmNJhnh1nyBUFEQTl9XK7Wdy+dmOF7c=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(376002)(366004)(396003)(346002)(66476007)(4326008)(26005)(53546011)(66946007)(8936002)(5660300002)(316002)(186003)(16576012)(31686004)(8676002)(38100700002)(4744005)(2906002)(66556008)(36756003)(6916009)(31696002)(956004)(2616005)(86362001)(478600001)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dHkwOUlZQjVLeFZGdklFby9JQmJ1a0ZGcm15LzhVN1BkdmFLd0ZhQ2NtWGc1?=
 =?utf-8?B?Q3gwR2xlT0RkVTZRelFUTmQ2M3Viem12YkVJZ0xMckM5alRDeEtwSUQzcFhX?=
 =?utf-8?B?MDRrWVlJZ1RWRW90NnlqQS96WjUrbXNyN25rNUFnaUZiU1BvVDJXZ2lKckxB?=
 =?utf-8?B?OHNLNGtnTFFLVUZMQVozT0g3cnBwQVdodE9waDU5eXlpdDRvWWt6WTI4VHZG?=
 =?utf-8?B?Mjg0UnFhTFZySWUveEVmaURyZjVDMWM3MmkzQUFMSTdoNzhIVGlZaFFZZmtT?=
 =?utf-8?B?cE1PRVE5aUFndFhEMzNlTWF4N1FLZjRwU2JqQWZDYkJMbW9DSlBOczdNbWoz?=
 =?utf-8?B?VVpPMG5JSzBKdEZZKy9WSlUxNGJreElydlVraUc4ZGk4Q0czbit6R3ZjVUJu?=
 =?utf-8?B?UCtINUN5R2tDUWlLSWdLSmYwb3M4SWx3SVFwcHJ4MVRDNXlUMVV5Ly90UmZ0?=
 =?utf-8?B?anN5RU4rTmFOUFJmOGlXQ1pCbDYxQjNtaHFkcGU1T0ZSUXBYZ3BZdDBvTUJQ?=
 =?utf-8?B?SmZIM3BUbGpwYXVvSGNYcU5FY3dkRkkvaEUwelNlaW95bXp3WDhpV0o4bFE5?=
 =?utf-8?B?ZjJTKzg4ckM2SlhITjVscXFMUVB1VDU3Y0p1RkpRWlpvRlJQME1hSGJaNlNh?=
 =?utf-8?B?cG53eitMT3BEcUdqZUxtTHk0bzV6U3l0VTJEQ0RVMkdicVc5ZFRjS3c5VGtH?=
 =?utf-8?B?ZG9sRlJGZ3hpTjNJajJuUWNpbjBYY2hPUWN1TzZvYkVaUnJqdzBKWXRGOW9Q?=
 =?utf-8?B?REROanNReXljU1R3V1JldGZHeTJKS2dMRHFUc3h2MFRwaGk5N1dHZkJDUFkv?=
 =?utf-8?B?TzV5R2VUWllaWUh5dStLbWpINmtqV1NIVlNKaDRGOUc4TlZRaGkvNEZob25t?=
 =?utf-8?B?Q2kxWE5HSmdhR2V3ZHU5MTdGMFI4SkFPUlBkcFJQYit5N2wrT2pEMCtGWGRV?=
 =?utf-8?B?cjZNUEVLV3FQd3hMRGIzbU9vdFd5VHV2M0R4YkpUaDhtM0I3M3pMc1cwVDJj?=
 =?utf-8?B?UlRDUkdFRTcvU1FMUnkzYlhpZmd2cGpkYVVLZTYvQlBjSEZwZEpTRlZjaVRa?=
 =?utf-8?B?MDdRemhhLzVobHpoTElVeUVpRzVCOTNRcGdkYzJjaU91N2s4aXliYll5MXJs?=
 =?utf-8?B?MWpRcnFPQ1pvMTd3YUxNYy8zVXJGblYreGxwUllUdjlsbU1hUzN3a2I2akVk?=
 =?utf-8?B?dkZIUFFRdVJISVVxajEwcHhRbzFheDNrZU5wbGxjRnZOaDZrZlhDQXBXOFlM?=
 =?utf-8?B?amRXNGdmUE5EUm1yL0Jad0R1THJZbkhLbFVGY1RMTjlpWkR0ZFVYVEZlckpZ?=
 =?utf-8?B?S0JCVHBBNmlyMllDMTRkYm5xYU5LS2tRdVR6MmVQTkVBODdKMXhKTlNmdVdG?=
 =?utf-8?B?QnphQ0pkTjVXZUNTMjArL1ZMR3p2MGhWZnRGRkU3YlVVdjVBN1JmWXN1aWha?=
 =?utf-8?B?TTFCRUZFU3hqTDNtb3I2cFlLc1dsWit5RWRFclEvUU1XMm8xTDJ1SkZLVGZk?=
 =?utf-8?B?TG8vNVQyNmxNTU12V2NtQy9UQmhCOTF3NUpQYTE3UVhuWkNwLzR1T2swMTMr?=
 =?utf-8?B?dXdvUkxzZU9FZUQ1RFNDUFQ3OURsZEN0b3VQK0NSeVo4ckd6SlE3YmREMG5j?=
 =?utf-8?B?R3U1a1JPQTB3eG5rVmNrdXpJcm40a3pCSHZtY3RsSGN3WW8vdEF0Z0JNckJ6?=
 =?utf-8?B?YW5vYmRBcTdDdlUrNENHTWJWYUROcitVNDFhYlF6R2hFYitVbWhiOXVDK1pO?=
 =?utf-8?Q?Ji2qhSSfKyj0Yn0A+i7LdHtnFFmRdoqanacr7yQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73adc729-ea34-4779-e5ee-08d96dfb8f7c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 10:22:26.4659
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yBB8pcU8XU7U+BXmep6BjcTMGJ33aNM/Hbavpa+rWk0/6gHXuliQN3Hj6AI2jnSrIiqteR6mS86BfMQeYlSoDg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5599

(trimming Cc list)

On 24.08.2021 12:49, Anthony PERARD wrote:
> Anthony PERARD (51):
>   build: introduce cpp_flags macro
>   build: use if_changed on built_in.o
>   build: use if_changed_rules with %.o:%.c targets
>   build: factorise generation of the linker scripts
>   x86/mm: avoid building multiple .o from a single .c file
>   build,include: rework compat-build-source.py
>   build,include: rework compat-build-header.py
>   build: fix clean targets when subdir-y is used
>   build: use subdir-y in test/Makefile
>   build,arm: move LDFLAGS change to arch.mk

Having gone through to here while skipping patch 5 for the moment, I
wonder in how far there are dependencies among the above (and, looking
forward, also for others further down on earlier ones). I don't think
it can reasonably be expected for the entire series to go in all at
the same time, so I'd consider it useful to put in whatever is ready
and independent of earlier changes perhaps still under discussion.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 11:21:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 11:21:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177236.322549 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLkml-0000yg-Mj; Thu, 02 Sep 2021 11:21:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177236.322549; Thu, 02 Sep 2021 11:21:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLkml-0000yZ-IL; Thu, 02 Sep 2021 11:21:35 +0000
Received: by outflank-mailman (input) for mailman id 177236;
 Thu, 02 Sep 2021 11:21:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEJZ=NY=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mLkmk-0000yT-RT
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 11:21:34 +0000
Received: from mail-pg1-x533.google.com (unknown [2607:f8b0:4864:20::533])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5cf3ac59-f97e-4c73-b1a8-67856855a759;
 Thu, 02 Sep 2021 11:21:33 +0000 (UTC)
Received: by mail-pg1-x533.google.com with SMTP id w8so1559841pgf.5
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 04:21:33 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 j6sm2394666pgq.0.2021.09.02.04.21.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 Sep 2021 04:21:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cf3ac59-f97e-4c73-b1a8-67856855a759
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=uvXbklsYzxIxtTfrV32ffa9Nuwizcf2v49dwd36P0II=;
        b=l3pHshNaH16SxCRdn9z7Xh25o4zr254bVE4gj2gcLe03eGaxqkakkGTJFxJqfTpw1V
         mgl7jhuQxVjIwW/UeX7ktsAfn4KtTM1QiXhueAKmpi1umldFhOhm/opbthWUVENXfmXz
         sByuEIse1tA9cQPTkGsfIdaE+UkFVQTBoS0K+F+XP4S1/PfM9AjDF1PXXeVM/2D0/+Wg
         GFVcsUwXqGqAsvdSyylORw5dpuBcJnKleE7Htyj6/Iy7J3TtLZUSPSx2JgomTUK9XCbK
         tkwtRAK0M+aqYoU4ijzQ2mRPYIgBubcVFek78cQ2L31j1R5JU4P5nO/87EvlyqPdofPx
         bNug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=uvXbklsYzxIxtTfrV32ffa9Nuwizcf2v49dwd36P0II=;
        b=VSLSz44j9gbLP9LfnFDgvpVj01pIiVl1f+nNM4SyaiuEflwGru/9w/nrI6P4y0xmgd
         ymAc0AY5rePz94CB/emFygLOr0ANhLHXJ4Q4F/0AXwWloZYvQWXl2N4Z5ozr/mPoUEUJ
         npYOLpmwj71tx4iFGE6MX01W9e+O8hXo6uE8+OHh4LcNJAs+T98YUMh2zU1nItlqkzKG
         NOU4xKQU5+m7oRrjIkF+kQE+110wjip1eMihJDEwObRX93qyMm0EldKEgtTvCiMuj3Pa
         wt1c1sayFiYwfyP3JxZXUrmb5H6SAOFWiYwJ2p9wcCjDfCwoQA3Ds6pntZO6/6PG/grP
         axxg==
X-Gm-Message-State: AOAM5305KrCOzkHNogix1pGZ60xRKpMjinfhCfu8M5mUrhExApcKDuJw
	G2JWPPy9gtayxvfB6DvPSVo=
X-Google-Smtp-Source: ABdhPJxldFu1U8M2V5yK1yTHMDs9K9anELSSV02bNY0qtLdwF/4+Etf8R4JVmeL3O1gUAcLcNnmdKQ==
X-Received: by 2002:a63:5b08:: with SMTP id p8mr2815838pgb.28.1630581692705;
        Thu, 02 Sep 2021 04:21:32 -0700 (PDT)
Subject: Re: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
To: Christoph Hellwig <hch@lst.de>, Michael Kelley <mikelley@microsoft.com>
Cc: KY Srinivasan <kys@microsoft.com>, Haiyang Zhang
 <haiyangz@microsoft.com>, Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
 "will@kernel.org" <will@kernel.org>, "tglx@linutronix.de"
 <tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>,
 "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
 "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
 <pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
 <hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
 <aneesh.kumar@linux.ibm.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "linux-arm-kernel@lists.infradead.org"
 <linux-arm-kernel@lists.infradead.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "rientjes@google.com" <rientjes@google.com>,
 "ardb@kernel.org" <ardb@kernel.org>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210830120036.GA22005@lst.de>
 <MWHPR21MB15933503E7C324167CB4132CD7CC9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <20210902075939.GB14986@lst.de>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <dc124c3d-a316-d36e-3ae4-21674280f55d@gmail.com>
Date: Thu, 2 Sep 2021 19:21:18 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902075939.GB14986@lst.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 9/2/2021 3:59 PM, Christoph Hellwig wrote:
> On Tue, Aug 31, 2021 at 05:16:19PM +0000, Michael Kelley wrote:
>> As a quick overview, I think there are four places where the
>> shared_gpa_boundary must be applied to adjust the guest physical
>> address that is used.  Each requires mapping a corresponding
>> virtual address range.  Here are the four places:
>>
>> 1)  The so-called "monitor pages" that are a core communication
>> mechanism between the guest and Hyper-V.  These are two single
>> pages, and the mapping is handled by calling memremap() for
>> each of the two pages.  See Patch 7 of Tianyu's series.
> 
> Ah, interesting.
> 
>> 3)  The network driver send and receive buffers.  vmap_phys_range()
>> should work here.
> 
> Actually it won't.  The problem with these buffers is that they are
> physically non-contiguous allocations.  We really have two sensible
> options:
> 
>   1) use vmap_pfn as in the current series.  But in that case I think
>      we should get rid of the other mapping created by vmalloc.  I
>      though a bit about finding a way to apply the offset in vmalloc
>      itself, but I think it would be too invasive to the normal fast
>      path.  So the other sub-option would be to allocate the pages
>      manually (maybe even using high order allocations to reduce TLB
>      pressure) and then remap them

Agree. In such case, the map for memory below shared_gpa_boundary is not 
necessary. allocate_pages() is limited by MAX_ORDER and needs to be 
called repeatedly to get enough memory.

>   2) do away with the contiguous kernel mapping entirely.  This means
>      the simple memcpy calls become loops over kmap_local_pfn.  As
>      I just found out for the send side that would be pretty easy,
>      but the receive side would be more work.  We'd also need to check
>      the performance implications.

kmap_local_pfn() requires pfn with backing struct page and this doesn't 
work pfn above shared_gpa_boundary.
> 
>> 4) The swiotlb memory used for bounce buffers.  vmap_phys_range()
>> should work here as well.
> 
> Or memremap if it works for 1.

Now use vmap_pfn() and the hv map function is reused in the netvsc driver.

> 
>> Case #2 above does unusual mapping.  The ring buffer consists of a ring
>> buffer header page, followed by one or more pages that are the actual
>> ring buffer.  The pages making up the actual ring buffer are mapped
>> twice in succession.  For example, if the ring buffer has 4 pages
>> (one header page and three ring buffer pages), the contiguous
>> virtual mapping must cover these seven pages:  0, 1, 2, 3, 1, 2, 3.
>> The duplicate contiguous mapping allows the code that is reading
>> or writing the actual ring buffer to not be concerned about wrap-around
>> because writing off the end of the ring buffer is automatically
>> wrapped-around by the mapping.  The amount of data read or
>> written in one batch never exceeds the size of the ring buffer, and
>> after a batch is read or written, the read or write indices are adjusted
>> to put them back into the range of the first mapping of the actual
>> ring buffer pages.  So there's method to the madness, and the
>> technique works pretty well.  But this kind of mapping is not
>> amenable to using vmap_phys_range().
> 
> Hmm.  Can you point me to where this is mapped?  Especially for the
> classic non-isolated case where no vmap/vmalloc mapping is involved
> at all?
> 

This is done via vmap() in the hv_ringbuffer_init()

182/* Initialize the ring buffer. */
183int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
184                       struct page *pages, u32 page_cnt, u32 
max_pkt_size)
185{
186        int i;
187        struct page **pages_wraparound;
188
189        BUILD_BUG_ON((sizeof(struct hv_ring_buffer) != PAGE_SIZE));
190
191        /*
192         * First page holds struct hv_ring_buffer, do wraparound 
mapping for
193         * the rest.
194         */
195        pages_wraparound = kcalloc(page_cnt * 2 - 1, sizeof(struct 
page *),
196                                   GFP_KERNEL);
197        if (!pages_wraparound)
198                return -ENOMEM;
199
/* prepare to wrap page array */
200        pages_wraparound[0] = pages;
201        for (i = 0; i < 2 * (page_cnt - 1); i++)
202                pages_wraparound[i + 1] = &pages[i % (page_cnt - 1) + 1];
203
/* map */
204        ring_info->ring_buffer = (struct hv_ring_buffer *)
205                vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, 
PAGE_KERNEL);
206
207        kfree(pages_wraparound);
208
209
210        if (!ring_info->ring_buffer)
211                return -ENOMEM;
212
213        ring_info->ring_buffer->read_index =
214                ring_info->ring_buffer->write_index = 0;




From xen-devel-bounces@lists.xenproject.org Thu Sep 02 11:42:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 11:42:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177244.322560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLl6o-0003f6-GB; Thu, 02 Sep 2021 11:42:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177244.322560; Thu, 02 Sep 2021 11:42:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLl6o-0003ez-Cy; Thu, 02 Sep 2021 11:42:18 +0000
Received: by outflank-mailman (input) for mailman id 177244;
 Thu, 02 Sep 2021 11:42:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLl6n-0003eo-0v
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 11:42:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 521b01a8-d630-469f-aa7d-b44328f3c1b3;
 Thu, 02 Sep 2021 11:42:16 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-HyzUUVwYNDeDBuR0TCYHUQ-1; Thu, 02 Sep 2021 13:42:13 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 11:42:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 11:42:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0014.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Thu, 2 Sep 2021 11:42:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 521b01a8-d630-469f-aa7d-b44328f3c1b3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630582935;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xXFVZSaYK4b8AaJ5+/aIrMW00hZGanLVCMXLN49C26c=;
	b=HWerCo+7Od7FvhkYA/FmgjjAN+LldpW923YIMwfkRXFSpfrV1uur+t5q7WTZBE7wh2ftqk
	ds2tmGoLbfYWX5oX0KnWocFK24lYNP9R+S9DYiPAkcr98kusmPvYzjKxkhtq2NkcqIoLw3
	v37SmVZjDufBzT/ctadTZ/GVPWtQe2A=
X-MC-Unique: HyzUUVwYNDeDBuR0TCYHUQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=edCBtRt22ezlv7rHVOgEMFSMVN86J+T7rDIgTPuWyB/1evYcATdlzTB+jNmy874LwZ8DgOYw1nPYb4iF5byKwdNjQosP+vnfk3MOHKXODRayJtUnIU/jCdnM/t/hZ5a3UeZZB6GuLVpGJKzetaXwownOfcWyt35Bgf6m/nnifKZ/fqwM5X33AhK7HdPS15vjIMLZYxUwSGZuMRo1PzClKviPd7KlRJo4cohVwkafeLwSZNp0aCLHjZXlh4rbLBAiNIH8Qg3sFjb0Dp0CMuS2TirbUzTR238W10lbPBCXs7bljh6YdC/KtHRn+kZzkQIMIwsyQnig9sRu3ZRdcnLRVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xXFVZSaYK4b8AaJ5+/aIrMW00hZGanLVCMXLN49C26c=;
 b=jheRC3phtRpE6pRTliRKpNp2eiLqNm+qEh/9spIiBXkVfdh5oyawuFdOf/y5aPdKzFORt7zzlqL73zoF/aET9nZ2RusvotCakhJFgTYUAJbuU8Mn+fwW0I9kC9AwErNnzztaaxvebrX7beKuxvYI1B1sZqqwarCNy8opF+0dCNL81XMaYazkNuaLtTIEpNNm7LHLrBVZkbU3Gi/PHpUc1cCatdoLzOvD0+KTt8cuEOxsvep+OiqTGLpOIf+2UtEvOBZWwmpDbcgKJNCv7gvVTn1ghxr33GOzMFcgIosKZWaFzOmRZRB9117vTjqV/elGb0vI8WI3ml+yvOiWYz4C/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 0/6] x86/PVH: Dom0 building adjustments
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Message-ID: <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>
Date: Thu, 2 Sep 2021 13:42:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0014.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2090ac4f-4ccd-45ed-db01-08d96e06b419
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024EE810965A7B8E470F59DB3CE9@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z5dG3a6giBK5ITZ3af/pUPNv5609L+ZerUQVpIgDI0UHTHjHiKz/ZMjKglUvcHfRwT2XAke/i9yroruz4YC6QXHIIdzizYGCBdyFrgFKkFCIb9T2PI09dXpgv1pjl4uCQOBD8ZHWGC7XjEhWy61tjBIKTQU2UviOA5AUJpcN05cFoh1LZai+xu77J2H8EFnzRZZPyy/OFGFOJ/Hlfn/I85NpqibHc+3Bl9U/UmtB8hAfhMxOYraN4xGFX30KgS7+4Pkw7kkR0wUS7tRvKDTVXIYOFTzsShebp/hwfIjAcOMZR9Ge15X2vO/NWDgakGj4EiQkRymPgkJn0Ta/IlRduuntHi2VuK817MZ1KZfv87visFjqMtv06rSPT74Oui0FikQzCmWOh8bSqfWoa3WUg+AIv6BQQPRekVhJ2vWH6D8xdkMnYRtpPI+zi5IEUqILaLpPa7Q+WGrLwjqM0T1ASFUrkPS43JYZZgJhbGe6zh3SjWSH6veN5ReNWx7fxmYp+Hnee3sqnhIY7xjaK32Fg3zZ5u1GvwKqFTnWkeWCGbvgyVclKNQFl2sZ2YlhbHEbgAVtEbcODlruXSdjhWXTIBWDB+vQBjyFKFt21NMLkiN+84eh4f+Ed3myB+ZmQ1vDyn6DcJMe5m54KW3JStVaoQpr8IkmVe4jguS17HK/PnHfI+ybud0ukjMEajP/w0QvsYewpFEIpJf4f/EwZvXae9grwOAamEY7oAhN+Hxfi2Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(346002)(396003)(39860400002)(366004)(53546011)(8936002)(38100700002)(31696002)(2616005)(26005)(16576012)(2906002)(5660300002)(6916009)(956004)(36756003)(186003)(54906003)(4326008)(316002)(8676002)(6486002)(83380400001)(66556008)(478600001)(31686004)(86362001)(66476007)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDB1azRXZkJUQm53dXRBME9RdWk4SzhiNUlDUmoxdjAvR09ObTQwZDJsbTNZ?=
 =?utf-8?B?VFRaOFQ0VHBmcGlWdzNSRHROVXZDSW03bE90UkNYdHVMRTlIb294M1gyMEo1?=
 =?utf-8?B?UjcrRksyQWZWSVpYcjhzemQ4NXhGcG4yL0dTdEU4UytVK05RTTFCU1krL1N1?=
 =?utf-8?B?QWNOaGErL1ZSMTY3ek9EMjVKa0JoOUI1ZHFMaVI2QlkzbEJvdDREak5vOEc0?=
 =?utf-8?B?TFRZRnNNZHQ0SWpqWGVMRkhkaWxRVm14VzZ5NW1obHlRblZJN2J4QmptS2J2?=
 =?utf-8?B?RkhGdnZndGZpUmdLeHduZmhGWWROUmcvWWVJWlRlbkV6QnUyWmFDUXVzZlZj?=
 =?utf-8?B?b0tscjhveWFmWFc0R243K0RXOW1CaGJPK09TYXpobDF0QUdNSy9ZUzZZRDNF?=
 =?utf-8?B?MXJrTHc1UEZ1YXR2azhBelhPa0s4T0FqdmVrTEptdXMzTVRtL01LMnZuNzBi?=
 =?utf-8?B?cnoyWHhyM1VBOHluOW1ES1ZBdjFHcmZ2Tjc3d0FGdElFL1krcFJyUW9yWnZ4?=
 =?utf-8?B?QXIyaVpOL0JxNU1OaUc4SGl4TGY2UDVXRmlJcElDT0xnTEVpaWdqbHBnRTBm?=
 =?utf-8?B?c1pXSUovWWlxeVUyZ3RBTXppS3YvT2JuK1g1THVVd21XeXA1aHVDVGxXS1lB?=
 =?utf-8?B?MngwQU55YnFDdkpBTWlBdzlwVDNKVnZBdmVtaVpYQXpEcEhKcVd3MDRyRm10?=
 =?utf-8?B?b3k3OXQ4M2V3N0JEUWF6QUZNWEFNeUZoU0UxV2RyaEwvaWllWjcraW95ZzY5?=
 =?utf-8?B?WW01SkVtWXIvc0pQZFNNMTNIZXFpOHFNaXZaSE9yaEF5ZmcyTWVJNURGNlI5?=
 =?utf-8?B?R3c0RVNlcGE2ZklabW81Zy9INU1yMWp0bEQxUUFKYVYyWnNqUXMvNDdzTEVz?=
 =?utf-8?B?WUxtWWYvcEZNSllndVUxeEhhNHpROUpqanZnUDV5Y1lrT1Y3OHM1RzFPTjEz?=
 =?utf-8?B?RkRUTnFBNzRDcmdrTVdPbld5ZXFnQW9qbFdjZHpicFZRaXkrL3hFdDlNbXc0?=
 =?utf-8?B?NFFlN0VhR3hvVExuUThIWGhRTFlQNkJMS1BjL2Ixb2E5TkQxMkd5eHJPOER2?=
 =?utf-8?B?bUlSTWgxeXhhakovdXVFd0ZndmhFY0syQVZrTU9Xak5CU2dUMlpuLytjK0l2?=
 =?utf-8?B?TlJtVHYyT3lOeTdyM2k2VUU1MXVhS0xqb2FxT3pLY1NCQ1Z0dUpmT2xkQ05L?=
 =?utf-8?B?RkhTbjlJYURQdHB4UFRwSFd2b2VCNUdXK0l5YXVYWlprdlVaNEl5dGRySHFS?=
 =?utf-8?B?dEVYL0tYT25WL3ZGRkF0QWJvUjRLWEdJd0lZSS9hRmFna3c0eHJ4bDBRMWNa?=
 =?utf-8?B?cW5xVFo4bEdvK3FveTNScEdMMTVaL1d6WkgyemZRcjFCeWE5N3NPRWxTMWsz?=
 =?utf-8?B?SlBHSTN3UnV6VmZycG1rOTFWdG9mSDFOWmxVZWUxNFBLaGVlU0RWOWVhcGlq?=
 =?utf-8?B?RGtlQWJEKzNBQU0zQjluRHRtOEQ0eE1UU210bXBSU0JWWEE5M2NQUXVZSU1h?=
 =?utf-8?B?azgwYjZIVkxNUVZHVitXUzh6QU5XSDZzajMrVHhHY29QQkxqb0pHK2JQS3BD?=
 =?utf-8?B?L2pBOEF2VTViNnpTL0dXeXVVenB4dTVIZCtlK3dkQWVWQm1jdGtzNU1KUEwy?=
 =?utf-8?B?N0g4aU01QWU1elBMUUpHV0k3WGVISE9CL3JiekNrL0NXZGNhVy9sYTFIbzJB?=
 =?utf-8?B?YW4zOHVsUUJHUEppQVBnNS9OYnM1eVVqcFZvY1M0a25SYnpMVDZEdmZZRTls?=
 =?utf-8?Q?syqoUeb8zjXOcbZ4JcGbtjkunB81RnXRzExJlD6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2090ac4f-4ccd-45ed-db01-08d96e06b419
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 11:42:12.1327
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XJajRYW0bdEo0ZXOhTGPN7a/VwcgKJToY2qQFBcWnxwLrnv+OQ+gbFxQm+SMNgw5JJlzSO0lzhF1vkoz2Ia6zA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 02.09.2021 10:30, Jan Beulich wrote:
> The code building PVH Dom0 made use of sequences of P2M changes
> which are disallowed as of XSA-378. First of all population of the
> first Mb of memory needs to be redone. Then, largely as a
> workaround, checking introduced by XSA-378 needs to be slightly
> relaxed.
> 
> Note that with these adjustments I get Dom0 to start booting on my
> development system, but the Dom0 kernel then gets stuck. Since it
> was the first time for me to try PVH Dom0 in this context (see
> below for why I was hesitant), I cannot tell yet whether this is
> due further fallout from the XSA, or some further unrelated
> problem. Dom0's BSP makes it all the way through to entering the
> idle loop while all APs are still in VPF_down.

This last part of the mystery is now solved: By cloning from my PV
.config, I've inherited the X86_X2APIC=n that I have there. Yet
ACPI's MADT gets populated with only x2APIC entries when building
Dom0, which such a kernel would mostly ignore (except for logging).
IOW in a way this was indeed a missing select, except that what's
needed wouldn't quite work yet:

--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -83,6 +83,6 @@ config XEN_PVH
 	bool "Xen PVH guest support"
 	depends on XEN && XEN_PVHVM && ACPI
 	select PVH
-	def_bool n
+	select X86_X2APIC if XEN_DOM0
 	help
 	  Support for running as a Xen PVH guest.

This is because, as mentioned, XEN_DOM0 gets turned off when XEN_PV
is off. Whereas x2APIC isn't strictly needed for DomU afaics
(MADT gets populated with LAPIC entries only), so the "select" also
shouldn't be unconditional.

While likely no-one will really care, I'd like to note that this
effectively means a 32-bit Linux PVH Dom0 would be impossible, as
X86_X2APIC depends on X86_64. This may want reflecting in the
eventual adjustment to the XEN_DOM0 dependencies.

Btw, I've meanwhile also checked timers: They're active and get
updated while Dom0 is in this funny idle state.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 11:48:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 11:48:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177252.322577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlCx-0004MG-8Y; Thu, 02 Sep 2021 11:48:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177252.322577; Thu, 02 Sep 2021 11:48:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlCx-0004M9-5N; Thu, 02 Sep 2021 11:48:39 +0000
Received: by outflank-mailman (input) for mailman id 177252;
 Thu, 02 Sep 2021 11:48:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLlCv-0004M3-Hu
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 11:48:37 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLlCt-0003Zn-1W; Thu, 02 Sep 2021 11:48:35 +0000
Received: from 54-240-197-224.amazon.com ([54.240.197.224]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLlCs-0002RI-RT; Thu, 02 Sep 2021 11:48:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=L0tCeXQQ0xIWkKx0h3OLF+sIBF+AtFWb5ivng8j3Ihs=; b=u1syOfYhFbWNxBbrx/kqAZuDlZ
	svczd0Uk+UKljZ/aD9daeMtL98EyjG4z4+hpYojPGi32ObgHlZ80bqXZ+Pw7qt4YvDCqRI3jJxLFP
	fqO9Oh92i+O+r8TXpdPPQFTpXZm0+WfWlMm+XQ8c/TRxVJxNqc4h30fOjFc0JAzwdhRQ=;
Subject: Re: vcpu_show_execution_state() difference between Arm and x86
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
 <8f295eeb-e8f2-a115-3f83-60922d0fe5f9@xen.org>
 <00457189-6ab5-24e4-b2ec-1050ebec67eb@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <ed59ccdc-5ac0-3b9a-eb4c-33499a316a34@xen.org>
Date: Thu, 2 Sep 2021 12:48:32 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <00457189-6ab5-24e4-b2ec-1050ebec67eb@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 02/09/2021 07:45, Jan Beulich wrote:
> On 01.09.2021 20:11, Julien Grall wrote:
>> On 01/09/2021 14:39, Jan Beulich wrote:
>>> back in 2016 Andrew added code to x86'es variant to avoid interleaving
>>> of output. The same issue ought to exist on Arm.
>>
>> Agree. I guess we got away so far because it is pretty rare to have two
>> CPUs running at the same.
> 
> I guess you've meant "crashing", not "running"?

Yes.

> 
>>> The lock acquired,
>>> or more importantly the turning off of IRQs while doing so, is now
>>> getting in the way of having PVH Dom0's state dumped the 2nd time.
>>
>> I am not quite too sure to understand the problem with PVH dom0. Do you
>> have a pointer to the issue?
> 
> Locking in both cases: For registers it was VMX'es vmx_vmcs_enter()
> acquiring a (non-IRQ-safe) lock, and for the stack it is the P2M lock.
> Neither can / should sensibly be made IRQ-safe.
> 
>>> For
>>> register state I did find a sufficiently simple (yet not pretty)
>>> workaround. For the stack, where I can't reasonably avoid using p2m
>>> functions, this is going to be more difficult. >
>>> Since I expect Arm to want to also have interleave protection at some
>>> point, and since Arm also acquires the p2m lock while accessing Dom0's
>>> stacks, I wonder whether anyone has any clever idea on how to avoid
>>> the (valid) triggering of check_lock()'s assertion without intrusive
>>> changes. (As to intrusive changes - acquiring the p2m lock up front in
>>> recursive mode, plus silencing check_lock() for nested acquires of a
>>> lock that's already being held by a CPU was my initial idea.)
>>
>> At least one Arm, the P2M lock is a rwlock which is not yet recursive.
> 
> Right; the same is tru on x86. Hence the expected intrusiveness.
> 
>> But then it feels to me that this solution is only going to cause us
>> more trouble in the future.
> 
> Same here - we'd need to be very careful not only when making such
> a change, but also going forward. Hence my desire to come up with a
> better approach.
> 
>> I looked at the original commit to find out the reason to use the
>> console lock. AFAICT, this was to allow console_force_unlock() to work
>> properly. But it is not entirely clear why we couldn't get a new lock
>> (with IRQ enabled) that could be forced unlocked in that function.
>>
>> Can either you or Andrew clarify it?
> 
> AIUI any new lock would need to be IRQ-safe as well, as the lock
> would be on paths taken to output stuff when the system crashed.

Hmmm... Just to confirm, what you are saying is some of the callers of 
vcpu_show_execution_state() & co may do it with IRQ-disabled. Is that 
correct?

I have tried to play with it on Arm but then I realized that 
check_lock() is not properly working on Arm because we don't call 
spin_debug_enable() at boot. :/ So it would make sense why we never saw 
any issue there...

> Hence it would be pointless to introduce yet another lock when the
> one we have is already good enough. But I may be missing aspects,
> in which case I'd have to defer to Andrew.
> 
>> The other solution I can think off is buffering the output for
>> show_registers and only print it once at the end. The downside is we may
>> not get any output if there is an issue in the middle of the dump.
> 
> Indeed; I'd prefer to avoid that, the more that it may be hard to
> predict how much output there's going to be.

And it is not going to work as we couldn't grab the P2M lock with IRQ 
disabled.

On Arm, the only problem is going to be the P2M lock for dump the guest 
trace. A possible controversial approach for Arm is to just not dump the 
guest stack or move it outside of the new lock and dump when IRQ is 
enabled (I am not aware of any places where the guest stack will be 
dumped and we have IRQ disabled).

Bertrand, Stefano? Do you use the guest stack in the dump? If so, what 
are the cases?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 11:54:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 11:54:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177258.322588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlIu-0005uG-UC; Thu, 02 Sep 2021 11:54:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177258.322588; Thu, 02 Sep 2021 11:54:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlIu-0005u9-Qs; Thu, 02 Sep 2021 11:54:48 +0000
Received: by outflank-mailman (input) for mailman id 177258;
 Thu, 02 Sep 2021 11:54:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLlIs-0005tz-S3; Thu, 02 Sep 2021 11:54:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLlIs-0003fq-LK; Thu, 02 Sep 2021 11:54:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLlIs-0003Tz-AZ; Thu, 02 Sep 2021 11:54:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLlIs-0003J4-A1; Thu, 02 Sep 2021 11:54:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LyYxfqqQ2gxMWPWCVQyIZPsnfImSwNvYkmlDd6f+LbY=; b=pZ08BKNDcY5F1b2k8nlXEKnA/z
	TJ3YbSfqVbukOJSZS4IcR7I4oQ/hO8YnWNtr0DyA2MmV6ZRNhQ05Gv6jPfwemcDAeiPuuzPscoebm
	7dIMnBk7C9S5gAHz1qftRi5qhQeRNpFxCaEJP4ORH49PF/8WhcVST6nTSHwZjOdJpuvc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164758-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164758: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 11:54:46 +0000

flight 164758 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164758/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   12 attempts
Testing same since   164740  2021-09-02 02:41:13 Z    0 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 11:55:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 11:55:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177266.322602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlJy-0006XS-EE; Thu, 02 Sep 2021 11:55:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177266.322602; Thu, 02 Sep 2021 11:55:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLlJy-0006XL-Af; Thu, 02 Sep 2021 11:55:54 +0000
Received: by outflank-mailman (input) for mailman id 177266;
 Thu, 02 Sep 2021 11:55:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLlJx-0006XF-4F
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 11:55:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b8275c86-0be4-11ec-ae59-12813bfff9fa;
 Thu, 02 Sep 2021 11:55:51 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2055.outbound.protection.outlook.com [104.47.4.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-RYp2SUa4Ou-VaJC3qdMmzA-1; Thu, 02 Sep 2021 13:55:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2957.eurprd04.prod.outlook.com (2603:10a6:802:4::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Thu, 2 Sep
 2021 11:55:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 11:55:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0175.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 11:55:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8275c86-0be4-11ec-ae59-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630583751;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=418HGwJdiDk5kaVjZbCAcJQlFgBBx59CLraGTj8kY8c=;
	b=O4o1b04w8Scij33BkNOESN6cWKf9fTcu6WIVZer4PBC9ypjhbGxONSvEctwhztHVsvT8Jf
	PEMxT0ECM/Ib9aCr1gsV62T8xVlZNv53AJj5Fakbq5mR0OBICXnZZsWvcExv4mgtjqWmL8
	TOtvfKS2f3Xpc/Rke+7ZeBPMAapJk/8=
X-MC-Unique: RYp2SUa4Ou-VaJC3qdMmzA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aFGRxR7iaqRPWBLUmJ5rm3tv8OOSUL5kCbziQHwnDLcgFFsOhRIus7T2K9jfZzdDbbNPk8BLE7uBXcCbcOT5v2Fd3yvpF4w6EkGEL31Fke5AqYOd3YarUOlHcTE5v049AOK9cqLVGhVOOZzGYFknvwmbFTJyg618n6SV+gtjD3aXhDz0Tm3/pbDu+zL88RKmyxF3WOUubxnMaIrm048E2bZCR7ugCg4PgjlmnREnctsjpVgdFPQr3x1HFRc09slFS35QjbDro+hjaJQkvYfxuLP88Jm1oXo66AvM1qxohC55Fo8oz/4a0yiKGvO3MtlWml5i79Wi0+sWijqgabuysQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=418HGwJdiDk5kaVjZbCAcJQlFgBBx59CLraGTj8kY8c=;
 b=RkTumLLJ/5Gwjhl6PpE3OZL2+P1ex7wrU+hsi/Kle2d6nNw//5sSwGvr+L8UUjmTNRToeoa/D5BrfdKwwWnORzimetg5Lmhx7+z97+/0iv5UxLzTY62W+oRr0bhZ3ed06vCgu+cWwhuNr0VQNhRdX0Y+yzYkk87lfnahkkd/42hs65TSHVQjM4/fFaHm1al6oounc1nLi/xD46DhIJuo5AveBaD7AmhlXvab4oNsQDyedA0EQS/1SkxSHtj0vvEgTdRtz2MOPhp2DvGecwkhausvSIMCw/kLXP5Md9Y/TWB/hRPGmMD7wZkEn009M1g+IWEqKbI5bmucIrpwxrM9/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: vcpu_show_execution_state() difference between Arm and x86
To: Julien Grall <julien@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <85dc06ff-04df-2148-81c5-26bd25d1142f@suse.com>
 <8f295eeb-e8f2-a115-3f83-60922d0fe5f9@xen.org>
 <00457189-6ab5-24e4-b2ec-1050ebec67eb@suse.com>
 <ed59ccdc-5ac0-3b9a-eb4c-33499a316a34@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <cd73d34a-5517-d608-bad2-1d06dfdd6900@suse.com>
Date: Thu, 2 Sep 2021 13:55:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <ed59ccdc-5ac0-3b9a-eb4c-33499a316a34@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0175.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1c::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c4b381b0-ea66-45ae-9969-08d96e089a4f
X-MS-TrafficTypeDiagnostic: VI1PR04MB2957:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2957A826083E424892D20EADB3CE9@VI1PR04MB2957.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	36UspjKN61wDRsP5MhevNPA13Fytw4OlwbhzrQhHOHoEelmsYTi4wV470Rj0A/maTgHB+JaO98SFarzANhclKEGn10BZ2s+SNeC5GWWUsQpeDynZk54/kVxZtWt6/rJuwZ4Nr8xmDWdImzZjSdpuumSTGiT3LgLyDevAVXJQW0BflnVnvHRGlFTyQIKWcdoRa34Kdh0F9E3oIH4VUkcZdHnJychzgJAWE5JY2kMyHJsyBIp8xpUAL2kUCT0VnXToKcgEjLwDtSiOvpBSbXu3svsqiF6E8sHT20ODZeIS1YYxlCGma5imJxM0YwSt7B08CrPJ6BkDw+m+2illOGgh39TbK7sbEBaM4eq/uphccxWYgIphZzcSUX9BCAlMSHQVWSg+yWeCR4wfWHRwrMtfw1kJa68rBGXLUXO0k8b5DWoNYrdb1oAgmWOBotAKo7mHoOHj8UW+Yex0ey8O4YZ9K1IHQ55kPCMa3VOGcfuQnIx4FtB3IpS8DErDm7X6Nuv2nHq9Ou7xSyJdlYrCndoVvmTYljc59aoD+oIXfg41DalCd9uVG9YUeeVUej5KCVw2IY1YuiuyAt3R5pzCGwzOTfSlE2DUa74vEy9WP7GoZ+T9ub37fdJm02YV6EZbbSSmiYBL5/99qNZQOGC/FEqyQJtiyWJkwv4U/1Mse9snS+vdRm3HNJCHTyy/qgZM9G8wwrM4mpwAU4V91XR+qjVT4/LfHXOMv3Y4M/0ApwoPvIE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(38100700002)(6486002)(8676002)(31686004)(66946007)(8936002)(4326008)(26005)(186003)(66476007)(66556008)(53546011)(86362001)(5660300002)(83380400001)(508600001)(16576012)(316002)(6916009)(956004)(36756003)(2616005)(31696002)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZWczWklPSExhMEQyTm84YXFNMFJTalIwU3FPMjJvNU5kalVDWkpzTFlhMGRH?=
 =?utf-8?B?NE9RdklDZnZMUUFEZ1RaRU5lYkdPNGJObmc3Q1k1L0tGTHNIOE1UY1BseUlG?=
 =?utf-8?B?ZXl3Y1dvaS9nOXRvRW02ODBHNmh6MVF2SUVtOFFVcGZqZ0R3Y05oazkvOHkv?=
 =?utf-8?B?eEk2ei9nWjZiRENMWXYzbHdvR3JKWk1VOXVISER3MFZMRnRyY0pmeUtlY1RF?=
 =?utf-8?B?NzB1NDQ0dkZCZnN6WU9GQ3ZweS94WkxENU9OdngyUkRYOGM1UlZibzhBSlJ0?=
 =?utf-8?B?a2hlVnBpby9jbGd2NnJ1RDYwUytzRjkrazVET0dWZ1V1M0ZacmtSQU52d1ZD?=
 =?utf-8?B?ZVFKYXNaRWJqQVFnL29xbTBmRUpPOTlkekI4dEE3OWdrcThKMnVQVFZaclZW?=
 =?utf-8?B?TmlHOFlpb3BXUWpNeVBuS2l4aEsrUmxPRVRZL0F0NjVNOHZPMk5EK2tDQjJG?=
 =?utf-8?B?MlRxZWRvT3VxeWtyR3lmQnpwSkpKd1dQZm5oY1FMUU51OTBMTEVBcGtjM2dm?=
 =?utf-8?B?WCtVVG1oZzJvR1Ird1paaCtQaE5hRGFGdG9lSXpybEhVTjA5Y1J4SS90dnkz?=
 =?utf-8?B?LzRsUlF0OSt2bnFjTmNZUDd4SnBleEFTQUZjbWNaSG85MTErNVFzUDR1Smsx?=
 =?utf-8?B?dlMvcTF0dHRTdVBBR1BrV2xQYkVuK1dGM3hxZlNjQ1MvZGlITFZtSTVmc2Fw?=
 =?utf-8?B?SUp1bmNiTEtIUTl1d2pyZXZoL3NuZGo3bUE2WlkrcjFndjJ0UG1PaXZWdmpm?=
 =?utf-8?B?b25lWXVVRlBUaW9QVXRJUy85L3NvSWd5R0Z4WkxYU1dzWWNDMXgvSlZHSEdl?=
 =?utf-8?B?V1FST2NXWS96RndiN1hhUkVycTdZelIrOTlHOFJvNGlYVWVYVjQ1TVdaRlZB?=
 =?utf-8?B?TkIvTW5ObjVNK1dUYWh1eThtd215ajNzd2xFUnJUcEtkdWNSL05VUVlQYkYx?=
 =?utf-8?B?Q0xQallMUno5ak4vZDFWUzJGajhaMmx0Y3pWUC9nemRnVjFlQXJzd1g3Y3Fy?=
 =?utf-8?B?OTFNdDNKUno4cDJlVU1XNExVdFdIckFFNTFtQ1A0eTQyWjlvRkY3WWtXRFg5?=
 =?utf-8?B?dzN3dWJuemFkWUl0NUFOYk9RS3ViS0oxRE1KUjNzTzNMejBNYmsxQjhHMFRY?=
 =?utf-8?B?S0xuWmhqN0Q2NFJWUHhXQ0lGYTJIc1hBaHdtY3dRdzhsU1R3RndXaXUzcHpQ?=
 =?utf-8?B?a2lHT1dFcTJXMXpWYmVrcHNYWHYvVzVSL1U1MnlndWtxVVF0R1NUZ0NucHVm?=
 =?utf-8?B?NW5XUGxIcm9obE9BUVVhWjE3TXRQNm1VNHptOSt6blF5Y1FpZ1NQc2NzOVRB?=
 =?utf-8?B?bGowN0pBem1LSnZ4RU5LYVExcHQ0K2xZNkIrMUJZL1JCNU1yNmo1WFZNby90?=
 =?utf-8?B?bTVGcmpESkc4cEVZT256cW10cXhjNlRzOUhKd1hLVHJWUTNXNGlBakhWWTJl?=
 =?utf-8?B?ZTNEVmlzWlNQV0x2WnkvTVZidDdrems4Y3FhQkVxU0FjVkZvVnNqeGlVTXlX?=
 =?utf-8?B?MWw3RVg0ZlRKVTloREQyRTNMOWtMN0dxMksxdUpqNy9PaUkyQlVPVHJid3FJ?=
 =?utf-8?B?ZG40NFJ1Q3hRWUEzZGJhTWlBT2hYKzh1NEdOeVpCNGNELzlMSkpwVUZIN3pm?=
 =?utf-8?B?SUJOSTJoajFkWEJiN0RtMFBQV1hyV0xaK0o0RWd2NkgvcVZidVREdjdaZmhJ?=
 =?utf-8?B?NDN4L1BZTUdxd3draHZZdXErSjZsNGxXSWU0b0E0VE5PVWE0OXVXMHRIVVVj?=
 =?utf-8?Q?4k5g4Y07Tmz5Idic8B6TzOTu8SUayNWPfmUAFxp?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c4b381b0-ea66-45ae-9969-08d96e089a4f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 11:55:47.9506
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bvfWUH1MUpTW2a9121SkwGm4b1y+LTfkMh4i/kyLAplYJKa5vlAbZhHZ+N5ArkjEjmTZljSYvfncpo5Z+4TGPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2957

On 02.09.2021 13:48, Julien Grall wrote:
> On 02/09/2021 07:45, Jan Beulich wrote:
>> On 01.09.2021 20:11, Julien Grall wrote:
>>> I looked at the original commit to find out the reason to use the
>>> console lock. AFAICT, this was to allow console_force_unlock() to work
>>> properly. But it is not entirely clear why we couldn't get a new lock
>>> (with IRQ enabled) that could be forced unlocked in that function.
>>>
>>> Can either you or Andrew clarify it?
>>
>> AIUI any new lock would need to be IRQ-safe as well, as the lock
>> would be on paths taken to output stuff when the system crashed.
> 
> Hmmm... Just to confirm, what you are saying is some of the callers of 
> vcpu_show_execution_state() & co may do it with IRQ-disabled. Is that 
> correct?

No, that's not what I was saying. What I was saying is that crash-
safety requires an IRQ-safe lock, because the approach taken here
should imo match that in show_execution_state(). And _that_
function can be called with IRQs in either state.

> I have tried to play with it on Arm but then I realized that 
> check_lock() is not properly working on Arm because we don't call 
> spin_debug_enable() at boot. :/ So it would make sense why we never saw 
> any issue there...

Oops.

>> Hence it would be pointless to introduce yet another lock when the
>> one we have is already good enough. But I may be missing aspects,
>> in which case I'd have to defer to Andrew.
>>
>>> The other solution I can think off is buffering the output for
>>> show_registers and only print it once at the end. The downside is we may
>>> not get any output if there is an issue in the middle of the dump.
>>
>> Indeed; I'd prefer to avoid that, the more that it may be hard to
>> predict how much output there's going to be.
> 
> And it is not going to work as we couldn't grab the P2M lock with IRQ 
> disabled.
> 
> On Arm, the only problem is going to be the P2M lock for dump the guest 
> trace. A possible controversial approach for Arm is to just not dump the 
> guest stack or move it outside of the new lock and dump when IRQ is 
> enabled (I am not aware of any places where the guest stack will be 
> dumped and we have IRQ disabled).

Well, you could certainly omit the stack dump on Arm. I for one find
it quite useful every now and then. On x86, that is.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 12:16:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 12:16:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177278.322612 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLldR-00011f-FV; Thu, 02 Sep 2021 12:16:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177278.322612; Thu, 02 Sep 2021 12:16:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLldR-00011Y-CW; Thu, 02 Sep 2021 12:16:01 +0000
Received: by outflank-mailman (input) for mailman id 177278;
 Thu, 02 Sep 2021 12:16:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fIFR=NY=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mLldQ-00011S-G5
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 12:16:00 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fa0ee40e-376a-44db-94ae-5e5dd74d07e5;
 Thu, 02 Sep 2021 12:15:59 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8C175224E2;
 Thu,  2 Sep 2021 12:15:58 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 5B07813424;
 Thu,  2 Sep 2021 12:15:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id jFKRFH7AMGGPAgAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 02 Sep 2021 12:15:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa0ee40e-376a-44db-94ae-5e5dd74d07e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630584958; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=nOOWdLJ2NSTnwVkgHcKg56yCR+4F5c9UzSH5e1+2gTo=;
	b=CbU7IDOjKXbwq5k8Byghe8I5ggchWVyFVWa2h4ZRPOhA3zMTobOgaUEjk/EfjrjYFme12q
	RIoDNlhXGzcEu8fkM3m9TZV42bB0/yJC/fpxtWF16i2RfgykJLQpbMr7UQ/MK2bgi/KXFP
	HLS2YH5Abv9S7ejRSuRcxbKisbpRuB4=
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 0/6] x86/PVH: Dom0 building adjustments
Message-ID: <ee80d3dd-86c2-9ef8-6ae3-da89e6bf8960@suse.com>
Date: Thu, 2 Sep 2021 14:15:57 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="XBwThObvyTg0stESRzAFiEtMk83rbO7Bl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--XBwThObvyTg0stESRzAFiEtMk83rbO7Bl
Content-Type: multipart/mixed; boundary="8Vw3qpsNEgCUuknWaRKx6oCBIYRG4u23m";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <ee80d3dd-86c2-9ef8-6ae3-da89e6bf8960@suse.com>
Subject: Re: [PATCH v2 0/6] x86/PVH: Dom0 building adjustments
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>
In-Reply-To: <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>

--8Vw3qpsNEgCUuknWaRKx6oCBIYRG4u23m
Content-Type: multipart/mixed;
 boundary="------------C99582B477A2C3CE5839A164"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------C99582B477A2C3CE5839A164
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 02.09.21 13:42, Jan Beulich wrote:
> On 02.09.2021 10:30, Jan Beulich wrote:
>> The code building PVH Dom0 made use of sequences of P2M changes
>> which are disallowed as of XSA-378. First of all population of the
>> first Mb of memory needs to be redone. Then, largely as a
>> workaround, checking introduced by XSA-378 needs to be slightly
>> relaxed.
>>
>> Note that with these adjustments I get Dom0 to start booting on my
>> development system, but the Dom0 kernel then gets stuck. Since it
>> was the first time for me to try PVH Dom0 in this context (see
>> below for why I was hesitant), I cannot tell yet whether this is
>> due further fallout from the XSA, or some further unrelated
>> problem. Dom0's BSP makes it all the way through to entering the
>> idle loop while all APs are still in VPF_down.
>=20
> This last part of the mystery is now solved: By cloning from my PV
> .config, I've inherited the X86_X2APIC=3Dn that I have there. Yet
> ACPI's MADT gets populated with only x2APIC entries when building
> Dom0, which such a kernel would mostly ignore (except for logging).
> IOW in a way this was indeed a missing select, except that what's
> needed wouldn't quite work yet:
>=20
> --- a/arch/x86/xen/Kconfig
> +++ b/arch/x86/xen/Kconfig
> @@ -83,6 +83,6 @@ config XEN_PVH
>   	bool "Xen PVH guest support"
>   	depends on XEN && XEN_PVHVM && ACPI
>   	select PVH
> -	def_bool n
> +	select X86_X2APIC if XEN_DOM0
>   	help
>   	  Support for running as a Xen PVH guest.
>=20
> This is because, as mentioned, XEN_DOM0 gets turned off when XEN_PV
> is off. Whereas x2APIC isn't strictly needed for DomU afaics
> (MADT gets populated with LAPIC entries only), so the "select" also
> shouldn't be unconditional.

Correct.

We should rename XEN_DOM0 to XEN_DOM0_PV, and add a "real" XEN_DOM0.

> While likely no-one will really care, I'd like to note that this
> effectively means a 32-bit Linux PVH Dom0 would be impossible, as
> X86_X2APIC depends on X86_64. This may want reflecting in the
> eventual adjustment to the XEN_DOM0 dependencies.

Indeed.


Juergen

--------------C99582B477A2C3CE5839A164
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------C99582B477A2C3CE5839A164--

--8Vw3qpsNEgCUuknWaRKx6oCBIYRG4u23m--

--XBwThObvyTg0stESRzAFiEtMk83rbO7Bl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEwwH0FAwAAAAAACgkQsN6d1ii/Ey/a
qAf+JRKKRtXuK5FRDM7fguBvJ+saHnpstIsdg3n0Prq5e8AguDaJnyBcjMM5cAXJa+k34dQbSqVA
85FM/c7n10ZoAjxh8qgiHXTdFOGsuGtugr/qlw5bFd5kA451tLemdZNGqs5VzVPsIK4BXeZACDxA
sljFXh9CBoQwCN96BsIST2qvFx4YQ02la4QtFP0qgnOriy05gX0KfThxr1Bf7Tx7tdBHvnK2FmZ5
rR2YfZCvK12OiBgxzvNQeP4QkFc/BAsv5COzG3gTVerS+H7sjmjL+GIAv7vOgXZT9zZarQiLYzYB
rUrulqOMBhHN7G3DeA38o0G8OOFiLXVfnxF0beqrbg==
=gmi+
-----END PGP SIGNATURE-----

--XBwThObvyTg0stESRzAFiEtMk83rbO7Bl--


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 12:40:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 12:40:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177285.322624 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLm0l-0004UX-CA; Thu, 02 Sep 2021 12:40:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177285.322624; Thu, 02 Sep 2021 12:40:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLm0l-0004UQ-97; Thu, 02 Sep 2021 12:40:07 +0000
Received: by outflank-mailman (input) for mailman id 177285;
 Thu, 02 Sep 2021 12:40:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=K8X1=NY=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mLm0k-0004TY-4t
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 12:40:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23508811-42cd-4b50-9756-2a48b320094a;
 Thu, 02 Sep 2021 12:40:04 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2050.outbound.protection.outlook.com [104.47.5.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-DHg7L3bDPB-3bMkE29wI7w-1; Thu, 02 Sep 2021 14:40:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 12:40:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 12:40:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0148.eurprd01.prod.exchangelabs.com (2603:10a6:208:aa::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 12:40:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23508811-42cd-4b50-9756-2a48b320094a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630586403;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HvglPhIv0cC1gQHqWcWdOwMhFrpQApUA2uzO4pvexEw=;
	b=nURaxN0MNYy7pATnLGepX7Dm+vdX1wk4lN8snIJYnX0aB6dVdOG/iQb5B0zDkH4xEtlr7W
	6+FwkPDuD1pbgo3KXT2RetbS1Z7FCZbicE//+2eTavZjjaaby9UkqtZOdKV22A5dBNQFB2
	bIAm7sZxWikVDGjnafzrRvEPJcpZT9c=
X-MC-Unique: DHg7L3bDPB-3bMkE29wI7w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z6v52/GYwaeFluSAUd4EEcyf5vIktYq0I2ZrFYAhpSENyEXK1mBBzj7bVPKcaERIQ1S2/s0wfEwdsD0/moENd1Mszb85D1Fyy3ozqaxzbKRSnZqUIOMcY+EMuwQU3zFczwUad7q8T12xxKHBmJ2O/wuJ+QY41Htecw69KVqPZDBlRzJMFTUVBU76gciafoClWIt51JuU2OHn2XBy493Ss/5d1Dqydfn6gx7HqmFFtguKji7pMn8ItepvLcLBg81poi8jKh3WagIs/pD0AZNhszyR1h1znpinNCMJbnjg6qD5ImIMmVf0d0TrrNnGqaZOd0szRQ/vUe24HU6FtqJs7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HvglPhIv0cC1gQHqWcWdOwMhFrpQApUA2uzO4pvexEw=;
 b=LnsQd33BSrgFGrl4dKFyG4VcUfubkoDGGogpsNuhffHJz4D1VpHxChVV4/2zVswPJmuANp8Y1xgIh4i4PkNKMB1B78nECYFYgWoedTHFD9NLhoElCPVSEP7EkIixq8kQ5M0EA4I7ybymFRhZIpHdwzNf98gkrtTEN+u9roYP9pwWIewj18TZtQJcEUCnRNkFwCWBAZktpTGk1ukKRsRkLE+KBaI4aKaPDd2zxZd2fwc+S6kAG41IKZ2m322AFeCiJ0Wrar3yINw+o+oCyQakPIWpZOgJEEhBMYoCS9CMVG5XjzfPfrodiW/LOOcgJM6S75R7jUtry2QNA5o8LmDZjQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 0/6] x86/PVH: Dom0 building adjustments
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <2cd2d15a-01f7-6bbf-86a8-85437991da7d@suse.com>
 <ee80d3dd-86c2-9ef8-6ae3-da89e6bf8960@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <90bad8ba-d816-b449-1c1c-07617ba6147f@suse.com>
Date: Thu, 2 Sep 2021 14:39:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <ee80d3dd-86c2-9ef8-6ae3-da89e6bf8960@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0148.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:aa::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c310b6bc-7534-44b0-a76d-08d96e0ec792
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB73608108F11006B3F20B610CB3CE9@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rdCB8OrH2mk+bEkctaZ9M6Vri1CfEFipjt+/LnlU2n42tVjQPdwOOu/baiIA4LO0wn4SeLQr/ZKeq0NhzRM/AVnH8zX7b5aehCB6qG9Y8kLjxACw1syY4QJTaiUzalO+ScNlYiGsNFF5g5uzL9/wRGDFzYEBAKjvL03FDi7iGOA2MFc09yznWPeoA+t34wrgeYpk1uUfV8s9PjdxM5vJYiEU+VznJF5JD3X5xT6Einz7gys0LdZpHSjq89Tr65ObEODQpv32PwPrSxaiZKrl4dQ2YerrCpc3mMNzRuXhoKvMfXbdDIAHECtksJTGY56WcK2vHjK9VUBh0dh1RakFmSZ8Zjtpiy8kOr2Ax7Eoo8LE6w7OiMkJ6JWHmY+xfeZhXSNZCN2pr4fb1dXXpsDKvtJBFWxZpN7TzMa8MvG7Un85HZ0gMSsGrndpCciO0oJ0VcOCiPE7M3644kS0hphiRQhiNHv3VCc0O6//skbeW4otNRPXYDblOTq//YSQBYbBj6SVttPPBBFwQasWDHWO1DlFlZ9FhDXO5p1MhheyKb07dKrq5KySTTAzwpUcBJrZbPk5FvteFqLxdhRJjTqNV+Gw3bSIjrRTCs0u36kis8uiIh1ONlqalfkSi9jALlaIYqJXYl5//kyR3VFhaNG5Hv8Kg5FFpyBUXsm3ZrPODIA+AbP+kgq6EFHvNAmtDPfk8QX6Ghvnf3wdqi1l4Dre9A==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(346002)(376002)(39860400002)(26005)(186003)(2616005)(4326008)(6862004)(956004)(6636002)(6486002)(54906003)(5660300002)(31686004)(37006003)(38100700002)(66946007)(8936002)(16576012)(8676002)(31696002)(316002)(66556008)(66476007)(36756003)(53546011)(478600001)(2906002)(86362001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NFRZdmF3MjNyR1ZsSmtac0JJTDVoREFYSUlIb0xtNStRNzQwQ1ZPTlFiVnpF?=
 =?utf-8?B?aDFtVXpFdmFFMnBmcVc5VklCUUhzRHRoSnlDQ0tFYXErbk5pNi9PS2IwNWhS?=
 =?utf-8?B?K3VsaTVYQnUybDd0cHF4c1lkOXpiNFh4QXgzbTJlY0d5a2NnbHE3QmdrWERk?=
 =?utf-8?B?SHNza3RMUEFYM0JQQVlMUy80WmpxMm1tbnBvSVcxQk5sMlhPOGhIaFhwVC9z?=
 =?utf-8?B?OWZsTkpRa1RySG9yb2tzMDFIdlY2SEdXenBjK2VxdGVtSEY4OTB5cXhIV1lx?=
 =?utf-8?B?L3ZVbVBwaXM1UE5iTVZQVzBMckFVM2xkcGlNbTdXN2tyUGcza05LTW42dytw?=
 =?utf-8?B?Wmc2TTRMTWZmVFpsd055RDdNS1lNc0tlcVNsMzRWWEtwWTBjVHZNUzRwaXIr?=
 =?utf-8?B?QXlPUGZLc1laRDd0OWRjNmhWd2lFbkd4dHVlbmtEWVlORER0YjRVWFdXRnpG?=
 =?utf-8?B?c3k0L011NG5XdDdiaEFlaWYvUTRITWI2TDA2NUl4RmJOL2lmL2U0Q1lUK1NU?=
 =?utf-8?B?bHRCRkUra0tja0FXMmFXOGNmTkExUE05TlpQWHFUWndBbUJibUxLNjNocGxC?=
 =?utf-8?B?WjdLQWQrZW5rVEo1bHJZcVBHNTJieE1DeEx1aXVSK0E0MmcrRHJXVCsxVU5n?=
 =?utf-8?B?UE5HK2U0NTFEN3djTjFZVzNyZVpmM2tKTHFLRS8zQWZnZm8xQ3pNaEd3eXpP?=
 =?utf-8?B?ZG45ejI0ZDRvamlSTnh1aFhhNjZlUjU1clF1NDdISmZLUXlLZ0hWS0M4bzdP?=
 =?utf-8?B?RDNGMHhjVUp4OWxUbTYzUlJ5SU01aFk0K0d0b3l3L1hGMXdRR1lSaXE0Y2Vq?=
 =?utf-8?B?b2NBQmlkSjhRWmdLdm9QTTdxVk4xUnp2dTNBUHNzazU0cTgvdS9LWDgrWmVv?=
 =?utf-8?B?K0tLT0taVlNZY3I1a2NqOHVCN0EzS2xoQzRpNUE2NGNYSktMSmljbm9NaHcx?=
 =?utf-8?B?TDZXT0N1RUY2VXp4bmlLejJJQlU2SW9oYkNGVUxEbUF5NFBmcEVBOVdwbDJs?=
 =?utf-8?B?blVSaUovK1Q1VFNJaHgvU1p0blA5YnMrQ2U3ZERMMlRrVVRqd3E1MGNmMUNF?=
 =?utf-8?B?Q2NxRWZGeTlzVEczYVdqdkxWa3dWa1hGbElNOUxRYkk5YzNHWmhQYW9Mb2Z6?=
 =?utf-8?B?ZFM5QnlFakFFanRtbXpDS2RoR21zcDc4eWVkMTBaMEw5NUhRNWlVMHYwVk9u?=
 =?utf-8?B?TlpldVFhRFZld2ZvNmVjUnVJOENzYms2YmtVSEVpUFVvVExUSWUwQndsL3hu?=
 =?utf-8?B?OFBXMVl0ZWwwODFSZEF1L1FUSmFQcGhkOHQwWGdQWFpBMmhVdmJseVFZNGRF?=
 =?utf-8?B?YndZVldudXc4RmlpR2owR09oemRrT213dmFuTnBaNjNaVVBrWXJsMldqY0NK?=
 =?utf-8?B?NmgzN1draXo4UEI4QVYxUFN3VnkwT1o1RDhPaE5tU2dML1I5K3VqakxmdXJI?=
 =?utf-8?B?QlpYMXlQSFZBRy9jVDg2dHNXU0xHMkI3SUM4SHZPRnRoOUxpUWdnVW0rR2Jy?=
 =?utf-8?B?VlF1TTJnRnFpOTcwanUxK2p4WGdxV0hTS3k0UkpyMCtlWWNEL2xEVHYyQjJL?=
 =?utf-8?B?VTdMSG9WYXhCMVgzeHdybkhabkIvclh0SlZVeVJnRzlpSjVNSTRFdGhJWUoy?=
 =?utf-8?B?S3VNRkdvd0ZpNHVISFRCZXNiYk1hSXlmckZadDR1VlhXVFFOMjAxYU1uUnBV?=
 =?utf-8?B?a3hBVVAxNDJmOGt2elA1QkUxYjNDK3gxY3M2YTJkMjhtblVSSlFvcWYyK3Bv?=
 =?utf-8?Q?hkESyDZF/1hBomtz2sDsD2gRTSDKtg026Jzltu/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c310b6bc-7534-44b0-a76d-08d96e0ec792
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 12:40:00.7959
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: poM49VxOng+SzLWmdmziG3BY1ouHjLlrPZ3iv9Hit9WrU0E7Cb+llGzdC6Zgmol0mMubuakT7RLvLJg6nH5VdQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 02.09.2021 14:15, Juergen Gross wrote:
> On 02.09.21 13:42, Jan Beulich wrote:
>> On 02.09.2021 10:30, Jan Beulich wrote:
>>> The code building PVH Dom0 made use of sequences of P2M changes
>>> which are disallowed as of XSA-378. First of all population of the
>>> first Mb of memory needs to be redone. Then, largely as a
>>> workaround, checking introduced by XSA-378 needs to be slightly
>>> relaxed.
>>>
>>> Note that with these adjustments I get Dom0 to start booting on my
>>> development system, but the Dom0 kernel then gets stuck. Since it
>>> was the first time for me to try PVH Dom0 in this context (see
>>> below for why I was hesitant), I cannot tell yet whether this is
>>> due further fallout from the XSA, or some further unrelated
>>> problem. Dom0's BSP makes it all the way through to entering the
>>> idle loop while all APs are still in VPF_down.
>>
>> This last part of the mystery is now solved: By cloning from my PV
>> .config, I've inherited the X86_X2APIC=n that I have there. Yet
>> ACPI's MADT gets populated with only x2APIC entries when building
>> Dom0, which such a kernel would mostly ignore (except for logging).
>> IOW in a way this was indeed a missing select, except that what's
>> needed wouldn't quite work yet:
>>
>> --- a/arch/x86/xen/Kconfig
>> +++ b/arch/x86/xen/Kconfig
>> @@ -83,6 +83,6 @@ config XEN_PVH
>>   	bool "Xen PVH guest support"
>>   	depends on XEN && XEN_PVHVM && ACPI
>>   	select PVH
>> -	def_bool n
>> +	select X86_X2APIC if XEN_DOM0
>>   	help
>>   	  Support for running as a Xen PVH guest.
>>
>> This is because, as mentioned, XEN_DOM0 gets turned off when XEN_PV
>> is off. Whereas x2APIC isn't strictly needed for DomU afaics
>> (MADT gets populated with LAPIC entries only), so the "select" also
>> shouldn't be unconditional.
> 
> Correct.
> 
> We should rename XEN_DOM0 to XEN_DOM0_PV, and add a "real" XEN_DOM0.

Actually, as I have just found, the lack of XEN_DOM0 _is_ a problem:
xen_initial_domain() gets hardcoded to 0 without it. I'll have to
make a(nother) patch along the lines of what you suggest; hvc-xen
and "earlyprintk=xen" also don't really work together, because the
first thing xenboot_write_console() does is a !xen_pv_domain() check.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 12:50:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 12:50:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177291.322635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLmAV-00068L-AS; Thu, 02 Sep 2021 12:50:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177291.322635; Thu, 02 Sep 2021 12:50:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLmAV-00068E-7H; Thu, 02 Sep 2021 12:50:11 +0000
Received: by outflank-mailman (input) for mailman id 177291;
 Thu, 02 Sep 2021 12:50:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLmAU-000688-5z
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 12:50:10 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLmAR-0004ce-Iz; Thu, 02 Sep 2021 12:50:07 +0000
Received: from 54-240-197-224.amazon.com ([54.240.197.224]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLmAR-0008LL-CW; Thu, 02 Sep 2021 12:50:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Y/sEGRohL9Kj4xLlhQ4GFVhQuLZvKB8nlgyOX3/T2lc=; b=u7CkeW8IlR6UJuC+kaZwH+r2Co
	OXvkYJ0mzeo/66XfzGgSioiS6duz2zqntjjfXGAHoqukVrImoRyiV8v973rrVHbzUx8kr7hhjTKAH
	+DfQl0F7TRTTKHe8bKiNLMUBfdheo5KsCg8DKJC8VZrBGh82T7rq49qJcvPG2D4yGi/4=;
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Michal Orzel <michal.orzel@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Ash Wilding <ash.j.wilding@gmail.com>, Ian Jackson <iwj@xenproject.org>
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <4a5ac822-3078-de0e-8f84-6881d47ad3eb@xen.org>
 <dd9859ec-a3a9-6a74-bdae-fde092f26ac0@arm.com>
 <e8bdb065-9b46-e0d8-1fcf-7bba38e02f9e@xen.org>
 <8974fc82-94c1-ef85-153c-0923f5faf331@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <f41e36bb-efb5-fc11-f87e-6906336bca45@xen.org>
Date: Thu, 2 Sep 2021 13:50:05 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <8974fc82-94c1-ef85-153c-0923f5faf331@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

On 01/09/2021 10:38, Michal Orzel wrote:
> Hi Julien,

Hi Michal,

> On 06.08.2021 13:12, Julien Grall wrote:
>>
>>
>> On 29/07/2021 12:47, Michal Orzel wrote:
>>> Hi Julien,
>>
>> Hi Michal,
>>
>>> On 29.07.2021 13:20, Julien Grall wrote:
>>>> Hi Michal,
>>>>
>>>> On 29/07/2021 11:42, Michal Orzel wrote:
>>>>> According to ARMv8A architecture, AArch64 registers
>>>>> are 64bit wide even though in many cases the upper
>>>>> 32bit is reserved. Therefore there is no need for
>>>>> function vreg_emulate_sysreg32 on arm64. This means
>>>>> that we can have just one function vreg_emulate_sysreg
>>>>> using new function pointer:
>>>>> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>>>>>                                  register_t *r, bool read);
>>>>>
>>>>> Modify vreg_emulate_cp32 to use the new function pointer
>>>>> as well.
>>>>>
>>>>> This change allows to properly use 64bit registers in AArch64
>>>>> state and in case of AArch32 the upper 32 bits of AArch64
>>>>> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
>>>>
>>>> What you wrote only says that the bits are ignored. It doesn't say whether the bits will be 0.
>>>>
>>>> They are probably, but as I wrote yesterday, I couldn't confirm it.
>>>>
>>> Should I then remove this part of the commit or write below?:
>>> "We can assume that those bits will be 0 but the architecture
>>> reference manual does not clarify this."
>>
>> There was some back and forth on security@xen.org about this. I will summarizing the discussion here as we considered this was a just a bug.
>>
>> I wasn't looking at the correct section in the Arm Arm. There is a paragraph clearly describing the expected behavior in a different section (thanks Ash for the pointer!). Per section D1.19.2 in DDI 0487F.c:
>>
>> "
>> If the general-purpose register was accessible from AArch32 state
>>
>> The upper 32 bits either become zero, or hold the value that the same architectural register held before any AArch32 execution. The choice between these two options is IMPLEMENTATION DEFINED, and might vary dynamically within an implementation. Correspondingly, software must
>> regard the value as being a CONSTRAINED UNPREDICTABLE choice between these two values.
>>
>> This behavior applies regardless of whether any execution occurred at the Exception level that was using AArch32. That is, this behavior applies even if AArch32 state was entered by an exception
>> return from AArch64 state, and another exception was immediately taken to AArch64 state without any instruction execution in AArch32 state.
>> "
>>
>> So we can't assume the top 32-bits are zeroed unless the hypervisor ensured they were. Today, we don't have that guarantee in Xen.
>>
>> This needs to be fixed. The two approachs we discussed are:
>>     1) Update set_user_reg() to zero the top 32-bit. We have a couple of places using directly the fields xN. So we would need to switch them to use set_user_reg()
>>     2) Only saving/restoring the bottom 32-bit when entering/leaving the hypervisor.
>>
>> At the moment, my preference goes towards the latter because we don't risk to introduce new place where set_user_reg() is not used.
>>
>> I have quickly hack the entry path. This would look like:
>>
>> diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
>> index fc3811ad0ad5..65e24c88b059 100644
>> --- a/xen/arch/arm/arm64/entry.S
>> +++ b/xen/arch/arm/arm64/entry.S
>> @@ -111,6 +111,11 @@
>>    */
>>           .macro  entry, hyp, compat, save_x0_x1=1
>>           sub     sp, sp, #(UREGS_SPSR_el1 - UREGS_LR) /* CPSR, PC, SP, LR */
>> +        .if \compat == 1 /* AArch32 mode */
>> +        /* Clobber the top 32-bit of the registers */
>> +        mov    w0, w0
>> +        mov    w1, w1
>> +        .endif
>>           push    x28, x29
>>           push    x26, x27
>>           push    x24, x25
>>
>> I haven't looked whether this can be optimized or the exit path would be easier to modify.
>>
>> Anyway, this is not a new bug so I would be fine to get this patch merged first. Although, I think this wants to be fixed for xen 4.16 (CCing Ian to track it).
>>
>> I will try to find sometimes in the next couple of weeks to fix it and have another review of this patch.
>>
> 
> As the 4.16 release is getting closer I wanted to ask whether you need help with creating a pre-work patch so that this patch can be merged.

Sorry I haven't looked at the bug yet. I don't think the bug I reported 
necessarily needs to go first. This is not a new bug and AFAICT your 
patch is not going to make it worse than the current state.

Your patch needs to have an updated commit message summarizing what we 
discussed and a second review. No need to resend a patch for the updated 
commit message, this can be discussed here and updated on commit (I 
assuming there is no other comments).

> I believe this patch wants to be merged for 4.16 as the other sysreg related patches are merged already, so I'm offering a help.

If you have spare time that would be great. There are a few bits I 
wanted to look other than the entry place:
   - The DOMCTL that update the context: I believe we may need to zero 
to top bits
   - hypercall continuation (see hypercall_create_continuation)
   - multicall

For 4.16, the entry (or exit) is probably going to be a good step. We 
can look at the rest for 4.17.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 13:36:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 13:36:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177300.322652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLmsw-0002dq-44; Thu, 02 Sep 2021 13:36:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177300.322652; Thu, 02 Sep 2021 13:36:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLmsw-0002dj-0z; Thu, 02 Sep 2021 13:36:06 +0000
Received: by outflank-mailman (input) for mailman id 177300;
 Thu, 02 Sep 2021 13:36:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEJZ=NY=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mLmsv-0002dc-90
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 13:36:05 +0000
Received: from mail-pj1-x1034.google.com (unknown [2607:f8b0:4864:20::1034])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c8100bc7-347c-477d-b54e-1199acc24b65;
 Thu, 02 Sep 2021 13:36:04 +0000 (UTC)
Received: by mail-pj1-x1034.google.com with SMTP id
 j10-20020a17090a94ca00b00181f17b7ef7so1469364pjw.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 06:36:04 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 a10sm2447079pja.14.2021.09.02.06.35.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 02 Sep 2021 06:36:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8100bc7-347c-477d-b54e-1199acc24b65
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=3ii/guKHn4LpF03RfgLBF/j9oSVP4qa5E4ECxRbe8hE=;
        b=SQm9Nj4ysgjXU+46HOkYd8Q+/51pgR0klYtCW81HunVSPpNqA+7K6ET2S66TXtFo5V
         dUkSrrWmLdz9TgWh/ywwICTIxatEgDnyhnQs5/AiLJGp5rcv9KvBnwyhADMsOmMIBDEE
         nzezmmPn+wpRYYcADoD+nWE8McbCwOsGC+BH28wlQdSj65Vv6aiEEaIhoTM3yHym8m8M
         uJ7pUAhfdy9O2NQ1ZT90Jct19BmiUoi4Lm8Un1aHkNzAhPQO1ioDu9bRjpEUheed7PwG
         NaFORAtAC9wGX5zQ/JUlDZFdP09/92z+FxuN3oVRGo0d4SLQnwGPjmGI7zp0OcLmFhww
         rJ3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=3ii/guKHn4LpF03RfgLBF/j9oSVP4qa5E4ECxRbe8hE=;
        b=KnX2E/Np4AaFGmWcTZnpj5S7+b2//nhkYzkEOCE439dEMtQGcr3yGpkHnHReGRt6Hp
         c4sa9efvf9/LkEoIVY+46XbDTGGprdWXi4T2cHt2J4N0V40GTvn/0ZNQ5kDTWUNGGf9h
         Nq9a6guLiAYD7N9SDH1rfqvRl/z7ruxKYsg4uM7sJoJ2HyK+i1wSOSLZqmEvI9Axd3QX
         /uGWA8QKZ2NtEowVaro0tuq4CPR72a9Vu/VqBt5OtFg/D7tjvVG1MKIpkMcshL1BI+Hz
         aW1e7tm7uUbPDLZvymT+Dad6r2KeNfUEbzjoMCMdhiUz+r5CZxOe05KDq7IP/nkHF4mC
         BSNA==
X-Gm-Message-State: AOAM533eEj4FHOb4Wt3oDCtUDAIWJe10/Xk8l/JQfgjOKKUdEjV/5DvQ
	rJVRIjeKk7PGyDeVbQBppS4=
X-Google-Smtp-Source: ABdhPJxb2ycIjPIdhzbaYk0Qkm84LE40W+f5tgz7hUwm6cNYqkebQcMmosgs/I437B2b32yx0P3oCw==
X-Received: by 2002:a17:902:c408:b0:138:e3df:e999 with SMTP id k8-20020a170902c40800b00138e3dfe999mr2981076plk.30.1630589763493;
        Thu, 02 Sep 2021 06:36:03 -0700 (PDT)
Subject: Re: [PATCH V4 08/13] hyperv/vmbus: Initialize VMbus ring buffer for
 Isolation VM
To: Michael Kelley <mikelley@microsoft.com>, KY Srinivasan
 <kys@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
 "will@kernel.org" <will@kernel.org>, "tglx@linutronix.de"
 <tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>,
 "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
 "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
 <pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
 <hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
 <aneesh.kumar@linux.ibm.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "linux-arm-kernel@lists.infradead.org"
 <linux-arm-kernel@lists.infradead.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "rientjes@google.com" <rientjes@google.com>,
 "ardb@kernel.org" <ardb@kernel.org>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-9-ltykernel@gmail.com>
 <MWHPR21MB1593B416ED91CD454FC036E5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <e864b95d-ecb7-074b-ff0b-85cc451bad52@gmail.com>
Date: Thu, 2 Sep 2021 21:35:48 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB1593B416ED91CD454FC036E5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/2/2021 8:23 AM, Michael Kelley wrote:
>> +	} else {
>> +		pages_wraparound = kcalloc(page_cnt * 2 - 1,
>> +					   sizeof(struct page *),
>> +					   GFP_KERNEL);
>> +
>> +		pages_wraparound[0] = pages;
>> +		for (i = 0; i < 2 * (page_cnt - 1); i++)
>> +			pages_wraparound[i + 1] =
>> +				&pages[i % (page_cnt - 1) + 1];
>> +
>> +		ring_info->ring_buffer = (struct hv_ring_buffer *)
>> +			vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP,
>> +				PAGE_KERNEL);
>> +
>> +		kfree(pages_wraparound);
>> +		if (!ring_info->ring_buffer)
>> +			return -ENOMEM;
>> +	}
> With this patch, the code is a big "if" statement with two halves -- one
> when SNP isolation is in effect, and the other when not.  The SNP isolation
> case does the work using PFNs with the shared_gpa_boundary added,
> while the other case does the same work but using struct page.  Perhaps
> I'm missing something, but can both halves be combined and always
> do the work using PFNs?  The only difference is whether to add the
> shared_gpa_boundary, and whether to zero the memory when done.
> So get the starting PFN, then have an "if" statement for whether to
> add the shared_gpa_boundary.  Then everything else is the same.
> At the end, use an "if" statement to decide whether to zero the
> memory.  It would really be better to have the logic in this algorithm
> coded only once.
> 

Hi Michael:
	I have tried this before. But vmap_pfn() only works for those pfns out 
of normal memory. Please see vmap_pfn_apply() for detail and
return error when the PFN is valid.




From xen-devel-bounces@lists.xenproject.org Thu Sep 02 14:15:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 14:15:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177307.322662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLnUv-0007OD-3Z; Thu, 02 Sep 2021 14:15:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177307.322662; Thu, 02 Sep 2021 14:15:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLnUv-0007O6-0c; Thu, 02 Sep 2021 14:15:21 +0000
Received: by outflank-mailman (input) for mailman id 177307;
 Thu, 02 Sep 2021 14:15:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do2V=NY=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mLnUt-0007O0-ET
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 14:15:19 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.5.66]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 32558fec-0bf8-11ec-ae73-12813bfff9fa;
 Thu, 02 Sep 2021 14:15:17 +0000 (UTC)
Received: from PR2PR09CA0011.eurprd09.prod.outlook.com (2603:10a6:101:16::23)
 by VE1PR08MB5728.eurprd08.prod.outlook.com (2603:10a6:800:1a0::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 2 Sep
 2021 14:15:15 +0000
Received: from VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:101:16:cafe::c3) by PR2PR09CA0011.outlook.office365.com
 (2603:10a6:101:16::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Thu, 2 Sep 2021 14:15:15 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT004.mail.protection.outlook.com (10.152.18.106) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 2 Sep 2021 14:15:14 +0000
Received: ("Tessian outbound 32695b2df2f8:v103");
 Thu, 02 Sep 2021 14:15:14 +0000
Received: from 538ca24e630e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7C765AFF-7C4E-4C69-A812-4351BC845DCA.1; 
 Thu, 02 Sep 2021 14:14:41 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 538ca24e630e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 02 Sep 2021 14:14:41 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB5338.eurprd08.prod.outlook.com (2603:10a6:10:111::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Thu, 2 Sep
 2021 14:14:39 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021
 14:14:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32558fec-0bf8-11ec-ae73-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JLCyG1nPoAs2Scph3OKb7y5oaPlFomSVrHudFHc5/fc=;
 b=WpwIzIXJ2EkBEz40BicJDri6QWkBKr9N4lQiLGICtkCCphNCNGXsGaKI1Cr6hqWzYDl+aRelrj95Rx/KQKYOLuevqAjEptfPBDoluKv5ExVRf3xxqGuZjqNJaRDTShN9DkjxXLKhqR/P/CmMTFIyeiLBJD0Wh7fYcPAQPzs17i0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a7981e90009dcd52
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=moCUcvJ5s7tImV10RgKRly2fUr3/fPEx/jpYw5FEcq1PjStFa9HTATLx3SIrSalqFdmg6REV89lDkmAKXKq22MG84i0GI37SA/VJAn9JM9MfsZ8zmzDU21my0sZPhNrxjQVv8mvVeRCdoSFuFIgmUPJultXQ6IA8qKXT3+4S4bQ9LmmbaNqY4YxYwgSY6OqKOFB5zgG/slnsNq+wL64g95OtkqS0nuhLVcu1hATExVOgX7hfbBdJC0Qn1bpayQtrLbxGfcBjhrcxDg8HjCAiL5Yi0RKTKjy6dFxuJRZLGvB42OjY0dcP4UQwvDo1PVDEsqs88gVMJUp0RguZmiVFBg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JLCyG1nPoAs2Scph3OKb7y5oaPlFomSVrHudFHc5/fc=;
 b=K5t81NQeQcE4eW9LoZlPFzwAl+zF+zWaQUlcU4tyC3DmP6cW8lxlc7vMBsHAep6UxZMyxmxrdTlFvr3zbadpWPeJ0WVIX7RZdUDfcJAWVRb3Ei1dlgQKqbkj5jWIPuaJRXNDsANM+yfvjLDrYrg8iYwWDKK44/0BM/FmNPtaiGBBIwvkO+sEHzCNyaZIP11zCWfPV+dEJGSJFnFwdePD7AtJ0KeN1LU876miW6WZiXSlkAVk/mKdM/BIGzUGSenEOAtkcqs3ytgs3kFgJrF7vo287RKgTTWU9pwWHUOjQQ7l9QmkOUW8Uv39/0hpI5IpaM70Coma8Uqfj98i8zByoQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JLCyG1nPoAs2Scph3OKb7y5oaPlFomSVrHudFHc5/fc=;
 b=WpwIzIXJ2EkBEz40BicJDri6QWkBKr9N4lQiLGICtkCCphNCNGXsGaKI1Cr6hqWzYDl+aRelrj95Rx/KQKYOLuevqAjEptfPBDoluKv5ExVRf3xxqGuZjqNJaRDTShN9DkjxXLKhqR/P/CmMTFIyeiLBJD0Wh7fYcPAQPzs17i0=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
Thread-Topic: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
 device tree NUMA distance map
Thread-Index:
 AQHXjps+iLbrOQu8n06b5z+EtCRt06uM5puAgABnI4CAAPWOAIAA3gOAgABcYACAAOS/gIAAh9fw
Date: Thu, 2 Sep 2021 14:14:39 +0000
Message-ID:
 <DB9PR08MB6857CBC9AFAC5DFBA423B1929ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-25-wei.chen@arm.com>
 <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
 <e28f6230-56cc-e716-0dbb-8e1dacec7df2@suse.com>
In-Reply-To: <e28f6230-56cc-e716-0dbb-8e1dacec7df2@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: B1C2CBA5107FD847AC1096268755BBB9.0
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: e9a6306c-039d-43a2-3ef7-08d96e1c1591
x-ms-traffictypediagnostic: DB8PR08MB5338:|VE1PR08MB5728:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB5728AB2B277CB2C62D7E4C2F9ECE9@VE1PR08MB5728.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AU9cInyw5hjhDGKCSUDbvl5Jmyrjowh5uiLTNQj3qfWKOcn47uBsFFlddhEaYBrH4RWrZVeTZyy3gYd0U+n29oUXFO+NxKwijwyypgXk+DTtD0pO4usfeHyBp9Dk9RRsQ9KTC1YlRZY0lUihA7+1Lucl4+DgUZYfaB+SdwJKLZIB1P9r4xk6nIXQ3hoNBYVEE3V9/suO45gGAsV2yGwT2bHIw/RvpDY3XCQPYG42l2huNKNn/ibNkGWnkPwX/3UTR9E/bQufojZMIkYhV6NKV89BJkGNB20TCE64Zg+Gm7KGdqEKgxu/VaAWie+mGqpQeKJTDpraFMZSV7Wx9dMxnt8tJ8z2tVwOz7FTo5A1W/9KephntsHlGkO/ti27AHii8jsX8A11NbsoeC19soPqgR0RFJ3YnuFRRHFWmzLEzXBbFRJBUh+bEgcWjXnYfFoUJ53meskxoXG9Zkbp3RS3I/rH8WZG32A25J/rzYGZ2572wtgUkewDloUbX9ZnyPdEgcQZTmWMIjxrHgfRSlHMQ0dM1/EGjmJcKRByfA/JxXv6Sbc/Q6JKZzZLiRYMeFM/Z1eWFVOYG/lbzyvWWzeYeCmfYLCcXilvjkloVq3TBMCoDj2wIK8iJPmp4VsoUVGt3/FA0BHmU0Vm9n3TcWbrIE1v8bP7ymiaDg27MBwTLbPguET7H6ueVeDQKJv9KZ/cXcTnl4fr8qzxQ8jrcp0I3kTNikJGLzFesZpcDBGk4Fgpiv39HRk/nhUzeUor2SAq0TpQcoVxIsipdGH5AYqdtBRVzBHonTRm1Mqso8a+ONI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(366004)(136003)(346002)(396003)(66556008)(7696005)(26005)(66446008)(4326008)(8936002)(64756008)(186003)(66476007)(8676002)(2906002)(76116006)(83380400001)(9686003)(38100700002)(71200400001)(66946007)(53546011)(33656002)(6506007)(122000001)(86362001)(966005)(54906003)(38070700005)(478600001)(55016002)(52536014)(110136005)(316002)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5338
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	97570c57-bd19-4693-31e7-08d96e1c0046
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vw8ZSKFXU6tuCrSTm8mmzGA/gyPH+swrt8FKGcN8api9LzHseRs3nt6gy+bdG7WuHRLw6e64SHOchze2+knt6XVy4C9tD9E+kVOnN6fv+jxkVrCARzSALYeVXpx049HzKlAIUUy3F/G0VQGkKXB5eceb31cUqBuMmJFArjqLkO+Q15JeEWZlMDK0CeESbcZ3Cc+ewWVVf+N//vOeYtCoh6mH4bzAkBe301mkoEFmd2P+bHonFVh8vIqOTTzKt1mJMlkF3ufT5YL80desO0dBosWRBsAmaB21wOyxB/SLzKzrTczl4ipWRQOAbkoww28gi0cwx9gj2oKSK/GNoypuFNZsQYaQlCgBGs7mKIzhHBOAh/v/hxjx0QHXy2u/5P+IVEzmwE+Y0qFq6436P257FUJddddbo2FhWPytt6RgF+2XLDlAJsWiSWbA2LCw5aimh7jUHSs9AuRixai0CQX8d5PGrvqWJh2ZhEP39ElnSzalaaDCoz3IsGIj+OtN1FRC/QGhBXR+fYwmFN4pe94AXJX03mq5iOoRRUVPGz3p7afvZxbbJ2rqaTnJMpRHsQ+1f7FFR8wHvjbU7IiE3AruH4vjaeVMmsnMxDpLx7KAYt2HjcqTQ06FnzuDsT8T0CRbi47qIX8gPO3kVaN0dyhibC5VInd+Qf3EnTpM76B/6/EjBaT3Ttlcr6K4PLUctFdqfV4moLNp1zy60WnL1JwmdpmfD9PSIkcClu9cWuUZQzQHVvXnoISBcMUA4dR1DtsbiouGYTkvUA1JzJbo8qhPEhOES8gO3C0q0JxOSJ9ChvY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(376002)(39860400002)(346002)(396003)(46966006)(36840700001)(966005)(2906002)(81166007)(4326008)(336012)(5660300002)(478600001)(8676002)(356005)(54906003)(55016002)(52536014)(9686003)(86362001)(82740400003)(110136005)(8936002)(70586007)(33656002)(83380400001)(70206006)(47076005)(36860700001)(316002)(6506007)(26005)(186003)(7696005)(82310400003)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 14:15:14.8216
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9a6306c-039d-43a2-3ef7-08d96e1c1591
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5728

SGkgSmFuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gU2VudDogMjAyMeW5tDnmnIgy5pelIDE0OjAwDQo+
IFRvOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyBXZWkgQ2hl
bg0KPiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0
Lm9yZzsganVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMNCj4gPEJlcnRyYW5kLk1hcnF1
aXNAYXJtLmNvbT4NCj4gU3ViamVjdDogUmU6IFtYRU4gUkZDIFBBVENIIDI0LzQwXSB4ZW4vYXJt
OiBpbnRyb2R1Y2UgYSBoZWxwZXIgdG8gcGFyc2UNCj4gZGV2aWNlIHRyZWUgTlVNQSBkaXN0YW5j
ZSBtYXANCj4gDQo+IE9uIDAxLjA5LjIwMjEgMTg6MjEsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90
ZToNCj4gPiBPbiBXZWQsIDEgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+Pj4gU3RlZmFu
byBTdGFiZWxsaW5pDQo+ID4+PiBTZW50OiAyMDIx5bm0OeaciDHml6UgNTozNg0KPiA+Pj4NCj4g
Pj4+IE9uIFR1ZSwgMzEgQXVnIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+Pj4+IEkgZG9uJ3Qg
cmVhbGx5IGxpa2UgdGhpcyBpbXBsZW1lbnRhdGlvbi4gSSB3YW50IHRoZSBiZWhhdmlvciBvZg0K
PiA+Pj4+IG51bWFfc2V0X2Rpc3RhbmNlIGp1c3QgbGlrZSB0aGUgZnVuY3Rpb24gbmFtZSwgZG8g
bm90IGluY2x1ZGUNCj4gPj4+PiBpbXBsaWNpdCBvcGVyYXRpb25zLiBPdGhlcndpc2UsIGV4Y2Vw
dCB0aGUgdXNlciByZWFkIHRoaXMgZnVuY3Rpb24NCj4gPj4+PiBpbXBsZW1lbnRhdGlvbiBiZWZv
cmUgaGUgdXNlIGl0LCBoZSBwcm9iYWJseSBkb2Vzbid0IGtub3cgdGhpcw0KPiA+Pj4+IGZ1bmN0
aW9uIGhhcyBkb25lIHNvIG1hbnkgdGhpbmdzLg0KPiA+Pj4NCj4gPj4+IFlvdSBjYW4gbGVhdmUg
bnVtYV9zZXRfZGlzdGFuY2UgYXMtaXMgd2l0aG91dCBhbnkgaW1wbGljaXQgb3BlcmF0aW9ucy4N
Cj4gPj4+DQo+ID4+PiBJbiB0aGF0IGNhc2UsIGp1c3QgY2FsbCBudW1hX3NldF9kaXN0YW5jZSB0
d2ljZSBmcm9tIG51bWFfc2V0X2Rpc3RhbmNlDQo+ID4+PiBmb3IgYm90aCBmcm9tL3RvIGFuZCB0
by9mcm9tLiBudW1hX3NldF9kaXN0YW5jZSBjb3VsZCByZXR1cm4gZXJyb3IgaXMNCj4gPj4NCj4g
Pj4gSSBhbSBPSyBmb3IgdGhlIGZpcnN0IHNlbnRlbmNlLiBCdXQuLi4NCj4gPj4NCj4gPj4+IHRo
ZSBlbnRyeSB3YXMgYWxyZWFkeSBzZXQgdG8gYSBkaWZmZXJlbnQgdmFsdWUgb3Igc3VjY2VzcyBv
dGhlcndpc2UNCj4gPj4+IChhbHNvIGluIHRoZSBjYXNlIGl0IHdhcyBhbHJlYWR5IHNldCB0byB0
aGUgc2FtZSB2YWx1ZSkuIFRoaXMgd291bGQNCj4gPj4NCj4gPj4gLi4uIEkgcHJlZmVyIG5vdCB0
byBjaGVjayB0aGUgcHJldmlvdXMgdmFsdWUuIFN1YnNlcXVlbnQNCj4gbnVtYV9zZXRfZGlzdGFu
Y2UNCj4gPj4gY2FsbCB3aWxsIG92ZXJyaWRlIHByZXZpb3VzIGNhbGxzLiBLZWVwIG51bWFfc2V0
X2Rpc3RhbmNlIGFzIHNpbXBsZSBhcw0KPiA+PiBpdCBjYW4uIEFuZCB3aGVuIHlvdSBwYXNzIG5l
dyBkYXRhIHRvIG51bWFfc2V0X2Rpc3RhbmNlLCBpdCBkb2Vzbid0DQo+ID4+IGtub3cgd2hldGhl
ciB0aGUgcHJldmlvdXMgZGF0YSB3YXMgY29ycmVjdCBvciB0aGUgbmV3IGRhdGEgaXMgY29ycmVj
dC4NCj4gPj4gT25seSBjYWxsZXIgbWF5IGhhdmUga25vd24uDQo+ID4NCj4gPiBUaGF0IG1pZ2h0
IGJlIE9LIGJ1dCBpZiBub3QgbnVtYV9zZXRfZGlzdGFuY2UgdGhlbiBzb21lYm9keSBlbHNlIG5l
ZWRzDQo+ID4gdG8gY2hlY2sgYWdhaW5zdCBvdmVyd3JpdGluZyBwcmV2aW91cyB2YWx1ZXMuIFRo
YXQgaXMgdG8gYmUgYWJsZSB0byBzcG90DQo+ID4gYmFkIGRldmljZSB0cmVlIGNhc2VzIGxpa2U6
DQo+ID4NCj4gPiAgIDAgMSAyMA0KPiA+ICAgMSAwIDQwDQo+IA0KPiBXaGF0J3Mgd3Jvbmcgd2l0
aCB0aGlzPyBBdCBsZWFzdCB0aGUgQUNQSSBzcGVjIGNhcmVzIHRvIHNwZWNpZmljYWxseQ0KPiBw
ZXJtaXQgdGhpcyBjYXNlOg0KPiANCg0KVGhpcyBpcyBiZWNhdXNlIHNvbWUgbm90ZXMgZGVzY3Jp
cHRpb24gaW4gZGV2aWNlIHRyZWUgTlVNQSBiaW5kaW5nOg0KLSBkaXN0YW5jZS1tYXRyaXgNCiAg
VGhpcyBwcm9wZXJ0eSBkZWZpbmVzIGEgbWF0cml4IHRvIGRlc2NyaWJlIHRoZSByZWxhdGl2ZSBk
aXN0YW5jZXMNCiAgYmV0d2VlbiBhbGwgbnVtYSBub2Rlcy4NCiAgSXQgaXMgcmVwcmVzZW50ZWQg
YXMgYSBsaXN0IG9mIG5vZGUgcGFpcnMgYW5kIHRoZWlyIHJlbGF0aXZlIGRpc3RhbmNlLg0KDQog
IE5vdGU6DQoJMS4gRWFjaCBlbnRyeSByZXByZXNlbnRzIGRpc3RhbmNlIGZyb20gZmlyc3Qgbm9k
ZSB0byBzZWNvbmQgbm9kZS4NCglUaGUgZGlzdGFuY2VzIGFyZSBlcXVhbCBpbiBlaXRoZXIgZGly
ZWN0aW9uLg0KaHR0cHM6Ly93d3cua2VybmVsLm9yZy9kb2MvRG9jdW1lbnRhdGlvbi9kZXZpY2V0
cmVlL2JpbmRpbmdzL251bWEudHh0DQoNClNvIHdlIHRyZWF0IHRoaXMgY2FzZSBpcyB3cm9uZyBp
biBEVC4gQnV0IGFzIHlvdSBzYWlkIEFDUEkgYWxsb3cgc3VjaA0KY2FzZSwgdGhhdCBtZWFucyBy
ZWFsIG1hY2hpbmVzIGFsbG93IGRpZmZlcmVudCBkaXN0YW5jZXMgZm9yIG9uZSBwYWlyDQpub2Rl
cyBpbiBkaWZmZXJlbnQgZGlyZWN0aW9ucy4NCg0KPiAiRXhjZXB0IGZvciB0aGUgcmVsYXRpdmUg
ZGlzdGFuY2UgZnJvbSBhIFN5c3RlbSBMb2NhbGl0eSB0byBpdHNlbGYsDQo+ICBlYWNoIHJlbGF0
aXZlIGRpc3RhbmNlIGlzIHN0b3JlZCB0d2ljZSBpbiB0aGUgbWF0cml4LiBUaGlzIHByb3ZpZGVz
DQo+ICB0aGUgY2FwYWJpbGl0eSB0byBkZXNjcmliZSB0aGUgc2NlbmFyaW8gd2hlcmUgdGhlIHJl
bGF0aXZlIGRpc3RhbmNlcw0KPiAgZm9yIHRoZSB0d28gZGlyZWN0aW9ucyBiZXR3ZWVuIFN5c3Rl
bSBMb2NhbGl0aWVzIGlzIGRpZmZlcmVudC4iDQo+IA0KPiBKYW4NCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 15:07:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 15:07:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177317.322686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLoJ9-0004dB-6E; Thu, 02 Sep 2021 15:07:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177317.322686; Thu, 02 Sep 2021 15:07:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLoJ9-0004d4-39; Thu, 02 Sep 2021 15:07:15 +0000
Received: by outflank-mailman (input) for mailman id 177317;
 Thu, 02 Sep 2021 15:07:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLoJ7-0004cr-L2; Thu, 02 Sep 2021 15:07:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLoJ7-00070q-EO; Thu, 02 Sep 2021 15:07:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLoJ7-0005Ll-3h; Thu, 02 Sep 2021 15:07:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLoJ7-00031v-36; Thu, 02 Sep 2021 15:07:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/VKoLjB506Ce5AJiUuydkSylSJThWFxV6MyYfcbA82k=; b=6v+kyJqq6BuQXkKDtInxWuv5Mz
	K8gglkOZUznGjnASx6OQPUpWOWAIGitTtA11WmcG9Z9rRE3CFnPaYCQFkXY8a3E7JBK6I/wTodUqg
	jzxjtkcp5o+9Smov4L2EqPNtpoIo1plJ9nqtkaowv45XJwvbZNlwTXLeYmtCW6BJXfxg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164739-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164739: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
X-Osstest-Versions-That:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 15:07:13 +0000

flight 164739 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164739/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   8 xen-boot         fail in 164687 pass in 164739
 test-armhf-armhf-xl-credit2   8 xen-boot                   fail pass in 164687

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 164687 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 164687 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164687
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164687
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164687
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164687
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164687
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164687
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164687
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164687
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164687
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164687
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164687
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618
baseline version:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618

Last test of basis   164739  2021-09-02 01:51:34 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 15:20:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 15:20:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177327.322705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLoVY-0006sd-IU; Thu, 02 Sep 2021 15:20:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177327.322705; Thu, 02 Sep 2021 15:20:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLoVY-0006rp-FC; Thu, 02 Sep 2021 15:20:04 +0000
Received: by outflank-mailman (input) for mailman id 177327;
 Thu, 02 Sep 2021 15:20:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLoVW-0006a7-UF
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 15:20:02 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3d1e1c10-0c01-11ec-ae7d-12813bfff9fa;
 Thu, 02 Sep 2021 15:20:00 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A9BF96023F;
 Thu,  2 Sep 2021 15:19:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d1e1c10-0c01-11ec-ae7d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630595999;
	bh=zBOKCPFNms5m2rbidR117XlvEpQMb/G1xiLl/tUDypQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bI6VUHcBYp+Ke4KVgkZXS1diLy7T43k2Z6YTD+AnFAItvVCel2R4Ac4kYHhBFYc/2
	 LKz3CYU2jiQLSeiBjtU/vIFdC20uvEK+qNB1qkvitDMy4r68FD9rrhaeMZe92YHih1
	 cS3Qyx9S2imtwS5/YPfdXdOiR+ib0IeTWtAc7CKwS8uUWoVVkQJ0KjKH5vGHMN6P/L
	 MGovG3SyCOL/nDXJ04mpDuzA2/tZRSg20f3jjUC3G9GZRDwMpBqYcQurcfmx4TM0b8
	 HDxhg4XXygwbhbg6F8dNQz+CDqG/V796vnWSK0Z725xpPOKEGQB0V0MQfPMEiV5FPR
	 sJSWv5s7Yjg4Q==
Date: Thu, 2 Sep 2021 08:19:59 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse device
 tree NUMA distance map
In-Reply-To: <DB9PR08MB68572F556DB52ADCA4DA07349ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109020816120.17925@sstabellini-ThinkPad-T480s>
References: <20210811102423.28908-1-wei.chen@arm.com> <20210811102423.28908-25-wei.chen@arm.com> <alpine.DEB.2.21.2108301740500.17851@sstabellini-ThinkPad-T480s> <DB9PR08MB685715E39333597911BAB6FD9ECC9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2108311428040.18862@sstabellini-ThinkPad-T480s> <DB9PR08MB6857F9954D269AE5EC45093B9ECD9@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109010920220.17925@sstabellini-ThinkPad-T480s>
 <DB9PR08MB68572F556DB52ADCA4DA07349ECE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-2029384931-1630595860=:17925"
Content-ID: <alpine.DEB.2.21.2109020819020.17925@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2029384931-1630595860=:17925
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109020819021.17925@sstabellini-ThinkPad-T480s>

On Thu, 2 Sep 2021, Wei Chen wrote:
> Hi Stefano,
> 
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月2日 0:22
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>
> > Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to parse
> > device tree NUMA distance map
> > 
> > On Wed, 1 Sep 2021, Wei Chen wrote:
> > > Hi Stefano,
> > >
> > > > -----Original Message-----
> > > > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > > > Stefano Stabellini
> > > > Sent: 2021年9月1日 5:36
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > <Bertrand.Marquis@arm.com>
> > > > Subject: RE: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to
> > parse
> > > > device tree NUMA distance map
> > > >
> > > > On Tue, 31 Aug 2021, Wei Chen wrote:
> > > > > Hi Stefano,
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > > Sent: 2021年8月31日 8:48
> > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > > julien@xen.org;
> > > > > > jbeulich@suse.com; Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > > > Subject: Re: [XEN RFC PATCH 24/40] xen/arm: introduce a helper to
> > > > parse
> > > > > > device tree NUMA distance map
> > > > > >
> > > > > > On Wed, 11 Aug 2021, Wei Chen wrote:
> > > > > > > A NUMA aware device tree will provide a "distance-map" node to
> > > > > > > describe distance between any two nodes. This patch introduce a
> > > > > > > new helper to parse this distance map.
> > > > > > >
> > > > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > > > ---
> > > > > > >  xen/arch/arm/numa_device_tree.c | 67
> > > > +++++++++++++++++++++++++++++++++
> > > > > > >  1 file changed, 67 insertions(+)
> > > > > > >
> > > > > > > diff --git a/xen/arch/arm/numa_device_tree.c
> > > > > > b/xen/arch/arm/numa_device_tree.c
> > > > > > > index bbe081dcd1..6e0d1d3d9f 100644
> > > > > > > --- a/xen/arch/arm/numa_device_tree.c
> > > > > > > +++ b/xen/arch/arm/numa_device_tree.c
> > > > > > > @@ -200,3 +200,70 @@ device_tree_parse_numa_memory_node(const
> > void
> > > > *fdt,
> > > > > > int node,
> > > > > > >
> > > > > > >      return 0;
> > > > > > >  }
> > > > > > > +
> > > > > > > +/* Parse NUMA distance map v1 */
> > > > > > > +int __init
> > > > > > > +device_tree_parse_numa_distance_map_v1(const void *fdt, int
> > node)
> > > > > > > +{
> > > > > > > +    const struct fdt_property *prop;
> > > > > > > +    const __be32 *matrix;
> > > > > > > +    int entry_count, len, i;
> > > > > > > +
> > > > > > > +    printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n");
> > > > > > > +
> > > > > > > +    prop = fdt_get_property(fdt, node, "distance-matrix", &len);
> > > > > > > +    if ( !prop )
> > > > > > > +    {
> > > > > > > +        printk(XENLOG_WARNING
> > > > > > > +               "NUMA: No distance-matrix property in distance-
> > > > map\n");
> > > > > > > +
> > > > > > > +        return -EINVAL;
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    if ( len % sizeof(uint32_t) != 0 )
> > > > > > > +    {
> > > > > > > +        printk(XENLOG_WARNING
> > > > > > > +               "distance-matrix in node is not a multiple of
> > > > u32\n");
> > > > > > > +        return -EINVAL;
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    entry_count = len / sizeof(uint32_t);
> > > > > > > +    if ( entry_count <= 0 )
> > > > > > > +    {
> > > > > > > +        printk(XENLOG_WARNING "NUMA: Invalid distance-
> > matrix\n");
> > > > > > > +
> > > > > > > +        return -EINVAL;
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    matrix = (const __be32 *)prop->data;
> > > > > > > +    for ( i = 0; i + 2 < entry_count; i += 3 )
> > > > > > > +    {
> > > > > > > +        uint32_t from, to, distance;
> > > > > > > +
> > > > > > > +        from = dt_read_number(matrix, 1);
> > > > > > > +        matrix++;
> > > > > > > +        to = dt_read_number(matrix, 1);
> > > > > > > +        matrix++;
> > > > > > > +        distance = dt_read_number(matrix, 1);
> > > > > > > +        matrix++;
> > > > > > > +
> > > > > > > +        if ( (from == to && distance != NUMA_LOCAL_DISTANCE) ||
> > > > > > > +            (from != to && distance <= NUMA_LOCAL_DISTANCE) )
> > > > > > > +        {
> > > > > > > +            printk(XENLOG_WARNING
> > > > > > > +                   "Invalid nodes' distance from node#%d to
> > node#%d
> > > > > > = %d\n",
> > > > > > > +                   from, to, distance);
> > > > > > > +            return -EINVAL;
> > > > > > > +        }
> > > > > > > +
> > > > > > > +        printk(XENLOG_INFO "NUMA: distance from node#%d to
> > node#%d
> > > > > > = %d\n",
> > > > > > > +               from, to, distance);
> > > > > > > +        numa_set_distance(from, to, distance);
> > > > > > > +
> > > > > > > +        /* Set default distance of node B->A same as A->B */
> > > > > > > +        if (to > from)
> > > > > > > +             numa_set_distance(to, from, distance);
> > > > > >
> > > > > > I am a bit unsure about this last 2 lines: why calling
> > > > numa_set_distance
> > > > > > in the opposite direction only when to > from? Wouldn't it be OK
> > to
> > > > > > always do both:
> > > > > >
> > > > > > numa_set_distance(from, to, distance);
> > > > > > numa_set_distance(to, from, distance);
> > > > > >
> > > > > > ?
> > > > > >
> > > > > I borrowed this code from Linux, but here is my understanding:
> > > > >
> > > > > First, I read some notes in
> > Documentation/devicetree/bindings/numa.txt
> > > > > 1. Each entry represents distance from first node to second node.
> > > > > The distances are equal in either direction.
> > > > > 2. distance-matrix should have entries in lexicographical ascending
> > > > > order of nodes.
> > > > >
> > > > > Here is an example of distance-map node in DTB:
> > > > > Sample#1, full list:
> > > > > 		distance-map {
> > > > > 			 compatible = "numa-distance-map-v1";
> > > > > 			 distance-matrix = <0 0  10>,
> > > > > 					   <0 1  20>,
> > > > > 					   <0 2  40>,
> > > > > 					   <0 3  20>,
> > > > > 					   <1 0  20>,
> > > > > 					   <1 1  10>,
> > > > > 					   <1 2  20>,
> > > > > 					   <1 3  40>,
> > > > > 					   <2 0  40>,
> > > > > 					   <2 1  20>,
> > > > > 					   <2 2  10>,
> > > > > 					   <2 3  20>,
> > > > > 					   <3 0  20>,
> > > > > 					   <3 1  40>,
> > > > > 					   <3 2  20>,
> > > > > 					   <3 3  10>;
> > > > > 		};
> > > > >
> > > > > Call numa_set_distance when "to > from" will prevent Xen to call
> > > > > numa_set_distance(0, 1, 20) again when it's setting distance for <1
> > 0
> > > > 20>.
> > > > > But, numa_set_distance(1, 0, 20) will be call twice.
> > > > >
> > > > > Normally, distance-map node will be optimized in following sample#2,
> > > > > all redundant entries are removed:
> > > > > Sample#2, partial list:
> > > > > 		distance-map {
> > > > > 			 compatible = "numa-distance-map-v1";
> > > > > 			 distance-matrix = <0 0  10>,
> > > > > 					   <0 1  20>,
> > > > > 					   <0 2  40>,
> > > > > 					   <0 3  20>,
> > > > > 					   <1 1  10>,
> > > > > 					   <1 2  20>,
> > > > > 					   <1 3  40>,
> > > > > 					   <2 2  10>,
> > > > > 					   <2 3  20>,
> > > > > 					   <3 3  10>;
> > > > > 		};
> > > > >
> > > > > There is not any "from > to" entry in the map. But using this
> > partial
> > > > map
> > > > > still can set all distances for all pairs. And numa_set_distance(1,
> > 0,
> > > > 20)
> > > > > will be only once.
> > > >
> > > > I see. I can't find in Documentation/devicetree/bindings/numa.txt
> > where
> > > > it says that "from > to" nodes can be omitted. If it is not written
> > > > down, then somebody could easily optimize it the opposite way:
> > > >
> > > >  			 distance-matrix = <0 0  10>,
> > > >  					   <1 0  20>,
> > > >  					   <2 0  40>,
> > > >  					   <3 0  20>,
> > > >  					   <1 1  10>,
> > > >  					   <2 1  20>,
> > > >  					   <3 1  40>,
> > > >  					   <2 2  10>,
> > > >  					   <3 2  20>,
> > > >  					   <3 3  10>;
> > > >
> > >
> > > Yes, you're right. Spec doesn't say opposite way is unallowed.
> > >
> > > > I think the code in Xen should be resilient and able to cope with a
> > > > device tree like the one you wrote or the one I wrote. From a code
> > > > perspective, it should be very easy to do. If nothing else it would
> > make
> > > > Xen more resilient against buggy firmware.
> > > >
> > > >
> > >
> > > I don't disagree with that.
> > >
> > > > > > But in any case, I have a different suggestion. The binding states
> > > > that
> > > > > > "distances are equal in either direction". Also it has an example
> > > > where
> > > > > > only one direction is expressed unfortunately (at the end of the
> > > > > > document).
> > > > > >
> > > > >
> > > > > Oh, I should see this comment first, then I will not post above
> > > > > comment : )
> > > > >
> > > > > > So my suggestion is to parse it as follows:
> > > > > >
> > > > > > - call numa_set_distance just once from
> > > > > >   device_tree_parse_numa_distance_map_v1
> > > > > >
> > > > > > - in numa_set_distance:
> > > > > >     - set node_distance_map[from][to] = distance;
> > > > > >     - check node_distance_map[to][from]
> > > > > >           - if unset, node_distance_map[to][from] = distance;
> > > > > >           - if already set to the same value, return success;
> > > > > >           - if already set to a different value, return error;
> > > > >
> > > > > I don't really like this implementation. I want the behavior of
> > > > > numa_set_distance just like the function name, do not include
> > > > > implicit operations. Otherwise, except the user read this function
> > > > > implementation before he use it, he probably doesn't know this
> > > > > function has done so many things.
> > > >
> > > > You can leave numa_set_distance as-is without any implicit operations.
> > > >
> > > > In that case, just call numa_set_distance twice from numa_set_distance
> > > > for both from/to and to/from. numa_set_distance could return error is
> > >
> > > I am OK for the first sentence. But...
> > >
> > > > the entry was already set to a different value or success otherwise
> > > > (also in the case it was already set to the same value). This would
> > >
> > > ... I prefer not to check the previous value. Subsequent
> > numa_set_distance
> > > call will override previous calls. Keep numa_set_distance as simple as
> > > it can. And when you pass new data to numa_set_distance, it doesn't
> > > know whether the previous data was correct or the new data is correct.
> > > Only caller may have known.
> > 
> > That might be OK but if not numa_set_distance then somebody else needs
> > to check against overwriting previous values. That is to be able to spot
> > bad device tree cases like:
> > 
> >   0 1 20
> >   1 0 40
> 
> 
> How about we check it still in NUMA distance parse function?
> Before setting the numa_set_distance for one pair nodes (e.g. a -> b),
> we can get its opposite way distance first.
> 
> distance_b_a = __node_distance(b, a); ==> get opposite way distance.
> if (distance_b_a == 0) ==> opposite way distance has not been set
> {
>     numa_set_distance(a, b, 20); ==> set both
>     numa_set_distance(b, a, 20)
> } else {
>     if (distance_b_a == 20) ==> opposite way distance has been set
>        numa_set_distance(a, b, 20); ==> set this way only
>     else ===> opposite way distance has been set, but is unmatched
>        // What can we do here?
>        Panic the system? or Just warning users? Or choose the bigger
>        distance for both ways?
>        
>        And distance_b_a == NUMA_NO_DISTANCE would be a special case
>        here.
> }

Yes, I think something like this would work. If we detect an error in
device tree I would probably print a warning and disable NUMA. It would
also be acceptable to panic.
--8323329-2029384931-1630595860=:17925--


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 15:57:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 15:57:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177393.322771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLp5x-0005mw-Hk; Thu, 02 Sep 2021 15:57:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177393.322771; Thu, 02 Sep 2021 15:57:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLp5x-0005mp-Dd; Thu, 02 Sep 2021 15:57:41 +0000
Received: by outflank-mailman (input) for mailman id 177393;
 Thu, 02 Sep 2021 15:57:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLp5v-0005mj-Fa
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 15:57:39 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.5])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7cbaedda-0c06-11ec-ae82-12813bfff9fa;
 Thu, 02 Sep 2021 15:57:34 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR2101MB0729.namprd21.prod.outlook.com (2603:10b6:301:76::39)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.1; Thu, 2 Sep
 2021 15:57:25 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 15:57:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7cbaedda-0c06-11ec-ae82-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KCAiBFS/0mCgB2qMN5U9gBLvp1U2Cyy92ELkvrC9T80fD4TYG+I/WxS/YPZIKOugn6fTW6Pj1TpRJhi8gbS00Rafsetoy/2Lc/w1ZJMpyW4OumOJrPFbLXen/VOsuYhCh/DUAxmh2ciewoPS3lF5hgJT1D3ot0TXIrZAhfvxqNONTMHKG31ePfmhU8tpvYJfR565aOA5fy6W4F6zaITNPWME2dsxbWFvzFV9oBHuH0cTJtCUSQmDwr29IfWXJb5vplgKPkmMDOvTJUwhljbHlUTQbz1jKcxZhiVsSFojaRwLF9nkpec0Hk8QT0b08MjR9jMcUDbOTp1M1SVh0RIoDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HuvGa1HIbj6xCIKr0VB/aMjA6tx/V75cyR5Rtnz4Cmo=;
 b=hX6PpM7beSSZeEmz9DUmaJ9pJyIkwjNdkTKfArPGom7dxrRA1MWEz+4y9+uxPH1xQ/w55bLGHYhFSX8eB7yLOh/u5NoGkyzrihATJth2kpIKd7wgGTrnYPm6XhYWHW3R6dG3+Bczq1WxwxsNMIT2fj8jVNsiYSXTsKBZU4KP0yZJ3/ZNPiweFdkQnQC1sa9fA/hrNf2LbPnbT74N1UwNPm2bPx2k1XgOUhJNsEbGH08/G2BFUfnfJsqUguv0tdXM5VWeaiCWbg/Y4tu0r0A28CTIR+MgpTwTUVZejfrljk9hc+LJMCKPO8t5vcxRvLaT/ZWJ4bBJAyk0vmd44Vs00Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HuvGa1HIbj6xCIKr0VB/aMjA6tx/V75cyR5Rtnz4Cmo=;
 b=BhPPiDHloa926sA3o5SBOABnbTxqZIOEXuJsLh4eYXVBnNr3+we4lcCH70qJWJ0JCTR7n6Zfsr/aBchdBEGRJnifOfVKLZFJwmT3UcPf2WTgCWs7k+jTugtAj/FRWa6bLqb+AbqkWGkRbtrf1n477usdwrbmPsPiBl92NdaCHys=
From: Michael Kelley <mikelley@microsoft.com>
To: Christoph Hellwig <hch@lst.de>
CC: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>,
	"m.szyprowski@samsung.com" <m.szyprowski@samsung.com>, "robin.murphy@arm.com"
	<robin.murphy@arm.com>, "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
	"thomas.lendacky@amd.com" <thomas.lendacky@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "pgonda@google.com" <pgonda@google.com>,
	"martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>,
	"iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
Thread-Topic: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
Thread-Index: AQHXm2f20hs45o24pkmChnbOnB/NPKuL9ngAgAHkV3CAAo9WgIAAfS2g
Date: Thu, 2 Sep 2021 15:57:24 +0000
Message-ID:
 <MWHPR21MB1593060DCFD854FDA14604D3D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210830120036.GA22005@lst.de>
 <MWHPR21MB15933503E7C324167CB4132CD7CC9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <20210902075939.GB14986@lst.de>
In-Reply-To: <20210902075939.GB14986@lst.de>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=341860f6-2dd3-4e6b-aaea-e3d22051d2db;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-02T15:27:40Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 29e260fb-9419-416f-e7a1-08d96e2a5b72
x-ms-traffictypediagnostic: MWHPR2101MB0729:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR2101MB072932751EF6DA0A12A9586DD7CE9@MWHPR2101MB0729.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 mrF4oMoXJSuRKL/XGzpZTJvFlKLDUPlBxhocIR348p9dCVr/qB/aSYqavDNWBtQhLYznx6PnBdzPIzuEu8muiihXZRDhc0FVPy5XsTO+p1l7SRkCY+mB0m63qcViFQhmw+6PAzn/E7FipdBaK4AhK5ZQ3dQVtrVE8aPoT4EO1UxWSIixV2cMtArTkpfY/t9Ek37zSyD3LzpQU26GYD7Z54ukfHYYdrSGOtxlDa3/caqm176zYfa21Mm8pcy2zPWa6iAASleD2A19TwfTcmBZRQ8q4SXTcT8o5UbAqeE9OsPy4IBrj4Yx7dfUaDLRdS9lySLGdIKsrcCFQ82KN3/11kJJWQ7GgQr3nl5JYk+5jezIGWSnvD3m+J4xMfPdwdjoBRtFfOnKx35p66gL60qd6WZIftZKspfKOGvaopwP+aBRbYXZIPcS6xvyhimqjR/CIWZZSgZhzSz3ho0/hNZP5IRZyGWU0TUhfzTo779XK5VIkzkHOi/I+wQKABXNOnIAzz4gYMyxGSA8wI4/XXSBH34OTNsHPQ3ufEXjWuwlXD5QLQ66kzcINxj7ivfCDzFL97r6bid/nlSlwuWg7B98nE8K4phLCDpmtLPECZZSRqt2XA4p99Vv24InBJBv5lGKFyTQOnTCY92zkMs3Fybe5w8Kl5GpNBPwECoMT2ofOdySGAqFHos0OzDlHss5FJ/cbS2KYfWc3Lw0lww1z+fA4A==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(9686003)(86362001)(10290500003)(8990500004)(76116006)(26005)(66556008)(33656002)(186003)(55016002)(6916009)(38070700005)(122000001)(54906003)(82960400001)(71200400001)(82950400001)(66476007)(64756008)(316002)(66446008)(4326008)(66946007)(38100700002)(7696005)(7406005)(2906002)(7366002)(5660300002)(8936002)(6506007)(7416002)(52536014)(8676002)(83380400001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?txgEKNhptKnTZhKRFxE9zXxPdblEEya4z3HY9Y9W+T8Go+v0NzILZI6Antk2?=
 =?us-ascii?Q?0AjU8L6C4+Pdz9J9yNTP3+W6bvhC6KHsZFFftpbRs8qBExE4XF1rEyej6XB6?=
 =?us-ascii?Q?TF3LP39SbQevIVDByyzHpIGKh0wRn8kGGWsI8RmMHR7duippFZbzzS+u5Omz?=
 =?us-ascii?Q?a7UpgUKenDyv008SIWttlg1AxbeKp8XDdLs02JClXg/lX55zviivWx25oqLq?=
 =?us-ascii?Q?h7wlY28mwkatvnzabVvfb+joZ2jgbo4wT4HBae10bX9QJJ2tD3+afv2tXXuE?=
 =?us-ascii?Q?ZJCBsAw09QlZVcG5X7AlVJP/Z54ztWI7perLhH243k87s00vfSlT6LYscL9r?=
 =?us-ascii?Q?rc1Nfq3Va8ZBovRoxJrhyMpY9TjA4zYDGKStDrQ0kTEcdGms2WHd1od6G1IW?=
 =?us-ascii?Q?5eRQoPhVcgWECYRILtl0370PKfMyQ01/zxJ5rOtXMqxuOzo959qRirNoWTbL?=
 =?us-ascii?Q?CvaUsKCwb81UHtb8QTr5BjfFG9tck0NsyXaB3NbaGN2zgMpZtkqZrKnmsM91?=
 =?us-ascii?Q?11vggvJNKD4erPN+76nEVLmrNJh7mGpPOfx9xWMwuY/nLbovZtZrAWsJ4i/I?=
 =?us-ascii?Q?uIJE1RTX1B8Ws09qDeD4AhxVR++baJjCDtKVcCxJd7GzbUR4UKxetYyHnoyW?=
 =?us-ascii?Q?+YZXn93BjZpK7W+yn8YuX5vlWh0gO1CUOqu9z8nIEJn4PNxo2q7pj9mYbElG?=
 =?us-ascii?Q?NGJeuXowqYgtxlBeiYiBdXUc1JNo5kpAXE/IEB8iSm8fuF5rE+h+LM1hpI0X?=
 =?us-ascii?Q?n5Cl5zan+tlWbr1vS9ZLyGUvgFUz8Oku3XCvIUybVz/m9cGbExAnrC0xzM/r?=
 =?us-ascii?Q?cpR3HBgNuJCQcEBBWv5tXPKrwBdeNwRvHsWwpT56Cjurg8BzAK9bLLXITIp6?=
 =?us-ascii?Q?KfBprR6CCu0wXbPU6GXdFSOI3R5V5ULB+jlscT8gBDf5vCAx+PlkQ2LLuc8u?=
 =?us-ascii?Q?gBiCPsLGalJ8FdBPXg7MpCUr51oGQI8SKyWqIkih++ORzKKulnjiNv2L2lO2?=
 =?us-ascii?Q?FMxdWIvmKSgHezvGgLE7QHJIBmFZvqW3mwOOLxKRJqSIEagN9abdW+cq0gCR?=
 =?us-ascii?Q?96OMxcP+W0DAtNyS8xUGJZnV5NIoh9fegO4HEWPwT3TRHRfoXI02MnKCLulT?=
 =?us-ascii?Q?kFe2v/kRAS82Ap2d+DHv9bjTGE/7UoPoxzd2rDZaFQ1rsfKFo5qWlXnjvcHd?=
 =?us-ascii?Q?nwqm4i88XU1RnK0+YolCKjff+k3PVg0Bk/Z4lvADn0xx80ridOQj2QBEFHyF?=
 =?us-ascii?Q?aiy4NeMdes3ZbGoZn69+7RlIE9s7Oe4NVOZnzbyJ52agJQbDEAWM5lA5osLb?=
 =?us-ascii?Q?uVjlCOUPx2PqzbxSIQG8WKYA?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 29e260fb-9419-416f-e7a1-08d96e2a5b72
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 15:57:24.8646
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: afQic2r0Oqvq+TKsAMcY2JuSksSLHirOl2XxJoYethj2b0nNTqDhmPNgN8NTmSHtCHvfeYm6BzmB8XhsRqMeI40O8olSiY1StN/gd4EMzMM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2101MB0729

From: Christoph Hellwig <hch@lst.de> Sent: Thursday, September 2, 2021 1:00=
 AM
>=20
> On Tue, Aug 31, 2021 at 05:16:19PM +0000, Michael Kelley wrote:
> > As a quick overview, I think there are four places where the
> > shared_gpa_boundary must be applied to adjust the guest physical
> > address that is used.  Each requires mapping a corresponding
> > virtual address range.  Here are the four places:
> >
> > 1)  The so-called "monitor pages" that are a core communication
> > mechanism between the guest and Hyper-V.  These are two single
> > pages, and the mapping is handled by calling memremap() for
> > each of the two pages.  See Patch 7 of Tianyu's series.
>=20
> Ah, interesting.
>=20
> > 3)  The network driver send and receive buffers.  vmap_phys_range()
> > should work here.
>=20
> Actually it won't.  The problem with these buffers is that they are
> physically non-contiguous allocations. =20

Indeed you are right.  These buffers are allocated with vzalloc().

> We really have two sensible options:
>=20
>  1) use vmap_pfn as in the current series.  But in that case I think
>     we should get rid of the other mapping created by vmalloc.  I
>     though a bit about finding a way to apply the offset in vmalloc
>     itself, but I think it would be too invasive to the normal fast
>     path.  So the other sub-option would be to allocate the pages
>     manually (maybe even using high order allocations to reduce TLB
>     pressure) and then remap them

What's the benefit of getting rid of the other mapping created by
vmalloc if it isn't referenced?  Just page table space?  The default sizes
are a 16 Meg receive buffer and a 1 Meg send buffer for each VMbus
channel used by netvsc, and usually the max number of channels
is 8.  So there's 128 Meg of virtual space to be saved on the receive
buffers,  which could be worth it.

Allocating the pages manually is also an option, but we have to
be careful about high order allocations.  While typically these buffers
are allocated during system boot, these synthetic NICs can be hot
added and removed while the VM is running.   The channel count
can also be changed while the VM is running.  So multiple 16 Meg
receive buffer allocations may need to be done after the system has
been running a long time.

>  2) do away with the contiguous kernel mapping entirely.  This means
>     the simple memcpy calls become loops over kmap_local_pfn.  As
>     I just found out for the send side that would be pretty easy,
>     but the receive side would be more work.  We'd also need to check
>     the performance implications.

Doing away with the contiguous kernel mapping entirely seems like
it would result in fairly messy code to access the buffer.  What's the
benefit of doing away with the mapping?  I'm not an expert on the
netvsc driver, but decoding the incoming packets is already fraught
with complexities because of the nature of the protocol with Hyper-V.
The contiguous kernel mapping at least keeps the basics sane.

>=20
> > 4) The swiotlb memory used for bounce buffers.  vmap_phys_range()
> > should work here as well.
>=20
> Or memremap if it works for 1.
>=20
> > Case #2 above does unusual mapping.  The ring buffer consists of a ring
> > buffer header page, followed by one or more pages that are the actual
> > ring buffer.  The pages making up the actual ring buffer are mapped
> > twice in succession.  For example, if the ring buffer has 4 pages
> > (one header page and three ring buffer pages), the contiguous
> > virtual mapping must cover these seven pages:  0, 1, 2, 3, 1, 2, 3.
> > The duplicate contiguous mapping allows the code that is reading
> > or writing the actual ring buffer to not be concerned about wrap-around
> > because writing off the end of the ring buffer is automatically
> > wrapped-around by the mapping.  The amount of data read or
> > written in one batch never exceeds the size of the ring buffer, and
> > after a batch is read or written, the read or write indices are adjuste=
d
> > to put them back into the range of the first mapping of the actual
> > ring buffer pages.  So there's method to the madness, and the
> > technique works pretty well.  But this kind of mapping is not
> > amenable to using vmap_phys_range().
>=20
> Hmm.  Can you point me to where this is mapped?  Especially for the
> classic non-isolated case where no vmap/vmalloc mapping is involved
> at all?

The existing code is in hv_ringbuffer_init() in drivers/hv/ring_buffer.c.
The code hasn't changed in a while, so any recent upstream code tree
is valid to look at.  The memory pages are typically allocated
in vmbus_alloc_ring() in drivers/hv/channel.c.

Michael


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:14:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:14:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177402.322788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpM7-0000Vu-5N; Thu, 02 Sep 2021 16:14:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177402.322788; Thu, 02 Sep 2021 16:14:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpM7-0000Vn-0q; Thu, 02 Sep 2021 16:14:23 +0000
Received: by outflank-mailman (input) for mailman id 177402;
 Thu, 02 Sep 2021 16:14:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9uwi=NY=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mLpM6-0000Vh-Bh
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:14:22 +0000
Received: from outbound.mail.eo.outlook.com (unknown [40.93.3.1])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d2ae41a4-0c08-11ec-ae8f-12813bfff9fa;
 Thu, 02 Sep 2021 16:14:17 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0287.namprd21.prod.outlook.com (2603:10b6:300:7a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.7; Thu, 2 Sep
 2021 16:14:04 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::3c8b:6387:cd5:7d86%8]) with mapi id 15.20.4478.014; Thu, 2 Sep 2021
 16:14:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2ae41a4-0c08-11ec-ae8f-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FNoAqQoI71meammxhfj/gx/c4tb7ezx8HjYle3+uh/lGS+KUgOHxk2xCR+m9lPzeQ4UqiOR0CNZ6slDBS3fO1CGM202k3dYbdbApMSIGBz9zRY/stncl6btwYCpR2M9FI+YBX9p+V53Lflh6Yhia5ir8QsDONyyw8HGewaV2YIcYSrIAzIRVjiOqtfO1Ul17Bp1IysbZxz5EommUMrNUbtySJg28jD7ZJ/Ja1QLSmGfnJCP5tAYJwuD2O0a/fbrMNvYBaw17RNoPYjyM/ak6B1Yonc7FtOhFvsCIrqVRCKcBrRofVLzQ0AAk47ltrFR24kiplPOhsTY42R6Ey5DJMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=h8kFc9GqemG3bgd5NrDd4yM/37itRP5ArxE57wxbEBg=;
 b=RIpmuxN6L/mRNY3I47H+xtO3SGaKme1u7YzdHF90Gx6MRF0jLM09QwOS54FQpv3/Cwot/atmlLIYXSqjh7L2cWt/wFJ6Zsas9JyVk2vxiWT8PcOL6/mGBq2abT3tY0oh6phldoRgDtfYLrP4e7tBLYJ6dK0g5BdT/g740+I8tpb71g6hPp7p+Sx6u+ci8pvRiwMEAJ7rT7ZsbzEcMeT8cQdRCQ8nbXULJL7+6RvpifaZIZkYOFTNi2KE/q0oNRXC++y4RWbvQbzyznu4dFkSt4S9xONtE2tP2SH4vDWMPELiBS8KlSacXyUnwf4Gp0kMt6L/u0HdnvCicxBO9QxWWg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=h8kFc9GqemG3bgd5NrDd4yM/37itRP5ArxE57wxbEBg=;
 b=bxo3sTyL6l/ajEMjZ4L7vv/WZ91eFA0PYD0Qm08JtECVJgnf10eigJ2XiYudU8qhGB/WxQbZ1VqS/az+2znapb61sHQFHw4p0tPsJ2W7y5zXIalp1ombnzzPBaSO8gem6IdwAUO09cIrWmse5A1T5/fqbZiLTw+QCIfkpezMqhg=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "catalin.marinas@arm.com"
	<catalin.marinas@arm.com>, "will@kernel.org" <will@kernel.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>, "mingo@redhat.com"
	<mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org"
	<x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "davem@davemloft.net"
	<davem@davemloft.net>, "kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
	<martin.petersen@oracle.com>, "gregkh@linuxfoundation.org"
	<gregkh@linuxfoundation.org>, "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de"
	<hch@lst.de>, "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "brijesh.singh@amd.com"
	<brijesh.singh@amd.com>, "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
	<pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
	<hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "krish.sadhukhan@oracle.com"
	<krish.sadhukhan@oracle.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "rientjes@google.com"
	<rientjes@google.com>, "ardb@kernel.org" <ardb@kernel.org>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V4 08/13] hyperv/vmbus: Initialize VMbus ring buffer for
 Isolation VM
Thread-Topic: [PATCH V4 08/13] hyperv/vmbus: Initialize VMbus ring buffer for
 Isolation VM
Thread-Index: AQHXm2gFIWSYU02+UUyReBHQOl5/ZauK4YnAgAXmhwCAACrvQA==
Date: Thu, 2 Sep 2021 16:14:04 +0000
Message-ID:
 <MWHPR21MB1593917C95C95128A4E835D5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210827172114.414281-9-ltykernel@gmail.com>
 <MWHPR21MB1593B416ED91CD454FC036E5D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <e864b95d-ecb7-074b-ff0b-85cc451bad52@gmail.com>
In-Reply-To: <e864b95d-ecb7-074b-ff0b-85cc451bad52@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=ef79cd5e-7331-4c0e-adbb-97b052469058;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-02T16:09:28Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1a10f30f-c31a-4b0c-12e9-08d96e2caf25
x-ms-traffictypediagnostic: MWHPR21MB0287:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB0287AA3A53ED847207E23899D7CE9@MWHPR21MB0287.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 vGOnTAxLePV4piCloHPg0HNrCCKjFqOJ03ruJTD+GCTP4UF1PaIK8gwhHPNNJd14psyj+2zOuqCghmd5/QHhATYperkokL2M12IxevdCyHmBPDh+Qgm6ulnw8OMa5ixVRiPXubatO98NxHkDlLHXQUYyQDkDpwkGuNzcMm0ceaN/LcPSGXJfiqeX56JxG/D/uUS39/B5AuEwZcK6As1J8g4sor69545UO6Zw+KPQgxxsQmYjmqMEJsdmrJCjFXrsx8JqF7WaVkuG80PlsWUK79QEgR3bcQ0nyfBortk07bKMHcZ35skTY6qzrPGorYewMWQgW7ai29WoD38r29USg8yQ3U2bhhWvFl37eFF9YTwinIwhxogysj2bI79io89Y7VyxlN7DU/n8mQJ60i8cjI/FYto8urP0xwEWSp12oOSmIna5dv1Gr7In4RIe6Aw1q0CXcDf9t9h7QQN5ITJWn3nLGev7fAQPS19uuHLBSlg/oUnWlfDmYUGgyet6+vRl1jbMMuoA0ZXO3Ll/QtTH1oVj29pRnpB2uZs9gQXV/nOD5I59sKFWbhIUffkaNQL7wGNZlKoQGOZVi7jO3Qhz4L/dqJKsEHjgAs+24BrkcJmvkQJbqI0aOXolLK/UdCD9mI4IJdtorvhNLuBa5KVLBkiBBNxKYAJcWS5CkREIS4XdcWAKUpOo9I951VnvK0yLIoLqeI6VUItSznjK6B2tuQG/8/7nHfE+YPP3bp8m8U0=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(10290500003)(186003)(86362001)(76116006)(8936002)(7416002)(7366002)(122000001)(38100700002)(71200400001)(82960400001)(2906002)(82950400001)(38070700005)(8676002)(55016002)(26005)(7406005)(53546011)(4326008)(921005)(64756008)(66446008)(66556008)(66476007)(66946007)(6506007)(316002)(52536014)(33656002)(8990500004)(110136005)(7696005)(9686003)(5660300002)(54906003)(508600001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?SCt0SWxWdDhBK1BQSUR2TUFWS1dNcWpkb2htRDJLT3VSTmp1aEdWaDRFZzFP?=
 =?utf-8?B?aUtuSW1QTEJSR3AzOTV4bFhWZ0RBOGkveUxzMWk3Q0pKRk52cmpwWW0wTTEz?=
 =?utf-8?B?MlNMYVFPYVk3bEJ1N1puSXJGSjJvWVZXS1dWL2U0T0RwYWJidmJMMDlEd2xj?=
 =?utf-8?B?TnJLREpxN2VJWE1vWTlJS1lkWDhMTExzd1BqbjFMRFFLSUQweGZpZmFXZllL?=
 =?utf-8?B?OStUY3VLc2h0STZFUWt4MG1Rb2F6MDlXdyttT0pHb28yWElzT3lPMEJobnhG?=
 =?utf-8?B?SjQzWWtVU1FwZkNuSXpvYXkwRWlOMUVFRDBsNG5oQ1FnTi90VXdYSitoQml4?=
 =?utf-8?B?TkNoZmZDbVpJWS81Q3BVTmxLZ0RpY0JqR1pjUjRPa29vT3RyYnZhcHBZSTJj?=
 =?utf-8?B?cVZMM2dOU1dmYnJzVFhFUDJETlBoWWdYYVU4Yml0TDFFS21uU094L21CaktI?=
 =?utf-8?B?dFYrSTkvd0I0dVMxL3JJZE9DRVlCeXNzck1ZRlFjaHBVVHhLaDE2ME5FN2hl?=
 =?utf-8?B?ZEx4eCtEOXNjcEN4RGFBQnNVSHM1cGMyN2VEajgxdW1ucGNPLzNONXdlRG1m?=
 =?utf-8?B?MUN1MnE1S2xISmx1bFl0YldCREFEM2UxSFhrM0R1M1IzR3VRSVRaN1czUzlR?=
 =?utf-8?B?TldwejRPcXNyQWdxNmVWeEdNcHJGcEt4dWRmQWljRFc2WXAxMkJFM2NOU2Mw?=
 =?utf-8?B?b2ZhdDJHU3gzZ1dLMll4b3JlYlg0NlRMYlV6UW5odVROL2hVdlZIVFdkdnVn?=
 =?utf-8?B?QUtIODVrTXVhd0hTVFVob0p1N0gycjRvSUJDTjdiWDZVZURUcGFlNkdFYzkz?=
 =?utf-8?B?R1JEVjRPWDVQa1Z4eVUzNCs2OFlJdWd6UXVyWWR5b2I1T0poWFo4cEZwMXJQ?=
 =?utf-8?B?YkdRby9pQTczUmdCd1NBcE95Um96SytST1NQWUtQaENwRkRReG1paTF5M3NG?=
 =?utf-8?B?VFhFSFlvSVAwMUVsUUNPNUF0NStDcGdNWVNvdjVHcWJaV25vdC9WdU4ySkUx?=
 =?utf-8?B?SFFnMXJZZDFlc01sTGk5YUJ3UWRqbHIvOUYxMXN0MW8xai9SaW5oblVWTXly?=
 =?utf-8?B?ZGg4cnVYKytXR1BUSVVGdndPL2ZyY1ZUdXFibWdHMkt0QUFBenAxYVZzdnRP?=
 =?utf-8?B?SmxwOWh5K1p5blQvZlFYZnN5TStHb0dnV3hNVFB3b1JGOEQvbFFzOXpOUlpa?=
 =?utf-8?B?S1IrTU1ROVZ6U0ExSXhhMTNCM0xyN00zRkN6d0pHUi9LOEl6YlI3VjNKVjdK?=
 =?utf-8?B?SDE5bnRGa01DQS92dGxET0p0YVJMNCtMYm9OOTVFa3dWZEpFeGtkbHVPQmhw?=
 =?utf-8?B?cnNVUVNYdFdXWW9SNzQ0ajFQWUpXc2Y5YmdNVUVpWW9QREozVlRyL3YyUHFW?=
 =?utf-8?B?cU1kVHFFRFJ3QWdZQm1WTFlGeU8rcWM4di9DTUc2dW5qMmJYZFZ6L2h4aVVM?=
 =?utf-8?B?RUx1VmVuY2VqODB4QjBoTEh3N0ZHUENkWjdHVktmcGVXV3F6TkhmRDNra1ht?=
 =?utf-8?B?UUtLQ0x5dGFJYVE5ejRaMnlKbVdKUzN4MGh1N3hIV3NVNDRLbEYvT2s1SXE5?=
 =?utf-8?B?bFVTOG1jQklFZkIrRlNFVkhQeldLeSszMXpUc2xwdk9KMThiYk8xL3ZxTmVR?=
 =?utf-8?B?bTBkelJ6aDVubW1GcmY2R0ZOSUIrcmJaM21pMFQzcmI1WHQwdUEySzBpUmZR?=
 =?utf-8?B?ZGJlTFFSdDBaMXlFMlhyZHFQSnhyV1dKUGZpdVBtQUt0Q1IxS1JlaXpQWkJH?=
 =?utf-8?Q?/KsbFS+FFu/eogW6zR47LkfqSJoFw1GvK9smoaR?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a10f30f-c31a-4b0c-12e9-08d96e2caf25
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2021 16:14:04.3146
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Yw5Wps3Lv7L2sgCGbAiKeaqoXpEbacvM3u28PlfRsJjUhekwDNXnedm/jETY+2tdzXKwgkE0gULqGhseUsLO+Qp5f6JA9+HASQ+PuOVd7Os=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0287

RnJvbTogVGlhbnl1IExhbiA8bHR5a2VybmVsQGdtYWlsLmNvbT4gU2VudDogVGh1cnNkYXksIFNl
cHRlbWJlciAyLCAyMDIxIDY6MzYgQU0NCj4gDQo+IE9uIDkvMi8yMDIxIDg6MjMgQU0sIE1pY2hh
ZWwgS2VsbGV5IHdyb3RlOg0KPiA+PiArCX0gZWxzZSB7DQo+ID4+ICsJCXBhZ2VzX3dyYXBhcm91
bmQgPSBrY2FsbG9jKHBhZ2VfY250ICogMiAtIDEsDQo+ID4+ICsJCQkJCSAgIHNpemVvZihzdHJ1
Y3QgcGFnZSAqKSwNCj4gPj4gKwkJCQkJICAgR0ZQX0tFUk5FTCk7DQo+ID4+ICsNCj4gPj4gKwkJ
cGFnZXNfd3JhcGFyb3VuZFswXSA9IHBhZ2VzOw0KPiA+PiArCQlmb3IgKGkgPSAwOyBpIDwgMiAq
IChwYWdlX2NudCAtIDEpOyBpKyspDQo+ID4+ICsJCQlwYWdlc193cmFwYXJvdW5kW2kgKyAxXSA9
DQo+ID4+ICsJCQkJJnBhZ2VzW2kgJSAocGFnZV9jbnQgLSAxKSArIDFdOw0KPiA+PiArDQo+ID4+
ICsJCXJpbmdfaW5mby0+cmluZ19idWZmZXIgPSAoc3RydWN0IGh2X3JpbmdfYnVmZmVyICopDQo+
ID4+ICsJCQl2bWFwKHBhZ2VzX3dyYXBhcm91bmQsIHBhZ2VfY250ICogMiAtIDEsIFZNX01BUCwN
Cj4gPj4gKwkJCQlQQUdFX0tFUk5FTCk7DQo+ID4+ICsNCj4gPj4gKwkJa2ZyZWUocGFnZXNfd3Jh
cGFyb3VuZCk7DQo+ID4+ICsJCWlmICghcmluZ19pbmZvLT5yaW5nX2J1ZmZlcikNCj4gPj4gKwkJ
CXJldHVybiAtRU5PTUVNOw0KPiA+PiArCX0NCj4gPiBXaXRoIHRoaXMgcGF0Y2gsIHRoZSBjb2Rl
IGlzIGEgYmlnICJpZiIgc3RhdGVtZW50IHdpdGggdHdvIGhhbHZlcyAtLSBvbmUNCj4gPiB3aGVu
IFNOUCBpc29sYXRpb24gaXMgaW4gZWZmZWN0LCBhbmQgdGhlIG90aGVyIHdoZW4gbm90LiAgVGhl
IFNOUCBpc29sYXRpb24NCj4gPiBjYXNlIGRvZXMgdGhlIHdvcmsgdXNpbmcgUEZOcyB3aXRoIHRo
ZSBzaGFyZWRfZ3BhX2JvdW5kYXJ5IGFkZGVkLA0KPiA+IHdoaWxlIHRoZSBvdGhlciBjYXNlIGRv
ZXMgdGhlIHNhbWUgd29yayBidXQgdXNpbmcgc3RydWN0IHBhZ2UuICBQZXJoYXBzDQo+ID4gSSdt
IG1pc3Npbmcgc29tZXRoaW5nLCBidXQgY2FuIGJvdGggaGFsdmVzIGJlIGNvbWJpbmVkIGFuZCBh
bHdheXMNCj4gPiBkbyB0aGUgd29yayB1c2luZyBQRk5zPyAgVGhlIG9ubHkgZGlmZmVyZW5jZSBp
cyB3aGV0aGVyIHRvIGFkZCB0aGUNCj4gPiBzaGFyZWRfZ3BhX2JvdW5kYXJ5LCBhbmQgd2hldGhl
ciB0byB6ZXJvIHRoZSBtZW1vcnkgd2hlbiBkb25lLg0KPiA+IFNvIGdldCB0aGUgc3RhcnRpbmcg
UEZOLCB0aGVuIGhhdmUgYW4gImlmIiBzdGF0ZW1lbnQgZm9yIHdoZXRoZXIgdG8NCj4gPiBhZGQg
dGhlIHNoYXJlZF9ncGFfYm91bmRhcnkuICBUaGVuIGV2ZXJ5dGhpbmcgZWxzZSBpcyB0aGUgc2Ft
ZS4NCj4gPiBBdCB0aGUgZW5kLCB1c2UgYW4gImlmIiBzdGF0ZW1lbnQgdG8gZGVjaWRlIHdoZXRo
ZXIgdG8gemVybyB0aGUNCj4gPiBtZW1vcnkuICBJdCB3b3VsZCByZWFsbHkgYmUgYmV0dGVyIHRv
IGhhdmUgdGhlIGxvZ2ljIGluIHRoaXMgYWxnb3JpdGhtDQo+ID4gY29kZWQgb25seSBvbmNlLg0K
PiA+DQo+IA0KPiBIaSBNaWNoYWVsOg0KPiAJSSBoYXZlIHRyaWVkIHRoaXMgYmVmb3JlLiBCdXQg
dm1hcF9wZm4oKSBvbmx5IHdvcmtzIGZvciB0aG9zZSBwZm5zIG91dA0KPiBvZiBub3JtYWwgbWVt
b3J5LiBQbGVhc2Ugc2VlIHZtYXBfcGZuX2FwcGx5KCkgZm9yIGRldGFpbCBhbmQNCj4gcmV0dXJu
IGVycm9yIHdoZW4gdGhlIFBGTiBpcyB2YWxpZC4NCj4gDQoNCkluZGVlZC4gIFRoaXMgdGllcyBp
bnRvIHRoZSBkaXNjdXNzaW9uIHdpdGggQ2hyaXN0b3BoIGFib3V0IGNvbWluZyB1cA0Kd2l0aCBn
ZW5lcmFsaXplZCBoZWxwZXIgZnVuY3Rpb25zIHRvIGFzc2lzdCBpbiBoYW5kbGluZyB0aGUNCnNo
YXJlZF9ncGFfYm91bmRhcnkuICAgSGF2aW5nIGEgc2luZ2xlIGltcGxlbWVudGF0aW9uIGhlcmUg
aW4NCmh2X3JpbmdidWZmZXJfaW5pdCgpIHdvdWxkIGJlIGEgZ29vZCBnb2FsIGFzIHdlbGwuDQoN
Ck1pY2hhZWwNCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:15:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:15:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177409.322798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNW-00015z-Fu; Thu, 02 Sep 2021 16:15:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177409.322798; Thu, 02 Sep 2021 16:15:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNW-00015s-CW; Thu, 02 Sep 2021 16:15:50 +0000
Received: by outflank-mailman (input) for mailman id 177409;
 Thu, 02 Sep 2021 16:15:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNU-00015i-QN
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:15:48 +0000
Received: from mail-wr1-x432.google.com (unknown [2a00:1450:4864:20::432])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0586f74-3c3b-45c3-9d4d-5e8b10b51373;
 Thu, 02 Sep 2021 16:15:47 +0000 (UTC)
Received: by mail-wr1-x432.google.com with SMTP id b10so3884094wru.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:15:47 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 j18sm2355853wrd.56.2021.09.02.09.15.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:15:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: d0586f74-3c3b-45c3-9d4d-5e8b10b51373
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=Cx9tmtcMe/yqb/L4FxX6s6fgCGjzmE2xzXTNn5n4WRY=;
        b=mcdkcgUkPmIymlgoHwDrJpvE9KHw9dvSG2MlkuM7+LMu3zkIYpqtCeCGgcISYDoKCb
         8696YyJt2CPh12bbX7QpwDA8IAMyJN7taR/RiiyUq4y4j+fSACZ5oqQ08yGPg+QlMR0S
         hitbbr51iTLrvFSzYtCDWEBhD9WU3oObbn7IgCnxkcpgU8/Bpb/PRbBi6oyJFhSCPB/4
         T4eJ4wL0xOuOcg0FizPKZv0AhuncICKZC1qKdxIbn5OpaC/dbV050AWnqohvIXKAkBb9
         MuFoocUeCUvdcKKf//7Z32DJC9YmwC2EKSjtLp0AMfq+R5Wwnd7FM6PpEyUZ4hvUxjzc
         2tig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :mime-version:content-transfer-encoding;
        bh=Cx9tmtcMe/yqb/L4FxX6s6fgCGjzmE2xzXTNn5n4WRY=;
        b=OwocEGUbclqWd/677oejQXsrZXjgc5UQ+9J8GvdslFFo/Vtv6tLZ6c5NUFLE3+hImw
         ckHmp67mWdqSqLEibxLjf5CCDhO51d08PAkBKWlodtkcCQ0GB2mSPb48nuneKm6vORtt
         Exhc/64bLOIUgZWlHbwUJNQsQoyHXCnD1uvGCHm6i1fR7kwPPq8D/UXNvw6WDySyzofL
         NMpBhkoZBmXZKK4KrRw2QcfygjvXixEKdshDkMhzMNhVs2fxd5Imgku2XtmrB+zN+DY5
         pnKZnG+GpDtQNF6GUT/uoQUiB7gMfj4sYO/5Lju21+VN7uQHq5gDkuNGGNa9514YvSLl
         MF9A==
X-Gm-Message-State: AOAM532VeWCOR6jst8TMpFBX3Crt5lC8mlUjGzbELNXKEFX7FDceZgT+
	vAv9dya/i53k0ZwTNDlAuiY=
X-Google-Smtp-Source: ABdhPJwMW19l1RT5uyseQGoPrvV+9UXH/fieGpgiPEuYOlcUopWj6VFIU2AjhhvSZ8tr8ff4GeIa0w==
X-Received: by 2002:adf:8144:: with SMTP id 62mr4741623wrm.144.1630599347027;
        Thu, 02 Sep 2021 09:15:47 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 00/30] accel: Move has_work() from SysemuCPUOps to AccelOpsClass
Date: Thu,  2 Sep 2021 18:15:13 +0200
Message-Id: <20210902161543.417092-1-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi,=0D
=0D
CPU has_work() is a per-accelerator handler. This series=0D
- explicit the KVM / WHPX implementations=0D
- moves TCG implementations in AccelOpsClass=0D
- explicit missing implementations (returning 'false').=0D
=0D
Since v2:=0D
- Full rewrite, no more RFC.=0D
=0D
Supersedes: <20210304222323.1954755-1-f4bug@amsat.org>=0D
"cpu: Move CPUClass::has_work() to TCGCPUOps"=0D
=0D
Philippe Mathieu-Daud=C3=A9 (30):=0D
  accel/tcg: Restrict cpu_handle_halt() to sysemu=0D
  hw/core: Restrict cpu_has_work() to sysemu=0D
  hw/core: Un-inline cpu_has_work()=0D
  sysemu: Introduce AccelOpsClass::has_work()=0D
  accel/kvm: Implement AccelOpsClass::has_work()=0D
  accel/whpx: Implement AccelOpsClass::has_work()=0D
  accel/tcg: Implement AccelOpsClass::has_work() as stub=0D
  target/alpha: Restrict has_work() handler to sysemu and TCG=0D
  target/arm: Restrict has_work() handler to sysemu and TCG=0D
  target/avr: Restrict has_work() handler to sysemu and TCG=0D
  target/cris: Restrict has_work() handler to sysemu and TCG=0D
  target/hexagon: Remove unused has_work() handler=0D
  target/hppa: Restrict has_work() handler to sysemu and TCG=0D
  target/i386: Restrict has_work() handler to sysemu and TCG=0D
  target/m68k: Restrict has_work() handler to sysemu and TCG=0D
  target/microblaze: Restrict has_work() handler to sysemu and TCG=0D
  target/mips: Restrict has_work() handler to sysemu and TCG=0D
  target/nios2: Restrict has_work() handler to sysemu and TCG=0D
  target/openrisc: Restrict has_work() handler to sysemu and TCG=0D
  target/ppc: Restrict has_work() handler to sysemu and TCG=0D
  target/ppc: Introduce PowerPCCPUClass::has_work()=0D
  target/ppc: Simplify has_work() handlers=0D
  target/riscv: Restrict has_work() handler to sysemu and TCG=0D
  target/rx: Restrict has_work() handler to sysemu and TCG=0D
  target/s390x: Restrict has_work() handler to sysemu and TCG=0D
  target/sh4: Restrict has_work() handler to sysemu and TCG=0D
  target/sparc: Restrict has_work() handler to sysemu and TCG=0D
  target/tricore: Restrict has_work() handler to sysemu and TCG=0D
  target/xtensa: Restrict has_work() handler to sysemu and TCG=0D
  accel: Add missing AccelOpsClass::has_work() and drop SysemuCPUOps one=0D
=0D
 include/hw/core/cpu.h             |  28 +--=0D
 include/hw/core/tcg-cpu-ops.h     |   4 +=0D
 include/sysemu/accel-ops.h        |   5 +=0D
 target/ppc/cpu-qom.h              |   3 +=0D
 accel/hvf/hvf-accel-ops.c         |   6 +=0D
 accel/kvm/kvm-accel-ops.c         |   6 +=0D
 accel/qtest/qtest.c               |   6 +=0D
 accel/tcg/cpu-exec.c              |   8 +-=0D
 accel/tcg/tcg-accel-ops.c         |  12 ++=0D
 accel/xen/xen-all.c               |   6 +=0D
 hw/core/cpu-common.c              |   6 -=0D
 softmmu/cpus.c                    |  10 +-=0D
 target/alpha/cpu.c                |   4 +-=0D
 target/arm/cpu.c                  |   7 +-=0D
 target/avr/cpu.c                  |   4 +-=0D
 target/cris/cpu.c                 |   4 +-=0D
 target/hexagon/cpu.c              |   6 -=0D
 target/hppa/cpu.c                 |   4 +-=0D
 target/i386/cpu.c                 |   6 -=0D
 target/i386/hax/hax-accel-ops.c   |   6 +=0D
 target/i386/nvmm/nvmm-accel-ops.c |   6 +=0D
 target/i386/tcg/tcg-cpu.c         |   8 +-=0D
 target/i386/whpx/whpx-accel-ops.c |   6 +=0D
 target/m68k/cpu.c                 |   4 +-=0D
 target/microblaze/cpu.c           |  10 +-=0D
 target/mips/cpu.c                 |   4 +-=0D
 target/nios2/cpu.c                |   4 +-=0D
 target/openrisc/cpu.c             |   4 +-=0D
 target/ppc/cpu_init.c             | 324 +++++++++++++++---------------=0D
 target/riscv/cpu.c                |   8 +-=0D
 target/rx/cpu.c                   |   4 +-=0D
 target/s390x/cpu.c                |   4 +-=0D
 target/sh4/cpu.c                  |   7 +-=0D
 target/sparc/cpu.c                |   4 +-=0D
 target/tricore/cpu.c              |   6 +-=0D
 target/xtensa/cpu.c               |  16 +-=0D
 36 files changed, 321 insertions(+), 239 deletions(-)=0D
=0D
-- =0D
2.31.1=0D
=0D


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:15:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:15:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177410.322810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNb-0001P0-Om; Thu, 02 Sep 2021 16:15:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177410.322810; Thu, 02 Sep 2021 16:15:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNb-0001On-Kk; Thu, 02 Sep 2021 16:15:55 +0000
Received: by outflank-mailman (input) for mailman id 177410;
 Thu, 02 Sep 2021 16:15:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNa-0001O1-ER
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:15:54 +0000
Received: from mail-wr1-x434.google.com (unknown [2a00:1450:4864:20::434])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23b39658-0c63-477d-af8b-3222810181c7;
 Thu, 02 Sep 2021 16:15:53 +0000 (UTC)
Received: by mail-wr1-x434.google.com with SMTP id z4so3826952wrr.6
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:15:53 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 j17sm2187403wrh.67.2021.09.02.09.15.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:15:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 23b39658-0c63-477d-af8b-3222810181c7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=/9vCQQZfSaTpKBQMegFXjFo179Y+24zi0HZDvs7SgMU=;
        b=nvF/c2Wc4Z15cqg2fwhp1NDrfdcdFXpuAmJa1KWBtzQQ/EXWFrb73xtrx8OZfbtAYH
         +ID89t8hrTCsxgu2Z9TktIv56PS86hvs5RilCbo/0PLhtchmK7Btfa7sZb3B2zUVbiHv
         /HU6lzJWGffGyztr061l6mMVTzRO0XEqa1r0BVkwLvpIuHEZFGdUMQyYCTDsiXNgmkvC
         mzGCP4B8dDeCO3p+Tu6JIdorMuA2pmQP2RRlgrSmz9a3spLyNyj6mRRUHHOVQRLc9UFi
         tO8C8CpGWtVF4XqV+ujfpe9fj5HRkBRHkHwyv6P8VevMMXH157Vbk0qwXp550NWCjx06
         CBMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=/9vCQQZfSaTpKBQMegFXjFo179Y+24zi0HZDvs7SgMU=;
        b=WL3BQx8x1N6N3//lKXNlq44llQPFA0y0sZtxbi1fmBnxXbUDjLIEgJINkdGIkSwOwX
         3kjySHcaFGGzF8WA/XcdJD0LdyfYFoMQgX1Ysn7IkUwk9/MXQxGf3gDMR6m3e4qUCm4t
         V5cVeV4WwY4zplXWk503JP05jSo1F6bCe/LEVS7GFn+ibUK5uVug5XqHzZvYtCaAHryt
         nA9taw61VwhQwkSNP6CxwquPmCOFd9DQnohy+gjzHL+HOzox1hjzspTsF/wF0jUjNXQJ
         Gy46xURJvwYxfCrErFBZkPQuYQHYTTi7O7GI77ayGenfNAzruqMFd9FnyTv142SsgMyk
         REOQ==
X-Gm-Message-State: AOAM533DWCk/Kz9hxbV9Zt/ADqILugl7KLDKz+/gK44zXFXWnYzj3ltP
	48d6crHEjBiKdzK2Ms1AThU=
X-Google-Smtp-Source: ABdhPJxxdNDVWzqPqP9AvO+fc8KPo1qKTWTpE1Z3SovcisiYMadm8391Ql4b1VZQyA/+Rjg0YRdLPg==
X-Received: by 2002:a05:6000:186:: with SMTP id p6mr4881125wrx.210.1630599352772;
        Thu, 02 Sep 2021 09:15:52 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 01/30] accel/tcg: Restrict cpu_handle_halt() to sysemu
Date: Thu,  2 Sep 2021 18:15:14 +0200
Message-Id: <20210902161543.417092-2-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Commit 372579427a5 ("tcg: enable thread-per-vCPU") added the following
comment describing EXCP_HALTED in qemu_tcg_cpu_thread_fn():

    case EXCP_HALTED:
         /* during start-up the vCPU is reset and the thread is
          * kicked several times. If we don't ensure we go back
          * to sleep in the halted state we won't cleanly
          * start-up when the vCPU is enabled.
          *
          * cpu->halted should ensure we sleep in wait_io_event
          */
         g_assert(cpu->halted);
         break;

qemu_wait_io_event() is sysemu-specific, so we can restrict the
cpu_handle_halt() call in cpu_exec() to system emulation.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 accel/tcg/cpu-exec.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 7a6dd9049f0..6b61262b151 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -586,10 +586,11 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
     return;
 }
 
+#ifndef CONFIG_USER_ONLY
 static inline bool cpu_handle_halt(CPUState *cpu)
 {
     if (cpu->halted) {
-#if defined(TARGET_I386) && !defined(CONFIG_USER_ONLY)
+#if defined(TARGET_I386)
         if (cpu->interrupt_request & CPU_INTERRUPT_POLL) {
             X86CPU *x86_cpu = X86_CPU(cpu);
             qemu_mutex_lock_iothread();
@@ -597,7 +598,7 @@ static inline bool cpu_handle_halt(CPUState *cpu)
             cpu_reset_interrupt(cpu, CPU_INTERRUPT_POLL);
             qemu_mutex_unlock_iothread();
         }
-#endif
+#endif /* TARGET_I386 */
         if (!cpu_has_work(cpu)) {
             return true;
         }
@@ -607,6 +608,7 @@ static inline bool cpu_handle_halt(CPUState *cpu)
 
     return false;
 }
+#endif /* !CONFIG_USER_ONLY */
 
 static inline void cpu_handle_debug_exception(CPUState *cpu)
 {
@@ -865,9 +867,11 @@ int cpu_exec(CPUState *cpu)
     /* replay_interrupt may need current_cpu */
     current_cpu = cpu;
 
+#ifndef CONFIG_USER_ONLY
     if (cpu_handle_halt(cpu)) {
         return EXCP_HALTED;
     }
+#endif
 
     rcu_read_lock();
 
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177411.322820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNh-0001kT-6Y; Thu, 02 Sep 2021 16:16:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177411.322820; Thu, 02 Sep 2021 16:16:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNh-0001kK-3Z; Thu, 02 Sep 2021 16:16:01 +0000
Received: by outflank-mailman (input) for mailman id 177411;
 Thu, 02 Sep 2021 16:16:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNg-0001jd-F9
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:00 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3e79055f-777a-4214-9dcd-58dbc5b4c0b9;
 Thu, 02 Sep 2021 16:15:59 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id z4so3827438wrr.6
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:15:59 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 x9sm1939663wmi.30.2021.09.02.09.15.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:15:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 3e79055f-777a-4214-9dcd-58dbc5b4c0b9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=IALuJS79v15vDPvWp0gEXOFVB8QcuCBujL3iQm4WVXc=;
        b=lyX2iDyZ4fZxIDfBFZ3tyjvTVxQHi5/DKPStvMf8XLdWBZf2Z4YEuBxLZ6HKfP2S0S
         9fzfqOWFn0kWTui3s6TURapTU3z+GQ8v8hJn1LfoSFOHPkdm35vCxTfT+cYkW1nrugty
         mCqdxM8mrNR5MLebQDolGLEo/7Lkgdj2gwYMGjuH7g4aAADt1Bq23c52IZGz+2AB+NL8
         GZ3E3+ftKA9QL43iXqO6Ak9f9Icd3LYBSFwdaV1YhiiVZy5M0xyjWQ/L7Ym3RUz2AJov
         QL67c+ediBlqlyYZ/UNrRKxOjNTuB98lMijtK7yEZi/9rGhzXp+Rvgvey6qwTDyAgy4I
         6qHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=IALuJS79v15vDPvWp0gEXOFVB8QcuCBujL3iQm4WVXc=;
        b=jA6rkLNq5onNhK18PKg4Jguw3KjL+xa4I351/p9hYDsaONywvLh2DiMpzoFfqfspym
         OdHGO7ValOFLZLWo2NdolFakqBxgmXArNPRqupE4g6hXetyFbh8Wzx7Bv35Un/P85jDi
         Mdpaa+ZLRS0f1ChnINBKNhyP5Zz9Djv9S5ucAocSSdnHLASAxURT8x0CDnzRRd6Tn2JP
         nh9DEE3HMgj2D8pBYQetxXaQewemPAeGfIYEGdGWe8yNdWYGROhe2NlnFVVsM83bY7Kf
         m8ybWyQ6BpgbbqwF5vypNql3QBVtm6Jw1+gZasaTQ6pHTSVjg0i8t/O43+CmhZpfu2P0
         LuBw==
X-Gm-Message-State: AOAM532dP8BsRnishddRUrFIICPTR4zitj2XbhoyxCUtU5ivcuarChxc
	dSLI/oa3b9uFpPsACSka9QNn9ub3IvE=
X-Google-Smtp-Source: ABdhPJy7JEkr/3dMu4L49Lw+gPGY8gHw7yAaZhdIf87oRC7v/4j/f/YWrayp4BQC2KvwiDQbjRMXEg==
X-Received: by 2002:adf:dbd0:: with SMTP id e16mr4727843wrj.402.1630599358916;
        Thu, 02 Sep 2021 09:15:58 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 02/30] hw/core: Restrict cpu_has_work() to sysemu
Date: Thu,  2 Sep 2021 18:15:15 +0200
Message-Id: <20210902161543.417092-3-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

cpu_has_work() is only called from system emulation code.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 include/hw/core/cpu.h | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index bc864564cee..2bd563e221f 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -538,6 +538,22 @@ enum CPUDumpFlags {
 void cpu_dump_state(CPUState *cpu, FILE *f, int flags);
 
 #ifndef CONFIG_USER_ONLY
+/**
+ * cpu_has_work:
+ * @cpu: The vCPU to check.
+ *
+ * Checks whether the CPU has work to do.
+ *
+ * Returns: %true if the CPU has work, %false otherwise.
+ */
+static inline bool cpu_has_work(CPUState *cpu)
+{
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+
+    g_assert(cc->has_work);
+    return cc->has_work(cpu);
+}
+
 /**
  * cpu_get_phys_page_attrs_debug:
  * @cpu: The CPU to obtain the physical page address for.
@@ -636,22 +652,6 @@ CPUState *cpu_create(const char *typename);
  */
 const char *parse_cpu_option(const char *cpu_option);
 
-/**
- * cpu_has_work:
- * @cpu: The vCPU to check.
- *
- * Checks whether the CPU has work to do.
- *
- * Returns: %true if the CPU has work, %false otherwise.
- */
-static inline bool cpu_has_work(CPUState *cpu)
-{
-    CPUClass *cc = CPU_GET_CLASS(cpu);
-
-    g_assert(cc->has_work);
-    return cc->has_work(cpu);
-}
-
 /**
  * qemu_cpu_is_self:
  * @cpu: The vCPU to check against.
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177413.322832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNn-0002Cm-Hz; Thu, 02 Sep 2021 16:16:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177413.322832; Thu, 02 Sep 2021 16:16:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNn-0002Ce-Cv; Thu, 02 Sep 2021 16:16:07 +0000
Received: by outflank-mailman (input) for mailman id 177413;
 Thu, 02 Sep 2021 16:16:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNm-00029c-8F
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:06 +0000
Received: from mail-wm1-x330.google.com (unknown [2a00:1450:4864:20::330])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a77c98f1-6fdb-4048-93e9-834adadafd23;
 Thu, 02 Sep 2021 16:16:05 +0000 (UTC)
Received: by mail-wm1-x330.google.com with SMTP id
 u26-20020a05600c441a00b002f66b2d8603so1866829wmn.4
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:05 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 p5sm2438468wrd.25.2021.09.02.09.16.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: a77c98f1-6fdb-4048-93e9-834adadafd23
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=wV14RcZMLIQqRNfbc2l4krY7RhS99j63lxSZtHhlqHE=;
        b=EKqirEKnM17VK0zDlsK2zMeCBYwgcIwGau/nM2ojAXQ1rvdOXkyoVbGuyEtcnzeBcf
         xImFuE98afvxno05q0YdC27nz6NANS7cu771VPDYoClXWYNKBPbXOfx9/ySz5LCrUECJ
         Q4xIJxN/kF3No93cpZwlCIAaJ6XH09BodtAvBDBdkeBdwCBDaphPwkcvpVDIxHiZ6ysU
         O+OUGJNK0KyuMps/0T4ys/RLGMV8UZ8emOYDNmBvg+JTSjUog7qhAQcBWa/t3lzLUf2T
         odXtoH2/j2F/4nd8pGQvQwtB64/pnwk36AxB7fddw33eEJVy5u1yikEgzNZHCoqMgJ3i
         wCuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=wV14RcZMLIQqRNfbc2l4krY7RhS99j63lxSZtHhlqHE=;
        b=B1/JoJulq2t4OsAb+f+Iwcof8/8nogzJjSsSeYK+ndN9RIbLLv7MqOObXsP3zigJPC
         QT6sZ7X9Gsx60qQjoIvTtzUllSTGwUxnqlXdrAPEJR9qiMNt7ksMXj72JbHAvkG5mIrG
         nfQffqdGhX9vmV+2DCvXd+wwYeUWI+geadKt5ZYlmwm7neBWVeXD40smnHAFotR6Iu8A
         vLWrUsmI/dXI9HAkILlkqCDzZlfemotf4KeC7PfD2izAyKXKnCjsn9dHhtw/epHv/H4J
         TWk/dOeYDrnaGH9JuKv7IwQ6eyb+YrO193FQE1uX/MyyCKhGGQeKW7HEzEbpdBt+wYMA
         kDzw==
X-Gm-Message-State: AOAM533sKL4jzmZ4JRxrrrQHFzUNx67241OAsPzhJ77CsJ9OPCSuNxaA
	TYZkGH9xZmE6gd74e2Htb+k=
X-Google-Smtp-Source: ABdhPJw7/6UEiBwptJyfAbxmtA0JM7jALL+mTrdaJbBQ6pe56ORuZs2k15EQns1EFPDQ4opAGTbfXw==
X-Received: by 2002:a05:600c:2101:: with SMTP id u1mr4147345wml.45.1630599364755;
        Thu, 02 Sep 2021 09:16:04 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 03/30] hw/core: Un-inline cpu_has_work()
Date: Thu,  2 Sep 2021 18:15:16 +0200
Message-Id: <20210902161543.417092-4-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We want to make cpu_has_work() per-accelerator. Only declare its
prototype and move its definition to softmmu/cpus.c.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 include/hw/core/cpu.h | 8 +-------
 softmmu/cpus.c        | 8 ++++++++
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 2bd563e221f..e2dd171a13f 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -546,13 +546,7 @@ void cpu_dump_state(CPUState *cpu, FILE *f, int flags);
  *
  * Returns: %true if the CPU has work, %false otherwise.
  */
-static inline bool cpu_has_work(CPUState *cpu)
-{
-    CPUClass *cc = CPU_GET_CLASS(cpu);
-
-    g_assert(cc->has_work);
-    return cc->has_work(cpu);
-}
+bool cpu_has_work(CPUState *cpu);
 
 /**
  * cpu_get_phys_page_attrs_debug:
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 071085f840b..7e2cb2c571b 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -251,6 +251,14 @@ void cpu_interrupt(CPUState *cpu, int mask)
     }
 }
 
+bool cpu_has_work(CPUState *cpu)
+{
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+
+    g_assert(cc->has_work);
+    return cc->has_work(cpu);
+}
+
 static int do_vm_stop(RunState state, bool send_stop)
 {
     int ret = 0;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177417.322843 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNu-0002n1-Qp; Thu, 02 Sep 2021 16:16:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177417.322843; Thu, 02 Sep 2021 16:16:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpNu-0002mu-LZ; Thu, 02 Sep 2021 16:16:14 +0000
Received: by outflank-mailman (input) for mailman id 177417;
 Thu, 02 Sep 2021 16:16:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNt-00029c-Nz
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:13 +0000
Received: from mail-wr1-x429.google.com (unknown [2a00:1450:4864:20::429])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f5586672-609a-4718-a2be-f5eaf5608fba;
 Thu, 02 Sep 2021 16:16:11 +0000 (UTC)
Received: by mail-wr1-x429.google.com with SMTP id n5so3803786wro.12
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:11 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 l1sm2293844wrb.15.2021.09.02.09.16.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: f5586672-609a-4718-a2be-f5eaf5608fba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=P0fp1qqpF59XMbIIHxComfWKKQIg8VIlTwHVvhBN59A=;
        b=G8+gv7/l02p+3pIOfkKqyIa3SJNJqqucn0S8o/siAkKaxmFM5RO224iTEbB1+wTzyT
         yH9VCpxFWBr0t0GXhQxQRUuO0CaUa3ZzEf3celJhLqjAbDckl1E6mqaq2dCEttikPZRW
         NTgh0ED5VO3rDtoACT4pDd4RK9nfKb9VvyIT09l58j9YQUutJPWiW1YUvmfRQD6MaciO
         hgQld4iTpFhaM+A7V2MEdU+/ZRnuTgXC5fBFAjpnxBcFkFzPQ9ZRGDj/1+tmyCyow8jw
         rBfrG9A0yoltXZKggfPRhQMqdG2T5CPTN8x3VdAjvEoK7uauTkkWkCFDPZbLzkrywDXE
         DO1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=P0fp1qqpF59XMbIIHxComfWKKQIg8VIlTwHVvhBN59A=;
        b=YfdnRZzuD/d1mvZa5Pw6Ez9HDlUz25GZZwQ6iY9ELsuSSEjPrTq3K3aLH4ylk4YNFf
         XAsnyJiw6fVLmTxBHI1u262nKJP5mSZ8fE9tvQUZ4ww3aGgcndnghHZ0J0jQpelINN8P
         cU2PROrYMrxr/tMYDV/NLL7d5Qj+7WrBUkq8xPYqKxGEJCQGRMWQcd14sARcAGB/3TkS
         keSuD/Ui6puLg2O8T2Yk8v2H4g08jA26HisroQl+kB5tRecU3//yKNYNLbUZeQQLBAwS
         9QJNM+zGvoR1W6VJgG3ofMc8gJAmMjv3r/JlxgPN1v4+GPT/84+dt4jk24SrobK4pDgU
         yjXQ==
X-Gm-Message-State: AOAM532osXSa/szmX+gyIMbs0u/RvSaOQlrc+mvqq3+jcnWv9bV95F1Z
	d5QxbobzSFO4viKllEcdgC4=
X-Google-Smtp-Source: ABdhPJxqYVZz8Vtdq+1rVuHAugWlbrJscT37rU3tWfkU/lcfKzRuzJ6gBh5u/Yi5SNrMElUTWFem6A==
X-Received: by 2002:adf:d191:: with SMTP id v17mr4847443wrc.345.1630599370860;
        Thu, 02 Sep 2021 09:16:10 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 04/30] sysemu: Introduce AccelOpsClass::has_work()
Date: Thu,  2 Sep 2021 18:15:17 +0200
Message-Id: <20210902161543.417092-5-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce an accelerator-specific has_work() handler.
Eventually call it from cpu_has_work().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 include/sysemu/accel-ops.h | 5 +++++
 softmmu/cpus.c             | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/include/sysemu/accel-ops.h b/include/sysemu/accel-ops.h
index 032f6979d76..de83f095f20 100644
--- a/include/sysemu/accel-ops.h
+++ b/include/sysemu/accel-ops.h
@@ -31,6 +31,11 @@ struct AccelOpsClass {
     void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL */
     void (*kick_vcpu_thread)(CPUState *cpu);
 
+    /**
+     * @has_work: Callback for checking if there is work to do.
+     */
+    bool (*has_work)(CPUState *cpu);
+
     void (*synchronize_post_reset)(CPUState *cpu);
     void (*synchronize_post_init)(CPUState *cpu);
     void (*synchronize_state)(CPUState *cpu);
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 7e2cb2c571b..2a61dfd6287 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -255,6 +255,9 @@ bool cpu_has_work(CPUState *cpu)
 {
     CPUClass *cc = CPU_GET_CLASS(cpu);
 
+    if (cpus_accel->has_work) {
+        return cpus_accel->has_work(cpu);
+    }
     g_assert(cc->has_work);
     return cc->has_work(cpu);
 }
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177421.322854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpO0-0003Hx-5H; Thu, 02 Sep 2021 16:16:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177421.322854; Thu, 02 Sep 2021 16:16:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpO0-0003Hg-1f; Thu, 02 Sep 2021 16:16:20 +0000
Received: by outflank-mailman (input) for mailman id 177421;
 Thu, 02 Sep 2021 16:16:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpNy-00029c-O4
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:18 +0000
Received: from mail-wm1-x333.google.com (unknown [2a00:1450:4864:20::333])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16e7e53f-64ec-4976-bcd4-b2f74f7f94a1;
 Thu, 02 Sep 2021 16:16:17 +0000 (UTC)
Received: by mail-wm1-x333.google.com with SMTP id
 m25-20020a7bcb99000000b002e751bcb5dbso1731680wmi.5
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:17 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 n3sm2097857wmi.0.2021.09.02.09.16.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 16e7e53f-64ec-4976-bcd4-b2f74f7f94a1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=RYRVLsfl9Ob+8zW8ulpQWg/n0bsy9p4rFkeC3sRFgzg=;
        b=cWSYDAq5Jp0McvzKfoJCBhOUKIrlk33QHDkz+m47acNIlb9JOx3tfp5kB+KXf5dDO/
         w4+OyKHGR2RiqPyFa7V7bzZyYSy6kLYW49zVrZ6Hy8QBCbCerYOKfrGjd0qNffHf9SpP
         r/Kq+8mSo19dNZRUfW1WN64ERD6JetVYvf41b1e1tWlJwcOL4P63Z/Mm9sSwkJyaYePZ
         Jroc0woaa+Fir71b0dp9xMglTvA3smVf5tzo/uq315Y7Ji+qHpf638YaJjs7F9y6JQ5A
         whXUesQqAhIyuS/27l3cznkFQWzDOa9H8TavWnRll7U6TXlgZEUv2l8XPpV83TA8h9zC
         OFCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=RYRVLsfl9Ob+8zW8ulpQWg/n0bsy9p4rFkeC3sRFgzg=;
        b=TIia/RMEQWe6dKwXHmsAI5rbL7+aZx0I+XFNFiWVqv2YHr5C3Ed7xp4daJftkUiXhT
         xHW+Hkieg+8c2HiMb/mUMtHo09g5wcUlcX9D5qDjS1koHJVAuOhaoKY6RP7Ga/M+9Sbp
         WUKrL6VhaMlOjpn83iVoPN5E/259NkvFD0fKsQ16Mrs3WGQrdn97B7O0plVKiTWlCmFG
         KAP3H65Td0huiQgfLw+Y1ImzZhVAl/i7JrTBHbN5wxQTIin+dHcdQxjG4L5rYsOgwsGs
         Azwxm7Ipb+AO0E9LorFYW/YiaYRuGr44ggo4zPtTPredM+1gGe418ZG6DvmSg8XsmnAr
         8ajw==
X-Gm-Message-State: AOAM53320BABRTZYX1ET6LNvaWWLO53oMn/deAadm2BUTOfeav3YFi1P
	nbzt1D9kXVBVP+vKPwzXMNU=
X-Google-Smtp-Source: ABdhPJwPjYjbwvv/6b7qFrtZZvNxWFscX+KmwH8SpAbQsTLfx+OHgnzX3KJXi3XGsfieLbi9LeXPtA==
X-Received: by 2002:a7b:c0c6:: with SMTP id s6mr3953111wmh.161.1630599376931;
        Thu, 02 Sep 2021 09:16:16 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 05/30] accel/kvm: Implement AccelOpsClass::has_work()
Date: Thu,  2 Sep 2021 18:15:18 +0200
Message-Id: <20210902161543.417092-6-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implement KVM has_work() handler in AccelOpsClass and
remove it from cpu_thread_is_idle() since cpu_has_work()
is already called.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 accel/kvm/kvm-accel-ops.c | 6 ++++++
 softmmu/cpus.c            | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/accel/kvm/kvm-accel-ops.c b/accel/kvm/kvm-accel-ops.c
index 7516c67a3f5..6f4d5df3a0d 100644
--- a/accel/kvm/kvm-accel-ops.c
+++ b/accel/kvm/kvm-accel-ops.c
@@ -74,6 +74,11 @@ static void kvm_start_vcpu_thread(CPUState *cpu)
                        cpu, QEMU_THREAD_JOINABLE);
 }
 
+static bool kvm_cpu_has_work(CPUState *cpu)
+{
+    return kvm_halt_in_kernel();
+}
+
 static void kvm_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
@@ -83,6 +88,7 @@ static void kvm_accel_ops_class_init(ObjectClass *oc, void *data)
     ops->synchronize_post_init = kvm_cpu_synchronize_post_init;
     ops->synchronize_state = kvm_cpu_synchronize_state;
     ops->synchronize_pre_loadvm = kvm_cpu_synchronize_pre_loadvm;
+    ops->has_work = kvm_cpu_has_work;
 }
 
 static const TypeInfo kvm_accel_ops_type = {
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 2a61dfd6287..3db7bd4eb4d 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -90,7 +90,7 @@ bool cpu_thread_is_idle(CPUState *cpu)
         return true;
     }
     if (!cpu->halted || cpu_has_work(cpu) ||
-        kvm_halt_in_kernel() || whpx_apic_in_platform()) {
+        whpx_apic_in_platform()) {
         return false;
     }
     return true;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177430.322865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpO9-00041h-Gt; Thu, 02 Sep 2021 16:16:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177430.322865; Thu, 02 Sep 2021 16:16:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpO9-00041V-DA; Thu, 02 Sep 2021 16:16:29 +0000
Received: by outflank-mailman (input) for mailman id 177430;
 Thu, 02 Sep 2021 16:16:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpO8-00029c-OZ
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:28 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d531d647-1b43-46ab-911b-29dfb596f7ed;
 Thu, 02 Sep 2021 16:16:23 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id q14so3833090wrp.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:23 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 k37sm1787439wms.18.2021.09.02.09.16.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: d531d647-1b43-46ab-911b-29dfb596f7ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=TscxpaThIFwXh/P1Ycdn5GNQgbZxc3kDvlQfbrdl29A=;
        b=qoOL8i+0Y9KqXMdm9wOQ3u/uPCdX/SQlDbKgluqhkScnDC2t1qHTT+2MTI//DWFAUT
         NEejCit7VB+NanpfXq6Qc1DSyRxFphKWmmaSUCPbUNDcmTFSjm/7IHDyWLstRl23/liL
         PZvnmlnErsZ1z8G7bMly2o0U/t5YGH3JGm4ZynAQjU//ks9rQoD8imqPSgj5ddgnhNdE
         jy4fS/nmBse8JHRRQZwgmcXXvfWbpfY46jYzcfi3LWel1xENjZOe87hdj9MZ//4GUVRq
         cQn36iCUZBXsphDCe0vsC3FLk68GDc7pyonHjq+49CjqDRjeD5GGI1ltJX+qXIdflDGH
         EuIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=TscxpaThIFwXh/P1Ycdn5GNQgbZxc3kDvlQfbrdl29A=;
        b=tEbKgKab5rC8PlryeZZ8NVtEze8MqmVaH0z+m003ejx8V66VSyqRNZGD/SWmx7W4ls
         okxHtY0MNJlrPWTVUb6pKD5ojuzQCU9K8zsk6fv6JPJx0KOQD+efccbo3+Fe3UKNZdpb
         Xqzc+2O1mWmwQBQpv+V/il1JUUowVM83GGr+bjEJg7TVW7gO13JreeNBlQAdfq/IMISM
         stwo48fJgaMd91R8NewgH+SAOXuBID4HlSzEw+uMoYXR05fmvrAKxN17EmS8GvTvyhex
         WszFqLqWs71w9tWoitTRrtVT63FhlXghdimkhhMXKebAYm6mCqlpVRaxQxlC66qXn6g0
         17nQ==
X-Gm-Message-State: AOAM530gfrH6gWl+jDDehOFXRwPTJoQRX+8692+MxadW1Bm7cQUgt7ui
	oGK8PyEs36sxl7vRAb/XKFo=
X-Google-Smtp-Source: ABdhPJwGGBwl9Y1sjN8okEHWYycyYyvr2xi+NUWJxoO5KxfkCZlVjxWvfmucwDATrcvJW3+aRgOqDw==
X-Received: by 2002:adf:b741:: with SMTP id n1mr4708081wre.354.1630599382926;
        Thu, 02 Sep 2021 09:16:22 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 06/30] accel/whpx: Implement AccelOpsClass::has_work()
Date: Thu,  2 Sep 2021 18:15:19 +0200
Message-Id: <20210902161543.417092-7-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implement WHPX has_work() handler in AccelOpsClass and
remove it from cpu_thread_is_idle() since cpu_has_work()
is already called.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 softmmu/cpus.c                    | 4 +---
 target/i386/whpx/whpx-accel-ops.c | 6 ++++++
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 3db7bd4eb4d..6bce52ce561 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -41,7 +41,6 @@
 #include "sysemu/replay.h"
 #include "sysemu/runstate.h"
 #include "sysemu/cpu-timers.h"
-#include "sysemu/whpx.h"
 #include "hw/boards.h"
 #include "hw/hw.h"
 #include "trace.h"
@@ -89,8 +88,7 @@ bool cpu_thread_is_idle(CPUState *cpu)
     if (cpu_is_stopped(cpu)) {
         return true;
     }
-    if (!cpu->halted || cpu_has_work(cpu) ||
-        whpx_apic_in_platform()) {
+    if (!cpu->halted || cpu_has_work(cpu)) {
         return false;
     }
     return true;
diff --git a/target/i386/whpx/whpx-accel-ops.c b/target/i386/whpx/whpx-accel-ops.c
index 6bc47c53098..1f9c6d52c27 100644
--- a/target/i386/whpx/whpx-accel-ops.c
+++ b/target/i386/whpx/whpx-accel-ops.c
@@ -83,6 +83,11 @@ static void whpx_kick_vcpu_thread(CPUState *cpu)
     }
 }
 
+static bool whpx_cpu_has_work(CPUState *cpu)
+{
+    return whpx_apic_in_platform();
+}
+
 static void whpx_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
@@ -94,6 +99,7 @@ static void whpx_accel_ops_class_init(ObjectClass *oc, void *data)
     ops->synchronize_post_init = whpx_cpu_synchronize_post_init;
     ops->synchronize_state = whpx_cpu_synchronize_state;
     ops->synchronize_pre_loadvm = whpx_cpu_synchronize_pre_loadvm;
+    ops->has_work = whpx_cpu_has_work;
 }
 
 static const TypeInfo whpx_accel_ops_type = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177435.322875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOE-0004eH-Sr; Thu, 02 Sep 2021 16:16:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177435.322875; Thu, 02 Sep 2021 16:16:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOE-0004dr-Ph; Thu, 02 Sep 2021 16:16:34 +0000
Received: by outflank-mailman (input) for mailman id 177435;
 Thu, 02 Sep 2021 16:16:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOD-00029c-Op
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:33 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5ab99782-1fbd-4846-a155-7ff202366422;
 Thu, 02 Sep 2021 16:16:29 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id t15so3835538wrg.7
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:29 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 g138sm2059442wmg.34.2021.09.02.09.16.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 5ab99782-1fbd-4846-a155-7ff202366422
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=gBDJiXTHT9IuEk9FfrTxtULybQQn9dfnXif9nUI7YC4=;
        b=LsbNjaXg9ZfDPBy35BgL3iTqoeboaHkvnknRwyAU2+xh3Y/GU5flK162rFrVD6PusK
         Zqye8j5GUtb9gwLuuAwBTaOPWqvSsNMI+LA1B5XlyhQj2OVug2mbnYS/clkuPBIuFEym
         pP4NuPbllHtklBm3Yk0D0W9rgb1jh68G001/gVtLBxzuFMcKbp53adEKS/t8pLvhfvjr
         /zEon0LvHHXUsmhiO9rLMnilC4Arm5Crf6gVF6CvpbMBshlkL+4jtharx5Q/STsgQBAC
         MvKjTe/KtyoK/YH9U42tNMwUXpXFN7GCzVSMFrxoghGcQKddKIXW2rcVsMa4qI3qYUFC
         llIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=gBDJiXTHT9IuEk9FfrTxtULybQQn9dfnXif9nUI7YC4=;
        b=kEv3BmFnyeHc6i4NdKXd76NahA4ttP4lhASayAo6lutF9BCJJ2V23iaZEmew8wfarp
         s2DrZZK6lVinQI6C35XXWmpK8V5SsSBdFz09xsRefLkeXGPOpTd+a1c4ep8VjrnN4K8v
         qiNWddsx1e2jcsXdgSbXLPIhKCzH0ItGOOPyRNNs0XqgzeEdAaI+gsaMFm5MnUC1OvHY
         Qyup9znmWkNl/Wm5B05iP1UF+JX6QruPuO/Us1w8yGkZGdtnt++FuGFLdeYfBqo4KZJT
         09GHANxeUiXjPnkNbPQU0txQXO/pcEUlPB11dgaGLPfKlKKjk/HgR0sMFIQ9CER1/k1L
         W00g==
X-Gm-Message-State: AOAM532lCxNXmb4vukUiuODmdi3Ztj03b8FOeqTYm7vn+4N0i8GvHLrM
	AJWr3zKYiNFpsphcvBf6NbI=
X-Google-Smtp-Source: ABdhPJxj90aNQOBb3bG354GuN1ofupsCwK8r7Qe99fRLxGQ8aaew/tNKGnIR32ntB4hlx8/hSq93OA==
X-Received: by 2002:a05:6000:36e:: with SMTP id f14mr4867472wrf.196.1630599389049;
        Thu, 02 Sep 2021 09:16:29 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 07/30] accel/tcg: Implement AccelOpsClass::has_work() as stub
Date: Thu,  2 Sep 2021 18:15:20 +0200
Message-Id: <20210902161543.417092-8-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add TCG target-specific has_work() handler in TCGCPUOps,
and add tcg_cpu_has_work() as AccelOpsClass has_work()
implementation.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 include/hw/core/tcg-cpu-ops.h |  4 ++++
 accel/tcg/tcg-accel-ops.c     | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/hw/core/tcg-cpu-ops.h b/include/hw/core/tcg-cpu-ops.h
index bbec7760f48..919d9006e24 100644
--- a/include/hw/core/tcg-cpu-ops.h
+++ b/include/hw/core/tcg-cpu-ops.h
@@ -66,6 +66,10 @@ struct TCGCPUOps {
     void (*do_interrupt)(CPUState *cpu);
 #endif /* !CONFIG_USER_ONLY || !TARGET_I386 */
 #ifdef CONFIG_SOFTMMU
+    /**
+     * @has_work: Callback for checking if there is work to do.
+     */
+    bool (*has_work)(CPUState *cpu);
     /** @cpu_exec_interrupt: Callback for processing interrupts in cpu_exec */
     bool (*cpu_exec_interrupt)(CPUState *cpu, int interrupt_request);
     /**
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 1a8e8390bd6..ed4ebe735fe 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -32,6 +32,7 @@
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
 #include "exec/exec-all.h"
+#include "hw/core/tcg-cpu-ops.h"
 
 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-mttcg.h"
@@ -73,6 +74,16 @@ int tcg_cpus_exec(CPUState *cpu)
     return ret;
 }
 
+static bool tcg_cpu_has_work(CPUState *cpu)
+{
+    CPUClass *cc = CPU_GET_CLASS(cpu);
+
+    if (!cc->tcg_ops->has_work) {
+        return false;
+    }
+    return cc->tcg_ops->has_work(cpu);
+}
+
 /* mask must never be zero, except for A20 change call */
 void tcg_handle_interrupt(CPUState *cpu, int mask)
 {
@@ -108,6 +119,7 @@ static void tcg_accel_ops_init(AccelOpsClass *ops)
         ops->kick_vcpu_thread = rr_kick_vcpu_thread;
         ops->handle_interrupt = tcg_handle_interrupt;
     }
+    ops->has_work = tcg_cpu_has_work;
 }
 
 static void tcg_accel_ops_class_init(ObjectClass *oc, void *data)
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177443.322887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOP-0005I1-7V; Thu, 02 Sep 2021 16:16:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177443.322887; Thu, 02 Sep 2021 16:16:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOP-0005Hp-3p; Thu, 02 Sep 2021 16:16:45 +0000
Received: by outflank-mailman (input) for mailman id 177443;
 Thu, 02 Sep 2021 16:16:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpON-00029c-PD
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:43 +0000
Received: from mail-wr1-x42f.google.com (unknown [2a00:1450:4864:20::42f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fa103d0b-92f2-4ae1-af99-c49ba28658ee;
 Thu, 02 Sep 2021 16:16:35 +0000 (UTC)
Received: by mail-wr1-x42f.google.com with SMTP id t15so3835941wrg.7
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:35 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 d9sm2161496wrm.21.2021.09.02.09.16.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: fa103d0b-92f2-4ae1-af99-c49ba28658ee
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=eXa1glKBQAZuhiqfHx9McrLJH+6vsqw3g1Zly0CLLW4=;
        b=ae0rJedtnyFXXpMPGaAYczb7D2gf5sAIBD78JSrM21inhCc6oobXicz8ZrAZbwKo+u
         LWgigyv/HdAEIiac6GDnYpFFEaYaskd9kEYeEtmG6nZ6xqWdpNpJ01xQm8qn97nDsycD
         JxK3rpivyOqG1ZF86VDCnWe7LDsUSWz4wzcJ/hvTp90Ehm2lEwn32lLuy6xZI/8yyuGo
         D3grkzcyRTx5k7pQjKg/3FGCaby6/Z3ejv2o7bs/xAAYUcfve3y2UqEM+TGGHH5UOrhG
         CAAu0Bd2wdhdwYLKumOwKFsEqKQ4OFbQ7TMnB4Cn4EM6ysG1/AHT1p0+QcU+g0Nj2W20
         hOjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=eXa1glKBQAZuhiqfHx9McrLJH+6vsqw3g1Zly0CLLW4=;
        b=jSlQ1lsubY4x8SnJQDWwEF1V/B3FiYRvCN/mPIfLJ0Phs3uhXdgL+JgMnK9C2CvY1y
         eOTBH+MxWVhBcz9Cm5aRQSmG1wx2xoEjxK9INOwlfTY3fC4ye7KC3fZBql/7gWt+Iaj9
         CR7UbXvorQ85//uOPiLlnx4m7yzXDxrpJDXmMY2v+K+suKC1QX591YKxUfhEaZHQj2oV
         xsYtT7g1MO+Wbv190oSxck92ZR0JFepUG71NeJb5rqsHiZHpoYYwKi6v7mU1qI/UsWFY
         RGA7qPGk1XaeYlee0xlivr1dazl3lhDVmweGHsOSIe6Cs3Ra9vrOQVJDhd7YGrAFJKfL
         aVhg==
X-Gm-Message-State: AOAM533PQCdY3sgmPxI+SXvBB/iPUafMfqP4MK33eVyAE4qsKFIcfD70
	8TozFj0F/mpPuDAU0qANeQU=
X-Google-Smtp-Source: ABdhPJzKZc/XQ+fV0m19x1zrzrTnVXlm9/5kK9IV3yDSnB/JNoam4AlRGKu+3WZKYWOwLNVhMsbvNg==
X-Received: by 2002:a05:6000:10:: with SMTP id h16mr4754013wrx.24.1630599394935;
        Thu, 02 Sep 2021 09:16:34 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 08/30] target/alpha: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:21 +0200
Message-Id: <20210902161543.417092-9-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/alpha/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
index 93e16a2ffb4..32cf5a2ea9f 100644
--- a/target/alpha/cpu.c
+++ b/target/alpha/cpu.c
@@ -33,6 +33,7 @@ static void alpha_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool alpha_cpu_has_work(CPUState *cs)
 {
     /* Here we are checking to see if the CPU should wake up from HALT.
@@ -47,6 +48,7 @@ static bool alpha_cpu_has_work(CPUState *cs)
                                     | CPU_INTERRUPT_SMP
                                     | CPU_INTERRUPT_MCHK);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void alpha_cpu_disas_set_info(CPUState *cpu, disassemble_info *info)
 {
@@ -221,6 +223,7 @@ static const struct TCGCPUOps alpha_tcg_ops = {
     .tlb_fill = alpha_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = alpha_cpu_has_work,
     .cpu_exec_interrupt = alpha_cpu_exec_interrupt,
     .do_interrupt = alpha_cpu_do_interrupt,
     .do_transaction_failed = alpha_cpu_do_transaction_failed,
@@ -238,7 +241,6 @@ static void alpha_cpu_class_init(ObjectClass *oc, void *data)
                                     &acc->parent_realize);
 
     cc->class_by_name = alpha_cpu_class_by_name;
-    cc->has_work = alpha_cpu_has_work;
     cc->dump_state = alpha_cpu_dump_state;
     cc->set_pc = alpha_cpu_set_pc;
     cc->gdb_read_register = alpha_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:16:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:16:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177449.322898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOT-0005mE-HH; Thu, 02 Sep 2021 16:16:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177449.322898; Thu, 02 Sep 2021 16:16:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpOT-0005m3-De; Thu, 02 Sep 2021 16:16:49 +0000
Received: by outflank-mailman (input) for mailman id 177449;
 Thu, 02 Sep 2021 16:16:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOS-00029c-P7
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:48 +0000
Received: from mail-wm1-x332.google.com (unknown [2a00:1450:4864:20::332])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4083f8f6-3927-4403-a69c-ad13ed63b36b;
 Thu, 02 Sep 2021 16:16:41 +0000 (UTC)
Received: by mail-wm1-x332.google.com with SMTP id
 192-20020a1c04c9000000b002f7a4ab0a49so1447222wme.0
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:41 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 s7sm2278245wra.75.2021.09.02.09.16.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 4083f8f6-3927-4403-a69c-ad13ed63b36b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=umRcFm5z0yg595NbXyjpfnNGeUl5eK5xZLxoXeUa+qc=;
        b=pz/nBk0UJK243d6EyhFjqhTq8St+3bylyw8UsBpEg2waosqkIlwbtba8uBILAoIr7Q
         lNryWY6OK0Yv0ei6QJ6YXdrGJmNPpUHm0iNE4fP7UqEe/h1YNv75NdMU0PYG192jLiEi
         nT8wtIR6GbLksC5eh0yUWGhRmDBX0INOYd68LwmG3Qv1zsVvvdfXM2Eh93Z4wPStdVNa
         KE8qNYenYdeo9Hw6npfh1Orrhaz/BR1l28Y8J4uAZJNC+LNfQ5fwkt6laufXunuUNo9o
         yefdGCG2pUPAo+hQ2RqYEKAbC+KVHHoFEwqvKykz0cg+3b9kRE25Iwy6z1ljbMNpTwU1
         QDYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=umRcFm5z0yg595NbXyjpfnNGeUl5eK5xZLxoXeUa+qc=;
        b=a1MTd6Sr5RuLWZnKxTOu5T/ioG2OW0lRu1s3mNvFLopb/6A4BDgspUA3X3lfPWhv1L
         tiP2kIUHbhSTV2DdxKIheyJm9CHa+BAky5tc34eXroUJHI9gvLi1WE2VeO3zZ1PtPZcK
         qaoK0/J1zA/lES8H24PJgw15NEVgDu4m1r2Ib7iaSCrsDpG2rrVlF0fD82/RCKsgGm+H
         HXZUVAqEWzBqTiaRkLYzOw2PfxB+V3D3vdz/+ToOefkpP/GQ1qKWETg+zDd8JQ6grBPE
         k5sEWuNpirUPtkNS4ZD4niecwcw+64B+Fw3ZRYoYCWFelY+z4pddcFRYqVO+8/9UweEo
         28Aw==
X-Gm-Message-State: AOAM533tPHR+YkfVgjDlquJk0Ci0/DyHbYwDpBHZWn+CrGKqfgjr4IOo
	AriSYN8d7YlwWRwbMHT+8hg=
X-Google-Smtp-Source: ABdhPJxG0Ekgmsw86aV3vYz0yeffJLjcyX9LXqOx701gt1hpjlWPvR2zb0A5F6AGLmteTS9Julrfnw==
X-Received: by 2002:a1c:4cd:: with SMTP id 196mr4020214wme.10.1630599401061;
        Thu, 02 Sep 2021 09:16:41 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 09/30] target/arm: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:22 +0200
Message-Id: <20210902161543.417092-10-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/arm/cpu.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index ba0741b20e4..e11aa625a5f 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -73,8 +73,8 @@ void arm_cpu_synchronize_from_tb(CPUState *cs,
         env->regs[15] = tb->pc;
     }
 }
-#endif /* CONFIG_TCG */
 
+#ifndef CONFIG_USER_ONLY
 static bool arm_cpu_has_work(CPUState *cs)
 {
     ARMCPU *cpu = ARM_CPU(cs);
@@ -85,6 +85,9 @@ static bool arm_cpu_has_work(CPUState *cs)
          | CPU_INTERRUPT_VFIQ | CPU_INTERRUPT_VIRQ
          | CPU_INTERRUPT_EXITTB);
 }
+#endif /* !CONFIG_USER_ONLY */
+
+#endif /* CONFIG_TCG */
 
 void arm_register_pre_el_change_hook(ARMCPU *cpu, ARMELChangeHookFn *hook,
                                  void *opaque)
@@ -2017,6 +2020,7 @@ static const struct TCGCPUOps arm_tcg_ops = {
     .debug_excp_handler = arm_debug_excp_handler,
 
 #if !defined(CONFIG_USER_ONLY)
+    .has_work = arm_cpu_has_work,
     .cpu_exec_interrupt = arm_cpu_exec_interrupt,
     .do_interrupt = arm_cpu_do_interrupt,
     .do_transaction_failed = arm_cpu_do_transaction_failed,
@@ -2041,7 +2045,6 @@ static void arm_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, arm_cpu_reset, &acc->parent_reset);
 
     cc->class_by_name = arm_cpu_class_by_name;
-    cc->has_work = arm_cpu_has_work;
     cc->dump_state = arm_cpu_dump_state;
     cc->set_pc = arm_cpu_set_pc;
     cc->gdb_read_register = arm_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:18:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:18:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177462.322909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPn-0007BT-Vq; Thu, 02 Sep 2021 16:18:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177462.322909; Thu, 02 Sep 2021 16:18:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPn-0007BM-Qm; Thu, 02 Sep 2021 16:18:11 +0000
Received: by outflank-mailman (input) for mailman id 177462;
 Thu, 02 Sep 2021 16:18:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPm-0007B4-Cu
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:18:10 +0000
Received: from mail-wm1-x32b.google.com (unknown [2a00:1450:4864:20::32b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01f06e99-8be3-4465-a2f9-a2b4690f76ce;
 Thu, 02 Sep 2021 16:18:08 +0000 (UTC)
Received: by mail-wm1-x32b.google.com with SMTP id
 m25-20020a7bcb99000000b002e751bcb5dbso1735750wmi.5
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:18:08 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 k14sm2234887wri.46.2021.09.02.09.18.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:18:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 01f06e99-8be3-4465-a2f9-a2b4690f76ce
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=UWFl0S8+QJmSRfKpaWNU58d5jvI099dEW79c4Eds8Co=;
        b=qyX0fbK8beMEiA2sBSgl3PW3ky5dGcSVCBtTwKFMLbBLTDoVjxKFiSsk0jDC/pFJh8
         EQJ9wt19ZKu7z9D2NO2X1GBhffNKLdpe3QtarXXg5bKru7SgrpNFq7ykSau/rj7BWEMU
         1lB3fKoh6dEiYxH/wEudtnGa9nyEo7DeltzASvLMnmAUaT41c/G+J+JRbl3ezoYTAza1
         Y+ETvUqkI+1eP+JhGzMn2Azr7F5hdjAx5ZUxT+4ueBQ0CSz4Bd282avi1jucNAchPVMA
         9SsBqrFRz0Y5wm79SMIMZa6Daop/L/LafP8vvrGNOlI29+ElFdjMqyjQo9hpcEC0Ade/
         GupA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=UWFl0S8+QJmSRfKpaWNU58d5jvI099dEW79c4Eds8Co=;
        b=qbYB+lHC3517KpzJLBT9MQiKuA4rWUW6Hviyi6tjq3zJ0kJqIV+Ngv9mZCgu9tyj7F
         faqEg4t7jBJcez6elKSA2n+GqJRlPk0fSy0vZI0EPuBk2vj3fUXRSgLjsE6180FpoU+W
         aZ/cqMLQWnM44zi32wC46EXZ67bytLm64mExhPw4CL8DhtOavQ1RuRg3O7phsHwAhi+g
         NUM2WW5A4QR3ojjFua2+pdmkfCosdGNS7CjPZpEV8dSMBTNsYX0XAAo6o/N6fXHxXD1Q
         nYCOsitfvSh1PaWGaQanJeNxziCMOvqKkhU6STNlbDz8DpmMX86AhRN4K/smuSpoqXhH
         RX7w==
X-Gm-Message-State: AOAM531pyzbwVShzwoh8XUduIFc/Q+p73tE0GjlwpSBVkc0qPHYnFv0D
	rZ6U6DvmgMRGU0hBaXXIkHk=
X-Google-Smtp-Source: ABdhPJxiiVaTYWvAG9DN++m79S0TQSTDr2C/1kQW8ctmk5/+oYLKUYcjyiQjFsrDz4LSrxOSg3eaUA==
X-Received: by 2002:a1c:202:: with SMTP id 2mr3928971wmc.122.1630599486805;
        Thu, 02 Sep 2021 09:18:06 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 22/30] target/ppc: Simplify has_work() handlers
Date: Thu,  2 Sep 2021 18:15:35 +0200
Message-Id: <20210902161543.417092-23-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The common ppc_cpu_has_work() handler already checks for cs->halted,
so we can simplify all callees.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/ppc/cpu_init.c | 294 ++++++++++++++++++++----------------------
 1 file changed, 138 insertions(+), 156 deletions(-)

diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index bbad16cc1ec..c8ec47d58fa 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7589,33 +7589,29 @@ static bool cpu_has_work_POWER7(CPUState *cs)
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
 
-    if (cs->halted) {
-        if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
-            return false;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
-            (env->spr[SPR_LPCR] & LPCR_P7_PECE0)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
-            (env->spr[SPR_LPCR] & LPCR_P7_PECE1)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
-            (env->spr[SPR_LPCR] & LPCR_P7_PECE2)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HMI)) &&
-            (env->spr[SPR_LPCR] & LPCR_P7_PECE2)) {
-            return true;
-        }
-        if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
-            return true;
-        }
+    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
         return false;
-    } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
+        (env->spr[SPR_LPCR] & LPCR_P7_PECE0)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
+        (env->spr[SPR_LPCR] & LPCR_P7_PECE1)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
+        (env->spr[SPR_LPCR] & LPCR_P7_PECE2)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HMI)) &&
+        (env->spr[SPR_LPCR] & LPCR_P7_PECE2)) {
+        return true;
+    }
+    if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
+        return true;
+    }
+    return false;
 }
 #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
@@ -7750,41 +7746,37 @@ static bool cpu_has_work_POWER8(CPUState *cs)
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
 
-    if (cs->halted) {
-        if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
-            return false;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE2)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE3)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE4)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HMI)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE4)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE0)) {
-            return true;
-        }
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_P8_PECE1)) {
-            return true;
-        }
-        if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
-            return true;
-        }
+    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
         return false;
-    } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE2)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE3)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE4)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HMI)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE4)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE0)) {
+        return true;
+    }
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_P8_PECE1)) {
+        return true;
+    }
+    if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
+        return true;
+    }
+    return false;
 }
 #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
@@ -7948,58 +7940,53 @@ static bool cpu_has_work_POWER9(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
+    uint64_t psscr = env->spr[SPR_PSSCR];
 
-    if (cs->halted) {
-        uint64_t psscr = env->spr[SPR_PSSCR];
-
-        if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
-            return false;
-        }
-
-        /* If EC is clear, just return true on any pending interrupt */
-        if (!(psscr & PSSCR_EC)) {
-            return true;
-        }
-        /* External Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
-            (env->spr[SPR_LPCR] & LPCR_EEE)) {
-            bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (heic == 0 || !msr_hv || msr_pr) {
-                return true;
-            }
-        }
-        /* Decrementer Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
-            (env->spr[SPR_LPCR] & LPCR_DEE)) {
-            return true;
-        }
-        /* Machine Check or Hypervisor Maintenance Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK |
-            1u << PPC_INTERRUPT_HMI)) && (env->spr[SPR_LPCR] & LPCR_OEE)) {
-            return true;
-        }
-        /* Privileged Doorbell Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_PDEE)) {
-            return true;
-        }
-        /* Hypervisor Doorbell Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_HDEE)) {
-            return true;
-        }
-        /* Hypervisor virtualization exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HVIRT)) &&
-            (env->spr[SPR_LPCR] & LPCR_HVEE)) {
-            return true;
-        }
-        if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
-            return true;
-        }
+    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
         return false;
-    } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
+
+    /* If EC is clear, just return true on any pending interrupt */
+    if (!(psscr & PSSCR_EC)) {
+        return true;
+    }
+    /* External Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
+        (env->spr[SPR_LPCR] & LPCR_EEE)) {
+        bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
+        if (heic == 0 || !msr_hv || msr_pr) {
+            return true;
+        }
+    }
+    /* Decrementer Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
+        (env->spr[SPR_LPCR] & LPCR_DEE)) {
+        return true;
+    }
+    /* Machine Check or Hypervisor Maintenance Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK |
+        1u << PPC_INTERRUPT_HMI)) && (env->spr[SPR_LPCR] & LPCR_OEE)) {
+        return true;
+    }
+    /* Privileged Doorbell Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_PDEE)) {
+        return true;
+    }
+    /* Hypervisor Doorbell Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_HDEE)) {
+        return true;
+    }
+    /* Hypervisor virtualization exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HVIRT)) &&
+        (env->spr[SPR_LPCR] & LPCR_HVEE)) {
+        return true;
+    }
+    if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
+        return true;
+    }
+    return false;
 }
 #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
@@ -8158,58 +8145,53 @@ static bool cpu_has_work_POWER10(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
+    uint64_t psscr = env->spr[SPR_PSSCR];
 
-    if (cs->halted) {
-        uint64_t psscr = env->spr[SPR_PSSCR];
-
-        if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
-            return false;
-        }
-
-        /* If EC is clear, just return true on any pending interrupt */
-        if (!(psscr & PSSCR_EC)) {
-            return true;
-        }
-        /* External Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
-            (env->spr[SPR_LPCR] & LPCR_EEE)) {
-            bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (heic == 0 || !msr_hv || msr_pr) {
-                return true;
-            }
-        }
-        /* Decrementer Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
-            (env->spr[SPR_LPCR] & LPCR_DEE)) {
-            return true;
-        }
-        /* Machine Check or Hypervisor Maintenance Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK |
-            1u << PPC_INTERRUPT_HMI)) && (env->spr[SPR_LPCR] & LPCR_OEE)) {
-            return true;
-        }
-        /* Privileged Doorbell Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_PDEE)) {
-            return true;
-        }
-        /* Hypervisor Doorbell Exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
-            (env->spr[SPR_LPCR] & LPCR_HDEE)) {
-            return true;
-        }
-        /* Hypervisor virtualization exception */
-        if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HVIRT)) &&
-            (env->spr[SPR_LPCR] & LPCR_HVEE)) {
-            return true;
-        }
-        if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
-            return true;
-        }
+    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD)) {
         return false;
-    } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
+
+    /* If EC is clear, just return true on any pending interrupt */
+    if (!(psscr & PSSCR_EC)) {
+        return true;
+    }
+    /* External Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
+        (env->spr[SPR_LPCR] & LPCR_EEE)) {
+        bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
+        if (heic == 0 || !msr_hv || msr_pr) {
+            return true;
+        }
+    }
+    /* Decrementer Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
+        (env->spr[SPR_LPCR] & LPCR_DEE)) {
+        return true;
+    }
+    /* Machine Check or Hypervisor Maintenance Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK |
+        1u << PPC_INTERRUPT_HMI)) && (env->spr[SPR_LPCR] & LPCR_OEE)) {
+        return true;
+    }
+    /* Privileged Doorbell Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_PDEE)) {
+        return true;
+    }
+    /* Hypervisor Doorbell Exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HDOORBELL)) &&
+        (env->spr[SPR_LPCR] & LPCR_HDEE)) {
+        return true;
+    }
+    /* Hypervisor virtualization exception */
+    if ((env->pending_interrupts & (1u << PPC_INTERRUPT_HVIRT)) &&
+        (env->spr[SPR_LPCR] & LPCR_HVEE)) {
+        return true;
+    }
+    if (env->pending_interrupts & (1u << PPC_INTERRUPT_RESET)) {
+        return true;
+    }
+    return false;
 }
 #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:18:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:18:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177463.322920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPs-0007UK-Ab; Thu, 02 Sep 2021 16:18:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177463.322920; Thu, 02 Sep 2021 16:18:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPs-0007UB-6e; Thu, 02 Sep 2021 16:18:16 +0000
Received: by outflank-mailman (input) for mailman id 177463;
 Thu, 02 Sep 2021 16:18:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPq-0007B4-Ej
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:18:14 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dc5d9528-0d0a-4ac3-9ada-94748a1401dd;
 Thu, 02 Sep 2021 16:18:13 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id n5so3813844wro.12
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:18:13 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 g1sm2930192wrb.27.2021.09.02.09.18.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:18:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: dc5d9528-0d0a-4ac3-9ada-94748a1401dd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=NcTQcOUeeehYkPycaMHvSzYk1/JpDe+W3tlOioHt6ec=;
        b=Qo3n3nT5T2uCxZDbpwI/2fMsolvJt4Y1gOrYSxa05AyniYxnVrYAyAl58E/mePDB+Q
         mp74y9slo+JHNdNbb124nVsmTrRmSYtRFeXvQ45hmAzQLS7SZdoREJlqjngb5ERIDHOz
         8fG18ynHtTwJKTgzScUP5EYAEGDfAkaAdwzs1bs8Mwbkci0fjxZIrzyNblhH2fEmvKn0
         14gWvYc8/SbUfS0Y8MjZjc8PX2H+MElpBG7G8YmI1yDXyIqgKhSWIpr7bVwK4JmVtxvW
         D4kk4tzADto1QBEDwX3w/FLijcMe32GJWkC5PPc+o9liJR70fQ003gNi58OOJ6evLU1W
         Khjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=NcTQcOUeeehYkPycaMHvSzYk1/JpDe+W3tlOioHt6ec=;
        b=SYjlQoR/J7tk5xKhuFLbP8mtHn0u5Y58HRYbmKJX6Gh1gn1SFrdvssMT4dRUWG5mOi
         0fnxWPaGTE5WY0PvLeUe592iAgIogd5nOkIYUyKCxLAs6R7FJtx+WXIJqwF1G+FUbkRN
         SzwqRz0bT0M/RlZzeGPKYbt+yyPjD4VTD/I80sjEB8duFb7vUEjXnE5hWDBhv1DTVog+
         pWZEPQjTDCu/XfK7aDhNSkjqcLQGNmKKC0RDyB0dQmsJAX4P5CU1KjtGbvARkoDJuazz
         B/hSezodbuoD//QQtaXv6IXzggdep5DQaRn1p+55k7ZfEvktTIr4ZOFsKzJaKu1yZiZh
         VGcQ==
X-Gm-Message-State: AOAM5318aK6bK+2yl3r5WBdGH4QLX+vA1wZmR1T4AwVVkW9zFPC4Ngy0
	LvN1day/ndVD9/dULbd2ac8=
X-Google-Smtp-Source: ABdhPJzq4gEgBOzEB6hIFdgS7MM2mMxbBTgou1tfWKrQyw8z4fa5k0Y3IhkCYtobsiFPzwgsy9U7mQ==
X-Received: by 2002:adf:c390:: with SMTP id p16mr4949292wrf.105.1630599492959;
        Thu, 02 Sep 2021 09:18:12 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 23/30] target/riscv: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:36 +0200
Message-Id: <20210902161543.417092-24-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/riscv/cpu.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 13575c14085..abb555a8bdb 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -335,9 +335,9 @@ static void riscv_cpu_synchronize_from_tb(CPUState *cs,
     env->pc = tb->pc;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool riscv_cpu_has_work(CPUState *cs)
 {
-#ifndef CONFIG_USER_ONLY
     RISCVCPU *cpu = RISCV_CPU(cs);
     CPURISCVState *env = &cpu->env;
     /*
@@ -345,10 +345,8 @@ static bool riscv_cpu_has_work(CPUState *cs)
      * mode and delegation registers, but respect individual enables
      */
     return (env->mip & env->mie) != 0;
-#else
-    return true;
-#endif
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 void restore_state_to_opc(CPURISCVState *env, TranslationBlock *tb,
                           target_ulong *data)
@@ -647,6 +645,7 @@ static const struct TCGCPUOps riscv_tcg_ops = {
     .tlb_fill = riscv_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = riscv_cpu_has_work,
     .cpu_exec_interrupt = riscv_cpu_exec_interrupt,
     .do_interrupt = riscv_cpu_do_interrupt,
     .do_transaction_failed = riscv_cpu_do_transaction_failed,
@@ -666,7 +665,6 @@ static void riscv_cpu_class_init(ObjectClass *c, void *data)
     device_class_set_parent_reset(dc, riscv_cpu_reset, &mcc->parent_reset);
 
     cc->class_by_name = riscv_cpu_class_by_name;
-    cc->has_work = riscv_cpu_has_work;
     cc->dump_state = riscv_cpu_dump_state;
     cc->set_pc = riscv_cpu_set_pc;
     cc->gdb_read_register = riscv_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:18:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:18:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177464.322931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPy-0007rZ-Kq; Thu, 02 Sep 2021 16:18:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177464.322931; Thu, 02 Sep 2021 16:18:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpPy-0007rK-GN; Thu, 02 Sep 2021 16:18:22 +0000
Received: by outflank-mailman (input) for mailman id 177464;
 Thu, 02 Sep 2021 16:18:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPw-0007of-Iv
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:18:20 +0000
Received: from mail-wm1-x329.google.com (unknown [2a00:1450:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b72cc44c-9d4f-42be-b49a-d342b8f6eda6;
 Thu, 02 Sep 2021 16:18:19 +0000 (UTC)
Received: by mail-wm1-x329.google.com with SMTP id e26so1676496wmk.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:18:19 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 l35sm1842816wms.40.2021.09.02.09.18.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:18:18 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: b72cc44c-9d4f-42be-b49a-d342b8f6eda6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=iBR7XZaVDWa3VEAQfJT4KESDdYv2ZsQvbER1+LPZ/aI=;
        b=b3oM2a7kfu59T16h1BQOivDnrY/oITcLEbYMDbyYiQhSRpMPPbkow/DjE+1Ih4EQL/
         DUMVJOdg0XVJbEcJ79XxMVenCZKrXPuNo4lfReSnNPQpSjZ4YOofjjl3erNURKNYsHtL
         HEHPXeiHX0YfdcBowdh8WFKOQbcQAU/wfMDvbfu4iq/jPgfcZfFUIVDjHXKT3CRk6R6M
         vj3MnraM9QYlz2Bf2AIl43yWvEv2b8GLOTUYPYAin5G610qJf8lQsTsudFi4GarHEOkf
         I84tHwdyec/ClmJMz9FqfnRo+aZy5mZWv4eOOqkltbqaFUUFGtiu6tZ9fx/UuTf496te
         WjkA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=iBR7XZaVDWa3VEAQfJT4KESDdYv2ZsQvbER1+LPZ/aI=;
        b=XC7H63v/lhUXbDyNEzU8+md8Q33329MaofzYASS1Mq3uAkryDjopPy10l2SFqURquz
         rypd3uWrS/XI2sdwbefbN5FCw0gPIGnAdQxEACoDsnPFktesRwu/8swgQ9wv/D+abhsA
         a7vPjgFklBktI90f3R7cO4OinSfwdf8yKneHny/iv3NJrH3vQTFlwZ0DfmQ1AM9jdNVX
         b2lpKZ81+bYon/Qa50+OqeAfnfdK4Xf7Eh1ihcaMXF3T/WFvSxIL91TeASmRk/uDpED9
         RAKjrUZG7VEHFcUcthBdGay2ZruJp3G+9HF7tq6RilzRRwdcFmr0mNVQuBA4KqIJkNPK
         OVtQ==
X-Gm-Message-State: AOAM530T+JziSXzZmMQWBcwGHaNGrFP7TX7pNrSZ1rFAZILHHFsNY26y
	p2JcI1tMAU+TxdVidVWyYYY=
X-Google-Smtp-Source: ABdhPJzOyuZQlMcSsHM1l7m0tMnQ7s93BFuzpn/h6Lj/C0+zH4RsxK7hVpmbT/Jm+TQOVSFAinpu2Q==
X-Received: by 2002:a1c:4682:: with SMTP id t124mr4067940wma.168.1630599499125;
        Thu, 02 Sep 2021 09:18:19 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 24/30] target/rx: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:37 +0200
Message-Id: <20210902161543.417092-25-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/rx/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/rx/cpu.c b/target/rx/cpu.c
index 25a4aa2976d..0d0cf6f9028 100644
--- a/target/rx/cpu.c
+++ b/target/rx/cpu.c
@@ -41,11 +41,13 @@ static void rx_cpu_synchronize_from_tb(CPUState *cs,
     cpu->env.pc = tb->pc;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool rx_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request &
         (CPU_INTERRUPT_HARD | CPU_INTERRUPT_FIR);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void rx_cpu_reset(DeviceState *dev)
 {
@@ -189,6 +191,7 @@ static const struct TCGCPUOps rx_tcg_ops = {
     .tlb_fill = rx_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = rx_cpu_has_work,
     .cpu_exec_interrupt = rx_cpu_exec_interrupt,
     .do_interrupt = rx_cpu_do_interrupt,
 #endif /* !CONFIG_USER_ONLY */
@@ -206,7 +209,6 @@ static void rx_cpu_class_init(ObjectClass *klass, void *data)
                                   &rcc->parent_reset);
 
     cc->class_by_name = rx_cpu_class_by_name;
-    cc->has_work = rx_cpu_has_work;
     cc->dump_state = rx_cpu_dump_state;
     cc->set_pc = rx_cpu_set_pc;
 
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177485.322953 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSe-0001le-Dr; Thu, 02 Sep 2021 16:21:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177485.322953; Thu, 02 Sep 2021 16:21:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSe-0001lX-9k; Thu, 02 Sep 2021 16:21:08 +0000
Received: by outflank-mailman (input) for mailman id 177485;
 Thu, 02 Sep 2021 16:21:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPf-00029c-Rf
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:18:03 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f94f3479-ee5e-4e24-8010-a4d3fd425a0c;
 Thu, 02 Sep 2021 16:18:01 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id q11so3831715wrr.9
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:18:01 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 c3sm2411819wrd.34.2021.09.02.09.17.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: f94f3479-ee5e-4e24-8010-a4d3fd425a0c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=MUUy/NWyVU2hg662wvWPJPkcXsVUh4rWx1BGsDTs1tA=;
        b=kEnHldIRSmSDQxyQdugw1Vx8L7xguo0M9caEjgWu9MaZq6gyOqkCAuD92VRjnuFYVu
         AFC1pCBYhs9ACTTeUQoUMMput3Qpo+G6rdKTIhD095mZXQogsq0uWanMgwmK05q4B113
         wHEl+rU0Lu5ECIbKPzs4FN4XtzXbG2tlJtl+zohx+CtLscNoURzNPXzDuLlX3udxNyRV
         ISqkEHEFnJiqPltUmbvOkyQ8UgtyH8boBBAz3eZMg7gCSDPvog7KSraXXpPQ+P/Y1jP/
         dnFgk5Q12dkH0yO8X+tMaBzTnLA6tV2u5dL5oZQj7YYuUZIHbs6Fs72PHNz4exx0IHfr
         tXUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=MUUy/NWyVU2hg662wvWPJPkcXsVUh4rWx1BGsDTs1tA=;
        b=LIuKnwEoVAx1nRgbaqjpxz2bR35k1JfN12ITN+9b2bZueszhmsZTZStouMzM0tMHH9
         47uuqJnF/oMEKIXpdhuMSHZTHydvxYDZruUSHPMxmDpjUDy15GBXZbZOtm8XkNw2YelO
         1mx+1Jyrwve1GbjPp3YHBi1fEwLHDIA/D/WLaQHuv40X7JgKQbd/hCPxUc2XGxvMUsOb
         H8cqqaG3GkLzx1bZ7fhwcwz5G/8LF3PTb8Ka9QF+gTx8PDKk+t/PdcTYNvWCSmEyXiuK
         1bnUd0LpeIRAkEGWkrH0LIBoqRLCrRrAa5v0Nkt+ThuAR6KhM88aSn2GlWVTRij7CvYe
         B/KA==
X-Gm-Message-State: AOAM530PjsfqhLXoEMxioICsu5vnCS9gy3YJxrzCqg1ToA+KtwkRDr8G
	24Qxbs5uIfXpdOfVYip+6/U=
X-Google-Smtp-Source: ABdhPJwaDZNhw8vo3XF1wL5OY+S/6RqfQvS7pZENs96vrPKDQEeSMVgbBjU4oik0CYAZjXMqLcm/Gg==
X-Received: by 2002:a5d:6cca:: with SMTP id c10mr4738535wrc.224.1630599480361;
        Thu, 02 Sep 2021 09:18:00 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 21/30] target/ppc: Introduce PowerPCCPUClass::has_work()
Date: Thu,  2 Sep 2021 18:15:34 +0200
Message-Id: <20210902161543.417092-22-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Each POWER cpu has its own has_work() implementation. Instead of
overloading CPUClass on each PowerPCCPUClass init, register the
generic ppc_cpu_has_work() handler, and have it call the POWER
specific has_work().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/ppc/cpu-qom.h  |  3 +++
 target/ppc/cpu_init.c | 26 ++++++++++++++++++--------
 2 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/target/ppc/cpu-qom.h b/target/ppc/cpu-qom.h
index 5800fa324e8..ff2bafcde6f 100644
--- a/target/ppc/cpu-qom.h
+++ b/target/ppc/cpu-qom.h
@@ -189,6 +189,9 @@ struct PowerPCCPUClass {
     int bfd_mach;
     uint32_t l1_dcache_size, l1_icache_size;
 #ifndef CONFIG_USER_ONLY
+#ifdef CONFIG_TCG
+    bool (*has_work)(CPUState *cpu);
+#endif /* CONFIG_TCG */
     unsigned int gdb_num_sprs;
     const char *gdb_spr_xml;
 #endif
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index e2e721c2b81..bbad16cc1ec 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7583,6 +7583,7 @@ static bool ppc_pvr_match_power7(PowerPCCPUClass *pcc, uint32_t pvr)
     return false;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool cpu_has_work_POWER7(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -7616,12 +7617,12 @@ static bool cpu_has_work_POWER7(CPUState *cs)
         return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(oc);
-    CPUClass *cc = CPU_CLASS(oc);
 
     dc->fw_name = "PowerPC,POWER7";
     dc->desc = "POWER7";
@@ -7630,7 +7631,6 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
     pcc->pcr_supported = PCR_COMPAT_2_06 | PCR_COMPAT_2_05;
     pcc->init_proc = init_proc_POWER7;
     pcc->check_pow = check_pow_nocheck;
-    cc->has_work = cpu_has_work_POWER7;
     pcc->insns_flags = PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MFTB |
                        PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
                        PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
@@ -7673,6 +7673,7 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
     pcc->lpcr_pm = LPCR_P7_PECE0 | LPCR_P7_PECE1 | LPCR_P7_PECE2;
     pcc->mmu_model = POWERPC_MMU_2_06;
 #if defined(CONFIG_SOFTMMU)
+    pcc->has_work = cpu_has_work_POWER7;
     pcc->hash64_opts = &ppc_hash64_opts_POWER7;
     pcc->lrg_decr_bits = 32;
 #endif
@@ -7743,6 +7744,7 @@ static bool ppc_pvr_match_power8(PowerPCCPUClass *pcc, uint32_t pvr)
     return false;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool cpu_has_work_POWER8(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -7784,12 +7786,12 @@ static bool cpu_has_work_POWER8(CPUState *cs)
         return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(oc);
-    CPUClass *cc = CPU_CLASS(oc);
 
     dc->fw_name = "PowerPC,POWER8";
     dc->desc = "POWER8";
@@ -7798,7 +7800,6 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
     pcc->pcr_supported = PCR_COMPAT_2_07 | PCR_COMPAT_2_06 | PCR_COMPAT_2_05;
     pcc->init_proc = init_proc_POWER8;
     pcc->check_pow = check_pow_nocheck;
-    cc->has_work = cpu_has_work_POWER8;
     pcc->insns_flags = PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MFTB |
                        PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
                        PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
@@ -7848,6 +7849,7 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
                    LPCR_P8_PECE3 | LPCR_P8_PECE4;
     pcc->mmu_model = POWERPC_MMU_2_07;
 #if defined(CONFIG_SOFTMMU)
+    pcc->has_work = cpu_has_work_POWER8;
     pcc->hash64_opts = &ppc_hash64_opts_POWER7;
     pcc->lrg_decr_bits = 32;
     pcc->n_host_threads = 8;
@@ -7941,6 +7943,7 @@ static bool ppc_pvr_match_power9(PowerPCCPUClass *pcc, uint32_t pvr)
     return false;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool cpu_has_work_POWER9(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -7998,12 +8001,12 @@ static bool cpu_has_work_POWER9(CPUState *cs)
         return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(oc);
-    CPUClass *cc = CPU_CLASS(oc);
 
     dc->fw_name = "PowerPC,POWER9";
     dc->desc = "POWER9";
@@ -8013,7 +8016,6 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
                          PCR_COMPAT_2_05;
     pcc->init_proc = init_proc_POWER9;
     pcc->check_pow = check_pow_nocheck;
-    cc->has_work = cpu_has_work_POWER9;
     pcc->insns_flags = PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MFTB |
                        PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
                        PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
@@ -8062,6 +8064,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
     pcc->lpcr_pm = LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | LPCR_OEE;
     pcc->mmu_model = POWERPC_MMU_3_00;
 #if defined(CONFIG_SOFTMMU)
+    pcc->has_work = cpu_has_work_POWER9;
     /* segment page size remain the same */
     pcc->hash64_opts = &ppc_hash64_opts_POWER7;
     pcc->radix_page_info = &POWER9_radix_page_info;
@@ -8150,6 +8153,7 @@ static bool ppc_pvr_match_power10(PowerPCCPUClass *pcc, uint32_t pvr)
     return false;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool cpu_has_work_POWER10(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -8207,12 +8211,12 @@ static bool cpu_has_work_POWER10(CPUState *cs)
         return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(oc);
-    CPUClass *cc = CPU_CLASS(oc);
 
     dc->fw_name = "PowerPC,POWER10";
     dc->desc = "POWER10";
@@ -8223,7 +8227,6 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
                          PCR_COMPAT_2_06 | PCR_COMPAT_2_05;
     pcc->init_proc = init_proc_POWER10;
     pcc->check_pow = check_pow_nocheck;
-    cc->has_work = cpu_has_work_POWER10;
     pcc->insns_flags = PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MFTB |
                        PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
                        PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
@@ -8275,6 +8278,7 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
     pcc->lpcr_pm = LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | LPCR_OEE;
     pcc->mmu_model = POWERPC_MMU_3_00;
 #if defined(CONFIG_SOFTMMU)
+    pcc->has_work = cpu_has_work_POWER10;
     /* segment page size remain the same */
     pcc->hash64_opts = &ppc_hash64_opts_POWER7;
     pcc->radix_page_info = &POWER10_radix_page_info;
@@ -8796,6 +8800,12 @@ static bool ppc_cpu_has_work(CPUState *cs)
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
 
+    if (cs->halted) {
+        PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
+
+        return pcc->has_work(cs);
+    }
+
     return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
 }
 #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177483.322942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSd-0001VW-4J; Thu, 02 Sep 2021 16:21:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177483.322942; Thu, 02 Sep 2021 16:21:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSd-0001VP-10; Thu, 02 Sep 2021 16:21:07 +0000
Received: by outflank-mailman (input) for mailman id 177483;
 Thu, 02 Sep 2021 16:21:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOr-00029c-Q7
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:13 +0000
Received: from mail-wr1-x42b.google.com (unknown [2a00:1450:4864:20::42b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 283a0621-2f95-4c3c-af23-46744b554ad2;
 Thu, 02 Sep 2021 16:17:00 +0000 (UTC)
Received: by mail-wr1-x42b.google.com with SMTP id n5so3807784wro.12
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:00 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 m5sm2067398wmi.1.2021.09.02.09.16.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 283a0621-2f95-4c3c-af23-46744b554ad2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=FVQ82fA2n+2YkhWVmssVrmq1ZgiebK+OWD8XFPkR2KQ=;
        b=NTN9poXw36sIQz7tT1cMpoOOwTG/lxJCt9n/hIDHpJ5oAGalZwShWoEdYjhHzzlMVv
         8/2soAA7WUYORMTBYBpUDtWNNxUFfDfUYJu04XkeEJLId1oUy0fpWdBO6D7Yyx1272JJ
         XKOsoC+njj5yavj0Qu1MgGlocXXaIcWXPI9T20BJ6avUCeJwQryVdBqiUkNfXQOnUX19
         Ji1BNyPOvloHuZqi99dMvFNWYRhFh5wijABsraMfCS9+BoMw9K93KiX8s+h3u2DNCNBy
         aYF39Fl9JigWDlhE1ZATlIq1qqUJ331MkmIOX3TRyMSoprNUbK+LRaCdaECy2hoGa1sh
         /wYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=FVQ82fA2n+2YkhWVmssVrmq1ZgiebK+OWD8XFPkR2KQ=;
        b=EaR3Us1frMpyXK5YUIWCBwqN+h2rllF1ZkloKOyiioL9daJb+0jiPTmT+RZ98C+cVy
         vCUhNLaDqoLrZRzBHSR9TI+b3qIFgZ1glTk9BdJbbcMiaZPs2qK0BVRg4hiWVhA8tSMI
         IZxM3biBTLE0/iVuQoVCTMXvX7P3KmA5VaEYJpwZrIPG/+u24FA9vlIH5/rgfY3HpH6O
         FKrC2IbMTMnHjmLrH3Jzrbc0mSKS3drTbyE9S9UQDquZ55CHm6w4ZuxVbXc0fdjOHDpT
         RtWM4Mti/gOXKlkxsUMoqh13CIskX+2J1Cs2edqje7HM8/th0PPA6/TB0TZvn6dkNWZl
         1raw==
X-Gm-Message-State: AOAM532Drrfvanph0Cnwpn/RhVYIunmNqamzXYkFLb+Jt/0XvFnOJ4A8
	0Qre1oKHg/Q7Yez1swhwtGU=
X-Google-Smtp-Source: ABdhPJwPHE7SqMAg84hdLm/drHbhhc9Rf1ir7m9E7UwL/bm+2kZWpbUhpR7xKiGkP3mn69DCmtzK4A==
X-Received: by 2002:adf:fd8c:: with SMTP id d12mr4712589wrr.21.1630599419723;
        Thu, 02 Sep 2021 09:16:59 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 12/30] target/hexagon: Remove unused has_work() handler
Date: Thu,  2 Sep 2021 18:15:25 +0200
Message-Id: <20210902161543.417092-13-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

has_work() is sysemu specific, and Hexagon target only provides
a linux-user implementation. Remove the unused hexagon_cpu_has_work().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/hexagon/cpu.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
index 3338365c16e..aa01974807c 100644
--- a/target/hexagon/cpu.c
+++ b/target/hexagon/cpu.c
@@ -189,11 +189,6 @@ static void hexagon_cpu_synchronize_from_tb(CPUState *cs,
     env->gpr[HEX_REG_PC] = tb->pc;
 }
 
-static bool hexagon_cpu_has_work(CPUState *cs)
-{
-    return true;
-}
-
 void restore_state_to_opc(CPUHexagonState *env, TranslationBlock *tb,
                           target_ulong *data)
 {
@@ -287,7 +282,6 @@ static void hexagon_cpu_class_init(ObjectClass *c, void *data)
     device_class_set_parent_reset(dc, hexagon_cpu_reset, &mcc->parent_reset);
 
     cc->class_by_name = hexagon_cpu_class_by_name;
-    cc->has_work = hexagon_cpu_has_work;
     cc->dump_state = hexagon_dump_state;
     cc->set_pc = hexagon_cpu_set_pc;
     cc->gdb_read_register = hexagon_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177488.322964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSp-0002FW-Rb; Thu, 02 Sep 2021 16:21:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177488.322964; Thu, 02 Sep 2021 16:21:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSp-0002FK-NU; Thu, 02 Sep 2021 16:21:19 +0000
Received: by outflank-mailman (input) for mailman id 177488;
 Thu, 02 Sep 2021 16:21:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPL-00029c-R8
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:43 +0000
Received: from mail-wm1-x331.google.com (unknown [2a00:1450:4864:20::331])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c3f37895-51ef-46b8-acb8-c1214b0ea298;
 Thu, 02 Sep 2021 16:17:36 +0000 (UTC)
Received: by mail-wm1-x331.google.com with SMTP id
 k5-20020a05600c1c8500b002f76c42214bso1866332wms.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:36 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 z17sm2350022wrh.66.2021.09.02.09.17.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: c3f37895-51ef-46b8-acb8-c1214b0ea298
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=oNTRH1yoq5af/chfn9I/5bXkTQKwubTn8iozem9IGSY=;
        b=PGBnfVPTt2uYSxX6RQU1qsSKDWiS62QWAFvPTZETHVyL2OfFEKZZM3g5e9WJ1/HCKp
         vwU/HD/XAeJVS5v2+My/H3taQ/5KpM2mCEqjy9lZxh2Z/pp5+KxKPIJb09S1d7E1zVbI
         3ZIcYz0tlufyP3eqE0RuxgxTothAxpq4vYSqZmx2K/VGuOwYk4ulhhYY34CvxwPWhOkc
         X6DZRjwitqLAQZOFu87Nxe9lRc1tqhLGZ8hROluYOSSfODd35hNP9l0+ra9n37CT0CeZ
         tWoQvZtrbSLJKAVJXRMVVda3fXJ9XRtlNf90jVOb68ZvlGwL1VX/T67Jlue0hgDZL0Fm
         SNZw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=oNTRH1yoq5af/chfn9I/5bXkTQKwubTn8iozem9IGSY=;
        b=iLqjt21ztDRh0OBVs+zX7k+Y0OdTQBSlJb7FqStIUEE1gBcu1a35LnUZWcssJxVemE
         Uyxafnx9LPaRx7vnD2ZxTprJsQLUycrJ6XreT+gc/xWAaSVqsGRWdSaHHz8ez0gxj2n/
         OLsJaZJe+gnan9DYU8shHiFY2wxYHla2jhUZHDTvfjRpx/qnyu1UALXqNbfayovI8I17
         5nct1bF7hJWagw6hQw8i7vizDIZlv4kYVbC5h++QLiINBTcG7tMPKqlVLjlK+T3TGiV2
         PAABtmx3NDP9O57iCoCds1XNoB5lqTODS6jjpGj+vcX8i3KnV/WdFdFSV+3ef9kKRs9T
         Um/A==
X-Gm-Message-State: AOAM531rrL5wJ+lZJNtEAgYw6XXA5y45Lr74qcpXmavItwb6O+do7XfQ
	S9ZOY8LzqfVCfwTPacUcNw4=
X-Google-Smtp-Source: ABdhPJxVc9r0FTKCTOqW8xIIldm9t05NRFuFTf+Kzcn3sLsTdbmtacHYR3pfbO5B7AXHWSOEzXQpFQ==
X-Received: by 2002:a05:600c:19ca:: with SMTP id u10mr3821810wmq.178.1630599453691;
        Thu, 02 Sep 2021 09:17:33 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 17/30] target/mips: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:30 +0200
Message-Id: <20210902161543.417092-18-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/mips/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/mips/cpu.c b/target/mips/cpu.c
index 00e0c55d0e4..3639c03f8ea 100644
--- a/target/mips/cpu.c
+++ b/target/mips/cpu.c
@@ -128,6 +128,7 @@ static void mips_cpu_set_pc(CPUState *cs, vaddr value)
     mips_env_set_pc(&cpu->env, value);
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool mips_cpu_has_work(CPUState *cs)
 {
     MIPSCPU *cpu = MIPS_CPU(cs);
@@ -172,6 +173,7 @@ static bool mips_cpu_has_work(CPUState *cs)
     }
     return has_work;
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 #include "cpu-defs.c.inc"
 
@@ -542,6 +544,7 @@ static const struct TCGCPUOps mips_tcg_ops = {
     .tlb_fill = mips_cpu_tlb_fill,
 
 #if !defined(CONFIG_USER_ONLY)
+    .has_work = mips_cpu_has_work,
     .cpu_exec_interrupt = mips_cpu_exec_interrupt,
     .do_interrupt = mips_cpu_do_interrupt,
     .do_transaction_failed = mips_cpu_do_transaction_failed,
@@ -563,7 +566,6 @@ static void mips_cpu_class_init(ObjectClass *c, void *data)
     device_class_set_props(dc, mips_cpu_properties);
 
     cc->class_by_name = mips_cpu_class_by_name;
-    cc->has_work = mips_cpu_has_work;
     cc->dump_state = mips_cpu_dump_state;
     cc->set_pc = mips_cpu_set_pc;
     cc->gdb_read_register = mips_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177492.322975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSw-0002lE-7h; Thu, 02 Sep 2021 16:21:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177492.322975; Thu, 02 Sep 2021 16:21:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSw-0002kp-30; Thu, 02 Sep 2021 16:21:26 +0000
Received: by outflank-mailman (input) for mailman id 177492;
 Thu, 02 Sep 2021 16:21:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPV-00029c-RN
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:53 +0000
Received: from mail-wr1-x436.google.com (unknown [2a00:1450:4864:20::436])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23c4919f-d8ba-48eb-ada9-6b5f71fd8ae6;
 Thu, 02 Sep 2021 16:17:49 +0000 (UTC)
Received: by mail-wr1-x436.google.com with SMTP id u16so3843166wrn.5
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:49 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 s15sm2202811wrb.22.2021.09.02.09.17.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 23c4919f-d8ba-48eb-ada9-6b5f71fd8ae6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=wiJyFwIKRQWjRkbu8SvGMr1PDMKMIYamMV7C3BB8oe8=;
        b=qAtoFIb8zeHmLJ+DXBMGpAdnbrphpEAfuI7XHEPw8NBmRW9/+WQnP+AZelTqnIhAyE
         XWWklQ0ZgUaW+TjGyBKgrmT5Awe/DhMqWt5iYbHcpZayB/1OpJXxLIVgIBvvvx9tbfbe
         MhBhX16/hQ/XQpM8/t7IWpZx8Wn0h5lsCbsMip4ljj1E3unFy90cJy4UzhkEFnhbknkU
         T8DR04fw7m7nWgaHWphfTdeObgKvp0ylXJqEGEN6mGrqmeM1GzS3/abyks0gzOofcGN7
         UjU06l66Hvh9mNoLvBN0ewV7GlShfdvdFlG07p47BqES4xriMHGxYVH/Zf/4POrSSA9s
         8XQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=wiJyFwIKRQWjRkbu8SvGMr1PDMKMIYamMV7C3BB8oe8=;
        b=TSgR5ffHhrhHIIW0U9o2qek/e8pXK11HMEHIRuYc4W2x67SZpZ2yWAfSR3x/RK4kZq
         EVglVcPm2RoTDG8nPxBCmtJAQ97MxJ0p2Uu/h3CDwC2bB/CkKMQScN+ZZFpyANFQOrG0
         mv3xejdE1URVO+giLkNaLVPKYAn1wgg8D4pHv7afsN62HFb6fX8bpXfOc/5PWe00NX9y
         hGFFTF90NUaPC51XgEmGSYp85kZ/0y4pwjTRZFELtka6PbEgZWq538KBwHL61KvgQ2gg
         gRV47F8SbBmIAGZLbzICiSEfX0orQ4QB03b15ddOa91yLSPWFkbQ4TUby14jmO1moxZj
         KcDw==
X-Gm-Message-State: AOAM533KiT6UL0hOmhezx0HibwF6YCmjW75ODLsITZNb/MlSL8JUWDys
	2M9R03qcdJ4CUI/gmgzQeiw=
X-Google-Smtp-Source: ABdhPJzK4QdOoxXsApVtGRumuSNyT10qkKd9QeSHbbUEQ3zItVBoCovI95FuxXm2STsnpfXGUzb9Vg==
X-Received: by 2002:a5d:4ed0:: with SMTP id s16mr4776030wrv.71.1630599468354;
        Thu, 02 Sep 2021 09:17:48 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 19/30] target/openrisc: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:32 +0200
Message-Id: <20210902161543.417092-20-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/openrisc/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
index 27cb04152f9..6544b549f12 100644
--- a/target/openrisc/cpu.c
+++ b/target/openrisc/cpu.c
@@ -30,11 +30,13 @@ static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.dflag = 0;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool openrisc_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD |
                                     CPU_INTERRUPT_TIMER);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void openrisc_disas_set_info(CPUState *cpu, disassemble_info *info)
 {
@@ -189,6 +191,7 @@ static const struct TCGCPUOps openrisc_tcg_ops = {
     .tlb_fill = openrisc_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = openrisc_cpu_has_work,
     .cpu_exec_interrupt = openrisc_cpu_exec_interrupt,
     .do_interrupt = openrisc_cpu_do_interrupt,
 #endif /* !CONFIG_USER_ONLY */
@@ -205,7 +208,6 @@ static void openrisc_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, openrisc_cpu_reset, &occ->parent_reset);
 
     cc->class_by_name = openrisc_cpu_class_by_name;
-    cc->has_work = openrisc_cpu_has_work;
     cc->dump_state = openrisc_cpu_dump_state;
     cc->set_pc = openrisc_cpu_set_pc;
     cc->gdb_read_register = openrisc_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177494.322985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSx-00034U-IY; Thu, 02 Sep 2021 16:21:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177494.322985; Thu, 02 Sep 2021 16:21:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpSx-000341-EA; Thu, 02 Sep 2021 16:21:27 +0000
Received: by outflank-mailman (input) for mailman id 177494;
 Thu, 02 Sep 2021 16:21:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpP1-00029c-QJ
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:23 +0000
Received: from mail-wm1-x32e.google.com (unknown [2a00:1450:4864:20::32e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 14cb6007-aed5-4a5e-98fd-b0c52838a1a8;
 Thu, 02 Sep 2021 16:17:16 +0000 (UTC)
Received: by mail-wm1-x32e.google.com with SMTP id
 k20-20020a05600c0b5400b002e87ad6956eso1751511wmr.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:16 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 c3sm2410025wrd.34.2021.09.02.09.17.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 14cb6007-aed5-4a5e-98fd-b0c52838a1a8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=k2G3IfvTSVM9dAo9x5r9x/QXUgni24b97nAmsANOOiA=;
        b=M+D+zo4/zdtMjYh01av69Zx4I3VwXPivjqKNQQ+G3Cy3iOhT3mTPvnPzXtpAFBC7IC
         bwDY97okiaHzzYZk/Nr+HJhmQWDbqsmtfcbNHfDxcB+wW5LVOcMLHk2ke6LUzT0b2vAG
         YqHMZm6pCnZF4sDmb5j0t8leoEPE4luIBbr+2Y/AHejZSaAIPRgw6F1l+RvTfdeQZQ/w
         KmEHFk+ys6eIeCl+10CsGxJv2WVk1ePshB2QpZEYQ9Rs3vY5QE1eYEOqT+aIoXIw8PR9
         WNsdSSeclnt9XXMY/k06ObG6uMJqiNqm40/pEaPsmniXZMMe5CRy4fWye4vqLWV5kU8p
         n7eA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=k2G3IfvTSVM9dAo9x5r9x/QXUgni24b97nAmsANOOiA=;
        b=ijz/wdJlkLFKFaCvoEFxFooZHM2zLtUxuCHt2lROYqeORIhVonBTd0qcajoXv50lM3
         jjcWcOHFS0R+zWlKbEbHRL0AWX1qFpv+4grQOf4XbUhxOShOKFbaQuhspQQXHuTeJj8M
         8SNqDdgc1il6EOYehLaRkRQwy3hWGiorW3czkmcWqbq9EaIzKHggkMRYPH2BQ9xC7uh/
         8RxWx1Dc0B/0KGCNa9XKupfsInZwwKQpiftJ30Ia8qzjjMcRuwVXhr/phs2okAf2755L
         jht8aAvSUjK2Qc+DkCubPL3x486Ph0DF4Er+7QzlT/FfaL7GsoIwt2wg8myWJnusj9aw
         f47Q==
X-Gm-Message-State: AOAM532utN0xyz6zq8QpT4AxXGsVsge9gIXOU2amkqBHs5sglX8X6GR5
	5yqVyn16JULC7FKbKC2wuvA=
X-Google-Smtp-Source: ABdhPJxFmnMfDsCiejdGeBE1y8KakVIkNHc7YRKHEHHLvGEXylx3fbGc1JSSwIBZyxQGjwxS5CvgPg==
X-Received: by 2002:a05:600c:1d27:: with SMTP id l39mr3986130wms.146.1630599431932;
        Thu, 02 Sep 2021 09:17:11 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 14/30] target/i386: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:27 +0200
Message-Id: <20210902161543.417092-15-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/i386/cpu.c         | 6 ------
 target/i386/tcg/tcg-cpu.c | 8 +++++++-
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 04f59043804..b7417d29f44 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6548,11 +6548,6 @@ int x86_cpu_pending_interrupt(CPUState *cs, int interrupt_request)
     return 0;
 }
 
-static bool x86_cpu_has_work(CPUState *cs)
-{
-    return x86_cpu_pending_interrupt(cs, cs->interrupt_request) != 0;
-}
-
 static void x86_disas_set_info(CPUState *cs, disassemble_info *info)
 {
     X86CPU *cpu = X86_CPU(cs);
@@ -6757,7 +6752,6 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->class_by_name = x86_cpu_class_by_name;
     cc->parse_features = x86_cpu_parse_featurestr;
-    cc->has_work = x86_cpu_has_work;
     cc->dump_state = x86_cpu_dump_state;
     cc->set_pc = x86_cpu_set_pc;
     cc->gdb_read_register = x86_cpu_gdb_read_register;
diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c
index fd86daf93d2..6cde53603ba 100644
--- a/target/i386/tcg/tcg-cpu.c
+++ b/target/i386/tcg/tcg-cpu.c
@@ -55,6 +55,11 @@ static void x86_cpu_synchronize_from_tb(CPUState *cs,
 }
 
 #ifndef CONFIG_USER_ONLY
+static bool x86_cpu_has_work(CPUState *cs)
+{
+    return x86_cpu_pending_interrupt(cs, cs->interrupt_request) != 0;
+}
+
 static bool x86_debug_check_breakpoint(CPUState *cs)
 {
     X86CPU *cpu = X86_CPU(cs);
@@ -63,7 +68,7 @@ static bool x86_debug_check_breakpoint(CPUState *cs)
     /* RF disables all architectural breakpoints. */
     return !(env->eflags & RF_MASK);
 }
-#endif
+#endif /* CONFIG_USER_ONLY */
 
 #include "hw/core/tcg-cpu-ops.h"
 
@@ -76,6 +81,7 @@ static const struct TCGCPUOps x86_tcg_ops = {
 #ifdef CONFIG_USER_ONLY
     .fake_user_exception = x86_cpu_do_interrupt,
 #else
+    .has_work = x86_cpu_has_work,
     .do_interrupt = x86_cpu_do_interrupt,
     .cpu_exec_interrupt = x86_cpu_exec_interrupt,
     .debug_excp_handler = breakpoint_handler,
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177497.322997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpT2-0003eh-SG; Thu, 02 Sep 2021 16:21:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177497.322997; Thu, 02 Sep 2021 16:21:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpT2-0003eS-Ov; Thu, 02 Sep 2021 16:21:32 +0000
Received: by outflank-mailman (input) for mailman id 177497;
 Thu, 02 Sep 2021 16:21:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPa-00029c-Ra
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:58 +0000
Received: from mail-wm1-x32c.google.com (unknown [2a00:1450:4864:20::32c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 59d35fc3-694e-4ea9-a48e-20ca3fdb9051;
 Thu, 02 Sep 2021 16:17:55 +0000 (UTC)
Received: by mail-wm1-x32c.google.com with SMTP id
 l7-20020a1c2507000000b002e6be5d86b3so1747123wml.3
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:55 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 f7sm2089431wmh.20.2021.09.02.09.17.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 59d35fc3-694e-4ea9-a48e-20ca3fdb9051
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=KB6w4zAa3ScKbWvcSeRs6nml5mmvjgABbPo/s2Hl9mE=;
        b=KudZd4akeMvkEQWUbX6szQOXRBHrwcFxsejJsDwXAwrTkhmJGlULM7aiBntpU6DlLD
         x1G792oH76CFsyLlbuYcUF0GY8rXRo+VR9Hu/GBgHJyiVzTP3OvIC53DxbyFxm7deDsy
         jywij08/HNg0mFST30fvOpvMZOkJgu6uZtVjcnTVWcKz4cejoxRKoHPRUUanT/GYimTW
         BuEEvjzL5rlc0uwntqWn8CT2tidHHSSa+1LvLL/36Uq3AShOfMPK6XKiF4/niGHWKMJq
         4jkolFQjzFn6TC5LBuIhdKNpLtstdk7EfVBatQiEEjt66rdtDDU/Oqn47gGLe726oHXk
         a0tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=KB6w4zAa3ScKbWvcSeRs6nml5mmvjgABbPo/s2Hl9mE=;
        b=S+ZBgS6+rjjUm7iKZGdfQttyEEVGsKTgcmP2G+1h5fdQB5HhKZfHB+j+3vV74V0vQk
         CvT3QJabTN8cQxBH6ON5N+1/PgJ9OKQSuHVUvGYkkUENfxm3VN/K7kbi/+q2xIEZh3O6
         /AMbcrs1VvKnZbrhLrlMM21JMIZMu7/dzDJcC3Z0v3irj411l+WtgJCVstR87xn2sTnT
         8JjisYxcBCIOy9wDWmygwTcLrEUYISYHyUidblw0b7Hpm9WVmjFuF2dChSrus8bWB3tA
         M0FeD6Su/CH89yvztpsIMcIn81BFMtfE8VTvzlHm4DHPpVvqPjXobS2adkGLM+XN8sfR
         oIjw==
X-Gm-Message-State: AOAM530lRcx6VXovEkQJ9D9jhQBlfovd6E+E1Sa8FAGoYtk5SA2rQY0X
	vPfvzM14WeSknC2C5DXX0dE=
X-Google-Smtp-Source: ABdhPJx3cFt1jrbYXnlPCjqrJf+8cwfXAXn4EeP6vsDC0dCbWo+FdkLPkzBtWW3nLTZkBTFw2ib2sw==
X-Received: by 2002:a1c:ed10:: with SMTP id l16mr3994398wmh.8.1630599474350;
        Thu, 02 Sep 2021 09:17:54 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 20/30] target/ppc: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:33 +0200
Message-Id: <20210902161543.417092-21-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/ppc/cpu_init.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 6aad01d1d3a..e2e721c2b81 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -8790,6 +8790,7 @@ static void ppc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.nip = value;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool ppc_cpu_has_work(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -8797,6 +8798,7 @@ static bool ppc_cpu_has_work(CPUState *cs)
 
     return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void ppc_cpu_reset(DeviceState *dev)
 {
@@ -9017,6 +9019,7 @@ static const struct TCGCPUOps ppc_tcg_ops = {
   .tlb_fill = ppc_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+  .has_work = ppc_cpu_has_work,
   .cpu_exec_interrupt = ppc_cpu_exec_interrupt,
   .do_interrupt = ppc_cpu_do_interrupt,
   .cpu_exec_enter = ppc_cpu_exec_enter,
@@ -9042,7 +9045,6 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, ppc_cpu_reset, &pcc->parent_reset);
 
     cc->class_by_name = ppc_cpu_class_by_name;
-    cc->has_work = ppc_cpu_has_work;
     cc->dump_state = ppc_cpu_dump_state;
     cc->set_pc = ppc_cpu_set_pc;
     cc->gdb_read_register = ppc_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177513.323008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTG-0004lg-5b; Thu, 02 Sep 2021 16:21:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177513.323008; Thu, 02 Sep 2021 16:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTG-0004lW-2J; Thu, 02 Sep 2021 16:21:46 +0000
Received: by outflank-mailman (input) for mailman id 177513;
 Thu, 02 Sep 2021 16:21:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOc-00029c-PS
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:16:58 +0000
Received: from mail-wm1-x334.google.com (unknown [2a00:1450:4864:20::334])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3adcf5b9-68b6-4973-9aaf-67106bde2e85;
 Thu, 02 Sep 2021 16:16:48 +0000 (UTC)
Received: by mail-wm1-x334.google.com with SMTP id
 v20-20020a1cf714000000b002e71f4d2026so1424714wmh.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:48 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 y24sm2479386wma.9.2021.09.02.09.16.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 3adcf5b9-68b6-4973-9aaf-67106bde2e85
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=1uiIX4B4XnBDTSuiPfQ6TXqVM80IFaG7yss59sugPek=;
        b=KqzWIKJ7D2fRmpGndmFmFNuwslKTxikjGxDMGA2xsPlBUegPAHynuzDusdrr5XjA+x
         UCJbwBQeZPJQtXG0Xtu7RxwzPdlThK6RVv2DMQ/DDE16jxbu2MLTQ+EKf6s/CZXPXeN6
         nzJIGn8UESvClj1N6xIA9j+rSpvqLJiMXVoYewPiMa355LZW9o2GW/afajWJc0/NEar0
         Q8DpjYGYUJuF0UC+706nFPlDCV62j7IN2KHM9wzg02+ZUE7+nf9cE7ogevav/S8CwMh4
         Jx4C3Bwaygo71B9Uga55OA34fqh3F6hfjdQCFZRtq0AWQ50+fnYTHxBfr074WpLdgF+1
         YThQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=1uiIX4B4XnBDTSuiPfQ6TXqVM80IFaG7yss59sugPek=;
        b=OcR6OYbN3d3+2/g4T4hIfhIUOPuSgrlIYkvMrSCCKVcwHrOzkhyTj27/29v92exeyX
         7JPELEhrW92u2JkyyYxEM+0T2A74FrU0I6kwO2v9goJHE4rl8EijW/t/W80Gz8+5ffe5
         XL/6yRk1dxlLX8JjN6Hx2z3MVNokbW2k64OTHcgMfPOWhpGvBvjuTOOTEX+RJH0WRYER
         gFrToV5q5PaGzh7O7eQ6O/M8sjhff6HLSeyLgHLgDjajIsfVPS4/uGqZQu+6Jt8qfK0E
         b4L8hjAofmc4J7d5ie3NpV5R5y+1fFb4sG+0JxUDgGhwIVcqmWrP8HqKjIwYdf+HJ5ep
         K7vw==
X-Gm-Message-State: AOAM530wdLK2ToSAzMr/x8CGIs4lvGpJNhADcJQV71/gfrit0Y9X73os
	IgPLuecZ3K8CHNnkONAn/jA=
X-Google-Smtp-Source: ABdhPJw3pSsMq4kZKK4hbVaJZjnjyoQFK4vJQiZ4XHFwYg2jDsiLD33cM2YA1HEu8cnvL7L44Mu7XQ==
X-Received: by 2002:a1c:44c5:: with SMTP id r188mr4089427wma.9.1630599407288;
        Thu, 02 Sep 2021 09:16:47 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 10/30] target/avr: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:23 +0200
Message-Id: <20210902161543.417092-11-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/avr/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/avr/cpu.c b/target/avr/cpu.c
index e9fa54c9777..6267cc6d530 100644
--- a/target/avr/cpu.c
+++ b/target/avr/cpu.c
@@ -32,6 +32,7 @@ static void avr_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc_w = value / 2; /* internally PC points to words */
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool avr_cpu_has_work(CPUState *cs)
 {
     AVRCPU *cpu = AVR_CPU(cs);
@@ -40,6 +41,7 @@ static bool avr_cpu_has_work(CPUState *cs)
     return (cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_RESET))
             && cpu_interrupts_enabled(env);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void avr_cpu_synchronize_from_tb(CPUState *cs,
                                         const TranslationBlock *tb)
@@ -198,6 +200,7 @@ static const struct TCGCPUOps avr_tcg_ops = {
     .tlb_fill = avr_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = avr_cpu_has_work,
     .cpu_exec_interrupt = avr_cpu_exec_interrupt,
     .do_interrupt = avr_cpu_do_interrupt,
 #endif /* !CONFIG_USER_ONLY */
@@ -214,7 +217,6 @@ static void avr_cpu_class_init(ObjectClass *oc, void *data)
 
     cc->class_by_name = avr_cpu_class_by_name;
 
-    cc->has_work = avr_cpu_has_work;
     cc->dump_state = avr_cpu_dump_state;
     cc->set_pc = avr_cpu_set_pc;
     cc->memory_rw_debug = avr_cpu_memory_rw_debug;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177515.323017 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTH-0004vN-0Y; Thu, 02 Sep 2021 16:21:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177515.323017; Thu, 02 Sep 2021 16:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTG-0004s5-Kd; Thu, 02 Sep 2021 16:21:46 +0000
Received: by outflank-mailman (input) for mailman id 177515;
 Thu, 02 Sep 2021 16:21:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOw-00029c-Q6
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:18 +0000
Received: from mail-wr1-x430.google.com (unknown [2a00:1450:4864:20::430])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c8146775-0b3e-4bd6-a71e-55562e409a43;
 Thu, 02 Sep 2021 16:17:06 +0000 (UTC)
Received: by mail-wr1-x430.google.com with SMTP id i6so3844396wrv.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:06 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 y4sm2068566wmi.22.2021.09.02.09.17.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: c8146775-0b3e-4bd6-a71e-55562e409a43
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=Pk3U565FofhMNBdH085OA9aU3aTnCsU54QbFBxwfMI8=;
        b=NaB65rpK4UK7+XbBO28oZxFusXeQWuGTndzQkKqs4x497OOBypBPU5uctaWGbkSibX
         gbzbf0uKBllL7IMK9jeerTaWXdNmmwfHar32fZ/NCC0Cj626RXCkU3d8hlfsPo25otel
         OL/nvXSML0UuV9hTKCtQARxXaxraFStiKn4xGZp+OALpmFXy+6sqXkWFwBJFR6EcXMpL
         +gEhWt3eW2TEXJjoPvyiNmFQPJOj0yOdhR7ZZu+zEVqND+25gS72HJlWm9AWHklxn9DD
         TkIjI2aJpOFpKyf44h+zUsWbE/UTizlwAkVSUgam/bJtjGLksKc8rvZbllkWj5mw988x
         UXUw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=Pk3U565FofhMNBdH085OA9aU3aTnCsU54QbFBxwfMI8=;
        b=MCyUV1oJcs5VGlyrKB6nHbQvPEcHLbkgS6FqfKTM07X9tN4LRSXo2L7t/6v08mYZUM
         1zA3wZtlulY5SkwR0bi8iBprxvHtzCEi0PwG3BKJX2isXumCYXpLXFL7pF3riq7jqxCR
         7qe8KB9hfcRi69cKyNWYwiz/99jRvXFqvQP78lBKIBkx4oOI/7f9NlGF/jWfHmMtvGTO
         sFaVIuKwFYythL74Csn0M9z8tdnmVJ9vEKp0zXgrxX9AcGBB0fl/2T66IJVyYlzP0Sox
         bbzAL8tWjKaRifdEK5Xci94JZBU3EN3XMfV8JGQLiJhLh/QYWzbH1yamHuGlKK8feF3R
         QomA==
X-Gm-Message-State: AOAM530gW0JLHBVyXLlcCjTOjML/xnUvFW8hfrITgPcLZzNxU2WxL6vb
	/UggOvOqfQ3++rdP8zQTt+E=
X-Google-Smtp-Source: ABdhPJxJ0WzmriUMp5BLtv/IKodEKm5HtzrJCRMHsGjbOs/WMzY0lZa1I+orMo+J9P/Vgaj+G+An4w==
X-Received: by 2002:adf:f991:: with SMTP id f17mr4751225wrr.56.1630599425756;
        Thu, 02 Sep 2021 09:17:05 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 13/30] target/hppa: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:26 +0200
Message-Id: <20210902161543.417092-14-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/hppa/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
index e8edd189bfc..cf1f656218f 100644
--- a/target/hppa/cpu.c
+++ b/target/hppa/cpu.c
@@ -60,10 +60,12 @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
     cpu->env.psw_n = (tb->flags & PSW_N) != 0;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool hppa_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & CPU_INTERRUPT_HARD;
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void hppa_cpu_disas_set_info(CPUState *cs, disassemble_info *info)
 {
@@ -147,6 +149,7 @@ static const struct TCGCPUOps hppa_tcg_ops = {
     .tlb_fill = hppa_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = hppa_cpu_has_work,
     .cpu_exec_interrupt = hppa_cpu_exec_interrupt,
     .do_interrupt = hppa_cpu_do_interrupt,
     .do_unaligned_access = hppa_cpu_do_unaligned_access,
@@ -163,7 +166,6 @@ static void hppa_cpu_class_init(ObjectClass *oc, void *data)
                                     &acc->parent_realize);
 
     cc->class_by_name = hppa_cpu_class_by_name;
-    cc->has_work = hppa_cpu_has_work;
     cc->dump_state = hppa_cpu_dump_state;
     cc->set_pc = hppa_cpu_set_pc;
     cc->gdb_read_register = hppa_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177516.323029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTI-0005MF-63; Thu, 02 Sep 2021 16:21:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177516.323029; Thu, 02 Sep 2021 16:21:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTI-0005M0-19; Thu, 02 Sep 2021 16:21:48 +0000
Received: by outflank-mailman (input) for mailman id 177516;
 Thu, 02 Sep 2021 16:21:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpOh-00029c-Pg
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:03 +0000
Received: from mail-wr1-x434.google.com (unknown [2a00:1450:4864:20::434])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 883c0791-ab03-401b-9eb3-be2241ff3bec;
 Thu, 02 Sep 2021 16:16:54 +0000 (UTC)
Received: by mail-wr1-x434.google.com with SMTP id v10so3835568wrd.4
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:16:54 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 l35sm1840348wms.40.2021.09.02.09.16.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:16:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 883c0791-ab03-401b-9eb3-be2241ff3bec
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ealGhID5/ZwlyOxoKUyhT8abP5eBfK+uMSNnBQgE1TY=;
        b=W7wWIRu899u62PiYNAofijA+z0Fl13DLdrj//JnOtB9nuav7HgFA25Nkb7zPP9qx0q
         V+T6UyuHV8lfYhIsfl6T3reP8G73iXnJAJA0ZI5pdqQK3b2QENsciaKQFL/VhPrWDk3o
         V9xTH3D7YAWgG+OE8yKD9gO/syxb0LKTwtCgItyeMl0A+kX7M5ZtD+Rg6R+o7Cnq1P8l
         69E8lhjYEQcYXk+iNCzjjt0R2+v6w/ULq5UCkJupima5EWr+YrbDz2jjvylypCOx3Xrd
         ioW7y8oVW/o1/Fd4QRhzX/tp9G+m2C+2tAs7vFR9FJH4g8+0Rq+iX11ED1zseb/+uUtD
         FA2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=ealGhID5/ZwlyOxoKUyhT8abP5eBfK+uMSNnBQgE1TY=;
        b=sdqiUu04J8O++JRqkvbqba+MFb5mXzFJUvwx7Yjw2LPEU/mBmB4QCTy5tpYQZTGr5h
         z3o4cUEFHl56rSKVzxPW80yiS83nRqpDV6J6NgJm3cKAhXh9M76GE6wkQqeaF9XUUM0V
         P3roardmwRDG+4ccztiKiu5zkp6jigyMBK7mh3xdCiuZ3pD4QtNoiNrHeY5LcJwM8nBw
         ExFM244J8mbjmXBhXGv+AvHPmRPcRlmLqoNVvjzI017D3wFnIZNe9kZbzf6KgtxM1prE
         AVkcvkTik8axI+iNci3QtK4VWrGjGAS0BZnHkNqkPKodYNEuaXH+U1xmVW3HgYuo2tPi
         0I3w==
X-Gm-Message-State: AOAM532TD803QR+83lQ51rGFrC6Wm2eGNV2BkK5aa/sAmeFG80Z45fzZ
	F+J/sL1sXGupcd8bBlxjhJg=
X-Google-Smtp-Source: ABdhPJw8/hywRF+QG+EoxPvaacbpI8TbV6wFHFx7ujXzHTlr6qNAq926vGsYV8Cac2S8ECAaju7k7A==
X-Received: by 2002:adf:916f:: with SMTP id j102mr4715417wrj.422.1630599413462;
        Thu, 02 Sep 2021 09:16:53 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 11/30] target/cris: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:24 +0200
Message-Id: <20210902161543.417092-12-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/cris/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/cris/cpu.c b/target/cris/cpu.c
index c2e7483f5bd..d6e486746be 100644
--- a/target/cris/cpu.c
+++ b/target/cris/cpu.c
@@ -35,10 +35,12 @@ static void cris_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool cris_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void cris_cpu_reset(DeviceState *dev)
 {
@@ -208,6 +210,7 @@ static const struct TCGCPUOps crisv10_tcg_ops = {
     .tlb_fill = cris_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = cris_cpu_has_work,
     .cpu_exec_interrupt = cris_cpu_exec_interrupt,
     .do_interrupt = crisv10_cpu_do_interrupt,
 #endif /* !CONFIG_USER_ONLY */
@@ -294,7 +297,6 @@ static void cris_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, cris_cpu_reset, &ccc->parent_reset);
 
     cc->class_by_name = cris_cpu_class_by_name;
-    cc->has_work = cris_cpu_has_work;
     cc->dump_state = cris_cpu_dump_state;
     cc->set_pc = cris_cpu_set_pc;
     cc->gdb_read_register = cris_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177521.323040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTM-0005v4-KJ; Thu, 02 Sep 2021 16:21:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177521.323040; Thu, 02 Sep 2021 16:21:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTM-0005um-F1; Thu, 02 Sep 2021 16:21:52 +0000
Received: by outflank-mailman (input) for mailman id 177521;
 Thu, 02 Sep 2021 16:21:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPG-00029c-R3
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:38 +0000
Received: from mail-wm1-x329.google.com (unknown [2a00:1450:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 50d9668c-5273-4f42-b2c0-07276964ddd5;
 Thu, 02 Sep 2021 16:17:28 +0000 (UTC)
Received: by mail-wm1-x329.google.com with SMTP id
 m25-20020a7bcb99000000b002e751bcb5dbso1734355wmi.5
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:28 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 f18sm2080257wmc.6.2021.09.02.09.17.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 50d9668c-5273-4f42-b2c0-07276964ddd5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=DdBo63POokH5fIMDIDGd3vZfaCJQwybOZKNS3BGh4nM=;
        b=SdUILgJd3l3KkosL5d+GJCCbjxa1kuvIi2eQH+QhbRZsxGZZeyT8B45EfAvb4vDqID
         Hvz12f+uLhG2sxeYjZh1eHobRoGFGmx8Vwc0IlXnnkf2IX5bx+BVhSr1cY3Ej8rlB7KC
         YO8ak4ncoqL6BecyY/e6YiTBs4ooYjlzwmmJGuU0TYXlpRik+gmohfcoQGfINeBVNh7H
         gfydQO5CesTCgYv9jzBSAmfhKaEn9YTmVKuXA9R7D2QLLI+i2p5DKESCrSigkULnbdYP
         q+rTwC0UOLH6fDszBS6ixFPljdENz9/dOQt5v2rlttm82OpvBLF+yYDt5s7Pj/jtLcYS
         Xgng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=DdBo63POokH5fIMDIDGd3vZfaCJQwybOZKNS3BGh4nM=;
        b=gscGy/7Izd5tcvChASYjY3Xc46zWncdfnQm4CzL2WrMP8G6U3kt8LJU1buyb08D0Au
         o8ydrhFGgwqZD86ffkp3iiDYfsplKbZ08Kwz7RhnwlBHn9meO0G972QB2nfY7coC5RIV
         aBjm1fn1pjlNQXEEFa+RKEzVVplsquUMFITBrqZWqSNpAI2PnMQZkw1vofyv6AP2fmp6
         qmA8YEtTIIx2SwFiNT7FPpHBRXokMh4MlPMkZwxIfoHt+dEpe9Jv0Pn1647pAB/FRsUn
         PlJQGgiFBIi54r4ilFGJ/xa35iETfHoG6TyUrp9f0pnb8JoENVtmF+VzrCSZztxkZZyd
         DayQ==
X-Gm-Message-State: AOAM533+JKAYooPg94eIiSAwa4nfsMEvvY9UbxGn7aKmYtLGHeECTnhx
	Mj/qOOr1U9TzLJUtnmr8E+0=
X-Google-Smtp-Source: ABdhPJxeXOWsxsq5sHAGX+bjrNwp1kbgUUARJY9Fvv8rdRD0ZNZFO5Z6zZ0qv0LrIPXbrhuPF3RRHw==
X-Received: by 2002:a1c:a50c:: with SMTP id o12mr4082621wme.4.1630599447661;
        Thu, 02 Sep 2021 09:17:27 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 16/30] target/microblaze: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:29 +0200
Message-Id: <20210902161543.417092-17-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/microblaze/cpu.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
index 15db277925f..74fbb5d201a 100644
--- a/target/microblaze/cpu.c
+++ b/target/microblaze/cpu.c
@@ -92,12 +92,15 @@ static void mb_cpu_synchronize_from_tb(CPUState *cs,
     cpu->env.iflags = tb->flags & IFLAGS_TB_MASK;
 }
 
+#ifndef CONFIG_USER_ONLY
+
+#ifdef CONFIG_TCG
 static bool mb_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
 }
+#endif /* CONFIG_TCG */
 
-#ifndef CONFIG_USER_ONLY
 static void mb_cpu_ns_axi_dp(void *opaque, int irq, int level)
 {
     MicroBlazeCPU *cpu = opaque;
@@ -142,7 +145,7 @@ static void microblaze_cpu_set_irq(void *opaque, int irq, int level)
         cpu_reset_interrupt(cs, type);
     }
 }
-#endif
+#endif /* !CONFIG_USER_ONLY */
 
 static void mb_cpu_reset(DeviceState *dev)
 {
@@ -368,6 +371,7 @@ static const struct TCGCPUOps mb_tcg_ops = {
     .tlb_fill = mb_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = mb_cpu_has_work,
     .cpu_exec_interrupt = mb_cpu_exec_interrupt,
     .do_interrupt = mb_cpu_do_interrupt,
     .do_transaction_failed = mb_cpu_transaction_failed,
@@ -386,8 +390,6 @@ static void mb_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, mb_cpu_reset, &mcc->parent_reset);
 
     cc->class_by_name = mb_cpu_class_by_name;
-    cc->has_work = mb_cpu_has_work;
-
     cc->dump_state = mb_cpu_dump_state;
     cc->set_pc = mb_cpu_set_pc;
     cc->gdb_read_register = mb_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:21:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:21:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177522.323051 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTN-0006EG-Uq; Thu, 02 Sep 2021 16:21:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177522.323051; Thu, 02 Sep 2021 16:21:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTN-0006Di-QM; Thu, 02 Sep 2021 16:21:53 +0000
Received: by outflank-mailman (input) for mailman id 177522;
 Thu, 02 Sep 2021 16:21:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpPQ-00029c-RL
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:48 +0000
Received: from mail-wm1-x330.google.com (unknown [2a00:1450:4864:20::330])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 35337728-37d5-4f15-99e8-bf5d5c0bc7c4;
 Thu, 02 Sep 2021 16:17:42 +0000 (UTC)
Received: by mail-wm1-x330.google.com with SMTP id
 k20-20020a05600c0b5400b002e87ad6956eso1752432wmr.1
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:42 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 b10sm2396452wrt.43.2021.09.02.09.17.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 35337728-37d5-4f15-99e8-bf5d5c0bc7c4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=zZDEgfpsGMQvs/Uf9L14U8iNfhU1RYAtDHEnPo7PYYM=;
        b=GULtcfLvxCYjpD88B1lNFiQq0+Fx5WhxaqykU8SQ2CHcFSQ46Oe+PaWfXkceFQ4mZo
         9SVUhNZ9o3T9wPipJYMj4yipSNtMQdK9k6FheUrkcj3bklOdIm+96woKK1z756yLjsg/
         MhcJnDjB6bDPTEVYbHN4OuhakMfbQHLPwIuQo7fhJI4LnxL+hhUixbbozhsjDk51Ju96
         gEVW+K7GtzvK1ZSbUGOUZzNQwoBSJHeiBUhu/bkunqvaYTW0krBKCnLFihT41xPY7/te
         UIojuKoYb6sy0krnR6uivIpg05XXUq83++TFLX9QNOJMrVv7jBEaRQx/6ic9TCOEtLyT
         HNGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=zZDEgfpsGMQvs/Uf9L14U8iNfhU1RYAtDHEnPo7PYYM=;
        b=s/ZlfdL2kPbg27cSacwE2l9+XTd6UgsmCFwLHqX5ZQEBOyLQZ/ih5yVl8LVtOJPBgY
         pHw6FDTkAyjPcQeCQcbneLIoxnPrTBswd8JJCWURFMLaFpQKDMlyE4lH6g9okWwlhge1
         PK/9qqBIjHm70UUj8XTCI0Tll+XqFWPHLUonS2d2vGfK9Jh9GR0V6OhPtHSnc2kVYGk4
         ouke6U000KZXsX9cDfD4101o8PlkaBRky+ahJZHOdB53WctZWPGLQIMJHADQtoVqPV0L
         ZC8iZdviq7Rhk1FxI+1BduhFztCxVXKGcWBeN8Nm7JcxgMieyL4ENl0N0dLivNDXWFjV
         oS9A==
X-Gm-Message-State: AOAM532+81vKHQ8Y6U/8zPktKodLsiFSkERZsoIO4SC4OfAr9ikDjtKG
	oJlLu2QMYb9U2hSSP+Eps1s=
X-Google-Smtp-Source: ABdhPJyVfC2B9mnkyBEM+dhp2QSoSDx2mJAJ4q0c4MKaFScZUxxMUDRDRaRsAykGgf3YQuNS/WTfdA==
X-Received: by 2002:a05:600c:3641:: with SMTP id y1mr3991806wmq.181.1630599461974;
        Thu, 02 Sep 2021 09:17:41 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 18/30] target/nios2: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:31 +0200
Message-Id: <20210902161543.417092-19-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/nios2/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/nios2/cpu.c b/target/nios2/cpu.c
index 947bb09bc1e..f1f976bdad7 100644
--- a/target/nios2/cpu.c
+++ b/target/nios2/cpu.c
@@ -34,10 +34,12 @@ static void nios2_cpu_set_pc(CPUState *cs, vaddr value)
     env->regs[R_PC] = value;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool nios2_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void nios2_cpu_reset(DeviceState *dev)
 {
@@ -223,6 +225,7 @@ static const struct TCGCPUOps nios2_tcg_ops = {
     .tlb_fill = nios2_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = nios2_cpu_has_work,
     .cpu_exec_interrupt = nios2_cpu_exec_interrupt,
     .do_interrupt = nios2_cpu_do_interrupt,
     .do_unaligned_access = nios2_cpu_do_unaligned_access,
@@ -241,7 +244,6 @@ static void nios2_cpu_class_init(ObjectClass *oc, void *data)
     device_class_set_parent_reset(dc, nios2_cpu_reset, &ncc->parent_reset);
 
     cc->class_by_name = nios2_cpu_class_by_name;
-    cc->has_work = nios2_cpu_has_work;
     cc->dump_state = nios2_cpu_dump_state;
     cc->set_pc = nios2_cpu_set_pc;
     cc->disas_set_info = nios2_cpu_disas_set_info;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:22:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:22:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177530.323063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTU-0006zM-FI; Thu, 02 Sep 2021 16:22:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177530.323063; Thu, 02 Sep 2021 16:22:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpTU-0006yx-AC; Thu, 02 Sep 2021 16:22:00 +0000
Received: by outflank-mailman (input) for mailman id 177530;
 Thu, 02 Sep 2021 16:21:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c9UA=NY=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mLpP6-00029c-QX
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:17:28 +0000
Received: from mail-wm1-x331.google.com (unknown [2a00:1450:4864:20::331])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9ef57b0c-3db4-49db-ab0a-e88961aeea9d;
 Thu, 02 Sep 2021 16:17:21 +0000 (UTC)
Received: by mail-wm1-x331.google.com with SMTP id
 o39-20020a05600c512700b002e74638b567so1756535wms.2
 for <xen-devel@lists.xenproject.org>; Thu, 02 Sep 2021 09:17:21 -0700 (PDT)
Received: from x1w.. (163.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.163])
 by smtp.gmail.com with ESMTPSA id
 u26sm2444655wrd.32.2021.09.02.09.17.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 02 Sep 2021 09:17:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 9ef57b0c-3db4-49db-ab0a-e88961aeea9d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=yI9HNtTgyr1D6LnyMBdjwFDJD3B27ntrVyNfg6pObmA=;
        b=U2uwqAQNdpnlzp+ESr25gYYFRQFiKkwGfbYBdEJEhK8WvNyB3W6SW/d8vS39ShykYt
         KjW8bmI0A51yksjJsztk3PnbWgrojrKVVqZEuWCXsll7maIHDOE+TUvhByg6eVxV2YO4
         Kck+INFoCYRSpbSsv9Whw88dorhpd2Vg3v9+eB5JpbKAbAeW/R5bh44TF5Lf3uEy+zLL
         RGSPDOO8yKxtK+VZKvOfcEExgvyvZTiR2XR3sSyBG6KljCmSQBYEj0d3C/Q4ABoW/H1Q
         /jDgwuiymH/q6Oj2C76+I+aEeA3QCh3LaEPc/5XOWBjjrnq8mhBxG0fKOPLpG6nd3yVW
         dFLA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=yI9HNtTgyr1D6LnyMBdjwFDJD3B27ntrVyNfg6pObmA=;
        b=Xm2kxRf5b4nwVM4gGhWtQqp68SCBHOhTxYxJ8VlHoTytT9r4qqYixLvzReNpDul4Ry
         0boISPHQSzecnaQ+rzvjeR+J0+kn2c4KG3LQWRLoLZtO+VsNAVSY4Bg7/WkcMSuAaDGl
         pjn1ldU4Jlqh64xtbqAwOoU5xjKqzzH9nKReC72Im6YP20DvJ8kl4RimmkihacX7REmV
         WXkI/+5nR4/poz2kX6cSqRdAmMIVf+b6bEsI95kZ2WHuYaV1hSSvcAm0dwCeyOHtyvgW
         OL66QDNw+EcrpFGw7/g+brfC9EU5DLqLgGTIUhP5jKOKoMX6XA4q/nLMMiZM4cnltVC/
         hIkA==
X-Gm-Message-State: AOAM5302pTLxcA5m9/9HpruRKO7jvmNMxMdcrEhqr3iPMGBstRVlvs2/
	KDLvX4jzaQq4b57KpD8uW/w=
X-Google-Smtp-Source: ABdhPJxszMtpghtVMFzBIiCctVAaFhaKkfegWVJMuLYTNpt1RcE+DF7+wgJu132GyNXvqWu6eKFzOQ==
X-Received: by 2002:a05:600c:3397:: with SMTP id o23mr3987673wmp.38.1630599441102;
        Thu, 02 Sep 2021 09:17:21 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org,
	Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>,
	qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>,
	qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	David Gibson <david@gibson.dropbear.id.au>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	xen-devel@lists.xenproject.org,
	Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>,
	Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	kvm@vger.kernel.org
Subject: [PATCH v3 15/30] target/m68k: Restrict has_work() handler to sysemu and TCG
Date: Thu,  2 Sep 2021 18:15:28 +0200
Message-Id: <20210902161543.417092-16-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Restrict has_work() to TCG sysemu.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 target/m68k/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
index 66d22d11895..94b35cb4a50 100644
--- a/target/m68k/cpu.c
+++ b/target/m68k/cpu.c
@@ -31,10 +31,12 @@ static void m68k_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool m68k_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & CPU_INTERRUPT_HARD;
 }
+#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
 
 static void m68k_set_feature(CPUM68KState *env, int feature)
 {
@@ -518,6 +520,7 @@ static const struct TCGCPUOps m68k_tcg_ops = {
     .tlb_fill = m68k_cpu_tlb_fill,
 
 #ifndef CONFIG_USER_ONLY
+    .has_work = m68k_cpu_has_work,
     .cpu_exec_interrupt = m68k_cpu_exec_interrupt,
     .do_interrupt = m68k_cpu_do_interrupt,
     .do_transaction_failed = m68k_cpu_transaction_failed,
@@ -535,7 +538,6 @@ static void m68k_cpu_class_init(ObjectClass *c, void *data)
     device_class_set_parent_reset(dc, m68k_cpu_reset, &mcc->parent_reset);
 
     cc->class_by_name = m68k_cpu_class_by_name;
-    cc->has_work = m68k_cpu_has_work;
     cc->dump_state = m68k_cpu_dump_state;
     cc->set_pc = m68k_cpu_set_pc;
     cc->gdb_read_register = m68k_cpu_gdb_read_register;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:36:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:36:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177565.323074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpht-0001mp-0d; Thu, 02 Sep 2021 16:36:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177565.323074; Thu, 02 Sep 2021 16:36:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLphs-0001mi-Ty; Thu, 02 Sep 2021 16:36:52 +0000
Received: by outflank-mailman (input) for mailman id 177565;
 Thu, 02 Sep 2021 16:36:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLphr-0001mc-M7
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:36:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLphr-0000ar-K7
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:36:51 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLphr-0000hc-JB
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:36:51 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLphm-0003SM-KY; Thu, 02 Sep 2021 17:36:46 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=D/t8Q47Sef2DkUJLCWNguQpG5Ves3E3v3+8vkz9f+9w=; b=MUZhn7bLOMouoGQJM3a8D7Xqfi
	lYx0BAxO1Pa3m4D1asDTOLZWA5qwrJoWXGRv9joHM2p+HuAbOrkWNck4LC+Z6Z7rwrHzgLxpUdt+U
	rRGLwHCB89pO8FwYcj7/eQcjUsqpSw138WJ94wslW1YjASUOYJv3NgPoX/zsxmeJx2l8=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24880.64926.93632.330726@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 17:36:46 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
    Wei Liu <wl@xen.org>,
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v2] tests/xenstore: link in librt if necessary
In-Reply-To: <c5df4be6-22e5-5f31-9ee5-1bc3b77b9ded@suse.com>
References: <3efcfc3d-54c8-7308-2288-9f57ed0fca0f@suse.com>
	<c5df4be6-22e5-5f31-9ee5-1bc3b77b9ded@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v2] tests/xenstore: link in librt if necessary"):
> On 17.08.21 16:18, Jan Beulich wrote:
> > Old enough glibc has clock_gettime() in librt.so, hence the library
> > needs to be specified to the linker. Newer glibc has the symbol
> > available in both libraries, so make sure that libc.so is preferred (to
> > avoid an unnecessary dependency on librt.so).
> > 
> > Fixes: 93c9edbef51b ("tests/xenstore: Rework Makefile")
> > Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Juergen Gross <jgross@suse.com>

Thanks.

While I peronally think this is not warranted, the extra Makefile
clutter is a very minor wart and I definitely don't want to stand in
the way of people dealing with things that are a problem for them.

So:

Acked-by: Ian Jackson <iwj@xenproject.org>

and queued.  I will push it today.

Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:40:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:40:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177572.323085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpll-00039F-Hx; Thu, 02 Sep 2021 16:40:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177572.323085; Thu, 02 Sep 2021 16:40:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLpll-000398-Dt; Thu, 02 Sep 2021 16:40:53 +0000
Received: by outflank-mailman (input) for mailman id 177572;
 Thu, 02 Sep 2021 16:40:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLplk-000392-M4
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:40:52 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLplk-0000fj-LF
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:40:52 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLplk-00019p-KT
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:40:52 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLplf-0003Tl-VM; Thu, 02 Sep 2021 17:40:47 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=GYDi4db046fx5JmmXElXDDtxBBBEsA4Th0cEYvzm6zg=; b=2LvnozFAb3yNbpSa4d5bY76QSt
	aKwgZG8n3+bo1Pb0Pusn2VsSSFiPRSTHekGmznA6JKDKyd9z9KlRCFdmoAZh6yc+YghF9/0kc61tc
	uF+RjQOXzouELYRA9waRq5MPmpqFVYl3tnyU8W7y6jh38UREyGtev43sF0Steb7HyYpw=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24880.65167.715236.551394@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 17:40:47 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools: ROUNDUP() related adjustments
In-Reply-To: <c1b59a02-cce2-b18d-6c2a-4e7057b0a0ed@suse.com>
References: <c5d84f5a-0350-f197-b98a-07512ac0dd91@suse.com>
	<c1b59a02-cce2-b18d-6c2a-4e7057b0a0ed@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH] tools: ROUNDUP() related adjustments"):
> On 10.08.21 12:03, Jan Beulich wrote:
> > For one xc_private.h needlessly repeats xen-tools/libs.h's definition.
> > 
> > And then there are two suspicious uses (resulting from the inconsistency
> > with the respective 2nd parameter of DIV_ROUNDUP()): While the one in
> > tools/console/daemon/io.c - as per the code comment - intentionally uses
> > 8 as the second argument (meaning to align to a multiple of 256), the
> > one in alloc_magic_pages_hvm() pretty certainly does not: There the goal
> > is to align to a uint64_t boundary, for the following module struct to
> > end up aligned.

This is really quite unpleasnt, to my mind.  ROUNDUP taking a bit
length is bad enough, but the magic knowledge about alignment is
really poor too.  It may not be right on all future architectures,
although I think your changae is correct on all the ones we support
(or which people are working on).

So IOW I think your change is correct and warranted, but I really
dislike the code here.

Therefore:

Acked-by: Ian Jackson <iwj@xenproject.org>

> > Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Juergen Gross <jgross@suse.com>

Thanks for the review!

Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 16:58:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 16:58:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177579.323095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLq2J-0005CL-Vx; Thu, 02 Sep 2021 16:57:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177579.323095; Thu, 02 Sep 2021 16:57:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLq2J-0005CE-T3; Thu, 02 Sep 2021 16:57:59 +0000
Received: by outflank-mailman (input) for mailman id 177579;
 Thu, 02 Sep 2021 16:57:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq2I-0005C6-Me
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:57:58 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq2I-00015Y-Ku
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:57:58 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq2I-00076K-Jq
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 16:57:58 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLq27-0003Vy-UT; Thu, 02 Sep 2021 17:57:47 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=VQ/pDRqduhH2wpfMLbgaHgM/4ZpTVM/+CEGro1GEMXk=; b=IB5fOPYC0qJdZkqXx8UfY1j0yU
	aXGHWedIpnm+dgMlrCkTqpOa1R4Wwi1mg0l1ZsB4HNXKdie85Zommd03YJgtweWsONI+XQiucADZI
	ZCK+KiF9lwpCFCNYee6QGBtJpBZREeY3/8AjuqAxQRZBQxQfggK4sLIfLojIZRQr0054=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24881.651.565806.53730@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 17:57:47 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 03/13] libxenguest: deal with log-dirty op stats
 overflow
In-Reply-To: <6d8b3104-f7e6-494c-30e1-1c06419fbfb0@suse.com>
References: <0bebfe8c-6897-dc8b-7fe0-9127d4996eb8@suse.com>
	<668ef720-389f-4cf1-608e-64aca4f7c73d@suse.com>
	<f69e5a2b-cd8c-3534-ab7b-427fb1fc1cdc@suse.com>
	<f0e9c8ad-dfa2-77ac-b3db-02cba59e843c@suse.com>
	<347edc1f-a104-1bc3-0cb2-2cfffcb6214a@suse.com>
	<a9fbd7c1-1675-50ae-6106-8a8625643930@suse.com>
	<6d8b3104-f7e6-494c-30e1-1c06419fbfb0@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [PATCH v2 03/13] libxenguest: deal with log-dirty op stats overflow"):
> And - just to make it very explicit - even if I went this route to
> address a controversial point, I'd still like to understand the
> reason for the original objection - if only for my own education.

I agree with this position.

I have reread the discussions with this patch title in the Subject
line.  I am still unclear on the precise nature of Andrew's objection.

All I could find was this from Andrew:

  >      ctx->save.stats = (struct precopy_stats){
  > -        .dirty_count = ctx->save.p2m_size,
  > +        .dirty_count = -1,

  This is an external interface, and I'm not sure it will tolerate finding
  more than p2m_size allegedly dirty.

and then the later comment by Andrew:

  I've already explained why [various things]

Unfortunately that message did not contain any links to these previous
explanations.  Andrew, where were they ?  If we are not to go round
in circles, we need to write these things down and then refer to them
when relevant.

We had a conversation on irc today.  I found today's IRC conversation
rather unsatisfactory.  I gleaned from Andrew's comments that this he
was saying this callback is used by something in Citrix's product to
do with nVidia migration.  Obviously we do not wish to break a
downstream.  But this interface is badly documented and clear
explanations are lacking.

While I recognise Andrew's expertise in this area, I think that as
tools maintainer I need to go with the information I have.
The overflow issue is real and Jan has proposed a fix.

Andrew: do you have time to engage with Jan here on these API
questions ?  If not, does anyone else have enough familiarity with
Citrix's use of this API to engage in a proper discussion about how
the overflow problem can be addressed ?

If effort is not available for this, that is of course fine.  But as
an Open Source project we need to be able to move forwsrd with the
available input and contributions.  So if no-one is able to explain
how this rather poor API is to be used, and help develop a solution to
the overflow problem, I should probably ack Jan's patch here.

Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 17:04:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 17:04:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177585.323106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLq8o-0006ob-MV; Thu, 02 Sep 2021 17:04:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177585.323106; Thu, 02 Sep 2021 17:04:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLq8o-0006oU-Ji; Thu, 02 Sep 2021 17:04:42 +0000
Received: by outflank-mailman (input) for mailman id 177585;
 Thu, 02 Sep 2021 17:04:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq8m-0006oO-Du
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:04:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq8m-0001Du-Bo
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:04:40 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLq8m-0007tR-An
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:04:40 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLq8h-0003Xd-Jy; Thu, 02 Sep 2021 18:04:35 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=EwrkcWG+KtHbspcp5DkvRxCnd1eWWHv+1cIErtfTVjg=; b=bDDDwbvXfOyL/MbBT9A/njs4zb
	em1e437Z8ewV8Lv6fGTbNJIZomyxOWXWYlm//NqfiS7MRZdRoeta/VN1GRMWGndWGkp+Ejq5SVvtT
	h5YmeOcJJ0DTtUGyR2/0eARjqYU+oV8vVR/yj7yjFffgKAyXKJUB0ro9395wQ1BLQ758=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24881.1059.344754.415908@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 18:04:35 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Anthony Perard <anthony.perard@citrix.com>,
    Marek Marczykowski <marmarek@invisiblethingslab.com>
Subject: Re: [PATCH v2 02/13] libxc: split xc_logdirty_control() from
 xc_shadow_control() [and 1 more messages]
In-Reply-To: <7284e7d0-99cc-e35d-2d44-46e7a3132498@suse.com>,
	<cb3b7a87-d480-d431-58a9-b47dab20d2ca@suse.com>
References: <0bebfe8c-6897-dc8b-7fe0-9127d4996eb8@suse.com>
	<4b68233b-259c-3308-3292-82e991e1e108@suse.com>
	<7284e7d0-99cc-e35d-2d44-46e7a3132498@suse.com>
	<eba44638-699e-36e4-afd5-5b78270baafa@suse.com>
	<cb3b7a87-d480-d431-58a9-b47dab20d2ca@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v2 02/13] libxc: split xc_logdirty_control() from xc_shadow_control()"):
> On 05.07.21 17:12, Jan Beulich wrote:
> >>> +long long xc_logdirty_control(xc_interface *xch,
> >>> +                              uint32_t domid,
> >>> +                              unsigned int sop,
> >>> +                              xc_hypercall_buffer_t *dirty_bitmap,
> >>> +                              unsigned long pages,
> >>> +                              unsigned int mode,
> >>> +                              xc_shadow_op_stats_t *stats)
> >>> +{
> >>> +    int rc;
> >>> +    struct xen_domctl domctl = {
> >>> +        .cmd         = XEN_DOMCTL_shadow_op,
> >>> +        .domain      = domid,
> >>> +        .u.shadow_op = {
> >>> +            .op    = sop,
> >>
> >> And same here the other way round: sop should really only be one of
> >> XEN_DOMCTL_SHADOW_OP_CLEAN or XEN_DOMCTL_SHADOW_OP_PEEK.
> >>
> >> With that fixed you can add my:
> >>
> >> Reviewed-by: Juergen Gross <jgross@suse.com>
> > 
> > Thanks, but I won't take this just yet, awaiting your (and maybe
> > others') view(s) on my reply above.
> 
> I'm not feeling really strong in this regard. Either way is fine for
> me.

OK.  My understanding then is that you are happy with Jan's patch and
he can add the Reviewed-by even though he's not checking the ops.
Is that right ?

If so, then Jan, please add my
Acked-by: Ian Jackson <iwj@xenproject.org>
along with Juergen's review.

If I have misunderstood please explain :-).

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 17:11:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 17:11:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177593.323124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLqFf-0008PQ-HE; Thu, 02 Sep 2021 17:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177593.323124; Thu, 02 Sep 2021 17:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLqFf-0008PJ-E7; Thu, 02 Sep 2021 17:11:47 +0000
Received: by outflank-mailman (input) for mailman id 177593;
 Thu, 02 Sep 2021 17:11:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqFe-0008PD-SV
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:11:46 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqFe-0001Mu-Rg
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:11:46 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqFe-0000MA-Qq
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:11:46 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLqFT-0003Zb-3Q; Thu, 02 Sep 2021 18:11:35 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=GeXJrQDQtclWxuJyi9dryyESGh+JES5AlryXjtCzhwo=; b=fVot05xTOOmNdteLjWY4uKKlm0
	8/Z6R/YFBUpdHdO18Cqd0KgA3lT12aevbOHftQsqJnw1PumWNigvx5ArAOzInL2u7UjFKNnZWG5Jf
	uAXlw004OWjPvZ4I1hLLjPwu6+d6bLLVKhI9QLfz4yPYVXTfmz5droqlPBcoGzPponUY=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24881.1478.795535.377133@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 18:11:34 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    Roger Pau =?iso-8859-1?Q?Monn=E9?=  <roger.pau@citrix.com>,
    Juergen Gross <jgross@suse.com>,
    George Dunlap <george.dunlap@citrix.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 03/12] libxenguest: short-circuit "all-dirty" handling
In-Reply-To: <f85ee373-b497-549b-242f-0dd9eda1b4cd@suse.com>
References: <912fa390-f9e9-198a-9aee-39fdb9a28fcc@suse.com>
	<55875a26-7f1d-a6d9-9384-b03b3b2cb86d@suse.com>
	<60be051f-7751-f15d-ae4d-2c7e9af82693@citrix.com>
	<f85ee373-b497-549b-242f-0dd9eda1b4cd@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [PATCH 03/12] libxenguest: short-circuit "all-dirty" handling"):
> On 25.06.2021 19:02, Andrew Cooper wrote:
> > Single all-dirty runs are a debugging technique only. All production
> > cases are live, and you don't want to fail midway through because a
> > late, large, memory allocation failed.
> 
> I'm afraid I don't understand: I don't move _when_ the allocation
> occurs; I only suppress the allocation (altogether) when the allocated
> memory remains unused.

Thanks for this reply, Jan.  I hope it is satisfactory to Andrew; if
not, I hope Andrew will be able to explain.

I am going to give this a provisional

Acked-by: Ian Jackson <iwj@xenproject.org>

I may of course withdraw that ack if it is explained to me that this
is patch is wrong.

> >> +        : (void *)-1L;
> > 
> > This is a pointer loaded with a timebomb, which doesn't trigger NULL
> > pointer checks, and for which {set,clear}_bit(dirty_bitmap, large_pfn)
> > won't fault and will instead corrupt random areas of the address space.
> 
> Yeah, this isn't very nice, and gets done away with again in a later
> patch. I'd prefer to keep it like it is (assuming the later change
> will also go in), but if really deemed necessary I can move that code
> re-arrangement here, such that the use of (void *)-1L wouldn't be
> necessary anymore. (You may have noticed that all I did this for is
> to be able to pass the !dirty_bitmap later in the function, and that
> I deliberately only update the local variable, not the hbuf, making
> pretty certain that this pointer isn't going to be de-referenced.)

I agree that this -1L is very unpleasant.

I'm not going to say that you need to restructure your series, but
please make sure you don't commit this patch without the fix.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 17:18:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 17:18:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177599.323135 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLqLb-0000fi-6b; Thu, 02 Sep 2021 17:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177599.323135; Thu, 02 Sep 2021 17:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLqLb-0000fb-3Q; Thu, 02 Sep 2021 17:17:55 +0000
Received: by outflank-mailman (input) for mailman id 177599;
 Thu, 02 Sep 2021 17:17:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqLa-0000fV-8G
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:17:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqLa-0001T6-6O
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:17:54 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mLqLa-0000rf-5N
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 17:17:54 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mLqLP-0003au-9j; Thu, 02 Sep 2021 18:17:43 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=umiEI7GI1ws1A9D4DUyhTQcphKTnzeVma7/jDhfL17Y=; b=u62kLIS/FgD59gVZ9daqK4fLPc
	XSgGpBGAF3SlzgA8WBEwl9yndpiFnBtpfw4nwNDnq8uPvlFWppzPYSeKoMqRHGUj7D8yRcn+j4oO2
	3aHlx4hFMP0TigC2PKqcQSb5JcQ1HdooYxMUpEocyn/+yZTDsdDzHPMwUvPnyACrFcYo=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24881.1846.830472.226170@mariner.uk.xensource.com>
Date: Thu, 2 Sep 2021 18:17:42 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    Roger Pau =?iso-8859-1?Q?Monn=E9?=  <roger.pau@citrix.com>,
    Juergen Gross <jgross@suse.com>,
    George Dunlap <george.dunlap@citrix.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] libxenguest: avoid allocating unused deferred-pages
 bitmap
In-Reply-To: <3aea7472-6c1d-f786-db5f-ead60eb03240@suse.com>
References: <912fa390-f9e9-198a-9aee-39fdb9a28fcc@suse.com>
	<61ff4f26-a9cc-d123-98a0-be6c23f21e9b@suse.com>
	<44825600-c27b-34ac-01b2-1ffb5e0bf0be@citrix.com>
	<3aea7472-6c1d-f786-db5f-ead60eb03240@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [PATCH 04/12] libxenguest: avoid allocating unused deferred-pages bitmap"):
> [stuff]

I have read this conversation several times and it is not clear to me
whether Andrew was saying Jan's patch is bad, or the existing code is
bad.

I'm hesitant to give an ack for an optimisation without understanding
what the implications might be.  Andrew, can you explain what will go
wrong if we take Jan's patch ?

(I amk not really familiar with this area of the code.  If necessary I
could go and read it to form my own opinion.)

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 18:15:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 18:15:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177605.323146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLrFa-0007Q9-Gj; Thu, 02 Sep 2021 18:15:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177605.323146; Thu, 02 Sep 2021 18:15:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLrFa-0007Q2-Df; Thu, 02 Sep 2021 18:15:46 +0000
Received: by outflank-mailman (input) for mailman id 177605;
 Thu, 02 Sep 2021 18:15:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrFZ-0007Ps-DJ; Thu, 02 Sep 2021 18:15:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrFZ-0002TH-5T; Thu, 02 Sep 2021 18:15:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrFY-00051A-SY; Thu, 02 Sep 2021 18:15:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrFY-0002Nm-S2; Thu, 02 Sep 2021 18:15:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SDIcP1OjsiM+oeKet/4l7rZjv861FdN57ePGZ2vAEp8=; b=j/Jg29RwqUnoXvECYRE3vXsLRW
	LwkPrPdy2LtNsrqecOAibPJQe2sASD4PpgwlPy0nzCvCK4gT/8MF9wE7uabNPJ2I9kHSwzi9y6D62
	dLsgkogoAFadgzIYEJe0iuH8uoJel2bCuABLIkTudIfpVnKP8qph0syDehNCs5V63IVg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164757-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164757: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=079b1252e9de384385c9da910262312ec2e574c8
X-Osstest-Versions-That:
    qemuu=4c41a1c595e1ce3fe29f3b7bb22ff7402be9c77d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 18:15:44 +0000

flight 164757 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164757/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164725
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164725
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164725
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164725
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164725
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164725
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164725
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                079b1252e9de384385c9da910262312ec2e574c8
baseline version:
 qemuu                4c41a1c595e1ce3fe29f3b7bb22ff7402be9c77d

Last test of basis   164725  2021-09-01 21:39:10 Z    0 days
Testing same since   164757  2021-09-02 08:06:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alistair Francis <alistair.francis@wdc.com>
  David Gibson <david@gibson.dropbear.id.au>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Shuuichirou Ishii <ishii.shuuichir@fujitsu.com>
  Yanan Wang <wangyanan55@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   4c41a1c595..079b1252e9  079b1252e9de384385c9da910262312ec2e574c8 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 18:39:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 18:39:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177617.323166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLrcp-0001li-SV; Thu, 02 Sep 2021 18:39:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177617.323166; Thu, 02 Sep 2021 18:39:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLrcp-0001lb-NZ; Thu, 02 Sep 2021 18:39:47 +0000
Received: by outflank-mailman (input) for mailman id 177617;
 Thu, 02 Sep 2021 18:39:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrco-0001lR-L9; Thu, 02 Sep 2021 18:39:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrco-0002pm-DO; Thu, 02 Sep 2021 18:39:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrco-0005jl-5H; Thu, 02 Sep 2021 18:39:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLrco-0001Q1-4o; Thu, 02 Sep 2021 18:39:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3oSTTXi2qZQMzCQ8lYQzWt46vZWAp8aVUy5kVV8WMks=; b=VtG0aiReSvAxsi3IH8+Nzu6saw
	IGZKuo+AI4KOKTudu8eT0mKd4q0eviEM84VmspwHKTgq2hkhEj1TmhgfUJe6gT99zwl7S1vZEKbek
	xEzDfRxaNAg8mV/9Iz7euV5D431ix8VDQBEavoNWbN1sFYTRyTfeC5SLyvvEj0nmbvhc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164764-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164764: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 18:39:46 +0000

flight 164764 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164764/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   13 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 19:55:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 19:55:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177626.323180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLsnT-000235-EP; Thu, 02 Sep 2021 19:54:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177626.323180; Thu, 02 Sep 2021 19:54:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLsnT-00022y-BG; Thu, 02 Sep 2021 19:54:51 +0000
Received: by outflank-mailman (input) for mailman id 177626;
 Thu, 02 Sep 2021 19:54:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLsnR-00022o-Uy; Thu, 02 Sep 2021 19:54:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLsnR-00041y-HV; Thu, 02 Sep 2021 19:54:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLsnR-0008V9-96; Thu, 02 Sep 2021 19:54:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLsnR-0005pk-8d; Thu, 02 Sep 2021 19:54:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gE81+iMR+tkr7Ly7/RydKpk8KRxexAegu4xRQUWSBFg=; b=CHDqHaDZf34kuNqA6WQJA1XGq3
	NPxPem6gLGKh2dD71T+dxDTY4H77EH8vP4M6HagzvipAuRIF7INj0uf3DyYcCWH33Lrb3G/KxFpEP
	QATxcfBbFzt5DsnnS32l54ntQS1R/WolYeg7Aup77S14UEB2u8qyr5c65zAeab9GSaQc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164773-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164773: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 19:54:49 +0000

flight 164773 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164773/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   14 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 20:55:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 20:55:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177639.323199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtkD-0000WU-C7; Thu, 02 Sep 2021 20:55:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177639.323199; Thu, 02 Sep 2021 20:55:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtkD-0000WN-9G; Thu, 02 Sep 2021 20:55:33 +0000
Received: by outflank-mailman (input) for mailman id 177639;
 Thu, 02 Sep 2021 20:55:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CQpA=NY=kernel.org=pr-tracker-bot@srs-us1.protection.inumbo.net>)
 id 1mLtkB-0000WH-Kt
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 20:55:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7e829113-e956-4e45-a777-b310a79e131e;
 Thu, 02 Sep 2021 20:55:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPS id EE4D661054;
 Thu,  2 Sep 2021 20:55:29 +0000 (UTC)
Received: from pdx-korg-docbuild-2.ci.codeaurora.org (localhost.localdomain
 [127.0.0.1])
 by pdx-korg-docbuild-2.ci.codeaurora.org (Postfix) with ESMTP id E838260A0C;
 Thu,  2 Sep 2021 20:55:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e829113-e956-4e45-a777-b310a79e131e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630616130;
	bh=FL6WX8B1Oxj9domd5qdIRAc3dG4IkXovTWt6/QvWHg4=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=EeI2FcfmIISJKB3YsGYEiGzci0TVG6yeokjQ6hY2PlF7ONujfcPBgPd4rxc2Xnx3v
	 SAzZI+mOgkvzHltRagDkX2J3ESex3rI99S600hqBlOYwxNvvag1AvoHa9rOuJu/pmo
	 zlYT7dfJi6l6kGaIddacWaLS3u1yRjQAyc+ZPmf1DfE9GVnriV3vPnU54Yq/y91M98
	 6qx/SQRuGg39uC5hGvSAgNzguYBbKTRhcOhsMoMcVQKHKwt01yK3Tfl5002W2Hm3Jn
	 gyy0+eq6L0w0GTPWBWEBjeXPVMJNURAiGORdCsY0WsXmVwwa1mFcAELOJWqS1ozADB
	 87Y/D0m2pSgDQ==
Subject: Re: [GIT PULL] xen: branch for v5.15-rc1
From: pr-tracker-bot@kernel.org
In-Reply-To: <20210902072921.27763-1-jgross@suse.com>
References: <20210902072921.27763-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20210902072921.27763-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15-rc1-tag
X-PR-Tracked-Commit-Id: 58e636039b512697554b579c2bb23774061877f5
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: 9ae5fceb9a20154d74586fe17d1096b981b23e34
Message-Id: <163061612994.4397.10660862212967182115.pr-tracker-bot@kernel.org>
Date: Thu, 02 Sep 2021 20:55:29 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, boris.ostrovsky@oracle.com

The pull request you sent on Thu,  2 Sep 2021 09:29:21 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15-rc1-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/9ae5fceb9a20154d74586fe17d1096b981b23e34

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 20:57:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 20:57:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177645.323211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtlu-00018L-Ob; Thu, 02 Sep 2021 20:57:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177645.323211; Thu, 02 Sep 2021 20:57:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtlu-00018E-LG; Thu, 02 Sep 2021 20:57:18 +0000
Received: by outflank-mailman (input) for mailman id 177645;
 Thu, 02 Sep 2021 20:57:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtlt-00017w-Tl; Thu, 02 Sep 2021 20:57:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtlt-0005CH-Kc; Thu, 02 Sep 2021 20:57:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtlt-00028W-8j; Thu, 02 Sep 2021 20:57:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtlt-0000Qd-8F; Thu, 02 Sep 2021 20:57:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Ozfvfgkec4E6NXTtSL2ngHRd6lu22+HqazEtSwgbGso=; b=hoW80vBZQX3yMRJ+29BaAVTEvo
	mL4489+GFh1HmlKBFMwrLTN18btsA54Vig4RZZKERiitxHlShhevjS3f9fOOTOn3T82Tg9jCxQ80F
	DIUlu+qUFaWoxNPtsVcZXwfjM+nDWFUYnDyfvHp+p/OK1cIOfsp4UChc+wdF5g+HXmsg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164760-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164760: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4ac6d90867a4de2e12117e755dbd76e08d88697f
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 20:57:17 +0000

flight 164760 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164760/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-armhf-armhf-xl-arndale   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4ac6d90867a4de2e12117e755dbd76e08d88697f
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  398 days
Failing since        152366  2020-08-01 20:49:34 Z  397 days  687 attempts
Testing same since   164760  2021-09-02 09:10:05 Z    0 days    1 attempts

------------------------------------------------------------
7137 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2177664 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 20:57:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 20:57:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177653.323231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtmX-0001jy-5R; Thu, 02 Sep 2021 20:57:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177653.323231; Thu, 02 Sep 2021 20:57:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLtmX-0001jr-1U; Thu, 02 Sep 2021 20:57:57 +0000
Received: by outflank-mailman (input) for mailman id 177653;
 Thu, 02 Sep 2021 20:57:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtmV-0001jY-QG; Thu, 02 Sep 2021 20:57:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtmV-0005Cg-MG; Thu, 02 Sep 2021 20:57:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtmV-0002AX-Ei; Thu, 02 Sep 2021 20:57:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLtmV-0000si-EC; Thu, 02 Sep 2021 20:57:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3Rbgeq0QRqigDgi3xVkRQ6D9OVmfPgX1xpin9VNOD3U=; b=6TRtFH4dFLJ6nQEBDu63koHRY+
	d4Msew1kiXTmU2Fgn9ntQX4tLTvWNzgm+5tGGpi7xuRc7Md5zVvYkHBVnTMrldL0hdWxOk6L8L5um
	lxXCzkyxULjnvWvnTw6fRilqZKt80d2nkAPQE5x+R32OQRujq/q6eorCreHIX/Dm0LWk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164775-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164775: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 20:57:55 +0000

flight 164775 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164775/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   15 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 21:23:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 21:23:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177664.323245 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuBZ-0005X6-Dy; Thu, 02 Sep 2021 21:23:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177664.323245; Thu, 02 Sep 2021 21:23:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuBZ-0005Wz-AB; Thu, 02 Sep 2021 21:23:49 +0000
Received: by outflank-mailman (input) for mailman id 177664;
 Thu, 02 Sep 2021 21:23:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLuBY-0005Wt-95
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 21:23:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0ea52c56-0c34-11ec-aecd-12813bfff9fa;
 Thu, 02 Sep 2021 21:23:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F27E061074;
 Thu,  2 Sep 2021 21:23:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ea52c56-0c34-11ec-aecd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630617826;
	bh=R33NS/1KqtwH0usN9pB4gLg6GNUdRpCt3mNX/jS3mZM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MEZ7nPh1YPE7bCire3PR2zog3wSb2lf6ddATIHdxyB2iVOvpOx44Z9ZFp/7EroCgC
	 9ZB/0fnsZPWOa5ZKLNElEoix/w5sJJ/zi59pArg6PtjVZQA1gGzV3d9dOjuvYs5gwn
	 hbtnggg5kB1g4WAneYP1eHdKA0YVtOw1seGp7pSE0xVYN9GSjltypXLq2vPd9IgCaw
	 Yuxii+rZkCFfyVh87Uj0dr429n2YdlVPp5+dZrr5/yHDyurFz3ie3DoP0hxN0zQpdD
	 cG87q2eadWrsYx48OaJZUje5aLmLMPX2f+rxAn4XBrtMHsolRTiHQXpOaMg+WCeE09
	 LUoGVzi9Z/ibw==
Date: Thu, 2 Sep 2021 14:23:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 4/7] xen/arm: static memory initialization
In-Reply-To: <20210824095045.2281500-5-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109021324590.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com> <20210824095045.2281500-5-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 24 Aug 2021, Penny Zheng wrote:
> This patch introduces static memory initialization, during system boot up.
> 
> The new function init_staticmem_pages is responsible for static memory
> initialization.
> 
> Helper free_staticmem_pages is the equivalent of free_heap_pages, to free
> nr_mfns pages of static memory.
> 
> This commit also introduces new CONFIG_STATIC_MEMORY to wrap all
                              ^ a                     ^ option

> static-allocation-related codes.
                            ^ code

> Put asynchronous scrubbing for pages of static memory in TODO list.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
> v5 change:
> - make CONFIG_STATIC_MEMORY user selectable and gated by UNSUPPORTED.
> - wrap all static-allocation-related codes with CONFIG_STATIC_MEMORY
> even in arm-specific file.
> - make bank_start/bank_end type of mfn_t, and rename bank_size to
> bank_pages.
> ---
>  xen/arch/arm/setup.c    | 31 +++++++++++++++++++++++++++++++
>  xen/common/Kconfig      | 17 +++++++++++++++++
>  xen/common/page_alloc.c | 22 +++++++++++++++++++++-
>  xen/include/xen/mm.h    |  6 ++++++
>  4 files changed, 75 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 63a908e325..44aca9f1b9 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -609,6 +609,29 @@ static void __init init_pdx(void)
>      }
>  }
>  
> +#ifdef CONFIG_STATIC_MEMORY
> +/* Static memory initialization */
> +static void __init init_staticmem_pages(void)
> +{
> +    unsigned int bank;
> +
> +    for ( bank = 0 ; bank < bootinfo.reserved_mem.nr_banks; bank++ )
> +    {
> +        if ( bootinfo.reserved_mem.bank[bank].xen_domain )
> +        {
> +            mfn_t bank_start = _mfn(PFN_UP(bootinfo.reserved_mem.bank[bank].start));
> +            unsigned long bank_pages = PFN_DOWN(bootinfo.reserved_mem.bank[bank].size);
> +            mfn_t bank_end = mfn_add(bank_start, bank_pages);
> +
> +            if ( mfn_x(bank_end) <= mfn_x(bank_start) )
> +                return;
> +
> +            free_staticmem_pages(mfn_to_page(bank_start), bank_pages, false);
> +        }
> +    }
> +}
> +#endif
> +
>  #ifdef CONFIG_ARM_32
>  static void __init setup_mm(void)
>  {
> @@ -736,6 +759,10 @@ static void __init setup_mm(void)
>      /* Add xenheap memory that was not already added to the boot allocator. */
>      init_xenheap_pages(mfn_to_maddr(xenheap_mfn_start),
>                         mfn_to_maddr(xenheap_mfn_end));
> +
> +#ifdef CONFIG_STATIC_MEMORY
> +    init_staticmem_pages();
> +#endif
>  }
>  #else /* CONFIG_ARM_64 */
>  static void __init setup_mm(void)
> @@ -789,6 +816,10 @@ static void __init setup_mm(void)
>  
>      setup_frametable_mappings(ram_start, ram_end);
>      max_page = PFN_DOWN(ram_end);
> +
> +#ifdef CONFIG_STATIC_MEMORY
> +    init_staticmem_pages();
> +#endif
>  }
>  #endif
>  
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 0ddd18e11a..514a2c9022 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -67,6 +67,23 @@ config MEM_ACCESS
>  config NEEDS_LIBELF
>  	bool
>  
> +config STATIC_MEMORY
> +        bool "Static Allocation Support (UNSUPPORTED)" if UNSUPPORTED
> +	depends on ARM
> +	---help---
> +	  Static Allocation refers to system or sub-system(domains) for
> +	  which memory areas are pre-defined by configuration using physical
> +          address ranges.
> +
> +	  Those pre-defined memory, -- Static Memory, as parts of RAM reserved
> +	  during system boot-up, shall never go to heap allocator or boot
> +	  allocator for any use.
> +
> +	  When enabled, memory can be statically allocated to a domain using
> +	  the property "xen,static-mem" defined in the domain configuration.
> +
> +	  If unsure, say Y.

I share Jan's comment about indentation, the kconfig message and the
little spurious change below. Aside from those, this patch looks fine to
me.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 21:30:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 21:30:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177670.323255 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuIP-0006uo-4b; Thu, 02 Sep 2021 21:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177670.323255; Thu, 02 Sep 2021 21:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuIP-0006uh-1M; Thu, 02 Sep 2021 21:30:53 +0000
Received: by outflank-mailman (input) for mailman id 177670;
 Thu, 02 Sep 2021 21:30:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLuIN-0006ub-3k
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 21:30:51 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0aa8547e-0c35-11ec-aece-12813bfff9fa;
 Thu, 02 Sep 2021 21:30:49 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BDEE4610E6;
 Thu,  2 Sep 2021 21:30:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0aa8547e-0c35-11ec-aece-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630618249;
	bh=SLLuobHZM2VYjHSWMTHYRwcUffU8KYJOq5ruqibBR64=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=f8FK93V2OYaaKdFvFnGFaHEAexfiuTJGZY97ZPySQ0CYfgMPs5/7QLlJLNCFtGSqr
	 V6rGbaRAlheJA3Abr6uN2DVei8eP3/Ts4SsIttdw7TfJCh/DgO6VbcCW447AUQbvZM
	 ph2xIHLoyjVrwLqPjl2M6tKtsDk8FG8jNq6N/jqnzeg1Svt+KdDWqtC8/16ttbDp9W
	 +b5XgW9bHlUj/dVJvOYZgUOFJt1dkuDJlS5jA2lhV20CbHBAYYj817eKDl01853ukn
	 w9R35gjESE5s3VPcnD96mB8XGlBeEivZVRtRMW9iAbX4pytLLhfLeS4zhV2ON7dkHN
	 cdJMoQML2ULsg==
Date: Thu, 2 Sep 2021 14:30:48 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 2/7] xen/arm: introduce domain on Static Allocation
In-Reply-To: <20210824095045.2281500-3-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109021312051.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com> <20210824095045.2281500-3-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 24 Aug 2021, Penny Zheng wrote:
> Static Allocation refers to system or sub-system(domains) for which memory
> areas are pre-defined by configuration using physical address ranges.
> Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
> beginning, shall never go to heap allocator or boot allocator for any use.
> 
> Memory can be statically allocated to a domain using the property "xen,static-
> mem" defined in the domain configuration. The number of cells for the address
> and the size must be defined using respectively the properties
> "#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
> 
> This patch introduces this new `xen,static-mem` feature, and also documents
> and parses this new attribute at boot time.
> 
> This patch also introduces a new field "bool xen_domain" in "struct membank"
> to tell the difference of one memory bank is reserved as the whole
> hardware resource, or bind to one specific xen domain node, through
> "xen,static-mem"
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
> v5 changes:
> - check the node using the Xen domain binding whether contains the property
> "xen,static-mem", not the property itself
> - add "rc = ..." to get the error propagated
> - introduce new field "bool xen_domain", then static memory shall be also stored
> as reserved memory(bootinfo.reserved_mem), but being bind to one
> specific Xen domain node.
> - doc refinement
> ---
>  docs/misc/arm/device-tree/booting.txt | 33 ++++++++++++++++++++++++
>  xen/arch/arm/bootfdt.c                | 36 +++++++++++++++++++++++++--
>  xen/include/asm-arm/setup.h           |  1 +
>  3 files changed, 68 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 5243bc7fd3..95b20ddc3a 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -268,3 +268,36 @@ The DTB fragment is loaded at 0xc000000 in the example above. It should
>  follow the convention explained in docs/misc/arm/passthrough.txt. The
>  DTB fragment will be added to the guest device tree, so that the guest
>  kernel will be able to discover the device.
> +
> +
> +Static Allocation
> +=============
> +
> +Static Allocation refers to system or sub-system(domains) for which memory
> +areas are pre-defined by configuration using physical address ranges.
> +
> +Memory can be statically allocated to a domain using the property "xen,static-
> +mem" defined in the domain configuration. The number of cells for the address
> +and the size must be defined using respectively the properties
> +"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
> +
> +Below is an example on how to specify the static memory region in the
> +device-tree:
> +
> +    / {
> +        chosen {
> +            domU1 {
> +                compatible = "xen,domain";
> +                #address-cells = <0x2>;
> +                #size-cells = <0x2>;
> +                cpus = <2>;
> +                #xen,static-mem-address-cells = <0x1>;
> +                #xen,static-mem-size-cells = <0x1>;
> +                xen,static-mem = <0x30000000 0x20000000>;
> +                ...
> +            };
> +        };
> +    };
> +
> +This will reserve a 512MB region starting at the host physical address
> +0x30000000 to be exclusively used by DomU1

This binding is OK.  We might want to clarify what is the purpose of the
"memory" property when "xen,static-mem" is present. I would suggest to
write that when "xen,static-mem" is present, the "memory" property
becomes optional. Or even better:

"""
When present, the xen,static-mem property supersedes the memory property.
"""


In the future when Xen will support direct mapping, I assume that we'll
also add a direct-map property to enable it.  Something like:

    domU1 {
        compatible = "xen,domain";
        #address-cells = <0x2>;
        #size-cells = <0x2>;
        cpus = <2>;
        #xen,static-mem-address-cells = <0x1>;
        #xen,static-mem-size-cells = <0x1>;
        xen,static-mem = <0x30000000 0x20000000>;
        direct-map;
        ...
    };

Maybe I would add a statement to clarify it that xen,static-mem doesn't
automatically imply direct mapping. Something like:

"""
The static memory will be mapped in the guest at the usual guest memory
addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.
"""

The rest of the patch looks OK. One minor NIT below.



> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 8c81be3379..00f34eec58 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -66,7 +66,7 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
>  static int __init device_tree_get_meminfo(const void *fdt, int node,
>                                            const char *prop_name,
>                                            u32 address_cells, u32 size_cells,
> -                                          void *data)
> +                                          void *data, bool xen_domain)
>  {
>      const struct fdt_property *prop;
>      unsigned int i, banks;
> @@ -90,6 +90,7 @@ static int __init device_tree_get_meminfo(const void *fdt, int node,
>              continue;
>          mem->bank[mem->nr_banks].start = start;
>          mem->bank[mem->nr_banks].size = size;
> +        mem->bank[mem->nr_banks].xen_domain = xen_domain;
>          mem->nr_banks++;
>      }
>  
> @@ -184,7 +185,8 @@ static int __init process_memory_node(const void *fdt, int node,
>          return -EINVAL;
>      }
>  
> -    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
> +    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells,
> +                                   data, false);
>  }
>  
>  static int __init process_reserved_memory_node(const void *fdt, int node,
> @@ -338,6 +340,34 @@ static void __init process_chosen_node(const void *fdt, int node,
>      add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
>  }
>  
> +static int __init process_domain_node(const void *fdt, int node,
> +                                       const char *name,
> +                                       u32 address_cells, u32 size_cells)
> +{
> +    const struct fdt_property *prop;
> +
> +    printk("Checking for \"xen,static-mem\" in domain node\n");
> +
> +    prop = fdt_get_property(fdt, node, "xen,static-mem", NULL);
> +    if ( !prop )
> +        /* No "xen,static-mem" present. */
> +        return 0;
> +
> +    address_cells = device_tree_get_u32(fdt, node,
> +                                        "#xen,static-mem-address-cells", 0);
> +    size_cells = device_tree_get_u32(fdt, node,
> +                                     "#xen,static-mem-size-cells", 0);
> +    if ( address_cells < 1 || size_cells < 1 )
> +    {
> +        printk("fdt: node `%s': invalid #xen,static-mem-address-cells or #xen,static-mem-size-cells",
> +               name);
> +        return -EINVAL;
> +    }
> +
> +    return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells,
> +                                   size_cells, &bootinfo.reserved_mem, true);
> +}
> +
>  static int __init early_scan_node(const void *fdt,
>                                    int node, const char *name, int depth,
>                                    u32 address_cells, u32 size_cells,
> @@ -356,6 +386,8 @@ static int __init early_scan_node(const void *fdt,
>          process_multiboot_node(fdt, node, name, address_cells, size_cells);
>      else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") )
>          process_chosen_node(fdt, node, name, address_cells, size_cells);
> +    else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") )
> +        rc = process_domain_node(fdt, node, name, address_cells, size_cells);
>  
>      if ( rc < 0 )
>          printk("fdt: node `%s': parsing failed\n", name);
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index c4b6af6029..6c3c16294b 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -24,6 +24,7 @@ typedef enum {
>  struct membank {
>      paddr_t start;
>      paddr_t size;
> +    bool xen_domain; /* whether memory bank is bind to Xen domain. */
                                  ^ a or the      ^ bound to a 


>  };
>  
>  struct meminfo {
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 21:33:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 21:33:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177676.323267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuKS-0007jO-IT; Thu, 02 Sep 2021 21:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177676.323267; Thu, 02 Sep 2021 21:33:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuKS-0007jH-Ee; Thu, 02 Sep 2021 21:33:00 +0000
Received: by outflank-mailman (input) for mailman id 177676;
 Thu, 02 Sep 2021 21:32:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLuKR-0007j9-TD
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 21:32:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f6592c34-d9a2-40a2-b6a0-1a0503a3100e;
 Thu, 02 Sep 2021 21:32:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D407F610CF;
 Thu,  2 Sep 2021 21:32:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6592c34-d9a2-40a2-b6a0-1a0503a3100e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630618378;
	bh=JBVvt8Y6dkmWs3GjjhGwiKm8mGMVJul8P/KSTHyo6b8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MtQ63zamEkOawpGx/Bo84fL7I2zWCVpgiF+veZhOMBxGxwGwqIOpOkMFQmZ6TzQcw
	 7/Wp/PTF6HWYVm8ViHtvpb2c4iqFTrgqM7WtcDgjj6xwMRjxCJr8bbHzFv7Wb682ve
	 0EvdiZHH3pKRw3AHPQAxcaDXu0JOygek2A4pS5dhjYoCtA/SJQko+oAPNS2GTMSAkm
	 Kj7jQUKfCulYbp3mGmMfnOSGLPvKcJqGIrnz/BPutTKfnmvowWrNtNUKpj6shIvEw0
	 sQf9UMq7DSMyUHtXa3Yl6EKnhhEz+lVuslC4eyt+KKwehNTlbo9CeFgN4DuKhy1BtT
	 J94VukGA1+Zew==
Date: Thu, 2 Sep 2021 14:32:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
In-Reply-To: <20210824095045.2281500-8-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com> <20210824095045.2281500-8-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 24 Aug 2021, Penny Zheng wrote:
> This commit introduces allocate_static_memory to allocate static memory as
> guest RAM for Domain on Static Allocation.
> 
> It uses acquire_domstatic_pages to acquire pre-configured static memory
> for this domain, and uses guest_physmap_add_pages to set up P2M table.
> These pre-defined static memory banks shall be mapped to the fixed guest RAM
> banks. And only when they exhausts the current guest RAM bank, it will seek
> to the next one.
> 
> In order to deal with the trouble of count-to-order conversion when page number
> is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
> a new function guest_physmap_add_pages to cope with adding guest RAM p2m
> mapping with nr_pages.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
> v5 changes:
> - don't split comment over multi-line (even they are more than 80 characters)
> - simply use dt_find_property(node, "xen,static-mem", NULL) to tell
> whether using allocate_static_memory, and add error comment when
> "xen,static-mem" is used but CONFIG_STATIC_MEMORY is not enabled.
> - exporting p2m_insert_mapping() and introduce guest_physmap_add_pages
> to cope with adding guest RAM p2m mapping with nr_pages.
> - check both pbase and psize are page aligned
> - simplify the code in the loops by moving append_static_memory_to_bank()
> outside of the if/else.
> ---
>  xen/arch/arm/domain_build.c | 156 +++++++++++++++++++++++++++++++++++-
>  xen/arch/arm/p2m.c          |   7 +-
>  xen/include/asm-arm/p2m.h   |  11 +++
>  3 files changed, 168 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 6c86d52781..843b8514c7 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -480,6 +480,148 @@ fail:
>            (unsigned long)kinfo->unassigned_mem >> 10);
>  }
>  
> +#ifdef CONFIG_STATIC_MEMORY
> +static bool __init append_static_memory_to_bank(struct domain *d,
> +                                                struct membank *bank,
> +                                                mfn_t smfn,
> +                                                paddr_t size)
> +{
> +    int res;
> +    unsigned int nr_pages = size >> PAGE_SHIFT;
                               ^ PFN_DOWN


> +    /* Infer next GFN. */
> +    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
> +
> +    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
> +    if ( res )
> +    {
> +        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
> +        return false;
> +    }
> +
> +    bank->size = bank->size + size;
> +
> +    return true;
> +}
> +
> +/* Allocate memory from static memory as RAM for one specific domain d. */
> +static void __init allocate_static_memory(struct domain *d,
> +                                          struct kernel_info *kinfo,
> +                                          const struct dt_device_node *node)
> +{
> +    const struct dt_property *prop;
> +    u32 addr_cells, size_cells, reg_cells;
> +    unsigned int nr_banks, gbank = 0, bank = 0;
> +    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
> +    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
> +    const __be32 *cell;
> +    u64 tot_size = 0;
> +    paddr_t pbase, psize, gsize;
> +    mfn_t smfn;
> +    int res;
> +
> +    prop = dt_find_property(node, "xen,static-mem", NULL);
> +    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
> +                               &addr_cells) )
> +    {
> +        printk(XENLOG_ERR
> +               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
> +        goto fail;
> +    }
> +
> +    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
> +                               &size_cells) )
> +    {
> +        printk(XENLOG_ERR
> +               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
> +        goto fail;
> +    }
> +    reg_cells = addr_cells + size_cells;
> +
> +    /* Start with GUEST_RAM0. */
> +    gsize = ramsize[gbank];
> +    kinfo->mem.bank[gbank].start = rambase[gbank];
> +
> +    cell = (const __be32 *)prop->value;
> +    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
> +
> +    for ( ; bank < nr_banks; bank++ )
> +    {
> +        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
> +        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
> +
> +        smfn = maddr_to_mfn(pbase);
> +        res = acquire_domstatic_pages(d, smfn, psize >> PAGE_SHIFT, 0);
                                                  ^ PFN_DOWN(psize)


> +        if ( res )
> +        {
> +            printk(XENLOG_ERR
> +                   "%pd: failed to acquire static memory: %d.\n", d, res);
> +            goto fail;
> +        }
> +
> +        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
> +               d, bank, pbase, pbase + psize);
> +
> +        /*
> +         * It shall be mapped to the fixed guest RAM banks(GUEST_RAM_BANK_BASES),
> +         * And only when it exhausts the current guest RAM bank, it will seek
> +         * to the next.
> +         */
> +        while ( 1 )
> +        {
> +            /* Map as much as possible the static range to the guest bank */
> +            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
> +                                               min(psize, gsize)) )
> +                goto fail;
> +
> +            /*
> +             * The current physical bank is fully mapped.
> +             * Handle the next physical bank.
> +             */
> +            if ( gsize >= psize )
> +            {
> +                gsize = gsize - psize;
> +                break;
> +            }
> +            /*
> +             * When current guest bank size is not enough to map.
> +             * Before seeking to the next, check if we still have available
> +             * guest bank.
> +             */
> +            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
> +            {
> +                printk(XENLOG_ERR "Exhausted all fixed guest banks.\n");
> +                goto fail;
> +            }
> +            else
> +            {
> +                psize = psize - gsize;
> +                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
                                        ^ PFN_DOWN


> +                /* Update to the next guest bank. */
> +                gbank++;
> +                gsize = ramsize[gbank];
> +                kinfo->mem.bank[gbank].start = rambase[gbank];
> +            }
> +        }
> +
> +        tot_size += psize;
> +    }
> +
> +    kinfo->mem.nr_banks = ++gbank;
> +    kinfo->unassigned_mem -= tot_size;
> +    if ( kinfo->unassigned_mem )
> +    {
> +        printk(XENLOG_ERR
> +               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\".\n");
> +        goto fail;

Do we need to make this a fatal failure?

I am asking because unassigned_mem comes from the "memory" property of
the domain in device tree which is kind of redundant with the
introduction of xen,static-mem. In fact, I think it would be better to
clarify the role of "memory" when "xen,static-mem" is also present.
In that case, we could even make "memory" optional.

In any case, even if we don't make "memory" optional, it might still be
good to turn this error into a warning and ignore the remaining
kinfo->unassigned_mem.


> +    }
> +
> +    return;
> +
> +fail:
> +    panic("Failed to allocate requested static memory for domain %pd.", d);
> +}
> +#endif
> +
>  static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                                     const struct dt_device_node *node)
>  {
> @@ -2452,7 +2594,19 @@ static int __init construct_domU(struct domain *d,
>      /* type must be set before allocate memory */
>      d->arch.type = kinfo.type;
>  #endif
> -    allocate_memory(d, &kinfo);
> +    if ( !dt_find_property(node, "xen,static-mem", NULL) )
> +        allocate_memory(d, &kinfo);
> +#ifdef CONFIG_STATIC_MEMORY
> +    else
> +        allocate_static_memory(d, &kinfo, node);
> +#else
> +    else
> +    {
> +        printk(XENLOG_ERR
> +               "CONFIG_STATIC_MEMORY must be enabled to use \"xen,static-mem\".\n");
> +        return -EINVAL;
> +    }
> +#endif

To avoid the double else in the code, this part could be written like
this which is a bit simpler I think:

    if ( !dt_find_property(node, "xen,static-mem", NULL) )
        allocate_memory(d, &kinfo);
    else
    {
#ifdef CONFIG_STATIC_MEMORY
        allocate_static_memory(d, &kinfo, node);
#else
        printk(XENLOG_ERR
               "CONFIG_STATIC_MEMORY must be enabled to use \"xen,static-mem\".\n");
        return -EINVAL;
#endif
    }

This is just a suggestion to improve readability, I am also OK with what
you wrote.

(Another alternative would be to provide a stub allocate_static_memory
implementation for !CONFIG_STATIC_MEMORY.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 21:33:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 21:33:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177681.323277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuKz-0008I8-RG; Thu, 02 Sep 2021 21:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177681.323277; Thu, 02 Sep 2021 21:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLuKz-0008I1-ON; Thu, 02 Sep 2021 21:33:33 +0000
Received: by outflank-mailman (input) for mailman id 177681;
 Thu, 02 Sep 2021 21:33:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLuKy-0008Hr-Qq
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 21:33:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d28cab53-46fe-448d-aef2-2bc219758426;
 Thu, 02 Sep 2021 21:33:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 48F6A610CF;
 Thu,  2 Sep 2021 21:33:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d28cab53-46fe-448d-aef2-2bc219758426
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630618411;
	bh=iI+f+PiR3CEK6K0nqtCXsTZumkL7k+UZRF03zicNu5I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jDOKDKMwyschsIyb505laIpMeD4OziXeu/U7R0ZbkabCj7drfHt/n5Vo8Xn1pFJU0
	 zxDu8hFfYGoRm2aCVxMgpMTp3HeXeGhU2CMaw69BsCu4QHm8jBhCJtEc/rpkx5pHQ3
	 u0Wund2RJDa01U1cOfRZxoDKZ5aAOyyUA401+5HnNd+pg6/i1TMUR/BdqYu76d0zGr
	 mlmG7e8nkvrvCuVFtWaVMMJyxDeOSIO8QKxbRjCs9zCFfIm+it9wtxEJFjbvAi6d/u
	 Axs7189/XKBNLDk+F6rPeDZdbUUboDWLH715F78x6ugSLq4nnPAzPuGrH7BSsfuzgB
	 VW38OfvxMIKig==
Date: Thu, 2 Sep 2021 14:33:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 0/7] Domain on Static Allocation
In-Reply-To: <20210824095045.2281500-1-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109021422170.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 24 Aug 2021, Penny Zheng wrote:
> Static Allocation refers to system or sub-system(domains) for which memory
> areas are pre-defined by configuration using physical address ranges.
> Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
> beginning, shall never go to heap allocator or boot allocator for any use.
> 
> Memory can be statically allocated to a domain using the property "xen,static-
> mem" defined in the domain configuration. The number of cells for the address
> and the size must be defined using respectively the properties
> "#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
> 
> This Patch Serie only talks about Domain on Static Allocation.
> 
> Looking into related [design link](
> https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg00882.html)
> for more details.
> 
> The whole design is about Static Allocation and 1:1 direct-map, and this
> Patch Serie only covers parts of it, which are Domain on Static Allocation.
> Other features will be delievered through different patch series.


I reviewed the whole series and I think it is in great shape and really
close to be committed. Probably the series only needs one more update
and I think we should target the 4.16 release.

I haven't replied to all patches because I agree
with Julien and Jan's comments and everything else looked fine.


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 21:52:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 21:52:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177689.323289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLudV-0002dF-LF; Thu, 02 Sep 2021 21:52:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177689.323289; Thu, 02 Sep 2021 21:52:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLudV-0002d8-IG; Thu, 02 Sep 2021 21:52:41 +0000
Received: by outflank-mailman (input) for mailman id 177689;
 Thu, 02 Sep 2021 21:52:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+wyV=NY=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLudT-0002d2-Oy
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 21:52:39 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01f29d1e-e211-4a89-b018-7e9f5ee1dd14;
 Thu, 02 Sep 2021 21:52:38 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A1C56610CF;
 Thu,  2 Sep 2021 21:52:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01f29d1e-e211-4a89-b018-7e9f5ee1dd14
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630619557;
	bh=1Qb985OcOh2tmU7a/TZayFbUIxluyDo779n2s/w0LyQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JU397CrVf7uaDaPJVR/aIYQyzKcG16haazs+tJKg0zlTLnZbzoW4U/a4Wnm5kKWnW
	 r6OKKbLLdlpcFBgkTXWVVjlhjaayWukx4z9Qe+pkjb0uSreJB+SzoHPFndxNdACb0s
	 Ho3gvS19ltu6nzqZQJ4dJL71F2VbWIwgqeuyZZlqxoQvHU3anRr0W+9us6dniwmC/I
	 NleDywM/jBOjJKLeRShFw/vNgUNfKLrC8MG9WBpr1PLkroQFHsN9e9M9SVZeo9WMQ+
	 Ojxnq4FTYZAQ11I71fHPUAdJEaKhfbn40M/oeeR7q9zgvEFQZrGVUYorjkuZHVzfIG
	 hnUFbrwymNnOw==
Date: Thu, 2 Sep 2021 14:52:36 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org, 
    julien@xen.org, Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
In-Reply-To: <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109021451070.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com> <20210824095045.2281500-8-penny.zheng@arm.com> <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 2 Sep 2021, Stefano Stabellini wrote:
> On Tue, 24 Aug 2021, Penny Zheng wrote:
> > This commit introduces allocate_static_memory to allocate static memory as
> > guest RAM for Domain on Static Allocation.
> > 
> > It uses acquire_domstatic_pages to acquire pre-configured static memory
> > for this domain, and uses guest_physmap_add_pages to set up P2M table.
> > These pre-defined static memory banks shall be mapped to the fixed guest RAM
> > banks. And only when they exhausts the current guest RAM bank, it will seek
> > to the next one.
> > 
> > In order to deal with the trouble of count-to-order conversion when page number
> > is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
> > a new function guest_physmap_add_pages to cope with adding guest RAM p2m
> > mapping with nr_pages.
> > 
> > Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> > ---
> > v5 changes:
> > - don't split comment over multi-line (even they are more than 80 characters)
> > - simply use dt_find_property(node, "xen,static-mem", NULL) to tell
> > whether using allocate_static_memory, and add error comment when
> > "xen,static-mem" is used but CONFIG_STATIC_MEMORY is not enabled.
> > - exporting p2m_insert_mapping() and introduce guest_physmap_add_pages
> > to cope with adding guest RAM p2m mapping with nr_pages.
> > - check both pbase and psize are page aligned
> > - simplify the code in the loops by moving append_static_memory_to_bank()
> > outside of the if/else.
> > ---
> >  xen/arch/arm/domain_build.c | 156 +++++++++++++++++++++++++++++++++++-
> >  xen/arch/arm/p2m.c          |   7 +-
> >  xen/include/asm-arm/p2m.h   |  11 +++
> >  3 files changed, 168 insertions(+), 6 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 6c86d52781..843b8514c7 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -480,6 +480,148 @@ fail:
> >            (unsigned long)kinfo->unassigned_mem >> 10);
> >  }
> >  
> > +#ifdef CONFIG_STATIC_MEMORY
> > +static bool __init append_static_memory_to_bank(struct domain *d,
> > +                                                struct membank *bank,
> > +                                                mfn_t smfn,
> > +                                                paddr_t size)
> > +{
> > +    int res;
> > +    unsigned int nr_pages = size >> PAGE_SHIFT;
>                                ^ PFN_DOWN
> 
> 
> > +    /* Infer next GFN. */
> > +    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
> > +
> > +    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
> > +    if ( res )
> > +    {
> > +        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
> > +        return false;
> > +    }
> > +
> > +    bank->size = bank->size + size;
> > +
> > +    return true;
> > +}
> > +
> > +/* Allocate memory from static memory as RAM for one specific domain d. */
> > +static void __init allocate_static_memory(struct domain *d,
> > +                                          struct kernel_info *kinfo,
> > +                                          const struct dt_device_node *node)
> > +{
> > +    const struct dt_property *prop;
> > +    u32 addr_cells, size_cells, reg_cells;
> > +    unsigned int nr_banks, gbank = 0, bank = 0;
> > +    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
> > +    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
> > +    const __be32 *cell;
> > +    u64 tot_size = 0;
> > +    paddr_t pbase, psize, gsize;
> > +    mfn_t smfn;
> > +    int res;
> > +
> > +    prop = dt_find_property(node, "xen,static-mem", NULL);
> > +    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
> > +                               &addr_cells) )
> > +    {
> > +        printk(XENLOG_ERR
> > +               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
> > +        goto fail;
> > +    }
> > +
> > +    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
> > +                               &size_cells) )
> > +    {
> > +        printk(XENLOG_ERR
> > +               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
> > +        goto fail;
> > +    }
> > +    reg_cells = addr_cells + size_cells;
> > +
> > +    /* Start with GUEST_RAM0. */
> > +    gsize = ramsize[gbank];
> > +    kinfo->mem.bank[gbank].start = rambase[gbank];
> > +
> > +    cell = (const __be32 *)prop->value;
> > +    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
> > +
> > +    for ( ; bank < nr_banks; bank++ )
> > +    {
> > +        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
> > +        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
> > +
> > +        smfn = maddr_to_mfn(pbase);
> > +        res = acquire_domstatic_pages(d, smfn, psize >> PAGE_SHIFT, 0);
>                                                   ^ PFN_DOWN(psize)
> 
> 
> > +        if ( res )
> > +        {
> > +            printk(XENLOG_ERR
> > +                   "%pd: failed to acquire static memory: %d.\n", d, res);
> > +            goto fail;
> > +        }
> > +
> > +        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
> > +               d, bank, pbase, pbase + psize);
> > +
> > +        /*
> > +         * It shall be mapped to the fixed guest RAM banks(GUEST_RAM_BANK_BASES),
> > +         * And only when it exhausts the current guest RAM bank, it will seek
> > +         * to the next.
> > +         */
> > +        while ( 1 )
> > +        {
> > +            /* Map as much as possible the static range to the guest bank */
> > +            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
> > +                                               min(psize, gsize)) )
> > +                goto fail;
> > +
> > +            /*
> > +             * The current physical bank is fully mapped.
> > +             * Handle the next physical bank.
> > +             */
> > +            if ( gsize >= psize )
> > +            {
> > +                gsize = gsize - psize;
> > +                break;
> > +            }
> > +            /*
> > +             * When current guest bank size is not enough to map.
> > +             * Before seeking to the next, check if we still have available
> > +             * guest bank.
> > +             */
> > +            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
> > +            {
> > +                printk(XENLOG_ERR "Exhausted all fixed guest banks.\n");
> > +                goto fail;
> > +            }
> > +            else
> > +            {
> > +                psize = psize - gsize;
> > +                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
>                                         ^ PFN_DOWN
> 
> 
> > +                /* Update to the next guest bank. */
> > +                gbank++;
> > +                gsize = ramsize[gbank];
> > +                kinfo->mem.bank[gbank].start = rambase[gbank];
> > +            }
> > +        }
> > +
> > +        tot_size += psize;
> > +    }
> > +
> > +    kinfo->mem.nr_banks = ++gbank;
> > +    kinfo->unassigned_mem -= tot_size;
> > +    if ( kinfo->unassigned_mem )
> > +    {
> > +        printk(XENLOG_ERR
> > +               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\".\n");
> > +        goto fail;
> 
> Do we need to make this a fatal failure?
> 
> I am asking because unassigned_mem comes from the "memory" property of
> the domain in device tree which is kind of redundant with the
> introduction of xen,static-mem. In fact, I think it would be better to
> clarify the role of "memory" when "xen,static-mem" is also present.
> In that case, we could even make "memory" optional.
> 
> In any case, even if we don't make "memory" optional, it might still be
> good to turn this error into a warning and ignore the remaining
> kinfo->unassigned_mem.

One more thing: if we decide to make "memory" optional, we need to avoid
failing if it is not present at the beginning of construct_domU (if
xen,static-mem is present).


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 22:08:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 22:08:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177697.323306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLusH-0004MN-1U; Thu, 02 Sep 2021 22:07:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177697.323306; Thu, 02 Sep 2021 22:07:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLusG-0004MG-UY; Thu, 02 Sep 2021 22:07:56 +0000
Received: by outflank-mailman (input) for mailman id 177697;
 Thu, 02 Sep 2021 22:07:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mLusF-0004MA-QJ
 for xen-devel@lists.xenproject.org; Thu, 02 Sep 2021 22:07:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLusF-0006Sq-Kl; Thu, 02 Sep 2021 22:07:55 +0000
Received: from 54-240-197-224.amazon.com ([54.240.197.224]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mLusF-0001cA-Ek; Thu, 02 Sep 2021 22:07:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=1T4g9ly1aVs9sxccoIF94nJdhxMvrAlrBSd/tuJ7YMc=; b=5Qo9VREyLbEKHxfKO/FkFT8fjG
	f8YnoVNVvw7ynszXmCX5KwKaPPG/28YVXOZJ5itcab0UFXaAco+j5k3OeK5sWWR1cM6n94izo/HJw
	qsVqJfP1+LeuWYjPT+PWYovt/m3Djo1YzMdIA5MDvvGU+sA/gt2imSGIBXQjoDdwoELQ=;
Subject: Re: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
To: Stefano Stabellini <sstabellini@kernel.org>,
 Penny Zheng <penny.zheng@arm.com>
Cc: xen-devel@lists.xenproject.org, Bertrand.Marquis@arm.com,
 Wei.Chen@arm.com, nd@arm.com
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-8-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <ac3a9fbd-7598-e400-6c6c-1214df7acee7@xen.org>
Date: Thu, 2 Sep 2021 23:07:53 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 02/09/2021 22:32, Stefano Stabellini wrote:
> On Tue, 24 Aug 2021, Penny Zheng wrote:
>> This commit introduces allocate_static_memory to allocate static memory as
>> guest RAM for Domain on Static Allocation.
>>
>> It uses acquire_domstatic_pages to acquire pre-configured static memory
>> for this domain, and uses guest_physmap_add_pages to set up P2M table.
>> These pre-defined static memory banks shall be mapped to the fixed guest RAM
>> banks. And only when they exhausts the current guest RAM bank, it will seek
>> to the next one.
>>
>> In order to deal with the trouble of count-to-order conversion when page number
>> is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
>> a new function guest_physmap_add_pages to cope with adding guest RAM p2m
>> mapping with nr_pages.
>>
>> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
>> ---
>> v5 changes:
>> - don't split comment over multi-line (even they are more than 80 characters)
>> - simply use dt_find_property(node, "xen,static-mem", NULL) to tell
>> whether using allocate_static_memory, and add error comment when
>> "xen,static-mem" is used but CONFIG_STATIC_MEMORY is not enabled.
>> - exporting p2m_insert_mapping() and introduce guest_physmap_add_pages
>> to cope with adding guest RAM p2m mapping with nr_pages.
>> - check both pbase and psize are page aligned
>> - simplify the code in the loops by moving append_static_memory_to_bank()
>> outside of the if/else.
>> ---
>>   xen/arch/arm/domain_build.c | 156 +++++++++++++++++++++++++++++++++++-
>>   xen/arch/arm/p2m.c          |   7 +-
>>   xen/include/asm-arm/p2m.h   |  11 +++
>>   3 files changed, 168 insertions(+), 6 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 6c86d52781..843b8514c7 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -480,6 +480,148 @@ fail:
>>             (unsigned long)kinfo->unassigned_mem >> 10);
>>   }
>>   
>> +#ifdef CONFIG_STATIC_MEMORY
>> +static bool __init append_static_memory_to_bank(struct domain *d,
>> +                                                struct membank *bank,
>> +                                                mfn_t smfn,
>> +                                                paddr_t size)
>> +{
>> +    int res;
>> +    unsigned int nr_pages = size >> PAGE_SHIFT;
>                                 ^ PFN_DOWN
> 
> 
>> +    /* Infer next GFN. */
>> +    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
>> +
>> +    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
>> +    if ( res )
>> +    {
>> +        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
>> +        return false;
>> +    }
>> +
>> +    bank->size = bank->size + size;
>> +
>> +    return true;
>> +}
>> +
>> +/* Allocate memory from static memory as RAM for one specific domain d. */
>> +static void __init allocate_static_memory(struct domain *d,
>> +                                          struct kernel_info *kinfo,
>> +                                          const struct dt_device_node *node)
>> +{
>> +    const struct dt_property *prop;
>> +    u32 addr_cells, size_cells, reg_cells;
>> +    unsigned int nr_banks, gbank = 0, bank = 0;
>> +    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
>> +    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
>> +    const __be32 *cell;
>> +    u64 tot_size = 0;
>> +    paddr_t pbase, psize, gsize;
>> +    mfn_t smfn;
>> +    int res;
>> +
>> +    prop = dt_find_property(node, "xen,static-mem", NULL);
>> +    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
>> +                               &addr_cells) )
>> +    {
>> +        printk(XENLOG_ERR
>> +               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
>> +        goto fail;
>> +    }
>> +
>> +    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
>> +                               &size_cells) )
>> +    {
>> +        printk(XENLOG_ERR
>> +               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
>> +        goto fail;
>> +    }
>> +    reg_cells = addr_cells + size_cells;
>> +
>> +    /* Start with GUEST_RAM0. */
>> +    gsize = ramsize[gbank];
>> +    kinfo->mem.bank[gbank].start = rambase[gbank];
>> +
>> +    cell = (const __be32 *)prop->value;
>> +    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
>> +
>> +    for ( ; bank < nr_banks; bank++ )
>> +    {
>> +        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
>> +        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
>> +
>> +        smfn = maddr_to_mfn(pbase);
>> +        res = acquire_domstatic_pages(d, smfn, psize >> PAGE_SHIFT, 0);
>                                                    ^ PFN_DOWN(psize)
> 
> 
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR
>> +                   "%pd: failed to acquire static memory: %d.\n", d, res);
>> +            goto fail;
>> +        }
>> +
>> +        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
>> +               d, bank, pbase, pbase + psize);
>> +
>> +        /*
>> +         * It shall be mapped to the fixed guest RAM banks(GUEST_RAM_BANK_BASES),
>> +         * And only when it exhausts the current guest RAM bank, it will seek
>> +         * to the next.
>> +         */
>> +        while ( 1 )
>> +        {
>> +            /* Map as much as possible the static range to the guest bank */
>> +            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
>> +                                               min(psize, gsize)) )
>> +                goto fail;
>> +
>> +            /*
>> +             * The current physical bank is fully mapped.
>> +             * Handle the next physical bank.
>> +             */
>> +            if ( gsize >= psize )
>> +            {
>> +                gsize = gsize - psize;
>> +                break;
>> +            }
>> +            /*
>> +             * When current guest bank size is not enough to map.
>> +             * Before seeking to the next, check if we still have available
>> +             * guest bank.
>> +             */
>> +            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
>> +            {
>> +                printk(XENLOG_ERR "Exhausted all fixed guest banks.\n");
>> +                goto fail;
>> +            }
>> +            else
>> +            {
>> +                psize = psize - gsize;
>> +                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
>                                          ^ PFN_DOWN
> 
> 
>> +                /* Update to the next guest bank. */
>> +                gbank++;
>> +                gsize = ramsize[gbank];
>> +                kinfo->mem.bank[gbank].start = rambase[gbank];
>> +            }
>> +        }
>> +
>> +        tot_size += psize;
>> +    }
>> +
>> +    kinfo->mem.nr_banks = ++gbank;
>> +    kinfo->unassigned_mem -= tot_size;
>> +    if ( kinfo->unassigned_mem )
>> +    {
>> +        printk(XENLOG_ERR
>> +               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\".\n");
>> +        goto fail;
> 
> Do we need to make this a fatal failure?
> 
> I am asking because unassigned_mem comes from the "memory" property of
> the domain in device tree which is kind of redundant with the
> introduction of xen,static-mem. In fact, I think it would be better to
> clarify the role of "memory" when "xen,static-mem" is also present.
> In that case, we could even make "memory" optional.

I requested to make it mandatory. Imagine you have a domU that has 1GB 
and now you want to switch to static memory. If we make the property 
optional, then there is a risk for the admin to not correctly pass the 
amount of memory. This may become unnoticed until late.

So I think making it mandatory is cheap for us and an easy way to 
confirm you properly sized the region. It also has the benefits to 
easily find out how much memory you gave to the guest (but that's just 
because I am lazy :)).

> In any case, even if we don't make "memory" optional, it might still be
> good to turn this error into a warning and ignore the remaining
> kinfo->unassigned_mem.

The behavior is matching the existing function and I think this is a 
good idea. If you ask 10MB of memory and we only give you 9MB, then at 
some point your guest is not going to be happy.

It is much better to know it in advance with a failure over discovering 
hours later when you see an OOM from your domain.

> 
> 
>> +    }
>> +
>> +    return;
>> +
>> +fail:
>> +    panic("Failed to allocate requested static memory for domain %pd.", d);
>> +}
>> +#endif
>> +
>>   static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>>                                      const struct dt_device_node *node)
>>   {
>> @@ -2452,7 +2594,19 @@ static int __init construct_domU(struct domain *d,
>>       /* type must be set before allocate memory */
>>       d->arch.type = kinfo.type;
>>   #endif
>> -    allocate_memory(d, &kinfo);
>> +    if ( !dt_find_property(node, "xen,static-mem", NULL) )
>> +        allocate_memory(d, &kinfo);
>> +#ifdef CONFIG_STATIC_MEMORY
>> +    else
>> +        allocate_static_memory(d, &kinfo, node);
>> +#else
>> +    else
>> +    {
>> +        printk(XENLOG_ERR
>> +               "CONFIG_STATIC_MEMORY must be enabled to use \"xen,static-mem\".\n");
>> +        return -EINVAL;
>> +    }
>> +#endif
> 
> To avoid the double else in the code, this part could be written like
> this which is a bit simpler I think:
> 
>      if ( !dt_find_property(node, "xen,static-mem", NULL) )
>          allocate_memory(d, &kinfo);
>      else
>      {
> #ifdef CONFIG_STATIC_MEMORY
>          allocate_static_memory(d, &kinfo, node);
> #else
>          printk(XENLOG_ERR
>                 "CONFIG_STATIC_MEMORY must be enabled to use \"xen,static-mem\".\n");
>          return -EINVAL;
> #endif
>      }
> 
> This is just a suggestion to improve readability, I am also OK with what
> you wrote.
> 
> (Another alternative would be to provide a stub allocate_static_memory
> implementation for !CONFIG_STATIC_MEMORY.)

My preference is 1) Stub function 2) your #ifdef proposal 3) Penny's 
approach.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 02 22:52:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 02 Sep 2021 22:52:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177707.323329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLvZF-0001fj-LP; Thu, 02 Sep 2021 22:52:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177707.323329; Thu, 02 Sep 2021 22:52:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLvZF-0001fc-Hf; Thu, 02 Sep 2021 22:52:21 +0000
Received: by outflank-mailman (input) for mailman id 177707;
 Thu, 02 Sep 2021 22:52:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLvZE-0001fS-Cn; Thu, 02 Sep 2021 22:52:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLvZE-0007HT-7u; Thu, 02 Sep 2021 22:52:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLvZE-0007nV-19; Thu, 02 Sep 2021 22:52:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLvZE-0001qR-0d; Thu, 02 Sep 2021 22:52:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tyQIheUlW3Ycxa0FlnKX+D5iFvAfJahaxYZjV5aGE7k=; b=r6fHYRr85dbb7oSjHEhdNZMcAs
	CXC1WR6oV4rrtm6scikL19bTnvNEnJ1kKc0V4BWRhEMg2AZ99xOwzD9tIgzCzM53hi2lykt5u4o4E
	/Epk8diUK7E1y3sFdeaDYgeiu/qrV+GAbjK7ma6HdjftPtbKw3aEgg0mMNohjCJEmTKk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164778-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164778: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 02 Sep 2021 22:52:20 +0000

flight 164778 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164778/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   16 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 00:26:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 00:26:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177719.323348 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLx28-0003uf-HT; Fri, 03 Sep 2021 00:26:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177719.323348; Fri, 03 Sep 2021 00:26:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLx28-0003uY-EF; Fri, 03 Sep 2021 00:26:16 +0000
Received: by outflank-mailman (input) for mailman id 177719;
 Fri, 03 Sep 2021 00:26:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx26-0003uO-NU; Fri, 03 Sep 2021 00:26:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx26-00011Y-BA; Fri, 03 Sep 2021 00:26:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx25-0004SC-W2; Fri, 03 Sep 2021 00:26:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx25-0004oj-VY; Fri, 03 Sep 2021 00:26:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=nrh3UH/pvC8HUOwb0DowZNjxIh5jbmJJITk0G8tPb1Q=; b=X0SBcsGZkI9MsRr+UA7Q0K1TQQ
	1Sa4AumaMSjvtWweVpC/sIcQOofpfJs+pAx/1/6pV1MFf7A5Vk/eCHRRZknZdd5pfSfVQchSi0SLq
	+yen51vdLMmkDtOJWU7/ejJnS9JCm5NyN8zWtRp7iM25Gj4szBxUdgZAZDgIQVAf4FG0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Subject: [ovmf bisection] complete build-i386
Message-Id: <E1mLx25-0004oj-VY@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 00:26:13 +0000

branch xen-unstable
xenbranch xen-unstable
job build-i386
testid xen-build

Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164784/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/ovmf/build-i386.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/ovmf/build-i386.xen-build --summary-out=tmp/164784.bisection-summary --basis-template=164674 --blessings=real,real-bisect,real-retry ovmf build-i386 xen-build
Searching for failure / basis pass:
 164778 fail [host=huxelrebe1] / 164674 [host=albana0] 164630 [host=huxelrebe0] 164556 [host=albana1] 164513 [host=albana1] 164461 [host=fiano0] 164406 [host=albana0] 164229 [host=huxelrebe0] 164173 [host=albana0] 164166 [host=albana1] 164160 ok.
Failure / basis pass flights: 164778 / 164160
(tree with no url: minios)
Tree: ovmf https://github.com/tianocore/edk2.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest b6bc203375b6efb6822ee4fe4a59be4f1918436b 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
Basis pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
Generating revisions with ./adhoc-revtuple-generator  https://github.com/tianocore/edk2.git#a776bbabd9e8f370feb089cdc016324b1d6a37c7-b6bc203375b6efb6822ee4fe4a59be4f1918436b git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823-b6e539830bf45e2d7a6bd86ddfdf003088b173b0 git://xenbits.xen.org/osstest/seabios.git#54082c81d96028ba8c76fbe6784085cf1df76b\
 20-54082c81d96028ba8c76fbe6784085cf1df76b20 git://xenbits.xen.org/xen.git#25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe-96607a8e680e7f965ca868d11f8b0636317d2618
Loaded 76351 nodes in revision graph
Searching for test results:
 164160 pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
 164166 [host=albana1]
 164173 [host=albana0]
 164229 [host=huxelrebe0]
 164406 [host=albana0]
 164461 [host=fiano0]
 164513 [host=albana1]
 164556 [host=albana1]
 164630 [host=huxelrebe0]
 164674 [host=albana0]
 164686 fail irrelevant
 164695 fail irrelevant
 164704 [host=fiano1]
 164718 fail irrelevant
 164728 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164733 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164737 fail 12e33dca4c0612a0975265e5ba641c6261a26455 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164740 [host=huxelrebe0]
 164743 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164746 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164754 pass a776bbabd9e8f370feb089cdc016324b1d6a37c7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 25da9455f1bb8a6d33039575a7b28bdfc4e3fcfe
 164755 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164751 fail e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164756 pass 7b4a99be8a39c12d3a7fc4b8db9f0eab4ac688d5 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 b1ee10be5625b7d502cef1e6ee3818610ab0d29c
 164759 pass 77d5fa80246e8784f89e109ff9dadfeb7089ff85 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 136c34c9bc4179dc64b15b2bb5f0c54ca4ddf823 54082c81d96028ba8c76fbe6784085cf1df76b20 8064488062641ae505b2a7369611c38057a7788b
 164761 pass 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164762 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164758 [host=huxelrebe0]
 164763 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164765 [host=huxelrebe0]
 164766 [host=huxelrebe0]
 164767 [host=huxelrebe0]
 164768 [host=huxelrebe0]
 164769 [host=huxelrebe0]
 164770 [host=huxelrebe0]
 164764 [host=huxelrebe0]
 164771 [host=huxelrebe0]
 164773 fail b6bc203375b6efb6822ee4fe4a59be4f1918436b 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164774 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164775 fail b6bc203375b6efb6822ee4fe4a59be4f1918436b 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164776 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164780 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164781 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164778 fail b6bc203375b6efb6822ee4fe4a59be4f1918436b 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 96607a8e680e7f965ca868d11f8b0636317d2618
 164782 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
 164784 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
Searching for interesting versions
 Result found: flight 164160 (pass), for basis pass
 For basis failure, parent search stopping at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356, results HASH(0x55576e7a07a0) HASH(0x555773b85a90) HASH(0x555773b87e38) For basis failure, parent search stopping at 52e2dabc0f8d3af09c213072ce8ba734302f585d 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d\
 96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356, results HASH(0x555773b71480) Result found: flight 164728 (fail), for basis failure (at ancestor ~18)
 Repro found: flight 164754 (pass), for basis pass
 Repro found: flight 164773 (fail), for basis failure
 0 revisions at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 b6e539830bf45e2d7a6bd86ddfdf003088b173b0 54082c81d96028ba8c76fbe6784085cf1df76b20 daaf007eb3467f900a2e20fadbc4c6f3bfcaa356
No revisions left to test, checking graph state.
 Result found: flight 164774 (pass), for last pass
 Result found: flight 164776 (fail), for first failure
 Repro found: flight 164780 (pass), for last pass
 Repro found: flight 164781 (fail), for first failure
 Repro found: flight 164782 (pass), for last pass
 Repro found: flight 164784 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf https://github.com/tianocore/edk2.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164784/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>

Revision graph left in /home/logs/results/bisect/ovmf/build-i386.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
164784: tolerable ALL FAIL

flight 164784 ovmf real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/164784/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-i386                    6 xen-build               fail baseline untested


jobs:
 build-i386                                                   fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 00:26:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 00:26:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177722.323363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLx2T-0004La-QZ; Fri, 03 Sep 2021 00:26:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177722.323363; Fri, 03 Sep 2021 00:26:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLx2T-0004LT-NL; Fri, 03 Sep 2021 00:26:37 +0000
Received: by outflank-mailman (input) for mailman id 177722;
 Fri, 03 Sep 2021 00:26:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx2S-0004L2-J3; Fri, 03 Sep 2021 00:26:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx2S-00011z-Ct; Fri, 03 Sep 2021 00:26:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx2S-0004T0-23; Fri, 03 Sep 2021 00:26:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLx2S-0007MB-1U; Fri, 03 Sep 2021 00:26:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=VCoF6OPfcN9kIDCgG0MkAQ/GagEHY4yXc/+JO99X2/o=; b=X4KflS1D4sjjH9+a+YOL8SEAGS
	bWLpAafW82bSO9EkHjYiByVKJ4Iovrqcgy06dkZyLP8m5AB2cXavvOdF80ZyHq0k7veC1DDNOJ+m7
	NK+FS2HG8F7NzTfwyPcxipsxa1wQgFSksssmW5lEcVa2mUfd9typ8CCMjsxmRN3dC7ow=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164783-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164783: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 00:26:36 +0000

flight 164783 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164783/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    2 days
Failing since        164686  2021-09-01 03:03:43 Z    1 days   17 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 00:39:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 00:39:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177734.323377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxEi-0006HT-1x; Fri, 03 Sep 2021 00:39:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177734.323377; Fri, 03 Sep 2021 00:39:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxEh-0006HM-Uv; Fri, 03 Sep 2021 00:39:15 +0000
Received: by outflank-mailman (input) for mailman id 177734;
 Fri, 03 Sep 2021 00:39:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mLxEg-0006HG-FY
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 00:39:14 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fad1be8b-f4cb-46f7-9bd6-f67864842b01;
 Fri, 03 Sep 2021 00:39:13 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7856860FD7;
 Fri,  3 Sep 2021 00:39:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fad1be8b-f4cb-46f7-9bd6-f67864842b01
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630629552;
	bh=y2icUBNg0/9LSlQTIW4ms/VZvVLzZCQ5l/Hx8gsPjFA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JbW+IilMz5RIKz2a4HoJk92KHJ3AgT2cFEVw5W2grEqIxPJeVUu7VDf7HXfEaPIu1
	 /kCNDi+CJnY0fsLdEzQIi8MGfqN2If96vHySwv8SVlkwnJXjaUW089cxAy2Qvwe6JM
	 41JNGPnCu2rSIaiw2bbiX9FLTPLz/IYqUBtA5NQY/wkz5nZxjtpaxXzQueLAWnMtZg
	 VnXqGdx6tKgcE+m8Kr+k4QMMy2cduN6Mvw9guj43uBGh8ktZzaWT+2PAudCUrjUNfi
	 0z6yxGkaH6R1YLlXeJ/AsgaWMq7fzicJFF9wjtbOZMDNFDZf0huEtS9QjHDIhzcy0p
	 foBGWbaOzib6A==
Date: Thu, 2 Sep 2021 17:39:11 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
In-Reply-To: <ac3a9fbd-7598-e400-6c6c-1214df7acee7@xen.org>
Message-ID: <alpine.DEB.2.21.2109021527330.26072@sstabellini-ThinkPad-T480s>
References: <20210824095045.2281500-1-penny.zheng@arm.com> <20210824095045.2281500-8-penny.zheng@arm.com> <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s> <ac3a9fbd-7598-e400-6c6c-1214df7acee7@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 2 Sep 2021, Julien Grall wrote:
> > > +    kinfo->mem.nr_banks = ++gbank;
> > > +    kinfo->unassigned_mem -= tot_size;
> > > +    if ( kinfo->unassigned_mem )
> > > +    {
> > > +        printk(XENLOG_ERR
> > > +               "Size of \"memory\" property doesn't match up with the
> > > sum-up of \"xen,static-mem\".\n");
> > > +        goto fail;
> > 
> > Do we need to make this a fatal failure?
> > 
> > I am asking because unassigned_mem comes from the "memory" property of
> > the domain in device tree which is kind of redundant with the
> > introduction of xen,static-mem. In fact, I think it would be better to
> > clarify the role of "memory" when "xen,static-mem" is also present.
> > In that case, we could even make "memory" optional.
> 
> I requested to make it mandatory. Imagine you have a domU that has 1GB and now
> you want to switch to static memory. If we make the property optional, then
> there is a risk for the admin to not correctly pass the amount of memory. This
> may become unnoticed until late.
> 
> So I think making it mandatory is cheap for us and an easy way to confirm you
> properly sized the region. It also has the benefits to easily find out how
> much memory you gave to the guest (but that's just because I am lazy :)).
> 
> > In any case, even if we don't make "memory" optional, it might still be
> > good to turn this error into a warning and ignore the remaining
> > kinfo->unassigned_mem.
> 
> The behavior is matching the existing function and I think this is a good
> idea. If you ask 10MB of memory and we only give you 9MB, then at some point
> your guest is not going to be happy.
> 
> It is much better to know it in advance with a failure over discovering hours
> later when you see an OOM from your domain.

OK, I didn't realize this was discussed already. Let's not revisit this
then.

My preference is primarily to make the device tree easier to write, but
nowadays nobody I know is writing the device tree by hand anymore (they
all use ImageBuilder). So if the device tree is generated then we are
fine either way as long as the binding is clear. So I am OK to drop my
suggestion of making "memory" optional. Let's think of a way to make
"memory" and "xen,static-mem" coexist instead.


There are two sides of the issue:
- the Xen implementation
- the binding


The Xen implementation is fine to panic if memory != xen,static-mem. In
that regard, the current patch is OK.


>From the binding perspective, I think it would be good to add a
statement to clarify. The binding doesn't necessarily need to match
exactly the implementation as the binding should be as future proof
and as flexible as possible.

>From the binding perspective two properties should mean different
things: memory the total memory amount and xen,static-mem the static
memory. If one day we'll have more types of memory, memory will cover
the total amount while xen,static-mem will cover a subset. So memory
must be greater or equal to xen,static-mem (even if today Xen only
supports memory == xen,static-mem).

So I would add:

"""
As the memory property represents the total memory of the domain, hence
the amount of memory selected by the memory property must be greater or
equal to the total amount specified by xen,static-mem. Other
configurations (memory amount less than xen,static-mem amount) are
invalid.
"""

This sentence has the purpose of clarifying that "memory" still need to
be populated and have a valid value. Then, it is OK for Xen to error
out if memory doesn't match xen,static-mem because that's the only
configuration supported. The error message could be:

Size of "memory" property doesn't match up with the sum-up of
"xen,static-mem". Unsupported configuration.


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 00:56:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 00:56:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177741.323399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxVi-0000kN-Nk; Fri, 03 Sep 2021 00:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177741.323399; Fri, 03 Sep 2021 00:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxVi-0000kG-KR; Fri, 03 Sep 2021 00:56:50 +0000
Received: by outflank-mailman (input) for mailman id 177741;
 Fri, 03 Sep 2021 00:56:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QtFM=NZ=ozlabs.org=dgibson@srs-us1.protection.inumbo.net>)
 id 1mLxVh-0000U1-DR
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 00:56:49 +0000
Received: from ozlabs.org (unknown [203.11.71.1])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc82b456-0c51-11ec-aee3-12813bfff9fa;
 Fri, 03 Sep 2021 00:56:43 +0000 (UTC)
Received: by ozlabs.org (Postfix, from userid 1007)
 id 4H0zrk2nScz9sX3; Fri,  3 Sep 2021 10:56:38 +1000 (AEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc82b456-0c51-11ec-aee3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=gibson.dropbear.id.au; s=201602; t=1630630598;
	bh=dYFvOAbI0hzwi7bgHPuL2nISN21A/HVkTWCIarLAldc=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=DMCqqC7Itikgm2VTUEE5MykJBeTtYQkUaBtP+KuJF/Y5jTcUC2LQWs1ef3S2IMUl3
	 RysX7Pwh5G3C+1QXEaTzQ5loyPMlyIRqnn+VJrtw1FLSH+R0ynhlUXGg+rbaiWSc1f
	 53PPEtkqp6WkbUnSX5FiujxAAdGniXO0crpUxZ60=
Date: Fri, 3 Sep 2021 10:50:45 +1000
From: David Gibson <david@gibson.dropbear.id.au>
To: Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= <f4bug@amsat.org>
Cc: qemu-devel@nongnu.org, Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>, Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com, Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org, Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	xen-devel@lists.xenproject.org, Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
Message-ID: <YTFxZb1Vg5pWVW9p@yekko>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="iHh9kZyoW/xDyYBn"
Content-Disposition: inline
In-Reply-To: <20210902161543.417092-22-f4bug@amsat.org>


--iHh9kZyoW/xDyYBn
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daud=E9 wrote:
> Each POWER cpu has its own has_work() implementation. Instead of
> overloading CPUClass on each PowerPCCPUClass init, register the
> generic ppc_cpu_has_work() handler, and have it call the POWER
> specific has_work().

I don't quite see the rationale for introducing a second layer of
indirection here.  What's wrong with switching the base has_work for
each cpu variant?

>=20
> Signed-off-by: Philippe Mathieu-Daud=E9 <f4bug@amsat.org>
> ---
>  target/ppc/cpu-qom.h  |  3 +++
>  target/ppc/cpu_init.c | 26 ++++++++++++++++++--------
>  2 files changed, 21 insertions(+), 8 deletions(-)
>=20
> diff --git a/target/ppc/cpu-qom.h b/target/ppc/cpu-qom.h
> index 5800fa324e8..ff2bafcde6f 100644
> --- a/target/ppc/cpu-qom.h
> +++ b/target/ppc/cpu-qom.h
> @@ -189,6 +189,9 @@ struct PowerPCCPUClass {
>      int bfd_mach;
>      uint32_t l1_dcache_size, l1_icache_size;
>  #ifndef CONFIG_USER_ONLY
> +#ifdef CONFIG_TCG
> +    bool (*has_work)(CPUState *cpu);
> +#endif /* CONFIG_TCG */
>      unsigned int gdb_num_sprs;
>      const char *gdb_spr_xml;
>  #endif
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index e2e721c2b81..bbad16cc1ec 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -7583,6 +7583,7 @@ static bool ppc_pvr_match_power7(PowerPCCPUClass *p=
cc, uint32_t pvr)
>      return false;
>  }
> =20
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>  static bool cpu_has_work_POWER7(CPUState *cs)
>  {
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
> @@ -7616,12 +7617,12 @@ static bool cpu_has_work_POWER7(CPUState *cs)
>          return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>      }
>  }
> +#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> =20
>  POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
>  {
>      DeviceClass *dc =3D DEVICE_CLASS(oc);
>      PowerPCCPUClass *pcc =3D POWERPC_CPU_CLASS(oc);
> -    CPUClass *cc =3D CPU_CLASS(oc);
> =20
>      dc->fw_name =3D "PowerPC,POWER7";
>      dc->desc =3D "POWER7";
> @@ -7630,7 +7631,6 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
>      pcc->pcr_supported =3D PCR_COMPAT_2_06 | PCR_COMPAT_2_05;
>      pcc->init_proc =3D init_proc_POWER7;
>      pcc->check_pow =3D check_pow_nocheck;
> -    cc->has_work =3D cpu_has_work_POWER7;
>      pcc->insns_flags =3D PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MF=
TB |
>                         PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
>                         PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
> @@ -7673,6 +7673,7 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
>      pcc->lpcr_pm =3D LPCR_P7_PECE0 | LPCR_P7_PECE1 | LPCR_P7_PECE2;
>      pcc->mmu_model =3D POWERPC_MMU_2_06;
>  #if defined(CONFIG_SOFTMMU)
> +    pcc->has_work =3D cpu_has_work_POWER7;
>      pcc->hash64_opts =3D &ppc_hash64_opts_POWER7;
>      pcc->lrg_decr_bits =3D 32;
>  #endif
> @@ -7743,6 +7744,7 @@ static bool ppc_pvr_match_power8(PowerPCCPUClass *p=
cc, uint32_t pvr)
>      return false;
>  }
> =20
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>  static bool cpu_has_work_POWER8(CPUState *cs)
>  {
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
> @@ -7784,12 +7786,12 @@ static bool cpu_has_work_POWER8(CPUState *cs)
>          return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>      }
>  }
> +#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> =20
>  POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
>  {
>      DeviceClass *dc =3D DEVICE_CLASS(oc);
>      PowerPCCPUClass *pcc =3D POWERPC_CPU_CLASS(oc);
> -    CPUClass *cc =3D CPU_CLASS(oc);
> =20
>      dc->fw_name =3D "PowerPC,POWER8";
>      dc->desc =3D "POWER8";
> @@ -7798,7 +7800,6 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
>      pcc->pcr_supported =3D PCR_COMPAT_2_07 | PCR_COMPAT_2_06 | PCR_COMPA=
T_2_05;
>      pcc->init_proc =3D init_proc_POWER8;
>      pcc->check_pow =3D check_pow_nocheck;
> -    cc->has_work =3D cpu_has_work_POWER8;
>      pcc->insns_flags =3D PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MF=
TB |
>                         PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
>                         PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
> @@ -7848,6 +7849,7 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
>                     LPCR_P8_PECE3 | LPCR_P8_PECE4;
>      pcc->mmu_model =3D POWERPC_MMU_2_07;
>  #if defined(CONFIG_SOFTMMU)
> +    pcc->has_work =3D cpu_has_work_POWER8;
>      pcc->hash64_opts =3D &ppc_hash64_opts_POWER7;
>      pcc->lrg_decr_bits =3D 32;
>      pcc->n_host_threads =3D 8;
> @@ -7941,6 +7943,7 @@ static bool ppc_pvr_match_power9(PowerPCCPUClass *p=
cc, uint32_t pvr)
>      return false;
>  }
> =20
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>  static bool cpu_has_work_POWER9(CPUState *cs)
>  {
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
> @@ -7998,12 +8001,12 @@ static bool cpu_has_work_POWER9(CPUState *cs)
>          return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>      }
>  }
> +#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> =20
>  POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
>  {
>      DeviceClass *dc =3D DEVICE_CLASS(oc);
>      PowerPCCPUClass *pcc =3D POWERPC_CPU_CLASS(oc);
> -    CPUClass *cc =3D CPU_CLASS(oc);
> =20
>      dc->fw_name =3D "PowerPC,POWER9";
>      dc->desc =3D "POWER9";
> @@ -8013,7 +8016,6 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
>                           PCR_COMPAT_2_05;
>      pcc->init_proc =3D init_proc_POWER9;
>      pcc->check_pow =3D check_pow_nocheck;
> -    cc->has_work =3D cpu_has_work_POWER9;
>      pcc->insns_flags =3D PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MF=
TB |
>                         PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
>                         PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
> @@ -8062,6 +8064,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
>      pcc->lpcr_pm =3D LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | LPCR_=
OEE;
>      pcc->mmu_model =3D POWERPC_MMU_3_00;
>  #if defined(CONFIG_SOFTMMU)
> +    pcc->has_work =3D cpu_has_work_POWER9;
>      /* segment page size remain the same */
>      pcc->hash64_opts =3D &ppc_hash64_opts_POWER7;
>      pcc->radix_page_info =3D &POWER9_radix_page_info;
> @@ -8150,6 +8153,7 @@ static bool ppc_pvr_match_power10(PowerPCCPUClass *=
pcc, uint32_t pvr)
>      return false;
>  }
> =20
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>  static bool cpu_has_work_POWER10(CPUState *cs)
>  {
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
> @@ -8207,12 +8211,12 @@ static bool cpu_has_work_POWER10(CPUState *cs)
>          return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>      }
>  }
> +#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> =20
>  POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
>  {
>      DeviceClass *dc =3D DEVICE_CLASS(oc);
>      PowerPCCPUClass *pcc =3D POWERPC_CPU_CLASS(oc);
> -    CPUClass *cc =3D CPU_CLASS(oc);
> =20
>      dc->fw_name =3D "PowerPC,POWER10";
>      dc->desc =3D "POWER10";
> @@ -8223,7 +8227,6 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
>                           PCR_COMPAT_2_06 | PCR_COMPAT_2_05;
>      pcc->init_proc =3D init_proc_POWER10;
>      pcc->check_pow =3D check_pow_nocheck;
> -    cc->has_work =3D cpu_has_work_POWER10;
>      pcc->insns_flags =3D PPC_INSNS_BASE | PPC_ISEL | PPC_STRING | PPC_MF=
TB |
>                         PPC_FLOAT | PPC_FLOAT_FSEL | PPC_FLOAT_FRES |
>                         PPC_FLOAT_FSQRT | PPC_FLOAT_FRSQRTE |
> @@ -8275,6 +8278,7 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
>      pcc->lpcr_pm =3D LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | LPCR_=
OEE;
>      pcc->mmu_model =3D POWERPC_MMU_3_00;
>  #if defined(CONFIG_SOFTMMU)
> +    pcc->has_work =3D cpu_has_work_POWER10;
>      /* segment page size remain the same */
>      pcc->hash64_opts =3D &ppc_hash64_opts_POWER7;
>      pcc->radix_page_info =3D &POWER10_radix_page_info;
> @@ -8796,6 +8800,12 @@ static bool ppc_cpu_has_work(CPUState *cs)
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
>      CPUPPCState *env =3D &cpu->env;
> =20
> +    if (cs->halted) {
> +        PowerPCCPUClass *pcc =3D POWERPC_CPU_GET_CLASS(cpu);
> +
> +        return pcc->has_work(cs);
> +    }
> +
>      return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>  }
>  #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */

--=20
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

--iHh9kZyoW/xDyYBn
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEdfRlhq5hpmzETofcbDjKyiDZs5IFAmExcWUACgkQbDjKyiDZ
s5Kd6Q/9ExvXgL0E8gsCJTsKVim2JIFxJm6lfW3Ljf0A7E4onyKybV1Gu4uLlUjF
j5ELZ1+AgWr3vOUIhcBT5V1pGKvYXc37CpiQ15Q92DTTfHCcku03DesecGmle2lS
SiUQFgxVA88pMj1bzbYdQhrwgNebNJzVUdmeGythMdPbN+U4NwFJ5BYs+hkj3v+U
maeTd4I/tJOySTZhNbNsRi+zahzOxt/idmrjbJ/XoT8z9EUCF54IE/Xf9xKvgcbQ
YMK+c4PePo2xQXuDozfJNurhJ++J94hysnxH1nihopOegCgBhy8ZyU2LjsXa6hdA
GkUnVb7tG//kw3uEYkm2z+0a80ZLmXKuR2M0gkkkGQFEftHLF2ZLNo9ou8HuWEzd
1BeejEEKrUKGb7IoNwAZ3XgXZ2dErbwrvzKEIMNo6e0w7nnLSPpEgkEYQe0c6JEE
ZUl0IAd541qcGbI8Ckerg652ymOvv4I3PhG7Rg3GhQhR4+7uByfOkGVAZwf+TPYM
/EBZjio3zJrj3j5tl6xuC6y1GBefg601jAXvJUQzTOuQyxsgcujgKtvexCDztUoG
fJIX+Pn1j5q54OAE7CRHS1YEiJXlzj58FDBMJ2p/sdqETP2iJ1KhfJQ8Vw9ObJ54
1qMEYGBJIChIYtsBLc2OkVECXoXqJr3yjVJ6wp5a+DW7vgtQtNs=
=ay8o
-----END PGP SIGNATURE-----

--iHh9kZyoW/xDyYBn--


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 00:56:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 00:56:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177740.323388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxVg-0000UE-GS; Fri, 03 Sep 2021 00:56:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177740.323388; Fri, 03 Sep 2021 00:56:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLxVg-0000U7-Cc; Fri, 03 Sep 2021 00:56:48 +0000
Received: by outflank-mailman (input) for mailman id 177740;
 Fri, 03 Sep 2021 00:56:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QtFM=NZ=ozlabs.org=dgibson@srs-us1.protection.inumbo.net>)
 id 1mLxVc-0000U1-JY
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 00:56:46 +0000
Received: from ozlabs.org (unknown [203.11.71.1])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc7b6e30-0c51-11ec-aee3-12813bfff9fa;
 Fri, 03 Sep 2021 00:56:43 +0000 (UTC)
Received: by ozlabs.org (Postfix, from userid 1007)
 id 4H0zrk2QpHz9sCD; Fri,  3 Sep 2021 10:56:38 +1000 (AEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc7b6e30-0c51-11ec-aee3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=gibson.dropbear.id.au; s=201602; t=1630630598;
	bh=XXRA9E9bkfiMuArjbNTfx+r3Un7G6QIyGWXT2AEimqw=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=S/oOJX4wVez+0NVv34NrPleBWRl5YwYrRYxkIYCYr2DBSXFoQW3ujaLS8VYNkCkdh
	 k+y3JMpbgn9tPsrOUDPXq+7t9kf9MvRy3873T3nWL9uuJ/xjlikC8YswJYUk9vfu76
	 vduXzNxO6Vij/+kbOiSMh1QGiktvH/h6gdwa0iwg=
Date: Fri, 3 Sep 2021 10:49:20 +1000
From: David Gibson <david@gibson.dropbear.id.au>
To: Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= <f4bug@amsat.org>
Cc: qemu-devel@nongnu.org, Bin Meng <bin.meng@windriver.com>,
	Eduardo Habkost <ehabkost@redhat.com>, Greg Kurz <groug@kaod.org>,
	haxm-team@intel.com, Kamil Rytarowski <kamil@netbsd.org>,
	qemu-ppc@nongnu.org, Anthony Perard <anthony.perard@citrix.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Jiaxun Yang <jiaxun.yang@flygoat.com>,
	Thomas Huth <thuth@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	Chris Wulff <crwulff@gmail.com>,
	Laurent Vivier <lvivier@redhat.com>,
	Cameron Esfahani <dirty@apple.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Max Filippov <jcmvbkbc@gmail.com>,
	Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
	Richard Henderson <richard.henderson@linaro.org>,
	Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Artyom Tarasenko <atar4qemu@gmail.com>,
	Aurelien Jarno <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	"Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Laurent Vivier <laurent@vivier.eu>,
	Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
	Wenchao Wang <wenchao.wang@intel.com>,
	xen-devel@lists.xenproject.org, Marek Vasut <marex@denx.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
	Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
	Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Stafford Horne <shorne@gmail.com>,
	Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
Subject: Re: [PATCH v3 20/30] target/ppc: Restrict has_work() handler to
 sysemu and TCG
Message-ID: <YTFxEPkY6Eq1+Xe/@yekko>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-21-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="6Kw+8nBbrWcu5Nha"
Content-Disposition: inline
In-Reply-To: <20210902161543.417092-21-f4bug@amsat.org>


--6Kw+8nBbrWcu5Nha
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Sep 02, 2021 at 06:15:33PM +0200, Philippe Mathieu-Daud=E9 wrote:
> Restrict has_work() to TCG sysemu.
>=20
> Signed-off-by: Philippe Mathieu-Daud=E9 <f4bug@amsat.org>

Acked-by: David Gibson <david@gibson.dropbear.id.au>

> ---
>  target/ppc/cpu_init.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>=20
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index 6aad01d1d3a..e2e721c2b81 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -8790,6 +8790,7 @@ static void ppc_cpu_set_pc(CPUState *cs, vaddr valu=
e)
>      cpu->env.nip =3D value;
>  }
> =20
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>  static bool ppc_cpu_has_work(CPUState *cs)
>  {
>      PowerPCCPU *cpu =3D POWERPC_CPU(cs);
> @@ -8797,6 +8798,7 @@ static bool ppc_cpu_has_work(CPUState *cs)
> =20
>      return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>  }
> +#endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> =20
>  static void ppc_cpu_reset(DeviceState *dev)
>  {
> @@ -9017,6 +9019,7 @@ static const struct TCGCPUOps ppc_tcg_ops =3D {
>    .tlb_fill =3D ppc_cpu_tlb_fill,
> =20
>  #ifndef CONFIG_USER_ONLY
> +  .has_work =3D ppc_cpu_has_work,
>    .cpu_exec_interrupt =3D ppc_cpu_exec_interrupt,
>    .do_interrupt =3D ppc_cpu_do_interrupt,
>    .cpu_exec_enter =3D ppc_cpu_exec_enter,
> @@ -9042,7 +9045,6 @@ static void ppc_cpu_class_init(ObjectClass *oc, voi=
d *data)
>      device_class_set_parent_reset(dc, ppc_cpu_reset, &pcc->parent_reset);
> =20
>      cc->class_by_name =3D ppc_cpu_class_by_name;
> -    cc->has_work =3D ppc_cpu_has_work;
>      cc->dump_state =3D ppc_cpu_dump_state;
>      cc->set_pc =3D ppc_cpu_set_pc;
>      cc->gdb_read_register =3D ppc_cpu_gdb_read_register;

--=20
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

--6Kw+8nBbrWcu5Nha
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEdfRlhq5hpmzETofcbDjKyiDZs5IFAmExcRAACgkQbDjKyiDZ
s5IeEBAAv3cpcInuLFJXsDvzVly7wSr5p5Tp9oNaBy8UvvGyoBX2eesw8Om/A3+P
S/tu8jqAI/GRfxWwBaE+wNg+Fmpdl3CK+ThR/6t9NA5BhoBwy5rwsM4V32BYkawN
NS5jP+L14XMC5MMa9o1IbnrNIPcpSA7kLTxq6sZxP1UaSvLecauVW2a14q630sUN
0mQEVEq0V/dO9bB01U775VN8Nzwxc6khrC8oufvhvSL22oYCjQTvOTslr7LEN5Ro
Bb/6zxAXhGR1A8aBLekm7RSlqY767gZjEtIdn1zjx6g/owGyARjDez0rzwhiBL8C
WqVsrRndk6jLRe1SJSRpZvgCx0EybrQyITVbeyMM4A/lXZYmDD2HKvOuXpUtnNZg
frxhAADG1KW6Y1Ufvklqyz9YGnFpztZL6XevPw1xfXXFvC3jyZpBNYP5XrilV44G
Sl7xomR8sRiJqzvJ2fWj475h07KiQ5NjXVvX5NvWy4okv9IphyXh8JoxyQsrA2b5
zIuqvUOy4GNPsunnxcNha90XiMsN6i5ds/ibiTyMwKbllU/hgUI3GEKVICncvbGt
LmIo3UFw/vTG7+lqCUxRZUtGBBgDZlsyxpkAGXfiBf+aISLrhsCHQeXxbD18AGBO
tECHtorVH7MbCNLHks/1Ey6DsZyaJnMdBoNjA+d4GpSUzASXzxI=
=o8a9
-----END PGP SIGNATURE-----

--6Kw+8nBbrWcu5Nha--


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 01:50:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 01:50:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177753.323410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLyLb-0005xR-K6; Fri, 03 Sep 2021 01:50:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177753.323410; Fri, 03 Sep 2021 01:50:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLyLb-0005xK-FO; Fri, 03 Sep 2021 01:50:27 +0000
Received: by outflank-mailman (input) for mailman id 177753;
 Fri, 03 Sep 2021 01:50:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyLZ-0005xA-U9; Fri, 03 Sep 2021 01:50:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyLZ-0000Df-Kr; Fri, 03 Sep 2021 01:50:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyLZ-0007ls-D3; Fri, 03 Sep 2021 01:50:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyLZ-0007dx-Cb; Fri, 03 Sep 2021 01:50:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1ujK+iTDU59k+1UNmmjPHa/4ecj4ziFwVX6SuXg0gsQ=; b=dcwsrY8OVC9eFEOhhxt0SPtqiJ
	HWH5HR5gPQNjam/R47Zm2YiImT/EqZpilUVMq4BFaiRKalE1V6MGmuKa4vWSAaaagAuReZ/dzmUHL
	3kxto20JZDwKvfq0LQCOUAZwxsvqY1qUE7q1P2fpU5G9UWWHmtE8KsAavjFXvY+Bj+do=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164779-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 164779: all pass - PUSHED
X-Osstest-Versions-This:
    xtf=0bb720b3c486bd3de62b8c32282eb5fa192b87f3
X-Osstest-Versions-That:
    xtf=bc26bc260cbfec1c6de1778ef17cf0faa54c0e03
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 01:50:25 +0000

flight 164779 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164779/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf                  0bb720b3c486bd3de62b8c32282eb5fa192b87f3
baseline version:
 xtf                  bc26bc260cbfec1c6de1778ef17cf0faa54c0e03

Last test of basis   164255  2021-08-19 12:11:19 Z   14 days
Testing same since   164779  2021-09-02 21:40:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   bc26bc2..0bb720b  0bb720b3c486bd3de62b8c32282eb5fa192b87f3 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 01:54:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 01:54:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177761.323424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLyPS-0006tC-9F; Fri, 03 Sep 2021 01:54:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177761.323424; Fri, 03 Sep 2021 01:54:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLyPS-0006t5-6E; Fri, 03 Sep 2021 01:54:26 +0000
Received: by outflank-mailman (input) for mailman id 177761;
 Fri, 03 Sep 2021 01:54:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyPR-0006sv-8a; Fri, 03 Sep 2021 01:54:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyPR-0000Gv-2T; Fri, 03 Sep 2021 01:54:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyPQ-0007sr-Qa; Fri, 03 Sep 2021 01:54:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLyPQ-0006FV-Q1; Fri, 03 Sep 2021 01:54:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jHB5/zQ4SzYASPkgspu/hZZx51XzGV9O2fY/3kBcv5U=; b=Y1nlaSDY2FuHAE47btf8+YTcXt
	IzKng/TtHKL5S4pv9YuplSVXLU0zXLJOvdDZfZLc1dsTmNahXn60MVe+mP0lHAvFknAB7E+bQi9Fj
	LX7i3hf/3iHFr0ncSZHGAPdgXjFBzyZ324Qov3wGvZlGJHo76GvbLJfobuIcn+0fIH1k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164772-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164772: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=9093028dd48c50bc0392791f78aab44afef57ead
X-Osstest-Versions-That:
    qemuu=079b1252e9de384385c9da910262312ec2e574c8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 01:54:24 +0000

flight 164772 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164772/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164757

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164757
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164757
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164757
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164757
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164757
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164757
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164757
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                9093028dd48c50bc0392791f78aab44afef57ead
baseline version:
 qemuu                079b1252e9de384385c9da910262312ec2e574c8

Last test of basis   164757  2021-09-02 08:06:50 Z    0 days
Testing same since   164772  2021-09-02 18:37:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Emanuele Giuseppe Esposito <eesposit@redhat.com>
  Fabrice Fontaine <fontaine.fabrice@gmail.com>
  Hanna Reitz <hreitz@redhat.com>
  Helge Konetzka <hk@zapateado.de>
  Igor Mammedov <imammedo@redhat.com>
  John Snow <jsnow@redhat.com>
  Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
  Markus Armbruster <armbru@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   079b1252e9..9093028dd4  9093028dd48c50bc0392791f78aab44afef57ead -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 03:14:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 03:14:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177768.323438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLzef-00083G-7l; Fri, 03 Sep 2021 03:14:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177768.323438; Fri, 03 Sep 2021 03:14:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mLzef-000839-4R; Fri, 03 Sep 2021 03:14:13 +0000
Received: by outflank-mailman (input) for mailman id 177768;
 Fri, 03 Sep 2021 03:14:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLzee-00082z-21; Fri, 03 Sep 2021 03:14:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLzed-00028j-HT; Fri, 03 Sep 2021 03:14:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mLzed-0002k4-Af; Fri, 03 Sep 2021 03:14:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mLzed-0006Qv-AA; Fri, 03 Sep 2021 03:14:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=WUWEXrQitFnaxbytGCo71zA7joD/Ycj1vPvOweG3KRk=; b=Zk/wflipNCqZMCD04eyEoaYCHm
	tQfZ9dFH50peqiQafQs+73RQoRKYXd0GZEMEhIEhSSzmQlcoFBx5w7yI8Jxr7m6prm4vGuFgSDZP8
	4/2NP1E2NuT2GcRiQaKIEwftreLN+W/ACPJzH8MZJ8pCOqG8K6woSWq6d6enHgHufYQo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164785-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164785: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=b6bc203375b6efb6822ee4fe4a59be4f1918436b
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 03:14:11 +0000

flight 164785 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164785/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 b6bc203375b6efb6822ee4fe4a59be4f1918436b
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   18 attempts
Testing same since   164764  2021-09-02 12:11:21 Z    0 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 04:03:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 04:03:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177776.323452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM0QH-0005ed-4w; Fri, 03 Sep 2021 04:03:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177776.323452; Fri, 03 Sep 2021 04:03:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM0QH-0005eW-1e; Fri, 03 Sep 2021 04:03:25 +0000
Received: by outflank-mailman (input) for mailman id 177776;
 Fri, 03 Sep 2021 04:03:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0QF-0005eM-PW; Fri, 03 Sep 2021 04:03:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0QF-00030n-LC; Fri, 03 Sep 2021 04:03:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0QF-0004SL-DB; Fri, 03 Sep 2021 04:03:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0QF-0008En-Cg; Fri, 03 Sep 2021 04:03:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rsXgzcxBi6Xc/X046RMTmmh7ckaC+iGBPRglI2cMeHQ=; b=LIk03MU2GCKpaWKjpwlDuGc4Fj
	kZncIy089+7AgjjDTMDzGwt+/YhsoKlp42BQBvUWl+tNAn1nfHXhx0NwrjCp31cwmMkX02Qej7wQh
	rdWqf1oxCT6w80Tndr4ZsbiCeoLtsGAA5aVHdcJ6BXAMumogsPwTwQOz9Pg/buGu3Guk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164787-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164787: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=c5e805ffe1098601e84fd501d5fa1d45e9d96a62
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 04:03:23 +0000

flight 164787 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164787/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 c5e805ffe1098601e84fd501d5fa1d45e9d96a62
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   19 attempts
Testing same since   164787  2021-09-03 03:15:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 04:35:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 04:35:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177783.323466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM0v0-0001Hb-Lx; Fri, 03 Sep 2021 04:35:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177783.323466; Fri, 03 Sep 2021 04:35:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM0v0-0001HU-I0; Fri, 03 Sep 2021 04:35:10 +0000
Received: by outflank-mailman (input) for mailman id 177783;
 Fri, 03 Sep 2021 04:35:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0uz-0001HG-QE; Fri, 03 Sep 2021 04:35:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0uz-0003Wx-I5; Fri, 03 Sep 2021 04:35:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0uz-0005Kq-6c; Fri, 03 Sep 2021 04:35:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM0uz-00081A-67; Fri, 03 Sep 2021 04:35:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yHD9xmRM2bbHsYrPGYq/UjbrF8F+/JToPxJjqJQ5w/I=; b=NToEGtHXH3s3WoLyFNdLBBQ1Qo
	yACDTq9uz2qwjJRckmi73jmvTjiqbO2Jct/ZXnZqFjnqGTNsK9uCG3G5s6TilaQk5gLYVYiO+m/nJ
	TKAdH1Ay0fIwWh8WfsZ/FSdkBHkYQru6M3fGtpU4KJFWXJJ618UuJAN50e7qf+4mOPHo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164788-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164788: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 04:35:09 +0000

flight 164788 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164788/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   20 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 04:52:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 04:52:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177791.323479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM1Bv-0004gG-Df; Fri, 03 Sep 2021 04:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177791.323479; Fri, 03 Sep 2021 04:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM1Bv-0004g9-AC; Fri, 03 Sep 2021 04:52:39 +0000
Received: by outflank-mailman (input) for mailman id 177791;
 Fri, 03 Sep 2021 04:52:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM1Bt-0004fz-Jm; Fri, 03 Sep 2021 04:52:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM1Bt-0003wb-Ey; Fri, 03 Sep 2021 04:52:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM1Bt-0005oT-1f; Fri, 03 Sep 2021 04:52:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM1Bt-0006Us-1B; Fri, 03 Sep 2021 04:52:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=h0SzT5xIkDw7uzw4mKAG3bLCJ4DQh7ulANlV/hoOod0=; b=f+0VnqL45Ycgj1fMJfRSpwYGsl
	ZW0+BPFiuPHm543bjUANn6xVedSJTf/ryaK4pIONj0XUGyfLN1W0DAi82dje/z7urzrmhICXmHUNP
	MT7wKmZXvppeEFy5AaNeDML3RyTWufyftqqN+vYFUloyA3yafuclMQAEZGu0egfDFDTU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164777-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164777: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit2:host-ping-check-xen:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7ba88a2a09f47e2e4f3e34215677a1d78a9e6a73
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 04:52:37 +0000

flight 164777 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164777/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2  10 host-ping-check-xen      fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7ba88a2a09f47e2e4f3e34215677a1d78a9e6a73
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  398 days
Failing since        152366  2020-08-01 20:49:34 Z  397 days  688 attempts
Testing same since   164777  2021-09-02 21:10:06 Z    0 days    1 attempts

------------------------------------------------------------
7153 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  fail    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2186225 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 07:32:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 07:32:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177800.323493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM3g0-0005iY-Qy; Fri, 03 Sep 2021 07:31:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177800.323493; Fri, 03 Sep 2021 07:31:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM3g0-0005iR-O2; Fri, 03 Sep 2021 07:31:52 +0000
Received: by outflank-mailman (input) for mailman id 177800;
 Fri, 03 Sep 2021 07:31:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM3fz-0005iH-7y; Fri, 03 Sep 2021 07:31:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM3fy-0006vz-Ve; Fri, 03 Sep 2021 07:31:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM3fy-0006Ec-OU; Fri, 03 Sep 2021 07:31:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM3fy-00034A-Nx; Fri, 03 Sep 2021 07:31:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3mutafMOaoMZwDIbXcKMXrrwEPJnIP3m+SG96J+1lyY=; b=Ln/cv7jkHbT+SqYgiZaNZTJF9O
	0jeP/eu8Js0Oc3qjimmmGRUrJYQw0g9LTzHb18yFOG/jPZdj63MR2jMk9d6B8tvIALCpW8c0sjBln
	YIoJ983Im2I40nB7e1NkpvjPGQ0lJg+NFk4qhTPuAIL3Vd7J3fw09UFr+ahLP6IUkgRM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164789-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164789: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=3dbaaab128fed98e015a938dada2e0824b0870b6
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 07:31:50 +0000

flight 164789 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164789/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              3dbaaab128fed98e015a938dada2e0824b0870b6
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  420 days
Failing since        151818  2020-07-11 04:18:52 Z  419 days  410 attempts
Testing same since   164789  2021-09-03 04:19:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71775 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 07:41:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 07:41:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177808.323508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM3pU-0007Qj-0A; Fri, 03 Sep 2021 07:41:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177808.323508; Fri, 03 Sep 2021 07:41:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM3pT-0007Qc-TS; Fri, 03 Sep 2021 07:41:39 +0000
Received: by outflank-mailman (input) for mailman id 177808;
 Fri, 03 Sep 2021 07:41:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mM3pS-0007QW-E1
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 07:41:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mM3pS-00075t-6C; Fri, 03 Sep 2021 07:41:38 +0000
Received: from 54-240-197-239.amazon.com ([54.240.197.239]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mM3pR-0002on-VM; Fri, 03 Sep 2021 07:41:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=ksqcf2jF0XIRZewBEItfZVaqUXtATCiTEIw7T76+bxM=; b=0225mFHrUzVqNYOkqlRquTHDE1
	Y62bkr4fGbW14yd6FQuHGaeGaNIRet0v/pY5CEX8dq9CPNLHEo4sBMfKfd5P0OyR/mvFVOjcHm47D
	nLNEDXwNhDKyuiwBcpoPRjGZtPE4rN7Q/UQvzoAuq//LAKqxXEhxMwqjmCLlvEj3hXUw=;
Subject: Re: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-8-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
 <ac3a9fbd-7598-e400-6c6c-1214df7acee7@xen.org>
 <alpine.DEB.2.21.2109021527330.26072@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <034628a6-5461-f4a9-3339-a0d6f3d68f80@xen.org>
Date: Fri, 3 Sep 2021 08:41:36 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109021527330.26072@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 03/09/2021 01:39, Stefano Stabellini wrote:
> On Thu, 2 Sep 2021, Julien Grall wrote:
>>>> +    kinfo->mem.nr_banks = ++gbank;
>>>> +    kinfo->unassigned_mem -= tot_size;
>>>> +    if ( kinfo->unassigned_mem )
>>>> +    {
>>>> +        printk(XENLOG_ERR
>>>> +               "Size of \"memory\" property doesn't match up with the
>>>> sum-up of \"xen,static-mem\".\n");
>>>> +        goto fail;
>>>
>>> Do we need to make this a fatal failure?
>>>
>>> I am asking because unassigned_mem comes from the "memory" property of
>>> the domain in device tree which is kind of redundant with the
>>> introduction of xen,static-mem. In fact, I think it would be better to
>>> clarify the role of "memory" when "xen,static-mem" is also present.
>>> In that case, we could even make "memory" optional.
>>
>> I requested to make it mandatory. Imagine you have a domU that has 1GB and now
>> you want to switch to static memory. If we make the property optional, then
>> there is a risk for the admin to not correctly pass the amount of memory. This
>> may become unnoticed until late.
>>
>> So I think making it mandatory is cheap for us and an easy way to confirm you
>> properly sized the region. It also has the benefits to easily find out how
>> much memory you gave to the guest (but that's just because I am lazy :)).
>>
>>> In any case, even if we don't make "memory" optional, it might still be
>>> good to turn this error into a warning and ignore the remaining
>>> kinfo->unassigned_mem.
>>
>> The behavior is matching the existing function and I think this is a good
>> idea. If you ask 10MB of memory and we only give you 9MB, then at some point
>> your guest is not going to be happy.
>>
>> It is much better to know it in advance with a failure over discovering hours
>> later when you see an OOM from your domain.
> 
> OK, I didn't realize this was discussed already. Let's not revisit this
> then.
> 
> My preference is primarily to make the device tree easier to write, but
> nowadays nobody I know is writing the device tree by hand anymore (they
> all use ImageBuilder).So if the device tree is generated then we are
> fine either way as long as the binding is clear. So I am OK to drop my
> suggestion of making "memory" optional. Let's think of a way to make
> "memory" and "xen,static-mem" coexist instead.
> 
> 
> There are two sides of the issue:
> - the Xen implementation
> - the binding
> 
> 
> The Xen implementation is fine to panic if memory != xen,static-mem. In
> that regard, the current patch is OK.
> 
> 
>  From the binding perspective, I think it would be good to add a
> statement to clarify. The binding doesn't necessarily need to match
> exactly the implementation as the binding should be as future proof
> and as flexible as possible.

So I agree that the binding doesn't have to match the implementation. 
But... the binding always have be more restrictive than the 
implementation. Otherwise, someone following the binding may be not able 
to use it with Xen.

> 
>  From the binding perspective two properties should mean different
> things: memory the total memory amount and xen,static-mem the static
> memory. If one day we'll have more types of memory, memory will cover
> the total amount while xen,static-mem will cover a subset. So memory
> must be greater or equal to xen,static-mem (even if today Xen only
> supports memory == xen,static-mem).
> 
> So I would add:
> 
> """
> As the memory property represents the total memory of the domain, hence
> the amount of memory selected by the memory property must be greater or
> equal to the total amount specified by xen,static-mem. Other
> configurations (memory amount less than xen,static-mem amount) are
> invalid.
> """
> 
> This sentence has the purpose of clarifying that "memory" still need to
> be populated and have a valid value. Then, it is OK for Xen to error
> out if memory doesn't match xen,static-mem because that's the only
> configuration supported. 
How about writing something like "The property 'memory' should match the 
amount of memory given to the guest. Currently, it is only possible to 
either allocate static memory or let Xen chose. *Mixing* is not supported'.

Then if we add the mixing, we could say 'From Xen XX.YY, mixing will be 
allowed'.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:06:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:06:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177820.323519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4DN-0002Lb-Ga; Fri, 03 Sep 2021 08:06:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177820.323519; Fri, 03 Sep 2021 08:06:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4DN-0002LU-Cr; Fri, 03 Sep 2021 08:06:21 +0000
Received: by outflank-mailman (input) for mailman id 177820;
 Fri, 03 Sep 2021 08:06:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=X61r=NZ=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mM4DM-0002LO-7F
 for xen-devel@lists.xen.org; Fri, 03 Sep 2021 08:06:20 +0000
Received: from mail-pj1-x102f.google.com (unknown [2607:f8b0:4864:20::102f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d96f000e-9f50-4c59-b499-15ab5ce0d2d4;
 Fri, 03 Sep 2021 08:06:19 +0000 (UTC)
Received: by mail-pj1-x102f.google.com with SMTP id
 j10-20020a17090a94ca00b00181f17b7ef7so3322057pjw.2
 for <xen-devel@lists.xen.org>; Fri, 03 Sep 2021 01:06:18 -0700 (PDT)
Received: from laputa (p784a44f4.tkyea130.ap.so-net.ne.jp. [120.74.68.244])
 by smtp.gmail.com with ESMTPSA id r8sm5109955pgp.30.2021.09.03.01.06.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:06:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d96f000e-9f50-4c59-b499-15ab5ce0d2d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:content-transfer-encoding:in-reply-to;
        bh=VSDI1b+kKShx1vFtwYGy2PerZEjlalFPEQbbPa66J9o=;
        b=Gorjw3gj2HqBc5LgjLHiFLqx01XeiuFebKz7161NrET5y7DATwcbm57599urNvtfwj
         Ag9PWaYhiOreAYtCB90WdV1AwXwKbFMtiOXd/XNQ082HRZMqLb50cyYPG/exk9xYjPho
         ToB0C1NGznEQirewsZZlxFDl6icG8eGgoykh+Q2XqjjI4Jb9AEB7ZJbQ2f7Dm3TrlfDd
         M7R7aF/Z/kkgxdEhNbnn2K1g/wmnNVAc8K+xA9wUrCHc2UuUy2zoMjWcOEfq6SgFbRKW
         6nVyjcbAsw4HDXpWErmzkNRqZlkUfW9y2kInKspLlg/WcSTsDAkpxA/ssSxnonlOH/GE
         aidA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:content-transfer-encoding
         :in-reply-to;
        bh=VSDI1b+kKShx1vFtwYGy2PerZEjlalFPEQbbPa66J9o=;
        b=lwc46S5kRapBDek1R+Mz5eJFB1uVMhTQVlvPb7mHVy2T66OMmS1AmPN5QgyiOUjiag
         Ulv4V1SG7OUt4s+zmTncqlLCYAee2zDsaF2qhT9ybrZgJ6UoY1fYGEypGHt8UXBYnQjB
         ZERO3lukUlGN50cRQGukNQgd0HMhoXZM6/Qn3VmFqW2a/5XxD0qhP8jG2zC/abb3TlNf
         MwWOfqPvo2Vn11wFxZh5qRKkRotffRuzAyeP/DjmoIBCUsRy8i/U4pFUy7lH2gmhZ4tD
         8f0Gq9IWWIPZI5pjkUAEy8B2Vczv176AakMzmvihCKlE7QWmVCmrvlQEK3nElEHaIMXB
         yBTA==
X-Gm-Message-State: AOAM5316mHk23jJl+F0lJOMr1hCJVo06nUH8MvMdBa47xL/PHjMHMfqf
	wuPe9jcGYk9EczVLTLl4TV/tJw==
X-Google-Smtp-Source: ABdhPJxqaZrzFD1/AwE3AQ1yYzaaKX9bSU33Qt7iVq3Iz0X2S2enbVNdAp2/mMETg7+Q06fRunv4Bg==
X-Received: by 2002:a17:90b:1085:: with SMTP id gj5mr2165236pjb.66.1630656377947;
        Fri, 03 Sep 2021 01:06:17 -0700 (PDT)
Date: Fri, 3 Sep 2021 17:06:09 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Alex Benn??e <alex.bennee@linaro.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	virtio-dev@lists.oasis-open.org,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>, Wei.Chen@arm.com,
	olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
	Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org,
	jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org,
	Elena Afanasova <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210903080609.GD47953@laputa>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
 <875yvkh1p1.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <875yvkh1p1.fsf@linaro.org>

Alex,

On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:
> 
> Stefan Hajnoczi <stefanha@redhat.com> writes:
> 
> > [[PGP Signed Part:Undecided]]
> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
> >> > Could we consider the kernel internally converting IOREQ messages from
> >> > the Xen hypervisor to eventfd events? Would this scale with other kernel
> >> > hypercall interfaces?
> >> > 
> >> > So any thoughts on what directions are worth experimenting with?
> >>  
> >> One option we should consider is for each backend to connect to Xen via
> >> the IOREQ interface. We could generalize the IOREQ interface and make it
> >> hypervisor agnostic. The interface is really trivial and easy to add.
> >> The only Xen-specific part is the notification mechanism, which is an
> >> event channel. If we replaced the event channel with something else the
> >> interface would be generic. See:
> >> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52
> >
> > There have been experiments with something kind of similar in KVM
> > recently (see struct ioregionfd_cmd):
> > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/
> 
> Reading the cover letter was very useful in showing how this provides a
> separate channel for signalling IO events to userspace instead of using
> the normal type-2 vmexit type event. I wonder how deeply tied the
> userspace facing side of this is to KVM? Could it provide a common FD
> type interface to IOREQ?

Why do you stick to a "FD" type interface?

> As I understand IOREQ this is currently a direct communication between
> userspace and the hypervisor using the existing Xen message bus. My

With IOREQ server, IO event occurrences are notified to BE via Xen's event
channel, while the actual contexts of IO events (see struct ioreq in ioreq.h)
are put in a queue on a single shared memory page which is to be assigned
beforehand with xenforeignmemory_map_resource hypervisor call.

> worry would be that by adding knowledge of what the underlying
> hypervisor is we'd end up with excess complexity in the kernel. For one
> thing we certainly wouldn't want an API version dependency on the kernel
> to understand which version of the Xen hypervisor it was running on.

That's exactly what virtio-proxy in my proposal[1] does; All the hypervisor-
specific details of IO event handlings are contained in virtio-proxy
and virtio BE will communicate with virtio-proxy through a virtqueue
(yes, virtio-proxy is seen as yet another virtio device on BE) and will
get IO event-related *RPC* callbacks, either MMIO read or write, from
virtio-proxy.

See page 8 (protocol flow) and 10 (interfaces) in [1].

If kvm's ioregionfd can fit into this protocol, virtio-proxy for kvm
will hopefully be implemented using ioregionfd.

-Takahiro Akashi

[1] https://op-lists.linaro.org/pipermail/stratos-dev/2021-August/000548.html

> >> There is also another problem. IOREQ is probably not be the only
> >> interface needed. Have a look at
> >> https://marc.info/?l=xen-devel&m=162373754705233&w=2. Don't we also need
> >> an interface for the backend to inject interrupts into the frontend? And
> >> if the backend requires dynamic memory mappings of frontend pages, then
> >> we would also need an interface to map/unmap domU pages.
> >> 
> >> These interfaces are a lot more problematic than IOREQ: IOREQ is tiny
> >> and self-contained. It is easy to add anywhere. A new interface to
> >> inject interrupts or map pages is more difficult to manage because it
> >> would require changes scattered across the various emulators.
> >
> > Something like ioreq is indeed necessary to implement arbitrary devices,
> > but if you are willing to restrict yourself to VIRTIO then other
> > interfaces are possible too because the VIRTIO device model is different
> > from the general purpose x86 PIO/MMIO that Xen's ioreq seems to
> > support.
> 
> It's true our focus is just VirtIO which does support alternative
> transport options however most implementations seem to be targeting
> virtio-mmio for it's relative simplicity and understood semantics
> (modulo a desire for MSI to reduce round trip latency handling
> signalling).
> 
> >
> > Stefan
> >
> > [[End of PGP Signed Part]]
> 
> 
> -- 
> Alex Bennée


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:15:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:15:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177826.323529 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4MP-0003zC-DQ; Fri, 03 Sep 2021 08:15:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177826.323529; Fri, 03 Sep 2021 08:15:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4MP-0003z5-AW; Fri, 03 Sep 2021 08:15:41 +0000
Received: by outflank-mailman (input) for mailman id 177826;
 Fri, 03 Sep 2021 08:15:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM4MO-0003yv-0t; Fri, 03 Sep 2021 08:15:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM4MN-0008Ci-J4; Fri, 03 Sep 2021 08:15:39 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM4MN-0008Js-BR; Fri, 03 Sep 2021 08:15:39 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM4MN-0000A2-At; Fri, 03 Sep 2021 08:15:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=z3y13KmUtLWDWt2g8jHJ5FzyJm9gqD8TLgA3Q4+HrX0=; b=2c6OwDooFrzZ/r4CIbZS8wKGGs
	Z2sL9aWkOEcKtCSujipBpozkjhSbD6Fs5WKk5wsUnoSGj/K7L18sc5dmrsJJbh/zRmTh7Pwo8Fbs7
	oMY3Be8WTRZ/QWygZMD1oI1n4gJdLeeEbvJ9WZPt8rUGw9M1/uPPQhzvmrfFO6fJ49H0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164790-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164790: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 08:15:39 +0000

flight 164790 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164790/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   21 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177836.323555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4e5-00071E-GG; Fri, 03 Sep 2021 08:33:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177836.323555; Fri, 03 Sep 2021 08:33:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4e5-000711-Ca; Fri, 03 Sep 2021 08:33:57 +0000
Received: by outflank-mailman (input) for mailman id 177836;
 Fri, 03 Sep 2021 08:33:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4e4-0006kQ-CN
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:33:56 +0000
Received: from mail-ed1-x52d.google.com (unknown [2a00:1450:4864:20::52d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 53a80e4d-17ef-4a2f-8cff-1eaa68946a8d;
 Fri, 03 Sep 2021 08:33:51 +0000 (UTC)
Received: by mail-ed1-x52d.google.com with SMTP id z19so6890788edi.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:51 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53a80e4d-17ef-4a2f-8cff-1eaa68946a8d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=zjlQkuHh6kZKwoZ8mUieXYtJ9esYGLvw5t0GDKDUbQk=;
        b=Xgp3IkxX/SfY/PfG+BOgIO+ADKUy/5Rb8UIuqEw2MjrDLDSn+HntfDhndHbMQEaScL
         FjmrDAnyZcW6+xVcPZwu1EP7ndImdZ0KeimUQLoaibXp19/yI5ifx9Mgd4XDHmgSsifl
         DEvRHIV40w0i9zsWwxea2m9bOGHPSDHt24SoDK7stOg+g1Tdc/9POfAGzlMJ9dfQ3CHs
         Di8p521g87IwoOeOcp2ltO9kq52QKbjG70LiiCsYOZqXZK9/EjouzbmL5WtPRAbiWcCF
         lxvSO0V1GfTAsxF6og3u/5lCxROSkhvfw5FJOZueqx+bwmhq+i8/2CAFs7DUgsQ90wSm
         4KEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=zjlQkuHh6kZKwoZ8mUieXYtJ9esYGLvw5t0GDKDUbQk=;
        b=UdVccc3g9rFVrXh81QNyHCPyMbdBIJTX7yfPGXxwB1o5FwwNVs2xqTL7hK2836W9M2
         35TtrgD+FzboohBUYt9XoxABD8ZjoRPHnejIO1eFgiyprFBi/mKMOvhy/EYoZWvnPEzj
         TmdmGEMGmoHoRTynt3ZY7X7jYsON2SrK/6UPq5u9CY39jk6e1KOvOqBu8kKOCiyXP1wX
         rp/IplvkD4uYtLTDmuwv8m2z9Sqy2mXSqrEplRVV3ZQ29r80+TGdVgMaRuL62IV7YQGC
         70QtshEMrGWbHKpTv6hN0Ep04QyFB+AAkgY/tkAbgAiHxNB+dvjIBciiltTAWZwtDKGe
         k+sg==
X-Gm-Message-State: AOAM530+wApQVPgt820BrvHk0yNdDbrcYVAx4xOuJjqbepovzBkHapVl
	nMDXIPzA7Spq0OXfBKNT+lCyNJPyikI=
X-Google-Smtp-Source: ABdhPJwdIkQ8KNRmfmDrpt7oR/v1jpWaKOTo/d+eapgA54xtdvm5mj3qW6yJP5PMshdGxMxg314tWA==
X-Received: by 2002:a05:6402:1cb4:: with SMTP id cz20mr2815841edb.0.1630658030445;
        Fri, 03 Sep 2021 01:33:50 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 01/11] xen/arm: Add new device type for PCI
Date: Fri,  3 Sep 2021 11:33:37 +0300
Message-Id: <20210903083347.131786-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add new device type (DEV_PCI) to distinguish PCI devices from platform
DT devices, so some drivers, like IOMMU, can handle PCI devices
differently.

While at it fix dev_is_dt macro.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/include/asm-arm/device.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 5ecd5e7bd15e..7bf040560363 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -4,6 +4,7 @@
 enum device_type
 {
     DEV_DT,
+    DEV_PCI,
 };
 
 struct dev_archdata {
@@ -25,9 +26,8 @@ typedef struct device device_t;
 
 #include <xen/device_tree.h>
 
-/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
-#define dev_is_pci(dev) ((void)(dev), 0)
-#define dev_is_dt(dev)  ((dev->type == DEV_DT)
+#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
+#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
 
 enum device_class
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177835.323544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4e1-0006kd-6M; Fri, 03 Sep 2021 08:33:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177835.323544; Fri, 03 Sep 2021 08:33:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4e1-0006kW-3I; Fri, 03 Sep 2021 08:33:53 +0000
Received: by outflank-mailman (input) for mailman id 177835;
 Fri, 03 Sep 2021 08:33:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4dz-0006kQ-Jc
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:33:51 +0000
Received: from mail-ed1-x529.google.com (unknown [2a00:1450:4864:20::529])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3764bfc1-031b-4d72-96d0-123a32cca3d1;
 Fri, 03 Sep 2021 08:33:50 +0000 (UTC)
Received: by mail-ed1-x529.google.com with SMTP id j13so6893466edv.13
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:50 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3764bfc1-031b-4d72-96d0-123a32cca3d1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=XT9IurSUZ2x5a+pdMA7R9m7eqgBNcFs0ABtEwva/rOk=;
        b=KH3T7H6XUTIO6kln10N/XlkP8zdINoBf8lwUi0XQJKd3eTct3CAeswZD1DLkoz411o
         HUui7uH/2IXPivd79xANiGs9dP1FXSRRH6o6HsQ+TGuJ1+wD/UP64P73vzqEovMOD2C2
         hyvQaXvyOA9fPsQOtEk69TV4XFur7vWkxPP50YZjReOl7udubtCsEtEx5rmon0ofhDVW
         qM3QQkgy3BS2RXc5yRPfi/5bUfBCI5mi0RMyXAfKaQ1lmxWDoUVkoIEbOeB5MZb6IxNi
         0idRKSrO3nWli6QKp4FxTiIg5+vmqcyzi6Ly8ksc5KOFpTlKK5o05svXNQpHVDStaCUE
         ynKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=XT9IurSUZ2x5a+pdMA7R9m7eqgBNcFs0ABtEwva/rOk=;
        b=WtVYx+mcZyc2lCj5KSAt8RZ1LucXX/DjXvq0K7MUtWrg6Iuzj5nDH2C464hBnitVWz
         FVWkcpZakU6dH7mMJsbHpGEVj7LCGEFU6Jq+CjqWbN5BUvUlTJ18Z1qTRi9jZ0W22unM
         0KOgLcPD7Noenn8UwDMrj5aoSeAwWA3QgV7VJRHOQSrwtTVXVu4pB8OH08z1ivmCuKVk
         EpsGg66GmHORg0aP15GWU/dlaD09PwjypjV7XcS5ZQJnHzuNyAUuQSMLMIRm3Z7REkfk
         I5eFTKi2jMrBpPmmlhhHcoOkjSYA1EZeKt6vXd6KL2+f2r/olJpX+AoYcrOFdMZpeHfD
         DWOw==
X-Gm-Message-State: AOAM531ISQXsqbWGA4UlIZgZ99+cld125r0qpy2Pc/A8GlqPjLvKC5ta
	A/49a+E+VMOrwYaF1dxtz0CdNVTJdGE=
X-Google-Smtp-Source: ABdhPJwOTG2NN7AeYtisr6AO0pYLJ2YbABVf7/DstvKMpb2rCJEQH5vNugO7k5U5hKNMG3xdF37o5w==
X-Received: by 2002:a05:6402:84e:: with SMTP id b14mr2731110edz.260.1630658029127;
        Fri, 03 Sep 2021 01:33:49 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 00/11] PCI devices passthrough on Arm, part 2
Date: Fri,  3 Sep 2021 11:33:36 +0300
Message-Id: <20210903083347.131786-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This is an assorted series of patches which aim is to make some further
basis for PCI passthrough on Arm support. The series continues the work
published earlier by Arm [1] and adds new helpers and clears the way for
vPCI changes which will follow.

Thank you,
Oleksandr

[1] https://patchwork.kernel.org/project/xen-devel/cover/cover.1629366665.git.rahul.singh@arm.com/

Oleksandr Andrushchenko (10):
  xen/arm: Add new device type for PCI
  xen/arm: Add dev_to_pci helper
  xen/arm: Introduce pci_find_host_bridge_node helper
  xen/device-tree: Make dt_find_node_by_phandle global
  xen/arm: Mark device as PCI while creating one
  libxl: Allow removing PCI devices for all types of domains
  libxl: Only map legacy PCI IRQs if they are supported
  xen/arm: Setup MMIO range trap handlers for hardware domain
  xen/arm: Do not map PCI ECAM space to Domain-0's p2m
  xen/arm: Process pending vPCI map/unmap operations

Oleksandr Tyshchenko (1):
  xen/domain: Call pci_release_devices() when releasing domain resources

 tools/libs/light/Makefile          |  4 +++
 tools/libs/light/libxl_pci.c       | 15 ++++++--
 xen/arch/arm/domain.c              |  9 ++++-
 xen/arch/arm/domain_build.c        |  3 ++
 xen/arch/arm/pci/ecam.c            | 28 +++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 55 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci.c             | 10 ++++++
 xen/arch/arm/traps.c               |  6 ++++
 xen/arch/arm/vpci.c                | 13 +++++++
 xen/common/device_tree.c           |  2 +-
 xen/drivers/passthrough/pci.c      |  3 ++
 xen/include/asm-arm/device.h       |  6 ++--
 xen/include/asm-arm/pci.h          | 30 +++++++++++++++-
 xen/include/xen/device_tree.h      |  2 ++
 14 files changed, 178 insertions(+), 8 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177837.323566 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eA-0007LT-Nx; Fri, 03 Sep 2021 08:34:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177837.323566; Fri, 03 Sep 2021 08:34:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eA-0007LH-KJ; Fri, 03 Sep 2021 08:34:02 +0000
Received: by outflank-mailman (input) for mailman id 177837;
 Fri, 03 Sep 2021 08:34:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4e9-0006kQ-Ce
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:01 +0000
Received: from mail-ej1-x629.google.com (unknown [2a00:1450:4864:20::629])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 24e6dbd2-53dd-4a2f-b109-cd7d7a4e1bc1;
 Fri, 03 Sep 2021 08:33:52 +0000 (UTC)
Received: by mail-ej1-x629.google.com with SMTP id mf2so10484692ejb.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:52 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24e6dbd2-53dd-4a2f-b109-cd7d7a4e1bc1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=e6QStZ2WIeXjyjg6BwJ+X2ETh+SFauYaLcc7bbXleUI=;
        b=nWqFogticuAbAM8eSH4t1N6ThkVzCw+SXxxF9K1ll2yiZMh1ba0KqHkEcuIICyZxA+
         SzrMGymV0AEdKm8ZZ/3WuQ6zOTDzu3Qxl2rYf1kjguYpIH6KYlhCyq4IRG8TfcKBRTEa
         iMEDEVpEa1K4k9kdeisup0kiYmqigISQlt8BUksAZ9OhIz75hqYC7L9u4YBATeJvnifO
         b13khRAluSa+dJjLlvArcmT+T3oNHIcHa6lln6cGdZNaR3DGxhgy9yz1WLBpn0CaTKgf
         JKlqdN0FSVOjxPDLSx+rh4zcWbdqnO+Fq6F81ZteVy6jV9HuhiWHdOcfUqqsdjHTdnlT
         suoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=e6QStZ2WIeXjyjg6BwJ+X2ETh+SFauYaLcc7bbXleUI=;
        b=DqfUBp3HYqyVJuvEWaAbOqwC+iP0kKM/BtDXEVHjL6xQ2EfXLT3VuN0ZFMiZ+SAzPs
         Moa683XQnMe0Xi8Erj6k69B5Pu6kd3hQVL0vD9jcAB4EQ6oK4YdrGWTsJ+3UUyVIazwE
         Xqgl56ITOKzRi0jItqS/W/TYqs7KWadXopTG5Dx0nrJVFWiEAxQMH3xKU8fsg805fRNV
         O9mUCJ2o4vAYxzTxCWuaRdUDiD93AVNdlAnPKG/SEzv5GO18QwEGM17T2hR8/G9SVUpp
         H1/an6zN52wDlP+iAFUudlGDxdULP1maooWYPv/L0au+vpqZJO/8+VJVl/XPVVvzOMZT
         C1dQ==
X-Gm-Message-State: AOAM532EVpJhMEJNMXKgNeiEuN+lD15r3Kf80L4/AiQdmRGklpph2xqw
	J6gNBPYL4kU14A7sBvyYlg0FOW8cayg=
X-Google-Smtp-Source: ABdhPJyTGMDmf4scjKeBebTeFdkThiwTB1tUX8NLSvZRs7ZtFuZfBJOLUJ5uJ8kRB25bkqYfTjUf0g==
X-Received: by 2002:a17:907:2721:: with SMTP id d1mr2923277ejl.24.1630658031656;
        Fri, 03 Sep 2021 01:33:51 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 02/11] xen/arm: Add dev_to_pci helper
Date: Fri,  3 Sep 2021 11:33:38 +0300
Message-Id: <20210903083347.131786-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add a helper which is when given a struct device returns the
corresponding struct pci_dev which this device is a part of.

Because of the header cross-dependencies, e.g. we need both
struct pci_dev and struct arch_pci_dev at the same time, this cannot be
done with an inline. Macro can be implemented, but looks scary:

 #define dev_to_pci_dev(dev) container_of((container_of((dev), \
                        struct arch_pci_dev, dev), struct pci_dev, arch)

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/arch/arm/pci/pci.c    | 10 ++++++++++
 xen/include/asm-arm/pci.h |  7 +++++++
 2 files changed, 17 insertions(+)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index dc63bbc2a2c1..6573f179af55 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -26,6 +26,16 @@ int arch_pci_clean_pirqs(struct domain *d)
     return 0;
 }
 
+struct pci_dev *dev_to_pci(struct device *dev)
+{
+    struct arch_pci_dev *arch_dev;
+
+    ASSERT(dev->type == DEV_PCI);
+
+    arch_dev = container_of((dev), struct arch_pci_dev, dev);
+    return container_of(arch_dev, struct pci_dev, arch);
+}
+
 static int __init dt_pci_init(void)
 {
     struct dt_device_node *np;
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 2d4610a23a25..e1aa05190bda 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -27,6 +27,13 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/*
+ * Because of the header cross-dependencies, e.g. we need both
+ * struct pci_dev and struct arch_pci_dev at the same time, this cannot be
+ * done with an inline here. Macro can be implemented, but looks scary.
+ */
+struct pci_dev *dev_to_pci(struct device *dev);
+
 /* Arch-specific MSI data for vPCI. */
 struct vpci_arch_msi {
 };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177838.323577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eG-0007k4-23; Fri, 03 Sep 2021 08:34:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177838.323577; Fri, 03 Sep 2021 08:34:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eF-0007jr-UN; Fri, 03 Sep 2021 08:34:07 +0000
Received: by outflank-mailman (input) for mailman id 177838;
 Fri, 03 Sep 2021 08:34:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4eE-0006kQ-Cm
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:06 +0000
Received: from mail-ed1-x529.google.com (unknown [2a00:1450:4864:20::529])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d791cd72-1871-4ccf-817a-e51d3db5b6c3;
 Fri, 03 Sep 2021 08:33:53 +0000 (UTC)
Received: by mail-ed1-x529.google.com with SMTP id r7so6926076edd.6
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:53 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d791cd72-1871-4ccf-817a-e51d3db5b6c3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=LGWgNsGMc39sVj5QGdwU3RC9MH61fd0rOAc+HjnetL4=;
        b=jzF3nEIJXkuxIUCpc0qM5Oiu8hLlldWQ+Lz0hhNI2Ub1RMF3Qbrij1SFl/emQEQscX
         OctS5kVKgfSmIlkAEokYDlh7KTdaWXM0wDAYf9/2NdD6neoTsa3t2DKhbRd5ZdUKiPOx
         S2PsK64LPXN+uZeNXjPPxgQjMJjKSFc6YGWqsQVbMF6xtxnZDnoHezWhH2nT/vwym0Yv
         0CNidcd1Mx+4PJKuDXmrq2uXK50XYtdBR/DUdsISNvXiS0MjoUtZx7LKVdsMWZlTQkxu
         ruWxAhOgTOoVzm1FC+ANk+ZBh3e633gonZWlfdEMs3pAoxm7xgjBcF1AwkgxVuzftbjM
         lmoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=LGWgNsGMc39sVj5QGdwU3RC9MH61fd0rOAc+HjnetL4=;
        b=gFEF3vI+84Y3nriEaeUXr/5nOWigp71+vuqb1g44w9QEEMom02M9m8dU9IVlOyNYhN
         AWRc1uk4LSOfu3GqcZwGnvreAVYrqjp7b1N6Kacs5p3hkVCgNPPtb6JF6d2hk1k9d5iw
         lYZfj4mx7X4+yEMowDQzY2EeXQYF5XQpuZ66Yw5Gv/9GRf864HTgAOEUbtk+t1AyY//c
         bvvWAiye7e1aNAQigt4y0UVP/CL/QACcniAUFHLhHZhHtgxDXBdvSATthqFlqO8V8lfy
         Cn0XzcsvgWI1LYYyBGMmCZqTeSU4/+ZaMw94Az14P/p+jCCc8in/H00i4t0K+jI/TxjW
         0JDA==
X-Gm-Message-State: AOAM533T29VmwjjdpxeGBjsIw1GF/1YGfe5TF4KSgBNOw6Q+nUGLh3O7
	OLuWyYoVesOzLVpY14INbxAW2qVrYXQ=
X-Google-Smtp-Source: ABdhPJzoCD96lMsn4O6MjQ9c1RPiG7IXqQPjSOWYZAYm2R5Ns/eebk9CaPXrFjkNmkqhirqr1Rp1sg==
X-Received: by 2002:a05:6402:4d:: with SMTP id f13mr2792315edu.275.1630658032768;
        Fri, 03 Sep 2021 01:33:52 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 03/11] xen/arm: Introduce pci_find_host_bridge_node helper
Date: Fri,  3 Sep 2021 11:33:39 +0300
Message-Id: <20210903083347.131786-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Get host bridge node given a PCI device attached to it.

This helper will be re-used for adding PCI devices by the subsequent
patches.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
 xen/arch/arm/pci/pci-host-common.c | 17 +++++++++++++++++
 xen/include/asm-arm/pci.h          |  1 +
 2 files changed, 18 insertions(+)

diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 5e34252deb9d..d2fef5476b8e 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -301,6 +301,23 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
     return -EINVAL;
 }
 
+/*
+ * Get host bridge node given a device attached to it.
+ */
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
+{
+    struct pci_host_bridge *bridge;
+    struct pci_dev *pdev = dev_to_pci(dev);
+
+    bridge = pci_find_host_bridge(pdev->seg, pdev->bus);
+    if ( unlikely(!bridge) )
+    {
+        printk(XENLOG_ERR "Unable to find PCI bridge for "PRI_pci"\n",
+               pdev->seg, pdev->bus, pdev->sbdf.dev, pdev->sbdf.fn);
+        return NULL;
+    }
+    return bridge->dt_node;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index e1aa05190bda..7dc4c8dc9026 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -105,6 +105,7 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
 
 #else   /*!CONFIG_HAS_PCI*/
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177840.323588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eL-0008Ff-CR; Fri, 03 Sep 2021 08:34:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177840.323588; Fri, 03 Sep 2021 08:34:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eL-0008FR-82; Fri, 03 Sep 2021 08:34:13 +0000
Received: by outflank-mailman (input) for mailman id 177840;
 Fri, 03 Sep 2021 08:34:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4eJ-0006kQ-Cz
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:11 +0000
Received: from mail-ed1-x531.google.com (unknown [2a00:1450:4864:20::531])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id abac851f-78b0-46af-ba55-f6067286fe72;
 Fri, 03 Sep 2021 08:33:55 +0000 (UTC)
Received: by mail-ed1-x531.google.com with SMTP id g22so6893686edy.12
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:55 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abac851f-78b0-46af-ba55-f6067286fe72
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=gT/ZFZNUCDilPZAOY4tWpXuhNvzOpcakatBoOBy/O1k=;
        b=AeeHVvoDQgATpWSZNB/AO4OIyw4Mg9oLVflwbPYmCY/dNrU1qFSQB6Db3RqYWroQIm
         7dL4DAlFEW2GS7PnIsgS7gE9t4WMgwHb0M5oO5gEtI/deFPJO4ufJeIIE6jGpdTIuNXv
         HO3CaROk5N0KkNEEdY7x1NBl3FxGPdgHarce5VLXc0j0PuLaobvKmi3gSljvQcSX+aPc
         //aaqhRjf3R96XiFwTYuAJ4GXlwIF3VVYkQFUG2Fu1uqXBJPRpNuEPYYXKO2Xf6aeIxP
         zG8rAY1KwYJ2z5GY75QLpNcSi/kpSMO3mJAHsjUmpyFmB7ff8JMJd20fYfkj258kxD4V
         PMSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=gT/ZFZNUCDilPZAOY4tWpXuhNvzOpcakatBoOBy/O1k=;
        b=AKT9Qirdlz2Tnu9mJr/AmWdARIaR25QIApx9cutv45Bb8poD82/RxZ+GlwzIWWK/jN
         vwH8m6sZI1dOy9tWslgbkcPUfjFkdXf/028gzwqLHU3N8EvrlBeGusD9iRJC4YyaDzYr
         ibmj6MAF23RwNDdatmZ5QKlYkPnTikZS5jTe3lxS1NFhPLwiT0spVHuhLSbMydSznAA3
         y2Nupp/ycyftCrSsBpYimh3hRpN4QPUrfwGmETZo7ldY324LnAeYB6VQbPxDzaORkEfK
         O+PecLffVpdYBV8Uj+cgaYzP7EtrQZ7KnHLZtm6Y/y1cev/RIbr5pYbGhoDiAy5nKaHK
         cEsQ==
X-Gm-Message-State: AOAM531b95LbkD8D/U3k/TYzb2/vrmJ7a8/LrjtzlF8XrJDv464yiVDU
	fWc4zqUnUhEkVvt/LHw8+E62ttNAkn4=
X-Google-Smtp-Source: ABdhPJwY8ZltlgwVhwjnryt/5IUzAOzh/IBbZ1FvlM/J72PFOBmQxVLdVpjv5yPJL3sHkLXKtBoe/g==
X-Received: by 2002:a05:6402:b9c:: with SMTP id cf28mr2924531edb.356.1630658034045;
        Fri, 03 Sep 2021 01:33:54 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 04/11] xen/device-tree: Make dt_find_node_by_phandle global
Date: Fri,  3 Sep 2021 11:33:40 +0300
Message-Id: <20210903083347.131786-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Make dt_find_node_by_phandle globally visible, so it can be re-used by
other frameworks.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/common/device_tree.c      | 2 +-
 xen/include/xen/device_tree.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 03d25a81cea8..c2e33b99832f 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -986,7 +986,7 @@ int dt_for_each_range(const struct dt_device_node *dev,
  *
  * Returns a node pointer.
  */
-static struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
 {
     struct dt_device_node *np;
 
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index b02696be9416..07393da1df90 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -776,6 +776,8 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
                                const char *list_name,
                                const char *cells_name);
 
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle);
+
 #ifdef CONFIG_DEVICE_TREE_DEBUG
 #define dt_dprintk(fmt, args...)  \
     printk(XENLOG_DEBUG fmt, ## args)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177845.323599 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eP-0000Nh-Rt; Fri, 03 Sep 2021 08:34:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177845.323599; Fri, 03 Sep 2021 08:34:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eP-0000MS-Lw; Fri, 03 Sep 2021 08:34:17 +0000
Received: by outflank-mailman (input) for mailman id 177845;
 Fri, 03 Sep 2021 08:34:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4eO-0006kQ-DJ
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:16 +0000
Received: from mail-ej1-x62f.google.com (unknown [2a00:1450:4864:20::62f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 27e92245-e9a7-4d08-b7db-06c719470729;
 Fri, 03 Sep 2021 08:33:56 +0000 (UTC)
Received: by mail-ej1-x62f.google.com with SMTP id x11so10579940ejv.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:56 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27e92245-e9a7-4d08-b7db-06c719470729
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=zEaEEkjSVbLwgqZ2cO/SvSyYHTCsAuDtk1o9tcPz34M=;
        b=HOEWMOzgAHfhqGy9RD08ptIgiNJDtfBpZENO+7wmukitpCXKU/tg2fBV+y4B5GPbeZ
         nc617mYgWSgBkpaW4vuo16HZ9MEpN6RQrgWJ6q05CQtakHUM9tZgI3oY9+bCavK6RfvD
         /Z5kGtwMD+pBzSNmg8rVMzrHjsh3Ac/qSBRoejs561XlEe8rtWJROHH3P8toAYjpOg0u
         GC5NtpzMwvW+z9I37LMc3EWyZY+bAC5A7dznB6kxyhp8224v3H81NjGIfPkG4Ng//ZK9
         jmNQ+aYz/U+d5xKG/+qYldB4flk+vPZS3BIDdQ7TKOzfrfzmsoFflWkGBScLO5yZ2+dc
         LL0Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=zEaEEkjSVbLwgqZ2cO/SvSyYHTCsAuDtk1o9tcPz34M=;
        b=DdGy+TRLNw447nRDWa8eJGApCfFbQfReNvgW8pH60UiCN/SorZTs60lprwxoqtb4hs
         rmgP1lGzTtV2swotSG1xTKz9qnrfWW/fDrQ9sj8KSea4C84Jqfm5NdtYq2JGrGbCm8fL
         fG805Mgc3AUF1E2QTo88vuwvadMZXvX2IQKpZkNYtYobbkYGN90hEL34gJrjxGHgrgAz
         I8WIIqLWAI2Ii3/BaOZnzSouaSc0YRFQQ3zD9UJF2xMRcDMuooDYCvQG+UFgNI6JwzTW
         mmApM5Ap2CcM5SZ6RbiO1a/5WKpxT5VRU1rk5mULWN0vjkr3MUQJJt48ANSvqLRzrocA
         u8eQ==
X-Gm-Message-State: AOAM530l5Uu+nBzJSIgtyQkQA41cdnN7ZvTYnI2/5urjS4An/BSc8jhD
	QrgONB206CkZ7AO4HjjL4+SqUNh3dO8=
X-Google-Smtp-Source: ABdhPJwRW77fMud+VU+y2+UHwAx3A+siV93Yru3KtspqID6C5FmqhT1A5wFmO04G72SDTjkT5t52Uw==
X-Received: by 2002:a17:906:ff41:: with SMTP id zo1mr2759320ejb.525.1630658035099;
        Fri, 03 Sep 2021 01:33:55 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 05/11] xen/arm: Mark device as PCI while creating one
Date: Fri,  3 Sep 2021 11:33:41 +0300
Message-Id: <20210903083347.131786-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

While adding a PCI device mark it as such, so other frameworks
can distinguish it form DT devices.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/passthrough/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 56e261e9bd08..25304dbe9956 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1301,6 +1301,9 @@ static int iommu_add_device(struct pci_dev *pdev)
     if ( !is_iommu_enabled(pdev->domain) )
         return 0;
 
+#ifdef CONFIG_ARM
+    pci_to_dev(pdev)->type = DEV_PCI;
+#endif
     rc = hd->platform_ops->add_device(pdev->devfn, pci_to_dev(pdev));
     if ( rc || !pdev->phantom_stride )
         return rc;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177848.323610 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eV-0000uw-AO; Fri, 03 Sep 2021 08:34:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177848.323610; Fri, 03 Sep 2021 08:34:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eV-0000ue-67; Fri, 03 Sep 2021 08:34:23 +0000
Received: by outflank-mailman (input) for mailman id 177848;
 Fri, 03 Sep 2021 08:34:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4eT-0006kQ-DN
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:21 +0000
Received: from mail-ed1-x52e.google.com (unknown [2a00:1450:4864:20::52e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fdf50016-d48a-46db-8d04-c3242be972e1;
 Fri, 03 Sep 2021 08:33:57 +0000 (UTC)
Received: by mail-ed1-x52e.google.com with SMTP id n11so6903634edv.11
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:57 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdf50016-d48a-46db-8d04-c3242be972e1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=/iiGZOLolgjA2ZPldrvDxiQ5Z5+rXkMHlrxZBGQ1V6g=;
        b=Thp1SViWRIliiAx3L7i8dGQs8jHSK15JDP1ZAcNVhPgnWdIyDDHzN7LnytjXMbIJGT
         2aj1NBBC9Fu1xWxM0uHHq4g2BnsU8a/mnvbjF5ljWBOSqkyfveqsDXi0U8D5UwEzaL+c
         CEj7XN/MJOAGR2zNbfqHuzkpEq2avwghGsdsSVSbRe7N2V4EIwDJsbOWGfI6CSgqlxeb
         mcHWEL6UK8lF5q/wFWsS0s5Kq9z5yV8KKrpO5t0pufBbczyE4q+Gw2WE96KobfzMF2BS
         lSpWrUci/mFJPcibc3S1lKaxt7Y7SV2Q08CoJNTxpxXB2eDudckd2zo5dQ76y5qmNNa2
         YJEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=/iiGZOLolgjA2ZPldrvDxiQ5Z5+rXkMHlrxZBGQ1V6g=;
        b=aTKcGwa9+82WQZhK8PxotLTJGD5bJdp/YKJmn6ttLaDFvAPQ/W5PuhrCI5PrXDMD9c
         4MLYRzzcs6/2CtCYVWVtTeMNUq2pnVeTRVi01/tbXVAc6n+PITQR3H9fwvCDdq/pwqnU
         zRJ8q8FTFzxCVmOtKWN3k1GjYVOrOLW6g208FTYnkvjkv1yqaUyFASmtc92slpNrU3S6
         GU7DqzvchSdfAyNy3MyRIxHEj9gUPJ0DCpICkXlpNF75piWYIFbTR2G46ec+mZSIURMt
         zAyOd75zuNEiBDd+ph5429G34WkCIVVl9+/GnDIHUEKD6JAXkvIXbgIqY/W4hhpBGwTF
         kijA==
X-Gm-Message-State: AOAM533j4pWEUlFIHjIEKFwB35po1xJXQZA9VKe1K4XrNP96LOZI5fuN
	CGi8JjQjg0mRC9nYjKJr3bgCpCoxHag=
X-Google-Smtp-Source: ABdhPJy4fYHKLqSsXdIUKB/e5T2+22yYDBhCrp7iQB/x8B/pbhcNDDdtAnMUCrCtYVr2mtO64vRMIA==
X-Received: by 2002:aa7:c94c:: with SMTP id h12mr2776075edt.378.1630658036325;
        Fri, 03 Sep 2021 01:33:56 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com
Subject: [PATCH 06/11] xen/domain: Call pci_release_devices() when releasing domain resources
Date: Fri,  3 Sep 2021 11:33:42 +0300
Message-Id: <20210903083347.131786-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

This is the very same what we do for DT devices. What is more
that x86 already calls pci_release_devices().

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
 xen/arch/arm/domain.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index d99c653626e4..4e40c4098280 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -985,7 +985,8 @@ static int relinquish_memory(struct domain *d, struct page_list_head *list)
  * function which may return -ERESTART.
  */
 enum {
-    PROG_tee = 1,
+    PROG_pci = 1,
+    PROG_tee,
     PROG_xen,
     PROG_page,
     PROG_mapping,
@@ -1022,6 +1023,12 @@ int domain_relinquish_resources(struct domain *d)
 #ifdef CONFIG_IOREQ_SERVER
         ioreq_server_destroy_all(d);
 #endif
+#ifdef CONFIG_HAS_PCI
+    PROGRESS(pci):
+        ret = pci_release_devices(d);
+        if ( ret )
+            return ret;
+#endif
 
     PROGRESS(tee):
         ret = tee_relinquish_resources(d);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177852.323621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eZ-0001TH-Kf; Fri, 03 Sep 2021 08:34:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177852.323621; Fri, 03 Sep 2021 08:34:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4eZ-0001T7-Gj; Fri, 03 Sep 2021 08:34:27 +0000
Received: by outflank-mailman (input) for mailman id 177852;
 Fri, 03 Sep 2021 08:34:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4eY-0006kQ-Df
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:26 +0000
Received: from mail-ej1-x62e.google.com (unknown [2a00:1450:4864:20::62e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0898630a-f5e4-4ab6-8354-bf16210b9e36;
 Fri, 03 Sep 2021 08:33:58 +0000 (UTC)
Received: by mail-ej1-x62e.google.com with SMTP id me10so10462374ejb.11
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:58 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0898630a-f5e4-4ab6-8354-bf16210b9e36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=G9YXRhru3uNLxB1PDUiun9J9pMrCuA67iL9Y2S0hbpXL6f5CH2X0QxIT5S1a9tUVI0
         ngdOqG5buY4ynepqOc4Mqw/59P8H5sjq34iVC/u0iXKce95prSaaImHKqsL0PaYH7vcE
         D1V6AOs2vm4fw/w3cgP9IhSPze4c2q6V0JyGGMv/a3ElMGbTNF0S7auFCdB0/MVNyqHt
         MSsg6wqzDA1lelto4azizexS43392uIQej/nGbEwF7OGVlS74cihi3/3uWxljNDqJhHU
         wFIzTiQMGRpJzgDNE4+dQa/3yv3zMkXoX1BmCUXGm3S/B7/wgbcEBjWuZCJOL6udh3zY
         kX1g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=AP9HIzrCzQkFkZvqieTGE4tmU+0QANe3qh/xgxiywNDWgviFk0YWP8YwbWKO5e0JoX
         7VfKO2dAT/dMkoNTCtzWwwdLHfKWWK9qIBrtnVuc1WTKjhyrob7hDBMRVYIXhABpeDui
         C9bvM3qArCy6hfhVIwyLJUw9EAfE+v1q7JQuFVXI9MX1vQyqzM/lrU8wt+dbj6+sVh7g
         6aqrsJZro0owjPWvFQUmw8zcltCArkgxX/pJmMnAXF91fxajQYfnItw2bHbm3S0fzuKi
         B8cGRUudIGaoFiJC5fUQqCop5n7MuaTYiGmiGPbjEN0EtEHQWASH645RT5eWuhp1ZAVq
         eKhA==
X-Gm-Message-State: AOAM533KzecWeoojC2rWm7Gni1Du+O0rT40VNhGHm+p+LhXB8GYxfmoQ
	XtAkmYVHree4A9HaC2c44t8VqYf/cMs=
X-Google-Smtp-Source: ABdhPJzGKTmhAkRttkjaVwij4GDfdpVmfRmx6aBuBdUijcAaoOwlRu3VoW1BGbNMMqkWyy5d8+/pYQ==
X-Received: by 2002:a17:906:9b1:: with SMTP id q17mr2879682eje.546.1630658037434;
        Fri, 03 Sep 2021 01:33:57 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 07/11] libxl: Allow removing PCI devices for all types of domains
Date: Fri,  3 Sep 2021 11:33:43 +0300
Message-Id: <20210903083347.131786-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

The PCI device remove path may now be used by PVH on ARM, so the
assert is no longer valid.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>
---
 tools/libs/light/libxl_pci.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 1a1c2630803b..59f3686fc85e 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1947,8 +1947,6 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
             goto out_fail;
         }
     } else {
-        assert(type == LIBXL_DOMAIN_TYPE_PV);
-
         char *sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", pci->domain,
                                      pci->bus, pci->dev, pci->func);
         FILE *f = fopen(sysfs_path, "r");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177859.323632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4ee-00025Z-WA; Fri, 03 Sep 2021 08:34:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177859.323632; Fri, 03 Sep 2021 08:34:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4ee-00025O-S0; Fri, 03 Sep 2021 08:34:32 +0000
Received: by outflank-mailman (input) for mailman id 177859;
 Fri, 03 Sep 2021 08:34:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4ed-0006kQ-Dn
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:31 +0000
Received: from mail-ej1-x633.google.com (unknown [2a00:1450:4864:20::633])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fcfff0c5-87bc-448e-bd89-368f58f11371;
 Fri, 03 Sep 2021 08:33:59 +0000 (UTC)
Received: by mail-ej1-x633.google.com with SMTP id jg16so7370339ejc.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:33:59 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcfff0c5-87bc-448e-bd89-368f58f11371
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=Hx54vdgUoheVTz/8kdwndGE5xFCEvdbxwnx2M5LV2vg=;
        b=RdDtLXI6Rg8R+9WLRJYoNMq8UIn9C9UYiKM3atwuaiVC9bWq2G1D1gEb8701s40UoB
         q3Vkp/Q+AaXXiFaOTz5uyjQxEyJLgSw8QJB7PO+sZnWNbsk2QivJXAbq5fg+Jqbdfysp
         +QSpl4wnAMWZmYNE5OUiwggYvVyrZQF2WSlhbdngi69rmyZVt6LY3os3UdU+56o7z0Qz
         aXLK85ahPpB0tLKxrNsEPTzR/jIdjbVD9jnl5/AcTadm4B0eFB/U0iKObJlOH+vNr5El
         RDPi9bmWAmk9pV5ibKbmzIYJq18buRGok9BqADlAyv9Sraqe2G5UHaYHUFkzWsJQvwxk
         vQEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=Hx54vdgUoheVTz/8kdwndGE5xFCEvdbxwnx2M5LV2vg=;
        b=ZU935E9As03e8TxEeil25qhW+riYm6XE9eWITrr/ZJ9UU6It0Y8jMfSb27gbCyZYQH
         7rVTysvwC+z+PqfePhYlj9jhf8iLKqItd5YCy/pp6AwTvmRxvSNdGRr8bw+AQ09xauuC
         aOg4wIrhnSUkmfE//8coPTDi1QWx1XoPJCDGAzhgqSsscpeuoyHZM93lvn+7p+tgkGCi
         tpLceNS3iPxbtrDhEmKVVVO/HGnb84qfctwWpCLdxhnV3eSyXNmGgI264Bg354M5o7g3
         KKqrCyHRhrobvkbroh5XCSw5ojMMTlDPWV22YeF7t3kwE3lpR6pbSAwVAN2wMIvsW1KJ
         aenw==
X-Gm-Message-State: AOAM5307TcEyC4GgLUAmz5c86aiXM/0OKVw8x0dRJJCxgTU1kneIREPD
	tkiVI4TVLaedressSxtb1tk39/tniUM=
X-Google-Smtp-Source: ABdhPJxbTzY7zXsW8PjrsejxN3DPMeW22l0e+skzCHi5HgMf4ECl+/PSNQ7TpHFwniDROzLZHbVUSw==
X-Received: by 2002:a17:906:a01:: with SMTP id w1mr2947033ejf.117.1630658038658;
        Fri, 03 Sep 2021 01:33:58 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are supported
Date: Fri,  3 Sep 2021 11:33:44 +0300
Message-Id: <20210903083347.131786-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Arm's PCI passthrough implementation doesn't support legacy interrupts,
but MSI/MSI-X. This can be the case for other platforms too.
For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
it to the CFLAGS and compile the relevant code in the toolstack only if
applicable.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>
---
 tools/libs/light/Makefile    |  4 ++++
 tools/libs/light/libxl_pci.c | 13 +++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 7d8c51d49242..bd3f6be2a183 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -46,6 +46,10 @@ CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
 	-Wno-declaration-after-statement -Wformat-nonliteral
 CFLAGS += -I.
 
+ifeq ($(CONFIG_X86),y)
+CFLAGS += -DCONFIG_PCI_SUPP_LEGACY_IRQ
+endif
+
 SRCS-$(CONFIG_X86) += libxl_cpuid.c
 SRCS-$(CONFIG_X86) += libxl_x86.c
 SRCS-$(CONFIG_X86) += libxl_psr.c
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 59f3686fc85e..cd4fea46c3f7 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1434,6 +1434,7 @@ static void pci_add_dm_done(libxl__egc *egc,
         }
     }
     fclose(f);
+#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
     sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                 pci->bus, pci->dev, pci->func);
     f = fopen(sysfs_path, "r");
@@ -1460,6 +1461,7 @@ static void pci_add_dm_done(libxl__egc *egc,
         }
     }
     fclose(f);
+#endif
 
     /* Don't restrict writes to the PCI config space from this VM */
     if (pci->permissive) {
@@ -1471,7 +1473,9 @@ static void pci_add_dm_done(libxl__egc *egc,
         }
     }
 
+#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
 out_no_irq:
+#endif
     if (!isstubdom) {
         if (pci->rdm_policy == LIBXL_RDM_RESERVE_POLICY_STRICT) {
             flag &= ~XEN_DOMCTL_DEV_RDM_RELAXED;
@@ -1951,7 +1955,9 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
                                      pci->bus, pci->dev, pci->func);
         FILE *f = fopen(sysfs_path, "r");
         unsigned int start = 0, end = 0, flags = 0, size = 0;
+#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
         int irq = 0;
+#endif
         int i;
 
         if (f == NULL) {
@@ -1983,6 +1989,7 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
         }
         fclose(f);
 skip1:
+#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
         sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                pci->bus, pci->dev, pci->func);
         f = fopen(sysfs_path, "r");
@@ -2001,8 +2008,14 @@ skip1:
             }
         }
         fclose(f);
+#else
+        /* Silence error: label at end of compound statement */
+        ;
+#endif
     }
+#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
 skip_irq:
+#endif
     rc = 0;
 out_fail:
     pci_remove_detached(egc, prs, rc); /* must be last */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:34:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:34:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177865.323642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4ek-0002hQ-B6; Fri, 03 Sep 2021 08:34:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177865.323642; Fri, 03 Sep 2021 08:34:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4ek-0002hH-7X; Fri, 03 Sep 2021 08:34:38 +0000
Received: by outflank-mailman (input) for mailman id 177865;
 Fri, 03 Sep 2021 08:34:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4ei-0006kQ-E5
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:36 +0000
Received: from mail-ej1-x630.google.com (unknown [2a00:1450:4864:20::630])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3bb67d0b-f39f-4b34-9864-f3691fc81bea;
 Fri, 03 Sep 2021 08:34:00 +0000 (UTC)
Received: by mail-ej1-x630.google.com with SMTP id mf2so10485515ejb.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:34:00 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:33:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bb67d0b-f39f-4b34-9864-f3691fc81bea
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=/GHtMwVtwcasyb8hly1Zcy56jKyqF6v6oGi9z4wXBlU=;
        b=mQ3lDyAu7ulppaS1JJy+SWOFwu+RvtpP1zI3cgCUuBtIuinOwyvLCIG9MxewYdI7wb
         dV/QNQp9gaBilxa1L3saoMIiNAKa4GjxcPpTanA20hGKCzz5QpJAiNHS/6eiO+aEby+f
         l7w6stPig8p506HVXOzw2BgAPYcOwkZ+OD0tZuyYui/AU5k+tg6rbLKS49n75aRhxpxL
         FFVTa9P9ikuYd6dbk2C9zfDC/kBVTCpJCMhTRepxDPdAUW12LYD4QcV3R1ooZ2wQyzED
         /H+subTI7EzgC9RApqZqRkYcedmNM4JZlibRP0m4S+GaRm+tZmLehtpvYwPU05nhGVEk
         4LLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=/GHtMwVtwcasyb8hly1Zcy56jKyqF6v6oGi9z4wXBlU=;
        b=O4lfH5P6mjh536gva7arPTU9Frb1mcd1r2huTLL9NwRMe4CMVErcHhWA1YyUz1Gn8F
         tWfkw4jaYubWiEFKwx6qq7PfIWMfh3MD4b+gmG5TbpkKLeHzxcAHMD8Elrjo7PNIUkrf
         H5cMpYSQSQfTGWG2V2unKdAlXz2FgOFAj6TtXZQol9SqCt3ULYAbvE/PYzawRwWMsMvj
         nrO9aLXEqip9GkQlLsUeqsJ7kojKilNOv3gnZ+7ULWhIniUF9ewwtlG0kNAqCJUjT7Yf
         c7rucReZ9HOS56umrG5LTZqKfPlceVAGC5FhP7KHWZidf1iVy9oGKU4S3aYtvP7iNLvJ
         02+w==
X-Gm-Message-State: AOAM532JkJfQzvOgGj6QtsslnH1vxGZEAnffzrJ39RlJGKs5sgjk4mFU
	wGuCSbTMJ19X/nPsKqJSSQtMf6X5OsU=
X-Google-Smtp-Source: ABdhPJzkdXq4/oqy7sAR6jAHLZTgcQFNU12ssgRybTwH6NU5CgLqCQqqVvZtmbSCS+UTenNgcBH03A==
X-Received: by 2002:a17:906:2a8e:: with SMTP id l14mr2909109eje.321.1630658039681;
        Fri, 03 Sep 2021 01:33:59 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for hardware domain
Date: Fri,  3 Sep 2021 11:33:45 +0300
Message-Id: <20210903083347.131786-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

In order vPCI to work it needs all access to PCI configuration space
(ECAM) to be synchronized among all entities, e.g. hardware domain and
guests. For that implement PCI host bridge specific callbacks to
properly setup those ranges depending on particular host bridge
implementation.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/arch/arm/pci/ecam.c            | 11 +++++++++++
 xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
 xen/arch/arm/vpci.c                | 13 +++++++++++++
 xen/include/asm-arm/pci.h          |  8 ++++++++
 4 files changed, 48 insertions(+)

diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
index 91c691b41fdf..92ecb2e0762b 100644
--- a/xen/arch/arm/pci/ecam.c
+++ b/xen/arch/arm/pci/ecam.c
@@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
     return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
 }
 
+static int pci_ecam_register_mmio_handler(struct domain *d,
+                                          struct pci_host_bridge *bridge,
+                                          const struct mmio_handler_ops *ops)
+{
+    struct pci_config_window *cfg = bridge->sysdata;
+
+    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
+    return 0;
+}
+
 /* ECAM ops */
 const struct pci_ecam_ops pci_generic_ecam_ops = {
     .bus_shift  = 20,
@@ -49,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
         .map_bus                = pci_ecam_map_bus,
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
+        .register_mmio_handler  = pci_ecam_register_mmio_handler,
     }
 };
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index d2fef5476b8e..a89112bfbb7c 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -318,6 +318,22 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
     }
     return bridge->dt_node;
 }
+
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*clb)(struct domain *d,
+                                        struct pci_host_bridge *bridge))
+{
+    struct pci_host_bridge *bridge;
+    int err;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        err = clb(d, bridge);
+        if ( err )
+            return err;
+    }
+    return 0;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index da8b1ca13c07..258134292458 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -74,11 +74,24 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
     .write = vpci_mmio_write,
 };
 
+static int vpci_setup_mmio_handler(struct domain *d,
+                                   struct pci_host_bridge *bridge)
+{
+    if ( bridge->ops->register_mmio_handler )
+        return bridge->ops->register_mmio_handler(d, bridge,
+                                                  &vpci_mmio_handler);
+    return 0;
+}
+
 int domain_vpci_init(struct domain *d)
 {
     if ( !has_vpci(d) )
         return 0;
 
+    if ( is_hardware_domain(d) )
+        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+
+    /* Guest domains use what is programmed in their device tree. */
     register_mmio_handler(d, &vpci_mmio_handler,
                           GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
 
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 7dc4c8dc9026..2c7c7649e00f 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -17,6 +17,8 @@
 #ifndef __ARM_PCI_H__
 #define __ARM_PCI_H__
 
+#include <asm/mmio.h>
+
 #ifdef CONFIG_HAS_PCI
 
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
@@ -77,6 +79,9 @@ struct pci_ops {
                 uint32_t reg, uint32_t len, uint32_t *value);
     int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
                  uint32_t reg, uint32_t len, uint32_t value);
+    int (*register_mmio_handler)(struct domain *d,
+                                 struct pci_host_bridge *bridge,
+                                 const struct mmio_handler_ops *ops);
 };
 
 /*
@@ -107,6 +112,9 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
 struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
 
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*clb)(struct domain *d,
+                                        struct pci_host_bridge *bridge));
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:41:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:41:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177892.323653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4lA-0005HT-4D; Fri, 03 Sep 2021 08:41:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177892.323653; Fri, 03 Sep 2021 08:41:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4lA-0005HM-1J; Fri, 03 Sep 2021 08:41:16 +0000
Received: by outflank-mailman (input) for mailman id 177892;
 Fri, 03 Sep 2021 08:41:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4es-0006kQ-EK
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:46 +0000
Received: from mail-ej1-x631.google.com (unknown [2a00:1450:4864:20::631])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d926dc1-80ff-4ebb-a723-1fd24234f40a;
 Fri, 03 Sep 2021 08:34:02 +0000 (UTC)
Received: by mail-ej1-x631.google.com with SMTP id lc21so10503499ejc.7
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:34:02 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.34.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:34:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d926dc1-80ff-4ebb-a723-1fd24234f40a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=Xy7BJSB6xaM30OyXxkPFQIY2JXEW71Y3R+/OfIHEOBE=;
        b=pnUw4Mlz8wE01ctr0MxVIgMnhwLnYDb2XDSr3Mz0utU10pdqiieFvIuxCCByexKtyf
         B06Oyzl54Te+zID2si0YkZIjwWSmkUavw3mR3hfVyzP1prT4JWTQ2NTIK4x6xtbcRpqc
         KxtBQtIPNwv9OGv7Awvjre3ntebXcDrWFI2SG529YegtLTDoGtS7s1bc/D2jQxUVo3N7
         BdhYDhJHI86NLX1Mc5Y1L3MlkT/LPFTYKEkS5Mb0pAz3/VB/Hn65DqjDFmLtvlvBfV5r
         z6WPuzwT0gYxxiczN3UQkXeeK6u22iH64pxd2K8aSWXJ8LKEUvYiJdgFvuoa8TTDahw0
         xKfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=Xy7BJSB6xaM30OyXxkPFQIY2JXEW71Y3R+/OfIHEOBE=;
        b=Tb8crlNetyA6rFLsr9BPepDhb+mT9n9jfsUBW8A0hc7C/6FqN8EkpaSUFDLxZx5Bm0
         EFyPIyQ/FBKiqjBTGdcEYbw27nAms0HXXuw4ith3GQIZT8uX8g1CEoOKpjrOVvimfx5H
         VFMdHmp7LNZHQUWt6F++n7mtvjQ/iZ8tJxutQaf0Kz6YLSq1dX3aT0TA8EMRT1vHN9rG
         0SszP9zXs6zXvrFAGNX6T38VwiduYKf+EWML+ycm9J+wssNO5DKunxRWtf/kP9F0INQM
         XVNDV/9u0aMfmHpyMCkGGxpYEBsyY5xUaq+gFNREXGgYvoPUo+WA5dUXo5WPfPCrtixp
         oKOw==
X-Gm-Message-State: AOAM5308j4Iig0X44JLaWCBB9ZOGEhuPoZvkJFtBcA9wOI/JQwhpGQ33
	8s99qzGcnqNjJqAC3WcSHIWGHWQ1amo=
X-Google-Smtp-Source: ABdhPJyJcdfQj9c9044SVO6J1UJQYGNr5fMQWFYrfhn9ETnMa9xx30aD81J57K8Jf8DgGximyczXvQ==
X-Received: by 2002:a17:907:2174:: with SMTP id rl20mr2987996ejb.199.1630658041886;
        Fri, 03 Sep 2021 01:34:01 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Date: Fri,  3 Sep 2021 11:33:47 +0300
Message-Id: <20210903083347.131786-12-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

vPCI may map and unmap PCI device memory (BARs) being passed through which
may take a lot of time. For this those operations may be deferred to be
performed later, so that they can be safely preempted.
Run the corresponding vPCI code while switching a vCPU.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/arch/arm/traps.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 219ab3c3fbde..1571fb8afd03 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -34,6 +34,7 @@
 #include <xen/symbols.h>
 #include <xen/version.h>
 #include <xen/virtual_region.h>
+#include <xen/vpci.h>
 
 #include <public/sched.h>
 #include <public/xen.h>
@@ -2304,6 +2305,11 @@ static bool check_for_vcpu_work(void)
     }
 #endif
 
+    local_irq_enable();
+    if ( has_vpci(v->domain) && vpci_process_pending(v) )
+        raise_softirq(SCHEDULE_SOFTIRQ);
+    local_irq_disable();
+
     if ( likely(!v->arch.need_flush_to_ram) )
         return false;
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:41:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:41:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177903.323664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4la-0005z8-Il; Fri, 03 Sep 2021 08:41:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177903.323664; Fri, 03 Sep 2021 08:41:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4la-0005z1-Fe; Fri, 03 Sep 2021 08:41:42 +0000
Received: by outflank-mailman (input) for mailman id 177903;
 Fri, 03 Sep 2021 08:41:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM4en-0006kQ-E9
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:34:41 +0000
Received: from mail-ed1-x531.google.com (unknown [2a00:1450:4864:20::531])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3cbf0903-612d-4e40-aaea-917bad9342ef;
 Fri, 03 Sep 2021 08:34:01 +0000 (UTC)
Received: by mail-ed1-x531.google.com with SMTP id v5so1613984edc.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 01:34:01 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id i7sm2311932ejx.73.2021.09.03.01.33.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 01:34:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cbf0903-612d-4e40-aaea-917bad9342ef
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=RKj8rEW+uLzxBa1xS9FCgER2pVUMxy97v84H9BueJo8=;
        b=Cc7VjnWxryLAC/tqYxNfb0Vqq0sB/g3r4rmDDY0lS2y4ND5fCxj+3AtTH0jMf3UD7I
         lArRxEur0IpiP8IRmM5bMXAd2tDTqstoOHd1vrOJQB+pAYbF+caMEs1aDP0y6JUJoUAC
         xEaHLm1ca7cVFaDLT1hlicqg/QeFfEAaQY7S1SpqIHJhYw6d6GtxExLoUnpq0E0zuH+m
         4NCp+Ya/PMN9z0TzuqE1T/2/rlEnc8XJFLpeu56yU2FZu5xlOk5bPOVjQrgrStQv7HvU
         TwhRciy77N17AemdAXrbaCnjfawSj07zHK5ot06VaJwpM/HtZHOSN1AfqKhUFuuc/1HO
         ob5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=RKj8rEW+uLzxBa1xS9FCgER2pVUMxy97v84H9BueJo8=;
        b=oBLpbapiCNQnIjv9Mo/sSyhL3oVakBxi1XwL1Wi7m4raUtBsNmjtGeWCoq7dTXXeTC
         KhocmQ1iynUJWKyuksEXh1BgFK/jRTRPbFN9qbQGg3QpeHcER3uNfb0r1fulY1GJFamB
         62zgQyF5dXtDweUZgg+/rJfYhR1agFpdUJtTWRgPVyZ8E3dkOV4MxOj72N2h1gVKp5b9
         vw15weokEfFftha6EM7IAETNzYpDgpzrZhG6Te4CYpQzeWncKy3llziRgqTWuOUkSF5w
         +q8Ng9ouUhGTopIjWQh/VhFVLNkp8wFO1HvkvKKBqNBPjPXhP9sPPQc9MbI4DGh/0pTk
         Kzfg==
X-Gm-Message-State: AOAM533uKOsAif0Gr5pPehfRtIiQNwbwfBJWC0recNqto4ZX7hQfZS3k
	Xjt3APquPw91Bq5Rgk6LA7LYf07qXZs=
X-Google-Smtp-Source: ABdhPJwyyY+X4Ff7jtwDvNQrMZN78ZPm6ovgltm1mRQGeZ/mmH2TfThaCYtmDMf1K+WgzeKR4mkDAg==
X-Received: by 2002:aa7:c9d6:: with SMTP id i22mr605692edt.307.1630658040780;
        Fri, 03 Sep 2021 01:34:00 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's p2m
Date: Fri,  3 Sep 2021 11:33:46 +0300
Message-Id: <20210903083347.131786-11-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903083347.131786-1-andr2000@gmail.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Host bridge controller's ECAM space is mapped into Domain-0's p2m,
thus it is not possible to trap the same for vPCI via MMIO handlers.
For this to work we need to not map those while constructing the domain.

Note, that during Domain-0 creation there is no pci_dev yet allocated for
host bridges, thus we cannot match PCI host and its associated
bridge by SBDF. Use dt_device_node field for checks instead.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/arch/arm/domain_build.c        |  3 +++
 xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
 xen/include/asm-arm/pci.h          | 12 ++++++++++++
 4 files changed, 54 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index da427f399711..76f5b513280c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
         }
     }
 
+    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI) )
+        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev, addr, len);
+
     if ( need_mapping )
     {
         res = map_regions_p2mt(d,
diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
index 92ecb2e0762b..d32efb7fcbd0 100644
--- a/xen/arch/arm/pci/ecam.c
+++ b/xen/arch/arm/pci/ecam.c
@@ -52,6 +52,22 @@ static int pci_ecam_register_mmio_handler(struct domain *d,
     return 0;
 }
 
+static int pci_ecam_need_p2m_mapping(struct domain *d,
+                                     struct pci_host_bridge *bridge,
+                                     uint64_t addr, uint64_t len)
+{
+    struct pci_config_window *cfg = bridge->sysdata;
+
+    if ( !is_hardware_domain(d) )
+        return true;
+
+    /*
+     * We do not want ECAM address space to be mapped in domain's p2m,
+     * so we can trap access to it.
+     */
+    return cfg->phys_addr != addr;
+}
+
 /* ECAM ops */
 const struct pci_ecam_ops pci_generic_ecam_ops = {
     .bus_shift  = 20,
@@ -60,6 +76,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
         .register_mmio_handler  = pci_ecam_register_mmio_handler,
+        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
     }
 };
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index a89112bfbb7c..c04be636452d 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -334,6 +334,28 @@ int pci_host_iterate_bridges(struct domain *d,
     }
     return 0;
 }
+
+bool pci_host_bridge_need_p2m_mapping(struct domain *d,
+                                      const struct dt_device_node *node,
+                                      uint64_t addr, uint64_t len)
+{
+    struct pci_host_bridge *bridge;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        if ( bridge->dt_node != node )
+            continue;
+
+        if ( !bridge->ops->need_p2m_mapping )
+            return true;
+
+        return bridge->ops->need_p2m_mapping(d, bridge, addr, len);
+    }
+    printk(XENLOG_ERR "Unable to find PCI bridge for %s segment %d, addr %lx\n",
+           node->full_name, bridge->segment, addr);
+    return true;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 2c7c7649e00f..9c28a4bdc4b7 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -82,6 +82,8 @@ struct pci_ops {
     int (*register_mmio_handler)(struct domain *d,
                                  struct pci_host_bridge *bridge,
                                  const struct mmio_handler_ops *ops);
+    int (*need_p2m_mapping)(struct domain *d, struct pci_host_bridge *bridge,
+                            uint64_t addr, uint64_t len);
 };
 
 /*
@@ -115,9 +117,19 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
 int pci_host_iterate_bridges(struct domain *d,
                              int (*clb)(struct domain *d,
                                         struct pci_host_bridge *bridge));
+bool pci_host_bridge_need_p2m_mapping(struct domain *d,
+                                      const struct dt_device_node *node,
+                                      uint64_t addr, uint64_t len);
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
 
+static inline bool
+pci_host_bridge_need_p2m_mapping(struct domain *d,
+                                 const struct dt_device_node *node,
+                                 uint64_t addr, uint64_t len)
+{
+    return true;
+}
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:49:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:49:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177911.323676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tL-0006nU-EZ; Fri, 03 Sep 2021 08:49:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177911.323676; Fri, 03 Sep 2021 08:49:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tL-0006nN-BR; Fri, 03 Sep 2021 08:49:43 +0000
Received: by outflank-mailman (input) for mailman id 177911;
 Fri, 03 Sep 2021 08:49:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FOYs=NZ=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mM4tK-0006nA-39
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:49:42 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5dadf295-4f13-42fd-b413-aa9abfb9d364;
 Fri, 03 Sep 2021 08:49:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 0FD5122663;
 Fri,  3 Sep 2021 08:49:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 6EBBF1374A;
 Fri,  3 Sep 2021 08:49:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id YQ4TGaPhMWFjdAAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 03 Sep 2021 08:49:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5dadf295-4f13-42fd-b413-aa9abfb9d364
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630658980; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Etfp5KRM0ihxlz1Fzg+3n+yV+gWrXCILk5YBRRiJ4ec=;
	b=g/xCu4uUEeLNVhfh4hdAtjtyZDkPPqU9rW19W5JYaL63cE3RPFwRsK8D5oSo5RKnUyhZef
	oRJKvg5r8mEK1qPZy47OOCWYT8N3Ja1dGzT5F+tieQLSPZ2csfGh8mApFEuhAiWEv2KpMv
	5oLIfL7iry/5ho/b6+Y9oBZRKXdYVLY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	linux-pm@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	x86@kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Len Brown <len.brown@intel.com>,
	Pavel Machek <pavel@ucw.cz>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	stable@vger.kernel.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH 0/2] xen: fix illegal rtc device usage of pv guests
Date: Fri,  3 Sep 2021 10:49:35 +0200
Message-Id: <20210903084937.19392-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

A recent change in mc146818_get_time() resulted in WARN splats when
booting a Xen PV guest.

The main reason is that there is a code path resulting in accessing a
RTC device which is not present, which has been made obvious by a
call of WARN() in this case.

This small series is fixing this issue by:

- avoiding the RTC device access from drivers/base/power/trace.c in
  cast there is no legacy RTC device available
- resetting the availability flag of a legacy RTC device for Xen PV
  guests

Juergen Gross (2):
  PM: base: power: don't try to use non-existing RTC for storing data
  xen: reset legacy rtc flag for PV domU

 arch/x86/xen/enlighten_pv.c |  7 +++++++
 drivers/base/power/trace.c  | 10 ++++++++++
 2 files changed, 17 insertions(+)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:49:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:49:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177912.323687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tM-00073S-MJ; Fri, 03 Sep 2021 08:49:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177912.323687; Fri, 03 Sep 2021 08:49:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tM-00073L-Ir; Fri, 03 Sep 2021 08:49:44 +0000
Received: by outflank-mailman (input) for mailman id 177912;
 Fri, 03 Sep 2021 08:49:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FOYs=NZ=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mM4tK-0006nG-Mc
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:49:42 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e0641ded-0c93-11ec-aefe-12813bfff9fa;
 Fri, 03 Sep 2021 08:49:41 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C9E082001B;
 Fri,  3 Sep 2021 08:49:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 71D981374A;
 Fri,  3 Sep 2021 08:49:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id eKY/GqThMWFjdAAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 03 Sep 2021 08:49:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0641ded-0c93-11ec-aefe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630658980; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cDxNZDOT6mQUMNCs/5bHO5A+JEd2XgxBAJXhmBdSbbw=;
	b=VbpC8r2iK1pOQ9eNi1TouFnvGAEG3Q2l//bLETrWhVa1Mvebnw9WTw+3fU42jh0vUotgsp
	luWP3rWesmgxuuRQkhvF72sZKVDy7LzQ6MpzomsIjNTl9HKI+OUwVrQXGEPTWdiRmsnWai
	o6EzteyS0M7u3UJpASPIGqLHXQUgGIo=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	stable@vger.kernel.org
Subject: [PATCH 2/2] xen: reset legacy rtc flag for PV domU
Date: Fri,  3 Sep 2021 10:49:37 +0200
Message-Id: <20210903084937.19392-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210903084937.19392-1-jgross@suse.com>
References: <20210903084937.19392-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

A Xen PV guest doesn't have a legacy RTC device, so reset the legacy
RTC flag. Otherwise the following WARN splat will occur at boot:

[    1.333404] WARNING: CPU: 1 PID: 1 at /home/gross/linux/head/drivers/rtc/rtc-mc146818-lib.c:25 mc146818_get_time+0x1be/0x210
[    1.333404] Modules linked in:
[    1.333404] CPU: 1 PID: 1 Comm: swapper/0 Tainted: G        W         5.14.0-rc7-default+ #282
[    1.333404] RIP: e030:mc146818_get_time+0x1be/0x210
[    1.333404] Code: c0 64 01 c5 83 fd 45 89 6b 14 7f 06 83 c5 64 89 6b 14 41 83 ec 01 b8 02 00 00 00 44 89 63 10 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b 48 c7 c7 30 0e ef 82 4c 89 e6 e8 71 2a 24 00 48 c7 c0 ff ff
[    1.333404] RSP: e02b:ffffc90040093df8 EFLAGS: 00010002
[    1.333404] RAX: 00000000000000ff RBX: ffffc90040093e34 RCX: 0000000000000000
[    1.333404] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 000000000000000d
[    1.333404] RBP: ffffffff82ef0e30 R08: ffff888005013e60 R09: 0000000000000000
[    1.333404] R10: ffffffff82373e9b R11: 0000000000033080 R12: 0000000000000200
[    1.333404] R13: 0000000000000000 R14: 0000000000000002 R15: ffffffff82cdc6d4
[    1.333404] FS:  0000000000000000(0000) GS:ffff88807d440000(0000) knlGS:0000000000000000
[    1.333404] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.333404] CR2: 0000000000000000 CR3: 000000000260a000 CR4: 0000000000050660
[    1.333404] Call Trace:
[    1.333404]  ? wakeup_sources_sysfs_init+0x30/0x30
[    1.333404]  ? rdinit_setup+0x2b/0x2b
[    1.333404]  early_resume_init+0x23/0xa4
[    1.333404]  ? cn_proc_init+0x36/0x36
[    1.333404]  do_one_initcall+0x3e/0x200
[    1.333404]  kernel_init_freeable+0x232/0x28e
[    1.333404]  ? rest_init+0xd0/0xd0
[    1.333404]  kernel_init+0x16/0x120
[    1.333404]  ret_from_fork+0x1f/0x30

Cc: <stable@vger.kernel.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/enlighten_pv.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 753f63734c13..349f780a1567 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1214,6 +1214,11 @@ static void __init xen_dom0_set_legacy_features(void)
 	x86_platform.legacy.rtc = 1;
 }
 
+static void __init xen_domu_set_legacy_features(void)
+{
+	x86_platform.legacy.rtc = 0;
+}
+
 /* First C function to be called on Xen boot */
 asmlinkage __visible void __init xen_start_kernel(void)
 {
@@ -1359,6 +1364,8 @@ asmlinkage __visible void __init xen_start_kernel(void)
 		add_preferred_console("xenboot", 0, NULL);
 		if (pci_xen)
 			x86_init.pci.arch_init = pci_xen_init;
+		x86_platform.set_legacy_features =
+				xen_domu_set_legacy_features;
 	} else {
 		const struct dom0_vga_console_info *info =
 			(void *)((char *)xen_start_info +
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:49:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:49:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177913.323698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tP-0007MH-VH; Fri, 03 Sep 2021 08:49:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177913.323698; Fri, 03 Sep 2021 08:49:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4tP-0007M5-Ru; Fri, 03 Sep 2021 08:49:47 +0000
Received: by outflank-mailman (input) for mailman id 177913;
 Fri, 03 Sep 2021 08:49:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FOYs=NZ=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mM4tP-0006nG-Gh
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:49:47 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e0641dec-0c93-11ec-aefe-12813bfff9fa;
 Fri, 03 Sep 2021 08:49:41 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 67AF722666;
 Fri,  3 Sep 2021 08:49:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 1527913754;
 Fri,  3 Sep 2021 08:49:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id SD+dA6ThMWFjdAAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 03 Sep 2021 08:49:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0641dec-0c93-11ec-aefe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630658980; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2A79ijv44MFRRI4yP9IKOdGZLkyzD6D8f46U5bOTMG8=;
	b=YaUyhefWzsHHkhY9meNyZ+nn5OZODEbPW68xXVIYL/Z3GsIY0wJ6ADl0f1uKxP/ao0IOzf
	7tkHLaKoHp2ra3Mq30rV0xmgUe0Zvl3BHYlchwLyMW2aH5QnXmYyUenSuhEv6x4NLizQQM
	xQiOoW4Iv3P3Ud7OZuNeMHRCHmstlzk=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	linux-pm@vger.kernel.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Len Brown <len.brown@intel.com>,
	Pavel Machek <pavel@ucw.cz>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	stable@vger.kernel.org
Subject: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC for storing data
Date: Fri,  3 Sep 2021 10:49:36 +0200
Message-Id: <20210903084937.19392-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210903084937.19392-1-jgross@suse.com>
References: <20210903084937.19392-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In there is no legacy RTC device, don't try to use it for storing trace
data across suspend/resume.

Cc: <stable@vger.kernel.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/base/power/trace.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c
index a97f33d0c59f..b7c80849455c 100644
--- a/drivers/base/power/trace.c
+++ b/drivers/base/power/trace.c
@@ -13,6 +13,7 @@
 #include <linux/export.h>
 #include <linux/rtc.h>
 #include <linux/suspend.h>
+#include <linux/init.h>
 
 #include <linux/mc146818rtc.h>
 
@@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, unsigned int user)
 	const char *file = *(const char **)(tracedata + 2);
 	unsigned int user_hash_value, file_hash_value;
 
+	if (!x86_platform.legacy.rtc)
+		return 0;
+
 	user_hash_value = user % USERHASH;
 	file_hash_value = hash_string(lineno, file, FILEHASH);
 	set_magic_time(user_hash_value, file_hash_value, dev_hash_value);
@@ -267,6 +271,9 @@ static struct notifier_block pm_trace_nb = {
 
 static int __init early_resume_init(void)
 {
+	if (!x86_platform.legacy.rtc)
+		return 0;
+
 	hash_value_early_read = read_magic_time();
 	register_pm_notifier(&pm_trace_nb);
 	return 0;
@@ -277,6 +284,9 @@ static int __init late_resume_init(void)
 	unsigned int val = hash_value_early_read;
 	unsigned int user, file, dev;
 
+	if (!x86_platform.legacy.rtc)
+		return 0;
+
 	user = val % USERHASH;
 	val = val / USERHASH;
 	file = val % FILEHASH;
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 08:56:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 08:56:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177928.323709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4zz-0001D7-MY; Fri, 03 Sep 2021 08:56:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177928.323709; Fri, 03 Sep 2021 08:56:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM4zz-0001D0-JQ; Fri, 03 Sep 2021 08:56:35 +0000
Received: by outflank-mailman (input) for mailman id 177928;
 Fri, 03 Sep 2021 08:56:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KfKZ=NZ=linuxfoundation.org=gregkh@srs-us1.protection.inumbo.net>)
 id 1mM4zy-0001Cu-43
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 08:56:34 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d610c1c8-0c94-11ec-aefe-12813bfff9fa;
 Fri, 03 Sep 2021 08:56:33 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F1B3161051;
 Fri,  3 Sep 2021 08:56:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d610c1c8-0c94-11ec-aefe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1630659392;
	bh=SsjjbztlEwoQ2/FB20kOEWlK/4U11FZMfaPIS5UzwPA=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=HIACXt/YT6pWCqM1ZXgIK14DP/a+8b2wZJAtH533JTB7bHiNh5hQjPgWr4l844+Hp
	 fsqjfJdlFyq4d5jAQMavIFBifH1me5UlrWLbbRy3fBUFxN2leYC9fYTFblIYrARotV
	 Weq2nXrRWULvKRmXB2e4DIi5A9/LS26+/JVxSY+U=
Date: Fri, 3 Sep 2021 10:56:29 +0200
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, linux-pm@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>,
	"Rafael J. Wysocki" <rafael@kernel.org>, stable@vger.kernel.org
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
Message-ID: <YTHjPbklWVDVaBfK@kroah.com>
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210903084937.19392-2-jgross@suse.com>

On Fri, Sep 03, 2021 at 10:49:36AM +0200, Juergen Gross wrote:
> In there is no legacy RTC device, don't try to use it for storing trace
> data across suspend/resume.
> 
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  drivers/base/power/trace.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c
> index a97f33d0c59f..b7c80849455c 100644
> --- a/drivers/base/power/trace.c
> +++ b/drivers/base/power/trace.c
> @@ -13,6 +13,7 @@
>  #include <linux/export.h>
>  #include <linux/rtc.h>
>  #include <linux/suspend.h>
> +#include <linux/init.h>
>  
>  #include <linux/mc146818rtc.h>
>  
> @@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, unsigned int user)
>  	const char *file = *(const char **)(tracedata + 2);
>  	unsigned int user_hash_value, file_hash_value;
>  
> +	if (!x86_platform.legacy.rtc)
> +		return 0;

Why does the driver core code here care about a platform/arch-specific
thing at all?  Did you just break all other arches?

thanks,

greg k-h


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 09:02:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 09:02:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177935.323719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM55G-0002rv-B1; Fri, 03 Sep 2021 09:02:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177935.323719; Fri, 03 Sep 2021 09:02:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM55G-0002ro-85; Fri, 03 Sep 2021 09:02:02 +0000
Received: by outflank-mailman (input) for mailman id 177935;
 Fri, 03 Sep 2021 09:02:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FOYs=NZ=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mM55F-0002rh-1X
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 09:02:01 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e6886402-fcf2-4f8d-a42a-beb599e3b106;
 Fri, 03 Sep 2021 09:02:00 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5EFA6225F7;
 Fri,  3 Sep 2021 09:01:59 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 0FB0C1374A;
 Fri,  3 Sep 2021 09:01:59 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 2qW5AYfkMWFBdwAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 03 Sep 2021 09:01:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6886402-fcf2-4f8d-a42a-beb599e3b106
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630659719; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=y2yOgR7xOsgNe3DptYo2QPyqQttv32W4xVBl15aqNN4=;
	b=pNNE6BX8asHX6WKrgFgKgJflKWF5Ifavakx2rqMPBZZDZpfs2PgmzaM61GHNjicbucQaA4
	gYG1qSy5fMSSHEv6bwSJbR+RVxktEdMYGR5MnqeZ6tCaBOKDNFCE3fMsHRcAyWj+Mm8ben
	LzQuKW/A7oCAYGQQFM9AdQ5A3LB1+N4=
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: xen-devel@lists.xenproject.org, linux-pm@vger.kernel.org,
 linux-kernel@vger.kernel.org, "Rafael J. Wysocki" <rjw@rjwysocki.net>,
 Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>,
 "Rafael J. Wysocki" <rafael@kernel.org>, stable@vger.kernel.org
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com> <YTHjPbklWVDVaBfK@kroah.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
Date: Fri, 3 Sep 2021 11:01:58 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YTHjPbklWVDVaBfK@kroah.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="8QrHnFxLhYwW5JVUDZedIeiKcRWLD8jDi"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--8QrHnFxLhYwW5JVUDZedIeiKcRWLD8jDi
Content-Type: multipart/mixed; boundary="DDwF9mzVWNkpBGBzATm4O5noHEiM7Qhxi";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: xen-devel@lists.xenproject.org, linux-pm@vger.kernel.org,
 linux-kernel@vger.kernel.org, "Rafael J. Wysocki" <rjw@rjwysocki.net>,
 Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>,
 "Rafael J. Wysocki" <rafael@kernel.org>, stable@vger.kernel.org
Message-ID: <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com> <YTHjPbklWVDVaBfK@kroah.com>
In-Reply-To: <YTHjPbklWVDVaBfK@kroah.com>

--DDwF9mzVWNkpBGBzATm4O5noHEiM7Qhxi
Content-Type: multipart/mixed;
 boundary="------------7C43D073BE29B721237268B8"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7C43D073BE29B721237268B8
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 03.09.21 10:56, Greg Kroah-Hartman wrote:
> On Fri, Sep 03, 2021 at 10:49:36AM +0200, Juergen Gross wrote:
>> In there is no legacy RTC device, don't try to use it for storing trac=
e
>> data across suspend/resume.
>>
>> Cc: <stable@vger.kernel.org>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   drivers/base/power/trace.c | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c
>> index a97f33d0c59f..b7c80849455c 100644
>> --- a/drivers/base/power/trace.c
>> +++ b/drivers/base/power/trace.c
>> @@ -13,6 +13,7 @@
>>   #include <linux/export.h>
>>   #include <linux/rtc.h>
>>   #include <linux/suspend.h>
>> +#include <linux/init.h>
>>  =20
>>   #include <linux/mc146818rtc.h>
>>  =20
>> @@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, unsi=
gned int user)
>>   	const char *file =3D *(const char **)(tracedata + 2);
>>   	unsigned int user_hash_value, file_hash_value;
>>  =20
>> +	if (!x86_platform.legacy.rtc)
>> +		return 0;
>=20
> Why does the driver core code here care about a platform/arch-specific
> thing at all?  Did you just break all other arches?

This file is only compiled for x86. It depends on CONFIG_PM_TRACE_RTC,
which has a "depends on X86" attribute.


Juergen

--------------7C43D073BE29B721237268B8
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7C43D073BE29B721237268B8--

--DDwF9mzVWNkpBGBzATm4O5noHEiM7Qhxi--

--8QrHnFxLhYwW5JVUDZedIeiKcRWLD8jDi
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEx5IYFAwAAAAAACgkQsN6d1ii/Ey8Z
Ywf+Pq3IKlaBuk34XWwT2ZcLn5H2s4d3HKoC1sokLsNtHiDJAveAMkhzsaIiSP8xfYzC0FKpGsB2
Y9iAFojypHyCF/BVEY1A48vxMFfYVcgqrohKB9MUVOhKDsaB5mzAZ7YeqQR5khtuc5J2B34Rij5c
VW0E0QjxjgcPL4O0ZFw9+p+f/BpIBECqUZV1hVoSiEtunFIt9xz+hJFJ9CF5P/ic0lsaNy2aTUos
3sHTrWdrBcHLE8pPN4wYXWz5SBj4HW64jG9C/nO6lw43EJ4JO4tw9KtgQ7VI4oQphdQmiKqk8Vzw
m3FWeab8bAWHHq9/MpyGfh+SBea4edJOy/pVI7CKng==
=vU83
-----END PGP SIGNATURE-----

--8QrHnFxLhYwW5JVUDZedIeiKcRWLD8jDi--


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 09:05:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 09:05:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177942.323731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM586-0003ZN-Ty; Fri, 03 Sep 2021 09:04:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177942.323731; Fri, 03 Sep 2021 09:04:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM586-0003ZG-Qg; Fri, 03 Sep 2021 09:04:58 +0000
Received: by outflank-mailman (input) for mailman id 177942;
 Fri, 03 Sep 2021 09:04:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mM584-0003ZA-Sn
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 09:04:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mM581-0000er-Sf; Fri, 03 Sep 2021 09:04:53 +0000
Received: from 54-240-197-239.amazon.com ([54.240.197.239]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mM581-0002uw-MR; Fri, 03 Sep 2021 09:04:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=GBrzngP5vJbdy9YlVKO13Ln629Mn55ISUbjkYvuttWo=; b=kcv2bQrhr9WWltUjml2Nfb25yz
	HWgnrzQwnnlM6x8Q3WujjW7esIL6kVj7aov6pPkAwz7IqXH7Po2kTCn/XVW56erKwwMqr8rTedZ8Y
	X3jqaAbzis2k7ekcwSeYhQ24qltM2vKDTYAbpnwbjJmc6VCVubc9ks5tqaJunNI8UF3g=;
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
Date: Fri, 3 Sep 2021 10:04:51 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210903083347.131786-12-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> vPCI may map and unmap PCI device memory (BARs) being passed through which
> may take a lot of time. For this those operations may be deferred to be
> performed later, so that they can be safely preempted.
> Run the corresponding vPCI code while switching a vCPU.

IIUC, you are talking about the function map_range() in 
xen/drivers/vpci/header. The function has the following todo for Arm:

         /*
          * ARM TODOs:
          * - On ARM whether the memory is prefetchable or not should be 
passed
          *   to map_mmio_regions in order to decide which memory attributes
          *   should be used.
          *
          * - {un}map_mmio_regions doesn't support preemption.
          */

This doesn't seem to be addressed in the two series for PCI passthrough 
sent so far. Do you have any plan to handle it?

> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>   xen/arch/arm/traps.c | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 219ab3c3fbde..1571fb8afd03 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -34,6 +34,7 @@
>   #include <xen/symbols.h>
>   #include <xen/version.h>
>   #include <xen/virtual_region.h>
> +#include <xen/vpci.h>
>   
>   #include <public/sched.h>
>   #include <public/xen.h>
> @@ -2304,6 +2305,11 @@ static bool check_for_vcpu_work(void)
>       }
>   #endif
>   
> +    local_irq_enable();
> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )

Looking at the code of vpci_process_pending(), it looks like there are 
some rework to do for guest. Do you plan to handle it as part of the 
vPCI series?

> +        raise_softirq(SCHEDULE_SOFTIRQ);
> +    local_irq_disable();
> +

 From my understanding of vcpi_process_pending(). The function will 
return true if there are more work to schedule. However, if 
check_for_vcpu_for_work() return false, then we will return to the guest 
before any work for vCPI has finished. This is because 
check_for_vcpu_work() will not be called again.

In this case, I think you want to return as soon as you know we need to 
reschedule.

However, looking at the rest of the code, we already have a check for 
vpci in the common IOREQ code. So we would end up to call twice 
vpci_process_pending(). Maybe we should move the call from the IOREQ to 
arch-code.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 09:09:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 09:09:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177948.323742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5C4-0004EB-E8; Fri, 03 Sep 2021 09:09:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177948.323742; Fri, 03 Sep 2021 09:09:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5C4-0004E4-B9; Fri, 03 Sep 2021 09:09:04 +0000
Received: by outflank-mailman (input) for mailman id 177948;
 Fri, 03 Sep 2021 09:09:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KfKZ=NZ=linuxfoundation.org=gregkh@srs-us1.protection.inumbo.net>)
 id 1mM5C2-0004Dy-7D
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 09:09:02 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9401bf88-0c96-11ec-aefe-12813bfff9fa;
 Fri, 03 Sep 2021 09:09:01 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 22EDB60FA0;
 Fri,  3 Sep 2021 09:08:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9401bf88-0c96-11ec-aefe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1630660140;
	bh=DcjjJ2WAyz1ejKxh5caZx2UjPxskQXotN1KC+CVTphM=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=AXVe52q+2qENaWaDMiNqezmL4BsUWssD8bJgO6b/o7VEIMHLNCgzHRygULaFTLwRR
	 oGz0JvrwOuCyWgZZPRJsgjb88aE0JCr9iBHnqDnHyutHvuocaJmuo0+o0dzEvLEmh3
	 rgQwV9xE7s8/DBL80QGApRESQeCjgwrWCZXPWS2g=
Date: Fri, 3 Sep 2021 11:08:58 +0200
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, linux-pm@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>,
	"Rafael J. Wysocki" <rafael@kernel.org>, stable@vger.kernel.org
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
Message-ID: <YTHmKvYz5j8ZT9Jt@kroah.com>
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com>
 <YTHjPbklWVDVaBfK@kroah.com>
 <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>

On Fri, Sep 03, 2021 at 11:01:58AM +0200, Juergen Gross wrote:
> On 03.09.21 10:56, Greg Kroah-Hartman wrote:
> > On Fri, Sep 03, 2021 at 10:49:36AM +0200, Juergen Gross wrote:
> > > In there is no legacy RTC device, don't try to use it for storing trace
> > > data across suspend/resume.
> > > 
> > > Cc: <stable@vger.kernel.org>
> > > Signed-off-by: Juergen Gross <jgross@suse.com>
> > > ---
> > >   drivers/base/power/trace.c | 10 ++++++++++
> > >   1 file changed, 10 insertions(+)
> > > 
> > > diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c
> > > index a97f33d0c59f..b7c80849455c 100644
> > > --- a/drivers/base/power/trace.c
> > > +++ b/drivers/base/power/trace.c
> > > @@ -13,6 +13,7 @@
> > >   #include <linux/export.h>
> > >   #include <linux/rtc.h>
> > >   #include <linux/suspend.h>
> > > +#include <linux/init.h>
> > >   #include <linux/mc146818rtc.h>
> > > @@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, unsigned int user)
> > >   	const char *file = *(const char **)(tracedata + 2);
> > >   	unsigned int user_hash_value, file_hash_value;
> > > +	if (!x86_platform.legacy.rtc)
> > > +		return 0;
> > 
> > Why does the driver core code here care about a platform/arch-specific
> > thing at all?  Did you just break all other arches?
> 
> This file is only compiled for x86. It depends on CONFIG_PM_TRACE_RTC,
> which has a "depends on X86" attribute.

Odd, and not obvious at all :(

Ok, I'll let Rafael review this...


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 09:17:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 09:17:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177954.323752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5Jr-0005rm-7S; Fri, 03 Sep 2021 09:17:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177954.323752; Fri, 03 Sep 2021 09:17:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5Jr-0005rf-4U; Fri, 03 Sep 2021 09:17:07 +0000
Received: by outflank-mailman (input) for mailman id 177954;
 Fri, 03 Sep 2021 09:17:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1yCI=NZ=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mM5Jq-0005rZ-1h
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 09:17:06 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 728e836a-9979-4ee7-9e3e-b594a3f5a662;
 Fri, 03 Sep 2021 09:17:03 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 348A11FB;
 Fri,  3 Sep 2021 02:17:03 -0700 (PDT)
Received: from [10.57.13.101] (unknown [10.57.13.101])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E59343F694;
 Fri,  3 Sep 2021 02:17:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 728e836a-9979-4ee7-9e3e-b594a3f5a662
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Ash Wilding <ash.j.wilding@gmail.com>, Ian Jackson <iwj@xenproject.org>
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <4a5ac822-3078-de0e-8f84-6881d47ad3eb@xen.org>
 <dd9859ec-a3a9-6a74-bdae-fde092f26ac0@arm.com>
 <e8bdb065-9b46-e0d8-1fcf-7bba38e02f9e@xen.org>
 <8974fc82-94c1-ef85-153c-0923f5faf331@arm.com>
 <f41e36bb-efb5-fc11-f87e-6906336bca45@xen.org>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <9502c0fa-3e46-3cd3-7de8-c7adccca1397@arm.com>
Date: Fri, 3 Sep 2021 11:16:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <f41e36bb-efb5-fc11-f87e-6906336bca45@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit



On 02.09.2021 14:50, Julien Grall wrote:
> On 01/09/2021 10:38, Michal Orzel wrote:
>> Hi Julien,
> 
> Hi Michal,
> 
>> On 06.08.2021 13:12, Julien Grall wrote:
>>>
>>>
>>> On 29/07/2021 12:47, Michal Orzel wrote:
>>>> Hi Julien,
>>>
>>> Hi Michal,
>>>
>>>> On 29.07.2021 13:20, Julien Grall wrote:
>>>>> Hi Michal,
>>>>>
>>>>> On 29/07/2021 11:42, Michal Orzel wrote:
>>>>>> According to ARMv8A architecture, AArch64 registers
>>>>>> are 64bit wide even though in many cases the upper
>>>>>> 32bit is reserved. Therefore there is no need for
>>>>>> function vreg_emulate_sysreg32 on arm64. This means
>>>>>> that we can have just one function vreg_emulate_sysreg
>>>>>> using new function pointer:
>>>>>> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>>>>>>                                  register_t *r, bool read);
>>>>>>
>>>>>> Modify vreg_emulate_cp32 to use the new function pointer
>>>>>> as well.
>>>>>>
>>>>>> This change allows to properly use 64bit registers in AArch64
>>>>>> state and in case of AArch32 the upper 32 bits of AArch64
>>>>>> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
>>>>>
>>>>> What you wrote only says that the bits are ignored. It doesn't say whether the bits will be 0.
>>>>>
>>>>> They are probably, but as I wrote yesterday, I couldn't confirm it.
>>>>>
>>>> Should I then remove this part of the commit or write below?:
>>>> "We can assume that those bits will be 0 but the architecture
>>>> reference manual does not clarify this."
>>>
>>> There was some back and forth on security@xen.org about this. I will summarizing the discussion here as we considered this was a just a bug.
>>>
>>> I wasn't looking at the correct section in the Arm Arm. There is a paragraph clearly describing the expected behavior in a different section (thanks Ash for the pointer!). Per section D1.19.2 in DDI 0487F.c:
>>>
>>> "
>>> If the general-purpose register was accessible from AArch32 state
>>>
>>> The upper 32 bits either become zero, or hold the value that the same architectural register held before any AArch32 execution. The choice between these two options is IMPLEMENTATION DEFINED, and might vary dynamically within an implementation. Correspondingly, software must
>>> regard the value as being a CONSTRAINED UNPREDICTABLE choice between these two values.
>>>
>>> This behavior applies regardless of whether any execution occurred at the Exception level that was using AArch32. That is, this behavior applies even if AArch32 state was entered by an exception
>>> return from AArch64 state, and another exception was immediately taken to AArch64 state without any instruction execution in AArch32 state.
>>> "
>>>
>>> So we can't assume the top 32-bits are zeroed unless the hypervisor ensured they were. Today, we don't have that guarantee in Xen.
>>>
>>> This needs to be fixed. The two approachs we discussed are:
>>>     1) Update set_user_reg() to zero the top 32-bit. We have a couple of places using directly the fields xN. So we would need to switch them to use set_user_reg()
>>>     2) Only saving/restoring the bottom 32-bit when entering/leaving the hypervisor.
>>>
>>> At the moment, my preference goes towards the latter because we don't risk to introduce new place where set_user_reg() is not used.
>>>
>>> I have quickly hack the entry path. This would look like:
>>>
>>> diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
>>> index fc3811ad0ad5..65e24c88b059 100644
>>> --- a/xen/arch/arm/arm64/entry.S
>>> +++ b/xen/arch/arm/arm64/entry.S
>>> @@ -111,6 +111,11 @@
>>>    */
>>>           .macro  entry, hyp, compat, save_x0_x1=1
>>>           sub     sp, sp, #(UREGS_SPSR_el1 - UREGS_LR) /* CPSR, PC, SP, LR */
>>> +        .if \compat == 1 /* AArch32 mode */
>>> +        /* Clobber the top 32-bit of the registers */
>>> +        mov    w0, w0
>>> +        mov    w1, w1
>>> +        .endif
>>>           push    x28, x29
>>>           push    x26, x27
>>>           push    x24, x25
>>>
>>> I haven't looked whether this can be optimized or the exit path would be easier to modify.
>>>
>>> Anyway, this is not a new bug so I would be fine to get this patch merged first. Although, I think this wants to be fixed for xen 4.16 (CCing Ian to track it).
>>>
>>> I will try to find sometimes in the next couple of weeks to fix it and have another review of this patch.
>>>
>>
>> As the 4.16 release is getting closer I wanted to ask whether you need help with creating a pre-work patch so that this patch can be merged.
> 
> Sorry I haven't looked at the bug yet. I don't think the bug I reported necessarily needs to go first. This is not a new bug and AFAICT your patch is not going to make it worse than the current state.
> 
> Your patch needs to have an updated commit message summarizing what we discussed and a second review. No need to resend a patch for the updated commit message, this can be discussed here and updated on commit (I assuming there is no other comments).
> 
If the following message is ok for you, please update the commit msg after acking this change:
"
According to ARMv8A architecture, AArch64 registers
are 64bit wide even though in many cases the upper
32bit is reserved. Therefore there is no need for
function vreg_emulate_sysreg32 on arm64. This means
that we can have just one function vreg_emulate_sysreg
using new function pointer:
typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
                              register_t *r, bool read);

Modify vreg_emulate_cp32 to use the new function pointer
as well.

This change allows to properly use 64bit registers in AArch64
state. In case of AArch32 the documantation (D1.20.2, DDI 0487A.j)
states that "the upper 32 bits either become zero, or hold the value
that the same architectural register held before any AArch32 execution."
As the choice between them is IMPLEMENTATION DEFINED we cannot assume they
are zeroed. Xen should ensure that but currently it does not. This is not
a new bug and must be fixed as agreed during a discussion over this patch.
"
>> I believe this patch wants to be merged for 4.16 as the other sysreg related patches are merged already, so I'm offering a help.
> 
> If you have spare time that would be great. There are a few bits I wanted to look other than the entry place:
>   - The DOMCTL that update the context: I believe we may need to zero to top bits
>   - hypercall continuation (see hypercall_create_continuation)
>   - multicall
> 
I'll take a look at it.
> For 4.16, the entry (or exit) is probably going to be a good step. We can look at the rest for 4.17.
> 
> Cheers,
> 
Cheers,


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 09:39:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 09:39:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177962.323764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5ey-0000Dn-35; Fri, 03 Sep 2021 09:38:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177962.323764; Fri, 03 Sep 2021 09:38:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM5ex-0000Dg-Uc; Fri, 03 Sep 2021 09:38:55 +0000
Received: by outflank-mailman (input) for mailman id 177962;
 Fri, 03 Sep 2021 09:38:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iCYc=NZ=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mM5ew-0000DV-JS
 for xen-devel@lists.xen.org; Fri, 03 Sep 2021 09:38:54 +0000
Received: from mail-wr1-x429.google.com (unknown [2a00:1450:4864:20::429])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b0ad83b8-e1a1-4018-b8eb-222b928869fe;
 Fri, 03 Sep 2021 09:38:52 +0000 (UTC)
Received: by mail-wr1-x429.google.com with SMTP id q14so7320267wrp.3
 for <xen-devel@lists.xen.org>; Fri, 03 Sep 2021 02:38:52 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id g143sm3863966wme.16.2021.09.03.02.38.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 02:38:50 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id D1CFC1FF96;
 Fri,  3 Sep 2021 10:38:49 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b0ad83b8-e1a1-4018-b8eb-222b928869fe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=eglj5ZpXfutr/eGVG8JtkghmoUPw+XxKGQoeUjdoPlo=;
        b=oYPdVmnHy4majCVmnTP+1CNbmbzW79nSLoTQ13OnPEJvgKek/h2Gi7AnN9hoFNGDJ+
         9oRio4Ppmw1h+mbh1Gaf7tAb7az0AGMNUsV10jA5po4tKmjRAm17aMCgxJjfccEn2hp3
         3Tsue8DDRoaE+1qJyZMoPdN9RpbX7ZKGM29rOcRYIGOR40wndTr9e6Mj0sippnBAKLA1
         hXmu02EkfOzX4jKVD7Aha6jbZIMl/QYGJtjJmORg59pL5uN+MV/fqh55driLyzThWsdi
         yfVj7UNlDTtg+5btsSE4yRhA6yb0Q4jY73uiT/bo+KloDFU/IePiG4YYh62qRBQUG/Ou
         njiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=eglj5ZpXfutr/eGVG8JtkghmoUPw+XxKGQoeUjdoPlo=;
        b=E1EMoo2bCvxxNlYLMh6KH+mziJhtd07q9Jt00KFQj5dyL2uz6eU2sYuifG0qQGkFes
         QkQssVnoEtl63hk+RvxJeNi2H/UwPzfbQwidLKUGjqzEu6ZJh2AHi2cvEoguQ67qLpVy
         Gtgjq93vg6ph3HGlVIkL0Q3yONBjBqtoafW6y4C/kz2F6C6oP37NPyRtp5wU/9bZilkL
         ySgC7Z+7LRIAOZL4ujnDODyWXRAnBvZQ2KjlrIifXsMY9OpShU/LEesRfdS4xLKBDQh0
         DmDRcWzNmT9QqYdYD1M6ILo5eIm8vNgQ7Iyt0KeE6BA3yGhKQeArmO6DfRKAcuNFMrRW
         9F7g==
X-Gm-Message-State: AOAM532Mn0+OK+1BaOhRB1597jN4K7IgYaQZK7xSicnIScJ/O752duPt
	tVJ4DcFO5RbClQ+btCXafJtLRQ==
X-Google-Smtp-Source: ABdhPJx6NdsYfccjIXK3bpXfNCSBcf1+do70OI6kBYhG6UJu6Ret7BmqXZyyO/01ILiNCDYEWnYxTw==
X-Received: by 2002:adf:eb02:: with SMTP id s2mr3012989wrn.294.1630661931736;
        Fri, 03 Sep 2021 02:38:51 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Arnd
 Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar
 <viresh.kumar@linaro.org>, Stefano Stabellini
 <stefano.stabellini@xilinx.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl
 van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com, Srivatsa
 Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker
 <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
 Wei.Chen@arm.com, olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
 Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org,
 jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org, Elena Afanasova
 <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Date: Fri, 03 Sep 2021 10:28:06 +0100
In-reply-to: <20210903080609.GD47953@laputa>
Message-ID: <87czpqq9qu.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


AKASHI Takahiro <takahiro.akashi@linaro.org> writes:

> Alex,
>
> On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:
>>=20
>> Stefan Hajnoczi <stefanha@redhat.com> writes:
>>=20
>> > [[PGP Signed Part:Undecided]]
>> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
>> >> > Could we consider the kernel internally converting IOREQ messages f=
rom
>> >> > the Xen hypervisor to eventfd events? Would this scale with other k=
ernel
>> >> > hypercall interfaces?
>> >> >=20
>> >> > So any thoughts on what directions are worth experimenting with?
>> >>=20=20
>> >> One option we should consider is for each backend to connect to Xen v=
ia
>> >> the IOREQ interface. We could generalize the IOREQ interface and make=
 it
>> >> hypervisor agnostic. The interface is really trivial and easy to add.
>> >> The only Xen-specific part is the notification mechanism, which is an
>> >> event channel. If we replaced the event channel with something else t=
he
>> >> interface would be generic. See:
>> >> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/=
hvm/ioreq.h#L52
>> >
>> > There have been experiments with something kind of similar in KVM
>> > recently (see struct ioregionfd_cmd):
>> > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1=
613828727.git.eafanasova@gmail.com/
>>=20
>> Reading the cover letter was very useful in showing how this provides a
>> separate channel for signalling IO events to userspace instead of using
>> the normal type-2 vmexit type event. I wonder how deeply tied the
>> userspace facing side of this is to KVM? Could it provide a common FD
>> type interface to IOREQ?
>
> Why do you stick to a "FD" type interface?

I mean most user space interfaces on POSIX start with a file descriptor
and the usual read/write semantics or a series of ioctls.

>> As I understand IOREQ this is currently a direct communication between
>> userspace and the hypervisor using the existing Xen message bus. My
>
> With IOREQ server, IO event occurrences are notified to BE via Xen's event
> channel, while the actual contexts of IO events (see struct ioreq in iore=
q.h)
> are put in a queue on a single shared memory page which is to be assigned
> beforehand with xenforeignmemory_map_resource hypervisor call.

If we abstracted the IOREQ via the kernel interface you would probably
just want to put the ioreq structure on a queue rather than expose the
shared page to userspace.=20

>> worry would be that by adding knowledge of what the underlying
>> hypervisor is we'd end up with excess complexity in the kernel. For one
>> thing we certainly wouldn't want an API version dependency on the kernel
>> to understand which version of the Xen hypervisor it was running on.
>
> That's exactly what virtio-proxy in my proposal[1] does; All the hypervis=
or-
> specific details of IO event handlings are contained in virtio-proxy
> and virtio BE will communicate with virtio-proxy through a virtqueue
> (yes, virtio-proxy is seen as yet another virtio device on BE) and will
> get IO event-related *RPC* callbacks, either MMIO read or write, from
> virtio-proxy.
>
> See page 8 (protocol flow) and 10 (interfaces) in [1].

There are two areas of concern with the proxy approach at the moment.
The first is how the bootstrap of the virtio-proxy channel happens and
the second is how many context switches are involved in a transaction.
Of course with all things there is a trade off. Things involving the
very tightest latency would probably opt for a bare metal backend which
I think would imply hypervisor knowledge in the backend binary.

>
> If kvm's ioregionfd can fit into this protocol, virtio-proxy for kvm
> will hopefully be implemented using ioregionfd.
>
> -Takahiro Akashi
>
> [1] https://op-lists.linaro.org/pipermail/stratos-dev/2021-August/000548.=
html

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:08:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:08:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177969.323775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67i-0004D4-Ed; Fri, 03 Sep 2021 10:08:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177969.323775; Fri, 03 Sep 2021 10:08:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67i-0004Cx-BO; Fri, 03 Sep 2021 10:08:38 +0000
Received: by outflank-mailman (input) for mailman id 177969;
 Fri, 03 Sep 2021 10:08:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM67g-0004Cr-Lf
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:36 +0000
Received: from mail-ej1-x632.google.com (unknown [2a00:1450:4864:20::632])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c434e2e5-586f-42b8-82f7-85921841c2e5;
 Fri, 03 Sep 2021 10:08:35 +0000 (UTC)
Received: by mail-ej1-x632.google.com with SMTP id t19so10998324ejr.8
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:35 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c434e2e5-586f-42b8-82f7-85921841c2e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=7H6Dr6JQKVJd8ZBAOuYHiT1TMijPI9lm94ml0YNkOmU=;
        b=IBD98SZW9KZZal7dGGuQAEjAduRLEntYcTgOwxwpzaCsJOYStXf09lvZqoqv3d00TR
         kCiLhDiAKTv4BvLyvjQAIiHzEK/9xxhTGDZbLRLt0sn98m8dMbHTU+GQn5lvI5/sLVUO
         crCsf6SVRQVa6+1N1/iKw773pDWJO4WSYgvK/Kcfu2dHXevb5j6eMvVAnZ+JAcdDIC3r
         ZVwlrmkcCshjGiXAg8qINGcNFhxs788QTSSDGBKd9SMDNDL4vGxhHpL6MJMae7Alv6EB
         agjJfky004dG/SytKYYnXdlBRFNi2AO3cE4bkyfEdYx52MuCc4X2olG0TMBXo/dBNjWU
         LnXw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=7H6Dr6JQKVJd8ZBAOuYHiT1TMijPI9lm94ml0YNkOmU=;
        b=JeFqIWH9GmpRccQUxSACtUw6gkbPUqcU9JHX2DQVEx5/jiIEjs3a1yw7vVJQ5aaD0J
         aTPS8kWMtTz24xXfKmsHAT/O3haTliNVTZcWlcu+Zlr4mcjVBGn8ztWQ+zvB+kNFZF3d
         9JABiNWBZj785GgHPXXwdHvn0pEU3pk4updKNGDWLEt1c77nnOx393/e3oEtGlhpDFcP
         aN+6Rg4eZc04Unci4bRyg+oNfz5WGuUUqBA2zijYvN7AxVLZH5jaEXukWSf+QF8mhh7K
         Ly2n3N3NuFXU0nnv1kFXRwyax2suM6In4+P6uFeoGDW3fAisZ9cQj2/bgIcATTo1i91i
         ffTQ==
X-Gm-Message-State: AOAM533M3ZqklRruXCykiLG4TaLH1E8juc8FXw1WlD0LGTWb+YeCOxLP
	OLCSmFNDNlzTGWidf4/0403xeNv8D80=
X-Google-Smtp-Source: ABdhPJw6mI/4aI/4ceFyYh6bdRsh+uhWebon0bgMPdhmq94GIcgA6dYEPZWF11K+3koGJZRrPc0/OQ==
X-Received: by 2002:a17:906:2f15:: with SMTP id v21mr3286141eji.444.1630663714345;
        Fri, 03 Sep 2021 03:08:34 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 0/9] PCI devices passthrough on Arm, part 3
Date: Fri,  3 Sep 2021 13:08:22 +0300
Message-Id: <20210903100831.177748-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This patch series is focusing on vPCI and adds support for non-identity
PCI BAR mappings which is required while passing through a PCI device to
a guest. The highlights are:

- Add relevant vpci register handlers when assigning PCI device to a domain
  and remove those when de-assigning. This allows having different
  handlers for different domains, e.g. hwdom and other guests.

- Emulate guest BAR register values based on physical BAR values.
  This allows creating a guest view of the registers and emulates
  size and properties probe as it is done during PCI device enumeration by
  the guest.

- Instead of handling a single range set, that contains all the memory
  regions of all the BARs and ROM, have them per BAR.

- Take into account guest's BAR view and program its p2m accordingly:
  gfn is guest's view of the BAR and mfn is the physical BAR value as set
  up by the host bridge in the hardware domain.
  This way hardware doamin sees physical BAR values and guest sees
  emulated ones.

The series was also tested on x86 PVH Dom0 and doesn't break it.

Thank you,
Oleksandr

Oleksandr Andrushchenko (8):
  vpci: Make vpci registers removal a dedicated function
  vpci: Add hooks for PCI device assign/de-assign
  vpci/header: Move register assignments from init_bars
  vpci/header: Add and remove register handlers dynamically
  vpci/header: Implement guest BAR register handlers
  vpci/header: Handle p2m range sets per BAR
  vpci/header: program p2m with guest BAR view
  vpci/header: Reset the command register when adding devices

Rahul Singh (1):
  vpci/header: Use pdev's domain instead of vCPU

 xen/drivers/passthrough/pci.c |   9 +
 xen/drivers/vpci/header.c     | 431 +++++++++++++++++++++++++++-------
 xen/drivers/vpci/vpci.c       |  28 ++-
 xen/include/xen/pci_regs.h    |   1 +
 xen/include/xen/vpci.h        |  28 ++-
 5 files changed, 413 insertions(+), 84 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:08:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:08:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177970.323786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67m-0004U7-QR; Fri, 03 Sep 2021 10:08:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177970.323786; Fri, 03 Sep 2021 10:08:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67m-0004U0-My; Fri, 03 Sep 2021 10:08:42 +0000
Received: by outflank-mailman (input) for mailman id 177970;
 Fri, 03 Sep 2021 10:08:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM67l-0004Cr-9V
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:41 +0000
Received: from mail-ej1-x62d.google.com (unknown [2a00:1450:4864:20::62d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a207c60b-2f84-4a2e-8c87-e4da339ca8fd;
 Fri, 03 Sep 2021 10:08:36 +0000 (UTC)
Received: by mail-ej1-x62d.google.com with SMTP id mf2so10990326ejb.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:36 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a207c60b-2f84-4a2e-8c87-e4da339ca8fd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ZadtWS/HmdzZ6ZB6hMvTOb7/437UPtLCPee6STSCqfE=;
        b=YhHIT73vy+PVxx26h1vPYEo5XBj3ro3OZ8ijz9+VhFJjzLOVDwpyT4NP5euCOnWr6F
         sXNIdIdwveV+4JJNloPws1E3n6Qi3ChdN9OgIrXxwDb3quCHFdbymal/0yvXMNj9eLKy
         rpvYV/4QQUS3mNCN08ose74UFLipgvvzzaThClvTPt6m8yytTwArmYRLrDbwUdlCjlwa
         Us4A/LRhlNmNfGnBUb+ygPZcwZ46eX0hpKznDPYPRaQBm59/7pFQJhV/aYIUkAeTNmGW
         YCjjIQo1QPg+2ZCq+OnF2tXhNd9qFuj+CzZ8qJS8zb8SFxw36Od7qiX+H7XIm3NLw3Pg
         JKjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ZadtWS/HmdzZ6ZB6hMvTOb7/437UPtLCPee6STSCqfE=;
        b=Uey5oSbYIwQXHH9NMZKn+7cI48afypAEdjXa55LJy1A+9R4MKMFUl49TIEpRjNybEB
         13dtLALW21QIbsUBSq6WP1+gdfH3xLAKEV91ac/4OTv8HAiIpIg8TdXFNfcVyUP3iJic
         L63ds6uBWA3us0EY5o13sTUHj2DhQ0aZYPqDrylJqHO5A2cvntCJr+jYVCxcxs1fBnsy
         pTIUve6hfQIyPMNxP46vNTsYQZ4SixRVfuft2/YPEa1Oxsmt6rVFFIPIa2Uc6zeB+9/K
         +q0ig0nEPa2PYN6/vNRpqE/7Abwmo5hbDgrAZCYolMxnVPxLoPDHE+0V1sl8Adn15pkn
         QV1Q==
X-Gm-Message-State: AOAM532yciggjHHlrkznHx4PQ140XKp6WbKSxiBfvBZDLj0Rv2g3sSnQ
	o7Nccq1vwXUrcAfJJz9Gr//naxT4rws=
X-Google-Smtp-Source: ABdhPJwgZPoQj1ss4jKZJbBH8b+YlK07Jv0KEwHZKKBxDLUyLP9WFuc8uyLUXAIwk7Pj5zBg8EBp3Q==
X-Received: by 2002:a17:907:362:: with SMTP id rs2mr3352436ejb.17.1630663715405;
        Fri, 03 Sep 2021 03:08:35 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 1/9] vpci: Make vpci registers removal a dedicated function
Date: Fri,  3 Sep 2021 13:08:23 +0300
Message-Id: <20210903100831.177748-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vpci register
handlers depending on the domain: hwdom or guest.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/vpci.c | 7 ++++++-
 xen/include/xen/vpci.h  | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index cbd1bac7fc33..b05530f2a6b0 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -35,7 +35,7 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
-void vpci_remove_device(struct pci_dev *pdev)
+void vpci_remove_device_registers(struct pci_dev *pdev)
 {
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
@@ -48,6 +48,11 @@ void vpci_remove_device(struct pci_dev *pdev)
         xfree(r);
     }
     spin_unlock(&pdev->vpci->lock);
+}
+
+void vpci_remove_device(struct pci_dev *pdev)
+{
+    vpci_remove_device_registers(pdev);
     xfree(pdev->vpci->msix);
     xfree(pdev->vpci->msi);
     xfree(pdev->vpci);
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 9f5b5d52e159..b861f438cc78 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -28,6 +28,8 @@ int __must_check vpci_add_handlers(struct pci_dev *dev);
 
 /* Remove all handlers and free vpci related structures. */
 void vpci_remove_device(struct pci_dev *pdev);
+/* Remove all handlers for the device given. */
+void vpci_remove_device_registers(struct pci_dev *pdev);
 
 /* Add/remove a register handler. */
 int __must_check vpci_add_register(struct vpci *vpci,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:08:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:08:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177971.323797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67s-0004p1-3u; Fri, 03 Sep 2021 10:08:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177971.323797; Fri, 03 Sep 2021 10:08:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67r-0004oq-VO; Fri, 03 Sep 2021 10:08:47 +0000
Received: by outflank-mailman (input) for mailman id 177971;
 Fri, 03 Sep 2021 10:08:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM67q-0004Cr-9p
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:46 +0000
Received: from mail-ej1-x62b.google.com (unknown [2a00:1450:4864:20::62b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f91edf82-e50f-4cf4-995d-7339dd9df84b;
 Fri, 03 Sep 2021 10:08:37 +0000 (UTC)
Received: by mail-ej1-x62b.google.com with SMTP id ia27so10983815ejc.10
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:37 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f91edf82-e50f-4cf4-995d-7339dd9df84b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=EU6daQfIrHKFvAQi2eWLYJvBvTorOIHCZnBp6xeFP3g=;
        b=kcubAD1Yrxu5RXJEbu7bj0a07h2QblbRY+s8jSOmtT8ZkdpKKBQGCcX5WvzOVQxHbS
         55eUezgwX9I0TbSWL9LToo0uUqvkN+I7fJZOt84M2creBFDahhPC94vgaafd+kHnigfs
         DNKtTUFK6Ibxfig4mz9B0SfhATEmqOExjB0cc+qPmEFCZiEZYlKxAY62tn4K5XHA2bm4
         DluMJoVFnrYQN3CvNURCZGzNvdnDc+48eMedOJKcKFfHLc/He6w52N5HxvkUmrxcolud
         3u9lfiOIhIVkADvUQQalsSwaH9feuKeguOXZ9unsAMQA/IpxrwZ8GvEpMu1xiLdcXTVR
         oSUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=EU6daQfIrHKFvAQi2eWLYJvBvTorOIHCZnBp6xeFP3g=;
        b=idmU1+ovw6G5NrPkYssghrV2ChJh/LzPdwy3MG/ApEHrp5GoaavT+I5dNB60eSTktN
         HgWdygLn0apuTFRpUQpgLekox//f5VAGdeXKMigSq+atohC+MCeABhOcuKlCxi6UV3vr
         QYfBbn38/lB4r+arfq2OEJMJs3Livvfkgw1CZS6JIvZ8s8FWkaGuQTFhGQBtZuy7ClYT
         pmDLsu9mHGLUfcG2EYPwmQ+V7kYXqThdbxAtX8MaoTRBAdM4lkzhDqa1HtHdEcN35t97
         Oleps0lYHqraC5hK8s3XUf/ZV0JXBRG+goSNiWJu6/SHn/izJln78+dkVr8J7akP0fuS
         voRQ==
X-Gm-Message-State: AOAM531Ff1UH+vuQfiIvWYw/6W2PNoubK9ygdflTG4Or7V5FEJMBi9S/
	Psl+wxbjX9KG29GVb75XzcQJgSln75o=
X-Google-Smtp-Source: ABdhPJz6mPKOr3Bp3qDr/3iV61C9/zLAySQ2YnRt9O7RgvETUrpMoCcyTRoUbjFWHZd9dkA+8OI+fA==
X-Received: by 2002:a17:906:1c8c:: with SMTP id g12mr3370387ejh.186.1630663716639;
        Fri, 03 Sep 2021 03:08:36 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign
Date: Fri,  3 Sep 2021 13:08:24 +0300
Message-Id: <20210903100831.177748-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

When a PCI device gets assigned/de-assigned some work on vPCI side needs
to be done for that device. Introduce a pair of hooks so vPCI can handle
that.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/passthrough/pci.c |  9 +++++++++
 xen/drivers/vpci/vpci.c       | 21 +++++++++++++++++++++
 xen/include/xen/vpci.h        | 16 ++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 25304dbe9956..deef986acbb4 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -864,6 +864,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
     if ( ret )
         goto out;
 
+    ret = vpci_deassign_device(d, pdev);
+    if ( ret )
+        goto out;
+
     if ( pdev->domain == hardware_domain  )
         pdev->quarantine = false;
 
@@ -1425,6 +1429,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
         rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
     }
 
+    if ( rc )
+        goto done;
+
+    rc = vpci_assign_device(d, pdev);
+
  done:
     if ( rc )
         printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index b05530f2a6b0..ee0ad63a3c12 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -86,6 +86,27 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
 
     return rc;
 }
+
+/* Notify vPCI that device is assigned to guest. */
+int vpci_assign_device(struct domain *d, struct pci_dev *dev)
+{
+    /* It only makes sense to assign for hwdom or guest domain. */
+    if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) )
+        return 0;
+
+    return 0;
+}
+
+/* Notify vPCI that device is de-assigned from guest. */
+int vpci_deassign_device(struct domain *d, struct pci_dev *dev)
+{
+    /* It only makes sense to de-assign from hwdom or guest domain. */
+    if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) )
+        return 0;
+
+    return 0;
+}
+
 #endif /* __XEN__ */
 
 static int vpci_register_cmp(const struct vpci_register *r1,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index b861f438cc78..e7a1a09ab4c9 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -26,6 +26,12 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 /* Add vPCI handlers to device. */
 int __must_check vpci_add_handlers(struct pci_dev *dev);
 
+/* Notify vPCI that device is assigned to guest. */
+int __must_check vpci_assign_device(struct domain *d, struct pci_dev *dev);
+
+/* Notify vPCI that device is de-assigned from guest. */
+int __must_check vpci_deassign_device(struct domain *d, struct pci_dev *dev);
+
 /* Remove all handlers and free vpci related structures. */
 void vpci_remove_device(struct pci_dev *pdev);
 /* Remove all handlers for the device given. */
@@ -220,6 +226,16 @@ static inline int vpci_add_handlers(struct pci_dev *pdev)
     return 0;
 }
 
+static inline int vpci_assign_device(struct domain *d, struct pci_dev *dev)
+{
+    return 0;
+};
+
+static inline int vpci_deassign_device(struct domain *d, struct pci_dev *dev)
+{
+    return 0;
+};
+
 static inline void vpci_dump_msi(void) { }
 
 static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:08:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:08:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177972.323808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67x-0005DG-De; Fri, 03 Sep 2021 10:08:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177972.323808; Fri, 03 Sep 2021 10:08:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM67x-0005D5-9t; Fri, 03 Sep 2021 10:08:53 +0000
Received: by outflank-mailman (input) for mailman id 177972;
 Fri, 03 Sep 2021 10:08:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM67v-0004Cr-9x
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:51 +0000
Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89cdb7c8-25df-49a2-9db1-259e6a8e8cb4;
 Fri, 03 Sep 2021 10:08:38 +0000 (UTC)
Received: by mail-ej1-x635.google.com with SMTP id me10so10971791ejb.11
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:38 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89cdb7c8-25df-49a2-9db1-259e6a8e8cb4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=3lxrhjHMCKgFgAktbulKfLSKY9SXLgEbqYmkCK+IHbI=;
        b=Cv6kNVKYxIEVmbcaX8DIKNQHN8wBj6cryOD0sKuPJKv/+o8YHRTdPVgVqD77l1hCb6
         vatzSz/C6yHUi7ZQZQ1JJlU9uCyOpq+GVxJBb9jeFDqhIQnX1cbkCtJinWZEU8eqJTQb
         jio/WmVPkE2k/zzaVXS7GL4dFAA2E0ouwzXpQCTsyFf2pUaDlvQFEaS78sYbu8YIPTXR
         U0FzaEvsPtV9sf174mXGL7Qw5/UuezecIBC+kpvG2EDzz17a/p2ZqvM6XZnGtCvqk3aL
         JIFwtBwu3xMljkBTPCWgfBV3fdb37XBeKMLOsqDZpBQznPnsEXb1R1MSjDFX+LUX9qeG
         nioA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=3lxrhjHMCKgFgAktbulKfLSKY9SXLgEbqYmkCK+IHbI=;
        b=Ar3aSlPCK0GtA4sCgxbYDimL2lZEujfGD2CTEiPA4eS5hh2KiOr+ahOg/9X58VTFh+
         v5lR3rXR1PGqZBPlv6NyAtoXOKwDm/GlvPQORD1cv9rpQng+6BEL/CrL/tjQaIaeOsLx
         RvUcwQiWdrc86goE79/rqqttWHovaGPNXawkzszYlLSZkzIyloeq+35lhBDgKREonL2E
         HY7eWCxYx2OJ1YAx3K2dVxroEccnwMuJyEQNYtGy25KnLRTzVJxrF6wWVINKeRkfq28i
         zk5netoP+L1mM+0e/Z1PeN9DzxSn+js6BPlupxFg73mQJCkXFXuJlL5oMNeuWRUsOoh5
         FBug==
X-Gm-Message-State: AOAM531FhXhUDauNFwiRD38UU1Lwv98u1OTOwNGaZvrk9+LAgVT1XWyj
	NZgM9J3TcaXszgvC0eIpF50lRxd6098=
X-Google-Smtp-Source: ABdhPJydxNWuh4lL4oZ/Fi+pB8Xp26xfg9TIMj2hvhsDyPP/rb6TmJbYwxmJmWH3EGwe4uJeauyAdQ==
X-Received: by 2002:a17:906:ff41:: with SMTP id zo1mr3148886ejb.525.1630663717693;
        Fri, 03 Sep 2021 03:08:37 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 3/9] vpci/header: Move register assignments from init_bars
Date: Fri,  3 Sep 2021 13:08:25 +0300
Message-Id: <20210903100831.177748-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vpci register
handlers depending on the domain: hwdom or guest.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 83 ++++++++++++++++++++++++++-------------
 1 file changed, 56 insertions(+), 27 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index f8cd55e7c024..31bca7a12942 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -445,6 +445,55 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
+static int add_bar_handlers(struct pci_dev *pdev)
+{
+    unsigned int i;
+    struct vpci_header *header = &pdev->vpci->header;
+    struct vpci_bar *bars = header->bars;
+    int rc;
+
+    /* Setup a handler for the command register. */
+    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
+                           2, header);
+    if ( rc )
+        return rc;
+
+    if ( pdev->ignore_bars )
+        return 0;
+
+    for ( i = 0; i < PCI_HEADER_NORMAL_NR_BARS + 1; i++ )
+    {
+        if ( (bars[i].type == VPCI_BAR_IO) || (bars[i].type == VPCI_BAR_EMPTY) )
+            continue;
+
+        if ( bars[i].type == VPCI_BAR_ROM )
+        {
+            unsigned int rom_reg;
+            uint8_t header_type = pci_conf_read8(pdev->sbdf,
+                                                 PCI_HEADER_TYPE) & 0x7f;
+            if ( header_type == PCI_HEADER_TYPE_NORMAL )
+                rom_reg = PCI_ROM_ADDRESS;
+            else
+                rom_reg = PCI_ROM_ADDRESS1;
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                   rom_reg, 4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+        else
+        {
+            uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
+
+            /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
+                                   4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+    }
+    return 0;
+}
+
 static int init_bars(struct pci_dev *pdev)
 {
     uint16_t cmd;
@@ -470,14 +519,8 @@ static int init_bars(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
-    if ( rc )
-        return rc;
-
     if ( pdev->ignore_bars )
-        return 0;
+        return add_bar_handlers(pdev);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -492,14 +535,6 @@ static int init_bars(struct pci_dev *pdev)
         if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO )
         {
             bars[i].type = VPCI_BAR_MEM64_HI;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
-            if ( rc )
-            {
-                pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-                return rc;
-            }
-
             continue;
         }
 
@@ -532,14 +567,6 @@ static int init_bars(struct pci_dev *pdev)
         bars[i].addr = addr;
         bars[i].size = size;
         bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH;
-
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4,
-                               &bars[i]);
-        if ( rc )
-        {
-            pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-            return rc;
-        }
     }
 
     /* Check expansion ROM. */
@@ -553,11 +580,13 @@ static int init_bars(struct pci_dev *pdev)
         rom->addr = addr;
         header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) &
                               PCI_ROM_ADDRESS_ENABLE;
+    }
 
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg,
-                               4, rom);
-        if ( rc )
-            rom->type = VPCI_BAR_EMPTY;
+    rc = add_bar_handlers(pdev);
+    if ( rc )
+    {
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+        return rc;
     }
 
     return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:08:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:08:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177975.323819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM681-0005gd-Ne; Fri, 03 Sep 2021 10:08:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177975.323819; Fri, 03 Sep 2021 10:08:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM681-0005gQ-Jh; Fri, 03 Sep 2021 10:08:57 +0000
Received: by outflank-mailman (input) for mailman id 177975;
 Fri, 03 Sep 2021 10:08:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM680-0004Cr-AA
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:56 +0000
Received: from mail-ed1-x533.google.com (unknown [2a00:1450:4864:20::533])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cfb4143e-4d10-43e8-a103-28cf681063e9;
 Fri, 03 Sep 2021 10:08:40 +0000 (UTC)
Received: by mail-ed1-x533.google.com with SMTP id z19so7227660edi.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:40 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfb4143e-4d10-43e8-a103-28cf681063e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=xpLp7pLNUQ/DHxUjEhr/yTY0NKqDW6A0T3pZsizfVtE=;
        b=aS6tQADQYeExZDhOABC9iBB8GoJc3IuoFCRob+Cctdwhth+nveel2KGxkARVYlU6Jd
         Qgnsw8axED8hMhxC2pYHQ9oNoSo/Olz1veZ4TZuqbKzQZivPkh6by4D4cHiKbYkHSK91
         YRf3tKX+OFy/jc8fvxtX6h9mWPRbWAPECFFsxFE+A6QKglW5XRGmbuMg67a0TGkoDIJw
         /eSmzbrBWGArGKWd2yxvzNtdGv662yFyuBYuegN1QYkQSgbHkEqDsAUOzpcLfZcumjrq
         cZ2XfDNZRq17L5VjX+W5pDzGFNH7tT+468gnBCcwp09gPnFouOqA3oPEThxmheEzgx36
         8JiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=xpLp7pLNUQ/DHxUjEhr/yTY0NKqDW6A0T3pZsizfVtE=;
        b=hGnApIAP71TtOCLXUjUKPy4WIrMPkur5k+NPSUeWn7p/Bf8CxvczUWGnlwjKqpPQcP
         9Y0eVWNEsj5402YA2Gbn//AJYvDDKoVXzV9QDF/Ps+iNmInd9cB0Cc+SOoFZS8/asqwT
         tTfsJeeSHJaC2dhpcFjiFNFectWoPuWWPGg0Z7aGKDHciQp/Z5hDb51wcF6vB7kI2N1r
         o0cFz+f9LWpM7xDisrSAzoVOdv0ooVvwsKGxRdZ2MfEA96Rzwy9NPLeVRX8dmbhQM0j6
         ZYj5JkfP2reHEmbEYz3M/2N8AeaN+RpskyCOLs7iDYTcCtetuKAJNg2RtePNh6AlqiEw
         d1Mw==
X-Gm-Message-State: AOAM5334/en9MEUJvPwecu2YmyDa6bOMJinsNafy5DPz5/8RswTn4lXw
	mt9V6eUOxVENPqJZI0u2GjZF5lc+2Bo=
X-Google-Smtp-Source: ABdhPJwZyi4jY7+BYhHRUtcHJ71NsVbFBf7jRjXcdY+sWwfFplhPMgxjOeq7kjOI9krtIeEBDoMxMA==
X-Received: by 2002:aa7:da8c:: with SMTP id q12mr3199500eds.380.1630663718806;
        Fri, 03 Sep 2021 03:08:38 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 4/9] vpci/header: Add and remove register handlers dynamically
Date: Fri,  3 Sep 2021 13:08:26 +0300
Message-Id: <20210903100831.177748-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add relevant vpci register handlers when assigning PCI device to a domain
and remove those when de-assigning. This allows having different
handlers for different domains, e.g. hwdom and other guests.

Use stubs for guest domains for now.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 78 +++++++++++++++++++++++++++++++++++----
 xen/drivers/vpci/vpci.c   |  4 +-
 xen/include/xen/vpci.h    |  4 ++
 3 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 31bca7a12942..5218b1af247e 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -397,6 +397,17 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
     pci_conf_write32(pdev->sbdf, reg, val);
 }
 
+static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
                       uint32_t val, void *data)
 {
@@ -445,14 +456,25 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
-static int add_bar_handlers(struct pci_dev *pdev)
+static void guest_rom_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_rom_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
+static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom)
 {
     unsigned int i;
     struct vpci_header *header = &pdev->vpci->header;
     struct vpci_bar *bars = header->bars;
     int rc;
 
-    /* Setup a handler for the command register. */
+    /* Setup a handler for the command register: same for hwdom and guests. */
     rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
                            2, header);
     if ( rc )
@@ -475,8 +497,13 @@ static int add_bar_handlers(struct pci_dev *pdev)
                 rom_reg = PCI_ROM_ADDRESS;
             else
                 rom_reg = PCI_ROM_ADDRESS1;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
-                                   rom_reg, 4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                       rom_reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_rom_read, guest_rom_write,
+                                       rom_reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -485,8 +512,13 @@ static int add_bar_handlers(struct pci_dev *pdev)
             uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
 
             /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write,
+                                       reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_bar_read, guest_bar_write,
+                                       reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -520,7 +552,7 @@ static int init_bars(struct pci_dev *pdev)
     }
 
     if ( pdev->ignore_bars )
-        return add_bar_handlers(pdev);
+        return add_bar_handlers(pdev, true);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -582,7 +614,7 @@ static int init_bars(struct pci_dev *pdev)
                               PCI_ROM_ADDRESS_ENABLE;
     }
 
-    rc = add_bar_handlers(pdev);
+    rc = add_bar_handlers(pdev, true);
     if ( rc )
     {
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
@@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
 }
 REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
 
+int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
+{
+    int rc;
+
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+
+    /* It only makes sense to add registers for hwdom or guest domain. */
+    if ( d->domain_id >= DOMID_FIRST_RESERVED )
+        return 0;
+
+    if ( is_hardware_domain(d) )
+        rc = add_bar_handlers(pdev, true);
+    else
+        rc = add_bar_handlers(pdev, false);
+
+    if ( rc )
+        gprintk(XENLOG_ERR,
+            "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
+            d->domain_id);
+    return rc;
+}
+
+int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev)
+{
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index ee0ad63a3c12..4530313f01e7 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -94,7 +94,7 @@ int vpci_assign_device(struct domain *d, struct pci_dev *dev)
     if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) )
         return 0;
 
-    return 0;
+    return vpci_bar_add_handlers(d, dev);
 }
 
 /* Notify vPCI that device is de-assigned from guest. */
@@ -104,7 +104,7 @@ int vpci_deassign_device(struct domain *d, struct pci_dev *dev)
     if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) )
         return 0;
 
-    return 0;
+    return vpci_bar_remove_handlers(d, dev);
 }
 
 #endif /* __XEN__ */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index e7a1a09ab4c9..4aa2941a1081 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -63,6 +63,10 @@ uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg,
  */
 bool __must_check vpci_process_pending(struct vcpu *v);
 
+/* Add/remove BAR handlers for a domain. */
+int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev);
+int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev);
+
 struct vpci {
     /* List of vPCI handlers for a device. */
     struct list_head handlers;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:09:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:09:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177978.323830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM686-0006Ar-8e; Fri, 03 Sep 2021 10:09:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177978.323830; Fri, 03 Sep 2021 10:09:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM686-0006AD-3I; Fri, 03 Sep 2021 10:09:02 +0000
Received: by outflank-mailman (input) for mailman id 177978;
 Fri, 03 Sep 2021 10:09:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM685-0004Cr-AL
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:01 +0000
Received: from mail-ed1-x535.google.com (unknown [2a00:1450:4864:20::535])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 37d65df0-7563-4c10-bccc-ae1dcb106e2e;
 Fri, 03 Sep 2021 10:08:40 +0000 (UTC)
Received: by mail-ed1-x535.google.com with SMTP id l6so7262298edb.7
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:40 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 37d65df0-7563-4c10-bccc-ae1dcb106e2e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=s1iI/pfNat4kQKr5zc/XN1JzcSSy61j1M1hI6YBK6Xs=;
        b=gBoRCW32iLvtnLPg39hDXD7cmyijkqaXRSJbAXbKOPEXyhDl0dgAaIZEz7n/8UZi4w
         xyP3FtbSUjU/63qIdjQ0mfR/gRCcCNLbo5+ikGBVrFTa2CYIa89ObRAV5x0LVELdz3K0
         gMdRG0CMUMNI1PoCqvx9gcF4dXo8MyFy60K9OjUcvHXSWX5viFQP5LDlfcUWYNyGXtIP
         zNLijBaZRzBMnCN3hJ3eHRipmudvgGiOpZcTXSfGXwt6qisNkUKhee+0WE34G69/e8mr
         HfTRJZRDqlnW++VApih2L91ZVfsGE+GAThkzBOdxFK99Io93rNBqo/1UlvVCckHjsX+9
         O+Wg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=s1iI/pfNat4kQKr5zc/XN1JzcSSy61j1M1hI6YBK6Xs=;
        b=ZH36fSUXi+4e9k9fED+FPhzapioOgKCe7C+6cxPN0UuaFxp4KN4lXQD87Uvbho9dbc
         ZZx86Su6a9Z8uVYtlcN4+kEjDJCgZ66LjCIPqAbzSjCFy51ucPxamBYSMFI7FJLj8TMS
         oFcpaTx+5DsPQW5xkXwvU8PHRN9REpyvveUCQtO8eWkMH1qjy05X2+/WO6c4SMpjXVFD
         zxI/g3d0pl6junt10RLbld6CQNUuUK2yZdwoOe+E11YUqcxfMLjHCIHlway3mXLBNnrk
         EKO+JALJdLx0ekrKEmzq1zohANVnfEMuKETO5Hrekci6wFI2BUOyMBmsV0fxqJSQNEnb
         rXUA==
X-Gm-Message-State: AOAM5320yZ6ZaDujZgZ7muxVMPZlWCCMjj6PlaSzBYzth8romaK4UiCc
	VI2SoTl4QY8DO4p+o22aqXANV8JE758=
X-Google-Smtp-Source: ABdhPJxrROUzQ4fz2kKiNwMn7jDNiM+KTMqFpM3OJ8BQuHdw/HwgIHYajSQwwLbnO5MBR/YkklXeKg==
X-Received: by 2002:a05:6402:424c:: with SMTP id g12mr3162442edb.37.1630663719821;
        Fri, 03 Sep 2021 03:08:39 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Date: Fri,  3 Sep 2021 13:08:27 +0300
Message-Id: <20210903100831.177748-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Emulate guest BAR register values: this allows creating a guest view
of the registers and emulates size and properties probe as it is done
during PCI device enumeration by the guest.

ROM BAR is only handled for the hardware domain and for guest domains
there is a stub: at the moment PCI expansion ROM is x86 only, so it
might not be used by other architectures without emulating x86. Other
use-cases may include using that expansion ROM before Xen boots, hence
no emulation is needed in Xen itself. Or when a guest wants to use the
ROM code which seems to be rare.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c  | 69 +++++++++++++++++++++++++++++++++++++-
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/vpci.h     |  3 ++
 3 files changed, 72 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 5218b1af247e..793f79ece831 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
 static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
                             uint32_t val, void *data)
 {
+    struct vpci_bar *bar = data;
+    bool hi = false;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+    {
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        hi = true;
+    }
+    else
+        val &= PCI_BASE_ADDRESS_MEM_MASK;
+    bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
+    bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
 }
 
 static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
                                void *data)
 {
-    return 0xffffffff;
+    struct vpci_bar *bar = data;
+    uint32_t val;
+    bool hi = false;
+
+    switch ( bar->type )
+    {
+    case VPCI_BAR_MEM64_HI:
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        hi = true;
+        /* fallthrough */
+    case VPCI_BAR_MEM64_LO:
+    {
+        if ( hi )
+            val = bar->guest_addr >> 32;
+        else
+            val = bar->guest_addr & 0xffffffff;
+        if ( (val & PCI_BASE_ADDRESS_MEM_MASK_32) ==  PCI_BASE_ADDRESS_MEM_MASK_32 )
+        {
+            /* Guests detects BAR's properties and sizes. */
+            if ( hi )
+                val = bar->size >> 32;
+            else
+                val = 0xffffffff & ~(bar->size - 1);
+        }
+        if ( !hi )
+        {
+            val |= PCI_BASE_ADDRESS_MEM_TYPE_64;
+            val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
+        }
+        bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
+        bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
+        break;
+    }
+    case VPCI_BAR_MEM32:
+    {
+        val = bar->guest_addr;
+        if ( (val & PCI_BASE_ADDRESS_MEM_MASK_32) ==  PCI_BASE_ADDRESS_MEM_MASK_32 )
+            val = 0xffffffff & ~(bar->size - 1);
+        val |= PCI_BASE_ADDRESS_MEM_TYPE_32;
+        val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
+        break;
+    }
+    default:
+        val = bar->guest_addr;
+        break;
+    }
+    return val;
 }
 
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
@@ -522,6 +582,13 @@ static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom)
             if ( rc )
                 return rc;
         }
+        /*
+         * It is neither safe nor secure to initialize guest's view of the BARs
+         * with real values which are used by the hardware domain, so assign
+         * all zeros to guest's view of the BARs, so the guest can perform
+         * proper PCI device enumeration and assign BARs on its own.
+         */
+        bars[i].guest_addr = 0;
     }
     return 0;
 }
diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h
index cc4ee3b83e5c..038eb18c5357 100644
--- a/xen/include/xen/pci_regs.h
+++ b/xen/include/xen/pci_regs.h
@@ -103,6 +103,7 @@
 #define  PCI_BASE_ADDRESS_MEM_TYPE_64	0x04	/* 64 bit address */
 #define  PCI_BASE_ADDRESS_MEM_PREFETCH	0x08	/* prefetchable? */
 #define  PCI_BASE_ADDRESS_MEM_MASK	(~0x0fUL)
+#define  PCI_BASE_ADDRESS_MEM_MASK_32	(~0x0fU)
 #define  PCI_BASE_ADDRESS_IO_MASK	(~0x03UL)
 /* bit 1 is reserved if address_space = 1 */
 
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 4aa2941a1081..db86b8e7fa3c 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -77,7 +77,10 @@ struct vpci {
     struct vpci_header {
         /* Information about the PCI BARs of this device. */
         struct vpci_bar {
+            /* Physical view of the BAR. */
             uint64_t addr;
+            /* Guest view of the BAR. */
+            uint64_t guest_addr;
             uint64_t size;
             enum {
                 VPCI_BAR_EMPTY,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:09:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:09:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177982.323841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68B-0006or-Lg; Fri, 03 Sep 2021 10:09:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177982.323841; Fri, 03 Sep 2021 10:09:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68B-0006od-HV; Fri, 03 Sep 2021 10:09:07 +0000
Received: by outflank-mailman (input) for mailman id 177982;
 Fri, 03 Sep 2021 10:09:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM68A-0004Cr-Ae
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:06 +0000
Received: from mail-ej1-x633.google.com (unknown [2a00:1450:4864:20::633])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 90692ff9-f747-4eed-8a0f-59677580be6d;
 Fri, 03 Sep 2021 10:08:42 +0000 (UTC)
Received: by mail-ej1-x633.google.com with SMTP id a25so11014156ejv.6
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:42 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90692ff9-f747-4eed-8a0f-59677580be6d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=PguLDO+PDisBHZWiq5SMa+3ZkndR3NGesmKLiuatRS8=;
        b=gvhPjQoucEcq+9aAL21Wr1DWTsBBuSYkSRqt2KCkvaHEiq66AaoEH4VEGm0T6rw7an
         eNLQMtKsECkxAj541srNzEB+llBTYXQJjMGYBcFjVb7/pNGvYqRgNv42tI5IQJQm63rf
         oCVEBOpCac6Lbf0xITzzcrfyNtd9pcv5tkjAn5lmD5IZLszS7BGsr2ifeUWBpqzOd4Aq
         8NhQxw3gLiIIWNZm1UGUBRrAl+24kJSXnaoRtcC4T59AZnJmalxLd/eSfBP5faMAjwiD
         tEzMHIJ4CDVHyHqvObg1+rtH46Gc+P/t/PtvGpW1Bds3YAdPHoG13TmR9RNbg+D2kVPZ
         +PQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=PguLDO+PDisBHZWiq5SMa+3ZkndR3NGesmKLiuatRS8=;
        b=tSPiQuPMmxsUDFhHlkZ83e/4oPxp4CwtNiMnM4BG0dwQzT/QjwUjq9RHSXdsE4oN6b
         TJwRwUe5xrRO+fYOxxnryU0Xhf4flVRT/Y42KEhvCnFf29Mv3Oc9uq2u6zqEbRZSUKD7
         H4GcNz5ja+bMQgaSSeOYE1UsVMd41V55vXufrVHgPyXSO8nW5vLcNV13MelEKm/RUfrj
         exwrq75nDzxsYPtfU68V/2K4hJs6w4RpeNfXpSely7PfVLY9ZXeHNukQ5HLOPoWLqcKA
         dcsT9jAJKx73nG8zLqjSQ8NElrSMrcyczltY/RyYq88mJjW2vH7qt1esatD8rT80YgYz
         tdsw==
X-Gm-Message-State: AOAM533RE6MO9/iDYyOSdcsGsLCM9oRMQuPIHq7jlojE8mv5ZvMiwyCu
	E4m8LkZBVuXbcMgUqRmTR15VicZvF18=
X-Google-Smtp-Source: ABdhPJyl+Tub9JIMSY/l/hoFxplCzEScXDX6jTh8LYN2njGF7bl9ybauJNv5gjjSPitXDu2IPjei+w==
X-Received: by 2002:a17:906:4a8e:: with SMTP id x14mr3341520eju.389.1630663720860;
        Fri, 03 Sep 2021 03:08:40 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Date: Fri,  3 Sep 2021 13:08:28 +0300
Message-Id: <20210903100831.177748-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Instead of handling a single range set, that contains all the memory
regions of all the BARs and ROM, have them per BAR.

This is in preparation of making non-identity mappings in p2m for the
MMIOs/ROM.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 172 ++++++++++++++++++++++++++------------
 xen/include/xen/vpci.h    |   3 +-
 2 files changed, 122 insertions(+), 53 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 793f79ece831..7f54199a3894 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -131,49 +131,75 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
 
 bool vpci_process_pending(struct vcpu *v)
 {
-    if ( v->vpci.mem )
+    if ( v->vpci.num_mem_ranges )
     {
         struct map_data data = {
             .d = v->domain,
             .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
         };
-        int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data);
+        struct pci_dev *pdev = v->vpci.pdev;
+        struct vpci_header *header = &pdev->vpci->header;
+        unsigned int i;
 
-        if ( rc == -ERESTART )
-            return true;
+        for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+        {
+            struct vpci_bar *bar = &header->bars[i];
+            int rc;
 
-        spin_lock(&v->vpci.pdev->vpci->lock);
-        /* Disable memory decoding unconditionally on failure. */
-        modify_decoding(v->vpci.pdev,
-                        rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
-                        !rc && v->vpci.rom_only);
-        spin_unlock(&v->vpci.pdev->vpci->lock);
+            if ( !bar->mem )
+                continue;
 
-        rangeset_destroy(v->vpci.mem);
-        v->vpci.mem = NULL;
-        if ( rc )
-            /*
-             * FIXME: in case of failure remove the device from the domain.
-             * Note that there might still be leftover mappings. While this is
-             * safe for Dom0, for DomUs the domain will likely need to be
-             * killed in order to avoid leaking stale p2m mappings on
-             * failure.
-             */
-            vpci_remove_device(v->vpci.pdev);
+            rc = rangeset_consume_ranges(bar->mem, map_range, &data);
+
+            if ( rc == -ERESTART )
+                return true;
+
+            spin_lock(&pdev->vpci->lock);
+            /* Disable memory decoding unconditionally on failure. */
+            modify_decoding(pdev,
+                            rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
+                            !rc && v->vpci.rom_only);
+            spin_unlock(&pdev->vpci->lock);
+
+            rangeset_destroy(bar->mem);
+            bar->mem = NULL;
+            v->vpci.num_mem_ranges--;
+            if ( rc )
+                /*
+                 * FIXME: in case of failure remove the device from the domain.
+                 * Note that there might still be leftover mappings. While this is
+                 * safe for Dom0, for DomUs the domain will likely need to be
+                 * killed in order to avoid leaking stale p2m mappings on
+                 * failure.
+                 */
+                vpci_remove_device(pdev);
+        }
     }
 
     return false;
 }
 
 static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
-                            struct rangeset *mem, uint16_t cmd)
+                            uint16_t cmd)
 {
     struct map_data data = { .d = d, .map = true };
-    int rc;
+    struct vpci_header *header = &pdev->vpci->header;
+    int rc = 0;
+    unsigned int i;
+
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
 
-    while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
-        process_pending_softirqs();
-    rangeset_destroy(mem);
+        if ( !bar->mem )
+            continue;
+
+        while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
+                                              &data)) == -ERESTART )
+            process_pending_softirqs();
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
     if ( !rc )
         modify_decoding(pdev, cmd, false);
 
@@ -181,7 +207,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
 }
 
 static void defer_map(struct domain *d, struct pci_dev *pdev,
-                      struct rangeset *mem, uint16_t cmd, bool rom_only)
+                      uint16_t cmd, bool rom_only, uint8_t num_mem_ranges)
 {
     struct vcpu *curr = current;
 
@@ -192,9 +218,9 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
      * started for the same device if the domain is not well-behaved.
      */
     curr->vpci.pdev = pdev;
-    curr->vpci.mem = mem;
     curr->vpci.cmd = cmd;
     curr->vpci.rom_only = rom_only;
+    curr->vpci.num_mem_ranges = num_mem_ranges;
     /*
      * Raise a scheduler softirq in order to prevent the guest from resuming
      * execution with pending mapping operations, to trigger the invocation
@@ -206,42 +232,47 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
 static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
 {
     struct vpci_header *header = &pdev->vpci->header;
-    struct rangeset *mem = rangeset_new(NULL, NULL, 0);
     struct pci_dev *tmp, *dev = NULL;
     const struct vpci_msix *msix = pdev->vpci->msix;
-    unsigned int i;
+    unsigned int i, j;
     int rc;
-
-    if ( !mem )
-        return -ENOMEM;
+    uint8_t num_mem_ranges;
 
     /*
-     * Create a rangeset that represents the current device BARs memory region
+     * Create a rangeset per BAR that represents the current device memory region
      * and compare it against all the currently active BAR memory regions. If
      * an overlap is found, subtract it from the region to be mapped/unmapped.
      *
-     * First fill the rangeset with all the BARs of this device or with the ROM
+     * First fill the rangesets with all the BARs of this device or with the ROM
      * BAR only, depending on whether the guest is toggling the memory decode
      * bit of the command register, or the enable bit of the ROM BAR register.
      */
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
-        const struct vpci_bar *bar = &header->bars[i];
+        struct vpci_bar *bar = &header->bars[i];
         unsigned long start = PFN_DOWN(bar->addr);
         unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
+        bar->mem = NULL;
+
         if ( !MAPPABLE_BAR(bar) ||
              (rom_only ? bar->type != VPCI_BAR_ROM
                        : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) )
             continue;
 
-        rc = rangeset_add_range(mem, start, end);
+        bar->mem = rangeset_new(NULL, NULL, 0);
+        if ( !bar->mem )
+        {
+            rc = -ENOMEM;
+            goto fail;
+        }
+
+        rc = rangeset_add_range(bar->mem, start, end);
         if ( rc )
         {
             printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n",
                    start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            goto fail;
         }
     }
 
@@ -252,14 +283,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
         unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) +
                                      vmsix_table_size(pdev->vpci, i) - 1);
 
-        rc = rangeset_remove_range(mem, start, end);
-        if ( rc )
+        for ( j = 0; j < ARRAY_SIZE(header->bars); j++ )
         {
-            printk(XENLOG_G_WARNING
-                   "Failed to remove MSIX table [%lx, %lx]: %d\n",
-                   start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            const struct vpci_bar *bar = &header->bars[j];
+
+            if ( !bar->mem )
+                continue;
+
+            rc = rangeset_remove_range(bar->mem, start, end);
+            if ( rc )
+            {
+                printk(XENLOG_G_WARNING
+                       "Failed to remove MSIX table [%lx, %lx]: %d\n",
+                       start, end, rc);
+                goto fail;
+            }
         }
     }
 
@@ -291,7 +329,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             unsigned long start = PFN_DOWN(bar->addr);
             unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
-            if ( !bar->enabled || !rangeset_overlaps_range(mem, start, end) ||
+            if ( !bar->enabled ||
+                 !rangeset_overlaps_range(bar->mem, start, end) ||
                  /*
                   * If only the ROM enable bit is toggled check against other
                   * BARs in the same device for overlaps, but not against the
@@ -300,13 +339,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
                  (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) )
                 continue;
 
-            rc = rangeset_remove_range(mem, start, end);
+            rc = rangeset_remove_range(bar->mem, start, end);
             if ( rc )
             {
                 printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n",
                        start, end, rc);
-                rangeset_destroy(mem);
-                return rc;
+                goto fail;
             }
         }
     }
@@ -324,12 +362,42 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
          * will always be to establish mappings and process all the BARs.
          */
         ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only);
-        return apply_map(pdev->domain, pdev, mem, cmd);
+        return apply_map(pdev->domain, pdev, cmd);
     }
 
-    defer_map(dev->domain, dev, mem, cmd, rom_only);
+    /* Find out how many memory ranges has left after MSI and overlaps. */
+    num_mem_ranges = 0;
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        if ( !rangeset_is_empty(bar->mem) )
+            num_mem_ranges++;
+    }
+
+    /*
+     * There are cases when PCI device, root port for example, has neither
+     * memory space nor IO. In this case PCI command register write is
+     * missed resulting in the underlying PCI device not functional, so:
+     *   - if there are no regions write the command register now
+     *   - if there are regions then defer work and write later on
+     */
+    if ( !num_mem_ranges )
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+    else
+        defer_map(dev->domain, dev, cmd, rom_only, num_mem_ranges);
 
     return 0;
+
+fail:
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
+    return rc;
 }
 
 static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index db86b8e7fa3c..a0cbdb4bf4fd 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -82,6 +82,7 @@ struct vpci {
             /* Guest view of the BAR. */
             uint64_t guest_addr;
             uint64_t size;
+            struct rangeset *mem;
             enum {
                 VPCI_BAR_EMPTY,
                 VPCI_BAR_IO,
@@ -156,9 +157,9 @@ struct vpci {
 
 struct vpci_vcpu {
     /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */
-    struct rangeset *mem;
     struct pci_dev *pdev;
     uint16_t cmd;
+    uint8_t num_mem_ranges;
     bool rom_only : 1;
 };
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:09:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:09:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177986.323851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68G-0007KL-0K; Fri, 03 Sep 2021 10:09:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177986.323851; Fri, 03 Sep 2021 10:09:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68F-0007K7-Rz; Fri, 03 Sep 2021 10:09:11 +0000
Received: by outflank-mailman (input) for mailman id 177986;
 Fri, 03 Sep 2021 10:09:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM68F-0004Cr-Ae
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:11 +0000
Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 574bafea-333f-4458-a85f-9d85f3dc68ab;
 Fri, 03 Sep 2021 10:08:43 +0000 (UTC)
Received: by mail-ej1-x635.google.com with SMTP id a25so11014222ejv.6
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:43 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 574bafea-333f-4458-a85f-9d85f3dc68ab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=EsmmWshqcRWEmqQF2VjxxyMLQRhse2D1gTlLi8+uzDk=;
        b=kYdmRslCs8NTALbXGc7+QzrWmSxEXO8+BDs+NVl9JKDEti73SNG8Bxa30Qe1HIM2o/
         XlN/YA1k2fkV4slRcmp6oFpn1xBHxhY1jlkZ1Y/5EYq9OhYgZ58xAhwDlgV61QuuCjyy
         UZii3Li6tECiunnl+jSszF+5X1YlnUZ9HhEU5Y2gyKLjYGcWaXw8AL7xRlLjh8K9LC+M
         +4maoZpwsGzHA2FVcxyn/f9WCz+4CdQ3r6f4kt8mNoAuN+GCdEjFEkFTERnznC8Yl17X
         LuJl09dTX5S6Z5iWPREoG5DbkdNx09ZLnW0hEfhE+vSTnWXUawIhyV5Rq8wX2xSQGD2o
         HtZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=EsmmWshqcRWEmqQF2VjxxyMLQRhse2D1gTlLi8+uzDk=;
        b=co2MPdxS2NYgbewPN0EUl+L5yydIua2nKlB3+oeYgTGZ0zPlgzdo1D56PN5tcyFvKJ
         4vQHsr+e07uCJbEdCSGdtklVDouMTgTQdtdaYKPMBm3Zs+ScDOdognLMaFA+ASCCXylm
         TDll1E0DA0d4uH/K2f2ukCOmRNuR3TqlWHvowtNnngrGDANJ08NgjB1HOtpianM6R7XH
         xC2TrdoRkqloWnkxhmZl/ZTkaOY4WBWv40FlvE856y14PP0EDn1EyRNJI89itn6xiasQ
         pu+nMJUwSicPHDF0/5sHMtnmX3CQAVlJGz7/dVbGjBF/F6xPGUbelDowCcNNgTXRByq6
         vXmg==
X-Gm-Message-State: AOAM532pMRFyhTXXnLPAfz6ZVLbUkWDOV8kD/yEfGcqduJjOczZzFvcV
	3kseNt3Fkh0dv4TZJ+euQ82rhIJlYDA=
X-Google-Smtp-Source: ABdhPJx8SGIw/Qab2uy5z2TUM5bpuiWzlOzdu1ih68KEF3FIN3G+q7L3S9fsjy63kr+jyeFlOUjLhQ==
X-Received: by 2002:a17:907:96a2:: with SMTP id hd34mr3270175ejc.113.1630663721926;
        Fri, 03 Sep 2021 03:08:41 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Date: Fri,  3 Sep 2021 13:08:29 +0300
Message-Id: <20210903100831.177748-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Take into account guest's BAR view and program its p2m accordingly:
gfn is guest's view of the BAR and mfn is the physical BAR value as set
up by the host bridge in the hardware domain.
This way hardware doamin sees physical BAR values and guest sees
emulated ones.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 7f54199a3894..7416ef1e1e06 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -30,6 +30,10 @@
 
 struct map_data {
     struct domain *d;
+    /* Start address of the BAR as seen by the guest. */
+    gfn_t start_gfn;
+    /* Physical start address of the BAR. */
+    mfn_t start_mfn;
     bool map;
 };
 
@@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
                      unsigned long *c)
 {
     const struct map_data *map = data;
+    gfn_t start_gfn;
     int rc;
 
     for ( ; ; )
     {
         unsigned long size = e - s + 1;
 
+        /*
+         * Any BAR may have holes in its memory we want to map, e.g.
+         * we don't want to map MSI regions which may be a part of that BAR,
+         * e.g. when a single BAR is used for both MMIO and MSI.
+         * In this case MSI regions are subtracted from the mapping, but
+         * map->start_gfn still points to the very beginning of the BAR.
+         * So if there is a hole present then we need to adjust start_gfn
+         * to reflect the fact of that substraction.
+         */
+        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
+
+        printk(XENLOG_G_DEBUG
+               "%smap [%lx, %lx] -> %#"PRI_gfn" for d%d\n",
+               map->map ? "" : "un", s, e, gfn_x(start_gfn),
+               map->d->domain_id);
         /*
          * ARM TODOs:
          * - On ARM whether the memory is prefetchable or not should be passed
@@ -52,8 +72,10 @@ static int map_range(unsigned long s, unsigned long e, void *data,
          * - {un}map_mmio_regions doesn't support preemption.
          */
 
-        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
-                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
+        rc = map->map ? map_mmio_regions(map->d, start_gfn,
+                                         size, _mfn(s))
+                      : unmap_mmio_regions(map->d, start_gfn,
+                                           size, _mfn(s));
         if ( rc == 0 )
         {
             *c += size;
@@ -69,6 +91,7 @@ static int map_range(unsigned long s, unsigned long e, void *data,
         ASSERT(rc < size);
         *c += rc;
         s += rc;
+        gfn_add(map->start_gfn, rc);
         if ( general_preempt_check() )
                 return -ERESTART;
     }
@@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
             if ( !bar->mem )
                 continue;
 
+            data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ?
+                _gfn(PFN_DOWN(bar->addr)) :
+                _gfn(PFN_DOWN(bar->guest_addr));
+            data.start_mfn = _mfn(PFN_DOWN(bar->addr));
             rc = rangeset_consume_ranges(bar->mem, map_range, &data);
 
             if ( rc == -ERESTART )
@@ -194,6 +221,10 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
         if ( !bar->mem )
             continue;
 
+        data.start_gfn = is_hardware_domain(d) ?
+            _gfn(PFN_DOWN(bar->addr)) :
+            _gfn(PFN_DOWN(bar->guest_addr));
+        data.start_mfn = _mfn(PFN_DOWN(bar->addr));
         while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
                                               &data)) == -ERESTART )
             process_pending_softirqs();
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:09:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:09:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177992.323863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68L-0007xj-Il; Fri, 03 Sep 2021 10:09:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177992.323863; Fri, 03 Sep 2021 10:09:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68L-0007xZ-EE; Fri, 03 Sep 2021 10:09:17 +0000
Received: by outflank-mailman (input) for mailman id 177992;
 Fri, 03 Sep 2021 10:09:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM68K-0004Cr-An
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:16 +0000
Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 81dc2936-e1af-40c8-8a27-a24992fd589f;
 Fri, 03 Sep 2021 10:08:44 +0000 (UTC)
Received: by mail-ej1-x635.google.com with SMTP id mf2so10990948ejb.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:43 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81dc2936-e1af-40c8-8a27-a24992fd589f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=9oZWyDoHrlYE4f9O9pDC/GUYZqq2tbn79VvRAgG5sU8=;
        b=G0v5SVgVhnp27fzhTFh/t3t6T3ZG3N3FQIFEBpPCV7HbEmNwdwHCsDna9Ka/KxV5FU
         DxTwEW485P0l8KGipmtwsR4PUeZba5wv3+Fu/ntpk5rL4UwSUtnBWGZYnBKUOM42NxuJ
         GFgt36WnRDm+PUd1QAGp4A2cQ2ZxqVUvShjebP00K+B1HTsHQQkU3uLmPWxgKL0AJPtb
         aznDETYoKDAewvu3eFK5aeZmP/KclS4BjcmZC+3sJHmxn1JpWg7lOPN/kWaaXmSgMb8e
         JqOVncQSXD/xA4F426v3mWQvQi+W2MGFum99JYmvYKVxmw6AeK1htZrltbHDiaSMYs2Q
         MMmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=9oZWyDoHrlYE4f9O9pDC/GUYZqq2tbn79VvRAgG5sU8=;
        b=Ul5ha2HbBHeFKU56A9FmD89nbB41SnxF5p4NO3s0lTyS/i2jSY94clHHGBlMRvu5Pb
         R+FZdRi1AssRf/R98fuBSuQbzexSwpDKH5D6WeWoNYBhRsbKzAFeAQ2dCCHVS3kW48Yr
         oh0uzN+gzfLxTuVdhpGJruzP39PZlHEXUzNy8YJk85Mh4PVoRco/Nz+SUyOGOaCCJVj/
         avBvwXftKBd0+zzkojEUvWHA+q9DBLrqWulwhrs7i+3wKSlOkmleWU4HyPUvssQCH+wi
         XcGMc3zMlannMfrlUBiW+YDS9TE9i+q50PJ0rEFPBNLxR91oYmCGOxEN986gTtmg2hya
         f8Kg==
X-Gm-Message-State: AOAM530t0rSAulO2Tr5mbpKDyYH5bn482wOLwLHWZUUq6kRc6muezO5m
	b5RsjRZOtucJLIyYUMYBWvAf0AVt3G8=
X-Google-Smtp-Source: ABdhPJwC0I+pj81VPq5Z8aniIDBjWrsyNx1aYeEbgRJViZP4d+n9MV4hh4STMk34EUBwp0NwdeCIrw==
X-Received: by 2002:a17:906:748e:: with SMTP id e14mr3277798ejl.152.1630663722931;
        Fri, 03 Sep 2021 03:08:42 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 8/9] vpci/header: Reset the command register when adding devices
Date: Fri,  3 Sep 2021 13:08:30 +0300
Message-Id: <20210903100831.177748-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reset the command register when passing through a PCI device:
it is possible that when passing through a PCI device its memory
decoding bits in the command register are already set. Thus, a
guest OS may not write to the command register to update memory
decoding, so guest mappings (guest's view of the BARs) are
left not updated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 7416ef1e1e06..dac973368b1e 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -811,6 +811,16 @@ int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
         gprintk(XENLOG_ERR,
             "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
             d->domain_id);
+
+    /*
+     * Reset the command register: it is possible that when passing
+     * through a PCI device its memory decoding bits in the command
+     * register are already set. Thus, a guest OS may not write to the
+     * command register to update memory decoding, so guest mappings
+     * (guest's view of the BARs) are left not updated.
+     */
+    pci_conf_write16(pdev->sbdf, PCI_COMMAND, 0);
+
     return rc;
 }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:09:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:09:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.177997.323874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68Q-00006V-RH; Fri, 03 Sep 2021 10:09:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 177997.323874; Fri, 03 Sep 2021 10:09:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM68Q-00006O-OD; Fri, 03 Sep 2021 10:09:22 +0000
Received: by outflank-mailman (input) for mailman id 177997;
 Fri, 03 Sep 2021 10:09:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=93bQ=NZ=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mM68P-0004Cr-Aw
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:21 +0000
Received: from mail-ej1-x62a.google.com (unknown [2a00:1450:4864:20::62a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cfd41a6e-29ed-4f0d-93f2-6a4adb13b1b1;
 Fri, 03 Sep 2021 10:08:45 +0000 (UTC)
Received: by mail-ej1-x62a.google.com with SMTP id jg16so7877055ejc.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 03:08:45 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 03:08:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfd41a6e-29ed-4f0d-93f2-6a4adb13b1b1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=hMt74Ggq0zLuhF8LFxm6cOM3G4Wenq/IlhMP/O3Mgu8=;
        b=V1tudX9W4mPKN2zjexbubTP4JGF55U3CiWlqiVVH/aom0cJVJ9DzTQIKOt0DiEqdeB
         gwJsiJ7+KLjAsmewkiv3WjPx0NljKuCC6KIu8e1A/0EQb3tPwt+Iw9O6ttCu1nEbMen9
         dPfIoN8NWovGilGVHpkSunNV38UJxPyiT5SWc9gLFIWuF7vISABGFOf/CkkqLy5U8eBT
         csUbFMOjSPYoczhbwliEbPFTcQ1pEyU+dcCSuBHQlOK/0Jns8dCF/muXfodFaHpq4rue
         NAmSsaWkuUYPr1nJ+O/wEdrM+g5lWOlPDEU6Ak9vRyYXvYs0OI0MpwfvQwziA7gVjDds
         pMxQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=hMt74Ggq0zLuhF8LFxm6cOM3G4Wenq/IlhMP/O3Mgu8=;
        b=rqp8P+819hb+Nc5KV1bfSgDPds2Qx4iqwmg8oFFbWh4ZqNIM6qVDcOCq7h6RawDZhd
         aEE62/tgVWnntBp55OnxsXazmB/MjUwRkpd4jY6duR1cEBqH3vVZuR+hJgW7BZsj4fER
         rmfXHct7z2ALSi7ZPhVn7iUoSjIX/debzlDvZ2RdKGyAlocz99WlZvuN7w0EVnGyLNw7
         S0R1n7VoXVIWzLqA9KtzcLpg/U1QkTQR56MuN7H3uP7CY1Xekbfi1kTc/C+edWW7mscV
         HowfSkqgG+Tn62lUY4V9ElvWTpBuESqgoibah25qOZY/qr4FkS5erpQHUvtVo+fozXUP
         CLfA==
X-Gm-Message-State: AOAM533PnJkO9S4kWSa0EgNTrODTxnqnnkoX83jVc6wYCeREriRFvzOK
	Ix1RSv1WQxCWpiQhuhKpoouQfIiBlUU=
X-Google-Smtp-Source: ABdhPJypiSxpzvy084GcT4yZmHBhr6SMk1ebVuiTgJuY67izy5QHVzN9Q+8vXBtuc3hYUclLgUGWEQ==
X-Received: by 2002:a17:906:e51:: with SMTP id q17mr3408604eji.76.1630663724006;
        Fri, 03 Sep 2021 03:08:44 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 9/9] vpci/header: Use pdev's domain instead of vCPU
Date: Fri,  3 Sep 2021 13:08:31 +0300
Message-Id: <20210903100831.177748-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Rahul Singh <rahul.singh@arm.com>

Fixes: 9c244fdef7e7 ("vpci: add header handlers")

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index dac973368b1e..688c69acbc23 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -157,7 +157,7 @@ bool vpci_process_pending(struct vcpu *v)
     if ( v->vpci.num_mem_ranges )
     {
         struct map_data data = {
-            .d = v->domain,
+            .d = v->vpci.pdev->domain,
             .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
         };
         struct pci_dev *pdev = v->vpci.pdev;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:27:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:27:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178035.323885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM6PR-0004BE-BP; Fri, 03 Sep 2021 10:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178035.323885; Fri, 03 Sep 2021 10:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM6PR-0004B7-84; Fri, 03 Sep 2021 10:26:57 +0000
Received: by outflank-mailman (input) for mailman id 178035;
 Fri, 03 Sep 2021 10:26:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FOYs=NZ=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mM6PQ-0004B1-IX
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:26:56 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a07fa6b6-d2eb-4ce5-aab4-a59e6090c514;
 Fri, 03 Sep 2021 10:26:55 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id AAE21222EF;
 Fri,  3 Sep 2021 10:26:54 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 4A9FC136DE;
 Fri,  3 Sep 2021 10:26:54 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id EnE/EG74MWGiDQAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 03 Sep 2021 10:26:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a07fa6b6-d2eb-4ce5-aab4-a59e6090c514
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630664814; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=XQ58Ucgkhc17h4IveJWXEz+P+uprNztu0MOEAIRPeBw=;
	b=oVz4x9WHA33YqEVhbNS3VUYbt/b23KgSR3YNXd+Ixq2RqytkH0aOz6hzy6TE+yUW95jBf8
	liDqdPwCXB64Gy5XsrKlZzYEfqd7kCGOIefuVWzRVJkm0KsDstcrF3PidyL/zgHgtiK223
	dlbt9ukmB3542BUT8UVsJq/q0wqR7dk=
Subject: Re: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-9-andr2000@gmail.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <e154c91d-bd24-9058-81d0-79e2e321140f@suse.com>
Date: Fri, 3 Sep 2021 12:26:53 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210903083347.131786-9-andr2000@gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="XFWj5qjpdEL7JHWD4ryS1CqPu61GJ7zlH"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--XFWj5qjpdEL7JHWD4ryS1CqPu61GJ7zlH
Content-Type: multipart/mixed; boundary="ARBrSBYpYYffIfVLBYOeH4vgLXjV1LrMR";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>
Message-ID: <e154c91d-bd24-9058-81d0-79e2e321140f@suse.com>
Subject: Re: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-9-andr2000@gmail.com>
In-Reply-To: <20210903083347.131786-9-andr2000@gmail.com>

--ARBrSBYpYYffIfVLBYOeH4vgLXjV1LrMR
Content-Type: multipart/mixed;
 boundary="------------E7B88EB43DA3CCAC89924545"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------E7B88EB43DA3CCAC89924545
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 03.09.21 10:33, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>=20
> Arm's PCI passthrough implementation doesn't support legacy interrupts,=

> but MSI/MSI-X. This can be the case for other platforms too.
> For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
> it to the CFLAGS and compile the relevant code in the toolstack only if=

> applicable.
>=20
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.co=
m>
> Cc: Ian Jackson <iwj@xenproject.org>
> Cc: Juergen Gross <jgross@suse.com>
> ---
>   tools/libs/light/Makefile    |  4 ++++
>   tools/libs/light/libxl_pci.c | 13 +++++++++++++
>   2 files changed, 17 insertions(+)
>=20
> diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
> index 7d8c51d49242..bd3f6be2a183 100644
> --- a/tools/libs/light/Makefile
> +++ b/tools/libs/light/Makefile
> @@ -46,6 +46,10 @@ CFLAGS +=3D -Wno-format-zero-length -Wmissing-declar=
ations \
>   	-Wno-declaration-after-statement -Wformat-nonliteral
>   CFLAGS +=3D -I.
>  =20
> +ifeq ($(CONFIG_X86),y)
> +CFLAGS +=3D -DCONFIG_PCI_SUPP_LEGACY_IRQ
> +endif
> +
>   SRCS-$(CONFIG_X86) +=3D libxl_cpuid.c
>   SRCS-$(CONFIG_X86) +=3D libxl_x86.c
>   SRCS-$(CONFIG_X86) +=3D libxl_psr.c
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.=
c
> index 59f3686fc85e..cd4fea46c3f7 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1434,6 +1434,7 @@ static void pci_add_dm_done(libxl__egc *egc,
>           }
>       }
>       fclose(f);
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ

Why #ifndef? Shouldn't this be #ifdef (same below multiple times)?


Juergen

--------------E7B88EB43DA3CCAC89924545
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------E7B88EB43DA3CCAC89924545--

--ARBrSBYpYYffIfVLBYOeH4vgLXjV1LrMR--

--XFWj5qjpdEL7JHWD4ryS1CqPu61GJ7zlH
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmEx+G0FAwAAAAAACgkQsN6d1ii/Ey+Q
1AgAiDNRCWXbfWqmsG3dAif15NqTb2IyU0f4CIK6LL5z+EEDoVnaa1pqds2Cj5xfuh4M/cdzJDKx
XV6vrhYvW/tUIVZXMzfg8T5GNGS+yJabgyN7SDYY3LR6WO6sbd2QVS24MxGyUrBvOFarTup1ihNN
4O1ufdl7guKej0CJmWrBz1XDj/KiCQhKyDB8cwK5lYOZkcJwjtMaj9NsfZJb1sOY2vnRo361ogTC
kV/dAlYZ2XdzU7Dqc8biLZ6uLqh+56169UtHCSgJXE+64oySto4ljAL2uUZySMwrX6qSj6jKzl38
a3Wpu45mmpd43xFu8YZUYmouhcmZ+6NtljDqKVhbPA==
=KWN7
-----END PGP SIGNATURE-----

--XFWj5qjpdEL7JHWD4ryS1CqPu61GJ7zlH--


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 10:31:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 10:31:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178041.323895 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM6TV-0005cD-T7; Fri, 03 Sep 2021 10:31:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178041.323895; Fri, 03 Sep 2021 10:31:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM6TV-0005c6-Q3; Fri, 03 Sep 2021 10:31:09 +0000
Received: by outflank-mailman (input) for mailman id 178041;
 Fri, 03 Sep 2021 10:31:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Xv6v=NZ=epam.com=prvs=9880a75e98=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mM6TU-0005bM-OD
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:31:08 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6acf58b6-7e27-4308-a979-c4927b2545f1;
 Fri, 03 Sep 2021 10:31:05 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 183AIQtC008940; 
 Fri, 3 Sep 2021 10:31:02 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0a-0039f301.pphosted.com with ESMTP id 3auhu681em-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 03 Sep 2021 10:31:02 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6881.eurprd03.prod.outlook.com (2603:10a6:20b:286::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Fri, 3 Sep
 2021 10:31:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8%5]) with mapi id 15.20.4478.022; Fri, 3 Sep 2021
 10:31:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6acf58b6-7e27-4308-a979-c4927b2545f1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=co8f4AWKsRTST6P4ipuy/UNZ0+L9aX9mgokwvNy1uK9hBCyb+xBuVoqEjpSWeIULAsbASidG+1nQfqA4nm1rQrFEUk8OmEsdgv7sb+O4rgQrJsqJ9o0R8TsA8CWW8AO+hKzj+FRbac14jsitnV1OsQlI3pgOQWvJ0fCH18+PKCbxT6ncDjNtRzt5OTtYcX07M/vdhiEVhZXx77j/WXLb6c9Y0LX22OSuWWzOAuPNnok/Js0Cqe0xu7uHlfM+Rc0810vHVphnBOT6wMr2io/di5plrrG5PU+pbGeteKf4f1f2G5lG1MAuLjAhte5ju5nM5poDfox1c7Sw67db1avQkQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=I1JrNBXFqVlvMtY1Qn3q/f0qiRWmgN9ym+/sXuGPkl8=;
 b=FQRcp0uWYrbOpcOdTD8BrEIn1Fbh4GoQKVerphzb1pk6uU/wMz99+ELzJENI1WRXB45lqWy27STwo6HBqieNRdmsVYc2tMZOl9SfNYfVFxuMBL29KKYDAZWMluWufCDZ9BzSLOYx8DStaYp+YBz/e0GiBv6sqQhMx4ww6eMVqwII3vXj8rCD+MhEfmDmJyIZ5Hn27XLJQmVk/kEmm/xvhnHVoHhssP3qjPdAr0rt3aNr87kWIXKTQ99nxwWPduu80uW6GTxELnx7WTWq3Yo4diPyxZkh/eZT/1ijNoN41bV2nFfDWoWjHI5TqsTR+DXvSAtALx/IR/gnwwnZno9gig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=I1JrNBXFqVlvMtY1Qn3q/f0qiRWmgN9ym+/sXuGPkl8=;
 b=04ETNyi+jyv0gOdhh60UZXJlTKT3pz7FAgep46b8OnlZPgJqQfsvx412iJ7g4WPqopKlxu5hXnUEaJCfeE3TOEc+Vn28HLERwzEp5b3j1PZ75Lv+0lK+hsPoMBvGWaPjW7eQrLXBLtzPlq7jhNWQuoUwhIKdSJCCb2vkNf4xQq8aEaCbVxcnYYiskwW6MyiuBAEzXf7/0BYNFqbHo3Mtx5J6psM21Jogt/tMgM3f9IaG3kGe87j5WM5Xem0ISgdRsiADxaruAXdtYLbhdUafV9SObDjqdz32ynbsatZ8WgD40SWj2oCSJ67I0R7X8yOKFEEbGNs9gXQNYQzW/AHOmQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>, Ian Jackson
	<iwj@xenproject.org>
Subject: Re: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Topic: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Index: AQHXoJ50aOdU36kT/0mQrZPJ6sbBAauSGzCAgAABJYA=
Date: Fri, 3 Sep 2021 10:30:59 +0000
Message-ID: <f285da98-ce0e-c120-80fc-26940104c1c6@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-9-andr2000@gmail.com>
 <e154c91d-bd24-9058-81d0-79e2e321140f@suse.com>
In-Reply-To: <e154c91d-bd24-9058-81d0-79e2e321140f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 111e0924-0885-4e37-950a-08d96ec5ec3e
x-ms-traffictypediagnostic: AM9PR03MB6881:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB6881CF3B4263A56A9AB68658E7CF9@AM9PR03MB6881.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 4JoyLQJyQVy8el1YBzscb/9jdMy50fcKhWEPxscJbb+Eh8q0dgIZjzIYUIs6nrobEJDlX18wBu4Wzoi1zhvfRIaGn2I6ezD/wDZCu3+O9/ngTWcv1eSFu1nUpwOEPX1flScpcPnkkuFIlYazs836NoOeEAkSHGz7nSf0p/BnIgICpK07vXgJScxG17PqJJCFlsn7J7Eq7Y7dyQ+UbVW9p7cbWgN8J/0rljM7SE7D/bfHXzCKsYLNYB+kEl5xjqurbOiTC6qI+Tnd8RTjv5N33f9x70BEgFhe4jEfUFMJsOsd7z/NaMCed4PColDl/ZFMhzhaDXzmd7+pZk2o0ubooYDcLD951ycQb8u54pWiDuCZGfEdMtvFIsAYMTzFyFR+H0roU+91dMZ99Im8R5Qd8eniGONcZOC/X6ZWnOpjBuYLf4AQkeGKMxEjGdQEA4pYYVkxbP9ZncA66lNZ3nMz0qQDDcgQp63PfoATcGucNjSjxu6egf0U71Y43bBL9etZijBahk29mTyHIbcMgWnFd14WCKdwfxelt+VbOMASybYDF2Vr6orTUqIDl03BAQjpmQhHRpLPqcoSI4kLKJlB+mhpg5/cgNKIKzJIg4xCAQeZB0i05woi+ywC/rlQ/rVLHIGeSKtODhT1amhXoN6XYgye3E0fULrgbaFSluCPjKGAQLMJpXxRwVODCtuih+7pTPHABgE/5mkPZ3gzmkLhC9b4lb8RbnJEqtJZaE2IKnsZ/dYg0pDfoc5Z4PXJsHnC
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(136003)(346002)(366004)(316002)(110136005)(54906003)(6486002)(71200400001)(6512007)(83380400001)(86362001)(5660300002)(2906002)(2616005)(478600001)(38100700002)(66476007)(122000001)(66556008)(64756008)(66446008)(55236004)(4326008)(8936002)(186003)(26005)(76116006)(31686004)(31696002)(38070700005)(8676002)(6506007)(53546011)(66946007)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?alU3aTU4REQvNkdvaEs5ajNZN0cySG5UNFZSdWJTelZxL1VqRXc3alhiTWNN?=
 =?utf-8?B?dTRIWVJlVEtWYllqeEZhU0dOSC8vaVpCbXlwL0ZKYzllVW0rQ2xlS29uV2Er?=
 =?utf-8?B?cWh3RDBINjdnbE82cnV3WkFjSkxBQVNKdnN4MkJ1VUxpeksycnY5TnZQcS9U?=
 =?utf-8?B?WDNVWmw4S0JTa0oyTzR6Mll5TnM0Wko5NDI1S2NoMFBweHQ0NDF4RGpPelB2?=
 =?utf-8?B?ZHpmK0pxYzdOZDRySFVxSE8vb01ETmNIak54WHFjWWlaV240eHE0MFNaZGJv?=
 =?utf-8?B?T3BEVTlWOC9mb0tUT29EcUVBZ0Q3VlF0L3VVOEpuTmFtWEMxVEl4aHR2M1dy?=
 =?utf-8?B?dXVpT3lvSUFwVVE0OWlYN2ZiaHNHdkkxeEpUOGM3aStrSkQxWGovdUozWHZw?=
 =?utf-8?B?NlNOazJFQnJDSG0rTzlWcHMrbHV6S2JMVTE2WEYxNlhVK3FmNi9lZllRdkF1?=
 =?utf-8?B?aWtYMnl2bGlma2owbmx5RWVOSVNLQWp4RmtpZ1h0MkdLRzJUSE1MTHJJaEpH?=
 =?utf-8?B?QndmSXFHSm14TklDK1ZzNnNNcFhMQ3RjWmptY1RkRm5xK2dQZFJOeUN2TUtH?=
 =?utf-8?B?Wlk3YzRHZFFZcVdndEVLdGszSm5qKzRpTEJXcC80RHowdFRNY3g1QnBxR1Va?=
 =?utf-8?B?TUVzL3Zsek9tM0s3eEtPV0RzbjRxbmJRSUtLKzJpTDN4N2prbUVNeHI0Ylli?=
 =?utf-8?B?d3FDUjgvem5LK2dveVlGZDBENzZiODNSOHFIemNjOVc1ZFdGKzJIN0hVZk5H?=
 =?utf-8?B?d1FFbUVHVnd1M2txMUZjenF1RGpsWDJuOUxkNm1SK3JXVDE4cnBKOU9ncUta?=
 =?utf-8?B?N3ozQnJ5ZVF5cHBSRnpyZzNQMGE0dG94bXZUT2lxcGl1cnVJOVlHVXB4YmJN?=
 =?utf-8?B?UjRXejZSUTc2N2dqT0FET2Vic2djMmRrNGg1dUlDYmVud1ZPRGJJNERNSnhi?=
 =?utf-8?B?blNZOWFpUTM4bXZ5ZDJRZk45c0h3bUVKWTZEbVU3djA3VTRSNUtCaXFoVFRY?=
 =?utf-8?B?dUJuQ2tUWmtZMHN6L0taTCtDY0VXNzVvbFFrbnJnUnNDRG8wL1JPYW1CL1RB?=
 =?utf-8?B?TFN0R2xISVlqWUNCeDQyUG9zVDFSKytkQnpSSkErQVM0YjNsajF1dFBVTEZH?=
 =?utf-8?B?bUthK2lqQVNZSHVLeVUwSUQ3TjF5TkpPa2VHQ3R1SU0rMzllTG96c3ptcWp2?=
 =?utf-8?B?VTdERHQ4ZnpWZlVWNzl6UFRodkdqQXBSc0Q1SnIxdldZbHU5SGZwL0VDeWJJ?=
 =?utf-8?B?WUJQMlVYZGJqUmhVb1VOeEtHRnl5V0hubzcvZ1lpdVd1VGplTzhJVC9ZVVpZ?=
 =?utf-8?B?TUtncjJoQnExRWkwcWRhRFowUDQvUUpweXYrRTlEL05KUlRMWGMxWW9FRDdE?=
 =?utf-8?B?bWgvQmRvN0JoRjJGdmZyOElPNEs0N0ppZVRYd2R0WmR2a3RCZ0ZJTU02ZTU5?=
 =?utf-8?B?NVM2TFpRVENXbzZTQVd6am9EQXhiZHdXamxlbnJibmRYc0MxVmdWYjN5Ri90?=
 =?utf-8?B?QkdmcUVsVUk3NS9YYjBzeTdCUnFPRnJ5QjhaR3lBZ2E5R1RRS3pnYmFodEUw?=
 =?utf-8?B?am9peGlUWnU4Y1lOQUVkVXpkVVQwNGpLN2xmd21FMWxkTnlJN0g3UjFsNlRL?=
 =?utf-8?B?WXFIdER4N3pZaEY0QVh3OUhZWGJuME5aQjdaQTQ2Y0NES3RuNGxGZDFNbnla?=
 =?utf-8?B?ZWRtbTRSbFlRVkZuZ1BCWitOT00yOUF0QWpzSUdXZUQ2T1BYNzZxUUhwK2g3?=
 =?utf-8?B?UHNOL2c1UXFQcm9lNUhwVjFxV29YR3FibkhOc0FyRTdxZHlFL2JIZTkyVUx0?=
 =?utf-8?B?bGszY1NzTWZFUWhmdmVHZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <FB986ED1967C164F91B707BA706D7C6A@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 111e0924-0885-4e37-950a-08d96ec5ec3e
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Sep 2021 10:30:59.9717
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dwTTRvMBwtG6wkkUNOPbnQOL1LS6cHGgolcGw9Ye+BxA8vjuMvqMfGpIyESsDk4/SZ1ianZ2LkJTgklmyq9IH9rY+J770KcKDHftkhwE9w/5aypZyLLTDrwe3IiwmUfL
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6881
X-Proofpoint-ORIG-GUID: -4YD1Sw6CRSks1MY1lcyDuQbmyM8uOXu
X-Proofpoint-GUID: -4YD1Sw6CRSks1MY1lcyDuQbmyM8uOXu
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-03_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0
 clxscore=1011 mlxscore=0 lowpriorityscore=0 spamscore=0 mlxlogscore=999
 malwarescore=0 suspectscore=0 bulkscore=0 priorityscore=1501 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109030064

SGVsbG8sIEp1ZXJnZW4hDQoNCk9uIDAzLjA5LjIxIDEzOjI2LCBKdWVyZ2VuIEdyb3NzIHdyb3Rl
Og0KPiBPbiAwMy4wOS4yMSAxMDozMywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+
PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29A
ZXBhbS5jb20+DQo+Pg0KPj4gQXJtJ3MgUENJIHBhc3N0aHJvdWdoIGltcGxlbWVudGF0aW9uIGRv
ZXNuJ3Qgc3VwcG9ydCBsZWdhY3kgaW50ZXJydXB0cywNCj4+IGJ1dCBNU0kvTVNJLVguIFRoaXMg
Y2FuIGJlIHRoZSBjYXNlIGZvciBvdGhlciBwbGF0Zm9ybXMgdG9vLg0KPj4gRm9yIHRoYXQgcmVh
c29uIGludHJvZHVjZSBhIG5ldyBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUSBhbmQgYWRkDQo+
PiBpdCB0byB0aGUgQ0ZMQUdTIGFuZCBjb21waWxlIHRoZSByZWxldmFudCBjb2RlIGluIHRoZSB0
b29sc3RhY2sgb25seSBpZg0KPj4gYXBwbGljYWJsZS4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBP
bGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+
DQo+PiBDYzogSWFuIEphY2tzb24gPGl3akB4ZW5wcm9qZWN0Lm9yZz4NCj4+IENjOiBKdWVyZ2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+DQo+PiAtLS0NCj4+IMKgIHRvb2xzL2xpYnMvbGlnaHQv
TWFrZWZpbGXCoMKgwqAgfMKgIDQgKysrKw0KPj4gwqAgdG9vbHMvbGlicy9saWdodC9saWJ4bF9w
Y2kuYyB8IDEzICsrKysrKysrKysrKysNCj4+IMKgIDIgZmlsZXMgY2hhbmdlZCwgMTcgaW5zZXJ0
aW9ucygrKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS90b29scy9saWJzL2xpZ2h0L01ha2VmaWxlIGIv
dG9vbHMvbGlicy9saWdodC9NYWtlZmlsZQ0KPj4gaW5kZXggN2Q4YzUxZDQ5MjQyLi5iZDNmNmJl
MmExODMgMTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L01ha2VmaWxlDQo+PiArKysg
Yi90b29scy9saWJzL2xpZ2h0L01ha2VmaWxlDQo+PiBAQCAtNDYsNiArNDYsMTAgQEAgQ0ZMQUdT
ICs9IC1Xbm8tZm9ybWF0LXplcm8tbGVuZ3RoIC1XbWlzc2luZy1kZWNsYXJhdGlvbnMgXA0KPj4g
wqDCoMKgwqDCoCAtV25vLWRlY2xhcmF0aW9uLWFmdGVyLXN0YXRlbWVudCAtV2Zvcm1hdC1ub25s
aXRlcmFsDQo+PiDCoCBDRkxBR1MgKz0gLUkuDQo+PiDCoCAraWZlcSAoJChDT05GSUdfWDg2KSx5
KQ0KPj4gK0NGTEFHUyArPSAtRENPTkZJR19QQ0lfU1VQUF9MRUdBQ1lfSVJRDQo+PiArZW5kaWYN
Cj4+ICsNCj4+IMKgIFNSQ1MtJChDT05GSUdfWDg2KSArPSBsaWJ4bF9jcHVpZC5jDQo+PiDCoCBT
UkNTLSQoQ09ORklHX1g4NikgKz0gbGlieGxfeDg2LmMNCj4+IMKgIFNSQ1MtJChDT05GSUdfWDg2
KSArPSBsaWJ4bF9wc3IuYw0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxf
cGNpLmMgYi90b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jDQo+PiBpbmRleCA1OWYzNjg2ZmM4
NWUuLmNkNGZlYTQ2YzNmNyAxMDA2NDQNCj4+IC0tLSBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxf
cGNpLmMNCj4+ICsrKyBiL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+IEBAIC0xNDM0
LDYgKzE0MzQsNyBAQCBzdGF0aWMgdm9pZCBwY2lfYWRkX2RtX2RvbmUobGlieGxfX2VnYyAqZWdj
LA0KPj4gwqDCoMKgwqDCoMKgwqDCoMKgIH0NCj4+IMKgwqDCoMKgwqAgfQ0KPj4gwqDCoMKgwqDC
oCBmY2xvc2UoZik7DQo+PiArI2lmbmRlZiBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0KPg0K
PiBXaHkgI2lmbmRlZj8gU2hvdWxkbid0IHRoaXMgYmUgI2lmZGVmIChzYW1lIGJlbG93IG11bHRp
cGxlIHRpbWVzKT8NCg0KWWVzLCB5b3UgYXJlIHJpZ2h0LiBJIGhhdmUgdG8gcmV2ZXJ0IHRoZSBs
b2dpYywgZS5nLiBzL2lmbmRlZi9pZmRlZg0KDQpPdGhlciB0aGFuIHRoYXQsIGFyZSB5b3Ugb2sg
d2l0aCBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUSBuYW1lPw0KDQpUaGFuayB5b3UgYW5kIHNv
cnJ5IGZvciB0aGUgbm9pc2UsDQoNCk9sZWtzYW5kcg0KDQo+DQo+DQo+IEp1ZXJnZW4=


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 11:22:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 11:22:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178050.323907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM7GU-0003bi-TR; Fri, 03 Sep 2021 11:21:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178050.323907; Fri, 03 Sep 2021 11:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM7GU-0003bb-PG; Fri, 03 Sep 2021 11:21:46 +0000
Received: by outflank-mailman (input) for mailman id 178050;
 Fri, 03 Sep 2021 11:21:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM7GT-0003bR-3d; Fri, 03 Sep 2021 11:21:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM7GS-000325-S1; Fri, 03 Sep 2021 11:21:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM7GS-0000v7-Hb; Fri, 03 Sep 2021 11:21:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM7GS-00016h-GI; Fri, 03 Sep 2021 11:21:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FPlSQJVroworsex5KXf59TIdvnygvElrQN7UTB9YKDI=; b=1qfzVA7SLokIXifEYtVHNcqCx2
	/QU4OlxzxEX+IcSDg6V7aM715We4ufVGkUXTVX/9yiqC37ZWxaEkmyEDrF6izzUi4C5yJGuHHGVvN
	fwqddNGKzNg3uzIr1KMyFGYWwCSfGYnLPkYQ9bTPilm+oiNukVRC8VvWdxiYMJee+smw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164786-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164786: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
X-Osstest-Versions-That:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 11:21:44 +0000

flight 164786 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164786/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164739
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164739
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164739
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164739
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164739
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164739
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164739
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164739
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164739
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164739
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164739
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618
baseline version:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618

Last test of basis   164786  2021-09-03 01:51:45 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 12:41:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 12:41:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178065.323921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM8Vh-0004Xh-0Y; Fri, 03 Sep 2021 12:41:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178065.323921; Fri, 03 Sep 2021 12:41:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM8Vg-0004Xa-TG; Fri, 03 Sep 2021 12:41:32 +0000
Received: by outflank-mailman (input) for mailman id 178065;
 Fri, 03 Sep 2021 12:41:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=etbn=NZ=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mM8Vf-0004XU-Ui
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 12:41:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9222c8ea-69bc-4cf3-8606-09a74fbeab7b;
 Fri, 03 Sep 2021 12:41:30 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2053.outbound.protection.outlook.com [104.47.5.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-LIyylY2YMj6tMTlQvMVfuw-1; Fri, 03 Sep 2021 14:41:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5903.eurprd04.prod.outlook.com (2603:10a6:803:e0::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Fri, 3 Sep
 2021 12:41:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.021; Fri, 3 Sep 2021
 12:41:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0063.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Fri, 3 Sep 2021 12:41:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9222c8ea-69bc-4cf3-8606-09a74fbeab7b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630672889;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EbrS5Qs1UIXDzktuDY0e79UHyHoqnt/wbQv5dVhWme4=;
	b=SA0v1wSI8hF4sjL2hFMBaKgX+jFl4HrmUMoL/2PToXrpOJu6rI/abwIq4cMXu/C4bnaesG
	kshKrBdhrTORfbfBf+VcQC7G3oU5hoZliqyjBa0R6RI2YfyTiFU5gJ/0a7F4oeq7+agsRs
	mkS5tY/qZ0++4rIKW6/S+RD2YV8bA1U=
X-MC-Unique: LIyylY2YMj6tMTlQvMVfuw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cjWebvd6BgjtC+0LvqsHTUHOHb8rx5hON3KQeKXN5d2vNjiYgabJWbI3XCx3ST9of4AgwHqLWepzO+7fcWLJ2coIdLG30zWqY/m5rZLB0aEM6+QeEModsU42xiDlzWSDqhU9Zq48IxzorvaEOqSkVCoHs47Qp5d4K9U22jnf/MC40AihSUJuyqmcWfd1CINvv2yMpOI4Sb02IwmXKEoAFwF2DjKZS/lB42W5DS53SIpeukvXKxRbyDAC5GBbQb86LhZBfhURcA9emgOhC3ggtI8RRovtYU1k9rOHeMTEgA3PmBD0bbGY4O2u1MQ+1fUgVoT64BlwmYnBwg+bNYSFpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EbrS5Qs1UIXDzktuDY0e79UHyHoqnt/wbQv5dVhWme4=;
 b=MtXi8j+n+Rsyn1wjfYVoO9O2Za/S3EceM7KDmDB1w6r83oMMYODtokGtdh3HC0O+duYYOOZTzleE/ZghupwPRYjRd2y4eI6v6l+ReB1G7Vz3qTFiQ4OnTotR2xx5rX2PsuLf2EbLu48ppEsfONMIvS5APETV91P+zYDZVub/71RAEZggLtRGnWwreGmGiqCxIC4uiaC+h3gDnGIKSAU0nX7Zr+F1HoSYQz86dmHR7XYjx7qRv15zCYk4H3F8VWYriaHwhFAL4Kn2LWi33S0l9wHcddGMjC1IrdIrK0Z7cKvk7qnMqXBPlBKp+6prz708+TKikx7/fI5xBP8ZIdj/jA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-6-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1b890e67-c907-6678-bc98-d810231d9b50@suse.com>
Date: Fri, 3 Sep 2021 14:41:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903083347.131786-6-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0063.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6a061db7-b876-4ee2-2881-08d96ed82517
X-MS-TrafficTypeDiagnostic: VI1PR04MB5903:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5903D145EBAD533118FC8072B3CF9@VI1PR04MB5903.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	I1mDtXKJhnyl/yZHIr8Fa6Ij7nODyJmXEahwicGcdrNx/7jHwW4g7J936zDMdCj8h8h8uysEvYiTj1NL0G/5UBAuMbuQAUms8WOwQD2Ub4PD2oZrOuiZyiGFd6bCB08mC2XtabItvKado6f1SkkQxwjtPPZN7bubbBHbO05eTFgQBq1lqx2tYtCbEeN9a5zaakJYQGqdhOmA9uHaduWxTj55azO3m1yfEOmcb17PUdsk1MOsXVdO+PAgWJuhpCUnhCA3yp3wu/FwQR/UrmplhnNxTLfWH6FT8+rQ6yGd6X0dPIrkrKFtPp/O34XViKLFP+PZ1xNVPzay/Ffgf21k3/K+LuG0BOxO+IiPBJff874c97p+RXN6mEy0+fQAISwSdaJ3E46TmQbdTc2uetgwsZPSuSDaDcP7E/O9sQSo9NXydmYsLkZnNOjmDv9huB5vt8DrrsuzMjXAXIMcLpIc9YPlUZn8QkNqyz7Lib1m9Qk9Szz+INGUSpElYLQgX4cqzArcLDZfrDXbFje/uG+aKtpQz1wZ53LOogpI8UzPjL29Ztw3myuMmafoagHi170Z1Ob7SSxTsUcFtMaart7Mh8gjxskCLqPTztS/ru12bL1zdrQwMOGdk6sk05pI9uR9+UI5XwLA1dMDlFjEgRoHWo8pn9En7nnvNF/tYEKJ2a9VHVWzusIKLMD2muXjM2t4Agl6sAxOwysWo+EvC/lCZi6qWNzZ+F5nNDjK1nk/ugM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(396003)(136003)(376002)(366004)(956004)(26005)(16576012)(7416002)(6486002)(478600001)(316002)(2616005)(6916009)(4744005)(5660300002)(4326008)(31686004)(2906002)(66556008)(186003)(66946007)(66476007)(36756003)(31696002)(86362001)(53546011)(38100700002)(8676002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDZJcHVXN2FMZGVQaWpjUlFqeW03RVgwb2ZGN2REcFE4TXRNMFczN0paRmRM?=
 =?utf-8?B?eTZZUmRGdFg3VENDK3g0SWF3VktUWERubGUzbzRyTGo3cDdMaTBpbnhIUEJS?=
 =?utf-8?B?a3pXSGJTNHFYU09VazMybm9mZk8xVFU4TVphUWFVTUV5dExtOEVYVXBZZy9N?=
 =?utf-8?B?ekJmT3JlTFRjRlpZcWFBN0xMNXZ3SVhJbDNSbTMrazMwVWxsUnhEQ2JoaHFx?=
 =?utf-8?B?RTQ5WUhBLzJWbFRXSVpBU1ZrRXZTVkVGeHdob3N3UGhIRTFOTzJVNllPa016?=
 =?utf-8?B?WWZzWllZeU9RN284TEswZkpsK1JSdXNhRWRGTnVJZ3VTcEtqdlpqOWRVdU1O?=
 =?utf-8?B?OG5CY1BXWlUrK1JVZXF5T2NpSzhqeUxhLzd1R3gzR09zSFQ1Q3UxOWhXZ2lO?=
 =?utf-8?B?RnZ5c0dyR2lFWG94VVpoRDRnTG1IcEhNWDJNcnY2eUoyM2dHZFNaSEt3bit3?=
 =?utf-8?B?d3BCTTlRajdEL2ZXcUlGVU1vakppTTZSUTVHS0RMa0lVT0F2Z2hHZjJKRTBW?=
 =?utf-8?B?YVZIOVp5MXFDZitrbnBZV3pjQldNQXJkWWdKUVpMSDAwU0h6RWNJU3k5QWZw?=
 =?utf-8?B?aFM2b2dpenROUE5wU2g4cW02Tm5SZTFvak9lbVFybklCMkI0ZVp3djMrT2Np?=
 =?utf-8?B?bFhvVis1SGU2akFwdDNzVXRsVitHeWxkQmg4amlGRVJuNkVZWDNwNWlLeldQ?=
 =?utf-8?B?eC9RUjFPSlNOMHg2dkxYMVRVZjhSLzQxV0V0Q2tRTCtMbWNYcE1WcWxDY1RJ?=
 =?utf-8?B?NWlpYTFCa2IzWnF5cGkyVXhaakRPUU80Y1pFWjZaRjJETjNlTlljam1tWkkv?=
 =?utf-8?B?d0wzRnJwWVdCc0FxUXRXY0R5Qnd0VnRoQmJFdjRkRGJwdUh3cUU3UkpzVHpW?=
 =?utf-8?B?NW5sRnpVL1MrT2w3MkZoM2xVZHl6UDVvMzhHV0p0MEtMcUpZakNNaFR3WDZK?=
 =?utf-8?B?VUNVdUpHZ1dsSnY4andMUXRCZ3dSak55UC9wclhoTThaZ0ZDVWtTVXpUdmVi?=
 =?utf-8?B?NnF4ektKbEN1ZlltZFJjZkpONFRjeHZISUFISllSbjFMU3FXdUlKQlE5VWZJ?=
 =?utf-8?B?dmhsVFN0YjVXeWlOeXNzcnk2ZmcwT3hVdE9Sbkx4UVZYd1lDR3Y1cUM0RnJy?=
 =?utf-8?B?UTVxKytBdzIyd0dldWEydlJzL3picjhQYVNrNm0wU21JekQ2RFhzMTdFSWMx?=
 =?utf-8?B?MzYwZHo3VGR2M0hvQUcxeTIzVGh6Y1RRang4RldhTHZoeXBMTDZaZGpkTHZZ?=
 =?utf-8?B?aHVjeVpXYkEvVThwUTFaRTdPVmhMQzBBVTFEQjkvMGxEeVFJakV3OW5qdGpz?=
 =?utf-8?B?MFQzNmI4QlRZeFZubmhGUHhRMU9xWkdrVDVUL3FMWGNqTGk4cWtRcFJOMFoz?=
 =?utf-8?B?RGl5cEZhb3FDQUllU213c2dveERJc2x3ODV6SVhpclFjVG9JaExpZitRaHg5?=
 =?utf-8?B?WEN5MU90dytQNk9GQUEzV3hQaFpqOWRrTzUzdVNoYXc0SFI2dm92YURyVXUr?=
 =?utf-8?B?cGFhK0ZsTVVlcmptRFNrSCt4c0xLZmJiMVoxdytPQ0UyekY1cUJJNDVnWHcv?=
 =?utf-8?B?R2VZREtqK3JRMDdPSVNya2JWelNqR2hIcXJmNUxibXlZcERFL0Zlby80UkVC?=
 =?utf-8?B?RDJBcXkzRDBRZ1MzWDEweUhKU2R5aFZ0Z2llNEpjNllDVHl3bEFxK0lFOC9o?=
 =?utf-8?B?b0NnSm8yeXhhcENnY29RRVg3cVN6T2Rqci95Zk1oVWw3RUcrTTdzMS9lUU10?=
 =?utf-8?Q?Q3IIIBW0jFzUo8x9h355byMsXgu0HQSr4c5hQ/7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a061db7-b876-4ee2-2881-08d96ed82517
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Sep 2021 12:41:26.6250
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: relRL+MFgHFf7TrmdR5q5a+iAf+9damWMM5MiPNccQpaZEy9KMs4NJ/zhSSmFp+1M4yQKQ5Ag36wRckubKQzMg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5903

On 03.09.2021 10:33, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -1301,6 +1301,9 @@ static int iommu_add_device(struct pci_dev *pdev)
>      if ( !is_iommu_enabled(pdev->domain) )
>          return 0;
>  
> +#ifdef CONFIG_ARM
> +    pci_to_dev(pdev)->type = DEV_PCI;
> +#endif

Why here instead of in alloc_pdev()? The field should be valid by the time
the new item gets inserted into the segment's list of devices, imo.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 13:03:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 13:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178072.323932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM8qc-0007IG-Ru; Fri, 03 Sep 2021 13:03:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178072.323932; Fri, 03 Sep 2021 13:03:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM8qc-0007I9-Nz; Fri, 03 Sep 2021 13:03:10 +0000
Received: by outflank-mailman (input) for mailman id 178072;
 Fri, 03 Sep 2021 13:03:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM8qb-0007Hz-O9; Fri, 03 Sep 2021 13:03:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM8qb-0004gd-GB; Fri, 03 Sep 2021 13:03:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM8qb-0004Vc-8W; Fri, 03 Sep 2021 13:03:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM8qb-0004IM-81; Fri, 03 Sep 2021 13:03:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nVM4epZLBzhC6myYmCTKRg5uCCc6chP9UyZBAGY6Apk=; b=AYNfOLjZzq36GwX69LaYgCuFQ6
	ycyMJknIS+ZWo9xx+KCd+eVnVWeNyIjY1cwY7wE2/xsHDoHSoVBrCgW9+ZVm632lv5XREE3j14HTy
	Dy14Q0eRtJ3Cru9OYyzLyH7hjnx78Pc2giB/bR7FX8syLbCRmSmhS1L3NShtd5lA0zuA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164794-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164794: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 13:03:09 +0000

flight 164794 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164794/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   22 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    0 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 13:26:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 13:26:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178090.323958 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9Cs-0001iX-U1; Fri, 03 Sep 2021 13:26:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178090.323958; Fri, 03 Sep 2021 13:26:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9Cs-0001iQ-Qv; Fri, 03 Sep 2021 13:26:10 +0000
Received: by outflank-mailman (input) for mailman id 178090;
 Fri, 03 Sep 2021 13:26:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Xv6v=NZ=epam.com=prvs=9880a75e98=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mM9Cr-0001iK-Aj
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 13:26:09 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7f5458c4-0cba-11ec-af16-12813bfff9fa;
 Fri, 03 Sep 2021 13:26:08 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 183DEFG3022979; 
 Fri, 3 Sep 2021 13:26:05 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2058.outbound.protection.outlook.com [104.47.14.58])
 by mx0b-0039f301.pphosted.com with ESMTP id 3aumdq01ga-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 03 Sep 2021 13:26:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5697.eurprd03.prod.outlook.com (2603:10a6:208:176::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Fri, 3 Sep
 2021 13:26:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8%5]) with mapi id 15.20.4478.022; Fri, 3 Sep 2021
 13:26:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f5458c4-0cba-11ec-af16-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Hj1sS21G0hxhYDsmoxMrwT9Qlg7scEdPifIVtS4F3i1kjyAgNjZaMfEkPZC98nywNJs5ZTNFsQ2LKnaRB7CeC366vj3wxXc7gLe4Nx3afpUKE8OWGDUHQtkLhCkz9JZI/P452D706ok2w1ABtQtNso+BWysdA2HsqpadlPPOnynj6JsiIup+0xEevir8wJDQCvBJTwW3fVtUYl+GfSdrc9dGstZGFjzhYXJKFS7Rj/dUocg8N7bN5/t/4XpWAt+Q0te/cbMQj6ByPQinaTaSUVAifRHoJ6wo5wo+WFw5LVprUEYqJzv8pAUNORGpdQYVlLrFqxZzVWfm3FlQiYxY/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6rcs18nY5JhoWpG4h295efE9YtJvCF4tu+ICLvI9ne4=;
 b=Zg1lrU1sJVm30bzA/GuYNC9PKopgFumn/Fx+rN0/sUd3fBVJEPDsvaFm/FE0cy/N8m7yL3Tmr5evelVc4EPvygDUoZeIm/e8rWYVsmQw/LrEV2zREK7cMPSiBPwiRFs+ItnUmJyOaajiEaNfjydOZAx7LYUDHFs7GNOr/1gAKjI7kQgnqR/4p+2yr6mXuW7PgvFif2p6/GgG/D5I1/lrl/984omQUhmSRVZlXLpBwUGgu5pD2gJqmlTspIS0g5wBNsoHqEZSm1Lb4sNbAQYyLdY6T1k5VSB1hBUHmdOQYSYsHnTLvecX2PvtfpFmKc0SFWWfYnoDnCmqNmQKfcwp6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6rcs18nY5JhoWpG4h295efE9YtJvCF4tu+ICLvI9ne4=;
 b=s9jGv3LBmF823BNOjmQjtyYOmsotUc2G0F1Io3nvVvh7L4LxIXnfGk43lFxT8OOgiyXAiqr9ZkkuU19V1W0GRVIYnj1qDNiJbJ/DLdjOypbPRee6G9u1BA3CsaxmmJYIG98O3V3gbQ/4AOJLauR9S7yhezMMVoS4/0YOoIUP9r7nf5Ccc1NtcAt5e+MagbfhVfZhQsohihMVqLTvOBKXTdvDG0W16t6HaWz2wM4BnZhM+OKNVDdXuZ/gIRWQTVMw04InLsni8/IJW9D2ADWWmRA+my9/i7TzcJXbD6oVC8G3zhIh6XU7E9Pdtla1MsWtsOh+cHe9U7kaQKfwdK8wVA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: AQHXoJ5xoGQK6kA9E0qeGYyDKoX2LquSQMSAgAAMdoA=
Date: Fri, 3 Sep 2021 13:26:00 +0000
Message-ID: <44a860ea-a93e-6167-25df-12ac0ae6e134@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-6-andr2000@gmail.com>
 <1b890e67-c907-6678-bc98-d810231d9b50@suse.com>
In-Reply-To: <1b890e67-c907-6678-bc98-d810231d9b50@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9a05e2d2-cb2d-4999-e15f-08d96ede5edc
x-ms-traffictypediagnostic: AM0PR03MB5697:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB56971682F0444905E6159138E7CF9@AM0PR03MB5697.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 WYBZqu6Hzj3obIrFV5DedsBYQDK7BfKPti6sZ2YpeDmiQc2rx4QfO1UsQ0mvtPgbA4JSwbdx8KL18NHgPI+eGC+ggRAwNGBjvA033fM9ptwHPAYpWvIpqufW++YCdHoWih8Ubhr84R/rZas99VwD90AjZwmZfNUCTYqoRyhbcWaC170X2+kRwfAzhYOlH7Fq5D1R30f8NlaKw/oonG+5EJPWp9vbtPh8HJ8eIuVg6utBaqNejkaXpAkQ+1WnN1Gaj+l1W2SDlqclSKBvD4rz1vcPG5vUtF/HToonl6MhclYm60b5n2UEIQV8+sW/dIHj+ZvRX/kimE89Lb191Axs+mYA7od+PeCdrvlm2aZ+aRIwcEzQT5GI6tob0rgPJf4AVn+mrQSA6nsJzu1ngVEBAocEwElI48QFWLnV+eO5Vi9ydLnbYUVnPxhiSumjCfLncAY6kew8iB05O5ciMj/X11v+Fci887qIhZdxMsRe72tmm9OqyrG02MD00FJjdfekUPNrzExGXn7kw8oELohEGW1ja337JHiKH3qDBTY6nvkets361VtR7GgqtTfXdd60DRuIz7hvoHToSMieb2OqOIEOwqvMLk9OaczmBY0c1BN+np3E3sNntrsaOpSqm84o9QoPhJI+MT0qy4Z9Q/UgtalmX/mJC9Hj5oqaFNgqCHtXlI2hjslBarFL7ZTc65RE4KL2JxUGmL096AuNdgsOORiuHnKc3AgoFzQWcgcA8ZdC+N7Fh/61NlknSshS5YdU
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(5660300002)(38100700002)(71200400001)(31696002)(8676002)(2616005)(508600001)(38070700005)(86362001)(110136005)(54906003)(2906002)(316002)(8936002)(4744005)(66476007)(66556008)(66446008)(64756008)(26005)(186003)(36756003)(53546011)(55236004)(6486002)(4326008)(6512007)(76116006)(66946007)(31686004)(91956017)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SEJQSXQvM3FtMEpQbm4rUFo1c3pNN0VrbUV3VG8rTnVaYU4rWjFVOCtDZmZJ?=
 =?utf-8?B?VCtudnVWTEdDNHJ0NmFKdGZidWk2YVpBQUpiRGU3L1U4cnZPQzBYQThkdkNM?=
 =?utf-8?B?VCs4VExibGM5eEh0bmdTVFJGSXU5dzEwTjRaaUJFZ0FWSTVBOVEyM0V6Y0J6?=
 =?utf-8?B?Q3Vud2d1d0FCTEZvV0pPekFWQ25oTUQ4UmlvbXhGMXR1N0lZSU1ZOVE1NFNB?=
 =?utf-8?B?WWJjY2dva1lBRDUwVCtWRzBUeEFXaGFOcjFmM0Y2akJjMUppMTJDUjM3V2ds?=
 =?utf-8?B?M0pRM3FsRDVFNjBaVDF6VTI0cUg4alF1SzRwMXU1cmpYMmtGZXplMy9pODNY?=
 =?utf-8?B?ZUtLT2o0MWNrVjh2SFdiNjRLTDF3TmFzSFQrK3dkcE9vSGJkRDdkQmdUcnlU?=
 =?utf-8?B?SVhRZ01oWjc4YzhMcEVqQWRsbXFmTEJRWXkwaDNGSGF2Q3ZCZmtPUVZpYWRU?=
 =?utf-8?B?TG1nWXd3Y0JuZkNtQWF3Rk1LeXlaNitKbWt6dHNDKzh6d1FwZzJnem16SHpk?=
 =?utf-8?B?RHJLblRKVUp3Q2JPKzFNNVZXRHBaR0Y3cXBtc2NIbHErWTRuR1Nuc1Y3elQy?=
 =?utf-8?B?UDNiQTVGZUJJQ3VEc3pmYkFIcE55RWNLVk5YUDdEanFkblowYjNXUC82M0lR?=
 =?utf-8?B?N2Fad2JNSTB4Z2w2eG1QR3habWpTWEpWbVpXd083WXMyWHBURzZsZG9ac0tL?=
 =?utf-8?B?N0lRRzdYY3BESkxZMjVvSmxVMlVjZ0tnU3FGZHZMUXhqajBzNk9JUlVBMlE0?=
 =?utf-8?B?UXlnSFB2cVlSWEFjRkRyZFFtbWVSRDRvb3ZRR2FVdG5MVVhCdzlBWk8vdlJz?=
 =?utf-8?B?dWovc0lhTVJ6SVRpU1A2VmZkUXFrUGVwQ3hhdXBXM2lRcmFORHdhN3U5ZUJo?=
 =?utf-8?B?MXI1akJld2dvY1hTUjRON3plaHZjSmN3OGpCL0tZRDNQNjYrM1JyWS9rU3dO?=
 =?utf-8?B?TWhTVmZCNURnV0JRcWFOUUp6QTBwQXRWOGlVVVdGaE1nTFo2MWwydDEzNWpE?=
 =?utf-8?B?WXNrc3ZxNHNTWUo4bXF0ZjFvazVDZXk3NTh3RTNOQ3REQ1N2YmdtbkN6b2tw?=
 =?utf-8?B?M3FmWGlSaFRRR094cTVxMTM2NGpzME5WalREd2NMT0dPaDg2eHVEUFI3MkNy?=
 =?utf-8?B?c0RHVVIrRWNvRFBLSVpPcjZLcmJKYzEvOG01bzN1Uno2SDdUZmJkN1EyNWpv?=
 =?utf-8?B?c2hxQmR4ek9yQW5Ga3FTc3JUdkZLZ2lpc0xsaW5ZQmdaaXQwSkJXOTlEVmhs?=
 =?utf-8?B?ZzlqSWF5dTNkUFdteld4dm4vNlFtd1hVRDRMSkRhY1U2aUNjRFdadEhSWkJW?=
 =?utf-8?B?Y1RlYTVQNjdTNU9qYzNkTVBIb3FNR0FQZTdYY0k0STJqS3g3ci9DdjFQa1cy?=
 =?utf-8?B?Snp4TmQ1cENDOUxpUk5RVHZkRnRWeHAxMFJvZ1pDcThTVTJRRWpEeXhiaXd4?=
 =?utf-8?B?NUpsa1VXVUFNbnY0d0lyQzEvMWVUejE0VDlDcGdKSlA0bisrTFBNY3NESFdV?=
 =?utf-8?B?Mlo4ZDE1dFBONzRsYkJvR0pKTHR1TkZra2hhbUF3OTBlTm0zanA4aXFySk1j?=
 =?utf-8?B?bjdaK3QrWmtGK2txZS9vMmlickZBbGY5cXRGNnNsTU5janZWbWJnVHB0Q1Nm?=
 =?utf-8?B?c0RuaUNvcndud3NhV3NKQTcxazRmMWtPMWMvdG9JYUpUOWsyVThWWFp1bCsr?=
 =?utf-8?B?MkRWOGNLV0R1MlI4VGwzVjk1c2U0RDRHQ0ZTaWdvazM0SWEwb1VLZ01RKzhQ?=
 =?utf-8?B?Sll6bWVhUnFoekc0aUpRM3Qzb0FSejh4NUE5ZEh3cHd0RW9OZjlPNjlDL3R2?=
 =?utf-8?B?Y3Bmay8xODI3YU1zblphUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <8393431B4FC6854D8177E3F627909D6B@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a05e2d2-cb2d-4999-e15f-08d96ede5edc
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Sep 2021 13:26:00.1829
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: IqmHj1XaQZUerY+KO1cpz+hYk34BfmcSrDUhSZlxnheGjnhVsCqt6GkXnR3QnahTJMYoxnEEvuR7jwHhgu096VV9VpsqzH76jTHErQo6G2gki5aOxpApEch/iM6iLbjv
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5697
X-Proofpoint-ORIG-GUID: H4sECB9vG4h-iOsAPH13KUqcJhN9pwlV
X-Proofpoint-GUID: H4sECB9vG4h-iOsAPH13KUqcJhN9pwlV
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-03_05,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0
 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=999 impostorscore=0
 adultscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109030082

DQpPbiAwMy4wOS4yMSAxNTo0MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTA6MzMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Bj
aS5jDQo+PiBAQCAtMTMwMSw2ICsxMzAxLDkgQEAgc3RhdGljIGludCBpb21tdV9hZGRfZGV2aWNl
KHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4gICAgICAgaWYgKCAhaXNfaW9tbXVfZW5hYmxlZChw
ZGV2LT5kb21haW4pICkNCj4+ICAgICAgICAgICByZXR1cm4gMDsNCj4+ICAgDQo+PiArI2lmZGVm
IENPTkZJR19BUk0NCj4+ICsgICAgcGNpX3RvX2RldihwZGV2KS0+dHlwZSA9IERFVl9QQ0k7DQo+
PiArI2VuZGlmDQo+IFdoeSBoZXJlIGluc3RlYWQgb2YgaW4gYWxsb2NfcGRldigpPyBUaGUgZmll
bGQgc2hvdWxkIGJlIHZhbGlkIGJ5IHRoZSB0aW1lDQo+IHRoZSBuZXcgaXRlbSBnZXRzIGluc2Vy
dGVkIGludG8gdGhlIHNlZ21lbnQncyBsaXN0IG9mIGRldmljZXMsIGltby4NCg0KWWVzLCBtYWtl
cyBzZW5zZS4NCg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg0KPg0KPiBKYW4NCj4=


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 13:40:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 13:40:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178100.323969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9QC-0003Tz-8g; Fri, 03 Sep 2021 13:39:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178100.323969; Fri, 03 Sep 2021 13:39:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9QC-0003Ts-5E; Fri, 03 Sep 2021 13:39:56 +0000
Received: by outflank-mailman (input) for mailman id 178100;
 Fri, 03 Sep 2021 13:39:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM9QB-0003Ti-F6; Fri, 03 Sep 2021 13:39:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM9QA-0005J7-W1; Fri, 03 Sep 2021 13:39:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mM9QA-0005gy-If; Fri, 03 Sep 2021 13:39:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mM9QA-0002vK-IB; Fri, 03 Sep 2021 13:39:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Gkwxrc440Tl5r1hYdJc5VfuLuzN27s7TC6KzhzIsknM=; b=VKVqfpdFUtQ16UiHbR0kf7b1b+
	LY9aNY2EaA5MxJOum5Y3qe41W4UxfuREREQ5Aty5ks0KBAimPucVLnan4coRvR4HZvZjZTB28fi6W
	izvQsbCT8tFGf9y5nhTiFKux8BaqR8twKdbWwb6PoEptBA9pi3HHplSRO2aruC+JrXUs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164791-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164791: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-localmigrate/x10:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a9c9a6f741cdaa2fa9ba24a790db8d07295761e3
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 13:39:54 +0000

flight 164791 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164791/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-amd64-amd64-qemuu-freebsd11-amd64 19 guest-localmigrate/x10 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-arndale   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                a9c9a6f741cdaa2fa9ba24a790db8d07295761e3
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  398 days
Failing since        152366  2020-08-01 20:49:34 Z  397 days  689 attempts
Testing same since   164791  2021-09-03 04:54:46 Z    0 days    1 attempts

------------------------------------------------------------
7181 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2197963 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 13:53:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 13:53:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178110.323983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9cn-000650-As; Fri, 03 Sep 2021 13:52:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178110.323983; Fri, 03 Sep 2021 13:52:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mM9cn-00064t-60; Fri, 03 Sep 2021 13:52:57 +0000
Received: by outflank-mailman (input) for mailman id 178110;
 Fri, 03 Sep 2021 13:52:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kbJo=NZ=linuxfoundation.org=aweltz@srs-us1.protection.inumbo.net>)
 id 1mM9cm-00064n-8H
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 13:52:56 +0000
Received: from mail-il1-x131.google.com (unknown [2607:f8b0:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c665ad12-4d74-43bb-9d32-d5cdc038559c;
 Fri, 03 Sep 2021 13:52:55 +0000 (UTC)
Received: by mail-il1-x131.google.com with SMTP id h29so5288303ila.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 06:52:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c665ad12-4d74-43bb-9d32-d5cdc038559c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linuxfoundation.org; s=google;
        h=mime-version:from:date:message-id:subject:to:cc;
        bh=LcflHDuWnt22drIHcdo3+GFGrkeK5QWOt9ac+oMd6kE=;
        b=eXkEl7Rm2aiiik4kPHuU4F1OnGGRqiaujgiCkqGXG0cpstYFYWa8wiKzdF0RReoPFJ
         8D9vFRc1d2VIkVdGREBEq3UOxb/qRHAhDVT94fPuqr1Q+3f2R6fQ8yy+KR+NxRQFtBme
         a3toHvO3iNa4cOQf0GChUAxiu3HF9IFfcwHhU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc;
        bh=LcflHDuWnt22drIHcdo3+GFGrkeK5QWOt9ac+oMd6kE=;
        b=tK/UOLulv7akLzkTJoQusSNUlKQl/DaoI2tS75dXzOWa09tYJ/Q50uoZiBroxWvfTL
         IQUY9faZ1pB/AmmsA5aNz6aw5uqli1j1g03EGwdrR+qwYFWyzwpFJ2Z+6GvnFt3XIM8/
         7oUN1asr+CP0HPqOrnzwt07Q6iNOEMUFOfyfW85PYx/QcrGv3wF2YCf9XRHPUqq0F+OT
         9Cg2jgDsDIZRP9w42dNHCUF+4vBFAwlFrtfcwzLlJ8k2BBdMhUnSsy+5klcraHg6RfRm
         ZFZhfDK86CtRgUnkcedxKtEJDKKGBQo0fmu9yCT8Dtnv6/BEwFjtZiuWvNQ+E1pd4+l9
         Taww==
X-Gm-Message-State: AOAM531MFjGaPic4U7v+PpPlcFYmoRwDDixrh3ErG6ui6GEvVlCbuWdT
	wDUJ2tbgHXTw5CgRDd6jUiYwBKPDDtFSn64UQq13MQ==
X-Google-Smtp-Source: ABdhPJx3adAS9hhK6cug5GLkwtGb1nPGLwRZ3S7Xjvj3gZC2aZvuo61R2uUSFjXy5lPujr2imEgnPUxbp5R9ru3PLIU=
X-Received: by 2002:a05:6e02:1353:: with SMTP id k19mr2671451ilr.157.1630677174665;
 Fri, 03 Sep 2021 06:52:54 -0700 (PDT)
MIME-Version: 1.0
From: Ashley Weltz <aweltz@linuxfoundation.org>
Date: Fri, 3 Sep 2021 09:52:43 -0400
Message-ID: <CADT4066k7HR=Bpcg1U0jT+w2Zt5u_4ApHXTZuAo-i3g-Z79mpQ@mail.gmail.com>
Subject: Call for agenda items for September 7th Community Call @ 1500 UTC
To: amit@infradead.org, andrew.cooper3@citrix.com, Artem_Mygaiev@epam.com, 
	ash.j.wilding@gmail.com, ben@exotanium.io, bobby.eshleman@gmail.com, 
	brendank310@gmail.com, brian.woods@xilinx.com, cardoe@cardoe.com, 
	christopher.w.clark@gmail.com, Corey Minyard <cminyard@mvista.com>, 
	daniel.kiper@oracle.com, deepthi.m@ltts.com, dpsmith@apertussolutions.com, 
	dwmw@amazon.co.uk, edgar.iglesias@xilinx.com, Ian.Jackson@citrix.com, 
	insurgo@riseup.net, intel-xen@intel.com, Jarvis.Roach@dornerworks.com, 
	Jeff.Kubascik@dornerworks.com, jgross@suse.com, john.ji@intel.com, 
	julien@xen.org, kevin.pearson@ortmanconsulting.com, Matt.Spencer@arm.com, 
	=?UTF-8?Q?Mirela_Simonovi=C4=87?= <mirela.simonovic@aggios.com>, 
	oleksandr_andrushchenko@epam.com, oleksandr_tyshchenko@epam.com, 
	olivier.lambert@vates.fr, pdurrant@amazon.com, persaur@gmail.com, 
	piotr.krol@3mdeb.com, Rahul.Singh@arm.com, rianquinn@gmail.com, 
	rob.townley@gmail.com, robin.randhawa@arm.com, roger.pau@citrix.com, 
	scottwd@gmail.com, sergey.dyasli@citrix.com, sstabellini@kernel.org, 
	Stewart.Hildebrand@dornerworks.com, tamas.k.lengyel@gmail.com, 
	varadgautam@gmail.com, volodymyr_babchuk@epam.com, wl@xen.org
Cc: xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000bd5c6605cb179deb"

--000000000000bd5c6605cb179deb
Content-Type: text/plain; charset="UTF-8"

Hi everyone,

Our next meeting is this upcoming Tuesday at 1500 UTC.

The proposed agenda is in
https://cryptpad.fr/pad/#/2/pad/edit/PXjRK6mJfWbUxSNW0Qt8xOqT/. Please add
or edit any items to this agenda. Alternatively, please feel free to email
me directly with agenda items.

Please put your name beside any items if you edit the document.

We will plan to start the meeting 5 minutes past the hour.

* If you want to be CC'ed please add or remove yourself from the
sign-up-sheet at
https://cryptpad.fr/pad/#/2/pad/edit/D9vGzihPxxAOe6RFPz0sRCf+/

Again, please let me know if you have any questions. See you on Tuesday!

Cheers,
Ashley

-- 
Ashley Weltz, Project Coordinator
The Linux Foundation
aweltz@linuxfoundation.org

--000000000000bd5c6605cb179deb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi everyone,</div><div><br></div><div>Our next meetin=
g is this upcoming Tuesday at 1500 UTC.=C2=A0</div><div><br></div><div>The =
proposed agenda is in=C2=A0<a href=3D"https://cryptpad.fr/pad/#/2/pad/edit/=
PXjRK6mJfWbUxSNW0Qt8xOqT/">https://cryptpad.fr/pad/#/2/pad/edit/PXjRK6mJfWb=
UxSNW0Qt8xOqT/</a>. Please add or edit any items to this agenda. Alternativ=
ely, please feel free to email me directly with agenda items.=C2=A0</div><d=
iv><br>Please put your name beside any items if you edit the document.<br><=
br>We will plan to start the meeting 5 minutes past the hour.=C2=A0<br><br>=
* If you want to be CC&#39;ed please add or remove yourself from the sign-u=
p-sheet at=C2=A0<a href=3D"https://cryptpad.fr/pad/#/2/pad/edit/D9vGzihPxxA=
Oe6RFPz0sRCf+/" rel=3D"noreferrer" target=3D"_blank">https://cryptpad.fr/pa=
d/#/2/pad/edit/D9vGzihPxxAOe6RFPz0sRCf+/</a></div><div><br></div><div>Again=
, please let me know if you have any questions. See you on Tuesday!</div><d=
iv><br></div><div>Cheers,</div><div>Ashley</div><div><br></div>-- <br><div =
dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><d=
iv dir=3D"ltr">Ashley Weltz, Project Coordinator=C2=A0<div>The Linux Founda=
tion</div><div><a href=3D"mailto:aweltz@linuxfoundation.org" target=3D"_bla=
nk">aweltz@linuxfoundation.org</a></div></div></div></div>

--000000000000bd5c6605cb179deb--


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:06:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:06:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178120.323994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAm8-0005l6-P3; Fri, 03 Sep 2021 15:06:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178120.323994; Fri, 03 Sep 2021 15:06:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAm8-0005kz-LT; Fri, 03 Sep 2021 15:06:40 +0000
Received: by outflank-mailman (input) for mailman id 178120;
 Fri, 03 Sep 2021 15:06:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAm7-0005kt-Nt
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:06:39 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c2c0082e-686b-4760-b90e-0f7fdcb53c2d;
 Fri, 03 Sep 2021 15:06:38 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681592656284.14797840719837;
 Fri, 3 Sep 2021 08:06:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2c0082e-686b-4760-b90e-0f7fdcb53c2d
ARC-Seal: i=1; a=rsa-sha256; t=1630681596; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=E2oZLva+iHX9+Zc88gM7a7iGmT8mb9EStMMvtU73/v3qRt+V+muRRXZw12qB5H+sNR+05kxZU1rfEd8iEAFhHsJggDyk7PTTFHm/28TIKSEONkZELEVtZ9A+3qHlyjiUvpuqBomYG4ZgaiArsUYBelM6VcAciOvKIGEmv2Z15Nk=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681596; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:MIME-Version:Message-ID:Subject:To; 
	bh=DY8SumKHy69+y3Ky8ZXpZgjykeZw2efDQnmXyQdCT7I=; 
	b=KzC69ufgi3jJTaSVSrXTOevHYq2d/Sx2ojmKZ3PsXiBsmc5bDjT4GCk6lxtL9j14ALyZ2KhQ9GEpYFsqy2cunu2hT/Zl56eTVasu/I0jIAJW6zHxQwacDJlEEv9lsFczrSfgHqowhzSpCyAH2EoLFR74cMWGpHl8T8xDioQ+ehs=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681596;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=DY8SumKHy69+y3Ky8ZXpZgjykeZw2efDQnmXyQdCT7I=;
	b=csHVsu2HGGmCXToKfqdM5cZAqNkSdhU0AnJAka7lU2tQX30O9p4ZE5KcnwUzhTfH
	/yNdQaj8UePYzjJ57chw/dbmLNY1IaRGOxBOtpoUcDilRvIOvW5ccyH2HOj6LNRF6bk
	zAmhlLuo01z5G+TRTQuitv1iSvRV/YzP7DG4QdeA=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: xen-devel@lists.xenproject.org
Cc: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v4 00/11] xsm: refactoring xsm hooks
Date: Fri,  3 Sep 2021 15:06:18 -0400
Message-Id: <20210903190629.11917-1-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Based on feedback from 2021 Xen Developers Summit the xsm-roles RFC patch set
is being split into two separate patch sets. This is the first patch set and is
focused purely on the clean up and refactoring of the XSM hooks.

This patch set refactors the xsm_ops wrapper hooks to use the alternative_call
infrastructure. Then proceeds to move and realign the headers to simplify the
unnecessarily complicated header inclusions, remove the facad of being able to
enable/disable XSM, and simplify the XSM hooks down to a single interface and
implementation. The remainder of the changes are clean up and removing no
longer necessary abstractions.

v2:
 - restructured the patches, breaking them up as needed
 - incorporate Andrew Cooper's alternative call common code
 - change XSM module registration, removing register_xsm
 - incoporate KConfig recommendations
 - reworded commit messages
 - incorporate macro expansion recommendations
 - misc clean-up fallout from recommendations

v3:
 - renamed struct xsm_operations to struct xsm_ops
 - flask and silo ops structs made __initconst
 - fixed misplacement of __init on flask/silo_init
 - lots of coding style alignment
 - further clean up from FLASK_DISABLE removal
 - addressed commit message comments
 - removed missed guard around alternative-call include
 - reworked approach to XSM hooks, merging the two interfaces instead of
   dropping one

v4:
- make __alt_call_maybe_initdata resolve to __read_mostly when
  CONFIG_ALTERNATIVE_CALL is not set
- removed the masking of void with xsm_op_t
- moved all the coding style conformity to an early commit
- fixed declaration on real and stub declarations in xsm.h
- corrected to __initconstrel
- made the xsm_ops global __read_mostly
- adjusted blank lines in xsm_ops struct to provide consistent grouping
- moved extern references to built in policy to where they are used
- Added back in the #ifdef CONFIG_XSM into struct evtchn
- split the patch removing the duplicate interface up further

Andrew Cooper (1):
  xen: Implement xen/alternative-call.h for use in common code

Daniel P. Smith (10):
  xsm: remove the ability to disable flask
  xsm: drop dubious xsm_op_t type
  xsm: apply coding style
  xsm: refactor xsm_ops handling
  xsm: convert xsm_ops hook calls to alternative call
  xsm: decouple xsm header inclusion selection
  xsm: drop generic event channel labeling exclusion
  silo: remove circular xsm hook call
  kconfig: update xsm config to reflect reality
  xsm: remove alternate xsm hook interface

 xen/arch/x86/Kconfig               |   1 +
 xen/common/Kconfig                 |  52 +-
 xen/include/public/xsm/flask_op.h  |   2 +-
 xen/include/xen/alternative-call.h |  63 +++
 xen/include/xen/hypercall.h        |   4 +-
 xen/include/xen/sched.h            |   4 +-
 xen/include/xsm/dummy.h            | 774 ---------------------------
 xen/include/xsm/xsm-core.h         | 273 ++++++++++
 xen/include/xsm/xsm.h              | 804 +++++++++++++----------------
 xen/xsm/Makefile                   |   4 +-
 xen/xsm/dummy.c                    |   7 +-
 xen/xsm/dummy.h                    | 739 ++++++++++++++++++++++++++
 xen/xsm/flask/flask_op.c           |  32 +-
 xen/xsm/flask/hooks.c              |  16 +-
 xen/xsm/silo.c                     |  25 +-
 xen/xsm/xsm_core.c                 |  94 ++--
 16 files changed, 1535 insertions(+), 1359 deletions(-)
 create mode 100644 xen/include/xen/alternative-call.h
 delete mode 100644 xen/include/xsm/dummy.h
 create mode 100644 xen/include/xsm/xsm-core.h
 create mode 100644 xen/xsm/dummy.h

-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:06:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:06:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178121.324005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmF-00062h-7U; Fri, 03 Sep 2021 15:06:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178121.324005; Fri, 03 Sep 2021 15:06:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmF-00062a-4U; Fri, 03 Sep 2021 15:06:47 +0000
Received: by outflank-mailman (input) for mailman id 178121;
 Fri, 03 Sep 2021 15:06:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAmD-000623-QD
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:06:45 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb533620-ee78-4d74-8252-62769fbfdca1;
 Fri, 03 Sep 2021 15:06:44 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681595567909.1171948642584;
 Fri, 3 Sep 2021 08:06:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb533620-ee78-4d74-8252-62769fbfdca1
ARC-Seal: i=1; a=rsa-sha256; t=1630681598; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=W/3McJcH+hgLEXU0Oay4LDpRFo+o9UmyNcl2T52y37Pfk55++hvihDEapiW2fcK3E2TP6aO2L6B0ysCgW2ntpHS23FK7yD6D0vvRfG46mTlgklV4vSQhTN+N8kCvv5POu9lECoMuYUAVPb5PVTIKe0QsIiJsLr69AOZV8oMb0/c=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681598; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=FwXfAdIv2IZUMTi0r+WhHxk4n1x2ncGtKlCWXV3sMlc=; 
	b=c8SfnD45zPwJ6Ashw9ih+Res04Da/g3ui7778oLj/rcvHtarNTDK+guDvCpcWar0Ze+kMg4B1BFOZwogPQjBOD5pae4ueq9r/ynCR71sUC5ESWGWIYQ39ogM82A/wbWoWqpY8TYsBsXXTK1G5me+uMt4eRYIuxoFMTkdu2AAkoU=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681598;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=FwXfAdIv2IZUMTi0r+WhHxk4n1x2ncGtKlCWXV3sMlc=;
	b=ZJBR/Tr+MzGvXgfIVcXhA7m7rmRYtb8FkKKBuQfAeA0npTHJWgmqWkg5/fe0StER
	7+fzsDItRiLRfyqhsp+LIjygyQ2C1b3WjX/9S5TCz+z/eKRbsNv3xvvfRAvTIS3z4/v
	64FEX45XVGtrIFqN1D2N7HiJGnoBPbOumzYKF2i0=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Connor Davis <connojdavis@gmail.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Jan Beulich <jbeulich@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 01/11] xen: Implement xen/alternative-call.h for use in common code
Date: Fri,  3 Sep 2021 15:06:19 -0400
Message-Id: <20210903190629.11917-2-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

From: Andrew Cooper <andrew.cooper3@citrix.com>

The alternative call infrastructure is x86-only for now, but the common iommu
code has a variant and more common code wants to use the infrastructure.

Introduce CONFIG_ALTERNATIVE_CALL and a conditional implemetnation so common
code can use the optimisation when available, without requiring all
architectures to implement no-op stubs.

Write some documentation, which was thus far entirely absent, covering the
requirements for an architecture to implement this optimsiation, and how to
use the infrastructure in general code.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bob Eshleman <bobbyeshleman@gmail.com>
CC: Alistair Francis <alistair.francis@wdc.com>
CC: Connor Davis <connojdavis@gmail.com>
CC: Daniel P. Smith <dpsmith@apertussolutions.com>

v3:
 * Drop __alt_call_maybe_initconst

This is a pre-requisite to "xsm: refactor xsm_ops handling" to avoid breaking
the ARM build.

Build test for the XSM code:

  diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
  index 5eab21e1b168..592074e8f41c 100644
  --- a/xen/xsm/xsm_core.c
  +++ b/xen/xsm/xsm_core.c
  @@ -195,6 +195,16 @@ bool __init has_xsm_magic(paddr_t start)
   }
    #endif

  +#include <xen/alternative-call.h>
  +struct foo {
  +    int (*bar)(void *);
  +} foo __alt_call_maybe_initdata;
  +
  +int test_alternative_call(void)
  +{
  +    return alternative_call(foo.bar, NULL);
  +}
  +
   int __init register_xsm(struct xsm_operations *ops)
    {
         if ( verify(ops) )
---
 xen/arch/x86/Kconfig               |  1 +
 xen/common/Kconfig                 |  3 ++
 xen/include/xen/alternative-call.h | 63 ++++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+)
 create mode 100644 xen/include/xen/alternative-call.h

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 9b164db641..1f83518ee0 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,6 +6,7 @@ config X86
 	def_bool y
 	select ACPI
 	select ACPI_LEGACY_TABLES_LOOKUP
+	select ALTERNATIVE_CALL
 	select ARCH_SUPPORTS_INT128
 	select CORE_PARKING
 	select HAS_ALTERNATIVE
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11a..ac5491b1cc 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -22,6 +22,9 @@ config GRANT_TABLE
 
 	  If unsure, say Y.
 
+config ALTERNATIVE_CALL
+	bool
+
 config HAS_ALTERNATIVE
 	bool
 
diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternative-call.h
new file mode 100644
index 0000000000..d10af90b1b
--- /dev/null
+++ b/xen/include/xen/alternative-call.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef XEN_ALTERNATIVE_CALL
+#define XEN_ALTERNATIVE_CALL
+
+/*
+ * Some subsystems in Xen may have multiple implementions, which can be
+ * resolved to a single implementation at boot time.  By default, this will
+ * result in the use of function pointers.
+ *
+ * Some architectures may have mechanisms for dynamically modifying .text.
+ * Using this mechnaism, function pointers can be converted to direct calls
+ * which are typically more efficient at runtime.
+ *
+ * For architectures to support:
+ *
+ * - Implement alternative_{,v}call() in asm/alternative.h.  Code generation
+ *   requirements are to emit a function pointer call at build time, and stash
+ *   enough metadata to simplify the call at boot once the implementation has
+ *   been resolved.
+ * - Select ALTERNATIVE_CALL in Kconfig.
+ *
+ * To use:
+ *
+ * Consider the following simplified example.
+ *
+ *  1) struct foo_ops __alt_call_maybe_initdata ops;
+ *
+ *  2) const struct foo_ops __initconst foo_a_ops = { ... };
+ *     const struct foo_ops __initconst foo_b_ops = { ... };
+ *
+ *     void foo_init(void)
+ *     {
+ *         ...
+ *         if ( use_impl_a )
+ *             ops = *foo_a_ops;
+ *         else if ( use_impl_b )
+ *             ops = *foo_b_ops;
+ *         ...
+ *     }
+ *
+ *  3) alternative_call(ops.bar, ...);
+ *
+ * There needs to a single ops object (1) which will eventually contain the
+ * function pointers.  This should be populated in foo's init() function (2)
+ * by one of the available implementations.  To call functions, use
+ * alternative_{,v}call() referencing the main ops object (3).
+ */
+
+#ifdef CONFIG_ALTERNATIVE_CALL
+
+#include <asm/alternative.h>
+
+#define __alt_call_maybe_initdata __initdata
+
+#else
+
+#define alternative_call(func, args...)  (func)(args)
+#define alternative_vcall(func, args...) (func)(args)
+
+#define __alt_call_maybe_initdata __read_mostly
+
+#endif /* !CONFIG_ALTERNATIVE_CALL */
+#endif /* XEN_ALTERNATIVE_CALL */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:07:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178123.324016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmU-0006XF-Gt; Fri, 03 Sep 2021 15:07:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178123.324016; Fri, 03 Sep 2021 15:07:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmU-0006X1-Cb; Fri, 03 Sep 2021 15:07:02 +0000
Received: by outflank-mailman (input) for mailman id 178123;
 Fri, 03 Sep 2021 15:07:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAmT-0006Uj-CA
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:07:01 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2caff1e1-690e-4e43-9507-43e8276cac68;
 Fri, 03 Sep 2021 15:06:59 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681598091395.8657927244592;
 Fri, 3 Sep 2021 08:06:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2caff1e1-690e-4e43-9507-43e8276cac68
ARC-Seal: i=1; a=rsa-sha256; t=1630681600; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=U3YG+tLKobR9wflqy2s9fIoOx+VtYcW5kHkGPVrruLfA5AmQYbCGTZ2kfX5OTEX7QllGconwszVSbOFbPSasY0pkC00SAZgkD2tR/WxR6z6yE/50339vyAAjCNsm41B7ls5bc+D/Z70PJ1SCx/1uF4nro/QKERAvlhU8Mo10DUU=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681600; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=tSLACVHOio6zEEE9Ar9yJitpL6mvf0UNHFD+1CvJyII=; 
	b=fM9UiAmm8+W9UiDy0mLksHjNJq7t6/45SznVb0C/QO4mS06XgIkHM4g0i1AJ1cW9H14U3QOKj3sJtb6urbfus81OnuOd8Sro1ePhCHcamy7PokGJ5IIhISHIBww5S+qRblyVf9tSuDe/irGwk0QEcRMwiHqumCJ2Gdqmyq1Lmrw=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681600;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=tSLACVHOio6zEEE9Ar9yJitpL6mvf0UNHFD+1CvJyII=;
	b=T7PxN4luxUgXYnC/GptfidDpQ8Qzr0aulBWZJ3lRiJEPb26P727ADdLOugwz77I8
	7XQT64jzSaJHgVxziOna40GDQaOyTy7E7oj2FKoofDFGVx4savVmrnSguRUjnlst4xT
	18yUymTBZTuvBiKa+t9oIPg4NTYN+Cr7Op/JAMGk=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 02/11] xsm: remove the ability to disable flask
Date: Fri,  3 Sep 2021 15:06:20 -0400
Message-Id: <20210903190629.11917-3-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On Linux when SELinux is put into permissive mode the descretionary access
controls are still in place. Whereas for Xen when the enforcing state of flask
is set to permissive, all operations for all domains would succeed, i.e. it
does not fall back to the default access controls. To provide a means to mimic
a similar but not equivalent behavior, a flask op is present to allow a
one-time switch back to the default access controls, aka the "dummy policy".

While this may be desirable for an OS, Xen is a hypervisor and should not allow
the switching of which security policy framework is being enforced after boot.
This patch removes the flask op to enforce the desired XSM usage model
requiring a reboot of Xen to change the XSM policy module in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/public/xsm/flask_op.h |  2 +-
 xen/xsm/flask/flask_op.c          | 30 ------------------------------
 2 files changed, 1 insertion(+), 31 deletions(-)

diff --git a/xen/include/public/xsm/flask_op.h b/xen/include/public/xsm/flask_op.h
index 16af7bc22f..b41dd6dac8 100644
--- a/xen/include/public/xsm/flask_op.h
+++ b/xen/include/public/xsm/flask_op.h
@@ -188,7 +188,7 @@ struct xen_flask_op {
 #define FLASK_SETBOOL           12
 #define FLASK_COMMITBOOLS       13
 #define FLASK_MLS               14
-#define FLASK_DISABLE           15
+#define FLASK_DISABLE           15 /* No longer implemented */
 #define FLASK_GETAVC_THRESHOLD  16
 #define FLASK_SETAVC_THRESHOLD  17
 #define FLASK_AVC_HASHSTATS     18
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 01e52138a1..f41c025391 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -223,32 +223,6 @@ static int flask_security_sid(struct xen_flask_sid_context *arg)
 
 #ifndef COMPAT
 
-static int flask_disable(void)
-{
-    static int flask_disabled = 0;
-
-    if ( ss_initialized )
-    {
-        /* Not permitted after initial policy load. */
-        return -EINVAL;
-    }
-
-    if ( flask_disabled )
-    {
-        /* Only do this once. */
-        return -EINVAL;
-    }
-
-    printk("Flask:  Disabled at runtime.\n");
-
-    flask_disabled = 1;
-
-    /* Reset xsm_ops to the original module. */
-    xsm_ops = &dummy_xsm_ops;
-
-    return 0;
-}
-
 static int flask_security_setavc_threshold(struct xen_flask_setavc_threshold *arg)
 {
     int rv = 0;
@@ -698,10 +672,6 @@ ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
         rv = flask_mls_enabled;
         break;    
 
-    case FLASK_DISABLE:
-        rv = flask_disable();
-        break;
-
     case FLASK_GETAVC_THRESHOLD:
         rv = avc_cache_threshold;
         break;
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:07:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:07:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178134.324027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmm-0007IV-PL; Fri, 03 Sep 2021 15:07:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178134.324027; Fri, 03 Sep 2021 15:07:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAmm-0007IO-Lz; Fri, 03 Sep 2021 15:07:20 +0000
Received: by outflank-mailman (input) for mailman id 178134;
 Fri, 03 Sep 2021 15:07:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAml-0006va-Rx
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:07:19 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9fa6f2f5-0cc8-11ec-af2e-12813bfff9fa;
 Fri, 03 Sep 2021 15:07:16 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681600351351.4561344219063;
 Fri, 3 Sep 2021 08:06:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9fa6f2f5-0cc8-11ec-af2e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1630681602; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=jLx1FP5XMnBmH5Si8hAulIa6JYl7fcmWYjOQ46NGOvzlpYLnRpCNnHR54O68cprM79eWlwviqKpch+ek4arkyUjp0FlMbanY+i8CZCTaX5NjJSP3gbMS4/0Ocov1rY2EWcTzJdOhGHyC8RXr5T+fxH9P6FEYu/pO9GJDqhLLTKA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681602; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=6kGTTbjxT8L+j+0w8ftNK7DzfHzYGwCxPp/WOutTjVk=; 
	b=O6Ge//VArCi2Jmd0u/I1nj1a0Q3s8PsCN6GQhl4GE482Vr/4DH1rEqJTjJ/KQgA+hf+MP8+zfC4qVqnoKWx+6vgjHkhimeEdo0z2/WBTnJbdXdzY7Q++wwkUXbQvcIoi/+/FQx7U504KXazryx1F5pLCNQAjmvA5fZGaw6D73QU=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681602;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=6kGTTbjxT8L+j+0w8ftNK7DzfHzYGwCxPp/WOutTjVk=;
	b=dyN+y7sLq+jzU7UE4JdsQTqZvDGIg+bRTTTykR+cDqumvfRvu+M7gZgKxN0fGxns
	hgOcqVhMn8BpZvEAQNprwiopsv4PcZUwyZL7d/+xsvff3ChSbaIaE+cO2wPrCuZ6B4u
	AFIbtN6DHIDT5QmucsfDJUrswDdU8mUmJ09TyIhM=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 03/11] xsm: drop dubious xsm_op_t type
Date: Fri,  3 Sep 2021 15:06:21 -0400
Message-Id: <20210903190629.11917-4-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

The type xsm_op_t masks the use of void pointers. This commit drops the xsm_op_t type and
replaces it and all its uses with an explicit void.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xen/hypercall.h |  4 ++--
 xen/include/xsm/dummy.h     |  4 ++--
 xen/include/xsm/xsm.h       | 11 ++++-------
 xen/xsm/flask/flask_op.c    |  2 +-
 xen/xsm/flask/hooks.c       |  4 ++--
 xen/xsm/xsm_core.c          |  4 ++--
 6 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 34b7f1fed6..3771487a30 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -127,7 +127,7 @@ do_kexec_op(
 
 extern long
 do_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_xsm_op);
+    XEN_GUEST_HANDLE_PARAM(void) u_xsm_op);
 
 #ifdef CONFIG_ARGO
 extern long do_argo_op(
@@ -198,7 +198,7 @@ compat_set_timer_op(
     s32 hi);
 
 extern int compat_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    XEN_GUEST_HANDLE_PARAM(void) op);
 
 extern int compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg);
 
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 363c6d7798..214b5408b1 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -442,13 +442,13 @@ static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
 
 #ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index ad3cddbf7d..9872bae502 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,9 +18,6 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
-typedef void xsm_op_t;
-DEFINE_XEN_GUEST_HANDLE(xsm_op_t);
-
 /* policy magic number (defined by XSM_MAGIC) */
 typedef u32 xsm_magic_t;
 
@@ -129,9 +126,9 @@ struct xsm_operations {
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
     int (*hvm_param) (struct domain *d, unsigned long op);
@@ -543,13 +540,13 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index f41c025391..221ff00fd3 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -607,7 +607,7 @@ static int flask_relabel_domain(struct xen_flask_relabel *arg)
 
 #endif /* !COMPAT */
 
-ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
+ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op)
 {
     xen_flask_op_t op;
     int rv;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index f1a1217c98..1465db125a 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1742,8 +1742,8 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 
 #endif
 
-long do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
-int compat_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
+long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
 static struct xsm_operations flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 5eab21e1b1..ac553f9c0d 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -213,13 +213,13 @@ int __init register_xsm(struct xsm_operations *ops)
 
 #endif
 
-long do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-int compat_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+int compat_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_compat_op(op);
 }
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:07:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178143.324038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAn4-0007uK-1s; Fri, 03 Sep 2021 15:07:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178143.324038; Fri, 03 Sep 2021 15:07:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAn3-0007uB-Uz; Fri, 03 Sep 2021 15:07:37 +0000
Received: by outflank-mailman (input) for mailman id 178143;
 Fri, 03 Sep 2021 15:07:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAn3-0007sS-0Y
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:07:37 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a9ad2160-0cc8-11ec-af2e-12813bfff9fa;
 Fri, 03 Sep 2021 15:07:32 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681602207418.47399076829913;
 Fri, 3 Sep 2021 08:06:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9ad2160-0cc8-11ec-af2e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1630681605; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=JVemV5ww9x7Ta5ZnHqAvJfb/QCcrmfwrK/HvnVFgrsgLYQ/KHrwCkGn/jvOac9kDwojSLLeWF2H8UXR+SZ7hk3U1JpWztpzjxVXjU91THB0uaUkkdv0FPmm2sssw3RCWh6odZ8caWGux0aastb2dFphwQVvD14byW8eea2kc5SQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681605; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=CDBkvs1fwyjC/r28e2VqrqXOwVPjbuNV/phvRXMPXyE=; 
	b=Jh5ZtrdvP/hSdphEV2n7ZMzPPgeW/71y4Nmh80uvBRwcTbI10hiGn13swNb/OvDncHgH3yxnJWhre9cmv+DNmeufF9WpSFSbF4M6NCtHRE8siEBKpPbe9qBcJtIHm+I6BvV8SEtgwxI2CEwbHk4DbXhMtFwooY/aYUqiyT+oBCM=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681605;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=CDBkvs1fwyjC/r28e2VqrqXOwVPjbuNV/phvRXMPXyE=;
	b=tqtSj6MYX7jOX8YPCzQPXmskymrz5BXh9msg9tXLNUbtAb/0FvUwb+kJRCtSElOB
	BtrLxiAujtO/OmjgAkz3BQVUHfZ//Y9uhPr2VyVuvPaC/zi27lCggTIZ2uxiGLs1G0Z
	RBWp18piZOaBxjRN2MrNxdI30pfk/s/1wzjivYPo=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 04/11] xsm: apply coding style
Date: Fri,  3 Sep 2021 15:06:22 -0400
Message-Id: <20210903190629.11917-5-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Instead of intermixing coding style changes with code changes as they
are come upon in this patch set, moving all coding style changes
into a single commit. The focus of coding style changes here are,

 - move trailing comments to line above
 - ensuring line length does not exceed 80 chars
 - ensuring proper indentation for 80 char wrapping
 - covert u32 type statements to  uint32_t
 - remove space between closing and opening parens
 - drop extern on function declarations

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h | 173 +++++++++-----
 xen/include/xsm/xsm.h   | 494 ++++++++++++++++++++++------------------
 xen/xsm/xsm_core.c      |   4 +-
 3 files changed, 389 insertions(+), 282 deletions(-)

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 214b5408b1..deaf23035e 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
 
 #endif /* CONFIG_XSM */
 
-static always_inline int xsm_default_action(
-    xsm_default_t action, struct domain *src, struct domain *target)
+static always_inline int xsm_default_action(xsm_default_t action,
+                                            struct domain *src,
+                                            struct domain *target)
 {
     switch ( action ) {
     case XSM_HOOK:
@@ -99,12 +100,13 @@ static always_inline int xsm_default_action(
 }
 
 static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
-                                    struct xen_domctl_getdomaininfo *info)
+    struct xen_domctl_getdomaininfo *info)
 {
     return;
 }
 
-static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d, u32 ssidref)
+static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d,
+                                        uint32_t ssidref)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -116,7 +118,8 @@ static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_domctl_scheduler_op(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int xsm_domctl_scheduler_op(XSM_DEFAULT_ARG struct domain *d,
+                                              int cmd)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -128,7 +131,8 @@ static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_set_target(XSM_DEFAULT_ARG struct domain *d, struct domain *e)
+static XSM_INLINE int xsm_set_target(XSM_DEFAULT_ARG struct domain *d,
+                                     struct domain *e)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, NULL);
@@ -173,38 +177,43 @@ static XSM_INLINE void xsm_free_security_domain(struct domain *d)
     return;
 }
 
-static XSM_INLINE int xsm_grant_mapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static XSM_INLINE int xsm_grant_mapref(XSM_DEFAULT_ARG struct domain *d1,
+                                       struct domain *d2, uint32_t flags)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_unmapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_unmapref(XSM_DEFAULT_ARG struct domain *d1,
+                                         struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_setup(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_setup(XSM_DEFAULT_ARG struct domain *d1,
+                                      struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_transfer(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_transfer(XSM_DEFAULT_ARG struct domain *d1,
+                                         struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_copy(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_copy(XSM_DEFAULT_ARG struct domain *d1,
+                                     struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_query_size(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_query_size(XSM_DEFAULT_ARG struct domain *d1,
+                                           struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -216,14 +225,17 @@ static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_memory_adjust_reservation(XSM_DEFAULT_ARG struct domain *d1,
-                                                            struct domain *d2)
+static XSM_INLINE int xsm_memory_adjust_reservation(XSM_DEFAULT_ARG
+                                                    struct domain *d1,
+                                                    struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG
+                                                  struct domain *d1,
+                                                  struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -253,13 +265,15 @@ static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_schedop_shutdown(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_schedop_shutdown(XSM_DEFAULT_ARG struct domain *d1,
+                                           struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_pin_page(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
+static XSM_INLINE int xsm_memory_pin_page(XSM_DEFAULT_ARG struct domain *d1,
+                                          struct domain *d2,
                                           struct page_info *page)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
@@ -272,15 +286,17 @@ static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_unbound(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn,
-                                         domid_t id2)
+static XSM_INLINE int xsm_evtchn_unbound(XSM_DEFAULT_ARG struct domain *d,
+                                         struct evtchn *chn, domid_t id2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_interdomain(XSM_DEFAULT_ARG struct domain *d1, struct evtchn
-                                *chan1, struct domain *d2, struct evtchn *chan2)
+static XSM_INLINE int xsm_evtchn_interdomain(XSM_DEFAULT_ARG struct domain *d1,
+                                             struct evtchn *chan1,
+                                             struct domain *d2,
+                                             struct evtchn *chan2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
@@ -291,37 +307,41 @@ static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
     return;
 }
 
-static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d,
+                                      struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d, NULL);
 }
 
-static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d,
+                                        struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1,
+                                       struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_alloc_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+static XSM_INLINE int xsm_alloc_security_evtchns(struct evtchn chn[],
+                                                 unsigned int nr)
 {
     return 0;
 }
 
-static XSM_INLINE void xsm_free_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+static XSM_INLINE void xsm_free_security_evtchns(struct evtchn chn[],
+                                                 unsigned int nr)
 {
     return;
 }
 
-static XSM_INLINE char *xsm_show_security_evtchn(struct domain *d, const struct evtchn *chn)
+static XSM_INLINE char *xsm_show_security_evtchn(struct domain *d,
+                                                 const struct evtchn *chn)
 {
     return NULL;
 }
@@ -357,13 +377,15 @@ static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_assign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_assign_device(XSM_DEFAULT_ARG struct domain *d,
+                                        uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d,
+                                          uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -400,19 +422,22 @@ static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_plug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_plug_pci(XSM_DEFAULT_ARG
+                                            uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_unplug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_unplug_pci(XSM_DEFAULT_ARG
+                                              uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_setup_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_setup_pci(XSM_DEFAULT_ARG
+                                             uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
@@ -478,13 +503,15 @@ static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_bind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_bind_pt_irq(XSM_DEFAULT_ARG struct domain *d,
+                                      struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_unbind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_unbind_pt_irq(XSM_DEFAULT_ARG struct domain *d,
+                                        struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -497,57 +524,68 @@ static XSM_INLINE int xsm_unmap_domain_irq(XSM_DEFAULT_ARG struct domain *d,
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_irq_permission(XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
+static XSM_INLINE int xsm_irq_permission(XSM_DEFAULT_ARG struct domain *d,
+                                         int pirq, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_permission(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_permission(XSM_DEFAULT_ARG struct domain *d,
+                                           uint64_t s, uint64_t e,
+                                           uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_mapping(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_mapping(XSM_DEFAULT_ARG struct domain *d,
+                                        uint64_t s, uint64_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf,
-                                        uint16_t start, uint16_t end,
-                                        uint8_t access)
+static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG
+                                                struct domain *d,
+                                                uint32_t machine_bdf,
+                                                uint16_t start, uint16_t end,
+                                                uint8_t access)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1,
+                                         struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_remove_from_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_remove_from_physmap(XSM_DEFAULT_ARG struct domain *d1,
+                                              struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_map_gmfn_foreign(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_map_gmfn_foreign(XSM_DEFAULT_ARG struct domain *d,
+                                           struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_hvm_param(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_param(XSM_DEFAULT_ARG struct domain *d,
+                                    unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_control(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_control(XSM_DEFAULT_ARG struct domain *d,
+                                      unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -559,7 +597,8 @@ static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
+static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d,
+                                           uint64_t mode, uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
 
@@ -578,7 +617,8 @@ static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uin
     }
 }
 
-static XSM_INLINE int xsm_vm_event_control(XSM_DEFAULT_ARG struct domain *d, int mode, int op)
+static XSM_INLINE int xsm_vm_event_control(XSM_DEFAULT_ARG struct domain *d,
+                                           int mode, int op)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -621,13 +661,15 @@ static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_shadow_control(XSM_DEFAULT_ARG struct domain *d, uint32_t op)
+static XSM_INLINE int xsm_shadow_control(XSM_DEFAULT_ARG struct domain *d,
+                                         uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mem_sharing_op(XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
+static XSM_INLINE int xsm_mem_sharing_op(XSM_DEFAULT_ARG struct domain *d,
+                                         struct domain *cd, int op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, cd);
@@ -651,8 +693,9 @@ static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct domain *t,
-                                     struct domain *f, uint32_t flags)
+static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d,
+                                     struct domain *t, struct domain *f,
+                                     uint32_t flags)
 {
     int rc = 0;
     XSM_ASSERT_ACTION(XSM_TARGET);
@@ -663,38 +706,44 @@ static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct do
     return rc;
 }
 
-static XSM_INLINE int xsm_mmuext_op(XSM_DEFAULT_ARG struct domain *d, struct domain *f)
+static XSM_INLINE int xsm_mmuext_op(XSM_DEFAULT_ARG struct domain *d,
+                                    struct domain *f)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_update_va_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *f, 
-                                                            l1_pgentry_t pte)
+static XSM_INLINE int xsm_update_va_mapping(XSM_DEFAULT_ARG struct domain *d,
+                                            struct domain *f, l1_pgentry_t pte)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_priv_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_priv_mapping(XSM_DEFAULT_ARG struct domain *d,
+                                       struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_ioport_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_permission(XSM_DEFAULT_ARG struct domain *d,
+                                            uint32_t s, uint32_t e,
+                                            uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_ioport_mapping(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_mapping(XSM_DEFAULT_ARG struct domain *d,
+                                         uint32_t s, uint32_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct domain *d, unsigned int op)
+static XSM_INLINE int xsm_pmu_op(XSM_DEFAULT_ARG struct domain *d,
+                                 unsigned int op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
@@ -743,7 +792,7 @@ static XSM_INLINE int xsm_argo_send(const struct domain *d,
 #endif /* CONFIG_ARGO */
 
 #include <public/version.h>
-static XSM_INLINE int xsm_xen_version (XSM_DEFAULT_ARG uint32_t op)
+static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 9872bae502..8878281eae 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -19,7 +19,7 @@
 #include <xen/multiboot.h>
 
 /* policy magic number (defined by XSM_MAGIC) */
-typedef u32 xsm_magic_t;
+typedef uint32_t xsm_magic_t;
 
 #ifdef CONFIG_XSM_FLASK
 #define XSM_MAGIC 0xf97cff8c
@@ -31,158 +31,171 @@ typedef u32 xsm_magic_t;
  * default actions of XSM hooks. They should be compiled out otherwise.
  */
 enum xsm_default {
-    XSM_HOOK,     /* Guests can normally access the hypercall */
-    XSM_DM_PRIV,  /* Device model can perform on its target domain */
-    XSM_TARGET,   /* Can perform on self or your target domain */
-    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
-    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
-    XSM_OTHER     /* Something more complex */
+    /* Guests can normally access the hypercall */
+    XSM_HOOK,
+    /* Device model can perform on its target domain */
+    XSM_DM_PRIV,
+    /* Can perform on self or your target domain */
+    XSM_TARGET,
+    /* Privileged - normally restricted to dom0 */
+    XSM_PRIV,
+    /* Xenstore domain - can do some privileged operations */
+    XSM_XS_PRIV,
+    /* Something more complex */
+    XSM_OTHER
 };
 typedef enum xsm_default xsm_default_t;
 
 struct xsm_operations {
-    void (*security_domaininfo) (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info);
-    int (*domain_create) (struct domain *d, u32 ssidref);
-    int (*getdomaininfo) (struct domain *d);
-    int (*domctl_scheduler_op) (struct domain *d, int op);
-    int (*sysctl_scheduler_op) (int op);
-    int (*set_target) (struct domain *d, struct domain *e);
-    int (*domctl) (struct domain *d, int cmd);
-    int (*sysctl) (int cmd);
-    int (*readconsole) (uint32_t clear);
-
-    int (*evtchn_unbound) (struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain) (struct domain *d1, struct evtchn *chn1,
-                                        struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post) (struct evtchn *chn);
-    int (*evtchn_send) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_status) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset) (struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref) (struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref) (struct domain *d1, struct domain *d2);
-    int (*grant_setup) (struct domain *d1, struct domain *d2);
-    int (*grant_transfer) (struct domain *d1, struct domain *d2);
-    int (*grant_copy) (struct domain *d1, struct domain *d2);
-    int (*grant_query_size) (struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain) (struct domain *d);
-    void (*free_security_domain) (struct domain *d);
-    int (*alloc_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn) (struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain) (struct domain *d);
-
-    int (*get_pod_target) (struct domain *d);
-    int (*set_pod_target) (struct domain *d);
-    int (*memory_exchange) (struct domain *d);
-    int (*memory_adjust_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_pin_page) (struct domain *d1, struct domain *d2, struct page_info *page);
-    int (*add_to_physmap) (struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap) (struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign) (struct domain *d, struct domain *t);
-    int (*claim_pages) (struct domain *d);
-
-    int (*console_io) (struct domain *d, int cmd);
-
-    int (*profile) (struct domain *d, int op);
-
-    int (*kexec) (void);
-    int (*schedop_shutdown) (struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid) (int irq);
-    int (*map_domain_pirq) (struct domain *d);
-    int (*map_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq) (struct domain *d);
-    int (*unmap_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission) (struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*iomem_mapping) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*pci_config_permission) (struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access);
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group) (uint32_t machine_bdf);
-    int (*assign_device) (struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device) (struct domain *d, uint32_t machine_bdf);
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
 #endif
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice) (struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice) (struct domain *d, const char *dtpath);
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
 #endif
 
-    int (*resource_plug_core) (void);
-    int (*resource_unplug_core) (void);
-    int (*resource_plug_pci) (uint32_t machine_bdf);
-    int (*resource_unplug_pci) (uint32_t machine_bdf);
-    int (*resource_setup_pci) (uint32_t machine_bdf);
-    int (*resource_setup_gsi) (int gsi);
-    int (*resource_setup_misc) (void);
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
 
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
-    int (*hvm_param) (struct domain *d, unsigned long op);
-    int (*hvm_control) (struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm) (struct domain *d);
-    int (*hvm_altp2mhvm_op) (struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo) (struct domain *d);
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
 
-    int (*vm_event_control) (struct domain *d, int mode, int op);
+    int (*vm_event_control)(struct domain *d, int mode, int op);
 
 #ifdef CONFIG_MEM_ACCESS
-    int (*mem_access) (struct domain *d);
+    int (*mem_access)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-    int (*mem_paging) (struct domain *d);
+    int (*mem_paging)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing) (struct domain *d);
+    int (*mem_sharing)(struct domain *d);
 #endif
 
-    int (*platform_op) (uint32_t cmd);
+    int (*platform_op)(uint32_t cmd);
 
 #ifdef CONFIG_X86
-    int (*do_mca) (void);
-    int (*shadow_control) (struct domain *d, uint32_t op);
-    int (*mem_sharing_op) (struct domain *d, struct domain *cd, int op);
-    int (*apic) (struct domain *d, int cmd);
-    int (*memtype) (uint32_t access);
-    int (*machine_memory_map) (void);
-    int (*domain_memory_map) (struct domain *d);
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*memtype)(uint32_t access);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
 #define XSM_MMU_UPDATE_READ      1
 #define XSM_MMU_UPDATE_WRITE     2
 #define XSM_MMU_NORMAL_UPDATE    4
 #define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update) (struct domain *d, struct domain *t,
-                       struct domain *f, uint32_t flags);
-    int (*mmuext_op) (struct domain *d, struct domain *f);
-    int (*update_va_mapping) (struct domain *d, struct domain *f, l1_pgentry_t pte);
-    int (*priv_mapping) (struct domain *d, struct domain *t);
-    int (*ioport_permission) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*pmu_op) (struct domain *d, unsigned int op);
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
 #endif
-    int (*dm_op) (struct domain *d);
-    int (*xen_version) (uint32_t cmd);
-    int (*domain_resource_map) (struct domain *d);
+    int (*dm_op)(struct domain *d);
+    int (*xen_version)(uint32_t cmd);
+    int (*domain_resource_map)(struct domain *d);
 #ifdef CONFIG_ARGO
-    int (*argo_enable) (const struct domain *d);
-    int (*argo_register_single_source) (const struct domain *d,
-                                        const struct domain *t);
-    int (*argo_register_any_source) (const struct domain *d);
-    int (*argo_send) (const struct domain *d, const struct domain *t);
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
 #endif
 };
 
@@ -192,191 +205,210 @@ extern struct xsm_operations *xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
-static inline void xsm_security_domaininfo (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info)
+static inline void xsm_security_domaininfo(struct domain *d,
+    struct xen_domctl_getdomaininfo *info)
 {
     xsm_ops->security_domaininfo(d, info);
 }
 
-static inline int xsm_domain_create (xsm_default_t def, struct domain *d, u32 ssidref)
+static inline int xsm_domain_create(xsm_default_t def, struct domain *d,
+                                    uint32_t ssidref)
 {
     return xsm_ops->domain_create(d, ssidref);
 }
 
-static inline int xsm_getdomaininfo (xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->getdomaininfo(d);
 }
 
-static inline int xsm_domctl_scheduler_op (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl_scheduler_op(xsm_default_t def, struct domain *d,
+                                          int cmd)
 {
     return xsm_ops->domctl_scheduler_op(d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op (xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl_scheduler_op(cmd);
 }
 
-static inline int xsm_set_target (xsm_default_t def, struct domain *d, struct domain *e)
+static inline int xsm_set_target(xsm_default_t def, struct domain *d,
+                                 struct domain *e)
 {
     return xsm_ops->set_target(d, e);
 }
 
-static inline int xsm_domctl (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->domctl(d, cmd);
 }
 
-static inline int xsm_sysctl (xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl(cmd);
 }
 
-static inline int xsm_readconsole (xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
     return xsm_ops->readconsole(clear);
 }
 
-static inline int xsm_evtchn_unbound (xsm_default_t def, struct domain *d1, struct evtchn *chn,
-                                                                    domid_t id2)
+static inline int xsm_evtchn_unbound(xsm_default_t def, struct domain *d1,
+                                     struct evtchn *chn, domid_t id2)
 {
     return xsm_ops->evtchn_unbound(d1, chn, id2);
 }
 
-static inline int xsm_evtchn_interdomain (xsm_default_t def, struct domain *d1,
-                struct evtchn *chan1, struct domain *d2, struct evtchn *chan2)
+static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
+                                         struct evtchn *chan1,
+                                         struct domain *d2,
+                                         struct evtchn *chan2)
 {
     return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
-static inline void xsm_evtchn_close_post (struct evtchn *chn)
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
     xsm_ops->evtchn_close_post(chn);
 }
 
-static inline int xsm_evtchn_send (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_send(xsm_default_t def, struct domain *d,
+                                  struct evtchn *chn)
 {
     return xsm_ops->evtchn_send(d, chn);
 }
 
-static inline int xsm_evtchn_status (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_status(xsm_default_t def, struct domain *d,
+                                    struct evtchn *chn)
 {
     return xsm_ops->evtchn_status(d, chn);
 }
 
-static inline int xsm_evtchn_reset (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_evtchn_reset(xsm_default_t def, struct domain *d1,
+                                   struct domain *d2)
 {
     return xsm_ops->evtchn_reset(d1, d2);
 }
 
-static inline int xsm_grant_mapref (xsm_default_t def, struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static inline int xsm_grant_mapref(xsm_default_t def, struct domain *d1,
+                                   struct domain *d2, uint32_t flags)
 {
     return xsm_ops->grant_mapref(d1, d2, flags);
 }
 
-static inline int xsm_grant_unmapref (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_unmapref(xsm_default_t def, struct domain *d1,
+                                     struct domain *d2)
 {
     return xsm_ops->grant_unmapref(d1, d2);
 }
 
-static inline int xsm_grant_setup (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_setup(xsm_default_t def, struct domain *d1,
+                                  struct domain *d2)
 {
     return xsm_ops->grant_setup(d1, d2);
 }
 
-static inline int xsm_grant_transfer (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_transfer(xsm_default_t def, struct domain *d1,
+                                     struct domain *d2)
 {
     return xsm_ops->grant_transfer(d1, d2);
 }
 
-static inline int xsm_grant_copy (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_copy(xsm_default_t def, struct domain *d1,
+                                 struct domain *d2)
 {
     return xsm_ops->grant_copy(d1, d2);
 }
 
-static inline int xsm_grant_query_size (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_query_size(xsm_default_t def, struct domain *d1,
+                                       struct domain *d2)
 {
     return xsm_ops->grant_query_size(d1, d2);
 }
 
-static inline int xsm_alloc_security_domain (struct domain *d)
+static inline int xsm_alloc_security_domain(struct domain *d)
 {
     return xsm_ops->alloc_security_domain(d);
 }
 
-static inline void xsm_free_security_domain (struct domain *d)
+static inline void xsm_free_security_domain(struct domain *d)
 {
     xsm_ops->free_security_domain(d);
 }
 
-static inline int xsm_alloc_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+static inline int xsm_alloc_security_evtchns(struct evtchn chn[],
+                                             unsigned int nr)
 {
     return xsm_ops->alloc_security_evtchns(chn, nr);
 }
 
-static inline void xsm_free_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+static inline void xsm_free_security_evtchns(struct evtchn chn[],
+                                             unsigned int nr)
 {
     xsm_ops->free_security_evtchns(chn, nr);
 }
 
-static inline char *xsm_show_security_evtchn (struct domain *d, const struct evtchn *chn)
+static inline char *xsm_show_security_evtchn(struct domain *d,
+                                             const struct evtchn *chn)
 {
     return xsm_ops->show_security_evtchn(d, chn);
 }
 
-static inline int xsm_init_hardware_domain (xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->init_hardware_domain(d);
 }
 
-static inline int xsm_get_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_pod_target(d);
 }
 
-static inline int xsm_set_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->set_pod_target(d);
 }
 
-static inline int xsm_memory_exchange (xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->memory_exchange(d);
 }
 
-static inline int xsm_memory_adjust_reservation (xsm_default_t def, struct domain *d1, struct
-                                                                    domain *d2)
+static inline int xsm_memory_adjust_reservation(xsm_default_t def,
+                                                struct domain *d1,
+                                                struct domain *d2)
 {
     return xsm_ops->memory_adjust_reservation(d1, d2);
 }
 
-static inline int xsm_memory_stat_reservation (xsm_default_t def, struct domain *d1,
-                                                            struct domain *d2)
+static inline int xsm_memory_stat_reservation(xsm_default_t def,
+                                              struct domain *d1,
+                                              struct domain *d2)
 {
     return xsm_ops->memory_stat_reservation(d1, d2);
 }
 
-static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1, struct domain *d2,
-                                      struct page_info *page)
+static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1,
+                                      struct domain *d2, struct page_info *page)
 {
     return xsm_ops->memory_pin_page(d1, d2, page);
 }
 
-static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1,
+                                     struct domain *d2)
 {
     return xsm_ops->add_to_physmap(d1, d2);
 }
 
-static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1,
+                                          struct domain *d2)
 {
     return xsm_ops->remove_from_physmap(d1, d2);
 }
 
-static inline int xsm_map_gmfn_foreign (xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_map_gmfn_foreign(xsm_default_t def, struct domain *d,
+                                       struct domain *t)
 {
     return xsm_ops->map_gmfn_foreign(d, t);
 }
@@ -386,47 +418,50 @@ static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
     return xsm_ops->claim_pages(d);
 }
 
-static inline int xsm_console_io (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->console_io(d, cmd);
 }
 
-static inline int xsm_profile (xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
     return xsm_ops->profile(d, op);
 }
 
-static inline int xsm_kexec (xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t def)
 {
     return xsm_ops->kexec();
 }
 
-static inline int xsm_schedop_shutdown (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_schedop_shutdown(xsm_default_t def, struct domain *d1,
+                                       struct domain *d2)
 {
     return xsm_ops->schedop_shutdown(d1, d2);
 }
 
-static inline char *xsm_show_irq_sid (int irq)
+static inline char *xsm_show_irq_sid(int irq)
 {
     return xsm_ops->show_irq_sid(irq);
 }
 
-static inline int xsm_map_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->map_domain_pirq(d);
 }
 
-static inline int xsm_map_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_map_domain_irq(xsm_default_t def, struct domain *d,
+                                     int irq, void *data)
 {
     return xsm_ops->map_domain_irq(d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->unmap_domain_pirq(d);
 }
 
-static inline int xsm_unmap_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_unmap_domain_irq(xsm_default_t def, struct domain *d,
+                                       int irq, void *data)
 {
     return xsm_ops->unmap_domain_irq(d, irq, data);
 }
@@ -443,22 +478,28 @@ static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
     return xsm_ops->unbind_pt_irq(d, bind);
 }
 
-static inline int xsm_irq_permission (xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
+static inline int xsm_irq_permission(xsm_default_t def, struct domain *d,
+                                     int pirq, uint8_t allow)
 {
     return xsm_ops->irq_permission(d, pirq, allow);
 }
 
-static inline int xsm_iomem_permission (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_permission(xsm_default_t def, struct domain *d,
+                                       uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_permission(d, s, e, allow);
 }
 
-static inline int xsm_iomem_mapping (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_mapping(xsm_default_t def, struct domain *d,
+                                    uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pci_config_permission (xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access)
+static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
+                                            uint32_t machine_bdf,
+                                            uint16_t start, uint16_t end,
+                                            uint8_t access)
 {
     return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
 }
@@ -469,12 +510,14 @@ static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
     return xsm_ops->get_device_group(machine_bdf);
 }
 
-static inline int xsm_assign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_assign_device(xsm_default_t def, struct domain *d,
+                                    uint32_t machine_bdf)
 {
     return xsm_ops->assign_device(d, machine_bdf);
 }
 
-static inline int xsm_deassign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
+                                      uint32_t machine_bdf)
 {
     return xsm_ops->deassign_device(d, machine_bdf);
 }
@@ -495,37 +538,39 @@ static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_plug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_unplug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_unplug_pci(xsm_default_t def,
+                                          uint32_t machine_bdf)
 {
     return xsm_ops->resource_unplug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_plug_core (xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t def)
 {
     return xsm_ops->resource_plug_core();
 }
 
-static inline int xsm_resource_unplug_core (xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
     return xsm_ops->resource_unplug_core();
 }
 
-static inline int xsm_resource_setup_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_setup_pci(xsm_default_t def,
+                                         uint32_t machine_bdf)
 {
     return xsm_ops->resource_setup_pci(machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi (xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
     return xsm_ops->resource_setup_gsi(gsi);
 }
 
-static inline int xsm_resource_setup_misc (xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
     return xsm_ops->resource_setup_misc();
 }
@@ -540,70 +585,74 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
 #endif
 
-static inline int xsm_hvm_param (xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_param(xsm_default_t def, struct domain *d,
+                                unsigned long op)
 {
     return xsm_ops->hvm_param(d, op);
 }
 
-static inline int xsm_hvm_control(xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_control(xsm_default_t def, struct domain *d,
+                                  unsigned long op)
 {
     return xsm_ops->hvm_control(d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm (xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->hvm_param_altp2mhvm(d);
 }
 
-static inline int xsm_hvm_altp2mhvm_op (xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
+static inline int xsm_hvm_altp2mhvm_op(xsm_default_t def, struct domain *d,
+                                       uint64_t mode, uint32_t op)
 {
     return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo (xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_vnumainfo(d);
 }
 
-static inline int xsm_vm_event_control (xsm_default_t def, struct domain *d, int mode, int op)
+static inline int xsm_vm_event_control(xsm_default_t def, struct domain *d,
+                                       int mode, int op)
 {
     return xsm_ops->vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_sharing(d);
 }
 #endif
 
-static inline int xsm_platform_op (xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->platform_op(op);
 }
@@ -614,22 +663,24 @@ static inline int xsm_do_mca(xsm_default_t def)
     return xsm_ops->do_mca();
 }
 
-static inline int xsm_shadow_control (xsm_default_t def, struct domain *d, uint32_t op)
+static inline int xsm_shadow_control(xsm_default_t def, struct domain *d,
+                                     uint32_t op)
 {
     return xsm_ops->shadow_control(d, op);
 }
 
-static inline int xsm_mem_sharing_op (xsm_default_t def, struct domain *d, struct domain *cd, int op)
+static inline int xsm_mem_sharing_op(xsm_default_t def, struct domain *d,
+                                     struct domain *cd, int op)
 {
     return xsm_ops->mem_sharing_op(d, cd, op);
 }
 
-static inline int xsm_apic (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->apic(d, cmd);
 }
 
-static inline int xsm_memtype (xsm_default_t def, uint32_t access)
+static inline int xsm_memtype(xsm_default_t def, uint32_t access)
 {
     return xsm_ops->memtype(access);
 }
@@ -644,39 +695,45 @@ static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
     return xsm_ops->domain_memory_map(d);
 }
 
-static inline int xsm_mmu_update (xsm_default_t def, struct domain *d, struct domain *t,
-                                  struct domain *f, uint32_t flags)
+static inline int xsm_mmu_update(xsm_default_t def, struct domain *d,
+                                 struct domain *t, struct domain *f,
+                                 uint32_t flags)
 {
     return xsm_ops->mmu_update(d, t, f, flags);
 }
 
-static inline int xsm_mmuext_op (xsm_default_t def, struct domain *d, struct domain *f)
+static inline int xsm_mmuext_op(xsm_default_t def, struct domain *d,
+                                struct domain *f)
 {
     return xsm_ops->mmuext_op(d, f);
 }
 
-static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d, struct domain *f,
-                                                            l1_pgentry_t pte)
+static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d,
+                                        struct domain *f, l1_pgentry_t pte)
 {
     return xsm_ops->update_va_mapping(d, f, pte);
 }
 
-static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d,
+                                   struct domain *t)
 {
     return xsm_ops->priv_mapping(d, t);
 }
 
-static inline int xsm_ioport_permission (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_permission(xsm_default_t def, struct domain *d,
+                                        uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_permission(d, s, e, allow);
 }
 
-static inline int xsm_ioport_mapping (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_mapping(xsm_default_t def, struct domain *d,
+                                     uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pmu_op (xsm_default_t def, struct domain *d, unsigned int op)
+static inline int xsm_pmu_op(xsm_default_t def, struct domain *d,
+                             unsigned int op)
 {
     return xsm_ops->pmu_op(d, op);
 }
@@ -688,7 +745,7 @@ static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
     return xsm_ops->dm_op(d);
 }
 
-static inline int xsm_xen_version (xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->xen_version(op);
 }
@@ -725,9 +782,9 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 #endif /* XSM_NO_WRAPPERS */
 
 #ifdef CONFIG_MULTIBOOT
-extern int xsm_multiboot_init(unsigned long *module_map,
+int xsm_multiboot_init(unsigned long *module_map,
                               const multiboot_info_t *mbi);
-extern int xsm_multiboot_policy_init(unsigned long *module_map,
+int xsm_multiboot_policy_init(unsigned long *module_map,
                                      const multiboot_info_t *mbi,
                                      void **policy_buffer,
                                      size_t *policy_size);
@@ -739,18 +796,19 @@ extern int xsm_multiboot_policy_init(unsigned long *module_map,
  *
  * On success, return 1 if using SILO mode else 0.
  */
-extern int xsm_dt_init(void);
-extern int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-extern bool has_xsm_magic(paddr_t);
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
 #endif
 
-extern int register_xsm(struct xsm_operations *ops);
+int register_xsm(struct xsm_operations *ops);
 
 extern struct xsm_operations dummy_xsm_ops;
-extern void xsm_fixup_ops(struct xsm_operations *ops);
+
+void xsm_fixup_ops(struct xsm_operations *ops);
 
 #ifdef CONFIG_XSM_FLASK
-extern void flask_init(const void *policy_buffer, size_t policy_size);
+void flask_init(const void *policy_buffer, size_t policy_size);
 #else
 static inline void flask_init(const void *policy_buffer, size_t policy_size)
 {
@@ -763,7 +821,7 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-extern void silo_init(void);
+void silo_init(void);
 #else
 static inline void silo_init(void) {}
 #endif
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index ac553f9c0d..55483292c5 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -128,8 +128,8 @@ int __init xsm_multiboot_init(unsigned long *module_map,
 
     if ( XSM_MAGIC )
     {
-        ret = xsm_multiboot_policy_init(module_map, mbi,
-                                        &policy_buffer, &policy_size);
+        ret = xsm_multiboot_policy_init(module_map, mbi, &policy_buffer,
+                                        &policy_size);
         if ( ret )
         {
             bootstrap_map(NULL);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:07:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:07:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178150.324048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnG-0008Qv-Gx; Fri, 03 Sep 2021 15:07:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178150.324048; Fri, 03 Sep 2021 15:07:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnG-0008Qo-Dl; Fri, 03 Sep 2021 15:07:50 +0000
Received: by outflank-mailman (input) for mailman id 178150;
 Fri, 03 Sep 2021 15:07:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAnF-0008Ng-0V
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:07:49 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b54bf82a-424d-4da4-8874-e24ba0bfc054;
 Fri, 03 Sep 2021 15:07:46 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681603149519.7405960574835;
 Fri, 3 Sep 2021 08:06:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b54bf82a-424d-4da4-8874-e24ba0bfc054
ARC-Seal: i=1; a=rsa-sha256; t=1630681606; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=eIFKDRxScrmARw/p8OWVj6UYH+9U3O5yIrXvnRa08cFoYdq+RvT5pR92aEvueq3tpvbGYI0jYPgxSlTaGLWbqoQTTTxarh1yJ7xFqbbBTS3dh+b+HTiBKYxqw7pBxNEAfJSP/c8X+y/PBYCfLamYzE0aytfL34v5IbGLatfg3WQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681606; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=X6+g8YK8ugcXGm10Da/blBL4IQsisq76l+m3qPb65X4=; 
	b=gNH2ChyRgro6MyqW8nJEXxu8GLOuVXGx57dOyP4yCnDrwhKk0Lt9SmW5H6/vonGtIol3mRNOReEFgBGMwM0ZaCCEa9r8mSi9FBlsWuALLxoFmMtgiPK2Pyi78I0gjp+3j11n7HGekBX2oLGgplsCsE9k1+rhWcoJeSwMpICuCJY=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681606;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=X6+g8YK8ugcXGm10Da/blBL4IQsisq76l+m3qPb65X4=;
	b=Y9JeDUu8fY+J7m3/RFW79S1nkc1pVjIcfOvBS7yPevkL6IBYpOiaP0uEChQbd3p8
	bx3LXocsuxNUqYmVok5VgqvyeU43bIABNkASSoaoP7dSMb5j+3lDMnxiVSD9fzncfhy
	8zR7o4Ljpd70au/OZPJX0YebnuDJyB2ezDIyhTks=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 05/11] xsm: refactor xsm_ops handling
Date: Fri,  3 Sep 2021 15:06:23 -0400
Message-Id: <20210903190629.11917-6-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

This renames the `struct xsm_operations` to the shorter `struct xsm_ops` and
converts the global xsm_ops from being a pointer to an explicit instance. As
part of this conversion, it reworks the XSM modules init function to return
their xsm_ops struct which is copied in to the global xsm_ops instance.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/xsm.h | 222 +++++++++++++++++++++---------------------
 xen/xsm/dummy.c       |   4 +-
 xen/xsm/flask/hooks.c |  12 +--
 xen/xsm/silo.c        |   7 +-
 xen/xsm/xsm_core.c    |  77 ++++++++-------
 5 files changed, 167 insertions(+), 155 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 8878281eae..3888172045 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -46,7 +46,7 @@ enum xsm_default {
 };
 typedef enum xsm_default xsm_default_t;
 
-struct xsm_operations {
+struct xsm_ops {
     void (*security_domaininfo)(struct domain *d,
                                 struct xen_domctl_getdomaininfo *info);
     int (*domain_create)(struct domain *d, uint32_t ssidref);
@@ -201,63 +201,63 @@ struct xsm_operations {
 
 #ifdef CONFIG_XSM
 
-extern struct xsm_operations *xsm_ops;
+extern struct xsm_ops xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
 static inline void xsm_security_domaininfo(struct domain *d,
     struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops->security_domaininfo(d, info);
+    xsm_ops.security_domaininfo(d, info);
 }
 
 static inline int xsm_domain_create(xsm_default_t def, struct domain *d,
                                     uint32_t ssidref)
 {
-    return xsm_ops->domain_create(d, ssidref);
+    return xsm_ops.domain_create(d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->getdomaininfo(d);
+    return xsm_ops.getdomaininfo(d);
 }
 
 static inline int xsm_domctl_scheduler_op(xsm_default_t def, struct domain *d,
                                           int cmd)
 {
-    return xsm_ops->domctl_scheduler_op(d, cmd);
+    return xsm_ops.domctl_scheduler_op(d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl_scheduler_op(cmd);
+    return xsm_ops.sysctl_scheduler_op(cmd);
 }
 
 static inline int xsm_set_target(xsm_default_t def, struct domain *d,
                                  struct domain *e)
 {
-    return xsm_ops->set_target(d, e);
+    return xsm_ops.set_target(d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->domctl(d, cmd);
+    return xsm_ops.domctl(d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl(cmd);
+    return xsm_ops.sysctl(cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops->readconsole(clear);
+    return xsm_ops.readconsole(clear);
 }
 
 static inline int xsm_evtchn_unbound(xsm_default_t def, struct domain *d1,
                                      struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops->evtchn_unbound(d1, chn, id2);
+    return xsm_ops.evtchn_unbound(d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
@@ -265,235 +265,235 @@ static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
                                          struct domain *d2,
                                          struct evtchn *chan2)
 {
-    return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
+    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops->evtchn_close_post(chn);
+    xsm_ops.evtchn_close_post(chn);
 }
 
 static inline int xsm_evtchn_send(xsm_default_t def, struct domain *d,
                                   struct evtchn *chn)
 {
-    return xsm_ops->evtchn_send(d, chn);
+    return xsm_ops.evtchn_send(d, chn);
 }
 
 static inline int xsm_evtchn_status(xsm_default_t def, struct domain *d,
                                     struct evtchn *chn)
 {
-    return xsm_ops->evtchn_status(d, chn);
+    return xsm_ops.evtchn_status(d, chn);
 }
 
 static inline int xsm_evtchn_reset(xsm_default_t def, struct domain *d1,
                                    struct domain *d2)
 {
-    return xsm_ops->evtchn_reset(d1, d2);
+    return xsm_ops.evtchn_reset(d1, d2);
 }
 
 static inline int xsm_grant_mapref(xsm_default_t def, struct domain *d1,
                                    struct domain *d2, uint32_t flags)
 {
-    return xsm_ops->grant_mapref(d1, d2, flags);
+    return xsm_ops.grant_mapref(d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops->grant_unmapref(d1, d2);
+    return xsm_ops.grant_unmapref(d1, d2);
 }
 
 static inline int xsm_grant_setup(xsm_default_t def, struct domain *d1,
                                   struct domain *d2)
 {
-    return xsm_ops->grant_setup(d1, d2);
+    return xsm_ops.grant_setup(d1, d2);
 }
 
 static inline int xsm_grant_transfer(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops->grant_transfer(d1, d2);
+    return xsm_ops.grant_transfer(d1, d2);
 }
 
 static inline int xsm_grant_copy(xsm_default_t def, struct domain *d1,
                                  struct domain *d2)
 {
-    return xsm_ops->grant_copy(d1, d2);
+    return xsm_ops.grant_copy(d1, d2);
 }
 
 static inline int xsm_grant_query_size(xsm_default_t def, struct domain *d1,
                                        struct domain *d2)
 {
-    return xsm_ops->grant_query_size(d1, d2);
+    return xsm_ops.grant_query_size(d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops->alloc_security_domain(d);
+    return xsm_ops.alloc_security_domain(d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops->free_security_domain(d);
+    xsm_ops.free_security_domain(d);
 }
 
 static inline int xsm_alloc_security_evtchns(struct evtchn chn[],
                                              unsigned int nr)
 {
-    return xsm_ops->alloc_security_evtchns(chn, nr);
+    return xsm_ops.alloc_security_evtchns(chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(struct evtchn chn[],
                                              unsigned int nr)
 {
-    xsm_ops->free_security_evtchns(chn, nr);
+    xsm_ops.free_security_evtchns(chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(struct domain *d,
                                              const struct evtchn *chn)
 {
-    return xsm_ops->show_security_evtchn(d, chn);
+    return xsm_ops.show_security_evtchn(d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->init_hardware_domain(d);
+    return xsm_ops.init_hardware_domain(d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_pod_target(d);
+    return xsm_ops.get_pod_target(d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->set_pod_target(d);
+    return xsm_ops.set_pod_target(d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->memory_exchange(d);
+    return xsm_ops.memory_exchange(d);
 }
 
 static inline int xsm_memory_adjust_reservation(xsm_default_t def,
                                                 struct domain *d1,
                                                 struct domain *d2)
 {
-    return xsm_ops->memory_adjust_reservation(d1, d2);
+    return xsm_ops.memory_adjust_reservation(d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(xsm_default_t def,
                                               struct domain *d1,
                                               struct domain *d2)
 {
-    return xsm_ops->memory_stat_reservation(d1, d2);
+    return xsm_ops.memory_stat_reservation(d1, d2);
 }
 
 static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1,
                                       struct domain *d2, struct page_info *page)
 {
-    return xsm_ops->memory_pin_page(d1, d2, page);
+    return xsm_ops.memory_pin_page(d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops->add_to_physmap(d1, d2);
+    return xsm_ops.add_to_physmap(d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1,
                                           struct domain *d2)
 {
-    return xsm_ops->remove_from_physmap(d1, d2);
+    return xsm_ops.remove_from_physmap(d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(xsm_default_t def, struct domain *d,
                                        struct domain *t)
 {
-    return xsm_ops->map_gmfn_foreign(d, t);
+    return xsm_ops.map_gmfn_foreign(d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->claim_pages(d);
+    return xsm_ops.claim_pages(d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->console_io(d, cmd);
+    return xsm_ops.console_io(d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops->profile(d, op);
+    return xsm_ops.profile(d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops->kexec();
+    return xsm_ops.kexec();
 }
 
 static inline int xsm_schedop_shutdown(xsm_default_t def, struct domain *d1,
                                        struct domain *d2)
 {
-    return xsm_ops->schedop_shutdown(d1, d2);
+    return xsm_ops.schedop_shutdown(d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops->show_irq_sid(irq);
+    return xsm_ops.show_irq_sid(irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->map_domain_pirq(d);
+    return xsm_ops.map_domain_pirq(d);
 }
 
 static inline int xsm_map_domain_irq(xsm_default_t def, struct domain *d,
                                      int irq, void *data)
 {
-    return xsm_ops->map_domain_irq(d, irq, data);
+    return xsm_ops.map_domain_irq(d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->unmap_domain_pirq(d);
+    return xsm_ops.unmap_domain_pirq(d);
 }
 
 static inline int xsm_unmap_domain_irq(xsm_default_t def, struct domain *d,
                                        int irq, void *data)
 {
-    return xsm_ops->unmap_domain_irq(d, irq, data);
+    return xsm_ops.unmap_domain_irq(d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(xsm_default_t def, struct domain *d,
                                   struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->bind_pt_irq(d, bind);
+    return xsm_ops.bind_pt_irq(d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
                                     struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->unbind_pt_irq(d, bind);
+    return xsm_ops.unbind_pt_irq(d, bind);
 }
 
 static inline int xsm_irq_permission(xsm_default_t def, struct domain *d,
                                      int pirq, uint8_t allow)
 {
-    return xsm_ops->irq_permission(d, pirq, allow);
+    return xsm_ops.irq_permission(d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(xsm_default_t def, struct domain *d,
                                        uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_permission(d, s, e, allow);
+    return xsm_ops.iomem_permission(d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(xsm_default_t def, struct domain *d,
                                     uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_mapping(d, s, e, allow);
+    return xsm_ops.iomem_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
@@ -501,25 +501,25 @@ static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
                                             uint16_t start, uint16_t end,
                                             uint8_t access)
 {
-    return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
+    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->get_device_group(machine_bdf);
+    return xsm_ops.get_device_group(machine_bdf);
 }
 
 static inline int xsm_assign_device(xsm_default_t def, struct domain *d,
                                     uint32_t machine_bdf)
 {
-    return xsm_ops->assign_device(d, machine_bdf);
+    return xsm_ops.assign_device(d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
                                       uint32_t machine_bdf)
 {
-    return xsm_ops->deassign_device(d, machine_bdf);
+    return xsm_ops.deassign_device(d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -527,254 +527,254 @@ static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
 static inline int xsm_assign_dtdevice(xsm_default_t def, struct domain *d,
                                       const char *dtpath)
 {
-    return xsm_ops->assign_dtdevice(d, dtpath);
+    return xsm_ops.assign_dtdevice(d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
                                         const char *dtpath)
 {
-    return xsm_ops->deassign_dtdevice(d, dtpath);
+    return xsm_ops.deassign_dtdevice(d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_plug_pci(machine_bdf);
+    return xsm_ops.resource_plug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(xsm_default_t def,
                                           uint32_t machine_bdf)
 {
-    return xsm_ops->resource_unplug_pci(machine_bdf);
+    return xsm_ops.resource_unplug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_plug_core();
+    return xsm_ops.resource_plug_core();
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_unplug_core();
+    return xsm_ops.resource_unplug_core();
 }
 
 static inline int xsm_resource_setup_pci(xsm_default_t def,
                                          uint32_t machine_bdf)
 {
-    return xsm_ops->resource_setup_pci(machine_bdf);
+    return xsm_ops.resource_setup_pci(machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops->resource_setup_gsi(gsi);
+    return xsm_ops.resource_setup_gsi(gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops->resource_setup_misc();
+    return xsm_ops.resource_setup_misc();
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops->page_offline(cmd);
+    return xsm_ops.page_offline(cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops->hypfs_op();
+    return xsm_ops.hypfs_op();
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_xsm_op(op);
+    return xsm_ops.do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
 static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_compat_op(op);
+    return xsm_ops.do_compat_op(op);
 }
 #endif
 
 static inline int xsm_hvm_param(xsm_default_t def, struct domain *d,
                                 unsigned long op)
 {
-    return xsm_ops->hvm_param(d, op);
+    return xsm_ops.hvm_param(d, op);
 }
 
 static inline int xsm_hvm_control(xsm_default_t def, struct domain *d,
                                   unsigned long op)
 {
-    return xsm_ops->hvm_control(d, op);
+    return xsm_ops.hvm_control(d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->hvm_param_altp2mhvm(d);
+    return xsm_ops.hvm_param_altp2mhvm(d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(xsm_default_t def, struct domain *d,
                                        uint64_t mode, uint32_t op)
 {
-    return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
+    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_vnumainfo(d);
+    return xsm_ops.get_vnumainfo(d);
 }
 
 static inline int xsm_vm_event_control(xsm_default_t def, struct domain *d,
                                        int mode, int op)
 {
-    return xsm_ops->vm_event_control(d, mode, op);
+    return xsm_ops.vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_access(d);
+    return xsm_ops.mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_paging(d);
+    return xsm_ops.mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_sharing(d);
+    return xsm_ops.mem_sharing(d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->platform_op(op);
+    return xsm_ops.platform_op(op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops->do_mca();
+    return xsm_ops.do_mca();
 }
 
 static inline int xsm_shadow_control(xsm_default_t def, struct domain *d,
                                      uint32_t op)
 {
-    return xsm_ops->shadow_control(d, op);
+    return xsm_ops.shadow_control(d, op);
 }
 
 static inline int xsm_mem_sharing_op(xsm_default_t def, struct domain *d,
                                      struct domain *cd, int op)
 {
-    return xsm_ops->mem_sharing_op(d, cd, op);
+    return xsm_ops.mem_sharing_op(d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->apic(d, cmd);
+    return xsm_ops.apic(d, cmd);
 }
 
 static inline int xsm_memtype(xsm_default_t def, uint32_t access)
 {
-    return xsm_ops->memtype(access);
+    return xsm_ops.memtype(access);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops->machine_memory_map();
+    return xsm_ops.machine_memory_map();
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_memory_map(d);
+    return xsm_ops.domain_memory_map(d);
 }
 
 static inline int xsm_mmu_update(xsm_default_t def, struct domain *d,
                                  struct domain *t, struct domain *f,
                                  uint32_t flags)
 {
-    return xsm_ops->mmu_update(d, t, f, flags);
+    return xsm_ops.mmu_update(d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(xsm_default_t def, struct domain *d,
                                 struct domain *f)
 {
-    return xsm_ops->mmuext_op(d, f);
+    return xsm_ops.mmuext_op(d, f);
 }
 
 static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d,
                                         struct domain *f, l1_pgentry_t pte)
 {
-    return xsm_ops->update_va_mapping(d, f, pte);
+    return xsm_ops.update_va_mapping(d, f, pte);
 }
 
 static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d,
                                    struct domain *t)
 {
-    return xsm_ops->priv_mapping(d, t);
+    return xsm_ops.priv_mapping(d, t);
 }
 
 static inline int xsm_ioport_permission(xsm_default_t def, struct domain *d,
                                         uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_permission(d, s, e, allow);
+    return xsm_ops.ioport_permission(d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(xsm_default_t def, struct domain *d,
                                      uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_mapping(d, s, e, allow);
+    return xsm_ops.ioport_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(xsm_default_t def, struct domain *d,
                              unsigned int op)
 {
-    return xsm_ops->pmu_op(d, op);
+    return xsm_ops.pmu_op(d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->dm_op(d);
+    return xsm_ops.dm_op(d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->xen_version(op);
+    return xsm_ops.xen_version(op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_resource_map(d);
+    return xsm_ops.domain_resource_map(d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops->argo_enable(d);
+    return xsm_ops.argo_enable(d);
 }
 
 static inline int xsm_argo_register_single_source(const struct domain *d,
                                                   const struct domain *t)
 {
-    return xsm_ops->argo_register_single_source(d, t);
+    return xsm_ops.argo_register_single_source(d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops->argo_register_any_source(d);
+    return xsm_ops.argo_register_any_source(d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops->argo_send(d, t);
+    return xsm_ops.argo_send(d, t);
 }
 
 #endif /* CONFIG_ARGO */
@@ -801,17 +801,18 @@ int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
 bool has_xsm_magic(paddr_t);
 #endif
 
-int register_xsm(struct xsm_operations *ops);
+extern struct xsm_ops dummy_xsm_ops;
 
-extern struct xsm_operations dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_operations *ops);
+void xsm_fixup_ops(struct xsm_ops *ops);
 
 #ifdef CONFIG_XSM_FLASK
-void flask_init(const void *policy_buffer, size_t policy_size);
+extern const struct xsm_ops *flask_init(const void *policy_buffer,
+                                        size_t policy_size);
 #else
-static inline void flask_init(const void *policy_buffer, size_t policy_size)
+static const inline struct xsm_ops *flask_init(const void *policy_buffer,
+                                               size_t policy_size)
 {
+    return NULL;
 }
 #endif
 
@@ -821,9 +822,12 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-void silo_init(void);
+extern const struct xsm_ops *silo_init(void);
 #else
-static inline void silo_init(void) {}
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
 #endif
 
 #else /* CONFIG_XSM */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index de44b10130..d8c935328e 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -13,15 +13,13 @@
 #define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
-struct xsm_operations dummy_xsm_ops;
-
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
         if ( !ops->function )                                          \
             ops->function = xsm_##function;                            \
     } while (0)
 
-void __init xsm_fixup_ops (struct xsm_operations *ops)
+void __init xsm_fixup_ops (struct xsm_ops *ops)
 {
     set_to_dummy_if_null(ops, security_domaininfo);
     set_to_dummy_if_null(ops, domain_create);
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 1465db125a..25e87180b4 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1745,7 +1745,7 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
-static struct xsm_operations flask_ops = {
+static const struct xsm_ops __initconstrel flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
     .domain_create = flask_domain_create,
     .getdomaininfo = flask_getdomaininfo,
@@ -1883,7 +1883,8 @@ static struct xsm_operations flask_ops = {
 #endif
 };
 
-void __init flask_init(const void *policy_buffer, size_t policy_size)
+const struct xsm_ops *__init flask_init(const void *policy_buffer,
+                                        size_t policy_size)
 {
     int ret = -ENOENT;
 
@@ -1891,7 +1892,7 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
     {
     case FLASK_BOOTPARAM_DISABLED:
         printk(XENLOG_INFO "Flask: Disabled at boot.\n");
-        return;
+        return NULL;
 
     case FLASK_BOOTPARAM_PERMISSIVE:
         flask_enforcing = 0;
@@ -1908,9 +1909,6 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
 
     avc_init();
 
-    if ( register_xsm(&flask_ops) )
-        panic("Flask: Unable to register with XSM\n");
-
     if ( policy_size && flask_bootparam != FLASK_BOOTPARAM_LATELOAD )
         ret = security_load_policy(policy_buffer, policy_size);
 
@@ -1923,6 +1921,8 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
         printk(XENLOG_INFO "Flask:  Starting in enforcing mode.\n");
     else
         printk(XENLOG_INFO "Flask:  Starting in permissive mode.\n");
+
+    return &flask_ops;
 }
 
 /*
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index fc2ca5cd2d..3550dded7b 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -100,7 +100,7 @@ static int silo_argo_send(const struct domain *d1, const struct domain *d2)
 
 #endif
 
-static struct xsm_operations silo_xsm_ops = {
+static const struct xsm_ops __initconstrel silo_xsm_ops = {
     .evtchn_unbound = silo_evtchn_unbound,
     .evtchn_interdomain = silo_evtchn_interdomain,
     .grant_mapref = silo_grant_mapref,
@@ -112,12 +112,11 @@ static struct xsm_operations silo_xsm_ops = {
 #endif
 };
 
-void __init silo_init(void)
+const struct xsm_ops *__init silo_init(void)
 {
     printk("Initialising XSM SILO mode\n");
 
-    if ( register_xsm(&silo_xsm_ops) )
-        panic("SILO: Unable to register with XSM\n");
+    return &silo_xsm_ops;
 }
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 55483292c5..859af3fe9a 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -28,9 +28,17 @@
 #include <asm/setup.h>
 #endif
 
-#define XSM_FRAMEWORK_VERSION    "1.0.0"
+#define XSM_FRAMEWORK_VERSION    "1.0.1"
 
-struct xsm_operations *xsm_ops;
+struct xsm_ops __read_mostly xsm_ops;
+
+enum xsm_ops_state {
+    XSM_OPS_UNREGISTERED,
+    XSM_OPS_REG_FAILED,
+    XSM_OPS_REGISTERED,
+};
+
+static enum xsm_ops_state xsm_ops_registered = XSM_OPS_UNREGISTERED;
 
 enum xsm_bootparam {
     XSM_BOOTPARAM_DUMMY,
@@ -68,17 +76,10 @@ static int __init parse_xsm_param(const char *s)
 }
 custom_param("xsm", parse_xsm_param);
 
-static inline int verify(struct xsm_operations *ops)
-{
-    /* verify the security_operations structure exists */
-    if ( !ops )
-        return -EINVAL;
-    xsm_fixup_ops(ops);
-    return 0;
-}
-
 static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
 {
+    const struct xsm_ops *ops = NULL;
+
 #ifdef CONFIG_XSM_FLASK_POLICY
     if ( policy_size == 0 )
     {
@@ -87,25 +88,35 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
     }
 #endif
 
-    if ( verify(&dummy_xsm_ops) )
+    if ( xsm_ops_registered != XSM_OPS_UNREGISTERED )
     {
-        printk(XENLOG_ERR "Could not verify dummy_xsm_ops structure\n");
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register already attempted\n");
         return -EIO;
     }
 
-    xsm_ops = &dummy_xsm_ops;
-
     switch ( xsm_bootparam )
     {
     case XSM_BOOTPARAM_DUMMY:
+        xsm_ops_registered = XSM_OPS_REGISTERED;
         break;
 
     case XSM_BOOTPARAM_FLASK:
-        flask_init(policy_buffer, policy_size);
+        ops = flask_init(policy_buffer, policy_size);
+        if ( ops )
+        {
+            xsm_ops_registered = XSM_OPS_REGISTERED;
+            xsm_ops = *ops;
+        }
         break;
 
     case XSM_BOOTPARAM_SILO:
-        silo_init();
+        ops = silo_init();
+        if ( ops )
+        {
+            xsm_ops_registered = XSM_OPS_REGISTERED;
+            xsm_ops = *ops;
+        }
         break;
 
     default:
@@ -113,6 +124,22 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
         break;
     }
 
+    /*
+     * This handles three cases,
+     *   - dummy policy module was selected
+     *   - a policy module does not provide all handlers
+     *   - a policy module failed to init
+     */
+    xsm_fixup_ops(&xsm_ops);
+
+    if ( xsm_ops_registered != XSM_OPS_REGISTERED )
+    {
+        xsm_ops_registered = XSM_OPS_REG_FAILED;
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register failed\n");
+        return -EFAULT;
+    }
+
     return 0;
 }
 
@@ -195,22 +222,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-int __init register_xsm(struct xsm_operations *ops)
-{
-    if ( verify(ops) )
-    {
-        printk(XENLOG_ERR "Could not verify xsm_operations structure\n");
-        return -EINVAL;
-    }
-
-    if ( xsm_ops != &dummy_xsm_ops )
-        return -EAGAIN;
-
-    xsm_ops = ops;
-
-    return 0;
-}
-
 #endif
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:07:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:07:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178154.324060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnO-0000SB-U2; Fri, 03 Sep 2021 15:07:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178154.324060; Fri, 03 Sep 2021 15:07:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnO-0000S0-Qq; Fri, 03 Sep 2021 15:07:58 +0000
Received: by outflank-mailman (input) for mailman id 178154;
 Fri, 03 Sep 2021 15:07:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAnN-0000Q7-A6
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:07:57 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bd47d666-a0d2-4a61-9a1b-0a2108c7308c;
 Fri, 03 Sep 2021 15:07:54 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681604081256.72926272893255;
 Fri, 3 Sep 2021 08:06:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd47d666-a0d2-4a61-9a1b-0a2108c7308c
ARC-Seal: i=1; a=rsa-sha256; t=1630681607; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=JbOkoHYCPySdEX9PZGt3yscfuAgkoQX8YqoESAfjPtK/XjQXJRr0QN2zkBpA022dk9IiM6ShQY2g8RKa756W5QjYA5tCGJ6o9engPp/ZdrVJLRCkzK/kG2als3kTLyzJuis5VPdpe6omYy3k0FgTLqITjwJB+hdT6LvHYHNeAXc=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681607; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=btatsPGI8kHRMfkQ9bgKzedkDOL0m29FjBRocaAF20Y=; 
	b=iXCe3z/Ns0O5dlX2CTpASRHNMIROVr6ELgDq40DXzGbIkXuAj37MfVak0egzD7usDnSj0xRF2MbL6if5JQ/MZAfZfhfBlNb89t8ZoqAyVwzFmawMkexe/F4pwNF1FVQ+By54/VSdao0GtvGVBi3ONZgvj6jQs+kaWw3tmbbCmu8=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681607;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=btatsPGI8kHRMfkQ9bgKzedkDOL0m29FjBRocaAF20Y=;
	b=nUDIe9c7qU4fS8RRNCitcuz818AOYSdaUFb2sDY4KfUmEgevhrnBvH+Xfa3owGu3
	ccRNtUEh6COwcG/tF9N8w7y4e1tLzyUrr8qOWpJesGdKvlnZ7m8VNVx5xP3BuMNYP+f
	nwICN6gbYDcL3eCC4pi8RyZ8cjFTLPiUhKnayYvQ=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 06/11] xsm: convert xsm_ops hook calls to alternative call
Date: Fri,  3 Sep 2021 15:06:24 -0400
Message-Id: <20210903190629.11917-7-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

To reduce retpolines convert all the pointer function calls of the
xsm_ops hooks over to the alternative_call infrastructure.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xsm/xsm.h | 193 +++++++++++++++++++++---------------------
 1 file changed, 97 insertions(+), 96 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 3888172045..d7ef412874 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -15,6 +15,7 @@
 #ifndef __XSM_H__
 #define __XSM_H__
 
+#include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
@@ -208,56 +209,56 @@ extern struct xsm_ops xsm_ops;
 static inline void xsm_security_domaininfo(struct domain *d,
     struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops.security_domaininfo(d, info);
+    alternative_vcall(xsm_ops.security_domaininfo, d, info);
 }
 
 static inline int xsm_domain_create(xsm_default_t def, struct domain *d,
                                     uint32_t ssidref)
 {
-    return xsm_ops.domain_create(d, ssidref);
+    return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.getdomaininfo(d);
+    return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
 static inline int xsm_domctl_scheduler_op(xsm_default_t def, struct domain *d,
                                           int cmd)
 {
-    return xsm_ops.domctl_scheduler_op(d, cmd);
+    return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl_scheduler_op(cmd);
+    return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
 static inline int xsm_set_target(xsm_default_t def, struct domain *d,
                                  struct domain *e)
 {
-    return xsm_ops.set_target(d, e);
+    return alternative_call(xsm_ops.set_target, d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.domctl(d, cmd);
+    return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl(cmd);
+    return alternative_call(xsm_ops.sysctl, cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops.readconsole(clear);
+    return alternative_call(xsm_ops.readconsole, clear);
 }
 
 static inline int xsm_evtchn_unbound(xsm_default_t def, struct domain *d1,
                                      struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops.evtchn_unbound(d1, chn, id2);
+    return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
@@ -265,235 +266,235 @@ static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
                                          struct domain *d2,
                                          struct evtchn *chan2)
 {
-    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
+    return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops.evtchn_close_post(chn);
+    alternative_vcall(xsm_ops.evtchn_close_post, chn);
 }
 
 static inline int xsm_evtchn_send(xsm_default_t def, struct domain *d,
                                   struct evtchn *chn)
 {
-    return xsm_ops.evtchn_send(d, chn);
+    return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
 static inline int xsm_evtchn_status(xsm_default_t def, struct domain *d,
                                     struct evtchn *chn)
 {
-    return xsm_ops.evtchn_status(d, chn);
+    return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
 static inline int xsm_evtchn_reset(xsm_default_t def, struct domain *d1,
                                    struct domain *d2)
 {
-    return xsm_ops.evtchn_reset(d1, d2);
+    return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
 static inline int xsm_grant_mapref(xsm_default_t def, struct domain *d1,
                                    struct domain *d2, uint32_t flags)
 {
-    return xsm_ops.grant_mapref(d1, d2, flags);
+    return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops.grant_unmapref(d1, d2);
+    return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
 static inline int xsm_grant_setup(xsm_default_t def, struct domain *d1,
                                   struct domain *d2)
 {
-    return xsm_ops.grant_setup(d1, d2);
+    return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
 static inline int xsm_grant_transfer(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops.grant_transfer(d1, d2);
+    return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
 static inline int xsm_grant_copy(xsm_default_t def, struct domain *d1,
                                  struct domain *d2)
 {
-    return xsm_ops.grant_copy(d1, d2);
+    return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
 static inline int xsm_grant_query_size(xsm_default_t def, struct domain *d1,
                                        struct domain *d2)
 {
-    return xsm_ops.grant_query_size(d1, d2);
+    return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops.alloc_security_domain(d);
+    return alternative_call(xsm_ops.alloc_security_domain, d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops.free_security_domain(d);
+    alternative_vcall(xsm_ops.free_security_domain, d);
 }
 
 static inline int xsm_alloc_security_evtchns(struct evtchn chn[],
                                              unsigned int nr)
 {
-    return xsm_ops.alloc_security_evtchns(chn, nr);
+    return alternative_call(xsm_ops.alloc_security_evtchns, chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(struct evtchn chn[],
                                              unsigned int nr)
 {
-    xsm_ops.free_security_evtchns(chn, nr);
+    alternative_vcall(xsm_ops.free_security_evtchns, chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(struct domain *d,
                                              const struct evtchn *chn)
 {
-    return xsm_ops.show_security_evtchn(d, chn);
+    return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.init_hardware_domain(d);
+    return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_pod_target(d);
+    return alternative_call(xsm_ops.get_pod_target, d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.set_pod_target(d);
+    return alternative_call(xsm_ops.set_pod_target, d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.memory_exchange(d);
+    return alternative_call(xsm_ops.memory_exchange, d);
 }
 
 static inline int xsm_memory_adjust_reservation(xsm_default_t def,
                                                 struct domain *d1,
                                                 struct domain *d2)
 {
-    return xsm_ops.memory_adjust_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(xsm_default_t def,
                                               struct domain *d1,
                                               struct domain *d2)
 {
-    return xsm_ops.memory_stat_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
 static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1,
                                       struct domain *d2, struct page_info *page)
 {
-    return xsm_ops.memory_pin_page(d1, d2, page);
+    return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1,
                                      struct domain *d2)
 {
-    return xsm_ops.add_to_physmap(d1, d2);
+    return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1,
                                           struct domain *d2)
 {
-    return xsm_ops.remove_from_physmap(d1, d2);
+    return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(xsm_default_t def, struct domain *d,
                                        struct domain *t)
 {
-    return xsm_ops.map_gmfn_foreign(d, t);
+    return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.claim_pages(d);
+    return alternative_call(xsm_ops.claim_pages, d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.console_io(d, cmd);
+    return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops.profile(d, op);
+    return alternative_call(xsm_ops.profile, d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops.kexec();
+    return alternative_call(xsm_ops.kexec);
 }
 
 static inline int xsm_schedop_shutdown(xsm_default_t def, struct domain *d1,
                                        struct domain *d2)
 {
-    return xsm_ops.schedop_shutdown(d1, d2);
+    return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops.show_irq_sid(irq);
+    return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.map_domain_pirq(d);
+    return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
 static inline int xsm_map_domain_irq(xsm_default_t def, struct domain *d,
                                      int irq, void *data)
 {
-    return xsm_ops.map_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.unmap_domain_pirq(d);
+    return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
 static inline int xsm_unmap_domain_irq(xsm_default_t def, struct domain *d,
                                        int irq, void *data)
 {
-    return xsm_ops.unmap_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(xsm_default_t def, struct domain *d,
                                   struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.bind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
                                     struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.unbind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
 static inline int xsm_irq_permission(xsm_default_t def, struct domain *d,
                                      int pirq, uint8_t allow)
 {
-    return xsm_ops.irq_permission(d, pirq, allow);
+    return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(xsm_default_t def, struct domain *d,
                                        uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(xsm_default_t def, struct domain *d,
                                     uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
@@ -501,25 +502,25 @@ static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
                                             uint16_t start, uint16_t end,
                                             uint8_t access)
 {
-    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.get_device_group(machine_bdf);
+    return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
 static inline int xsm_assign_device(xsm_default_t def, struct domain *d,
                                     uint32_t machine_bdf)
 {
-    return xsm_ops.assign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
                                       uint32_t machine_bdf)
 {
-    return xsm_ops.deassign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -527,62 +528,62 @@ static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
 static inline int xsm_assign_dtdevice(xsm_default_t def, struct domain *d,
                                       const char *dtpath)
 {
-    return xsm_ops.assign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
                                         const char *dtpath)
 {
-    return xsm_ops.deassign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_plug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(xsm_default_t def,
                                           uint32_t machine_bdf)
 {
-    return xsm_ops.resource_unplug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_plug_core();
+    return alternative_call(xsm_ops.resource_plug_core);
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_unplug_core();
+    return alternative_call(xsm_ops.resource_unplug_core);
 }
 
 static inline int xsm_resource_setup_pci(xsm_default_t def,
                                          uint32_t machine_bdf)
 {
-    return xsm_ops.resource_setup_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops.resource_setup_gsi(gsi);
+    return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops.resource_setup_misc();
+    return alternative_call(xsm_ops.resource_setup_misc);
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops.page_offline(cmd);
+    return alternative_call(xsm_ops.page_offline, cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops.hypfs_op();
+    return alternative_call(xsm_ops.hypfs_op);
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
@@ -600,112 +601,112 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 static inline int xsm_hvm_param(xsm_default_t def, struct domain *d,
                                 unsigned long op)
 {
-    return xsm_ops.hvm_param(d, op);
+    return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
 static inline int xsm_hvm_control(xsm_default_t def, struct domain *d,
                                   unsigned long op)
 {
-    return xsm_ops.hvm_control(d, op);
+    return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.hvm_param_altp2mhvm(d);
+    return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(xsm_default_t def, struct domain *d,
                                        uint64_t mode, uint32_t op)
 {
-    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
+    return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_vnumainfo(d);
+    return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
 static inline int xsm_vm_event_control(xsm_default_t def, struct domain *d,
                                        int mode, int op)
 {
-    return xsm_ops.vm_event_control(d, mode, op);
+    return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_access(d);
+    return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_paging(d);
+    return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_sharing(d);
+    return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.platform_op(op);
+    return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops.do_mca();
+    return alternative_call(xsm_ops.do_mca);
 }
 
 static inline int xsm_shadow_control(xsm_default_t def, struct domain *d,
                                      uint32_t op)
 {
-    return xsm_ops.shadow_control(d, op);
+    return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
 static inline int xsm_mem_sharing_op(xsm_default_t def, struct domain *d,
                                      struct domain *cd, int op)
 {
-    return xsm_ops.mem_sharing_op(d, cd, op);
+    return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.apic(d, cmd);
+    return alternative_call(xsm_ops.apic, d, cmd);
 }
 
 static inline int xsm_memtype(xsm_default_t def, uint32_t access)
 {
-    return xsm_ops.memtype(access);
+    return alternative_call(xsm_ops.memtype, access);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops.machine_memory_map();
+    return alternative_call(xsm_ops.machine_memory_map);
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_memory_map(d);
+    return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
 static inline int xsm_mmu_update(xsm_default_t def, struct domain *d,
                                  struct domain *t, struct domain *f,
                                  uint32_t flags)
 {
-    return xsm_ops.mmu_update(d, t, f, flags);
+    return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(xsm_default_t def, struct domain *d,
                                 struct domain *f)
 {
-    return xsm_ops.mmuext_op(d, f);
+    return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
 static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d,
@@ -717,64 +718,64 @@ static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d,
 static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d,
                                    struct domain *t)
 {
-    return xsm_ops.priv_mapping(d, t);
+    return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
 static inline int xsm_ioport_permission(xsm_default_t def, struct domain *d,
                                         uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(xsm_default_t def, struct domain *d,
                                      uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(xsm_default_t def, struct domain *d,
                              unsigned int op)
 {
-    return xsm_ops.pmu_op(d, op);
+    return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.dm_op(d);
+    return alternative_call(xsm_ops.dm_op, d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.xen_version(op);
+    return alternative_call(xsm_ops.xen_version, op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_resource_map(d);
+    return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops.argo_enable(d);
+    return alternative_call(xsm_ops.argo_enable, d);
 }
 
 static inline int xsm_argo_register_single_source(const struct domain *d,
                                                   const struct domain *t)
 {
-    return xsm_ops.argo_register_single_source(d, t);
+    return alternative_call(xsm_ops.argo_register_single_source, d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops.argo_register_any_source(d);
+    return alternative_call(xsm_ops.argo_register_any_source, d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops.argo_send(d, t);
+    return alternative_call(xsm_ops.argo_send, d, t);
 }
 
 #endif /* CONFIG_ARGO */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:08:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:08:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178157.324070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnU-0000sR-6e; Fri, 03 Sep 2021 15:08:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178157.324070; Fri, 03 Sep 2021 15:08:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnU-0000sK-3N; Fri, 03 Sep 2021 15:08:04 +0000
Received: by outflank-mailman (input) for mailman id 178157;
 Fri, 03 Sep 2021 15:08:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAnT-0000pX-0h
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:08:03 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2297f24f-9416-4fb5-bc0e-68d572178f48;
 Fri, 03 Sep 2021 15:08:00 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681605135159.2520943143012;
 Fri, 3 Sep 2021 08:06:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2297f24f-9416-4fb5-bc0e-68d572178f48
ARC-Seal: i=1; a=rsa-sha256; t=1630681608; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=IQmOrMmsWZWhcfJ9fvxJAdohJmDd40GONJ3M5td8t5OdqcLTbICTt1ALGf6ldTJTBkJNV5gCoN1c6+B+yoEhbVrF+yfEPuRkqbOxWHt6gjeP0Wc7J+mLTG1vY07W658gFV7ttiJQnew4HRc1FQ6pXjLl3TZfkm6DULmW8fM4HUU=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681608; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=Ck3QWAp7r648NxHUI1GTuSv4TTuwb5biDai25m+bty4=; 
	b=nsi9j5FJcmTMPj2cVVQ6HdXTH5z/865Ykr7IGoh7FUtKc6joz+Gm1pr6uNppNQ6B9IIXtvSX92Sky1CJn1JqWWRhHh/UtqMPdo82CUPN51YIiZRjhcIUxcXMoujDx8UEDZxvVV+Y8F/AHNNQJ08onneexhbu8bSlUKJx1gMFHR8=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681608;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=Ck3QWAp7r648NxHUI1GTuSv4TTuwb5biDai25m+bty4=;
	b=N04u9Idu71mnjJY/MdL8vU0GRbyPX8+G/XWTguSk7mi73yh9b/lLKlhtPKSdokb1
	99gfY7JBgQuxmoz/Rv5Xty7HEkl4B0m0S8Tis5mupM/sPR+4JTxVKkXmmoyb9O9JLZ6
	QGfADe5hHvXA4B5FiguhKklw4/ZXV1iVKBQmSCbo=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 07/11] xsm: decouple xsm header inclusion selection
Date: Fri,  3 Sep 2021 15:06:25 -0400
Message-Id: <20210903190629.11917-8-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Multiple preprocessor defines were used as a mechanism to selective include
parts of the xsm.h header file. This makes it difficult to know which portion
is being included at any one time. This commit works to simplify this by
separating the core structures and functions of XSM into xsm-core.h away from
the wrapper functions which remain in xsm.h and dummy.h.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h    |   2 +-
 xen/include/xsm/xsm-core.h | 274 +++++++++++++++++++++++++++++++++++++
 xen/include/xsm/xsm.h      | 255 +---------------------------------
 xen/xsm/dummy.c            |   1 -
 xen/xsm/silo.c             |   1 -
 xen/xsm/xsm_core.c         |   5 +
 6 files changed, 281 insertions(+), 257 deletions(-)
 create mode 100644 xen/include/xsm/xsm-core.h

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index deaf23035e..78d0b7d07a 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -16,7 +16,7 @@
  */
 
 #include <xen/sched.h>
-#include <xsm/xsm.h>
+#include <xsm/xsm-core.h>
 #include <public/hvm/params.h>
 
 /* Cannot use BUILD_BUG_ON here because the expressions we check are not
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
new file mode 100644
index 0000000000..4555e111dc
--- /dev/null
+++ b/xen/include/xsm/xsm-core.h
@@ -0,0 +1,274 @@
+/*
+ *  This file contains the XSM hook definitions for Xen.
+ *
+ *  This work is based on the LSM implementation in Linux 2.6.13.4.
+ *
+ *  Author:  George Coker, <gscoker@alpha.ncsc.mil>
+ *
+ *  Contributors: Michael LeMay, <mdlemay@epoch.ncsc.mil>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ */
+
+#ifndef __XSM_CORE_H__
+#define __XSM_CORE_H__
+
+#include <xen/sched.h>
+#include <xen/multiboot.h>
+
+/* policy magic number (defined by XSM_MAGIC) */
+typedef uint32_t xsm_magic_t;
+
+#ifdef CONFIG_XSM_FLASK
+#define XSM_MAGIC 0xf97cff8c
+#else
+#define XSM_MAGIC 0x0
+#endif
+
+/* These annotations are used by callers and in dummy.h to document the
+ * default actions of XSM hooks. They should be compiled out otherwise.
+ */
+enum xsm_default {
+    /* Guests can normally access the hypercall */
+    XSM_HOOK,
+    /* Device model can perform on its target domain */
+    XSM_DM_PRIV,
+    /* Can perform on self or your target domain */
+    XSM_TARGET,
+    /* Privileged - normally restricted to dom0 */
+    XSM_PRIV,
+    /* Xenstore domain - can do some privileged operations */
+    XSM_XS_PRIV,
+    /* Something more complex */
+    XSM_OTHER
+};
+typedef enum xsm_default xsm_default_t;
+
+struct xsm_ops {
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
+#endif
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
+#endif
+
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
+
+    int (*page_offline)(uint32_t cmd);
+    int (*hypfs_op)(void);
+
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#ifdef CONFIG_COMPAT
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#endif
+
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
+
+    int (*vm_event_control)(struct domain *d, int mode, int op);
+
+#ifdef CONFIG_MEM_ACCESS
+    int (*mem_access)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+    int (*mem_paging)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+    int (*mem_sharing)(struct domain *d);
+#endif
+
+    int (*platform_op)(uint32_t cmd);
+
+#ifdef CONFIG_X86
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*memtype)(uint32_t access);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
+#define XSM_MMU_UPDATE_READ      1
+#define XSM_MMU_UPDATE_WRITE     2
+#define XSM_MMU_NORMAL_UPDATE    4
+#define XSM_MMU_MACHPHYS_UPDATE  8
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
+#endif
+
+    int (*dm_op)(struct domain *d);
+
+    int (*xen_version)(uint32_t cmd);
+
+    int (*domain_resource_map)(struct domain *d);
+
+#ifdef CONFIG_ARGO
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
+#endif
+};
+
+void xsm_fixup_ops(struct xsm_ops *ops);
+
+#ifdef CONFIG_XSM
+
+#ifdef CONFIG_MULTIBOOT
+int xsm_multiboot_init(unsigned long *module_map,
+                       const multiboot_info_t *mbi);
+int xsm_multiboot_policy_init(unsigned long *module_map,
+                              const multiboot_info_t *mbi,
+                              void **policy_buffer,
+                              size_t *policy_size);
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+/*
+ * Initialize XSM
+ *
+ * On success, return 1 if using SILO mode else 0.
+ */
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
+#endif
+
+#ifdef CONFIG_XSM_FLASK
+const struct xsm_ops *flask_init(const void *policy_buffer,
+                                 size_t policy_size);
+#else
+static inline const struct xsm_ops *flask_init(const void *policy_buffer,
+                                               size_t policy_size)
+{
+    return NULL;
+}
+#endif
+
+#ifdef CONFIG_XSM_SILO
+const struct xsm_ops *silo_init(void);
+#else
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
+#endif
+
+#else /* CONFIG_XSM */
+
+#ifdef CONFIG_MULTIBOOT
+static inline int xsm_multiboot_init(unsigned long *module_map,
+                                     const multiboot_info_t *mbi)
+{
+    return 0;
+}
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+static inline int xsm_dt_init(void)
+{
+    return 0;
+}
+
+static inline bool has_xsm_magic(paddr_t start)
+{
+    return false;
+}
+#endif /* CONFIG_HAS_DEVICE_TREE */
+
+#endif /* CONFIG_XSM */
+
+#endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index d7ef412874..1517443173 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,194 +18,12 @@
 #include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
-
-/* policy magic number (defined by XSM_MAGIC) */
-typedef uint32_t xsm_magic_t;
-
-#ifdef CONFIG_XSM_FLASK
-#define XSM_MAGIC 0xf97cff8c
-#else
-#define XSM_MAGIC 0x0
-#endif
-
-/* These annotations are used by callers and in dummy.h to document the
- * default actions of XSM hooks. They should be compiled out otherwise.
- */
-enum xsm_default {
-    /* Guests can normally access the hypercall */
-    XSM_HOOK,
-    /* Device model can perform on its target domain */
-    XSM_DM_PRIV,
-    /* Can perform on self or your target domain */
-    XSM_TARGET,
-    /* Privileged - normally restricted to dom0 */
-    XSM_PRIV,
-    /* Xenstore domain - can do some privileged operations */
-    XSM_XS_PRIV,
-    /* Something more complex */
-    XSM_OTHER
-};
-typedef enum xsm_default xsm_default_t;
-
-struct xsm_ops {
-    void (*security_domaininfo)(struct domain *d,
-                                struct xen_domctl_getdomaininfo *info);
-    int (*domain_create)(struct domain *d, uint32_t ssidref);
-    int (*getdomaininfo)(struct domain *d);
-    int (*domctl_scheduler_op)(struct domain *d, int op);
-    int (*sysctl_scheduler_op)(int op);
-    int (*set_target)(struct domain *d, struct domain *e);
-    int (*domctl)(struct domain *d, int cmd);
-    int (*sysctl)(int cmd);
-    int (*readconsole)(uint32_t clear);
-
-    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
-                              struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post)(struct evtchn *chn);
-    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
-    int (*grant_setup)(struct domain *d1, struct domain *d2);
-    int (*grant_transfer)(struct domain *d1, struct domain *d2);
-    int (*grant_copy)(struct domain *d1, struct domain *d2);
-    int (*grant_query_size)(struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain)(struct domain *d);
-    void (*free_security_domain)(struct domain *d);
-    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain)(struct domain *d);
-
-    int (*get_pod_target)(struct domain *d);
-    int (*set_pod_target)(struct domain *d);
-    int (*memory_exchange)(struct domain *d);
-    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
-                           struct page_info *page);
-    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
-    int (*claim_pages)(struct domain *d);
-
-    int (*console_io)(struct domain *d, int cmd);
-
-    int (*profile)(struct domain *d, int op);
-
-    int (*kexec)(void);
-    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid)(int irq);
-    int (*map_domain_pirq)(struct domain *d);
-    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq)(struct domain *d);
-    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
-                            uint8_t allow);
-    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
-                         uint8_t allow);
-    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
-                                 uint16_t start, uint16_t end, uint8_t access);
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group)(uint32_t machine_bdf);
-    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
-#endif
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
-#endif
-
-    int (*resource_plug_core)(void);
-    int (*resource_unplug_core)(void);
-    int (*resource_plug_pci)(uint32_t machine_bdf);
-    int (*resource_unplug_pci)(uint32_t machine_bdf);
-    int (*resource_setup_pci)(uint32_t machine_bdf);
-    int (*resource_setup_gsi)(int gsi);
-    int (*resource_setup_misc)(void);
-
-    int (*page_offline)(uint32_t cmd);
-    int (*hypfs_op)(void);
-
-    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#ifdef CONFIG_COMPAT
-    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#endif
-
-    int (*hvm_param)(struct domain *d, unsigned long op);
-    int (*hvm_control)(struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm)(struct domain *d);
-    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo)(struct domain *d);
-
-    int (*vm_event_control)(struct domain *d, int mode, int op);
-
-#ifdef CONFIG_MEM_ACCESS
-    int (*mem_access)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_PAGING
-    int (*mem_paging)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing)(struct domain *d);
-#endif
-
-    int (*platform_op)(uint32_t cmd);
-
-#ifdef CONFIG_X86
-    int (*do_mca)(void);
-    int (*shadow_control)(struct domain *d, uint32_t op);
-    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
-    int (*apic)(struct domain *d, int cmd);
-    int (*memtype)(uint32_t access);
-    int (*machine_memory_map)(void);
-    int (*domain_memory_map)(struct domain *d);
-#define XSM_MMU_UPDATE_READ      1
-#define XSM_MMU_UPDATE_WRITE     2
-#define XSM_MMU_NORMAL_UPDATE    4
-#define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update)(struct domain *d, struct domain *t,
-                      struct domain *f, uint32_t flags);
-    int (*mmuext_op)(struct domain *d, struct domain *f);
-    int (*update_va_mapping)(struct domain *d, struct domain *f,
-                             l1_pgentry_t pte);
-    int (*priv_mapping)(struct domain *d, struct domain *t);
-    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
-                             uint8_t allow);
-    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
-                          uint8_t allow);
-    int (*pmu_op)(struct domain *d, unsigned int op);
-#endif
-    int (*dm_op)(struct domain *d);
-    int (*xen_version)(uint32_t cmd);
-    int (*domain_resource_map)(struct domain *d);
-#ifdef CONFIG_ARGO
-    int (*argo_enable)(const struct domain *d);
-    int (*argo_register_single_source)(const struct domain *d,
-                                       const struct domain *t);
-    int (*argo_register_any_source)(const struct domain *d);
-    int (*argo_send)(const struct domain *d, const struct domain *t);
-#endif
-};
+#include <xsm/xsm-core.h>
 
 #ifdef CONFIG_XSM
 
 extern struct xsm_ops xsm_ops;
 
-#ifndef XSM_NO_WRAPPERS
-
 static inline void xsm_security_domaininfo(struct domain *d,
     struct xen_domctl_getdomaininfo *info)
 {
@@ -780,81 +598,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#endif /* XSM_NO_WRAPPERS */
-
-#ifdef CONFIG_MULTIBOOT
-int xsm_multiboot_init(unsigned long *module_map,
-                              const multiboot_info_t *mbi);
-int xsm_multiboot_policy_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi,
-                                     void **policy_buffer,
-                                     size_t *policy_size);
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-/*
- * Initialize XSM
- *
- * On success, return 1 if using SILO mode else 0.
- */
-int xsm_dt_init(void);
-int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-bool has_xsm_magic(paddr_t);
-#endif
-
-extern struct xsm_ops dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_ops *ops);
-
-#ifdef CONFIG_XSM_FLASK
-extern const struct xsm_ops *flask_init(const void *policy_buffer,
-                                        size_t policy_size);
-#else
-static const inline struct xsm_ops *flask_init(const void *policy_buffer,
-                                               size_t policy_size)
-{
-    return NULL;
-}
-#endif
-
-#ifdef CONFIG_XSM_FLASK_POLICY
-extern const unsigned char xsm_flask_init_policy[];
-extern const unsigned int xsm_flask_init_policy_size;
-#endif
-
-#ifdef CONFIG_XSM_SILO
-extern const struct xsm_ops *silo_init(void);
-#else
-static const inline struct xsm_ops *silo_init(void)
-{
-    return NULL;
-}
-#endif
-
 #else /* CONFIG_XSM */
 
 #include <xsm/dummy.h>
 
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init (unsigned long *module_map,
-                                      const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
 #endif /* CONFIG_XSM */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index d8c935328e..b848580eaa 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,6 @@
  *  as published by the Free Software Foundation.
  */
 
-#define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
 #define set_to_dummy_if_null(ops, function)                            \
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 3550dded7b..6db793f35c 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -17,7 +17,6 @@
  * You should have received a copy of the GNU General Public License along with
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
-#define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 859af3fe9a..112d11fa6a 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -32,6 +32,11 @@
 
 struct xsm_ops __read_mostly xsm_ops;
 
+#ifdef CONFIG_XSM_FLASK_POLICY
+extern const unsigned char xsm_flask_init_policy[];
+extern const unsigned int xsm_flask_init_policy_size;
+#endif
+
 enum xsm_ops_state {
     XSM_OPS_UNREGISTERED,
     XSM_OPS_REG_FAILED,
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:08:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178164.324082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnf-0001eX-N7; Fri, 03 Sep 2021 15:08:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178164.324082; Fri, 03 Sep 2021 15:08:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnf-0001eK-JL; Fri, 03 Sep 2021 15:08:15 +0000
Received: by outflank-mailman (input) for mailman id 178164;
 Fri, 03 Sep 2021 15:08:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAne-0000pX-09
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:08:14 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d872ea58-6b0c-4a80-96e7-c4bd4ba6c50b;
 Fri, 03 Sep 2021 15:08:08 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681606215260.8066699495482;
 Fri, 3 Sep 2021 08:06:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d872ea58-6b0c-4a80-96e7-c4bd4ba6c50b
ARC-Seal: i=1; a=rsa-sha256; t=1630681610; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=lAaLajPmJgFfFXG9/fAvb2ro5HO85Q3nsofHT2js9kIjJLUIcmAEwVnVRByD18B9BbfWzRwjXmHiRQ1NkHqkCAR6baJbqVuXyFDQKewron6VMgyp1EwK0vRuauCP8k7v0/w3eJpo6xNKA1Yg6FnkV3iStlXyd6g2aenWpPZrl4M=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681610; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=OgL0twSQXxCO75U01QAyE+snmmbBrwyx4tBfZzhsUPE=; 
	b=jriZcMk0QAlQDwqd9hUmAFmBdN4GJoYRaEmd+YHpKX0SL+UPXmbNtZcqoMhQSCtEyt1kPYe0QYzzE9pdr7wGpidWVQ7jEM4kx+MPlTs4NJxhi84N7kK6LaUKJviuftYYlGwX8nBEU1JwBLrLCELr/mp/Ju0n3Ae8nXv46//zZ5k=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681610;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=OgL0twSQXxCO75U01QAyE+snmmbBrwyx4tBfZzhsUPE=;
	b=lApT1YGLT1E23o2KEJ6XDdo9IKv1h1NXRUckpo4eBRBVW4gSMl9PBq8GGeC791od
	A3VugirBc8gh4HsVUaozTDgdsQtcEV7hVDMzE4XkxvlSmbQG/607flA0hM1f6Mc/Spq
	3vljexfm5F+I1+JGiPfg0RrmordO6hguPURTRL/Y=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: xen-devel@lists.xenproject.org
Cc: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 08/11] xsm: drop generic event channel labeling exclusion
Date: Fri,  3 Sep 2021 15:06:26 -0400
Message-Id: <20210903190629.11917-9-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

The internal define flag is not used by any XSM module, removing the #ifdef
leaving the generic event channel labeling as always present.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xen/sched.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404..e56690cd2b 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -122,13 +122,11 @@ struct evtchn
 
 #ifdef CONFIG_XSM
     union {
-#ifdef XSM_NEED_GENERIC_EVTCHN_SSID
         /*
          * If an XSM module needs more space for its event channel context,
          * this pointer stores the necessary data for the security server.
          */
         void *generic;
-#endif
 #ifdef CONFIG_XSM_FLASK
         /*
          * Inlining the contents of the structure for FLASK avoids unneeded
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:08:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178172.324093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnn-0002Gp-1L; Fri, 03 Sep 2021 15:08:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178172.324093; Fri, 03 Sep 2021 15:08:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAnm-0002Ge-Tm; Fri, 03 Sep 2021 15:08:22 +0000
Received: by outflank-mailman (input) for mailman id 178172;
 Fri, 03 Sep 2021 15:08:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAnm-00020o-6O
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:08:22 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c49e68bc-0cc8-11ec-af2e-12813bfff9fa;
 Fri, 03 Sep 2021 15:08:18 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681607871996.5359561303381;
 Fri, 3 Sep 2021 08:06:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c49e68bc-0cc8-11ec-af2e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1630681613; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=AmnjgUxavzvlOgQThsUztE6rwLA9AR1eB9LLbjf1gN6C9YJEpPb01RiYrBLPwTDUqvUcyfBoETtfz+FrM9lGAzjdji/RjUI86EsOCH/2SMRu3MhhtE63l7MfRGbo/gF7Zp82gneuAiP36Etp/u0NPTQH9LMmKPgvOh3kSCJaViA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681613; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=DHYiPbkr+NbOPJst4/lduNG0E/MNzNsNVs5R1f5AFDk=; 
	b=J0EIGjS1XMflXx/TZjog1zNaIfzg8OscC2cCqd50CL4ka5IynK+OIcQvt+Hv0dVFHiDFt2riVKSHWXyRDjEJJ2YsX3lTD4xGa+tnHLgH9Y7wJsHTC9JmpZcvfnlvqFLzT001Liu2eUURuPzN3pDcVXc6FpXq3Tfme/8GwKafWMQ=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681613;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=DHYiPbkr+NbOPJst4/lduNG0E/MNzNsNVs5R1f5AFDk=;
	b=Brrsy+5tS9Yt7RWJtCGbyKNNj4+oiAse7W5CsQOOu0cqWs+vJD/flMRPYzEEF45K
	CxCT7RaTtN8LPe06fEy/6YHMs8elfUoHwWS0CCI0qM4VAYIBlc+QAlZtM598UcKZpi5
	wKa6SJJi+0qdf8SdhusQ8iixemr3TFZBa8eLMnHE=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 09/11] silo: remove circular xsm hook call
Date: Fri,  3 Sep 2021 15:06:27 -0400
Message-Id: <20210903190629.11917-10-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

SILO implements a few XSM hooks to extended the decision logic beyond
what is defined in the dummy/default policy. For each of the hooks, it
falls back to the dummy/default policy. The fall back is done a slight
round-about way. This commit makes the direct call to the default policy's
logic, xsm_default_action().

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/xsm/silo.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 6db793f35c..56a330a831 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -17,6 +17,7 @@
  * You should have received a copy of the GNU General Public License along with
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
+#include <xsm/xsm-core.h>
 #include <xsm/dummy.h>
 
 /*
@@ -43,7 +44,7 @@ static int silo_evtchn_unbound(struct domain *d1, struct evtchn *chn,
     else
     {
         if ( silo_mode_dom_check(d1, d2) )
-            rc = xsm_evtchn_unbound(d1, chn, id2);
+            rc = xsm_default_action(XSM_TARGET, current->domain, d1);
         rcu_unlock_domain(d2);
     }
 
@@ -54,7 +55,7 @@ static int silo_evtchn_interdomain(struct domain *d1, struct evtchn *chan1,
                                    struct domain *d2, struct evtchn *chan2)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_evtchn_interdomain(d1, chan1, d2, chan2);
+        return xsm_default_action(XSM_HOOK, d1, d2);
     return -EPERM;
 }
 
@@ -62,21 +63,21 @@ static int silo_grant_mapref(struct domain *d1, struct domain *d2,
                              uint32_t flags)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_grant_mapref(d1, d2, flags);
+        return xsm_default_action(XSM_HOOK, d1, d2);
     return -EPERM;
 }
 
 static int silo_grant_transfer(struct domain *d1, struct domain *d2)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_grant_transfer(d1, d2);
+        return xsm_default_action(XSM_HOOK, d1, d2);
     return -EPERM;
 }
 
 static int silo_grant_copy(struct domain *d1, struct domain *d2)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_grant_copy(d1, d2);
+        return xsm_default_action(XSM_HOOK, d1, d2);
     return -EPERM;
 }
 
@@ -86,14 +87,14 @@ static int silo_argo_register_single_source(const struct domain *d1,
                                             const struct domain *d2)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_argo_register_single_source(d1, d2);
+        return 0;
     return -EPERM;
 }
 
 static int silo_argo_send(const struct domain *d1, const struct domain *d2)
 {
     if ( silo_mode_dom_check(d1, d2) )
-        return xsm_argo_send(d1, d2);
+        return 0;
     return -EPERM;
 }
 
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:08:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:08:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178186.324104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAo2-00036i-CL; Fri, 03 Sep 2021 15:08:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178186.324104; Fri, 03 Sep 2021 15:08:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAo2-00036V-8L; Fri, 03 Sep 2021 15:08:38 +0000
Received: by outflank-mailman (input) for mailman id 178186;
 Fri, 03 Sep 2021 15:08:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAo1-00020o-7E
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:08:37 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c88d6a5e-0cc8-11ec-af2e-12813bfff9fa;
 Fri, 03 Sep 2021 15:08:24 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681609048555.1692564591247;
 Fri, 3 Sep 2021 08:06:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c88d6a5e-0cc8-11ec-af2e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1630681615; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=gW6P5jBpju7RaplNSLoo/QOOJyGDhIhA3hMpxMZwgliJhH2vQT34hWZXiikGXZIL5myeJE1aywU04Yp67H9v2WXSdHRe0cWzK6H+MYvJ3AvcggaoZW8BULAypKN8lUPSvl+soM5OUANE4FhbXTFcIeTpxM4WQYzUR6qf1ymtwso=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681615; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=EWTwq6nNcnfro7pqzmCHy9mDrCWwYOniEdIAUXfrX50=; 
	b=e/fzjYuPWVBzdWlMzbTgVXAMhEFLd+n6Kv38RzMPakdtPYiwd7E9FGXmZgVqe9SC7oRtcFR0N9gO0rrSQ3vm9s1vc6hduEDZ8xCu9t6l8LV+Yktfy6r9z4iqRIfrBSiwsa35kkuV0o30u/5caWiftxxEeN3DvZndmnFgOj7YH5Q=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681615;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=EWTwq6nNcnfro7pqzmCHy9mDrCWwYOniEdIAUXfrX50=;
	b=t2tWHwtGCWiWfkWZAHE1fPkFWJXbbeiD1drHgc8RnY3cvt5Ntwldy2WRmiDvco48
	xM74PJM63Exn/TiIqx4iwiNKetM/ZS5wkQ5ZyD1mPamhulPF3C+3lDl0dOiznQNPSmC
	vWX13+lZv96YbP69XJpBW/Rk9K0lxqor5aUiTmfM=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 10/11] kconfig: update xsm config to reflect reality
Date: Fri,  3 Sep 2021 15:06:28 -0400
Message-Id: <20210903190629.11917-11-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

It has been a very long time since XSM Flask was the only XSM module, yet the
concenpt of turning XSM on/off continues to be synonymous with enabling and
disabling XSM Flask. Even when XSM Flask was the only module, turning XSM
on/off did not disable or remove the XSM hooks but simply controlled whether
they were implemented as direct inline functions or dispatch calls.

This commit updates XSM kconfig to ensure that it is clear in the code as well
to the user, via the help messages, that the option is about configuring which
XSM policy module(s) are available and which is the default.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/common/Kconfig         | 49 ++++++++++++++++++--------------------
 xen/include/xen/sched.h    |  2 +-
 xen/include/xsm/dummy.h    | 23 +++++++++---------
 xen/include/xsm/xsm-core.h |  6 ++---
 xen/include/xsm/xsm.h      |  6 ++---
 xen/xsm/Makefile           |  4 ++--
 xen/xsm/xsm_core.c         |  4 ++--
 7 files changed, 46 insertions(+), 48 deletions(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index ac5491b1cc..2f85538920 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -200,23 +200,20 @@ config XENOPROF
 
 	  If unsure, say Y.
 
-config XSM
-	bool "Xen Security Modules support"
+config XSM_CONFIGURABLE
+	bool "Configure Xen Security Modules"
 	default ARM
-	---help---
-	  Enables the security framework known as Xen Security Modules which
-	  allows administrators fine-grained control over a Xen domain and
-	  its capabilities by defining permissible interactions between domains,
-	  the hypervisor itself, and related resources such as memory and
-	  devices.
+	help
+	  Allows for configuring the Xen Security Modules (XSM) policy or policies
+	  modules that will be availble and which will be the default.
 
 	  If unsure, say N.
 
 config XSM_FLASK
-	def_bool y
-	prompt "FLux Advanced Security Kernel support"
-	depends on XSM
-	---help---
+	bool "FLux Advanced Security Kernel support"
+	depends on XSM_CONFIGURABLE
+	select XSM_EVTCHN_LABELING
+	help
 	  Enables FLASK (FLux Advanced Security Kernel) as the access control
 	  mechanism used by the XSM framework.  This provides a mandatory access
 	  control framework by which security enforcement, isolation, and
@@ -226,10 +223,10 @@ config XSM_FLASK
 	  If unsure, say Y.
 
 config XSM_FLASK_AVC_STATS
-	def_bool y
-	prompt "Maintain statistics on the FLASK access vector cache" if EXPERT
+	bool "Maintain statistics on the FLASK access vector cache" if EXPERT
+	default y
 	depends on XSM_FLASK
-	---help---
+	help
 	  Maintain counters on the access vector cache that can be viewed using
 	  the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.  Disabling
 	  this will save a tiny amount of memory and time to update the stats.
@@ -240,7 +237,7 @@ config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
 	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
 	depends on XSM_FLASK
-	---help---
+	help
 	  This includes a default XSM policy in the hypervisor so that the
 	  bootloader does not need to load a policy to get sane behavior from an
 	  XSM-enabled hypervisor.  If this is disabled, a policy must be
@@ -253,10 +250,10 @@ config XSM_FLASK_POLICY
 	  If unsure, say Y.
 
 config XSM_SILO
-	def_bool y
-	prompt "SILO support"
-	depends on XSM
-	---help---
+	bool "SILO support"
+	default y if ARM
+	depends on XSM_CONFIGURABLE
+	help
 	  Enables SILO as the access control mechanism used by the XSM framework.
 	  This is not the default module, add boot parameter xsm=silo to choose
 	  it. This will deny any unmediated communication channels (grant tables
@@ -265,14 +262,14 @@ config XSM_SILO
 	  If unsure, say Y.
 
 choice
-	prompt "Default XSM implementation"
-	depends on XSM
+	prompt "Default XSM module"
+	depends on XSM_CONFIGURABLE
 	default XSM_SILO_DEFAULT if XSM_SILO && ARM
 	default XSM_FLASK_DEFAULT if XSM_FLASK
 	default XSM_SILO_DEFAULT if XSM_SILO
 	default XSM_DUMMY_DEFAULT
 	config XSM_DUMMY_DEFAULT
-		bool "Match non-XSM behavior"
+		bool "Classic Dom0 behavior"
 	config XSM_FLASK_DEFAULT
 		bool "FLux Advanced Security Kernel" if XSM_FLASK
 	config XSM_SILO_DEFAULT
@@ -282,15 +279,15 @@ endchoice
 config LATE_HWDOM
 	bool "Dedicated hardware domain"
 	default n
-	depends on XSM && X86
-	---help---
+	depends on XSM_FLASK && X86
+	help
 	  Allows the creation of a dedicated hardware domain distinct from
 	  domain 0 that manages devices without needing access to other
 	  privileged functionality such as the ability to manage domains.
 	  This requires that the actual domain 0 be a stub domain that
 	  constructs the actual hardware domain instead of initializing the
 	  hardware itself.  Because the hardware domain needs access to
-	  hypercalls not available to unprivileged guests, an XSM policy
+	  hypercalls not available to unprivileged guests, an XSM Flask policy
 	  is required to properly define the privilege of these domains.
 
 	  This feature does nothing if the "hardware_dom" boot parameter is
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index e56690cd2b..a61586d8b6 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -120,7 +120,7 @@ struct evtchn
     unsigned short notify_vcpu_id; /* VCPU for local delivery notification */
     uint32_t fifo_lastq;           /* Data for identifying last queue. */
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
     union {
         /*
          * If an XSM module needs more space for its event channel context,
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 78d0b7d07a..d2a005c521 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -42,12 +42,12 @@ static inline void __xsm_action_mismatch_detected(void)
 void __xsm_action_mismatch_detected(void);
 #endif
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
-/* In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
- * contains static (not inline) functions compiled to the dummy XSM module.
- * There is no xsm_default_t argument available, so the value from the assertion
- * is used to initialize the variable.
+/* In CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/dummy.c, and contains static (not inline) functions compiled to the
+ * dummy XSM module.  There is no xsm_default_t argument available, so the
+ * value from the assertion is used to initialize the variable.
  */
 #define XSM_INLINE __maybe_unused
 
@@ -55,19 +55,20 @@ void __xsm_action_mismatch_detected(void);
 #define XSM_DEFAULT_VOID void
 #define XSM_ASSERT_ACTION(def) xsm_default_t action = def; (void)action
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
-/* In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
- * contains inline functions for each XSM hook. These functions also perform
- * compile-time checks on the xsm_default_t argument to ensure that the behavior
- * of the dummy XSM module is the same as the behavior with XSM disabled.
+/* In !CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/xsm.h, and contains inline functions for each XSM hook. These functions
+ * also perform compile-time checks on the xsm_default_t argument to ensure
+ * that the behavior of the dummy XSM module is the same as the behavior with
+ * XSM disabled.
  */
 #define XSM_INLINE always_inline
 #define XSM_DEFAULT_ARG xsm_default_t action,
 #define XSM_DEFAULT_VOID xsm_default_t action
 #define XSM_ASSERT_ACTION(def) LINKER_BUG_ON(def != action)
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 static always_inline int xsm_default_action(xsm_default_t action,
                                             struct domain *src,
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index 4555e111dc..a80d6a981f 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -205,7 +205,7 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(unsigned long *module_map,
@@ -247,7 +247,7 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #ifdef CONFIG_MULTIBOOT
 static inline int xsm_multiboot_init(unsigned long *module_map,
@@ -269,6 +269,6 @@ static inline bool has_xsm_magic(paddr_t start)
 }
 #endif /* CONFIG_HAS_DEVICE_TREE */
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 1517443173..4d5f441935 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,7 +20,7 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 extern struct xsm_ops xsm_ops;
 
@@ -598,10 +598,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #include <xsm/dummy.h>
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index cf0a728f1c..09b9311b1d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
-obj-$(CONFIG_XSM) += xsm_policy.o
-obj-$(CONFIG_XSM) += dummy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 112d11fa6a..87c2980459 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,7 +18,7 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
@@ -227,7 +227,7 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:08:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:08:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178194.324115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAoH-0003kV-Lt; Fri, 03 Sep 2021 15:08:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178194.324115; Fri, 03 Sep 2021 15:08:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMAoH-0003kO-I3; Fri, 03 Sep 2021 15:08:53 +0000
Received: by outflank-mailman (input) for mailman id 178194;
 Fri, 03 Sep 2021 15:08:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=yFRs=NZ=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mMAoG-00020o-7L
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:08:52 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cfa37400-0cc8-11ec-af2e-12813bfff9fa;
 Fri, 03 Sep 2021 15:08:36 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1630681611173769.2077493218004;
 Fri, 3 Sep 2021 08:06:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfa37400-0cc8-11ec-af2e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1630681616; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=ARTuBemhfHD7PHmg2/V8DzU/9Dn/d49oG7L/TAk/ojBTw+nGjCc7mgzgm19YQF6BcdidPONvz9lYt8ee5WQ1VSmqclPBRtlxbCSOgMB3ufE2FU2Q8o9SQoS/oJl9muE3vU7jnIUGLk3ayNnM28ctnqmIQcDBKvurysbgexTurQY=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1630681616; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=u7PRBaKNvpJUcvrNoVjDcU/pRAPQ2hgSBvfk9KJrnqw=; 
	b=XMWdENT0WMuT9rlQokC/uyNTR2hphm6KIUcVn4Ge/wo9Jd+GE25NfTzZBUZ1EHnrn3A/VyZ6Lpj9/pBGGRRkPM5Ne5SoMCE576G2PPp/iuxO+bKLc9piTUuGpqZF2GXTCW4vMVgE2/QhXnuz705WQSfOpHx+wOfuBxhXJm6l9bA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1630681616;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=u7PRBaKNvpJUcvrNoVjDcU/pRAPQ2hgSBvfk9KJrnqw=;
	b=OzwCtjbU4kYMAtY1yhSuNuLDvoFFlER0mEE5EWyiXzLJdEQ9PHsEE+HbZ4u+jjKi
	BYt0KZYp0443rj5w0liaHuP7Z2qXctYh+fDnr0muhUOKbZFKc37Mkzg3xY7PdtsSElF
	IFX+pQlzGXRki4bhyRbZNZvZKMSVfaVXFntkCGrQ=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v4 11/11] xsm: remove alternate xsm hook interface
Date: Fri,  3 Sep 2021 15:06:29 -0400
Message-Id: <20210903190629.11917-12-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210903190629.11917-1-dpsmith@apertussolutions.com>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Hidden behind macro magic is an alternative xsm hook interface dedicated for
use when the dummy/default policy is the only one built. This alternative
interface increases code complexity and code size in the core security
framework of Xen.  This results in code requiring additional maintanence and
additional risk for securit-relevant bugs.

This patch removes this additional interface, making Xen's security framework
have a single, consistent interface that works in a single and consistent
manner regardless of which XSM policy is in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h    | 824 -------------------------------------
 xen/include/xsm/xsm-core.h |  51 ++-
 xen/include/xsm/xsm.h      | 279 ++++++++-----
 xen/xsm/Makefile           |   2 +-
 xen/xsm/dummy.c            |   2 +-
 xen/xsm/dummy.h            | 739 +++++++++++++++++++++++++++++++++
 xen/xsm/silo.c             |   2 +-
 xen/xsm/xsm_core.c         |   4 -
 8 files changed, 952 insertions(+), 951 deletions(-)
 delete mode 100644 xen/include/xsm/dummy.h
 create mode 100644 xen/xsm/dummy.h

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
deleted file mode 100644
index d2a005c521..0000000000
--- a/xen/include/xsm/dummy.h
+++ /dev/null
@@ -1,824 +0,0 @@
-/*
- *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
- *
- *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License version 2,
- *  as published by the Free Software Foundation.
- *
- *
- *  Each XSM hook implementing an access check should have its first parameter
- *  preceded by XSM_DEFAULT_ARG (or use XSM_DEFAULT_VOID if it has no
- *  arguments). The first non-declaration statement shold be XSM_ASSERT_ACTION
- *  with the expected type of the hook, which will either define or check the
- *  value of action.
- */
-
-#include <xen/sched.h>
-#include <xsm/xsm-core.h>
-#include <public/hvm/params.h>
-
-/* Cannot use BUILD_BUG_ON here because the expressions we check are not
- * considered constant at compile time. Instead, rely on constant propagation to
- * inline out the calls to this invalid function, which will cause linker errors
- * if references remain at link time.
- */
-#define LINKER_BUG_ON(x) do { if (x) __xsm_action_mismatch_detected(); } while (0)
-
-#if defined(CONFIG_COVERAGE) && defined(__clang__)
-/*
- * LLVM coverage support seems to disable some of the optimizations needed in
- * order for XSM to compile. Since coverage should not be used in production
- * provide an implementation of __xsm_action_mismatch_detected to satisfy the
- * linker.
- */
-static inline void __xsm_action_mismatch_detected(void)
-{
-    ASSERT_UNREACHABLE();
-}
-#else
-/* DO NOT implement this function; it is supposed to trigger link errors */
-void __xsm_action_mismatch_detected(void);
-#endif
-
-#ifdef CONFIG_XSM_CONFIGURABLE
-
-/* In CONFIG_XSM_CONFIGURABLE builds, this header file is included from
- * xsm/dummy.c, and contains static (not inline) functions compiled to the
- * dummy XSM module.  There is no xsm_default_t argument available, so the
- * value from the assertion is used to initialize the variable.
- */
-#define XSM_INLINE __maybe_unused
-
-#define XSM_DEFAULT_ARG /* */
-#define XSM_DEFAULT_VOID void
-#define XSM_ASSERT_ACTION(def) xsm_default_t action = def; (void)action
-
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-/* In !CONFIG_XSM_CONFIGURABLE builds, this header file is included from
- * xsm/xsm.h, and contains inline functions for each XSM hook. These functions
- * also perform compile-time checks on the xsm_default_t argument to ensure
- * that the behavior of the dummy XSM module is the same as the behavior with
- * XSM disabled.
- */
-#define XSM_INLINE always_inline
-#define XSM_DEFAULT_ARG xsm_default_t action,
-#define XSM_DEFAULT_VOID xsm_default_t action
-#define XSM_ASSERT_ACTION(def) LINKER_BUG_ON(def != action)
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
-static always_inline int xsm_default_action(xsm_default_t action,
-                                            struct domain *src,
-                                            struct domain *target)
-{
-    switch ( action ) {
-    case XSM_HOOK:
-        return 0;
-    case XSM_TARGET:
-        if ( evaluate_nospec(src == target) )
-        {
-            return 0;
-    case XSM_XS_PRIV:
-            if ( evaluate_nospec(is_xenstore_domain(src)) )
-                return 0;
-        }
-        /* fall through */
-    case XSM_DM_PRIV:
-        if ( target && evaluate_nospec(src->target == target) )
-            return 0;
-        /* fall through */
-    case XSM_PRIV:
-        if ( is_control_domain(src) )
-            return 0;
-        return -EPERM;
-    default:
-        LINKER_BUG_ON(1);
-        return -EPERM;
-    }
-}
-
-static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
-    struct xen_domctl_getdomaininfo *info)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d,
-                                        uint32_t ssidref)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_domctl_scheduler_op(XSM_DEFAULT_ARG struct domain *d,
-                                              int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_set_target(XSM_DEFAULT_ARG struct domain *d,
-                                     struct domain *e)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_domctl(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( cmd )
-    {
-    case XEN_DOMCTL_ioport_mapping:
-    case XEN_DOMCTL_memory_mapping:
-    case XEN_DOMCTL_bind_pt_irq:
-    case XEN_DOMCTL_unbind_pt_irq:
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    case XEN_DOMCTL_getdomaininfo:
-        return xsm_default_action(XSM_XS_PRIV, current->domain, d);
-    default:
-        return xsm_default_action(XSM_PRIV, current->domain, d);
-    }
-}
-
-static XSM_INLINE int xsm_sysctl(XSM_DEFAULT_ARG int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG uint32_t clear)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_alloc_security_domain(struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE void xsm_free_security_domain(struct domain *d)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_grant_mapref(XSM_DEFAULT_ARG struct domain *d1,
-                                       struct domain *d2, uint32_t flags)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_unmapref(XSM_DEFAULT_ARG struct domain *d1,
-                                         struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_setup(XSM_DEFAULT_ARG struct domain *d1,
-                                      struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_transfer(XSM_DEFAULT_ARG struct domain *d1,
-                                         struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_copy(XSM_DEFAULT_ARG struct domain *d1,
-                                     struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_query_size(XSM_DEFAULT_ARG struct domain *d1,
-                                           struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_memory_adjust_reservation(XSM_DEFAULT_ARG
-                                                    struct domain *d1,
-                                                    struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG
-                                                  struct domain *d1,
-                                                  struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_console_io(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    if ( d->is_console )
-        return xsm_default_action(XSM_HOOK, d, NULL);
-#ifdef CONFIG_VERBOSE_DEBUG
-    if ( cmd == CONSOLEIO_write )
-        return xsm_default_action(XSM_HOOK, d, NULL);
-#endif
-    return xsm_default_action(XSM_PRIV, d, NULL);
-}
-
-static XSM_INLINE int xsm_profile(XSM_DEFAULT_ARG struct domain *d, int op)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_schedop_shutdown(XSM_DEFAULT_ARG struct domain *d1,
-                                           struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_pin_page(XSM_DEFAULT_ARG struct domain *d1,
-                                          struct domain *d2,
-                                          struct page_info *page)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_unbound(XSM_DEFAULT_ARG struct domain *d,
-                                         struct evtchn *chn, domid_t id2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_interdomain(XSM_DEFAULT_ARG struct domain *d1,
-                                             struct evtchn *chan1,
-                                             struct domain *d2,
-                                             struct evtchn *chan2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d,
-                                      struct evtchn *chn)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d,
-                                        struct evtchn *chn)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1,
-                                       struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_alloc_security_evtchns(struct evtchn chn[],
-                                                 unsigned int nr)
-{
-    return 0;
-}
-
-static XSM_INLINE void xsm_free_security_evtchns(struct evtchn chn[],
-                                                 unsigned int nr)
-{
-    return;
-}
-
-static XSM_INLINE char *xsm_show_security_evtchn(struct domain *d,
-                                                 const struct evtchn *chn)
-{
-    return NULL;
-}
-
-static XSM_INLINE int xsm_init_hardware_domain(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_get_pod_target(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_set_pod_target(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_get_vnumainfo(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_assign_device(XSM_DEFAULT_ARG struct domain *d,
-                                        uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d,
-                                          uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#endif /* HAS_PASSTHROUGH && HAS_PCI */
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static XSM_INLINE int xsm_assign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                          const char *dtpath)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_deassign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                            const char *dtpath)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
-
-static XSM_INLINE int xsm_resource_plug_core(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_plug_pci(XSM_DEFAULT_ARG
-                                            uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_unplug_pci(XSM_DEFAULT_ARG
-                                              uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_pci(XSM_DEFAULT_ARG
-                                             uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_gsi(XSM_DEFAULT_ARG int gsi)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_misc(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_page_offline(XSM_DEFAULT_ARG uint32_t cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
-{
-    return -ENOSYS;
-}
-
-#ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
-{
-    return -ENOSYS;
-}
-#endif
-
-static XSM_INLINE char *xsm_show_irq_sid(int irq)
-{
-    return NULL;
-}
-
-static XSM_INLINE int xsm_map_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_map_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                         int irq, const void *data)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_bind_pt_irq(XSM_DEFAULT_ARG struct domain *d,
-                                      struct xen_domctl_bind_pt_irq *bind)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unbind_pt_irq(XSM_DEFAULT_ARG struct domain *d,
-                                        struct xen_domctl_bind_pt_irq *bind)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unmap_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                           int irq, const void *data)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_irq_permission(XSM_DEFAULT_ARG struct domain *d,
-                                         int pirq, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_iomem_permission(XSM_DEFAULT_ARG struct domain *d,
-                                           uint64_t s, uint64_t e,
-                                           uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_iomem_mapping(XSM_DEFAULT_ARG struct domain *d,
-                                        uint64_t s, uint64_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG
-                                                struct domain *d,
-                                                uint32_t machine_bdf,
-                                                uint16_t start, uint16_t end,
-                                                uint8_t access)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1,
-                                         struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_remove_from_physmap(XSM_DEFAULT_ARG struct domain *d1,
-                                              struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_map_gmfn_foreign(XSM_DEFAULT_ARG struct domain *d,
-                                           struct domain *t)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, t);
-}
-
-static XSM_INLINE int xsm_hvm_param(XSM_DEFAULT_ARG struct domain *d,
-                                    unsigned long op)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_control(XSM_DEFAULT_ARG struct domain *d,
-                                      unsigned long op)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d,
-                                           uint64_t mode, uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-
-    switch ( mode )
-    {
-    case XEN_ALTP2M_mixed:
-        return xsm_default_action(XSM_TARGET, current->domain, d);
-    case XEN_ALTP2M_external:
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    case XEN_ALTP2M_limited:
-        if ( HVMOP_altp2m_vcpu_enable_notify == op )
-            return xsm_default_action(XSM_TARGET, current->domain, d);
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    default:
-        return -EPERM;
-    }
-}
-
-static XSM_INLINE int xsm_vm_event_control(XSM_DEFAULT_ARG struct domain *d,
-                                           int mode, int op)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#ifdef CONFIG_MEM_ACCESS
-static XSM_INLINE int xsm_mem_access(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-#ifdef CONFIG_MEM_PAGING
-static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-#ifdef CONFIG_MEM_SHARING
-static XSM_INLINE int xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-static XSM_INLINE int xsm_platform_op(XSM_DEFAULT_ARG uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-#ifdef CONFIG_X86
-static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_shadow_control(XSM_DEFAULT_ARG struct domain *d,
-                                         uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_mem_sharing_op(XSM_DEFAULT_ARG struct domain *d,
-                                         struct domain *cd, int op)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, cd);
-}
-
-static XSM_INLINE int xsm_apic(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_machine_memory_map(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d,
-                                     struct domain *t, struct domain *f,
-                                     uint32_t flags)
-{
-    int rc = 0;
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    if ( f != dom_io )
-        rc = xsm_default_action(action, d, f);
-    if ( evaluate_nospec(t) && !rc )
-        rc = xsm_default_action(action, d, t);
-    return rc;
-}
-
-static XSM_INLINE int xsm_mmuext_op(XSM_DEFAULT_ARG struct domain *d,
-                                    struct domain *f)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, f);
-}
-
-static XSM_INLINE int xsm_update_va_mapping(XSM_DEFAULT_ARG struct domain *d,
-                                            struct domain *f, l1_pgentry_t pte)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, f);
-}
-
-static XSM_INLINE int xsm_priv_mapping(XSM_DEFAULT_ARG struct domain *d,
-                                       struct domain *t)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, t);
-}
-
-static XSM_INLINE int xsm_ioport_permission(XSM_DEFAULT_ARG struct domain *d,
-                                            uint32_t s, uint32_t e,
-                                            uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_ioport_mapping(XSM_DEFAULT_ARG struct domain *d,
-                                         uint32_t s, uint32_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_pmu_op(XSM_DEFAULT_ARG struct domain *d,
-                                 unsigned int op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( op )
-    {
-    case XENPMU_init:
-    case XENPMU_finish:
-    case XENPMU_lvtpc_set:
-    case XENPMU_flush:
-        return xsm_default_action(XSM_HOOK, d, current->domain);
-    default:
-        return xsm_default_action(XSM_PRIV, d, current->domain);
-    }
-}
-
-#endif /* CONFIG_X86 */
-
-static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#ifdef CONFIG_ARGO
-static XSM_INLINE int xsm_argo_enable(const struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_register_single_source(const struct domain *d,
-                                                      const struct domain *t)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_register_any_source(const struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_send(const struct domain *d,
-                                    const struct domain *t)
-{
-    return 0;
-}
-
-#endif /* CONFIG_ARGO */
-
-#include <public/version.h>
-static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( op )
-    {
-    case XENVER_version:
-    case XENVER_platform_parameters:
-    case XENVER_get_features:
-        /* These sub-ops ignore the permission checks and return data. */
-        block_speculation();
-        return 0;
-    case XENVER_extraversion:
-    case XENVER_compile_info:
-    case XENVER_capabilities:
-    case XENVER_changeset:
-    case XENVER_pagesize:
-    case XENVER_guest_handle:
-        /* These MUST always be accessible to any guest by default. */
-        return xsm_default_action(XSM_HOOK, current->domain, NULL);
-    default:
-        return xsm_default_action(XSM_PRIV, current->domain, NULL);
-    }
-}
-
-static XSM_INLINE int xsm_domain_resource_map(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index a80d6a981f..0da625317e 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -18,6 +18,31 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
+/* Cannot use BUILD_BUG_ON here because the expressions we check are not
+ * considered constant at compile time. Instead, rely on constant propagation to
+ * inline out the calls to this invalid function, which will cause linker errors
+ * if references remain at link time.
+ */
+#define XSM_LINKER_BUG_ON(x) do { if (x) __xsm_action_mismatch_detected(); } while (0)
+
+#if defined(CONFIG_COVERAGE) && defined(__clang__)
+/*
+ * LLVM coverage support seems to disable some of the optimizations needed in
+ * order for XSM to compile. Since coverage should not be used in production
+ * provide an implementation of __xsm_action_mismatch_detected to satisfy the
+ * linker.
+ */
+static inline void __xsm_action_mismatch_detected(void)
+{
+    ASSERT_UNREACHABLE();
+}
+#else
+/* DO NOT implement this function; it is supposed to trigger link errors */
+void __xsm_action_mismatch_detected(void);
+#endif
+
+#define XSM_ASSERT_ACTION(def) XSM_LINKER_BUG_ON(def != action)
+
 /* policy magic number (defined by XSM_MAGIC) */
 typedef uint32_t xsm_magic_t;
 
@@ -205,8 +230,6 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(unsigned long *module_map,
                        const multiboot_info_t *mbi);
@@ -247,28 +270,4 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 4d5f441935..33bc2570bf 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,8 +20,6 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 extern struct xsm_ops xsm_ops;
 
 static inline void xsm_security_domaininfo(struct domain *d,
@@ -30,60 +28,72 @@ static inline void xsm_security_domaininfo(struct domain *d,
     alternative_vcall(xsm_ops.security_domaininfo, d, info);
 }
 
-static inline int xsm_domain_create(xsm_default_t def, struct domain *d,
+static inline int xsm_domain_create(xsm_default_t action, struct domain *d,
                                     uint32_t ssidref)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
-static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
-static inline int xsm_domctl_scheduler_op(xsm_default_t def, struct domain *d,
+static inline int xsm_domctl_scheduler_op(xsm_default_t action,
+                                          struct domain *d,
                                           int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
-static inline int xsm_set_target(xsm_default_t def, struct domain *d,
+static inline int xsm_set_target(xsm_default_t action, struct domain *d,
                                  struct domain *e)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.set_target, d, e);
 }
 
-static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
-static inline int xsm_sysctl(xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.sysctl, cmd);
 }
 
-static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t action, uint32_t clear)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.readconsole, clear);
 }
 
-static inline int xsm_evtchn_unbound(xsm_default_t def, struct domain *d1,
+static inline int xsm_evtchn_unbound(xsm_default_t action, struct domain *d1,
                                      struct evtchn *chn, domid_t id2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
-static inline int xsm_evtchn_interdomain(xsm_default_t def, struct domain *d1,
+static inline int xsm_evtchn_interdomain(xsm_default_t action,
+                                         struct domain *d1,
                                          struct evtchn *chan1,
                                          struct domain *d2,
                                          struct evtchn *chan2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
@@ -92,57 +102,66 @@ static inline void xsm_evtchn_close_post(struct evtchn *chn)
     alternative_vcall(xsm_ops.evtchn_close_post, chn);
 }
 
-static inline int xsm_evtchn_send(xsm_default_t def, struct domain *d,
+static inline int xsm_evtchn_send(xsm_default_t action, struct domain *d,
                                   struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
-static inline int xsm_evtchn_status(xsm_default_t def, struct domain *d,
+static inline int xsm_evtchn_status(xsm_default_t action, struct domain *d,
                                     struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
-static inline int xsm_evtchn_reset(xsm_default_t def, struct domain *d1,
+static inline int xsm_evtchn_reset(xsm_default_t action, struct domain *d1,
                                    struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
-static inline int xsm_grant_mapref(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_mapref(xsm_default_t action, struct domain *d1,
                                    struct domain *d2, uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
-static inline int xsm_grant_unmapref(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_unmapref(xsm_default_t action, struct domain *d1,
                                      struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
-static inline int xsm_grant_setup(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_setup(xsm_default_t action, struct domain *d1,
                                   struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
-static inline int xsm_grant_transfer(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_transfer(xsm_default_t action, struct domain *d1,
                                      struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
-static inline int xsm_grant_copy(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_copy(xsm_default_t action, struct domain *d1,
                                  struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
-static inline int xsm_grant_query_size(xsm_default_t def, struct domain *d1,
+static inline int xsm_grant_query_size(xsm_default_t action, struct domain *d1,
                                        struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
@@ -174,87 +193,106 @@ static inline char *xsm_show_security_evtchn(struct domain *d,
     return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
-static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t action,
+                                           struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
-static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.get_pod_target, d);
 }
 
-static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.set_pod_target, d);
 }
 
-static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_exchange, d);
 }
 
-static inline int xsm_memory_adjust_reservation(xsm_default_t def,
+static inline int xsm_memory_adjust_reservation(xsm_default_t action,
                                                 struct domain *d1,
                                                 struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
-static inline int xsm_memory_stat_reservation(xsm_default_t def,
+static inline int xsm_memory_stat_reservation(xsm_default_t action,
                                               struct domain *d1,
                                               struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
-static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1,
+static inline int xsm_memory_pin_page(xsm_default_t action, struct domain *d1,
                                       struct domain *d2, struct page_info *page)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
-static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1,
+static inline int xsm_add_to_physmap(xsm_default_t action, struct domain *d1,
                                      struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
-static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1,
+static inline int xsm_remove_from_physmap(xsm_default_t action,
+                                          struct domain *d1,
                                           struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
-static inline int xsm_map_gmfn_foreign(xsm_default_t def, struct domain *d,
+static inline int xsm_map_gmfn_foreign(xsm_default_t action, struct domain *d,
                                        struct domain *t)
+
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
-static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
+static inline int xsm_claim_pages(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.claim_pages, d);
 }
 
-static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t action, struct domain *d,
+                                 int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
-static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t action, struct domain *d, int op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.profile, d, op);
 }
 
-static inline int xsm_kexec(xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.kexec);
 }
 
-static inline int xsm_schedop_shutdown(xsm_default_t def, struct domain *d1,
+static inline int xsm_schedop_shutdown(xsm_default_t action, struct domain *d1,
                                        struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
@@ -263,144 +301,172 @@ static inline char *xsm_show_irq_sid(int irq)
     return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
-static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
-static inline int xsm_map_domain_irq(xsm_default_t def, struct domain *d,
+static inline int xsm_map_domain_irq(xsm_default_t action, struct domain *d,
                                      int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
-static inline int xsm_unmap_domain_irq(xsm_default_t def, struct domain *d,
+static inline int xsm_unmap_domain_irq(xsm_default_t action, struct domain *d,
                                        int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
-static inline int xsm_bind_pt_irq(xsm_default_t def, struct domain *d,
+static inline int xsm_bind_pt_irq(xsm_default_t action, struct domain *d,
                                   struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
-static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
+static inline int xsm_unbind_pt_irq(xsm_default_t action, struct domain *d,
                                     struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
-static inline int xsm_irq_permission(xsm_default_t def, struct domain *d,
+static inline int xsm_irq_permission(xsm_default_t action, struct domain *d,
                                      int pirq, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
-static inline int xsm_iomem_permission(xsm_default_t def, struct domain *d,
+static inline int xsm_iomem_permission(xsm_default_t action, struct domain *d,
                                        uint64_t s, uint64_t e, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
-static inline int xsm_iomem_mapping(xsm_default_t def, struct domain *d,
+static inline int xsm_iomem_mapping(xsm_default_t action, struct domain *d,
                                     uint64_t s, uint64_t e, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
-static inline int xsm_pci_config_permission(xsm_default_t def, struct domain *d,
+static inline int xsm_pci_config_permission(xsm_default_t action,
+                                            struct domain *d,
                                             uint32_t machine_bdf,
                                             uint16_t start, uint16_t end,
                                             uint8_t access)
 {
-    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
+    XSM_ASSERT_ACTION(XSM_HOOK);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf,
+                            start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_get_device_group(xsm_default_t action,
+                                       uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
-static inline int xsm_assign_device(xsm_default_t def, struct domain *d,
+static inline int xsm_assign_device(xsm_default_t action, struct domain *d,
                                     uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
-static inline int xsm_deassign_device(xsm_default_t def, struct domain *d,
+static inline int xsm_deassign_device(xsm_default_t action, struct domain *d,
                                       uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static inline int xsm_assign_dtdevice(xsm_default_t def, struct domain *d,
+static inline int xsm_assign_dtdevice(xsm_default_t action, struct domain *d,
                                       const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
-static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
+static inline int xsm_deassign_dtdevice(xsm_default_t action, struct domain *d,
                                         const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t action,
+                                        uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
-static inline int xsm_resource_unplug_pci(xsm_default_t def,
+static inline int xsm_resource_unplug_pci(xsm_default_t action,
                                           uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
-static inline int xsm_resource_plug_core(xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_plug_core);
 }
 
-static inline int xsm_resource_unplug_core(xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_unplug_core);
 }
 
-static inline int xsm_resource_setup_pci(xsm_default_t def,
+static inline int xsm_resource_setup_pci(xsm_default_t action,
                                          uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t action, int gsi)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
-static inline int xsm_resource_setup_misc(xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_misc);
 }
 
-static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
+static inline int xsm_page_offline(xsm_default_t action, uint32_t cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.page_offline, cmd);
 }
 
-static inline int xsm_hypfs_op(xsm_default_t def)
+static inline int xsm_hypfs_op(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hypfs_op);
 }
 
@@ -416,161 +482,192 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 }
 #endif
 
-static inline int xsm_hvm_param(xsm_default_t def, struct domain *d,
+static inline int xsm_hvm_param(xsm_default_t action, struct domain *d,
                                 unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
-static inline int xsm_hvm_control(xsm_default_t def, struct domain *d,
+static inline int xsm_hvm_control(xsm_default_t action, struct domain *d,
                                   unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
-static inline int xsm_hvm_altp2mhvm_op(xsm_default_t def, struct domain *d,
+static inline int xsm_hvm_altp2mhvm_op(xsm_default_t action, struct domain *d,
                                        uint64_t mode, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
-static inline int xsm_vm_event_control(xsm_default_t def, struct domain *d,
+static inline int xsm_vm_event_control(xsm_default_t action, struct domain *d,
                                        int mode, int op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
-static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
-static inline int xsm_do_mca(xsm_default_t def)
+static inline int xsm_do_mca(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.do_mca);
 }
 
-static inline int xsm_shadow_control(xsm_default_t def, struct domain *d,
+static inline int xsm_shadow_control(xsm_default_t action, struct domain *d,
                                      uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
-static inline int xsm_mem_sharing_op(xsm_default_t def, struct domain *d,
+static inline int xsm_mem_sharing_op(xsm_default_t action, struct domain *d,
                                      struct domain *cd, int op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
-static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.apic, d, cmd);
 }
 
-static inline int xsm_memtype(xsm_default_t def, uint32_t access)
+#if 0
+/* Could not find any usages */
+static inline int xsm_memtype(xsm_default_t action, uint32_t access)
 {
     return alternative_call(xsm_ops.memtype, access);
 }
+#endif
 
-static inline int xsm_machine_memory_map(xsm_default_t def)
+static inline int xsm_machine_memory_map(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.machine_memory_map);
 }
 
-static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_memory_map(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
-static inline int xsm_mmu_update(xsm_default_t def, struct domain *d,
+static inline int xsm_mmu_update(xsm_default_t action, struct domain *d,
                                  struct domain *t, struct domain *f,
                                  uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
-static inline int xsm_mmuext_op(xsm_default_t def, struct domain *d,
+static inline int xsm_mmuext_op(xsm_default_t action, struct domain *d,
                                 struct domain *f)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
-static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d,
+static inline int xsm_update_va_mapping(xsm_default_t action, struct domain *d,
                                         struct domain *f, l1_pgentry_t pte)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_ops.update_va_mapping(d, f, pte);
 }
 
-static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d,
+static inline int xsm_priv_mapping(xsm_default_t action, struct domain *d,
                                    struct domain *t)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
-static inline int xsm_ioport_permission(xsm_default_t def, struct domain *d,
+static inline int xsm_ioport_permission(xsm_default_t action, struct domain *d,
                                         uint32_t s, uint32_t e, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
-static inline int xsm_ioport_mapping(xsm_default_t def, struct domain *d,
+static inline int xsm_ioport_mapping(xsm_default_t action, struct domain *d,
                                      uint32_t s, uint32_t e, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
-static inline int xsm_pmu_op(xsm_default_t def, struct domain *d,
+static inline int xsm_pmu_op(xsm_default_t action, struct domain *d,
                              unsigned int op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
-static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
+static inline int xsm_dm_op(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.dm_op, d);
 }
 
-static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.xen_version, op);
 }
 
-static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_resource_map(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
@@ -598,10 +695,4 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#include <xsm/dummy.h>
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index 09b9311b1d..121a0eda7d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
+obj-y += dummy.o
 obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
-obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index b848580eaa..e18afe0673 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,7 @@
  *  as published by the Free Software Foundation.
  */
 
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
diff --git a/xen/xsm/dummy.h b/xen/xsm/dummy.h
new file mode 100644
index 0000000000..b9a7e8c40f
--- /dev/null
+++ b/xen/xsm/dummy.h
@@ -0,0 +1,739 @@
+/*
+ *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
+ *
+ *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ *
+ *
+ *  Dummy XSM hooks implementing the default access check. Each hook should
+ *  have as its first line XSM_DEFAULT_ACTION declaring the privilege level
+ *  required for this access.
+ */
+
+#ifndef __XSM_DUMMY_H__
+#define __XSM_DUMMY_H__
+
+#include <xen/sched.h>
+#include <xsm/xsm-core.h>
+#include <public/hvm/params.h>
+
+#define XSM_DEFAULT_ACTION(def) xsm_default_t action = def; (void)action
+
+static always_inline int xsm_default_action(
+    xsm_default_t action, struct domain *src, struct domain *target)
+{
+    switch ( action ) {
+    case XSM_HOOK:
+        return 0;
+    case XSM_TARGET:
+        if ( evaluate_nospec(src == target) )
+        {
+            return 0;
+    case XSM_XS_PRIV:
+            if ( evaluate_nospec(is_xenstore_domain(src)) )
+                return 0;
+        }
+        /* fall through */
+    case XSM_DM_PRIV:
+        if ( target && evaluate_nospec(src->target == target) )
+            return 0;
+        /* fall through */
+    case XSM_PRIV:
+        if ( is_control_domain(src) )
+            return 0;
+        return -EPERM;
+    default:
+        XSM_LINKER_BUG_ON(1);
+        return -EPERM;
+    }
+}
+
+static inline void xsm_security_domaininfo(struct domain *d,
+    struct xen_domctl_getdomaininfo *info)
+{
+    return;
+}
+
+static inline int xsm_domain_create(struct domain *d, uint32_t ssidref)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_getdomaininfo(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_domctl_scheduler_op(struct domain *d, int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_sysctl_scheduler_op(int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_set_target(struct domain *d, struct domain *e)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_domctl(struct domain *d, int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_OTHER);
+    switch ( cmd )
+    {
+    case XEN_DOMCTL_ioport_mapping:
+    case XEN_DOMCTL_memory_mapping:
+    case XEN_DOMCTL_bind_pt_irq:
+    case XEN_DOMCTL_unbind_pt_irq:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_DOMCTL_getdomaininfo:
+        return xsm_default_action(XSM_XS_PRIV, current->domain, d);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, d);
+    }
+}
+
+static inline int xsm_sysctl(int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_readconsole(uint32_t clear)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_alloc_security_domain(struct domain *d)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_domain(struct domain *d)
+{
+    return;
+}
+
+static inline int xsm_grant_mapref(struct domain *d1, struct domain *d2,
+                                   uint32_t flags)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_grant_unmapref(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_grant_setup(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_grant_transfer(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_grant_copy(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_grant_query_size(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_memory_exchange(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_memory_adjust_reservation(struct domain *d1,
+                                                struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_memory_stat_reservation(struct domain *d1,
+                                              struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_console_io(struct domain *d, int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_OTHER);
+    if ( d->is_console )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#ifdef CONFIG_VERBOSE_DEBUG
+    if ( cmd == CONSOLEIO_write )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#endif
+    return xsm_default_action(XSM_PRIV, d, NULL);
+}
+
+static inline int xsm_profile(struct domain *d, int op)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d, NULL);
+}
+
+static inline int xsm_kexec(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_schedop_shutdown(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_memory_pin_page(struct domain *d1, struct domain *d2,
+                                      struct page_info *page)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_claim_pages(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_evtchn_unbound(struct domain *d, struct evtchn *chn,
+                                     domid_t id2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_evtchn_interdomain(struct domain *d1,
+                                         struct evtchn *chan1,
+                                         struct domain *d2,
+                                         struct evtchn *chan2)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
+{
+    return;
+}
+
+static inline int xsm_evtchn_send(struct domain *d, struct evtchn *chn)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, d, NULL);
+}
+
+static inline int xsm_evtchn_status(struct domain *d, struct evtchn *chn)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_evtchn_reset(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_alloc_security_evtchns(struct evtchn chn[],
+                                             unsigned int nr)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_evtchns(struct evtchn chn[],
+                                             unsigned int nr)
+{
+    return;
+}
+
+static inline char *xsm_show_security_evtchn(struct domain *d,
+                                             const struct evtchn *chn)
+{
+    return NULL;
+}
+
+static inline int xsm_init_hardware_domain(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_get_pod_target(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_set_pod_target(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_get_vnumainfo(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+static inline int xsm_get_device_group(uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_assign_device(struct domain *d, uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_deassign_device(struct domain *d, uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_PCI */
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+static inline int xsm_assign_dtdevice(struct domain *d, const char *dtpath)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_deassign_dtdevice(struct domain *d, const char *dtpath)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
+
+static inline int xsm_resource_plug_core(void)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_core(void)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_plug_pci(uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_pci(uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_pci(uint32_t machine_bdf)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_gsi(int gsi)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_misc(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_page_offline(uint32_t cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_hypfs_op(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+
+#ifdef CONFIG_COMPAT
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+#endif
+
+static inline char *xsm_show_irq_sid(int irq)
+{
+    return NULL;
+}
+
+static inline int xsm_map_domain_pirq(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_map_domain_irq(struct domain *d, int irq,
+                                     const void *data)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_pirq(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_bind_pt_irq(struct domain *d,
+                                  struct xen_domctl_bind_pt_irq *bind)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_unbind_pt_irq(struct domain *d,
+                                    struct xen_domctl_bind_pt_irq *bind)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_irq(struct domain *d, int irq,
+                                       const void *data)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_irq_permission(struct domain *d, int pirq, uint8_t allow)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_iomem_permission(struct domain *d, uint64_t s, uint64_t e,
+                                       uint8_t allow)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_iomem_mapping(struct domain *d, uint64_t s, uint64_t e,
+                                    uint8_t allow)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_pci_config_permission(struct domain *d,
+                                            uint32_t machine_bdf,
+                                            uint16_t start, uint16_t end,
+                                            uint8_t access)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d1, d2);
+}
+
+static inline int xsm_map_gmfn_foreign(struct domain *d, struct domain *t)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d, t);
+}
+
+static inline int xsm_hvm_param(struct domain *d, unsigned long op)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_hvm_control(struct domain *d, unsigned long op)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_hvm_param_altp2mhvm(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_hvm_altp2mhvm_op(struct domain *d, uint64_t mode,
+                                       uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_OTHER);
+
+    switch ( mode )
+    {
+    case XEN_ALTP2M_mixed:
+        return xsm_default_action(XSM_TARGET, current->domain, d);
+    case XEN_ALTP2M_external:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_ALTP2M_limited:
+        if ( HVMOP_altp2m_vcpu_enable_notify == op )
+            return xsm_default_action(XSM_TARGET, current->domain, d);
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    default:
+        return -EPERM;
+    }
+}
+
+static inline int xsm_vm_event_control(struct domain *d, int mode, int op)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#ifdef CONFIG_MEM_ACCESS
+static inline int xsm_mem_access(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+static inline int xsm_mem_paging(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+static inline int xsm_mem_sharing(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+#endif
+
+static inline int xsm_platform_op(uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+#ifdef CONFIG_X86
+static inline int xsm_do_mca(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_shadow_control(struct domain *d, uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_mem_sharing_op(struct domain *d, struct domain *cd,
+                                     int op)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, cd);
+}
+
+static inline int xsm_apic(struct domain *d, int cmd)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, d, NULL);
+}
+
+static inline int xsm_machine_memory_map(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(action, current->domain, NULL);
+}
+
+static inline int xsm_domain_memory_map(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_mmu_update(struct domain *d, struct domain *t,
+                                 struct domain *f, uint32_t flags)
+{
+    int rc = 0;
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    if ( f != dom_io )
+        rc = xsm_default_action(action, d, f);
+    if ( evaluate_nospec(t) && !rc )
+        rc = xsm_default_action(action, d, t);
+    return rc;
+}
+
+static inline int xsm_mmuext_op(struct domain *d, struct domain *f)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d, f);
+}
+
+static inline int xsm_update_va_mapping(struct domain *d, struct domain *f,
+                                        l1_pgentry_t pte)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d, f);
+}
+
+static inline int xsm_priv_mapping(struct domain *d, struct domain *t)
+{
+    XSM_DEFAULT_ACTION(XSM_TARGET);
+    return xsm_default_action(action, d, t);
+}
+
+static inline int xsm_ioport_permission(struct domain *d, uint32_t s,
+                                        uint32_t e, uint8_t allow)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_ioport_mapping(struct domain *d, uint32_t s, uint32_t e,
+                                     uint8_t allow)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(action, current->domain, d);
+}
+
+static inline int xsm_pmu_op(struct domain *d, unsigned int op)
+{
+    XSM_DEFAULT_ACTION(XSM_OTHER);
+    switch ( op )
+    {
+    case XENPMU_init:
+    case XENPMU_finish:
+    case XENPMU_lvtpc_set:
+    case XENPMU_flush:
+        return xsm_default_action(XSM_HOOK, d, current->domain);
+    default:
+        return xsm_default_action(XSM_PRIV, d, current->domain);
+    }
+}
+
+#endif /* CONFIG_X86 */
+
+static inline int xsm_dm_op(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#ifdef CONFIG_ARGO
+static inline int xsm_argo_enable(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_single_source(const struct domain *d,
+                                                  const struct domain *t)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_any_source(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
+{
+    return 0;
+}
+
+#endif /* CONFIG_ARGO */
+
+#include <public/version.h>
+static inline int xsm_xen_version(uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_OTHER);
+    switch ( op )
+    {
+    case XENVER_version:
+    case XENVER_platform_parameters:
+    case XENVER_get_features:
+        /* These sub-ops ignore the permission checks and return data. */
+        block_speculation();
+        return 0;
+    case XENVER_extraversion:
+    case XENVER_compile_info:
+    case XENVER_capabilities:
+    case XENVER_changeset:
+    case XENVER_pagesize:
+    case XENVER_guest_handle:
+        /* These MUST always be accessible to any guest by default. */
+        return xsm_default_action(XSM_HOOK, current->domain, NULL);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, NULL);
+    }
+}
+
+static inline int xsm_domain_resource_map(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(action, current->domain, d);
+}
+
+#endif /* __XSM_DUMMY_H__ */
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 56a330a831..1bc8277b85 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -18,7 +18,7 @@
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
 #include <xsm/xsm-core.h>
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 /*
  * Check if inter-domain communication is allowed.
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 87c2980459..f899950d67 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,8 +18,6 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
 #endif
@@ -227,8 +225,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 15:42:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 15:42:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178233.324126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMBKO-0000m9-Cr; Fri, 03 Sep 2021 15:42:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178233.324126; Fri, 03 Sep 2021 15:42:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMBKO-0000m2-9W; Fri, 03 Sep 2021 15:42:04 +0000
Received: by outflank-mailman (input) for mailman id 178233;
 Fri, 03 Sep 2021 15:42:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kvJD=NZ=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mMBKM-0000lw-W4
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 15:42:03 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.87]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 13f325b0-82b0-4577-8d40-c31b587b4a9a;
 Fri, 03 Sep 2021 15:42:01 +0000 (UTC)
Received: from DB7PR05CA0008.eurprd05.prod.outlook.com (2603:10a6:10:36::21)
 by VI1PR08MB3869.eurprd08.prod.outlook.com (2603:10a6:803:bf::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Fri, 3 Sep
 2021 15:41:57 +0000
Received: from DB5EUR03FT056.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:36:cafe::55) by DB7PR05CA0008.outlook.office365.com
 (2603:10a6:10:36::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Fri, 3 Sep 2021 15:41:57 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT056.mail.protection.outlook.com (10.152.21.124) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Fri, 3 Sep 2021 15:41:56 +0000
Received: ("Tessian outbound 5918cb94a4b5:v103");
 Fri, 03 Sep 2021 15:41:56 +0000
Received: from 225ad6a0dd7c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 667FB746-F948-4870-BAF1-10147472F887.1; 
 Fri, 03 Sep 2021 15:41:49 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 225ad6a0dd7c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 03 Sep 2021 15:41:49 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6799.eurprd08.prod.outlook.com (2603:10a6:102:13b::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Fri, 3 Sep
 2021 15:41:47 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.022; Fri, 3 Sep 2021
 15:41:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13f325b0-82b0-4577-8d40-c31b587b4a9a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=S/2w3w+Uf621jp6T0BcYnlrni+StxWj3wQqAu2bVjOk=;
 b=VlcFFBHWdSbfRa/jMXlr9MArzklV7JmpLw4iXSzvbjDqH9oS7CTdlOdkGmUaOkePemZVQkUKPGRlmLYt9YM0udgi4nFA7xS9YpJMHCdEIzVhg6cv2ePmgnak7f1ajA0qJhwAhODooPygtzY9S5aw2MwNHBZc0OHJEHDKNJFWQuA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 674871f30fdc51aa
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mAJ//xP/Ak0GYtnAYkTXpGRaapCgREl0F3WrxCOLn3K+eNgdC89Yvsy4vkmQp+TuyJQVjDdoMcynUDK0MOXpt6VcrN23EqgVULtg2ha14CJurPCvcjHzM/qR8pLsWnoSXDZdkXJHYW6C1+OVTA5NXE1qv6zwQZv6yH+GtdT+n1cRvBRYDOoMi9QdoMy2vQP9NMbeMr7tnXBxFPiD8IKyLFTM2kKY1A5aDcBef8H2Vsf83gpe61jNq+SBwY2jJghdjrg5vv8bEVd0p7ixtfkXSrJvihDRHeYX+k0xxfvlY44vYdeXNQkNbDFodgSC2AJD9WK+2zZ2foipzvvV17QJCw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=S/2w3w+Uf621jp6T0BcYnlrni+StxWj3wQqAu2bVjOk=;
 b=jHNuHUXlhPpY9T0jkDt2WRo1n7B1G1ZbhN6mvC1pqJELbXN5Eny8bPjr0ViAUKjwluzNYnPVgoE3rkEz2TP7DW/Yl80dci7svzxcSom/S98NRz9GRWReufPS4q3ixFLKqpEdiCSoC0dGhOd9gJ6Z9YGoZBYLgEp0v/CJyWs5vZtbMBiKvdlBx8T9kIEQrBiKh3hferlWI4B3LpDwK59qymG41bD9krrajJf21XSUqggt4v5YE/reQMJI7S9hUiAmcBdPW/GO7ew+y7kybA2vPRf2laAjgiMDK0AXzE2L2Wuu0YyOqG1EtnBgFxeS9/wxzXpdeXaHj+09FnO72OPwkQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=S/2w3w+Uf621jp6T0BcYnlrni+StxWj3wQqAu2bVjOk=;
 b=VlcFFBHWdSbfRa/jMXlr9MArzklV7JmpLw4iXSzvbjDqH9oS7CTdlOdkGmUaOkePemZVQkUKPGRlmLYt9YM0udgi4nFA7xS9YpJMHCdEIzVhg6cv2ePmgnak7f1ajA0qJhwAhODooPygtzY9S5aw2MwNHBZc0OHJEHDKNJFWQuA=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Juergen Gross <jgross@suse.com>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: Crash when using cpupools
Thread-Topic: Crash when using cpupools
Thread-Index: AQHXoNo0buBDGTl+VEiGgPdGr1E/CQ==
Date: Fri, 3 Sep 2021 15:41:47 +0000
Message-ID: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: lists.xenproject.org; dkim=none (message not
 signed) header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 136f5789-ccd3-453b-df0a-08d96ef15c98
x-ms-traffictypediagnostic: PAXPR08MB6799:|VI1PR08MB3869:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3869DE9D4E2C81276D2984F69DCF9@VI1PR08MB3869.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 7dym35IrXBwamjNRfh7cop5gJF4z4KZ2UiPBiWKXjnuUB3g8lI4r46eREK55+Fw3yPJ4dcLuJo9qii6X+vZ8PwmBLehYkzhZT1dOhsDGRt/9NBfgQnMOX9BgU5oElgupoLDtPfGNVQ1/LB5OsE4jQferXw99AlMZyLtdCsrPi6kTEZMXWIhrRQxb8qDIhCMWs+6guTYQirirAqejcOWw0v/cu6VYxcivJWQG8wR0Z/xSx4O/4JU0Mc5fKehgSiChtPuL1vKHDfQEwTnYDyM4QvAuk4I7uVdpw+2Iirn30QE5mPeNGsanBHQ9Q8sn5jhQ+Pm7HSA5I5NDA088PBAk2qkCiCxZ32XQJQ9RqijlcXdsN4AndDWLQ7LCgrQoQ63xU0lbJBaENYDWZVgMn5+QksND/JqkMnTzWcZ5FHkwBC+h4tCpP/2JmDLPLtlarvgU2OawlB3F6PX73CNaBuS0Kfx0jg8Uu/m/RwMtAfTTkOz+Rnd0CoHsKt72RDdV1fEnVWx7cDb3kXLe6wP8SWGLxbqeaX3hetbD2WpvQ+tuAMc0kqrFtHMXYvGy76UZIsuluHslFazMAxH/+DlVg0JcMOVjT+Y2JKuylZuWdlUGXTrPpcjIEMsyjmToohei5dMuOcfXazm5WzZHoxEkOJQ1FY6j9wAoow5XtONCHlPpt4AWiSbrvCb6y+QE0Wkm/Qm/pwndWcPr9mF9bJ+MQwmdzcPsw8xaMptZnecFjeB2lV428P/sAvF1RMMkR4QTP/Mu
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(366004)(39860400002)(376002)(346002)(136003)(83380400001)(2906002)(5660300002)(66476007)(66946007)(66556008)(86362001)(71200400001)(2616005)(8936002)(8676002)(54906003)(478600001)(6506007)(33656002)(38100700002)(6916009)(36756003)(122000001)(6512007)(186003)(26005)(316002)(64756008)(3480700007)(6486002)(4326008)(66446008)(91956017)(76116006)(38070700005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <C7AC348BCC95A142BB182AA77835CC61@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6799
Original-Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6f00d02b-faf6-4da4-fb27-08d96ef15704
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xr+UaOTRAtJTdVTBqZSKnBPROHXpDV7fPSIiN4n3uKEzkCclHsIbHqI6l2kddFAJEfWP0nt7jOPhycN+vcoWTuA/kTZOSEolMPMK1qbfkgJ0WDqVvrBA/YiHY6F9RdaayPY3s4MMuMD5gBlY0mQmTcOLFuMpYX41NWlJBfobapPFrBfxq5DtyMXyeoNKsCNumRY0SVjVOWdOgjQOuSQdwmbNp4SZ67N6UD7j9akq0fJTQgYlRHlYsjlGQfpb37fckwcPPgMZrnRU7D6Kxp0j4pS/pNoyGE0p5qgJXxIOU5/r127/dRlk3KGyvudC19OzYKgleTu2YdSEQ/B2Q84cti/vH9IStt3MG0hRX6A2w8IgGs1GkcVFP3jwzNwrp772v9X3jMvbi7mCbhvimXOtvnhx9TQyNa70TanJziCMlJbZ8wObpf9CrJNmY5LDa6+3LtDHM62oZGjm7ujQzDLVgg/HRB8GKy6L3fcPFlRW8nJZDqk0UQNNWZfY1sMxN9ekYd1+5k3paPaSGMlem3UsELdeHuJ34u997Qb4jVsgIz91lIEMbhJKU7T//MLjhdlk2Uy+Gev1hHPDTn+XA999Qo0LdgaIDtrdavashXb6b55gsYegmjAm0CjdqLJWKTzMdzJ9R630KQtN60oOmlDsAzR7sO1Tnlli/tyeiWb8pAEpHH8XMe3BZDe7u5buWRpy7+gF+jvDM0MvjFlEIymz9w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(107886003)(508600001)(5660300002)(356005)(26005)(4326008)(8936002)(3480700007)(6512007)(6486002)(186003)(82310400003)(70586007)(86362001)(47076005)(83380400001)(70206006)(36756003)(36860700001)(2906002)(6916009)(8676002)(6506007)(2616005)(54906003)(336012)(316002)(81166007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Sep 2021 15:41:56.9242
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 136f5789-ccd3-453b-df0a-08d96ef15c98
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3869

SGksDQoNCldoaWxlIGRvaW5nIHNvbWUgaW52ZXN0aWdhdGlvbiB3aXRoIGNwdXBvb2xzIEkgZW5j
b3VudGVyZWQgYSBjcmFzaCB3aGVuIHRyeWluZyB0byBpc29sYXRlIGEgZ3Vlc3QgdG8gaXRzIG93
biBwaHlzaWNhbCBjcHUuDQoNCkkgYW0gdXNpbmcgY3VycmVudCBzdGFnaW5nIHN0YXR1cy4NCg0K
SSBkaWQgdGhlIGZvbGxvd2luZyAob24gRlZQIHdpdGggOCBjb3Jlcyk6DQotIHN0YXJ0IGRvbTAg
d2l0aCBkb20wX21heF92Y3B1cz0xDQotIHJlbW92ZSBjb3JlIDEgZnJvbSBkb20wIGNwdXBvb2w6
IHhsIGNwdXBvb2wtY3B1LXJlbW92ZSBQb29sLTAgMQ0KLSBjcmVhdGUgYSBuZXcgcG9vbDogeGwg
Y3B1cG9vbC1jcmVhdGUgbmFtZT1cIk5ldFBvb2xc4oCdDQotIGFkZCBjb3JlIDEgdG8gdGhlIHBv
b2w6IHhsIGNwdXBvb2wtY3B1LWFkZCBOZXRQb29sIDENCi0gY3JlYXRlIGEgZ3Vlc3QgaW4gTmV0
UG9vbCB1c2luZyB0aGUgZm9sbG93aW5nIGluIHRoZSBndWVzdCBjb25maWc6IHBvb2w94oCcTmV0
UG9vbCINCg0KSSBlbmQgd2l0aCBhIGNyYXNoIHdpdGggdGhlIGZvbGxvd2luZyBjYWxsIHRyYWNl
IGR1cmluZyBndWVzdCBjcmVhdGlvbjoNCihYRU4pIFhlbiBjYWxsIHRyYWNlOg0KKFhFTikgICAg
WzwwMDAwMDAwMDAwMjM0Y2IwPl0gY3JlZGl0Mi5jI2NzY2hlZDJfYWxsb2NfdWRhdGErMHg1OC8w
eGZjIChQQykNCihYRU4pICAgIFs8MDAwMDAwMDAwMDIzNGM4MD5dIGNyZWRpdDIuYyNjc2NoZWQy
X2FsbG9jX3VkYXRhKzB4MjgvMHhmYyAoTFIpDQooWEVOKSAgICBbPDAwMDAwMDAwMDAyNDJkMzg+
XSBzY2hlZF9tb3ZlX2RvbWFpbisweDE0NC8weDZjMA0KKFhFTikgICAgWzwwMDAwMDAwMDAwMjJk
ZDE4Pl0gY3B1cG9vbC5jI2NwdXBvb2xfbW92ZV9kb21haW5fbG9ja2VkKzB4MzgvMHg3MA0KKFhF
TikgICAgWzwwMDAwMDAwMDAwMjJmYWRjPl0gY3B1cG9vbF9kb19zeXNjdGwrMHg3M2MvMHg3ODAN
CihYRU4pICAgIFs8MDAwMDAwMDAwMDIyZDhlMD5dIGRvX3N5c2N0bCsweDc4OC8weGE1OA0KKFhF
TikgICAgWzwwMDAwMDAwMDAwMjczYjY4Pl0gdHJhcHMuYyNkb190cmFwX2h5cGVyY2FsbCsweDc4
LzB4MTcwDQooWEVOKSAgICBbPDAwMDAwMDAwMDAyNzRmNzA+XSBkb190cmFwX2d1ZXN0X3N5bmMr
MHgxMzgvMHg2MTgNCihYRU4pICAgIFs8MDAwMDAwMDAwMDI2MDQ1OD5dIGVudHJ5Lm8jZ3Vlc3Rf
c3luY19zbG93cGF0aCsweGE0LzB4ZDQNCg0KQWZ0ZXIgc29tZSBkZWJ1Z2dpbmcgSSBmb3VuZCBv
dXQgdGhhdCB1bml0LT52Y3B1X2xpc3QgaXMgTlVMTCBhZnRlciB1bml0LT52Y3B1X2xpc3QgPSBk
LT52Y3B1W3VuaXQtPnVuaXRfaWRdOyB3aXRoIHVuaXRfaWQgMCBpbiBjb21tb24vc2NoZWQvY29y
ZS5jOjY4OA0KVGhpcyBtYWtlcyB0aGUgY2FsbCB0byBpc19pZGxlX3VuaXQodW5pdCkgaW4gY3Nj
aGVkMl9hbGxvY191ZGF0YSB0cmlnZ2VyIHRoZSBjcmFzaC4NCg0KQW0gSSBkb2luZyBzb21ldGhp
bmcgd3JvbmcgPw0KSXMgaXQgYSBrbm93IHByb2JsZW0gPw0KRG9lcyBhbnlib2R5IGhhdmUgYSBw
b2ludGVyIG9mIHdoZXJlIHRoZSBzb3VyY2Ugb2YgdGhlIGlzc3VlIG1pZ2h0IGJlID8NCg0KSSBo
YXZlIGEgd29ya2luZyBzZXR1cCB0byBkZWJ1ZyB0aGlzIGlmIGFueSBvdGhlciBpbmZvIGlzIHJl
cXVpcmVkIHRvIHVuZGVyc3RhbmQgdGhlIGlzc3VlLg0KDQpDaGVlcnMNCkJlcnRyYW5kDQoNCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 16:32:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 16:32:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178246.324137 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMC6V-00076K-1T; Fri, 03 Sep 2021 16:31:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178246.324137; Fri, 03 Sep 2021 16:31:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMC6U-00076D-Ul; Fri, 03 Sep 2021 16:31:46 +0000
Received: by outflank-mailman (input) for mailman id 178246;
 Fri, 03 Sep 2021 16:31:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mMC6U-000767-0P
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 16:31:46 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mMC6R-0000Qk-Ot; Fri, 03 Sep 2021 16:31:43 +0000
Received: from 54-240-197-239.amazon.com ([54.240.197.239]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mMC6R-000422-I3; Fri, 03 Sep 2021 16:31:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=L890QpMZCiO1y4oZR4rTouOouMUVodQHVV5GineFeIA=; b=UQ0wpUazcNWDt94uph65luMvXT
	mUTkmLzPCdA6yLy6TQwFKq6nWFU/FF7hRMAz+QTx1SnP6sZQvJfls+wy0Jf5TlcjVQn+q8uoRpkZp
	2sSSotRocqXtWs89UhmqShvLTtRJTKqWhxOnjG0WEI4x6jlkOwhFfQnUzuQ0fp90gENM=;
Subject: Re: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9db57ad3-ce25-c753-4f24-ff8396ad5d18@xen.org>
Date: Fri, 3 Sep 2021 17:31:41 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 01/09/2021 09:45, Jan Beulich wrote:
> Commit 806448806264 ("xen/domain: Fix label position in
> domain_teardown()" has caused Coverity to report a _new_ supposedly
> un-annotated fall-through in a switch(). I find this (once again)
> puzzling; I'm having an increasingly hard time figuring what patterns
> the tool is actually after. I would have expected that the tool would
> either have spotted an issue also before this change, or not at all. Yet
> if it had spotted one before, the statistics report should have included
> an eliminated instance alongside the new one (because then the issue
> would simply have moved by a few lines).
> 
> Hence the only thing I could guess is that the treatment of comments in
> macro expansions might be subtly different. Therefore try whether
> switching the comments to the still relatively new "fallthrough" pseudo
> keyword actually helps.
> 
> Coverity-ID: 1490865
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Julien Grall <jgrall@amazon.com>

> ---
> If this doesn't help, I'm afraid I'm lost as to what Coverity means us
> to do to silence the reporting.
> 
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -401,13 +401,13 @@ static int domain_teardown(struct domain
>            */
>   #define PROGRESS(x)                             \
>           d->teardown.val = PROG_ ## x;           \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>       case PROG_ ## x
>   
>   #define PROGRESS_VCPU(x)                        \
>           d->teardown.val = PROG_vcpu_ ## x;      \
>           d->teardown.vcpu = v;                   \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>       case PROG_vcpu_ ## x:                       \
>           v = d->teardown.vcpu
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 17:35:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 17:35:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178293.324183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMD5v-0007ea-Mi; Fri, 03 Sep 2021 17:35:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178293.324183; Fri, 03 Sep 2021 17:35:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMD5v-0007eT-Jj; Fri, 03 Sep 2021 17:35:15 +0000
Received: by outflank-mailman (input) for mailman id 178293;
 Fri, 03 Sep 2021 17:35:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMD5u-0007eJ-Fu; Fri, 03 Sep 2021 17:35:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMD5u-0001UE-5C; Fri, 03 Sep 2021 17:35:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMD5t-0001RP-TY; Fri, 03 Sep 2021 17:35:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMD5t-0006Ww-T4; Fri, 03 Sep 2021 17:35:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3YLQ4d55VlFI+VvAEzCVZUzqI9QqXYuP/0B4u0KPlPo=; b=eiUPv/rB8k26Z1qgWXpfLS7DQB
	cw0hYPinaCCVr8xIivlatg/qmiUdf2DvTGTGw6bIgU1xmklUfuFcSd04/2AgY4uLNq5iKhyPx1cvb
	retAjp9DSRQWTym13Dx8PUNnUfmc0S7NBqmYt4+pyJxqnotLEJ3FUDHhQF4diXHXCK/k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164796-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164796: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 17:35:13 +0000

flight 164796 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164796/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618

Last test of basis   164684  2021-08-31 20:01:34 Z    2 days
Testing same since   164796  2021-09-03 14:01:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Christian Lindig <christian.lindig@citrix.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   96607a8e68..2d4978ead4  2d4978ead4b98452bc45ebe47952e524b12e2b41 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 18:20:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 18:20:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178300.324198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMDnE-0004AW-4c; Fri, 03 Sep 2021 18:20:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178300.324198; Fri, 03 Sep 2021 18:20:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMDnE-0004AP-0O; Fri, 03 Sep 2021 18:20:00 +0000
Received: by outflank-mailman (input) for mailman id 178300;
 Fri, 03 Sep 2021 18:19:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wQk4=NZ=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mMDnD-0004AJ-1e
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 18:19:59 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d25ceb54-55f7-4eb0-a243-faf6ec6d229e;
 Fri, 03 Sep 2021 18:19:57 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id m9so9411815wrb.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 11:19:57 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id u23sm136370wmc.24.2021.09.03.11.19.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 Sep 2021 11:19:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: d25ceb54-55f7-4eb0-a243-faf6ec6d229e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=k3EjGTKaI5g2mQLOHoezT/qbWejrZejERiPKCtXq8gw=;
        b=ojXrCIeyfH+YVO9iz1+TNZ9ezRMcW6h0jHjOAtJni4y0ai5GgfLbbVKAXjbpWwN84L
         Wm56dutgcXCYft6ubwqE6izEV7rcqTas8IDw2060mRAfdL2U6dvcLmlcGNbagstYZVSf
         jve/9McQZrfnans26GBi94oDOXS/nSfcjfw8rvKdqsA1IniKQVKQQbc7TUA0wMVS7nvT
         TBnmW1VCvwKuQQGF8INVzA1baogQfGV8Cas8UMEOVGUt7odj+mlXP4aVg2IUpEe1kmu8
         QYWmHgIkGuFnhk+83UH6zljZXUsTAJP42n6FB7tIoIdWXO+avlqow18bOmY8gb6Phemw
         R0jQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=k3EjGTKaI5g2mQLOHoezT/qbWejrZejERiPKCtXq8gw=;
        b=XfF9cAChzRC/MmnQT32mfgNCp8u3lnquqpHa9u8I867G7NBDSL5GpRvvE0HhrVnqFi
         5/CR/YW60buIQD9xIkZjtH8MTq23Tg+/0YfQoYQdYoNtl+fEHH5kWfLX9Vxk9N1gQhmN
         /ur+V6+oazKy5mlOlFY8ataHczTfXu7dAcsYnBx9DBgt/oL4zWO9JQKdh98OJEF3T3o3
         Ac50sI3HDLCOkzZESnwvEPWKhs/FEaU8TKWmV9ijzso3JIKqaGsURGZQQ++eait9izdj
         seEWHNO73G7t/ERwFhMlzjfA8DP9Y1ESudysEG9+bfBzpUaX46sPfMr2+JjiPAkMfzAw
         mbfg==
X-Gm-Message-State: AOAM530JmU7js84t8JMQtmy8G5oRDSq47XBIdl7ZUGmx3JH7RJKQM+Vw
	XCEPjoTd1EyeaMlnORYioN4=
X-Google-Smtp-Source: ABdhPJx0dZCgp0zUQLbWfMU8EmsUBXRCSeLlfDHicQVXCHRyEM3DqNc6Zbv9ZU3Q+HTVaH2wLnC6rg==
X-Received: by 2002:a5d:5642:: with SMTP id j2mr433139wrw.264.1630693196710;
        Fri, 03 Sep 2021 11:19:56 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Cameron Esfahani <dirty@apple.com>,
	Claudio Fontana <cfontana@suse.de>,
	Colin Xu <colin.xu@intel.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	haxm-team@intel.com,
	Kamil Rytarowski <kamil@netbsd.org>,
	Laurent Vivier <lvivier@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Paul Durrant <paul@xen.org>,
	Reinoud Zandijk <reinoud@netbsd.org>,
	Roman Bolshakov <r.bolshakov@yadro.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Huth <thuth@redhat.com>,
	Wenchao Wang <wenchao.wang@intel.com>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Subject: [PATCH v3 30/30] accel: Add missing AccelOpsClass::has_work() and drop SysemuCPUOps one
Date: Fri,  3 Sep 2021 20:19:43 +0200
Message-Id: <20210903181943.763360-1-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210902161543.417092-1-f4bug@amsat.org>
References: <20210902161543.417092-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

cpu_common_has_work() is the default has_work() implementation
and returns 'false'.

Explicit it for the QTest / HAX / HVF / NVMM / Xen accelerators
and remove cpu_common_has_work().

Since there are no more implementations of SysemuCPUOps::has_work,
remove it along with the assertion in cpu_has_work().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 include/hw/core/cpu.h             | 2 --
 accel/hvf/hvf-accel-ops.c         | 6 ++++++
 accel/qtest/qtest.c               | 6 ++++++
 accel/xen/xen-all.c               | 6 ++++++
 hw/core/cpu-common.c              | 6 ------
 softmmu/cpus.c                    | 9 ++-------
 target/i386/hax/hax-accel-ops.c   | 6 ++++++
 target/i386/nvmm/nvmm-accel-ops.c | 6 ++++++
 8 files changed, 32 insertions(+), 15 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index e2dd171a13f..c64709b898c 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -89,7 +89,6 @@ struct SysemuCPUOps;
  * instantiatable CPU type.
  * @parse_features: Callback to parse command line arguments.
  * @reset_dump_flags: #CPUDumpFlags to use for reset logging.
- * @has_work: Callback for checking if there is work to do.
  * @memory_rw_debug: Callback for GDB memory access.
  * @dump_state: Callback for dumping state.
  * @get_arch_id: Callback for getting architecture-dependent CPU ID.
@@ -132,7 +131,6 @@ struct CPUClass {
     void (*parse_features)(const char *typename, char *str, Error **errp);
 
     int reset_dump_flags;
-    bool (*has_work)(CPUState *cpu);
     int (*memory_rw_debug)(CPUState *cpu, vaddr addr,
                            uint8_t *buf, int len, bool is_write);
     void (*dump_state)(CPUState *cpu, FILE *, int flags);
diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index d1691be9896..53c427ee42e 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -446,6 +446,11 @@ static void hvf_start_vcpu_thread(CPUState *cpu)
                        cpu, QEMU_THREAD_JOINABLE);
 }
 
+static bool hvf_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void hvf_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
@@ -456,6 +461,7 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, void *data)
     ops->synchronize_post_init = hvf_cpu_synchronize_post_init;
     ops->synchronize_state = hvf_cpu_synchronize_state;
     ops->synchronize_pre_loadvm = hvf_cpu_synchronize_pre_loadvm;
+    ops->has_work = hvf_cpu_has_work;
 };
 static const TypeInfo hvf_accel_ops_type = {
     .name = ACCEL_OPS_NAME("hvf"),
diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c
index 7e6b8110d52..eb5a17cef18 100644
--- a/accel/qtest/qtest.c
+++ b/accel/qtest/qtest.c
@@ -47,12 +47,18 @@ static const TypeInfo qtest_accel_type = {
 };
 module_obj(TYPE_QTEST_ACCEL);
 
+static bool qtest_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void qtest_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
     ops->create_vcpu_thread = dummy_start_vcpu_thread;
     ops->get_virtual_clock = qtest_get_virtual_clock;
+    ops->has_work = qtest_cpu_has_work;
 };
 
 static const TypeInfo qtest_accel_ops_type = {
diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index 69aa7d018b2..fe5a37fa2e6 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -215,11 +215,17 @@ static const TypeInfo xen_accel_type = {
     .class_init = xen_accel_class_init,
 };
 
+static bool xen_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void xen_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
     ops->create_vcpu_thread = dummy_start_vcpu_thread;
+    ops->has_work = xen_cpu_has_work;
 }
 
 static const TypeInfo xen_accel_ops_type = {
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index e2f5a646046..5ed1ccdfdd5 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -143,11 +143,6 @@ static void cpu_common_reset(DeviceState *dev)
     }
 }
 
-static bool cpu_common_has_work(CPUState *cs)
-{
-    return false;
-}
-
 ObjectClass *cpu_class_by_name(const char *typename, const char *cpu_model)
 {
     CPUClass *cc = CPU_CLASS(object_class_by_name(typename));
@@ -279,7 +274,6 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->parse_features = cpu_common_parse_features;
     k->get_arch_id = cpu_common_get_arch_id;
-    k->has_work = cpu_common_has_work;
     k->gdb_read_register = cpu_common_gdb_read_register;
     k->gdb_write_register = cpu_common_gdb_write_register;
     set_bit(DEVICE_CATEGORY_CPU, dc->categories);
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 6bce52ce561..e6dad2243c6 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -251,13 +251,8 @@ void cpu_interrupt(CPUState *cpu, int mask)
 
 bool cpu_has_work(CPUState *cpu)
 {
-    CPUClass *cc = CPU_GET_CLASS(cpu);
-
-    if (cpus_accel->has_work) {
-        return cpus_accel->has_work(cpu);
-    }
-    g_assert(cc->has_work);
-    return cc->has_work(cpu);
+    g_assert(cpus_accel->has_work);
+    return cpus_accel->has_work(cpu);
 }
 
 static int do_vm_stop(RunState state, bool send_stop)
diff --git a/target/i386/hax/hax-accel-ops.c b/target/i386/hax/hax-accel-ops.c
index 136630e9b23..5407ba17eaf 100644
--- a/target/i386/hax/hax-accel-ops.c
+++ b/target/i386/hax/hax-accel-ops.c
@@ -74,6 +74,11 @@ static void hax_start_vcpu_thread(CPUState *cpu)
 #endif
 }
 
+static bool hax_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void hax_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
@@ -85,6 +90,7 @@ static void hax_accel_ops_class_init(ObjectClass *oc, void *data)
     ops->synchronize_post_init = hax_cpu_synchronize_post_init;
     ops->synchronize_state = hax_cpu_synchronize_state;
     ops->synchronize_pre_loadvm = hax_cpu_synchronize_pre_loadvm;
+    ops->has_work = hax_cpu_has_work;
 }
 
 static const TypeInfo hax_accel_ops_type = {
diff --git a/target/i386/nvmm/nvmm-accel-ops.c b/target/i386/nvmm/nvmm-accel-ops.c
index f788f75289f..36296f79ff8 100644
--- a/target/i386/nvmm/nvmm-accel-ops.c
+++ b/target/i386/nvmm/nvmm-accel-ops.c
@@ -83,6 +83,11 @@ static void nvmm_kick_vcpu_thread(CPUState *cpu)
     cpus_kick_thread(cpu);
 }
 
+static bool nvmm_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void nvmm_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
@@ -94,6 +99,7 @@ static void nvmm_accel_ops_class_init(ObjectClass *oc, void *data)
     ops->synchronize_post_init = nvmm_cpu_synchronize_post_init;
     ops->synchronize_state = nvmm_cpu_synchronize_state;
     ops->synchronize_pre_loadvm = nvmm_cpu_synchronize_pre_loadvm;
+    ops->has_work = nvmm_cpu_has_work;
 }
 
 static const TypeInfo nvmm_accel_ops_type = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 03 19:18:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 19:18:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178309.324209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMEi7-0002en-Kp; Fri, 03 Sep 2021 19:18:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178309.324209; Fri, 03 Sep 2021 19:18:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMEi7-0002eg-Hr; Fri, 03 Sep 2021 19:18:47 +0000
Received: by outflank-mailman (input) for mailman id 178309;
 Fri, 03 Sep 2021 19:18:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMEi6-0002eW-OW; Fri, 03 Sep 2021 19:18:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMEi6-0003FF-Bg; Fri, 03 Sep 2021 19:18:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMEi5-0006JE-Vm; Fri, 03 Sep 2021 19:18:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMEi5-0006pr-VG; Fri, 03 Sep 2021 19:18:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CYPHhQXpI/icUiou3nYStBOWQ/ZFnJpR6PiaB6dFq8g=; b=f+WX48WlKocIa/4KEEQdwMOWZr
	5Hasqd+Z3ZIrq78IuRkNoKdc59udXLuvPUggRcLk7vTb/uVEkBnxBOEGyD0lpWAQ6fOeysd8fDPpA
	NaWgEwP/MM06roekjKcEyV+FVOGhraR9hEUAD+rGYTwWdz2jntXo4m/7yanqRb9leJFo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164792-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164792: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=8664d30a30fd676b56b4c29dbcbdd5c5538acfc1
X-Osstest-Versions-That:
    qemuu=9093028dd48c50bc0392791f78aab44afef57ead
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 19:18:45 +0000

flight 164792 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164792/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164772
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164772
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164772
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164772
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164772
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164772
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164772
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164772
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                8664d30a30fd676b56b4c29dbcbdd5c5538acfc1
baseline version:
 qemuu                9093028dd48c50bc0392791f78aab44afef57ead

Last test of basis   164772  2021-09-02 18:37:11 Z    1 days
Testing same since   164792  2021-09-03 07:39:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Bulekov <alxndr@bu.edu>
  Darren Kenny <darren.kenny@oracle.com>
  Peter Maydell <peter.maydell@linaro.org>
  Qiuhao Li <Qiuhao.Li@outlook.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   9093028dd4..8664d30a30  8664d30a30fd676b56b4c29dbcbdd5c5538acfc1 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 19:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 19:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178317.324222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMEu7-0005EV-QW; Fri, 03 Sep 2021 19:31:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178317.324222; Fri, 03 Sep 2021 19:31:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMEu7-0005EO-NY; Fri, 03 Sep 2021 19:31:11 +0000
Received: by outflank-mailman (input) for mailman id 178317;
 Fri, 03 Sep 2021 19:31:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMEu6-0005EI-TY
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 19:31:10 +0000
Received: from mail-wm1-x335.google.com (unknown [2a00:1450:4864:20::335])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0b5a6aae-0869-4a71-8c58-ac91c4e78552;
 Fri, 03 Sep 2021 19:31:09 +0000 (UTC)
Received: by mail-wm1-x335.google.com with SMTP id
 x2-20020a1c7c02000000b002e6f1f69a1eso232852wmc.5
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 12:31:09 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id f17sm286460wmf.4.2021.09.03.12.31.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 12:31:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b5a6aae-0869-4a71-8c58-ac91c4e78552
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=5AI83WewnZnYkPXwZnd2Q/eU9eygrBsbivngRE4BXiQ=;
        b=P4ISVXfvobqG12Z6RoTOI5UGIr/4McKEzBRjkpnJVeCZevklJ/WV/0lQBT+voH2SUY
         t0dVslctqlM5DOtyjeqjkUvPjGWjj1dhiEdyZQeWioK8rQRoWy5A5NcKlSGuzKW3aWqj
         t2bT7b9xG9bD7XISjXwWDKsCHqd+100iir2zh/3XRm8EAkGO06KgFLng+vuhbRGyvSoR
         AObhwaMSw7osyojtzotiYTY7cxvr3JYasdODvCk6KW0vBIVPJ0cpLiVEP0EFP/cmNIcL
         vTd4VlnM3ebyqP3INvEw9GWo/Z410XL/Oo2itgtLLE/TkG/P7c4oBYv9h2IqwiQYnS7X
         J5sg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=5AI83WewnZnYkPXwZnd2Q/eU9eygrBsbivngRE4BXiQ=;
        b=B6AfWkm6iJPPwOFjof/JZTbWIte1i/DyMNkXA3zQ4z5Bs8RLa+dPYVfWVe7qIO9UlQ
         oF70qyOjkOiutCrvn9M4mGZmLZrYM2IZD6gZBr0J6L3Ni+N19G/02cDfpJAP4NAvWsFJ
         1xt/z5wmxYI3kdK1UB4Rv7g44yV3D4QuYXCXxiqm+xX4BdQtX7u1AefH4v1HI8Ku00fl
         bG1hcp/MlrdHt1KTYpViovlsd/pgTSg8NvGAPO4SmpxYtGwb1ChctSQi1IE90UsYnuus
         d3wvy39SR9P0YpupTi341QCF1UZwAV73/ufV+INEQP6qQ1QMp+2uAqcMQs1hGoR2J3xU
         7mgw==
X-Gm-Message-State: AOAM5322nGVaKe073evTpjpQww+giObdZ/qboNQv4yqwOjcKDs2PS9p9
	cdfA8R/H1eiuAUu54pUjPgcbmQ==
X-Google-Smtp-Source: ABdhPJyNPnhhhAtbo2njo6bCAr3WAaEKFNGmuE7LF325krCnfwymK4ah4w6eYAokj7v99c7wD74wnA==
X-Received: by 2002:a1c:1c2:: with SMTP id 185mr289419wmb.11.1630697468395;
        Fri, 03 Sep 2021 12:31:08 -0700 (PDT)
Subject: Re: [PATCH v3 01/30] accel/tcg: Restrict cpu_handle_halt() to sysemu
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-2-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <cdbffa64-98a4-859d-d8f1-50d959197542@linaro.org>
Date: Fri, 3 Sep 2021 21:31:02 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-2-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> +#ifndef CONFIG_USER_ONLY
>   static inline bool cpu_handle_halt(CPUState *cpu)
>   {

Hmm, slightly better to move the ifdef just inside here,

> @@ -607,6 +608,7 @@ static inline bool cpu_handle_halt(CPUState *cpu)
>   

and here,

>       return false;
>   }
> +#endif /* !CONFIG_USER_ONLY */
>   
>   static inline void cpu_handle_debug_exception(CPUState *cpu)
>   {
> @@ -865,9 +867,11 @@ int cpu_exec(CPUState *cpu)
>       /* replay_interrupt may need current_cpu */
>       current_cpu = cpu;
>   
> +#ifndef CONFIG_USER_ONLY
>       if (cpu_handle_halt(cpu)) {
>           return EXCP_HALTED;
>       }
> +#endif

which would eliminate this one.


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:11:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:11:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178325.324233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFX8-0001e0-Vo; Fri, 03 Sep 2021 20:11:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178325.324233; Fri, 03 Sep 2021 20:11:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFX8-0001dt-Sv; Fri, 03 Sep 2021 20:11:30 +0000
Received: by outflank-mailman (input) for mailman id 178325;
 Fri, 03 Sep 2021 20:11:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFX7-0001dk-Mb
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:11:29 +0000
Received: from mail-wr1-x435.google.com (unknown [2a00:1450:4864:20::435])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 437a5473-0e83-4d53-95a7-0d22b3314cfa;
 Fri, 03 Sep 2021 20:11:28 +0000 (UTC)
Received: by mail-wr1-x435.google.com with SMTP id b10so351300wru.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:11:28 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id o26sm331500wmc.17.2021.09.03.13.11.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:11:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 437a5473-0e83-4d53-95a7-0d22b3314cfa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=EaCWOOldFpTJW/62GNcO8hnd0wqc8elIcEfaX69jVvw=;
        b=YyrzMqwR7pTY4UVwmQkF/b6/6vs1u7pbVcrnFbJ2IcW6P4fipRJVT8PmaEsbbFpg5m
         Ymv96X1royzInIGobPF2j09Z1yc16HGEdliJIXhC7BdkL4CkDrSv4ViHU6bzRWnn7dZV
         qZBXE7dHM+4+DbAAXk77dXGymyVlTIpYSq7C1i3JEZlTfiNoWgj2wqws++sPnSgcptat
         mA18BowRdXGHFl+/0fRO7ZQ0GtRH00yJ/XcRK/OYqcLDp/RR8ktqw1Cg+Aoha/XNDyBg
         8cWnsyWPtksyXCOTBNPJZ0SnF8sr/Z1rLMwoCoGs5MDwwc19FX55U9jh0cE3apvw4e/O
         +6oQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=EaCWOOldFpTJW/62GNcO8hnd0wqc8elIcEfaX69jVvw=;
        b=uJPXVZfNjPKYxgmllbDIpsOu6ps0CcCNR6OjRLXaRw30hJ60XI4HtmCYXbPOqOrG1W
         lh0hLBVg4OmTEgLYbuEGVHBn0cnoSXdkd0UCqfjmehnRZTXBz+gKW2riWE+fwOVmybQK
         92M5BFIGWKTtZuSP3aONNYG1v7o3M+03+iglczXGFYxaixsEiHZDogqypLOnFkSgDvP6
         NRWL76IBy2xhBR7cQBzjM1s6tlIi0svuuyD632A0DySuzSpkbVFH2Uqjt0hYGJaV2qPr
         jxKLdGZW+8iMq3N7rQLBiLBhqK71gnzRyniTeVmsYURjC8B7VPNFl+aYptLAuUEYgYVT
         Lm+Q==
X-Gm-Message-State: AOAM532llcgkgH4jDFvLV4vL/OXVJ9Kx0Mj/1rxeh0SDa+TsXt2uGbhR
	TF8SwLE9bgPE/SeK5LJFSGmAmQ==
X-Google-Smtp-Source: ABdhPJwhD7ZLlhNK9IfRlsfSxTpaQWDz0Xn28SyMeYUyzG5cv49AdD3Ojc4LM0ShmrKJ0X1u+d3JqQ==
X-Received: by 2002:a5d:5712:: with SMTP id a18mr758059wrv.367.1630699887867;
        Fri, 03 Sep 2021 13:11:27 -0700 (PDT)
Subject: Re: [PATCH v3 02/30] hw/core: Restrict cpu_has_work() to sysemu
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-3-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <ee2f2d27-cc3d-c398-74a0-c0ca439d84ef@linaro.org>
Date: Fri, 3 Sep 2021 22:11:21 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-3-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> cpu_has_work() is only called from system emulation code.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   include/hw/core/cpu.h | 32 ++++++++++++++++----------------
>   1 file changed, 16 insertions(+), 16 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:12:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:12:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178329.324245 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFXl-0002BN-9h; Fri, 03 Sep 2021 20:12:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178329.324245; Fri, 03 Sep 2021 20:12:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFXl-0002BG-5j; Fri, 03 Sep 2021 20:12:09 +0000
Received: by outflank-mailman (input) for mailman id 178329;
 Fri, 03 Sep 2021 20:12:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFXj-00027S-M5
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:12:07 +0000
Received: from mail-wm1-x32d.google.com (unknown [2a00:1450:4864:20::32d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6cb8a81b-98a6-42d3-9930-9832eff5d001;
 Fri, 03 Sep 2021 20:12:02 +0000 (UTC)
Received: by mail-wm1-x32d.google.com with SMTP id
 u26-20020a05600c441a00b002f66b2d8603so297450wmn.4
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:12:02 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id d24sm273952wmb.35.2021.09.03.13.11.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:12:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6cb8a81b-98a6-42d3-9930-9832eff5d001
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=mxyctmapl79Ub3Q/zYX+Nkj85YoukkH7BHTHMekfamE=;
        b=t9RFyV/8T9K6nv3dHRjG1hWQUIpgeAPiWzbli6rc/aUuQ31x1HtnbcmsiJVOb8T6rR
         m4qiCVH84qQK8xPbNGXOZUpCRPvxDeX7tTBSsZh8M0374UGQg6dF6PLZwyo9ISjfc9bJ
         CWfpxjGwhMr7vTpK1fPtfwbm1iJ8HnTJdrYosgJK9izgHdZ0ztxg8B2uyepCOhJplxAp
         prdXjw8n/KT5kuDuS96LqrCZaNEgUC6kDP7SmktLzm+AQ+9mJ5wvgVVAeU25jfhlO6eI
         BLzbMQN2f0dppbQCTMKOQkiUcLDlvJ+v4vT/tXsEEKXyNLSD/44J84hSUEVubACT1vl5
         uKhQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=mxyctmapl79Ub3Q/zYX+Nkj85YoukkH7BHTHMekfamE=;
        b=E3sT4A+kmXTE6nMqoHUQl5zhxL96RMev8hIGOCggNJ2QSdLo7Url16Mo9p+vsDliTQ
         rmptr9bpz2J42YkLnvEqfGckE7MSMgZsZEz/kR41ReweZoU4Gge6bnC0uK5tk9mqKpqX
         CG51FgXMNRnMedO6kouwBBxbLakqMUGIpBzgp2E88ImAOjlrGojATYjkhdcPJE2kr/rE
         dbKyeam8gNx3RLoy4p6JombUm0Jdh9V2dtFNtnUVLzRy4B00i3ZXRqtx9CzfRZP6JiqA
         T2oxOp6e0gxJxdJYyfls+qYt7QFRhZhzkrZ+mTklF1vDBgw9+54DPjuii5L6/pspAttG
         lhFQ==
X-Gm-Message-State: AOAM53197H9cpRisvnY7JdzdT64DZGPc878/8MCRATPpnOIxgKTZdvnL
	ubJYnWV14YRExO+DqHGjC7vB/w==
X-Google-Smtp-Source: ABdhPJy+MRQgwX4y9J2xsMLOLDvh4kR8c6BOyH1OQWePN8Q6sLyIRNmiAvv2gW9kqjIBssHC4EjXwA==
X-Received: by 2002:a1c:23cb:: with SMTP id j194mr455976wmj.1.1630699921311;
        Fri, 03 Sep 2021 13:12:01 -0700 (PDT)
Subject: Re: [PATCH v3 03/30] hw/core: Un-inline cpu_has_work()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-4-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <a72010ab-56d8-a6ca-14a0-7caaec76eca0@linaro.org>
Date: Fri, 3 Sep 2021 22:11:53 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-4-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> We want to make cpu_has_work() per-accelerator. Only declare its
> prototype and move its definition to softmmu/cpus.c.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   include/hw/core/cpu.h | 8 +-------
>   softmmu/cpus.c        | 8 ++++++++
>   2 files changed, 9 insertions(+), 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:15:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:15:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178338.324256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFaX-0002yl-ST; Fri, 03 Sep 2021 20:15:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178338.324256; Fri, 03 Sep 2021 20:15:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFaX-0002ye-PO; Fri, 03 Sep 2021 20:15:01 +0000
Received: by outflank-mailman (input) for mailman id 178338;
 Fri, 03 Sep 2021 20:15:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFaX-0002yY-6x
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:15:01 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7aad5d17-db47-491c-9c16-9b1737dfe478;
 Fri, 03 Sep 2021 20:15:00 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id q14so322741wrp.3
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:15:00 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id l124sm343991wml.8.2021.09.03.13.14.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:14:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7aad5d17-db47-491c-9c16-9b1737dfe478
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=4eHHYFkoRaMsvUTt4qW8qWzKOCRjBx/Npk4Tb094AvQ=;
        b=cLO22hnYLpEg/1RD0FCiO3YdVJ9j79J1/7mkf4AG2YXIXe2aFGD359ZxjySxf7C/iF
         tzfinMqq37GvKuJlfgORv3JQsLRcmYumsBYo2QPaZMoawXGhFMbSYBGnHCvKL0IbQik1
         45MV1RijIs6tVAHVpo7wuxRVTGnn6P3tAg9pOxmHYSpWbVd9DiYxU9bQGzq/RI9TejCi
         IuFmIe0QtC8F7fLOeMyaxPg7y3lcT3EAD/GnAD/B9pA4KX7M04jhATDZ7A8/fr7gbEdx
         fOoED7iAFaPcN+AM1JnCIYtXX45hgsKvOLuQ5XpA+AqILLfo9hIxUKs545+6dSzo6DRU
         0nsw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=4eHHYFkoRaMsvUTt4qW8qWzKOCRjBx/Npk4Tb094AvQ=;
        b=f7e9szPaS9KnFo2+8yttE9sm9fN47hliljWAiiqqQDU5JitalRq7TbQD/3b0bZg3a6
         uEWXZqMR1o/6MwPSMEtxPWY5c3Q3TLgvU8MNnvmdQ+o4ylHi6j3hwgDoKw4qk8sysEDV
         DN1gYh9QK8872wFtmQ8GLcF7/oQ6xJOFNRIkSa/JFYvg9z11f8BqJ//h0WvaeFhbMJcK
         Vqwfdu4Bty7kQa3iqUbfi4aWhSRHdY1W2GsfyWA7P1ouRV7zvA0eztHIxqovr734oRJj
         9UWQk+t8jb4IKjfnXFfSmCu92JasIHwIyl8ZKnu6u91wDNwWyrHs0bDIzIhKnlZRxC1p
         q0gA==
X-Gm-Message-State: AOAM533hKAvEA5DEl4zaAF+//4pbkU8TqYOvE7BVC4AUg1nrONuQVjNt
	/gUR8IYxgeRXLO1UaMGBUy5vTQ==
X-Google-Smtp-Source: ABdhPJyHZZGXFXfUxy0q9drjOdzklbpmSyqwplr6BLWUGT4wjUUKjoKaSTakGLOyf3y9u9KCe6dQnw==
X-Received: by 2002:a5d:6cc9:: with SMTP id c9mr814056wrc.12.1630700099566;
        Fri, 03 Sep 2021 13:14:59 -0700 (PDT)
Subject: Re: [PATCH v3 04/30] sysemu: Introduce AccelOpsClass::has_work()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-5-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <d97a0dbf-842c-5d90-e4d9-c2faa5d618cc@linaro.org>
Date: Fri, 3 Sep 2021 22:14:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-5-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Introduce an accelerator-specific has_work() handler.
> Eventually call it from cpu_has_work().
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   include/sysemu/accel-ops.h | 5 +++++
>   softmmu/cpus.c             | 3 +++
>   2 files changed, 8 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:15:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:15:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178344.324267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFbO-0003YE-6g; Fri, 03 Sep 2021 20:15:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178344.324267; Fri, 03 Sep 2021 20:15:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFbO-0003Y7-2X; Fri, 03 Sep 2021 20:15:54 +0000
Received: by outflank-mailman (input) for mailman id 178344;
 Fri, 03 Sep 2021 20:15:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFbN-0003Xg-2u
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:15:53 +0000
Received: from mail-wm1-x329.google.com (unknown [2a00:1450:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d5b75ea2-91a8-40bc-9e12-af0037baca71;
 Fri, 03 Sep 2021 20:15:52 +0000 (UTC)
Received: by mail-wm1-x329.google.com with SMTP id
 o39-20020a05600c512700b002e74638b567so300057wms.2
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:15:52 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id m4sm353497wml.28.2021.09.03.13.15.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:15:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5b75ea2-91a8-40bc-9e12-af0037baca71
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=OIdER+ntU5pnE+dAq/LvT2j6xOcRVpXDcpqShg7WpU0=;
        b=uxvnHzNv132fzgkc218/R0lqQKuK+t8JtxKc9v7MGrkKPRxDYU93G8sAz5OB0Qa73S
         HE9cI2H2Z0CH+z6tU7HILrt73xJLER1ScmOcHWVCSynT7na5DZ8aQUWZ73RzZS1vzDMQ
         vqeZlVUhDraI2JH+QdE7CnrbM86ihr8OahFggiYzaiiPqr9S9Cooveeavxo3806rUBCc
         wAmH4n70qiWJfcK+7NmYB575D8ru3ANLV2HDRDh0a+ujAxTAi2JXNetHl39JI8em8dQt
         /MjpEL+5e1BeWhbg4mKgjTybiTcJ5pJVRqEATLrqAPvRDkDzcFl9morqxai4fdjaAREV
         wsLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=OIdER+ntU5pnE+dAq/LvT2j6xOcRVpXDcpqShg7WpU0=;
        b=lnmxU22xwuqKs2dy4xF4UdH0u0kpnfuFVONRsTIR//o8WO//Ftp6rGZrRFM4sbArrp
         5/I+sdqAFuk+6yLo8tRRozdbOeyz9sekeBeQ8sLnvT127SHfI8ypfugTnNeJENFxTISx
         L17R8/CGylV0SJ383LKfKpVbTNJPpj4BCsyI/7y4VYCKo7HfuLU3BiOhC8gdPGvfZWqz
         U6cw1EmKpxVNzSB3j6P62utk+63cAwaqltEaHn34wA2m5W95WdkfkII6KOCKVsw11I3w
         3CK7rpNGwHJ+b2ZjNwDzOo+sTb/YIfMX92cmFi2RdrvPkMHiZffEvEo8wxZlw3dwRoPA
         Jk0Q==
X-Gm-Message-State: AOAM5337r3g4K1VYUD8/iY8sv2GeayEbTH6Uh2cmHaeYDIhWXek9iQpk
	vAUt1xoeqBalQ8AvPVI+wkqfPQ==
X-Google-Smtp-Source: ABdhPJyMWwBTg93pyBo5sv0A6QW1ur9FJeS4HTsMD79+WiDYC0sQ7drSYJ0pDX3u+yQQ7riYEaCIGg==
X-Received: by 2002:a7b:cc16:: with SMTP id f22mr461323wmh.99.1630700151552;
        Fri, 03 Sep 2021 13:15:51 -0700 (PDT)
Subject: Re: [PATCH v3 05/30] accel/kvm: Implement AccelOpsClass::has_work()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-6-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <27a1a7f4-10f3-e089-6f37-4eed9542a6c6@linaro.org>
Date: Fri, 3 Sep 2021 22:15:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-6-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Implement KVM has_work() handler in AccelOpsClass and
> remove it from cpu_thread_is_idle() since cpu_has_work()
> is already called.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   accel/kvm/kvm-accel-ops.c | 6 ++++++
>   softmmu/cpus.c            | 2 +-
>   2 files changed, 7 insertions(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:16:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:16:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178350.324278 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFcC-00048t-GE; Fri, 03 Sep 2021 20:16:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178350.324278; Fri, 03 Sep 2021 20:16:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFcC-00048m-CG; Fri, 03 Sep 2021 20:16:44 +0000
Received: by outflank-mailman (input) for mailman id 178350;
 Fri, 03 Sep 2021 20:16:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFcA-00048c-Pj
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:16:42 +0000
Received: from mail-wr1-x433.google.com (unknown [2a00:1450:4864:20::433])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 51c84082-b43f-4e93-b677-905a8c27a938;
 Fri, 03 Sep 2021 20:16:41 +0000 (UTC)
Received: by mail-wr1-x433.google.com with SMTP id q11so303104wrr.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:16:41 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id b18sm201319wrr.89.2021.09.03.13.16.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:16:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51c84082-b43f-4e93-b677-905a8c27a938
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=ksvR3vXBtwALfl0I6bUYOq6zF1YdgeJMV8dFDRL1P+g=;
        b=F+LmEPrSCssgf1eqjM+WBloAvOVq2tL7NMyM6LZCHirhU39DXiDO2uLD9ug3iNaHCK
         qAQxo2bwgbel/l5ZzJa4uPRl/EqjN0sQA7XIe3XklYehe60QyUJs5qg1vnpnfs/LYzJ8
         F50mpECSrD9JTS8cWWsDSVPn2dEFp7h2GfA9zAOn6e5EfN/aiCylKqANnhUX+0GyzxMA
         5YbCx/uH9oIK3BVZImpikPWK6g5wC7VESYux9bpW/knmD2hNngQDnsSLtz91Hsh54IH+
         4JT7IufJEjtYH/tbJ32xqBZvfx5N5H001O4TnBoFg7L9NuRj3KiL2d3ZfaMPEXO0z6EN
         fEBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=ksvR3vXBtwALfl0I6bUYOq6zF1YdgeJMV8dFDRL1P+g=;
        b=mEWZqKV/mrsQGoQguwhD752Us+6NTsDddwM82XLeRiFXvXuuFVazp/j+sZSej/NZrR
         i9+VIMJv04YQy6c9HcDZgdtVKR7v6jBzLEmr1AxhphuJYCKd8QFm7S45yCZX0y3X/9NR
         qMS1QBqkyEhlqCvGjHvyD2ro1xCCGhPz+jTArVHSkC/UZzXARrfI+josk+RsPehnL0YM
         dYSgCecPvqRljT1hpCProqgCn0cnS4+Ax8o4R87bITnlEVqXYOlidrmGKkmkdk7fGgNC
         DBW2hDW92sS3jF1g9Kft/jFhmwXKEgGvsjwNbPRTt7GA5eZq913vKmUpAIWAjGmKSycp
         kFow==
X-Gm-Message-State: AOAM530zwJUITx+Fd6Rupzp4r4RyIQNuNP8GhWnRUIl1hIKOM5voP21/
	Dkohf8CsxiwrvVxgTXg9fENyjw==
X-Google-Smtp-Source: ABdhPJzV+oCZLVl1R3jywIXOYAACRgU/ouMC2nXN7qXT1ry456hLKIT4YtHfUNsVqaTI8qbV2+dfAA==
X-Received: by 2002:adf:eb4a:: with SMTP id u10mr794817wrn.11.1630700201045;
        Fri, 03 Sep 2021 13:16:41 -0700 (PDT)
Subject: Re: [PATCH v3 06/30] accel/whpx: Implement AccelOpsClass::has_work()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-7-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <c7d58bfa-20f7-502b-f307-9ed8f9f4def6@linaro.org>
Date: Fri, 3 Sep 2021 22:16:34 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-7-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Implement WHPX has_work() handler in AccelOpsClass and
> remove it from cpu_thread_is_idle() since cpu_has_work()
> is already called.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   softmmu/cpus.c                    | 4 +---
>   target/i386/whpx/whpx-accel-ops.c | 6 ++++++
>   2 files changed, 7 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:17:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:17:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178356.324289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFcs-0004kx-PX; Fri, 03 Sep 2021 20:17:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178356.324289; Fri, 03 Sep 2021 20:17:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFcs-0004kq-Lw; Fri, 03 Sep 2021 20:17:26 +0000
Received: by outflank-mailman (input) for mailman id 178356;
 Fri, 03 Sep 2021 20:17:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFcr-0004kg-C8
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:17:25 +0000
Received: from mail-wr1-x436.google.com (unknown [2a00:1450:4864:20::436])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 84ab1444-2426-4487-9fd3-de92cb7480df;
 Fri, 03 Sep 2021 20:17:24 +0000 (UTC)
Received: by mail-wr1-x436.google.com with SMTP id u16so318827wrn.5
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:17:24 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id n4sm212131wri.78.2021.09.03.13.17.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:17:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84ab1444-2426-4487-9fd3-de92cb7480df
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=Q8RRtCMhTGBNat58ZoO2h/0Fc8GkNe0Oprx1Qcmt+DQ=;
        b=dwmv1KqM52aR7rH3nFosWXsm4FD08tdhGNndv5xGMzStvCDMhzyZVE8JBIwPUqwq2F
         bCLRGHMXtpcQxrFHLNsfcJ75PuxEZqI9fgeJ+R64/zxYVNYGjwzAkdzQcnV1G0aiGJeW
         LTJZC1LjtZnsolxgQDMU0UUFNoUHXBPLt2RyL3DgdCqDzCR5fV8RIduiPWvrI4huqI2x
         14xoPIKRj+tYSoQLnz+qpU2iP9/qF7o+j2jiFMjWkvX25zBXsZzxhgUCz6Znrep8fiq8
         IBBZaGnjNh2PebWfRMYGVTv6N4GT+lZ2aNLKT+f0rlGZl04V61ntDZBBZCMp6HS8v9vZ
         WEdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=Q8RRtCMhTGBNat58ZoO2h/0Fc8GkNe0Oprx1Qcmt+DQ=;
        b=VMtFp0TDoo0ZYWC8LeXgt9poz3w6LEJFVPi5I6UmFJocA7M9TaZkOZe5FFEDBgFVbD
         gemmPe+/R/gcckT9RkYz46kzeNZj2YF2CxCAYNkzRbM/WGMwhlDozTysYxCfrHqrFmcj
         7LtjM5EsFSmR371xuRZUssqCeF9om3Qlr6dajoPU9aTAzenytxtvY8za3RVnkXdElUBh
         pzBYY8OCCoP1/xzAbD33hTE6G8AfXIT95hyfOECVfA182S7skSS0IAGQBqfBMPv+VYz1
         62TEdW+ikFtS39NVg3TMTbRUFcnKvqdX0Wd9BiId9gkxZdViuFjQyQGt7VtBzXIK86yW
         elIw==
X-Gm-Message-State: AOAM530PP01zreNvSRH6jVgJCxPaCh1Nz2gCgJL98/KKWpyx2zJHg1sV
	yScAT/9RJrivx9QPx6aWV9ILTg==
X-Google-Smtp-Source: ABdhPJymtYq5kcbCfG6OUgkQoqRmbaJKl89J4TFcqlkWTH+OL3rMdlakO2CQQpDxn27LYKv0Y23GMQ==
X-Received: by 2002:a5d:53c1:: with SMTP id a1mr784175wrw.209.1630700243751;
        Fri, 03 Sep 2021 13:17:23 -0700 (PDT)
Subject: Re: [PATCH v3 07/30] accel/tcg: Implement AccelOpsClass::has_work()
 as stub
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-8-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <618da04b-c9e4-bd22-d527-412c3fd31386@linaro.org>
Date: Fri, 3 Sep 2021 22:17:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-8-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Add TCG target-specific has_work() handler in TCGCPUOps,
> and add tcg_cpu_has_work() as AccelOpsClass has_work()
> implementation.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   include/hw/core/tcg-cpu-ops.h |  4 ++++
>   accel/tcg/tcg-accel-ops.c     | 12 ++++++++++++
>   2 files changed, 16 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:18:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:18:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178362.324300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFdy-0005Oi-5S; Fri, 03 Sep 2021 20:18:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178362.324300; Fri, 03 Sep 2021 20:18:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFdy-0005Ob-1T; Fri, 03 Sep 2021 20:18:34 +0000
Received: by outflank-mailman (input) for mailman id 178362;
 Fri, 03 Sep 2021 20:18:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMFdw-0005OL-F3; Fri, 03 Sep 2021 20:18:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMFdw-0004Iu-82; Fri, 03 Sep 2021 20:18:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMFdv-0001Br-UA; Fri, 03 Sep 2021 20:18:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMFdv-0003P3-Te; Fri, 03 Sep 2021 20:18:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=kv18ucImg/AIqQOVrt8p+pQ3jTd/7X6tCx213toIVgU=; b=QooggekwkFbtnz99tY5JP0CZWB
	KcaljnM8uEPPS0b3l7AW4h553Tne1uSAsS3SNrqlT4KVJYtzTSK8nSaPY6rsafPmV3Hf6G6PqUl3s
	6iGLmEix9/zBs0Y8SO3n1mkKynUYTmZEeSHvsdMITcVlRhVXefCD5QoewnRjPdZFYal4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164795-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164795: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 20:18:31 +0000

flight 164795 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164795/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    3 days
Failing since        164686  2021-09-01 03:03:43 Z    2 days   23 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    0 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:18:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:18:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178365.324313 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFeE-0005rB-Jl; Fri, 03 Sep 2021 20:18:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178365.324313; Fri, 03 Sep 2021 20:18:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFeE-0005r4-Ga; Fri, 03 Sep 2021 20:18:50 +0000
Received: by outflank-mailman (input) for mailman id 178365;
 Fri, 03 Sep 2021 20:18:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFeC-0005qI-T9
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:18:48 +0000
Received: from mail-wm1-x334.google.com (unknown [2a00:1450:4864:20::334])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cb6e1bf4-0d8b-4b88-9e55-6134c1e51da7;
 Fri, 03 Sep 2021 20:18:48 +0000 (UTC)
Received: by mail-wm1-x334.google.com with SMTP id
 192-20020a1c04c9000000b002f7a4ab0a49so274899wme.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:18:48 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id e3sm292523wrv.18.2021.09.03.13.18.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:18:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb6e1bf4-0d8b-4b88-9e55-6134c1e51da7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=LqQ2qGQh8nV1ZvT9x+U3BKt+BXWg+vkVFqLIEGHhpMw=;
        b=V8kfIBCEJT52V1fp3BG4eEUjASX4Rx05CBYVPUwSnXcrpvu9DyPd9YLrQ1EFixujIi
         lUO2S1zkZF5PG/5VBU6Rk3aTdfu/NohdFpDaoPBTWr2+EAlAEqDrzmgPhi1sQ9/EZI5O
         sp0T10IQyhYcG2l5hY7SahzlV5/YDV007peO7QbcQiw6uki0FNbyDs+4u092DrKL854h
         BofPE+Yc042W1WfjRNZgBmoOLdNGI5J41nf004XrJoendGTO+R9LadQCXGvxwYAslhrk
         awNCuxHsEq//V4tBYb37j34Ybdls/wwtEv1je9PRsUGuN+bp2kKIvMshhTUfobepCiRM
         Cayg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=LqQ2qGQh8nV1ZvT9x+U3BKt+BXWg+vkVFqLIEGHhpMw=;
        b=fFS616hdkAZ4xc9P2JiDpTWswMFgmI9g7KUWwZ1NOSshdy+B49z5e7Rc+9lTIFtTSb
         pdXewz0X/CjPZuH1nThaMyDOVXrDKMtaKLNLotn3UlHFmV6X+cVcE4KXkhU9tIplrwwL
         7Tn+b752FQDkC7rvn+5tpfmVLj8cXVO37Oz1+axdh0Kwij57BsTcLfsO6xnE+7SHFVHr
         FVfDSD1KmZTQWC6aCiJ2PbxHudt6UeN2ywkvZlx6g+vDoBGEvh+y/TfOL5XEe734ziml
         OiacWeYjn8LhKQ/Qgk7Yga9QsPfSlkGYNT6DuUhmIzxXnZ9JqL6QcQ8AP9iaEdtlsaOS
         4L/A==
X-Gm-Message-State: AOAM530brx33j1oqyxOludQIFMv/ZSH6rhZd4Rkr/2mZ+0u9iQYJ4hcl
	Ri3LODKTBASYCDleawJ5HTLiyA==
X-Google-Smtp-Source: ABdhPJw3r9Ix8xv+Nu5+AHr+Q3zSVCPcKA2sZb2hwjS2kDrQMzRlRnU83nDo49PObR1jR+RXBXRDHg==
X-Received: by 2002:a1c:7c12:: with SMTP id x18mr498453wmc.114.1630700327288;
        Fri, 03 Sep 2021 13:18:47 -0700 (PDT)
Subject: Re: [PATCH v3 08/30] target/alpha: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-9-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <3cd48aba-a1a1-cde3-3175-e9c462fcb220@linaro.org>
Date: Fri, 3 Sep 2021 22:18:39 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-9-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/alpha/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
> index 93e16a2ffb4..32cf5a2ea9f 100644
> --- a/target/alpha/cpu.c
> +++ b/target/alpha/cpu.c
> @@ -33,6 +33,7 @@ static void alpha_cpu_set_pc(CPUState *cs, vaddr value)
>       cpu->env.pc = value;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool alpha_cpu_has_work(CPUState *cs)

Drop CONFIG_TCG for alpha; it's always true.

Otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:19:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178376.324324 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFem-0006bH-SZ; Fri, 03 Sep 2021 20:19:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178376.324324; Fri, 03 Sep 2021 20:19:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFem-0006b8-PS; Fri, 03 Sep 2021 20:19:24 +0000
Received: by outflank-mailman (input) for mailman id 178376;
 Fri, 03 Sep 2021 20:19:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFel-0006ah-4m
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:19:23 +0000
Received: from mail-wr1-x433.google.com (unknown [2a00:1450:4864:20::433])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a6511396-d53c-4344-92d3-8d3fd82629ab;
 Fri, 03 Sep 2021 20:19:22 +0000 (UTC)
Received: by mail-wr1-x433.google.com with SMTP id t15so316203wrg.7
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:19:22 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id i4sm360865wmd.5.2021.09.03.13.19.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:19:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6511396-d53c-4344-92d3-8d3fd82629ab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=hrfT6D6r8NgV0kbWbMFB7eJ0H5Q19iawpNSQkwrO3Zw=;
        b=UP4L3x90K9R0Xv9kUPD2k1ldA8cppgKzxB41aZwVXYD7x/qndJVah80MG87/CBUxKF
         TQsp6BHab8hbEiq0EVctWDjgUQRASZHj3+eAB2JftZU4rDgjFNuJfnOUODTXjmvHq+Wb
         CO5SRo4MZKWNpVLFkz1S9LBbag1wL73cRaGdVxll7VdzLyexwvlp0cZN9dy4/9ct+32O
         xuGIsIHPC9o6Pv+TjnwbzRCalBEKUw0bgwH95SVmBcNFszNxj5KOrbihoohVlg16IcU/
         cuE0LBDTSZg+bzpqf7VDLPEdO9axqPBJ8doVS6whQR+pz75s6Bv4Yx4dkv7Omkpgzy1I
         2uGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=hrfT6D6r8NgV0kbWbMFB7eJ0H5Q19iawpNSQkwrO3Zw=;
        b=TD+tcvo9inebleM+w/QhYd0j/Qx73orif6Y00HgD+awkan/RNyI5c7a9yNqCMZUT9w
         Y1sp4eQuCP6F7nmZo5VXLZccFRUgudITWdEW8XF07gvHbJ1uDCVs9Dt84s5bFgg6FA6c
         YVSzoEgbuureyy3qarRu6w1DxanNir2aJ+OtbRw6LQL8sO0Io06W9HGwx7S+7Dpl6KbT
         heMTfBOuV1fEcyPILM3n2M8z9NJ3v723c4D02OhEFyq2AFU1hreRy2xh/eVvdF+8EPMG
         arfGIzPFj1oOmdh8TPj6k8s+/59AmVv85LilfOoIUMDea9UAeImRk6b411To9z11COHC
         6UMA==
X-Gm-Message-State: AOAM531lIpJFfgQA3p4nfXE0p9TEUXB9Kx2Q2gGWPl2tAurvxmeUG71d
	5H5mAZD1unlcOT0FZkES5d0p0A==
X-Google-Smtp-Source: ABdhPJxOkyr3cz4gxEh2Ke6ki62uWZywliZS86PJZb/UyH5xriM04K4qsIBYKGhxLIpR8DxkeTkVHg==
X-Received: by 2002:a5d:5262:: with SMTP id l2mr826597wrc.190.1630700361562;
        Fri, 03 Sep 2021 13:19:21 -0700 (PDT)
Subject: Re: [PATCH v3 09/30] target/arm: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-10-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <318478f2-6cea-e9ee-dc9d-442589aa8f56@linaro.org>
Date: Fri, 3 Sep 2021 22:19:13 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-10-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/arm/cpu.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:20:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:20:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178382.324335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFfb-0007xU-7C; Fri, 03 Sep 2021 20:20:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178382.324335; Fri, 03 Sep 2021 20:20:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFfb-0007xN-4D; Fri, 03 Sep 2021 20:20:15 +0000
Received: by outflank-mailman (input) for mailman id 178382;
 Fri, 03 Sep 2021 20:20:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFfZ-0007xD-QQ
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:20:13 +0000
Received: from mail-wr1-x435.google.com (unknown [2a00:1450:4864:20::435])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42206d79-e52a-438d-9045-ae305ad8bfb6;
 Fri, 03 Sep 2021 20:20:13 +0000 (UTC)
Received: by mail-wr1-x435.google.com with SMTP id z4so322463wrr.6
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:20:13 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id o8sm266012wmp.42.2021.09.03.13.20.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:20:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42206d79-e52a-438d-9045-ae305ad8bfb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=xsif4pme6nhSpsi7vEWmXFg/Hnrb7mhQtjIRT3kxFkM=;
        b=muTFofn/yHs0EYKIsGha4ps6z/nsFLgk7o50hQ1bBL3mlCLJN3dqh2EbQA90EnwZiK
         NECjyG+jXGItzTA4QlZoqJEuj4/2GopLUtjAu+ncqdo/FFyEUWUPbBu236/GFMudcuCK
         D9DyjDbkUkta9IAKG6+3m2r8+8jv4lYsuurqFUi0rWz/gxe73U7hLmg/mrm5hqKdTbiF
         Opmfkr9N/UDCqcJTgk0JtowcRFkQ/LeFaRp+RBrZIUFfJTbIekWq6l07SbB0VGTU8eH+
         /SKP943AfE5+gEB6opgLoUtMGZRq5vBaEZuA49fASyXmNkP+xk+3PmUF6tFbonUn2CoS
         6yWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=xsif4pme6nhSpsi7vEWmXFg/Hnrb7mhQtjIRT3kxFkM=;
        b=WM1qaP9Nuobmr0ThmP9LOEw7Q6EJK5TgxVAvO6vqYNxWz26yTxi8DfEQBhq8vxoKFO
         hGG7Y0Os/5zuSpcXwIcSlM5gIgCQL8GIMCGDeYr+Ki+OHq2Hnmz6KDe+Wn2ChIhwIpDD
         EVlnCQe4l54bOZaxg6v84J4R7mIi8P4niFKAiIh/r9GH/HIG97chxOP4IbJj3FMq7WUN
         qCSHF6oGhJxT0pOZM6oA9VZTsEF1sTR7zD50T2XUqTjc2VOtDotxWImPs7ubQV7s2V+V
         zthsXK/D9F0VrWcXgu/WJUeEnSe6UAwrrQioFyhO/BcNt43wQBrvfFhvm8C/8DQHu2Is
         9nEw==
X-Gm-Message-State: AOAM533xozJkdYOtc7u6dtOZQf1WWA1jVqRrQPmf129hf2vfD9SSZMa2
	hqeIovRl0zbygR2uItsAJ+6hKA==
X-Google-Smtp-Source: ABdhPJxCxEUNKk9CrXrBQLK7GjqB1EVrT8+iyYBGoXf53+iVgrOG7SDx81iHjmDeuA92Igp3EWy9+w==
X-Received: by 2002:adf:dd4f:: with SMTP id u15mr825520wrm.237.1630700412182;
        Fri, 03 Sep 2021 13:20:12 -0700 (PDT)
Subject: Re: [PATCH v3 10/30] target/avr: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-11-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <1833210b-e250-33f5-be38-9d543539b4aa@linaro.org>
Date: Fri, 3 Sep 2021 22:20:03 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-11-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/avr/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/avr/cpu.c b/target/avr/cpu.c
> index e9fa54c9777..6267cc6d530 100644
> --- a/target/avr/cpu.c
> +++ b/target/avr/cpu.c
> @@ -32,6 +32,7 @@ static void avr_cpu_set_pc(CPUState *cs, vaddr value)
>       cpu->env.pc_w = value / 2; /* internally PC points to words */
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool avr_cpu_has_work(CPUState *cs)

No CONFIG_TCG or CONFIG_USER_ONLY test for avr.

Otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:21:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:21:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178388.324347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFgl-0000On-HH; Fri, 03 Sep 2021 20:21:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178388.324347; Fri, 03 Sep 2021 20:21:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFgl-0000Oe-EN; Fri, 03 Sep 2021 20:21:27 +0000
Received: by outflank-mailman (input) for mailman id 178388;
 Fri, 03 Sep 2021 20:21:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFgj-0000OJ-Ff
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:21:25 +0000
Received: from mail-wr1-x435.google.com (unknown [2a00:1450:4864:20::435])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cbb01131-55d6-40e8-aad9-439950fc1951;
 Fri, 03 Sep 2021 20:21:24 +0000 (UTC)
Received: by mail-wr1-x435.google.com with SMTP id t15so322008wrg.7
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:21:24 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id s15sm232969wrb.22.2021.09.03.13.21.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:21:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbb01131-55d6-40e8-aad9-439950fc1951
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=q+SSfq4073OmWN5SjfGpr2ImvdVTltAm2Ap93b1Y88E=;
        b=PZ2KiViiXR0yMgtsi9ubJd49eXX8m7kgtJ7JWAf0xocPcQ4Oxw4avSPsHVo9QZcW0N
         BByueunHxBruDfx2gZo+Ix58peWhu5wGIHIMCSjfvS8XdfqDnHodPv7hZLnIQBhld9/a
         y0lWLp8aIQKnuXGpPmhHwQpf/DGQHqb2SPZDygfQELNsjREPaTcvh71MJU4d2M9P2WuC
         AZHX/t3lV1bfa8h49PxfG8kq4e5hCt+9BPtVsMQjLzebnh3VaBZTuTGJBK+rNkK7BCfW
         EvnOQKajlmpMw2NfupoVnEo8oI6UDgZtiqq4mNRFKiSDClkKz+9ZYBjtDFyqInTaIFO/
         szhw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=q+SSfq4073OmWN5SjfGpr2ImvdVTltAm2Ap93b1Y88E=;
        b=nMikvhKWxS2K7zDj/loG+pNS1Px6FswKXZQf2k5wujLZwXJp8H4etffh4dhaDKjvMz
         kHGVfG2okp+j3nboJFp5JTVq+i5mkYINgwSklOOjf+4T572IeJa42E17km1arizjajtZ
         EvZB9E51EWm+e+Se2wB+Yy6yHYXMGgeNEtwIqZD+4jwyG3iWXgq/lMxmW+m4yR33re3h
         PXihYd4RYlwMAeXqLZ/GXQ1wejTWJVr4TjypUbEzpfS+6JRTHVDaWMMbYfjjT27r9pEu
         PACEj2xW7lqsDCZyah4SRdaQ6mg+xEuGk7Dr6Z7ECZe9JUINcxa0YQM4gpK4jz7yh6C0
         Lb+w==
X-Gm-Message-State: AOAM531UdRIJjPYOd/FZnqD3/tusaBiy6m+sbIhhAmtF9JMc4jz0LsKt
	BDkNjnYRlNX03vqPeo8mIimRsw==
X-Google-Smtp-Source: ABdhPJyd1De7PuFzuLP4i8MhbWXrBu0OhprhhfEXRkHThwdYNyCL8/AmxbAlET5sIL14xwjI9QRBeg==
X-Received: by 2002:adf:c149:: with SMTP id w9mr804955wre.127.1630700483905;
        Fri, 03 Sep 2021 13:21:23 -0700 (PDT)
Subject: Re: [PATCH v3 11/30] target/cris: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-12-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <51e42a3f-6064-be49-26b2-20eafd1106a2@linaro.org>
Date: Fri, 3 Sep 2021 22:21:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-12-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool cris_cpu_has_work(CPUState *cs)

No CONFIG_TCG for cris.  Otherwise,

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:21:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:21:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178392.324358 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFh8-0000xP-Pw; Fri, 03 Sep 2021 20:21:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178392.324358; Fri, 03 Sep 2021 20:21:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFh8-0000xI-Mx; Fri, 03 Sep 2021 20:21:50 +0000
Received: by outflank-mailman (input) for mailman id 178392;
 Fri, 03 Sep 2021 20:21:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFh7-0000x6-TY
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:21:49 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 76992963-d02d-4d59-9f16-1d9497d94eaf;
 Fri, 03 Sep 2021 20:21:49 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id b10so382251wru.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:21:48 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id r15sm402884wmh.27.2021.09.03.13.21.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:21:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76992963-d02d-4d59-9f16-1d9497d94eaf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=mnk5zSmpT/p699xgSKxNbIV/1GCHbsuA6wEF56QNf+E=;
        b=GhgL9+LGgM+gjslEr/EdoI7/xsxfmmw67TehO/6U7pPlE/OPtA9LjE4SDDmXvYLAmN
         yZeX0gl3cDixUofaJ53d7PYirmx3VephL7Fp2AHgbYGFY+2vJtFBsCzJiTNZI+nj8zOU
         lX3C3n2CsiqPN2JfeGez5wPcTov60f+AYR19eDiQGHsrHnbpM5alfslIIIHhsT0nTxma
         DXlSCsL0edylOOeUn7jofbIz89KxP2tF8ZT5uMLxWfF0tNPmLeVePpEpIaz8SindXxiw
         pPRklR9Iklk2ipRx/ia9X3h7Q3ffP7aRj506nP6FYfGI6tshEguc/NBqS4OKI2Ci9Ann
         FGJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=mnk5zSmpT/p699xgSKxNbIV/1GCHbsuA6wEF56QNf+E=;
        b=ZJ0EbI4yCqYK1W/l2g8S9Hv3xZJiYQGN+YYsJY60GH2xtOIpaKCw14sQCqORs8a81B
         5oj0S15wInCjtsHhN6o+89dylxZxuIfoiQROUg4xDzOmr8xLv82nQYH1uM8cCDU0QMD1
         kUznd8E1wu56lNhFFzlCszhPe1mqrG7YY3KC0V03SDH971aOHXx5SnpCek0ilHuDekdv
         hkB/d+man8bZUztrCwLUNUQccX6ZjlC5s1IXKr/n/iAooqkU6ThA/jsHJ/D9IBq2h2Fr
         ABcdrc8LYeyUMSt15CkT7cfcgvYSONdhI6iecxWAQRbAGkP0TNgp9c+J3eSl7RtruKBq
         CRuw==
X-Gm-Message-State: AOAM532PKoKaqMegNpv10UWyrkslaq61Q8pAac/7ak/eQcjW5uLf7s9w
	CWf9IGgy+trperHT0wztpu+Xbw==
X-Google-Smtp-Source: ABdhPJwPe1mnKhrxerAyhvhbZ2seI44kln/Bm0nN+tszrWb4ThZb+maUMAoke8HDY1589uhlj92r0g==
X-Received: by 2002:adf:8b03:: with SMTP id n3mr768062wra.439.1630700508096;
        Fri, 03 Sep 2021 13:21:48 -0700 (PDT)
Subject: Re: [PATCH v3 12/30] target/hexagon: Remove unused has_work() handler
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-13-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <5c91731f-a11c-3555-93d8-cd3379fb727e@linaro.org>
Date: Fri, 3 Sep 2021 22:21:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-13-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> has_work() is sysemu specific, and Hexagon target only provides
> a linux-user implementation. Remove the unused hexagon_cpu_has_work().
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/hexagon/cpu.c | 6 ------
>   1 file changed, 6 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:22:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:22:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178401.324369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFhu-0001gG-7R; Fri, 03 Sep 2021 20:22:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178401.324369; Fri, 03 Sep 2021 20:22:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFhu-0001g7-4N; Fri, 03 Sep 2021 20:22:38 +0000
Received: by outflank-mailman (input) for mailman id 178401;
 Fri, 03 Sep 2021 20:22:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFhs-0001fz-Uy
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:22:37 +0000
Received: from mail-wm1-x32e.google.com (unknown [2a00:1450:4864:20::32e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7adb07b7-15d0-4505-a490-b04c17cc036e;
 Fri, 03 Sep 2021 20:22:36 +0000 (UTC)
Received: by mail-wm1-x32e.google.com with SMTP id
 u15-20020a05600c19cf00b002f6445b8f55so334024wmq.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:22:36 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id p13sm244311wro.8.2021.09.03.13.22.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:22:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7adb07b7-15d0-4505-a490-b04c17cc036e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=UnbOiyADd1ZMlxRvNb9HZP0pt1hL83FL8WVZjS8Ogcw=;
        b=COG3NPL3Vxpz88wlWI3lGdCLg/LDdsQZJexfhJnrghpwFEqcukZveIdAk/fFiP0wKB
         7HNAWQIl6rmgV04BpIshszyQzk9qq0Q2NwrgljJCneyfFgOJ0xAXKUZsaU/hrF0B0/AT
         3w+VRd6HCQDmcDCjwEVjMOfRX4d+x+Lya4wftnpVkB2KXN3pr43vNkh+vseHQcZijosS
         Q3FflSz6b5FTlNEGFg6AgPTW3cZ3FSliEWoFoIHYG+zwm5ACts2RQXEeZ5OBwv2FsHiM
         zzHD5y43SWWwvU4sjxJD/8SKKcD/qNm8NbTssviVjN3cg7xbLdiiJSNeDTXPZ9eqrqkH
         KpWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=UnbOiyADd1ZMlxRvNb9HZP0pt1hL83FL8WVZjS8Ogcw=;
        b=sazlLata/lmlAqyxE3LUM1sbHpFP4c2M5QWmnvp2LeDRKFdXoijg/Pn9ksGFWLcjMu
         lbkiwUujh1xCoSqUCPeQ+3AwNjzMNliB1Q7ovwGDyZBeSerq+EQd1kVQ7qIa58+PhkPn
         2KtYWWiS3as1tw741pMCTbQxBsPubTDKlbF+LQmVeXaacMzYrvH9UDIQuhit8Bdzlj0c
         spVeyo+9SNhuSRWyNCtb28RWrsk91sI8cGQxZ08NikvH4Fw+5JiRWQ5+7C6Qd9cRr3gE
         fX5XIAGHqwm9sytZxoVUsCT24bCLheNMioj2Ng4bC0EOEuqEasFm/lohvG1k1S4cVaE+
         CQbg==
X-Gm-Message-State: AOAM533d53cJ9UyoA9ZAfBXLftC7FgXHNJjFnXj32xZt4qyHqEwInrsn
	arckgeZ4eBvQEVCwQl8gQZujng==
X-Google-Smtp-Source: ABdhPJxPwO0cHTUIrMGmZMtVOl8TUayc1aTCNm++vMgNkxvRpyKHD/crU8kG261DTumMzBNsNkV5GA==
X-Received: by 2002:a05:600c:19ca:: with SMTP id u10mr432498wmq.178.1630700555348;
        Fri, 03 Sep 2021 13:22:35 -0700 (PDT)
Subject: Re: [PATCH v3 13/30] target/hppa: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-14-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <1f644d61-92e9-2161-284b-0643edd4892b@linaro.org>
Date: Fri, 3 Sep 2021 22:22:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-14-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/hppa/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
> index e8edd189bfc..cf1f656218f 100644
> --- a/target/hppa/cpu.c
> +++ b/target/hppa/cpu.c
> @@ -60,10 +60,12 @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
>       cpu->env.psw_n = (tb->flags & PSW_N) != 0;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool hppa_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:23:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178407.324379 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFj8-0002LK-Im; Fri, 03 Sep 2021 20:23:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178407.324379; Fri, 03 Sep 2021 20:23:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFj8-0002LD-FL; Fri, 03 Sep 2021 20:23:54 +0000
Received: by outflank-mailman (input) for mailman id 178407;
 Fri, 03 Sep 2021 20:23:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFj7-0002Kk-BJ
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:23:53 +0000
Received: from mail-wr1-x435.google.com (unknown [2a00:1450:4864:20::435])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 76ebf9ff-0cf2-492a-891f-020586c0e0e5;
 Fri, 03 Sep 2021 20:23:52 +0000 (UTC)
Received: by mail-wr1-x435.google.com with SMTP id b10so387887wru.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:23:52 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id j20sm258635wrb.5.2021.09.03.13.23.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:23:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76ebf9ff-0cf2-492a-891f-020586c0e0e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=wXBwe+C96NVMAw/WVJPmEePxSXgR8UnAw4mAKidy+Uc=;
        b=QHLQ3rMDWNB/26AcEt/tmQB5kk5HXskxmUCeZxjWkZY+/+gmxKxtLYBbUrbEJ9IIVl
         TXxNm1cMDVlWjjJE1EL8SyZCs1686ubATja87bAh8hUlCMrmrU+p/h2HnsSi4f6M2VVv
         ZijvBz00fTjJ0Pey6nM7GppsZ7HFYqOWG62UIeKnTBmuInsYh9s3ltuexL+iuerhbSSN
         uMg0jOQeNSsrtwUY8ODf5A01NMcBOal5Wp3/QlINZyEPfodQvP1VTMgezqqMzuZLU7iB
         OiJNsnH57C+xyoKu+8wTdisD8sJn3vaHgIGWHGCi7wCCfdTlsvskKx4OEy1NR1/zo36u
         mnOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=wXBwe+C96NVMAw/WVJPmEePxSXgR8UnAw4mAKidy+Uc=;
        b=jks338WhTHfECjgtr1U4zauTz6hfppYTkFokgmX5iCtZRvDfN4VJkCCOXRRlLILUAp
         YlR8Q6LCm5Kl7FKn+KEGJcVnrmY/6bg7uQNRiBwKrQCA2MoWZVbCmevuEPgN33momOnz
         WVVklKSTZ4K+276yW/Dig0p/uzO5yr620HGz8NoVFEv+uKRawGHjZHWBCo8hh7Ymj+cm
         Jl+NdqAYcx4Vwn1m68jUYqjculxsJTGuSGxqKK4L4DfMjDuKoHmqJe094KGE9xr1pcUD
         3ZqG+XkUf9aPif1go4Mz1YM6g7AG0z5zWFB6hONKaeBUaP89r7YKlN2o2Yp4i2Eusx2a
         IJnA==
X-Gm-Message-State: AOAM530SBEiY0L2QQ8QmU0S3zl/AMtPyKsNnPY6SwyfsMXvTkCM5h07U
	oxnurNZdY7jz952G0t3pii1VEA==
X-Google-Smtp-Source: ABdhPJz826d62i3FadVITpOZWzNuXjQqqxILWUC/72776fLbfHJdkqtyjig/TilRLtZ8efOdunSk1g==
X-Received: by 2002:a5d:5262:: with SMTP id l2mr839408wrc.190.1630700631792;
        Fri, 03 Sep 2021 13:23:51 -0700 (PDT)
Subject: Re: [PATCH v3 14/30] target/i386: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-15-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <b2796bcf-8358-1faf-cf30-0ad87cd0ac08@linaro.org>
Date: Fri, 3 Sep 2021 22:23:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-15-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/i386/cpu.c         | 6 ------
>   target/i386/tcg/tcg-cpu.c | 8 +++++++-
>   2 files changed, 7 insertions(+), 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:24:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:24:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178413.324391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFjm-0002vu-Ry; Fri, 03 Sep 2021 20:24:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178413.324391; Fri, 03 Sep 2021 20:24:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFjm-0002vl-OX; Fri, 03 Sep 2021 20:24:34 +0000
Received: by outflank-mailman (input) for mailman id 178413;
 Fri, 03 Sep 2021 20:24:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFjk-0002vT-Mf
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:24:32 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3df121d7-77d4-470d-8279-ac0f63c31d72;
 Fri, 03 Sep 2021 20:24:31 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id u9so327078wrg.8
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:24:31 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id l15sm315117wms.38.2021.09.03.13.24.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:24:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3df121d7-77d4-470d-8279-ac0f63c31d72
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=MUAOnWb7Jr2o1NB8tpwx1E/Lm7U1xeRRhxPNvAXjZbg=;
        b=vzZJ3StsffaZE2XWFvvqf2xJqEy05bbfJRhtDaeaWq7QuAU52n/TB/ULQ8WeVNs54M
         UGk4VZfAumjAQ6YHZqLXwehiRMweE/3kJmX4Exi6/vcM0AbVsP9DyQ+kTZSyXOjSTQXE
         yltj+XKDtFQcrEjs98IbsnvIXwqtEH/TjR8pwEw8KEUkc8NYtmXPjP513FwOCCGUt9wr
         aStX5sxRH5/5TVq63meFggk5pkY4HjNNTzztQF0zM0dnsU7O4t2Ixsf6lRKnmGoibQxd
         a4FPgOEbY/12bxwaYsGxLbjms3hn+0Bpm8n2QXKQeM0XRr72OBCjWTcA7StsxwdGykZR
         MZRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=MUAOnWb7Jr2o1NB8tpwx1E/Lm7U1xeRRhxPNvAXjZbg=;
        b=JOYTnnJAZUa5Dj7c6hA/CHUvkpGrG6mFHpK0TPc56S07LSbsvxJ4EcyHfA+TxxQCj7
         db0HDBeZMwmMJv03cMvNaiqfvtL7RF7u7lJiGQ2k+SzQhDTO+f5L3dauIh/3p3DVu5Bu
         u+M2vhyV37FGJfvM5vnVeVhBuuCkzXbDl13OFQWtpZAl8skERJegw5Mht+OsSjSNapAo
         88CgEx6SWnIWQet5yDugTNMf4p3EomArIiaPmVwAmUR6QL0uTOZ6UsnZ4TpzbNCjG5+T
         xAyTFZ19W5UOfKErFnAu77Kb+5EyrDqbApLFOQx2dubjSDr6YOoLRGbBUaHNDEoNqoAv
         Jo/Q==
X-Gm-Message-State: AOAM533slL4RzxBXWPC65IwtOhkQpqhRFhuQIqleKkxxqEX1U1TEsX3A
	Tt54VkVnu76Q3AN/RvtArMIuJA==
X-Google-Smtp-Source: ABdhPJzUsw/5kYywwKhFVX1rjFqmUKa3i/mI2bjH4AHQJ6YuS1HfmSJytNGtWbSvomY0u97BFeRM4Q==
X-Received: by 2002:adf:d4c2:: with SMTP id w2mr799125wrk.235.1630700671118;
        Fri, 03 Sep 2021 13:24:31 -0700 (PDT)
Subject: Re: [PATCH v3 15/30] target/m68k: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-16-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <4ac98aaa-ca63-0db6-87ce-a77193044dd1@linaro.org>
Date: Fri, 3 Sep 2021 22:24:23 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-16-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/m68k/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
> index 66d22d11895..94b35cb4a50 100644
> --- a/target/m68k/cpu.c
> +++ b/target/m68k/cpu.c
> @@ -31,10 +31,12 @@ static void m68k_cpu_set_pc(CPUState *cs, vaddr value)
>       cpu->env.pc = value;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool m68k_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:25:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:25:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178419.324401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFko-0003ZG-5N; Fri, 03 Sep 2021 20:25:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178419.324401; Fri, 03 Sep 2021 20:25:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFko-0003Z9-2L; Fri, 03 Sep 2021 20:25:38 +0000
Received: by outflank-mailman (input) for mailman id 178419;
 Fri, 03 Sep 2021 20:25:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFkm-0003Yw-Ua
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:25:36 +0000
Received: from mail-wr1-x42c.google.com (unknown [2a00:1450:4864:20::42c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5c79fb02-b334-4c5b-90db-2703ef3efbad;
 Fri, 03 Sep 2021 20:25:36 +0000 (UTC)
Received: by mail-wr1-x42c.google.com with SMTP id x6so306880wrv.13
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:25:36 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id q13sm324320wmj.46.2021.09.03.13.25.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:25:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c79fb02-b334-4c5b-90db-2703ef3efbad
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=W7k4RtqOZy7ya1WayabP+HxNcp5Fq3Mk2ZbTc+R+8wk=;
        b=Y7PtSwX3LA3cO29hKgEQ0rkI844j3EG/DnwVbuMutIDQuUE90Uvns4x6mk4yf7B+cm
         Z6Ehutu4130di1yQJkVCvyamcRHY9V7H8+/nW9cFA4cQP7R4+86sspF6SjVkmgr6mnbK
         taRoXaEV9tG9Hfk01UC455nNMEqk+FHQaJULcihgVHUwNroTfGeteSjAaIcGVZ3lfe7k
         vWQNvEvJXomiaKpwJ4tHqL+1JgHxOaDk5BcG2O5Ba51DpymToNVYzbCqx9P90GXzyM8j
         2b3ZUyaRty9XmOwlTzWm7qEP3nm+1Lkh4CwG0qqunwT4jbB+PrEQJvI5TjsNGJMxJsoD
         DntA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=W7k4RtqOZy7ya1WayabP+HxNcp5Fq3Mk2ZbTc+R+8wk=;
        b=AiP9q96TBkyUl2OWBmPi6OqDblElP0cTyrelP6EEQV4z35oD41XPuIWMxg3tqQO+Fo
         bARSU/8Q4/M/yqHc1psV5XY/u34nxXrXgycZvVhLMiFsmASkLl8RAVsWkPAOoXNjOo4H
         Lmr4kftJ7Q1vZw3H+BsAQsFdOS1CCNTdMHG5kFmRAbY0LJwIZHtykT14aVIOOPb2WGR6
         ysJ0C3FVmM0imHfXhLYnq2Y5o91O0OLCCj15qd5jdodUpTf0vzPNc78O3pswui+Yq8qO
         gAepvqfHcnZweUqndktduy3GiSp5ycJn9f82MP7AGVZeUNVglttjjrwP+OCrelEt7Puq
         Egjw==
X-Gm-Message-State: AOAM532uIie3Vg+pTseiq5mRuPmnyi4G+xRZ2kEtaWrl+Vfr18qXRPBK
	sDkxAl8pmjLDvlbQxnUIMaxz5Q==
X-Google-Smtp-Source: ABdhPJy/o6kXb1fKJrZGJ7NA/+HnxHPvtkph89J5ZY0ajIZpI/IL3HlrDNTpZbKsJbsJhN5+OKI2UQ==
X-Received: by 2002:adf:d239:: with SMTP id k25mr830865wrh.314.1630700735383;
        Fri, 03 Sep 2021 13:25:35 -0700 (PDT)
Subject: Re: [PATCH v3 16/30] target/microblaze: Restrict has_work() handler
 to sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-17-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <acaab338-bc0c-3c06-37ea-dee2ba753900@linaro.org>
Date: Fri, 3 Sep 2021 22:25:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-17-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/microblaze/cpu.c | 10 ++++++----
>   1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
> index 15db277925f..74fbb5d201a 100644
> --- a/target/microblaze/cpu.c
> +++ b/target/microblaze/cpu.c
> @@ -92,12 +92,15 @@ static void mb_cpu_synchronize_from_tb(CPUState *cs,
>       cpu->env.iflags = tb->flags & IFLAGS_TB_MASK;
>   }
>   
> +#ifndef CONFIG_USER_ONLY
> +
> +#ifdef CONFIG_TCG
>   static bool mb_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:26:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:26:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178425.324413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFla-0004AJ-G0; Fri, 03 Sep 2021 20:26:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178425.324413; Fri, 03 Sep 2021 20:26:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFla-0004AC-Bq; Fri, 03 Sep 2021 20:26:26 +0000
Received: by outflank-mailman (input) for mailman id 178425;
 Fri, 03 Sep 2021 20:26:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFlZ-0004A3-Es
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:26:25 +0000
Received: from mail-wm1-x32d.google.com (unknown [2a00:1450:4864:20::32d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e4397a1e-e8a3-4803-ac39-0d29a1f114d7;
 Fri, 03 Sep 2021 20:26:24 +0000 (UTC)
Received: by mail-wm1-x32d.google.com with SMTP id
 k5-20020a05600c1c8500b002f76c42214bso318379wms.3
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:26:24 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id c24sm245010wrb.57.2021.09.03.13.26.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:26:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4397a1e-e8a3-4803-ac39-0d29a1f114d7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=9EhQsb/ACMR+ZUYvfsQ8sm7Iu0dalRztguOzy6Crcdk=;
        b=QYSXBDXbshWafmBVS2xgefEvLphaKwA0sQfPQhGCg4AdEMozIzMcXqiXVGYB7CIewz
         jGF6cmCkebhdq+OE+6RWrDO4qTxvTVl4BSUMt65GZglcAtMPB7hoGF8O6DZlqsExbX4s
         XAzARgy5lmbAjguMkMH9lOvN2ERwa8Zk1z1kUQwdBWqSZWZPNk39la+7ZY1kN/pV4gyV
         RBRSsHPMZGhaBmnt8AGJWS+r15xgowV56WK1jX05FsVJKFg3NVzyfEkg92ZjiSGXwFOY
         XPliUy/Mwpe/M5POTE0+dgsKNGe4w2JH0ejOk5iEpYzt1oiH6GsVpGrvEbVnBC4nn0nZ
         rJzw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=9EhQsb/ACMR+ZUYvfsQ8sm7Iu0dalRztguOzy6Crcdk=;
        b=PCNHejtiT1CLRnZmh45VDJ5OdBHtjmbUE87Q5fZrGu6cn6ACsHiCPYmm516q1mNaEo
         gPKhXleycghBiLOoHH2HZfBjTYXjqhrOs50lZtc4bJ2T2HICLz90GPw36j7rPnzCbeL8
         0fq+CCl42zRREpxaZ2hjMwJz4PbqdJOmSGkmbIETTKsViE9gFl9wpYCHBmzED9VTvmUU
         XAEooRizrbYVqWi/XgfuHgpuqhVhfouNxRnoRPGhpcWtWK4I/2lqWQEqmOjxZx/Aqtgw
         8GF6/hN3NEP7YkalyPJRHxeQvPoimI/1ditnMSrg2Hiegjge47TCfxf4TDGbTAd7tJJ9
         Gdng==
X-Gm-Message-State: AOAM533an0b6nSzYAGxrtto6QlD/VwAhrujbx40lmqC/WL813iQuHgNd
	tvcA6j87xT87WZ2BzYCj6jTicA==
X-Google-Smtp-Source: ABdhPJxBhvIh1VZtcwLHBRW/mfjmLMhAmkSWajrXDW4E0STizvGidXuGiX92vd10z3qXEwsTB/9ddw==
X-Received: by 2002:a1c:202:: with SMTP id 2mr456635wmc.122.1630700783750;
        Fri, 03 Sep 2021 13:26:23 -0700 (PDT)
Subject: Re: [PATCH v3 17/30] target/mips: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-18-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <f24d9add-497c-8342-f37b-3d299a383d73@linaro.org>
Date: Fri, 3 Sep 2021 22:26:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-18-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/mips/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:31:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:31:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178432.324424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFqC-0005sJ-63; Fri, 03 Sep 2021 20:31:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178432.324424; Fri, 03 Sep 2021 20:31:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFqC-0005sC-1S; Fri, 03 Sep 2021 20:31:12 +0000
Received: by outflank-mailman (input) for mailman id 178432;
 Fri, 03 Sep 2021 20:31:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFqB-0005s6-3E
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:31:11 +0000
Received: from mail-wm1-x330.google.com (unknown [2a00:1450:4864:20::330])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fe8d4ae7-09e8-4557-aa50-c0513c956a23;
 Fri, 03 Sep 2021 20:31:10 +0000 (UTC)
Received: by mail-wm1-x330.google.com with SMTP id g135so86906wme.5
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:31:10 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id i21sm232068wrb.62.2021.09.03.13.31.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:31:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe8d4ae7-09e8-4557-aa50-c0513c956a23
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=bg/lxOaKgNK7p03NwBu4EagG8gbF3wogDiecV24XpwU=;
        b=N7Y7UOCwkEiqMP/DA2rhef2OLc1uTSqNeETW5xALwf+2bPM9vLPwLP2mwNJoCACMW9
         VGKi6Y2sF5A5a/NW+D+AWWckUMjZ4g3RW+QjWA4qk3jIut3SSJ9spjBUDhjvwevFox5I
         JxUNi0LSnwXZj39KDzqlWU4jMSwm5sc60TEToEgbjTBaHRQCxznMjmrAUesETWr1pxNW
         YyW+6rvnEyxw8H9JQrzfuQGZib07c4Ny8cSP8XSD4x2scI1MkvpGzBsmG8mPGMPdfgDz
         mq9c4Q6tDvPpU/qVPLs0vNhp9EqbjHmLIcoKB7Y3JPSTxJq3EY1M5NeWFCnMB3KfYlPd
         C/RA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=bg/lxOaKgNK7p03NwBu4EagG8gbF3wogDiecV24XpwU=;
        b=dyGKD20n76EzEB1MwLfuEDj+jVdAoURheShco0xeoS1L64ZlwZWsX2DozUVJuVReSA
         yXpqVx9wkrH0jOT7Fm1AaRPDlDzuqvHI6Vz1+E1dbBaJ5iTXQVHJrPjtUVauLAmCrJQo
         lUR/IfWKnS3udWIXxNNEUcMEnOcOtnCM0IXT/o/n9CSe++yoBsWobf/1LaGViVGiVKta
         3RVU2zOP/dtob3mWvSTbrDSyjCnXiG5iBGiM12v82lZTbQ41xQ9eJ2qJGiroYrucmY+r
         Qi+2zLoXsgXyHVxyfIsZQhuOUyN+TFHoQemSSGB/r/+i2IdzYXLrS+ztbtKqMJXO/Ub5
         Laow==
X-Gm-Message-State: AOAM533xfFQyNOW/7D6CQoMRdoWUns81cLMZVIGYvddfA2N7d+bW9tEm
	wRL9EBdtGCup1+Z5ofk7kn/juA==
X-Google-Smtp-Source: ABdhPJwzsD1YoBLjkg4zHzKfIgPz4jC9G2HwCKaXpDjC3rpGD+JAqToGSBQB4JeRK7m1b0G3qcn/dw==
X-Received: by 2002:a05:600c:35cd:: with SMTP id r13mr538951wmq.24.1630701069292;
        Fri, 03 Sep 2021 13:31:09 -0700 (PDT)
Subject: Re: [PATCH v3 18/30] target/nios2: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-19-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <f58740c8-2199-61de-014a-fcb1db4835f8@linaro.org>
Date: Fri, 3 Sep 2021 22:31:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-19-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/nios2/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/nios2/cpu.c b/target/nios2/cpu.c
> index 947bb09bc1e..f1f976bdad7 100644
> --- a/target/nios2/cpu.c
> +++ b/target/nios2/cpu.c
> @@ -34,10 +34,12 @@ static void nios2_cpu_set_pc(CPUState *cs, vaddr value)
>       env->regs[R_PC] = value;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool nios2_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:31:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:31:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178437.324434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFqp-0006QT-Db; Fri, 03 Sep 2021 20:31:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178437.324434; Fri, 03 Sep 2021 20:31:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFqp-0006QM-Ae; Fri, 03 Sep 2021 20:31:51 +0000
Received: by outflank-mailman (input) for mailman id 178437;
 Fri, 03 Sep 2021 20:31:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFqo-0006Ok-6q
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:31:50 +0000
Received: from mail-wm1-x32a.google.com (unknown [2a00:1450:4864:20::32a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f49976a1-f222-4986-9403-0fc003c84747;
 Fri, 03 Sep 2021 20:31:48 +0000 (UTC)
Received: by mail-wm1-x32a.google.com with SMTP id m2so118211wmm.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:31:48 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id 129sm315688wmz.26.2021.09.03.13.31.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:31:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f49976a1-f222-4986-9403-0fc003c84747
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=IuI8V71Jj+hTTxPmLaVm9RgBYo3XervExPJzr/+wck8=;
        b=BHJmUzDO3YywOUa9sdMc6M7Mu/GQRndqEiNv6lAlw/Rx2vJ69SftU14BI/7FyKvVj9
         /uSejEVcW3KrPYDJAJyYChPMo1COohaJL/FXfRq2f06sB0IBPSkSyKXLV8C/rFTDVb7E
         WjHNSLL5Ss9dhHvBCzA6nwVi4BQ/020CB2mCBU5FNscWowKriFlIsU05ePAcI11HBhBZ
         8EJaMt/LSnFNof7n0feMVm2QdfWC+CGhuNk0uCoYDLrGLAXIEX9D0/LRxoU1O0AF4UA4
         kzHj1+GLCmB1fy3OyPTx6ajgae2BTAFMx+usPTXWXhBd7mz+rL1Z/GcPM+49esXseeLB
         SgmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=IuI8V71Jj+hTTxPmLaVm9RgBYo3XervExPJzr/+wck8=;
        b=oL7TTvZ8MnH3A0eGix+nqMs/+0M2ladZXXM2mXM+78bJziLnCy5Ronv2K2OoX1ulyX
         +gE4dicmKzCwtC1rXgR0P2c/iv5Q0Q/xAlDCkDypchSRj3DBouDDDNmW+TV1YdTDox1L
         k48cAtflmOgRHDaUGljjHhFZDj6P0JSPEqa00JS5cO9dx2SxX95/F4VtOXzBDr5rkFWe
         2Oj9MlRBcof+S8VHFIh3frOxlUrQ8vHVJ8qS+uykGPAJUUUdd/3VJvEyqlLWcD3aed18
         jk3ZXBrTJyYJ1aNaLXiVgEATtzsQMP0nyYtoa9DfrxXIyFWYKbMMbGVaOXbsA7Kap0/R
         w0HQ==
X-Gm-Message-State: AOAM531f55EtLlEgCgnUElBkBWiYtufxN7V3O6cbyMHdTMqNL9xM+z5c
	fxK4VV/iXY1f5PJNMIIAAcwqDA==
X-Google-Smtp-Source: ABdhPJxfq3NlZYeWiDE5W/APiCzDJdzufksHOX18ryVfGevh4sxLdu17AHeHdIzgstpOHsuJ9Y+KsA==
X-Received: by 2002:a05:600c:cc:: with SMTP id u12mr440384wmm.182.1630701108006;
        Fri, 03 Sep 2021 13:31:48 -0700 (PDT)
Subject: Re: [PATCH v3 19/30] target/openrisc: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-20-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <ed2e67da-0ff5-88af-6522-1fd029511e3c@linaro.org>
Date: Fri, 3 Sep 2021 22:31:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-20-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/openrisc/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
> index 27cb04152f9..6544b549f12 100644
> --- a/target/openrisc/cpu.c
> +++ b/target/openrisc/cpu.c
> @@ -30,11 +30,13 @@ static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
>       cpu->env.dflag = 0;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool openrisc_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:34:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:34:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178444.324446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFtU-00078Y-RE; Fri, 03 Sep 2021 20:34:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178444.324446; Fri, 03 Sep 2021 20:34:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFtU-00078R-O8; Fri, 03 Sep 2021 20:34:36 +0000
Received: by outflank-mailman (input) for mailman id 178444;
 Fri, 03 Sep 2021 20:34:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wQk4=NZ=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mMFtS-00078L-Or
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:34:34 +0000
Received: from mail-wr1-x42e.google.com (unknown [2a00:1450:4864:20::42e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 34b6fe07-1500-404c-8bd7-17c66be5d3a3;
 Fri, 03 Sep 2021 20:34:34 +0000 (UTC)
Received: by mail-wr1-x42e.google.com with SMTP id u9so356496wrg.8
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:34:33 -0700 (PDT)
Received: from [192.168.1.36] (21.red-83-52-55.dynamicip.rima-tde.net.
 [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id d7sm258631wrs.39.2021.09.03.13.34.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:34:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 34b6fe07-1500-404c-8bd7-17c66be5d3a3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=LFeEcrxPmiQ0y9mkjFms4J4GrCvxgbbAZyrEPk3foT8=;
        b=blhwVscSGxsnvMX9mpwF1WKnN0xyca3wo+jPJrvx7iJfDyuaVBJ0KE2ZZYoLwyB4Qb
         4ovLs/5Mxxmi9wMRp2vY8kz61AoNMdOvCCJQ1ezWt/9nx9cQ0hjn+MeBnxsE8IfF+HWm
         MRXjSCyRODB4tHb2j9tg9SRHAw9DJhbv0/cKEArMyfVM1qOCRF3s4YzT2Dln7MMoqaZq
         y6iKsF7H+hxlB36Mu4fKWpUJ5EAdcOvfMVCI3XguqcRlT/JRvp5sK6I5Rgfg+ycwLNfj
         b/WnJWkRX3vTttRdmBA7xpv8BARViFg/xzi1CERUqrSWq87BIGBc2CT5TLVm4/NdQPul
         vUWw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
         :date:user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=LFeEcrxPmiQ0y9mkjFms4J4GrCvxgbbAZyrEPk3foT8=;
        b=Pg/bU+MXW2o8zWwj3yGcyIdnzQQ8i5zPWyxRRKE23dMRuwTVI3ZWtHqtk2XsF/yHqq
         4WYpG3mT/I9G3/llBmkMBsaSNqmRUAGUdTRSGmlLRdVnPyH87XXUbq4/BZpqhmnh2rUe
         mLn40IKquLmJOmSgmuUfWi+qlfDQI7bZFP0xLBCgapPMvh0VSsffWB3apiaIKhUzWd8k
         COtMXbG1P9WHRHKK4AM2HwS2vzFt3fVnF7HL5VQ/gO83g3TJWzLvMVByVYTBexKabtnu
         F5Od49oYqPE/wBczIXlE6LAguGS+jA3G5l7TPqrCKE0FoiSUjWWpMGFvESyC+fWs+GFp
         33vA==
X-Gm-Message-State: AOAM533TKda7709aZDLiRQy4AdGngZTqIbkWIyAkm/nnW9KLBo7Z74Qb
	FIWnuq58DHT8g5o23+rk2VA=
X-Google-Smtp-Source: ABdhPJzlEMbpTMPYknNobW93n1KQ+VYIJEKgvJhtzWFM57YHy480w/4A7Y7tiGaFvSIUZksLBnn7iw==
X-Received: by 2002:adf:804a:: with SMTP id 68mr886849wrk.236.1630701273190;
        Fri, 03 Sep 2021 13:34:33 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
Subject: Re: [PATCH v3 08/30] target/alpha: Restrict has_work() handler to
 sysemu and TCG
To: Richard Henderson <richard.henderson@linaro.org>, qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Laurent Vivier <laurent@vivier.eu>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, haxm-team@intel.com,
 Colin Xu <colin.xu@intel.com>, "Edgar E. Iglesias"
 <edgar.iglesias@gmail.com>, Stafford Horne <shorne@gmail.com>,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, Cameron Esfahani <dirty@apple.com>,
 qemu-s390x@nongnu.org, qemu-arm@nongnu.org,
 Michael Rolnik <mrolnik@gmail.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 David Gibson <david@gibson.dropbear.id.au>, qemu-riscv@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Chris Wulff <crwulff@gmail.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 qemu-ppc@nongnu.org, Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Paolo Bonzini <pbonzini@redhat.com>, Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-9-f4bug@amsat.org>
 <3cd48aba-a1a1-cde3-3175-e9c462fcb220@linaro.org>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>
Message-ID: <740a2e5c-3dad-fc7d-b54a-0c405faa605e@amsat.org>
Date: Fri, 3 Sep 2021 22:34:30 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <3cd48aba-a1a1-cde3-3175-e9c462fcb220@linaro.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 10:18 PM, Richard Henderson wrote:
> On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
>> Restrict has_work() to TCG sysemu.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   target/alpha/cpu.c | 4 +++-
>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
>> index 93e16a2ffb4..32cf5a2ea9f 100644
>> --- a/target/alpha/cpu.c
>> +++ b/target/alpha/cpu.c
>> @@ -33,6 +33,7 @@ static void alpha_cpu_set_pc(CPUState *cs, vaddr value)
>>       cpu->env.pc = value;
>>   }
>>   +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>>   static bool alpha_cpu_has_work(CPUState *cs)
> 
> Drop CONFIG_TCG for alpha; it's always true.

What is the rational? "Old" architectures (with no active /
official hw development) are unlikely to add hardware
acceleration, so TCG is the single one possible? Thus no
need to clutter the code with obvious #ifdef'ry?

> Otherwise,
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> 
> r~
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:38:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:38:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178450.324457 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFwv-0007oE-AI; Fri, 03 Sep 2021 20:38:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178450.324457; Fri, 03 Sep 2021 20:38:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFwv-0007o7-6B; Fri, 03 Sep 2021 20:38:09 +0000
Received: by outflank-mailman (input) for mailman id 178450;
 Fri, 03 Sep 2021 20:38:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wQk4=NZ=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mMFwu-0007o1-HC
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:38:08 +0000
Received: from mail-wr1-x42e.google.com (unknown [2a00:1450:4864:20::42e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d6addfb9-5549-498b-b27b-2b8be1ab465b;
 Fri, 03 Sep 2021 20:38:07 +0000 (UTC)
Received: by mail-wr1-x42e.google.com with SMTP id b10so429128wru.0
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:38:07 -0700 (PDT)
Received: from [192.168.1.36] (21.red-83-52-55.dynamicip.rima-tde.net.
 [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id d8sm283071wrv.20.2021.09.03.13.38.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:38:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: d6addfb9-5549-498b-b27b-2b8be1ab465b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=fNIhsoAMPvZrQEmEvDEa/ofLQCAFlDFx6OPwYJbakLA=;
        b=OL7bcNknLzAqSVlHwkuNrFgmHEdJJTT0ywZpM33E+WzKRYLTjn/SmGnREPXGBif/IU
         PqPcK3SjLG5eogScB3jtfBXh29opHuqZSJGeNZd9Ke+t/AT5Cr5VwhVWY6Gtra5FS44y
         zIqcsHP71yoBrqPd/tqJ/b53dF7tqywgJ6KeJPrt2jdTFotMdIWIPMYLMJAvvMhlNjSQ
         H2ILUDze/pO7mpdvET5jz2pXK4k5IhHgCgzK6c3TFxwQ/zgNvWS7KYfj6KO8XO/AjwPO
         FYIYxhtgYZeXyzLZ3F3EqMh3ogFwJ5iW7yzCb/zV+nhXKNxtMozTPxiBRByP9xmArzZh
         7eKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
         :date:user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=fNIhsoAMPvZrQEmEvDEa/ofLQCAFlDFx6OPwYJbakLA=;
        b=cxEMaYPnkonMt3PmJ7YXDzxmVYiP6gVk7XPAa/XOWjX9Kio7bKVs6D4daS3tCm17ti
         M2urxuU7iXlRVK0KP351lKoXvuQ9YeaO49bYsSQH0+4Xy/IjbKKyVEe74A+W463c5U/M
         ozf2lYiF6MOqP+CvOalPBsbVdbaeBLdPy5iz+AiDXV00Es0RIvj40HUzOkw6Fzx6k9xj
         ujcGqcVyMeoZMgE5DeopaCkAgHKXOBsPmprTZu3hHiDV/e3vd9ZOuJax/PgN1zig3sv3
         hjLMg1gIXixJvRDquv9RPowK/kWdSxa1wWhhs1Flc0uvDi4X4XdRzJPo1sZTewk5W+9r
         JDlg==
X-Gm-Message-State: AOAM531ppCrPIIweoaqsr7bxac7SuZ+szL0o/6OEmxJgb4oO0E9kWJW+
	g5lzCcx3GYs8nd7sebufCDA=
X-Google-Smtp-Source: ABdhPJz6x+pWgErAspkEq96cAfi2JjZZYOWbgKHTQiSSXfqb5WO1BMDSjMrLjiY2+IXe9aaRofWxVQ==
X-Received: by 2002:adf:80e5:: with SMTP id 92mr881885wrl.300.1630701486851;
        Fri, 03 Sep 2021 13:38:06 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
To: David Gibson <david@gibson.dropbear.id.au>
Cc: Peter Maydell <peter.maydell@linaro.org>,
 Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, qemu-devel@nongnu.org,
 Jiaxun Yang <jiaxun.yang@flygoat.com>, Laurent Vivier <laurent@vivier.eu>,
 Max Filippov <jcmvbkbc@gmail.com>, Taylor Simpson <tsimpson@quicinc.com>,
 haxm-team@intel.com, Colin Xu <colin.xu@intel.com>,
 Stafford Horne <shorne@gmail.com>, Marek Vasut <marex@denx.de>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Richard Henderson <richard.henderson@linaro.org>, Greg Kurz
 <groug@kaod.org>, Cameron Esfahani <dirty@apple.com>, qemu-s390x@nongnu.org,
 qemu-arm@nongnu.org, Michael Rolnik <mrolnik@gmail.com>,
 Sunil Muthuswamy <sunilmut@microsoft.com>,
 Palmer Dabbelt <palmer@dabbelt.com>, qemu-riscv@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Chris Wulff <crwulff@gmail.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 qemu-ppc@nongnu.org, Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Paolo Bonzini <pbonzini@redhat.com>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org> <YTFxZb1Vg5pWVW9p@yekko>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>
Message-ID: <1d8ea9ea-dda0-bf0e-a2c8-13d36c891d9c@amsat.org>
Date: Fri, 3 Sep 2021 22:38:04 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <YTFxZb1Vg5pWVW9p@yekko>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 2:50 AM, David Gibson wrote:
> On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daud wrote:
>> Each POWER cpu has its own has_work() implementation. Instead of
>> overloading CPUClass on each PowerPCCPUClass init, register the
>> generic ppc_cpu_has_work() handler, and have it call the POWER
>> specific has_work().
> 
> I don't quite see the rationale for introducing a second layer of
> indirection here.  What's wrong with switching the base has_work for
> each cpu variant?
> 
>>
>> Signed-off-by: Philippe Mathieu-Daud <f4bug@amsat.org>
>> ---
>>  target/ppc/cpu-qom.h  |  3 +++
>>  target/ppc/cpu_init.c | 26 ++++++++++++++++++--------
>>  2 files changed, 21 insertions(+), 8 deletions(-)

>> @@ -8796,6 +8800,12 @@ static bool ppc_cpu_has_work(CPUState *cs)
>>      PowerPCCPU *cpu = POWERPC_CPU(cs);
>>      CPUPPCState *env = &cpu->env;
>>  
>> +    if (cs->halted) {
>> +        PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);

Maybe I should have kept the 'if (cs->halted)' for the next patch,
simply dispatch here, then in the next patch the code simplification
is more apparent. I thought this approach would involve less #ifdef'ry
but haven't checked the other way around. Will do now.

>> +        return pcc->has_work(cs);
>> +    }
>> +
>>      return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
>>  }
>>  #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:38:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:38:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178454.324468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFxa-0008Me-LS; Fri, 03 Sep 2021 20:38:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178454.324468; Fri, 03 Sep 2021 20:38:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMFxa-0008MX-Go; Fri, 03 Sep 2021 20:38:50 +0000
Received: by outflank-mailman (input) for mailman id 178454;
 Fri, 03 Sep 2021 20:38:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMFxZ-0008ML-9W
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:38:49 +0000
Received: from mail-wm1-x333.google.com (unknown [2a00:1450:4864:20::333])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bc088393-fe84-4f62-bef7-11e0480e6a94;
 Fri, 03 Sep 2021 20:38:48 +0000 (UTC)
Received: by mail-wm1-x333.google.com with SMTP id g138so98139wmg.4
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:38:48 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id w9sm286608wrs.7.2021.09.03.13.38.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:38:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc088393-fe84-4f62-bef7-11e0480e6a94
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=GL9M66YPePviEeJIanaWkTBsfl1Vxvfhn+ruVwm7BCY=;
        b=M/vaAbEmKAIlbeJ47KCqDEuoW1fU8OLbdaCXy0Pvl+0fPEdToeVM22GDpxEzsjWt9v
         WbAqalwQ/WT+6kAtmN1B0nIOitKaIhJ157SyDHY79vxDGA3ucdaq3AJgPv1Ovs1Fv/aB
         1lSLTga+m6SS4hdqpDGzD3Th0/yrB0aZiBPiOV1Y9NoKc6DtcGD50FEYbqTQdUvsRoIH
         T4tRlzq2k7HSirKjelg2CrSPUYVuJ22AUaoX7n0jXoOTIEfbt+eiTX7puG0QvsvjemoW
         Tfwd8Zr/L/8FyDF1XuV7Wb/Ws8ZaMEPMWJ3/JlWVrBtMcJoHyoGb+mo3WTtGcH+kKg2w
         oQuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=GL9M66YPePviEeJIanaWkTBsfl1Vxvfhn+ruVwm7BCY=;
        b=qKhVfiRINwUR4SOv/yzH2Z5s8oTD+UQbZOcUBILm+Ggn9srTEzYNXaTPv72fhfmujn
         +R20a5MNP7NATCtlI4IR+rCn7wQa5pZi1gJQBC/Ulz79In4B0WRZODTtvDXd6rzexRaB
         5B12Hoa/dlCAwk2n8BuS1vTPB1Gu6VAjjGG5sVVU4YqnqMzipdhGoskDvZMIRlVlB2SQ
         wbvX9/+Cm5GWFpgLtB9fB+VA0ihdN72HigNg5bY04eb6dfG6fHvnZTZllaqM9D0it3Cm
         /Eh3dX3EilkqkaAKMgbLEHrp2hPoqZ9D+lMJ19qkLcBL3uTQnNgnFcCfALh+E2ElqOLE
         Gfww==
X-Gm-Message-State: AOAM5302iAA3z3G09/oMnf/b4ouYV6nBjN9rOp/F96B2qz/3NoVjWrmN
	ga0P0XM3A/u0jkNQU4zlgyBvAg==
X-Google-Smtp-Source: ABdhPJyQ/VEZImP2sey8PyCscUHWehgMFHJDN9IuQ3kxtIigmhduRyDKYPM4Bvm1IEM1SgOYGjMY+A==
X-Received: by 2002:a1c:2547:: with SMTP id l68mr468593wml.23.1630701527658;
        Fri, 03 Sep 2021 13:38:47 -0700 (PDT)
Subject: Re: [PATCH v3 08/30] target/alpha: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Laurent Vivier <laurent@vivier.eu>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, haxm-team@intel.com,
 Colin Xu <colin.xu@intel.com>, "Edgar E. Iglesias"
 <edgar.iglesias@gmail.com>, Stafford Horne <shorne@gmail.com>,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, Cameron Esfahani <dirty@apple.com>,
 qemu-s390x@nongnu.org, qemu-arm@nongnu.org,
 Michael Rolnik <mrolnik@gmail.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 David Gibson <david@gibson.dropbear.id.au>, qemu-riscv@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Chris Wulff <crwulff@gmail.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 qemu-ppc@nongnu.org, Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Paolo Bonzini <pbonzini@redhat.com>, Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-9-f4bug@amsat.org>
 <3cd48aba-a1a1-cde3-3175-e9c462fcb220@linaro.org>
 <740a2e5c-3dad-fc7d-b54a-0c405faa605e@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <8c70c273-cb11-c6da-a456-c906df86326b@linaro.org>
Date: Fri, 3 Sep 2021 22:38:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <740a2e5c-3dad-fc7d-b54a-0c405faa605e@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 10:34 PM, Philippe Mathieu-Daudé wrote:
>> Drop CONFIG_TCG for alpha; it's always true.
> 
> What is the rational? "Old" architectures (with no active /
> official hw development) are unlikely to add hardware
> acceleration, so TCG is the single one possible? Thus no
> need to clutter the code with obvious #ifdef'ry?

Correct.  Broadly, if git grep CONFIG_TCG is empty in a subdirectory, don't add the first 
instance.


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:42:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:42:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178462.324479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG0r-0001dt-6T; Fri, 03 Sep 2021 20:42:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178462.324479; Fri, 03 Sep 2021 20:42:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG0r-0001dm-3T; Fri, 03 Sep 2021 20:42:13 +0000
Received: by outflank-mailman (input) for mailman id 178462;
 Fri, 03 Sep 2021 20:42:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG0p-0001df-66
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:42:11 +0000
Received: from mail-wm1-x334.google.com (unknown [2a00:1450:4864:20::334])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da4756ff-0aeb-4025-b282-098aaafd534c;
 Fri, 03 Sep 2021 20:42:10 +0000 (UTC)
Received: by mail-wm1-x334.google.com with SMTP id
 k20-20020a05600c0b5400b002e87ad6956eso337030wmr.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:42:10 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id y15sm462739wmi.18.2021.09.03.13.42.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:42:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da4756ff-0aeb-4025-b282-098aaafd534c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=KO1an/5CmjrwEhdc2mhCZ+S9p7TgIYwX9ZNZ9M9TFAE=;
        b=cu2bS4up1KUaZFvfqZzOd2qoAgEXmB9czyJg7Ld9C9RhrRFPlF7SwFqEwPUxttKJHP
         I1xS+ex3bHbMetfNUUCLVWvrfESiUFlOEywVjEsNBviabapOnUNOdhvyN9zKTl2NKPkI
         oFLzpz1YOXk4f6pHR/sY67Y1o2wn+XVbP1qwIi2sdi5BDgf0tM7J6FGdCmto3LcLIaIX
         PKJGhIKO37mpf5Ce+QN4EnowFcFFR+4AUTl4t3FVeWIkVZSObVUv39QZ23RCI1PJPmoc
         scHBKkHA2wztn0q+7QOHwGhnRETaGgrejgdp9/upgRUBCHz7lFrOigJB0pB0zh/LJOTn
         XY+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=KO1an/5CmjrwEhdc2mhCZ+S9p7TgIYwX9ZNZ9M9TFAE=;
        b=rW5bDh5BEYvzpFSqTiNxiPUrF10e0Glnf1w6pgIG+YyqFBNzDc26ktLQBh7epeATzQ
         dWedb90fHZdJPAUuggHZwYUzuHaBl7LjZ1eL0c4XwmVibqiraFukMkmyf1XlKIrRsEyw
         pVcaqBdmpAus5WlPzFlW+QnbruDtzSBUce/g8pTNJrWqajLOfgjW6oOxXcIU++LXhW4C
         6tMVQJAmXTx97W+scJg3OBKeUwU8sSy2WLEA782aWIQNVWZiemJE0vGKdGYicFYJnGlz
         TOG2rwpHOpnro/Qnm4WsPk08Ej2JknrsTzYm5BOnvboI2rGP5TQTspVGnX+ZhR1Thbt5
         tKaQ==
X-Gm-Message-State: AOAM532aSHLlx8q7bXWkWzTNr3itEXkO6HnFLaKSg9o6OxBfi+qiHq0b
	SACAoPpZfgWpr9IjAQOFPPuKrw==
X-Google-Smtp-Source: ABdhPJwceVbbv7hWOXS37awGetSIGUJik1eFEfP7Dd+3VwgeGBrwgNpP3UDG2KidWTsBePHMk6Y+ng==
X-Received: by 2002:a05:600c:a08:: with SMTP id z8mr522391wmp.165.1630701729599;
        Fri, 03 Sep 2021 13:42:09 -0700 (PDT)
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
To: David Gibson <david@gibson.dropbear.id.au>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>
Cc: qemu-devel@nongnu.org, Bin Meng <bin.meng@windriver.com>,
 Eduardo Habkost <ehabkost@redhat.com>, Greg Kurz <groug@kaod.org>,
 haxm-team@intel.com, Kamil Rytarowski <kamil@netbsd.org>,
 qemu-ppc@nongnu.org, Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org> <YTFxZb1Vg5pWVW9p@yekko>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <fd383a02-fb9f-8641-937f-ebe1d8bb065f@linaro.org>
Date: Fri, 3 Sep 2021 22:42:02 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <YTFxZb1Vg5pWVW9p@yekko>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 2:50 AM, David Gibson wrote:
> On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daudé wrote:
>> Each POWER cpu has its own has_work() implementation. Instead of
>> overloading CPUClass on each PowerPCCPUClass init, register the
>> generic ppc_cpu_has_work() handler, and have it call the POWER
>> specific has_work().
> 
> I don't quite see the rationale for introducing a second layer of
> indirection here.  What's wrong with switching the base has_work for
> each cpu variant?

We're moving the hook from CPUState to TCGCPUOps.
Phil was trying to avoid creating N versions of

static const struct TCGCPUOps ppc_tcg_ops = {
     ...
};

A plausible alternative is to remove the const from this struct and modify it, just as we 
do for CPUState, on the assumption that we cannot mix and match ppc cpu types in any one 
machine.


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:43:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:43:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178468.324490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG1w-0002F5-Fm; Fri, 03 Sep 2021 20:43:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178468.324490; Fri, 03 Sep 2021 20:43:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG1w-0002Ey-Cl; Fri, 03 Sep 2021 20:43:20 +0000
Received: by outflank-mailman (input) for mailman id 178468;
 Fri, 03 Sep 2021 20:43:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG1u-0002Eq-CB
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:43:18 +0000
Received: from mail-wm1-x330.google.com (unknown [2a00:1450:4864:20::330])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9533dc19-474a-4853-bf2c-4538a46493f5;
 Fri, 03 Sep 2021 20:43:17 +0000 (UTC)
Received: by mail-wm1-x330.google.com with SMTP id
 k20-20020a05600c0b5400b002e87ad6956eso338365wmr.1
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:43:17 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id w9sm294701wrs.7.2021.09.03.13.43.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:43:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9533dc19-474a-4853-bf2c-4538a46493f5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=vBLdwX3ZChNXVS/9UqNF4wJjMelZK55qA391msJX3qU=;
        b=sz9KbnkkagCr/GFF3OMVQhtS6VhgzaCiOfnUEvPJVL2O8jwbbscHft/H9IJkpP0NsU
         TGizEk76lcvi8mLRugPLNXDpebTnYO3vqlZQK48MJI9TPSqJFF4fhUuswhN7CqY2Ghwk
         3vqpHJ2pzTxkD4Alw5fP1VVDaMLadtQr5cqiT1MBE/uvf9J116rMMc6TMrhbx3UMJdIj
         /DMhuBGeqgleldVui5C3jsYuqvxxkrLkQtp2u6KAbqe0LNNwZJBYdNlmGqmCrDRJX6/D
         H9oo/QoelvlvKhE6ZH5vxKSx6LJafL8LYNlCMQZjRXX4mct49OSCXeOxBiNBVInXZevi
         wD3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=vBLdwX3ZChNXVS/9UqNF4wJjMelZK55qA391msJX3qU=;
        b=IGpODACODIJ3dZm2I+T70rRofLkXDfd2xpYoMVIFs6gbDugYFXzWCJNXmIz3HRqfbG
         QXjMkPP3Ftc0MauyCmX3gpWR0ELoUfseqIR0FN/CdFLIB3HnjxSS4Eh62x8PzCQdgSY5
         g+TmDiltFFtOYQQJuRj+sZ7M5bq3atmKOD5r74Ns1swHO6v9o9x8NQOWv8PKLP5RXDyU
         fqtaY01oJ4QyaWiptVMr+SnTy9E1Sa+NwF3oi+WmgvtJa6HtUTzBN24QtQ1w3K9n+G4i
         boTe96ruzos1hf8ETLkpDAN9xpxe9gVescIyEupNcCQINrbVHUYHJ9IU1jo4bQ/C9ykZ
         zuTg==
X-Gm-Message-State: AOAM532fiBvjkt8ZPHjqNJdwLG9km+wG1O/+kGZSJDeLsN32fTXHLw1f
	C9zVh8HV+umkfe3YwRz+YJVhPA==
X-Google-Smtp-Source: ABdhPJyTwcg40ZXBf0kRQeFkHvEtxWVastgigZVfDdU3EBMLa1vjXC+xX2MEzRyEH8NNNNiGPWLbnA==
X-Received: by 2002:a1c:1c2:: with SMTP id 185mr479522wmb.11.1630701796724;
        Fri, 03 Sep 2021 13:43:16 -0700 (PDT)
Subject: Re: [PATCH v3 22/30] target/ppc: Simplify has_work() handlers
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-23-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <007f0874-b82f-5853-0c08-00528fb22bcb@linaro.org>
Date: Fri, 3 Sep 2021 22:43:03 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-23-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> The common ppc_cpu_has_work() handler already checks for cs->halted,
> so we can simplify all callees.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/ppc/cpu_init.c | 294 ++++++++++++++++++++----------------------
>   1 file changed, 138 insertions(+), 156 deletions(-)

Well, I'm not actually a fan of this; I'd rather the outer function merely dispatch here, 
or preferably arrange to arrive here to begin.


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:43:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:43:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178472.324501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG2I-0002nR-Qh; Fri, 03 Sep 2021 20:43:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178472.324501; Fri, 03 Sep 2021 20:43:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG2I-0002nI-Lo; Fri, 03 Sep 2021 20:43:42 +0000
Received: by outflank-mailman (input) for mailman id 178472;
 Fri, 03 Sep 2021 20:43:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG2I-0002kD-8E
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:43:42 +0000
Received: from mail-wr1-x432.google.com (unknown [2a00:1450:4864:20::432])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d463abf-0a47-4652-abe6-62e2ce844bab;
 Fri, 03 Sep 2021 20:43:37 +0000 (UTC)
Received: by mail-wr1-x432.google.com with SMTP id u9so381612wrg.8
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:43:37 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id d7sm274068wrs.39.2021.09.03.13.43.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:43:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d463abf-0a47-4652-abe6-62e2ce844bab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=OCS3x2IM2IhJ7L7NOzo3OsfaFZSDUFR4VR+0j8dzaJ0=;
        b=H8ozIcixUveIbw9k7stg4KdS0gx7rTV2e5s41Gx6botMrAh8hYIIWhIu+AvbtT54dW
         Tl8+brbL8I6gSbmldeLB6TkvxB9mAloeVrSwAdDFUCxAz3A9ONWLJnZnRcf7/3YYCKiI
         mKw5CM9pdGmfYUyVULTwZdtG4Zo2cC0TJu5c2IH8dI7SsNAjx2c3JA5uxkgkaF3syZYO
         bS8qs1qwUmnUfdlgL3bsdR1L8kzzDcc0saGHUOJA/DshHL9WLii6vhGNcqmBnmKhwPEJ
         E/skqUuaJ6aFM7UfknaO/MQDQsX7XIC1JB3/bhICCmdu8qPCP1VV/4dZA8n9iYUOsjZI
         4+YQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=OCS3x2IM2IhJ7L7NOzo3OsfaFZSDUFR4VR+0j8dzaJ0=;
        b=khCrd3B+fHgkXT2gYGCyZ9w9hPNKH7udY4YcW+II0FWD1TvL/RLybLD4wzx+w12gjx
         CHz1Fm8LD5NCfOsUy5PRpUg7OoJEBNasV1XAtm5TO/8Uy66ysTbXdzZsf5ye2AGWn6lM
         g8KZKa6VDrYVrZoxcbkFVu56T77s1CosjOLXDAdqiOmVRdY7Pkf4ybOrJv0xoDfYjTwf
         py+keLyShGyHVBqpinZ6sycMd5obKrc/OylDwyRKRfraRmAEbeOSq86xdAjiXBRbOYUr
         ehXlE3y4YYxkGARVwluyCX9+wCJVQp9dawNBLAYIkZKlV252bPxSAVOzUymu+A2oq7vK
         YCkA==
X-Gm-Message-State: AOAM531Viow/P47ksqMXyaudtnJPTaK9G14E3xs/cwaOJuihD7C/CLJT
	6B2UdmoJw9MbSEnvkxPjuvRvWQ==
X-Google-Smtp-Source: ABdhPJzDzA0nqf/Zf1bg1Haee/zHM2hXytKc0QH8nIr2MjWxDRi+4jQmQKWadcbSkZgCEwSPJcyooA==
X-Received: by 2002:adf:eac3:: with SMTP id o3mr939918wrn.60.1630701816896;
        Fri, 03 Sep 2021 13:43:36 -0700 (PDT)
Subject: Re: [PATCH v3 23/30] target/riscv: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-24-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <56180d64-090a-1f6f-5c0a-22387c8a2af5@linaro.org>
Date: Fri, 3 Sep 2021 22:43:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-24-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/riscv/cpu.c | 8 +++-----
>   1 file changed, 3 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:44:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:44:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178479.324512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG2j-0003NU-3t; Fri, 03 Sep 2021 20:44:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178479.324512; Fri, 03 Sep 2021 20:44:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG2i-0003NN-W7; Fri, 03 Sep 2021 20:44:08 +0000
Received: by outflank-mailman (input) for mailman id 178479;
 Fri, 03 Sep 2021 20:44:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG2h-0003N3-PF
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:44:07 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f1d3555d-3435-4062-8bcf-9affcb3d7dc5;
 Fri, 03 Sep 2021 20:44:07 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id q11so381219wrr.9
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:44:07 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id h16sm264457wre.52.2021.09.03.13.44.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:44:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1d3555d-3435-4062-8bcf-9affcb3d7dc5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=kHY8X+3WAxrdeSKfSV5cKvDPcFHMSr1zqsSabCFDuj4=;
        b=y2JE2znGpCHgDzsU2I9wK0RJ8GE2JJWPW9THeKM+zR9sq8qQv438t3kOx206Jt2Fg9
         HCBj4K5J9LrMW8VCOyReTekXa1iDDwX12LjR7w51Hp6FgTpMfV7mK57QPRaKgTN+mKSM
         c4ydaiundmZ/zqYpEcQ7fTBQRpS/g1xvCPjoSorsUTdNxikH8pbFeAJIqDRqkEeBcdOc
         o6bXO2ZIp6ZGu2T9H1FE0GxSedCTSSR2B/P/X59CRLG9yK22Gt+PLAznez/m/7vmSLM1
         nNLeBSmIfDEWuEX2lL68vLI9//wt72neSXBsIxn05T13a5hBYl9JWV1zFbdtz3FU6mYb
         fpgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=kHY8X+3WAxrdeSKfSV5cKvDPcFHMSr1zqsSabCFDuj4=;
        b=tnF7QaEAcubsueSkXIe4fv9uWUBpEY9UmErJYaO2zYb8OWuj+KDsHuQXmii6QrV9/p
         JpXpzn8P2prkRKJt+QoiOcHt7/iX05bkPf0l3QGV0A0UmRTYuAQoubLtlUMfljFLMZcN
         p2E76z1u7IsjdKawthLGN207k6hvesVwPtM3yjohl3VqBTR3mW0ZH/At6an52bvWNdli
         gIbtN5JOcbdd3ZLD/UnA01E+sA5YHttbMpzwtuNJ35HsVa91pwSYc+O6j8+yU57rb1+7
         NKZ69eXjPzE3c8+FYFAqvz8JDishDve3K4NkqQyBHPBe3isnroJu3cU5nOahUda/k1ut
         eU7g==
X-Gm-Message-State: AOAM5301doh9a0MYtX0WEU7xIN0vtznE2nJ5tUvJqbjYzy/isU8LqdBl
	Q5nw23gDRcQy+mwGADZJEjekDQ==
X-Google-Smtp-Source: ABdhPJwF66mC8rAdAxTJMbRm7Tnb/TmgI1HpOm8MMDv09XxJRL4qVfHzOz4F+luA4Rwup1L4RWjtvA==
X-Received: by 2002:a5d:6ac7:: with SMTP id u7mr889163wrw.390.1630701846250;
        Fri, 03 Sep 2021 13:44:06 -0700 (PDT)
Subject: Re: [PATCH v3 20/30] target/ppc: Restrict has_work() handler to
 sysemu and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-21-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <860bd94b-89ce-3aa1-4c8b-b53e4d83b80f@linaro.org>
Date: Fri, 3 Sep 2021 22:43:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-21-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   target/ppc/cpu_init.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:44:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:44:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178486.324523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG39-0003y4-Bd; Fri, 03 Sep 2021 20:44:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178486.324523; Fri, 03 Sep 2021 20:44:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG39-0003xx-8b; Fri, 03 Sep 2021 20:44:35 +0000
Received: by outflank-mailman (input) for mailman id 178486;
 Fri, 03 Sep 2021 20:44:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG37-0003vk-PE
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:44:33 +0000
Received: from mail-wr1-x430.google.com (unknown [2a00:1450:4864:20::430])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f17a0e54-f5d3-418e-ad55-3ab4e9f433c2;
 Fri, 03 Sep 2021 20:44:30 +0000 (UTC)
Received: by mail-wr1-x430.google.com with SMTP id g18so370577wrc.11
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:44:30 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id c24sm280794wrb.57.2021.09.03.13.44.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:44:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f17a0e54-f5d3-418e-ad55-3ab4e9f433c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=KQ6VNEDT2RTSrtudl0wF85JMAcGQkDFdqJ9dexlByDI=;
        b=j0YbQMDCCUEbh8WfA/4C6tjTCZz5YVgUlnkvm7vBt137y8oCJJCgceYmnd1W45xDsj
         fH2oliFfUD9c9XqutLEe1u5GTFJ6NAM0xeLY5W7fHmz3N4JrZIxA65d+kOsL2+ohTMMZ
         pz2NWVSauNgwHAR9TfxgbMUUxFdzWPsbV7R8NCfsWMr3X1UEPkWtchjxP20wvoSCi3RN
         2F0iPe5rDeOWIgYfpiwvyG9SdVoyXltSrkbqSEyUhJSapGdh1kKfq26I6utdBnwZlpAN
         hMZ0E1VMPwkZiA/nmt6T74JUpWM/2vzkg/YXgRLLDqbAk0z+L4XVz0pmGyj5wrmMUf7O
         sEtA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=KQ6VNEDT2RTSrtudl0wF85JMAcGQkDFdqJ9dexlByDI=;
        b=puG1glLUZzKB8V3FIux5L6bUK8HHumPi9yZ8qzNMK9flFQSP6UIFOlbLX+fs0ljBek
         QGoew9tq2IsLIIn4RU/5V+YflecudxjS5gMf10K9wyAA16WMCen9RhwZfXA/Wv6/IKZc
         IfMXNWCE2OJse24oD0GUYpMgOOoCo6u4W/I/R3uq7Wxh3ON7H7WUfEkyyaykPXFjOPIj
         O4MoKQvAxZ2oT31nUOtPYhmZeNmp2h1AY6wlG47MlO6awvAs71AhDAIjX4drfYaCyuAi
         3VE7fTUnN0YAmggDwNg0PCwgmmvWEaEmINo2MGQDQaOHUlJKnYkZQxqm2xTkmv30haDn
         ir3g==
X-Gm-Message-State: AOAM5334zDmhmaeSQZBgD+WRZ4w8o3VJfekJmhfKId0QChVz2B6atj5n
	mK16K1nwPyUeyFbf/Ig9IJQjTw==
X-Google-Smtp-Source: ABdhPJz3eHG2XBUgqkGosMAvUqwA6yjsJz1vhZ/AXgPH+VG7PO6AKFhg7sg0B/HYBLrKSYyiTUNh5w==
X-Received: by 2002:adf:b789:: with SMTP id s9mr857553wre.315.1630701869461;
        Fri, 03 Sep 2021 13:44:29 -0700 (PDT)
Subject: Re: [PATCH v3 24/30] target/rx: Restrict has_work() handler to sysemu
 and TCG
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Bin Meng <bin.meng@windriver.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, haxm-team@intel.com,
 Kamil Rytarowski <kamil@netbsd.org>, qemu-ppc@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Michael Rolnik <mrolnik@gmail.com>, qemu-riscv@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Thomas Huth <thuth@redhat.com>, David Hildenbrand <david@redhat.com>,
 Chris Wulff <crwulff@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, qemu-s390x@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Artyom Tarasenko <atar4qemu@gmail.com>, Aurelien Jarno
 <aurelien@aurel32.net>, Paul Durrant <paul@xen.org>,
 Peter Maydell <peter.maydell@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>,
 Alistair Francis <alistair.francis@wdc.com>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Roman Bolshakov <r.bolshakov@yadro.com>, Laurent Vivier <laurent@vivier.eu>,
 Cornelia Huck <cohuck@redhat.com>, qemu-arm@nongnu.org,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Colin Xu <colin.xu@intel.com>, Claudio Fontana <cfontana@suse.de>,
 Palmer Dabbelt <palmer@dabbelt.com>, Stafford Horne <shorne@gmail.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, kvm@vger.kernel.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-25-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <5f0e217b-2580-49b5-5cee-61d8ab41d706@linaro.org>
Date: Fri, 3 Sep 2021 22:44:22 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210902161543.417092-25-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/2/21 6:15 PM, Philippe Mathieu-Daudé wrote:
> Restrict has_work() to TCG sysemu.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   target/rx/cpu.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/target/rx/cpu.c b/target/rx/cpu.c
> index 25a4aa2976d..0d0cf6f9028 100644
> --- a/target/rx/cpu.c
> +++ b/target/rx/cpu.c
> @@ -41,11 +41,13 @@ static void rx_cpu_synchronize_from_tb(CPUState *cs,
>       cpu->env.pc = tb->pc;
>   }
>   
> +#if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
>   static bool rx_cpu_has_work(CPUState *cs)

No CONFIG_TCG, otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 20:48:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 20:48:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178494.324534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG74-0004pF-22; Fri, 03 Sep 2021 20:48:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178494.324534; Fri, 03 Sep 2021 20:48:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMG73-0004p8-Uc; Fri, 03 Sep 2021 20:48:37 +0000
Received: by outflank-mailman (input) for mailman id 178494;
 Fri, 03 Sep 2021 20:48:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lnve=NZ=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mMG71-0004ox-Uy
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 20:48:36 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cbd8ebd2-f9d7-4f70-b376-9c2795757b2f;
 Fri, 03 Sep 2021 20:48:34 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id x6so372036wrv.13
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 13:48:34 -0700 (PDT)
Received: from [192.168.8.107] (190.red-2-142-216.dynamicip.rima-tde.net.
 [2.142.216.190])
 by smtp.gmail.com with ESMTPSA id t23sm273507wrb.71.2021.09.03.13.48.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 13:48:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbd8ebd2-f9d7-4f70-b376-9c2795757b2f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=JR2N3hW+DAAOOkmrYLuui81slBAvGFUWkGnERbEB/4A=;
        b=oIoi1f5ZWFOc0RmycwGdPB1omD0xRAxTNPvzzN59dsRqEHrjnFus9NLUNjhn6i17pj
         ivO2uKGYQ/5srtBFw7S7dxLFCPr0Dg84Ff6W1/9vJTqoS6LzG+xh4J2hCztqivPqnava
         u/7bE8JnlgID+82FLwTwyyj/Qz3U6Qik1/Tf5wLbcyLfdBYFCdE7+xNUeGNupoQceOtK
         DuiPtlLD47X/8EiY6wq+LG78nSM5X+oluNwRPdUvyRQTAk0pPxYuJQ+0FSqNABy+klPa
         /Gw/OIgSOH/QuL9vGuSTxQAkWanGr1FY2e4K3sb0MB5auKQ+KdAP6aCeWBFl5rG2tuai
         4CPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=JR2N3hW+DAAOOkmrYLuui81slBAvGFUWkGnERbEB/4A=;
        b=R4VRmijr0Yo1qLLJ+affaxnJgfi6/TS8eNggVwo2P2usLW2mAd9Re8ol84KRTjFA5V
         uQyo9aZ9CPBdVRwFN/5t/nPRRWVidA9k1sFKPUr2YkmmS2C56Jd8xxtq3noEyCFjEL4w
         tvDTNZaOGXIlDgGbSCuEC4MM3FhAJmQytFTUvfWbFenkZ63UxpXxLnasHB95ESA/bWcu
         FzOy9BtMws46TEruYQqtATHJVcArlbxzAIYqiuXryf6QOo1GSX4W7KbSFQVPu0uwejEu
         Vb1+fyu/O52J0aMy7C/zg7OHuV9OlR/uToJLhI2OTltd/OooXRXvh1jXrp5aSRp/qp91
         wrVA==
X-Gm-Message-State: AOAM533mF7xGV4iAT81N1LiQIwe6oiDubNQ8+PWrxxFdUmengaGTpEgp
	UsviioydSIKTAO+UYh0KZSRrPoZRGyF8cPxyLXU=
X-Google-Smtp-Source: ABdhPJwYxL1wBdNoflELnyXgoIfkyp1X+C47Clew7iV3DpVZ/QyfLSQUi3F7MAdXmgdYugc/uDUBHg==
X-Received: by 2002:adf:e6c5:: with SMTP id y5mr908323wrm.198.1630702113942;
        Fri, 03 Sep 2021 13:48:33 -0700 (PDT)
Subject: Re: [PATCH v3 30/30] accel: Add missing AccelOpsClass::has_work() and
 drop SysemuCPUOps one
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Anthony Perard <anthony.perard@citrix.com>,
 Cameron Esfahani <dirty@apple.com>, Claudio Fontana <cfontana@suse.de>,
 Colin Xu <colin.xu@intel.com>, Eduardo Habkost <ehabkost@redhat.com>,
 haxm-team@intel.com, Kamil Rytarowski <kamil@netbsd.org>,
 Laurent Vivier <lvivier@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 Paul Durrant <paul@xen.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Roman Bolshakov <r.bolshakov@yadro.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Thomas Huth <thuth@redhat.com>,
 Wenchao Wang <wenchao.wang@intel.com>, xen-devel@lists.xenproject.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210903181943.763360-1-f4bug@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <5b1a24c2-cdd6-ec88-d0d8-540bf5f96c38@linaro.org>
Date: Fri, 3 Sep 2021 22:48:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210903181943.763360-1-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 8:19 PM, Philippe Mathieu-Daudé wrote:
> cpu_common_has_work() is the default has_work() implementation
> and returns 'false'.
> 
> Explicit it for the QTest / HAX / HVF / NVMM / Xen accelerators
> and remove cpu_common_has_work().
> 
> Since there are no more implementations of SysemuCPUOps::has_work,
> remove it along with the assertion in cpu_has_work().
> 
> Signed-off-by: Philippe Mathieu-Daudé<f4bug@amsat.org>
> ---
>   include/hw/core/cpu.h             | 2 --
>   accel/hvf/hvf-accel-ops.c         | 6 ++++++
>   accel/qtest/qtest.c               | 6 ++++++
>   accel/xen/xen-all.c               | 6 ++++++
>   hw/core/cpu-common.c              | 6 ------
>   softmmu/cpus.c                    | 9 ++-------
>   target/i386/hax/hax-accel-ops.c   | 6 ++++++
>   target/i386/nvmm/nvmm-accel-ops.c | 6 ++++++
>   8 files changed, 32 insertions(+), 15 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 21:11:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 21:11:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178503.324545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMGTX-0008Vt-Ta; Fri, 03 Sep 2021 21:11:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178503.324545; Fri, 03 Sep 2021 21:11:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMGTX-0008Vm-QI; Fri, 03 Sep 2021 21:11:51 +0000
Received: by outflank-mailman (input) for mailman id 178503;
 Fri, 03 Sep 2021 21:11:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wQk4=NZ=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mMGTX-0008Vg-0W
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 21:11:51 +0000
Received: from mail-wr1-x434.google.com (unknown [2a00:1450:4864:20::434])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bd593490-6877-4ccb-b84e-dcd243829007;
 Fri, 03 Sep 2021 21:11:50 +0000 (UTC)
Received: by mail-wr1-x434.google.com with SMTP id b6so457037wrh.10
 for <xen-devel@lists.xenproject.org>; Fri, 03 Sep 2021 14:11:50 -0700 (PDT)
Received: from [192.168.1.36] (21.red-83-52-55.dynamicip.rima-tde.net.
 [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id s205sm411532wme.4.2021.09.03.14.11.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 03 Sep 2021 14:11:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: bd593490-6877-4ccb-b84e-dcd243829007
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=jI43SKJbesP12DNmL3/e0Usr50r2XgSNCcYV6kiyq6k=;
        b=XH5LRShQJoZBhky3eh03Pmb9SkQ32XBteRsyogRiPstHFr6orcBCJeqTh6+8uA9Ugj
         hwUHRkbv3Iki29kH6SULZWWaPVeHLm9dOZGMJp/KvL+iV9qeA3IsgO7aPDi6dPOxdM5Q
         ws2BSkCdqzZ3QGSkRU4XFRPZC9C8RO5eAPwO0xHZyZ51hswyi5ZlgnIK0hazE40GI5Vy
         pDzx5Z2/NQxO+YOF01K7Wot3MRs8tFcfTdIvEFWt3PL9aHwjDLKgNplb2i8xQwHyzDKi
         JVFun0OPCzM5tItFSKRVH8Yfw5eizrEfOkYRFD8WnJK9txMG72i4kstdXmKL8/qxt3TH
         7oMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
         :date:user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=jI43SKJbesP12DNmL3/e0Usr50r2XgSNCcYV6kiyq6k=;
        b=QaLqBsoiTZipwKiltx1WkeslU9Yq6UFpgzyQKEmB+fEoCphbc271rGkjP2E4uNqdTo
         vSfOiEFzaBOTrc31aM/n66Q7+Nmu68ezfX9SoK0IPnovXrs8h/5o7D2GA87TBNTmXP+f
         1Npht6E1mnD0c0PNNzGtg0WeWcMT/NSLFiEpM6QRw4FXFvmNhBB2lBS5yekOaTKOTOzH
         LwAdxNCs1Lj1L5Bv6O59FfKGU7byDaUkfvBPehqZz+VxmHbgZc+WycBzBi8RVXgFmGZm
         RSWfh0RZKj0swin51jmJQCrUzs62UghacP4hicwY4Fg2wXnh1fVgaWGL67wJkRLD33kI
         y11w==
X-Gm-Message-State: AOAM5314DauI7bUhWK3I/wPgPN1hoDv7LxLkE0kxhgHIw3bxLbLARYPF
	8jv5tukkhhtsr6fsF9sthfU=
X-Google-Smtp-Source: ABdhPJx9z+1i4IwrqpeoM/m67lAEAeVLF6XhWOMCvs9W+7ge742xu6/mLFLJYvAuNvjtGUuedCA5Yg==
X-Received: by 2002:a5d:40d1:: with SMTP id b17mr974069wrq.47.1630703509319;
        Fri, 03 Sep 2021 14:11:49 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
To: Richard Henderson <richard.henderson@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>, =?UTF-8?Q?C=c3=a9dric_Le_Goater?=
 <clg@kaod.org>, Greg Kurz <groug@kaod.org>
Cc: Peter Maydell <peter.maydell@linaro.org>,
 Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, qemu-devel@nongnu.org,
 Jiaxun Yang <jiaxun.yang@flygoat.com>, Laurent Vivier <laurent@vivier.eu>,
 Max Filippov <jcmvbkbc@gmail.com>, Taylor Simpson <tsimpson@quicinc.com>,
 haxm-team@intel.com, Colin Xu <colin.xu@intel.com>,
 Stafford Horne <shorne@gmail.com>, Marek Vasut <marex@denx.de>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Greg Kurz <groug@kaod.org>, Cameron Esfahani <dirty@apple.com>,
 qemu-s390x@nongnu.org, qemu-arm@nongnu.org,
 Michael Rolnik <mrolnik@gmail.com>, Sunil Muthuswamy
 <sunilmut@microsoft.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 qemu-riscv@nongnu.org, Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Chris Wulff <crwulff@gmail.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 qemu-ppc@nongnu.org, Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Paolo Bonzini <pbonzini@redhat.com>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org> <YTFxZb1Vg5pWVW9p@yekko>
 <fd383a02-fb9f-8641-937f-ebe1d8bb065f@linaro.org>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>
Message-ID: <fc98e293-f2ba-8ca0-99c8-f07758b79d73@amsat.org>
Date: Fri, 3 Sep 2021 23:11:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <fd383a02-fb9f-8641-937f-ebe1d8bb065f@linaro.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 10:42 PM, Richard Henderson wrote:
> On 9/3/21 2:50 AM, David Gibson wrote:
>> On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daudé wrote:
>>> Each POWER cpu has its own has_work() implementation. Instead of
>>> overloading CPUClass on each PowerPCCPUClass init, register the
>>> generic ppc_cpu_has_work() handler, and have it call the POWER
>>> specific has_work().
>>
>> I don't quite see the rationale for introducing a second layer of
>> indirection here.  What's wrong with switching the base has_work for
>> each cpu variant?
> 
> We're moving the hook from CPUState to TCGCPUOps.
> Phil was trying to avoid creating N versions of
> 
> static const struct TCGCPUOps ppc_tcg_ops = {
>     ...
> };

Ah yes this is the reason! Too many context switching so
I forgot about it.

> A plausible alternative is to remove the const from this struct and
> modify it, just as we do for CPUState, on the assumption that we cannot
> mix and match ppc cpu types in any one machine.

I thought about this case and remembered how it works on the ARM arch,
i.e. ZynqMP machine uses both Cortex-R5F and Cortex-A53. Even if no
similar PPC machine exists, IMHO we should try to generally allow to
possibility to experiment machine with different CPUs. Restricting it
on PPC goes the other way around. Thoughts?


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 21:24:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 21:24:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178509.324556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMGg2-0001mV-4F; Fri, 03 Sep 2021 21:24:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178509.324556; Fri, 03 Sep 2021 21:24:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMGg2-0001mO-0T; Fri, 03 Sep 2021 21:24:46 +0000
Received: by outflank-mailman (input) for mailman id 178509;
 Fri, 03 Sep 2021 21:24:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMGg1-0001mE-4n; Fri, 03 Sep 2021 21:24:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMGg0-0005Wh-To; Fri, 03 Sep 2021 21:24:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMGg0-00056A-Ij; Fri, 03 Sep 2021 21:24:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMGg0-0007C3-Ht; Fri, 03 Sep 2021 21:24:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tiBMzIEwgeeuAYbfzs9FxQCxkvjLao1jbvXSqezZC2s=; b=sXGHIOkIKVwMBdEVYmM03H9ykc
	wPoqjPOGtQjqH3ugxSa6bOr7yNKL4JdvgQATV1k+hBxH4Zjlaq1eShyLzoX/rSe+HNh23m69OxgZP
	qwy5DesM8JEi9vGJVmwXy9xenscLMeRVn/JmTsPIKOJYI2iW1lOd/S25Tn/ghYRI0fsw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164793-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 164793: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924
X-Osstest-Versions-That:
    linux=fd80923202c6bfd723742fc32426a7aa3632abaa
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 03 Sep 2021 21:24:44 +0000

flight 164793 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164793/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164501
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164501
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164501
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164501
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164501
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164501
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164501
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164501
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164501
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164501
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164501
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924
baseline version:
 linux                fd80923202c6bfd723742fc32426a7aa3632abaa

Last test of basis   164501  2021-08-26 13:42:10 Z    8 days
Testing same since   164793  2021-09-03 08:40:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adrian Hunter <adrian.hunter@intel.com>
  Anand Jain <anand.jain@oracle.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Ignatov <rdna@fb.com>
  Andrii Nakryiko <andriin@fb.com>
  Ariel Elior <aelior@marvell.com>
  Ben Skeggs <bskeggs@redhat.com>
  Bjorn Andersson <bjorn.andersson@linaro.org>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Colin Ian King <colin.king@canonical.com>
  Daniel Borkmann <daniel@iogearbox.net>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  DENG Qingfang <dqfext@gmail.com>
  Denis Efremov <efremov@linux.com>
  Dvora Fuxbrumer <dvorax.fuxbrumer@linux.intel.com>
  Felipe Balbi <balbi@kernel.org>
  Filipe Manana <fdmanana@suse.com>
  Florian Fainelli <f.fainelli@gmail.com>
  Florian Westphal <fw@strlen.de>
  Gal Pressman <galpress@amazon.com>
  Gerd Rausch <gerd.rausch@oracle.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guangbin Huang <huangguangbin2@huawei.com>
  Guenter Roeck <linux@roeck-us.net>
  Guojia Liao <liaoguojia@huawei.com>
  Hannes Frederic Sowa <hannes@stressinduktion.org>
  Helge Deller <deller@gmx.de>
  Hulk Robot <hulkrobot@huawei.com>
  Ingo Molnar <mingo@kernel.org>
  Jakub Kicinski <kuba@kernel.org>
  Jason Gunthorpe <jgg@nvidia.com>
  Jason Wang <jasowang@redhat.com>
  Jerome Brunet <jbrunet@baylibre.com>
  Johan Hovold <johan@kernel.org>
  John Harrison <John.C.Harrison@Intel.com>
  Jon Hunter <jonathanh@nvidia.com>
  Kefeng Wang <wangkefeng.wang@huawei.com>
  Li Jinlin <lijinlin3@huawei.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Marc Kleine-Budde <mkl@pengutronix.de>
  Mark Yacoub <markyacoub@chromium.org>
  Mark Yacoub <markyacoub@google.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Martin KaFai Lau <kafai@fb.com>
  Matthew Brost <matthew.brost@intel.com>
  Maxim Kiselev <bigunclemax@gmail.com>
  Michael S. Tsirkin <mst@redhat.com>
  Michal Hocko <mhocko@suse.com>
  Michał Mirosław <mirq-linux@rere.qmqm.pl>
  Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
  Miklos Szeredi <mszeredi@redhat.com>
  Naresh Kamboju <naresh.kamboju@linaro.org>
  Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>
  Neeraj Upadhyay <neeraju@codeaurora.org>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Bonzini <pbonzini@redhat.com>
  Parav Pandit <parav@nvidia.com>
  Paul Moore <paul@paul-moore.com>
  Peter Collingbourne <pcc@google.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Petr Vorel <petr.vorel@gmail.com>
  Qiu Laibin <qiulaibin@huawei.com>
  Qu Wenruo <wqu@suse.com>
  Rafael David Tinoco <rafaeldtinoco@gmail.com>
  Richard Guy Briggs <rgb@redhat.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Salil Mehta <salil.mehta@huawei.com>
  Santosh Shilimkar <santosh.shilimkar@oracle.com>
  Sasha Levin <sashal@kernel.org>
  Sasha Neftin <sasha.neftin@intel.com>
  Sean Christopherson <seanjc@google.com>
  Sean Paul <seanpaul@chromium.org>
  Selvin Xavier <selvin.xavier@broadcom.com>
  Shai Malin <smalin@marvell.com>
  Shaik Sajida Bhanu <sbhanu@codeaurora.org>
  Shreyansh Chouhan <chouhan.shreyansh630@gmail.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Stefan Mätje <stefan.maetje@esd.eu>
  Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
  Sudip Mukherjee <sudipm.mukherjee@gmail.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Tuo Li <islituo@gmail.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Vineet Gupta <vgupta@synopsys.com>
  Viresh Kumar <viresh.kumar@linaro.org>
  Wesley Cheng <wcheng@codeaurora.org>
  Xiaolong Huang <butterflyhuangxx@gmail.com>
  Yafang Shao <laoar.shao@gmail.com>
  Yufeng Mo <moyufeng@huawei.com>
  Zhengjun Zhang <zhangzhengjun@aicrobo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   fd80923202c6..c6bf0ed9d1a7  c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924 -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:48:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:48:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178519.324581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHyq-0002rF-UN; Fri, 03 Sep 2021 22:48:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178519.324581; Fri, 03 Sep 2021 22:48:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHyq-0002r6-R7; Fri, 03 Sep 2021 22:48:16 +0000
Received: by outflank-mailman (input) for mailman id 178519;
 Fri, 03 Sep 2021 22:48:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMHyp-0002qd-Dr
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:48:15 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ce82eaaf-8eab-471f-a37c-8c963c90fb6d;
 Fri, 03 Sep 2021 22:48:14 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2E3A461056;
 Fri,  3 Sep 2021 22:48:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce82eaaf-8eab-471f-a37c-8c963c90fb6d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709293;
	bh=D3TKVtXRF0k7je6UxI5S91fhLgoCCJ2RQ6UxqDcfnA0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=SkypW8AB/LVsQX17U1lF7SVw119y3enF4tgUOj0R9Ry6KSegHjQD8hwpTTFwdOaUU
	 un4idD6fP73zahznH66T7Ttc4KYexYORWAREKHoUinVaBaTD4N6s9pzlHv0o+q6naS
	 DT7wsA/SfsTq6uAShajW2T/029Oad7O+x53oVLdUBPJvKFtlbxel2L8K96MZRNzrbl
	 ei1etvyU4SHPo06FUYuSPu5RUTzFqgXmoLRsuFEhIJhTuPvJix6yJRIrcynrCZEWXP
	 Wx4yteW+qD7zfaNDvymFB2LbgpiAb784I8xB7Jy3St2jYZDjgJ1xBiE6XI/0CqiNHU
	 0JF7uH5/SSwHw==
Date: Fri, 3 Sep 2021 15:48:12 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo
In-Reply-To: <588f4cdd02ce3f907c4b97b1513d90f153d127c0.1629897306.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109031419130.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <588f4cdd02ce3f907c4b97b1513d90f153d127c0.1629897306.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> As we will sanitize the content of boot_cpu_data it will not really
> contain the boot cpu information but the system sanitize information.
> Rename the structure to system_cpuinfo so the user is informed that this
> is the system wide available feature and not anymore the features of the
> boot cpu.
> The original boot cpu data is still available in cpu_data.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v3: none
> Changes in v2:
>   - patch introduced in v2
> ---
>  xen/arch/arm/cpufeature.c        |  8 ++------
>  xen/arch/arm/setup.c             | 34 ++++++++++++++++++--------------
>  xen/arch/arm/smpboot.c           |  6 +++---
>  xen/include/asm-arm/cpufeature.h |  6 +++---
>  4 files changed, 27 insertions(+), 27 deletions(-)
> 
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 1d88783809..f600a611bd 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -169,12 +169,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>   */
>  static int __init create_guest_cpuinfo(void)
>  {
> -    /*
> -     * TODO: The code is currently using only the features detected on the boot
> -     * core. In the long term we should try to compute values containing only
> -     * features supported by all cores.
> -     */
> -    guest_cpuinfo = boot_cpu_data;
> +    /* Use the sanitized cpuinfo as initial guest cpuinfo */
> +    guest_cpuinfo = system_cpuinfo;
>  
>  #ifdef CONFIG_ARM_64
>      /* Hide MPAM support as xen does not support it */
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 63a908e325..3798c5ade0 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -56,7 +56,11 @@
>  
>  struct bootinfo __initdata bootinfo;
>  
> -struct cpuinfo_arm __read_mostly boot_cpu_data;
> +/*
> + * Sanitized version of cpuinfo containing only features available on all
> + * cores (only on arm64 as there is no sanitization support on arm32).
> + */
> +struct cpuinfo_arm __read_mostly system_cpuinfo;
>  
>  #ifdef CONFIG_ACPI
>  bool __read_mostly acpi_disabled;
> @@ -100,7 +104,7 @@ static const char * __initdata processor_implementers[] = {
>  static void __init processor_id(void)
>  {
>      const char *implementer = "Unknown";
> -    struct cpuinfo_arm *c = &boot_cpu_data;
> +    struct cpuinfo_arm *c = &system_cpuinfo;
>  
>      identify_cpu(c);
>      current_cpu_data = *c;
> @@ -120,7 +124,7 @@ static void __init processor_id(void)
>  #if defined(CONFIG_ARM_64)
>      printk("64-bit Execution:\n");
>      printk("  Processor Features: %016"PRIx64" %016"PRIx64"\n",
> -           boot_cpu_data.pfr64.bits[0], boot_cpu_data.pfr64.bits[1]);
> +           system_cpuinfo.pfr64.bits[0], system_cpuinfo.pfr64.bits[1]);
>      printk("    Exception Levels: EL3:%s EL2:%s EL1:%s EL0:%s\n",
>             cpu_has_el3_32 ? "64+32" : cpu_has_el3_64 ? "64" : "No",
>             cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
> @@ -144,13 +148,13 @@ static void __init processor_id(void)
>                 boot_cpu_feature64(simd));
>  
>      printk("  Debug Features: %016"PRIx64" %016"PRIx64"\n",
> -           boot_cpu_data.dbg64.bits[0], boot_cpu_data.dbg64.bits[1]);
> +           system_cpuinfo.dbg64.bits[0], system_cpuinfo.dbg64.bits[1]);
>      printk("  Auxiliary Features: %016"PRIx64" %016"PRIx64"\n",
> -           boot_cpu_data.aux64.bits[0], boot_cpu_data.aux64.bits[1]);
> +           system_cpuinfo.aux64.bits[0], system_cpuinfo.aux64.bits[1]);
>      printk("  Memory Model Features: %016"PRIx64" %016"PRIx64"\n",
> -           boot_cpu_data.mm64.bits[0], boot_cpu_data.mm64.bits[1]);
> +           system_cpuinfo.mm64.bits[0], system_cpuinfo.mm64.bits[1]);
>      printk("  ISA Features:  %016"PRIx64" %016"PRIx64"\n",
> -           boot_cpu_data.isa64.bits[0], boot_cpu_data.isa64.bits[1]);
> +           system_cpuinfo.isa64.bits[0], system_cpuinfo.isa64.bits[1]);
>  #endif
>  
>      /*
> @@ -161,7 +165,7 @@ static void __init processor_id(void)
>      {
>          printk("32-bit Execution:\n");
>          printk("  Processor Features: %"PRIregister":%"PRIregister"\n",
> -               boot_cpu_data.pfr32.bits[0], boot_cpu_data.pfr32.bits[1]);
> +               system_cpuinfo.pfr32.bits[0], system_cpuinfo.pfr32.bits[1]);
>          printk("    Instruction Sets:%s%s%s%s%s%s\n",
>                 cpu_has_aarch32 ? " AArch32" : "",
>                 cpu_has_arm ? " A32" : "",
> @@ -174,18 +178,18 @@ static void __init processor_id(void)
>                 cpu_has_security ? " Security" : "");
>  
>          printk("  Debug Features: %"PRIregister"\n",
> -               boot_cpu_data.dbg32.bits[0]);
> +               system_cpuinfo.dbg32.bits[0]);
>          printk("  Auxiliary Features: %"PRIregister"\n",
> -               boot_cpu_data.aux32.bits[0]);
> +               system_cpuinfo.aux32.bits[0]);
>          printk("  Memory Model Features: %"PRIregister" %"PRIregister"\n"
>                 "                         %"PRIregister" %"PRIregister"\n",
> -               boot_cpu_data.mm32.bits[0], boot_cpu_data.mm32.bits[1],
> -               boot_cpu_data.mm32.bits[2], boot_cpu_data.mm32.bits[3]);
> +               system_cpuinfo.mm32.bits[0], system_cpuinfo.mm32.bits[1],
> +               system_cpuinfo.mm32.bits[2], system_cpuinfo.mm32.bits[3]);
>          printk("  ISA Features: %"PRIregister" %"PRIregister" %"PRIregister"\n"
>                 "                %"PRIregister" %"PRIregister" %"PRIregister"\n",
> -               boot_cpu_data.isa32.bits[0], boot_cpu_data.isa32.bits[1],
> -               boot_cpu_data.isa32.bits[2], boot_cpu_data.isa32.bits[3],
> -               boot_cpu_data.isa32.bits[4], boot_cpu_data.isa32.bits[5]);
> +               system_cpuinfo.isa32.bits[0], system_cpuinfo.isa32.bits[1],
> +               system_cpuinfo.isa32.bits[2], system_cpuinfo.isa32.bits[3],
> +               system_cpuinfo.isa32.bits[4], system_cpuinfo.isa32.bits[5]);
>      }
>      else
>      {
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a1ee3146ef..c9f2827d56 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -124,7 +124,7 @@ static void __init dt_smp_init_cpus(void)
>      bool bootcpu_valid = false;
>      int rc;
>  
> -    mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK;
> +    mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
>  
>      if ( !cpus )
>      {
> @@ -319,13 +319,13 @@ void start_secondary(void)
>       * now.
>       */
>      if ( !opt_hmp_unsafe &&
> -         current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
> +         current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
>      {
>          printk(XENLOG_ERR
>                 "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
>                 XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
>                 smp_processor_id(), current_cpu_data.midr.bits,
> -               boot_cpu_data.midr.bits);
> +               system_cpuinfo.midr.bits);
>          stop_cpu();
>      }
>  
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index ba48db3eac..8f2b8e7830 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -3,7 +3,7 @@
>  
>  #ifdef CONFIG_ARM_64
>  #define cpu_feature64(c, feat)         ((c)->pfr64.feat)
> -#define boot_cpu_feature64(feat)       (boot_cpu_data.pfr64.feat)
> +#define boot_cpu_feature64(feat)       (system_cpuinfo.pfr64.feat)
>  
>  #define cpu_feature64_has_el0_32(c)    (cpu_feature64(c, el0) == 2)
>  
> @@ -21,7 +21,7 @@
>  #endif
>  
>  #define cpu_feature32(c, feat)         ((c)->pfr32.feat)
> -#define boot_cpu_feature32(feat)       (boot_cpu_data.pfr32.feat)
> +#define boot_cpu_feature32(feat)       (system_cpuinfo.pfr32.feat)
>  
>  #define cpu_has_arm       (boot_cpu_feature32(arm) == 1)
>  #define cpu_has_thumb     (boot_cpu_feature32(thumb) >= 1)
> @@ -326,7 +326,7 @@ struct cpuinfo_arm {
>      } mvfr;
>  };
>  
> -extern struct cpuinfo_arm boot_cpu_data;
> +extern struct cpuinfo_arm system_cpuinfo;
>  
>  extern void identify_cpu(struct cpuinfo_arm *);
>  
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:48:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:48:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178518.324570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHym-0002aR-HY; Fri, 03 Sep 2021 22:48:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178518.324570; Fri, 03 Sep 2021 22:48:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHym-0002aK-Dx; Fri, 03 Sep 2021 22:48:12 +0000
Received: by outflank-mailman (input) for mailman id 178518;
 Fri, 03 Sep 2021 22:48:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMHyk-0002aC-Rx
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:48:11 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 017ea770-0d09-11ec-af6a-12813bfff9fa;
 Fri, 03 Sep 2021 22:48:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BC8D360FA0;
 Fri,  3 Sep 2021 22:48:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 017ea770-0d09-11ec-af6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709286;
	bh=x4cpnhoGOcwYa/KDT85UxSjYeA5q7x+4pHkzioVIWUE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=omdgbZWN7MZn9RXb8n1wdxKN10lRQvX0T5sfAokOgcVw2KiZAS3DT85I9Q/LACyvU
	 RmE/GpyIFQ7ABm7HZVeo7sV7GybnTdZc1jDWSe1JYna3ubRYmOiZhdPWUx1nVP4Q07
	 GW3jLHMCmMzN8KA/LN2c62RaqVgKNIHCKvWIU7+SZycQV1ejsXnuok+rwQLhEYGKUj
	 vP35gH872y6lYXB5c88tbjRFxhrWNZYz1bqOaXPijY+rm0kstAiD7dwSmWO493bGMQ
	 Hl8bDpu7XRvOhkGCKrLLCx62dEG6euaOHHC2LuZM0ANnoQUkiL6nRWQ8pcSzSTzvSo
	 mjlNDrdGzdrJA==
Date: Fri, 3 Sep 2021 15:48:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 2/7] xen/arm: Import ID features sanitize from linux
In-Reply-To: <61de0ae1aaec35cb32f0ec2ead6ce9c4c6ef154e.1629897306.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109031408160.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <61de0ae1aaec35cb32f0ec2ead6ce9c4c6ef154e.1629897306.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Import structures declared in Linux file arch/arm64/kernel/cpufeature.c
> and the required types from arch/arm64/include/asm/cpufeature.h.
> 
> Current code has been imported from Linux 5.13-rc5 (Commit ID
> cd1245d75ce93b8fd206f4b34eb58bcfe156d5e9) and copied into cpufeature.c
> in arm64 code and cpufeature.h in arm64 specific headers.
> 
> Those structure will be used to sanitize the cpu features available to
> the ones availble on all cores of a system even if we are on an
> heterogeneous platform (from example a big/LITTLE).
> 
> For each feature field of all ID registers, those structures define what
> is the safest value and if we can allow to have different values in
> different cores.
> 
> This patch is introducing Linux code without any changes to it.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v3: none
> Changes in v2:
> - Move add to Makefile to following patch to allow bisection
> - Remove GPL text as SPDL is there
> - Re-add introduction comment from Linux Kernel file
> - Rename cpusanitize.c to cpufeature.c to keep Linux file name
> - Move structures imported from linux headers into a new cpufeature.h
> header in asm-arm/arm64.
> - Move comment about imported code origin to the file header
> - Remove not needed linux function declarations instead of removing them
> in the following patch
> - Add original arm64_ftr_safe_value from Linux
> - include kernel.h to use max()
> - remove unused ftr_single32 as we will not use it
> - remove ctr associated structures that we cannot use (keep the one
> defining sanitization bits)
> ---
>  xen/arch/arm/arm64/cpufeature.c        | 504 +++++++++++++++++++++++++
>  xen/include/asm-arm/arm64/cpufeature.h | 104 +++++
>  2 files changed, 608 insertions(+)
>  create mode 100644 xen/arch/arm/arm64/cpufeature.c
>  create mode 100644 xen/include/asm-arm/arm64/cpufeature.h
> 
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> new file mode 100644
> index 0000000000..5777e33e5c
> --- /dev/null
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -0,0 +1,504 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Contains CPU feature definitions
> + *
> + * The following structures have been imported directly from Linux kernel and
> + * should be kept in sync.
> + * The current version has been imported from arch/arm64/kernel/cpufeature.c
> + *  from kernel version 5.13-rc5 together with the required structures and
> + *  macros from arch/arm64/include/asm/cpufeature.h which are stored in
> + *  include/asm-arm/arm64/cpufeature.h
> + *
> + * Copyright (C) 2021 Arm Ltd.
> + * based on code from the Linux kernel, which is:
> + *  Copyright (C) 2015 ARM Ltd.
> + *
> + * A note for the weary kernel hacker: the code here is confusing and hard to
> + * follow! That's partly because it's solving a nasty problem, but also because
> + * there's a little bit of over-abstraction that tends to obscure what's going
> + * on behind a maze of helper functions and macros.
> + *
> + * The basic problem is that hardware folks have started gluing together CPUs
> + * with distinct architectural features; in some cases even creating SoCs where
> + * user-visible instructions are available only on a subset of the available
> + * cores. We try to address this by snapshotting the feature registers of the
> + * boot CPU and comparing these with the feature registers of each secondary
> + * CPU when bringing them up. If there is a mismatch, then we update the
> + * snapshot state to indicate the lowest-common denominator of the feature,
> + * known as the "safe" value. This snapshot state can be queried to view the
> + * "sanitised" value of a feature register.
> + *
> + * The sanitised register values are used to decide which capabilities we
> + * have in the system. These may be in the form of traditional "hwcaps"
> + * advertised to userspace or internal "cpucaps" which are used to configure
> + * things like alternative patching and static keys. While a feature mismatch
> + * may result in a TAINT_CPU_OUT_OF_SPEC kernel taint, a capability mismatch
> + * may prevent a CPU from being onlined at all.
> + *
> + * Some implementation details worth remembering:
> + *
> + * - Mismatched features are *always* sanitised to a "safe" value, which
> + *   usually indicates that the feature is not supported.
> + *
> + * - A mismatched feature marked with FTR_STRICT will cause a "SANITY CHECK"
> + *   warning when onlining an offending CPU and the kernel will be tainted
> + *   with TAINT_CPU_OUT_OF_SPEC.
> + *
> + * - Features marked as FTR_VISIBLE have their sanitised value visible to
> + *   userspace. FTR_VISIBLE features in registers that are only visible
> + *   to EL0 by trapping *must* have a corresponding HWCAP so that late
> + *   onlining of CPUs cannot lead to features disappearing at runtime.
> + *
> + * - A "feature" is typically a 4-bit register field. A "capability" is the
> + *   high-level description derived from the sanitised field value.
> + *
> + * - Read the Arm ARM (DDI 0487F.a) section D13.1.3 ("Principles of the ID
> + *   scheme for fields in ID registers") to understand when feature fields
> + *   may be signed or unsigned (FTR_SIGNED and FTR_UNSIGNED accordingly).
> + *
> + * - KVM exposes its own view of the feature registers to guest operating
> + *   systems regardless of FTR_VISIBLE. This is typically driven from the
> + *   sanitised register values to allow virtual CPUs to be migrated between
> + *   arbitrary physical CPUs, but some features not present on the host are
> + *   also advertised and emulated. Look at sys_reg_descs[] for the gory
> + *   details.
> + *
> + * - If the arm64_ftr_bits[] for a register has a missing field, then this
> + *   field is treated as STRICT RES0, including for read_sanitised_ftr_reg().
> + *   This is stronger than FTR_HIDDEN and can be used to hide features from
> + *   KVM guests.
> + */
> +
> +#include <xen/types.h>
> +#include <xen/kernel.h>
> +#include <asm/sysregs.h>
> +#include <asm/cpufeature.h>
> +#include <asm/arm64/cpufeature.h>
> +
> +#define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	{						\
> +		.sign = SIGNED,				\
> +		.visible = VISIBLE,			\
> +		.strict = STRICT,			\
> +		.type = TYPE,				\
> +		.shift = SHIFT,				\
> +		.width = WIDTH,				\
> +		.safe_val = SAFE_VAL,			\
> +	}
> +
> +/* Define a feature with unsigned values */
> +#define ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	__ARM64_FTR_BITS(FTR_UNSIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +/* Define a feature with a signed value */
> +#define S_ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	__ARM64_FTR_BITS(FTR_SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +#define ARM64_FTR_END					\
> +	{						\
> +		.width = 0,				\
> +	}
> +
> +/*
> + * NOTE: Any changes to the visibility of features should be kept in
> + * sync with the documentation of the CPU feature register ABI.
> + */
> +static const struct arm64_ftr_bits ftr_id_aa64isar0[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RNDR_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TLB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_FHM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_DP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM4_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RDM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_ATOMICS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_CRC32_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_AES_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DGH_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SPECRES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_API_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_APA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_MPAM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SEL2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +				   FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
> +	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
> +	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_SHIFT, 4, ID_AA64PFR0_EL1_64BIT_ONLY),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL0_SHIFT, 4, ID_AA64PFR0_EL0_64BIT_ONLY),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MPAMFRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_RASFRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_BTI),
> +				    FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_BT_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F64MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F32MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
> +	/*
> +	 * Page size not being supported at Stage-2 is not fatal. You
> +	 * just give up KVM if PAGE_SIZE isn't supported there. Go fix
> +	 * your favourite nesting hypervisor.
> +	 *
> +	 * There is a small corner case where the hypervisor explicitly
> +	 * advertises a given granule size at Stage-2 (value 2) on some
> +	 * vCPUs, and uses the fallback to Stage-1 (value 0) for other
> +	 * vCPUs. Although this is not forbidden by the architecture, it
> +	 * indicates that the hypervisor is being silly (or buggy).
> +	 *
> +	 * We make no effort to cope with this and pretend that if these
> +	 * fields are inconsistent across vCPUs, then it isn't worth
> +	 * trying to bring KVM up.
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN4_2_SHIFT, 4, 1),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN64_2_SHIFT, 4, 1),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN16_2_SHIFT, 4, 1),
> +	/*
> +	 * We already refuse to boot CPUs that don't support our configured
> +	 * page size, so we can only detect mismatches for a page size other
> +	 * than the one we're currently using. Unfortunately, SoCs like this
> +	 * exist in the wild so, even though we don't like it, we'll have to go
> +	 * along with it and treat them as non-strict.
> +	 */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN16_SHIFT, 4, ID_AA64MMFR0_TGRAN16_NI),
> +
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL0_SHIFT, 4, 0),
> +	/* Linux shouldn't care about secure memory */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_SNSMEM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ASID_SHIFT, 4, 0),
> +	/*
> +	 * Differing PARange is fine as long as all peripherals and memory are mapped
> +	 * within the minimum PARange of all CPUs
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_PARANGE_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VHE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VMIDBITS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HADBS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_UAO_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CNP_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_ctr[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IDC_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_CWG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_ERG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DMINLINE_SHIFT, 4, 1),
> +	/*
> +	 * Linux can handle differing I-cache policies. Userspace JITs will
> +	 * make use of *minLine.
> +	 * If we have differing I-cache policies, report it as the weakest - VIPT.
> +	 */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT),	/* L1Ip */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_BRPS_SHIFT, 4, 0),
> +	/*
> +	 * We can instantiate multiple PMU instances with different levels
> +	 * of support.
> +	 */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_mvfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_dczid[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DEBUG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_COPROC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_CMPBRANCH_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITFIELD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITCOUNT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_SWAP_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar5[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_RDM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_CRC32_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_AES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SEVL_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr4[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_EVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CCIDX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_LSM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
> +
> +	/*
> +	 * SpecSEI = 1 indicates that the PE might generate an SError on an
> +	 * external abort on speculative read. It is safe to assume that an
> +	 * SError might be generated than it will not be. Hence it has been
> +	 * classified as FTR_HIGHER_SAFE.
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_MMFR4_SPECSEI_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar4[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SWP_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_PSR_M_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_BARRIER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SMC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WRITEBACK_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WITHSHIFTS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_UNPRIV_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr5[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR5_ETS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar6[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SPECRES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_FHM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_DP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_JSCVT_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GIC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRT_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SEC_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GENTIMER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRTUALIZATION_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_MPROGMOD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SECURITY_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_PROGMOD_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_SSBS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_CSV3_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> +	/* [31:28] TraceFilt */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr1[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_MTPMU_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> +		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
> +	ARM64_FTR_END,
> +};
> +
> +/*
> + * Common ftr bits for a 32bit register with all hidden, strict
> + * attributes, with 4bit feature fields and a default safe value of
> + * 0. Covers the following 32bit registers:
> + * id_isar[1-4], id_mmfr[1-3], id_pfr1, mvfr[0-1]
> + */
> +static const struct arm64_ftr_bits ftr_generic_32bits[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_raz[] = {
> +	ARM64_FTR_END,
> +};
> +
> +static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
> +				s64 cur)
> +{
> +	s64 ret = 0;
> +
> +	switch (ftrp->type) {
> +	case FTR_EXACT:
> +		ret = ftrp->safe_val;
> +		break;
> +	case FTR_LOWER_SAFE:
> +		ret = min(new, cur);
> +		break;
> +	case FTR_HIGHER_OR_ZERO_SAFE:
> +		if (!cur || !new)
> +			break;
> +		fallthrough;
> +	case FTR_HIGHER_SAFE:
> +		ret = max(new, cur);
> +		break;
> +	default:
> +		BUG();
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * End of imported linux structures and code
> + */
> +
> diff --git a/xen/include/asm-arm/arm64/cpufeature.h b/xen/include/asm-arm/arm64/cpufeature.h
> new file mode 100644
> index 0000000000..d9b9fa77cb
> --- /dev/null
> +++ b/xen/include/asm-arm/arm64/cpufeature.h
> @@ -0,0 +1,104 @@
> +#ifndef __ASM_ARM_ARM64_CPUFEATURES_H
> +#define __ASM_ARM_ARM64_CPUFEATURES_H
> +
> +/*
> + * CPU feature register tracking
> + *
> + * The safe value of a CPUID feature field is dependent on the implications
> + * of the values assigned to it by the architecture. Based on the relationship
> + * between the values, the features are classified into 3 types - LOWER_SAFE,
> + * HIGHER_SAFE and EXACT.
> + *
> + * The lowest value of all the CPUs is chosen for LOWER_SAFE and highest
> + * for HIGHER_SAFE. It is expected that all CPUs have the same value for
> + * a field when EXACT is specified, failing which, the safe value specified
> + * in the table is chosen.
> + */
> +
> +enum ftr_type {
> +	FTR_EXACT,			/* Use a predefined safe value */
> +	FTR_LOWER_SAFE,			/* Smaller value is safe */
> +	FTR_HIGHER_SAFE,		/* Bigger value is safe */
> +	FTR_HIGHER_OR_ZERO_SAFE,	/* Bigger value is safe, but 0 is biggest */
> +};
> +
> +#define FTR_STRICT	true	/* SANITY check strict matching required */
> +#define FTR_NONSTRICT	false	/* SANITY check ignored */
> +
> +#define FTR_SIGNED	true	/* Value should be treated as signed */
> +#define FTR_UNSIGNED	false	/* Value should be treated as unsigned */
> +
> +#define FTR_VISIBLE	true	/* Feature visible to the user space */
> +#define FTR_HIDDEN	false	/* Feature is hidden from the user */
> +
> +#define FTR_VISIBLE_IF_IS_ENABLED(config)		\
> +	(IS_ENABLED(config) ? FTR_VISIBLE : FTR_HIDDEN)
> +
> +struct arm64_ftr_bits {
> +	bool		sign;	/* Value is signed ? */
> +	bool		visible;
> +	bool		strict;	/* CPU Sanity check: strict matching required ? */
> +	enum ftr_type	type;
> +	u8		shift;
> +	u8		width;
> +	s64		safe_val; /* safe value for FTR_EXACT features */
> +};
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field_width(u64 features, int field, int width)
> +{
> +	return (s64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field(u64 features, int field)
> +{
> +	return cpuid_feature_extract_signed_field_width(features, field, 4);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width)
> +{
> +	return (u64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field(u64 features, int field)
> +{
> +	return cpuid_feature_extract_unsigned_field_width(features, field, 4);
> +}
> +
> +static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
> +{
> +	return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign)
> +{
> +	return (sign) ?
> +		cpuid_feature_extract_signed_field_width(features, field, width) :
> +		cpuid_feature_extract_unsigned_field_width(features, field, width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field(u64 features, int field, bool sign)
> +{
> +	return cpuid_feature_extract_field_width(features, field, 4, sign);
> +}
> +
> +static inline s64 arm64_ftr_value(const struct arm64_ftr_bits *ftrp, u64 val)
> +{
> +	return (s64)cpuid_feature_extract_field_width(val, ftrp->shift, ftrp->width, ftrp->sign);
> +}
> +
> +#endif /* _ASM_ARM_ARM64_CPUFEATURES_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:48:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:48:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178520.324592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHyy-0003DL-81; Fri, 03 Sep 2021 22:48:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178520.324592; Fri, 03 Sep 2021 22:48:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHyy-0003D5-2r; Fri, 03 Sep 2021 22:48:24 +0000
Received: by outflank-mailman (input) for mailman id 178520;
 Fri, 03 Sep 2021 22:48:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMHyw-0003BT-Im
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:48:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ffaf7407-2dd6-460a-9fe1-af28644ee89c;
 Fri, 03 Sep 2021 22:48:21 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 716676101A;
 Fri,  3 Sep 2021 22:48:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffaf7407-2dd6-460a-9fe1-af28644ee89c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709300;
	bh=L0jZaRecm61EmSsPjf7fpVTRHQ/Ab8AzvN4SzS0bsD8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VL+W57e/93aDPPY0itPIQAtj2qj4CAGw6qU/InbYWzei28csxrIeJhKknRalZ4M1z
	 SvIf8fJvqog+D7AlFAAlb94p1Jrso8HtBEKEaUJ/mHAUMlm+FyRMZeyXzkGzR0Ghzw
	 wWFYycWaGRtshNUBcZrnClMWu0EH3LcsEyx7HQfj9lkHUpnQJ8FArjKcQokdxUAPEG
	 LetOBjdfgfE6uB5+OCQY7A4g/N5o6Q2FqpHnSTSU0f+TbOaY32RCh70p6kQhgHsFHu
	 4Nan2eW59E0G0lzR/E6LFK3y/dD4AIC+qfecpVPwMfRZ9FNK+8eVqagalgkZZNm4Zb
	 rVU2Aqya/KCrg==
Date: Fri, 3 Sep 2021 15:48:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
In-Reply-To: <6710cc0ac3fbc9c8030dfa3d6dfc3ecb79de4b71.1629897306.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109031425030.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <6710cc0ac3fbc9c8030dfa3d6dfc3ecb79de4b71.1629897306.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Define a sanitize_cpu function to be called on secondary cores to
> sanitize the system cpuinfo structure.
> 
> The safest value is taken when possible and the system is marked tainted
> if we encounter values which are incompatible with each other.
> 
> Call the update_system_features function on all secondary cores that are
> kept running and taint the system if different midr are found between
> cores but hmp-unsafe=true was passed on Xen command line.
> 
> This is only supported on arm64 so update_system_features is an empty
> static inline on arm32.
> 
> The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
> Xen is running on a system with features differences between cores which
> are not supported.
> 
> The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
> as this patch is not handling sanitization of those registers.
> CTR_EL0/DCZID will be handled in a future patch to properly handle
> different cache attributes when possible.
> ZCR should be sanitize once we add support for SVE in Xen.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Great patch! It looks good I only have a couple of minor questions
below.


> ---
> Changes in v3:
> - in case of different midr but hmp-unsafe passed on the command line,
> enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
> - use current core info to sanitize cpu only if we keep it on
> Changes in v2:
> - add compilation of cpufeature.c in this patch instead of previous one
> - remove functions reused from linux code and moved to header
> - rename sanitize_cpu to update_system_features
> - change to Linux coding style
> - remove dev comments
> - surround currently not used Linux structures with #if 0 and adapt the
> commit message
> - add missing aa64dfr1 register
> - add TODO for CTR, DCZID and ZCR
> - add CPU_OUT_OF_SPEC support to print_taint
> - use system_cpuinfo instead of boot_cpu_data
> ---
>  xen/arch/arm/arm64/Makefile      |   1 +
>  xen/arch/arm/arm64/cpufeature.c  | 121 +++++++++++++++++++++++++++++++
>  xen/arch/arm/smpboot.c           |  34 +++++++--
>  xen/common/kernel.c              |   6 +-
>  xen/include/asm-arm/cpufeature.h |   9 +++
>  xen/include/xen/lib.h            |   1 +
>  6 files changed, 162 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
> index 40642ff574..701d66883d 100644
> --- a/xen/arch/arm/arm64/Makefile
> +++ b/xen/arch/arm/arm64/Makefile
> @@ -1,6 +1,7 @@
>  obj-y += lib/
>  
>  obj-y += cache.o
> +obj-y += cpufeature.o
>  obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
>  obj-$(CONFIG_EARLY_PRINTK) += debug.o
>  obj-y += domctl.o
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index 5777e33e5c..61f629ebaa 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: use this to sanitize the cache line size among cores */
> +
>  static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
>  	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: handle this when sanitizing cache related registers */
>  static const struct arm64_ftr_bits ftr_dczid[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  static const struct arm64_ftr_bits ftr_id_isar0[] = {
>  	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> @@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: use this to sanitize SVE once we support it */
> +
>  static const struct arm64_ftr_bits ftr_zcr[] = {
>  	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>  		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  /*
>   * Common ftr bits for a 32bit register with all hidden, strict
> @@ -502,3 +513,113 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
>   * End of imported linux structures and code
>   */
>  
> +static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
> +						const struct arm64_ftr_bits *ftrp)
> +{
> +	int taint = 0;
> +	u64 old_reg = *cur_reg;
> +
> +	for (;ftrp->width != 0;ftrp++)
> +	{
> +		u64 mask;
> +		s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
> +		s64 new_field = arm64_ftr_value(ftrp, new_reg);
> +
> +		if (cur_field == new_field)
> +			continue;
> +
> +		if (ftrp->strict)
> +			taint = 1;
> +
> +		mask = arm64_ftr_mask(ftrp);
> +
> +		*cur_reg &= ~mask;
> +		*cur_reg |= (arm64_ftr_safe_value(ftrp, new_field, cur_field)
> +					<< ftrp->shift) & mask;

I wonder why you haven't also imported arm64_ftr_set_value?  This seems
to be the open-coded version of it.


> +	}
> +
> +	if (old_reg != new_reg)
> +		printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> +				reg_name, old_reg, new_reg);
> +	if (old_reg != *cur_reg)
> +		printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> +				reg_name, old_reg, *cur_reg);
> +
> +	if (taint)
> +	{
> +		printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
> +				reg_name);
> +		add_taint(TAINT_CPU_OUT_OF_SPEC);
> +	}
> +}
> +
> +
> +/*
> + * This function should be called on secondary cores to sanitize the boot cpu
> + * cpuinfo.
> + */
> +void update_system_features(const struct cpuinfo_arm *new)
> +{
> +
> +#define SANITIZE_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				 #reg, ftr_##reg)
> +
> +#define SANITIZE_ID_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_id_##reg)
> +
> +#define SANITIZE_RAZ_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_raz)
> +
> +#define SANITIZE_GENERIC_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_generic_32bits)
> +
> +	SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
> +	SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
> +
> +	SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
> +	SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
> +
> +	SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
> +	SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
> +	SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
> +
> +	SANITIZE_ID_REG(isa64, 0, aa64isar0);
> +	SANITIZE_ID_REG(isa64, 1, aa64isar1);
> +
> +	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
> +
> +	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> +	{
> +		SANITIZE_ID_REG(pfr32, 0, pfr0);
> +		SANITIZE_ID_REG(pfr32, 1, pfr1);
> +		SANITIZE_ID_REG(pfr32, 2, pfr2);
> +
> +		SANITIZE_ID_REG(dbg32, 0, dfr0);
> +		SANITIZE_ID_REG(dbg32, 1, dfr1);
> +
> +		SANITIZE_ID_REG(mm32, 0, mmfr0);
> +		SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
> +		SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
> +		SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
> +		SANITIZE_ID_REG(mm32, 4, mmfr4);
> +		SANITIZE_ID_REG(mm32, 5, mmfr5);
> +
> +		SANITIZE_ID_REG(isa32, 0, isar0);
> +		SANITIZE_GENERIC_REG(isa32, 1, isar1);
> +		SANITIZE_GENERIC_REG(isa32, 2, isar2);
> +		SANITIZE_GENERIC_REG(isa32, 3, isar3);
> +		SANITIZE_ID_REG(isa32, 4, isar4);
> +		SANITIZE_ID_REG(isa32, 5, isar5);
> +		SANITIZE_ID_REG(isa32, 6, isar6);
> +
> +		SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
> +		SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
> +#ifndef MVFR2_MAYBE_UNDEFINED
> +		SANITIZE_REG(mvfr, 2, mvfr2);
> +#endif
> +	}
> +}

Looking at the list of registers we are sanitizing here we are only
missing aux32 and aux64 compared to struct cpuinfo_arm. Is that because
there is nothing to sanitize there?


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:48:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:48:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178521.324603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHz2-0003aX-Gy; Fri, 03 Sep 2021 22:48:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178521.324603; Fri, 03 Sep 2021 22:48:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHz2-0003aO-D3; Fri, 03 Sep 2021 22:48:28 +0000
Received: by outflank-mailman (input) for mailman id 178521;
 Fri, 03 Sep 2021 22:48:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMHz1-0003BT-KI
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:48:27 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fc5df30c-29ca-4894-a89b-066a0aa2911f;
 Fri, 03 Sep 2021 22:48:26 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 119826101A;
 Fri,  3 Sep 2021 22:48:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc5df30c-29ca-4894-a89b-066a0aa2911f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709306;
	bh=jT3rD3kg3gdlzCnptl4+hGg4HVNl2HPkKyI+a61ElnA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=aEtBse5+IXOr4g/JSLovXb6l/if47EWlzriShx6CzKz1NLZ+t56xxxYeSpGo/eclf
	 +Ky9yqiR2uB7b3KcjClz9JJA6/AIjHR1Y9TpRfF5I/j0QnB9Wnv2LDYYQ07MykpjqP
	 NPpfrwDMfNyTafG+SdN5FcLiYoAI9FdEN87CroTBhBXnlMdH0ii77mU8tPQHhQ/PGU
	 jd+2O5d1+irmcLfvOW0ud187rgh37TxyavhudmVzpgCVSHEH/uav4GeQc8FCMUVeyn
	 uL9G4v8TGebawV2191LY7dhq48tX1LLBUCQJmFOHlnUNtnwSjRKHlhBeAr5KX0QY6+
	 nkaieVjWOMvDQ==
Date: Fri, 3 Sep 2021 15:48:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 5/7] xen/arm: Use sanitize values for p2m
In-Reply-To: <dbc702844a4322db69a1b0a56ab57be66f4d6174.1629897306.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109031527070.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <dbc702844a4322db69a1b0a56ab57be66f4d6174.1629897306.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Replace the code in p2m trying to find a sane value for the VMID size
> supported and the PAR to use. We are now using the boot cpuinfo as the
> values there are sanitized during boot and the value for those
> parameters is now the safest possible value on the system.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v3: drop arm32 mention in commmit message
> Changes in v2:
>  - use system_cpuinfo
> ---
>  xen/arch/arm/p2m.c | 30 ++++++++++--------------------
>  1 file changed, 10 insertions(+), 20 deletions(-)
> 
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index eff9a105e7..41b6430c30 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -2045,31 +2045,21 @@ void __init setup_virt_paging(void)
>          [7] = { 0 }  /* Invalid */
>      };
>  
> -    unsigned int i, cpu;
> +    unsigned int i;
>      unsigned int pa_range = 0x10; /* Larger than any possible value */
> -    bool vmid_8_bit = false;
> -
> -    for_each_online_cpu ( cpu )
> -    {
> -        const struct cpuinfo_arm *info = &cpu_data[cpu];
>  
> -        /*
> -         * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
> -         * with IPA bits == PA bits, compare against "pabits".
> -         */
> -        if ( pa_range_info[info->mm64.pa_range].pabits < p2m_ipa_bits )
> -            p2m_ipa_bits = pa_range_info[info->mm64.pa_range].pabits;
> -
> -        /* Set a flag if the current cpu does not support 16 bit VMIDs. */
> -        if ( info->mm64.vmid_bits != MM64_VMID_16_BITS_SUPPORT )
> -            vmid_8_bit = true;
> -    }
> +    /*
> +     * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
> +     * with IPA bits == PA bits, compare against "pabits".
> +     */
> +    if ( pa_range_info[system_cpuinfo.mm64.pa_range].pabits < p2m_ipa_bits )
> +        p2m_ipa_bits = pa_range_info[system_cpuinfo.mm64.pa_range].pabits;
>  
>      /*
> -     * If the flag is not set then it means all CPUs support 16-bit
> -     * VMIDs.
> +     * cpu info sanitization made sure we support 16bits VMID only if all
> +     * cores are supporting it.
>       */
> -    if ( !vmid_8_bit )
> +    if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
>          max_vmid = MAX_VMID_16_BIT;
>  
>      /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:49:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:49:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178543.324614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHzh-0004r7-Sj; Fri, 03 Sep 2021 22:49:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178543.324614; Fri, 03 Sep 2021 22:49:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMHzh-0004r0-Oy; Fri, 03 Sep 2021 22:49:09 +0000
Received: by outflank-mailman (input) for mailman id 178543;
 Fri, 03 Sep 2021 22:49:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMHzg-0004pW-5x
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:49:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 59256c8a-01da-49cc-a5ab-53bb46601b0c;
 Fri, 03 Sep 2021 22:49:04 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9DC7E6101A;
 Fri,  3 Sep 2021 22:49:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59256c8a-01da-49cc-a5ab-53bb46601b0c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709343;
	bh=S/hCI4aNd0cSDI8CmUHd0XFH3442g/fhnIUYiVIArNA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ZgA3V3osAHiiWa8wy42E9oJxR4wvoMSZDXARPUT8fYpa+9FEPaMH1XtpIXsb2SWVe
	 7OENlv0thp8D6iYy0RQD9FG15Aa1b6nJut2G5X/eo/QDhTi0z4TNhTNk12ZJOWC25S
	 csVfmz0CpJQ4Eeig3dBQzEIJMdwZOPWLfKCwkzezPmEIRrNvoo2CbsMR2V/2oVOcgB
	 OiIPRxNg1D9PEdDIHMoeV0E0U61qeljMIggl6Ex7Mj2+bfjOHzMz5n7+4bSE7F1z12
	 1XT5aZJviY3U37+nnYFNFsoP8rC1jMDNxIyeTvO+EDhZ7yiB/BUR1kfmHlraCAKxyG
	 I5bonXuCQUjTA==
Date: Fri, 3 Sep 2021 15:49:03 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
In-Reply-To: <c1868fce1bcb0bcddf7bc786be166007f91f2f67.1629897306.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109031529400.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <c1868fce1bcb0bcddf7bc786be166007f91f2f67.1629897306.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
                                        ^  TAINT


> are found (ftr_dczid is using only STRICT method).
> In this case actual memory being cleaned by DC ZVA operations would be
> different depending on the cores which could make a guest zeroing too
> much or too little memory if it is merged between CPUs.
> 
> We could, on processor supporting it, trap access to DCZID_EL0 register
               ^ processors

> using HFGRTR_EL2 register but this would not solve the case where a
> process is being migrated during a copy or if it cached the value of the
> register.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> Change in v3: none
> Change in v2: Patch introduced in v2
> ---
>  xen/arch/arm/arm64/cpufeature.c  | 14 +++++++++++---
>  xen/arch/arm/cpufeature.c        |  2 ++
>  xen/include/asm-arm/cpufeature.h |  8 ++++++++
>  3 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index 61f629ebaa..b1936ef1d6 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> -#if 0
> -/* TODO: handle this when sanitizing cache related registers */
>  static const struct arm64_ftr_bits ftr_dczid[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> -#endif
>  
>  static const struct arm64_ftr_bits ftr_id_isar0[] = {
>  	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
>  
>  	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>  
> +	/*
> +	 * Comment from Linux:

I don't know if I would keep or remove "Comment from Linux"

In any case:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


Also I gave a quick test of the series on a ZCU102 and it worked fine.


> +	 * Userspace may perform DC ZVA instructions. Mismatched block sizes
> +	 * could result in too much or too little memory being zeroed if a
> +	 * process is preempted and migrated between CPUs.
> +	 *
> +	 * ftr_dczid is using STRICT comparison so we will taint Xen if different
> +	 * values are found.
> +	 */
> +	SANITIZE_REG(dczid, 0, dczid);
> +
>  	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>  	{
>  		SANITIZE_ID_REG(pfr32, 0, pfr0);
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index f600a611bd..113f20f601 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>  
>      c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
>  
> +    c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
> +
>      aarch32_el0 = cpu_feature64_has_el0_32(c);
>  #endif
>  
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 52cb3133e0..5219fd3bab 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -259,6 +259,14 @@ struct cpuinfo_arm {
>          register_t bits[1];
>      } zfr64;
>  
> +    /*
> +     * DCZID is only used to check for incoherent values between cores
> +     * and taint Xen in this case
> +     */
> +    struct {
> +        register_t bits[1];
> +    } dczid;
> +
>  #endif
>  
>      /*
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 03 22:49:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 03 Sep 2021 22:49:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178547.324624 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMI0A-0005Xr-9D; Fri, 03 Sep 2021 22:49:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178547.324624; Fri, 03 Sep 2021 22:49:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMI0A-0005Xk-5l; Fri, 03 Sep 2021 22:49:38 +0000
Received: by outflank-mailman (input) for mailman id 178547;
 Fri, 03 Sep 2021 22:49:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UKUJ=NZ=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mMI08-0005XF-DJ
 for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 22:49:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 92e6d110-16a5-49ea-9fed-1136efe8e605;
 Fri, 03 Sep 2021 22:49:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E69C60FA0;
 Fri,  3 Sep 2021 22:49:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92e6d110-16a5-49ea-9fed-1136efe8e605
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1630709374;
	bh=MelmbXi6qQvN2rHzNWcwaBtUUsR83J5tHhU/VWOnA1s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mcilNlKENYopNymU1GVqplsCs8G0/C4btMrZJGuTed3q3TrBbiG8T3gEGlnzljwVX
	 SUtsli8oSfJiIkWIfRVRxWRGC3HKGIqd9aYs7K4EQCzWqXM9CU8l4dEdGjt2oYKAYi
	 BrHh+LjMyGVhE9yH0oufb0eWPTNoftb9qmiyTagEG4p1GBTV12Z8h9WYVDrHlseIyt
	 HV6Hf5OvsINegFSIlXRHAAErKX0+3zkRUfz2QgFNSiKhwUDCLp8kIUYlqXw7O2fnA9
	 enW0pUxNRQCJ8rI7q4zLaA9BrUItINft9SpdUgs8WyjEreechqYFS7a1EJPxizLBd+
	 wPafcGzr9n/Hg==
Date: Fri, 3 Sep 2021 15:49:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
cc: Julien Grall <julien@xen.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if
 needed
In-Reply-To: <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com>
Message-ID: <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <a7482653a0c29d68854846f25553f4938a6279b9.1629897306.git.bertrand.marquis@arm.com> <14b1639e-c9e4-f8fb-e921-73d7f28681ac@xen.org> <98E8FB9D-3372-47D2-92F7-8AD401AC1092@arm.com>
 <6705127a-bde2-36cd-0fed-900eb1eace03@xen.org> <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 31 Aug 2021, Bertrand Marquis wrote:
> Hi Julien,
> 
> > On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
> > 
> > 
> > 
> > On 31/08/2021 14:17, Bertrand Marquis wrote:
> >> Hi Julien,
> > 
> > Hi Bertrand,
> > 
> >>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
> >>> 
> >>> Hi Bertrand,
> >>> 
> >>> On 25/08/2021 14:18, Bertrand Marquis wrote:
> >>>> Sanitize CTR_EL0 value between cores.
> >>>> In most cases different values will taint Xen but if different
> >>>> i-cache policies are found, we choose the one which will be compatible
> >>>> between all cores in terms of invalidation/data cache flushing strategy.
> >>> 
> >>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
> >>> 
> >>>> In this case we need to activate the TID2 bit in HCR to emulate the
> >>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
> >>>> the time to limit the overhead when possible.
> >>> 
> >>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
> >>> 
> >>> So I think we should be able to get away from trappings the registers.
> >> I do agree that we should be able to get away from that in the long term once
> >> we have cpupools properly set but right now this is the only way to have
> >> something useable (I will not say right).
> >> I will work on finding a way to setup properly cpupools (or something else as
> >> we discussed earlier) but in the short term I think this is the best we can do.
> > 
> > My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
> 
> I agree and this is why Xen is tainted.
> 
> > 
> >> An other solution would be to discard this patch from the serie for now until
> >> I have worked a proper solution for this case.
> >> Should we discard or merge or do you have an other idea ?
> > Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
> 
> Not sure it depends on what the final solution would be but this is highly possible I agree.
> 
> > 
> > This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
> > 
> > No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
> 
> Ok, I will wait for feedback and next serie will not include this patch anymore.

Would it be worth keeping just the part that sanitizes ctr, without any
of the emulation stuff? That way we could still detect cases where there
is a mismatch between CPUs, print a useful message and taint Xen.

For clarity something like the appended.


diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index b1936ef1d6..d2456af2bf 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
 	ARM64_FTR_END,
 };
 
-#if 0
-/* TODO: use this to sanitize the cache line size among cores */
-
 static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
-#endif
 
 static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -600,6 +596,8 @@ void update_system_features(const struct cpuinfo_arm *new)
 	 */
 	SANITIZE_REG(dczid, 0, dczid);
 
+	SANITIZE_REG(ctr, 0, ctr);
+
 	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
 	{
 		SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 113f20f601..6e51f530a8 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
 
     c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
 
+    c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
+
     aarch32_el0 = cpu_feature64_has_el0_32(c);
 #endif
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 3798c5ade0..33b7bfb59c 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -627,7 +627,7 @@ static void __init setup_mm(void)
         panic("No memory bank\n");
 
     /* We only supports instruction caches implementing the IVIPT extension. */
-    if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
+    if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
         panic("AIVIVT instruction cache not supported\n");
 
     init_pdx();
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 5219fd3bab..ca6e827fcb 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -267,6 +267,14 @@ struct cpuinfo_arm {
         register_t bits[1];
     } dczid;
 
+    /*
+     * CTR is only used to check for different cache types or policies and
+     * taint Xen in this case
+     */
+    struct {
+        register_t bits[1];
+    } ctr;
+
 #endif
 
     /*
@@ -339,6 +347,9 @@ extern struct cpuinfo_arm system_cpuinfo;
 extern void identify_cpu(struct cpuinfo_arm *);
 
 #ifdef CONFIG_ARM_64
+
+extern bool mismatched_cache_type;
+
 extern void update_system_features(const struct cpuinfo_arm *);
 #else
 static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 2577e9a244..8c9843e12b 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -7,9 +7,21 @@
 #include <public/arch-arm.h>
 
 /* CTR Cache Type Register */
-#define CTR_L1Ip_MASK       0x3
-#define CTR_L1Ip_SHIFT      14
-#define CTR_L1Ip_AIVIVT     0x1
+#define CTR_L1IP_MASK       0x3
+#define CTR_L1IP_SHIFT      14
+#define CTR_DMINLINE_SHIFT  16
+#define CTR_IMINLINE_SHIFT  0
+#define CTR_IMINLINE_MASK   0xf
+#define CTR_ERG_SHIFT       20
+#define CTR_CWG_SHIFT       24
+#define CTR_CWG_MASK        15
+#define CTR_IDC_SHIFT       28
+#define CTR_DIC_SHIFT       29
+
+#define ICACHE_POLICY_VPIPT  0
+#define ICACHE_POLICY_AIVIVT 1
+#define ICACHE_POLICY_VIPT   2
+#define ICACHE_POLICY_PIPT   3
 
 /* MIDR Main ID Register */
 #define MIDR_REVISION_MASK      0xf


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 00:47:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 00:47:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178583.324660 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMJpl-00042a-Hf; Sat, 04 Sep 2021 00:47:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178583.324660; Sat, 04 Sep 2021 00:47:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMJpl-00042T-EJ; Sat, 04 Sep 2021 00:47:01 +0000
Received: by outflank-mailman (input) for mailman id 178583;
 Sat, 04 Sep 2021 00:46:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMJpj-00042J-FT; Sat, 04 Sep 2021 00:46:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMJpj-00012Y-4V; Sat, 04 Sep 2021 00:46:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMJpi-0003T1-Mk; Sat, 04 Sep 2021 00:46:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMJpi-0005cd-MC; Sat, 04 Sep 2021 00:46:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0bncTkJNzJmmLEieCHFUIGy1mpdW2MYnUjAw5cCSBZI=; b=IenDntL6kuulVAjl/OK5XoEsG+
	4tBwgZNO85bKyk42FZoizfnyYY1qun/eb1GiEtAzcWhbiUpINtT/h3/Qep/PZisOqe6RajHpIa6bO
	VGFgEbxOVxVbVnlrjzzf6OrN0iXaJRYeMrWNAwFVIbIh/LAQaTKFs5ahfJccbRtC06ic=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164797-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164797: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd11-amd64:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-arndale:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a9c9a6f741cdaa2fa9ba24a790db8d07295761e3
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 00:46:58 +0000

flight 164797 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164797/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 19 guest-localmigrate/x10 fail in 164791 pass in 164797
 test-armhf-armhf-xl-arndale   8 xen-boot         fail in 164791 pass in 164797
 test-amd64-amd64-examine    4 memdisk-try-append fail in 164791 pass in 164797
 test-arm64-arm64-xl-credit2  13 debian-fixup               fail pass in 164791
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164791

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2 15 migrate-support-check fail in 164791 never pass
 test-arm64-arm64-xl-credit2 16 saverestore-support-check fail in 164791 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                a9c9a6f741cdaa2fa9ba24a790db8d07295761e3
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  399 days
Failing since        152366  2020-08-01 20:49:34 Z  398 days  690 attempts
Testing same since   164791  2021-09-03 04:54:46 Z    0 days    2 attempts

------------------------------------------------------------
7181 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2197963 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 03:44:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 03:44:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178601.324674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMMb3-0005XM-Du; Sat, 04 Sep 2021 03:44:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178601.324674; Sat, 04 Sep 2021 03:44:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMMb3-0005Wu-74; Sat, 04 Sep 2021 03:44:01 +0000
Received: by outflank-mailman (input) for mailman id 178601;
 Sat, 04 Sep 2021 03:44:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMMb2-0005Wk-0F; Sat, 04 Sep 2021 03:44:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMMb1-0002Aw-Qn; Sat, 04 Sep 2021 03:43:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMMb1-0005FZ-GF; Sat, 04 Sep 2021 03:43:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMMb1-00055m-Fl; Sat, 04 Sep 2021 03:43:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DwGiL8QW2xyzCkVDtcNqaKi/z8rH3Mzn6XEREdvRhbg=; b=f++5trXf/rEZMscuCvDXjXDYi0
	mitrxmE4VE53/Ptrp7AKg/CbSoQqfl4yQMgGBF9R1TqULU6uVeqdUzfXqE33Mm89mRD3Mm5PoJ5+4
	3QnyMt9bXjAmKos8zKfsdEJRLTn6uFGTDiKqLZizAFNaJhhed6J5GCsFBZHiA0vrOo6I=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164800-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164800: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 03:43:59 +0000

flight 164800 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164800/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   24 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    0 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 06:38:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 06:38:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178611.324688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMPJW-0007p9-SN; Sat, 04 Sep 2021 06:38:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178611.324688; Sat, 04 Sep 2021 06:38:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMPJW-0007p2-Op; Sat, 04 Sep 2021 06:38:06 +0000
Received: by outflank-mailman (input) for mailman id 178611;
 Sat, 04 Sep 2021 06:38:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMPJV-0007oq-Kq; Sat, 04 Sep 2021 06:38:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMPJV-0005XQ-Bb; Sat, 04 Sep 2021 06:38:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMPJV-0003md-0S; Sat, 04 Sep 2021 06:38:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMPJU-0002YX-WD; Sat, 04 Sep 2021 06:38:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SrH0xbq8PMwzRBkCG1Z2oCZs1PbIgQc0nT6JJRq59jU=; b=0IHtH+F0eh5l1u39CcfCGtU5Sc
	cwZFWJYLrSdw2JUA+WIVJD4vkifpqrGCk3UAFn5i3T5l4TmKcu9tmDgpQ8O94xXEy/up+uDwffrvK
	tQUSP7SuJL6jJZDtJ7JePxhkTQzGeo8meBGie6DnSrVZos43XJW+mMVZQD3FGY5XQ9nM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164802-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164802: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 06:38:04 +0000

flight 164802 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164802/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   25 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    1 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 07:48:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 07:48:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178644.324720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMQPW-00004r-Mz; Sat, 04 Sep 2021 07:48:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178644.324720; Sat, 04 Sep 2021 07:48:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMQPW-0008WQ-JV; Sat, 04 Sep 2021 07:48:22 +0000
Received: by outflank-mailman (input) for mailman id 178644;
 Sat, 04 Sep 2021 07:48:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMQPV-0008WG-K8; Sat, 04 Sep 2021 07:48:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMQPV-0006hn-Ef; Sat, 04 Sep 2021 07:48:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMQPV-000096-3a; Sat, 04 Sep 2021 07:48:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMQPV-0007Kk-36; Sat, 04 Sep 2021 07:48:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tx6n7jqUgf2Jb8dXbHRt/okq7iZnCXTSEOEJfvDRPMU=; b=yXZ59f9hfrFth4RmqsHW/hKgdP
	sAAw0StGAvzWFSBKPO2PBtSzyZKb3rZZmng6jU/tc/C19QWqun+SwmtgqPNpFB4syUvEcO1/iELXt
	xSo9PA8odrF+PUBc65hnY3lw8bwUcgG7yAx6KKOKORmb2R9Bs3Ed/7Ft4t+Fuz29699U=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164798-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164798: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 07:48:21 +0000

flight 164798 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164798/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164786
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164786
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164786
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164786
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164786
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164786
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164786
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164786
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164786
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164786
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164786
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618

Last test of basis   164786  2021-09-03 01:51:45 Z    1 days
Testing same since   164798  2021-09-03 17:37:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Christian Lindig <christian.lindig@citrix.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   96607a8e68..2d4978ead4  2d4978ead4b98452bc45ebe47952e524b12e2b41 -> master


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 09:29:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 09:29:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178669.324746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMRyr-0003UP-Jz; Sat, 04 Sep 2021 09:28:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178669.324746; Sat, 04 Sep 2021 09:28:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMRyr-0003UI-GP; Sat, 04 Sep 2021 09:28:57 +0000
Received: by outflank-mailman (input) for mailman id 178669;
 Sat, 04 Sep 2021 09:28:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMRyp-0003U8-Lh; Sat, 04 Sep 2021 09:28:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMRyp-0000gL-BM; Sat, 04 Sep 2021 09:28:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMRyp-0005b2-31; Sat, 04 Sep 2021 09:28:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMRyp-0005v1-2X; Sat, 04 Sep 2021 09:28:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/Cfieg3LyQJJJNaxExgkYGhHXNGypfCl0VIfHdwXYZA=; b=u4YQ7lGOFYJyhCUKhopCjzJXKi
	zPrUC5ec3t1+aylIZ5vTl4HPXrGzfn1sb6/zACtuhltTzwx/X7BKFM98XFWWE59CPbkU5Rm6/R/9d
	4IM79KIOKG5o1iLQFVQ480p2EaxrclrnMHzSbLxEVW1z6uvRDaHAzNIpt56icMrekSvs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164804-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164804: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=81d71fb86e1ad676e94becb7ffffb403cefd3019
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 09:28:55 +0000

flight 164804 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164804/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 81d71fb86e1ad676e94becb7ffffb403cefd3019
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   26 attempts
Testing same since   164788  2021-09-03 04:12:12 Z    1 days    7 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 81d71fb86e1ad676e94becb7ffffb403cefd3019
Author: Hao A Wu <hao.a.wu@intel.com>
Date:   Fri Aug 20 13:36:56 2021 +0800

    Maintainers.txt: Update maintainer/reviewer roles in MdeModulePkg
    
    Remove Hao A Wu as the MdeModulePkg maintainer.
    Add Liming Gao as the MdeModulePkg maintainer.
    Remove Hao A Wu as the MdeModulePkg: Firmware Update modules reviewer.
    Remove Hao A Wu as the MdeModulePkg: Serial modules reviewer.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Guomin Jiang <guomin.jiang@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Zhichao Gao <zhichao.gao@intel.com>
    Signed-off-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
    Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>

commit c5e805ffe1098601e84fd501d5fa1d45e9d96a62
Author: Rebecca Cran <rebecca@bsdio.com>
Date:   Tue Aug 31 06:36:36 2021 +0800

    MdeModulePkg: Fix typo of "memory" in RamDiskDxe debug message
    
    Fix a typo of "memory" in a debug message in RamDiskProtocol.c.
    
    Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
    Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>

commit b6bc203375b6efb6822ee4fe4a59be4f1918436b
Author: Wenyi Xie <xiewenyi2@huawei.com>
Date:   Thu Aug 26 09:16:03 2021 +0800

    MdeModulePkg/HiiDatabaseDxe:remove dead code block
    
    As the if statement outside has confirmed that
    BlockData->Name == NULL, so the if statement inside
    is always false.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Dandan Bi <dandan.bi@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
    Reviewed-by: Dandan Bi <dandan.bi@intel.com>

commit e3ee8c8dbd7a9f7d7905abe17be60354c9f5f9a3
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:37 2021 +0800

    .azurepipelines: Add UefiPayloadPkg in gate-build-job.yml and CISetting.py
    
    Add UefiPayloadPkg in gate-build-job.yml to enable Core ci for UefiPayloadPkg.
    Add UefiPayloadPkg to supported Packages in CISettings.
    
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 63fddc98e06c380e654b1cfbaf24b9fd922adfad
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 14:43:36 2021 +0800

    UefiPayloadPkg: Create .yaml file in UefiPayloadPkg
    
    Create .yaml file in UefiPayloadPkg to enable Core ci for UefiPayloadPkg
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 12e33dca4c0612a0975265e5ba641c6261a26455
Author: Loo, Tung Lun <tung.lun.loo@intel.com>
Date:   Tue Aug 17 15:43:12 2021 +0800

    IntelFsp2Pkg: Support Config File and Binary delta comparison
    
    BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3567
    
    This patch is to enable config editor to have a new feature that
    can spell out the delta between the default configuration files'
    data, such as YAML and BSF, against the data stored in the binary.
    This can help users understand and track the difference when
    modifications are made.
    
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
    Cc: Star Zeng <star.zeng@intel.com>
    Cc: Chasel Chiu <chasel.chiu@intel.com>
    Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
    Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>

commit b170806518c1e414939c8b085866544814e1ce8e
Author: Mark Wilson <Mark.Wilson@amd.com>
Date:   Fri Nov 13 08:05:18 2020 +0800

    UefiCpuPkg: Clean up save state boundary checks and comments.
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2956
    
    In functions ReadSaveStateRegisterByIndex and WriteSaveStateRegister:
    * check width > 4 instead of >= 4 when writing upper 32 bytes.
      - This improves the code but will not affect functionality.
    
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Signed-off-by: Mark Wilson <Mark.Wilson@amd.com>

commit 3b3f88228812e2c642eea8746858a4dad928ebf7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 14:09:25 2021 +0800

    MdeModulePkg/PiSmmCore: Drop deprecated image profiling commands
    
    The legacy codebase allowed SMM images to be registered for profiling
    from DXE. Support for this has been dropped entirely, so remove the
    remaining handlers.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Hao A Wu <hao.a.wu@intel.com>
    Cc: Eric Dong <eric.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit cdda3f74a1327663a5d48cca13507085ba403af7
Author: Marvin H?user <mhaeuser@posteo.de>
Date:   Mon Aug 9 03:39:53 2021 +0800

    UefiPayloadPkg/UefiPayloadEntry: Fix memory corruption
    
    UefiPayloadEntry's AllocatePool() applies the "sizeof" operator to
    HOB index rather than the HOB header structure. This yields 4 Bytes
    compared to the 8 Bytes the structure header requires. Fix the call
    to allocate the required space instead.
    
    Reviewed-by: Guo Dong <guo.dong@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Cc: Vitaly Cheptsov <vit9696@protonmail.com>
    Signed-off-by: Marvin H?user <mhaeuser@posteo.de>

commit 5d34cc49d5d348012fe8acf9fb618826bd541a7c
Author: Wenxing Hou <wenxing.hou@intel.com>
Date:   Tue Aug 24 16:11:11 2021 +0800

    UefiCpuPkg/PiSmmCpuDxeSmm: Update mPatchCetSupported set condition
    
    REF:https://bugzilla.tianocore.org/show_bug.cgi?id=3584
    
    Function AsmCpuid should first check the value for Basic CPUID Information.
    The fix is to update the mPatchCetSupported judgment statement.
    
    Signed-off-by: Wenxing Hou <wenxing.hou@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>
    Cc: Eric Dong   <eric.dong@intel.com>
    Cc: Ray Ni      <ray.ni@intel.com>
    Cc: Rahul Kumar <rahul1.kumar@intel.com>
    Cc: Sheng W     <w.sheng@intel.com>
    Cc: Yao Jiewen  <jiewen.yao@intel.com>

commit f0fe55bca4651734abf1752a1d7c69fb5bee00b9
Author: duntan <dun.tan@intel.com>
Date:   Fri Aug 20 13:04:29 2021 +0800

    UefiPayloadPkg: Fix the build error when enable Core ci for UefiPayloadPkg
    
    V1: Add quotes when using $(ARCH) in .dsc and .fdf file.
    The quotes are added due to the way by which Core ci parse the .dsc file.
    Add UINTN in Hob.c to fix cast from pointer to integer of different size error.
    V2: Delete lines which reference ShellBinPkg.The pkg doesn't exist in edk2.
    
    Cc: Guo Dong <guo.dong@intel.com>
    Cc: Ray Ni <ray.ni@intel.com>
    Cc: Maurice Ma <maurice.ma@intel.com>
    Cc: Benjamin You <benjamin.you@intel.com>
    Signed-off-by: DunTan <dun.tan@intel.com>
    Reviewed-by: Ray Ni <ray.ni@intel.com>

commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:29:48 2021 +0800

    OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
    
    Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
    the kernel, initrd, and kernel command line from QEMU's fw_cfg.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:15 2021 +0800

    OvmfPkg/SmmControl2Dxe: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies SmmControl2Dxe to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 52e2dabc0f8d3af09c213072ce8ba734302f585d
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:14 2021 +0800

    OvmfPkg/PlatformBootManagerLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies PlatformBootManagerLib to
    detect S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit 28152333bccb778b62e6e97446b28bfa0e92ef82
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:13 2021 +0800

    OvmfPkg/LockBoxLib: use PcdAcpiS3Enable to detect S3 support
    
    To avoid the potential inconsistency between PcdAcpiS3Enable and
    QemuFwCfgS3Enabled(), this commit modifies LockBoxLib to detect
    S3 support by PcdAcpiS3Enable as modules in MdeModulePkg do.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>

commit cb0d24637dfdd869618b9635dfb8e3b0746393a6
Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
Date:   Tue Aug 31 09:31:12 2021 +0800

    OvmfPkg/OvmfXen: set PcdAcpiS3Enable at initialization
    
    There are several functions in OvmfPkg/Library using
    QemuFwCfgS3Enabled() to detect the S3 support status. However, in
    MdeModulePkg, PcdAcpiS3Enable is used to check S3 support. Since
    InitializeXenPlatform() didn't set PcdAcpiS3Enable as
    InitializePlatform() did, this made the inconsistency between
    drivers/functions.
    
    For example, S3SaveStateDxe checked PcdAcpiS3Enable and skipped
    S3BootScript because the default value is FALSE. On the other hand,
    PlatformBootManagerBeforeConsole() from OvmfPkg/Library called
    QemuFwCfgS3Enabled() and found it returned TRUE, so it invoked
    SaveS3BootScript(). However, S3SaveStateDxe skipped S3BootScript, so
    SaveS3BootScript() asserted due to EFI_NOT_FOUND.
    
    This issue mainly affects "HVM Direct Kernel Boot". When used,
    "fw_cfg" is enabled in QEMU and QemuFwCfgS3Enabled() returns true in
    that case.
    
    Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3573
    Signed-off-by: Gary Lin <gary.lin@hpe.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
    Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
    Tested-by: Jim Fehlig <jfehlig@suse.com>


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 10:25:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 10:25:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178679.324760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMSrQ-0002B4-2F; Sat, 04 Sep 2021 10:25:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178679.324760; Sat, 04 Sep 2021 10:25:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMSrP-0002Ax-Te; Sat, 04 Sep 2021 10:25:19 +0000
Received: by outflank-mailman (input) for mailman id 178679;
 Sat, 04 Sep 2021 10:25:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSrO-0002An-RV; Sat, 04 Sep 2021 10:25:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSrO-0001hL-N7; Sat, 04 Sep 2021 10:25:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSrO-0007uB-Di; Sat, 04 Sep 2021 10:25:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSrO-0001Be-DA; Sat, 04 Sep 2021 10:25:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4MAAGzAf8C3aoSrTzXOJZvvCtrTOVf5RkZ8al5VcYIk=; b=pJNxb23TnPq8lqzfNowEMUz2By
	T4et/y/KCX9KJa/qFOHiJ5jWjJW/uWMNuY0RPzVDITpkRYmKS5u4nXliL+jJY361olsoYQTBQ9t8N
	6aWE3fDm3heaEAQsVbHUwSVejIsBhlm6yN04nerxkmsHabOUy6wp9i7qgNJY3vKAUrck=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164799-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164799: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=8880cc4362fde4ecdac0b2092318893118206fcf
X-Osstest-Versions-That:
    qemuu=8664d30a30fd676b56b4c29dbcbdd5c5538acfc1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 10:25:18 +0000

flight 164799 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164799/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164792
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164792
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164792
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164792
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164792
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164792
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164792
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                8880cc4362fde4ecdac0b2092318893118206fcf
baseline version:
 qemuu                8664d30a30fd676b56b4c29dbcbdd5c5538acfc1

Last test of basis   164792  2021-09-03 07:39:10 Z    1 days
Testing same since   164799  2021-09-03 19:38:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   8664d30a30..8880cc4362  8880cc4362fde4ecdac0b2092318893118206fcf -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 10:29:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 10:29:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178686.324774 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMSvD-0002sN-If; Sat, 04 Sep 2021 10:29:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178686.324774; Sat, 04 Sep 2021 10:29:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMSvD-0002sG-FO; Sat, 04 Sep 2021 10:29:15 +0000
Received: by outflank-mailman (input) for mailman id 178686;
 Sat, 04 Sep 2021 10:29:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSvB-0002s6-TD; Sat, 04 Sep 2021 10:29:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSvB-0001k2-Pq; Sat, 04 Sep 2021 10:29:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSvB-00084H-GD; Sat, 04 Sep 2021 10:29:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMSvB-0003k2-Fh; Sat, 04 Sep 2021 10:29:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yjsxcA4NL86rfu6zlr1Qcpmjsacil1hSskvxdkkpu7o=; b=af4XRfQ9HosA1NW6ROZ9vtRlRJ
	EB8oxnDGD1f0EjPSxK2halFkbXJHvbTZg3dWwZDTvfBqOGES6hOgdid6A/eaI1Id9EcoZrKsM2s5f
	3CCBMNxXhNgh7PwtSJ4ARAeXVkELYFbYmgcntaZzJOelf3WkCvC+TOYLX7PF2ZgljZ88=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164806-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164806: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 10:29:13 +0000

flight 164806 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164806/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   27 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 12:05:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 12:05:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178699.324788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMUPd-0005m3-RA; Sat, 04 Sep 2021 12:04:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178699.324788; Sat, 04 Sep 2021 12:04:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMUPd-0005lw-NH; Sat, 04 Sep 2021 12:04:45 +0000
Received: by outflank-mailman (input) for mailman id 178699;
 Sat, 04 Sep 2021 12:04:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMUPc-0005lm-Ni; Sat, 04 Sep 2021 12:04:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMUPc-0003GM-HL; Sat, 04 Sep 2021 12:04:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMUPc-0003CV-8T; Sat, 04 Sep 2021 12:04:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMUPc-0008N8-80; Sat, 04 Sep 2021 12:04:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PFMNO0QEM5lG58zh52CPD1s//6Ms9MNrf4aY0nhcpwM=; b=1ViEmBH/EW3vss63NxEVxX/5uq
	BDU2TBoDoBoJ+bOQYsILfiKz4Z5SjXM/NZB6X1+ys4Ta2dAGLfMYYZD7IdnMAmYc0pVRSn8xwH8IG
	JK/0tlllm8OZUQ1bFRYGKd3w9fpFiehJFggVKTMZk8mFCr7RQz1ZkVL4ScnBYX8gSXNI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164803-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164803: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=b6ec01b82154c9dc25f18c9c28df30d0c053defe
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 12:04:44 +0000

flight 164803 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164803/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              b6ec01b82154c9dc25f18c9c28df30d0c053defe
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  421 days
Failing since        151818  2020-07-11 04:18:52 Z  420 days  411 attempts
Testing same since   164803  2021-09-04 04:20:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71803 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 14:28:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 14:28:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178734.324802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMWet-0004PU-W5; Sat, 04 Sep 2021 14:28:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178734.324802; Sat, 04 Sep 2021 14:28:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMWet-0004PN-SO; Sat, 04 Sep 2021 14:28:39 +0000
Received: by outflank-mailman (input) for mailman id 178734;
 Sat, 04 Sep 2021 14:28:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMWes-0004PD-6X; Sat, 04 Sep 2021 14:28:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMWer-0005eF-Ne; Sat, 04 Sep 2021 14:28:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMWer-0000rc-EE; Sat, 04 Sep 2021 14:28:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMWer-0006RF-Dg; Sat, 04 Sep 2021 14:28:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=L4dfjSPnMwfuD0VJbj+aT9SXnk8TfkWrBOjQJNU9DVU=; b=FEVzBDHsRHwcwjylWnpIfHXW5v
	zsncELZua2UkjhmGVbt4fawgcev851bdyddMe8CjjebxTIymcCw3BVOZhwIOnhcOg6SCz18gzJee6
	laefZP2+IlalGIiMYZ02XaDVkdBS4EFy8L9u7x4WQwytwHBW0/vdQ/dx1rz3o4A9uMac=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164801-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164801: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f1583cb1be35c23df60b1c39e3e7e6704d749d0b
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 14:28:37 +0000

flight 164801 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164801/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow  22 guest-start/debian.repeat fail REGR. vs. 152332
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                f1583cb1be35c23df60b1c39e3e7e6704d749d0b
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  399 days
Failing since        152366  2020-08-01 20:49:34 Z  398 days  691 attempts
Testing same since   164801  2021-09-04 01:12:40 Z    0 days    1 attempts

------------------------------------------------------------
7203 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   fail    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2211696 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 15:12:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 15:12:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178747.324834 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMXLA-0002Do-Qz; Sat, 04 Sep 2021 15:12:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178747.324834; Sat, 04 Sep 2021 15:12:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMXLA-0002Dh-Nq; Sat, 04 Sep 2021 15:12:20 +0000
Received: by outflank-mailman (input) for mailman id 178747;
 Sat, 04 Sep 2021 15:12:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iNAo=N2=unixindia.com=bhasker@srs-us1.protection.inumbo.net>)
 id 1mMXL9-0001n1-DE
 for xen-devel@lists.xenproject.org; Sat, 04 Sep 2021 15:12:19 +0000
Received: from mail-io1-xd2a.google.com (unknown [2607:f8b0:4864:20::d2a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 02e1509f-60cd-4cfd-943a-dd85528f12ed;
 Sat, 04 Sep 2021 15:12:13 +0000 (UTC)
Received: by mail-io1-xd2a.google.com with SMTP id z1so2674159ioh.7
 for <xen-devel@lists.xenproject.org>; Sat, 04 Sep 2021 08:12:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02e1509f-60cd-4cfd-943a-dd85528f12ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=unixindia.com; s=mail;
        h=mime-version:from:date:message-id:subject:to;
        bh=pWt61Hm0SnYMak2zE6TxuNgLfbDZ8EXWGBHuDC9vfNo=;
        b=g88JVThzb2JAnZfM/5MOidhBsqJY+bJvMZgZqXlMWGinkwCFeq6kzUINkKBYJAYZU7
         mYr+f+5Z3gDw+R8VI+K8mdd+Su2OJdYP9clU+oUBjVYw7jyUmY2XBhJe77jtxFoTNOBN
         u42zDC4CPdplnPvdfsOfplyVgUF5p/OzvORvBNJS1uX2yCjbjevnMc2Su6WhxxBQO9hP
         loQVTTSEjLS6GHfh0tcsjbuS3oT2yAaC5zrH1SYWiLrl5PY3xs4bUg+j7tBXCuiHuZs2
         TYCs7RkwHEyklYsItJ0DaTjznHKhm9pJPypxk+DDmuoA4YLPUeEfG9jT2jYmxtVx4gjZ
         R/5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
        bh=pWt61Hm0SnYMak2zE6TxuNgLfbDZ8EXWGBHuDC9vfNo=;
        b=fkPPAmN2Bf4JlMwdCXsI3agwEGYMlTb0cq7nRhUMfJu1Uq6mAaYq2z2WhQqeZRjMV3
         GSGvAGPs6pDLZvwKvpUEwRqqoayjCaVpRsJFkXRn/oS6xYX/Pf9klouEUFj1wB1d9A8s
         RD1Y1tJLTowayLUskZGHVx8hUl5iRg5iVpj0G6RWcyfIv7wxMDvWsUMMsJt6bGp/QCBx
         RIQls3VMJ4cWgnABw5hKavo4AyVvcTPIYGUFPHul5s+iJr/COl2Vw0z5xp2j+i+oGc2q
         nJgD6mDnxpyfQcILb9f+HK3POPty766s5WG+Er0gy6H1DJ0869pdyD5/yqfbFUZ3EHF6
         bi6g==
X-Gm-Message-State: AOAM5331kJ12gHtiq8zcB/WKWXphhgrliELyRmu5Ovg65k2V/3ii4P/w
	wPPHQvZkxozjmPJuvw/hx7lruCUGwY1qY0Cvl8dEX1JwTud3wpVvsfQ=
X-Google-Smtp-Source: ABdhPJwSvIUeRvrvCIQfenu9f5WaY1Iqrvp5MotteTEVGvNe+xr2GDWj0n4ors9QO4Fx1DXafKJtLkQjLOyQDeZ51BI=
X-Received: by 2002:a05:6638:4195:: with SMTP id az21mr3666596jab.40.1630768332265;
 Sat, 04 Sep 2021 08:12:12 -0700 (PDT)
MIME-Version: 1.0
From: Bhasker C V <bhasker@unixindia.com>
Date: Sat, 4 Sep 2021 16:12:01 +0100
Message-ID: <CAPLCSGCkdZMph=iRrqf3G5kT881mN5kYaG=qXrMtWS=fTn2O=g@mail.gmail.com>
Subject: Help with Network SR-IOV (PF still in reset state. Is the PF
 interface up?)
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000027f14e05cb2cd71c"

--00000000000027f14e05cb2cd71c
Content-Type: text/plain; charset="UTF-8"

I run  xen-4.14.3-pre from debian

I created 6 vfs for my igb card
06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network
Connection (rev 01)
I got these
07:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.5 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:11.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)

bcv@bcvibmb1:~$ sudo xl pci-assignable-list
0000:07:10.1

However the guest is not able to see the link up.
bcv@utils:~$ sudo ethtool  enp0s0f1



Settings for enp0s0f1:

        Supported ports: [ ]
...

        Link detected: no

I see this error on the guest

[    6.134890] igbvf 0000:00:00.1: Xen PCI mapped GSI0 to IRQ19

[    6.149577] igbvf 0000:00:00.1: PF still in reset state. Is the PF
interface up?

[    6.149595] igbvf 0000:00:00.1: Assigning random MAC address.



[    6.151552] igbvf 0000:00:00.1: PF still resetting

[    6.153807] igbvf 0000:00:00.1: Intel(R) I350 Virtual Function

[    6.153822] igbvf 0000:00:00.1: Address: 32:91:70:c5:62:bd


and thiis on the host:
[  390.776779] xen-blkback: backend/vbd/3/51713: using 1 queues, protocol 1
(x86_64-abi) persistent grants


[  395.849996] pciback 0000:07:10.1: xen-pciback: Driver tried to write to
a read-only configuration space field at offset 0x1d6, size 2. This may be
harmless, but if you have problems with your device:

[  395.849996] 1) see permissive attribute in sysfs



[  395.849996] 2) report problems to the xen-devel mailing list along with
details of your device obtained from lspci.


[  396.103595] pciback 0000:07:10.1: enabling device (0000 -> 0002)




However if I pass through 07:10.4 it works fine. The host still reports
this  above error but all works well in the guest.


Please could someone help ? I have been using  KVM these years and just
moved to xen and hence new to xen.


Regards
Bhasker C V

--00000000000027f14e05cb2cd71c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+PGRpdj5JIHJ1bsKgIHhlbi00LjE0LjMtcHJlIGZyb20gZGViaWFuPC9k
aXY+PGRpdj48YnI+PC9kaXY+PGRpdj5JIGNyZWF0ZWQgNiB2ZnMgZm9yIG15IGlnYiBjYXJkIDxi
cj48L2Rpdj48ZGl2PjA2OjAwLjAgRXRoZXJuZXQgY29udHJvbGxlcjogSW50ZWwgQ29ycG9yYXRp
b24gSTM1MCBHaWdhYml0IE5ldHdvcmsgQ29ubmVjdGlvbiAocmV2IDAxKTwvZGl2PjxkaXY+SSBn
b3QgdGhlc2U8L2Rpdj48ZGl2PjA3OjEwLjAgRXRoZXJuZXQgY29udHJvbGxlcjogSW50ZWwgQ29y
cG9yYXRpb24gSTM1MCBFdGhlcm5ldCBDb250cm9sbGVyIFZpcnR1YWwgRnVuY3Rpb24gKHJldiAw
MSk8YnI+MDc6MTAuMSBFdGhlcm5ldCBjb250cm9sbGVyOiBJbnRlbCBDb3Jwb3JhdGlvbiBJMzUw
IEV0aGVybmV0IENvbnRyb2xsZXIgVmlydHVhbCBGdW5jdGlvbiAocmV2IDAxKTxicj4wNzoxMC40
IEV0aGVybmV0IGNvbnRyb2xsZXI6IEludGVsIENvcnBvcmF0aW9uIEkzNTAgRXRoZXJuZXQgQ29u
dHJvbGxlciBWaXJ0dWFsIEZ1bmN0aW9uIChyZXYgMDEpPGJyPjA3OjEwLjUgRXRoZXJuZXQgY29u
dHJvbGxlcjogSW50ZWwgQ29ycG9yYXRpb24gSTM1MCBFdGhlcm5ldCBDb250cm9sbGVyIFZpcnR1
YWwgRnVuY3Rpb24gKHJldiAwMSk8YnI+MDc6MTEuMCBFdGhlcm5ldCBjb250cm9sbGVyOiBJbnRl
bCBDb3Jwb3JhdGlvbiBJMzUwIEV0aGVybmV0IENvbnRyb2xsZXIgVmlydHVhbCBGdW5jdGlvbiAo
cmV2IDAxKTxicj4wNzoxMS4xIEV0aGVybmV0IGNvbnRyb2xsZXI6IEludGVsIENvcnBvcmF0aW9u
IEkzNTAgRXRoZXJuZXQgQ29udHJvbGxlciBWaXJ0dWFsIEZ1bmN0aW9uIChyZXYgMDEpPC9kaXY+
PGRpdj48YnI+PC9kaXY+PGRpdj5iY3ZAYmN2aWJtYjE6fiQgc3VkbyB4bCBwY2ktYXNzaWduYWJs
ZS1saXN0PGJyPjAwMDA6MDc6MTAuMTwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+SG93ZXZlciB0
aGUgZ3Vlc3QgaXMgbm90IGFibGUgdG8gc2VlIHRoZSBsaW5rIHVwLiA8YnI+PC9kaXY+PGRpdj5i
Y3ZAdXRpbHM6fiQgc3VkbyBldGh0b29sIMKgZW5wMHMwZjEgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgPGJyPlNldHRpbmdzIGZvciBlbnAwczBmMTogwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqA8YnI+wqAgwqAgwqAgwqAgU3VwcG9ydGVkIHBvcnRzOiBbIF0gwqAgPGJy
PjwvZGl2PjxkaXY+Li4uwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqDCoCA8YnI+wqAgwqAgwqAgwqAgTGluayBkZXRlY3RlZDogbm8gwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgPGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2
PjxkaXY+SSBzZWUgdGhpcyBlcnJvciBvbiB0aGUgZ3Vlc3Q8YnI+PC9kaXY+PGRpdj48YnI+PC9k
aXY+PGRpdj5bIMKgIMKgNi4xMzQ4OTBdIGlnYnZmIDAwMDA6MDA6MDAuMTogWGVuIFBDSSBtYXBw
ZWQgR1NJMCB0byBJUlExOSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA8YnI+WyDCoCDCoDYuMTQ5NTc3XSBpZ2J2ZiAw
MDAwOjAwOjAwLjE6IFBGIHN0aWxsIGluIHJlc2V0IHN0YXRlLiBJcyB0aGUgUEYgaW50ZXJmYWNl
IHVwPyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA8YnI+WyDCoCDCoDYu
MTQ5NTk1XSBpZ2J2ZiAwMDAwOjAwOjAwLjE6IEFzc2lnbmluZyByYW5kb20gTUFDIGFkZHJlc3Mu
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgPGJyPlsgwqAgwqA2LjE1MTU1Ml0gaWdidmYgMDAwMDowMDowMC4xOiBQRiBzdGlsbCBy
ZXNldHRpbmcgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgPGJyPlsgwqAgwqA2LjE1MzgwN10g
aWdidmYgMDAwMDowMDowMC4xOiBJbnRlbChSKSBJMzUwIFZpcnR1YWwgRnVuY3Rpb24gwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgPGJyPlsgwqAgwqA2LjE1MzgyMl0gaWdidmYgMDAwMDowMDowMC4xOiBBZGRyZXNzOiAzMjo5
MTo3MDpjNTo2MjpiZCDCoCDCoCA8YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48YnI+PC9k
aXY+PGRpdj5hbmQgdGhpaXMgb24gdGhlIGhvc3Q6PC9kaXY+PGRpdj5bIMKgMzkwLjc3Njc3OV0g
eGVuLWJsa2JhY2s6IGJhY2tlbmQvdmJkLzMvNTE3MTM6IHVzaW5nIDEgcXVldWVzLCBwcm90b2Nv
bCAxICh4ODZfNjQtYWJpKSBwZXJzaXN0ZW50IGdyYW50cyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDxicj5bIMKgMzk1Ljg0OTk5Nl0gcGNp
YmFjayAwMDAwOjA3OjEwLjE6IHhlbi1wY2liYWNrOiBEcml2ZXIgdHJpZWQgdG8gd3JpdGUgdG8g
YSByZWFkLW9ubHkgY29uZmlndXJhdGlvbiBzcGFjZSBmaWVsZCBhdCBvZmZzZXQgMHgxZDYsIHNp
emUgMi4gVGhpcyBtYXkgYmUgaGFybWxlc3MsIGJ1dCBpZiB5b3UgaGF2ZSBwcm9ibGVtcyB3aXRo
IHlvdXIgZGV2aWNlOiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDxicj5bIMKgMzk1Ljg0OTk5
Nl0gMSkgc2VlIHBlcm1pc3NpdmUgYXR0cmlidXRlIGluIHN5c2ZzIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIDxicj5bIMKgMzk1Ljg0OTk5Nl0gMikgcmVwb3J0IHByb2JsZW1zIHRvIHRoZSB4ZW4t
ZGV2ZWwgbWFpbGluZyBsaXN0IGFsb25nIHdpdGggZGV0YWlscyBvZiB5b3VyIGRldmljZSBvYnRh
aW5lZCBmcm9tIGxzcGNpLiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDxi
cj5bIMKgMzk2LjEwMzU5NV0gcGNpYmFjayAwMDAwOjA3OjEwLjE6IGVuYWJsaW5nIGRldmljZSAo
MDAwMCAtJmd0OyAwMDAyKSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgIDxicj48L2Rpdj48ZGl2
Pjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pkhvd2V2ZXIgaWYgSSBwYXNzIHRocm91Z2gg
MDc6MTAuNCBpdCB3b3JrcyBmaW5lLiBUaGUgaG9zdCBzdGlsbCByZXBvcnRzIHRoaXPCoCBhYm92
ZSBlcnJvciBidXQgYWxsIHdvcmtzIHdlbGwgaW4gdGhlIGd1ZXN0LiA8YnI+PC9kaXY+PGRpdj48
YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5QbGVhc2UgY291bGQgc29tZW9uZSBoZWxwID8g
SSBoYXZlIGJlZW4gdXNpbmfCoCBLVk0gdGhlc2UgeWVhcnMgYW5kIGp1c3QgbW92ZWQgdG8geGVu
IGFuZCBoZW5jZSBuZXcgdG8geGVuLiA8YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48YnI+
PC9kaXY+PGRpdj5SZWdhcmRzPC9kaXY+PGRpdj5CaGFza2VyIEMgViA8YnI+PC9kaXY+PC9kaXY+
DQo=
--00000000000027f14e05cb2cd71c--


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 16:00:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 16:00:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178774.324845 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMY5f-0000TH-JO; Sat, 04 Sep 2021 16:00:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178774.324845; Sat, 04 Sep 2021 16:00:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMY5f-0000TA-GR; Sat, 04 Sep 2021 16:00:23 +0000
Received: by outflank-mailman (input) for mailman id 178774;
 Sat, 04 Sep 2021 16:00:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMY5e-0000T0-Fa; Sat, 04 Sep 2021 16:00:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMY5e-0007hq-9r; Sat, 04 Sep 2021 16:00:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMY5d-0006lP-VM; Sat, 04 Sep 2021 16:00:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMY5d-0001HQ-Un; Sat, 04 Sep 2021 16:00:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=q3+FfTdm1kQmBM3m39fmMOD8YsgQcNKDLtjj7gPBDvs=; b=3a3rmehfQtK2K1QINqKzTc8vHA
	eUNZt1IbkJ7129v0Vvt+EOaUiPjkTEyY9ShSkPOAyoojNXWT9QAWV80BIL7+mpt6FvbHrbSJOhjz8
	vxnh6WlnrSsnLwga97kaqLQ6AZb1iwk2JW3kbY9T8EZuGHAvV94G0Lco1b9Btmn60/LQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164807-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164807: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 16:00:21 +0000

flight 164807 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164807/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   28 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 16:46:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 16:46:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178782.324859 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMYo3-0005T3-5B; Sat, 04 Sep 2021 16:46:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178782.324859; Sat, 04 Sep 2021 16:46:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMYo3-0005Sw-1Y; Sat, 04 Sep 2021 16:46:15 +0000
Received: by outflank-mailman (input) for mailman id 178782;
 Sat, 04 Sep 2021 16:46:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMYo1-0005Sm-W6; Sat, 04 Sep 2021 16:46:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMYo1-0008S1-QY; Sat, 04 Sep 2021 16:46:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMYo1-0000Q7-GW; Sat, 04 Sep 2021 16:46:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMYo1-0002ta-Fu; Sat, 04 Sep 2021 16:46:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=AwVyevIkjJoT6zcKO5brXg2C3X9huyq5lS+YkCe8IFk=; b=b6izB5v3G9PsDzQAdmb/ZG48+p
	DDD+TB0gaChMtWbLMleQRZtJa3ii5wu0uKrptzgZm9pn3nGbTmh7oDddfld+bDzsH7e3UxDOPFVXF
	+Sy7XJ6fLSSDltwqvJIYb8xCT0PpTS2TwQXnnosC9wzD+7MrDNArM5v3X1sPRJy7rv24=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164809-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164809: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 16:46:13 +0000

flight 164809 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164809/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   29 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 17:59:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 17:59:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178793.324873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMZwM-00053l-Jp; Sat, 04 Sep 2021 17:58:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178793.324873; Sat, 04 Sep 2021 17:58:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMZwM-00053e-Gn; Sat, 04 Sep 2021 17:58:54 +0000
Received: by outflank-mailman (input) for mailman id 178793;
 Sat, 04 Sep 2021 17:58:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMZwL-00053T-97; Sat, 04 Sep 2021 17:58:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMZwL-0001Ck-4l; Sat, 04 Sep 2021 17:58:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMZwK-0003zM-TO; Sat, 04 Sep 2021 17:58:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMZwK-00071R-Sp; Sat, 04 Sep 2021 17:58:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HB9mNXQmN/WgFWwkPdT9tYaWiu8gr7t+ts0tREWVgRo=; b=1HXsyaJeNgLYtZAjZWewfB/Nm0
	ZYTEfn6G/uzd8GBsY7V+VjYoRCAYKnDOK2ZkGpzIAlFepq4TQojdGlQkqQ1PJhMEEKbl8l+OxeKia
	GojuNRqkhELdilRW5NJFCcQnV1d409HHKoNfcb9xAg50ioHBIvjSmRQgY4yPrMhLtSGU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164810-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164810: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 17:58:52 +0000

flight 164810 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164810/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   30 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 18:30:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 18:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178801.324887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMaQq-0001CV-2d; Sat, 04 Sep 2021 18:30:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178801.324887; Sat, 04 Sep 2021 18:30:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMaQp-0001CO-Vj; Sat, 04 Sep 2021 18:30:23 +0000
Received: by outflank-mailman (input) for mailman id 178801;
 Sat, 04 Sep 2021 18:30:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMaQo-0001CE-N4; Sat, 04 Sep 2021 18:30:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMaQo-0001q2-ID; Sat, 04 Sep 2021 18:30:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMaQo-0004t5-8K; Sat, 04 Sep 2021 18:30:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMaQo-0000fy-7r; Sat, 04 Sep 2021 18:30:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=GDQ5HNgOXvGnGl+eo+lPZdzeyV57exp7uq77fJPi3BA=; b=StZdsILmMtvMvQSJ977nFumLLP
	xATeaYeL0Bpp7IzFVpQP3jNYK3xWTOtZ+zcLQZuAChGlVN4AaPQDoI+I/dkdPqW+XXCdKEDA7kQk7
	j9i9K1jz7mQN0sMJUNo+8fIBk6fcidkrFzqXz3jFOsOUr3or9oV55j0/yX5EWTC6NeS4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164811-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164811: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 18:30:22 +0000

flight 164811 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164811/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   31 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 18:45:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 18:45:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178808.324901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMafF-00035K-E5; Sat, 04 Sep 2021 18:45:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178808.324901; Sat, 04 Sep 2021 18:45:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMafF-00035D-B4; Sat, 04 Sep 2021 18:45:17 +0000
Received: by outflank-mailman (input) for mailman id 178808;
 Sat, 04 Sep 2021 18:45:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMafE-000353-4J; Sat, 04 Sep 2021 18:45:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMafD-00024k-Ob; Sat, 04 Sep 2021 18:45:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMafD-0005HZ-DQ; Sat, 04 Sep 2021 18:45:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMafD-00058B-D0; Sat, 04 Sep 2021 18:45:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=A7dIeSiEAANhcoTXS32pJzlQXY/UJCootyM8uL/n2ug=; b=Byhock5rhDA36ZiPBD6rHNp8u3
	JNNk5CZk7xYl16e8MSm4ZisO879JooajO5KkyuzTARwRsu23OjmNWXP0r9uiqRQKU4G2t9PCSRizi
	02cse2n9bJklsSpQovf+COz37oxtjGlFZzChWVG8jjkM45cKSF5khvlGW4fplbxJRsZ0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164805-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164805: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 18:45:15 +0000

flight 164805 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164805/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail pass in 164798

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164798
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164798
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164798
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164798
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164798
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164798
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164798
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164798
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164798
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164798
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164798
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164805  2021-09-04 07:50:13 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sat Sep 04 19:55:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 19:55:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178816.324915 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMblF-0002WM-Td; Sat, 04 Sep 2021 19:55:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178816.324915; Sat, 04 Sep 2021 19:55:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMblF-0002WF-PT; Sat, 04 Sep 2021 19:55:33 +0000
Received: by outflank-mailman (input) for mailman id 178816;
 Sat, 04 Sep 2021 19:55:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMblE-0002W5-Gi; Sat, 04 Sep 2021 19:55:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMblE-0003Mm-Bu; Sat, 04 Sep 2021 19:55:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMblE-0007YM-3c; Sat, 04 Sep 2021 19:55:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMblE-0007pU-32; Sat, 04 Sep 2021 19:55:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NKiU4iFKWN+jJqM36rnU6s02nGmpDXUAt9myllV5FdI=; b=28YSqCtezJ6SehNdpYh9D6xyrK
	E2w7Klo1kyFt7uUBJeAzbMl0NLVP0bIphgMdIgB10Y8TUfjNYXX4Rr4I4t+XveH8XnjFFpX7ugsGP
	ItuOEEx2advRe2lJ3OiJ7dJPzWmAE4uYfkK6QZ6hS1KkMTZD00Lv0jrybppLopYplQYo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164813-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164813: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 19:55:32 +0000

flight 164813 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164813/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   32 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 21:12:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 21:12:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178828.324929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMcxf-00038d-RC; Sat, 04 Sep 2021 21:12:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178828.324929; Sat, 04 Sep 2021 21:12:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMcxf-00038W-OB; Sat, 04 Sep 2021 21:12:27 +0000
Received: by outflank-mailman (input) for mailman id 178828;
 Sat, 04 Sep 2021 21:12:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMcxf-00038M-84; Sat, 04 Sep 2021 21:12:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMcxe-0004jd-Pw; Sat, 04 Sep 2021 21:12:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMcxe-0001kF-Gy; Sat, 04 Sep 2021 21:12:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMcxe-0007HK-GR; Sat, 04 Sep 2021 21:12:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=S4QbXQrJ6ct940T99uQdSQC4rlnJHB8hF68DdyRzobw=; b=PE8gZC4L/A5Tf0NlPTjuBxo4LX
	NYic5TayfoYqX3M86E2Lu2Kh5m/WglkgavcxeiRY8KWXxZISw+Ywm55u7j0rf7vEYEe8Vh4JvjQuV
	tHoz0y67H60c9V2Lqr7XzGckOsQyX1i1orlDzY0K7qOrLq6dwgGsTLiLLOb5y+mGXiPY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164814-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164814: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 21:12:26 +0000

flight 164814 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164814/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   33 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    7 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 21:50:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 21:50:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178836.324943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMdYA-0007fP-OV; Sat, 04 Sep 2021 21:50:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178836.324943; Sat, 04 Sep 2021 21:50:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMdYA-0007fI-LM; Sat, 04 Sep 2021 21:50:10 +0000
Received: by outflank-mailman (input) for mailman id 178836;
 Sat, 04 Sep 2021 21:50:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMdY9-0007ep-Hx; Sat, 04 Sep 2021 21:50:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMdY9-0005L6-9J; Sat, 04 Sep 2021 21:50:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMdY8-00038R-V6; Sat, 04 Sep 2021 21:50:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMdY8-0003RK-Uc; Sat, 04 Sep 2021 21:50:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aI51K37+Hebz6ZjITKztpH420BB54RTCivzXOfkg/xE=; b=Iq+0CFIYiHNQjIOHEhnSiqkj2t
	nqGZbovzElJAuFJOe55PfvehABjmLW3J67+N9bibjMcyeu8HaQCfpz4OF7LMwi4r6qOeRTJj4Xf3M
	6IjIXH+Y4d0KyLRP/Fw0MTQdySDLqXjg0f/rWQfc+JLuZNqAz4OPpAm9qd+DAYMbXXR4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164808-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164808: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-credit2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f1583cb1be35c23df60b1c39e3e7e6704d749d0b
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 21:50:08 +0000

flight 164808 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164808/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-shadow 22 guest-start/debian.repeat fail in 164801 pass in 164808
 test-amd64-amd64-examine    4 memdisk-try-append fail in 164801 pass in 164808
 test-amd64-amd64-xl-credit2  22 guest-start/debian.repeat  fail pass in 164801

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 164801 like 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                f1583cb1be35c23df60b1c39e3e7e6704d749d0b
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  400 days
Failing since        152366  2020-08-01 20:49:34 Z  399 days  692 attempts
Testing same since   164801  2021-09-04 01:12:40 Z    0 days    2 attempts

------------------------------------------------------------
7203 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2211696 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 04 23:33:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 04 Sep 2021 23:33:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178844.324956 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMf9n-0002BL-FO; Sat, 04 Sep 2021 23:33:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178844.324956; Sat, 04 Sep 2021 23:33:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMf9n-0002BE-CQ; Sat, 04 Sep 2021 23:33:07 +0000
Received: by outflank-mailman (input) for mailman id 178844;
 Sat, 04 Sep 2021 23:33:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMf9m-0002B4-00; Sat, 04 Sep 2021 23:33:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMf9l-0006zh-Ox; Sat, 04 Sep 2021 23:33:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMf9l-0008AY-Hh; Sat, 04 Sep 2021 23:33:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMf9l-00046C-HA; Sat, 04 Sep 2021 23:33:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HuBX95klywY3zgntBcarUM5bpGmFXwEqmQCoRV4ULL4=; b=g/j26je5Vvs9FKj7Qk2SQWI0PG
	ynp0a/9WTnMtoVsgxohhIspGX383x+LuYD04l0CjV9MCi/DX2iiGZFi0925wtnGlCCDqoseQvL3Zz
	Cv8OeQ40syCYqq11J+ED0BQjw8hfoBdMVOJXSoI+kcutQW96Z9FORsr076WTx8eZbiiI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164815-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164815: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 04 Sep 2021 23:33:05 +0000

flight 164815 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164815/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   34 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    8 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 00:28:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 00:28:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178854.324970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMg1I-0000F7-AO; Sun, 05 Sep 2021 00:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178854.324970; Sun, 05 Sep 2021 00:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMg1I-0000F0-7N; Sun, 05 Sep 2021 00:28:24 +0000
Received: by outflank-mailman (input) for mailman id 178854;
 Sun, 05 Sep 2021 00:28:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMg1H-0000Eq-N0; Sun, 05 Sep 2021 00:28:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMg1H-0008Vc-I8; Sun, 05 Sep 2021 00:28:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMg1H-0003rz-AI; Sun, 05 Sep 2021 00:28:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMg1H-0005ZG-9l; Sun, 05 Sep 2021 00:28:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=VCdKH9Ao3bzA3bm4VVT/EmzYfL0np/NifRsqL2VGpK8=; b=GkMVYYOhUKjJoa/injtdoYqHaa
	hqlBwy7v57m6B9dpM6hMoeECw669q8E877+NxH728TQgGZd4LLmac0ySg3IjRbMY5r7xazGVRITBI
	+ba5W9PtxodpuqchMpqt1lr02505mScXhzRgBJADTR9+c4x5baXpG6gvfpAd5qWkBSCo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164817-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164817: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 00:28:23 +0000

flight 164817 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164817/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   35 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days    9 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 01:41:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 01:41:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178861.324985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMhA4-0007PL-SJ; Sun, 05 Sep 2021 01:41:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178861.324985; Sun, 05 Sep 2021 01:41:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMhA4-0007PE-P2; Sun, 05 Sep 2021 01:41:32 +0000
Received: by outflank-mailman (input) for mailman id 178861;
 Sun, 05 Sep 2021 01:41:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMhA3-0007P4-SE; Sun, 05 Sep 2021 01:41:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMhA3-0007Xv-Lr; Sun, 05 Sep 2021 01:41:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMhA2-0006C1-K5; Sun, 05 Sep 2021 01:41:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMhA2-00076D-Jd; Sun, 05 Sep 2021 01:41:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MU8m52ijS5WrhVsxtKKd3HoNjysbbjeIIJhAx9iRDtg=; b=aC5ry0AvVNMke2QVjJkE9ulHom
	+zJHc86zMcU8gt4w2o32QWQPMK6yceb17CFBV7ZjDnTeOJRwuw4RP48i8E5vtKWnnx5H0hkEUQYYL
	/JJTnZBTc9VtwckBlgsYNGKcRMJ7B9FaEU6e+gDJfU6GLEF0nOlcMaa3zcK8c3WpIywc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164818-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164818: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 01:41:30 +0000

flight 164818 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164818/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   36 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days   10 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 02:31:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 02:31:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178869.324999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMhvv-0004wK-OG; Sun, 05 Sep 2021 02:30:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178869.324999; Sun, 05 Sep 2021 02:30:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMhvv-0004vs-Hc; Sun, 05 Sep 2021 02:30:59 +0000
Received: by outflank-mailman (input) for mailman id 178869;
 Sun, 05 Sep 2021 02:30:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=af6+=N3=unixindia.com=bhasker@srs-us1.protection.inumbo.net>)
 id 1mMhvt-0004vm-OJ
 for xen-devel@lists.xenproject.org; Sun, 05 Sep 2021 02:30:57 +0000
Received: from mail-io1-xd30.google.com (unknown [2607:f8b0:4864:20::d30])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 61eaba39-4c97-49ab-9a2d-50a97823c5c7;
 Sun, 05 Sep 2021 02:30:56 +0000 (UTC)
Received: by mail-io1-xd30.google.com with SMTP id b10so4020459ioq.9
 for <xen-devel@lists.xenproject.org>; Sat, 04 Sep 2021 19:30:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61eaba39-4c97-49ab-9a2d-50a97823c5c7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=unixindia.com; s=mail;
        h=mime-version:from:date:message-id:subject:to;
        bh=9GhxT6iwrAUFJk5X6XPrE5T9ku0+A2eEn9VbjHQLQIo=;
        b=wD34T0Gvd/G8aM6SWzrcuQIBuyaUkEE5aP2179vfJDvOPiIIEYbEniiZSWqCH7VYcy
         sEE5wMVn1gp7c6rFEiTC7Jd8svNigMfs/8BvnLyFPKvZyYH4iPWu+a2lDcveeMi1NDAX
         rnGA4JMIeko96oI3krWpHZ2npAQuvw1xzlu14cGNveGMQRcKgGun480dONIzkIc/QS5c
         NKdPYTApOsFRxDDeLmO2VN4XZotAxi2BcUBX0lm4FpL26dMShXT5CdCxwP4eLz9yI/AD
         FwWRJ0Ja0iOo4UZPAbZTAMoNJoxW70/ZYcMXz0ZIle06Sosv12kjBjqB7S+A7XbSRz87
         abPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
        bh=9GhxT6iwrAUFJk5X6XPrE5T9ku0+A2eEn9VbjHQLQIo=;
        b=WtG97C1xZiMU0klDUrJzDwvj3XEYdhzpCScGsdjhlvVI7LymvqBzwlhWAcml9qI+Uq
         TnPuJiUyC0J1s71c0jvH4tpSfHEuZYlPy7yMXRWkm562H1pFT9yueYlJ4Mw7ljDzKy5j
         vcH0D4R8BXsNO1wTNGQExA8YvKYAuZG/iIkVExhzc4maHuyg0x7RKjZElKnuFyQCi1Vt
         s9PZ73U8lz0ovhTal1iC1txlurLLMvvIfKXtUYsrARYkkgI3h0PB5hGAdW9jPbq10Q8l
         1Qt1uvX3l15cdsVq46YznWN1HxgWK74g9lulk0tMazUmlj/ZwdH90+HpmtGZfgoq2JrV
         bj/Q==
X-Gm-Message-State: AOAM532IwbA/LAVoFxYQgkYAgeVn/hqEh1+AMRUp3eY3NQxZxajkF7zi
	IwSQApI4h/KTblG4LPoflxHt9NWg9HBNklpvnvw3Nv0e8HYBz2CR560=
X-Google-Smtp-Source: ABdhPJwdKE+lU/CkGaWvd3SU3J4iNHihj1Pae38TvnF0g7XLrRJhQ9OIpnsmKu6R3H/ZJowz7UtGLPagw3hxNWODII8=
X-Received: by 2002:a02:1dc9:: with SMTP id 192mr5354954jaj.92.1630809055894;
 Sat, 04 Sep 2021 19:30:55 -0700 (PDT)
MIME-Version: 1.0
From: Bhasker C V <bhasker@unixindia.com>
Date: Sun, 5 Sep 2021 03:30:45 +0100
Message-ID: <CAPLCSGD+Tw3M1GcU6thqE4YiN6bWqJVqwc1YPPL7-rNb63J2pw@mail.gmail.com>
Subject: Pygrub crash when trying to edit kernel commandline
To: xen-users@lists.xenproject.org, xen-devel@lists.xenproject.org
Content-Type: multipart/mixed; boundary="0000000000007950e905cb36526c"

--0000000000007950e905cb36526c
Content-Type: multipart/alternative; boundary="0000000000007950e705cb36526a"

--0000000000007950e705cb36526a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi
 When I try to edit kernel command line of domU, pygrub crashes

bcv@bcvibmb1:~$ sudo xl create -c /etc/xen/utils.cfg
Parsing config from /etc/xen/utils.cfg
Using <class 'grub.GrubConf.GrubConfigFile'> to parse /boot/grub/menu.lst

    pyGRUB  version 0.6
 =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=90
 =E2=94=82 root            (hd0,0)                                         =
       =E2=94=82
 =E2=94=82 kernel          /boot/vmlinuz-4.19.0-17-amd64 root=3D/dev/xvda2 =
ro elev> =E2=94=82
 =E2=94=82 initrd          /boot/initrd.img-4.19.0-17-amd64                =
       =E2=94=82
 =E2=94=82                                                                 =
       =E2=94=82
 =E2=94=82                                                                 =
       =E2=94=82
 =E2=94=82                                                                 =
       =E2=94=82
 =E2=94=82                                                                 =
       =E2=94=82
 =E2=94=82                                                                 =
       =E2=94=82
 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98
     Use the =E2=86=91 and =E2=86=93 keys to select which entry is highligh=
ted.
     Press 'b' to boot, 'e' to edit the selected command in the
     boot sequence, 'c' for a command-line, 'o' to open a new line
     after ('O' for before) the selected line, 'd' to remove the
     selected line, or escape to go back to the main menu.


Traceback (most recent call last):
                                    File "/usr/lib/xen-4.14/bin/pygrub",
line 884, in <module>

                      raise RuntimeError("Unable to find partition
containing kernel")
             RuntimeError: Unable to find partition containing kernel
                                                                     libxl:
error: libxl_bootloader.c:648:bootloader_finished: Domain 4:bootloader
failed - consult logfile /var/log/xen/bootloader.4.log
                                                    libxl: error:
libxl_exec.c:117:libxl_report_child_exitstatus: bootloader [1316] exited
with error status 1
         libxl: error: libxl_create.c:1576:domcreate_rebuild_done: Domain
4:cannot (re-)build domain: -3

                            libxl: error:
libxl_domain.c:1182:libxl__destroy_domid: Domain 4:Non-existant domain
libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain 4:Unable
to destroy guest
libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain 4:Destruction
of domain failed
bcv@bcvibmb1:~$


Attached is the log
Am I doing something wrong ? or is this a known issue ?

Thanks
Bhasker C V

--0000000000007950e705cb36526a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+PGRpdj5IaTwvZGl2PjxkaXY+wqBXaGVuIEkgdHJ5IHRvIGVkaXQga2Vy
bmVsIGNvbW1hbmQgbGluZSBvZiBkb21VLCBweWdydWIgY3Jhc2hlczwvZGl2PjxkaXY+PHNwYW4g
c3R5bGU9ImNvbG9yOnJnYig2MSwxMzMsMTk4KSI+PGJyPjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFu
IHN0eWxlPSJjb2xvcjpyZ2IoNjEsMTMzLDE5OCkiPmJjdkBiY3ZpYm1iMTp+JCBzdWRvIHhsIGNy
ZWF0ZSAtYyAvZXRjL3hlbi91dGlscy5jZmc8YnI+UGFyc2luZyBjb25maWcgZnJvbSAvZXRjL3hl
bi91dGlscy5jZmc8YnI+VXNpbmcgJmx0O2NsYXNzICYjMzk7Z3J1Yi5HcnViQ29uZi5HcnViQ29u
ZmlnRmlsZSYjMzk7Jmd0OyB0byBwYXJzZSAvYm9vdC9ncnViL21lbnUubHN0PGJyPjxicj7CoCDC
oCBweUdSVUIgwqB2ZXJzaW9uIDAuNjxicj7CoOKUjOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUkDxicj7CoOKUgiByb290IMKgIMKgIMKgIMKgIMKgIMKgKGhk
MCwwKSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoOKUgjxicj7CoOKUgiBrZXJuZWwgwqAgwqAgwqAgwqAgwqAvYm9v
dC92bWxpbnV6LTQuMTkuMC0xNy1hbWQ2NCByb290PS9kZXYveHZkYTIgcm8gZWxldiZndDsg4pSC
PGJyPsKg4pSCIGluaXRyZCDCoCDCoCDCoCDCoCDCoC9ib290L2luaXRyZC5pbWctNC4xOS4wLTE3
LWFtZDY0IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIOKUgjxicj7CoOKUgiDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoOKUgjxicj7CoOKU
giDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoOKUgjxi
cj7CoOKUgiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oOKUgjxicj7CoOKUgiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoOKUgjxicj7CoOKUgiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoOKUgjxicj7CoOKUlOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUmDxicj7CoCDCoCDCoFVzZSB0aGUg4oaRIGFuZCDihpMga2V5cyB0byBz
ZWxlY3Qgd2hpY2ggZW50cnkgaXMgaGlnaGxpZ2h0ZWQuPGJyPsKgIMKgIMKgUHJlc3MgJiMzOTti
JiMzOTsgdG8gYm9vdCwgJiMzOTtlJiMzOTsgdG8gZWRpdCB0aGUgc2VsZWN0ZWQgY29tbWFuZCBp
biB0aGU8YnI+wqAgwqAgwqBib290IHNlcXVlbmNlLCAmIzM5O2MmIzM5OyBmb3IgYSBjb21tYW5k
LWxpbmUsICYjMzk7byYjMzk7IHRvIG9wZW4gYSBuZXcgbGluZTxicj7CoCDCoCDCoGFmdGVyICgm
IzM5O08mIzM5OyBmb3IgYmVmb3JlKSB0aGUgc2VsZWN0ZWQgbGluZSwgJiMzOTtkJiMzOTsgdG8g
cmVtb3ZlIHRoZTxicj7CoCDCoCDCoHNlbGVjdGVkIGxpbmUsIG9yIGVzY2FwZSB0byBnbyBiYWNr
IHRvIHRoZSBtYWluIG1lbnUuPGJyPjxicj48YnI+VHJhY2ViYWNrIChtb3N0IHJlY2VudCBjYWxs
IGxhc3QpOjxicj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCBGaWxlICZxdW90Oy91c3IvbGliL3hlbi00LjE0L2Jpbi9weWdydWImcXVvdDssIGxp
bmUgODg0LCBpbiAmbHQ7bW9kdWxlJmd0Ozxicj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCByYWlzZSBSdW50aW1lRXJyb3IoJnF1b3Q7VW5hYmxlIHRvIGZpbmQgcGFydGl0aW9uIGNv
bnRhaW5pbmcga2VybmVsJnF1b3Q7KTxicj7CoCDCoCDCoCDCoCDCoCDCoCDCoFJ1bnRpbWVFcnJv
cjogVW5hYmxlIHRvIGZpbmQgcGFydGl0aW9uIGNvbnRhaW5pbmcga2VybmVsPGJyPsKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgbGlieGw6IGVycm9yOiBsaWJ4
bF9ib290bG9hZGVyLmM6NjQ4OmJvb3Rsb2FkZXJfZmluaXNoZWQ6IERvbWFpbiA0OmJvb3Rsb2Fk
ZXIgZmFpbGVkIC0gY29uc3VsdCBsb2dmaWxlIC92YXIvbG9nL3hlbi9ib290bG9hZGVyLjQubG9n
PGJyPsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGxpYnhsOiBlcnJvcjogbGlieGxfZXhlYy5jOjExNzps
aWJ4bF9yZXBvcnRfY2hpbGRfZXhpdHN0YXR1czogYm9vdGxvYWRlciBbMTMxNl0gZXhpdGVkIHdp
dGggZXJyb3Igc3RhdHVzIDE8YnI+wqAgwqAgwqAgwqAgwqBsaWJ4bDogZXJyb3I6IGxpYnhsX2Ny
ZWF0ZS5jOjE1NzY6ZG9tY3JlYXRlX3JlYnVpbGRfZG9uZTogRG9tYWluIDQ6Y2Fubm90IChyZS0p
YnVpbGQgZG9tYWluOiAtMzxicj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCBsaWJ4bDogZXJyb3I6IGxpYnhsX2RvbWFpbi5jOjExODI6bGlieGxfX2Rlc3Ryb3lfZG9t
aWQ6IERvbWFpbiA0Ok5vbi1leGlzdGFudCBkb21haW48YnI+bGlieGw6IGVycm9yOiBsaWJ4bF9k
b21haW4uYzoxMTM2OmRvbWFpbl9kZXN0cm95X2NhbGxiYWNrOiBEb21haW4gNDpVbmFibGUgdG8g
ZGVzdHJveSBndWVzdDxicj5saWJ4bDogZXJyb3I6IGxpYnhsX2RvbWFpbi5jOjEwNjM6ZG9tYWlu
X2Rlc3Ryb3lfY2I6IERvbWFpbiA0OkRlc3RydWN0aW9uIG9mIGRvbWFpbiBmYWlsZWQ8YnI+YmN2
QGJjdmlibWIxOn4kwqA8L3NwYW4+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+
PGRpdj48L2Rpdj48ZGl2PkF0dGFjaGVkIGlzIHRoZSBsb2c8L2Rpdj48ZGl2PkFtIEkgZG9pbmcg
c29tZXRoaW5nIHdyb25nID8gb3IgaXMgdGhpcyBhIGtub3duIGlzc3VlID88L2Rpdj48ZGl2Pjxi
cj48L2Rpdj48ZGl2PlRoYW5rczwvZGl2PjxkaXY+Qmhhc2tlciBDIFYgPGJyPjwvZGl2PjwvZGl2
Pg0K
--0000000000007950e705cb36526a--
--0000000000007950e905cb36526c
Content-Type: text/x-log; charset="x-binaryenc"; name="bootloader.4.log"
Content-Disposition: attachment; filename="bootloader.4.log"
Content-Transfer-Encoding: base64
Content-ID: <f_kt6ll68k0>
X-Attachment-Id: f_kt6ll68k0

VXNpbmcgPGNsYXNzICdncnViLkdydWJDb25mLkdydWJDb25maWdGaWxlJz4gdG8gcGFyc2UgL2Jv
b3QvZ3J1Yi9tZW51LmxzdAobKEIbKTAbWzE7MjRyG1ttDxtbPzdoG1s/MWgbPRtbSBtbShtbPzFo
Gz0bWzFCICAgIHB5R1JVQiAgdmVyc2lvbiAwLjYNG1sxQiAbWzBtDmxxcXFxcXFxcXFxcXFxcXFx
cXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFr
Dw0bWzFCIBtbMG0OeA8gG1swOzdtD0RlYmlhbiBHTlUvTGludXgsIGtlcm5lbCA0LjE5LjAtMTct
YW1kNjQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAbW20PIBtbMG0OeA8NG1sxQiAbWzBt
DngPIERlYmlhbiBHTlUvTGludXgsIGtlcm5lbCA0LjE5LjAtMTctYW1kNjQgKHNpbmdsZS11c2Vy
IG1vZGUpG1sxMkMbWzBtDngPDRtbMUIgG1swbQ54DyBEZWJpYW4gR05VL0xpbnV4IDEwG1s1MkMb
WzBtDngPDRtbMUIgG1swbQ54DyBEZWJpYW4gR05VL0xpbnV4IDEwIChTaW5nbGUtVXNlcikbWzM4
QxtbMG0OeA8NG1sxQiAbWzBtDngPIERlYmlhbiBHTlUvTGludXggMTAgKERlZmF1bHQgS2VybmVs
KRtbMzVDG1swbQ54Dw0bWzFCIBtbMG0OeA8gRGViaWFuIEdOVS9MaW51eCAxMCAoRGVmYXVsdCBL
ZXJuZWwsIFNpbmdsZS1Vc2VyKRtbMjJDG1swbQ54Dw0bWzFCIBtbMG0OeA8bWzcyQxtbMG0OeA8N
G1sxQiAbWzBtDngPG1s3MkMbWzBtDngPDRtbMUIgG1swbQ5tcXFxcXFxcXFxcXFxcXFxcXFxcXFx
cXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxag8bWzFC
G1s3MEQbWzBtDg9Vc2UgdGhlIOKGkSBhbmQg4oaTIGtleXMgdG8gc2VsZWN0IHdoaWNoIGVudHJ5
IGlzIGhpZ2hsaWdodGVkLhtbMUIbWzU4RFByZXNzIGVudGVyIHRvIGJvb3QgdGhlIHNlbGVjdGVk
IE9TLCAnZScgdG8gZWRpdCB0aGUbWzFCG1s1MkRjb21tYW5kcyBiZWZvcmUgYm9vdGluZywgJ2En
IHRvIG1vZGlmeSB0aGUga2VybmVsIGFyZ3VtZW50cxtbMUIbWzU5RGJlZm9yZSBib290aW5nLCBv
ciAnYycgZm9yIGEgY29tbWFuZCBsaW5lLhtbN0EbWzI2QxtbMTJCG1s2OERXaWxsIGJvb3Qgc2Vs
ZWN0ZWQgZW50cnkgaW4gIDIgc2Vjb25kcxtbPzFoGz0NG1tKG1sxMkEbWzczQxtbMTJCG1s2OERX
aWxsIGJvb3Qgc2VsZWN0ZWQgZW50cnkgaW4gIDEgc2Vjb25kcxtbPzFoGz0NG1tKG1sxN0EbWzND
RGViaWFuIEdOVS9MaW51eCwga2VybmVsIDQuMTkuMC0xNy1hbWQ2NCAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIBtbMUIbWzcwRBtbMDs3bQ9EZWJpYW4gR05VL0xpbnV4LCBrZXJuZWwgNC4x
OS4wLTE3LWFtZDY0IChzaW5nbGUtdXNlciBtb2RlKSAgICAgICAgICAgG1s0QhtbbQ8bWzEzQhtb
NjhEG1s/MWgbPRtbMThBCAgbWzA7N20PRGViaWFuIEdOVS9MaW51eCwga2VybmVsIDQuMTkuMC0x
Ny1hbWQ2NCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIBtbMUIbWzcwRBtbbQ9EZWJpYW4g
R05VL0xpbnV4LCBrZXJuZWwgNC4xOS4wLTE3LWFtZDY0IChzaW5nbGUtdXNlciBtb2RlKSAgICAg
ICAgICAgG1s0QhtbMTNCG1s2OEQbWz8xaBs9G1sxOEEICBtbMDs3bQ9yb290ICAgICAgICAgICAg
KGhkMCwwKSAgICAgICAgICAgICAgICAgG1sxQhtbNDBEG1ttD2tlcm5lbCAgICAgICAgICAvYm9v
dC92bWxpbnV6LTQuMTkuMC0xNy1hbWQ2NCByb290PS9kZXYveHZkYTIgcm8gZWxldj4bWzFCG1s3
MERpbml0cmQgICAgICAgICAgL2Jvb3QvaW5pdHJkLmltZy00LjE5LjAtMTctYW1kNjQbWzFCG1s0
OEQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAbWzFCG1szM0QgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAbWzFCG1szNkQgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgG1sxNDsxMkgnYicgdG8gYm9vdCwgJ2UnIHRvIGVkaXQg
dGhlIHNlbGVjdGVkIGNvbW1hbmQgaW4gdGhlG1sxQhtbNThEYm9vdCBzZXF1ZW5jZSwgJ2MnIGZv
ciBhIGNvbW1hbmQtbGluZSwgJ28nIHRvIG9wZW4gYSBuZXcgbGluZRtbMUIbWzYxRGFmdGVyICgn
TycgZm9yIGJlZm9yZSkgdGhlIHNlbGVjdGVkIGxpbmUsICdkJyB0byByZW1vdmUgdGhlG1sxQhtb
NTlEc2VsZWN0ZWQgbGluZSwgb3IgZXNjYXBlIHRvIGdvIGJhY2sgdG8gdGhlIG1haW4gbWVudS4b
WzExQRtbMTVDG1s/MWgbPRtbMkEbWzcwRHJvb3QgICAgICAgICAgICAoaGQwLDApICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAbWzFCG1s3MEQbWzA7N20Pa2Vy
bmVsICAgICAgICAgIC9ib290L3ZtbGludXotNC4xOS4wLTE3LWFtZDY0IHJvb3Q9L2Rldi94dmRh
MiBybyBlbGV2PhtbMUIbW20PG1s/MWgbPRtbPzFsGz4bWzI0OzFIDRtbPzFsGz5UcmFjZWJhY2sg
KG1vc3QgcmVjZW50IGNhbGwgbGFzdCk6CiAgRmlsZSAiL3Vzci9saWIveGVuLTQuMTQvYmluL3B5
Z3J1YiIsIGxpbmUgODg0LCBpbiA8bW9kdWxlPgogICAgcmFpc2UgUnVudGltZUVycm9yKCJVbmFi
bGUgdG8gZmluZCBwYXJ0aXRpb24gY29udGFpbmluZyBrZXJuZWwiKQpSdW50aW1lRXJyb3I6IFVu
YWJsZSB0byBmaW5kIHBhcnRpdGlvbiBjb250YWluaW5nIGtlcm5lbAo=
--0000000000007950e905cb36526c--


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 02:42:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 02:42:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178893.325028 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMi6s-0007so-46; Sun, 05 Sep 2021 02:42:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178893.325028; Sun, 05 Sep 2021 02:42:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMi6r-0007sg-Te; Sun, 05 Sep 2021 02:42:17 +0000
Received: by outflank-mailman (input) for mailman id 178893;
 Sun, 05 Sep 2021 02:42:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMi6q-0007sW-U6; Sun, 05 Sep 2021 02:42:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMi6q-0000XE-MJ; Sun, 05 Sep 2021 02:42:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMi6p-00088Q-Bi; Sun, 05 Sep 2021 02:42:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMi6p-0000eU-8o; Sun, 05 Sep 2021 02:42:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=e6oav8WQi/rXmE07hJn8bLGXuRcjUEI0ZGjRpEqCi7c=; b=tvKZKwqaeVWQ8x468VhjgT+H4x
	kckTKUtizJbhq2lLbAwQEw8BwSsBdgv+0uBpLdjhxCPLMrUdNdrc28ysN2JrW+yPCHHgKQQj3xJo7
	myWE1z54QfHUF24wE8OAE7ew2w6IgoajKqMjAO0bT/CxdkTJbSOfBaabG3kWigwWdMSg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164820-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164820: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 02:42:15 +0000

flight 164820 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164820/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    4 days
Failing since        164686  2021-09-01 03:03:43 Z    3 days   37 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days   11 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 03:01:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 03:01:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178901.325042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMiOx-00021D-Qw; Sun, 05 Sep 2021 03:00:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178901.325042; Sun, 05 Sep 2021 03:00:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMiOx-00020l-KB; Sun, 05 Sep 2021 03:00:59 +0000
Received: by outflank-mailman (input) for mailman id 178901;
 Sun, 05 Sep 2021 03:00:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMiOw-00020b-9X; Sun, 05 Sep 2021 03:00:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMiOw-0000wb-2Q; Sun, 05 Sep 2021 03:00:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMiOv-0001AF-M7; Sun, 05 Sep 2021 03:00:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMiOv-00024R-Lc; Sun, 05 Sep 2021 03:00:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LM0EZtkWk5vBezeciignTkSyJhwR0eboXJm5QcmP/SE=; b=gLoo205Qxy1hag8O8+fd3UaQ2z
	cGaBxKC3FSH85MWIhKq8opyaah+E9ymolewBc/9kX2A6eo9bI/Fo5MUKFiItqg3aeRp/vv42TvgLc
	6HwGQ0Q351oxfeqnMhmQq4N4mAFHwreolDFDY629exOYS/hTMVh0SIsMwXZhxWrfb5iQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164812-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164812: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=9c03aa87e52567f6c9a7bf456e5dd94dc84088de
X-Osstest-Versions-That:
    qemuu=8880cc4362fde4ecdac0b2092318893118206fcf
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 03:00:57 +0000

flight 164812 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164812/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164799
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164799
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164799
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164799
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164799
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164799
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164799
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                9c03aa87e52567f6c9a7bf456e5dd94dc84088de
baseline version:
 qemuu                8880cc4362fde4ecdac0b2092318893118206fcf

Last test of basis   164799  2021-09-03 19:38:03 Z    1 days
Testing same since   164812  2021-09-04 18:37:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Bennée <alex.bennee@linaro.org>
  Mahmoud Mandour <ma.mandourr@gmail.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   8880cc4362..9c03aa87e5  9c03aa87e52567f6c9a7bf456e5dd94dc84088de -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 04:25:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 04:25:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178912.325056 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMjiw-000341-6T; Sun, 05 Sep 2021 04:25:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178912.325056; Sun, 05 Sep 2021 04:25:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMjiv-00033Z-Uq; Sun, 05 Sep 2021 04:25:41 +0000
Received: by outflank-mailman (input) for mailman id 178912;
 Sun, 05 Sep 2021 04:25:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMjiu-00033P-Mm; Sun, 05 Sep 2021 04:25:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMjiu-0002cP-C7; Sun, 05 Sep 2021 04:25:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMjit-0004a1-Ld; Sun, 05 Sep 2021 04:25:39 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMjit-0004vb-Ki; Sun, 05 Sep 2021 04:25:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ONfQiNGc/OJruT+72rIIEEesV5sbIfFZO3OtQmChQps=; b=N60Y8Z/ICGfCsJpibRtDj0VhuE
	jAh76TvLGUvzw0+UVBp6iCu4VOvBfSBmIo4we2mqIz5RZXc2thGKhQCameGPFjJAhLVSfuWo+GfUj
	LEH/udUEepiY4oi+xK7OSCCj/UQ4kxCwDdi3A5rvchnT8icNEWytTy5J+jDo/S2jSqng=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164821-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164821: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 04:25:39 +0000

flight 164821 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164821/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   38 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days   12 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 07:30:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 07:30:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178921.325070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMmbO-0006ek-Ej; Sun, 05 Sep 2021 07:30:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178921.325070; Sun, 05 Sep 2021 07:30:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMmbO-0006eC-Al; Sun, 05 Sep 2021 07:30:06 +0000
Received: by outflank-mailman (input) for mailman id 178921;
 Sun, 05 Sep 2021 07:30:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMmbM-0006S5-Rr; Sun, 05 Sep 2021 07:30:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMmbM-0005xm-KI; Sun, 05 Sep 2021 07:30:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMmbM-0004fR-Ap; Sun, 05 Sep 2021 07:30:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMmbM-0000Fo-AJ; Sun, 05 Sep 2021 07:30:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tpnnEtsJRkqGl+hesrnpeC+PmZVYwLzwiRajer4BQOE=; b=pAbtapvDagyKG2omxK4ngDciCf
	nZSQQg0YhTja7l9R7i78adwKGkJ7cOTiuKKfXzUfov+w7COJD5Fay3GBp/Fcqo9lFxcO9KaIZPV/D
	OWbWnKEpCgHJh2Cth3jIHtljC+jycDJbsZy8GGvnVSFKq+J7NJ9HvdVlhhoRDbSZ5aQY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164816-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164816: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:guest-localmigrate:fail:allowable
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 07:30:04 +0000

flight 164816 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164816/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2 22 guest-start/debian.repeat fail REGR. vs. 152332
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 17 guest-localmigrate fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  400 days
Failing since        152366  2020-08-01 20:49:34 Z  399 days  693 attempts
Testing same since   164816  2021-09-04 22:09:46 Z    0 days    1 attempts

------------------------------------------------------------
7209 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2214305 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 08:06:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 08:06:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178935.325083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMnAa-0002qL-Oi; Sun, 05 Sep 2021 08:06:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178935.325083; Sun, 05 Sep 2021 08:06:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMnAa-0002qE-Lo; Sun, 05 Sep 2021 08:06:28 +0000
Received: by outflank-mailman (input) for mailman id 178935;
 Sun, 05 Sep 2021 08:06:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnAZ-0002q4-A5; Sun, 05 Sep 2021 08:06:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnAZ-00076l-3s; Sun, 05 Sep 2021 08:06:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnAY-000689-Qd; Sun, 05 Sep 2021 08:06:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnAY-0005RR-Q3; Sun, 05 Sep 2021 08:06:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wOGgP2/SIeFwpMxID02PyRrxLC+4tQW6/QvQk4clNyA=; b=bFQYLuKMmYUU9M1foNdZ6+rwMf
	V1nDbAWbPanNoOthToqZTzochiPMg3KFxwDg5cJ6RFo1VqR81hOYwHKmHQGDEGk+QMQ8YFBs9JCuI
	3MpHDnmgiZ81GPBTQFpaOWZNu3LNQh9iHeHDMxqoKiSz1AF0mKeiVOUA0SU252+YbET0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164823-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164823: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 08:06:26 +0000

flight 164823 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164823/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   39 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days   13 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 08:17:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 08:17:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178943.325097 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMnKr-0004V7-PB; Sun, 05 Sep 2021 08:17:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178943.325097; Sun, 05 Sep 2021 08:17:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMnKr-0004V0-La; Sun, 05 Sep 2021 08:17:05 +0000
Received: by outflank-mailman (input) for mailman id 178943;
 Sun, 05 Sep 2021 08:17:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnKr-0004Uq-0d; Sun, 05 Sep 2021 08:17:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnKq-0007HG-QH; Sun, 05 Sep 2021 08:17:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnKq-0006cG-IF; Sun, 05 Sep 2021 08:17:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMnKq-00062g-EX; Sun, 05 Sep 2021 08:17:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=83xelHTQ+U9uaRN3ZSnabUgjU6rzz35B2j6wDQ9ZiXo=; b=q0PYteCe7+f5l8H0tPzy14Kevu
	SPC5GbMFnzeNGTMQPEhukOFLovQP7UuDv6omgnsCvSl50WzD8PIyZkjUwbRTA3FGFgXSRp87K2ecE
	1rvR3ru5jz6SWY+pT9xk+gVdRHWSNc5JchEEXcHqo31FkgGGgfjFDEZ5s5xnJubTlUjk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164822-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164822: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=b6ec01b82154c9dc25f18c9c28df30d0c053defe
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 08:17:04 +0000

flight 164822 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164822/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              b6ec01b82154c9dc25f18c9c28df30d0c053defe
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  422 days
Failing since        151818  2020-07-11 04:18:52 Z  421 days  412 attempts
Testing same since   164803  2021-09-04 04:20:08 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71803 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 08:57:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 08:57:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178956.325112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMny6-0000sA-Sm; Sun, 05 Sep 2021 08:57:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178956.325112; Sun, 05 Sep 2021 08:57:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMny6-0000s3-PT; Sun, 05 Sep 2021 08:57:38 +0000
Received: by outflank-mailman (input) for mailman id 178956;
 Sun, 05 Sep 2021 08:57:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMny5-0000rt-Mr; Sun, 05 Sep 2021 08:57:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMny5-0007wD-Gl; Sun, 05 Sep 2021 08:57:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMny5-0007vu-9J; Sun, 05 Sep 2021 08:57:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMny5-0001qo-8p; Sun, 05 Sep 2021 08:57:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=k/RtWgL7Td3zHnT6KXm+B09/We7AL8SG/eAypxaG8W4=; b=Q52P8/95bkUE9gojrKktzhym04
	ZGtjWbiSlSIAONi04vVLmesYzNX54AS2I3ESx+wd4ysTlymoVLA/Sv6sEkfgOSa1R3EbXvmhFroNF
	y6+oqSMeIrtE4fe51OI5Uq/OYnO6GzEpa8MDD/+s3C6ECvmPY7ZkL3mDOkUsgOzXy/aA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164825-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164825: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 08:57:37 +0000

flight 164825 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164825/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   40 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    0 days   14 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 09:49:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 09:49:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178977.325126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMomZ-0006iA-TH; Sun, 05 Sep 2021 09:49:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178977.325126; Sun, 05 Sep 2021 09:49:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMomZ-0006i3-Q9; Sun, 05 Sep 2021 09:49:47 +0000
Received: by outflank-mailman (input) for mailman id 178977;
 Sun, 05 Sep 2021 09:49:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMomY-0006ht-58; Sun, 05 Sep 2021 09:49:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMomX-0000Nw-Vk; Sun, 05 Sep 2021 09:49:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMomX-0001T7-La; Sun, 05 Sep 2021 09:49:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMomX-0004mc-L6; Sun, 05 Sep 2021 09:49:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TLBc9IA7OFUPDy3YXurDMcY26IZLSdZrTOmeh0+afIo=; b=o66Cg3efFsAU6BBlwI3PznuuMx
	pb8jdeszatq6k85BPSwf7JBq82iWofOew6yXeQdw5evmTBWHK+BktqJxNqJiAZhl5v428K2mNWWc2
	EOSHtJwCQqzulCdSr6qAyCSMxUNeHBuN4sK5a8keDA/0Ojb+KGAvd3hqkiwMMhQmR5I0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164826-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164826: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 09:49:45 +0000

flight 164826 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164826/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   41 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   15 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 10:10:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 10:10:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.178989.325140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMp63-0001Ai-Mi; Sun, 05 Sep 2021 10:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 178989.325140; Sun, 05 Sep 2021 10:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMp63-0001Ab-JY; Sun, 05 Sep 2021 10:09:55 +0000
Received: by outflank-mailman (input) for mailman id 178989;
 Sun, 05 Sep 2021 10:09:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMp61-0001AN-OM; Sun, 05 Sep 2021 10:09:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMp61-0000oO-Ih; Sun, 05 Sep 2021 10:09:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMp61-000254-A1; Sun, 05 Sep 2021 10:09:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMp61-0004yE-9W; Sun, 05 Sep 2021 10:09:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3pRFNPLpQv5EN8fsk2irALFSUyUxUI1Oh/3I2qCKr0g=; b=fs12lc+vSecB5Ym2qoWkLibRdY
	I6NhxezZKgrMhDudxK+c+4plZkxW6vvQy9IIfaHPuNlpmi0Lqy225MOa6ckZJgPLNvGnw2NOvkuwm
	ZaeMJXTL5osHMtCU1hfUZL5pLpB7e/Nwo5U7xvr1H94jLdVHS50r+QLD1vh/jtcFS0Ew=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164827-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 164827: all pass - PUSHED
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=96607a8e680e7f965ca868d11f8b0636317d2618
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 10:09:53 +0000

flight 164827 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164827/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  96607a8e680e7f965ca868d11f8b0636317d2618

Last test of basis   164692  2021-09-01 09:18:30 Z    4 days
Testing same since   164827  2021-09-05 09:18:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Christian Lindig <christian.lindig@citrix.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   96607a8e68..2d4978ead4  2d4978ead4b98452bc45ebe47952e524b12e2b41 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 10:47:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 10:47:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179000.325154 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMpgC-0005uY-L0; Sun, 05 Sep 2021 10:47:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179000.325154; Sun, 05 Sep 2021 10:47:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMpgC-0005uR-Gd; Sun, 05 Sep 2021 10:47:16 +0000
Received: by outflank-mailman (input) for mailman id 179000;
 Sun, 05 Sep 2021 10:47:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMpgB-0005uH-4i; Sun, 05 Sep 2021 10:47:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMpgA-0001Wq-WA; Sun, 05 Sep 2021 10:47:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMpgA-00043Y-Mw; Sun, 05 Sep 2021 10:47:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMpgA-0004IA-ML; Sun, 05 Sep 2021 10:47:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HkLJhTlZb89vRvAC3abpXsxnDMdi4iAQCcx4LQ+zvso=; b=4XCfzITAhIdTvLRr2pzAwF6sBy
	gW/HeM/KH3oMetB/i7OQyHew7UmHNBS9peKHNBMR5OV1OtARdaDP1lTEthcIKdWHZjbSSh1oiGLsc
	wVeYzJEhyNS6P8rt2emiJa+sDZitpUr1+qVzMTkTCfrUFZSTBMfEsafElq1UergcFMAo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164828-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164828: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 10:47:14 +0000

flight 164828 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164828/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   42 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   16 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 11:49:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 11:49:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179009.325168 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMqeY-0004Fq-Ai; Sun, 05 Sep 2021 11:49:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179009.325168; Sun, 05 Sep 2021 11:49:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMqeY-0004Fj-6z; Sun, 05 Sep 2021 11:49:38 +0000
Received: by outflank-mailman (input) for mailman id 179009;
 Sun, 05 Sep 2021 11:49:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqeW-0004FZ-OE; Sun, 05 Sep 2021 11:49:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqeW-0002aw-Hv; Sun, 05 Sep 2021 11:49:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqeW-0007pv-7D; Sun, 05 Sep 2021 11:49:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqeW-0000np-6j; Sun, 05 Sep 2021 11:49:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EPnSfGwbjasYKIcT8A8b0nREtc5NXpVUs4o5vBrwA88=; b=0FHrRgFwZTRGMCE3gembDYUob6
	SD5IkChxcW+GUEDhnlpO79zh8JJ9pW9NGemUa7XOOe6N9hkyydB5LHpIRgBrmR/L7jR14jsPSjv8k
	gK/pM7UWoSDUAyKUFDV9D7qU99vHJQoYYxlzLuMQy5VicrGSPwNX+ClWF/WKHsnUMoNY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164819-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164819: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:guest-start/freebsd.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 11:49:36 +0000

flight 164819 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164819/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 21 guest-start/freebsd.repeat fail in 164805 pass in 164819
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 164805

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164805
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164805
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164805
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164805
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164805
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164805
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164805
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164805
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164805
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164805
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164805
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164819  2021-09-05 01:51:58 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Sep 05 11:50:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 11:50:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179011.325182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMqeu-0004he-Nb; Sun, 05 Sep 2021 11:50:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179011.325182; Sun, 05 Sep 2021 11:50:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMqeu-0004hV-KH; Sun, 05 Sep 2021 11:50:00 +0000
Received: by outflank-mailman (input) for mailman id 179011;
 Sun, 05 Sep 2021 11:49:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqet-0004eQ-9y; Sun, 05 Sep 2021 11:49:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqet-0002ba-4X; Sun, 05 Sep 2021 11:49:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqes-0007qQ-NZ; Sun, 05 Sep 2021 11:49:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMqes-0000uC-N1; Sun, 05 Sep 2021 11:49:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aqDfAzm3/tvuHlNVWaYf6KFuVsKB2oDXlhKlIc/Hvco=; b=OixmnyFWEdY+tE6VP+d4bclbBM
	TizIcwCD6U41zashzirH09buaoNin+hnQprUVEuTVjZcRsva1pJVOWQ5DtXXLz+DMLUt0kolKERNr
	8hs4R6Eo9b+sjXlJINVXLjWOUoB0cXrIxGpEpbzq2IE8p/U9X4q7AfSWy//b3bszte5Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164829-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164829: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 11:49:58 +0000

flight 164829 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164829/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   43 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   17 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 12:30:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 12:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179031.325195 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMrI3-0002KO-4Q; Sun, 05 Sep 2021 12:30:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179031.325195; Sun, 05 Sep 2021 12:30:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMrI3-0002KH-1O; Sun, 05 Sep 2021 12:30:27 +0000
Received: by outflank-mailman (input) for mailman id 179031;
 Sun, 05 Sep 2021 12:30:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMrI1-0002K5-Gm; Sun, 05 Sep 2021 12:30:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMrI1-0003PZ-97; Sun, 05 Sep 2021 12:30:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMrI1-0000d4-11; Sun, 05 Sep 2021 12:30:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMrI1-0007ub-0V; Sun, 05 Sep 2021 12:30:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=i/tbuT/AcFQ7viktCDHZjqCbnRpSmlO2EGBgXbcAVnQ=; b=dvOs5FQqRLMrUY8XZMjKNz4KBT
	L/wVUloteamz+wwPAgcEeA1qrbkGJ+SA5e4THRWql77mucK0BY1trDCVZvQAPNyjlkQLTrF5H0sHm
	mrbN2fw6yCQ4sQ1ZcD0A49PfuEPAoedV+9jtcq4dl8G3oMPgdIHCR67j3fVgV/7B446g=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164830-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164830: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 12:30:25 +0000

flight 164830 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164830/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   44 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   18 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 13:28:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 13:28:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179064.325228 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMsC6-00016q-RB; Sun, 05 Sep 2021 13:28:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179064.325228; Sun, 05 Sep 2021 13:28:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMsC6-00016j-Ny; Sun, 05 Sep 2021 13:28:22 +0000
Received: by outflank-mailman (input) for mailman id 179064;
 Sun, 05 Sep 2021 13:28:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMsC5-00016Z-Qt; Sun, 05 Sep 2021 13:28:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMsC5-0004Un-KM; Sun, 05 Sep 2021 13:28:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMsC5-0002EA-AV; Sun, 05 Sep 2021 13:28:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMsC5-0006uZ-9v; Sun, 05 Sep 2021 13:28:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JuLfHvWGucj4lmIkMu+vEhVYSdG+lzX1IdEoOtsfUeM=; b=ifKBuFifi/JgzKN9rIs8gO2yJk
	RbcddSxDowB211/hy0tzroye1nYTz2Nr7b/wXLKKqZq0KYLi/r6y6659g1MiyuL36pDoaI5yz6wEw
	mkiqF/GiEtlwrr8TGoxJyERx0Rq5qd0zmO2K/cXZ3GOOOPw3b2w2vdmfITfKXpRNDdhk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164831-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164831: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 13:28:21 +0000

flight 164831 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164831/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   45 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   19 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 14:23:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 14:23:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179084.325242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMt3U-0007sh-Ti; Sun, 05 Sep 2021 14:23:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179084.325242; Sun, 05 Sep 2021 14:23:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMt3U-0007sa-QV; Sun, 05 Sep 2021 14:23:32 +0000
Received: by outflank-mailman (input) for mailman id 179084;
 Sun, 05 Sep 2021 14:23:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMt3T-0007sQ-Ne; Sun, 05 Sep 2021 14:23:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMt3T-0005SU-HE; Sun, 05 Sep 2021 14:23:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMt3T-0003bj-8G; Sun, 05 Sep 2021 14:23:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMt3T-0001sq-7k; Sun, 05 Sep 2021 14:23:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yVnm4OeAa2CbBkfDWUmlmjJDE0oUKaEjM9tMJvAE1es=; b=LDoN2hFT+rQQ+rHGOvdSBQrl2L
	zWiX4xeTnVHuSwl4rpn7nqekZTyeSTbfi6TdCMLDD5ouBFQBTMsjV5VwW6IjHexXIR3FDWSMgKh70
	LREI9MNSJcLoKdV3xXVPHa+DhhIyZG+nXCOX3F3CIEwYduS6DhzU9+xmiKT3X33jXT+o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164832-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164832: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 14:23:31 +0000

flight 164832 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164832/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   46 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   20 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 14:58:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 14:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179094.325256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMtbE-0003AF-Nc; Sun, 05 Sep 2021 14:58:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179094.325256; Sun, 05 Sep 2021 14:58:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMtbE-0003A8-Jx; Sun, 05 Sep 2021 14:58:24 +0000
Received: by outflank-mailman (input) for mailman id 179094;
 Sun, 05 Sep 2021 14:58:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMtbD-00039y-C8; Sun, 05 Sep 2021 14:58:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMtbD-0005zk-5y; Sun, 05 Sep 2021 14:58:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMtbC-0004V7-Sr; Sun, 05 Sep 2021 14:58:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMtbC-0003S2-SQ; Sun, 05 Sep 2021 14:58:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NzurV4w00HWx/ttMFtefRVC9SXVM94Oq4/BiDm2wB+c=; b=WVKMWu84Ioc4EljMLg2Bpmri0F
	XMi7cMcO3i4rQ35aQCUfrnf5lfrffcV9OWX9h6Gy9Jq5phJiz0YxJ6rvRZkeGjX1tUbTi8qPUXaUX
	DdxclgLArM+2kDkTvmxBZ55qn+UdiPXbhuabBv2yLNSMKZtCqreh7Nwo0kzD8cs+sCxk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164833-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164833: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 14:58:22 +0000

flight 164833 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164833/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   47 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   21 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 16:20:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 16:20:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179106.325270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMusn-0004qi-Rf; Sun, 05 Sep 2021 16:20:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179106.325270; Sun, 05 Sep 2021 16:20:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMusn-0004qb-NV; Sun, 05 Sep 2021 16:20:37 +0000
Received: by outflank-mailman (input) for mailman id 179106;
 Sun, 05 Sep 2021 16:20:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMusm-0004qR-2w; Sun, 05 Sep 2021 16:20:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMusl-0007wJ-O1; Sun, 05 Sep 2021 16:20:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMusl-0006Wu-Ew; Sun, 05 Sep 2021 16:20:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMusl-0008EE-EW; Sun, 05 Sep 2021 16:20:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Dm8wP27U53P13eQPZIlId+ZuCVmN3e7RcEsk6ALcYiA=; b=YsJZOFbc+IitoBhc0Q7f/CZe8L
	YqdaGuB1gUUVMDqlLF+IpR9tzbDFcA4tEQB3FJNDRCt2qUN+UXE7df0XQv5b5na8N8DSquSm1clzF
	Q7Lx4WvkM13F5nzgq8bgALsYbQmga+HPtVD5fxJewk9Dh8/8DImjaOs+fAPPysNA+wI8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164835-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164835: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 16:20:35 +0000

flight 164835 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164835/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   48 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   22 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 16:35:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 16:35:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179118.325284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMv6v-0006n9-B9; Sun, 05 Sep 2021 16:35:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179118.325284; Sun, 05 Sep 2021 16:35:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMv6v-0006n2-7b; Sun, 05 Sep 2021 16:35:13 +0000
Received: by outflank-mailman (input) for mailman id 179118;
 Sun, 05 Sep 2021 16:35:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMv6u-0006ms-6X; Sun, 05 Sep 2021 16:35:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMv6t-0008CC-D5; Sun, 05 Sep 2021 16:35:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMv6t-0007IQ-1w; Sun, 05 Sep 2021 16:35:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMv6t-0007DB-1T; Sun, 05 Sep 2021 16:35:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6wwR1sl1hRZSWMwZViDJvKTx0hBICzF65MrZcuoS+hY=; b=5r9hTpjJWR6f4gSaQoh/qphR0d
	+kRC+si7q02ij/SsDrpz3wG/JGuXt+y8pRJ35YriH/BlTvW1+/2EEjH+0zivoEnosHp1BZT+v83qM
	+0PjncoUGUSqOZlbZe5Ntfo5FG/HU0FFVMHemMrSODg/8vUBmlAkdUzAdDkuiMfYP/nk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164824-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164824: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl:guest-start:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:guest-localmigrate:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-credit2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:heisenbug
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:guest-start:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 16:35:11 +0000

flight 164824 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164824/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl          14 guest-start    fail in 164816 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 17 guest-localmigrate fail in 164816 pass in 164824
 test-amd64-amd64-xl-credit2 22 guest-start/debian.repeat fail in 164816 pass in 164824
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail in 164816 pass in 164824
 test-amd64-amd64-xl-rtds     17 guest-saverestore          fail pass in 164816
 test-arm64-arm64-xl          13 debian-fixup               fail pass in 164816
 test-armhf-armhf-xl-vhd      13 guest-start                fail pass in 164816

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-vhd     14 migrate-support-check fail in 164816 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 164816 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  400 days
Failing since        152366  2020-08-01 20:49:34 Z  399 days  694 attempts
Testing same since   164816  2021-09-04 22:09:46 Z    0 days    2 attempts

------------------------------------------------------------
7209 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2214305 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 17:59:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 17:59:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179127.325297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMwQV-0007Cb-O9; Sun, 05 Sep 2021 17:59:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179127.325297; Sun, 05 Sep 2021 17:59:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMwQV-0007CU-LL; Sun, 05 Sep 2021 17:59:31 +0000
Received: by outflank-mailman (input) for mailman id 179127;
 Sun, 05 Sep 2021 17:59:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMwQU-0007CI-9r; Sun, 05 Sep 2021 17:59:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMwQU-00014N-35; Sun, 05 Sep 2021 17:59:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMwQT-0001jH-QH; Sun, 05 Sep 2021 17:59:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMwQT-0007ZW-Pk; Sun, 05 Sep 2021 17:59:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jJVRUsbjxefEH7Vhk+PZK/ZjgQhL0iPYKFrHYg10w2E=; b=Y1kg0Jxbj+QjyOcWxxeewKe2lK
	zjs0gv4jiwa5ijL6W2HoXkRz0SixV9VKZcVpXuNKVYNE61xTlURkRzFPi4k4YBh+ThA+CJKoCDLBP
	FyaD0151Iu7B/jPYSs2DG9KB6IIcd1Rp9MBrzQJoCRWgqZ+9kaYPONdsa598dTyXwECw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164837-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164837: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 17:59:29 +0000

flight 164837 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164837/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   49 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   23 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 18:50:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 18:50:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179139.325312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMxE8-0005Je-Ma; Sun, 05 Sep 2021 18:50:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179139.325312; Sun, 05 Sep 2021 18:50:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMxE8-0005JX-Ii; Sun, 05 Sep 2021 18:50:48 +0000
Received: by outflank-mailman (input) for mailman id 179139;
 Sun, 05 Sep 2021 18:50:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMxE7-0005JN-BV; Sun, 05 Sep 2021 18:50:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMxE7-000221-7Q; Sun, 05 Sep 2021 18:50:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMxE6-0003zQ-Tt; Sun, 05 Sep 2021 18:50:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMxE6-0006wb-T1; Sun, 05 Sep 2021 18:50:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gojIG+khwv6sv7LoG2oNZDn1ZVMShQU+HHsa0a0ABMo=; b=gcPw/ytwoRMwrzY+eiM5d6i0jV
	XEGegvK9UKPjriFc9IuhClNCHlw/89jbqZ4fQqTwNq6VVq6dUZIu4l5yeJft9wp94riKOKmGWQn6W
	MKEQecIJy9L/q3FvpENEKorvSYPPi01dRKj6jurf9MdkJRvf9vWDtYJJQsHs5420K1r0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164838-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164838: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 18:50:46 +0000

flight 164838 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164838/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   50 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   24 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 18:54:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 18:54:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179148.325343 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMxHn-0006gD-Pj; Sun, 05 Sep 2021 18:54:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179148.325343; Sun, 05 Sep 2021 18:54:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMxHn-0006g6-Mk; Sun, 05 Sep 2021 18:54:35 +0000
Received: by outflank-mailman (input) for mailman id 179148;
 Sun, 05 Sep 2021 18:54:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=af6+=N3=unixindia.com=bhasker@srs-us1.protection.inumbo.net>)
 id 1mMxHm-0006Ep-4J
 for xen-devel@lists.xenproject.org; Sun, 05 Sep 2021 18:54:34 +0000
Received: from mail-io1-xd2f.google.com (unknown [2607:f8b0:4864:20::d2f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6b2adada-8c5e-499a-811c-e6b2302b1dc4;
 Sun, 05 Sep 2021 18:54:28 +0000 (UTC)
Received: by mail-io1-xd2f.google.com with SMTP id y18so5972885ioc.1
 for <xen-devel@lists.xenproject.org>; Sun, 05 Sep 2021 11:54:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b2adada-8c5e-499a-811c-e6b2302b1dc4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=unixindia.com; s=mail;
        h=mime-version:from:date:message-id:subject:to;
        bh=8EXbD+WXl5NKpRmiVXpJMhwGpvUlCOqYs4PO3k+SXdM=;
        b=QhVxTKwXCOp/+GRfDwt5AusfMdwcmKgpbXoyasGZkYjJ23jNtJVeH8rNoR5qH8gEp/
         XZag0Tc02yo1GV0hDB1CnwQxyDJCe+hoeCwTNsyMCelrbp4RTD6/7mFnay/+uGAgvmuZ
         3EbQKfoE2OASvDAOSpy6YvIdZ04pWdnB0GtCtJEfW2OREj+ZGNAU9zHvwhxN1vfbKdx9
         hvKOunJ3icZrJ7NIFBgFVuIhBQRlBAr2JlizxbKnjVQXAvEmhqXoBvzJKuCHCZVf6+yc
         OobrX8cjTJZY2LybdFV+F8GOoZS7D/TGKwtE4hLaZ9DLIrMOgI2YZUU9emVo6KwgxXj4
         x45A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
        bh=8EXbD+WXl5NKpRmiVXpJMhwGpvUlCOqYs4PO3k+SXdM=;
        b=Ey4JoLUXXO50gZZpuEcb5KJpSOPgi8bbHZtzTrdBigxLR8HSGMlrxtj5exfiKv/rsL
         pVqrobjQdqY2CZMcs3WOC2k6/AP11Yw0opSCIGEQvxLuENiPh4dgCSBGuMrCf1x2CTCr
         bs/m8BwJKmc1CcgczW7Osxs1e/bYGdtu0V5I6urtiFcTFpJLt+qMx9a/WW38HVuHBnpv
         QUKuhFFLhuXbonDz/4ne34GAQhPr5eRL5BzAe2wBlzre2nPS945oE7YBAl0MC3DLvngH
         XefmIKdY5JEX4nEfwR0ipODEhja2bMWI8Q1c0A+DeW7m1GjbSFHIGkc9i8c+4rShJ2te
         mFbQ==
X-Gm-Message-State: AOAM530u5ht66i6mcauPnkcZZ2tDYiS16jZBEkuCP61CFDQVvQ+W+VFA
	mZScULtrulCgECwrqLvnLgZv77i9dbpFBUoOq2DM27mmpDTyxQ==
X-Google-Smtp-Source: ABdhPJxqI/bI7DpkdZFdjU9QxRKbM7tokg6OUGG/BLGUIkM3YYpxKmWQKyNu+HR8t3sEf72Lc1pg0f66PC3MmKV2NAo=
X-Received: by 2002:a6b:e70f:: with SMTP id b15mr6949747ioh.140.1630868067583;
 Sun, 05 Sep 2021 11:54:27 -0700 (PDT)
MIME-Version: 1.0
From: Bhasker C V <bhasker@unixindia.com>
Date: Sun, 5 Sep 2021 19:54:16 +0100
Message-ID: <CAPLCSGBP6vT4Fp2fr5RH7XVsRM=rJ2CGjuVN2aN0v8TSdAWOUg@mail.gmail.com>
Subject: USB passthrough
To: xen-users@lists.xenproject.org, xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000d81cc505cb440f26"

--000000000000d81cc505cb440f26
Content-Type: text/plain; charset="UTF-8"

Hi all,
 Sorry if I am starting a well-discussed thread again.
 I was reading through  https://wiki.xenproject.org/wiki/Xen_USB_Passthrough
and I understand that a xen_usbfront and xen_usbback.ko is needed.

Where do i get these ? I see these provided for k 2.6.18 but that is way
old. Is there a new version for these ?

I tried this:

$ sudo xl usbctrl-attach  14  type=pv version=2 ports=8
libxl: error: libxl_device.c:1103:device_backend_callback: Domain 14:unable
to add device with path /local/domain/0/backend/vusb/14/1
libxl: error: libxl_device.c:1402:device_addrm_aocomplete: Domain 14:Unable
to add vusb with id 1
libxl_device_usbctrl_add failed.

and any attempt to usbdev-attach fails too

Please help

Thanks and regards
Bhasker C V

--000000000000d81cc505cb440f26
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi all,</div><div>=C2=A0Sorry if I am starting a well=
-discussed thread again. <br></div><div>=C2=A0I was reading through=C2=A0 <=
a href=3D"https://wiki.xenproject.org/wiki/Xen_USB_Passthrough">https://wik=
i.xenproject.org/wiki/Xen_USB_Passthrough</a> and I understand that a xen_u=
sbfront and xen_usbback.ko is needed.</div><div><br></div><div>Where do i g=
et these ? I see these provided for k 2.6.18 but that is way old. Is there =
a new version for these ? <br></div><div><br></div><div>I tried this:</div>=
<div><span style=3D"color:rgb(61,133,198)"><br></span></div><div><span styl=
e=3D"color:rgb(61,133,198)">$ sudo xl usbctrl-attach =C2=A014 =C2=A0type=3D=
pv version=3D2 ports=3D8<br>libxl: error: libxl_device.c:1103:device_backen=
d_callback: Domain 14:unable to add device with path /local/domain/0/backen=
d/vusb/14/1<br>libxl: error: libxl_device.c:1402:device_addrm_aocomplete: D=
omain 14:Unable to add vusb with id 1<br>libxl_device_usbctrl_add failed.</=
span></div><div><br></div><div>and any attempt to usbdev-attach fails too</=
div><div><br></div><div>Please help <br></div><div><br></div><div>Thanks an=
d=C2=A0regards</div><div>Bhasker C V<br></div></div>

--000000000000d81cc505cb440f26--


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 19:58:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 19:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179172.325354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMyHV-0005fq-KF; Sun, 05 Sep 2021 19:58:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179172.325354; Sun, 05 Sep 2021 19:58:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMyHV-0005fj-HL; Sun, 05 Sep 2021 19:58:21 +0000
Received: by outflank-mailman (input) for mailman id 179172;
 Sun, 05 Sep 2021 19:58:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMyHU-0005fZ-TE; Sun, 05 Sep 2021 19:58:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMyHU-00039e-Kt; Sun, 05 Sep 2021 19:58:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMyHU-000064-Bo; Sun, 05 Sep 2021 19:58:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMyHU-0005LM-BH; Sun, 05 Sep 2021 19:58:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=L9hErdsUzD0R5chV0wNmmm6ai0FxpL2ZYx588iOTh2o=; b=yaSzhbmqGSGyUZOau5gRnE7Grp
	+hFHuclo76dhKfezRmvLqtlcq0SSpXp3tHbXbCUhNhLpw8v9FgojFxCijmRtBc6g9P/ATy49JTfjG
	9Qj14Eaq/RLPQuLDITANuhJxvxfyqveRZjHF7q02xd43vX2nkXiCj1krScl+MPoNKi5Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164839-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164839: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 19:58:20 +0000

flight 164839 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164839/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   51 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   25 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 21:22:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 21:22:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179181.325368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMzas-0007EJ-UC; Sun, 05 Sep 2021 21:22:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179181.325368; Sun, 05 Sep 2021 21:22:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mMzas-0007EC-R8; Sun, 05 Sep 2021 21:22:26 +0000
Received: by outflank-mailman (input) for mailman id 179181;
 Sun, 05 Sep 2021 21:22:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMzar-0007Dj-QS; Sun, 05 Sep 2021 21:22:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMzar-0004dA-MI; Sun, 05 Sep 2021 21:22:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mMzar-0004Le-B3; Sun, 05 Sep 2021 21:22:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mMzar-0002RP-AY; Sun, 05 Sep 2021 21:22:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SgmPazDYXVewKCwkXxNBO2S/4EpW0pWg4M3fhKT3DBU=; b=Jjzo8hQFUnOVD489GicmBGDvFZ
	t2U+30tPrppFhqFrKKhgGrZb6ixtQWbyHC4aZyAVeEyYbEsrudnJ2rmagMTeFoGfK5nseVTtCU5Ht
	6nA/D6PzmkctTaQQKg7LnWn1FH7Qjz8HhhMwd/4p4ity+mv5hXNxuAXydarhdYg78GYI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164840-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164840: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 21:22:25 +0000

flight 164840 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164840/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   52 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   26 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 21:58:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 21:58:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179188.325383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN09z-0002X1-ND; Sun, 05 Sep 2021 21:58:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179188.325383; Sun, 05 Sep 2021 21:58:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN09z-0002Wu-Ju; Sun, 05 Sep 2021 21:58:43 +0000
Received: by outflank-mailman (input) for mailman id 179188;
 Sun, 05 Sep 2021 21:58:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN09x-0002Wk-Ks; Sun, 05 Sep 2021 21:58:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN09x-0005Ox-E9; Sun, 05 Sep 2021 21:58:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN09x-0005kZ-56; Sun, 05 Sep 2021 21:58:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN09x-0000E3-4f; Sun, 05 Sep 2021 21:58:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7oRiAF+UuXjwp/zPSmEzgdbT10FCFJ9l346YRH2sggY=; b=o0KwI8Qa0prNNjvVIee7wMF4yU
	rtBS2AfXUzj6P1L1O70NoEDdlo/tL8wKKUQpovgmuoMzix/cMV/UyfNDgXEnKjU3kDI7YrIV+GLLt
	9N2Xrt+7sfiGL2qeJL3VP9+szX+ALEmRhGNRscaRnuS5kgb3M5x91dvC+zYmZfsMeaV4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164841-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164841: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 21:58:41 +0000

flight 164841 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164841/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   53 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   27 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 22:10:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 22:10:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179195.325397 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0LS-0004uJ-RF; Sun, 05 Sep 2021 22:10:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179195.325397; Sun, 05 Sep 2021 22:10:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0LS-0004uC-Nh; Sun, 05 Sep 2021 22:10:34 +0000
Received: by outflank-mailman (input) for mailman id 179195;
 Sun, 05 Sep 2021 22:10:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vhsZ=N3=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mN0LR-0004u6-Bi
 for xen-devel@lists.xenproject.org; Sun, 05 Sep 2021 22:10:33 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 162f4d44-0e96-11ec-b06d-12813bfff9fa;
 Sun, 05 Sep 2021 22:10:32 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 185MANtd061312
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO)
 for <xen-devel@lists.xenproject.org>; Sun, 5 Sep 2021 18:10:29 -0400 (EDT)
 (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 185MANaZ061311
 for xen-devel@lists.xenproject.org; Sun, 5 Sep 2021 15:10:23 -0700 (PDT)
 (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 162f4d44-0e96-11ec-b06d-12813bfff9fa
Date: Sun, 5 Sep 2021 15:10:23 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: xen-devel@lists.xenproject.org
Subject: HVM/PVH Ballon crash
Message-ID: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

I brought this up a while back, but it still appears to be present and
the latest observations appear rather serious.

I'm unsure of the entire set of conditions for reproduction.

Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
this is an older AMD IOMMU).

This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
Debian's patches, but those are mostly backports or environment
adjustments.

Domain 0 is presently using a 4.19 kernel.

The trigger is creating a HVM or PVH domain where memory does not equal
maxmem.


New observations:

I discovered this occurs with PVH domains in addition to HVM ones.

I got PVH GRUB operational.  PVH GRUB appeared at to operate normally
and not trigger the crash/panic.

The crash/panic occurred some number of seconds after the Linux kernel
was loaded.


Mitigation by not using ballooning with HVM/PVH is workable, but this is
quite a large mine in the configuration.

I'm wondering if perhaps it is actually the Linux kernel in Domain 0
which is panicing.

The crash/panic occurring AFTER the main kernel loads suggests some
action by the user domain is doing is the actual trigger of the
crash/panic.


That last point is actually rather worrisome.  There might be a security
hole lurking here.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Sun Sep 05 22:28:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 22:28:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179201.325407 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0cs-0006pK-Ay; Sun, 05 Sep 2021 22:28:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179201.325407; Sun, 05 Sep 2021 22:28:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0cs-0006pD-81; Sun, 05 Sep 2021 22:28:34 +0000
Received: by outflank-mailman (input) for mailman id 179201;
 Sun, 05 Sep 2021 22:28:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0cq-0006p3-UZ; Sun, 05 Sep 2021 22:28:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0cq-0005v2-MK; Sun, 05 Sep 2021 22:28:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0cq-0006o8-Ce; Sun, 05 Sep 2021 22:28:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0cq-00020b-CB; Sun, 05 Sep 2021 22:28:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=axnSHiXodJ9jMEzWBW8wEMbJouerLroLOegMPFJpIQI=; b=lBVraYAqTcJLgH7lY08zF/F3+f
	CcemINncuAKxF3XqpKs7Ety8B7PT1EUiv/UzjoEYn8iqvZXMihAxHl7OToenTTQqU3BAmqmonxXGb
	v8jEcHvjxo/kS4XqvuaKV5/RrpeZaHsMzAcqsuFI8IKvEEo02lE4346da8P1lcENh2V4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164842-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164842: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 22:28:32 +0000

flight 164842 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164842/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   54 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   28 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 22:40:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 22:40:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179209.325421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0o2-0000pS-Gu; Sun, 05 Sep 2021 22:40:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179209.325421; Sun, 05 Sep 2021 22:40:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN0o2-0000pL-Ds; Sun, 05 Sep 2021 22:40:06 +0000
Received: by outflank-mailman (input) for mailman id 179209;
 Sun, 05 Sep 2021 22:40:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0o0-0000a2-CT; Sun, 05 Sep 2021 22:40:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0o0-00066F-5Z; Sun, 05 Sep 2021 22:40:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0nz-00078M-P6; Sun, 05 Sep 2021 22:40:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN0nz-0003O6-Od; Sun, 05 Sep 2021 22:40:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5iCv/XOfyeM/UNtavfJ9Ptt0eMd8y392HUbKjkhS4wg=; b=eDh0Iy6dqP78xx9Ya/8fPWL9J7
	UaLAYtJundpRvHL8tXrsJwyCRJk6zHeMu9FTRiV8bNPIRfOIfCVAqZGouoDJUsPs8dPIwBeSMC0Y6
	QrBMlBVWeghMwu+uVX3D8lS1jZSpamyRWY+OLVN3s3aORFNEhc9D729dDjcWRZ9Bq1Zo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164834-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164834: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=31ebff513fad11f315377f6b07447169be8d9f86
X-Osstest-Versions-That:
    qemuu=9c03aa87e52567f6c9a7bf456e5dd94dc84088de
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 22:40:03 +0000

flight 164834 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164834/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164812
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164812
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164812
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164812
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164812
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164812
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164812
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                31ebff513fad11f315377f6b07447169be8d9f86
baseline version:
 qemuu                9c03aa87e52567f6c9a7bf456e5dd94dc84088de

Last test of basis   164812  2021-09-04 18:37:54 Z    1 days
Testing same since   164834  2021-09-05 15:08:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Guoyi Tu <tugy@chinatelecom.cn>
  Markus Armbruster <armbru@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   9c03aa87e5..31ebff513f  31ebff513fad11f315377f6b07447169be8d9f86 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 23:30:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 23:30:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179217.325436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN1ab-0006Yy-CQ; Sun, 05 Sep 2021 23:30:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179217.325436; Sun, 05 Sep 2021 23:30:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN1ab-0006Yr-9B; Sun, 05 Sep 2021 23:30:17 +0000
Received: by outflank-mailman (input) for mailman id 179217;
 Sun, 05 Sep 2021 23:30:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN1aa-0006Yh-8h; Sun, 05 Sep 2021 23:30:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN1aa-0006w2-1W; Sun, 05 Sep 2021 23:30:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN1aZ-0008T9-Po; Sun, 05 Sep 2021 23:30:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN1aZ-0002V9-PH; Sun, 05 Sep 2021 23:30:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xAyIJmn83P4y4PchSS09I4vJ7w4pd9BHksCIycItHbM=; b=B/wDE+KdQPv3OJetjtl5Y8HCbe
	rICG0KookFG+Y2PRzUtdIu1iUoq9ZMsk54+8Xd4Uu7OCfZ/HI0UPZ8diTd2fR7YQXSFklNlEyhMmm
	F31VyUfm5DKK9mZhRByElrs4FCKTimzEZ4+7FamcOSzXeL7WUNGcMpC54EIEGIYekESE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164843-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164843: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 23:30:15 +0000

flight 164843 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164843/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   55 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   29 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 05 23:58:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 05 Sep 2021 23:58:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179225.325450 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN220-00017g-My; Sun, 05 Sep 2021 23:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179225.325450; Sun, 05 Sep 2021 23:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN220-00017Z-Ju; Sun, 05 Sep 2021 23:58:36 +0000
Received: by outflank-mailman (input) for mailman id 179225;
 Sun, 05 Sep 2021 23:58:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN21z-00017P-4y; Sun, 05 Sep 2021 23:58:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN21z-0007Mp-0p; Sun, 05 Sep 2021 23:58:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN21y-0000oQ-Pn; Sun, 05 Sep 2021 23:58:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN21y-00024x-PK; Sun, 05 Sep 2021 23:58:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TN/LJvJsvq8eUTj9KwEtNJ4MqKZ65qSIGifMKhJ8Fao=; b=63erZShl6gvLbeb18AcUE787Js
	Lo9ZWWzAA8FhMjKyoSEOG8AendzvAgxilzX+d7PWU3YH4XjhcPWUx5tqPJxu5rb2UEtvbNfXMLWKz
	MaVOdIdwVeAatcK997bhQRV/H2wJUKvIbljtjZuCyhGEMmXW6q5XCjIJ55TyXfiNOu4s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164844-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164844: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 05 Sep 2021 23:58:34 +0000

flight 164844 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164844/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   56 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   30 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 00:06:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 00:06:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179233.325464 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN29O-0003Ke-EI; Mon, 06 Sep 2021 00:06:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179233.325464; Mon, 06 Sep 2021 00:06:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN29O-0003KX-AQ; Mon, 06 Sep 2021 00:06:14 +0000
Received: by outflank-mailman (input) for mailman id 179233;
 Mon, 06 Sep 2021 00:06:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZwUt=N4=unixindia.com=bhasker@srs-us1.protection.inumbo.net>)
 id 1mN29M-0003KK-Av
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 00:06:12 +0000
Received: from mail-il1-x12e.google.com (unknown [2607:f8b0:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a0ebf0cd-8a17-492a-baf9-6cf81701f4d9;
 Mon, 06 Sep 2021 00:06:11 +0000 (UTC)
Received: by mail-il1-x12e.google.com with SMTP id a1so5200397ilj.6
 for <xen-devel@lists.xenproject.org>; Sun, 05 Sep 2021 17:06:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0ebf0cd-8a17-492a-baf9-6cf81701f4d9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=unixindia.com; s=mail;
        h=mime-version:from:date:message-id:subject:to;
        bh=YrSeOqW8u+/b+kn90enjWBIEuYxqEEa6/DzSOYyPeAY=;
        b=gcldrFazzFF5WFdtYu2jX31ym0BBjR1auHkSXfPsxG96O8W3kkW9MnNMGCjoS8drWX
         6jS8oPTp48QV+bW60fft6BFuU8yqSJMAmMwcCE7Qpvjf7MgjMxK5aqvMV1H8pzrH/k8r
         qA2INpgGQJb19vRncTdl1jj4zsNTSvsH85ttq5L10Nxj3csSuV9T0eRmcXmP6O4boU51
         Ri2kSZcSg0ct1b82FEWC6z4LJSxlP/2eMMIFZJp9EkAxx8iiitpo4TrLfa+t1/0ftmpM
         7uk4dldo2Am4l4e+JK/rcCKASX+gCs0drZy/EZ8o23RIneSg/zEH1O7cYEeqY25f9KJo
         tI/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
        bh=YrSeOqW8u+/b+kn90enjWBIEuYxqEEa6/DzSOYyPeAY=;
        b=RpcbtifRF4JBPnP6tq+ekbY5LnkcX0DW+cQHN84ySmH1Hc4c40A8CDAoWdoAQG0o7B
         +blGfa+0Tb6MZI9cx2rLY1R1JZH1Zc+cP4AltYcP6R4BA+yHmF4bDPWVOaCqsNGNHJWG
         yCYc5299iGLLG4ZfrkypKZRAZo19dQlSdjoRbV12m8nP8MztFhWv7rlUd5KOyGze4gs2
         CNWQrjZqjh+GtUPMB+x16di7UirI2gbClzp3Ltz+vECrilM25K21JP++7mz4bZ2fj+bT
         wV67q7aZbgOIQDlbf3+Bl8XeRooqe3QWM5CEzkh4cvXjBZ86PNxcLPJIOfqM5HjutC/3
         hN3Q==
X-Gm-Message-State: AOAM530SzCuNHHAXFz8bfra7G0fLzwBEaBISKAFO+7PoIYicxqjZlPaW
	p10LnTuCL/aJ/NwhGeNRx+089gMkLsPWBxamkKl8EZZZp6w=
X-Google-Smtp-Source: ABdhPJzPn1n3lZMZ0mAzdV/rhaVBfKhWFWOE69Ok0QaASId1ikfXjRzDX3Cey8Kp8fWgoo2qabbpw4FUkkOhE1h4+U0=
X-Received: by 2002:a92:c109:: with SMTP id p9mr6578692ile.115.1630886771096;
 Sun, 05 Sep 2021 17:06:11 -0700 (PDT)
MIME-Version: 1.0
From: Bhasker C V <bhasker@unixindia.com>
Date: Mon, 6 Sep 2021 01:06:00 +0100
Message-ID: <CAPLCSGD9ZTaoMSqgo6EEdW4VYs9wrz+YvChR_Cx0Rkm8x5iq0w@mail.gmail.com>
Subject: DomU crashes when restored if it has pci passthrough
To: xen-users@lists.xenproject.org, xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000a918c205cb486aff"

--000000000000a918c205cb486aff
Content-Type: text/plain; charset="UTF-8"

Hi,
 I have a domU and that domU has network vf attached to it.
 The save and restore leads to crash of the domU after restore.
 Am I doing anything wrong?

Regards
Bhasker C V

--000000000000a918c205cb486aff
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi,</div><div>=C2=A0I have a domU and that domU has n=
etwork vf attached to it. <br></div><div>=C2=A0The save and restore leads t=
o crash of the domU after restore. <br></div><div>=C2=A0Am I doing anything=
 wrong?</div><div><br></div><div>Regards</div><div>Bhasker C V<br></div></d=
iv>

--000000000000a918c205cb486aff--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 00:10:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 00:10:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179258.325492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN2DZ-0005oq-Ga; Mon, 06 Sep 2021 00:10:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179258.325492; Mon, 06 Sep 2021 00:10:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN2DZ-0005oj-DL; Mon, 06 Sep 2021 00:10:33 +0000
Received: by outflank-mailman (input) for mailman id 179258;
 Mon, 06 Sep 2021 00:10:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZwUt=N4=unixindia.com=bhasker@srs-us1.protection.inumbo.net>)
 id 1mN2DX-0005ob-KC
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 00:10:31 +0000
Received: from mail-io1-xd2e.google.com (unknown [2607:f8b0:4864:20::d2e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 69973039-38c7-4729-a02f-b33b902cadbd;
 Mon, 06 Sep 2021 00:10:30 +0000 (UTC)
Received: by mail-io1-xd2e.google.com with SMTP id y18so6627259ioc.1
 for <xen-devel@lists.xenproject.org>; Sun, 05 Sep 2021 17:10:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69973039-38c7-4729-a02f-b33b902cadbd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=unixindia.com; s=mail;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
        bh=GxF+ayA/gKbFEeVpCODk2PNYX4glvADmxV7Gx3ybgIU=;
        b=a8NpY+rVtH87EPNsbyAx0M6Um/29TPQMeYPdrsOjv/NqC1UYXo5ZP13tBiVQaI9Fee
         NRm5yL4TakS2hi/uhoP+BAthc2gxkEioNi+ANZjWV/jiLBMO/oqRDYiiIpnDVJ2gsxmQ
         9ToMF7Noge9u6nV/QysfeR25koGJKfRKYwyigATOyBjjmwEOtVHfIpRM294q7TjL01Og
         dUhSr2ZdosJlyqsPh+cjR3ZnkZz0Zs/iNko3hsVv3NTy5S7OOMQ8CqraDzpAK/3A3fDk
         xLDhB5QWPPKkX0U0RA+M091QxeE/45HCPVrNMK5RzqJP2KI3+vI4To9CpGYeqLqGWZwT
         tjmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to;
        bh=GxF+ayA/gKbFEeVpCODk2PNYX4glvADmxV7Gx3ybgIU=;
        b=WnsZfxIig20zWd1H1pQakq5qoRlIv8BO/E26EboniMAOmNMw4ONumRCXo0nf5Yc4RK
         qW+iuUS9C8cUyg3gWSFWDhZW91v172/Ru0je7f9cTOgUrszkV/BHBwpw7Wfu6URa/u6f
         kX3Iw8eojbwKiVuEJJqy3QeHWD/Kt0PB4XbW2bQbn58U0s/zIxlZ+PvI/nHng0Kwt9+t
         Aph+3CAV0cTF8S5YLqLuGhD5zNlXdQUEu+V350/X5/mdPE5lkx5V5jQzB53zvSn1VR/l
         +l9kEeQBDFJ7N3tWjemDyiyl5kZ+x1CKwZ0N3JMQ0EbPZSBwhiSFcm5ClH43437l2Oz3
         TIFw==
X-Gm-Message-State: AOAM530cdVIvUbGhHmMIyBJhjcU6KoGC/pJPA5ECUd9GbRYaTGyx39P+
	QGz672a3Gu7yJeE18DDDeostCbcBLLQR04pTzLcKCQ==
X-Google-Smtp-Source: ABdhPJzmlGrjP9v8qzWH6l5kaYjSyu5HD1L9QeBkTQuvo0jy1HBRGTcHJYUOCBUpM+mCQagBFP/gvxx7+hS33W+ySPI=
X-Received: by 2002:a6b:f203:: with SMTP id q3mr7596862ioh.32.1630887030484;
 Sun, 05 Sep 2021 17:10:30 -0700 (PDT)
MIME-Version: 1.0
References: <CAPLCSGD9ZTaoMSqgo6EEdW4VYs9wrz+YvChR_Cx0Rkm8x5iq0w@mail.gmail.com>
In-Reply-To: <CAPLCSGD9ZTaoMSqgo6EEdW4VYs9wrz+YvChR_Cx0Rkm8x5iq0w@mail.gmail.com>
From: Bhasker C V <bhasker@unixindia.com>
Date: Mon, 6 Sep 2021 01:10:19 +0100
Message-ID: <CAPLCSGAsCi=WEy+=3Cit4_e_G=EfcN9TB4B1CkeVxyJvc7o7EQ@mail.gmail.com>
Subject: Re: DomU crashes when restored if it has pci passthrough
To: xen-users@lists.xenproject.org, xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="0000000000001f044005cb487a35"

--0000000000001f044005cb487a35
Content-Type: text/plain; charset="UTF-8"

Ignore me on this
https://wiki.xenproject.org/wiki/Xen_PCI_Passthrough
says it is not possible

On Mon, Sep 6, 2021 at 1:06 AM Bhasker C V <bhasker@unixindia.com> wrote:

> Hi,
>  I have a domU and that domU has network vf attached to it.
>  The save and restore leads to crash of the domU after restore.
>  Am I doing anything wrong?
>
> Regards
> Bhasker C V
>

--0000000000001f044005cb487a35
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Ignore me on this <br></div><div><a href=3D"https://w=
iki.xenproject.org/wiki/Xen_PCI_Passthrough">https://wiki.xenproject.org/wi=
ki/Xen_PCI_Passthrough</a></div><div>says it is not possible<br></div></div=
><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mo=
n, Sep 6, 2021 at 1:06 AM Bhasker C V &lt;<a href=3D"mailto:bhasker@unixind=
ia.com">bhasker@unixindia.com</a>&gt; wrote:<br></div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Hi,</div><div>=C2=A0I ha=
ve a domU and that domU has network vf attached to it. <br></div><div>=C2=
=A0The save and restore leads to crash of the domU after restore. <br></div=
><div>=C2=A0Am I doing anything wrong?</div><div><br></div><div>Regards</di=
v><div>Bhasker C V<br></div></div>
</blockquote></div>

--0000000000001f044005cb487a35--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 00:29:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 00:29:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179287.325522 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN2Vw-0000RS-I4; Mon, 06 Sep 2021 00:29:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179287.325522; Mon, 06 Sep 2021 00:29:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN2Vw-0000RL-Dy; Mon, 06 Sep 2021 00:29:32 +0000
Received: by outflank-mailman (input) for mailman id 179287;
 Mon, 06 Sep 2021 00:29:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN2Vv-0000R9-KI; Mon, 06 Sep 2021 00:29:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN2Vv-0008Vw-Ee; Mon, 06 Sep 2021 00:29:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN2Vv-0001Wq-43; Mon, 06 Sep 2021 00:29:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN2Vv-00013D-3W; Mon, 06 Sep 2021 00:29:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=t6ACaiYYpsM00TI5pG2Of+1jhHOD/xDwLJkkAPYT35s=; b=FvtEt3xI6Wt+TqsICII2pzsqev
	7eGvcsFYdEflDfEuhIsN9mOp35s24ELS/mpGPLXiTQyivplquv1Ae8pHHKuZmnS6R50yPZfGs+nE+
	adF1j/Rkc2v0e/qdjmcezRjtG/Akg8J1pzPcpmWnZopdWkKIVPSGDDmKPlVyL2tZrVcI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164845-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164845: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 00:29:31 +0000

flight 164845 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164845/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   57 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   31 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 01:28:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 01:28:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179296.325536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN3R3-0005bD-2H; Mon, 06 Sep 2021 01:28:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179296.325536; Mon, 06 Sep 2021 01:28:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN3R2-0005b6-VB; Mon, 06 Sep 2021 01:28:32 +0000
Received: by outflank-mailman (input) for mailman id 179296;
 Mon, 06 Sep 2021 01:28:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3R1-0005aw-DT; Mon, 06 Sep 2021 01:28:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3R1-0007Iz-5q; Mon, 06 Sep 2021 01:28:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3R0-000307-U8; Mon, 06 Sep 2021 01:28:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3R0-0006Eb-Tc; Mon, 06 Sep 2021 01:28:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=IFZLvMtNKtje9Fyc/tk7eFpwGUk3iyCIQIxjcs7WUUo=; b=wFcVcyLAepghwq+PCsiim7GR7a
	PhJ5ikRzvMuF26wobP2Ri4nh9BvXJl7HFluBSbrPMmzPQ6qRqi+urtzR13i4Ty2646gDXMYCFniy8
	korN3qXOEoQ5CW0GYWAGJnvzwKLdB6yeKUmN4/x9ChPrPrdE4AfheyOmSP8Sw6NLnbPs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164846-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164846: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 01:28:30 +0000

flight 164846 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164846/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   58 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   32 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 01:59:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 01:59:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179306.325550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN3ul-0000t8-IF; Mon, 06 Sep 2021 01:59:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179306.325550; Mon, 06 Sep 2021 01:59:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN3ul-0000t1-Do; Mon, 06 Sep 2021 01:59:15 +0000
Received: by outflank-mailman (input) for mailman id 179306;
 Mon, 06 Sep 2021 01:59:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3uj-0000sr-8b; Mon, 06 Sep 2021 01:59:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3uj-0007nO-4R; Mon, 06 Sep 2021 01:59:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3ui-0003lF-T3; Mon, 06 Sep 2021 01:59:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN3ui-00038E-SY; Mon, 06 Sep 2021 01:59:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=orQPH5E08YAc2dCabRRtT3xNghF7XWfee9MNw3o6ouI=; b=Oh/ftyvOFPSKL818AL/70mLao5
	X+43pfOTU65b+2gpDqAoiSzvmGwXShX7FSK2ymA4nymPb2PklNrHLAceWY8UfMtgfl4u5JiwlIdga
	9tlSQoJEpEEa8Mv9AFqOMHToDRgH2UvWBNyY0hUKVK5i8+HRaAv9FsLFHpyWYSnJDTTI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164847-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164847: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 01:59:12 +0000

flight 164847 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164847/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    5 days
Failing since        164686  2021-09-01 03:03:43 Z    4 days   59 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   33 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 02:24:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 02:24:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179315.325564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN4Iq-0004vk-Mf; Mon, 06 Sep 2021 02:24:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179315.325564; Mon, 06 Sep 2021 02:24:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN4Iq-0004vd-JP; Mon, 06 Sep 2021 02:24:08 +0000
Received: by outflank-mailman (input) for mailman id 179315;
 Mon, 06 Sep 2021 02:24:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AiNf=N4=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mN4Ip-0004vX-T6
 for xen-devel@lists.xen.org; Mon, 06 Sep 2021 02:24:07 +0000
Received: from mail-pj1-x102d.google.com (unknown [2607:f8b0:4864:20::102d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a15e629f-1d3c-455a-8653-b9e4a053d60b;
 Mon, 06 Sep 2021 02:24:06 +0000 (UTC)
Received: by mail-pj1-x102d.google.com with SMTP id
 n13-20020a17090a4e0d00b0017946980d8dso3340172pjh.5
 for <xen-devel@lists.xen.org>; Sun, 05 Sep 2021 19:24:05 -0700 (PDT)
Received: from laputa (p784a2304.tkyea130.ap.so-net.ne.jp. [120.74.35.4])
 by smtp.gmail.com with ESMTPSA id b7sm5582312pfl.195.2021.09.05.19.23.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 05 Sep 2021 19:24:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a15e629f-1d3c-455a-8653-b9e4a053d60b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:content-transfer-encoding:in-reply-to;
        bh=TIye2OuKCRdKK2inz4e7KcA5V+jm8PQZVNAbxUlcF9M=;
        b=ugJmyzJYLmTJvdVs2vfCx6yDxQZp2kbg8om7kdStgI1azE3b3NpW5qCcscYjm1HgC7
         0S76RUOxRRMB9DW6xpkkbmtB6hDTxL0QtdONXvmwXW3ovHbnhTz9lRi/H0QJElADrgPR
         VC/oHUaSKizAnxCRfK43xQXtznpVEeV8UGJZXPfFSbCd6nCGKJ8r7lHeX7GeDQ9V+xru
         ibhLrT25bjKCDD6V1xfNLfN2tkMq9i2ChFJeg1gpFGAEiP+8liY5o3PC3Am56bFvKYb1
         X73JlQzIvZ5LtUn8xy3KwwitUu0ZeZko324byhqkUtaO+/fgWzN7py2yQQMBH0jSEU5s
         yiBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:content-transfer-encoding
         :in-reply-to;
        bh=TIye2OuKCRdKK2inz4e7KcA5V+jm8PQZVNAbxUlcF9M=;
        b=UngTuoNSphFBxHKHdIk8Vrvhi8eRTxRvxEgLVbNKeFmZ8V0mAXWnlx5SylzHiFwa8O
         oWmh18gMYQERNQtLwfMrnCnMZWJlXcvsPjMvvXC9c3X7VIVDQ187mL1j6OSXuPJY0YeO
         t2xm0YJPCOJ1LIU4y1rCwldGf1Tm+qoxh2v2furI/8QEVnWHfrxvDeMaH7JSpj3vxstA
         EWdsBgF5CTgQS8QBiHP0XrPEFA5AP163S+l7TTlEJ3I19PgIRbJbvlOhu2Ckv87DK6iK
         ZKMgS0MktntscIRoqCWa2UzSmvluribb3/Oxnq/zLuh0g9N4+wYe6glQbu0gVdYHu+ZM
         KvUQ==
X-Gm-Message-State: AOAM530XvEnXcjATh0kVNWU57oLsECF+u2lbPMyloot1tgpP1LwHuoML
	b8DG2j9/WzWVdy2wm+MRhyxKgg==
X-Google-Smtp-Source: ABdhPJwCO1zSOXyxpTWEILD/F2gLRiZRWkwd2ZxF74/o/wUpG2r2d3jsgevbkR2rmE4PDW1bcWPNoQ==
X-Received: by 2002:a17:90a:de04:: with SMTP id m4mr11661221pjv.187.1630895045015;
        Sun, 05 Sep 2021 19:24:05 -0700 (PDT)
Date: Mon, 6 Sep 2021 11:23:56 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Alex Benn??e <alex.bennee@linaro.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	virtio-dev@lists.oasis-open.org,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>, Wei.Chen@arm.com,
	olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com,
	Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org,
	jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org,
	Elena Afanasova <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210906022356.GD40187@laputa>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
 <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa>
 <87czpqq9qu.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <87czpqq9qu.fsf@linaro.org>

Alex,

On Fri, Sep 03, 2021 at 10:28:06AM +0100, Alex Benn??e wrote:
> 
> AKASHI Takahiro <takahiro.akashi@linaro.org> writes:
> 
> > Alex,
> >
> > On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:
> >> 
> >> Stefan Hajnoczi <stefanha@redhat.com> writes:
> >> 
> >> > [[PGP Signed Part:Undecided]]
> >> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
> >> >> > Could we consider the kernel internally converting IOREQ messages from
> >> >> > the Xen hypervisor to eventfd events? Would this scale with other kernel
> >> >> > hypercall interfaces?
> >> >> > 
> >> >> > So any thoughts on what directions are worth experimenting with?
> >> >>  
> >> >> One option we should consider is for each backend to connect to Xen via
> >> >> the IOREQ interface. We could generalize the IOREQ interface and make it
> >> >> hypervisor agnostic. The interface is really trivial and easy to add.
> >> >> The only Xen-specific part is the notification mechanism, which is an
> >> >> event channel. If we replaced the event channel with something else the
> >> >> interface would be generic. See:
> >> >> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52
> >> >
> >> > There have been experiments with something kind of similar in KVM
> >> > recently (see struct ioregionfd_cmd):
> >> > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/
> >> 
> >> Reading the cover letter was very useful in showing how this provides a
> >> separate channel for signalling IO events to userspace instead of using
> >> the normal type-2 vmexit type event. I wonder how deeply tied the
> >> userspace facing side of this is to KVM? Could it provide a common FD
> >> type interface to IOREQ?
> >
> > Why do you stick to a "FD" type interface?
> 
> I mean most user space interfaces on POSIX start with a file descriptor
> and the usual read/write semantics or a series of ioctls.

Who do you assume is responsible for implementing this kind of
fd semantics, OSs on BE or hypervisor itself?

I think such interfaces can only be easily implemented on type-2 hypervisors.

# In this sense, I don't think rust-vmm, as it is, cannot be
# a general solution.

> >> As I understand IOREQ this is currently a direct communication between
> >> userspace and the hypervisor using the existing Xen message bus. My
> >
> > With IOREQ server, IO event occurrences are notified to BE via Xen's event
> > channel, while the actual contexts of IO events (see struct ioreq in ioreq.h)
> > are put in a queue on a single shared memory page which is to be assigned
> > beforehand with xenforeignmemory_map_resource hypervisor call.
> 
> If we abstracted the IOREQ via the kernel interface you would probably
> just want to put the ioreq structure on a queue rather than expose the
> shared page to userspace. 

Where is that queue?

> >> worry would be that by adding knowledge of what the underlying
> >> hypervisor is we'd end up with excess complexity in the kernel. For one
> >> thing we certainly wouldn't want an API version dependency on the kernel
> >> to understand which version of the Xen hypervisor it was running on.
> >
> > That's exactly what virtio-proxy in my proposal[1] does; All the hypervisor-
> > specific details of IO event handlings are contained in virtio-proxy
> > and virtio BE will communicate with virtio-proxy through a virtqueue
> > (yes, virtio-proxy is seen as yet another virtio device on BE) and will
> > get IO event-related *RPC* callbacks, either MMIO read or write, from
> > virtio-proxy.
> >
> > See page 8 (protocol flow) and 10 (interfaces) in [1].
> 
> There are two areas of concern with the proxy approach at the moment.
> The first is how the bootstrap of the virtio-proxy channel happens and

As I said, from BE point of view, virtio-proxy would be seen
as yet another virtio device by which BE could talk to "virtio
proxy" vm or whatever else.

This way we guarantee BE's hypervisor-agnosticism instead of having
"common" hypervisor interfaces. That is the base of my idea.

> the second is how many context switches are involved in a transaction.
> Of course with all things there is a trade off. Things involving the
> very tightest latency would probably opt for a bare metal backend which
> I think would imply hypervisor knowledge in the backend binary.

In configuration phase of virtio device, the latency won't be a big matter.
In device operations (i.e. read/write to block devices), if we can
resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue is
how efficiently we can deliver notification to the opposite side. Right?
And this is a very common problem whatever approach we would take.

Anyhow, if we do care the latency in my approach, most of virtio-proxy-
related code can be re-implemented just as a stub (or shim?) library
since the protocols are defined as RPCs.
In this case, however, we would lose the benefit of providing "single binary"
BE.
(I know this is is an arguable requirement, though.)

# Would we better discuss what "hypervisor-agnosticism" means?

-Takahiro Akashi

> >
> > If kvm's ioregionfd can fit into this protocol, virtio-proxy for kvm
> > will hopefully be implemented using ioregionfd.
> >
> > -Takahiro Akashi
> >
> > [1] https://op-lists.linaro.org/pipermail/stratos-dev/2021-August/000548.html
> 
> -- 
> Alex Bennée


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 02:32:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 02:32:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179324.325575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN4R0-0006cb-Md; Mon, 06 Sep 2021 02:32:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179324.325575; Mon, 06 Sep 2021 02:32:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN4R0-0006cU-Im; Mon, 06 Sep 2021 02:32:34 +0000
Received: by outflank-mailman (input) for mailman id 179324;
 Mon, 06 Sep 2021 02:32:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN4Qz-0006cK-F4; Mon, 06 Sep 2021 02:32:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN4Qz-0000NZ-5f; Mon, 06 Sep 2021 02:32:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN4Qy-0004Sn-Rn; Mon, 06 Sep 2021 02:32:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN4Qy-0006Gw-Qc; Mon, 06 Sep 2021 02:32:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eRLYvhcg0uu4eqzt61OkoYUoEs1z3GvbnRQM52xnbEs=; b=H+vxTSfcHkdkGhB5W0OnXkzOyZ
	pbnsy98Wll2IyIILiQ9iHthhnKyjtLANiQ5HYI6Gya4eihzle65n2YYNjHtyNbAf+ZyJdC9mAvjoy
	SktDGMwCzwRt2zGWCuWSjI51jKKstn96t6wdvNaZvZE7zQVf2AymiZ9M4tKzfQzJ6ZcA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164836-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164836: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-qemuu-freebsd12-amd64:guest-localmigrate:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-credit2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:guest-start:fail:heisenbug
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 02:32:32 +0000

flight 164836 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164836/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl          14 guest-start    fail in 164816 REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup   fail in 164824 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 17 guest-localmigrate fail in 164816 pass in 164836
 test-amd64-amd64-xl-credit2 22 guest-start/debian.repeat fail in 164816 pass in 164836
 test-amd64-amd64-xl-rtds    17 guest-saverestore fail in 164824 pass in 164836
 test-armhf-armhf-xl-vhd      13 guest-start      fail in 164824 pass in 164836
 test-arm64-arm64-xl          13 debian-fixup               fail pass in 164816
 test-arm64-arm64-xl-xsm       8 xen-boot                   fail pass in 164824
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164824

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                49624efa65ac9889f4e7c7b2452b2e6ce42ba37d
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  401 days
Failing since        152366  2020-08-01 20:49:34 Z  400 days  695 attempts
Testing same since   164816  2021-09-04 22:09:46 Z    1 days    3 attempts

------------------------------------------------------------
7209 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2214305 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 03:15:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 03:15:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179334.325589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN564-0003RV-UJ; Mon, 06 Sep 2021 03:15:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179334.325589; Mon, 06 Sep 2021 03:15:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN564-0003RO-Qy; Mon, 06 Sep 2021 03:15:00 +0000
Received: by outflank-mailman (input) for mailman id 179334;
 Mon, 06 Sep 2021 03:15:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN564-0003RE-1d; Mon, 06 Sep 2021 03:15:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN563-0001XE-TT; Mon, 06 Sep 2021 03:14:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN563-0005lU-LO; Mon, 06 Sep 2021 03:14:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN563-0007sz-Kw; Mon, 06 Sep 2021 03:14:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DC4VoHwFw7nN5BJLNdxU4GM3D/LLQe3DRsqAWhMFWUU=; b=6F4H7HuulC/4+Kh9ounmsAr9uR
	SMQ7lCR7ao+KhTGxqVxSwlUitE14cY3p+xAZ1GZgPZ+67i+YTA6Pgpt1KMh+xTNqIIDrpRKP3CfnU
	ATfOEcRFTloIhFwuN/SGZLm27FbhsR0kuso2j4q5MPbC6DcFwN84flN9N1KtAwgtE/Os=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164849-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164849: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 03:14:59 +0000

flight 164849 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164849/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   60 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   34 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 04:55:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 04:55:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179344.325603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN6eh-0006E4-7r; Mon, 06 Sep 2021 04:54:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179344.325603; Mon, 06 Sep 2021 04:54:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN6eh-0006Dx-4b; Mon, 06 Sep 2021 04:54:51 +0000
Received: by outflank-mailman (input) for mailman id 179344;
 Mon, 06 Sep 2021 04:54:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN6ef-0006Dn-AQ; Mon, 06 Sep 2021 04:54:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN6ef-0003Kp-2J; Mon, 06 Sep 2021 04:54:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN6ee-0000oG-OW; Mon, 06 Sep 2021 04:54:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN6ee-0005Uh-Np; Mon, 06 Sep 2021 04:54:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UVg3TRteyt8KraxQtdIkqzOY247NIzbdDoVLohD1La0=; b=u7Z9Vz5iLgVrfOHQoaSEZTZ7f1
	Eaz+RBNVZTMi+qeFaNqIGmjjjhVMoxlU7cjmvzywDdTsAD7PDV1Vw2CSkAwmcXzdpUNnvwei8AZP8
	CNSxPPeSqA2N8ADaYVoiaqnd8NDW+vTYHq0xlnbISFxiklGtg1vuMwzl8J1WXuTlwMWg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164851-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164851: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 04:54:48 +0000

flight 164851 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164851/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   61 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   35 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 06:49:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 06:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179360.325616 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8Qw-0001yl-4n; Mon, 06 Sep 2021 06:48:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179360.325616; Mon, 06 Sep 2021 06:48:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8Qw-0001ye-1m; Mon, 06 Sep 2021 06:48:46 +0000
Received: by outflank-mailman (input) for mailman id 179360;
 Mon, 06 Sep 2021 06:48:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NW6E=N4=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mN8Qu-0001yY-DL
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 06:48:44 +0000
Received: from mail-wm1-x329.google.com (unknown [2a00:1450:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 71175502-8074-44d1-8cfb-976cad8ddc83;
 Mon, 06 Sep 2021 06:48:43 +0000 (UTC)
Received: by mail-wm1-x329.google.com with SMTP id n39so2055687wms.1
 for <xen-devel@lists.xenproject.org>; Sun, 05 Sep 2021 23:48:43 -0700 (PDT)
Received: from ?IPv6:2a00:23c5:5785:9a01:f581:5504:73aa:6d39?
 ([2a00:23c5:5785:9a01:f581:5504:73aa:6d39])
 by smtp.gmail.com with ESMTPSA id k16sm6930834wrh.24.2021.09.05.23.48.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 05 Sep 2021 23:48:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71175502-8074-44d1-8cfb-976cad8ddc83
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:reply-to:subject:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=d2q96vSBWYolsynVYZ8qCx75g4juhICH/4ouPIjinkI=;
        b=Jyyedaj9+sm7NqUcI1nGPWdF9/qLNUsEdhKM2iNOsLkMAEpsa+1DRaYXva8ydqaQ2K
         EbfPLuPpu9rGlxyst45Fa9sTFZ/sR+wMAu0S5CghrRIPSz2lgf0B/qCkEbRf5i4mzviw
         c7lAAd9Le+RXQn96K0SUv8C5x/ACANaaqjqJunVOleHN4LheJOhKvA9Ndew/VzZ3010T
         1bxbCL9e3dI9pVDJFzd/W8H6VhrQCssi6Fp1HiYxjwUycMmgE5fP1Hw4qCSQR6oySAgu
         TGQTohXJCy331cXN8h6t+jGsBJwTWweeV/oF2scVLMYYNWGpK+GNzJEHKPjDrQByIboF
         GA3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:reply-to:subject:to:cc:references
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=d2q96vSBWYolsynVYZ8qCx75g4juhICH/4ouPIjinkI=;
        b=AiPpZwPng5xAwWAPBBkCHbcUbK2+8KkxaQ2LmgnNuMfd4JxXIfDccPfrqrc44sZAGN
         KZImObD4/sYXEff8/3s0xK9N7Ou6C9SAzUArvFa7MdHk/My1bu9xI5VgwvdX3A9mlkNb
         j4PjIAQa6IGwBSw6lZfR7v5Ns7raMEoEXQSPTHcFL6ij/x2u9FTtp4AAE2BlQ2m/AjN7
         1+hvm4Kn18FRLVHubmFKZl96sZFcAwDpOWGWOqZhH6KChP907WoiDgy3N6WKCw+s8HfP
         MwisSA6mEZfJaJLHSIxfcrbZXRb3aHcs6UM/b13BIZm6Dv28FhAV3zU/YWSrnVqrtG71
         AuYQ==
X-Gm-Message-State: AOAM531GVRNO17YvbCfzc7G5dYswnY5n8tf+x76YtS62Y2neQTTF6eLl
	zPZcNO6YsqPD51oWCAQBbZoz3moOmqY=
X-Google-Smtp-Source: ABdhPJz5GXTy42bFINdV+bVQ/4IrI84/Mcf+IJ/fqGcFF/rg1XE+6IOHLmrnlMvlR77t4cVIc38DfA==
X-Received: by 2002:a1c:cc13:: with SMTP id h19mr9722348wmb.187.1630910922999;
        Sun, 05 Sep 2021 23:48:42 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Reply-To: paul@xen.org
Subject: Re: [PATCH v3 30/30] accel: Add missing AccelOpsClass::has_work() and
 drop SysemuCPUOps one
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
 Anthony Perard <anthony.perard@citrix.com>,
 Cameron Esfahani <dirty@apple.com>, Claudio Fontana <cfontana@suse.de>,
 Colin Xu <colin.xu@intel.com>, Eduardo Habkost <ehabkost@redhat.com>,
 haxm-team@intel.com, Kamil Rytarowski <kamil@netbsd.org>,
 Laurent Vivier <lvivier@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 Reinoud Zandijk <reinoud@netbsd.org>, Roman Bolshakov
 <r.bolshakov@yadro.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Huth <thuth@redhat.com>, Wenchao Wang <wenchao.wang@intel.com>,
 xen-devel@lists.xenproject.org
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210903181943.763360-1-f4bug@amsat.org>
Message-ID: <f039f2e5-2558-2305-c30f-aaa89084e3fd@xen.org>
Date: Mon, 6 Sep 2021 07:48:41 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210903181943.763360-1-f4bug@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 03/09/2021 19:19, Philippe Mathieu-Daudé wrote:
> cpu_common_has_work() is the default has_work() implementation
> and returns 'false'.
> 
> Explicit it for the QTest / HAX / HVF / NVMM / Xen accelerators
> and remove cpu_common_has_work().
> 
> Since there are no more implementations of SysemuCPUOps::has_work,
> remove it along with the assertion in cpu_has_work().
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Xen parts...

Acked-by: Paul Durrant <paul@xen.org>


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:03:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:03:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179382.325628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8eg-0004g3-FF; Mon, 06 Sep 2021 07:02:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179382.325628; Mon, 06 Sep 2021 07:02:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8eg-0004fw-Bu; Mon, 06 Sep 2021 07:02:58 +0000
Received: by outflank-mailman (input) for mailman id 179382;
 Mon, 06 Sep 2021 07:02:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ykgp=N4=epam.com=prvs=9883458851=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mN8ee-0004fq-Rj
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:02:57 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 75965472-0ee0-11ec-b097-12813bfff9fa;
 Mon, 06 Sep 2021 07:02:55 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18664srY000909; 
 Mon, 6 Sep 2021 07:02:49 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50])
 by mx0b-0039f301.pphosted.com with ESMTP id 3aw28ps0r0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 06 Sep 2021 07:02:49 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6242.eurprd03.prod.outlook.com (2603:10a6:20b:15c::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 07:02:46 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::641e:b269:6d21:b7f8%5]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 07:02:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75965472-0ee0-11ec-b097-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PQk7TwGRFQCFaBgnv0VXOZWWeM6NYZTEkh6Hztv50rc03PEx21G27h6gO1VXNJ06PRc6PYxw2bwcA/TwInJbr9TJpPs3oGK5QmYR4pQLaaQjj4NoDWBZVpEZUMGkwF9a9Hlbin8gGQKkw7LeG0darT46wGL4hEoMn4U0+UHSpAtayS0fr3Hmq++4ZhkGjt3X0GKBa4XC238xuC3A2vCmlTRTE7mRFdUeOeb4XHm1bDOrcAFI+JOOGFKl5rtq4EDNHKjD5Dm2lduHAzkovIILa4T/g7/A+P0gm4co9G7MLveESpusmr8wJYe6Ltsg7StWn0xBX9w1J77CJNyc4Rq7pg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GDTAVpAbHwGYMZmKPNl2kjdBtJNWPwV7jhug3pQvpwE=;
 b=WswR/JJ/zS2K7staqMNNw3o7is1s+64X2SOraKIbH33BRxVZIv0IAC7Fp8fQ5aMIlzuOo3RMs/TdD69UEwHhmq7+DssjXr2M1FE2XkcPwlEZs+EMTHr5tL0rJBTWTYKGZqpx3X5i7xg8UUWh1nTe7eDAARRqd1MXUUWAekzTnUbGTBF/MjZKpqeO0PZKabr5zBNiAapDgt3oGbqkCtRgBgD7Emu+lnKu1SemlwUdH/MdkcLhOCeIsxfecz0NkRerFJLCwkA9PBHM4mIwpmAMPihfuBXt7Xieo14yG9EQg0ka6uTHWQtjIjKRLqapAzRX70FnpEkIALw821gVDx8wjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GDTAVpAbHwGYMZmKPNl2kjdBtJNWPwV7jhug3pQvpwE=;
 b=4InjGgXhNRt1Td4at2b1KE54GRLJqv7kF93wP/i6v+P6g/fn6Jc7oLPilTQB1+zEn6a3G2JkP7QFXjeb8pw4dOU4A74fmY0IJhK3XwicdBWvRisoAlGaiA2FFb4ZPkmEByoPlhNZ3csVhgJI0WlNGERfR1L417gsajI1fmr29DiamF4wNZNdZ9/zANO+tGHxdwU6YUpTl//+v4v9ftSbx8cMKrNSqwz5ApY0mLH6z7fwuxEpMoLpeUN0W7FuQtnZhkU8R3bn/uByQ+bSMR5L9Rwohfs+JQ1/LqGr3ejxwXr3t0LttFI9s9uA7bwwe3TvbcOhVs6BzSePTp2Xdp02mw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Topic: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Index: AQHXoJ51zxJWpxhHEEGXNqYYBqazw6uSBESAgASU4wA=
Date: Mon, 6 Sep 2021 07:02:46 +0000
Message-ID: <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
In-Reply-To: <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 12ba1be1-c0e4-4060-655e-08d9710454e5
x-ms-traffictypediagnostic: AM0PR03MB6242:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB6242EE81780CF4F8B3D25460E7D29@AM0PR03MB6242.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 TRdmlDVjN4rVdMieKN+iJtO+wd13W06nOiyDFtHW/adnlGRDtCofvsYSIs0z1QKYoU9K0/Z86mFFK16oxZQs+49YVw/QxzSgtAYaHD0xyXsZn7vpUizZXZWHqkjZyJOlzmuub4qnOyhtS6SyatYL23U8Z8KKHAUoUncDR1fFR+0NsYvXxeb7VllURkfOgtKXTPi5mCE9ZClKqxASlbEK5BEySy3dIR9j9w7aRP5IlvTD/M1fno0NagKRRGDc9bi7AR6tQNfseJ832AhPkjRjGV5CRjvyqoFy1EbGdAID9qSYIOrnlv3oaE2GRzxeKamCnLOVyl8BuoH8EoBXKa1X5JMUJCkO2hZzO3g1DLaLeBQrXZSLkD2yl26HybfVqXgvE1uPL6MC3mvnKmqjzODDBN5ckQ3ccMo27n8MZeKHURjK9yguv4P9wB7FeFVJR8JDBLkqsfNGJ1QDHk8600bNjFEv8v81jeBWyPw1l8cy28ETXpPndGVd+sfHn1CbaRYHatL9YrAT2Bw1kxUYKDH6eyaXIWQajLfvC1lxgKcSJOBi0qG29NRZA8Fn11kROG/P2ZI76y27eYCfpuqbdcj1GdavrrYSUj1WauaCcm+2Xno/i9+iOQyWteFE1I4uWfeTWUTqUZS8neA/5cZon5onN1IK24kcPrQlVGguo1g4VZ1HFbUpJY4KDnkJapJebGsU/bTxu7eWxJ9UauI/RuA3ZE+tfDPTWD4pE+VPbzYWAopKqIFS/fL3LvqsTVHrRKsu
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(376002)(396003)(136003)(39860400002)(66556008)(66476007)(4326008)(8936002)(64756008)(6512007)(66946007)(76116006)(66446008)(478600001)(122000001)(26005)(31686004)(71200400001)(38100700002)(54906003)(110136005)(2616005)(5660300002)(8676002)(316002)(38070700005)(36756003)(2906002)(31696002)(6506007)(53546011)(83380400001)(86362001)(186003)(6486002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SDZ2cmR2RS8xWkFlcHJWOXhnN2ltZ2pmUzNHZWd6bHVlZUJwamlyR2IvQWxt?=
 =?utf-8?B?WW1mTFRQbGUzWDZPdFdQRzNFWk1kNWdrMzZUbzhtZVB1V2tYRUZWU3VlZ1JM?=
 =?utf-8?B?TlFvV3hwWWY0azRETmFGNWFDTGl2T0JZbnQ1L0VwU0k0Ykczb29KVTRBQlFa?=
 =?utf-8?B?R3RFS3hiazgrbS9lTUY3NXZPV2EzY0RDZUpVOTcrZkRKN3FRZzM2ZGcxRmg3?=
 =?utf-8?B?NldwYWlSdGlYTWtyMWJUY2NYcHJ4bUY4Y0N3UU1PbElGUEtDWHk5eGdDNzFV?=
 =?utf-8?B?SWxubFRLVXJmcnR1WjV3WHllemZUTitoa0dNb0dYdHgreWhXbU4zR1VjMHJE?=
 =?utf-8?B?OWJZZERGWHd2US9NT1VHa05hd2loRVEwSDR1UTFGRWdvTjhHVk90c1NsSFkv?=
 =?utf-8?B?N3pJY2VnWnZiNUlzN1ovU2JqRXN4Yi9maWpRTVcwVEVJcGJyTHpKdEJ1OC9r?=
 =?utf-8?B?bHFiVzg3SWhuRGw0S2E0WU4rZkFxUUVCNWJRN29WZGJPeFFKWXFNYU4wazdV?=
 =?utf-8?B?UG9IOWIvZENHWTdYa25kQldyQTB2TFVVU2g1eEtyK1dCbkp0Z0lPOFdHbHlk?=
 =?utf-8?B?bFE4T29RaDBpNEZXSGFJQi9QRzlFaG90WTJUaU53UHhsSDBiTlJnSDBwLzVh?=
 =?utf-8?B?bVBGOWFaOGZMelp6c0RVWmZvL3VwaTNjQXhlcS9KSkx5RGZhRE9RUGp3b0c4?=
 =?utf-8?B?azJoQ2pzMnkycC9lQU5obWR2YWFuVnphOGlSRHJCcEdBOS9QUERCNnVuRTlk?=
 =?utf-8?B?K2NTRVcyYVY5NUFxeWtRWHRrTWhyUXlMc2ZrcjAxdG9ncDNKdTlPZW1iWWFV?=
 =?utf-8?B?cFQ0cmFtUFluR1VSK0xDQzZhQUxlNFBGZ2ZHUDJ1YTNLb3FRcS9PeHpxT2dL?=
 =?utf-8?B?R0N4YkFqZUpsQU5IczF5QjRaQU5MNUM2SDlWMjlYMHJUVmw0RHdnQUw3bW1F?=
 =?utf-8?B?bk83ZUF6bi8yeS96RVA3ZXJoN0RGZUVSbzZyVVZXUUpOR0k0RWF2bGZMaVJG?=
 =?utf-8?B?Vm9tR0Zac0txY3FVc08xb2M2MFZxOHh5UWJrNWQ5R2NyNjZtUHkvblBlT2c2?=
 =?utf-8?B?bzJwVEkxSHp6bFpZNHhGZ1JSZUQwSW5QdUd4RTAreE5BUXI1YzNIcFFKejhB?=
 =?utf-8?B?cElUMU1YMmJobmpGOE9waE5USk4veUs2SWhDcnBCREcyMXlMTmU0TWc0VGVp?=
 =?utf-8?B?eHQ3dXVyYllsbWVRcnA4YVc4Y2JlQ2FvNlh3L1FaQmhNMkpzOW1MUjQybGZl?=
 =?utf-8?B?QjJqQ2s4V3R2SHJFVERSWDBRNFBwMHIyTkp4MVRmRC9Qdzl4YzZwWEtDU0Zn?=
 =?utf-8?B?bm12ZXJaaGFoZVQxSE5PMm1qd2dZdlY4M3AwQThBS0Z3MnBtTWF2ZzZmRGZk?=
 =?utf-8?B?VWRaNGl0K2t3MlZOaHFaM0EwMjEremlxWDZmaDRrcUFGeVNSYkhkbm1yU0ov?=
 =?utf-8?B?YitPb0JSNUFDSm1ScWJYUHJCSC91WGRlcmlxQUtndkxrZXhjRmhLc20zWlI4?=
 =?utf-8?B?NXllZHZTOHh3YnRCNExIQ2MyQ3JvRityODc4YVR3RmtxRzJFaWFzYytBV0k5?=
 =?utf-8?B?bkd4TEZHQTQ0ZEtNL1JET1ZiTEdVNEZJbzF6Y1pBQjhMczFLNW9WaldETGs3?=
 =?utf-8?B?ZlBjQk1FNDN4ckFaVGswelROZFhFSVFQZy81dFpnTUd1QUJpejdTZTZYSG9T?=
 =?utf-8?B?MDdqUGUvMTVjL3BJK092cFVCZTdrVlVLaklrMFhmNFJPekxaWUpzb3ZlRTNY?=
 =?utf-8?Q?k3+n6lPi4pIDygo1HWkwSqfkQ/S2PYg5MZ+ZBDt?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D6712A4FCBCCF549864B2A4160802F56@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 12ba1be1-c0e4-4060-655e-08d9710454e5
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2021 07:02:46.5732
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: NAi5LS3ymOmmbqwzhL+RJL/e7+QtFWPunQcrXqcd1AjHIZLut5SWF8x8zDXDydx4R3Wgp7of7EmB9r0cEZVvmAJhLOCYGQDuziLOhRMAn2BhzeJyMpmvKEK+BPZVW0wU
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6242
X-Proofpoint-GUID: LtIWBbLKz_VVlkAk4Ee9cQ2XEUi3PdeH
X-Proofpoint-ORIG-GUID: LtIWBbLKz_VVlkAk4Ee9cQ2XEUi3PdeH
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-06_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0
 lowpriorityscore=0 spamscore=0 impostorscore=0 adultscore=0 phishscore=0
 clxscore=1015 priorityscore=1501 bulkscore=0 mlxscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109060044

SGksIEp1bGllbiENCg0KT24gMDMuMDkuMjEgMTI6MDQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4g
SGkgT2xla3NhbmRyLA0KPg0KPiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiB2UENJIG1heSBtYXAgYW5kIHVubWFw
IFBDSSBkZXZpY2UgbWVtb3J5IChCQVJzKSBiZWluZyBwYXNzZWQgdGhyb3VnaCB3aGljaA0KPj4g
bWF5IHRha2UgYSBsb3Qgb2YgdGltZS4gRm9yIHRoaXMgdGhvc2Ugb3BlcmF0aW9ucyBtYXkgYmUg
ZGVmZXJyZWQgdG8gYmUNCj4+IHBlcmZvcm1lZCBsYXRlciwgc28gdGhhdCB0aGV5IGNhbiBiZSBz
YWZlbHkgcHJlZW1wdGVkLg0KPj4gUnVuIHRoZSBjb3JyZXNwb25kaW5nIHZQQ0kgY29kZSB3aGls
ZSBzd2l0Y2hpbmcgYSB2Q1BVLg0KPg0KPiBJSVVDLCB5b3UgYXJlIHRhbGtpbmcgYWJvdXQgdGhl
IGZ1bmN0aW9uIG1hcF9yYW5nZSgpIGluIHhlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLiBUaGUgZnVu
Y3Rpb24gaGFzIHRoZSBmb2xsb3dpbmcgdG9kbyBmb3IgQXJtOg0KPg0KPiDCoMKgwqDCoMKgwqDC
oCAvKg0KPiDCoMKgwqDCoMKgwqDCoMKgICogQVJNIFRPRE9zOg0KPiDCoMKgwqDCoMKgwqDCoMKg
ICogLSBPbiBBUk0gd2hldGhlciB0aGUgbWVtb3J5IGlzIHByZWZldGNoYWJsZSBvciBub3Qgc2hv
dWxkIGJlIHBhc3NlZA0KPiDCoMKgwqDCoMKgwqDCoMKgICrCoMKgIHRvIG1hcF9tbWlvX3JlZ2lv
bnMgaW4gb3JkZXIgdG8gZGVjaWRlIHdoaWNoIG1lbW9yeSBhdHRyaWJ1dGVzDQo+IMKgwqDCoMKg
wqDCoMKgwqAgKsKgwqAgc2hvdWxkIGJlIHVzZWQuDQo+IMKgwqDCoMKgwqDCoMKgwqAgKg0KPiDC
oMKgwqDCoMKgwqDCoMKgICogLSB7dW59bWFwX21taW9fcmVnaW9ucyBkb2Vzbid0IHN1cHBvcnQg
cHJlZW1wdGlvbi4NCj4gwqDCoMKgwqDCoMKgwqDCoCAqLw0KPg0KPiBUaGlzIGRvZXNuJ3Qgc2Vl
bSB0byBiZSBhZGRyZXNzZWQgaW4gdGhlIHR3byBzZXJpZXMgZm9yIFBDSSBwYXNzdGhyb3VnaCBz
ZW50IHNvIGZhci4gRG8geW91IGhhdmUgYW55IHBsYW4gdG8gaGFuZGxlIGl0Pw0KDQpObyBwbGFu
IHlldC4NCg0KQWxsIHRoZSBtYXBwaW5ncyBhcmUgaGFwcGVuaW5nIHdpdGggcDJtX21taW9fZGly
ZWN0X2RldiBhcyBvZiBub3cuDQoNCj4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIg
QW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+PiAtLS0N
Cj4+IMKgIHhlbi9hcmNoL2FybS90cmFwcy5jIHwgNiArKysrKysNCj4+IMKgIDEgZmlsZSBjaGFu
Z2VkLCA2IGluc2VydGlvbnMoKykNCj4+DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3Ry
YXBzLmMgYi94ZW4vYXJjaC9hcm0vdHJhcHMuYw0KPj4gaW5kZXggMjE5YWIzYzNmYmRlLi4xNTcx
ZmI4YWZkMDMgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdHJhcHMuYw0KPj4gKysrIGIv
eGVuL2FyY2gvYXJtL3RyYXBzLmMNCj4+IEBAIC0zNCw2ICszNCw3IEBADQo+PiDCoCAjaW5jbHVk
ZSA8eGVuL3N5bWJvbHMuaD4NCj4+IMKgICNpbmNsdWRlIDx4ZW4vdmVyc2lvbi5oPg0KPj4gwqAg
I2luY2x1ZGUgPHhlbi92aXJ0dWFsX3JlZ2lvbi5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vdnBjaS5o
Pg0KPj4gwqAgwqAgI2luY2x1ZGUgPHB1YmxpYy9zY2hlZC5oPg0KPj4gwqAgI2luY2x1ZGUgPHB1
YmxpYy94ZW4uaD4NCj4+IEBAIC0yMzA0LDYgKzIzMDUsMTEgQEAgc3RhdGljIGJvb2wgY2hlY2tf
Zm9yX3ZjcHVfd29yayh2b2lkKQ0KPj4gwqDCoMKgwqDCoCB9DQo+PiDCoCAjZW5kaWYNCj4+IMKg
ICvCoMKgwqAgbG9jYWxfaXJxX2VuYWJsZSgpOw0KPj4gK8KgwqDCoCBpZiAoIGhhc192cGNpKHYt
PmRvbWFpbikgJiYgdnBjaV9wcm9jZXNzX3BlbmRpbmcodikgKQ0KPg0KPiBMb29raW5nIGF0IHRo
ZSBjb2RlIG9mIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCksIGl0IGxvb2tzIGxpa2UgdGhlcmUgYXJl
IHNvbWUgcmV3b3JrIHRvIGRvIGZvciBndWVzdC4gRG8geW91IHBsYW4gdG8gaGFuZGxlIGl0IGFz
IHBhcnQgb2YgdGhlIHZQQ0kgc2VyaWVzPw0KWWVzLCB2UENJIGNvZGUgaXMgaGVhdmlseSB0b3Vj
aGVkIHRvIHN1cHBvcnQgZ3Vlc3Qgbm9uLWlkZW50aXR5IG1hcHBpbmdzDQo+DQo+PiArwqDCoMKg
wqDCoMKgwqAgcmFpc2Vfc29mdGlycShTQ0hFRFVMRV9TT0ZUSVJRKTsNCj4+ICvCoMKgwqAgbG9j
YWxfaXJxX2Rpc2FibGUoKTsNCj4+ICsNCj4NCj4gRnJvbSBteSB1bmRlcnN0YW5kaW5nIG9mIHZj
cGlfcHJvY2Vzc19wZW5kaW5nKCkuIFRoZSBmdW5jdGlvbiB3aWxsIHJldHVybiB0cnVlIGlmIHRo
ZXJlIGFyZSBtb3JlIHdvcmsgdG8gc2NoZWR1bGUuDQpZZXMNCj4gSG93ZXZlciwgaWYgY2hlY2tf
Zm9yX3ZjcHVfZm9yX3dvcmsoKSByZXR1cm4gZmFsc2UsIHRoZW4gd2Ugd2lsbCByZXR1cm4gdG8g
dGhlIGd1ZXN0IGJlZm9yZSBhbnkgd29yayBmb3IgdkNQSSBoYXMgZmluaXNoZWQuIFRoaXMgaXMg
YmVjYXVzZSBjaGVja19mb3JfdmNwdV93b3JrKCkgd2lsbCBub3QgYmUgY2FsbGVkIGFnYWluLg0K
Q29ycmVjdA0KPg0KPiBJbiB0aGlzIGNhc2UsIEkgdGhpbmsgeW91IHdhbnQgdG8gcmV0dXJuIGFz
IHNvb24gYXMgeW91IGtub3cgd2UgbmVlZCB0byByZXNjaGVkdWxlLg0KTm90IHN1cmUgSSB1bmRl
cnN0YW5kIHRoaXMNCj4NCj4gSG93ZXZlciwgbG9va2luZyBhdCB0aGUgcmVzdCBvZiB0aGUgY29k
ZSwgd2UgYWxyZWFkeSBoYXZlIGEgY2hlY2sgZm9yIHZwY2kgaW4gdGhlIGNvbW1vbiBJT1JFUSBj
b2RlLg0KDQpXaGljaCBtYXkgbm90IGJlIGVuYWJsZWQgYXMgaXQgZGVwZW5kcyBvbiBDT05GSUdf
SU9SRVFfU0VSVkVSLg0KDQpNeSB1bmRlcnN0YW5kaW5nIGlzIHRoYXQgZm9yIHg4NiBpdCBpcyBh
bHdheXMgZW5hYmxlZCwgYnV0IHRoaXMgbWlnaHQgbm90IGJlIHRoZSBjYXNlIGZvciBBcm0NCg0K
PiBTbyB3ZSB3b3VsZCBlbmQgdXAgdG8gY2FsbCB0d2ljZSB2cGNpX3Byb2Nlc3NfcGVuZGluZygp
Lg0KU28sIGlmIENPTkZJR19JT1JFUV9TRVJWRVIgaXMgbm90IGVuYWJsZWQgdGhlbiB3ZSBlbmQg
dXAgd2l0aCBvbmx5IGNhbGxpbmcgaXQgZnJvbSB0cmFwcy5jIG9uIEFybQ0KPiBNYXliZSB3ZSBz
aG91bGQgbW92ZSB0aGUgY2FsbCBmcm9tIHRoZSBJT1JFUSB0byBhcmNoLWNvZGUuDQoNCkhtLiBJ
IHdvdWxkIGJldHRlciB0aGluayBvZiBtb3ZpbmcgaXQgZnJvbSBJT1JFUSB0byBzb21lIG90aGVy
IGNvbW1vbiBjb2RlOiBmb3IgeDg2IChpZg0KDQpteSB1bmRlcnN0YW5kaW5nIGNvcnJlY3QgYWJv
dXQgQ09ORklHX0lPUkVRX1NFUlZFUikgaXQgaXMgYnkgY29pbmNpZGVuY2UgdGhhdCB3ZSBjYWxs
IHZQQ0kNCg0KY29kZSBmcm9tIHRoZXJlIGFuZCBJT1JFUSBpcyBhbHdheXMgZW5hYmxlZC4NCg0K
Pg0KPiBDaGVlcnMsDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:17:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:17:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179389.325638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8sO-0006KX-NM; Mon, 06 Sep 2021 07:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179389.325638; Mon, 06 Sep 2021 07:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN8sO-0006KQ-KP; Mon, 06 Sep 2021 07:17:08 +0000
Received: by outflank-mailman (input) for mailman id 179389;
 Mon, 06 Sep 2021 07:17:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN8sN-0006KE-G5; Mon, 06 Sep 2021 07:17:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN8sN-00064p-8w; Mon, 06 Sep 2021 07:17:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mN8sM-0007yK-UI; Mon, 06 Sep 2021 07:17:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mN8sM-00068e-Ti; Mon, 06 Sep 2021 07:17:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=e8OWnnB2XLJmpnd87ePW0WMHjn7FMJMQJMK5jwVzx8o=; b=1a+1sYKSZKgmmhKP7muXbUo9pt
	CJXXqYpiaOfZfihs8FWRvm6Jdx4oZFSVVnUY9yANHVsa/qDcJN0m47QeYKQDkPhqfCOfK/KIUV6+O
	UODuMKRI0BDY+4t17ITkBdklX0IC236unA8NwdlWz59u+XqzVsDeVYb0sE4BXa5FrI/A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164853-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164853: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 07:17:06 +0000

flight 164853 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164853/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   62 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    1 days   36 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:42:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:42:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179400.325653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9HD-0001YU-Rx; Mon, 06 Sep 2021 07:42:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179400.325653; Mon, 06 Sep 2021 07:42:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9HD-0001YN-OZ; Mon, 06 Sep 2021 07:42:47 +0000
Received: by outflank-mailman (input) for mailman id 179400;
 Mon, 06 Sep 2021 07:42:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mN9HC-0001YH-I2
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:42:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 05e1e3f2-0ee6-11ec-b099-12813bfff9fa;
 Mon, 06 Sep 2021 07:42:45 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6--VJjzU7QPaSfY_f2SYh3PQ-1; Mon, 06 Sep 2021 09:42:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 07:42:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 07:42:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0074.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 07:42:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05e1e3f2-0ee6-11ec-b099-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630914164;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=thr+KWDNBf4RXgukMPAM+yfG2ZtaAJHpOlaYkocEhOU=;
	b=fOrW/MsJJa7aHe6PBLsb2lnTluUtLDmBz1mcEa0xaA0mBeKjWVu2txQ//ukfdo69vIsjIJ
	T2PrAT0xoyn+iB+LrkXcph7s5u4+pv0FqQjNUaTTNcMKedE6KvrrbRavV6mR4z8jmcUGg5
	goM287RYt87Wx8fJjVyzjR6/bRBXRwg=
X-MC-Unique: -VJjzU7QPaSfY_f2SYh3PQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oe2ussa7S5SlyO7NRMUHxRj5gFZ3JEo7d7CJFEqPvsM/1Tq2Hi9g+GYatJUOnQqezXs1z6/lAtnXnSmiUM6XuBhWJfcJtGDS8LDhfy1v3kH/7AZ+rWAST+8sQ+6FQwBsBVK9hO6qUNQyasiEglhqT2LnhdK94NRh1zQNBAronCD9imvRkR4kwq4/QrjJ8Cbb9zUMA60LjPGeIYfh+Wm+5FuMMCqcpZryz1qcv8+1cw9A2WGRrUVTBG5rGytq/t91jDcPz+KuNQwQn035IzKWyZK45JznydZLPWA+NSq9HZafYoz/H32kckeQ4f8Ai9K/oi+xt4NDScDI6bgbx+YM7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=thr+KWDNBf4RXgukMPAM+yfG2ZtaAJHpOlaYkocEhOU=;
 b=Xaall/BESC/SB9UCTlJvcaU0oN/D9ierH/Sp9K0BqZpYEKQlKkYXNnfpsJadA+ouX6WcyyCKPs52GRxdJRVh74Bqy1c+MKGnkCLQc0+tH72Fy36UDQzazaPHt6fav/5j4/K8hrv2LAAyzTUoOTx3+8Jmoa7zIGJk85IkwoZ9bRpiikrRq46IMmlC9r3E7jlQi82SH0Z29gExn8Dpj8s4u/kGbvK9sr+siRSweJwBGQwrYNOTIDW8O85P5VphaUIfMCW6q+ZnpZbEJTF7NUT0Kxfc3ptCzQr+GX+Wik3tr24W5U3OGOB57PQc0N6ZM/4jdbJQMDFdJAckRTm1yPaLsQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: Help with Network SR-IOV (PF still in reset state. Is the PF
 interface up?)
To: Bhasker C V <bhasker@unixindia.com>
References: <CAPLCSGCkdZMph=iRrqf3G5kT881mN5kYaG=qXrMtWS=fTn2O=g@mail.gmail.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8282a111-0481-410b-4a04-f0edd4bdb326@suse.com>
Date: Mon, 6 Sep 2021 09:42:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <CAPLCSGCkdZMph=iRrqf3G5kT881mN5kYaG=qXrMtWS=fTn2O=g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0074.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f6c020da-8081-4d40-40d9-08d97109e805
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2959889FC3C66F2BC1974EF2B3D29@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xrg9MqHQZJR0J7R/OFehNXU4zQjtflIUdXKWaP/Hj+t+rovV9SpH4Stjj7S8wzpAH9BTip28SR00HpL/vcPSFdfjQAqSAoSptUge1VC/3RqcwiJnXd63EqbNfOpa5JkX638f/WHc8CA02aGIq09gGoLxquEehOsL+zR84cToxkBoGUbouObLJeSta9RcdOpKTPS1xNsS4GfFx0ckCz4EJ6SncO/1DsSvLb6+iRgk8lWYmAwzRi2T9ip7BYACQJz+fsrUycMSypiqKoc2TXE6RfkH40ox9SsiwNeIwR/BW0IUpKJRfK4FGyguaSCECFMEQIMtiETYjolhh6Fetr1OjjmcDg5nr0SGx/iE4L3OXjwYO8kKdesLfSKRQM5qY/ndFF6ZDLpddOsNPTWWHiNFU3bBwkK+8zERKvIWTjIu0K5YIP6BILR3LRzyAIKDCAx+Fen0Wl2Gj7kY9CDjixlSgoEr+urKKEoaKamti1KqNQ8vNTqBwitpbVp0IiyGHFDdjmMB8pXb6RbtXjmo3WNEwVt6harF4wzgthjWb/wvxJFG/rIlvrTysa4vPLaMcmkT4iFm4qDIHHWGHdzN9EHVXTqYyrd/m4wJ9kvoYHmT9RhaGpleVyzO2FwuNnGq04WOTNCpFObOqeWAAjwsHDaJ7LHDxGucw0CyvpUWaw/tUDmcq3djjfOnsL81UB2V8CZqEeaalWqcdBkozwMAjQlmf446xjHhBSV/rxFF2U+9ckU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(5660300002)(6486002)(186003)(26005)(66946007)(31696002)(8676002)(2906002)(956004)(2616005)(316002)(53546011)(31686004)(83380400001)(508600001)(38100700002)(16576012)(4326008)(66556008)(66476007)(36756003)(86362001)(6916009)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SCt5UkE3cnZRMWFBdHJ5RFBYMUkyazhxWS82d2VKbmVYVHFEOS9RV1RCenpV?=
 =?utf-8?B?QVFSSlpGUHZDUGFrWklaeUNibk5NVXQ1c3M0RWtBRzh4UzdacTBycVlCVVh5?=
 =?utf-8?B?WXV6SnRVYWo5MEtoYWZ6UlZVRVA1M2JVRjNSMGFsS3RpQUVDaCtnQ3NQbTVa?=
 =?utf-8?B?UmkyVmJvUnNJTWoxb0lzeVdmMVZZU2JnNm81b1BZamxOYXlOdUlUZ0tSVTBj?=
 =?utf-8?B?QXNBMjRtNlZNdWtIMVJDbzdZaFJoYWdka21najZPQUpGWVFhQyt1VWY0dzRS?=
 =?utf-8?B?VmtmS3VBbTdQRDEvWG9KcCtqRGRkbEVVNGdYNURDdHR4SEw1M1BlL3padEZI?=
 =?utf-8?B?Rjd4L002ck5UOHB4RWYxL0FpOW1WYytVN1VKaU1ORWxnSy9sZUZDaU96emJu?=
 =?utf-8?B?aS9iaHQ3WDV1K1hocWpLdDFNY0VaOEFOOWZKaXRWS213VG5TK0VlNHo2bXRh?=
 =?utf-8?B?dWRJUG5UUVFnTzVPd0tIU2VhQW15ZU4yMkFjV3doUW5DT204SFM2VjE1OTRL?=
 =?utf-8?B?ZTdZS0FIQjhJOEsxcGczUE1KSzZtbXgvY3RlQ0pBOUEyR1JzZFdjT0trRUg2?=
 =?utf-8?B?eW01bW9rRnRvcThwWDF5OWdoVVJlb3JFM3RmaTlLbWN5czA2K0lHMWVEWTFE?=
 =?utf-8?B?Q05IcHhaMjArb1ZwV2JpdjlNSWJoRzlUTkI2aEFwNWM5YXlDbWxHV2NDeWYy?=
 =?utf-8?B?ckhYY2VnaHAzT2ZSL1haZllrRndXM0JNNFlZYjhqWXB3OE4zUXZydGp6ZjQ1?=
 =?utf-8?B?TFJRd2lHU0FzWThNM0NzTU8rVnVvRFRqTFF4YnNGT1dzemh4Um05VHZHZlQ2?=
 =?utf-8?B?ODduclp6TmZ0SFhieVVFSWtRNDBKSkU1Nit4enF6b3ZpZ3lsVEpFUGt0djNr?=
 =?utf-8?B?cVA1U2xSaGh2RXF6MlczSWk0ZTZDK3NqWUM3OE9VZEgxNUg5QktJc1JRUjEx?=
 =?utf-8?B?L1NWRVBVcEREejY3UlpZRjFpM3RML0dOMDVYSS9mYkM5aFhUclBQNGtmTXVB?=
 =?utf-8?B?WlNEY0tBMVhmVjhML0l6NFBueWwrU0psNEdTRzNVOU5RSmY1bFdtYjQ4aE1u?=
 =?utf-8?B?a2lac3gySFFZd3BCQ2RheGZHbEdsaGQ0d3ZKeHhSK05CU0JTRDg2Zk0zenRt?=
 =?utf-8?B?UXZCVGs4YU5LNm9XRFBlRE5FVlJVTXRtcTZLN2Jod0tBNlBmdngxV05hcVNK?=
 =?utf-8?B?MGxQOWduNkNsbFZCUnY0OHBiV29mOSt1VmE5MEppYzd4ZUQ5M3lNSkgrbWJq?=
 =?utf-8?B?aGlNemVGcGhHSXlpVTJxV3Ayc1lqZUFSTm4rakdWRXkyK1dkNXN3eURRTVAv?=
 =?utf-8?B?dHhvOUJZWUh6Z0UzZjhQSHU4cVpSK3MzQXc1YmxWTlFSQ0VNd2wyMjFJbDRu?=
 =?utf-8?B?ZU9RL1hWVzRxd21sMGZwbDF1VVU2OEE3OUk5b0RGQmd3YkxXRHBQVHRuVURn?=
 =?utf-8?B?TkZ5YUJCU2FYY3BQWk4zN3hvS0JPWWxaYWUzL0V5WEM2RTlyaXVjRHFjM3Fr?=
 =?utf-8?B?M08rTW1vejFuanBReHFPby9PSytDdmNlRTNSTXpOVVdMR2NSN0hrYmN5TTgx?=
 =?utf-8?B?QWMrN2lyckhiOFp5R3RORldtcU1pSkRuRVVzc05WckVHZUdveERXSDlkL1Zz?=
 =?utf-8?B?QWdyUWFYVlNXUEM5dG9zLytLeUhTYjN0M1MxYXpCaTE5L2s3VGhUOXYyRHZG?=
 =?utf-8?B?K1lOeUMzR0h0WVdHS0kyMjZ6TzlhZ1dnd0VFMGNHblVDSjR4SUk2RVc4K1h3?=
 =?utf-8?Q?BJ9lgyi8wq4QkjRa4YNxDFT8d5SX039QrQPcjFK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f6c020da-8081-4d40-40d9-08d97109e805
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 07:42:41.2107
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XIW1kEQetet1uchw+/4oTYHGZIf6L3rScpXeQ10DsVPAJn6RtvfJ+7yNgu1P8nbUwSGYBChIulLiIorwWXEJnQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

(please don't cross-post; removing xen-users)

On 04.09.2021 17:12, Bhasker C V wrote:
> I run  xen-4.14.3-pre from debian
> 
> I created 6 vfs for my igb card
> 06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network
> Connection (rev 01)
> I got these
> 07:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.5 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:11.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 
> bcv@bcvibmb1:~$ sudo xl pci-assignable-list
> 0000:07:10.1
> 
> However the guest is not able to see the link up.
> bcv@utils:~$ sudo ethtool  enp0s0f1
> 
> 
> 
> Settings for enp0s0f1:
> 
>         Supported ports: [ ]
> ...
> 
>         Link detected: no
> 
> I see this error on the guest
> 
> [    6.134890] igbvf 0000:00:00.1: Xen PCI mapped GSI0 to IRQ19
> 
> [    6.149577] igbvf 0000:00:00.1: PF still in reset state. Is the PF
> interface up?
> 
> [    6.149595] igbvf 0000:00:00.1: Assigning random MAC address.
> 
> 
> 
> [    6.151552] igbvf 0000:00:00.1: PF still resetting
> 
> [    6.153807] igbvf 0000:00:00.1: Intel(R) I350 Virtual Function
> 
> [    6.153822] igbvf 0000:00:00.1: Address: 32:91:70:c5:62:bd

All of this is kernel functionality, which driver maintainers may be
in a better position to help with. At the first glance the Xen
version in use may be of no relevance here.

> and thiis on the host:
> [  390.776779] xen-blkback: backend/vbd/3/51713: using 1 queues, protocol 1
> (x86_64-abi) persistent grants
> 
> 
> [  395.849996] pciback 0000:07:10.1: xen-pciback: Driver tried to write to
> a read-only configuration space field at offset 0x1d6, size 2. This may be
> harmless, but if you have problems with your device:
> 
> [  395.849996] 1) see permissive attribute in sysfs
> 
> 
> 
> [  395.849996] 2) report problems to the xen-devel mailing list along with
> details of your device obtained from lspci.
> 
> 
> [  396.103595] pciback 0000:07:10.1: enabling device (0000 -> 0002)
> 
> 
> 
> 
> However if I pass through 07:10.4 it works fine. The host still reports
> this  above error but all works well in the guest.

"This above error" being which one? "PF still resetting" or "Driver tried
to write ..."?

> Please could someone help ? I have been using  KVM these years and just
> moved to xen and hence new to xen.

Well, if one of the VFs works and the other doesn't, then a close
comparison of the various logs (hypervisor, kernels, tool stack) may
already provide an indication of the problem. Above you only supply
individual lines from logs, which isn't going to be sufficient to derive
anything.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:52:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:52:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179407.325664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9QX-0003FS-UQ; Mon, 06 Sep 2021 07:52:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179407.325664; Mon, 06 Sep 2021 07:52:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9QX-0003FL-QX; Mon, 06 Sep 2021 07:52:25 +0000
Received: by outflank-mailman (input) for mailman id 179407;
 Mon, 06 Sep 2021 07:52:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mN9QW-0003FF-HI
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:52:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b5d175d5-203b-4096-b9e9-d4f224e47cbd;
 Mon, 06 Sep 2021 07:52:23 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-J4sBdIofMLyCw3Xre7vYZA-1;
 Mon, 06 Sep 2021 09:52:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Mon, 6 Sep
 2021 07:52:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 07:52:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0169.eurprd01.prod.exchangelabs.com (2603:10a6:208:aa::38) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Mon, 6 Sep 2021 07:52:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5d175d5-203b-4096-b9e9-d4f224e47cbd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630914742;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7qfYe0MVEmrcUZ1IysB0i36pFwGoNwanHSmnwKiOeh0=;
	b=ioffoHOUvvwp7B6rWTm0TLucXCpthR08D+Luu53cmOQQEghIVbcAvfHcB8YyrjI1O3PscG
	IMdhKAkM35XtQpgYNUzrhQjx5vvikol7OyjjGiw1L3/dXnZl1ktYnPXCfFBGcr3UkQ75yr
	OE+LCUAVhkgwg97sWs6uyPc3Y3ltYdI=
X-MC-Unique: J4sBdIofMLyCw3Xre7vYZA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kSBZKt7WpvrLaPsX36LVFe2MBkhIPHObVSsG0VaaX/FzPh5DQpIB1A0vhKuxLCqML/unMl/CHGi+LnJsPamYyS59nmUwMFM+kcz8efuuHI3vvrvMr3KVzciQEUg8bWjZPXU+kc1tmfrdmGLAGxnsSwZkUv1UcBali2jeFcv/KmEyRIR433GzfetW15VM/dRqngA1p42dzvDZkIPUAXbUKUX4Kxsh2mYA5qCU8HWs38KE8uhcKIO6o+/ByA1PCE7zROgggoItfoqcfs+R8zQMjB3ooY3glRxTmv8pqXzEpKOel8XQpmcejF6ZziwIUtMq4RnzeDE+8IRnCQ6fU1+O4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7qfYe0MVEmrcUZ1IysB0i36pFwGoNwanHSmnwKiOeh0=;
 b=W3oVsyHyxIPcBbqn8IWX4EDYbv2rWkdUDxI4K/3kHQoCx3+W7BPrmy2knvRE1hBjShoQxNx8LxrrFEvV+XJwx6hGEksm/ucHkqfSVRW9wgk2/0x469lOvJBdt49wzMkj8vfrc4M1ir5ylLJ7DlWeVkbtr0HlScdZ9LwJTg/k+C3CFVK3bJXBzsGlc7iaugicQUh3OAtVdZ8pgYmJQERLVJjq9I0dWNE/RF6EDUI0pG+OSlotIHg5vvyzsDsFSqD+VEifX0+R9+36HY/4Vg2ryAQHP/hrpA0fmoNcvnk6Q4RLp2ZZfXk4AZRGT93WV0rceyqXbH9CLRlIxUAauE1C7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Ballon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
Date: Mon, 6 Sep 2021 09:52:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0169.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:aa::38) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 330106e2-2405-4fb5-be32-08d9710b4083
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260727A0C4D6A75617949D2FB3D29@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CUF4WvX9MiLywgBKoR4NXK1izs8+nkW6MmnYvksLWK6v8QH3QfW+nmklUo7uM7qJpEelGSt4m1VI+btS7WmgoDIqmASuCXYiRhz57t+kRImiuUVGbJmng3OeuhJyf17ZjUo6dDvWVoFfSPxido2xxuutLmwN8x1iosxzpyuBS8uZshmHYgSPeGAQPO9fww/whGmGOEcQfDZhOLmj8GAOkkZAYmjKkDxDLNuPUwDq5Jt5j7J90WuX+6I+qWA5oNcG18l4riPkn6emsLi4ReT5Hq7l9Sw5RvGYtlT3+oCS/5ru9P7ITvNM/+yIR2x4ja1sXxDwKeG9T25MH45kf6H/m37s0fN5IkNgcqSzYshHLmFT2R+q5LqPVGregy+mCpYTM7w9o7Er05JtfI8NJ/VkoHw3LeT6OGzvR0CkgmgrT8hZkueb56VbvvvvGYkmQCcjf8HMzo/ko6tVSqbzS+3+xeLQBNsXWMdJyg7IwHXhZWtRLmxbK0D58PkXht6qJvUbkKgyuhv5JHBWCasS4JS/6Sg6lZITiK3hGAjx6jzNUQqnLqxslu+u3dMq5RFqkcRZmtkN2OQDJLBM6a+PDVXyJjVMJ9BX+0jXFmOW7uVI4UKuCBVF34xheBRqkGqoHduqPIm64io544anmmYDC7fayNDHeWZA1xTomx0T5J9qekjdoJWUm6abBpg4c4oUdSOFn8URGxQ2LAxs7SH6ydGN6A==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(396003)(39860400002)(136003)(376002)(366004)(478600001)(36756003)(31696002)(66476007)(8936002)(66946007)(2616005)(956004)(2906002)(3480700007)(31686004)(53546011)(66556008)(316002)(86362001)(6486002)(16576012)(4326008)(5660300002)(38100700002)(26005)(8676002)(83380400001)(186003)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QTd5eit1Sm9XU0FQTmdHVEdpb0dVV1pHanBaek5HdkUzRDJzUVhyZ2piS3ds?=
 =?utf-8?B?UytoRDVBbExHUWM5ak9YRUNSYUhNTHU0SUpUa3hKMFlrSXZ1c1lrWXRhRWlV?=
 =?utf-8?B?S0tNQzczMlBaOXJDUERPK0lISkx3WGxuVFhWUkZHa3luUlB5UFY1eGNYMHF6?=
 =?utf-8?B?RjZBNlc5ZUZyQ0c4MUsrS09aL2NndyttdWtvOUNYMitOSFNjMEtqNVVHQVVS?=
 =?utf-8?B?aElHQ2xIU1hMajhtajR5dFIyUEFLYytXeEY1cGI3OHVtRTZwd0xZUGRHYWI3?=
 =?utf-8?B?WTNQdTJSQ243WHNaWlFSdUVJSUdpa0dab3VqYXdWUGFmcVcrUmZ4WkswT2RR?=
 =?utf-8?B?UGMrbmdrSnRBMGR1L045SjE5RGxsOHRyV1F3VmVBYStVWjNBZGdiVk8rak9k?=
 =?utf-8?B?Q3d6eFg3L2wwZkY0bkN1ZkNGaFMvU2RpRVRNa08yMFFXTzIzYzFZdVREdENw?=
 =?utf-8?B?Sk5BLzNYVUtqREFialQranhoNkFvaUYvZnIzY05DY2ExUVFsZGVlRjhva3hh?=
 =?utf-8?B?K1lMTlh1dVpCaG9McFQrWDZnWHoxOEVtV1hZUjhEN0ZRTFo0cFJJWHJzUzgy?=
 =?utf-8?B?QUpRRXB6bnMyNjVnVHVrdWx1WFkweWtmY1R1QXVxTHFBeTNabjFVQVlJRGU3?=
 =?utf-8?B?UXV1LzRwUzIxOUplWjRtakZWRHIzeTVkWHg5K2dNMS92N3ZHcVErQTA5azRO?=
 =?utf-8?B?aU1UMnhvRlVsN0Z1Q2UzL2ZEVmZuZGovdU1MdW9qYXZOZFFsWkdIOExVNnNG?=
 =?utf-8?B?SW5JUDBYUjRCZkVsT1V5UWg3NlNDLzFhN2QxbUZ5OXBKUy9pazBqcTNGWUJ5?=
 =?utf-8?B?bDBRMDd3T3dwZFJNeTFGdGlVOVpzSGxzcXZIaCtFclNuYjd5eURUT0tSWHZx?=
 =?utf-8?B?cStXbmdBZ3MzbzRzRkttZTBOcGpaRGNLSXlTL0pHYzRnMXgwQVFQRXJYYlF3?=
 =?utf-8?B?R2grOWhRaDFQdU1zbGJIbXBjY0E1QmEwc0daSk45VGR5WktRbEpkRE94WDhJ?=
 =?utf-8?B?K3lIaWFsbmt2djE1Z2I3UHhEa0l0T05BUGxucUcwMWcwZnpUMUQ1UGRualZS?=
 =?utf-8?B?RHVYcVlldWRDMWJzYWhuSEpxTE81QVJDVmptMmQ2MmYwZTRWOFgrNXBUYXUv?=
 =?utf-8?B?OGlmK1pFa25KbDhVbE1kMlZlTmRldCtHMjgxSEhQbnVoM1dRaEdORS93eVdm?=
 =?utf-8?B?MGdYdThiUWpzRittcE5xVVQzTGdWbDV3T2IzRjVkYjRrT0ZVb0R0QlZZRWho?=
 =?utf-8?B?L1dqeWpxQ01zZHR4VHZGSDJhUW13aVRubVFmRnhpTVVCN0l3NUdGaUJGTUNX?=
 =?utf-8?B?bkozSWw5NXUzMkY5L1BoZk4xWk5zNnRFZ1ZPUm5YanpFRmdjc1dpTUgwRTZH?=
 =?utf-8?B?Rjl3WFNaZWNkZXNzM3p5c0N4SFQrZlY1OUp4QTJ1dmxnRFBxS3ZXSDZUSG1j?=
 =?utf-8?B?QkZPL1NqZXduMHdRVFhYTnlVMUI5aGhwS2FJcDMvNW5wRmJJK05kQzZrdDNF?=
 =?utf-8?B?akZwQk53RkVXRVc2aHExQ3Rtd2F2VlhxSXQ4WE15NURSNUxHTTNML0xERVgw?=
 =?utf-8?B?SVMzK2Z2NUUrWHVnMHgzT0wxaTdiaEJSQy8wWC95T1llWnhTZjI0c0NrRm5O?=
 =?utf-8?B?NmdkWmdCb015OXB3eDJIR1JGZlhXZkFKcnlvbmNoQkFwSmtTejdybWhnMXhH?=
 =?utf-8?B?MnYzWGF4TU02N1FjdjdWK2VtZjNJR3UxWWJzcXMva2o2U2hUUzZpOVNDNzFT?=
 =?utf-8?Q?16F+saPMZBsHHN8/V4I7ccIsgzL2RIzFt5yTT1R?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 330106e2-2405-4fb5-be32-08d9710b4083
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 07:52:19.2669
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: N4+tzR+CZWYFj46pFiCEM0ItKO4MTF29fJGWUhhcFF2yremcsivMnXVV5zWjErPUPd1goG22F/WxEmx6EtUhMA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

On 06.09.2021 00:10, Elliott Mitchell wrote:
> I brought this up a while back, but it still appears to be present and
> the latest observations appear rather serious.
> 
> I'm unsure of the entire set of conditions for reproduction.
> 
> Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
> this is an older AMD IOMMU).
> 
> This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
> Debian's patches, but those are mostly backports or environment
> adjustments.
> 
> Domain 0 is presently using a 4.19 kernel.
> 
> The trigger is creating a HVM or PVH domain where memory does not equal
> maxmem.

I take it you refer to "[PATCH] x86/pod: Do not fragment PoD memory
allocations" submitted very early this year? There you said the issue
was with a guest's maxmem exceeding host memory size. Here you seem to
be talking of PoD in its normal form of use. Personally I uses this
all the time (unless enabling PCI pass-through for a guest, for being
incompatible). I've not observed any badness as severe as you've
described.

> New observations:
> 
> I discovered this occurs with PVH domains in addition to HVM ones.
> 
> I got PVH GRUB operational.  PVH GRUB appeared at to operate normally
> and not trigger the crash/panic.
> 
> The crash/panic occurred some number of seconds after the Linux kernel
> was loaded.
> 
> 
> Mitigation by not using ballooning with HVM/PVH is workable, but this is
> quite a large mine in the configuration.
> 
> I'm wondering if perhaps it is actually the Linux kernel in Domain 0
> which is panicing.
> 
> The crash/panic occurring AFTER the main kernel loads suggests some
> action by the user domain is doing is the actual trigger of the
> crash/panic.

All of this is pretty vague: If you don't even know what component it
is that crashes / panics, I don't suppose you have any logs. Yet what
do you expect us to do without any technical detail?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:58:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:58:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179415.325674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9WL-0003vf-K8; Mon, 06 Sep 2021 07:58:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179415.325674; Mon, 06 Sep 2021 07:58:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9WL-0003vY-HC; Mon, 06 Sep 2021 07:58:25 +0000
Received: by outflank-mailman (input) for mailman id 179415;
 Mon, 06 Sep 2021 07:58:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mN9WK-0003vS-7d
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:58:24 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 28db596b-d2d7-4fdb-a851-1798376585ed;
 Mon, 06 Sep 2021 07:58:23 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 168C920090;
 Mon,  6 Sep 2021 07:58:22 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id BD3A31332A;
 Mon,  6 Sep 2021 07:58:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id vXixLB3KNWElZQAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 07:58:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28db596b-d2d7-4fdb-a851-1798376585ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630915102; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=1Za9VHhRvpXITUKCSZL629zvN/BxLscJdKXjSFiZ2+g=;
	b=SiROr0rslCXdR3AGf61xOFA29IRHYyXipM6FrpCmTeFR6QzuOwuwIj9O6tGyi4w7IdgpQY
	rQUKoV6RQUh034xIePoQ9aFy1OPw1hvpPdV677UUuE1v9rvG9YHepYgMnzSgcJQTs7YNN8
	Hx2rR3JWdu7uFe9jp0mFaE9gyHnmsgs=
Subject: Re: [PATCH] x86: xen: platform-pci-unplug: use pr_err() and pr_warn()
 instead of raw printk()
To: zhaoxiao <long870912@gmail.com>, tglx@linutronix.de, mingo@redhat.com,
 bp@alien8.de, x86@kernel.org
Cc: hpa@zytor.com, xen-devel@lists.xenproject.org,
 linux-kernel@vger.kernel.org, boris.ostrovsky@oracle.com,
 sstabellini@kernel.org
References: <20210906075516.15066-1-long870912@gmail.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <af68f9f5-3b6e-162f-5624-e398b744a268@suse.com>
Date: Mon, 6 Sep 2021 09:58:21 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210906075516.15066-1-long870912@gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="0NiT60Arq9oUmIaGGHFc8tMs0HEluEKwq"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--0NiT60Arq9oUmIaGGHFc8tMs0HEluEKwq
Content-Type: multipart/mixed; boundary="OnXWnoypXks5k2gkdaUdrAwQMOYsNyAmr";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: zhaoxiao <long870912@gmail.com>, tglx@linutronix.de, mingo@redhat.com,
 bp@alien8.de, x86@kernel.org
Cc: hpa@zytor.com, xen-devel@lists.xenproject.org,
 linux-kernel@vger.kernel.org, boris.ostrovsky@oracle.com,
 sstabellini@kernel.org
Message-ID: <af68f9f5-3b6e-162f-5624-e398b744a268@suse.com>
Subject: Re: [PATCH] x86: xen: platform-pci-unplug: use pr_err() and pr_warn()
 instead of raw printk()
References: <20210906075516.15066-1-long870912@gmail.com>
In-Reply-To: <20210906075516.15066-1-long870912@gmail.com>

--OnXWnoypXks5k2gkdaUdrAwQMOYsNyAmr
Content-Type: multipart/mixed;
 boundary="------------6D93742B032AE25FF0D27A00"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------6D93742B032AE25FF0D27A00
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 09:55, zhaoxiao wrote:
> Since we have the nice helpers pr_err() and pr_warn(), use them instead=

> of raw printk().
>=20
> Signed-off-by: zhaoxiao <long870912@gmail.com>

Any reason you are sending this patch again? It has already been applied
through the Xen tree.


Juergen

--------------6D93742B032AE25FF0D27A00
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------6D93742B032AE25FF0D27A00--

--OnXWnoypXks5k2gkdaUdrAwQMOYsNyAmr--

--0NiT60Arq9oUmIaGGHFc8tMs0HEluEKwq
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE1yh0FAwAAAAAACgkQsN6d1ii/Ey98
Lwf/QVhNeKbTqGC3/WByP2vXlbyE9+y9YugswvSoksaY4qBNYUsKbey1CU/ZTeZ1rKmzTzKUoh2I
Rq/atOQS41km+txxg1iOdLcKotJ8+HSu1oz72cdi5SYXxPDV6PUBtyIzgcgOAmmyn5fImCMZgLYs
dMSR6W46ffQ4KEE/Es7shi/MgADXR2H5axMyNaQqqdwvfIxrcLV2RLiEnZIYGHXyvO8WvFk8jZo4
xAyAxcOZ+/RNLl+4XtWYX8VpcHRVawzjDIZAP4o12ei77G2mKXJFhlcLfNpmIb6osbrEXvInK/pR
ZIVjJAonB1PpZjTRq0dPqxLAcfa8jI+O1jLoJ/4O5g==
=8ISr
-----END PGP SIGNATURE-----

--0NiT60Arq9oUmIaGGHFc8tMs0HEluEKwq--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 07:59:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 07:59:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179421.325686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9X8-0004U7-VK; Mon, 06 Sep 2021 07:59:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179421.325686; Mon, 06 Sep 2021 07:59:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9X8-0004U0-Rd; Mon, 06 Sep 2021 07:59:14 +0000
Received: by outflank-mailman (input) for mailman id 179421;
 Mon, 06 Sep 2021 07:59:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kxtw=N4=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mN9X8-0004Ts-8R
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:59:14 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.67]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 02416458-7fb2-4eb4-adef-abbbdb4859e6;
 Mon, 06 Sep 2021 07:59:12 +0000 (UTC)
Received: from AM6PR10CA0084.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:8c::25)
 by AM6PR08MB4294.eurprd08.prod.outlook.com (2603:10a6:20b:bd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Mon, 6 Sep
 2021 07:59:10 +0000
Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8c:cafe::6a) by AM6PR10CA0084.outlook.office365.com
 (2603:10a6:209:8c::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Mon, 6 Sep 2021 07:59:10 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 07:59:10 +0000
Received: ("Tessian outbound 19a632698c93:v103");
 Mon, 06 Sep 2021 07:59:09 +0000
Received: from bf2941014b77.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EDCB66D2-3E83-4673-962B-47BE55C9FD1B.1; 
 Mon, 06 Sep 2021 07:59:02 +0000
Received: from FRA01-PR2-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bf2941014b77.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 07:59:02 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR2PR08MB4874.eurprd08.prod.outlook.com (2603:10a6:101:1d::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 07:59:01 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 07:59:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02416458-7fb2-4eb4-adef-abbbdb4859e6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wqT3OTunmQEuYn/hPnbl+2ZS4xB6pnzJeMCS5J/gFo0=;
 b=1CigprJlfSjfO99h6CWRiRHI1AL4fV+efrm2/TjS5mCFgzwplQb4yVGvY+CTYFJ1JJZfruCkcigPDIHmurzsL/iYJzioTas0ASTs9q7wPCLIXimt2lPFC2h8hf8EFALoYhVEiKftUDL6u77cximqotCYRsqHKDkBie6uJHTbN4M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cbf86ecf56087163
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kOaNrl38Xj3PI4AOCqtzL5SsAYpkh8esndWQhkQ2W5ffZNG84QPmVRJjwCLfMaRU9HFkooDrKoqan7XVLQrQZhV55dbM6/wWPbPA+Nc0wCgrzZG9nrQdybJfxGBGZ+3uZ2fUj+fvzBD3tyV8lsud7BDVdHX6QYmt5h3t1ytwDjA1uFh4BXIi+/RZRUkgrlBzMSDrs7/b+ht3Pu0qUVRM2xw/Jv83EfJD1GGNikJtkSjutN5EkGrqSBKdmOHGjJEVCVU5CIdu9NCiTG5TRTdnKMOpy3YRmjkmj7v5bot59Vb5JN0FGiyjjwqwZJng44GtTNDk4qAVEYi8eWjTXpn86A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wqT3OTunmQEuYn/hPnbl+2ZS4xB6pnzJeMCS5J/gFo0=;
 b=HsNcM+Eb2ckqiboIeDKnPVu/CB1TgSWGd2KpPkrAFHm3ljb9XD6287WiXFKLnxLaca8bUQUnz4gQYeO5CSYO18GVIt1dl6sVJI9jTvfXSaDQIE+6U9pgywloq6/ctZ/B4UpZ2iDM/kmmfaEwk18C97bvL9UboAGyDC9CYqx+H3t1T04nJkorMzupdmfSvWwUM8nC5uk3Gu0c19THvcLHGeesUjFOsStMZwKBxXxWN+hUqIk0G62uKNvbB7Qph0nWj9/FTXNiS70fzK+KHFmQcDL76FoTCZgcqw1nVuQdwLjHYb5MAUgeH+yKsitlPXjoIR2zju2P7aQ5b6Hr/Q5RZg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wqT3OTunmQEuYn/hPnbl+2ZS4xB6pnzJeMCS5J/gFo0=;
 b=1CigprJlfSjfO99h6CWRiRHI1AL4fV+efrm2/TjS5mCFgzwplQb4yVGvY+CTYFJ1JJZfruCkcigPDIHmurzsL/iYJzioTas0ASTs9q7wPCLIXimt2lPFC2h8hf8EFALoYhVEiKftUDL6u77cximqotCYRsqHKDkBie6uJHTbN4M=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
Thread-Topic: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
Thread-Index: AQHXmbPyvc9RSYz5OUKBEYoCXmX0t6uS+GGAgAO+T4A=
Date: Mon, 6 Sep 2021 07:59:00 +0000
Message-ID: <DDB381B4-1519-43B5-9DA5-DE0C0E11916B@arm.com>
References: <cover.1629897306.git.bertrand.marquis@arm.com>
 <c1868fce1bcb0bcddf7bc786be166007f91f2f67.1629897306.git.bertrand.marquis@arm.com>
 <alpine.DEB.2.21.2109031529400.26072@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109031529400.26072@sstabellini-ThinkPad-T480s>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b915c3f8-054c-4e7c-cd21-08d9710c3597
x-ms-traffictypediagnostic: PR2PR08MB4874:|AM6PR08MB4294:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB42949E96D58FDB136F4F6A929DD29@AM6PR08MB4294.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 3kXZxKy3Nxd+lLfGRINpjkMOHc74N/vpeTzhNSMzG6leXwfDMaIeSXQQzDyWioFZtk0fFNGuxC2wdDrvlL4/Tje4L9vZd8NORw0EaVN8JbjoK+oXMh2Q3hHxaIjLIt8p390ZDFL3l1PTdm6gt+vmCzzYAZKyVeHEoAthD70xQMCkwosGN3IfcxiIdhh0NwDFekt7vRYc8fttuwr+pmdFRL9A7Axn3nou2E/LogJxNjDsFBl0oUpYAnn7Ew+Bd7igGnWn4CAL4KjXIZ1JdV3SBl9wrTVN1UXe/VO2+WiVRPPntICtAjStrPyNUN1hoEGrPAwfx3xvdVmpIZwgfYZE54UvuhEuSRzC0RJybXrYAyzpPUm1d+CXzgHLYF1lMn4Usbx+fA6XEjZjLYCdaH0gklpcReLM+JnqgCeLIlMECvniD7wyTwG0y3pwZuPvrEqH9Yx+n7YquwlXDfnrFFmKmg/09/zia0YZx+gFc/nTsbN+jqIPbn0HkxX0csj02mDigdh5+lw7Ekz2Y1L35HjnVIRIdvdhXXchB37kmNXLyVhS+I6Dpb/9JkfNSEUiyfSbUow+K1kYWmgTpp6Z+shKvs9dYRQOJgoJBZFgyG/4AJ5hvVhLy+P8LTKPLjwqnkKOJYS60nMH1sk4+WK/qMWsSo9iNBGyQDTKAuvTzJrwtRI6iHU/AMm9bW0ga/rrfoAd5sn8Ed6LydXF+TzbsZS4mLVbbE8omm28/Y/qdaEKNXRLDNH05hOtCeWW1AkAqa4e
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(39850400004)(366004)(346002)(136003)(54906003)(316002)(5660300002)(2906002)(122000001)(2616005)(38070700005)(64756008)(6506007)(66946007)(53546011)(66446008)(6486002)(76116006)(91956017)(83380400001)(8676002)(66556008)(8936002)(66476007)(38100700002)(26005)(6916009)(86362001)(186003)(71200400001)(36756003)(478600001)(33656002)(6512007)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <CCDE26D932ED4648AD77B2D677D78978@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4874
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d153bbb4-0551-4513-384b-08d9710c3014
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	e7gteVfGcZkygifGTQs7+6Hjnd8+8eU8mK/kGobYyJRsk6hgL7HGUzxpfWhglH6E3jgwfdPULlsd07A3pLUl81hhEe3w9uLojy9B1rACZYg+LzZoBgQZURTN2Scc2Wv6j5DhDmsmoStIroXRbvCQP3wRj44f2O3lhYICYIjuey93nHzC7VGxl+QUOQsmg6p14w4wsxGOUOeMTVy0aRuDlwTD55JVJb2sIi37MlgM9E6P5qi6oyE1lawegdguyKehMaD0sQB5dtorfUGTFvQnUvdx7aw83dMqQRL9S6IgQT766piwInYqgypSWtExWWMMbxYWktbhFFR7ZTpMegi+MJcOT6YZ23tFt4gFvghZgw9/ZmP+/UTaqayBiuXNyTs4f7Cf+ABKXhEnga6A/4DmoOtQ6V7rCnkqYlYKsrGRi2za06F4RFMOERWR5d+gjKWqtE4eqD/NsAq40aKMf2kUQbUsaUz7F6mYFMARnjbbOwkY5zRWUET95dcPtV+vIH9YewpY3OAMfLnfYe0Nox+5y75qroxG1W49xjQF88Z3uQt6Ozly4fcE+gLuXYHTeuHxFWywuztBQscDWURAC3h+0LAcGrp8wyhJ4Jx8ykLsIeiVEIlp1B/mR/LGr+x2PJAVva2ImPEJ2IiKWZ4sw8X3Q0EvoiMy3ewzZbWi5q9za4seYjPtc4ofprm9PhbWof+4ol19QPlyKr3aHQSXLodvKA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39850400004)(136003)(376002)(396003)(46966006)(36840700001)(5660300002)(82310400003)(8936002)(186003)(6512007)(6862004)(26005)(6506007)(83380400001)(107886003)(54906003)(478600001)(8676002)(356005)(81166007)(4326008)(316002)(36756003)(36860700001)(33656002)(336012)(2616005)(6486002)(82740400003)(70586007)(2906002)(47076005)(70206006)(86362001)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 07:59:10.0910
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b915c3f8-054c-4e7c-cd21-08d9710c3597
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4294



> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrot=
e:
>=20
> On Wed, 25 Aug 2021, Bertrand Marquis wrote:
>> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
>                                        ^  TAINT
>=20
>=20
>> are found (ftr_dczid is using only STRICT method).
>> In this case actual memory being cleaned by DC ZVA operations would be
>> different depending on the cores which could make a guest zeroing too
>> much or too little memory if it is merged between CPUs.
>>=20
>> We could, on processor supporting it, trap access to DCZID_EL0 register
>               ^ processors

Could those typos be fixed during commit ?

>=20
>> using HFGRTR_EL2 register but this would not solve the case where a
>> process is being migrated during a copy or if it cached the value of the
>> register.
>>=20
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> ---
>> Change in v3: none
>> Change in v2: Patch introduced in v2
>> ---
>> xen/arch/arm/arm64/cpufeature.c  | 14 +++++++++++---
>> xen/arch/arm/cpufeature.c        |  2 ++
>> xen/include/asm-arm/cpufeature.h |  8 ++++++++
>> 3 files changed, 21 insertions(+), 3 deletions(-)
>>=20
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufea=
ture.c
>> index 61f629ebaa..b1936ef1d6 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] =3D=
 {
>> 	ARM64_FTR_END,
>> };
>>=20
>> -#if 0
>> -/* TODO: handle this when sanitizing cache related registers */
>> static const struct arm64_ftr_bits ftr_dczid[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, =
1),
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT,=
 4, 0),
>> 	ARM64_FTR_END,
>> };
>> -#endif
>>=20
>> static const struct arm64_ftr_bits ftr_id_isar0[] =3D {
>> 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_=
SHIFT, 4, 0),
>> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_ar=
m *new)
>>=20
>> 	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>>=20
>> +	/*
>> +	 * Comment from Linux:
>=20
> I don't know if I would keep or remove "Comment from Linux"

I added that because the comment itself does not really apply to Xen.
I could have rephrased the comment/
Anyway I have no objection to remove that statement.

Do I need to send a v2 for that ?

Cheers
Bertrand


>=20
> In any case:
>=20
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>=20
>=20
> Also I gave a quick test of the series on a ZCU102 and it worked fine.
>=20
>=20
>> +	 * Userspace may perform DC ZVA instructions. Mismatched block sizes
>> +	 * could result in too much or too little memory being zeroed if a
>> +	 * process is preempted and migrated between CPUs.
>> +	 *
>> +	 * ftr_dczid is using STRICT comparison so we will taint Xen if differ=
ent
>> +	 * values are found.
>> +	 */
>> +	SANITIZE_REG(dczid, 0, dczid);
>> +
>> 	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> 	{
>> 		SANITIZE_ID_REG(pfr32, 0, pfr0);
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index f600a611bd..113f20f601 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>=20
>>     c->zfr64.bits[0] =3D READ_SYSREG(ID_AA64ZFR0_EL1);
>>=20
>> +    c->dczid.bits[0] =3D READ_SYSREG(DCZID_EL0);
>> +
>>     aarch32_el0 =3D cpu_feature64_has_el0_32(c);
>> #endif
>>=20
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpuf=
eature.h
>> index 52cb3133e0..5219fd3bab 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -259,6 +259,14 @@ struct cpuinfo_arm {
>>         register_t bits[1];
>>     } zfr64;
>>=20
>> +    /*
>> +     * DCZID is only used to check for incoherent values between cores
>> +     * and taint Xen in this case
>> +     */
>> +    struct {
>> +        register_t bits[1];
>> +    } dczid;
>> +
>> #endif
>>=20
>>     /*
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:00:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:00:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179413.325696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9YV-0006Sj-Pv; Mon, 06 Sep 2021 08:00:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179413.325696; Mon, 06 Sep 2021 08:00:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9YV-0006Sc-MJ; Mon, 06 Sep 2021 08:00:39 +0000
Received: by outflank-mailman (input) for mailman id 179413;
 Mon, 06 Sep 2021 07:55:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tq2V=N4=gmail.com=long870912@srs-us1.protection.inumbo.net>)
 id 1mN9TR-0003rK-3Z
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 07:55:25 +0000
Received: from mail-pj1-x102b.google.com (unknown [2607:f8b0:4864:20::102b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 038a7c4b-e7d3-4c03-98ac-9450a760c7b3;
 Mon, 06 Sep 2021 07:55:24 +0000 (UTC)
Received: by mail-pj1-x102b.google.com with SMTP id
 mj9-20020a17090b368900b001965618d019so3770772pjb.4
 for <xen-devel@lists.xenproject.org>; Mon, 06 Sep 2021 00:55:24 -0700 (PDT)
Received: from localhost.localdomain ([111.207.172.18])
 by smtp.gmail.com with ESMTPSA id u10sm6789616pfn.147.2021.09.06.00.55.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 06 Sep 2021 00:55:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 038a7c4b-e7d3-4c03-98ac-9450a760c7b3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=2Xf56OLSp+hppzWvWhEGSXjWhxDg/ErT9f349mtnYmE=;
        b=Tj/b5umNNUQO9OTj3KVUsNndVD9R/+79WMav7vc/xEFnF8t6aarL8lMZq2aqHDGjCK
         ccCjcuxmO+xSG+V1DpEK/wSnmGUtxvRS7In/KdYAdecRoFxbTVPjHkMJhX1og+8YJtaC
         OLziIHDZfUChb6k7TVrMezl5ZwtSGzqPzoYTQg4s2nYvNQyXISHO0HeOxBs6f8ad4BLt
         zQIqvCluOoylVSb2kc/zCJtLB4aF9PnF1gOhDpZ+nCqm2OsEIGsvyInV7GyNTUgoO+X1
         h1701OZ+VbZSUDUHA7PgUN50uYWh65fuh9owuw2BMzW/e2BF0BOAevtd8BEoSCpMPMaW
         f9mA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=2Xf56OLSp+hppzWvWhEGSXjWhxDg/ErT9f349mtnYmE=;
        b=q+YEqNrje2pq+4BM0nsVB3W8OnKZux099mFCHWIrVmglO7mDvJNJBolSn8dVIMy0ME
         41RdbsdOpCW77n0/MQH9Um5qXCQ+gyghn/FxnMVV9EK/mdknDq8DW6jEiIHRtZdauEq4
         y3ws8SDyENJnCIGI79SLtLLZD19lf8//M+NJej8Er7/9jL7QrPxCyRJK6xrPuHJ94tC1
         2tO7bnWofFQkWb7eMtgW9UXsWfg47SIIJNC1J1t1gc2UAsku37LIz9y/YyHSQO9gO46E
         BG1pq+SdZOmykjjeCcaKPL2TO8UYuPZG/ue6W/gxBqi01bbiDFk6kouBFJIKB3yQzmO5
         JZSw==
X-Gm-Message-State: AOAM530Z9oTg09pKxbVCiaL3s1GZzqEAL1OmpIRyWBj9Vp/ZuX6rPJOZ
	P8HZgRyIf/GL0RaxGze3jPg=
X-Google-Smtp-Source: ABdhPJxd0pw9THgbbgqH1OA2JXyGSDCt9tbjRnSLlddXY9HZasP9LDEct/XwtdvVy+oXt3MppnsVUg==
X-Received: by 2002:a17:902:a503:b029:12b:2429:385e with SMTP id s3-20020a170902a503b029012b2429385emr9853168plq.64.1630914923382;
        Mon, 06 Sep 2021 00:55:23 -0700 (PDT)
From: zhaoxiao <long870912@gmail.com>
To: tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org
Cc: hpa@zytor.com,
	xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	zhaoxiao <long870912@gmail.com>
Subject: [PATCH] x86: xen: platform-pci-unplug: use pr_err() and pr_warn() instead of raw printk()
Date: Mon,  6 Sep 2021 15:55:16 +0800
Message-Id: <20210906075516.15066-1-long870912@gmail.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Since we have the nice helpers pr_err() and pr_warn(), use them instead
of raw printk().

Signed-off-by: zhaoxiao <long870912@gmail.com>
---
 arch/x86/xen/platform-pci-unplug.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/x86/xen/platform-pci-unplug.c b/arch/x86/xen/platform-pci-unplug.c
index 96d7f7d39cb9..6586eaf59c00 100644
--- a/arch/x86/xen/platform-pci-unplug.c
+++ b/arch/x86/xen/platform-pci-unplug.c
@@ -19,6 +19,8 @@
 #define XEN_PLATFORM_ERR_PROTOCOL -2
 #define XEN_PLATFORM_ERR_BLACKLIST -3
 
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
 /* store the value of xen_emul_unplug after the unplug is done */
 static int xen_platform_pci_unplug;
 static int xen_emul_unplug;
@@ -30,13 +32,13 @@ static int check_platform_magic(void)
 
 	magic = inw(XEN_IOPORT_MAGIC);
 	if (magic != XEN_IOPORT_MAGIC_VAL) {
-		printk(KERN_ERR "Xen Platform PCI: unrecognised magic value\n");
+		pr_err("Xen Platform PCI: unrecognised magic value\n");
 		return XEN_PLATFORM_ERR_MAGIC;
 	}
 
 	protocol = inb(XEN_IOPORT_PROTOVER);
 
-	printk(KERN_DEBUG "Xen Platform PCI: I/O protocol version %d\n",
+	pr_debug("Xen Platform PCI: I/O protocol version %d\n",
 			protocol);
 
 	switch (protocol) {
@@ -44,12 +46,12 @@ static int check_platform_magic(void)
 		outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM);
 		outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER);
 		if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) {
-			printk(KERN_ERR "Xen Platform: blacklisted by host\n");
+			pr_err("Xen Platform: blacklisted by host\n");
 			return XEN_PLATFORM_ERR_BLACKLIST;
 		}
 		break;
 	default:
-		printk(KERN_WARNING "Xen Platform PCI: unknown I/O protocol version\n");
+		pr_warn("Xen Platform PCI: unknown I/O protocol version\n");
 		return XEN_PLATFORM_ERR_PROTOCOL;
 	}
 
@@ -155,12 +157,12 @@ void xen_unplug_emulated_devices(void)
 	 * been compiled for this kernel (modules or built-in are both OK). */
 	if (!xen_emul_unplug) {
 		if (xen_must_unplug_nics()) {
-			printk(KERN_INFO "Netfront and the Xen platform PCI driver have "
+			pr_info("Netfront and the Xen platform PCI driver have "
 					"been compiled for this kernel: unplug emulated NICs.\n");
 			xen_emul_unplug |= XEN_UNPLUG_ALL_NICS;
 		}
 		if (xen_must_unplug_disks()) {
-			printk(KERN_INFO "Blkfront and the Xen platform PCI driver have "
+			pr_info("Blkfront and the Xen platform PCI driver have "
 					"been compiled for this kernel: unplug emulated disks.\n"
 					"You might have to change the root device\n"
 					"from /dev/hd[a-d] to /dev/xvd[a-d]\n"
@@ -200,7 +202,7 @@ static int __init parse_xen_emul_unplug(char *arg)
 		else if (!strncmp(p, "never", l))
 			xen_emul_unplug |= XEN_UNPLUG_NEVER;
 		else
-			printk(KERN_WARNING "unrecognised option '%s' "
+			pr_warn("unrecognised option '%s' "
 				 "in parameter 'xen_emul_unplug'\n", p);
 	}
 	return 0;
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:11:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:11:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179443.325708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9jG-0008KG-Qy; Mon, 06 Sep 2021 08:11:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179443.325708; Mon, 06 Sep 2021 08:11:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9jG-0008K9-N8; Mon, 06 Sep 2021 08:11:46 +0000
Received: by outflank-mailman (input) for mailman id 179443;
 Mon, 06 Sep 2021 08:11:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tq2V=N4=gmail.com=long870912@srs-us1.protection.inumbo.net>)
 id 1mN9jF-0008K3-Lj
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:11:45 +0000
Received: from mail-ot1-x329.google.com (unknown [2607:f8b0:4864:20::329])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1060a139-3710-4982-bfd4-ad1603b2a747;
 Mon, 06 Sep 2021 08:11:44 +0000 (UTC)
Received: by mail-ot1-x329.google.com with SMTP id
 i8-20020a056830402800b0051afc3e373aso7884094ots.5
 for <xen-devel@lists.xenproject.org>; Mon, 06 Sep 2021 01:11:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1060a139-3710-4982-bfd4-ad1603b2a747
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=Jqw0MwknMj1RijM4alevMH+POuK2y8El0iNuvJ4Z7sI=;
        b=fAULSwf4Pj+E1UknxqjuJ/OdTFuiM7YxgvSqmyiVuhepLdtk8HrH3yeP++ilsATvZQ
         5uUedwNnU2hl/Gmjit3Ykk/Y3BCHR93wDqaLZ7QJCWw0rAmnccIhKnwv5CdxzmyYq8eS
         SPKhucRIXtC7Kxli/P/k2eIB8OMVbsEmbEphr3fpvHiAKzPHKETrIG6q5URdo2VIVtXg
         DWJVQF9BPsQUYDkq//AiIh5sQWLLCqnK+8b1wogRQKfZZ/8dCu6bipeYSjhCz/DtiBFR
         O59TGtBnYwHvt1PEMtpSZ4fag3sx9r3sJwVlPWzfNthiZ/dNYL4F4HfPEshQFDJcFpVR
         zsgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=Jqw0MwknMj1RijM4alevMH+POuK2y8El0iNuvJ4Z7sI=;
        b=iliIaYAvbLuHpIS2crv9LTzUuT0M3Ie1IayG7m+CaDwOXgv4/Hr/dfnwSKpGH3RqQp
         EwXwKtgXQ6e0qelQX498ifCc1g5FE+W5IxE+9AqIBt1mh+47rcGu6BweFI1JsbZPUUaZ
         oIb8HFx87dXfR910+LfTyilURNS+EAZP9NQfmScakMUlC3+0vnvmc984Qurp4Lc8753d
         KuxFQLDRrVfvhXvy/N3W4YU/DL7kitippxDnNm163kgpJOmcKFXAWpp4yyrJfe0ru/2t
         tmKGF6U4bJvI3DnSWbdRufWTTxcEHFlVH9Y6UysLFvvHSFpvkVRoqUNJIK3kUGV/hml+
         2Usw==
X-Gm-Message-State: AOAM533z5KL7dsQzF22UoCwEwFOXn2lVO3PNHAIp0HB2AfSrg2SclZZ/
	VQFZf+Ak96M28Vsa5VaJmPwDLDim+eCkyKWFNyg=
X-Google-Smtp-Source: ABdhPJz++uZDYjY85Lmv2nL7XDUdCXXJBPnGtWjT5ffovBGzVwDowugFqSssiyb2f7K0TOpCH2pam3qeCpee4wMP704=
X-Received: by 2002:a9d:6e0d:: with SMTP id e13mr9580893otr.304.1630915904431;
 Mon, 06 Sep 2021 01:11:44 -0700 (PDT)
MIME-Version: 1.0
References: <20210906075516.15066-1-long870912@gmail.com> <af68f9f5-3b6e-162f-5624-e398b744a268@suse.com>
In-Reply-To: <af68f9f5-3b6e-162f-5624-e398b744a268@suse.com>
From: xiao zhao <long870912@gmail.com>
Date: Mon, 6 Sep 2021 16:11:33 +0800
Message-ID: <CAMw=WFQn7Zh_iJxt+KKqDjHr7HAkA9T0e9VF6N5nNvvfOpN7Ng@mail.gmail.com>
Subject: Re: [PATCH] x86: xen: platform-pci-unplug: use pr_err() and pr_warn()
 instead of raw printk()
To: Juergen Gross <jgross@suse.com>
Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, 
	hpa@zytor.com, xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
	boris.ostrovsky@oracle.com, sstabellini@kernel.org
Content-Type: multipart/alternative; boundary="00000000000024674305cb4f335e"

--00000000000024674305cb4f335e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Sorry, I don't know the patch has already been applied through the Xen
tree. Thanks.

Juergen Gross <jgross@suse.com> =E4=BA=8E2021=E5=B9=B49=E6=9C=886=E6=97=A5=
=E5=91=A8=E4=B8=80 =E4=B8=8B=E5=8D=883:58=E5=86=99=E9=81=93=EF=BC=9A

> On 06.09.21 09:55, zhaoxiao wrote:
> > Since we have the nice helpers pr_err() and pr_warn(), use them instead
> > of raw printk().
> >
> > Signed-off-by: zhaoxiao <long870912@gmail.com>
>
> Any reason you are sending this patch again? It has already been applied
> through the Xen tree.
>
>
> Juergen
>

--00000000000024674305cb4f335e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Sorry, I don&#39;t know the patch=C2=A0has already been ap=
plied through the Xen tree. Thanks.</div><br><div class=3D"gmail_quote"><di=
v dir=3D"ltr" class=3D"gmail_attr">Juergen Gross &lt;<a href=3D"mailto:jgro=
ss@suse.com">jgross@suse.com</a>&gt; =E4=BA=8E2021=E5=B9=B49=E6=9C=886=E6=
=97=A5=E5=91=A8=E4=B8=80 =E4=B8=8B=E5=8D=883:58=E5=86=99=E9=81=93=EF=BC=9A<=
br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 06.09.21 09:5=
5, zhaoxiao wrote:<br>
&gt; Since we have the nice helpers pr_err() and pr_warn(), use them instea=
d<br>
&gt; of raw printk().<br>
&gt; <br>
&gt; Signed-off-by: zhaoxiao &lt;<a href=3D"mailto:long870912@gmail.com" ta=
rget=3D"_blank">long870912@gmail.com</a>&gt;<br>
<br>
Any reason you are sending this patch again? It has already been applied<br=
>
through the Xen tree.<br>
<br>
<br>
Juergen<br>
</blockquote></div>

--00000000000024674305cb4f335e--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:23:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:23:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179457.325723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9uv-0001dN-UJ; Mon, 06 Sep 2021 08:23:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179457.325723; Mon, 06 Sep 2021 08:23:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9uv-0001dG-Ql; Mon, 06 Sep 2021 08:23:49 +0000
Received: by outflank-mailman (input) for mailman id 179457;
 Mon, 06 Sep 2021 08:23:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mN9uu-0001dA-OW
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:23:48 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 360c8a42-d7be-4d2a-8c97-c6ef0df8db6d;
 Mon, 06 Sep 2021 08:23:47 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id CEC6720097;
 Mon,  6 Sep 2021 08:23:46 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 99F4C1363C;
 Mon,  6 Sep 2021 08:23:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id bk/6IxLQNWEvbAAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 08:23:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 360c8a42-d7be-4d2a-8c97-c6ef0df8db6d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630916626; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=57S1nNH9F/vO+SKTjE94uCVaSzV8n1fOeTrRwN+v1Wc=;
	b=WgmWKT7pJqmwfIHmJW8QEWohpMCBKo/+GOgmLwAIScLW0Ch5fwHOlR4y19nTTHLslkfbiZ
	QZi1om9eBKiZ4kMWIAjumO40fHfIY7+bmqJKscIaIXeB1Lhl+RE7Idt0A6NtAKm8Y8kSS5
	HikzdsPX62R4jKI5vO2SB4i4kbMMx44=
Subject: Re: Crash when using cpupools
To: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
Date: Mon, 6 Sep 2021 10:23:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="wY65b36q0bSXFLeiVc8vg5hcmacMuivLC"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--wY65b36q0bSXFLeiVc8vg5hcmacMuivLC
Content-Type: multipart/mixed; boundary="enopubJSFoztANg8vadMnHvwG2acC1EI6";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
Message-ID: <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
Subject: Re: Crash when using cpupools
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
In-Reply-To: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>

--enopubJSFoztANg8vadMnHvwG2acC1EI6
Content-Type: multipart/mixed;
 boundary="------------415838FAAE7F29D92DDEEB04"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------415838FAAE7F29D92DDEEB04
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 03.09.21 17:41, Bertrand Marquis wrote:
> Hi,
>=20
> While doing some investigation with cpupools I encountered a crash when=
 trying to isolate a guest to its own physical cpu.
>=20
> I am using current staging status.
>=20
> I did the following (on FVP with 8 cores):
> - start dom0 with dom0_max_vcpus=3D1
> - remove core 1 from dom0 cpupool: xl cpupool-cpu-remove Pool-0 1
> - create a new pool: xl cpupool-create name=3D\"NetPool\=E2=80=9D
> - add core 1 to the pool: xl cpupool-cpu-add NetPool 1
> - create a guest in NetPool using the following in the guest config: po=
ol=3D=E2=80=9CNetPool"
>=20
> I end with a crash with the following call trace during guest creation:=

> (XEN) Xen call trace:
> (XEN)    [<0000000000234cb0>] credit2.c#csched2_alloc_udata+0x58/0xfc (=
PC)
> (XEN)    [<0000000000234c80>] credit2.c#csched2_alloc_udata+0x28/0xfc (=
LR)
> (XEN)    [<0000000000242d38>] sched_move_domain+0x144/0x6c0
> (XEN)    [<000000000022dd18>] cpupool.c#cpupool_move_domain_locked+0x38=
/0x70
> (XEN)    [<000000000022fadc>] cpupool_do_sysctl+0x73c/0x780
> (XEN)    [<000000000022d8e0>] do_sysctl+0x788/0xa58
> (XEN)    [<0000000000273b68>] traps.c#do_trap_hypercall+0x78/0x170
> (XEN)    [<0000000000274f70>] do_trap_guest_sync+0x138/0x618
> (XEN)    [<0000000000260458>] entry.o#guest_sync_slowpath+0xa4/0xd4
>=20
> After some debugging I found out that unit->vcpu_list is NULL after uni=
t->vcpu_list =3D d->vcpu[unit->unit_id]; with unit_id 0 in common/sched/c=
ore.c:688
> This makes the call to is_idle_unit(unit) in csched2_alloc_udata trigge=
r the crash.

So there is no vcpu 0 in that domain? How is this possible?


Juergen

--------------415838FAAE7F29D92DDEEB04
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------415838FAAE7F29D92DDEEB04--

--enopubJSFoztANg8vadMnHvwG2acC1EI6--

--wY65b36q0bSXFLeiVc8vg5hcmacMuivLC
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE10BIFAwAAAAAACgkQsN6d1ii/Ey9w
dgf/SCez1XZCmBzFoS8n1p7BJfT4ftHXsGBrvPYx12GK2n9lXm2wd8tV3piFR1LLhotDQV7eP0Aj
6g/9oZoi/WiqgVYWivxLTGwAmFJNmUGkErZYywVRUg/AfS/vvSPEyMVwFd0ZWDoXXQZLqN6KBWzF
wjULi95YK/P1EhTGFGpLcrDk7Gra2cyOYoHrYfrQ5TK779Sbqr4v2M0B4alTRx8cxNkKIDbL/Lhv
K5OfhCvNB3QE+S9Pku6wXklbAevL/iHq8jCS6K9UUN+jpMwITyNhBHXdHgzJRBHmtVRCG7YeuS/X
t7BiATwoYkKNj/q3tr0eE8cIP7tl1+mx6iP28vfN7g==
=ftUM
-----END PGP SIGNATURE-----

--wY65b36q0bSXFLeiVc8vg5hcmacMuivLC--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:24:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:24:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179463.325734 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9vx-0002CV-8E; Mon, 06 Sep 2021 08:24:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179463.325734; Mon, 06 Sep 2021 08:24:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mN9vx-0002CO-4I; Mon, 06 Sep 2021 08:24:53 +0000
Received: by outflank-mailman (input) for mailman id 179463;
 Mon, 06 Sep 2021 08:24:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kxtw=N4=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mN9vv-0002CG-N8
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:24:51 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.59]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e7a0ee50-0eeb-11ec-b09b-12813bfff9fa;
 Mon, 06 Sep 2021 08:24:51 +0000 (UTC)
Received: from DU2PR04CA0167.eurprd04.prod.outlook.com (2603:10a6:10:2b0::22)
 by DB6PR08MB2663.eurprd08.prod.outlook.com (2603:10a6:6:21::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 08:24:39 +0000
Received: from DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::b5) by DU2PR04CA0167.outlook.office365.com
 (2603:10a6:10:2b0::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Mon, 6 Sep 2021 08:24:39 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT023.mail.protection.outlook.com (10.152.20.68) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 08:24:39 +0000
Received: ("Tessian outbound 5918cb94a4b5:v103");
 Mon, 06 Sep 2021 08:24:39 +0000
Received: from d993e605fcea.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BCFFB8A1-1D20-4CD7-827A-2EFA6490215F.1; 
 Mon, 06 Sep 2021 08:24:32 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d993e605fcea.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 08:24:32 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5707.eurprd08.prod.outlook.com (2603:10a6:102:8b::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 08:24:30 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 08:24:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7a0ee50-0eeb-11ec-b09b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r9quR08hWtYNSsdIKzLQ+LhdPucH7fi6yjeXX0EcObI=;
 b=bGEFN9PQjFtVTa/QYC3EUCsQ9bX1se+s5be+Gz6ls7tM11chJFLa6KHuFkxiEQTtpiZSFDRxrzJkiZr/ZbyYqru/v7Ub1gEyHWJYXYmskX4q2Vy97EnBQVkYTHUUpSw1+VTXEnqYhistEJzXGVzK54BpKmdItPmW1s4RGaXY2VM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 35b6929ec9af1a07
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GHZ1tyoVJznr+V/ahN3pJIP6eaykjvYKgIJFUSdHdekqZ2UcUvFqSs7wf+p4CQ2MvwGUU9gaBXdl1s15QKa/RuXRA7dblIeqGMmBt7cMZ3zfrTqKf5Ahc4RRl03/wQ3s/n4Ve34qQ66k9sc6tlxXSIAEiAVO0DdSRjdGhPfTUbsGH1x1L4AY2TztlFTj73HgqbsVukuQGOTUwy2ZoYCqWKnAgKMl8x0BMMCgoMunJojpPqQKrEEPmWum5Q6kecICw5pV2znNL/gWgNmeSPIEhll+R+aZXrzv/vg2LCB3KkL3mlWOQe8ggoGLj9uKbAQfHAOhTMV5A6c7QklFAJg8Rw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=r9quR08hWtYNSsdIKzLQ+LhdPucH7fi6yjeXX0EcObI=;
 b=kj99V3tP1IAJvjv2e53fRjm6zBLqzsT3zWPWrSJ7uykBlEQlAP2fq5Y0dfJ/zLQb3NUz6MWY88ctWESgX5w8fKfup2XkyaKLosg3ZSaP+FMpFyAXITAFzMmd9VZdN5mlNJQQARMDDFO8f0juHd8JWOdfEx7zpDIe5Ask9CCvW9ZKEJ8abk4EH7a9DEVFEnK/eKqov3HxGmaz45jtc3vpcZKaA1So8oUGd3mTHG8YBy4tfD4gl7KZdOMi5glAEyLzRBQq6u8t3DdpakEBcUI4+Hc9/RNi8VjxpAzBf8iFFMutc1xS4WkxL0BJbphR4X00uhDQX0eh8prrH2hQP4l3Ag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r9quR08hWtYNSsdIKzLQ+LhdPucH7fi6yjeXX0EcObI=;
 b=bGEFN9PQjFtVTa/QYC3EUCsQ9bX1se+s5be+Gz6ls7tM11chJFLa6KHuFkxiEQTtpiZSFDRxrzJkiZr/ZbyYqru/v7Ub1gEyHWJYXYmskX4q2Vy97EnBQVkYTHUUpSw1+VTXEnqYhistEJzXGVzK54BpKmdItPmW1s4RGaXY2VM=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
Thread-Topic: [PATCH v3 4/7] xen/arm: Sanitize cpuinfo ID registers fields
Thread-Index: AQHXmbPuT8O7GIjNskafsumV8a3YcquS+CyAgAPFowA=
Date: Mon, 6 Sep 2021 08:24:29 +0000
Message-ID: <3A692AFD-CFD5-4E90-BEF8-5D29E3A9821B@arm.com>
References: <cover.1629897306.git.bertrand.marquis@arm.com>
 <6710cc0ac3fbc9c8030dfa3d6dfc3ecb79de4b71.1629897306.git.bertrand.marquis@arm.com>
 <alpine.DEB.2.21.2109031425030.26072@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109031425030.26072@sstabellini-ThinkPad-T480s>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: e9779a59-db3f-42c4-33cc-08d9710fc4e7
x-ms-traffictypediagnostic: PR3PR08MB5707:|DB6PR08MB2663:
X-Microsoft-Antispam-PRVS:
	<DB6PR08MB2663BBBF528BDD35DB10CEC19DD29@DB6PR08MB2663.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:556;OLM:556;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 c2vy3/kqsm0SFth62usHfZM9sf7Ck197UGqPe0QpHH0uNXnSTxKBuYvB2sXRc84yOXg7WFBP9RCmQu0GMe9HO79JtUhkbZEWKx4ulXMzos7dU0OuE4FiI2DcT3Rzjfz1IsGDRmw+RAzSfQ11Gp/3h/jAz3jLKEPmw04Wpp92Q5+viTKQJCgPPr31Auhv63nk768gGAihHoX+xzAjLUJIvgj0Z9gFgEraAu3TDNOoda/YExL+SC2OW0qFZqDS/lBDosy446/f/orDQ9Jg01Y0LLQg3WVBL2BiBI+OyLINqA0VbyXPcHtwKXyVBu9HJoMiKaegS5Zp6C0c1unV9gYB+UsWzVS+Zs0ERvBoLnEg69D648Ccy71SXwEOK6IH6/S24ZTu46remKanJzjbI67VJ0J1feRJx3SYxR2ptjvKcCud4XKajEQAfS4dbo6ja7Gkui7Nnkf44LVBgKRSpNBA9tE9HJYq09bVTbJSJJpAP9n4+XbAsru5kWTESJeaW6eirdmAHt8Aa1pp7NQIcGG+m9M7c6/LRPk2JA7TYCZUdwAwLGqjdokWESA1CR6WP6qqm+XPrWge6YxZLpsPZEdcWqlQMkeOu9rKf5MI2cPeDEEWdZWPopOs53/nfCFc8AZva5aI+wgWmTtTHKHwyUBlKPWyEz5b/1QaCb7+eY7hj4R0D3AEmJfqLm0OOVE8+LrrwOsWdZ5aF3DGkL4MBqWzhE0tJzzjwSpoyEjrL4X9qMV5GhtK6Dv/WkgfLCSasEZp
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(33656002)(316002)(6916009)(38100700002)(6486002)(83380400001)(86362001)(2906002)(54906003)(2616005)(8676002)(36756003)(91956017)(76116006)(4326008)(26005)(38070700005)(508600001)(71200400001)(53546011)(8936002)(6506007)(6512007)(66946007)(5660300002)(64756008)(66446008)(66556008)(66476007)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <6C9677CDE8FB0741AA53416A35C173E6@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5707
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1ae03149-6456-44fb-73e4-08d9710fbf2b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9azTzZTES3EjOBhFC0hoMX+C5+9GUarXijMq0D5oBzKku2ZsVcQBjfZEJ61gu1Dy+M0GOyyGDOi4jf0yLFxxpGeNhrEGXp5LPk8r5PvdM1nnBqE9X4iODhFQ/dhH2UdwMOaNsWh+GY3DaBZiXyzKrcQNOmP+8mSL8Rt+MrNlkVmrIwcAgAZEqdPE069nXpp+2ULC88UKaVvLt0kmXteDqhM4L30hZRQGIEOldNX94f2a8WU79icrxjeYPNrjltIUgevvi9DQ/0M6//Eu3RrW8Vuxccf2b6mnhWjOiIJeETxQdMygpcHFAWRuF9Lh6wm3g0W/+ETdsSgZKBdCGIF0SWedKkONjTTFp10s8Mt2gqSd5gysPhyHCnbNJpzpFdFxZvgc/ZtFZWv5PDxDYqp7bltifh5NqERhpe1IXaGIaG2Iotxrh0IIl2j17Q8LT7nNqG9t02ssorqvSY69crsrGuojk5JlSxAUDKWGjW0Bpq2ATkssJCqHMHv33zlWXewC1eqYsu6+qtXYz4sx8X2pJg2JWhwopGsZCJ3q93RzssBX3Hw5fq3H6fTo2w5N0SlCCOmZV8QhZ7Z//JGc3eNpiJdajNs9KRuM1KMKU0mF2bFg8qNhPCGIm5MmISpBCyl641rcIx0vvVyQB88r0nOMXpf+IJkW/D3BWKSJk+4OErZEEN2L+OBfcGhta6Vvc7REoX+7XJvAt50i4x9aThyDZA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(376002)(39860400002)(396003)(346002)(46966006)(36840700001)(6506007)(53546011)(2616005)(6862004)(4326008)(47076005)(316002)(6512007)(83380400001)(82310400003)(5660300002)(36756003)(6486002)(82740400003)(8676002)(54906003)(186003)(8936002)(36860700001)(26005)(356005)(81166007)(70206006)(70586007)(336012)(478600001)(33656002)(2906002)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 08:24:39.1490
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e9779a59-db3f-42c4-33cc-08d9710fc4e7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2663

Hi Stefano,

> On 3 Sep 2021, at 23:48, Stefano Stabellini <sstabellini@kernel.org> wrot=
e:
>=20
> On Wed, 25 Aug 2021, Bertrand Marquis wrote:
>> Define a sanitize_cpu function to be called on secondary cores to
>> sanitize the system cpuinfo structure.
>>=20
>> The safest value is taken when possible and the system is marked tainted
>> if we encounter values which are incompatible with each other.
>>=20
>> Call the update_system_features function on all secondary cores that are
>> kept running and taint the system if different midr are found between
>> cores but hmp-unsafe=3Dtrue was passed on Xen command line.
>>=20
>> This is only supported on arm64 so update_system_features is an empty
>> static inline on arm32.
>>=20
>> The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
>> Xen is running on a system with features differences between cores which
>> are not supported.
>>=20
>> The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
>> as this patch is not handling sanitization of those registers.
>> CTR_EL0/DCZID will be handled in a future patch to properly handle
>> different cache attributes when possible.
>> ZCR should be sanitize once we add support for SVE in Xen.
>>=20
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>=20
> Great patch! It looks good I only have a couple of minor questions
> below.

Thanks
>=20
>=20
>> ---
>> Changes in v3:
>> - in case of different midr but hmp-unsafe passed on the command line,
>> enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
>> - use current core info to sanitize cpu only if we keep it on
>> Changes in v2:
>> - add compilation of cpufeature.c in this patch instead of previous one
>> - remove functions reused from linux code and moved to header
>> - rename sanitize_cpu to update_system_features
>> - change to Linux coding style
>> - remove dev comments
>> - surround currently not used Linux structures with #if 0 and adapt the
>> commit message
>> - add missing aa64dfr1 register
>> - add TODO for CTR, DCZID and ZCR
>> - add CPU_OUT_OF_SPEC support to print_taint
>> - use system_cpuinfo instead of boot_cpu_data
>> ---
>> xen/arch/arm/arm64/Makefile      |   1 +
>> xen/arch/arm/arm64/cpufeature.c  | 121 +++++++++++++++++++++++++++++++
>> xen/arch/arm/smpboot.c           |  34 +++++++--
>> xen/common/kernel.c              |   6 +-
>> xen/include/asm-arm/cpufeature.h |   9 +++
>> xen/include/xen/lib.h            |   1 +
>> 6 files changed, 162 insertions(+), 10 deletions(-)
>>=20
>> diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
>> index 40642ff574..701d66883d 100644
>> --- a/xen/arch/arm/arm64/Makefile
>> +++ b/xen/arch/arm/arm64/Makefile
>> @@ -1,6 +1,7 @@
>> obj-y +=3D lib/
>>=20
>> obj-y +=3D cache.o
>> +obj-y +=3D cpufeature.o
>> obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) +=3D bpi.o
>> obj-$(CONFIG_EARLY_PRINTK) +=3D debug.o
>> obj-y +=3D domctl.o
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufea=
ture.c
>> index 5777e33e5c..61f629ebaa 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[=
] =3D {
>> 	ARM64_FTR_END,
>> };
>>=20
>> +#if 0
>> +/* TODO: use this to sanitize the cache line size among cores */
>> +
>> static const struct arm64_ftr_bits ftr_ctr[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 *=
/
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, =
1, 1),
>> @@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SH=
IFT, 4, 0),
>> 	ARM64_FTR_END,
>> };
>> +#endif
>>=20
>> static const struct arm64_ftr_bits ftr_id_mmfr0[] =3D {
>> 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNER=
SHR_SHIFT, 4, 0xf),
>> @@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] =3D=
 {
>> 	ARM64_FTR_END,
>> };
>>=20
>> +#if 0
>> +/* TODO: handle this when sanitizing cache related registers */
>> static const struct arm64_ftr_bits ftr_dczid[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, =
1),
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT,=
 4, 0),
>> 	ARM64_FTR_END,
>> };
>> +#endif
>>=20
>> static const struct arm64_ftr_bits ftr_id_isar0[] =3D {
>> 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_=
SHIFT, 4, 0),
>> @@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] =
=3D {
>> 	ARM64_FTR_END,
>> };
>>=20
>> +#if 0
>> +/* TODO: use this to sanitize SVE once we support it */
>> +
>> static const struct arm64_ftr_bits ftr_zcr[] =3D {
>> 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> 		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
>> 	ARM64_FTR_END,
>> };
>> +#endif
>>=20
>> /*
>>  * Common ftr bits for a 32bit register with all hidden, strict
>> @@ -502,3 +513,113 @@ static s64 arm64_ftr_safe_value(const struct arm64=
_ftr_bits *ftrp, s64 new,
>>  * End of imported linux structures and code
>>  */
>>=20
>> +static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_nam=
e,
>> +						const struct arm64_ftr_bits *ftrp)
>> +{
>> +	int taint =3D 0;
>> +	u64 old_reg =3D *cur_reg;
>> +
>> +	for (;ftrp->width !=3D 0;ftrp++)
>> +	{
>> +		u64 mask;
>> +		s64 cur_field =3D arm64_ftr_value(ftrp, *cur_reg);
>> +		s64 new_field =3D arm64_ftr_value(ftrp, new_reg);
>> +
>> +		if (cur_field =3D=3D new_field)
>> +			continue;
>> +
>> +		if (ftrp->strict)
>> +			taint =3D 1;
>> +
>> +		mask =3D arm64_ftr_mask(ftrp);
>> +
>> +		*cur_reg &=3D ~mask;
>> +		*cur_reg |=3D (arm64_ftr_safe_value(ftrp, new_field, cur_field)
>> +					<< ftrp->shift) & mask;
>=20
> I wonder why you haven't also imported arm64_ftr_set_value?  This seems
> to be the open-coded version of it.

You are right I could have used it instead (no idea why I did not).

I will modify that and send an update.

Good finding :-)

>=20
>=20
>> +	}
>> +
>> +	if (old_reg !=3D new_reg)
>> +		printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
>> +				reg_name, old_reg, new_reg);
>> +	if (old_reg !=3D *cur_reg)
>> +		printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
>> +				reg_name, old_reg, *cur_reg);
>> +
>> +	if (taint)
>> +	{
>> +		printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
>> +				reg_name);
>> +		add_taint(TAINT_CPU_OUT_OF_SPEC);
>> +	}
>> +}
>> +
>> +
>> +/*
>> + * This function should be called on secondary cores to sanitize the bo=
ot cpu
>> + * cpuinfo.
>> + */
>> +void update_system_features(const struct cpuinfo_arm *new)
>> +{
>> +
>> +#define SANITIZE_REG(field, num, reg)  \
>> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> +				 #reg, ftr_##reg)
>> +
>> +#define SANITIZE_ID_REG(field, num, reg)  \
>> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> +				#reg, ftr_id_##reg)
>> +
>> +#define SANITIZE_RAZ_REG(field, num, reg)  \
>> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> +				#reg, ftr_raz)
>> +
>> +#define SANITIZE_GENERIC_REG(field, num, reg)  \
>> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
>> +				#reg, ftr_generic_32bits)
>> +
>> +	SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
>> +	SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
>> +
>> +	SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
>> +	SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
>> +
>> +	SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
>> +	SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
>> +	SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
>> +
>> +	SANITIZE_ID_REG(isa64, 0, aa64isar0);
>> +	SANITIZE_ID_REG(isa64, 1, aa64isar1);
>> +
>> +	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
>> +
>> +	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> +	{
>> +		SANITIZE_ID_REG(pfr32, 0, pfr0);
>> +		SANITIZE_ID_REG(pfr32, 1, pfr1);
>> +		SANITIZE_ID_REG(pfr32, 2, pfr2);
>> +
>> +		SANITIZE_ID_REG(dbg32, 0, dfr0);
>> +		SANITIZE_ID_REG(dbg32, 1, dfr1);
>> +
>> +		SANITIZE_ID_REG(mm32, 0, mmfr0);
>> +		SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
>> +		SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
>> +		SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
>> +		SANITIZE_ID_REG(mm32, 4, mmfr4);
>> +		SANITIZE_ID_REG(mm32, 5, mmfr5);
>> +
>> +		SANITIZE_ID_REG(isa32, 0, isar0);
>> +		SANITIZE_GENERIC_REG(isa32, 1, isar1);
>> +		SANITIZE_GENERIC_REG(isa32, 2, isar2);
>> +		SANITIZE_GENERIC_REG(isa32, 3, isar3);
>> +		SANITIZE_ID_REG(isa32, 4, isar4);
>> +		SANITIZE_ID_REG(isa32, 5, isar5);
>> +		SANITIZE_ID_REG(isa32, 6, isar6);
>> +
>> +		SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
>> +		SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
>> +#ifndef MVFR2_MAYBE_UNDEFINED
>> +		SANITIZE_REG(mvfr, 2, mvfr2);
>> +#endif
>> +	}
>> +}
>=20
> Looking at the list of registers we are sanitizing here we are only
> missing aux32 and aux64 compared to struct cpuinfo_arm. Is that because
> there is nothing to sanitize there?

Aux registers have no standard bit definition and as such cannot be treated=
.
Linux is doing the same.

Cheers
Bertrand





From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:29:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:29:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179470.325745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNA0N-0002z3-UZ; Mon, 06 Sep 2021 08:29:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179470.325745; Mon, 06 Sep 2021 08:29:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNA0N-0002yw-RV; Mon, 06 Sep 2021 08:29:27 +0000
Received: by outflank-mailman (input) for mailman id 179470;
 Mon, 06 Sep 2021 08:29:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kxtw=N4=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mNA0M-0002yq-P2
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:29:26 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::621])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 29668a3e-8a1e-4067-a903-58ec13f63491;
 Mon, 06 Sep 2021 08:29:24 +0000 (UTC)
Received: from AS9PR06CA0138.eurprd06.prod.outlook.com (2603:10a6:20b:467::24)
 by PAXPR08MB7185.eurprd08.prod.outlook.com (2603:10a6:102:207::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 08:29:19 +0000
Received: from AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:467:cafe::27) by AS9PR06CA0138.outlook.office365.com
 (2603:10a6:20b:467::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Mon, 6 Sep 2021 08:29:19 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT048.mail.protection.outlook.com (10.152.17.177) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 08:29:19 +0000
Received: ("Tessian outbound f11f34576ce3:v103");
 Mon, 06 Sep 2021 08:29:19 +0000
Received: from 7268d7fd9e47.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 08D861F5-D650-4DC4-9101-51D0454602AF.1; 
 Mon, 06 Sep 2021 08:29:13 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7268d7fd9e47.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 08:29:13 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6576.eurprd08.prod.outlook.com (2603:10a6:102:15a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Mon, 6 Sep
 2021 08:29:05 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 08:29:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29668a3e-8a1e-4067-a903-58ec13f63491
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=mIvUYbF7S8gDAZN+6fmeLMclWA1FIIhVDIW9rAoYCzI=;
 b=zmMjejXwXcG1V0B8VjDTjLSMvSG5Do4ByGvVk1eifdsqxZjUfmhMA1aBdraH6vbtHdVEZhhXxhzlofRtcoTSqXYmvdFxiTVwtOaZGOefv5Z6qB5YjPNGwpTv3jIM27CcDz9s2uHyeZxGN00WS2cJH4PBKhwWqPKLR1Wb658p9BI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fd24a59af78e937c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mM9kUiyLvgj1Sg+xs0VuwoWaQE9IgxS1fsHVp5dlG5sXl5e03Ixn1iYuIyqlmXJxA7sK13P2NMoOUv0rDscMnepZcRoIwxl0ul46vxsLBLd/I3kl4ECtJ1L5kBIVu56O3PvGKLkdoQfyQVfmqF2cFwYzxC9xPVonN4JlFFVyMNj3DxNWr35YQZAxfLU1dLqyWxQSt6ej9mIieEY/xjYBEAboOVXJvMOIaJjkBfiW0ItJmL1mIv4pqbOyzMMI7ruU4CkkMnhnlv3lOm/rOsAH4XG7pY7A4PZ+yzV21aMJQzfu/AUqFd+fWvTKlID3gmwIwv8DlmHdJ/xKmzlPyUT/hQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mIvUYbF7S8gDAZN+6fmeLMclWA1FIIhVDIW9rAoYCzI=;
 b=cQEXMJqmWp2d8+DuQAI77ekdflX15VhPCVZmffopdetamqW1AzflX7+oIOLkunBVVAoiRjRdjlKfmIGSXiPgYneHja4NlvEZNtLZnZpOE3K98rMXPpT6lu9SBBWFUJJ7szt3o1CDFg1s20/v78mtGqPcMZvsVyy8+9YIi7B3gZXVx29ktrn22czo46E6rjY1qN5M33pWeETWn5bdwicgD4YTcNIjKxZLu7gdLI53y0ykmoDvhCyFbbI+nPp9VRLGG55wwBKo9JUtz0zARt5mVfwSROD2wvnH2qfdH555Tf8O0APVGCXoA8YF/GvWknxYwkuACwujEwDMG0yZMc213w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=mIvUYbF7S8gDAZN+6fmeLMclWA1FIIhVDIW9rAoYCzI=;
 b=zmMjejXwXcG1V0B8VjDTjLSMvSG5Do4ByGvVk1eifdsqxZjUfmhMA1aBdraH6vbtHdVEZhhXxhzlofRtcoTSqXYmvdFxiTVwtOaZGOefv5Z6qB5YjPNGwpTv3jIM27CcDz9s2uHyeZxGN00WS2cJH4PBKhwWqPKLR1Wb658p9BI=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
Thread-Topic: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if
 needed
Thread-Index:
 AQHXmbPyaqNt71/0B0WMx26sIln5R6uHdm2AgAYrXACAABkoAIAAGG+AgAUlJICAA8aUAA==
Date: Mon, 6 Sep 2021 08:29:04 +0000
Message-ID: <D0CCB87F-1CC1-450B-A612-E4B505B36B5B@arm.com>
References: <cover.1629897306.git.bertrand.marquis@arm.com>
 <a7482653a0c29d68854846f25553f4938a6279b9.1629897306.git.bertrand.marquis@arm.com>
 <14b1639e-c9e4-f8fb-e921-73d7f28681ac@xen.org>
 <98E8FB9D-3372-47D2-92F7-8AD401AC1092@arm.com>
 <6705127a-bde2-36cd-0fed-900eb1eace03@xen.org>
 <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com>
 <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b188e162-6490-42e4-4141-08d971106bf2
x-ms-traffictypediagnostic: PAXPR08MB6576:|PAXPR08MB7185:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB7185B1D78C8D3A2634D028C49DD29@PAXPR08MB7185.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 vS3hw7RjagwvyY0IecFOD5DpcsJs9nAkkBLLaJm60l4V/pfhmGQkWmH7fL5JLixaFSuG+oZDaP5+aNO21zuNNQJcvfMtrbgIsDgS64ZFAQr0aHjDkrXd9dZEh3ndJcnHLRlzDE7N0fiUh00ejayqnuSALwweH6vtdWW/yEffdJQ+kKWGUz2ltsiB/IgF4ZPyaffUTezqEJXPxZWbf6T/QpRt/1RP83aHw8fZw0c53HYzo4qUIJs0YbtMHpmnajBAAlCzfvL5/OmchK4I1EVwPHSkhxJQwHnQxC6wwrTIHUwJHAwtl8XkPLsjYwiIn3tPFMdnQ5r4bBn3QpqQF6VDyTKAlHJvIMqJdWA+OtBcP/UkFnlemSDSKKFclA+P8dEKHUpV86ov9iCbzjxnedxaQsBYCZ3q54ZL+QL/34FUWAcxfxc6mx0UlD/D2E3OsJIs32EApeXg7PLZP3wWtfjJApnZnoWZu1cZ/e5vo1fyNdmNMkQHpT4K/XilC5mQbhXFHgJY8WjsnEbcKWLf+4fPrzINTFMpfMLl5y1+Wxb+sckkf04e0VXNcjgEdd55OlkKuuAqL3l1541OsyS0vX9t2EE4lV7mMVcKFnNky4U0kB5hSNXezj7NZkuEWAEXuCx+55AVsHmEWMOSnSfs650Y8efxuaxs3T7g1kXTZ0rTGJiQ5ZI4Fb54AvyzTILjZbqmJ6x+Si2lQ2SIDVTmG4/21d2GiSv9sXvTzWGAM3iEIenjnIEbQhco16xqXHsljbiW
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(2906002)(86362001)(4326008)(38100700002)(316002)(122000001)(54906003)(26005)(6916009)(2616005)(5660300002)(38070700005)(33656002)(6486002)(478600001)(186003)(53546011)(6506007)(91956017)(66556008)(66476007)(76116006)(66946007)(66446008)(64756008)(8676002)(36756003)(8936002)(83380400001)(6512007)(71200400001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <84D31BB05B88A9479F9D79FD63BC079D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6576
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5c849654-0661-4f27-89d8-08d97110637a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EsKaSiSORlrVwgVHogxGgRANhDMusjrWlT+cKNmXvoAgJTOClfKmFCVYC7wy/B2+oTOWyxleHjYt1U9apUm81Pbz8GMcbjfDpWHZv/v51bX22r1T5yrvrbdImy/ksprQ+QwGXrSnyLi9BbWlTCqK9zE05wExNZgN53IxrRufZKESsgjV68Dt0jNYFDW692HP9VslUlLPgAhapEHcbnWPygDDH7y/bIXd4l5pGk1NpScWJ/X31Cu1pFcUUzWCQkveer4IHzUYRi1fmp6qCU5pzlQTOuBivtlkez7Oy9RhMdurvvu77T1spVIxirNCO6c4P0fKe2KXlwGrKsdxeWWckgwP4w6gpeWl18uuasDyuj2oJnAl2ZIFanBXbt4/P/DnGnozJSvjABloT8fC7oFg+3udtOaQJs/VHfTxVPIaxhOkAroRMgWxLkkA6h5ul7ITtlZH2KA+VONLh64eTN0MKIFGiVoRgusSgrJagRn2LLCR7XSD4TtLOARP6UiLntbQ/Sj7Q1ufeGremmyoJYx/6M8UsrVpXgQQO8lqqhirMc4JvDjzpUVEQDhdrJwncmW6hU6OfwuMIzbiPujeH8p2LTR9p3+0ItlXeecz0whtTUj/pMSZ3xlTpX/CY1DTQXzeuTAWyzBzRZ+KhBAvuBhxpbCPuabvABD4DyQEUq7EwsGhnxlkxfc4urJWuRI/6D8mJAyknLkpfcp7M6iy3fDDvA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6506007)(6862004)(53546011)(33656002)(54906003)(83380400001)(2616005)(36860700001)(336012)(8936002)(81166007)(356005)(186003)(26005)(8676002)(4326008)(316002)(107886003)(5660300002)(36756003)(86362001)(6512007)(47076005)(82310400003)(70586007)(70206006)(508600001)(6486002)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 08:29:19.3320
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b188e162-6490-42e4-4141-08d971106bf2
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7185

SGkgU3RlZmFubywNCg0KPiBPbiAzIFNlcCAyMDIxLCBhdCAyMzo0OSwgU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFR1ZSwgMzEgQXVn
IDIwMjEsIEJlcnRyYW5kIE1hcnF1aXMgd3JvdGU6DQo+PiBIaSBKdWxpZW4sDQo+PiANCj4+PiBP
biAzMSBBdWcgMjAyMSwgYXQgMTU6NDcsIEp1bGllbiBHcmFsbCA8anVsaWVuQHhlbi5vcmc+IHdy
b3RlOg0KPj4+IA0KPj4+IA0KPj4+IA0KPj4+IE9uIDMxLzA4LzIwMjEgMTQ6MTcsIEJlcnRyYW5k
IE1hcnF1aXMgd3JvdGU6DQo+Pj4+IEhpIEp1bGllbiwNCj4+PiANCj4+PiBIaSBCZXJ0cmFuZCwN
Cj4+PiANCj4+Pj4+IE9uIDI3IEF1ZyAyMDIxLCBhdCAxNjowNSwgSnVsaWVuIEdyYWxsIDxqdWxp
ZW5AeGVuLm9yZz4gd3JvdGU6DQo+Pj4+PiANCj4+Pj4+IEhpIEJlcnRyYW5kLA0KPj4+Pj4gDQo+
Pj4+PiBPbiAyNS8wOC8yMDIxIDE0OjE4LCBCZXJ0cmFuZCBNYXJxdWlzIHdyb3RlOg0KPj4+Pj4+
IFNhbml0aXplIENUUl9FTDAgdmFsdWUgYmV0d2VlbiBjb3Jlcy4NCj4+Pj4+PiBJbiBtb3N0IGNh
c2VzIGRpZmZlcmVudCB2YWx1ZXMgd2lsbCB0YWludCBYZW4gYnV0IGlmIGRpZmZlcmVudA0KPj4+
Pj4+IGktY2FjaGUgcG9saWNpZXMgYXJlIGZvdW5kLCB3ZSBjaG9vc2UgdGhlIG9uZSB3aGljaCB3
aWxsIGJlIGNvbXBhdGlibGUNCj4+Pj4+PiBiZXR3ZWVuIGFsbCBjb3JlcyBpbiB0ZXJtcyBvZiBp
bnZhbGlkYXRpb24vZGF0YSBjYWNoZSBmbHVzaGluZyBzdHJhdGVneS4NCj4+Pj4+IA0KPj4+Pj4g
SSB1bmRlcnN0YW5kIHRoYXQgYWxsIHRoZSBDUFVzIGluIFhlbiBuZWVkcyB0byBhZ3JlZSBvbiB0
aGUgY2FjaGUgZmx1c2ggc3RyYXRlZ3kuIEhvd2V2ZXIuLi4NCj4+Pj4+IA0KPj4+Pj4+IEluIHRo
aXMgY2FzZSB3ZSBuZWVkIHRvIGFjdGl2YXRlIHRoZSBUSUQyIGJpdCBpbiBIQ1IgdG8gZW11bGF0
ZSB0aGUNCj4+Pj4+PiBUQ1JfRUwwIHJlZ2lzdGVyIGZvciBndWVzdHMuIFRoaXMgcGF0Y2ggaXMg
bm90IGFjdGl2YXRpbmcgVElEMiBiaXQgYWxsDQo+Pj4+Pj4gdGhlIHRpbWUgdG8gbGltaXQgdGhl
IG92ZXJoZWFkIHdoZW4gcG9zc2libGUuDQo+Pj4+PiANCj4+Pj4+IGFzIHdlIGRpc2N1c3NlZCBp
biBhbiBlYXJsaWVyIHZlcnNpb24sIGEgdkNQVSBpcyB1bmxpa2VseSAoYXQgbGVhc3QgaW4gc2hv
cnQvbWVkaXVtKSB0byBiZSBhYmxlIG1vdmUgYWNyb3NzIHBDUFUgb2YgZGlmZmVyZW50IHR5cGUu
IFNvIHRoZSB2Q1BVIHdvdWxkIGJlIHBpbm5lZCB0byBhIHNldCBvZiBwQ1BVcy4gSU9XLCB0aGUg
Z3Vlc3Qgd291bGQgaGF2ZSB0byBiZSBiaWcuTElUVExFIGF3YXJlIGFuZCB0aGVyZWZvcmUgd291
bGQgYmUgYWJsZSB0byBkbyBpdHMgb3duIHN0cmF0ZWd5IGRlY2lzaW9uLg0KPj4+Pj4gDQo+Pj4+
PiBTbyBJIHRoaW5rIHdlIHNob3VsZCBiZSBhYmxlIHRvIGdldCBhd2F5IGZyb20gdHJhcHBpbmdz
IHRoZSByZWdpc3RlcnMuDQo+Pj4+IEkgZG8gYWdyZWUgdGhhdCB3ZSBzaG91bGQgYmUgYWJsZSB0
byBnZXQgYXdheSBmcm9tIHRoYXQgaW4gdGhlIGxvbmcgdGVybSBvbmNlDQo+Pj4+IHdlIGhhdmUg
Y3B1cG9vbHMgcHJvcGVybHkgc2V0IGJ1dCByaWdodCBub3cgdGhpcyBpcyB0aGUgb25seSB3YXkg
dG8gaGF2ZQ0KPj4+PiBzb21ldGhpbmcgdXNlYWJsZSAoSSB3aWxsIG5vdCBzYXkgcmlnaHQpLg0K
Pj4+PiBJIHdpbGwgd29yayBvbiBmaW5kaW5nIGEgd2F5IHRvIHNldHVwIHByb3Blcmx5IGNwdXBv
b2xzIChvciBzb21ldGhpbmcgZWxzZSBhcw0KPj4+PiB3ZSBkaXNjdXNzZWQgZWFybGllcikgYnV0
IGluIHRoZSBzaG9ydCB0ZXJtIEkgdGhpbmsgdGhpcyBpcyB0aGUgYmVzdCB3ZSBjYW4gZG8uDQo+
Pj4gDQo+Pj4gTXkgY29uY2VybiBpcyB5b3UgYXJlIG1ha2luZyBsb29rIGxpa2UgWGVuIHdpbGwg
YmUgYWJsZSB0byBkZWFsIG5pY2VseSB3aXRoIGJpZy5MSVRUTEUgd2hlbiBpbiBmYWN0IHRoZXJl
IGFyZSBhIGxvdCBtb3JlIHBvdGVudGlhbCBpc3N1ZSBieSBhbGxvdyBhIHZDUFUgbW92aW5nIGFj
Y3Jvc3MgcENQVSBvZiBkaWZmZXJlbnQgdHlwZSAodGhlIGVycmF0YSBpcyBvbmUgZXhhbXBsZSku
DQo+PiANCj4+IEkgYWdyZWUgYW5kIHRoaXMgaXMgd2h5IFhlbiBpcyB0YWludGVkLg0KPj4gDQo+
Pj4gDQo+Pj4+IEFuIG90aGVyIHNvbHV0aW9uIHdvdWxkIGJlIHRvIGRpc2NhcmQgdGhpcyBwYXRj
aCBmcm9tIHRoZSBzZXJpZSBmb3Igbm93IHVudGlsDQo+Pj4+IEkgaGF2ZSB3b3JrZWQgYSBwcm9w
ZXIgc29sdXRpb24gZm9yIHRoaXMgY2FzZS4NCj4+Pj4gU2hvdWxkIHdlIGRpc2NhcmQgb3IgbWVy
Z2Ugb3IgZG8geW91IGhhdmUgYW4gb3RoZXIgaWRlYSA/DQo+Pj4gUGxlYXNlIGNvcnJlY3QgbWUg
aWYgSSBhbSB3cm9uZywgYXQgdGhlIG1vbWVudCwgaXQgZG9lc24ndCBsb29rIGxpa2UgdGhpcyBw
YXRjaCB3aWxsIGJlIHBhcnQgb2YgdGhlIGxvbmdlciBwbGFuLiBJZiBzbywgdGhlbiBJIHRoaW5r
IGl0IHNob3VsZCBiZSBwYXJrZWQgZm9yIG5vdy4NCj4+IA0KPj4gTm90IHN1cmUgaXQgZGVwZW5k
cyBvbiB3aGF0IHRoZSBmaW5hbCBzb2x1dGlvbiB3b3VsZCBiZSBidXQgdGhpcyBpcyBoaWdobHkg
cG9zc2libGUgSSBhZ3JlZS4NCj4+IA0KPj4+IA0KPj4+IFRoaXMgd291bGQgYWxzbyBoYXZlIHRo
ZSBhZHZhbnRhZ2UgdG8gYXZvaWQgc3BlbmRpbmcgdG9vIG11Y2ggdGltZSBvbiByZXNvbHZpbmcg
dGhlIGVtdWxhdGlvbiBpc3N1ZSBJIG1lbnRpb25lZCBpbiBteSBwcmV2aW91cyBhbnN3ZXIuDQo+
Pj4gDQo+Pj4gTm8gbmVlZCB0byByZXNlbmQgYSBuZXcgdmVyc2lvbiBvZiB0aGlzIHNlcmllcyB5
ZXQuIFlvdSBjYW4gd2FpdCB1bnRpbCB0aGUgcmVzdCBvZiB0aGUgc2VyaWVzIGdldCBtb3JlIGZl
ZWRiYWNrLg0KPj4gDQo+PiBPaywgSSB3aWxsIHdhaXQgZm9yIGZlZWRiYWNrIGFuZCBuZXh0IHNl
cmllIHdpbGwgbm90IGluY2x1ZGUgdGhpcyBwYXRjaCBhbnltb3JlLg0KPiANCj4gV291bGQgaXQg
YmUgd29ydGgga2VlcGluZyBqdXN0IHRoZSBwYXJ0IHRoYXQgc2FuaXRpemVzIGN0ciwgd2l0aG91
dCBhbnkNCj4gb2YgdGhlIGVtdWxhdGlvbiBzdHVmZj8gVGhhdCB3YXkgd2UgY291bGQgc3RpbGwg
ZGV0ZWN0IGNhc2VzIHdoZXJlIHRoZXJlDQo+IGlzIGEgbWlzbWF0Y2ggYmV0d2VlbiBDUFVzLCBw
cmludCBhIHVzZWZ1bCBtZXNzYWdlIGFuZCB0YWludCBYZW4uDQoNClRoYXTigJlzIGEgZ29vZCBp
ZGVhLCBpdCBtZWFucyByZW1vdmluZyB0aGUgZW11bGF0aW9uIHBhcnQgYW5kIGp1c3Qga2VlcCB0
aGUgc2FuaXRpemF0aW9uLg0KDQpASnVsaWVuOiB3b3VsZCB5b3UgYmUgb2sgd2l0aCB0aGF0ID8N
Cg0KU2hvdWxkIEkgc2VuZCBhIHY0IG9yIHNob3VsZCB3ZSB1c2UgU3RlZmFub+KAmXMgcGF0Y2gg
ZGlyZWN0bHkgaW5zdGVhZCA/DQoNCkNoZWVycw0KQmVydHJhbmQNCg0KPiANCj4gRm9yIGNsYXJp
dHkgc29tZXRoaW5nIGxpa2UgdGhlIGFwcGVuZGVkLg0KPiANCj4gDQo+IGRpZmYgLS1naXQgYS94
ZW4vYXJjaC9hcm0vYXJtNjQvY3B1ZmVhdHVyZS5jIGIveGVuL2FyY2gvYXJtL2FybTY0L2NwdWZl
YXR1cmUuYw0KPiBpbmRleCBiMTkzNmVmMWQ2Li5kMjQ1NmFmMmJmIDEwMDY0NA0KPiAtLS0gYS94
ZW4vYXJjaC9hcm0vYXJtNjQvY3B1ZmVhdHVyZS5jDQo+ICsrKyBiL3hlbi9hcmNoL2FybS9hcm02
NC9jcHVmZWF0dXJlLmMNCj4gQEAgLTI3NSw5ICsyNzUsNiBAQCBzdGF0aWMgY29uc3Qgc3RydWN0
IGFybTY0X2Z0cl9iaXRzIGZ0cl9pZF9hYTY0bW1mcjJbXSA9IHsNCj4gCUFSTTY0X0ZUUl9FTkQs
DQo+IH07DQo+IA0KPiAtI2lmIDANCj4gLS8qIFRPRE86IHVzZSB0aGlzIHRvIHNhbml0aXplIHRo
ZSBjYWNoZSBsaW5lIHNpemUgYW1vbmcgY29yZXMgKi8NCj4gLQ0KPiBzdGF0aWMgY29uc3Qgc3Ry
dWN0IGFybTY0X2Z0cl9iaXRzIGZ0cl9jdHJbXSA9IHsNCj4gCUFSTTY0X0ZUUl9CSVRTKEZUUl9W
SVNJQkxFLCBGVFJfU1RSSUNULCBGVFJfRVhBQ1QsIDMxLCAxLCAxKSwgLyogUkVTMSAqLw0KPiAJ
QVJNNjRfRlRSX0JJVFMoRlRSX1ZJU0lCTEUsIEZUUl9TVFJJQ1QsIEZUUl9MT1dFUl9TQUZFLCBD
VFJfRElDX1NISUZULCAxLCAxKSwNCj4gQEAgLTI5NCw3ICsyOTEsNiBAQCBzdGF0aWMgY29uc3Qg
c3RydWN0IGFybTY0X2Z0cl9iaXRzIGZ0cl9jdHJbXSA9IHsNCj4gCUFSTTY0X0ZUUl9CSVRTKEZU
Ul9WSVNJQkxFLCBGVFJfU1RSSUNULCBGVFJfTE9XRVJfU0FGRSwgQ1RSX0lNSU5MSU5FX1NISUZU
LCA0LCAwKSwNCj4gCUFSTTY0X0ZUUl9FTkQsDQo+IH07DQo+IC0jZW5kaWYNCj4gDQo+IHN0YXRp
YyBjb25zdCBzdHJ1Y3QgYXJtNjRfZnRyX2JpdHMgZnRyX2lkX21tZnIwW10gPSB7DQo+IAlTX0FS
TTY0X0ZUUl9CSVRTKEZUUl9ISURERU4sIEZUUl9TVFJJQ1QsIEZUUl9MT1dFUl9TQUZFLCBJRF9N
TUZSMF9JTk5FUlNIUl9TSElGVCwgNCwgMHhmKSwNCj4gQEAgLTYwMCw2ICs1OTYsOCBAQCB2b2lk
IHVwZGF0ZV9zeXN0ZW1fZmVhdHVyZXMoY29uc3Qgc3RydWN0IGNwdWluZm9fYXJtICpuZXcpDQo+
IAkgKi8NCj4gCVNBTklUSVpFX1JFRyhkY3ppZCwgMCwgZGN6aWQpOw0KPiANCj4gKwlTQU5JVEla
RV9SRUcoY3RyLCAwLCBjdHIpOw0KPiArDQo+IAlpZiAoIGNwdV9mZWF0dXJlNjRfaGFzX2VsMF8z
Migmc3lzdGVtX2NwdWluZm8pICkNCj4gCXsNCj4gCQlTQU5JVElaRV9JRF9SRUcocGZyMzIsIDAs
IHBmcjApOw0KPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL2NwdWZlYXR1cmUuYyBiL3hlbi9h
cmNoL2FybS9jcHVmZWF0dXJlLmMNCj4gaW5kZXggMTEzZjIwZjYwMS4uNmU1MWY1MzBhOCAxMDA2
NDQNCj4gLS0tIGEveGVuL2FyY2gvYXJtL2NwdWZlYXR1cmUuYw0KPiArKysgYi94ZW4vYXJjaC9h
cm0vY3B1ZmVhdHVyZS5jDQo+IEBAIC0xMjcsNiArMTI3LDggQEAgdm9pZCBpZGVudGlmeV9jcHUo
c3RydWN0IGNwdWluZm9fYXJtICpjKQ0KPiANCj4gICAgIGMtPmRjemlkLmJpdHNbMF0gPSBSRUFE
X1NZU1JFRyhEQ1pJRF9FTDApOw0KPiANCj4gKyAgICBjLT5jdHIuYml0c1swXSA9IFJFQURfU1lT
UkVHKENUUl9FTDApOw0KPiArDQo+ICAgICBhYXJjaDMyX2VsMCA9IGNwdV9mZWF0dXJlNjRfaGFz
X2VsMF8zMihjKTsNCj4gI2VuZGlmDQo+IA0KPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3Nl
dHVwLmMgYi94ZW4vYXJjaC9hcm0vc2V0dXAuYw0KPiBpbmRleCAzNzk4YzVhZGUwLi4zM2I3YmZi
NTljIDEwMDY0NA0KPiAtLS0gYS94ZW4vYXJjaC9hcm0vc2V0dXAuYw0KPiArKysgYi94ZW4vYXJj
aC9hcm0vc2V0dXAuYw0KPiBAQCAtNjI3LDcgKzYyNyw3IEBAIHN0YXRpYyB2b2lkIF9faW5pdCBz
ZXR1cF9tbSh2b2lkKQ0KPiAgICAgICAgIHBhbmljKCJObyBtZW1vcnkgYmFua1xuIik7DQo+IA0K
PiAgICAgLyogV2Ugb25seSBzdXBwb3J0cyBpbnN0cnVjdGlvbiBjYWNoZXMgaW1wbGVtZW50aW5n
IHRoZSBJVklQVCBleHRlbnNpb24uICovDQo+IC0gICAgaWYgKCAoKGN0ciA+PiBDVFJfTDFJcF9T
SElGVCkgJiBDVFJfTDFJcF9NQVNLKSA9PSBDVFJfTDFJcF9BSVZJVlQgKQ0KPiArICAgIGlmICgg
KChjdHIgPj4gQ1RSX0wxSVBfU0hJRlQpICYgQ1RSX0wxSVBfTUFTSykgPT0gSUNBQ0hFX1BPTElD
WV9BSVZJVlQgKQ0KPiAgICAgICAgIHBhbmljKCJBSVZJVlQgaW5zdHJ1Y3Rpb24gY2FjaGUgbm90
IHN1cHBvcnRlZFxuIik7DQo+IA0KPiAgICAgaW5pdF9wZHgoKTsNCj4gZGlmZiAtLWdpdCBhL3hl
bi9pbmNsdWRlL2FzbS1hcm0vY3B1ZmVhdHVyZS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9jcHVm
ZWF0dXJlLmgNCj4gaW5kZXggNTIxOWZkM2JhYi4uY2E2ZTgyN2ZjYiAxMDA2NDQNCj4gLS0tIGEv
eGVuL2luY2x1ZGUvYXNtLWFybS9jcHVmZWF0dXJlLmgNCj4gKysrIGIveGVuL2luY2x1ZGUvYXNt
LWFybS9jcHVmZWF0dXJlLmgNCj4gQEAgLTI2Nyw2ICsyNjcsMTQgQEAgc3RydWN0IGNwdWluZm9f
YXJtIHsNCj4gICAgICAgICByZWdpc3Rlcl90IGJpdHNbMV07DQo+ICAgICB9IGRjemlkOw0KPiAN
Cj4gKyAgICAvKg0KPiArICAgICAqIENUUiBpcyBvbmx5IHVzZWQgdG8gY2hlY2sgZm9yIGRpZmZl
cmVudCBjYWNoZSB0eXBlcyBvciBwb2xpY2llcyBhbmQNCj4gKyAgICAgKiB0YWludCBYZW4gaW4g
dGhpcyBjYXNlDQo+ICsgICAgICovDQo+ICsgICAgc3RydWN0IHsNCj4gKyAgICAgICAgcmVnaXN0
ZXJfdCBiaXRzWzFdOw0KPiArICAgIH0gY3RyOw0KPiArDQo+ICNlbmRpZg0KPiANCj4gICAgIC8q
DQo+IEBAIC0zMzksNiArMzQ3LDkgQEAgZXh0ZXJuIHN0cnVjdCBjcHVpbmZvX2FybSBzeXN0ZW1f
Y3B1aW5mbzsNCj4gZXh0ZXJuIHZvaWQgaWRlbnRpZnlfY3B1KHN0cnVjdCBjcHVpbmZvX2FybSAq
KTsNCj4gDQo+ICNpZmRlZiBDT05GSUdfQVJNXzY0DQo+ICsNCj4gK2V4dGVybiBib29sIG1pc21h
dGNoZWRfY2FjaGVfdHlwZTsNCj4gKw0KPiBleHRlcm4gdm9pZCB1cGRhdGVfc3lzdGVtX2ZlYXR1
cmVzKGNvbnN0IHN0cnVjdCBjcHVpbmZvX2FybSAqKTsNCj4gI2Vsc2UNCj4gc3RhdGljIGlubGlu
ZSB2b2lkIHVwZGF0ZV9zeXN0ZW1fZmVhdHVyZXMoY29uc3Qgc3RydWN0IGNwdWluZm9fYXJtICpj
cHVpbmZvKQ0KPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9wcm9jZXNzb3IuaCBi
L3hlbi9pbmNsdWRlL2FzbS1hcm0vcHJvY2Vzc29yLmgNCj4gaW5kZXggMjU3N2U5YTI0NC4uOGM5
ODQzZTEyYiAxMDA2NDQNCj4gLS0tIGEveGVuL2luY2x1ZGUvYXNtLWFybS9wcm9jZXNzb3IuaA0K
PiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL3Byb2Nlc3Nvci5oDQo+IEBAIC03LDkgKzcsMjEg
QEANCj4gI2luY2x1ZGUgPHB1YmxpYy9hcmNoLWFybS5oPg0KPiANCj4gLyogQ1RSIENhY2hlIFR5
cGUgUmVnaXN0ZXIgKi8NCj4gLSNkZWZpbmUgQ1RSX0wxSXBfTUFTSyAgICAgICAweDMNCj4gLSNk
ZWZpbmUgQ1RSX0wxSXBfU0hJRlQgICAgICAxNA0KPiAtI2RlZmluZSBDVFJfTDFJcF9BSVZJVlQg
ICAgIDB4MQ0KPiArI2RlZmluZSBDVFJfTDFJUF9NQVNLICAgICAgIDB4Mw0KPiArI2RlZmluZSBD
VFJfTDFJUF9TSElGVCAgICAgIDE0DQo+ICsjZGVmaW5lIENUUl9ETUlOTElORV9TSElGVCAgMTYN
Cj4gKyNkZWZpbmUgQ1RSX0lNSU5MSU5FX1NISUZUICAwDQo+ICsjZGVmaW5lIENUUl9JTUlOTElO
RV9NQVNLICAgMHhmDQo+ICsjZGVmaW5lIENUUl9FUkdfU0hJRlQgICAgICAgMjANCj4gKyNkZWZp
bmUgQ1RSX0NXR19TSElGVCAgICAgICAyNA0KPiArI2RlZmluZSBDVFJfQ1dHX01BU0sgICAgICAg
IDE1DQo+ICsjZGVmaW5lIENUUl9JRENfU0hJRlQgICAgICAgMjgNCj4gKyNkZWZpbmUgQ1RSX0RJ
Q19TSElGVCAgICAgICAyOQ0KPiArDQo+ICsjZGVmaW5lIElDQUNIRV9QT0xJQ1lfVlBJUFQgIDAN
Cj4gKyNkZWZpbmUgSUNBQ0hFX1BPTElDWV9BSVZJVlQgMQ0KPiArI2RlZmluZSBJQ0FDSEVfUE9M
SUNZX1ZJUFQgICAyDQo+ICsjZGVmaW5lIElDQUNIRV9QT0xJQ1lfUElQVCAgIDMNCj4gDQo+IC8q
IE1JRFIgTWFpbiBJRCBSZWdpc3RlciAqLw0KPiAjZGVmaW5lIE1JRFJfUkVWSVNJT05fTUFTSyAg
ICAgIDB4Zg0KDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:31:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:31:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179476.325756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNA29-0004U6-CP; Mon, 06 Sep 2021 08:31:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179476.325756; Mon, 06 Sep 2021 08:31:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNA29-0004Tz-88; Mon, 06 Sep 2021 08:31:17 +0000
Received: by outflank-mailman (input) for mailman id 179476;
 Mon, 06 Sep 2021 08:31:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kxtw=N4=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mNA27-0004Tt-3W
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:31:15 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe07::62b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d188a28c-ef78-4998-8dea-688d24e0d721;
 Mon, 06 Sep 2021 08:31:13 +0000 (UTC)
Received: from DB9PR05CA0023.eurprd05.prod.outlook.com (2603:10a6:10:1da::28)
 by DB8PR08MB4955.eurprd08.prod.outlook.com (2603:10a6:10:38::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Mon, 6 Sep
 2021 08:31:07 +0000
Received: from DB5EUR03FT010.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:1da:cafe::9e) by DB9PR05CA0023.outlook.office365.com
 (2603:10a6:10:1da::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Mon, 6 Sep 2021 08:31:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT010.mail.protection.outlook.com (10.152.20.96) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 08:31:07 +0000
Received: ("Tessian outbound 1a0c40aa17d8:v103");
 Mon, 06 Sep 2021 08:31:07 +0000
Received: from 2ae092774d4a.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B08C1D8F-5A14-4DF5-8735-7D1A131F7C4D.1; 
 Mon, 06 Sep 2021 08:30:59 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2ae092774d4a.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 08:30:59 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5788.eurprd08.prod.outlook.com (2603:10a6:102:8b::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 08:30:57 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 08:30:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d188a28c-ef78-4998-8dea-688d24e0d721
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9aXBdmofQAsnNEK7ex5fwtBRjD0v15b+Jrmhf2x2xRg=;
 b=zSXe527rxXA/jU5csRzONoV3jFb+ms7GOEmXE4UGkGhAiOvRJfOHWet8JEihUl01agm55hJJ2a5b1TWzdjOIGttJXPIwaQgwDpwhO3xAVFo/9TCFksuZ6x+EXccRIK+OS79jD224VhG1N4JEidDWWgbU9hBYIajwDFhngs+rtL4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 55ccc14e93903f37
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bi9vdWf6Ov5vRQAnSVzECFRi4N7Z9i2F5SS47UpfJCmS/LZ6Z3oz1kN/k4PDlpPzmkVEN0ry2cnRCGN5CZGidk0Otj3zpZtYb9zDn8i3uwMHcGwFg5zR4Jg8GZjzTSwsle98SNKumfHf/gR13ImI+NAn2HoL2DVAGzzNTvTLROgDv42xE07YGglSmUuL/5XfHw0oUWGa8SlUAQsQxZxkrtdlV/KxvO7PkbY6jSB3NSh9wNaY1Fis9mMnpFvuvHmWo/KOPWBTjmCEGj4qZoPjrVHiDnkMdILXnBU5K97uJnDTQ6PQgcgwDWMkb7Ue2yHufC+HaWE8GztMuPP/KiyJrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9aXBdmofQAsnNEK7ex5fwtBRjD0v15b+Jrmhf2x2xRg=;
 b=Jrl3DPwPhNUFpzLuD9/TynUeyHLAJVkbo4yCMVVFpBWMiY2n2NhfUo2FOk1/Syr49CrUuSyLGWlNcecu2NDdq1kNH3a4QodoCwvejupt3dQjDKSUmycHz5tRsZAxe+QfR+FMudne0H1oWoo+skt7LGdx/Ux5vcFLArmHcSFh/tR+fpMhY4Q+HXMMD4a+6H5soOOL+cJwAa3Zb1KhMICizw1oRFO/SQ5GrfropCK0BpXwW3QkmTb8L/dVvYYBRp0E8vxFGsAlicb5hb+5YD3rsPB2wa834ZY0DWpH2Z/81T1wTpo41GRWDYmLNZTH9mic8Mq7mfgt3v5vM4F88Fc1OQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9aXBdmofQAsnNEK7ex5fwtBRjD0v15b+Jrmhf2x2xRg=;
 b=zSXe527rxXA/jU5csRzONoV3jFb+ms7GOEmXE4UGkGhAiOvRJfOHWet8JEihUl01agm55hJJ2a5b1TWzdjOIGttJXPIwaQgwDpwhO3xAVFo/9TCFksuZ6x+EXccRIK+OS79jD224VhG1N4JEidDWWgbU9hBYIajwDFhngs+rtL4=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Juergen Gross <jgross@suse.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: Re: Crash when using cpupools
Thread-Topic: Crash when using cpupools
Thread-Index: AQHXoNo0twqKn1RLAkCotEsfXja8k6uWr0+AgAACAYA=
Date: Mon, 6 Sep 2021 08:30:56 +0000
Message-ID: <626655D0-D7C5-457D-A99E-EF5CEDFD188D@arm.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
In-Reply-To: <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 996d2d14-9b48-4b1a-2c2b-08d97110ac47
x-ms-traffictypediagnostic: PR3PR08MB5788:|DB8PR08MB4955:
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB49554A1BE605964897960EE39DD29@DB8PR08MB4955.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 N8woIjHFwJGhpXrfpGLmo9p40k/Okp0NOcelnIhi1S2atsc+3L0fw1Xk+7UxbVtp4tYOQFfJrCKFzZPMPyN6MweW1ad+bT9LOZOm+4ECPt3PwUGD8QJ6WHzcNmpfHwy+UT7ADUmbep984RnDjStLHX2TlwCmlRNY7h2YY87rNQh5OqC+UkP+KR+Ao8RGjfMan7AqXFRQwpryRk5WA2WJtutyJupYJba6OTiLz5rLkUMXiP8y2yZ7dgN2R5pfmkQUbKUoeL2xkJaW25a99kKE3isLVxc8EMxqNmmWGArM1GOo8+W3mpso2/fAxSuenLDbqN5vZtv8panOwLnOTJMee43jV/Z3nr26pCqGf+gvNYjk8Ong5HdNsN9tx5gGjAhsDKBbyQnxxEdNupUGPKwKjYj59inq1e8vyDVGAMBiYOIS0ThYUekdSAzD1jipFb0PFQe1zCFKgVAFTQHB8/J/d0TVNdwNNEqRt2VUTVD2N4/lJc6V3el8uRAM9D1GPub8mjvTFE+newU/KUvF6WXBcwzB+tdbfHlMFJbTaJR+XdHGhyg6SxFyRyJdpe84HPnNpo2U1NAdha43oaKSq5hYrcf0y8bje6w8W1Jxl8eiYNHHeQlNB5ByXU14LaCQrU/yF5AeHVprspLWLLLkSMoem3NszZxu+Qbf4FB3mrncZo1O2XM4RWTgxBsevZmP1F+LZganvlwEhBiUZgKfopF5UR00mDOo8y7uJXcRjiF+Hu6F3HZvusa0YgjVHWhntu0M
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(366004)(39860400002)(396003)(376002)(71200400001)(6916009)(186003)(4326008)(33656002)(26005)(86362001)(3480700007)(478600001)(36756003)(6512007)(6486002)(2616005)(316002)(5660300002)(54906003)(122000001)(38070700005)(2906002)(83380400001)(8676002)(8936002)(38100700002)(66556008)(66476007)(6506007)(66446008)(64756008)(66946007)(53546011)(91956017)(76116006)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?NjZkd2UvMW04ZUFEOTE2cHFLREtJTWpVTkxpTnVPMDVUMEwzZHFuSzliMFdn?=
 =?utf-8?B?U1BGVWlFR28vMXMrMXBLUkFaeFpHaDRrOFR0U3Y0VldJQlRmYUNmZDZ0M1Ix?=
 =?utf-8?B?bXUrK3ZQVTB4bnpDbEh4RTBybVVnQklFTzVnZ3VDZ2poaE1IQTZjemdUbnR3?=
 =?utf-8?B?T2xUbnJ6VmFBQ3ZwMGtod004cjdraFcyeEpFZGs5eWJxOStmalpzcHo0eVl6?=
 =?utf-8?B?SnNKcXNLOUVoUWc2Uzl5b2NFaS96TlVpSFg5dUNWTmNHbWRNbkRIL3o5dW1j?=
 =?utf-8?B?S1o3Qk9QVWtUNTNvOUZ1dEpCOTl4RWh6OVRHd3FRWDh3Z1Z0SFZqTjEwZENn?=
 =?utf-8?B?SEd6cXB6b28vTU1PeUI4MVVvV2JCZnRVaTBxZGhUaEI4MkprbFZLQzYyNnFl?=
 =?utf-8?B?MGVlRTM5Q3BaM3hJMDJqS2RsRWRwM1lFbGM5YnpIclFtYUVaZzRvQytiZ0la?=
 =?utf-8?B?RjE5M3hSZmxVc29LRHltek42WWNraFBBbUE2c0cwWW9BdFBLRVFMdnZncHFi?=
 =?utf-8?B?WEtVZGlXZUpLWkxPRmppY0UrRHRQbzd2cHptYlZWeUQ2MjBRQnZ1dktobmJB?=
 =?utf-8?B?ODFCNUdSY0ZJYXFUeXN4KzBvMTE5UUNUYWVMcVVCcU16WTBkN2RidXVjSlQ2?=
 =?utf-8?B?eVdkbTlhUUVnVGJzMHptNUNlbkR5dW9uTEVTL3Z3Z2V1V0IxYjJsUWx0TS9p?=
 =?utf-8?B?ODI0L1pSM0lrVzl3a2tKMUkya20xNDVtZTJYcWZYOUVlNzVyOTBZYkJoRWJt?=
 =?utf-8?B?MGN2THBTUVhqNDR2akc2SWxaOXprVnRXQjA1bGJrbWI2Vll1VktTY2VLelNE?=
 =?utf-8?B?V1dJMU9aMk9DM2dmRWcyOVFPd3JFdGVrNld5RTRKZ0J2WGNSYlRQK1RoRVNT?=
 =?utf-8?B?MlMrK3RmckZxaU0zQTMxUzF4dTg1bkpTZnlQdTQ3T2NPQzloaytIa29uT3Fz?=
 =?utf-8?B?ZThid1RxUVNaTnppSUNabmJUUi9SSUFWWDI4SVdVOUxZbk1QVndIWmVIQnB6?=
 =?utf-8?B?Z3RJN2RqWGk2aUNDTmNUQTcxbFJoVHkrQzc5cUZQN3JVcVFDNHhkS2l3WHBj?=
 =?utf-8?B?dit4ZG1Ka3lxZTQraGFyeWpxSUd3YWxtTHYrYmtWMm5LRm1yNFZZN01Ob2RI?=
 =?utf-8?B?N0x0WDJmeWV0MDRxeWUwelhuY1EvalFzUnpiWTdvaWtoeFl6US9DQlpZQ3h1?=
 =?utf-8?B?K2ZYOVZzUEZqd292cTIxZTA2RWJiZ1lUeHNCanhsM0ZnU0FGandML0MvemFC?=
 =?utf-8?B?OGFPZlVjNjBSTTY3b1BaK0MwUjY4Zkh0RGNySHlHOG5EQUh1eko2NTA2d3Rm?=
 =?utf-8?B?TDBGRUNmbDkzU2ZoZHJ6Zy9Jd2U3WURoeVQ4UXU0dng4emVmQmxZa0VCODEz?=
 =?utf-8?B?WnFLVTRJbW5rQnFLV3NmcDBWMkZIOXV6VGlPYXJpNGE4Z3V6ZVRnakpvRU5O?=
 =?utf-8?B?SWI3MDFEeGY4U0pJQmI4NkhSOUlsb2dDWkNMZFZuemVNMjhGTVJuKzYrM1FN?=
 =?utf-8?B?S3VSUVFmVE1maE9MeTdVUGRKY2xHR1hvWDJvZzZYR2FrYldDa21uMDhGeXdI?=
 =?utf-8?B?MmVMdk92bk1KdGFnSndydnNKcGpwQVhxL3dqUWxYNm9BQjlUbGpTWkdHSHZU?=
 =?utf-8?B?ZEt1Y1FkVHBWdnord0lPenBuaXpMK0U0c0hwRnRpd3ZRVzZnQ2JyTndpODRF?=
 =?utf-8?B?d3BscEpHRUhqZ1B4cWVFMW9SUW5tRGkwNWpJVld2QXBuUjlEaXBsNmpvWnA5?=
 =?utf-8?Q?lkoLZ0LAJwJ+LwXb2Olp5aRnNUnCL4UbFqc2AUt?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <FDCDDF354BC77945A912FA8F623F67D8@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5788
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f67df28d-40af-495d-791f-08d97110a61a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9K++IhcLfoof8o5dj2M2RwbKrPwZc+98zGvANYHwyIYUKlBBR3Fa2QON5gW/+s49ACthN8KLHPGiWyxy15J8b7s3TmguHXSci05aRAKULNHPIRVWh5wqvJYVn/2SQoUp8xTxD6y3r92iEaiUEAjwpnVPRExShdPgJMTbpdHZEIuMvVQhCGJnekwn+DfAQBmCD8fDVHgkwOUAHdC6e4bio72EBNDnLfinbmY4b0k+Ig4OLjCdxBMTeBCDZom0+gnoPZHXYWYGEPhHqEYJ512RQccS4vvyPh2YhQqeg9HOxzc3Msd36hbQ5gWFQJMfHQkiOubrEl0ptRm/7oGv8VNt5ZW78D/zclB5uR1wIPeZUbtwaG7BUme2DXq4EVPaAumd9YW4YwUfNP/ZmAuF3AplCW4D7oqfDhb471DVabd2e8qFeE3LOePUBDRXTeJmsXB8+pCLlc4Msivk8uWwreS+eyEujRZONWUcjlFtKrjVqPSUg4uPSyESY6vILV2VsoePUAlVcxNj0cld/0jQ89tkEseih/A4c4GjV0LLydi1QHemxCqoWQoSJtzA+tJm9X7OCRAtamscgb35wZqCsPvgpJPNjkr312kfLMvlm5qBmeLPSI0RDWHi+GYo4TCOq860xm3RmBmE77Y72HKOJidyWZ0uhDa4FluATPDqaEh/oT3zhjpEsaeYHbXhTl7E0lvtJqj64Nq4c9HCrwCQx0dDUQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(136003)(39860400002)(46966006)(36840700001)(26005)(478600001)(70206006)(86362001)(356005)(6506007)(6862004)(316002)(53546011)(54906003)(82310400003)(36756003)(81166007)(82740400003)(47076005)(107886003)(3480700007)(2616005)(83380400001)(2906002)(6512007)(36860700001)(6486002)(8936002)(8676002)(5660300002)(186003)(33656002)(336012)(70586007)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 08:31:07.3206
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 996d2d14-9b48-4b1a-2c2b-08d97110ac47
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4955

SGkgSnVlcmdlbiwNCg0KPiBPbiA2IFNlcCAyMDIxLCBhdCAwOToyMywgSnVlcmdlbiBHcm9zcyA8
amdyb3NzQHN1c2UuY29tPiB3cm90ZToNCj4gDQo+IE9uIDAzLjA5LjIxIDE3OjQxLCBCZXJ0cmFu
ZCBNYXJxdWlzIHdyb3RlOg0KPj4gSGksDQo+PiBXaGlsZSBkb2luZyBzb21lIGludmVzdGlnYXRp
b24gd2l0aCBjcHVwb29scyBJIGVuY291bnRlcmVkIGEgY3Jhc2ggd2hlbiB0cnlpbmcgdG8gaXNv
bGF0ZSBhIGd1ZXN0IHRvIGl0cyBvd24gcGh5c2ljYWwgY3B1Lg0KPj4gSSBhbSB1c2luZyBjdXJy
ZW50IHN0YWdpbmcgc3RhdHVzLg0KPj4gSSBkaWQgdGhlIGZvbGxvd2luZyAob24gRlZQIHdpdGgg
OCBjb3Jlcyk6DQo+PiAtIHN0YXJ0IGRvbTAgd2l0aCBkb20wX21heF92Y3B1cz0xDQo+PiAtIHJl
bW92ZSBjb3JlIDEgZnJvbSBkb20wIGNwdXBvb2w6IHhsIGNwdXBvb2wtY3B1LXJlbW92ZSBQb29s
LTAgMQ0KPj4gLSBjcmVhdGUgYSBuZXcgcG9vbDogeGwgY3B1cG9vbC1jcmVhdGUgbmFtZT1cIk5l
dFBvb2xc4oCdDQo+PiAtIGFkZCBjb3JlIDEgdG8gdGhlIHBvb2w6IHhsIGNwdXBvb2wtY3B1LWFk
ZCBOZXRQb29sIDENCj4+IC0gY3JlYXRlIGEgZ3Vlc3QgaW4gTmV0UG9vbCB1c2luZyB0aGUgZm9s
bG93aW5nIGluIHRoZSBndWVzdCBjb25maWc6IHBvb2w94oCcTmV0UG9vbCINCj4+IEkgZW5kIHdp
dGggYSBjcmFzaCB3aXRoIHRoZSBmb2xsb3dpbmcgY2FsbCB0cmFjZSBkdXJpbmcgZ3Vlc3QgY3Jl
YXRpb246DQo+PiAoWEVOKSBYZW4gY2FsbCB0cmFjZToNCj4+IChYRU4pICAgIFs8MDAwMDAwMDAw
MDIzNGNiMD5dIGNyZWRpdDIuYyNjc2NoZWQyX2FsbG9jX3VkYXRhKzB4NTgvMHhmYyAoUEMpDQo+
PiAoWEVOKSAgICBbPDAwMDAwMDAwMDAyMzRjODA+XSBjcmVkaXQyLmMjY3NjaGVkMl9hbGxvY191
ZGF0YSsweDI4LzB4ZmMgKExSKQ0KPj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjQyZDM4Pl0gc2No
ZWRfbW92ZV9kb21haW4rMHgxNDQvMHg2YzANCj4+IChYRU4pICAgIFs8MDAwMDAwMDAwMDIyZGQx
OD5dIGNwdXBvb2wuYyNjcHVwb29sX21vdmVfZG9tYWluX2xvY2tlZCsweDM4LzB4NzANCj4+IChY
RU4pICAgIFs8MDAwMDAwMDAwMDIyZmFkYz5dIGNwdXBvb2xfZG9fc3lzY3RsKzB4NzNjLzB4Nzgw
DQo+PiAoWEVOKSAgICBbPDAwMDAwMDAwMDAyMmQ4ZTA+XSBkb19zeXNjdGwrMHg3ODgvMHhhNTgN
Cj4+IChYRU4pICAgIFs8MDAwMDAwMDAwMDI3M2I2OD5dIHRyYXBzLmMjZG9fdHJhcF9oeXBlcmNh
bGwrMHg3OC8weDE3MA0KPj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjc0ZjcwPl0gZG9fdHJhcF9n
dWVzdF9zeW5jKzB4MTM4LzB4NjE4DQo+PiAoWEVOKSAgICBbPDAwMDAwMDAwMDAyNjA0NTg+XSBl
bnRyeS5vI2d1ZXN0X3N5bmNfc2xvd3BhdGgrMHhhNC8weGQ0DQo+PiBBZnRlciBzb21lIGRlYnVn
Z2luZyBJIGZvdW5kIG91dCB0aGF0IHVuaXQtPnZjcHVfbGlzdCBpcyBOVUxMIGFmdGVyIHVuaXQt
PnZjcHVfbGlzdCA9IGQtPnZjcHVbdW5pdC0+dW5pdF9pZF07IHdpdGggdW5pdF9pZCAwIGluIGNv
bW1vbi9zY2hlZC9jb3JlLmM6Njg4DQo+PiBUaGlzIG1ha2VzIHRoZSBjYWxsIHRvIGlzX2lkbGVf
dW5pdCh1bml0KSBpbiBjc2NoZWQyX2FsbG9jX3VkYXRhIHRyaWdnZXIgdGhlIGNyYXNoLg0KPiAN
Cj4gU28gdGhlcmUgaXMgbm8gdmNwdSAwIGluIHRoYXQgZG9tYWluPyBIb3cgaXMgdGhpcyBwb3Nz
aWJsZT8NCg0KTm8gaWRlYSwgSSB3aWxsIG5lZWQgdG8gZGlnIGRlZXBlciBhcyB0aGUgc3RhdGUg
SSBjb21lIHRvIGRvZXMgbm90IG1ha2Ugc2Vuc2UuDQoNCkNvdWxkIHlvdSBqdXN0IGNvbmZpcm0g
dGhhdCBteSBvcGVyYXRpb25zIGFyZSByaWdodCBhbmQgdGhpcyBzaG91bGQgd29yayBiZWZvcmUg
SSBzdGFydCBkaWdnaW5nID8NCg0KQ2hlZXJzDQpCZXJ0cmFuZA0KDQo+IA0KPiANCj4gSnVlcmdl
bg0KPiA8T3BlblBHUF8weEIwREU5REQ2MjhCRjEzMkYuYXNjPg0KDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:46:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:46:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179491.325767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAGq-0006HH-Ur; Mon, 06 Sep 2021 08:46:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179491.325767; Mon, 06 Sep 2021 08:46:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAGq-0006HA-Pf; Mon, 06 Sep 2021 08:46:28 +0000
Received: by outflank-mailman (input) for mailman id 179491;
 Mon, 06 Sep 2021 08:46:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNAGp-0006H4-U2
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:46:28 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ead0f9aa-0eee-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 08:46:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ead0f9aa-0eee-11ec-b09c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630917986;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=YJpL75vOv1uu/c6yyOoRd6o8Efhvw6/U6AAGNgCeSJ0=;
  b=F/KM+AoXPwZwIC9G/kHJvBQHQX0VfK2KRTstCdtcs5hA4FYoKcujqTVJ
   IuLI0HebM9DG4GcChA+kzP8rkEatO/F5cRpaccgigxWyB+esElLq9kEkF
   1XbkcWwgLNduaCsPCbecYEyshsQM9YtaJfwABVNPals4HUQYGrl5eU4Fu
   0=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 0z+8TH3Pv/VgxCy7RjjW41oHdfRa/etgPAXFg1ex0PctC0AC6QIvnDhavS2YfEVAxw0HUrC2w7
 3P6VQDowWhJJMAzL6di+qjBS4vjhFAtkZeysFbAuM0ma50jDOhQfqkCqESqxNkjOEYxKygP1hK
 1WLR3IrOiH2bw6GosjdmnSXpoz+/Vn2Tg6vEueVPdmBDHdOMVNdHKpUh+XkijYELUhwPKAxPKl
 5aQFVv4fCIO5nc7+67s1rMEUBctBkfFEl9SOsn2Hcuz/qwgPwj6GXlI9kTvg7OkkvhCsfx7yGi
 B1NLcweEKgHK1UtSxrx17igz
X-SBRS: 5.1
X-MesageID: 51680239
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:ByibLqjJ+5mQBR2jh3RrPltxQXBQX0h13DAbv31ZSRFFG/FwyP
 rBoB1L73DJYWgqNE3I+erhBEGBKUmskaKdkrNhQ4tKOzOWx1dATbsSkbcKpgeAJ8SQzJ8n6U
 4NSdkZNDS0NykGsS+Y2njKLz9D+qj+zEnAv463pB0BPGIaCdAT0+46MHf9LqQffng3OXNTLu
 vk2iMonUvERZ1aVLXAOpFTNNKz1uEj2aiWLSIuNloC0k2jnDmo4Ln1H1yx2QofaSpGxfMH/X
 LemwL0y62/u7XjoyWsmVP73tBzop/M29FDDMuDhow8LSjtsB+hYMBEV6eZtD44jemz4BIBkc
 XKoT0nI8NvgkmhMV2dkF/I4U3NwTwu43jtxRuxhmbim9XwQHYAB89IletiA1DkwntlmOs5/L
 NA3mqfuZYSJwjHhj7B69/BUAwvvlaooFI5+NRjzEB3YM87Uvt8vIYf9ERaHNMrByTh8r0qF+
 FoEYX1+OtWS1WHdHrU11MfgOBEZk5DWytuf3Jy/vB8i1Nt7TdEJgojtY0id047hdAAo8Iu3Z
 WDDkxq/Is+BvP+I5gNXdvpevHHflAldyi8eV56EW6XYZ3vBEi93KIfwI9Frt1CK6Z4gafbpv
 z6ISVlXCgJChrTNfE=
X-IronPort-AV: E=Sophos;i="5.85,271,1624334400"; 
   d="scan'208";a="51680239"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Al5qySRNiilD8T4leYlpt+sy8m8JPRx5uwqCjR2AJ57DH206eyitVZ/yiGF3L1woOVwJMZ9s3GGOsFucXzKWdaU6qIO0jU7vrbvlgpgOndDZvDAVXbpuc09JKLdeOF/UE1XrXQIselVanaUp9lDw50tk5bVt8jR03DTza3tuLTOj1iLB7b5YRiTCocKm18blCQjpoCXewVUdrRU64ZIBnG0nQFeMdDQLahqP9JnFS+xOQEaNKcm/aKypTJ5771s7NZDvGgkwIhPbq6/V8+unoj82iJVbTdfSnAbpkXzxwJjsauDndECmqjdLkRa0zQQO1/Cslfzlm+Sg2Fk+gcrDmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YJpL75vOv1uu/c6yyOoRd6o8Efhvw6/U6AAGNgCeSJ0=;
 b=D4M6qXOaR05hOFyZ5mjVUD2Gbqj/SqgLd3O2MQ9C2uhSjzki0U4sKrgakyTovAav9liZxUa1c8nAg6micPXQPPSTUxaqE+i3GTl7JvJDJlT5YlxaG5hrOF5YpRPM7VE5Iy2PcdFMuaKSUBF9eU8A/KORWE6jBgp10mgNuYT/lPGlvcaw7U1aWXNruSA+D0D/7E5KGpwjJssOAKAEK2RkdIkeL7zfgoR2opMkeGdHWZy0b0RUtstToMKX+6UPcNbPfrURSKiWvjpYAjOlywxumeNC14hEu6znZ8MNIa6g8M9vbBR57OtP9kYwZx2Ugi2ztCAVeH+7N6DswuedMeg9Bg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YJpL75vOv1uu/c6yyOoRd6o8Efhvw6/U6AAGNgCeSJ0=;
 b=CqsTUhvP+CsCJiLGTnc9u0xa585YmgzJRpJaoE0dd1ln0FRX3ZLY/s2czt+rV/mROxBfWJI6zeitHMYp0TeEAatsGtaiohxKl19iYJTNVsoRgvM0aUwZBdCg872wlFOfoxO0UYUcXd9Qsfb7bHaXSXf2SwmgYoWRiaNS5pG8liM=
To: Juergen Gross <jgross@suse.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Xen-devel <xen-devel@lists.xenproject.org>
CC: Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Crash when using cpupools
Message-ID: <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
Date: Mon, 6 Sep 2021 09:46:13 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LNXP265CA0072.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:5d::36) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8df172f7-ae1b-45b4-5282-08d97112cc3c
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5437:
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5437D3425DCA87DE885C124DBAD29@SJ0PR03MB5437.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: DPXAKvFhAiDVtkNA9WklNP1J5rWIiz+Xfwl2ZzT5/9YAdJooeKf1zmOzbBAfrC7oOWYEGc/eOef3XKczWHNaKjvhp8bqDZXib/7rKKwvdpgjMj29pA/Jmd+jQXxmpTZLyd45qI5xveouN2yQDlig2P0CdLuRtfT16guTZVwnJonrY+vvoXCAXu8LGxZxV4UEc4syOkCIYKcS1MM90ujfHjZW24vXgyy2vnIeWjAUk9eqVfD0okDu7COQRl+iHR7j0isHuNuUKCBnC0XSfMtX1ZJFqNS/uOi3VmfZNWi4WE9oZRcRdvr/GFraQ/b0pqAy2tmwQoUPr+FdiTAe+YHHUl7DfLuTLli2vl3E1CvIYFp7knkduRPkXgX0iQw1cpVJZQya5VMyjLlVX1hYs1piXN6tXTptbwhMOcxjDXhGcjOX+De1mx3/9ow/ZNy0XLomRuIoyEihyAy+mNx4SM4kqAtder4dA0PNGAU5hpIBK9/tgslMUS7h13wslR5KbIR16QdnIJ49llNMVW4qV9yRxdL4DiIOVMCuAgZfjU/1eMvU5+U9DHY5pmaF6v60nbxRNyW23+na58LO7Y6iY8kduiwAFp4B1GfksfIBjRhZWGTGwKcMRLVt35THGNeratr+rkJDALPpAOOdS7nAwXdT2kQUDd5MSzt6awE521xZ94qJm0p+7XygGSOYs0gi7RMEow/AbxqUkX7JpMQ9CYiQxCW5k+C7IP694LPKptWWf3M=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(136003)(366004)(346002)(31696002)(53546011)(26005)(186003)(316002)(38100700002)(86362001)(55236004)(478600001)(4326008)(2616005)(6486002)(2906002)(110136005)(83380400001)(54906003)(956004)(8936002)(6666004)(3480700007)(31686004)(8676002)(66556008)(66476007)(66946007)(16576012)(36756003)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Wm9DK0ZFc3h3U0IweHBNM0dPODFjMGJjR29NVU4rWUtqNFJGWkc4VThqWWhK?=
 =?utf-8?B?OERSK2h3WFFZdWZXV3E1U1FaUWh2NjRKa1piVTk4bXQ3SlBoRGpaUHdxVEtV?=
 =?utf-8?B?T3E3bm5iTndPQkN5MlQ0TjRKVnpoTXBrTittdU9HTkhrejZOREJmU2JKb1NK?=
 =?utf-8?B?cUIwUklRb3BDdXdGTW10L2dybnd4eDhNTnBwYktzdGYxZ1VLOWQxek1vN21F?=
 =?utf-8?B?Rk9VVGZvY1RIRFZYNmtqMWNyTUprSnFBKzlIZGQzU052UWN3MmN0QmJxanlw?=
 =?utf-8?B?ME1KUms5Sm1xTFNDUWdHaHdxeUF5ZmRvaGJYcXEyUFNRRFI0bjhRTE51eGJW?=
 =?utf-8?B?L1JOdjFPMysxUEpvVHlDaCtaVittbFkyS29zZkEwaS9UTHk1QWtsVlZIdlNt?=
 =?utf-8?B?cXBzRlFlL1dVVlNZQVFuYTJQcENadzFNL1ZPMHcxK3BLRlNLNGpqbFBzMSt5?=
 =?utf-8?B?bzcwSnF1VmxTWHd5OTFLMFp1SW1aRUs0RkdZMzAwQXpGTElIc3dXNUw2Qi96?=
 =?utf-8?B?TEQ3SHY5d0xVR1VRcWFJSmxXSGF2SVMrT0NZTEZoMmR6ZCt4U0paVVlqSXNB?=
 =?utf-8?B?TTdtNU5IYVdweXpBcHl6WU9ibzQzUmxybzNpWFltc2o4QkMyT1ZLRHMrTjFZ?=
 =?utf-8?B?T0ZEa1NXb0cxZkxxb2hxZDgxck1OQ3d0WTZDSEwrT0R2RkgzSkNsb0lTWVVB?=
 =?utf-8?B?VEYranlZSEtySlJaaWR5Qy9IcG93cElKN3YxbDVySmhndmtNZFJkTnUxaHpj?=
 =?utf-8?B?cUVhaDNFS3FlMTd1ZlY1V1VJZjBJWUpZQkNiSWVvdVBhT01oSmlRa1pva0NR?=
 =?utf-8?B?VnkxQ3l6ZjRsQ2IzaklNUTMwSVI1VSsyZmtVNDFjQ3ViUzVBbmFmQVVaTjlN?=
 =?utf-8?B?UG43UHkrd2JML3VFeG1EdC9LdEVueERoUlpGY3BoK25qdEJqSjZkajZQWERt?=
 =?utf-8?B?Y0tjdzhiRzlaTjVBNk1WKzhlQVlxZ3RDWXZ2NVo1MEVtNzV6U2JjeGhjR3E5?=
 =?utf-8?B?cTlBNE5PSjIvYWp1OGRCR3VFa3NYN1Y5YVRSNEt6eG9ZMWlWNWZ4NUdsaGNU?=
 =?utf-8?B?WnlMQlBWc0YwTXNFRW1ORXNmNVpaYm1ZNEpPV2xORUZKSlB5czhobjUycjlv?=
 =?utf-8?B?S1lQVlEzOEU1Q3VIL0wrOEZDWmpyemFNYUtZWWhDSkJra1FRN1FnVmdIdzg1?=
 =?utf-8?B?YktvUm5pT3NkVWlCMEc3MHVzWC9xNHNVbnQ5T1h5cTNBMjFwTmFBVzFXclZj?=
 =?utf-8?B?RFQxWHE5Z1lFNnRlQXZRNGh3dkZDVW92UDhQSWpSbE5IN1kwQ05ZMjh6N0J6?=
 =?utf-8?B?VDZNaTRzbVhlSGZlYWdtdUFIejMxL1dpUk9QbURnaHMzeHlWb3cxbkY3bTRJ?=
 =?utf-8?B?Y0RCWUI1TE1Ca2lLS1NxZkw4Q0pPTWVhS1oxTjkwVUJRRDIrYXg5RkVsMVBT?=
 =?utf-8?B?azM5di83TnI0Z1NuTWNpZEJmdVNlNGl1TnVVK0hnZnVLblduSHN6THo0eFNE?=
 =?utf-8?B?RnQzczc2UVcwczR2NDhrNWUwN3NuSHNHZFBITHp0aUs1TXcrYmpvMm84Y0I2?=
 =?utf-8?B?a2RLWkkxTE1sMXYvNldjZzhDb3YxeWYwcmMrQkVCKzlmTU9kbUE4UUZBUGNw?=
 =?utf-8?B?ZmJzYkNtbW9weWpxR2ViekVlcldRNDJTWGRkbXZYQXdTeFA3bUtucGk4bm5u?=
 =?utf-8?B?V2ZtRC90aE5uOXR5aWZVZjlBQ1VwY3BFVzg2T01DK0xLTnllZjVjaVZFY1hF?=
 =?utf-8?Q?8oa+g9RU4smc6oRsS1tBpuLp2GLMzuv6Tad8e6H?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8df172f7-ae1b-45b4-5282-08d97112cc3c
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 08:46:20.1470
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: orid3NBW0Hg8Ay7PiHOLc7tnUUYbIn2KlEV4N/FVr9QdppjX8KRZ0T1txRJPmaXx41a532IZyLZd1REvu4ebevO7u6kbV2UC5mRijClQ2kY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5437
X-OriginatorOrg: citrix.com

On 06/09/2021 09:23, Juergen Gross wrote:
> On 03.09.21 17:41, Bertrand Marquis wrote:
>> Hi,
>>
>> While doing some investigation with cpupools I encountered a crash
>> when trying to isolate a guest to its own physical cpu.
>>
>> I am using current staging status.
>>
>> I did the following (on FVP with 8 cores):
>> - start dom0 with dom0_max_vcpus=3D1
>> - remove core 1 from dom0 cpupool: xl cpupool-cpu-remove Pool-0 1
>> - create a new pool: xl cpupool-create name=3D\"NetPool\=E2=80=9D
>> - add core 1 to the pool: xl cpupool-cpu-add NetPool 1
>> - create a guest in NetPool using the following in the guest config:
>> pool=3D=E2=80=9CNetPool"
>>
>> I end with a crash with the following call trace during guest creation:
>> (XEN) Xen call trace:
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234cb0>] credit2.c#csched2_alloc_uda=
ta+0x58/0xfc
>> (PC)
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234c80>] credit2.c#csched2_alloc_uda=
ta+0x28/0xfc
>> (LR)
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000242d38>] sched_move_domain+0x144/0x6=
c0
>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022dd18>]
>> cpupool.c#cpupool_move_domain_locked+0x38/0x70
>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022fadc>] cpupool_do_sysctl+0x73c/0x7=
80
>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022d8e0>] do_sysctl+0x788/0xa58
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000273b68>] traps.c#do_trap_hypercall+0=
x78/0x170
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000274f70>] do_trap_guest_sync+0x138/0x=
618
>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000260458>] entry.o#guest_sync_slowpath=
+0xa4/0xd4
>>
>> After some debugging I found out that unit->vcpu_list is NULL after
>> unit->vcpu_list =3D d->vcpu[unit->unit_id]; with unit_id 0 in
>> common/sched/core.c:688
>> This makes the call to is_idle_unit(unit) in csched2_alloc_udata
>> trigger the crash.
>
> So there is no vcpu 0 in that domain? How is this possible?

Easy, depending on the order of hypercalls issued by the toolstack.

Between DOMCTL_createdomain and DOMCTL_max_vcpus, the domain exists but
the vcpus haven't been allocated.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:48:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:48:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179497.325777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAIh-0006tu-95; Mon, 06 Sep 2021 08:48:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179497.325777; Mon, 06 Sep 2021 08:48:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAIh-0006tk-6B; Mon, 06 Sep 2021 08:48:23 +0000
Received: by outflank-mailman (input) for mailman id 179497;
 Mon, 06 Sep 2021 08:48:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNAIg-0006te-6c
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:48:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAIc-00088p-UG; Mon, 06 Sep 2021 08:48:18 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAIc-0001IX-O6; Mon, 06 Sep 2021 08:48:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=qgP+hT12nMxjdDowc+pmDpM6YfKK2qq0BIsypq9S8TQ=; b=f1IymJEEkbNVWgm6AmeL9FLn2R
	CQNqwFkVqniWLiVKx6jxtsIHtLj3pt45extwmdqpFTEViIEKMGvDxQf6uv82ISB8odz1xU/r1AZTw
	p8ZZ2ChPVfjMIWp2mhug7KsZarI/nrqr3ZGc8uGr0i4PqXIExgKPsfMGwoPN/YFHKBVY=;
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
Date: Mon, 6 Sep 2021 09:48:16 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

On 06/09/2021 08:02, Oleksandr Andrushchenko wrote:
> Hi, Julien!

Hi Oleksandr,

> On 03.09.21 12:04, Julien Grall wrote:
>> Hi Oleksandr,
>>
>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> vPCI may map and unmap PCI device memory (BARs) being passed through which
>>> may take a lot of time. For this those operations may be deferred to be
>>> performed later, so that they can be safely preempted.
>>> Run the corresponding vPCI code while switching a vCPU.
>>
>> IIUC, you are talking about the function map_range() in xen/drivers/vpci/header. The function has the following todo for Arm:
>>
>>          /*
>>           * ARM TODOs:
>>           * - On ARM whether the memory is prefetchable or not should be passed
>>           *   to map_mmio_regions in order to decide which memory attributes
>>           *   should be used.
>>           *
>>           * - {un}map_mmio_regions doesn't support preemption.
>>           */
>>
>> This doesn't seem to be addressed in the two series for PCI passthrough sent so far. Do you have any plan to handle it?
> 
> No plan yet.
> 
> All the mappings are happening with p2m_mmio_direct_dev as of now.

So this address the first TODO but how about the second TODO? It refers 
to the lack of preemption on Arm but in this patch you suggest there are 
some and hence we need to call vpci_process_pending().

For a tech preview, the lack of preemption would be OK. However, the 
commit message should be updated to make clear there are no such 
preemption yet or avoid to mention it.

> 
>>
>>>
>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>> ---
>>>    xen/arch/arm/traps.c | 6 ++++++
>>>    1 file changed, 6 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>>> index 219ab3c3fbde..1571fb8afd03 100644
>>> --- a/xen/arch/arm/traps.c
>>> +++ b/xen/arch/arm/traps.c
>>> @@ -34,6 +34,7 @@
>>>    #include <xen/symbols.h>
>>>    #include <xen/version.h>
>>>    #include <xen/virtual_region.h>
>>> +#include <xen/vpci.h>
>>>      #include <public/sched.h>
>>>    #include <public/xen.h>
>>> @@ -2304,6 +2305,11 @@ static bool check_for_vcpu_work(void)
>>>        }
>>>    #endif
>>>    +    local_irq_enable();
>>> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )
>>
>> Looking at the code of vpci_process_pending(), it looks like there are some rework to do for guest. Do you plan to handle it as part of the vPCI series?
> Yes, vPCI code is heavily touched to support guest non-identity mappings

I wasn't referring to the non-identity mappings here. I was referring to 
TODOs such as:

             /*
              * FIXME: in case of failure remove the device from the domain.
              * Note that there might still be leftover mappings. While 
this is
              * safe for Dom0, for DomUs the domain will likely need to be
              * killed in order to avoid leaking stale p2m mappings on
              * failure.
              */

You still have them after the series reworking the vPCI. As for the 
preemption this is OK to ignore it for a tech preview. Although, we want 
to at least track them.

>>
>>> +        raise_softirq(SCHEDULE_SOFTIRQ);
>>> +    local_irq_disable();
>>> +
>>
>>  From my understanding of vcpi_process_pending(). The function will return true if there are more work to schedule.
> Yes
>> However, if check_for_vcpu_for_work() return false, then we will return to the guest before any work for vCPI has finished. This is because check_for_vcpu_work() will not be called again.
> Correct
>>
>> In this case, I think you want to return as soon as you know we need to reschedule.
> Not sure I understand this

The return value of check_for_vcpu_for_work() indicates whether we have 
more work to do before returning to return to the guest.

When vpci_process_pending() returns true, it tells us we need to call 
the function at least one more time before returning to the guest.

In your current implementation, you leave that decision to whoeever is 
next in the function.

It is not safe to return to the guest as long as vpci_process_pending() 
returns true. So you want to write something like:

if ( vpci_process_pending() )
   return true;

>>
>> However, looking at the rest of the code, we already have a check for vpci in the common IOREQ code.
> 
> Which may not be enabled as it depends on CONFIG_IOREQ_SERVER.

Right. My point is when CONFIG_IOREQ_SERVER is set then you would end up 
to call twice vpci_process_pending(). This will have an impact how on 
long your vCPU is going to running because you are doubling the work.

> 
> My understanding is that for x86 it is always enabled, but this might not be the case for Arm
> 
>> So we would end up to call twice vpci_process_pending().
> So, if CONFIG_IOREQ_SERVER is not enabled then we end up with only calling it from traps.c on Arm
>> Maybe we should move the call from the IOREQ to arch-code.
> 
> Hm. I would better think of moving it from IOREQ to some other common code: for x86 (if
> 
> my understanding correct about CONFIG_IOREQ_SERVER) it is by coincidence that we call vPCI
> 
> code from there and IOREQ is always enabled.

I am not aware of another suitable common helper that would be called on 
the return to the guest path. Hence why I suggest to possibly duplicated 
the code in each arch path.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 08:53:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 08:53:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179503.325789 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAN4-0008U1-SI; Mon, 06 Sep 2021 08:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179503.325789; Mon, 06 Sep 2021 08:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAN4-0008Tu-P8; Mon, 06 Sep 2021 08:52:54 +0000
Received: by outflank-mailman (input) for mailman id 179503;
 Mon, 06 Sep 2021 08:52:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNAN2-0008To-LU
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 08:52:52 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAN0-0008DW-Rl; Mon, 06 Sep 2021 08:52:50 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAN0-0001YF-L9; Mon, 06 Sep 2021 08:52:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=65UVgM6ReCYo2fAUvHxEoeQ5/yGHQxH9PkAc5H3Ip5Q=; b=gi0SGXbJYvZk5BKbGX50do6dsv
	5ex6MFpHN6vzZWOx0mCEtr8jIKBDFzM96jZ/eygMua1vDyInBk+m65pscKriVmesU4lfOw9W+ABNu
	R+t4MuUP2Sc8bPpqPjBRN2PiMVSVhBac1NCOCDNKQ0G5kp0jFjJTTuIdNXuvSeMZiAcY=;
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Michal Orzel <michal.orzel@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Ash Wilding <ash.j.wilding@gmail.com>, Ian Jackson <iwj@xenproject.org>
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <4a5ac822-3078-de0e-8f84-6881d47ad3eb@xen.org>
 <dd9859ec-a3a9-6a74-bdae-fde092f26ac0@arm.com>
 <e8bdb065-9b46-e0d8-1fcf-7bba38e02f9e@xen.org>
 <8974fc82-94c1-ef85-153c-0923f5faf331@arm.com>
 <f41e36bb-efb5-fc11-f87e-6906336bca45@xen.org>
 <9502c0fa-3e46-3cd3-7de8-c7adccca1397@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <71432e6c-1ef6-f2c2-623b-d60df607957d@xen.org>
Date: Mon, 6 Sep 2021 09:52:48 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <9502c0fa-3e46-3cd3-7de8-c7adccca1397@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Michal,

On 03/09/2021 10:16, Michal Orzel wrote:
> 
> 
> On 02.09.2021 14:50, Julien Grall wrote:
>> On 01/09/2021 10:38, Michal Orzel wrote:
>>> Hi Julien,
>>
>> Hi Michal,
>>
>>> On 06.08.2021 13:12, Julien Grall wrote:
>>>>
>>>>
>>>> On 29/07/2021 12:47, Michal Orzel wrote:
>>>>> Hi Julien,
>>>>
>>>> Hi Michal,
>>>>
>>>>> On 29.07.2021 13:20, Julien Grall wrote:
>>>>>> Hi Michal,
>>>>>>
>>>>>> On 29/07/2021 11:42, Michal Orzel wrote:
>>>>>>> According to ARMv8A architecture, AArch64 registers
>>>>>>> are 64bit wide even though in many cases the upper
>>>>>>> 32bit is reserved. Therefore there is no need for
>>>>>>> function vreg_emulate_sysreg32 on arm64. This means
>>>>>>> that we can have just one function vreg_emulate_sysreg
>>>>>>> using new function pointer:
>>>>>>> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>>>>>>>                                   register_t *r, bool read);
>>>>>>>
>>>>>>> Modify vreg_emulate_cp32 to use the new function pointer
>>>>>>> as well.
>>>>>>>
>>>>>>> This change allows to properly use 64bit registers in AArch64
>>>>>>> state and in case of AArch32 the upper 32 bits of AArch64
>>>>>>> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
>>>>>>
>>>>>> What you wrote only says that the bits are ignored. It doesn't say whether the bits will be 0.
>>>>>>
>>>>>> They are probably, but as I wrote yesterday, I couldn't confirm it.
>>>>>>
>>>>> Should I then remove this part of the commit or write below?:
>>>>> "We can assume that those bits will be 0 but the architecture
>>>>> reference manual does not clarify this."
>>>>
>>>> There was some back and forth on security@xen.org about this. I will summarizing the discussion here as we considered this was a just a bug.
>>>>
>>>> I wasn't looking at the correct section in the Arm Arm. There is a paragraph clearly describing the expected behavior in a different section (thanks Ash for the pointer!). Per section D1.19.2 in DDI 0487F.c:
>>>>
>>>> "
>>>> If the general-purpose register was accessible from AArch32 state
>>>>
>>>> The upper 32 bits either become zero, or hold the value that the same architectural register held before any AArch32 execution. The choice between these two options is IMPLEMENTATION DEFINED, and might vary dynamically within an implementation. Correspondingly, software must
>>>> regard the value as being a CONSTRAINED UNPREDICTABLE choice between these two values.
>>>>
>>>> This behavior applies regardless of whether any execution occurred at the Exception level that was using AArch32. That is, this behavior applies even if AArch32 state was entered by an exception
>>>> return from AArch64 state, and another exception was immediately taken to AArch64 state without any instruction execution in AArch32 state.
>>>> "
>>>>
>>>> So we can't assume the top 32-bits are zeroed unless the hypervisor ensured they were. Today, we don't have that guarantee in Xen.
>>>>
>>>> This needs to be fixed. The two approachs we discussed are:
>>>>      1) Update set_user_reg() to zero the top 32-bit. We have a couple of places using directly the fields xN. So we would need to switch them to use set_user_reg()
>>>>      2) Only saving/restoring the bottom 32-bit when entering/leaving the hypervisor.
>>>>
>>>> At the moment, my preference goes towards the latter because we don't risk to introduce new place where set_user_reg() is not used.
>>>>
>>>> I have quickly hack the entry path. This would look like:
>>>>
>>>> diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
>>>> index fc3811ad0ad5..65e24c88b059 100644
>>>> --- a/xen/arch/arm/arm64/entry.S
>>>> +++ b/xen/arch/arm/arm64/entry.S
>>>> @@ -111,6 +111,11 @@
>>>>     */
>>>>            .macro  entry, hyp, compat, save_x0_x1=1
>>>>            sub     sp, sp, #(UREGS_SPSR_el1 - UREGS_LR) /* CPSR, PC, SP, LR */
>>>> +        .if \compat == 1 /* AArch32 mode */
>>>> +        /* Clobber the top 32-bit of the registers */
>>>> +        mov    w0, w0
>>>> +        mov    w1, w1
>>>> +        .endif
>>>>            push    x28, x29
>>>>            push    x26, x27
>>>>            push    x24, x25
>>>>
>>>> I haven't looked whether this can be optimized or the exit path would be easier to modify.
>>>>
>>>> Anyway, this is not a new bug so I would be fine to get this patch merged first. Although, I think this wants to be fixed for xen 4.16 (CCing Ian to track it).
>>>>
>>>> I will try to find sometimes in the next couple of weeks to fix it and have another review of this patch.
>>>>
>>>
>>> As the 4.16 release is getting closer I wanted to ask whether you need help with creating a pre-work patch so that this patch can be merged.
>>
>> Sorry I haven't looked at the bug yet. I don't think the bug I reported necessarily needs to go first. This is not a new bug and AFAICT your patch is not going to make it worse than the current state.
>>
>> Your patch needs to have an updated commit message summarizing what we discussed and a second review. No need to resend a patch for the updated commit message, this can be discussed here and updated on commit (I assuming there is no other comments).
>>
> If the following message is ok for you, please update the commit msg after acking this change:
> "
> According to ARMv8A architecture, AArch64 registers
> are 64bit wide even though in many cases the upper
> 32bit is reserved. Therefore there is no need for
> function vreg_emulate_sysreg32 on arm64. This means
> that we can have just one function vreg_emulate_sysreg
> using new function pointer:
> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>                                register_t *r, bool read);
> 
> Modify vreg_emulate_cp32 to use the new function pointer
> as well.
> 
> This change allows to properly use 64bit registers in AArch64
> state. In case of AArch32 the documantation (D1.20.2, DDI 0487A.j)

s/documantation/documentation/

> states that "the upper 32 bits either become zero, or hold the value
> that the same architectural register held before any AArch32 execution."
> As the choice between them is IMPLEMENTATION DEFINED we cannot assume they
> are zeroed. Xen should ensure that but currently it does not. This is not
> a new bug and must be fixed as agreed during a discussion over this patch.
> "

THis looks good to me. I will fold it on commit.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:05:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:05:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179510.325800 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAZ4-0001jQ-W3; Mon, 06 Sep 2021 09:05:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179510.325800; Mon, 06 Sep 2021 09:05:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAZ4-0001jJ-T4; Mon, 06 Sep 2021 09:05:18 +0000
Received: by outflank-mailman (input) for mailman id 179510;
 Mon, 06 Sep 2021 09:05:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=oa8G=N4=arm.com=Hongda.Deng@srs-us1.protection.inumbo.net>)
 id 1mNAZ3-0001jD-Oo
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:05:17 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.73]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8ce56666-0ef1-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 09:05:16 +0000 (UTC)
Received: from PR0P264CA0140.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::32)
 by AM0PR08MB5443.eurprd08.prod.outlook.com (2603:10a6:208:183::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 09:05:14 +0000
Received: from VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1a:cafe::2f) by PR0P264CA0140.outlook.office365.com
 (2603:10a6:100:1a::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Mon, 6 Sep 2021 09:05:14 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT037.mail.protection.outlook.com (10.152.19.70) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 09:05:14 +0000
Received: ("Tessian outbound 19a632698c93:v103");
 Mon, 06 Sep 2021 09:05:13 +0000
Received: from 0031496d2023.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 06815203-31B7-4BCB-9BBE-677527EA3505.1; 
 Mon, 06 Sep 2021 09:05:02 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0031496d2023.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 09:05:02 +0000
Received: from VE1PR08MB5677.eurprd08.prod.outlook.com (2603:10a6:800:1ab::17)
 by VI1PR0801MB1967.eurprd08.prod.outlook.com (2603:10a6:800:8c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 09:04:59 +0000
Received: from VE1PR08MB5677.eurprd08.prod.outlook.com
 ([fe80::d810:52a3:dce3:4d06]) by VE1PR08MB5677.eurprd08.prod.outlook.com
 ([fe80::d810:52a3:dce3:4d06%5]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 09:04:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ce56666-0ef1-11ec-b09c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VQ8S4ENHD1kcr09LTDYDkrm1ZbBEVK9tOxw2UuxhjNg=;
 b=nodtvef8gMM28EzyBM6VGGxJXZP1BnfgoFd/TVEta4MlOcRlez8QafyprKrvs/uL9WiNsKzlpL5+M0uzPV4Q7x1Y/wqSZcbFYKXkXT69EQxeTGQ4k6C2H8ne8nmSmnxlzdHxU0rJtpR1V8Yafz6T7WY8yQxP2G+1zpFBfEp8XfQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HN0sKJibKDo/blIlWl85ccwtSKNfa2kuygtgrJmfQiE+T5d1za6aU1y0xzhicF3Rfpvr/IXx3ECjuyGUTwxE0AbUBscj5JZZxyLcCYWzIdy4lBP+Alclixdn0sQ8aKppUhLYkHQPOdJ97ggBoj+HGsrwuUFDDCeqwqQ2UG4qAfrEpjoaswDei4ev//dbf3EY5lHoTO/bJXfGcyihP3xBouu3YBOqfg3dVlUgMWW+5/eCrXJirSNYKqo44RQbW6e4oC/Pe0e/WXb7AEvgbIe5hQVKqXJr//Zzsn7KN4yw8W+Yj7gHdMEaf7SUtBh5ci8BEi671KzD1CyPZZJekTASCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VQ8S4ENHD1kcr09LTDYDkrm1ZbBEVK9tOxw2UuxhjNg=;
 b=jG804fkFsxTqSXfiks5B1hZUOjQRG4tNZx2i5uj5JH4npXwdJvqM/JyjKjZeNVeZB2sbptaMeWT3Jj6BW2MMFc3jzE2Qx2Ov8WH5FDzgw4FNSfzaYP4dD/ERvE5NA8Jy5gviyO7Ysvc5bU5WP3/G3NL0K1Fu3KGgrvxqIhUYQH950xHnEOc1Or64s4muKcw6RMGup7u4OZ9H2ACihOzFb3aFUQoVBdkYn5z7TXm/4mmYw6E6/427DJDcQhK0jNyWD/NNHN1HlpfCUJUh4Ot12+D/Ki84cMmTMD/lTraS62OiMSZ0iVGGqwK5bSlTe7ILF8P8BmuJKaSqTONjSupPeQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VQ8S4ENHD1kcr09LTDYDkrm1ZbBEVK9tOxw2UuxhjNg=;
 b=nodtvef8gMM28EzyBM6VGGxJXZP1BnfgoFd/TVEta4MlOcRlez8QafyprKrvs/uL9WiNsKzlpL5+M0uzPV4Q7x1Y/wqSZcbFYKXkXT69EQxeTGQ4k6C2H8ne8nmSmnxlzdHxU0rJtpR1V8Yafz6T7WY8yQxP2G+1zpFBfEp8XfQ=
From: Hongda Deng <Hongda.Deng@arm.com>
To: "julien@xen.org" <julien@xen.org>, Wei Chen <Wei.Chen@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, nd <nd@arm.com>
Subject: About Arm guests accessing to GICD_ICPENR 
Thread-Topic: About Arm guests accessing to GICD_ICPENR 
Thread-Index: Adei/iBluoJQFkGjReWesRkbibwmOA==
Date: Mon, 6 Sep 2021 09:04:59 +0000
Message-ID:
 <VE1PR08MB5677BABDFBBCA24B4ADA9C16E6D29@VE1PR08MB5677.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 026BE3F40A943F45A19D0F3648C58C6C.0
x-checkrecipientchecked: true
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 644850ed-7d68-47a0-3d83-08d97115706c
x-ms-traffictypediagnostic: VI1PR0801MB1967:|AM0PR08MB5443:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB54431355A200535FE21012F0E6D29@AM0PR08MB5443.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 U+JsCEUnT1/WS903E+Wf7RF9MCZRlXBZqeliXiTaLzeNvVqOavKpcd+J1uaAjKIbIhGlZzk5hYncVCqk+rMkumdPcrnalpdlRoSQlRNUPb5z+f9FyiwVxBWCbxV1DNqfpBtPy/wU88fhpEDOMC5atc51UTaFrrgWnU7FRBxWjzkTqKDoAkDB7DJfFTU80NAdC7TU/qlh3d8QGs1YH0oW1yO3x8IXqIg4Pcitt3AEXq4C2r5kr8lGrgoLxi/UjytTERHiVROq+Z+TZz3jP45AuBDh2S/xtrHdaoK2hru/Guy0fhrFoFMYZ8fITjoqO8UkiZu/HBlzHBlazz2mfeupR1Mv4WcVvXmoHzJcSxOdXNmNHlUyButkjP2ggqNjG0mRe5EY8Hwe0jP/TUF8EtVdvIHl9c4TU+vnvIm8GqTRNZdWnxjGTWej3AUVN6BvrHddPcBtcawDBFILzxwpeRbvkzygaraqGakRpGYGOqVEKOS5l/tCts+GoXzH6xjp8pF2maqLYLo0+evURPll75HfaMHPQvlLlsPNgoTnPqrcWoG89/c7tmwxaGPOorI9q2g632IGbXaJScwWuhhGPUbAaZJ5pd2bNrMFl7O8jmz3iR7bajZqKzUPx/i03AcXeFohk98ykvo0zhIuQbjmmQ5ujpFZUkuGWqmLQJDf/Hz7B9vkey7KIi9LWlWVtBg6D8TLN4Q4MHAmNjzD5nnV+9KUg5Ckzvgf7KZluRdCdPM7kJPmm0DzPUOwruZxAmmBd6cgtWJuIivE12K+gCSH1xaSP3+NkuHPolowldy9P8+UM5g=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5677.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(396003)(366004)(39860400002)(346002)(54906003)(186003)(4743002)(26005)(38100700002)(966005)(2906002)(66946007)(478600001)(122000001)(55016002)(316002)(33656002)(9686003)(110136005)(76116006)(86362001)(71200400001)(7696005)(52536014)(66476007)(66556008)(66446008)(64756008)(83380400001)(5660300002)(8936002)(6506007)(4326008)(38070700005)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1967
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	abd6a0e2-6da9-4081-2d9c-08d971156796
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jD1RlvswgEv2o5dgHiAatjNNHTOVujMPHlsC1GTpt6ijl83FLYbZ5uFW9HY+J03/eiQjXiejlnJwZwOH1coZ1S3PakGmdiZQiwZK2xlaRp4kEbgjUmO2LW/HFs9m7BeiQ5PYksUnCQbk7PwKxv3we1GS3wFYeiKYUUXrDgGhO8aOfrntNSnz2IES3nOKFNNNKg3vuh42wuR0Qf4zU8H84QUso8dAKBsajA15FucYO5dPL7dTdCO1lsWZZDpGiptWLvtwAA2XmB0T+8rmIVDUV6T5P+0dLXA6P+DRZx9IsGcqIK8KuqoErN7NTsfncpszRmE5w9vBihJm+4MksBlC5spD0bvckRjavX2K/vFl5ClQRWtC5dNUzTvmz0x65iryP4lheF6Hu2OuWTWcVbvc1tcbF4VYWstmX8JkiDSEViQvk+UC5U7QG2DVEHl9d3qUEeL4vmLvG1jTWwPI1uezzpEWPs9QoX1GcRx9FoDPKcoETdquS7ebo6mEbUCrwqyoHc0Rdd0OsxP57LYmEi7cmax4asdvk6gQsRg/P4Wo33J4u9rCDYKLjYV1Sv+xHYm2bAK4P9acHcwLVOFzPjjumTTU35v9B7WJDDYStlN9Z4gbyDPaVkglZPFbCZK1ImBSLcQMnroI+3a0gT0NiE/DwJeDJUNuywkP5yJ6UW8PQb7NPBO1EWZ7UkFZpFMo2icm+VFqP9RGYirGGbdJyZ7cqpEE6sK34B1+t80w1cqVtJZcBbKRLcLh8pp89m5S9dbf9F0GGCLnf8weiJ5p/hTjNk5jJ8ZSdGw/1KPWzMQn79E=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(39860400002)(136003)(396003)(346002)(46966006)(36840700001)(70206006)(36860700001)(356005)(5660300002)(52536014)(70586007)(966005)(316002)(33656002)(9686003)(478600001)(8676002)(55016002)(82310400003)(8936002)(110136005)(4326008)(54906003)(83380400001)(81166007)(4743002)(47076005)(2906002)(82740400003)(7696005)(336012)(6506007)(186003)(86362001)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 09:05:14.2547
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 644850ed-7d68-47a0-3d83-08d97115706c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5443

Hi Julien,
=20
Xen provides vGIC to support Xen guests, and currently xen will return IO=20
unhandled when guests access GICD ICPENRn registers. This works fine with L=
inux=20
guests, for Linux won't access these registers. But for Zephyr, this mechan=
ism=20
will cause IO dataabort on Zephyr's initialization which makes Zephyr get i=
n=20
fatal error.
=20
One method to ease this is to let vGIC ignore GICD ICPENRn registers access=
. I=20
tested it with Linux guests and Zephyr guests, and both works fine. And I f=
ound=20
in this patch[1] that it would be more complex to touch the read part for=20
I{S,C}PENDR and the write part for ICPENDR, so could we ignore GICD ICPENDR=
n=20
registers access to ease Zephyr's initialization problem?

If Xen wants a complete GICD ICPENDRn emulation to fix it, do you have any=
=20
suggestions?

[1] https://www.mail-archive.com/search?l=3Dxen-devel@lists.xenproject.org&=
q=3D
subject:%22Re%5C%3A+%5C%5BPATCH+for%5C-4.15%5C%5D+xen%5C%2Fvgic%5C%3A
+Implement+write+to+ISPENDR+in+vGICv%5C%7B2%2C+3%5C%7D%22&o=3Dnewest&f=3D1

Cheers
Hongda


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:07:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:07:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179517.325811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAbO-0002RQ-HO; Mon, 06 Sep 2021 09:07:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179517.325811; Mon, 06 Sep 2021 09:07:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAbO-0002RJ-D2; Mon, 06 Sep 2021 09:07:42 +0000
Received: by outflank-mailman (input) for mailman id 179517;
 Mon, 06 Sep 2021 09:07:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNAbM-0002RD-Ur
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:07:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAbL-0008Ud-RK; Mon, 06 Sep 2021 09:07:39 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNAbL-0003Ez-Ky; Mon, 06 Sep 2021 09:07:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=qDjbz5x+lhQVqDghklbiyW58OCiSPI+82nSf0BsV7VA=; b=OakPuhnoFXchkbRDXMKpxDoRFi
	aX+BL9NnxJYXfPgvdL5pO8Y5s37h/bctRncqVtK2A0AtwEW3cxXXHHmk3iX/NLB5x5E3vrpRpmkSt
	fqNgdSR0U//Ot8zjxMFXB01pcYTDKjGEpn20/o+1RI6QZvIxFFKxc8YVM5pFxuFFcddM=;
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Michal Orzel <michal.orzel@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com
References: <20210729104258.6320-1-michal.orzel@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <24ba08cc-acd4-c64a-1e90-dc4c8efbfe48@xen.org>
Date: Mon, 6 Sep 2021 10:07:37 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210729104258.6320-1-michal.orzel@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Michal,

On 29/07/2021 11:42, Michal Orzel wrote:
> According to ARMv8A architecture, AArch64 registers
> are 64bit wide even though in many cases the upper
> 32bit is reserved. Therefore there is no need for
> function vreg_emulate_sysreg32 on arm64. This means
> that we can have just one function vreg_emulate_sysreg
> using new function pointer:
> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>                                register_t *r, bool read);
> 
> Modify vreg_emulate_cp32 to use the new function pointer
> as well.
> 
> This change allows to properly use 64bit registers in AArch64
> state and in case of AArch32 the upper 32 bits of AArch64
> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
> 
> Signed-off-by: Michal Orzel <michal.orzel@arm.com>
> ---
>   xen/arch/arm/arm64/vsysreg.c    |  2 +-
>   xen/arch/arm/vcpreg.c           | 16 ++++++++++----
>   xen/arch/arm/vgic-v3.c          |  2 +-
>   xen/arch/arm/vtimer.c           | 11 +++++-----
>   xen/include/asm-arm/processor.h |  4 ++--
>   xen/include/asm-arm/vreg.h      | 38 ++++++---------------------------
>   6 files changed, 29 insertions(+), 44 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index caf17174b8..73fa2ca9ae 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -64,7 +64,7 @@ TVM_REG(CONTEXTIDR_EL1)
>       {                                                                   \
>           bool res;                                                       \
>                                                                           \
> -        res = vreg_emulate_sysreg64(regs, hsr, vreg_emulate_##reg);     \
> +        res = vreg_emulate_sysreg(regs, hsr, vreg_emulate_##reg);       \
>           ASSERT(res);                                                    \
>           break;                                                          \
>       }
> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
> index e3ce56d875..be1ec08159 100644
> --- a/xen/arch/arm/vcpreg.c
> +++ b/xen/arch/arm/vcpreg.c
> @@ -57,9 +57,17 @@
>   #define WRITE_SYSREG_SZ(sz, val, sysreg...)  WRITE_SYSREG##sz(val, sysreg)
>   #endif
>   
> +/*
> + * type32_t is defined as register_t due to the vreg_emulate_cp32 and
> + * vreg_emulate_sysreg taking function pointer with register_t type used for
> + * passing register's value.
> + */
> +typedef register_t type32_t;
> +typedef uint64_t type64_t ;

NIT: spurious space before ;.

> +
>   /* The name is passed from the upper macro to workaround macro expansion. */
>   #define TVM_REG(sz, func, reg...)                                           \
> -static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read)    \
> +static bool func(struct cpu_user_regs *regs, type##sz##_t *r, bool read)    \
>   {                                                                           \
>       struct vcpu *v = current;                                               \
>       bool cache_enabled = vcpu_has_cache_enabled(v);                         \
> @@ -83,7 +91,7 @@ static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read)    \
>   
>   #else /* CONFIG_ARM_64 */
>   #define TVM_REG32_COMBINED(lowreg, hireg, xreg)                             \
> -static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r,    \
> +static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, register_t *r,  \
>                                   bool read, bool hi)                         \
>   {                                                                           \
>       struct vcpu *v = current;                                               \
> @@ -108,13 +116,13 @@ static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r,    \
>       return true;                                                            \
>   }                                                                           \
>                                                                               \
> -static bool vreg_emulate_##lowreg(struct cpu_user_regs *regs, uint32_t *r,  \
> +static bool vreg_emulate_##lowreg(struct cpu_user_regs *regs, register_t *r,\
>                                     bool read)                                \
>   {                                                                           \
>       return vreg_emulate_##xreg(regs, r, read, false);                       \
>   }                                                                           \
>                                                                               \
> -static bool vreg_emulate_##hireg(struct cpu_user_regs *regs, uint32_t *r,   \
> +static bool vreg_emulate_##hireg(struct cpu_user_regs *regs, register_t *r, \
>                                    bool read)                                 \
>   {                                                                           \
>       return vreg_emulate_##xreg(regs, r, read, true);                        \
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 613f37abab..cb5a70c42e 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1531,7 +1531,7 @@ static bool vgic_v3_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
>       switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
>       {
>       case HSR_SYSREG_ICC_SGI1R_EL1:
> -        return vreg_emulate_sysreg64(regs, hsr, vgic_v3_emulate_sgi1r);
> +        return vreg_emulate_sysreg(regs, hsr, vgic_v3_emulate_sgi1r);
>   
>       default:
>           return false;
> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
> index 167fc6127a..0196951af4 100644
> --- a/xen/arch/arm/vtimer.c
> +++ b/xen/arch/arm/vtimer.c
> @@ -162,7 +162,8 @@ void virt_timer_restore(struct vcpu *v)
>       WRITE_SYSREG(v->arch.virt_timer.ctl, CNTV_CTL_EL0);
>   }
>   
> -static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, uint32_t *r, bool read)
> +static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, register_t *r,
> +                            bool read)
>   {
>       struct vcpu *v = current;
>       s_time_t expires;
> @@ -197,7 +198,7 @@ static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, uint32_t *r, bool read)
>       return true;
>   }
>   
> -static bool vtimer_cntp_tval(struct cpu_user_regs *regs, uint32_t *r,
> +static bool vtimer_cntp_tval(struct cpu_user_regs *regs, register_t *r,
>                                bool read)
>   {
>       struct vcpu *v = current;
> @@ -316,11 +317,11 @@ static bool vtimer_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
>       switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
>       {
>       case HSR_SYSREG_CNTP_CTL_EL0:
> -        return vreg_emulate_sysreg32(regs, hsr, vtimer_cntp_ctl);
> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_ctl);
>       case HSR_SYSREG_CNTP_TVAL_EL0:
> -        return vreg_emulate_sysreg32(regs, hsr, vtimer_cntp_tval);
> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_tval);
>       case HSR_SYSREG_CNTP_CVAL_EL0:
> -        return vreg_emulate_sysreg64(regs, hsr, vtimer_cntp_cval);
> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_cval);
>   
>       default:
>           return false;
> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
> index 2577e9a244..2058b69447 100644
> --- a/xen/include/asm-arm/processor.h
> +++ b/xen/include/asm-arm/processor.h
> @@ -484,9 +484,9 @@ extern register_t __cpu_logical_map[];
>   #define CNTKCTL_EL1_EL0PTEN  (1u<<9) /* Expose phys timer registers to EL0 */
>   
>   /* Timer control registers */
> -#define CNTx_CTL_ENABLE   (1u<<0)  /* Enable timer */
> +#define CNTx_CTL_ENABLE   (1ul<<0)  /* Enable timer */
>   #define CNTx_CTL_MASK     (1ul<<1)  /* Mask IRQ */
> -#define CNTx_CTL_PENDING  (1u<<2)  /* IRQ pending */
> +#define CNTx_CTL_PENDING  (1ul<<2)  /* IRQ pending */
I would suggest to mention in the commit message why this is necessary. 
AFAICT, it is not strictly necessary because you left ctl defined as a 
uin32_t. So I am guessing you only keep it for hardening purpose?

If so how about adding:

"Take the opportunity to switch CNTx_CTL_* to use UL to avoid any 
surprise with the negation of any bits (as used in vtimer_cntp_ctl)".

The rest of the patch looks fine. So I would be happy to deal with the 
fixes on commit:

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:10:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:10:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179523.325822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAdi-0003lp-VX; Mon, 06 Sep 2021 09:10:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179523.325822; Mon, 06 Sep 2021 09:10:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAdi-0003l8-Qc; Mon, 06 Sep 2021 09:10:06 +0000
Received: by outflank-mailman (input) for mailman id 179523;
 Mon, 06 Sep 2021 09:10:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LxUx=N4=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mNAdh-0003X8-17
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:10:05 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 38aac932-0ef2-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 09:10:03 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 65D59D6E;
 Mon,  6 Sep 2021 02:10:03 -0700 (PDT)
Received: from [10.57.15.22] (unknown [10.57.15.22])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1B3143F73D;
 Mon,  6 Sep 2021 02:10:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38aac932-0ef2-11ec-b09c-12813bfff9fa
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <24ba08cc-acd4-c64a-1e90-dc4c8efbfe48@xen.org>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <f4ee4519-9153-1671-179b-3595d632a4fb@arm.com>
Date: Mon, 6 Sep 2021 11:09:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <24ba08cc-acd4-c64a-1e90-dc4c8efbfe48@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Julien,

On 06.09.2021 11:07, Julien Grall wrote:
> Hi Michal,
> 
> On 29/07/2021 11:42, Michal Orzel wrote:
>> According to ARMv8A architecture, AArch64 registers
>> are 64bit wide even though in many cases the upper
>> 32bit is reserved. Therefore there is no need for
>> function vreg_emulate_sysreg32 on arm64. This means
>> that we can have just one function vreg_emulate_sysreg
>> using new function pointer:
>> typedef bool (*vreg_reg_fn_t)(struct cpu_user_regs *regs,
>>                                register_t *r, bool read);
>>
>> Modify vreg_emulate_cp32 to use the new function pointer
>> as well.
>>
>> This change allows to properly use 64bit registers in AArch64
>> state and in case of AArch32 the upper 32 bits of AArch64
>> registers are inaccessible and are ignored(D1.20.1 ARM DDI 0487A.j).
>>
>> Signed-off-by: Michal Orzel <michal.orzel@arm.com>
>> ---
>>   xen/arch/arm/arm64/vsysreg.c    |  2 +-
>>   xen/arch/arm/vcpreg.c           | 16 ++++++++++----
>>   xen/arch/arm/vgic-v3.c          |  2 +-
>>   xen/arch/arm/vtimer.c           | 11 +++++-----
>>   xen/include/asm-arm/processor.h |  4 ++--
>>   xen/include/asm-arm/vreg.h      | 38 ++++++---------------------------
>>   6 files changed, 29 insertions(+), 44 deletions(-)
>>
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index caf17174b8..73fa2ca9ae 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -64,7 +64,7 @@ TVM_REG(CONTEXTIDR_EL1)
>>       {                                                                   \
>>           bool res;                                                       \
>>                                                                           \
>> -        res = vreg_emulate_sysreg64(regs, hsr, vreg_emulate_##reg);     \
>> +        res = vreg_emulate_sysreg(regs, hsr, vreg_emulate_##reg);       \
>>           ASSERT(res);                                                    \
>>           break;                                                          \
>>       }
>> diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
>> index e3ce56d875..be1ec08159 100644
>> --- a/xen/arch/arm/vcpreg.c
>> +++ b/xen/arch/arm/vcpreg.c
>> @@ -57,9 +57,17 @@
>>   #define WRITE_SYSREG_SZ(sz, val, sysreg...)  WRITE_SYSREG##sz(val, sysreg)
>>   #endif
>>   +/*
>> + * type32_t is defined as register_t due to the vreg_emulate_cp32 and
>> + * vreg_emulate_sysreg taking function pointer with register_t type used for
>> + * passing register's value.
>> + */
>> +typedef register_t type32_t;
>> +typedef uint64_t type64_t ;
> 
> NIT: spurious space before ;.
> 
>> +
>>   /* The name is passed from the upper macro to workaround macro expansion. */
>>   #define TVM_REG(sz, func, reg...)                                           \
>> -static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read)    \
>> +static bool func(struct cpu_user_regs *regs, type##sz##_t *r, bool read)    \
>>   {                                                                           \
>>       struct vcpu *v = current;                                               \
>>       bool cache_enabled = vcpu_has_cache_enabled(v);                         \
>> @@ -83,7 +91,7 @@ static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read)    \
>>     #else /* CONFIG_ARM_64 */
>>   #define TVM_REG32_COMBINED(lowreg, hireg, xreg)                             \
>> -static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r,    \
>> +static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, register_t *r,  \
>>                                   bool read, bool hi)                         \
>>   {                                                                           \
>>       struct vcpu *v = current;                                               \
>> @@ -108,13 +116,13 @@ static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r,    \
>>       return true;                                                            \
>>   }                                                                           \
>>                                                                               \
>> -static bool vreg_emulate_##lowreg(struct cpu_user_regs *regs, uint32_t *r,  \
>> +static bool vreg_emulate_##lowreg(struct cpu_user_regs *regs, register_t *r,\
>>                                     bool read)                                \
>>   {                                                                           \
>>       return vreg_emulate_##xreg(regs, r, read, false);                       \
>>   }                                                                           \
>>                                                                               \
>> -static bool vreg_emulate_##hireg(struct cpu_user_regs *regs, uint32_t *r,   \
>> +static bool vreg_emulate_##hireg(struct cpu_user_regs *regs, register_t *r, \
>>                                    bool read)                                 \
>>   {                                                                           \
>>       return vreg_emulate_##xreg(regs, r, read, true);                        \
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 613f37abab..cb5a70c42e 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -1531,7 +1531,7 @@ static bool vgic_v3_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
>>       switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
>>       {
>>       case HSR_SYSREG_ICC_SGI1R_EL1:
>> -        return vreg_emulate_sysreg64(regs, hsr, vgic_v3_emulate_sgi1r);
>> +        return vreg_emulate_sysreg(regs, hsr, vgic_v3_emulate_sgi1r);
>>         default:
>>           return false;
>> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
>> index 167fc6127a..0196951af4 100644
>> --- a/xen/arch/arm/vtimer.c
>> +++ b/xen/arch/arm/vtimer.c
>> @@ -162,7 +162,8 @@ void virt_timer_restore(struct vcpu *v)
>>       WRITE_SYSREG(v->arch.virt_timer.ctl, CNTV_CTL_EL0);
>>   }
>>   -static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, uint32_t *r, bool read)
>> +static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, register_t *r,
>> +                            bool read)
>>   {
>>       struct vcpu *v = current;
>>       s_time_t expires;
>> @@ -197,7 +198,7 @@ static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, uint32_t *r, bool read)
>>       return true;
>>   }
>>   -static bool vtimer_cntp_tval(struct cpu_user_regs *regs, uint32_t *r,
>> +static bool vtimer_cntp_tval(struct cpu_user_regs *regs, register_t *r,
>>                                bool read)
>>   {
>>       struct vcpu *v = current;
>> @@ -316,11 +317,11 @@ static bool vtimer_emulate_sysreg(struct cpu_user_regs *regs, union hsr hsr)
>>       switch ( hsr.bits & HSR_SYSREG_REGS_MASK )
>>       {
>>       case HSR_SYSREG_CNTP_CTL_EL0:
>> -        return vreg_emulate_sysreg32(regs, hsr, vtimer_cntp_ctl);
>> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_ctl);
>>       case HSR_SYSREG_CNTP_TVAL_EL0:
>> -        return vreg_emulate_sysreg32(regs, hsr, vtimer_cntp_tval);
>> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_tval);
>>       case HSR_SYSREG_CNTP_CVAL_EL0:
>> -        return vreg_emulate_sysreg64(regs, hsr, vtimer_cntp_cval);
>> +        return vreg_emulate_sysreg(regs, hsr, vtimer_cntp_cval);
>>         default:
>>           return false;
>> diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
>> index 2577e9a244..2058b69447 100644
>> --- a/xen/include/asm-arm/processor.h
>> +++ b/xen/include/asm-arm/processor.h
>> @@ -484,9 +484,9 @@ extern register_t __cpu_logical_map[];
>>   #define CNTKCTL_EL1_EL0PTEN  (1u<<9) /* Expose phys timer registers to EL0 */
>>     /* Timer control registers */
>> -#define CNTx_CTL_ENABLE   (1u<<0)  /* Enable timer */
>> +#define CNTx_CTL_ENABLE   (1ul<<0)  /* Enable timer */
>>   #define CNTx_CTL_MASK     (1ul<<1)  /* Mask IRQ */
>> -#define CNTx_CTL_PENDING  (1u<<2)  /* IRQ pending */
>> +#define CNTx_CTL_PENDING  (1ul<<2)  /* IRQ pending */
> I would suggest to mention in the commit message why this is necessary. AFAICT, it is not strictly necessary because you left ctl defined as a uin32_t. So I am guessing you only keep it for hardening purpose?
> 
> If so how about adding:
> 
> "Take the opportunity to switch CNTx_CTL_* to use UL to avoid any surprise with the negation of any bits (as used in vtimer_cntp_ctl)".
> 
> The rest of the patch looks fine. So I would be happy to deal with the fixes on commit:
Please do. Thanks.
> 
> Reviewed-by: Julien Grall <jgrall@amazon.com>
> 
> Cheers,
> 
Cheers,
Michal


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:14:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:14:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179530.325833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAhl-0004dp-Fc; Mon, 06 Sep 2021 09:14:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179530.325833; Mon, 06 Sep 2021 09:14:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAhl-0004di-CK; Mon, 06 Sep 2021 09:14:17 +0000
Received: by outflank-mailman (input) for mailman id 179530;
 Mon, 06 Sep 2021 09:14:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ykgp=N4=epam.com=prvs=9883458851=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNAhj-0004dc-E4
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:14:15 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cd8057ca-0ef2-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 09:14:13 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18664TJi010446; 
 Mon, 6 Sep 2021 09:14:10 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 by mx0b-0039f301.pphosted.com with ESMTP id 3awdcp0hpa-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 06 Sep 2021 09:14:10 +0000
Received: from DB8PR03MB6329.eurprd03.prod.outlook.com (2603:10a6:10:13e::9)
 by DB9PR03MB7243.eurprd03.prod.outlook.com (2603:10a6:10:220::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 09:14:06 +0000
Received: from DB8PR03MB6329.eurprd03.prod.outlook.com
 ([fe80::14ca:2f6f:15bd:3470]) by DB8PR03MB6329.eurprd03.prod.outlook.com
 ([fe80::14ca:2f6f:15bd:3470%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 09:14:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd8057ca-0ef2-11ec-b09c-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dxJo+4xILY1PE3eZqZCVGaGq+uaQznYVxCBenToiXsKsRTzu3ugXq0n+zrztyOQ1D43m9/vCBl3ARXd1m6LHkfI/gr2y/HxeQxVAURGyh73e2bnNaHzmCEFhi10QTJ3RZtxozP/k8KClb4PET7q24ElXrLS+yEvtJBzTTlMSYSzNkdcrtGti1CN18UjuTj2k9VIk/kgrQc+LogGdo5AFsuWgd7c+1+DrjkuPnzw/jF9gg+FTNcfMCQYWiYJw37GtgMiEWhLHBPREkYJ2oOi7Je90N3tp7DREh+syRXfW5vjz2R/qJk3+UTnUZRDfsGmdMidB/TvpJuqrhNp0vnuGCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6req6beNU/YGZj4m82FSm1lkYHxl5L6WrIjISmnzq9I=;
 b=JnAad68rzEAwhiClHf6VKuhwnr2kvoYijPnUOadXRysC5mBo4JzEPgj1bkvMD4jQclsYaroXv4EzH+tAw8OqFVT817YLm+b8w8AnsmzAj8W/Mr94/1lojJtwKP6Pi6qWRqT385Jzj3EQO2/nfCxEtybAse6dWX1rQRzL0B1bIBxXICO0cRaVpV62HxiPFIBuw9/M1uK19/7SwkOD2bMws62nAyYhwjF3lTEuC/R3EkZql1g9EWjJKaaS0CPcf7NnWm6OXBK9+JDizQkb85DMiBmqBFNG26T6Ppk3r6sUiN4TvsFdZd2FZZNFkVcPlUQiktWPISkGYnhksodKgfIhIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6req6beNU/YGZj4m82FSm1lkYHxl5L6WrIjISmnzq9I=;
 b=3yrcfv8bhJlnIwF7513v/GukUl9eLosHzIoso61ax11LmBE7RF0JJRFMPxyyhJEwAK0I18aZPbshPRqk0a1GJUdkuL+EhhAyam6ujDq9+8bScOWHvzNc5o+BQQIGcxbMMRTJ3K9e49Q/8pAGQs3vi002VVS6U0SdO40E6e++iaM5r6WztUO3s9S4OSSo0Nxx24PNBD1QCHUSv3T7ndDoLYAqekcNq9yywVyQj+XUl6VXauvuEQheOzLn2V4GcZa50AaQrK1g0HgwljGXeYGGQAQjR4Xo4mcp7LmwPhE959NmvMFjFT+o7eYOcq2gAeo+1MyMBvn/U9LP3hviaxS8Pg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Topic: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Index: AQHXoJ51zxJWpxhHEEGXNqYYBqazw6uSBESAgASU4wCAAB15AIAABzeA
Date: Mon, 6 Sep 2021 09:14:06 +0000
Message-ID: <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
 <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
In-Reply-To: <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 20f18d5d-4b87-4d45-d5d6-08d97116ad77
x-ms-traffictypediagnostic: DB9PR03MB7243:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <DB9PR03MB724317053005879A689405BFE7D29@DB9PR03MB7243.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 MgKIuLBizgFk/yy+aaCHwtitXP0QG48p7ikmTbZlXsQQNRxUZVkifJzOF4cvvIEq3u8nKPbj8x7JY0aX2rbkYmKS1pa2EPpEiAUlbem2xntnEoPz50lZjeJlktH2fcUqAmZsZULCes9DMfySPoQsWyrEtdI9HtiQ/vqVdeXYNgvMncvidzT6i7SSPgtZDLr2iDS7pN/0sWMsH4vhwW5M7+anheMt3JjyQwgsCpmBiazbrrHuW/OGvEBCqUlxJbFi8fBkM8KHQPcBVNOKuetmvB8LhOF6j4ZgOul9vK6AtRQPhn/HItrEBG8nes8081E9liW9vnhylfCTJHuBfBYwqdqNtni38/nnyBquCDFCBf1CPq7hvk8KABw7S1xZtEcHRT4durDenKGTaL0j9/HXdmDNMKvGMVAsHlmaQHqxJZLXWr8EJ1nw1MIIbSVipA+vhCz2pjCXJP6hSZj3wYyGieAf4B2RHxD2ggAytG5PB8f9WnWWRNYJZ54MeBqdrbLqXgvCky1y1TyU2rLVf4AwjRS2JCM1TEkm6e6J0QSxRwYoQRnKV3/KZj25tQA2Aspyu+OAOI+Q5WbIl0BK2s4SentuH7iSeJtMUIwGn3wXIi5RN1yLnWlP77XeCAxspb7xCf2X/z3fdM63C9TjwFDiamG3wiBgD8aeOHG0fJPvW8OYKglUK+V3uxQMrqwaASITAEWPHjlaufdIoa7SfZO/QcU8RWoaWR2vMFIEB4ERtFDqx1ZfXAWHLmlCP4sbaJHd
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6329.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(346002)(376002)(366004)(8676002)(31696002)(64756008)(2906002)(66946007)(66446008)(478600001)(4326008)(76116006)(8936002)(38070700005)(91956017)(71200400001)(316002)(31686004)(54906003)(110136005)(26005)(36756003)(86362001)(53546011)(6512007)(186003)(5660300002)(122000001)(83380400001)(38100700002)(6506007)(6486002)(2616005)(66556008)(66476007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UFFudVVDT2JxYVhoTm9YUWVkU2dYWDE3aHZSLzlWRmx0cUdFRUYxOEQ5RTNG?=
 =?utf-8?B?dDJwVzJobFFmS1UvL0NzV3ZEQ1I3WkVpSFRPdkN1QXowUCt3ZWxzOCtDWG5j?=
 =?utf-8?B?em9saVgreElQY2lIdGxTL1VGT3NtWXR4NkZOcDBFaFdzeTFBemRJSWMxUHhY?=
 =?utf-8?B?dUowTkdlc2lLejAvcm1tbGMrVlpONUtTUURzUnVERlFabCtxczFhRTFUS05h?=
 =?utf-8?B?d3NmUjk3WXJocjg1Q3I4SGdwRTFET3UrV0FyUTJuQ1JFWHdWM3NieHdwOG5L?=
 =?utf-8?B?MDAwQ2pIVk1YVGFUczI1VEdJdUF1ZVV4SEhXaStBL0JnUXN6N1JCQlZuQzlm?=
 =?utf-8?B?UGk2Y0dBSm9SVXVOdTRXOVBOejFpQ0xRQW1aTFlkZWdvcTVRTjdYTnNtaGRC?=
 =?utf-8?B?WGJXREpGZWFiMDdBcmtmQ0FuQ0RCZElJUHZsdjIycWpYdnlrbENuZURYSkRR?=
 =?utf-8?B?YVdnSnFhcDRpbVV5L3UwL3JqNVgxWERaNWtUa2tCVDlVSFlXay9XWERtdlZ6?=
 =?utf-8?B?WkI5Rk82SHV3RWVtNWNqa3RuajF0M21INFIzdWpKMERGODBTUnVOckhvQ2lC?=
 =?utf-8?B?Wk52VjlkNkFFcWxtbEtYRnFqb0p1ejlUZE1xTHJUc0R6MC9BREcwWTZGb2FP?=
 =?utf-8?B?MFM2N1BLSHhYcGV3cE5kazJpZ1ppVTlLRlFHamRYeWsrRXcvbllwWmlxN1Ez?=
 =?utf-8?B?UWFYQUZXSUZlSVNLaTlBUjNHMUs3SCsyVnNjSzE1WG00VU5FNHdkMmZBRWZn?=
 =?utf-8?B?RXZxWEt6RTI2RGQ4dmREYVBoVXhnSEZ3RjJsSWEvRUpsUzhzUTBzSUJHN1Rm?=
 =?utf-8?B?S21jdTQzRnEzQ0xIYWs4ZmFYUkxZWWZoNWdjMmcreHZWSFlOQWU2cDluUDhK?=
 =?utf-8?B?NVZybEVBbWt6Q3pVcWRUQzNVOUVNV2k4WXJEUnBsK0IwdEZuWHJHemptRElp?=
 =?utf-8?B?eitoZDk2OHphSU16UkVkaTVDcU5zN1dxc3pxUjhKanhPZzV0WXNuYyt4SDc3?=
 =?utf-8?B?RXJnVlFBMGhJY2ZNNDh3cS81MndEenFsRE9MVkFHRTkya2dubFY0RXNoUXlQ?=
 =?utf-8?B?Y2JDUDJ0MjI5UUo1bnhrbTZqazZqN1Y4TDBMQmhuRWxYU25UOUd2NDBiYXZJ?=
 =?utf-8?B?QTR1Y1I4T0MwM3R4WGlBL2lOV2l3RTB1VzBDN3MrOUx0ZDVzMlppc0sveThp?=
 =?utf-8?B?VDdabEhVb0pCaUZGRG1vQ2c5dUIrdG82QnVUNzlVZXZwcUp0ZlBmU3NPTm5o?=
 =?utf-8?B?UlhBeHl3a2t0SktibEtxU2FiZXNxN3FiTFVjOG9yaDRxaXkzY2V4RGg1NWMr?=
 =?utf-8?B?enRXMkJNNVB0WjJ0cTJTMWJCWmh2TDRNd1YybHdrOEx0SnBxNjloTU54d2Ra?=
 =?utf-8?B?Rk4raGp6d0EyQ1I4NGhlQkkrMUJvQlJnbys4MGRXMFFLYXlWUHlQNUd5WTEw?=
 =?utf-8?B?ZUROZFJ0YzBxWDU2TjlJdHVXTit0WHlBWXJ0OXNQajNOcTgvZFYzYnNLL0dU?=
 =?utf-8?B?N2JkUkpxQ0VGbzIvTDU3WFpkcEp0dVV5aW44a2NtTmp0c1VXb0dPS2JhSzZp?=
 =?utf-8?B?YS9rdlNVNmNNRm9ZNGF5b1E1RWV4SU1kRUJSQVhod25mdnJhU3BCVHhYTGpa?=
 =?utf-8?B?VUJ0T0F2Qm1hejltYzh2NUxUOU9EcGhBNUtjZnlwaDk1WjZZRnAyNDY2dFpt?=
 =?utf-8?B?L1lKWXNJdWVIREJtVTVKWVRObm5IUXpwcGFXdklMWEU4Tzh4czhsbHUvaXgv?=
 =?utf-8?Q?/usaBxQ7b5SpCYvzENfJkpOaf05gUa4O8ykePfz?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C1A72FC2385BAF40A9FFB33EF81D11A8@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6329.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 20f18d5d-4b87-4d45-d5d6-08d97116ad77
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2021 09:14:06.2041
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FatD2sSPMF/bjlKo/M2LPmR1YETF4+WMxSv6B/UJFgOHlVNOznciCizzbXEMIvrD5sc7+Nq7ObaAY/Xv4ENYzZMTb3Qkl44S12ThFlK+bZHrDzduox13wDCoeMjlJ4Sy
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR03MB7243
X-Proofpoint-GUID: TRDYvTpmQRXsf1pZnLJxxhhY9FGGApxx
X-Proofpoint-ORIG-GUID: TRDYvTpmQRXsf1pZnLJxxhhY9FGGApxx
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-06_04,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 spamscore=0
 malwarescore=0 priorityscore=1501 bulkscore=0 suspectscore=0 adultscore=0
 mlxscore=0 impostorscore=0 clxscore=1015 lowpriorityscore=0 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109060059

DQpPbiAwNi4wOS4yMSAxMTo0OCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPiBPbiAwNi8wOS8yMDIx
IDA4OjAyLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEhpLCBKdWxpZW4hDQo+
DQo+IEhpIE9sZWtzYW5kciwNCj4NCj4+IE9uIDAzLjA5LjIxIDEyOjA0LCBKdWxpZW4gR3JhbGwg
d3JvdGU6DQo+Pj4gSGkgT2xla3NhbmRyLA0KPj4+DQo+Pj4gT24gMDMvMDkvMjAyMSAwOTozMywg
T2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IEZyb206IE9sZWtzYW5kciBBbmRy
dXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4NCj4+Pj4g
dlBDSSBtYXkgbWFwIGFuZCB1bm1hcCBQQ0kgZGV2aWNlIG1lbW9yeSAoQkFScykgYmVpbmcgcGFz
c2VkIHRocm91Z2ggd2hpY2gNCj4+Pj4gbWF5IHRha2UgYSBsb3Qgb2YgdGltZS4gRm9yIHRoaXMg
dGhvc2Ugb3BlcmF0aW9ucyBtYXkgYmUgZGVmZXJyZWQgdG8gYmUNCj4+Pj4gcGVyZm9ybWVkIGxh
dGVyLCBzbyB0aGF0IHRoZXkgY2FuIGJlIHNhZmVseSBwcmVlbXB0ZWQuDQo+Pj4+IFJ1biB0aGUg
Y29ycmVzcG9uZGluZyB2UENJIGNvZGUgd2hpbGUgc3dpdGNoaW5nIGEgdkNQVS4NCj4+Pg0KPj4+
IElJVUMsIHlvdSBhcmUgdGFsa2luZyBhYm91dCB0aGUgZnVuY3Rpb24gbWFwX3JhbmdlKCkgaW4g
eGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuIFRoZSBmdW5jdGlvbiBoYXMgdGhlIGZvbGxvd2luZyB0
b2RvIGZvciBBcm06DQo+Pj4NCj4+PiDCoMKgwqDCoMKgwqDCoMKgIC8qDQo+Pj4gwqDCoMKgwqDC
oMKgwqDCoMKgICogQVJNIFRPRE9zOg0KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoCAqIC0gT24gQVJN
IHdoZXRoZXIgdGhlIG1lbW9yeSBpcyBwcmVmZXRjaGFibGUgb3Igbm90IHNob3VsZCBiZSBwYXNz
ZWQNCj4+PiDCoMKgwqDCoMKgwqDCoMKgwqAgKsKgwqAgdG8gbWFwX21taW9fcmVnaW9ucyBpbiBv
cmRlciB0byBkZWNpZGUgd2hpY2ggbWVtb3J5IGF0dHJpYnV0ZXMNCj4+PiDCoMKgwqDCoMKgwqDC
oMKgwqAgKsKgwqAgc2hvdWxkIGJlIHVzZWQuDQo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgICoNCj4+
PiDCoMKgwqDCoMKgwqDCoMKgwqAgKiAtIHt1bn1tYXBfbW1pb19yZWdpb25zIGRvZXNuJ3Qgc3Vw
cG9ydCBwcmVlbXB0aW9uLg0KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoCAqLw0KPj4+DQo+Pj4gVGhp
cyBkb2Vzbid0IHNlZW0gdG8gYmUgYWRkcmVzc2VkIGluIHRoZSB0d28gc2VyaWVzIGZvciBQQ0kg
cGFzc3Rocm91Z2ggc2VudCBzbyBmYXIuIERvIHlvdSBoYXZlIGFueSBwbGFuIHRvIGhhbmRsZSBp
dD8NCj4+DQo+PiBObyBwbGFuIHlldC4NCj4+DQo+PiBBbGwgdGhlIG1hcHBpbmdzIGFyZSBoYXBw
ZW5pbmcgd2l0aCBwMm1fbW1pb19kaXJlY3RfZGV2IGFzIG9mIG5vdy4NCj4NCj4gU28gdGhpcyBh
ZGRyZXNzIHRoZSBmaXJzdCBUT0RPIGJ1dCBob3cgYWJvdXQgdGhlIHNlY29uZCBUT0RPPyBJdCBy
ZWZlcnMgdG8gdGhlIGxhY2sgb2YgcHJlZW1wdGlvbiBvbiBBcm0gYnV0IGluIHRoaXMgcGF0Y2gg
eW91IHN1Z2dlc3QgdGhlcmUgYXJlIHNvbWUgYW5kIGhlbmNlIHdlIG5lZWQgdG8gY2FsbCB2cGNp
X3Byb2Nlc3NfcGVuZGluZygpLg0KPg0KPiBGb3IgYSB0ZWNoIHByZXZpZXcsIHRoZSBsYWNrIG9m
IHByZWVtcHRpb24gd291bGQgYmUgT0suIEhvd2V2ZXIsIHRoZSBjb21taXQgbWVzc2FnZSBzaG91
bGQgYmUgdXBkYXRlZCB0byBtYWtlIGNsZWFyIHRoZXJlIGFyZSBubyBzdWNoIHByZWVtcHRpb24g
eWV0IG9yIGF2b2lkIHRvIG1lbnRpb24gaXQuDQoNCldlbGwsIHRoZSBjb21tZW50IHdhcyBub3Qg
YWRkZWQgYnkgbWUgKGJ5IFJvZ2VyIEkgZ3Vlc3MpLCBJIGp1c3Qga2VlcCBpdC4NCg0KQXMgdG8g
dGhlIHByZWVtcHRpb24gYm90aCBtYXAgYW5kIHVubWFwIGFyZSBoYXBwZW5pbmcgdmlhIHZwY2lf
cHJvY2Vzc19wZW5kaW5nLCBzbw0KDQp3aGF0IGlzIHRydWUgZm9yIG1hcCBpcyBhbHNvIHRydWUg
Zm9yIHVubWFwIHdpdGggdGhpcyByZXNwZWN0DQoNCj4NCj4+DQo+Pj4NCj4+Pj4NCj4+Pj4gU2ln
bmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hl
bmtvQGVwYW0uY29tPg0KPj4+PiAtLS0NCj4+Pj4gwqDCoCB4ZW4vYXJjaC9hcm0vdHJhcHMuYyB8
IDYgKysrKysrDQo+Pj4+IMKgwqAgMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQ0KPj4+
Pg0KPj4+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3RyYXBzLmMgYi94ZW4vYXJjaC9hcm0v
dHJhcHMuYw0KPj4+PiBpbmRleCAyMTlhYjNjM2ZiZGUuLjE1NzFmYjhhZmQwMyAxMDA2NDQNCj4+
Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3RyYXBzLmMNCj4+Pj4gKysrIGIveGVuL2FyY2gvYXJtL3Ry
YXBzLmMNCj4+Pj4gQEAgLTM0LDYgKzM0LDcgQEANCj4+Pj4gwqDCoCAjaW5jbHVkZSA8eGVuL3N5
bWJvbHMuaD4NCj4+Pj4gwqDCoCAjaW5jbHVkZSA8eGVuL3ZlcnNpb24uaD4NCj4+Pj4gwqDCoCAj
aW5jbHVkZSA8eGVuL3ZpcnR1YWxfcmVnaW9uLmg+DQo+Pj4+ICsjaW5jbHVkZSA8eGVuL3ZwY2ku
aD4NCj4+Pj4gwqDCoCDCoCAjaW5jbHVkZSA8cHVibGljL3NjaGVkLmg+DQo+Pj4+IMKgwqAgI2lu
Y2x1ZGUgPHB1YmxpYy94ZW4uaD4NCj4+Pj4gQEAgLTIzMDQsNiArMjMwNSwxMSBAQCBzdGF0aWMg
Ym9vbCBjaGVja19mb3JfdmNwdV93b3JrKHZvaWQpDQo+Pj4+IMKgwqDCoMKgwqDCoCB9DQo+Pj4+
IMKgwqAgI2VuZGlmDQo+Pj4+IMKgwqAgK8KgwqDCoCBsb2NhbF9pcnFfZW5hYmxlKCk7DQo+Pj4+
ICvCoMKgwqAgaWYgKCBoYXNfdnBjaSh2LT5kb21haW4pICYmIHZwY2lfcHJvY2Vzc19wZW5kaW5n
KHYpICkNCj4+Pg0KPj4+IExvb2tpbmcgYXQgdGhlIGNvZGUgb2YgdnBjaV9wcm9jZXNzX3BlbmRp
bmcoKSwgaXQgbG9va3MgbGlrZSB0aGVyZSBhcmUgc29tZSByZXdvcmsgdG8gZG8gZm9yIGd1ZXN0
LiBEbyB5b3UgcGxhbiB0byBoYW5kbGUgaXQgYXMgcGFydCBvZiB0aGUgdlBDSSBzZXJpZXM/DQo+
PiBZZXMsIHZQQ0kgY29kZSBpcyBoZWF2aWx5IHRvdWNoZWQgdG8gc3VwcG9ydCBndWVzdCBub24t
aWRlbnRpdHkgbWFwcGluZ3MNCj4NCj4gSSB3YXNuJ3QgcmVmZXJyaW5nIHRvIHRoZSBub24taWRl
bnRpdHkgbWFwcGluZ3MgaGVyZS4gSSB3YXMgcmVmZXJyaW5nIHRvIFRPRE9zIHN1Y2ggYXM6DQo+
DQo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLyoNCj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
ICogRklYTUU6IGluIGNhc2Ugb2YgZmFpbHVyZSByZW1vdmUgdGhlIGRldmljZSBmcm9tIHRoZSBk
b21haW4uDQo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAqIE5vdGUgdGhhdCB0aGVyZSBtaWdo
dCBzdGlsbCBiZSBsZWZ0b3ZlciBtYXBwaW5ncy4gV2hpbGUgdGhpcyBpcw0KPiDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgKiBzYWZlIGZvciBEb20wLCBmb3IgRG9tVXMgdGhlIGRvbWFpbiB3aWxs
IGxpa2VseSBuZWVkIHRvIGJlDQo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAqIGtpbGxlZCBp
biBvcmRlciB0byBhdm9pZCBsZWFraW5nIHN0YWxlIHAybSBtYXBwaW5ncyBvbg0KPiDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgKiBmYWlsdXJlLg0KPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
Ki8NCj4NCj4gWW91IHN0aWxsIGhhdmUgdGhlbSBhZnRlciB0aGUgc2VyaWVzIHJld29ya2luZyB0
aGUgdlBDSS4gQXMgZm9yIHRoZSBwcmVlbXB0aW9uIHRoaXMgaXMgT0sgdG8gaWdub3JlIGl0IGZv
ciBhIHRlY2ggcHJldmlldy4gQWx0aG91Z2gsIHdlIHdhbnQgdG8gYXQgbGVhc3QgdHJhY2sgdGhl
bS4NClBsZWFzZSBzZWUgYWJvdmU6IGJvdGggbWFwIGFuZCB1bm1hcCBhcmUgaGFwcGVuaW5nIHZp
YSB2cGNpX3Byb2Nlc3NfcGVuZGluZw0KPg0KPj4+DQo+Pj4+ICsgcmFpc2Vfc29mdGlycShTQ0hF
RFVMRV9TT0ZUSVJRKTsNCj4+Pj4gK8KgwqDCoCBsb2NhbF9pcnFfZGlzYWJsZSgpOw0KPj4+PiAr
DQo+Pj4NCj4+PiDCoEZyb20gbXkgdW5kZXJzdGFuZGluZyBvZiB2Y3BpX3Byb2Nlc3NfcGVuZGlu
ZygpLiBUaGUgZnVuY3Rpb24gd2lsbCByZXR1cm4gdHJ1ZSBpZiB0aGVyZSBhcmUgbW9yZSB3b3Jr
IHRvIHNjaGVkdWxlLg0KPj4gWWVzDQo+Pj4gSG93ZXZlciwgaWYgY2hlY2tfZm9yX3ZjcHVfZm9y
X3dvcmsoKSByZXR1cm4gZmFsc2UsIHRoZW4gd2Ugd2lsbCByZXR1cm4gdG8gdGhlIGd1ZXN0IGJl
Zm9yZSBhbnkgd29yayBmb3IgdkNQSSBoYXMgZmluaXNoZWQuIFRoaXMgaXMgYmVjYXVzZSBjaGVj
a19mb3JfdmNwdV93b3JrKCkgd2lsbCBub3QgYmUgY2FsbGVkIGFnYWluLg0KPj4gQ29ycmVjdA0K
Pj4+DQo+Pj4gSW4gdGhpcyBjYXNlLCBJIHRoaW5rIHlvdSB3YW50IHRvIHJldHVybiBhcyBzb29u
IGFzIHlvdSBrbm93IHdlIG5lZWQgdG8gcmVzY2hlZHVsZS4NCj4+IE5vdCBzdXJlIEkgdW5kZXJz
dGFuZCB0aGlzDQo+DQpJIHdhcyBtb3JlIHJlZmVycmluZyB0byAiSSB0aGluayB5b3Ugd2FudCB0
byByZXR1cm4gYXMgc29vbiBhcyB5b3Uga25vdyB3ZSBuZWVkIHRvIHJlc2NoZWR1bGUuIg0KPiBU
aGUgcmV0dXJuIHZhbHVlIG9mIGNoZWNrX2Zvcl92Y3B1X2Zvcl93b3JrKCkgaW5kaWNhdGVzIHdo
ZXRoZXIgd2UgaGF2ZSBtb3JlIHdvcmsgdG8gZG8gYmVmb3JlIHJldHVybmluZyB0byByZXR1cm4g
dG8gdGhlIGd1ZXN0Lg0KPg0KPiBXaGVuIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCkgcmV0dXJucyB0
cnVlLCBpdCB0ZWxscyB1cyB3ZSBuZWVkIHRvIGNhbGwgdGhlIGZ1bmN0aW9uIGF0IGxlYXN0IG9u
ZSBtb3JlIHRpbWUgYmVmb3JlIHJldHVybmluZyB0byB0aGUgZ3Vlc3QuDQo+DQo+IEluIHlvdXIg
Y3VycmVudCBpbXBsZW1lbnRhdGlvbiwgeW91IGxlYXZlIHRoYXQgZGVjaXNpb24gdG8gd2hvZWV2
ZXIgaXMgbmV4dCBpbiB0aGUgZnVuY3Rpb24uDQo+DQo+IEl0IGlzIG5vdCBzYWZlIHRvIHJldHVy
biB0byB0aGUgZ3Vlc3QgYXMgbG9uZyBhcyB2cGNpX3Byb2Nlc3NfcGVuZGluZygpIHJldHVybnMg
dHJ1ZS4gU28geW91IHdhbnQgdG8gd3JpdGUgc29tZXRoaW5nIGxpa2U6DQo+DQo+IGlmICggdnBj
aV9wcm9jZXNzX3BlbmRpbmcoKSApDQo+IMKgIHJldHVybiB0cnVlOw0KLS0tIGEveGVuL2FyY2gv
YXJtL3RyYXBzLmMNCg0KKysrIGIveGVuL2FyY2gvYXJtL3RyYXBzLmMNCkBAIC0yMjkxLDYgKzIy
OTEsOSBAQCBzdGF0aWMgYm9vbCBjaGVja19mb3JfdmNwdV93b3JrKHZvaWQpDQogwqB7DQogwqDC
oMKgwqAgc3RydWN0IHZjcHUgKnYgPSBjdXJyZW50Ow0KDQorwqDCoMKgIGlmICggdnBjaV9wcm9j
ZXNzX3BlbmRpbmcoKSApDQorwqDCoMKgwqDCoCByZXR1cm4gdHJ1ZTsNCisNCiDCoCNpZmRlZiBD
T05GSUdfSU9SRVFfU0VSVkVSDQogwqDCoMKgwqAgaWYgKCBkb21haW5faGFzX2lvcmVxX3NlcnZl
cih2LT5kb21haW4pICkNCiDCoMKgwqDCoCB7DQpEbyB5b3UgbWVhbiBzb21ldGhpbmcgbGlrZSB0
aGlzPw0KDQoNCj4NCj4+Pg0KPj4+IEhvd2V2ZXIsIGxvb2tpbmcgYXQgdGhlIHJlc3Qgb2YgdGhl
IGNvZGUsIHdlIGFscmVhZHkgaGF2ZSBhIGNoZWNrIGZvciB2cGNpIGluIHRoZSBjb21tb24gSU9S
RVEgY29kZS4NCj4+DQo+PiBXaGljaCBtYXkgbm90IGJlIGVuYWJsZWQgYXMgaXQgZGVwZW5kcyBv
biBDT05GSUdfSU9SRVFfU0VSVkVSLg0KPg0KPiBSaWdodC4gTXkgcG9pbnQgaXMgd2hlbiBDT05G
SUdfSU9SRVFfU0VSVkVSIGlzIHNldCB0aGVuIHlvdSB3b3VsZCBlbmQgdXAgdG8gY2FsbCB0d2lj
ZSB2cGNpX3Byb2Nlc3NfcGVuZGluZygpLiBUaGlzIHdpbGwgaGF2ZSBhbiBpbXBhY3QgaG93IG9u
IGxvbmcgeW91ciB2Q1BVIGlzIGdvaW5nIHRvIHJ1bm5pbmcgYmVjYXVzZSB5b3UgYXJlIGRvdWJs
aW5nIHRoZSB3b3JrLg0KDQpTbywgeW91IHN1Z2dlc3QgdGhhdCB3ZSBoYXZlIGluIHRoZSBjb21t
b24gSU9SRVEgY29kZSBzb21ldGhpbmcgY2FsbCBsaWtlDQoNCmFyY2hfdnBjaV9wcm9jZXNzX3Bl
bmRpbmc/IEluIGNhc2Ugb2YgeDg2IGl0IHdpbGwgaGF2ZSB0aGUgY29kZSBjdXJyZW50bHkgZm91
bmQgaW4gdGhlDQoNCmNvbW1vbiBJT1JFUSBzb3VyY2VzIGFuZCBmb3IgQXJtIGl0IHdpbGwgYmUg
bm9wPw0KDQpBbnkgYmV0dGVyIHN1Z2dlc3Rpb24gZm9yIHRoZSBuYW1lPw0KDQo+DQo+Pg0KPj4g
TXkgdW5kZXJzdGFuZGluZyBpcyB0aGF0IGZvciB4ODYgaXQgaXMgYWx3YXlzIGVuYWJsZWQsIGJ1
dCB0aGlzIG1pZ2h0IG5vdCBiZSB0aGUgY2FzZSBmb3IgQXJtDQo+Pg0KPj4+IFNvIHdlIHdvdWxk
IGVuZCB1cCB0byBjYWxsIHR3aWNlIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCkuDQo+PiBTbywgaWYg
Q09ORklHX0lPUkVRX1NFUlZFUiBpcyBub3QgZW5hYmxlZCB0aGVuIHdlIGVuZCB1cCB3aXRoIG9u
bHkgY2FsbGluZyBpdCBmcm9tIHRyYXBzLmMgb24gQXJtDQo+Pj4gTWF5YmUgd2Ugc2hvdWxkIG1v
dmUgdGhlIGNhbGwgZnJvbSB0aGUgSU9SRVEgdG8gYXJjaC1jb2RlLg0KPj4NCj4+IEhtLiBJIHdv
dWxkIGJldHRlciB0aGluayBvZiBtb3ZpbmcgaXQgZnJvbSBJT1JFUSB0byBzb21lIG90aGVyIGNv
bW1vbiBjb2RlOiBmb3IgeDg2IChpZg0KPj4NCj4+IG15IHVuZGVyc3RhbmRpbmcgY29ycmVjdCBh
Ym91dCBDT05GSUdfSU9SRVFfU0VSVkVSKSBpdCBpcyBieSBjb2luY2lkZW5jZSB0aGF0IHdlIGNh
bGwgdlBDSQ0KPj4NCj4+IGNvZGUgZnJvbSB0aGVyZSBhbmQgSU9SRVEgaXMgYWx3YXlzIGVuYWJs
ZWQuDQo+DQo+IEkgYW0gbm90IGF3YXJlIG9mIGFub3RoZXIgc3VpdGFibGUgY29tbW9uIGhlbHBl
ciB0aGF0IHdvdWxkIGJlIGNhbGxlZCBvbiB0aGUgcmV0dXJuIHRvIHRoZSBndWVzdCBwYXRoLiBI
ZW5jZSB3aHkgSSBzdWdnZXN0IHRvIHBvc3NpYmx5IGR1cGxpY2F0ZWQgdGhlIGNvZGUgaW4gZWFj
aCBhcmNoIHBhdGguDQpJIHNlZQ0KPg0KPiBDaGVlcnMsDQo+


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:28:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:28:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179538.325844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAvg-0006PB-Sc; Mon, 06 Sep 2021 09:28:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179538.325844; Mon, 06 Sep 2021 09:28:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNAvg-0006P4-Oe; Mon, 06 Sep 2021 09:28:40 +0000
Received: by outflank-mailman (input) for mailman id 179538;
 Mon, 06 Sep 2021 09:28:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNAvg-0006Oy-AQ
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:28:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4d9ce224-eeba-4533-aa2a-cb5e95a60e95;
 Mon, 06 Sep 2021 09:28:38 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C14F82211D;
 Mon,  6 Sep 2021 09:28:37 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 8FC3B13942;
 Mon,  6 Sep 2021 09:28:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id RNmoIUXfNWGLfQAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 09:28:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d9ce224-eeba-4533-aa2a-cb5e95a60e95
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630920517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=Fnt3pUFGD23kUHjHSeZH4LhPgNOomRuK/bJBrzAtC5U=;
	b=GqlRiGZ133bWZD5NWHgpAcBhkIHJJU1TzARdyNIbPiTDYF2U3oDad5bflg1wN1E9ehH8lA
	0QTKJq1oM/aB8+r7Rqh+d6mH0xTCUlVhNsNMSagGY6x7/W8SzxAl028c154TgK0tEebBqE
	KqZplS48RWusqytsxiIZcJz6RkuwixI=
Subject: Re: Crash when using cpupools
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
 <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
Date: Mon, 6 Sep 2021 11:28:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="twv2WvE7CpSkqTNbG1UY0sa9JKUBnNFJl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--twv2WvE7CpSkqTNbG1UY0sa9JKUBnNFJl
Content-Type: multipart/mixed; boundary="gZGi62OkMEfsZvlf2YOBXBajlHysrdJnd";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
Message-ID: <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
Subject: Re: Crash when using cpupools
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
 <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
In-Reply-To: <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>

--gZGi62OkMEfsZvlf2YOBXBajlHysrdJnd
Content-Type: multipart/mixed;
 boundary="------------7D5E77FB0A544101293057AF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7D5E77FB0A544101293057AF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 10:46, Andrew Cooper wrote:
> On 06/09/2021 09:23, Juergen Gross wrote:
>> On 03.09.21 17:41, Bertrand Marquis wrote:
>>> Hi,
>>>
>>> While doing some investigation with cpupools I encountered a crash
>>> when trying to isolate a guest to its own physical cpu.
>>>
>>> I am using current staging status.
>>>
>>> I did the following (on FVP with 8 cores):
>>> - start dom0 with dom0_max_vcpus=3D1
>>> - remove core 1 from dom0 cpupool: xl cpupool-cpu-remove Pool-0 1
>>> - create a new pool: xl cpupool-create name=3D\"NetPool\=E2=80=9D
>>> - add core 1 to the pool: xl cpupool-cpu-add NetPool 1
>>> - create a guest in NetPool using the following in the guest config:
>>> pool=3D=E2=80=9CNetPool"
>>>
>>> I end with a crash with the following call trace during guest creatio=
n:
>>> (XEN) Xen call trace:
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234cb0>] credit2.c#csched2_alloc_=
udata+0x58/0xfc
>>> (PC)
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234c80>] credit2.c#csched2_alloc_=
udata+0x28/0xfc
>>> (LR)
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000242d38>] sched_move_domain+0x144/=
0x6c0
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022dd18>]
>>> cpupool.c#cpupool_move_domain_locked+0x38/0x70
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022fadc>] cpupool_do_sysctl+0x73c/=
0x780
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022d8e0>] do_sysctl+0x788/0xa58
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000273b68>] traps.c#do_trap_hypercal=
l+0x78/0x170
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000274f70>] do_trap_guest_sync+0x138=
/0x618
>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000260458>] entry.o#guest_sync_slowp=
ath+0xa4/0xd4
>>>
>>> After some debugging I found out that unit->vcpu_list is NULL after
>>> unit->vcpu_list =3D d->vcpu[unit->unit_id]; with unit_id 0 in
>>> common/sched/core.c:688
>>> This makes the call to is_idle_unit(unit) in csched2_alloc_udata
>>> trigger the crash.
>>
>> So there is no vcpu 0 in that domain? How is this possible?
>=20
> Easy, depending on the order of hypercalls issued by the toolstack.
>=20
> Between DOMCTL_createdomain and DOMCTL_max_vcpus, the domain exists but=

> the vcpus haven't been allocated.

Oh yes, indeed.

Bertrand, does the attached patch fix the issue for you?


Juergen

--------------7D5E77FB0A544101293057AF
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-xen-sched-fix-sched_move_domain-for-domain-without-v.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename*0="0001-xen-sched-fix-sched_move_domain-for-domain-without-v.pa";
 filename*1="tch"

=46rom 82af7d22a69a8cac633a6b2a40bc7d52dac5c5e8 Mon Sep 17 00:00:00 2001
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>
Cc: Dario Faggioli <dfaggioli@suse.com>
Date: Mon, 6 Sep 2021 11:19:12 +0200
Subject: [PATCH] xen/sched: fix sched_move_domain() for domain without vc=
pus

In case a domain is created with a cpupool other than Pool-0 specified
it will be moved to that cpupool before any vcpus are allocated.

This will lead to a NULL pointer dereference in sched_move_domain().

Fix that by tolerating vcpus not being allocated yet.

Fixes: 61649709421a5a7c1 ("xen/domain: Allocate d->vcpu[] in domain_creat=
e()")
Reported-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/common/sched/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8d178baf3d..79c9100680 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cpupo=
ol *c)
=20
     for ( unit_idx =3D 0; unit_idx < n_units; unit_idx++ )
     {
+        /* Special case for move at domain creation time. */
+        if ( !d->vcpu[unit_idx * gran] )
+            break;
+
         unit =3D sched_alloc_unit_mem();
         if ( unit )
         {
--=20
2.26.2


--------------7D5E77FB0A544101293057AF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7D5E77FB0A544101293057AF--

--gZGi62OkMEfsZvlf2YOBXBajlHysrdJnd--

--twv2WvE7CpSkqTNbG1UY0sa9JKUBnNFJl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE130UFAwAAAAAACgkQsN6d1ii/Ey+V
MQf/ZpNqqtNsdtSCVkQjFcgLGaa53spI7ht9azXcuSTf6KFCXHaeHsQ136jHVRpL/kTUVylGcaZA
xSFgHZoIEzecLFgZxD3CxD4yRKjMm/o8CgYeV+Yljvdyn/2TWE9c61iTcbO7t5waMG/D7rmpfQqr
YFpl1NvfhuzEK46Fi2jk57anyKsVFHRZnRPykNjwz80+dJ4i1V7FNgP3xiR7jsSslvAqBOkV8unK
2VzVmnxT0LTTDtPT5V3QyQmpqNe0KEhEVZmLRLL84wD36IyiIh1TtpOVV8trPt/2oRrt2zwBux3a
Ec5T6/pQBsa2estR75ahTNejHe8VDwlWIhRI+DGUZw==
=z1sL
-----END PGP SIGNATURE-----

--twv2WvE7CpSkqTNbG1UY0sa9JKUBnNFJl--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:46:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179544.325855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBCn-0000aj-Ds; Mon, 06 Sep 2021 09:46:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179544.325855; Mon, 06 Sep 2021 09:46:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBCn-0000ac-8o; Mon, 06 Sep 2021 09:46:21 +0000
Received: by outflank-mailman (input) for mailman id 179544;
 Mon, 06 Sep 2021 09:46:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNBCl-0000aW-Rh
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:46:19 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 481c6971-0ef7-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 09:46:18 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 1CAF4200AF;
 Mon,  6 Sep 2021 09:46:17 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id DD1911363C;
 Mon,  6 Sep 2021 09:46:16 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id YrZZNGjjNWEMBAAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 09:46:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 481c6971-0ef7-11ec-b09c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630921577; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=T2wXaDRUM1flFstth7VtHRzmPEa+revKjFjRGmU2dE8=;
	b=XnnHBEXw5n3TXEe3oGUoKmdCZa8R5XDwq160rFXTUcOJ1qGfJPwHy+Q05UXWv1ea3dyNAV
	yZdIfR3zsfzdqWO6a1C0xNa7FZMLhpLHkOWekAUaMmgU3DhgEm5ouQXS9L6iygmC6v/a7N
	gPTsZC3jwvk6dF02CuUU0qFKB9VFbhY=
Subject: Re: Crash when using cpupools
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
 <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
 <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
Message-ID: <01dd62b1-db4c-d8a1-b9b6-f80e4fb403cf@suse.com>
Date: Mon, 6 Sep 2021 11:46:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="1LWtuK8YRW5IouvXB1ISMO4sEVi2t9yAk"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--1LWtuK8YRW5IouvXB1ISMO4sEVi2t9yAk
Content-Type: multipart/mixed; boundary="AFZA3OYofPA9zSLtZxcJqtD5axn1XBhqR";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
Message-ID: <01dd62b1-db4c-d8a1-b9b6-f80e4fb403cf@suse.com>
Subject: Re: Crash when using cpupools
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
 <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
 <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
In-Reply-To: <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>

--AFZA3OYofPA9zSLtZxcJqtD5axn1XBhqR
Content-Type: multipart/mixed;
 boundary="------------8D0B8B444F64EA343F4320B7"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8D0B8B444F64EA343F4320B7
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 11:28, Juergen Gross wrote:
> On 06.09.21 10:46, Andrew Cooper wrote:
>> On 06/09/2021 09:23, Juergen Gross wrote:
>>> On 03.09.21 17:41, Bertrand Marquis wrote:
>>>> Hi,
>>>>
>>>> While doing some investigation with cpupools I encountered a crash
>>>> when trying to isolate a guest to its own physical cpu.
>>>>
>>>> I am using current staging status.
>>>>
>>>> I did the following (on FVP with 8 cores):
>>>> - start dom0 with dom0_max_vcpus=3D1
>>>> - remove core 1 from dom0 cpupool: xl cpupool-cpu-remove Pool-0 1
>>>> - create a new pool: xl cpupool-create name=3D\"NetPool\=E2=80=9D
>>>> - add core 1 to the pool: xl cpupool-cpu-add NetPool 1
>>>> - create a guest in NetPool using the following in the guest config:=

>>>> pool=3D=E2=80=9CNetPool"
>>>>
>>>> I end with a crash with the following call trace during guest creati=
on:
>>>> (XEN) Xen call trace:
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234cb0>] credit2.c#csched2_alloc=
_udata+0x58/0xfc
>>>> (PC)
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000234c80>] credit2.c#csched2_alloc=
_udata+0x28/0xfc
>>>> (LR)
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000242d38>] sched_move_domain+0x144=
/0x6c0
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022dd18>]
>>>> cpupool.c#cpupool_move_domain_locked+0x38/0x70
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022fadc>] cpupool_do_sysctl+0x73c=
/0x780
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<000000000022d8e0>] do_sysctl+0x788/0xa58
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000273b68>] traps.c#do_trap_hyperca=
ll+0x78/0x170
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000274f70>] do_trap_guest_sync+0x13=
8/0x618
>>>> (XEN)=C2=A0=C2=A0=C2=A0 [<0000000000260458>] entry.o#guest_sync_slow=
path+0xa4/0xd4
>>>>
>>>> After some debugging I found out that unit->vcpu_list is NULL after
>>>> unit->vcpu_list =3D d->vcpu[unit->unit_id]; with unit_id 0 in
>>>> common/sched/core.c:688
>>>> This makes the call to is_idle_unit(unit) in csched2_alloc_udata
>>>> trigger the crash.
>>>
>>> So there is no vcpu 0 in that domain? How is this possible?
>>
>> Easy, depending on the order of hypercalls issued by the toolstack.
>>
>> Between DOMCTL_createdomain and DOMCTL_max_vcpus, the domain exists bu=
t
>> the vcpus haven't been allocated.
>=20
> Oh yes, indeed.
>=20
> Bertrand, does the attached patch fix the issue for you?

Oh, just realized that the "Fixes:" tag is referencing the wrong commit,
that should be 70fadc41635b9b62


Juergen


--------------8D0B8B444F64EA343F4320B7
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8D0B8B444F64EA343F4320B7--

--AFZA3OYofPA9zSLtZxcJqtD5axn1XBhqR--

--1LWtuK8YRW5IouvXB1ISMO4sEVi2t9yAk
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE142gFAwAAAAAACgkQsN6d1ii/Ey+2
kgf9GzRyUFgrBGLj3kmwmJhHWBS8/JoO1QBsEys4PQ70rX5qLoXjR5babB02MFLlJhkmDWwFEr9S
RHdD3gvLku4HU9aOgdaCIRK/Qt5wRiokVGlcKoQfK9g3mEt+7ITL+RCOQpxpeYD9rsM5iV+JMham
uxJsCZLffXD5EAdUM1BPc4lOEd87KUZaDiWZ2sH/XTPQzntPC7kvGWLCK6Y7FoUs7vrpptv5bU5r
WCqNPt5i81PF1K7qho0qfmSErIso8uBaJvuo95OZ8GUeQL/6Vg0n6Hkl/ZTsQsIP18WFY/tVZpaX
FVKPwO5RjfU3LyqozS24xVSf0Gr7NudxVZwrDiVw2w==
=aZBU
-----END PGP SIGNATURE-----

--1LWtuK8YRW5IouvXB1ISMO4sEVi2t9yAk--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 09:54:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 09:54:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179550.325865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBKH-0002Dc-7B; Mon, 06 Sep 2021 09:54:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179550.325865; Mon, 06 Sep 2021 09:54:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBKH-0002DV-3A; Mon, 06 Sep 2021 09:54:05 +0000
Received: by outflank-mailman (input) for mailman id 179550;
 Mon, 06 Sep 2021 09:54:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNBKF-0002DP-KK
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 09:54:03 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNBKD-0000ne-9L; Mon, 06 Sep 2021 09:54:01 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNBKD-0008FJ-2H; Mon, 06 Sep 2021 09:54:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=WRQ1tyZJg50eo3tR13+7InnQQ2Ofz1ksNdc1tiT75eA=; b=Vl8ZlsqiotCLob4Ffdqc+siYQp
	gpz1UCO4z6kxuKSXKOTV4sxakDsH7ZZM0N4hF/FeDnnPQ1p/7e3SsIGBC1FcG0RLDmzdoMGdD8AX6
	wlvW3l8WvhBrnKLj6zmsSfR5T0U4Zx7aEI5X0MBCFaDxeXIZaFSi89G/me1o1wjNqqxM=;
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
 <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
 <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <f950a37d-2773-fce5-9e77-163a64925d7d@xen.org>
Date: Mon, 6 Sep 2021 10:53:58 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Oleksandr,

On 06/09/2021 10:14, Oleksandr Andrushchenko wrote:
> 
> On 06.09.21 11:48, Julien Grall wrote:
>> On 06/09/2021 08:02, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>
>> Hi Oleksandr,
>>
>>> On 03.09.21 12:04, Julien Grall wrote:
>>>> Hi Oleksandr,
>>>>
>>>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> vPCI may map and unmap PCI device memory (BARs) being passed through which
>>>>> may take a lot of time. For this those operations may be deferred to be
>>>>> performed later, so that they can be safely preempted.
>>>>> Run the corresponding vPCI code while switching a vCPU.
>>>>
>>>> IIUC, you are talking about the function map_range() in xen/drivers/vpci/header. The function has the following todo for Arm:
>>>>
>>>>           /*
>>>>            * ARM TODOs:
>>>>            * - On ARM whether the memory is prefetchable or not should be passed
>>>>            *   to map_mmio_regions in order to decide which memory attributes
>>>>            *   should be used.
>>>>            *
>>>>            * - {un}map_mmio_regions doesn't support preemption.
>>>>            */
>>>>
>>>> This doesn't seem to be addressed in the two series for PCI passthrough sent so far. Do you have any plan to handle it?
>>>
>>> No plan yet.
>>>
>>> All the mappings are happening with p2m_mmio_direct_dev as of now.
>>
>> So this address the first TODO but how about the second TODO? It refers to the lack of preemption on Arm but in this patch you suggest there are some and hence we need to call vpci_process_pending().
>>
>> For a tech preview, the lack of preemption would be OK. However, the commit message should be updated to make clear there are no such preemption yet or avoid to mention it.
> 
> Well, the comment was not added by me (by Roger I guess), I just keep it.

I don't think it matters to know who added it. What matters is when 
those comments are going to be handled. If they are already handled, 
then they should be dropped.

If they are not, the two TODOs listed above are probably OK to defer as 
you only plan a tech preview. But they would need to be handled before 
vCPI is selected by default and used in production.

Note that I specifically wrote "the two TODOs listed above" because I 
haven't looked at the other TODOs/FIXMEs and figued out they are fine to 
defer.

> 
> As to the preemption both map and unmap are happening via vpci_process_pending, so

Right... this doesn't mean preemption is actually supported on Arm. 
vpci_process_pending() doesn't do the preemption itself. It relies on 
map_range() to do it.

But even map_range() relies on the arch specific helper 
{,un}map_mmio_regions() to do it. If you look at the x86 implementation 
they are adding at max MAX_MMIO_MAX_ITER entries per call. On Arm, there 
are not such limit. Therefore the function will always do the full 
{,un}mapping before returning. IOW there are no preemption supported.

> 
> what is true for map is also true for unmap with this respect
> 
>>
>>>
>>>>
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>> ---
>>>>>     xen/arch/arm/traps.c | 6 ++++++
>>>>>     1 file changed, 6 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>>>>> index 219ab3c3fbde..1571fb8afd03 100644
>>>>> --- a/xen/arch/arm/traps.c
>>>>> +++ b/xen/arch/arm/traps.c
>>>>> @@ -34,6 +34,7 @@
>>>>>     #include <xen/symbols.h>
>>>>>     #include <xen/version.h>
>>>>>     #include <xen/virtual_region.h>
>>>>> +#include <xen/vpci.h>
>>>>>       #include <public/sched.h>
>>>>>     #include <public/xen.h>
>>>>> @@ -2304,6 +2305,11 @@ static bool check_for_vcpu_work(void)
>>>>>         }
>>>>>     #endif
>>>>>     +    local_irq_enable();
>>>>> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )
>>>>
>>>> Looking at the code of vpci_process_pending(), it looks like there are some rework to do for guest. Do you plan to handle it as part of the vPCI series?
>>> Yes, vPCI code is heavily touched to support guest non-identity mappings
>>
>> I wasn't referring to the non-identity mappings here. I was referring to TODOs such as:
>>
>>              /*
>>               * FIXME: in case of failure remove the device from the domain.
>>               * Note that there might still be leftover mappings. While this is
>>               * safe for Dom0, for DomUs the domain will likely need to be
>>               * killed in order to avoid leaking stale p2m mappings on
>>               * failure.
>>               */
>>
>> You still have them after the series reworking the vPCI. As for the preemption this is OK to ignore it for a tech preview. Although, we want to at least track them.
> Please see above: both map and unmap are happening via vpci_process_pending

I am not sure how this is relevant to what I just mentionned.

>>
>>>>
>>>>> + raise_softirq(SCHEDULE_SOFTIRQ);
>>>>> +    local_irq_disable();
>>>>> +
>>>>
>>>>   From my understanding of vcpi_process_pending(). The function will return true if there are more work to schedule.
>>> Yes
>>>> However, if check_for_vcpu_for_work() return false, then we will return to the guest before any work for vCPI has finished. This is because check_for_vcpu_work() will not be called again.
>>> Correct
>>>>
>>>> In this case, I think you want to return as soon as you know we need to reschedule.
>>> Not sure I understand this
>>
> I was more referring to "I think you want to return as soon as you know we need to reschedule."
>> The return value of check_for_vcpu_for_work() indicates whether we have more work to do before returning to return to the guest.
>>
>> When vpci_process_pending() returns true, it tells us we need to call the function at least one more time before returning to the guest.
>>
>> In your current implementation, you leave that decision to whoeever is next in the function.
>>
>> It is not safe to return to the guest as long as vpci_process_pending() returns true. So you want to write something like:
>>
>> if ( vpci_process_pending() )
>>    return true;
> --- a/xen/arch/arm/traps.c
> 
> +++ b/xen/arch/arm/traps.c
> @@ -2291,6 +2291,9 @@ static bool check_for_vcpu_work(void)
>    {
>        struct vcpu *v = current;
> 
> +    if ( vpci_process_pending() )
> +      return true;
> +
>    #ifdef CONFIG_IOREQ_SERVER
>        if ( domain_has_ioreq_server(v->domain) )
>        {
> Do you mean something like this?

Yes.

>>>> However, looking at the rest of the code, we already have a check for vpci in the common IOREQ code.
>>>
>>> Which may not be enabled as it depends on CONFIG_IOREQ_SERVER.
>>
>> Right. My point is when CONFIG_IOREQ_SERVER is set then you would end up to call twice vpci_process_pending(). This will have an impact how on long your vCPU is going to running because you are doubling the work.
> 
> So, you suggest that we have in the common IOREQ code something call like
> 
> arch_vpci_process_pending? In case of x86 it will have the code currently found in the
> 
> common IOREQ sources and for Arm it will be nop?

No I am suggesting to move the call of the IOREQ code to hvm_do_resume() 
(on x86) and check_for_vcpu_work() (on Arm).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:06:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:06:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179558.325877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBVz-000424-EW; Mon, 06 Sep 2021 10:06:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179558.325877; Mon, 06 Sep 2021 10:06:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBVz-00041x-BG; Mon, 06 Sep 2021 10:06:11 +0000
Received: by outflank-mailman (input) for mailman id 179558;
 Mon, 06 Sep 2021 10:06:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Prhc=N4=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNBVx-00041r-Fx
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:06:09 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 07395f79-cf56-48d6-84eb-f99b86417c34;
 Mon, 06 Sep 2021 10:06:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07395f79-cf56-48d6-84eb-f99b86417c34
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630922765;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=I+fziuhvRpesrSNJ0c58bAv+DQLU4/GtqeroDqk51ko=;
  b=Txa8Q1FrwWSYqb/7kg10gt9OvL4/smv5v490hv3SJmlSN3zQPZEVPVLr
   Ij5zWv9IZQQ3hnx5x+IssLtSZl1cD1+/ZKdkhY7eSVssduiwm9B+lmXmg
   q71FK9u/mKlOQoYEbFlCQSpGOIALn8cjLsP/9AVcJsOSVdOtvRirMNShg
   c=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: So0UWvXWJbMpiG7Qti2/no7YCzlCG/49Hbao8u9m0kWwGa0lkGO5dvExafKxklAzj6OIWJamLg
 cp8/WRGR8mmTOnsMmWeFPRem/SIRvIqxXhQlaNtBpdnRTlGed76w2mPf2U5QL0msZnaYmANnsh
 Bciibf9T7J6C6ZsT/cv4YyO1XNErb0FJvTrKxPQSje3IBPOWRlR9upE5+nzu4vJ21rcZ9c/ywK
 CizNEtVFksmMTBKez0xygl1B+nroj0HtV5W71BMYJs8AocF9ixeEsMlfJ/LWf0DKPSYXkA3IM4
 0FJk65YWwePoNZZvCarQLjN7
X-SBRS: 5.1
X-MesageID: 52079071
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:AGu+uavt4251vxZfbGl7V1O47skDdtV00zEX/kB9WHVpmszxra
 6TddAgpHvJYVcqKRQdcL+7VZVoLUmxyXcx2/h3AV7AZniFhILLFuFfBOLZqlWKcREWtNQttp
 uIG5IObuEYZmIasS+V2maFL+o=
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52079071"
Date: Mon, 6 Sep 2021 11:06:01 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, "Tim
 Deegan" <tim@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 01/51] build: introduce cpp_flags macro
Message-ID: <YTXoCQ3SSvN4eU/C@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-2-anthony.perard@citrix.com>
 <ba3cfb06-1d89-ff62-1012-6edc2a9c5259@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ba3cfb06-1d89-ff62-1012-6edc2a9c5259@suse.com>

On Thu, Sep 02, 2021 at 12:08:58PM +0200, Jan Beulich wrote:
> On 24.08.2021 12:49, Anthony PERARD wrote:
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> albeit with a remark:
> 
> > --- a/xen/Rules.mk
> > +++ b/xen/Rules.mk
> > @@ -133,6 +133,9 @@ endif
> >  # Always build obj-bin files as binary even if they come from C source. 
> >  $(obj-bin-y): XEN_CFLAGS := $(filter-out -flto,$(XEN_CFLAGS))
> >  
> > +# To be use with $(a_flags) or $(c_flags) to produce CPP flags
> > +cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))
> 
> Afaics this has nothing to do with Linux'es cpp_flags, so what we do here
> is entirely up to us. If this is strictly intended to be used the another
> macro, wouldn't it make sense to have
> 
> cpp_flags = $(filter-out -Wa$(comma)% -flto,$($(1)))
> 
> here and then e.g. ...
> 
> > @@ -222,13 +225,13 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
> >  	$(call if_changed,obj_init_o)
> >  
> >  quiet_cmd_cpp_i_c = CPP     $@
> > -cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -MQ $@ -o $@ $<
> > +cmd_cpp_i_c = $(CPP) $(call cpp_flags,$(c_flags)) -MQ $@ -o $@ $<
> 
> ... the slightly simpler / easier to read
> 
> cmd_cpp_i_c = $(CPP) $(call cpp_flags,c_flags) -MQ $@ -o $@ $<
> 
> here?

I don't think this is better or simpler. "cpp_flags" don't need to know
the name of the variable to be useful. I think it is better to know that
"cpp_flags" act on the value of the variable rather than the variable
itself, when reading "$(call cpp_flags, $(a_flags))".

But thanks for the suggestion, and for the review,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:06:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:06:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179560.325888 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBWF-0004SN-Nm; Mon, 06 Sep 2021 10:06:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179560.325888; Mon, 06 Sep 2021 10:06:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBWF-0004SG-Kn; Mon, 06 Sep 2021 10:06:27 +0000
Received: by outflank-mailman (input) for mailman id 179560;
 Mon, 06 Sep 2021 10:06:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ykgp=N4=epam.com=prvs=9883458851=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNBWE-0004P7-Fq
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:06:26 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 17d65e08-0efa-11ec-b09c-12813bfff9fa;
 Mon, 06 Sep 2021 10:06:25 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1867BWMK005392; 
 Mon, 6 Sep 2021 10:06:22 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2053.outbound.protection.outlook.com [104.47.4.53])
 by mx0a-0039f301.pphosted.com with ESMTP id 3aweck0jve-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 06 Sep 2021 10:06:21 +0000
Received: from DB8PR03MB6329.eurprd03.prod.outlook.com (2603:10a6:10:13e::9)
 by DBBPR03MB5366.eurprd03.prod.outlook.com (2603:10a6:10:f5::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Mon, 6 Sep
 2021 10:06:19 +0000
Received: from DB8PR03MB6329.eurprd03.prod.outlook.com
 ([fe80::14ca:2f6f:15bd:3470]) by DB8PR03MB6329.eurprd03.prod.outlook.com
 ([fe80::14ca:2f6f:15bd:3470%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 10:06:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17d65e08-0efa-11ec-b09c-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nki9kBPfoIcBa6pHODa3MDHY7fGTSDiXP1DWGAPLPTRrf1MZO9rcmSUOCT+i7VenlULQNKKy9DmRGC87uwmXyGjSDN46rtJcT0Ix2NcRpD2AFfRYZ51m1DpTVR8WfeL8c3QkgLqnJ3+50iiA9yVd/+240ps0THXrpgk9JXil+fhS8Kc0dRnSbIOvlVaR+qe2wIdGhxe3kkX6ZdTo6HsP0d1NHmmHKpI/E2rdOmyxrNZQiHANaibsybuazQADGaik1qM2BsKkeXKFBlwMZ6HWG5kmEENxAPUub5Ig43kHBBbdNNLfulta9EUNCKgt0hRfwMqQJKmyz7TA+3r4Ea36VA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GlvqxbD5b03wI0GYjnBPDqAcfMokceyjdbHCD/df8wE=;
 b=X5SFxg04p/sflxga31oA/OmwDp0ia+WuoiaCMBGSdMhb513W35Kaz5TNMAYnGlUNZeH9wDzuuk1MsHSOpX3VeSUG2cq2ZIJSShAaikaCBjvOaMs3ZZeMV00xcIWK5t03QA2Au3NU6uGYoP1Ks7IqRUogBTcG1TQjxXvsqkIpwyUsxtl6SRwmMbbLP9sA2Bthc6iQTm0eKg3nom5BRt1RYetOReEKCgi45KO+gS4MSgSFrKpazNZNMdDOFXDx/PlRiq0kx+a1PF77D8jvocqEfGPFofWMR3BsY/6/hMN5JPpaMipaAE6gNS347ts2EjiZRp9NDtwRDW4JaUzkPBUdVg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GlvqxbD5b03wI0GYjnBPDqAcfMokceyjdbHCD/df8wE=;
 b=bOQ8XheyxwU4+FRV4/hXnpJr1ujFkxHLj3lj9RS+k8vYgmgRuFzQhATkLL9imaVzYld4Uws5NXLITjo64cgp1wA0/HUKY+GWdcARPUesxBRz6DBdunBLq0uhghixEuU5gNTaVKwP+uCwrPbYBh345klD0USRQfdolvww+/ehXIRisVXtjnVilQka0Z1a0OMcrfLsL+gIkItmKFfHmJp6PjN7QDudgdFkI3fMIOD9b5ZJNjyyAjJTbUyfC1WEpcQvo0wooRTsuZKS6V+WOAKHpVr4UgnSrSiMrdJap0+2KbrhcywRCiImqvc5CVVNZzAQhLFbxRsJC7Cosy7lL7MOow==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Topic: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Index: 
 AQHXoJ51zxJWpxhHEEGXNqYYBqazw6uSBESAgASU4wCAAB15AIAABzeAgAALJQCAAANyAA==
Date: Mon, 6 Sep 2021 10:06:18 +0000
Message-ID: <71e0a54f-da69-5712-f826-9c38eecfa244@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
 <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
 <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
 <f950a37d-2773-fce5-9e77-163a64925d7d@xen.org>
In-Reply-To: <f950a37d-2773-fce5-9e77-163a64925d7d@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8f6b2752-80c3-4351-2437-08d9711df8c6
x-ms-traffictypediagnostic: DBBPR03MB5366:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <DBBPR03MB536604ACC49EA83F4E241861E7D29@DBBPR03MB5366.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 rH2AO/pSY6cTQPn6n+wtAAItbeiAPYCzlKvw5fXWbGeh+Wpr+DJmjtp4EmLK4eDdQmFWvgZPSbKhTJSrrrl5lP6bFl0nk6Y7UGGYJCCvCoZRPTzSWmVPyVT1me//G8EPsntMwVdiymBS4JcLj/LnI8Ml/1kvmPb5+vErFuNa1NBPYSmgSt0HU3dZYcdebfJopIlnMDmJyXSEsUFumGHG38icPsBK/9aLSniBrwmOORW9CsTFFhq138YkenBvp6i54IXZVFQs18L+BB+rU4ZyBfZfK4Si5cORVj7Hy2NjXSa4PK+gZkQ1BwrtBjwkDbaN6ciL4fDu5e/MSOz0IoPseXfK0TTNTkN/6eA2c7zM8gTT1aBMZMgtjng9BY8VzlkBz8wWRAJ7qviJ06wyO+IqAmCLo1hea2c+OsWvVmy7//j4L+44mKOQKB1p73vOQxrJVYEMW2dJb66d4/MTgumumvl9oO3BlmHXquf1glLYAVSUNbvH9/pIRdbjqiVig1K0xBgjpZQk+T2DwXh3qjO3WtWTPem7ycrTqHSNFw3ldnqgKa4iqjr50kFr6fdq5KGDjJ1NTX+59UXN2NWk4a3Ew8H4K6DLS1AEMXndjmNYJuyskKAxQAIrpN8/VU6ScmJMw9dus9IJCGQt48ZnvoUn8wXmilkgjTBzAS88fpVxUw70C7Nu3y+vGDOT0TkVV70c6/QPnQ9ppW3aOTx4k7oDaeKE2udFOSG4K11IMwXaVG38mmsOb1G/I8H4ip5KmUtV
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6329.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(83380400001)(38070700005)(8936002)(4326008)(2616005)(86362001)(31696002)(8676002)(38100700002)(36756003)(31686004)(122000001)(2906002)(6512007)(71200400001)(6486002)(6506007)(54906003)(5660300002)(316002)(110136005)(66446008)(64756008)(66556008)(66946007)(66476007)(76116006)(91956017)(26005)(53546011)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?a0N3Rzc4eHRqdFR4WVpsR2gzT3BaUklxS0dzTitNUCt2Tnc0clVRbjRWaEN5?=
 =?utf-8?B?ekN6K3g0Y01QTVhzaC96NUdjZ1dOeEo2SHBoalkramJOSjhFWmF3ZTJQODk3?=
 =?utf-8?B?ODJmSUp4UkRXMFNKT1VYVm80MytyUUFXbHlGNGRTNG5tdnFtRDFqNzdzN2tJ?=
 =?utf-8?B?YmtFa0tzMTQ1UkhUVWtmTlJ3UWJkYlNkdlA0ME4wZ2padSt4ejBhWUl3d0gv?=
 =?utf-8?B?c2tzbkVkOVl6L1J1QW1PQVpRbGxqOGZlUXBTYjRuOHdneUxWMGtTWm1ickZ1?=
 =?utf-8?B?LzBWeDdKL0lLb1drT25IS0pzVGt3bUlweUU5c0JQZ3BFb2pHWmFycGx1Q0lj?=
 =?utf-8?B?SnMrdExldDBTcFRCa01wQ3pkN1B6UXRZMWs2bHY0U0VXMksyK1ZuVU1SU1Ni?=
 =?utf-8?B?eEp3VXp1bS8rdTB2UmdQMmZKcXFjSkpuT3l6NW5qZi9NclQvanJwQ0JwanA0?=
 =?utf-8?B?SHlDU0wySEpFc0E4RVNoRGNHUmZXNnV4TnV3QTJsRjJid3pDMk1wRlBqOWE1?=
 =?utf-8?B?ZW05UVcraHkzT2dCUEYvYXZoUUJVd2E5QkZHR2twek5RWS9PSkZWU1FaUG5X?=
 =?utf-8?B?d3k4OElvSWRUZHh2R25GNHp0S2F1SWFRMGU2NXUvTEZ3a09TRkx0bE56ZVF3?=
 =?utf-8?B?bTdGeGxmcWE4ZWFVdENtTWFHbi9idUZiTndtODRZOUYrNnRvNldqbmZGcmZq?=
 =?utf-8?B?NGMwV2NrRlNRMkJoSUtNWlp3dEMwYVp3MU04U1RKL3JQS2tCV0tVdlJXZUx1?=
 =?utf-8?B?bk52TUFpRkJ4eFZlaUswYVdiUW1HUGMrN0U2Uk5JemZ5bnZhV1BQN0FyZkdN?=
 =?utf-8?B?bTFsVlRwWk12YXJqL0crYmtwR3hHMnkyZFpLejlsaVNxRnQvNXlMczY3dmNX?=
 =?utf-8?B?WGVZTXpFYzRObEc3cmwyZlBtbXVPams1dHFlWkdwVWlFcVBCOEdhK2VJTmVT?=
 =?utf-8?B?ZzJ1ZTBrcGJPMmplaW5TRHZibmxnaGR3YUFyc3NBTkFBN0xKSis5SjBGdlVw?=
 =?utf-8?B?NHlEU25jUGZPb1RQTnoxUHNYTHh1VGJZcmt2VTFrNmpmMWpZNWNEK3JySVJm?=
 =?utf-8?B?Y1F2Zk1ZdXZ5WFFvWHFxMXF6TmFncVJ5OW9INHZBUm1MKzhhYVozOWZSemxx?=
 =?utf-8?B?Qi9objBnWlBMbCtHd2VsTFJyMHlDSmQ2OXNMZ2tCclcrQUltL3pZeThRL2lB?=
 =?utf-8?B?WTRPaHV4YktENnZJdVlnUE9LbWRTd3B4OFd0dlhTWkxhYng0OGdYNTNnaVND?=
 =?utf-8?B?US9TZTd6RzY5RmtwQVltZWx3Y0pBaW1JY0RwVlNDaGhHbHBYRHhhclFJL3py?=
 =?utf-8?B?dXhKeGhFWDJ4bDJyUFl1Qy8vUW5hSTZoVjVpdGJWVEk3WngrUkgxdURUbXNw?=
 =?utf-8?B?cmFmN3dWMVZ4aHQ4aEdRTHJaR2NSSjV3UDBkTG1ZT2xoemhoQWl2U2p1aklo?=
 =?utf-8?B?alRyRXZsdWNCSFZQMU9vTGpEem9OcDJGM2pFcjROZDhCQmhrL2xqekN4c2hD?=
 =?utf-8?B?MWxtYW93QkVMSHBNYnlLUE9pRFkzZXQrck11TzQvWnhFcTFKdkV5V3pxaGps?=
 =?utf-8?B?d0Ftb3hzaFJicGNpQ2lRbWUydTBmQXFSNUJTUEsydUZyazNYSVdlN2t4YWl0?=
 =?utf-8?B?TjlYNE1mRy9tSDBHUDNDQldhN0diM3pTcmU5YjFLNVJZMUtNR2NoRmVmOFdh?=
 =?utf-8?B?c3k0akVjRHlPUUUrNnZrTW9mcmRmUXQwdUhMRk5vdWc1VzNaR2tMbHVXWUY1?=
 =?utf-8?Q?RbQq4dqvpEw/OrG7Q4h9z5uMjfps4MGffoSKHjm?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <10D4F8D17A2DC44C89797B5ED8F2E223@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6329.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f6b2752-80c3-4351-2437-08d9711df8c6
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2021 10:06:18.9738
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +4jDX0tMJfe32pxZLPB6sD1KiNKqNNY7yTsfZBYPIKqrFrxShzxBcFSjOB8ZD03k2m9ve12KvbxOd8BcQkgwL3BSYveWGyNmvWWUfuPX3uU0BGBVr9KVqbqG69182n2t
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR03MB5366
X-Proofpoint-GUID: vkyPgN_bW_Kr5Lq7MUtFzHKsC0Ni8Sm0
X-Proofpoint-ORIG-GUID: vkyPgN_bW_Kr5Lq7MUtFzHKsC0Ni8Sm0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-06_05,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxscore=0
 mlxlogscore=999 adultscore=0 impostorscore=0 suspectscore=0
 priorityscore=1501 clxscore=1015 spamscore=0 malwarescore=0 phishscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109060064

DQpPbiAwNi4wOS4yMSAxMjo1MywgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPiBIaSBPbGVrc2FuZHIs
DQo+DQo+IE9uIDA2LzA5LzIwMjEgMTA6MTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4NCj4+IE9uIDA2LjA5LjIxIDExOjQ4LCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+Pj4gT24g
MDYvMDkvMjAyMSAwODowMiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IEhp
LCBKdWxpZW4hDQo+Pj4NCj4+PiBIaSBPbGVrc2FuZHIsDQo+Pj4NCj4+Pj4gT24gMDMuMDkuMjEg
MTI6MDQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+DQo+
Pj4+PiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToN
Cj4+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hj
aGVua29AZXBhbS5jb20+DQo+Pj4+Pj4NCj4+Pj4+PiB2UENJIG1heSBtYXAgYW5kIHVubWFwIFBD
SSBkZXZpY2UgbWVtb3J5IChCQVJzKSBiZWluZyBwYXNzZWQgdGhyb3VnaCB3aGljaA0KPj4+Pj4+
IG1heSB0YWtlIGEgbG90IG9mIHRpbWUuIEZvciB0aGlzIHRob3NlIG9wZXJhdGlvbnMgbWF5IGJl
IGRlZmVycmVkIHRvIGJlDQo+Pj4+Pj4gcGVyZm9ybWVkIGxhdGVyLCBzbyB0aGF0IHRoZXkgY2Fu
IGJlIHNhZmVseSBwcmVlbXB0ZWQuDQo+Pj4+Pj4gUnVuIHRoZSBjb3JyZXNwb25kaW5nIHZQQ0kg
Y29kZSB3aGlsZSBzd2l0Y2hpbmcgYSB2Q1BVLg0KPj4+Pj4NCj4+Pj4+IElJVUMsIHlvdSBhcmUg
dGFsa2luZyBhYm91dCB0aGUgZnVuY3Rpb24gbWFwX3JhbmdlKCkgaW4geGVuL2RyaXZlcnMvdnBj
aS9oZWFkZXIuIFRoZSBmdW5jdGlvbiBoYXMgdGhlIGZvbGxvd2luZyB0b2RvIGZvciBBcm06DQo+
Pj4+Pg0KPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgIC8qDQo+Pj4+PiDCoMKgwqDCoMKgwqDCoMKg
wqDCoCAqIEFSTSBUT0RPczoNCj4+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgICogLSBPbiBBUk0g
d2hldGhlciB0aGUgbWVtb3J5IGlzIHByZWZldGNoYWJsZSBvciBub3Qgc2hvdWxkIGJlIHBhc3Nl
ZA0KPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgKsKgwqAgdG8gbWFwX21taW9fcmVnaW9ucyBp
biBvcmRlciB0byBkZWNpZGUgd2hpY2ggbWVtb3J5IGF0dHJpYnV0ZXMNCj4+Pj4+IMKgwqDCoMKg
wqDCoMKgwqDCoMKgICrCoMKgIHNob3VsZCBiZSB1c2VkLg0KPj4+Pj4gwqDCoMKgwqDCoMKgwqDC
oMKgwqAgKg0KPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiAtIHt1bn1tYXBfbW1pb19yZWdp
b25zIGRvZXNuJ3Qgc3VwcG9ydCBwcmVlbXB0aW9uLg0KPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKg
wqAgKi8NCj4+Pj4+DQo+Pj4+PiBUaGlzIGRvZXNuJ3Qgc2VlbSB0byBiZSBhZGRyZXNzZWQgaW4g
dGhlIHR3byBzZXJpZXMgZm9yIFBDSSBwYXNzdGhyb3VnaCBzZW50IHNvIGZhci4gRG8geW91IGhh
dmUgYW55IHBsYW4gdG8gaGFuZGxlIGl0Pw0KPj4+Pg0KPj4+PiBObyBwbGFuIHlldC4NCj4+Pj4N
Cj4+Pj4gQWxsIHRoZSBtYXBwaW5ncyBhcmUgaGFwcGVuaW5nIHdpdGggcDJtX21taW9fZGlyZWN0
X2RldiBhcyBvZiBub3cuDQo+Pj4NCj4+PiBTbyB0aGlzIGFkZHJlc3MgdGhlIGZpcnN0IFRPRE8g
YnV0IGhvdyBhYm91dCB0aGUgc2Vjb25kIFRPRE8/IEl0IHJlZmVycyB0byB0aGUgbGFjayBvZiBw
cmVlbXB0aW9uIG9uIEFybSBidXQgaW4gdGhpcyBwYXRjaCB5b3Ugc3VnZ2VzdCB0aGVyZSBhcmUg
c29tZSBhbmQgaGVuY2Ugd2UgbmVlZCB0byBjYWxsIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCkuDQo+
Pj4NCj4+PiBGb3IgYSB0ZWNoIHByZXZpZXcsIHRoZSBsYWNrIG9mIHByZWVtcHRpb24gd291bGQg
YmUgT0suIEhvd2V2ZXIsIHRoZSBjb21taXQgbWVzc2FnZSBzaG91bGQgYmUgdXBkYXRlZCB0byBt
YWtlIGNsZWFyIHRoZXJlIGFyZSBubyBzdWNoIHByZWVtcHRpb24geWV0IG9yIGF2b2lkIHRvIG1l
bnRpb24gaXQuDQo+Pg0KPj4gV2VsbCwgdGhlIGNvbW1lbnQgd2FzIG5vdCBhZGRlZCBieSBtZSAo
YnkgUm9nZXIgSSBndWVzcyksIEkganVzdCBrZWVwIGl0Lg0KPg0KPiBJIGRvbid0IHRoaW5rIGl0
IG1hdHRlcnMgdG8ga25vdyB3aG8gYWRkZWQgaXQuIFdoYXQgbWF0dGVycyBpcyB3aGVuIHRob3Nl
IGNvbW1lbnRzIGFyZSBnb2luZyB0byBiZSBoYW5kbGVkLiBJZiB0aGV5IGFyZSBhbHJlYWR5IGhh
bmRsZWQsIHRoZW4gdGhleSBzaG91bGQgYmUgZHJvcHBlZC4NCj4NCj4gSWYgdGhleSBhcmUgbm90
LCB0aGUgdHdvIFRPRE9zIGxpc3RlZCBhYm92ZSBhcmUgcHJvYmFibHkgT0sgdG8gZGVmZXIgYXMg
eW91IG9ubHkgcGxhbiBhIHRlY2ggcHJldmlldy4gQnV0IHRoZXkgd291bGQgbmVlZCB0byBiZSBo
YW5kbGVkIGJlZm9yZSB2Q1BJIGlzIHNlbGVjdGVkIGJ5IGRlZmF1bHQgYW5kIHVzZWQgaW4gcHJv
ZHVjdGlvbi4NCj4NCj4gTm90ZSB0aGF0IEkgc3BlY2lmaWNhbGx5IHdyb3RlICJ0aGUgdHdvIFRP
RE9zIGxpc3RlZCBhYm92ZSIgYmVjYXVzZSBJIGhhdmVuJ3QgbG9va2VkIGF0IHRoZSBvdGhlciBU
T0RPcy9GSVhNRXMgYW5kIGZpZ3VlZCBvdXQgdGhleSBhcmUgZmluZSB0byBkZWZlci4NCk9rLCB0
aGVuIEkgbGVhdmUgdGhlIFRPRE9zIGFzIHRoZXkgYXJlDQo+DQo+Pg0KPj4gQXMgdG8gdGhlIHBy
ZWVtcHRpb24gYm90aCBtYXAgYW5kIHVubWFwIGFyZSBoYXBwZW5pbmcgdmlhIHZwY2lfcHJvY2Vz
c19wZW5kaW5nLCBzbw0KPg0KPiBSaWdodC4uLiB0aGlzIGRvZXNuJ3QgbWVhbiBwcmVlbXB0aW9u
IGlzIGFjdHVhbGx5IHN1cHBvcnRlZCBvbiBBcm0uIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCkgZG9l
c24ndCBkbyB0aGUgcHJlZW1wdGlvbiBpdHNlbGYuIEl0IHJlbGllcyBvbiBtYXBfcmFuZ2UoKSB0
byBkbyBpdC4NCj4NCj4gQnV0IGV2ZW4gbWFwX3JhbmdlKCkgcmVsaWVzIG9uIHRoZSBhcmNoIHNw
ZWNpZmljIGhlbHBlciB7LHVufW1hcF9tbWlvX3JlZ2lvbnMoKSB0byBkbyBpdC4gSWYgeW91IGxv
b2sgYXQgdGhlIHg4NiBpbXBsZW1lbnRhdGlvbiB0aGV5IGFyZSBhZGRpbmcgYXQgbWF4IE1BWF9N
TUlPX01BWF9JVEVSIGVudHJpZXMgcGVyIGNhbGwuIE9uIEFybSwgdGhlcmUgYXJlIG5vdCBzdWNo
IGxpbWl0LiBUaGVyZWZvcmUgdGhlIGZ1bmN0aW9uIHdpbGwgYWx3YXlzIGRvIHRoZSBmdWxsIHss
dW59bWFwcGluZyBiZWZvcmUgcmV0dXJuaW5nLiBJT1cgdGhlcmUgYXJlIG5vIHByZWVtcHRpb24g
c3VwcG9ydGVkLg0KT2sNCj4NCj4+DQo+PiB3aGF0IGlzIHRydWUgZm9yIG1hcCBpcyBhbHNvIHRy
dWUgZm9yIHVubWFwIHdpdGggdGhpcyByZXNwZWN0DQo+Pg0KPj4+DQo+Pj4+DQo+Pj4+Pg0KPj4+
Pj4+DQo+Pj4+Pj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtz
YW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4+Pj4+IC0tLQ0KPj4+Pj4+IMKgwqDCoCB4
ZW4vYXJjaC9hcm0vdHJhcHMuYyB8IDYgKysrKysrDQo+Pj4+Pj4gwqDCoMKgIDEgZmlsZSBjaGFu
Z2VkLCA2IGluc2VydGlvbnMoKykNCj4+Pj4+Pg0KPj4+Pj4+IGRpZmYgLS1naXQgYS94ZW4vYXJj
aC9hcm0vdHJhcHMuYyBiL3hlbi9hcmNoL2FybS90cmFwcy5jDQo+Pj4+Pj4gaW5kZXggMjE5YWIz
YzNmYmRlLi4xNTcxZmI4YWZkMDMgMTAwNjQ0DQo+Pj4+Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3Ry
YXBzLmMNCj4+Pj4+PiArKysgYi94ZW4vYXJjaC9hcm0vdHJhcHMuYw0KPj4+Pj4+IEBAIC0zNCw2
ICszNCw3IEBADQo+Pj4+Pj4gwqDCoMKgICNpbmNsdWRlIDx4ZW4vc3ltYm9scy5oPg0KPj4+Pj4+
IMKgwqDCoCAjaW5jbHVkZSA8eGVuL3ZlcnNpb24uaD4NCj4+Pj4+PiDCoMKgwqAgI2luY2x1ZGUg
PHhlbi92aXJ0dWFsX3JlZ2lvbi5oPg0KPj4+Pj4+ICsjaW5jbHVkZSA8eGVuL3ZwY2kuaD4NCj4+
Pj4+PiDCoMKgwqAgwqAgI2luY2x1ZGUgPHB1YmxpYy9zY2hlZC5oPg0KPj4+Pj4+IMKgwqDCoCAj
aW5jbHVkZSA8cHVibGljL3hlbi5oPg0KPj4+Pj4+IEBAIC0yMzA0LDYgKzIzMDUsMTEgQEAgc3Rh
dGljIGJvb2wgY2hlY2tfZm9yX3ZjcHVfd29yayh2b2lkKQ0KPj4+Pj4+IMKgwqDCoMKgwqDCoMKg
IH0NCj4+Pj4+PiDCoMKgwqAgI2VuZGlmDQo+Pj4+Pj4gwqDCoMKgICvCoMKgwqAgbG9jYWxfaXJx
X2VuYWJsZSgpOw0KPj4+Pj4+ICvCoMKgwqAgaWYgKCBoYXNfdnBjaSh2LT5kb21haW4pICYmIHZw
Y2lfcHJvY2Vzc19wZW5kaW5nKHYpICkNCj4+Pj4+DQo+Pj4+PiBMb29raW5nIGF0IHRoZSBjb2Rl
IG9mIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCksIGl0IGxvb2tzIGxpa2UgdGhlcmUgYXJlIHNvbWUg
cmV3b3JrIHRvIGRvIGZvciBndWVzdC4gRG8geW91IHBsYW4gdG8gaGFuZGxlIGl0IGFzIHBhcnQg
b2YgdGhlIHZQQ0kgc2VyaWVzPw0KPj4+PiBZZXMsIHZQQ0kgY29kZSBpcyBoZWF2aWx5IHRvdWNo
ZWQgdG8gc3VwcG9ydCBndWVzdCBub24taWRlbnRpdHkgbWFwcGluZ3MNCj4+Pg0KPj4+IEkgd2Fz
bid0IHJlZmVycmluZyB0byB0aGUgbm9uLWlkZW50aXR5IG1hcHBpbmdzIGhlcmUuIEkgd2FzIHJl
ZmVycmluZyB0byBUT0RPcyBzdWNoIGFzOg0KPj4+DQo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIC8qDQo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBGSVhNRTogaW4gY2FzZSBv
ZiBmYWlsdXJlIHJlbW92ZSB0aGUgZGV2aWNlIGZyb20gdGhlIGRvbWFpbi4NCj4+PiDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCAqIE5vdGUgdGhhdCB0aGVyZSBtaWdodCBzdGlsbCBiZSBsZWZ0
b3ZlciBtYXBwaW5ncy4gV2hpbGUgdGhpcyBpcw0KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgICogc2FmZSBmb3IgRG9tMCwgZm9yIERvbVVzIHRoZSBkb21haW4gd2lsbCBsaWtlbHkgbmVl
ZCB0byBiZQ0KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICoga2lsbGVkIGluIG9yZGVy
IHRvIGF2b2lkIGxlYWtpbmcgc3RhbGUgcDJtIG1hcHBpbmdzIG9uDQo+Pj4gwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgKiBmYWlsdXJlLg0KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
ICovDQo+Pj4NCj4+PiBZb3Ugc3RpbGwgaGF2ZSB0aGVtIGFmdGVyIHRoZSBzZXJpZXMgcmV3b3Jr
aW5nIHRoZSB2UENJLiBBcyBmb3IgdGhlIHByZWVtcHRpb24gdGhpcyBpcyBPSyB0byBpZ25vcmUg
aXQgZm9yIGEgdGVjaCBwcmV2aWV3LiBBbHRob3VnaCwgd2Ugd2FudCB0byBhdCBsZWFzdCB0cmFj
ayB0aGVtLg0KPj4gUGxlYXNlIHNlZSBhYm92ZTogYm90aCBtYXAgYW5kIHVubWFwIGFyZSBoYXBw
ZW5pbmcgdmlhIHZwY2lfcHJvY2Vzc19wZW5kaW5nDQo+DQo+IEkgYW0gbm90IHN1cmUgaG93IHRo
aXMgaXMgcmVsZXZhbnQgdG8gd2hhdCBJIGp1c3QgbWVudGlvbm5lZC4NCj4NCj4+Pg0KPj4+Pj4N
Cj4+Pj4+PiArIHJhaXNlX3NvZnRpcnEoU0NIRURVTEVfU09GVElSUSk7DQo+Pj4+Pj4gK8KgwqDC
oCBsb2NhbF9pcnFfZGlzYWJsZSgpOw0KPj4+Pj4+ICsNCj4+Pj4+DQo+Pj4+PiDCoMKgRnJvbSBt
eSB1bmRlcnN0YW5kaW5nIG9mIHZjcGlfcHJvY2Vzc19wZW5kaW5nKCkuIFRoZSBmdW5jdGlvbiB3
aWxsIHJldHVybiB0cnVlIGlmIHRoZXJlIGFyZSBtb3JlIHdvcmsgdG8gc2NoZWR1bGUuDQo+Pj4+
IFllcw0KPj4+Pj4gSG93ZXZlciwgaWYgY2hlY2tfZm9yX3ZjcHVfZm9yX3dvcmsoKSByZXR1cm4g
ZmFsc2UsIHRoZW4gd2Ugd2lsbCByZXR1cm4gdG8gdGhlIGd1ZXN0IGJlZm9yZSBhbnkgd29yayBm
b3IgdkNQSSBoYXMgZmluaXNoZWQuIFRoaXMgaXMgYmVjYXVzZSBjaGVja19mb3JfdmNwdV93b3Jr
KCkgd2lsbCBub3QgYmUgY2FsbGVkIGFnYWluLg0KPj4+PiBDb3JyZWN0DQo+Pj4+Pg0KPj4+Pj4g
SW4gdGhpcyBjYXNlLCBJIHRoaW5rIHlvdSB3YW50IHRvIHJldHVybiBhcyBzb29uIGFzIHlvdSBr
bm93IHdlIG5lZWQgdG8gcmVzY2hlZHVsZS4NCj4+Pj4gTm90IHN1cmUgSSB1bmRlcnN0YW5kIHRo
aXMNCj4+Pg0KPj4gSSB3YXMgbW9yZSByZWZlcnJpbmcgdG8gIkkgdGhpbmsgeW91IHdhbnQgdG8g
cmV0dXJuIGFzIHNvb24gYXMgeW91IGtub3cgd2UgbmVlZCB0byByZXNjaGVkdWxlLiINCj4+PiBU
aGUgcmV0dXJuIHZhbHVlIG9mIGNoZWNrX2Zvcl92Y3B1X2Zvcl93b3JrKCkgaW5kaWNhdGVzIHdo
ZXRoZXIgd2UgaGF2ZSBtb3JlIHdvcmsgdG8gZG8gYmVmb3JlIHJldHVybmluZyB0byByZXR1cm4g
dG8gdGhlIGd1ZXN0Lg0KPj4+DQo+Pj4gV2hlbiB2cGNpX3Byb2Nlc3NfcGVuZGluZygpIHJldHVy
bnMgdHJ1ZSwgaXQgdGVsbHMgdXMgd2UgbmVlZCB0byBjYWxsIHRoZSBmdW5jdGlvbiBhdCBsZWFz
dCBvbmUgbW9yZSB0aW1lIGJlZm9yZSByZXR1cm5pbmcgdG8gdGhlIGd1ZXN0Lg0KPj4+DQo+Pj4g
SW4geW91ciBjdXJyZW50IGltcGxlbWVudGF0aW9uLCB5b3UgbGVhdmUgdGhhdCBkZWNpc2lvbiB0
byB3aG9lZXZlciBpcyBuZXh0IGluIHRoZSBmdW5jdGlvbi4NCj4+Pg0KPj4+IEl0IGlzIG5vdCBz
YWZlIHRvIHJldHVybiB0byB0aGUgZ3Vlc3QgYXMgbG9uZyBhcyB2cGNpX3Byb2Nlc3NfcGVuZGlu
ZygpIHJldHVybnMgdHJ1ZS4gU28geW91IHdhbnQgdG8gd3JpdGUgc29tZXRoaW5nIGxpa2U6DQo+
Pj4NCj4+PiBpZiAoIHZwY2lfcHJvY2Vzc19wZW5kaW5nKCkgKQ0KPj4+IMKgwqAgcmV0dXJuIHRy
dWU7DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdHJhcHMuYw0KPj4NCj4+ICsrKyBiL3hlbi9hcmNo
L2FybS90cmFwcy5jDQo+PiBAQCAtMjI5MSw2ICsyMjkxLDkgQEAgc3RhdGljIGJvb2wgY2hlY2tf
Zm9yX3ZjcHVfd29yayh2b2lkKQ0KPj4gwqAgwqB7DQo+PiDCoCDCoMKgwqDCoCBzdHJ1Y3QgdmNw
dSAqdiA9IGN1cnJlbnQ7DQo+Pg0KPj4gK8KgwqDCoCBpZiAoIHZwY2lfcHJvY2Vzc19wZW5kaW5n
KCkgKQ0KPj4gK8KgwqDCoMKgwqAgcmV0dXJuIHRydWU7DQo+PiArDQo+PiDCoCDCoCNpZmRlZiBD
T05GSUdfSU9SRVFfU0VSVkVSDQo+PiDCoCDCoMKgwqDCoCBpZiAoIGRvbWFpbl9oYXNfaW9yZXFf
c2VydmVyKHYtPmRvbWFpbikgKQ0KPj4gwqAgwqDCoMKgwqAgew0KPj4gRG8geW91IG1lYW4gc29t
ZXRoaW5nIGxpa2UgdGhpcz8NCj4NCj4gWWVzLg0KT2ssIEknbGwgYWRkIHRoaXMgY2hlY2sNCj4N
Cj4+Pj4+IEhvd2V2ZXIsIGxvb2tpbmcgYXQgdGhlIHJlc3Qgb2YgdGhlIGNvZGUsIHdlIGFscmVh
ZHkgaGF2ZSBhIGNoZWNrIGZvciB2cGNpIGluIHRoZSBjb21tb24gSU9SRVEgY29kZS4NCj4+Pj4N
Cj4+Pj4gV2hpY2ggbWF5IG5vdCBiZSBlbmFibGVkIGFzIGl0IGRlcGVuZHMgb24gQ09ORklHX0lP
UkVRX1NFUlZFUi4NCj4+Pg0KPj4+IFJpZ2h0LiBNeSBwb2ludCBpcyB3aGVuIENPTkZJR19JT1JF
UV9TRVJWRVIgaXMgc2V0IHRoZW4geW91IHdvdWxkIGVuZCB1cCB0byBjYWxsIHR3aWNlIHZwY2lf
cHJvY2Vzc19wZW5kaW5nKCkuIFRoaXMgd2lsbCBoYXZlIGFuIGltcGFjdCBob3cgb24gbG9uZyB5
b3VyIHZDUFUgaXMgZ29pbmcgdG8gcnVubmluZyBiZWNhdXNlIHlvdSBhcmUgZG91YmxpbmcgdGhl
IHdvcmsuDQo+Pg0KPj4gU28sIHlvdSBzdWdnZXN0IHRoYXQgd2UgaGF2ZSBpbiB0aGUgY29tbW9u
IElPUkVRIGNvZGUgc29tZXRoaW5nIGNhbGwgbGlrZQ0KPj4NCj4+IGFyY2hfdnBjaV9wcm9jZXNz
X3BlbmRpbmc/IEluIGNhc2Ugb2YgeDg2IGl0IHdpbGwgaGF2ZSB0aGUgY29kZSBjdXJyZW50bHkg
Zm91bmQgaW4gdGhlDQo+Pg0KPj4gY29tbW9uIElPUkVRIHNvdXJjZXMgYW5kIGZvciBBcm0gaXQg
d2lsbCBiZSBub3A/DQo+DQo+IE5vIEkgYW0gc3VnZ2VzdGluZyB0byBtb3ZlIHRoZSBjYWxsIG9m
IHRoZSBJT1JFUSBjb2RlIHRvIGh2bV9kb19yZXN1bWUoKSAob24geDg2KSBhbmQgY2hlY2tfZm9y
X3ZjcHVfd29yaygpIChvbiBBcm0pLg0KDQpPaywgSSBjYW4gbW92ZSB2UENJIGNvZGUgdG8gaHZt
X2RvX3Jlc3VtZSwgYnV0IHZQQ0kgaXMgb25seSB1c2VkIGZvciB4ODYgUFZIIERvbTAuDQoNCkRv
IHlvdSBzdGlsbCB0aGluayBodm1fZG9fcmVzdW1lIGlzIHRoZSByaWdodCBwbGFjZT8NCg0KPg0K
PiBDaGVlcnMsDQo+DQpUaGFua3MsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:11:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:11:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179572.325899 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBau-0006DJ-Ay; Mon, 06 Sep 2021 10:11:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179572.325899; Mon, 06 Sep 2021 10:11:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBau-0006DC-7l; Mon, 06 Sep 2021 10:11:16 +0000
Received: by outflank-mailman (input) for mailman id 179572;
 Mon, 06 Sep 2021 10:11:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBat-000682-1A; Mon, 06 Sep 2021 10:11:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBas-0001Cp-GD; Mon, 06 Sep 2021 10:11:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBas-0001Of-6y; Mon, 06 Sep 2021 10:11:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBas-00019K-6P; Mon, 06 Sep 2021 10:11:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=MJhPdTBfKkaJ1rOsrpkR6ZFKuUZSALrBCC9jR+rOdtc=; b=J4M9uFjRnIbhh9amHbJe48klkE
	M19CGQmR4PFpHQxKcm/iqeuu7uo7LARAzVu4i9mYT36IBDbOMqJCy5+NYPX5lClTd3QchJ7QtoLiQ
	IOVF4d6YTPfMJ6x2fpLnRVP8+OcVquPyLu8kvmYCZKqiRJ+AH46rSfHJmcm6XJrHkNf0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164852-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164852: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=b6ec01b82154c9dc25f18c9c28df30d0c053defe
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 10:11:14 +0000

flight 164852 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164852/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              b6ec01b82154c9dc25f18c9c28df30d0c053defe
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  423 days
Failing since        151818  2020-07-11 04:18:52 Z  422 days  413 attempts
Testing same since   164803  2021-09-04 04:20:08 Z    2 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71803 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:14:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:14:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179580.325913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBeT-0006wQ-Vi; Mon, 06 Sep 2021 10:14:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179580.325913; Mon, 06 Sep 2021 10:14:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBeT-0006wJ-SS; Mon, 06 Sep 2021 10:14:57 +0000
Received: by outflank-mailman (input) for mailman id 179580;
 Mon, 06 Sep 2021 10:14:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kxtw=N4=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mNBeS-0006wC-0Z
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:14:56 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.71]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3dd26814-38ef-4cfa-a295-bb1acdd50157;
 Mon, 06 Sep 2021 10:14:54 +0000 (UTC)
Received: from AM5PR0601CA0033.eurprd06.prod.outlook.com
 (2603:10a6:203:68::19) by DBBPR08MB5899.eurprd08.prod.outlook.com
 (2603:10a6:10:208::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 10:14:51 +0000
Received: from VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:68:cafe::e8) by AM5PR0601CA0033.outlook.office365.com
 (2603:10a6:203:68::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Mon, 6 Sep 2021 10:14:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT051.mail.protection.outlook.com (10.152.19.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 10:14:51 +0000
Received: ("Tessian outbound 620209b93b95:v105");
 Mon, 06 Sep 2021 10:14:51 +0000
Received: from f3725336c883.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8922800A-C75C-4DE7-B23E-14EE9528B054.1; 
 Mon, 06 Sep 2021 10:14:42 +0000
Received: from FRA01-MR2-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f3725336c883.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 06 Sep 2021 10:14:42 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR2PR08MB4665.eurprd08.prod.outlook.com (2603:10a6:101:25::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Mon, 6 Sep
 2021 10:14:40 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 10:14:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dd26814-38ef-4cfa-a295-bb1acdd50157
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uzHDmyjnh5LGIpbyruutxpYRl2HOE5gt7OKhwpi2jLE=;
 b=7wzyXKu9xb8dGzDWGVozFOJf5zwm5PpWjQ9IkZ4kBP/IWYs2WMMLZcxxjzcwVEJ1K7/Iyf5+wua2fj9RAE/lI9pM590LHJNoOxd3mgDic/pR8JZOLMHBwVgz4M+Z8uvoPoYtt234SYyLS62Sr0HFRslIsAJcLwkqREr5CS8lrd4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c83646bd47ee1174
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dxYMsosEaTZ9CSJ4MU1ItPJz+pLEzyMfBWLzOAtzt27pYkb5Bk5t2inN4O/H59Ah8VnxhXkZAflD63xx0z30rPyXZo4vqxHuS0nYmEKGNu1whZPc6u+e/Ut4sec/UQ+DUhKr6ADWySQFCVM0YbpvdL7BkBj3/lzG7arjDYRnxT5gB2Cn57s6zwCZbBWjjIBq8ynl2gzovo6fVuJhbWH3+b8q91/nFciDFOsGc28wQbgkY7GqKUjWUV27BCe5sIB7DsRW/G2xhieorFX0f6DFowUTIFjMRk8MbGotCM6kFIW581S1yHYsMJztU3DwRMQo5fNsIvN+ngIyDCMG1d1Ptw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uzHDmyjnh5LGIpbyruutxpYRl2HOE5gt7OKhwpi2jLE=;
 b=nynDTTETRYYsOgNWHfleHhkmJT7MEihvuZYAPN/8nklf+zEftzYjhsjyJhAyBpIVOlTXkLRUEWQvmWVnv8tRDUbRf5GElm0hikf72zA6CZ6uTZkWDVKPeewuTRxRcsnYvftlBPc7mfqQyhOeMryhEzyb9JqWb6LYS+VaRAX1LUHnFPQWza/stnQ5zT1RjJ9ty3JpcmvJacsxtlmOk9EbTGwwc8lZBNSKw8n2KAZmK1a7TRDa03E+KTTODZ0dxcKTNdkbZ5C+hM2eKbPFxMQqX/ZgQ2r8kFvC1jg9LQ694fhMe+SiVlqPalX5Rl+2kvd/dx86kxP/zgzIHL0Co9hS6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uzHDmyjnh5LGIpbyruutxpYRl2HOE5gt7OKhwpi2jLE=;
 b=7wzyXKu9xb8dGzDWGVozFOJf5zwm5PpWjQ9IkZ4kBP/IWYs2WMMLZcxxjzcwVEJ1K7/Iyf5+wua2fj9RAE/lI9pM590LHJNoOxd3mgDic/pR8JZOLMHBwVgz4M+Z8uvoPoYtt234SYyLS62Sr0HFRslIsAJcLwkqREr5CS8lrd4=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Juergen Gross <jgross@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Dario Faggioli <dfaggioli@suse.com>
Subject: Re: Crash when using cpupools
Thread-Topic: Crash when using cpupools
Thread-Index: AQHXoNo0twqKn1RLAkCotEsfXja8k6uWr0+AgAAGR4CAAAvYAIAADN6A
Date: Mon, 6 Sep 2021 10:14:39 +0000
Message-ID: <FC6628F1-46A0-4992-A510-7AFD47149C79@arm.com>
References: <FBF65354-8C41-41B4-8A24-BA1485CAE073@arm.com>
 <5b0a4530-72df-a186-9bd5-50759b3dd151@suse.com>
 <c1554429-07d7-1f9f-ef0e-1931675f01e8@citrix.com>
 <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
In-Reply-To: <41b5e46e-daad-82f0-63f0-1efe431ca695@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 79cc6872-6e4f-4a59-c34f-08d9711f2a2b
x-ms-traffictypediagnostic: PR2PR08MB4665:|DBBPR08MB5899:
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB58996F178E9604226208F1839DD29@DBBPR08MB5899.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HiGygZbc+05kS2syjfEmYL3TKeGDyJwrUphxidA9Mc5EHCleZOlZOdbEx8heb2Bu5uPhsQDdl47ky23KDLqov8b8p0JDpUllGgvr8mkRHgH+747C3nc6GdO0yUYzEK7YeOs8VNa3l3GoEubxt6JIl44soyTDki3maMNAbbIBxqT8O+HoruRy8RfHi6Zq1frUzqtG2a+qBrTHn0D+GA7KJVp3MVsz+r4WRzdJzBkJY58M5fdzrU4PRxepc41xae25kislEnqG6Fo+0G0ifLfkdqgKkKma9ap8Jj8Cu1DEBV8gOsEY5jFN4XIcswYoJdUjUhvnTPp2iHzceHxATRxUpe4xLTZ24wU7vxCpyjhE3YBRxPa8Ago094+FCRNEp9+KFBzxMzvGTrPiotNtEduI/1+MhhP6HzUq9irebL3Fithz4L6RUsejkIpSHqUsYuer0xo1GqVB71YkLETU+GR1+c9SWSU3tubYZEONtSONvRZXzmFoUGAVdxeK8/UxvN0jONNnpSnQ2sHNUQ47hrmLS9v669B/z+WAnZxrvTu2EmURScXLyWOnUhCx2qz4g1ls7NAxgicoEdTS/lSBsXRjue+QW72FzTaKQfBqQ0T5t/zpsUJTraVupdORDn2bZCzWcFhevnI7rFF3/U0StwdEelpExCU8Pc0OlVuUn40VxQRXkunbj8N6SSxfVAA3vrqAuGJDd/FvEqeZL7Isbsa3T0t35JY1CdVdjkxpukU0z1M=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(366004)(39860400002)(136003)(396003)(346002)(8676002)(6512007)(66556008)(478600001)(186003)(66476007)(38070700005)(64756008)(36756003)(6916009)(66446008)(76116006)(91956017)(2906002)(86362001)(83380400001)(6506007)(53546011)(3480700007)(71200400001)(54906003)(5660300002)(2616005)(4326008)(122000001)(316002)(66946007)(38100700002)(8936002)(26005)(6486002)(33656002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <B7F8479821F8D94EAC83556DE029CBE0@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4665
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6ef4d185-d307-47ee-88dc-08d9711f2341
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lmV0IZJ4pSh8wRUNd26QEZzegHO2z1kxCLV1tDw81hkfUKg8fSte+Zpw8+Zp/GOmG+/i9L2SGTsS4LwJmV42/ofgbh9Gmqb/HO5eJFwhnwpJlycSIuzUy5/uwAlvN2IxmPWMwFujAvMUiPGHhXL3yYtiC4XCE59ZbbuF0EUEWNl1QbEqarUs3Jb4eDz5OaTgSG5msZw5Hr+9jns38exEwebI3HgOVpZCmC38+yKq3S0MPci2jfB3pYAQzKs+1QTHRvRCR+NaSoIsivvO4yofTKkvdlH2/WptPTmmE6r4aidYQj5q+Whr9HdP3vF9AUtqXSIxoEZ2UbD3pJmJZxEviwIrJN3js+K+s8yaiPUOeOhACJbrwlc8kLVSph17gyqmUhdOSBcaOw84kjn9rFlPQGc5STOooonLwJxrTAjJsAXrSoQrXup+sAKtEcB2vEPUfXrVwEc7WrKtX8imYoUGonzj8+qPT8PB0O/978ck/vJnhwX+SoCuyPnj7RjJxtikABfTiSi3io9zcD0PawsEccCeFiX35WDm22TPvNNplFTVoTTbBrP1H90HrnGWhQ8cgocpNqYHejlGQ93Ztk1KgG9bgGYT/fF/sgg2VxROWdUZ9ViFJH8Wn+SJAPf5b+p+8wsSyJWljinaXE6OhygMODY/AclNL5FDmMq3+SnEJYJDwCR1A1DXjWL/rQXUG0wO7J2XSZ4caXGPsRFPuvQf1A==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(136003)(346002)(36840700001)(46966006)(26005)(107886003)(356005)(36756003)(3480700007)(86362001)(6486002)(70206006)(70586007)(186003)(8676002)(83380400001)(2906002)(4326008)(53546011)(6506007)(82740400003)(81166007)(6862004)(336012)(5660300002)(47076005)(36860700001)(2616005)(82310400003)(8936002)(316002)(33656002)(478600001)(54906003)(6512007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 10:14:51.3972
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 79cc6872-6e4f-4a59-c34f-08d9711f2a2b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5899

SEkgSnVlcmdlbiwNCg0KPiBPbiA2IFNlcCAyMDIxLCBhdCAxMDoyOCwgSnVlcmdlbiBHcm9zcyA8
amdyb3NzQHN1c2UuY29tPiB3cm90ZToNCj4gDQo+IE9uIDA2LjA5LjIxIDEwOjQ2LCBBbmRyZXcg
Q29vcGVyIHdyb3RlOg0KPj4gT24gMDYvMDkvMjAyMSAwOToyMywgSnVlcmdlbiBHcm9zcyB3cm90
ZToNCj4+PiBPbiAwMy4wOS4yMSAxNzo0MSwgQmVydHJhbmQgTWFycXVpcyB3cm90ZToNCj4+Pj4g
SGksDQo+Pj4+IA0KPj4+PiBXaGlsZSBkb2luZyBzb21lIGludmVzdGlnYXRpb24gd2l0aCBjcHVw
b29scyBJIGVuY291bnRlcmVkIGEgY3Jhc2gNCj4+Pj4gd2hlbiB0cnlpbmcgdG8gaXNvbGF0ZSBh
IGd1ZXN0IHRvIGl0cyBvd24gcGh5c2ljYWwgY3B1Lg0KPj4+PiANCj4+Pj4gSSBhbSB1c2luZyBj
dXJyZW50IHN0YWdpbmcgc3RhdHVzLg0KPj4+PiANCj4+Pj4gSSBkaWQgdGhlIGZvbGxvd2luZyAo
b24gRlZQIHdpdGggOCBjb3Jlcyk6DQo+Pj4+IC0gc3RhcnQgZG9tMCB3aXRoIGRvbTBfbWF4X3Zj
cHVzPTENCj4+Pj4gLSByZW1vdmUgY29yZSAxIGZyb20gZG9tMCBjcHVwb29sOiB4bCBjcHVwb29s
LWNwdS1yZW1vdmUgUG9vbC0wIDENCj4+Pj4gLSBjcmVhdGUgYSBuZXcgcG9vbDogeGwgY3B1cG9v
bC1jcmVhdGUgbmFtZT1cIk5ldFBvb2xc4oCdDQo+Pj4+IC0gYWRkIGNvcmUgMSB0byB0aGUgcG9v
bDogeGwgY3B1cG9vbC1jcHUtYWRkIE5ldFBvb2wgMQ0KPj4+PiAtIGNyZWF0ZSBhIGd1ZXN0IGlu
IE5ldFBvb2wgdXNpbmcgdGhlIGZvbGxvd2luZyBpbiB0aGUgZ3Vlc3QgY29uZmlnOg0KPj4+PiBw
b29sPeKAnE5ldFBvb2wiDQo+Pj4+IA0KPj4+PiBJIGVuZCB3aXRoIGEgY3Jhc2ggd2l0aCB0aGUg
Zm9sbG93aW5nIGNhbGwgdHJhY2UgZHVyaW5nIGd1ZXN0IGNyZWF0aW9uOg0KPj4+PiAoWEVOKSBY
ZW4gY2FsbCB0cmFjZToNCj4+Pj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjM0Y2IwPl0gY3JlZGl0
Mi5jI2NzY2hlZDJfYWxsb2NfdWRhdGErMHg1OC8weGZjDQo+Pj4+IChQQykNCj4+Pj4gKFhFTikg
ICAgWzwwMDAwMDAwMDAwMjM0YzgwPl0gY3JlZGl0Mi5jI2NzY2hlZDJfYWxsb2NfdWRhdGErMHgy
OC8weGZjDQo+Pj4+IChMUikNCj4+Pj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjQyZDM4Pl0gc2No
ZWRfbW92ZV9kb21haW4rMHgxNDQvMHg2YzANCj4+Pj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjJk
ZDE4Pl0NCj4+Pj4gY3B1cG9vbC5jI2NwdXBvb2xfbW92ZV9kb21haW5fbG9ja2VkKzB4MzgvMHg3
MA0KPj4+PiAoWEVOKSAgICBbPDAwMDAwMDAwMDAyMmZhZGM+XSBjcHVwb29sX2RvX3N5c2N0bCsw
eDczYy8weDc4MA0KPj4+PiAoWEVOKSAgICBbPDAwMDAwMDAwMDAyMmQ4ZTA+XSBkb19zeXNjdGwr
MHg3ODgvMHhhNTgNCj4+Pj4gKFhFTikgICAgWzwwMDAwMDAwMDAwMjczYjY4Pl0gdHJhcHMuYyNk
b190cmFwX2h5cGVyY2FsbCsweDc4LzB4MTcwDQo+Pj4+IChYRU4pICAgIFs8MDAwMDAwMDAwMDI3
NGY3MD5dIGRvX3RyYXBfZ3Vlc3Rfc3luYysweDEzOC8weDYxOA0KPj4+PiAoWEVOKSAgICBbPDAw
MDAwMDAwMDAyNjA0NTg+XSBlbnRyeS5vI2d1ZXN0X3N5bmNfc2xvd3BhdGgrMHhhNC8weGQ0DQo+
Pj4+IA0KPj4+PiBBZnRlciBzb21lIGRlYnVnZ2luZyBJIGZvdW5kIG91dCB0aGF0IHVuaXQtPnZj
cHVfbGlzdCBpcyBOVUxMIGFmdGVyDQo+Pj4+IHVuaXQtPnZjcHVfbGlzdCA9IGQtPnZjcHVbdW5p
dC0+dW5pdF9pZF07IHdpdGggdW5pdF9pZCAwIGluDQo+Pj4+IGNvbW1vbi9zY2hlZC9jb3JlLmM6
Njg4DQo+Pj4+IFRoaXMgbWFrZXMgdGhlIGNhbGwgdG8gaXNfaWRsZV91bml0KHVuaXQpIGluIGNz
Y2hlZDJfYWxsb2NfdWRhdGENCj4+Pj4gdHJpZ2dlciB0aGUgY3Jhc2guDQo+Pj4gDQo+Pj4gU28g
dGhlcmUgaXMgbm8gdmNwdSAwIGluIHRoYXQgZG9tYWluPyBIb3cgaXMgdGhpcyBwb3NzaWJsZT8N
Cj4+IEVhc3ksIGRlcGVuZGluZyBvbiB0aGUgb3JkZXIgb2YgaHlwZXJjYWxscyBpc3N1ZWQgYnkg
dGhlIHRvb2xzdGFjay4NCj4+IEJldHdlZW4gRE9NQ1RMX2NyZWF0ZWRvbWFpbiBhbmQgRE9NQ1RM
X21heF92Y3B1cywgdGhlIGRvbWFpbiBleGlzdHMgYnV0DQo+PiB0aGUgdmNwdXMgaGF2ZW4ndCBi
ZWVuIGFsbG9jYXRlZC4NCj4gDQo+IE9oIHllcywgaW5kZWVkLg0KPiANCj4gQmVydHJhbmQsIGRv
ZXMgdGhlIGF0dGFjaGVkIHBhdGNoIGZpeCB0aGUgaXNzdWUgZm9yIHlvdT8NCg0KSXQgZG9lcywg
bXkgZ3Vlc3QgaXMgbm93IGJvb3RpbmcgcHJvcGVybHkgOi0pDQpTbyB0aGlzIGlzIHNvbHZpbmcg
dGhlIGlzc3VlIG9uIGFybSAoYW5kIHByb2JhYmx5IG9uIHg4NiBpZiBpdCB3YXMgcHJlc2VudCBi
dXQgdW50ZXN0ZWQgYnkgbWUpLg0KDQpGZWVsIGZyZWUgdG8gYWRkIHRvIHlvdXIgcGF0Y2ggbXk6
DQpSZXZpZXdlZC1ieTogQmVydHJhbmQgTWFycXVpcyA8YmVydHJhbmQubWFycXVpc0Bhcm0uY29t
Pg0KDQpUaGFua3MgYSBsb3QgZm9yIHRoZSBxdWljayBmaXgNCkNoZWVycw0KQmVydHJhbmQNCg0K
PiANCj4gDQo+IEp1ZXJnZW4NCj4gPDAwMDEteGVuLXNjaGVkLWZpeC1zY2hlZF9tb3ZlX2RvbWFp
bi1mb3ItZG9tYWluLXdpdGhvdXQtdi5wYXRjaD48T3BlblBHUF8weEIwREU5REQ2MjhCRjEzMkYu
YXNjPg0KDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:17:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:17:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179586.325923 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBhI-0007aQ-D2; Mon, 06 Sep 2021 10:17:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179586.325923; Mon, 06 Sep 2021 10:17:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBhI-0007aJ-A8; Mon, 06 Sep 2021 10:17:52 +0000
Received: by outflank-mailman (input) for mailman id 179586;
 Mon, 06 Sep 2021 10:17:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNBhG-0007aD-7o
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:17:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNBhG-0001JG-1F; Mon, 06 Sep 2021 10:17:50 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNBhF-00030S-R7; Mon, 06 Sep 2021 10:17:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=2AS3K2FbI2bd9Qg1w3F4RKn4dZ2PjVJc8kXf8TTYLZE=; b=DZo8XLD/EXOqvOcc85kZL0s9pa
	zz4LMwrPN1yu0acKVKzXtdGEgjtridSe1DScoiuanx6fGTjQwMZCdVKK0P3GNeRKI3a7AJgptiWJa
	Q/Rwihi2aUwpbXRFBApoTkwXm4p/pL8F8AZdIio58CLh7ctUNt0nyX3zfn+HJf09kJyA=;
Subject: Re: About Arm guests accessing to GICD_ICPENR
To: Hongda Deng <Hongda.Deng@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>, nd <nd@arm.com>
References: <VE1PR08MB5677BABDFBBCA24B4ADA9C16E6D29@VE1PR08MB5677.eurprd08.prod.outlook.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9152003b-87c6-f9ac-1754-db1eba86a6c3@xen.org>
Date: Mon, 6 Sep 2021 11:17:48 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <VE1PR08MB5677BABDFBBCA24B4ADA9C16E6D29@VE1PR08MB5677.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 06/09/2021 10:04, Hongda Deng wrote:
> Hi Julien,

Hi Hongda,

> Xen provides vGIC to support Xen guests, and currently xen will return IO
> unhandled when guests access GICD ICPENRn registers. This works fine with Linux
> guests, for Linux won't access these registers. But for Zephyr, this mechanism
> will cause IO dataabort on Zephyr's initialization which makes Zephyr get in
> fatal error.
>   
> One method to ease this is to let vGIC ignore GICD ICPENRn registers access. I
> tested it with Linux guests and Zephyr guests, and both works fine. And I found
> in this patch[1] that it would be more complex to touch the read part for
> I{S,C}PENDR and the write part for ICPENDR,

Read to I{S,C}PENDR should already return. AFAIK, what's left 
unimplemented is write to ICPENDR.

> so could we ignore GICD ICPENDRn
> registers access to ease Zephyr's initialization problem?
Would you be able to provide more information on how Zephyr is using it?

> 
> If Xen wants a complete GICD ICPENDRn emulation to fix it, do you have any
> suggestions.

Emulating ICPENDR is not easy with the existing vGIC. It would be great 
to finally have a vGIC spec compliant, but I also appreciate this is 
going to take some time.

Ignoring the access is probably the best we can do. However, this is 
also a risky approach because Zephyr (or another OS) may genuinely want 
to clear an already pending interrupt. So I would suggest to walk the 
interrupts that are meant to be modified and check whether they are 
already pending. If they are then print a message.

This would make clear to the developper that something may go wrong 
later on.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:18:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:18:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179592.325935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBi8-00089t-O6; Mon, 06 Sep 2021 10:18:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179592.325935; Mon, 06 Sep 2021 10:18:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBi8-00089m-Kz; Mon, 06 Sep 2021 10:18:44 +0000
Received: by outflank-mailman (input) for mailman id 179592;
 Mon, 06 Sep 2021 10:18:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBi7-00089a-Dm; Mon, 06 Sep 2021 10:18:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBi7-0001Jz-8d; Mon, 06 Sep 2021 10:18:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBi7-0001dM-03; Mon, 06 Sep 2021 10:18:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNBi6-0002w9-Vn; Mon, 06 Sep 2021 10:18:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CTYTAy+nlz9uYwpVeMto9bCcMl9eD5MZtSvV6DyBN58=; b=MPAyFi4IQxWudmHZFrT9E77gKG
	7vrrHLd3KHsFk0vAeoSkkS8Hx+okBNlPP69LciVBT1/F8xLff0dYwjy3PzCA7eYibyPi52TBPK0Fo
	wqhFhnQbM94I2aQV+9N7OMaVummHhEL8GWh6UvhqkK0UMYKD95fp5XdyvxQ75kUWyJG8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164854-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164854: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 10:18:42 +0000

flight 164854 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164854/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   63 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   37 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:30:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:30:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179601.325949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBtI-0002CP-SH; Mon, 06 Sep 2021 10:30:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179601.325949; Mon, 06 Sep 2021 10:30:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNBtI-0002CI-Oa; Mon, 06 Sep 2021 10:30:16 +0000
Received: by outflank-mailman (input) for mailman id 179601;
 Mon, 06 Sep 2021 10:30:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNBtH-0002CC-PY
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:30:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 06a6541f-ce3f-4f1a-a113-55bf8351e0b5;
 Mon, 06 Sep 2021 10:30:14 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-r2swLOZBMM-JLY12322v_A-1;
 Mon, 06 Sep 2021 12:30:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5470.eurprd04.prod.outlook.com (2603:10a6:803:d6::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 10:30:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 10:30:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0046.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Mon, 6 Sep 2021 10:30:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06a6541f-ce3f-4f1a-a113-55bf8351e0b5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630924213;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ld/f2dx1cE58n/YMzJDEy0DH6VXjkwcYdaqKpjFUdSg=;
	b=Eb+7ryE+zak+472LJ4pLf1WaaZXc1mAsJDvOOXyt+SNYN1E9i1evQ+HmxuqTtR3XbFBb18
	cLjySN9krS3zASj6735P/9pHpAY4FpN81SBcI5jqClT7MTxD3a+NpFmke5BZlN9kgByb/a
	V5DJdWQjMWDfuVXINnsGbYnJMWJeE9s=
X-MC-Unique: r2swLOZBMM-JLY12322v_A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HKLLUw5OGe9ljUCFPEp1GmutQSR6e5A1oJ6vFxbMgk67g3EUht+39Im0yr5dECSdSH7C0dmCluOojrp4Z3pQl+FqM3mKUA59Iq0+NS7eiS6IYBh7lLnboKG+g/vU/D5YU2ADztMfZ0B8IVWjOgv8DMHuS8QFijFsC+6eXO5apgOWzj7vwC5wfhM6BQureYg9r4MjElcsa0zLypfa7EdVFylar5R5vnu1dwpLHfd4B2A7wsvrjKmetmFErKMqqSe/fMtBpn7FvRXKiuOAYf2YnRrcAQdSoF5p9q1ItA4aspXKdJVPlINvoXySROIawgTm13TkSy+ZAn26bTRHGK8aIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ld/f2dx1cE58n/YMzJDEy0DH6VXjkwcYdaqKpjFUdSg=;
 b=R1PjiE7mdimW/MIqkvFGXBxs52KySlIne9ltnszRY4jj9trvpkYO85rnhuc8x3zZBra4Gnc2pxv4zH5F0vfOAzPICGMRgs8mltWeQI80s0y0Tz9t0yCyGQqZxgGxQtAk8upLIChiREoZuvOVPMr5zcmVoDrx6vK26p5fX87JDIqhAJp71icL0dWCDWZbLSB8f3/ZiD6SwzWXdvUr6j06ngc1tNBwHQu4/i87Agc65pAr00E4ZNpgiSPDpxBcg6a49sBUlLv7UrUVElZi6FCTW5QtENf6+utHYcFUiSLt6A0NJzdaHK4+13gtWswRRTn53bXDH1wLBErFsS311LqfHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 01/51] build: introduce cpp_flags macro
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-2-anthony.perard@citrix.com>
 <ba3cfb06-1d89-ff62-1012-6edc2a9c5259@suse.com> <YTXoCQ3SSvN4eU/C@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <280c5579-ee62-ed36-1f08-58fe5b9ca34f@suse.com>
Date: Mon, 6 Sep 2021 12:30:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTXoCQ3SSvN4eU/C@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0046.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b42c405e-6058-4ebf-6842-08d971214d69
X-MS-TrafficTypeDiagnostic: VI1PR04MB5470:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5470D9706271C37D03730CEEB3D29@VI1PR04MB5470.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	H5tffAjQbLpV02jAOP50qHOcz4dp92GBmz8+6WBT0agGzkjyP8RzNn3o2qxZbsZZ8hSeYt/gZuUNL128utTmaczxJ5qWFaWGil6J4gkREMv4oXfdxm1DQ/9YopVQ1FKu9vaKWMTOv0E4YRZ1D0K+/EYo+2kscBitq8P6f5eOtpCZvjRdrSYwNgxg+L9OzEKUgC4Vu8theYoIgbFLjBM9mbUF34vG5pRtM974eisM+CapuX0nUwLRAVsXPxJndQPSImGKzbed+HtdEPV1IKH40oS/1EXtlajl6v4EVm2qbVyUhr157AKFUI7Izmf8mO5NNELf/04z2AXsBsgOPRqnH0CMqBKNNONlPIAG9FPBbOu2M+ZBuEsi5ec2CUItXhOqmfe2mwEylLkxvCmi4XzZ01Miw2fq0+NXEO6GUJw8LcqGLeFVV/uSmQk/dnGXK+JWe6sZcNjpqi8SKgSWyH395rmxn+2veUiepAGm4pt4C/O5cdcwdS0T7aJtcT3bE+qflZQOuqn3AJuOHyjV/sMqocMy//3MCP9hxbWWE26AH3x6N7c/Sf4XUIAtJxT2utyhwGooFH3g8EvDZeLP1iwHYcD3DH2GKkC32l6WHAFtwN4qteoLTWQvxBKeSDgQGBmrI5Fr/rUEyR/0D2rWyVjpXn6D7Hz65qCfL1l+G6r67gZLIQr8b59JSKxCaAeYgd8lE6EyPMpTaLykpgJNUdqhKdxb6jueQ4NzrV6cL+q/z9I=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(39860400002)(366004)(346002)(396003)(66476007)(66556008)(54906003)(478600001)(86362001)(186003)(7416002)(8936002)(956004)(36756003)(53546011)(31696002)(4326008)(2616005)(16576012)(8676002)(2906002)(316002)(66946007)(38100700002)(6916009)(31686004)(6486002)(83380400001)(26005)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OFFKK1hlV2Nxc3lZMGFBM2lkZGhHa0xmcU9SZ2JLK0RBL0xURFVlYWFKYVFt?=
 =?utf-8?B?VlhURHA4Mk9pMTRkRUxMbzIrY3ZUVDZXcTZKL3BRaVo2b2x5dldjZEUwYUpw?=
 =?utf-8?B?ZTBlKzVVdndhNzcvVzNUZUtXa21YSnVneFNRZ2xRSXhDQkJvT2d3Q1dBdjU0?=
 =?utf-8?B?ZlgwOHJ1ZUYyR3ZhYVg1TnFtMXp6aW9uYm1RZUZSZHRYL2paLzFCdEtDQVBD?=
 =?utf-8?B?MXE0dkY5SnJUdG05dTExZEJ1RGk5ZGVrSkNYVC9jQTZnbGhpRXMyUDZBQm15?=
 =?utf-8?B?SkQvVkcvZWUzMmtQNU8wN2VUUUd3ZUw1NzB1NFFXQ0QxOVAvN2hDbHFvTXNX?=
 =?utf-8?B?YVhYeFdKMXFJd29uRFVuckl6RENrdHRXVzFxbWxiYTlzNksyYm5LVndRay9l?=
 =?utf-8?B?ZWVaa2pJYTBMU1BySkttYTJDS0Y3bTVKd2R4b3RaMHpCVDFVbTlOcU4yTXA0?=
 =?utf-8?B?NGRHcnpkTjNQR01RekMxMEh2M2N1Mitpa0RQMytvMU95M1FGcjVjWFBHWkJJ?=
 =?utf-8?B?RldWdG16UFc1K2Eza2JheENIOUVMUU5rTHNLTldjK1pET2hHREhCcHZxS3k0?=
 =?utf-8?B?Q2pWclZzcGtRSDgydFZ0amhMVWxvTU90d2V0TVlxSEptR1VQODNVQWpPenZX?=
 =?utf-8?B?WTFPRCtKQmgzYUZSR2k1Tzk2cnNUN2QxNmdlSjVOTEhFSHNNZWhoU0RlUzhx?=
 =?utf-8?B?SlhOVk9ocjlqeVNjK0JEd28rVUZrWmZudGszU0E0V0JVQnZoaEhEUGpzSjJG?=
 =?utf-8?B?LzBkSUlJdkdWc2dWL1lZbVBOSDllRi9nOHR1Ty9qSnJwcHZVak1aTXhQQkM1?=
 =?utf-8?B?Ym1mSmJXbmNXakxTRDBRQWM1SlpRUEREaGdMem5WWXBNTnZXdTFNbjdNY2l5?=
 =?utf-8?B?dmNxa1ZLZGx1RWFpTENubVJFNU5mdnVoSXIxcERTdWIxbjdQb2FTYm1HQTl3?=
 =?utf-8?B?NVBxaXFSZ00vMG9NbEIwL1Y4Y1VJVVhOaVAxOXh6ZFc4T2k4czdnYnEwUGhI?=
 =?utf-8?B?bklVcGxWa2w3TThwUTZZaE1EeStWLzVZYkliRjArQWVuWlNjSlAxcGFQMnZk?=
 =?utf-8?B?dGlHc2UrSVB6akd6ZThmOU1LRUpqUEZJelVjcWM2eUVDakNqREc5aTQ5VC9K?=
 =?utf-8?B?cXZpMWRNWWN4RUoxNXlqVTIyZnFFaFZOWWVQOHBzY04xT2hXblNNZ05QdE9k?=
 =?utf-8?B?SVdtd291NnNrYmN4MlptK3JVaDdncXhLemppK2c3YjBTemZBaFg2RkZKQlVC?=
 =?utf-8?B?czB1T3ZVMFRMVkkvbzlKdnUzR1d2T3BOQnpvMmM4TWRCNFd1dVk0TE1HMTFL?=
 =?utf-8?B?UWtmd3o2VVRIYjBhMDZla2daR2lnWWpKZ1hjdmhtMFN2Z0liaXBQTlhROStJ?=
 =?utf-8?B?Y1diY1lWMzBIa3k0SVVWZmRKckk2QXAwNTMzdTU3d0hHUHR4S0NyanpQY29D?=
 =?utf-8?B?NE16WlRFN2I1b281Ty9YS3pkOEdhT3k5VHkzMGZsWlJLRHpyNjVGcWNVMHM4?=
 =?utf-8?B?NTkyTHBvTWJ1bmpGcFlTQ2F4dExWM1hPeGdkVFFYTTF3V29aQmx3WHFCVnFl?=
 =?utf-8?B?eGo5MGVkRk92UWlTa0xWZ3VpdlZNVHlxQWpEaW1aUng1d1l6cVRtUFBKeURV?=
 =?utf-8?B?ODNmNHNSalhyVUl1bVpOQThiWGdFK2l3T3UyeHF0VUp3U3hzcXF5TElmL3kz?=
 =?utf-8?B?d3p1SnkyMTh3R2xLTTlqeTRnRjljbk4ra213d2thUGZDdi9RaGEwbnl1T0k5?=
 =?utf-8?Q?Fptj31m2heZSr06CwIGEEUemStvSo8KZth/aBkf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b42c405e-6058-4ebf-6842-08d971214d69
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 10:30:09.7937
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YUfEkUo9myD/ZgFwmDA5yc4uNyaYRrGdU14+FmpWeub1gCbRqRFjj4S0/YyTXbINqQoxuwY+UHKpm1km6DP0wg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5470

On 06.09.2021 12:06, Anthony PERARD wrote:
> On Thu, Sep 02, 2021 at 12:08:58PM +0200, Jan Beulich wrote:
>> On 24.08.2021 12:49, Anthony PERARD wrote:
>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>> albeit with a remark:
>>
>>> --- a/xen/Rules.mk
>>> +++ b/xen/Rules.mk
>>> @@ -133,6 +133,9 @@ endif
>>>  # Always build obj-bin files as binary even if they come from C source. 
>>>  $(obj-bin-y): XEN_CFLAGS := $(filter-out -flto,$(XEN_CFLAGS))
>>>  
>>> +# To be use with $(a_flags) or $(c_flags) to produce CPP flags
>>> +cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))
>>
>> Afaics this has nothing to do with Linux'es cpp_flags, so what we do here
>> is entirely up to us. If this is strictly intended to be used the another
>> macro, wouldn't it make sense to have
>>
>> cpp_flags = $(filter-out -Wa$(comma)% -flto,$($(1)))
>>
>> here and then e.g. ...
>>
>>> @@ -222,13 +225,13 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
>>>  	$(call if_changed,obj_init_o)
>>>  
>>>  quiet_cmd_cpp_i_c = CPP     $@
>>> -cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -MQ $@ -o $@ $<
>>> +cmd_cpp_i_c = $(CPP) $(call cpp_flags,$(c_flags)) -MQ $@ -o $@ $<
>>
>> ... the slightly simpler / easier to read
>>
>> cmd_cpp_i_c = $(CPP) $(call cpp_flags,c_flags) -MQ $@ -o $@ $<
>>
>> here?
> 
> I don't think this is better or simpler. "cpp_flags" don't need to know
> the name of the variable to be useful. I think it is better to know that
> "cpp_flags" act on the value of the variable rather than the variable
> itself, when reading "$(call cpp_flags, $(a_flags))".

Well, yes. This way one could also pass more than just the expansion of
either of these two variables. The thing that made me think of the
alternative is the comment: Would you mind if I inserted "e.g." in there,
to make clear this isn't limited to these two variables?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:38:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:38:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179609.325960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNC1C-0003CF-Qq; Mon, 06 Sep 2021 10:38:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179609.325960; Mon, 06 Sep 2021 10:38:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNC1C-0003C8-NJ; Mon, 06 Sep 2021 10:38:26 +0000
Received: by outflank-mailman (input) for mailman id 179609;
 Mon, 06 Sep 2021 10:38:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNC1B-0003C2-BE
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:38:25 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNC18-0001ey-1b; Mon, 06 Sep 2021 10:38:22 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNC17-0005RK-R2; Mon, 06 Sep 2021 10:38:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=vRQKbF9dIKn56RN0QBCOcsQEJRISlyCnZKN8myk/D8Q=; b=A2FI9XHsrW4R+hJqG8RLklILc4
	+jqR2/HiZZsgEJ7qd5LKq/H+b5zoIkxsdrW9rXAOOMu58TJQ1tAHkJDJR0bJ0EfuvYDMHjL4BxyPI
	9g9i6+gMZ+giS19dbtYl/ImncQ9QvLwgUTqQCj/DG6fhI7UDrW2u+i03QWw05rwhcwkY=;
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
 <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
 <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
 <f950a37d-2773-fce5-9e77-163a64925d7d@xen.org>
 <71e0a54f-da69-5712-f826-9c38eecfa244@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9482457a-fae2-eb8e-79dc-b5d7bcf2fa0a@xen.org>
Date: Mon, 6 Sep 2021 11:38:19 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <71e0a54f-da69-5712-f826-9c38eecfa244@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 06/09/2021 11:06, Oleksandr Andrushchenko wrote:
> On 06.09.21 12:53, Julien Grall wrote:
>>>>>> However, looking at the rest of the code, we already have a check for vpci in the common IOREQ code.
>>>>>
>>>>> Which may not be enabled as it depends on CONFIG_IOREQ_SERVER.
>>>>
>>>> Right. My point is when CONFIG_IOREQ_SERVER is set then you would end up to call twice vpci_process_pending(). This will have an impact how on long your vCPU is going to running because you are doubling the work.
>>>
>>> So, you suggest that we have in the common IOREQ code something call like
>>>
>>> arch_vpci_process_pending? In case of x86 it will have the code currently found in the
>>>
>>> common IOREQ sources and for Arm it will be nop?
>>
>> No I am suggesting to move the call of the IOREQ code to hvm_do_resume() (on x86) and check_for_vcpu_work() (on Arm).
> 
> Ok, I can move vPCI code to hvm_do_resume, but vPCI is only used for x86 PVH Dom0.

AFAIK, Roger is planning to use it for x86 PVH guest.

> 
> Do you still think hvm_do_resume is the right place?
I think so. AFAICT, on x86, the only caller of 
vcpu_ioreq_handle_completion() is hvm_do_resume(). So it makes sense to 
push one layer up.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:43:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:43:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179615.325970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNC5p-0004ls-Ce; Mon, 06 Sep 2021 10:43:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179615.325970; Mon, 06 Sep 2021 10:43:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNC5p-0004ll-9k; Mon, 06 Sep 2021 10:43:13 +0000
Received: by outflank-mailman (input) for mailman id 179615;
 Mon, 06 Sep 2021 10:43:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNC5o-0004lf-KT
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:43:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNC5m-0001kT-Vu; Mon, 06 Sep 2021 10:43:10 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNC5m-0005x2-PP; Mon, 06 Sep 2021 10:43:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=ZCb0AKomNjh90Gk0tOHoLBirjLw+JK+7N6jSubaVZtI=; b=GTbTw7PclL7DoJ7msE18HSBIsh
	jZ4zTIk4BIazeIcE7Y+/WF8agIZibSSD0qhSLNtiQX3qAoQQzGagnIU7Ovk6ZPz5oFx5+r28taVjS
	fw3+rOD1Uzv3tihlqagvfii8YiXvRUVqfVdoEWQSNvavp9zD5Ti0lA2ZzQYTmCp+IGyM=;
Subject: Re: [PATCH v3 1/4] public: Add page related definitions for accessing
 guests memory
To: Jan Beulich <jbeulich@suse.com>, Costin Lupu <costin.lupu@cs.pub.ro>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <cover.1629737453.git.costin.lupu@cs.pub.ro>
 <22031be8466bb18d1dd891481ccc67d8c2b2dd55.1629737453.git.costin.lupu@cs.pub.ro>
 <accc3026-1043-6b90-eda4-1951ef808bdc@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9a5e836c-32ae-50e2-1091-086a40beb4b2@xen.org>
Date: Mon, 6 Sep 2021 11:43:08 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <accc3026-1043-6b90-eda4-1951ef808bdc@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 24/08/2021 07:11, Jan Beulich wrote:
> On 23.08.2021 19:02, Costin Lupu wrote:
>> These changes introduce the page related definitions needed for mapping and
>> accessing guests memory. These values are intended to be used by any toolstack
>> component that needs to map guests memory. Until now, the values were defined
>> by the xenctrl.h header, therefore whenever a component had to use them it also
>> had to add a dependency for the xenctrl library.
>>
>> This patch also introduces xen_mk_long() macrodefinition for defining long
>> constants both for C and assembler code.
>>
>> Signed-off-by: Costin Lupu <costin.lupu@cs.pub.ro>
> 
> x86 part:
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> This extends to the common parts only if the Arm side gets an ack,

For Arm:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

> since - as said before - there you're treating use of one unstable
> interface (libxc) for another (the ABI) in supposedly stable
> libraries, or - if the ABI is to be stable despite being exposed
> to the tool stack only - you make it impossible to make the page
> size variable down the road.
> 
> Just yesterday we've been (internally) talking about the similar
> "maximum vCPU-s" aspect: This shouldn't be taken directly from the
> ABI by tool stacks, as imo we ought to allow the upper bounds to
> be configurable in the hypervisor (with the present limits merely
> becoming limits of what can be configured). This would similarly
> require a library function (or two, as HVM and PV limits are
> likely different). I wonder whether we shouldn't have a stable
> library providing functions to retrieve such limits. Initially the
> library would return constants, short of the hypervisor providing
> the needed data.
> 
> Jan
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 10:56:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 10:56:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179621.325981 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCIQ-0006Qo-I9; Mon, 06 Sep 2021 10:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179621.325981; Mon, 06 Sep 2021 10:56:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCIQ-0006Qh-FH; Mon, 06 Sep 2021 10:56:14 +0000
Received: by outflank-mailman (input) for mailman id 179621;
 Mon, 06 Sep 2021 10:56:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNCIP-0006Qb-0b
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 10:56:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNCIN-0001yO-A9; Mon, 06 Sep 2021 10:56:11 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNCIN-0007KY-1T; Mon, 06 Sep 2021 10:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=TbH4O9e020jx0sGPeovPiNA2vcT/yAA6hKIcUQf61zI=; b=t0Fgi76jJkub4DJtwu6u80uIh1
	7hbZBfxlc2aDiJ4AuTPrIn+C13Z6tKE2rpTRgun5p7tznUMcaTLWhb2oS+WGKk/1oozXUbfJ0Dlrd
	wuSVjBtGShHkZCRABV+DKWqCt96HCNge6qGhMdFwdvAv8tF6edwJcdnPbjE3nCRKTsoI=;
Subject: Re: [PATCH] gnttab: adjust unmap checking of dev_bus_addr
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <1d2d28ac-bcf2-2a96-a6dd-8282fc59a16c@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <79c21c7b-c833-acfb-644a-907707a9fe1d@xen.org>
Date: Mon, 6 Sep 2021 11:56:08 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <1d2d28ac-bcf2-2a96-a6dd-8282fc59a16c@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 30/08/2021 15:26, Jan Beulich wrote:
> There's no point checking ->dev_bus_addr when GNTMAP_device_map isn't
> set (and hence the field isn't going to be consumed). And if there is a
> mismatch, use the so far unused GNTST_bad_dev_addr error indicator - if
> not here, where else would this (so far unused) value be used?
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Julien Grall <jgrall@amazon.com>

> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -1450,9 +1450,9 @@ unmap_common(
>   
>       op->mfn = act->mfn;
>   
> -    if ( op->dev_bus_addr &&
> +    if ( op->dev_bus_addr && (flags & GNTMAP_device_map) &&
>            unlikely(op->dev_bus_addr != mfn_to_maddr(act->mfn)) )
> -        PIN_FAIL(act_release_out, GNTST_general_error,
> +        PIN_FAIL(act_release_out, GNTST_bad_dev_addr,
>                    "Bus address doesn't match gntref (%"PRIx64" != %"PRIpaddr")\n",
>                    op->dev_bus_addr, mfn_to_maddr(act->mfn));
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:01:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:01:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179627.325993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCN8-00082G-4O; Mon, 06 Sep 2021 11:01:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179627.325993; Mon, 06 Sep 2021 11:01:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCN8-000829-1F; Mon, 06 Sep 2021 11:01:06 +0000
Received: by outflank-mailman (input) for mailman id 179627;
 Mon, 06 Sep 2021 11:01:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNCN7-000823-0l
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:01:05 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b655369a-290f-481c-ae76-3e0f87498c2b;
 Mon, 06 Sep 2021 11:01:04 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6D415220EA;
 Mon,  6 Sep 2021 11:01:03 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 3298213942;
 Mon,  6 Sep 2021 11:01:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id gFtICu/0NWFbGQAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 11:01:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b655369a-290f-481c-ae76-3e0f87498c2b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630926063; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=BrS40Tf/KPD9CQCWVNVJ4t3+lx1YVOcmmtx8WpJDnlY=;
	b=bejh5fc5k+w1U6ChtsJEM6gYGML9shAdNIMcvQWDi+PEXtdSQJBOTjexaAnNTUqHjpiSNi
	VTRoOLFZWRlX8K7yRTmMpcpyPzKYfkpiFOQt4dNDzOGWujkiLaoTiIs//4sRYDI9LHd45i
	gzOgfWigAns23QVOD9Zgar5PfP5VXfk=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>
Subject: [PATCH] xen/sched: fix sched_move_domain() for domain without vcpus
Date: Mon,  6 Sep 2021 13:00:57 +0200
Message-Id: <20210906110057.15384-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In case a domain is created with a cpupool other than Pool-0 specified
it will be moved to that cpupool before any vcpus are allocated.

This will lead to a NULL pointer dereference in sched_move_domain().

Fix that by tolerating vcpus not being allocated yet.

Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpupools with different granularity")
Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/common/sched/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8d178baf3d..79c9100680 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cpupool *c)
 
     for ( unit_idx = 0; unit_idx < n_units; unit_idx++ )
     {
+        /* Special case for move at domain creation time. */
+        if ( !d->vcpu[unit_idx * gran] )
+            break;
+
         unit = sched_alloc_unit_mem();
         if ( unit )
         {
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:05:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:05:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179633.326003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCQs-0000Gu-LR; Mon, 06 Sep 2021 11:04:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179633.326003; Mon, 06 Sep 2021 11:04:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCQs-0000Gn-Ic; Mon, 06 Sep 2021 11:04:58 +0000
Received: by outflank-mailman (input) for mailman id 179633;
 Mon, 06 Sep 2021 11:04:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNCQr-0000Gh-2n
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:04:57 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNCQp-00028G-An; Mon, 06 Sep 2021 11:04:55 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNCQp-0008Lo-46; Mon, 06 Sep 2021 11:04:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=cUUe6zzqW0wvOLi12xZic2f3L00lviKdlvhQ+zAy3fs=; b=0G+ZCTjQWV78luFw20ktNazUe7
	pXeAXYqqVsiVHa4xMP7cltFmcw3uKqLxi+Ndlen7GxAH/sznP1zOnzCptX1/bjm/ahZtLZLvgGT+m
	6a8k+vHCcYOIbeX2LJikd3ce45UYi2DCnT4gwD5L2HvJ3OecNw/P9bKf//9RV8N8/boM=;
Subject: Re: [PATCH] gnttab: maptrack handle shortage is not IOMMU related
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e28eea8c-fd54-c3a7-a10a-5eeac2c7a8a1@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <29fd1d29-1077-4513-a8b6-5d220b06f4d3@xen.org>
Date: Mon, 6 Sep 2021 12:04:53 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <e28eea8c-fd54-c3a7-a10a-5eeac2c7a8a1@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 30/08/2021 15:27, Jan Beulich wrote:
> Both comment and message string associated with GNTST_no_device_space
> suggest a connection to the IOMMU. A lack of maptrack handles has
> nothing to do with that; it's unclear to me why commit 6213b696ba65
> ("Grant-table interface redone") introduced it this way. Introduce a
> new error indicator.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Or should this be a more specific one, e.g. GNTST_no_mt_handle? If not,
> I would intend to also use GNTST_no_space for -ENOMEM like situations.

I guess the guest could use GNTSTS_no_mt_handle to reduce the number of 
persistent mappings (if there are any).

Although, I am not sure how involved it would be in OS like Linux.

Anyway, the new error status is better than the previous one. So I am 
happy with either solutions:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:15:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:15:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179639.326014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCaZ-0001vL-KN; Mon, 06 Sep 2021 11:14:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179639.326014; Mon, 06 Sep 2021 11:14:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCaZ-0001vE-HR; Mon, 06 Sep 2021 11:14:59 +0000
Received: by outflank-mailman (input) for mailman id 179639;
 Mon, 06 Sep 2021 11:14:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNCaX-0001v8-AA
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:14:57 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id aa40d6e8-0f03-11ec-b09e-12813bfff9fa;
 Mon, 06 Sep 2021 11:14:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa40d6e8-0f03-11ec-b09e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630926896;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=oQV7QHWU6OhwydPZsJEIiXObTnGSTwrrzOwsyzrQ4C8=;
  b=CGCwooLj5I3XYEHoqM8dDeJ4sCXe9n+ZgyAgYC6gyk89HPdbeqa5jkJ5
   66ie8t0BcDYJNX0QjB4UpUVwo7ewMD4n7Cv3Q0mDOVTCKssMxTdwzitw2
   92sW+ShPcl2Setz3TsC1wJzPidfss4MlFmC2ktb46Xr+N+51j3vSXd491
   A=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: yyVzsrerJc31Vq4XhX3BSvdEhgA5sZEjHA742tSj20s9+xubdWoiSaMDCSH6hmbZI400VsWs1B
 u//yIyX9p9pGYrSjQ5UnJ3vAqAwN7oaS0f1xvAoW2gCzEQunYN0bvunW+9FfV+7Koi27tqAoMJ
 B4zp6BWdsj/RB8RctWGYdp44K/JkHHmFHrBD1wCv2G+DAzgskNQPbQRfKE1Oat1oc3BPyZXY3J
 tebEh9AXWiSgOWDriF/pvDEIqCiYXLBk5WB9uptuowsUGq8bKLkvfH8HjRbPDSBeZ7CPoFKF0J
 Vfy5+SESMhoBns0iUUAMAzNa
X-SBRS: 5.1
X-MesageID: 53838646
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:aLpIX60KMs9qunmdlIZzSgqjBSVyeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5OEtApTiBUJPwJE80hqQFnrX5Wo3SIDUO2VHYUb2KiLGN/9SOIVyHygcw79
 YGT0E6MqyLMbEYt7eL3ODbKadY/DDvysnB7o2/vhQdPj2CKZsQizuRYjzrY3GeLzM2Y6bReq
 DshPav6wDQAkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="53838646"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jTFVkScVpNPVsclnXnFuj4JvvGK/1s0ApPARsh0d/iWpD8pSSiLkAwf6VIXPHOu3j2aWD0dE4F0hDKP/eRn9xZpaEvT5NUOwsMCMWqVg5CpGAxSbR2fwSTySX16N+KVikuxLzcFEr/u45282oV90Zyyw7Rz2F9O+2qKvB+luH0GeZDtnoeHxQhLRb0Go24e3/2/eQAAaxARPOuWb8hhNGBeL5ehxG76o2n3HibThUIkdMoofuP+kMqr+mFb+kPfvycL0nLECrpLxIRWA0hJgQSZah/EHyMq4ql99iqaxoeApHYSt8hdNKqQdBA1dMBer5ubGvFMHnP1emKCQNVo7TA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8DIosnqOWGFq84M1MwFZbsweglLBrZR4689wCKGnoEo=;
 b=WTHfwF0DMLPA753PCnSHlnYAIx01fG8ynSdC2nMEC2hgmfYwhF1IdpJoyHR7sjX6BuoWpDpZyA6EIwwegF/U8dM1Nzg2wYgVoAUtK0vp2s9zMH5FqPIRWFZ/K9LAXUsuuht/bqiqGAsgVBl+ny1q16n6MYKewc3ZFNbrlgO5yD1Q4mycL3s3ex41h+8rH34E+Bvidopp7Fx7tup/P4dXkOZK19E6w3Kfs1mEbcy6Tsw7jdBtvh2O4tCHN5H17kTDxLAYfkbovMRbCNKfZEGj8yPNZMywwDM6lTOiVlL7sqJs5P8xl3d2f87dZdP/yZe3mhXpY77EggI85YcyWasPAw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8DIosnqOWGFq84M1MwFZbsweglLBrZR4689wCKGnoEo=;
 b=tKS3XzxezhVNWqq8dERxaqZinaSdwFM6ZV9z3GkNvtjlaOiCSyD/nGXf4uvdasoaaHqg7GIAuH/36H6x9n2xEO/ANsZ9g7NtDFVWJ0MeR1u18+WUu0tBGohqwUimPVDJNCEGnadaKWQNxHEYikEVVldlZ12NEXc4eDgtB/XdXBM=
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Dario Faggioli
	<dfaggioli@suse.com>, Bertrand Marquis <bertrand.marquis@arm.com>
References: <20210906110057.15384-1-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
Message-ID: <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
Date: Mon, 6 Sep 2021 12:14:48 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210906110057.15384-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0479.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a8::16) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bf2db25f-b70b-40e5-c3cf-08d971278cc8
X-MS-TrafficTypeDiagnostic: BYAPR03MB3991:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3991EF2368100BB28F15D143BAD29@BYAPR03MB3991.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dRMdJT59jD9HEbsbKrDV21D/67nPcMkBsI+56z0WjGfIDNFw7MgVMvf7twq7lmI9hkAPslZnIDy79AhiFxSkeerzWahE7stTGUmaV96GGlqkZgYy4PIXfK8gUUPFQ0Ae9l7Ofm2bH2cBPao3OpcjhjG05eaJZ6dPSTBkKuYpE1VldiqXmv0z52gMeFTObWz5HIfluoViFbBjK3iq/wsCh7jlyisjQaTNcUmpuLTW04j0n6XBJqwPXf4+zQ6F7DtqQS0WQM1HAwbaeQLEZ+6HO75zdyrXFZyT3+1/UgWcvmr+grhCkU5yPht/fWWemGqaXtuA30jIf0SFT2FWYS1PjibV7jYXQ3nsHBND8d1cLTaU2Spvm/AlKaUo45wmN2ECkbWnzmROfAcmkQAd7Hf6wxH2zsMyf6Ik3+an2moUmzfujCj6LYMEu3GMfgxhcor8FvRC1HQ4k9AvEVuEWhaePJpM1NFDK7iLtNlyFJnscuApeJsB9S5/1XEe7HLdaxm0JOvbXQ5FoNL/YWxnpthbfuvU13VNKzIqri8p6wG8EA7/PGVGecz7NyAQanpirPaqG2defy6Gy45v+v8sLVJhHyBII+A3lf3JF9bagp+o0LXF0/iz8gcwEZlUuYSDfcims3kXNIif3DiIQ+PUflmBju2AZX9v+xaDnDYRBlYAiFTHh8qER/4M3/bIRjzk/OWIkOXkwbpKZbJniGhdHWJGBexqjk5tvwR9Pujsdn9UYR4=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(39860400002)(366004)(396003)(4326008)(86362001)(66476007)(66556008)(66946007)(31686004)(38100700002)(31696002)(16576012)(316002)(2616005)(54906003)(2906002)(956004)(36756003)(5660300002)(478600001)(6486002)(6666004)(26005)(8936002)(8676002)(186003)(55236004)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Tlg3T2hYTU5oZGR1Z0dub1UySWZDeXNpSndIRGk3VmVMTndDUDBnQ0l3SmVF?=
 =?utf-8?B?bmZxVS9JVUFDSzR2MDNoUkt5TnlyQStRbkx4VVV5Mlh6NlYyMUxrUHlZclov?=
 =?utf-8?B?U29WQWJ2WjlOaDA5WkU1bVJ0c2N3K3lKdGxlNEhKZlZOd0FNWEJNdzlHeUx1?=
 =?utf-8?B?cjZldy82OVNZNnpvVXlBWS9DQ1VyTFdiWHVTckhrZmthRXBkQm90NVhzZEpm?=
 =?utf-8?B?RFh5WEo3NVZwQ3d5MnBkVWNWTVA4a1N1OERBVkpxeWpJZ0VvNlRwRWpxem9G?=
 =?utf-8?B?VjZpb0xEUUhwSGdZVGNmVHRJWkFFM01pOXhETDY3U0F4Sm10YVlXa25jNmR2?=
 =?utf-8?B?UjQzT0hEQVN1MjY3MGU4a2hFR0RLUk5tV01aZTZVbzQvTE4wQkNYbWMwNXBx?=
 =?utf-8?B?ZUsvaitHbEFmZzhEdXZ4ZWVBdHowMnVrY1pZdVFxdGp2MzBmT09Nb1RNQmMx?=
 =?utf-8?B?SGo0TVROckZkS3NRRWxrNjArbEVuQ0NPSUREZFY3L2dBaGxpVmNINUVSWWFW?=
 =?utf-8?B?bG0yQW5VM1BKU2lCQUZoT09ZQ3JCQXRlREdtV0Y2L0pON0t0eEZmQTZhWUlS?=
 =?utf-8?B?Tm5qMkJXRTNrUlpzL0pOOFBIaFI5anp0dzFPY3RGZ2g3Ymx4K3R6M0tBMS95?=
 =?utf-8?B?VkhOeWhKSndITFIxQXlhWkduVnJ5VklyRmNKWmtlamZPMDZHM3lLMzBOVnp4?=
 =?utf-8?B?RFp1SVJtb1J0NkpKcnpiWDRTNFEwbU5sZHE4SVlDN3FYd010d1duS1hVaGx3?=
 =?utf-8?B?b1Z3amFJam5LS1YyYUVYWEFXbngvWi91WHhNWURHVis5TEYyeGh1WER1cy8z?=
 =?utf-8?B?VmFTUUlBQ0dLait6V2g0MEFYQXBMTTM5YjdlYm9qeGJqTlJjN3Irc1VRZDhW?=
 =?utf-8?B?Uk5GU1RycGN6WlREQ0R3UFJ4NGhrL0dPem9WeWlqK0JZYWo3NUlsYWJoZnYw?=
 =?utf-8?B?NS9qRGtyWlp3Y1RyYXo5ZUpTMWEvZkNqdTN5UjlQbENPNXJ6WHh0eXB5QzJG?=
 =?utf-8?B?bkg5ZFJadWh0WWxUN0dLbEhVaHF2U3YxRFBEdnRvcUM0dUV4VFRMOHZ2ZWVu?=
 =?utf-8?B?c1d3Ulp4Mmh0TW1lTzVIeVI0RGg4RVg5WWVHMzFzWTdoR3YyeFdBcTloN2k3?=
 =?utf-8?B?VDNvR3ZITy9UYXBJbVRpV25CRHluUjFiWWRiTHZrMi9yOHNReEMxRzIwaytv?=
 =?utf-8?B?M3orSm9TTUlKaXBzTFJqaUtoVGxHTkdEcE9oWE8ybDRvM2FUNXhsc1hYNmph?=
 =?utf-8?B?TGp2OG1CSXhKam1PYTNxQVdub0VzMmVCS2d4RC8zQkFnckF0TnFLZkdUcyts?=
 =?utf-8?B?bjNJNHREd0JiWUUvY2J5SUhLSkVhYU5IUGI3cEJlcWhNR3hqZTJEbXpDNCtT?=
 =?utf-8?B?YXJHQUhDbWsveHBJM2l5LzZldVgvRy9abG5KR01NVHgrOVhhRTVWMmhENExw?=
 =?utf-8?B?RElra3Y2cktLTEtwaCsrVmVYWHpmQVgvQlRKdUJaZ2R4Qlg3TTdNMWdvTHNH?=
 =?utf-8?B?MFBKbDF2clN2TCthKzZIYkFQWmZaOVArVk9wRVNaT3U1N3VDM25oZ1pKMFc1?=
 =?utf-8?B?U2lUMjBkL2d3VFU0MXpQOFRFN3JHd3hOTkdXZVBoblFjQklKeFMrUyt2Vmta?=
 =?utf-8?B?V2hVTkNqTlNNMWppbjNLcjI4N3NKOTRrSklKd3NhUE9RenRZRzVHSXVjZG1R?=
 =?utf-8?B?bC9sTEljbUl0K0QvbkYybDRqcDNMNjhZd2VSQWkybkxwMjBCZFdXcVBVVDMv?=
 =?utf-8?Q?zBUUJvdpOL/EisBr6qEN0nrQZf2z3ORwgIjAZVd?=
X-MS-Exchange-CrossTenant-Network-Message-Id: bf2db25f-b70b-40e5-c3cf-08d971278cc8
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 11:14:53.1255
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fMhuUD/SMf1NFDcp42cw3DqgZen+tXZIJFW8udj0JUiN5rya9nJktGpoElUOeBqcwxmUSY+AaLqChpX6+6wBUG1ZGyE+nSWZgtiPx2+Ln6Q=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3991
X-OriginatorOrg: citrix.com

On 06/09/2021 12:00, Juergen Gross wrote:
> In case a domain is created with a cpupool other than Pool-0 specified
> it will be moved to that cpupool before any vcpus are allocated.
>
> This will lead to a NULL pointer dereference in sched_move_domain().
>
> Fix that by tolerating vcpus not being allocated yet.
>
> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpupo=
ols with different granularity")
> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  xen/common/sched/core.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
> index 8d178baf3d..79c9100680 100644
> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cpupo=
ol *c)
> =20
>      for ( unit_idx =3D 0; unit_idx < n_units; unit_idx++ )
>      {
> +        /* Special case for move at domain creation time. */
> +        if ( !d->vcpu[unit_idx * gran] )
> +            break;
> +
>          unit =3D sched_alloc_unit_mem();
>          if ( unit )
>          {

I think the logic would be clearer if you wrap the entire for loop in if
( d->max_vcpus ).=C2=A0 This loop is only allocating units in the new
scheduler for existing vcpus, so there's no point entering the loop at
all during domain creation.

Also, this removes a non-speculatively-guarded d->vcpu[] deference.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:18:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:18:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179646.326026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCeK-0002eE-9n; Mon, 06 Sep 2021 11:18:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179646.326026; Mon, 06 Sep 2021 11:18:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCeK-0002e7-5e; Mon, 06 Sep 2021 11:18:52 +0000
Received: by outflank-mailman (input) for mailman id 179646;
 Mon, 06 Sep 2021 11:18:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNCeI-0002e1-NK
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:18:50 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 35136e66-0f04-11ec-b09e-12813bfff9fa;
 Mon, 06 Sep 2021 11:18:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35136e66-0f04-11ec-b09e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630927128;
  h=subject:from:to:cc:references:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=UGxqvqQGjENugmIZlJdv6hd3iip9nYlhxFfwYqsejpA=;
  b=O1pZjZRvTM276/T8roanNsFyDFoG4+krJRgHIG+XSDegMVOmoCGKLjMe
   uVyP7QsojVGfb9KYF8Jkw4V87w1OYl2/Q182Mdh5cuNk+vUPBrr/VJYYM
   QtehVa9S0QqxVhdk/6qtpYRZmLPpwnXeCFiRF7CUKe2VwE01ibIglJUeU
   E=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 9UqJlYe1eqL2Dv9Z2t16LuKYX0yFqSP+7scu9rBlrvCeY3EYfNfMIda9FVfG+YfjFUVBp8VDMt
 vRRFR3Ealss/X9PCPiDpJs5iKxkEH0ezMAuCDw+Pd5URBhkeYSUpYHKUGS9qiwYYeCWn1JfU06
 Pqkfh+6wHTzL5tEOgjBpraWtQxEhPpRJjfaRzz2fdHigUEdyJIJxkLlvVX20ptI9Eqn3XHv70S
 kw98ZgxjuxdL4RSVnysg/0MZ1TyKl7zOnJ26kJo548hV9m2rbM0fuM8dL9ko3P/CEu8IqOAGxx
 iZ3K+XuMJbIKlZrgJOI2cLn3
X-SBRS: 5.1
X-MesageID: 52082718
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:1Ym+2KACDVsh0T/lHeglsceALOsnbusQ8zAXPh9KJyC9I/b2qy
 nxppgmPH/P6Ar4WBkb6La90dq7MA3hHPlOkPYs1NaZLXXbUQ6TTb2KgrGSuAEIdxeOktK1kJ
 0QDpSWa+eAfmSS7/yKmDVQeuxIqLLsndHK9IWuvUuFDzsaEp2Ihz0JejpzeXcGITWua6BJc6
 Z0qvA33QZJLh8sH7WG7zQ+Lqf+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+SemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lkdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNxN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wiJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABCnhkjizy1SKeGXLzMO9k/seDlFhiXV6UkXoJlB9Tpc+CRF9U1wra7USPF/lq
 /52+pT5elzpmJ/V9MKOA47e7rCNoX6e2OFDIujGyWTKEg5AQO7l3fW2sR52Aj4Qu1F8HMN8K
 6xGW+w81RCIH7TNQ==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52082718"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nfjvxeuYzk9xRximLHfT1uCp3BqMPTwM+t1MFhP7VM4LKf4cuYzKWsvXDDp/fsVWsJIYWSzMREH96vwZxb2kNTkh/srzEdU37ZlVkYtFL6KJJ91uCBlZFCg/RS0PuZHaa0OEyf0sz0Rn8UZ3qJbGqan6nBpROCVkmCjMmD+KYUJJBJnRDjQGpoWhB7QGVcQ+TJ54OwTHe51aZ15TjlJRaixe4tPX1KffAVPScdFAyjhpG95GuWPrl0npPB2ZbF05ovnITDtrPF3Z/d0HkfjGp95Trs3AE/B9OtTFajeBVadT9Jky6lvUZjcJscmUNUWbZMacHJCqI7xX10/+cEktRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PRyRSKeSK8xNzfD/CkdS3fjOL8eQZf6W+On4hkPJgyY=;
 b=EVZATBHR2QTHRMBDr6eciyk5kLKhOS+R3maEnPpRg+DhwVcU5VV6TZ+1e2D2ntozFc7PbEKPN7IlK1JFghYz9MZGXC6rHLDhA33J9RTC7XwLkexUYqbINpZ9qKQ+55+jSUBjyuB226eGRBM0G9s4ln9ZInYdqkX2C5uio+xHzd/ND3O1iXkaAeRjy8Ywt6jBYtlnzshvMf9iR1qyoOwDUziLuauTjbLdpN/vM5qjm5W4q3jASWD5MasqEihSZS0hVI8BmsKw4AiH+vmIQ2e7U7lF2W3uVi2uOgB8EUaSwLz/x1p1TB4kWDAOeIz8XONpyRuefMJt8fNLJ9PMBF8YZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PRyRSKeSK8xNzfD/CkdS3fjOL8eQZf6W+On4hkPJgyY=;
 b=mK6qUL4ivudI9fYpmaG9Uy/AAWlLffwiXIRivkdvlmcU0OBn0bNz5wB2SOhhibADxvbbPIu73iSGtOeRUgu/6ou95Rj99aTt1r18SW8WmfVg2NGawzsxnUCnNfaRTbUBnq5CoqNindzC0jme6r/EvAj5sCOQV7aePu6w5ZqXRco=
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Dario Faggioli
	<dfaggioli@suse.com>, Bertrand Marquis <bertrand.marquis@arm.com>
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
Message-ID: <2a42660c-e1ae-c791-784a-da39e64473f3@citrix.com>
Date: Mon, 6 Sep 2021 12:18:37 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0167.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9::35) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3192ee62-46e8-4a02-093f-08d97128162d
X-MS-TrafficTypeDiagnostic: BYAPR03MB3669:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB36693622FF85E26EC78E8FC7BAD29@BYAPR03MB3669.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: jt4BjobyesaZRW/Oxf4xa/EFF/Ar6/NQ6VO64zIE0xRXa/4CuAMnIcthjTlmFLww5OF3nUPqSx1X9zuB5uFAeER1jD21VpdVE7VuDpD3wsgCRhbXPFQBEJ9EQdfgXLSR0MKXVxZ2Q6MCkYXIdUZm3kHOJSNLv3spKs3L6JHkLm06CksIfhgtE+xiX2BOwYRrkR7EIwS1e4Ghq3zsduo4jDlELPSNqq1js2AioLaN6Kwv++wCi/AKATjbpRzSQKizg9zXqsKGeCICM2Cf8hRY1SY+O4NGdzvVE36Yw66D8LmefaGJxKZ4YSsjTwcrNedevaYhFzmZ2sd3UQrcdX0htAJJEn5R636kE3tkSPCOpeBoGgkC/5ZJeYts5hiC5buXcTZM8jZduv9ckui+dBHRzqoue6rLCaa6d5Qk54Q8K5hLmfbP11N8ochgkHrcZUZALnm4DE1y1V77xPrb2eX7YO6cTpDZrPkEeGh7JumfuqGulZW47eMpma1L7PBsE5v5zTvQYaZoi51DEPIXowOdv1ibAsZtDaaTNSvZwYNuD2NhRCICLVKPPnHxgwV1OFb2LjRvKxGe+ukI1MQIxgPm+TU6VRzEyXqgCUOzL0Mx4f6RZwoYsqZea4ZWb7/iB7A6TIJPtT9ocENehYnS0W5n5LxCVmeigTCA5910ayBSykn2KMb4OxoYWaBJbLJIo49xrG6ktewQv+xAsnnOLWuoIQ3Sy2YtBCkJ1fsjFyVe/EI=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(376002)(346002)(39860400002)(366004)(66476007)(31686004)(8676002)(36756003)(5660300002)(8936002)(31696002)(86362001)(6666004)(54906003)(16576012)(66946007)(55236004)(66556008)(6486002)(316002)(2616005)(956004)(4326008)(478600001)(186003)(26005)(2906002)(38100700002)(53546011)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aEhtdkpnMTU5WjdNZzRnMUVLYVVhSlhwVjZ3bmZKYW5mQUM4eVRRcTlYTXI4?=
 =?utf-8?B?ZEprM21pMTREVzlPRExrZjZjQjFsQWl1b0RVNHRkSGR2cS9MemkxdUZHZUQ2?=
 =?utf-8?B?YXMvSUUyc3k1dnFLVXdqd2s3WTJjUHJqUXBSRTRKTUtsQ1VRVjQ1c1R0ZUw2?=
 =?utf-8?B?VGJXVzZ5c1V3TUtPTFdYZjllWHAwQlo4eXNRTjNwQ1BPcWwwOG5wbzJFT0Fz?=
 =?utf-8?B?blV6K3AxdDZCSjU4OWFLSENxNHB3bHNuT2JKVWpnN3ByK0N1UUJvUzBQdlhU?=
 =?utf-8?B?UG1mV2cwalVkQ3BsVGpocnZmSVQwWndQUEtBOXgzOXdTT1lVeTFYcFplMFp3?=
 =?utf-8?B?c0Q3bDkvb2NCNzE1eXl5UWVwR0FHWng1MVFKQ1F4dWNPSHYzYTc0aDRPQ1c0?=
 =?utf-8?B?TUQzeTJrNlNtSG9xN1I3S2FLOU9PeUJsd1l0Q3E4eHZwcWJuT2c2MjFNQk5E?=
 =?utf-8?B?UWFNN2hTY0ZrUDZXY3Q5cjRCdUdxNVN4QXQ2dCtPZFYyWmhmU0VSdC9yWDZJ?=
 =?utf-8?B?SmNLTThGa3BWS2Zucm5CWFZUMS9uVXBQWHNlZm9PYm9LdGg5YnJKK0lFVHh2?=
 =?utf-8?B?aGY4bUFvMll0QjZjWnZGYStJTFRiUVhFbWUrdDZ2SzNWalJVbVpES3QvZ2NK?=
 =?utf-8?B?WEFybURzN1N2cDU5bjFWQmJHZmFGdFl5OHBYQkJGdDVmTTFKbEV4RXBqUTJx?=
 =?utf-8?B?VW0rQ2sxY2xpUWQ3b1dTNXVXZG5GT2dIa1hwWUN3bFh3Zis0ZEpQbS9DaUJv?=
 =?utf-8?B?dkJKbHBjZk5wUEl4TnllNnArbkl0bVdSVXRIS2hQWVZrOGIvMVNaNE90bmVr?=
 =?utf-8?B?dEkydzZJSG1tbXRVdE9pQ2xNdTFwU3dTWFJTYmRQTTFtd2k5SmhFc1lpdDRX?=
 =?utf-8?B?aS82bkttK3Rod3c5K3c2UlJmWEk4UlJGdCtkdUl1eGtvTjdCN3d2YlhOYzEw?=
 =?utf-8?B?dlFJbldGbXdKVmtQZi8rVFNLWmRmNllYSXVPOFF1cHpWdzU0dWtmd29MVUJK?=
 =?utf-8?B?WkFnOWdGVEJWbnp5eEdITjFjSUcvMExZMmpjZi9UQlA0aHB0bEZoSGRTcENH?=
 =?utf-8?B?RUtiRHVrb0FoNmlwNWpNSEs4czdyVHBmZnFLM1o4TnB0emd6NlFoL0ZIVHNm?=
 =?utf-8?B?eWIrMDFzOXc2YjNQdm8zcllPSFRKSW1vdXpMaWl3OG1reWZXRFc4bjJzMm9F?=
 =?utf-8?B?MnVHeno4aVh0eVN4SmJrdzk5cVlxakRnVGpZUzFDZFFmSGZvYU85dloyS2Zl?=
 =?utf-8?B?ZDV0cWVLbU1ybnR0SU9RZFFLZ3dHaTkwQldOZ3REcEpiNHlzWlZWYWN2Skl6?=
 =?utf-8?B?RU10S25JRjhKK1FjeUp1T0h0L090a3ZnZEx1Y0NSbUxwcEx3YWNGc2VmR0dt?=
 =?utf-8?B?VXhEN0pJTklqM3hZTWVXaVNHc3NkQjRlM3BoMmNiQjJWcm5aa3J3SGpNT1pi?=
 =?utf-8?B?K0xaelpvdEZwNzVNRVA0NWFNK1lVTVNDVW1yUWZrS0VRdlVRZlBwSnFrUzBU?=
 =?utf-8?B?WXF0ZmJXUlBNaDQ5d0U5OHg4SmVZQ1VJRTE0dnAyRnc1SE1WOG1teWI4MXlJ?=
 =?utf-8?B?d283anFFdUhFeUtUamRJdXAyd0RDejdqb0tPZTRzTjZ3SXMrUFRjcFJISzJq?=
 =?utf-8?B?cnNkWVYvdlNMUGlLMmlsdjY5T3VUbHJ3WmpKYnlsTzBFTmVIRHVCSjBKS2Vi?=
 =?utf-8?B?S1Vnb0RZaWpJZ2xSKzI0WTgvVlJBbDZvOXZ5cmdBZjZJQlZRbmpjakw0cVk2?=
 =?utf-8?Q?pVQ0VIgY854DbH4SZBk71eKkHJGJM/HlUryC1NA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3192ee62-46e8-4a02-093f-08d97128162d
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 11:18:43.5795
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: X6kutlrTSiw1q/o0x5sTJaZ4ylVdG1TLZSnRidwCxdf190au1FI7WTdxf/H+bNsLcMQCjVbX4nh+Y1ofy6xc95OQjPLEzNV1CPMq4k+An2g=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3669
X-OriginatorOrg: citrix.com

On 06/09/2021 12:14, Andrew Cooper wrote:
> On 06/09/2021 12:00, Juergen Gross wrote:
>> In case a domain is created with a cpupool other than Pool-0 specified
>> it will be moved to that cpupool before any vcpus are allocated.
>>
>> This will lead to a NULL pointer dereference in sched_move_domain().
>>
>> Fix that by tolerating vcpus not being allocated yet.
>>
>> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpupools with different granularity")
>> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>  xen/common/sched/core.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
>> index 8d178baf3d..79c9100680 100644
>> --- a/xen/common/sched/core.c
>> +++ b/xen/common/sched/core.c
>> @@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cpupool *c)
>>  
>>      for ( unit_idx = 0; unit_idx < n_units; unit_idx++ )
>>      {
>> +        /* Special case for move at domain creation time. */
>> +        if ( !d->vcpu[unit_idx * gran] )
>> +            break;
>> +
>>          unit = sched_alloc_unit_mem();
>>          if ( unit )
>>          {
> I think the logic would be clearer if you wrap the entire for loop in if
> ( d->max_vcpus ).

And of course, this is wrong.  Turns out the domain_has_vcpus()
predicate still hasn't been committed, but d->vcpu[0] is the correct
internal.

~Andrew

>   This loop is only allocating units in the new
> scheduler for existing vcpus, so there's no point entering the loop at
> all during domain creation.
>
> Also, this removes a non-speculatively-guarded d->vcpu[] deference.
>
> ~Andrew
>
>



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:21:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:21:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179652.326037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCgg-0004Bz-N9; Mon, 06 Sep 2021 11:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179652.326037; Mon, 06 Sep 2021 11:21:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCgg-0004Bs-JP; Mon, 06 Sep 2021 11:21:18 +0000
Received: by outflank-mailman (input) for mailman id 179652;
 Mon, 06 Sep 2021 11:21:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Prhc=N4=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNCgf-0004Bk-Er
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:21:17 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0bbcb4be-55f2-430d-a5d9-8eb7ee204be3;
 Mon, 06 Sep 2021 11:21:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0bbcb4be-55f2-430d-a5d9-8eb7ee204be3
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630927276;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=ytoXKKZBgq2BBDO9li+7iEmPK3N6KZ6iFt5jVQxn6DQ=;
  b=HsWUEmMj5g5K5a0fpfwxhzJXRVOlbEDI6ahyamk9SAP515a4BX7KhCBy
   otlRaAnNHcekaN2/ORxXymUEgAn9Gc74517r4J+y1OjKPJzdUNMO+Ns70
   7EqEOoVFxzzpHUOQu02u2R4LAHW9nBPthonNWavsGGHDQZV9x8eerSVSn
   w=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: xeB6+cUldNGZTvDItGjVErmqjASTCsTVVfkPWK+6B2Rqqv5Hv4n6V7J9v6Tl/NsoMBR8fdwpuy
 auNR9FNlKJHExAJ6L3DIz5rfs/gdicgXRM71dDSYfZSu9WF1Nzw8llywlQ+QQ6Cuh2nEzOl/pV
 yPqQH1MTHF5Gb83QuCrgthhX0dk204VXcoZR8aMWhEw6CHS2x014BFOShN7ncfhqPwAnjr+YVZ
 Orm2iTV2v7dXFCY2Rr48afLqC40tLiL0bNjHZOjwGJf/rsTOi/EnbStdcnfngtzr9Y7MOaJx7+
 2wyRtrKAUQ4db7TVnTxuUO/F
X-SBRS: 5.1
X-MesageID: 52484249
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Ith7NqFoh3gH0OgxpLqEP8eALOsnbusQ8zAXPhhKOH5om7+j5q
 WTdZUgpGXJYVkqNk3I9erqBEDEewK+yXcX2/htAV7BZmjbUQKTRelfBMnZowEIcBeSygcp78
 hdmt9FZOEYY2IRsS+w2njdLz9p+qjhzIm4werRi2xgQmhRGsddxhY8BwjeCUtsS2B9dOME/F
 nw3Ls9m9OPQwVuUviG
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52484249"
Date: Mon, 6 Sep 2021 12:21:06 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, "Tim
 Deegan" <tim@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 01/51] build: introduce cpp_flags macro
Message-ID: <YTX5oi25Ac3ANEiU@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-2-anthony.perard@citrix.com>
 <ba3cfb06-1d89-ff62-1012-6edc2a9c5259@suse.com>
 <YTXoCQ3SSvN4eU/C@perard>
 <280c5579-ee62-ed36-1f08-58fe5b9ca34f@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <280c5579-ee62-ed36-1f08-58fe5b9ca34f@suse.com>

On Mon, Sep 06, 2021 at 12:30:07PM +0200, Jan Beulich wrote:
> On 06.09.2021 12:06, Anthony PERARD wrote:
> > On Thu, Sep 02, 2021 at 12:08:58PM +0200, Jan Beulich wrote:
> >> On 24.08.2021 12:49, Anthony PERARD wrote:
> >>> +# To be use with $(a_flags) or $(c_flags) to produce CPP flags
> >>> +cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))
[..]
> Well, yes. This way one could also pass more than just the expansion of
> either of these two variables. The thing that made me think of the
> alternative is the comment: Would you mind if I inserted "e.g." in there,
> to make clear this isn't limited to these two variables?

Adding "e.g." is fine. Thanks.

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:22:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:22:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179658.326047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNChV-0004mD-0J; Mon, 06 Sep 2021 11:22:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179658.326047; Mon, 06 Sep 2021 11:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNChU-0004m6-TA; Mon, 06 Sep 2021 11:22:08 +0000
Received: by outflank-mailman (input) for mailman id 179658;
 Mon, 06 Sep 2021 11:22:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=t7g0=N4=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNChT-0004lB-2U
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:22:07 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6da9db43-d88a-499b-b0fd-cecfad644a93;
 Mon, 06 Sep 2021 11:22:05 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 0F59A1FEDA;
 Mon,  6 Sep 2021 11:22:05 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id D706F1395F;
 Mon,  6 Sep 2021 11:22:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id vfZkMtz5NWHMHgAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 06 Sep 2021 11:22:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6da9db43-d88a-499b-b0fd-cecfad644a93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630927325; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=bZxp9VPY5c6s8wGWdcldGfjpDN2Ri4rZPWfB5DP4AhM=;
	b=bEaIQLE5MuOXofj7XevBMD0xy+9D6CkEGI6CYk3d3+8iq+g/iihWrHggeKk2q1oegYM5nh
	CURX71E2d7T1M4vuz5eax8R5OIoub50wNLYiv1z+l+wn5AYA2uuxlnskgfja76+bxCNWMb
	f0qRM0QDeXn+VI593KSgj6yLSr62CBY=
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <72a7728d-1b30-5cc7-981d-50471e77126b@suse.com>
Date: Mon, 6 Sep 2021 13:22:04 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="DqXVcNp9qsxotgnM55gXR6Fc3Gf5QKbfg"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--DqXVcNp9qsxotgnM55gXR6Fc3Gf5QKbfg
Content-Type: multipart/mixed; boundary="VzyTPZaAbTyoAKrBL7tBm5kVgPMOxo8kl";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
Message-ID: <72a7728d-1b30-5cc7-981d-50471e77126b@suse.com>
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
In-Reply-To: <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>

--VzyTPZaAbTyoAKrBL7tBm5kVgPMOxo8kl
Content-Type: multipart/mixed;
 boundary="------------DF3084C52409BDBB90E4E232"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------DF3084C52409BDBB90E4E232
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 13:14, Andrew Cooper wrote:
> On 06/09/2021 12:00, Juergen Gross wrote:
>> In case a domain is created with a cpupool other than Pool-0 specified=

>> it will be moved to that cpupool before any vcpus are allocated.
>>
>> This will lead to a NULL pointer dereference in sched_move_domain().
>>
>> Fix that by tolerating vcpus not being allocated yet.
>>
>> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cp=
upools with different granularity")
>> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   xen/common/sched/core.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
>> index 8d178baf3d..79c9100680 100644
>> --- a/xen/common/sched/core.c
>> +++ b/xen/common/sched/core.c
>> @@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cp=
upool *c)
>>  =20
>>       for ( unit_idx =3D 0; unit_idx < n_units; unit_idx++ )
>>       {
>> +        /* Special case for move at domain creation time. */
>> +        if ( !d->vcpu[unit_idx * gran] )
>> +            break;
>> +
>>           unit =3D sched_alloc_unit_mem();
>>           if ( unit )
>>           {
>=20
> I think the logic would be clearer if you wrap the entire for loop in i=
f
> ( d->max_vcpus ).

No, d->max_vcpus is not 0 here, otherwise n_units would be 0.

> This loop is only allocating units in the new
> scheduler for existing vcpus, so there's no point entering the loop at
> all during domain creation.
>=20
> Also, this removes a non-speculatively-guarded d->vcpu[] deference.

I don't think this dereference is a real problem. In case you are
worried about it we should replace the one further below, too.


Juergen

--------------DF3084C52409BDBB90E4E232
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------DF3084C52409BDBB90E4E232--

--VzyTPZaAbTyoAKrBL7tBm5kVgPMOxo8kl--

--DqXVcNp9qsxotgnM55gXR6Fc3Gf5QKbfg
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE1+dwFAwAAAAAACgkQsN6d1ii/Ey8T
+gf/d689AFSV3hds3y/QXlpXUIPF3DagRR4HOLSSJeQ2q7/NgGTQ9fMDm0p2qMwCP4fIjNzHnsg9
Jb2W9kvV/y7PJ/LyhoXEwSyXc51zFqPgH7lJrCTP59lq2N7n6pxK8GxDOyX4maaIpQNmzDkwvCmz
3vIjbNbKzShLHV+0CZajwxoPEGaLOPcV10gI+VnfWU9Z+GtieDroDm028sqopjiz42yqCyE+eeHW
YcDHIPGrlwknjlZnQowcXfadMDvYKBX9Cp397jDP8z0q6a0oxQOeukAG7/tJ3hTv2EE70CUwGt9j
GUv6OK3UfK9ujo3fkbBN4kznB/dMJsLrx4g/rIkJWA==
=vY95
-----END PGP SIGNATURE-----

--DqXVcNp9qsxotgnM55gXR6Fc3Gf5QKbfg--


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:23:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:23:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179664.326059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCip-0005PU-C5; Mon, 06 Sep 2021 11:23:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179664.326059; Mon, 06 Sep 2021 11:23:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCip-0005PN-7O; Mon, 06 Sep 2021 11:23:31 +0000
Received: by outflank-mailman (input) for mailman id 179664;
 Mon, 06 Sep 2021 11:23:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCin-0005PC-A8; Mon, 06 Sep 2021 11:23:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCin-0002Ut-5w; Mon, 06 Sep 2021 11:23:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCim-0003d5-PA; Mon, 06 Sep 2021 11:23:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCim-0005DX-Oi; Mon, 06 Sep 2021 11:23:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6sEuM6y8dZxSOdmA6FXON8rgpZtbf1M8ejPiOziwm3A=; b=Hu+wiVpMyuGvExkvKEiBthljaE
	VSAixurXyY0A8hAkkRXcIuamhtA9W3Xmh/dDjy1/8qfb6IJoSMH71ZBlg2Ov0+Xg3JcpxLwWbn+nU
	5ArZiGgDc50CvtIX5kK1lC2ybwguPDvtf9dEZPVkquvSitUhZK4V4PGDxvRsPk/B4gpw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164848-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164848: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 11:23:28 +0000

flight 164848 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164848/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine    4 memdisk-try-append fail in 164819 pass in 164848
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10     fail pass in 164819

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164819
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164819
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164819
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164819
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164819
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164819
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164819
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164819
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164819
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164819
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164819
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164848  2021-09-06 01:51:30 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:23:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:23:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179666.326073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCj0-0005np-SN; Mon, 06 Sep 2021 11:23:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179666.326073; Mon, 06 Sep 2021 11:23:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCj0-0005ni-OA; Mon, 06 Sep 2021 11:23:42 +0000
Received: by outflank-mailman (input) for mailman id 179666;
 Mon, 06 Sep 2021 11:23:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNCiz-0005ms-Kl
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 11:23:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 79dfadaa-d577-4815-9aab-3aadf84dc4c8;
 Mon, 06 Sep 2021 11:23:40 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2059.outbound.protection.outlook.com [104.47.8.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-NqeZ9Tc4NUSIe3A-WzbJOQ-1; Mon, 06 Sep 2021 13:23:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4192.eurprd04.prod.outlook.com (2603:10a6:803:4c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Mon, 6 Sep
 2021 11:23:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 11:23:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0176.eurprd02.prod.outlook.com (2603:10a6:20b:28e::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Mon, 6 Sep 2021 11:23:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79dfadaa-d577-4815-9aab-3aadf84dc4c8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630927419;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xx0HZQkV4tYv17/m2KPYg7jrcsCcKDrCsNTIkEptX9k=;
	b=WJCvI7C7nNkLsdRwpOT3a4MAI2JNZijyNRB0sH1Se81kGatKSnIrjnSJVFk/sdhcBqj/EU
	WgTXeLSeRHZXX5uLCc/KxMiXXngkHFUk9KThewrIZhYi95fXfYb3AZWgSK/suN5PwmrT5w
	n1SQ0S/Yz/hwDAExs3LFwHsWXbLfPTs=
X-MC-Unique: NqeZ9Tc4NUSIe3A-WzbJOQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nv4AqZAEXed4h/4/31nXsI8HDGBgLiqohdxyp7hkBikhUv74YGeNmwk618m+7Z81++uXeQvZSOTdcfQNqU4xqBQfJAaaE3Pk4nu9cvSQg04WG8F6eP3lNG//PxpVcoT3rAJa+2K00b8nz67uI2uS71M4gADgZ7eRE1Xaj4TvB5QVK60SXXvGf8Z/dh6sfHdWYTmVfdX8rQj+fZ1QIUznaUB4uiAkyJlozRHBRuuIvKXFofzhfS5gzokETtGclqLFbZ1TcNQpcyqvjODBnz3mjPpHGLp5qB1bu/1izvywnGE+4zRfon5cYCREjfFOZRecUz31mm6kf5NfhsvsW8DZFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=P10PInH1DfhGLms21QzcUomgnt98pv9+SUigsNNpRbw=;
 b=ctIUMbPmrAqRPB2SMPfaizx+Dr7nyPBafW3aTN47XsS9fOIzSeciOgSbO3X2/QugePwO0VAp+OVA3OygUEIznoYa8qjcdj6c2vylTo1bA91UHz68TrPpfhO5l+aO+JreUMa8KLhADmGBmvKBYsoUpiAXtBbg4i//t7lQu6j9cigffc7tn06rwD74ED8rKAe70/yh3nxxuUNNfQkhsd5h4EgaaQbJCMyJ8H7KupQhSNHsd/DYbpYDvXaM4sMpG+7rJdqTW0YXfvoZKoDKBKevkCrVwsLui8msaoU8lHbzyQDzdReZlee2PfI669jvkIrNFPbxcKK+7phl0RS5GRuRrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
To: Andrew Cooper <andrew.cooper3@citrix.com>, Juergen Gross <jgross@suse.com>
CC: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, xen-devel@lists.xenproject.org
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
 <2a42660c-e1ae-c791-784a-da39e64473f3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <598640b9-758a-5a0c-83cd-38b3e41d7b39@suse.com>
Date: Mon, 6 Sep 2021 13:23:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <2a42660c-e1ae-c791-784a-da39e64473f3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0176.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 52483e6e-a830-4d49-26f8-08d97128c51e
X-MS-TrafficTypeDiagnostic: VI1PR04MB4192:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4192F81526FA4C65E0B72084B3D29@VI1PR04MB4192.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZSDvWTTR7eQMSLoBbChbX6usSKHGvRClQNok3u/gLQI9UbErC+ymyMkvews1ip1rRARbIKz2SavqwSyqCFbQgtcPOpQm028JNP2307VFw3roPWxjYWt/fyRJX/9HjE4y8SpkcRufemYbw//YWUy1wxb7QE0iZ89IBZRxQoHOIU4h+hCnUuqA//GLXrEl1AitcyERB92DC8fadvLkIFnRmtddTMWMn5wQT5leZvSvN9a86EKVrsqEH6L3UeGAgRQahX+zi2dcqbNaN313kSA3V88+2Sauwuqn4+m79Yta6qiPkSfXI7zkzmY8HPBtIkj7hkV/SAqBj4Wf67ckLW39cP2WhO3cBENoR+5QYqeTO92QZwa5EqysjvCFajV7/9eJvr/6eUt13FJu3T4tre9pZZb1Mp9Q+gv+baB2FFxJM7nybIIsw8rVL+kdXNvQ5kwF8vNjvLgjcNEXlhnqWHAWaPpOThT8BtZcDuOXlGT+v2JnmTrG1CFVC+m6VvW4RXOqDw3dC8jPzjLYtk73/swHi+yOVNJhaJ40Wa3w4zCJx70xbbYpT0tbpk9cB90N0x19Qt6tdNfsPqhGdgr2N46aNISb/Bnl+wMA1k4awJ8E58lGZDCNlujmIIZEaqiXLT/OKJJK6/NKuFF7YM3SwRL6o3m4tZeYcun6zVeRLF8et1RCjzRbssnIriJuHwKwg6RkeiXBsq83plAHx0fg0IcnzuakeYtmtkEb1jauPZcDhNU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(396003)(346002)(376002)(136003)(31696002)(2616005)(956004)(110136005)(4326008)(36756003)(2906002)(6486002)(31686004)(66946007)(5660300002)(8676002)(8936002)(478600001)(38100700002)(316002)(54906003)(16576012)(86362001)(6636002)(66556008)(53546011)(26005)(66476007)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?2NCDR6Fz4fI3v9SC516A+ZGD7IPHdRrqnoqV0C7KuIHG0sABTvhWmJJiaSr+?=
 =?us-ascii?Q?AEp6RGR9VqZtsQlbOhBZZ/BKqMMibEkrfrLj//98rmt64ZUnlip6XhRqiBJM?=
 =?us-ascii?Q?ltBR0qP18KSpMx6opOaXawGMMgxtkMaEhpbJ0+y6YZq82VOwSywlQ7w6vznE?=
 =?us-ascii?Q?WJr0Qnau+wuan3uRhj0C84czggxcRPz1GPxuuJ5hTk/gqrsbrCctYO2VOi2D?=
 =?us-ascii?Q?B+Ml/KYhu7pB6MOWsyCmrFFtG0GkN64W+7vhqOpVMcPD9md2DYOTU1RnkOQP?=
 =?us-ascii?Q?ARV3hWU6i2/NidB6Vgmeup9JoKB5l+LFzM1k7AG6j6FcQ9jVVSPQukL8MoqU?=
 =?us-ascii?Q?2kojc4dBLs13mWAJ2vN8Tft9qqJeZQz29tMdc5q6UQ491RJDH4S7rR6By36/?=
 =?us-ascii?Q?Pbs/n0h4JiNOn+yuJjVgNvXw+7OUav+fQyjTb2F4mWux8tiWVTodQHx+ZOs7?=
 =?us-ascii?Q?n6nECBsbIbTumj2yhSEDZ4MmmlUMq85qp06W8tCxiTPRsr7ttptlnhHfW+oe?=
 =?us-ascii?Q?sBsX9ks2eMKrK14X3DlhmuvSKngT1oZSJdk9D5IgA8muomsyIwKD2RZ5cmoI?=
 =?us-ascii?Q?0OCGgDTjl8oHicvqjB1ykywU0XfLfR/ztxr3YSOv2vzbd3OE6CkQAnVRj3lo?=
 =?us-ascii?Q?E67deN1VjgOoE6N1Keh6or1ArH7DPQOdLnHrlimusvzb9OTxEzAFk5SVrIx+?=
 =?us-ascii?Q?PcDwZpmWi0aCm3oc93AeQ6EmRlZox6duXP1ZeGp8MeUfU3+tirLZpxvUyf2i?=
 =?us-ascii?Q?5GRBIuVf52lWJIjuMAz+LOEEWVXcufHSPQP4h5bUZFJh0F66MzFS4PBHBNGl?=
 =?us-ascii?Q?iDCs/29195XQmoJ30cBibHebNFbGMFfmicJph3M+U8ZXrrFZJxjGndEVQCGI?=
 =?us-ascii?Q?I6b8PYkXDyZT/w1XJ4d0bhFAt7Y7XMqyYwZuwq8yyHjHi+7TlD/8asH0HSzu?=
 =?us-ascii?Q?PYdhn6Z6YYS+JU2HoUf5qLAKujl727nwa5Vb/np4Jre6YCX2LDdsD8HyG9v0?=
 =?us-ascii?Q?uFwnStE/2/jxxFMoBIY8AJAvBU5u4gNP2keMpqwJVItu2eDO2MsX501CHPCh?=
 =?us-ascii?Q?Jg2izlXzh4zsRfGn51ynOMJYUwSALn+glif84FOpdOGUS3cmLJbbhnNq56wO?=
 =?us-ascii?Q?0ugaVF1cPSAK1A50tnZiATkkW9KaPRi0nI/xEOD6pbQqxJLPuFPZB77sQWvi?=
 =?us-ascii?Q?7Qmk3xGZ3wvDUa95WESOK057MD9+Nje5TN8x9tii/j8/fbkFBSv+F43KLDuB?=
 =?us-ascii?Q?Cu8iNeIcHd9LfEFVxIdbZ9hrzTOfPZ6z0Z9TYDsweVqIOoCtOqNm9wMAUtVc?=
 =?us-ascii?Q?AbRQQygqrFLNNK2+9ADR2QNb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 52483e6e-a830-4d49-26f8-08d97128c51e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 11:23:37.1023
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Fq2+jMR1+ZTo0z/4NiHsmI0dSAg4WalspNEz8xgiQL4ZCOpyjNBrpLjJqcBf8ZscmfYmH9YRvl56TGCeOU++Og==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4192

On 06.09.2021 13:18, Andrew Cooper wrote:
> On 06/09/2021 12:14, Andrew Cooper wrote:
>> On 06/09/2021 12:00, Juergen Gross wrote:
>>> In case a domain is created with a cpupool other than Pool-0 specified
>>> it will be moved to that cpupool before any vcpus are allocated.
>>>
>>> This will lead to a NULL pointer dereference in sched_move_domain().
>>>
>>> Fix that by tolerating vcpus not being allocated yet.
>>>
>>> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpu=
pools with different granularity")
>>> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ---
>>>  xen/common/sched/core.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
>>> index 8d178baf3d..79c9100680 100644
>>> --- a/xen/common/sched/core.c
>>> +++ b/xen/common/sched/core.c
>>> @@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct cpu=
pool *c)
>>> =20
>>>      for ( unit_idx =3D 0; unit_idx < n_units; unit_idx++ )
>>>      {
>>> +        /* Special case for move at domain creation time. */
>>> +        if ( !d->vcpu[unit_idx * gran] )
>>> +            break;
>>> +
>>>          unit =3D sched_alloc_unit_mem();
>>>          if ( unit )
>>>          {
>> I think the logic would be clearer if you wrap the entire for loop in if
>> ( d->max_vcpus ).
>=20
> And of course, this is wrong.=C2=A0 Turns out the domain_has_vcpus()
> predicate still hasn't been committed, but d->vcpu[0] is the correct
> internal.

Which in turn might want to be done by setting n_units to zero when
d->vcpus[0] is NULL?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 11:34:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 11:34:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179679.326084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCtT-0007lI-SR; Mon, 06 Sep 2021 11:34:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179679.326084; Mon, 06 Sep 2021 11:34:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNCtT-0007lB-PI; Mon, 06 Sep 2021 11:34:31 +0000
Received: by outflank-mailman (input) for mailman id 179679;
 Mon, 06 Sep 2021 11:34:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCtS-0007kz-56; Mon, 06 Sep 2021 11:34:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCtR-0002fx-Pr; Mon, 06 Sep 2021 11:34:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCtR-0003zb-JV; Mon, 06 Sep 2021 11:34:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNCtR-0002ju-J1; Mon, 06 Sep 2021 11:34:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0eDTU8jfiGc8cMZ4DBxu2q7eVcDUGX1ebfvzaY6rne8=; b=tZcR3BjNMopZvKEQI/mopTV7qm
	4M+U1/lxeSaEvJ19xX4z20WdAzWephgl52S77SihPh1ZkQ5BNx8rGKTZtgtzDLc+evPrZ5YS/ogW8
	PIPn9vAqMxHgkDPnzcVrWRt1NVqWMiQnlxo1qBNjLPNUvmQZgsYHomPaegWA9M2Vcof0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164856-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164856: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 11:34:29 +0000

flight 164856 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164856/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   64 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   38 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:02:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:02:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179699.326101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDJu-0003H4-Gb; Mon, 06 Sep 2021 12:01:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179699.326101; Mon, 06 Sep 2021 12:01:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDJu-0003Gx-DY; Mon, 06 Sep 2021 12:01:50 +0000
Received: by outflank-mailman (input) for mailman id 179699;
 Mon, 06 Sep 2021 12:01:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PlzZ=N4=citrix.com=Jane.Malalane@srs-us1.protection.inumbo.net>)
 id 1mNDJt-0003Gb-Mz
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:01:49 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 35354346-0f0a-11ec-b0a0-12813bfff9fa;
 Mon, 06 Sep 2021 12:01:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35354346-0f0a-11ec-b0a0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630929707;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=Pf01GPhpRixaBcZCDc0yr0gLSvWSydMEpj3W90Q2OU4=;
  b=QzizcM47aFmngZqVWQQk3jdKDp7o238ck78qOwlmbzxLXCAoptelpLkH
   n/AZtJHtmheaN/+/OyjPUqeVxqPqwZKRxCgwv9wRl2tG7ScXwuguKKigN
   UnoeWhyPoAFit14aPsVNWX04hT8KEp9oVtl9NPpUz9d+eThaUtWR0jN5O
   A=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: KrnqQUvoYQ1Xtnsjv+2HZft3xh1JM5c3Bmix5ZKZyETZzEyfTYCmerJ8fuCQu/RtSp3h39P4/Z
 CwBl859DY//SyLHaO5NggCnnPmPso640oN4UT5r8dYdl2mYYuriluJZPINeyfUWYth027tHzjv
 0+wzl517PfFLpLV38+vcO4SbCU61UuwLKy8K6bcHAxLsaMjZ5WucHIcll3ysKyTtCwZYFsJGAL
 QibNliFBx9d98Zlbp7kR0XaT/4Lgv/gjSVNlB7yruc96llPuTinH/PvOUicjws91DZBrHxDmvD
 k62KYAMNFG0OosI7ceSUItSH
X-SBRS: 5.1
X-MesageID: 52084907
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:NVjIUKHLX7E6fgoSpLqE5MeALOsnbusQ8zAXP0AYc3Jom6uj5q
 eTdZUgpHvJYVkqOE3I9ertBEDiewK4yXcW2/hzAV7KZmCP0wHEEGgL1/qF/9SKIUzDH4Bmup
 uIC5IOauHNMQ==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52084907"
From: Jane Malalane <jane.malalane@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Jane Malalane <jane.malalane@citrix.com>
Subject: [PATCH v1 0/2] x86/cpuid: Use AMD's NullSelectorClearsBase CPUID bit
Date: Mon, 6 Sep 2021 13:00:02 +0100
Message-ID: <cover.1630929059.git.jane.malalane@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Jane Malalane (2):
  x86/cpuid: Expose NullSelectorClearsBase CPUID bit to guests
  x86/cpuid: Detect null segment behaviour on Zen2 CPUs

 tools/libs/light/libxl_cpuid.c              |  1 +
 tools/misc/xen-cpuid.c                      |  1 +
 xen/arch/x86/cpu/amd.c                      | 18 ++++++++++++++++++
 xen/arch/x86/cpu/cpu.h                      |  1 +
 xen/arch/x86/cpu/hygon.c                    |  5 +++++
 xen/include/asm-x86/cpufeature.h            |  1 +
 xen/include/public/arch-x86/cpufeatureset.h |  1 +
 7 files changed, 28 insertions(+)

-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:02:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:02:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179710.326112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDKt-0003wp-Qh; Mon, 06 Sep 2021 12:02:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179710.326112; Mon, 06 Sep 2021 12:02:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDKt-0003wi-Nb; Mon, 06 Sep 2021 12:02:51 +0000
Received: by outflank-mailman (input) for mailman id 179710;
 Mon, 06 Sep 2021 12:02:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PlzZ=N4=citrix.com=Jane.Malalane@srs-us1.protection.inumbo.net>)
 id 1mNDKs-0003wa-VD
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:02:50 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5b13c8b2-0f0a-11ec-b0a0-12813bfff9fa;
 Mon, 06 Sep 2021 12:02:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b13c8b2-0f0a-11ec-b0a0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630929769;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=FkKh25/O7W8PkkcqGWWAdNe03OfrUwvjmG6W/c0Qi7s=;
  b=W3ElB7tmteFnCQuGFTx74y3yUz8GDGPgoB9yFCsFHVPo9mQGLNc07aPW
   XymkO45suNTqXWM6cuc6MP4JmgndwHhebvaJYr0vE0siNTBYdWHGmtbN8
   G7zOkDZV09Mee+UbftjgiqVP6ONVyX1hTSk6wM4uIJqrApVHAxj6vWIWN
   M=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: qrKOgZfF7Ems9LLXxm7kvfpwSlebblNmSCRhZv7CySQmC/DPOy38zO5mE/RFli4ZnmRNtzZr9x
 BWqzRjpQ5BRE+CSMLG6ubqdb+fH+OUZ8fP/RHhoTO+xaQ/ILxhOpSWjm4TuB7jIgZWOBsNJA+c
 oScgX0tH+LlH1e7V+z0Lh6axejXU7MInsGQ7xpNUr6fb61bjKl31Hp9DRyWvcL0oMNxqC6dsif
 +doS5jx0t+ejYRH6WwxbbUBcDN72QmFRYnXB2ZPpBc7fadylWDko9p00pVuySuBsTOjV1iYgS3
 g2oAW0DYiQQcU71Rb75DvvAs
X-SBRS: 5.1
X-MesageID: 51690423
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:B/PZD6/oXBQEQxBtr7Fuk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P
 re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM
 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="51690423"
From: Jane Malalane <jane.malalane@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Jane Malalane <jane.malalane@citrix.com>, Wei Liu <wl@xen.org>, "Jan
 Beulich" <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Pu Wen
	<puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v1 1/2] x86/cpuid: Expose NullSelectorClearsBase CPUID bit to guests
Date: Mon, 6 Sep 2021 13:00:03 +0100
Message-ID: <239f7624f3ac3793993dc045d451079aa555b90d.1630929059.git.jane.malalane@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <cover.1630929059.git.jane.malalane@citrix.com>
References: <cover.1630929059.git.jane.malalane@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

AMD Zen3 adds the NullSelectorClearsBase bit to indicate that loading
a NULL segment selector zeroes the base and limit fields, as well as
just attributes.

Expose bit to all guests.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jane Malalane <jane.malalane@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Pu Wen <puwen@hygon.cn>
CC: Andy Lutomirski <luto@kernel.org>
---
 tools/libs/light/libxl_cpuid.c              | 1 +
 tools/misc/xen-cpuid.c                      | 1 +
 xen/include/public/arch-x86/cpufeatureset.h | 1 +
 3 files changed, 3 insertions(+)

diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index b2c673841a..d667c36f31 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -289,6 +289,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
         {"svm_pausefilt",0x8000000a, NA, CPUID_REG_EDX, 10,  1},
 
         {"lfence+",      0x80000021, NA, CPUID_REG_EAX,  2,  1},
+        {"nscb",         0x80000021, NA, CPUID_REG_EAX,  6,  1},
 
         {"maxhvleaf",    0x40000000, NA, CPUID_REG_EAX,  0,  8},
 
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 735bcf8f0e..d79e67ecfb 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -185,6 +185,7 @@ static const char *const str_7a1[32] =
 static const char *const str_e21a[32] =
 {
     [ 2] = "lfence+",
+    [ 6] = "nscb",
 };
 
 static const struct {
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 380b51b1b3..e5a7c94c78 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -285,6 +285,7 @@ XEN_CPUFEATURE(FSRCS,        10*32+12) /*A  Fast Short REP CMPSB/SCASB */
 
 /* AMD-defined CPU features, CPUID level 0x80000021.eax, word 11 */
 XEN_CPUFEATURE(LFENCE_DISPATCH,    11*32+ 2) /*A  LFENCE always serializing */
+XEN_CPUFEATURE(NSCB,               11*32+ 6) /*A  Null Selector Clears Base */
 
 #endif /* XEN_CPUFEATURE */
 
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:03:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:03:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179711.326124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDL7-0004Ja-3Q; Mon, 06 Sep 2021 12:03:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179711.326124; Mon, 06 Sep 2021 12:03:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDL6-0004JT-Vx; Mon, 06 Sep 2021 12:03:04 +0000
Received: by outflank-mailman (input) for mailman id 179711;
 Mon, 06 Sep 2021 12:03:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PlzZ=N4=citrix.com=Jane.Malalane@srs-us1.protection.inumbo.net>)
 id 1mNDL5-0004Im-Qs
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:03:03 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 624bf758-0f0a-11ec-b0a0-12813bfff9fa;
 Mon, 06 Sep 2021 12:03:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 624bf758-0f0a-11ec-b0a0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630929782;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=/rO6h35/M0MTFcSt9DOaPPseqG4z+McXNWqLbx32HoQ=;
  b=J6nhEADilOC8KYSU8QnOO4JAM+JjbszA3b4z2OdKJXLYECgqpwMu36d+
   KEGg32OsacZXebRbbFzMpkkgLFjGIcmQfqK+wrVwQtUMXwXrSVI/gftLP
   IeyZ9lLRJaVpMLiz7Icu3klH9AmPnZEhej4iWjnshovLim/JhjmCSNMT6
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 2yxqHK4UoAn9Tf3hwbH09y0L2oHBa7OhCWCfPCeDayrroAYgWSup1sXhjxISpYkXvgzfxWiQjd
 Ke24p72vg7DHS4UbxvqUFTVHPU7FGMcSa0xK7uG1Ix3k5T4xLLCTuQzH1oSIdiAC79+wmBHegC
 YZuYtF6U9NRE2T0Mev4Hc6yayUwOeXKwGfD34t73IptbcztfsESVz4eSO8ZT3gNhBTTX1MfXXI
 Toi6ePZYTZZvnRTf97k/0nT0RKQmXYMhmccNvmNoM+EHu22R7DNNFvmxTG2y1mlL3O7us3DEfb
 Al33XHISP2E3tmH7XBRq2XLM
X-SBRS: 5.1
X-MesageID: 51690456
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:RtKG1qyMl9JJEBXjb1RrKrPwFL1zdoMgy1knxilNoRw8SKKlfq
 eV7Y0mPH7P+VAssR4b+exoVJPtfZqYz+8R3WBzB8bEYOCFghrKEGgK1+KLqFeMJ8S9zJ846U
 4JSdkHNDSaNzlHZKjBjzVQa+xQouW6zA==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="51690456"
From: Jane Malalane <jane.malalane@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Jane Malalane <jane.malalane@citrix.com>, Wei Liu <wl@xen.org>, "Jan
 Beulich" <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Pu Wen
	<puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2 CPUs
Date: Mon, 6 Sep 2021 13:00:04 +0100
Message-ID: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <cover.1630929059.git.jane.malalane@citrix.com>
References: <cover.1630929059.git.jane.malalane@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Zen2 CPUs actually have this behaviour, but the CPUID bit couldn't be
introduced into Zen2 due to a lack of leaves. So, it was added in a
new leaf in Zen3. Nonetheless, hypervisors can synthesize the CPUID
bit in software.

So, on Zen2 hardware, Xen probes for NSCB (NullSelectorClearsBit) and
synthesizes the bit.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jane Malalane <jane.malalane@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Pu Wen <puwen@hygon.cn>
CC: Andy Lutomirski <luto@kernel.org>
---
 xen/arch/x86/cpu/amd.c           | 18 ++++++++++++++++++
 xen/arch/x86/cpu/cpu.h           |  1 +
 xen/arch/x86/cpu/hygon.c         |  5 +++++
 xen/include/asm-x86/cpufeature.h |  1 +
 4 files changed, 25 insertions(+)

diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 2260eef3aa..654f82e2cb 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
 			  c->x86_capability);
 }
 
+void detect_zen2_null_seg_behaviour(void)
+{
+	uint64_t base;
+
+	wrmsrl(MSR_FS_BASE, 1);
+	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
+	rdmsrl(MSR_FS_BASE, base);
+
+	if (base == 0)
+		setup_force_cpu_cap(X86_FEATURE_NSCB);
+
+}
+
 static void init_amd(struct cpuinfo_x86 *c)
 {
 	u32 l, h;
@@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
 	else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */
 		amd_init_lfence(c);
 
+	/* Probe for NSCB on Zen2 CPUs when not virtualised */
+	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
+	    c->x86 == 0x17 && c->x86_model >= 30 && c->x86_model <= 0x5f)
+		detect_zen2_null_seg_behaviour();
+
 	/*
 	 * If the user has explicitly chosen to disable Memory Disambiguation
 	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
diff --git a/xen/arch/x86/cpu/cpu.h b/xen/arch/x86/cpu/cpu.h
index 1ac3b2867a..0dd1b762ff 100644
--- a/xen/arch/x86/cpu/cpu.h
+++ b/xen/arch/x86/cpu/cpu.h
@@ -21,3 +21,4 @@ extern bool detect_extended_topology(struct cpuinfo_x86 *c);
 void early_init_amd(struct cpuinfo_x86 *c);
 void amd_log_freq(const struct cpuinfo_x86 *c);
 void amd_init_lfence(struct cpuinfo_x86 *c);
+void detect_zen2_null_seg_behaviour(void);
diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c
index 67e23c5df9..232edb0c4d 100644
--- a/xen/arch/x86/cpu/hygon.c
+++ b/xen/arch/x86/cpu/hygon.c
@@ -34,6 +34,11 @@ static void init_hygon(struct cpuinfo_x86 *c)
 
 	amd_init_lfence(c);
 
+	/* Probe for NSCB on Zen2 CPUs when not virtualised */
+	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
+	    c->x86 == 0x18 && c->x86_model >= 4)
+		detect_zen2_null_seg_behaviour();
+
 	/*
 	 * If the user has explicitly chosen to disable Memory Disambiguation
 	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 5f6b83f71c..4faf9bff29 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -146,6 +146,7 @@
 #define cpu_has_cpuid_faulting  boot_cpu_has(X86_FEATURE_CPUID_FAULTING)
 #define cpu_has_aperfmperf      boot_cpu_has(X86_FEATURE_APERFMPERF)
 #define cpu_has_lfence_dispatch boot_cpu_has(X86_FEATURE_LFENCE_DISPATCH)
+#define cpu_has_nscb            boot_cpu_has(X86_FEATURE_NSCB)
 #define cpu_has_xen_lbr         boot_cpu_has(X86_FEATURE_XEN_LBR)
 #define cpu_has_xen_shstk       boot_cpu_has(X86_FEATURE_XEN_SHSTK)
 
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:19:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:19:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179748.326146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDah-0006YK-KW; Mon, 06 Sep 2021 12:19:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179748.326146; Mon, 06 Sep 2021 12:19:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNDah-0006YD-HZ; Mon, 06 Sep 2021 12:19:11 +0000
Received: by outflank-mailman (input) for mailman id 179748;
 Mon, 06 Sep 2021 12:19:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNDah-0006Y3-5e; Mon, 06 Sep 2021 12:19:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNDag-0003TH-Vg; Mon, 06 Sep 2021 12:19:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNDag-0005hY-OP; Mon, 06 Sep 2021 12:19:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNDag-0002TC-Nv; Mon, 06 Sep 2021 12:19:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=L36sy1FnWcaWdqr/FCKX8yMIgrwzydWVdnPZfgZJAco=; b=w1zGB0Sl+myEEwXjky2ciUiVbZ
	hY1oWDQxL7Jk5lXnrGRa4xP0zvsjFdUmDM/kjC1U9OqxYhc7mZ5m9py7wabmkdwB9lqYNo+1qveT7
	ds71xzIdbg7XIMvKeTIBdNjrkYb0MmubY94uPjzzPKgLAAp4NXnlTbBpF01CBN0GYDg8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164857-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164857: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 12:19:10 +0000

flight 164857 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164857/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   65 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   39 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:48:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:48:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179791.326177 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNE3M-0002Bk-5V; Mon, 06 Sep 2021 12:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179791.326177; Mon, 06 Sep 2021 12:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNE3M-0002Bd-1t; Mon, 06 Sep 2021 12:48:48 +0000
Received: by outflank-mailman (input) for mailman id 179791;
 Mon, 06 Sep 2021 12:48:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNE3L-0002BX-0N
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:48:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNE3J-0003xM-5w; Mon, 06 Sep 2021 12:48:45 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNE3I-00010W-VM; Mon, 06 Sep 2021 12:48:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=rwy7AxNwfFwlsMnF38Gr4XwnPq0UVzT9byovRhnT+jY=; b=A4i8Ezp016dum75EUVFwvLVBqg
	PmENjNsYxSmwhI+J7psgcyB+22jjdZxTK+XfTnlzDVua2bXpMZJEzlrbh4ivrHOPSdC1DdsvwqK0c
	8zwubBuX5aXb5TPWIN6vaGwPz3PBOroCJ+JLByoH2oKDG0lFqo4uw1EVJ8+V3B/k/vLs=;
Subject: Re: [PATCH] ns16550: MMIO r/o ranges are maintained at page
 granularity
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <05d055aa-2981-8c0f-33c6-62139859aeb2@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <a3b91f7a-ceb0-cf10-04cf-4249667c81fa@xen.org>
Date: Mon, 6 Sep 2021 13:48:42 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <05d055aa-2981-8c0f-33c6-62139859aeb2@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 30/08/2021 14:05, Jan Beulich wrote:
> Passing byte granular values will not have the intended effect. Address
> the immediate issue, but I don't think what we do is actually
> sufficient: At least some devices allow access to their registers via
> either I/O ports or MMIO. In such aliasing cases we'd need to protect
> the MMIO range even when we use I/O port accesses to drive the port.
> 
> Note that this way we may write-protect MMIO ranges of unrelated devices
> as well. To deal with this, faults resulting from this would need
> handling, to emulate the accesses outside of the protected range. (An
> alternative would be to relocate the BAR, but I'm afraid this might end
> up even more challenging.)
> 
> Fixes: c9f8e0aee507 ("ns16550: Add support for UART present in Broadcom TruManage capable NetXtreme chips")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:58:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:58:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179807.326192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNECo-0003v1-5V; Mon, 06 Sep 2021 12:58:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179807.326192; Mon, 06 Sep 2021 12:58:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNECo-0003uu-2F; Mon, 06 Sep 2021 12:58:34 +0000
Received: by outflank-mailman (input) for mailman id 179807;
 Mon, 06 Sep 2021 12:58:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNECm-0003uo-Nq
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:58:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1394b0de-52ea-462d-b2b3-68937a13ce89;
 Mon, 06 Sep 2021 12:58:31 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2056.outbound.protection.outlook.com [104.47.2.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-GtL87o-fNGGo4nSSiBoaVQ-1; Mon, 06 Sep 2021 14:58:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5742.eurprd04.prod.outlook.com (2603:10a6:803:e5::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 12:58:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 12:58:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0002.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 12:58:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1394b0de-52ea-462d-b2b3-68937a13ce89
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933110;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=hK3ZJrK7cB1H1je6taAi+GVM+bdxDIHytD92JlsUDMg=;
	b=Z95Gn8QSa9CEPVo3/8jqxEwwYaQzrCJJbuLSrOd5uRtGO/2RlzAmybNPQrmVGymbVFCyju
	Z8ZLIuJ5cPtOGJ4HD3aEkW4C+iGyAk7B4/1nWveqlJS5bN2T4Pvz+IxINqI1DiIMUr3aVk
	H1atRdaf3JsG1+A0r9f+M8pJ+/dpn5Y=
X-MC-Unique: GtL87o-fNGGo4nSSiBoaVQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XRV38NCOiS6jDyL4h1pGek9xAztpaUckuOPeRr0/1hGFEgqKGBEY8BG7aaLw7ngQZS1cg/UgTwrLcH1L8KQTlFKm+e+Rc2YAcKeoHZRKnJq5o1rvYZqm0yP40VlMmloJW2Gv/vOoBt4KAxEc7enxBu8g/t/PphBNUxylQxwn3ZEhPhgzdXHWulx/SoyQPUs46Kagu2fX0JWkonykNBlPpIp8HcWQyOSe+O2BC2GC8MmX1Pi+09LV2sZCaOWqFAlGEepSGsConMUnJvOVFEuHJyhw5M2h6cD0HXSTeSWLC/CH1y5yuj/0faLi8UEvESIgjV6ExTo6PQOwxEcai2Uz2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hK3ZJrK7cB1H1je6taAi+GVM+bdxDIHytD92JlsUDMg=;
 b=KNMe2wfs7S1Ji68YWSY5fjhqcgx3LbrHmr7TTGmfmCckBac2bR9wCeSVly1JNn/4B9td+3VQ8pB4rhw/Pnb0JXjeQE5aVLYtbqGRiraYnqZxIoKnf3tMP/FoYA0MzG/wKT8j3nzcdqC2/xqDDlEOBGO6DKSfBsvQaHYsgTdcHx9LHrvwMiCPEbtvNXGlR09NWC4HCT6fl7kpA9stkTzdp5VtrBAy0XjoSIabIIPnm4gG7OgNj/niSNkP30YvMdWYwPpE6aAR+kUK4oxhbew4DzJpaUe5DcxhNSxO7qC/vdcxT2isjAa5RWP1WT72OD3o+QpBF4xOTMg9qcQJif/2/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/5] x86/mwait-idle: updates from Linux
Message-ID: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Date: Mon, 6 Sep 2021 14:58:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P191CA0002.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bbc0d966-9c8d-4ca9-c26e-08d97135fc71
X-MS-TrafficTypeDiagnostic: VI1PR04MB5742:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB574288122EDF0B7591241D0BB3D29@VI1PR04MB5742.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HdR13lwz7wB4w/KpVQmi6z17nnkObvE0lCxl9+lm4ZC+5dip+3B4NB1j2P2ItzPeVF3dhH3Rzz3Bz+wQw+TJxZFk7Xcokuc5fCH+PkgZaIT+rc7xywc96So+gCbMtF4YpR2gTT6DUvl63b6slzA0cC1YEfHovsnf6//as4bB3VRnUiJ9d1eyPzDBpSZ+aRPShSS2zRYHfh0cP/VYeeUMKKq21V+dFnJ/W2/GyjS8qiGLY/vfCaJ/lg1K00sfZhMq9/7IB3ruMpL8sWbZ+bbUN7qMAk2+KxxxlR1CiQY9C8PuWi83MmkaGB1f8U1iaLoFIWfKSc9q33cpdj5MFM8YyrmEhkadO8uzgUHQhn1KlcIMmEZk5UB5Ia9Jd37jGL382cNC1JqQj1+6DztJf1Ru2Azsi3I3Go8YPmOps4EKfDIu7bjFZMgnu+o2+E7VPZuGyueRe9djedIQ9pL3mrwnzahm680DRbbS+S7NTsxYLz4VHvSEbnNFEU1D09PK2frSraABEJmFLJZGqTd3o6FhPjPV8oSCvAscOssgpZiqoBHT7f82fOSujUScI8VNsyD5Akq1RIB2msjk8dOvQrXRHkjdxda4Txj04bb9r2AKr4U0VoCzFW6kn53Iy4lHPtzkASO5qD9h0/R1DrUo6NIwo1Gf1/mNYX+uDiubISEoVVCSHj6/N3OTxHYzoShfuOEk5FlZwCs+y/FLKQu+gpUD5xoUz6a4RrYe9SLJY3Ao1Ik=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(346002)(396003)(366004)(376002)(31686004)(186003)(36756003)(478600001)(16576012)(8676002)(316002)(66556008)(26005)(86362001)(54906003)(6486002)(31696002)(6916009)(2616005)(5660300002)(956004)(38100700002)(15650500001)(4744005)(66476007)(2906002)(8936002)(83380400001)(66946007)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RUZobEt3Mi8xSkpwcTM3aE5aazMxTW9PbU5tcFYyeUFIS2ZQSStpOGw0SE90?=
 =?utf-8?B?RTFubFl5UlpFRjIxTFBxZjRMeXJOUlBTYTJIb0tUM3htaDR0M3dMa3pBdy9O?=
 =?utf-8?B?cDFqdk03OWFBRC9WbGpyZ05MWUhJbm44aHgyVEtybGN2a0hiV0V1anBFa241?=
 =?utf-8?B?dDRuVTN5VGk0UzRubXVVZ2hrY1I4T2pRT1VRNDNIdDhLeW5hQTcvalRGdk9y?=
 =?utf-8?B?N00xV1VhRmR1akJ6R0dLbWk3MlRRZ1BRclVTTkZ1K1VWdW93K3dGc1VCUU9J?=
 =?utf-8?B?VDc0NFExajZIcjZlRWx1cno1U0hOYjc0bHJtRzZ0YTVDMSsxQXFIRXIyTGdy?=
 =?utf-8?B?RkthS3NvUzA2Z3hyUzkwWm9hODVjejFidFZHUzVtbXVER05FaWp5RitubDVy?=
 =?utf-8?B?N1lHTERKMnBSTDgvOHU4bGZ6bWk1ZCtmZHlXUmZEaTZDM1J1N3R5dHZqMkpW?=
 =?utf-8?B?Q1BPNTQ5NDRMcWg2Z21rTGlsSVJUZDBOT1NiUmdBNGxCR3gwS1FWdm5jSTBr?=
 =?utf-8?B?NHZSWkx2NCtPYXNDRUtpbEQ4MXBTQ0NKRlRkTlFmMHc2UlJzNXF4SWE1NXR2?=
 =?utf-8?B?QnQwMmFqK3dsak5Nc3ZyaEpDRXp4R2l3eFg0cFhJN3NnZjNDZ09RelZuWE5j?=
 =?utf-8?B?Uzl5dlRqb3Z3SFdwL201M0duVzdNMGRNcjdqdnFEM0YvdUpMOWFJdEUrNXgz?=
 =?utf-8?B?R2o0MUZmRnB1a2NKMVU1cGg2a2xlSFl1SldxdUVZNm92ckt1dU54ZE44SG1y?=
 =?utf-8?B?NERPWExOeEdTNlprbkN3YW4wUm1VSHhoYTc1TkxBZnpHb05PTUswNmNvd0NL?=
 =?utf-8?B?UnM2WjdwbFdMVFVwRWFQbThUR0FOZWVkS3N0MWpNS29XMk5CcnRSUmEzSmcr?=
 =?utf-8?B?eWVIN2tQdW54UUlVV1IxajdqOTRFUjQ5R2VlbU1DYUJnSVorb3MyRUVBODhD?=
 =?utf-8?B?LzVua0FKdXJnNm1RdmRvdnl0MTRyQUJ3bjVxeHlBM29IcmJrZUkybjMyUFlL?=
 =?utf-8?B?R3ZDUHRuOENBRkNoblVYTXF4V1hHRWRGZHR2d2U3d1FLVFdrbWNwa3RTZmp2?=
 =?utf-8?B?cVB3YnpGazhrL21ma2N2TWVQeDg1cEVNWlcxQ3ZkS2ltY00zbnBneHQwWGdE?=
 =?utf-8?B?UE5MVkxPM2Rpdm96NUV4ajlhcDh1Z05sY2JlZXl2VWJGUVdCM3RhUG9wQmJx?=
 =?utf-8?B?cmRRdWxEWngyV0tpQk5ZcVdySDZFQ1B5S3ZqdFYwMWhVbmJLNElvQ2QzSzI1?=
 =?utf-8?B?YUtsWEJ3cjV2NU12L0doeVhGV2R3Uk94eFE3NE5wREplTUhwMzVxNktqa3JY?=
 =?utf-8?B?ejJ0RGM1a2kxdjBWTmhVUDMvUEhIanYzNWJTU2pOazVMUjlTWVVLeDVmRGxl?=
 =?utf-8?B?K0E3bEtOMjNPNEl5ZHFYL3Myc1ZOOS9La2tSUktPMDRzT3Ava29OT0NKTUdo?=
 =?utf-8?B?bjladVp6N1ZTaDFpRkFQaUs2bktNajB4MXRRWCtDbXRpV0F6QTRvV2JsZGtw?=
 =?utf-8?B?N0xremMxRjhlZHpkeThhb2loZFZxekJualZ0UHJ1U3NjdEZrNE1ydGhXYVBo?=
 =?utf-8?B?MjVROSt3RktvOFNiS3Fqb2JIMzBuSS82Q1hyanFCalE4Ujg5TCsrVnlkQURK?=
 =?utf-8?B?VktpVWM0aVBaNWVmNEFkSkdudXorM1lrU2QwUDc0aDNFbkFndG9TVGY0RU5p?=
 =?utf-8?B?TWcxK1ZzdmE0UkNVbnQ0SFhkVkNiVnpSN0VqL0NWK0Nxbmh2bnVLMEFNNWNu?=
 =?utf-8?Q?fVhroLwt4cHSoJkgY3uGcQwafYSscrXtZuwAr0m?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bbc0d966-9c8d-4ca9-c26e-08d97135fc71
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 12:58:25.2466
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8rsOOt/AiOE88BWUcPA+vKPIyaKTvuvfAL/GSuEKbiE4WNGqfOi3PR3efc1o/fuOUAs8P9vmMOIqmvN7gxwcCg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5742

Before the code freeze I thought I'd check the original driver again,
and indeed there were a few changes we want to inherit.

1: mention assumption that WBINVD is not needed
2: add SnowRidge C-state table
3: update ICX C6 data
4: add Icelake-D support
5: adjust the SKX C6 parameters if PC6 is disabled

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:59:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:59:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179814.326203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEE2-0004ZU-Jm; Mon, 06 Sep 2021 12:59:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179814.326203; Mon, 06 Sep 2021 12:59:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEE2-0004ZN-Gi; Mon, 06 Sep 2021 12:59:50 +0000
Received: by outflank-mailman (input) for mailman id 179814;
 Mon, 06 Sep 2021 12:59:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEE1-0004ZD-TG; Mon, 06 Sep 2021 12:59:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEE1-00048f-Pf; Mon, 06 Sep 2021 12:59:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEE1-0007Ep-FZ; Mon, 06 Sep 2021 12:59:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEE1-00048p-F8; Mon, 06 Sep 2021 12:59:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6J93bbgmocMXPLFDfWfU4Rp3VcdHlwIwtC0eij1xjH0=; b=VaCrmG+m8rCX7h6PtvHAjgWbqH
	bEHbvSn2gqDXjgsYtCFCMcIowdXyqcr3P3Wo/lWu2vyCTKBJcTyNIWaXFBYTioUUh9eK093DI3EQF
	Bx2VKc0WKFWzSHDY6i+S4gtzsLu0tOGqI2vAp1wPXXIIqY7C9o0pKYwNFp84j39CkVMg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164858-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164858: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 12:59:49 +0000

flight 164858 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164858/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   66 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   40 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 12:59:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 12:59:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179816.326217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEE5-0004rA-TO; Mon, 06 Sep 2021 12:59:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179816.326217; Mon, 06 Sep 2021 12:59:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEE5-0004r3-PV; Mon, 06 Sep 2021 12:59:53 +0000
Received: by outflank-mailman (input) for mailman id 179816;
 Mon, 06 Sep 2021 12:59:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEE4-0004qY-Gq
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 12:59:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 52036839-0f12-11ec-b0a4-12813bfff9fa;
 Mon, 06 Sep 2021 12:59:51 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-xAUjZGraMhuknGFuXbTleA-1;
 Mon, 06 Sep 2021 14:59:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3533.eurprd04.prod.outlook.com (2603:10a6:803:b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Mon, 6 Sep
 2021 12:59:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 12:59:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0288.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 12:59:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52036839-0f12-11ec-b0a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933190;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lYxRIiU55Up939f84d4QmJrnjJ7T0dm9h+/746Rq8MI=;
	b=TAsfWiHE7FZWYTewzBgpb+QG9gVW3kWFnzbDI6TYQZ4q61WIEVh0JLmWYfqCz7kJjy6nvA
	1qLtePpTs+Wnj9fILoNt7kw9GlELIxxNVHIGvBKGLTJsNhYP47HtI9sp3/JMwOheLQWum8
	/uC25cCoklG6xZH6UXkaBi6w6stIP9I=
X-MC-Unique: xAUjZGraMhuknGFuXbTleA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZLjHtcBlvctdvTt75iYoXLbCptmC1uoUlrqNiQmQOWmN5NMRBe+ubNquiHZn7Wq5b/C/z67M+RdwyNcjKJKh02ATQmXYOrLkVubin4yYYHIRfRSzSvkoVYatGqs0as1BQP0L+Y97lHeLfoZHK94JsMRT8IrlWfP3wQ2jgihURSI0tk14jvyx/GZ9p78FXtC4g63uV4MhEyoDHahdiEqGvXATD+T+FhreW3qXYUw3+ZaajRM/IfMyzUTDycSGZXgW88Gb2CFkJbNWnzUs08XmUmhZVZitKoQ55psDDzosUvTaLRNcIc80BeFRWQroEalriCgOHyeLqx2fVWiCJZc//Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lYxRIiU55Up939f84d4QmJrnjJ7T0dm9h+/746Rq8MI=;
 b=PF+1ThupuWkV23rBEb/AVG9YKXiiGvWzWk8N0RVdNnbRiY8A9Y1xfd0Ys2B6n9B8bIi5+yrhKDP5LlyHid8BZcSECQY5ToTb05h/ieblw7n0uV7xrluquc6TLnGeefGIMyWcvJLD4YPtwWKMfERKsEhykRZ0NCb1HRs3UY+d0sH4dsrTgiBMt7bSutsb4uqPk4cbmG9rhfWQ7CvulLnIJqeo1EOA1JeVxKl+J+t8wUXxcf0yL4fJPYZV6zu2vnAfKaEYcXMbeglndhz4/v5FSKxibFcor1TaHlfRrY1cAtnjJQHwHkvQUhi+InvBm1GMzwe/wH+JNRYImUUulv2h9A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/5] x86/mwait-idle: mention assumption that WBINVD is not
 needed
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Message-ID: <0ebb0de3-2500-a9a2-8128-64b9d4a27cb7@suse.com>
Date: Mon, 6 Sep 2021 14:59:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0288.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7171f000-7582-4922-d127-08d97136349e
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3533:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB35336395B11F7488EE05CD03B3D29@VI1PR0402MB3533.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3044;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wklRinhSRVpDYvJdmegP10VvhuZJ/azd6SK3EM0JRoO+jeONCwEo0dDrPHZIuQDrxm31/i3doby2IO5X3Wz68ierdfXcSEQM2b1PfiWoOmdmpUqO+6Oi03hx01V0Eo6076ekcEruFFs4nxaAbfG5zttp/Y/N4OJSmiY7ON7Z8UlRomzcDiWYsd4cO02iRWeIb0oJEQiAKz3YSsgpkDBoABzVH2N6/OzjzKNjFLnhVPvddGwkOn/Dd8tij4ivx3BaLAmWM+UVlCVgSONrteD0Xertv70pBHzmFKTjmKlyilsJvsnbL1hswMy4m/7yzCahsRflgR0HKzcUii8TplJ501Nwe4EIDsQfYy2tsPWfJZDjEYQKQvvEZBstvNfexOmA1z6O8oqTOPZegnxl2mIkOQNz4B/LU6Q5CvnMfanNE8MmrDOmbejoWYX/dA9YCrjxaKZKjeuUDhkZPS+PTlOs6jRKke0xOy4FkA+c74HRnfIOVYl9RZkoe9SeMhT8ftCWHnYgMcTBbhV8pMtYhtXg8GMdw/5oNRmk1yUFURpWsPN6wL6YyPf9JQ6lqjaS45Skf4pCWP6emhg5ORp4CcytYlwwIF0wfU3w4cmADcR5BKnI8oLy4p0uuLLtKAtK4YIIAzzRzFjdHiGwTYy2ukJpPTBFn9qjiBMS1lyB4S4YK9JRh8pSndGvuwK6PL+fHSpVLmUh/g4RWA5QTvEAJqlS6Bt8ORLpgHHUMVQDcWzSPwc/sudAHLc8DanABn8qZWic
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(346002)(396003)(39860400002)(376002)(16576012)(54906003)(478600001)(316002)(6916009)(2906002)(31696002)(66476007)(8676002)(186003)(8936002)(26005)(5660300002)(4326008)(6486002)(956004)(31686004)(66946007)(38100700002)(36756003)(86362001)(2616005)(83380400001)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WjdKRVdDUHVmT1dlNmlJcW9TekoyUk14eXJOeU1iY1YrK3ZPd3FRUU5KeFRF?=
 =?utf-8?B?YlpubWpLMWtuaDM3dm1ndVJpVXFQSnRUUzArYUJKVHpiaGRrZkJuM2FyRWhB?=
 =?utf-8?B?amI3bm16QkYxRmRIY2liVkJHRE5hRkVGNWpuQ3U2SWpLajBQWEJaTG5QRW5M?=
 =?utf-8?B?TVZZSUxFSU8zcXgzZmJnZU56SWtWaWhiU0hQd0wxZjIycGYvZUZ4ei9zK0FE?=
 =?utf-8?B?aGRmVVdjSjZQMzY3VGIydFhqTFJmUzc4UDFMVDlIQXN1aEhpRTkrVU00MTUx?=
 =?utf-8?B?SkxvZ3RtR2Y3Y2FtTTNYYm1FbjFmTklhQnhETC9TdkxIV2xBMSsxenRkWUtz?=
 =?utf-8?B?ZVlzdVdEQlE4Q2t4clhDYmVsemRLYzZjSFhhYm1VeWx2S0x6TnB0RGg2eUtH?=
 =?utf-8?B?S1RVRVVPSVA0bVE5NTExTzJ5bWpYUVpzNHhIMHZNVnBCU0ZNS2ppUDZySnkx?=
 =?utf-8?B?N0dZUFVpSGlNYlMwSEI4QWVzek9lU3Z4OUl1UTd1ckJvRTJNR09PUmtheVdW?=
 =?utf-8?B?OExReC9nVHliVVArNVNQUVI5dDVRamYzbGd3NXhTRFBNZU1qdy9rZzVTUlFq?=
 =?utf-8?B?VG9BUGFPeG9BQitHWUtMV3ZjbkF6NHF3K3MydnVOVngzTkZNVVZoNG1jQkxC?=
 =?utf-8?B?aEFyUUJuY3lEV2oyb1hxTWNxUU95V0YyN2dVRHRTZWh3WE1OOHl0R1dNbkxX?=
 =?utf-8?B?R29qNWhSTjBBZDQxQWhjSW5kYkhpZXlPY1kyM1FmR2NEeW9SMHpQMi9VSllx?=
 =?utf-8?B?QXRSWUlBTk52c0JKVDB2S3YxT0VwMkNYOVIzTllZWGtPZmI3VThpMEtabkcr?=
 =?utf-8?B?ekhjMUJhT1N3OENRRmttcDJFRlZYL0o4bVA1ZVlQVUtyZFhPdGNBcTZZNjFK?=
 =?utf-8?B?RTN4d29NbDNob1ZqZXBoaE1HR1ZyOUhrMHM0QUNERHhTZ2VidmVqVkt0RUY5?=
 =?utf-8?B?Y1pCNEsvUUR1RW1mVGFscm5ockNYZVZFQU5QNzB1dCtFWnlXeWR3VDVUWTBB?=
 =?utf-8?B?SlZjVjRlUEU3bDNnRllnOFRCL0VUUEFnV3N6M1pVNGREa2IyYmF6UzdBeGRv?=
 =?utf-8?B?Wk5OWGh3QXUyV3Jzb0pIdDhlUDdQaXVSb2Zra1NMbVh1LzNheVZrNnIvaitm?=
 =?utf-8?B?ajZqcmZOYmNLVzhPb3h0d2M4UkpiOEpxL1VzTTMzRVMzNGVydFBwUEJmam5V?=
 =?utf-8?B?bGQyWlpGb0VVVTJPd2xLUENPVFcrYStIbUQyN2V0bnVxbXZUbXk5Yko5SXNP?=
 =?utf-8?B?NXhFejNackpNRUtYQW5HSzNjS2NNRUN0WlRpRERwVXlUUTU5VDdGQ1plYXB0?=
 =?utf-8?B?U3NVTW9CWGp2b2s0ajdMeTJmRjQ0b3lBZDhzc0Jxc0dzYU12VGhNbCtoOXBT?=
 =?utf-8?B?dE1GK2Rhb0FiQUJEVHh5TEM1NzJCMUxZWjJSTDl2MDgxdW5ZaU5mMTJiYSty?=
 =?utf-8?B?ak1Wc2lSU21qSEdDYjZ1bkkxUlM1eHphVHRtbU92RkFtSHFlcHp6ZnV0SVJB?=
 =?utf-8?B?c2xBQlNQM042aDkrNHU5UzBIdEdtZzljcWFUeDFNVnZaSWFCbG9MeTdhOTU3?=
 =?utf-8?B?TUszZkFWdmZVQ1RBMmhDcDFETVhvV3RYWUZUT3pHZWtWVTJ3U1dmNHE0dVl4?=
 =?utf-8?B?Y1hkaTRmNFVNNDhyMm9VSHpUNUJqRWxPTFZYczR2WHpkenRXdk5XRjhmOUtM?=
 =?utf-8?B?U1p6QWkxL0ErSXZyNkUraFlJUmFQTnY4anczMkVjVUxkUU9MZVQ5TUU1Kzkv?=
 =?utf-8?Q?PI9XIL1uEwWqncUrWy/aSwU36jrDd1X1UsZNY4Z?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7171f000-7582-4922-d127-08d97136349e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 12:59:47.5361
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HqSobVfJ3c2p3fXDjWInCq+5ruoGY8x9/vU4bs9CSCmzW2E0UsiD53GkKIhd4o6HQl1fd2aqdwkm16KZmow+PQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3533

From: Alexander Monakov <amonakov@ispras.ru>

Intel SDM does not explicitly say that entering a C-state via MWAIT will
implicitly flush CPU caches as appropriate for that C-state. However,
documentation for individual Intel CPU generations does mention this
behavior.

Since intel_idle binds to any Intel CPU with MWAIT, list this assumption
of MWAIT behavior.

In passing, reword opening comment to make it clear that the driver can
load on any old and future Intel CPU with MWAIT.

Signed-off-by: Alexander Monakov <amonakov@ispras.ru>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
[Linux commit: 8bb2e2a887afdf8a39e68fa0dccf82a168aae655]

Dropped "reword opending comment" part - this doesn't apply to our code:
First thing mwait_idle_probe() does is call x86_match_cpu(); we do not
have a 2nd such call looking for just MWAIT (in order to the use _CST
data directly, which we can't get our hands at _CST at this point yet).

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -31,6 +31,10 @@
  *
  * Chipset BM_STS (bus master status) bit is a NOP
  *	for preventing entry into deep C-states
+ *
+ * CPU will flush caches as needed when entering a C-state via MWAIT
+ *	(in contrast to entering ACPI C3, in which case the WBINVD
+ *	instruction needs to be executed to flush the caches)
  */
 
 /*



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:00:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:00:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179827.326228 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEF3-0006Tj-7k; Mon, 06 Sep 2021 13:00:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179827.326228; Mon, 06 Sep 2021 13:00:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEF3-0006Tc-4R; Mon, 06 Sep 2021 13:00:53 +0000
Received: by outflank-mailman (input) for mailman id 179827;
 Mon, 06 Sep 2021 13:00:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEF2-0006TU-AQ
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:00:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d7f79551-92a2-43cf-8f1d-71134edeb319;
 Mon, 06 Sep 2021 13:00:51 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2058.outbound.protection.outlook.com [104.47.5.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-9Bx13LP0PrKwqSeciJeBJQ-2;
 Mon, 06 Sep 2021 15:00:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 13:00:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:00:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0185.eurprd02.prod.outlook.com (2603:10a6:20b:28e::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Mon, 6 Sep 2021 13:00:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7f79551-92a2-43cf-8f1d-71134edeb319
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933250;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=dbwG033IYOy80/NWLE0KFiBS2xDM7Ke8HGRpzdh07P8=;
	b=XgWq3RIqJvoN4psFBGnchKCfQbvC/8Vta8PECiyF1/8RoOfHmaZxWexuxyco2G0CRhjoNd
	VBmbaz8UpiaZjE8WQIL/NX8NwjqiNsHv2aB/sHDnPoABDalEWCh83pXp5wjVfNE4lyv8aO
	sOOhlD0Jmf8bXIJNvUdk1iCztpU4CWU=
X-MC-Unique: 9Bx13LP0PrKwqSeciJeBJQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iFaY0g6K971tMrHClPEJn3tNCVWy2LI8T2tjRGy6BHgpPB4FB4MKe7Wy6mZI4qifrcH3+S/CnUsN7DuggklVQ+yJCElaEH/lPkKFMxBdIz0Chnu+NtHRVuxhZAGQY2o3JMOvzQc5tdQ9dmJ5RVAgICtYP1UF7nJ1qeNmpJ2PNSXAkJnagNIQkGu8ZyMc8EGwoYIzunBVlJN5cz5lilnI9x8c3pOc9xJLUp9lFJNJ7QAMEo9IWOGgB603EMwqsIFw7DQhgzbcaq7aID9DntCXxF3+bSFWHGMnbK+t0EgMDGl/pIARbRsJVzfo+SwiQClGwjxnZFqwWhHzbKNFmVvm1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dbwG033IYOy80/NWLE0KFiBS2xDM7Ke8HGRpzdh07P8=;
 b=QVkNP6lP5NN+6SQIXjVDOKxQpBEn9ySo6Qa4+Gj49AEMyj8RQolBr+S/Oz5m5eLffTMqpjkIPtzOePT/YiJVNltyKwh76oZbFPHrkkQ3Bl2oTVnt4g1+vrCNGleu6ibRZvSJnypAbtL+hOSqYFAMcKBsb5Xd67ukHETiDC8VfyLmwc/GAR/zbKjYHuZHiwOFQufO0w45g8vVI0HWFdoPOyhGaeKaBAsJqpx4ThuoYeiYiLVlMQJsub2qB9+H2Obk0fxFOb7HFAdWg+LC9oD9AUGbK3vtoqVCFnhYfAAPCMlN0aRNJofdosJ5kIhKC9fsLxPEUAJK8fRxcOUp4tco+Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/5] x86/mwait-idle: add SnowRidge C-state table
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Message-ID: <b21c106b-b852-b4f0-efa9-fb5c465bfdac@suse.com>
Date: Mon, 6 Sep 2021 15:00:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0185.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a5413c7-79c1-4ce5-2931-08d97136587e
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4608A11A2CBBD78936236FF7B3D29@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2399;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UvanncNRt3CVO5T4rfui0iaRvjPOLgPVLYFajDhaE8raOtDWDiFid9eL5OCMnTAPwJPTFEkJ/fVkSd01EZGhsIKu39kgI42aIeeVe6m0Mau29B8Puqd6/JI5TXi3RaLs4T/R2Qw6sGXbWGM5F7qAyBTwh8kO7dgJ0610WFWpo2R+xBtdbY5luhgrAlSsF5dNxOTXJel+XR4vG0K2sKSDvZkOQd/7AroTBsLbCs9VMI1oRBfRWvoFNPST4Q/BnxuUnrPpAip7c/u4CpChdzKSqm3WDkw4G8z7SL+vtRF5bGBP1kIKmiED1/QLuKkEDtCsa0SvdkpzdqHdHTzf/AbqrWlJZjg4XTHp4x/9wU4sUEwLc9UOxYaBhG+1ltjPmoFpA9dOa6lbePFtpx92XFnRf3OHeCMxVFFFuaLMiyhklj7CC/AAhkr6k4FNLAywSSsI0EYz79kscZMb9BgJJ0PksmYTb7EqPk3S0kQyE/brt5HTWFON1YPbOFG9a5NiuHKYZCrqwrsGeKF50r5OLlLxIakCUQcT9+2RgqLmeeb2YCqJ1qHLz9l8Cv+bnmf/ZxOnN7DJwSCCx92GRYOb6rj/KYOcE+8jlsOLCHWLQcIkzBy3XdBx8tXK4dSVMX6Lwg4sxDajZzPK5sysUYN2U52XKwrysxejAkablqoVE8eMc3QPXLt3frMEmvjdkI8LcXxVJrt/aegDYidFiC9A2GPfoP4AYn33NolW8hMXKIrN9lk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(366004)(396003)(136003)(39860400002)(66476007)(66946007)(54906003)(478600001)(5660300002)(2616005)(6486002)(16576012)(8936002)(2906002)(31686004)(8676002)(316002)(956004)(4326008)(26005)(6916009)(86362001)(186003)(31696002)(38100700002)(36756003)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NGdQazlYVVRFQ2FvcnVYYVRPQndnMUVUelRUSWNFaFdmRWxrcW8vSzJSWGNt?=
 =?utf-8?B?akdHTXp6bzU5c2lQTjdqelZkcDZrSW4wY1NsRXpwcEpxZlBnTGlTZTVTNWZK?=
 =?utf-8?B?KzJjb1k5aHY4QWFnOWJGWWhPSXM3THN6b0k5YmRRa0xJUUpqc1R0dWg1VVNl?=
 =?utf-8?B?cjJpTXZ6cTkzNFhBdlA0NHUyM1FraHJSZ2VabjMvZFZ6azZRK0tybnh1NUVp?=
 =?utf-8?B?ZU9saE44WkJHclJOQVAwbFNWRjBKb24xK3puMTVHWGJya2xqaVkvQWVDMWJy?=
 =?utf-8?B?U0k5VXZNNllVUEdmQWJqYWxTUEtmTEVuWlNtSHBIZFVyTCtINEQ2bWJEVFYx?=
 =?utf-8?B?Q1IwYWF4bkZsbVUxSW13TlpFNEs4NzJKT3Q3OERhaktueU9VM0psYWZLQksx?=
 =?utf-8?B?VDZiUVo5b0l1TG9UWXhTTmZjYzM0UzhUV0dlNXA3cStJZjlXa21WTHhZUkls?=
 =?utf-8?B?aFBwU1ZJejRaa05HM014VVMxd3NQbGhpUVd3VHZjS0FnZ2dSVnJCOXVpYXNJ?=
 =?utf-8?B?aVBKMVNCL055NzRlSVhuQU1zZFpYVFBORDlUUkNWMW03VE9XNFFQdGRURU5C?=
 =?utf-8?B?SmV4V3VjdWVDM2o1MUg4MVpiVjRNbEYzTmN3UnJScTVTYTQvSDl1aHIvN3Q5?=
 =?utf-8?B?Vng3d241UDlUU0U2enFLRGlITFdET3hVblNERU9OaVpNSFd1NnlWUEViVzFL?=
 =?utf-8?B?RGo4ZEdlbndWeVBKbm9jZ1UwaTBnd1U0R2FIM3ljYktOYXNaN3FPR0ZPd0h4?=
 =?utf-8?B?ZnVJdmVROEFNMzd1TWl3OFV1dGFYaityRDFVcDZnMzByRm9ZeE1UTUVvVDV3?=
 =?utf-8?B?VENHRjFFaCt0SFpSR3RJQU5JSkxlUlJRblNnVm93dlJWd0lCbTFWdWdhMXhF?=
 =?utf-8?B?U1VVVTh2eE1mL2ZqTXNBYzlyY1Q0VFBCUS83a0pEdVJzWVhQZDVVeXlxcCtF?=
 =?utf-8?B?cWl2aUFha0FDcEZET2x0ZzBFYXFXbnhsc0p4dk8rRTlJcDU2ZEJMNSs5K1Zx?=
 =?utf-8?B?TFY2UERwVGpJNlBrZnMrWmhrVXlQSzEzVzM4NlhKbm9lVUtTNGFsVmE3dGx3?=
 =?utf-8?B?VkcrL2FLcGZDcjBmMUtkc2dscmpRTUh0NDlSQ3pwY1hZeHh3R2l1Sk4reGNk?=
 =?utf-8?B?TVhscXk0RHc2VVdBbnMyNnJ6UXVGTXJxSWFvTGlyblBrUnBrYTZOYnF3T0tB?=
 =?utf-8?B?dWhUU2JIZW5YakV0c29STmpDRmhMNU1KMnh1Rnh4Mk8rZ1kvQkRqU2YzMlFa?=
 =?utf-8?B?S1J1cVgwczkzc0VXSDZJRHZ3bS94ME5Weks0b00vcEE2dFdMTDBzekRxeVNL?=
 =?utf-8?B?UFNYaUJObmx2eGFvcEVwSDBkS25CWWZHTUtKTmw0SkR1Tm12QVQ0MGh6Tjlv?=
 =?utf-8?B?Y21TQ2NIemU5SE0vK1IyenlsOXQyenhVOWFGTVRkTTJ1YWloRWJoTWVucVJC?=
 =?utf-8?B?U0VOVGdKTXRJZjZWb2VJbW5HQXlHbU84VlB0TE5FOFkrTzRubkdYOGZyQkJT?=
 =?utf-8?B?bjRKMWU0K1pwc3I2UTc2N1hlMi85WXFpMlg4MWY1VlhOSzJOQ0hnQTVGczl1?=
 =?utf-8?B?THp3UU9GRWtsVVFPSHI5Sm1pK0dRdmtjbVFpdEMrVzUvcUpSRmJ2QUJTeTZX?=
 =?utf-8?B?RFNGTVY1VmUrWHZLZTlBVEpWQ3U5aTBaSUVubzRsaE5sNW02UDZTS05GYUFQ?=
 =?utf-8?B?MHRFeExQcDB4OThkVUwvMGE2MUV5NzVnRmFDRjlLN3hHczYwQjBQUXQ2RlE4?=
 =?utf-8?Q?WYU2dVH9UHMJXzxfqcdUVpFYl3z8QEef2XWjqF9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a5413c7-79c1-4ce5-2931-08d97136587e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:00:47.7291
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LM5RyPaoQrPL30amNzacVtuxbHakgb9YTZUAkyIQJQG2u9hpUvBixHNvhy39UxNHDy8hP0FOPy+s09sEBdP+2A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Add C-state table for the SnowRidge SoC which is found on Intel Jacobsville
platforms.

The following has been changed.

 1. C1E latency changed from 10us to 15us. It was measured using the
    open source "wult" tool (the "nic" method, 15us is the 99.99th
    percentile).

 2. C1E power break even changed from 20us to 25us, which may result
    in less C1E residency in some workloads.

 3. C6 latency changed from 50us to 130us. Measured the same way as C1E.

The C6 C-state is supported only by some SnowRidge revisions, so add a C-state
table commentary about this.

On SnowRidge, C6 support is enumerated via the usual mechanism: "mwait" leaf of
the "cpuid" instruction. The 'intel_idle' driver does check this leaf, so even
though C6 is present in the table, the driver will only use it if the CPU does
support it.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
[Linux commit: 9cf93f056f783f986c19f40d5304d1bcffa0fc0d]
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -742,6 +742,32 @@ static const struct cpuidle_state dnv_cs
 	{}
 };
 
+/*
+ * Note, depending on HW and FW revision, SnowRidge SoC may or may not support
+ * C6, and this is indicated in the CPUID mwait leaf.
+ */
+static const struct cpuidle_state snr_cstates[] = {
+	{
+		.name = "C1",
+		.flags = MWAIT2flg(0x00),
+		.exit_latency = 2,
+		.target_residency = 2,
+	},
+	{
+		.name = "C1E",
+		.flags = MWAIT2flg(0x01),
+		.exit_latency = 15,
+		.target_residency = 25,
+	},
+	{
+		.name = "C6",
+		.flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 130,
+		.target_residency = 500,
+	},
+	{}
+};
+
 static void mwait_idle(void)
 {
 	unsigned int cpu = smp_processor_id();
@@ -954,6 +980,11 @@ static const struct idle_cpu idle_cpu_dn
 	.disable_promotion_to_c1e = 1,
 };
 
+static const struct idle_cpu idle_cpu_snr = {
+	.state_table = snr_cstates,
+	.disable_promotion_to_c1e = true,
+};
+
 #define ICPU(model, cpu) \
 	{ X86_VENDOR_INTEL, 6, model, X86_FEATURE_ALWAYS, &idle_cpu_##cpu}
 
@@ -995,7 +1026,7 @@ static const struct x86_cpu_id intel_idl
 	ICPU(0x5c, bxt),
 	ICPU(0x7a, bxt),
 	ICPU(0x5f, dnv),
-	ICPU(0x86, dnv),
+	ICPU(0x86, snr),
 	{}
 };
 



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:01:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:01:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179831.326239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEFY-0007F7-Gz; Mon, 06 Sep 2021 13:01:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179831.326239; Mon, 06 Sep 2021 13:01:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEFY-0007F0-Dv; Mon, 06 Sep 2021 13:01:24 +0000
Received: by outflank-mailman (input) for mailman id 179831;
 Mon, 06 Sep 2021 13:01:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEFW-00070n-L8
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:01:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 85213f56-0f12-11ec-b0a4-12813bfff9fa;
 Mon, 06 Sep 2021 13:01:16 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-4zTxguOhNJyKD5lOYhCkVA-1; Mon, 06 Sep 2021 15:01:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 13:01:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:01:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0181.eurprd02.prod.outlook.com (2603:10a6:20b:28e::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Mon, 6 Sep 2021 13:01:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85213f56-0f12-11ec-b0a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933275;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gXqC7kdE6vgcxzHsZu6LKPSCCwekAZOdkt0dRfJilCw=;
	b=Ktcpexp4X1FT3W4pAC7pUsPQzrkGO2TVGTntuRgBWNeIFnI16yoYesiVD+RAxa7GN53VrY
	shfEaP0flxV2POE8B/TgQsqvVC3o0bYlbIzqzKUOYKSUhWrZYuRenu4Jlo5xtrnun3hVLM
	YQHEn2bR6i4iXVDjQxSoQXxcJfUxda0=
X-MC-Unique: 4zTxguOhNJyKD5lOYhCkVA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OQ1hhvpzshOWV7M8HGtnj/xEGUliPqE+LcKtpQ9eTUvHV5LNbZzIAHP7dZsySbv1uMF328nXodfDda0OqBvFrE10edK+XIcNXYzvcsBAlNcPkw7Su5JhKA6/J6CQN8eQdpczJC5t2FWtKD9UM33HkaY8aQQem7Yu2nz9zFE8cbBv9ld+4zndlbw/Ag2SSg8PWixxSYFXpK/FI5VrTLGKGAk7BOqcJt6LP/VRRK5v+qpYC1DjfKf+zc1ZcgiEXmQtG0RMdqmY4BgwEPX59Dsvw+khjgvylPJbVeFZ8E3ETDbblvW8+Xka5Qq3P+9fJmwV21rZ5BI+eNuBNtL2REUeFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gXqC7kdE6vgcxzHsZu6LKPSCCwekAZOdkt0dRfJilCw=;
 b=Hc02hEB4Ogp1mEAvFDowtjsJA98bTzyr1E/ahMO1JN3d0GewcSEFwl9NeHAvXQboEPS2A6tX1ArvVagrydPWp57slRMa9qYg7NNfCjN7tDCNpSNMRhGZp7F6bCV1sjOvxWUK4CoThYJpFPN7czwXTBZysHYNNweOTmS8vhi40jcmhvDgho9U15zGAR64xcDw9dc74Rhny8deo84Wc+qieTrA+oeHgD29998eZbgVNHth657OcAiC4nt68jvYqupWLZEiqIrF44HDL6EAXxPNU2/oLV6E1iTzvX8FMbvZjRgPs8FnMdUAfuZGf1wFx0+iVHerZMqCSJT5sZl4155ACA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/5] x86/mwait-idle: update ICX C6 data
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Message-ID: <fbe74c84-c9f9-dc00-a614-151634747f33@suse.com>
Date: Mon, 6 Sep 2021 15:01:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0181.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b5dc565c-0990-49af-0609-08d9713667e4
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4608868B81A5D4EAB470088AB3D29@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1850;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Foml9KDVDTIII9oDRVhzQetiXWAASLxxWUp+CozyqOhQXp3CKUw21a/FoF0QBZ4cW/7cN94Kk27e2OeYJpxFY4oQpBNIlcxjYBytawQ2gZMGxUOppibIuhwHjCqTkvmTs4oNcd4qbbXh8SS77Fdboek8FoTA+ViiB7zytSRsidBGkf3fbtdQczBWZaf0JN/6WRNfi2t9yaLBGXMiW8vL5WPt7e5wPOfvtTZzBGPirM8JCWvckrShRe77ZsUXFuZfm9aqfU/kmflRTD46YLrBq4PA9a3AoQSZtJ47o9dOAF2NL+IvB8IisdX1vAsAU9axU1QJPFhBVqpECboPf2YiTezuUJWRtO2TcMy30w4HY8FKpkU4lpaODbQVUk0U5Hns72JpUBLQkNUGDq0WmmWTZWX2XnHTxjnd1cdHtccJ43cVIEts3fPkGjl12F5zI34nQHlgwofk7m7npf9GOt1axwEVcvogytnH0Mi2zcYPOaO7scaTk4m4bSg1DgnEaCogPItvXFFLGH79Pt+EffDNV4cbz/ZjZFsDzFZJjPhLREA3MX2Rm7Y3fYzl33b+KVxr0ekZR3XcjMQ0JpQSY8xg6iONG2LihnNiFIObnSM/ICu6XiFu8hYFtp2gvQVu38nmDy4bAtJGqF3dJx2BIPcS2VknG5u9rgCFyy4ke/08Av/69IB52+fggWpNzI6hgYkgcd0kVPykHGaRL99my2w2VwMDEqeHY+VJ/QWH8lV9V5Cb36vvHWAT7eZb87kv4hWN
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(366004)(396003)(136003)(39860400002)(66476007)(66946007)(54906003)(478600001)(5660300002)(2616005)(6486002)(16576012)(8936002)(2906002)(31686004)(8676002)(316002)(956004)(4326008)(26005)(6916009)(86362001)(186003)(31696002)(38100700002)(36756003)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YjFqVWpTdktXT3hJV1RDVmlvMFgrR1JxNWtNd0c3WExQNHpZUE9VZXhObHNL?=
 =?utf-8?B?UThQaWxrTEdqTG81MzZrTDRKRzNMSk1zM3FVUHNOOGdIcGlBS1VmM2lNTmEy?=
 =?utf-8?B?TnNoSjdzbWVhbVhWTEFxNU95L3RDWkNrK2UzQnZjVzN6a0xuQnNtMUhkbTRH?=
 =?utf-8?B?dW1xUlN6VlRTYzJTVy9KdWhVVkpjY2lBSEQ5cGw4SDJjL3gyNnpMbm5OdHdq?=
 =?utf-8?B?eGJjTnVNT3RWRDlLNTlpMlFhbSttQ3RvMy9OK1dpOHlaVVUzMHBoL1ZBeUc1?=
 =?utf-8?B?M0ZMRVhtTEdkQUxmQVA4QTlzNTFJOFJxWFFtYXJad09uT0dubGRLTE5nY1Zk?=
 =?utf-8?B?TXl0OXhYamE0SkpwQkRzVFJxbVJHZUlXRDNKSk1aZDkyTnhwL1YrMWlFdUxY?=
 =?utf-8?B?d2pEbUNTRy8vQzQyUTRKOVcxdGR0MVhFNVJTNVlMZFhZRy9RSzJ2YWdGSWVj?=
 =?utf-8?B?c213VG5ycHhpcU1oSlk5ZzQyT3RRYXFWOC9ZN2Jmd2JUM1M0dkl3R0VRS3o3?=
 =?utf-8?B?MUxHTFNWbko0bit1NlAxczRHVzhsQW5tTktPdWRBME1pQWFzYXNNSHNwamhJ?=
 =?utf-8?B?SmROOWJhSDU4S2FFRnMzdktjY01nZDg4bWNNRmo4YVZMQjBUejJhZ1BILzBU?=
 =?utf-8?B?SWtHT29jU0hKMEQycnFVY1MyTDFqUklURmVrcXlqMnRmMmVIVERVR2tuR0Qx?=
 =?utf-8?B?TlpGN0l2WHIrS1I4Z0lwM0svMGlHa2FpaXRrUFdPaGExQW5lcW1zZml2aGh0?=
 =?utf-8?B?eHRJbmFnQ0N3Q3NNdjR3U0U3V2ZmMHdBN1dzSlhqc3hKd0g5a2lBWEpHdElP?=
 =?utf-8?B?SnVqeHg0R0hlUXNuMjExa1ZZa2I0TWVJVXdWNlNIajcyT2s5WG5PRzdmSUky?=
 =?utf-8?B?Z0ZNNkduMUZpQ01KY0JLZ1YxaTRGUm9aRXlaRDd5blNiSDBqRms4YVRoeTFU?=
 =?utf-8?B?aU1RbzFiSlpRM2c2Y3hqQVlvY1N3RE1rOVV2NzZEQkU0THI4WnB4aDVCTitm?=
 =?utf-8?B?cEJLeVpscGs1VHpKWEpxa2FBVWJxRmo0czllYUFPeEM0QU9iVmJoWVpEQUxr?=
 =?utf-8?B?UE5rOE1sTmhLTk84eExiZ3J4UDhTTktESkMvalA2cm9oQU16b0dnUjdwUTZn?=
 =?utf-8?B?a0pZbkJMQUNQWkljcTVRcnlLUHc3Q0UwMklQVlJNWXBkcG5TSkVIOGxFNzlr?=
 =?utf-8?B?bE05MDVHcmxCYS9kU3NCZm9NSWFtWHRrdnM0TFZ5bVJLaWJqc2NpU0hNN2hM?=
 =?utf-8?B?aktIWmpXVnFnMGM2SjBiNUJFNHdvNUEreXdHaEZPSFJINzE3L2R6aEhDbU0z?=
 =?utf-8?B?cGlhWHR5TktBVnRNVi9rUmZ2MlJYYnAzYzQ5VFRCNlM0ZE5xSDd3Q1p3OWlE?=
 =?utf-8?B?SjZjS1NqM3MvbXcrcytNY2xqaWU5d0tZZlBHUjZuZkZiOXlJVVdjamc1UDhI?=
 =?utf-8?B?TlJmMXdRU0NFLysvSXFGV1hia0t4TVdvN0Y0SUh4ZGM2aDcyeU9iMitHZ3FC?=
 =?utf-8?B?MHI0UndwTFRUdW9hbTF3VE5sa0JpRE8xVDVQWXpVZm8xKytibHlTTnQ5akdR?=
 =?utf-8?B?MnZlS3c2TmplN2dBVDYweVFPcEJ4MzBFbnYyMmIrdmFyWWNqK0pYaG83NUtF?=
 =?utf-8?B?ZXcrQitOamFBUWJsS05BSFpGQ05jaFVQWjd0QVIyZkIxT1hia084UTBRTXE4?=
 =?utf-8?B?QnpNNWlWUHZGbGRMYm5TRFNpTnJpVExobFlBYmdXMzFOdzRyOU1VYVhaSjB2?=
 =?utf-8?Q?Rov4S/z/iy3boIpeBy/AWAYbIFGvkp9GjsKV3rQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5dc565c-0990-49af-0609-08d9713667e4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:01:13.5545
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e86jtHR11cuJbPnyo8CNIznS+QrImzwD0T8IPf+jtAG84+eAbtziWKRHjawtb5uL0JY7IywiZSgCX6HHKrIe5A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Change IceLake Xeon C6 latency from 128 us to 170 us. The latency
was measured with the "wult" tool and corresponds to the 99.99th
percentile when measuring with the "nic" method. Note, the 128 us
figure correspond to the median latency, but in intel_idle we use
the "worst case" latency figure instead.

C6 target residency was increased from 384 us to 600 us, which may
result in less C6 residency in some workloads. This value was tested
and compared to values 384, and 1000. Value 600 is a reasonable
tradeoff between power and performance.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Acked-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
[Linux commit: d484b8bfc6fa71a088e4ac85d9ce11aa0385867e]
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -574,8 +574,8 @@ static const struct cpuidle_state icx_cs
        {
                .name = "C6-ICX",
                .flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
-               .exit_latency = 128,
-               .target_residency = 384,
+               .exit_latency = 170,
+               .target_residency = 600,
        },
        {}
 };



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:01:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:01:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179837.326250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEG0-0007pb-Vp; Mon, 06 Sep 2021 13:01:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179837.326250; Mon, 06 Sep 2021 13:01:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEG0-0007pU-R3; Mon, 06 Sep 2021 13:01:52 +0000
Received: by outflank-mailman (input) for mailman id 179837;
 Mon, 06 Sep 2021 13:01:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEG0-0007pK-0a
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:01:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 994aa0fa-0f12-11ec-b0a4-12813bfff9fa;
 Mon, 06 Sep 2021 13:01:51 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-A-cj4z9TPxa0qOumc9_p6w-1; Mon, 06 Sep 2021 15:01:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 13:01:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:01:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0226.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 13:01:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 994aa0fa-0f12-11ec-b0a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933310;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=C0VetijaR8xkFGTQTUwV6cJiozc54HeEusdPV3iHAi8=;
	b=UOa/xj8QqxMsM91USdtP/TgHQ3lZcGg68peADaIWNS7N8KIflac2ukZ4SCZwSGbAHJgcsz
	uZSjEJfZwEg+BlVennC51N3cTkfv43ak/AYdCak5MucW9+8bofx9uCBYnUW5O7a1H0lfmY
	j//yN4NOCz2bVRDrUV9bzJsgba4a6pA=
X-MC-Unique: A-cj4z9TPxa0qOumc9_p6w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KrsUmg3yW6KZu0bukEOz0NRdeY/4rpJse07BhCHEWkbgLQr8Itr1Kj3+cMcdbccwLiD3Cjj/lDmfUGZpw2rEmwtrOV1o0zwwl6y15Ve02ICKH3cU75wHQyKO5h4hB1oxCtfte6QmpVe9bbpA5EVsEpnZWe4laLDAjRzp2S43QHg5VehRI0mE6oglth8FYbgoQmxQtvbLeo8t7LXLg6pDqLFMWzL0osvGNAAWSWO2SSQY6i7C1gF7ECg3ZFk/54orM+/zxFlF3UX74rkUwIcadlJEiizeLTWFEYZlEEPxcMMSiRF8Do0EYoBCuxn2vjj8kX65t8M/svQG7jWCfEgGZg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=C0VetijaR8xkFGTQTUwV6cJiozc54HeEusdPV3iHAi8=;
 b=JyFs+DCOG0DB8e3UF2xtNB4E5nlICNQquHcqO9YeI21xgUbJt/Vg2TJZxT7hedCORe0uorQL/si889p/Y8+Pmlq7R9/7KLOX8Q/y65xWuwVzBmgGeNt1MfZcfelqoVOha65tP/pfsSaPxhjDjvguhEbalMD3bBwHQBhIWDIsH6xZemzlKH8BMg6U8En7MQWXMRTi4tD6nY5xXHXFOQRcQ8DAgvqLCUAjSMHn5u8eM1Kj40yB3rRbOjdeOzPj+RrfrMMx+3dTPSHKN89QGMWGZbxRZaWhiMYDhCsTZhZ/DjuGBZdUHL847Ae+HmoJO9xzMcCEPtVSjpGSlZQiHd61vQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 4/5] x86/mwait-idle: add Icelake-D support
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Message-ID: <f18d6203-7cde-40be-e645-f5460259c416@suse.com>
Date: Mon, 6 Sep 2021 15:01:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0226.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 877498ab-33e6-4fa5-75b7-08d971367c48
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4608D8A7F316FC9D680208E5B3D29@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:483;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	txKJ/x3HL6XEsk8VSDAPxfPxRVWf5wotqkw6RHCyU57bMCSD4kCO1DwYDnTOmfMLPxuR/T9ldhGy5bwR8KFQmL15w+2aFnVhdH/PfzBOtse6flTqTTbnkv3trrQ4vmrF4xA/rXpwRRDnYLKCpk9ZbtXtdItZTcwqKs6vEMsLWxE0z0sEdWTStnbe243cFJjFHWY+YmPT2P60mVw6a2Jw9MJZtX9tHOl2neDJ71JulngDDYuDyIHhMvR9rIPk+bvfZH6mOLzl9YxetKjNhPS/tEoyuQcGUthu8KkD0fpFIzbNCdptKyRCsSiPf1uq9UnyBOcVI2wRYDiG0kBrVyEwZ0/QI3zllb3965UANPvZPhGFvKXKWQUJvXcmhs+6XEAmO9neJh6PY6ejTrrDtFjASito7CfCQA0Gut9m9z0gBPHCQtCgOGpaJCgw57WcCOK3Zakrq1v1zAy5mY28+n9Ymj3Oci//GJZ/7qB195xHEfyjoeVrj2cEEBwZFGMFruUwUnIw+JZ7vD/6WJ9aFRjgfdZYfIJJ6bAut+Qh6OysRIAsmbJqKbZgWJAOvs3AYDb7V91PnFCbdbwKEHNs+nXxs320h2xKIp5ddK/6AoSjWg1ynpZzAtHbbtjYgXkardvsqS/YLAEom6AuQp9I6hTsN2hgpteLF8H1r/6zWKOIub7IuoU+qmAypx/cwHe8thwhuTtPK5X3HLmqJi3eAdFmY1IOe93KyoLUPt80W4Vhd/8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(366004)(346002)(376002)(26005)(4326008)(86362001)(186003)(6916009)(316002)(956004)(66556008)(4744005)(38100700002)(31696002)(36756003)(5660300002)(2616005)(66946007)(54906003)(66476007)(478600001)(8676002)(31686004)(6486002)(16576012)(8936002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VWF3UkpMK2hyUmV0M1hTUDNNb3l0ZmRkQm83OEh0VmQyVzJ0dFBqT3pFQVcx?=
 =?utf-8?B?eDdNSVQ1SE8wWEsxUFRJcFBMYkxOQ0ltTUE3SWRML0wvNk1kQ0VLNU5IR29N?=
 =?utf-8?B?ZUhMWnJISFhQR1BOR3hSL1JZZURsS3JFNWtNRFRQUUhadlQxL3kxSUVyeEUv?=
 =?utf-8?B?Z1VrMzcvbHI2SzhYUzY1RzdoTVVmTGphaWpZUHVvM3JUeUoyWW5wdFMxcGJl?=
 =?utf-8?B?VzdtY0twZkFBR0sxSHdSWk9FamJWS0JGelJNcWI4OExtQXA0WTFIZi9zTmxh?=
 =?utf-8?B?Z0IxUmhxRDEvclRSOHRHa29DaytUTVV0UG5KbzJ5enpXdXJJeDkrUHkzWmlQ?=
 =?utf-8?B?Ymc5bktNMzhMbnByZEhWZ2VqNmFGTjRwTTVUM2JjRmN4bVk3Mm5oelFqOWwr?=
 =?utf-8?B?Wm9SN0FJV0JTS1NQVUROTUQvdEtabWJsYlgvNTJuS2Z2RDMxU0lsRXRMQ2dp?=
 =?utf-8?B?QmhRL0F3bEJ4aWlvZ0NTcGZ2L3AxVWtqWnRaOVdFU0t4KzFPRWpBRkVyRkM3?=
 =?utf-8?B?NkZudk5vLy8vdS94eXQ2a1UwVFgrME1LSEhKWklGZVA4TWhJQmdLbEwrMmR0?=
 =?utf-8?B?WXM5WjVuWEFOSXplczQwR3lwTHo3T3hzeU9zNm9BTzZDTXR5MWlkRnBkbVNS?=
 =?utf-8?B?RSt6elJzS2tGM0dZK0gyM0FsUU5Fb0hZZ0U1bUhHSmJnNEJ1SWdaNW5IRUEz?=
 =?utf-8?B?WlNPenhMQ1ZhTTNvU2JaaGVIWFJMdWFLRTEvdjFua0ZBaUFaRUtuYmhxeS9a?=
 =?utf-8?B?dkRQVGhWL0lvbEdNVUpRbXZGQWsrQ2dhdWV4RURFVGJ6WDlHSi9aR1lwODR2?=
 =?utf-8?B?NWc5dFdQclNNMXJMamwyKzJBTDlmd2l6S09uMXRVWldsVVB4UkMwT04rMlZB?=
 =?utf-8?B?MXRwOVpMT2YvS0s1Y3VLanVRYTlZZnhrekhMcklKWGUydGFTK2h0eWhNcWd0?=
 =?utf-8?B?SU8vTW51SS9lTXFPTnREeCszU0pHbmJVZHhhM1hKcXpMWm5yWXpHTDdyMGFW?=
 =?utf-8?B?MHRFdWtUR09ld0VhR0owandNY0RBK3J1N1N1SEhIVWNyNUYyQmMwRFpSRHI4?=
 =?utf-8?B?TzJ5MG5hZ0I3SXRid2VjSmswUUlMN1RRbnN5c3RQcnNkT3JFQUJUMkhBbnVL?=
 =?utf-8?B?c2pDNG45Y05DOWMvelBTRzgreVJMRi82QlRCbVl6ZnBTTW1ycDJYd0JBaGNU?=
 =?utf-8?B?THdrcVMxV242MEVuTmdwZ3pJNHd1Wms4TFZFMVJ2TjlTM2lIVGZTT0srMEoz?=
 =?utf-8?B?MUZLV2l3MjhtcWRlazNmOUVpeW1vdWdIQVQxdWIzYTJvcWd0WEx3VVRmNm16?=
 =?utf-8?B?RG01dVFvTlphdkgxSHFTQ1pCTEtqSUI1a2RUTDBJMko0WEFmSzBxK0pEeUhH?=
 =?utf-8?B?UytCejZRU3FidSt2K1dMYjdQOHpjN0RtdVVyYksyNkZ3OEdTdmt4RTd4bWdQ?=
 =?utf-8?B?OU14MWEvTEFWR0xRdStTRER4VUFjTkNSTWZCNS8rUXZzZXc3NWdBaHQrZndP?=
 =?utf-8?B?cUpJRzJsL0FhMTNMWW0wRXIxR0h5SHFjWUVwd21iYWkzcWlVRXY1NnE0WFR6?=
 =?utf-8?B?RG9majhKMjZ6YzZ5WHBMZ3hOOGJKaGdNcC9scHltWmxkOVpZcmEvZlNSdE9r?=
 =?utf-8?B?dnJ4QXVzUVZTak90U1NEWHA3cXl0WDdqZW0zSmVEYXhSTWY2RmxKeWpTQkZK?=
 =?utf-8?B?dElrZnNyZ2NsSnl4SzQ3c0FkV0g1S2FDZHJzM1laVVdMQ1B3Yms0R2QzdjVi?=
 =?utf-8?Q?g6SjCp7nkgPZiibm/vS9jUAWACVYG7MG/lQ8/BP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 877498ab-33e6-4fa5-75b7-08d971367c48
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:01:47.7651
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LG5szUu94mJS1RPw6ilyEDfhU1Gv6yB8pB8NaGL9KMs+Ovb+MB0KUw/FnSh1/nCSAk5xjlXiv2amEQbsUcTMSg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

This patch adds Icelake Xeon D support to the intel_idle driver.

Since Icelake D and Icelake SP C-state characteristics the same,
we use Icelake SP C-states table for Icelake D as well.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Acked-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
[Linux commit: 22141d5f411895bb1b0df2a6b05f702e11e63918]
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -1021,6 +1021,7 @@ static const struct x86_cpu_id intel_idl
 	ICPU(0x9e, skl),
 	ICPU(0x55, skx),
 	ICPU(0x6a, icx),
+	ICPU(0x6c, icx),
 	ICPU(0x57, knl),
 	ICPU(0x85, knl),
 	ICPU(0x5c, bxt),



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:02:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:02:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179844.326261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEGR-0008V2-7s; Mon, 06 Sep 2021 13:02:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179844.326261; Mon, 06 Sep 2021 13:02:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEGR-0008Ut-4V; Mon, 06 Sep 2021 13:02:19 +0000
Received: by outflank-mailman (input) for mailman id 179844;
 Mon, 06 Sep 2021 13:02:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEGQ-0008Ue-FX
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:02:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a92c2aa0-0f12-11ec-b0a4-12813bfff9fa;
 Mon, 06 Sep 2021 13:02:17 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2050.outbound.protection.outlook.com [104.47.5.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-QMFLyJeKMY-g26Ww_Z7L6g-2; Mon, 06 Sep 2021 15:02:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 13:02:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:02:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0042.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Mon, 6 Sep 2021 13:02:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a92c2aa0-0f12-11ec-b0a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630933336;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=q25vTfg9n5hflZMUVunRhlsx6PeE8kCcVjZSgImDym4=;
	b=Hil60oNA16WPL7nAR9GmglMgALqsmElPcmp7kYa1e99cTME/rfDrrxEITbHYqgWF4taPnx
	COm/MrHVqkV3SqXlYOPPhyQHaF+qouiLrLVht3Z/9Fkh+BBNp9jDtRP4ENGGgnZE486WiB
	AnMAzkBjxOao7l0ShqA1uwv9cFad578=
X-MC-Unique: QMFLyJeKMY-g26Ww_Z7L6g-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wpk2a+RIrLwrozxDrlSrKSr+lrUEz7Qyzi0XBq5rCfDv1GoM4sHuR4rZFTI3lklQ5wW2FPvQLAJc71WboulW5/qvV94Vb61eHbWOkKEP6u4S/y182aEEumftrSyFyIXUVsGrd+Z6eQTV7eMUQA/8kcPdnDo3CVX4HF+EPRqaMpTWef9lYO+KLFlJQeaTWZYxsMsGtfsqbFljLGmhSW/cLG9dmrqWxtIHKtg6ZDKhLQG1YyKUiUxq5jxPdepKaeYlZpQNsUxUireO6QGkmSUky2RN0ZLilno7H7un/V3sqFTWDS2TIpL2qvB4dFR+AiDPTRO2CosbI1TkhBO9rX1X3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=q25vTfg9n5hflZMUVunRhlsx6PeE8kCcVjZSgImDym4=;
 b=hYJTTisj4WdVmIQ17WL2uPySfg1kK7K6LElXWrUDjAsXNoXYCNlNO7yYjFKgzEwN3FKA3boWAXwq2nJ6cVdwi6PNtjPKKKMThaxVSt0gyB6RgDmGalP1zpzIU14K5FnO3pIhArPr70TaT1NgCslFBXl3tPZtodqO57L2V/lvJgSJ03omu7gvOav990u28gRUN51F2H2qZkkz11SvfcDMj14cV2yRBt7IijKYcuarT2ZilM8eJO4nByXvs0ajsuI4XySd1Kt9DSnWX4DzdobNZaD6+LnB7HlFT8vqEs1/loegrkDjc5eCIj5LswxMes3b/kP6otqvQ3b6cJUMTcyPUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 5/5] x86/mwait-idle: adjust the SKX C6 parameters if PC6 is
 disabled
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Message-ID: <aa7dc2e4-6a80-dd7a-81d6-92690f6e0bfb@suse.com>
Date: Mon, 6 Sep 2021 15:02:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <e16ff1ad-b1c5-f4e0-9336-716eceb93a9f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0042.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e6c4a2ae-3b3a-4497-6e39-08d971368be8
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4608D15FB7D47D1C0CF87EC1B3D29@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sOgmmceWZFhjWuATx/r6y2TEs4E0O4koN+SZvZW1sgtpSVSlr+gW88szl4fS/17a6xf2zHUFAoBKtRl6GuIl6qtATTMIYeepIFt+IJy4FqAAplMGMibR/ZRaYSnNtmY0QcVLYrA3udvnmlL4qm3tOqfFhIHmYucLNv278yLxoegEwVwv1N446PKm/JHD8sjtI4E3cd5yKmCZuQU/gvDtk0FwZflk/Q6QTsPjHzxIUNyE6MwnsnDjhfX95Sqq9FQEU/39Uylu74NYVpyp/JXRLv0WOw7yGk7ZsB10fGQmrLg1R9xQcdVe1DzNo7j6XMTNrliiBZwF7m8bfo5bdkyEjlVf689DvmR5aonHh2TfDDg5zE43I05sOY24doNkR5MNkqy6vqYEqtVDNLNxqBFrcvyvuXy8cZaODeli7HZJHK5+9EdrjADOHKfVxpNQLalZ/02XNLlMi7OdSEQd5mCjx7I9x5H1EElUQL33en9XOWa6hrRjo3uw0h56At1BTNHSlAa6csds4KFBT+WqQYESiItiv7TqYK3wphunlJWHJW4ClCzU8tPsl/IhpC166btTw736q6voN+7Xb4x0f2FEkHj1EX8q9EXy66R5e+fyANtw567Xt+H4co4FuNKJkxVh3OCCD9mEKVwsi60riGakBvRuOv/6zuDul08Sl66CUbIkccrvo+t9mHXdr+jJ6uhv8R91JyJBa1PR8C8B2OTMgRYyoYlslVfGNPXhwWEFtOP7U0yBW2gYMHLkhNaO+kU/2REB8YRtvo3pY+8hTLOmc+ehYzPAay3AfBIoM6h/IEVzJ8tuVrGeW1S/hUwxZVuw
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(366004)(346002)(376002)(26005)(4326008)(86362001)(186003)(6916009)(316002)(956004)(66556008)(966005)(38100700002)(31696002)(36756003)(5660300002)(83380400001)(2616005)(66946007)(54906003)(66476007)(478600001)(8676002)(31686004)(6486002)(16576012)(8936002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cTdJZ2ttbGdxNXF0WE9QdVNPQU9nVW1CTGxyamkvUlNsdzZTL04reDhrN2tj?=
 =?utf-8?B?TFRWaFRBeFF1OEQ0bzhXYzk2UG5WZG0vd21aZk1WK0JSbE1MV00vM1hVWHZM?=
 =?utf-8?B?WU9lMkp1M25yU3pqZ2hHSUkyZzdYQU1rT203b0YvY2dLQnZzdHEzUE93SEdO?=
 =?utf-8?B?VDdCU2RwYkdQSTFOMm1aazR4OUh2dzZ3M25xckgwS0JGSlFWcnZ1M1RPWXJu?=
 =?utf-8?B?Uy9pTUhZRGhHUVU1cWNZSDBSVU9lWkplWEplaTJGU3BXbzR1QmhPUTVpVDhW?=
 =?utf-8?B?NUZ6THgwQStqRUIxcEZhdEFybklZY1JUdGVvTUhLcG4vRm5sb3h0TG5WSlZE?=
 =?utf-8?B?ejZ1TjJBelZqcWdtdmdXZHNiQXB0RHVGdkM1Y0xsRmh4MzhIazJnazFnYzA4?=
 =?utf-8?B?cFBBazBSRGN2SnJ3dGVWQ09FL0Z2T1hjNGgwck5NSVVhU3ZNQ2JBcFlSUjFX?=
 =?utf-8?B?NHJNYzRlekJBemFnM3F6WXB5ejFFZUlSaHhqUkVWaDRObTNaNHd0bEI0TERk?=
 =?utf-8?B?dVpSTzJHbDlSczJrZ1o0M3BFWnhoM0VKZzRFUEhIQTNqUDR6V0xVUkFDcUlR?=
 =?utf-8?B?djZpdnpGSGZCVVh0aVJKUmoxR2lMY3FwemRDeEQwcStoMUpiRSt6NkIxZ3po?=
 =?utf-8?B?WGhrNTZmQTkrdG9KK1ZKbHF0QUZwYzk0eUJZUTRlbnlRekhJRUZQeG1qUGQ5?=
 =?utf-8?B?WVozQmtKNTRCaU1wQ1pNR2h1dFd5ekE5MVphQVh1cmF1d0EwekRhYTg3L0dr?=
 =?utf-8?B?a3E4UUVQMUE3MFlmQldOK2d3TTV0WEtxQWE4ZkVyTnpub0F6ZHc3Q1ZNVGwv?=
 =?utf-8?B?RVFaRVV3d01UZGE5TktlQ3daVkpxYXRzY0NqMGFBcGhHT0piQkJkeHpRY0Nj?=
 =?utf-8?B?WE9mU2E2YWZrVEhidkJubXJ2TEZPZktyQnAzcnpWWHJYd2VTWXhZVHBwY3lB?=
 =?utf-8?B?ek5PT2liVlpHL2RzNzNzK2RXNzlvZ0g2TWQzUVFBS2loTE4xUEs4a09kRFQv?=
 =?utf-8?B?MXZFMzNQcGZudG1oSjNwSWlHQ0VQTnJmcndsUEl4QWJ5aGFGZjBKazVzMUpH?=
 =?utf-8?B?Y3c5ZlVsRWg5NVp2TS9lSG9CWDF6VFBHQmxIOFJGK2NGb0Q3RFg2eUFWQ3pO?=
 =?utf-8?B?eFljTzkwSWRuWEZWek5FUTNsZHRpRVQveEtiSmMxSGxqY0N1aTl3dXR2cjdl?=
 =?utf-8?B?ZnBXd2VSRnZRSE4wYnRoN1pmb0xrSUc5cjRPVjlWYWJLeGJFT21xTzBYdjRr?=
 =?utf-8?B?SEdMYkM3em0vNGI2b0NRaWZpb1A1bU04aW1iV0tSWUVlVjlVRHBHNk5sdUo5?=
 =?utf-8?B?Z0hTamJFSUw5cGpXNnFZK0RjVjgvYVgzMzJaNjBWUGI3UGpmUnZHYWgrSnpE?=
 =?utf-8?B?djFYcGltYmdHeTBGOHkxSTZvb1lYdmhmZjFCYU5ONzRWK21tendXZytLeElG?=
 =?utf-8?B?SUc1MkhOOVJpRytaMUtMTWQvV2VmS1lGT3p5Y3orVUNJL3lvdldKY0hSN2c2?=
 =?utf-8?B?VVhoREY1TjFvdFl3anQ2cFRiZWpPaE9WMzlHM20xOFEwZWtMY0V2alFwRFJt?=
 =?utf-8?B?Z3lKcEJ6dWRaTHI1M2syN1Yrc25uVjBVMGhDVnVJczFKL053ajZ1LzFuR21B?=
 =?utf-8?B?S0tnaFZPYmVEM09QQzAvY3pHYk1pY3RPYkx5M1QzMjhqRFRiQzltVHM0T1JP?=
 =?utf-8?B?eGhPZ2E3VGgvMDdsS3cwcStDVnNTSWlUazFXZTdyRjgyQnBpSEY0QW5DdUVo?=
 =?utf-8?Q?6orz3kj96UWtAkQqJpg0HdyNl9tp7twDD8UoWyt?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e6c4a2ae-3b3a-4497-6e39-08d971368be8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:02:13.9873
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qfIPbEufWImrkCsk6wMr8vC8dfpNpdXMi9d95/6Es/JsTHPQD9knzDbeBrYwJWERQZu70yt3oTC6PlFrSeBS8w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

From: Chen Yu <yu.c.chen@intel.com>

Because cpuidle assumes worst-case C-state parameters, PC6 parameters
are used for describing C6, which is worst-case for requesting CC6.
When PC6 is enabled, this is appropriate. But if PC6 is disabled
in the BIOS, the exit latency and target residency should be adjusted
accordingly.

Exit latency:
Previously the C6 exit latency was measured as the PC6 exit latency.
With PC6 disabled, the C6 exit latency should be the one of CC6.

Target residency:
With PC6 disabled, the idle duration within [CC6, PC6) would make the
idle governor choose C1E over C6. This would cause low energy-efficiency.
We should lower the bar to request C6 when PC6 is disabled.

To fill this gap, check if PC6 is disabled in the BIOS in the
MSR_PKG_CST_CONFIG_CONTROL(0xe2) register. If so, use the CC6 exit latency
for C6 and set target_residency to 3 times of the new exit latency. [This
is consistent with how intel_idle driver uses _CST to calculate the
target_residency.] As a result, the OS would be more likely to choose C6
over C1E when PC6 is disabled, which is reasonable, because if C6 is
enabled, it implies that the user cares about energy, so choosing C6 more
frequently makes sense.

The new CC6 exit latency of 92us was measured with wult[1] on SKX via NIC
wakeup as the 99.99th percentile. Also CLX and CPX both have the same CPU
model number as SkX, but their CC6 exit latencies are similar to the SKX
one, 96us and 89us respectively, so reuse the SKX value for them.

There is a concern that it might be better to use a more generic approach
instead of optimizing every platform. However, if the required code
complexity and different PC6 bit interpretation on different platforms
are taken into account, tuning the code per platform seems to be an
acceptable tradeoff.

Link: https://intel.github.io/wult/ # [1]
Suggested-by: Len Brown <len.brown@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Reviewed-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
[ rjw: Subject and changelog edits ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
[Linux commit: 64233338499126c5c31e07165735ab5441c7e45a]

Pull in Linux'es MSR_PKG_CST_CONFIG_CONTROL. Alongside the dropping of
"const" from skx_cstates[] add __read_mostly, and extend that to other
similar non-const tables.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -484,7 +484,7 @@ static const struct cpuidle_state bdw_cs
 	{}
 };
 
-static struct cpuidle_state skl_cstates[] = {
+static struct cpuidle_state __read_mostly skl_cstates[] = {
 	{
 		.name = "C1-SKL",
 		.flags = MWAIT2flg(0x00),
@@ -536,7 +536,7 @@ static struct cpuidle_state skl_cstates[
 	{}
 };
 
-static const struct cpuidle_state skx_cstates[] = {
+static struct cpuidle_state __read_mostly skx_cstates[] = {
 	{
 		.name = "C1-SKX",
 		.flags = MWAIT2flg(0x00),
@@ -674,7 +674,7 @@ static const struct cpuidle_state knl_cs
 	{}
 };
 
-static struct cpuidle_state bxt_cstates[] = {
+static struct cpuidle_state __read_mostly bxt_cstates[] = {
 	{
 		.name = "C1-BXT",
 		.flags = MWAIT2flg(0x00),
@@ -870,9 +870,9 @@ static void auto_demotion_disable(void *
 {
 	u64 msr_bits;
 
-	rdmsrl(MSR_NHM_SNB_PKG_CST_CFG_CTL, msr_bits);
+	rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
 	msr_bits &= ~(icpu->auto_demotion_disable_flags);
-	wrmsrl(MSR_NHM_SNB_PKG_CST_CFG_CTL, msr_bits);
+	wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
 }
 
 static void byt_auto_demotion_disable(void *dummy)
@@ -1141,7 +1141,7 @@ static void __init sklh_idle_state_table
 	if ((mwait_substates & (MWAIT_CSTATE_MASK << 28)) == 0)
 		return;
 
-	rdmsrl(MSR_NHM_SNB_PKG_CST_CFG_CTL, msr);
+	rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr);
 
 	/* PC10 is not enabled in PKG C-state limit */
 	if ((msr & 0xF) != 8)
@@ -1161,6 +1161,36 @@ static void __init sklh_idle_state_table
 }
 
 /*
+ * skx_idle_state_table_update - Adjust the Sky Lake/Cascade Lake
+ * idle states table.
+ */
+static void __init skx_idle_state_table_update(void)
+{
+	unsigned long long msr;
+
+	rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr);
+
+	/*
+	 * 000b: C0/C1 (no package C-state support)
+	 * 001b: C2
+	 * 010b: C6 (non-retention)
+	 * 011b: C6 (retention)
+	 * 111b: No Package C state limits.
+	 */
+	if ((msr & 0x7) < 2) {
+		/*
+		 * Uses the CC6 + PC0 latency and 3 times of
+		 * latency for target_residency if the PC6
+		 * is disabled in BIOS. This is consistent
+		 * with how intel_idle driver uses _CST
+		 * to set the target_residency.
+		 */
+		skx_cstates[2].exit_latency = 92;
+		skx_cstates[2].target_residency = 276;
+	}
+}
+
+/*
  * mwait_idle_state_table_update()
  *
  * Update the default state_table for this CPU-id
@@ -1178,6 +1208,9 @@ static void __init mwait_idle_state_tabl
 	case 0x5e: /* SKL-H */
 		sklh_idle_state_table_update();
 		break;
+	case 0x55: /* SKL-X */
+		skx_idle_state_table_update();
+		break;
  	}
 }
 
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -45,6 +45,13 @@
 #define MSR_CORE_CAPABILITIES               0x000000cf
 #define  CORE_CAPS_SPLITLOCK_DETECT         (_AC(1, ULL) <<  5)
 
+#define MSR_PKG_CST_CONFIG_CONTROL          0x000000e2
+#define  NHM_C3_AUTO_DEMOTE                 (_AC(1, ULL) << 25)
+#define  NHM_C1_AUTO_DEMOTE                 (_AC(1, ULL) << 26)
+#define  ATM_LNC_C6_AUTO_DEMOTE             (_AC(1, ULL) << 25)
+#define  SNB_C3_AUTO_UNDEMOTE               (_AC(1, ULL) << 27)
+#define  SNB_C1_AUTO_UNDEMOTE               (_AC(1, ULL) << 28)
+
 #define MSR_ARCH_CAPABILITIES               0x0000010a
 #define  ARCH_CAPS_RDCL_NO                  (_AC(1, ULL) <<  0)
 #define  ARCH_CAPS_IBRS_ALL                 (_AC(1, ULL) <<  1)
@@ -175,11 +182,6 @@
 #define MSR_IA32_A_PERFCTR0		0x000004c1
 #define MSR_FSB_FREQ			0x000000cd
 
-#define MSR_NHM_SNB_PKG_CST_CFG_CTL	0x000000e2
-#define NHM_C3_AUTO_DEMOTE		(1UL << 25)
-#define NHM_C1_AUTO_DEMOTE		(1UL << 26)
-#define ATM_LNC_C6_AUTO_DEMOTE		(1UL << 25)
-
 #define MSR_MTRRcap			0x000000fe
 #define MTRRcap_VCNT			0x000000ff
 



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:13:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:13:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179860.326276 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNERF-00020H-B0; Mon, 06 Sep 2021 13:13:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179860.326276; Mon, 06 Sep 2021 13:13:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNERF-00020A-7r; Mon, 06 Sep 2021 13:13:29 +0000
Received: by outflank-mailman (input) for mailman id 179860;
 Mon, 06 Sep 2021 13:13:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNERE-000204-A0
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:13:28 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNERB-0004RC-I9; Mon, 06 Sep 2021 13:13:25 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNERB-0003Lx-Bb; Mon, 06 Sep 2021 13:13:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Y12aQVs2B09axgUlZvQNFaktwJQZPR6LaeC5gO9J3Zo=; b=Wr4XyjbkMShdiCiWcLXwSdIOCH
	fkgKPuhnWJ469AQgsKrVQoZLuuJsY8xSdHNVQjRRoa8YPveSdetCTV2E4r0Qquh/8+xDMGX6uM96E
	VIgA/NosXJ8i1tZUDtjM7oZCblYewEvXrquYxVobPSOYHkQF9HcvSmV2N22moyt/3sIA=;
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <7b8fba1a-767e-87f8-d0eb-5af5e5427bb4@xen.org>
Date: Mon, 6 Sep 2021 14:13:23 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 26/08/2021 11:09, Jan Beulich wrote:
> By default all guests are permitted to have up to 1024 maptrack frames,
> which on 64-bit means an 8k frame table. Yet except for driver domains
> guests normally don't make use of grant mappings. Defer allocating the
> table until a map track handle is first requested.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I continue to be unconvinced that it is a good idea to allow all DomU-s
> 1024 maptrack frames by default. While I'm still of the opinion that a
> hypervisor enforced upper bound is okay, I question this upper bound
> also getting used as the default value - this is perhaps okay for Dom0,
> but not elsewhere.

I agree here. I think having a per-domain limit maptrack is a good idea.

> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -633,6 +633,34 @@ get_maptrack_handle(
>       if ( likely(handle != INVALID_MAPTRACK_HANDLE) )
>           return handle;
>   
> +    if ( unlikely(!read_atomic(&lgt->maptrack)) )
> +    {
> +        struct grant_mapping **maptrack = NULL;
> +
> +        if ( lgt->max_maptrack_frames )
> +            maptrack = vzalloc(lgt->max_maptrack_frames * sizeof(*maptrack));

While I understand that allocating with a lock is bad idea, I don't like 
the fact that multiple vCPUs racing each other would result to 
temporarily allocate more memory.

If moving the call within the lock is not a solution, would using a loop 
with a cmpxchg() a solution to block the other vCPU?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:15:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:15:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179876.326290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNETS-0002l4-U1; Mon, 06 Sep 2021 13:15:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179876.326290; Mon, 06 Sep 2021 13:15:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNETS-0002kx-Qd; Mon, 06 Sep 2021 13:15:46 +0000
Received: by outflank-mailman (input) for mailman id 179876;
 Mon, 06 Sep 2021 13:15:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNETR-0002kp-LG
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:15:45 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNETP-0004V7-U6; Mon, 06 Sep 2021 13:15:43 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNETP-0003Vs-OG; Mon, 06 Sep 2021 13:15:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=NJgZOIAhM0FRVum0rn4oVDRo/18ADiT0ruigwcfl41Q=; b=XW0IShM2XbHG4wk6yifxpLIXOJ
	auG9rIrvW9Oldb8T0hnV61Rt299IwvLueOVjPdpa4bDlb9kekT42qOAL/o/ZJFJxembj0fFUudmC9
	+VRpfR4Bn7BiakMNDsfQ8+pyuCEf4sJnreGjZPBcb8GeTVJnoOMx9AQRIda6LIxHb/5o=;
Subject: Re: [PATCH 2/9] gnttab: drop a redundant expression from
 gnttab_release_mappings()
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <80426a2b-81ac-acc1-db93-077934468b1b@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <eac94bb4-d9bd-f416-9c43-a9bb708ed008@xen.org>
Date: Mon, 6 Sep 2021 14:15:41 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <80426a2b-81ac-acc1-db93-077934468b1b@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 26/08/2021 11:11, Jan Beulich wrote:
> This gnttab_host_mapping_get_page_type() invocation sits in the "else"
> path of a conditional controlled by "map->flags & GNTMAP_readonly".
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -3816,9 +3816,7 @@ int gnttab_release_mappings(struct domai
>                   if ( gnttab_release_host_mappings(d) &&
>                        !is_iomem_page(act->mfn) )
>                   {
> -                    if ( gnttab_host_mapping_get_page_type((map->flags &
> -                                                            GNTMAP_readonly),
> -                                                           d, rd) )
> +                    if ( gnttab_host_mapping_get_page_type(false, d, rd) )
>                           put_page_type(pg);
>                       put_page(pg);
>                   }
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:19:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:19:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179883.326301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEXF-0003QL-EZ; Mon, 06 Sep 2021 13:19:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179883.326301; Mon, 06 Sep 2021 13:19:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEXF-0003QE-BY; Mon, 06 Sep 2021 13:19:41 +0000
Received: by outflank-mailman (input) for mailman id 179883;
 Mon, 06 Sep 2021 13:19:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEXE-0003Q4-FJ; Mon, 06 Sep 2021 13:19:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEXE-0004YB-6x; Mon, 06 Sep 2021 13:19:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEXD-0008MQ-OU; Mon, 06 Sep 2021 13:19:39 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEXD-00012B-O1; Mon, 06 Sep 2021 13:19:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NO45CK5PSRlWsYOEwHen1TOace1XYi38q9DOoPBe/h0=; b=W6Dux8/uaB3VLdu6YhVNOaK3Rk
	fe+ZFN6+Nob+v+cKKMnTsU+T2eh/5QRQWYMtroexZekVR0xATzUNeePuu5QwxWysPsGvAIRtNQySw
	Lhnvj6Ws9pzTtaYcLa6f1o9RecIjmcNqSTldkzpuzxgy8bCC/JgcMkvNUNOzEAlMWCLQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164850-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164850: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=27151f177827d478508e756c7657273261aaf8a9
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 13:19:39 +0000

flight 164850 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164850/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 14 guest-start             fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                27151f177827d478508e756c7657273261aaf8a9
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  401 days
Failing since        152366  2020-08-01 20:49:34 Z  400 days  696 attempts
Testing same since   164850  2021-09-06 02:37:34 Z    0 days    1 attempts

------------------------------------------------------------
7219 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2221077 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:23:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:23:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179890.326316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEam-00051B-1r; Mon, 06 Sep 2021 13:23:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179890.326316; Mon, 06 Sep 2021 13:23:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEal-000514-Uj; Mon, 06 Sep 2021 13:23:19 +0000
Received: by outflank-mailman (input) for mailman id 179890;
 Mon, 06 Sep 2021 13:23:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEak-00050y-AV
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:23:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a39a1414-ead7-490e-bae0-26b0da986191;
 Mon, 06 Sep 2021 13:23:17 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2055.outbound.protection.outlook.com [104.47.2.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-htmKaAk_NEqJWFquy_URbQ-1;
 Mon, 06 Sep 2021 15:23:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4848.eurprd04.prod.outlook.com (2603:10a6:803:55::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 13:23:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:23:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0086.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Mon, 6 Sep 2021 13:23:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a39a1414-ead7-490e-bae0-26b0da986191
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630934596;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xCq+T/w9za/+vXVmsi/kL7xE4T9zITd+xcOCO9ublJ0=;
	b=cHkJQa4uDFteIpHTdew1TpIQZKV05vHd2p51n7UaXdViG+/er6VnXPuRdwPTJc/4fia8Tw
	WCEOKCpEi8nE9o03uOze4xySd/Mmcmy6/q3v1DA4ausJClKBqY2nxzpuDpNlqS/e/IvWsB
	fqQarpkQ7RSWWKy51lKMz+3kvcBllHw=
X-MC-Unique: htmKaAk_NEqJWFquy_URbQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HhMlIlDF6bEYB3ozbM+KscYSqqEhkfQN/rd5rXwgBDIpfqg3venWAn+QQsoyonh+drME+6VsMgeLDMmM8s02/6WJs5aeRdhIb/96HbLQg8fnOZmqzoFFwZkGvNH/+JOG54pwmyup4i8hrLYEZlcZ0iisqaMp1B+/iI/SZv3/QesBpuOTcLZMZMLoo6RszDizzfVV4XBnhPfRgIH+lMIjC5aKX5rjd2sAgJd/gCBrPS7ZnkRhbnKyhUn9ZJaSXpIjhVDUt97Tq2cBnIEAtflIdr1lxigIndooVRRpI2816/LyDqgWNTkSglmk9gmarOlrKO9jnVpCHXAdEtPwxfN+6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xCq+T/w9za/+vXVmsi/kL7xE4T9zITd+xcOCO9ublJ0=;
 b=JnVGxIU75PhBdciQs+8Efy4TB0uw/Xu+FGQMdZiFVUfdKuWJQ8XCLsNfY3KsuIIKmzkVK2DsBmi/mMZYNFk+Lrb/vjroxCWwmGIYi8wl6C+lfL/YMlreKZzl4pU6zZrrLoatI67UDuTTkUZbuko1/jypEReD0/c4nhOBvXJEgemQjujJ/D+wrfJFO9/6UTy6sEmHNzcXQnKM9xQ72ZmtH+w2rGFvhFE4b9BMbo2OBdLXZ+XX0aSKJKUhWwo223kUO8zBQdlIY0QWVXHEXpqmlDCMcn0pnVJchKNExspfiSgRppyZqpKd/626laliH1ctMby5pKZZhUYXoUBivUhRBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-3-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <371a403c-adec-f1e1-8887-5664a749b169@suse.com>
Date: Mon, 6 Sep 2021 15:23:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-3-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0086.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cfe7cddd-96ee-4935-f6a0-08d971397a59
X-MS-TrafficTypeDiagnostic: VI1PR04MB4848:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4848A708E281DC730EA66B21B3D29@VI1PR04MB4848.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4b0nSyWvCjOOkRuuyRE3JhF+USvLjsUZOyiv00TqLcdQ8diaHwqQ+wisziUIKwLb2g+wQGVn5szYrmbciszzjW9C3rUdHsUi/M3ISdTvQIwUmTtK+QesJge9r9IShJTaIVm/fJCgnzFGB9I1+2NY51+Fneu9UqfHxO1s65Ds4FlvEQAUgp1lWyNIeAL4jZCKYBMo/jgQzgUyNZRT7gCttgS4svpiVS+MTZ5SddZCcuu/M5Q6Wg2F+2uLDggHLrW5L0bdJJALk3XvABPM6wdmwNSJ7xqTJ1JBKBYo6N41nupbTuNx//RsxDLhsJrt4vRNarXq56ggIYnNjRSQzEXiBqOUbZzSIfVr+QluohzreIe3RDpKKw2jbzgrC5FiML34UfnoCeLcnrdOr+5yXvkF0noqxeOikkDem1IfIcxoWCl3LnOrxn8OTuSP/y9Ll8CoYUCS9EzkYy0EXRPq3JC4zc912mTG3Er1nX6Pcj83sYs2AhVVxK/VDcAuyDTuuT2hg/X0TLDcPVsbBcen/JRXprPG5sITyFJddMVOF0xIVxJ1AJJShOnUN6vNXjmy2K0e1h6FiHLD25bngcS94pbX4rwpC9IhzWOsBWHq8WyWoxZwRq5AnePZEnje+t+PrcNGKwisBX1/5h4IokXikaKWj0DWkyvbfq8YgNeik8gQS61p9cxMbqSqrT8M229oCifBz5DYgLQMSKZvueBzgcTz7Wh1SLa8wlqLC9QpnJ8cBu4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(376002)(346002)(136003)(396003)(2906002)(53546011)(956004)(5660300002)(8676002)(2616005)(7416002)(8936002)(186003)(38100700002)(478600001)(6486002)(4326008)(36756003)(31686004)(86362001)(6916009)(66556008)(26005)(83380400001)(16576012)(66476007)(31696002)(66946007)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K0ZlcjB5WTR4enlGdlhzNkYxRnREN1czYURoMFFFZHdqYi9MVlFyZXJTTEw2?=
 =?utf-8?B?bzdEaGp4b3VZU0NZRTZOaEw2aUZHalBqeDIxK2JjNXdrUjRaT2U1aFhnQTdn?=
 =?utf-8?B?R01vMWhMVno0M0NkckVhZFd5ckd2S2RLQzJkL1BTcXBrdjFGRlFZY3ZSaFFM?=
 =?utf-8?B?L3dzay9sdGRSV0lISG96Ym00NVE4RUJQOFRFWW55cFEyeHF3VlZnK25tNlRa?=
 =?utf-8?B?RGhFMjhJS1NmRENtM1BraWpWWGxkeUdrNzA5VW9mbU9xNW16WEVsZG9zMWZB?=
 =?utf-8?B?aGRXM0huRXlZMGt0WlJoaTM0Wkc0TlhZa3loM3hHVmswOWNEK1NxUVQxWG9o?=
 =?utf-8?B?RFJCTTdUMmF2YWMySXk3MmJFYkxuOTlvdEswT1ZEaHhpVjNlOFdUQk5xNmNp?=
 =?utf-8?B?amdIbXBZMGJybVBtNkZ5Tk1BWEVrUWtzVjB6QWJRRjg3NFE3Q1dJWW90YS9B?=
 =?utf-8?B?Z2wrUmsyNjNrMHl1S1Eyb2VoZnpWV0VjbHZCMDdQVDJGeEZzRlRyYTRVUkto?=
 =?utf-8?B?SnRDdnhQK0tLSldhalV4dGEydlZWaEs3bXpLZDB5WVdoV2Q2bUc2OUZKU2FZ?=
 =?utf-8?B?L0tzcWxuZ2xHejNiVm9kRDJQWGNIajJwTWVBZ0xyNjU5SVIyV0p5VkR3MnM2?=
 =?utf-8?B?cUVsVmxCQjAzdXBJTWVCM3lMaUhhUjFnL05mYWpveGhpaWZqZjRvUm9tell2?=
 =?utf-8?B?MitUdk1WN2J2ZEZ1ak9pN2NMdjkxTEZkcGFlMUpTK3liWFA0d3hpUGNvVGFs?=
 =?utf-8?B?OUViaFcrb29pRmRDR3lCTTMrb2FJY2lBTTZmc2VMZisyWWxtUU5yOGJnTWdI?=
 =?utf-8?B?UkxTNCszenppQUNwV1FEK09HTnk5UnQ3U211WDVHeS9kekpyclpMLzRJd2ha?=
 =?utf-8?B?Q0pLS3kveWQ2R290VmhEeXBFV3lXMENKdDFkanhaNS85V2hENklzYUw2Tkgx?=
 =?utf-8?B?UXhoUWgzN090MUNEV2FxclJnNVU0TkpaRHczdzB0REZLKzZwOE0rSEhzK3Zi?=
 =?utf-8?B?OERCUjRzMHVuYmVyTTFVNHlTVTF6R0FHQVE4RTlpZWdDNWVBMjdtU1lrSWpv?=
 =?utf-8?B?VER1dDZEUEdKYVVDUXArb3ZscDdHYk4xMjJvWkJKRDEwR0V0V2ZueUFlYzR4?=
 =?utf-8?B?aG5EaTIzMmE0U0h2ZXVDODZsTkJpVEJhMm0xTzBsZTdQYkRpeDFnR1J5cm5N?=
 =?utf-8?B?Rm50R1o2bFk1MDFlMlFhbXhzQTBwRUVmRWRBdjUvN3JaQTZJb1l3MjlrUEJE?=
 =?utf-8?B?MjE5M2ZoeTliOXIzbDdwUG9uc1lXOFZ5SFJJaGJHdlcraHdnVGw1dXNpenI5?=
 =?utf-8?B?ZUZYeDc2Z010cjJlcVpyUzJHaG8zM201R3VjNHNNRHlWRDd0ekJ2bzA0SVRF?=
 =?utf-8?B?a3R2VnVvcmNLNysrQzV6TFRLNGMyUjNrM3BMSXlqeVB0S3ZJbW9YQ1ZES1lh?=
 =?utf-8?B?Y2ZNcFAyQWpla2Evc3N1K085ZGRmdmoza0lyYVlrcVBLWG9CSjhDUkc3Q2Y3?=
 =?utf-8?B?QTBjeTF4bDYrMTdTZFBudDlVT0RpNjN6R3FCZTVHenZPS1E1Rnl1aUw2NER1?=
 =?utf-8?B?a1Z0RHpYV3Rtb0NTQkhIWm9LS1p4R1ZWWHhlNFBpNnEwUEw3bWQydUFCc0Jw?=
 =?utf-8?B?RnI0Ty9DZWkxa3p5YVdyNGtQbi9MdVVRbXRoVFhyMjlZR3J6RkZKbGt5L21j?=
 =?utf-8?B?eWZEd0taejF6RG41aEhNb3picURIRm4vQkt3NFJzWVRpanBsSUVyam50QmYv?=
 =?utf-8?Q?AMiuc6YiWdgNPZOf+lt1dhKN79FclEYDKc2gV1k?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cfe7cddd-96ee-4935-f6a0-08d971397a59
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:23:13.0436
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: O/zupNZRPlBvBXbEnA36NKl/5hZztPczuA0cIRc9Bi6daTlmcqlABsSDQT1HxIu8kO/bDQXbxV6baQtzpCLy4Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4848

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -864,6 +864,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
>      if ( ret )
>          goto out;
>  
> +    ret = vpci_deassign_device(d, pdev);
> +    if ( ret )
> +        goto out;
> +
>      if ( pdev->domain == hardware_domain  )
>          pdev->quarantine = false;
>  
> @@ -1425,6 +1429,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>          rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
>      }
>  
> +    if ( rc )
> +        goto done;
> +
> +    rc = vpci_assign_device(d, pdev);
> +
>   done:
>      if ( rc )
>          printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",

I have to admit that I'm worried by the further lack of unwinding in
case of error: We're not really good at this, I agree, but it would
be quite nice if the problem didn't get worse. At the very least if
the device was de-assigned from Dom0 and assignment to a DomU failed,
imo you will want to restore Dom0's settings.

Also in the latter case don't you need to additionally call
vpci_deassign_device() for the prior owner of the device?

> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -86,6 +86,27 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>  
>      return rc;
>  }
> +
> +/* Notify vPCI that device is assigned to guest. */
> +int vpci_assign_device(struct domain *d, struct pci_dev *dev)
> +{
> +    /* It only makes sense to assign for hwdom or guest domain. */
> +    if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) )

Please don't open-code is_system_domain(). I also think you want to
flip the two sides of the ||, to avoid evaluating whatever has_vcpi()
expands to for system domains. (Both again below.)

> --- a/xen/include/xen/vpci.h
> +++ b/xen/include/xen/vpci.h
> @@ -26,6 +26,12 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
>  /* Add vPCI handlers to device. */
>  int __must_check vpci_add_handlers(struct pci_dev *dev);
>  
> +/* Notify vPCI that device is assigned to guest. */
> +int __must_check vpci_assign_device(struct domain *d, struct pci_dev *dev);
> +
> +/* Notify vPCI that device is de-assigned from guest. */
> +int __must_check vpci_deassign_device(struct domain *d, struct pci_dev *dev);

Is the expectation that "dev" may get altered? If not, it may want to
become pointer-to-const. (For "d" there might be the need to acquire
locks, so I guess it might not be a god idea to constify that one.)

I also think that one comment ought to be enough for the two functions.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:28:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:28:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179898.326327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEg1-0005l1-QY; Mon, 06 Sep 2021 13:28:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179898.326327; Mon, 06 Sep 2021 13:28:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEg1-0005ku-NW; Mon, 06 Sep 2021 13:28:45 +0000
Received: by outflank-mailman (input) for mailman id 179898;
 Mon, 06 Sep 2021 13:28:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEg0-0005kk-9S; Mon, 06 Sep 2021 13:28:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEfz-0004hY-HC; Mon, 06 Sep 2021 13:28:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEfz-0000oV-6u; Mon, 06 Sep 2021 13:28:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNEfz-0006Vx-6S; Mon, 06 Sep 2021 13:28:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cVjaJDiCiMn+BDMSqpaPNBryK8K9OTNeYNSswEf6PBA=; b=gW5sQrHGjrxzGn9cryMQZNNVS5
	SsvrzqDWhelu60nf1j5et+ODhe9+eVwwN7SuvekqKOEW69C9WpVZ6mIfjQBoEY97JEdwa/9ANkHCk
	k5twsOZdPUlMZfDcG8jRKAFcMSAWRLCpzg1GAVwTGshPppOYCBIqYBCOJl7lExEBTq+0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164859-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164859: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 13:28:43 +0000

flight 164859 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164859/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   67 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   41 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:29:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:29:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179905.326341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEge-0006JE-4i; Mon, 06 Sep 2021 13:29:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179905.326341; Mon, 06 Sep 2021 13:29:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEge-0006J7-1M; Mon, 06 Sep 2021 13:29:24 +0000
Received: by outflank-mailman (input) for mailman id 179905;
 Mon, 06 Sep 2021 13:29:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNEgc-0006Ix-7m
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:29:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cecdbee3-918a-4524-9703-f2dc5c642011;
 Mon, 06 Sep 2021 13:29:21 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2113.outbound.protection.outlook.com [104.47.18.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-4-YjVHstorM1SLv8-ofyYDbw-1; Mon, 06 Sep 2021 15:29:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 13:29:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:29:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P250CA0029.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 13:29:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cecdbee3-918a-4524-9703-f2dc5c642011
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630934960;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=G3OCOX0BR8N3bNEaVRo41nPsf4k/oWlMETypATXxQfg=;
	b=ddVWkeYSNY6jZwDbzgb4goYuXhZThyaQGuXVxaeBmzR/dSHC/vutwANf6oQqzddMYbpDWP
	NqB+eouYkFC7MUdDzhzAntgIlEhcHa+MobSo1b48ZoDwLjjc0cvogGBp/8CWnzAM55vCFD
	AqoHpAYKabobOMxJLKcOlINxTspS+Bk=
X-MC-Unique: YjVHstorM1SLv8-ofyYDbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TrwQPq9DNZIcxM2j9SlRty6gLcEdn6aTmreLx2n/ifvO3fFzkC2UGQrnAAnkiagTFMmsDBfjFSwKvWI3kV/GdE+92lokw2uFPoUQCRyikrKQNcxx2ewu8ziEpucyVR2E/W8iE4IZitLcyJfCF8sJX+P1jaHadcrP1nsc8aZMLnsueNdh3ui0EX5sHhwvHFpT85nskDWClXeJ03GPchkYXyizZijAxKzyuojY2ZHqLz5O5YuJxx6fOxOXuNoNsIGnv+Ndn8JoQmYh9hM2kKhvOhXV5pLi49uDHJ8u8cjytuiJLQvIJ+3bZaj/o45MIqQA6MSnequgFhUXQxKgTYsEDA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=G3OCOX0BR8N3bNEaVRo41nPsf4k/oWlMETypATXxQfg=;
 b=diUTTM9UGY6I+wylvSBInV5iPwdi5k/au3IuaEB4r3sT5mzx39mzpF4WR3uzWZkbniKXe0vFCKetzJWuKPWvjLNJUUYRU9Eb3yh5seFekASTg8pFU1jcmLbGfyRsAiqDzJbKaUn4ub8XW5p5EIkHJvAwghx6acHU18Nh4eA63+gMwCrEB2nm3EVl6NigVIhGIlT5/AbsDFr3jbe5P87IQXf+UmlR/gHLE0IXNbjiWGVZ0WMKWXYigIkRC1i3xuAQqq4NpVwiXDefF0nrErO/7wh+fZiw6bFwIzQO4s5dQG48cfIUynFk1Kx6jdomkRr0R7IVo3/RzBfwVafOkJ9SSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
 <7b8fba1a-767e-87f8-d0eb-5af5e5427bb4@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <60acd486-1641-6db4-4451-247edcc11522@suse.com>
Date: Mon, 6 Sep 2021 15:29:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <7b8fba1a-767e-87f8-d0eb-5af5e5427bb4@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P250CA0029.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:102:57::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 38cbc938-7c44-4dfa-f8d4-08d9713a53b0
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6479766F1F06E92615A57BA9B3D29@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yQm0psxsr09FTyI8IrwMoJ1zG7VcnMBxQKPALDMawJ3qlVTZZ+wmhiy3XQXk8ruKpc2YpqzwG1j9pTa7zdS5aj5/AM0vtPGkL7Nz5JZkxoL2jBSE92sYI99kx2duyM37j8XRone7OfVxofrz90y/m5vNoTOP6gY1iqD9b2riYpxuHUIuA9SPPgO5OAVYBZG3qWCbk3GfxmhpMciKy5xT5+ZTm7zhSyoW9myL0g/KpNmwQnhvB1ktB+2SMAqcuuhBWc1bme540MVQbwAxVnkY4dBZSzpQKWwb6OhHD4pJK8zdatG8+E7BOc85USAxxBWgei8AAgtWbUHeQXyLwYpVBVv60VVs4IOSzZPFK8/Vpn7817AkdcedTK5zU1a2vhGE+qy67ViS1D60efgNLtxuC/664wI0wgL5GgeDGRxERbcJPYJDIApjiYXP4429bJUBNSfjyK+GnmjbArBTsmiY4ls4tNdUQzf/1zXjqxBxOuRWeNbah9gQyuEGuuzfgqWpFDxFVJ497bX/VcNo9/dKi+Hfaoc4DTCa8RLkdZlLKWauiyEZPC0Y+Gmv5W//ufPXKZLRLh0TIF3YMClZIBeMLpVHm/Q2DlP5Rf/fV7n70yMu7jZklfcOB4xU2mdTUeIHdKigqUn3gQgoat3X8gsop9oQ7CVthzMH/zbA+zepuTCzWpAaZMczovkxeFPpj1pDqOinpfN6m9uH9PtMlBm6bmjrC+VNS4IOCkKPvjGSzT8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(396003)(136003)(39860400002)(376002)(316002)(54906003)(8676002)(38100700002)(66946007)(16576012)(6666004)(2906002)(5660300002)(36756003)(478600001)(956004)(6916009)(4326008)(66476007)(8936002)(31686004)(83380400001)(66556008)(2616005)(31696002)(53546011)(86362001)(186003)(26005)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Y3c5WUREV1orVnRNUjhyTEQxYkZpakR2bHlDOGpNTyswd05BMW5DSmdZRlA2?=
 =?utf-8?B?QmhaMEx4Q2I5K1BJS1VtcWV1dERDUHVPMG14T1BBY3IwWUVaYUNmWEFhSXZk?=
 =?utf-8?B?b000ZnFFbEZwZVEyOTlPVDkwMHNDZHRNcmRvdEhOUyt0Y3NXaEIwelBldUVv?=
 =?utf-8?B?MlFzcmx2MGFrQXdHTmxSWVNFRElzdUZJWHAzazBnWGY2cEhLa1BjQnFUVDB1?=
 =?utf-8?B?TEIvNWYrMllqczViYzBJc3U1djhqdTVFWklOSDJzSWhDRGVNaHFhcEZMdy96?=
 =?utf-8?B?SFcvL1o5bXJ0S2lFWmpRSGxlcWtWU2I2RkVRZEtUeURkVWc0SmtJR0NJUnor?=
 =?utf-8?B?TmRmL2IyUTZJK1hwd0VlR3N4Vzd1VnhUS1huY1dCZmMya1J4VTIyRG5EeEhU?=
 =?utf-8?B?SjhVcGptSXdScWRBYzh4aktvNDRhY0grbHZDdVZ5aTVMcU5GYWhnN2I3Q1Mz?=
 =?utf-8?B?YTlTLzc4VmZZMjZWR0FTOGhWMlB1emlEemtHeXcvcW5WdU9TdUkzS0xSa1pm?=
 =?utf-8?B?NmVSNmg3NFpvdUc1cW5jYnNIRGFtekJKWXdDeHpYYnpMRHVxaUpxVjQrTXUr?=
 =?utf-8?B?L2hFTmtnT0x5YVBJd0xYMWVhWnB2djI4N1MrOTdtbkl4bS9FcXNtYXJrc1My?=
 =?utf-8?B?YlhSallvTEN4OTRWa01QV2tJWWtlaGZvUGx1ZjdhUE1qSmJxRGFxSW93N1ZF?=
 =?utf-8?B?MEFCSm9SeUR3bWN0VlFhM3QxZmgvc2NkMGU3a0VJZkdEN1oxWFBCc2IzTFZt?=
 =?utf-8?B?N29aajB6cTVIY1NvZWVGa2xqYTAzM1Z1cE13M251d2FMc2tIUGV4Ny9Vdkl5?=
 =?utf-8?B?TFk3cmlHZ01ha2p3NjE0SVNicVk1Q2I3c0hnamdMZTU3cnluK09qV29rbHhT?=
 =?utf-8?B?K1RkNUNFSllqVGhCeHdQVTRTaUlWb3hqRyt3QXVZM0NVN3lKQnJpMlBUQjZR?=
 =?utf-8?B?RkhlMEtCSjBqSVRBOGp0dlBoeVZHcHdyYis2UWdWd1RiTFcxNkdrNEl1NVhi?=
 =?utf-8?B?YXpaQ0thRVVjRktNOEYyM2VrbEk2NUFycHNtVVQ4MmdJeHVqQnpPOHk4SndF?=
 =?utf-8?B?SHJIOS8rTlFwaVlZU29aeXBlYktTbzVsYlhpWTlhVTl5UXBVMlpnanlpbkdo?=
 =?utf-8?B?QVh3TEswYnRoQ2VVZktBUmx5VVNvdmlGaHdnUEhpMExKUGpTN2Mxc1RXQzNl?=
 =?utf-8?B?VWQ5ckU2MFU4QlhmTTRES2VBMVZtMDBXSmxWaXU0amlLd3hUck1PMVRnRStC?=
 =?utf-8?B?VGtJVVVBZUpjakZXZGJpbFdKWEVjSFhkTW5mYk44YVVwSEtzMjlIUVIzRGR3?=
 =?utf-8?B?MTZvN2ZLSEFKYk5DV1VHcjZ0VEhaOG1nT2dEYzY2V2d0aDBHV2ZtR1Fzb1cy?=
 =?utf-8?B?WEI3RmdGeHN2dldIaU9zNThlb2ZXcmZadjZpb2g2bnF2RDE1WUw4UmJiSXk2?=
 =?utf-8?B?dlRkMW0rOTlYb1ZMMDErdnlmM2c5TFRHUG9oSGJUNENJcnZwOHBETk90aHN0?=
 =?utf-8?B?WDVGUGFsaUFNOTVMWjBEOGVhMkR6VG9wanFiL3d5SHlUUWJTd2IzckpiNDcy?=
 =?utf-8?B?RkRXeUV6NDFJS2V2WU9mMXdIYVRieC8yeEhEVXNkZUR1NGJ3QmQ2ZGZWTHEr?=
 =?utf-8?B?T1IvWTY5SUh2dHRNTFIyS1dJR2dTb2IveGZYNzVVMXo3NjZ4VHI1UElsYncr?=
 =?utf-8?B?RWI1eDFuUWE1d29Rd2dIR0NsTzRmdnhWcDQrdGZYU3NRUDZocWtHNHBCSmpS?=
 =?utf-8?Q?Pg0a2RA2RZ2QOHbYsdFGUTee+1GM/cewq8prwOP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 38cbc938-7c44-4dfa-f8d4-08d9713a53b0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:29:17.7043
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HTl/eKVymEaVKobkZkht67y6gbWDqjWy/3aJ+CzL8pIhbox1kixJML9x/KUfQ4tnNx5w1G90TXzyXYeQsFwlnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

On 06.09.2021 15:13, Julien Grall wrote:
> On 26/08/2021 11:09, Jan Beulich wrote:
>> --- a/xen/common/grant_table.c
>> +++ b/xen/common/grant_table.c
>> @@ -633,6 +633,34 @@ get_maptrack_handle(
>>       if ( likely(handle != INVALID_MAPTRACK_HANDLE) )
>>           return handle;
>>   
>> +    if ( unlikely(!read_atomic(&lgt->maptrack)) )
>> +    {
>> +        struct grant_mapping **maptrack = NULL;
>> +
>> +        if ( lgt->max_maptrack_frames )
>> +            maptrack = vzalloc(lgt->max_maptrack_frames * sizeof(*maptrack));
> 
> While I understand that allocating with a lock is bad idea, I don't like 
> the fact that multiple vCPUs racing each other would result to 
> temporarily allocate more memory.
> 
> If moving the call within the lock is not a solution, would using a loop 
> with a cmpxchg() a solution to block the other vCPU?

As with any such loop the question then is for how long to retry. No matter
what (static) loop bound you choose, if you exceed it you would return an
error to the caller for no reason.

As to the value to store by cmpxchg() - were you thinking of some "alloc in
progress" marker? You obviously can't store the result of the allocation
before actually doing the allocation, yet it is unnecessary allocations
that you want to avoid (i.e. to achieve your goal the allocation would need
to come after the cmpxchg()). A marker would further complicate the other
code here, even if (maybe) just slightly ...

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:33:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:33:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179912.326351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEkc-0007yL-PA; Mon, 06 Sep 2021 13:33:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179912.326351; Mon, 06 Sep 2021 13:33:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEkc-0007yE-MN; Mon, 06 Sep 2021 13:33:30 +0000
Received: by outflank-mailman (input) for mailman id 179912;
 Mon, 06 Sep 2021 13:33:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNEkc-0007y8-1C
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:33:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEka-0004oI-Is; Mon, 06 Sep 2021 13:33:28 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEka-00057U-CS; Mon, 06 Sep 2021 13:33:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=APik1CbMbgF6wJTSvB432A/CGZAYI0LBfC+hISwtP1Y=; b=63RDQMgBCf/ln7NWRC2xpZ64S2
	KlkhERBFLjEhXiRn5exhoxEEX8ugn8IPls5vlAQxiNgdmUzMQi+aAwsFy8vcvksAtu+6QOpPgbzdC
	31yt9Kr4ens/sAyQGPVSjIsZdaRtCyg+veQ5X6JdM68OI9loOmzCh3PCGTkOJfcr7chY=;
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
 <7b8fba1a-767e-87f8-d0eb-5af5e5427bb4@xen.org>
 <60acd486-1641-6db4-4451-247edcc11522@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <26669864-09e2-a026-67f6-1f6dc48267db@xen.org>
Date: Mon, 6 Sep 2021 14:33:26 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <60acd486-1641-6db4-4451-247edcc11522@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 06/09/2021 14:29, Jan Beulich wrote:
> On 06.09.2021 15:13, Julien Grall wrote:
>> On 26/08/2021 11:09, Jan Beulich wrote:
>>> --- a/xen/common/grant_table.c
>>> +++ b/xen/common/grant_table.c
>>> @@ -633,6 +633,34 @@ get_maptrack_handle(
>>>        if ( likely(handle != INVALID_MAPTRACK_HANDLE) )
>>>            return handle;
>>>    
>>> +    if ( unlikely(!read_atomic(&lgt->maptrack)) )
>>> +    {
>>> +        struct grant_mapping **maptrack = NULL;
>>> +
>>> +        if ( lgt->max_maptrack_frames )
>>> +            maptrack = vzalloc(lgt->max_maptrack_frames * sizeof(*maptrack));
>>
>> While I understand that allocating with a lock is bad idea, I don't like
>> the fact that multiple vCPUs racing each other would result to
>> temporarily allocate more memory.
>>
>> If moving the call within the lock is not a solution, would using a loop
>> with a cmpxchg() a solution to block the other vCPU?
> 
> As with any such loop the question then is for how long to retry. No matter
> what (static) loop bound you choose, if you exceed it you would return an
> error to the caller for no reason.

I was thinking to have an unbound loop. This would be no better no worth 
than the current implementation because of the existing lock.

> 
> As to the value to store by cmpxchg() - were you thinking of some "alloc in
> progress" marker?

Yes.

> You obviously can't store the result of the allocation
> before actually doing the allocation, yet it is unnecessary allocations
> that you want to avoid (i.e. to achieve your goal the allocation would need
> to come after the cmpxchg()). A marker would further complicate the other
> code here, even if (maybe) just slightly ...

Right, the code will be a bit more complicated (although it would not be 
that bad if moved in a separate function...) but I feel it is better 
than the multiple vzalloc().

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:36:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:36:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179918.326362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEn3-000097-6Q; Mon, 06 Sep 2021 13:36:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179918.326362; Mon, 06 Sep 2021 13:36:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEn3-000090-3a; Mon, 06 Sep 2021 13:36:01 +0000
Received: by outflank-mailman (input) for mailman id 179918;
 Mon, 06 Sep 2021 13:35:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNEn1-00008u-Il
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:35:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEmz-0004ro-ES; Mon, 06 Sep 2021 13:35:57 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEmz-0005KV-8I; Mon, 06 Sep 2021 13:35:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=DuwvLpcFkRjxKrXDfwsHp4QlfklE89rlmF3HveJUEkA=; b=KxnoRDPGQjSREn2uBbVJS9rOn3
	dzXKq1ngXnj3mM1rJbT1VSiwt0XruhakkC770maae9aWNXJW33RkOC/a+JH4DNZN7J8QY9BzSvRnd
	F8CJnvzqYUGjFU2TVhetf8jyRLJpofcHeJkCnIs2LiiGGoQWLsYnIBY7qzQ//ubpSjK8=;
Subject: Re: [PATCH 3/9] gnttab: fold recurring is_iomem_page()
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <06eeea76-adf8-4f40-eced-57db1526eba8@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <cb4f6360-4d80-be4a-e631-553b00b06a29@xen.org>
Date: Mon, 6 Sep 2021 14:35:55 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <06eeea76-adf8-4f40-eced-57db1526eba8@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 26/08/2021 11:12, Jan Beulich wrote:
> In all cases call the function just once instead of up to four times, at

extra NIT: It is more like two because there is a else in 
gnttab_release_mappings() :)

> the same time avoiding to store a dangling pointer in a local variable.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c

[...]

Everything below looks a duplicate. Might be an issue in your tools?

> In all cases call the function just once instead of up to four times, at
> the same time avoiding to store a dangling pointer in a local variable.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -1587,11 +1587,11 @@ unmap_common_complete(struct gnttab_unma
>       else
>           status = &status_entry(rgt, op->ref);
>   
> -    pg = mfn_to_page(op->mfn);
> +    pg = !is_iomem_page(act->mfn) ? mfn_to_page(op->mfn) : NULL;
>   
>       if ( op->done & GNTMAP_device_map )
>       {
> -        if ( !is_iomem_page(act->mfn) )
> +        if ( pg )
>           {
>               if ( op->done & GNTMAP_readonly )
>                   put_page(pg);
> @@ -1608,7 +1608,7 @@ unmap_common_complete(struct gnttab_unma
>   
>       if ( op->done & GNTMAP_host_map )
>       {
> -        if ( !is_iomem_page(op->mfn) )
> +        if ( pg )
>           {
>               if ( gnttab_host_mapping_get_page_type(op->done & GNTMAP_readonly,
>                                                      ld, rd) )
> @@ -3778,7 +3778,7 @@ int gnttab_release_mappings(struct domai
>           else
>               status = &status_entry(rgt, ref);
>   
> -        pg = mfn_to_page(act->mfn);
> +        pg = !is_iomem_page(act->mfn) ? mfn_to_page(act->mfn) : NULL;
>   
>           if ( map->flags & GNTMAP_readonly )
>           {
> @@ -3786,7 +3786,7 @@ int gnttab_release_mappings(struct domai
>               {
>                   BUG_ON(!(act->pin & GNTPIN_devr_mask));
>                   act->pin -= GNTPIN_devr_inc;
> -                if ( !is_iomem_page(act->mfn) )
> +                if ( pg )
>                       put_page(pg);
>               }
>   
> @@ -3794,8 +3794,7 @@ int gnttab_release_mappings(struct domai
>               {
>                   BUG_ON(!(act->pin & GNTPIN_hstr_mask));
>                   act->pin -= GNTPIN_hstr_inc;
> -                if ( gnttab_release_host_mappings(d) &&
> -                     !is_iomem_page(act->mfn) )
> +                if ( pg && gnttab_release_host_mappings(d) )
>                       put_page(pg);
>               }
>           }
> @@ -3805,7 +3804,7 @@ int gnttab_release_mappings(struct domai
>               {
>                   BUG_ON(!(act->pin & GNTPIN_devw_mask));
>                   act->pin -= GNTPIN_devw_inc;
> -                if ( !is_iomem_page(act->mfn) )
> +                if ( pg )
>                       put_page_and_type(pg);
>               }
>   
> @@ -3813,8 +3812,7 @@ int gnttab_release_mappings(struct domai
>               {
>                   BUG_ON(!(act->pin & GNTPIN_hstw_mask));
>                   act->pin -= GNTPIN_hstw_inc;
> -                if ( gnttab_release_host_mappings(d) &&
> -                     !is_iomem_page(act->mfn) )
> +                if ( pg && gnttab_release_host_mappings(d) )
>                   {
>                       if ( gnttab_host_mapping_get_page_type(false, d, rd) )
>                           put_page_type(pg);
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:41:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:41:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179925.326374 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEsY-0001ms-Sh; Mon, 06 Sep 2021 13:41:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179925.326374; Mon, 06 Sep 2021 13:41:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEsY-0001ml-Od; Mon, 06 Sep 2021 13:41:42 +0000
Received: by outflank-mailman (input) for mailman id 179925;
 Mon, 06 Sep 2021 13:41:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNEsX-0001mf-80
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:41:41 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEsV-0004xi-Mb; Mon, 06 Sep 2021 13:41:39 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNEsV-0005qx-G4; Mon, 06 Sep 2021 13:41:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Boy2zv/vfXIzRDN1u+uY3cgR2WQrfnHGiF9oilq+fg0=; b=ScdiiQJHlMAIB21htUpL6FgnEf
	sawNDL3sx2Bu/GhC2DPZU1ZKMGJN+DZeWa+sii/UIor4GnZ/Tw7MomWZUWHAtAJJEuis6TMPU+NJf
	QeUJ8iRTZM1Fk+IEbezyJzkX57uz3xRjrxBkmVW3l172VIILymcV3AlDc5OrCI+aThVc=;
Subject: Re: [PATCH 6/9] gnttab: check handle early in
 gnttab_get_status_frames()
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <049c4f02-2af6-542e-cc51-5c245f33341c@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <194da6a8-5818-fb5f-d3f7-3b374fd3c17f@xen.org>
Date: Mon, 6 Sep 2021 14:41:37 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <049c4f02-2af6-542e-cc51-5c245f33341c@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Jan,

On 26/08/2021 11:13, Jan Beulich wrote:
> Like done in gnttab_setup_table(), check the handle once early in the
> function and use the lighter-weight (for PV) copying function in the
> loop.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -3261,6 +3261,9 @@ gnttab_get_status_frames(XEN_GUEST_HANDL
>           return -EFAULT;
>       }
>   
> +    if ( !guest_handle_okay(op.frame_list, op.nr_frames) )
> +        return -EFAULT;
> +
>       d = rcu_lock_domain_by_any_id(op.dom);
>       if ( d == NULL )
>       {
> @@ -3301,7 +3304,7 @@ gnttab_get_status_frames(XEN_GUEST_HANDL
>       for ( i = 0; i < op.nr_frames; i++ )
>       {
>           gmfn = gfn_x(gnttab_status_gfn(d, gt, i));
> -        if ( copy_to_guest_offset(op.frame_list, i, &gmfn, 1) )
> +        if ( __copy_to_guest_offset(op.frame_list, i, &gmfn, 1) )
>               op.status = GNTST_bad_virt_addr;
>       }
>   
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:42:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:42:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179932.326385 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEtM-0002RE-9a; Mon, 06 Sep 2021 13:42:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179932.326385; Mon, 06 Sep 2021 13:42:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNEtM-0002R7-6F; Mon, 06 Sep 2021 13:42:32 +0000
Received: by outflank-mailman (input) for mailman id 179932;
 Mon, 06 Sep 2021 13:42:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/HKT=N4=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNEtL-0002Qv-PG
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:42:31 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f9f0901-a3f0-434a-a066-225fbfd0f79f;
 Mon, 06 Sep 2021 13:42:30 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id x27so13602366lfu.5
 for <xen-devel@lists.xenproject.org>; Mon, 06 Sep 2021 06:42:30 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id j20sm746556lfr.248.2021.09.06.06.42.29
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 06 Sep 2021 06:42:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f9f0901-a3f0-434a-a066-225fbfd0f79f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=1Op4eoAO59qaq+mkjX+eiujVM6Ur2TNXFe7+MnMWPAo=;
        b=h4C3JZ0PaIs5BAjfC12yjm/XbJG+Sg9p+OZEUfx6eyBgaJw16VkFji68h8VfPdxIED
         P6NK7acmk/yKEkPtyzd8dOEYML8uVsibT46B4+PTyLzRROJ9UOz50n3CKRTwqI4BtZID
         jYMegg7iXrsRGKkDxwGNXMUZtijc3jxaTR4SDPtzc6IhfF4/RKhw1B+8L7VqCWlFd72a
         Nw1aOjXB8yydiaSexP00VHw/XHaEC/OsrgKO90QMBaaRZ+NBjerGSvjiOcjwgLYsnxKN
         41vUrRQsPxLno+ecJ+ts8wOK7MH7c+2sUWZ35uvbub1655t2VThcR2w8nj7iJLkWrbvY
         MJSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=1Op4eoAO59qaq+mkjX+eiujVM6Ur2TNXFe7+MnMWPAo=;
        b=dtplSFlS2iLmP9mfCONq2rU9nXDXoRV8en7BPQgB1K1/Zv+kyugvTEITYW2gK0aExV
         p3Q3f1e7EijKTDTVroOH+KieAtV+vXNK2z/fZDc4J7/1cbEgjxS1J5BukQN9XCpFGn5s
         QrdPFq5EyeUXtSiKmVjcf1h4OVXRma9z2KwdmlpslhduhExOoWu76OaRcITbzYx0Kr3J
         8zRxEJyKXxW6+jqMoeUqC5YGEcaaqarCiERgmvpgsewDBm6HGakr+kLUHgdM3nBhk2Mm
         Fm0+tWe0p6XlIIo0vZxhq1s2n8AlimIWcCdZoBk9Vwk+2t/D7NPnX87iHgiSFO4+jaTF
         B/vQ==
X-Gm-Message-State: AOAM531Un2z+GEK1RiKtHeqDwJTLdhla0rIIqSE+iXiZn0EpbsY3p1fN
	HZn+ZUL71tL95iuM/3f99MZfyVyHbBY=
X-Google-Smtp-Source: ABdhPJywLRGtr+1AwJ1jlC5wYfXjRpuRgZv/48wcKg6PKsZfGss8UwtlELHpiu/sosOoKvpQOWBbWQ==
X-Received: by 2002:ac2:5f78:: with SMTP id c24mr2880538lfc.554.1630935749687;
        Mon, 06 Sep 2021 06:42:29 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
Date: Mon,  6 Sep 2021 16:42:21 +0300
Message-Id: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Allocate anonymous domheap pages as there is no strict need to
account them to a particular domain.

Since XSA-383 "xen/arm: Restrict the amount of memory that dom0less
domU and dom0 can allocate" the dom0 cannot allocate memory outside
of the pre-allocated region. This means if we try to allocate
non-anonymous page to be accounted to dom0 we will get an
over-allocation issue when assigning that page to the domain.
The anonymous page, in turn, is not assigned to any domain.

CC: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
---
 xen/arch/arm/tee/optee.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 3453615..83b4994 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -410,7 +410,7 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
     if ( !shm_rpc )
         return ERR_PTR(-ENOMEM);
 
-    shm_rpc->xen_arg_pg = alloc_domheap_page(current->domain, 0);
+    shm_rpc->xen_arg_pg = alloc_domheap_page(NULL, 0);
     if ( !shm_rpc->xen_arg_pg )
     {
         xfree(shm_rpc);
@@ -774,7 +774,7 @@ static int translate_noncontig(struct optee_domain *ctx,
      * - There is a plan to implement preemption in the code below, which
      *   will allow use to increase default MAX_SHM_BUFFER_PG value.
      */
-    xen_pgs = alloc_domheap_pages(current->domain, order, 0);
+    xen_pgs = alloc_domheap_pages(NULL, order, 0);
     if ( !xen_pgs )
         return -ENOMEM;
 
@@ -938,7 +938,7 @@ static bool copy_std_request(struct cpu_user_regs *regs,
 
     BUILD_BUG_ON(OPTEE_MSG_NONCONTIG_PAGE_SIZE > PAGE_SIZE);
 
-    call->xen_arg_pg = alloc_domheap_page(current->domain, 0);
+    call->xen_arg_pg = alloc_domheap_page(NULL, 0);
     if ( !call->xen_arg_pg )
     {
         set_user_reg(regs, 0, OPTEE_SMC_RETURN_ENOMEM);
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:53:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:53:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179946.326400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNF3i-0004Dp-BG; Mon, 06 Sep 2021 13:53:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179946.326400; Mon, 06 Sep 2021 13:53:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNF3i-0004Di-7e; Mon, 06 Sep 2021 13:53:14 +0000
Received: by outflank-mailman (input) for mailman id 179946;
 Mon, 06 Sep 2021 13:53:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNF3g-0004Dc-Sk
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:53:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c5e79a4c-0f19-11ec-b0a6-12813bfff9fa;
 Mon, 06 Sep 2021 13:53:11 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-57hMhSFSMBSVrQVWPxqeDw-1; Mon, 06 Sep 2021 15:53:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2958.eurprd04.prod.outlook.com (2603:10a6:802:a::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Mon, 6 Sep
 2021 13:53:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:53:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0271.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 13:53:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5e79a4c-0f19-11ec-b0a6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630936390;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GhLZGAONCDDxsmgW/I/T1OjSqyvPMPUBLOx7J8nGXuA=;
	b=KvsoEaTyXPNlntYJgtfcZcKi7Hs1dPn+z312ZOKTuciFj83jgo8VRuLvgk3iDrqmrWB2pw
	9THrqlalj5gz0WAOIwWtuWh+HjkJAmHg2U4+AmAh3mzD4nAg1eq7uCEerHnTcSVrr2CCJe
	OytdjxyQk638DRn9KFfbJ+RWUogtFvw=
X-MC-Unique: 57hMhSFSMBSVrQVWPxqeDw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iENllwtlBNZr9CQMofON5J4F1V5EhvxRYfNbCvWF8rQqhY/OmjJpkGKR4iz3eIiIkmmUsP82BZglQW6yftOMvW+yE7bK9H0qts0qlHDJBacet+mprGmGY0GkY2xUH+nAtSFghNAo3XqpG9g8YqmY0c5w0UEwzeKVgCnl5OPYhqYWOMq6plL26mbu9+sVb7YZJ0Fv0tdMO3FyVtlBt8JyrdvqRwGGP54H8QXNiJpPA2gmy/PL9fNqaknI8nLq2i9RDgnLb7a/kB3Ubbl4vQgnUY2PuszS3OL7ax0fenMpOtZzKEpiw/QgQQu+GFnktwlZdtKN0eicCN9SDdSMcIXhCw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GhLZGAONCDDxsmgW/I/T1OjSqyvPMPUBLOx7J8nGXuA=;
 b=BVtoVgWwlfrRM045Nz8GoNGZe6QFWZu7PSgbw+Fq5sFzSzJvSr212bEvmGm13Z/+P3Whhh+e55QBUjCu/6eUxKzyMQP4F5EGiyW+KUR3/SnvZshB0O2KnBwVH36VjV6uO4sbsyhLKXVJE9Q3xMUJ1Ho1QENs3zrXB1YeHnQMmyoGXjX3ytz/2HMcTKXm6XCNLGnhUBLfR54xJrcws1JC/cMQMHlsXIvrozIROk+V5uM8dYgVJs8ELUmuPfMtW5+ItGe60RhFucWWXHXJ4Jh1j4CFbCOyeFMOF0UWiQmyuMzHKyV04a+WcSZWAzicR1kUs8EzsqdHjb2R6vMhVGFWaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 3/9] vpci/header: Move register assignments from init_bars
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-4-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4007d84d-9e76-32df-58f6-af5ff26fdf6e@suse.com>
Date: Mon, 6 Sep 2021 15:53:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-4-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0271.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 91454edf-7940-401b-bdf5-08d9713da8b3
X-MS-TrafficTypeDiagnostic: VI1PR04MB2958:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2958AFFCC2205163BE6DDCD0B3D29@VI1PR04MB2958.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	w6fRXdngvvZMQEcXFxwtsB+i4DD+N/IBg3k/NuHplPd5zzLk7EPBKx3Qxn4sh2K1Pj2p13e0RVe0I4TPKepL6JdkcQOp71SLNLedR5ys1arHR9NFWtkyRqOjjDB5abfNehBOOek+fd25EWny4xvXJgEg6IS8XHjHjSu8TVH0+lGGABZxuXo66pgqHD6oFLKph+lEBt9sDeFfWKgD2VB93oEsvp5MtG+hHISARCT2Bo+Mq1lVEGL0O8duFRazU2vER73jKXi72NTfq7DWpLrJB4gPOFvo5i8udB5inl/C0FUsopDEvQzqblGtEKFqmkd2xeaHsU2uNEw/dyrD76ETtnDedSnntTX2ItYuaWl1nirNcVX/3D9okZCE05Gtd8CGFvXAXA4w07gOPz4CtuJbkPllwMeajXIDM3YpvGcW+sR/CRmPYEJSZ/Jj2HKrtqXTHs9HIw0cGnGoo5fxXCqIB4WygalYr5BXnH6LCIX8Fdw4Hd0uQ5QZN0gGKfTWM4NMiQSMWfP1bpO1zs63aR7LYYvC7LlWVEWusWnhk5xqZlKC7AfCg8L0292+8fLgwYtr8gpWw3AkbXL1J+1luv7KDifQ40BzUIusbVqIk5tajNEW7AF5+kaF7N4aqJjDQV4Ht2IZPHMIYKBIABUOYgUL8orGGxM6bPFVfVazAnMIKL5CqnqESvh8qHg2omvlzlaavtJipZcW3tBOBM4zXJJPuu2oa3xbXHrNNBoLW0UT5sw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(136003)(39860400002)(376002)(346002)(38100700002)(4326008)(26005)(31686004)(53546011)(8936002)(316002)(8676002)(66556008)(956004)(7416002)(186003)(5660300002)(2616005)(66476007)(66946007)(6486002)(110136005)(16576012)(36756003)(86362001)(83380400001)(478600001)(31696002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OTZZMWRqazhCanBaeTVJeEZQY3dwSXphWk1Ca0ZvQ0FHNktsRUgrM1FMQmZh?=
 =?utf-8?B?dmZncGR6VXFBdkVvdW96bzZPZ3BSQjFJckJoQnUreGhlVG10bUR3dFRVb1VP?=
 =?utf-8?B?UUVmRnhxVWo4c2JORVpBTlVBeXhBbnJHOWlIRGp3N1BxWGxzN0g5dnVOQ0J3?=
 =?utf-8?B?SGgwY2JBQmJVcXBTaHZ5Z0M5a3M3aXdENHB6bnJkWlJVeHlvU2p2aU9FTVRF?=
 =?utf-8?B?RnVmbndxSmhkSnhrdWVCMnNjVENLcU82d1Y1Zk8rQklwOTZHYWE4VHR4Z1hN?=
 =?utf-8?B?VGM4Y0lnR1R6cWp4YVVDTjBJbUtIUnJjMUxvb2JsRVB3VkVEWkFHWFNRbG00?=
 =?utf-8?B?TlZsS2kzeXZGNEoxVm5sSHIrMU00WSttY21JelFJMEpQUEtMamp6Z2hnbnpF?=
 =?utf-8?B?VnMyTXc5R3lzTnVJQkM4V0l1TW9JQ3BiUWxTM0MvK3E0bWVBd2R1dU5uQUdY?=
 =?utf-8?B?QThVdDNlM2dVQ3A5bU5LZGV4UnRNbWI2R3h3a1dwdSsyeWQ5cGNaaFVaSEtJ?=
 =?utf-8?B?cEpPLzRzenQwRjUybWxuVmh0QzNMSk9lOE5rVnpleVczOGdjK2JjRDdyVTRj?=
 =?utf-8?B?RVIxTHM1TitrTXhqZEM2N0hDZjUzQm5FY0ZYSWFpSlhjR05Zb1pYd2YwWWtX?=
 =?utf-8?B?TUl6VEh0dTkxb1dWc0x2S25pTUhGRnA4V2RnM0t5V1lGRGpqSGdtdnlxYnpM?=
 =?utf-8?B?Z3NtSkc1VzFOQ0dzaHJDdmlDSXFicE5zeDJoVWNiUEFjc2h3VmhKQ2hjWWlF?=
 =?utf-8?B?eXVwWmRVTXpPdDJJSVlFSUl6ZWpsNXV2NDlmUVhqR0tJWFZpOThpVWFzSlNB?=
 =?utf-8?B?ZjNMUWppTDhPT2VrUFhiaE1jR3U4ZGJqUllNUW5QSlluNndPVEVGVlF6NkR4?=
 =?utf-8?B?Y00zM0pXaVlzMlJvTDVuckhuU0hzcm1pTlN0WGo3ZmRMSVkydWZnR1hLUWxl?=
 =?utf-8?B?YnE2VjJoNXI3Q2JmMVVxV2tLQkNNcW05MmNoNzB4RndON2dkZ1g1WVhsTERS?=
 =?utf-8?B?U2VwWVEzMVJjWEhjT3hYUlJKS2ViNmRSQ2RYTzRhbzJMMEFsdENLRlhQZ0l6?=
 =?utf-8?B?a3hUT2c1d0VXbGJtMU0xTDgvTzlQc1pQK2pFMit0NFJobGZnYTltandnTDJn?=
 =?utf-8?B?aGZaNUVYa3pHemFCQnAxWjlvanRvK3UrbnlENTE0d0ZUZEs4cXd6TWVMVmhI?=
 =?utf-8?B?TlpQTWRLditSanRXdlRKb1pYeDdxNUN4c0FtckladG1XS3o1TGZPMjlBamtz?=
 =?utf-8?B?aW9tSyt5V0NuU203YkIxNjFXdElSL3FKdUFxQittWGthTElnU1RvMEc1NXZx?=
 =?utf-8?B?RUp5dE5BRWoxV25Fb2MrcUI5Q1FnUzgzVC9VOGR1QUs2WERBOFhZTnA5MzNL?=
 =?utf-8?B?Vi9SUzh0Z0FVRk0xc2F5WmUvblREdTVoQnhicU80R09NMUhJa3ZmZG52Y2xy?=
 =?utf-8?B?ZzBmdUkyd1RMSW5ld3FEdUZBZzNHSldOTUQwc0JwNGhPT2xTTWU5VkVCZlpO?=
 =?utf-8?B?OWRyTWV6VG1vMTZVOUJPekhiNElxS1QrZWd4WkJCK29wQzV0TXNaYjdjWWRI?=
 =?utf-8?B?SWtIWmhVMzRjRFhMdERqQ1NjeVZNKzdLN2QxSDk5SFhERzhrQW10UTF4dEEr?=
 =?utf-8?B?UEtMV0tWekwyUmJPT1E0OEExbnI1VFpQN2RielBXc0h3ZmJQaWlDY0srQ0VV?=
 =?utf-8?B?MHpHdTlQc1hzenhTVE5NVkd1TnFpVUJBL2Zieko0U0NockdGN2RPTThIb2RW?=
 =?utf-8?Q?tePr6N0jUStceAZYO+MaSie5g94FaGzszZDWi3N?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 91454edf-7940-401b-bdf5-08d9713da8b3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:53:08.8640
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rM3Z4LV21P1n4pwaJlSRTBk9mmlMggVVkpTzzF+6jkJWK5pc+WYyUq6Pp7B3Z3cuAFBc3zf9Tmk2PeLBq9ck2w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2958

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> This is in preparation for dynamic assignment of the vpci register
> handlers depending on the domain: hwdom or guest.

I guess why exactly this is going to help is going to be seen in
subsequent patches. To aid review (i.e. to not force reviewers to
peek ahead) it would imo be helpful if you outlined how the result
is going to help. After all ...

> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -445,6 +445,55 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
>          rom->addr = val & PCI_ROM_ADDRESS_MASK;
>  }
>  
> +static int add_bar_handlers(struct pci_dev *pdev)

... this function name, for example, isn't Dom0-specific, so one
might expect the function body to gain conditionals. Yet then the
question is why these conditionals can't live in the original
function.

> +{
> +    unsigned int i;
> +    struct vpci_header *header = &pdev->vpci->header;
> +    struct vpci_bar *bars = header->bars;
> +    int rc;
> +
> +    /* Setup a handler for the command register. */
> +    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
> +                           2, header);
> +    if ( rc )
> +        return rc;
> +
> +    if ( pdev->ignore_bars )
> +        return 0;
> +
> +    for ( i = 0; i < PCI_HEADER_NORMAL_NR_BARS + 1; i++ )
> +    {
> +        if ( (bars[i].type == VPCI_BAR_IO) || (bars[i].type == VPCI_BAR_EMPTY) )
> +            continue;
> +
> +        if ( bars[i].type == VPCI_BAR_ROM )
> +        {
> +            unsigned int rom_reg;
> +            uint8_t header_type = pci_conf_read8(pdev->sbdf,
> +                                                 PCI_HEADER_TYPE) & 0x7f;
> +            if ( header_type == PCI_HEADER_TYPE_NORMAL )
> +                rom_reg = PCI_ROM_ADDRESS;
> +            else
> +                rom_reg = PCI_ROM_ADDRESS1;
> +            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
> +                                   rom_reg, 4, &bars[i]);
> +            if ( rc )
> +                return rc;

I'm not the maintainer of this code, but if I was I'd ask for this and ...

> +        }
> +        else
> +        {
> +            uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
> +
> +            /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
> +            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
> +                                   4, &bars[i]);
> +            if ( rc )
> +                return rc;

... this to be moved ...

> +        }

... here to reduce redundancy.

> @@ -553,11 +580,13 @@ static int init_bars(struct pci_dev *pdev)
>          rom->addr = addr;
>          header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) &
>                                PCI_ROM_ADDRESS_ENABLE;
> +    }
>  
> -        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg,
> -                               4, rom);
> -        if ( rc )
> -            rom->type = VPCI_BAR_EMPTY;
> +    rc = add_bar_handlers(pdev);
> +    if ( rc )
> +    {
> +        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
> +        return rc;
>      }

Seeing this moved (hence perhaps more a question to Roger than to
you) restoring of the command register - why is it that the error
path(s) here care(s) about restoring this, but ...

>      return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;

... ones in modify_bars() (and downwards) don't? I was wondering
whether the restore could actually be done prior to the two calls
(or, in the original code, the one call), or perhaps even right
after the last call to pci_size_mem_bar(). At the very least the
comment further up suggests memory decode only gets disabled for
sizing BARs, which we're done with at this point.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 13:58:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 13:58:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179960.326414 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNF99-0004xO-1Q; Mon, 06 Sep 2021 13:58:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179960.326414; Mon, 06 Sep 2021 13:58:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNF98-0004xH-Uf; Mon, 06 Sep 2021 13:58:50 +0000
Received: by outflank-mailman (input) for mailman id 179960;
 Mon, 06 Sep 2021 13:58:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNF97-0004xB-Id
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 13:58:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3c5d87d3-169e-4820-a44f-fba01049b859;
 Mon, 06 Sep 2021 13:58:48 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-ebv1iwU3NkmyAHjdTbZQbw-1; Mon, 06 Sep 2021 15:58:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 13:58:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 13:58:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0006.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Mon, 6 Sep 2021 13:58:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c5d87d3-169e-4820-a44f-fba01049b859
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630936727;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NbkbrBJpOYpfUoH/UPnckwv83IdFu2ZkE2MCOi9Y3VM=;
	b=Xvht80B43TV1ig1AtxQm6jmmM554MiLTs7KrlmURg5TbLXImbcjfZVj46m8IWWQYqQg+1A
	FM62FqyYr568RsoBVJzK4Yw9MDuUouw+eITYRkGFq2lZAKS3fBy+hb9cW+Q6fix5YllquE
	1OUgUCN+nGh0lReYH3lVa+65QBXWAIU=
X-MC-Unique: ebv1iwU3NkmyAHjdTbZQbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W/98m7EE/vaNsWFlRjnTFXQoozQBocGC/8V4/k1orNkAwjibJQRUp1lDNnNb2sM52VEyzBFGnzyx3QmloZCpQ2FFL9p0YfyB3R/Lr/bAJ8SaS3AiUYagria4SHQDKFcH47deVEs9B+8djyjpj7pFxFDkDx64jOYz7vOQiuuprJozH/Q25MwlqCRGfy3rQziH+eEtNuapug0S92GLuSZYEL8M+llktfVwleylnaNUcIyclb3kV13Ep43hHrdsqnTSAklUl5bKCJEl/fjcnal9kdH38HZNQqyZesAXKBSIAX02esRoDIVqrCMKQeFeEWJZ30PyAlRaQ5Y4W2r6H7XaPg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NbkbrBJpOYpfUoH/UPnckwv83IdFu2ZkE2MCOi9Y3VM=;
 b=LcfZQPEIjveA2wV8SuJFoXoXOTqnZsbfYKCcp8qmggsRYlFe+QntvsRYM83YnWImj+kgqcVWfBL/Z58FEzzce5ym1fpXneVxzQS4xQtwOb5oNAARzr8Wwj6USSs/SbvJ3kKRmE4ALiynacV6TXkBkdqvEX1KEf3UCCt+DYMYKT0tnpEtr5gPTuRog+Q8edCs5c23IM+mRT54/6PfInnmVt/Vur4xBu3H0xy/8b/n5HEpTR7kI9ruGm7cO+DJuGUIYhc3VzoVOwVUKtlz0L249s2B5lWE1y+iIGRh6xDvUrFgWD9OUkivCGor69eVANoAyCKJENV2Kbhg7zzNTKLiXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
 <7b8fba1a-767e-87f8-d0eb-5af5e5427bb4@xen.org>
 <60acd486-1641-6db4-4451-247edcc11522@suse.com>
 <26669864-09e2-a026-67f6-1f6dc48267db@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9fa952e9-086a-559f-bcd6-b1630e3bf5fe@suse.com>
Date: Mon, 6 Sep 2021 15:58:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <26669864-09e2-a026-67f6-1f6dc48267db@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0006.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9f102340-cc8a-43f3-a1ab-08d9713e70a6
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2959926C84B7B9B43B79F474B3D29@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GQpnch5kMXUBIn+0QNlyeh3YgQCxaGY/x8tjjta1c77M+32qhRQ/Q6z82VGuSxHmEmQJPH2xKR436cz4rXUsoDG7wPi545QnWSWH4vCStNC7s/PY5Q/BheJezpMN50tbfFiw4T/A8oXK+LwfLxL4ImwEW3Hz16uSLYoQszaSS9seE8aUypQQivHb2MXjajRvM9zkcU4iyYLMI0bq6oNz6bFymGwauAyRz61K4xfLWsUfzWiwEhFexoGPNrGMYkSgW5Qw56jhg5IIVhi1gRJD/aOKiyZats1Yvx0UEBYsvy7mL2T05fLOzMBu8Utv3gFQF0AVrTOlZ+JK1OC2J+CwCrGeluCrVoA0Y98tvgTXcfrrc/S7raEgw/QasHQ7wbWjgZpksF++COw+uCCd9ywc72wsEE/u7qI+KoJge/FciHpe3lnrYdPrDThMHP06D+FI7UHQuVqxpBJFj4zix06BwzCjQxorgvkK4pN3FWTvlSuYBE3d0uRD9PXIhJIfguyhUbXzriLsJYFYMGIza8m8McsdPTlY1uixq3ydWQMjKKtXq0xD5HPV6tdnpCHBf835nMwazDLZY2mPJwDpceJoPqaeDfx1UWe2lSHtFzYhVfv+d/ugdazig3GEjG+ZzRL/TA5Q0C7zygNTmNbvACfugGaFMAWavnXMe/Bsa5cIywSRs6Iqj/hl9pCiNaEu3FV/6GIUtVTG7DiO03xoMMsgmcUqXFL9/QTR/sjDN5EOBFU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(54906003)(36756003)(38100700002)(16576012)(66556008)(66476007)(508600001)(83380400001)(6916009)(86362001)(2906002)(6486002)(66946007)(8936002)(26005)(5660300002)(186003)(8676002)(31696002)(31686004)(53546011)(956004)(2616005)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UHF3ejdMYlRGOFZ4VzZLbEg0WDRlWEpBZlhzQVNpKzlydXhxdUdJQkl2TXk4?=
 =?utf-8?B?d0oyUjN4UjhFODlyb0dxSURQazNNQTNLMC9XTGc1dGtKdHA4b09BZnQ2cHZj?=
 =?utf-8?B?THVoTDRPc3BEYXZTRGI2MkdBK2ZhRnp6OEYwL2NQUHgrZFhMaGl4YVZVTDVn?=
 =?utf-8?B?Q3F4bDdqbFFCblBsRnhTKzhrZ2RhVXdvVGlwQkREdkVvUmdzVDEycHlXbndP?=
 =?utf-8?B?dS8zVzhBZUJLZTFBRHVJZHg1RmdTYmF4cS9xbEZGbmFmVVlocVhVSWRLdGxF?=
 =?utf-8?B?eWNBN1RsbmhJKzNNZTFSdi9wZXZNR0JrbVA0L01oQjJlUm5QcmY3YUYzMzVJ?=
 =?utf-8?B?dElwNEd1YSthTU12dzlWejdLd3VacllOT0w4c0luNXI5NzI3YWh4NkZtQUZ6?=
 =?utf-8?B?MUZDZDFXVmV5VXNXWjRTeXU1Wm5HYk1YZVdmRlVoZzQ0bGVKUUo1SUpXeHhU?=
 =?utf-8?B?ZkdJbkJHbVc2dmFVOWUvdFRVKy9vTGt1ZVlaZzB3RkxzZm1YMnVOVUlpYW1a?=
 =?utf-8?B?NVoyU08xbnhBaGcvVnNVL0Z3Rmc5RjdXakpTdXc4TUpnck1mZktkNm01VlJn?=
 =?utf-8?B?MXM0Vkp6REYwOHd4RmI0Z3lBcm8rei9QSTdRY3BvdW05TDFhdEdPa25uVnM3?=
 =?utf-8?B?NkZtZTJiTjJkSUcvZ1FVYjlUekc0bXpHcnpKT1g1SC9VOFlKMlRvTXN4NFZH?=
 =?utf-8?B?TlkzUkFmKzZobTNoYUtEYlBjOFpjNFZJNkFXdG5qNEpDWFRUUnZmK3ZNRnlF?=
 =?utf-8?B?aGhKTDc4ZTZ0RDY0T3E4d0N4Q3NEaU1McmNjdWd4bWdKbFBXN1ZzclF4bytM?=
 =?utf-8?B?T09ZNnhCcjJ1b2RjVEh2MllIRkhpNGczamFRVThXSzR5OG5PczVkbWIwVXBH?=
 =?utf-8?B?bzlZUmwrNUdadld1WUxlcUVuYVZTSDdjSWJuV2VHZGhiVU1QbDhvd0hhSmNC?=
 =?utf-8?B?TTBpci9oQTJDM3VJcEJHV3lwbXB0SGJneXh0V05pZkwwK0l1VHZNdi9PRXN0?=
 =?utf-8?B?QUFRTXd3RUtYaDM5QWI0U2Q1Qi9PUTc3Sm1NVWhFNnc5TU9EbjViTVdkTlV6?=
 =?utf-8?B?bVQ0QzNYWjYvL3pqN2R1dVRleFA4TGQycDd1QTBNVEYxSkgxdEdQeG5CNUVD?=
 =?utf-8?B?M3NGM0Y1akZwODJRT0lESlZmKzRtUWl6TktWUytqUGNVWXdMNUZCMlk4VzJX?=
 =?utf-8?B?eUtPQ0MxTEt5MmxzREZUUVNWQkUzVDFybDlMWW9icGNwaEdQVmVzNFFEOVpU?=
 =?utf-8?B?UGJScnhlWVhLNjlsbHRiSVhCOHkwMndDWVROWkY4UW9JNE8yOERQVDNJSHYz?=
 =?utf-8?B?TTJNVTkxV2VNMVl6RnAyZ0Z3OVRtUGsrRTdDMGpVS1YyMDVGVUozQnlJa243?=
 =?utf-8?B?ZGRKdUZRbHpWaEwwZUhOVEIvSDcwajgvdXBNV3VkK2pGNkhDdXVRR1NBcHR4?=
 =?utf-8?B?bUM4dm5oSWxNeC9vMm1wbXE4d2tQSHpCaHJNaGZ5aGdGdFBDWitIdEVKSGtW?=
 =?utf-8?B?QUhqeEtwdGZINEdxMlhkZjlLbjkzN213OVF1dFFBelA4blZwY09oRVVDL0lZ?=
 =?utf-8?B?QmpkNjFtb1dqelU4VThiNkgyUE9BN0pVdE5obmhQL2pKbnNMeVp3dDg2Rk9H?=
 =?utf-8?B?ZXJzajhxekh3V0dXUEFrRkJaTGdvYnhUY3ZlYVRTZE5uUVRVU0cwSElDY0hW?=
 =?utf-8?B?eElreU51OUkwTFlIaTZrRU93bGJLZHhPT1FVV043Y3pxMXFERXVXaTRCemc4?=
 =?utf-8?Q?zQP0qNiX4T1HvYfO2LeMOwufv9VXfg/BV3VVvVw?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f102340-cc8a-43f3-a1ab-08d9713e70a6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 13:58:44.3974
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A0DeTbpVrGQo58ekRmMSxmp8ivech/v0qGO1a9wCYTEIv44fzj/IXOH4dtofQ653bjYV8mM6DxhGSsf28KOI8Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

On 06.09.2021 15:33, Julien Grall wrote:
> On 06/09/2021 14:29, Jan Beulich wrote:
>> On 06.09.2021 15:13, Julien Grall wrote:
>>> On 26/08/2021 11:09, Jan Beulich wrote:
>>>> --- a/xen/common/grant_table.c
>>>> +++ b/xen/common/grant_table.c
>>>> @@ -633,6 +633,34 @@ get_maptrack_handle(
>>>>        if ( likely(handle != INVALID_MAPTRACK_HANDLE) )
>>>>            return handle;
>>>>    
>>>> +    if ( unlikely(!read_atomic(&lgt->maptrack)) )
>>>> +    {
>>>> +        struct grant_mapping **maptrack = NULL;
>>>> +
>>>> +        if ( lgt->max_maptrack_frames )
>>>> +            maptrack = vzalloc(lgt->max_maptrack_frames * sizeof(*maptrack));
>>>
>>> While I understand that allocating with a lock is bad idea, I don't like
>>> the fact that multiple vCPUs racing each other would result to
>>> temporarily allocate more memory.
>>>
>>> If moving the call within the lock is not a solution, would using a loop
>>> with a cmpxchg() a solution to block the other vCPU?
>>
>> As with any such loop the question then is for how long to retry. No matter
>> what (static) loop bound you choose, if you exceed it you would return an
>> error to the caller for no reason.
> 
> I was thinking to have an unbound loop. This would be no better no worth 
> than the current implementation because of the existing lock.

Not quite: Ticket locks grant access to the locked region in FIFO manner.
Such an open-coded loop wouldn't, i.e. there would be a risk of a loop
becoming (close to) infinite. Granted this is largely a theoretical risk,
but still ...

>> As to the value to store by cmpxchg() - were you thinking of some "alloc in
>> progress" marker?
> 
> Yes.
> 
>> You obviously can't store the result of the allocation
>> before actually doing the allocation, yet it is unnecessary allocations
>> that you want to avoid (i.e. to achieve your goal the allocation would need
>> to come after the cmpxchg()). A marker would further complicate the other
>> code here, even if (maybe) just slightly ...
> 
> Right, the code will be a bit more complicated (although it would not be 
> that bad if moved in a separate function...) but I feel it is better 
> than the multiple vzalloc().

It's the other way around here - to me it feels better the way I've coded
it. I don't think the risk of an actual race is overly high, the more that
this is a one time event for every domain.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:02:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:02:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179967.326426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFCQ-0006gp-MB; Mon, 06 Sep 2021 14:02:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179967.326426; Mon, 06 Sep 2021 14:02:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFCQ-0006gi-Gy; Mon, 06 Sep 2021 14:02:14 +0000
Received: by outflank-mailman (input) for mailman id 179967;
 Mon, 06 Sep 2021 14:02:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFCP-0006gZ-33
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:02:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78630ac1-b7f9-430c-9a05-e82bf196302a;
 Mon, 06 Sep 2021 14:02:12 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6-Eyatc_WFNDCP8D4IV-zIgw-1; Mon, 06 Sep 2021 16:02:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 14:02:08 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:02:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0501CA0060.eurprd05.prod.outlook.com (2603:10a6:200:68::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Mon, 6 Sep 2021 14:02:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78630ac1-b7f9-430c-9a05-e82bf196302a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630936931;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cKa39fBdUuKKjVZJgUwDa3lk0XUVFaHOI2TGt9dFD3s=;
	b=VYuYQG0BsbzWN+jyxdREgeQYRyjpGkkYHbqVYnG6qOIC2anAQKSL8wgN5q7Un5M7QdHm4o
	IeYrTq4Sn83KvBwahHb2fAq7zEsSdzTw1fS3jk31K91V9dJjt0kNf24UjeMN4BlnI/fD5u
	1z2IIhpAvUeUZxOo4v57M+eWxkrYLkg=
X-MC-Unique: Eyatc_WFNDCP8D4IV-zIgw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nBbCop4iwZ/1vJQhPYwnF0RG0nIBJNsg+txUdd9x4e6zVBL6fCgspB9H6NgnFDmYmFQUhKy4CalZLwo3ZmTC7WbE2N1g9e/+tpi8wQLSCMIfb4F1g9CezJCozlIH1zxblftsIKAi4rTBanZhOCTjoqz6h/bkDjithrqfWKWFeC+0Te4NHV4LnDJT9dgAiM0RkAPBbv6itR4MYBYM/z4tOUjYKT6B6vHeuy0YZCPhtPBZghRHw/wQWlWvng8upywK7g79AyUP+hfwtRM9Fsf1Tae8LxluumwgRvi7/+KtK0Wu3OU7QhW89r2RwBfjCaORy9+vNjN/byZnDh2wgTPZOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cKa39fBdUuKKjVZJgUwDa3lk0XUVFaHOI2TGt9dFD3s=;
 b=U0FhI7l3QY6+6kb7R8iCDS4voC/A/nkRC/GyVuCOxQp/j7np6B5piRyZ7pZaD9SmnNJPTBEdw0KT/xOh74I75mMvMGbxinWQoP3tDeRQcekSONcetQq4rWQHwJNjH2aRpxi1INUWYMFoVhPKcyYFsG9SJDMFGCaHzWsIqkkQgTktqnrQr765jtC4S2gwgt9ATb5Wnuqexz8npeddnUKFDRijcGqZ1ZIWy+GX7FTpdtzYdfSn8nwSx48wv3H5hZSStcrIAth8w87Lz5nfXJRn9nAvHlMxV/nq36QOtik0A6cuEUPoAePLwYGMDnNNfuqtCIKqOjbZwYUTLmlEIURxCA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 3/9] gnttab: fold recurring is_iomem_page()
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <06eeea76-adf8-4f40-eced-57db1526eba8@suse.com>
 <cb4f6360-4d80-be4a-e631-553b00b06a29@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5c960a54-7860-2de4-d3c3-0075740a69e8@suse.com>
Date: Mon, 6 Sep 2021 16:02:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <cb4f6360-4d80-be4a-e631-553b00b06a29@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0501CA0060.eurprd05.prod.outlook.com
 (2603:10a6:200:68::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 88148904-6c42-46c1-acce-08d9713ee9ff
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71505BACE9EAFC12C4AEF277B3D29@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	906ee7+/RJst/2GwNTTyxZy9QjJaRDNHme48ltmvGZigVpShOcbraNUy1O3fvAVj1IKVZtCT3g1Qdu95cETg4BT5CHcPmNKPy5hBJ+AlRhsopb6CAdEk5WWVjfmWWc8zYn39V0sWhTrNsZpOAOLXZhXPjnxnwsdgzP3gTqcGBm8Cxbc/VKHQTRt84fNgs8y7u/jBiGsw94/IbVR80a2Sn6I9c3R4amdhtXv+OzveS17lLtXrjt/dO4d2oI9zuK/A7fAEJvX+cAmFXMfqmrCtzxDCpfPrzM4YQmBE2yXyefdLGbsyEsFLxUYMjaRrizbSAc4F9oqUqQfQiFVgTQR4gR6PJaBX5dbxJf06a/3UaYDjGDKFNvWFkdflt0bJEu8Mw0KA1gfbFT0fY4PPEBPgLPLpPsh3st46rx381Bpw6wdIxeZHitzG2WZhAZCYDv/jQbWIve5gJoUQaWyqFmRkoptOn/Ed0dUzDQZOQrDCpT+lEtCVZuQzRJ+SPS/zsMjx/gxVoSlkC5kHY0XDcsecZSUTuoumTe10EOrj6pgsiqd7UVlLovATrnn3eCGTX14Vw0QfJevwiPrsmarwPymvyaUl7zoIZo3zT20RmtO4MFYc0e5fdKyzWpzVqzVlJoR4okl3rVcCw/wr+fhwSTkwxk9kq8QcE831QMEvmfCZLcsphev6RTmU29VuHR3LK49Mb2J/Z2sNOwhEPwlRGeUzzWp+oRk9fDKYrVXnSBjF/Cw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(376002)(136003)(366004)(346002)(54906003)(5660300002)(2616005)(2906002)(956004)(316002)(53546011)(66946007)(6486002)(66476007)(4744005)(8676002)(66556008)(8936002)(38100700002)(86362001)(26005)(6916009)(16576012)(186003)(31686004)(478600001)(4326008)(31696002)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?enRqRDlxTk1hMzRST0l2cHJxdFlhWXlYWFBDakZGWUowbDFITy94ZG1zelhp?=
 =?utf-8?B?dTVxQnVKUDBmVEN6Uk9zOWZlQUdZMEdZcG5HQmpsMzlNUmtOUjZiUm4vYmVk?=
 =?utf-8?B?czhaS2NSS3duYjBEUVRtbnlUWCs4eDNkS1oyV1J1QUNCV012L0ljc3VjMmlN?=
 =?utf-8?B?NjQzV0UyOGJLUVNQZnJ2ZXdDdDZjcmpqZnFkVlI3Z0M0SXFzWFMzOFZUa2xh?=
 =?utf-8?B?bHdrSTBXQXVJazcxZVVXZHlxeWFPbm9hUWE1VFhIL3RWb1JHVi91bC9sSU5F?=
 =?utf-8?B?RDZPN05JVjJ4UlhaMTkvOHQxNHUrWVI0RU02RmtFdllXUWt1RU5aZTQ5UDFR?=
 =?utf-8?B?bUwybU5UenQvcXpOS0pHeTBCS3AvZWp5ejExcEdFRnhyUnFKUnk4RXMrbXlP?=
 =?utf-8?B?WEhIRTJUcTlqRUtUamRKcUt1cURkaGdHUU5JdzFERjlpTHpXZ0NtRFZHVUZN?=
 =?utf-8?B?YXF1ZmJmeFRCY1QxWUtVL1FNVVAxSHQwNnpoUFFLaTBzN1pYYlRCbXV6RDVn?=
 =?utf-8?B?R3FaK2x5cUZscFRWU3JleEVRNUtyVm9YdWcvQnphZGFPZ3JtRTRMdXYvbldq?=
 =?utf-8?B?L3BMQWJoRTd2UUdMNVd5TTdlaC9SQnllY0wwb0VWRWtsOWhnYWw0dXZMSlo0?=
 =?utf-8?B?cDFKRkZ2MWJRZzh1RTdvSHlsdGx6RlhRd2YwVEdRWTM5L3AxMWl6Q2FucE5n?=
 =?utf-8?B?Z2lLYUdsQUFNSXFKbGRRVldVUmFsalRKc3pRVWtnZ29xR0FtdU1CT1NoRzdl?=
 =?utf-8?B?ZmxJZ2djSEpoMWR3YXpuWTYrVmpSN3BvSWtuZ2I0TzhsT2ZLbm5qdEt0b1M4?=
 =?utf-8?B?N09YRGtZbjVKNnZZdUQxT0JicFNmSGNvNmtYSEQ2V2RBbUZVUTMzTEp6TWV3?=
 =?utf-8?B?QzR1ZWJFNHJtOENCOWNxQ2J4RVFLWEtYT0tOc3hlWGxvNkgzNlA4OVk1MW15?=
 =?utf-8?B?Zm9sK3F0ZUpBbW9qTG9qN2JsVU9WVzczRTh4MWVQeC9pZG9mUnYzYUhiY3dZ?=
 =?utf-8?B?VTFWMmdZTDZUelZuYTJSK3AvTlZRaWpTeTFtL3U5Skp4bWh4NDAya3FielZ0?=
 =?utf-8?B?TG91cmlZNHJrUnFXeHVjZVR2a08yczlrM2VCSmtBdU8yQjNISDhsbkdVVk5U?=
 =?utf-8?B?Wjk2bXRUcis4ZWV5ZlltenhrR2h6MGVaYk8rSUEvKzhURFhFUnZyZzNTYWJu?=
 =?utf-8?B?ZHZoNmhrcUt6QnljbUNVdWs4cEhMU3l6SkY2NlFHZTY1czc0UzFwNk13MkhD?=
 =?utf-8?B?cUZPdnBYZk1UT282NGZWQ2hBYU9rblpVSTBHYjRIaVhXSWFUVXd5N2JvaHpY?=
 =?utf-8?B?MC82bUdhV0Z5OFd6SGNlRFFPTUxGVWNlQlFWV2tuVzlaUkRWR000bllQeXBL?=
 =?utf-8?B?VjlJVURzZlpvQkdaN2ZEVGlqdGtvTmxZYVJ2aXdteUFienJUT0NvWVVOdXRD?=
 =?utf-8?B?dHc1ZVVGdlREYVN4cnNGenNkMVRnODlJSUdNMTJ4Z0xjd2NrbjVRZkZldjdY?=
 =?utf-8?B?TzNsS0lVTWo4N09Ha3V3RjNSTWcyT3pTckFtcjNBenpiS3YrNGRaSUt5RFVI?=
 =?utf-8?B?WkF1RzVORm5hSDc2YkorakErdGRkQnZuaURtU0hGNDhWR1Y5dEtWSzdRWkor?=
 =?utf-8?B?U1AycFM5dFlBNFI2MzBhb3lja0VPQlgyQzlBZWlwZ3U0cUtwUFFsb1ZwcXBv?=
 =?utf-8?B?WHlYa2JPYzlBaTlkQjBDTUJhMk9HeS9CWXNjUkFhZlRFRmhxUUpvdElqRHpu?=
 =?utf-8?Q?TtAQiaQZA3C05NtNUwkADiyIWPy/KajbXDF/dOH?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88148904-6c42-46c1-acce-08d9713ee9ff
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:02:07.8703
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 42m5QWFXHVwN7qHOwcvb287TO4mTAsGuLYUo9pMGAdzIsoJnI3w3/xZ06T7R5lCZRpDnhiC82D6RGy0/8oR+6g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

On 06.09.2021 15:35, Julien Grall wrote:
> On 26/08/2021 11:12, Jan Beulich wrote:
>> In all cases call the function just once instead of up to four times, at
> 
> extra NIT: It is more like two because there is a else in 
> gnttab_release_mappings() :)

Well, I was viewing things from code gen pov, not so much execution
paths. And of course "two" is not wrongly covered by "up to four" ;-)

>> the same time avoiding to store a dangling pointer in a local variable.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Julien Grall <jgrall@amazon.com>

Thanks.

>> --- a/xen/common/grant_table.c
>> +++ b/xen/common/grant_table.c
> 
> [...]
> 
> Everything below looks a duplicate. Might be an issue in your tools?

Oops, indeed - some kind of glitch. Odd.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:06:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:06:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179973.326436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFGE-0007Jp-4z; Mon, 06 Sep 2021 14:06:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179973.326436; Mon, 06 Sep 2021 14:06:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFGE-0007Ji-29; Mon, 06 Sep 2021 14:06:10 +0000
Received: by outflank-mailman (input) for mailman id 179973;
 Mon, 06 Sep 2021 14:06:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNFGD-0007Jc-E4
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:06:09 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0765f5c-35f7-48eb-a7c0-38d7e605088a;
 Mon, 06 Sep 2021 14:06:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0765f5c-35f7-48eb-a7c0-38d7e605088a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630937168;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=dP5NkI57FysLi5fpG0vY0d9VkOPvehodDZ8TCsHz7rs=;
  b=e/lePDMKb88YSVyT1UCF/+7Tsz6RyIvQSpkNsz7uvqaS6JD3xYTpcmvI
   Nk+0ERwR1I8UiTGRqMSDv3Q6HLC8iU8pug7l2UhFFRCr9XfQx7qoIlrUc
   vgpxIDU+/vMO/LyZmyCjty6xvj+hveVyWSzmCsRoi07dh4i2ZXEFNlKxD
   Q=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: i20T5t4R1ZJa3d26qjyEsHMuRn1W1jpNe4yzzfe46ZNz1PT5dGaOdjW3307cp9wLCdOw4PsjMT
 Xf+0/guVrIPzMESAugdIQKIBmdM2U66COih1hkrU0WpbtXXAbZUXhZSrNX220sdnG4r8j7vkOz
 OCBo0XShrW146ES54FFx2rj58Cd2Rf2Z/w77Ia1JU00lT0XIkvjIhFOT/dtRgi6RRIeQENoW9s
 hz1aeWUy8jlnwX5H9WHOYek79JIoZ0K8tNqKWfvKGprgBajCkUsrPJFGMfnuREQBp0zjaUTI4N
 PNkFfkKktkreKjpiUI/s/Z/V
X-SBRS: 5.1
X-MesageID: 52080161
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:xkFLT6h4cUPukwHoDc8dBVSmhHBQX1N13DAbv31ZSRFFG/FwyP
 rBoB1L73DJYWgqNE3I+erhBEGBKUmskaKdkrNhQ4tKOzOWx1dATbsSkbcKpgeAJ8SQzJ8n6U
 4NSdkZNDS0NykGsS+Y2njKLz9D+qj+zEnAv463pB0BPGIaCdAT0+46MHf9LqQffng3OXNTLu
 vk2iMonUvERZ1aVLXAOpFTNNKz1uEj2aiWLiIuNloC0k2jnDmo4Ln1H1yx2QofaSpGxfMH/X
 LemwL0y62/u7XjoyWsllP73tBzop/M29FDDMuDhow8LSjtsB+hYMBEV6eZtD44jemz4BIBkc
 XKoT0nI8NvgkmhMF2dkF/I4U3NwTwu43jtxRuxhmbim9XwQHYAB89IletiA1HkwntlmOs5/L
 NA3mqfuZYSJwjHhj7B69/BUAwvvlaooFI5+NRjzUB3YM87Uvt8vIYf9ERaHNMrByTh8r0qF+
 FoEYX1+OtWS1WHdHrU11Mfg+BEZk5DWituf3Jy/fB8i1Nt7TZEJgojtYwid047hdcAo8Iu3Z
 WcDkxq/Is+BfP+I5gNXNvo7KOMexrwqCn3QRavyGLcZek60kL22uvKCYoOlZaXkbwzvdYPcc
 f6IR9lXVBbQTOmNSTJ5uwRzizw
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52080161"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oICJRuD1Wix1rzV8+iChMMRYDECxQK+2Bc/XhW4aYl9uyaqm7Rcze38/T2+FhgXuXe4IDcWvfCxS9p39GIrUBjr0zE/0p4Qa8n2pGGQKREeSZJNeFNhqwwkyxXK28Z6UL/g3qfaynMouf97sWhDPy+Q56AnUDcp2yl292zsGxfh5GpIX9WwY0rZgbobu5gDZvuOKhkoZeEhb6q/n0igNMoB+0BPTtjWGDqT5LUTWa94/LNafvaPx80NTsuYpU2IdvwgtdKc/c5hEPLBM/rJ6xGN5fzlruaI6FsamV+vdxEZHJjNCOU9623+XNieUlAIu1xr6aTvXw7E1jic348/Erg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dP5NkI57FysLi5fpG0vY0d9VkOPvehodDZ8TCsHz7rs=;
 b=nddb8BYhRQSx//I2snXs91d55aNAE1zhEBSvvabNUc6D+axS7/8Uv2QzNx/32H47YfLM8W0LkzmL0ALnsYJ+3zgKLN6vof+jdDs5LVdBtHePTQxZVRr52NT56HPntKs1nvyG8Js5+kZtMTxNRsHtG362aJLRiO6hMCwGJVypnjKeGEn760z+9W67ZKAEwdpkeWBezg1Plahw6YgJVnSutKKaffizerFDRv8I1HXB3oN93eYYym8QY2Yt65gG40NzrJDNJjoiKZuw0D66wLfOOjvsALFEQR6zBoGXWIIjODFoFEqDJ81OWEvTaZsFtrE7WgDrUrScmGCT7Ql8fhnL2Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dP5NkI57FysLi5fpG0vY0d9VkOPvehodDZ8TCsHz7rs=;
 b=Xd5F4ABKfnfSk8EoPKzG8+st5IyRaDFwRbWTyWVsweJz34TtrGXwAtvl+YvMvatlomvsfJnJuKjmS6EBnS7frfwaeXlxz6YGxJgU1URm1dgEQFs8R49o0r1oijrIaQ4Ca5+gYjyefed7VoP0PtomVTyygcO+SRtyna0DKQwHNaE=
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
Message-ID: <c3c20938-ba34-965b-0f1e-8d72c8004cc6@citrix.com>
Date: Mon, 6 Sep 2021 15:05:28 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0250.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8a::22) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a6d49d6d-5b04-40ff-3dce-08d9713f6555
X-MS-TrafficTypeDiagnostic: BYAPR03MB3688:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3688862C6869B863433C842CBAD29@BYAPR03MB3688.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +La/BxplHFP6Ntc3uPovdy7OBfJnaV2wou+x7wSE7NdYDNU09LhPs7HGQYuJsejafF6EQHpgL1DAH1WCSvj6ut2SLF+h/F3VU9SNTN/q9NJTEh61y5As1SzN7jEGCOR53ol2eNB7bBARFU/bq9JTWHc3bF645btDuE6z9o0SZCxEBOdAd28vS6inaeocCAGTuFX7NQSEN60LNmWAA59ksf6h+wZh+kXbbep5UCAjbCiaviDX1XpXOjjdZH0+PTFi7phgTyn6HCmrdO8s2LewwVYLfz+UP3XAfQPbseMcQty4QE0KLpJ8rrmUyf04gfKp7gqSDbgVCZLXYNG9Mr++1P+CUpEtZSQ9e2+LGVpcI4O7mBWOzxSdFRzA3Svinn2ar58v5XFcl9rmOJTfbsldV0pONNmHLs2rQe/vGd4I4OjeM2m4Yt4Gts7G3zRk6quYEbGIkphPUuwQK/X5bOEYUcv5vsxiaQCDLDjcwLPM8L8ylXCwviMLNIuj979t4mVQWOKM+XJ+yL2+34Uwpmsm/BeD32S/+GRFTVxakuU3qpI1op6zTALa4Vw7ytRH+wP7TxjRGmU3Tm/Ci3onRxxYjVOynvl3cnP2pbcup2AijlDw7m50Qc8LVa0e5wpD7u5bVwKqe8sIfqmnQbDEWcuLwSnSPpCa9nTxFoVaZVleMrcZw5VAb0lIspw/2WaWRUCXjoGujKrbQNeToW/B5B+WegVLITT0XPuXLIgN79g0OmU=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(376002)(346002)(366004)(39860400002)(6486002)(55236004)(31696002)(86362001)(53546011)(110136005)(2616005)(66476007)(54906003)(66556008)(186003)(956004)(6666004)(8676002)(66946007)(316002)(4326008)(16576012)(5660300002)(8936002)(478600001)(31686004)(2906002)(26005)(36756003)(38100700002)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q0NJWHg5K1FrSVpsK0tVRUVLeDlJcG1KU2laY2RhVno3VUhON2ZhdnpXcHdW?=
 =?utf-8?B?aXcwOXBnVkhyVnZCRkgyU3pCTFhxUjhmRk9iZ1dSMmszRTRJSjhpcFdobDZC?=
 =?utf-8?B?RGt2WUFTNmZWM0xPT3NuTzdtOFJWdWZTalNCYWxhNWdwWk4xWDB2cXBGSnBl?=
 =?utf-8?B?eklVQnJHYU9JNGNPdCtLZ3VRR1V0b0l4QU5qUER6WW5ib09mU0srZFBqRnFj?=
 =?utf-8?B?VkdRMGl6TStuWkh5MEdlWFBIeVlkRUZqaHZucFBBaTVicXBlamVKamV5MmhS?=
 =?utf-8?B?QitEUk1CTTNhZ0tCb040MVdtWVZFN3d1bExKK05qOENOai9sMUdPdmExNFFD?=
 =?utf-8?B?NUZtaGtWSXVlTko3aVRob3V3dk1HY2xlNEo4RHRuN3hCbE85RTVlaUtOUTVG?=
 =?utf-8?B?Z0g2ZklKL09xQjJKY2RmYkJReG1Qb2QyYWl1NEtnT2g3RHM4dDlMZkZrUHhk?=
 =?utf-8?B?b0JJREpEemFwTTF3OXk4Uk0vWjBVd3E5b2hQVVMzRnVDUUpHN1duU0NZRGlR?=
 =?utf-8?B?TnRDN3FJRm1Ha0lHUVpSbmtsNVgwRzRwTGoxN1pJd3EzKzUxY29pQ1doV0Qz?=
 =?utf-8?B?U2VOb01ON2xNVlFUeTBaTnUrYWd5Uk1menU0ZzV5eDg2YzdIVStya3lWcVVh?=
 =?utf-8?B?eGJtNlRpcW5KUlNPZVkveGJDOTExZFJsQm5sbTJYWklOcHdDWml2S1orVkFG?=
 =?utf-8?B?OXJ0a1h4RWc3MHlXKy9kc0ovZGozdytCL2JvZE5vdHhJUGZoanZkbUplQUtD?=
 =?utf-8?B?MlF3SHdVNk9qZFllYjZVWlhsYk5CSHZiUlFTSVNNaUNSMTluYlRWZVBrcXgy?=
 =?utf-8?B?aXE5ckcyeDZnS0NvYys2a0tsOWJmRnN1QXdLZFVKNGZMZXJUMURFVEZYMDdu?=
 =?utf-8?B?bkxQcjI0dkRCZGM5cUJ0c2RYb2loR0g5elpzRnVaSjN0R0RxZThrSnlmQnBF?=
 =?utf-8?B?ck91UWpDdzRLcTl0bGc3RXlobnVPWHRrdjUzSU9mRGczZi9kSlhtMldSSVRn?=
 =?utf-8?B?bThFZTlqZmZHY2p6WnZmdFY1eXM5NUxUQXFSL0ZIU2Z0dnR2Nk96QWRxM0Rk?=
 =?utf-8?B?c2hTaGgwb2lLSUhZWVZ2TGRQdlJucVVoQXlROTc4VjQwNlZuV1ZUZXhyRzhD?=
 =?utf-8?B?VXBXMWI2aHpNWS9jZ0Z6YVQzb3I4Vm1HQ2N0bGdtV0dLUzZCUThzVEYvdXJE?=
 =?utf-8?B?VGtaajVtSTRpK3E2blk0STZsUWh2Ti9sVlZIMDF4cUdvZmg4bUJsTTY4c1Vq?=
 =?utf-8?B?ZnpBMlBMNWNZQmllcVQrSldCb1hDUmdON0ZrWjZSTWI1WHNRWjlNUmJpL0Jp?=
 =?utf-8?B?NkJZY2F3VlRnWTEwRkRNZldMQ2hjckh4K1djVS91NTFvNmlXclZKdnRmSDd5?=
 =?utf-8?B?ZUxkelo4ZG56aTg0NGJtTFkzTmxsWU12dU40bGUvNHJKM1V0eGJQaUZPQ2Zj?=
 =?utf-8?B?OGxiQlV1NGZzaWtpb0lZQU1uWDVLU3c0b3Y3Qjh2TlhLTXpRVG93QmRKaFdq?=
 =?utf-8?B?TlRzMXB0VTkzOEpWazJXcFVkK1JpZ2hSTFR0RXloWGpkVkxYa2gwOWppYnRZ?=
 =?utf-8?B?eUF4L1lZUDZyUWJyV3V2eVRCbytEeHFCTVphMmV1WThwb21jelhqNXlJOTVI?=
 =?utf-8?B?R2lxOENKWUkrYVlUbjdmN2lzYlk0WXBVejZ2ZkRhU3F1d0k3bm5oMnFKbGlC?=
 =?utf-8?B?K2ZrNHdUSDhaWE1aTTRMa044NlhZTUZtTnZLbXpzZDM2eFZ2M29WcDgxYXUw?=
 =?utf-8?Q?IQmQYD+E9gKEW9Qb9U+n+Zwy/G1S7+V6xjhmKPM?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a6d49d6d-5b04-40ff-3dce-08d9713f6555
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:05:34.7748
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dX1LEfbM2fED8zEFrlDH58w2WQ/yb/6FfwrLBPjGAtmzn/83luso7XKX/E4Pp2779IsXCeV2TSsNX1C5XoEL/xt4EY59kYzinFjB5pSmYk0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3688
X-OriginatorOrg: citrix.com

On 26/08/2021 11:09, Jan Beulich wrote:
> By default all guests are permitted to have up to 1024 maptrack frames,
> which on 64-bit means an 8k frame table. Yet except for driver domains
> guests normally don't make use of grant mappings. Defer allocating the
> table until a map track handle is first requested.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Nack.=C2=A0 This creates new dynamic failures outside the VM's control,
therefore regressing Xen's usability.

The maptrack array (and frames for that matter) should be allocated at
domain creation time, like we do for most other structures in the
hypervisor.

Furthermore, seeing as this has come up on multiple community calls, I
will remind you that it is not just Citrix as a downstream which is
firmly of this opinion.

This entire patch should be replaced with one that...

> ---
> I continue to be unconvinced that it is a good idea to allow all DomU-s
> 1024 maptrack frames by default.

... defaults to something smaller for plain domUs, because this improves
the general case without leaving VMs more liable to crash at runtime.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:11:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:11:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179979.326448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFLk-0000W3-T9; Mon, 06 Sep 2021 14:11:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179979.326448; Mon, 06 Sep 2021 14:11:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFLk-0000Vw-OE; Mon, 06 Sep 2021 14:11:52 +0000
Received: by outflank-mailman (input) for mailman id 179979;
 Mon, 06 Sep 2021 14:11:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFLj-0000Vq-HB
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:11:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 60bd661c-0f1c-11ec-b0a8-12813bfff9fa;
 Mon, 06 Sep 2021 14:11:50 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-g9Zj8tjBNdOf4RIkj_6ArA-1; Mon, 06 Sep 2021 16:11:45 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6302.eurprd04.prod.outlook.com (2603:10a6:803:102::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 14:11:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:11:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0009.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 14:11:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60bd661c-0f1c-11ec-b0a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630937509;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wk5cxx/2uXDEEOICbmsgCL4++M8We3UgJBTequBverE=;
	b=PyZimjdsg7KnKY3N6z9iRGAQkkpXopFnR4p01R85vbfIsSeIUYso/JPfzq5kYZErt+R+nw
	WYKviCGWffwIntw52Go3zhI0NHd0gd1ECS/bmm2iX3jY7ShBpULX+lxSt7Tn2gU6qnp7s6
	0KJeVHPdHO7tM1O79VNeJ949sSmbj8Q=
X-MC-Unique: g9Zj8tjBNdOf4RIkj_6ArA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZW2PYW0XbiG3aDIDg+hQ19vOBv4fnmnjGXayQrUFvRkxfZ/FzRDHSblJMs2hrF6flgYnY5THKFjpIn1oPulDbHZl5egkIY3kAtqLEYGCYHtmIBLv/0x6vVjqjwfu76kncEJHUaGonWSyvypMcKa7wh12p7qoWbsUcyBpb62gkAGHow3iJqqPsfTM6XmZe7RgrJxQhLctDlLqjv2jxtmGQ9le2Hyvzlk29J6OIVsaflqTUnqSYYdHXlj5ySx99C9odrGPqSP1Dpsi4bh8DE9oy0/LMv/zIZh4rwg7Cjg0xhKF4o+sVgS5tSWMYIX9Dr6pmE+gkub+LUFu6jzMUMqn7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wk5cxx/2uXDEEOICbmsgCL4++M8We3UgJBTequBverE=;
 b=JY++zwrXZ0+4C5WT5U9jIgby+YwDfgNw78aWSOd99wQ89otkjr35/AozkHEcNnOnxjXB4e11e2HcCnfKVOsfSQ6AOnA3vnmQD8kKzSmWYWWdPplKM9U/eiasSsEb7lcy+fwu1cUGJ6W/7mqJQtG6Jn7QeeWsLXHZCEg/cnV0m4Gbwr5C+VGrDpQYhsDss6fcWNKJRLBL/Li5uNO/oaIxaytjemykk12009zqKP+sJpq+YKs2hHzrc66gp9sbw+3m9GC7m6pJmirhwxxx1rmFe5iiVm0yyhvKU58g2HyId9Z8g9YnZBnwdW6dzOTqKfQ8tRspAC0yAdZ65Mi1mZzMKw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
Date: Mon, 6 Sep 2021 16:11:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-5-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0009.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cf17f6f0-582d-4da6-60a5-08d971404192
X-MS-TrafficTypeDiagnostic: VI1PR04MB6302:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB630205BA4619EBFEECC5FA75B3D29@VI1PR04MB6302.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gJbQd3rKPYaaWwN2SXBZhLFsr3D6TNULu2PxXCI4Lj3Jh+b4ZHuNMn9aB7lQdcP0aa77N1BUGsm1kQMN3gAe6SQ+tSZK9d9bVx49kq1F+mAcYUgGrEqtgbbt0sNAwj4YXd7w6a4TqWx0GXBwB1/8SO9WFJ2GSTnPviM/0kpW25D2R78VWuX/y74EIxZsWS2fxCfVQUERpEa7mdYWAbB2LLRbQdmPv7iTguovJ4XLsppI/Q/jbRdlNX3VuuVL5GPzf7jkhmeuFu5hXFcxbjrcAWICbjhuM2aPoVyuydJXkrMsWGtHir84c508w3oVgPkw0ZNRzUDxsWPJA6QrlExw3/Ot0tGr8EvNGGdXWVnVj3WhMyoAZXY7IgiosDteEV7xbS+PyVXrcR7dHyoXU0aGtYeS4hf8WkIQpLoUs6nnGncHkIbHlO3GSyVN4Lq0HYQn8Uq26W5YLATdHm28rvlaQ9FzlI4O/arev8kO3eZ9hQPp4IwtQzfQFjo0r5JSjl1M0tOnGGQ5o5BntJpcxCu6B0K0ii8dGGZkFb/33hZ3E2lvsR0aKaiqESItEY8pX/CdGnu2nH2z2StMVk/cejRynJtHNnQymCXez+Zp5Rsv9ZdZzLNTB1UKQwuzIUFVdxxpQKjbQrfhRnQ+2Jp5XVdhjhTjZAod6jsrU0VhcuPTlkEKoXiSPUnfebg5pOmg5PvPivyRbvypkpvyXOKxkrBqTx+kUcOVqAXl6nzmtzkS2PY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(136003)(366004)(396003)(376002)(39860400002)(31686004)(86362001)(26005)(16576012)(186003)(6916009)(478600001)(36756003)(31696002)(4326008)(7416002)(5660300002)(2616005)(956004)(316002)(2906002)(8676002)(38100700002)(8936002)(66476007)(66556008)(66946007)(53546011)(83380400001)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MWpTc0FqdVVTT29EM3R0ZGtLOGNOSUVOOGsvY00vai91MmxSeEdDOWgxYWI0?=
 =?utf-8?B?T3BqR1p0K3hnaUM3czI1K3E4UWg5ZnowelhCRnQyMVVFeFkxK3ZKdHBPcEFF?=
 =?utf-8?B?cEw2bEJiVjlUT0djTHczTnFRNWNYd3VuNXFvbVBWb2dKY2Z0anFPYmFYTUsw?=
 =?utf-8?B?TmJMcVdTYndsbGJrYUQrQ1lvR0k3RjN0Ty9FNTNBT1RCeXUwN2tnQW5IdTNE?=
 =?utf-8?B?WEJhUVpLeVFVb3lDRnVTeEtSSFNkak0vM1RGcWNYcnJiUnpNUUNnKzNybWhV?=
 =?utf-8?B?SVM4V0RKbmpDdTltM1V5bW5ScEliL0JvUVhaV1Y0MHB2a1Uzcm42MEltZFZP?=
 =?utf-8?B?NkRpQzF4S1RNT09qSkNhSGowSTZMUElvUlJHNlRQbHFQZzl0NDNIVHUwa3Zn?=
 =?utf-8?B?cElqVXV5bDNlMEt5aDE4NnBOaGtpMTNkOTNNVmtXMlNXb2orOUZ0ZXhFeWRX?=
 =?utf-8?B?ejZNOUpENVBvcUNtRUNQYldIZ3pXSXl1ZU5oZ2ZZTytmU1pWUk4xbkt0VEhs?=
 =?utf-8?B?V0N6QXp1MWM3QU9QWTBjMExCQTRzd3dSeG0yK1JyNVdkcmVNa09ZMmRpdnpy?=
 =?utf-8?B?R3Z3UVFldHRoWmxhendsclRkejFaNUZqZno4b3pxZU5hUzM4RUdsbXZvRkFH?=
 =?utf-8?B?c3JRWnZZdHYzWGljRDQ4Rnp6MlZqb2xHUFdwdG03bHR2dkNCdnFyYSt6eS9P?=
 =?utf-8?B?ZXZKZHhsNCtGNmdjRjMrQjZPdW03L29OME55SDd0RWQwOEZxS0ZBc05Ob0c2?=
 =?utf-8?B?N1dDenVOUmQ4VEZPcjNqcmpQVkJPMnZMUElvMnl1WXRVeGt6RGk0Yk9kM2RF?=
 =?utf-8?B?L1RoT2paTVY1b1BDbTBKZDhJbWdQeEVWR1RDN0Q0c2NlcEJkcEpLOEtPaFFH?=
 =?utf-8?B?Tk1CRnFUdDhrdzkwMkFVei8yMkRRM3h0TUtESnhEOWpYbVcwTXB5QTlBNk9M?=
 =?utf-8?B?L3VSUkVGbTJKZXJEbHREZ0oxbGQ2QjhITFFJWGpISkJZVlVybHY3dkc5bHBj?=
 =?utf-8?B?Z1M1ZzRhU1cvcEF3Mi9FMHlPUEluSWRwMmFwY1NxcldqZm50NURlNlByOVg1?=
 =?utf-8?B?Q2Z2QkhrbWlWWVlYb2M0SEt4NStheGV3TlBPbmdDNzloVFFGNVNrWFpjSHAr?=
 =?utf-8?B?Y3ZjUGtGWGl1STNMUlhQNzZsdURSVW5TV0JxY3NkUUZobmFaQWp6R1NKUkZC?=
 =?utf-8?B?anJ3UDBQVHdPclVjbGJyZjNVOUJhZHMxZFRqTEpISWdIM2poVXo2eEhBR3hC?=
 =?utf-8?B?ODR5Qk8rWFFRakpaVytsdy9Bc2JxTy9KMVJEMEN5UVQ2Z0Vwb2Y1OTFjYUZa?=
 =?utf-8?B?ZThkbTE2eGJCYnYxaHZsWVV3UWdQTmhVUTZ3NGpFallNejJLNWtDQ1BqWi9S?=
 =?utf-8?B?bElGcTRxeTZyenNlZk9NUUtDOXNZV2pNRWc3VndyYllrQ0kweEVaeUVxb1ZU?=
 =?utf-8?B?WktOeTBuMWxESDlnRTUyQ0NjOE1LOUxVRTJaNHBaSG1wMkFuSDlGOTd4dFFS?=
 =?utf-8?B?MGtEZWRJWmxLVTF2QkxDUWhlYzZUZEdZZmcvalpLbnZxUlpVbmg2djBoOWp3?=
 =?utf-8?B?VmUvZmhBZ05VZm9KTnlpYm9xa3QzWUo2VGlIYW44enJPeDQ3TFhEK3ZFaTVz?=
 =?utf-8?B?eFZLNTgwczQ2LzEyTEdTdjMveDF0cVpscHlkWWFpZkdDcW9GVlB3MHVrYUhS?=
 =?utf-8?B?c0VROGxPbXRya3NYbStyTG5iY0VYUzFMOEFDaW1pMUpUbWtJSS80bmFXaVNI?=
 =?utf-8?Q?4q7q/urLMEpl54WfPpEquZYTNFVwPuZ7LLRn8Mq?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cf17f6f0-582d-4da6-60a5-08d971404192
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:11:44.3472
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RlnpXi7VmkqPOlu5/3GlsTfGtmk3snBG/srXujd2aGIKInQvQltI+Ox5S0MrN/kjfxcWULp9otJj+XIkqjiXCg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6302

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
>  }
>  REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
>  
> +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
> +{
> +    int rc;
> +
> +    /* Remove previously added registers. */
> +    vpci_remove_device_registers(pdev);
> +
> +    /* It only makes sense to add registers for hwdom or guest domain. */
> +    if ( d->domain_id >= DOMID_FIRST_RESERVED )
> +        return 0;
> +
> +    if ( is_hardware_domain(d) )
> +        rc = add_bar_handlers(pdev, true);
> +    else
> +        rc = add_bar_handlers(pdev, false);

    rc = add_bar_handlers(pdev, is_hardware_domain(d));

> +    if ( rc )
> +        gprintk(XENLOG_ERR,
> +            "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
> +            d->domain_id);

Please use %pd and correct indentation. Logging the error code might
also help some in diagnosing issues. Further I'm not sure this is a
message we want in release builds - perhaps gdprintk()?

> +    return rc;
> +}
> +
> +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev)
> +{
> +    /* Remove previously added registers. */
> +    vpci_remove_device_registers(pdev);
> +    return 0;
> +}

Also - in how far is the goal of your work to also make vPCI work for
x86 DomU-s? If that's not a goal, I'd like to ask that you limit the
introduction of code that ends up dead there.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:32:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:32:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179987.326458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFfA-0003KJ-Lv; Mon, 06 Sep 2021 14:31:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179987.326458; Mon, 06 Sep 2021 14:31:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFfA-0003KC-Iy; Mon, 06 Sep 2021 14:31:56 +0000
Received: by outflank-mailman (input) for mailman id 179987;
 Mon, 06 Sep 2021 14:31:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFf9-0003K6-Fo
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:31:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cbb728e0-af9f-43bc-8888-dc28a429f2c3;
 Mon, 06 Sep 2021 14:31:54 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-YH-nePbLM0mQl2P3n33QFw-1; Mon, 06 Sep 2021 16:31:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 14:31:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:31:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0020.eurprd09.prod.outlook.com (2603:10a6:101:16::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Mon, 6 Sep 2021 14:31:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbb728e0-af9f-43bc-8888-dc28a429f2c3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630938713;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EoJ9m66Pji9O6SyXYQfehum/FULzvCHrym3EdmuH0a4=;
	b=BM4hAG9WNwJZ+BCCoBvTO2GKcYgSeDnEILdo8e6/s1I3CocmRFGkN6x0sI8vPI1EnV7nqE
	8DgeYleszC2KmKlfSFTnCdmFWEC4Gu5Npaycqjq5AJN+GivZFBVMXiTLdWgbQ4joXfBMp8
	VXtdXTjaFnyDRCNtor02GL1k72Vaysk=
X-MC-Unique: YH-nePbLM0mQl2P3n33QFw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ign0+7h1XqoCzh5q+/7D4sRdvuLgoh9mwKAcYADT0Y+A95Oa9Ci4/qgc7Dz/J+ojrKtVoyucvcuRtGC7Eh0jI/Hkuq4eIwERKRe9FrAq4eagoxqSxAIs5eTQTGrt1k9L4xzFyCy6F3spL3UhCKKpwijnkwv52BSYMsh1LGHGgJ57fbVo3Fbq2ZXjnC+QOqPph62wjcL+7IVy2dLGiDspW8ftTZqUhygH5Ff/Y5wZYW11zWHhuu6kqo2JQKwceyoIhz6P+ZUl6fGHyoBiv88G+9xwus1F1wBdfMyEHqPR3lX5IbjMjHeryWvSmNoIbC0DLuMNTXnV1O353ob8rC+M2Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EoJ9m66Pji9O6SyXYQfehum/FULzvCHrym3EdmuH0a4=;
 b=XQMOnuvXPFYpYBXalxwSPSNNGH0AajrCdJ6ajNR5ELdpSIIlM6MqEfYA9aICGvbKNnjRiiKO5lrvb1iOfIRN0+nktOVkSYOhTDnLgvf3LxduK4/vjKaptOmZrJN6wSowjaqgmhQtfQROfI6UroL5X8ojSMS7n0U3Az+u3hIVrstggjCR5kRDFh34bAFWlJMFwdSP+TKbwgp8L6pM8+4zJ/scfVUSEFXqJ/bZVmMcyKl6KFRYBJZ7FL50vg0tTsAVySw0+IZ/P00456zvMdpQmRQCkKITasjTES/aJRJOOEkFPGZ8DrE388+UhYnvyVZZn5+rCbnnAxGMBmMHHlZhPw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
Date: Mon, 6 Sep 2021 16:31:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-6-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0020.eurprd09.prod.outlook.com
 (2603:10a6:101:16::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d5f45ac3-d324-499f-354a-08d97143102f
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70373E83E2912D45F355AECFB3D29@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	U7lwUZOpVePGij1GeL+s9zkHgRRcaOStPh7AgLoJTqCMK7ckk1EEOVgb7zp9Q4Lvvmbpz+o8OK7NqqxoMhZPXxZV2F/fJ7SQvEOFfs5Syl53a8oS49DcFR5kGiErAOgYoB1BwcHLm5AeXZc0WPKhl5v1eUBXxdI5nos9AI96FFImKlpEMCRF3LHn2VYb81rXFg5DQbXj1WfQUHl7Fslso7r3qL3J69Uc79a70KG+ZKHN2A10zWyFOtQozCrW0ajKhrm+RRUQ7dJTbdh+HyBiG04yzJH0cLnVJnvOrtAL2KsY7f6Rdf1G+khZNPkeuAEZTQSxcl+q7wJVjh8b32tbo7aMFpLAjYGSJ9WZHu0S+TCQlPFXOJc8JZwAtaMJkTbphVOn2kgCIjNGXATKPssYUX6s/bUDxQghFSiZNwYZUyXg/5M0ZiguDS/IvsHPevRZipS7ayEnIixDyI6ZHP4RXv9NO2R6r7wsH8zzLXmV7cyUodgyBLhE5btmyld1m9V+D7QXZ+N0p+Y2qtf8h2x+J1DebNH/UfMIx7WJBDd60zD4u9K8jCdkAzb9EjEUtDCKhJSZOChwy1Rc7GbbjKgjP24VmdeEojHxGdEyBdxO6iMSVT0R5fBggNYsekR2KqrsleA1RZMlJi5WIDeUsDvVH5dM6wdB/+f+KPTsvCfSqg0KXdk4GuWQMaxfYEptquZYd7XZrhqMtfs3cnF/Pk8NLd2hyvPLvu1PhJTtKGzeCGs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(39860400002)(376002)(396003)(366004)(66556008)(66476007)(8936002)(66946007)(478600001)(2906002)(83380400001)(36756003)(2616005)(186003)(31696002)(5660300002)(53546011)(7416002)(26005)(31686004)(86362001)(38100700002)(316002)(16576012)(6486002)(6916009)(4326008)(956004)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z0x2MU9rQjQzakthaHhJVmQ1bk01RGgraVFGMDNJL2ZpVFJyY1JZeGFUdFJX?=
 =?utf-8?B?RUlONjFpYzlCdW5DcjVnVjBZUUx1dVZTMWQrYlpUNnovSTdWbnpDK2pxYzhz?=
 =?utf-8?B?RWppWU5hNklFZjMzKzFFLzZWVmNiRldudUQ4M2d6azBWZHJaWW9NZWU4L21r?=
 =?utf-8?B?NDBuTGxicG9XVlVjY04xVDZGZmVXbzNNTXE1TlgxU21GOHBuRTlWdXBEa1pO?=
 =?utf-8?B?N2VJNDJzYVQzNjFZUlpvaGR1N1BQRUt1eXNYdThsMktib0FsSVZFMFhyQzhR?=
 =?utf-8?B?bGZyck1odG82VGxNMGFEd3VXbThjTFFXb2c3bW05SzRYTlMzTUFJM29ma1d5?=
 =?utf-8?B?a2xSTlNVMVZNQzZoSHo3a3NjTnBQS0xiZDJvejE5Q1lweGk0Y085V1FONHFF?=
 =?utf-8?B?cFo3WEhRQ2pDMVJsNFRsNTZSZ2xJbmVLVWd5amZCRzdncGJGYnJTajAyOGNJ?=
 =?utf-8?B?UGtxNHd1Z2dXMkNnVVdvR0RwUVpEa0hETGlWSDAyL0pZV3RWdWRIZ1RnTGV6?=
 =?utf-8?B?QmdHRDloN3VOQUtFamluYjI0a2RZeGV4Nk0xMW9zNEMrLzN5eUJuRXk1bUtm?=
 =?utf-8?B?V1FSNEhKSDZjUzlNUjR6b1RWMXp2SHVMTTliZFlSYnY2QWxUWER4Sjh4ZUxC?=
 =?utf-8?B?SSt5d0o5SkQyeExxRC9JWWJkdXErbEJPbzhtbFZqc3ROZWJhYmk5bkk5czZY?=
 =?utf-8?B?d0ozSGc1NjdYaTNQU3YybmhUdUpyclk3Nk1Tb2ZzeUxtMkFHZ0lESkhjT3Y1?=
 =?utf-8?B?WVJsSjQzcUZ3cmprWkpKcFNkL1pJenA4U3BoMVpsYWE0NGpUWnRKcmZKbW9U?=
 =?utf-8?B?dVVvaURsNGhDMG1lMm9EMFVUK3Y4UEh1ODV1aFh0cC9hNG1aUGkzRjB0c01N?=
 =?utf-8?B?d1VxQVBWZE5sU0d5cUNOb3Q5RjJubXRvYklxUnV1N0Z6VGUxS1hCMVZtai9W?=
 =?utf-8?B?WlBtSEZSUWF6N1ZBaUROVnlESUlncFNvYzlyRDg5bzBVN3NNQXBNVVBPZk9p?=
 =?utf-8?B?TkVLVS95TEgwZUkrS1NiZ2tCaDljY2R6S0xhUUlkZzAvNEZscG1UN3BTN3ZK?=
 =?utf-8?B?cWlUZkpESHJQOTAydmwwaVdFWlB2T21ocmZFRWNJdUU0cjZ2TUZmVVQ5eWZj?=
 =?utf-8?B?czVtNGtwc0V4ZDZlSkhXMk13eXRDOWVqQXA2TUxCVjU4Qm1CVGN1NG5JZnNB?=
 =?utf-8?B?cFFtR3dHZFFCVjgxMVd2TDdCbXFlNGtIYW5QaHNWMC9tRDB3NXdCV2FKL3c2?=
 =?utf-8?B?SVZYQngrWW9vZElqNTZldG9SWThtQ3RFaEc5ZlUvQ0R5RmNJbzFQVGM1MEg1?=
 =?utf-8?B?OFJTWlBHeE5SVXB2bE9RbkZRZUc4emZHa0hRbkZhV3FjUWErN0xGWExwWi9M?=
 =?utf-8?B?TXlwemp5OGlZMWFhaTFzK2IwNzl5RGhpdytMMGtQVk9mMzdjZTdwSjd4eG1J?=
 =?utf-8?B?WTR6azZjYmUyQytEdXJiOWR3aFNFYUpjVmRrR0h5NitmZjVjNzVFNGN5WTl0?=
 =?utf-8?B?b0dkUXZZZDI3c2RvN0ZSMWVKSFF1Rk1idFpvOHpnNmpRK2lZajVIMlRON2lr?=
 =?utf-8?B?ZTQxYjhEYUdVMVhnbGlka1hlS1E1VHAzbzRYTUUxaHhaOWVYdkF3N2FKNTNp?=
 =?utf-8?B?WGxiTit3Q3A1RUppVXVSWGZ1Z1p1eHdGS1lGcUxNZE1zMVpRbUxCemxpK2dZ?=
 =?utf-8?B?OURuTVpUSnZINzNxMTl5bk45bzNkQlBuMyt0eUdCaW1MZHBqcngyK3EySG1I?=
 =?utf-8?Q?Yq50CZLOQ8WxK8vw3a3VJlx/SkcDGlB4KceARx0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d5f45ac3-d324-499f-354a-08d97143102f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:31:49.9181
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: y82xFa4D+Nygj2ZL8d0b+X9ByfNkhzAcguHgiJvTh5Aw94D9ikVXaHkXad/3Ai/j0Ngb4Je8yP43TYTfy1399g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
>  static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
>                              uint32_t val, void *data)
>  {
> +    struct vpci_bar *bar = data;
> +    bool hi = false;
> +
> +    if ( bar->type == VPCI_BAR_MEM64_HI )
> +    {
> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
> +        bar--;
> +        hi = true;
> +    }
> +    else
> +        val &= PCI_BASE_ADDRESS_MEM_MASK;
> +    bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
> +    bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);

What you store here is not the address that's going to be used, as
you don't mask off the low bits (to account for the BAR's size).
When a BAR gets written with all ones, all writable bits get these
ones stored. The address of the BAR, aiui, really changes to
(typically) close below 4Gb (in the case of a 32-bit BAR), which
is why memory / I/O decoding should be off while sizing BARs.
Therefore you shouldn't look for the specific "all writable bits
are ones" pattern (or worse, as you presently do, the "all bits
outside of the type specifier are ones" one) on the read path.
Instead mask the value appropriately here, and simply return back
the stored value from the read path.

>  }
>  
>  static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
>                                 void *data)
>  {
> -    return 0xffffffff;
> +    struct vpci_bar *bar = data;
> +    uint32_t val;
> +    bool hi = false;
> +
> +    switch ( bar->type )
> +    {
> +    case VPCI_BAR_MEM64_HI:
> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
> +        bar--;
> +        hi = true;
> +        /* fallthrough */
> +    case VPCI_BAR_MEM64_LO:
> +    {

Please don't add braces to case blocks when they're not needed.

> +        if ( hi )
> +            val = bar->guest_addr >> 32;
> +        else
> +            val = bar->guest_addr & 0xffffffff;
> +        if ( (val & PCI_BASE_ADDRESS_MEM_MASK_32) ==  PCI_BASE_ADDRESS_MEM_MASK_32 )

This is wrong when falling through to here from VPCI_BAR_MEM64_HI:
All 32 bits need to be looked at. Yet as per the comment further
up I think it isn't right anyway to apply the mask here.

Also: Stray double blanks.

> +        {
> +            /* Guests detects BAR's properties and sizes. */
> +            if ( hi )
> +                val = bar->size >> 32;
> +            else
> +                val = 0xffffffff & ~(bar->size - 1);
> +        }
> +        if ( !hi )
> +        {
> +            val |= PCI_BASE_ADDRESS_MEM_TYPE_64;
> +            val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
> +        }
> +        bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
> +        bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
> +        break;
> +    }
> +    case VPCI_BAR_MEM32:

Please separate non-fall-through case blocks by a blank line.

> @@ -522,6 +582,13 @@ static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom)
>              if ( rc )
>                  return rc;
>          }
> +        /*
> +         * It is neither safe nor secure to initialize guest's view of the BARs
> +         * with real values which are used by the hardware domain, so assign
> +         * all zeros to guest's view of the BARs, so the guest can perform
> +         * proper PCI device enumeration and assign BARs on its own.
> +         */
> +        bars[i].guest_addr = 0;

I'm afraid I don't understand the comment: Without memory decoding
enabled, the BARs are simple registers (with a few r/o bits).

> --- a/xen/include/xen/pci_regs.h
> +++ b/xen/include/xen/pci_regs.h
> @@ -103,6 +103,7 @@
>  #define  PCI_BASE_ADDRESS_MEM_TYPE_64	0x04	/* 64 bit address */
>  #define  PCI_BASE_ADDRESS_MEM_PREFETCH	0x08	/* prefetchable? */
>  #define  PCI_BASE_ADDRESS_MEM_MASK	(~0x0fUL)
> +#define  PCI_BASE_ADDRESS_MEM_MASK_32	(~0x0fU)

Please don't introduce an identical constant that's merely of
different type. (uint32_t)PCI_BASE_ADDRESS_MEM_MASK at the use
site (if actually still needed as per the comment above) would
seem more clear to me.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:44:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:44:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.179996.326470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFqs-00052d-Ro; Mon, 06 Sep 2021 14:44:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 179996.326470; Mon, 06 Sep 2021 14:44:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFqs-00052W-OZ; Mon, 06 Sep 2021 14:44:02 +0000
Received: by outflank-mailman (input) for mailman id 179996;
 Mon, 06 Sep 2021 14:44:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFqr-00052Q-I2
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:44:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 31cb0069-ef88-4bf1-a74b-5629dadc2d36;
 Mon, 06 Sep 2021 14:44:00 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-6WtBcZGBPkyJDcJH7aE1_g-1;
 Mon, 06 Sep 2021 16:43:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 14:43:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:43:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0167.eurprd02.prod.outlook.com (2603:10a6:20b:28d::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Mon, 6 Sep 2021 14:43:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31cb0069-ef88-4bf1-a74b-5629dadc2d36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630939439;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=E8XZ4I1QoHTLVYKTUoACejOGqLW+G0EiotA4aZqRjRM=;
	b=Rqn0F9EPF8fZG/kbo0VZBlfQ8SIU5ysN7+L3VxIlqn7Rhdjla7JdK7Q4DVWUMkPgEH/b2Y
	TuRMh2hZCCGBKqwT5itDIsapOgJmgVpISXAJb+16hr+z6fzWi1vPPPsB0HwITq3+NyLtgj
	rOM/1Qj9PpNDbDQpHcQa4uqd80apIzo=
X-MC-Unique: 6WtBcZGBPkyJDcJH7aE1_g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Bv4Dc8/gSdBkB2dl2gTnrBywWW6QecIEz/XpXvU4XJ7Iww6qSKQmalZoKzWYh8LqOd0eL1NMCx31b0m36LW+puVxmm6ziGfk9JT2iTYCs0T7gYH4HTEZYON5aJo42BHezMmXAeuDNk3z7QIklCpUE+66EvrPE8QqqLGvN5UUt6l92n9BczciVbEqRnX/p4hKd0rLk0jaT7sf0ISyLRRra/xeNQOjkL+1ID0DKzc7jukgTdCugmTDCIBQm13Cp+PMDN3fID+oTHlayuy2AqEjGp91GlqxEvMoC+3VxDDz0cPsCIR6armlB2wCM7SMwQUZX3Bi8D45LjO5ApwE3Ba+bw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q2f1EWZnO8vB5o8YP7/43ffRbfMbHlhAQBauZVv+dfE=;
 b=cuwwSlHO44Pu0CtI+5JesZm4ieE08G5ICY7KgXitP/k5l5FTMynXq0COg4IqI6XDbMN4kd8s7ffvYeDTKX7pSz5OWEUO1CEhtKWc2ve2g0roeApN5I8EtQxKilseMmWQRODjZEAbt+iPg/rW0zj85kCE5Nd81vV5GUKMG22FxPm2EgWobVD6hLbBj0B6UIjdvbwBftdL7ufju4sQojIjwL0MT9HcbJKUAnQEHi3gqvMXTbcmI+IAcf3BfIeJFrViGNIJDtkAzlHs7uowZygTM4ZD5xhDDWzQt9CHqrvt4j7m5Aoei9ftcWqRESl7iWJz9jnmTsMPVhrTfM38fa1kxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/9] gnttab: defer allocation of maptrack frames table
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <eb7bbd51-1da3-9a7f-27c7-49dea1a41488@suse.com>
 <062bcbd3-420e-e1c0-3aa0-0dfb229e6ae9@suse.com>
 <c3c20938-ba34-965b-0f1e-8d72c8004cc6@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d26de0f2-12b5-4b3c-0956-0c2d2aa977b9@suse.com>
Date: Mon, 6 Sep 2021 16:43:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <c3c20938-ba34-965b-0f1e-8d72c8004cc6@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0167.eurprd02.prod.outlook.com
 (2603:10a6:20b:28d::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d3b38919-ea7c-4041-b2fa-08d97144c17d
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2447B19275D08CC1950A98F8B3D29@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	G+slpr7YOPTee+eS13ie60O9wLJm5UCNd5s+ckkPjwLKgs/lUknJVXWeutQrha+puo9k98PSt8gK+lc+cJXp4qhg/T1RozoYrOEKZxMFZbeZ9syX+i8Hj9zu5DS46CNvkLbQwEkxm5V/QE4mHNBTuthV/mNzrAFGdbLtceSkFTZrDUp9Toiqd9U5XY1+xwLuTB2IBt4WpFE/Ar46dgHyyUKjkenM47s45I9cYJ7Q3Y8W2j0yqtJXogMaCamHTCd6oj5QwyKfpofJeHtUR9tSnf621Ztnu6g9GVxLkVGnpNl9LRX5BUJxeMLw0dQPqgUU5NWAitOQ729SXITIHQSRJQHX0fRrtwZkD4+0qe2GL/ghPNCclpLkdfUhzq+5jPl3JiBsJ30H2lYT23U0nhxT+oPtTrOwyfvPdca+eZ9ITaVClCgAt5fopTdjJs/sq5Z9IrPO5pw8/2DEFbVY6lzR9mN75AAu3HCCvw4LL8cWAAqMdRbkMV3vCqX12JPKTcZI3hNRebFGVGD06D7roXfOpH1PErmBGnLmT+Pe3vbSgnKnLCrQqxD8OEoDLr0Rhl5oEemgdGTt9/2Y6UEICeXrd6mVYdvJVtp5n0PgGPrniSGFi5OMZglhx/y0kTo3qjuuWpxpsbIPk0/I6eW1BXm09WQC+TqAVdGi4Uw411b7hzi2ekQYs7u/sRyTrjJtWyedHl7SYyTWnPmvFnq24jS24bDqap1feQRShvYl0Fs6Naw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(376002)(136003)(39860400002)(346002)(53546011)(31686004)(26005)(31696002)(2616005)(186003)(5660300002)(4326008)(956004)(54906003)(8676002)(86362001)(38100700002)(316002)(6486002)(6916009)(16576012)(478600001)(66476007)(66556008)(8936002)(66946007)(36756003)(83380400001)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?kIhSJ/OKTAb0u/vlR0jDgW4iPgXZ1+BTzbg5S2QcF9nVeSiChr1noGjccoLe?=
 =?us-ascii?Q?m239xnQ1XrpLbI06VzLiy5jwJMlx2TlRUWiDr119glblLnaHmNmhJpHIDQgj?=
 =?us-ascii?Q?KenC3ZtNUyt/q4VAMyQPp/RdNW/FRkJEXJet2Vd6mUN01jcwCJu9hATTxf98?=
 =?us-ascii?Q?jdOjNBEzGOfa+Y1fFCZFp6Aiw7MfeuSMUdhm6Yr+poZoq8kXXVu2lpcVP+2K?=
 =?us-ascii?Q?OWvSUo98AAABYDsELHp8Mo6YvV7YYK7x+sCz6L8Kb2uQg6exr+ynHleHD3l4?=
 =?us-ascii?Q?HMWTWe+/zshDCDMexcpgvAgrDl1wAEHpMOBL4L9DD5NT54vJhNkmZ8VfcJxZ?=
 =?us-ascii?Q?BHpHAtVKR1pXkOfmGvInmYGRhNDZK7KVYdaMm+nP3teNT5Z4fZpaRPsG561F?=
 =?us-ascii?Q?eaL2JnFvpg+QHlLNBMl4C+ky3QSevQnt331SRZpoxbcKmJ2vMlHiDtFW4Rki?=
 =?us-ascii?Q?Tu1tT8yraAcnXV4SyJOtcEhukrOoW7g+ovabH6gjP1553YCewq/4Sj4FGPZA?=
 =?us-ascii?Q?Ueyt4DX8YX8IRCzCgfUJip1UavNp9v5BYYM07RwaFyPGMUxYbBKdB/4qgJ0J?=
 =?us-ascii?Q?hB9I7coOcrZBLblLAP+9gZH/mS9GhRMLZfEXvayC674dhdBeD4ycY7VtX2+t?=
 =?us-ascii?Q?xLAy8d5s8O6ZFfVg45TECuS/oYsB1OPNcYCX63QdcXcxOr3nNrAlZgEDOqwU?=
 =?us-ascii?Q?uwuSNVTwUqXSRUmk8Re0Ra55F29jUQQIdmH5W/KZqooRI+cXpvvdPndEysEQ?=
 =?us-ascii?Q?dQhS19WzROS7gBuyRB6RUsee+N6LTPkGQ+nNAeF5evvEt49yafRSliOf0dTa?=
 =?us-ascii?Q?ur7TFfHxg5kxRt4F2+4sIn9IB1cMBnUqTaCTkvOE/TKAxS0nB00l/V01WZUc?=
 =?us-ascii?Q?BfZvow9VSmMbLG0xixShyVOEKrvnBKPvtr3FiVvKgOOG9995LX1aGETBc4bv?=
 =?us-ascii?Q?rZx/gbdAYYjyIEW1ccPgRHuRmld238slGm3n8ItEpKoUJXcOGCvaBv+EOkmj?=
 =?us-ascii?Q?AkJcZL5BFHvz8sND4PtdUj6svxCqK2/+uFPRtl8twr4NGAl+u14gpC4bxris?=
 =?us-ascii?Q?4WchkeBDoYPPao4t9dMA1+fglrMvttIH0f8F2QsnYWW04uTnDgkAhLX60Qaz?=
 =?us-ascii?Q?2DuB+zW4tZ3wj8xUiiABFQ82vfEEb2glHZkC8kicSgchm4NCprR4BTdkXa6o?=
 =?us-ascii?Q?CPxevhxsUxJ6gLcElivNvDZ4JXgIoALODnqUw0Y5oVRX8vnCgbyZs1/AY9gf?=
 =?us-ascii?Q?5asPb7NPBNbt16bcDwqEnBJxkkbWRBOGFNi0a/B/iKMsaExpd5rU2IU6Skgc?=
 =?us-ascii?Q?VQjNsz1veZZkM2kRwrBbr9bQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d3b38919-ea7c-4041-b2fa-08d97144c17d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:43:56.8890
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: k7Wtow/z8Ao5wWMkELMANmih4lr7IcDNUhvNOGjV0+45MX6oZJWu+7wG5ybaqoWr5xvkVSy4X4ps4kxClz4arA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

On 06.09.2021 16:05, Andrew Cooper wrote:
> On 26/08/2021 11:09, Jan Beulich wrote:
>> By default all guests are permitted to have up to 1024 maptrack frames,
>> which on 64-bit means an 8k frame table. Yet except for driver domains
>> guests normally don't make use of grant mappings. Defer allocating the
>> table until a map track handle is first requested.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> Nack.=C2=A0 This creates new dynamic failures outside the VM's control,
> therefore regressing Xen's usability.

As with the v2 status frames tracking array, the memory demand of
the actual table will typically be quite a bit higher than that of
the frame tracking table. Therefore guests already are at risk of
observing failures from these paths; the increase of that risk from
the change here is typically quite small.

> The maptrack array (and frames for that matter) should be allocated at
> domain creation time, like we do for most other structures in the
> hypervisor.

Structures we allocate at domain creation time are typically ones
which we know will get used (to at least some degree), or where
allocation can't be done later because paths this would be needed on
have no way to indicate respective failure. Neither of this is true
for the maptrack frame table (or the v2 status frame table, for that
matter).

This said, I could see us _switch_ to a policy like the one you
describe, or even allow either behavior depending on some kind of
setting. But then this needs to be done consistently for all forms
of resources (e.g. also the gnttab v2 frames, not just the frame
table).

> Furthermore, seeing as this has come up on multiple community calls, I
> will remind you that it is not just Citrix as a downstream which is
> firmly of this opinion.

No-one but you has voiced such an opinion, so far.

> This entire patch should be replaced with one that...
>=20
>> ---
>> I continue to be unconvinced that it is a good idea to allow all DomU-s
>> 1024 maptrack frames by default.
>=20
> ... defaults to something smaller for plain domUs, because this improves
> the general case without leaving VMs more liable to crash at runtime.

Just to continue to waste memory? Or to force the admin to know very
precisely how many maptrack entries a guest may need to create, such
that they will neither cause large amounts of unused memory nor risk
the guest running out of handles?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:47:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:47:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180002.326480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFuc-0005h6-Dr; Mon, 06 Sep 2021 14:47:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180002.326480; Mon, 06 Sep 2021 14:47:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFuc-0005gz-Ag; Mon, 06 Sep 2021 14:47:54 +0000
Received: by outflank-mailman (input) for mailman id 180002;
 Mon, 06 Sep 2021 14:47:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFub-0005gt-61
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:47:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 93362bb3-838f-409e-8ef7-20d2ec162293;
 Mon, 06 Sep 2021 14:47:52 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-ZUAwH7HBNbeAZCKnY8ICBw-1; Mon, 06 Sep 2021 16:47:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4445.eurprd04.prod.outlook.com (2603:10a6:803:6e::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 14:47:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:47:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0154.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Mon, 6 Sep 2021 14:47:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93362bb3-838f-409e-8ef7-20d2ec162293
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630939671;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AwH30p3wBzytY59LExMHtIgmcTmt2MaZkmO+Zyddd+c=;
	b=EwGChiis+7h69KPouAgKmLw9skNB1Pxkx9kvoopuNtlgJgPop5nCxIj4mtrsFf/g7s2C/N
	N3CWadQ4QJomH+f/9x6A4XKAwEY9hApJzcM5Hzi+UN++O+HKGo2s2UVy7AM+beb7IM6/e7
	IbvqGhdlyC4eVDlDlQOKU2GJPgVfchg=
X-MC-Unique: ZUAwH7HBNbeAZCKnY8ICBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OTfbOVi/et79nELYmQ8BNq5hkBRpCAPpse63+n3f45zLQcO1I8bXM/5C/tzpVawm+6ukcu+FIf0Jc3ix63zkfJDN3UJFSAq9xqsyY/Je2wmkaMzP4pk3aay0x8Y8hYRZOFwrNhfospa9wXWGK+tWUYX/pMhQfYHnQSxfxgc+W1wl/BrNGx/NqXjz2f21B24M+6g68TT64MjV2ls55u+lPGicIwETsMkPznfGKprZs7y2XKThPwamZSvx6TknHxSKXYL59PSKhFSixdjV/VaITlayAzak/SfWeV8Xuu4eYrwVPlV2ahX9eG1XCFLZh/E5fC8CH2gvW73y+t5tClTy9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AwH30p3wBzytY59LExMHtIgmcTmt2MaZkmO+Zyddd+c=;
 b=WirNdtYCR68ybkJpUYfjzFTRXSYMd29JPEu5R9tki/RxVVLJSLHmEhlP73I5NI/8ciX3HI4T5LnidemDPT0lU8Zats97XMdLBI+GKseJOjNNg/JL4cR9y74VBi9zZ3GiamfpIEIi3YIrQPCaXx4MJSFkAX3W3psU4vq1gQFB8+Z5RJOl90J2KtndwsVa84gLvIo14t8bYLQPgC+y7inVr1nbkN0eZCm0FCS1qeZ6Mvt6FtqIhf/FndZVlerMcPZzQLTEionSWciiaC6Yi6a/EzeXQOsMHxup4Q/yapY1ZHanmRUq2yOUpsSYYlVyIWhJuRPsqwoImefsrypTsY6M+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
Date: Mon, 6 Sep 2021 16:47:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-7-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0154.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4bfab33e-f55e-4ccc-05c6-08d971454b8d
X-MS-TrafficTypeDiagnostic: VI1PR04MB4445:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4445CF7748DA3ED4BC290119B3D29@VI1PR04MB4445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	r5NAMpvGAwb0UtPPZ9NXqtMPLGHTnhpPsw6J4r8lG222Q4qNZybNsgHe3mdvJOb7seTf81dBXqRJ+A21YVmil9opouWWYwnSwIePjohlsucCwDcP8feaSusvmGtnFbPrFjz7wyUJQN/7LKeH+yYziYGs8SUHFxtGnsm2aZAyelLOWyL9eccYH13hA7oQ/VjJv3JtrSL/95xxdPX77YBkUzYpyuGTLyE5O9Kl0i6v+SDrOsZQe5yXlVaq6QpIGehuj7TThVsbC3KgmjzAyThwwXj1zFblnEQHLyX517eXpzJyqxZcHAZiUZHR8A8QFPTcHj3577dGp06lm/vbqI75bEGBk6V/bMc0c+x4LLJlR4X1/WjXllFgTk9fhdTbaK6Yxxoai3lDJ3DrV7FY6aCz38+DZMlHjPa6tuKQn53IODR146nHvFME/PdXFXta+VctOMz2w1mUStFXAZbaxtfkWM6bTIZSNoqcYwoH8DYeKcHFjG9HmLihmSi8hzL7wavQaNMc0DLay6zO2nc8A3r2b2XNtV9IsGUsMHecDl8kINh7MIM1XLiPtrSX0gXSIhDxdJCG6v50A9cKfRAkN2KEDCSjGFQiresGSxPy8BS+9MMCNgtQWBqOg7FKGQ8WXFttVYLbZR8h9Tvtaw4znO36GJPy3OQMdMe2dvPK6C7nDQr4+hkFWCf5riX1fG7idx3z9abwbBL7tCYmHux6WumNT1UNXOsHTFh7k8QnbCd61E4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(956004)(4744005)(6916009)(16576012)(26005)(508600001)(7416002)(86362001)(31686004)(6486002)(36756003)(31696002)(316002)(53546011)(66476007)(66946007)(186003)(4326008)(38100700002)(66556008)(2906002)(5660300002)(8676002)(8936002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M3dueGYzYjhtUTZOWjZjY1E2R1pRN0U0TnRISUFTSEl2Q2dZTXpJc2NaeERl?=
 =?utf-8?B?NWs4bWd0V3VESmVMY242MHBibE9IK1NNa2c5MVk1NDVHeGFmQW41cm0xSmp1?=
 =?utf-8?B?Mmd5a0g5TUptMGk2ZHJseW1XU1JEWHR2UlJUa2xNczFKeFhobzNVU2l2eVVp?=
 =?utf-8?B?K1g1Y2xyK1UvaFBueUpTQTNZQnJkVGlGNVpmM1R3WUFNc213K1JqRitzRE1H?=
 =?utf-8?B?ZTJ4QUZuNFMxRGZmWmE4dlFkdlRCQk54b1JmRUQvMGVoN3NnUTV1WmRxeGZP?=
 =?utf-8?B?cGU1YVptTDZtNld4THZlVnpuZkViQ2ZiZWZvTVpEYzdGaStMZW4rSkhEQVZq?=
 =?utf-8?B?KzJLTkUrZFZwOVZQWXdXTW1Eem5PbHZpdm9HUDJVaWlIeGtieU1zNjZnVTMr?=
 =?utf-8?B?OGFOeWV3c3VkTXhPMDZ1YllvUHpRaE83Uy9WUjB1anBhOGxIYm5mVEViYndX?=
 =?utf-8?B?cVRJVFZoZ0VpZy9KM0VEVzUzdDNJSjk5QTVBVEEzR2JqOFZjL3hsQXUwWE1i?=
 =?utf-8?B?TVE0K1NEVmVZd3NSV1pHSDhhY1ZldTFIMG9TM3RXSEYvOThZUm5QMGUwcEIw?=
 =?utf-8?B?UCs4dFdZSHNGOTZPaVR1VWpWWERCWkdwclk4eksvaVo0NkNQU0ZBcVFEc1o4?=
 =?utf-8?B?RWFNVVpucDcycHZIVndiUFZNbVNjdlVXZm1CdVk5MENMSG1TQzBjRk9Xenox?=
 =?utf-8?B?ZlhLbXA1emd3WUpzUXYrMkYzRFg5eDhzVmlSNGsvSmtodkNybjZpT1VEZWZa?=
 =?utf-8?B?NTdBdUdjSXgydlkwTm94TzZpYlRXWktPbExiamQyd21Ca3luZUpRVjVsNGpS?=
 =?utf-8?B?WUtaMTNHTk1NN3NHWTVoeXVzMnlTNXBNTDg4dkRHMCs4cjF0TU42UWtBYnNB?=
 =?utf-8?B?aUx3d1Mzam5xVGR0Q0tyQVJzQzhhMmhhSXFwOEFYMzhWN1E5blM3aW41bjhs?=
 =?utf-8?B?d1hnSGFudlhQcHNyRjB4MnVJVzBjdzF2ZFlIWFEvc3ZrOEJ3aWZwcjRDQm5w?=
 =?utf-8?B?dXQxUFJ2OGl4NW9oTHovS3lKd09xSU9QNStOS0Y5YmpSQjRhMXdjZHM4djVX?=
 =?utf-8?B?SVluOFZnNmc0eGpIbXZuVUV6bWlDM2tIVHNhOEhJaEJuSlIraW12UjNPNHZr?=
 =?utf-8?B?S09nbXQzalBIc0RaUEcrdnZMY2s1TU9WM2RGRTRUZGthU0x3NUowUGlkZTc2?=
 =?utf-8?B?OTJESU5WaG0wWjkxcFd4MWNTMkZpM0xPVDl2Y085aEtza3NvTk9aYWU5aWw4?=
 =?utf-8?B?NEd6QnlWZ2h6ZnZ5WmN2MnVaOXhoUDBlekVlOXRVRTJ1WWxnejhOZTV3Smdm?=
 =?utf-8?B?aExmUVd3WGFPWExtMHl6S3AraFM0UlpIWjFCWHVyWXpJUWZnSUFQVzBpZm9L?=
 =?utf-8?B?d1VEUVhBcDBxdEM2aDRGV0hwV2g1Mnh2VFd3SXVueGRndUxJUmRENU9NMHZp?=
 =?utf-8?B?Q2pLbFVLYkFuVTFoaDZkbVZXd21NT1V1TWU5eGhsZmtGZWxKOGJxaWJEbEla?=
 =?utf-8?B?RHNIcUFsUHNqNXJiQy9OazFhZW1JQjNsUWU0NkZGWktHcjVIdmVMaHFkYmxW?=
 =?utf-8?B?UDVndXdBQ2V1TWplbnBxUkFEMG9RNnNPSGtFdTlpa1RZajVJQ1YrUUFaUTgw?=
 =?utf-8?B?dThIRnNFb3E3Nk1yTnFvcHhVdlN5QTV6cTdrdWlOckEvaFVyUkszUGFEdjZi?=
 =?utf-8?B?TnljWkpwdndyNjF0V0EyY00rQ1RxU1JhWGxIeUtXQnV3VlZmcGJVVGNBcGty?=
 =?utf-8?Q?4wmloHMG1g4fagcRWwWVjyXN3Jm4i0iJ7iheFbh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4bfab33e-f55e-4ccc-05c6-08d971454b8d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:47:48.4990
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 67JahmkBh+xlSHPwUvmcVDxQ1vBbKF0GOdcKWUTthT4riqXHbZ1UtbzL7M0Kf0QfUJf+qrHRgFtkuK5rnwBRVQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4445

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Instead of handling a single range set, that contains all the memory
> regions of all the BARs and ROM, have them per BAR.

Without looking at how you carry out this change - this look wrong (as
in: wasteful) to me. Despite ...

> This is in preparation of making non-identity mappings in p2m for the
> MMIOs/ROM.

... the need for this, every individual BAR is still contiguous in both
host and guest address spaces, so can be represented as a single
(start,end) tuple (or a pair thereof, to account for both host and guest
values). No need to use a rangeset for this.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:51:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:51:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180009.326491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFxe-00075n-1a; Mon, 06 Sep 2021 14:51:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180009.326491; Mon, 06 Sep 2021 14:51:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFxd-00075g-Uf; Mon, 06 Sep 2021 14:51:01 +0000
Received: by outflank-mailman (input) for mailman id 180009;
 Mon, 06 Sep 2021 14:51:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNFxc-00075W-Uj; Mon, 06 Sep 2021 14:51:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNFxc-0006FP-P6; Mon, 06 Sep 2021 14:51:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNFxc-0005Zf-Hv; Mon, 06 Sep 2021 14:51:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNFxc-0001of-HN; Mon, 06 Sep 2021 14:51:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Cxe8CFbjk/dvGVcyw1QMza334AQQ56t8EzbCmtg5fhU=; b=KGYImgVvFB0o7e8HRi6sngIiUv
	uopwabiJ0KBCc/bRMHSwWW45baaFOZPw3LohVGc1vdXw6hcFc8jHkpAO+EntxU2vRyEhYhF14wNGn
	3vDQNmBpWY1sBZsmHuPTFUNd3CpygIzQ80p3bi0pyAXr/17dzOXaJX5ByH/m1CY4T0b8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164861-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164861: regressions - FAIL
X-Osstest-Failures:
    ovmf:build-amd64-xsm:xen-build:fail:regression
    ovmf:build-amd64:xen-build:fail:regression
    ovmf:build-i386-xsm:xen-build:fail:regression
    ovmf:build-i386:xen-build:fail:regression
    ovmf:build-amd64-libvirt:build-check(1):blocked:nonblocking
    ovmf:build-i386-libvirt:build-check(1):blocked:nonblocking
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    ovmf=edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 14:51:00 +0000

flight 164861 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164861/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm               6 xen-build                fail REGR. vs. 164674
 build-amd64                   6 xen-build                fail REGR. vs. 164674
 build-i386-xsm                6 xen-build                fail REGR. vs. 164674
 build-i386                    6 xen-build                fail REGR. vs. 164674

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 build-i386-libvirt            1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a

version targeted for testing:
 ovmf                 edf8bc6d24c6f490c27de3144eee88f1ca40ce6a
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   68 attempts
Testing same since   164806  2021-09-04 09:41:21 Z    2 days   42 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              fail    
 build-i386-xsm                                               fail    
 build-amd64                                                  fail    
 build-i386                                                   fail    
 build-amd64-libvirt                                          blocked 
 build-i386-libvirt                                           blocked 
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 302 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:51:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180016.326506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFyE-0007tb-Bj; Mon, 06 Sep 2021 14:51:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180016.326506; Mon, 06 Sep 2021 14:51:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNFyE-0007tU-8l; Mon, 06 Sep 2021 14:51:38 +0000
Received: by outflank-mailman (input) for mailman id 180016;
 Mon, 06 Sep 2021 14:51:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNFyC-0007t6-DZ
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:51:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b1779ef4-d67c-425a-bb44-16905e5f7d68;
 Mon, 06 Sep 2021 14:51:35 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-AicrIZkDPiGpM8Q08cBCzQ-1; Mon, 06 Sep 2021 16:51:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 14:51:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:51:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0075.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 14:51:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1779ef4-d67c-425a-bb44-16905e5f7d68
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630939894;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w+e3TMfIQ3h7vzuGFjSUuI1jlMp8WLtee/hmiATvu5Y=;
	b=c2bG7a1cDwWAkNrSGeXra5lsf4/OmlcbnychRFXnLmrCMdrf4hlrJewR8DqWaXpgS880sj
	hule9+7EyVUmOxZzxtLPp5v2rZDgo4P6dRF6/JHIxSeRqSG6hZ0liwrZmpP1kKLjZgNHRo
	hywICSg21uIZTrfnDL9HibPd6eXracM=
X-MC-Unique: AicrIZkDPiGpM8Q08cBCzQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W9oiN8EiXENiLveTSwmZdfT24vfgz83PH9xzFjxOhbZwXRYCSaw/bJez+tY2VYVmbkhHZFpagBLYrMQd8r0wnkhE8YBfX1Rb5a8dFIVhQlW6h1rfy9AM7DNPcv1n4/EzBuKrjNyk2k44DhrtNBzN5AnK97+oysdCSrqlMrEb/y1ifHhs/Y/+J96O1GIKWBwUB5B6w0mHK8kDYgYmMi5dtdgnIKBJZwup9OuFK0ng2AQGWFHLsgMwofgDTTmfwmsx0I9m5w2vxzoV9Gcg5Tc8KAxuJ+UY83lk0oKn6f/Tkk9/e2p6Zf5pu16k0s44amDohPoX0fQlPxIsbIMacZoZ9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=w+e3TMfIQ3h7vzuGFjSUuI1jlMp8WLtee/hmiATvu5Y=;
 b=mRALFWuwLamhmvp5l5pH8Y6azbeGOcWmYtnlvx2fKnZYlWLk2OKAObp4NBIAgm3pZTOGJkw8jdnEMqJS174U6jGSJ5L/zg1/fy+h8T0XJT3wxxuZSMHB9Z5ITjLVBSkS4PY9kCqHOJbqrsnaAknyly2xEL7KIPPKG4DGawTnCUfUSlBrEm3gCwQtbtR14fVsEIFshCTmKqur5mBgK+IGGH2ITsm0lVAmbadsDAsmr/OGByxdmuchmzoVg9/0Jt3Nz6+wgbju1jOiXTkQvheBXA0noNnVeHdIybsD7Wyl28CJH+Rn/WfBSdo0TWwnM6ZyHFrJQqV9I5sMEk8ZFPioHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
Date: Mon, 6 Sep 2021 16:51:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-8-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0075.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 55f704b4-e29f-4c46-0d01-08d97145d101
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68625A2052A58D2841710113B3D29@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nLHflvI2dgXVcHIKnK5CUAPowX26T6oAB4oD/sgsA4VxGi90Z2QxXfY032buko6GNrKMj7TDVg7m1FlfJCFPEM4t2e40vyeZOqGor6dAHE/R9vHDEOX1EQkcrh4t6I026uy4M+Gp/z7/l9o363ceBb9ryXyuvDVwY3qOxk7kkHqPIj715IIDkz43tYRfYMyZAd5sxuU0yTkyb8dpBwd1lN0qAxFc38LTiH7EMjDTymXaDvC8Ri+H1bu3Hqqu5u+42EkNDspR26B/8hsGmt1C9L+u24lwq8Zqirb+/Ej4mmm9ILfsxURbjTHAyoKopCjZ/B/w6nKIJtcPb8A/8Zs62pfmiZrZDVh4sjNoYWeeqYWmllOzzbfuZj0g6J/cCD9mBsCn2I9qjkzdbyCO+/SXK8GMkBezTa+d9pJIxpfpjY1taIyUD99/tprQgcx7fDN4eHlaZ0luC1OoMYWFE96Jg0ytCGewSclZkmFt94WNPCSBcL99w7Y89WqxbMu6J1/RvIFKXJ+kbQkFfnWhkOsdJttyBqe8ByZGioVFLcGmtkO3UAQx1E2g8XQhqTsO1k551AZRB2zRgFDrls6w77UfAKvXAfEIaKQhAAQsefv+q7Jok6ZgyZSwH5gmX5rhD2lV0Zb/t5gSalsjyJoW6KxDMIcbNR6hkz0A+g6Vkd6brxYkZwjc1DQFCXWf15c+RtSCj7a4lkMI4tOz/9jpH60Wz6JtwqJ6/rHDUiVAjwc+0+c=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(346002)(39860400002)(366004)(376002)(186003)(31696002)(26005)(8936002)(956004)(2616005)(6916009)(66946007)(53546011)(36756003)(8676002)(38100700002)(4326008)(66556008)(5660300002)(7416002)(31686004)(16576012)(6486002)(316002)(478600001)(83380400001)(86362001)(2906002)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RFhBNGFpeXpBck5PNHJaV3B3czc1aWZ1aGZSalhHNk0rdC91dWZzOHNrcVRO?=
 =?utf-8?B?bU92UkhNbXV6dHVseW1vdlpIdENpT1VhQXZ1K0JtRG5nVmRvNHpReXNEVWI0?=
 =?utf-8?B?dG1icmsvNWxWZ0dCdExyOU5leU5KM0J5eEdXYnFuQXBHNHlWNk05aWs4U2p0?=
 =?utf-8?B?UEUwYmRYTm5DVEdNUFFydkp3V3pWbFFBTVdVZVE1dFdHRmdZL05rMWdIUDAv?=
 =?utf-8?B?cGxxMWtuMEkrTTF5U3hrbldKRXphNVRkVnZrdEpWdldUMzk2V0I5Z3lsMkZu?=
 =?utf-8?B?bk9Lci9vVXllQ2NaeTZtNjBkaE9LTGYrbEI5RVoxYVhyVGliYXRKNTRmcjRK?=
 =?utf-8?B?TTRjWUN1STNvSG9oUFByckhoU2I1MDN0OC9sQnF3NzdMckN4N1Y5dDVsWmox?=
 =?utf-8?B?ZmZoNVNtN0FuSFdib1k1OXg5WkV6VHZmUS8zRlA1b1I0cGxEblJrbk8rRG9p?=
 =?utf-8?B?UURwMlo4T0hEdmNYdEdaWlRiVWVkR0hGakNTb2FXY1RVUTZkeWQzclcwQ2Rq?=
 =?utf-8?B?ZTF4Qys5aU9BcS9aQ3VkQllHMXQxNmVBdHo1MTlVdEtBWDh6cWQvU045LzNy?=
 =?utf-8?B?NEp0S2JsU3B2ZmozVU9YT3hMbEtYeUU2Sm16ZWlHdzRtcXh4VEhDTTZlcEhE?=
 =?utf-8?B?NzY5cDhDZE1RZzNGNDRQc0FKZkJvWXRJTG1ISFZiWk1mSGlFY2xWMjcwRnNE?=
 =?utf-8?B?OUd3WHVGbEFBWXc4dkJjeUpLQmVGMzJFdnVoaHh1WllzY0t3UmlFY2I3bll3?=
 =?utf-8?B?SmhSME5helVOb0IwWThJTUtLY3VnbGl5QU1DQkRzSUJzc3pJS290NGpLSmNL?=
 =?utf-8?B?Qlg2bU4rSzZpeGU1c3ppbE9kVFo4b1QzQWQxL0NlN3M0eW9QVXhlUng2RE1i?=
 =?utf-8?B?ZFA5WFZ5TFdwbUUwZDE4K1BVUmNVdVNCbG02cTU4WS9pRGo1RENuS0ZRWHN5?=
 =?utf-8?B?RllEbGd3OTc4RjZtdXZpcm5zazd2REU0REZNYThOdytCRXZuOXhjZ3RoUnND?=
 =?utf-8?B?a1NEdjhNS05OM1dJQzVhK1g0UlNBa04xeVJ6OXRxam1NVnAyOGlhSzBaMDFq?=
 =?utf-8?B?UWRydjN4OWtnU09iNkhXVG9TcUlQSFVIVGZGTk9GbGxLYlZxKzNVM3Q0TkJa?=
 =?utf-8?B?aWRrakxCZ0pSbzBUS2tZd25oWTdqYnFQTFJTYys1UklIalNxdkczT3hJOXNH?=
 =?utf-8?B?Y3BkVFVSUk9Va1VDcXJBRzJpQ3Y2bzRwL3lXLyttdDU5SUZiR3FXZmMyNUJR?=
 =?utf-8?B?WmpseUlXc3M0eTJQVXcxTE1DVW00aW9iQmN1Slh2RGZabForYkJ0Rm9oR2ZQ?=
 =?utf-8?B?TlFodHpUb2s5MHBqazliSXZheWNiMDRIK2h1UGJlaGw0UTJjOXRPd25ReVVt?=
 =?utf-8?B?d3lKSTB0NXNuS2FrdTRTTWhCQlVIWGtnQXcwVHRPdzYza0dxaVpFUjlDclBG?=
 =?utf-8?B?eXc4bDllMllGWWJtalYya3dPSi9SeExROTEyN1NyWjBxS2FMaVpwcjNwUUJV?=
 =?utf-8?B?QklnbVIyT0ZQMW83WTc3d2JVaWttamMrRjhoWjV1YzA5cER5RGtkVnV6SCsx?=
 =?utf-8?B?WDl5bXA2N3p2djQzSFJVcVVOVGhtMFRBTmxYaWVsMjAxbFJmQUt4azdyaHJj?=
 =?utf-8?B?aU41clRNZ2I1MEF2RHAvK0FCZHlJWGdtd21uWTRhYzNTT0gwclErNThPY2Jj?=
 =?utf-8?B?WlkvY1k2UXV0MHNvMllhVmtQVzR3c2daK2UvaGsyREVaZ2ZmbDRkVG9UNFR1?=
 =?utf-8?Q?wpsG9PmYRfFLtNOmU4ZrEH4MbazGv0jisGYC750?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 55f704b4-e29f-4c46-0d01-08d97145d101
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:51:32.3744
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dRNhdyZN6gjMGRBq5g4cPSOU8ktaGjC96t8cH2x1o9UAqf7xVOjbOrqFZLEYMzWgKlvzJfWcpvx/birmV0jwbg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> @@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>                       unsigned long *c)
>  {
>      const struct map_data *map = data;
> +    gfn_t start_gfn;
>      int rc;
>  
>      for ( ; ; )
>      {
>          unsigned long size = e - s + 1;
>  
> +        /*
> +         * Any BAR may have holes in its memory we want to map, e.g.
> +         * we don't want to map MSI regions which may be a part of that BAR,
> +         * e.g. when a single BAR is used for both MMIO and MSI.
> +         * In this case MSI regions are subtracted from the mapping, but
> +         * map->start_gfn still points to the very beginning of the BAR.
> +         * So if there is a hole present then we need to adjust start_gfn
> +         * to reflect the fact of that substraction.
> +         */
> +        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));

I may be missing something, but don't you need to adjust "size" then
as well? And don't you need to account for the "hole" not being at
the start? (As an aside - do you mean "MSI-X regions" everywhere you
say just "MSI" above?)

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:55:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:55:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180022.326517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNG25-00009r-Tn; Mon, 06 Sep 2021 14:55:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180022.326517; Mon, 06 Sep 2021 14:55:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNG25-00009h-QI; Mon, 06 Sep 2021 14:55:37 +0000
Received: by outflank-mailman (input) for mailman id 180022;
 Mon, 06 Sep 2021 14:55:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNG24-00009b-6L
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:55:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7d4867c2-0f22-11ec-b0a9-12813bfff9fa;
 Mon, 06 Sep 2021 14:55:35 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2113.outbound.protection.outlook.com [104.47.18.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-EAfCa_UaPdGfC6z9bz9vUQ-1; Mon, 06 Sep 2021 16:55:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4845.eurprd04.prod.outlook.com (2603:10a6:803:51::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Mon, 6 Sep
 2021 14:55:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:55:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0219.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17 via Frontend Transport; Mon, 6 Sep 2021 14:55:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d4867c2-0f22-11ec-b0a9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630940134;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Bv9XkP8auOWVn7yNSqj8qEom+vUYrV73C8AOIGJpxcM=;
	b=j3d2CkaD7MYhW3K+gjo7PPim+vsUgCcXVXWYz06JBY+v/5zxpz+pchuHNVAqHuo2/W3YsM
	aQbitXNgVg+4j3VnqQEtmyt5oRKSrgCTm+7jV/n4LJ7E4mqsvn49qfV0mKqhJA6cvoRI3+
	Pqdltzxge4AIjDjJoatZigs4TbgZrg4=
X-MC-Unique: EAfCa_UaPdGfC6z9bz9vUQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kOZU9rIc9MSfr+JyiiMZZdYx9pxdnDaE8vCTVMIoxWw5O662f5akmDI+1+obqGcpPFzHGjMol5prvdlfivynaB4a9bZxbFNYQI8IKk+R/UsFDCVzD+BxBc87JdLFnu9+G9/vGlAraaNbWq+0lsPfKgntM4GEkNNTtMW1hdZP35I7R27MZE9urYxUnH3m6P78R0LzMXSglO2vlHy0eEUe71ks7XbAAxr5nxni7nGS1h/5dCXd26UQ/xd5jARRw2TzBCAZ2iS0/l+CbrouIzyHP6BalilaCMwQ9m9WwhUDmNG76/VWlC3uxQFhuGuHfTlIey7rzZl3paE5VpTXmUty6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Bv9XkP8auOWVn7yNSqj8qEom+vUYrV73C8AOIGJpxcM=;
 b=TP36ndniocItnaNg5dEvChGUUqU5cgTfXS9RWxHLBmREp76Gh+lMZBHYeD0ZrDUjqYzRopFVEum8cmS7rEhuCT3S1Qf5Zeqf0o8Gi2QzEm86EWs2rbCWotYRLvULIzonL4V/4xnMnlZ45fgRzVVJwcduhlaluvJFq9L6idEJymPOqBlo3G4nPzZ2DzMNIdxM4j68mXPW6EyElD7LFQlwJa53mUBu8bzcqvfFszmvonG3+PcQMsXxQXQU8t17au52RrMGocVEBCnq4qqxe8u/UqjyPUaT8qj3BMs2YYKexxK914/ImHe50WW/lgBMJ2NJunOW3ktFKysMQWMW+kRqjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
Date: Mon, 6 Sep 2021 16:55:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-9-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0219.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 042c7078-9ffb-435b-057b-08d971465f7b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4845:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB48454FEE8A389CBD15FD7798B3D29@VI1PR04MB4845.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1284;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DP6pgYsCEek2J4ubnvlbLXqs2kLl00f0vSO4HSj1pBp3qfNcKiIKpHvd72nIhKd+FmQfHKnrBK9p1IxK/mSHtpNsMdjOm8rBa98oPQ9FkJp9+TVAtZmu6oxkF7iiP5DGdpKP4MAfV1WAEr0di9KQ2Yo3KcUhUwcbcLoh9Yp87QcdQrdkO7sBJHmo3SD2VAypjelmydcDeHtR+VVGUAxOV/aOwIa1uOeeWh9of8sfViDUGykLcJsSa7EAKGym7Q7EI4gZKxR/sygtXppU3Z4oBIUzIpQ0D4hQBkdZ9bayUQOVmY3J3imIlOjFW+c0ioTasm4Y7dCLSy2eNSsMVhOVEoctUIOCTD08g3OEKu5y4xmK6jsGv2D80t3wL4r8PZJzrsbpQM/lpY/1GKFzGyCbmBsA9O7EZPPC3KDegQ5zqEI5CjoN4Xso7hOU9Z7/vc3JxcSIKqcLw7eEb/WgdJMNfrBarx+h6yWhWB6fsQ9WYld1zFMIlgFcxqliN/isjCxahOq8o64spyy35unbYUJzxiwNjtljaRORNX1LxJnIJPu/FWiJ9edZ5tOv6ElDTYbak67Bh4rwrvb1N7UtSQar5/tQB5RZ9nP/8Pw+aLB5YnmJuS/h9iD2HajG5mqTchIgS5+dITiLBCrHzx/q8blEAeV8hM06xNYvfQ+fIvwsq20fHIM6aC07cmiNhwdy+6aR7m/OG4M/0j2yBUbT7Rs0aLG9i94U2vhRzXMHRUnraUU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(4326008)(31686004)(7416002)(186003)(6486002)(83380400001)(26005)(5660300002)(2616005)(508600001)(8676002)(6916009)(53546011)(66556008)(66476007)(31696002)(66946007)(8936002)(36756003)(2906002)(16576012)(86362001)(316002)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VVlGcGI2LzhDdE80dHBOQk01U1Z4WDQ3OXR1d2Yrc0VCeTY2bCs5NWpCYXBV?=
 =?utf-8?B?KzJqT1duMlJhRExEcHN5cnVKcE45Y09yL3puLy9OQUFsMU9kMHJ0b21heDlF?=
 =?utf-8?B?UDJwMFpPVmRja3RDalA5SE9xaGh4aDVzZythRzZ6UkdicG9jRkJzNGxsekdE?=
 =?utf-8?B?eTVFL0Y2ZmE2ajVzQ2Zlb0toQnZheDcxbkNEeDZZR3hVYWwxSHc1cmtKZ1lE?=
 =?utf-8?B?eFFGT1ZrZU9LKzczc3FDYVZlU25OdTJTcTZhckJMMGY1d1hZZUtFUFNzSWVZ?=
 =?utf-8?B?cUd1bmNDN3lrTGlDQktvNVdWTXdFQmdUMHZBZThuMDcvR01ZV09lbjVTRnhG?=
 =?utf-8?B?cXFheEdSejB1eFgzZExsWEFJQjN6cmZjK2xBSFcvMmtxWjR1SWdxRVhsNGhK?=
 =?utf-8?B?UXpmVGIySXE4ajZKcGk3R0didnF2R2FPc1JuTUlmelRpa1hEd0cyeE0zeHpV?=
 =?utf-8?B?MDVXc0ZPRzdXbXJpbzRHYjJVVy9rYXBiZXV0Tk51Yy9sOUpWWnFpdUZzaGQ1?=
 =?utf-8?B?bGxHUVhnVzFMd1hOd1BOZUg1Myszd0dpUE9zcmZ4eGF3VVVzNExRdUIyLzJw?=
 =?utf-8?B?TkNlQzZkaEdjRFdUL09sMGFoby9WZnp5enR5eXlPTC9UcTRkaTU1eVhoZUh5?=
 =?utf-8?B?ckpHcVFYUFkwblhicEd0ZlkxdklSTDdBMHlWaU8vZ1h0VHZ0a0NqMnR2aTI5?=
 =?utf-8?B?bE1uajdmZ1dKS1dEKzJXbDBJbVlKZXdWRzRzaStSd0REVG9JcndRM2p2bTZn?=
 =?utf-8?B?ZUpkTlV1K3hZSWF3U1p2V2hjSjJsUzh6NGF6VzRiSkh0eVgzNFpWdnRsYmxq?=
 =?utf-8?B?ZldXaGFjcnVzekpudm5rdzNCTGlGOFMzK0tlV0o3TFJDN2QxUk1LUm50L2Z6?=
 =?utf-8?B?d21tb1ljTVR4QUZqR0pwc1NYVHUrZk0zVkVqOEFXSjF6S3EwWFU2c3RNRTcr?=
 =?utf-8?B?MzJVSXVrcEFkcXBsNzV1Q2VRcXF4WE5HNFI4RzZqZnJ1djlrMFE5YkFGTVVs?=
 =?utf-8?B?TW1tQUVycTVFNHRXSjR5d0JPK2FsbldaQ2d2OVgyUDY5Z1ZKNnpPYmJXN0cx?=
 =?utf-8?B?QmZoYklzQ0lGYUhwckZNK01PNXlkQ3dQaXZReVZMUXo2aE43QitiNFVsTmV1?=
 =?utf-8?B?ekx4RXFIVHNpS0ptSUNLMVZnc1EyUGJqMzd4d2N4M1E1SmpqRVlVV3FBN1di?=
 =?utf-8?B?RTNIbndwMlE1akpYNTI3Q0VsTVZ2YmY2dEw2bE9iSFlMVzVMOUljS1hFS1kz?=
 =?utf-8?B?a1c2dDNxOE5yUktsM3U2a3BCVzVYNGhrSldSL1R3aGJzWWROTkJKaXE5RWVj?=
 =?utf-8?B?R3VqUVBNMnRUQVlLTU4zTGoxeExZbHBVMXBzMlFRSWxjb1B5a2NxV3ZwR2Q2?=
 =?utf-8?B?cXBWS2EyTXBNQk1IK3M3NE9lVzZIaTRvVjl6QU1rSUxCVjIyQWVsdjgycDNr?=
 =?utf-8?B?cnRXQS9XbVNydVNoQ3JNNjF5aXUva1YzRjk5S1lIQmVVTFFnUk04b1ZvVm9I?=
 =?utf-8?B?WmdnaTBSNURRbXYyV1Fka0JPSzQvalVRM0VZdS9UZE9va1FiWTVSM2VVVzR0?=
 =?utf-8?B?d1B6d29DRlZwTC8rejdzRGpqRDdqU2pSVmIxMmxqWWMzVytDTXVpTXlXTmxB?=
 =?utf-8?B?REc1RTc1YWNFdUFOakMxeXJ0bnpkRmphbmdNeHhQeGJzUEJFaGdxQllMdU5x?=
 =?utf-8?B?WEptZmVjYlFhYU1rZENzdzQxUzQra1c3bGgyZGpiZThEamI0eGN6eEhsbE5H?=
 =?utf-8?Q?/x7L4ZEuuVKQi83GvhbeNQ0m0596ot3hjIg9kR3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 042c7078-9ffb-435b-057b-08d971465f7b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:55:31.4132
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LQD1tIYNncaNkT3WVx6usSciRqbrmX7p2hVGsGbIzgD42Rc4cBqUrhnkjJJ8CrymCqdulIk3fGOHGsSmWCjyIw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4845

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -811,6 +811,16 @@ int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
>          gprintk(XENLOG_ERR,
>              "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
>              d->domain_id);
> +
> +    /*
> +     * Reset the command register: it is possible that when passing
> +     * through a PCI device its memory decoding bits in the command
> +     * register are already set. Thus, a guest OS may not write to the
> +     * command register to update memory decoding, so guest mappings
> +     * (guest's view of the BARs) are left not updated.
> +     */
> +    pci_conf_write16(pdev->sbdf, PCI_COMMAND, 0);

Can you really blindly write 0 here? What about bits that have to be
under host control, e.g. INTX_DISABLE? I can see that you may want to
hand off with I/O and memory decoding off and bus mastering disabled,
but for every other bit (including reserved ones) I'd expect separate
justification (in the commit message).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 14:57:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 14:57:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180028.326528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNG3o-0000oU-9q; Mon, 06 Sep 2021 14:57:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180028.326528; Mon, 06 Sep 2021 14:57:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNG3o-0000oN-6o; Mon, 06 Sep 2021 14:57:24 +0000
Received: by outflank-mailman (input) for mailman id 180028;
 Mon, 06 Sep 2021 14:57:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNG3n-0000oH-56
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 14:57:23 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bd050988-0f22-11ec-b0a9-12813bfff9fa;
 Mon, 06 Sep 2021 14:57:22 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-xl1GQZ_MNoeOQNhEmmZ5jA-2; Mon, 06 Sep 2021 16:57:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5744.eurprd04.prod.outlook.com (2603:10a6:803:e4::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Mon, 6 Sep
 2021 14:57:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 14:57:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0055.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22 via Frontend Transport; Mon, 6 Sep 2021 14:57:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd050988-0f22-11ec-b0a9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630940241;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CNSM9aGdDV9WVXdpSczpiqZ5PzSE+iG54YbevdVOLVA=;
	b=Aj6JQBDKTboW+LnBDQ1PezGm9DKpiSLxcm/cqghdiPBCgbvjO9EHG4bVvaCtxR6MOyus1d
	0MbDiJPnu7NRLmZtpRykwAN02dxbltQ7qcaarQ/RcLiMGsqqk/vfVhKjduXFuJwPtHjm/X
	42DSjQwP1j8weRLCQHnqvKtCIFxOjMQ=
X-MC-Unique: xl1GQZ_MNoeOQNhEmmZ5jA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eoV6waeUU42V5jrePYS3N0ueKaPI4dv65AXSjYxS2ZhBu+8hiHNOY253nED/6BzgH2P+zfCZz+UgCDN4KSoFLRRX5c+JgscgqR6p8cL0ch9nnM4D9h9VstMk8Hcxs3bxk9n9RMsnKr2EdPCYCQSdwQkWctm/s77td4EeAgoTwYhLFa1T8lw9HVkZoFv9SVr6JvaCBV6gb7vQsji/1iG4j5p0d8u0N3lWngN4neynAk0E2f2Fqih7lpNRH9PYPTxPqWI2vjRpyaLWAsUoHga/ngsMyyaJnov7t5nQ7erL3DHxVkMpyO9d7w2eylCeI5Drf2Efcc3ShL4lkp/AvJJlpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CNSM9aGdDV9WVXdpSczpiqZ5PzSE+iG54YbevdVOLVA=;
 b=Z1ud5JtNboivYcmO8CltTTEyQOyClvUyMP8C8sCUNkUSdZtYNAd2u4MJ9DY0MtAWkHUG6OFBC5X6+NBC+dSfbeD0fSywd/BiHhK/3aW979Oa2ESbWYJZ0Tp3Gxxdis4vKpxu1NMKOiLXrlwdF/akoP3p8B+iOavDkHUjQ9V17hIKNUwR6TA6wk7biimpUrZBQ/i5+Qr8By6dhTpxvz/0do95RIKfy7NJtRlIEyu9K2XEuDnQ5HfbHgCdC6ik42loGPKw/AuDhOZCH56evbGGyV68NJYbP/X7Fyzgnksx+vvJ8csYUKYw6+zXw6wo83LVYmJeA3Gf+b41G/JGVW6jcw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 9/9] vpci/header: Use pdev's domain instead of vCPU
To: Oleksandr Andrushchenko <andr2000@gmail.com>, rahul.singh@arm.com
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-10-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <214a8300-e818-321c-7618-185c56684db0@suse.com>
Date: Mon, 6 Sep 2021 16:57:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903100831.177748-10-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0055.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ddbb2616-ce5b-4ac1-37b7-08d971469f4c
X-MS-TrafficTypeDiagnostic: VI1PR04MB5744:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5744103CEA4B3172531B51B4B3D29@VI1PR04MB5744.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iCKw+/27HlMB6LnhDHuHC3dHURHfAdhZDdl9u0/8aObUrgGozNPCGLMrjMCKlkRZ+mb16B1Sz5Y4ZtHHBWLF/Em7kGuF1rpXaE9hUaYrMHctw+bY9AXRadVxGw2PioUhAC7PsFy6Nh4pSAZt7xGTyMoC0wJIvYdXD8cGTHGxBGSPciT1F9tWXVdaUF0N/zqjOhzl9QBs8pjWT2xnQRhRbZiUQMdEOHFMXUuTxywqI7LoDTftTgiF0Uc27t+ip1XOIHaGUsfIDS5mQeMnAzf6koRRzDIb4Txtol+qPIIvD4TkJpXQ/HUCYklCaAzcFOC5HdwB0u6fjM14jND14yNJP65Mo4yrQqhsWpdqZTOMDHaK0ox4pZf2+6iUKDP6j0JifZooLKJLhXYr03K5phNdymKouIHJDmoKobeVDEpvQNDq+xS06vtt9cqGEhO2No4+AWhRWEt8jU+f2crCaDaPOmMSIVE2PPyd76s4gLrmGMCbFLrKlnrZFUnjFAs3sBDwxTUkDAwWTWpWBW1uAGBVFSwe2LO5JBQnR/cSNwcz6EAcOOhJJnrzi/nL+SGzaUqWuK0bYXHqhscy+T20i8B7Ts21ni89RiGV8l9/gd3V4ptFC/NyGD/FoJajzERCyP5qGYGZZZgKs/a2m9zshVaM7KmmmX4TAeLSNG3ln0AxeWqo7Tj486q3TkaIF2nSqJZBicoZYkXzf40gk9+6bd3E0Cz0TQRY4KcT0cAtGer37mY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(346002)(366004)(396003)(16576012)(5660300002)(186003)(478600001)(316002)(86362001)(7416002)(36756003)(38100700002)(4326008)(6486002)(8936002)(8676002)(2616005)(956004)(31686004)(66556008)(558084003)(2906002)(26005)(53546011)(66476007)(66946007)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R2QwdHBCdnlzeHJpRHdNWktsU2tzUDA5aDZ2engxT0Nja2c3azI0NEJ3c05K?=
 =?utf-8?B?NFVmU0pCL2UxOTZKbG9zcGJBR2VxOU5jaWV6SlhQaEJpOE5vT1o1akRuaHNq?=
 =?utf-8?B?djZnZ2ZXUXd4dnNxcDZaL05nbUIvczVvUU1CYWtxYW9MMjBFUkFhWGRuMlZs?=
 =?utf-8?B?N05raDl4bk5OYzFtTlZsVWtKTHFSRWt2UUVCS21LQ3FuQTFOdUpOV1N2Q1JM?=
 =?utf-8?B?SS9xTXZHT1NLVTYwbys0NURiNU13aks3dEhDVTZDbU5PWXdKZWlnWnhydlht?=
 =?utf-8?B?QUhhRmpDcDVUUjlsdEhPRit3SytldW1sMjkyV3Q3bUI3azBud21TckpvU3JV?=
 =?utf-8?B?R3lDUk1ia3pvTms0ZEZDTDZQT0Nqa21ISWhPMXRuUDVwSkRmS3lFc0NzQUQ0?=
 =?utf-8?B?RVZLTkxST1NMSWlRM2lCWUZvdm5tNU9ibFF0WkJoNTJMQWlGSTZnNlZGdjVO?=
 =?utf-8?B?bExmTzdWUUJnTVpWZEhhNlU3RU9MTnMva1dvbW1JZTFDSEc1bVdGK1dEOVJl?=
 =?utf-8?B?Y05pOEU3aEFVV0xxM1ZLRVlQK2lhaXJ5aWxvTFhXelQ0TzBiZC9UNXMweHlL?=
 =?utf-8?B?ZVBqYnFMNHpBQUYyUWp4cExkWm5vcURKWGVNWllGR2NZWTg2dDF1K0xBME91?=
 =?utf-8?B?Z3JwbnJGMTRsU3EyWVNuYkpsd2pSaTBUUEZhTHVQdlVTOUo4L3BkV3JlcWky?=
 =?utf-8?B?Yk9Nc1dOSjFTR1N2ZUplNHYxTFlJWW9CSEFvclNnRSsxZEdGVGlCTXkzZE5n?=
 =?utf-8?B?ckRPdTBaR2dhYXhqSlJaV0c4ZGV5RHMzallEeHIrcGpZbUoyOWRmdE82SzdO?=
 =?utf-8?B?ZnJZa1ZFc0hEZFcrRllGU2NzS3Z3aVFIZlJNSFUrTWcxcTIyOERTeDVQU1pE?=
 =?utf-8?B?OHhFUm5PdkRXZi9FRjRod3pjdC9oYnM1Zm5rWC9IaXovM1RwWUZlODZjZVcz?=
 =?utf-8?B?WWxMdEhNMHl3VUNWWFVXZVo2Z1JIVnJIVnhKTUk3ZmNkTlFFNEhhdkowYW1i?=
 =?utf-8?B?eTdQR25SbWhwc3JOMVlsTFY2TDZxTTAvS3cxUTNhc2FBVVNqQWpEbHpLUEFk?=
 =?utf-8?B?OUxmSm5zMjZnc1JpMFZ4KzhMRG93UTFLbzhjQlQwOXVKRkF2SHpGbDFxMVZZ?=
 =?utf-8?B?cjBUZXptb05vS1M5aGYvbGh1MW5vd05xMGhTNGwwZnFtYW9HYkNnR0VjdE5L?=
 =?utf-8?B?c3BxZkp3LzBURURBVHVLV1QrWVhWWUQ4VWJjRUEvSWZIaWJQMzl4VUJKT240?=
 =?utf-8?B?ZWtlM0xTWUxmQzJSN2NKQ3JSYm9wMTdkMGN1VjdxS0xLbkwwQ1hTMDI3bkNM?=
 =?utf-8?B?aTB0UWtqT3IvTzRqc1lXZko5NXpvRkZSRzZvVHN0UFBxdEs5bjFsdHIxL1FT?=
 =?utf-8?B?aU1EaTc1Y1JsQUdZU3NRSzVNM3RUOVcwdkNlZHdRa3ZYbkw5aERwYXMxT1pp?=
 =?utf-8?B?TTRJdlZvUk9zaG1Qek54bE1XNUExUFZ1SjZ0Umg0bEUyTGgrSzhFS2t2dTdS?=
 =?utf-8?B?OGQvdFJQMkRvRklMcnJkSktuZEtqbHJqUC94alpGL3N4VkpKa3J3V2xGVGVD?=
 =?utf-8?B?UUV1MkF4N2ZvSjhXV28rc3BWSThpVTJmb3p3SitVYVZHOFUzeForSVc1Y0w3?=
 =?utf-8?B?REpLSXRBaHVLZURlYXkwOUpVVVpocUtRcFpnMGZrQUF6VTVQcnhaMjdNN0M3?=
 =?utf-8?B?MkFuZzZHWkw2anI4KzlVL0FFdlpoYkZ4dWx2UGcrM3kwaVRQY0N5a214UnBu?=
 =?utf-8?Q?vuBAxG3ai6bcByaRbG6P40tfswOobfW97Mn4aRL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ddbb2616-ce5b-4ac1-37b7-08d971469f4c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:57:18.4797
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: G6/d6yR/MC4L8o+bNxrgNtnQksFbKVdI62Rp8R0sAuZ484tZAMqO0/8L64eqld/PvY88MCg2RbAJC0sR0rbxHw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5744

On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Fixes: 9c244fdef7e7 ("vpci: add header handlers")

In which way is that original change broken? The title doesn't
clarify this, and the description is empty ...

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:04:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:04:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180035.326538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGAv-0002Zi-6X; Mon, 06 Sep 2021 15:04:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180035.326538; Mon, 06 Sep 2021 15:04:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGAv-0002Zb-3G; Mon, 06 Sep 2021 15:04:45 +0000
Received: by outflank-mailman (input) for mailman id 180035;
 Mon, 06 Sep 2021 15:04:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNGAt-0002ZT-Cg
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:04:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 97c3c93b-02ee-4ad9-9075-52bebe7f9dd2;
 Mon, 06 Sep 2021 15:04:42 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-pzqLZhm0Ora2kiQ_tWtLnQ-1; Mon, 06 Sep 2021 17:04:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6480.eurprd04.prod.outlook.com (2603:10a6:803:11d::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 15:04:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 15:04:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0226.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 15:04:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97c3c93b-02ee-4ad9-9075-52bebe7f9dd2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630940681;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=bNdZ6Cx7slUiIr3muINtSAbKCyL8zuOoHJm+tONX/9I=;
	b=PeiVHg+fd/Fkex+5eLSYxCmkND8DYrY/FS03vOhLrn7nKw02ZUdSqIQZYQBn5/XGIMGXCm
	B7F43D1CDhK687dWexGsvQDSpgGaIlkeWvZjj4T7B9vbX9OkclrEyLBDdsS7FvXdzMobQa
	5/MkWF0T3tg8CEJe5zoK0UmRwkvIHWQ=
X-MC-Unique: pzqLZhm0Ora2kiQ_tWtLnQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Fd8VZxNT/agSyYAo64p3ultBMpBaZGyDLSDWifE1TJ+DLW/YJspKcqDGtxD47yIWu01jNDMd8nODMoZIYDv8zIklyTPEjq/wIN9pOPhzTmNCLiBKLBbrMAY6Bw5m8IUzq7xF/Z/nacR0356TKarZ9SuNL9/uJAbDlnE2qi1VtvApQoCLU2Q/Po0uXZEl5K4gax/qvZbXOc4n7zEEJpdyxGS105QSAPfnydVSM0WL+zWekaUd0o/wBZrgQ8EPN0BXqLoxgWBVy5p81Hoqgb76yN17AfTLq0t7wkCBVYmPLIp3pnfpw0yJHH7cZscvMJQTcwcrN8KEjMa1InfnT/hUjQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bNdZ6Cx7slUiIr3muINtSAbKCyL8zuOoHJm+tONX/9I=;
 b=CKssZ+W7EUToHwa/ywYAUUs/vQJIDcaza7LKbvqn8ZHhSZo7J8sxRZtJzKRZAUfXy3JoWDC0+F75pCADtL7SmawcKmCPgdRMCwspfi0RXrhHb8C6wIqRw91OuUTMt97I7IuZSCjlf/Os2duTsVifJFhlzMxZSMrleMW197Ng8NyMbf4voR3Ie+wY0WoEpxpplNLLydHPaCajr2V9Y0D05U0aZ7le5ncBRxxigo+PwQwYH6t6PwuwPJBiv8Gdm07kgumNI1P1ecTX/IoO0TDRCDwp6kSPSc8aE6c2XmRB9cj52TSd7uA9a4MFe/DnCyQYc7CGEjzF5RmsfRMCMIKuig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v1 1/2] x86/cpuid: Expose NullSelectorClearsBase CPUID bit
 to guests
To: Jane Malalane <jane.malalane@citrix.com>
Cc: Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Pu Wen <puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <239f7624f3ac3793993dc045d451079aa555b90d.1630929059.git.jane.malalane@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <203359a1-1f76-70f1-e314-0c65c4413361@suse.com>
Date: Mon, 6 Sep 2021 17:04:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <239f7624f3ac3793993dc045d451079aa555b90d.1630929059.git.jane.malalane@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0226.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 08b92ff5-2072-44f9-4d29-08d97147a53b
X-MS-TrafficTypeDiagnostic: VE1PR04MB6480:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6480F9185CC8575B060E6E21B3D29@VE1PR04MB6480.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jcSSGkWp18dNTu5c+s7GR1YSXcU7SpAu3LHqh3pqF2NMQzWxm2nuVeyh7nxpfhT/VMJF20QtxoJJV4+IwenZuzUkPuJdfm5nhVzJaD4m6ejaxUAsxsdkhLTi5BtTZtYVqBJ6rReNsEtMtuI1GyUSBojAuqm+w6EOYP9WVdahJbkwqMubPC3c6SfNyuX/5uOTdsv00JI1BB/q9d/yrh3x3o4duAZk6FkX7RbB4ohJg5nhbzSE8j5e6cfGq/K8b9ICEGmSYo5BQDQl9u6BQp8waQ1MQL3Vg22vixnobpNnfN1LPgkemkzTf7mWFBe7LobuaFucrjF5TzP70A7F38U9jVI4RPJFZWTn89BRN62rp4um1ZgNEul3MwFkduJke8pIgLEK6LNj8Zq6x4Rj3ACmryoRIP6Oqvsx0ZQgWuL+UW+EOnrbu/L5lSvshmVIJHSQ3hExgUKl/Lxo3GqqXfJ4S7g7Ip8sXeVhB4cg9ZVPjB189JApu5B9gDiJysDDyvtEPB4T9h/mmdWPS4XZvMT9qUygWjs82m3IH/4ZLDhzdf8iAYxNFBqgg/mCD5/HqAJ+2Xz2MHsIxE2X7zu/6Kq9kkSRsEX0qCJVPY/RIiuJHyW/Y5y9oehXggj769TBsE3Sp/6h+ihnkcfRnHIxYa/fufpmCY1X/1ZI/jbS2mpmk/q9WrAW3mdTTDzBOkIt1WWTeTVPK1ML+RoTvQm9gFdwMncMqBg1gKuRo6F8+ALaJdc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(346002)(376002)(39860400002)(136003)(36756003)(8936002)(8676002)(16576012)(5660300002)(31696002)(38100700002)(54906003)(53546011)(66476007)(4744005)(66556008)(6486002)(86362001)(26005)(186003)(316002)(6916009)(956004)(4326008)(31686004)(2906002)(83380400001)(2616005)(66946007)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?czRiSmpkWC9PQm41WlZTUzlpR3h5aEpiZVlTUDdhRDhwTG4zdVFKUVppejVq?=
 =?utf-8?B?TDduaVRQbHJWVzBEa2hta0VodmV5RTZkc01jUFZMSmxnU1RmS3NFZmwrOENv?=
 =?utf-8?B?QWVacDNxUUFKTTFTbzhPM1c1MTNJak5IRVpoUDl2UTJnY0ZjTzQ3MXozTDlT?=
 =?utf-8?B?RWdqcWRGVFJYOWw5Tk1nUWo2bXVsd3ZiODlwU3VvcktwUkc0OFVLMEtWbDNS?=
 =?utf-8?B?ZVVDT2dwd202RThJdGh0SmpYWGxIb0VFM0V3cEcxdlhjRUNaamxvMlVpcytp?=
 =?utf-8?B?RmszTGpMb0ZpcFVzVnJSSm1ENERrZEtZMTJCSk9vVXJZQzNocWxUTkFSL3h1?=
 =?utf-8?B?LzBhSWZ2bHVFbDJlSTV5Znh0YUlqU1BTdllXdkNWM0QwdG93dDRwSjhSNmR5?=
 =?utf-8?B?M3lJeUQwMjdCSDFBWTVQWDgzWi8wZkkyQlo3VUlhMjdlcnQrRm45cE1QWFNn?=
 =?utf-8?B?OTJXMGVoMlBOTWVDUWF6SlFOcWVVTHU1eGNaMTZyYmI3S0cwaUJGR1lscjZW?=
 =?utf-8?B?RWIxVkxJazJVUDZWWVEzOTF6aTBKNEdvdmFwL1dSYWhyRGI0TWhDSXB2Q1FT?=
 =?utf-8?B?bmFzak91TW5wL1BSdUhCODVRZUVTS0NUU2xUeFlSSFBFeU9pQWlhaGU2SWNR?=
 =?utf-8?B?L1RyVTBEOGx0SzFFVFkxNytCRFlZOWRvYlE3MmE3N2wyT0J6NFloUnM2dk5u?=
 =?utf-8?B?WEE1NW9PUVhHQXdvRTNla2ZVUkRPTDlBaTBzSmxyT3V0K3U3WkI5eGZEWVAz?=
 =?utf-8?B?cnJ1aGpoNnJKVUNRK3Nvblh0UmR1Tjc5QW1LbHcxdGtIeW94LzJRM1drWlVN?=
 =?utf-8?B?SWxNM0thQVA5Ny9leHFUVEFaeFNVY3BLTUZPUi84UG5pYStGbzBlSUswV3p2?=
 =?utf-8?B?bWJqUTllbHd3c3FYSmxqRW16cEZyZ1o4bjFKNm54b0dNS0NwVDdOT1BuYVdN?=
 =?utf-8?B?L1ZUWlcycDliRUNiY1BTU05uVVVBYSsveEZvN1dKRTFuRnZRUGpkdmRicURO?=
 =?utf-8?B?dFN0bW5kdGNEN2pIN0I1RDNKL1NEYmhuNXN2aWlKWW95aGlKNVlUbk9OaFF6?=
 =?utf-8?B?dTdBZTNoNVJqeXVicXZYZFJFZ2FYN05OWG9rZjNLVDQrblVYYWI5aFl5Smcx?=
 =?utf-8?B?aVVZYmhOVmtIQ2g2ZHc3Y3dZSVVUeHVmUit2ZVZSYk5SUVFJYzJHQmVTNHhz?=
 =?utf-8?B?STVHamFtVUlEWDllUm16dkVGY25iem1YWHhsYk5lenhIVkRxT056OVFzcXVJ?=
 =?utf-8?B?UU9SQWtyWHF2WksvZVlXSHdMaGVoaHMzS0FIMU9kdXdFMUpEUk9KbVdZcllq?=
 =?utf-8?B?NFB3RU8yYzFuTVQ5R3ZQbm9rYjB3NkgvMGlXbmtiNkZjNE4vcmJ1dERHSkJ6?=
 =?utf-8?B?c0pjbHR4OTg4eTR2bXJ0OTJ3NGVTVXFmTVZTWTV1YTZuSVNRQjFoY1RkWlFH?=
 =?utf-8?B?VFJEVUdSMDc4VThwSUxWVTBib0ZvdGE1NTNtaDdmZUF2TkYyZnN3Yko5NUl1?=
 =?utf-8?B?UzJwY255ZEVyZU1pVTFTanBrNm1oWWt3dmxNUlR3YzNHQ1lOTlc5UWpPM1dw?=
 =?utf-8?B?Y0RyU1dHaHRvTzZNdFhzNDlNbkY4d2NCUHo5cUh6elRHcXRwTk5mWGd2RTNj?=
 =?utf-8?B?M2Ntc0h2WklPaEt1YVJXWG1HVE81bGxXaWF6Vm5qcGdlOE8yK0pLV0JLSkN3?=
 =?utf-8?B?TjhYbnFwUE5US0N0cEluaGFZSmtjbUpJL2dyOE51b0UrSXpkL05ON0o3VzMw?=
 =?utf-8?Q?QMtA1UU/I0BduqEg838FE1w/QFr9ENloHIyRAyf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 08b92ff5-2072-44f9-4d29-08d97147a53b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:04:37.9299
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZpTkiRznDRwpQnuMzQHo1QyrMfm/lyaj2LJ/rXetqy4y8YWWimaE4yCSO6OphzSsJtv7p14qvVxpV2AiASwG3w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6480

On 06.09.2021 14:00, Jane Malalane wrote:
> AMD Zen3 adds the NullSelectorClearsBase bit to indicate that loading
> a NULL segment selector zeroes the base and limit fields, as well as
> just attributes.
> 
> Expose bit to all guests.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jane Malalane <jane.malalane@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:18:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:18:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180042.326550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGNo-0004IJ-Ce; Mon, 06 Sep 2021 15:18:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180042.326550; Mon, 06 Sep 2021 15:18:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGNo-0004IC-9d; Mon, 06 Sep 2021 15:18:04 +0000
Received: by outflank-mailman (input) for mailman id 180042;
 Mon, 06 Sep 2021 15:18:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNGNm-0004I6-Dr
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:18:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d433e749-219c-45e6-afda-c2433cf10c6c;
 Mon, 06 Sep 2021 15:18:01 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2051.outbound.protection.outlook.com [104.47.8.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-LYPiVYPMO4u0kudWfCMQQA-1;
 Mon, 06 Sep 2021 17:17:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6302.eurprd04.prod.outlook.com (2603:10a6:803:102::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep
 2021 15:17:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 15:17:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0701CA0023.eurprd07.prod.outlook.com (2603:10a6:200:42::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4 via Frontend
 Transport; Mon, 6 Sep 2021 15:17:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d433e749-219c-45e6-afda-c2433cf10c6c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630941480;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FdCd0JKyDD9QE5Baa1mY81Z0aybW8BdpAIrhBi9YKkA=;
	b=EnJHLqwQRXkfbqWrR6BABUmC69eWRN/XKDlAt4xKS64iVX5EyYCGwyUGUKfGc5KySKAlyH
	1dC+80k0q4MpaVQJaOCNPKRMAYAsSYlswiVwCPu1dUjgCMqCRB7XwZublHbROVNPRi1qE9
	Oz+umVYCE5MPh11+9CNEdlyqw3Ym5z0=
X-MC-Unique: LYPiVYPMO4u0kudWfCMQQA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NcH7BRB4QfsE4qI5YLnKnB++J17UcwM358d9O/Han3EoY6zLOSjxMpQgSRY/IRDzIkPKNjeSifekkxJ+6lb8fCJoImhvujMnhGjKbYEx5HLv3OtB40OaLut5ulDaUSAbmxB0yfByS7ndg+jQMe/gA2ADq8ejbZ03XpkGcYg8f7XQ7dpNjSW36ntwtGeDXRLmyrPeyFc/+v9LJV1x0oBYSvAXlSCf7mZL3I3m9WzyEOnOILakMZnXCv4CBpl6iriahfjlOJeglsu49KGXAvfPOtzT4DPN1erquhaRO8xF1X24dfbDv5xyae02QPLJ/nMqiFulVnLxwIXuL5hX5RVFAw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FdCd0JKyDD9QE5Baa1mY81Z0aybW8BdpAIrhBi9YKkA=;
 b=ik+HONU9/etGr8GuVTVlDtswh3MRn/Za4SCUSOoXAAZ8ra/+nM7jdhpOCp/KL4G0V8LHnEWB2aKtsdgG2U/H2mx17R9YPMsieFFFJl3w6TnLAcw2WHyjOe/7OSed4PnxwIiLybDNHR48wwbUpI4mLKMNsSfX4lwWFHXmaswrTChgVWZ8VV3PcS76uaDw3LLQqg9TE/bUbFi/X+4tJfQji0yj5742RlRpkl448adRS+rotDRFmxM/KJ/NbUKvh2D2S6KDfvXLAh/0acPz6vLJ2JvAwMsw/2Uuw1f4kqo3qANOtXV2SDGKTEfCPxaro+qMDhqjmtDiCrosm+dhT22VcA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
To: Jane Malalane <jane.malalane@citrix.com>
Cc: Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Pu Wen <puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
Date: Mon, 6 Sep 2021 17:17:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0701CA0023.eurprd07.prod.outlook.com
 (2603:10a6:200:42::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5e6f0b92-df84-4f07-0e95-08d9714981e6
X-MS-TrafficTypeDiagnostic: VI1PR04MB6302:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB630217C99B6A833C496D4085B3D29@VI1PR04MB6302.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VvmXU54w1o+pBl//WH8iStYC4PNZt5OOgQHEcKvf9JqaVNmpb2vyGp5Vfaik7Au0PwI1em92KHjHLVUuNsRREfS3y7KGvkuYzF+vGOeihhxkWboORVyzb9k2nh7301A3GFcssvtnHGgkfgq/L9EvWrmZIsmif37qIsvEormJS7zOmbU6+q+Vwmv0IZhGK2Mef1bwydgaCGa1F0dbjhd90alwF9nsiWDBU7Ccq4WGLIueMpisyEVNAlyDfADW6XC6xC3V1t5TBvO7MA/TTS+GhrvH4p2qmYWH9jHTH9GxQJAHd0iVCh9pvxMJT7oiEt9NszBDd7A70opKaNRhDEmBlL4UrZQckRblviMZouHyOEHhKxhMuSNt6h6nuIxN5WsD7jFx2gTbyLQZlniXb3fuQkI9wD5lCcACFqJT0rtzzBb7WB0mrh4tYK/pj7ko3OvTau+VBD7B23POgXWhcwnhY7/Ux1YM162uPrtRPisgHEcRNpEY4LA+9M6Q2pW3b2sqGlAAt235snQTyJQTpTA/HF7PZSnk/KjdEtYmVR67Nf+nOM0PzW/+04bSlDQ8GiYCgFvO5URBTBT/423Tgy29LzYklHxfM4uDBacB7F5hCQ4jpgQAH7lR14QOPfRPA/kfoKPPqyLxZi4G9GzVzjO5kTnQNWT8Cle5cIjZ9za4G+cEiZQ2uIei5+Ibw9As3hPCcojvpt8hTSxkGeNIVsF9tXOTt5FD5ZofU3h/UuYmaPo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(396003)(346002)(39860400002)(136003)(31686004)(86362001)(26005)(16576012)(186003)(6916009)(478600001)(31696002)(36756003)(4326008)(5660300002)(54906003)(2616005)(956004)(316002)(2906002)(8676002)(38100700002)(8936002)(66476007)(66556008)(66946007)(53546011)(83380400001)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?LzNoZHEyWXQwQVRhU2N3bXpCUGNjazlXLzYzeUFiL2pEYjhob3JqNStTY1dl?=
 =?utf-8?B?NU51aVpKa25TS1lUUmpSZWp0ODZETVZsb29lZ3ZhNkhaNk5TaW1ZYzNhZTc1?=
 =?utf-8?B?SHZPd3ZoaXJrUVZMYUJ0dVBKa0pWNnpBdXgyZC9Va1RTMHM3WXJ3Q1V2enRl?=
 =?utf-8?B?VStBUUVZdEd0b3ZCTGMva3U2S2ZPTVNTRFBzbENKbkhiVnRoUXhiUFhNR3cr?=
 =?utf-8?B?TEh5dCszbHdtWjlZeFQzd1hMVTUvaEgwZk9sZUgyTkpQWDkzZzBaSVFUU1p4?=
 =?utf-8?B?RHRUVjd3QkFNckFydmMxVUl3Y3QwVStnRzJ3QTJKMmUzcDVBdVJGMldTSVY5?=
 =?utf-8?B?SGZUbnpSM0pCOWsvdHBydnhNSzRKNDZFUVhLSDFScTlJM0I5c1B5Z2tNNFlh?=
 =?utf-8?B?djIzVnVlZ3l5dnIvTEpQdS90SkxsMG1ZQzV0NG5GYXEvcjM3VUQ2L0NscFhZ?=
 =?utf-8?B?UWJzTHdFUzRDdkdPYVk4R1ZCTVNjY0hBaXZkS1lxdXd1dzl1M093cjdVU1N2?=
 =?utf-8?B?UFFPaEIvZnlxY0h3NlZkbGtkc3NaVU96YzNHS2tOS0g5cmZ5eUNzaTdjTk5E?=
 =?utf-8?B?WGpaWHhNZFgzamxLTlFacURjZ0dUdnZrT3gyd1BuQWJSdjBpa05NRXZ1VGxV?=
 =?utf-8?B?cHAvY1AvMlYyby92cU9TMEd4TnRWcjFhRTY1OEpuSkpKYWhuR2EwRFpBaEJu?=
 =?utf-8?B?NzVVWTB6NnU3K0hKUFRsWXBZMXJWOWNOTE8zNy81blU3dWtDMmQ2SjlRV1Va?=
 =?utf-8?B?TmZvcGdvVUdVWVFSUkhtTzE5aFlyKzQ5NXN1elU2L3dZRGJpZi9MNGRoeTNQ?=
 =?utf-8?B?bThYZG5DUUdQRHFseXl6QzlIWjl3bTNiTTA5c3REc3JpaHRpYUpNbFBXaytP?=
 =?utf-8?B?T3ZBUCswbFlsMWxZd1QvNGNZVE9ZY1hqOVd0Y2k5NTVMZWlZdCtPcnFQTE5w?=
 =?utf-8?B?ZVduZk04dzNCcHRQSUJxU0Z5aENUK3k5WmJPWTNlY1NVNWNXWVluK0swY1Rt?=
 =?utf-8?B?S1dqc2pDdkNjQU5EeXFkVFZhRHBXblQ5U2ZOYWEvQmdhbmFXK2NDU1dncm84?=
 =?utf-8?B?cGRDeklEQVJiTW5oaFlUMUJRT2dWcllOUkVrWllGTHJHMzQ2alBwWDJjQmxZ?=
 =?utf-8?B?QnlESEtTSnlwT2p6dTRodzIxbXJzRktDck9VZ3NoeXc4T2NmRDlJMDNpWXJF?=
 =?utf-8?B?YUE3QUM2ak1aQVhodFBRK3B4K3lWR1pxYmpTZDhqa3dqQ1laSXBHRzZmVHhj?=
 =?utf-8?B?T09VRFVPRkhSb2hFUXg4NFVRaVRoVE5QY3I0L3ExWGZWbkl5WGxyeS9jSkRp?=
 =?utf-8?B?YTNRK1JsL25KYi95dTN2Kzc4RW5IeUlXUXg4NlBwNGxkT3krUEpyNHZLVHJB?=
 =?utf-8?B?SFVjTDFSNm9EcWZLMVhVOXFPMGs1NVEyc0RFcHZzZFQxbS8zOVkwbHJWT1Yy?=
 =?utf-8?B?allBbHArNnJoM1hzNnVCcXZsTGoxT3M1V29FbzF6MWN1Z0hhc0RoU0h3bXlZ?=
 =?utf-8?B?KzhxUGdHdmRvRklGbzl4bS9objR6WVZPVXR5dGtac3YrdDBTbk1GaHFMNzE1?=
 =?utf-8?B?QWkxcmp4clgxd1Q0emhXeWlIYjFRdjVJSXhzempVNUdsK0MvUVhCcTRFUEFs?=
 =?utf-8?B?VkhpSEVkQ2tCZmFJdXducWtCQjMrZTRwYmFRLzlCTDhxM1RQbDErZHJhMnha?=
 =?utf-8?B?TE1oaTkyZFNyS1RzaVVjZ1l0OUV3enQwb3Jtd09aTWI5QnNJakNJeWtzVFhX?=
 =?utf-8?Q?BAAE8R3oH7Q0/U4tbd6GgLQono/Twy5ZTANsjFP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5e6f0b92-df84-4f07-0e95-08d9714981e6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:17:57.6584
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 21lLh7WhMg4mm5/yIXse6kZU+56GJcq85eFhh77Z/wvErLfnPNnBm0v+dkmHsQHhiA2oqWJPWITthk2W3shkiw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6302

On 06.09.2021 14:00, Jane Malalane wrote:
> --- a/xen/arch/x86/cpu/amd.c
> +++ b/xen/arch/x86/cpu/amd.c
> @@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
>  			  c->x86_capability);
>  }
>  
> +void detect_zen2_null_seg_behaviour(void)

This can in principle be marked __init.

> +{
> +	uint64_t base;
> +
> +	wrmsrl(MSR_FS_BASE, 1);
> +	asm volatile ( "mov %0, %%fs" :: "rm" (0) );

While I don't strictly mind the "m" part of the constraint to remain
there (in the hope for compilers actually to support this), iirc it's
not useful to have when the value is a constant: Last time I checked,
the compiler would not instantiate an anonymous (stack) variable to
fulfill this constraint (as can be seen when dropping the "r" part of
the constraint).

> @@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
>  	else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */
>  		amd_init_lfence(c);
>  
> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
> +	    c->x86 == 0x17 && c->x86_model >= 30 && c->x86_model <= 0x5f)

DYM 0x30 here? Or 0x1e? In any event 0x5f should be accompanied by
another hex constant. And it would also help if in the description
you said where these bounds as well as ...

> --- a/xen/arch/x86/cpu/hygon.c
> +++ b/xen/arch/x86/cpu/hygon.c
> @@ -34,6 +34,11 @@ static void init_hygon(struct cpuinfo_x86 *c)
>  
>  	amd_init_lfence(c);
>  
> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
> +	    c->x86 == 0x18 && c->x86_model >= 4)

... this one come from.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:48:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:48:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180058.326561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGr7-00088h-Qg; Mon, 06 Sep 2021 15:48:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180058.326561; Mon, 06 Sep 2021 15:48:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGr7-00088a-NZ; Mon, 06 Sep 2021 15:48:21 +0000
Received: by outflank-mailman (input) for mailman id 180058;
 Mon, 06 Sep 2021 15:48:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNGr5-00088U-9s
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:48:19 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d40a1fa-d63f-473a-9c81-61d5e52bc2fa;
 Mon, 06 Sep 2021 15:48:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d40a1fa-d63f-473a-9c81-61d5e52bc2fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630943298;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=ZW8rLL2Ju+LwKmfKg28mVm/orxZTxzhvsjw8xsAcAL8=;
  b=SW3HaML5diHR0KeK+K6mn3ZcUwit83U9Yad6VgeDWo0I1e/LIbY3TxKV
   TTUCzSwut/Xxz4+vfdZJ3a1DFEdNzbRsLYV6mg6eZZU01fDCxB7nmxIia
   e0Oyy9sw4paP8ZrJwR5G1xbCjjDP0q6wsWhaIPrsAGuR0NAEEhHS12w4q
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: szFn5Rx2CcSuwwNQSgdCA4V9pzjhPGxgwYXW4zhWAc8u8kbqhc0/avN1YHIM9EPT+q6qH/K2Xy
 amKg1jbA77nB6bRwsS7Bw0UL83AJcFIMGRTC177pmGimOmEEjayrapbqPDf6v6wOwp1l09fCUh
 V9PTFDsiaoXw+wivSubxBvA835Z1c+UE+mBe/G+H23rZyi1zgx89ifsPCsW+2G280kzzYElELp
 2Bi23k1lajzKsYwON8W0RXQZU1+968rUpNlqrfVdhMyoASlObHkkg3kLdvSSD7wkeku3/n3ax6
 Dt3q/SRhRog2nMUsqBjjNkQ+
X-SBRS: 5.1
X-MesageID: 51703951
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:j2H4TK5mw9yxhWnR3APXwD7XdLJyesId70hD6qkQc3FomwKj9/
 xG/c5rsyMc7Qx6ZJhOo7+90cW7L080sKQFg7X5Xo3SOzUO2lHYT72KhLGKq1Hd8m/Fh4tgPM
 9bGJSWY+eAaWSS4/ya3OG5eexQv+Vu8sqT9JnjJ6EGd3AaV0lihT0JejpyCidNNXB77QJSLu
 vg2iJAzQDQAUg/X4CAKVQuefPMnNHPnIKOW297O/Z2gDP+9g9B8dTBYmKl4is=
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="51703951"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P4ahkLnBPGH/J6zIrsH2R47SfkgSH3cyayht7Is/8Wt6EdzxP5+9LRaQ6dStqADi2qSZMFYKSLF5vZ8L8ZUDJd6aoAehMDZsGLhNuL2oVum+5Xp7HWx9H38QWW3aQV/rTVe0dMimydRql1+F/tEEnsqSdmoCb7ep6s6SG9oSPDcHkxlH/krBhvbIrDIRp8p97hsp6xWlaf7/cnYSh7WjvN677xbebI3Lj0/bZYtWIgdENNyrJv0QmbYY23nnW18oRoBenfJP0689txVaky24DoZODKc08siJuPq/RRvNLTtCs/Ihux3xykeuok0JRwlozlq2MD4TyBjyMvRAxjrwXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vLg9oQJLQc4EgCHbSr6T90rQDY6+Gti+7IBwBOjZGbY=;
 b=SJbpy3EgCZ8UXkgwW3Dhq/zC7VIU5ttzfDwL5Rk5sFJNx3jJcEEOYGAjBxH09hU8kYok5IO3D+glN3oHKtw2p31DIBPf3f17bFL+5QSRs81JwYyLtwfPUtONwUs93pOTKTsXo4ri3YivfpbdqdytfSzHzIaIoEgj/eYqHB/V+SLETTsqIhvOKwDFHf+o4UHyB3gSWb+CSWmpHY3ksGdXr9tarJyQqqz9jK5udCOPzyGWHmCzuAk93Wh6Cj1TykKZcl2ZRxe8z03HQ3tP5jnRZ+ONjZdEp5q391/XsLhD2QU7oi5+dohlZkfcvZnemXBjK6NoS8PiXt7esxY89O6j2Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vLg9oQJLQc4EgCHbSr6T90rQDY6+Gti+7IBwBOjZGbY=;
 b=fvMhLCeYD61czJreXW+pOAnmpMyj33eXcw7h6FMjlsUGVy2EndwOXFspifROUJkFJp93LbRAsoEmwJ+XHbBXtmncIJrUmGzFYKpCAQaMhndBoUbP79kiAIsxYbP2ryunq2rdKYxUooaa9otDUl2lvLKu3Zk4GrG2SO90u4w5GpU=
Subject: Re: [PATCH v2 2/6] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <457637e2-d35c-18d0-ead8-0b513b257927@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <9b7936fc-f81a-d5e7-5ceb-6e10588503dc@citrix.com>
Date: Mon, 6 Sep 2021 16:48:07 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <457637e2-d35c-18d0-ead8-0b513b257927@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0096.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8::36) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5a2441a8-bb9e-441e-ec9f-08d9714dbc3d
X-MS-TrafficTypeDiagnostic: BY5PR03MB4966:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB4966E30AA9FFEF551B9047DEBAD29@BY5PR03MB4966.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: QxgfqcIIXmpfgh+TmsFMbnCev7yqppudYl2WK9Bh+U62reyc+eGhs/HcyhP/XzeLx97mt4G/aUvuv0etePk+bzkBIpFSTYqhlX8ZzV58pAlYiY27h8JWBa0bFqyCghFRJ+/+VpoVPGCKyaVrmvigXj3FTMwBwQ9zyIhTNREBy5QdKK3pyBHMk9vZoCBMpM/eHNIKIhovKWB9Y++usOHoXO9SySXfzpkz2ef7c4TO/YmbTLatzmXIHOKslh9997Rt4e/y/cyTDPuINv0x/0wOieDWZAM3eW6rrw1WHOPf6eB51gsTcuqleUl8dDNEFUr34dl/D0w8ypcU2lXw2R4xyyvXvXV8Yf1OEpvP1rKlOeDz9V8pb3T8D/EWw7qPHso0ueeOXCzup00K9zAg5rB8zBOnY6gxpbQahZ6bdIzgLy+Uq01mYQyIwBCxs5jSgLMyb2bM7nIvIldJahb5V3Fa3gYveJOkTT/s3mOIpdj63/1mqimMaC16ioC0yT8bvKgQ7Juw7Qu0lyDlERzemf/zbg4Hf+NewcjpMMCFdulLs83jw6evChvXYplF5RJPd3sVdywgOnwGvVp1R7oHcRC/iopySp0CCm7Du3SK8HWCUl4aW2SgOTlTWW43OhoLcQvEaj2Osr2Et5SF2zVUBx1wfPlnVnM+6zZT8Q7b9ZRAC8g1mYL00ZZ9ig4bPPWfxDhP9dcQ7rzox9KDbMJKgnmR676OIBNLgOBhqdMYRy/zNkI=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(86362001)(2906002)(53546011)(2616005)(8936002)(186003)(8676002)(55236004)(6486002)(31696002)(66556008)(36756003)(54906003)(6666004)(26005)(16576012)(4326008)(316002)(66476007)(66946007)(508600001)(83380400001)(956004)(31686004)(38100700002)(107886003)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TzF2UDdLTVA4alZweVBaQ25Xcnk1bnd0NUNnTWNWT3BrKzVpWHZhSGkvL3B6?=
 =?utf-8?B?TkJZYnBBdmpocHNxZnJYSUZ6ZVRpNElwc2lMbEZKbFlMU0ZXNnRLckVjM2Nq?=
 =?utf-8?B?blJ5eFNUQmFHU0tyeDV4Yk15ZlRwS2J5UTh4c3N1VFE1aE5QNTRvVnJoT2Fk?=
 =?utf-8?B?THZSZ1hIaUQvYTl4UVY4VXpMdDEzWStpNFFybElOMzBGQm1FVzZ4VWRvRlFK?=
 =?utf-8?B?bnYzMDRUemFBcWszNjQwYTFaTzZURWM5dWo5eW4rNG8xUlE3V0EzajFYVm5X?=
 =?utf-8?B?N05WeFgyQkJ6eld3RnlveGlrTnlvbkhpYW5JZUtvdndhbnlaYTBXMVIxNzRK?=
 =?utf-8?B?UFJjVFdtTElDYnduYi83cGh1MGg5ZjFFdkNpRjdBTmxtY25kN2FOVDc5WFFV?=
 =?utf-8?B?MC9zV2hhWUxmWVQraWlkVENuYTBPU3hVczRKeGwxMXVvWm43bDZtaTBhcHZv?=
 =?utf-8?B?MkYrUVR2UGpUYVhEbUxnVXhXZVlQMTBIUDVMc0tzQWFwN1ZPd0FUR01DM0o1?=
 =?utf-8?B?Rm5nSU5lWFpzRzRqaXFmeHN4WFE1cUlLRWFSVHFManYrWk5HY3AyK1krTkVY?=
 =?utf-8?B?b2ZndjhiV1BuT21mYzJMQThINzlJRkIzK0NWSkNsZ3ZGditRUFRneDJvVUpm?=
 =?utf-8?B?TmFsME9wNUl0Y1grRXI0K2VjdE42c250RnRvekRwQUFlaEYvQXhOVFZaWmZW?=
 =?utf-8?B?S3R0clNLaVlIekk4VlJkbkRQS1FObGoyV21RSTZuNWVjWGhRWXliOG1QVnhm?=
 =?utf-8?B?UTJIdUduMjJrbEg5SGp2NDM4eklCczZXbUlscEsreks3aURYQXZFcEd1Rmxk?=
 =?utf-8?B?QVpiM1RZWXYzeGFDaU5JMWNDa2E2Vlcrb2RGTDUvUEJBTi9wZm1CSjhaZWFx?=
 =?utf-8?B?UVBycGQ0eWVJdXJJU1g1aEZmZHFhUWtJUkpPQVFTcGlORHR1UU14dk5RKzE0?=
 =?utf-8?B?QXZSa2F3TDJvYStXZmZHVkV0c25xMTFKUDcxdTd2dUt4MU1jemw0YmRoOUh2?=
 =?utf-8?B?by9VNnA4dGo2amY0ZlhhaTVoeGVSditkMlprQVlOT0c4MHpXSEFSREpkOENX?=
 =?utf-8?B?RzBRUkI2b0Y5VXJmcTFZN3BNQTVKU1JQemlBbC9yN1BaaDBPZlRZOHdydmxT?=
 =?utf-8?B?SzB2TWpCNUJYOXlScW9xcVdZaWg4TmlYV3ZkWGhzMHRQTnArOGJobjdnYk9E?=
 =?utf-8?B?aUowdGRXWmhWeEVLZzNKUmNZdEVyZGpOd0pYaDcxcHFqeXBqcHU5MDRFTjdG?=
 =?utf-8?B?TEdrUzhFb1ErWmRKZDBLR1V4c1hYK09rWHMzU2JrQUFhWXc1SVA0eFNvMUgy?=
 =?utf-8?B?WHl3RUdhM2dTdG9XNng3V2xkMHFsbU5mOGxDOW5qRmx6OVc2MmtNUW5TUjdW?=
 =?utf-8?B?dnIrUDVlMDdYdmVuY3orWkR5ajJmeWVGdTZLYThPaGdidXFSYUc0RDdQYVRk?=
 =?utf-8?B?REQ1S2YvOWltK0owai9rZ2x6THBxd1paZE9UcE9kMGZDK1doMU9RQWVVUld5?=
 =?utf-8?B?ZFNqUnVJZ3ROc25EdFdTL2RJaWRzaXFsdlRUcmVRTWF1VnlCcGxCamFuUFdz?=
 =?utf-8?B?VnhUcUFhVmJ6RTcvWmJlbXQxSEFDQXpIQTRwRnNreldXWExlTUpQTWJ3aytU?=
 =?utf-8?B?SWpLbGNyU3ZDbGgwcjFzZ0VpS3A2anNsem5KU2hwdlFhQS9WZlc0a09GVlpa?=
 =?utf-8?B?Y1poa3MwWGtPdkpHVzBFM1IrSTRVTllHMG5XeWNQWWlFbDMyeXFxOXVtSVNL?=
 =?utf-8?Q?foK6awgck0okiIU/bfleyTi8+RCuzX6oTWozsK5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5a2441a8-bb9e-441e-ec9f-08d9714dbc3d
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:48:14.0077
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9EPCwDyiPSVK93v8mLg/Wpa/UwxGc1Yl9Llx0Ew/7Ud+raiHrd/SLQHLITTKETJ7IDYWlIlVoO3wOLFc/1qDmFkdpqfTztMekTjxuRlip6Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4966
X-OriginatorOrg: citrix.com

On 02/09/2021 09:33, Jan Beulich wrote:
> To become independent of the sequence of mapping operations, permit
> "access" to accumulate for Dom0, noting that there's not going to be an
> introspection agent for it which this might interfere with. While e.g.
> ideally only ROM regions would get mapped with X set, getting there is
> quite a bit of work.

?

That's literally the opposite of what needs to happen to fix this bug. 
Introspection is the only interface which should be restricting X
permissions.

>  Plus the use of p2m_access_* here is abusive in the
> first place.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v2: Split off from original patch.
>
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
>              return -EPERM;
>          }
>  
> +        /*
> +         * Gross bodge, to go away again rather sooner than later:
> +         *
> +         * For MMIO allow access permissions to accumulate, but only for Dom0.
> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
> +         * the way they specify "access", this will allow the ultimate result
> +         * be independent of the sequence of operations.

"result to be"

~Andrew

> +         */
> +        if ( is_hardware_domain(d) && gfn_p2mt == p2m_mmio_direct &&
> +             access <= p2m_access_rwx && a <= p2m_access_rwx )
> +            access |= a;
> +
>          if ( access == a )
>          {
>              gfn_unlock(p2m, gfn, order);
>
>



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:52:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:52:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180065.326572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGux-0001Oi-HE; Mon, 06 Sep 2021 15:52:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180065.326572; Mon, 06 Sep 2021 15:52:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGux-0001Ob-DO; Mon, 06 Sep 2021 15:52:19 +0000
Received: by outflank-mailman (input) for mailman id 180065;
 Mon, 06 Sep 2021 15:52:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNGuw-0001OS-DT
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:52:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5917e309-8132-4e59-9e5d-232a1e1460da;
 Mon, 06 Sep 2021 15:52:17 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-B6XkD3duPKSlItBJVN3VAQ-1;
 Mon, 06 Sep 2021 17:52:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5325.eurprd04.prod.outlook.com (2603:10a6:803:60::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep
 2021 15:52:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 15:52:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0028.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Mon, 6 Sep 2021 15:52:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5917e309-8132-4e59-9e5d-232a1e1460da
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630943536;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=L7UKcM4S9tKAzyGGYNFS3Wo9PSeC76IYZoeIv6naCYk=;
	b=NP/XvRsOF3kwlIVghVZwNHuFV6xZen329F2RgfzPmeiYtveyYyRaHA1ULTdDn/Akyxfnis
	MbzJwQDm/D6WaQnOGrFrK2AizLjmRqNJf3C57dDjx8Di1hanwzhLsiXhI/ywIJw3bOY69r
	trLOANHfUELv5dc8Bfuo/enF/1a6veM=
X-MC-Unique: B6XkD3duPKSlItBJVN3VAQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eqCdVsoSwNMZymkJeu6shrlDE7askClulZsXxw6Dr03LBi9DhUNYfy1fWuopynjT20WVdRvju2qHrvvI5zwY7H9sNQamNBQRCD1u8UXPaihz9ymBvoMM/iMLzbgEHIxg+QxJt6D2RAnYGmW2iZ/KXe8a0mSpIMOmd9nPz0G0/zM5iwgTFgqG6l32K6u9Ar63MNYCeELtS9gQAxYhoXZxn15Fm6IZ3kruE6u2+i38Mzs1RrbWOGryS6dtuRZKLRoy7W+6ySZ9Aly4bSG3COP2xoWdgF4A9HNFmXP7I5rZgzNvEazHIEi+UVMRjIkz5A76EwROn+OLfnLB5rYADNLZfw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=L7UKcM4S9tKAzyGGYNFS3Wo9PSeC76IYZoeIv6naCYk=;
 b=jkg7Kywyvh3UFDR37yqIl2Z/sJVA21A9MSr4GIA7cmvuhFAV0UHen3NjKkr9ZCy4isqyfyCh9P7EVP5ew7eWfbkEnnykkfpUIpKyp5HuTrjsCSn20uuGQDfZz1DfeE6REZ+3/Gx1el437aaTwoOPf8snj7vmf4dxns6xflKNphUDuxL1MRsuHjrEY4pPFmP6RwylFY8WDQrp2e/AjIRXKePaby6SF2UwV1mjacCLPL1KpFGkyEuMUQDcPQ9u4+e5izcjcS9ouZDeVTF5eNa7utuvBEht3/TKVFnD//Z+/562JmvD75sbnUX1krWdq46upc6YofmGf2nVdiZgRAFoQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 01/11] xen: Implement xen/alternative-call.h for use in
 common code
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-2-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3a91e4ad-50c4-205f-3d90-f23c996b2938@suse.com>
Date: Mon, 6 Sep 2021 17:52:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210903190629.11917-2-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0028.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 70eaedd1-bfd6-462e-d501-08d9714e4ad8
X-MS-TrafficTypeDiagnostic: VI1PR04MB5325:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5325B7217303F3CD0D0D2333B3D29@VI1PR04MB5325.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XPBuEpIZJHXJ/zq749VVD0uO1AyOnXa0eT28Ib65HMWa7fcK2VkJyrpah1baaEPUeMRASbvXalkR3s6RX2RE+1eXwFD0VzC2WQV7jYEK6xTOWnwjd3AwIKqTTuGeHhrSHN5CsiYXRLh394YwWdjI7p8El7cRU6WlAHPlOpxmhhRHfwufZ52xTBzdHsleBThHYAJq7Ykb0ZpZbpNt5vG/O/iEszfea4uw7JO7hucDgHAHbzH0vkLhzmy+6sMgG/mDqfn4q8XujUZ1HYBC6dCppHoXldh69Gz8C5hvGI2SQERXxnDGIWP8pYhEbd3KS1DH2znZqJhQ21rhVTku994y9g/9apy1hIv9WCKBLnU9lzwOtoZj6ham2vphgUF7YYenOrT+jCTxlo7idufc3gZ1Hv3mROzxD4SYJVM7oxvNDbfx3KnXFzOZlCSjTqxMA0PPn971eY4lHxPIJudJh19o2hPtl3cVfmrvL4clioqCwlVbXa26WXbcyGDFMnKeNUMflDJLhdWB6VGmIJvitouYXsL6kosVRmsEe6IPdBVp/OWqKxDrlc1FvkRDfk/JSjws+fLClJmNMioZggCWHC1dohRAitqEQ+B2qVOp0GRopesJiLigPlWbt6Kn+7NHrmG9syBbaYsgooXb7Rg0Zlih7+pVX9x183drs+/BgtyWC0gxu4tkuU18JaD2msYDGcT0Bm0AsSLzflJUj46xdycEWJxP6F2+R7N1o6uRYYj+dJY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(376002)(346002)(39860400002)(396003)(54906003)(53546011)(5660300002)(4326008)(8936002)(956004)(38100700002)(6486002)(2616005)(83380400001)(31696002)(6916009)(8676002)(86362001)(26005)(316002)(36756003)(66946007)(7416002)(31686004)(2906002)(478600001)(66476007)(66556008)(186003)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MEt6Vm9xaEtkRjR1QWZGV3hwdk9JSSt4MDFSOThnUEZtTlZCQnIrWllCTko0?=
 =?utf-8?B?ZFlFN2N3QUh1b3RQMVV3RmhadmlRd1pWeG8wS0dZRWxNVkM2eTJlZXVyOHRG?=
 =?utf-8?B?SVc2SklyNEV2Z1ZKMG9Xdm9uYnZ2OERhSWNnMmhBRSticVMyY05JUTdPdEE1?=
 =?utf-8?B?dWw0ZHA5NU8wNVkzK2hPaCtEMlphT2NDR1IrNVBjeE1MN0l4TUNtdUNyYm9a?=
 =?utf-8?B?Ty9naXo3L2tFNU1xYVBlalV3VlVaWW10MG40N3djZ05ndXRaRzBrbUFvdDU3?=
 =?utf-8?B?ckxBbldlRXo5SnhNb09xT1l2WHprZExtdTdXcUp5R2l1Q3l5aHQrakxibXF5?=
 =?utf-8?B?VWRTanQwTEJpcnRlYVY5VGR5RVRlNUFheG1XWGJwaFk5QkNCUGtZUHZYbldq?=
 =?utf-8?B?VFBrWi9wL0dkbTZkd3lXS2h4c1h5TkYxSVVqMS9neHFFWmswTmNETnlacW9o?=
 =?utf-8?B?YlJQNWdzUTB6azFUSzdxOXJTZ1N1UVhET2tJeVcrSlZLRzZaRXdlMlZJem1Z?=
 =?utf-8?B?aWZTbVdMVHJjUjJhWTY2RGZLem5lS0JSRkJCT0lUUkk4cGY5bk91ZmhDcjlJ?=
 =?utf-8?B?WTIrcGIwQTZCeXNheEh5Y3lxN0lCOFlyTmZUZHJOcHJUU3ZIMnJLZXQ0bWZa?=
 =?utf-8?B?bjNpc2ZLUmpzczVYMnlXYWRNMDBnQkVFT2w1cUgwT2N5c2dFY2JpSmFtTkdo?=
 =?utf-8?B?TWpoUVgzdmo1ZXdwalBZNWNrMmpuWGZZSWFzclpzbnpYWE5JWUJPQzlRMHJ3?=
 =?utf-8?B?dzRDNktaekxXeVBCWnUvVTFGd1paOUgycVVoeUJ5VCsvV3ZGdE5rWEd1ZjFM?=
 =?utf-8?B?aHo1LzFSVTNUZjhuZTJ6MFg1SGppNzh5VDZFY1JJOXRsOHhTT1RWOTRkZ0Fn?=
 =?utf-8?B?ZThTN09wS0YrdEJQa3pWNTdENU1VY2tJN281L1h0czJkekVQajlsR3ZLYXhr?=
 =?utf-8?B?SE0zSW1LdnBWMmJMa2tsQXpTN0dVVC9QUkJ5S0ZocmthUFNrOXB5WW5SZlBQ?=
 =?utf-8?B?aGlPOVIzSnN2SkhZaTE1Nm1kRmVhZFN6MUlGTGtOVm5YSWE5UWlEc1oyZko4?=
 =?utf-8?B?NnRma3F2a1BRM2N6TFlzNUx2dFRoMG9IZ05Ic3BSUU1kWVlzbXJJL0sxdTFo?=
 =?utf-8?B?UlNXYXFhcmpzellLcGxnbXJsbGlPR3pxNERLcGg3d3VnWWJXam11K3hoQnM3?=
 =?utf-8?B?R2Rxa1J5M3duYm5oYXBuQ3dBOVJydWRpa040TEJVc2M0d2E3UXl4ME0ybkdo?=
 =?utf-8?B?QVhSTnphZ3dDZ0d5a1kwcmF6S0RqeTl1YnlBTVp0TVlHRjhIY3B5bWRrcFdZ?=
 =?utf-8?B?cDg4dUxkOE80MGxnT242Q3dKTFVCZ3RTcWU5RGNkOHRCU0xucGcvZnFjZUJK?=
 =?utf-8?B?RUtHOWVjV0l2ZFluWGJweUVjYnlGcWM4SE9DU2FQYUFOeWI4TXk1TGIyYUlY?=
 =?utf-8?B?NkV4WEc4c1pSOG4zcHI0QTlOYnM2b2NxeWRLWUJHbFZUYWxVLzBXRGZYdE54?=
 =?utf-8?B?SnhvcllaSkNxdW5BQUFXMVNyYUNGamZDM1Q5cUhtSXRwZVB0YjV0dUdIekpT?=
 =?utf-8?B?a2twQzFCSkpWNjd5UlVTS3R5SXpBZ0dVSTBMZzhWcE1jRVJZSWJyK1pyQUZs?=
 =?utf-8?B?d3hITnYzK202Skh2TVVIRXB1Q3BvbEt4clIxYnU5QmJMd1g5enhGRThxTVlv?=
 =?utf-8?B?NWMxc3JlbUI0N0Q5eHVpc0ZCcTFINmcvVDNmaUlnKzloY2R2dTF5aXBsVzl1?=
 =?utf-8?Q?vQ9KjBXEIKAoFT396tRK/1QLJ+kHwHPzEyibdHx?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 70eaedd1-bfd6-462e-d501-08d9714e4ad8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:52:12.8776
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QVZidnb6RsPIUEX5vXOxm8KwJoii/LRbYbXp1XbivD5BmsJ8hGwPmGT5YBL8hFn4fNKpiipOjwcXE4Qjox+urg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5325

On 03.09.2021 21:06, Daniel P. Smith wrote:
> --- /dev/null
> +++ b/xen/include/xen/alternative-call.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef XEN_ALTERNATIVE_CALL
> +#define XEN_ALTERNATIVE_CALL
> +
> +/*
> + * Some subsystems in Xen may have multiple implementions, which can be
> + * resolved to a single implementation at boot time.  By default, this will
> + * result in the use of function pointers.
> + *
> + * Some architectures may have mechanisms for dynamically modifying .text.
> + * Using this mechnaism, function pointers can be converted to direct calls
> + * which are typically more efficient at runtime.
> + *
> + * For architectures to support:
> + *
> + * - Implement alternative_{,v}call() in asm/alternative.h.  Code generation
> + *   requirements are to emit a function pointer call at build time, and stash
> + *   enough metadata to simplify the call at boot once the implementation has
> + *   been resolved.
> + * - Select ALTERNATIVE_CALL in Kconfig.
> + *
> + * To use:
> + *
> + * Consider the following simplified example.
> + *
> + *  1) struct foo_ops __alt_call_maybe_initdata ops;
> + *
> + *  2) const struct foo_ops __initconst foo_a_ops = { ... };
> + *     const struct foo_ops __initconst foo_b_ops = { ... };
> + *
> + *     void foo_init(void)
> + *     {
> + *         ...
> + *         if ( use_impl_a )
> + *             ops = *foo_a_ops;
> + *         else if ( use_impl_b )
> + *             ops = *foo_b_ops;
> + *         ...
> + *     }
> + *
> + *  3) alternative_call(ops.bar, ...);
> + *
> + * There needs to a single ops object (1) which will eventually contain the
> + * function pointers.  This should be populated in foo's init() function (2)
> + * by one of the available implementations.  To call functions, use
> + * alternative_{,v}call() referencing the main ops object (3).
> + */
> +
> +#ifdef CONFIG_ALTERNATIVE_CALL
> +
> +#include <asm/alternative.h>
> +
> +#define __alt_call_maybe_initdata __initdata

My v3 comment here was:

"I think it wants (needs) clarifying that this may only be used if
 the ops object is used exclusively in alternative_{,v}call()
 instances (besides the original assignments to it, of course)."

I realize this was slightly too strict, as other uses from .init.*
are of course also okay, but I continue to think that - in
particular with the example using it - there should be a warning
about this possible pitfall. Or am I merely unable to spot the
wording change somewhere in the comment?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:55:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:55:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180071.326583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGxY-00020s-VW; Mon, 06 Sep 2021 15:55:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180071.326583; Mon, 06 Sep 2021 15:55:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGxY-00020l-SM; Mon, 06 Sep 2021 15:55:00 +0000
Received: by outflank-mailman (input) for mailman id 180071;
 Mon, 06 Sep 2021 15:54:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNGxX-00020d-Dk
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:54:59 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c8ec2a44-0f2a-11ec-b0ab-12813bfff9fa;
 Mon, 06 Sep 2021 15:54:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8ec2a44-0f2a-11ec-b0ab-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630943696;
  h=from:subject:to:cc:references:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=iFu3zhxevD19T38OTBC4f/Ly8O9/qxuVANg6BKssmGw=;
  b=MMklRHFLM2DrltER2yO0x7m4s2923mrh/87ERbW6P8nRLKe4FwmsSTR0
   yO4t3nkU0PGM6Ji37bHOUMSXRm9OdR6nTgDNnXYFa7bwTRZVR/pikr2QV
   Bnopb4SJJwmzGk8zmvlLYoI6DXSnWP4sciThw423nGKPvBby4mY/+namF
   0=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 3cgpXJT0NZj5KsX2WolJ0HvD6SNovf9HEoo8OhyUGiOwJXc+C6exDVtoXnZ1RKejOczXPEK59A
 PAbiECxEFUOAM67eAYix485vlzLBpYiyaZ0KFPdDZhA9SRRytuOs5mNJMW0OBq5F8/uhoYinUL
 GcEszmLYpGo9S1rmabOj+DFCRz04i/rDi+dp3ljtWyjZOrhmCyVM2WBP9sKfV27tO3muhwWGlr
 AuWMFncw3Q+UCFWa13pl0AyuXIWX4utryogBaZPVdmOXd0coKZfDn8hme3UF7a9oZPyx4TUElF
 RALj7WFsUe+DjO7CxoCskDl7
X-SBRS: 5.1
X-MesageID: 52086313
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:lGbq5arUxEKnUT+NoQ7iNocaV5uiL9V00zEX/kB9WHVpm5Oj+f
 xGzc516farslossREb+expOMG7MBXhHO1OkPcs1NCZLXbbUQqTXf1fBO7ZogEIdBeOjdK1uZ
 0QFpSWTeeAcGSS7vyKkzVQcexQuOVvmZrA7Yy1ogYPPGMaGZ2IrT0JczpzencGNDWubqBJcK
 Z0iPA3wAZINU5nFPhSURI+Lpn+TpDw5d3bSC9DIyRixBiFjDuu5rK/Ox+E3i0GWzcK5bs562
 DKnyHw+63m6piAu17h/l6Wy64TtMrqy9NFCsDJos8JKg/0ggLtQIh6QbWNsB08venqwlc3l9
 vnpQsmIq1Imj3sV1DwhSGo9xjr0T4o5XOn4ViEgUH7qci8fz4+A9opv/MSTjLpr24b+P1s2q
 NC2GyU87BNCwnboSj779/UEzl3i0uduxMZ4Kwupk0adbFbRK5arIQZ8k8QOowHBjjG5IcuF/
 QrJN3A5cxRbUiRYxnizypSKeSXLzAO9yq9Mw8/UpT/6UkRoJk59TpZ+CUnpAZEyHpnIKM0vt
 gtMcxT5fpzp4EtHPpA7Epoe7rANoX3e2O5DIulGyWuKEg2AQO5l3fJ2sRD2AiLQu1E8HJgou
 WMbLtn3VRCMn4GT/f+h6F2zg==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52086313"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xpord6Lul3uguoS/aSYBYiKyYp8hupYGMY0kTcgsKLDVRe1CNsVhJLOKVw2yPHykxhGcQ6KdXPMVO/koux+BZsFR5/sckE2l7NzSsRpLm2pR1S6cxfX8Jh9laL1HE8fAxJllMJePAapmlocuIa4Og2Yuoes0w07VPToLj5syxBS9uhMqRRtcCVkO70EDQp/K2aSEcuJuO++1yqPC6qGhalBEdgwIBgBfvLzvWBDeyEDlAUukXLO6C94gLUu3ia4hDY5jhzwHn5ncroyTM3A/3jp7Hm2mCckCOemnnWxrTP8ZvRpDcWLCu4kopuhfbRsJVEx2FsLZaoICfOUruW51gA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=iFu3zhxevD19T38OTBC4f/Ly8O9/qxuVANg6BKssmGw=;
 b=ZUPZ3aRZzzYGKTl3U65UZIA2jKcEB3cxy+Nveyw5s5JyKZa/4noOyzW2FgmoqqwaqzwzNCA4yC7gNQmygh6ly82i33j8QXt9+S3lrI233dtilcZsy9NAqg/kW74YcWVTT0ZNdoSOUSlMWQ+OMuyNLsjbCjYWDIU6YDqG+blF0v6QEysy/yBN+cTa/xLL4MJtd8vFPQS+MLdQPS/y/9ET0PD7RuJQd+B5HbAT1bzFG7/WVDSBgfKC1VFiRQVu+jtAfvavPOKKsTLi6L61+wpMukgQtlLfRYwH2WF650Q5dziu/MuBSnt4pFF/aOF80y06c+EjQTsToNMKVm/sUyBDJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iFu3zhxevD19T38OTBC4f/Ly8O9/qxuVANg6BKssmGw=;
 b=qjBPTzVsszUErkfx3PC604fELBLqTiWLRGEJxwa5vhz2zVDe8YjuUqMEo5T03dasgSu1OHVSk8u0zM4HKHxYxzmcyV+xuX4G/5zGWQZo+JKaptNER/Y5yk9lV14P/frTNt7Sz6SCw9LECqFvZxsnjR3aXNbEI7OMp+iVKdlmfRM=
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 1/6] x86/P2M: relax guarding of MMIO entries
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <9ab471d8-877c-b569-96c3-c3145800adcf@suse.com>
Message-ID: <4f794b1a-b1c9-2612-c3be-4dd710bef73a@citrix.com>
Date: Mon, 6 Sep 2021 16:54:44 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9ab471d8-877c-b569-96c3-c3145800adcf@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0254.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:194::7) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c140d6c3-d73b-4087-cf4f-08d9714ea906
X-MS-TrafficTypeDiagnostic: BYAPR03MB4805:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB48057D8FFAFBA71D118F9318BAD29@BYAPR03MB4805.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: aaATC9MZA1LgM/AAKxeZSEG+Tr2JC32fsa+oWOAYlewJKhnU9yA15ucyZuw3/ljBYMAd1JiNgm08rxqACIPEFRJkBdfKM12cWBEqgxbwQ6i1MeGcDEqaGzQQOy4zA9ZwxtHr2050HzZv8gxbM8Ulg19c0CJge87DwRa0TYHteyq87AB9PRt9DBLQv1foKkA68dAJXPZGjpzYWQ5mfHGOYg3G2KQWvYuwf7KlCJMiE9q+obAl3v2kWP3Q6HO1PibjcUA3JE8bCfYvAxLUb+Crml70YqyIlUiNbzD1A354PliX07fAwierV/umq12Rm/aw0+PQPYf1MxqBEQzpmlBhf8xjgiXOHnML7EZPGwdG+ykhjPXtjyn7Qavm1tHflSGSU74Bj5I8sw/Tg6uYVZpagcdxqOAydbvgJeM4HJXulUPYJ22gwVrwCHcu1Bwjg9jju1zVkq9o7c76sSBQurs/GECJswmkCbtFWp7FNWHrIUqc2Xnr6TvKB3PunqrFllztJ2yij9zrVTSM2dSYbKx7pUu/E+6GvfWB6/PwpSGGgYoLLr4Tib0kBPghY1cyzUuBlUtC5wubtntxKXDgBBzqBdP1QQ+NAdk7JFnSNG96EBe2aVe3GkgNS13LMScCBHY0m8qCL5UTGo7/+bzJx59hM0q4m4OMZ9bbmSZdAWL6arl69IcIBrnmrV4ezIL6UsNOc/DLYZddgcBOacqGtXtdCspYuU0N+ERIgiu4+r7ok8sVEk2Ws0uM0EydDKtuaJej
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(136003)(396003)(366004)(376002)(16576012)(31686004)(38100700002)(4326008)(316002)(36756003)(54906003)(110136005)(86362001)(66476007)(66556008)(66946007)(5660300002)(6666004)(31696002)(83380400001)(8936002)(8676002)(53546011)(186003)(2906002)(26005)(6486002)(55236004)(956004)(2616005)(478600001)(107886003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WnIwcjBCczdsSXExNVc4WmlCN0lCallFN3RPUjlzQUIwVGRSc1BNcGVUcTJr?=
 =?utf-8?B?OGJpcWZKd0Y1TXhVWWxBdGlPaGJ4aThVZmhMWjlDMHQvU3lZWlM3ZWgzcG9C?=
 =?utf-8?B?Lzh1US9EVW5PR1NIaGthd2FaUW9pTXRyTllkbkFGNnF1NEpZblYwTlBKV2pW?=
 =?utf-8?B?UU9KWjJ5ZkY0WGU2Q1NnbHdINlBUWHhHai9PMjFaMlh5WFNicmVHOTJjUTJS?=
 =?utf-8?B?czF5TWZMdGZxSU1UMWVMY0VVeUtmSHBHanlPb2xRNXl6ajZlMmVnOTJpVVU2?=
 =?utf-8?B?akwxcWhZWVp0MFdqU01iRGRIM1I4ektJVTRPYW9KVSt1bWNwcHgzU2cvSW1G?=
 =?utf-8?B?eENCYkJzdWlHMks5T0xnQnEvMFkzM2VJQmp1a21XUlE3RnVNcU9wckxJQ0Vh?=
 =?utf-8?B?c205WUpGMS9kam9rN2pZR3V0OFB1MjVuWk50dnV4aFJrNUNVbE9QWHJBTDdB?=
 =?utf-8?B?eklHRnZBWWxmbVFObll4MWVJUXZUa01PbExXNEFieGlEWmR4N0RSKzN4azdY?=
 =?utf-8?B?WDVBSFg1TkFxcXZaNDJocThSMEUwRXpIMGxYVjRZQU9tTzJINUtnVEdRc08x?=
 =?utf-8?B?V1Jka0YvbnY1TXhNblZQRkFZYmovNnVNdDlscEdGQW1wQUIxb3VuRFJobjRp?=
 =?utf-8?B?QXhOd0JWT1JFYzVTZGpzNithS3ZING9TVVpkcGxKem8wYjArSm5rT09zUXNy?=
 =?utf-8?B?QUdqSnZVbko2Mjhhblk2UlFJR1NTU2ZiQ2NGN3RyUitUTG9EVjFQMW9JclVE?=
 =?utf-8?B?US9VV05aVWQyUVJLTXdaMzRoclhsRGR0TkM1NWh1cCtBdWRDejY5TFR1cVJn?=
 =?utf-8?B?VWlpS0NadWl0ajBBczZkUzRNVG5IZ29mU0dKbmMrc24yLy9HdVpUKzB5SDk1?=
 =?utf-8?B?ZjFFUTJuY2RJTXBKdjVUMnVmV05xRS8xa2toeE5mRW9aUk5ZMWRPMnpsbmZp?=
 =?utf-8?B?QTlSS05wTVNIeGF0Mlk2R1VBZC9IL0dmNzl1THJVNWN4N3N2WWdzUXN2VnMz?=
 =?utf-8?B?Q1lCcWNrLzNJeTNacEtWUE5VSDk4QnZzRENrczl4cEJPeGRCOWt6NUJZTFkr?=
 =?utf-8?B?SGZ6VEVzaVVISWxPWjFjb0U5dzcvUjFDWXNoL08rZlBlc1gzYTFzT0tLeFda?=
 =?utf-8?B?QVNxREFadHEyVEJ4Vlo0bE1VZkRvREs0WGtIODNIcTZ4M3ozSm0vbjBON1ho?=
 =?utf-8?B?NzZSMjNYT0RxTGM3YXNtWDVzVjBiS3FpTWo2MjZFM3Q4RGRMckVkLzczaVVG?=
 =?utf-8?B?Z00xbk5lcCtodDExS0RaM2ZRalh6YnlKOFYzbGlVV1Nub04xd1lXQmhKTEIr?=
 =?utf-8?B?YW1IQ2d0ZmNMRVJHQWJyMnFuTi9WL240Y0U4M1hNb2c4N2NMeExIcUs1Y0dm?=
 =?utf-8?B?ZU9XUHRIZEFUd01EdTg1N09JQmJ2VkYvcG1Qb1Q4Vi9MTDBXM1ByWGxSTTBl?=
 =?utf-8?B?aDVDM0FTNjhoc2hQdm5mV2FETElmeFdvRkRGb0RQbG4rOXFza25pSFNEZk43?=
 =?utf-8?B?SVF4VWhDdklBYkNDdnUybHZDSFowTy9FaW05MXBOakE5eUJER1AzbXlPSk5L?=
 =?utf-8?B?bEpnTFNFK25oSzgzWFlOVDVMY0ZGSVFzU3VnSE9pSzhHVUR4SGttRVVtZmtu?=
 =?utf-8?B?ZzNKQ2tNTE1qS20vSjhFQWtJejlKamM5WmNrazBSbnV3ci9FNjZndXBJaE5q?=
 =?utf-8?B?dlpDaGQ1dVdkbEo5cXdKOXJBU2lCTDI5UEdXbGpoNmtPblNTRi9HYWFPZU9K?=
 =?utf-8?Q?XkWY4NScNXe35bWQvg16/TtQGBWsABp57j/V5yE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c140d6c3-d73b-4087-cf4f-08d9714ea906
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:54:50.8268
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7E5OilklUTMU6p1bHGzlaiG9gClfm14WsSVQLxWI6Iz0fdqwzFA6/NePEb7GZtZOfWTTL5vdJvL5qjbmFS8FkJj+GiEoiKJMbLZKw1AKb2Q=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4805
X-OriginatorOrg: citrix.com

On 02/09/2021 09:32, Jan Beulich wrote:
> One of the changes comprising the fixes for XSA-378 disallows replacing
> MMIO mappings by code paths not intended for this purpose. At least in
> the case of PVH Dom0 hitting an RMRR covered by an E820 ACPI region,
> this is too strict. Generally short-circuit requests establishing the
> same kind of mapping that's already in place, while otherwise adjusting
> permissions without - as before - allowing MFN or type to change.

"Generally short-circuit requests establishing the same kind of mapping
(mfn, type) but allow the permissions to differ".

> While there, also add a log message to the other domain_crash()
> invocation that did prevent PVH Dom0 from coming up after the XSA-378
> changes.
>
> Fixes: 753cb68e6530 ("x86/p2m: guard (in particular) identity mapping entries")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I may have gone too far by allowing "access" to change for all special
> types now.

I think this is appropriate.  After all, it is the pre-existing
behaviour, and the type change is the important thing to restrict.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 15:56:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 15:56:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180076.326594 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGyd-0002bV-9s; Mon, 06 Sep 2021 15:56:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180076.326594; Mon, 06 Sep 2021 15:56:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNGyd-0002bO-6a; Mon, 06 Sep 2021 15:56:07 +0000
Received: by outflank-mailman (input) for mailman id 180076;
 Mon, 06 Sep 2021 15:56:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WUJr=N4=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNGyb-0002b8-Ro
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 15:56:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f0b55398-0f2a-11ec-b0ab-12813bfff9fa;
 Mon, 06 Sep 2021 15:56:05 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-va3uDO4RM2y9I5DnaNZr0A-1;
 Mon, 06 Sep 2021 17:56:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Mon, 6 Sep
 2021 15:56:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021
 15:56:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR05CA0094.eurprd05.prod.outlook.com (2603:10a6:208:136::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend
 Transport; Mon, 6 Sep 2021 15:56:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0b55398-0f2a-11ec-b0ab-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630943764;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=g3VIIzhR4VPWUylr8MJh6UdRLw6fQBScFUBQK+lSfh4=;
	b=Y+c/YjHDXnFGZS6F7wHUpPP3RjH7n9hDJZQezId5NEoOxTHvDmZp+5hlkgGnEppu1J5NJd
	fqSMreL6glZmbstkyWwC9CVb+mUf9w76up56vaShGpMmX1OCMu3+giz098E3OnFJAQ9oug
	lkusKzhV63UPq6aGGJ92zBCH8V7bALo=
X-MC-Unique: va3uDO4RM2y9I5DnaNZr0A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Sa4U2cSGjzNInatLJPKW4bsGiTjipHz3GBD4+9hjcxgNXYMpX1Ujz77ItvoB69KvGjZEj6rTZ8vqqB7uGtqzWc2RfIrQdMBfNBn7ItH2q0Ya4FE3EVxgMcg727J/7tUQJYp+1U2O9txHEJPEZv3/YIqpV3RBdUHot1GGeIzZ20cbxk5sADfDMjkxKk277aonkZtWtYH7SDas2qWrPYq8Fz7/6VOpGyqq/hl1zfCBgebk+u7aXfLRgDpAD+seGftOs0yu9aG2nCq4+M2nZv3+WuzeMSBiZWVrEbAswp2HwfXVhL0AnnMTPgv2I6qH8hA+hRnRO3xSInDKhH0dhsuJHQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0eMpjxEXhNm6omFEl8aBZ2ChtVfNPks2EaXMUmuJAqg=;
 b=oW2mU8h4+nJF+7gaYSuD086lsqKeg0UyEeQEhKIrVOLWyy3/xqbH8KL91jiQAsCM3picT6K9N0harL+UWaSPC5REiP8k90Qu9LJjNdLs+J09qqUVv6+qdkkPuD0gtJZXTMTBcAAjjHFDsaJX2T67XCL7CnCCbfYrwnRW3u4Owx374Iuvxb84ssXCorTTJ5J9d4H6SqofSXt6WLsEUPepd71TpdgDaS6vuCUSkWXT/XWjYpJGh/n8i3TFDFBfoTTqTPNgBmrtEuwGL42izeARRAIc1zA1wdjwiCiD36O+zwYAM1zUcRAvmHnzngjlSzlS5mASN9Ik9UHktr4hxu8KtA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 2/6] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <457637e2-d35c-18d0-ead8-0b513b257927@suse.com>
 <9b7936fc-f81a-d5e7-5ceb-6e10588503dc@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <28646a17-420e-bfaf-8f7d-6d109818ffc3@suse.com>
Date: Mon, 6 Sep 2021 17:55:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <9b7936fc-f81a-d5e7-5ceb-6e10588503dc@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR05CA0094.eurprd05.prod.outlook.com
 (2603:10a6:208:136::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7ff10b55-8923-4283-84b1-08d9714ed333
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41901BA90D360869B8127B9CB3D29@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XCUvxD2aG0bcXQbCaYJIUTp+ZbB9g/stXeR60kRa1vr6hihvPRLyiZq7p612wxmA/KlPBuIqBk3F6m+9g4LJhS6V8+l8mzkzI0WhHTBPis32j/MqzUtySbYUKW4wnffCbcbocABdNmIyKd4kV3hytAt9SrV+TtAZ5qgkYA960GqAANmrWkljwUN7AWahnbfHGje2Ddmj63hTfNKJGbRAecETR8kOJe/O/q2LqrEY8T/T1gm7/hRtW9R2Z1KTfv/NBbu0Lnym3R7hh8e6GUF+OEY47I6rlRlaxCm3MwRFkAfB0m4cmgg8p/37rZaXKg4LWyWI/dQxv5KWbgPaEZ4KbQMv6TYHskJbMHGwdD6nywsXzvwyn+Xxz7Sro1clouyj5wI3lCGDlyySP2UidHfocp8vg3MIHnFwio6p/GqVX5cUOHON/br0zKN2pYPiMnrDfPj/p2zY00vhKTo0NH6iYBhMMSy75Ogi0pAhkmvUZ+Ci9IGRpVJICFVlH6Tbj0hzN6A/QYaO5Km8Y9f+96Xjpz8vQLE4GtxFYRdhC0eFkikLxckosR981Oez2iyOw+Tka2oVnpb6SL9dsZIYuVYJ2NSWVxMN7HosZS+tyod3nqqLBK3QX8A4hNZxiubotrUeD6AYpC1aF/vfVSNS0jYfhuG5d8pj18rdG5LLEFciLBgwbZPgfl9sySc/c/F+ve75S5gOvf2lhpo2n9q3Shj4Ez+UYxYCwgw1iToV0ExYLPo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(376002)(346002)(366004)(39860400002)(36756003)(6486002)(53546011)(66476007)(66946007)(31696002)(6916009)(186003)(4744005)(16576012)(316002)(8676002)(54906003)(86362001)(31686004)(5660300002)(2906002)(4326008)(8936002)(478600001)(83380400001)(26005)(956004)(2616005)(38100700002)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?heEbXrQDAcKTe2mtwvp09Q5xqcIgKsXCaSr1gHO/f19hUXxJPOtdTG3yjL4G?=
 =?us-ascii?Q?Rl8TFZn85zWCdk8FXs4Tqpe+lfEHa8um978Zkd+e585aMDkuiXI1Dh/yeyWG?=
 =?us-ascii?Q?c+CcGWJD8HtUoWmEelD2gRDltz++lYYomjzTU7ck955hWl9WQgOTHoouFfLe?=
 =?us-ascii?Q?N+pBHx21wEcrQ9hjjD+vGwATzBxfAV0RCa8idSwusGo958Ql/9rSQIUnNQTt?=
 =?us-ascii?Q?43YzxQ4sTbAp4ZjkH5MEnap4xVMVpcUPwSw92Z1LNwndSsau+DUPm0WDlLaE?=
 =?us-ascii?Q?AT+pUHW/+kMWb6UAaoJRsGmN6U/yOvykzRiPYiQiLIavNyGnqrN/ysHmVeOz?=
 =?us-ascii?Q?je/7h8IWOQ7fO4AYuVnB48295rYysJHNMp53mvj0CS/n7fzDDtzrbjiqCei6?=
 =?us-ascii?Q?xlHd29A/VROax0zwq0O4h9GTNPoPJ9OlW5XRok2b4qimP/2rmFIcBMm547WF?=
 =?us-ascii?Q?pNAt2PQkywgCB+A49j89uGOsjl6bQEPmYF/e+K7EW/T1To0eTnAhlD915oRp?=
 =?us-ascii?Q?BDwkgWyIPM9EfxLNm5GNw/d5xu1h4O2MyCQr5BuTdp3Wets/5RazOC66OtuR?=
 =?us-ascii?Q?CXbGH4NpfWSdiVC64nw0MocS/sSZcANQ0nrLmJjJ+nZRPfdJb7QkqQkyJr86?=
 =?us-ascii?Q?uXZB7mQb02J4aj3bOBnOIUH13v8jweFIYUfWMIdQxFTRm5SFLqYGY9CQBHOB?=
 =?us-ascii?Q?C++jQVi/EdW0+EvhboP1mVwuVTplp0W7Atyz1/s3541WTWjbKnUbj7D0jDns?=
 =?us-ascii?Q?LWuoKNzbjZQ3MojuOLTnbmq9YETWYSX9qDx7pUMI2Fi22X8WEhpzKZ5nIr34?=
 =?us-ascii?Q?QEZUZ33tIKJ1DUHDfwZ/60CwNbkV1BKSkhX9t5loTZIy2lq34Nm/ei1YcI1F?=
 =?us-ascii?Q?Jv5U/IQq/Opvso4NumnlGXeQyurLWRXnYHZmCQqe4TdMaNqWNoJunFcxGTgs?=
 =?us-ascii?Q?cr/HZQXemGW5heWty/GFrjjOHPGlo3vIp+Ka5Jo1LeC/U6QmAQVsYFXNLUV/?=
 =?us-ascii?Q?672GvoKZsWJc3y7PS8k2tvwwZlHP/pJopI2cStxRktutI1+gL37unQ350Uyu?=
 =?us-ascii?Q?Riq/bVb7lKRib8j8+86JfQhWtJxxWz80egFsQvGuMjc41Nrb0bHYWpdvz+d2?=
 =?us-ascii?Q?lgN0LjoJ2Z+Za+0QMKOMk1Nkuvp63+JTVwE0NDb70jq6W+cc/1ZacmqiBUtv?=
 =?us-ascii?Q?/7nlGBJPf3Vzb0+ks1O8zdXEcpO1V776ATJrbbbNn0X1CYQY5betv7fWbBG2?=
 =?us-ascii?Q?3shhpdefCRkb7fuI0CsX+3y07e2CnJ86AWRALnxtmWbhhnRoa9YXWfCBgtw+?=
 =?us-ascii?Q?1kxNZMrfUbtU/luvoYj6/eAD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ff10b55-8923-4283-84b1-08d9714ed333
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:56:01.5571
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eS0wVgOD70J+2WIq9vpMJ7GEhlL/Ed0NPhAWtm7ixfCso9bOM/2klZ+IpYMgXJDCYM8l2KdHA66ZYvMgHrlOBA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

On 06.09.2021 17:48, Andrew Cooper wrote:
> On 02/09/2021 09:33, Jan Beulich wrote:
>> To become independent of the sequence of mapping operations, permit
>> "access" to accumulate for Dom0, noting that there's not going to be an
>> introspection agent for it which this might interfere with. While e.g.
>> ideally only ROM regions would get mapped with X set, getting there is
>> quite a bit of work.
>=20
> ?
>=20
> That's literally the opposite of what needs to happen to fix this bug.=C2=
=A0
> Introspection is the only interface which should be restricting X
> permissions.

What agent would be handling access violations in Dom0? The description
(now) focuses on entirely Dom0; I agree that DomU wants things the way
you describe (provided all p2m_access_t abuses are gone).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 16:14:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 16:14:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180085.326604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNHFr-0005xs-VI; Mon, 06 Sep 2021 16:13:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180085.326604; Mon, 06 Sep 2021 16:13:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNHFr-0005xl-SP; Mon, 06 Sep 2021 16:13:55 +0000
Received: by outflank-mailman (input) for mailman id 180085;
 Mon, 06 Sep 2021 16:13:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Prhc=N4=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNHFq-0005xf-Rz
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 16:13:54 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6dc0a926-0f2d-11ec-b0ab-12813bfff9fa;
 Mon, 06 Sep 2021 16:13:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6dc0a926-0f2d-11ec-b0ab-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630944833;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=sPS1iR3/1qB/kh7mRnkHpyZCa1OlC8FRIQTogK8xtOU=;
  b=IOSLeBP5dKMNpqHtA6oyRJZuNh5pCtw9WmIruSVHXuyvlCxcb/BevhcA
   JR+PVzXxI56LErIQrsnlqrSponcidzUDOMjryeEBdMn/Ua+2le4q9q4uI
   /LrXYPC2WV5UUuvtrHHMky2iNqqqTvS4oaRnTj8B9yg4T9PaO6HIm6diG
   M=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: PAneNPwE5aDq+Ll/CpoUPWdhhv9tg4/BUy8AeMc0O2PUU4INBbhv5Kz40Hb/zEJtzX+XAoc7gJ
 O9ERxtzFxcj6M1tVuvfoztfdcc/8gIo2gvWq/zckgyvjznb38WzOdn374IJDFgw+Aa2XS1OG/V
 07XxUl0ltJ0qf6p7fTkzbaA0dH+ZZURnXnyGfBUKg+VSbi5A/gMfk6whFAdXAuDwhl/tngPSp8
 Wpq1PdQGMnTgTGZVOf7/6fuMNmlztdlySu217xAvpDov71s9kfSy6u+ulo1g6rW/1+Cmm7O3dR
 S76Ima7LCUdkkcsCUtYgTuOf
X-SBRS: 5.1
X-MesageID: 52111338
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:c23qnKDy7XGChmDlHelo55DYdb4zR+YMi2TDt3oddfU1SL38qy
 nKpp4mPHDP5wr5NEtPpTniAtjjfZq/z/5ICOAqVN/PYOCPggCVxepZnOjfKlPbehEX9oRmpN
 1dm6oVMqyMMbCt5/yKnDVRELwbsaa6GLjDv5a785/0JzsaE52J6W1Ce2GmO3wzfiZqL7wjGq
 GR48JWzgDQAkj+PqyAdx84t/GonayzqK7b
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52111338"
Date: Mon, 6 Sep 2021 17:13:48 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 00/51] xen: Build system improvements, now with
 out-of-tree build!
Message-ID: <YTY+PNBccQYPhX0a@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20210824105038.1257926-1-anthony.perard@citrix.com>

I hope this is useful:

On Tue, Aug 24, 2021 at 11:49:47AM +0100, Anthony PERARD wrote:
> Anthony PERARD (51):
>   build: introduce cpp_flags macro
>   build: use if_changed on built_in.o
>   build: use if_changed_rules with %.o:%.c targets

all 3 ready to commit

>   build: factorise generation of the linker scripts

may need a ack from arm maintainers, does it? Otherwise, ready.

>   x86/mm: avoid building multiple .o from a single .c file
>   build,include: rework compat-build-source.py
>   build,include: rework compat-build-header.py
>   build: fix clean targets when subdir-y is used
>   build: use subdir-y in test/Makefile

all 5 ready to commit

>   build,arm: move LDFLAGS change to arch.mk
need edit commit description, but otherwise ready
    not needed before [PATCH 21/51] build: set ALL_OBJS to main Makefile; move prelink.o to main Makefile

>   build: move make option changes check earlier
>   build: avoid building arm/arm/*/head.o twice
both ready

So up to here (12/51) all patch seems to be ready to commit.

>   build: convert binfile use to if_changed
>   xen: move include/asm-* to arch/*/include/asm
both need a ack

>   build,riscv: tell the build system about riscv64/head.S
patch ready
but depends on "[PATCH 14/51] xen: move include/asm-* to arch/*/include/asm"
    for the rune written in the commit message to work.


>   build: generate "include/xen/compile.h" with if_changed
>   build: set XEN_BUILD_EFI earlier
>   build: fix $(TARGET).efi creation in arch/arm
patches needs review

>   build: fix arch/x86/note.o rule
ready, standalone


The rest of the series needs review.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 16:23:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 16:23:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180113.326615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNHOo-0007fV-Ss; Mon, 06 Sep 2021 16:23:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180113.326615; Mon, 06 Sep 2021 16:23:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNHOo-0007fO-Ps; Mon, 06 Sep 2021 16:23:10 +0000
Received: by outflank-mailman (input) for mailman id 180113;
 Mon, 06 Sep 2021 16:23:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNHOn-0007fI-7T
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 16:23:09 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0a42d75-f702-48f0-996f-eb4c9c6b15d5;
 Mon, 06 Sep 2021 16:23:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0a42d75-f702-48f0-996f-eb4c9c6b15d5
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630945388;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=u1wCfwPEvija+X5nLmaV5L/jHHDMUlypZ4kYBWYJFLc=;
  b=UVeHqI3PbgBIWWRqVlPlJ31Wg2bNj4pUzJq9e51VR/Hxfp9Od9/ivGVF
   4M1vEyqsyEM4LtZJwCxRBplEki+J32prWw1y/d4ncHZEKCnM8WKnsHZdy
   keEY4DDPFFpe+4hWImhU9xfQlIfE3+xshbx1B9cXjaOBJbzzcsmaPurV0
   U=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: aITdVDxBJwvPSCi6ug98L7fiFhUqZAEmE49LuZRb8tAR+VQHXlYJYu6vDfuMmiDcdZxk7S2VWY
 1l6zwaYvqJlHhUOb/8BX9AaRxvZHvSdlZWAyXaNGnTLP/59iSbaB6oQR04NbQJUkMX+g58peyN
 jIPD8T9zJLxCAjtgSqEPLKuK4lDj+wjb7e2DWVl0MrjATjWYMlXxT102P/DqGcPyNuk60xSGXJ
 TVGNBm73WHkwzeNY1KkrNreOGkmLc3BZ13QK2AaOhxoGQw/Tho1qzul7gpxLVllFtSZwL47sCm
 2ZuJWT1uuiWLInjdkfKuYXjY
X-SBRS: 5.1
X-MesageID: 52087671
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:zXrAXKx/Q+h0wKGZ1WfOKrPxu+skLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjkfZquz+8L3WB3B8bfYOCGghrUEGgG1+XfKlLbalXDH4JmpM
 Bdmu1FeafN5DtB/LbHCWuDYq8dKbC8mcjC74eurAYfcegpUdAF0+4QMHfrLqQcfnghOXNWLu
 v/2iMKnUvaRZxBBf7LeEXtEtKz6+HjpdbDW1orFhQn4A6BgXeB76P7KQGR2lM7XylUybkv3G
 DZm0ihj5/T/c2T+1v57Sv+/p5WkNzuxp9qA9GNsNEcLnHJhhyzbIpsdrWetHQeof2p6nwtjN
 7Qyi1QcPhb2jf0RCWYsBHt0w7v3HIH7GLj80aRhT/ZrcnwVFsBeoF8rLMcViGcx1srvdl63q
 4O9XmerYBrARTJmzm4z8TUVjlx/3DE4kYKoKo2tThyQIEeYLheocg050VOCqoNGyr89cQODP
 RuNsfB//xbGGnqL0wxhlMfheBEY05DWitvGiM5y4uoOnlt7TFEJnIjtY4idixqzuN6d3FGj9
 60epiA2os+F/P/wMpGdZA8qPCMexnwqCT3QSuvyGTcZdM60k322urKCZUOlauXkc8zvdYPcK
 qoaiIviYd1QTO3NfGz
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52087671"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KOO1SQlkNfM4CFHZATNIChtWTYk0nmSUW/JPaop/kFLLa3t8LA03ZmrL06/alYXUrmgoiOZlPgValUNeyagmNkDnLrzAd4pcxEerdJ7NGyj2oK6FrLBVUkiG9/4pIJJu3z+YZIar57IvP0009aBjOf7e+OqHOctWTW8hbSVhNnhzjTfHp1szmO3Yw7KkU1zkQOjWvCKghGcgElGfL3/EvhAYTaFwpML0KgDvtZUjUaUecQnGFvr5CD8sIsXNVqU4rYK2NblAKKr5nkcS94G3PC8dRvacbGSxJJ672c1GWBcxxDOJq8/FznaG+hgJ8X7UnMzfqcvEKVYVOQXIjQKtZg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NAWywoEhPXo8wkSkHOONStkbpBdXvcsyCBc2lneyBb8=;
 b=JY9L4atVHGv4TcCF0173og2a5Jt4HI92+8mWy+xYjUxC6tp5MH/2COBVn63AVApXqr49LmtG+UBjJMN7npXtIl08eoCLjnQ6CziWiubRy2kG7cIIMtvwsNCFGHm/7noK4v1VDK7Dz45qICgD4yoz07TmrqRJICc1hHz0e4lTTNvBKxqgi1KDvxnGFG8cf2VaprdTDRwlrfVJWtSAUczNBucP9JBbEeKdvxON0gh4XuZX63p/hgZ9fvBj3WAYcGabMMDx6jifPoRnGkC//uODfk4gMmdJCRgHy3GJwVQGG8Pb0RAS61tM1Y5lJS1P2txSZtiKgReLjEjmBCIQkFRpBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NAWywoEhPXo8wkSkHOONStkbpBdXvcsyCBc2lneyBb8=;
 b=Lvk3Pn0XCfqaIefxTbHUGsKn+aUC3NsmIDeX5NOcNoubMPyZrkHYicQOAJ2CZ6rOoIgCOKNm2mXP/I1zcnLQOL0PqNsA1nQtyfo8dE6PnlwKdY+DZvLku7YMQXvW9do/LYFCh1UKokpbhnXKsjeniGxz0xX7LvZAbvs26wjhf/g=
To: Jan Beulich <jbeulich@suse.com>, "Daniel P. Smith"
	<dpsmith@apertussolutions.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Bob Eshleman <bobbyeshleman@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>, Connor Davis
	<connojdavis@gmail.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
	<xen-devel@lists.xenproject.org>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-2-dpsmith@apertussolutions.com>
 <3a91e4ad-50c4-205f-3d90-f23c996b2938@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 01/11] xen: Implement xen/alternative-call.h for use in
 common code
Message-ID: <8c3dfbf5-5c98-c923-ca81-28af3d13c4a4@citrix.com>
Date: Mon, 6 Sep 2021 17:22:58 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <3a91e4ad-50c4-205f-3d90-f23c996b2938@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0073.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:190::6) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ed4c017-7bdb-482c-0433-08d971529a9d
X-MS-TrafficTypeDiagnostic: BY5PR03MB4999:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB4999EB93616B75F091DFFC13BAD29@BY5PR03MB4999.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2803;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: S8M8Pzy2Km16DGbI2YEXgFY/yzi/jufCOAVbZw7CZi9T/6st6i5gCw0vTIRSsYEVsEr4RtoWiZqRPhGV63bRIi1QDDAHeuuunQ0d/hZ3NvHEm3hGmXHS0bN4OsVYmMJU4GQGboyljTX+lDF74Qe9ANpJTfttvABUbsT8GAy5k2dFhzfw4DWPxkypv1mRB2xqAlPrJVJCGYv9UADyiv+y/d+wITJES2DRePZPMF1KHmXD+cAUFTXPAiwutcmprWq2JSTSq2cO+Gqe3UjjXvtAJAaerITQ9tX106avc/Ij0XYmOZ3juBrZ+dECfSUS7X21tKTHSLyfPaBefaZuscrvKZDzayF60/Q+SjO0m9n/yRC0XXbaWUaOGO/YjtG7nhtKjQuVP8ymkDjhR3z5i3FqDB3bzIDf+gNU9pWfVZUns3VM4YwXUW9zPWpM17bwJG4JKQU8EeENNVfyFYq843EVg97z3UR5tSNE8BnhskIv8ttF8a4gU+pQXvm95HX88jQm+YW08SEO5K1V7ovvqm/htRw6S4NYjqXt43A4KpsfwCVItGnnqbV5xRZchQxAAisZNiBQOb1/2pFRoJfAvCItL4CymOo7emov9CQch4SBJDO/Qy1dF0PVhHBE0qhhfWDOCuqq+KgZIo0ov/v6uI353UR168YKGbYNgXUvSIPWfw40wXBIrq0BJaS9sWbqb8YKR8cbm73ZKd7MdFutNDk1vFbJfKOpnYLi1DG8oV8ga3rE2pM6+Png7Fe6OvWkiau2yQR+/AnqtKefyXkvzGpn/knDkQFjg2UqMjTxCmkchTtPHG4SaDJdqMm4SDAsySgk
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(366004)(346002)(376002)(38100700002)(6666004)(8936002)(4326008)(53546011)(2906002)(6486002)(66946007)(5660300002)(26005)(956004)(966005)(186003)(478600001)(110136005)(66476007)(16576012)(66556008)(2616005)(316002)(36756003)(83380400001)(54906003)(31696002)(31686004)(86362001)(7416002)(55236004)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TTJuOFNxbklqSzZMZ0Vjc2Q4cVlHNytDbGI2WTlmT0JJVURxVVBYUHBQRVdG?=
 =?utf-8?B?YTdsWkVtb21yVGdic0tqZ2IyV2M4eFFIL2I4QUs1KzQ1b0xOY1FVMVNsUFEy?=
 =?utf-8?B?K1hlUHJ2K1RBZmw3RTc3UFRIdWJUUG42SkkwdEN6Wk5raGRBTUFkSCt1OTh1?=
 =?utf-8?B?QUNub2pLR0VLNm5qcGtQRkpmK2I0aVM4SEZ0YnhDWHFBUUtuWTA3bDVmNGdQ?=
 =?utf-8?B?NXhXWERVVWZBNFNSQlU5bE1lN2NoMlpaNVBaZEh3c2lLODMyU0wwaS9NbHRL?=
 =?utf-8?B?bGVzTzVVNjJHTmc5OE5mK3VTVUJmejdrMDZYRE5COGxKWE1tbnNUZWhZQ1Ju?=
 =?utf-8?B?U2NmSFd0OEZxdG1PQ3Z3c2JyVUJFdi80UXpRSThQbTh4am1BQXZweXJhWE9x?=
 =?utf-8?B?Qk9saXRSZzFXN3F3NmdOUWlKR096SVVvTWhHS2poZFhMb1JRWldHSFpiREtp?=
 =?utf-8?B?R3JoV0NUQ1BNZDJ2czNqZDFLRmk3Snh0ZnRocXF1THlIL25iYmhMUXU5NHc2?=
 =?utf-8?B?TnNzUVhyRVpjOWpQdHNHRUNObFhtbVR6aHE1UEg0aGNCUFVXRVJOMG9wSGw2?=
 =?utf-8?B?T2RlRE4rM0k3bkRnVkFmRUZXQ3EzQTdMcTJtNEtpK0FCdmo4dzhjZVJmY1A2?=
 =?utf-8?B?eUs4QlF6L052YUk5dWRKRUZYNFNTbkZUdWZ1TlhEaDhBdjRsaFVTNHI5Yzl3?=
 =?utf-8?B?U1ZTVnJWZVkxa2Z6cDdqMi8rSjVnYm16TVFBQjA5Y2xzeFJpb1FjUzVUTEt5?=
 =?utf-8?B?Uk0zTGlnVVZWRHJyaVJFdkVQdmdZczYyY3NtVmlmaUdzeVBKWWs4akxoZk04?=
 =?utf-8?B?TmFkNDZWNzJlT0E4Z2Zwak9jQ1hZV0tVOUtSaXlxdlBLRHc2NURoV2JrekVI?=
 =?utf-8?B?b0RzRVVaV1pxZ1ZmMGE4bXgyQ2JDK1R0UzhHMHlVRFowMmdDaWExTERnc3dV?=
 =?utf-8?B?UTVNN09NeElyL3VZNmd3eXQ3R083dVdtVTJXR2JjZ3VWS3ZCNnJvM3BFVE5G?=
 =?utf-8?B?citGUkNKNzRaL0svdHN2RGNZQ1RmSm9wSjdIUHMvdElTdEYrSWhDaHZwYlU4?=
 =?utf-8?B?SGo1RkgwQ0VlaFc1REhTOERGdS9IV1Jrdmg4NitqVXRJY0dNQU5wSFJlUjVC?=
 =?utf-8?B?KzFDMHpiNFZ3cTlueFRnU3RGbjRzNG5nbjRMZXJqMW9kYnV4azVOR1Z4Yldy?=
 =?utf-8?B?aWNZc2x4Q2VEeVFhTGJYYm42WG5Xbmg5NU5IT09vYlZIR2hWbW9ibEVHU20v?=
 =?utf-8?B?VzR0VE5Uc0RSTjRDMFRxVm1FYVpyWi8xSlpISHFEWTVHVmZOemNPQ0RKd3VU?=
 =?utf-8?B?YzRya243V2VHd01JOE56aC9ZSFJjZzliZUlrckt2QnN2N2U3dGhTbEk2dGhU?=
 =?utf-8?B?bDcyTDdWcmJHYUZ6L0c4MmtLZUMvUENjNzFqYWdmTHVPMUlEaS9DZE5Vb0pX?=
 =?utf-8?B?Y0RkQmplWWFVeUF1U0xHMXlxU0xENFgxekUvKzRrL0xZMGJFcXg0RzNFUE5z?=
 =?utf-8?B?cjRIa2xPeVFNYlkxbit4ayswSVBhNURISXVRSTYyRkhKdXpXZkpybWhwSFVt?=
 =?utf-8?B?RHhHRzRSNDFEUnVyRUZjSHFaVmE5Y1JXa0w0dEpIN0NVY25GV2JVNmFMRW03?=
 =?utf-8?B?bWhYMWNwLzA0a1BJY3l6QnhLSkdFbFhwSWtudjRpa1h3ZDZ3bFNSZzZJZW1V?=
 =?utf-8?B?WldGWGhiNGN2WmZWbloxNFl0ZmFlNE9aQVQwR0EwSVhkYVlwWkJjbXBXOTdE?=
 =?utf-8?Q?pRr0iqgym/G0s25DznuxOH0eM61WXZ4kRCRwAYD?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ed4c017-7bdb-482c-0433-08d971529a9d
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 16:23:04.6363
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: b8JpsDCd1aPMHBe6+AtHdoddaCrt7jQ8dqi19yI4P5bWk2gA/jBNHnMtBPPaN0pcQsI9BXBdrquNkL4ha5QVNHwCIm8k2LEqK+IiD7IQ+vM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4999
X-OriginatorOrg: citrix.com

On 06/09/2021 16:52, Jan Beulich wrote:
> On 03.09.2021 21:06, Daniel P. Smith wrote:
>> --- /dev/null
>> +++ b/xen/include/xen/alternative-call.h
>> @@ -0,0 +1,63 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +#ifndef XEN_ALTERNATIVE_CALL
>> +#define XEN_ALTERNATIVE_CALL
>> +
>> +/*
>> + * Some subsystems in Xen may have multiple implementions, which can be
>> + * resolved to a single implementation at boot time.  By default, this =
will
>> + * result in the use of function pointers.
>> + *
>> + * Some architectures may have mechanisms for dynamically modifying .te=
xt.
>> + * Using this mechnaism, function pointers can be converted to direct c=
alls
>> + * which are typically more efficient at runtime.
>> + *
>> + * For architectures to support:
>> + *
>> + * - Implement alternative_{,v}call() in asm/alternative.h.  Code gener=
ation
>> + *   requirements are to emit a function pointer call at build time, an=
d stash
>> + *   enough metadata to simplify the call at boot once the implementati=
on has
>> + *   been resolved.
>> + * - Select ALTERNATIVE_CALL in Kconfig.
>> + *
>> + * To use:
>> + *
>> + * Consider the following simplified example.
>> + *
>> + *  1) struct foo_ops __alt_call_maybe_initdata ops;
>> + *
>> + *  2) const struct foo_ops __initconst foo_a_ops =3D { ... };
>> + *     const struct foo_ops __initconst foo_b_ops =3D { ... };
>> + *
>> + *     void foo_init(void)
>> + *     {
>> + *         ...
>> + *         if ( use_impl_a )
>> + *             ops =3D *foo_a_ops;
>> + *         else if ( use_impl_b )
>> + *             ops =3D *foo_b_ops;
>> + *         ...
>> + *     }
>> + *
>> + *  3) alternative_call(ops.bar, ...);
>> + *
>> + * There needs to a single ops object (1) which will eventually contain=
 the
>> + * function pointers.  This should be populated in foo's init() functio=
n (2)
>> + * by one of the available implementations.  To call functions, use
>> + * alternative_{,v}call() referencing the main ops object (3).
>> + */
>> +
>> +#ifdef CONFIG_ALTERNATIVE_CALL
>> +
>> +#include <asm/alternative.h>
>> +
>> +#define __alt_call_maybe_initdata __initdata
> My v3 comment here was:
>
> "I think it wants (needs) clarifying that this may only be used if
>  the ops object is used exclusively in alternative_{,v}call()
>  instances (besides the original assignments to it, of course)."
>
> I realize this was slightly too strict, as other uses from .init.*
> are of course also okay, but I continue to think that - in
> particular with the example using it - there should be a warning
> about this possible pitfall. Or am I merely unable to spot the
> wording change somewhere in the comment?

Such a comment is utterly pointless.=C2=A0 initdata has a well known meanin=
g,
and a comment warning about the effects of it is just teaching
developers to suck eggs[1]

~Andrew

[1] For the non-english speakers,
https://en.wikipedia.org/wiki/Teaching_grandmother_to_suck_eggs



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 17:07:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 17:07:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180126.326626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNI64-0004CJ-Bq; Mon, 06 Sep 2021 17:07:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180126.326626; Mon, 06 Sep 2021 17:07:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNI64-0004CC-8w; Mon, 06 Sep 2021 17:07:52 +0000
Received: by outflank-mailman (input) for mailman id 180126;
 Mon, 06 Sep 2021 17:07:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LYSO=N4=gmail.com=rjwysocki@srs-us1.protection.inumbo.net>)
 id 1mNI62-0004C6-MP
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 17:07:50 +0000
Received: from mail-ot1-f45.google.com (unknown [209.85.210.45])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f6eb9e84-0f34-11ec-b0b0-12813bfff9fa;
 Mon, 06 Sep 2021 17:07:49 +0000 (UTC)
Received: by mail-ot1-f45.google.com with SMTP id
 m7-20020a9d4c87000000b0051875f56b95so9471479otf.6
 for <xen-devel@lists.xenproject.org>; Mon, 06 Sep 2021 10:07:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6eb9e84-0f34-11ec-b0b0-12813bfff9fa
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=++LIn4BzMT/yfmvrIHegIKxrKLsnvbzGkInouRJvYyc=;
        b=CokLebOuvCoJE4SwzVfhnpNJLez5olRg6cY8guhMYST2+xtcuOhn9gOrGTmKNBiXYq
         2lIUy1t3Qpr9lujq2Ux5CDSV6l/07ylN7DaD8pStMUBwZySwCq38+9pEWKfwtZgZ9Btq
         LCGiCkOOXozqkUbyH69kTt4wHrQYYuejV2YYvJpT6ZS8K2yfBY8w/BsDaYe0rVngEQAb
         OlzNE0BpEb0OXqHtUZ0Rgb4Ye56PyvfwUyZlzQEzHyLCSqVPHZUGgxMaVr6ophtvRpyT
         Ct0LPhOvI3Yasw7bTp2gxAgsybNm9pY5O1RxX8dzFu0GSX27wVSC46bmRqBQzV8Xqn4W
         1Gmg==
X-Gm-Message-State: AOAM533/DDBU4FFnz44WAx6aYHbFFwl3uPsfBxoUXZRC7T9XxDKH9LVa
	DDgZJOiNjpkPI8GFk9eOhuyUoG86/bOimuszkps=
X-Google-Smtp-Source: ABdhPJxFe+shqvt53iInVBtVbEwgHxwTsvQqM7BZ8tlMMEWr+cag6cxnMCqsopLy1hSSKR9VzP0DVY4VLq48C4xu4fg=
X-Received: by 2002:a9d:4d93:: with SMTP id u19mr11579566otk.86.1630948068687;
 Mon, 06 Sep 2021 10:07:48 -0700 (PDT)
MIME-Version: 1.0
References: <20210903084937.19392-1-jgross@suse.com> <20210903084937.19392-2-jgross@suse.com>
 <YTHjPbklWVDVaBfK@kroah.com> <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
In-Reply-To: <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
From: "Rafael J. Wysocki" <rafael@kernel.org>
Date: Mon, 6 Sep 2021 19:07:37 +0200
Message-ID: <CAJZ5v0g_WVFqDKCBYnoPtqR5VzH-eBMk+7M1bAmgGsyX0XGpgw@mail.gmail.com>
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
To: Juergen Gross <jgross@suse.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, xen-devel@lists.xenproject.org, 
	Linux PM <linux-pm@vger.kernel.org>, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, "Rafael J. Wysocki" <rjw@rjwysocki.net>, 
	Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>, 
	"Rafael J. Wysocki" <rafael@kernel.org>, Stable <stable@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"

On Fri, Sep 3, 2021 at 11:02 AM Juergen Gross <jgross@suse.com> wrote:
>
> On 03.09.21 10:56, Greg Kroah-Hartman wrote:
> > On Fri, Sep 03, 2021 at 10:49:36AM +0200, Juergen Gross wrote:
> >> In there is no legacy RTC device, don't try to use it for storing trace
> >> data across suspend/resume.
> >>
> >> Cc: <stable@vger.kernel.org>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >> ---
> >>   drivers/base/power/trace.c | 10 ++++++++++
> >>   1 file changed, 10 insertions(+)
> >>
> >> diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c
> >> index a97f33d0c59f..b7c80849455c 100644
> >> --- a/drivers/base/power/trace.c
> >> +++ b/drivers/base/power/trace.c
> >> @@ -13,6 +13,7 @@
> >>   #include <linux/export.h>
> >>   #include <linux/rtc.h>
> >>   #include <linux/suspend.h>
> >> +#include <linux/init.h>
> >>
> >>   #include <linux/mc146818rtc.h>
> >>
> >> @@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, unsigned int user)
> >>      const char *file = *(const char **)(tracedata + 2);
> >>      unsigned int user_hash_value, file_hash_value;
> >>
> >> +    if (!x86_platform.legacy.rtc)
> >> +            return 0;
> >
> > Why does the driver core code here care about a platform/arch-specific
> > thing at all?  Did you just break all other arches?
>
> This file is only compiled for x86. It depends on CONFIG_PM_TRACE_RTC,
> which has a "depends on X86" attribute.

This feature uses the CMOS RTC memory to store data, so if that memory
is not present, it's better to avoid using it.

Please feel free to add

Reviewed-by: Rafael J. Wysocki <rafael@kernel.org>

to this patch or let me know if you want me to take it.


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 17:36:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 17:36:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180135.326638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIY1-00084c-Kh; Mon, 06 Sep 2021 17:36:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180135.326638; Mon, 06 Sep 2021 17:36:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIY1-00084V-Hj; Mon, 06 Sep 2021 17:36:45 +0000
Received: by outflank-mailman (input) for mailman id 180135;
 Mon, 06 Sep 2021 17:36:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNIY0-00084P-NE
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 17:36:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNIY0-00019c-7Y; Mon, 06 Sep 2021 17:36:44 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNIY0-0003cC-12; Mon, 06 Sep 2021 17:36:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=pfsAuKPr2auWUICkRafD2Xiy/uvIwmtJIwYIY7f4TAk=; b=wJtyMdrzHWQK6ZAa4IRuSE4Z3a
	SlbWGc8jfPzzDAOFtcZzZtNypBcL9z7GDOolAq8V9GtWQ+ZTQtLlN7TNeAnpCRMyxo2cXgs0l6dTq
	g/tHed8xO9LW7VqF2ikte83D8D/i3+d8fCc6Itrd0KSvCSZfCbCWeL3Ts9RtgXPXYgH8=;
Subject: Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
To: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629897306.git.bertrand.marquis@arm.com>
 <a7482653a0c29d68854846f25553f4938a6279b9.1629897306.git.bertrand.marquis@arm.com>
 <14b1639e-c9e4-f8fb-e921-73d7f28681ac@xen.org>
 <98E8FB9D-3372-47D2-92F7-8AD401AC1092@arm.com>
 <6705127a-bde2-36cd-0fed-900eb1eace03@xen.org>
 <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com>
 <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s>
 <D0CCB87F-1CC1-450B-A612-E4B505B36B5B@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <7fc6e6b6-464a-a305-2693-42cba40ad869@xen.org>
Date: Mon, 6 Sep 2021 18:36:42 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <D0CCB87F-1CC1-450B-A612-E4B505B36B5B@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Bertrand,

On 06/09/2021 09:29, Bertrand Marquis wrote:
>> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
>>> Hi Julien,
>>>
>>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
>>>>
>>>>
>>>>
>>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
>>>>> Hi Julien,
>>>>
>>>> Hi Bertrand,
>>>>
>>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
>>>>>>
>>>>>> Hi Bertrand,
>>>>>>
>>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
>>>>>>> Sanitize CTR_EL0 value between cores.
>>>>>>> In most cases different values will taint Xen but if different
>>>>>>> i-cache policies are found, we choose the one which will be compatible
>>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
>>>>>>
>>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
>>>>>>
>>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
>>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
>>>>>>> the time to limit the overhead when possible.
>>>>>>
>>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
>>>>>>
>>>>>> So I think we should be able to get away from trappings the registers.
>>>>> I do agree that we should be able to get away from that in the long term once
>>>>> we have cpupools properly set but right now this is the only way to have
>>>>> something useable (I will not say right).
>>>>> I will work on finding a way to setup properly cpupools (or something else as
>>>>> we discussed earlier) but in the short term I think this is the best we can do.
>>>>
>>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
>>>
>>> I agree and this is why Xen is tainted.
>>>
>>>>
>>>>> An other solution would be to discard this patch from the serie for now until
>>>>> I have worked a proper solution for this case.
>>>>> Should we discard or merge or do you have an other idea ?
>>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
>>>
>>> Not sure it depends on what the final solution would be but this is highly possible I agree.
>>>
>>>>
>>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
>>>>
>>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
>>>
>>> Ok, I will wait for feedback and next serie will not include this patch anymore.
>>
>> Would it be worth keeping just the part that sanitizes ctr, without any
>> of the emulation stuff? That way we could still detect cases where there
>> is a mismatch between CPUs, print a useful message and taint Xen.
> 
> That’s a good idea, it means removing the emulation part and just keep the sanitization.
> 
> @Julien: would you be ok with that ?

I actually thought about suggesting it before Stefano did it. So I am OK 
with that.

> 
> Should I send a v4 or should we use Stefano’s patch directly instead ?

I would suggest to send a v4. This needs a signed-off-by from Stefano 
and a new commit message.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 17:46:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 17:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180141.326649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIhL-0001Mu-JG; Mon, 06 Sep 2021 17:46:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180141.326649; Mon, 06 Sep 2021 17:46:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIhL-0001Mn-Fx; Mon, 06 Sep 2021 17:46:23 +0000
Received: by outflank-mailman (input) for mailman id 180141;
 Mon, 06 Sep 2021 17:46:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNIhK-0001Md-D3; Mon, 06 Sep 2021 17:46:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNIhK-0001K3-0G; Mon, 06 Sep 2021 17:46:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNIhJ-0004T6-O8; Mon, 06 Sep 2021 17:46:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNIhJ-0005RC-Ng; Mon, 06 Sep 2021 17:46:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4OtFCz7kZQXVcpAkuAPWawUFRMQ35ileKtWaSsqHdCE=; b=aOR4utRrlPLumiky2ipDS3q048
	eDNhEjDGDdVnKrmgXkxo9BZVVtAnJaFmRPktmhTD14UQ6bApx6P+W45q6LtcnbRBSONMmYPoJOMpG
	6Io3hzztZzQV7glxZh5P1WWAjIw4vu6jZxu/PFkEWKE/OzEsOTdU+6yVcL0KkjGEGiu0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164855-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164855: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=88afdc92b644120e0182c8567e1b1d236e471b12
X-Osstest-Versions-That:
    qemuu=31ebff513fad11f315377f6b07447169be8d9f86
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 17:46:21 +0000

flight 164855 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164855/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164834
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164834
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164834
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164834
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164834
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164834
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164834
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                88afdc92b644120e0182c8567e1b1d236e471b12
baseline version:
 qemuu                31ebff513fad11f315377f6b07447169be8d9f86

Last test of basis   164834  2021-09-05 15:08:59 Z    1 days
Testing same since   164855  2021-09-06 10:06:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alyssa Ross <hi@alyssa.is>
  Ani Sinha <ani@anisinha.ca>
  David Hildenbrand <david@redhat.com>
  Denis Plotnikov <den-plotnikov@yandex-team.ru>
  Eduardo Habkost <ehabkost@redhat.com>
  Eugenio Pérez <eperezma@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Jason Wang <jasowang@redhat.com>
  Jingqi Liu <jingqi.liu@intel.com>
  Michael S. Tsirkin <mst@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Tiberiu Georgescu <tiberiu.georgescu@nutanix.com>
  Yajun Wu <yajunw@nvidia.com>
  Yuwei Zhang <zhangyuwei.9149@bytedance.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   31ebff513f..88afdc92b6  88afdc92b644120e0182c8567e1b1d236e471b12 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 17:48:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 17:48:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180149.326663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIjl-00025G-6C; Mon, 06 Sep 2021 17:48:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180149.326663; Mon, 06 Sep 2021 17:48:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIjl-000259-2b; Mon, 06 Sep 2021 17:48:53 +0000
Received: by outflank-mailman (input) for mailman id 180149;
 Mon, 06 Sep 2021 17:48:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNIjj-000253-9G
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 17:48:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNIji-0001Lg-9S; Mon, 06 Sep 2021 17:48:50 +0000
Received: from 54-240-197-236.amazon.com ([54.240.197.236]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNIji-0005Bv-35; Mon, 06 Sep 2021 17:48:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=4TxAf+1W+KDaf8dtm2znwRRH5Pti4W0MUcRVqefKbqo=; b=3kyYO9DDEwEBE7eT/hbpGJ16Na
	HcoZWjMalx12VR6nNq0fb+1Lfk05VUBFqPye4/ZV4B9c4O0WwUVughtJUcTviw3CwBwj5cL43dqJ5
	a9NMPFOHXedMoOVLPXxP23ljZEaHMFMfrhVkz1F+fAsPwHxEDG99wCXkSWViGD3dgoeo=;
Subject: Re: [PATCH v2] xen/arm64: Remove vreg_emulate_sysreg32
To: Michal Orzel <michal.orzel@arm.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com
References: <20210729104258.6320-1-michal.orzel@arm.com>
 <24ba08cc-acd4-c64a-1e90-dc4c8efbfe48@xen.org>
 <f4ee4519-9153-1671-179b-3595d632a4fb@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <6ce114f6-ba2b-7262-ee0c-7a339465c9a6@xen.org>
Date: Mon, 6 Sep 2021 18:48:48 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <f4ee4519-9153-1671-179b-3595d632a4fb@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 06/09/2021 10:09, Michal Orzel wrote:
> On 06.09.2021 11:07, Julien Grall wrote:
>> The rest of the patch looks fine. So I would be happy to deal with the fixes on commit:
> Please do. Thanks.

Pushed. I have also re-wrapped the commit message to 72 characters per line.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 17:56:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 17:56:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180157.326678 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIqr-0003lc-WD; Mon, 06 Sep 2021 17:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180157.326678; Mon, 06 Sep 2021 17:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIqr-0003lV-SJ; Mon, 06 Sep 2021 17:56:13 +0000
Received: by outflank-mailman (input) for mailman id 180157;
 Mon, 06 Sep 2021 17:56:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNIqq-0003lK-1j
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 17:56:12 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b7e7d00c-0f3b-11ec-b0b1-12813bfff9fa;
 Mon, 06 Sep 2021 17:56:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7e7d00c-0f3b-11ec-b0b1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630950970;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=nUUgfuP/5TszjZLkwkCThXs/nKbossEJ3ZFdbxVQy/4=;
  b=VdP9eEEgjfZxZsH8EqLNkzkC8ElqsmQnrFjQ58ZgBTAVvlu4EVt/Fj9R
   DBx4BifmgQCc1QhGMdPA/WmPvdHFIz69lirh4wKIEcawbQ821RcZWKSuE
   Ht/2eW/BPy/1vbS2rzjChXxMIi+2tzeIb7rVkYMguMVgp1w1R1bqCsbPb
   k=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: VTba+8dEPt6A6J6iaSKByEmEkw0aMFkoYtidQv1hRx0YxY5+EnAN7UpcUB5kDVL2JHjW2+26lO
 XpO/mMUdodxRiegRNk0BDKY0GGZ/Rw6Y+FSuWwSYxAODA8ZGYcraUOiUjW/qkyeqiBLSlnkLlu
 xO5m7LWenx1qQQX/KWO9CDAV/D8OUFUqzsGavDv8zjCl+FSxj8jVGjxhk3DlFz+OzNJP+33JHK
 mnAEUzQjty6ag8jF4nhFQPEYNGwR75bVVh5XProTc5HcYuJHrJtl1UK9EJp+oqavx4C3xescs6
 ivsGM41ChjMZAGDb4BJj09g9
X-SBRS: 5.1
X-MesageID: 52104379
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:+OsgBaHFsgLfSlk/pLqFf5HXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HlBEDyewKiyXcT2/hsAV7CZniahILMFu9fBOTZskXd8kHFh4hgPO
 JbAtJD4b7LfChHZKTBkXCF+r8bqbHtmsDY5pas854ud3AMV0gJ1XYJNu/xKDwVeOApP+tdKH
 PR3Ls9m9L2Ek5nEPhTS0N1E9TrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqJmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87CsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXkHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1XackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMW9yV0qp+1WH/ebcGkjaRny9Mw8/U42uonlrdUlCvgklLJd1pAZHyHo/I6M0r9
 gsfJ4YzY2n46ctHNVA7dw6ML6K41r2MFvx2VKpUCba/Z48SgbwQr7Mkf8IDbKRCdE1JKVbou
 W2bLofjx9rR37T
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52104379"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JB6JAMJ/Pj8vs7eF5ZcC9l8HXHzELkpIKv51WtTu3KsdoJpGSOP4M7Ay1b4euGIXes/QJFSItg9E4r8/wM9zCu3DS75EH8tEiOpFDOaUn6MsnYNf+g+yQZxeLrbBWynXOnRWngo0qj5zZl1Lhp4vMv0V9Ym5iEPUhGnlWlIUGUeJijRAG8uZgerxLlI5KyXO8cLB02hhPNkqyJYuyaRP30VTS5ZjpPIisZCa5UVXiuaMgr7hGj+sm6C5RjNNQzo5eIyA++cHaXg/jmRggeOsHXzjrkVR9YJUViyi0djsye2h3KQRpQVLExFI1mYBy5HfP7T5/bxwWG8RKor85GpO3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nUUgfuP/5TszjZLkwkCThXs/nKbossEJ3ZFdbxVQy/4=;
 b=MpLN5pe95YsLzzPIYsAX7svDgN/b3sg0GnV9xye/8v6D7M8RyunHwTSWxYFkWXCn+Fte0dlIyvMdswYArko722r2gDElDFf4WdbdP88sxDJ4DiSiGaYAay6PfYZ30VKbl9jRhaqR2TbY1UyR8XMeysVrzlWWTwi2fOhM08swALeb7IpzPyJEq+pPpsn8UMXGNJNgmb67z2ZEB/mVgzlc75pqwoZW/Ks/F5Oc5hKO51HnCq7GBIT0HhfqeTXavkvfXpEZKUffneQ4+ByOv09B2do+05qFB+hm9cgcyXD37i+mruoYAiALWzJE2K2vo/afjO+IE1gnAjfxeqrbZ2UnXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nUUgfuP/5TszjZLkwkCThXs/nKbossEJ3ZFdbxVQy/4=;
 b=jYd7M13FwTE+2ZwIBNvVGapfUtt3pCPVWZ38HXQIqtlNEcyqlX3mw5CPSyM8Ae9tG3r61QqVexu4vF/ecNvdVLLk9Rs7HEsl42TwmRXGUo65AvPeHGHGMBncLYcCC8MMpSbUKPGMV2vOsacyJ/Srvg1d8iwS4IFC341FjCyS4D0=
Subject: Re: [PATCH v4 02/11] xsm: remove the ability to disable flask
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-3-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <fc0a3a17-8896-e2d9-bc10-7ca4c277d4bd@citrix.com>
Date: Mon, 6 Sep 2021 18:56:00 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-3-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0337.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::18) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2cbd5bce-78c7-4ec9-d9a0-08d9715f9a95
X-MS-TrafficTypeDiagnostic: BY5PR03MB5000:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB50002FB7A663D42015EEFE7EBAD29@BY5PR03MB5000.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: kVqkyrgTbep0EFVGZSZOxvecFukYG9u1J1AXU824VmPIBVA8XV+4YSS89eFJxCaKZ1l9h2vt5agK98ue1BV4CmkM2F04cB9/kP1pGqbUU0gxA3546/vPDee/r2LAE/ZeojzUz6f4aipBqiiiL/yKuspqaYnJZRL/YO9eNOE5GUTdY4kHT4Bz0fO5/6fkOgeQDFyZEs9R+PcQyRcabVM58SCcpUbgsvYYbBw1UB+5dgeOh9SScGhbz8oXyqx+9uFzpY6mo1OSV7ADhqRiZkDxCCLbVKIHuOitmOXdi6binNInlIZQ28EIL4qbOWSmTswqJdhYEa5o77+MD16C1ozERyLizIzm1dPDJv4POGJVGO+bewqwGtqwYG8VmODNRW+h+0ykNkzaWYvxPzuPEuM3EDaPMPqoL67IvLpqIOlpwOwIb0VTJStt0Ahas3KHpSShSHzoxQDFzBl45zD8FhHxnHon4yniVkgykR1r/FgXtnwg1dtzx5Ht9LfJeFfj/Z1snW8hsoyEh1lM/ZASqsrjPhhlqqKelR+kbj0+RE8hpqS02MhFZ3MMykt/rEkdu9aPpQ4mw8NSXxAUYb4VQY4UjEh2Ytro+6XJlqv+Z5BfzrQzhboGYZUkka2R8WAu2RpgM1PdFV8dU4VW/wvC49cKEqcA/jT4KMDXrHSkjgFf7lsuJj1tzcGQAoYEppTtJf+Yp/YJulUZqXt6EklVdJfTrs7VixxOzw792g3wWAw1Z8A=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(346002)(376002)(366004)(396003)(4744005)(66476007)(83380400001)(2906002)(31696002)(8676002)(8936002)(66946007)(66556008)(5660300002)(31686004)(16576012)(36756003)(6666004)(2616005)(54906003)(186003)(55236004)(478600001)(316002)(38100700002)(956004)(86362001)(6486002)(26005)(53546011)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TXdndnArVThQWHpTM1IwZFZCSCt2Q1RWLy9wNDZ5a0R0L09lWlozT3RydUUx?=
 =?utf-8?B?TWsrMFk3MVFpMmM0L0dkanV5QXRyY3h6VENRVk1KRmwvMm0zWE5UL3YwTVZJ?=
 =?utf-8?B?VjdaUTQxOW9VVjczc2RGcG1FcTVOMmFmbVRYNUNBRTFlZUFFYnRYcFY2QU5s?=
 =?utf-8?B?bE1peFNpMkdDWE9kSWMzN2hYdDFwbTRPd1hUQXg3TisvYWcvT0dtOEtoOGoy?=
 =?utf-8?B?ZGk1SGJxbkVXUzV0NHpzVWhRVmJYcG9EOHZGMzUvcnBKZXpWTVZTa0ROaFdj?=
 =?utf-8?B?ZGdWY1E4ZlE0SGIzMkxCV3dLaFk3cW8zQVVwNVFjSmRxNEowNnNkSkprMkpl?=
 =?utf-8?B?eUpTdFVWWm1LZzJldmxMVG5UTmNBc2dXdTFxeTdwc1JNTlE5THErU2plWjh1?=
 =?utf-8?B?bWxiSVpESlcwVjlWWUprdmpEMjhuVXRUTWNSdXpXTUtoN3U0citDYWhBc2lZ?=
 =?utf-8?B?NHVtdmY2eVFlc0NKUDhDQ0lNUUttaDRONys2TmlWWUdqalBJKzRyblJWemhG?=
 =?utf-8?B?ZVg2MGE5R1VjNU5tUUx0Y0l3aG9vVENQUkExRDlEQ0VWNDIxMk0vNmxuZys1?=
 =?utf-8?B?Z2RxbGZxQ1VvVGFnOUlxWUVGRkg4WEU2MGxNSktmbGZZeEtDYzZzTjQyQ09h?=
 =?utf-8?B?RnJuM0lqZFgxNHFVREloZVBVa1dKTExRaEhPNDVCRjY2a3ZqVGlHd2VCa0Rj?=
 =?utf-8?B?S3JsM2cxeXdwSzJzNkRGZ0lUNnpRa21RU0FzK215eWxia2pkQzhKNW5OWUU0?=
 =?utf-8?B?TkhyMU1JUnBUTDk2VDJ5TUJkTCt2U0NKYWpsYzY2WEFBelRUZmNoeWU4dTY1?=
 =?utf-8?B?SzlSV0g5UWI5eCtwdEtibk9jL1BWKzh4R3ZtQ3pvYXJDeCt2N3RtWUJxWDhU?=
 =?utf-8?B?RFM2bno1ZHNYcWR2cThHdUFkNmVzYW51c1ZWWVFyQnREMEM2RWhPMFlIQmJr?=
 =?utf-8?B?TkVwZlFzYmsyZ0hna3VCMzd6c1ZuMHRONlVhLzRDemhCOWlGWUZsV3pwdXEw?=
 =?utf-8?B?YVM0c1k3Rlp4SC92ZVBoejMreWkzS3NqYytKYU5tMERDdEhLTU1ySE9iU2p4?=
 =?utf-8?B?K3lETWozMG0rNTkvSHA2dHlrdnljQzgzUjFUVFhtVlR3UjJrb1ZFN2l1bnVq?=
 =?utf-8?B?dGU5amxFTy8zVE9MQ1gramExaEF6d05WU3orazVvajdRQU55WmVnRjdzcW45?=
 =?utf-8?B?WW96a0tFOE5JNVV3QkZSczY1dDA3MmFQOU1ZRUR1VkdNeWlUUlNWczNnc2Vh?=
 =?utf-8?B?dmxVbXkxWDF5RkJXRVU2SU9xNnduSkVIdGJlRzRGRlM5NktWLzJrR0Y3Qktj?=
 =?utf-8?B?KzJEZUFacE9LbnV4dERyMHl3Rk0xTjZrRm9ZaTdOcytXc0U1K0MyRG15S3ht?=
 =?utf-8?B?bXRXY085NC9YdkVheVBUZS9VWDhobG9yeklTTGZkcXhxMjdiTFFlNDZkditD?=
 =?utf-8?B?QUdBeElZbDFoZWdRUE5JRjFqQU13RjRBanFtY3dXL2lnUVlaS0M5MmZORDM1?=
 =?utf-8?B?MmdmSUttUGptTmNqdHk0RkJsZTh2a1I5dWkrQXNUU2JsR1ZDRE11N1VnNmFE?=
 =?utf-8?B?b0VpaDY5Nlcxd2pkTDhGOHJzUVVVY04yZXlNVENPRWxXVE91R21IMVJGOE51?=
 =?utf-8?B?Y0I5T3RHcHNrVmc4S1BPZEpWaS8yRmxGWmQwalB6QjUzZjVuak5XcDB2V1hP?=
 =?utf-8?B?aURRRkhHT2hQVzNyZDlHTVIwdmVEZ29uOFphS2hoNTZ5aXdTdVlFSHhMNTlS?=
 =?utf-8?Q?pBl7gKf1L6t/vlW3UiSvEVWnAdTZBxQo1C/eNnH?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2cbd5bce-78c7-4ec9-d9a0-08d9715f9a95
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 17:56:08.0348
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ntAV3uvyfkLAcTeYHRu8x7TH9y+13DYdQCu4ydXwazvhcZ6aMLw77icvvhntVxD3rc2hV2+4JYmvJTTDQdWaiFWfFm8NfX6pX6qNMWWelZY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5000
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> On Linux when SELinux is put into permissive mode the descretionary access
> controls are still in place. Whereas for Xen when the enforcing state of flask
> is set to permissive, all operations for all domains would succeed, i.e. it
> does not fall back to the default access controls. To provide a means to mimic
> a similar but not equivalent behavior, a flask op is present to allow a
> one-time switch back to the default access controls, aka the "dummy policy".
>
> While this may be desirable for an OS, Xen is a hypervisor and should not allow
> the switching of which security policy framework is being enforced after boot.
> This patch removes the flask op to enforce the desired XSM usage model
> requiring a reboot of Xen to change the XSM policy module in use.
>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:00:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:00:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180163.326689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIuz-0005D4-Gx; Mon, 06 Sep 2021 18:00:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180163.326689; Mon, 06 Sep 2021 18:00:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNIuz-0005Cx-DY; Mon, 06 Sep 2021 18:00:29 +0000
Received: by outflank-mailman (input) for mailman id 180163;
 Mon, 06 Sep 2021 18:00:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNIuy-0005Cr-5l
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:00:28 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 508fddc2-0f3c-11ec-b0b1-12813bfff9fa;
 Mon, 06 Sep 2021 18:00:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 508fddc2-0f3c-11ec-b0b1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630951227;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=hVReezva1bSFHzc08+Zdr6xsT6LCHgeopJi15GjeCc8=;
  b=ORcYJxRG/gBeSSFTPSYlVhh6Y/rwgSqVK1IDAXDe6jdzt+Xe598DGJ5z
   1a6TBRqnHmhlwxuj4LgLWwszzuJT6NB/II3c5F1wPmKS1ZURIROz+ZS/5
   39RGD91Prjx3GkDLxN5uc8Idg6bsJEKIxiUVRjQOdQvU9mgc1YFoJ0aYO
   c=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: X7eUQp6tuzQ90zkby4zuBFKbzv6oaSOv4wR0xGuSIdaM2RVYJGXuc+gMZSllMEhnhyKD/mTDCz
 pPo4VLTrZra7G7yeufcb0koKr3XalC57dshSmC71JXivZJJnG6ItRJHNgmqsTuir6l4R58Clim
 9cYtRC3dmlXtCvo0qHjU6PvqoLTomD4bl0lJGNdyKfOjNm73a9ab0GBfvVZorz7p5eIwwWEGO1
 7zHHW3h/cbYnaTh27qwNdScBDzPpnC/7zj9GKuWmRiwtt8OcYw9AZXNs+uOzBBAcKX2W2hmIhl
 989RUCyN48Mxee/xQTi1flVU
X-SBRS: 5.1
X-MesageID: 52115295
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:8RiGzK18jX10sI76Q5oOxgqjBSVyeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5OEtApTiBUJPwJU80hqQFnrX5Wo3SIDUO2VHYUb2KiLGN/9SOIVyHygcw79
 YHT0E6MqyLMbEYt7eL3ODbKadY/DDvysnB7o2/vhQdPj2CKZsQiDuRYjzrY3GeLzM2Z6bReq
 Dsg/av6wDQAkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="52115295"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e7TNtoxXe6LCg5dz499W2bhUflsE+gEht+DnQm6UbnptoOp9BbMqJP+R0BhFRu4vtAnB6CPDufjk1AKyApvXc045G52mSyIcBa1NdccHI4HRZSiUknEBRbeV9S9cQxjAvUpNZIyIImi+2mAF/VoeP+LkgfJ2OhhxsInYmyDhrbHEYk3fdeMZZr9AKQ93quoWdjnFygTAoYS3A4GkW2jUsorcHsI4MXDarW4oEum2LMXmeZm+daV+ipYTj7BYI1tI7Tby7Nc3iJ5e1dT/8ZYwmYd+90Oj1CT4VXGseHA0/vdc5yqGKEgrR9TN1wUzaSLuXXJDcmFAZoBCNQyawWVNKw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hVReezva1bSFHzc08+Zdr6xsT6LCHgeopJi15GjeCc8=;
 b=OhHy/CMhQl2QEGSdLSNbj57wevfGcKZBUbbZRr5Zlbdf/K/ExqgOPgA2c1ir5S43M4uJmxx7ahuTWeMYqJWiC24zLWpqptJJrFH4LBEMQlqKhSHCFKMgZRKzO32UZIvdPslobFygUAHEtZHzgJcu0JUABQKodLe8BNZGvMgvTUMUuG6SIO08SXwFXz5nd0PH1pYim2NKWMZz2CP1PAp5rqHFPZNZ12uQ3meq3+6DbejCavWFMtOAxKPc9sXlSWW/YeBdm1enE1/BxTARPBIat8IQP/GYLmL/MEcRAG4vhYjnGuGbq6E3FWAXCs0LLIIrxu+/NQijUuulIfGkKfEvvg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hVReezva1bSFHzc08+Zdr6xsT6LCHgeopJi15GjeCc8=;
 b=BM6XOLAvBYZUfwHFbBN9+i8gCufwmmwL0znbbGmvOGK29eGjTuwVw62DUvJ4NuTBbc9sK7goAKtsFANuxFXZZ2Sd1r2OkC48xWdGZmWIko9Cb7d8fiWqNhHWbnSTrotn9K52nBe9YrnUoVjzU1icVSy0t+NKzJbOnOjUnF1MDa8=
Subject: Re: [PATCH v4 03/11] xsm: drop dubious xsm_op_t type
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-4-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <923afe88-ac02-77b8-06c5-480ddb2afe43@citrix.com>
Date: Mon, 6 Sep 2021 19:00:13 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-4-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0078.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::11) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 79473da4-3a1c-419a-a61c-08d9716030b4
X-MS-TrafficTypeDiagnostic: BYAPR03MB4741:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB47416A97F04DF9648788A62DBAD29@BYAPR03MB4741.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1332;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +7fZnZ/lN0p4q6mLzuc+LUICJI+wdXud8CWOQMptCNrXmVqWQiGOyjqGBrVGjttzmK46SjnmiSH0VPQYmSywbJt6FbRiYucpc8xKCEc0OZcKwVHaAFdwzfIM3DAfzknC1ekxJ+SVD6zuqafRtTKcUD0KDEWZieHEy4Ub5QFQWH6l5X3tR1kx/hdeHGFmFj8MuopYFi2zi//CswvGbJK4EWGdrh9kacuPkaMbANhUs5nvucwh+8THvQKL0+aGtz7cAXhyTAvtWBTcpmy9Ydsy52VIxSh6upFOlr6kP4PFKj8uMksnoxCice/ntzC6tonavoSU3RtmPtSQAOenHBg088nmJnTD/L1uOnUBWwtgszTxzV6oiIqqObz1cvXq2P8LwSjctmKg+dn5YQpI5GK6BwFYi6Fp7el+ktAaMSIssrh+VqX5v6NueKoO/eAUyJoYAF/mBXzk+GjFnmrEUHRx3m5uZ9+CBYvAoc6wNIWjkor9zRAbdRcpzv2RqKe5YUhOZjrmmHTG+pAsjvT8rGFbFZWBf+04rfIGWpRsA6QMHGIxqKeO51naZzlX0T0G9nH83ZcrLdZQxE7jDZF/tPH1g4N6ejaJ06jKqJY+sgU+hTYmYZKyPTJqXCFkU0Pd28rh9yCEoMxEZfWD0fnA919Vcy0SfdKYvf/Tr7SxyetEY+0JugNv6PuO9VOeOY1viDh0H4Uj7exo2LWXmxycDTXIZRn0K2PvBGGcdvuBD52pvPURRGAuQTOshKRP8gaIEmfL
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(66476007)(66946007)(31686004)(66556008)(5660300002)(16576012)(8936002)(4744005)(2906002)(8676002)(316002)(26005)(2616005)(6486002)(36756003)(55236004)(186003)(53546011)(4326008)(6666004)(54906003)(86362001)(38100700002)(956004)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Yi9reTE0SVJFL3BqM3BhTjNEc2FsMnZrVGdPL0g0VEFjWk91cDVBTHVQcUlC?=
 =?utf-8?B?SWJlcE1pYzJiSW1TMCs3dzl4SVprd09sdU5udktteDJwalpPUjJYbDZmdzVM?=
 =?utf-8?B?cGhuWm5VMU1OczRmUGZJc1B4UEdYTjdxa2lIVEUrb01QeU1DOWR2WC92MjF6?=
 =?utf-8?B?NHl5Uk1sa29yeGcrRzJsMmhIMVNsV3BuOU1TalF1ZmNUeFNHeS9YWTQ2WWVY?=
 =?utf-8?B?L3NITXZYd3ZHRGZEZkt6OWZCY0w0dDB5dllFUU9sMlZBWE9pcWZNdVAyd3Ft?=
 =?utf-8?B?QmxQOERKMVZpT1d3KzBuTG00UWpDaFBWT2M3cWI1TXk4RlZUM0pxcW5LYXZD?=
 =?utf-8?B?NnRaOTByY1NmbkVjK09GVk9zQzFrNk5OTms3K1g3U2pXMUxzQTdvNm5TdnVz?=
 =?utf-8?B?YlpDUGpJWWViVG4wSkRKZXkyL2RLV094Uk9RTkk0cmV3Zm1qak5qTDZJbUhT?=
 =?utf-8?B?QU1sN2plcTJNZkdMQWg2UXRRa0Jta28xZ01hVlNvcXBrS1JWQ2R5MjVIOGVh?=
 =?utf-8?B?Yk9qYTM5TVFFYWp4OXMxNytpQTIxVWZBdTFvVTNRdGdpZHhkb0NGQjR2NzRn?=
 =?utf-8?B?cy95em9PY2xuc21qbUcxTXFSVFlVeG9IRk9HeUYwZWl4Y05jTmZHblpkMm1H?=
 =?utf-8?B?ZjZhSDY5WUxhNXNvTkdqV3BwOG41dmVlbVJIVEJKT05TRnVhUWw2cWNUT1oy?=
 =?utf-8?B?QVJLQTNiVE9XNEFkYmpKQjh6cUpHUDNjd2FMcGpMaFlZR0sxaFh2b1ExTG5Y?=
 =?utf-8?B?cHZPb3VxVzNtb000dUdvaXdsWlRlTmFPVlZld1JHQU9KUGdJOUlYOUZBVU9S?=
 =?utf-8?B?a1RzdmcrUTBsNXp3TllWTVFJZU50RjlGU0pzS01uc0lBRjMrSXh0TzROU2dL?=
 =?utf-8?B?c2E2ODQvMStVWjBLRDVtTTRSVW00RUdGQlhTekllY2pjTnB1M0JOMnhaWTZ1?=
 =?utf-8?B?bTFVdDVONUVtd3hJdmxyTXgvUFlWWDVNV2VYVE5yKzlnaGR0Z2FRWHJENVZv?=
 =?utf-8?B?L2FBeng1a0xkM2V3UVdwNHpTVnlTMjQ3eVQxNWJUN05Ta2U2NFRHODZ0RHUw?=
 =?utf-8?B?Q1F0YVp2U0VXKzgxVkIxeFl4RDV3eWUwYTVQWVMvRFMyN3M2SHYvVFRad2Ur?=
 =?utf-8?B?d1AzYmRaeWlUejh3bnhpMUljK0xOZ0dNaXVnUWZXWnQwRnB3M2NJcUU4NXhu?=
 =?utf-8?B?ZXJSbW5jSnF3WlZiWCt6OEw4M1J1S0JsMlVRYUJKR1MzQWFENitRYXFLRlU5?=
 =?utf-8?B?NkIyVUx5amFjbi9JWTk1QUtQZTNwK1c0emNjckNCaGNua3J4cVpmM3lwZXUy?=
 =?utf-8?B?eHdRYkxsNzJaL3JoUTFNOEZReXhiVllKRk9ZWG1haWZWcFZ2MVdpY0VoSEdP?=
 =?utf-8?B?OXo3dkxNQ3Z6cCtnSk9iM0hIZVoyNHFCYXdxSFo3N3BjeUpXQW4yWEYxSUNm?=
 =?utf-8?B?bkY2M0NyUTI5VGtHL0RaSzZsS2Vvb0ZIMUpOK0wwQVhJdzM3Wlc0ejJyYW1a?=
 =?utf-8?B?Z284UGs1VWc5Nm5MU2xQUE9kZUZMMndrY05udDRJQnc4eXlKeFZLWUl1dGxI?=
 =?utf-8?B?cWZjVXRTeHNhcG1YYTF4eWRHeUgxZTdKVkR5Z1EwcUJkMUJGbUdpNkNsczlF?=
 =?utf-8?B?QVNlaFNDUndaWjlVVUNIbGh4MnlNOXF0bGw4WFR2bGpOWDJKNHFjbm9DTloy?=
 =?utf-8?B?VnYvQVdvU1JuOUdHOUhLdDlISlJ0bVJFUjZ2QWlYQnFNbVV1d1RkUHFabG80?=
 =?utf-8?Q?lcC3Y2+LjhJ84Sq1ANqwlDQSkAG7Vq+0EKtnTe0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 79473da4-3a1c-419a-a61c-08d9716030b4
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:00:19.9119
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xBRfsa+IvyMX7SZXZeIY51QvHrjigniW5TS27cgJp6ZCRx0ZRMiJRqnRD10scyjZML+iNYF2qVOVTVndOoLZ9xxItmdE4kV0psXmbbpmA6U=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4741
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> The type xsm_op_t masks the use of void pointers. This commit drops the xsm_op_t type and
> replaces it and all its uses with an explicit void.
>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

HYPERCALL_xsm_op really ought to be renamed to reflect the reality that
it is "flask op" and we cannot possibly reuse it for other XSM modules,
but that can be added to the todo list.


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:07:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:07:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180169.326699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJ2B-000696-9h; Mon, 06 Sep 2021 18:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180169.326699; Mon, 06 Sep 2021 18:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJ2B-00068z-6j; Mon, 06 Sep 2021 18:07:55 +0000
Received: by outflank-mailman (input) for mailman id 180169;
 Mon, 06 Sep 2021 18:07:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNJ29-00068t-K4
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:07:53 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5a02463d-0f3d-11ec-b0b3-12813bfff9fa;
 Mon, 06 Sep 2021 18:07:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a02463d-0f3d-11ec-b0b3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630951672;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=tatBAklfoTz0IHa6ERQ2r3u844FuxIM8q6wiCalO/oA=;
  b=UPScWOQ3csiWpfJ+cGBYjScDBO7H/tNvoWYFPxBw8odbS+498NOpU+gP
   28OzAPXucHm3l7u8LZcadtJJ6jmZn+6iYgnq33hX4fWc23Edv1Df7B9h6
   pK/6UecmXYPf6GOUaWcvbew9Y2FxvrJNqN1Ad/iHyHyGHmJsZPqkyJlAL
   s=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: dEaSWqBoj63gUhgCCQ+WlgRYldYcv7Eds+6ZAI+mjCtvfQTL5D/hm9//plqEFfMNSkrW713r9J
 Nj2h1d/9RKP/VTvg3MMyI5Kv/mi62S4M/Qxh1xVqbPO3cF7VYp5vACZ62EbyhbloBBlQ/pDgaM
 ttE4QrcIYbNb/ugqmIBbLZorthWMk6AHPlyaqo9Jxh3ZO4epj1VSqn+PCWtwbEPFWvW1Slm9H2
 NZu/Dr7t4D7zAGj2/LaUErcxvSrHFcK03ZYzWLp4kqZZIXdcRfwseoq66C9NqBV27LLa+Ye15M
 JLp1yXyebgh0TJFYuo/4UYDa
X-SBRS: 5.1
X-MesageID: 53860036
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:6na8C60Dd1DnAUXj3ligFQqjBLYkLtp133Aq2lEZdPUzSL3+qy
 nOpoV+6faQsl0ssR4b9exoVJPufZq+z/5ICOsqU4tKNTOO0AHEEGgI1+rf6gylNyri9vNMkY
 dMGpIObeEY1GIK7voSNjPIceod/A==
X-IronPort-AV: E=Sophos;i="5.85,272,1624334400"; 
   d="scan'208";a="53860036"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bZ2VpO9mG+T2NomCSkFdTBk6E8i4I2vO52EcQbhCplkCSy+2HGAcXbDxkwlZCdR6NIZtlsDrVQZcxTDJendbn0r4kNmYOUwHRP8gg2I23H6ytuTsIFFiJ+cQsfINESBEfS9gbGg+XAi+fuu7MjXpO4OopXPU5jE6BPnQrXAZBrZDCIVoe9db1d22aq//BCpmRoqFVN0gQCa1OfD1IlZpqmnKnhQX46OpORsLKQcE3OqiBS3aZwdBh6EDjwN/HuGeMOgXdMKEIRH3fwhdXlO0puHo8UJqHoIa0f1jgVx4B0zhgJP4tRhBhwCm3KDY8iJzNrqQJOscH1Gqwwb3sG6fFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0Mncjb0+JPGtlBvhg0XIVHAc93gaDfzgSaMk3YdIyJU=;
 b=ae+2/oq0Jzfk8uYwF0/lk1izupG7YWdXafG3G6ll7M5604SJODsMqptpVEGWj056AshtpKW/YqhrFfU0SaN7Q2fVCWJhh1Cnfg8QPSMySInZ+0crvyUUSADKH1Odv4ShQUCrMt/bIQ0QhDytMUGiOIyIM8id/tv8L44LMUuIKLjGwrXjaBYRXvByhOqc/eogob4v8fge2rYAjFL57GDW2xmTE8ODg+UqJobZwroRphA/ORl04/tr3MoShrHJXnxX1blPSC/Lw/wupNtTiVmh46+Ed79Cr43gbt1RiKswHxwQCMjOXfA2wE2Am3PGku1f/KEJVVW8WIo0I6fywClFIw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0Mncjb0+JPGtlBvhg0XIVHAc93gaDfzgSaMk3YdIyJU=;
 b=xnX6zHR0yZjXHyfBffcjLF41hgf00ZSlykbyFrBJWWbf8pDU00oyA4dH4ZIW/2Ae8UG0TdC5IXBKe7j8RduGK4m8rFcE5iP8SGVFsrzPnQF1zQduoEHc/Fg857AdXjtZb9q9LXBJAvpBHnUnbq5xmyjpFZxNlCzEqw9jgsY+xMA=
To: Jan Beulich <jbeulich@suse.com>, Jane Malalane <jane.malalane@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Pu Wen <puwen@hygon.cn>, Andy Lutomirski
	<luto@kernel.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
Message-ID: <2972c71a-5fed-0813-8ec0-d179eacff14d@citrix.com>
Date: Mon, 6 Sep 2021 19:07:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LNXP265CA0095.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:76::35) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7e97e60b-760e-4a32-97f0-08d971613b65
X-MS-TrafficTypeDiagnostic: BY5PR03MB4997:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB49970E3F23473704B3B6FF93BAD29@BY5PR03MB4997.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: yNZHqmSxIatNQYb18rRAi6XBJxigwx3fUFH5MpYv1DDMLgz4oF3irB3mVEMAVEA5546aw62LMASQR9eolrmyq4gvIVsu3yq0KbFE6led3WvGI+SopDLX88iBjoi98tbNAFT8PTT3IGdvGMhm1gcZ1lLd8QQA+ve3toWgTfi7fWu9zB9DMyhNY+H0uu3MDfuHSWgMGav1mgPoHIdq1r1p2VTkVRKbBaLux3YjtRJ9dIEfkXilv/FnzgP+hm2mQDBoCeq+T5FwC0E/CDzRjMZZ9mh9snOgKrrhcvSMWxDsn+fA2xrn+MlOmPLMN5o7FhO8qYOB7diPDVL+eYe/yQ9pEE3+HMmQxYGtzzoZd1sV+G5odnAKYhRPDnP0MDVeSKK7HEAJtrSX3Fh/vskZ6ncy7/yc2qoGOqFnqpDVWkmKly1OrgIGH1C+9PgL0J5qHrcVTn+v8ZFro3rfkKYH27lRl2k8yvg9UEa8Gqk3GHWDo1TMQM9XxGRJFIgzx/5qjj0qjPl7kJsRDLIYn+TzNPTadyxgeaeSO7z2stWm3LSstZa6lqfZFwLjhLTDYG1DybPN/Y7vDQEPyeSmGCYZwhMZk2Yi49NfnHbq2k7mfn1qwGrtWkU6XGpvYKS5XjFwwkZJbK7QHTbaAwvIoA47WSDU8yGIqn/64vlwSPZ2dGxIhjaIo73Bk10U5W7ftASlGzo1+CgnD0os0aNSRvdhuoCSUEc+mZsRQWm9G4KEj4GRL9zHVkIKUfpmHeMInG+oXQGV
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(136003)(39860400002)(396003)(366004)(55236004)(316002)(53546011)(83380400001)(8676002)(956004)(26005)(2616005)(8936002)(36756003)(31686004)(186003)(16576012)(110136005)(4326008)(478600001)(6666004)(54906003)(2906002)(6636002)(66946007)(66476007)(86362001)(31696002)(5660300002)(66556008)(6486002)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q2J5N3YzWTBIZENDRnJaWVV4NkhUdHFhSlFKdlBBVFRDQW8zemFhV2MxWSs2?=
 =?utf-8?B?M3E0MXRRTjNqS2x5VzFkbFlBOTc1djZyc0dsWngyNG5QN2p6SjJucGNFbjVH?=
 =?utf-8?B?Z1BQc1BoWXAyNHR1WkRMeUdGVGp2UnQ5bE5VQ0ZoSmVHWFFEaC9aL3RKcU1w?=
 =?utf-8?B?NDJ4K2MxeUJxQUQ5YU9IanZITTl2U25wZVh6b1hyekFBZUFOZHNzUFFHN3Y4?=
 =?utf-8?B?UGJoZitmZk9DakJEL3FScURtWHlYeS82Nko2WW1vZUdxaHFBT1BISWdESWZJ?=
 =?utf-8?B?bFVJenhNTFp2WmptMTlLSFh5NnNtVlVVb0wxRmhVeHZQUnRWSVdkLzdQOHJr?=
 =?utf-8?B?cVZ6T3VzRjVwbGFIbTlFbkYvMnN3aG8raTRXWkR2Um9uUDEyYU9wQmcyYUVq?=
 =?utf-8?B?K2J2Q1dzZnlLSFNDOExsSk5mSDBEdGpGK3dKOXZxTHhNaE8zTGNGQWQzWmVl?=
 =?utf-8?B?K1ZZOWdRL1plaEZSdnJwcy9xY3RqRFJhR2k5RFBwajhwakVXcmhjTENjRitB?=
 =?utf-8?B?cFJ0SWd3bEt4ek1pMlZXQmpIMXFPbGdNTitGT29lUkphU1ViZ2ZzS1N1YUJa?=
 =?utf-8?B?OTVSY3YzSDh3ckFnRCt1TEROL2pOVmZHL0xJcG1iRkxadGEzZEtXNEYzT2th?=
 =?utf-8?B?SGU3NHNheWx1SWs2Rlo2SVRLbWpCZGQvbzZ2NGRDKy9RNkNNSmJjT3phbzU3?=
 =?utf-8?B?eitXRW5iWWpxdHdSTEp4K1VyZ25MTlg2MjB5dTNyTWxrQjMzaDh1UG9kZGVn?=
 =?utf-8?B?REIwNloxYmdiNzFwOFVyT3ZONjBVaHBVZXBEMElxK1hXdzlITjltWU92L1RP?=
 =?utf-8?B?RGxYODUzcUVkOXYyREtIbUFqZXhIbDhBUXNIK3JpejRVWW02T3dLUnloMzVu?=
 =?utf-8?B?eDg5K2VFU1ZFc3pVQ0t3OE91R0dkK1RnVS9pd2F4WUx5dGZhWmlLZGNWdEtJ?=
 =?utf-8?B?L3VYb3Z1S1JBQno5aHhJMWh2M1VTbVpnQlh3dDU1dUhKQ1BZSHlFR2J6Z2gz?=
 =?utf-8?B?cGpwNUw4QS9jcXdrbFZhSlJ3SkZ0V2NtVy9JbC9Fc0I3aVF2RmNvZVBaWDEr?=
 =?utf-8?B?R3p4SEJrM1R4dndDZ3RJTjJRbldWbVVXK2NGQ0FxcE5NTXNZSmVRNUs0dzk3?=
 =?utf-8?B?YjBhSGxpT2doTkwyblllTEJaUEl0SE81bVZLZmVZQ2NzdWhPcjlJSFlTV0tY?=
 =?utf-8?B?YmNRODJBWDdhaTk4YytyTHV3WGY2NGpEejFxcWxhenhvSmlPTC9neXZ0bXRQ?=
 =?utf-8?B?T2dZZTg1c0hjMkVwZStiNXFkS1RwcDRXZDQ3b2c3aGMwYjJTMnc2bzRPZ2Fl?=
 =?utf-8?B?S0hQc1BTeTNIOHpGamcyRFNZSlR0SjViZ0NWZm9NRWswNjUrOEJKbjRxbUpD?=
 =?utf-8?B?RUkyNlgvZHo4RWYveTZUR25JVE9RYVduaVoyV205UWFnTTNtb0dqWmx6T3RX?=
 =?utf-8?B?NGQ3VjY5SXVFYkN3SmNBaTZhMTRuYVE5SnRLdlZKN0orSFJhZFNvUSswalB4?=
 =?utf-8?B?Y0xZWWVZTllHWStPdTI1cTdYbVRYMHJDc0NVWEdDSjduMTAzR0F0OVZsOWZG?=
 =?utf-8?B?VU5iRGJVaW5sYTVqMk1Gc05veEM0VkRJcExURWRXQ0VZc2prUXdRZ3VDQzkw?=
 =?utf-8?B?aFhGUkg2V2dHQXNDRElsUUgwYVBSUnl5ekFObm9tOVhpYjQyOUpPLzFtV0xj?=
 =?utf-8?B?WldteTJaTnRyckV0TVE3TXpVZjk0OVFtZGlJc1pDQXJWMHR2NG1jT3kwK0E3?=
 =?utf-8?Q?KdTslSsxjAgfCwqCMFhc9rZ9mPUC4izT+EdTWGZ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e97e60b-760e-4a32-97f0-08d971613b65
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:07:47.3045
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1POKSrEGRmFY4zTTBpwp50UhM/6sP/EJq5QlxM7n7uNq34mW+8nkCYSCaz9MQwo5/me1Mezze9qtOC1pvs17wH/JpbD4oED30DOOmtJPBTo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4997
X-OriginatorOrg: citrix.com

On 06/09/2021 16:17, Jan Beulich wrote:
> On 06.09.2021 14:00, Jane Malalane wrote:
>> --- a/xen/arch/x86/cpu/amd.c
>> +++ b/xen/arch/x86/cpu/amd.c
>> @@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
>>  			  c->x86_capability);
>>  }
>> =20
>> +void detect_zen2_null_seg_behaviour(void)
> This can in principle be marked __init.
>
>> +{
>> +	uint64_t base;
>> +
>> +	wrmsrl(MSR_FS_BASE, 1);
>> +	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
> While I don't strictly mind the "m" part of the constraint to remain
> there (in the hope for compilers actually to support this), iirc it's
> not useful to have when the value is a constant: Last time I checked,
> the compiler would not instantiate an anonymous (stack) variable to
> fulfill this constraint (as can be seen when dropping the "r" part of
> the constraint).

This is "rm" because it is what we use elsewhere in Xen for selectors,
and because it is the correct constraints based on the legal instruction
encodings.

If you want to work around what you perceive to be bugs in compilers
then submit a independent change yourself.

>> @@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
>>  	else /* Implicily "=3D=3D 0x10 || >=3D 0x12" by being 64bit. */
>>  		amd_init_lfence(c);
>> =20
>> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
>> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c =3D=3D &boot_cpu_data &&
>> +	    c->x86 =3D=3D 0x17 && c->x86_model >=3D 30 && c->x86_model <=3D 0x=
5f)
> DYM 0x30 here?

0x30, although it turns out that some of the mobile Zen2 CPUs exceed
0x60 in terms of model number.

As Zen3 changes the family number to 0x19, I'd just drop the upper bound.

>  Or 0x1e? In any event 0x5f should be accompanied by
> another hex constant. And it would also help if in the description
> you said where these bounds

>From talking to people at AMD.

>  as well as ...
>
>> --- a/xen/arch/x86/cpu/hygon.c
>> +++ b/xen/arch/x86/cpu/hygon.c
>> @@ -34,6 +34,11 @@ static void init_hygon(struct cpuinfo_x86 *c)
>> =20
>>  	amd_init_lfence(c);
>> =20
>> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
>> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c =3D=3D &boot_cpu_data &&
>> +	    c->x86 =3D=3D 0x18 && c->x86_model >=3D 4)
> ... this one come from.

>From talking to people at Hygon.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:17:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:17:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180176.326711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJBF-0007rw-Dk; Mon, 06 Sep 2021 18:17:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180176.326711; Mon, 06 Sep 2021 18:17:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJBF-0007rp-8S; Mon, 06 Sep 2021 18:17:17 +0000
Received: by outflank-mailman (input) for mailman id 180176;
 Mon, 06 Sep 2021 18:17:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNJBE-0007rj-4y
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:17:16 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a94a317c-0f3e-11ec-b0b4-12813bfff9fa;
 Mon, 06 Sep 2021 18:17:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a94a317c-0f3e-11ec-b0b4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630952234;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=bC+EJiQzJ/3oT+llAauPcqganhRQKwalHRfyaI9+9Tw=;
  b=UezwOD+EOtgh0fPRXulGw58z9WTzbDOBqiK5cnkwZvBltQhCdJYZRgog
   IZJSIPSyf4eGiDUr9rXPb4KJzLrA60CaLiE03QS9CD5WlnTlQ6dbsJFx/
   hyjWyeEOtG/2g0B648mibq6hZekbmLGa4ySXBbPDvMCyM7FSvdcgvkfCb
   Y=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 3O2i/bW5wEkgN2rYiFXpS6Zu53HhjTcrLQivkUprjMnlCrQboLUGdz9To1FsjgsxmYXi9XU/tZ
 9lrTUaoxNa6QwnsJdbfizSk4O7ZdiujrkL/yf5DHpGrYcdaqBeBOwR7aCzWPCGo2NC91LMLnAV
 90aSmINcb4Byr0O2vgB3Ysr4PXbBK7x6gLt5I1LA56wM1PTbCGcVC/+vN7emKpFodAhK1phmAK
 Vbo9UqwtKUJGU8L+9SH6qzrHAs03omVMTut6r3+HtuekjTDpWt3HaFT/Hsl5XD7sDxvFR9t7rT
 DEHD64zkBYhmCJ7jVAKGr5gv
X-SBRS: 5.1
X-MesageID: 52105057
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:DwSEQaxFyy9SLGcW7Pm9KrPxvuskLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjlfZquz+8K3WB3B8bcYOCGghrVEGgG1+rfKlLbalbDH4JmpM
 Fdmu1FeaDN5DtB/LXHCWuDYq4dKbC8mcjC74qurAYOPHNXguNbnmBE426gYz1LrWJ9dOME/f
 Snl696TnabCA4qhpPRPAh1YwGPnayGqLvWJTo9QzI34giHij2lrJb8Dhijxx8bFxdC260r/2
 TpmxHwovzLiYD09jbsk0voq7hGktrozdVOQOSKl8guMz3pziKlfp5oVbGutC085Muv9FEput
 /RpApIBbU911rhOkWO5Tf90Qjp1zgjr1fk1F+jmHPm5ff0QTorYvAxzr5xQ1/80Q4Nrdt82K
 VE0yayrJxMFy7Nmyz7+pzhSwxqvlDcmwtgrccjy1hkFacOYr5YqoISuGlPFo0bIS784Ic7VM
 FzEcDn4upMe1/yVQGYgoBW+q3oYp0PJGbDfqBb0fbllAS+3UoJjnfw/fZv3Evpr/kGOt95D+
 etCNUhqFgBdL5OUUtHPpZ0fSKAMB2Fffv9ChPmHb3ZLtBxB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52105057"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=chiLBGFVJPSEpysz1lpQqJNjZC43D+rFqTOt+wtJmahEFMd8otEPIfkJjwO3FeX3366vXHQAD18IgSAdq0uFeqSftqhLjtt+qpYSvXpcT7qq1PlDqd4x9n7DYf4psM0Siip4oSokG3bGEjF/egEyjJLGoA3JANBrWOAmsq/3916tGUvLXXsptxKhOIfJCNmaiuc1vv8SL6CK7P21bdME+/dTorlXtwrqLpJJtI8Hop4M6GyFlKnI0FZkbgZler9H4LdTxM6Ov5ztuHeZEHYIWqJP1c/56G66lVxqF/OIoZ6UblJd0Lwg98e63aPhzKhZrctvHyYIoK49L2WuNL5/Iw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0694yTCPGieLg9B9FN4YDvoVX+IZkzSt6FqWq++RqgQ=;
 b=jKSvGNIKr0tjVgU6NnrBVv53PlUodnuexs10miwY96ZYLSCOrOl8BmMs38nKN5VkIYcTA7PqNKgBXGBF8hO1TpuHIxDLylMQcDfrNFAD6e+iqaQS/7+Z+D0By+sG8f9DW6UVQTBaLTZov50xESoxh24vawVTTd7FnLTRcI3P+OyylB93PPVp1R217dDyhIpfUAA1OVFHahU4gTh0Pk6LIWpuZ6rx1f63tgpKDPqSAPzCfDNtPo4hurxKwAoIL+FesLTmzq0ZCHUE1gdr2UNOx4czmN8dlXmhg7d86hW9rxsvACAEoaKB/1hYoEIrtN9XVPGLo4EXYEdQ4LvGhAxc1A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0694yTCPGieLg9B9FN4YDvoVX+IZkzSt6FqWq++RqgQ=;
 b=NuniERP551CZ9ZHg2P4cauD4vyGOCdcuAhy8WQNHD0PcIO4kmV0h1Na0dcA91m6iHIHgWQR7AWPN2QHtzffo/ph4bvUPa5V5+GeMZ9QDkETYMh9bNSojKJDmCx+ou2/CmLTPAfzwjByBaMhOAhnEjtE9vdltM5vvcShY7os4YYg=
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
Message-ID: <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
Date: Mon, 6 Sep 2021 19:17:02 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-5-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0026.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::14)
 To BYAPR03MB3623.namprd03.prod.outlook.com (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f8e8915a-9a06-4e25-7728-08d9716289e2
X-MS-TrafficTypeDiagnostic: BYAPR03MB3670:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3670C1D3B4AD229930CCE152BAD29@BYAPR03MB3670.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: a6PZ/TDbf344B4JInXfmap5ABOq96iIEUJGbtaM2XsQwKs4iNQmu4G1AvS+Fc7HH2/bZwyERHyYrkTkc/TcVDz0CHHQevCXM0IM7wqIvpVxIeDh8R5MLkdK2mPgWBPDjedmn1mEajFrtUlrGKo38hvgltwuhpFBFz5hPv/Ls+kwHSklX1XU1kaUWeorPaGMs0C36h8G4bplKThg5y834/8IWkS8Sps1NfkRL2ebpdSttlZldzi3+YnSIVxUBHfRfNGhbSlq31SL5hq+hlceDExbadCDBJGaIOzSfMBBidkym5NNIiREMiwwpHRmYa1UkcaQ6fw4QnEe0CxWJqF1EhxV+cwBDJFqiF8Sj7IHBX8TZWYjH+6AArFarlE0avXL6fXlbwH0xa0TiVv40XRSiWS1rfsKG1F8mFHOero+64OxkeJLZbtpEoDomV2X+ZIiOYtzpWORxbV6/gmdKifMSbSlVkDmnAJ+VjepK7ppZj4d7VVr9ufoZW3i+9IC7tIoHCXhTqVP/yAQ8DhEWKTSmIxdNcDwY1X3oaAA1aOErzIgDP5td79dM/nxSTlQqiTOdUZMgTSF3SmwcuO5gvTlETOjCdiIiyQCHtvS4pTkZhxKGhLOdtX+wqXRc8fbcstdwwMcXac6SHSs7Aki2l9naR9c+UT+0LYgdRmcrrL7PXqs7VNldGFAqr/E9ZngvQgj3Ne+ZDCQYZ3qSu4stlnLxH/9ITTWoh1qvSTMpe0jD5PgcWtH1V9QX+9/D97+erB9Y
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39860400002)(366004)(376002)(136003)(396003)(8936002)(16576012)(5660300002)(4326008)(316002)(66946007)(6666004)(186003)(956004)(8676002)(38100700002)(26005)(83380400001)(36756003)(478600001)(2906002)(31686004)(6486002)(31696002)(55236004)(66556008)(66476007)(2616005)(53546011)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VkNsV2JtSFBGNUNGN2NmUUtrcXV2aCtTS2dNQVdhZnFESXUwVElwNldtbm9I?=
 =?utf-8?B?eTZOZDdmNkhhZjJPUDJQRXdsREpreE9NM3JHZXRiUzZzOG9ySnFaUHRaR21p?=
 =?utf-8?B?aWNoVGtrUXV4TG9EYzQ3N24xVmE0WEFwcmx1N05VVlg0c2lEbmFtMS9qZXVU?=
 =?utf-8?B?a1ExSzZnZGVtSlkrNkloTmM1dXloaGFNcmVkUWFtcTJEWE41bm1Gd3ZPNy9t?=
 =?utf-8?B?SGtIdjlXb0Y4L2t4TUVxamlXTGdYcHhlUGo2OTk1RXkrRmIyd0ZNKytNcFBZ?=
 =?utf-8?B?VUJrNmZ3OTQzanBkMG9YVGFWb1JocXdtZ0d2OUJVTWoxdFJuMklXZ2Fwc25Z?=
 =?utf-8?B?YnZZSTNqcWVOQjRIcDdld1I1OUM5ZzBqNjFZeTlxVGRNN0IvM1EzR0luTlpJ?=
 =?utf-8?B?UWV1SnZRWjdUVDdCUVh2VzB3UGxLa0ZtUTJNSm9HRmFCdnUyaVM3aFRXQ2E4?=
 =?utf-8?B?UEJrWENnK1I0OWsxc0gvcWRNa2QyQTRHcU1jVDJxZG9Ga0FCRm1uYjljWnp0?=
 =?utf-8?B?NGxLeUZiVHdSUGZ0UU9qeDRvZklldm4zTURIZFFlUXpsRFBZMGZZZkRhTGEz?=
 =?utf-8?B?YUMrZkErMThXOWpnbnlxM2tRQUF3RTVUZFVreFp3QlFYRmdYRUtyeWZzVU55?=
 =?utf-8?B?THE5QWl5TjRoRDJybmVBRzhCUnpoaDBRR0pTSGVJVndIRTFCcFVVTmlrT2VT?=
 =?utf-8?B?RytGR0pITHprTGVLaUwzdjBBT0Q5V0RCUUdTbW1EZU1Hb0kvNXArOXpTZHFW?=
 =?utf-8?B?MEhUMFZ2MHB5NmlaSjFkSGFjUnMrRjBNck1zaFo3VDg5cGxIMTl0Q1doQyto?=
 =?utf-8?B?dW1oSEtvU2t3bzRmbmhLejdUK2hxYXR1UjNzZjhKbVRJcUdDMW5nV2toNENV?=
 =?utf-8?B?UEt2U1QrS01pZGFmcTVCTXh5S1BKSFE5dUhYVGtHQ3RMQ3lMbjZ4WHloWGRq?=
 =?utf-8?B?aXFlWVBrL1FJRUtFSUJ0WXdjSW1NSkZocFhZVHUvZU9qcVo4UmJ1ZXdGZmxP?=
 =?utf-8?B?ZHI3RXYrSHRQcGhnbE5ZcjRSM2Vwa0ZFUm01U01uMlpscndFTkhnVHplTGhw?=
 =?utf-8?B?ZG1aeVlXRnRLVkZpd0d0Vllac0JxWFcxTzl0OFQrcHNuOGZFODJWM0srWC9B?=
 =?utf-8?B?NElhUlVIcmJxV25XNGNYekxOUzhJd09ZbktQZndWS2Y1WlZWMnNCMmJGem51?=
 =?utf-8?B?RUZHSHJyUTZ2YzN5anN3MUl0c3pjbldnVndzQ2ZqK2NIYmxIMDc5M29NM2Vv?=
 =?utf-8?B?b3lIQmQwVWRpbUw4NFNCY3BGYk5XQnpkWVNVR00rMll3d0RBQ1FtQ291SXd6?=
 =?utf-8?B?NStFbDVJeTdvWUNBWTM2Z3hDakRBQzAwVjcyWHlxc3B3cVA5UnpOR204Zk9S?=
 =?utf-8?B?WGxTVlZXZ2kvSnBOVnB2UHhxOTRNOUNHZEpna0hFZ1VFcVJUdE9iQVA5NndP?=
 =?utf-8?B?QkJqQmVDVzdmQVZKamI5d0xCVkxOd3B2U1l3QWlNK2kzbjcwRVBZc2FOdXha?=
 =?utf-8?B?WjRJQXNUUy9GZWJoM3N2VllkQ0pTR0I4ay9GOWlEZjU5UjlwL25Vdk82S28x?=
 =?utf-8?B?R1U4TFFObHpjUnF5dmpQVjN3bHJsT2VaZGRIYkNHVHhvZlYrUWRvdE9SdHlP?=
 =?utf-8?B?MzlCcU9tSGhuS0FDTExDQnhsMlJSSGxXM2ZFMUVuOHNwMFlvSS9qeXlEcm9B?=
 =?utf-8?B?MndrbGV0L1F5VGFJWlZ4Tkw0OFlpSm9YU0JJL0Vla2pIZXFscStRVzM0S2RL?=
 =?utf-8?Q?8p6YepYFOofhNmNFv258q0hcK8g3SiBdAiG/uHn?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f8e8915a-9a06-4e25-7728-08d9716289e2
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:17:08.6837
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OD8QvT/kiWxR2XjqyiYJ7J1flEiq1/2bmjeGNYGj2Ni7HmZCEaZJou5Djnf5JhHhW0QaODNzRPMIVW7USo2V6REvY/S5emjvwjE4Vo/+JsI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3670
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> Instead of intermixing coding style changes with code changes as they
> are come upon in this patch set, moving all coding style changes
> into a single commit. The focus of coding style changes here are,
>
>  - move trailing comments to line above
>  - ensuring line length does not exceed 80 chars
>  - ensuring proper indentation for 80 char wrapping
>  - covert u32 type statements to  uint32_t
>  - remove space between closing and opening parens
>  - drop extern on function declarations
>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> ---
>  xen/include/xsm/dummy.h | 173 +++++++++-----
>  xen/include/xsm/xsm.h   | 494 ++++++++++++++++++++++------------------
>  xen/xsm/xsm_core.c      |   4 +-
>  3 files changed, 389 insertions(+), 282 deletions(-)
>
> diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
> index 214b5408b1..deaf23035e 100644
> --- a/xen/include/xsm/dummy.h
> +++ b/xen/include/xsm/dummy.h
> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
> =20
>  #endif /* CONFIG_XSM */
> =20
> -static always_inline int xsm_default_action(
> -    xsm_default_t action, struct domain *src, struct domain *target)
> +static always_inline int xsm_default_action(xsm_default_t action,
> +                                            struct domain *src,
> +                                            struct domain *target)

The old code is correct.=C2=A0 We have plenty of examples of this in Xen, a=
nd
I have been adding new ones when appropriate.

It avoids squashing everything on the RHS and ballooning the line count
to compensate.=C2=A0 (This isn't a particularly bad example, but we've had
worse cases in the past).

>  {
>      switch ( action ) {
>      case XSM_HOOK:
> @@ -99,12 +100,13 @@ static always_inline int xsm_default_action(
>  }
> =20
>  static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
> -                                    struct xen_domctl_getdomaininfo *inf=
o)
> +    struct xen_domctl_getdomaininfo *info)

This doesn't match any styles I'm aware of.=C2=A0 Either struct domain on t=
he
new line, or the two structs vertically aligned.

It more obviously highlights why squashing all parameters on the RHS is
a bad move.

> @@ -291,37 +307,41 @@ static XSM_INLINE void xsm_evtchn_close_post(struct=
 evtchn *chn)
>      return;
>  }
> =20
> -static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d, =
struct evtchn *chn)
> +static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d,
> +                                      struct evtchn *chn)
>  {
>      XSM_ASSERT_ACTION(XSM_HOOK);
>      return xsm_default_action(action, d, NULL);
>  }
> =20
> -static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d=
, struct evtchn *chn)
> +static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d=
,
> +                                        struct evtchn *chn)
>  {
>      XSM_ASSERT_ACTION(XSM_TARGET);
>      return xsm_default_action(action, current->domain, d);
>  }
> =20
> -static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1=
, struct domain *d2)
> +static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1=
,
> +                                       struct domain *d2)
>  {
>      XSM_ASSERT_ACTION(XSM_TARGET);
>      return xsm_default_action(action, d1, d2);
>  }
> =20
> -static XSM_INLINE int xsm_alloc_security_evtchns(
> -    struct evtchn chn[], unsigned int nr)
> +static XSM_INLINE int xsm_alloc_security_evtchns(struct evtchn chn[],
> +                                                 unsigned int nr)

I maintain that this was correct before.

> diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
> index 9872bae502..8878281eae 100644
> --- a/xen/include/xsm/xsm.h
> +++ b/xen/include/xsm/xsm.h
> @@ -19,7 +19,7 @@
>  #include <xen/multiboot.h>
> =20
>  /* policy magic number (defined by XSM_MAGIC) */
> -typedef u32 xsm_magic_t;
> +typedef uint32_t xsm_magic_t;
> =20
>  #ifdef CONFIG_XSM_FLASK
>  #define XSM_MAGIC 0xf97cff8c
> @@ -31,158 +31,171 @@ typedef u32 xsm_magic_t;
>   * default actions of XSM hooks. They should be compiled out otherwise.
>   */
>  enum xsm_default {
> -    XSM_HOOK,     /* Guests can normally access the hypercall */
> -    XSM_DM_PRIV,  /* Device model can perform on its target domain */
> -    XSM_TARGET,   /* Can perform on self or your target domain */
> -    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
> -    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations=
 */
> -    XSM_OTHER     /* Something more complex */
> +    /* Guests can normally access the hypercall */
> +    XSM_HOOK,
> +    /* Device model can perform on its target domain */
> +    XSM_DM_PRIV,
> +    /* Can perform on self or your target domain */
> +    XSM_TARGET,
> +    /* Privileged - normally restricted to dom0 */
> +    XSM_PRIV,
> +    /* Xenstore domain - can do some privileged operations */
> +    XSM_XS_PRIV,
> +    /* Something more complex */
> +    XSM_OTHER
>  };

Why?=C2=A0 This takes a table which was unambiguous to read, and makes it
ambiguous at a glance.=C2=A0 You want either no change at all, or blank lin=
es
between comment/constant pairs so you don't need to read to either end
to figure out how to parse the comments.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:27:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:27:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180183.326722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJKb-00016Y-8Y; Mon, 06 Sep 2021 18:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180183.326722; Mon, 06 Sep 2021 18:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJKb-00016R-5b; Mon, 06 Sep 2021 18:26:57 +0000
Received: by outflank-mailman (input) for mailman id 180183;
 Mon, 06 Sep 2021 18:26:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNJKa-00016H-7J; Mon, 06 Sep 2021 18:26:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNJKa-00027m-0y; Mon, 06 Sep 2021 18:26:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNJKZ-0005bA-NP; Mon, 06 Sep 2021 18:26:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNJKZ-0004PX-Mt; Mon, 06 Sep 2021 18:26:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lWdWNKadwgxsJNhNnduT+sZdtFoV9PHBnDpc+LCqw2k=; b=ajhqKq/NWQoPP/2xQukCuNGorP
	JvA/oNrbcKOz2eu6RcZrB+Iv5dwIdzXiprvVUpPS4nMjHHzUzJ0az9bpGOyEU9VLZvzEIUyzSDl+Q
	s54tVYptwUsqmG5BHBQyVKgeyIcu+2TsQKAlBXiIMz7gR7nxl0P5NpqV1tc83YBHbIsc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164862-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164862: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=4473834e7d49c555eca81f96383a1d6d6f5f4bb2
X-Osstest-Versions-That:
    ovmf=cae735f61328d64e2e8991036707b9e78c0f5f63
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 18:26:55 +0000

flight 164862 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164862/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 4473834e7d49c555eca81f96383a1d6d6f5f4bb2
baseline version:
 ovmf                 cae735f61328d64e2e8991036707b9e78c0f5f63

Last test of basis   164674  2021-08-31 02:56:52 Z    6 days
Failing since        164686  2021-09-01 03:03:43 Z    5 days   69 attempts
Testing same since   164862  2021-09-06 15:10:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  DunTan <dun.tan@intel.com>
  Gary Lin <gary.lin@hpe.com>
  Hao A Wu <hao.a.wu@intel.com>
  Jim Fehlig <jfehlig@suse.com>
  Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Loo Tung Lun <tung.lun.loo@intel.com>
  Loo, Tung Lun <tung.lun.loo@intel.com>
  Mark Wilson <Mark.Wilson@amd.com>
  Marvin H?user <mhaeuser@posteo.de>
  Michael Kubacki <michael.kubacki@microsoft.com>
  Rebecca Cran <rebecca@bsdio.com>
  Wenxing Hou <wenxing.hou@intel.com>
  Wenyi Xie <xiewenyi2@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   cae735f613..4473834e7d  4473834e7d49c555eca81f96383a1d6d6f5f4bb2 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:31:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:31:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180190.326736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJOq-0002if-Rm; Mon, 06 Sep 2021 18:31:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180190.326736; Mon, 06 Sep 2021 18:31:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJOq-0002iY-OS; Mon, 06 Sep 2021 18:31:20 +0000
Received: by outflank-mailman (input) for mailman id 180190;
 Mon, 06 Sep 2021 18:31:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNJOp-0002iS-B7
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:31:19 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a00a8fe2-0f40-11ec-b0b5-12813bfff9fa;
 Mon, 06 Sep 2021 18:31:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a00a8fe2-0f40-11ec-b0b5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630953078;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=8cVMSSNJXfkVJJAEzk8daSd7j4AUn+dcJl/UkByNwW4=;
  b=ILqVKZtmchjMuajH10PxIaNDcKCvA5XhwLsP9aTGG421mzUYyXt6JkxI
   yV5EnYe7C28URzDQ4KoQnL9d3/LYrtuVhcSOscFRJiHM75QegNpmO3jLA
   OOdpyERrOOlqbcq2GRS3fz194kjf52DKFAeXcy2a1XUgUDEI2JANl6r9S
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: f3l083uFU2Pk3PCfh8bZTRZZkEJr1TAghVd2Qds3PcS9aZydkc7hSGIgAfpN+ADcCypXcj6BzB
 hAFqmkCxg+NWgJaNECHrLiMa2Yy/0EMjzNBd6GAaC9MRXj2HNy9yudc7rXhg3T969k+REy6lCF
 e0T2PFBG8pDXi38jy3PKWfUftAj6P7em3DY8Bc2nDC9D3Zp85FZjrTIDS23BOHOfweNz/0mY6m
 3XvGqfZ+V+xUIiwoM93KMMN88RlcZrnf65ryGZiN9gyErKQLkIE8iGYsEoE2hX/rHq3D114UaS
 CdXmpda65LNNHIyBLcJK3QBY
X-SBRS: 5.1
X-MesageID: 52507244
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:6q6oeaMOWHKamMBcT1P155DYdb4zR+YMi2TDiHoedfUFSKOlfp
 6V8MjztSWVtN4QMEtQ/exoS5PwP080kqQFnrX5XI3SIDUO3VHIEGgM1/qY/9SNIVyZygcZ79
 YcT0EcMqyCMbEZt7eD3ODQKb9Jq7PrgcPY55at854ud3AMV0gJ1XYINu/xKDwOeOApP+tdKH
 PR3Ls8m9L2Ek5nH/hTS0N1E9TrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqJmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87CsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXkHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1XackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMW9yV0qp+1WH/ebcGkjaRny9Mw8/U42uonlrdUlCvgklLJd1pAZHyHo/I6M0r9
 gsfJ4YzY2mdfVmGZ6VMt1xCPdfOla9NC4kD1jiVmgPNJt3cU4l+KSHrYnc2omRCeo1Jd0J6c
 z8bG8=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52507244"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fwa9ylUfSuABZrSS4VpoSNl8yzUqMZFvQLdcnji+e7Z4yYxOTjomuz+gLD8XwjBZUYFLlw6DCM49OWNf7IeIciCrNG7H0YAGHM/5leksFZ4FtmzTiO+ChYA0gRty1tZODKDjcNIHC/LAe9HiqfTxrfnK4EH8ReqSJf0oVurJCDWHRPh9CxPyuvChpZhc74MtAN/RDSQfJPS09HKncGBHM6JXK867EOWFzH3YjwouQKM/5klr66W4+QSKnqVYDmy5Kkv1ETusf7IZJJpJLt07PF0DFFLOL+weG+TRZCE78CM087X5gPgYUdXALNHXsDp7LFbouVccxicmtARACx4Cbw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UygBNLCUMU/3ZqHyK+ayd1mRTwySTUAf+mwn1mECLiE=;
 b=CSSVsWFQk6N3WwQGZemL4ItwcNOfTYDoPBsq8CK/wiArGQ9xZP85j7XN1QHeo+JkN5TdfVJFE3CWkiBRpNMYPRrH4c8uD9GkLJMpeQmPhNw1E3Rl/rz4pukuXXjfeoIwYbsxHBDB1I/oHfk1z2k2+j3xzHXtPHv0g+8GHLfBqotOlX35tVMVBb8ZwYb714m6TwFJsnr4MGT8Lid0oVVf5GnTX/h/zv4UCO6vvORQoG4yoCNiSz8HdN1bE8ZRWtsFkCEqtckbcNeF4MWWvqn4zsPtM464XYs/1ViQEvSboW7OCbLAAL/3G0JOxIesbcIncyMP9gbxeGSrfkNqsZSsFQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UygBNLCUMU/3ZqHyK+ayd1mRTwySTUAf+mwn1mECLiE=;
 b=H1L7UWXsM6V4y6upF2osuLaLzEZVcTuD4eyaO0jpF7LE3BwkjyErma9wJiGwKyVJUD3ZVMkvKgPGkRH9Q9FtncARmnWxb2PZQHpgQ6M/H1xKfNzulvc7RV8DEzJLm1jkSZL5wYVKk7GB7TN9mC6t8qTRO809NvTnu8fvd900Odk=
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-6-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 05/11] xsm: refactor xsm_ops handling
Message-ID: <6924c3ea-de14-60e5-d3dc-eaff6c228f1c@citrix.com>
Date: Mon, 6 Sep 2021 19:31:06 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-6-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0189.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::33) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2dba57e0-6047-412d-26ff-08d971648152
X-MS-TrafficTypeDiagnostic: BYAPR03MB3670:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3670B3BD362C2134D712D0F4BAD29@BYAPR03MB3670.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dT1eH4rIP3aS86VlDkv2x5BJ60C/cH1bVZw+toenIapafmdyTK5v/m6cil3uDwYU0XW7v+Jpms3TWwCKnc0j2G9pJUQt+EMgPV8xwEnKNcEJP2xK8x6IS51rAIxH2gAP1bH9sado7MBTUFatQs9v2PhmOfe/dWwEnqsyH/EpOAs7WO0OIl4M5/7IAbiucc1tOZBxA/pTsw/Jx/NpSaHs0o0CSEgINk3Gu2EP8yOvzCOYF7CZEoRg+gGNbGgdWI3JW2jA+HPrcnlaHi+nnk5vgHbQhWjN3n9ldlA18j4cRUFv+s/eWpQqHUCuDSjUtexJ8/4YskW/OsrKQbzC3WQbiWtj6pYOs4/M+o4teEQuc/L23N+ICVdktLtXZSus5lM3KgvwVp45lr9MUObjLFl/KoCglQ9CAxSGX/geGeAVrQmssUN5FyNxCHAyQ0wFfaLO6JhSbEKKTZdTes/rxQYdqjfosG75cI6UgbR+Ewt2HqM7MoRzDcbaKtG0PfB6no+rcyv+YPWlTjCyGXJfRj0H5ji+UxcWMJqYk48KDVcTAZzk5N3E94U4jIse5Yfn7AxiH5hIiSCG3HzgZkLI1bp1Sw5NI1/Lh5COxnO++IT8dMB8P/pyhIQv1dgQ9TkZSIDcoSFqguV/fe3x++Nnc5jtIGRWUUaPxot1vau5eiTPmuk5PV4o+C9iFB1fANsuGuUR1tURv5JrazOLsvOmDg2BafvmUulwd42Kl2jIyjUG2p8=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(366004)(39860400002)(346002)(6486002)(55236004)(31696002)(86362001)(53546011)(2616005)(66556008)(66476007)(66946007)(8676002)(6666004)(186003)(956004)(316002)(16576012)(5660300002)(4326008)(8936002)(478600001)(2906002)(31686004)(83380400001)(38100700002)(26005)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MUV0UnRWcVovMWNsalFzQnB3MWhlZ2J5Rlc1cWdSSzBxZWU0aHJjUTBoVFAr?=
 =?utf-8?B?d21SVjIydXk1QUVWdmgrbGdEYzlDOGROeTJsNDhieXQ5VlgyTnAyRWdsNTJw?=
 =?utf-8?B?S1owTHRZRmxyMGRlK1dVR2hLM0F2SllGUG9ITDJZZURhaFVJVXo3eVpIOTNs?=
 =?utf-8?B?c1lsM1pJTUFhQStWZGNGWVRsV2crS1ZkcGJQTUREZ05HeHNMTzNlbUxTd2da?=
 =?utf-8?B?N1hpaFhPS2tuMURrU1dNMmxpZHVYeWsrajRVZG5KamtzSlNnZmc0SU1mL0tZ?=
 =?utf-8?B?TzdkaFlpcmFyK2FKa0pXdkpuY2Y2dThnUElwQy9GQjVNQ3pqSS8vR3cwTXVu?=
 =?utf-8?B?R1RBVFRjTTJqWVQrUlpJbWNVc25hMUpGN0Q4c1lMbVkreUpNZnV3YXNBYnhw?=
 =?utf-8?B?cy9SeTByb2RHR1VEUUViNGtoRHdyR040SnRxQkRwOTVHNW04SXBnS1N0T0xU?=
 =?utf-8?B?dDZqMjIzbm5ZSmdDNTdPTGF3S3pWd0JVcXNRS2ZhemNyOVNwdVZDMzlqcGpx?=
 =?utf-8?B?SE5XM2FId1I5K1BuU0NCZDdlTXhnbVRQS08rUFFJamdkVmR1ckZmNjZkMUlp?=
 =?utf-8?B?bnBpRm1aWmg2SWJUTUxKb1h1OWc0aUcrY09lQnhLUXlrcjBGRjJzNy9VN0o3?=
 =?utf-8?B?a1o4SjJ4MDVNS0luR3R2QmtOYlBLeTJDSFV3MU5UQjkyZFB5K1JUVWRMa2Rv?=
 =?utf-8?B?OUhqTU04VmpYQ05ub0M3aklFaGtYQ01RTUh4c3dpeURydUhxaHpBVkJGVEY0?=
 =?utf-8?B?eEwvSTBrajQ3dkozcUpvQ1BKQXF1TnBhV3NKVXFkWlZCSTZCbDlCNUtDMTB6?=
 =?utf-8?B?V2lOc3EwWVRkYzNzZHhIbEhnZ3g1WGNiQnVSMjJGcVBKNzVWS01xaHNWb0xh?=
 =?utf-8?B?b24rRE5XZTJuZnhlNmJKNEgvMTExODg5aXB1RDBaWVczbFJPREFGcHVwcnIy?=
 =?utf-8?B?NTVnR2pZeXZyN0xPWWF1andkaGdadEFpSmNRcGlBREVtWHBTYm5jTWk1d3Vk?=
 =?utf-8?B?bEdHK29DOEcrdEdJdy9ZRS91dk0vQ3pkUXBxQm1yWjFPWVZ5N2lqTlFlKzk1?=
 =?utf-8?B?ZENsWnhSNFRveDgvaU5DeXpwL3p2ZnRLVE9TL0NTcDV5aHVNY2x5SCtOMmRV?=
 =?utf-8?B?NVBoMjIrUFRNNDdiZzlSWWJwcEpFQXhVYi9VNjZUcE82OVhnb29ERmJleG53?=
 =?utf-8?B?Wnloc0JVLy9GVld4SDJTOEprNThVWDByQXgxbDQ1UUxSSGkwRmRCNkJ2M0c1?=
 =?utf-8?B?dTNsS1RGUEM4TkQrbzBpQlpZYnQrRmRJaVFYUERidGloRkl0TkRTdUFoTkFi?=
 =?utf-8?B?SXhkN0dXNkVzTjFldlhBeTlkWXgxd0FaM3hFZGY2MzNFSmE2U0ZmZlhSSUl4?=
 =?utf-8?B?RHNuU0hUL0ZrY21wUEhvYThhYXhuWjU0K29PYmFCcDgrY3orazNEQ2N2QWpo?=
 =?utf-8?B?aEZoQVJaa3p2V0MybklRbmJ5em5xWGxjaVVHTXh3MVYvTmlRMmU2ckJuU2FK?=
 =?utf-8?B?ajdWYmtaYk1yQllJaktJR1VhTm9LM29NNDExYXBMbmw5SUpwVjc4QitCUWhI?=
 =?utf-8?B?Uko3aXZrOXUxM04wd2RQVFR1Z3dTL0VNSEJGMHlkM2VJN01ieWFkMllKb0Rj?=
 =?utf-8?B?M1FxZGs1TnQyeXEyNm0xV2JRU3VWaVFLR3pGbjkzM0MvVFBXeS9xd1ZxRTJI?=
 =?utf-8?B?N3JiTm9OUzk5eUVqYmtIaFVJUXYxUVBWdXlLRkE4UENWV3lVZUlFSHlNSCtU?=
 =?utf-8?Q?PyNVQupjJsp1upH/dTCmWPxOppur4HzznyqVfEX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2dba57e0-6047-412d-26ff-08d971648152
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:31:13.3017
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 868MW9CBky2jNc+ruARLvLc5apMjSc72Ah+7sGhQ/frFMy1I92p9jUuAEe1gEtJaSLAnGnBGYF3YeiQp0x/bj/JBREdngnVqFwWQKS63/RA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3670
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> This renames the `struct xsm_operations` to the shorter `struct xsm_ops` =
and
> converts the global xsm_ops from being a pointer to an explicit instance.=
 As
> part of this conversion, it reworks the XSM modules init function to retu=
rn
> their xsm_ops struct which is copied in to the global xsm_ops instance.
>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

However, some suggestions...

> diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
> index 55483292c5..859af3fe9a 100644
> --- a/xen/xsm/xsm_core.c
> +++ b/xen/xsm/xsm_core.c
> @@ -28,9 +28,17 @@
>  #include <asm/setup.h>
>  #endif
> =20
> -#define XSM_FRAMEWORK_VERSION    "1.0.0"
> +#define XSM_FRAMEWORK_VERSION    "1.0.1"
> =20
> -struct xsm_operations *xsm_ops;
> +struct xsm_ops __read_mostly xsm_ops;
> +
> +enum xsm_ops_state {
> +    XSM_OPS_UNREGISTERED,
> +    XSM_OPS_REG_FAILED,
> +    XSM_OPS_REGISTERED,
> +};
> +
> +static enum xsm_ops_state xsm_ops_registered =3D XSM_OPS_UNREGISTERED;

__read_mostly, or can this be __initdata ?

> @@ -87,25 +88,35 @@ static int __init xsm_core_init(const void *policy_bu=
ffer, size_t policy_size)
>      }
>  #endif
> =20
> -    if ( verify(&dummy_xsm_ops) )
> +    if ( xsm_ops_registered !=3D XSM_OPS_UNREGISTERED )
>      {
> -        printk(XENLOG_ERR "Could not verify dummy_xsm_ops structure\n");
> +        printk(XENLOG_ERR
> +               "Could not init XSM, xsm_ops register already attempted\n=
");
>          return -EIO;
>      }
> =20
> -    xsm_ops =3D &dummy_xsm_ops;
> -
>      switch ( xsm_bootparam )
>      {
>      case XSM_BOOTPARAM_DUMMY:
> +        xsm_ops_registered =3D XSM_OPS_REGISTERED;
>          break;
> =20
>      case XSM_BOOTPARAM_FLASK:
> -        flask_init(policy_buffer, policy_size);
> +        ops =3D flask_init(policy_buffer, policy_size);
> +        if ( ops )
> +        {
> +            xsm_ops_registered =3D XSM_OPS_REGISTERED;
> +            xsm_ops =3D *ops;
> +        }
>          break;
> =20
>      case XSM_BOOTPARAM_SILO:
> -        silo_init();
> +        ops =3D silo_init();
> +        if ( ops )
> +        {
> +            xsm_ops_registered =3D XSM_OPS_REGISTERED;
> +            xsm_ops =3D *ops;
> +        }

This if ( ops ) block can be deduplicated by moving after the switch()
statement.=C2=A0 It's going to be common to all everything except dummy.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:48:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:48:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180198.326747 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJez-0004Xk-Am; Mon, 06 Sep 2021 18:48:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180198.326747; Mon, 06 Sep 2021 18:48:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJez-0004Xd-7U; Mon, 06 Sep 2021 18:48:01 +0000
Received: by outflank-mailman (input) for mailman id 180198;
 Mon, 06 Sep 2021 18:48:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNJey-0004XX-3S
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:48:00 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f40e3060-0f42-11ec-b0b5-12813bfff9fa;
 Mon, 06 Sep 2021 18:47:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f40e3060-0f42-11ec-b0b5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630954077;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=r6dJB/O4rfn/f1vnVoxveZOWS+xUToPTgwygMcGwfpI=;
  b=Rsjv8lehegNjo5qlw5lNcflDBVWAr7HOW+jQRK8K6VmmMoR6tzvMUSRS
   Ua/9cl6ZIlqcKooSGPBqECPnzOs8G0F96YnwB+xcwb0nqrRyRP2riLqjJ
   99n6CfJ32ccEpjMmLMIuwP3n7k6ub/qifQvXOTNE8dj6oN3KXUR+OdwmQ
   k=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: AJXT4rlBSSGvZOOwuOBvuo4L/XXCjH0iB/CLg+0YhgkIHdyZY3o1gHnF7eu8qJCMNB/i9+9TNi
 HWUy60jzdt5mqsHpu+gAK6OloYu9Wt4aok1lFoZq6xmkU+v6ueyV/1V4UyAyCYloEpQfh3BHKj
 V9VQGid+3UGlX02V2svaAw1Wty4JfBZVXUXHQ88m6kcKWgxhBtCeWf0nZnXQeSoVneDG14yZmf
 TqRWTfYXEqjO5jt31PaKcxoxcBgDc4CzDE7pCJjEPYaRzmxyMQRcXYXo7hEdKmsLm3wHi+yGdb
 zHyNJK98xOp4tw/N/Sp/wbx3
X-SBRS: 5.1
X-MesageID: 52092572
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:krThmqrv7lTshhETEpBGsAQaV5u7L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBbhHO1OkPYs1NCZLXXbUQqTXfxfBO7ZrQEIdBeOjtK1uZ
 0QFJSWTeeAd2SS7vyKkDVQcexQueVvmZrA7Yy1rwYPPHJXguNbnmNE426gYzxLrWJ9dPwE/f
 Snl6h6TnabCA8qhpPRPAh6YwGPnayHqLvWJTo9QzI34giHij2lrJb8Dhijxx8bFxdC260r/2
 TpmxHwovzLiYD19jbsk0voq7hGktrozdVOQOSKl8guMz3pziKlfp5oVbGutC085Muv9FEput
 /RpApIBbU+11rhOkWO5Tf90Qjp1zgjr1fk1F+jmHPm5ff0QTorYvAxzb5xQ1/80Q4Nrdt82K
 VE0yayrJxMFy7Nmyz7+pzhSwxqvlDcmwtjrccjy1hkFacOYr5YqoISuGlPFo0bIS784Ic7VM
 FzEcDn4upMe1/yVQGZgoBW+q3vYp0PJGbCfqBb0fbllwS+3UoJgXfw/fZv3Uvpr/kGOt55D+
 etCNUgqFgBdL5RUUtHPpZ1fSKAMB26ffv9ChPhHb3ZLtByB5vske+93Fxn3pDhRHQ3pKFC76
 gpFmko7FIPRw==
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52092572"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CBKZzdLwZtsB+VXBUp2mxSzyT6moFjPoWyL4WCfVs0n2ntpJe1w9x+Zx8bmXfCUasUiIPXnDqd/ECk3wsYoF4t5n9o/JGxXObv6ri+b/+7DcXaZscSFyyNGNsVjN1admH4bG1n5IZHIbY83Xz2SL3pN4H6J1IYABTTAq/95NSFuqOnsjv/ZSaPWJylLQcMFN/QZPNltDywJ4TBPYxrnucVpADbCBjdUoSwPKzrjdeUhdEj9tKCagi07dLIlrryAZ89EunokQwW3EBz7EaKTDU7Rlpp73dyrc8S3nQuy9F9fOjUsqw/Mbh/IGdMmhP++ho0SZa7UaqGT5BgOV1QLWsg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=e4AQjytToAgMKCwoiUEbPXNarq36A5NlNm1hQZ3oCF4=;
 b=kaEut+8XyJVY9pgOb4iHnBOwttBa0umwAIzZ0SzgRV4SNdVU4xe+Ilec6Fc1rh/wVISRSXn4anfxrdtB0kRpwbqRD0OUZp31wyXt+DlQJXPfwV6IGXKRhwoJLAATlNJWf2dHLa/bWJmr2SiOB/ODT6s42fwnoFHXbO7ZQAVQ0Iz5ggNVU5TUD1UxfHSVXQtdhStx73WXAFE72N/DudbQZK4N0h1aUmAqe6CdVWHoSG/4uOBkUbtMFNj6U+YM6HnsiRQzIQzY1QYnZqcsSfSQZdNLRo2IQhx2mQyfVoKVsQs2ZQHUHcj0o3hDGSJ3HWriiOwSzsv3C8Hssoy6WjMpxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e4AQjytToAgMKCwoiUEbPXNarq36A5NlNm1hQZ3oCF4=;
 b=ZHGKSmrC8P2b22k35rEseaVQq0SD0aukCRAqHAxLz1LZOsdenGFnWWttGeSWUkTu6ZKPvSqnWzD17ci6q296hbAjWup3gTHCX170PQTjWDnUO5EclDXCjxKQ8YSKEpemzYTc7bB5iWwVVUMnAUqzAPXXNye3R/7Vl7sl3LVTL4E=
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-8-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 07/11] xsm: decouple xsm header inclusion selection
Message-ID: <17155620-d091-3205-6af0-65e19dfa1bfb@citrix.com>
Date: Mon, 6 Sep 2021 19:47:43 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-8-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0387.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18f::14) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 63831b78-91ee-4d2c-4469-08d97166d309
X-MS-TrafficTypeDiagnostic: BYAPR03MB4486:
X-Microsoft-Antispam-PRVS: <BYAPR03MB448689936667101A67D816D0BAD29@BYAPR03MB4486.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: f9UTy/Dwkc1ycStsa0hSSBSjyk6nDxfVvcaJj2+Ib/xKnvnm9mIOTWRiOmFVjWAj7iFqOKbHEhzDfWVaqOP5EDdEo1QDmSglp3g5T/89TEFf59iGWo7btKfYjIluc49lQbXHgLPnd6+k2MC9HiT4FhgOgeYV7vozUDmYdVJUR19BntOPkyel2RxiC+S0jt6ImspW/nQI1eRpWzpgK8sFxYQao6ngf1ysLZFFV6ff0HUtHAhCtGqOYAIQMXaTH5/oNJsQ5vZD6CwjQMkxJRv4gE+DyulIo+3Ghrm+BB6FBYtaK/QMNTvKKUVyUBR4D9WX7x4zYj4TUUF93RODMcr1+F/LA2ykj0i4KQBzRrknML0o9h79eVu73ulnIqoxfZy7BxTGdPiMcIR/LQeidomd7FS0A/Dft5Tuzqu3O3Zu1k6MvurqQ+P8ReO9aZQ+1iihrqHkmvnlrIuug8JPRHUnEEUQBFrKOKrsSN7xTlBuu5kjezHE3o9mUTc6bkyHWBhfed/yirYwMblX1xynBnqZyc+fVID3W6G9Wxh7uTl4mHNzsEBG7veii9Ta0u07P6HF+CO87kmreW4Z0wnmdHGvxwrTnJp0uEPwks2bmDVaEldpmrFVl22HHfttgnuqGXmjDcm3ukecAuHgdKGo3AFGABbGXj1WgTiXyRxRTKLCqgEcakyJB3qyF2ixv5RY0hqpAROo/QUjGrFL1U8CrdFqEuNWJEbPQdLO2+c1uudLyUWZ44lZ6evLE07LI3hqZzhX6EUuE5ChZMRgQ2K5BqZAhw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(366004)(346002)(39860400002)(396003)(36756003)(31696002)(66556008)(8676002)(66476007)(86362001)(2906002)(38100700002)(2616005)(956004)(6486002)(26005)(55236004)(478600001)(31686004)(4326008)(6666004)(53546011)(83380400001)(8936002)(316002)(16576012)(5660300002)(66946007)(186003)(2004002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TXBYYlBZR21mY0ZIMk11Z04vaUk0ZVNZbHd2d3BIUEF6WWxBVXVUNDk4SzJO?=
 =?utf-8?B?V2xtQU03QlFaMnY5dnZIWUxlZFRGZHJWVnJ1WGk1UWVDbFRKRENSKzJmeUF2?=
 =?utf-8?B?Z25ueWRKZjRnWnl4Y1VTYUJ3eHVBZmRqbUpBUmhvYTU3ZmJtT1cyL1R6RnhB?=
 =?utf-8?B?UDdwbWZtdys2eWxXNjdsaTF6cVlZYVUxU2tja3pJM1BMZWlycGxHc0FCSGFX?=
 =?utf-8?B?NU9CanZGMmgrcitkdUovVS8rdE1QVUo3Y3VOekZ6TkhuM1FWZ09uNERpVmw4?=
 =?utf-8?B?UDgzS2FFRzRtcUxQekdpYStZVW5GZk02VDVtcWVMY28yL05BL1JiVTNaMFJw?=
 =?utf-8?B?UDljT3RCZ1FMZXNhOWxNSk0xT0ROd2ZHT3RVWElyUU1UOGhaeldVd1NCblRt?=
 =?utf-8?B?WElEdm5qeEg3bUlDOW0za3BnSEoxRGNiSnU4YzBMb0JVRGhKaFV0U1JXL3Ix?=
 =?utf-8?B?K3pLMkpSMkYzMmE3TmVYY3ZaVnFIL0p1SU0xZXVkV05rUGtJODFyUlVMSU16?=
 =?utf-8?B?NU5ZYWpQaXcxcWZEK0EvVXJtZ1JrWkQydnhQbnZyVW0zR3lOZFRYSjUvNmpN?=
 =?utf-8?B?RFJIQ0hkVDc4a2xyYSt2cGx1OVdsUzNXU3cya1B1ODNaMDZIcldhVlJZczVB?=
 =?utf-8?B?Q043WHgvMjhkZ3AyYUZKSHFmNWRMSjBDaXhtMmJXWHpIV3diOTFWaysrV2xa?=
 =?utf-8?B?K2ozNGYzUXE4THkyQkNPaC9IRjhQdVRoTXNhZWp4aERrYm4yeEVudGxMUUM1?=
 =?utf-8?B?em5ERWV3OHlDK0NTQmk1SEJTRmpQdTN4bVI4aHBKaXhxS3FvQ3doWlBtVGYr?=
 =?utf-8?B?UFR2a1BXdEdJb3BjZWQxcmdxTHNsSWlneDYvMFpsdndPMkd4N0dyL2VpcUlU?=
 =?utf-8?B?cG9VWWVubkh4ak9Md21QaHRWeWpLQTgvNjRJdjlUOUF1Nm5HdUoxc1lRRFhU?=
 =?utf-8?B?eHpxNEtGeUE2Ui94TWxTbTNRNGRDdWY0N0FReXFJbDBZNlVNV0MzcElYa2dK?=
 =?utf-8?B?TFlYeFR3Z3I2UFhpcGpYZWkzamttQUdvWXZYUXYyVUVuUDY4M0VkRGxLWVdW?=
 =?utf-8?B?WmhOS054ckRMSytxR3VWOWNvaGhicHp3eFZRYXplRzFZY2thSE5OVFgreEh3?=
 =?utf-8?B?M2FDUDRybHByRks2aFM4emhleS81UnV5Vk8rZi9sZlZVWHl3UXRhb0NidDdK?=
 =?utf-8?B?Z0dSbjdTeS9iSXBYZjlQTEJxTWZTdVZQUEw4Zkp0dHllQ25JUi9RYkgyTWVy?=
 =?utf-8?B?SU8zM21Ia3RQeitHSGFqLys1WjFPQVlxN0dpdzd3VGZ2UEpkdW5pWDlSZ25I?=
 =?utf-8?B?UlhxSTJ6RnFyUXRIaGZDcEQwQkU3N3RpWDhqV0YyYTZuRGZ6eU9kaFBCVUVq?=
 =?utf-8?B?NUVGWlB0S3JMTVU5cTZUOGlwTE5DWXk0S2diMjBjL3FqcHRnUmdMSmZFVEFD?=
 =?utf-8?B?VGFLZjgxZDNWWVNYKzU3RkhXOTZheG1uL01lWjRGRUlKL2pCa3RIcVZzb3pu?=
 =?utf-8?B?YXJId0hWaUhueHo4cld0UVJ0Y2J0UTlyQitKZjF3NCt1clZkQ2JvK0YzOVBI?=
 =?utf-8?B?c28zVk5TVUVNYnNBd1Q3SWpKN3lZczJJb01tNlc3VDNscXJzQm5WT1ZZb3Fv?=
 =?utf-8?B?RVhNcW5HbVpTRHVhb0VWSDQvSExnelhJdCszWE96cnE5RjVoK29hTnBlL2kx?=
 =?utf-8?B?ZTM0T2hJSDl6R0p3NVdIU3hSK25LcG5JOUo4QmJqcVBhZndQT2lPbzFIaUF2?=
 =?utf-8?Q?AxC7FWQIjPZW5IhY7ZZI+ghemaCPZ9YNxy6fCEp?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 63831b78-91ee-4d2c-4469-08d97166d309
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:47:49.2757
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6Ee5h9aRE/tlvyyacY8LXzVAV3io5aOiAho/akPkCD4X9bQo2veBzcMdQd2zHb1eZG7s6KAjFI/pHgZq3kM5EIQqhs/HlP6rWrtMTSrkqWU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4486
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
> new file mode 100644
> index 0000000000..4555e111dc
> --- /dev/null
> +++ b/xen/include/xsm/xsm-core.h
> @@ -0,0 +1,274 @@
> +/*
> + *  This file contains the XSM hook definitions for Xen.
> + *
> + *  This work is based on the LSM implementation in Linux 2.6.13.4.
> + *
> + *  Author:  George Coker, <gscoker@alpha.ncsc.mil>
> + *
> + *  Contributors: Michael LeMay, <mdlemay@epoch.ncsc.mil>
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License version 2,
> + *  as published by the Free Software Foundation.
> + */
> +
> +#ifndef __XSM_CORE_H__
> +#define __XSM_CORE_H__
> +
> +#include <xen/sched.h>
> +#include <xen/multiboot.h>
> +
> +/* policy magic number (defined by XSM_MAGIC) */
> +typedef uint32_t xsm_magic_t;
> +
> +#ifdef CONFIG_XSM_FLASK
> +#define XSM_MAGIC 0xf97cff8c
> +#else
> +#define XSM_MAGIC 0x0
> +#endif

Eww.=C2=A0 I know you're only moving code, but this construct is broken
(right from XSM's introduction in c/s d046f361dc937), and creates a
fairly-severe bug.

It causes xsm_multiboot_policy_init() to malfunction and accept a module
which starts with 4 zeroes, rather than the flask magic number.=C2=A0 The o=
ne
caller is suitably guarded so this is only a latent bug right now, but I
don't think we could credibly security support the code without this
being fixed.=C2=A0 (Again - fine to add to the todo list.=C2=A0 I know ther=
e's
loads to do)

> +
> +/* These annotations are used by callers and in dummy.h to document the
> + * default actions of XSM hooks. They should be compiled out otherwise.
> + */

For the coding style patch, this should be

/*
=C2=A0* These ...

> +#ifdef CONFIG_XSM
> +
> +#ifdef CONFIG_MULTIBOOT
> +int xsm_multiboot_init(unsigned long *module_map,
> +                       const multiboot_info_t *mbi);
> +int xsm_multiboot_policy_init(unsigned long *module_map,
> +                              const multiboot_info_t *mbi,
> +                              void **policy_buffer,
> +                              size_t *policy_size);
> +#endif
> +
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +/*
> + * Initialize XSM
> + *
> + * On success, return 1 if using SILO mode else 0.
> + */
> +int xsm_dt_init(void);
> +int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
> +bool has_xsm_magic(paddr_t);
> +#endif
> +
> +#ifdef CONFIG_XSM_FLASK
> +const struct xsm_ops *flask_init(const void *policy_buffer,
> +                                 size_t policy_size);
> +#else
> +static inline const struct xsm_ops *flask_init(const void *policy_buffer=
,
> +                                               size_t policy_size)
> +{
> +    return NULL;
> +}
> +#endif
> +
> +#ifdef CONFIG_XSM_SILO
> +const struct xsm_ops *silo_init(void);
> +#else
> +static const inline struct xsm_ops *silo_init(void)
> +{
> +    return NULL;
> +}
> +#endif
> +
> +#else /* CONFIG_XSM */
> +
> +#ifdef CONFIG_MULTIBOOT
> +static inline int xsm_multiboot_init(unsigned long *module_map,
> +                                     const multiboot_info_t *mbi)
> +{
> +    return 0;
> +}
> +#endif
> +
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +static inline int xsm_dt_init(void)
> +{
> +    return 0;
> +}
> +
> +static inline bool has_xsm_magic(paddr_t start)
> +{
> +    return false;
> +}
> +#endif /* CONFIG_HAS_DEVICE_TREE */

Shouldn't this be an #ifndef CONFIG_HAS_DEVICE_TREE ?

And the answer is no because of the #else /* CONFIG_XSM */ higher up,
but it is incredibly deceptive to read.


I think this logic would be far easier to follow as:

#if IS_ENABLED(CONFIG_XSM) && IS_ENABLED(CONFIG_MULTIBOOT)
...
#else
...
#endif

etc.

rather than having two separate #ifdef CONFIG_MULTIBOOT blocks doing
opposite things due to the position of intermixed #ifdef CONFIG_XSM.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 18:55:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 18:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180204.326758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJm5-0006A4-3p; Mon, 06 Sep 2021 18:55:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180204.326758; Mon, 06 Sep 2021 18:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNJm5-00069x-0T; Mon, 06 Sep 2021 18:55:21 +0000
Received: by outflank-mailman (input) for mailman id 180204;
 Mon, 06 Sep 2021 18:55:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNJm3-00069r-HC
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 18:55:19 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fa1f1ec8-0f43-11ec-b0b5-12813bfff9fa;
 Mon, 06 Sep 2021 18:55:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa1f1ec8-0f43-11ec-b0b5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630954517;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=78kqkbp/8KkJXYvxGOcNoRYv+d1QDAIOBRdg4h03mHw=;
  b=eqdXULvlQB3cHWYKjKYra+flJsOVPRhWc0x3VzmighZJd7JtMzofjUqQ
   8nX4dl93fO3GtAUMZWIe9IajfJpyKvuO1P70QIdn0WtpPquaKI5mFQlrc
   nucfuQA06dAF8P4qX6MAhos3832m+gKJqd6YRC2ScP1+9bXgQiA7bDAUS
   Q=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: o6T8GI+uOEzF3SSOpElMJ/udZGNIFI8HyePgw/f/YFZSp2oJpJpjlynhgU6txRtrYymqVHrqzs
 214jrw+V9CyTzY/9FfskEQEqwtXv/pUZV6uOLO2685M3Jm3wfF7qJy1rwU4DDvORR2LBCrBNQT
 NTblttk4KAwgMcqGMC4HZJKXE1evl+YjzlUevggSPOSZMx7cERQ/ZIqbhYUqasuZinnzDILhEc
 TKDfscrenrBt5DsVhBS92a7pWwom1MaB6g8qv0nLrMdNqah8p8YM1InxxEqHKB1peFtxnaDC/s
 mZ6UBolSxBL4l42CS9deU+Kn
X-SBRS: 5.1
X-MesageID: 52507810
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:WBgp6KuWThjwJeXCEp1b6VOF7skC5oMji2hC6mlwRA09TyXGra
 +TdaUguSMc1gx9ZJhBo7G90KnpewK4yXcH2/hvAV7EZnibhILIFvAe0WKG+VPd8kLFh5ZgPM
 tbAs9D4ZjLfCJHZKXBkXmF+rQbsaC6GcmT7I+0pRcdLnAYV0gj1XYcNu/yKDwGeOAsP+teKH
 Pz3Lskm9PtQwVtUiztbUN1IdQr6ue72a7OUFojPVoK+QOOhTSn5PrTFAWZ5A4XV3dqza05+W
 bIvgTl7uH72svLhSP05iv21dB7idHhwtxMCIiljdUUECzljkKNaJ56U7OPkTgpqKWE6Uoskv
 PLvxA8Vv4Dp0/5TyWQm1/AygPg2DEh5zvLzkKZu2LqpYjDSDczG6N69MpkWyqcz3BlkMB30a
 pN0W7cnYFQFwn8kCP04MWNfw12l2KvyEBS0dI7vjh6a88zebVRpYsQ8Ad+C5EbBh/374ghDa
 1HENzc3vBLalmXBkqp/1WH+ObcHEjbIy32B3Tr4qeuonxrdTFCvgUlLfUk7zQ9HMlXcegC2w
 zGWp4Y3Y2mAPVmK56VP91xNPdfPFa9Ny4kAFjiU2gPK5t3T04li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52507810"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N01OWPYe/+p9tuJjLX6zqVHoR4i7uIFst9lr62Lcse3ehhx6gblYvA8TofqTZr5Fc9Yy35VQbcrSEiHHUFQmzvq8AEyzyxUvOkcLWM2Oi8lUXMtuM9KJ6UlAT9QqYSlajJ521X2wp1efiZG2QNpgBTuvLRHAjzv4sPR3QSqq7zyPgo65/gnDZjOHiMGtqXO+QQHbx+fFh2lwuEjU1ArymIANV4X5LUDg3WokBw87gKPaDg6IBiVpXAlvI5P0/HGyJ19ONlviIAQXZfQXsth7A14rosVJMqdjE81zfG5l9dVjbEE7H8ls/8t+jCkajLItzyzrw8Nmqxl/p5h8A73RtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7yWjg6RICr/TwDqra08velTyo7XtSAg1czseNajXQ4w=;
 b=HtY0uIqk0aBYBSqEF2MiXQ85JB+O2cogloBDo4z3j7VcEYe0fneV9T1ZjOrV/kSmyBnlF+n2kI/1B+rbNXt7Q5SwdOmimEsRzn5WfNhNMRnW/cAhQ/KsDBbjIMoQpnWtkv5LjDFo9CzteVvmNXV5uuyoiABIumTra/oCZCR7Pi5m7x5AMkObv7Lqkkb2fnkvbvEYFCVdk/7zeazEj/KtJ9YsOlf7dIXS81zYB2IHYFpSYmeDf2rJ/m/q0Q6ckEusbx20eVtFnIGmQC7TeJpGQTfgoXHUN8W9A7LiCVvBW2SWd2i1dtFa79WSqt/b1MfBjdw/cazhUBYOtKV3VXmhxw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7yWjg6RICr/TwDqra08velTyo7XtSAg1czseNajXQ4w=;
 b=R0NXAynlhfI1WS90VlPjd6uvi04adwt6fnXcaTpb3dWrJqE3Sey73H/9G/lT+ST8loMJeasuC3FEGFQ4462nvRZ/jAf8LFFrm7lhMwztbKIrtuv3FXguxSHwsif52YfYq7ySCNWywColj2MKJg6uSEH7VhMR585cDylEGgiSOKM=
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-10-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 09/11] silo: remove circular xsm hook call
Message-ID: <d415ab82-ca83-98e8-0b07-0e093a1fa7e1@citrix.com>
Date: Mon, 6 Sep 2021 19:55:07 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-10-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0168.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18a::11) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f27be822-55ae-4309-3ec5-08d97167db7a
X-MS-TrafficTypeDiagnostic: BYAPR03MB3941:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3941DFE8779FC82341110242BAD29@BYAPR03MB3941.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:595;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +zY+YJchn0vKL1shRH3kH5Yat0y9+YgaMqGSumkFkY/F36j4lvyqY5K6n89ub0cUIUhVel8LXe3PxaVOdj/aCMTdzwaYZD7jBKeO8FQ592RSUD4f3Qf66jsI12kqwJINecNRW3VfFFpJ9NxVMUDYNSMil5KU5M5OdPa+SxowpGMacKX1oHQL0kRZaIWOSGFeR//AAtfRgVCQNcLkyqqirMdPCizQk54Fxb6nTqFvHyxHefJovTKjtTgHR/qIr9KPMhHCrKKH/jAHNTGWmbURuT9CDNpagApjaWY+yKp5hdFJpSrS7sCInSzj6Gwy68OynRwcNMWlwAwkx8kzBV7tAsgkiBabxsz0mg5vNwi58knZefJ57NW+ThveWNHI4L0c6Ya1N/dFdv4cO9FSX5yzxu+b+Vi2l1Qt0Htpf2GRFiYF3eqqTEjSK3C3eKqWYBPn7+UzZcX1biNAqeD88ZCqCiDXy68TP7HKaQSmo7gjsVmI1AGknqL/TLju96GLCLLnS7wl/DoatqBlfoCo1y6c5QcnGduHmOfn4TzWRo+je8gg7NXIe1cfmTQ2GLyiVEPlcTHrtdj29uJWRm9933YcMXnPhy8w7EqID3zBe4b+mDaiqwR+hAzcpyddfSou0pw/51WnAM+IyWZxmTD8zUidq43CnxZxoNbWKl6ib6S8vK0eChwhr8UAzKRLt7JsTj73+MLiZEJCCo6NdXQ1Svo97/xYO1hyzfWSgDnvamQyooAVQfF3JgyhNv6XB9GrdfRlHa2sXnyMu6r01YU8zZEqvzPHwzpINq6sbyms7ahMoNH8u7/CTae9manONHYf74qc
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(39860400002)(376002)(346002)(396003)(136003)(53546011)(2616005)(5660300002)(55236004)(36756003)(8936002)(86362001)(26005)(478600001)(956004)(66946007)(66476007)(66556008)(83380400001)(31686004)(186003)(38100700002)(6666004)(2906002)(16576012)(8676002)(4326008)(31696002)(316002)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VlNWeHNIY0V2UDdXYjEvMVNzN0pTT3dOcHEzd3lPTmpHWGpuL09mTENWa0ZG?=
 =?utf-8?B?M2tCejE0ODVpYXVwLzV3ZmJuTytXSE1YMU9xaERzU2g4VDdORGdjWm8rVTZV?=
 =?utf-8?B?c28wV2ZHMDJTN1lpM0pFN3FBNzFiajNXRTVQY2h3ZUpERlM4dGNUbHlGMFZs?=
 =?utf-8?B?bGJmSkZLejd2KzZXTUtCTXU3cXJMWlRkK1JKUWFxNzRTblp1eFdYN1A0K2cw?=
 =?utf-8?B?VThIbCtSL3VlMXFFdnVlNVc3S3FXYXdxYTJub1Z2TU80UFh2OFJDRmc5Z0Mz?=
 =?utf-8?B?dVVjRW5qUWh5bHZJS1pLVUQxZlM5WjVyNndJb2Vrd1JvWFE2TDIyZmc3NHZK?=
 =?utf-8?B?WTZlU2dudDNQbDNxMEVkT1g3NkcrMGpremRHMEdOVHg0MlB1VVBvNm9jdG5S?=
 =?utf-8?B?YkEyK1d3NVB2NjF4SG5JczZaTTFIREs1MDhMTGF6dFdGZUNtbVRsZkk4Q2JV?=
 =?utf-8?B?Vk1IQVk0V04vb3R3cjd6VUlkQ0xvNGc1UDZsN0k2dEZibERHcHVGZHZNeTA2?=
 =?utf-8?B?K1h5Nm4rdE90OEdmR1ppNWpDMlp6VWt2MGlUMXhlblE4MThpazRNYmd2OVls?=
 =?utf-8?B?emtWRFliQTRXMERkZlg0ZDhrSlB4MGtwbjBOME9vWWZrRUhkUHkyZTRMTjlQ?=
 =?utf-8?B?TjVoSlhuNGZqbGNVdUVXL2lESUdubkJFcE9jbUp5L000d0NNUm0xd3dpUFBS?=
 =?utf-8?B?b0pXcDRIQ0wvRE5Gdmp3b3I4QzZEZUlnSDZDc2JDYmhXQzAxTjlIeW03ejBw?=
 =?utf-8?B?c3FSVkVraVlxSjVJRDFadTgwcU01Tld6bFdFWXNUYy84S2svbUJTbmdKNGhP?=
 =?utf-8?B?QUxMaDdQQ2kvV1NPditHT0h0Z1pxWWM5OVdPbGQ0OFp5c2M3b3R0bzdWZlRU?=
 =?utf-8?B?dFVtWGl4aEZobEVqbjZwQWtrSHZyN1NvRGJlWHM1YlJwc2kyaHcwYkpXTDhi?=
 =?utf-8?B?Q1ZYMEZLSk5LblZLeWZUcnc4aXJtbEpjbFd4NmMvcmZFcFBieUpCc29PQU94?=
 =?utf-8?B?N3ViV2EvMEtUbUxkTmZhYkJrYVpoMXBzdkptMmpTRzJvdWZIaHA1ckc1K1Va?=
 =?utf-8?B?ZTdxckZaWjhCalBxL1huSEg0R3p6ZVRHWmN3ZFZiWHJVRUhvOElQeDAwdTVS?=
 =?utf-8?B?amtZZGdmYWdvWWtNMitFZm03bTkrSWtFR242dUhBV1VVMit6eDc5dUhSdFk2?=
 =?utf-8?B?V3orK1BsMmUzREMvTmpwcC9NbjNjemhiRnd0Qmt2TVJTSDVkdnFTRmRYekZI?=
 =?utf-8?B?WklUUWF2TGpKUC84NWVLeUhya0Z6QnV4QWFiL3p3TUNnNnZncWY5UVVsdnBQ?=
 =?utf-8?B?QmFYYzFOMk12SEtwdEhPOHZJenNBVlh5czl5MnIxZzlJL2FMeE1VL25ySjVq?=
 =?utf-8?B?eWZXbEc2azMraXcrUXViZ29pSSszQzF1MUR5MHpUS0FKRVdPT1RKOWZmTVRj?=
 =?utf-8?B?UFJOcXhwaVJ4WGM0ZS9zZzUzbFF6K1hZNHJYWEpTUkgzVTVVRlZmaEF6eW1S?=
 =?utf-8?B?ZUVyMVowSlRic25SUUZUekdEazNNT3MyUW9BcFVBTncxL2VRc3JtemN3ZXZY?=
 =?utf-8?B?OXVxUkR6bnZ1eldzNDlJdlVGK1l3Q2RRbkRtVTRKMC9LYnd5OXd4WTdFS3Zh?=
 =?utf-8?B?ZDZ5L1h4eTZpT2RtNTBDTDlQa05KZnVrS0JsYlMwSjcrdzFvVGI0SDRsVHRv?=
 =?utf-8?B?b2F0ck01dEVCNUpKRzdndnY0K3lIdzJnNzkvYTQxWHJpaUMrSEtxK1RXeUQ1?=
 =?utf-8?Q?8d4wefKVd6ykQd4dB3Om7jWp76lQQM0OFEvFdfk?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f27be822-55ae-4309-3ec5-08d97167db7a
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 18:55:12.9216
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3DcxsiQpdNP1ShTxmc1+3JAbpiL0CWoAgBeHyRSoc1JiAS4xbeGPgshMNT8VbjCWtWZNg5hg7JskkwBXn37vkhzuxBA6djmeOoBXnTN87H4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3941
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> SILO implements a few XSM hooks to extended the decision logic beyond
> what is defined in the dummy/default policy. For each of the hooks, it
> falls back to the dummy/default policy. The fall back is done a slight
> round-about way.

"done in a slightly" ?

>  This commit makes the direct call to the default policy's
> logic, xsm_default_action().
>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> ---
>  xen/xsm/silo.c | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
> index 6db793f35c..56a330a831 100644
> --- a/xen/xsm/silo.c
> +++ b/xen/xsm/silo.c
> @@ -17,6 +17,7 @@
>   * You should have received a copy of the GNU General Public License alo=
ng with
>   * this program; If not, see <http://www.gnu.org/licenses/>.
>   */
> +#include <xsm/xsm-core.h>
>  #include <xsm/dummy.h>
> =20
>  /*
> @@ -43,7 +44,7 @@ static int silo_evtchn_unbound(struct domain *d1, struc=
t evtchn *chn,
>      else
>      {
>          if ( silo_mode_dom_check(d1, d2) )
> -            rc =3D xsm_evtchn_unbound(d1, chn, id2);
> +            rc =3D xsm_default_action(XSM_TARGET, current->domain, d1);
>          rcu_unlock_domain(d2);
>      }
> =20
> @@ -54,7 +55,7 @@ static int silo_evtchn_interdomain(struct domain *d1, s=
truct evtchn *chan1,
>                                     struct domain *d2, struct evtchn *cha=
n2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_evtchn_interdomain(d1, chan1, d2, chan2);
> +        return xsm_default_action(XSM_HOOK, d1, d2);
>      return -EPERM;
>  }
> =20
> @@ -62,21 +63,21 @@ static int silo_grant_mapref(struct domain *d1, struc=
t domain *d2,
>                               uint32_t flags)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_grant_mapref(d1, d2, flags);
> +        return xsm_default_action(XSM_HOOK, d1, d2);
>      return -EPERM;
>  }
> =20
>  static int silo_grant_transfer(struct domain *d1, struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_grant_transfer(d1, d2);
> +        return xsm_default_action(XSM_HOOK, d1, d2);
>      return -EPERM;
>  }
> =20
>  static int silo_grant_copy(struct domain *d1, struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_grant_copy(d1, d2);
> +        return xsm_default_action(XSM_HOOK, d1, d2);
>      return -EPERM;
>  }
> =20
> @@ -86,14 +87,14 @@ static int silo_argo_register_single_source(const str=
uct domain *d1,
>                                              const struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_argo_register_single_source(d1, d2);
> +        return 0;
>      return -EPERM;
>  }
> =20
>  static int silo_argo_send(const struct domain *d1, const struct domain *=
d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_argo_send(d1, d2);
> +        return 0;

Shouldn't these be XSM_HOOK too?=C2=A0 Or should all other XSM_HOOK's be
short-circuted to 0?

The asymmetry here seems weird.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 19:18:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 19:18:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180210.326769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNK8M-0000ZC-Vz; Mon, 06 Sep 2021 19:18:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180210.326769; Mon, 06 Sep 2021 19:18:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNK8M-0000Z5-Se; Mon, 06 Sep 2021 19:18:22 +0000
Received: by outflank-mailman (input) for mailman id 180210;
 Mon, 06 Sep 2021 19:18:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNK8L-0000Yz-BF
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 19:18:21 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id aa6e8fd0-f2e3-41cc-83af-cf6374ad21fb;
 Mon, 06 Sep 2021 19:18:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa6e8fd0-f2e3-41cc-83af-cf6374ad21fb
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630955899;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=QtUSp3pvvZZv808XCaHgs/kPOkAFV28WuV+5wXilHK0=;
  b=A5FX0Naz0LBvGYH+HwWbyzi3T+aVfkE1FiQN5k+X8ss28e4w3Etp7Rud
   FqxwSgh/5XuUhbxfwEVehxNbj/nolurW8wz1e+f7MIHcE1NmKuo5Spn9A
   SywQplyfvyhSQoV4uQi982h/nbOrZNyJjktE+kaLjMDRkaFOF+/Ne3fzQ
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 6B/Tzs0QA46+CNAFqe01cQ9nVn+/Aqt6rO72D7+txkl0ZpJ94XpmeO1Ff8SGxJ2jmDgmhez3AX
 RZY0EVxnC3G6dHZ56ffxWBATfgDJkqOuZL1DOZwA7yOWHBx1kVMKapM6bMjfVV3uuCHrX73Rqh
 4uSu9m0ljuK8n1dv2gs0ENCwQ8CMZfj/hBPgPWLa+fyJomvrHW5on32MwEywbSmjK8NcJKSERD
 TGwkfUYuqtCji1rAJ0XXRiYtnZqija+1RtPqLADRDDHSq6qAaaAuA9t/lPuhygjy8RNssmp4OX
 8JzKtLu3LWwQq9LZDn37bw8I
X-SBRS: 5.1
X-MesageID: 52508607
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:XV9dIqHKGLcyP+ESpLqFeJHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HkBEDyewKiyXcT2/hsAV7CZniahILMFu9fBOTZskXd8kHFh4lgPO
 JbAtJD4b7LfChHZKTBkXCF+r8bqbHtmsDY5pas854ud3APV0gJ1XYJNu/xKDwReOApP+taKH
 PR3Ls9m9L2Ek5nEPhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2mdfVmGJ6VMt1xN/dfOla9My4kD1jiVWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52508607"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dTSxZ79Nmu07vS74qNVRDIamhRusfk2uBEp3y7QnxeCqKXozhxHMDTDxAu708Y7ys4+LTEcEfE7obviTOAoSoo9FGmf8+0E3/WxIHKdHgg/bcBfp9GgF5V6rWzNeTkwJtQGwYD9pXMwsnwS0XLOkkrlBIpFHA/rRILcfv5MvBomBZ6sX4vaTYdKkmFOEud/RCzPhtpAD8qS44a1L93pCTKXWTtTrKPPdrzz/t0h1lUOFD+lGp5gCjiR1SzSjLqkmBTIBEDLIWrQbJRcoCbfVMbBdTNQT9sJi/ZB7XRIKiTHHZoLUgdhCIYcc7LDwJFFgvQLUyMNeiKRYd3JB2UDblg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gvTsw0FNSL5+s3kxRMn/xBfkLyPZHLX0DVmJG321wyA=;
 b=h+3P0OgDMcsGTxgrTvp/Voz8nDws28mFvKLPVHWnO7p7vhvzSiBHDLESIMSvkh23N1KC11txf1gHgJw/vXAYOxnBbs/LgouUXi7kNhakVOiuKJ7Jn93OEsABa4CZJfUE5sy0nyUKPbszs4ItUvblH+mULCJt/msdV/Yk1/81vCIXyQSMgvlvAAeXHv3sx+t81ZykqW2X6SBI314gg0DlBTmGz/x5BBCS43UNcAuIIaU+g6+ENFqjy6+WMpaWNsLDdeYT7YxUH8J0qnNcBf0TnnEc1z5KbOL/RWimqxJXa6RXLFf3N1l0D5MPzw919rfPsqDeGXnXALUD/GHLb6jk4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gvTsw0FNSL5+s3kxRMn/xBfkLyPZHLX0DVmJG321wyA=;
 b=jZK1tR2Gr+pHvu4O4CTwSEzAnAAEE1EDGdwA+AD6/JdmDCYPy6+BeUe6Dps1JYnrWbDX0s7P+5YzfTfxEI2SYi2hUMqve0EEPlUMFPWeDae9rbkph1oxfmVTxz+sld2O8LdM8kB69RGS4H8UNAVrgMan8Rj6PQ25q8s+GltRVnE=
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	<xen-devel@lists.xenproject.org>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-12-dpsmith@apertussolutions.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 11/11] xsm: remove alternate xsm hook interface
Message-ID: <b35d8b3a-c398-a101-8893-6a3ca6906e94@citrix.com>
Date: Mon, 6 Sep 2021 20:18:03 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-12-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0064.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:60::28) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fec7ad68-569d-44aa-57fe-08d9716b0fcf
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5597:
X-Microsoft-Antispam-PRVS: <SJ0PR03MB559794347B6707B055624D3EBAD29@SJ0PR03MB5597.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: My09vHhQTv9vimrF9fR6V7n2ANeaRjupRzQa98VxXNOBOUia8rROr7fP9LqIlCn/gxJ4XyjQiTGOh3yp6DqmkdFA/hHw+3q7zD9P8D9zMJff0jWx6prlTMPJ2ANijDRB6MUUTtH3cg8syWHHdxJ6vtRO9pP5ZBuX5YaAztOjVdjtmhS2HwiGMqO7LDITZuKhyibXEQyOZNA0aUsSeTpgW+Vc5ar66sBGFRP5QE3oc3sfCehZZNLTCO0huO11+sBqDqIyTEIqpt+V45Yay5sKja+SwBfHiwPAeuKDq1FmgK4hyejI1WT0tRFZDfvc7ffjrB22tRQ1o9tsK4DXpm8G+k1zLlmRnElEJOTI+F4ycBDMs0Z3R9kygP8EWLeAnOHgAa/oI6RxcCfQ3qLnWb95pg9hsgSUqvXEGpzdU/eCqUqBihDK/jPC97DyDsV+CbPKHoDK2S3buxLwTnqceL/m88EBqYVsZ3iMX03cvv5ZH5Vl2RExU9WE4fnGHIP5GzxDcgsfUSLwYBcPuua1C61tnkODWthn/9SyQgPHgfipzPe4L/c0AZnDYkSYZufwqm5h5EOUquhJouamOQc2/Brgx7UCIopAhqIEQxreOCWAi6G5VK3+NZBOKn+ACpp1UgX8o3nWlgm72SIv49B4DBwOxmY8sE1TnJfqoBxHpnfWPSCNmqwwwBIcyAQy/TE45Ee3ItTbPJcfy9+OvbobTkjOOyV3zaQLSuWEdNLR1qtPTSCZQ5uzRblVdYCovZ79Vgw4Aanuw7MTJCg+EYkS6FbYgQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(366004)(396003)(39860400002)(376002)(31696002)(2906002)(4326008)(956004)(16576012)(2616005)(55236004)(36756003)(316002)(186003)(26005)(6666004)(31686004)(5660300002)(38100700002)(53546011)(66946007)(66476007)(66556008)(478600001)(86362001)(6486002)(83380400001)(8936002)(8676002)(2004002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TkNBWmhHOURZUkpuWEU2Y3NpNVljWDIvdURWa0ZEWXRwSHJxYnNCQlgydWVa?=
 =?utf-8?B?WTI4eml4UE1WcHNxYVpDK0tpQ0xwTGs4aG9kR2E2U2FxMk1EQ0lGNUt4ZWxk?=
 =?utf-8?B?Zy96aklBOGZPcHY3dm4yK3dmUEhROEZla0l4YWROeUFIVzY4ZENaUTlZMkRE?=
 =?utf-8?B?Vi9kSURNaFZobVNFVXFUbGRLRDFNdDVJWjNHSnZGYy8xNHhGSFUyaVZRa05x?=
 =?utf-8?B?RTJ2Tm5FcWdoSnpuU0doR3B4Uk5CQ2NldmdtUmYvVmNJYmR2MkNMb2lXdDhy?=
 =?utf-8?B?WFV6OElxc1hVNHF4b0xVNDEyTldvbnZQWU5OVktGOGlqQ3BRWFJCVkJwZFV2?=
 =?utf-8?B?cS9CR2MrN0g1RExPaFlGWk1FeWkvMnpXbUdsSzRuMkVXWEpFdmNiZjR5ck5p?=
 =?utf-8?B?ZzVpYlVOZW02dzYzU21SVVhsZEJlWGhhMVdjUk0xY1VIM0xuNmZHNHdJTE9i?=
 =?utf-8?B?bnViNXNHTGx1Y0NTZWljUFczSUtBZzhjK2tRbDYvV0x6djhaaXNyeitwdHdR?=
 =?utf-8?B?SGxpZjVIdG1ZbU5mZWVoUUFpNDFvK3U5Q1VGanBidHhBS1lieFhlbVNKTy91?=
 =?utf-8?B?b21BYVQ0OFd5T29aaUhacmZEaFhMN0t6RUZja05YWFNMdlpHNlRpZ3JwZmh6?=
 =?utf-8?B?U3NCZ01PR3p1MkgzeXVnbE9sK2trYVhMaTV5MW9DTU9paDRuRC9mbUtFaEJu?=
 =?utf-8?B?Wm5zUDJWdHBlVlZXa2E2RzlIcXp0eUlPNVRVc2I5Ti9uQUp6REM3RE1VMDRY?=
 =?utf-8?B?YXozWHhTVFZZeERSM1ZyREpwQXlHWUVpbXpJVVdVNitTNlJScXBMMGFKaWcv?=
 =?utf-8?B?elFOVGkxRW1xNlplMndHUXozdFl0cEQwcFNLMjBCTEp2K1RtSEcxVFlTVi9S?=
 =?utf-8?B?MzJBbEVvYXA1clUrZDhVSStXb0Rwbk91c3dlWm5zQUhWZnlaSklFcmNrbm1R?=
 =?utf-8?B?RVBvdUlzczA5NTdlY1dpc3pmSEl4UXhqT05vU2FOSUtxSXloYlRybUx1VUxR?=
 =?utf-8?B?SVFqVUhHQkcxS2NvQ1ZGbE9Vams5Q0VvM1YxeStBRWFCa3A2bnZTaVFIN2VS?=
 =?utf-8?B?UVZhaFNpMXpIVDNMeXQvUDNLRys3UE1VZmJFWWpIaEtWdkoydkNWQ1VzY3VH?=
 =?utf-8?B?S3BOUkk4UFdkV3I4Ny80V2llYnBxTEgyaUM4dCtOM3Z1OXZiUGgxekJBSXU4?=
 =?utf-8?B?STE5ODFjZ2ZwTVRhdjVEUTIrdGxmVmg4WVhGdTJkZjhPOXZRVnd2QkxSZlQy?=
 =?utf-8?B?ZG5ERkh1RnFIMGhnR2tBWGtZMGhuL2pPd2xDbWU0dUZQVmp3ZkpxWlhLWSto?=
 =?utf-8?B?UHltUzZ1bEtFaGMzM0h2ZUVPRWRzNFplQk9XZ2g3dmR2dFh0YUF4dkt6bnQx?=
 =?utf-8?B?ZGc0dDllZ29RR2g4dUdmZ1M2MzhQWE54blpKRDBhWndzeXdrSDlkaExqTjRK?=
 =?utf-8?B?TWJZei96R0htbTM3eU5UMUtBTkNJenE4OFRVMUk0ZzV2RkpUN3VlNjNQNVNK?=
 =?utf-8?B?am9pQjZaZUQyRlZsZjlhdEt0aHdtem1XWDN6Vmt5TXIwU21DUk9iRHJqdDE3?=
 =?utf-8?B?dHVxRG9QejJYVE9hSEhKbXRFdlBEQTVJQ3Evdkt0QU1HMENCWndIL2VCZ1NR?=
 =?utf-8?B?UUcrSGdpcnBDVTArNUMyWmhSWUk5em1vbmcyUmtvd1hCK3RrUEo3NUpKREhm?=
 =?utf-8?B?b2Z3Q1JMek1uQzY0VmJJdmNsOThmNnp2WUMrZEdjRW9QU0s3dTRhT3E5TFBa?=
 =?utf-8?Q?i8OE+E5U/7iu0YTne6nyz0QP0oEHoO3fxy5Vixs?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fec7ad68-569d-44aa-57fe-08d9716b0fcf
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 19:18:09.2060
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xNXRHAmPAbFfK2WzzHaJ3muilch85ZR9USj3jpf+/uGB/+w1plrHQfUaNK1+ONnYlS0To7DN6UvK+yZZXEsr/8ChaHsyXK585ik1+V/LPnU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5597
X-OriginatorOrg: citrix.com

On 03/09/2021 20:06, Daniel P. Smith wrote:
> -static inline int xsm_memtype(xsm_default_t def, uint32_t access)
> +#if 0
> +/* Could not find any usages */
> +static inline int xsm_memtype(xsm_default_t action, uint32_t access)
>  {
>      return alternative_call(xsm_ops.memtype, access);
>  }
> +#endif

There wants to be an earlier patch deleting dead code.=C2=A0 We don't want =
to
retain this if-0'd out.

> diff --git a/xen/xsm/dummy.h b/xen/xsm/dummy.h
> new file mode 100644
> index 0000000000..b9a7e8c40f
> --- /dev/null
> +++ b/xen/xsm/dummy.h
> @@ -0,0 +1,739 @@
> +/*
> + *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
> + *
> + *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License version 2,
> + *  as published by the Free Software Foundation.
> + *
> + *
> + *  Dummy XSM hooks implementing the default access check. Each hook sho=
uld
> + *  have as its first line XSM_DEFAULT_ACTION declaring the privilege le=
vel
> + *  required for this access.
> + */
> +
> +#ifndef __XSM_DUMMY_H__
> +#define __XSM_DUMMY_H__
> +
> +#include <xen/sched.h>
> +#include <xsm/xsm-core.h>
> +#include <public/hvm/params.h>
> +
> +#define XSM_DEFAULT_ACTION(def) xsm_default_t action =3D def; (void)acti=
on
> +
> +static always_inline int xsm_default_action(
> +    xsm_default_t action, struct domain *src, struct domain *target)
> +{
> +    switch ( action ) {

Either here (because you're moving code), or in the style fix, the brace
wants to be on the next line.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 19:20:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 19:20:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180217.326780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKAh-0001wd-Gn; Mon, 06 Sep 2021 19:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180217.326780; Mon, 06 Sep 2021 19:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKAh-0001wW-DU; Mon, 06 Sep 2021 19:20:47 +0000
Received: by outflank-mailman (input) for mailman id 180217;
 Mon, 06 Sep 2021 19:20:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNKAg-0001wO-5F
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 19:20:46 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 886da6f6-0f47-11ec-b0b7-12813bfff9fa;
 Mon, 06 Sep 2021 19:20:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 886da6f6-0f47-11ec-b0b7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630956045;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=H61zcukOsd/RVg3l1xWthoBS5YMimXZXBl98tbnD6FI=;
  b=Df2OdeIy/91r3qNvShzwS1NF/u6F++BNrhn2vX6knTtpfyNrvdbvlWkB
   +e3OMrmHWJKonTJzrdC6/HPVluM35t7uVf0lujoFZcoWUeMWU6TysX6bU
   jfSPch2/3QL+eE3aUxk6B/8uKOAj9t53J1rgEmDfZ7vpCfQBwhonCfb2l
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: chvutixBMXO4o9wIBt7vAERNdpsTi9K22ZuepRfmz0jbKVhtqmqH1n67h0VE2dHk6G472gBwFd
 SSa/G/D5zr++T9NkRIt0q4yZbZWL/LherjNmlm9QpJVkKhP94/S/eVXS0uSoWvWSP8ozQCc0WB
 xN2Kor5HpT3YD63lON/ofI6owo7P1tLYJpvr0s/L4e8szxvPohLATv1L7AHpsActgzL4VoWiDm
 4lPHcsCuYpK08ZUutAno1B6KOyeIqRavMcImxDW/4QELluCifYGPiad9USUSi3WDmesU5LQTM5
 VSrka9UlwV9tFjJ5BWGkvFlr
X-SBRS: 5.1
X-MesageID: 51711369
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:QiVGLaBckZwM40blHeg7sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LS90dq7MAzhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSuwEIdxeOkNK1kJ
 0QCZSWa+eAfWSS7/yKmTVQeuxIqLLskNHK9JTjJjVWPGZXgslbnnZE422gYy9LrWd9dP8E/d
 anl7F6T23KQwVoUi33PAhIY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX222oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iCnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDw4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWAoqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocXTbqjVQGdgoBT+q3pYpxqdS32BXTq+/blkgS+pUoJjXfxn6ck7zE9HJFUcegN2w
 2LCNUwqFniJvVmGp6VP91xNPdfPFa9CC4kAFjiU2gPK5t3T04li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="51711369"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CHw+64Wk2BTMd+YYZFxdW6DqgeYcdyEoeSZwE7ry3VaDncZQ90fPaiWWMDzbzmbVKfAt5Km030hAqFbyR7sbq3PBbhPNFnFlP3uSdK+XYHVy2TU5p80C4sFNq2LEXHFueT15aJy0gDooHRRC3cOgU1vXEPxK9Mx8uspnOR1pvo8gtTctvmXwSsUoUMXyjIiyJ73njzqpuyxOUh2mZ4A2AEklQ5wWqw268mGmUXlGG9sFqo/M832Vsj63WUSncv3Q7kPGdGe54euxKxjN/5xBp8jrql9TmjHe8dPnlQe5YPz4Osr+H++xvXwZ6nT9fDY3lQpdxRhesuNppn8afRvRRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=068ibvkm9tu8zkK4YojlwHtkjfAJ7acXP/CA6lRGMLE=;
 b=j2t8LqYRK4Wm1KvdTLCxsqQDPhZbmXumtoPKtErpvfP64VyXGfqOo1yHcHEGHEFpf732nGJ2vNtYkCtY9FNWS7+Jc8SazGXTvBFKbc3CODnudTYI/+Z+A6qGlFQ+c4tSaFPZ1q2RGNYxZ896ZHJb4iuDOWWUnJVw6MBPvZ6Us2iwqQs/KL4+R3AqvtbAu9dAz9aVUmr8WJLuW5+uWxweKj1O9bR62AlEY0jqYInuWDx0ZppW5rJPJLDqeCdf2L8LD8qqbM3dhxTV+lvdbocSHL2qBIz6hSTWcgDjRB8VdZT8Cw92RN3db5nlntC/0fHYHgksYW6MgjkjV+U5LAjJMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=068ibvkm9tu8zkK4YojlwHtkjfAJ7acXP/CA6lRGMLE=;
 b=X50ZueyKqIDDI6jcm8U+aRaJAxwncZKFyimLHMrsLW4FilmLuVhsOjEaahQeNnat0YRupyZJRsvWXz69grjkYJQn+SIy9Sk9FmLR3vaEtDs1o3xmlecHPyQmdmi+yU59uxbpWaDA4UkIlJBLnhmWdWG9s71h+RZlcKUCBwFrGdw=
Subject: Re: [PATCH v1 1/2] x86/cpuid: Expose NullSelectorClearsBase CPUID bit
 to guests
To: Jane Malalane <jane.malalane@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, Jan Beulich <jbeulich@suse.com>,
	=?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Pu Wen
	<puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <239f7624f3ac3793993dc045d451079aa555b90d.1630929059.git.jane.malalane@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <7a87358a-37b9-0292-b6bd-ef33026426df@citrix.com>
Date: Mon, 6 Sep 2021 20:20:34 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <239f7624f3ac3793993dc045d451079aa555b90d.1630929059.git.jane.malalane@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0298.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a5::22) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 12ed33ea-445c-464f-425d-08d9716b69af
X-MS-TrafficTypeDiagnostic: BYAPR03MB4613:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB46137B5BA3A1667BE4F2C02DBAD29@BYAPR03MB4613.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:473;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: bHLgaBrnZVAhvZtlQF7KmqGveJTVlUhoml1r8gFhUzKSXgmraMn7osDf1qkAuoM+eoCqV4WqIZn8ujU3OhhHIIBwBaiEi18Qp3hAxmzl3K/Pg9o+/EOatKy0Scb/x6LAvaJPZG6hVLZCRhNFDEXbQhQlovQuFD2oXCsGNFMp49zdMMOmHyrwap5segWk/RVXBpus18yeeli2X+/U6gtAKFZB/bRR0O/MCmC0+3Fmv0fiwPAvT8tF+W8QccK8AHLK8wOUQoOpyfgSCz0VKJ9ALn+FXip7xdAkZk7shbEWJDPcRT87WJLgJMZGzAJfUFC75jySvZJKDLKCzE6h5KnpOQ6UHAXs34lVD25ZIRVy1cfU1pWVKZNvcnJojoEjmZ53kFiRu1lylpCBd7H3tF2fR3cj08LOI+KgA1JcZ2ic4xvoQ7rA+i1tOxr8Chkz+I73r5E3p5gCnUPEWT8mQbvpcOyo3589sc5Tw5L+q/wm80/YltREpLyiFNaAefPdS1/zD2REKpgPK/3aY9urofSB3l+LFYhS3PCk2hzHockacEpOinchZ/81PnSVE8H5mD1CpPvF8lGF9mO56QOLSOaa+r0flSwFXqI+D5c71cIdf4N5ROPPktC0KmZtwTTCXcrIGYODSdy1YVD7II8m3KEA4mzdyt3IsOh9UToAxrVeOz2pxC9eSBTm49cHjJJAoYq4Gf9rEK5H7RyXmzT+QBlHUjtxX5cpuGqYXwvHqqDzSlUTrbsA8nQRkSK0ITEfAFTv
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(136003)(376002)(39860400002)(396003)(8936002)(110136005)(54906003)(478600001)(66556008)(66476007)(31696002)(316002)(38100700002)(2906002)(16576012)(2616005)(66946007)(31686004)(26005)(956004)(36756003)(6486002)(55236004)(186003)(53546011)(83380400001)(5660300002)(4326008)(8676002)(86362001)(4744005)(6666004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dUNIOTREK1BSZzhwUU0ybW9DYW5NcVZUR1lEOWNOK0JVb3JBeVRLL0g5N1Rm?=
 =?utf-8?B?OER6WVAyTmsvTm5BaTN4bmRTMXVZUnlRK3V1MVJReGV2VHhvbTlVaDhiaHM2?=
 =?utf-8?B?UEQ5T2lUVys5QXQvRGZGOHRNOGc1TkNjbnVxRjB1KzVjclRtVlpHOVRoK055?=
 =?utf-8?B?c2dUZnVOZERqRFYxYk5xelRReXNkZkRHMklCNGtvd3Q1S0dDQStVeGpBV0cy?=
 =?utf-8?B?NlFWOXhSWFgyY29zZ2NESEEyS0FNbSsxbXFraGhUa01pdWpjM05mOE5zbzdH?=
 =?utf-8?B?VjIzaUJHTUxxQTUzYm9CRmszd1AyVHBwMWM4WjFrUkRCdmU1Tk52RlUvQ1Jr?=
 =?utf-8?B?MEFuejd6aUVzK0dDbEpNMHE2c0RQZm9uRUp1N3kyUmozYzIveHU0OFM1dzVI?=
 =?utf-8?B?TklJN0RRcWVuOHpUTGJZV21WRFZtV0xySGhWd2lBVkdJUGxpZzI5L1crYkM3?=
 =?utf-8?B?eWxBZnRhdk5OL0VuUUtLcHdnc3dWZUlOL1RYWk1KU3pPOHpJY2tEUTZLNmk3?=
 =?utf-8?B?MFI3dk9RYmhRM3d4YVAvWDZkbk5NSndFSEp6eGVLZitIMCswclFzdVljNHU2?=
 =?utf-8?B?V2s0RGx5c3I0dUxZdldRRVNvOGluY1JZOC9sOWNEMTZNK3FKN0FyS1VkbGNM?=
 =?utf-8?B?V0plM2N1NFBSek9mSCt1aHdmclNTQmpIU0ZmMSs5VzJ2V3FhQm1GZHdGZ28w?=
 =?utf-8?B?ZkVIUXJiU2taUjVKSjNpMmpMZkZlWmt4UjJHTjlLUlZFSC95WUxvMDBKeEhV?=
 =?utf-8?B?YXVsR016WkxEY01pTzdxUFFyNkNEUVNJdjZLTWh1ZTZuZTcxVzFsUmtmTUNY?=
 =?utf-8?B?clF6blJOZzZ0S29PeHNUaEdMWWdGTVNPditqVnF2bXMzL0pJMXpCRm1ZOHZm?=
 =?utf-8?B?RkNRdnRvMUdBYnViMmNGQXdQR3AvcnFBOTdJQjlvOEtPbkhUZTdIeDZqQTFU?=
 =?utf-8?B?dTR0UXh2TlQveXdSVDJ0WUNNeEVkMGpUY2Fib3hlNFozSmF6UlRxWHN3R3Nw?=
 =?utf-8?B?b0JMRDVEdkJTUm5qVDVGSXN5VUpRempKOVJvc01ZdTkzTGxaU1hqWjBmMW1B?=
 =?utf-8?B?R2dKNGdKVW1taVNOdHZXalhMb0JDdFBOMlhmWGMvMHdUK2Z1WnZtbWpTd0Rl?=
 =?utf-8?B?ckgzeUFGV3U1U29xVlNBTFZDaEdidEdOb0lSTFErSWF5NDNId3Q3b2U0Qlly?=
 =?utf-8?B?YUVlWHFFSC9LREh3Vlg1MjZqa3pqcFozUlA4M3dsdllFRVdKNmpUVFVqaTYv?=
 =?utf-8?B?eUtoekg4Ni9hRXg3R3BwVlhGR1lOeU5QcENkZGRlZUlTeTlGOEZBckxDMUJy?=
 =?utf-8?B?eEhvQUJjUWRlR3o1Y3dIamJGazdUb1JTNmNNWGdxSi90V1duU3NXNVpzMEE3?=
 =?utf-8?B?UjlUdWJneVl2aUVqTUFJamdVWTUwNEJTMks0UFNKRDB0dk5FNWdjVzNmZGZi?=
 =?utf-8?B?cDBhazErOWsxRGZFVjBMZU90dkpjY2JWakhXVUhYTXNDZ0I4SENLNEN1WkJ0?=
 =?utf-8?B?azc3UDhkNXh4Z3hFRkdWVkRBNEpuM08xU2xDNkI3aGJ0TkVKWVVwNlBiZ1gv?=
 =?utf-8?B?NHVvVDZzQjN1dFBiTGFXUkhDNUpLTFowVXhxcGdGYUUzTFFPR1MvZXdJUita?=
 =?utf-8?B?QmxsbTBsMmVNZ0hVNHRUa2FBQ2t0aWNMN1F6dUEvMVRjU09SMFRwL0pqMGky?=
 =?utf-8?B?Z29ZcVUxb0c2NmxYK09NOGYySjM4SlI2S09UWGp4WEY5NEp3Uy9FbXp1TEZ2?=
 =?utf-8?Q?IpKHpJl04r1ncYD+FvDL1ie/y2In0JAGSgCP3F3?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 12ed33ea-445c-464f-425d-08d9716b69af
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 19:20:39.9318
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 86owFCx1sifHqc1llDU1ZFMf6FVCSNdgxChXxjlSdIQBukaXwtwFtio6zacjNiXUMrp3xorTVenO+9iGjv8J57s8axY5JwOylbIqWFVRTDk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4613
X-OriginatorOrg: citrix.com

On 06/09/2021 13:00, Jane Malalane wrote:
> diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
> index 380b51b1b3..e5a7c94c78 100644
> --- a/xen/include/public/arch-x86/cpufeatureset.h
> +++ b/xen/include/public/arch-x86/cpufeatureset.h
> @@ -285,6 +285,7 @@ XEN_CPUFEATURE(FSRCS,        10*32+12) /*A  Fast Short REP CMPSB/SCASB */
>  
>  /* AMD-defined CPU features, CPUID level 0x80000021.eax, word 11 */
>  XEN_CPUFEATURE(LFENCE_DISPATCH,    11*32+ 2) /*A  LFENCE always serializing */
> +XEN_CPUFEATURE(NSCB,               11*32+ 6) /*A  Null Selector Clears Base */

On second thoughts, I'm tempted to add " (and limit too)" to the comment.

Can be fixed on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 19:23:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 19:23:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180223.326791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKDT-0002qs-0R; Mon, 06 Sep 2021 19:23:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180223.326791; Mon, 06 Sep 2021 19:23:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKDS-0002ql-SD; Mon, 06 Sep 2021 19:23:38 +0000
Received: by outflank-mailman (input) for mailman id 180223;
 Mon, 06 Sep 2021 19:23:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNKDR-0002qf-M9
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 19:23:37 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eeafac2a-0f47-11ec-b0b7-12813bfff9fa;
 Mon, 06 Sep 2021 19:23:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eeafac2a-0f47-11ec-b0b7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630956216;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=HhdcBXvKAHOjeRXnQDNqYRtYE6DpLidMQx5V6Wm9xSg=;
  b=K8R0nO0A054KveB+MX64tWirwlVFL9uYE7KdIxeU20BXDVk1r6DiAIYD
   UEl27BA8c49e3N4XxhVeNtN62uZC8aFSx9oHbvcxBIeUxbsbNcSXQ6sH0
   ol8n8vC6Yi3RIZTUCdTfFgpApEAW6saciFDpg5dZtrxT7OQbP2rdT08B+
   Y=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 0lMz85qtU7JE6sIzM2Gmge97+Vp5v8v4YaLVP0BLPbmi0Em5JRTSnCV9nd7wsyALhX3Z8hr5Wl
 DTtwt3x5lS6SLEbGmcPhHZ4i9xRgbt041D7gyd+i4cuj+sUOSlWxgaOYcaLbOV4mNAnMrOp2ko
 H1yE0pNlElOGBJIVhutncGeIOegC1JCatwM2LLPYW4P7wm3dpzSREwIKO9uk2C3ALhoFWUt5RN
 ERmcyUbPdx1aLuhHgVjHpPN8Z/D7J1bsrCfsQRkVMDSacfSS4qrPzNw2RZWi1zOUMI9DRxQoix
 iZymO7lNz4YoQZ1m7k/WMq5n
X-SBRS: 5.1
X-MesageID: 52508760
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:OyIpnqCr86S9l/PlHeg7sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LS90dq7MAzhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSuwEIdxeOkNK1kJ
 0QCZSWa+eAfWSS7/yKmTVQeuxIqLLskNHK9JTjJjVWPGZXgslbnnZE422gYy9LrWd9dP8E/d
 anl7F6T23KQwVoUi33PAhIY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX222oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iCnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDw4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWAoqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocXTbqjVQGdgoBT+q3pYpxqdS32BXTq+/blkgS+pUoJjXfxn6ck7zE9HJFUcegN2w
 2LCNUwqFniJvVmGp6VP91xNPdfPFa9CC4kAFjiU2gPK5t3T04li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52508760"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ag9pRZsMEtDDmFEPfgaiAV78PBfhrV7rpAweLAWjqWYfTBfhzv85CtRCSeE8m3ErB0YKQNja0izaZ55r0rM9R2S22LtR0HuaewiyUVpFCFXbMz+0pW4mYqzpZbqb2SmfN9fPnjf9KurkCGUaE1knUR8ITqq+2q7LY6EHf28iShh5VR4t5ZdqNy0jGTz9J9T71yNGmyqlt3jlL2w3E//Zwp3/2ps5sYTV6DrLhIl3FCXxFn6IkwitAcN8pRRroFEmJcpXtJQyyVCkZ/dMNceXS0PyVBgISu61vzwMLGV5hBdUuMd8JZm0ao/maG7NA4Ct3bDF1H+tPHUtIdfFPaqK2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HhdcBXvKAHOjeRXnQDNqYRtYE6DpLidMQx5V6Wm9xSg=;
 b=CJEx8dAL9Eqrb4KSQJKYTEy1zeBS6BzZ07/WtWERETKh7ATe8z2VbNxIa+Sr/we6cr6EISTqzsMTn5Xr5n6TDIQLkSnGqkcNeqpgvkQpFV1Stk7p7qVKNjsxKXmTiqpVFsFKFb0eknr4hrT+4iLQJ5deVC3z+CDDpeGXhTyHGyW+UcsQzhC2bWJc8ysYZnCPzmIxhtbAUzLmWGUg4hwcsWP7YPSYSoysU9vVlMncB+RNP9I890FmHCcJgWRTzVz48a4rbjXJCu1tIbn0vF/Rk6TS71JubZnbJU/LSpW83yVUcjgc0DaVzIVM7YPDt6O1MAhEWt6IwxrQ3aeufuYTXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HhdcBXvKAHOjeRXnQDNqYRtYE6DpLidMQx5V6Wm9xSg=;
 b=B733muLDt5WehOn24iLdHi2mC+sVGziaf7tVrMqFHRMjpZo65dsSsKdwJpsKYsb4t6Sp10/Q9uvsVvSSo2eB30ecePVdcMFWRaOsbsn3lt04sQWBpKr6QHkG3UK4tybS70fnxBvwVeqOIEw4oOVetLi9+HyD+rc5OEbeh+yczUk=
Subject: Re: [PATCH v2 2/6] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <5972ce4c-bdb0-6d07-afad-cf4dd5615488@suse.com>
 <457637e2-d35c-18d0-ead8-0b513b257927@suse.com>
 <9b7936fc-f81a-d5e7-5ceb-6e10588503dc@citrix.com>
 <28646a17-420e-bfaf-8f7d-6d109818ffc3@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <80dc4625-a53e-1f18-2a77-a4e45c690cc2@citrix.com>
Date: Mon, 6 Sep 2021 20:23:24 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <28646a17-420e-bfaf-8f7d-6d109818ffc3@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0100.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:c::16) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 996d6716-c469-434c-54f8-08d9716bcfb5
X-MS-TrafficTypeDiagnostic: BYAPR03MB3735:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3735F7C8CCFD9EC29CE9AE77BAD29@BYAPR03MB3735.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Zqpel8VDUBjZRih4xbXby2J2PELVLN9eAGPPtIiGdwhumNXnyxKKg3Os8LUeN5GQSNtRWZ/0WDOub2A+wZLROEB2cZKQkKIMzv4CqzK+BFJ12ztoNUuDi5iZxdABPKw0WUHK5w/xc1dso2yguat+4VO+d/dobcYIM+vp9WmIB6bfp12p7U5qvKATmLRWWsq9+Aadt63Uanb6p61Wk2AGz4ieYfnconzfYcGBA8lUCWBpH6C4/4K6A/1b5KrD6C/Nd6la3BE0rYTA+wZSQKjMVn/sbEwRqUvqY20U5hEDWVtwUaKK4w569MlTHlzUxZvMEl9FWPMRXBNq7267Sqhpdu57mxyzMvLnBMO1Mnxd07OTL8ZOJ+1Qh3FSoZw0I+4eb/ljQHsb6saghtCdSfH1/Qmb8cDwuI4+dNvKE36l3sXQER6o9QFelqRiR4RevJKcPCuyFXtU6Blpv4lUT1ev41PxkolYZQSuILyNOZBKYpBu5K+536NU2YPR/E/504Sfky2fV2GqiezFpghNfXAt8F2zFP88nl7nXxVFbQCOr5afKjewVOdodjNuH+KL5ZKbrwjOFVg2WJhhZwMiTEj+QYMX6GY5irZ0qKeoZOfMJKRrnUtexsUNg1mHY5CuloY38lTPepB5cXfkLwgZFTR9qDIYPvkng8WDvnKE5jT6TFfEQt6ytCdT9rOkk/nFwHhYK+4fkRa1bT3lSRj5F3/uo3p6h1tBcCub9Jdpdo1D7ScjWXXovK8I5HE+8BFRheeN
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(136003)(376002)(346002)(39860400002)(186003)(53546011)(36756003)(956004)(31686004)(6666004)(38100700002)(66556008)(8936002)(26005)(6486002)(8676002)(54906003)(66476007)(86362001)(83380400001)(4744005)(316002)(2906002)(16576012)(4326008)(6916009)(2616005)(5660300002)(478600001)(55236004)(31696002)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VEFyOTFsVFZjaTl6Q1JzcmhCbGFnZHpYNnFVbTNEWFBDU1NGeTIvM3NZTTFW?=
 =?utf-8?B?eUQraWVTQWxJWnlybGZVM0tSNTZaMGlWMWJoa1RYdW5weWxKbW9YallmU3Rq?=
 =?utf-8?B?TVR1WkRab0swYkQrVytPcnRNdnByNDZtNHU3WklMYmpFR096b3Q3aldzbVh0?=
 =?utf-8?B?Z2dGNnVFK0xSVWhGOUlKd3lINHBCMkhrYU40N2tzUUNPQTRtckhwblRCVE5Z?=
 =?utf-8?B?NEVrbytBZzZWcXdDUCtmTkd0M09MNlJZQ0xYcS9UN2NyUExuTjUyQmlXcVlX?=
 =?utf-8?B?dHdwVDllTkZpSnppa0xaN3FZUVk1THI2aHYwUEtqMDJYYkxvWnRkTURCZkFT?=
 =?utf-8?B?Zk1saVpVczh3cjJsUy9YdDZ5ekpwcUptc1BRUCsxWUQ0Q3NyKytCTEMyd2do?=
 =?utf-8?B?a0RCUE1NZjBDQ0pUeUxBWUVJQjB0WnRia1ByRUdxWkYzSjJVUm9WSnp6Rnhm?=
 =?utf-8?B?cHdOV2NWODVlT2tyOFpDZlRRZWtPZFdEZlA4UGVuVDhReGRtQTBXeDJsZDRV?=
 =?utf-8?B?TjhjYU1JVi9mUGRIdUdFNnhGbU5SVFQ0K2Y1dXRqRHFWaGE0NEh3bEUwVyt3?=
 =?utf-8?B?a2tFdFVCMGM3SEdtVlVmRnhZVlllNU1Da1NKMXVySWZiOGJiVDhhUUplRnBJ?=
 =?utf-8?B?a3AybHNiU051bEwwaVprMmQybkZsa2xVQ2hOZDdpZG9jeE0vSUlYZVNjclkz?=
 =?utf-8?B?SXZJdGNUYTdZOW40d3llYUpCdk45djZPdWYreHdoY0lGOWRtZW41QUpvTjZo?=
 =?utf-8?B?RGZ4a1NnOTNWNllzMFdKTGFKeEtZc2hLQVk1T0UxTFltQlVNRzdxbFJaR2tX?=
 =?utf-8?B?cXFlMThJNEFnajJBWEtKVmhWWFJnQ09DalRPN3J0NC9NMEF6aVlVc3MyaXRw?=
 =?utf-8?B?WFdKZ2N5RHdTWllSZHZHRmNjNCtsTFJ4YlUrSlU3YXVpSFRnd0JXZWV4dVcz?=
 =?utf-8?B?U3hOREFnQ2c1bVZ5eno2aWtqU2drSHZ4OEtjc2wxQjE1ZWk0K2VmVHVTejVh?=
 =?utf-8?B?Vkc3RjJGaDBnV05JOGhhaitDUVNETm94K1hnTEhad1l0NEJDSFVid21QTHNZ?=
 =?utf-8?B?RmZCZ09KQkROd2lPRkVjRnErcWVFZE9yQzRRSnlWUXMzRGxsdEZ2NFMyVmZT?=
 =?utf-8?B?WGZJUko5V1IxNU1ackVvY3pDZit0L0lSVjVnUmFFbmRYdHp4Rk9Pemk2SnVF?=
 =?utf-8?B?cll0elYydjRocHJpR1NNUjJ0dlMrcDlramxQRDdYUUNjWnh5TEkzNGpvZEwz?=
 =?utf-8?B?MmlGQ2UyZVNPMjZRSEVVR3g1VHFsclFUck91RUtvNU1Bc0x6b2ppaFpWVHRj?=
 =?utf-8?B?N09zK2JOTGN2ZVUyYXdUQmZyclZ5YXhPRXNUbnBZMWVRc21GUEtiUmlkSmFE?=
 =?utf-8?B?Rm0yTzZ0amMwWHpQbFpXNEZEd2lSZmxOM0NLVlRCekkrRm1pQ3ZjQThxSi84?=
 =?utf-8?B?ZVpaUWNKdUE1RUkzd0c5MFZRK1VVeXBzVitpMFhDRXhVTnJVN041Wis5VS9v?=
 =?utf-8?B?WFROYkpKUE9nV2xDdnNuRExaK2EyNHdKaUQ0YWJDWU9VTCsxMzJZQUZqdjNK?=
 =?utf-8?B?ZUZQMWNSVG9SMGpsdXR2bm1DU3BWU3doZ05Kdnljb0NRb09ZbTM2VVVHWlkr?=
 =?utf-8?B?bWlOSE16T2FNUm9LdXYzcktZYjN5MnE2c3dsSDVJNk84dGdMaVplNEhYelk1?=
 =?utf-8?B?eGlrd0wrdVV6R1dUY3B3U1JvZjBVWHl3ZHhSTDNWRjBEaG5yZGRVckx6YjQx?=
 =?utf-8?Q?pDvMxIA+3YtgogMEbqMCwGPqUVJrJb/ZXy+X+1U?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 996d6716-c469-434c-54f8-08d9716bcfb5
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 19:23:31.1002
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eBGUND6Unvhm42Cnj2nBrod5BhUh8tP3ANN9mI3c9dal8+EoYITZzsqdA8Gn1WB1gXRnVw+IhyNRKXlocQbG/LPA5/T2VhtJhg+7n3F4GE4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3735
X-OriginatorOrg: citrix.com

On 06/09/2021 16:55, Jan Beulich wrote:
> On 06.09.2021 17:48, Andrew Cooper wrote:
>> On 02/09/2021 09:33, Jan Beulich wrote:
>>> To become independent of the sequence of mapping operations, permit
>>> "access" to accumulate for Dom0, noting that there's not going to be an
>>> introspection agent for it which this might interfere with. While e.g.
>>> ideally only ROM regions would get mapped with X set, getting there is
>>> quite a bit of work.
>> ?
>>
>> That's literally the opposite of what needs to happen to fix this bug. 
>> Introspection is the only interface which should be restricting X
>> permissions.
> What agent would be handling access violations in Dom0?

None.  dom0 really shouldn't have any NX mappings in the first place.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 19:27:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 19:27:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180230.326820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKHE-0003wR-Qe; Mon, 06 Sep 2021 19:27:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180230.326820; Mon, 06 Sep 2021 19:27:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKHE-0003wK-NN; Mon, 06 Sep 2021 19:27:32 +0000
Received: by outflank-mailman (input) for mailman id 180230;
 Mon, 06 Sep 2021 19:27:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNKHC-0003VO-Rl
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 19:27:30 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 76499510-0f48-11ec-b0b7-12813bfff9fa;
 Mon, 06 Sep 2021 19:27:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76499510-0f48-11ec-b0b7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630956444;
  h=subject:to:references:cc:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=YiUT4mNjlZ92jk3RwIuC/v40/jTL0JJz3cuHc6HT3CE=;
  b=bqZvMwPik9E/dlI6xjSQhFZ6z/XLBBeAYbgeT4tTZyjFN5pFRl+g5h3Z
   rLa+h8b4Jsor1gsV6S500phZbsEIUCpjilqowr92Uz7EzH9OCuPxZF/AF
   wNleulWBv89id2uaz4YL/BobadM+kx3dvKZCD2TmagWbVfL9cfvXrPkN0
   M=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: muftSrz5kPvlD9/8HA8ixQ+XB8VqvebWULeKAvwli3Goy1hGAGRhGgS9qyErASdkKDgksYD4tW
 3UXeWt4I8W+wHPjc0R/EmDzZlg8THlv9mUUV86YVpKhlGR5yWdvw9BERKagx/hC1GvCgzciHnX
 Nq+JRGPgDWzvnclp+k1xNxqysOLssoLpbmiT0fAKme1nFwiEGzuEP/3d7LhFiD4vedwv2ZYu9R
 L+ZZmlJFEdXbPdqPxUcBnyi0vTRu6FppTUEdIVfKvzQxVosCXG9//LYvaqUHl3/0gx+BVPwEfi
 3F4X5CoaiBFy9pk81shadGVy
X-SBRS: 5.1
X-MesageID: 52117783
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:vFpb9a9FPigGqSG8CoZuk+FPdb1zdoMgy1knxilNoENuHfBwxv
 rDoB1E73LJYVYqOU3Jmbi7Sc29qADnhOFICO4qTMuftWjdyRaVxeRZg7cKrAeQYxEWmtQts5
 uINpIOcuEYbmIK/foSgjPIaurIqePvmMvD5Za8vgRQpENRGsZdBm9Ce3em+yZNNWt77PQCZf
 +hD4Z81kGdkSN9VLXEOpBJZZmOm/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTsj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZuA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQPZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv2nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLbU5nghBgr/DWQZAV2Iv/fKXJy/fB9kgIm3UyR9nFohvD2xRw7hdQAo5ot3Z
 WNDk0nrsAWcie6BZgNc9vpevHHf1Aldyi8eV56EW6XZp3vBEi936IfwI9Frt1CK6Z4gafbpv
 z6ISVlXCgJChrTNfE=
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52117783"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=I/p6R6dXyjhuqaq8YcR+fIG9mhyXkOUAurGrEy4KswqPdaDfvEC8kU2w0D4PDQXnvrRN1qua4A2xSVzSVZJIkdEe+4nhT+k7KsQ2ygfYXiXSP6dz9oa+/q+67BuTHb9AJYohML78/DP4++KmMUtmqF1+8SjHkBaMtSFy5NjvpmDH8v3XJ5MLnM0OCbWTZYm0thhVfYdy8Hag95jm6NJsLtDMOPoP2M2SiuVgSo/wXxu0030n2XtTFNcfT9bLNdctmZ3EuP732zryuOTG6xTs6avVsFMcUJS4Z95dalICpBD6JZvovZuS0mjTaxWa+pgQZaq2z3C5qZD/K7cAQJ5sqw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YiUT4mNjlZ92jk3RwIuC/v40/jTL0JJz3cuHc6HT3CE=;
 b=lm83mPmT8xLKC+iEsopt4sfgdY5m3voxhqYD4E1/mulXQ3fy0dJykKotMTbvUatWeZBLpfB2bGAvRYNCzg9gHSXtNE4fW7crPGOBp2r0NxOL1jWrY4QvyhNdPoBgFotXCxvoHjAWiiPdM7D4sL0hqDOfm8yLs6WiG+6OfDTI6tXWT5wjB+z/jDMBOgXhvn70Qnb8tclHxvFMibrWbsvpFdaSjjhbEQBOfbwErN1b+G1SjY+fCoWVgxmioMlfWa/lCSDcrfxEfDPjvIkgMSC3LAJeKsFDI2ces6wCppSd1rz4wyqzNQ81t95WtSuXY7G+MU/vrSFOqIV8nfx7qVAInw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YiUT4mNjlZ92jk3RwIuC/v40/jTL0JJz3cuHc6HT3CE=;
 b=rK7GqcBhPhf+jCDtJyCMZXbEAWx/r8UrO2FCakqhlgO8XF53exF3VVaB42E29YcVEu8h2hrXoMI2B9L/YXcgNGGjHpsou2Hml8gRLI0FQe3BPzTkk6wdzT1YJfoiOguttz0Bgq1fyOfe1yyRX84mHY7rn+0s0BeayF89CSfsZJs=
Subject: Re: DomU crashes when restored if it has pci passthrough
To: Bhasker C V <bhasker@unixindia.com>, <xen-users@lists.xenproject.org>,
	<xen-devel@lists.xenproject.org>
References: <CAPLCSGD9ZTaoMSqgo6EEdW4VYs9wrz+YvChR_Cx0Rkm8x5iq0w@mail.gmail.com>
CC: Wei Liu <wl@xen.org>, Ian Jackson <iwj@xenproject.org>, Anthony PERARD
	<anthony.perard@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <65f4630f-546e-57e8-7147-b5add37d594e@citrix.com>
Date: Mon, 6 Sep 2021 20:27:15 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CAPLCSGD9ZTaoMSqgo6EEdW4VYs9wrz+YvChR_Cx0Rkm8x5iq0w@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0456.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:e::36) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6221555b-8e66-4d15-1bcf-08d9716c590b
X-MS-TrafficTypeDiagnostic: BY5PR03MB5000:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB5000556A10E6BF9460C4975BBAD29@BY5PR03MB5000.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: eFNOS8bYy2gBtslgL45WsNV7fJjcEwe/qiNHKoXk+07DMLQIjHAxKfapJiRCihaEICAM7amsw7zFKl6dTtwMFkElHe3F0epm3/9PHWlVSsubK1k2yk32UezesoNW4+87dIdBIIC0APWx5U8XffC+/5NuJAZtqzcl2ieIxlcdJQWVmTVp7SZgcegb7z/8XWSbyuBApzGJy3V6TZHFe90fxbsZpr9RL4gE+dOOh77G6e2R3ZjVzsZMEdDTfzoFgih4dhG4f6uoNCNAhgcBGxmrx2ilIL1gBNvwBzVWWzyOHbsTOD6mzMkiAXAjPoeTpGlCdEMx/piTWfUxuL95Q11Xgwu/bgLm7yDa+CaU/BV7Lz1XH97jqeQ6mLYuOALYajWAUQYjL2FpJ8xi9T6YLP5Y1yQPM6uTVeZwCgA0MTbqMJbdzoG9JJAiW7FHblxYkcA2r9T19OL2Wkgq9IoiYTBZNFtSh2S/StU838nkU6KEYPaG1zLLigU/fXZBhkOpzhVJzGMlf/tFAGwECyKMsq0SRFhIG4lzfst4SJJqnzMSw8IAsC0qwJptt5zjN2DMwissbzQwdsEyBoSFAxaOUpsZBMJd9aGkreq9xqExAWwbxLnuSjRTSjoLzvfKGYG0eVIoffgn0oSUB9OBQu+UY3MFk8xeM5s1YztpBzfdnFYC0EVSOPCAur90fwHqlGu7VmAKJvXjxcuJMn2qGInQcLgBoOTuVF5ekDw2wr/pNZADOYUOdGH51p/8eptuu/rKYjcv
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(39860400002)(396003)(346002)(376002)(136003)(4744005)(66476007)(83380400001)(2906002)(8676002)(8936002)(66946007)(66556008)(5660300002)(31686004)(107886003)(16576012)(31696002)(6666004)(36756003)(2616005)(54906003)(55236004)(478600001)(316002)(38100700002)(956004)(186003)(6486002)(86362001)(26005)(53546011)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VkFSQjlFMjVOZ1lreVF5Y3ZqQ0lUR0owZVEyUlg3MnpTbDdQM2JYRWdodnlN?=
 =?utf-8?B?dEh3emtrWVc4OFBwTXhvZ28zcUljdlVQdTV0S0NtaTBmRFo0b3RHbitGaW9m?=
 =?utf-8?B?S3ZUOVgyNDhueWUyNHBzVDZYZUNadkk5eXNCS3JWQzA1bkxwZmpUbVZtSGdv?=
 =?utf-8?B?djNyU2lnZ3BUN2xrTVM3Ly9hNXV1SjBPajdJM2RQU1pYWWc2R1ord2g0ZDNj?=
 =?utf-8?B?YVMwelY4cWdVbEw3elU1N3JvU0taT1RwRk5oWG9tZkFTUFNCTkhUS0E1MlRt?=
 =?utf-8?B?RDNXRmlrd0N5cjBOMnBoMVVGS2lwZ3ZnZmRoVm5tRkRuVHlFd2QxNDFIWDRV?=
 =?utf-8?B?Wkw3TUI3Znd0UWJNUHZwbTJia2p4bXZyUTdzcWJlcWRxN1pOZnVwNUREdGt2?=
 =?utf-8?B?Y1FBU2xmMTBkeU1LcllndnZDS0ZCNUo1Z2I2ejZQcWZoN29pV3Z4YVJVTjNv?=
 =?utf-8?B?ak9Yd1BzMlpETGlDNWtIT0IrMWkrMUl3SGpkY0FFdHZaYVcyMkdCUW5rS1hT?=
 =?utf-8?B?QXZaTzdKSFVSZzdSMzlDMXl3UzNuVWxYemsxRnpOREZiQWZtaHgxd290bkV0?=
 =?utf-8?B?L2hyY3lXZGhzbnJCTDhZREIxN2FwaS90RkNmdzUyVUYvMUVpYXRWK0Q3Q3g2?=
 =?utf-8?B?d2VraDk0b0dJQktsNkNVakhzdlo1N2orQVIyc1FMWDArQVZCdGpGK0x6b24r?=
 =?utf-8?B?aDMzWTBkNDNlZytKN2JmUzFmbWdLdVY1NTFDQ25EaWhIblFRKzk0b3V3YjJQ?=
 =?utf-8?B?cFpwVTJPVnhBSTc1bVVOY2diSU1EcFljWVZ0Y3ZCSWRrcUM1THE0T2N3a0t2?=
 =?utf-8?B?dmZIM1Zwb3lNZ3VYcHdDN0EwejJ3NzFCRnovZ1hXd25rZk04OVV6OHN5dklj?=
 =?utf-8?B?RWdZNnZhb3RXN0NRbkNwQS85TGFKQWRyUUxwR1FlODFZK1AzSHlsQnU1N1dl?=
 =?utf-8?B?azdDcUJaaUc0THNZelpxRFJUSmdwd2JFTm00UnlhK0JNbTgwcy9WY1JNQzIz?=
 =?utf-8?B?eENsSHBZMjlWZVl1aTQ0Z2pnN2s2R29COHQzL1dVK0pDSWlDdUFHVnFVWi9O?=
 =?utf-8?B?L2VhblVQaHV2RHRkc0doY05HYW16eHZxejJpMXMxNHlLOEVLcW5mQllyOFVY?=
 =?utf-8?B?dzNtU3ZGNktlcW5KRVU3SnFsRjQrZ3lZSVhoak85YjIwNXgzckxTMXYxbW1O?=
 =?utf-8?B?c2xHMGZLTWhiR1h1MlZWRTliS0w1MjdOT0pkbEJ1Nk91aTJ0bloxeTVZSTAr?=
 =?utf-8?B?b3hUQ2pmWTRtZmhqM0Zyc1I0dTAvbDJMdVNnZmMzekc4ZVVlRVVLdElxcCtn?=
 =?utf-8?B?RS9BOVl2dDFRcTZJS2x1dUtrcmNrSUN2eFdCSjZENFdNb00rZGR2ZSswaUlY?=
 =?utf-8?B?UVdFbGpEdVR3MGZSUjJhZVU3dFp2TEZKenFPMW4wYkQwQ3lza3hrOC8vazVF?=
 =?utf-8?B?NkhYMlk1bWxpMExMb3piM1pZUC9lQ1B3cHpTeENIMUg4Y1F2TnVPTGtUby9H?=
 =?utf-8?B?eFhCaXN6ck95V2tQV3ZRbkNZM0xwdGJMcWppVFE2dnpDQlg1d3ZaU201aUxX?=
 =?utf-8?B?SlhIdjZCMTFhemk3enlyUjQ0UXdVSGVqNld4NUlWYm9FR2laUHZYc0h5RVc2?=
 =?utf-8?B?YmphVE5XR2UvWXoxOTY1eXNQQXlhZDVTcTRCeE9TSlBuTk9uaWNsRzVBM2RS?=
 =?utf-8?B?VCs0cGx6ZjgwbEwyK2ZyZjY0NitmNUQvTlduSW1yUThXUFlER1R1Qi9mbThv?=
 =?utf-8?Q?Pr82dw0aXxI4Q+07sRWbh1/o+e/ML6OGVmgTpxc?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6221555b-8e66-4d15-1bcf-08d9716c590b
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 19:27:21.5631
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LXgfyeaaLSrj5imdTwnouNIIU4vfJ/FkHQEhEtoipGOtncKN4khFUcCbJ/dZTeiCBqD9VF42AJn7kJ17McBvxAteXn99IMbrBnVzzpno1Ms=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5000
X-OriginatorOrg: citrix.com

On 06/09/2021 01:06, Bhasker C V wrote:
> Hi,
>  I have a domU and that domU has network vf attached to it.
>  The save and restore leads to crash of the domU after restore.
>  Am I doing anything wrong?

xl save/restore really ought to give up early and refuse the operation. 
CC-ing the toolstack maintainers.

It is an error that it continues and creates a corrupt vm image.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 19:54:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 19:54:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180255.326831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKgj-0008QE-4T; Mon, 06 Sep 2021 19:53:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180255.326831; Mon, 06 Sep 2021 19:53:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNKgj-0008Q7-0Q; Mon, 06 Sep 2021 19:53:53 +0000
Received: by outflank-mailman (input) for mailman id 180255;
 Mon, 06 Sep 2021 19:53:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4nFA=N4=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNKgi-0008Q1-1G
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 19:53:52 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 27b88fce-0f4c-11ec-b0b8-12813bfff9fa;
 Mon, 06 Sep 2021 19:53:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27b88fce-0f4c-11ec-b0b8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1630958030;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=bwrJ/82EidgVnL+FTnzOnWMM8fNlJ/yK8puMT+f780s=;
  b=KrXajhqxGyp0g8XIpCYkNwcTJi6ju0oK7+cXZv8WebSwOI2D42ewF6RW
   jhz1FIy6hGJ31PBA+Z25QHd+L30CAU+cBZzn09obS3wBqsnlpa1Y+WzcW
   904UzDI6QY1r5cAnbvIO78BKUOQibvxWzuTgakbENuQXIaAPp3q+OGfgb
   g=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: IOkwJFIQ3Kn6ZeoXcmOr5/oyU+86gutetzvZrJ3XG7s20uOYXKsj8TpuI9E9+MtUzwF8UreioB
 ZeViExfhPgtal+nt+SAvgEjEr4vgVAKT+cGVLK2l3WE91a38fCP6GWyo+Vj6u+pXJhYxbBrxjl
 iOBsIAVXJANghcWmTZiSMXVCaBHVIodRGmBV9FjC6XFGbjhtNl5lRIOHMmhvpAqDbgJ7Ji+L3F
 Id1bvtuBhGUTdnqOxYDugAAdeAcZu8KcvlE33enX8kK2fg1I442PRMjvOttYHv+WrxRMF2tUbV
 /2d8jgPw3njOIm8R2OKVbXU8
X-SBRS: 5.1
X-MesageID: 52509495
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:kKuF+61PuK2X2RSyQlVg2gqjBTtyeYIsimQD101hICG9Lfb3qy
 n+ppsmPEHP5Ar5OEtBpTiBUJPwJU80hqQFn7X5Wo3SIzUO2VHYUL2KiLGC/9SOIVyEygcw79
 YHT0E6MqyMMbEYt7eI3ODbKadZ/DDvysnB7o2yvhQdL3AZV0gj1XYfNu/yKDwHeOAsP+tBKH
 Pz3Lsjm9PtQwVsUiztbUN1LtQr6ue7267OUFojPVoK+QOOhTSn5PrTFAWZ5A4XV3dqza05+W
 bIvgTl7uH72svLiyP05iv21dB7idHhwtxMCIiljdUUECzljkKNaJ56U7OPkTgpqKWE6Uoskv
 PLvxA8Vv4DpU/5TyWQm1/AygPg2DEh5zvLzkKZu2LqpYjDSDczG6N69MhkWyqcz3BlkMB30a
 pN0W7cnYFQFwn8kCP04MWNfw12l2KvyEBS09I7vjh6a88zebVRpYsQ8Ad+C5EbBh/374ghDa
 1HENzc3vBLalmXBkqp/VWH+ObcGkjbIy32BXTr4qeuon5rdTFCvgslLfUk7zI9HMlXcegc2w
 zGWp4Y342mAPVmNZ6UqY86ML2K41f2MGbx2VSpUBza/ZE8SgfwQqHMkcIIDcGRCdE1JcgJ6d
 j8uG0xjx96R6upM7zU4KF2
X-IronPort-AV: E=Sophos;i="5.85,273,1624334400"; 
   d="scan'208";a="52509495"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PzjeqbfEaMkqdKFIf6Z7afvOHn8asoEZvQbT6Rck0lmMWwji3P1hHQhstQDSyuJ6gk+mWx9GyxsPKWGFh1cy8t9rqjSRDCN4KX+E77Yvy9nBb43mZnDEZ6MHSEOwhG2OkdZZth6NO+B0Ml0NVVZjs2IaN7/35yZHYrVSLyeLM3GvKoQJdqm+rfF4SvGW4a94UrUt6SsgES/tuxBAVr6L8JR0fpUwDOyZJ+6Pqe05wu+Xd65z2Iqsxpf7kdn7W1AhsU6tUdR+2Gknym/gLnig6B4M4dkj3d7Cvxl74WOG9z+KS84DY+qTHbtckzxVcXyQlWCI2ibmWiKxivbcfPE06w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JUF5ZQ9qHhiOdd1nZnSVwMywJawM174/ekfRo7ZYZ6M=;
 b=F+ngGyUtzO8vsg4gG2Fd07aR9CD/S0N+tGkS0MBeuHEIVagmSm75qcFAkprVXJkiryOK0eAByo94Otc1gaqXAtvr9WIT7fM4VNFtXOhrouxYYhk0BdF6q3iVpSWXoLOAyhdfruQfT9m84eS94S+p1ct1A+r72ziJNzrSenIJ5Na9FmEOcu5sCdj8gI4BSWC+nJRqSxwVXZJFSFdyzlSsUDtgcDnhCT9N1OQxQq5i3kxmT67ts+pxUkCh7P2aeZah2z7KYt7vNleQ2Z3FYgG2fEjVwDjPoNgeg4gnopD19kpm6OKqdGbIqMhojFLKdceonSa0URrNfRcUcAMv+XSsbg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JUF5ZQ9qHhiOdd1nZnSVwMywJawM174/ekfRo7ZYZ6M=;
 b=WkJtN2bBrw7IT0pRt35GuXahFfBbABdQ3dPOuQHj4B4Bua/yBp6P4aMD+/vXur3YqN56tsnN8bNAapiAKuf610y6zdX2H/NGOQW3D9EGzB26hS1/EUkx1INe88Vd1nVfuDqKFaXuZripmoSovovvFQmYZPG80HRiFj/ux9OhZBY=
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <62cb78c9-f4fd-76e7-225d-bc575f10850f@citrix.com>
 <818cc08e-8372-35c4-935f-86a87cceec75@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
Message-ID: <bf091ebc-95e7-b357-9f84-4ec3a6346fac@citrix.com>
Date: Mon, 6 Sep 2021 20:53:38 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <818cc08e-8372-35c4-935f-86a87cceec75@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0042.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::30)
 To BYAPR03MB3623.namprd03.prod.outlook.com (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c14ef7f8-3948-48fc-e7e9-08d971700853
X-MS-TrafficTypeDiagnostic: BYAPR03MB4120:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB41209150C37D0F0F3D882C5EBAD29@BYAPR03MB4120.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: eBVRbVfcZPgqWF2GZ6GZTrfb2LM9BcDMwWqtR6fjSXyOnHyWaBoCzwAoIDwyY9UY6L2sWPfPkHjpmNpFZnpuUjCibCI/eVfkgnn7q5+lG3uGIK5SEgZXUeKkKhRBtP76ikCAxt9nePVoMsSftJV0xWOhgg2NoPlRWcbI5mjjTAQscQa5A7tkltqT1YEN1G5W9xXUVCAbkxjFzsE1N3v4LJuWLfwOAsLQduv5s23r5qWEmLeM0YUmE2hseSqRiRao12iqB/9czxxuL7AJnfV43Pd6yCM3KSBi0oY/2xCa6ID+qb3gj/Zr3cQwlSzmF8Xj5lP5UpMsRpv7RML3y/sPDNnb3a7HEHiQ4rO9+xvES1eVnZefNpXTqHuQ/6EAjQDinlbgicptONFA37QSytCQMn49WNa11aeQKvIOTi7hHh4Ji50nknmiOAVdYMm+Cvu8r9eux9+8AexNRy5aLrZQZVd/CZ+hHHQPay2V2kHqliNQDgj9YGJiArwZEuDLyMk8gFS5jllWV2LsIphm/zd+Mnfa/cwXviIHtrsVVKps9v7hj8mCoFlFiAtMTv+AD65eV1opkZsMhcLseNv/K2MTtlYREaDNGdy2M4x1rF0UcyicWe0glXF2rMBw4KqeNJk/ts+vIgLUmZsja5C2lTguTmv1CfIuA4bztqGRdlF6fk2On9EebPQHzcpTtVUo/YHGAJT7nEt8Q96gFzWTFxGk4C0BnfqF8jq4Tm6yZMclKdhNMiwDh6ZtnlXif+Zf5Pjl
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(366004)(346002)(376002)(39860400002)(6666004)(55236004)(2616005)(16576012)(38100700002)(478600001)(26005)(6916009)(2906002)(6486002)(83380400001)(36756003)(5660300002)(86362001)(31686004)(53546011)(66556008)(66476007)(66946007)(31696002)(4326008)(8936002)(186003)(316002)(956004)(8676002)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dGFWdDhOZG8rVHpvYXFxcVlTeXMxL1QxTGFJS0RGQWVIbW4vUE5COTE2NXFQ?=
 =?utf-8?B?Wkg3aGsrWkJvclNrRERkU1VkamhrRTIyR09oQ0M3N01TaTh0NEdKTmJpeGZ2?=
 =?utf-8?B?dTJsMVdNMDhuaXU2V0Y5Y2pUdmkxbWllR3JhM0c0aGZhN01QbHRzYytmY0RK?=
 =?utf-8?B?WlNmbFA3SklZV2MvbVo2b3dQbXcvQ0liREt5MXNHVUdyNlhJbnBhcXpwSHJS?=
 =?utf-8?B?TnV1YU1mMXovQmwvZVd4elk2eHFIbVFjRGtBRitidmNHRFp0YlVoZE85WDVK?=
 =?utf-8?B?NXVQVzFtN3lYc1dqVjhJSTk1SWMzakRwUEFvazVZaXE4R3psc0J3QWRRSEMy?=
 =?utf-8?B?WmxUVWYwWi9oUXI2K04zbG9VU1dRUWdsUWxlUEhwU0V0M1pIa1VJbDFGc1h2?=
 =?utf-8?B?Y01YTmExV3NKemlsVVVUQ2ZVc01qQW4zNWdkS3dFZ0Q3K0grdDJ0WS9LNFFz?=
 =?utf-8?B?bVJJTnVZakc1ZEtCMUE5ZmZ0WHU3N0pKVVcxWU9aZkd6ZXFMMHpwR3VRNGpB?=
 =?utf-8?B?YjlGblFvajhzcmdZL0tQREx4Qm9tbnpGeW9XRm85YUt3U3hDakVST3Nqc2Vh?=
 =?utf-8?B?eWFvdm9JbloxZXo4cjdWTVdqYm5ORlUzd1A3anN6TFdHSTR2a1R4akFRZkZ0?=
 =?utf-8?B?d1k3TUpOTFdTTktrZ1RnbWwxZnZxM1FCZkZJWWR3MlJDNXN0NUZWMkx0T2ZE?=
 =?utf-8?B?L1I4L1hiOGN5dnZLajlmZDlnMWpDSDNEQnVjU0dvK2w2TFZUNFllNDdjVnV6?=
 =?utf-8?B?T0J6RVIxMXRQb3Nvd0RPU1hvSzI3ZE9rVlhiU1JrSkhCcmREN2xRRWlYQ1Nv?=
 =?utf-8?B?MjIxV0JqcEgrdi84NjFiT25tL0UrU0tWRVkyUVhVNFFvakJxdTJzSDMwMkFE?=
 =?utf-8?B?NnpicW9xTnplSHVYVXFHcEswdmI5dmdlcFBDQi9kSGRwSkNlLzVrSERiV1F3?=
 =?utf-8?B?NUtsZmpTdjFtZFYyeU1hcG81MG5WVGlvMXlkcHAxQXlEZDN4Z2FNZG9OT08x?=
 =?utf-8?B?NEQrK1JCQ045bUY0azlpUUFPcUtSa2pFNkpxM1lkVHE0S2R2MEVmdEZNaXZS?=
 =?utf-8?B?OFVmQUxWNklwbzZ6V0NVSUQzaWdsclpLK0NRdXZ5MzlKbWRHNEVEb3gzUXpM?=
 =?utf-8?B?R3BqNUZvQlpUUkxWaHZtRU44ZjFTenUrMDh0YzlXNFcrQitxVEJ1eGIweWRm?=
 =?utf-8?B?UmpUaDVuVXI4eWcwcENiUXFHM0hwRjk0dUlJQW5UZ2JQbjRDL0I3QkVQTmgw?=
 =?utf-8?B?cXNaSXdGRlA1T2s3RktPcnZocjlMMXpqRmkyUzBROUpNVlRNM2hMTE83YTA4?=
 =?utf-8?B?UUpXdGVpencxZTMyM0dlcHlrODlES2ZzUkxEVzJlYVVaRjJ1T1BYV2lWOHpv?=
 =?utf-8?B?dEJCeEtmeS9KSkMvenVpTmJRQVMyay8yQkZIeW1rVmpuK0cya2hWVEhrM2NX?=
 =?utf-8?B?b21PbThCUXAvZk5XOXNoT3NFWmRMQnBSdmRrOXF1SXFtSlpnWlVDYWZ4RFIy?=
 =?utf-8?B?YWRCS3oxT0NkbVk0YTNxa0RuR3hkcEM2REo2aWFHbXI0VWlSUTBEYTBKVStT?=
 =?utf-8?B?OUZOdFZTbGtKWnBucHFWbXBiUWlJaEJ4VHowL3ZGNXhaeDF6V1UzcHFSVnRm?=
 =?utf-8?B?Kys2bEZGYmIrS0Y3eGRiTk91Qk1hSTJHNWVkc216N1J3VUxOc2hJaHFmRkRH?=
 =?utf-8?B?ek5mNkZNZkdzK0p5T3RoWlJPKy9iMHdnYW1aZEJ6SlZzZlFHaFNGaUdjUnhG?=
 =?utf-8?Q?Iwj4mlC7kZS19pPzIHLowvXmVDq6WuOp3Vi6Qko?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c14ef7f8-3948-48fc-e7e9-08d971700853
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 19:53:44.2056
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: R21Ot0oL9zY59GAHlnBrvjYNmaWlI6fnETb5ACFBroFhixRBjxho9//pCEF6Cl+X5y8fMLnX2hacOdvM6MJwMa71Emr5PYjqOq0lNtD+XpM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4120
X-OriginatorOrg: citrix.com

On 01/09/2021 14:08, Jan Beulich wrote:
>>>> Restricting execute permissions is something unique to virt.=C2=A0 It =
doesn't
>>>> exist in a non-virtualised system, as I and D side reads are
>>>> indistinguishable outside of the core.
>>>>
>>>> Furthermore, it is inexpressible on some systems/configurations.
>>>>
>>>> Introspection is the only technology which should be restricting execu=
te
>>>> permissions in the p2m, and only when it takes responsibility for
>>>> dealing with the fallout.
>>> IOW are you saying that the calls to set_identity_p2m_entry()
>>> (pre-dating XSA-378) were wrong to use p2m_access_rw?
>> Yes.
>>
>>>  Because that's
>>> what's getting the way here.
>> On a real machine, you really can write some executable code into an
>> E820 reserved region and jump to it.=C2=A0 You can also execute code fro=
m
>> real BARs is you happen to know that they are prefetchable (or you're a
>> glutton for UC reads...)
>>
>> And there is the WPBT ACPI table which exists specifically to let
>> firmware inject drivers/applications into a windows environment, and may
>> come out of the SPI ROM in the first place.
>>
>>
>> Is it sensible to execute an E820 reserved region, or unmarked BAR?=C2=
=A0
>> Probably not.
>>
>> Should it work, because that's how real hardware behaves?=C2=A0 Absolute=
ly.
>>
>> Any restrictions beyond that want handling by some kind of introspection
>> agent which has a policy of what to do with legal-but-dodgy-looking acti=
ons.
> IOW you suggest we remove p2m_access_t parameters from various functions,
> going with just default access?

p2m_access_t was very obviously a bodge when introduced, and I doubt it
would pass today's review standards.

It is definitely a mis-design, given its ill-specified and overlapping
semantics with respect to the p2m type.

>  Of course, as pointed out in another
> reply, we'll need to split p2m_mmio_direct into multiple types then, at
> the very least to honor the potential r/o restriction of AMD IOMMU unity
> mapped regions. (FAOD all of this isn't a short term plan anyway, at leas=
t
> afaic.)

I don't think that will be necessary.

IVMDs are almost non-existent, and given how many other areas of the AMD
IOMMU spec are totally unused, I wouldn't be surprised if r/o unity
mappings were in that category too.=C2=A0 There's no obvious usecase for r/=
o,
as anything critical enough in the platform to have an IVMD in the first
place will also be non-trivial enough to require bidirectional
communication somehow.

The unity mapping only says "this device requires read-only access".=C2=A0 =
It
doesn't say "this must be mapped read-only", and it is legitimate to map
a r/o unity mapping as r/w.

If such a case actually exists, there's clearly one agent in the system
with r/w access into the r/o range, and mapping it r/w is equivalent to
the "IOMMU not enabled in the first place" case which is the default
case for most software for the past decade-and-a-bit.

In other words, I don't think the r/o unit maps on their own are a good
enough reasons to split the type.=C2=A0 If we gain other reasons then fine,
but this seems like chunk of complexity with no real users.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 06 20:47:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 20:47:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180262.326841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNLWl-00068Z-8F; Mon, 06 Sep 2021 20:47:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180262.326841; Mon, 06 Sep 2021 20:47:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNLWl-00068S-5F; Mon, 06 Sep 2021 20:47:39 +0000
Received: by outflank-mailman (input) for mailman id 180262;
 Mon, 06 Sep 2021 20:47:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=BJbD=N4=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mNLWj-00068K-VM
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 20:47:37 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e44ad8c3-e821-4c62-9ee6-fe2b280bd656;
 Mon, 06 Sep 2021 20:47:36 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 186KlQcf066300
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Mon, 6 Sep 2021 16:47:32 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 186KlQdT066299;
 Mon, 6 Sep 2021 13:47:26 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e44ad8c3-e821-4c62-9ee6-fe2b280bd656
Date: Mon, 6 Sep 2021 13:47:26 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Mon, Sep 06, 2021 at 09:52:17AM +0200, Jan Beulich wrote:
> On 06.09.2021 00:10, Elliott Mitchell wrote:
> > I brought this up a while back, but it still appears to be present and
> > the latest observations appear rather serious.
> > 
> > I'm unsure of the entire set of conditions for reproduction.
> > 
> > Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
> > this is an older AMD IOMMU).
> > 
> > This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
> > Debian's patches, but those are mostly backports or environment
> > adjustments.
> > 
> > Domain 0 is presently using a 4.19 kernel.
> > 
> > The trigger is creating a HVM or PVH domain where memory does not equal
> > maxmem.
> 
> I take it you refer to "[PATCH] x86/pod: Do not fragment PoD memory
> allocations" submitted very early this year? There you said the issue
> was with a guest's maxmem exceeding host memory size. Here you seem to
> be talking of PoD in its normal form of use. Personally I uses this
> all the time (unless enabling PCI pass-through for a guest, for being
> incompatible). I've not observed any badness as severe as you've
> described.

I've got very little idea what is occurring as I'm expecting to be doing
ARM debugging, not x86 debugging.

I was starting to wonder whether this was widespread or not.  As such I
was reporting the factors which might be different in my environment.

The one which sticks out is the computer has an older AMD processor (you
a 100% Intel shop?).  The processor has the AMD NPT feature, but a very
early/limited IOMMU (according to Linux "AMD IOMMUv2 functionality not
available").

Xen 4.14 refused to load the Domain 0 kernel as PVH (not enough of an
IOMMU).


There is also the possibility Debian added a bad patch, but that seems
improbable as there aren't enough bug reports.


> > New observations:
> > 
> > I discovered this occurs with PVH domains in addition to HVM ones.
> > 
> > I got PVH GRUB operational.  PVH GRUB appeared at to operate normally
> > and not trigger the crash/panic.
> > 
> > The crash/panic occurred some number of seconds after the Linux kernel
> > was loaded.
> > 
> > 
> > Mitigation by not using ballooning with HVM/PVH is workable, but this is
> > quite a large mine in the configuration.
> > 
> > I'm wondering if perhaps it is actually the Linux kernel in Domain 0
> > which is panicing.
> > 
> > The crash/panic occurring AFTER the main kernel loads suggests some
> > action by the user domain is doing is the actual trigger of the
> > crash/panic.
> 
> All of this is pretty vague: If you don't even know what component it
> is that crashes / panics, I don't suppose you have any logs. Yet what
> do you expect us to do without any technical detail?

Initially this had looked so spectacular as to be easy to reproduce.

No logs, I wasn't expecting to be doing hardware-level debugging on x86.
I've got several USB to TTL-serial cables (ARM/MIPS debug), I may need to
hunt a USB to full voltage EIA-232C cable.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Mon Sep 06 21:14:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 21:14:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180271.326853 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNLx2-0001aJ-G9; Mon, 06 Sep 2021 21:14:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180271.326853; Mon, 06 Sep 2021 21:14:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNLx2-0001aC-DA; Mon, 06 Sep 2021 21:14:48 +0000
Received: by outflank-mailman (input) for mailman id 180271;
 Mon, 06 Sep 2021 21:14:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNLx0-0001a2-PB; Mon, 06 Sep 2021 21:14:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNLx0-00050h-H5; Mon, 06 Sep 2021 21:14:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNLx0-00032T-8G; Mon, 06 Sep 2021 21:14:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNLx0-0003Fd-7m; Mon, 06 Sep 2021 21:14:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2pzxwQvyzuFHN1hB7tBsMg7NU+mnhOSdl6abd4CKkaY=; b=0XnAZlbfTEX8E203RPF3hjrDyn
	wv6zEfi3vCqFhx5rpPH3P9Pd4eTzt/s7Rom3tBI1CwZr+vJxpFdth3LM4vrldMIlm34L8NfxBgE1z
	OGQshb6JwP46JH5sjL6AYDPKmALSlWnqc1qUuRxp5S4AqXJ6CNxItopMCwun4u/r9bkM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164860-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164860: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:guest-start.2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=27151f177827d478508e756c7657273261aaf8a9
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 21:14:46 +0000

flight 164860 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164860/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds     19 guest-start.2           fail blocked in 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                27151f177827d478508e756c7657273261aaf8a9
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  402 days
Failing since        152366  2020-08-01 20:49:34 Z  401 days  697 attempts
Testing same since   164850  2021-09-06 02:37:34 Z    0 days    2 attempts

------------------------------------------------------------
7219 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2221077 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 21:35:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 21:35:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180281.326867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNMH3-0004Qa-Bi; Mon, 06 Sep 2021 21:35:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180281.326867; Mon, 06 Sep 2021 21:35:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNMH3-0004QT-8K; Mon, 06 Sep 2021 21:35:29 +0000
Received: by outflank-mailman (input) for mailman id 180281;
 Mon, 06 Sep 2021 21:35:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=k3uM=N4=eikelenboom.it=linux@srs-us1.protection.inumbo.net>)
 id 1mNMH2-0004QN-A6
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 21:35:28 +0000
Received: from server.eikelenboom.it (unknown [91.121.65.215])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d167496c-c553-4b02-b94b-d68a6f126275;
 Mon, 06 Sep 2021 21:35:25 +0000 (UTC)
Received: from 76-24-144-85.ftth.glasoperator.nl ([85.144.24.76]:59648
 helo=[172.16.1.50]) by server.eikelenboom.it with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <linux@eikelenboom.it>) id 1mNMGn-0003UQ-35
 for xen-devel@lists.xenproject.org; Mon, 06 Sep 2021 23:35:13 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d167496c-c553-4b02-b94b-d68a6f126275
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=eikelenboom.it; s=20180706; h=Content-Transfer-Encoding:Content-Type:
	MIME-Version:Date:Message-ID:To:Subject:From:Sender:Reply-To:Cc:Content-ID:
	Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
	:Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:
	List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=p2NdIgEhC3QeQFfY0U3KvL+PnROpHFsvtadWOxPAqvA=; b=RmMZnXucFtIIpaT/DJfQ7pYpcp
	dbojjmoySx/+QdSwQpAzVMKKElcG6gZ2roHByYsIK2lokmHryAduwES2NXaSn/qet1Bfz/Q3f9jq2
	wH83yrlPCcLmjaMLSoMrId+UW1Xek6tq/DPxISb9SnC5Dyy6RHtHge792hQl2gzFgrO0=;
From: Sander Eikelenboom <linux@eikelenboom.it>
Subject: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
Date: Mon, 6 Sep 2021 23:35:23 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: nl-NL
Content-Transfer-Encoding: 7bit

L.S.,

On my AMD box running:
     xen-unstable changeset: Fri Sep 3 15:10:43 2021 +0200 git:2d4978ead4
     linux kernel: 5.14.1

With this setup I'm encountering some issues in dom0, see below.

--
Sander

xl dmesg gives:

(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63b936 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a0622 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6b63da already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 638dd9 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 68a7bc already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63c27d already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a04f2 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 690d49 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6959a0 already pinned
(XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a055e already pinned
(XEN) [2021-09-06 18:15:04.090] mm.c:3506:d0v0 mfn 639437 already pinned


dmesg gives:

[34321.304270] ------------[ cut here ]------------
[34321.304277] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
[34321.304288] Modules linked in:
[34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted 5.14.1-20210906-doflr-mac80211debug+ #1
[34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1 09/13/2010
[34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
[34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
[34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
[34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807d416398
[34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefdeadf00d
[34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff888006160cc0
[34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12: 0000000000000000
[34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15: 0000000000000001
[34321.304320] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlGS:0000000000000000
[34321.304322] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 0000000000000660
[34321.304326] Call Trace:
[34321.304331]  xen_alloc_pte+0x294/0x320
[34321.304334]  move_pgt_entry+0x165/0x4b0
[34321.304339]  move_page_tables+0x6fa/0x8d0
[34321.304342]  move_vma.isra.44+0x138/0x500
[34321.304345]  __x64_sys_mremap+0x296/0x410
[34321.304348]  do_syscall_64+0x3a/0x80
[34321.304352]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[34321.304355] RIP: 0033:0x7f507196301a
[34321.304358] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
[34321.304360] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000000000000019
[34321.304362] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507196301a
[34321.304363] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506dc56000
[34321.304364] RBP: 0000000001a00000 R08: 0000000000000010 R09: 0000000000000004
[34321.304365] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506dc56060
[34321.304367] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205f950f30
[34321.304368] ---[ end trace a19885b78fe8f33e ]---
[34321.304370] 1 of 2 multicall(s) failed: cpu 0
[34321.304371]   call  2: op=12297829382473034410 arg=[aaaaaaaaaaaaaaaa] result=-22
[34321.304398] ------------[ cut here ]------------
[34321.304399] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
[34321.304403] Modules linked in:
[34321.304404] CPU: 0 PID: 23628 Comm: apt-get Tainted: G        W         5.14.1-20210906-doflr-mac80211debug+ #1
[34321.304406] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1 09/13/2010
[34321.304407] RIP: e030:xen_mc_flush+0x176/0x1a0
[34321.304410] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
[34321.304411] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
[34321.304413] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807d416398
[34321.304414] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefdeadf00d
[34321.304415] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff888006160cc0
[34321.304417] R10: deadbeefdeadf00d R11: ffffc90000a97ab8 R12: 0000000000000000
[34321.304418] R13: ffff888009e6a000 R14: 0000000009e6a000 R15: 0000000000000001
[34321.304422] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlGS:0000000000000000
[34321.304423] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[34321.304425] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 0000000000000660
[34321.304427] Call Trace:
[34321.304428]  xen_alloc_pte+0x294/0x320
[34321.304431]  move_pgt_entry+0x165/0x4b0
[34321.304433]  move_page_tables+0x6fa/0x8d0
[34321.304436]  move_vma.isra.44+0x138/0x500
[34321.304439]  __x64_sys_mremap+0x296/0x410
[34321.304442]  do_syscall_64+0x3a/0x80
[34321.304444]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[34321.304446] RIP: 0033:0x7f507196301a
[34321.304448] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
[34321.304449] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000000000000019
[34321.304451] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507196301a
[34321.304452] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506dc56000
[34321.304453] RBP: 0000000001a00000 R08: 0000000000000010 R09: 0000000000000004
[34321.304455] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506dc56060
[34321.304456] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205f950f30
[34321.304457] ---[ end trace a19885b78fe8f33f ]---
[34321.304458] 1 of 2 multicall(s) failed: cpu 0
[34321.304459]   call  2: op=12297829382473034410 arg=[aaaaaaaaaaaaaaaa] result=-22
[34321.304477] ------------[ cut here ]------------
[34321.304478] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
[34321.304482] Modules linked in:
[34321.304483] CPU: 0 PID: 23628 Comm: apt-get Tainted: G        W         5.14.1-20210906-doflr-mac80211debug+ #1
[34321.304485] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1 09/13/2010
[34321.304486] RIP: e030:xen_mc_flush+0x176/0x1a0
[34321.304488] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
[34321.304490] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
[34321.304491] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807d416398
[34321.304493] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefdeadf00d
[34321.304494] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff888006160cc0
[34321.304495] R10: deadbeefdeadf00d R11: ffffc90000a97ab8 R12: 0000000000000000
[34321.304496] R13: ffff88801833e000 R14: 000000001833e000 R15: 0000000000000001
[34321.304500] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlGS:0000000000000000
[34321.304502] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[34321.304503] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 0000000000000660
[34321.304505] Call Trace:
[34321.304506]  xen_alloc_pte+0x294/0x320
[34321.304509]  move_pgt_entry+0x165/0x4b0
[34321.304511]  move_page_tables+0x6fa/0x8d0
[34321.304514]  move_vma.isra.44+0x138/0x500
[34321.304516]  __x64_sys_mremap+0x296/0x410
[34321.304519]  do_syscall_64+0x3a/0x80
[34321.304521]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[34321.304523] RIP: 0033:0x7f507196301a
[34321.304525] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
[34321.304526] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000000000000019
[34321.304528] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507196301a
[34321.304529] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506dc56000
[34321.304530] RBP: 0000000001a00000 R08: 0000000000000010 R09: 0000000000000004
[34321.304531] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506dc56060
[34321.304533] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205f950f30
[34321.304534] ---[ end trace a19885b78fe8f340 ]---
[34321.304535] 1 of 2 multicall(s) failed: cpu 0
[34321.304536]   call  2: op=12297829382473034410 arg=[aaaaaaaaaaaaaaaa] result=-22


From xen-devel-bounces@lists.xenproject.org Mon Sep 06 22:45:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 06 Sep 2021 22:45:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180291.326878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNNMI-0004B3-Di; Mon, 06 Sep 2021 22:44:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180291.326878; Mon, 06 Sep 2021 22:44:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNNMI-0004Aw-Aq; Mon, 06 Sep 2021 22:44:58 +0000
Received: by outflank-mailman (input) for mailman id 180291;
 Mon, 06 Sep 2021 22:44:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNNMH-0004Am-I0; Mon, 06 Sep 2021 22:44:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNNMH-0006SZ-AN; Mon, 06 Sep 2021 22:44:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNNMH-0007Ow-0E; Mon, 06 Sep 2021 22:44:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNNMG-0000An-Vy; Mon, 06 Sep 2021 22:44:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=L9mpehhOwPie4jfUUdfv7cpPvpbBZ13JSHkykAadb4s=; b=j/ivdFpbgb2T8+IhJOv/nvuqV3
	iSwMEkMuXo+ZiiH/YjrcmPCzc6IIfBTyV8oM+8NAPxgK+XAAVME0wJKEhphKc4Vbsk6yIBNGswufp
	wjVxFyIxLHVuopACJkL/DWTPfc0dy925+qn3Nyq52GS6xmjZ105TLi0dbiAmHsHhxNcE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164863-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164863: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 06 Sep 2021 22:44:56 +0000

flight 164863 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164863/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164796  2021-09-03 14:01:38 Z    3 days
Testing same since   164863  2021-09-06 18:02:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Orzel <michal.orzel@arm.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2d4978ead4..6c27a8da8d  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 00:58:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 00:58:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180302.326892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNPR5-0002fs-CE; Tue, 07 Sep 2021 00:58:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180302.326892; Tue, 07 Sep 2021 00:58:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNPR5-0002fl-93; Tue, 07 Sep 2021 00:58:03 +0000
Received: by outflank-mailman (input) for mailman id 180302;
 Tue, 07 Sep 2021 00:58:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M3oQ=N5=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mNPR4-0002ff-1t
 for xen-devel@lists.xen.org; Tue, 07 Sep 2021 00:58:02 +0000
Received: from mail-oo1-xc2d.google.com (unknown [2607:f8b0:4864:20::c2d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6ad94e00-1f18-4281-9f6d-40b1bf59f82f;
 Tue, 07 Sep 2021 00:57:59 +0000 (UTC)
Received: by mail-oo1-xc2d.google.com with SMTP id
 y3-20020a4ab403000000b00290e2a52c71so2439885oon.2
 for <xen-devel@lists.xen.org>; Mon, 06 Sep 2021 17:57:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ad94e00-1f18-4281-9f6d-40b1bf59f82f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=VUq8hhhVF+UOcOQr157X5bZtPmm1X78HQGLXPKMFzuI=;
        b=GEJPVWizDy1/cHuxqsGCcbg4TXZyPYnuD2g4BhcY8nN+MtooHN44X6f+M15WWQeELk
         ZHxS6OcsHF8b1HPdRfPrF0lUe8C581xHWFOWapCJIP8aHnoh2ey9vULULhfumbo7LXAT
         GDviqgNRWwGILs8ON3yjB8FPK4bq4Hi+rBj1qjbLgkk61ZhZRn1NnD6Nd/KiOCIm5ZTC
         9QWRj/+vjWiOpxsKLgEDePZkeYbpkjTc6l36kXMd9zHsHuh1j4C2f5lCraLKARu9LJ1k
         D1F/za65wIi/G7aKw3Hss9BiMcAXuyaWlgCm51v4c9Oa/kRqeJDe+S0cQ893K3/ePIKq
         PTsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=VUq8hhhVF+UOcOQr157X5bZtPmm1X78HQGLXPKMFzuI=;
        b=YgIOaxgppSIFvy/s4gT6nXj1X2pb52KckWwceEN1plRGQCyigxEnkySZunRF/brX9O
         HV9oybniTOv9xgdadVkKUuCx8w9MSpC2DdpW/tp6RouPSG6lZROf1qkycF7A23DtEeOZ
         JCZvSYCYX7ZoLTY5HqrWHyK8tWJUW9f6bGV+4m8DziF8VVO3y42sQYD0hrEHJW0YDQNm
         CXvgsXissc7bPQ/QSoiFe++m+k56zTNWcOhYCDZwwcRYVcgcY1xIkG2vHqi46b0JWxxm
         HlCxaw5EYgyBPgTgMNnfxAM6iWgnq+4dite4koUDnqT6zd53sSVxyhxD9yD0Xcp01aMd
         zoLg==
X-Gm-Message-State: AOAM531PQAwSMAwFkoa6NyaCl+f6ZlBIEU4AYr7Fyhd/ryOwK8vVwobd
	yw+d7RrRZnz0CnZshXMWcakddqcNM0UGjq9OQmQ=
X-Google-Smtp-Source: ABdhPJzkmVgMjlkimhOnxYqmnzjr8MYlQDbviubm9j0fp6M02lYE+H2P8fCiYO93E0jJO8KWexrxbJ0mIeTaa254QNk=
X-Received: by 2002:a4a:e297:: with SMTP id k23mr9231450oot.4.1630976278325;
 Mon, 06 Sep 2021 17:57:58 -0700 (PDT)
MIME-Version: 1.0
References: <DB9PR08MB685767CFAA4A8BCE7D2225A89EFD9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210817080757.GC43203@laputa> <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa> <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CACMJ4GbmNgbB5ponYt3NGEk3j6YCksot+kDy2qs8HMdFXWnQbw@mail.gmail.com>
 <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com> <20210902071902.GC71098@laputa>
In-Reply-To: <20210902071902.GC71098@laputa>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Mon, 6 Sep 2021 17:57:43 -0700
Message-ID: <CACMJ4GYY0wkm0vOzJfq8M9dCxPq1ZFwWT8Q73rMjqtrP7if-Zw@mail.gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Wei Chen <Wei.Chen@arm.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, "Alex Benn??e" <alex.bennee@linaro.org>, 
	Kaly Xin <Kaly.Xin@arm.com>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	Viresh Kumar <viresh.kumar@linaro.org>, 
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com" <stefanha@redhat.com>, 
	Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>, 
	"pratikp@quicinc.com" <pratikp@quicinc.com>, Srivatsa Vaddagiri <vatsa@codeaurora.org>, 
	Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>, 
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, 
	Paul Durrant <paul@xen.org>, Xen Devel <xen-devel@lists.xen.org>, Rich Persaud <persaur@gmail.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, James McKenzie <james@bromium.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000b501cf05cb5d41dd"

--000000000000b501cf05cb5d41dd
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Sep 2, 2021 at 12:19 AM AKASHI Takahiro <takahiro.akashi@linaro.org=
>
wrote:

> Hi Christopher,
>
> Thank you for your feedback.
>
> On Mon, Aug 30, 2021 at 12:53:00PM -0700, Christopher Clark wrote:
> > [ resending message to ensure delivery to the CCd mailing lists
> > post-subscription ]
> >
> > Apologies for being late to this thread, but I hope to be able to
> > contribute to
> > this discussion in a meaningful way. I am grateful for the level of
> > interest in
> > this topic. I would like to draw your attention to Argo as a suitable
> > technology for development of VirtIO's hypervisor-agnostic interfaces.
> >
> > * Argo is an interdomain communication mechanism in Xen (on x86 and Arm=
)
> > that
> >   can send and receive hypervisor-mediated notifications and messages
> > between
> >   domains (VMs). [1] The hypervisor can enforce Mandatory Access Contro=
l
> > over
> >   all communication between domains. It is derived from the earlier v4v=
,
> > which
> >   has been deployed on millions of machines with the HP/Bromium uXen
> > hypervisor
> >   and with OpenXT.
> >
> > * Argo has a simple interface with a small number of operations that wa=
s
> >   designed for ease of integration into OS primitives on both Linux
> > (sockets)
> >   and Windows (ReadFile/WriteFile) [2].
> >     - A unikernel example of using it has also been developed for XTF.
> [3]
> >
> > * There has been recent discussion and support in the Xen community for
> > making
> >   revisions to the Argo interface to make it hypervisor-agnostic, and
> > support
> >   implementations of Argo on other hypervisors. This will enable a sing=
le
> >   interface for an OS kernel binary to use for inter-VM communication
> that
> > will
> >   work on multiple hypervisors -- this applies equally to both backends
> and
> >   frontend implementations. [4]
>
> Regarding virtio-over-Argo, let me ask a few questions:
> (In figure "Virtual device buffer access:Virtio+Argo" in [4])
>

(for ref, this diagram is from this document:
 https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698 )

Takahiro, thanks for reading the Virtio-Argo materials.

Some relevant context before answering your questions below: the Argo
request
interface from the hypervisor to a guest, which is currently exposed only
via a
dedicated hypercall op, has been discussed within the Xen community and is
open
to being changed in order to better enable support for guest VM access to
Argo
functions in a hypervisor-agnostic way.

The proposal is to allow hypervisors the option to implement and expose any
of
multiple access mechanisms for Argo, and then enable a guest device driver
to
probe the hypervisor for methods that it is aware of and able to use. The
hypercall op is likely to be retained (in some form), and complemented at
least
on x86 with another interface via MSRs presented to the guests.



> 1) How the configuration is managed?
>    On either virtio-mmio or virtio-pci, there always takes place
>    some negotiation between the FE and BE through the "configuration"
>    space. How can this be done in virtio-over-Argo?
>

Just to be clear about my understanding: your question, in the context of a
Linux kernel virtio device driver implementation, is about how a virtio-arg=
o
transport driver would implement the get_features function of the
virtio_config_ops, as a parallel to the work that vp_get_features does for
virtio-pci, and vm_get_features does for virtio-mmio.

The design is still open on this and options have been discussed, including=
:

* an extension to Argo to allow the system toolstack (which is responsible
for
  managing guest VMs and enabling connections from front-to-backends)
  to manage a table of "implicit destinations", so a guest can transmit Arg=
o
  messages to eg. "my storage service" port and the hypervisor will deliver
it
  based on a destination table pre-programmed by the toolstack for the VM.
  [1]
     - ref: Notes from the December 2019 Xen F2F meeting in Cambridge, UK:
       [1] https://lists.archive.carbon60.com/xen/devel/577800#577800

  So within that feature negotiation function, communication with the
backend
  via that Argo channel will occur.

* IOREQ
The Xen IOREQ implementation is not currently appropriate for virtio-argo
since
it requires the use of foreign memory mappings of frontend memory in the
backend
guest. However, a new HMX interface from the hypervisor could support a new
DMA
Device Model Op to allow the backend to request the hypervisor to retrieve
specified
bytes from the frontend guest, which would enable plumbing for device
configuration
between an IOREQ server (device model backend implementation) and the guest
driver.
[2]

Feature negotiation in the front end in this case would look very similar t=
o
the virtio-mmio implementation.

ref: Argo HMX Transport for VirtIO meeting minutes, from January 2021:
[2]
https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html

* guest ACPI tables that surface the address of a remote Argo endpoint
  on behalf of the toolstack, and Argo communication can then negotiate
features

* emulation of a basic PCI device by the hypervisor (though details not
determined)



> 2) Do there physically exist virtio's available/used vrings as well as
>    descriptors, or are they virtually emulated over Argo (rings)?
>

In short: the latter.

In the analysis that I did when looking at this, my observation was that
each
side (front and backend) should be able to accurately maintain their own
local
copy of the available/used vrings as well as descriptors, and both be kept
synchronized by ensuring that updates are transmitted to the other side whe=
n
they are written to. eg. As part of this, in the Linux front end
implementation
the virtqueue_notify function uses a function pointer in the virtqueue that
is
populated by the transport driver, ie. the virtio-argo driver in this case,
which can implement the necessary logic to coordinate with the backend.


> 3) The payload in a request will be copied into the receiver's Argo ring.
>    What does the address in a descriptor mean?
>    Address/offset in a ring buffer?
>

Effectively yes. I would treat it as a handle that is used to identify and
retrieve data from messages exchanged between frontend transport driver and
the backend via Argo rings established for moving data for the data path.
In the diagram, those are "Argo ring for reads" and "Argo ring for writes".


> 4) Estimate of performance or latency?
>

Different access methods to Argo (ie. related to my answer to your question
'1)'
above --) will have different performance characteristics.

Data copying will necessarily involved for any Hypervisor-Mediated data
eXchange
(HMX) mechanism[1], such as Argo, where there is no shared memory between
guest
VMs, but the performance profile on modern CPUs with sizable caches has bee=
n
demonstrated to be acceptable for the guest virtual device drivers use case
in
the HP/Bromium vSentry uXen product. The VirtIO structure is somewhat
different
though.

Further performance profiling and measurement will be valuable for enabling
tuning of the implementation and development of additional interfaces (eg.
such
as an asynchronous send primitive) - some of this has been discussed and
described on the VirtIO-Argo-Development-Phase-1 wiki page[2].

[1]
https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_f=
or_Xen

[2]
https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+De=
velopment%3A+Phase+1


>    It appears that, on FE side, at least three hypervisor calls (and data
>    copying) need to be invoked at every request, right?
>

For a write, counting FE sendv ops:
1: the write data payload is sent via the "Argo ring for writes"
2: the descriptor is sent via a sync of the available/descriptor ring
  -- is there a third one that I am missing?

Christopher


>
> Thanks,
> -Takahiro Akashi
>
>
> > * Here are the design documents for building VirtIO-over-Argo, to
> support a
> >   hypervisor-agnostic frontend VirtIO transport driver using Argo.
> >
> > The Development Plan to build VirtIO virtual device support over Argo
> > transport:
> >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1
> >
> > A design for using VirtIO over Argo, describing how VirtIO data
> structures
> > and communication is handled over the Argo transport:
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Arg=
o
> >
> > Diagram (from the above document) showing how VirtIO rings are
> synchronized
> > between domains without using shared memory:
> >
> https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-b=
lob-url=3Dtrue&id=3D01f7d0e1-7686-4f0b-88e1-457c1d30df40&collection=3Dconte=
ntId-1348763698&contextId=3D1348763698&mimeType=3Dimage%2Fpng&name=3Ddevice=
-buffer-access-virtio-argo.png&size=3D243175&width=3D1106&height=3D1241
> >
> > Please note that the above design documents show that the existing Virt=
IO
> > device drivers, and both vring and virtqueue data structures can be
> > preserved
> > while interdomain communication can be performed with no shared memory
> > required
> > for most drivers; (the exceptions where further design is required are
> those
> > such as virtual framebuffer devices where shared memory regions are
> > intentionally
> > added to the communication structure beyond the vrings and virtqueues).
> >
> > An analysis of VirtIO and Argo, informing the design:
> >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+=
Argo+as+a+transport+medium+for+VirtIO
> >
> > * Argo can be used for a communication path for configuration between t=
he
> > backend
> >   and the toolstack, avoiding the need for a dependency on XenStore,
> which
> > is an
> >   advantage for any hypervisor-agnostic design. It is also amenable to =
a
> > notification
> >   mechanism that is not based on Xen event channels.
> >
> > * Argo does not use or require shared memory between VMs and provides a=
n
> > alternative
> >   to the use of foreign shared memory mappings. It avoids some of the
> > complexities
> >   involved with using grants (eg. XSA-300).
> >
> > * Argo supports Mandatory Access Control by the hypervisor, satisfying =
a
> > common
> >   certification requirement.
> >
> > * The Argo headers are BSD-licensed and the Xen hypervisor implementati=
on
> > is GPLv2 but
> >   accessible via the hypercall interface. The licensing should not
> present
> > an obstacle
> >   to adoption of Argo in guest software or implementation by other
> > hypervisors.
> >
> > * Since the interface that Argo presents to a guest VM is similar to
> DMA, a
> > VirtIO-Argo
> >   frontend transport driver should be able to operate with a physical
> > VirtIO-enabled
> >   smart-NIC if the toolstack and an Argo-aware backend provide support.
> >
> > The next Xen Community Call is next week and I would be happy to answer
> > questions
> > about Argo and on this topic. I will also be following this thread.
> >
> > Christopher
> > (Argo maintainer, Xen Community)
> >
> >
> -------------------------------------------------------------------------=
-------
> > [1]
> > An introduction to Argo:
> >
> https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-=
%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> > https://www.youtube.com/watch?v=3DcnC0Tg3jqJQ
> > Xen Wiki page for Argo:
> >
> https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)=
_for_Xen
> >
> > [2]
> > OpenXT Linux Argo driver and userspace library:
> > https://github.com/openxt/linux-xen-argo
> >
> > Windows V4V at OpenXT wiki:
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> > Windows v4v driver source:
> > https://github.com/OpenXT/xc-windows/tree/master/xenv4v
> >
> > HP/Bromium uXen V4V driver:
> > https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
> >
> > [3]
> > v2 of the Argo test unikernel for XTF:
> >
> https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.htm=
l
> >
> > [4]
> > Argo HMX Transport for VirtIO meeting minutes:
> >
> https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.htm=
l
> >
> > VirtIO-Argo Development wiki page:
> >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1
> >
>
>

--000000000000b501cf05cb5d41dd
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Thu, Sep 2, 2021 at 12:19 AM AKASH=
I Takahiro &lt;<a href=3D"mailto:takahiro.akashi@linaro.org">takahiro.akash=
i@linaro.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin=
g-left:1ex">Hi Christopher,<br>
<br>
Thank you for your feedback.<br>
<br>
On Mon, Aug 30, 2021 at 12:53:00PM -0700, Christopher Clark wrote:<br>
&gt; [ resending message to ensure delivery to the CCd mailing lists<br>
&gt; post-subscription ]<br>
&gt; <br>
&gt; Apologies for being late to this thread, but I hope to be able to<br>
&gt; contribute to<br>
&gt; this discussion in a meaningful way. I am grateful for the level of<br=
>
&gt; interest in<br>
&gt; this topic. I would like to draw your attention to Argo as a suitable<=
br>
&gt; technology for development of VirtIO&#39;s hypervisor-agnostic interfa=
ces.<br>
&gt; <br>
&gt; * Argo is an interdomain communication mechanism in Xen (on x86 and Ar=
m)<br>
&gt; that<br>
&gt;=C2=A0 =C2=A0can send and receive hypervisor-mediated notifications and=
 messages<br>
&gt; between<br>
&gt;=C2=A0 =C2=A0domains (VMs). [1] The hypervisor can enforce Mandatory Ac=
cess Control<br>
&gt; over<br>
&gt;=C2=A0 =C2=A0all communication between domains. It is derived from the =
earlier v4v,<br>
&gt; which<br>
&gt;=C2=A0 =C2=A0has been deployed on millions of machines with the HP/Brom=
ium uXen<br>
&gt; hypervisor<br>
&gt;=C2=A0 =C2=A0and with OpenXT.<br>
&gt; <br>
&gt; * Argo has a simple interface with a small number of operations that w=
as<br>
&gt;=C2=A0 =C2=A0designed for ease of integration into OS primitives on bot=
h Linux<br>
&gt; (sockets)<br>
&gt;=C2=A0 =C2=A0and Windows (ReadFile/WriteFile) [2].<br>
&gt;=C2=A0 =C2=A0 =C2=A0- A unikernel example of using it has also been dev=
eloped for XTF. [3]<br>
&gt; <br>
&gt; * There has been recent discussion and support in the Xen community fo=
r<br>
&gt; making<br>
&gt;=C2=A0 =C2=A0revisions to the Argo interface to make it hypervisor-agno=
stic, and<br>
&gt; support<br>
&gt;=C2=A0 =C2=A0implementations of Argo on other hypervisors. This will en=
able a single<br>
&gt;=C2=A0 =C2=A0interface for an OS kernel binary to use for inter-VM comm=
unication that<br>
&gt; will<br>
&gt;=C2=A0 =C2=A0work on multiple hypervisors -- this applies equally to bo=
th backends and<br>
&gt;=C2=A0 =C2=A0frontend implementations. [4]<br>
<br>
Regarding virtio-over-Argo, let me ask a few questions:<br>
(In figure &quot;Virtual device buffer access:Virtio+Argo&quot; in [4])<br>=
</blockquote><div><br></div><div>(for ref, this diagram is from this docume=
nt:<br>=C2=A0<a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/1=
348763698">https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698</a>=
 )<br><br>Takahiro, thanks for reading the Virtio-Argo materials.<br><br>So=
me relevant context before answering your questions below: the Argo request=
<br>interface from the hypervisor to a guest, which is currently exposed on=
ly via a<br>dedicated hypercall op, has been discussed within the Xen commu=
nity and is open<br>to being changed in order to better enable support for =
guest VM access to Argo<br>functions in a hypervisor-agnostic way.<br><br>T=
he proposal is to allow hypervisors the option to implement and expose any =
of<br>multiple access mechanisms for Argo, and then enable a guest device d=
river to<br>probe the hypervisor for methods that it is aware of and able t=
o use. The<br>hypercall op is likely to be retained (in some form), and com=
plemented at least<br>on x86 with another interface via MSRs presented to t=
he guests.<br></div><div><br></div><div>=C2=A0</div><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,=
204,204);padding-left:1ex">
1) How the configuration is managed?<br>
=C2=A0 =C2=A0On either virtio-mmio or virtio-pci, there always takes place<=
br>
=C2=A0 =C2=A0some negotiation between the FE and BE through the &quot;confi=
guration&quot;<br>
=C2=A0 =C2=A0space. How can this be done in virtio-over-Argo?<br></blockquo=
te><div><br></div><div>Just to be clear about my understanding: your questi=
on, in the context of a<br>Linux kernel virtio device driver implementation=
, is about how a virtio-argo<br>transport driver would implement the get_fe=
atures function of the<br>virtio_config_ops, as a parallel to the work that=
 vp_get_features does for<br>virtio-pci, and vm_get_features does for virti=
o-mmio.<br><br>The design is still open on this and options have been discu=
ssed, including:<br><br>* an extension to Argo to allow the system toolstac=
k (which is responsible for<br>=C2=A0 managing guest VMs and enabling conne=
ctions from front-to-backends)<br>=C2=A0 to manage a table of &quot;implici=
t destinations&quot;, so a guest can transmit Argo<br>=C2=A0 messages to eg=
. &quot;my storage service&quot; port and the hypervisor will deliver it<br=
>=C2=A0 based on a destination table pre-programmed by the toolstack for th=
e VM.<br>=C2=A0 [1]<br>=C2=A0 =C2=A0 =C2=A0- ref: Notes from the December 2=
019 Xen F2F meeting in Cambridge, UK:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0[1] <a =
href=3D"https://lists.archive.carbon60.com/xen/devel/577800#577800">https:/=
/lists.archive.carbon60.com/xen/devel/577800#577800</a><br><br>=C2=A0 So wi=
thin that feature negotiation function, communication with the backend<br>=
=C2=A0 via that Argo channel will occur.<br><br>* IOREQ<br>The Xen IOREQ im=
plementation is not currently appropriate for virtio-argo since<br>it requi=
res the use of foreign memory mappings of frontend memory in the backend<br=
>guest. However, a new HMX interface from the hypervisor could support a ne=
w DMA<br>Device Model Op to allow the backend to request the hypervisor to =
retrieve specified<br>bytes from the frontend guest, which would enable plu=
mbing for device configuration<br>between an IOREQ server (device model bac=
kend implementation) and the guest driver.<br>[2]<br><br>Feature negotiatio=
n in the front end in this case would look very similar to<br>the virtio-mm=
io implementation.<br><br>ref: Argo HMX Transport for VirtIO meeting minute=
s, from January 2021:<br>[2] <a href=3D"https://lists.xenproject.org/archiv=
es/html/xen-devel/2021-02/msg01422.html">https://lists.xenproject.org/archi=
ves/html/xen-devel/2021-02/msg01422.html</a><br><br>* guest ACPI tables tha=
t surface the address of a remote Argo endpoint<br>=C2=A0 on behalf of the =
toolstack, and Argo communication can then negotiate features<br><br>* emul=
ation of a basic PCI device by the hypervisor (though details not determine=
d)<br></div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204)=
;padding-left:1ex">
2) Do there physically exist virtio&#39;s available/used vrings as well as<=
br>
=C2=A0 =C2=A0descriptors, or are they virtually emulated over Argo (rings)?=
<br></blockquote><div><br></div><div>In short: the latter.<br><br>In the an=
alysis that I did when looking at this, my observation was that each<br>sid=
e (front and backend) should be able to accurately maintain their own local=
<br>copy of the available/used vrings as well as descriptors, and both be k=
ept<br>synchronized by ensuring that updates are transmitted to the other s=
ide when<br>they are written to. eg. As part of this, in the Linux front en=
d implementation<br>the virtqueue_notify function uses a function pointer i=
n the virtqueue that is<br>populated by the transport driver, ie. the virti=
o-argo driver in this case,<br>which can implement the necessary logic to c=
oordinate with the backend.<br></div><div>=C2=A0</div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex">
3) The payload in a request will be copied into the receiver&#39;s Argo rin=
g.<br>
=C2=A0 =C2=A0What does the address in a descriptor mean?<br>
=C2=A0 =C2=A0Address/offset in a ring buffer?<br></blockquote><div><br></di=
v><div>Effectively yes. I would treat it as a handle that is used to identi=
fy and<br>retrieve data from messages exchanged between frontend transport =
driver and<br>the backend via Argo rings established for moving data for th=
e data path.<br>In the diagram, those are &quot;Argo ring for reads&quot; a=
nd &quot;Argo ring for writes&quot;.<br></div><div>=C2=A0</div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex">
4) Estimate of performance or latency?<br></blockquote><div><br></div><div>=
Different access methods to Argo (ie. related to my answer to your question=
 &#39;1)&#39;<br>above --) will have different performance characteristics.=
<br><br>Data copying will necessarily involved for any Hypervisor-Mediated =
data eXchange<br>(HMX) mechanism[1], such as Argo, where there is no shared=
 memory between guest<br>VMs, but the performance profile on modern CPUs wi=
th sizable caches has been<br>demonstrated to be acceptable for the guest v=
irtual device drivers use case in<br>the HP/Bromium vSentry uXen product. T=
he VirtIO structure is somewhat different<br>though.<br><br>Further perform=
ance profiling and measurement will be valuable for enabling<br>tuning of t=
he implementation and development of additional interfaces (eg. such<br>as =
an asynchronous send primitive) - some of this has been discussed and<br>de=
scribed on the VirtIO-Argo-Development-Phase-1 wiki page[2].<br><br>[1]<br>=
<a href=3D"https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Excha=
nge_(HMX)_for_Xen">https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediat=
ed_Exchange_(HMX)_for_Xen</a><br><br>[2]<br><a href=3D"https://openxt.atlas=
sian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development%3A+Phase+1=
">https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development%3A+Phase+1</a><br></div><div>=C2=A0</div><blockquote class=3D"g=
mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204=
,204,204);padding-left:1ex">
=C2=A0 =C2=A0It appears that, on FE side, at least three hypervisor calls (=
and data<br>
=C2=A0 =C2=A0copying) need to be invoked at every request, right?<br></bloc=
kquote><div><br></div><div>For a write, counting FE sendv ops:<br>1: the wr=
ite data payload is sent via the &quot;Argo ring for writes&quot;<br>2: the=
 descriptor is sent via a sync of the available/descriptor ring<br>=C2=A0 -=
- is there a third one that I am missing?<br><br>Christopher<br></div><div>=
=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thanks,<br>
-Takahiro Akashi<br>
<br>
<br>
&gt; * Here are the design documents for building VirtIO-over-Argo, to supp=
ort a<br>
&gt;=C2=A0 =C2=A0hypervisor-agnostic frontend VirtIO transport driver using=
 Argo.<br>
&gt; <br>
&gt; The Development Plan to build VirtIO virtual device support over Argo<=
br>
&gt; transport:<br>
&gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/169616998=
5/VirtIO-Argo+Development+Phase+1" rel=3D"noreferrer" target=3D"_blank">htt=
ps://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Devel=
opment+Phase+1</a><br>
&gt; <br>
&gt; A design for using VirtIO over Argo, describing how VirtIO data struct=
ures<br>
&gt; and communication is handled over the Argo transport:<br>
&gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/134876369=
8/VirtIO+Argo" rel=3D"noreferrer" target=3D"_blank">https://openxt.atlassia=
n.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo</a><br>
&gt; <br>
&gt; Diagram (from the above document) showing how VirtIO rings are synchro=
nized<br>
&gt; between domains without using shared memory:<br>
&gt; <a href=3D"https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd437=
7a1194#media-blob-url=3Dtrue&amp;id=3D01f7d0e1-7686-4f0b-88e1-457c1d30df40&=
amp;collection=3DcontentId-1348763698&amp;contextId=3D1348763698&amp;mimeTy=
pe=3Dimage%2Fpng&amp;name=3Ddevice-buffer-access-virtio-argo.png&amp;size=
=3D243175&amp;width=3D1106&amp;height=3D1241" rel=3D"noreferrer" target=3D"=
_blank">https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#m=
edia-blob-url=3Dtrue&amp;id=3D01f7d0e1-7686-4f0b-88e1-457c1d30df40&amp;coll=
ection=3DcontentId-1348763698&amp;contextId=3D1348763698&amp;mimeType=3Dima=
ge%2Fpng&amp;name=3Ddevice-buffer-access-virtio-argo.png&amp;size=3D243175&=
amp;width=3D1106&amp;height=3D1241</a><br>
&gt; <br>
&gt; Please note that the above design documents show that the existing Vir=
tIO<br>
&gt; device drivers, and both vring and virtqueue data structures can be<br=
>
&gt; preserved<br>
&gt; while interdomain communication can be performed with no shared memory=
<br>
&gt; required<br>
&gt; for most drivers; (the exceptions where further design is required are=
 those<br>
&gt; such as virtual framebuffer devices where shared memory regions are<br=
>
&gt; intentionally<br>
&gt; added to the communication structure beyond the vrings and virtqueues)=
.<br>
&gt; <br>
&gt; An analysis of VirtIO and Argo, informing the design:<br>
&gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/133342822=
5/Analysis+of+Argo+as+a+transport+medium+for+VirtIO" rel=3D"noreferrer" tar=
get=3D"_blank">https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225=
/Analysis+of+Argo+as+a+transport+medium+for+VirtIO</a><br>
&gt; <br>
&gt; * Argo can be used for a communication path for configuration between =
the<br>
&gt; backend<br>
&gt;=C2=A0 =C2=A0and the toolstack, avoiding the need for a dependency on X=
enStore, which<br>
&gt; is an<br>
&gt;=C2=A0 =C2=A0advantage for any hypervisor-agnostic design. It is also a=
menable to a<br>
&gt; notification<br>
&gt;=C2=A0 =C2=A0mechanism that is not based on Xen event channels.<br>
&gt; <br>
&gt; * Argo does not use or require shared memory between VMs and provides =
an<br>
&gt; alternative<br>
&gt;=C2=A0 =C2=A0to the use of foreign shared memory mappings. It avoids so=
me of the<br>
&gt; complexities<br>
&gt;=C2=A0 =C2=A0involved with using grants (eg. XSA-300).<br>
&gt; <br>
&gt; * Argo supports Mandatory Access Control by the hypervisor, satisfying=
 a<br>
&gt; common<br>
&gt;=C2=A0 =C2=A0certification requirement.<br>
&gt; <br>
&gt; * The Argo headers are BSD-licensed and the Xen hypervisor implementat=
ion<br>
&gt; is GPLv2 but<br>
&gt;=C2=A0 =C2=A0accessible via the hypercall interface. The licensing shou=
ld not present<br>
&gt; an obstacle<br>
&gt;=C2=A0 =C2=A0to adoption of Argo in guest software or implementation by=
 other<br>
&gt; hypervisors.<br>
&gt; <br>
&gt; * Since the interface that Argo presents to a guest VM is similar to D=
MA, a<br>
&gt; VirtIO-Argo<br>
&gt;=C2=A0 =C2=A0frontend transport driver should be able to operate with a=
 physical<br>
&gt; VirtIO-enabled<br>
&gt;=C2=A0 =C2=A0smart-NIC if the toolstack and an Argo-aware backend provi=
de support.<br>
&gt; <br>
&gt; The next Xen Community Call is next week and I would be happy to answe=
r<br>
&gt; questions<br>
&gt; about Argo and on this topic. I will also be following this thread.<br=
>
&gt; <br>
&gt; Christopher<br>
&gt; (Argo maintainer, Xen Community)<br>
&gt; <br>
&gt; ----------------------------------------------------------------------=
----------<br>
&gt; [1]<br>
&gt; An introduction to Argo:<br>
&gt; <a href=3D"https://static.sched.com/hosted_files/xensummit19/92/Argo%2=
0and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019=
.pdf" rel=3D"noreferrer" target=3D"_blank">https://static.sched.com/hosted_=
files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clar=
k%20-%20Xen%20Summit%202019.pdf</a><br>
&gt; <a href=3D"https://www.youtube.com/watch?v=3DcnC0Tg3jqJQ" rel=3D"noref=
errer" target=3D"_blank">https://www.youtube.com/watch?v=3DcnC0Tg3jqJQ</a><=
br>
&gt; Xen Wiki page for Argo:<br>
&gt; <a href=3D"https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_=
Exchange_(HMX)_for_Xen" rel=3D"noreferrer" target=3D"_blank">https://wiki.x=
enproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen</a><br>
&gt; <br>
&gt; [2]<br>
&gt; OpenXT Linux Argo driver and userspace library:<br>
&gt; <a href=3D"https://github.com/openxt/linux-xen-argo" rel=3D"noreferrer=
" target=3D"_blank">https://github.com/openxt/linux-xen-argo</a><br>
&gt; <br>
&gt; Windows V4V at OpenXT wiki:<br>
&gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/=
V4V" rel=3D"noreferrer" target=3D"_blank">https://openxt.atlassian.net/wiki=
/spaces/DC/pages/14844007/V4V</a><br>
&gt; Windows v4v driver source:<br>
&gt; <a href=3D"https://github.com/OpenXT/xc-windows/tree/master/xenv4v" re=
l=3D"noreferrer" target=3D"_blank">https://github.com/OpenXT/xc-windows/tre=
e/master/xenv4v</a><br>
&gt; <br>
&gt; HP/Bromium uXen V4V driver:<br>
&gt; <a href=3D"https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv=
4vlib" rel=3D"noreferrer" target=3D"_blank">https://github.com/uxen-virt/ux=
en/tree/ascara/windows/uxenv4vlib</a><br>
&gt; <br>
&gt; [3]<br>
&gt; v2 of the Argo test unikernel for XTF:<br>
&gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-devel/2021-0=
1/msg02234.html" rel=3D"noreferrer" target=3D"_blank">https://lists.xenproj=
ect.org/archives/html/xen-devel/2021-01/msg02234.html</a><br>
&gt; <br>
&gt; [4]<br>
&gt; Argo HMX Transport for VirtIO meeting minutes:<br>
&gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-devel/2021-0=
2/msg01422.html" rel=3D"noreferrer" target=3D"_blank">https://lists.xenproj=
ect.org/archives/html/xen-devel/2021-02/msg01422.html</a><br>
&gt; <br>
&gt; VirtIO-Argo Development wiki page:<br>
&gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/169616998=
5/VirtIO-Argo+Development+Phase+1" rel=3D"noreferrer" target=3D"_blank">htt=
ps://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Devel=
opment+Phase+1</a><br>
&gt; <br><br>
</blockquote></div></div>

--000000000000b501cf05cb5d41dd--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 01:43:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 01:43:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180313.326903 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNQ8a-00071y-LO; Tue, 07 Sep 2021 01:43:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180313.326903; Tue, 07 Sep 2021 01:43:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNQ8a-00071r-HG; Tue, 07 Sep 2021 01:43:00 +0000
Received: by outflank-mailman (input) for mailman id 180313;
 Tue, 07 Sep 2021 01:42:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M3oQ=N5=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mNQ8Y-00071l-VO
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 01:42:59 +0000
Received: from mail-oo1-xc2b.google.com (unknown [2607:f8b0:4864:20::c2b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f67aa032-282b-44eb-bc5c-ffc11927bb96;
 Tue, 07 Sep 2021 01:42:53 +0000 (UTC)
Received: by mail-oo1-xc2b.google.com with SMTP id
 k18-20020a4abd92000000b002915ed21fb8so2468578oop.11
 for <xen-devel@lists.xenproject.org>; Mon, 06 Sep 2021 18:42:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f67aa032-282b-44eb-bc5c-ffc11927bb96
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:from:date:message-id:subject:to:cc;
        bh=AmTYCsTMxsxUaV+vrl73d/gbiuPrWmazNxtA2VpDiE4=;
        b=F2cSITu7H6zodAIKEv07Q1UXJNjdAmUHud5D8ZNM4iteDHDTuhTSM1laFUOwcuvo+j
         GF1VwFDyEBZHMtuv6On3rpsc4EXDSYGet1ZuqppQYR13E/0jxEeDHnr+c5U1Ba7u4Go1
         3yMtTxviC1wLPfcXvas1a6xSozLNCorc+hQvj+PeRu88S5p7uW5LqqCBesmmZzsbngbI
         D0uupFhGHO2oYlw87f7EvTfVn2RrzZ0XPjPj+3KfwYobiLuwzDu10SaO+v3ugsytrWMr
         FVRl0F8dygQlu9Ao2u5wCPpNN0iuqZKOFOaPOu/2dg3GAQhPjISTLa8iDFpx3snPmkKq
         yGHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc;
        bh=AmTYCsTMxsxUaV+vrl73d/gbiuPrWmazNxtA2VpDiE4=;
        b=IGGPibdq2DnKSHyzxBm5b3ciAkCxhfxgPEqPElbNZHsiM+mo201HgKUcw0jk0OwBiV
         IOaEnqxqlGPJSz25kcmMT2KfUc+9k8AZrBajxTJU9VsVuSBoAlh2AFXmfhR00bUpkt6f
         vDGBjf7w+1MLQ0d8A5dPt4OYDIX/zQ0kGr/MXebz6NcIRYzbmCsptxkUbKhi61ohVGaa
         6GoS+U6/ncuKj0fl5MMJUPAcmPGoV+JIUPV5066fK41VOasDtoXdGZi4kJcghaXtAVXt
         Db4rNseiBnvVUbvl9KI7VfJS0/OzlJpRDznBUjawRAnskesCwMyJX/BTs5Nk9wfhI62X
         QSeQ==
X-Gm-Message-State: AOAM533lnSdvUEB7c7yXWKJTvpVP6A9npbZZQson7BjQmGQzhcTMnmcm
	mu7hdKxC4gSd6WbxfBEcNUWjXtpbVmTmw1cCkgC4QnriD159oA==
X-Google-Smtp-Source: ABdhPJyqyXvInMjZtKtyVn1MU4FPSFnwbM+SxbhnaCAPYw9c+kHOMzZpytZTtEokXXQMt3YOn814gLY+OvHFIAoi7VI=
X-Received: by 2002:a4a:e3cf:: with SMTP id m15mr15993837oov.21.1630978972478;
 Mon, 06 Sep 2021 18:42:52 -0700 (PDT)
MIME-Version: 1.0
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Mon, 6 Sep 2021 18:42:36 -0700
Message-ID: <CACMJ4GarXj5uAiNuoxvKHPhbv40jVtqgH0TxpB22PjyD2mLMRQ@mail.gmail.com>
Subject: Notes from the Xen Summit 2021: Design Session VirtIO Cross-Project
 BoF (Birds of a Feather) for Xen and Guest OS (Linux, Windows, FreeBSD) developers
To: xen-devel <xen-devel@lists.xenproject.org>
Cc: Juergen Gross <jgross@suse.com>, Rich Persaud <persaur@gmail.com>, stratos-dev@linaro.org, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Julien Grall <jgrall@amazon.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Julien Grall <julien.grall.oss@gmail.com>, James McKenzie <james@bromium.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <pdurrant@amazon.co.uk>, 
	Paul Durrant <paul@xen.org>, Jean-Philippe Ouellet <jpo@vt.edu>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
	Jason Andryuk <jandryuk@gmail.com>, eric chanudet <eric.chanudet@gmail.com>, 
	Chris Rogers <rogersc@ainfosec.com>, Rich Turner <turnerr@ainfosec.com>, 
	Stefano Stabellini <stefano.stabellini@xilinx.com>, Wei Liu <wl@xen.org>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, demi@invisiblethingslab.com, 
	George Dunlap <George.Dunlap@citrix.com>, damien.thenot@vates.fr, 
	Wei Chen <Wei.Chen@arm.com>
Content-Type: multipart/alternative; boundary="0000000000004a724505cb5de215"

--0000000000004a724505cb5de215
Content-Type: text/plain; charset="UTF-8"

Design Session notes for: VirtIO Cross-Project BoF (Birds of a Feather) for
Xen and Guest OS (Linux, Windows, FreeBSD) developers
---------------------------
Xen Design & Developer Summit, 27th May 2021
Session Host: Juergen Gross
Notes by: Christopher Clark, with thanks to Rich Persaud

Apologies for the delay in posting these, solely my responsibility.
- Christopher

============================================================
Session Context:

There are three separate recent approaches within the Xen community to
enabling
use of VirtIO device drivers in guest virtual machines with the Xen
hypervisor,
and a fourth older completed project.
(Placeholder names are assigned to each of these below for ease of
reference.)

In addition, Linaro has an active project 'Stratos' pursuing:
    "Establish virtio as the standard interface between hypervisors,
    freeing a mobile, industrial or automotive platform to migrate between
    hypervisors and reuse the backend implementation."

    - https://linaro.atlassian.net/wiki/spaces/STR/overview
    - https://projects.linaro.org/projects/STR/summary
    - https://op-lists.linaro.org/mailman/listinfo/stratos-dev

--------------------
* 'VirtIO-EPAM': enabling existing VirtIO-MMIO transport on Xen, using
foreign
mappings and an IOREQ server.
by EPAM and others, with focus on Xen on Arm platforms.

Enables use of the existing standardized VirtIO-MMIO transport driver,
which is
present in the mainline Linux kernel, using Xen's IOREQ emulation
infrastructure and use of privileged foreign mappings to establish shared
memory for access to guest data by the device model backend.

Status: Patches are in progress towards Xen on the xen-devel mailing list.
The presentation at Linaro Connect 2021 includes a working demonstration.

VirtIO on Xen hypervisor (Arm), Oleksandr Tyshchenko, EPAM, Linaro Connect
2021
https://static.linaro.org/connect/lvc21/presentations/lvc21-314.pdf
https://www.youtube.com/watch?v=XE5Rn8KFunk

--------------------
* 'VirtIO-SuSE': introducing a new VirtIO transport driver that uses Xen
grants.
by SuSE, presented at this Xen Summit.

A new VirtIO transport device driver is added to the guest kernel, to
translate guest physical addresses into grant references, enabling VirtIO
data
path communication over mutually-negociated shared memory regions between
the
guest virtual machine and the device model backend. Improves isolation as
backend does not need privilege over the guest to perform foreign mappings.
Grant references are a Xen-specific interface. Design supports driver
domains.

Status: A prototype is described in the presentation at this Xen Summit
2021.

VirtIO and Xen with Full Grant Support:
https://static.sched.com/hosted_files/xen2021/bf/Thursday_2021-Xen-Summit-virtio.pdf
https://www.youtube.com/watch?v=IrlEdaIUDPk&list=PLYyw7IQjL-zGcRPN6EjiTuFVGo4A6KCNf&index=25

--------------------
* VirtIO-Argo: introducing a new VirtIO transport driver that uses Argo for
interdomain communication, supporting isolation and Mandatory Access
Control.
Design and analysis performed within the OpenXT and Xen communities.

A new VirtIO transport device driver is added to the guest kernel to
transmit
data between the guest domain and the domain hosting the device model via
Argo
rings: a Hypervisor-Mediated data eXchange protocol where the hypervisor
transfers the data, being trusted to strictly adhere to the delivery
protocol.
Supports stronger isolation properties and enforcement of Mandatory Access
Control security policy over interdomain communication. Does not use shared
memory between domains.
Development of a Hypervisor-agnostic interface for Argo has been proposed
and discussed within the Xen community. Design supports driver domains.

Status: Design and analysis published; funding required for development to
proceed.

VirtIO-Argo: Documentation at the OpenXT wiki:
https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO%2BArgo
VirtIO-Argo Development:
https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1
Minutes from the Argo HMX Transport for VirtIO topic call, 14th January
2021:
https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html

--------------------
* 'VirtIO-Xen-GSoC': 2011 VirtIO on Xen student project
A Google Summer of Code project by Wei Liu investigated enabling VirtIO on
Xen.

A working prototype was produced for both PV and HVM guests, using XenBus
and
the Qemu VirtIO backends. PV guests require a guest kernel patch to
translate
guest physical addresses to machine addresses in VirtIO rings.

Status: project completed.

https://wiki.xenproject.org/wiki/Virtio_On_Xen

============================================================
Summary of the VirtIO Design Session:

xl toolstack guest config file syntax for VirtIO devices:
    - Recommend: mix of: device-specific config (eg. disk, net) plus
support for
                 generic VirtIO device config (eg. esoteric devices)

VirtIO spec:
    - spec change needed? understood as not mandatory
    - v1.1 has a platform feature for DMA addr translation: enables
transport
      driver to use any of grants, pre-shared memory, Argo in the data path
        - ie. does not force use of (guest) physical addrs in the ring
    - open question re: Linux and qemu acceptance of non-standardized driver

Guest access to new VirtIO transport
    - for Argo or grants, add new transport driver (eg. out of tree module)

Performance, standardization
    - different characteristics on different architectures; motivates
      development and deployment of different transports for separate cases
    - difficulty of ensuring correctness with shared memory vs. performance
      achievable with some configurations

Atomics in shared memory, Arm, "Fat Virtqueue" development
    - Arm and RISC-V have challenges with atomic inst in shared memory
    - VirtIO-SuSE (using grants) useful enough for consideration
    - Wind River preshared-memory memcpy approach (OpenAMP/kvmtool)
    - Linaro "Fat Virtqueue" under development: pass data within enlarged
rings

Data copies with Argo and shared memory transports
    - discussion of copies made by different transports, different
conditions
    - data copies performed to protect correctness in communication

Ecosystems, differentiation
    - standarization of VirtIO shifted economics for hypervisor developers
    - Mandatory Access Control with Xen technologies will be a Xen advantage


============================================================
Detailed notes for the Design Session:

Rich (OpenXT): introducing session; to talk about:
    - how any solution will be ratified by the VirtIO/OASIS community
        - esp automotive use cases: a lot of attention paid to formal
          acceptance of any VirtIO solution

    - non-Linux guest VMs
        - Windows is used in OpenXT, Qubes
        - FreeBSD is popular for networking and storage

Roger (Citrix; x86 maintainer) [chat]: I wanted to ask whether the Linux
kernel
/QEMU code could be accepted without any spec change?

--- topic: xl toolstack guest config file syntax for VirtIO devices

Juergen (SUSE; PV interfaces maintainer): shared window with VM config:
------------------------
disk=[ 'file:/home/vm/upstream/image,hda,w', ]
vif=[ 'mac:00:16:3e:06:a7:21,bridge=br0', ]
vfb=[ 'vnclisten=localhost:0', ]
# device_model_args=[ '-drive',
'file=/home/vm/sle15sp3/image,if=virtio,index=1,media=disk,format=raw,cache=writeback'
]
device_model_args=[ '-drive',
'file=/home/vm/sle15sp3/image,if=none,id=drive-virtio-disk0,format=raw',
'-device',
'virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=0,disable-legacy=on'
]
------------------------

Juergen: Request for input on xl config file syntax for configuring VirtIO
devices
    - see device_model_args in example (force VirtIO block to non-legacy
mode)
    - alt option: add a 'virtio' specifier to the disk line
        - see recent patch by Artem
    - need to consider generic devices: non-network, non-disk device classes

Q: Should VirtIO disk be added under 'disk', or a new special VirtIO disk?

Julien (Amazon; Arm maintainer):
    - want to avoid having one vif, one framebuffer, one gps, etc.
        - so instead have generic device options (eg. block, gps)
            - ie. similar to Qemu

Marek (Qubes; Python bindings maintainer):
    - libvirt bindings perspective: Preference for the other way
        - don't want duplicated eg. disk handling, for the VirtIO case
        - unified handling for a device class is more convenient for API
users

Stefano (Xilinx; Arm maintainer):
    - Disk is a special case: config already has a line for disk,
      already have a way to specify different types - network may be similar
    - alt method is better for other devices, eg. virtio-rng
        - for those, device_model_arg is probably the best

Andy (Citrix, x86 maintainer):
    - a lot of this is PCI-based, even if non-DMA transport so:
        - need to preserve support for multi-IOREQ servers
        - feasible to have VirtIO backends that are not Qemu
    - so: don't tie the configuration to Qemu

OleksandrT (EPAM) [chat]: Initially I created a new "vdisk" property for
virto.disk, but was asked to reuse existing "disk" configuration in libxl
if possible.
https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg01174.html

Juergen: ack; esp for Arm, mmio-based config: so we should be flexible.
Aiming to determine:
  1: Configuration format for VirtIO devices in guests
  2: Surfacing VirtIO devices in guest VMs:
        - tie it to the PCI specification or not?
        - challenge if guests do not have PCI detection

    - Responses:
        - PCI is fairly standard
        - issue with mmio: hotplugging devices
              - ranges to be configured => need number of devices in advance
              - hotplug already requires upfront work, due to grants, events
        - PCI and mmio are similar; choice depends on guest and ecosystem

George (Citrix; Xen Community Manager):
    - makes sense for a case to standardize on Argo?
    - Argo vs grant table vs shared memory : why duplicate the extra effort?
    - how much effort duplicated for multiple? is it pretty easy to swap
them
      out?

Rich [chat]: Important for automotive, FuSA, security

Christopher (Apertus/Star Lab; Argo maintainer):
    - Options are not exclusive: VirtIO can use different transports
per-device
    - Transports orthogonal eg. Argo devices enumerable via the VirtIO-Argo
      driver; Argo enables Mandatory Access Control over the data transport


--- topic: VirtIO spec change; addresses on VirtIO rings

Concern: requirement for a VirtIO spec change

- Discussion re: is spec change mandatory?
    - may be needed: VirtIO spec says pass guest physical addr on ring
    - may not be: translation step in the transport layer;
                  enables translation so don't need to change spec
    - is spec change needed for config negotiation, to announce feature?
    - response: no; transport driver does enumerations, so up to the driver

Stefano: "VirtIO" means different things: spec, transport, drivers
    - since transports are part of the spec: VirtIO-PCI, VirtIO-MMIO, ...,
      adding one (VirtIO-Argo) means adding to the spec
Daniel: no ..

Juergen: VirtIO spec does not demand physical addresses on the ring pages
    - DMA addresses are allowed
    - eg. a special DMA mechanism, which happens to be grants, is fine.
        - ie. We have a special DMA engine.
Andy: Yes. DMA addresses != identity map onto physical or guest phys
addresses

Daniel: there is a spec for the transports: if implementing a PCI
transport, or
  a MMIO transport, has certain characteristics of how those should behave;
  so the driver-to-transport interface is fixed.
  It doesn't fix the transport back, which was the analysis that
Christopher and
  I were looking at.
  Fixed from front-end perspective: interface between the driver and
transport.
  If the transport presents as a PCI device transport, there's an interface
that
  the driver expects that transport to provide.
     - It doesn't fix how it happens from the transport back.

George: Frontend VirtIO driver hands phys addr to frontend transport and the
       frontend transport can convert it to eg. grant tables to do what it
wants

Daniel: right
    - plus more behaviours, esp for PCI: eg. structures expected to be there
      so the drivers can expect a common behaviour for any PCI transport
        - transport is translation layer, can handle dealing with actual
          physical backend mechanism that you're going over

Stefano: so plan is to add new transport: would that need to be in the spec?
Daniel: no


--- topic: Guest access to new VirtIO transport

George: VirtIO frontend drivers would work with the new transport.
    - a current VM image won't have transport implemented
        - have to update to a new version of kernel, or
        - a new version of image to get new transport?

Daniel: Well, that's one way; for Argo, can present as either a PCI
transport or
a MMIO transport to the driver, so the driver understands how to interact
with
the transport; and the transport handles moving data back. For Argo, we move
everything onto Argo rings.
To gain that on Linux, can do an out of tree kernel module, and load that
without having to recompile the entire kernel (unless they don't have
loadable
module support).
- responses:
    - needs the frontend transport in the guest;
    - so can't just use an existing image unmodified -- need the compiled
      kernel driver loaded to make the transport available
    - objective will be to add it to the upstream Linux kernel

Andy: might be able to get a frontend Windows driver somewhere useful

Stefano: fewer changes to the spec are better: looks like nothing, fantastic
Second is changes to the code; seems limited to transport driver, is a lot
better than having to change all the frontend drivers

    - Christopher: yes; point is to keep all of those and should be able to
                   plug them into backend drivers in the Qemu
implementation.
    - Juergen: Yes. Actual backends unchanged and only lower levels of
               transport layer need some modifications.
        - ie. for all implementations of backend infrastructure:
            - Qemu
            - the kernel for the vhost backends
            - vhost-user, all the user daemons implementing the backend
    - Andy: sounds great overall

---- topic: Performance

George: whether to use Argo or grant tables or something else:
    - Andy: think the answer is that we need both
    - Juergen: Yes.
    - Andy: there are people that need to not have shared memory, and Argo
is an
            option there, and there are people who will want shared memory
            because its faster.
    - ...: Is it?
    - Andy: Yes.

Damien Thenot [chat]: Shared memory would be interesting for DPU too

Stefano: Performance is a good metric;
- Able to convince third party proprietary vendor to add a driver?
    - some sort of shared memory useful as a last resort, fallback:
        - ie. easier to say "just add a memcpy" and to copy to the
pre-shared
          memory region

George: Ian Pratt presentation said:
    - copying is faster than sharing memory
    - sharing memory is so difficult that noone is smart enough to do it,
and we
      were all fools back in the 2000s to think we were smart enough to do
it

Christopher [chat]: https://platformsecuritysummit.com/2018/speaker/pratt

Andy: Not using shared memory can get around a lot of common bugs, but if
you
      know how to do shared memory, it is still faster than data copying in
      enough cases for it to be relevant, for us to care.
Marek: if you can avoid mapping and unmapping all the time
      eg. can use well known regions.

Andy: a lot of network performance in XenServer is from not mapping
    - dom0 constructs a scatter-gather over the guest memory
      (a granted area, so permissions work) - never actually touched
    - Can't do that with copying mechanism; would force a memcpy into a
      zero overhead transmit path

Stefano: measurements so far always demonstrated grant table is slower than
any
         memcpy mechanisms
    - Julien: related to atomic instructions, ref: the previous talk?
    - Stefano: could be; all my latest measurements are on Arm.
    - Andy: Different architectures will make massive differences here

Jason Andryuk (OpenXT) [chat]: @Andrew, you use netfront/back but your dom0
nic
                               scatter/gathers over the guest's granted
network
                               frames directly?
Andy [chat]: Yeah. A consequence is that dom0 never touches the mapping, so
             never sets the A bit, so you can skip the TLB flush on the
unmap
             side and this is great for performance
Roger [chat]: Note you need to patch Xen to not do the flush on unmap if
the A
              bit isn't set


---- topic: Atomics in shared memory, Arm, "Fat Virtqueue" development

Juergen: regarding Arm and shared pages: how does VirtIO and KVM on Arm
work?

Julien: don't do atomics in shared page. Lesson learned a while ago.
- Xen has many uses of atomics in shared pages, eg. evtchn, etc.
  For VirtIO, would be nicer to avoid doing the same again.

Stefano: Could we require Arm 8.1? (Is 8.1 introduces the new atomics, or
8.2?)

Julien: Yes, but limited hardware available today
- latest may have it, but not Qemu
- looks like RISC-V may have the same issues

Demi Marie (Qubes OS) [chat]: RISC-V only has 32-bit atomics

Andy: VirtIO-Grant [ie. VirtIO-SuSE in these notes] may not be perfect for
all
      future use cases and other options exist - but still enough cases
that it
      is worth considering

Demi Marie [chat]: Long-term fix is probably to change the guest-visible
ABI to
                   not require atomics in shared memory

Julien: how can it be used with other hypervisors?
- eg. for Argo, looking at something that can be used in other hypervisors
- not possible with grant table, because of the problems I mentioned.
- want to avoid having a Xen-specific driver, ie. tying VirtIO transport to
Xen
  due to use of grant table

Stefano: two related activities, both based on shared memory:
    - one driven by Wind River: memcpy-based approach
        - presharing memory, and picking addresses off the shared region

Stefano [chat]: This is WindRiver KVMtools that does shared memory for
Virtio
https://github.com/OpenAMP/kvmtool

Stefano: second approach is from Linaro: "Fat Virtqueue"
        - increase the ring size and pick addresses within the ring only
            - So the shared memory region is only the ring, and the buffers
are
              within the ring. Pointer points to things within the ring.

George: summarizing:
- on Arm copying might be faster, and there are other advantages as well;
- on x86, things may be different
so:
- SuSE and Citrix may work on the grant table one, and
- Epam and Arm, etc. may work on the Argo one
and people can understand why they're choosing which one.

George: Can you also do driver domain things with Argo?
Daniel: Yes


--- topic: Data copies with Argo and shared memory transports

Stefano: with Argo is there a memcpy, or more than one memcpy? who would do
it?

Christopher: hypervisor does the copy for the transport.
- guest hypercall op to register a ring, allows hypervisor to blit data in;
    - can think of as a DMA-style operation to deliver data to the guest

Juergen: So for each I/O data, there are three copies:
    - first, into the ring, then by the hypervisor to the other domain,
      and then out of the ring of the other domain to the target address,
right?
Andy: That's correct if both the source and destination are buffering the
I/O in
      and out of the transport.
Daniel: Right

Andy: Buffering is probably because the backend is normally shared memory.
If you've got a guarantee that it's not shared memory between frontend and
backend, then neither the front or the back VirtIO driver need to buffer in
and
out of the transport

Juergen: ok, for a disk I/O: have user data in the guest, want to write to
disk
Arbitrary data in memory, needs to be written to the Argo ring buffer,
right?
That's copy one.
Andy: No; only destination has an Argo ring. Source just passes arbitrary
      addresses, like scatter-gather DMA.
Juergen: OK, so like grant copy
Andy: Yes.

Julien: With Argo, are the pages always mapped in Xen?
    - Christopher: Yes, for the rings the way that Argo works at the moment
is,
                   yes, once you register a ring that mapping is retained so
                   the hypervisor can send to it when a hypercall is
invoked.
    - Julien: OK

Demi Marie: is it possible for Argo to copy memory from guest one userspace
to
            the ring in guest two?
    - re: disk I/O: from ZFS issue tracker: have to take copies because is
not
      safe to operate on data shared with untrusted parties
    - So most cases you're going to have to have a copy.
        - Only exception: extremely carefully written data sources,
destinations
            - careful to only make loads exactly once
            - or tools like everparse, or another tool that generates
              formally-verified binary parsers, immune to double-fetch bugs
        - so: prefer to just take the copy
            - because memcpy on modern processors is very, very fast
            - especially while it's stayed in cache

George: Andy was saying: some people definitely going to want Argo, because
it's copying instead of shared memory, and double-fetch issues with shared
memory is definitely an issue

Andy: Yes. A lot of reasons why using shared memory is hard to do correctly;
not impossible but it is hard.


--- topic: Configuration syntax (continued)

Juergen: choice: generic virtio device configuration, vs individual
         device-specific stuff, or just mix variants?

Roger [chat]: I think both

Christopher: good to have generic + allow specialization if you care

Marek: Yes; eg. for network devices, might want a different copying
mechanism if
you can avoid copies for network. Same may not be doable for disk, so might
want
different mechanisms for disk and network devices.

Juergen: Should VirtIO disk be labelled under the disk item of the guest
config,
or a specific virtio disk item? Or virtio-device type = disk, or whatever.

Roger:
- keep the old disk options, expand to support VirtIO
- add a generic VirtIO option for more esoteric devices that we don't
support in
  any other way.
Disks and vifs need a type parameter to select VirtIO as backend.

Juergen: then have a transport layer specification:
(eg. don't care, or grants, or Argo, or whatever)

George: We have the same thing for PCI (strict, relaxed, ...)
ie. Set a global default for xl, plus override in specific domain config


--- topic: Interface to adapt to platform transport

Juergen: spec bit is: VIRTIO_F_ACCESS_PLATFORM feature bit
- indicates "the device can be used on a platform where the device
  access to data memory is limited, and/or translated"
  Exactly what we want; works for grants, etc.
    - is a generic bit from newer virtio spec, not a Xen-specific bit
    - in current impl, assumes running on Xen and means grants, because
      assumes that everything is done for addresses going into the ring


--- topic: Ecosystems, differentiation

Rich: On standardization of VirtIO: it commoditized the driver interface -
now
all hypervisors implement VirtIO because it's too expensive to do otherwise.
- using Argo transport to add Mandatory Access Control (needed by many
guests)
  is an advantage for Xen, since only with Xen will you get the strong
  guarantees from Argo. Especially with Hyperlaunch, and all the other
  integrity pieces.
    - Challenging for other hypervisors to do MAC transport for VirtIO.


End of session

============================================================
Open Issues / Technical questions:

- Method of discovery of available transport drivers, negociation which to
use
    - OASIS/VirtIO Standardization required? (eg. feature bits needed?)

- Upstreaming path for different pieces to: Linux kernel, Qemu, elsewhere
    - including enabling guest support for FreeBSD, Windows, etc.

- Development of VirtIO "Fat Virtqueues": how does this affect system
design?

- Design for interaction with guest virtual IOMMU
    - ref: VirtIO-iommu

- VFIO for device model backends

- Support for complex VirtIO drivers: eg. framebuffers (shared memory
regions)


============================================================
Areas suitable for investment to support VirtIO on Xen:

- Development of the Hypervisor-agnostic interface for Argo
Ref: Xen and OpenXT community discussion minutes:
https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html

- Development of the VirtIO-Argo system
Project analysis and design documentation:

https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1

- VirtIO-focussed test cases for the Xen Test Framework
    - to provide coverage of the hypervisor interfaces used by VirtIO
transports
    - extend the framework capabilities to enable new tests (eg. multi-VM
cases)
  https://xenbits.xen.org/docs/xtf/index.html
    - development of support for Arm platforms with XTF

- Rust and Go languages toolstack development for Xen
    - Investment in the Xen toolstack is needed to support hypervisor
feature
      development. Toolstack support is necessary for enabling VirtIO
guests.
    - Work towards enabling Xen tools in Rust has started:
        https://github.com/xcp-ng/xenopsd-ng
    - Work towards enabling Xen tools in Go has started:

https://lists.xenproject.org/archives/html/xen-devel/2021-04/msg01744.html

- System image test integration with Yocto/Openembedded and Qemu
    - Xen and the Xen Test Framework can already be run within Qemu for x86
      in the standard meta-virtualization environment; Arm platform support
      needs to be completed, with coverage for VirtIO guests, and Continuous
      Integration enabled
    https://lists.yoctoproject.org/g/meta-virtualization/message/6405
    - to evaluate: VirtIO as a DISTRO_FEATURE or MACHINE_FEATURE

- Creation of documentation for Argo
    - implementation documentation exists:
      https://xenbits.xen.org/docs/4.15-testing/designs/argo.html

https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
      https://www.youtube.com/watch?v=cnC0Tg3jqJQ
    - user interface / client / kernel developer documentation is needed

- Development items for Argo:

https://openxt.atlassian.net/wiki/spaces/DC/pages/737345538/Argo+Hypervisor-Mediated+data+eXchange+Development

- Performance profiling of VirtIO-on-Xen systems across hardware
architectures

============================================================
Additional References:

VirtIO Specification v1.1, OASIS, 11th April 2019:
https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html
VirtIO spec maintainers repository:
https://github.com/oasis-tcs/virtio-spec
Organization for the Advancement of Structured Information Standards
https://www.oasis-open.org
ACPI Virtal I/O Translation Table (VIOT) DRAFT v9, December 2020:
https://jpbrucker.net/virtio-iommu/viot/viot-v9.pdf
https://lore.kernel.org/linux-acpi/20210610075130.67517-2-jean-philippe@linaro.org/T/

Linaro: Stratos Project: Auto / Industrial Demonstrable Milestones
https://linaro.atlassian.net/wiki/spaces/STR/pages/28503377401/Demonstrable+artifacts

OpenXT: VirtIO and Argo:
https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO%2BArgo

OpenXT: VirtIO-Argo Development:
https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1

OpenXT: Analysis of Argo as a transport medium for VirtIO:
https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO

Virtio with Argo for Xen, Mandatory Access Control
Meeting of Virtualization Experts Group of Automotive Grade Linux, 18th
Aug, 2020
https://lists.automotivelinux.org/g/agl-dev-community/attachment/8595/0/Argo%20and%20VirtIO.pdf

VirtIO on Xen hypervisor (Arm), Oleksandr Tyshchenko, EPAM, Linaro Connect
2021
https://static.linaro.org/connect/lvc21/presentations/lvc21-314.pdf
https://www.youtube.com/watch?v=XE5Rn8KFunk

Xen wiki: Argo
https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen

OpenXT: Argo: Hypervisor-Mediated data eXchange: Development:
https://openxt.atlassian.net/wiki/spaces/DC/pages/737345538/Argo%2BHypervisor-Mediated%2Bdata%2BeXchange%2BDevelopment

Xen wiki: Virtio On Xen
https://wiki.xenproject.org/wiki/Virtio_On_Xen

XCP-ng: IOREQ Server: Device Emulation in the Xen Hypervisor
https://xcp-ng.org/blog/2020/06/03/device-emulation-in-the-xen-hypervisor/

DEMU
https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git

Video for this Design Session:
https://www.youtube.com/watch?v=Ib0OdpHZ-G0&list=PLYyw7IQjL-zGcRPN6EjiTuFVGo4A6KCNf&index=29

--0000000000004a724505cb5de215
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Design Session notes for: VirtIO Cross-Project BoF (Birds =
of a Feather) for Xen and Guest OS (Linux, Windows, FreeBSD) developers<br>=
---------------------------<br>Xen Design &amp; Developer Summit, 27th May =
2021<br>Session Host: Juergen Gross<br>Notes by: Christopher Clark, with th=
anks to Rich Persaud<br><br>Apologies for the delay in posting these, solel=
y my responsibility.<br>- Christopher<div><br><div>=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D<br>Session Context:<div><br></div><div>There are three separate rece=
nt approaches within the Xen community to enabling<br>use of VirtIO device =
drivers in guest virtual machines with the Xen hypervisor,<br>and a fourth =
older completed project.<br>(Placeholder names are assigned to each of thes=
e below for ease of reference.)<br><br>In addition, Linaro has an active pr=
oject &#39;Stratos&#39; pursuing:<br>=C2=A0 =C2=A0 &quot;Establish virtio a=
s the standard interface between hypervisors,<br>=C2=A0 =C2=A0 freeing a mo=
bile, industrial or automotive platform to migrate between<br>=C2=A0 =C2=A0=
 hypervisors and reuse the backend implementation.&quot;<br><br>=C2=A0 =C2=
=A0 - <a href=3D"https://linaro.atlassian.net/wiki/spaces/STR/overview">htt=
ps://linaro.atlassian.net/wiki/spaces/STR/overview</a><br>=C2=A0 =C2=A0 - <=
a href=3D"https://projects.linaro.org/projects/STR/summary">https://project=
s.linaro.org/projects/STR/summary</a><br>=C2=A0 =C2=A0 - <a href=3D"https:/=
/op-lists.linaro.org/mailman/listinfo/stratos-dev">https://op-lists.linaro.=
org/mailman/listinfo/stratos-dev</a><br><br>--------------------<br>* &#39;=
VirtIO-EPAM&#39;: enabling existing VirtIO-MMIO transport on Xen, using for=
eign<br>mappings and an IOREQ server.<br>by EPAM and others, with focus on =
Xen on Arm platforms.<br><br>Enables use of the existing standardized VirtI=
O-MMIO transport driver, which is<br>present in the mainline Linux kernel, =
using Xen&#39;s IOREQ emulation<br>infrastructure and use of privileged for=
eign mappings to establish shared<br>memory for access to guest data by the=
 device model backend.<br><br>Status: Patches are in progress towards Xen o=
n the xen-devel mailing list.<br>The presentation at Linaro Connect 2021 in=
cludes a working demonstration.<br><br>VirtIO on Xen hypervisor (Arm), Olek=
sandr Tyshchenko, EPAM, Linaro Connect 2021<br><a href=3D"https://static.li=
naro.org/connect/lvc21/presentations/lvc21-314.pdf">https://static.linaro.o=
rg/connect/lvc21/presentations/lvc21-314.pdf</a><br><a href=3D"https://www.=
youtube.com/watch?v=3DXE5Rn8KFunk">https://www.youtube.com/watch?v=3DXE5Rn8=
KFunk</a><br><br>--------------------<br>* &#39;VirtIO-SuSE&#39;: introduci=
ng a new VirtIO transport driver that uses Xen grants.<br>by SuSE, presente=
d at this Xen Summit.<br><br>A new VirtIO transport device driver is added =
to the guest kernel, to<br>translate guest physical addresses into grant re=
ferences, enabling VirtIO data<br>path communication over mutually-negociat=
ed shared memory regions between the<br>guest virtual machine and the devic=
e model backend. Improves isolation as<br>backend does not need privilege o=
ver the guest to perform foreign mappings.<br>Grant references are a Xen-sp=
ecific interface. Design supports driver domains.<br><br>Status: A prototyp=
e is described in the presentation at this Xen Summit 2021.<br><br>VirtIO a=
nd Xen with Full Grant Support:<br><a href=3D"https://static.sched.com/host=
ed_files/xen2021/bf/Thursday_2021-Xen-Summit-virtio.pdf">https://static.sch=
ed.com/hosted_files/xen2021/bf/Thursday_2021-Xen-Summit-virtio.pdf</a><br><=
a href=3D"https://www.youtube.com/watch?v=3DIrlEdaIUDPk&amp;list=3DPLYyw7IQ=
jL-zGcRPN6EjiTuFVGo4A6KCNf&amp;index=3D25">https://www.youtube.com/watch?v=
=3DIrlEdaIUDPk&amp;list=3DPLYyw7IQjL-zGcRPN6EjiTuFVGo4A6KCNf&amp;index=3D25=
</a><br><br>--------------------<br>* VirtIO-Argo: introducing a new VirtIO=
 transport driver that uses Argo for<br>interdomain communication, supporti=
ng isolation and Mandatory Access Control.<br>Design and analysis performed=
 within the OpenXT and Xen communities.<br><br>A new VirtIO transport devic=
e driver is added to the guest kernel to transmit<br>data between the guest=
 domain and the domain hosting the device model via Argo<br>rings: a Hyperv=
isor-Mediated data eXchange protocol where the hypervisor<br>transfers the =
data, being trusted to strictly adhere to the delivery protocol.<br>Support=
s stronger isolation properties and enforcement of Mandatory Access<br>Cont=
rol security policy over interdomain communication. Does not use shared<br>=
memory between domains.<br>Development of a Hypervisor-agnostic interface f=
or Argo has been proposed<br>and discussed within the Xen community. Design=
 supports driver domains.<br><br>Status: Design and analysis published; fun=
ding required for development to<br>proceed.<br><br>VirtIO-Argo: Documentat=
ion at the OpenXT wiki:<br><a href=3D"https://openxt.atlassian.net/wiki/spa=
ces/DC/pages/1348763698/VirtIO%2BArgo">https://openxt.atlassian.net/wiki/sp=
aces/DC/pages/1348763698/VirtIO%2BArgo</a><br>VirtIO-Argo Development:<br><=
a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/Virt=
IO-Argo%2BDevelopment%2BPhase%2B1">https://openxt.atlassian.net/wiki/spaces=
/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1</a><br>Minutes f=
rom the Argo HMX Transport for VirtIO topic call, 14th January 2021:<br><a =
href=3D"https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg014=
22.html">https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01=
422.html</a><br><br>--------------------<br>* &#39;VirtIO-Xen-GSoC&#39;: 20=
11 VirtIO on Xen student project<br>A Google Summer of Code project by Wei =
Liu investigated enabling VirtIO on Xen.<br><br>A working prototype was pro=
duced for both PV and HVM guests, using XenBus and<br>the Qemu VirtIO backe=
nds. PV guests require a guest kernel patch to translate<br>guest physical =
addresses to machine addresses in VirtIO rings.<br><br>Status: project comp=
leted.<br><br><a href=3D"https://wiki.xenproject.org/wiki/Virtio_On_Xen">ht=
tps://wiki.xenproject.org/wiki/Virtio_On_Xen</a><br><br>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D<br>Summary of the VirtIO Design Session:<br><br>xl toolstack g=
uest config file syntax for VirtIO devices:<br>=C2=A0 =C2=A0 - Recommend: m=
ix of: device-specific config (eg. disk, net) plus support for<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0generic VirtIO devic=
e config (eg. esoteric devices)<br><br>VirtIO spec:<br>=C2=A0 =C2=A0 - spec=
 change needed? understood as not mandatory<br>=C2=A0 =C2=A0 - v1.1 has a p=
latform feature for DMA addr translation: enables transport<br>=C2=A0 =C2=
=A0 =C2=A0 driver to use any of grants, pre-shared memory, Argo in the data=
 path<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - ie. does not force use of (guest) ph=
ysical addrs in the ring<br>=C2=A0 =C2=A0 - open question re: Linux and qem=
u acceptance of non-standardized driver<br><br>Guest access to new VirtIO t=
ransport<br>=C2=A0 =C2=A0 - for Argo or grants, add new transport driver (e=
g. out of tree module)<br><br>Performance, standardization<br>=C2=A0 =C2=A0=
 - different characteristics on different architectures; motivates<br>=C2=
=A0 =C2=A0 =C2=A0 development and deployment of different transports for se=
parate cases<br>=C2=A0 =C2=A0 - difficulty of ensuring correctness with sha=
red memory vs. performance<br>=C2=A0 =C2=A0 =C2=A0 achievable with some con=
figurations<br><br>Atomics in shared memory, Arm, &quot;Fat Virtqueue&quot;=
 development<br>=C2=A0 =C2=A0 - Arm and RISC-V have challenges with atomic =
inst in shared memory<br>=C2=A0 =C2=A0 - VirtIO-SuSE (using grants) useful =
enough for consideration<br>=C2=A0 =C2=A0 - Wind River preshared-memory mem=
cpy approach (OpenAMP/kvmtool)<br>=C2=A0 =C2=A0 - Linaro &quot;Fat Virtqueu=
e&quot; under development: pass data within enlarged rings<br><br>Data copi=
es with Argo and shared memory transports<br>=C2=A0 =C2=A0 - discussion of =
copies made by different transports, different conditions<br>=C2=A0 =C2=A0 =
- data copies performed to protect correctness in communication<br><br>Ecos=
ystems, differentiation<br>=C2=A0 =C2=A0 - standarization of VirtIO shifted=
 economics for hypervisor developers<br>=C2=A0 =C2=A0 - Mandatory Access Co=
ntrol with Xen technologies will be a Xen advantage<br><br><br>=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D<br>Detailed notes for the Design Session:<br><br>Rich (O=
penXT): introducing session; to talk about:<br>=C2=A0 =C2=A0 - how any solu=
tion will be ratified by the VirtIO/OASIS community<br>=C2=A0 =C2=A0 =C2=A0=
 =C2=A0 - esp automotive use cases: a lot of attention paid to formal<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 acceptance of any VirtIO solution<br><br=
>=C2=A0 =C2=A0 - non-Linux guest VMs<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - Windo=
ws is used in OpenXT, Qubes<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - FreeBSD is pop=
ular for networking and storage<br><br>Roger (Citrix; x86 maintainer) [chat=
]: I wanted to ask whether the Linux kernel<br>/QEMU code could be accepted=
 without any spec change?<br><br>--- topic: xl toolstack guest config file =
syntax for VirtIO devices<br><br>Juergen (SUSE; PV interfaces maintainer): =
shared window with VM config:<br>------------------------<br>disk=3D[ &#39;=
file:/home/vm/upstream/image,hda,w&#39;, ]<br>vif=3D[ &#39;mac:00:16:3e:06:=
a7:21,bridge=3Dbr0&#39;, ]<br>vfb=3D[ &#39;vnclisten=3Dlocalhost:0&#39;, ]<=
br># device_model_args=3D[ &#39;-drive&#39;, &#39;file=3D/home/vm/sle15sp3/=
image,if=3Dvirtio,index=3D1,media=3Ddisk,format=3Draw,cache=3Dwriteback&#39=
; ]<br>device_model_args=3D[ &#39;-drive&#39;, &#39;file=3D/home/vm/sle15sp=
3/image,if=3Dnone,id=3Ddrive-virtio-disk0,format=3Draw&#39;, &#39;-device&#=
39;, &#39;virtio-blk-pci,scsi=3Doff,drive=3Ddrive-virtio-disk0,id=3Dvirtio-=
disk0,bootindex=3D0,disable-legacy=3Don&#39; ]<br>------------------------<=
br><br>Juergen: Request for input on xl config file syntax for configuring =
VirtIO<br>devices<br>=C2=A0 =C2=A0 - see device_model_args in example (forc=
e VirtIO block to non-legacy mode)<br>=C2=A0 =C2=A0 - alt option: add a &#3=
9;virtio&#39; specifier to the disk line<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - s=
ee recent patch by Artem<br>=C2=A0 =C2=A0 - need to consider generic device=
s: non-network, non-disk device classes<br><br>Q: Should VirtIO disk be add=
ed under &#39;disk&#39;, or a new special VirtIO disk?<br><br>Julien (Amazo=
n; Arm maintainer):<br>=C2=A0 =C2=A0 - want to avoid having one vif, one fr=
amebuffer, one gps, etc.<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - so instead have g=
eneric device options (eg. block, gps)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 - ie. similar to Qemu<br><br>Marek (Qubes; Python bindings maint=
ainer):<br>=C2=A0 =C2=A0 - libvirt bindings perspective: Preference for the=
 other way<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - don&#39;t want duplicated eg. d=
isk handling, for the VirtIO case<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - unified =
handling for a device class is more convenient for API users<br><br>Stefano=
 (Xilinx; Arm maintainer):<br>=C2=A0 =C2=A0 - Disk is a special case: confi=
g already has a line for disk,<br>=C2=A0 =C2=A0 =C2=A0 already have a way t=
o specify different types - network may be similar<br>=C2=A0 =C2=A0 - alt m=
ethod is better for other devices, eg. virtio-rng<br>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 - for those, device_model_arg is probably the best<br><br>Andy (Citr=
ix, x86 maintainer):<br>=C2=A0 =C2=A0 - a lot of this is PCI-based, even if=
 non-DMA transport so:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - need to preserve su=
pport for multi-IOREQ servers<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - feasible to =
have VirtIO backends that are not Qemu<br>=C2=A0 =C2=A0 - so: don&#39;t tie=
 the configuration to Qemu<br><br>OleksandrT (EPAM) [chat]: Initially I cre=
ated a new &quot;vdisk&quot; property for<br>virto.disk, but was asked to r=
euse existing &quot;disk&quot; configuration in libxl<br>if possible.<br><a=
 href=3D"https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg01=
174.html">https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg0=
1174.html</a><br><br>Juergen: ack; esp for Arm, mmio-based config: so we sh=
ould be flexible.<br>Aiming to determine:<br>=C2=A0 1: Configuration format=
 for VirtIO devices in guests<br>=C2=A0 2: Surfacing VirtIO devices in gues=
t VMs:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - tie it to the PCI specification or =
not?<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - challenge if guests do not have PCI d=
etection<br><br>=C2=A0 =C2=A0 - Responses:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -=
 PCI is fairly standard<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - issue with mmio: h=
otplugging devices<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - ra=
nges to be configured =3D&gt; need number of devices in advance<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - hotplug already requires upfron=
t work, due to grants, events<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - PCI and mmio=
 are similar; choice depends on guest and ecosystem<br><br>George (Citrix; =
Xen Community Manager):<br>=C2=A0 =C2=A0 - makes sense for a case to standa=
rdize on Argo? <br>=C2=A0 =C2=A0 - Argo vs grant table vs shared memory : w=
hy duplicate the extra effort?<br>=C2=A0 =C2=A0 - how much effort duplicate=
d for multiple? is it pretty easy to swap them<br>=C2=A0 =C2=A0 =C2=A0 out?=
<br><br>Rich [chat]: Important for automotive, FuSA, security<br><br>Christ=
opher (Apertus/Star Lab; Argo maintainer):<br>=C2=A0 =C2=A0 - Options are n=
ot exclusive: VirtIO can use different transports per-device<br>=C2=A0 =C2=
=A0 - Transports orthogonal eg. Argo devices enumerable via the VirtIO-Argo=
<br>=C2=A0 =C2=A0 =C2=A0 driver; Argo enables Mandatory Access Control over=
 the data transport<br><br><br>--- topic: VirtIO spec change; addresses on =
VirtIO rings<br><br>Concern: requirement for a VirtIO spec change<br><br>- =
Discussion re: is spec change mandatory?<br>=C2=A0 =C2=A0 - may be needed: =
VirtIO spec says pass guest physical addr on ring<br>=C2=A0 =C2=A0 - may no=
t be: translation step in the transport layer;<br>=C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 enables translation so don&#39;t nee=
d to change spec<br>=C2=A0 =C2=A0 - is spec change needed for config negoti=
ation, to announce feature?<br>=C2=A0 =C2=A0 - response: no; transport driv=
er does enumerations, so up to the driver<br><br>Stefano: &quot;VirtIO&quot=
; means different things: spec, transport, drivers<br>=C2=A0 =C2=A0 - since=
 transports are part of the spec: VirtIO-PCI, VirtIO-MMIO, ...,<br>=C2=A0 =
=C2=A0 =C2=A0 adding one (VirtIO-Argo) means adding to the spec<br>Daniel: =
no ..<br><br>Juergen: VirtIO spec does not demand physical addresses on the=
 ring pages<br>=C2=A0 =C2=A0 - DMA addresses are allowed<br>=C2=A0 =C2=A0 -=
 eg. a special DMA mechanism, which happens to be grants, is fine.<br>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 - ie. We have a special DMA engine.<br>Andy: Yes. =
DMA addresses !=3D identity map onto physical or guest phys addresses<br><b=
r>Daniel: there is a spec for the transports: if implementing a PCI transpo=
rt, or<br>=C2=A0 a MMIO transport, has certain characteristics of how those=
 should behave;<br>=C2=A0 so the driver-to-transport interface is fixed.<br=
>=C2=A0 It doesn&#39;t fix the transport back, which was the analysis that =
Christopher and<br>=C2=A0 I were looking at.<br>=C2=A0 Fixed from front-end=
 perspective: interface between the driver and transport.<br>=C2=A0 If the =
transport presents as a PCI device transport, there&#39;s an interface that=
<br>=C2=A0 the driver expects that transport to provide.<br>=C2=A0 =C2=A0 =
=C2=A0- It doesn&#39;t fix how it happens from the transport back.<br><br>G=
eorge: Frontend VirtIO driver hands phys addr to frontend transport and the=
<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0frontend transport can convert it to eg. gra=
nt tables to do what it wants<br><br>Daniel: right<br>=C2=A0 =C2=A0 - plus =
more behaviours, esp for PCI: eg. structures expected to be there<br>=C2=A0=
 =C2=A0 =C2=A0 so the drivers can expect a common behaviour for any PCI tra=
nsport<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - transport is translation layer, can=
 handle dealing with actual<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 physical =
backend mechanism that you&#39;re going over<br><br>Stefano: so plan is to =
add new transport: would that need to be in the spec?<br>Daniel: no<br><br>=
<br>--- topic: Guest access to new VirtIO transport<br><br>George: VirtIO f=
rontend drivers would work with the new transport.<br>=C2=A0 =C2=A0 - a cur=
rent VM image won&#39;t have transport implemented<br>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 - have to update to a new version of kernel, or<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 - a new version of image to get new transport?<br><br>Daniel: We=
ll, that&#39;s one way; for Argo, can present as either a PCI transport or<=
br>a MMIO transport to the driver, so the driver understands how to interac=
t with<br>the transport; and the transport handles moving data back. For Ar=
go, we move<br>everything onto Argo rings. =C2=A0<br>To gain that on Linux,=
 can do an out of tree kernel module, and load that<br>without having to re=
compile the entire kernel (unless they don&#39;t have loadable<br>module su=
pport).<br>- responses:<br>=C2=A0 =C2=A0 - needs the frontend transport in =
the guest;<br>=C2=A0 =C2=A0 - so can&#39;t just use an existing image unmod=
ified -- need the compiled<br>=C2=A0 =C2=A0 =C2=A0 kernel driver loaded to =
make the transport available<br>=C2=A0 =C2=A0 - objective will be to add it=
 to the upstream Linux kernel<br><br>Andy: might be able to get a frontend =
Windows driver somewhere useful<br><br>Stefano: fewer changes to the spec a=
re better: looks like nothing, fantastic<br>Second is changes to the code; =
seems limited to transport driver, is a lot<br>better than having to change=
 all the frontend drivers<br><br>=C2=A0 =C2=A0 - Christopher: yes; point is=
 to keep all of those and should be able to<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0plug them into backend drivers in =
the Qemu implementation.<br>=C2=A0 =C2=A0 - Juergen: Yes. Actual backends u=
nchanged and only lower levels of<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0transport layer need some modifications.<br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 - ie. for all implementations of backend infrastructure:<br>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Qemu<br>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 - the kernel for the vhost backends<br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 - vhost-user, all the user daemons implementing=
 the backend<br>=C2=A0 =C2=A0 - Andy: sounds great overall<br><br>---- topi=
c: Performance<br><br>George: whether to use Argo or grant tables or someth=
ing else:<br>=C2=A0 =C2=A0 - Andy: think the answer is that we need both<br=
>=C2=A0 =C2=A0 - Juergen: Yes.<br>=C2=A0 =C2=A0 - Andy: there are people th=
at need to not have shared memory, and Argo is an<br>=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 option there, and there are people who will want share=
d memory<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 because its faster.<b=
r>=C2=A0 =C2=A0 - ...: Is it?<br>=C2=A0 =C2=A0 - Andy: Yes.<br><br>Damien T=
henot [chat]: Shared memory would be interesting for DPU too<br><br>Stefano=
: Performance is a good metric;<br>- Able to convince third party proprieta=
ry vendor to add a driver?<br>=C2=A0 =C2=A0 - some sort of shared memory us=
eful as a last resort, fallback:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - ie. easie=
r to say &quot;just add a memcpy&quot; and to copy to the pre-shared<br>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 memory region<br><br>George: Ian Pratt pres=
entation said:<br>=C2=A0 =C2=A0 - copying is faster than sharing memory<br>=
=C2=A0 =C2=A0 - sharing memory is so difficult that noone is smart enough t=
o do it, and we<br>=C2=A0 =C2=A0 =C2=A0 were all fools back in the 2000s to=
 think we were smart enough to do it<br><br>Christopher [chat]: <a href=3D"=
https://platformsecuritysummit.com/2018/speaker/pratt">https://platformsecu=
ritysummit.com/2018/speaker/pratt</a><br><br>Andy: Not using shared memory =
can get around a lot of common bugs, but if you<br>=C2=A0 =C2=A0 =C2=A0 kno=
w how to do shared memory, it is still faster than data copying in<br>=C2=
=A0 =C2=A0 =C2=A0 enough cases for it to be relevant, for us to care.<br>Ma=
rek: if you can avoid mapping and unmapping all the time<br>=C2=A0 =C2=A0 =
=C2=A0 eg. can use well known regions.<br><br>Andy: a lot of network perfor=
mance in XenServer is from not mapping<br>=C2=A0 =C2=A0 - dom0 constructs a=
 scatter-gather over the guest memory<br>=C2=A0 =C2=A0 =C2=A0 (a granted ar=
ea, so permissions work) - never actually touched<br>=C2=A0 =C2=A0 - Can&#3=
9;t do that with copying mechanism; would force a memcpy into a<br>=C2=A0 =
=C2=A0 =C2=A0 zero overhead transmit path<br><br>Stefano: measurements so f=
ar always demonstrated grant table is slower than any <br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0memcpy mechanisms<br>=C2=A0 =C2=A0 - Julien: related to at=
omic instructions, ref: the previous talk?<br>=C2=A0 =C2=A0 - Stefano: coul=
d be; all my latest measurements are on Arm.<br>=C2=A0 =C2=A0 - Andy: Diffe=
rent architectures will make massive differences here<br><br>Jason Andryuk =
(OpenXT) [chat]: @Andrew, you use netfront/back but your dom0 nic<br>=C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0scatter/gathers over the guest&#39;s granted=
 network<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0frames directly?<br>Andy [c=
hat]: Yeah. A consequence is that dom0 never touches the mapping, so<br>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0never sets the A bit, so you c=
an skip the TLB flush on the unmap<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0side and this is great for performance<br>Roger [chat]: Note y=
ou need to patch Xen to not do the flush on unmap if the A<br>=C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bit isn&#39;t set<br><br><br>---- topic=
: Atomics in shared memory, Arm, &quot;Fat Virtqueue&quot; development<br><=
br>Juergen: regarding Arm and shared pages: how does VirtIO and KVM on Arm =
work?<br><br>Julien: don&#39;t do atomics in shared page. Lesson learned a =
while ago.<br>- Xen has many uses of atomics in shared pages, eg. evtchn, e=
tc.<br>=C2=A0 For VirtIO, would be nicer to avoid doing the same again.<br>=
<br>Stefano: Could we require Arm 8.1? (Is 8.1 introduces the new atomics, =
or 8.2?)<br><br>Julien: Yes, but limited hardware available today<br>- late=
st may have it, but not Qemu<br>- looks like RISC-V may have the same issue=
s<br><br>Demi Marie (Qubes OS) [chat]: RISC-V only has 32-bit atomics<br><b=
r>Andy: VirtIO-Grant [ie. VirtIO-SuSE in these notes] may not be perfect fo=
r all<br>=C2=A0 =C2=A0 =C2=A0 future use cases and other options exist - bu=
t still enough cases that it<br>=C2=A0 =C2=A0 =C2=A0 is worth considering<b=
r><br>Demi Marie [chat]: Long-term fix is probably to change the guest-visi=
ble ABI to<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0not require atomics in shared memory<br><br>Julien: how can it be=
 used with other hypervisors?<br>- eg. for Argo, looking at something that =
can be used in other hypervisors<br>- not possible with grant table, becaus=
e of the problems I mentioned.<br>- want to avoid having a Xen-specific dri=
ver, ie. tying VirtIO transport to Xen<br>=C2=A0 due to use of grant table<=
br><br>Stefano: two related activities, both based on shared memory:<br>=C2=
=A0 =C2=A0 - one driven by Wind River: memcpy-based approach<br>=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 - presharing memory, and picking addresses off the shared=
 region<br><br>Stefano [chat]: This is WindRiver KVMtools that does shared =
memory for Virtio<br><a href=3D"https://github.com/OpenAMP/kvmtool">https:/=
/github.com/OpenAMP/kvmtool</a><br><br>Stefano: second approach is from Lin=
aro: &quot;Fat Virtqueue&quot;<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - increase th=
e ring size and pick addresses within the ring only<br>=C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 - So the shared memory region is only the ring, and t=
he buffers are<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 within t=
he ring. Pointer points to things within the ring.<br><br>George: summarizi=
ng:<br>- on Arm copying might be faster, and there are other advantages as =
well;<br>- on x86, things may be different<br>so:<br>- SuSE and Citrix may =
work on the grant table one, and <br>- Epam and Arm, etc. may work on the A=
rgo one<br>and people can understand why they&#39;re choosing which one.<br=
><br>George: Can you also do driver domain things with Argo?<br>Daniel: Yes=
<br><br><br>--- topic: Data copies with Argo and shared memory transports<b=
r><br>Stefano: with Argo is there a memcpy, or more than one memcpy? who wo=
uld do it?<br><br>Christopher: hypervisor does the copy for the transport. =
<br>- guest hypercall op to register a ring, allows hypervisor to blit data=
 in;<br>=C2=A0 =C2=A0 - can think of as a DMA-style operation to deliver da=
ta to the guest<br><br>Juergen: So for each I/O data, there are three copie=
s:<br>=C2=A0 =C2=A0 - first, into the ring, then by the hypervisor to the o=
ther domain,<br>=C2=A0 =C2=A0 =C2=A0 and then out of the ring of the other =
domain to the target address, right?<br>Andy: That&#39;s correct if both th=
e source and destination are buffering the I/O in<br>=C2=A0 =C2=A0 =C2=A0 a=
nd out of the transport.<br>Daniel: Right<br><br>Andy: Buffering is probabl=
y because the backend is normally shared memory.<br>If you&#39;ve got a gua=
rantee that it&#39;s not shared memory between frontend and<br>backend, the=
n neither the front or the back VirtIO driver need to buffer in and <br>out=
 of the transport<br><br>Juergen: ok, for a disk I/O: have user data in the=
 guest, want to write to disk<br>Arbitrary data in memory, needs to be writ=
ten to the Argo ring buffer, right?<br>That&#39;s copy one.<br>Andy: No; on=
ly destination has an Argo ring. Source just passes arbitrary<br>=C2=A0 =C2=
=A0 =C2=A0 addresses, like scatter-gather DMA.<br>Juergen: OK, so like gran=
t copy<br>Andy: Yes.<br><br>Julien: With Argo, are the pages always mapped =
in Xen?<br>=C2=A0 =C2=A0 - Christopher: Yes, for the rings the way that Arg=
o works at the moment is, <br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0yes, once you register a ring that mapping is retai=
ned so<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0the hypervisor can send to it when a hypercall is invoked.<br>=C2=A0 =
=C2=A0 - Julien: OK<br><br>Demi Marie: is it possible for Argo to copy memo=
ry from guest one userspace to<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 the ring in guest two?<br>=C2=A0 =C2=A0 - re: disk I/O: from ZFS issue tra=
cker: have to take copies because is not <br>=C2=A0 =C2=A0 =C2=A0 safe to o=
perate on data shared with untrusted parties<br>=C2=A0 =C2=A0 - So most cas=
es you&#39;re going to have to have a copy.<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
- Only exception: extremely carefully written data sources, destinations<br=
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - careful to only make loads exa=
ctly once<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - or tools like ever=
parse, or another tool that generates<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 formally-verified binary parsers, immune to double-fetch bug=
s<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 - so: prefer to just take the copy<br>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - because memcpy on modern processor=
s is very, very fast<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - especia=
lly while it&#39;s stayed in cache<br><br>George: Andy was saying: some peo=
ple definitely going to want Argo, because<br>it&#39;s copying instead of s=
hared memory, and double-fetch issues with shared<br>memory is definitely a=
n issue<br><br>Andy: Yes. A lot of reasons why using shared memory is hard =
to do correctly;<br>not impossible but it is hard.<br><br><br>--- topic: Co=
nfiguration syntax (continued)<br><br>Juergen: choice: generic virtio devic=
e configuration, vs individual<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0device-=
specific stuff, or just mix variants?<br><br>Roger [chat]: I think both<br>=
<br>Christopher: good to have generic + allow specialization if you care<br=
><br>Marek: Yes; eg. for network devices, might want a different copying me=
chanism if<br>you can avoid copies for network. Same may not be doable for =
disk, so might want<br>different mechanisms for disk and network devices.<b=
r><br>Juergen: Should VirtIO disk be labelled under the disk item of the gu=
est config,<br>or a specific virtio disk item? Or virtio-device type =3D di=
sk, or whatever.<br><br>Roger:<br>- keep the old disk options, expand to su=
pport VirtIO<br>- add a generic VirtIO option for more esoteric devices tha=
t we don&#39;t support in<br>=C2=A0 any other way.<br>Disks and vifs need a=
 type parameter to select VirtIO as backend.<br><br>Juergen: then have a tr=
ansport layer specification:<br>(eg. don&#39;t care, or grants, or Argo, or=
 whatever)<br><br>George: We have the same thing for PCI (strict, relaxed, =
...)<br>ie. Set a global default for xl, plus override in specific domain c=
onfig<br><br><br>--- topic: Interface to adapt to platform transport<br><br=
>Juergen: spec bit is: VIRTIO_F_ACCESS_PLATFORM feature bit<br>- indicates =
&quot;the device can be used on a platform where the device<br>=C2=A0 acces=
s to data memory is limited, and/or translated&quot;<br>=C2=A0 Exactly what=
 we want; works for grants, etc.<br>=C2=A0 =C2=A0 - is a generic bit from n=
ewer virtio spec, not a Xen-specific bit<br>=C2=A0 =C2=A0 - in current impl=
, assumes running on Xen and means grants, because<br>=C2=A0 =C2=A0 =C2=A0 =
assumes that everything is done for addresses going into the ring<br><br><b=
r>--- topic: Ecosystems, differentiation<br><br>Rich: On standardization of=
 VirtIO: it commoditized the driver interface - now<br>all hypervisors impl=
ement VirtIO because it&#39;s too expensive to do otherwise.<br>- using Arg=
o transport to add Mandatory Access Control (needed by many guests)<br>=C2=
=A0 is an advantage for Xen, since only with Xen will you get the strong<br=
>=C2=A0 guarantees from Argo. Especially with Hyperlaunch, and all the othe=
r<br>=C2=A0 integrity pieces.<br>=C2=A0 =C2=A0 - Challenging for other hype=
rvisors to do MAC transport for VirtIO.<br><br><br>End of session<br><br>=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>Open Issues / Technical questions:<br><br=
>- Method of discovery of available transport drivers, negociation which to=
 use<br>=C2=A0 =C2=A0 - OASIS/VirtIO Standardization required? (eg. feature=
 bits needed?)<br><br>- Upstreaming path for different pieces to: Linux ker=
nel, Qemu, elsewhere<br>=C2=A0 =C2=A0 - including enabling guest support fo=
r FreeBSD, Windows, etc.<br><br>- Development of VirtIO &quot;Fat Virtqueue=
s&quot;: how does this affect system design?<br><br>- Design for interactio=
n with guest virtual IOMMU<br>=C2=A0 =C2=A0 - ref: VirtIO-iommu<br><br>- VF=
IO for device model backends<br><br>- Support for complex VirtIO drivers: e=
g. framebuffers (shared memory regions)<br><br><br>=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D<br>Areas suitable for investment to support VirtIO on Xen:<br><br>- =
Development of the Hypervisor-agnostic interface for Argo<br>Ref: Xen and O=
penXT community discussion minutes:<br><a href=3D"https://lists.xenproject.=
org/archives/html/xen-devel/2021-02/msg01422.html">https://lists.xenproject=
.org/archives/html/xen-devel/2021-02/msg01422.html</a><br><br>- Development=
 of the VirtIO-Argo system<br>Project analysis and design documentation:<br=
>=C2=A0 <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/169616=
9985/VirtIO-Argo%2BDevelopment%2BPhase%2B1">https://openxt.atlassian.net/wi=
ki/spaces/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1</a><br>=
<br>- VirtIO-focussed test cases for the Xen Test Framework<br>=C2=A0 =C2=
=A0 - to provide coverage of the hypervisor interfaces used by VirtIO trans=
ports<br>=C2=A0 =C2=A0 - extend the framework capabilities to enable new te=
sts (eg. multi-VM cases)<br>=C2=A0 <a href=3D"https://xenbits.xen.org/docs/=
xtf/index.html">https://xenbits.xen.org/docs/xtf/index.html</a><br>=C2=A0 =
=C2=A0 - development of support for Arm platforms with XTF<br><br>- Rust an=
d Go languages toolstack development for Xen<br>=C2=A0 =C2=A0 - Investment =
in the Xen toolstack is needed to support hypervisor feature<br>=C2=A0 =C2=
=A0 =C2=A0 development. Toolstack support is necessary for enabling VirtIO =
guests.<br>=C2=A0 =C2=A0 - Work towards enabling Xen tools in Rust has star=
ted:<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"https://github.com/xcp-ng/xe=
nopsd-ng">https://github.com/xcp-ng/xenopsd-ng</a><br>=C2=A0 =C2=A0 - Work =
towards enabling Xen tools in Go has started:<br>=C2=A0 =C2=A0 =C2=A0 <a hr=
ef=3D"https://lists.xenproject.org/archives/html/xen-devel/2021-04/msg01744=
.html">https://lists.xenproject.org/archives/html/xen-devel/2021-04/msg0174=
4.html</a><br><br>- System image test integration with Yocto/Openembedded a=
nd Qemu<br>=C2=A0 =C2=A0 - Xen and the Xen Test Framework can already be ru=
n within Qemu for x86<br>=C2=A0 =C2=A0 =C2=A0 in the standard meta-virtuali=
zation environment; Arm platform support<br>=C2=A0 =C2=A0 =C2=A0 needs to b=
e completed, with coverage for VirtIO guests, and Continuous<br>=C2=A0 =C2=
=A0 =C2=A0 Integration enabled<br>=C2=A0 =C2=A0 <a href=3D"https://lists.yo=
ctoproject.org/g/meta-virtualization/message/6405">https://lists.yoctoproje=
ct.org/g/meta-virtualization/message/6405</a><br>=C2=A0 =C2=A0 - to evaluat=
e: VirtIO as a DISTRO_FEATURE or MACHINE_FEATURE<br><br>- Creation of docum=
entation for Argo<br>=C2=A0 =C2=A0 - implementation documentation exists:<b=
r>=C2=A0 =C2=A0 =C2=A0 <a href=3D"https://xenbits.xen.org/docs/4.15-testing=
/designs/argo.html">https://xenbits.xen.org/docs/4.15-testing/designs/argo.=
html</a><br>=C2=A0 =C2=A0 =C2=A0 <a href=3D"https://static.sched.com/hosted=
_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Cla=
rk%20-%20Xen%20Summit%202019.pdf">https://static.sched.com/hosted_files/xen=
summit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20X=
en%20Summit%202019.pdf</a><br>=C2=A0 =C2=A0 =C2=A0 <a href=3D"https://www.y=
outube.com/watch?v=3DcnC0Tg3jqJQ">https://www.youtube.com/watch?v=3DcnC0Tg3=
jqJQ</a><br>=C2=A0 =C2=A0 - user interface / client / kernel developer docu=
mentation is needed<br><br>- Development items for Argo:<br>=C2=A0 <a href=
=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/737345538/Argo+Hyperv=
isor-Mediated+data+eXchange+Development">https://openxt.atlassian.net/wiki/=
spaces/DC/pages/737345538/Argo+Hypervisor-Mediated+data+eXchange+Developmen=
t</a><br><br>- Performance profiling of VirtIO-on-Xen systems across hardwa=
re architectures<br><br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>Additional Refere=
nces:<br><br>VirtIO Specification v1.1, OASIS, 11th April 2019:<br><a href=
=3D"https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html">https:=
//docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html</a><br>VirtIO spe=
c maintainers repository:<br><a href=3D"https://github.com/oasis-tcs/virtio=
-spec">https://github.com/oasis-tcs/virtio-spec</a><br>Organization for the=
 Advancement of Structured Information Standards<br><a href=3D"https://www.=
oasis-open.org">https://www.oasis-open.org</a><br>ACPI Virtal I/O Translati=
on Table (VIOT) DRAFT v9, December 2020:<br><a href=3D"https://jpbrucker.ne=
t/virtio-iommu/viot/viot-v9.pdf">https://jpbrucker.net/virtio-iommu/viot/vi=
ot-v9.pdf</a><br><a href=3D"https://lore.kernel.org/linux-acpi/202106100751=
30.67517-2-jean-philippe@linaro.org/T/">https://lore.kernel.org/linux-acpi/=
20210610075130.67517-2-jean-philippe@linaro.org/T/</a><br><br>Linaro: Strat=
os Project: Auto / Industrial Demonstrable Milestones<br><a href=3D"https:/=
/linaro.atlassian.net/wiki/spaces/STR/pages/28503377401/Demonstrable+artifa=
cts">https://linaro.atlassian.net/wiki/spaces/STR/pages/28503377401/Demonst=
rable+artifacts</a><br><br>OpenXT: VirtIO and Argo:<br><a href=3D"https://o=
penxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO%2BArgo">https://=
openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO%2BArgo</a><br><=
br>OpenXT: VirtIO-Argo Development:<br><a href=3D"https://openxt.atlassian.=
net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo%2BDevelopment%2BPhase%2B1">=
https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo%2B=
Development%2BPhase%2B1</a><br><br>OpenXT: Analysis of Argo as a transport =
medium for VirtIO:<br><a href=3D"https://openxt.atlassian.net/wiki/spaces/D=
C/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO">https=
://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as=
+a+transport+medium+for+VirtIO</a><br><br>Virtio with Argo for Xen, Mandato=
ry Access Control<br>Meeting of Virtualization Experts Group of Automotive =
Grade Linux, 18th Aug, 2020<br><a href=3D"https://lists.automotivelinux.org=
/g/agl-dev-community/attachment/8595/0/Argo%20and%20VirtIO.pdf">https://lis=
ts.automotivelinux.org/g/agl-dev-community/attachment/8595/0/Argo%20and%20V=
irtIO.pdf</a><br><br>VirtIO on Xen hypervisor (Arm), Oleksandr Tyshchenko, =
EPAM, Linaro Connect 2021<br><a href=3D"https://static.linaro.org/connect/l=
vc21/presentations/lvc21-314.pdf">https://static.linaro.org/connect/lvc21/p=
resentations/lvc21-314.pdf</a><br><a href=3D"https://www.youtube.com/watch?=
v=3DXE5Rn8KFunk">https://www.youtube.com/watch?v=3DXE5Rn8KFunk</a><br><br>X=
en wiki: Argo<br><a href=3D"https://wiki.xenproject.org/wiki/Argo:_Hypervis=
or-Mediated_Exchange_(HMX)_for_Xen">https://wiki.xenproject.org/wiki/Argo:_=
Hypervisor-Mediated_Exchange_(HMX)_for_Xen</a><br><br>OpenXT: Argo: Hypervi=
sor-Mediated data eXchange: Development:<br><a href=3D"https://openxt.atlas=
sian.net/wiki/spaces/DC/pages/737345538/Argo%2BHypervisor-Mediated%2Bdata%2=
BeXchange%2BDevelopment">https://openxt.atlassian.net/wiki/spaces/DC/pages/=
737345538/Argo%2BHypervisor-Mediated%2Bdata%2BeXchange%2BDevelopment</a><br=
><br>Xen wiki: Virtio On Xen<br><a href=3D"https://wiki.xenproject.org/wiki=
/Virtio_On_Xen">https://wiki.xenproject.org/wiki/Virtio_On_Xen</a><br><br>X=
CP-ng: IOREQ Server: Device Emulation in the Xen Hypervisor<br><a href=3D"h=
ttps://xcp-ng.org/blog/2020/06/03/device-emulation-in-the-xen-hypervisor/">=
https://xcp-ng.org/blog/2020/06/03/device-emulation-in-the-xen-hypervisor/<=
/a><br><br>DEMU<br><a href=3D"https://xenbits.xen.org/gitweb/?p=3Dpeople/pa=
uldu/demu.git">https://xenbits.xen.org/gitweb/?p=3Dpeople/pauldu/demu.git</=
a><br><br>Video for this Design Session:<br><a href=3D"https://www.youtube.=
com/watch?v=3DIb0OdpHZ-G0&amp;list=3DPLYyw7IQjL-zGcRPN6EjiTuFVGo4A6KCNf&amp=
;index=3D29">https://www.youtube.com/watch?v=3DIb0OdpHZ-G0&amp;list=3DPLYyw=
7IQjL-zGcRPN6EjiTuFVGo4A6KCNf&amp;index=3D29</a><br></div></div></div></div=
>

--0000000000004a724505cb5de215--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 02:42:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 02:42:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180322.326914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNR3j-00066x-B6; Tue, 07 Sep 2021 02:42:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180322.326914; Tue, 07 Sep 2021 02:42:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNR3j-00066q-86; Tue, 07 Sep 2021 02:42:03 +0000
Received: by outflank-mailman (input) for mailman id 180322;
 Tue, 07 Sep 2021 02:42:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M3oQ=N5=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mNR3i-00066k-JU
 for xen-devel@lists.xen.org; Tue, 07 Sep 2021 02:42:02 +0000
Received: from mail-oi1-x22b.google.com (unknown [2607:f8b0:4864:20::22b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bacbc3a1-6530-4402-99d4-5ffb8b8e8b8b;
 Tue, 07 Sep 2021 02:42:00 +0000 (UTC)
Received: by mail-oi1-x22b.google.com with SMTP id q39so10977071oiw.12
 for <xen-devel@lists.xen.org>; Mon, 06 Sep 2021 19:42:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bacbc3a1-6530-4402-99d4-5ffb8b8e8b8b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=DW9xyVq0VMsOnG7pEIDynSXF5/evQSkaZLzLtVFxop8=;
        b=MYeiHb2HeVfr6ZyokGIU//eh9yOTOkLkp/PvkD60nIV4JNNddtRjL9TxKpAjVa5tUS
         T5G0Nhb33f0j6xoP5yIqx8wgfGn5cFoy8zb0JtJ5qYwG1Yp8ao19CH2/UVoPZjHJtz6k
         SIrMQuNabuaI65bwrtyLZ9A1yHM2rpCNFA+otPGstCW02Zax1VsJd926iy9WVyd/YPoD
         BaGAxD9WJJf8m188fypJuYTVeeyFMLC0kO+cg5HKuLyPpojsh/pwMM0FC4XgwZbrmQGQ
         wASM7bwRFAs0iMBvPiNqa4qJ0uNLtZ9+ASyFijT+t9aYMH76u797wCLvu7jKKFI2U325
         1A2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=DW9xyVq0VMsOnG7pEIDynSXF5/evQSkaZLzLtVFxop8=;
        b=YnMpDEpRf8JFVELI4RFrBeb26f2iAYmmedG9kFLRX8gejD+UuSO4YF1teNJWplJVdt
         +TQkqztBvxrlvo2zbSad9ni0pzfg5XMNb/cgTBkM03db29s4dgJjCc1ndDlqhpDfa5XY
         5V1Bc32oaXZndj+E9ZRnm3CRUByx65EDfTUamHBgDRTieVsqDN44hj1ysh5EokTDBYkX
         IrxXED9sX0T6/14rX5JeHeGUCAWE+rllCWclnx34x+jB5SNKo6QXwgHZoSVvFZ18UMBP
         1dzUyFTfzfKeGn1Gd3TSdfwgaas+l9ifbVUT21emrIXuUYe8GZDrVCAwFYsAM0a6ieDk
         5tww==
X-Gm-Message-State: AOAM532/6U4cR/V9hqj8wq0QIVaghyfNskzBDdI15XE0pt/A1WOSE/Pq
	z+hJtsd1dpCHSjxtq+SzO4R5q448dHi7ZEiLAE4=
X-Google-Smtp-Source: ABdhPJz5qwyNwRm9AtgL03IQFlT0nPkWIPMKhsYmaWd2teMLjVt7RoQQburO6AFCfN6vWU1lbvUjcSX6Rx6V6uTzTMI=
X-Received: by 2002:a54:4197:: with SMTP id 23mr1373584oiy.122.1630982520003;
 Mon, 06 Sep 2021 19:42:00 -0700 (PDT)
MIME-Version: 1.0
References: <87v94ldrqq.fsf@linaro.org> <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org> <20210906022356.GD40187@laputa>
In-Reply-To: <20210906022356.GD40187@laputa>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Mon, 6 Sep 2021 19:41:48 -0700
Message-ID: <CACMJ4GaJyAnguzAEH87DSNN_+GhEa5jRbw11hVj-yWMAXx8V7w@mail.gmail.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: "Alex Benn??e" <alex.bennee@linaro.org>, Wei Chen <Wei.Chen@arm.com>, 
	Paul Durrant <paul@xen.org>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
	virtio-dev@lists.oasis-open.org, Stefano Stabellini <sstabellini@kernel.org>, 
	Jan Kiszka <jan.kiszka@siemens.com>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	Juergen Gross <jgross@suse.com>, Julien Grall <julien@xen.org>, 
	Carl van Schaik <cvanscha@qti.qualcomm.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Stefan Hajnoczi <stefanha@redhat.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>, 
	Xen-devel <xen-devel@lists.xen.org>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>, 
	James McKenzie <james@bromium.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Rich Persaud <persaur@gmail.com>, Daniel Smith <dpsmith@apertussolutions.com>, 
	Jason Andryuk <jandryuk@gmail.com>, eric chanudet <eric.chanudet@gmail.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000bd5eb305cb5eb55e"

--000000000000bd5eb305cb5eb55e
Content-Type: text/plain; charset="UTF-8"

On Sun, Sep 5, 2021 at 7:24 PM AKASHI Takahiro via Stratos-dev <
stratos-dev@op-lists.linaro.org> wrote:

> Alex,
>
> On Fri, Sep 03, 2021 at 10:28:06AM +0100, Alex Benn??e wrote:
> >
> > AKASHI Takahiro <takahiro.akashi@linaro.org> writes:
> >
> > > Alex,
> > >
> > > On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:
> > >>
> > >> Stefan Hajnoczi <stefanha@redhat.com> writes:
> > >>
> > >> > [[PGP Signed Part:Undecided]]
> > >> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
> > >> >> > Could we consider the kernel internally converting IOREQ
> messages from
> > >> >> > the Xen hypervisor to eventfd events? Would this scale with
> other kernel
> > >> >> > hypercall interfaces?
> > >> >> >
> > >> >> > So any thoughts on what directions are worth experimenting with?
> > >> >>
> > >> >> One option we should consider is for each backend to connect to
> Xen via
> > >> >> the IOREQ interface. We could generalize the IOREQ interface and
> make it
> > >> >> hypervisor agnostic. The interface is really trivial and easy to
> add.
> > >> >> The only Xen-specific part is the notification mechanism, which is
> an
> > >> >> event channel. If we replaced the event channel with something
> else the
> > >> >> interface would be generic. See:
> > >> >>
> https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52
> > >> >
> > >> > There have been experiments with something kind of similar in KVM
> > >> > recently (see struct ioregionfd_cmd):
> > >> >
> https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/
> > >>
> > >> Reading the cover letter was very useful in showing how this provides
> a
> > >> separate channel for signalling IO events to userspace instead of
> using
> > >> the normal type-2 vmexit type event. I wonder how deeply tied the
> > >> userspace facing side of this is to KVM? Could it provide a common FD
> > >> type interface to IOREQ?
> > >
> > > Why do you stick to a "FD" type interface?
> >
> > I mean most user space interfaces on POSIX start with a file descriptor
> > and the usual read/write semantics or a series of ioctls.
>
> Who do you assume is responsible for implementing this kind of
> fd semantics, OSs on BE or hypervisor itself?
>
> I think such interfaces can only be easily implemented on type-2
> hypervisors.
>
> # In this sense, I don't think rust-vmm, as it is, cannot be
> # a general solution.
>
> > >> As I understand IOREQ this is currently a direct communication between
> > >> userspace and the hypervisor using the existing Xen message bus. My
> > >
> > > With IOREQ server, IO event occurrences are notified to BE via Xen's
> event
> > > channel, while the actual contexts of IO events (see struct ioreq in
> ioreq.h)
> > > are put in a queue on a single shared memory page which is to be
> assigned
> > > beforehand with xenforeignmemory_map_resource hypervisor call.
> >
> > If we abstracted the IOREQ via the kernel interface you would probably
> > just want to put the ioreq structure on a queue rather than expose the
> > shared page to userspace.
>
> Where is that queue?
>
> > >> worry would be that by adding knowledge of what the underlying
> > >> hypervisor is we'd end up with excess complexity in the kernel. For
> one
> > >> thing we certainly wouldn't want an API version dependency on the
> kernel
> > >> to understand which version of the Xen hypervisor it was running on.
> > >
> > > That's exactly what virtio-proxy in my proposal[1] does; All the
> hypervisor-
> > > specific details of IO event handlings are contained in virtio-proxy
> > > and virtio BE will communicate with virtio-proxy through a virtqueue
> > > (yes, virtio-proxy is seen as yet another virtio device on BE) and will
> > > get IO event-related *RPC* callbacks, either MMIO read or write, from
> > > virtio-proxy.
> > >
> > > See page 8 (protocol flow) and 10 (interfaces) in [1].
> >
> > There are two areas of concern with the proxy approach at the moment.
> > The first is how the bootstrap of the virtio-proxy channel happens and
>
> As I said, from BE point of view, virtio-proxy would be seen
> as yet another virtio device by which BE could talk to "virtio
> proxy" vm or whatever else.
>
> This way we guarantee BE's hypervisor-agnosticism instead of having
> "common" hypervisor interfaces. That is the base of my idea.
>
> > the second is how many context switches are involved in a transaction.
> > Of course with all things there is a trade off. Things involving the
> > very tightest latency would probably opt for a bare metal backend which
> > I think would imply hypervisor knowledge in the backend binary.
>
> In configuration phase of virtio device, the latency won't be a big matter.
> In device operations (i.e. read/write to block devices), if we can
> resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue
> is
> how efficiently we can deliver notification to the opposite side. Right?
> And this is a very common problem whatever approach we would take.
>
> Anyhow, if we do care the latency in my approach, most of virtio-proxy-
> related code can be re-implemented just as a stub (or shim?) library
> since the protocols are defined as RPCs.
> In this case, however, we would lose the benefit of providing "single
> binary"
> BE.
> (I know this is is an arguable requirement, though.)
>
> # Would we better discuss what "hypervisor-agnosticism" means?
>
> Is there a call that you could recommend that we join to discuss this and
the topics of this thread?
There is definitely interest in pursuing a new interface for Argo that can
be implemented in other hypervisors and enable guest binary portability
between them, at least on the same hardware architecture, with VirtIO
transport as a primary use case.

The notes from the Xen Summit Design Session on VirtIO Cross-Project BoF
for Xen and Guest OS, which include context about the several separate
approaches to VirtIO on Xen, have now been posted here:
https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00472.html

Christopher



> -Takahiro Akashi
>
>
>

--000000000000bd5eb305cb5eb55e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Sun, Sep 5, 2021 at 7:24 PM AKASHI=
 Takahiro via Stratos-dev &lt;<a href=3D"mailto:stratos-dev@op-lists.linaro=
.org">stratos-dev@op-lists.linaro.org</a>&gt; wrote:<br></div><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px soli=
d rgb(204,204,204);padding-left:1ex">Alex,<br>
<br>
On Fri, Sep 03, 2021 at 10:28:06AM +0100, Alex Benn??e wrote:<br>
&gt; <br>
&gt; AKASHI Takahiro &lt;<a href=3D"mailto:takahiro.akashi@linaro.org" targ=
et=3D"_blank">takahiro.akashi@linaro.org</a>&gt; writes:<br>
&gt; <br>
&gt; &gt; Alex,<br>
&gt; &gt;<br>
&gt; &gt; On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; Stefan Hajnoczi &lt;<a href=3D"mailto:stefanha@redhat.com" ta=
rget=3D"_blank">stefanha@redhat.com</a>&gt; writes:<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; &gt; [[PGP Signed Part:Undecided]]<br>
&gt; &gt;&gt; &gt; On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabel=
lini wrote:<br>
&gt; &gt;&gt; &gt;&gt; &gt; Could we consider the kernel internally convert=
ing IOREQ messages from<br>
&gt; &gt;&gt; &gt;&gt; &gt; the Xen hypervisor to eventfd events? Would thi=
s scale with other kernel<br>
&gt; &gt;&gt; &gt;&gt; &gt; hypercall interfaces?<br>
&gt; &gt;&gt; &gt;&gt; &gt; <br>
&gt; &gt;&gt; &gt;&gt; &gt; So any thoughts on what directions are worth ex=
perimenting with?<br>
&gt; &gt;&gt; &gt;&gt;=C2=A0 <br>
&gt; &gt;&gt; &gt;&gt; One option we should consider is for each backend to=
 connect to Xen via<br>
&gt; &gt;&gt; &gt;&gt; the IOREQ interface. We could generalize the IOREQ i=
nterface and make it<br>
&gt; &gt;&gt; &gt;&gt; hypervisor agnostic. The interface is really trivial=
 and easy to add.<br>
&gt; &gt;&gt; &gt;&gt; The only Xen-specific part is the notification mecha=
nism, which is an<br>
&gt; &gt;&gt; &gt;&gt; event channel. If we replaced the event channel with=
 something else the<br>
&gt; &gt;&gt; &gt;&gt; interface would be generic. See:<br>
&gt; &gt;&gt; &gt;&gt; <a href=3D"https://gitlab.com/xen-project/xen/-/blob=
/staging/xen/include/public/hvm/ioreq.h#L52" rel=3D"noreferrer" target=3D"_=
blank">https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public=
/hvm/ioreq.h#L52</a><br>
&gt; &gt;&gt; &gt;<br>
&gt; &gt;&gt; &gt; There have been experiments with something kind of simil=
ar in KVM<br>
&gt; &gt;&gt; &gt; recently (see struct ioregionfd_cmd):<br>
&gt; &gt;&gt; &gt; <a href=3D"https://lore.kernel.org/kvm/dad3d025bcf15ece1=
1d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/" rel=3D"norefe=
rrer" target=3D"_blank">https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0=
ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/</a><br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; Reading the cover letter was very useful in showing how this =
provides a<br>
&gt; &gt;&gt; separate channel for signalling IO events to userspace instea=
d of using<br>
&gt; &gt;&gt; the normal type-2 vmexit type event. I wonder how deeply tied=
 the<br>
&gt; &gt;&gt; userspace facing side of this is to KVM? Could it provide a c=
ommon FD<br>
&gt; &gt;&gt; type interface to IOREQ?<br>
&gt; &gt;<br>
&gt; &gt; Why do you stick to a &quot;FD&quot; type interface?<br>
&gt; <br>
&gt; I mean most user space interfaces on POSIX start with a file descripto=
r<br>
&gt; and the usual read/write semantics or a series of ioctls.<br>
<br>
Who do you assume is responsible for implementing this kind of<br>
fd semantics, OSs on BE or hypervisor itself?<br>
<br>
I think such interfaces can only be easily implemented on type-2 hypervisor=
s.<br>
<br>
# In this sense, I don&#39;t think rust-vmm, as it is, cannot be<br>
# a general solution.<br>
<br>
&gt; &gt;&gt; As I understand IOREQ this is currently a direct communicatio=
n between<br>
&gt; &gt;&gt; userspace and the hypervisor using the existing Xen message b=
us. My<br>
&gt; &gt;<br>
&gt; &gt; With IOREQ server, IO event occurrences are notified to BE via Xe=
n&#39;s event<br>
&gt; &gt; channel, while the actual contexts of IO events (see struct ioreq=
 in ioreq.h)<br>
&gt; &gt; are put in a queue on a single shared memory page which is to be =
assigned<br>
&gt; &gt; beforehand with xenforeignmemory_map_resource hypervisor call.<br=
>
&gt; <br>
&gt; If we abstracted the IOREQ via the kernel interface you would probably=
<br>
&gt; just want to put the ioreq structure on a queue rather than expose the=
<br>
&gt; shared page to userspace. <br>
<br>
Where is that queue?<br>
<br>
&gt; &gt;&gt; worry would be that by adding knowledge of what the underlyin=
g<br>
&gt; &gt;&gt; hypervisor is we&#39;d end up with excess complexity in the k=
ernel. For one<br>
&gt; &gt;&gt; thing we certainly wouldn&#39;t want an API version dependenc=
y on the kernel<br>
&gt; &gt;&gt; to understand which version of the Xen hypervisor it was runn=
ing on.<br>
&gt; &gt;<br>
&gt; &gt; That&#39;s exactly what virtio-proxy in my proposal[1] does; All =
the hypervisor-<br>
&gt; &gt; specific details of IO event handlings are contained in virtio-pr=
oxy<br>
&gt; &gt; and virtio BE will communicate with virtio-proxy through a virtqu=
eue<br>
&gt; &gt; (yes, virtio-proxy is seen as yet another virtio device on BE) an=
d will<br>
&gt; &gt; get IO event-related *RPC* callbacks, either MMIO read or write, =
from<br>
&gt; &gt; virtio-proxy.<br>
&gt; &gt;<br>
&gt; &gt; See page 8 (protocol flow) and 10 (interfaces) in [1].<br>
&gt; <br>
&gt; There are two areas of concern with the proxy approach at the moment.<=
br>
&gt; The first is how the bootstrap of the virtio-proxy channel happens and=
<br>
<br>
As I said, from BE point of view, virtio-proxy would be seen<br>
as yet another virtio device by which BE could talk to &quot;virtio<br>
proxy&quot; vm or whatever else.<br>
<br>
This way we guarantee BE&#39;s hypervisor-agnosticism instead of having<br>
&quot;common&quot; hypervisor interfaces. That is the base of my idea.<br>
<br>
&gt; the second is how many context switches are involved in a transaction.=
<br>
&gt; Of course with all things there is a trade off. Things involving the<b=
r>
&gt; very tightest latency would probably opt for a bare metal backend whic=
h<br>
&gt; I think would imply hypervisor knowledge in the backend binary.<br>
<br>
In configuration phase of virtio device, the latency won&#39;t be a big mat=
ter.<br>
In device operations (i.e. read/write to block devices), if we can<br>
resolve &#39;mmap&#39; issue, as Oleksandr is proposing right now, the only=
 issue is<br>
how efficiently we can deliver notification to the opposite side. Right?<br=
>
And this is a very common problem whatever approach we would take.<br>
<br>
Anyhow, if we do care the latency in my approach, most of virtio-proxy-<br>
related code can be re-implemented just as a stub (or shim?) library<br>
since the protocols are defined as RPCs.<br>
In this case, however, we would lose the benefit of providing &quot;single =
binary&quot;<br>
BE.<br>
(I know this is is an arguable requirement, though.)<br>
<br>
# Would we better discuss what &quot;hypervisor-agnosticism&quot; means?<br=
>
<br></blockquote><div>Is there a call that you could recommend that we join=
 to discuss this and the topics of this thread?</div><div>There is definite=
ly interest in pursuing a new interface for Argo that can be implemented in=
 other hypervisors and enable guest binary portability between them, at lea=
st on the same hardware architecture, with VirtIO transport as a primary us=
e case.</div><div><br></div><div>The notes from the Xen Summit Design Sessi=
on on VirtIO Cross-Project BoF for Xen and Guest OS, which include context =
about the several separate approaches to VirtIO on Xen, have now been poste=
d here:</div><div><a href=3D"https://lists.xenproject.org/archives/html/xen=
-devel/2021-09/msg00472.html">https://lists.xenproject.org/archives/html/xe=
n-devel/2021-09/msg00472.html</a><br></div><div><br></div><div>Christopher<=
/div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin=
g-left:1ex">
-Takahiro Akashi<br>
<br><br>
</blockquote></div></div>

--000000000000bd5eb305cb5eb55e--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 03:06:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 03:06:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180328.326925 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRRF-0000XF-CW; Tue, 07 Sep 2021 03:06:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180328.326925; Tue, 07 Sep 2021 03:06:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRRF-0000X8-8X; Tue, 07 Sep 2021 03:06:21 +0000
Received: by outflank-mailman (input) for mailman id 180328;
 Tue, 07 Sep 2021 03:06:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GNGS=N5=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNRRD-0000X2-NO
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 03:06:19 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::612])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8ae9452c-2b70-4c21-970a-d27f2542b225;
 Tue, 07 Sep 2021 03:06:16 +0000 (UTC)
Received: from DB6PR0601CA0021.eurprd06.prod.outlook.com (2603:10a6:4:7b::31)
 by AM8PR08MB6339.eurprd08.prod.outlook.com (2603:10a6:20b:317::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 03:06:15 +0000
Received: from DB5EUR03FT005.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:7b::4) by DB6PR0601CA0021.outlook.office365.com
 (2603:10a6:4:7b::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 03:06:14 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT005.mail.protection.outlook.com (10.152.20.122) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 03:06:14 +0000
Received: ("Tessian outbound 610b9df843d5:v105");
 Tue, 07 Sep 2021 03:06:14 +0000
Received: from fa98366dd5e0.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B5C8ABBC-0598-449B-9455-F5BE0C669D43.1; 
 Tue, 07 Sep 2021 03:06:08 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id fa98366dd5e0.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 03:06:08 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB4318.eurprd08.prod.outlook.com (2603:10a6:803:f8::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 03:05:50 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 03:05:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ae9452c-2b70-4c21-970a-d27f2542b225
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uRCisFhpggo291C6aobRFvXGwFA/5l47T6YR9tpJq30=;
 b=X6Jg0Q30d77huaZYeQ1swI0vNboAhgy+8r4dLxWetixqgyUlpH58x+Qfq0hjb+mZmeMxCirE1tmu3c01ZxSKc2cXEywbSFo3rlgMkIBC5hfQuzwxX4Dl0194YXNzRqlRteigj/U0CiC5cMunHLb9HmWqX/H7GIGJOnsfoZrm+ec=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nOqDZk45ztnl5sPmS/exIS4JdOkueecBfm903RmYtWQco20jmhz6LrS7zqMgYLucx4bqa9cllGxH9uBePzwk7JNwGt5veEnPBV3Rbx4zq0JWdWmf3QSS8w68nuACF4yHx+1Rsoa0XYq44EEPdXwP3NP0jTRmC/onE5xn721JuM8MnCHQQW4eBdgQkBHLr8CEhkbpBpW4L2sjjcdFqFzPbmyX0y1Suo0fC9h18lO3bJl0zLEj0B4p9NWUYelmQDv9bF3syfx7oyz8XMl4JWEshNVZARtbsVxWqQJzr+mh0y8URnJVmTCGIenMsd9QQdZytPLbnc5cVKpvmDDz2wpktA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uRCisFhpggo291C6aobRFvXGwFA/5l47T6YR9tpJq30=;
 b=jUh+OV2HjmuW8aCK9cDCwGrLCQqZtgf8zda0tA5hAj/rfTqN9YRt0ZghuqxaBa4oqscr6R+YmuQzoGKGQuUQYBAgDYHzjbQiZuiG3LG8mjbdZ4Sex0BNoFilzaOrVkJ/yFfsXl8VGBl7Zv1o8jA3t6SockLmxdolES6/JHqrNxuWDp1XFBRmg10UmszTil5tf/0c4LSRTSN1VT0gOo/F+XVCrdoaznLPc3thipnugI9gTr+/g1RPdtzpGUKmnWhDhmL+FI2x+Uh/Pc7xlLg0buVdH/EDBxAQyjLbfFCzJnqEjjosctXu6utHrqQc6LuUngkRZByAPLGDOxkj77CDYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uRCisFhpggo291C6aobRFvXGwFA/5l47T6YR9tpJq30=;
 b=X6Jg0Q30d77huaZYeQ1swI0vNboAhgy+8r4dLxWetixqgyUlpH58x+Qfq0hjb+mZmeMxCirE1tmu3c01ZxSKc2cXEywbSFo3rlgMkIBC5hfQuzwxX4Dl0194YXNzRqlRteigj/U0CiC5cMunHLb9HmWqX/H7GIGJOnsfoZrm+ec=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Julien Grall <julien@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
	nd <nd@arm.com>
Subject: RE: [PATCH v5 1/7] xen/arm: introduce new helper
 device_tree_get_meminfo
Thread-Topic: [PATCH v5 1/7] xen/arm: introduce new helper
 device_tree_get_meminfo
Thread-Index: AQHXmM2TtDweULKD40q5wirMZQfui6uO7RIAgAkLAqA=
Date: Tue, 7 Sep 2021 03:05:49 +0000
Message-ID:
 <VE1PR08MB52153FC6C86128102A1A2A05F7D39@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-2-penny.zheng@arm.com>
 <06395a17-24fb-35d5-8332-a6ca42dbc183@xen.org>
In-Reply-To: <06395a17-24fb-35d5-8332-a6ca42dbc183@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: ECEF68F408207442B32D4A429459FAB8.0
x-checkrecipientchecked: true
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: fa26699c-2db9-4240-c0b4-08d971ac742d
x-ms-traffictypediagnostic: VI1PR08MB4318:|AM8PR08MB6339:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM8PR08MB6339FB31B5EB63727D8F6CD7F7D39@AM8PR08MB6339.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:1122;OLM:1122;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AAKiH1ZB7CgwbFK/f4sAXCyZ70wABNBj+O8nlddF+0ANu6neF0Atpo11hF7TSbjDIj5w9nC7x2ki5mObY9laC0MWYxYPyXo4igogiHk0BPESBTmWObM8IehwGdFbIl7Hbycxpv9gj8Y0TyD1HaWKwkyybPyfqKfOIHzcC9mT3fHTYbnA9vAk3PRMgJQKVmUT9b12KlOEdsDo8qH6O8HKLbeqxSWiOvPibKps81c1sUMop0hqnr7dKAhWRD2YYuB2IT2bEKuPYWcvGDAvx/T53vtkOfTRF6snQrv9zWlDVNNCBqWH614itB7yg1CSNMgaPpqrIfeMof2/q1C6sCdfr/i8CJ91E/rLISUptpUrnml26SV/PuhnE7LCTnLu/E+R7Ztwu5kfpvXGwWxqQo9pw/cwa2i28I4/PtBKHYBqjshL5l1hD0jxLfJfOWXpgq/YFQV+fV/YCqk3r7xyZ2fY4CqNegaUH12jzRk+GXLoG/beQr5sy/Pv+mB3mT1M14MtC1Kxb6QFKQj6Ern2CW4dGEGUaXRieoAgj7Yy6cpeQihqrJrnEt3RB9SIoG4OpaQdQ1p/5g+YyJ0M/Z19sBjQ2AAKw8TmOX5snVUs4/y+CHppgv07L3QmdmhN6FRnwTzYFy0MtmLmy0uuzWjFBrwveO1uYoWiThaeuOIetymtfkomK+Q0WoVsrOccYWzeXXpN6um+2u9m+KT985uSuZF1pw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(38070700005)(186003)(64756008)(66556008)(66476007)(71200400001)(7696005)(26005)(38100700002)(5660300002)(52536014)(316002)(508600001)(9686003)(86362001)(83380400001)(8676002)(55016002)(76116006)(53546011)(6506007)(4326008)(66946007)(66446008)(33656002)(122000001)(110136005)(54906003)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4318
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT005.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8c28562c-8dd3-4b47-da8e-08d971ac657c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nj7zY7ECgQL+EsfUWWc3tgMGn41LW1p7lK2hhv5v6ZEsY11/2mcljdtrOVm1tOJozbXbmRB0iSeI7n/9Qe2JvqHDbk/p/4MEWMPyR4qkbMxmCezJk+7Qva+/57/JWWA0CPhwN0xbgPlHWJEJos+s51+np+7U08DtDO5PMuH2LqVKVed0KwNaJXFrJtOtlX3OX0N7wAjG79/vZQOskOF7O7t7oNv+sVGMu1laji4ynPuZ314pE1B0sEH0OyTUo7hGvlVEdkwOVnuaPBS97IghyTw0NUhbiBhVurKR6RRoh+WC6+DhIqHWNMO8kUEnJVuMVTeNAZLh95tA1EeG/J4rxqc4q8OCjGK3MayBRQNOn75UFcFbbKa2cLw45PxP55TQiV4g8h0eE2RZossW2YeIvoGQknWkecZFls/uvl1z5xsGnC/3+su/12Ogsa5sTTdg3y1LrAD5y++J1wnLr8j50wCPu1JsWfrWHajoSA/LkwktcfqI3Gi35P7mNZNntB/o7mjcvCQyYzNk9QnDNZv3UjPvibMR2/nAuzhQCzuuf/pEwcsKZGKHmuxTzLQ6pg796Wg7EOCdjuRnnJ+RaJEDJTI5jtBhyLVyYxyz6dwKzfI3FGxQJMj7YvTnxXRzjKi7AKhtrtT1XM9KkWK8Clh2LzzaZv7usiQjWZ0ry91A56wKTztK5LDrWpBxrQ2Y5u+uWyOIoNsb+2Anuxaz88mthA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(376002)(136003)(396003)(346002)(46966006)(36840700001)(70206006)(70586007)(5660300002)(53546011)(54906003)(356005)(86362001)(6506007)(81166007)(82310400003)(110136005)(82740400003)(55016002)(8936002)(316002)(8676002)(33656002)(4326008)(478600001)(47076005)(7696005)(2906002)(9686003)(336012)(83380400001)(52536014)(36860700001)(186003)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 03:06:14.7014
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fa26699c-2db9-4240-c0b4-08d971ac742d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT005.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6339

SGkgSnVsaWVuDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogSnVsaWVu
IEdyYWxsIDxqdWxpZW5AeGVuLm9yZz4NCj4gU2VudDogV2VkbmVzZGF5LCBTZXB0ZW1iZXIgMSwg
MjAyMSA0OjU3IFBNDQo+IFRvOiBQZW5ueSBaaGVuZyA8UGVubnkuWmhlbmdAYXJtLmNvbT47IHhl
bi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsNCj4gc3N0YWJlbGxpbmlAa2VybmVsLm9yZw0K
PiBDYzogQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgV2VpIENo
ZW4NCj4gPFdlaS5DaGVuQGFybS5jb20+OyBuZCA8bmRAYXJtLmNvbT4NCj4gU3ViamVjdDogUmU6
IFtQQVRDSCB2NSAxLzddIHhlbi9hcm06IGludHJvZHVjZSBuZXcgaGVscGVyDQo+IGRldmljZV90
cmVlX2dldF9tZW1pbmZvDQo+IA0KPiBIaSBQZW5ueSwNCj4gDQo+IE9uIDI0LzA4LzIwMjEgMTA6
NTAsIFBlbm55IFpoZW5nIHdyb3RlOg0KPiA+IEEgZmV3IGZ1bmN0aW9ucyBpdGVyYXRlIG92ZXIg
dGhlIGRldmljZSB0cmVlIHByb3BlcnR5IHRvIGdldCBtZW1vcnkNCj4gPiBpbmZvLCBsaWtlICJy
ZWciIG9yIHRoZSBsYXRlciAieGVuLHN0YXRpYy1tZW0iLCBzbyB0aGlzIGNvbW1pdCBjcmVhdGVz
DQo+ID4gYSBuZXcgaGVscGVyIGRldmljZV90cmVlX2dldF9tZW1pbmZvIHRvIGV4dHJhY3QgdGhl
IGNvbW1vbiBjb2Rlcy4NCj4gDQo+IFRoZSBjb21taXQgbWVzc2FnZSBuZWVkcyB0byBiZSB1cGRh
dGVkIGFzIHRoZSBwYXRjaCBoYXMgYmVlbiByZXNodWZmbGVkLg0KPiANCg0KWWVzISBJJ2xsIHJl
Y29uc3RydWN0Lg0KPiA+IFNpZ25lZC1vZmYtYnk6IFBlbm55IFpoZW5nIDxwZW5ueS56aGVuZ0Bh
cm0uY29tPg0KPiA+IC0tLQ0KPiA+ICAgeGVuL2FyY2gvYXJtL2Jvb3RmZHQuYyB8IDY4ICsrKysr
KysrKysrKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0tLS0tLQ0KPiA+ICAgMSBmaWxlIGNoYW5n
ZWQsIDM5IGluc2VydGlvbnMoKyksIDI5IGRlbGV0aW9ucygtKQ0KPiA+DQo+ID4gZGlmZiAtLWdp
dCBhL3hlbi9hcmNoL2FybS9ib290ZmR0LmMgYi94ZW4vYXJjaC9hcm0vYm9vdGZkdC5jIGluZGV4
DQo+ID4gNDc2ZTMyZTBmNS4uOGM4MWJlMzM3OSAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vYXJjaC9h
cm0vYm9vdGZkdC5jDQo+ID4gKysrIGIveGVuL2FyY2gvYXJtL2Jvb3RmZHQuYw0KPiA+IEBAIC02
Myw2ICs2Myw0NCBAQCB2b2lkIF9faW5pdCBkZXZpY2VfdHJlZV9nZXRfcmVnKGNvbnN0IF9fYmUz
MiAqKmNlbGwsDQo+IHUzMiBhZGRyZXNzX2NlbGxzLA0KPiA+ICAgICAgICpzaXplID0gZHRfbmV4
dF9jZWxsKHNpemVfY2VsbHMsIGNlbGwpOw0KPiA+ICAgfQ0KPiA+DQo+ID4gK3N0YXRpYyBpbnQg
X19pbml0IGRldmljZV90cmVlX2dldF9tZW1pbmZvKGNvbnN0IHZvaWQgKmZkdCwgaW50IG5vZGUs
DQo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IGNo
YXIgKnByb3BfbmFtZSwNCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdTMyIGFkZHJlc3NfY2VsbHMsIHUzMiBzaXplX2NlbGxzLA0KPiA+ICsgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2b2lkICpkYXRhKSB7DQo+ID4gKyAgICBj
b25zdCBzdHJ1Y3QgZmR0X3Byb3BlcnR5ICpwcm9wOw0KPiA+ICsgICAgdW5zaWduZWQgaW50IGks
IGJhbmtzOw0KPiA+ICsgICAgY29uc3QgX19iZTMyICpjZWxsOw0KPiA+ICsgICAgdTMyIHJlZ19j
ZWxscyA9IGFkZHJlc3NfY2VsbHMgKyBzaXplX2NlbGxzOw0KPiA+ICsgICAgcGFkZHJfdCBzdGFy
dCwgc2l6ZTsNCj4gPiArICAgIHN0cnVjdCBtZW1pbmZvICptZW0gPSBkYXRhOw0KPiA+ICsNCj4g
PiArICAgIHByb3AgPSBmZHRfZ2V0X3Byb3BlcnR5KGZkdCwgbm9kZSwgcHJvcF9uYW1lLCBOVUxM
KTsNCj4gPiArICAgIGlmICggIXByb3AgKQ0KPiA+ICsgICAgICAgIHJldHVybiAtRU5PRU5UOw0K
PiA+ICsNCj4gPiArICAgIGNlbGwgPSAoY29uc3QgX19iZTMyICopcHJvcC0+ZGF0YTsNCj4gPiAr
ICAgIGJhbmtzID0gZmR0MzJfdG9fY3B1KHByb3AtPmxlbikgLyAocmVnX2NlbGxzICogc2l6ZW9m
ICh1MzIpKTsNCj4gPiArDQo+ID4gKyAgICBmb3IgKCBpID0gMDsgaSA8IGJhbmtzICYmIG1lbS0+
bnJfYmFua3MgPCBOUl9NRU1fQkFOS1M7IGkrKyApDQo+ID4gKyAgICB7DQo+ID4gKyAgICAgICAg
ZGV2aWNlX3RyZWVfZ2V0X3JlZygmY2VsbCwgYWRkcmVzc19jZWxscywgc2l6ZV9jZWxscywgJnN0
YXJ0LCAmc2l6ZSk7DQo+ID4gKyAgICAgICAgLyogU29tZSBEVCBtYXkgZGVzY3JpYmUgZW1wdHkg
YmFuaywgaWdub3JlIHRoZW0gKi8NCj4gPiArICAgICAgICBpZiAoICFzaXplICkNCj4gPiArICAg
ICAgICAgICAgY29udGludWU7DQo+ID4gKyAgICAgICAgbWVtLT5iYW5rW21lbS0+bnJfYmFua3Nd
LnN0YXJ0ID0gc3RhcnQ7DQo+ID4gKyAgICAgICAgbWVtLT5iYW5rW21lbS0+bnJfYmFua3NdLnNp
emUgPSBzaXplOw0KPiA+ICsgICAgICAgIG1lbS0+bnJfYmFua3MrKzsNCj4gPiArICAgIH0NCj4g
PiArDQo+ID4gKyAgICBpZiAoIGkgPCBiYW5rcyApDQo+ID4gKyAgICB7DQo+ID4gKyAgICAgICAg
cHJpbnRrKCJXYXJuaW5nOiBNYXggbnVtYmVyIG9mIHN1cHBvcnRlZCBtZW1vcnkgcmVnaW9ucw0K
PiByZWFjaGVkLlxuIik7DQo+ID4gKyAgICAgICAgcmV0dXJuIC1FTk9TUEM7DQo+ID4gKyAgICB9
DQo+ID4gKyAgICByZXR1cm4gMDsNCj4gPiArfQ0KPiA+ICsNCj4gPiAgIHUzMiBfX2luaXQgZGV2
aWNlX3RyZWVfZ2V0X3UzMihjb25zdCB2b2lkICpmZHQsIGludCBub2RlLA0KPiA+ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IGNoYXIgKnByb3BfbmFtZSwgdTMyIGRmbHQp
DQo+ID4gICB7DQo+ID4gQEAgLTEzOSwxNCArMTc3LDYgQEAgc3RhdGljIGludCBfX2luaXQgcHJv
Y2Vzc19tZW1vcnlfbm9kZShjb25zdCB2b2lkDQo+ICpmZHQsIGludCBub2RlLA0KPiA+ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1MzIgYWRkcmVzc19jZWxscywgdTMy
IHNpemVfY2VsbHMsDQo+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHZvaWQgKmRhdGEpDQo+ID4gICB7DQo+ID4gLSAgICBjb25zdCBzdHJ1Y3QgZmR0X3Byb3BlcnR5
ICpwcm9wOw0KPiA+IC0gICAgaW50IGk7DQo+ID4gLSAgICBpbnQgYmFua3M7DQo+ID4gLSAgICBj
b25zdCBfX2JlMzIgKmNlbGw7DQo+ID4gLSAgICBwYWRkcl90IHN0YXJ0LCBzaXplOw0KPiA+IC0g
ICAgdTMyIHJlZ19jZWxscyA9IGFkZHJlc3NfY2VsbHMgKyBzaXplX2NlbGxzOw0KPiA+IC0gICAg
c3RydWN0IG1lbWluZm8gKm1lbSA9IGRhdGE7DQo+ID4gLQ0KPiA+ICAgICAgIGlmICggYWRkcmVz
c19jZWxscyA8IDEgfHwgc2l6ZV9jZWxscyA8IDEgKQ0KPiANCj4gVGhpcyBjaGVjayB3aWxsIGJl
IHRoZSBzYW1lIGZvciAieGVuLHN0YXRpYy1tZW0iLiBTbyBjYW4gaXQgYmUgbW92ZWQgdG8NCj4g
ZGV2aWNlX3RyZWVfZ2V0X21lbWluZm8oKT8NCj4gDQoNClN1cmUuIEknbGwgbW92ZSBpdC4NCg0K
PiA+ICAgICAgIHsNCj4gPiAgICAgICAgICAgcHJpbnRrKCJmZHQ6IG5vZGUgYCVzJzogaW52YWxp
ZCAjYWRkcmVzcy1jZWxscyBvcg0KPiA+ICNzaXplLWNlbGxzIiwgQEAgLTE1NCwyNyArMTg0LDcg
QEAgc3RhdGljIGludCBfX2luaXQNCj4gcHJvY2Vzc19tZW1vcnlfbm9kZShjb25zdCB2b2lkICpm
ZHQsIGludCBub2RlLA0KPiA+ICAgICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4gPiAgICAgICB9
DQo+ID4NCj4gPiAtICAgIHByb3AgPSBmZHRfZ2V0X3Byb3BlcnR5KGZkdCwgbm9kZSwgInJlZyIs
IE5VTEwpOw0KPiA+IC0gICAgaWYgKCAhcHJvcCApDQo+ID4gLSAgICAgICAgcmV0dXJuIC1FTk9F
TlQ7DQo+ID4gLQ0KPiA+IC0gICAgY2VsbCA9IChjb25zdCBfX2JlMzIgKilwcm9wLT5kYXRhOw0K
PiA+IC0gICAgYmFua3MgPSBmZHQzMl90b19jcHUocHJvcC0+bGVuKSAvIChyZWdfY2VsbHMgKiBz
aXplb2YgKHUzMikpOw0KPiA+IC0NCj4gPiAtICAgIGZvciAoIGkgPSAwOyBpIDwgYmFua3MgJiYg
bWVtLT5ucl9iYW5rcyA8IE5SX01FTV9CQU5LUzsgaSsrICkNCj4gPiAtICAgIHsNCj4gPiAtICAg
ICAgICBkZXZpY2VfdHJlZV9nZXRfcmVnKCZjZWxsLCBhZGRyZXNzX2NlbGxzLCBzaXplX2NlbGxz
LCAmc3RhcnQsICZzaXplKTsNCj4gPiAtICAgICAgICAvKiBTb21lIERUIG1heSBkZXNjcmliZSBl
bXB0eSBiYW5rLCBpZ25vcmUgdGhlbSAqLw0KPiA+IC0gICAgICAgIGlmICggIXNpemUgKQ0KPiA+
IC0gICAgICAgICAgICBjb250aW51ZTsNCj4gPiAtICAgICAgICBtZW0tPmJhbmtbbWVtLT5ucl9i
YW5rc10uc3RhcnQgPSBzdGFydDsNCj4gPiAtICAgICAgICBtZW0tPmJhbmtbbWVtLT5ucl9iYW5r
c10uc2l6ZSA9IHNpemU7DQo+ID4gLSAgICAgICAgbWVtLT5ucl9iYW5rcysrOw0KPiA+IC0gICAg
fQ0KPiA+IC0NCj4gPiAtICAgIGlmICggaSA8IGJhbmtzICkNCj4gPiAtICAgICAgICByZXR1cm4g
LUVOT1NQQzsNCj4gPiAtICAgIHJldHVybiAwOw0KPiA+ICsgICAgcmV0dXJuIGRldmljZV90cmVl
X2dldF9tZW1pbmZvKGZkdCwgbm9kZSwgInJlZyIsIGFkZHJlc3NfY2VsbHMsDQo+ID4gKyBzaXpl
X2NlbGxzLCBkYXRhKTsNCj4gPiAgIH0NCj4gPg0KPiA+ICAgc3RhdGljIGludCBfX2luaXQgcHJv
Y2Vzc19yZXNlcnZlZF9tZW1vcnlfbm9kZShjb25zdCB2b2lkICpmZHQsIGludA0KPiA+IG5vZGUs
DQo+ID4NCj4gDQo+IENoZWVycywNCj4gDQo+IC0tDQo+IEp1bGllbiBHcmFsbA0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 03:13:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 03:13:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180335.326936 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRYH-0002Fp-8H; Tue, 07 Sep 2021 03:13:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180335.326936; Tue, 07 Sep 2021 03:13:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRYH-0002Fi-5B; Tue, 07 Sep 2021 03:13:37 +0000
Received: by outflank-mailman (input) for mailman id 180335;
 Tue, 07 Sep 2021 03:13:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GNGS=N5=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNRYF-0002Fc-R4
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 03:13:35 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1b::631])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 645f2ba7-1609-4ee8-a0f3-771b2d8469d7;
 Tue, 07 Sep 2021 03:13:33 +0000 (UTC)
Received: from DB6PR07CA0172.eurprd07.prod.outlook.com (2603:10a6:6:43::26) by
 AS8PR08MB7208.eurprd08.prod.outlook.com (2603:10a6:20b:407::6) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.24; Tue, 7 Sep 2021 03:13:24 +0000
Received: from DB5EUR03FT034.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:43:cafe::f3) by DB6PR07CA0172.outlook.office365.com
 (2603:10a6:6:43::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4 via Frontend
 Transport; Tue, 7 Sep 2021 03:13:24 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT034.mail.protection.outlook.com (10.152.20.87) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 03:13:23 +0000
Received: ("Tessian outbound 0ec886cb54dd:v105");
 Tue, 07 Sep 2021 03:13:23 +0000
Received: from c3af3d769bd7.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C3890B58-DDC9-4EF5-BA66-F6A0E8520F84.1; 
 Tue, 07 Sep 2021 03:13:17 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c3af3d769bd7.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 03:13:17 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB4173.eurprd08.prod.outlook.com (2603:10a6:803:e8::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 03:13:14 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 03:13:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 645f2ba7-1609-4ee8-a0f3-771b2d8469d7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GxQnPRoj1/mbk5l4OOqDbjtc4ZfJJmMN5bdSpP8ux2w=;
 b=mx3dccJz0S3p5NwVMRCoJgfTlYTReqEs2VH3mvim4M52de1ZicpyaH6BwWKpN96xwekh7+LsnLh3ewrXt2HPVfqVlL/sR6RuzkuhIiEZ7V5ylt3gRDwrCqZdNJGFdPYsQ8RHNaVhxcljePXH8OwO0K6I3NTXFENFYt4fumaHlxw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Qk3BT03R+YajOifu6FCKdkdAwUkp6Ps9ohAHZHW9PWOUmh0arMmP0EI+JMIfBlcIPiNDh9fJqVXrGA0DmG9C5UIi3PJzIeJk61qetZihonRaHOm/ghMwhb4LCMvxLrDMtkrlbrwqmq4PoeFNdniCzxY5KmPmQPcBFZDoIzCe2qHcE8ijDG0EjDqFVTnbiCfrp1Hq5zAOi/gXoX2Gr2vQlPTe6wy53gJOFRx6/DFG39CoQPklp7Bc2T9sGpRj2ZvmtDvCOuBSw41MfNWWtp/mFxqQ8L9UYir2J83EhGZpmuqbbilsyMDsGbTzlROFwpsYCm835jEdAlOoiFPWNB4xqg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GxQnPRoj1/mbk5l4OOqDbjtc4ZfJJmMN5bdSpP8ux2w=;
 b=DTcW97IKgjYtuOeaojgbAEx+p49de08d/hfSb5rDtu8r/GruDrbPxfFdB1T9E8eM3st3s5ynTDwg3NYSa55lgbNGtFLWki4jzCogMQA9Hs+5ts3/0876Vho66CCHB1D6VwExAW9b9cD6bac4XVwWo2PSsQMrPZF2YJAZyEOs4pQzLeoMfJG5hBcJHLXXfFiWBvUV7O4/W67oIcY8psE3LatYhdszyfqAB64lgm5SFPDP0Kvh8wrD+dlJm//e2zmktdY1dW1hmT1bdJw5wFZxIPww1gActOnXkLNLQdvi2Cs5hVgVpPFeEkolyJPB+0S2ygCIR84Cu5EHg32/BNURjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GxQnPRoj1/mbk5l4OOqDbjtc4ZfJJmMN5bdSpP8ux2w=;
 b=mx3dccJz0S3p5NwVMRCoJgfTlYTReqEs2VH3mvim4M52de1ZicpyaH6BwWKpN96xwekh7+LsnLh3ewrXt2HPVfqVlL/sR6RuzkuhIiEZ7V5ylt3gRDwrCqZdNJGFdPYsQ8RHNaVhxcljePXH8OwO0K6I3NTXFENFYt4fumaHlxw=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, nd
	<nd@arm.com>
Subject: RE: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
Thread-Topic: [PATCH v5 7/7] xen/arm: introduce allocate_static_memory
Thread-Index: AQHXmM2d1FH9kUvkk0y8p4F7V64vPKuRUpWAgAAJw4CAACpGgIAAdgUAgAX8taA=
Date: Tue, 7 Sep 2021 03:13:13 +0000
Message-ID:
 <VE1PR08MB52157E2C65AF90195F4E1516F7D39@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-8-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109021352020.26072@sstabellini-ThinkPad-T480s>
 <ac3a9fbd-7598-e400-6c6c-1214df7acee7@xen.org>
 <alpine.DEB.2.21.2109021527330.26072@sstabellini-ThinkPad-T480s>
 <034628a6-5461-f4a9-3339-a0d6f3d68f80@xen.org>
In-Reply-To: <034628a6-5461-f4a9-3339-a0d6f3d68f80@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 3844190897E13D46905699D8AF8D48F0.0
x-checkrecipientchecked: true
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a7ca33b8-1ad3-4261-9ca9-08d971ad73ce
x-ms-traffictypediagnostic: VI1PR08MB4173:|AS8PR08MB7208:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB720835F6469E2658B9D8CE8EF7D39@AS8PR08MB7208.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GGYtz82v59GjVWRW1s3hlQz39B/+tL559MfJVajg2c61fNqmcMFZ+h7VHxkFb+YS6lkx8NuSiDL4a7QcHDI24nEEHt3Dou0pbnnzJVqlseByrTHf4PQqSHqI8wJ0XXSgNyGhilNikSHbs8hhOmJCFQzFUpEjfz88jfAMgvmKeGqBcmQq0s/qpoc/JFNepfzkw9xSawfPPwT3QrKhVujfEMIkCzpTZkC8ct1YVVz+tv1yB76qg7rEDNAgwrVkeZWgPVCPLKxU4+cpR3wQIOgrmQei8WG/ErDfi+PCAgQBrlR5NOVk2i/67L0tUYWswnI7CXbjLgmiMddqKhEdUNNDb2paxKgoVJEsgrwuExba0PFbo78nggPBfhLxGAdlg6Ouq0rydF+U4LM666JUHd80kY8x0tRnNuSdbyEVeT+dM7dIqPjmLNvtt4936rDVIF4Y3WisFvWrfHdnxgTETUC3zhjXia6uZt1Ct37va5rWOGjjBydZIuxjFsuCk78XviGrui0LRmQheUgWr92P4ZyR/eApz98HYi/hOC7vHH5Km+JlrEzrqRlL3AApUHVKA6Exlu/ySgbkil1xtcZeLnTDal0P2gtoPxOlnV+GBTycsOUp64a4KMhJGoShjrPm/H3hc74799PyM0WCqJh/pLAocr+FiUpK3udt7oL27vsHS/FkOeOJA9gxI0LBkfxuUUoiHG1VbBVHRTHAr0J4PHBgSg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(366004)(346002)(376002)(110136005)(83380400001)(38070700005)(8936002)(9686003)(52536014)(66476007)(76116006)(4326008)(66946007)(478600001)(5660300002)(38100700002)(71200400001)(33656002)(316002)(8676002)(66446008)(66556008)(64756008)(122000001)(26005)(6506007)(54906003)(2906002)(53546011)(186003)(55016002)(7696005)(86362001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4173
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7e4c8623-abfa-4512-b479-08d971ad6e47
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CgfayPYRC7LdigTivWMEPAEHbQimkfuCOY4UEQcMqL5tkZlvghb6l+lG/iZMGFfkl0L5kTlVVN2pNIeRP8d+fjfSrZtrZlOWfYoWXoUTJQgyZHMbnFKv0BIHAPyaLoNHVXtL7KqumLsYqC6LrYp1ZUhOgttniZBqykYoJ0jiJcrSaKT7YTMEHjDkXkfVxrthYT0gGvO7W3UkytjrbOPGW7H/TVoGadEFQ8kTw6GqOaeQq0cgFqBsYy0IO6f7BQWsboFwTmVnHPWZ3qJeFI44DxoSuKse14P7QGPwRpRv8cOQF3QN1mvW/p44wBAWNJ4rit+QQGwriUgm9Y6+L1Ni6FNmMxxdozI9Zj10WaK712gBAb1TKNsD9iPvfLX271QeMpBYjLQNN+NKwhJ4BmyHJDR2Pa910pYj2s4khE2rqDOauD+UfPReVhvIyWa0RRmyT3iFL9D/a5V4FB2A+3s5wRRCWgM+TqgUPgZoD1L+aaoy/b5lLQfOIgQC5nrSnnTYx7uJiSaDSuyLhU4V0hB+8sBX9hSGlQiBVXor3yw+4qw1OiDmIpGDxosy75ru5qQTQhKYZj4Xng5bzNxGKGdE7Ybl4UzR73pVLk5E/otTZ9OdOFIu/QeM0X41t5ysUlDs5G3T+3SMRR5Ri6AqJKIvjSc8JjiSPRB1q/pZFohT43fl4uz8nY5ggYOkvZ7t41PcGSSR7E6iuXZqotC5RS8JkQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(346002)(376002)(396003)(39860400002)(36840700001)(46966006)(356005)(33656002)(110136005)(55016002)(6506007)(70206006)(336012)(86362001)(8676002)(9686003)(26005)(70586007)(54906003)(5660300002)(186003)(8936002)(82740400003)(81166007)(53546011)(83380400001)(4326008)(478600001)(2906002)(82310400003)(52536014)(316002)(36860700001)(47076005)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 03:13:23.5757
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a7ca33b8-1ad3-4261-9ca9-08d971ad73ce
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7208

SGkgSnVsaWVuIGFuZCBTdGVmYW5vDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4g
RnJvbTogSnVsaWVuIEdyYWxsIDxqdWxpZW5AeGVuLm9yZz4NCj4gU2VudDogRnJpZGF5LCBTZXB0
ZW1iZXIgMywgMjAyMSAzOjQyIFBNDQo+IFRvOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxs
aW5pQGtlcm5lbC5vcmc+DQo+IENjOiBQZW5ueSBaaGVuZyA8UGVubnkuWmhlbmdAYXJtLmNvbT47
IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVy
dHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgV2VpIENoZW4NCj4gPFdlaS5DaGVuQGFybS5jb20+OyBu
ZCA8bmRAYXJtLmNvbT4NCj4gU3ViamVjdDogUmU6IFtQQVRDSCB2NSA3LzddIHhlbi9hcm06IGlu
dHJvZHVjZSBhbGxvY2F0ZV9zdGF0aWNfbWVtb3J5DQo+IA0KPiANCj4gDQo+IE9uIDAzLzA5LzIw
MjEgMDE6MzksIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4gPiBPbiBUaHUsIDIgU2VwIDIw
MjEsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gPj4+PiArICAgIGtpbmZvLT5tZW0ubnJfYmFua3Mg
PSArK2diYW5rOw0KPiA+Pj4+ICsgICAga2luZm8tPnVuYXNzaWduZWRfbWVtIC09IHRvdF9zaXpl
Ow0KPiA+Pj4+ICsgICAgaWYgKCBraW5mby0+dW5hc3NpZ25lZF9tZW0gKQ0KPiA+Pj4+ICsgICAg
ew0KPiA+Pj4+ICsgICAgICAgIHByaW50ayhYRU5MT0dfRVJSDQo+ID4+Pj4gKyAgICAgICAgICAg
ICAgICJTaXplIG9mIFwibWVtb3J5XCIgcHJvcGVydHkgZG9lc24ndCBtYXRjaCB1cCB3aXRoDQo+
ID4+Pj4gKyB0aGUNCj4gPj4+PiBzdW0tdXAgb2YgXCJ4ZW4sc3RhdGljLW1lbVwiLlxuIik7DQo+
ID4+Pj4gKyAgICAgICAgZ290byBmYWlsOw0KPiA+Pj4NCj4gPj4+IERvIHdlIG5lZWQgdG8gbWFr
ZSB0aGlzIGEgZmF0YWwgZmFpbHVyZT8NCj4gPj4+DQo+ID4+PiBJIGFtIGFza2luZyBiZWNhdXNl
IHVuYXNzaWduZWRfbWVtIGNvbWVzIGZyb20gdGhlICJtZW1vcnkiIHByb3BlcnR5DQo+ID4+PiBv
ZiB0aGUgZG9tYWluIGluIGRldmljZSB0cmVlIHdoaWNoIGlzIGtpbmQgb2YgcmVkdW5kYW50IHdp
dGggdGhlDQo+ID4+PiBpbnRyb2R1Y3Rpb24gb2YgeGVuLHN0YXRpYy1tZW0uIEluIGZhY3QsIEkg
dGhpbmsgaXQgd291bGQgYmUgYmV0dGVyDQo+ID4+PiB0byBjbGFyaWZ5IHRoZSByb2xlIG9mICJt
ZW1vcnkiIHdoZW4gInhlbixzdGF0aWMtbWVtIiBpcyBhbHNvIHByZXNlbnQuDQo+ID4+PiBJbiB0
aGF0IGNhc2UsIHdlIGNvdWxkIGV2ZW4gbWFrZSAibWVtb3J5IiBvcHRpb25hbC4NCj4gPj4NCj4g
Pj4gSSByZXF1ZXN0ZWQgdG8gbWFrZSBpdCBtYW5kYXRvcnkuIEltYWdpbmUgeW91IGhhdmUgYSBk
b21VIHRoYXQgaGFzDQo+ID4+IDFHQiBhbmQgbm93IHlvdSB3YW50IHRvIHN3aXRjaCB0byBzdGF0
aWMgbWVtb3J5LiBJZiB3ZSBtYWtlIHRoZQ0KPiA+PiBwcm9wZXJ0eSBvcHRpb25hbCwgdGhlbiB0
aGVyZSBpcyBhIHJpc2sgZm9yIHRoZSBhZG1pbiB0byBub3QNCj4gPj4gY29ycmVjdGx5IHBhc3Mg
dGhlIGFtb3VudCBvZiBtZW1vcnkuIFRoaXMgbWF5IGJlY29tZSB1bm5vdGljZWQgdW50aWwNCj4g
bGF0ZS4NCj4gPj4NCj4gPj4gU28gSSB0aGluayBtYWtpbmcgaXQgbWFuZGF0b3J5IGlzIGNoZWFw
IGZvciB1cyBhbmQgYW4gZWFzeSB3YXkgdG8NCj4gPj4gY29uZmlybSB5b3UgcHJvcGVybHkgc2l6
ZWQgdGhlIHJlZ2lvbi4gSXQgYWxzbyBoYXMgdGhlIGJlbmVmaXRzIHRvDQo+ID4+IGVhc2lseSBm
aW5kIG91dCBob3cgbXVjaCBtZW1vcnkgeW91IGdhdmUgdG8gdGhlIGd1ZXN0IChidXQgdGhhdCdz
IGp1c3QNCj4gYmVjYXVzZSBJIGFtIGxhenkgOikpLg0KPiA+Pg0KPiA+Pj4gSW4gYW55IGNhc2Us
IGV2ZW4gaWYgd2UgZG9uJ3QgbWFrZSAibWVtb3J5IiBvcHRpb25hbCwgaXQgbWlnaHQgc3RpbGwN
Cj4gPj4+IGJlIGdvb2QgdG8gdHVybiB0aGlzIGVycm9yIGludG8gYSB3YXJuaW5nIGFuZCBpZ25v
cmUgdGhlIHJlbWFpbmluZw0KPiA+Pj4ga2luZm8tPnVuYXNzaWduZWRfbWVtLg0KPiA+Pg0KPiA+
PiBUaGUgYmVoYXZpb3IgaXMgbWF0Y2hpbmcgdGhlIGV4aXN0aW5nIGZ1bmN0aW9uIGFuZCBJIHRo
aW5rIHRoaXMgaXMgYQ0KPiA+PiBnb29kIGlkZWEuIElmIHlvdSBhc2sgMTBNQiBvZiBtZW1vcnkg
YW5kIHdlIG9ubHkgZ2l2ZSB5b3UgOU1CLCB0aGVuDQo+ID4+IGF0IHNvbWUgcG9pbnQgeW91ciBn
dWVzdCBpcyBub3QgZ29pbmcgdG8gYmUgaGFwcHkuDQo+ID4+DQo+ID4+IEl0IGlzIG11Y2ggYmV0
dGVyIHRvIGtub3cgaXQgaW4gYWR2YW5jZSB3aXRoIGEgZmFpbHVyZSBvdmVyDQo+ID4+IGRpc2Nv
dmVyaW5nIGhvdXJzIGxhdGVyIHdoZW4geW91IHNlZSBhbiBPT00gZnJvbSB5b3VyIGRvbWFpbi4N
Cj4gPg0KPiA+IE9LLCBJIGRpZG4ndCByZWFsaXplIHRoaXMgd2FzIGRpc2N1c3NlZCBhbHJlYWR5
LiBMZXQncyBub3QgcmV2aXNpdA0KPiA+IHRoaXMgdGhlbi4NCj4gPg0KPiA+IE15IHByZWZlcmVu
Y2UgaXMgcHJpbWFyaWx5IHRvIG1ha2UgdGhlIGRldmljZSB0cmVlIGVhc2llciB0byB3cml0ZSwN
Cj4gPiBidXQgbm93YWRheXMgbm9ib2R5IEkga25vdyBpcyB3cml0aW5nIHRoZSBkZXZpY2UgdHJl
ZSBieSBoYW5kIGFueW1vcmUNCj4gPiAodGhleSBhbGwgdXNlIEltYWdlQnVpbGRlcikuU28gaWYg
dGhlIGRldmljZSB0cmVlIGlzIGdlbmVyYXRlZCB0aGVuIHdlDQo+ID4gYXJlIGZpbmUgZWl0aGVy
IHdheSBhcyBsb25nIGFzIHRoZSBiaW5kaW5nIGlzIGNsZWFyLiBTbyBJIGFtIE9LIHRvDQo+ID4g
ZHJvcCBteSBzdWdnZXN0aW9uIG9mIG1ha2luZyAibWVtb3J5IiBvcHRpb25hbC4gTGV0J3MgdGhp
bmsgb2YgYSB3YXkNCj4gPiB0byBtYWtlICJtZW1vcnkiIGFuZCAieGVuLHN0YXRpYy1tZW0iIGNv
ZXhpc3QgaW5zdGVhZC4NCj4gPg0KPiA+DQo+ID4gVGhlcmUgYXJlIHR3byBzaWRlcyBvZiB0aGUg
aXNzdWU6DQo+ID4gLSB0aGUgWGVuIGltcGxlbWVudGF0aW9uDQo+ID4gLSB0aGUgYmluZGluZw0K
PiA+DQo+ID4NCj4gPiBUaGUgWGVuIGltcGxlbWVudGF0aW9uIGlzIGZpbmUgdG8gcGFuaWMgaWYg
bWVtb3J5ICE9IHhlbixzdGF0aWMtbWVtLg0KPiA+IEluIHRoYXQgcmVnYXJkLCB0aGUgY3VycmVu
dCBwYXRjaCBpcyBPSy4NCj4gPg0KPiA+DQo+ID4gIEZyb20gdGhlIGJpbmRpbmcgcGVyc3BlY3Rp
dmUsIEkgdGhpbmsgaXQgd291bGQgYmUgZ29vZCB0byBhZGQgYQ0KPiA+IHN0YXRlbWVudCB0byBj
bGFyaWZ5LiBUaGUgYmluZGluZyBkb2Vzbid0IG5lY2Vzc2FyaWx5IG5lZWQgdG8gbWF0Y2gNCj4g
PiBleGFjdGx5IHRoZSBpbXBsZW1lbnRhdGlvbiBhcyB0aGUgYmluZGluZyBzaG91bGQgYmUgYXMg
ZnV0dXJlIHByb29mDQo+ID4gYW5kIGFzIGZsZXhpYmxlIGFzIHBvc3NpYmxlLg0KPiANCj4gU28g
SSBhZ3JlZSB0aGF0IHRoZSBiaW5kaW5nIGRvZXNuJ3QgaGF2ZSB0byBtYXRjaCB0aGUgaW1wbGVt
ZW50YXRpb24uDQo+IEJ1dC4uLiB0aGUgYmluZGluZyBhbHdheXMgaGF2ZSBiZSBtb3JlIHJlc3Ry
aWN0aXZlIHRoYW4gdGhlIGltcGxlbWVudGF0aW9uLg0KPiBPdGhlcndpc2UsIHNvbWVvbmUgZm9s
bG93aW5nIHRoZSBiaW5kaW5nIG1heSBiZSBub3QgYWJsZSB0byB1c2UgaXQgd2l0aCBYZW4uDQo+
IA0KPiA+DQo+ID4gIEZyb20gdGhlIGJpbmRpbmcgcGVyc3BlY3RpdmUgdHdvIHByb3BlcnRpZXMg
c2hvdWxkIG1lYW4gZGlmZmVyZW50DQo+ID4gdGhpbmdzOiBtZW1vcnkgdGhlIHRvdGFsIG1lbW9y
eSBhbW91bnQgYW5kIHhlbixzdGF0aWMtbWVtIHRoZSBzdGF0aWMNCj4gPiBtZW1vcnkuIElmIG9u
ZSBkYXkgd2UnbGwgaGF2ZSBtb3JlIHR5cGVzIG9mIG1lbW9yeSwgbWVtb3J5IHdpbGwgY292ZXIN
Cj4gPiB0aGUgdG90YWwgYW1vdW50IHdoaWxlIHhlbixzdGF0aWMtbWVtIHdpbGwgY292ZXIgYSBz
dWJzZXQuIFNvIG1lbW9yeQ0KPiA+IG11c3QgYmUgZ3JlYXRlciBvciBlcXVhbCB0byB4ZW4sc3Rh
dGljLW1lbSAoZXZlbiBpZiB0b2RheSBYZW4gb25seQ0KPiA+IHN1cHBvcnRzIG1lbW9yeSA9PSB4
ZW4sc3RhdGljLW1lbSkuDQo+ID4NCj4gPiBTbyBJIHdvdWxkIGFkZDoNCj4gPg0KPiA+ICIiIg0K
PiA+IEFzIHRoZSBtZW1vcnkgcHJvcGVydHkgcmVwcmVzZW50cyB0aGUgdG90YWwgbWVtb3J5IG9m
IHRoZSBkb21haW4sDQo+ID4gaGVuY2UgdGhlIGFtb3VudCBvZiBtZW1vcnkgc2VsZWN0ZWQgYnkg
dGhlIG1lbW9yeSBwcm9wZXJ0eSBtdXN0IGJlDQo+ID4gZ3JlYXRlciBvciBlcXVhbCB0byB0aGUg
dG90YWwgYW1vdW50IHNwZWNpZmllZCBieSB4ZW4sc3RhdGljLW1lbS4NCj4gPiBPdGhlciBjb25m
aWd1cmF0aW9ucyAobWVtb3J5IGFtb3VudCBsZXNzIHRoYW4geGVuLHN0YXRpYy1tZW0gYW1vdW50
KQ0KPiA+IGFyZSBpbnZhbGlkLg0KPiA+ICIiIg0KPiA+DQo+ID4gVGhpcyBzZW50ZW5jZSBoYXMg
dGhlIHB1cnBvc2Ugb2YgY2xhcmlmeWluZyB0aGF0ICJtZW1vcnkiIHN0aWxsIG5lZWQNCj4gPiB0
byBiZSBwb3B1bGF0ZWQgYW5kIGhhdmUgYSB2YWxpZCB2YWx1ZS4gVGhlbiwgaXQgaXMgT0sgZm9y
IFhlbiB0bw0KPiA+IGVycm9yIG91dCBpZiBtZW1vcnkgZG9lc24ndCBtYXRjaCB4ZW4sc3RhdGlj
LW1lbSBiZWNhdXNlIHRoYXQncyB0aGUNCj4gPiBvbmx5IGNvbmZpZ3VyYXRpb24gc3VwcG9ydGVk
Lg0KPiBIb3cgYWJvdXQgd3JpdGluZyBzb21ldGhpbmcgbGlrZSAiVGhlIHByb3BlcnR5ICdtZW1v
cnknIHNob3VsZCBtYXRjaCB0aGUNCj4gYW1vdW50IG9mIG1lbW9yeSBnaXZlbiB0byB0aGUgZ3Vl
c3QuIEN1cnJlbnRseSwgaXQgaXMgb25seSBwb3NzaWJsZSB0byBlaXRoZXINCj4gYWxsb2NhdGUg
c3RhdGljIG1lbW9yeSBvciBsZXQgWGVuIGNob3NlLiAqTWl4aW5nKiBpcyBub3Qgc3VwcG9ydGVk
Jy4NCj4gDQo+IFRoZW4gaWYgd2UgYWRkIHRoZSBtaXhpbmcsIHdlIGNvdWxkIHNheSAnRnJvbSBY
ZW4gWFguWVksIG1peGluZyB3aWxsIGJlDQo+IGFsbG93ZWQnLg0KPiANCg0KT2suIEknbGwgYWRk
IHRoZSBzdGF0ZW1lbnQgdG8gY2xhcmlmeSB0aGUgYmluZGluZy4gDQoNCj4gQ2hlZXJzLA0KPiAN
Cj4gLS0NCj4gSnVsaWVuIEdyYWxsDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 03:19:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 03:19:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180341.326947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRdU-0002us-UP; Tue, 07 Sep 2021 03:19:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180341.326947; Tue, 07 Sep 2021 03:19:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNRdU-0002ul-Q6; Tue, 07 Sep 2021 03:19:00 +0000
Received: by outflank-mailman (input) for mailman id 180341;
 Tue, 07 Sep 2021 03:18:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GNGS=N5=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNRdT-0002uf-1v
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 03:18:59 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.52]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0fd35b04-a34b-4e54-a5a4-7a4df8557624;
 Tue, 07 Sep 2021 03:18:56 +0000 (UTC)
Received: from AM6P192CA0105.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::46)
 by VI1PR0802MB2445.eurprd08.prod.outlook.com (2603:10a6:800:bb::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 03:18:49 +0000
Received: from AM5EUR03FT047.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8d:cafe::bf) by AM6P192CA0105.outlook.office365.com
 (2603:10a6:209:8d::46) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 03:18:49 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT047.mail.protection.outlook.com (10.152.16.197) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 03:18:49 +0000
Received: ("Tessian outbound cc1d52552731:v105");
 Tue, 07 Sep 2021 03:18:48 +0000
Received: from 3016fb1a370b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 AEB6A185-06C1-4A94-A51E-75245AE3AD0B.1; 
 Tue, 07 Sep 2021 03:18:38 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3016fb1a370b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 03:18:38 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB4173.eurprd08.prod.outlook.com (2603:10a6:803:e8::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 03:18:29 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 03:18:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fd35b04-a34b-4e54-a5a4-7a4df8557624
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eAutKulNSqbSaQcjB/rY66Exbgv3sh5ur6/FakAOZtA=;
 b=b7wZ/vhM196EVRBtO4mY2nJLhxrmiYdDZalLfeItQXXrd/ZQVnCd18Iyz3BhkU+FwG5VRHtF7t5i3u8E1RrajnxoR/0QmnnT8JWk0vlyf2VfxCFL/eFkhWDl/FnpsUsvECT0OijLAorXfvhdSm7WTUIHvuPt/diptz4WrMnEMQk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=flJU45vtVZWsrH3V/GSEaDRg8d2nZ8E0Fz4+b45WeddQQccdg7RBGaD3v0JLizRhN4d7E8TeHqFoehEdZ2vaySpqNX8WX6kyFsR4hW7k9i1IZPy3gbdByeD7BSuHpkG+oq3Cr3YKeN51S5mM6b8kas8rmFq+kauESOHH87kHCV07VqRqiGCKIYQo5TFHmzuWW7DryQEc0x1a+TnR3/SgU5JHefiCyQ3uDTdgtcgD/+VCt/oVcCkyVYkZdX2KksPYR5yvezxitJWI8skh9mwhoTvFSvM2qSKtH58SAiXam9Ip0lclqKiLx83m0p8SBBEB/bIp91yaxXlA99YCVCDEtw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eAutKulNSqbSaQcjB/rY66Exbgv3sh5ur6/FakAOZtA=;
 b=C4YRXHg8qCsbFdjkyOmaU7IcLDX7lEeAV7zgYEooU7xnpRhbOqnNNqWQb2FquARAHBzyjt5/5otDjOCFBl2wufEiT86K4pOGp3sCT6QupPFMiGGFuMlzckxIAwLIDo5YbUiyx8tsXRnwqzOKIvEQJ2KhVxo01eWMtgRjhatgvx74NWDYR3ZMzGbqxkAU4iYtUebvtBujcp3XyBz6GOUFavbRimHNoBO2N6WXMZiLrssqC+koqN6+xMOD0d6A8FwzhuzNh0FzZLjHp+cjeMo0Iv/3tOAlQE44QDw1eQC139+uCsrHc5qBquskqiIEM7hOgDyxVxwaK5OWUcMhZFqadw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eAutKulNSqbSaQcjB/rY66Exbgv3sh5ur6/FakAOZtA=;
 b=b7wZ/vhM196EVRBtO4mY2nJLhxrmiYdDZalLfeItQXXrd/ZQVnCd18Iyz3BhkU+FwG5VRHtF7t5i3u8E1RrajnxoR/0QmnnT8JWk0vlyf2VfxCFL/eFkhWDl/FnpsUsvECT0OijLAorXfvhdSm7WTUIHvuPt/diptz4WrMnEMQk=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, nd <nd@arm.com>
Subject: RE: [PATCH v5 2/7] xen/arm: introduce domain on Static Allocation
Thread-Topic: [PATCH v5 2/7] xen/arm: introduce domain on Static Allocation
Thread-Index: AQHXmM2YqjVc2LVowEyczFIN0YMASquRUfwAgAapJVA=
Date: Tue, 7 Sep 2021 03:18:29 +0000
Message-ID:
 <VE1PR08MB52153131D62829CBB8152B93F7D39@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210824095045.2281500-1-penny.zheng@arm.com>
 <20210824095045.2281500-3-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109021312051.26072@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109021312051.26072@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: EEA958F302EBDA4CB8D1B1C352665A34.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3798125a-0c09-4557-c977-08d971ae360e
x-ms-traffictypediagnostic: VI1PR08MB4173:|VI1PR0802MB2445:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0802MB244519C516DA266F52C08A29F7D39@VI1PR0802MB2445.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:1148;OLM:1148;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 joSyrx2GucuBrj2gtO+eOEiQMgSlcJ9mBxLGV3ZsYNNlDPBvz8AG1CNKBxvPfBR/7QJyJaAPdqJvwR/gsVnfYLqTiC0ZgZV3Bx6ibABQQQFFhLndgWXEBFBRzXepUGbsxOn8i8fGZF0tS+wORcgc17fun6CirM0+7dcGJhYOVgoHkhsGRfMM69fYZvJfTl+tq0cCSWlDaL3YCM79wXPAaqqkVOkYry00oUH/yjUXN7e7LdT0FpocHWkdq5ZuOQmSutiZs97NGK1SdTqkkHnUhAqCTyi4JzFdX0tZMAaUyWeNGbGuC7mVzRWk0bic73oCY57CbxA9OaHyzIIsS3eRw2zI+2zHiTL7D11AA9b2/i1WFlPnEM6I1/ih04Zbv3fOoGMOwo9w3K5YhHCqjY2sjO87rPAKG6jHWvLMnf1DNwSPWB8uL6kP0EKPytoBTILieT0duntDsKIYtClrdw0a+o5R+rRwBHAODNYWJ+xww//1uVR8LUrTMkZu6YADFTZxLIOW3f4kXdnJbFfe2O9HXF0hQedzjmFljdkxSKMXon0buJVoHNnMy8nkg6RrX1opJhWDEyFIEZwydTRsEyKQxButbMvrRBcnyisyJQjh1QlwpElT3Fs/RfLZfokUlxG555Pz64hxdosdyGB+74gFSJOQ1WU/5/7vl1uF1gtIZ/y5SOB3KRxfGhrEPYOaowf+ApeDsq3NW4fqgcy6lruedw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(366004)(396003)(136003)(39860400002)(186003)(54906003)(2906002)(53546011)(55016002)(26005)(6506007)(7696005)(86362001)(38070700005)(52536014)(6916009)(8936002)(9686003)(83380400001)(8676002)(33656002)(316002)(64756008)(66476007)(122000001)(66446008)(66556008)(71200400001)(66946007)(76116006)(4326008)(5660300002)(38100700002)(478600001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4173
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	4ce8522d-29ea-41e6-9b9a-08d971ae2a3b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Vzfy021VVqWWyaWOHxJV67yFO3GIUiLdTICAqZYoy8KaMjFkdlPm+xk18vuilTpOXwNR3F2Lmgc2vZS0rOvXskxYF+lNgNtDokJNBg+VJUAFryBejTtfOX0KQ7tRtnyCsZhWk7zxj9XdwI6MIgNDpZvaPkHV7/uYjfowiy4v081dpTDT1B5Nep/6Nh3PqvYrKgnBXQvdoMNjRTsA7zwd4Xu/rAJlhhvbX7LBbYX48Gn0cwZMFc+uD8r9UujJVuaSfhrxPviWNNN76qjot1Movk7zz8KxmPECpCmiyIWb5Az+VSE9xP92BhcNXHjbzogFpPrkuWLzBe5NsWkI3WmGUBfw/XlYNf1hGXVMu/KOn7T+4fXD2RFAU0ME+q3K0xYxe2klZ+6zonTclxs6zyjvtZWBYrm9tksP1l+JjlQdV2HOwOz13bJUkSsYPyjVKjhstXi1/KEQiEmL+ozGz/hnWPScggJofW5Fl7XuBvipjP7mUo28xKJDLQlTujAQubP6n94670dfyJ3IlgRslzYbF8e02Y4INiCHrdQ9F5EzHKHT6HdORDiDrUYcRrfy87xazw+i0shnx8zWtqmbgsqDbqefiVDxxt4iolpJ7uGVNHOsNibC8i2NwdUNu4wldQ2bLBFvrp8adqccMnRq4P0iqRoBUNqJsq738BsFR2/vtoas0RoIG5TONGl5DO3L4Nv54+BlgHC8awyvxqH1C9Z1GA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(396003)(136003)(46966006)(36840700001)(316002)(26005)(36860700001)(8676002)(5660300002)(6862004)(8936002)(186003)(86362001)(4326008)(83380400001)(55016002)(54906003)(52536014)(70206006)(33656002)(9686003)(47076005)(7696005)(70586007)(478600001)(53546011)(6506007)(82310400003)(82740400003)(356005)(336012)(2906002)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 03:18:49.4073
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3798125a-0c09-4557-c977-08d971ae360e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2445

Hi Stefano

> -----Original Message-----
> From: Stefano Stabellini <sstabellini@kernel.org>
> Sent: Friday, September 3, 2021 5:31 AM
> To: Penny Zheng <Penny.Zheng@arm.com>
> Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.or=
g;
> Bertrand Marquis <Bertrand.Marquis@arm.com>; Wei Chen
> <Wei.Chen@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v5 2/7] xen/arm: introduce domain on Static Allocatio=
n
>=20
> On Tue, 24 Aug 2021, Penny Zheng wrote:
> > Static Allocation refers to system or sub-system(domains) for which
> > memory areas are pre-defined by configuration using physical address
> ranges.
> > Those pre-defined memory, -- Static Memory, as parts of RAM reserved
> > in the beginning, shall never go to heap allocator or boot allocator fo=
r any
> use.
> >
> > Memory can be statically allocated to a domain using the property
> > "xen,static- mem" defined in the domain configuration. The number of
> > cells for the address and the size must be defined using respectively
> > the properties "#xen,static-mem-address-cells" and "#xen,static-mem-siz=
e-
> cells".
> >
> > This patch introduces this new `xen,static-mem` feature, and also
> > documents and parses this new attribute at boot time.
> >
> > This patch also introduces a new field "bool xen_domain" in "struct
> membank"
> > to tell the difference of one memory bank is reserved as the whole
> > hardware resource, or bind to one specific xen domain node, through
> > "xen,static-mem"
> >
> > Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> > ---
> > v5 changes:
> > - check the node using the Xen domain binding whether contains the
> > property "xen,static-mem", not the property itself
> > - add "rc =3D ..." to get the error propagated
> > - introduce new field "bool xen_domain", then static memory shall be
> > also stored as reserved memory(bootinfo.reserved_mem), but being bind
> > to one specific Xen domain node.
> > - doc refinement
> > ---
> >  docs/misc/arm/device-tree/booting.txt | 33 ++++++++++++++++++++++++
> >  xen/arch/arm/bootfdt.c                | 36 +++++++++++++++++++++++++--
> >  xen/include/asm-arm/setup.h           |  1 +
> >  3 files changed, 68 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/misc/arm/device-tree/booting.txt
> > b/docs/misc/arm/device-tree/booting.txt
> > index 5243bc7fd3..95b20ddc3a 100644
> > --- a/docs/misc/arm/device-tree/booting.txt
> > +++ b/docs/misc/arm/device-tree/booting.txt
> > @@ -268,3 +268,36 @@ The DTB fragment is loaded at 0xc000000 in the
> > example above. It should  follow the convention explained in
> > docs/misc/arm/passthrough.txt. The  DTB fragment will be added to the
> > guest device tree, so that the guest  kernel will be able to discover t=
he device.
> > +
> > +
> > +Static Allocation
> > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > +
> > +Static Allocation refers to system or sub-system(domains) for which
> > +memory areas are pre-defined by configuration using physical address
> ranges.
> > +
> > +Memory can be statically allocated to a domain using the property
> > +"xen,static- mem" defined in the domain configuration. The number of
> > +cells for the address and the size must be defined using respectively
> > +the properties "#xen,static-mem-address-cells" and "#xen,static-mem-si=
ze-
> cells".
> > +
> > +Below is an example on how to specify the static memory region in the
> > +device-tree:
> > +
> > +    / {
> > +        chosen {
> > +            domU1 {
> > +                compatible =3D "xen,domain";
> > +                #address-cells =3D <0x2>;
> > +                #size-cells =3D <0x2>;
> > +                cpus =3D <2>;
> > +                #xen,static-mem-address-cells =3D <0x1>;
> > +                #xen,static-mem-size-cells =3D <0x1>;
> > +                xen,static-mem =3D <0x30000000 0x20000000>;
> > +                ...
> > +            };
> > +        };
> > +    };
> > +
> > +This will reserve a 512MB region starting at the host physical
> > +address
> > +0x30000000 to be exclusively used by DomU1
>=20
> This binding is OK.  We might want to clarify what is the purpose of the
> "memory" property when "xen,static-mem" is present. I would suggest to wr=
ite
> that when "xen,static-mem" is present, the "memory" property becomes
> optional. Or even better:
>=20
> """
> When present, the xen,static-mem property supersedes the memory property.
> """
>=20

oh, "supersede" learned! ;)

>=20
> In the future when Xen will support direct mapping, I assume that we'll a=
lso
> add a direct-map property to enable it.  Something like:
>=20
>     domU1 {
>         compatible =3D "xen,domain";
>         #address-cells =3D <0x2>;
>         #size-cells =3D <0x2>;
>         cpus =3D <2>;
>         #xen,static-mem-address-cells =3D <0x1>;
>         #xen,static-mem-size-cells =3D <0x1>;
>         xen,static-mem =3D <0x30000000 0x20000000>;
>         direct-map;
>         ...
>     };
>=20
> Maybe I would add a statement to clarify it that xen,static-mem doesn't
> automatically imply direct mapping. Something like:
>=20
> """
> The static memory will be mapped in the guest at the usual guest memory
> addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> xen/include/public/arch-arm.h.
> """
>=20

Thanks for the detailed suggestion. I'll just take it.

> The rest of the patch looks OK. One minor NIT below.
>=20
>=20
>=20
> > diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c index
> > 8c81be3379..00f34eec58 100644
> > --- a/xen/arch/arm/bootfdt.c
> > +++ b/xen/arch/arm/bootfdt.c
> > @@ -66,7 +66,7 @@ void __init device_tree_get_reg(const __be32 **cell,
> > u32 address_cells,  static int __init device_tree_get_meminfo(const voi=
d *fdt,
> int node,
> >                                            const char *prop_name,
> >                                            u32 address_cells, u32 size_=
cells,
> > -                                          void *data)
> > +                                          void *data, bool
> > + xen_domain)
> >  {
> >      const struct fdt_property *prop;
> >      unsigned int i, banks;
> > @@ -90,6 +90,7 @@ static int __init device_tree_get_meminfo(const void
> *fdt, int node,
> >              continue;
> >          mem->bank[mem->nr_banks].start =3D start;
> >          mem->bank[mem->nr_banks].size =3D size;
> > +        mem->bank[mem->nr_banks].xen_domain =3D xen_domain;
> >          mem->nr_banks++;
> >      }
> >
> > @@ -184,7 +185,8 @@ static int __init process_memory_node(const void
> *fdt, int node,
> >          return -EINVAL;
> >      }
> >
> > -    return device_tree_get_meminfo(fdt, node, "reg", address_cells,
> size_cells, data);
> > +    return device_tree_get_meminfo(fdt, node, "reg", address_cells,
> size_cells,
> > +                                   data, false);
> >  }
> >
> >  static int __init process_reserved_memory_node(const void *fdt, int
> > node, @@ -338,6 +340,34 @@ static void __init process_chosen_node(const
> void *fdt, int node,
> >      add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);  }
> >
> > +static int __init process_domain_node(const void *fdt, int node,
> > +                                       const char *name,
> > +                                       u32 address_cells, u32
> > +size_cells) {
> > +    const struct fdt_property *prop;
> > +
> > +    printk("Checking for \"xen,static-mem\" in domain node\n");
> > +
> > +    prop =3D fdt_get_property(fdt, node, "xen,static-mem", NULL);
> > +    if ( !prop )
> > +        /* No "xen,static-mem" present. */
> > +        return 0;
> > +
> > +    address_cells =3D device_tree_get_u32(fdt, node,
> > +                                        "#xen,static-mem-address-cells=
", 0);
> > +    size_cells =3D device_tree_get_u32(fdt, node,
> > +                                     "#xen,static-mem-size-cells", 0);
> > +    if ( address_cells < 1 || size_cells < 1 )
> > +    {
> > +        printk("fdt: node `%s': invalid #xen,static-mem-address-cells =
or
> #xen,static-mem-size-cells",
> > +               name);
> > +        return -EINVAL;
> > +    }
> > +
> > +    return device_tree_get_meminfo(fdt, node, "xen,static-mem",
> address_cells,
> > +                                   size_cells,
> > +&bootinfo.reserved_mem, true); }
> > +
> >  static int __init early_scan_node(const void *fdt,
> >                                    int node, const char *name, int dept=
h,
> >                                    u32 address_cells, u32 size_cells,
> > @@ -356,6 +386,8 @@ static int __init early_scan_node(const void *fdt,
> >          process_multiboot_node(fdt, node, name, address_cells, size_ce=
lls);
> >      else if ( depth =3D=3D 1 && device_tree_node_matches(fdt, node, "c=
hosen") )
> >          process_chosen_node(fdt, node, name, address_cells,
> > size_cells);
> > +    else if ( depth =3D=3D 2 && device_tree_node_compatible(fdt, node,
> "xen,domain") )
> > +        rc =3D process_domain_node(fdt, node, name, address_cells,
> > + size_cells);
> >
> >      if ( rc < 0 )
> >          printk("fdt: node `%s': parsing failed\n", name); diff --git
> > a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h index
> > c4b6af6029..6c3c16294b 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -24,6 +24,7 @@ typedef enum {
> >  struct membank {
> >      paddr_t start;
> >      paddr_t size;
> > +    bool xen_domain; /* whether memory bank is bind to Xen domain. */
>                                   ^ a or the      ^ bound to a
>=20

Sure. Will fix it.

>=20
> >  };
> >
> >  struct meminfo {
> > --
> > 2.25.1
> >


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 03:54:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 03:54:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180349.326957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNSBZ-0007ve-JU; Tue, 07 Sep 2021 03:54:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180349.326957; Tue, 07 Sep 2021 03:54:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNSBZ-0007vX-GV; Tue, 07 Sep 2021 03:54:13 +0000
Received: by outflank-mailman (input) for mailman id 180349;
 Tue, 07 Sep 2021 03:54:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNSBY-0007vN-AZ; Tue, 07 Sep 2021 03:54:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNSBY-00021d-0j; Tue, 07 Sep 2021 03:54:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNSBX-0004Yg-Jc; Tue, 07 Sep 2021 03:54:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNSBX-00032x-Ik; Tue, 07 Sep 2021 03:54:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TJwIEulUiBltS3/OiOSnKXFQSw0k5U3f7zwUaqBCPKQ=; b=ZX9rmJo9WSX/IXtVPVa/sdqKRl
	8hCpu4B3Fg2IeYSebpZO1l4Nr+ARPSVeUZ1w1/UDAFXekBAbHmMq0HGhFg4PMXaZusl5Kuwpv8MR1
	8yRAB21cuBSGFb1unwtaPWlBnuzuP01JWy1/O/fLBLMvSXu8nWcYwjxxcJih61q/tIkQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164864-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164864: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=935efca6c246c108253b0e4e51cc87648fc7ca10
X-Osstest-Versions-That:
    qemuu=88afdc92b644120e0182c8567e1b1d236e471b12
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 03:54:11 +0000

flight 164864 qemu-mainline real [real]
flight 164868 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164864/
http://logs.test-lab.xenproject.org/osstest/logs/164868/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164868-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     14 guest-start              fail REGR. vs. 164855

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164855
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164855
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164855
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164855
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164855
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164855
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164855
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                935efca6c246c108253b0e4e51cc87648fc7ca10
baseline version:
 qemuu                88afdc92b644120e0182c8567e1b1d236e471b12

Last test of basis   164855  2021-09-06 10:06:44 Z    0 days
Testing same since   164864  2021-09-06 18:07:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Peter Krempa <pkrempa@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Willian Rampazzo <willianr@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   88afdc92b6..935efca6c2  935efca6c246c108253b0e4e51cc87648fc7ca10 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 04:18:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 04:18:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180358.326972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNSZ2-0002Qb-Fi; Tue, 07 Sep 2021 04:18:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180358.326972; Tue, 07 Sep 2021 04:18:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNSZ2-0002QU-Ce; Tue, 07 Sep 2021 04:18:28 +0000
Received: by outflank-mailman (input) for mailman id 180358;
 Tue, 07 Sep 2021 04:18:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xlGX=N5=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNSZ1-0002QO-6k
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 04:18:27 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a545b804-0f92-11ec-b0cc-12813bfff9fa;
 Tue, 07 Sep 2021 04:18:26 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 278BE1FF56;
 Tue,  7 Sep 2021 04:18:25 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id B8D0612FF9;
 Tue,  7 Sep 2021 04:18:24 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 5CvvKhDoNmE4awAAGKfGzw
 (envelope-from <jgross@suse.com>); Tue, 07 Sep 2021 04:18:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a545b804-0f92-11ec-b0cc-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630988305; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=h49ksqWpNBrOpV9ozMeMCOB3BPOEfvBt8DuU3qwVRqc=;
	b=sZNNeEPwCdCRgkir273n5eFwHDgdw09J9nVsrNWcwRIcB8eB9BEzwnPVCPOPZCdJRaUvtb
	NxyFo5jPrQPrxrxIirIpNeQ1LSKCvGhkbtLkmmgRNtGv89i+lkQsylxBGUAM4rdp7dydgg
	ePYUWKRc0W2M5DRpLXYO2RjSjFhzsS0=
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
To: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
 xen-devel@lists.xenproject.org, Linux PM <linux-pm@vger.kernel.org>,
 Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
 "Rafael J. Wysocki" <rjw@rjwysocki.net>, Len Brown <len.brown@intel.com>,
 Pavel Machek <pavel@ucw.cz>, Stable <stable@vger.kernel.org>
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com> <YTHjPbklWVDVaBfK@kroah.com>
 <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
 <CAJZ5v0g_WVFqDKCBYnoPtqR5VzH-eBMk+7M1bAmgGsyX0XGpgw@mail.gmail.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <3f08f433-a228-d512-3608-3ed0c797e653@suse.com>
Date: Tue, 7 Sep 2021 06:18:23 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <CAJZ5v0g_WVFqDKCBYnoPtqR5VzH-eBMk+7M1bAmgGsyX0XGpgw@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="2WDWaUILT1zcTAfwQw13cP6arEOvVJ7NT"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--2WDWaUILT1zcTAfwQw13cP6arEOvVJ7NT
Content-Type: multipart/mixed; boundary="40AjFM7hF7VBwEyptUN1cRO8vL4caXhmG";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
 xen-devel@lists.xenproject.org, Linux PM <linux-pm@vger.kernel.org>,
 Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
 "Rafael J. Wysocki" <rjw@rjwysocki.net>, Len Brown <len.brown@intel.com>,
 Pavel Machek <pavel@ucw.cz>, Stable <stable@vger.kernel.org>
Message-ID: <3f08f433-a228-d512-3608-3ed0c797e653@suse.com>
Subject: Re: [PATCH 1/2] PM: base: power: don't try to use non-existing RTC
 for storing data
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-2-jgross@suse.com> <YTHjPbklWVDVaBfK@kroah.com>
 <1b6a8f9c-2a5f-e97e-c89d-5983ceeb20e5@suse.com>
 <CAJZ5v0g_WVFqDKCBYnoPtqR5VzH-eBMk+7M1bAmgGsyX0XGpgw@mail.gmail.com>
In-Reply-To: <CAJZ5v0g_WVFqDKCBYnoPtqR5VzH-eBMk+7M1bAmgGsyX0XGpgw@mail.gmail.com>

--40AjFM7hF7VBwEyptUN1cRO8vL4caXhmG
Content-Type: multipart/mixed;
 boundary="------------9B3DCA2E79466DA893357F5F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------9B3DCA2E79466DA893357F5F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 19:07, Rafael J. Wysocki wrote:
> On Fri, Sep 3, 2021 at 11:02 AM Juergen Gross <jgross@suse.com> wrote:
>>
>> On 03.09.21 10:56, Greg Kroah-Hartman wrote:
>>> On Fri, Sep 03, 2021 at 10:49:36AM +0200, Juergen Gross wrote:
>>>> In there is no legacy RTC device, don't try to use it for storing tr=
ace
>>>> data across suspend/resume.
>>>>
>>>> Cc: <stable@vger.kernel.org>
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>> ---
>>>>    drivers/base/power/trace.c | 10 ++++++++++
>>>>    1 file changed, 10 insertions(+)
>>>>
>>>> diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c=

>>>> index a97f33d0c59f..b7c80849455c 100644
>>>> --- a/drivers/base/power/trace.c
>>>> +++ b/drivers/base/power/trace.c
>>>> @@ -13,6 +13,7 @@
>>>>    #include <linux/export.h>
>>>>    #include <linux/rtc.h>
>>>>    #include <linux/suspend.h>
>>>> +#include <linux/init.h>
>>>>
>>>>    #include <linux/mc146818rtc.h>
>>>>
>>>> @@ -165,6 +166,9 @@ void generate_pm_trace(const void *tracedata, un=
signed int user)
>>>>       const char *file =3D *(const char **)(tracedata + 2);
>>>>       unsigned int user_hash_value, file_hash_value;
>>>>
>>>> +    if (!x86_platform.legacy.rtc)
>>>> +            return 0;
>>>
>>> Why does the driver core code here care about a platform/arch-specifi=
c
>>> thing at all?  Did you just break all other arches?
>>
>> This file is only compiled for x86. It depends on CONFIG_PM_TRACE_RTC,=

>> which has a "depends on X86" attribute.
>=20
> This feature uses the CMOS RTC memory to store data, so if that memory
> is not present, it's better to avoid using it.
>=20
> Please feel free to add
>=20
> Reviewed-by: Rafael J. Wysocki <rafael@kernel.org>

Thanks!

>=20
> to this patch or let me know if you want me to take it.
>=20

No, I can take it with the other patch of this small series, thanks.


Juergen

--------------9B3DCA2E79466DA893357F5F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------9B3DCA2E79466DA893357F5F--

--40AjFM7hF7VBwEyptUN1cRO8vL4caXhmG--

--2WDWaUILT1zcTAfwQw13cP6arEOvVJ7NT
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE26BAFAwAAAAAACgkQsN6d1ii/Ey9r
Hwf/c5XiAVGxbZw83ONwePCE+bf8D3q8KJTcmCkhNwelw0RK36RQnVYNwSa9bzaSR89Lzi1swmXh
Lr82MokgGFWKUx9MPteFyc4i5PyxI2YNuktNKru/bQNoiVjNzBtppdt1N2GozS0vE2SeeAr+VNIf
OYYpAWaHOSV0UtyOSbD5pQYD8+dAfs87HJ+1iX3p9RLGNZI+zO0MHdzqVLdTxEaVhOitakKC1sPR
WXjVRTPTduUUqcnQUt93kt/1rz4zUcHoO5KdI859qIY4EmLKL4psTa2SEyjU0oB0JrUQxpoceul6
05bb8ymydiOv8M2YIpC1tRIiQMXnSWLYtp7FTgO7bg==
=1oDE
-----END PGP SIGNATURE-----

--2WDWaUILT1zcTAfwQw13cP6arEOvVJ7NT--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 05:09:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 05:09:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180366.326983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTM0-0000Uy-9c; Tue, 07 Sep 2021 05:09:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180366.326983; Tue, 07 Sep 2021 05:09:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTM0-0000Ur-6S; Tue, 07 Sep 2021 05:09:04 +0000
Received: by outflank-mailman (input) for mailman id 180366;
 Tue, 07 Sep 2021 05:09:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xlGX=N5=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNTLy-0000Ul-OE
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 05:09:02 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b665aacb-0f99-11ec-b0d1-12813bfff9fa;
 Tue, 07 Sep 2021 05:09:01 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 717AB1FD69;
 Tue,  7 Sep 2021 05:09:00 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 3AC84132AB;
 Tue,  7 Sep 2021 05:09:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 9xPCDOzzNmHRcgAAGKfGzw
 (envelope-from <jgross@suse.com>); Tue, 07 Sep 2021 05:09:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b665aacb-0f99-11ec-b0d1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1630991340; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=ULAENWFlyyXXMa0pYWOTC/kHazB6xVrcl7laUjdiR1o=;
	b=b+DEtPYHQiAp6Pcat+VpwIVL6YlRfsjaqf2YhV+fat/Att9rKkNa70v05Vn2sOIP+nOuDj
	9m9g43Vafq9RYst/aUlvpHQwVo8M16n0AnrEUbz8yv7RQ94Z7v5PZ2GpBI2fwCCxe2EdVq
	XF0d4lBqC3xbXff1QkeRnZc7jO5ytd0=
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, xen-devel@lists.xenproject.org
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
 <2a42660c-e1ae-c791-784a-da39e64473f3@citrix.com>
 <598640b9-758a-5a0c-83cd-38b3e41d7b39@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <d42f3023-7701-f695-ddf9-5a7330eb7a65@suse.com>
Date: Tue, 7 Sep 2021 07:08:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <598640b9-758a-5a0c-83cd-38b3e41d7b39@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="eYCTYiqErDDNw369RivENVePujGNhH1yJ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--eYCTYiqErDDNw369RivENVePujGNhH1yJ
Content-Type: multipart/mixed; boundary="MOnJc8pN4CJCBCSVPPUd7uwGYTUt8x5wY";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, xen-devel@lists.xenproject.org
Message-ID: <d42f3023-7701-f695-ddf9-5a7330eb7a65@suse.com>
Subject: Re: [PATCH] xen/sched: fix sched_move_domain() for domain without
 vcpus
References: <20210906110057.15384-1-jgross@suse.com>
 <0006eb7d-a9d5-16a3-1418-a1d28c4c9a92@citrix.com>
 <2a42660c-e1ae-c791-784a-da39e64473f3@citrix.com>
 <598640b9-758a-5a0c-83cd-38b3e41d7b39@suse.com>
In-Reply-To: <598640b9-758a-5a0c-83cd-38b3e41d7b39@suse.com>

--MOnJc8pN4CJCBCSVPPUd7uwGYTUt8x5wY
Content-Type: multipart/mixed;
 boundary="------------AB3182B2E1CC8A09D81AD162"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------AB3182B2E1CC8A09D81AD162
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 13:23, Jan Beulich wrote:
> On 06.09.2021 13:18, Andrew Cooper wrote:
>> On 06/09/2021 12:14, Andrew Cooper wrote:
>>> On 06/09/2021 12:00, Juergen Gross wrote:
>>>> In case a domain is created with a cpupool other than Pool-0 specifi=
ed
>>>> it will be moved to that cpupool before any vcpus are allocated.
>>>>
>>>> This will lead to a NULL pointer dereference in sched_move_domain().=

>>>>
>>>> Fix that by tolerating vcpus not being allocated yet.
>>>>
>>>> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between =
cpupools with different granularity")
>>>> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>> ---
>>>>   xen/common/sched/core.c | 4 ++++
>>>>   1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
>>>> index 8d178baf3d..79c9100680 100644
>>>> --- a/xen/common/sched/core.c
>>>> +++ b/xen/common/sched/core.c
>>>> @@ -671,6 +671,10 @@ int sched_move_domain(struct domain *d, struct =
cpupool *c)
>>>>  =20
>>>>       for ( unit_idx =3D 0; unit_idx < n_units; unit_idx++ )
>>>>       {
>>>> +        /* Special case for move at domain creation time. */
>>>> +        if ( !d->vcpu[unit_idx * gran] )
>>>> +            break;
>>>> +
>>>>           unit =3D sched_alloc_unit_mem();
>>>>           if ( unit )
>>>>           {
>>> I think the logic would be clearer if you wrap the entire for loop in=
 if
>>> ( d->max_vcpus ).
>>
>> And of course, this is wrong.=C2=A0 Turns out the domain_has_vcpus()
>> predicate still hasn't been committed, but d->vcpu[0] is the correct
>> internal.
>=20
> Which in turn might want to be done by setting n_units to zero when
> d->vcpus[0] is NULL?

Yes, this would be possible.

OTOH my variant is more robust in case not all vcpus are allocated,
but I guess this will explode somewhere else anyway.

In case I don't get any other comment today I'll change the patch to set
n_units to 0 if d->vcpus[0] is NULL.


Juergen


--------------AB3182B2E1CC8A09D81AD162
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------AB3182B2E1CC8A09D81AD162--

--MOnJc8pN4CJCBCSVPPUd7uwGYTUt8x5wY--

--eYCTYiqErDDNw369RivENVePujGNhH1yJ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE28+sFAwAAAAAACgkQsN6d1ii/Ey/7
Awf+I0zzj1zIYNjaDvWvdeVjGMhyk4ecIzDbA1809V1MqDFOY4Iuv8z+WwoM7zg+dGiuDXXT1ZeI
RZmU8kjnGX08NGr43NaAghXebsFpwodJ43Jhwnpx6Xng66cNnjYtZcmQfbXGe51lNOTxBT9MyKGu
QPeSIN+OWjvr6ER9CXLm9rBzaQzgXtE5EeyQlRbs52gnmwZLQftIyL/MKQ2zaB7QUkzvm59XGWyb
vn3RdYQ7DPupE3ECYF0Sy2Ize6J0mdlrQhJgwkcJxa4Crp3chL0wiHtK2Thrji6Steq8tvawHbRB
iWG4dOc6IioiZN1roj+ipI4IR3rAe63lZO7mQ7YnNA==
=tH3r
-----END PGP SIGNATURE-----

--eYCTYiqErDDNw369RivENVePujGNhH1yJ--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 05:16:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 05:16:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180373.326994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTT3-0002CL-4o; Tue, 07 Sep 2021 05:16:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180373.326994; Tue, 07 Sep 2021 05:16:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTT3-0002CE-1E; Tue, 07 Sep 2021 05:16:21 +0000
Received: by outflank-mailman (input) for mailman id 180373;
 Tue, 07 Sep 2021 05:16:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTT1-0002C4-3j; Tue, 07 Sep 2021 05:16:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTT0-0003vg-Oc; Tue, 07 Sep 2021 05:16:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTT0-0000Y4-HV; Tue, 07 Sep 2021 05:16:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTT0-0003Me-Gz; Tue, 07 Sep 2021 05:16:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=maTfaYDWmwiW+WzogC0xfR3utfB9F1Ck0mkDpBgjATE=; b=b6OoRUz6X9ZrOia5P2pmSUAY6y
	4vF18/DwyNkdXIbcOnw5FaXIEC1trLdJlL8LZPrIiES93esMZb7pX+Fy6bS3KS5WmNaV4YAfKo177
	7k/hg2vOJUfczUd55j9kxC/3CDAWdlC7+eahahqcSrhSR2RXhyCSyi5kzZU1Oa7A3w+Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164867-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 164867: all pass - PUSHED
X-Osstest-Versions-This:
    xtf=91d215a4ed1463ab14d1f68e497117ac1255e05e
X-Osstest-Versions-That:
    xtf=0bb720b3c486bd3de62b8c32282eb5fa192b87f3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 05:16:18 +0000

flight 164867 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164867/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf                  91d215a4ed1463ab14d1f68e497117ac1255e05e
baseline version:
 xtf                  0bb720b3c486bd3de62b8c32282eb5fa192b87f3

Last test of basis   164779  2021-09-02 21:40:04 Z    4 days
Testing same since   164867  2021-09-07 00:10:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Michal Orzel <michal.orzel@arm.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   0bb720b..91d215a  91d215a4ed1463ab14d1f68e497117ac1255e05e -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 05:48:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 05:48:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180380.327008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTyU-0006K0-I7; Tue, 07 Sep 2021 05:48:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180380.327008; Tue, 07 Sep 2021 05:48:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNTyU-0006Jt-Et; Tue, 07 Sep 2021 05:48:50 +0000
Received: by outflank-mailman (input) for mailman id 180380;
 Tue, 07 Sep 2021 05:48:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTyS-0006Jj-Sy; Tue, 07 Sep 2021 05:48:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTyS-0004S0-Mr; Tue, 07 Sep 2021 05:48:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTyS-00028l-B3; Tue, 07 Sep 2021 05:48:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNTyS-0005Il-AY; Tue, 07 Sep 2021 05:48:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=m0NPz1K5DqGH2f2fWD7tNhQm6+HJEi1nopP/LaUQsXY=; b=TuI5Sj1tg81gL6wXUBhXQB3Mgh
	PGMWCL2MTUgFGePDP+d2yM2jAIkvh49TClfvGxG1YSJUHOPmhDezv1D0x1FHFJw5zDnn/l+U4T48z
	RFRiIvfscYoSbwmu58WW+4mdmDeRmIF9cx7t/LpJhoP0i3OI16/TdBQhqUTkVUZ0mkHs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164865-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164865: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b93c544e90e2b28326182d31ee008eb80e02074
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 05:48:48 +0000

flight 164865 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164865/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                4b93c544e90e2b28326182d31ee008eb80e02074
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  402 days
Failing since        152366  2020-08-01 20:49:34 Z  401 days  698 attempts
Testing same since   164865  2021-09-06 21:40:45 Z    0 days    1 attempts

------------------------------------------------------------
7221 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2221825 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:00:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:00:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180387.327021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNU9Y-0000Nz-M7; Tue, 07 Sep 2021 06:00:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180387.327021; Tue, 07 Sep 2021 06:00:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNU9Y-0000Ns-Iv; Tue, 07 Sep 2021 06:00:16 +0000
Received: by outflank-mailman (input) for mailman id 180387;
 Tue, 07 Sep 2021 06:00:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNU9T-0000Nm-Fk
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:00:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 52529593-b699-4746-8d3d-60c68498e07e;
 Tue, 07 Sep 2021 06:00:08 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-_x-3ZE5wNjy6Jx3_XoJB2w-1; Tue, 07 Sep 2021 08:00:06 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM0PR04MB5905.eurprd04.prod.outlook.com (2603:10a6:208:125::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 06:00:04 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 06:00:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0101CA0052.eurprd01.prod.exchangelabs.com (2603:10a6:200:41::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 06:00:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52529593-b699-4746-8d3d-60c68498e07e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630994407;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w83dYj+PDR5QgdknZkEX8CLqhAE+6ALYyEqWNgI6TFA=;
	b=hPm1l70O/RipXoriEuaSty5kILEoPs61WyX3P3HcEuYJeFuXliVMuEX07XlqP9PhjdfTLM
	AvbCfcgRl3P6eRS59+klkadfN88kPyV/iUHd2dukFCgnTNLkTZvbjiPE+ZSjZcKA7rmQSw
	q2J6871EMckI/Wwa34Y4GsqDzquJQVg=
X-MC-Unique: _x-3ZE5wNjy6Jx3_XoJB2w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Lmsgw4LESNtND2ApvG38Km/fbF2zJCXhgiwJnvUBl5P4TbT4lEzJCD4tMl3j5bySPBXHrJpSDHX49l9MzgGb+MTPX1o4NMfQutgSFllTlH0GoroZVHoQBYD+6CcwDicSCUkrTbmh/hg3AuNgyFfaVfTFKT65/Xg6kRB9U1Q+5VdqgB2CGkj2YPgVr6u4TsfbIVD1ao+TI5z8xBnBn1lPTPHhIdv7PghiQUUFcdtuHMnMGjODr7W6ikDUXjWcW3hHklRF5OxcP91sXJCe9WPYyDDA6NVDDyTPlI1f4aQXje9HpFAf0t0KqoiJVKDASxwa6uVau6Pl9AvvpJwH1eicnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wdBhc89sJrcmeLMeJkCx0mR3ZXIDHKOAM3UfkYNp9/0=;
 b=KbkZAH9+YVhOQBmnTvrJSjk+KaLrQZ03RXMP8rwCxnic11rb0VvCx0JNBCISGW/K5Y+35omo8fqPcY3ke9T1arYCwzfuixqa13miilNGqw+s/21s0uQBzEWpgqUTIdB2cmoi4FT/DLzb1KKmYlSYe6KsIu9MoB77D9hBuIVs0gaKLJq6poc3FHLAMcDkQvUE8CmG5zrcbGVAyWVwOdWerqE1KIM/mYpem92zBrrN9Ppo0MBdu6W3Zu2A15frNZySgW1HG/V9/XF2dh836LSlcSB6KDttTi2iVAy6hcr1SXnCzroQ4ldnq//gcbhNsC2cxtAg2khOk+G+QO1xgZEotA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 01/11] xen: Implement xen/alternative-call.h for use in
 common code
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-2-dpsmith@apertussolutions.com>
 <3a91e4ad-50c4-205f-3d90-f23c996b2938@suse.com>
 <8c3dfbf5-5c98-c923-ca81-28af3d13c4a4@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7e71ce74-dd81-21c3-db78-c37d95d5b239@suse.com>
Date: Tue, 7 Sep 2021 08:00:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <8c3dfbf5-5c98-c923-ca81-28af3d13c4a4@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM4PR0101CA0052.eurprd01.prod.exchangelabs.com
 (2603:10a6:200:41::20) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 14001ceb-ff84-4416-7d5a-08d971c4bccc
X-MS-TrafficTypeDiagnostic: AM0PR04MB5905:
X-Microsoft-Antispam-PRVS:
	<AM0PR04MB590556E6E119ECBD8ADABE84B3D39@AM0PR04MB5905.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	H7ttFfQ3qWqnMmw99mi5eUcYejrNwksYux9Ys8WEZdMAHfkpUftxRtrl1ijmFmOqgKsMaVLo23hDnqR2sVKYRGx5JqWvJL3hGZsEZJ7xXXvIdET3fIH+0PFdCXSNJSrN6OkCimuKAepqnUR7FVNjsAuKxTQS9Y5IRE2yuDRQOFDOwUJyw8YW9s9MxKMJnAr93LfCWtIUJxduzqvBxr5huJQrAJ9Ab8vNw0c40m26eAufr+pXaG9dRfEJ+HNYF8BFcYcUahRJxozWIH0b/RdsHkL9mnvVm9aP+d5oJIsnuF7bXArERm3r2ZAObEf10LQWwizRpaTk1GRosvcKv5nCu0rRLVJGsvrW5v8Y6G2T7UhC9SEZNuwhcb7ar64bynfp4kB4leFbs/N9qxJ3tOJtu5r3wMNLmZSwkEkloi78DmnpRWdoAK9DbvtivMSHSc4b+GUbYZr9M93GM3ox19Ilc/q+DweykZ9t2CPuQO5RXwgOaSSjVuNVkZgnTPf1H5EEvvvakwnp8UiXZjp24CzIf3sKoYfBi7iXg6aerB0zgjqgYYDqRv0UsCVdsxnRmEbYfsrQS64PyB7vd4mGMmGfv3xYY2q6OGnZYyFfNKtu28UKpG24DZwypha2Q1oOZQ8DqcY51ewq0O8QBjY7P5qzg2K17gb4L9RbqR36cvYQ82kP9C2ximGChMu5IVMbGZsWmS6bJ0uyFyPgSpN12iuE1w==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(376002)(346002)(366004)(86362001)(8936002)(26005)(38100700002)(5660300002)(53546011)(316002)(110136005)(4326008)(54906003)(186003)(83380400001)(16576012)(31696002)(2906002)(956004)(2616005)(478600001)(7416002)(36756003)(66946007)(66556008)(31686004)(66476007)(6486002)(8676002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?dzyN9tY8BYalUgFoqgWIGUxwOGT2FrB7VfTjX8O+hAXX6/aekdTj9KOGk67t?=
 =?us-ascii?Q?VAtLNTLvBXOvx9m0YnsHyDHH8gZk2xjymkEvGKLFP/uS9wSrOHkTJFywjRAa?=
 =?us-ascii?Q?ySNuTu0SSCK9R2uyELBG7jY30nOF2k7xkkTdkIFCGDA0dAPap8p9a2gp/eUd?=
 =?us-ascii?Q?4Sk5LGvriljUpAH016ikdtfvtA5HrbIXRVUMtZkrJVa/CJhGkIv1aHK6lo/4?=
 =?us-ascii?Q?Fwmcu9b/eGHAnL1HHz62O2aCRA+41oMTpurpYCkKE0jkhd0YOklG8bpF0yX0?=
 =?us-ascii?Q?G1nFpg20I60elkc2Ab+ElNFWkox0ej1JEtaIewhdvZxsPyipt+ITxuugnVSy?=
 =?us-ascii?Q?uVhQ3I1g3BWAii0zII1Bu+SDXkbTk1JC6gzmmcNBEdskIT/aikjgUh13fbsX?=
 =?us-ascii?Q?YmVFqEd4Kd3SDsYCdiTiEAzYT5gX3nNx//AZiYccKR07UDM8oOKA2ya8TO0u?=
 =?us-ascii?Q?lVFOuUbMaLEf2KWJRd3sx+1QDd8WTd3W3OltUSOCKtGvHl/LnZ6m+5p2hUqO?=
 =?us-ascii?Q?783l0UHSC6reoIJePsuqnO6wkv9Cd/9bYyy23upli1wA7no7verHoJUVEExZ?=
 =?us-ascii?Q?MUqlYmCku13Nfjr2dNHYJ86z+JMKU0/rtLDOh5Hm0Oi6LJgv8xArGXiw1oAH?=
 =?us-ascii?Q?62TSWorCVPn78xPR8xPgsaZrcSC3dy/e3rvFSV+oRfd26yWijX+MZZPFwHNr?=
 =?us-ascii?Q?tZKAk0w9kkIV2a8I5R/psS6t9N3PD3BsZWFbFHVlkCOiFM4c1iwaouocCOrL?=
 =?us-ascii?Q?WA70SsBH12BB5XbxnBqu1DgkGD5e3d5jCM3gSebaRglyOYJp9KJG7x5maoYL?=
 =?us-ascii?Q?7c9qivnuoyl9sDyPWJQNl60rfyiJTpokPPtTyT4vfDN8eTu3Bdg/BDCTRO1q?=
 =?us-ascii?Q?BIKlKVd5njTicuIYa/hCw0b2L7wkwzmUrownAcRtI68eehEnhl5ldPtyRuT+?=
 =?us-ascii?Q?M51GH2jSkaaMxhTW7JauWA32TuWt1BReWUkNVThhYBKGSFsD+vmYORynU6k4?=
 =?us-ascii?Q?A399oGrAy+57eyr0DwQtl4yT83lHfj28gYGDhlq0QuBRJypHdpro5o0+nDmQ?=
 =?us-ascii?Q?WtBa7y//F1WvBK3mOD21n9dwE9kvDbHNl51HwSS9sTkkOpEjbqPbt5jVuhjV?=
 =?us-ascii?Q?g59+5Qqr7NrFWHkgPWt9QQX+IMcNSK6Rterjf5sMRk1MsXJzOjtD2GLbN2NN?=
 =?us-ascii?Q?mUu/EdqOOG5RsuO1waWrWvN7MtbEsnVWrAPTj30qKfioHokN+UVMoAtgDYvX?=
 =?us-ascii?Q?oCfn4fYKYSl97LoHDfNlKu4YDaM0p2cMXGAIVv3BczdsvJBmgdWVvAjpsxTC?=
 =?us-ascii?Q?u9+hj8Ssyhmuhbv3xrYDokWV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 14001ceb-ff84-4416-7d5a-08d971c4bccc
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:00:04.6501
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UNB7QpZ9odRRqHEJQQzmwZcymWzZdUI65i3gN3l2Ijwx2f6qPwS2A9HgJH8NXRwBM8U6XeJgDBsYCdFGXkdyjA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB5905

On 06.09.2021 18:22, Andrew Cooper wrote:
> On 06/09/2021 16:52, Jan Beulich wrote:
>> On 03.09.2021 21:06, Daniel P. Smith wrote:
>>> --- /dev/null
>>> +++ b/xen/include/xen/alternative-call.h
>>> @@ -0,0 +1,63 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +#ifndef XEN_ALTERNATIVE_CALL
>>> +#define XEN_ALTERNATIVE_CALL
>>> +
>>> +/*
>>> + * Some subsystems in Xen may have multiple implementions, which can b=
e
>>> + * resolved to a single implementation at boot time.  By default, this=
 will
>>> + * result in the use of function pointers.
>>> + *
>>> + * Some architectures may have mechanisms for dynamically modifying .t=
ext.
>>> + * Using this mechnaism, function pointers can be converted to direct =
calls
>>> + * which are typically more efficient at runtime.
>>> + *
>>> + * For architectures to support:
>>> + *
>>> + * - Implement alternative_{,v}call() in asm/alternative.h.  Code gene=
ration
>>> + *   requirements are to emit a function pointer call at build time, a=
nd stash
>>> + *   enough metadata to simplify the call at boot once the implementat=
ion has
>>> + *   been resolved.
>>> + * - Select ALTERNATIVE_CALL in Kconfig.
>>> + *
>>> + * To use:
>>> + *
>>> + * Consider the following simplified example.
>>> + *
>>> + *  1) struct foo_ops __alt_call_maybe_initdata ops;
>>> + *
>>> + *  2) const struct foo_ops __initconst foo_a_ops =3D { ... };
>>> + *     const struct foo_ops __initconst foo_b_ops =3D { ... };
>>> + *
>>> + *     void foo_init(void)
>>> + *     {
>>> + *         ...
>>> + *         if ( use_impl_a )
>>> + *             ops =3D *foo_a_ops;
>>> + *         else if ( use_impl_b )
>>> + *             ops =3D *foo_b_ops;
>>> + *         ...
>>> + *     }
>>> + *
>>> + *  3) alternative_call(ops.bar, ...);
>>> + *
>>> + * There needs to a single ops object (1) which will eventually contai=
n the
>>> + * function pointers.  This should be populated in foo's init() functi=
on (2)
>>> + * by one of the available implementations.  To call functions, use
>>> + * alternative_{,v}call() referencing the main ops object (3).
>>> + */
>>> +
>>> +#ifdef CONFIG_ALTERNATIVE_CALL
>>> +
>>> +#include <asm/alternative.h>
>>> +
>>> +#define __alt_call_maybe_initdata __initdata
>> My v3 comment here was:
>>
>> "I think it wants (needs) clarifying that this may only be used if
>>  the ops object is used exclusively in alternative_{,v}call()
>>  instances (besides the original assignments to it, of course)."
>>
>> I realize this was slightly too strict, as other uses from .init.*
>> are of course also okay, but I continue to think that - in
>> particular with the example using it - there should be a warning
>> about this possible pitfall. Or am I merely unable to spot the
>> wording change somewhere in the comment?
>=20
> Such a comment is utterly pointless.=C2=A0 initdata has a well known mean=
ing,
> and a comment warning about the effects of it is just teaching
> developers to suck eggs[1]

Well, okay then - at least the definition of __alt_call_maybe_initdata
isn't far away from the comment. (What I'm not convinced of is that
people knowing __initdata's meaning necessarily need to correctly
infer __alt_call_maybe_initdata's.)

Two other observations about the comment though, which I'd like to be
taken care of (perhaps while committing):

- __initconst wants to become __initconstrel.
- foo_init(), seeing that there are section annotations elsewhere,
  wants to be marked __init.

Then
Acked-by: Jan Beulich <jbeulich@suse.com>

Daniel, you having made changes (even if just minor ones) imo requires
you S-o-b on the patch alongside Andrew's.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:10:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:10:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180394.327033 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUIo-0001QF-NG; Tue, 07 Sep 2021 06:09:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180394.327033; Tue, 07 Sep 2021 06:09:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUIo-0001Q8-K9; Tue, 07 Sep 2021 06:09:50 +0000
Received: by outflank-mailman (input) for mailman id 180394;
 Tue, 07 Sep 2021 06:09:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNUIn-0001Q2-Ji
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:09:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3422ff0a-0fa2-11ec-b0d5-12813bfff9fa;
 Tue, 07 Sep 2021 06:09:48 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-I2SFiyC0PHSf8gq9h58a1A-1; Tue, 07 Sep 2021 08:09:46 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM0PR04MB4611.eurprd04.prod.outlook.com (2603:10a6:208:71::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 06:09:44 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 06:09:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0701CA0002.eurprd07.prod.outlook.com (2603:10a6:200:42::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4 via Frontend
 Transport; Tue, 7 Sep 2021 06:09:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3422ff0a-0fa2-11ec-b0d5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630994987;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=intlh2v0bqwJEaN4Bk097uOqhuTDV3eIWchBM22Tee4=;
	b=aV4wV8NhcS1f/dnBgkrI0DM4ytGBv68YsQGT2XISstUDig9eS8fzbXtc6llASR4mN8WkOW
	MWW61lbUQCbOLtMyDnEBEalc90acpljJV+BRaW9PnXzIctkvbL9EqPcaDtdsZotJwx4j2w
	AWyWDHjd7ZTGDR2xoqTe4h/PWSuyXiI=
X-MC-Unique: I2SFiyC0PHSf8gq9h58a1A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hcbsu/NNkyMxzR3bZfLME8+s1JV0JyI2Uhr3w4fNcvbXqtGc/uCrJkeSavDumnckzNMCBQ3T1LcfbrkAmrkqR1BzKOZVAVtimIdh0rlnuc96XZ8+VwhvtkrTLIVJpHaXJ5KgrEh33Ul6+FdvysuaVesped2xxWgL6nAPmWNVAXSkbdJmWcQIAmZK+dgH2Sqt7yTNbOOVVunyFRcfhcDqzO0zsh+4ffTJNQmK0EKsapsL32OpzU9FWDI4tok+Ci7EkSxyWAAWtCeXPH2w/yPDbxn9hXDEvpoplGD9zfyiQ4oTzFdtdyNrBWCBpWq21SJblw+VfoGSr6eWLBzd4HSuDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=intlh2v0bqwJEaN4Bk097uOqhuTDV3eIWchBM22Tee4=;
 b=WbNhEBKMFas0v7so4B4EXMK9VDttK5CrggaDVaZqYQx1XKXr+YTHtmInwHsLFbOIxE0XUGU3swoxoFCi15mYDKRMb9PGa2+ehxPnS8FJmToBtyP9sqcG+DEBg8lXbrN4V6qLsJ5tD1nrAac1QpfXdXnE/7wIAAbpTar39N2+K1Z8/lxcy/LL/Eo3JqOwA+GQ5zamfLgaqfjKu0uCNPQvUccrUp2FrDS0IfNPEiwrfBweBiKlh7N7SUQrDII/Fr7Yb1A1BV53i/dF+VN7hlsMC9gHSXkbntc/35IE9BZaR3sQD5LOsyK4txW306mZhWyzPgvMYotPkjwldPEXK3ZtjQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Pu Wen <puwen@hygon.cn>,
 Andy Lutomirski <luto@kernel.org>, Xen-devel
 <xen-devel@lists.xenproject.org>, Jane Malalane <jane.malalane@citrix.com>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
 <2972c71a-5fed-0813-8ec0-d179eacff14d@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <70f376c6-eef2-42ce-a8eb-67c1130a41e4@suse.com>
Date: Tue, 7 Sep 2021 08:09:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <2972c71a-5fed-0813-8ec0-d179eacff14d@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0701CA0002.eurprd07.prod.outlook.com
 (2603:10a6:200:42::12) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1f42b0e4-3df2-4116-88cd-08d971c61668
X-MS-TrafficTypeDiagnostic: AM0PR04MB4611:
X-Microsoft-Antispam-PRVS:
	<AM0PR04MB46116793E746B4C372CF9757B3D39@AM0PR04MB4611.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oLzwCsgJlpWEVEYU8ia0oPbj8WWhy6mWDF4u/iJixc/tU0IECKC+E36AMCdW44Pu6cLxEGThCFvrhq9Qt4a0KbWphiRcLPiotMZjZ/SoCDSNDWCMfB4rLKcMvWlvp5zJosOFJzYaX9s9R+S51dwp5WK8LlkEo+tchDWgBQ6I7QVsG2si85F2Ye+VsovxkiTo++8IECHw6HWdcx4RDmgVe5Gie9CI/6M/AYC/KldiurePOlvDkh+KWWQ9vAlNpSA9Wh7E6nnySubtU7tf44J8goSa50/jt9OQ45FJ8nR8LPAEHckmzx0brCG01SVRlZxnpW3QWUxsnKPTIts6zeUFFZi73AYg1nAVYw3ZvKlhNM6V8/PFVWuvKjue79udDYMmqfiWHoDc9VOdFLarwPNY7WENCXsPKzrP/9L0aZLOFQ2A4jvD0ICSDZLA5YdAeQKQBrYV0f6RtonNvYWulAjI3tPpAnXkQ7dOPj6YEhRAyEoU3Y1c7F1Jv1/Ha1MO7bTvo5gwSPmSTnxpxGfYPcqOL7rBIplvgxCVc7ZaxjqZISeAjPUtLdgnBW7EUv0OG3t/afKHt9ZDDMod6TRlfef5aWR1hoZb3SaIr6af3EXQCEr8VJUC5nGUn4MtCnojFiglDbmj4cB9o2xxdsnLsi7pb92Fzm2vQNHCyT0X1d0FUUB8zd7d0ZzZbxtKvU72RXhSpGjhagZj14BUKfkn4fWzNvdevARVQ3Bc6f1xkBPwDlYtddcTSpVrqt5kL+fswrsi
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(346002)(366004)(396003)(31696002)(2616005)(478600001)(8936002)(4326008)(66556008)(6916009)(66476007)(66946007)(38100700002)(316002)(16576012)(6486002)(31686004)(54906003)(86362001)(53546011)(83380400001)(36756003)(956004)(26005)(186003)(5660300002)(8676002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VWhEM1dTQm53Nnh4NGlkN1V5bkN3ck5kaHd3ZFhINlZ5SXIwM0RndUw3alc2?=
 =?utf-8?B?RlNYZ3hXYWRpY0JLeitsU29peVR5blpqTGU0N0thTnhrdnM5L1NmOGF5OWpK?=
 =?utf-8?B?djJZVmVOL0EzUlVmdm8rR1hvU1NQUVVtdlZQQWtycFdjZzB2OGhMd04vSUwy?=
 =?utf-8?B?NVNlbjhGcHI3bDN0aTB3MStRMUdRRC9kbDJtc0xFQnpOQUZBdmpOVHo4Zk5R?=
 =?utf-8?B?YVNnMytVNEpRTUJNZDk0Nm41RENGS2o0aFVNaFhlMWRsamVVTnJrSUpXWGZv?=
 =?utf-8?B?ZnVibWlsTGZoN0dPN0dvRUlVbW5EWUMrUTNnM3pHOEwzL0pvcWp1bXFmRUJ5?=
 =?utf-8?B?bDExSlo3MGJUSmVOU1ZUalhpU3ZkT1drcjRZaWVUc09yVXV4SzZ2cXNtYzhN?=
 =?utf-8?B?ekQwdm92REdnVHl1OC9GdDIrUlZ6L0hLTXJndWJTZkJXeWV2clg5M2hRdUhy?=
 =?utf-8?B?eXpIaFFHbnRlem1lc0daWjZRUFpjYTFPZjdabVJkRThHcHFROEJuTkNkejBE?=
 =?utf-8?B?MHhFaGN5YkY0N2lUS043em1TVzdPeTZKdjJpbUMxQWxyN2Ezdm1vTTNEcXVY?=
 =?utf-8?B?RldJWGc0bVZ3SXAyWGFuNGJtV0ZNSDFqaVpidm1zOWpZYzRZblFkaUoxaFNO?=
 =?utf-8?B?N3JiaEdheVZnWXhzdUE0VXhSNzVBVlh1VUJISjFXZnV0a1A5M1ZhSURIODZS?=
 =?utf-8?B?d0ovOWxueHZjNFU2Z3FDcUw2QlJEVmJoblkrTXZKcjllZTN6OHNXODA3RVZp?=
 =?utf-8?B?T0Z0bGdlWHY2K3dXcG9EUXlSNEJOU2trbmJkUWt5RmM1RGJKK1ozWm56VXE5?=
 =?utf-8?B?dWhMVUxXSmMzQjBZSmU3QWZIYS9pSkNMWjdqVko0SGIrRWQ0dnlZdjlYclZj?=
 =?utf-8?B?RDNMQTRhT1N6V0NOa0FmdlNGVDc2bXUwY2NSR3R5SEpDbGZqazVTNkVJWWJP?=
 =?utf-8?B?RXpUajliZDJ3Q1NaUlRDWk4yUXRKaHN3b3FUT3pLUlFmVTRyODRvUXNBWXJB?=
 =?utf-8?B?UDEvQUEydHYyeWVvbE9oUXg4VjVyVWs2ZDB5aGsvT00zY2xUQ3pTcWpwdjVU?=
 =?utf-8?B?NlRWaDNuZjRXMjFNYmF5UTNNNzRkdGFQWmhmS2F1dTdwU2w3dlVEZmVscnov?=
 =?utf-8?B?ZHU1bllYbVMrN2pmRG4zQUcrcXBwYktWalhVSkJJWTY1NnJJT20rQVRCb1hh?=
 =?utf-8?B?RC94UVhZNWF3ZVMzY1NTM2V6M1NPRENzcm96MzV4TW1LK1VQakJ1SFM5MGtE?=
 =?utf-8?B?ODV5VjV3WkZFdmVSYjBKS0UzREgxWWxyaE1pTGhvUTFaeCtYNldxVGlzUlRR?=
 =?utf-8?B?OUU2amRRMzRVUmdJbnF6OGZzMDNPc09iRXlqYVErZTRlWEtwbVdERnJkQ3RC?=
 =?utf-8?B?ZXUwdmFidkwrMHRZNlJlRkdoTm5kMHlRdDFZS3pYV29KbjNpYkc5U0NqRlNN?=
 =?utf-8?B?OVYwcC9oa2c3L3grM1hhRTFtcmlzdFJwSGN3OUdYa09SMWpEdkE0aFBWQWRl?=
 =?utf-8?B?TXp4a2I3VytHMTBQUmIvOTZFZTJHRkkwWXZEdVRJaFBRYkRXZDlyTFYvOWZq?=
 =?utf-8?B?U0tNQ29BOXh3d1Z0aGp2UXFHNzlvRnBDZlVtYmVUbDFITHBEUUpVdk9hV3JB?=
 =?utf-8?B?amhGN3AveEpyVyt5bmZkbU84VlBLbndhMFBNOXNVQlQzOUNNR0I2TiswM21K?=
 =?utf-8?B?R0kxNmk5bTRzTEFmeURHNzM3eDNPMmlMbFkycWQwdGZIa2FDRk5iOGxBQlZx?=
 =?utf-8?Q?frwx11arucvmqtG/L+Z5vi9OjXR9sye0svjP/3Q?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1f42b0e4-3df2-4116-88cd-08d971c61668
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:09:44.4237
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zrlXNOH2d3K10CZXYFx7pC+O0tNHhdwiSffOj27flzzms69sKdbMJoU/ZievS9k1nRDdpvbotinVz7WafPficA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4611

On 06.09.2021 20:07, Andrew Cooper wrote:
> On 06/09/2021 16:17, Jan Beulich wrote:
>> On 06.09.2021 14:00, Jane Malalane wrote:
>>> --- a/xen/arch/x86/cpu/amd.c
>>> +++ b/xen/arch/x86/cpu/amd.c
>>> @@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
>>>  			  c->x86_capability);
>>>  }
>>>  
>>> +void detect_zen2_null_seg_behaviour(void)
>> This can in principle be marked __init.
>>
>>> +{
>>> +	uint64_t base;
>>> +
>>> +	wrmsrl(MSR_FS_BASE, 1);
>>> +	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
>> While I don't strictly mind the "m" part of the constraint to remain
>> there (in the hope for compilers actually to support this), iirc it's
>> not useful to have when the value is a constant: Last time I checked,
>> the compiler would not instantiate an anonymous (stack) variable to
>> fulfill this constraint (as can be seen when dropping the "r" part of
>> the constraint).
> 
> This is "rm" because it is what we use elsewhere in Xen for selectors,
> and because it is the correct constraints based on the legal instruction
> encodings.

grep-ing for "%%[defgs]s" reveals:

efi_arch_post_exit_boot(), svm_ctxt_switch_to(), and
do_set_segment_base() all use just "r". This grep has not produced
any use of "rm". What are you talking about?

> If you want to work around what you perceive to be bugs in compilers
> then submit a independent change yourself.

I don't perceive this as a bug; perhaps a desirable feature. I also
did start my response with "While I don't strictly mind the "m"
part ..." - was this not careful enough to indicate I'm not going
to insist on the change, but I'd prefer it to be made?

>>> @@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
>>>  	else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */
>>>  		amd_init_lfence(c);
>>>  
>>> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
>>> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
>>> +	    c->x86 == 0x17 && c->x86_model >= 30 && c->x86_model <= 0x5f)
>> DYM 0x30 here?
> 
> 0x30, although it turns out that some of the mobile Zen2 CPUs exceed
> 0x60 in terms of model number.
> 
> As Zen3 changes the family number to 0x19, I'd just drop the upper bound.

Minor note: Even if it didn't, the !cpu_has_nscb would also be enough
to avoid the probing there.

>>  Or 0x1e? In any event 0x5f should be accompanied by
>> another hex constant. And it would also help if in the description
>> you said where these bounds
> 
> From talking to people at AMD.
> 
>>  as well as ...
>>
>>> --- a/xen/arch/x86/cpu/hygon.c
>>> +++ b/xen/arch/x86/cpu/hygon.c
>>> @@ -34,6 +34,11 @@ static void init_hygon(struct cpuinfo_x86 *c)
>>>  
>>>  	amd_init_lfence(c);
>>>  
>>> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
>>> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
>>> +	    c->x86 == 0x18 && c->x86_model >= 4)
>> ... this one come from.
> 
> From talking to people at Hygon.

Fair enough, but imo this wants mentioning in the description.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:14:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:14:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180400.327044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUND-00032k-A9; Tue, 07 Sep 2021 06:14:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180400.327044; Tue, 07 Sep 2021 06:14:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUND-00032d-6z; Tue, 07 Sep 2021 06:14:23 +0000
Received: by outflank-mailman (input) for mailman id 180400;
 Tue, 07 Sep 2021 06:14:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNUNB-00032X-6u
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:14:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7ff516a7-4b3d-4c57-8e0a-ffd3c287b952;
 Tue, 07 Sep 2021 06:14:20 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-V1VvL1hMNVeMrtjdYwZhcw-1; Tue, 07 Sep 2021 08:14:18 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM9PR04MB7569.eurprd04.prod.outlook.com (2603:10a6:20b:2d8::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 06:14:16 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 06:14:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0023.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Tue, 7 Sep 2021 06:14:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ff516a7-4b3d-4c57-8e0a-ffd3c287b952
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630995259;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lE+pv+2t6gwcK6U4MSVYO6DoEvX4rlBIfkZVESIv7zc=;
	b=URLymQr+qMErGR58zs0ydrkxAtuFJ6oWB28kCFNV2IM0Jvm26T5G/ZkxOsc+snaOb/a0Vg
	R2GE3M6htWSJ6BTZJDIgNkxcHylhYSXGJG3Dqa2Iver6tYhw11fM3wc2lt4Sq7tejk/iPU
	CWF2URwmXNRji8UPhy3j+ZW1GZ4fKKA=
X-MC-Unique: V1VvL1hMNVeMrtjdYwZhcw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hr3DKHK7A9LutM5P9cRMQIn50PlzDY+ZN17mvsB1JAqyMRB1K3u90/sLb4BAVYUrq/cZjr2GCSKjGh4/B62LseCj6HHJ9TCYdjXjSjbFoZfW50d5uzJtbl8qRp2azfeP5zEv/mHSMF0Cfxl8IFhJcg32O7PdFRTsbmB+xrzSocsEvVFp7naLprZDYRVVhGR4G4NXlClc+bIjjSmU4932se8LXS/oGvp4K/x6COdrb8j9wk3663uVNoTrOzz9gSjmP3PGaWRZIsUR8vukObQUk/sDBWrNaQmdrnmN9TJ6Tx/VIA530eBxaTMSm+5iFtrId14CwxRWQs5frSUWs8JufA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lE+pv+2t6gwcK6U4MSVYO6DoEvX4rlBIfkZVESIv7zc=;
 b=TPd7QSzH3Xrhv5V4PAzNTasoHSu8826fko0zq4zmw8nIMTcMlAyFMLkTK356baQx6KgrDveeLh4Ril1q2FTeUEJJX5sQYJXRsV+N/4dtXjhxt1pf0U8LYjMpqMT2+djFN5P7tcUT2vVDszXiPIYFf8wKx0UTvo4ReH0KGYLIK+9T11f13AUXzIzB3FQrjaBnLSmPR2ANiyrqjBiEwHrlJR6tL9HqEriQp2x+l7PxQEqFXuiHLjPNLcb8vP+Aoh8AGyLuTMi6QXbfQHtSIkKHtSKDdX+bVrLW8PTskg82ALf+d/tR8Ipf/YSm6iVSLXKO9eo9jP0y2AkyalLgcU6NSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a
 single .c file
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-6-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com>
Date: Tue, 7 Sep 2021 08:14:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210824105038.1257926-6-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0023.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::28) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 85cb63a9-dcdc-418c-f445-08d971c6b876
X-MS-TrafficTypeDiagnostic: AM9PR04MB7569:
X-Microsoft-Antispam-PRVS:
	<AM9PR04MB75692A51FCDDE7621C4B46E6B3D39@AM9PR04MB7569.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7sAxio4bnaBRVvzEcBzNpwkR1auwlW/FZwG/PTFPuyHEC0BYmHbbntGwyh16slzFhEl0MuZ2z/68Mhyx+3VPUoRHvEUJOE0EdxGWtfW1McYnLQu+qPvkjLfORM9BKPfLOBMinTSmXX7Qf6wqt5wwNC1UWJn0imjmRFdxkGXyujX9apHGzx+qoZ238OT0suIMze8+pKyfDa9zaxW1fAVqTVXuzM8QdsByuFkAeDxJnjoslnc5YrVtpCLqhmFVoB3WWtJzepYJyWo/yA7v9ZbYxWC6euSU9BsNkSmPJco0RePG0ogv94Ar3CDuoFV/mBgu0DENm5S1F0tRn8XugiMO5goiQt4xVhcBhuwmlx5y8+awCoFtiKt3i2bC7S0oYc2vbfeDGP7J8drm6DCDswZ3mbnwslUtnH+vKCIBQweUcvpmh+rVL3FADM49tzaY40aVBUYEXJw6vS/77yVyBQiULjtALQFeIcl9nedY0ZzFa2ZEWJtbcR27s3ha0oA3WR1WlzgZgzYJhIoO3QACmMr13Mq/wN6CaEIRDW+r35Kih52INx0377E1IpejQf50qvkOSUt85vYDJP/AaK8hfP/OLJRkC0lDGRR5IUkvk3nkCVICr9py3n78JbwggtjXIaPvDR9op/fBTszu0RHK54IP2zYBmz1pTlB150JWRtXtTFIRED9Rw8CigV5l4+NKO0ksfBifSazPP3b8jeO4rrWq3NbYPRry6XZLB+VQOzppbYVFxlkLnhZwz0gaXtpWZDCaIorpTJSDGY2G8h2tkddXCg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(396003)(376002)(366004)(39860400002)(4326008)(2906002)(8676002)(66946007)(66556008)(2616005)(5660300002)(186003)(66476007)(956004)(6486002)(53546011)(31686004)(8936002)(26005)(36756003)(38100700002)(86362001)(316002)(31696002)(16576012)(54906003)(7416002)(83380400001)(478600001)(6916009)(142923001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MFNNbDBFSThZZ08yeFNKT3RwaWpYVUkvSGdZWmoySFg1ZnRPU0J6ZVFuL1BS?=
 =?utf-8?B?ZUVDYjU3Nmo0enFnZFNkSUhyZjFpcnFKeTNMRHV1dnJFQ0Y5YlhUUStsTkhX?=
 =?utf-8?B?b1JVNnFPNHRxOW8rbDZ6bVVab2FKSmFOTSsvTG1LQThJTjUrTkFzZHJLYlU3?=
 =?utf-8?B?Vk1VUGpSZjRNdGlZdGNoOFRhdHErLy9PeFNqdGgyQmtPa096dHhXNWw1K1Y0?=
 =?utf-8?B?Z3oyc3A4eksydnMwNXQ2YkUvMk1PcEFMMG1HSjZzNTJFOHh2M2FoelhpdkEr?=
 =?utf-8?B?MkhMR1o2azFOdm1Ma3Nrb2hVUlRWQ0J2Q2V0VjMwSmR0T3NQcEJ4ZHQrbFpl?=
 =?utf-8?B?SXBzMUI2eEE2S1JHWVlHT2NoWkZFTXY5Zm9DY1BoSXRaV25Nc2VlamlvcFBu?=
 =?utf-8?B?QldMbmw3QnkrREVXT3E3Z0hLNUQ5clM5c3BtNTRHTVYwZVF0b3hzU2dPbHAw?=
 =?utf-8?B?RFRxK3g5dFpTQ21XZDd3OUlMaUhYdUZ4bHl4U2RlUjNUYmRsODR2NmNSdHE3?=
 =?utf-8?B?SUhBVGp3TE1zVVI4TVp0OEloYlJtUGJDYWVhVk0wcjhjOVJmWlJvTkdIK2Ni?=
 =?utf-8?B?eWxFdytVQmU0U05RbjgzYlpFSUpMT0RSZVBPdG5nTGxWOWVMQUp2OFdSSTVW?=
 =?utf-8?B?ZGtlY0ZES0lpdGRxc0V0MHllQWVBK0RYdXZ5MWswUi9nTXNuTFFSZzh3cGxU?=
 =?utf-8?B?MUZOdzFFcGpjRzJMS0ZTOXN5WlR3UEUrOWpZTU03US9sUTZ0ZFNlQTJTc3R5?=
 =?utf-8?B?SEJKY21tRkRLSmxOdWsxaTVFR3JjV0xsMnZRQkZkd0JrTzhMcWo3dnZSWFNk?=
 =?utf-8?B?RTBBWlgxdFJpbjRISkJPeGVPMzdnaE8zWE11MzFQWnl1SndHUUZQWTRSWWI3?=
 =?utf-8?B?MnFXTzhzM3RMVDZsQUlDSHNaTjZ0TzF0Sm15YVRycTd4Q2M2S1V1U3BzZUtl?=
 =?utf-8?B?SWxjN1ZPZk9hZ0RCODZLMlVHelhobm5UU1ZVdFBLRUZiVXE1WGpxQ3Axdytj?=
 =?utf-8?B?T0NSZHFtRDMzSEhtV09MTWlMSGFiMHdvaDVWMTBnRXNMWDVsb1FDMS8xMzcr?=
 =?utf-8?B?aWtnSFhDZ3NFVjZNaW45SUl4M1dhdndZQ1kzM3NPZkJBdFE0N0tJSnZMbEMw?=
 =?utf-8?B?R29YU01VMW1RaDJUNCt4ZGJtUi84WUtMNGY5a1FmYXM5elBtUGVCbS9qWWtt?=
 =?utf-8?B?dHJVRm1IdmdnZWlUYm5SSm5mck5Kc2dEcmkrMHUyVG5ZSTlJcTQ2Tmp2RnRy?=
 =?utf-8?B?SWxKNm95Nm5rVUpadXpBNE9xTTBKTDNiMTU3d3NiQVhOSkI4ald6TXVlMEhY?=
 =?utf-8?B?cFIzUFFyMWVtZ3JEMGh2ekNTZXhJZ0xjcGFUWkl4dW84WUNSd0tHbC9sUkdF?=
 =?utf-8?B?Qit0OWdWcG1yN20yc2VhYUNjVzdUaVRjbGJxbFpCYmg2am5sNEViS2lkUkhF?=
 =?utf-8?B?cG1QQlV0Uk5wd3FrVG5CSi91dS83NDloY2NObFBiRzJsdkpxTWxOcWlTcVNu?=
 =?utf-8?B?YWZEbmdBUUYzVzY0MjA4bWtiQU9XVDc5b3IwTytuL2xreWxsVjVqNDZJWEg0?=
 =?utf-8?B?bXp4dzdQYTA5QlNnMGhrRFJxMG9UQnE5dlBPa21KYzR5S2JsRFhJZ25qbnNn?=
 =?utf-8?B?RVYvOG5PZkdrNGJOdWxFWUcybWI2Vll1bmF5MDZwM3UyNlhIMG1OckprREpa?=
 =?utf-8?B?dHl2Uk8vVzFFem1LWTI1VEIyTHV2WGN2WlRxZnRrOFB4Y3hDRDRTckN4MkI4?=
 =?utf-8?Q?Bugx0rXrTL2ZYJYSJkCBweGFz/sLNdBooApVy3w?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 85cb63a9-dcdc-418c-f445-08d971c6b876
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:14:16.2882
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sJCqVni4EaS7DeOD/h21VrvC8Idlkc9It+rTXHSWS2ckgNKIzLl1HvUdBe8BYfZyuLoxTkKiZwm7sqzkXnBOWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB7569

On 24.08.2021 12:49, Anthony PERARD wrote:
> This replace the use of a single .c file use for multiple .o file by
> creating multiple .c file including the first one.
> 
> There's quite a few issues with trying to build more than one object
> file from a single source file: there's is a duplication of the make
> rules to generate those targets; there is an additional ".file" symbol
> added in order to differentiate between the object files; and the
> tools/symbols have an heuristic to try to pick up the right ".file".
> 
> This patch adds new .c source file which avoid the need to add a
> second ".file" symbol and thus avoid the need to deal with those
> issues.
> 
> Also remove __OBJECT_FILE__ from $(CC) command line as it isn't used
> anywhere anymore. And remove the macro "build-intermediate" since the
> generic rules for single targets can be used.
> 
> And rename the objects in mm/hap/ to remove the extra "level".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Hmm, when replying to 00/51 I didn't recall I gave an R-b for this one
already. I'd like to restrict it some: It should be taken to stand for
the technical correctness of the change. Nevertheless I'm not really
happy with the introduction of the various tiny source files. I've
meanwhile been wondering: Can't these be generated (in the build tree,
as soon as that's possible to be separate) rather than getting put in
the repo?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:20:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180406.327055 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUT8-0004Qd-0p; Tue, 07 Sep 2021 06:20:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180406.327055; Tue, 07 Sep 2021 06:20:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUT7-0004QW-U3; Tue, 07 Sep 2021 06:20:29 +0000
Received: by outflank-mailman (input) for mailman id 180406;
 Tue, 07 Sep 2021 06:20:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNUT6-0004QQ-Qo
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:20:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b10da5b4-0fa3-11ec-b0d5-12813bfff9fa;
 Tue, 07 Sep 2021 06:20:27 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-uqOFY_bUNVGWqceMPrDZlQ-1; Tue, 07 Sep 2021 08:20:25 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM8PR04MB8002.eurprd04.prod.outlook.com (2603:10a6:20b:247::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 06:20:23 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 06:20:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0082.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 06:20:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b10da5b4-0fa3-11ec-b0d5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630995626;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TSFfFqofenQbMlBfkKFanf9ysGnmn1I9sU3PxpTZ6YM=;
	b=SIszgxswv6zVjENgiiLb2JiRBX8Hr8gEME5U+xuIbY1bSgXHrnBBiLGfn9PersrzGjS0oz
	1XjPeOR9UZr+x/0KkBSWQj1/RQZX4MzEL5z7SKuA5LQYFrXuSgz0TdNIQbCKFKVEqAPS/u
	hsbQJjzieW0CRPxL6vVBstqxPw6cThs=
X-MC-Unique: uqOFY_bUNVGWqceMPrDZlQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PTpj4yYzUJYGIZnUY8JOmKpSLkb4CFZSAEteaYSVJJfw5W6a87NlYpB+5CKP9eeng++dJYux5BROo6iXNZv249NLbee233dRCsorY1sTYpc6+4x6sDL2QtAhjYQMdueuL6UvU9Q/vlsDHvOXs9bECVIykE2n4dzO6sDGCdBEoDE1Mp5DqqotERzHz49L5E4BYk1OFI5aK1BFaEY9r4o7RU8wlcnq/ruF7MvC9eUc0C4DXLb9pkVgqHOlJbK0yxqAT8hNNYcFXpvNjoFXK8IBjD1krcLcynidS2BLGAeoago+fkGhyW6G+0LYjsLOuHXzrZtfQFutFjDAFsRYf/J1Iw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TSFfFqofenQbMlBfkKFanf9ysGnmn1I9sU3PxpTZ6YM=;
 b=DoZNcrW0AKbLpwpZhBnfeScXCMMbfylW0e0JiTecqpKji29aL0eNqiTp7Gj6vdomVUfPnsGZaSBLMqTB7aY7bAwzI54WQUxRmTi6x+WHEjbOA2SIPTobapef5q4MCbVJfzbNg6KtDKvYuNpb+X7+E3DoYA/rNKZ/aw4J0JgIzdidiEMoo2hegY+sis44o34HEhyys/6SF81oJNfyhXwuSK9b0uDJ8pxHHy2AabrOHOa/tTUtscsqMduUhEac/YmpV2tsz9SB7aVMYP7VGCmgHIt2glMcaQv5ZHqP9DOG6j08/bphzAGExP6cG3j5qfj9c6xJoXIStTNfocIHyahIdw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 00/51] xen: Build system improvements, now with
 out-of-tree build!
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <YTY+PNBccQYPhX0a@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ffcd6c45-62df-18a2-74e0-8fc121ad9f6e@suse.com>
Date: Tue, 7 Sep 2021 08:20:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTY+PNBccQYPhX0a@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0082.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::21) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0e6a8f8a-ff46-406d-3fdb-08d971c79337
X-MS-TrafficTypeDiagnostic: AM8PR04MB8002:
X-Microsoft-Antispam-PRVS:
	<AM8PR04MB80023D2C8D23DB775C1B30DEB3D39@AM8PR04MB8002.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2449;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8nkkzatVohMxafv5zc5UYs4lvChbT6NKCleWNeENzySayD0QTr1glm6ts291ZDOVc6OYmz12HPl6hoBFpzpipjgslU+tPqfl4fPFOn+oNQLmpmSNUDKAMmv6BiPYKB0d9jjm1k27zqu/hRRhQSZj6vdxZSaJbuyFFwmLFM+0WB5JiKBj+umJmr9+bMAHDsIEbGAIYo+GM3/+y4FHnSqfqYiGZhginphavw2GH/kY0fIPPv7f1TPNsTfmED7eU8gjobKM7qIx9KVz5X6/pkSIgn9+FiU2IimfhRXs7Ob3aXRSqL6ku/7WH9pLBsBqHzPkfOTvs13Yyyub9YmsnLZ6U354dUwOzyftZyFsRdE4EQwb4pzSmBf+Mm2GxtWXVE8Kb0E13NTQQn0TYESNVvnwjKZnP42cuGOL9wmK/RVSZ/lQtDEmRfc/IFhKo9cGFBgZe8snZNUGN2XsbE912I3OBU9USb8ci25cALr7ENzkIcOiYO5douZNzu/lu4loyDTtihhi46BEqQxrTWobAtrL0yd88mOfW2obb5qzLNb/VZP0AZ9ghvT5PWwyM4XFgrhPHvRprgKMJpzuJQ2S/oR2+ogURaYLv16TXOEGvQUkkQIpwtx5iKcInwvaWqXf/Pa9lNyN5dQSckzKRshTOjo0GTf9kr3dBINruJIp3vLlWl8RbA7Vq1QS2x7NS7n6eurqdHvk2hjcVHP7Bmn+On9svI1CK2MFWPxaXaZW3TLWvCk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(136003)(39860400002)(376002)(366004)(346002)(316002)(4326008)(83380400001)(16576012)(478600001)(5660300002)(6916009)(26005)(66476007)(66556008)(53546011)(66946007)(186003)(31696002)(38100700002)(86362001)(6486002)(2616005)(956004)(36756003)(8936002)(2906002)(31686004)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dHNBOFRQVGVUd2Vlc2tPT0JtT29HYjF4U0N4MVlVNlBFbmJxRTBOTDltZFE0?=
 =?utf-8?B?c1JUMmZENy94ZHllK0VnUnM1L0wzdXVFd21CUFhJSlprK3ppUEVTejZNeDVY?=
 =?utf-8?B?Q091bGgwRkdKSmh5cEdwUVhUMUJqTXpjOUxhNUJGeWJKbGJaNHZFY2IreTRR?=
 =?utf-8?B?WEhoV1grUXFBUVhVeG10MVlDQjYyU3QxNGQ1ODQ0U2hrVS9lWVRidURoWlJ4?=
 =?utf-8?B?dkJpS0NjU1R0K0luaHpBbmlFRUkyVjU1Ly9HMjlkZXR5djg5RWtaKzQ1ZGpJ?=
 =?utf-8?B?UGRYdDRvbEZFK2lTOVF1dnFoZWNXbmk1TkhibEZtN2dzWHF2cEkrRjBaWWIx?=
 =?utf-8?B?c2JkbU8reXpQZytnWEozU2Q2UEVHVWFBWEMzZEdESm5yRVFKbGh1QmZQYm1v?=
 =?utf-8?B?VXJzZlJGN09tblVzMHdvd1lqRWRucWpza3BLaHIxRmMvdW1SVU0zYjV6SnZE?=
 =?utf-8?B?VjF1bk1lL0ZDaUEyL1ZlZExYN1VIU0lFNlZ6UTJFbTZMSWtaRmFpNGhNcnpG?=
 =?utf-8?B?cEpvUW52MTVYMERBM3Q4NFdUVjZzanp4bDNSWlF4QS9SMUxOWFZ4SVROcHpm?=
 =?utf-8?B?cEQvMDlSZ0xMYkIyMWVwTTh0RENLTkI2KzQ4QUwyUkNFVDJ4YzN6QlZCVEw4?=
 =?utf-8?B?aFF0cHlhOEdRNkNyWGQwUjFrRFY4S1FxRUdOVVo3ZFFwanNhMEJpaTR1Sjc2?=
 =?utf-8?B?d2k0QTlJTVdVQUJrMGJHRkE4TWtwSlIvS211aGRvbDBFdkQ2YW5RcHZNV1o4?=
 =?utf-8?B?QTcveWtVb3RQaVlSUmRBYm5ZN253T2MzTVg1L2VpTGRjdVE2b3FQR3V6T0FC?=
 =?utf-8?B?TzhPQXJrT3lzVDgxd0tSMW04b1FlNmd5MGl6NEdzRmdIOVVWRzlqTGhUUFRu?=
 =?utf-8?B?S3l3THRlOGd6ZER4RUREOStiL1F3VWltSnlmcEJzNVRlUTJGQ3UwSGxRL09L?=
 =?utf-8?B?S2pxdGtFU0Q1bThaNGVQVXVXVGRGd2ZkOGdGSnJiU2M2VWwrdTRMT28zSkUv?=
 =?utf-8?B?aEJXWXRIZXpOSVJ1SDZha0cyM3RGdk9PQk93M3FIMVRETEc0a0V3bVptOHNm?=
 =?utf-8?B?MmdoUFYwV0M1ZWpnQ0hYZ1Y5RkhwR0ZHcVdLN1BFUzJBa3A4bXZPak1mc2dv?=
 =?utf-8?B?OWo4OTJuRTIrSTAvYm8vMk1vTUF0OHAwaDk5d29EQk9kanpVWWFyT0xJMlNj?=
 =?utf-8?B?d2VLTE92aSsxZlY4UDJjV1BVRGh0dWtVaWY5bXVJYTlZbXcyeHJlREtFUjZ0?=
 =?utf-8?B?WnB3Z3hyTy9UREp1VDRCNGdSUkpwdDhiYjl6ejdpZU9MMUdOY3Npck5VaGRu?=
 =?utf-8?B?cmlLWDNJcllBZUJ1SFVId296NU5pS2ZaYUhINTBwdWc2UzRFc2R4K0xaMWJr?=
 =?utf-8?B?aGRvblkrcHE3c0lLVXlwaDZDWmRQUzBTUW1ya2hjR0dQZFRwMUhpZis0VmY5?=
 =?utf-8?B?YXBHbHhmZGNYM2tvY3hXdzJRV0lnNHRuL1lyamxTYTg4cEp3MlVFZWF5bmZl?=
 =?utf-8?B?T2NMdFVPYTdxd0NJLzg5blpuYUx3blUvcnBSTVBZTlFsOEcvbVpmV2RzdG8r?=
 =?utf-8?B?TkF5ZmdKbGx4TUpZeno2OVFzaHFnWXZiMzdxbS9OU0xWQTZZbmdETzM2Ukd0?=
 =?utf-8?B?WXcrRGxaRTlkd09ZS3hkOXg2ZXRXTnhWN01ZMmpPdmJYVFVKRWkwWlYyMXd6?=
 =?utf-8?B?OGNWZ0RCcGtmM2pUSmhLYk1Jc3NvSkFrekk3YjVLSCtiUzZIb3ZkQk5MQ1ow?=
 =?utf-8?Q?v/Schfsc8f4Ejl3CFoj0d2lu4X55njg1puBQZBb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e6a8f8a-ff46-406d-3fdb-08d971c79337
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:20:23.3177
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WkCqiiGJ2wYU4lIOUUaqZqN0svwuA1DbSbT6kSOC/As43UeWDtpSwaTFSA6DQ2MV5H/ug1eVwvsNm65C5Jo6wA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR04MB8002

On 06.09.2021 18:13, Anthony PERARD wrote:
> I hope this is useful:
> 
> On Tue, Aug 24, 2021 at 11:49:47AM +0100, Anthony PERARD wrote:
>> Anthony PERARD (51):
>>   build: introduce cpp_flags macro
>>   build: use if_changed on built_in.o
>>   build: use if_changed_rules with %.o:%.c targets
> 
> all 3 ready to commit
> 
>>   build: factorise generation of the linker scripts
> 
> may need a ack from arm maintainers, does it? Otherwise, ready.

Ah yes. Will need to wait. Which is the first place pointing out that
you answer my question only partly: Does "ready to commit" include
"independent of earlier patches in this series"? Same e.g. ...

>>   x86/mm: avoid building multiple .o from a single .c file
>>   build,include: rework compat-build-source.py
>>   build,include: rework compat-build-header.py
>>   build: fix clean targets when subdir-y is used
>>   build: use subdir-y in test/Makefile
> 
> all 5 ready to commit

... here: In my reply to this cover letter I did specifically that
I've skipped patch 5. As said in a reply just sent there, when writing
that I didn't realize I had already given R-b for it.

I intend to put in some of this series in a couple of minutes; I'll
try to judge what is okay to commit independent of earlier skipped
patches.

Thanks for the summary nevertheless,
Jan

>>   build,arm: move LDFLAGS change to arch.mk
> need edit commit description, but otherwise ready
>     not needed before [PATCH 21/51] build: set ALL_OBJS to main Makefile; move prelink.o to main Makefile
> 
>>   build: move make option changes check earlier
>>   build: avoid building arm/arm/*/head.o twice
> both ready
> 
> So up to here (12/51) all patch seems to be ready to commit.
> 
>>   build: convert binfile use to if_changed
>>   xen: move include/asm-* to arch/*/include/asm
> both need a ack
> 
>>   build,riscv: tell the build system about riscv64/head.S
> patch ready
> but depends on "[PATCH 14/51] xen: move include/asm-* to arch/*/include/asm"
>     for the rune written in the commit message to work.
> 
> 
>>   build: generate "include/xen/compile.h" with if_changed
>>   build: set XEN_BUILD_EFI earlier
>>   build: fix $(TARGET).efi creation in arch/arm
> patches needs review
> 
>>   build: fix arch/x86/note.o rule
> ready, standalone
> 
> 
> The rest of the series needs review.
> 
> Cheers,
> 



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:27:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:27:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180413.327066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUZy-0005Sh-TE; Tue, 07 Sep 2021 06:27:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180413.327066; Tue, 07 Sep 2021 06:27:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUZy-0005Sa-PX; Tue, 07 Sep 2021 06:27:34 +0000
Received: by outflank-mailman (input) for mailman id 180413;
 Tue, 07 Sep 2021 06:27:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNUZx-0005SU-8J
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:27:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ae3d6b16-0fa4-11ec-b0d6-12813bfff9fa;
 Tue, 07 Sep 2021 06:27:32 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-FrpMNNnVMK6I20FejC95iQ-1; Tue, 07 Sep 2021 08:27:30 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM9PR04MB7634.eurprd04.prod.outlook.com (2603:10a6:20b:2d5::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 06:27:29 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 06:27:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0064.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 06:27:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae3d6b16-0fa4-11ec-b0d6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630996051;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4X413M352Bz+g6WOu/MmDm2tizrfnb8E3vuzxpYuBgc=;
	b=BmMXInTDOy5p81/giVQd3TednnXNdaBxj/99my+eCh3W0eB4Xtasu+yO4FXK65j5MuvE2o
	SdQuRrrvebU8yCPhgcub9RiSa83HkECG9AORPHmeBucH/zNVsqZJ4m4BhHg3hhN3EJP/65
	x2CPZudmWVcsmEUMKKhwqYnXHqUH1AE=
X-MC-Unique: FrpMNNnVMK6I20FejC95iQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JeeNiYH+ScbJodh/gqcrJpJbzdLR1nzFOS6uj5bNaf1TFAyrG7wGfjZuLzMAd8onABRPTLhZrny/En5D4v08IKWJLqNTTexCcIpzXtHd7aHfTVDYRLO+kxSpYHiMqbQpnoIyPsGey5t7mwMgCOCoonxfXrmOgBB0kXlHb+Pe0Jbl5A5E6kAnqrHhrIqSnc4aSp63EJ3jcTf4mHiPdrrx2g8Vb1Au7hkihJzWhZu0RK9k+xFiTJcnFmi3LEDC+7FSJfcPV3JYjhIjQJ6ELWY8zEYQW3Xv341H/TeyzIVXXwmUUcUoB36KEU+BTEixXpogqn5IjoUEU8SKD0dL0ukvlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oLQFaWi4QB5p7y9edaTmQxOxvVOecW8XZ9bI4P+cP6Q=;
 b=YVxfm+fb5paYsyeS1VBZUWKjRkDDM4pjVW0Pnz0G8PFshgrr6KF6CExIS/w8NuGEaH3yhsp4ty1oTJifc75RhQSJ100727l5qfv4B4vGH0iv89kuy39r4ByvkEu3OHjTGXtixqF3XvZaV5PkzpdY5Qy199nTxVW6XHn6fKgrYthW7898NC/7VOZESqXls6K9VG5gN3Z8yZgyxZKP6xuny4T8i89jntqDyl96ejartMxiIV58FJBBk8rfzpgT4vK/IZRECufg0pXe6p8MaZXNls8hky3KqWP/19RPFihdE1T8T460XmMTMtc5Fe6XKABO6XYFBQKupLwlflXsTzOCJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] x86/P2M: relax guarding of MMIO entries
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <aae19abb-e88b-8bf8-6f1d-502ac6645d6b@suse.com>
 <93ecc397-1293-69b1-e70d-3c5f9317229a@suse.com>
 <ad1e33b5-d729-2680-7f9f-aa875c2d9cfa@citrix.com>
 <f684e203-9967-8653-d59f-ad11f5070aef@suse.com>
 <7f56c625-eee6-31c4-2d6f-bd7334867d83@citrix.com>
 <5d69d4c0-7a04-c3da-f971-616363c6d63d@suse.com>
 <62cb78c9-f4fd-76e7-225d-bc575f10850f@citrix.com>
 <818cc08e-8372-35c4-935f-86a87cceec75@suse.com>
 <bf091ebc-95e7-b357-9f84-4ec3a6346fac@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8a45ba41-e659-c472-dbd0-56bbe92bb5c0@suse.com>
Date: Tue, 7 Sep 2021 08:27:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <bf091ebc-95e7-b357-9f84-4ec3a6346fac@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0064.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::12) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 222932d7-0b7e-4832-27c2-08d971c89116
X-MS-TrafficTypeDiagnostic: AM9PR04MB7634:
X-Microsoft-Antispam-PRVS:
	<AM9PR04MB76342EFCC1CFF098346B5939B3D39@AM9PR04MB7634.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5C5wzyRBWgnLYYb/a7R1EJ2jPYH6HVDgTNTHOKgRmrJI9jcv7GI8y/FzSAMxYmA1BoWxdz9UgOH8qJtOLtrp3Ogou31ZeaYfE3Ew5o76UhBGrZ1aOhJ/DvwW8t6MwpW57m2YK9ZLq4IKG1R+up6VgRVmD01+18hqRNkdpa4YA5psB/MdlbBSHRp6+YrbMmaFxQpX7YWOmE+JuRKkwHTtE2KUmJ9pTc+XrGJQjQLdinuAir92nWzit2e8ik9YbPnh0QCLVqYpUn76MGGYQ+9hQAfejC0dA3S1PEUb76WyU9KMiWqLNvHW0LYRxtJYtfdXoQqc1Sr/2945rzgnSL6tQd6ecKVdI3o5jaq+97DEH1UIOFKqaJxATSmjaJYUL7k0snySt9hgHPx7+zlgxXiMnzhhzFrk8tRGpyGQqxuZSAQGA/LoZgjjkoOd9ICXttVqT8NZRZ0nVUYdEKM79OsMq9NNhnFpsu2YIHdPkm6wPPcCVsGWFV3/WqZIMM1hpYdtN2yxcLE+Mnj0BlOhAw25mMutPKbfbTaIIk7FhzGCZ/JXhjMLxrXUIFAD3qiRCv0+wt3oHPvaj5ZaK1rVcztULKi0w22MSrlfIn3vz25Qhw04yQHJpxlPoXj9B40iAu5SgZVx2lLxGWsAN3JHrCKmU4ZGJDykS6hhtCZqy1OKFRQXpis7lSwhi4eGGAGRwkGLGOYxyDnGvjA984wHiwzNoxoxG6bBIGWU3YQ9+9uCeOA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(376002)(396003)(346002)(2616005)(5660300002)(478600001)(66476007)(38100700002)(8676002)(83380400001)(8936002)(31686004)(956004)(6486002)(6916009)(53546011)(26005)(2906002)(316002)(31696002)(4326008)(16576012)(54906003)(36756003)(186003)(66946007)(86362001)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?OxAcRGvYer51QZlj8wC/p8tFLFQAqRGQxoJYLjFj1aAPHVNoy0IV6hB911vt?=
 =?us-ascii?Q?LBSrF9NLqXb3kyZ7gZ0EhANdt28shbwMcnwEU0TFbDJXZOldA2Ww+XZ9JJNm?=
 =?us-ascii?Q?Y811A6cwA2vExlmULFs4Ln+C30YiSjSWpWpRTGNB5AIb0kEvDgsccx2rkpSl?=
 =?us-ascii?Q?6EpB+4XGYHnbcAUz2RLPuIKu1vVmzvdc316FYumSP6XaKvCW7OiRw6YK0UBo?=
 =?us-ascii?Q?Ol61t4UkwEl4GOkRrxJVWndPstnRoG5RaFKOs8/zx9lE7xe9uuRbJPbtkEx4?=
 =?us-ascii?Q?n3FTtGVr06sRnLU/ak9DGZHjCgbosCoT5KWGKbl0NpIP/ZE8n3CXMlj4eeex?=
 =?us-ascii?Q?fDEJcrebDfc8cGcK2RgDvMhJh/mH0sAa+N7DoA1PBx5CmjbR8jj1nXEkX8tA?=
 =?us-ascii?Q?d+MVKofXjH/W1yTZeuOK/9Xcy0tvJHlpbXpCUCSqgrZditPXgisfQ/PcjX18?=
 =?us-ascii?Q?KkQhcmBYwVZqRWhoHwJ0D9TMK4e1wE6837EiPt/dyYd3gIIBhFsiNPX4CtGy?=
 =?us-ascii?Q?hfhxH3wwBIb1lf7otxWKvfg5wVBrsWQM4eZqhAlbbAdeQB35wsXW8cl5ZdF6?=
 =?us-ascii?Q?wKsZu4xG7Puc9nWIXSVScjl9pRDBf9gW7ExTPtEkM3/s6gu29/0KdOVi5wqD?=
 =?us-ascii?Q?FS6/p/wweUakq14P1BDI5inEDD9cxC3VEyFHgsXbL/Z1CTwUGdGJPOPCA8Zr?=
 =?us-ascii?Q?EXf5RIBFGT0PY1f9vr2SYr5pA8E5K7cU57TxIRiUoOILABlgild0seH/fskS?=
 =?us-ascii?Q?zoiV3el4QT7dwNeB3mSE/oBTczmL4R8Hprj4iQrOobLXaQzmciDQeTnkGhwt?=
 =?us-ascii?Q?zRqoSQLLFxpS/MD8JzaIOIDEfQOBgRwQjPK2ECCnLNZ+f3bKXYUijItsD2pv?=
 =?us-ascii?Q?/DdgUYdN3rQ+n/ucd6ztMvarP3K9Xv66vp0M7OcjWR8qphA8jMbPLD/JqHy1?=
 =?us-ascii?Q?FqUqRr9YP/qYze5KextykAxw+SXZ3zZtcLTz7EswtvJsQjUrvCiCgfvV1uqo?=
 =?us-ascii?Q?AuIR747tNSvoqySOI3zN6nmYLV1fwHM4ullteZaTw9e+EoXub3k0lwjzmICk?=
 =?us-ascii?Q?LKwNLNq366DvLpOzwiOTCRDCdX2JTccXR6sARx1Zp1mJ6SkLp2uFZ25cjIxH?=
 =?us-ascii?Q?C8UTsBiCR6pV9CqzYqqDmZ4fCKr9so+xGoVuP6cwPREBy/+zYfMif5KGpt33?=
 =?us-ascii?Q?oeH9Up++lpw2en5I0bR7iQQtCLA2l3ngpQ0JRRI2Vn1Ee63M4eDF5jo8m+n8?=
 =?us-ascii?Q?cJQ4NZdtb/1NPU/P09XFpBxq0Qbk/VPVAaJ6mKNhhke3OU2r3Wk+du1GQCeg?=
 =?us-ascii?Q?L58Z3Cjx/dsotAPkVGd5mt35?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 222932d7-0b7e-4832-27c2-08d971c89116
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:27:29.2381
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +p2OT2AkhIzDgWeuNVG0v8g6e4TGNpq/hrt8hfNK5Qd2G3uOJbWei7JTwheOSvekvs0IITbZys+72oJxpJgIBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB7634

On 06.09.2021 21:53, Andrew Cooper wrote:
> On 01/09/2021 14:08, Jan Beulich wrote:
>>>>> Restricting execute permissions is something unique to virt.=C2=A0 It=
 doesn't
>>>>> exist in a non-virtualised system, as I and D side reads are
>>>>> indistinguishable outside of the core.
>>>>>
>>>>> Furthermore, it is inexpressible on some systems/configurations.
>>>>>
>>>>> Introspection is the only technology which should be restricting exec=
ute
>>>>> permissions in the p2m, and only when it takes responsibility for
>>>>> dealing with the fallout.
>>>> IOW are you saying that the calls to set_identity_p2m_entry()
>>>> (pre-dating XSA-378) were wrong to use p2m_access_rw?
>>> Yes.
>>>
>>>>  Because that's
>>>> what's getting the way here.
>>> On a real machine, you really can write some executable code into an
>>> E820 reserved region and jump to it.=C2=A0 You can also execute code fr=
om
>>> real BARs is you happen to know that they are prefetchable (or you're a
>>> glutton for UC reads...)
>>>
>>> And there is the WPBT ACPI table which exists specifically to let
>>> firmware inject drivers/applications into a windows environment, and ma=
y
>>> come out of the SPI ROM in the first place.
>>>
>>>
>>> Is it sensible to execute an E820 reserved region, or unmarked BAR?=C2=
=A0
>>> Probably not.
>>>
>>> Should it work, because that's how real hardware behaves?=C2=A0 Absolut=
ely.
>>>
>>> Any restrictions beyond that want handling by some kind of introspectio=
n
>>> agent which has a policy of what to do with legal-but-dodgy-looking act=
ions.
>> IOW you suggest we remove p2m_access_t parameters from various functions=
,
>> going with just default access?
>=20
> p2m_access_t was very obviously a bodge when introduced, and I doubt it
> would pass today's review standards.
>=20
> It is definitely a mis-design, given its ill-specified and overlapping
> semantics with respect to the p2m type.
>=20
>>  Of course, as pointed out in another
>> reply, we'll need to split p2m_mmio_direct into multiple types then, at
>> the very least to honor the potential r/o restriction of AMD IOMMU unity
>> mapped regions. (FAOD all of this isn't a short term plan anyway, at lea=
st
>> afaic.)
>=20
> I don't think that will be necessary.
>=20
> IVMDs are almost non-existent, and given how many other areas of the AMD
> IOMMU spec are totally unused, I wouldn't be surprised if r/o unity
> mappings were in that category too.=C2=A0 There's no obvious usecase for =
r/o,
> as anything critical enough in the platform to have an IVMD in the first
> place will also be non-trivial enough to require bidirectional
> communication somehow.
>=20
> The unity mapping only says "this device requires read-only access".=C2=
=A0 It
> doesn't say "this must be mapped read-only", and it is legitimate to map
> a r/o unity mapping as r/w.

Well, imo that's extending what "Write permission. 1b=3Dwriteable, 0b=3Dnot
writeable" and "Read permission. 1b=3Dreadable, 0b=3Dnot readable" in the
spec say. "Permission" to me doesn't mean what you say.

Nevertheless I would perhaps not insist (as I've already made clear I
don't see a strong need to support w/o mappings), if ...

> If such a case actually exists, there's clearly one agent in the system
> with r/w access into the r/o range, and mapping it r/w is equivalent to
> the "IOMMU not enabled in the first place" case which is the default
> case for most software for the past decade-and-a-bit.
>=20
> In other words, I don't think the r/o unit maps on their own are a good
> enough reasons to split the type.=C2=A0 If we gain other reasons then fin=
e,
> but this seems like chunk of complexity with no real users.

... there wasn't already a 2nd use for this: The IO-APIC mappings (see
my respective pending patch).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:34:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:34:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180419.327076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUgR-00076e-Ip; Tue, 07 Sep 2021 06:34:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180419.327076; Tue, 07 Sep 2021 06:34:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUgR-00076X-Fs; Tue, 07 Sep 2021 06:34:15 +0000
Received: by outflank-mailman (input) for mailman id 180419;
 Tue, 07 Sep 2021 06:34:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNUgQ-00076R-Ax
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:34:14 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 14c46350-cb4d-48b5-8944-7fa3ee01311a;
 Tue, 07 Sep 2021 06:34:13 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1875gv9o008521; 
 Tue, 7 Sep 2021 06:34:10 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53])
 by mx0b-0039f301.pphosted.com with ESMTP id 3aws4h13bn-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 06:34:10 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM7PR03MB6548.eurprd03.prod.outlook.com (2603:10a6:20b:1c2::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 06:34:05 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 06:34:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14c46350-cb4d-48b5-8944-7fa3ee01311a
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Io9ggiQeyG7jx0BilvoNnv0Y5zon76kEnRVsj4ppbDCnWrfa+Plo9dSAeKFUXss+H4n/da4IfAqam5QJ9AMTfnBqJEivsX8dtTgRuNc8lQkNBpWOy27183ZZeU5sIeonDDx6jEyi+JgYb7wYQDw66yjuu/ia1Ezi3FNnVunE9DaEVbRa9SwT4XfC/0o6eT0G8GAkE3ESE2ZwwbhO7I64drCZ+Vm1oSA5k7pCBeJ6AiANZJF40VkAqnoJSd+cOhZgz47xSVotJN7xFhLe88sWX/BOKwwGTIPMcTN+QKyZW/yu5lZw0bi/xJwzmah5KELdO6nLESu+AKk+ecEjE/fTDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OwCGYDqY51fg4A+y0BsNIv4jmEll0CQOVvaIhZYta9U=;
 b=Lk74jwz1fTs4WBc3ZDudkOkkHKyzaegLlR0dk/7xvuNzaZ7elTi7NJD4rhfHIUNi5tT1SIVVHuxpaTXHuqo7ibULEFB7bxihR5LD51cxGbcF1qH63rE6Qv72Sz2ce4+UXCI32UZNnEqelASe4yVWnpCj0EXXvZC7GN4blafQju8JC0gBC3NzeqTXZB1XMy/AIuwNTgeE0EzVElCoentvXnET0AcI32o6dS/LdiimDrFIGv5Hy2QNrLy2QgS7wPZD2GbtpZ9DWjellhaLZgn0/Sw5lTgVsSFLLuIaFP9hs4ilS+950bNAEL6fedar3ZT6mL2pBu9q9FME+t+9hE07EA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OwCGYDqY51fg4A+y0BsNIv4jmEll0CQOVvaIhZYta9U=;
 b=xOvwgJkHt0cV+OfD9dZVK8KGHoR6cPAbw1gomM/wPGMA0caYlY2lY9iZWrvurwxaTZ3duXdTKY7vwGtlRnXudaCc/yF2UDpjC0O9JVbKGp+616q+j0QzdhdPaEuRIRnNX1bPdZ04LSfEaErvlaDNkmhl6VGz2fA5ZvPaDH0bPLu126fS1bm+w4yFuGvfUt58lU+RA9Kzk/b6ckkGjiMhnm83QdzQS1jQD8IohTomEq3jNabnTNGTaTpLEAB2BvwTwpYiUsqNvcqKGlc+nGdbYg7quKIsE3gpua9OIKazfQDw9IcKAZJdaVxpeAz7btiahU44Nwl7ewYFWI2puAXGcw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Topic: [PATCH 11/11] xen/arm: Process pending vPCI map/unmap operations
Thread-Index: 
 AQHXoJ51zxJWpxhHEEGXNqYYBqazw6uSBESAgASU4wCAAB15AIAABzeAgAALJQCAAANyAIAACPKAgAFOFwA=
Date: Tue, 7 Sep 2021 06:34:05 +0000
Message-ID: <16d93740-1016-19b4-ca58-65a9dc1aadfc@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-12-andr2000@gmail.com>
 <bc3bc53a-4a86-8ef3-b040-6e983f02ebf1@xen.org>
 <f8760f77-1ba7-b0ed-0562-d63527aa1cfa@epam.com>
 <7da7f759-eab0-5e97-e8e4-980db5db1e79@xen.org>
 <b5b761bd-f5d7-34e6-691d-735af7c8b0ae@epam.com>
 <f950a37d-2773-fce5-9e77-163a64925d7d@xen.org>
 <71e0a54f-da69-5712-f826-9c38eecfa244@epam.com>
 <9482457a-fae2-eb8e-79dc-b5d7bcf2fa0a@xen.org>
In-Reply-To: <9482457a-fae2-eb8e-79dc-b5d7bcf2fa0a@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8c69a9c5-d084-4531-9f40-08d971c97d3a
x-ms-traffictypediagnostic: AM7PR03MB6548:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM7PR03MB6548D72544E29C2455DF59E3E7D39@AM7PR03MB6548.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 vI3ioV4HyYVzpt9GBULiZUek3KzInzxGAGp8ytiHZXBetbFhTaXeJkWvzWNae2jSxSQ7n6k2GSDjgGLXZccLFWyyootM7gUai9Qu1XinGptzrcojll8/vMo8msfuFm99bvcZPbIwkmz+/srOJaip5K3oeyje5cUazRjI/5eOARyoDL/hVoI9HIXeSwhIuJ42LdGFoWTJZaPCHKcL528ZrpHG4P6PmPY+UUonqVgNn3470A9Hhg9bncsiJS+V4F6OpE8u8/yC6FzLC3tDrYTxIUFYtYnteaQIyUfIuTPth/yDwMmqgsSEnHcTDSRClX0sY9waPNffA4eKFmJgXxGyfRUUSFAopDZGug6mZ/S0o4n/sSWnAgov+wEQk7qZgMd1lB6WD454F9YWf0xsC1t7wR90wZElSgOcyDFRsjMSs0TgrhpF9m1LI6wlINmWa89Gu8lPyn784NnWmxNWIKO1QUBhqz1jAoq63ElLfRa2vGUVLweixSEycwFVDTK1fRo+Ay/03MKVUucF6CTEfwl0bLRQDnb+fM4+3ey0M+zsm2ARJjjtMaxY5qQZ5J9BCoBaxyGgsOudTTcV/3CVxFcrDpQ/uxXsv9ni1YR2uViwi0S5g7bZgabkg+IVHOEZRXlCP6e0srgcKW7+dK8Kkmu3cKJgEWIRdUS9TlRpoZLJTJiGDxcihQNfSn2nnRNyPavwFq7KqjIoyjGMIT3VI+/ZQ4NjHbI5CMzejrjHxPq2mOXo8Af9/csGcBE5uMSgvBgX
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(136003)(396003)(39860400002)(346002)(376002)(6486002)(478600001)(66946007)(66476007)(6512007)(2906002)(5660300002)(53546011)(6506007)(86362001)(38100700002)(186003)(71200400001)(4326008)(26005)(31696002)(36756003)(31686004)(316002)(2616005)(66556008)(122000001)(54906003)(8676002)(64756008)(8936002)(38070700005)(91956017)(66446008)(76116006)(110136005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UUozUVY5Y3l2QWcwcS9LOUt0YkNYUTRTRFJZNFM0eWRKNFdMSmlmbXArOWls?=
 =?utf-8?B?UkJKLzdBTlFGZEkzVkltc0NGQ3FJYmNYcmNrUUVtUWdrSE0zSTVQOTlYaFEr?=
 =?utf-8?B?ZDhOSUxxWWxUN1Q2c20vdnJTR3JmQmg3WFdtVzAzSFZnUElwYnJSVXZHK3Qz?=
 =?utf-8?B?OTdEaWZRSDh0L1FhUHB0eXVWVUMyaTR1TDZkbER4R1pBendDTzNrRDJEdXpW?=
 =?utf-8?B?NzdwZ2plMW9SS29SSElidXRERjVVMVEwa3hrSk9RVllTTXI2bHNWeWxZMkJL?=
 =?utf-8?B?VStKWVRBSDYzU3BDY3hGUnVLaUk4R0xsN1VnT3BnSmswb2NUL2xySjVqTTJX?=
 =?utf-8?B?b3BYT2pPdjRFNmgrNDZSbDJaK00wOGNCaU0xQlZqeGlFVnNYL2tTV3d2VFBh?=
 =?utf-8?B?TFIvNi8rUStuRGlFbCtRd1Mva1plam0vOHZCVHlaL3E0eXhoSWVXUElGcHA1?=
 =?utf-8?B?WlgwcFdvSkRNdnYzQ1M3K3NQUHBTOU10RlpOUU1OSkNkYStvc2cwY29RU0Qr?=
 =?utf-8?B?YmtZRDlpb1ZXOUhncVZEZFBIY2xEREUycysrVWVBVHV0Zld0Q1ZwcEJGYVk1?=
 =?utf-8?B?QnFyUXhVK1RDS0RpNzhLY2JKa1VpcTNuZ0w5N2wxK1RGeklkS01IbXBablVl?=
 =?utf-8?B?M3FzMXppZW9DREZBcXVXbkNtWmF1NEZEbjZZTzF5MXRHWW5PN1lGeVBXKysy?=
 =?utf-8?B?TzBkc1RlZHlpNHdXY0Z5NzdWeitDSDM2bXRZck5mWmdiOCtpamttaC9nU3ox?=
 =?utf-8?B?SldvYWp3OUpEbVU1emRiVXZLQzZoa2grblRZa1BKRlhtODZEM3l3MUFMMXVh?=
 =?utf-8?B?RUg0V0crblB6aHFoNFRkWW9Pc2hIbzdLUmErcWlvZi9XVWJqTjdWeUxaMkpp?=
 =?utf-8?B?QkRySEc2blZMSHNaU283ejk5NmN4b2l3RitPWW02bHJlNU9hRWJRRWhHQkZS?=
 =?utf-8?B?cVp2MTdiendmdjY1MHd1ZEcrV2FMRkxpT01IcGZxdjNGOUFxSytuMkRWRnBS?=
 =?utf-8?B?NmVvbEw2N3d1VlJMMzBNT0piek9HMURscE5VRWhsOXkxMFN4cS9pU2ZQM2lM?=
 =?utf-8?B?bmNOZmFmZUE1Y0srMjlTcnREM2VqTTh0U3hxZ1FiaUZVOFZIRC90T2Z3dVBJ?=
 =?utf-8?B?S0NIWnQydXpZc0NabWRXMmttK0ZGVWgyTFJFck1sc01rVWFFQkk2ekZkNkVR?=
 =?utf-8?B?Ky82anpncWpFeks0YjRFcnJMS0d3dXR2Ym53SWJjcWl4ZVVIVWQzTVZLNjht?=
 =?utf-8?B?eGlSMi96VnEwRWZaSk90MlZyMkc1eTRaR3ZmcTVIV2FaaW5wRjRYckJQa2NM?=
 =?utf-8?B?RnV1Q1lwbzF0OER4WHBaZEtHVW0xMEc3V29yemRVTHh2bk1HODFBcUxIMjQv?=
 =?utf-8?B?cTRQdUkwM3NpMFNydDEvUXFjZm9rTHYzd0V5dFpNbC82dko3ZTRIUW1LZW5l?=
 =?utf-8?B?MEVqSm92eXp0dXczWTdQN0M5aVF3cGRwY0llRUk1dEJlNmJ1cjZTUGFlbEdm?=
 =?utf-8?B?dDF1MFRuOVlGeXJBeTV4dWJPKzNQOXdlRkRsNlJIQjUzODI4RGorZG1CR2dz?=
 =?utf-8?B?b3RneEYvM0loczJiQXphazJ5eFQ4N0Viekt3Mm1EeWsxeng2TkVMZ3Y0dkJt?=
 =?utf-8?B?Ym5IaytwRkhXVUtKTEVDaEFsMHduOXFETUh3UWNZZWZKQXFYZEJOQUEwaXRR?=
 =?utf-8?B?WjJkeGJPNTFKNy92NzVSYk1uM1RwL0gvTnNiZ1pDU3RSdC8wVTZ2SVF6bzRM?=
 =?utf-8?Q?GgZ/EBXNLx08E7ZS9YAeJluusU+raL4u9vOUBNO?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <AE4A125459D4004594DAE15F5D2D578B@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c69a9c5-d084-4531-9f40-08d971c97d3a
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 06:34:05.1610
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 8a9u+qeUiNQ/PImUze2o2BlWgmIHk8XCbhzj5C/Xds48Hy8mG1YVsjf4sX4BcO0MU5chByJyKIeOPlzsrRBlFv87bpOZP7Au2U4ATq+jnZQIbjMHMutA6nwRKHDDwauh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR03MB6548
X-Proofpoint-GUID: hPkc-jlmIuwvvC_O6sxvj_loaVVUptEu
X-Proofpoint-ORIG-GUID: hPkc-jlmIuwvvC_O6sxvj_loaVVUptEu
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0
 priorityscore=1501 mlxscore=0 malwarescore=0 suspectscore=0 clxscore=1015
 spamscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 impostorscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070043

DQpPbiAwNi4wOS4yMSAxMzozOCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPiBIaSBPbGVrc2FuZHIs
DQo+DQo+IE9uIDA2LzA5LzIwMjEgMTE6MDYsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4gT24gMDYuMDkuMjEgMTI6NTMsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+Pj4+Pj4gSG93
ZXZlciwgbG9va2luZyBhdCB0aGUgcmVzdCBvZiB0aGUgY29kZSwgd2UgYWxyZWFkeSBoYXZlIGEg
Y2hlY2sgZm9yIHZwY2kgaW4gdGhlIGNvbW1vbiBJT1JFUSBjb2RlLg0KPj4+Pj4+DQo+Pj4+Pj4g
V2hpY2ggbWF5IG5vdCBiZSBlbmFibGVkIGFzIGl0IGRlcGVuZHMgb24gQ09ORklHX0lPUkVRX1NF
UlZFUi4NCj4+Pj4+DQo+Pj4+PiBSaWdodC4gTXkgcG9pbnQgaXMgd2hlbiBDT05GSUdfSU9SRVFf
U0VSVkVSIGlzIHNldCB0aGVuIHlvdSB3b3VsZCBlbmQgdXAgdG8gY2FsbCB0d2ljZSB2cGNpX3By
b2Nlc3NfcGVuZGluZygpLiBUaGlzIHdpbGwgaGF2ZSBhbiBpbXBhY3QgaG93IG9uIGxvbmcgeW91
ciB2Q1BVIGlzIGdvaW5nIHRvIHJ1bm5pbmcgYmVjYXVzZSB5b3UgYXJlIGRvdWJsaW5nIHRoZSB3
b3JrLg0KPj4+Pg0KPj4+PiBTbywgeW91IHN1Z2dlc3QgdGhhdCB3ZSBoYXZlIGluIHRoZSBjb21t
b24gSU9SRVEgY29kZSBzb21ldGhpbmcgY2FsbCBsaWtlDQo+Pj4+DQo+Pj4+IGFyY2hfdnBjaV9w
cm9jZXNzX3BlbmRpbmc/IEluIGNhc2Ugb2YgeDg2IGl0IHdpbGwgaGF2ZSB0aGUgY29kZSBjdXJy
ZW50bHkgZm91bmQgaW4gdGhlDQo+Pj4+DQo+Pj4+IGNvbW1vbiBJT1JFUSBzb3VyY2VzIGFuZCBm
b3IgQXJtIGl0IHdpbGwgYmUgbm9wPw0KPj4+DQo+Pj4gTm8gSSBhbSBzdWdnZXN0aW5nIHRvIG1v
dmUgdGhlIGNhbGwgb2YgdGhlIElPUkVRIGNvZGUgdG8gaHZtX2RvX3Jlc3VtZSgpIChvbiB4ODYp
IGFuZCBjaGVja19mb3JfdmNwdV93b3JrKCkgKG9uIEFybSkuDQo+Pg0KPj4gT2ssIEkgY2FuIG1v
dmUgdlBDSSBjb2RlIHRvIGh2bV9kb19yZXN1bWUsIGJ1dCB2UENJIGlzIG9ubHkgdXNlZCBmb3Ig
eDg2IFBWSCBEb20wLg0KPg0KPiBBRkFJSywgUm9nZXIgaXMgcGxhbm5pbmcgdG8gdXNlIGl0IGZv
ciB4ODYgUFZIIGd1ZXN0Lg0KPg0KPj4NCj4+IERvIHlvdSBzdGlsbCB0aGluayBodm1fZG9fcmVz
dW1lIGlzIHRoZSByaWdodCBwbGFjZT8NCj4gSSB0aGluayBzby4gQUZBSUNULCBvbiB4ODYsIHRo
ZSBvbmx5IGNhbGxlciBvZiB2Y3B1X2lvcmVxX2hhbmRsZV9jb21wbGV0aW9uKCkgaXMgaHZtX2Rv
X3Jlc3VtZSgpLiBTbyBpdCBtYWtlcyBzZW5zZSB0byBwdXNoIG9uZSBsYXllciB1cC4NCg0KT2ss
IHNvIEkgZW5kZWQgdXAgd2l0aDoNCg0KLS0tIGEveGVuL2FyY2gvYXJtL3RyYXBzLmMNCisrKyBi
L3hlbi9hcmNoL2FybS90cmFwcy5jDQpAQCAtMjMwNSwxMCArMjMwNSwxNyBAQCBzdGF0aWMgYm9v
bCBjaGVja19mb3JfdmNwdV93b3JrKHZvaWQpDQogwqDCoMKgwqAgfQ0KIMKgI2VuZGlmDQoNCi3C
oMKgwqAgbG9jYWxfaXJxX2VuYWJsZSgpOw0KLcKgwqDCoCBpZiAoIGhhc192cGNpKHYtPmRvbWFp
bikgJiYgdnBjaV9wcm9jZXNzX3BlbmRpbmcodikgKQ0KLcKgwqDCoMKgwqDCoMKgIHJhaXNlX3Nv
ZnRpcnEoU0NIRURVTEVfU09GVElSUSk7DQotwqDCoMKgIGxvY2FsX2lycV9kaXNhYmxlKCk7DQor
wqDCoMKgIGlmICggaGFzX3ZwY2kodi0+ZG9tYWluKSApDQorwqDCoMKgIHsNCivCoMKgwqDCoMKg
wqDCoCBib29sIHBlbmRpbmc7DQorDQorwqDCoMKgwqDCoMKgwqAgbG9jYWxfaXJxX2VuYWJsZSgp
Ow0KK8KgwqDCoMKgwqDCoMKgIHBlbmRpbmcgPSB2cGNpX3Byb2Nlc3NfcGVuZGluZyh2KTsNCivC
oMKgwqDCoMKgwqDCoCBsb2NhbF9pcnFfZGlzYWJsZSgpOw0KKw0KK8KgwqDCoMKgwqDCoMKgIGlm
ICggcGVuZGluZyApDQorwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gdHJ1ZTsNCivCoMKg
wqAgfQ0KVGhpcyBpcyBob3cgaXQgaXMgZG9uZSBmb3IgSU9SRVEuIEl0IHNlZW1zIHRoZXJlIGlz
IG5vIG5lZWQgdG8gcmFpc2Ugc29mdGlycS4NCg0KSSBhbHNvIG1vdmVkIHZQQ0kgZnJvbSBjb21t
b24gY29kZSB0byBodm1fZG9fcmVzdW1lIGZvciB4ODYNCg0KPg0KPiBDaGVlcnMsDQo+DQpUaGFu
a3MsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:52:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:52:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180441.327105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUyI-0002YK-H5; Tue, 07 Sep 2021 06:52:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180441.327105; Tue, 07 Sep 2021 06:52:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNUyI-0002YD-Do; Tue, 07 Sep 2021 06:52:42 +0000
Received: by outflank-mailman (input) for mailman id 180441;
 Tue, 07 Sep 2021 06:52:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qx60=N5=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mNUyH-0002Y7-F7
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:52:41 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 31791194-0fa8-11ec-b0d7-12813bfff9fa;
 Tue, 07 Sep 2021 06:52:40 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B06AF31B;
 Mon,  6 Sep 2021 23:52:39 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5DA4C3F5A1;
 Mon,  6 Sep 2021 23:52:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31791194-0fa8-11ec-b0d7-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [RFC PATCH] xen/design: Add design for EFI dom0less system start
Date: Tue,  7 Sep 2021 07:52:28 +0100
Message-Id: <20210907065228.21794-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1

Add a design describing a proposal to improve the EFI
configuration file, adding keywords to describe domU
guests and allowing to start a dom0less system.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
 docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 docs/designs/efi-arm-dom0less.md

diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
new file mode 100644
index 0000000000..8d8fa2243f
--- /dev/null
+++ b/docs/designs/efi-arm-dom0less.md
@@ -0,0 +1,105 @@
+# Xen EFI configuration file
+
+The current configuration file used by Xen when it is started as an EFI
+application is considering only the dom0 guest and doesn't have any
+property to describe and load in memory domU guests.
+Hence currently it's impossible to start a dom0less system using EFI.
+
+# Objective
+
+This document describes the proposed improvement to the Xen EFI
+configuration file to list properly both the dom0 guest and the domU
+guests as well.
+The final goal is to be able to start a dom0less system using EFI.
+
+# Current Xen EFI configuration file
+
+The current configuration file is described by the documentation page
+https://xenbits.xenproject.org/docs/unstable/misc/efi.html.
+
+Here an example:
+
+```
+[global]
+default=section1
+
+[section1]
+options=console=vga,com1 com1=57600 loglvl=all noreboot
+kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
+ramdisk=initrd-3.0.31-0.4-xen
+xsm=<filename>
+dtb=devtree.dtb
+```
+
+# Proposed improvement
+
+The proposed improvement to the current configuration file is the
+introduction of new keywords to describe additional domUs.
+
+Here follows the proposed new keywords:
+  - domu#_kernel=<kernel file> [domU command line options]
+    - Mandatory kernel file for the domU#
+  - domu#_ramdisk=<ramdisk file>
+    - Optional ramdisk file for the domU#
+  - domu#_dtb=<dtb file>
+    - Optional dtb fragment file for the domU#, it is used for device
+      assignment (passthrough).
+  - domu#_property=cpus=2
+    - Properties that should be added to the dtb in the domU node to
+      properly describe the domU guest. Refer to the documentation:
+      https://xenbits.xenproject.org/docs/unstable/misc/arm/device-tree/booting.txt,
+      section "Creating Multiple Domains directly from Xen".
+
+For all the keywords above, the # is a number that uniquely identifies
+the guest.
+The keywords domu#_kernel, domu#_ramdisk, domu#_dtb are unique, therefore there
+must not be specified the same keyword twice in a section.
+The # number is not enforcing any domid, it is just used to link each property
+to the right guest, so there can be domu1_* guests that are started with domid 2
+and so on.
+
+The domu#_property can appear multiple times and it specifies an additional
+property to be listed in the domU node inside the device tree, Xen will
+not check if the same content is specified multiple times.
+
+There are some property whose name starts with an hash symbol (#address-cells,
+#size-cells), in this case the line will be considered as a comment, so to
+specify them, they have to be listed without the hash symbol, the documentation
+will be updated as well to see the implemented handling of these special
+properties.
+
+# Example of a configuration file describing a dom0less system
+
+The following configuration file is describing a dom0less system starting two
+guests.
+
+```
+[global]
+default=xen
+
+[xen]
+# Xen boot arguments
+options=noreboot console=dtuart dtuart=serial0 bootscrub=0
+# Xen device tree
+dtb=devtree.dtb
+
+# Guest 1
+domu1_property=address-cells=2
+domu1_property=size-cells=2
+domu1_kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
+domu1_property=cpus=1
+domu1_property=memory=0xC0000
+domu1_dtb=domu.dtb
+
+# Guest 2
+domu2_kernel=Image-domu2.bin console=ttyAMA0 root=/dev/ram0 rw
+domu2_property=cpus=2
+domu2_property=memory=0x100000
+domu2_property=vpl011
+```
+
+In this example the Guest 1 is loading Image-domu1.bin as kernel and it is using
+the passed boot arguments, it is also loading a device tree snippet for device
+passthrough.
+The second guest, Guest 2, is using another kernel and it will use the vpl011
+device.
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 06:54:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 06:54:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180450.327117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNV02-0003DZ-2Q; Tue, 07 Sep 2021 06:54:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180450.327117; Tue, 07 Sep 2021 06:54:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNV01-0003DS-Uh; Tue, 07 Sep 2021 06:54:29 +0000
Received: by outflank-mailman (input) for mailman id 180450;
 Tue, 07 Sep 2021 06:54:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNV01-0003DM-1R
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 06:54:29 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2c4fef43-8188-4913-a64a-d6de9c1a02b7;
 Tue, 07 Sep 2021 06:54:27 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2058.outbound.protection.outlook.com [104.47.8.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-Tx5IdKCpMGGYJcqTF6j-Cw-1; Tue, 07 Sep 2021 08:54:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4192.eurprd04.prod.outlook.com (2603:10a6:803:4c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 06:54:24 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 06:54:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P251CA0027.EURP251.PROD.OUTLOOK.COM (2603:10a6:102:b5::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 06:54:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c4fef43-8188-4913-a64a-d6de9c1a02b7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1630997667;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hqt8KpC+/cLPQGezf9kbhcn7tWO956T4Rz/kcs8uU+Y=;
	b=nHEndOrvXG0TETMQaPDvkGsH9YxyEVR7pRXsd5SqzcFMbS9z62K8EDkO+X6ZXpcZw6lcxO
	sWZFFq7diA0ZQM9vyFrNQBj6Mecuks7RhIY7mshwKszUCgupcIUJirG/iLgiyQ8UgSlzVG
	OFXl7P3hh7HrZIn+46ypNsAjLhsCmXg=
X-MC-Unique: Tx5IdKCpMGGYJcqTF6j-Cw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IMppmzkvfoDfCDXioiJMvUs3WB4q73YFpeiuLgVf8zOj/qsaIqm8oiBG7h9ad14PdvM/8bBNpQ8uQwRV1Ax6YHJJFsluiZlVkz8sBQoa2LM/iXdwFDmz1iMewNGsU8aDhkVfYprGs5yb4Ed5V5jToGTguu89KsvCxb3wroZeyVFfVdqqTPFC7Z8S+PZuLRBrObMnM0Y9Mxc25pXAogKzOYuoOmLUxdSlsP3KPZQeBJ76OKSx8TKS63vPoKs66Ub+ottIFRotGqgUY8Nfj8/05FDZqoO4A6OId3nnCPzeBjlF6/a7sP8K4l2CWVp/pkM+WU5GdjeWlTMujCxBw0MM0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hqt8KpC+/cLPQGezf9kbhcn7tWO956T4Rz/kcs8uU+Y=;
 b=SCDoFaNgwzabALppQMsCmPza1E6jAgogXmAGhjBObvhJ1yiNMvhNUSNfs9j3Z7tRJ7NEcuKJfK5VFB+pkefEpNEjvKVIqRGc8S68RquqjGAT+L7Z6CakNf5sY5JVsQCTEJkG5PTpq5qfRsvg0uoRblRkgqZCsdTYzfWCeqpI8dNLqYd2GSkn3h70mEJJUadBWD+fFladbyrZTpcnq+ZzmVHGasdX9ICNJZpV0W/Tp0bFlrD54LAUr+q0q6/fpp1Ek7veacVQM2ISvisAhXehm9ExpeQbRcoUsA/QCkxDMbeMD66DaZQqcK2ls9rms9sbykyZxwQGlXQyIcXoio3ZnQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: Sander Eikelenboom <linux@eikelenboom.it>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <468f4d74-d469-51f9-a0a0-3fcbe24d298c@suse.com>
Date: Tue, 7 Sep 2021 08:54:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P251CA0027.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:102:b5::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a47430a2-5556-4036-3873-08d971cc5316
X-MS-TrafficTypeDiagnostic: VI1PR04MB4192:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4192EE6A52A7F002D2D326ADB3D39@VI1PR04MB4192.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Mt28rYVlfml3dEboFw6TkvNRYSiO8CiJZypLeK2Ii1i0qN2hT2XTEQaRrt9jmOn1wCpb2Tg9UbhqJpTCQd6gc4kuln5VBR1+OeuAB6uFHfJJExcecVt11mJI4j/qPJwoeHBSpGSR+Mv4u9MtfxzzIqCAL3MPrBiiEfTP2vuxP1/ORs9536sTTnJWyai4N7UqvOvWDBpBRCk4iK2R6A5aAd5t4VuWpIXO1iQIq0i8/XgUUjpjco5ZBaRyDiEnjGq6NLr4LFaqKBf2h4XBv59T6/RkB9dW375bRie25y+eY6JX9tpVbR9rZkRm9EsU0SroPvPuMrjVZ1pIfWWZ4MgsiMyhTiHrHGlMy29skl+Z6slzPT0X4kV04VjP+AekPJPkcLjUd5r9gkR6unT9iKpWNFHwylmofDePCh+7QEEkZUQj37EYx5oMSORdFWCqcstioYtvW4uitlj5iBQlXawdx0Kty+ltL1GnYB9YPBVrjctIDv65WJUpEkdHiu3D4WncUhn3HPOEHFnQ+uKt7fB6gsLU6ExB2kly1CplEhX7rmEFdepUQaTsf6cpO8/UvfN+0hYVCUa9TCjueDxhQ2v4yxkx5JIdoWR3PjkQtXKubP3zKpcbqTeKgA7hV70bFzx7ba5CQeGW4/Xlkq8krLrcTFn3JjMQYRCI0g/P7VURawlZA3XqoSDtFcDDYiVwVnuzFaqtjhiRGHtk4gLAL/suXi3qoq0+PzcFaJm3Js0ehbU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(39860400002)(376002)(366004)(396003)(31696002)(956004)(36756003)(8936002)(2616005)(6486002)(2906002)(31686004)(4326008)(83380400001)(45080400002)(8676002)(66946007)(86362001)(478600001)(316002)(5660300002)(16576012)(38100700002)(6916009)(66556008)(53546011)(186003)(26005)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TkVhQ1gzY2NTWjBSUWxKRkY2b0FRK2JySWpVR2hIaGQ5eElKblEydHZ1azgz?=
 =?utf-8?B?WGFkODRtdkxtdjRkU3EvOXFJcFBPQ2toU1JrcmZQcHV0QUl3Z3ZZbDU0cDh3?=
 =?utf-8?B?YUdOakNlRFJTZkx3eHBld0N6VHRkYzE1c2xJY2xHcTVKamprTjdJMnpya0w1?=
 =?utf-8?B?SVliSzZ1NXdnaUVTWWhJeWlzSzBjN2cyTE15dzVPYjdYTnRCamFyUDM2MStB?=
 =?utf-8?B?b3RGd1ZBMnRUNTZwV0dyNVU1WVNTeXBiN1dRNWY3a3RGbUZodVEzb2dWcE5j?=
 =?utf-8?B?VWs4ZGp2Tm42M2RXTnQ0d3l3amJTK3VrSWpldUdNa2tsYzc4WUZPdjNUR0dl?=
 =?utf-8?B?Z3ozaGhPWlVDNG1lWWpFNlRoQXNkNi9FdXptS1phaUwxMmVUMTY4MkZPbkpp?=
 =?utf-8?B?b3FnNy9xOERrb1V5NXZBTWhNd2JLZ0NVTWlhWlI2VUN1SXY2RzlVdHA2bGtP?=
 =?utf-8?B?S1ZDdWg5ZDlwQURzdlh1WUJoRnU2dURsbVpwWnM4NkRnYWhmMmFyYTVWUDFi?=
 =?utf-8?B?TkxlamlYWVZLY01na0UrRjdLMzVISGRTcUtFeE9nYVN2YVRVdzFyaldqYmZr?=
 =?utf-8?B?WGhCYWVWSktsR0lYZDJ2bEZBSU9XOXNWUS9wY2FLOGNRZ1RXeEpGcGIzeThi?=
 =?utf-8?B?d09qUzBndG1zaXJMbHU5bVFwcWVoTys0bGl6WHZFOUUyL2FwTGtFRnJ2WG5D?=
 =?utf-8?B?QUljRG8rem1zZVk4ZjF4UUIzMEVMTjMyb1dZVFBUR2doVk5HVi9JbExJS3lk?=
 =?utf-8?B?YXhSTEU5TStBdG5zd2tNemxjczY5ZW5NbzFrOVNCb3VzZmZVL0VFM3dva21B?=
 =?utf-8?B?TjFDaXUveWpSWGZxRjd6VFRhVWJIeUo1NlV6M2xFSzFYSEZBNE5RMWEzMy81?=
 =?utf-8?B?bGxZR3FoZGI0dzc3NkZpYXdlZnlncTZFa1Frc1VkRnUxem9Cbzd2bFdDci9Z?=
 =?utf-8?B?QnhaSXMrcUhSc3pUam54NDlmV0lPMmorMVN3bHJiUkVOZkdNSGNQd2wvRWVW?=
 =?utf-8?B?OG1aaUJrZ2hoZWhWekJIMXI3TXhEaU9ZSmROdXd6OG1ISDhZeWt5OFE1Qk1t?=
 =?utf-8?B?Qyt3S1o0bk90Y2xiZTBUdlJBT21CaEVoVFJ1Sk8wOUZPcVNpWGlBSnJTb2k3?=
 =?utf-8?B?OVl2bzNQTDI0OVU3NUZNckxCM1pscmk5OHJDbEVtNlJlMno1QTJSaFZOSzZ5?=
 =?utf-8?B?S3YzZXRuTGVETk5GWUJRdE5GWW1XWDQyY1dKSUpJQ3VLaTBaTmM2VDdpbXZ0?=
 =?utf-8?B?U1dEZHlYaGgrWVZiRzJWZEFDNXY3VnM1SzlTa0ViaHZhdFdpdHdUNEtJand6?=
 =?utf-8?B?clhiRHUzYzNvaklhU3ZYZG9KRkI2MzRReWdUZ2hYSGpOR3dnd0JyZFVFVmFI?=
 =?utf-8?B?Q1ozNUVaUHhXMFZkMHl5aHE3V2NaaENrSDdqR1dxb1l6WFdWa0I1YkpXditw?=
 =?utf-8?B?YzE2YmRJWFVLUmxDaVJKWUYyWWJXTWhEdVVqQ3dpQWJSdk5nbkFuamJHZzF6?=
 =?utf-8?B?Y0R3MFVOWno3RXNZUjdPaUN3endscmlCTEZTMGc3UmM3N0FiV3RRd201TjVu?=
 =?utf-8?B?a0d6MkE0dDBZeHRQcDRockYrU3pVL1QvaWFhTXVoK0JDUEdKVGw0WGpGZGtj?=
 =?utf-8?B?Zk9Zd1R4dUI0MmtzdW1TTDVtQmtyWEhjRVczaXUwR2FGY0UrakUvR1crRlY4?=
 =?utf-8?B?endPTWtpaWJwdCtZazNFbTkzYVdjNW5wMHpOSjMvUnZDbDBiOVkxb0FuZE40?=
 =?utf-8?Q?ckx4J0UnpUVHkgIrF/mv4j54pP8iIYTDwxVs+SL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a47430a2-5556-4036-3873-08d971cc5316
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 06:54:23.5775
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kftMAkS9f9pR2u+l8Wa9sjIRe8wwadWXS6YNx7hl3SXS8fx1TMjQSDecHKRtTyTWlkL/mqRTPa+H80WuN/9+Cw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4192

On 06.09.2021 23:35, Sander Eikelenboom wrote:
> L.S.,
> 
> On my AMD box running:
>      xen-unstable changeset: Fri Sep 3 15:10:43 2021 +0200 git:2d4978ead4
>      linux kernel: 5.14.1
> 
> With this setup I'm encountering some issues in dom0, see below.

Any indication as to "since when"? Xen update, kernel update, config
change, ...?

> xl dmesg gives:
> 
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63b936 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a0622 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6b63da already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 638dd9 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 68a7bc already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63c27d already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a04f2 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 690d49 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6959a0 already pinned
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a055e already pinned
> (XEN) [2021-09-06 18:15:04.090] mm.c:3506:d0v0 mfn 639437 already pinned

This hints more towards a kernel issue, but I wouldn't outright exclude
other reasons.

> dmesg gives:
> 
> [34321.304270] ------------[ cut here ]------------
> [34321.304277] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
> [34321.304288] Modules linked in:
> [34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted 5.14.1-20210906-doflr-mac80211debug+ #1
> [34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1 09/13/2010
> [34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
> [34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
> [34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
> [34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807d416398
> [34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefdeadf00d
> [34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff888006160cc0
> [34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12: 0000000000000000
> [34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15: 0000000000000001
> [34321.304320] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlGS:0000000000000000
> [34321.304322] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
> [34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 0000000000000660
> [34321.304326] Call Trace:
> [34321.304331]  xen_alloc_pte+0x294/0x320
> [34321.304334]  move_pgt_entry+0x165/0x4b0
> [34321.304339]  move_page_tables+0x6fa/0x8d0
> [34321.304342]  move_vma.isra.44+0x138/0x500
> [34321.304345]  __x64_sys_mremap+0x296/0x410
> [34321.304348]  do_syscall_64+0x3a/0x80
> [34321.304352]  entry_SYSCALL_64_after_hwframe+0x44/0xae
> [34321.304355] RIP: 0033:0x7f507196301a
> [34321.304358] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
> [34321.304360] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000000000000019
> [34321.304362] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507196301a
> [34321.304363] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506dc56000
> [34321.304364] RBP: 0000000001a00000 R08: 0000000000000010 R09: 0000000000000004
> [34321.304365] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506dc56060
> [34321.304367] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205f950f30
> [34321.304368] ---[ end trace a19885b78fe8f33e ]---
> [34321.304370] 1 of 2 multicall(s) failed: cpu 0
> [34321.304371]   call  2: op=12297829382473034410 arg=[aaaaaaaaaaaaaaaa] result=-22

Just as an observation: This (decimal) output of course isn't very helpful
for investigating issues like this one. But the 0xAA pattern indicates
anyway that what is being logged has been clobbered by the hypervisor.
For the output to be useful, MC_DEBUG needs to be enabled in the sources.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 07:38:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 07:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180477.327136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVg6-0000DK-K0; Tue, 07 Sep 2021 07:37:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180477.327136; Tue, 07 Sep 2021 07:37:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVg6-0000DD-GC; Tue, 07 Sep 2021 07:37:58 +0000
Received: by outflank-mailman (input) for mailman id 180477;
 Tue, 07 Sep 2021 07:37:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eOj3=N5=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mNVg4-0000D7-BP
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 07:37:56 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.82]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 839a3b3c-0fae-11ec-b0d8-12813bfff9fa;
 Tue, 07 Sep 2021 07:37:55 +0000 (UTC)
Received: from AM5PR04CA0015.eurprd04.prod.outlook.com (2603:10a6:206:1::28)
 by AM9PR08MB6787.eurprd08.prod.outlook.com (2603:10a6:20b:2fe::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 07:37:53 +0000
Received: from AM5EUR03FT034.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:1:cafe::46) by AM5PR04CA0015.outlook.office365.com
 (2603:10a6:206:1::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 7 Sep 2021 07:37:53 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT034.mail.protection.outlook.com (10.152.16.81) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 07:37:52 +0000
Received: ("Tessian outbound b48d04bf8afd:v105");
 Tue, 07 Sep 2021 07:37:52 +0000
Received: from cc337d4a303e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5A90B823-E9D3-4480-A8EE-8118E84F0E17.1; 
 Tue, 07 Sep 2021 07:37:45 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cc337d4a303e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 07:37:45 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6687.eurprd08.prod.outlook.com (2603:10a6:102:132::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 07:37:44 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 07:37:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 839a3b3c-0fae-11ec-b0d8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JacijTGwJwH9kAXer4cp3e7Zs95IEF6URsV3LKVTH0g=;
 b=2xP39ivKE620IZmZc0sPTQb/f02xxJjYGgkbTjbDWF1qg8+jpxXd8Iyz73hCtClrmZvGxwGcTw4s/0IefEB+YG8PkoUx2P1TZp1NXCHavl1N4D6g1KMcCxXP9uDG6FCTVTrzrgYCiKOU50K90Bj1tY4heGCOWDlKx+v+W46rL8w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 897d52c3b5177b73
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dh6O08MnqxBafjwLpw3NuWMTa57YVQoKhy1ii6sWKhJRt2jvTEu5dfnftxx1/pgR+O/OMaILp6LnGzB4iuYCqulPGqYqzVqYcdKzR/0uOJoClrp6Ci/L183wDjVHgtBgergHu67ioJdMTM+6UdMh+AjUF8B584eKcBNSdTDvfwdhzV/8FOOhoSWtzvd+tXbN+Sp85H74L1taU92dTVjc8LLv3vZmQ9Ou2wgqmKApXStpw+cWk9YeoBBxOhA+5Ki7Hu4pr8VALfDQmHo55vDf2VyPcmQDjmsbESVuToE3MeJJJoGMW06/9OBtO5YVZONDZk3Q2Ir352PYZh87LJR7MQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JacijTGwJwH9kAXer4cp3e7Zs95IEF6URsV3LKVTH0g=;
 b=RuvOj5Clm/Dl0pYzr4N0muh6zTunpCQomv6ax0wGyv+carmfhFa8u48dPx6QeSca4wUr3w1H86qsp74e9tG+fyBBo9sW89DJdnCuatqTh8Nf/pKjYMP0+XoNE0PS/Rjgu3hXNhB7sFuy0mXeJhTkezrbB+Y5f3ALA0xtJbtXMQP3gnjsIlmAypDDbR4elfqG9ho1Y0K7I34PYu64JtyKRk4BrrgN1PadY2Oh+8hQP9iaAtewz3F2T3HcP4SZnPwSVQ5rGaK63pg6swF7afrZEP1iB7fbLIX0AEm8OKaEr78Yytl80LU906jB8oo/Jaw46DoX5PU5jFRwekdi5Sofgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JacijTGwJwH9kAXer4cp3e7Zs95IEF6URsV3LKVTH0g=;
 b=2xP39ivKE620IZmZc0sPTQb/f02xxJjYGgkbTjbDWF1qg8+jpxXd8Iyz73hCtClrmZvGxwGcTw4s/0IefEB+YG8PkoUx2P1TZp1NXCHavl1N4D6g1KMcCxXP9uDG6FCTVTrzrgYCiKOU50K90Bj1tY4heGCOWDlKx+v+W46rL8w=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if needed
Thread-Topic: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if
 needed
Thread-Index:
 AQHXmbPyaqNt71/0B0WMx26sIln5R6uHdm2AgAYrXACAABkoAIAAGG+AgAUlJICAA8aUAIAAmQIAgADq+oA=
Date: Tue, 7 Sep 2021 07:37:44 +0000
Message-ID: <F675C327-DA08-478D-ADA8-51D9DA63DE6D@arm.com>
References: <cover.1629897306.git.bertrand.marquis@arm.com>
 <a7482653a0c29d68854846f25553f4938a6279b9.1629897306.git.bertrand.marquis@arm.com>
 <14b1639e-c9e4-f8fb-e921-73d7f28681ac@xen.org>
 <98E8FB9D-3372-47D2-92F7-8AD401AC1092@arm.com>
 <6705127a-bde2-36cd-0fed-900eb1eace03@xen.org>
 <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com>
 <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s>
 <D0CCB87F-1CC1-450B-A612-E4B505B36B5B@arm.com>
 <7fc6e6b6-464a-a305-2693-42cba40ad869@xen.org>
In-Reply-To: <7fc6e6b6-464a-a305-2693-42cba40ad869@xen.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 5527bb9c-947b-401a-ad77-08d971d2668e
x-ms-traffictypediagnostic: PAXPR08MB6687:|AM9PR08MB6787:
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB67876BC7A8EFF7EAD93A0DCB9DD39@AM9PR08MB6787.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ibyjGh0gjRbk+LfTaFbfKuH+LE8AkIHTI5lPwvoNVKNO/8+0cfwq5eSuKym7J5IjzxmXTw1HBQD0CDGi83AE0nQjFPV20hnp1YJdAhqhxlmWddCoPyYUC2npo+eT2sjWPFzp+2Hrf6bPyTepUIc8ZC0N4bjPXa5U3wMZSf7YwRWeFfFJ4oTVpmiBpB3VQWAIiw+YroljIamzyOur5Bpmh2ctS/zukOi7tB6yj5ST0T8BQisWtfTGBloyRdLoRfdyyDzxQus/70XOuG+yTAtaS68vGE5dLuBqZ/f+l3IS9Hqz7l0RKqUazEykNUo/oNOeYVUP0zuM4M0xcTbgPLouBv84FIfSLH77Xva8HFpWCzykhTwiTeM8DW1qNBC7vHlI9NTi9g1noCYDN2WrQ6ocljbE0dd54j0EWmWUgflB5gqRudV+pZYaqAtUKSIG/AJ4inBimgAl3g6GLwc9JvpevUaGg9IJ8ZjHD5bdCcYTEYSml5ifhDLhLC8AsMJkpUsJ6UM3YOmrYrBvdwOxhp4DuHqhSLzuihMmsGqkcfnr3LT1H75kSUYvJ0eSqhurbNJ/Db8Yh8IAyhFz18L4ttgyeANc7+xOYq1LHmyybZxEiVbg9I91EZymLyiMrok5UdTOda8oFAsmZbfa7GDQhcFooDvFIlYvwZ+I461IpNKOEpB+LX9pVqO3GoepM1Aq/ejWRYQZ6GbCR5xo4BogrhH4HrL3xULqNatId2ZxSMAvTPaA7QKPPG3UeO6wzznxIn1S
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(396003)(136003)(366004)(346002)(64756008)(66476007)(66556008)(53546011)(66446008)(6506007)(38100700002)(6486002)(26005)(36756003)(6512007)(71200400001)(186003)(66946007)(8676002)(2906002)(5660300002)(2616005)(4326008)(478600001)(54906003)(76116006)(8936002)(91956017)(6916009)(86362001)(38070700005)(122000001)(83380400001)(316002)(33656002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <D63EEFAFA8AA294CBD7C4A4B2359E064@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6687
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d15b2021-f01d-48f0-4952-08d971d26191
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	O9ArztvDSpNstsErMxyHEuITQsjYKjpOMuyX3rtMCERTmYu3RZmcgb/PXDzlzD/nXq/R0zfTD1LrPWZWPO0K1CLcoPOzJWIYKzBh/S0a/Jo56wCuABlJTcM8XQMj2gBOSGg0mP6npbeZYcUg6sOFvlCHo6psRVgPTeRQYQoSYmJeltRH4zZJuqE6KBn2Q+v/92s3+jW8/JwFd3vJgkBG1HO8F8AZfMN4kIcnmKl3BRM+MlvnXBQ4WErEJte4NbR4rqpUahsg3OCp/A+sm4hi8K9ouLUKuuXMovCiN2OPISF8A0ww8+5W+Oun29VvcpzsCTDJwqs4zikavkNk/K2WIRQ/Ivgv+ucHtamRruGhHGkGaVLG7JU21PlsH5/Bj2jvor8waGCszkWGho0Cz9JBvDZW+48CUnEHjY/GLb6vnBjuKP3ho14hfdQpSKhXa/sq6a9qOKSutICgHsuYm2RJwMmIHEC5msdrT0nVu7qNB+kUBjqoLu4Y1eWdbS074tvnZgBOMxPs02tKJHOmLqMZyBy62HQyZVcU3RndU8hgJhImRskLAMjbDEBtZVurolwsLLSFDilaYpNz35UKwLnykW9R3yYrls23SFE9j/RfFaNImYSwtsD01neTeVGxlKnagfN72lBMSVYHkuCzQMNxKfdzXBO3hRUdPWjoanFemYmQJP0khCqpvw6potWVQ/PEdHjMnD/fJsV3L4cghpzITg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(376002)(396003)(136003)(46966006)(36840700001)(54906003)(107886003)(33656002)(70586007)(36860700001)(4326008)(6486002)(70206006)(2616005)(36756003)(2906002)(53546011)(336012)(6506007)(82310400003)(6862004)(186003)(316002)(5660300002)(26005)(8936002)(8676002)(47076005)(6512007)(81166007)(478600001)(86362001)(83380400001)(82740400003)(356005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 07:37:52.6553
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5527bb9c-947b-401a-ad77-08d971d2668e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6787

SGkgSnVsaWVuLA0KDQo+IE9uIDYgU2VwIDIwMjEsIGF0IDE4OjM2LCBKdWxpZW4gR3JhbGwgPGp1
bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4gDQo+IEhpIEJlcnRyYW5kLA0KPiANCj4gT24gMDYvMDkv
MjAyMSAwOToyOSwgQmVydHJhbmQgTWFycXVpcyB3cm90ZToNCj4+PiBPbiAzIFNlcCAyMDIxLCBh
dCAyMzo0OSwgU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90
ZToNCj4+PiANCj4+PiBPbiBUdWUsIDMxIEF1ZyAyMDIxLCBCZXJ0cmFuZCBNYXJxdWlzIHdyb3Rl
Og0KPj4+PiBIaSBKdWxpZW4sDQo+Pj4+IA0KPj4+Pj4gT24gMzEgQXVnIDIwMjEsIGF0IDE1OjQ3
LCBKdWxpZW4gR3JhbGwgPGp1bGllbkB4ZW4ub3JnPiB3cm90ZToNCj4+Pj4+IA0KPj4+Pj4gDQo+
Pj4+PiANCj4+Pj4+IE9uIDMxLzA4LzIwMjEgMTQ6MTcsIEJlcnRyYW5kIE1hcnF1aXMgd3JvdGU6
DQo+Pj4+Pj4gSGkgSnVsaWVuLA0KPj4+Pj4gDQo+Pj4+PiBIaSBCZXJ0cmFuZCwNCj4+Pj4+IA0K
Pj4+Pj4+PiBPbiAyNyBBdWcgMjAyMSwgYXQgMTY6MDUsIEp1bGllbiBHcmFsbCA8anVsaWVuQHhl
bi5vcmc+IHdyb3RlOg0KPj4+Pj4+PiANCj4+Pj4+Pj4gSGkgQmVydHJhbmQsDQo+Pj4+Pj4+IA0K
Pj4+Pj4+PiBPbiAyNS8wOC8yMDIxIDE0OjE4LCBCZXJ0cmFuZCBNYXJxdWlzIHdyb3RlOg0KPj4+
Pj4+Pj4gU2FuaXRpemUgQ1RSX0VMMCB2YWx1ZSBiZXR3ZWVuIGNvcmVzLg0KPj4+Pj4+Pj4gSW4g
bW9zdCBjYXNlcyBkaWZmZXJlbnQgdmFsdWVzIHdpbGwgdGFpbnQgWGVuIGJ1dCBpZiBkaWZmZXJl
bnQNCj4+Pj4+Pj4+IGktY2FjaGUgcG9saWNpZXMgYXJlIGZvdW5kLCB3ZSBjaG9vc2UgdGhlIG9u
ZSB3aGljaCB3aWxsIGJlIGNvbXBhdGlibGUNCj4+Pj4+Pj4+IGJldHdlZW4gYWxsIGNvcmVzIGlu
IHRlcm1zIG9mIGludmFsaWRhdGlvbi9kYXRhIGNhY2hlIGZsdXNoaW5nIHN0cmF0ZWd5Lg0KPj4+
Pj4+PiANCj4+Pj4+Pj4gSSB1bmRlcnN0YW5kIHRoYXQgYWxsIHRoZSBDUFVzIGluIFhlbiBuZWVk
cyB0byBhZ3JlZSBvbiB0aGUgY2FjaGUgZmx1c2ggc3RyYXRlZ3kuIEhvd2V2ZXIuLi4NCj4+Pj4+
Pj4gDQo+Pj4+Pj4+PiBJbiB0aGlzIGNhc2Ugd2UgbmVlZCB0byBhY3RpdmF0ZSB0aGUgVElEMiBi
aXQgaW4gSENSIHRvIGVtdWxhdGUgdGhlDQo+Pj4+Pj4+PiBUQ1JfRUwwIHJlZ2lzdGVyIGZvciBn
dWVzdHMuIFRoaXMgcGF0Y2ggaXMgbm90IGFjdGl2YXRpbmcgVElEMiBiaXQgYWxsDQo+Pj4+Pj4+
PiB0aGUgdGltZSB0byBsaW1pdCB0aGUgb3ZlcmhlYWQgd2hlbiBwb3NzaWJsZS4NCj4+Pj4+Pj4g
DQo+Pj4+Pj4+IGFzIHdlIGRpc2N1c3NlZCBpbiBhbiBlYXJsaWVyIHZlcnNpb24sIGEgdkNQVSBp
cyB1bmxpa2VseSAoYXQgbGVhc3QgaW4gc2hvcnQvbWVkaXVtKSB0byBiZSBhYmxlIG1vdmUgYWNy
b3NzIHBDUFUgb2YgZGlmZmVyZW50IHR5cGUuIFNvIHRoZSB2Q1BVIHdvdWxkIGJlIHBpbm5lZCB0
byBhIHNldCBvZiBwQ1BVcy4gSU9XLCB0aGUgZ3Vlc3Qgd291bGQgaGF2ZSB0byBiZSBiaWcuTElU
VExFIGF3YXJlIGFuZCB0aGVyZWZvcmUgd291bGQgYmUgYWJsZSB0byBkbyBpdHMgb3duIHN0cmF0
ZWd5IGRlY2lzaW9uLg0KPj4+Pj4+PiANCj4+Pj4+Pj4gU28gSSB0aGluayB3ZSBzaG91bGQgYmUg
YWJsZSB0byBnZXQgYXdheSBmcm9tIHRyYXBwaW5ncyB0aGUgcmVnaXN0ZXJzLg0KPj4+Pj4+IEkg
ZG8gYWdyZWUgdGhhdCB3ZSBzaG91bGQgYmUgYWJsZSB0byBnZXQgYXdheSBmcm9tIHRoYXQgaW4g
dGhlIGxvbmcgdGVybSBvbmNlDQo+Pj4+Pj4gd2UgaGF2ZSBjcHVwb29scyBwcm9wZXJseSBzZXQg
YnV0IHJpZ2h0IG5vdyB0aGlzIGlzIHRoZSBvbmx5IHdheSB0byBoYXZlDQo+Pj4+Pj4gc29tZXRo
aW5nIHVzZWFibGUgKEkgd2lsbCBub3Qgc2F5IHJpZ2h0KS4NCj4+Pj4+PiBJIHdpbGwgd29yayBv
biBmaW5kaW5nIGEgd2F5IHRvIHNldHVwIHByb3Blcmx5IGNwdXBvb2xzIChvciBzb21ldGhpbmcg
ZWxzZSBhcw0KPj4+Pj4+IHdlIGRpc2N1c3NlZCBlYXJsaWVyKSBidXQgaW4gdGhlIHNob3J0IHRl
cm0gSSB0aGluayB0aGlzIGlzIHRoZSBiZXN0IHdlIGNhbiBkby4NCj4+Pj4+IA0KPj4+Pj4gTXkg
Y29uY2VybiBpcyB5b3UgYXJlIG1ha2luZyBsb29rIGxpa2UgWGVuIHdpbGwgYmUgYWJsZSB0byBk
ZWFsIG5pY2VseSB3aXRoIGJpZy5MSVRUTEUgd2hlbiBpbiBmYWN0IHRoZXJlIGFyZSBhIGxvdCBt
b3JlIHBvdGVudGlhbCBpc3N1ZSBieSBhbGxvdyBhIHZDUFUgbW92aW5nIGFjY3Jvc3MgcENQVSBv
ZiBkaWZmZXJlbnQgdHlwZSAodGhlIGVycmF0YSBpcyBvbmUgZXhhbXBsZSkuDQo+Pj4+IA0KPj4+
PiBJIGFncmVlIGFuZCB0aGlzIGlzIHdoeSBYZW4gaXMgdGFpbnRlZC4NCj4+Pj4gDQo+Pj4+PiAN
Cj4+Pj4+PiBBbiBvdGhlciBzb2x1dGlvbiB3b3VsZCBiZSB0byBkaXNjYXJkIHRoaXMgcGF0Y2gg
ZnJvbSB0aGUgc2VyaWUgZm9yIG5vdyB1bnRpbA0KPj4+Pj4+IEkgaGF2ZSB3b3JrZWQgYSBwcm9w
ZXIgc29sdXRpb24gZm9yIHRoaXMgY2FzZS4NCj4+Pj4+PiBTaG91bGQgd2UgZGlzY2FyZCBvciBt
ZXJnZSBvciBkbyB5b3UgaGF2ZSBhbiBvdGhlciBpZGVhID8NCj4+Pj4+IFBsZWFzZSBjb3JyZWN0
IG1lIGlmIEkgYW0gd3JvbmcsIGF0IHRoZSBtb21lbnQsIGl0IGRvZXNuJ3QgbG9vayBsaWtlIHRo
aXMgcGF0Y2ggd2lsbCBiZSBwYXJ0IG9mIHRoZSBsb25nZXIgcGxhbi4gSWYgc28sIHRoZW4gSSB0
aGluayBpdCBzaG91bGQgYmUgcGFya2VkIGZvciBub3cuDQo+Pj4+IA0KPj4+PiBOb3Qgc3VyZSBp
dCBkZXBlbmRzIG9uIHdoYXQgdGhlIGZpbmFsIHNvbHV0aW9uIHdvdWxkIGJlIGJ1dCB0aGlzIGlz
IGhpZ2hseSBwb3NzaWJsZSBJIGFncmVlLg0KPj4+PiANCj4+Pj4+IA0KPj4+Pj4gVGhpcyB3b3Vs
ZCBhbHNvIGhhdmUgdGhlIGFkdmFudGFnZSB0byBhdm9pZCBzcGVuZGluZyB0b28gbXVjaCB0aW1l
IG9uIHJlc29sdmluZyB0aGUgZW11bGF0aW9uIGlzc3VlIEkgbWVudGlvbmVkIGluIG15IHByZXZp
b3VzIGFuc3dlci4NCj4+Pj4+IA0KPj4+Pj4gTm8gbmVlZCB0byByZXNlbmQgYSBuZXcgdmVyc2lv
biBvZiB0aGlzIHNlcmllcyB5ZXQuIFlvdSBjYW4gd2FpdCB1bnRpbCB0aGUgcmVzdCBvZiB0aGUg
c2VyaWVzIGdldCBtb3JlIGZlZWRiYWNrLg0KPj4+PiANCj4+Pj4gT2ssIEkgd2lsbCB3YWl0IGZv
ciBmZWVkYmFjayBhbmQgbmV4dCBzZXJpZSB3aWxsIG5vdCBpbmNsdWRlIHRoaXMgcGF0Y2ggYW55
bW9yZS4NCj4+PiANCj4+PiBXb3VsZCBpdCBiZSB3b3J0aCBrZWVwaW5nIGp1c3QgdGhlIHBhcnQg
dGhhdCBzYW5pdGl6ZXMgY3RyLCB3aXRob3V0IGFueQ0KPj4+IG9mIHRoZSBlbXVsYXRpb24gc3R1
ZmY/IFRoYXQgd2F5IHdlIGNvdWxkIHN0aWxsIGRldGVjdCBjYXNlcyB3aGVyZSB0aGVyZQ0KPj4+
IGlzIGEgbWlzbWF0Y2ggYmV0d2VlbiBDUFVzLCBwcmludCBhIHVzZWZ1bCBtZXNzYWdlIGFuZCB0
YWludCBYZW4uDQo+PiBUaGF04oCZcyBhIGdvb2QgaWRlYSwgaXQgbWVhbnMgcmVtb3ZpbmcgdGhl
IGVtdWxhdGlvbiBwYXJ0IGFuZCBqdXN0IGtlZXAgdGhlIHNhbml0aXphdGlvbi4NCj4+IEBKdWxp
ZW46IHdvdWxkIHlvdSBiZSBvayB3aXRoIHRoYXQgPw0KPiANCj4gSSBhY3R1YWxseSB0aG91Z2h0
IGFib3V0IHN1Z2dlc3RpbmcgaXQgYmVmb3JlIFN0ZWZhbm8gZGlkIGl0LiBTbyBJIGFtIE9LIHdp
dGggdGhhdC4NCj4gDQo+PiBTaG91bGQgSSBzZW5kIGEgdjQgb3Igc2hvdWxkIHdlIHVzZSBTdGVm
YW5v4oCZcyBwYXRjaCBkaXJlY3RseSBpbnN0ZWFkID8NCj4gDQo+IEkgd291bGQgc3VnZ2VzdCB0
byBzZW5kIGEgdjQuIFRoaXMgbmVlZHMgYSBzaWduZWQtb2ZmLWJ5IGZyb20gU3RlZmFubyBhbmQg
YSBuZXcgY29tbWl0IG1lc3NhZ2UuDQoNCk9rIEkgd2lsbCBkbyB0aGF0IGJlZ2lubmluZyBvZiBu
ZXh0IHdlZWsuDQoNCkNoZWVycw0KQmVydHJhbmQNCg0KPiANCj4gQ2hlZXJzLA0KPiANCj4gLS0g
DQo+IEp1bGllbiBHcmFsbA0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 07:43:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 07:43:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180485.327147 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVlW-0001x3-Ba; Tue, 07 Sep 2021 07:43:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180485.327147; Tue, 07 Sep 2021 07:43:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVlW-0001ww-8H; Tue, 07 Sep 2021 07:43:34 +0000
Received: by outflank-mailman (input) for mailman id 180485;
 Tue, 07 Sep 2021 07:43:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNVlU-0001wq-R1
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 07:43:33 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4b64b5a2-0faf-11ec-b0d9-12813bfff9fa;
 Tue, 07 Sep 2021 07:43:30 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1877hF5Q015024; 
 Tue, 7 Sep 2021 07:43:26 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 by mx0b-0039f301.pphosted.com with ESMTP id 3awxwj0rcr-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 07:43:26 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB7253.eurprd03.prod.outlook.com (2603:10a6:20b:2e8::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 07:43:24 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 07:43:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b64b5a2-0faf-11ec-b0d9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HuUhm10ZF3UHBEcJP/PqxNGPc81E9ery6yQihapkOwESeMehOhQdYDMMaK3PpUBnMnFGwPD38p9cgKIDnjlblVr79iu8cSmBDvPfcwQWQZbySbW2D/t4NvrDh9UUfA/ojk2Mz35CTGSQ/qIT7GMztwO8alvq8a0miZKdlbtuhBF62RLXUK/M02xj6bUaD4rq+R67xbhNdELJU9EHDXtqNhAj6yGHvcn6p7SbrcdveHM3NjUiOkw192bpeSOylYsbjHNv4G8FheLOwsWJs4JKcKuPiaY1aAxu96O/ExZ4i1nmcsuBsWNJ8iwwB3Emevy8J1GEc+d+1XkNTUXkHHnnbw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=H05GsjdJnDqyBr0UhP6j4Tyahjtcs2/I/LUt2RtCWRM=;
 b=UDiDSFZJ2FzFCuCb+IdodVoJGV7fxWUsHvOiJ3ULHlmJKTPdNCaG6IYfs7ASb2zufAfxMqerGFWTVIanLiBaSWP1n44c28m/KjDvRNdaZIqRsjZPUBTQxT7e0+dK8livQOBs29d1vHi62WzIF6hFpU0Qe3qYyTACB0MGT38xyVWJcERFlbzqZqQDL/Av9w67jaFtRyX1rPleknjqW5TBhQ64R5FmMEhsWUmHCdcGCM34qeEbSIdUN8QqjqGP7zD0UmhplSCMK2lZubHbIBYDSwUo8TmqlIgNmt5Ip6KSEeXKmZbUetvXtiiKQq+jCEoe+Q+0fujRwTv4GhwEruB2xA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=H05GsjdJnDqyBr0UhP6j4Tyahjtcs2/I/LUt2RtCWRM=;
 b=EdMKpoIjkmB7owIiFFYVf3i+z0xjp4CtJ5cxcOWHrBQJcJJe2+KoA9mgqhHL1nvKb3AnKsjEWhVSEO6ydChFirbSvHILkwMsDzroijKPD1mNfWafSwEKbqSD6cMCt+5RR7klpCxMUxxRRPaBTEEIzf6+vPGDRflVuPnBM/kqmfLO2cRQM25WOy5M41oT4dL0YM9KZg/TgXW1oREVkbiBf5f6uSWOS2UkaShj+Hu2BJutjXE+c+ATLfbxR2/6JHHC5IfuDh2uD+/KVwUXkIRwJGjtM/7IxBNAi8Uo4Xq8Z25+RdGJ+L0X3wrS69lXYsjffP+lh+9LSbcDXF1ShEHtTg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4A=
Date: Tue, 7 Sep 2021 07:43:24 +0000
Message-ID: <5366265c-d169-041d-f812-1c49260f6499@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
In-Reply-To: <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 855fbb1c-f7c5-4b94-e44a-08d971d32c20
x-ms-traffictypediagnostic: AS8PR03MB7253:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB72531729002997BC33CF3C94E7D39@AS8PR03MB7253.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 0gSKp3SZyqf4i9b9Ftv/HqXkJf/TT6ZrvgOFkhmR9Ok/p4yVDT+j5fhUfUds/RZfS4n1FBtRaEuTTV4/eAs2X9gbCs1reNIFvbKZGZBcTLRJ5BlxSo7aT1+L5KPo/zcEseLbSmB+BnGWaI6AnuxxdTi7vLR64XhalPBqWdBUg2hhQIKKoJ5j3Km3jykXSreyL5EKeT6g4eObrrcKjRZ0vVvjLzooeV5egrw4jt40YvghjuCqT3XKok0O96aM3TrJvKJDd5praZadegWclnbU3aDt/G52rVtBrpHyL7gryJFBR/GtJSAn1e0sfev+UFbIy0RG1PPq0qy98LvBAGbzDP2YkmLlvprhooZCaplVFStzpWiBCFMepg/RPF+XqdwOZuktcERUi34Uv/ASEWDL4NoxeXm+/My/TrjobXL13/w9ZiUUp1CRGtBlJEFBSFzfqOvsckR4gUvFWTFjXhmxFPNQrCZnsbe0103jSPnvKYxFbp2E8SMGDPwtR/u3m2a4eik8VPDgDwh/z+1xSNcQAGBVDyNwAUABV7lGxq/UFzMRLYWqJEvexRkTzLlWuT85bFFhRRPiZEZhpDksZQAegdzKLwTLYaechlQK6AuYEUkNI1kAWaqDosMGsO6skJaf2R1WfSX9bsjiBZ9heDv6rgTUkp+JwB7S3dv1sdhyK6qkgCz4rFrAdlQkx8rp+1VyUwtZBLhPMjFsAeNFoEa/cdd7uceYqICnocP4py9FfZgS5HWIy0kn9AeKl7Q1LvV4
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(39860400002)(366004)(396003)(346002)(31686004)(38100700002)(122000001)(31696002)(316002)(4326008)(26005)(71200400001)(86362001)(66446008)(66476007)(66946007)(66556008)(64756008)(5660300002)(8676002)(186003)(8936002)(38070700005)(91956017)(83380400001)(76116006)(36756003)(54906003)(110136005)(6506007)(53546011)(2906002)(6512007)(6486002)(2616005)(478600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RlVXRmorMW9UMzQ5SDh5SGtNVGQxc2huY2tabk1NSTB2aXJEZkdSbXR3NjJk?=
 =?utf-8?B?M0c3QXloSW93Q3ZVVGhBRlc3cFNXS25rczQwbHEvNGxnenFhME03TmIycUVE?=
 =?utf-8?B?bHJNSkNScXNKcm56SnJ1REpsRHpXWkNNNDJEcHJYdHVpdjgzNEoweEhiQ2pF?=
 =?utf-8?B?K2t5Y0ZVN1FNWlR0M1ZvWXNxa0VCUFg1ZFQ4V1hPbUQ2eEdpZmYrNmJZc2Fo?=
 =?utf-8?B?bFRtSFozRzJ5K3g3eEVTTUgwNHU3NU1sYTNlMHBhWFF1aXprbU5VRko3R2ZV?=
 =?utf-8?B?cGRDTDk3RGdmVUszb3pzZGEvS3NrNm1vNnlkdldKR0JGQjRQeEhRc25TVWpx?=
 =?utf-8?B?cGVqWGR1NklmUzVUUXdMeGNBSFRoZXRJRGpXUjhKUkNWbkQyeGN5RUxTblV1?=
 =?utf-8?B?bnVTUXY2dUkxaERycTBwaGZJYXp4SEVOVUlHZy9wWk0xOEZxditrSE5aS2d6?=
 =?utf-8?B?RVRxS1FzK1UxWXkxSFhiaE5aWVNURGZFbDRIb09DQmVGMHJsZno4Zmw2aVR2?=
 =?utf-8?B?dWZZNVlXTnBkUE5qNnRNcEFPWHQ2R3YzclZWT0EreXhnZmtpWUd6c04yL085?=
 =?utf-8?B?bzM4RUlDcm1lZE9DMmlIWGJFOGdoaUhXWWMxNFE4aWxJQnArRXJYOWk3b05M?=
 =?utf-8?B?Y2lRUTNlYkJacmFqRE43bXJoaXIxSEphcE9BeU1iK01IRnlsdThNaTBIeXdV?=
 =?utf-8?B?aGZpSHlRVnNFYktlV3E4ZERFSFZpeWFDOWN1eE5ESHhnZkhFVkVyTk1ZVk5o?=
 =?utf-8?B?SDZxMGFPeFE1a0JMVFZRRjZCNkRlNkxzWkU4Q3ZvdUcvYlRhQU9FMlZZWkZo?=
 =?utf-8?B?ZHptMTFuRHRnK1YwMkVVQlYvNTZIWjdzMk1nR1pKQUVydFgrTFh1REFXMkVn?=
 =?utf-8?B?VGx3SWxsQUtTWTlRNlJIdDNZdVIvVEVGTjBtNHhqUEN0Z3Y3ZUxzRVI0eFlo?=
 =?utf-8?B?VlFNKzRaUTZlL0crQWZLME5iQStDdEd5M1EvU2NZMHlvTHQwZkNXSE5odWor?=
 =?utf-8?B?dExmM09GZUVTY0ZWcTMzc2tyN2JoSm1jbzI2em54alJOVzRtZmNIRFlLZitD?=
 =?utf-8?B?UDVYMlJqVU9NU2lRc0FDMWU5ZkNqVDhaQjRSMmlkNkluZHliL29nZU10WGth?=
 =?utf-8?B?bDZBQzZieGhERmpTeUtBUUJuT2EyOW9sMWVNeStWTDR3cFNQeGVYTE1JaWw3?=
 =?utf-8?B?d21NSmdIbS9wbFU1TGcvWS9kdCt3S1FZSk91c0lkbzk3TFNvU1lVOTM5bFZ4?=
 =?utf-8?B?NXBXODQ2RWh1Z3ZhMWtFOWQ1K3dNM2JFUkJMMk9GcFRteWs4WERlNW1lc25C?=
 =?utf-8?B?TVlRQlJSdjVMSmhHWVl6K05UeUc4d1BIbnB4eGVodDdJU0VJQkViY1ZFUHNL?=
 =?utf-8?B?MGxFZnQvZ3dMcjZ6V2NYT1VCdHNpOXdaMjZ3ODlueUcvalpHUElwWlUwSEJ6?=
 =?utf-8?B?NStOcGltQ1pKTzcwNjNVTEFNdWc2RDhVaU5TWEJycm9rTWE0UFM2TXhQRFZM?=
 =?utf-8?B?eGt3S3U0RzZVb2QrV0JsazRXWG8zeitBa1JjdHAzK1NUVGxqZzZMcFYvNmZy?=
 =?utf-8?B?MUJpaTdxcDNTd016TFhSb212eDNnY2h4QVNnK2xDNnNRTUdtMEEzV3NOcTFk?=
 =?utf-8?B?ck91R25xM1NPWDJVM0VrS1VpYUJ4SnEyR2NMZTVhRjBvREx2Zi9IOTczdFRX?=
 =?utf-8?B?Y09PWDFUOEphODROTEYyYVhhaDBUR3g0bVpibWlrQkdjOTlsOXhSK1Bic3Nu?=
 =?utf-8?Q?lWLyuD2YC5/FHXevnxaItcQsJm6Ik1g5iT+nZ4M?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A930BBE9B4863B47A2FFB397577D1C62@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 855fbb1c-f7c5-4b94-e44a-08d971d32c20
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 07:43:24.1350
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kP4aR03bofpX5KhR0/sT1U4h/5/gUw9XaTG3NwTKInipxfNjda43jPU/zSuPIHNz3kJFZZ/R5AWroRU2NqTkzh3BOw2WFi375LYUisZHA23uLplUxC7mPt9/p/z6ho9V
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB7253
X-Proofpoint-ORIG-GUID: ws8YyR2UnlOfJ03RvriM9lie45fON0yU
X-Proofpoint-GUID: ws8YyR2UnlOfJ03RvriM9lie45fON0yU
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0
 mlxscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=782 clxscore=1015
 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109070051

DQpPbiAwNi4wOS4yMSAxNzo1NSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEveGVuL2RyaXZl
cnMvdnBjaS9oZWFkZXIuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4g
QEAgLTgxMSw2ICs4MTEsMTYgQEAgaW50IHZwY2lfYmFyX2FkZF9oYW5kbGVycyhjb25zdCBzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+ICAgICAgICAgICBncHJpbnRr
KFhFTkxPR19FUlIsDQo+PiAgICAgICAgICAgICAgICIlcHA6IGZhaWxlZCB0byBhZGQgQkFSIGhh
bmRsZXJzIGZvciBkb20lZFxuIiwgJnBkZXYtPnNiZGYsDQo+PiAgICAgICAgICAgICAgIGQtPmRv
bWFpbl9pZCk7DQo+PiArDQo+PiArICAgIC8qDQo+PiArICAgICAqIFJlc2V0IHRoZSBjb21tYW5k
IHJlZ2lzdGVyOiBpdCBpcyBwb3NzaWJsZSB0aGF0IHdoZW4gcGFzc2luZw0KPj4gKyAgICAgKiB0
aHJvdWdoIGEgUENJIGRldmljZSBpdHMgbWVtb3J5IGRlY29kaW5nIGJpdHMgaW4gdGhlIGNvbW1h
bmQNCj4+ICsgICAgICogcmVnaXN0ZXIgYXJlIGFscmVhZHkgc2V0LiBUaHVzLCBhIGd1ZXN0IE9T
IG1heSBub3Qgd3JpdGUgdG8gdGhlDQo+PiArICAgICAqIGNvbW1hbmQgcmVnaXN0ZXIgdG8gdXBk
YXRlIG1lbW9yeSBkZWNvZGluZywgc28gZ3Vlc3QgbWFwcGluZ3MNCj4+ICsgICAgICogKGd1ZXN0
J3MgdmlldyBvZiB0aGUgQkFScykgYXJlIGxlZnQgbm90IHVwZGF0ZWQuDQo+PiArICAgICAqLw0K
Pj4gKyAgICBwY2lfY29uZl93cml0ZTE2KHBkZXYtPnNiZGYsIFBDSV9DT01NQU5ELCAwKTsNCj4g
Q2FuIHlvdSByZWFsbHkgYmxpbmRseSB3cml0ZSAwIGhlcmU/IFdoYXQgYWJvdXQgYml0cyB0aGF0
IGhhdmUgdG8gYmUNCj4gdW5kZXIgaG9zdCBjb250cm9sLCBlLmcuIElOVFhfRElTQUJMRT8gSSBj
YW4gc2VlIHRoYXQgeW91IG1heSB3YW50IHRvDQo+IGhhbmQgb2ZmIHdpdGggSS9PIGFuZCBtZW1v
cnkgZGVjb2Rpbmcgb2ZmIGFuZCBidXMgbWFzdGVyaW5nIGRpc2FibGVkLA0KPiBidXQgZm9yIGV2
ZXJ5IG90aGVyIGJpdCAoaW5jbHVkaW5nIHJlc2VydmVkIG9uZXMpIEknZCBleHBlY3Qgc2VwYXJh
dGUNCj4ganVzdGlmaWNhdGlvbiAoaW4gdGhlIGNvbW1pdCBtZXNzYWdlKS4NCkFjY29yZGluZyB0
byAiUENJIExPQ0FMIEJVUyBTUEVDSUZJQ0FUSU9OLCBSRVYuIDMuMCIgSSBoYXZlIGF0IGhhbmQs
DQpzZWN0aW9uICI2LjIuMiBEZXZpY2UgQ29udHJvbCIgc2F5cyB0aGF0IHRoZSByZXNldCBzdGF0
ZSBvZiB0aGUgY29tbWFuZA0KcmVnaXN0ZXIgaXMgdHlwaWNhbGx5IDAsIHNvIHRoaXMgaXMgd2h5
IEkgY2hvc2UgdG8gd3JpdGUgMCBoZXJlLCBlLmcuDQptYWtlIHRoZSBjb21tYW5kIHJlZ2lzdGVy
IGFzIGlmIGl0IGlzIGFmdGVyIHRoZSByZXNldC4NCg0KV2l0aCByZXNwZWN0IHRvIGhvc3QgY29u
dHJvbDogd2UgY3VycmVudGx5IGRvIG5vdCByZWFsbHkgZW11bGF0ZSBjb21tYW5kDQpyZWdpc3Rl
ciB3aGljaCBwcm9iYWJseSB3YXMgb2sgZm9yIHg4NiBQVkggRG9tMCBhbmQgdGhpcyBtaWdodCBu
b3QgYmUgdGhlDQpjYXNlIG5vdyBhcyB3ZSBhZGQgRG9tVSdzLiBUaGF0IGJlaW5nIHNhaWQ6IGlu
IG15IGltcGxlbWVudGF0aW9uIGd1ZXN0IGNhbg0KYWx0ZXIgY29tbWFuZCByZWdpc3RlciBhcyBp
dCB3YW50cyB3aXRob3V0IHJlc3RyaWN0aW9ucy4NCklmIHdlIHNlZSBpdCBkb2VzIG5lZWQgcHJv
cGVyIGVtdWxhdGlvbiB0aGVuIHdlIHdvdWxkIG5lZWQgYWRkaW5nIHRoYXQgYXMNCndlbGwgKGlz
IG5vdCBwYXJ0IG9mIHRoaXMgc2VyaWVzIHRob3VnaCkuDQoNCk1lYW53aGlsZSwgSSBhZ3JlZSB0
aGF0IHdlIGNhbiBvbmx5IHJlc2V0IElPIHNwYWNlLCBtZW1vcnkgc3BhY2UgYW5kIGJ1cw0KbWFz
dGVyIGJpdHMgYW5kIGxlYXZlIHRoZSByZXN0IHVudG91Y2hlZC4gQnV0IGFnYWluLCB3aXRob3V0
IHByb3BlciBjb21tYW5kDQpyZWdpc3RlciBlbXVsYXRpb24gZ3Vlc3RzIGNhbiBzdGlsbCBzZXQg
d2hhdCB0aGV5IHdhbnQuDQoNCg0KUGxlYXNlIGxldCBtZSBrbm93IHlvdXIgb3BpbmlvbiBvbiBo
b3cgd2UgY2FuIHByb2NlZWQuDQoNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5k
cg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 07:47:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 07:47:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180527.327226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVot-0003P2-Ks; Tue, 07 Sep 2021 07:47:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180527.327226; Tue, 07 Sep 2021 07:47:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNVot-0003Ov-Gv; Tue, 07 Sep 2021 07:47:03 +0000
Received: by outflank-mailman (input) for mailman id 180527;
 Tue, 07 Sep 2021 07:47:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNVos-0003Op-DW
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 07:47:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1e6f59a7-31cc-42c9-b7fd-e76f51f3e32c;
 Tue, 07 Sep 2021 07:47:01 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-9m-ODolLOeKbciDOBV25xQ-1; Tue, 07 Sep 2021 09:46:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5599.eurprd04.prod.outlook.com (2603:10a6:803:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20; Tue, 7 Sep
 2021 07:46:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 07:46:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0090.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 07:46:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e6f59a7-31cc-42c9-b7fd-e76f51f3e32c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631000820;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=89Wcxu97YlBjEmvf+rIXeD4kJNc16BNfojpE+slgxM8=;
	b=VsBQjFhjEZ8Od2Ywenq/rvsk+tign6RYfhR5/rWsKzXR/nNSgwCW3IsR64mGdCWQjOyLmX
	JJCVR6w3gywpd1dN+4dlaEfMLsxvhPLq+1BC1IoaVOQMEXuVSkyJTD3YicLgxfMA+Jq1A4
	4RDfMou8mWcouJFRxlcYZMwT/S4eON4=
X-MC-Unique: 9m-ODolLOeKbciDOBV25xQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kZHNJIy7H3pI1z1hzTSmbsVUvLU+kwiHA/zEsPFOx+wlQCW/2GCQ3sZbpJVf6Jg+gCTvrfQJBVs5uQYEIOM8kDp0fEUBTl9hM2dJ1T5sfgFHNPWJFZIo3TFAJh8FS8btcxwXgwWFr7PhbElUld7aQlhRTVJEnact7n8zLhtVPiQwlAkkkT1laO3uIikrN/bce5XgRDsUzmHPh1RuGSEJEztVkgTo+FASZwKj38DW2ZwAlMQqMsfytzN/4BfxpKQ0SKFvV7fiIKANxNdtGUH7keT2v0PKHteD/orDNdj+/On/+RYkg0VwwIp7I3umGs8UJ75q9M8HJChdUYM8fv3aSA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=89Wcxu97YlBjEmvf+rIXeD4kJNc16BNfojpE+slgxM8=;
 b=OSz7iR0XVgfTPztoWg4KWq66VC/+YfFIMjBzN9XrVHg5VqwpzybP1j8HhE1qOR5WGCZM9RH3sz+0lO6HrWW4KXr+gAN4p8zkf67LhRXiLahtmSe8aJMbf1qh++FWvC7BXD0l68l6mIWfLxUw7yfCX8wJWAhA435myEPzJwLjvD9Fqe/+5Per/Ah4ePOJ4uCKnyJ5f+qLBbCfa6aFoJ4E6D5hnDrblnHiYdLzZcsn72VQCdx8fmEv23yiAY0wzozYRAhwUZUbdslZog9ZCDDffA9NGoP9+q90Zh1vr7Y6yJBgiOo/7FPiXGxnXk5Nb3ZvPsXsKI2uCgnKJRAGynGzxw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 00/51] xen: Build system improvements, now with
 out-of-tree build!
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <YTY+PNBccQYPhX0a@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8d23409c-8334-b507-082d-9d6472b9d8d2@suse.com>
Date: Tue, 7 Sep 2021 09:46:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTY+PNBccQYPhX0a@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0090.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 89f6afc9-8405-4d3f-0aef-08d971d3ab5e
X-MS-TrafficTypeDiagnostic: VI1PR04MB5599:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5599F97204DFF9B693BA9D69B3D39@VI1PR04MB5599.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1q1CH1tPWWM+nD0vnOByfWeK/W47qo9JrPg2EudA3uFHUURNtt/HycTuRlsl//OBLd/L1NKxB2QFvaPk4PwJKgZ+avi/KpZheESy+nmyYulUXccFRD0rzK/vrRbPntEDLE51YPc8To5+3FhpWX836uueK74zmwx9ltzX99PAeeTQmUXzN58Lm48fHo8XiyMdbGOEqnO5mHD18Y5RsW63nAAxLKeV3bR/aUQgfEV+k1mNEtSEcynXxl1XQqYg7It1oAB6oCXwtCtwKyPq3hGz1xVf80DWOh5YtC3MQq80001RJl5jH5Rm6Rv4YakBKm7/NQmwOWjy27kD8rTHH0L7SVfaAfegELRRdX8NU9Px9SUyM/mm2CRMuzcACqujOudAM17yUtvpqfzc/0rZ6PaiGNnp0TcEjW3cRu3WGGzkuf7kP+4ls0LoccabQCaZneSAVWil18vsBZL3f2ZeMa5eiaNEPoqVYwHxe+Uff4p/4/b6mC3lSlgrKN3mZtBiGf3FvOCJo9NARGoKuhjtZPu2UcwPZaT07u27heDfRE4kMrhyb4P8DewlZWawqGzUVHXyxPlFId9mQxVzUdi1V7calyfzrDiROLqNDBFoMaAb/olXjp4W1uUA/1vfWy2LTVcIPSterxUNzlCsWh8S62tU7+hcFtZeQgbHWJ4nlG0fTj1U5XS/hZBQWFkQfh6SO4vVZughFVDES5DjXzlCSWqmy9Hxka2kf82tDY9Y8ftmnLuy554I/V2QIMQoBemPyb0V
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(136003)(376002)(366004)(396003)(66476007)(66556008)(186003)(53546011)(6916009)(5660300002)(38100700002)(478600001)(4326008)(6486002)(31686004)(66946007)(16576012)(956004)(8676002)(86362001)(4744005)(36756003)(2616005)(31696002)(316002)(26005)(2906002)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?U3BqQjJTVWVnL3JjU0RScFFtN1RBTVk3SGxsdlhvOFVrc3A1THByVlBvd2ZQ?=
 =?utf-8?B?eDlIVzdjQXQxSlEyNjRmZmpvVk45NG9Ld0VzQ1FWNXhHSmF3ZFR4ekFuRk93?=
 =?utf-8?B?ZFRSWE5HMkswN2IvbC9vQWhzbEt3R3FsZ3VsTHRERkNSUWNaMTdUVG42QUty?=
 =?utf-8?B?dGIrWlliYndTYWhkcWk2SERJYWZrQnZ4Z2Zrb2dOczZnd2Y4M3MvQjZlRE82?=
 =?utf-8?B?SkZoTUJrSlE3RDRiaEpoVFpMeExrYndML0p0ZHFENTlPa1lzQ2dnVENFZjNF?=
 =?utf-8?B?bVUxR2MydEhoalpUYzNxUU9SN3RvME5CK0hyWGZLczhHZlBiWFFUdEtMMDRF?=
 =?utf-8?B?cnpLVFZydTZMV1FGR0QrcTdib1gwTmJNU3V0a3lpeVJuUUhmWmpCM1dGTld0?=
 =?utf-8?B?NUhuV0xHUGZPWGp3L2FSL1BMNEVCbDVXUCtsN3E2azh6VDZDUFpSN0wwQzRJ?=
 =?utf-8?B?T1FZcU1mZnJ5amdtM1JTQ0pHZXNWZDg4VGNxUGkzeHpWYkxoS1pOMHViTldz?=
 =?utf-8?B?L1ZvcTBJTUIreHR5ZCt6RXJTamNEWjRwa1M1WUZSTElLU1VEZ2Mrb0hadkVi?=
 =?utf-8?B?VW9CL011SmV2TFh1ZG9ZZDZXZ015c01VSmNWNnBBM1FPWCtiazQwbGgrQTZl?=
 =?utf-8?B?ei9qcTNNdXd0bWJPSEJRei9vd3JRT2IycmpOdkVZMTNEQzRWOGxDa3JiY1dr?=
 =?utf-8?B?aEw5SXRqYUFtV3lQYlU5bWs3aURJdUMwcUI4TXhhd0YwR1AvSkNjOTVaNkgx?=
 =?utf-8?B?VExDSVFVUUMrQkZxZWs3Rm95eFNWUm1QMU9sOHMzYkFQc0xHam9FRk1iNndu?=
 =?utf-8?B?SU5xclV3V1MyZzBkMjAyUlk2YmFsemhCNFB4TUdnR3BUL1Q4R1pBYytaOUtB?=
 =?utf-8?B?Q29vSHFPZXY2dWRJVmptSlNaMEtkMTltVUdHMGFHR2JqeUx3TUFZOTJURndh?=
 =?utf-8?B?NWh1SUVjYW5uTHhYZW9TMlNUN3kvRzZ3QWM1RjNkZ0Y2WWlJVVNlVFJ5b284?=
 =?utf-8?B?am5pbDFGZWhQZkVMNUpRdTJRVVVhbklsR2svZmFzWnE0VXhMdnpuQnltOS8v?=
 =?utf-8?B?azVjQXNGcFBGdUFkMk9rdWhqM0Fnd0tZMFQxUGJaV3NYQUlhbkxFQmNxenJI?=
 =?utf-8?B?UnhEbk1ZdWVpRXA2a1QrSXVLUjdzc2FWeXlTRnAyQlJwZnphcTNvbW5jZ2tJ?=
 =?utf-8?B?dnlSNjgrUXNBcTlNRDEyZ1REelZYNlRmUWN5OE5icVQrWnBjaUI5TzJsc2VW?=
 =?utf-8?B?NytsYnNmNjRiRE1NcXhiR0Zjamc2dXNkNFV1OWVnMHdVUnF1M3BOY1hMa1hU?=
 =?utf-8?B?NHJFMzFqVEZFc1FIUE56QTRkTUw1TUROSFhUY3lraTU4UmVoVURrRm5JVDZx?=
 =?utf-8?B?anJ2RCtVTnMwOURMNExpOFdSaTN5ZEErNjEwTXVSNFVwV0hST1Fkc0tlWVBa?=
 =?utf-8?B?M0x4M1E2aHF6NGJSN3lIWTZNT1c3YjZKTjNyWDNHUWdZQVI0WW54S0gyemRU?=
 =?utf-8?B?VWpxYTFMMktKcXBXS1psRlpRcFZ2Nks2RVNqRXhtVkNDZXhxMGdyajlwWm0r?=
 =?utf-8?B?RjEyd0ZOWmhZS2VkWUhQMUFORWdlN1p4Nm54ajRpN1ZoR1dYeWJQV1VLcjlB?=
 =?utf-8?B?eERDUTVHZFUyb29FNFl6b0xQSnFqTzZwRkMrcUtPK0I2VUpLU2poVjBoejdX?=
 =?utf-8?B?MzFmUkJROU45R0ExNmFOc3dXZUZrVEtNZEppNXhwbkY4M1BCcEtWN3pyODF5?=
 =?utf-8?Q?7g5pPFfWcuEg/9f6CQDvX2ZIOOxAMnkmAUqJ3NT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89f6afc9-8405-4d3f-0aef-08d971d3ab5e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 07:46:57.8010
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bq0+wk9JkVdWlN4fYuHfeGwnQaTanIiuPI3zBO8eMrUdGjemly0v4qnNyaq0uAGUR6NsVBfkNyydNnH1Tk4KKQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5599

On 06.09.2021 18:13, Anthony PERARD wrote:
> On Tue, Aug 24, 2021 at 11:49:47AM +0100, Anthony PERARD wrote:
>>   build,arm: move LDFLAGS change to arch.mk
> need edit commit description, but otherwise ready
>     not needed before [PATCH 21/51] build: set ALL_OBJS to main Makefile; move prelink.o to main Makefile

While you did indicate you'd be fine with Julien's suggested edit,
he did indicate he would take care of this Arm specific change, so
I've left it out.

>>   build: move make option changes check earlier
>>   build: avoid building arm/arm/*/head.o twice
> both ready

The latter didn't apply cleanly for me; I didn't go look specifically
why that might have been.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 07:59:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 07:59:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180534.327236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW0O-00059a-LJ; Tue, 07 Sep 2021 07:58:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180534.327236; Tue, 07 Sep 2021 07:58:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW0O-00059T-ID; Tue, 07 Sep 2021 07:58:56 +0000
Received: by outflank-mailman (input) for mailman id 180534;
 Tue, 07 Sep 2021 07:58:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xlGX=N5=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNW0N-00059N-Af
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 07:58:55 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed1a5942-98c0-438d-aaa1-09e80813426e;
 Tue, 07 Sep 2021 07:58:54 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2B7291FD3E;
 Tue,  7 Sep 2021 07:58:53 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 035A1132AB;
 Tue,  7 Sep 2021 07:58:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id ziGrOrwbN2GhGQAAGKfGzw
 (envelope-from <jgross@suse.com>); Tue, 07 Sep 2021 07:58:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed1a5942-98c0-438d-aaa1-09e80813426e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631001533; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=AMcdZLNTncTH+Yy7z6MnCAdXPsgZXnKDxbbM9Ifr0u0=;
	b=PxeXKKp54XMGPMbkcv7Vx+UZCB2T09V8TQYm2L5sIV+Pxaf8MTDIomVJLReoZltvA0v5TT
	z4t+q4ijLQK5eWxnBIBTAdxczNQOGajPDXh+xmWjFLsKdH2cGArE1hXJdSKkuw8GdL0+Dh
	AAzZycC4rnOZt/IhRR4A9yx9jtcpvCk=
To: Sander Eikelenboom <linux@eikelenboom.it>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
From: Juergen Gross <jgross@suse.com>
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
Message-ID: <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
Date: Tue, 7 Sep 2021 09:58:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="nkepnofagqj0Zs3mMsn1mbJe64GyvUYdo"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--nkepnofagqj0Zs3mMsn1mbJe64GyvUYdo
Content-Type: multipart/mixed; boundary="8M0QKdzDc7d04oORWPj6gBj8yDHBPi3So";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Sander Eikelenboom <linux@eikelenboom.it>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
In-Reply-To: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>

--8M0QKdzDc7d04oORWPj6gBj8yDHBPi3So
Content-Type: multipart/mixed;
 boundary="------------26F5A7C1DA5D359581D4BECD"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------26F5A7C1DA5D359581D4BECD
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 23:35, Sander Eikelenboom wrote:
> L.S.,
>=20
> On my AMD box running:
>  =C2=A0=C2=A0=C2=A0 xen-unstable changeset: Fri Sep 3 15:10:43 2021 +02=
00 git:2d4978ead4
>  =C2=A0=C2=A0=C2=A0 linux kernel: 5.14.1
>=20
> With this setup I'm encountering some issues in dom0, see below.
>=20
> --=20
> Sander
>=20
> xl dmesg gives:
>=20
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63b936 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a0622 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6b63da already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 638dd9 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 68a7bc already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63c27d already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a04f2 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 690d49 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6959a0 already pinne=
d
> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a055e already pinne=
d
> (XEN) [2021-09-06 18:15:04.090] mm.c:3506:d0v0 mfn 639437 already pinne=
d
>=20
>=20
> dmesg gives:
>=20
> [34321.304270] ------------[ cut here ]------------
> [34321.304277] WARNING: CPU: 0 PID: 23628 at=20
> arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
> [34321.304288] Modules linked in:
> [34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted=20
> 5.14.1-20210906-doflr-mac80211debug+ #1
> [34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)=C2=A0 ,=
 BIOS=20
> V1.8B1 09/13/2010
> [34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
> [34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60=
=20
> 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00=
=20
> 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
> [34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
> [34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX:=20
> ffff88807d416398
> [34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI:=20
> deadbeefdeadf00d
> [34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09:=20
> ffff888006160cc0
> [34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12:=20
> 0000000000000000
> [34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15:=20
> 0000000000000001
> [34321.304320] FS:=C2=A0 00007f5071177800(0000) GS:ffff88807d400000(000=
0)=20
> knlGS:0000000000000000
> [34321.304322] CS:=C2=A0 10000e030 DS: 0000 ES: 0000 CR0: 0000000080050=
033
> [34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4:=20
> 0000000000000660
> [34321.304326] Call Trace:
> [34321.304331]=C2=A0 xen_alloc_pte+0x294/0x320
> [34321.304334]=C2=A0 move_pgt_entry+0x165/0x4b0
> [34321.304339]=C2=A0 move_page_tables+0x6fa/0x8d0
> [34321.304342]=C2=A0 move_vma.isra.44+0x138/0x500
> [34321.304345]=C2=A0 __x64_sys_mremap+0x296/0x410
> [34321.304348]=C2=A0 do_syscall_64+0x3a/0x80
> [34321.304352]=C2=A0 entry_SYSCALL_64_after_hwframe+0x44/0xae
> [34321.304355] RIP: 0033:0x7f507196301a

I can see why this failure is occurring, but I'm not sure which way is
the best to fix it.

The problem is that a pinned page table is moved: the pmd entry
referencing it is cleared and a new reference is put into the pmd.
This is done by getting the old pmd entry, clearing that entry, and then
using pmd_populate() to write the new pmd entry. pmd_populate() will
lead to a call of xen_pte_alloc() trying to pin the referenced page
table, which is failing, as it is already pinned.

The problem has been introduced by commit 0881ace292b662d2 in kernel
5.14.

Following solutions would be possible:

1. When running as PV guest skip the optimization of move_pgt_entry()
    by letting arch_supports_page_table_move() return false. This will
    result in a performance drop in some cases.

2. Unpin the page table before calling pmd_populate(). This adds some
    unneeded hypercall and without flushing the TLB I'm feeling uneasy
    to do that.

3. Add a check in xen_pte_alloc() if the page table is pinned already
    and if this is the case, don't do the pinning. This is a rather clean=

    solution, but will result in other failures if a page table is used
    multiple times (this case would be caught today as in the failure
    above).

My tendency is towards solution 3 as it is local to Xen code and has the
best performance.

Any thoughts?


Juergen

--------------26F5A7C1DA5D359581D4BECD
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------26F5A7C1DA5D359581D4BECD--

--8M0QKdzDc7d04oORWPj6gBj8yDHBPi3So--

--nkepnofagqj0Zs3mMsn1mbJe64GyvUYdo
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE3G7wFAwAAAAAACgkQsN6d1ii/Ey+E
TQf/fpxzZlRXgmKEMF/OYUVa6cmBJ/w1lRjz5L9orGtjrbsGzCFW1VTwTimiaEhvR1fyy6gWxfr0
6QNHpmx1SRIlNDtTq09z48Vcq2sx3l394mv7cdiTnrSa2z7gZb2197VmLfT6DfCf1eb97QCsQeXQ
q9JtoPHHaMoXfqnPkga1CurulKm3yGebk4Ecl1c/EWx0T5XVdBrLhZRpWlLbo2UD/UutA4MDina0
YimVdc1TasQXd6KCBQG0H9+Q50vJVGcgluFXDVOrf1mB+nWvsemb0svCklBU6+ZQe0w/SeFEGdhT
b92x+mApf2/awQd6erf5XXaoF4/LR3NphJnJ5mi8pw==
=bYZD
-----END PGP SIGNATURE-----

--nkepnofagqj0Zs3mMsn1mbJe64GyvUYdo--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:00:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:00:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180546.327248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW1h-0006wk-DA; Tue, 07 Sep 2021 08:00:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180546.327248; Tue, 07 Sep 2021 08:00:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW1h-0006wd-9i; Tue, 07 Sep 2021 08:00:17 +0000
Received: by outflank-mailman (input) for mailman id 180546;
 Tue, 07 Sep 2021 08:00:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNW1f-0006wN-8v
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:00:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1be4cc95-5834-47be-bbd0-023698e99f46;
 Tue, 07 Sep 2021 08:00:14 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-RbVMblSFOeGXKIM1TETcQg-1; Tue, 07 Sep 2021 10:00:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (20.178.127.208) by
 VI1PR0401MB2605.eurprd04.prod.outlook.com (10.168.65.146) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22; Tue, 7 Sep 2021 08:00:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:00:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0158.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 08:00:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1be4cc95-5834-47be-bbd0-023698e99f46
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631001613;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=qGYW5LYytLbES3ZLTz8QqvgRsEvCsh8vaEda5ensums=;
	b=BJEwoJ7Yb+YOMp0EvEWTyvVtwrVqKMfVw4/SesvqB6zHsc4AZffJMYgwoHe1VW/2MmydEW
	LRl4m7kGQWZroGSXkiR/OhsDFH/+A87nbSoERVz+oFf0c7qsHFVEnHQrQPRWMbzu6x2gFz
	KfuCkeDZHFrULCVjHUNDzBsbzyBKFok=
X-MC-Unique: RbVMblSFOeGXKIM1TETcQg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G8JOlJDb8Lp2O8VHnfiauI8z86sTs4NWpdicP7UfTonCKu2NQ+wNuDxjMavY2Hq+ePXzoJH0DQxBSxy9A1ZipOmlaV2qI2IYiwe2ug38Pvn1J3LyW3Gx5LHBkxE2NVwFIuI3m9fw/8f0STgZcetX4g5sV/9pNRAymZkLzVTCnmYrOiJw8Hut417aV3UB/PLu9Ih4nuuqjcxpdExEHUqPmndXqNACcukm5OVZBe9/0jmoMu0e7kuJK8hdM2iJwRv7vCgjXohoaimUMCBL9Vq6X47MhTaNuNqmm85f63bNLqXrOvkLsJUfnmCVrRg1zIqRafrT22/l9zE82KJv18OHig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qGYW5LYytLbES3ZLTz8QqvgRsEvCsh8vaEda5ensums=;
 b=iNFLr6hdUFsDa+uDTSAjoA3I44L5iYlpo7TWelAH77Xd/cdo0V81fVYuS/BiB2ZVlfnPLopCRjzHVFIWIKMi4yc7vJMZzcNnbGriwKP+xu/wc9IPgw+WQcfGdbYupBx5HcOPJbymCgFPAbPNitwjT7lcAvNa/RC4BwIgPy2TuXQ24JsebwkRsPnIIqK5NR6jIN5kH9TlGxQrVEC4TeYe4qUB9z68isEgx12DbF1brlVk+o/9kX/bFg9kEqLVDXY1phI0BnEy2LTUws1l2u1ZxGza4Oplez/wYny8tOiKWiGCvJ4H+uABam2qrediKAKNGlemJrO2S3Am/sUMsruoYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
Date: Tue, 7 Sep 2021 10:00:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5366265c-d169-041d-f812-1c49260f6499@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0158.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 223bb35e-1efb-4fb7-3dd7-08d971d583c2
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260581B9B2FC7C1813931FC6B3D39@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	W44jmgzxr1Bqfl5Ur/sgExxDR1isKpyvDiD/S1Nj9ebyDjNkIFg5zmvJR8RzhLuQKyWoZCADr+eemcZsXbPvxeS7Zyg/D2JQD9py+QGuleF3ylSNU9dzEjVOrUuOlvhnaVnI1pwLSLFyxetDCi6u7u6l8pgZw9ZI/aCZNXfHDv66Qv+8J25ACejM+PvD6oBIheXdOO42QEEGxnyYbNKS3+JWNDGWGdAcWtHb2eTtYR3XPmRdns6UMbxz4S+qV9Qpd7pT2QiZ3ifJvOgD45N/o4cTDjJdsfzT7NYC1St3IpQa8OmRpEYLIaxqsrjYsug+CZdlTWudOMBFCBO7iN3U9R/gf1bZzZV08LE9MsmbkQ69dJ6kkLOHWl3kzC1TgZZPmrPwS88Hi2tcmyVfybxBCBVXDFiWsi6i8xdTvw78NE1tfKNPXkjJ8TrelGCh8BctJgNadZul5UVW2NLVvJSalTAQasKfIBVzpKIMDsjarZGonuVt7WKETkAEh9a2Q5byePxacRUIw4RuEUHDy/1AKF8BTI3SlLJWrIimJAovYNkjkufstZoozjhsYJ/VhgicSLIu6iMQNg5y5NaxNAT9QC9DMI5DRQmhhXJtAJl78n6qfCMqmGOfH3G8g3tChiBqu2Kw+y8q70FvS0i1w/5ocj0euHEZZxg5CICjl1yD1ZS1NgwKVr0rldFlOQdtehW4oi40u61aLFliqe0OLlL9IVIMrmriAdCeiqr9kknE5855TY33sLwpEJEgMONwcvyM
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(396003)(39860400002)(376002)(366004)(136003)(5660300002)(7416002)(53546011)(31686004)(36756003)(316002)(16576012)(66946007)(83380400001)(6486002)(66476007)(66556008)(186003)(2906002)(26005)(956004)(38100700002)(31696002)(2616005)(478600001)(8936002)(8676002)(4326008)(86362001)(54906003)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VlpPMitNdi80enhCTTlhemxDSTVVK1dwNEJQdm5zMXBqcDR1MWhIb29hVTdY?=
 =?utf-8?B?RUxEV3JCRzJ1RW1KUjJLbWxLTTR3N3FmNXpwQm5ZdXgxWjAvbmNzYS9hY3E5?=
 =?utf-8?B?Z3pYZEw0UG8xUWdhRVdUaTZqaGNybzkxU1RzQW4vb3pXQ2Y2VmQ1MjU4U3VF?=
 =?utf-8?B?ZlBTd3V2UE9DeDNiMk9relArbTE1ZEJ1M1RhVnFTV2VWeVJYOThxb2g5alJh?=
 =?utf-8?B?S1daZkV0NnBLYXVPSXZYejRSY2dxNGpQNmhLVjlHVWVWWlRQdktBWC8zOFpE?=
 =?utf-8?B?cXdTUmxWaCt1VC9LbHVVUmI3Qy9zNDZDSUlQdE54TWVXU0ZHRlhyNHQ1QjRu?=
 =?utf-8?B?L3BLSWNoME1lUk1laWJ2ekVCbnNua0c5ZUhOK1NCbVh4VmdKQkdDNFUyK2dj?=
 =?utf-8?B?dzdiQ25FNWlCZE1mcGg2NjdSbnBDSDdxNlZ2Rk1uNHp6NGJaV3lsRHpBblZG?=
 =?utf-8?B?UjAwYnhKbmY2N3JSVmJMcE1kcE9wVkhCNEg2VDYrNEE0dDJ6c3Vud0xlbWR3?=
 =?utf-8?B?OWxEaEhBODlNZEQ2NlpvNDBYZEd2M1Rka3o0djU3cWJ2YmhzVXF4RUpVSjdC?=
 =?utf-8?B?NUtaVCsycm1tZVVKWTdUYzBXcEpMNU5scUUrZTdZSmJGK3lhZEV3Yk9lYnBG?=
 =?utf-8?B?NFlFQll6QXFiZTBVYmdSSFBtYjJHUU1nZFBUWWQyOEM0djdTdllnR0hvZE0v?=
 =?utf-8?B?Y2pVcnd5Q2tLOVVKU2JETSsrVEoyakxBL0tQekVLOWhaUFZsS1p2Mzlta3Mz?=
 =?utf-8?B?ZkQ3SVJQcDI3STVwVmYwN1ZZcnVCNTVPaGtjUXdNWGl4NTIyTElUQUQwU21Z?=
 =?utf-8?B?QnJTM2RHelNocVduOWl2aFRybVgzQ2ZLVGE3eGljMUptWk1zeTZLODE5VXpz?=
 =?utf-8?B?eGRvdEl4OG9tbFlNaUhLZmRMN3d3M1o0T3RUdkR4S3EzeStMOUpVMFJaazdW?=
 =?utf-8?B?bHIwZ0twbi92M0dLQTVKYVNTMXB2eE15bkkvOFpRRkg2emlpRUh0VzBIejFj?=
 =?utf-8?B?VFBGa3hrdjRQTFp3eUl1bTI2RXhXSm9RR3lRNDkrSGJ5Q05Ec1lsb1VMd3Rp?=
 =?utf-8?B?TmxpamFJaWRIaXE2YmtwbHJFb2RXaUlnUldtdGRMcE1wUVF3T2lFcE9pd0hy?=
 =?utf-8?B?RnVnRzV4VTAvbWlyT2UvdExkakhJOHNVeWhHbkJVUTdMZERPZWVYVHZuR3hZ?=
 =?utf-8?B?d2dXbEd3a0pIOE8xbk9jcHdIcXV5VTZhWU1iWThFRUI4K0lGL1JvQmlYOVdM?=
 =?utf-8?B?c0FiNVMycG1rViswa0g4TVhCN3BFR2RtU2MzS2ZObmJuaXV5ckFFL0V1L2VF?=
 =?utf-8?B?eVpnQ3g4VHg0UmZsdjN0elBmR3lSZTlTZkJ1bVVSdnBEdFR5K2hsOUNlQWdB?=
 =?utf-8?B?a1JyMDJMNGhpd1lXdUdhdzl5TzQzWTUzYVpaejV1SlVmT0d3S24xMXVRSEhD?=
 =?utf-8?B?Uk5KN29zWWU2YTV6dVc1WHJJL3JEalgzVVdBK3RhcmFhdlJjZUJkQWhVN2Jy?=
 =?utf-8?B?SWVPeE5Wa05rdzVMUkVRVnViVjA2V1lSS0Yzc0lJRUExczdXM3plTmowZ3c5?=
 =?utf-8?B?WlIwaDNDVkgrVkhseStYMTZnZVJKNGtvTXdPWE9LT1dzYjhlY0QvQWhTaVJr?=
 =?utf-8?B?VDVvYTBjUk1CYlJhcUhzazZFd1lKNlN1UWdEMUpDZ0lmeXdSR09KV2hFVmgr?=
 =?utf-8?B?YlIyZUZHSGFjVjVLMFpUMGowQ1RsamV5RldXRjFvd3l3NU1zKzVKMFk0NDJs?=
 =?utf-8?Q?PqKd+raBG1YqgM2IKrDake8vGx9V5fO7H/0mO4L?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 223bb35e-1efb-4fb7-3dd7-08d971d583c2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:00:10.3273
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CppwKVVBl30tD5wgR07UVvKzn08D8WT+tB8mfkilSOW19cqXFMMHgDGSWBWtU7gt7Zb6n3XTAaRoFzaEhVc6dw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 07.09.2021 09:43, Oleksandr Andrushchenko wrote:
> 
> On 06.09.21 17:55, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/vpci/header.c
>>> +++ b/xen/drivers/vpci/header.c
>>> @@ -811,6 +811,16 @@ int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
>>>           gprintk(XENLOG_ERR,
>>>               "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
>>>               d->domain_id);
>>> +
>>> +    /*
>>> +     * Reset the command register: it is possible that when passing
>>> +     * through a PCI device its memory decoding bits in the command
>>> +     * register are already set. Thus, a guest OS may not write to the
>>> +     * command register to update memory decoding, so guest mappings
>>> +     * (guest's view of the BARs) are left not updated.
>>> +     */
>>> +    pci_conf_write16(pdev->sbdf, PCI_COMMAND, 0);
>> Can you really blindly write 0 here? What about bits that have to be
>> under host control, e.g. INTX_DISABLE? I can see that you may want to
>> hand off with I/O and memory decoding off and bus mastering disabled,
>> but for every other bit (including reserved ones) I'd expect separate
>> justification (in the commit message).
> According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0" I have at hand,
> section "6.2.2 Device Control" says that the reset state of the command
> register is typically 0, so this is why I chose to write 0 here, e.g.
> make the command register as if it is after the reset.
> 
> With respect to host control: we currently do not really emulate command
> register which probably was ok for x86 PVH Dom0 and this might not be the
> case now as we add DomU's. That being said: in my implementation guest can
> alter command register as it wants without restrictions.
> If we see it does need proper emulation then we would need adding that as
> well (is not part of this series though).
> 
> Meanwhile, I agree that we can only reset IO space, memory space and bus
> master bits and leave the rest untouched. But again, without proper command
> register emulation guests can still set what they want.

Yes, writes to the register will need emulating for DomU. Reporting the
emulated register as zero initially is probably also quite fine (to
match, as you say, mandated reset state).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:04:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:04:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180552.327259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW5G-00081E-S1; Tue, 07 Sep 2021 08:03:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180552.327259; Tue, 07 Sep 2021 08:03:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW5G-000817-Oo; Tue, 07 Sep 2021 08:03:58 +0000
Received: by outflank-mailman (input) for mailman id 180552;
 Tue, 07 Sep 2021 08:03:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNW5G-000811-4P
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:03:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ecd9df2c-25d5-407c-a70e-980aed8a372c;
 Tue, 07 Sep 2021 08:03:57 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-s4LotqQnP-2KPd2_f5FqLQ-1; Tue, 07 Sep 2021 10:03:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6301.eurprd04.prod.outlook.com (2603:10a6:803:f1::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 08:03:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:03:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Tue, 7 Sep 2021 08:03:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecd9df2c-25d5-407c-a70e-980aed8a372c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631001836;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=A5EGaKlsiNTjnRo/nw/nllmUB/AG6tuR/OJmZarPtZ8=;
	b=Ut+xOJaWeQ6Gy92qgJ/Agv2UB4D4XHi7I7fPmac4sdf/j15duaVnNDC0qnMYIfHpdLEN5J
	ViuLCTUJiiCKceyJKVA5isSr3wWWgxsUEXbYSmIzPHYrejvVDg4PSD3wGMfpqNWNzXwJpU
	Rzz9LKbVJZECgHltKo6ExgiubVn+Hkk=
X-MC-Unique: s4LotqQnP-2KPd2_f5FqLQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L3AmYJCtUZXOyyhLbTMQcllVPOL+RGMhLZqx8fi+HiIbQ7hszKWMV822/82ivYBTynjVI24R7QhNt3cJy/4dnxkapIEPRyzyQgUxpuVJLbcepjBynz3nGbLV8LaYw8XMnRvE7dVkxQ1ymZZ/j10UMQDa3HM6nvVzrawuxhhVcCABZ6Gb6/2VcCIwvM7WQZqGz5WQ6CfXPOc9izHIJ8M1N1oDI0cB1PZZ9k3hECDKP9ibOglpE/m/fxm9vvFdtiCB+EDF5pmjSxzYOXPPR5O32PKJg8AKN1vQ/HmVK0zmwpRDDcNkbOxWkwK4gqkAgjSS5PjKAUsFH3ijAHqrF6O2Og==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=A5EGaKlsiNTjnRo/nw/nllmUB/AG6tuR/OJmZarPtZ8=;
 b=djq5Vmyc4BHPYuA68x5tizrkd6sAmZL1lZURsRrI7kXeP30BnpU0q6wDLkv47dxMYrVhD10sf4i/UKj7nP52RSnFED+ZiDbyoiYg/wSdkb4yb5hykgVpeZqwzkEdU6wiDu6aqlxarvw1M6Be0kB/0/bVSM3PkO2RFqoETED/O9B82yIHwjgns7WS3PehcIqURdIRo2vrcoUCc0k/+RpVWT4AYVel/TJ5EM0Mugl8Ua3Pg09zYgmFU9WkzlEZuiaB3RsAI60eF36Q1DXfGLju3BvrXLoiUwYdOOiiQuvAZR16jPRxcSaEd42wvndtru/2qYtS6REkfwxdtA8+N3S9kQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
Date: Tue, 7 Sep 2021 10:03:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8977a94f-4175-4f22-c12c-08d971d60880
X-MS-TrafficTypeDiagnostic: VI1PR04MB6301:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6301DBA0CF18E6FACC64FAB7B3D39@VI1PR04MB6301.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	POxqRJaYlTUXSV0JGrUwU3LlydJJ+rNpo5ScsLwUJpqfacPlUPYMapOLkNZXpwlcwnVZNTxMtJhse69k38FGoZsS/5ZFHW+xm3CIldKz0m+tlXIRvMT8ParDN5WVoox5NC8KBa3G7qnnN+iyl2PypzCsBYkI2snlmBsnlOW7d57mbt4mimwGpwREY0cs0hxQsdbGhXO5CRifmpFCvm+Eo+AdYuTdwInDfA+JfJlUUEHaEZ0BMUbWaS/kbDHc1GnX9bGxth3y+GjeyoVN/afmQsYmn20J+JENsTmws+JUMSHzLPMzyGG1RHZwn81JdCvGpzbxfY06gD4KnSVWHtD8YgD2D4dmZaY61AHlTl9+n5dBV5xcwNcH1mGlp8Wx7+qh8CEvcBrpWmOfKmU2fTtIvaeeYTLXGdayxNDkT0ztLxoTjvDla+yh/TE1SW8QuguIKDL3T7XYxs5ztuxBI8LPCnsEFBIEHT9ZT06VpvIUH1aHKctSOQUZGVpcX3EjNHyy/iUX3xOeLHGEpZKQh7FcdfU+qtSNpAS8crdHHWX3H6yhJXm8alNoHh5oDcCw29PGogztS+hZpX7boDGHcBXg0nXJt2FWF6W5u6cO6+4N6W/D1L8fmurbyQHVS7A0YSvhd35iDalq+gBbK/auOJsarKcFpWvZ+vF4FpuXgjO8/+WeDS2zqLeKHGC+ydE4iUR0Y8/MRVEfkZ/ZrMirqWzsh8eXQqA/ZAmV52xu3t1bxCZTa/hFebPU9NCzLFl2yXJ3
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(136003)(346002)(396003)(39860400002)(31696002)(86362001)(186003)(2906002)(26005)(8936002)(8676002)(4326008)(53546011)(316002)(956004)(38100700002)(2616005)(16576012)(3480700007)(83380400001)(6486002)(66946007)(66476007)(66556008)(36756003)(5660300002)(31686004)(478600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?enREKzFIK3BFKzl3ckluRXM5T2hVNnhpaG1nSGxwZFlQUkh1Z0JMZXA1K3BJ?=
 =?utf-8?B?eTZXemw3NGxsSUd2MlpOdDEreDBtTGRoOFk3eG9qaGR6Vy9EbkxqWVYzaFkx?=
 =?utf-8?B?UE40bHNUQTJqNStGUkliZERVNm9tT01oY01kakZxYThEYUJVclk4WmJVekU1?=
 =?utf-8?B?TmdCL1VlMmJrZG9CY1hUS2drRnhFNzBaNzBkTjd0SVJyK25PenVWV1U0ZnIv?=
 =?utf-8?B?cHNCckJldklrUXFYemVNNTBhMUtwaWxvZWxZREtXUkU2ZHMwNVdrWG1ac25j?=
 =?utf-8?B?LzNYR3EzRHhKL1BQSS9JK2NRRjZHTkxseTNZQzBRSWcxL2xUdGljTXM3NnA0?=
 =?utf-8?B?WWlPaDJyM3lPT1luUjBnRk02MmxmVWhBNSt2cDVMUE9PeExQTVhPWVEyKzQ5?=
 =?utf-8?B?REFGL1FNeWVEMnJiMTFWVTNxWExtRXhKVzVZV3RCTklvRGVxMVVDNGFaTzc5?=
 =?utf-8?B?Zlk5Umk1YjF3OVhaK0dvSHYyL2taVFVQRUhYWDhIVHdmVFlQbmtrTEpqWXNt?=
 =?utf-8?B?L05ET1lzSkxyQ1RTOHo0U0IrcjNQbm5uWDNOTzRnWklqd0dKWHhrRjBQY2FS?=
 =?utf-8?B?czN1WkRiWm1hWXdBSzY2ekZHQkplU0hMczV3RHpKUkQ1Z2R1Z0RSOE9Dekd6?=
 =?utf-8?B?UFZEN2pKMjBQazVDTmtmandtTUhZejNFZTE4aUhMZzlNS3QwMElvY0RjTjBQ?=
 =?utf-8?B?M2JnRU91TXBHSktTS2NhcTd1RFFzR01BSEd5UWdCY3Y4WjdUOHl0dFB4bThS?=
 =?utf-8?B?bmNtUjN2TmpKSHRVN1NwR1JZODhhSzFlSmNGYno1aEk3Vitid1Rla25LbjQ0?=
 =?utf-8?B?SjV2N2xKS0xPTXR1YStReW1XKzYxR0ptZHU5cXFJUzRWcHYva2t2dTNWa1F1?=
 =?utf-8?B?RjZwYkxmNmlFb1pHeWFCTlZCZzdsSElJOTBxV0VnWVVSM3JvQytTMmpydDVE?=
 =?utf-8?B?WmNIdjI3bmtxWW5xOXljK3p6VG9zTEF2ajNrdExFenpIQWNDZStHT1NIWjFz?=
 =?utf-8?B?RDFCb3NjSHBURjdJNk5HbzR6VmxOMElKd3F4QjVVS2s4YjNnTGVFSERrRGFQ?=
 =?utf-8?B?R3VWUDNEbnc1RVBQZ2xaVE9vYjk4OEtNNGdaSzZWb3lWRm9qeFBHeCtKQkFv?=
 =?utf-8?B?MnhpZkdoQ0ZIQ0xSdU0rNEhlUEROdEpLRHVvU3hHVFVqTGZ6T1VpeHk1OXRk?=
 =?utf-8?B?MHhUVkxLOHRobnUwcWdwUlpaOUhOUHFsN0dwS1pyWnpqZDNHMnpBUGd2MkZy?=
 =?utf-8?B?STRCWmJyR25CZ0lmMTlZWGhnQmcxR2YycFM5VGNvWFFhVGc4Q2w1UWZ6OVhu?=
 =?utf-8?B?WCtTVW51YWd5REdWblNFUTA0RDdrT3pTVmRxUzdiYlg3WkJOTjhpeFpKc3hW?=
 =?utf-8?B?dUNrV0ZJZHoweWxyZk4zSEEzcnlyTGV2R3hRdm5ib0FxdC9wUGxNbHhRYzFj?=
 =?utf-8?B?WjZWUmdYRXRDVlJKcGtoUS92c0JScFhFVUpyK2lVVDZ5OHZOQ1JpL1laY0xP?=
 =?utf-8?B?aS93SVJydldwRHQwN1ZveisvcDBKL1dlTWJua1NtWVEzNTVlbHFidmlLZ09X?=
 =?utf-8?B?dTBkVk5oeWV2bENPV253WTBFNDJLN3hEUlRDY1pwODdjOEVUeVhEUGRYSUVv?=
 =?utf-8?B?UG1xUzlsVlNhQUxiT0xpemlxR0RjaE1wSzNNUVQ4YU1VY0ZNd0U3OGVpNkNW?=
 =?utf-8?B?MTVFSkM4MHFRRnduTHJkUDBJV1AyUE1LQjJqdUhlRXY5YWppU1RGOERBdW1p?=
 =?utf-8?Q?mhFC+fbTYs4nEm/YvK1xiuMntWC7eVoAt80CWua?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8977a94f-4175-4f22-c12c-08d971d60880
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:03:53.0520
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hu1TXXxQgUX4yQAGIMWGiU4KgfBJPrRgpDHHA/N4HonWQ+5VX3Qc76VmIPJM98sFlCaOG+piGbGNKzSKuFvDOA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6301

On 06.09.2021 22:47, Elliott Mitchell wrote:
> On Mon, Sep 06, 2021 at 09:52:17AM +0200, Jan Beulich wrote:
>> On 06.09.2021 00:10, Elliott Mitchell wrote:
>>> I brought this up a while back, but it still appears to be present and
>>> the latest observations appear rather serious.
>>>
>>> I'm unsure of the entire set of conditions for reproduction.
>>>
>>> Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
>>> this is an older AMD IOMMU).
>>>
>>> This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
>>> Debian's patches, but those are mostly backports or environment
>>> adjustments.
>>>
>>> Domain 0 is presently using a 4.19 kernel.
>>>
>>> The trigger is creating a HVM or PVH domain where memory does not equal
>>> maxmem.
>>
>> I take it you refer to "[PATCH] x86/pod: Do not fragment PoD memory
>> allocations" submitted very early this year? There you said the issue
>> was with a guest's maxmem exceeding host memory size. Here you seem to
>> be talking of PoD in its normal form of use. Personally I uses this
>> all the time (unless enabling PCI pass-through for a guest, for being
>> incompatible). I've not observed any badness as severe as you've
>> described.
> 
> I've got very little idea what is occurring as I'm expecting to be doing
> ARM debugging, not x86 debugging.
> 
> I was starting to wonder whether this was widespread or not.  As such I
> was reporting the factors which might be different in my environment.
> 
> The one which sticks out is the computer has an older AMD processor (you
> a 100% Intel shop?).

No, AMD is as relevant to us as is Intel.

>  The processor has the AMD NPT feature, but a very
> early/limited IOMMU (according to Linux "AMD IOMMUv2 functionality not
> available").
> 
> Xen 4.14 refused to load the Domain 0 kernel as PVH (not enough of an
> IOMMU).

That sounds odd at the first glance - PVH simply requires that there be
an (enabled) IOMMU. Hence the only thing I could imagine is that Xen
doesn't enable the IOMMU in the first place for some reason.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:05:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:05:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180558.327270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW76-0000BA-9r; Tue, 07 Sep 2021 08:05:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180558.327270; Tue, 07 Sep 2021 08:05:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNW76-0000B3-5c; Tue, 07 Sep 2021 08:05:52 +0000
Received: by outflank-mailman (input) for mailman id 180558;
 Tue, 07 Sep 2021 08:05:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0UdZ=N5=arm.com=Hongda.Deng@srs-us1.protection.inumbo.net>)
 id 1mNW75-0000Av-7L
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:05:51 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe1e::619])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 292e35d6-c6f2-425d-8dec-2a84e5adcba3;
 Tue, 07 Sep 2021 08:05:48 +0000 (UTC)
Received: from DB6PR07CA0175.eurprd07.prod.outlook.com (2603:10a6:6:43::29) by
 AM9PR08MB5985.eurprd08.prod.outlook.com (2603:10a6:20b:286::5) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.17; Tue, 7 Sep 2021 08:05:46 +0000
Received: from DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:43:cafe::fa) by DB6PR07CA0175.outlook.office365.com
 (2603:10a6:6:43::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6 via Frontend
 Transport; Tue, 7 Sep 2021 08:05:46 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT011.mail.protection.outlook.com (10.152.20.95) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 08:05:46 +0000
Received: ("Tessian outbound 610b9df843d5:v105");
 Tue, 07 Sep 2021 08:05:46 +0000
Received: from 66fdde6d019e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3E15A6A9-9FC5-49EC-A7C7-AC7D04436CE8.1; 
 Tue, 07 Sep 2021 08:05:39 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 66fdde6d019e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 08:05:39 +0000
Received: from VE1PR08MB5677.eurprd08.prod.outlook.com (2603:10a6:800:1ab::17)
 by VI1PR08MB4190.eurprd08.prod.outlook.com (2603:10a6:803:eb::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 08:05:33 +0000
Received: from VE1PR08MB5677.eurprd08.prod.outlook.com
 ([fe80::d810:52a3:dce3:4d06]) by VE1PR08MB5677.eurprd08.prod.outlook.com
 ([fe80::d810:52a3:dce3:4d06%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 08:05:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 292e35d6-c6f2-425d-8dec-2a84e5adcba3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QV8b/SNLW0Tjhqdk11/RrqSAJCoIZ01fAAKzO7ivC+4=;
 b=Llu9plT9+lgVEzSlrojJUGwGwaI5QRZTkxTJkfoa5QbN7uDYqNCTDhA5FJGLLIOU4suO0j5cns+va0KmqeoXF90Nj0GNmT/rDr5RNjw/17jhtagamC5Km6j0AQ7zlCdff7B4JGJL3CxHKNpypPGhhgGmEbrhfjeEcwL6NXVt6qc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QMRYR7P8r53exVm7qnHJMIZp3PJQPhNaNQCa9oxXrU0CaR8ohsxdtewRTm2WKt6vnJRCYTXMDBJcws7KK8k9eKGzH2xys34utm2CDCYuFWWt7NXyGgyGYm+Gaqagx3451yTe4qoOG2086NbAtl343OLEPmz2dWuXRtNkbd50HSh3FKlv6nJ+D7I2543PUVKMsuAEcVUInLep3mChy0jSQk2LmcAKI3BIQzAdlAmFa+vDKi3kplYZHmba3/b2HVx/hUnqj6FNW9FkePmT2j6/sOPiF7ODnlgdqNnreQuUndr8tN6DTa/zQ7h0LdopXH5jWDSG0qMkSFqCiVQhh5p0ig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QV8b/SNLW0Tjhqdk11/RrqSAJCoIZ01fAAKzO7ivC+4=;
 b=PDxu0c5eqWgrZuvPAVWn4NdCZ6ytlBPiZKsOTxlz3sY4aMxwNXW7nZuzbkhhgMoEX05SnguB7b0zFG27MrS1rF9yiLcgIxURQTe1/gG3Gu3faCT2pNdEoK+20U1e29h55CdW+T9k1X6rShGxHrDDdcinL1WmCu/jJsyzdSST4KvUGYVHCRH6uGNSQuzdeOsWoT6GfLUF+7Ljh8y4E4dFgbB7f7/bR7RbO5//FfrUoMIBEHkOo+IHXbwTIQTh295zc2hcU3SuU7b3aSFpP5mxLLG3AR54LaeQCdKAEd6kVRKwRIS80SfQ/UE+52bNADmBG/3aAIMMeyPthfeI2aZA7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QV8b/SNLW0Tjhqdk11/RrqSAJCoIZ01fAAKzO7ivC+4=;
 b=Llu9plT9+lgVEzSlrojJUGwGwaI5QRZTkxTJkfoa5QbN7uDYqNCTDhA5FJGLLIOU4suO0j5cns+va0KmqeoXF90Nj0GNmT/rDr5RNjw/17jhtagamC5Km6j0AQ7zlCdff7B4JGJL3CxHKNpypPGhhgGmEbrhfjeEcwL6NXVt6qc=
From: Hongda Deng <Hongda.Deng@arm.com>
To: Julien Grall <julien@xen.org>, Wei Chen <Wei.Chen@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, nd <nd@arm.com>
Subject: RE: About Arm guests accessing to GICD_ICPENR
Thread-Topic: About Arm guests accessing to GICD_ICPENR
Thread-Index: AdejvyF8C7fiPQEXRRSQQzNI/V0p1g==
Date: Tue, 7 Sep 2021 08:05:33 +0000
Message-ID:
 <VE1PR08MB567729BD416FCCD32FCBD6CCE6D39@VE1PR08MB5677.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 3A117852C6EAA740B0A5374012329AE5.0
x-checkrecipientchecked: true
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: fbca13c2-b529-4a44-5c46-08d971d64c2f
x-ms-traffictypediagnostic: VI1PR08MB4190:|AM9PR08MB5985:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB5985AC5CE5FE087D546D2F96E6D39@AM9PR08MB5985.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NYCTz5HY7d7u6eKzrsty7ns6ESFZArVEhYi4/34rmo7sWqQPaPlvdFExUqajvAcQwVT9B4Q+mNegtD0TtJ9VQn/OeAdNsB+ZHQyJdw4EPUT05qhd1ShGsYCvSmgUqeJA67c4gbnUk1QZDRPb8FFAuJratTkWyp2qIV1Pc/tIYOTonDf/b7+xl6ytM6EWaJqaS5oytedaOL+0KiM2LEGoazkmWndSHOkGyLsxUZgsc4cnMsYwv/7bueXuZKVSnR8QqKkgKrTU9GXSxNGTWvFowjbaimINtF+ARmgVyLK0ZQGLRIvn0lyrlJC5umivqRA48z4m3v9TMROXxHV5VInhsdOIq/OdEVjttD4Au1ywC2N5A3OHrhDh+2JPbO4nSm+1lTa/8K7z0dySzzI4TUzaR5Hz7owMxRQsa/BWbfENyFa4RpAYgzWhr2JRdvv+CM7Px+LimO9UApTrX+NTFgbtURUtrusJYdYpEPSIVio3FXw6jQhhAB9Ometgnz9q9YFNRTMJaIMUkG+WdtAuTqB8ZQbvV96OkOORDRBLk8FoDzsqJMQX2DaebJXpseM38Km/0wW2qCVUGtq6rjn0vu9W8kIJ474Zdn6KNRLTpdUBojj3GxHlRAb4WNdEydwzfkpMY4FazGSa9ubW9u3MgqEVp9KXuYMG+hC5fXAKa4Euzk50conuNqyzgy7mqm0drE57xJGdRrckd6eYHu864aYH9AEzxnosKiSh8iYPPgQNhFWu1gX+MFlC/dGG9P6xjulXDqKuV483cu3i2iQ/taOaBGdeMz6rBnx2+Hin3GQU/S4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5677.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(346002)(136003)(366004)(376002)(52536014)(66446008)(66556008)(66476007)(5660300002)(66946007)(33656002)(64756008)(122000001)(76116006)(38100700002)(7696005)(478600001)(2906002)(186003)(8676002)(26005)(83380400001)(54906003)(316002)(8936002)(38070700005)(55016002)(86362001)(71200400001)(4326008)(966005)(110136005)(6506007)(9686003)(53546011);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4190
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	da997a58-c9b1-4fa1-10a5-08d971d64480
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NjFyaEp0ZKDtV0TSlR54FyaOyuysWTsrLU8k2m1exzMsxeoZEtZ3/2Be6Ku7IWQfavD7rinXN0JYtmZ3ydyIRefL5Dw1jJ0b9A+kI3eQIzxGoqrk8bQszfZl8x0p2Dc4oDYQp4mornSQmmqQcnU8YmLhssLzTq6t1RIGu1Ary5qztpUk0DyE/OLD4AW7yVSP+ycV48aF6GOCL0f6j64zZgxwUmvk6PQ4x8iI5tQvkWupVTod8sGxdmN3st/mMRYrRVEnKo5EruqSmO/GkPMj2N3vn1jiZTNwl2iAyv5xgN3a7PxcuG8InwavqCEVwbSSYh1dZSWBrCnPZj7ZU3C2zKCedJQN7hnlejF5Oi2VNCOkiCkxhzZG4LTOn39X550P/6QSaknbG2VIBDGhwMf1HMhiwfHLOS9KFqlGORZzGIArxrx7uP+aAN5o9AV7r5xd32ZoNNM5OFdx2t5nPUduqlWk/B61/QHR7VhvFiaynNLLVZF6cHBVErU6fdwdKePYafYkohT05f/M9rc/1hB8CiuG/ds1MFdJVHowdRU7ED1xlBORr3MOaQGy0avLmqc9AhWMvvMMLwgxgKyNFU0gKAbi2sRbOwF47Pu39Bk/ddgWs4GfmVW1xxybTwo+LwYatnDhZq1Gs86AB3xluNL6h2Krr7UIvf3fP8OY+e9sLy8l9qlBueWnqFbVpAJSijXAl3fUAuireIkXmH77Jr//NQitmICsXTtexs5on2DqK/FDNDFgcUaB9Ksnc/nA4ZJOvBXMlsHmU8AxTH8ZVK85zr36/dkAZBa4vt7rFN/I7z4=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(376002)(136003)(396003)(46966006)(36840700001)(7696005)(82740400003)(52536014)(82310400003)(8676002)(47076005)(54906003)(33656002)(53546011)(6506007)(81166007)(336012)(356005)(5660300002)(36860700001)(70206006)(83380400001)(2906002)(110136005)(9686003)(966005)(316002)(26005)(186003)(86362001)(4326008)(55016002)(478600001)(8936002)(70586007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:05:46.4678
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fbca13c2-b529-4a44-5c46-08d971d64c2f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB5985

SGkgSnVsaWVuLA0KDQo+IE9uIDA2LzA5LzIwMjEgMTA6MDQsIEhvbmdkYSBEZW5nIHdyb3RlOg0K
PiA+IEhpIEp1bGllbiwNCj4gDQo+IEhpIEhvbmdkYSwNCj4gDQo+ID4gWGVuIHByb3ZpZGVzIHZH
SUMgdG8gc3VwcG9ydCBYZW4gZ3Vlc3RzLCBhbmQgY3VycmVudGx5IHhlbiB3aWxsIHJldHVybiBJ
Tw0KPiA+IHVuaGFuZGxlZCB3aGVuIGd1ZXN0cyBhY2Nlc3MgR0lDRCBJQ1BFTlJuIHJlZ2lzdGVy
cy4gVGhpcyB3b3JrcyBmaW5lIHdpdGgNCj4gTGludXgNCj4gPiBndWVzdHMsIGZvciBMaW51eCB3
b24ndCBhY2Nlc3MgdGhlc2UgcmVnaXN0ZXJzLiBCdXQgZm9yIFplcGh5ciwgdGhpcyBtZWNoYW5p
c20NCj4gPiB3aWxsIGNhdXNlIElPIGRhdGFhYm9ydCBvbiBaZXBoeXIncyBpbml0aWFsaXphdGlv
biB3aGljaCBtYWtlcyBaZXBoeXIgZ2V0IGluDQo+ID4gZmF0YWwgZXJyb3IuDQo+ID4NCj4gPiBP
bmUgbWV0aG9kIHRvIGVhc2UgdGhpcyBpcyB0byBsZXQgdkdJQyBpZ25vcmUgR0lDRCBJQ1BFTlJu
IHJlZ2lzdGVycyBhY2Nlc3MuIEkNCj4gPiB0ZXN0ZWQgaXQgd2l0aCBMaW51eCBndWVzdHMgYW5k
IFplcGh5ciBndWVzdHMsIGFuZCBib3RoIHdvcmtzIGZpbmUuIEFuZCBJIGZvdW5kDQo+ID4gaW4g
dGhpcyBwYXRjaFsxXSB0aGF0IGl0IHdvdWxkIGJlIG1vcmUgY29tcGxleCB0byB0b3VjaCB0aGUg
cmVhZCBwYXJ0IGZvcg0KPiA+IEl7UyxDfVBFTkRSIGFuZCB0aGUgd3JpdGUgcGFydCBmb3IgSUNQ
RU5EUiwNCj4gDQo+IFJlYWQgdG8gSXtTLEN9UEVORFIgc2hvdWxkIGFscmVhZHkgcmV0dXJuLiBB
RkFJSywgd2hhdCdzIGxlZnQNCj4gdW5pbXBsZW1lbnRlZCBpcyB3cml0ZSB0byBJQ1BFTkRSLg0K
PiANCj4gPiBzbyBjb3VsZCB3ZSBpZ25vcmUgR0lDRCBJQ1BFTkRSbg0KPiA+IHJlZ2lzdGVycyBh
Y2Nlc3MgdG8gZWFzZSBaZXBoeXIncyBpbml0aWFsaXphdGlvbiBwcm9ibGVtPw0KPiBXb3VsZCB5
b3UgYmUgYWJsZSB0byBwcm92aWRlIG1vcmUgaW5mb3JtYXRpb24gb24gaG93IFplcGh5ciBpcyB1
c2luZyBpdD8NCg0KWmVwaHlyIHdpbGwgdHJ5IHRvIGNsZWFyIHBlbmRpbmcgc3RhdGUgYXQgaXRz
IGluaXRpYWxpemF0aW9uLCBhbmQgdGhlIGNvZGUgY2FuIGJlDQpmb3VuZCBoZXJlWzFdLg0KDQo+
IA0KPiA+DQo+ID4gSWYgWGVuIHdhbnRzIGEgY29tcGxldGUgR0lDRCBJQ1BFTkRSbiBlbXVsYXRp
b24gdG8gZml4IGl0LCBkbyB5b3UgaGF2ZSBhbnkNCj4gPiBzdWdnZXN0aW9ucy4NCj4gDQo+IEVt
dWxhdGluZyBJQ1BFTkRSIGlzIG5vdCBlYXN5IHdpdGggdGhlIGV4aXN0aW5nIHZHSUMuIEl0IHdv
dWxkIGJlIGdyZWF0DQo+IHRvIGZpbmFsbHkgaGF2ZSBhIHZHSUMgc3BlYyBjb21wbGlhbnQsIGJ1
dCBJIGFsc28gYXBwcmVjaWF0ZSB0aGlzIGlzDQo+IGdvaW5nIHRvIHRha2Ugc29tZSB0aW1lLg0K
DQpZZXMsIEkgYWdyZWUgd2l0aCB0aGF0Lg0KDQo+IA0KPiBJZ25vcmluZyB0aGUgYWNjZXNzIGlz
IHByb2JhYmx5IHRoZSBiZXN0IHdlIGNhbiBkby4gSG93ZXZlciwgdGhpcyBpcw0KPiBhbHNvIGEg
cmlza3kgYXBwcm9hY2ggYmVjYXVzZSBaZXBoeXIgKG9yIGFub3RoZXIgT1MpIG1heSBnZW51aW5l
bHkgd2FudA0KPiB0byBjbGVhciBhbiBhbHJlYWR5IHBlbmRpbmcgaW50ZXJydXB0LiBTbyBJIHdv
dWxkIHN1Z2dlc3QgdG8gd2FsayB0aGUNCj4gaW50ZXJydXB0cyB0aGF0IGFyZSBtZWFudCB0byBi
ZSBtb2RpZmllZCBhbmQgY2hlY2sgd2hldGhlciB0aGV5IGFyZQ0KPiBhbHJlYWR5IHBlbmRpbmcu
IElmIHRoZXkgYXJlIHRoZW4gcHJpbnQgYSBtZXNzYWdlLg0KDQpPSywgSSB3b3VsZCBhZGQgdGhp
cyBjaGVjayBhbmQgd2FybmluZyBtZXNzYWdlIHRvIG15IGN1cnJlbnQgcGF0Y2ggdG8gZml4IGl0
IA0KaW4gY3VycmVudCBzdGFnZS4gSXMgaXQgT0s/DQoNCj4gDQo+IFRoaXMgd291bGQgbWFrZSBj
bGVhciB0byB0aGUgZGV2ZWxvcHBlciB0aGF0IHNvbWV0aGluZyBtYXkgZ28gd3JvbmcNCj4gbGF0
ZXIgb24uDQo+IA0KPiBDaGVlcnMsDQo+IA0KPiAtLQ0KPiBKdWxpZW4gR3JhbGwNCg0KWzFdIGh0
dHBzOi8vZ2l0aHViLmNvbS96ZXBoeXJwcm9qZWN0LXJ0b3MvemVwaHlyL2Jsb2IvZWFmNmNmNzQ1
ZGYzODA3ZTZlDQpjYzk0MWMzYTMwZGU2YzE3OWFlMzU5L2RyaXZlcnMvaW50ZXJydXB0X2NvbnRy
b2xsZXIvaW50Y19naWN2My5jI0wyNzQNCg0KQ2hlZXJzDQpIb25nZGENCg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:11:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:11:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180566.327280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWCc-0001ui-0l; Tue, 07 Sep 2021 08:11:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180566.327280; Tue, 07 Sep 2021 08:11:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWCb-0001ub-Tw; Tue, 07 Sep 2021 08:11:33 +0000
Received: by outflank-mailman (input) for mailman id 180566;
 Tue, 07 Sep 2021 08:11:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNWCa-0001uV-3l
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:11:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0fd28f35-4acf-4b82-a357-48ad7d4a734c;
 Tue, 07 Sep 2021 08:11:30 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-WNOwULU5P1-pdkk2dJTp-Q-2; Tue, 07 Sep 2021 10:11:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4848.eurprd04.prod.outlook.com (2603:10a6:803:55::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 08:11:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:11:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0107.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 08:11:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fd28f35-4acf-4b82-a357-48ad7d4a734c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631002289;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NL5+XZXMZuyGIIbtbG3QIPRva4YiO/MGTh6ZA1O4+5Q=;
	b=LZHnLs+0psfwctVP47zmDYmB6CBiSh5PeGQwksJZ6J1VyK7FBMEK+F61DJaChXXzF//Pyk
	AwhEvnMgiIVwYq8c22Jsg+2JhEsH2XObER7L6IhNcwp3JnEIs+ZNNkhV81vdg88+e6XLt6
	UsPqY4LBPfcgJGXdgSSQMjp8nZ/PAHs=
X-MC-Unique: WNOwULU5P1-pdkk2dJTp-Q-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HYDgMW6ar0nnsQeWaPyczkp4e9EnuxWFXNZYiYnE/o+cKh5jTEVXZ+cLWFB9AO+D80I1GVKLijyrNA0fX7rMfehTgegpgmhntQ+FW3qmj77BEg8vkHh7fVS99Qj86oo8PsXRs35N0rXLI+A6waZ4qeTJAVwmM4QdchYrCl5JnFhYD9J8h84YMxbBGAtgKYiIwoPN1jmz6OOCeSVNLnJLWWeHEZQuAkwYOn1PvsQyL6yG7iudk2hoyCOGaoaftvmzU6x3mwaOoejsUGKtf+NuyKhtszYG3QU1XJh5UoM7I21W5ZF2Bl4L3v8cuei7wDFF4kQSaiN3fLRH9POuJ4ZLOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=95mzxOnnCf3gr5z3HF9NfKZisG77yTQkq6d66SJJ3zc=;
 b=hShkfj+tuwxIFAK3UUaOvGm0sMKwF9RJqGCjFst8IoXM3pi76pihBz6hBNEoh0FkO1DoYX3SbIRuYyfctrpDqMtnpl8MgUyhGnBMKWQ4ZX5TiK0YTd6fNyO7TFCVShdg0jgDh3wc9zsqrCNQADMrjkD+j913l1nCXmP2ioUs56z2G71lUzyGzE1MUflD/6CJGHx+SZomRN+9dFVBPKF94QnoCGc4yUymj3o7RELQPj8l8Lj8X7AtXi7IK+vTRtfXdH7a+BoSC7zdPR/O7U0Y/dwWA1AO/b4NG58DriIirkOAViXJNejY/b+IPqP566m4hRIA0cARLMJQjJnlcoGmsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: eikelenboom.it; dkim=none (message not signed)
 header.d=none;eikelenboom.it; dmarc=none action=none header.from=suse.com;
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: Juergen Gross <jgross@suse.com>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
 <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Sander Eikelenboom <linux@eikelenboom.it>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <36e8dec0-e777-a4d1-de8a-8991d95407cb@suse.com>
Date: Tue, 7 Sep 2021 10:11:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0107.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e3344e7c-dcb5-468a-6cc6-08d971d716c3
X-MS-TrafficTypeDiagnostic: VI1PR04MB4848:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB48483C381199B09E09529119B3D39@VI1PR04MB4848.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YKAV0qSviyZeu0E2pe6klOFT9CbT8K8sTd8snsUZIdQ3VAEM1hvwWQfacymFxlDHSsW18TT0tYSOycySklJjPwJ+Sfz+9TuWx4Kbbt1awzT9RtVpd6eSYe/Yj1Y9Jv/MmOYkYN5hf8QU78sSJ6iA2iaMZ0IttvnpF3pRAYzVu4+rbR2FvRG8ZwYMhV2lBEdO8S61MII9lYUCWBZoyghJpPcUE1cmn6ogAXSKv6tgw7vlyofTEmKCTuCn0rw0xnbCsZ5S8/au00jA8kqqEl4wDany8Us/QFIsanbQuAVuMornBE2eUtCTiddWtlxfNzsQRABsoyzhu9oNTdEIzlhCPXuhVV+qTY76PIvS5oSg2lcElx6ihWPeDTXDH768Pnll0bBQBKZRg1SgA44wtrjxUdW60qP/uTZYm4R2VqIKaAcX+mnE12K+tqJyviO0DSlFIbY6YW0IzeCNRDXA8GVSrqDsygqp3ume/Jp+ohyLXwZr/GmBkvqxKi+oBw/6PixnMfY8PQwiXQlEopi/xcPGDw0aH7I6TWHcpSILvwpaob4zl/M868j9EdqRHzXNl+5n9AOxSpQMQMYRxb9C3pJbuHRIlJU6GsrZVdANW43fBR69b9UxrKmQtDDqDs3HJrVBv6knWaVoixTzO7n9SbVWI8aV+R4Sf5O0kp2lMozur91GoxOwo9fY1oB+6/byeNuONellYK+sYuyobN2RWb8TEU88pYGm80jGUsKWb1MOAK6rP8i4KlNfR7MiXSmQgbOt
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(366004)(346002)(376002)(396003)(39860400002)(6636002)(31686004)(26005)(8676002)(53546011)(4326008)(2906002)(186003)(38100700002)(5660300002)(6862004)(478600001)(86362001)(66946007)(66476007)(66556008)(83380400001)(54906003)(956004)(6486002)(16576012)(31696002)(45080400002)(37006003)(36756003)(316002)(8936002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?334//3fKJiwriJdtQBrBaZTrLKEkUyDtivoQuwpw/KB8Yues/I4IhDab6xaQ?=
 =?us-ascii?Q?2UYYpIIGVqxiDL8xaRKiesqYxCqvv2Lutohikoce6VlLmCqSiWYgTbS1iaB3?=
 =?us-ascii?Q?I+iXdOMqsx+C4fKcKUrs9cFHpXFYED2Eyl/VoM/c5BFg5dMTigUtTGq5h7NL?=
 =?us-ascii?Q?9z9T+HQGFvCly/RZKEPFGyAzbi1n14qmM8tz6ecgYl5rOfUBgmk6WeENIxld?=
 =?us-ascii?Q?ac3TFQSA+GXv3jFI9U8u4Oix3izdrKDXXxcPNQtAZkbmKlp9+0adztLCiKLg?=
 =?us-ascii?Q?GTW6eWqrMEKffZsMJV9ntWQpl22Guyh8VtZ6TJDjdf/rMTqNBbt6+77vuLgz?=
 =?us-ascii?Q?TeFrtgIIMnrE+sMCHKLnt07JgYTjtI4lCqMr0fo++pLg8yk7g+c4m8Rfuy1x?=
 =?us-ascii?Q?W9KHDhnT+WSwBcU1Qx6CK1v36QlqDcDBn6qNTfYk7/VBLzbNyTMtaKRfovHr?=
 =?us-ascii?Q?Vyhi6p98enDK4GgKu6yjXAMKsBzJNhB8fFDxsuhWI6FV6pqje5C1fMuzSq/p?=
 =?us-ascii?Q?W1ATr8bTiG+7y8e6OR9wlFypw5rT5Bvmzy/BPvnL1kincBgPffl3Ov7WO80/?=
 =?us-ascii?Q?WkECkUBUk0EbiFAbx7/Sz0m0M6BUHI38isZNR+BSzZ8mu/wYYraZ5j+jige3?=
 =?us-ascii?Q?zxqoEtsJHIQl71rqKkalOGsWZpLqrsGJkJ4WBalEM8Hf/ME5LdcMHZ82Vedn?=
 =?us-ascii?Q?floygQPq20r8YJHAplendf3zIm+fPLGAQbvet3J69AcQz+fDBXLOEgwq8ReY?=
 =?us-ascii?Q?VmnzRpoFie6SXMgVz/yRl00fF4aI0vgKdmAjMduXZju7YfSu9ay/P/QDPQQx?=
 =?us-ascii?Q?THfsbx9iCORpfwyPrFY0ouWHZptwhAgeUP8rtlO58SXrxyMeIFjzQPdVVgQr?=
 =?us-ascii?Q?6Bg17upT/X+388HUsOA5rL3y5zft+MyMP3hvk7ueQuEQ54Vp8nWGYEgbQDxO?=
 =?us-ascii?Q?Pd6LrZVp/Hcgk+TkUawmnp9H/Qq/BtG54qds4ax5hRzlgZv1/gQnOnYei0LM?=
 =?us-ascii?Q?waPxh3YYq4ffwaTPpYo5b4WK0K6gBEoNlcPKJrttGrF3LUUok6//qmOFb6S1?=
 =?us-ascii?Q?FOOrdXtW8YY5RYgRgS3DxMgpg22UYKsOj2rWpMnZSdvrDg/S8auIfaRoH2n3?=
 =?us-ascii?Q?rl/TaIDdVSJDhVEyp08P/MWoAcfbSFXYy2NG8+enyRtWQlwDaymGuzkhOztA?=
 =?us-ascii?Q?88jBwHZdlDT/QQReLVwwdEIqrzfANNdH8F9qYVuCa/KIRavWtpR4P0P0UcYZ?=
 =?us-ascii?Q?6y+rEW9YvbH0tGML0lH4HlYDfHwcRwk0WPNwhk+pf1d21/J1W1rpp4qR+WeV?=
 =?us-ascii?Q?ItZtnDGFyUx8Wic9x5KjYTZA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e3344e7c-dcb5-468a-6cc6-08d971d716c3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:11:26.5288
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +aPmfOsaRHhEX/E8tYL8pmiotLzZVpmxU1L3DBNTepnZvNmOVdB7vgOrPbhPQozXuVSWn6ZTRmCJpvQqc9yhTg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4848

On 07.09.2021 09:58, Juergen Gross wrote:
> On 06.09.21 23:35, Sander Eikelenboom wrote:
>> L.S.,
>>
>> On my AMD box running:
>>  =C2=A0=C2=A0=C2=A0 xen-unstable changeset: Fri Sep 3 15:10:43 2021 +020=
0 git:2d4978ead4
>>  =C2=A0=C2=A0=C2=A0 linux kernel: 5.14.1
>>
>> With this setup I'm encountering some issues in dom0, see below.
>>
>> --=20
>> Sander
>>
>> xl dmesg gives:
>>
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63b936 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a0622 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6b63da already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 638dd9 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 68a7bc already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63c27d already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a04f2 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 690d49 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6959a0 already pinned
>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a055e already pinned
>> (XEN) [2021-09-06 18:15:04.090] mm.c:3506:d0v0 mfn 639437 already pinned
>>
>>
>> dmesg gives:
>>
>> [34321.304270] ------------[ cut here ]------------
>> [34321.304277] WARNING: CPU: 0 PID: 23628 at=20
>> arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
>> [34321.304288] Modules linked in:
>> [34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted=20
>> 5.14.1-20210906-doflr-mac80211debug+ #1
>> [34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)=C2=A0 , =
BIOS=20
>> V1.8B1 09/13/2010
>> [34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
>> [34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60=
=20
>> 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00=
=20
>> 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
>> [34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
>> [34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX:=20
>> ffff88807d416398
>> [34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI:=20
>> deadbeefdeadf00d
>> [34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09:=20
>> ffff888006160cc0
>> [34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12:=20
>> 0000000000000000
>> [34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15:=20
>> 0000000000000001
>> [34321.304320] FS:=C2=A0 00007f5071177800(0000) GS:ffff88807d400000(0000=
)=20
>> knlGS:0000000000000000
>> [34321.304322] CS:=C2=A0 10000e030 DS: 0000 ES: 0000 CR0: 00000000800500=
33
>> [34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4:=20
>> 0000000000000660
>> [34321.304326] Call Trace:
>> [34321.304331]=C2=A0 xen_alloc_pte+0x294/0x320
>> [34321.304334]=C2=A0 move_pgt_entry+0x165/0x4b0
>> [34321.304339]=C2=A0 move_page_tables+0x6fa/0x8d0
>> [34321.304342]=C2=A0 move_vma.isra.44+0x138/0x500
>> [34321.304345]=C2=A0 __x64_sys_mremap+0x296/0x410
>> [34321.304348]=C2=A0 do_syscall_64+0x3a/0x80
>> [34321.304352]=C2=A0 entry_SYSCALL_64_after_hwframe+0x44/0xae
>> [34321.304355] RIP: 0033:0x7f507196301a
>=20
> I can see why this failure is occurring, but I'm not sure which way is
> the best to fix it.
>=20
> The problem is that a pinned page table is moved: the pmd entry
> referencing it is cleared and a new reference is put into the pmd.
> This is done by getting the old pmd entry, clearing that entry, and then
> using pmd_populate() to write the new pmd entry. pmd_populate() will
> lead to a call of xen_pte_alloc() trying to pin the referenced page
> table, which is failing, as it is already pinned.
>=20
> The problem has been introduced by commit 0881ace292b662d2 in kernel
> 5.14.
>=20
> Following solutions would be possible:
>=20
> 1. When running as PV guest skip the optimization of move_pgt_entry()
>     by letting arch_supports_page_table_move() return false. This will
>     result in a performance drop in some cases.
>=20
> 2. Unpin the page table before calling pmd_populate(). This adds some
>     unneeded hypercall and without flushing the TLB I'm feeling uneasy
>     to do that.

I agree as far as the "unneeded hypercall" aspect goes, but I don't
see any connection to the TLB (or a need to flush it): Pinning has
nothing to do with insertion into a live page table; a pinned page
table can be entirely free floating. It's the removal from a
(possibly) live page table which would require a flush.

> 3. Add a check in xen_pte_alloc() if the page table is pinned already
>     and if this is the case, don't do the pinning. This is a rather clean
>     solution, but will result in other failures if a page table is used
>     multiple times (this case would be caught today as in the failure
>     above).
>=20
> My tendency is towards solution 3 as it is local to Xen code and has the
> best performance.

I agree 3 looks most promising. I can't judge how big of a risk
there is for a page table to get used in more than one place, and
hence how important it is to be able to detect that case.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:16:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:16:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180572.327292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWGw-0002ZH-Jx; Tue, 07 Sep 2021 08:16:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180572.327292; Tue, 07 Sep 2021 08:16:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWGw-0002ZA-EA; Tue, 07 Sep 2021 08:16:02 +0000
Received: by outflank-mailman (input) for mailman id 180572;
 Tue, 07 Sep 2021 08:16:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xlGX=N5=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNWGv-0002Yy-86
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:16:01 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d545c744-0fb3-11ec-b0da-12813bfff9fa;
 Tue, 07 Sep 2021 08:15:59 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 02C2B1FD3E;
 Tue,  7 Sep 2021 08:15:59 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id D17FB132AB;
 Tue,  7 Sep 2021 08:15:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id zpc8L74fN2FKHgAAGKfGzw
 (envelope-from <jgross@suse.com>); Tue, 07 Sep 2021 08:15:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d545c744-0fb3-11ec-b0da-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631002559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=f6zXc+LSbfebUefAjRoXqLW7t85EtxUizUQ2aQ73x/0=;
	b=gzbyRSyonhBjA+jvjd6JqRiJKCTbqKUWaChVq35WHoHLjJpnUn7x7SM//07TPvom7T2Jx8
	zaRVytzbgMBHNS/pf/1LvLEqs0+Y1aTZ/CAdNLDl9HksxD7b0oJVeC3z/1knuk2VVacpKe
	yDeDwJ+/jZs/Y+xulyUPyitGkMdxsY8=
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Sander Eikelenboom <linux@eikelenboom.it>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
 <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
 <36e8dec0-e777-a4d1-de8a-8991d95407cb@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <96b25976-4109-8aa2-b763-f00cfb1c20f7@suse.com>
Date: Tue, 7 Sep 2021 10:15:58 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <36e8dec0-e777-a4d1-de8a-8991d95407cb@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="rkLvAjn7JyDnXoy3ThW9ioqDZPHBBuMI3"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--rkLvAjn7JyDnXoy3ThW9ioqDZPHBBuMI3
Content-Type: multipart/mixed; boundary="s002Bm9I1HUjHEKtUbAXCiPi5Qbn6KY6W";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Sander Eikelenboom <linux@eikelenboom.it>
Message-ID: <96b25976-4109-8aa2-b763-f00cfb1c20f7@suse.com>
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
 <0ae9bb1e-dacc-a3d2-ed90-f85cb52c698d@suse.com>
 <36e8dec0-e777-a4d1-de8a-8991d95407cb@suse.com>
In-Reply-To: <36e8dec0-e777-a4d1-de8a-8991d95407cb@suse.com>

--s002Bm9I1HUjHEKtUbAXCiPi5Qbn6KY6W
Content-Type: multipart/mixed;
 boundary="------------D51AD51E6B56470FE6CB0F0C"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------D51AD51E6B56470FE6CB0F0C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 10:11, Jan Beulich wrote:
> On 07.09.2021 09:58, Juergen Gross wrote:
>> On 06.09.21 23:35, Sander Eikelenboom wrote:
>>> L.S.,
>>>
>>> On my AMD box running:
>>>   =C2=A0=C2=A0=C2=A0 xen-unstable changeset: Fri Sep 3 15:10:43 2021 =
+0200 git:2d4978ead4
>>>   =C2=A0=C2=A0=C2=A0 linux kernel: 5.14.1
>>>
>>> With this setup I'm encountering some issues in dom0, see below.
>>>
>>> --=20
>>> Sander
>>>
>>> xl dmesg gives:
>>>
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63b936 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a0622 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6b63da already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 638dd9 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 68a7bc already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 63c27d already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a04f2 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 690d49 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6959a0 already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.089] mm.c:3506:d0v0 mfn 6a055e already pin=
ned
>>> (XEN) [2021-09-06 18:15:04.090] mm.c:3506:d0v0 mfn 639437 already pin=
ned
>>>
>>>
>>> dmesg gives:
>>>
>>> [34321.304270] ------------[ cut here ]------------
>>> [34321.304277] WARNING: CPU: 0 PID: 23628 at
>>> arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
>>> [34321.304288] Modules linked in:
>>> [34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted
>>> 5.14.1-20210906-doflr-mac80211debug+ #1
>>> [34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)=C2=A0=
 , BIOS
>>> V1.8B1 09/13/2010
>>> [34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
>>> [34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 =
60
>>> 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 =
00
>>> 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 6=
5
>>> [34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
>>> [34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX:
>>> ffff88807d416398
>>> [34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI:
>>> deadbeefdeadf00d
>>> [34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09:
>>> ffff888006160cc0
>>> [34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12:
>>> 0000000000000000
>>> [34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15:
>>> 0000000000000001
>>> [34321.304320] FS:=C2=A0 00007f5071177800(0000) GS:ffff88807d400000(0=
000)
>>> knlGS:0000000000000000
>>> [34321.304322] CS:=C2=A0 10000e030 DS: 0000 ES: 0000 CR0: 00000000800=
50033
>>> [34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4:
>>> 0000000000000660
>>> [34321.304326] Call Trace:
>>> [34321.304331]=C2=A0 xen_alloc_pte+0x294/0x320
>>> [34321.304334]=C2=A0 move_pgt_entry+0x165/0x4b0
>>> [34321.304339]=C2=A0 move_page_tables+0x6fa/0x8d0
>>> [34321.304342]=C2=A0 move_vma.isra.44+0x138/0x500
>>> [34321.304345]=C2=A0 __x64_sys_mremap+0x296/0x410
>>> [34321.304348]=C2=A0 do_syscall_64+0x3a/0x80
>>> [34321.304352]=C2=A0 entry_SYSCALL_64_after_hwframe+0x44/0xae
>>> [34321.304355] RIP: 0033:0x7f507196301a
>>
>> I can see why this failure is occurring, but I'm not sure which way is=

>> the best to fix it.
>>
>> The problem is that a pinned page table is moved: the pmd entry
>> referencing it is cleared and a new reference is put into the pmd.
>> This is done by getting the old pmd entry, clearing that entry, and th=
en
>> using pmd_populate() to write the new pmd entry. pmd_populate() will
>> lead to a call of xen_pte_alloc() trying to pin the referenced page
>> table, which is failing, as it is already pinned.
>>
>> The problem has been introduced by commit 0881ace292b662d2 in kernel
>> 5.14.
>>
>> Following solutions would be possible:
>>
>> 1. When running as PV guest skip the optimization of move_pgt_entry()
>>      by letting arch_supports_page_table_move() return false. This wil=
l
>>      result in a performance drop in some cases.
>>
>> 2. Unpin the page table before calling pmd_populate(). This adds some
>>      unneeded hypercall and without flushing the TLB I'm feeling uneas=
y
>>      to do that.
>=20
> I agree as far as the "unneeded hypercall" aspect goes, but I don't
> see any connection to the TLB (or a need to flush it): Pinning has
> nothing to do with insertion into a live page table; a pinned page
> table can be entirely free floating. It's the removal from a
> (possibly) live page table which would require a flush.

And this removal is happening:

         /* Clear the pmd */
         pmd =3D *old_pmd;
         pmd_clear(old_pmd);

         VM_BUG_ON(!pmd_none(*new_pmd));

         pmd_populate(mm, new_pmd, pmd_pgtable(pmd));

So unpinning after calling pmd_clear() seems to be risky.

>=20
>> 3. Add a check in xen_pte_alloc() if the page table is pinned already
>>      and if this is the case, don't do the pinning. This is a rather c=
lean
>>      solution, but will result in other failures if a page table is us=
ed
>>      multiple times (this case would be caught today as in the failure=

>>      above).
>>
>> My tendency is towards solution 3 as it is local to Xen code and has t=
he
>> best performance.
>=20
> I agree 3 looks most promising. I can't judge how big of a risk
> there is for a page table to get used in more than one place, and
> hence how important it is to be able to detect that case.

Thanks. I'm going that route then.


Juergen

--------------D51AD51E6B56470FE6CB0F0C
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------D51AD51E6B56470FE6CB0F0C--

--s002Bm9I1HUjHEKtUbAXCiPi5Qbn6KY6W--

--rkLvAjn7JyDnXoy3ThW9ioqDZPHBBuMI3
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE3H74FAwAAAAAACgkQsN6d1ii/Ey+R
2wf/T8k4DXCL0IjMFs5iazUnq4wLJn+r9VvSXMFGOPepylzZ6ZLtnB7VqXksgOtoz62aIVt0LUHC
rxR662Mno0CP4jbyONfxh+fxC3ijHsIylXp7bctAtEoEUTbrqdFDZGIaFadzUiGFGrvNr5H/2bUA
fwzqoF6wmKwmpb3vMYu/SaRS3x8Lfu6LgkZLby78LFfi34oVE/vTrHmo3TSVeygtIBA2kPgjd74g
wqfKoILPxujt4bYrR0Z4e6FzySLugxpJX3g2a207sJheF7IOyHPX0gzVjRyrM33PySLelNpOI4Pi
iSVHCVXafnfVcMebQBY6FM4ImjEi0p2vDMOJhb64bg==
=IP9j
-----END PGP SIGNATURE-----

--rkLvAjn7JyDnXoy3ThW9ioqDZPHBBuMI3--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:18:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:18:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180585.327307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWJX-0003HQ-1i; Tue, 07 Sep 2021 08:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180585.327307; Tue, 07 Sep 2021 08:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWJW-0003HJ-UF; Tue, 07 Sep 2021 08:18:42 +0000
Received: by outflank-mailman (input) for mailman id 180585;
 Tue, 07 Sep 2021 08:18:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNWJU-0003H6-Mf
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:18:40 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 34d25f74-0fb4-11ec-b0da-12813bfff9fa;
 Tue, 07 Sep 2021 08:18:39 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1877hJUh002674; 
 Tue, 7 Sep 2021 08:18:37 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ax2jngdvj-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 08:18:37 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM6PR03MB4695.eurprd03.prod.outlook.com (2603:10a6:20b:11::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 08:18:34 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 08:18:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34d25f74-0fb4-11ec-b0da-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lTSnc8k0z9+pblZq22nD0t1fgXmhduQuhih1IY1gGEZl9W/2EI1a8b5PJVRc5N5tw2dVx6UrWE7xqxcy7pm/glHLya2OP2+3XlSVSIwRZxhLbJrt6SQIO6xjmPOltCfVOFp1hyf7QZ1P4T4M/pts5B5A8QR0oVrh62VlsW31zT2l6nZsU3raScdiRO9Si8qKGyi6mrdhzJoleqWxJwYBA2lWl5Hsdz0QXboq5wPml3FNjwZSdcKwRfRX+h3F5/94tXNsBryoXomC9uUMBFYRnEIdPunBYzAt2YwN2IhdNSEizzSiFEa7iqrCoWAcQo/9qG6SchK0FylxksP+91t9Mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g9FL1dpPHRZyfhmVFJHfBWM1lqcz7ec+X2EeUF80IHM=;
 b=GFtRRX5L56B5MHIbu7t5W6adWjo5EgbAd3N9xb5AbiCijZGEsPbLm1bg90xzukZAYXZhkHtNTQvQfpyKfEwUQHAQc1SQpFpVNPsZnfRri4bWLpRmXxxKfPGZjOZUsoJzTa4gGX8WFv8GIMalsdAXYHDHDRAC6s6ds3W7fXhyzssyA4g3nXu/EiWqwzEHuI45rDSEWTNErHvFy2gHf/P00akRtDMD8aDJLpQWiF4dAwVdYiu76bgDQfc+z2D32hRnHCbxB5VeMWYdvwCPP27Yh5GnK1XoCgcYQutkYNCsnVIrLxwgxVRA9Ll01oRojey2DlmUfxSgfmxQXEqAt776gA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g9FL1dpPHRZyfhmVFJHfBWM1lqcz7ec+X2EeUF80IHM=;
 b=wuW6nht+sO8HhQhW0txFhOjracm/au/xYy4QuwiYu5XyZQDfsn+Q7pbF0H9qTUDRhzc68e39PUZhWApXTKkdOTRHp1ZVM7IRq7Nn8faFmYiUl1aktMh4OXvBC/X6YYMyVp9oFJzUVJLPd4Fk5Q2aO9fuSrZ7cZQOXPx4PjvxIl5K4oB/X4NXgv9ZMHNxC+zBhtZGVJFhIoggxXz2P7FoBnVdFqlbe61PCBk/wvPfYW/ilUiU0dfFqEV/jbGrZ4oF7E2LwV+t+/1mjvgBZ8r13YJR0UilwnBzepfHLxuJklO4IlRjOf4u/179geYNk0iv9PPfc15Vmn2FWHYUh0OBgQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgA
Date: Tue, 7 Sep 2021 08:18:34 +0000
Message-ID: <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
In-Reply-To: <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 80ec8ef6-7e8b-4b10-3570-08d971d8160d
x-ms-traffictypediagnostic: AM6PR03MB4695:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM6PR03MB469543BB4D93577C73B49E74E7D39@AM6PR03MB4695.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 /EailDQw6DYN+N3MSDGmyX3V9KS1+MFivTTkztG2iC4uwDH7ch+fnvwO+9XzJvREChYRNDRC+DImLJuJLX7B+7IpX7F93mJd0QShSXktMhhK/BQTtylX+3CMvF4nBMjp6Cg0no6ol2AqRDC5SwaHpNj4EEUSHMI2wtjO2VGXEMEDVAEQTeFJ7gY/w0q69atHRxR1ov4hTgN3GApF+UjStkPexrol6tfLrFKd7oXUHW4q9cj2cc3wq14kEHzr+6QOYQ9IfW3f2he/mt12Vkrxeup5Wo8/R1rkYFv4ZBp/HcFdPMTL0YtjyfUheAXpvMxhV4RYN2h9A+ZNlUV7KED+AJO9TUPOhAec3SWFJ/x2ib5VwVS3OCyGOcQuzVJcPPsB6w18bPzp893fl74EdTKvqZKNPg4YCZlsnIWG1KLYH5WMFR6n7/xFiiHFqEt+KXXduketbUNM5iIbJuQLKQk9oIDHCu2DUdgeQJK7/rFxoB9EMHi1iZEtRjiL21s4+uZ1fQzJsh7leqIGH5gj26KvMgictdZA4XSpnPgTHiwmTo0rn7kK68jeyXHLvE2psY/CXosrOwIStIKh8JqYUGeUHTcMcnNERG7ydE3dxObD5cBdYRZdl1vhz3IeBW2+IGAx8oOKCFXUfJDVbwb/J6jBvs/Mh4Scby9BJNYyPbQySscZZfr2eafKIhmHpmTFXWU3RX+PwUUpjgXR4KBHcatwxVFoaGonV5/D0kNoqT4sOBRRFKx5lTmVBF/ZC7dfXMqi
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(366004)(376002)(346002)(396003)(36756003)(66446008)(66556008)(31686004)(83380400001)(2616005)(66476007)(2906002)(76116006)(110136005)(66946007)(122000001)(38070700005)(38100700002)(91956017)(86362001)(31696002)(8936002)(6486002)(8676002)(64756008)(316002)(71200400001)(54906003)(478600001)(6506007)(5660300002)(53546011)(186003)(26005)(4326008)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RDZpK2JhYjNqRFpOVVNHT2J3N2JlWndVVW1KN2sxeE9NRTFOSnY3ZkYyNXFm?=
 =?utf-8?B?VVlhSVFLT3VPSjI4U0I0OVArbDBNUFVwRXJRWGtnUXJ2dkROZzh5RlpOSTlR?=
 =?utf-8?B?Yyt1SGlab0pkR1hKa1JQQzJ6UnlSM1llNU1XZ05FVnF5dk8ram9Zb2g4MG9F?=
 =?utf-8?B?Y2dGMkszd01KQkdNdEd5eTcwQWorZ0UxK1pCUEswUUVyd3JvcmlVSnVHTWEy?=
 =?utf-8?B?UGRIR1Ztc2tpMHY5Qmk3aFFLc0c2R2syNVlyR3BpU0lFYVhFaStRRlR6aTdh?=
 =?utf-8?B?Y2ZNaElCdThsY09pVzViL2JuMzVYOXBncld2V0ZTMFJ2a1FrZFhpZlBHWXEw?=
 =?utf-8?B?RkxHUmR2L0twSGRPNk5SL1c0SXhGUURWcEhwZHRyNzlPLytoMm45UHl1TTNr?=
 =?utf-8?B?c2RqZmF5MkE4Zjd3c1pkYjYvWEFvYVdZdmx2eHpnWUJLY2Z2V3FBWjlSbDdi?=
 =?utf-8?B?dXA4ZUZvamZ1alJ4bHFXcmZHaEk0MVlyWEZnWmhYcy80NWU0a3FKWThRQk5K?=
 =?utf-8?B?MXZZYXd3YktnNEtUY0QzTDhqQzBRcTdBRC96MU9OeVlWV2tnSWttQzVIMUlt?=
 =?utf-8?B?ZjFSa0xvaWhTSHRzL1ZJZzgrdTZjM1g4NjREOVMyLzRFNHdWNXFUb3VPWmJF?=
 =?utf-8?B?QjkxQkN6WEVmK1dGY3BRNzRpQWlRamsvdUZ5T29zVW5zTTJNWW1NZzN5elly?=
 =?utf-8?B?Q0FKUHB2NGEwZWFBZUlOWDBCbXpxUG1HWnNNc1BaS2g2NFpUSHZVdFlxazRP?=
 =?utf-8?B?UThKSk9ybUdza28vT2c4azhpSm9tQW1SK2czRDlTYkxZYW5DWXh4WTJjWW1n?=
 =?utf-8?B?Y2p3N3hXZmIzL2hTa2ZyZ09kOXpPTDZxQVBBVVBvMWsxU3B2S3ZFa2JnWWNK?=
 =?utf-8?B?aTMzQUxuWG1VRVU4U1NnRkMvVkYxZ0JLMWFNbHJwYnlsQ1NUVnQyQ2ZUcWpt?=
 =?utf-8?B?dS9vbjNhcUZaWHVRek53RGRET2lic2JSSjJHZkFERUVkOHNST1dZNFRwS0ZB?=
 =?utf-8?B?cTl5RUp2VlVPNVVpdWF1NWlESWpLVTF5a1dSenNEOXgrNWtJSGExSk92MVBp?=
 =?utf-8?B?TVdBQU5Ra2FKMG5KWERXMTZZeFNlUkM5bUo1VlJMY0JRNEdkYTQvMW5oQW9t?=
 =?utf-8?B?RWhIREtubVFEV1NXMnlmdjF3bkZmTUNrbmluYWI0YmxrMkpGTjVxYU9UbUhs?=
 =?utf-8?B?Qy9wdlQwL2dKZW1sMTdvZkIzVlRGTzIzdXJGQUtHOXZLWDBmWUV1anlKYk5Y?=
 =?utf-8?B?dTFCdm5vYTExWEtFVHA0Wk1qT3pDTlV0SCtLN0NaL1JPay8zL1NJVy83cE0v?=
 =?utf-8?B?YUNEa0ZqbU5RRWtGM2JWcCtWems1QldFd0JaNDZwL3d5OFdSUFFRb1hDSjJQ?=
 =?utf-8?B?ZVlYSkxMMEQ0UWxVdmlhVVRGL01tRWpWSnZUbEVmbjBwUThKaXhIN0NUQWl1?=
 =?utf-8?B?MlcyUHkrZFAwSmRFdkNEaUVWV2JtZ2xJNmFJdWtwTFdvZU1FQTJtOUhKT3Ev?=
 =?utf-8?B?M3B0OVRjZUgyOHBhSHQyT21WNUJJYnYxd3hVOEpRQ3NYRWd5am5saHk5Qk0y?=
 =?utf-8?B?R09JbHFEcXlHaVpBakZWYzlkRXprdmwwQ0NWd1BKVGpUMExncVZ0TENnRHRi?=
 =?utf-8?B?LzVLREs3V3pNL05Wa1VqMXRBdFBCL2F2MkpEZGRhSWt6VXdjSkdHSFVSSGd4?=
 =?utf-8?B?VFlzd1IzRDUwVFZWUGRlQWtvTE8vQ1pBSCtPQ1NhZkNOSzFWSm4wTlUxWHVY?=
 =?utf-8?Q?1fvXBoHJYZ6GRDY91VkUcbbo2YolLRGRAk36uOD?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <5EF03F4C88D4E44E810E69FA9E17AFE9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 80ec8ef6-7e8b-4b10-3570-08d971d8160d
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 08:18:34.5608
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: s8t+v7+sps+QIUclH28VBXhcdovCSRxu0gIXEqxa3rmqjtbexIDWfYjLC+O/Mgsnzp80JCXWPnKNxA0LvmpvXaa2d5R0kjrUo3DkIcVAblVEH5l1oOgUIMK2c71GKM8B
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB4695
X-Proofpoint-ORIG-GUID: MOPuO5Nv1ur17sk8Con6XVQ1JhWNLWkG
X-Proofpoint-GUID: MOPuO5Nv1ur17sk8Con6XVQ1JhWNLWkG
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0
 clxscore=1015 priorityscore=1501 impostorscore=0 phishscore=0 spamscore=0
 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070055

DQpPbiAwNy4wOS4yMSAxMTowMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MDk6NDMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDYuMDkuMjEgMTc6
NTUsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hlYWRl
ci5jDQo+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+Pj4gQEAgLTgxMSw2
ICs4MTEsMTYgQEAgaW50IHZwY2lfYmFyX2FkZF9oYW5kbGVycyhjb25zdCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4gICAgICAgICAgICBncHJpbnRrKFhFTkxP
R19FUlIsDQo+Pj4+ICAgICAgICAgICAgICAgICIlcHA6IGZhaWxlZCB0byBhZGQgQkFSIGhhbmRs
ZXJzIGZvciBkb20lZFxuIiwgJnBkZXYtPnNiZGYsDQo+Pj4+ICAgICAgICAgICAgICAgIGQtPmRv
bWFpbl9pZCk7DQo+Pj4+ICsNCj4+Pj4gKyAgICAvKg0KPj4+PiArICAgICAqIFJlc2V0IHRoZSBj
b21tYW5kIHJlZ2lzdGVyOiBpdCBpcyBwb3NzaWJsZSB0aGF0IHdoZW4gcGFzc2luZw0KPj4+PiAr
ICAgICAqIHRocm91Z2ggYSBQQ0kgZGV2aWNlIGl0cyBtZW1vcnkgZGVjb2RpbmcgYml0cyBpbiB0
aGUgY29tbWFuZA0KPj4+PiArICAgICAqIHJlZ2lzdGVyIGFyZSBhbHJlYWR5IHNldC4gVGh1cywg
YSBndWVzdCBPUyBtYXkgbm90IHdyaXRlIHRvIHRoZQ0KPj4+PiArICAgICAqIGNvbW1hbmQgcmVn
aXN0ZXIgdG8gdXBkYXRlIG1lbW9yeSBkZWNvZGluZywgc28gZ3Vlc3QgbWFwcGluZ3MNCj4+Pj4g
KyAgICAgKiAoZ3Vlc3QncyB2aWV3IG9mIHRoZSBCQVJzKSBhcmUgbGVmdCBub3QgdXBkYXRlZC4N
Cj4+Pj4gKyAgICAgKi8NCj4+Pj4gKyAgICBwY2lfY29uZl93cml0ZTE2KHBkZXYtPnNiZGYsIFBD
SV9DT01NQU5ELCAwKTsNCj4+PiBDYW4geW91IHJlYWxseSBibGluZGx5IHdyaXRlIDAgaGVyZT8g
V2hhdCBhYm91dCBiaXRzIHRoYXQgaGF2ZSB0byBiZQ0KPj4+IHVuZGVyIGhvc3QgY29udHJvbCwg
ZS5nLiBJTlRYX0RJU0FCTEU/IEkgY2FuIHNlZSB0aGF0IHlvdSBtYXkgd2FudCB0bw0KPj4+IGhh
bmQgb2ZmIHdpdGggSS9PIGFuZCBtZW1vcnkgZGVjb2Rpbmcgb2ZmIGFuZCBidXMgbWFzdGVyaW5n
IGRpc2FibGVkLA0KPj4+IGJ1dCBmb3IgZXZlcnkgb3RoZXIgYml0IChpbmNsdWRpbmcgcmVzZXJ2
ZWQgb25lcykgSSdkIGV4cGVjdCBzZXBhcmF0ZQ0KPj4+IGp1c3RpZmljYXRpb24gKGluIHRoZSBj
b21taXQgbWVzc2FnZSkuDQo+PiBBY2NvcmRpbmcgdG8gIlBDSSBMT0NBTCBCVVMgU1BFQ0lGSUNB
VElPTiwgUkVWLiAzLjAiIEkgaGF2ZSBhdCBoYW5kLA0KPj4gc2VjdGlvbiAiNi4yLjIgRGV2aWNl
IENvbnRyb2wiIHNheXMgdGhhdCB0aGUgcmVzZXQgc3RhdGUgb2YgdGhlIGNvbW1hbmQNCj4+IHJl
Z2lzdGVyIGlzIHR5cGljYWxseSAwLCBzbyB0aGlzIGlzIHdoeSBJIGNob3NlIHRvIHdyaXRlIDAg
aGVyZSwgZS5nLg0KPj4gbWFrZSB0aGUgY29tbWFuZCByZWdpc3RlciBhcyBpZiBpdCBpcyBhZnRl
ciB0aGUgcmVzZXQuDQo+Pg0KPj4gV2l0aCByZXNwZWN0IHRvIGhvc3QgY29udHJvbDogd2UgY3Vy
cmVudGx5IGRvIG5vdCByZWFsbHkgZW11bGF0ZSBjb21tYW5kDQo+PiByZWdpc3RlciB3aGljaCBw
cm9iYWJseSB3YXMgb2sgZm9yIHg4NiBQVkggRG9tMCBhbmQgdGhpcyBtaWdodCBub3QgYmUgdGhl
DQo+PiBjYXNlIG5vdyBhcyB3ZSBhZGQgRG9tVSdzLiBUaGF0IGJlaW5nIHNhaWQ6IGluIG15IGlt
cGxlbWVudGF0aW9uIGd1ZXN0IGNhbg0KPj4gYWx0ZXIgY29tbWFuZCByZWdpc3RlciBhcyBpdCB3
YW50cyB3aXRob3V0IHJlc3RyaWN0aW9ucy4NCj4+IElmIHdlIHNlZSBpdCBkb2VzIG5lZWQgcHJv
cGVyIGVtdWxhdGlvbiB0aGVuIHdlIHdvdWxkIG5lZWQgYWRkaW5nIHRoYXQgYXMNCj4+IHdlbGwg
KGlzIG5vdCBwYXJ0IG9mIHRoaXMgc2VyaWVzIHRob3VnaCkuDQo+Pg0KPj4gTWVhbndoaWxlLCBJ
IGFncmVlIHRoYXQgd2UgY2FuIG9ubHkgcmVzZXQgSU8gc3BhY2UsIG1lbW9yeSBzcGFjZSBhbmQg
YnVzDQo+PiBtYXN0ZXIgYml0cyBhbmQgbGVhdmUgdGhlIHJlc3QgdW50b3VjaGVkLiBCdXQgYWdh
aW4sIHdpdGhvdXQgcHJvcGVyIGNvbW1hbmQNCj4+IHJlZ2lzdGVyIGVtdWxhdGlvbiBndWVzdHMg
Y2FuIHN0aWxsIHNldCB3aGF0IHRoZXkgd2FudC4NCj4gWWVzLCB3cml0ZXMgdG8gdGhlIHJlZ2lz
dGVyIHdpbGwgbmVlZCBlbXVsYXRpbmcgZm9yIERvbVUuDQoNCkJ1dCB0aGVuIEkgYW0gd29uZGVy
aW5nIHRvIHdoYXQgZXh0ZW50IHdlIG5lZWQgdG8gZW11bGF0ZSB0aGUgY29tbWFuZA0KDQpyZWdp
c3Rlcj8gV2UgaGF2ZSB0aGUgZm9sbG93aW5nIGJpdHMgaW4gdGhlIGNvbW1hbmQgcmVnaXN0ZXI6
DQoNCiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9JT8KgwqDCoCDCoMKgwqAgMHgxwqDCoMKgIC8qIEVu
YWJsZSByZXNwb25zZSBpbiBJL08gc3BhY2UgKi8NCiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9NRU1P
UlnCoMKgwqAgMHgywqDCoMKgIC8qIEVuYWJsZSByZXNwb25zZSBpbiBNZW1vcnkgc3BhY2UgKi8N
CiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9NQVNURVLCoMKgwqAgMHg0wqDCoMKgIC8qIEVuYWJsZSBi
dXMgbWFzdGVyaW5nICovDQojZGVmaW5lwqAgUENJX0NPTU1BTkRfU1BFQ0lBTMKgwqDCoCAweDjC
oMKgwqAgLyogRW5hYmxlIHJlc3BvbnNlIHRvIHNwZWNpYWwgY3ljbGVzICovDQojZGVmaW5lwqAg
UENJX0NPTU1BTkRfSU5WQUxJREFURcKgwqDCoCAweDEwwqDCoMKgIC8qIFVzZSBtZW1vcnkgd3Jp
dGUgYW5kIGludmFsaWRhdGUgKi8NCiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9WR0FfUEFMRVRURSAw
eDIwwqDCoMKgIC8qIEVuYWJsZSBwYWxldHRlIHNub29waW5nICovDQojZGVmaW5lwqAgUENJX0NP
TU1BTkRfUEFSSVRZwqDCoMKgIDB4NDDCoMKgwqAgLyogRW5hYmxlIHBhcml0eSBjaGVja2luZyAq
Lw0KI2RlZmluZcKgIFBDSV9DT01NQU5EX1dBSVQgwqDCoMKgIDB4ODDCoMKgwqAgLyogRW5hYmxl
IGFkZHJlc3MvZGF0YSBzdGVwcGluZyAqLw0KI2RlZmluZcKgIFBDSV9DT01NQU5EX1NFUlLCoMKg
wqAgMHgxMDDCoMKgwqAgLyogRW5hYmxlIFNFUlIgKi8NCiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9G
QVNUX0JBQ0vCoMKgwqAgMHgyMDDCoMKgwqAgLyogRW5hYmxlIGJhY2stdG8tYmFjayB3cml0ZXMg
Ki8NCiNkZWZpbmXCoCBQQ0lfQ09NTUFORF9JTlRYX0RJU0FCTEUgMHg0MDAgLyogSU5UeCBFbXVs
YXRpb24gRGlzYWJsZSAqLw0KDQpXZSB3YW50IHRoZSBndWVzdCB0byBhY2Nlc3MgZGlyZWN0bHkg
YXQgbGVhc3QgSS9PIGFuZCBtZW1vcnkgZGVjb2RpbmcgYW5kIGJ1cyBtYXN0ZXJpbmcNCg0KYml0
cywgYnV0IGhvdyBkbyB3ZSBlbXVsYXRlIHRoZSByZXN0PyBEbyB5b3UgbWVhbiB3ZSBjYW4gbWF0
Y2ggdGhlIHJlc3QgdG8gd2hhdCBob3N0DQoNCnVzZXMgZm9yIHRoZSBkZXZpY2UsIGxpa2UgUENJ
X0NPTU1BTkRfSU5UWF9ESVNBQkxFIGJpdD8gSWYgc28sIGFzIHBlciBteSB1bmRlcnN0YW5kaW5n
LA0KDQp0aG9zZSBiaXRzIGdldCBzZXQvY2xlYXJlZCB3aGVuIGEgZGV2aWNlIGlzIGVuYWJsZWQs
IGUuZy4gYnkgTGludXgga2VybmVsL2RldmljZSBkcml2ZXIgZm9yIGV4YW1wbGUuDQoNClNvLCBp
ZiB3ZSBoYXZlIGEgaGlkZGVuIFBDSSBkZXZpY2Ugd2hpY2ggY2FuIGJlIGFzc2lnbmVkIHRvIGEg
Z3Vlc3QgYW5kIGl0IGlzIGxpdGVyYWxseSB1bnRvdWNoZWQNCg0KKG5vdCBlbmFibGVkIGluIERv
bTApIHRoZW4gSSB0aGluayB0aGVyZSB3aWxsIGJlIG5vIHN1Y2ggcmVmZXJlbmNlIGFzICJob3N0
IGFzc2lnbmVkIHZhbHVlcyIgYXMNCg0KbW9zdCBwcm9iYWJseSB0aGUgY29tbWFuZCByZWdpc3Rl
ciB3aWxsIHJlbWFpbiBpbiBpdHMgYWZ0ZXIgcmVzZXQgc3RhdGUuDQoNClRodXMsIEkgYW0gbm90
IHF1aXRlIHN1cmUgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgY2FuIGVhc2lseSBiZSBlbXVsYXRlZC4N
Cg0KUGxlYXNlIGNvcnJlY3QgbWUgaWYgbXkgdW5kZXJzdGFuZGluZyBpcyB3cm9uZyBoZXJlLg0K
DQo+ICAgUmVwb3J0aW5nIHRoZQ0KPiBlbXVsYXRlZCByZWdpc3RlciBhcyB6ZXJvIGluaXRpYWxs
eSBpcyBwcm9iYWJseSBhbHNvIHF1aXRlIGZpbmUgKHRvDQo+IG1hdGNoLCBhcyB5b3Ugc2F5LCBt
YW5kYXRlZCByZXNldCBzdGF0ZSkuDQo+DQo+IEphbg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2Fu
ZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:20:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180593.327318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWLJ-0004eu-HA; Tue, 07 Sep 2021 08:20:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180593.327318; Tue, 07 Sep 2021 08:20:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWLJ-0004en-E1; Tue, 07 Sep 2021 08:20:33 +0000
Received: by outflank-mailman (input) for mailman id 180593;
 Tue, 07 Sep 2021 08:20:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNWLI-0004eh-Bo
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:20:32 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNWLH-0007kv-9O; Tue, 07 Sep 2021 08:20:31 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNWLH-0000GH-2C; Tue, 07 Sep 2021 08:20:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=iPUVsAG9/4VmfVlRTjN/f3rMxYdXOSMmyOhnkZw4B9g=; b=bkNFcR2gBy2HkDwApzOo5UBl7S
	qMIgQzSHz22C3kvVFaGNDiLQzVKvbhLm2dG4P/AHWi6v7dxynbT6OcI0bPHtbagYD4S8efYzvkIJf
	GcCYNIoLXvzCwrIzOReI/u0WDjvCjk8BizmtY6KPU3dioXOWSiWgH01AXRTvw+oCH2c4=;
Subject: Re: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
To: Rahul Singh <rahul.singh@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9bbf553c-80ff-7a1a-a87e-e2162085caca@xen.org>
Date: Tue, 7 Sep 2021 09:20:29 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Rahul,

On 19/08/2021 13:02, Rahul Singh wrote:
> pci_init(..) will be called during xen startup to initialize and probe
> the PCI host-bridge driver.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>   xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>   xen/include/asm-arm/device.h |  1 +
>   2 files changed, 55 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index dc55d23778..d1c9cf997d 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -14,13 +14,67 @@
>    * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>    */
>   
> +#include <xen/acpi.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
>   #include <xen/pci.h>
> +#include <xen/param.h>

This include doesn't look to be necessary (yet?).

>   
>   int arch_pci_clean_pirqs(struct domain *d)
>   {
>       return 0;
>   }
>   
> +static int __init dt_pci_init(void)
> +{
> +    struct dt_device_node *np;
> +    int rc;
> +
> +    dt_for_each_device_node(dt_host, np)
> +    {
> +        rc = device_init(np, DEVICE_PCI, NULL);
> +        if( !rc )
> +            continue;
> +        /*
> +         * Ignore the following error codes:
> +         *   - EBADF: Indicate the current is not an pci
> +         *   - ENODEV: The pci device is not present or cannot be used by
> +         *     Xen.
> +         */
> +        else if ( rc != -EBADF && rc != -ENODEV )
> +        {
> +            printk(XENLOG_ERR "No driver found in XEN or driver init error.\n");
> +            return rc;
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +#ifdef CONFIG_ACPI
> +static void __init acpi_pci_init(void)
> +{
> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
> +    return;
> +}
> +#else
> +static inline void __init acpi_pci_init(void) { }
> +#endif
> +
> +static int __init pci_init(void)
> +{
> +    if ( acpi_disabled )
> +        dt_pci_init();
> +    else
> +        acpi_pci_init();
> +
> +    pci_segments_init();

Shouldn't this happen before the PCI initialization?

> +
> +    return 0;
> +}
> +__initcall(pci_init);
> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index ee7cff2d44..5ecd5e7bd1 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -34,6 +34,7 @@ enum device_class
>       DEVICE_SERIAL,
>       DEVICE_IOMMU,
>       DEVICE_GIC,
> +    DEVICE_PCI,
>       /* Use for error */
>       DEVICE_UNKNOWN,
>   };
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:33:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:33:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180608.327333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWXt-0006oj-O3; Tue, 07 Sep 2021 08:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180608.327333; Tue, 07 Sep 2021 08:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWXt-0006oc-Ke; Tue, 07 Sep 2021 08:33:33 +0000
Received: by outflank-mailman (input) for mailman id 180608;
 Tue, 07 Sep 2021 08:33:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNWXs-0006oW-5o
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:33:32 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2e865283-6a99-4ed7-bedd-169b92d7e73d;
 Tue, 07 Sep 2021 08:33:31 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1877mgC7024165; 
 Tue, 7 Sep 2021 08:33:28 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ax413r56p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 08:33:28 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB6981.eurprd03.prod.outlook.com (2603:10a6:20b:299::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 08:33:26 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 08:33:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e865283-6a99-4ed7-bedd-169b92d7e73d
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JDy87PHSNQYi95Ze6iPaEQuNglrE13n+PSqy4ugqUQkDwI7OBG7DHq4700Y0H90DB9eRvMh7qxLK9N9/dgvekH88y4WcThESRdaiT5eKR/Ga+7WXxFsqAOGoFahvny42mMPbRmjX/SUqnbIkTbrd7cJhRRZgdnYgOiNdwB2DFUWfrkq2VR0Z3agQJVQ7zBfug6l4rSZEa/RXB5XSisrmo9nN27bnGwq8hZy9j5DUZ8Tw938if8CoMubDuBkwTpzNAf0KCsDuEo8SwTAL0rThOSATLzEVZJGqxkuDsh0n8xzhEgWB1Xxqdt/DAMB9G+bHXoeT8x9k2cGBeEq50JAsNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rY4+RI5kbn/YbnzJ+0E6KYuPaJLPUxjFO5sxV5cpIII=;
 b=LZHIPId4CZTACxlfYQ9jjNtyuTQHhsk+v9NmfXOqK5KYTfFAbGgdWiDB2o3aj+p0Dhmm3eljWrdMF9mlqChMB3EfVpOLPFWBdlQBL3MvZZhXrbAa5HKW/y62h4A3msyZ4KuWnbnOJhjAYJVyof17arkEf7JIgAPUJsVhqa7EpUF2YrcS/g29IiGvzHmr1EfnVxRyfL5FH4SVsJfYOKvn7ipyntU23jAUw9bFAk7Ax7Xnjo0xoLPDQ9bq0p75DFPc2+TLjY3qRRim6KDdOIZK2ZJ+ZbdkQjtSSdiqErZc0vhY9C6OBIFhKW1Hu8KF9vpJKLn7B/0URPilxRT0yYQw3Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rY4+RI5kbn/YbnzJ+0E6KYuPaJLPUxjFO5sxV5cpIII=;
 b=E4AUFXCC24F7y/pKbrN8Fvm8ivJ+SQ1l1mod79jikW8u7bimg8hGUr7TbEv5ZK1S482pIDWuUd4xkADRnxKkWA5/IpNWkXDcuL/oXSkUEtnOIIJ8Zx/40jNcpz74fSGX+rEeYaCURL2WoTnJVpKdpNtT2p2WvKjW+tXQNELww4sAcF420nwOlFc6WOZ/7c5HHVMFgS4IlxYoTid8fzxrV1cgbTLU9SzwzjbBbEMaO0WA4ULdIZGXDN/QcgY2gX2J9pLZgDHvTdK4h1apuYe9e14dXTuzKEPRecBqObTjzHPnmphmAQttFCWWxN2ybxD63ko5Cc8H9nVQdngxJCuKSA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign
Thread-Topic: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign
Thread-Index: AQHXoKuuObg1egaUxUKTttuKCAaWP6uXA1aAgAFBYAA=
Date: Tue, 7 Sep 2021 08:33:26 +0000
Message-ID: <c299a9fb-f84a-a042-f452-006c9d082256@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-3-andr2000@gmail.com>
 <371a403c-adec-f1e1-8887-5664a749b169@suse.com>
In-Reply-To: <371a403c-adec-f1e1-8887-5664a749b169@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b34e9d35-644b-44f9-999f-08d971da29c2
x-ms-traffictypediagnostic: AS8PR03MB6981:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB698147728B69630307AEE12DE7D39@AS8PR03MB6981.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 VrFiOUWqRn48ftTr1w5Ey/OMqWTFeNzPZM0252d6VzJdFNmDYR4mtAzXYuHroK0L+vfX+jU+xxlwvsyf9itbJh+oxBEA2MxLaDHqAxFP8kBz5/rbsi136j4lAd82UYR1eWUEsw22PksFJoVrXzSV1xclBj8rrAfcnWV3F0cXzXjF2BDWuNVfgdkFWAUGeYIxUinUIOJeq3CumpMrfQuvkc3/KoMy0e0wQ75Au9kUJT9+YEn4R1PN+a9BCoiJIhslEw75RddnhKmezHDdEEVUnvoC/9poMXedJie5gVhYZNluDUkou1+ygH2La0wouV+8vQ4/js9fhl3Ofqf7gjbDI1O0Xt1Ggbms5gVzeoBnirzMDF2vUxZqkfnZFo6ZthGpIQXVXKcb2VhWDTET+zLhh+31TMpux76NNP8wuEiGOl6rjI5vSTFlM9qhNs5RrpkC4p+4OQuY2XWO+iYBm67Pa7ITFrwKT+H5Q1w4HdPoa8hjUIyLO5VRl6b5zAduR3UV0j6kd42uYnzFDHp5jF1WuxEZI/SeJLYqPHE4Mmff1B1ZdaAKx0PZ5L+wOxMFMXHrBJx2403lOpZc08VzoVe95v2i/d5sqSEtNO1BDZbIyOf6QpjehkoP6KIJRWZWzXQ61CosIKzQYwpPlGVu5RAAc3fa7Ixr/Dxq3ps3wTfm9K9dJgaNBi18U+xzkIpb6m6GprIpIuz6ly+T+2orQ/c1l32LF+okZ+GWtlIsxprcau60eq3GKgSsI5GJtZiNaYSj
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(366004)(136003)(396003)(39860400002)(76116006)(66946007)(36756003)(66476007)(2906002)(6506007)(66446008)(110136005)(316002)(54906003)(83380400001)(31696002)(5660300002)(31686004)(86362001)(38070700005)(186003)(64756008)(91956017)(66556008)(6486002)(122000001)(38100700002)(6512007)(2616005)(478600001)(8936002)(71200400001)(4326008)(8676002)(53546011)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aDIwTS9UMS9wWHZ5SXRmUW9SU2RLbkphcWN1VVVPQ1M4LzZzQnJ1dExtVTZY?=
 =?utf-8?B?M29lNWdwRHVRYzFabEhndHhhQXNTRzNkajFaWDErbkJqMmZML0Yrdm5YT0Fr?=
 =?utf-8?B?QWFGblZxV05tRE5UUUxTUkpDcmlrU2hNWmJMT09tdEFXZEVZNEVkZnRUTzBC?=
 =?utf-8?B?aEJRQVlpa2J1RGJoWkRCNkNlMXZNZWg5dnl1L0FySVlJMlkxRW1NU3B3TUZJ?=
 =?utf-8?B?WHpiRFo1bjhlL1Rlb0x5aGF4MVEzZHI4UkFrY2V0L2R6QUFmeHlBYVZMNzRy?=
 =?utf-8?B?S0trWGRtYmJhUUJ1WVgvTnFrNk4xbkxuWUV6T1ZXZzM1bG1ibDQvQi9GZXBw?=
 =?utf-8?B?c3RJN2dzUi94UDRkc0tERW4vcUxMM1NTcFdUOE1zU1p4VTlVUitabVdjQlR4?=
 =?utf-8?B?b2ZDc29IdFZKZ1RZOEozd3Evdzd3MVU1RG9aNnJNODhkckRBeVlNYS9sa3R4?=
 =?utf-8?B?bUNvK3lETitybCtpb0hSRUd3QnJQUjJtS1dpVkJGRmdoQVRXTkhoZDI4Wld6?=
 =?utf-8?B?c0dzeUFpRnNqcGJqS0ZUb0MyNmI3eEg0bUUrc0kyYjVhc081TVg5VGpMK0Rn?=
 =?utf-8?B?WHI4MlZYa0MvdU1aRTlGQUJHMndmbjY1QjhmKzU2aDlpTFlsNnRlVTNxMzE2?=
 =?utf-8?B?RXJWUm9lUUhtMEQ2dDZoY0svRVhjN2NxdDJoaGhSOVdGbGZFUkd2NndyWWZj?=
 =?utf-8?B?d2c4SnUrUlNIZ04yNFpBNE1pUThleS8yVjJKajRyU0tiM2daZitHM3l1b3Az?=
 =?utf-8?B?bzlHMy9PSzZadVpuOTZvTWJseCtpV01lV1loKzM4TFpPOVppb0RwUGIzdXI3?=
 =?utf-8?B?VStWeEZIRVpaOWIxL1hZemY2ajlESGduTllLclc4alUwRXF6V0lJdlo4VWdp?=
 =?utf-8?B?MGp2Q2pLa0I5akJrS1hkQTZCczlOSG5GU3Y1Y0VjN1RUazhPa1d3dll6MFU5?=
 =?utf-8?B?M1NiSmxVTXNxSmVBbDFmWi9tbWFlcS9JNWxMVFlVRWl2SUNFUy9TeVBvRmRi?=
 =?utf-8?B?Qk5rSkdTUkR0YnpLekxpckNHS2JxaG1sUEs2aEdnTVVOeUx6UDlKZ3NDWnJq?=
 =?utf-8?B?eVdSVGNLNWMvMEtPOEZJTS9kRXN0U1p2c2RpT2FVMmlUUDRDV2dRQkRPditU?=
 =?utf-8?B?TjBTcjJRdW5FdDEva01FTkk5MWZFOUIzdXFZWjJRbXM2TkFDQW9GV29rZXZG?=
 =?utf-8?B?Z1pXcU1MMjhkVVJYL0F6R0Y4OFNaQjNhUW5qSklwNzErVjIzN1BsY2VjU0xs?=
 =?utf-8?B?c2IxZlBoNDY3NzRWdU1VOFpoV2psTHFhNHdHdjJ1c2RUczQ4N0tpc0Y1TW9R?=
 =?utf-8?B?SFVmdWVBWTBXQ2tlVmFxZGRpYmFBL2R5NlloZkx0ZlR3Ymc2ck5YSHc3bE5j?=
 =?utf-8?B?cHU5RGJqZHYwcVY5TXJqYUNZUmdzN1ppWnJ4VllydnJIby9RWithRWNsT21F?=
 =?utf-8?B?c2svSWNwa2JPN0g0U0VoTjk2WU9jYkRmd0krNC9YSitLYTQySDNMdkhDL0tn?=
 =?utf-8?B?Z0JUSTdNSEV5Mm1sRmpBamxLeTBEMXNLLzZyZE5xNG9zZTcvSWI3R3AyaFNo?=
 =?utf-8?B?OGpEV3RsNDArb0piakRPcVlBOW9EdHZJbXV4cHhXOEl5Z3JBR2hlTkhSakwy?=
 =?utf-8?B?Ym9RVk1IZ2pnTzRSbUNyYTN1SktFZTBWd2hxd3l2ZWhLMFN5NVd1RGg3SVBR?=
 =?utf-8?B?N0dnQlUwcGR5L2I2d2psbkI0VkNPNzlIcjFkeUd1cWhZQlgxN3ppMVBwbXJ6?=
 =?utf-8?Q?t23m5DQ1MbLQNoPWQ2Nc7W2pnr2gJQfum9nbyHc?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E257E7D0E558EC4AB9FE1BD1DE22AEB3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b34e9d35-644b-44f9-999f-08d971da29c2
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 08:33:26.5998
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vp3q+I2AJF8IYZinVhlwMlGfbHRWvFxLPEnI/e4Mx0tOZeoMCq8Hns/WG1Wutj/Qpz9t/NtsC15XUpQHbfqw8Hp+CH/3DiXjp74qfuLnhRFqPpqu+bp6Mn60eMvoFm3m
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB6981
X-Proofpoint-ORIG-GUID: yzRev8ctFvb79gzde9I-IYo0tvpgGNoK
X-Proofpoint-GUID: yzRev8ctFvb79gzde9I-IYo0tvpgGNoK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_02,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0
 priorityscore=1501 clxscore=1015 spamscore=0 malwarescore=0 suspectscore=0
 adultscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109070056

SGVsbG8sIEphbiENCg0KT24gMDYuMDkuMjEgMTY6MjMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBP
biAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IC0t
LSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+PiArKysgYi94ZW4vZHJpdmVycy9w
YXNzdGhyb3VnaC9wY2kuYw0KPj4gQEAgLTg2NCw2ICs4NjQsMTAgQEAgc3RhdGljIGludCBkZWFz
c2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgdWludDE2X3Qgc2VnLCB1aW50OF90IGJ1cywN
Cj4+ICAgICAgIGlmICggcmV0ICkNCj4+ICAgICAgICAgICBnb3RvIG91dDsNCj4+ICAgDQo+PiAr
ICAgIHJldCA9IHZwY2lfZGVhc3NpZ25fZGV2aWNlKGQsIHBkZXYpOw0KPj4gKyAgICBpZiAoIHJl
dCApDQo+PiArICAgICAgICBnb3RvIG91dDsNCj4+ICsNCj4+ICAgICAgIGlmICggcGRldi0+ZG9t
YWluID09IGhhcmR3YXJlX2RvbWFpbiAgKQ0KPj4gICAgICAgICAgIHBkZXYtPnF1YXJhbnRpbmUg
PSBmYWxzZTsNCj4+ICAgDQo+PiBAQCAtMTQyNSw2ICsxNDI5LDExIEBAIHN0YXRpYyBpbnQgYXNz
aWduX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLCB1MTYgc2VnLCB1OCBidXMsIHU4IGRldmZuLCB1
MzIgZmxhZykNCj4+ICAgICAgICAgICByYyA9IGhkLT5wbGF0Zm9ybV9vcHMtPmFzc2lnbl9kZXZp
Y2UoZCwgZGV2Zm4sIHBjaV90b19kZXYocGRldiksIGZsYWcpOw0KPj4gICAgICAgfQ0KPj4gICAN
Cj4+ICsgICAgaWYgKCByYyApDQo+PiArICAgICAgICBnb3RvIGRvbmU7DQo+PiArDQo+PiArICAg
IHJjID0gdnBjaV9hc3NpZ25fZGV2aWNlKGQsIHBkZXYpOw0KPj4gKw0KPj4gICAgZG9uZToNCj4+
ICAgICAgIGlmICggcmMgKQ0KPj4gICAgICAgICAgIHByaW50ayhYRU5MT0dfR19XQVJOSU5HICIl
cGQ6IGFzc2lnbiAoJXBwKSBmYWlsZWQgKCVkKVxuIiwNCj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQg
SSdtIHdvcnJpZWQgYnkgdGhlIGZ1cnRoZXIgbGFjayBvZiB1bndpbmRpbmcgaW4NCj4gY2FzZSBv
ZiBlcnJvcjogV2UncmUgbm90IHJlYWxseSBnb29kIGF0IHRoaXMsIEkgYWdyZWUsIGJ1dCBpdCB3
b3VsZA0KPiBiZSBxdWl0ZSBuaWNlIGlmIHRoZSBwcm9ibGVtIGRpZG4ndCBnZXQgd29yc2UuIEF0
IHRoZSB2ZXJ5IGxlYXN0IGlmDQo+IHRoZSBkZXZpY2Ugd2FzIGRlLWFzc2lnbmVkIGZyb20gRG9t
MCBhbmQgYXNzaWdubWVudCB0byBhIERvbVUgZmFpbGVkLA0KPiBpbW8geW91IHdpbGwgd2FudCB0
byByZXN0b3JlIERvbTAncyBzZXR0aW5ncy4NCg0KSW4gdGhlIGN1cnJlbnQgZGVzaWduIHRoZSBl
cnJvciBwYXRoIGlzIGhhbmRsZWQgYnkgdGhlIHRvb2xzdGFjaw0KDQp2aWEgWEVOX0RPTUNUTF9h
c3NpZ25fZGV2aWNlL1hFTl9ET01DVExfZGVhc3NpZ25fZGV2aWNlLA0KDQpzbyB0aGlzIGlzIHdo
eSBpdCBpcyAib2siIHRvIGhhdmUgdGhlIGNvZGUgc3RydWN0dXJlZCBpbiB0aGUNCg0KYXNzaWdu
X2RldmljZSBhcyBpdCBpcywgZS5nLiByb2xsIGJhY2sgd2lsbCBiZSBoYW5kbGVkIG9uIGRlYXNz
aWduX2RldmljZS4NCg0KU28sIGl0IGlzIHVwIHRvIHRoZSB0b29sc3RhY2sgdG8gcmUtYXNzaWdu
IHRoZSBkZXZpY2UgdG8gRG9tMCBvciBEb21JTyg/KQ0KDQppbiBjYXNlIG9mIGVycm9yIGFuZCB3
ZSBkbyByZWx5IG9uIHRoZSB0b29sc3RhY2sgaW4gWGVuLg0KDQo+DQo+IEFsc28gaW4gdGhlIGxh
dHRlciBjYXNlIGRvbid0IHlvdSBuZWVkIHRvIGFkZGl0aW9uYWxseSBjYWxsDQo+IHZwY2lfZGVh
c3NpZ25fZGV2aWNlKCkgZm9yIHRoZSBwcmlvciBvd25lciBvZiB0aGUgZGV2aWNlPw0KDQpFdmVu
IGlmIHdlIHdhbnRlZCB0byBoZWxwIHRoZSB0b29sc3RhY2sgd2l0aCB0aGUgcm9sbC1iYWNrIGlu
IGNhc2Ugb2YgYW4gZXJyb3INCg0KdGhpcyB3b3VsZCBJTU8gbWFrZSB0aGluZ3MgZXZlbiB3b3J0
aCwgZS5nLiB3ZSB3aWxsIGRlLWFzc2lnbiBmb3IgdlBDSSwgYnV0IHdpbGwNCg0KbGVhdmUgSU9N
TVUgcGF0aCB1bnRvdWNoZWQgd2hpY2ggd291bGQgcmVzdWx0IGluIHNvbWUgbWVzcy4NCg0KU28s
IG15IG9ubHkgZ3Vlc3MgaGVyZSBpcyB0aGF0IHdlIHNob3VsZCByZWx5IG9uIHRoZSB0b29sc3Rh
Y2sgY29tcGxldGVseSBhcw0KDQppdCB3YXMgYmVmb3JlIFBDSSBwYXNzdGhyb3VnaCBvbiBBcm0u
DQoNCj4NCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvdnBjaS5jDQo+PiArKysgYi94ZW4vZHJp
dmVycy92cGNpL3ZwY2kuYw0KPj4gQEAgLTg2LDYgKzg2LDI3IEBAIGludCBfX2h3ZG9tX2luaXQg
dnBjaV9hZGRfaGFuZGxlcnMoc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+PiAgIA0KPj4gICAgICAg
cmV0dXJuIHJjOw0KPj4gICB9DQo+PiArDQo+PiArLyogTm90aWZ5IHZQQ0kgdGhhdCBkZXZpY2Ug
aXMgYXNzaWduZWQgdG8gZ3Vlc3QuICovDQo+PiAraW50IHZwY2lfYXNzaWduX2RldmljZShzdHJ1
Y3QgZG9tYWluICpkLCBzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gK3sNCj4+ICsgICAgLyogSXQg
b25seSBtYWtlcyBzZW5zZSB0byBhc3NpZ24gZm9yIGh3ZG9tIG9yIGd1ZXN0IGRvbWFpbi4gKi8N
Cj4+ICsgICAgaWYgKCAhaGFzX3ZwY2koZCkgfHwgKGQtPmRvbWFpbl9pZCA+PSBET01JRF9GSVJT
VF9SRVNFUlZFRCkgKQ0KPiBQbGVhc2UgZG9uJ3Qgb3Blbi1jb2RlIGlzX3N5c3RlbV9kb21haW4o
KS4gSSBhbHNvIHRoaW5rIHlvdSB3YW50IHRvDQo+IGZsaXAgdGhlIHR3byBzaWRlcyBvZiB0aGUg
fHwsIHRvIGF2b2lkIGV2YWx1YXRpbmcgd2hhdGV2ZXIgaGFzX3ZjcGkoKQ0KPiBleHBhbmRzIHRv
IGZvciBzeXN0ZW0gZG9tYWlucy4gKEJvdGggYWdhaW4gYmVsb3cuKQ0KR29vZCBjYXRjaCwgSSBt
aXNzZWQgaXNfc3lzdGVtX2RvbWFpbiBjb21wbGV0ZWx5LCB0aGFuayB5b3UhDQo+DQo+PiAtLS0g
YS94ZW4vaW5jbHVkZS94ZW4vdnBjaS5oDQo+PiArKysgYi94ZW4vaW5jbHVkZS94ZW4vdnBjaS5o
DQo+PiBAQCAtMjYsNiArMjYsMTIgQEAgdHlwZWRlZiBpbnQgdnBjaV9yZWdpc3Rlcl9pbml0X3Qo
c3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAgIC8qIEFkZCB2UENJIGhhbmRsZXJzIHRvIGRldmlj
ZS4gKi8NCj4+ICAgaW50IF9fbXVzdF9jaGVjayB2cGNpX2FkZF9oYW5kbGVycyhzdHJ1Y3QgcGNp
X2RldiAqZGV2KTsNCj4+ICAgDQo+PiArLyogTm90aWZ5IHZQQ0kgdGhhdCBkZXZpY2UgaXMgYXNz
aWduZWQgdG8gZ3Vlc3QuICovDQo+PiAraW50IF9fbXVzdF9jaGVjayB2cGNpX2Fzc2lnbl9kZXZp
Y2Uoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiArDQo+PiArLyog
Tm90aWZ5IHZQQ0kgdGhhdCBkZXZpY2UgaXMgZGUtYXNzaWduZWQgZnJvbSBndWVzdC4gKi8NCj4+
ICtpbnQgX19tdXN0X2NoZWNrIHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBwY2lfZGV2ICpkZXYpOw0KPiBJcyB0aGUgZXhwZWN0YXRpb24gdGhhdCAiZGV2IiBt
YXkgZ2V0IGFsdGVyZWQ/IElmIG5vdCwgaXQgbWF5IHdhbnQgdG8NCj4gYmVjb21lIHBvaW50ZXIt
dG8tY29uc3QuIChGb3IgImQiIHRoZXJlIG1pZ2h0IGJlIHRoZSBuZWVkIHRvIGFjcXVpcmUNCj4g
bG9ja3MsIHNvIEkgZ3Vlc3MgaXQgbWlnaHQgbm90IGJlIGEgZ29kIGlkZWEgdG8gY29uc3RpZnkg
dGhhdCBvbmUuKQ0KSnVzdCBjaGVja2VkIHRoYXQgYW5kIGl0IGlzIGluZGVlZCBwb3NzaWJsZSB0
byBjb25zdGlmeS4gV2lsbCBkbw0KPg0KPiBJIGFsc28gdGhpbmsgdGhhdCBvbmUgY29tbWVudCBv
dWdodCB0byBiZSBlbm91Z2ggZm9yIHRoZSB0d28gZnVuY3Rpb25zLg0KU3VyZQ0KPg0KPiBKYW4N
Cj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:33:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:33:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180609.327344 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWXz-00075w-0Y; Tue, 07 Sep 2021 08:33:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180609.327344; Tue, 07 Sep 2021 08:33:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWXy-00075p-TU; Tue, 07 Sep 2021 08:33:38 +0000
Received: by outflank-mailman (input) for mailman id 180609;
 Tue, 07 Sep 2021 08:33:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNWXx-00075D-HM
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:33:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4a0cac9e-0fb6-11ec-b0da-12813bfff9fa;
 Tue, 07 Sep 2021 08:33:35 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-mD_xAjpyMvmvUX887eO22A-1;
 Tue, 07 Sep 2021 10:33:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4349.eurprd04.prod.outlook.com (2603:10a6:803:40::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 08:33:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:33:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 08:33:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a0cac9e-0fb6-11ec-b0da-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631003614;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xY1dQKa7ru5yIZJ5fPY1MRo8fV5l6mrUGhuHVi2kSps=;
	b=augjI1XbJAnmw05LPgJpySd5LUqiChPmO/YpM9Otk6VHFnsqydDE3b+bN2JZEB28PDxc7w
	nRoVKk7I1CJqd1ZvFSS/AdX3fBsGSV8ubffUaSyWR6ej9fZZ+AuUDDj0Bt/xJwBjbf8LCN
	EjO/7RTSiMq17kORABOTzc4WE2iYXNo=
X-MC-Unique: mD_xAjpyMvmvUX887eO22A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MLYeF5LjiBVT25/PCBJSed9tqDmEokCY8QzemNAa6ddp6XL1iu27Q58zsmZBy8VU3twu30ATUzeSB1oLoJDNkv4HTDuxKVZFCesSmHAPRWU0BwswRe+eEvtKEzRX98zI+ayXjISrpXLpQzjjM8K6zZRR/szvGKi2v5v+AAzgv1v6gck7y2SLjcWV88XlIOibAeXgspJEK+DoXZUoEygTaHkeJ4z+X+/PTS9zb0Bm4PlunZz9zlfy+SpZUVggz5pSPHKhWsj/NRr6PwApEa5UIRwWXlhdK08qzlypLNT/sva5aYr7VlgABURAVQ44/KCj6dIr1v43ugFsJZwyxkx7gg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xY1dQKa7ru5yIZJ5fPY1MRo8fV5l6mrUGhuHVi2kSps=;
 b=KrwPTgezsbvUJ4Ehm9vrdSFKOyD9R8N/KkcIYtUsoVN5PwdwEBY1nYkqej+NON8LZPmcv2Uah7F1cbjAyD84ixI+8gkRlbqrSx0MwgsDQQT8yC2oesTlCJjZf0Koc+kFLdsfcCfDsG4KaU52jJBPVuliL6HMt2np2byKS19i93Qc9nt6FmFWLGa0nAfOkV1iC/NyiOL5yL7mq+gzsF1F/ONjKGxMqDFTS367YMLNEh2KWn56oyJl+3TpMpO1uN11AHP6HEPcQ05McKkbIiKQPvuSdAPzMdcMHuqy0F2DsgJ7sHsE/ZE50hV0uDOVAsGEg0ZxfwVZsulckCWbWbJSyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210907065228.21794-1-luca.fancellu@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ea736db5-e3cf-7eea-cffe-98ecb0290048@suse.com>
Date: Tue, 7 Sep 2021 10:33:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210907065228.21794-1-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6e312c61-01f0-4ad5-3803-08d971da2bf4
X-MS-TrafficTypeDiagnostic: VI1PR04MB4349:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB434991F34E0587797C0F4A21B3D39@VI1PR04MB4349.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kWB/Pj/5Cgl0Qc2/VcsHEfof8G7vwurxa1QGmnIgMe99GhCdoWA0xyOowLhWef3rOpeHnV3Cn5lXzqGzcVHYqw9NgoAUFu/A4s/I/XCAGPSs8+an5NJ+ykvjsl3E20zSekYCH+kjeuqK4L02pSHpi2OjnVy2MqjA5b84kF4w5VCMaZ3jp+il2WHgfPYpK6e/xSvLRs1nXhBpW0dkFluArhEBSOWKdaWOgGzIY0R8XzsuMicjg1sjxYqIEPe+NtAM2urC1eu4OeelybjzzW9mQVC7z6INHps4qR6WcF/CjnG7NHULeutRKFREVeHlFXtubXkYzWvGn1eTGR+iy0eK295Uz+xHluV/vdudwcfKuigX4TODP5/pJQ4jpy8BDt8d/XWJ1PZ3sTC5TwLlRsE33O2OQN1ywKn1KXmWZ1I/LwqmSZxV2nBkzeUrVjuGc27Tk+UbIM+9buz917BEM7CCMGqUHlYbf/e9YNbYabv1TeQha+qXgCaJRUmUXQbtM2+E7dieacNk8dTZ+tGAKlfuyry3EFt/L+A4q779gV7DD6716vty2vst7Z2h0hyOKUM8gKktvOn1XojWg9TX1l59gfx4vrscpeH5AoLkdO52L/5A1P3k3/Q9RkRATgXUFJZEc3famdXE1xwiAgRbr7ghHhM3IWC2FcpmR1o+Rp7SKH3PdlOg3excc8EzVdjBvgKdOCHLMYJOduM/2jAhSxYoSl+7kCt9dZaGaLAn9BgFvt5ExFlmubxYxxEJIpO7IcPwR9ycA4C0wpp/Ti4Luo0NfG/PfqDatN9PcQiVzA0uGhPOjImNb0tNULiNF+SlbY6b/aXSL00awXME5uylrV0kcOp30wWwn1WceB7MgVN615Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(376002)(346002)(6486002)(478600001)(2906002)(38100700002)(186003)(36756003)(6916009)(316002)(16576012)(31686004)(5660300002)(966005)(26005)(54906003)(31696002)(86362001)(53546011)(2616005)(956004)(66946007)(7416002)(8936002)(83380400001)(66556008)(66476007)(4326008)(8676002)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?a0tsTHFUcldYV243bnhpMVl6YzY5WU9DUnFmZ1N0NG9KNzlzOVpqWEtON0VX?=
 =?utf-8?B?OEJsQjI1bGU2K2w2TVZvRytTeVNXZ0ZqRkJNVjU3NC9QNUI2NEFlNmd1NUhx?=
 =?utf-8?B?dDFLM1lKdUxrbmMyWXpwR0pjYlQxN3dZaXNrcjhiKy9sck8zZEZzN3VDdnNF?=
 =?utf-8?B?RkV6VnEyd2RyWUtvQzQvMitkN2ZiRUxIVGlRc1UxaVJ0MGdEOXVVM0F3d0Y4?=
 =?utf-8?B?WXI3MWdkQzJEa1JBelBMOGR6aFhCUkZ0R1lHcmw4enQ3aHYyNkZMQTdNbmlk?=
 =?utf-8?B?WFdFcjdQVGNWbDVYcXRreDBTajlrYUdqSTQ2WGxjSFVZMWNSWGd2QUJ0dFY1?=
 =?utf-8?B?TGkwQ055aU1vTU5sa1JCQXBrakVKaDRhK2xXWEtWM1VJZFdiNEQzNmxoQTBG?=
 =?utf-8?B?Zno0S0JCWjV5bzJVS2RZT2lySXdlNkdRWUlyY2FRWmVxNmpFY1ZXQ015bHVZ?=
 =?utf-8?B?MGpnWjIvTkFZWm5JZllNS016SWdRTlZFR29YbmthY1RVcC9TeUUwN1dabTdJ?=
 =?utf-8?B?cVBJWUFSWlRIdHh4VFU2d0kxTUdwL1M3aVoycnoxVytNWk1MbzJRTiticGww?=
 =?utf-8?B?K3hGNkZCU3RQWC9WenhLM0xSMFlNVlBMS2RiUnN6K3VmY2lvSS94MmdRWjc4?=
 =?utf-8?B?RlFJSmpoajhNUU4zVkxLc25VbVBXREh5Mkp2azdwVTV1U29VK3I1dzQ1MXl6?=
 =?utf-8?B?c0E0Q3R2Y2JjYks1ZkFpS05aa25hUFYwc2hnaHlaNml6NCtTSG81RWdCRm5k?=
 =?utf-8?B?aDQ2NURJUGVWZGUzaGlycmdjUEg5aU5rOVdwbnJLMWpUc1RrTWc2RElEd1Nx?=
 =?utf-8?B?TnNoRVZOUWZNa2w1WkVuVHFyRkZZaHBSL2hUcms2U1pERllYRUtzRndaOXJZ?=
 =?utf-8?B?VGdTZy91MFptbHMxV1pNU3U5eFYrM2FwSnllYWd1ZTdMSGJpVjZJM1BKRUFJ?=
 =?utf-8?B?TXp5dzU3b2tXQ0dMNW1OdG5uWmJhZDUyVnNZcXczNXMyb1JJL2dsZXE0bHlQ?=
 =?utf-8?B?UUI3N0Z5T2NBanV4RU5vaTAyOU9SR3UwdWVVZTVjT3ZuTTRUaTZNQ3dmUEk4?=
 =?utf-8?B?eittRGtQVFBGVTVGQ1Q0K1ZUZm9nZWdFc0ptNFJFVHEvM3RqanRHWEFxcG1N?=
 =?utf-8?B?YmlMdGhSeUw1SnVOY21lMDlhQjF6NzNXTGllM0pLZnlCdDdQd3k3b2VtdDJs?=
 =?utf-8?B?VWpmNmJlMlkydlJYaytudjVYbVFRbmVaMWRXeDhYc2FRY1FacExuK2t6UFhG?=
 =?utf-8?B?M3dzdCtDbG56TU40UzZlSTIyOHlxTUdCWFdwNkxIYzgva3g2NVhKQWtWRWxm?=
 =?utf-8?B?NS9lK2FnRjUwZUZvYmRCc1ZyaFhQT3IrckI3dXV4bWxxdFpGZjBOWTdOSkov?=
 =?utf-8?B?S2NLWDdqcGdBNDBBYVZpdnlMOVV5OU1ENy9GV2tSbGVvb1IwclgzVTdKNjJ3?=
 =?utf-8?B?WTBnWlkwWFNsT3llbWRqTjlQbUVXMWY1MWhBeXNDT2pxZUxyYkxlQjh4MFV0?=
 =?utf-8?B?QWVSSVprdHk0NVhyczJEclo4Z1VkSkJpWE1uRk91UDA5UGlrRUJuRk9ndU00?=
 =?utf-8?B?dkxyT0F0NVZoZXpoYVNCOGJaVnl2WENmTktIa2d1NVVjMHZralhCRG5TY3VK?=
 =?utf-8?B?YmIwejRqYmVwTlh3cy9SdFFkWkVha3hnZVNoMS9ldEFkMlBkZDROMFh2Q3VB?=
 =?utf-8?B?cmNyUHMrNU1aTy9WcjhGa0p5Rng0eHdrL05XZXlER0hoL0o3VVZwaHZNVW1O?=
 =?utf-8?Q?CY1OR7JZJ/bDorqS3XjTQcVL94Hkr5RKwjt+2P9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e312c61-01f0-4ad5-3803-08d971da2bf4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:33:30.5029
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2lSh2h1WW40Igaz0NBp81Oei5QcOM9qOHMvazOVgN0i/on/mn5anHZjSYiwJRtt7HAgbBK5JkgE0EK6B7q3Vlg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4349

On 07.09.2021 08:52, Luca Fancellu wrote:
> Add a design describing a proposal to improve the EFI
> configuration file, adding keywords to describe domU
> guests and allowing to start a dom0less system.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>  1 file changed, 105 insertions(+)
>  create mode 100644 docs/designs/efi-arm-dom0less.md
> 
> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
> new file mode 100644
> index 0000000000..8d8fa2243f
> --- /dev/null
> +++ b/docs/designs/efi-arm-dom0less.md
> @@ -0,0 +1,105 @@
> +# Xen EFI configuration file
> +
> +The current configuration file used by Xen when it is started as an EFI
> +application is considering only the dom0 guest and doesn't have any
> +property to describe and load in memory domU guests.
> +Hence currently it's impossible to start a dom0less system using EFI.
> +
> +# Objective
> +
> +This document describes the proposed improvement to the Xen EFI
> +configuration file to list properly both the dom0 guest and the domU
> +guests as well.
> +The final goal is to be able to start a dom0less system using EFI.
> +
> +# Current Xen EFI configuration file
> +
> +The current configuration file is described by the documentation page
> +https://xenbits.xenproject.org/docs/unstable/misc/efi.html.
> +
> +Here an example:
> +
> +```
> +[global]
> +default=section1
> +
> +[section1]
> +options=console=vga,com1 com1=57600 loglvl=all noreboot
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +xsm=<filename>
> +dtb=devtree.dtb
> +```
> +
> +# Proposed improvement
> +
> +The proposed improvement to the current configuration file is the
> +introduction of new keywords to describe additional domUs.
> +
> +Here follows the proposed new keywords:
> +  - domu#_kernel=<kernel file> [domU command line options]
> +    - Mandatory kernel file for the domU#
> +  - domu#_ramdisk=<ramdisk file>
> +    - Optional ramdisk file for the domU#
> +  - domu#_dtb=<dtb file>
> +    - Optional dtb fragment file for the domU#, it is used for device
> +      assignment (passthrough).
> +  - domu#_property=cpus=2
> +    - Properties that should be added to the dtb in the domU node to
> +      properly describe the domU guest. Refer to the documentation:
> +      https://xenbits.xenproject.org/docs/unstable/misc/arm/device-tree/booting.txt,
> +      section "Creating Multiple Domains directly from Xen".
> +
> +For all the keywords above, the # is a number that uniquely identifies
> +the guest.
> +The keywords domu#_kernel, domu#_ramdisk, domu#_dtb are unique, therefore there
> +must not be specified the same keyword twice in a section.
> +The # number is not enforcing any domid, it is just used to link each property
> +to the right guest, so there can be domu1_* guests that are started with domid 2
> +and so on.
> +
> +The domu#_property can appear multiple times and it specifies an additional
> +property to be listed in the domU node inside the device tree, Xen will
> +not check if the same content is specified multiple times.
> +
> +There are some property whose name starts with an hash symbol (#address-cells,
> +#size-cells), in this case the line will be considered as a comment, so to
> +specify them, they have to be listed without the hash symbol, the documentation
> +will be updated as well to see the implemented handling of these special
> +properties.
> +
> +# Example of a configuration file describing a dom0less system
> +
> +The following configuration file is describing a dom0less system starting two
> +guests.
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +# Xen boot arguments
> +options=noreboot console=dtuart dtuart=serial0 bootscrub=0
> +# Xen device tree
> +dtb=devtree.dtb
> +
> +# Guest 1
> +domu1_property=address-cells=2
> +domu1_property=size-cells=2
> +domu1_kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
> +domu1_property=cpus=1
> +domu1_property=memory=0xC0000
> +domu1_dtb=domu.dtb
> +
> +# Guest 2
> +domu2_kernel=Image-domu2.bin console=ttyAMA0 root=/dev/ram0 rw
> +domu2_property=cpus=2
> +domu2_property=memory=0x100000
> +domu2_property=vpl011
> +```

I'd like to suggest a different scheme, not the least because I expect
the individual domains being independent of e.g. hypervisor command
line options or Dom0 kernel versions. Yet varying sets of these are,
for example, a reason to have multiple sections in the current scheme.
Every dom0less guest would then require spelling out in every such
section. Hence I think we'd be better off having a section per guest:

[guest1]
kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
property=cpus=1
property=memory=0xC0000
dtb=domu.dtb

These sections would then be referenced by other sections, e.g. by a
new "guests" (or "domus", but this ends up looking a little odd for
its matching of an unrelated latin word) keyword:

guests=guest1,guest2

If it is deemed necessary to make sure such a section can't be
(mistakenly) used to create Dom0, such sections would need identifying
in some way. Presence of property= (or, as per below, properties=)
could be one means (allowing an empty setting would then be desirable).

As to the properties, is there anything wrong with having them all on
one line:

[guest1]
kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
dtb=domu.dtb
properties=cpus=1 memory=0xC0000

(just like is the case for options=)?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:37:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:37:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180621.327354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWba-00088Q-Kg; Tue, 07 Sep 2021 08:37:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180621.327354; Tue, 07 Sep 2021 08:37:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWba-00088J-Hc; Tue, 07 Sep 2021 08:37:22 +0000
Received: by outflank-mailman (input) for mailman id 180621;
 Tue, 07 Sep 2021 08:37:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNWbZ-000889-Pj; Tue, 07 Sep 2021 08:37:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNWbZ-00081q-L0; Tue, 07 Sep 2021 08:37:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNWbZ-0002pt-By; Tue, 07 Sep 2021 08:37:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNWbZ-0001cQ-BS; Tue, 07 Sep 2021 08:37:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=GpezyJmmiaztqkYLkEph0l5QPZZXhnLxPiQKIpkd1fM=; b=lwh5daN8GfWxPj0ZjrOENzMH+3
	A4MRCG5iFU0b/H0RXmohO7CliwmucvYGoArO/xUzrxvJrYgcQSZOuuJoe/rJU2j/CcyKuhh3V1R9W
	ewDzoTL6Rkmp+k+Ix+Ldj7gGX1ZLr1haOAgM9uFmF0AF4tPNzhhIC74tDpOxgsCOAVNo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164869-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164869: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0
X-Osstest-Versions-That:
    ovmf=4473834e7d49c555eca81f96383a1d6d6f5f4bb2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 08:37:21 +0000

flight 164869 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164869/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0
baseline version:
 ovmf                 4473834e7d49c555eca81f96383a1d6d6f5f4bb2

Last test of basis   164862  2021-09-06 15:10:03 Z    0 days
Testing same since   164869  2021-09-07 04:11:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abner Chang <abner.chang@hpe.com>
  Cheng Zhou <zhoucheng@phytium.com.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   4473834e7d..a7cf2c5664  a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:44:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:44:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180639.327369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWiO-0001O1-DE; Tue, 07 Sep 2021 08:44:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180639.327369; Tue, 07 Sep 2021 08:44:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWiO-0001Nu-A0; Tue, 07 Sep 2021 08:44:24 +0000
Received: by outflank-mailman (input) for mailman id 180639;
 Tue, 07 Sep 2021 08:44:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNWiN-0001No-Io
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:44:23 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc12e3ba-0fb7-11ec-b0da-12813bfff9fa;
 Tue, 07 Sep 2021 08:44:22 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-oBpkjDe3M9inU-ss14bDRA-1; Tue, 07 Sep 2021 10:44:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6478.eurprd04.prod.outlook.com (2603:10a6:803:12a::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 08:44:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:44:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0059.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.21 via Frontend Transport; Tue, 7 Sep 2021 08:44:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc12e3ba-0fb7-11ec-b0da-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631004261;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=y4e7cPHUvDvUZfKXDaQ86Rvpo//kr0KR6cJuZ42srw8=;
	b=YdMmxJB6LCzBqyjr+DM5N/SKAO3o7KNK6zUW8qkEMnfnrOiIz5X1OQ2ktnzRUwkH0d8HAz
	w9sdMQaDQrZ6d7PNsRfHq0ogbzjXXItdbAVBlymFazNkZgpftm1ZKKExRaXiPjzEWlDnnx
	6S3+tMkTaL8Z6Uou5HOHwRzAZHhQURM=
X-MC-Unique: oBpkjDe3M9inU-ss14bDRA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gOow3NjjInyVCfxS9vqeqY63pJ0KBiESWjsfyp+1ATipzvH2igUzh11pjTShd2KncD7rR/ayi3gTvIomuKNu0I6Y2mxGdFpQadArg3Y1+BmmL8jknM0cAsmHWUO5/j1Xkx0yZn4thNgYX5/6ypI9XZBNZ7pbj/PnlPprdgnnMaCwvMELkITT61+N0MtyYAHQE/okgKKkgCB88CEujXNlwJDW0FUUR7lbdkH3iybMe24zZsrBS1D00FekTjdR+KzXdCvYgYm2yDDXH84RmRRrYMylwAHLlPo8NUA9Pn9jArRndw0YjX1MV/ajdjx1YjhCZJgG3PBJEBkB7FOpTNdP8Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=y4e7cPHUvDvUZfKXDaQ86Rvpo//kr0KR6cJuZ42srw8=;
 b=DUY8S9+3T4PqbPkfbNPukhp/vXabM3oi+pV5oxCLfJLWJMbsu/FbVEBZerI+OB6tzwZNcQTxZSo+lPjkrK2ZcPJLZCJx1q4lJLQbtbhBh190aakPZBYPi5jVjQdimTsKVXwCGFgOuP28uz4R5RENQhyd6/VzwvrI9XK+JSXxlnzV3DyfbdAUu2Xisx8FZX4B/lOfx+HD7icfHKhSJw0/AYp+IHXiMvfq9bSHjULMDaPY7J52Tuqko0xGkE1Aiw6n0nbLLBbq1mz1eNkjUZUyX/AMSClxc+u+aLkOShZ40b4AMfcPBXyea67SWwRCWvhaC3eql7A+391cL06fLToCGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-3-andr2000@gmail.com>
 <371a403c-adec-f1e1-8887-5664a749b169@suse.com>
 <c299a9fb-f84a-a042-f452-006c9d082256@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c1df422b-4e10-40d1-2726-a4d5b76673f0@suse.com>
Date: Tue, 7 Sep 2021 10:44:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <c299a9fb-f84a-a042-f452-006c9d082256@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0059.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c996cd76-f1d4-41ad-3d7d-08d971dbaecc
X-MS-TrafficTypeDiagnostic: VE1PR04MB6478:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6478ED99925F54CAEBC35FEBB3D39@VE1PR04MB6478.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VF2FFk0KGCV3MQlxoEH+MsOlzoP1NcNWFOgje6DRluLaRJaeMdnvbI2PwFUkbvPeoSrwDZ3iegVl6iIVZBqguzRlI/h2VBvn1dgakh4VOI2+9EvkKnON5Co+6lPf6CmZQd4aftwx89hbu0zzKVM3xhwet3gDpxHd0SWM6Jme9YUVZTLw6zeEMpskmYAUY//QxB67njQAE56fNaWKTcVaUELa/izF9Mx5qO69TxZL0XfCu6rRyAeVY6Ie9u72wj8cvcfgoDSGBvaNhhCPXnVda4X+aVn1z/oO5Kxw0RdsUsyrW74WnzNBuOpL4eRskbJOVzD70kGOrUSg7uXRR7IfyMTM69LnfeGQSMGav02SV+MH0jRa1GNzKF3YSAJOEy5jAyZzYwiM7hXXw5Jr+F9+nq+r3ZGzS5sgvt5yaZ07OVkwo1e0t2DwcgYdLMNI9H+bcq5oM0G8fn0goHQoYzUToy8UR8ZbkgYuE2uQO0zwTxYwkkKXQpWfBoXkpJkO9UocTQGJOJonddXlKC5JPTDm2OLcMgJfsQ9TCM6qA/IcGTucxnvE3OPDRMH/dfwFJ68Vfjy3rv9W1mx/6Kn6LW4eQWvrU3AyV/PkZlY1fd162XtFWNEcDZhPAco3zvgTEa41zV7sc8Cff+KvUUlkcLUrcGBpGNUwbryPGBZwIDxj03SoQx1EAWNymVVj+um0hU/+frLAI+fCOaCGu7xU25DXniaWnBfA/0x9zu/PG1L68tQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(376002)(366004)(346002)(316002)(16576012)(110136005)(54906003)(4326008)(83380400001)(5660300002)(478600001)(26005)(7416002)(66556008)(53546011)(66476007)(66946007)(186003)(38100700002)(86362001)(31696002)(6486002)(2616005)(956004)(31686004)(2906002)(36756003)(8936002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SXdaMHcxNVN3VzJ5a0VoUG5Md04wNHBPcWZmckI2c3gycVE3Z29rVW9USDNB?=
 =?utf-8?B?WjRzQ2pYMFNPR0REd09zUDhLWmdIWWJjVHRLUEhwNDVROWNnakJhcU9OSXNj?=
 =?utf-8?B?aTNBSml1MW1qWnp0cDlBZmp2cy8zQW44TkZ5QW9IbFpJQnpqYmVoNUJPUk80?=
 =?utf-8?B?d0hqTTZZN2ZrTlkvYzJPeDI1Ly9MYk5lM0twOEdWRHozbW9kQ2VpcDE3KzEy?=
 =?utf-8?B?U0lwMmlqVVdDYWova2J5T0EvZzFsMndhQXM0RXNOVkM0WnNNNzUrNG9hSWh4?=
 =?utf-8?B?M2Jab2hsZkVFczdQOHQ1cUxiVWVUa3VBY202VWRReTNpL0oxM254cWNXak56?=
 =?utf-8?B?ajVabmdyWDNvOHF5KzNvY3Q2dlBBUUVDRHl5d29OVDhRemNJL20xZjZRUmlG?=
 =?utf-8?B?SFA3dHB1TmVpMWE1ei9lSE9BaWdMNkU2aGdFZTF2bXk4UUl5T2t6YTZUN0Rj?=
 =?utf-8?B?dUVZWlBaM0lZQTV4bVhpMWdCY2cxdkx1MWRjVmlYbTZxMVExVkRlVnhrcVZp?=
 =?utf-8?B?cVlWeWc1STBlOHRDV1RJcmYzeWJVVjdwY0ZhRi9lVldMWDlUS3lLUTI1YW5w?=
 =?utf-8?B?TlZvbmJJckc1TnpIQzM5NDNKS2paZDNPMXBJR3k3dzRsa3hjblAveHUwMEV2?=
 =?utf-8?B?K2RZMzFqeXZQUU5zT214OHozck5XbE5EMitkc0p3MmI3ZmUySzdLTGVqUHZQ?=
 =?utf-8?B?M3F3azZVL1hDYU55T1d5NnBSeFVhenRqVEZXeTM2a3FIdmtVTDRnL0hzb2M1?=
 =?utf-8?B?K0JlTHJPVDgxYWlRcUhFbk9xTGpSb29veiszRVpLaFRhbVpDMlhzSWp0Wjhm?=
 =?utf-8?B?ZitPb3FVeXYrTVgydlUxWG9NbnFQZTZXMzFMRlNTVTF2WnNDQlNrWnZSVnRS?=
 =?utf-8?B?bzZjRU55Ylpaa0xrR0hCNDVTSC9Oa1dRZlIrWHgxQnVxNERkZEtocVBwTGMw?=
 =?utf-8?B?T1VqTHY4cGRON3VsbEFBMlhXbFVseFpHN0lzRnE4Rm1NVjJSSHZId3JFVWYz?=
 =?utf-8?B?NkxReks5d1N1VGZpNlhGeVBXbXFXWFg2c21weFBZVU9jSVJGbmpJL3Rzdko5?=
 =?utf-8?B?ZjYrOE9NMlJycFoyQTNiY25XN1NoQlliMXNhaDdWNHZxZXY1emhiNHhXbDFi?=
 =?utf-8?B?bmJIV1d3SWt0Vm54amVNRm1MdUk3M20rUUhrb2N1c3JhNTRlOW5RbXZZMlZC?=
 =?utf-8?B?Ym5taGdzUjVDYXRVdnBGVnkxWEg3SWRhOVk4UU5pOUhyZUd6ZjlDZ1ovWHFH?=
 =?utf-8?B?aElsSHJQTmgxVFV1SzNaRmltMUlGY0hwa3NWMy91RTZNc3MzNGtnOUVEcFlk?=
 =?utf-8?B?SVNCNG5aU1dmaGkwQXFxM3BuSGx6bVFZVnMxMWt0SkpaRUl3QlF2MUw5eG9Y?=
 =?utf-8?B?NXJ6L2dldFN4WXdNNnBOYXJveDQrNGNDbkEzYkpEaVlCQ3c0alBLRSsyMEd0?=
 =?utf-8?B?RkxKYVIxaEo5U0xtbzhFN1Z0c0VnTUo4OEp6dVRmeUFlYmFJdEdQbVdWUTFq?=
 =?utf-8?B?aEF6T3FVb0FCc0JLTW5qUEE1TmY2c3MzNzF2bzNMTmkyRHpVUWFyQjVCcGJm?=
 =?utf-8?B?RHNQNE9VaWMyUXFCenVFaEt0QmZ2aURtVFB6WUZuTEw3K2hoTllYRGFWeTBE?=
 =?utf-8?B?Q290NnRkeW4rQnZEeHpST3ZPVVVpTUNpWW1Bd1ZvQ2xYVE1vK1B2WUZQeklX?=
 =?utf-8?B?TUxySEVJa3FrLzFRSTB6UG9ydEVuM3F4bkNSYW1nSDNDQXA2QmZzNGNCODA3?=
 =?utf-8?Q?tK61S0zdo1rRJMkw2vquIRETKrJH6w0eQ9ifkMC?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c996cd76-f1d4-41ad-3d7d-08d971dbaecc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:44:19.5302
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HchVqPhG4yzsYm02HKtDhZNhZiHstDe+4WMddXT/g3GBiLb+MLl899LV++udI191a/RB5Vdzt5ttPYDx7c7vCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6478

On 07.09.2021 10:33, Oleksandr Andrushchenko wrote:
> On 06.09.21 16:23, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -864,6 +864,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
>>>       if ( ret )
>>>           goto out;
>>>   
>>> +    ret = vpci_deassign_device(d, pdev);
>>> +    if ( ret )
>>> +        goto out;
>>> +
>>>       if ( pdev->domain == hardware_domain  )
>>>           pdev->quarantine = false;
>>>   
>>> @@ -1425,6 +1429,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>>>           rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
>>>       }
>>>   
>>> +    if ( rc )
>>> +        goto done;
>>> +
>>> +    rc = vpci_assign_device(d, pdev);
>>> +
>>>    done:
>>>       if ( rc )
>>>           printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",
>> I have to admit that I'm worried by the further lack of unwinding in
>> case of error: We're not really good at this, I agree, but it would
>> be quite nice if the problem didn't get worse. At the very least if
>> the device was de-assigned from Dom0 and assignment to a DomU failed,
>> imo you will want to restore Dom0's settings.
> 
> In the current design the error path is handled by the toolstack
> via XEN_DOMCTL_assign_device/XEN_DOMCTL_deassign_device,
> so this is why it is "ok" to have the code structured in the
> assign_device as it is, e.g. roll back will be handled on deassign_device.
> So, it is up to the toolstack to re-assign the device to Dom0 or DomIO(?)
> in case of error and we do rely on the toolstack in Xen.
> 
>>
>> Also in the latter case don't you need to additionally call
>> vpci_deassign_device() for the prior owner of the device?
> 
> Even if we wanted to help the toolstack with the roll-back in case of an error
> this would IMO make things even worth, e.g. we will de-assign for vPCI, but will
> leave IOMMU path untouched which would result in some mess.
> So, my only guess here is that we should rely on the toolstack completely as
> it was before PCI passthrough on Arm.

Well, okay, but please make this explicit in the description then.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:49:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:49:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180645.327380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWnd-00022u-3C; Tue, 07 Sep 2021 08:49:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180645.327380; Tue, 07 Sep 2021 08:49:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWnc-00022n-V2; Tue, 07 Sep 2021 08:49:48 +0000
Received: by outflank-mailman (input) for mailman id 180645;
 Tue, 07 Sep 2021 08:49:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNWnb-00022h-5C
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:49:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8cca3e32-0fb8-11ec-b0dd-12813bfff9fa;
 Tue, 07 Sep 2021 08:49:46 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2050.outbound.protection.outlook.com [104.47.13.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-_uOl7SUxNhK_wGlpSXP27Q-1; Tue, 07 Sep 2021 10:49:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6669.eurprd04.prod.outlook.com (2603:10a6:803:125::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 08:49:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 08:49:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0120.eurprd02.prod.outlook.com (2603:10a6:20b:28c::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 08:49:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8cca3e32-0fb8-11ec-b0dd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631004585;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+USiVCtL27+QQVariAoWrWCDUJ0hi/rb0UFRwLTlaow=;
	b=SYDMJFrpR7gpCRgiYpYjofoYikWKfk9roc1lp9retPps7XGuA9L8ftdsbC50vJovjCgYAK
	VOLq/cM7rm5qIb8anrvQSMGM5V0kLjskLrph/hFBaiIJun6tio/pszBr6q/aDnxHdxafM+
	9sKSngbhuTW6hJKncBURWVbyXp1kZE8=
X-MC-Unique: _uOl7SUxNhK_wGlpSXP27Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ec6JgUnFhyybd6anCfRarEA/03/sBMUfW9BG0eSLitlfHjvZeWUBhENH43rjKmwrG+W7+SW0v1Bg1PKN6na16IkTHiUSOUB6yQ3bbtQ9z+0gwdFlTEhBkCfFhwiePCwUrI0vEYsoriCd34mLHFpqvSWgyyFLGxcIDx51JRpi3WhJ5eOgk9sp9zrrcZZrCC4SNY5dmtXoCFVkr5o65Rcpqd4WBevNRugpJ58iOgB6KXK7NSJuWmQhszq5p0S75kuPy43gUmNVEmpiiBGwpkReuD+0T1jbjNZFpOJgybkhpxjjuVq7KFuUSC6TyrmanmtV+Z2bl39lKbFAQLj31UUPaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=m0kuWnlTREyx4UISZpn1Ak+Ryu5Www7B1hzmyh1mIPU=;
 b=O+LzsgfB8sER4uDHJ7yGBhX6Fw3VX80mmKc8HnCIB6BS9ZfZdMrxE+WyAfoUhN9udpbo+ejDNylk9FJUCZLGeiMUJg2P9L+MZKbyBguyoBSLf2wj8GCZzBSDr33r0DVskdAt+xXPAB3dtFQGPezAOhSGVNQtww/hHqS72jlmmB89srWmAsXeSHr25J9WkjLHgAe0FngmZ2bTGdx6Sx2jAj1pU0UZ2p2ZvsIM6bkf9i9IIOhOT9/jRuf9+blcFi+hNNRoaAzmPbsFctDjm8jcAMEKVmouBlc3UMO0AlSBVqV9LZpKwaKWAp3p1fXg5dX5ZieAAr05eyv0Bk0NETa6Fg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
Date: Tue, 7 Sep 2021 10:49:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0120.eurprd02.prod.outlook.com
 (2603:10a6:20b:28c::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d6a9fe48-1b83-4bec-108f-08d971dc6e86
X-MS-TrafficTypeDiagnostic: VE1PR04MB6669:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66692B7B073FD05795F08E91B3D39@VE1PR04MB6669.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GnOff/IUUpeRC0jvHqH3QlMUKu89A+rAXavLqdve8muAYenIFul/ALNCkWEHgLdObZuA0PjY9u27cQqiDKfrbGeP+ZKS6BnBHAO8q5vXIov9z6Opw1BoadxLsJbwNmtb7uLr6EvnxmXUmJ3q2Ftm3u1oQ+ekk9mad8RR27l6P8oU1Fw3XqdB8U0tKE8Sn3t/a2b6UDuNQfhZlqdVhjzLhi26Z42vaziNvtlUt8eKriK37t5r2p7fLx3FRozGN4QarK5urjo5gE8YPE7+tHp2WK/cXEeCyMoCVM/AmrY9LkRK79FJQoryddF59kKzSrom3+sXhERYmjDEbjkCm6KBYlMb79E2x1VlvNNbbA1yHkDjzv37hfACELAM8mPMlW8OXwoP2KSgBfa/T4wZCYtE9wxb2Dxwm288g4MR/7LHGMQdMBCGiygTfDmX/shWiTwqUZ9UBaJyf1ju145SBIajYzK2I2+dUNtz38+mNqcL+06044bHKwfAMV13R/xIao7rjBWLtzw8EbFysjGxTnbAz6epnJHgNzfUpiqtFo4JPoW3wQFgEwprraKRsbQ+31499xfUjpErE76dMBNVYCyxOxz330P59Pn2qHzUs3pBKHcfW6TzUxV8wcxlwRZOX6QYGLjTjmlunQouakKPB/QdXvTCUEOWZSPQ3kI2NyXWBxUUs02dgJfIbJ8c8InWxj8GM720ITbHSHkoGzhZn9SmuhtybKvWI8OgowoxP1GgkareVWe1f5dp+ZezaVfLbRsz
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(346002)(136003)(39860400002)(54906003)(110136005)(36756003)(38100700002)(53546011)(8936002)(7416002)(186003)(26005)(66946007)(66476007)(8676002)(2616005)(478600001)(956004)(66556008)(5660300002)(2906002)(4326008)(83380400001)(6486002)(16576012)(31696002)(316002)(86362001)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?JJasR3zm8M7Mh6GuhMxIWpP7hL4CS4grKhNx71AcXJMgxezQOePwyM35dU8y?=
 =?us-ascii?Q?Fn5LFobznt4+XmSAavLgr/TncGgq0W28xDSYmqkM7vou03IMEiunZt82x56I?=
 =?us-ascii?Q?xZTDzsz7dY9/UwYjJuo3w45RiYGnvzIUAxc7LiD4XQmKE12Y+MnLOqGnghcQ?=
 =?us-ascii?Q?B7b2qFYZcvbtFXDnOdE2EA3vDbcTO7FieE6Mz2Qh/gDMGBo5YwsuzA0wB2rc?=
 =?us-ascii?Q?vQCwrmwvhmRyz3T8DDAUeP5w6PF7wl3DXooi5ZzbCHydNNCuqzyL4sDe/93j?=
 =?us-ascii?Q?43GFfzBp5sm+5+nvfNv4sDHP9+Ur+34ZbC0lCKLaTxXqcbT+ofnVS6vm6yoU?=
 =?us-ascii?Q?xWP66URUnzs6Wtwg/bcTbyL8WFzfgTiN3s1h8Fm8bqTCVllpxuJgIFqcMY4G?=
 =?us-ascii?Q?KqfBfPCzsWtsE2/yOEqwgHPi4RNNYxxb4t0YE6zu1BIna74GjfrcsHBFfA24?=
 =?us-ascii?Q?NL1hBM3VRbTFVbhdD7paPocTX9Z/pYJ2n4hEVDv0MmYM58NdkiJI2LOHvoZp?=
 =?us-ascii?Q?xiMa2aRCuzAockgGA6gKBXZCyu27RjyeQzCAw5QFvRcdMcbf52QEFL0GOi7p?=
 =?us-ascii?Q?rWRbfN6z1E+LdE2b+vO0c/ojGkLtsAR0g82m4z3h5GoE6OAZRE45hzkEt6jI?=
 =?us-ascii?Q?i8Fyy8rjeMHBG5TySf+wDx92KWRfBiTL9AxleGK/FYSMxLjeTUzP3sxLl5bA?=
 =?us-ascii?Q?EHofjFkmqcIzwJzEtIUazPdnwb8SDD8/PlSRT4MfE8uy0R1owt8h2PDF1TLk?=
 =?us-ascii?Q?9egnKEyEk1vv2EvUWj6Ukg4XkAH4BOusI18sCrkHeBE/RPTAPRqee8YOYTSM?=
 =?us-ascii?Q?ZotSMWl8+fxphaXMLkHp6HFwvoEU+hZXH7BQrXJzb70jKTAXMz1LP8mv91xj?=
 =?us-ascii?Q?F326q6Zhy8VkpPi7qbDTp4GFA2Kbc4Cfl38YYWTNSUVKnqK4dzthkpUrdBed?=
 =?us-ascii?Q?Ekr4EpK1Nhz+lxYmnnYXPQ/Y/agT/riwO99xda8GCKJt6u1V68nJVBOlTLoW?=
 =?us-ascii?Q?dTqBiy6UUHO0tQD4sLoy32TYX2AiEB/1yccUgPC9Vs+u5G5U/Ol7EVBEr77Z?=
 =?us-ascii?Q?tEWc2ny7LQC3SI/OJ0qtrdAZUvEPBYI1loQslZgC5BwVJOxohk/wvbtG5KTO?=
 =?us-ascii?Q?7PKZUZGK5QManJOLxbkd8FJWqLpjrIMS6FHvmH//OpuiBj1oJR7TnKt3D6wT?=
 =?us-ascii?Q?iOmiw9LyQ30Ou8xzxD8Pbx0erg//KtU5CXzVnnWrgSvoPjAN+hW3yRqe1d6x?=
 =?us-ascii?Q?fktusEporvHSbCnw8x0uTXPymY0DMszGjMKRxQcGaLuLsfvmYB/7Nwg8KrM3?=
 =?us-ascii?Q?lGWJbRtOmEdTXWA8JyxbOem0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d6a9fe48-1b83-4bec-108f-08d971dc6e86
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:49:42.1373
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iUna6nDs2FsHh1Ppp1OZWQNRynqh1YADuz3Iy2JqGbYHxoR7qelKXsdtZ5VBr0o89arxGqWJuKRHOw8xdq3//A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6669

On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>=20
> On 07.09.21 11:00, Jan Beulich wrote:
>> On 07.09.2021 09:43, Oleksandr Andrushchenko wrote:
>>> On 06.09.21 17:55, Jan Beulich wrote:
>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>> --- a/xen/drivers/vpci/header.c
>>>>> +++ b/xen/drivers/vpci/header.c
>>>>> @@ -811,6 +811,16 @@ int vpci_bar_add_handlers(const struct domain *d=
, struct pci_dev *pdev)
>>>>>            gprintk(XENLOG_ERR,
>>>>>                "%pp: failed to add BAR handlers for dom%d\n", &pdev->=
sbdf,
>>>>>                d->domain_id);
>>>>> +
>>>>> +    /*
>>>>> +     * Reset the command register: it is possible that when passing
>>>>> +     * through a PCI device its memory decoding bits in the command
>>>>> +     * register are already set. Thus, a guest OS may not write to t=
he
>>>>> +     * command register to update memory decoding, so guest mappings
>>>>> +     * (guest's view of the BARs) are left not updated.
>>>>> +     */
>>>>> +    pci_conf_write16(pdev->sbdf, PCI_COMMAND, 0);
>>>> Can you really blindly write 0 here? What about bits that have to be
>>>> under host control, e.g. INTX_DISABLE? I can see that you may want to
>>>> hand off with I/O and memory decoding off and bus mastering disabled,
>>>> but for every other bit (including reserved ones) I'd expect separate
>>>> justification (in the commit message).
>>> According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0" I have at hand,
>>> section "6.2.2 Device Control" says that the reset state of the command
>>> register is typically 0, so this is why I chose to write 0 here, e.g.
>>> make the command register as if it is after the reset.
>>>
>>> With respect to host control: we currently do not really emulate comman=
d
>>> register which probably was ok for x86 PVH Dom0 and this might not be t=
he
>>> case now as we add DomU's. That being said: in my implementation guest =
can
>>> alter command register as it wants without restrictions.
>>> If we see it does need proper emulation then we would need adding that =
as
>>> well (is not part of this series though).
>>>
>>> Meanwhile, I agree that we can only reset IO space, memory space and bu=
s
>>> master bits and leave the rest untouched. But again, without proper com=
mand
>>> register emulation guests can still set what they want.
>> Yes, writes to the register will need emulating for DomU.
>=20
> But then I am wondering to what extent we need to emulate the command
>=20
> register? We have the following bits in the command register:
>=20
> #define=C2=A0 PCI_COMMAND_IO=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 0x1=C2=
=A0=C2=A0=C2=A0 /* Enable response in I/O space */
> #define=C2=A0 PCI_COMMAND_MEMORY=C2=A0=C2=A0=C2=A0 0x2=C2=A0=C2=A0=C2=A0 =
/* Enable response in Memory space */
> #define=C2=A0 PCI_COMMAND_MASTER=C2=A0=C2=A0=C2=A0 0x4=C2=A0=C2=A0=C2=A0 =
/* Enable bus mastering */
> #define=C2=A0 PCI_COMMAND_SPECIAL=C2=A0=C2=A0=C2=A0 0x8=C2=A0=C2=A0=C2=A0=
 /* Enable response to special cycles */
> #define=C2=A0 PCI_COMMAND_INVALIDATE=C2=A0=C2=A0=C2=A0 0x10=C2=A0=C2=A0=
=C2=A0 /* Use memory write and invalidate */
> #define=C2=A0 PCI_COMMAND_VGA_PALETTE 0x20=C2=A0=C2=A0=C2=A0 /* Enable pa=
lette snooping */
> #define=C2=A0 PCI_COMMAND_PARITY=C2=A0=C2=A0=C2=A0 0x40=C2=A0=C2=A0=C2=A0=
 /* Enable parity checking */
> #define=C2=A0 PCI_COMMAND_WAIT =C2=A0=C2=A0=C2=A0 0x80=C2=A0=C2=A0=C2=A0 =
/* Enable address/data stepping */
> #define=C2=A0 PCI_COMMAND_SERR=C2=A0=C2=A0=C2=A0 0x100=C2=A0=C2=A0=C2=A0 =
/* Enable SERR */
> #define=C2=A0 PCI_COMMAND_FAST_BACK=C2=A0=C2=A0=C2=A0 0x200=C2=A0=C2=A0=
=C2=A0 /* Enable back-to-back writes */
> #define=C2=A0 PCI_COMMAND_INTX_DISABLE 0x400 /* INTx Emulation Disable */
>=20
> We want the guest to access directly at least I/O and memory decoding and=
 bus mastering
> bits, but how do we emulate the rest? Do you mean we can match the rest t=
o what host
> uses for the device, like PCI_COMMAND_INTX_DISABLE bit? If so, as per my =
understanding,
> those bits get set/cleared when a device is enabled, e.g. by Linux kernel=
/device driver for example.

I would suggest to take qemu's emulation as a starting point.

> So, if we have a hidden PCI device which can be assigned to a guest and i=
t is literally untouched
> (not enabled in Dom0) then I think there will be no such reference as "ho=
st assigned values" as
> most probably the command register will remain in its after reset state.

What meaning of "hidden" do you imply here? Devices passed to
pci_{hide,ro}_device() may not be assigned to guests ...

For any other meaning of "hidden", even if the device is completely
ignored by Dom0, certain of the properties still cannot be allowed
to be DomU-controlled. (I'm therefore not sure in how far Dom0 can
actually legitimately "ignore" devices. It may decide to not enable
them, but that's not "ignoring".)

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 08:54:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 08:54:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180656.327391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWrp-0003lL-O9; Tue, 07 Sep 2021 08:54:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180656.327391; Tue, 07 Sep 2021 08:54:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNWrp-0003lE-L1; Tue, 07 Sep 2021 08:54:09 +0000
Received: by outflank-mailman (input) for mailman id 180656;
 Tue, 07 Sep 2021 08:54:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Xbzw=N5=arm.com=Henry.Wang@srs-us1.protection.inumbo.net>)
 id 1mNWro-0003l8-4g
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 08:54:08 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.87]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 656bb1bb-3465-4a4c-a600-cf12a2c49f3a;
 Tue, 07 Sep 2021 08:54:04 +0000 (UTC)
Received: from AM6PR08CA0018.eurprd08.prod.outlook.com (2603:10a6:20b:b2::30)
 by VI1PR08MB5533.eurprd08.prod.outlook.com (2603:10a6:803:12d::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 08:54:00 +0000
Received: from AM5EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:b2:cafe::26) by AM6PR08CA0018.outlook.office365.com
 (2603:10a6:20b:b2::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 08:54:00 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT035.mail.protection.outlook.com (10.152.16.119) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 08:53:59 +0000
Received: ("Tessian outbound b48d04bf8afd:v105");
 Tue, 07 Sep 2021 08:53:59 +0000
Received: from 157a9ae1b450.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4F7A7C14-4190-4850-BC8A-D468F262965C.1; 
 Tue, 07 Sep 2021 08:53:53 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 157a9ae1b450.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 08:53:53 +0000
Received: from PA4PR08MB6253.eurprd08.prod.outlook.com (2603:10a6:102:e4::8)
 by PA4PR08MB6063.eurprd08.prod.outlook.com (2603:10a6:102:ed::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 08:53:52 +0000
Received: from PA4PR08MB6253.eurprd08.prod.outlook.com
 ([fe80::e4b3:8a19:4478:2f6e]) by PA4PR08MB6253.eurprd08.prod.outlook.com
 ([fe80::e4b3:8a19:4478:2f6e%7]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 08:53:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 656bb1bb-3465-4a4c-a600-cf12a2c49f3a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YL8Pk4V+1+v3exfGE2oOhB0Xp/3yEFdKkhD+1Klxtig=;
 b=zPYfCHsrM4Rs27JP3pM/HrNedV1T+IzxAwRjYZ2bTWONiz8UXclKDYg3IiR7+mKdGhVCvmL7bS4lIlYNxro0EC2tDKRz7gU2avYQ8RYURZZX4VYOGSGkibBE2nTZja4XUfoIM22ARy5hDoVuUSlNSdtICdASJTWOh3t6ENbh4jw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e84oTZfk8Zcl52z7i4r7VtQBj4cOzj1kxmwZgyG+uJVoblQHNkfOeETP7+wccaonACmwpxXl1lZVqZVtA8x7a/fKiLCfNGKFo+P8VlNwLH9MKncFMoCADEcSfSFxL53TFPGAiXKfvMLZ2LzRv7U4rIuB/Nlb5VCV3O7wnsYNwc60Mb+WijLpxhF6GtrtDVssdtjw+W1+PkjRdL9Pt+LMeZiaBxTKe3PFETDmFXrx3UE107VCcnUrZE6qf3KPmXaL3x4PJS9LR6BJbBiai0kT8FDqaESKlNzr9HWYlhZziHF60tXdobsNk+hueCmRNCReWkHeUOznCw4u+b1v+8TxQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YL8Pk4V+1+v3exfGE2oOhB0Xp/3yEFdKkhD+1Klxtig=;
 b=hooCY5EuNb25T0NPUd+eF++yCaNNNYH/xqruKxWv8p+siSd9U4xX9MX2REOfg5umSltAPEmqOoRWb7BhBWfKJ9C9tgJ2Iu++dzjVcmThgyVfLBqW6Q0899LOpJayQtWdaYReXyHTRKZMs72UPe7xrqRXfEDMrGb5r+QFLFzj5YSh3SdqHeF7Z5Y3vM85zvELBjR64rkq2MwtjzAxGRMCqdL4aXvABPS5GrGX/l6OPJ0S/mlnFaEuBzkS02SvLnob/Rl39cqT/U22MNtNQ0YEHJoCtWBTuxw2ptDU1hyGtfbqP0XwgOdm0Eet5wACS2HSQ7eb0POVM58le+alLVGWew==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YL8Pk4V+1+v3exfGE2oOhB0Xp/3yEFdKkhD+1Klxtig=;
 b=zPYfCHsrM4Rs27JP3pM/HrNedV1T+IzxAwRjYZ2bTWONiz8UXclKDYg3IiR7+mKdGhVCvmL7bS4lIlYNxro0EC2tDKRz7gU2avYQ8RYURZZX4VYOGSGkibBE2nTZja4XUfoIM22ARy5hDoVuUSlNSdtICdASJTWOh3t6ENbh4jw=
From: Henry Wang <Henry.Wang@arm.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, Daniel De Graaf
	<dgdegra@tycho.nsa.gov>, "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen
	<Wei.Chen@arm.com>
Subject: RE: [RFC PATCH] xen/memory: Introduce a hypercall to provide
 unallocated space
Thread-Topic: [RFC PATCH] xen/memory: Introduce a hypercall to provide
 unallocated space
Thread-Index: AQHXg8xFOJw2l6FsN06730YVyibR0auYgwfg
Date: Tue, 7 Sep 2021 08:53:52 +0000
Message-ID:
 <PA4PR08MB6253F59F4C63FFE50CC6ADBC92D39@PA4PR08MB6253.eurprd08.prod.outlook.com>
References: <1627489110-25633-1-git-send-email-olekstysh@gmail.com>
In-Reply-To: <1627489110-25633-1-git-send-email-olekstysh@gmail.com>
Accept-Language: zh-CN, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: C01E3F5F0E5BE947987B72EC078054FB.0
x-checkrecipientchecked: true
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 998654e2-1ebe-4345-80a6-08d971dd08d6
x-ms-traffictypediagnostic: PA4PR08MB6063:|VI1PR08MB5533:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB55331C080D8AEDA5B9A8648A92D39@VI1PR08MB5533.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:3513;OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HfcNRZBTiqINTCcQVCKSjPv3AncExrg0PDIFEbSqqqbWcmhk9FJGydA1HNy8vNJ5mvg0MDO+dtOhKLOGoQGP/GHhXNBwFmzLk/fRcXpwD96QRBcqSQYiGc3qP/JLavW81FkSj539fDw6Xh6+hzqthmS7BvGTTYfgN3M55HurKb+Ybe1nY7E6dKJ3L4PtRJC5Ta1J7RhkT+MKIGzn2dpxf9fuTIpORA7VU3u9LNyzD0kRvzQ21ZxzeS/P012dd8/lA26EG2fuPmTDbzAsWSVnpuFYSzzu9b1i4neLn0iyuWKU7L+th721P8DhzOSc2UuIuQGkH1jAPleKdW15Icbq8WgqTP5MVjhsRit34eLY93MT40iVjkf2b0rxYPwGUZyLIwniGHt5V+P2c1mGh+Ecssh3I24/6rglQdOORUdHk6eM1i+2cNqmc5WVj5+MR1vzTMfCUexQvHJALAVeaENm7wtGT/jRU0ptE7eS26VoYhRYqb2TFBRbIHhCWqYuoelgtR35G2SNFVv0O8fmzVBiTPcJ1MMPIIXC9/aQbFU6lVgQCauLzZuNcNUbyoZi+36jb/p/GsQC7SfBu3AK4hluGn2deFl+2x5T/wJaS2c/oL46Y+pijv/yRPeto2bnh3pdcatT0v1Y9QdVip7rniSERXVRBkzssBtBp/iVZ7emBVFAhs88F8Y8XyboFvfcW0DQpYdxg9ejdQ50k9WvUUwkrVFiMurfzWt1l1JVCW9EbgBZsOjrBv0WN/FRz87D1DsCbG6mrO+igoBnwOFk/KC9BbzcJFPp5bmf32QFflaFH7cvzKk30p7mEvY0RET58cvx40Yh9lfVnTXWQ9eVBM3Zdg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR08MB6253.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(366004)(396003)(346002)(39860400002)(6506007)(53546011)(8676002)(316002)(5660300002)(83380400001)(7696005)(8936002)(7416002)(33656002)(55016002)(26005)(86362001)(2906002)(38100700002)(71200400001)(122000001)(30864003)(52536014)(966005)(4326008)(9686003)(64756008)(66446008)(186003)(66556008)(478600001)(54906003)(66476007)(38070700005)(76116006)(66946007)(110136005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6063
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8da9d086-81a9-473e-071e-08d971dd0442
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9Yjuy6o/733GvZGxuCCoMVUP2Gr9cpzdEvggx+fkOv7AmH5RdCfQVP9Yb3M/hEaFHr2GGZJKhFGHyLnFqHoju+TVYe7xfSuO7Lw6Te3+b5aI+GsA4GmN3yyYZfJbEa2tKRJM0azMC3raQczmIZhcVslNWNwtBzEHL56l6XfmkfE5aTgrqdu2RbVu82c/S6I2ZaZkFYTC4Kq7hfm6kfkIaHvqRm/JO7kJGaFPQ78q18DSTjhTlZ+FXhebfr2aqi716+E/bO5pLRrxzUn1mrbNG/xLbi1B7QdL80jf5pu1ELv3yUGem1XYqXPyVWIi5h33+wwq2ljnHWatm3TFDenot3wdWF9c7GmA1u6c02jZRhvG26MuasgM2D8a+pj1/wQxjmPIlW0rZmR2YJuRmyURebWkaWCIj3p2/+gux9Ch/X0EV+VeAEGZrZ7semvO6+sKhIp74WmuahKpRgET2FOcL4a1sMT9LJaU6uC5OWQaptvGYzwbXu07mNGUj/olWqFhmVhkhXSFj4S7UJIMie/z5dJlmEO92n46eR2CaWEBejIY0nXl42JD4xvGF7HU8ChuA4HeHF/deRJqt0xGWIAiAvWsgz/+mITOsRR6n2ft22IcWRASfZMHXex0wqkPJh43cJ7GEMHhMPPCLzIYW/KgB1JETtJwwhhPm6KrmP8ZrGpnqW1FRx6myV+fKASD90W9GqZZwxw1Zgqff7Sc40ANS/XymoaYluygqFCQ7jvSkK+ZdqxzYodoRXbKH7YP87ItQe/sZ/L2SEX2ptpb0rf7CoKTerQPmJlVnBiZZ62LPKFxmc1ieLpdlhqvUCrHDGvBXu6I0SfpyxUQleBjw3p3Rg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(39860400002)(346002)(36840700001)(46966006)(83380400001)(7696005)(86362001)(316002)(4326008)(70206006)(70586007)(82310400003)(81166007)(54906003)(966005)(186003)(26005)(33656002)(53546011)(478600001)(336012)(8676002)(47076005)(82740400003)(52536014)(110136005)(8936002)(356005)(6506007)(5660300002)(55016002)(2906002)(36860700001)(30864003)(9686003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 08:53:59.8852
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 998654e2-1ebe-4345-80a6-08d971dd08d6
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5533

Hi Oleksandr,

> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Oleksandr Tyshchenko
> Sent: Thursday, July 29, 2021 12:19 AM
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Daniel De
> Graaf <dgdegra@tycho.nsa.gov>; Daniel P. Smith
> <dpsmith@apertussolutions.com>; Ian Jackson <iwj@xenproject.org>; Wei
> Liu <wl@xen.org>; Andrew Cooper <andrew.cooper3@citrix.com>; George
> Dunlap <george.dunlap@citrix.com>; Jan Beulich <jbeulich@suse.com>;
> Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org=
>;
> Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Roger Pau Monn=E9
> <roger.pau@citrix.com>; Bertrand Marquis <Bertrand.Marquis@arm.com>;
> Wei Chen <Wei.Chen@arm.com>
> Subject: [RFC PATCH] xen/memory: Introduce a hypercall to provide
> unallocated space
>=20
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>=20
> Add XENMEM_get_unallocated_space hypercall which purpose is to
> query hypervisor to find regions of guest physical address space
> which are unused and can be used to create grant/foreign mappings
> instead of wasting real pages from the domain memory for
> establishing these mappings. The problem with the current Linux
> on Xen on Arm behaviour is if we want to map some guest memory
> regions in advance or to perform cache mappings in the backend
> we might run out of memory in the host (see XSA-300).
> This of course, depends on the both host and guest memory sizes.
>=20
> The "unallocated space" can't be figured out precisely by
> the domain on Arm without hypervisor involvement:
> - not all device I/O regions are known by the time domain starts
>   creating grant/foreign mappings
> - the Dom0 is not aware of memory regions used for the identity
>   mappings needed for the PV drivers to work
> In both cases we might end up re-using these regions by
> a mistake. So, the hypervisor which maintains the P2M for the domain
> is in the best position to provide "unallocated space".
>=20
> The arch code is in charge of finding these regions and filling
> in corresponding array in new helper arch_get_unallocated_space().
>=20
> This patch implements common and Arm parts, the x86 specific bits
> are left uniplemented for now.
>=20
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Tested-by: Henry Wang <Henry.Wang@arm.com>

Kind regards,

Henry

> ---
> Current patch is based on the latest staging branch:
> 73c932d tools/libxc: use uint32_t for pirq in xc_domain_irq_permission
> and also available at:
> https://github.com/otyshchenko1/xen/commits/map_opt_ml1
>=20
> The corresponding Linux changes you can find at:
> https://github.com/otyshchenko1/linux/commits/map_opt_ml1
> I am going to publish them soon.
> ---
>  tools/flask/policy/modules/dom0.te  |  2 +-
>  xen/arch/arm/mm.c                   | 18 ++++++++++++
>  xen/common/memory.c                 | 56
> +++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/mm.h            |  4 +++
>  xen/include/asm-x86/mm.h            |  8 ++++++
>  xen/include/public/memory.h         | 37 +++++++++++++++++++++++-
>  xen/include/xsm/dummy.h             |  6 ++++
>  xen/include/xsm/xsm.h               |  6 ++++
>  xen/xsm/dummy.c                     |  1 +
>  xen/xsm/flask/hooks.c               |  6 ++++
>  xen/xsm/flask/policy/access_vectors |  2 ++
>  11 files changed, 144 insertions(+), 2 deletions(-)
>=20
> diff --git a/tools/flask/policy/modules/dom0.te
> b/tools/flask/policy/modules/dom0.te
> index 0a63ce1..b40091f 100644
> --- a/tools/flask/policy/modules/dom0.te
> +++ b/tools/flask/policy/modules/dom0.te
> @@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
>  };
>  allow dom0_t dom0_t:domain2 {
>  	set_cpu_policy gettsc settsc setscheduler set_vnumainfo
> -	get_vnumainfo psr_cmt_op psr_alloc get_cpu_policy
> +	get_vnumainfo psr_cmt_op psr_alloc get_cpu_policy
> get_unallocated_space
>  };
>  allow dom0_t dom0_t:resource { add remove };
>=20
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index 0e07335..8a70fe0 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -1635,6 +1635,24 @@ unsigned long get_upper_mfn_bound(void)
>      return max_page - 1;
>  }
>=20
> +#define GPFN_ALIGN_TO_GB(x) (((x) + (1UL << 18) - 1) & (~((1UL << 18) - =
1)))
> +
> +int arch_get_unallocated_space(struct domain *d,
> +                               struct xen_unallocated_region *regions,
> +                               unsigned int *nr_regions)
> +{
> +    /*
> +     * Everything not mapped into P2M could be treated as unused. Taking
> into
> +     * the account that we have a lot of unallocated space above
> max_mapped_gfn
> +     * and in order to simplify things, just provide a single 8GB memory
> +     * region aligned to 1GB boundary for now.
> +     */
> +    regions->start_gpfn =3D
> GPFN_ALIGN_TO_GB(domain_get_maximum_gpfn(d) + 1);
> +    regions->nr_gpfns =3D (1UL << 18) * 8;
> +    *nr_regions =3D 1;
> +
> +    return 0;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index e07bd9a..3f9b816 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -1811,6 +1811,62 @@ long do_memory_op(unsigned long cmd,
> XEN_GUEST_HANDLE_PARAM(void) arg)
>              start_extent);
>          break;
>=20
> +    case XENMEM_get_unallocated_space:
> +    {
> +        struct xen_get_unallocated_space xgus;
> +        struct xen_unallocated_region *regions;
> +
> +        if ( unlikely(start_extent) )
> +            return -EINVAL;
> +
> +        if ( copy_from_guest(&xgus, arg, 1) ||
> +             !guest_handle_okay(xgus.buffer, xgus.nr_regions) )
> +            return -EFAULT;
> +
> +        d =3D rcu_lock_domain_by_any_id(xgus.domid);
> +        if ( d =3D=3D NULL )
> +            return -ESRCH;
> +
> +        rc =3D xsm_get_unallocated_space(XSM_HOOK, d);
> +        if ( rc )
> +        {
> +            rcu_unlock_domain(d);
> +            return rc;
> +        }
> +
> +        if ( !xgus.nr_regions || xgus.nr_regions >
> XEN_MAX_UNALLOCATED_REGIONS )
> +        {
> +            rcu_unlock_domain(d);
> +            return -EINVAL;
> +        }
> +
> +        regions =3D xzalloc_array(struct xen_unallocated_region,
> xgus.nr_regions);
> +        if ( !regions )
> +        {
> +            rcu_unlock_domain(d);
> +            return -ENOMEM;
> +        }
> +
> +        rc =3D arch_get_unallocated_space(d, regions, &xgus.nr_regions);
> +        if ( rc )
> +            goto unallocated_out;
> +
> +        if ( __copy_to_guest(xgus.buffer, regions, xgus.nr_regions) )
> +        {
> +            rc =3D -EFAULT;
> +            goto unallocated_out;
> +        }
> +
> +        if ( __copy_to_guest(arg, &xgus, 1) )
> +            rc =3D -EFAULT;
> +
> +unallocated_out:
> +        rcu_unlock_domain(d);
> +        xfree(regions);
> +
> +        break;
> +    }
> +
>      default:
>          rc =3D arch_memory_op(cmd, arg);
>          break;
> diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
> index ded74d2..99a2824 100644
> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -359,6 +359,10 @@ void clear_and_clean_page(struct page_info *page);
>=20
>  unsigned int arch_get_dma_bitsize(void);
>=20
> +int arch_get_unallocated_space(struct domain *d,
> +                               struct xen_unallocated_region *regions,
> +                               unsigned int *nr_regions);
> +
>  #endif /*  __ARCH_ARM_MM__ */
>  /*
>   * Local variables:
> diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
> index cb90527..6244bc4 100644
> --- a/xen/include/asm-x86/mm.h
> +++ b/xen/include/asm-x86/mm.h
> @@ -652,4 +652,12 @@ static inline bool arch_mfn_in_directmap(unsigned
> long mfn)
>      return mfn <=3D (virt_to_mfn(eva - 1) + 1);
>  }
>=20
> +static inline
> +int arch_get_unallocated_space(struct domain *d,
> +                               struct xen_unallocated_region *regions,
> +                               unsigned int *nr_regions)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
>  #endif /* __ASM_X86_MM_H__ */
> diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
> index 383a946..040201b 100644
> --- a/xen/include/public/memory.h
> +++ b/xen/include/public/memory.h
> @@ -739,7 +739,42 @@ struct xen_vnuma_topology_info {
>  typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
>  DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
>=20
> -/* Next available subop number is 29 */
> +/*
> + * Get the unallocated space (regions of guest physical address space wh=
ich
> + * are unused) and can be used to create grant/foreign mappings.
> + */
> +#define XENMEM_get_unallocated_space 29
> +struct xen_unallocated_region {
> +    xen_pfn_t start_gpfn;
> +    xen_ulong_t nr_gpfns;
> +};
> +typedef struct xen_unallocated_region xen_unallocated_region_t;
> +DEFINE_XEN_GUEST_HANDLE(xen_unallocated_region_t);
> +
> +#define XEN_MAX_UNALLOCATED_REGIONS 32
> +
> +struct xen_get_unallocated_space {
> +    /* IN - Which domain to provide unallocated space for */
> +    domid_t domid;
> +
> +    /*
> +     * IN/OUT - As an IN parameter number of memory regions which
> +     *          can be written to the buffer (maximum size of the array)
> +     *          As OUT parameter number of memory regions which
> +     *          have been written to the buffer
> +     */
> +    unsigned int nr_regions;
> +
> +    /*
> +     * OUT - An array of memory regions, the regions must be placed in
> +     *       ascending order, there must be no overlap between them.
> +     */
> +    XEN_GUEST_HANDLE(xen_unallocated_region_t) buffer;
> +};
> +typedef struct xen_get_unallocated_space xen_get_unallocated_space_t;
> +DEFINE_XEN_GUEST_HANDLE(xen_get_unallocated_space_t);
> +
> +/* Next available subop number is 30 */
>=20
>  #endif /* __XEN_PUBLIC_MEMORY_H__ */
>=20
> diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
> index 363c6d7..2761fbd 100644
> --- a/xen/include/xsm/dummy.h
> +++ b/xen/include/xsm/dummy.h
> @@ -772,3 +772,9 @@ static XSM_INLINE int
> xsm_domain_resource_map(XSM_DEFAULT_ARG struct domain *d)
>      XSM_ASSERT_ACTION(XSM_DM_PRIV);
>      return xsm_default_action(action, current->domain, d);
>  }
> +
> +static XSM_INLINE int xsm_get_unallocated_space(XSM_DEFAULT_ARG
> struct domain *d)
> +{
> +    XSM_ASSERT_ACTION(XSM_HOOK);
> +    return xsm_default_action(action, current->domain, d);
> +}
> diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
> index ad3cddb..b01619c 100644
> --- a/xen/include/xsm/xsm.h
> +++ b/xen/include/xsm/xsm.h
> @@ -180,6 +180,7 @@ struct xsm_operations {
>      int (*dm_op) (struct domain *d);
>      int (*xen_version) (uint32_t cmd);
>      int (*domain_resource_map) (struct domain *d);
> +    int (*get_unallocated_space) (struct domain *d);
>  #ifdef CONFIG_ARGO
>      int (*argo_enable) (const struct domain *d);
>      int (*argo_register_single_source) (const struct domain *d,
> @@ -701,6 +702,11 @@ static inline int
> xsm_domain_resource_map(xsm_default_t def, struct domain *d)
>      return xsm_ops->domain_resource_map(d);
>  }
>=20
> +static inline int xsm_get_unallocated_space(xsm_default_t def, struct
> domain *d)
> +{
> +    return xsm_ops->get_unallocated_space(d);
> +}
> +
>  #ifdef CONFIG_ARGO
>  static inline int xsm_argo_enable(const struct domain *d)
>  {
> diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
> index de44b10..45efadb 100644
> --- a/xen/xsm/dummy.c
> +++ b/xen/xsm/dummy.c
> @@ -151,6 +151,7 @@ void __init xsm_fixup_ops (struct xsm_operations
> *ops)
>      set_to_dummy_if_null(ops, dm_op);
>      set_to_dummy_if_null(ops, xen_version);
>      set_to_dummy_if_null(ops, domain_resource_map);
> +    set_to_dummy_if_null(ops, get_unallocated_space);
>  #ifdef CONFIG_ARGO
>      set_to_dummy_if_null(ops, argo_enable);
>      set_to_dummy_if_null(ops, argo_register_single_source);
> diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
> index f1a1217..38a9b20 100644
> --- a/xen/xsm/flask/hooks.c
> +++ b/xen/xsm/flask/hooks.c
> @@ -1715,6 +1715,11 @@ static int flask_domain_resource_map(struct
> domain *d)
>      return current_has_perm(d, SECCLASS_DOMAIN2,
> DOMAIN2__RESOURCE_MAP);
>  }
>=20
> +static int flask_get_unallocated_space(struct domain *d)
> +{
> +    return current_has_perm(d, SECCLASS_DOMAIN2,
> DOMAIN2__GET_UNALLOCATED_SPACE);
> +}
> +
>  #ifdef CONFIG_ARGO
>  static int flask_argo_enable(const struct domain *d)
>  {
> @@ -1875,6 +1880,7 @@ static struct xsm_operations flask_ops =3D {
>      .dm_op =3D flask_dm_op,
>      .xen_version =3D flask_xen_version,
>      .domain_resource_map =3D flask_domain_resource_map,
> +    .get_unallocated_space =3D flask_get_unallocated_space,
>  #ifdef CONFIG_ARGO
>      .argo_enable =3D flask_argo_enable,
>      .argo_register_single_source =3D flask_argo_register_single_source,
> diff --git a/xen/xsm/flask/policy/access_vectors
> b/xen/xsm/flask/policy/access_vectors
> index 6359c7f..3cbdc19 100644
> --- a/xen/xsm/flask/policy/access_vectors
> +++ b/xen/xsm/flask/policy/access_vectors
> @@ -245,6 +245,8 @@ class domain2
>      resource_map
>  # XEN_DOMCTL_get_cpu_policy
>      get_cpu_policy
> +# XENMEM_get_unallocated_space
> +    get_unallocated_space
>  }
>=20
>  # Similar to class domain, but primarily contains domctls related to HVM
> domains
> --
> 2.7.4
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:05:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:05:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180663.327402 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNX2d-0005Wl-RM; Tue, 07 Sep 2021 09:05:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180663.327402; Tue, 07 Sep 2021 09:05:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNX2d-0005We-Nq; Tue, 07 Sep 2021 09:05:19 +0000
Received: by outflank-mailman (input) for mailman id 180663;
 Tue, 07 Sep 2021 09:05:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNX2c-0005WY-QM
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:05:18 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNX2b-00005T-PO; Tue, 07 Sep 2021 09:05:17 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNX2b-0004Qi-Eu; Tue, 07 Sep 2021 09:05:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=o9uZx/FtwTATu6iK2Vt8bQFkyHvBw8HnDgDM6Gd6PC0=; b=T6r3AcXHqiT1ZEsFp6ljuOmC43
	Kk/cA7SpQCm0UrvFCRIWFB68miptBD4zmrLgkFZYciCMpnEAJn5sVRlMRdI3g4jq/V34ftAe0AxOG
	bZ2dx2nM9JBxxUG0yyoeELg3U7KZO90h4mTjFPpBK88DS+kt89E7ybntIOTcWmXj5gzk=;
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
To: Rahul Singh <rahul.singh@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <1f4a0284-302e-47e9-6958-47dca449d580@xen.org>
Date: Tue, 7 Sep 2021 10:05:15 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Rahul,

On 19/08/2021 13:02, Rahul Singh wrote:
> XEN during boot will read the PCI device tree node “reg” property
> and will map the PCI config space to the XEN memory.
> 
> As of now "pci-host-ecam-generic" compatible board is supported.

I think the word "only" is missing.

> 
> "linux,pci-domain" device tree property assigns a fixed PCI domain
> number to a host bridge, otherwise an unstable (across boots) unique
> number will be assigned by Linux.This property has to be in sync with

Typo: missing space after the '.'

> XEN to access the PCI devices.

I would expand a little bit the last sentence to explain why the need to 
be sync-ed.

> > XEN will read the “linux,pci-domain” property from the device tree node
> and configure the host bridge segment number accordingly. If this
> property is not available XEN will allocate the unique segment number
> to the host bridge.
> 
> dt_get_pci_domain_nr(..) and dt_pci_bus_find_domain_nr(..) are directly
> imported from the Linux source tree.

What was the Linux commit used? I also read "directly imported" as a 
verbartim copy but AFAICT the implementation has been slightly reworked.

> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>   xen/arch/arm/pci/Makefile           |   2 +
>   xen/arch/arm/pci/pci-host-common.c  | 261 ++++++++++++++++++++++++++++
>   xen/arch/arm/pci/pci-host-generic.c |  55 ++++++
>   xen/include/asm-arm/pci.h           |  28 +++
>   4 files changed, 346 insertions(+)
>   create mode 100644 xen/arch/arm/pci/pci-host-common.c
>   create mode 100644 xen/arch/arm/pci/pci-host-generic.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index a9ee0b9b44..f3d97f859e 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -1,2 +1,4 @@
>   obj-y += pci.o
>   obj-y += pci-access.o
> +obj-y += pci-host-generic.o
> +obj-y += pci-host-common.o
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> new file mode 100644
> index 0000000000..9dd9b02271
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -0,0 +1,261 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/ecam.c
> + * Copyright 2016 Broadcom.
> + *
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
> + *
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>

AFAICT, <xen/pci.h> already includes <asm/pci.h>. So this looks 
unneccessary.

> +#include <xen/rwlock.h>
> +#include <xen/sched.h>
> +#include <xen/vmap.h>
> +
> +/*
> + * List for all the pci host bridges.
> + */
> +
> +static LIST_HEAD(pci_host_bridges);
> +
> +static atomic_t domain_nr = ATOMIC_INIT(-1);
> +
> +bool dt_pci_parse_bus_range(struct dt_device_node *dev,
> +                            struct pci_config_window *cfg)

Aside, "pci_config_window", the function is not Arm specific. Would it 
be possible to consider to introduce "struct resource" in Xen so this 
function can be moved in common/device_tree.c?

> +{
> +    const __be32 *cells;
> +    uint32_t len;
> +
> +    cells = dt_get_property(dev, "bus-range", &len);
> +    /* bus-range should at least be 2 cells */
> +    if ( !cells || (len < (sizeof(*cells) * 2)) )
> +        return false;

How about introducing dt_property_read_u32_array()?

> +
> +    cfg->busn_start = dt_next_cell(1, &cells);
> +    cfg->busn_end = dt_next_cell(1, &cells);
> +
> +    return true;
> +}
> +
> +static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
> +{
> +    return ioremap_nocache(start, len);
> +}
> +
> +static void pci_ecam_free(struct pci_config_window *cfg)
> +{
> +    if ( cfg->win )
> +        iounmap(cfg->win);
> +
> +    xfree(cfg);
> +}
> +
> +static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
> +                                              int ecam_reg_idx)
> +{
> +    int err;
> +    struct pci_config_window *cfg;
> +    paddr_t addr, size;
> +
> +    cfg = xzalloc(struct pci_config_window);
> +    if ( !cfg )
> +        return NULL;
> +
> +    err = dt_pci_parse_bus_range(dev, cfg);
> +    if ( !err ) {
> +        cfg->busn_start = 0;
> +        cfg->busn_end = 0xff;
> +        printk(XENLOG_ERR "%s:No bus range found for pci controller\n",

Typo: Missing space after ':'.

> +               dt_node_full_name(dev));
> +    } else {
> +        if ( cfg->busn_end > cfg->busn_start + 0xff )
> +            cfg->busn_end = cfg->busn_start + 0xff;
> +    }
> +
> +    /* Parse our PCI ecam register address*/
> +    err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size);
> +    if ( err )
> +        goto err_exit;
> +
> +    cfg->phys_addr = addr;
> +    cfg->size = size;
> +
> +    /*
> +     * On 64-bit systems, we do a single ioremap for the whole config space
> +     * since we have enough virtual address range available.  On 32-bit, we

In Xen on Arm64, the VMAP is actually only 1GB. So it is not that big 
and this will compete with other mapping like ITS, global domain mapping...

So I think the vMAP area will need to increase to cater to increase usage.

> +     * ioremap the config space for each bus individually.
> +     *
> +     * As of now only 64-bit is supported 32-bit is not supported.
> +     */
> +    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
> +    if ( !cfg->win )
> +        goto err_exit_remap;
> +
> +    printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr,

The physical address is a paddr_t. So this needs to use PRIpaddr. Also, 
please use preprent hexadecimal with 0x. This makes a lot easier to 
differentiate hexa vs decimal in the log.

> +            cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end);
> +
> +    return cfg;
> +
> +err_exit_remap:
> +    printk(XENLOG_ERR "ECAM ioremap failed\n");
> +err_exit:
> +    pci_ecam_free(cfg);

Coding style: Please add a new line before return.

> +    return NULL;
> +}
> +
> +struct pci_host_bridge *pci_alloc_host_bridge(void)
> +{
> +    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
> +
> +    if ( !bridge )
> +        return NULL;
> +
> +    INIT_LIST_HEAD(&bridge->node);
> +    bridge->bus_start = ~0;
> +    bridge->bus_end = ~0;

Coding style: Please add a new line before return.


> +    return bridge;
> +}
> +
> +void pci_add_host_bridge(struct pci_host_bridge *bridge)
> +{
> +    list_add_tail(&bridge->node, &pci_host_bridges);
> +}
> +
> +static int pci_get_new_domain_nr(void)
> +{
> +    return atomic_inc_return(&domain_nr);
> +}
> +
> +/*
> + * This function will try to obtain the host bridge domain number by
> + * finding a property called "linux,pci-domain" of the given device node.
> + *
> + * @node: device tree node with the domain information
> + *
> + * Returns the associated domain number from DT in the range [0-0xffff], or
> + * a negative value if the required property is not found.
> + */
> +static int dt_get_pci_domain_nr(struct dt_device_node *node)

Nothing looks Arm specific for this function. Can you move it in 
device_tree.c?

> +{
> +    u32 domain;
> +    int error;
> +
> +    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
> +    if ( !error )
> +        return -EINVAL;
> +
> +    return (u16)domain;
> +}
> +
> +static int pci_bus_find_domain_nr(struct dt_device_node *dev)
> +{
> +    static int use_dt_domains = -1;
> +    int domain;
> +
> +    domain = dt_get_pci_domain_nr(dev);
> +
> +    /*
> +     * Check DT domain and use_dt_domains values.
> +     *
> +     * If DT domain property is valid (domain >= 0) and
> +     * use_dt_domains != 0, the DT assignment is valid since this means
> +     * we have not previously allocated a domain number by using
> +     * pci_get_new_domain_nr(); we should also update use_dt_domains to
> +     * 1, to indicate that we have just assigned a domain number from
> +     * DT.
> +     *
> +     * If DT domain property value is not valid (ie domain < 0), and we
> +     * have not previously assigned a domain number from DT
> +     * (use_dt_domains != 1) we should assign a domain number by
> +     * using the:
> +     *
> +     * pci_get_new_domain_nr()
> +     *
> +     * API and update the use_dt_domains value to keep track of method we
> +     * are using to assign domain numbers (use_dt_domains = 0).
> +     *
> +     * All other combinations imply we have a platform that is trying
> +     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
> +     * which is a recipe for domain mishandling and it is prevented by
> +     * invalidating the domain value (domain = -1) and printing a
> +     * corresponding error.
> +     */
> +    if ( domain >= 0 && use_dt_domains )
> +    {
> +        use_dt_domains = 1;
> +    }
> +    else if ( domain < 0 && use_dt_domains != 1 )
> +    {
> +        use_dt_domains = 0;
> +        domain = pci_get_new_domain_nr();
> +    }
> +    else
> +    {
> +        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
> +        BUG();

I don't think pci_bus_find_domain_nr() should be the function that 
crashes Xen. Instead, this should be propagated to the highest possible 
callers and let it decide what to do.

> +    }
> +
> +    return domain;
> +}
> +
> +int pci_host_common_probe(struct dt_device_node *dev,
> +                          int ecam_reg_idx)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct pci_config_window *cfg;
> +    int err;
> +
> +    bridge = pci_alloc_host_bridge();
> +    if ( !bridge )
> +        return -ENOMEM;
> +
> +    /* Parse and map our Configuration Space windows */
> +    cfg = gen_pci_init(dev, ecam_reg_idx);
> +    if ( !cfg )
> +    {
> +        err = -ENOMEM;
> +        goto err_exit;
> +    }
> +
> +    bridge->dt_node = dev;
> +    bridge->sysdata = cfg;
> +    bridge->bus_start = cfg->busn_start;
> +    bridge->bus_end = cfg->busn_end;
> +
> +    bridge->segment = pci_bus_find_domain_nr(dev);
> +
> +    pci_add_host_bridge(bridge);
> +
> +    return 0;
> +
> +err_exit:
> +    xfree(bridge);
> +    return err;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> new file mode 100644
> index 0000000000..13d0f7f999
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -0,0 +1,55 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static const struct dt_device_match gen_pci_dt_match[] = {
> +    { .compatible = "pci-host-ecam-generic" },
> +    { },
> +};
> +
> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
> +{
> +    const struct dt_device_match *of_id;
> +
> +    of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);

This seems to be a bit pointless to me as you already know the 
compatible (there is only one possible...).

> +
> +    printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n",
> +           dt_node_full_name(dev), of_id->compatible);
> +
> +    return pci_host_common_probe(dev, 0);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> +.dt_match = gen_pci_dt_match,
> +.init = gen_pci_dt_init,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 61e43da088..58a51e724e 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,34 @@ struct arch_pci_dev {
>       struct device dev;
>   };
>   
> +/*
> + * struct to hold the mappings of a config space window. This
> + * is expected to be used as sysdata for PCI controllers that
> + * use ECAM.
> + */
> +struct pci_config_window {
> +    paddr_t         phys_addr;
> +    paddr_t         size;
> +    uint8_t         busn_start;
> +    uint8_t         busn_end;
> +    void __iomem    *win;
> +};
> +
> +/*
> + * struct to hold pci host bridge information
> + * for a PCI controller.
> + */
> +struct pci_host_bridge {
> +    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
> +    struct list_head node;           /* Node in list of host bridges */
> +    uint16_t segment;                /* Segment number */
> +    u8 bus_start;                    /* Bus start of this bridge. */
> +    u8 bus_end;                      /* Bus end of this bridge. */

Please use uint8_t rather than u8.

> +    void *sysdata;                   /* Pointer to the config space window*/
> +};
> +
> +int pci_host_common_probe(struct dt_device_node *dev,
> +                          int ecam_reg_idx);
>   #else   /*!CONFIG_HAS_PCI*/
>   
>   struct arch_pci_dev { };
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:07:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:07:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180670.327413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNX4x-0006Dl-Dh; Tue, 07 Sep 2021 09:07:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180670.327413; Tue, 07 Sep 2021 09:07:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNX4x-0006De-9Z; Tue, 07 Sep 2021 09:07:43 +0000
Received: by outflank-mailman (input) for mailman id 180670;
 Tue, 07 Sep 2021 09:07:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNX4v-0006DW-Cn
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:07:41 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0a95279b-1286-4476-a61b-23b964065993;
 Tue, 07 Sep 2021 09:07:40 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1877hNKW015205; 
 Tue, 7 Sep 2021 09:07:36 GMT
Received: from eur02-he1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3awwwf94hj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 09:07:36 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB6982.eurprd03.prod.outlook.com (2603:10a6:20b:290::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 09:07:27 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 09:07:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a95279b-1286-4476-a61b-23b964065993
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f+ta1QLp89UHIR9J/cwaHW0np4ecTqJ4FmG/2+AAujQ2YteC6/Frx0oEQIH4103DAQWCSKTGpg83FOa4Y+eHBmel26nkFoVuLQPyp+yTyit6g1+u1ZFco/ClGoI51SakDu/LYnRogjibxvy+lcjdj/YRYdAMY2BW1+wJNlCrhcBUHyznVCZo80JHBuAtSO/M1U6hqZqmpmKj4Tfsu/dhmlVZL1NfGFNkhZ2KSXvSxhqyz+TFfVlpyZUL6SaXVraieFXxIxk2ZTN4bdbt7XU5dcpvnybr3uPGN4dZU+Zw+2AAP4rPyfJ2OhHFlmr/E31Dsbk4JE2AJ7aQpN+jgaD0NQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=iU06uG9UQghA0xEp3Kzkgh9kVYDqeW2pkTfpg5vmOdM=;
 b=eL3KyX+GJhjuPeIG96P/qSrdMTxvxsQ6grnVshpgnAjyNeqWYj+egtBO7AexJCKhMvpUws2rW4hLAN6KnR0kJbSFhjcXEilq9kwRorisKULGBvvFUhwZE1AOnD3ibm8pLQelAyilOhL9H9UIfASmNlYOe+WOfwnpM1gBKdZFD0vOyOXKmY+q1Km2GvqRafZjJ2RRvaIXr7yDhpNp9gtlPqhfr57bIHTdnhxMfX/UFSRbgRc0SloAdsU65Gazp5iDBAXncLGhPm+qOZzsl73mHL2q6kbAs6o6BDMWK/NNtw82BvTLewVEvINcpSVlXO3jbdRBVP0QGKv/6QbtLio6qA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iU06uG9UQghA0xEp3Kzkgh9kVYDqeW2pkTfpg5vmOdM=;
 b=yRDwwYuhb8aPYAHqWR4lpaCDun7U7me78zYKjU8giykZZeH/bCC5Fp5Bvu+LsszkRGjrmu/8ebFsYyEPJRY6MS4vO4tJeMCvR6M0VVtfAzohEpCj9ox6rYuCuzlmkffsjk8/DBHEfUTVWGv/6m1LJETFOf/jZq23lHPolFWbxBj7JuIRcMcPzRjYfKEB97zcCF1ruO/qSrskiaLq6Y0AYiMHvm6cILCQ5JxTw8MDmXqIuCk1N9OsB18xrIljvfcdvPL+wVwTpmDGTE05/7+zmTcvpZmR0s3DWn5xGcqk5hf64H6zLXFYGtDsJSkZQ4ppk7cL7iq4e3wZDweD1qtDcw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AA==
Date: Tue, 7 Sep 2021 09:07:27 +0000
Message-ID: <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
In-Reply-To: <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 92bba6b3-6a9c-491e-e1b7-08d971deea26
x-ms-traffictypediagnostic: AS8PR03MB6982:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB69827F167C1F893442A3A0C2E7D39@AS8PR03MB6982.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 CLqJcsRE6v6umDppl37yYDWl3klabIMmmbDDF/+Wxd5fGGlVe3zaAamXHl6P+dHPY7gWDLjqr7wV2lbaFNvkL/riiDyHvb7XdtYDiSzcCZNoJ259/F9QlflOvdJRIn2djFC/T9kGixgbE/P9bKzsnjWR6A25bjL5eqVIiZ0nJecWuTghtBrurU8IBYWgYIzQ41IQn5knDMW/ZgJGZmf82BYmZiOJ2GJLWXo2fJkc61RcfzcaGb17X86gbEmHiaXK35/nv8POAO2UlXPeVRcRp3NgZV6A9WGiA8kpmN7mwJS45+0N07fpwzePE9WWRInZpsvg4ElbWGA1GNAi/zSDb90vY4jEv8PEEoR5crJCbDXPgAEacSKC5MyH/vGe8E8poFoIy4/HdfXrI9oxExZDzXTFnyo6Pe0r426JWvA10cf8X614tH0Zs2nE/ldCmZySiiI+ZhpvLf/aTY4mOyGrsnDSjMdJqHAjRGt9nzXlt6bo/blt4wWYhE3DHwe7DHu6RDtGKiiqRLO25ETxkgLhgzvz1wpW3geGYL1shYqlCbIDdna2EFjrW9dOJEMmK0mowVwyXULdjSsv+0tgSBRvwqvt7YDDkvrml00/u3BNNIZtaT6nVWL9XeNSW1hn0ODhqL3JugGFTcdUAPOhbxRYtLI2hx8vAxAIPwskHR1l96uwub/THpw/rx83QJ7nu/eKnU2TLcKBB0WYt/6bLOJ+iwrW5/oShWyxQDjB5bZYZnh67tGRW7ack4+SSNVwi9eD
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(136003)(39860400002)(376002)(346002)(8936002)(2616005)(66556008)(66446008)(66476007)(8676002)(66946007)(26005)(36756003)(64756008)(316002)(2906002)(4326008)(6486002)(76116006)(71200400001)(91956017)(31686004)(53546011)(110136005)(54906003)(122000001)(5660300002)(478600001)(38100700002)(186003)(6512007)(38070700005)(86362001)(83380400001)(31696002)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UExGMWdDeE1VTld2RHhzOGVvRjlHRkVDVVp0U3FlcFhTUzI5UWtJNW5TdDNn?=
 =?utf-8?B?VWcyRkR5NmJodHdtdGk0YVFGSXM0T0J1MG5iMzVmYTNkQmc2d0dRYXpwK1pB?=
 =?utf-8?B?UjYvZDNpSi9LYXgxNCtNaTdpL2pERnZKYzZIa0Y4WHNzSHBGVGdjNGt1WXJP?=
 =?utf-8?B?N205eDYzVnFkTmNSOEhQSlN1UXR2RzFXYzVGeTlvSkpWcWZXY2ZJWW92T0lo?=
 =?utf-8?B?Q29MZ25iRHdtTitGUHhSMjJIK1FCWjhwWFV5QjlJZFJwSVp3ODEydG5NL2JK?=
 =?utf-8?B?RzMzRzZwYmUyY3lkRkx4MW9nTUYrNzJ6aFVHOGRORHh2bkhxWFdLT2E2R3d5?=
 =?utf-8?B?SE00RzRUSUJqOWtJRzNvSWFGTTA0QmxGdmk2NEl0Nmt5bEVROXpGakszVGUw?=
 =?utf-8?B?ZityTW00dTN4OXRuY2wrTVFERFI1R1VodFFkNDRGc240UVJxVkpJN290TGVG?=
 =?utf-8?B?Sk5NU1pCcDF5cWI1aGFTTHd1MlpGck1DY1ZNN0ZzamZVcTNCTUJORU1aSWpB?=
 =?utf-8?B?YTVKOW03dzRDNk0wcWMwdS9SSzA3TXZPVCtTUk5uMFh6Y3Z3cTVnNnB3WGNt?=
 =?utf-8?B?Y3pqdUxSQkcwQmVNcVBSRlFYQUY5aDc2SDNuaVVzT1hzK2NvbHVJNkNQRm4r?=
 =?utf-8?B?dEFOZmZwY0ZTMGVRVitkeFVPMTNxVXRrRHEvRXBkb1ZMckFWT0dBeE5PMkk3?=
 =?utf-8?B?TVhhamVIbzZTOHVVUjJRbFl3RFNWYzZiTWlCQTVocjB5VFNFSzdTUXp4cVlC?=
 =?utf-8?B?azdrZzBKU1IxT2hJSkNZZGJqeGpwZDlXZTJXaGNFL3QrUHZXNi9xeStlVzZW?=
 =?utf-8?B?RTBJTm02Rkh4S3lQSHBSVldqQ2dRUmxhKzB0SzVkNUI3c21wNU1mbFJhdjM1?=
 =?utf-8?B?NUJYTWJOVUUrMzVtMTFkYmZtYVQxZUlmOG1KdnRoSWZ1bldpTzdwY2xBNmRh?=
 =?utf-8?B?aFh3cXppUkR4b3dRSS9wbWVsN1RYUzRHMXBVWWw4ZStoTk1BOEpqdVdmRlhU?=
 =?utf-8?B?NVV2cDFzT3BWcjR2bG1GTFI2aTlHelhwcWtaUG8vbHJ6YW9acEdsR2Ztclhr?=
 =?utf-8?B?N0tON1Q4cFc3MEMyeGRDL3FBQTdkVllKdXpISlNrdVhrZDRFbE82NWpQdXVF?=
 =?utf-8?B?NHl5Z25PcHp2OXk0MlNjNmNIcENsTXFOdmxZQWQ4WnowTGtWR3RYMWRlaFg1?=
 =?utf-8?B?amxEWkZERjVyVWlzT3NKSjZWQjBKejZUVkJNbW9rSjhvemlpQ1FjQVFDSjF0?=
 =?utf-8?B?b0VMUWZyZVN6TkZYSWwyd2ZmRjlqNDFQd3RNMXB3VG5uYkEvRUdRV1A3VEk1?=
 =?utf-8?B?ZzJucktMWVBkYmVveE5KZlJPalBCcEhOOENTZW1zalVxTXpGSklJTXFFelBQ?=
 =?utf-8?B?ZzVPTkc0R3dvLzVseHhQeUNZNFF0MjMySS9IMnlZb0Vvdy9SYk1LNGI0WnJp?=
 =?utf-8?B?NEZsSFI0bVgyUDBkbE5CZ2pwRHhrbTVoemFiMlk5Q1Bodk1WSW1zNzBhVW00?=
 =?utf-8?B?VUZFcWFTeWliQkJoN25zRll5S2pMQVliUDY2bEx2RkViQUNwK2U0MS9lZ0tX?=
 =?utf-8?B?dlE4Z3VNNm9UK21mdnZvVHhyalBYVlhZcVVrbm5TTmFveVord01uUzc3aGdK?=
 =?utf-8?B?N0VxVXI4cG0vSUNhc24wOHg5VFhuejgzOUNpMDJEWk9yRDFxV3YzU3ZYNTgw?=
 =?utf-8?B?amIyeTd2bTFoZ2lCY0dUdG5aWU9WR1pTcjBXRklCS0ViOGlkdmFmb3AweHF1?=
 =?utf-8?Q?Q9pPfwWuiDm0Lk9DPtCvyyEIv6gJmtqxVa76iLF?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <5294CDD42335214FA71A1334D7A6EDD8@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 92bba6b3-6a9c-491e-e1b7-08d971deea26
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 09:07:27.3924
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sWMAApXv0ofYfZSHYR7IdbA5Dr24P3Ssd/xx55njutynxSFuw13PnM9e16YiXhOJzbu2J2dvXHcllfkQMLKMxgzf96yrj/SeJEGKKeIpEzpEpJ+XFOM0Gy+PN4NvZoQw
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB6982
X-Proofpoint-GUID: fHIcg41XHYZlkLIiiZ7b4H4uxl28MIOu
X-Proofpoint-ORIG-GUID: fHIcg41XHYZlkLIiiZ7b4H4uxl28MIOu
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_03,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0
 bulkscore=0 phishscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0
 spamscore=0 mlxscore=0 mlxlogscore=999 impostorscore=0 priorityscore=1501
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109070060

DQpPbiAwNy4wOS4yMSAxMTo0OSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTA6MTgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTE6
MDAsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMDk6NDMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNi4wOS4yMSAxNzo1NSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hlYWRlci5jDQo+Pj4+
Pj4gKysrIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4+Pj4+IEBAIC04MTEsNiArODEx
LDE2IEBAIGludCB2cGNpX2Jhcl9hZGRfaGFuZGxlcnMoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCwg
c3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+Pj4+Pj4gICAgICAgICAgICAgZ3ByaW50ayhYRU5MT0df
RVJSLA0KPj4+Pj4+ICAgICAgICAgICAgICAgICAiJXBwOiBmYWlsZWQgdG8gYWRkIEJBUiBoYW5k
bGVycyBmb3IgZG9tJWRcbiIsICZwZGV2LT5zYmRmLA0KPj4+Pj4+ICAgICAgICAgICAgICAgICBk
LT5kb21haW5faWQpOw0KPj4+Pj4+ICsNCj4+Pj4+PiArICAgIC8qDQo+Pj4+Pj4gKyAgICAgKiBS
ZXNldCB0aGUgY29tbWFuZCByZWdpc3RlcjogaXQgaXMgcG9zc2libGUgdGhhdCB3aGVuIHBhc3Np
bmcNCj4+Pj4+PiArICAgICAqIHRocm91Z2ggYSBQQ0kgZGV2aWNlIGl0cyBtZW1vcnkgZGVjb2Rp
bmcgYml0cyBpbiB0aGUgY29tbWFuZA0KPj4+Pj4+ICsgICAgICogcmVnaXN0ZXIgYXJlIGFscmVh
ZHkgc2V0LiBUaHVzLCBhIGd1ZXN0IE9TIG1heSBub3Qgd3JpdGUgdG8gdGhlDQo+Pj4+Pj4gKyAg
ICAgKiBjb21tYW5kIHJlZ2lzdGVyIHRvIHVwZGF0ZSBtZW1vcnkgZGVjb2RpbmcsIHNvIGd1ZXN0
IG1hcHBpbmdzDQo+Pj4+Pj4gKyAgICAgKiAoZ3Vlc3QncyB2aWV3IG9mIHRoZSBCQVJzKSBhcmUg
bGVmdCBub3QgdXBkYXRlZC4NCj4+Pj4+PiArICAgICAqLw0KPj4+Pj4+ICsgICAgcGNpX2NvbmZf
d3JpdGUxNihwZGV2LT5zYmRmLCBQQ0lfQ09NTUFORCwgMCk7DQo+Pj4+PiBDYW4geW91IHJlYWxs
eSBibGluZGx5IHdyaXRlIDAgaGVyZT8gV2hhdCBhYm91dCBiaXRzIHRoYXQgaGF2ZSB0byBiZQ0K
Pj4+Pj4gdW5kZXIgaG9zdCBjb250cm9sLCBlLmcuIElOVFhfRElTQUJMRT8gSSBjYW4gc2VlIHRo
YXQgeW91IG1heSB3YW50IHRvDQo+Pj4+PiBoYW5kIG9mZiB3aXRoIEkvTyBhbmQgbWVtb3J5IGRl
Y29kaW5nIG9mZiBhbmQgYnVzIG1hc3RlcmluZyBkaXNhYmxlZCwNCj4+Pj4+IGJ1dCBmb3IgZXZl
cnkgb3RoZXIgYml0IChpbmNsdWRpbmcgcmVzZXJ2ZWQgb25lcykgSSdkIGV4cGVjdCBzZXBhcmF0
ZQ0KPj4+Pj4ganVzdGlmaWNhdGlvbiAoaW4gdGhlIGNvbW1pdCBtZXNzYWdlKS4NCj4+Pj4gQWNj
b3JkaW5nIHRvICJQQ0kgTE9DQUwgQlVTIFNQRUNJRklDQVRJT04sIFJFVi4gMy4wIiBJIGhhdmUg
YXQgaGFuZCwNCj4+Pj4gc2VjdGlvbiAiNi4yLjIgRGV2aWNlIENvbnRyb2wiIHNheXMgdGhhdCB0
aGUgcmVzZXQgc3RhdGUgb2YgdGhlIGNvbW1hbmQNCj4+Pj4gcmVnaXN0ZXIgaXMgdHlwaWNhbGx5
IDAsIHNvIHRoaXMgaXMgd2h5IEkgY2hvc2UgdG8gd3JpdGUgMCBoZXJlLCBlLmcuDQo+Pj4+IG1h
a2UgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgYXMgaWYgaXQgaXMgYWZ0ZXIgdGhlIHJlc2V0Lg0KPj4+
Pg0KPj4+PiBXaXRoIHJlc3BlY3QgdG8gaG9zdCBjb250cm9sOiB3ZSBjdXJyZW50bHkgZG8gbm90
IHJlYWxseSBlbXVsYXRlIGNvbW1hbmQNCj4+Pj4gcmVnaXN0ZXIgd2hpY2ggcHJvYmFibHkgd2Fz
IG9rIGZvciB4ODYgUFZIIERvbTAgYW5kIHRoaXMgbWlnaHQgbm90IGJlIHRoZQ0KPj4+PiBjYXNl
IG5vdyBhcyB3ZSBhZGQgRG9tVSdzLiBUaGF0IGJlaW5nIHNhaWQ6IGluIG15IGltcGxlbWVudGF0
aW9uIGd1ZXN0IGNhbg0KPj4+PiBhbHRlciBjb21tYW5kIHJlZ2lzdGVyIGFzIGl0IHdhbnRzIHdp
dGhvdXQgcmVzdHJpY3Rpb25zLg0KPj4+PiBJZiB3ZSBzZWUgaXQgZG9lcyBuZWVkIHByb3BlciBl
bXVsYXRpb24gdGhlbiB3ZSB3b3VsZCBuZWVkIGFkZGluZyB0aGF0IGFzDQo+Pj4+IHdlbGwgKGlz
IG5vdCBwYXJ0IG9mIHRoaXMgc2VyaWVzIHRob3VnaCkuDQo+Pj4+DQo+Pj4+IE1lYW53aGlsZSwg
SSBhZ3JlZSB0aGF0IHdlIGNhbiBvbmx5IHJlc2V0IElPIHNwYWNlLCBtZW1vcnkgc3BhY2UgYW5k
IGJ1cw0KPj4+PiBtYXN0ZXIgYml0cyBhbmQgbGVhdmUgdGhlIHJlc3QgdW50b3VjaGVkLiBCdXQg
YWdhaW4sIHdpdGhvdXQgcHJvcGVyIGNvbW1hbmQNCj4+Pj4gcmVnaXN0ZXIgZW11bGF0aW9uIGd1
ZXN0cyBjYW4gc3RpbGwgc2V0IHdoYXQgdGhleSB3YW50Lg0KPj4+IFllcywgd3JpdGVzIHRvIHRo
ZSByZWdpc3RlciB3aWxsIG5lZWQgZW11bGF0aW5nIGZvciBEb21VLg0KPj4gQnV0IHRoZW4gSSBh
bSB3b25kZXJpbmcgdG8gd2hhdCBleHRlbnQgd2UgbmVlZCB0byBlbXVsYXRlIHRoZSBjb21tYW5k
DQo+Pg0KPj4gcmVnaXN0ZXI/IFdlIGhhdmUgdGhlIGZvbGxvd2luZyBiaXRzIGluIHRoZSBjb21t
YW5kIHJlZ2lzdGVyOg0KPj4NCj4+ICNkZWZpbmXCoCBQQ0lfQ09NTUFORF9JT8KgwqDCoCDCoMKg
wqAgMHgxwqDCoMKgIC8qIEVuYWJsZSByZXNwb25zZSBpbiBJL08gc3BhY2UgKi8NCj4+ICNkZWZp
bmXCoCBQQ0lfQ09NTUFORF9NRU1PUlnCoMKgwqAgMHgywqDCoMKgIC8qIEVuYWJsZSByZXNwb25z
ZSBpbiBNZW1vcnkgc3BhY2UgKi8NCj4+ICNkZWZpbmXCoCBQQ0lfQ09NTUFORF9NQVNURVLCoMKg
wqAgMHg0wqDCoMKgIC8qIEVuYWJsZSBidXMgbWFzdGVyaW5nICovDQo+PiAjZGVmaW5lwqAgUENJ
X0NPTU1BTkRfU1BFQ0lBTMKgwqDCoCAweDjCoMKgwqAgLyogRW5hYmxlIHJlc3BvbnNlIHRvIHNw
ZWNpYWwgY3ljbGVzICovDQo+PiAjZGVmaW5lwqAgUENJX0NPTU1BTkRfSU5WQUxJREFURcKgwqDC
oCAweDEwwqDCoMKgIC8qIFVzZSBtZW1vcnkgd3JpdGUgYW5kIGludmFsaWRhdGUgKi8NCj4+ICNk
ZWZpbmXCoCBQQ0lfQ09NTUFORF9WR0FfUEFMRVRURSAweDIwwqDCoMKgIC8qIEVuYWJsZSBwYWxl
dHRlIHNub29waW5nICovDQo+PiAjZGVmaW5lwqAgUENJX0NPTU1BTkRfUEFSSVRZwqDCoMKgIDB4
NDDCoMKgwqAgLyogRW5hYmxlIHBhcml0eSBjaGVja2luZyAqLw0KPj4gI2RlZmluZcKgIFBDSV9D
T01NQU5EX1dBSVQgwqDCoMKgIDB4ODDCoMKgwqAgLyogRW5hYmxlIGFkZHJlc3MvZGF0YSBzdGVw
cGluZyAqLw0KPj4gI2RlZmluZcKgIFBDSV9DT01NQU5EX1NFUlLCoMKgwqAgMHgxMDDCoMKgwqAg
LyogRW5hYmxlIFNFUlIgKi8NCj4+ICNkZWZpbmXCoCBQQ0lfQ09NTUFORF9GQVNUX0JBQ0vCoMKg
wqAgMHgyMDDCoMKgwqAgLyogRW5hYmxlIGJhY2stdG8tYmFjayB3cml0ZXMgKi8NCj4+ICNkZWZp
bmXCoCBQQ0lfQ09NTUFORF9JTlRYX0RJU0FCTEUgMHg0MDAgLyogSU5UeCBFbXVsYXRpb24gRGlz
YWJsZSAqLw0KPj4NCj4+IFdlIHdhbnQgdGhlIGd1ZXN0IHRvIGFjY2VzcyBkaXJlY3RseSBhdCBs
ZWFzdCBJL08gYW5kIG1lbW9yeSBkZWNvZGluZyBhbmQgYnVzIG1hc3RlcmluZw0KPj4gYml0cywg
YnV0IGhvdyBkbyB3ZSBlbXVsYXRlIHRoZSByZXN0PyBEbyB5b3UgbWVhbiB3ZSBjYW4gbWF0Y2gg
dGhlIHJlc3QgdG8gd2hhdCBob3N0DQo+PiB1c2VzIGZvciB0aGUgZGV2aWNlLCBsaWtlIFBDSV9D
T01NQU5EX0lOVFhfRElTQUJMRSBiaXQ/IElmIHNvLCBhcyBwZXIgbXkgdW5kZXJzdGFuZGluZywN
Cj4+IHRob3NlIGJpdHMgZ2V0IHNldC9jbGVhcmVkIHdoZW4gYSBkZXZpY2UgaXMgZW5hYmxlZCwg
ZS5nLiBieSBMaW51eCBrZXJuZWwvZGV2aWNlIGRyaXZlciBmb3IgZXhhbXBsZS4NCj4gSSB3b3Vs
ZCBzdWdnZXN0IHRvIHRha2UgcWVtdSdzIGVtdWxhdGlvbiBhcyBhIHN0YXJ0aW5nIHBvaW50Lg0K
DQpTdXJlLCBJJ2xsIHRha2UgYSBsb29rIHdoYXQgUUVNVSBkb2VzLiBCdXQgSSBndWVzcyB0aGF0
IGVtdWxhdGlvbiBtYXkgZGVwZW5kDQoNCm9uIGhvc3QgYnJpZGdlIGVtdWxhdGlvbiBldGMuIHdo
aWNoIG1heSBub3QgYmUgYXBwbGljYWJsZSBmb3Igb3VyIGNhc2Ugd2l0aG91dA0KDQpzZXJpb3Vz
IGNvbXBsaWNhdGlvbnMuDQoNCj4NCj4+IFNvLCBpZiB3ZSBoYXZlIGEgaGlkZGVuIFBDSSBkZXZp
Y2Ugd2hpY2ggY2FuIGJlIGFzc2lnbmVkIHRvIGEgZ3Vlc3QgYW5kIGl0IGlzIGxpdGVyYWxseSB1
bnRvdWNoZWQNCj4+IChub3QgZW5hYmxlZCBpbiBEb20wKSB0aGVuIEkgdGhpbmsgdGhlcmUgd2ls
bCBiZSBubyBzdWNoIHJlZmVyZW5jZSBhcyAiaG9zdCBhc3NpZ25lZCB2YWx1ZXMiIGFzDQo+PiBt
b3N0IHByb2JhYmx5IHRoZSBjb21tYW5kIHJlZ2lzdGVyIHdpbGwgcmVtYWluIGluIGl0cyBhZnRl
ciByZXNldCBzdGF0ZS4NCj4gV2hhdCBtZWFuaW5nIG9mICJoaWRkZW4iIGRvIHlvdSBpbXBseSBo
ZXJlPyBEZXZpY2VzIHBhc3NlZCB0bw0KPiBwY2lfe2hpZGUscm99X2RldmljZSgpIG1heSBub3Qg
YmUgYXNzaWduZWQgdG8gZ3Vlc3RzIC4uLg0KWW91IGFyZSBjb21wbGV0ZWx5IHJpZ2h0IGhlcmUu
DQo+DQo+IEZvciBhbnkgb3RoZXIgbWVhbmluZyBvZiAiaGlkZGVuIiwgZXZlbiBpZiB0aGUgZGV2
aWNlIGlzIGNvbXBsZXRlbHkNCj4gaWdub3JlZCBieSBEb20wLA0KDQpEb20wbGVzcyBpcyBzdWNo
IGEgY2FzZSB3aGVuIGEgZGV2aWNlIGlzIGFzc2lnbmVkIHRvIHRoZSBndWVzdA0KDQp3aXRob3V0
IERvbTAgYXQgYWxsPw0KDQo+ICAgY2VydGFpbiBvZiB0aGUgcHJvcGVydGllcyBzdGlsbCBjYW5u
b3QgYmUgYWxsb3dlZA0KPiB0byBiZSBEb21VLWNvbnRyb2xsZWQuDQoNClRoZSBsaXN0IGlzIG5v
dCB0aGF0IGJpZywgY291bGQgeW91IHBsZWFzZSBuYW1lIGEgZmV3IHlvdSB0aGluayBjYW5ub3QN
Cg0KYmUgY29udHJvbGxlZCBieSBhIGd1ZXN0PyBJIGNhbiB0aGluayBvZiBQQ0lfQ09NTUFORF9T
UEVDSUFMKD8pLA0KDQpQQ0lfQ09NTUFORF9JTlZBTElEQVRFKD8pLCBQQ0lfQ09NTUFORF9QQVJJ
VFksIFBDSV9DT01NQU5EX1dBSVQsDQoNClBDSV9DT01NQU5EX1NFUlIsIFBDSV9DT01NQU5EX0lO
VFhfRElTQUJMRSB3aGljaCB3ZSBtYXkgd2FudCB0bw0KDQpiZSBhbGlnbmVkIHdpdGggdGhlICJo
b3N0IHJlZmVyZW5jZSIgdmFsdWVzLCBlLmcuIHdlIG9ubHkgYWxsb3cgdGhvc2UgYml0cw0KDQp0
byBiZSBzZXQgYXMgdGhleSBhcmUgaW4gRG9tMC4NCg0KPiAgIChJJ20gdGhlcmVmb3JlIG5vdCBz
dXJlIGluIGhvdyBmYXIgRG9tMCBjYW4NCj4gYWN0dWFsbHkgbGVnaXRpbWF0ZWx5ICJpZ25vcmUi
IGRldmljZXMuIEl0IG1heSBkZWNpZGUgdG8gbm90IGVuYWJsZQ0KPiB0aGVtLCBidXQgdGhhdCdz
IG5vdCAiaWdub3JpbmciLikNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:18:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:18:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180679.327424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXEp-0007wb-Bz; Tue, 07 Sep 2021 09:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180679.327424; Tue, 07 Sep 2021 09:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXEp-0007wU-98; Tue, 07 Sep 2021 09:17:55 +0000
Received: by outflank-mailman (input) for mailman id 180679;
 Tue, 07 Sep 2021 09:17:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNXEn-0007wO-Qd
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:17:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXEn-0000JL-0s; Tue, 07 Sep 2021 09:17:53 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXEm-0005Iv-QG; Tue, 07 Sep 2021 09:17:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=xaPEDbTrlEdxGiPgvs3r61r+qm7NtTnC/Clpi0CTVhs=; b=twy/ZRMOwHvl1PKqlHGOecN0VG
	EcNXYeGjhtAstKYZwwpqW8Os7RIjOTnrgap5Tbb8/Algg4NyE54DKRtU8mRUAN+kgEB5UAIxn8i9F
	O8GHOtDfguExvTn7bUggb641ILmDYUCFhSiEzgjyztcgDZ7M76YRNPX4Zr0fhLbKzW70=;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <ea736db5-e3cf-7eea-cffe-98ecb0290048@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <3222722f-b12e-3991-d0de-b455b9fca063@xen.org>
Date: Tue, 7 Sep 2021 10:17:50 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <ea736db5-e3cf-7eea-cffe-98ecb0290048@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 07/09/2021 09:33, Jan Beulich wrote:
> On 07.09.2021 08:52, Luca Fancellu wrote:
>> Add a design describing a proposal to improve the EFI
>> configuration file, adding keywords to describe domU
>> guests and allowing to start a dom0less system.
>>
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>> ---
>>   docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>>   1 file changed, 105 insertions(+)
>>   create mode 100644 docs/designs/efi-arm-dom0less.md
>>
>> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
>> new file mode 100644
>> index 0000000000..8d8fa2243f
>> --- /dev/null
>> +++ b/docs/designs/efi-arm-dom0less.md
>> @@ -0,0 +1,105 @@
>> +# Xen EFI configuration file
>> +
>> +The current configuration file used by Xen when it is started as an EFI
>> +application is considering only the dom0 guest and doesn't have any
>> +property to describe and load in memory domU guests.
>> +Hence currently it's impossible to start a dom0less system using EFI.
>> +
>> +# Objective
>> +
>> +This document describes the proposed improvement to the Xen EFI
>> +configuration file to list properly both the dom0 guest and the domU
>> +guests as well.
>> +The final goal is to be able to start a dom0less system using EFI.
>> +
>> +# Current Xen EFI configuration file
>> +
>> +The current configuration file is described by the documentation page
>> +https://xenbits.xenproject.org/docs/unstable/misc/efi.html.
>> +
>> +Here an example:
>> +
>> +```
>> +[global]
>> +default=section1
>> +
>> +[section1]
>> +options=console=vga,com1 com1=57600 loglvl=all noreboot
>> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>> +ramdisk=initrd-3.0.31-0.4-xen
>> +xsm=<filename>
>> +dtb=devtree.dtb
>> +```
>> +
>> +# Proposed improvement
>> +
>> +The proposed improvement to the current configuration file is the
>> +introduction of new keywords to describe additional domUs.
>> +
>> +Here follows the proposed new keywords:
>> +  - domu#_kernel=<kernel file> [domU command line options]
>> +    - Mandatory kernel file for the domU#
>> +  - domu#_ramdisk=<ramdisk file>
>> +    - Optional ramdisk file for the domU#
>> +  - domu#_dtb=<dtb file>
>> +    - Optional dtb fragment file for the domU#, it is used for device
>> +      assignment (passthrough).
>> +  - domu#_property=cpus=2
>> +    - Properties that should be added to the dtb in the domU node to
>> +      properly describe the domU guest. Refer to the documentation:
>> +      https://xenbits.xenproject.org/docs/unstable/misc/arm/device-tree/booting.txt,
>> +      section "Creating Multiple Domains directly from Xen".
>> +
>> +For all the keywords above, the # is a number that uniquely identifies
>> +the guest.
>> +The keywords domu#_kernel, domu#_ramdisk, domu#_dtb are unique, therefore there
>> +must not be specified the same keyword twice in a section.
>> +The # number is not enforcing any domid, it is just used to link each property
>> +to the right guest, so there can be domu1_* guests that are started with domid 2
>> +and so on.
>> +
>> +The domu#_property can appear multiple times and it specifies an additional
>> +property to be listed in the domU node inside the device tree, Xen will
>> +not check if the same content is specified multiple times.
>> +
>> +There are some property whose name starts with an hash symbol (#address-cells,
>> +#size-cells), in this case the line will be considered as a comment, so to
>> +specify them, they have to be listed without the hash symbol, the documentation
>> +will be updated as well to see the implemented handling of these special
>> +properties.
>> +
>> +# Example of a configuration file describing a dom0less system
>> +
>> +The following configuration file is describing a dom0less system starting two
>> +guests.
>> +
>> +```
>> +[global]
>> +default=xen
>> +
>> +[xen]
>> +# Xen boot arguments
>> +options=noreboot console=dtuart dtuart=serial0 bootscrub=0
>> +# Xen device tree
>> +dtb=devtree.dtb
>> +
>> +# Guest 1
>> +domu1_property=address-cells=2
>> +domu1_property=size-cells=2
>> +domu1_kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
>> +domu1_property=cpus=1
>> +domu1_property=memory=0xC0000
>> +domu1_dtb=domu.dtb
>> +
>> +# Guest 2
>> +domu2_kernel=Image-domu2.bin console=ttyAMA0 root=/dev/ram0 rw
>> +domu2_property=cpus=2
>> +domu2_property=memory=0x100000
>> +domu2_property=vpl011
>> +```
> 
> I'd like to suggest a different scheme, not the least because I expect
> the individual domains being independent of e.g. hypervisor command
> line options or Dom0 kernel versions. Yet varying sets of these are,
> for example, a reason to have multiple sections in the current scheme.
> Every dom0less guest would then require spelling out in every such
> section. Hence I think we'd be better off having a section per guest:
> 
> [guest1]
> kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
> property=cpus=1
> property=memory=0xC0000
> dtb=domu.dtb

I much prefer the idea of the section. This is going to be easier to 
parse the configuration file as we would not have to look for "domuX_" 
and then distinguishing X.

> 
> These sections would then be referenced by other sections, e.g. by a
> new "guests" (or "domus", but this ends up looking a little odd for
> its matching of an unrelated latin word) keyword:
> 
> guests=guest1,guest2
> 
> If it is deemed necessary to make sure such a section can't be
> (mistakenly) used to create Dom0, such sections would need identifying
> in some way. Presence of property= (or, as per below, properties=)
> could be one means (allowing an empty setting would then be desirable).

I would expect dom0 to be described in the similar fashion at some 
point. So maybe we should name the property "domains=...".

> 
> As to the properties, is there anything wrong with having them all on
> one line:
> 
> [guest1]
> kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
> dtb=domu.dtb
> properties=cpus=1 memory=0xC0000

It depends on the number of properties for the domain, this may become 
quickly unreadable.

But... if we use sections, then I think it would be better to have:

kernel=..
dtb=...
cpu=1
memory=0xC0000

This would also allow us to create more complex setup (such as for the 
static memory allocation).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:19:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:19:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180685.327435 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXGI-00004t-Ob; Tue, 07 Sep 2021 09:19:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180685.327435; Tue, 07 Sep 2021 09:19:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXGI-0008WR-Ka; Tue, 07 Sep 2021 09:19:26 +0000
Received: by outflank-mailman (input) for mailman id 180685;
 Tue, 07 Sep 2021 09:19:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNXGG-0008WJ-Vv
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:19:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b0a2198e-0fbc-11ec-b0dd-12813bfff9fa;
 Tue, 07 Sep 2021 09:19:24 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2173.outbound.protection.outlook.com [104.47.17.173])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-5Pj7idR_M5Kp9voGTPgQmQ-1; Tue, 07 Sep 2021 11:19:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3392.eurprd04.prod.outlook.com (2603:10a6:803:7::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 09:19:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 09:19:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0111.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 09:19:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b0a2198e-0fbc-11ec-b0dd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631006363;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AfkubQ/XUUoMzWb4P1EI2bnBghkZcRYqwXc3h2OW3OA=;
	b=Qk+wryJM5YTr4Bk8yzj35R+D58lI71PKCzQR1iSia+rxare/P8j94hu2ovL4nhza7ybCUt
	6tE8A8g9/ikbuhiHNA5RP24IAwiPUQph3yEdTLhzV0CltJG416o6BuACyxNjhNJCWeeusX
	J5N6ktVFY3t1SJ3mjdZ108AUV8wIjFE=
X-MC-Unique: 5Pj7idR_M5Kp9voGTPgQmQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=O4eV2BFg8QDjioBXGlM5Vy2DM1xW+mlurUUvHt85SNQfbFXjHyACG7r1Alu6ZhymIBZQvtX8V0yepjUKuUHKmjrD4Cw/mvq7gI4b5mpzFldZrRfArZ2xOzP5TH+IeN8hIix4yKLmbPBT+hYCxJ4dH8JKMAQhghZunPNIZeiH2QkF112rCT+NxuluI8XinayV0wyly6XUzyBL9pBHyU3f+eBE6Dq3/49MbQNAA3i+UpnFtFYJxcgKSKKAz8olK4fa8UXEGmEZLDcp6MqYTA0fxahOZIX1xh8Wr1B4/Tex7T+iGXEwNSlmWHebzhuhhdl7NnXh+9fGkcbQYD6txH9bkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AfkubQ/XUUoMzWb4P1EI2bnBghkZcRYqwXc3h2OW3OA=;
 b=IhGbeBi7cOP3SKgUw7wqyvgpb40L93xtaPgw4WapCflni583Wg7jOv6IlV0t1JYELj3kfVykM0JCFBPayDdYa2ANMHEbtOPmgX3MO8BC9C9dUGQIxotfgDjMQw/a5ihBIT7pusWIuWivMBS7uj1oZqu2vdVskADi4GNFTBRY7+ioPaUtelEo3IhR7p39rs5HIap+T7eVKO2UhBJUYOH0y/sanzTszxtNYD/vJ+sJy5stFiW08/DfAh1UQE+BHfRhxi5IZngi4nS5aAxu0Hz2oQpZWxHSsFzXScASYzz8EtoqEgeYXnNZe2fu3JxK03Q8NA0bqW06WcBC3nBSmUvuYA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
Date: Tue, 7 Sep 2021 11:19:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0111.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e2c8330-1659-4126-a703-08d971e09349
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3392:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB339266B00F98D1EE7CD38844B3D39@VI1PR0402MB3392.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cDx2KHKtTSgILVVDEBk3Mfl3GJS1uQUPx8fUietNRl56XBmQg0QzKYjplxTdOs7JqB8PIEwGLc6CWeFFJD1KFjxd1Yl7/GXBqeQK54+UcE4Xj0Yo0GyesDKQs3H1FvR2vWHmgoI0Nr6QVLFYtv8nXtXIOrSxyYEJwbp49mWfnitqjGJ8uswYpVNDL9LADv/lPH7L08nyCELL716j0wc/TJVTZG+CkjOYT2IHVkCTCKuy8Cr3o22N+iixsojD2d0UQ8lgPAEWLyGuQx2jON37D1IkCSqBw0cr45nfKsEGuv+l/nmO0lCY/cOUTCnTRyyw9tCuKtvmC4tOMElzBk/R1hoKYsW9yOV7p4QM8FnYiOr5NosKeWzKU4UJvtU72I/12g61YtEgv0GGFYQ9W9ZzVQoJXME5zABoIXvogWGyUUa5CvZKseo4l0r2y6ddZuCCLkCHwFCWW1mDe8Ilg6BfsmrHJ3Mj4+tZCzYF+h3Sk72XwdtAuJ13Ky5Dv74xKWZVqumTddhCyhAIPLOyvcES565UOtTh3yVJ9169ngGIvReXXOCHIAi449ErhTExNSrNOtY3C2K05sge/8NBwB9kC0YVc+voTaoyF+O6M+AGO1moxA5UttflFEoDz3RFKuLyJw/MJwNHA5grEP2bvAwnmQ4tNQjoCBsDmkZab9Xnthd5nNhCBoyHW9y+NWMEfGjFy8TnTrFFMATm2vFtymjfMA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(110136005)(38100700002)(31696002)(508600001)(956004)(31686004)(7416002)(26005)(54906003)(4326008)(186003)(83380400001)(6486002)(66476007)(66946007)(8936002)(53546011)(2906002)(66556008)(316002)(5660300002)(2616005)(16576012)(8676002)(36756003)(86362001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MFA5QjNRcHlyT05vMEN0TzdUYjBtK21FcGszUDRsVVFiZzJWK3pGcjQ5ODI5?=
 =?utf-8?B?dWtzMU1pTFlYQXZ2SE9XdCtKTTVyMHBaTW9EcjBKeUhXZzdzMlNRdzlzT1cr?=
 =?utf-8?B?SUtGdGE0dnA5MUdXeGdya3JOeTUwdmVBZm1qYkUybFJ2RVdjbGtFY2pYRGY5?=
 =?utf-8?B?SHUxUjFaK1IydGJsK0xBbDZmRHduT0JTM0xRamVTU3pJSE41SFZoMWl1Ui83?=
 =?utf-8?B?UDgxcGFGbGlnMnRNaGJxcEhUUTd0NjFaZ1JrMFlwMlpyVmJlRkY3MlZyVzlF?=
 =?utf-8?B?WG1LMy9FeHBYaWRMbkszWmcvZExCOXdYclF1K2VScVhmQ2p2S1RrWDU4OHo4?=
 =?utf-8?B?QTlyVVl4QmZseEVnZHE2TXQwc05Vd1hYbWdackkvaDU1NFpBL05saWpWZS9W?=
 =?utf-8?B?QjV3cTJtMDdIVTVIYndOS29keTlIUHlPbGpmcXF4RStHUlJ0RkNiRW0wNDYy?=
 =?utf-8?B?UzI2bXRvMVl3RzJVc3MxamFuRzVaSGdpTHdrMGhvNS9QMkdhV2YzZGoyYUY0?=
 =?utf-8?B?VkpDZTZOYy9BdE5SRW8yc3ZJM09vZUVvS3NwQVUyQ2wvMGkvK3hKN0lEWGhv?=
 =?utf-8?B?NGZoakdmM0EzNU5hblFaNUs2UVFkRHYwMGQ1c3lsekdXSmJBTERZazh6ck9r?=
 =?utf-8?B?QWhZWlI3alFrd1kweVI0d2czUC9JdnE2c2NHcFBramVyMjhsTkhscEsvb3lm?=
 =?utf-8?B?TnFpYnpqZUY3L2x0U29PWlg3akxZOC8rVVBNNlRSWnBZZTVvZ3R6VjkrWUIy?=
 =?utf-8?B?eU0zN0UwYnpXclZFOHVwY1JsRUI5bjJNMSs4eFUrbEcyRE1aUXJQckNzL1Zq?=
 =?utf-8?B?UHRsWUdyQ1N4ODhuVld3YzFwcHViVHovZjd5ZFNQdktXWEZvSkdGVTI3L1Yy?=
 =?utf-8?B?d0xNdFY4N0QzcWVUK01HdWJpS0FPbXhadDVHZlNvQzBCQW1GZVB5bEZMeVRp?=
 =?utf-8?B?b1pJVHZpUU9XdERJS3AyVzBWRi9aQ2pDQllXK3NINGczTzJmTXN3Q1lpbTls?=
 =?utf-8?B?LzNJYzVjMk9CVTg3cWJFc3EycFlJdE55cG9OdnpKRmhSelNzdWJ2RXhVWVRh?=
 =?utf-8?B?ZGlWWDA4YWZqRXQwT3hSQXhuOGErc0NPalN3SW9vRjZrTXRpbVpmZ011KzYv?=
 =?utf-8?B?YVNScG9wSlNMQ0RiL2pVT2Fjdk1rZ2JQUEhyNnZlTXRhT3NCMk53bFlhVGF0?=
 =?utf-8?B?UkdJcDBVNzlOeXZXcVlkempzNXBmYjVWY3Q5QW5QZDdYOEpPTWtUNU0rbUxn?=
 =?utf-8?B?Q3ZDbzJKZjVxdVdmUXpqTENxd0NmWUJpcURESVczSmZsWFNVRDdKWFNvNXpZ?=
 =?utf-8?B?S3kyU1lEQXBwbG1mTXVqcWZxOFNxZDJWYTBqaDlkYzZVSVE3WmMrS1JVcmVy?=
 =?utf-8?B?OHF6bTdXVHVmU3QyN0NlL3NUT1dSdjk5ZEMzZHp6REgwN0hZcjhlZlpZTElG?=
 =?utf-8?B?dkRZNWNYQ2lzbzJrUEFoKzhwMW91WWx2WW54VVZJWDJRUVdjcEhOa3pzK1Jj?=
 =?utf-8?B?aGZ6ZWJkNCtYUzFXaHlGL0dQQkQrQllKNWtyYkV2VHB2QXpOdWd2ZEs2S3N5?=
 =?utf-8?B?Y2FBY25CcXhUem96NHk4dDlldEg3V3hJMmFLRjZ4cTdiQ1Y5OWtlbUpWYzBG?=
 =?utf-8?B?M21nWXpwMVVNQWxnQURDbXFSNkdWK3p1eGxHY1lJdjdRZnVOY2NkdnA1YW93?=
 =?utf-8?B?NjNpZWU4M2ZaTDJ4VFhJbjdxQ2VnZ2crQzRUZVJ5R3VncC9lTmhzVVBsOG11?=
 =?utf-8?Q?Dts3p59VenTxTH4+Rb4/g/rsvDEMtTnNq98C5y2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e2c8330-1659-4126-a703-08d971e09349
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 09:19:20.8335
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6aNQ9eM1WTM/qW4MT2A4VS29LIj/1w8MRlpvMvz5WrnPDNSTwVD549G5jJ5VHzJKUq1u8JyTltj8K2E9amceZQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3392

On 07.09.2021 11:07, Oleksandr Andrushchenko wrote:
> On 07.09.21 11:49, Jan Beulich wrote:
>> On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>>> So, if we have a hidden PCI device which can be assigned to a guest and it is literally untouched
>>> (not enabled in Dom0) then I think there will be no such reference as "host assigned values" as
>>> most probably the command register will remain in its after reset state.
>> What meaning of "hidden" do you imply here? Devices passed to
>> pci_{hide,ro}_device() may not be assigned to guests ...
> You are completely right here.
>>
>> For any other meaning of "hidden", even if the device is completely
>> ignored by Dom0,
> 
> Dom0less is such a case when a device is assigned to the guest
> without Dom0 at all?

In this case it is entirely unclear to me what entity it is to have
a global view on the PCI subsystem.

>>   certain of the properties still cannot be allowed
>> to be DomU-controlled.
> 
> The list is not that big, could you please name a few you think cannot
> be controlled by a guest? I can think of PCI_COMMAND_SPECIAL(?),
> PCI_COMMAND_INVALIDATE(?), PCI_COMMAND_PARITY, PCI_COMMAND_WAIT,
> PCI_COMMAND_SERR, PCI_COMMAND_INTX_DISABLE which we may want to
> be aligned with the "host reference" values, e.g. we only allow those bits
> to be set as they are in Dom0.

Well, you've compile a list already, and I did say so before as well:
Everything except I/O and memory decoding as well as bus mastering
needs at least closely looking at. INTX_DISABLE, for example, is
something I don't think a guest should be able to directly control.
It may still be the case that the host permits it control, but then
only indirectly, allowing the host to appropriately adjust its
internals.

Note that even for I/O and memory decoding as well as bus mastering
it may be necessary to limit guest control: In case the host wants
to disable any of these (perhaps transiently) despite the guest
wanting them enabled.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:24:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:24:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180693.327446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXLS-0001pi-GM; Tue, 07 Sep 2021 09:24:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180693.327446; Tue, 07 Sep 2021 09:24:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXLS-0001pb-D7; Tue, 07 Sep 2021 09:24:46 +0000
Received: by outflank-mailman (input) for mailman id 180693;
 Tue, 07 Sep 2021 09:24:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNXLR-0001pV-PT
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:24:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6f73a761-0fbd-11ec-b0dd-12813bfff9fa;
 Tue, 07 Sep 2021 09:24:44 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-cfbypJ0_Nrq764JVaelNew-2; Tue, 07 Sep 2021 11:24:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 09:24:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 09:24:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0047.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22 via Frontend Transport; Tue, 7 Sep 2021 09:24:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f73a761-0fbd-11ec-b0dd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631006683;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=92ya5n/p8bGdMWSZW+gUJtmHj66H2FVO1aSkxTQSLw4=;
	b=QzJJVhKi/BbrifPvFvsjTgC2Y0ncm0cfJZUMoUwXXHA+HnziDTkSefGwt2u1q6R7SDNOAk
	ozYFMudZfjxymSf4GAZjO06BTnxgYTS/Gid4yzwlEeU80LZ7IIQXcYvav+7D6GcaDVGL58
	iQjiESJEsSn8gcO45g/dlgzYwVrHNQw=
X-MC-Unique: cfbypJ0_Nrq764JVaelNew-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SoCUmH7KES2lcCuH28BWCZKl6vv2eWfI4HT8dr0g/0nO7odmgcFf73a1vWyY/QFbloxiHlu/o+d0vlQSBVX4LZEvflsPxUpmlwl/0uC51RhmDyLKqUeQuI7BCEebM6t9EgdP41pRPXSxYj30li/EUSHICh5qiCp9JU+OXpaIdcU5yEGCaXDgNtjrSApw0cV5lZ8EgdilziI3rLLqn5ZpQxBg8BhDyZSqRTmwT78tx6qIagR0zevvkKpqAC39hG2wpguJRw3nmnA/HMYh/TIVDXZlJwFFVOvby/dxCyzB3mrcpensH3ChPuU6tpjpF8l+7brB+f+4FzfPK1Iehy9Ffg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=92ya5n/p8bGdMWSZW+gUJtmHj66H2FVO1aSkxTQSLw4=;
 b=h7S+x70DHmBh76ah0GwTyHqR+dRxkrb1+iJZBBYadrJTUly4NEMHQV1ARKb75e/hcUAkTt2TkuTh03PnvuaXrbULUCWVEIa4zZaMf+h3kWsqCIAybCCOxKqQfSw25BugGDZCxvuR+tWoRKqfEhBB+qMv8KMa4C5zMp91KF4TVXeaulCWc3NPNRCgk7+ksg0s3J5nsXTKNlZ6qOZwpsFHdcT6WESKaf4L/q7XiKwMBw4/XFfm/JHefzdFKryRIe3UEbdiVB1DIsC0gMd4wqj+jjLDlh1imh+842GF/BC659oLPxtrJnVWh+CL9gLCUXcOopoFjlG2YQ+BG9aJyaUqXw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Julien Grall <julien@xen.org>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Luca Fancellu <luca.fancellu@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <ea736db5-e3cf-7eea-cffe-98ecb0290048@suse.com>
 <3222722f-b12e-3991-d0de-b455b9fca063@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1b4eef88-78cb-7201-c326-551261f9e9ad@suse.com>
Date: Tue, 7 Sep 2021 11:24:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <3222722f-b12e-3991-d0de-b455b9fca063@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0047.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c45b6758-e959-49a3-d3c9-08d971e15136
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB715211BDE62E1AF911727DEFB3D39@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	T4Nz5BsL4QGfoQ9urWOtzoRhB8eKZYHxT3Wrc2gHfWtx7WzBfYsnhJ5NaCN0s6NFKtpYCsKP99HK+fCJKtscsuDFiVMwgE0efAKpk7ZcdlaPGAqsMXUZRc2LKNCksens/v9ieFiliwOB0U0kEWN+JKDe+hrJEukSwnDYe3hD+jArlaWDSI1UqK1illwgwiIIfIyKgqx1sTTLOGTp5CA6V0xywsx8sYBcco6UV5sGzA5CoZHsOwqUGqVN28VkyBoqQOew3xIBdS/434fQAJBAxagKPk40lWPbT5ysTCRDecIhR97vSxmj6hNZmkElwHzCzn3J96yG3GTGmn3smDCVCbB2nYwGlNs0PeM1MbA0ozHKVLXp5edX4xBxckNf+NtSNoJy0x4t8XoD/Q6BbbEpFY8ig5oeEKYcUmvUs894JTY57LcUKNWe2jn1txBPB2Rvw+ixvzrMVpb1rAeJtomCtPhqrrNjCJq7QEXj72Up3mWDrMcq7kgl944a9rhv6aBj+3uZhyNK4MsNwIK0oluJyDxmY1/bW+CncwDMJp9oOAplEedRMhIeJJcLMRx93T5f7JzudC3Osiui1OGP81npXi3nzVkdpYrFeog43ymEE1vmuV6JfgC1+ccnUoUPwv/1gLYWHRxue2oPUdY/dolVakrp0uBHEkIXk0WmTDqnwPZKBnCCjzEBundkwbVTXYZMvKX8hq8JEKa0t4vNOqfefd24tdljXpHX2Rw043HWid/O3qT3lB/QS8Sy6tIf4TzG
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(346002)(136003)(366004)(39860400002)(66946007)(6486002)(66476007)(66556008)(83380400001)(478600001)(31686004)(36756003)(5660300002)(7416002)(86362001)(31696002)(6916009)(54906003)(316002)(956004)(38100700002)(2616005)(16576012)(2906002)(186003)(26005)(53546011)(8936002)(4326008)(8676002)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SDR3NFJFVUsraW9XZTd3N2paYjByL2xqUDBHcis2b2x0YklJNXFZR2JuTFdF?=
 =?utf-8?B?RlhDalhkUmhtQjlUZkVhWmN4djJ3NzY3T1dVSmNBdXlEdUNibDV2SjZTRUw5?=
 =?utf-8?B?OG9heEo3M1NnUzVJeWwzRGgwS1JxZ0FrdUFnaWVvSG5wUVRQSHVHNW96Q0dI?=
 =?utf-8?B?WldkNncva1JoVXMrZDkwbGRvZy9yeHNPMW9QUVRuNVdnbHQ4TWZNTEhVUFYw?=
 =?utf-8?B?ZXpuU3ZrUENSTXE1UVQ1Ynk2WllnNkM4Vi8zekEraytBTlRMNTlzMjVraDdJ?=
 =?utf-8?B?c0JPaG0yQnpxaHg5VGFmZjJiRGVaMUMyRGlYTy9HenBHY2UyUDJhenlsSWg3?=
 =?utf-8?B?ZHN5ejhVQ1VYVU9OeThVRHRvbmdESWxLMUU1RmFSUXUyb1hDaWRoQ2Z2bXdn?=
 =?utf-8?B?V1VSRm1tU0FnMTFIUFZkazd0anRmd3dkMlJvN09ENDNrZ1BncmtIRTIwWXNQ?=
 =?utf-8?B?bEFKUEdRQzhFK2UyYk5DYnpwQ2c3NU92L2FxTGFxWERmUm5rYkkzRVRITXZ0?=
 =?utf-8?B?QnIrWlEycURYNitnTU1CejVxbFVHYUpZWlRYWmd0OVBzZUVqTWQ2UHBZV3ZP?=
 =?utf-8?B?MDVDa1FVaEtuUUlidGNXRm8yT1gxcnFmN3QyM1RWUERYcFJqcU5SUWYzR3dm?=
 =?utf-8?B?dTdJaklVYmxQREFwMnZqU0R5dWJMc3dwUk9OUmh3NXREZ1NCM1NtdlU2R01B?=
 =?utf-8?B?RHNiMDg5Zk1CWklRYzZoNFRjakM1SWxzbjlYU0E5Rno1bmxnN0VONkgxY3ZM?=
 =?utf-8?B?VThCbTRXYXUzSm5GL0NJcTQySlQwMENxWHY0U1dGVnl3cHljWURVU0g2cGJE?=
 =?utf-8?B?d3lnL2w5cFJRMDR6R3orUUNyWFIvNEF4TGZSZFgvalAyMkhEYVVrUVJCbWdp?=
 =?utf-8?B?am52RnVCcFl2dU9WTkhISG5OSnpIdFQ1VGZMUlZod2V3U09RZVd1Y0pva2pF?=
 =?utf-8?B?aFpia1JVMGQ2dVUwR2ZHY0g4R3BYLzBTYStJcTgxbXZGdzJYZGFYYlMyVll5?=
 =?utf-8?B?ejFNUy9aSVkvQjVzRXVFOEJNQk4yT3J2QjF4Y2JVbHRJdTRSUG8xYzF5eUd0?=
 =?utf-8?B?U01USXlkdUpYYml3TFRmc0lpelhORnRnMVNWMmF3SVBLSWQwandrVU9FL0d1?=
 =?utf-8?B?Nnp5UzBrUm8ycXJjLzNsUy8rSTFvbnc0ZE92dmpTQnVDemhuYkppeDlGSmY0?=
 =?utf-8?B?eFNjNEhSMGNEa0Z5YUJDem9HcVBTbzZpTnRhME44NUR0cEVOU2Z1dWxmQk5M?=
 =?utf-8?B?WWloSGliZklEV1lld21HOEo3K3ZQamQ3U1FLUmxkWktNc1B3ZGg3MEpiYmdu?=
 =?utf-8?B?dnRBNDc5WXN1U2ZQZXdnTmx3QkdWY0hIZlpiUUYrMExGZEZDS2NSMlZuZkJZ?=
 =?utf-8?B?TXlwdjgwOEVtY2Y1MlpoUVZmR25jTlNzWTc2VTRPeE9ISG1QME5MZWlTRjZ2?=
 =?utf-8?B?TFBaMzUwQ0pLOUpERXNURmIwNnByeWxwL3lMN3EySHdPSjMvYWdYanJpdEVE?=
 =?utf-8?B?YVgxRXZ5YzJKN0JSUlAwOVRHSjk2T0RNaEtoUFhiZS9qZGxRa3RBWFM4SXNP?=
 =?utf-8?B?R1hSK0FqUjJCOWhzeis0cUxFckFjdFZ3aDUvcllxS3lZUkNuU2toOXlwOUZx?=
 =?utf-8?B?c0hnUEJDTmNUQXJ6a3lYZUc2Q2cvWVBUUnVab3d4SVlSOTlKY0dUZ0RIYTBa?=
 =?utf-8?B?QmpSeEVHaFd3RVQzb1Q3RnNWNnEwSmE5YjFQSTE1SWVJdnI0MzFpU2p1emla?=
 =?utf-8?Q?NMGkA7AfmeB1l8lUbX3a15/OIHd+Ra4oeDBFGLY?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c45b6758-e959-49a3-d3c9-08d971e15136
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 09:24:39.5312
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PNgdy4r1gZD2LWgrT6GFIy+FwTuDdZ48xSCbYQ9RBzK0hv1tKrfhWtzqHScbPblez7R8jpkePLvoBsaTpoZXLg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

On 07.09.2021 11:17, Julien Grall wrote:
> On 07/09/2021 09:33, Jan Beulich wrote:
>> I'd like to suggest a different scheme, not the least because I expect
>> the individual domains being independent of e.g. hypervisor command
>> line options or Dom0 kernel versions. Yet varying sets of these are,
>> for example, a reason to have multiple sections in the current scheme.
>> Every dom0less guest would then require spelling out in every such
>> section. Hence I think we'd be better off having a section per guest:
>>
>> [guest1]
>> kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
>> property=cpus=1
>> property=memory=0xC0000
>> dtb=domu.dtb
> 
> I much prefer the idea of the section. This is going to be easier to 
> parse the configuration file as we would not have to look for "domuX_" 
> and then distinguishing X.
> 
>>
>> These sections would then be referenced by other sections, e.g. by a
>> new "guests" (or "domus", but this ends up looking a little odd for
>> its matching of an unrelated latin word) keyword:
>>
>> guests=guest1,guest2
>>
>> If it is deemed necessary to make sure such a section can't be
>> (mistakenly) used to create Dom0, such sections would need identifying
>> in some way. Presence of property= (or, as per below, properties=)
>> could be one means (allowing an empty setting would then be desirable).
> 
> I would expect dom0 to be described in the similar fashion at some 
> point. So maybe we should name the property "domains=...".

Not sure - the order above doesn't mandate domain IDs, yet Dom0 needs
creating with ID 0. IOW I was deliberately suggesting "guests=".

>> As to the properties, is there anything wrong with having them all on
>> one line:
>>
>> [guest1]
>> kernel=Image-domu1.bin console=ttyAMA0 root=/dev/ram0 rw
>> dtb=domu.dtb
>> properties=cpus=1 memory=0xC0000
> 
> It depends on the number of properties for the domain, this may become 
> quickly unreadable.
> 
> But... if we use sections, then I think it would be better to have:
> 
> kernel=..
> dtb=...
> cpu=1
> memory=0xC0000
> 
> This would also allow us to create more complex setup (such as for the 
> static memory allocation).

If that's feasible parsing-wise - sure. I was first thinking to suggest
separate keywords, but then decided there was a reason this wasn't done
in the original proposal (with respective dom#_ prefixes).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:36:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:36:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180701.327457 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXWL-0003Z5-JF; Tue, 07 Sep 2021 09:36:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180701.327457; Tue, 07 Sep 2021 09:36:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXWL-0003Yy-G4; Tue, 07 Sep 2021 09:36:01 +0000
Received: by outflank-mailman (input) for mailman id 180701;
 Tue, 07 Sep 2021 09:36:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNXWK-0003Ys-DX
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:36:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXWH-0000cW-Ju; Tue, 07 Sep 2021 09:35:57 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXWH-0006k8-D8; Tue, 07 Sep 2021 09:35:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=zt8g/1egKtbllKcQNJg4DQLiOBnwt0kBFJGt3Sn7rSw=; b=fAU76dfu5dTxcGnb6q78D7EEIf
	vgLRNFq9OK13HumYIYU8rV4V9Dc3a/EA5lUOZ/1g4NUD9j1WPolP5C/z+uaLAQravuRyKTcowdkG9
	wT2w/gpJ33ZqIckMV/iFIheuX4j2h1IgLo9A1veD9Kb+DDfp+kP4lpnw9ZZ/eImULGR4=;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
Date: Tue, 7 Sep 2021 10:35:54 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <20210907065228.21794-1-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Luca,

On 07/09/2021 07:52, Luca Fancellu wrote:
> Add a design describing a proposal to improve the EFI
> configuration file, adding keywords to describe domU
> guests and allowing to start a dom0less system.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>   docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>   1 file changed, 105 insertions(+)
>   create mode 100644 docs/designs/efi-arm-dom0less.md
> 
> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
> new file mode 100644
> index 0000000000..8d8fa2243f
> --- /dev/null
> +++ b/docs/designs/efi-arm-dom0less.md
> @@ -0,0 +1,105 @@
> +# Xen EFI configuration file
> +
> +The current configuration file used by Xen when it is started as an EFI
> +application is considering only the dom0 guest and doesn't have any
> +property to describe and load in memory domU guests.

 From my understanding, the problem is less about properties (we already 
have them in the Device-Tree) but more about where are the binaries 
located in memory as we don't know in advance.

So I would like to propose something that build on top of the 
Device-Tree work we did. Note this is early thoughts.

The problematic nodes in the DT are:

         module@0x4a000000 {
             compatible = "multiboot,kernel", "multiboot,module";
             reg = <0x0 0x4a000000 0xffffff>;
             bootargs = "console=ttyAMA0 init=/bin/sh";
         };

         module@0x4b000000 {
             compatible = "multiboot,ramdisk", "multiboot,module";
             reg = <0x0 0x4b000000 0xffffff>;
         };

In particular the property "reg" cannot be known in advance because the 
UEFI stub will be responsible to load the binaries in memory.

What we could do is providing a list of binaries to load and associate a 
key for each of them. Something like:

binary=<binary> <key>
binary=<binary2> <key2>
....

We can then replace the property "reg" with a new property "uefi,key" 
that will contain the name of the binary.

What do you think?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 09:52:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 09:52:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180708.327468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXmF-0006Mm-1B; Tue, 07 Sep 2021 09:52:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180708.327468; Tue, 07 Sep 2021 09:52:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXmE-0006Mf-UQ; Tue, 07 Sep 2021 09:52:26 +0000
Received: by outflank-mailman (input) for mailman id 180708;
 Tue, 07 Sep 2021 09:52:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNXmD-0006MZ-9e
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 09:52:25 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ac617ce9-c16b-4a33-9cd1-aeeaca2b86fe;
 Tue, 07 Sep 2021 09:52:24 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1879h5nS030224; 
 Tue, 7 Sep 2021 09:52:21 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ax3d2rker-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 09:52:20 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM7PR03MB6327.eurprd03.prod.outlook.com (2603:10a6:20b:137::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 09:52:17 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 09:52:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac617ce9-c16b-4a33-9cd1-aeeaca2b86fe
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jK7A8sAL19CiTLD3ZU6JXJHyA2iHkaS+uQeSNQi0nDL/oyZSmdhku4rxDffaQyY/EcxelDcXiE7h0DU8H+6m1I3HvzXvbbMPsucKSUt3WwbWz+KiFojcErDaOTnDZw5yBPnD+rFWfmtTUNQU4DlbFty7ok2XJSjqHBEIt2P1MCq2o8rezPzapNlwgc7GFK3lA9Dt+P/S/y5CSLSOtU+tiv5/K9YzhT8UKicyNo/iwUsv+WfmMGUqGbu38DyoZqgjAae9/N6U/JUinPkrg6xP5MoDWY7wlwwGRIES4bx9D5cXh7PtaUiOrGZgZo1DVtXh8soyGIqInh/IJ6mjxFIyfQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+ol761Dc91+0hUI/Jo1B4DDOAwiNGaIXzUP1n4/xJDE=;
 b=Sd2oFJsoJrYc6fz9/hrKGBWkdnTvscWMMZ6dPyztpbqnZmImNdwxjm4LEbbRBsQYOgke7W0ndt9p6ZmCS+uff+SG8cuYzMTVOK7Hlz/+A4cP3xcMQJQZu0o1DVYbZtU2GFA6o8AkKfLM7oAQW+W5DVlO5gC4kAxbUmzpGGKAbHiE9LSEhiYdXRuS5bZ6IwIrU5Xb/dsLHbKYmC1m3gFreN5aXpBWkOxKmGj2vGThNsSKz0wKB5RnyCVbYiUR1akSFiFLPTFH5i1bm40bUZkhW707oWDCGimO9du1SIAj/T27heUYOsjqY4RCrmRhaAOT95UAE0f919zm/YA/TRvqfA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+ol761Dc91+0hUI/Jo1B4DDOAwiNGaIXzUP1n4/xJDE=;
 b=NSnbO4ohcGGbWE+3TOU1Gqlv4LAnWL2jt2px9mYqCXsyXyAYlnlIszf+Ct3wBfJz3QXLmJFO8YmbV1w6bddBMX8f48sKllLuEwMFmf5JKkK44SSOvtnvenmuDp6UM/GCy7TkrAmhFxb5QD6WPtNdo8sjd+/obczV/Okp0omiIlhYZ+84IIESm9VImoKDBO37n8BdiHr4UF123XmWBYoZKBCQHvqOoNfXFao60hIScKDOreKoaBeelj5tfHUh+mLceGn5wFa8tMHfBE5H0MD7he+FfaBob+1WAPJN4DCg26oZUS5hEVicFoX8k3rRUsfZHSiy6e0G69ae3QIb/31Nhg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYA=
Date: Tue, 7 Sep 2021 09:52:16 +0000
Message-ID: <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
In-Reply-To: <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 47fbaf89-b629-4a2c-c6c6-08d971e52cfd
x-ms-traffictypediagnostic: AM7PR03MB6327:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM7PR03MB63275072D29C6387071D780EE7D39@AM7PR03MB6327.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 PYuXB+pwgV16n1NQeQQHDt2Kc1yitg8csf0lx4VNEaLkXosO5IzY8yAHr12wvtVEgC985gkUG/iFdVVozG6EV4dnKkKboLze6IsQb6BTHJiiOcQnAFzhZkepTef8QAkslMqqv7cXGaUK2/0TPzppWFYX+JLTg9/dOA7nkHjcadagzm8s33q73kPmdzU6dEjjPE7uhp6acHe8lhgI+Ce2vkWnr4/2fORMv0pXn6lYDIfdlRf+S7gtUWLRoveCYqiUllM8BqwTbTaM1ncWvFwB2FAkQgrK/+kREaIRWQXW4BaYjyjhY02BcVqsSJtJVIcjIOWTXdifanzXsh4Rz7tsFcbRuUBacFGKV0uhh/5yoE/yEbfy8uACeC1YrOogNEyccoIsn0uraIDa9lXqsmv2qvotHd/CcU1Bl+QAu+tDLDL8ufc1Cg/GMBnQLAsfkICnyFseTKQI2tyHQ+VMdUJJs+WfGPqWBGyN9NmC161kVZ5b89mR1DYYO4gjHN1ge7i4qEbwYaAc6Beqi6usBZNYtAX0l7JfGnZ6BtoxLKU10sb3aH7MuMO/krvnUzl9OvuHZjCyicSKr2FBzxJ6PIwcL3j77wpmdq6LTBbegsOElmbc/tnhiKhqT5Ti5LfPp4Z5JCSKXv/Jh0p44B2FylrXCQY7PDNtDVSuKFqBfcZT5Gokhf3tShD1YVpFJzA0jdOJzxzQJh2Gt7eh3xIacNoDbe4N+FpH2428HCAvu0XCAf8HsKztIvt/CKp7plgj1eRu
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(376002)(39860400002)(136003)(396003)(86362001)(91956017)(6512007)(26005)(31686004)(66556008)(76116006)(66446008)(36756003)(4326008)(31696002)(66946007)(66476007)(6506007)(54906003)(110136005)(316002)(83380400001)(53546011)(122000001)(2906002)(71200400001)(38100700002)(478600001)(8936002)(186003)(6486002)(38070700005)(5660300002)(8676002)(64756008)(2616005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SEVJcTNFV0V5bTBOSDgrOFlMOXk3Y0ViN1BzMTI3Rm5FeUF3dE43UzZ1WXRp?=
 =?utf-8?B?MEpYLzY0VGozMGxlelVzNEI0dGtObTdQUEVySnR2ZTFBYlIvbXVHUUhVRUtP?=
 =?utf-8?B?VDUyeldabWlUMkRqdXB0L0dIcjlYOHZnUzJRK1BVSjlLVkJ5S2VBQTdwcm85?=
 =?utf-8?B?L1lDWllLWW02V0svYjZjenlQQzIzYm5CdkgwL1ZwRlVzMU9DWC9vbEUrQ2hS?=
 =?utf-8?B?cGZkUHBsT0pRaWtsZWptWDZ3dCtINTNyVmVCb0R6YkdXcE05dzYxeFNmcjhS?=
 =?utf-8?B?L2RubDU4aGlRWEdIU2N3N3puNGNGRFZMa1d6N1gwbTRVWUVWM1J5SlVtSTM0?=
 =?utf-8?B?RnF3ai8wUUJvUjI5SysrSlYyV29teFN6UUlLaXNnbUpnVjJucVQ1MjdwdFps?=
 =?utf-8?B?RTdwZGFkN2Y0cWE5ZFlvbE02YlZHd2JVWjNkUnZrdlZWaWJuZ1JDWW9zTGhT?=
 =?utf-8?B?dGFQYS9UVCtXZjJkL1BvZzRJYitrODgvM0N4RkdqOXNlVnhQWnV2NFRpT2hR?=
 =?utf-8?B?RlBEVjIwRzRrK0hIYmhNckFoWnFvcGtyZndDaTY5Y1Q4aFRzQUxERkFzVnR6?=
 =?utf-8?B?RytFZC9lNnJmcHdYY3Z5aFdSMXBqUG1Hdlk0bTcxeXh5UlNMby8rNUlTUGtC?=
 =?utf-8?B?MXFTS2xFcFV2N1BzazNtR1dFWm96STRRU3g5UDRBeWwyeklyejdIVG5PMzND?=
 =?utf-8?B?K3Z1c1I4bHFrYjdWdm56Nm40ckFVSFBwOW5oVDNhSW9RemNTWitmTnVhL01w?=
 =?utf-8?B?MzBPUGEvQ0FxYnV3dDVNMFBGOTg5OWRCTHZRT1I2V0FsOUxHblA5a3hzektk?=
 =?utf-8?B?aGgxUkw5enJaYlZrYllMMWtyVWkyQkU5aDdTL3BXTzdlNVFJb1U5Q2gzWlhI?=
 =?utf-8?B?dnJWMTc4Wmk2dURMZTY5UkxTSjkvSVVudWFOaUM3d1ByMUY5aGpKWGlmT1VJ?=
 =?utf-8?B?RWU4b1JjOWNhSEN2cUFpdmlBVHdUZERmSDFxcG9jNTZpSS80RFRNNVdPZk9O?=
 =?utf-8?B?NkVrRjU4NW9wMnh4RTQzcW9YWkZIS2J6YUZkQ080ODJFTmM3MjhRSGR4UVpm?=
 =?utf-8?B?c1NTR0J0Y3R6dytMZU5JTmFva1Yyci9IVXZRL1QrWFZneSs5QWY3ZlJndDV3?=
 =?utf-8?B?VzJoQXhiL2gzWTJ2bGNva3g4ZTcvSE1DaWQ4cE43YjZsTDg0WEFRZXFzY2Fm?=
 =?utf-8?B?cklGZytyT3FVN1VYVnZPZWlUNjBBTFFhTGxsa2M4eENjbzdWdlZaaGcwNHQz?=
 =?utf-8?B?NE9wNWV5RjVLY0RGc01tYi9wenhxS0w4OG1wRGVOL2I5OW50STEydENidy9y?=
 =?utf-8?B?bUxtOXdSTDhMNksySUVXQnZpWDc3UXlLTHZRdVpQUmx2SW9zNmxrdlNKRU5j?=
 =?utf-8?B?aFIyWWpEZ0NNdGJMb2JSOGtDNFJST01QczN6YmFGNHZXSEY0MlVrZ1NhMW16?=
 =?utf-8?B?RndmVEtzcTcrQ2E1NTJsWXZyaFYxTUloU1dneWJjYjZtRkJrK25sZ2ZTbkZU?=
 =?utf-8?B?TTlUai9PYmVOaFF4M0JibkF3SzlPRk83clhVOURCZ3ROQ2VvS2FFKzd1S1Bu?=
 =?utf-8?B?YUJ4UmpOckR1cE5LdnR5QU5SYk5MTHlEeHlsVXUyb3IvQjRiZVlKS2tQWDV1?=
 =?utf-8?B?M3c4ZEFDUURTQkd3SlN0cERPYmJhV3VQa3Jiem1ZTXNZVnJUcGlYSHZxbWtL?=
 =?utf-8?B?eFlXTENKdmpWUUhqTVY1WWxlMWsydEhUSkY2ditpQXltcXRPVEtJNzlUcGgv?=
 =?utf-8?Q?EZnJBRRwOWekLxsQUwU9i5yf58AW3cTMmbz3+v0?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C1AC6BD172CC76429FEBDB55999C5EF9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 47fbaf89-b629-4a2c-c6c6-08d971e52cfd
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 09:52:16.4878
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: T0Aiqn7dKBW2mZoOWh6C0jmvZajLWT1aAjsup6G65xR6duyNsIdJciBjVHD3+bOQh90E0GBMIENqC9hZbyyxu/A8Bf42vrp5vqCD7o/OvvNNJ7W7XaeG0wfMaDTWA5EZ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR03MB6327
X-Proofpoint-ORIG-GUID: YLtnS5ThGfSCc62F1oTtxLjhQ7D_RJJz
X-Proofpoint-GUID: YLtnS5ThGfSCc62F1oTtxLjhQ7D_RJJz
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_03,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0
 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxlogscore=999
 suspectscore=0 mlxscore=0 bulkscore=0 priorityscore=1501 phishscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070065

DQpPbiAwNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTE6MDcsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTE6
NDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTA6MTgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBTbywgaWYgd2UgaGF2ZSBhIGhpZGRlbiBQQ0kg
ZGV2aWNlIHdoaWNoIGNhbiBiZSBhc3NpZ25lZCB0byBhIGd1ZXN0IGFuZCBpdCBpcyBsaXRlcmFs
bHkgdW50b3VjaGVkDQo+Pj4+IChub3QgZW5hYmxlZCBpbiBEb20wKSB0aGVuIEkgdGhpbmsgdGhl
cmUgd2lsbCBiZSBubyBzdWNoIHJlZmVyZW5jZSBhcyAiaG9zdCBhc3NpZ25lZCB2YWx1ZXMiIGFz
DQo+Pj4+IG1vc3QgcHJvYmFibHkgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgd2lsbCByZW1haW4gaW4g
aXRzIGFmdGVyIHJlc2V0IHN0YXRlLg0KPj4+IFdoYXQgbWVhbmluZyBvZiAiaGlkZGVuIiBkbyB5
b3UgaW1wbHkgaGVyZT8gRGV2aWNlcyBwYXNzZWQgdG8NCj4+PiBwY2lfe2hpZGUscm99X2Rldmlj
ZSgpIG1heSBub3QgYmUgYXNzaWduZWQgdG8gZ3Vlc3RzIC4uLg0KPj4gWW91IGFyZSBjb21wbGV0
ZWx5IHJpZ2h0IGhlcmUuDQo+Pj4gRm9yIGFueSBvdGhlciBtZWFuaW5nIG9mICJoaWRkZW4iLCBl
dmVuIGlmIHRoZSBkZXZpY2UgaXMgY29tcGxldGVseQ0KPj4+IGlnbm9yZWQgYnkgRG9tMCwNCj4+
IERvbTBsZXNzIGlzIHN1Y2ggYSBjYXNlIHdoZW4gYSBkZXZpY2UgaXMgYXNzaWduZWQgdG8gdGhl
IGd1ZXN0DQo+PiB3aXRob3V0IERvbTAgYXQgYWxsPw0KPiBJbiB0aGlzIGNhc2UgaXQgaXMgZW50
aXJlbHkgdW5jbGVhciB0byBtZSB3aGF0IGVudGl0eSBpdCBpcyB0byBoYXZlDQo+IGEgZ2xvYmFs
IHZpZXcgb24gdGhlIFBDSSBzdWJzeXN0ZW0uDQo+DQo+Pj4gICAgY2VydGFpbiBvZiB0aGUgcHJv
cGVydGllcyBzdGlsbCBjYW5ub3QgYmUgYWxsb3dlZA0KPj4+IHRvIGJlIERvbVUtY29udHJvbGxl
ZC4NCj4+IFRoZSBsaXN0IGlzIG5vdCB0aGF0IGJpZywgY291bGQgeW91IHBsZWFzZSBuYW1lIGEg
ZmV3IHlvdSB0aGluayBjYW5ub3QNCj4+IGJlIGNvbnRyb2xsZWQgYnkgYSBndWVzdD8gSSBjYW4g
dGhpbmsgb2YgUENJX0NPTU1BTkRfU1BFQ0lBTCg/KSwNCj4+IFBDSV9DT01NQU5EX0lOVkFMSURB
VEUoPyksIFBDSV9DT01NQU5EX1BBUklUWSwgUENJX0NPTU1BTkRfV0FJVCwNCj4+IFBDSV9DT01N
QU5EX1NFUlIsIFBDSV9DT01NQU5EX0lOVFhfRElTQUJMRSB3aGljaCB3ZSBtYXkgd2FudCB0bw0K
Pj4gYmUgYWxpZ25lZCB3aXRoIHRoZSAiaG9zdCByZWZlcmVuY2UiIHZhbHVlcywgZS5nLiB3ZSBv
bmx5IGFsbG93IHRob3NlIGJpdHMNCj4+IHRvIGJlIHNldCBhcyB0aGV5IGFyZSBpbiBEb20wLg0K
PiBXZWxsLCB5b3UndmUgY29tcGlsZSBhIGxpc3QgYWxyZWFkeSwgYW5kIEkgZGlkIHNheSBzbyBi
ZWZvcmUgYXMgd2VsbDoNCj4gRXZlcnl0aGluZyBleGNlcHQgSS9PIGFuZCBtZW1vcnkgZGVjb2Rp
bmcgYXMgd2VsbCBhcyBidXMgbWFzdGVyaW5nDQo+IG5lZWRzIGF0IGxlYXN0IGNsb3NlbHkgbG9v
a2luZyBhdC4gSU5UWF9ESVNBQkxFLCBmb3IgZXhhbXBsZSwgaXMNCj4gc29tZXRoaW5nIEkgZG9u
J3QgdGhpbmsgYSBndWVzdCBzaG91bGQgYmUgYWJsZSB0byBkaXJlY3RseSBjb250cm9sLg0KPiBJ
dCBtYXkgc3RpbGwgYmUgdGhlIGNhc2UgdGhhdCB0aGUgaG9zdCBwZXJtaXRzIGl0IGNvbnRyb2ws
IGJ1dCB0aGVuDQo+IG9ubHkgaW5kaXJlY3RseSwgYWxsb3dpbmcgdGhlIGhvc3QgdG8gYXBwcm9w
cmlhdGVseSBhZGp1c3QgaXRzDQo+IGludGVybmFscy4NCj4NCj4gTm90ZSB0aGF0IGV2ZW4gZm9y
IEkvTyBhbmQgbWVtb3J5IGRlY29kaW5nIGFzIHdlbGwgYXMgYnVzIG1hc3RlcmluZw0KPiBpdCBt
YXkgYmUgbmVjZXNzYXJ5IHRvIGxpbWl0IGd1ZXN0IGNvbnRyb2w6IEluIGNhc2UgdGhlIGhvc3Qg
d2FudHMNCj4gdG8gZGlzYWJsZSBhbnkgb2YgdGhlc2UgKHBlcmhhcHMgdHJhbnNpZW50bHkpIGRl
c3BpdGUgdGhlIGd1ZXN0DQo+IHdhbnRpbmcgdGhlbSBlbmFibGVkLg0KDQpPaywgc28gaXQgaXMg
bm93IGNsZWFyIHRoYXQgd2UgbmVlZCBhIHlldCBhbm90aGVyIHBhdGNoIHRvIGFkZCBhIHByb3Bl
cg0KDQpjb21tYW5kIHJlZ2lzdGVyIGVtdWxhdGlvbi4gV2hhdCBpcyB5b3VyIHByZWZlcmVuY2U6
IGRyb3AgdGhlIGN1cnJlbnQNCg0KcGF0Y2gsIGltcGxlbWVudCBjb21tYW5kIHJlZ2lzdGVyIGVt
dWxhdGlvbiBhbmQgYWRkIGEgInJlc2V0IHBhdGNoIg0KDQphZnRlciB0aGF0IG9yIHdlIGNhbiBo
YXZlIHRoZSBwYXRjaCBhcyBpcyBub3csIGJ1dCBJJ2xsIG9ubHkgcmVzZXQgSU8vbWVtIGFuZCBi
dXMNCg0KbWFzdGVyIGJpdHMsIGUuZy4gcmVhZCB0aGUgcmVhbCB2YWx1ZSwgbWFzayB0aGUgd2Fu
dGVkIGJpdHMgYW5kIHdyaXRlIGJhY2s/DQoNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9s
ZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:01:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:01:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180715.327479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXuv-0008Av-2H; Tue, 07 Sep 2021 10:01:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180715.327479; Tue, 07 Sep 2021 10:01:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXuu-0008Ao-Uz; Tue, 07 Sep 2021 10:01:24 +0000
Received: by outflank-mailman (input) for mailman id 180715;
 Tue, 07 Sep 2021 10:01:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNXuu-0008Ai-0M
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:01:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXul-00019S-DD; Tue, 07 Sep 2021 10:01:15 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNXul-0000A5-6A; Tue, 07 Sep 2021 10:01:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=PkCgRNrbFPgPtKA37oxT20cFd+oz/vvNiad1HjOCvq0=; b=GV/ghNPTj7NjTiTquDGTwIIuK3
	wHp+5s0AAtFICUwV6aX8CXeqpg8pNsNqw6jSWj0C248h9LtaaLOHaGk6rA11l5GW2uzkdB7kYFADy
	W6efVik1PCy1enFFxYA1QfdowzGGbOLUXpvl8qFaauFHQVxeJq1w2IcaRE35MvORsOTo=;
Subject: Re: [PATCH v1 01/14] xen/pci: Refactor MSI code that implements MSI
 functionality within XEN
To: Rahul Singh <Rahul.Singh@arm.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>, Jan Beulich
 <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Paul Durrant <paul@xen.org>,
 Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <d0b7b457762d481b19c8da6c2d55ff4acb4d6291.1629366665.git.rahul.singh@arm.com>
 <9022cf38-532f-05bd-d78d-50f46cdbd8f5@xen.org>
 <4909CE53-15AE-4A6D-A32D-69B5E6AC6DDD@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <10505763-238a-1a9b-88cc-b19312a16e44@xen.org>
Date: Tue, 7 Sep 2021 11:01:12 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <4909CE53-15AE-4A6D-A32D-69B5E6AC6DDD@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 19/08/2021 15:16, Rahul Singh wrote:
> Hi Julien,

Hi Rahul,

Sorry for the late reply.

>> On 19 Aug 2021, at 1:18 pm, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Rahul,
>>
>> On 19/08/2021 13:02, Rahul Singh wrote:
>>> MSI code that implements MSI functionality to support MSI within XEN is
>>> not usable on ARM. Move the code under CONFIG_HAS_PCI_MSI flag to gate
>>
>> Can you clarify what you mean by not usable? Is it because we lack of support or we have no plan to use it?
> 
> We have no plan to use it. Code moved to CONFIG_HAS_PCI_MSI will only be required for ARM if we
I read "not usable", as the code doesn't yet work on Arm. This is quite 
possible, but from you wrote the main reason is that the code is not 
necessary yet (see below) on Arm.

> decide to support PCI device access (PCI MSI interrupt support) within XEN. As of now, we are planning
> to add support for PCI device access for DOM0/DOMU guests not for XEN.

This is probably going to happen sooner rather than later as, AFAICT, 
there are SMMUs out which signals using MSIs.

But, AFAICT, this code would also be used if we need to manage the MSIs 
in Xen on behalf of the guest (such as if the platform is using GICv2m).

That said, I would be fine with gating the MSI code behind a new config. 
However, I think the commit message wants to be clarified into why we 
don't need the option. Maybe something like:

"On Arm, the inital plan is to only support GICv3 ITS which doesn't 
require us to manage the MSIs because the HW will protect against 
spoofing. Introduce a new option XXXX to protect the MSI code.
"

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:04:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:04:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180721.327490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXy6-0000OC-IB; Tue, 07 Sep 2021 10:04:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180721.327490; Tue, 07 Sep 2021 10:04:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXy6-0000O5-EM; Tue, 07 Sep 2021 10:04:42 +0000
Received: by outflank-mailman (input) for mailman id 180721;
 Tue, 07 Sep 2021 10:04:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNXy4-0000Nz-Rn
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:04:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8c641a18-2079-4ef0-b785-ddea4c9fce01;
 Tue, 07 Sep 2021 10:04:39 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-9-hpqGuoH9PGq115orbvAkig-1; Tue, 07 Sep 2021 12:04:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 10:04:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:04:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0086.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 10:04:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c641a18-2079-4ef0-b785-ddea4c9fce01
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009079;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=mESkZMCn3zmsbiDZ07Z3XiuHmv0bzZgcJCIsETD9bV4=;
	b=f9te18dHO8t7eLXe6jKnWru5DmTbFdpaqOanYQOAK/1Ki+v7RUdsnr6jzD6IlHl+x3cbwW
	6nfYDgOasNNen5DFzxQrk0Sj73GVnhAyPWoL/kYIvndiqCoDe2c3rsM51qAh7MEk9+IS07
	+7psCLuIGL+JJbr1cJOOHY+FmQ9QANU=
X-MC-Unique: hpqGuoH9PGq115orbvAkig-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FBFcSFwxe8k9APKb6ChixWW5QTh3KUK3lQrk/5ZSjQzECd983whDtHVuXV+zVKW3hLWUTZYNZQ9ibCDanbCTJ93aBRVDLYcxCHUlEyOBl8f3ZhZJ2DD8zJ7ijJblnMNH7C25LN3d//X65rooVLtr4ZXBgKbnmuouQh8JQOj1kpinW0Ve9UBRmo/MgYJgSX3TMYiOaloEOclRIW/YnjwAGZUg+4h3bc0adJEKyl92UkdDjUx2eaSSnFoI22Hb/gwaVMYD+x9mPFZFsIWtDZwMqQPD7v7Ys9O6qqZalJlhF39PVdOsVDn9sxR8Oy+A4pQRzHagpwNkYDsMAzRIDCngPA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mESkZMCn3zmsbiDZ07Z3XiuHmv0bzZgcJCIsETD9bV4=;
 b=OJWnPLAL1KTnRcJZuu48HMMOASP8nA6t5tNHyXgJIZkMuEFwwbiMnLfqz7rWylxbJrNj2HpnhQ9e+gL+f3/4wfsR/rlkPa2erXsW0TCpnevVHY10YH3Nc0Khtgpa+X5ZlqCCGHMDBqrCz0AzFmIDckWUxd6lYysjv3nTHeJLT9U29m2VPikckQ/Gn9TBngC94rXbjWxd3j8NQcV0R55ksXRGcPe8VdIxoVcT17iVvOSfOQ58xr1UbchOBHHfSnCsINO2oYqPYTUjiZk1EaapZLR8ZIxwVLxKlqvNeN3u1f/z6vFQ5Z9x2tZFWHBA6U2YDiuipOnXXTQ1b7Lg2pUsUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
Message-ID: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Date: Tue, 7 Sep 2021 12:04:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0086.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3f7bf4a6-61b8-4bae-9838-08d971e6e5de
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6861E95D1B4BAD83CC93F0DAB3D39@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2HFuW6qjrpDU+nMRA4sqT+UmRulKiRMvir2UGiWWhZJVi+TyTBfaarxs2jLscRYOX5OtsWFqOt7FbB4P0DYcKn9DufThRExjmgt7Qm7vaEUR8A/OhixtQbLZMnp3Ak8n/PLU1U/z1fFfgYKCiqhgHJJQxeqdRg8f5AcX19KkPEw9Ws2AZ0GKrAxnesGTzL9ASdZP60khriLDDb5CI1igK1JHMJ3b5Ub+gKrCUUQ52grVKgsLqKkB9kubmo7d4D+DKC1lYi6j8Z30pW4DCZneL4oroOkFrSZIylgOxiOJlI7yGbYKyo8lJEMSRKCccNyPmRBYFUYxPqtHFLUzVgap5PEARjmd6OfMvtqWQh2Z5iUQuJOiUipIX1WFt3GGBuN3WS1EzZySj4k3ZTj0ZgXDUZjzwb8pjZ8YWMRdrdT3EZuvXq2ci4tt4R4OGLc1wI89PN32cXBq0c06CjKCY0pT0LwIYTsxDb9uhx3Ip+qlj1sEd/AyOO6rkK8dCoO/PrM/a/udwDKVwft+fYsuThECxK1ml9beKZct7g06RzHyTGqktSzoHTB+lmT8Mh0sYUWT6nvSA+g05bKDWgF74YHCVPpPa90mH9o0jMHEbGK4tc8cKWIMAQMYGRLAZ84HdklWRmczRpOMXyqRLamPT2idtD2MtaiQQViYst8nF8Zu8eh3gMTsX14UfN4TAlF7qLfBWu7663XALdWvltHnjoN1gQdfj7A2uhKI5Mgoq5Oa1anf14sfQpYuXs8BB5Oy73ZA
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(376002)(366004)(346002)(316002)(16576012)(54906003)(83380400001)(86362001)(2906002)(31696002)(110136005)(6486002)(478600001)(36756003)(38100700002)(4326008)(31686004)(186003)(26005)(66476007)(66556008)(8676002)(66946007)(8936002)(956004)(5660300002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VlJSSjlBZ09IbHV2Z0NoZXZzS2M0eGxYb0JLeTlyR0V5QkJZT25RUkcrNDVN?=
 =?utf-8?B?QjBHUEVOaFIyYVlMT1lWSFBPN2V5MC9wV3EvUUU5MzhSWmpmcmcyakIrck5z?=
 =?utf-8?B?U25yRE9QbmNIaHRvazYyc1FsUDFaaWR5cUtGdzFmREQzV0pVTWhRVU5lRzV0?=
 =?utf-8?B?TWlWVmZRMkx0bFNjSzBLdHdWbEZwak55K1Q3czRZWVVpcDl4aGFYUERqUnBw?=
 =?utf-8?B?dXpuZVJ1S01qZVEwaTdXK0lsUkMyeWwvRHRxbkVLZW96M0Z3N0lsWjdnbG9Z?=
 =?utf-8?B?NDRiclFMRjRSNjB4NkI4WDJydUUyL3VYOG5hYThHWEZWbUxhQjl2ZlZBeVZX?=
 =?utf-8?B?QXZHR0VMcHRNR1RRNHl5R2xhbGJqZW1udnJ3ZndleEIvSWlCK2hvUlhTa3dE?=
 =?utf-8?B?a1FmaWoyTFBMNENRYXRnTVRSNnROTHRkZlY4dnZUVjlQQkZGbTVDeUJTK3Bq?=
 =?utf-8?B?Q25TZCtJU0RwQTA2cGcxL1JIUnZDUHlsRFV1cW9oTnRwalZnQzZKdUx1aGFj?=
 =?utf-8?B?N0x2T3RwcmtsbENPQlZrSTBFOS9kNzNQTHBpZ0xubERmc0p6USs0R0duR3d1?=
 =?utf-8?B?QkZOZWdVSDJoZHZlMnUxKzhwOXZpYjRRNnV1ZVNRU1crVFA5ZS90OTh1SUFU?=
 =?utf-8?B?cEgvSHVSb21BOHgzM2FlSEdFdHZtSXlKQmFUQVo5NFFQVS9CcWQrRjZ3R2dk?=
 =?utf-8?B?bmVXelU4UGlIMUVuRTZuQnY2NkNwRzRIUzFpZnc5bWMzdktIMWtQZW9QdHpl?=
 =?utf-8?B?TFlNNXphT1g0Q2dJTTlqZi9zblk0WEdaTVFsRnVxVThlNDJzaUtVbU1WVnpP?=
 =?utf-8?B?ZGRqbGhUZjh4YUJ1ZjNERmFiUjl4QjM3cWMrZXFtMXBiZXhPdjg0aWp4OVpB?=
 =?utf-8?B?cWc0b2tXQ2wvWlV0dEJQcjlxUU5QQ0orVzhmUU1LaldXcXNCQkpjcVNUbDgx?=
 =?utf-8?B?MVdYSGR6UkNmL2FmYk1vRlFRaHhNTGRjaWpUWTVMUEhENXFZNk02WWlGS2R1?=
 =?utf-8?B?dVNmVDlwUmFXQTBITUhQZzlzOXE3MStVcXVkRWduYWxNSXpYUUxLMDhBUG5J?=
 =?utf-8?B?OHROU3RTKzlHa3c4UzVSaUlGQytHOWVXZFFOVnZkQzRQUERjaGsraDFWVnF6?=
 =?utf-8?B?ZHhpcTZHcWVXbkdrNjFuRmFoY0duODhSTk1xRmtZbTdVWDNhRkxPeUpSczFj?=
 =?utf-8?B?bzlldTF1em04TnM4VWFNbCtSZktOSzhKck1GWGFHaFMzWXJRc2pBV1ZNaEM5?=
 =?utf-8?B?TEx0ZjAxS2dFUTJGNDZSRVhsanRFS1hCd1YzdFAvekQvOU5lN0xCcFFTY0t4?=
 =?utf-8?B?U2JPOC9sVHU3MC9ySXduU2J0SUdMU2wwKzFnZXFmL3hrdWtyWkpkbXBpRTNH?=
 =?utf-8?B?Qjd2UUMvY3JzQUFhdHU2TUNvdGJQN2VtbkhiZXhPbVBXa1FDeEJQL2hNZkc0?=
 =?utf-8?B?cDFsejdWd1dPaEFIZnVsbW9CL1ZDOVhhaFRJZVdteUhkeTRQaHhjMTVTaWEv?=
 =?utf-8?B?dEwydG1Ydjc4M29XZEFFaGxZeG5WendIOFdKUm9admFodllESVJhTDhNRzFZ?=
 =?utf-8?B?K2dmOXN1SkZ1RjJBRWhBbklvb2oxVStteUN5am8xa0hDRnZvUXYvVVB2TCsy?=
 =?utf-8?B?ejNJb21XeWV4MzArejRZNHYreEpaZWE5cXFScTlFOUI4VjgyU2hiMU1IQW1y?=
 =?utf-8?B?MTFWcXRoVC8xUWxWZldZSnI1NzdUaWpySGFta2VjWHFmM2t3RjUvNzdyRGNU?=
 =?utf-8?Q?t/TZlfT7VPwDUXdKhEkGyH+b5fvEBIJB5aqBUYh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f7bf4a6-61b8-4bae-9838-08d971e6e5de
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:04:36.3542
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jUdCjTt80pqdIGPC1zPRU8QD1iXXM9sL+g3kvZyTWuaPBXu9MbyA21lMB215uSiHDcysixCMNOPLx3hCRfwWkQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

In order to try to debug hypervisor side breakage from XSA-378 I found
myself urged to finally give PVH Dom0 a try. Sadly things didn't work
quite as expected. In the course of investigating these issues I actually
spotted one piece of PV Dom0 breakage as well, a fix for which is also
included here.

There are two immediate remaining issues (also mentioned in affected
patches):

1) It is not clear to me how PCI device reporting is to work. PV Dom0
   reports devices as they're discovered, including ones the hypervisor
   may not have been able to discover itself (ones on segments other
   than 0 or hotplugged ones). The respective hypercall, however, is
   inaccessible to PVH Dom0. Depending on the answer to this, either
   the hypervisor will need changing (to permit the call) or patch 2
   here will need further refinement.

2) Dom0, unlike in the PV case, cannot access the screen (to use as a
   console) when in a non-default mode (i.e. not 80x25 text), as the
   necessary information (in particular about VESA-bases LFB modes) is
   not communicated. On the hypervisor side this looks like deliberate
   behavior, but it is unclear to me what the intentions were towards
   an alternative model. (X may be able to access the screen depending
   on whether it has a suitable driver besides the presently unusable
   /dev/fb<N> based one.)

1: xen/x86: prevent PVH type from getting clobbered
2: xen/x86: allow PVH Dom0 without XEN_PV=y
3: xen/x86: make "earlyprintk=xen" work better for PVH Dom0
4: xen/x86: allow "earlyprintk=xen" to work for PV Dom0
5: xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
6: xen/x86: generalize preferred console model from PV to PVH Dom0
7: xen/x86: hook up xen_banner() also for PVH 
8: x86/PVH: adjust function/data placement
9: xen/x86: adjust data placement

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:05:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:05:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180725.327501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXyV-0000w9-RN; Tue, 07 Sep 2021 10:05:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180725.327501; Tue, 07 Sep 2021 10:05:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXyV-0000w2-Nk; Tue, 07 Sep 2021 10:05:07 +0000
Received: by outflank-mailman (input) for mailman id 180725;
 Tue, 07 Sep 2021 10:05:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNXyU-0000ue-4w
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:05:06 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 129d972a-0fc3-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:05:04 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1879h7RO005781; 
 Tue, 7 Sep 2021 10:05:02 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ax2jngtf4-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 10:05:01 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB7143.eurprd03.prod.outlook.com (2603:10a6:20b:23e::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 10:04:59 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 10:04:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 129d972a-0fc3-11ec-b0e1-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ayg+E8KbkwmM/To98WalRkwreO+Jsd6A+YUrsNzqgMfpfHN1QJnwQArMvKHqhIe0sCOGGBGOPnge+gYWcJNLVS5KA2mMRrMSJMhKF7qtQfEcujPYajHWhaMpdm+mRdspGSb3hsyZNlINjIbyP9a8Sq8mKwf45mydQ+d+Pjw2M22nu2Y4Q6d/N5vsCbiwK0zOmgT0C7gF5Ia9k+emuZFM76IqPrA6Lh/URBYz5LqRGFVv/SEx0fqNUGb+TtSQtPr3qaE9TFYsBaT7fhlJ2fWbu9dLiTi96xfJMOxSiVt1XMSgBeYDoWRE+ZPJ7SFO/vKSAJ5kiPIDNV4fSKbht88RGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/TQLdNk+Reu+foPLnB0xZnQ6fPvTin+Fuvfe22LzTIA=;
 b=ogkdl417/AE/+0h6wxo3Nt++pni2qrzHHSPjHPO5hs/39SowMTII0VBv1yDt4Glj68ZN0ZCug5mbz731uEJyjjjUbrMHTLIPX8C534sCTSQSQff7wymIkmWS1mudcvhc/bxSGvBng5ogsVDcwJowfnGHQpw930l144cAGnrlNnQgn+QOesTI7y/XTixafbY4MXnDoUPiSGzlIehsrN/NFxdsNb8x6arJg4T5E/BWMeLslEcRgQzBJeJnu6YOiif2kAVtZy29V8xmlGkEvuhoX+7EkiW7O0NpBCDlTIDIAoJlI3aH5fjdM/tVWVMXwJzUgeEcnyVMdEEN5CH+vKVo4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/TQLdNk+Reu+foPLnB0xZnQ6fPvTin+Fuvfe22LzTIA=;
 b=LkiCAsqVH+bGUx7o/BahbNM8k7E1C3mLq9UV/ff4M0pjfR4Bmri0bpuaIhMus4zECeTAkY9FtqsIYPO2k1r+9qpmp3BBEYUyUTtHS9VNOUhpquZdUx0YtF1tGaFLYar/VlHl1ea6SWby8kLLwZ/3crUtAJoFArjdcdm43jUf3xTFEVYqWO0dyDp2w2SYZNSJS1viPhj9pIzMgf9mWzriXIC50ZtZVmN07OSIaVC4DkLvNUJGs8F52G9NrN0rmanEK78KqjKVvX8epkXL/ui6Bkr2EXZ7iatDFemHSHWvZYem5OsNhX3bJUcruzCcCoejL3jr+EFRmkWbM6oRIPYuFQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?=
	<roger.pau@citrix.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        Rahul Singh
	<rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 3/9] vpci/header: Move register assignments from init_bars
Thread-Topic: [PATCH 3/9] vpci/header: Move register assignments from
 init_bars
Thread-Index: AQHXoKwh6IBgDEFZ2k+LKlDV/Oq/mKuXC7EAgAFSlwA=
Date: Tue, 7 Sep 2021 10:04:59 +0000
Message-ID: <218da10d-2280-4f15-4ede-b5fadcb8c59d@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-4-andr2000@gmail.com>
 <4007d84d-9e76-32df-58f6-af5ff26fdf6e@suse.com>
In-Reply-To: <4007d84d-9e76-32df-58f6-af5ff26fdf6e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1029a4a1-8877-4620-33e7-08d971e6f3ba
x-ms-traffictypediagnostic: AS8PR03MB7143:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB71432F484D3BC1037C9F0BDFE7D39@AS8PR03MB7143.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 gMV8TXCLUjV5f4GQOCYNGLik8bzWsYRiwSPS0QepALIwMsH6m1WEI0qWiaypF2vX3cjgwwfabjcWaV/qNNX/F70KVQ7V+uFmTODDcTSPAhEBK/hMPN3c+fOTkyeFOTpegojSUF0T1p6+l1kgooJCXDzQNHG7DbDjCgJ3NzqQl7q3aetiigcmkxcer0jQATSnInncnxJn/8V8DtzJsv/yKEO2ZmhbrYvYrfJ5LsYnwkq4Wzfb6i/FENTTMqLnDSXlqJV/8RQMCIdtwAQ81PMi4xbSDZVdYpOSJPUcgftBG0yBJNxQ831SCim0bJkDBu3rholf1u9VmJz5XOmnuWVFcLyRH8VI4ziP+Qu5xR6vzcewvSJg41lUES/7+sObL7Rbmp27fiO0pvDk1gn93+787KJzA7ToaLmzgNC670hcCEdHe6XZqgRv8cRVEdEg+dqveTuQhza4f6llx09lCouPKXyVE0afasuR+3uUpkIM5yxMNA/ASTFBFkykSqKRgahKfNtw1n5oGel5TWjKdIvOTS7zZSOaUBZzgHCL5WtbpjcR0y2XjwtdqDQN4wLt8VfWWSm1f42xfKGJkgoXId3yQOXDsBUaTtIfZ7grKFVlnWSATTsIKMe8IDlaICHMGUTROVn/P7bdtG4BdRhPc6OUFJ+bU2fUegmot+RwvbQo42cA+O7iqKs3QRx2BV9y1dAJUlCWwTNpadebhlu2HWQJRZe5+dOkxTaXPl0gk+/QDErKVbAUDcFWEGN419gUBhUO
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(376002)(136003)(346002)(39860400002)(396003)(2906002)(8936002)(478600001)(8676002)(71200400001)(86362001)(38100700002)(2616005)(5660300002)(38070700005)(31696002)(4326008)(122000001)(76116006)(6486002)(91956017)(31686004)(53546011)(66556008)(66946007)(66446008)(64756008)(186003)(83380400001)(66476007)(36756003)(110136005)(54906003)(26005)(6512007)(6506007)(316002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aFZRa29KbTZxUzNydUlhQ2h2NnJJQ3V0SjNEcURDK1VSVVdUV0VUZDRXb0ZL?=
 =?utf-8?B?QlpWd1NndlJxeWNuVEhHWnZya294UVlGME52TWxHdE1xYVd4RVowRDJTSUtW?=
 =?utf-8?B?M010MitmbStpaUtwdmZMOEN1TEhweS9CWERpU1gzRUpWVDN0bk1mTjNWNlAv?=
 =?utf-8?B?OW1sU2NrbWFnMTljbW15eTcrQWV5ZldWUXRYNlV2Z3B5Q2dWWmwyMXJsT0lt?=
 =?utf-8?B?QkN6TUc5V3ZXd0xveXlqSFdGMFVLMnNtb2hucEpILzNlSU9DOEdsUHM3S0xI?=
 =?utf-8?B?eUYraEg0ZTBsLzZWTklxUTZVbnNVUXErWlFGZ3dhTjJIMU1qczRkdmJlUkhZ?=
 =?utf-8?B?RG5mSk84SDY0bENnOE14RlBXTXVQd3JOeEdNWHdhTHRUVjhRbDhhQUpDZVdJ?=
 =?utf-8?B?b3NBd213Q3JrNnZsaDRUamhodVZhQmVQTzRNSkJVZUhjVWtpMlFMY0Q0azlK?=
 =?utf-8?B?SnNnVHRtVXRpNkZ2ZWpFY01GdGtMZmdEaEZjam1vNjRUZm55WnJkcVpBdnly?=
 =?utf-8?B?Y1FKS3BsZVZRcHdMWkM0b0tTcFl3ZnZmSXlUY05uR2wySHhmWit1V3l5MUJj?=
 =?utf-8?B?bnB3SGxXSnhPOUd2Z1A4a2pLWFFFKzg0OEFZOXBZMjY4UUpjK3N4MXdyR2Vv?=
 =?utf-8?B?QjJwU3A2MGVWcVdMN3JubTN1cStxNWtjQkJTVDIvcDZmc1RSc1p0T2NyZVl6?=
 =?utf-8?B?eDhDOGdkVHhPcVV5b1NYWndpM2M2azIyR3VEWnJrZE9zeXNyc1ZCSGJpR3M5?=
 =?utf-8?B?d3BEK0h6K2hla2Z1ZGFYK3pLUzBQN05kSmVodFo3NVI5T1R0TDZLWldtaGht?=
 =?utf-8?B?eDExQ0pHYXBpZW1MZStNS0lQZmpwbW9IZzk4dzV5ZlB1bkdlSUtnRDIrK3dC?=
 =?utf-8?B?UktHbDRmR3VRVjN3Zyt4TjIvRWlvUlNYZDdiWHZtNlZOL1BwYnkxbXdueEkr?=
 =?utf-8?B?eFFZTXd5MEhocmVKUDhwVndqTHZ3MXhNdTNGTm1Dc3dEdzBSYjJjNjhIMStm?=
 =?utf-8?B?RjRYOWlZVy9aYUlHZm9JaUVCSXkxQ3VBbW9ZVm9vK1ZqNVpGSHE2d1pEdm54?=
 =?utf-8?B?OTZqVTlIZzNRZERJbDB0Z2g1RHVHOTh6TVl2SzJQcmNQTDNjVXNTaCtXbHBF?=
 =?utf-8?B?UXRJWEorVjdoMit3UFRIdDhqMVF4QlBETHB2WmFEWHJGMEY0azFWZ0pHMDEw?=
 =?utf-8?B?WG9hS2JUWTFCbzdqTWZKd2s2NjYzaVhTQXNpdDhDY29oUTRwNi8vNDlyVGVN?=
 =?utf-8?B?Y2FSU01lZm52YkhrQjRVdGorSUl0NzQyRUpNd2hsRXkyY2hseEdlNHU2NDdO?=
 =?utf-8?B?a01xWVRmQ2c3NEVYMVd1cGNvMk1TNjhSSklydUY4V3RIaUdvZUF3U1hqb0RX?=
 =?utf-8?B?dG5jbVEyR210bWlLNUE4bFRGL283WWdZT2w3eFJVRS9ZdllleUdnNnY0REh6?=
 =?utf-8?B?amRjS0ZFeVBoejlhRmpWNW9YU3NLVHVKa0J1YXZtVE1RM2hLZGFJRlhNNnNw?=
 =?utf-8?B?OVlBVUxLVDhiL0NwSXE3N3R6dk8vSWdlVFRLY0dOWjhtV2xOQitJVmQvN2t3?=
 =?utf-8?B?Yklmc3VWMkcwWEZ2VHBna2dDd01pbkdDMysvREdQaSthcmY0bWZBOUV3UXEx?=
 =?utf-8?B?WER6bXFMOWozd2RYK1daMlVLcllsa0ZCblRXM3phdiswTlp3VDQvaDcvSnBu?=
 =?utf-8?B?dmQ2UFQ3VmlDaWh1UVZpWmE0UGl2SzdRMjlOV00wSHQvdkgrbERmcTc2WVdS?=
 =?utf-8?Q?HUnwe21BfJs596oqiQACyXFrwm9VA5BahQm3WQa?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <7DEB72B1234C29479B61DD896A6910AC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1029a4a1-8877-4620-33e7-08d971e6f3ba
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 10:04:59.4103
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: JRSmn8T68mW3t0qHf94B3kpILE8lHDnhxYd0bXPvdecV6yqMOKV+o+9qQMd7khxKdLxWelIqml2AKbqBdE9T1vaalR3SfCEsssYv16jq21LuzvSDruqMLTLKmSyNZdgv
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB7143
X-Proofpoint-ORIG-GUID: p4zJE7BS5DzZqRP-HuOsAJv7Lh9KXsnn
X-Proofpoint-GUID: p4zJE7BS5DzZqRP-HuOsAJv7Lh9KXsnn
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_03,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0
 clxscore=1015 priorityscore=1501 impostorscore=0 phishscore=0 spamscore=0
 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070066

DQpPbiAwNi4wOS4yMSAxNjo1MywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+
IFRoaXMgaXMgaW4gcHJlcGFyYXRpb24gZm9yIGR5bmFtaWMgYXNzaWdubWVudCBvZiB0aGUgdnBj
aSByZWdpc3Rlcg0KPj4gaGFuZGxlcnMgZGVwZW5kaW5nIG9uIHRoZSBkb21haW46IGh3ZG9tIG9y
IGd1ZXN0Lg0KPiBJIGd1ZXNzIHdoeSBleGFjdGx5IHRoaXMgaXMgZ29pbmcgdG8gaGVscCBpcyBn
b2luZyB0byBiZSBzZWVuIGluDQo+IHN1YnNlcXVlbnQgcGF0Y2hlcy4gVG8gYWlkIHJldmlldyAo
aS5lLiB0byBub3QgZm9yY2UgcmV2aWV3ZXJzIHRvDQo+IHBlZWsgYWhlYWQpIGl0IHdvdWxkIGlt
byBiZSBoZWxwZnVsIGlmIHlvdSBvdXRsaW5lZCBob3cgdGhlIHJlc3VsdA0KPiBpcyBnb2luZyB0
byBoZWxwLg0KDQpTdXJlLCB3aWxsIGRvIG5leHQgdGltZS4gVGhlIG5lZWQgZm9yIHRoaXMgc3Rl
cCBpcyB0aGF0IGlzIGl0IGVhc2llciB0byBoYXZlDQoNCmFsbCByZWxhdGVkIGZ1bmN0aW9uYWxp
dHkgKEJBUnMgaGVyZSkgcHV0IGF0IG9uZSBwbGFjZSBhbmQgd2hlbiB0aGUgc3Vic2VxdWVudA0K
DQpwYXRjaGVzIGFkZCBkZWNpc2lvbnMgb24gd2hpY2ggaGFuZGxlcnMgdG8gaW5zdGFsbCwgZS5n
LiBod2RvbSBvciBndWVzdCBoYW5kbGVycywNCg0KdGhpcyBmdW5jdGlvbiBpcyBleHRlbmRlZCB0
byBhY2NlcHQgYSBvbmUgbW9yZSBwYXJhbWV0ZXIsIGlzX2h3ZG9tLCBhbmQgYWxsDQoNCnRoZSBh
c3NpZ25tZW50IGxvZ2ljIGlzIHB1dCBoZXJlLiBPZiBjb3Vyc2UgSSBjb3VsZCBoYXZlIGFsbCB0
aGUgImlmIChpc19od2RvbSkiJ3MNCg0KcHV0IGF0IHRoZSBvcmlnaW5hbCBsb2NhdGlvbiwgYnV0
IGRlZGljYXRlZCBmdW5jdGlvbiBsb29rZWQgY2xlYW5lciB0byBtZS4NCg0KPiAgIEFmdGVyIGFs
bCAuLi4NCj4NCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+ICsrKyBiL3hl
bi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+IEBAIC00NDUsNiArNDQ1LDU1IEBAIHN0YXRpYyB2
b2lkIHJvbV93cml0ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldiwgdW5zaWduZWQgaW50IHJl
ZywNCj4+ICAgICAgICAgICByb20tPmFkZHIgPSB2YWwgJiBQQ0lfUk9NX0FERFJFU1NfTUFTSzsN
Cj4+ICAgfQ0KPj4gICANCj4+ICtzdGF0aWMgaW50IGFkZF9iYXJfaGFuZGxlcnMoc3RydWN0IHBj
aV9kZXYgKnBkZXYpDQo+IC4uLiB0aGlzIGZ1bmN0aW9uIG5hbWUsIGZvciBleGFtcGxlLCBpc24n
dCBEb20wLXNwZWNpZmljLCBzbyBvbmUNCj4gbWlnaHQgZXhwZWN0IHRoZSBmdW5jdGlvbiBib2R5
IHRvIGdhaW4gY29uZGl0aW9uYWxzLiBZZXQgdGhlbiB0aGUNCj4gcXVlc3Rpb24gaXMgd2h5IHRo
ZXNlIGNvbmRpdGlvbmFscyBjYW4ndCBsaXZlIGluIHRoZSBvcmlnaW5hbA0KPiBmdW5jdGlvbi4N
Cg0KQW5zd2VyZWQgYWJvdmUuIEkgdGhpbmsgaXQgbWFrZXMgY29kZSBjbGVhbmVyIGFuZCBlYXNp
ZXIgZm9yIG1vZGlmaWNhdGlvbg0KDQphcyBoYW5kbGVycycgYXNzaWdubWVudCBmb3IgQkFScyBi
ZWNvbWVzIGdyb3VwZWQgYXMgaXQgaXMgZG9uZSBmb3IgTVNJL01TSS1YLg0KDQoNCj4NCj4+ICt7
DQo+PiArICAgIHVuc2lnbmVkIGludCBpOw0KPj4gKyAgICBzdHJ1Y3QgdnBjaV9oZWFkZXIgKmhl
YWRlciA9ICZwZGV2LT52cGNpLT5oZWFkZXI7DQo+PiArICAgIHN0cnVjdCB2cGNpX2JhciAqYmFy
cyA9IGhlYWRlci0+YmFyczsNCj4+ICsgICAgaW50IHJjOw0KPj4gKw0KPj4gKyAgICAvKiBTZXR1
cCBhIGhhbmRsZXIgZm9yIHRoZSBjb21tYW5kIHJlZ2lzdGVyLiAqLw0KPj4gKyAgICByYyA9IHZw
Y2lfYWRkX3JlZ2lzdGVyKHBkZXYtPnZwY2ksIHZwY2lfaHdfcmVhZDE2LCBjbWRfd3JpdGUsIFBD
SV9DT01NQU5ELA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgIDIsIGhlYWRlcik7DQo+
PiArICAgIGlmICggcmMgKQ0KPj4gKyAgICAgICAgcmV0dXJuIHJjOw0KPj4gKw0KPj4gKyAgICBp
ZiAoIHBkZXYtPmlnbm9yZV9iYXJzICkNCj4+ICsgICAgICAgIHJldHVybiAwOw0KPj4gKw0KPj4g
KyAgICBmb3IgKCBpID0gMDsgaSA8IFBDSV9IRUFERVJfTk9STUFMX05SX0JBUlMgKyAxOyBpKysg
KQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBpZiAoIChiYXJzW2ldLnR5cGUgPT0gVlBDSV9CQVJf
SU8pIHx8IChiYXJzW2ldLnR5cGUgPT0gVlBDSV9CQVJfRU1QVFkpICkNCj4+ICsgICAgICAgICAg
ICBjb250aW51ZTsNCj4+ICsNCj4+ICsgICAgICAgIGlmICggYmFyc1tpXS50eXBlID09IFZQQ0lf
QkFSX1JPTSApDQo+PiArICAgICAgICB7DQo+PiArICAgICAgICAgICAgdW5zaWduZWQgaW50IHJv
bV9yZWc7DQo+PiArICAgICAgICAgICAgdWludDhfdCBoZWFkZXJfdHlwZSA9IHBjaV9jb25mX3Jl
YWQ4KHBkZXYtPnNiZGYsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIFBDSV9IRUFERVJfVFlQRSkgJiAweDdmOw0KPj4gKyAgICAgICAgICAgIGlm
ICggaGVhZGVyX3R5cGUgPT0gUENJX0hFQURFUl9UWVBFX05PUk1BTCApDQo+PiArICAgICAgICAg
ICAgICAgIHJvbV9yZWcgPSBQQ0lfUk9NX0FERFJFU1M7DQo+PiArICAgICAgICAgICAgZWxzZQ0K
Pj4gKyAgICAgICAgICAgICAgICByb21fcmVnID0gUENJX1JPTV9BRERSRVNTMTsNCj4+ICsgICAg
ICAgICAgICByYyA9IHZwY2lfYWRkX3JlZ2lzdGVyKHBkZXYtPnZwY2ksIHZwY2lfaHdfcmVhZDMy
LCByb21fd3JpdGUsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByb21f
cmVnLCA0LCAmYmFyc1tpXSk7DQo+PiArICAgICAgICAgICAgaWYgKCByYyApDQo+PiArICAgICAg
ICAgICAgICAgIHJldHVybiByYzsNCj4gSSdtIG5vdCB0aGUgbWFpbnRhaW5lciBvZiB0aGlzIGNv
ZGUsIGJ1dCBpZiBJIHdhcyBJJ2QgYXNrIGZvciB0aGlzIGFuZCAuLi4NCj4NCj4+ICsgICAgICAg
IH0NCj4+ICsgICAgICAgIGVsc2UNCj4+ICsgICAgICAgIHsNCj4+ICsgICAgICAgICAgICB1aW50
OF90IHJlZyA9IFBDSV9CQVNFX0FERFJFU1NfMCArIGkgKiA0Ow0KPj4gKw0KPj4gKyAgICAgICAg
ICAgIC8qIFRoaXMgaXMgZWl0aGVyIFZQQ0lfQkFSX01FTTMyIG9yIFZQQ0lfQkFSX01FTTY0X3tM
T3xISX0uICovDQo+PiArICAgICAgICAgICAgcmMgPSB2cGNpX2FkZF9yZWdpc3RlcihwZGV2LT52
cGNpLCB2cGNpX2h3X3JlYWQzMiwgYmFyX3dyaXRlLCByZWcsDQo+PiArICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICA0LCAmYmFyc1tpXSk7DQo+PiArICAgICAgICAgICAgaWYgKCBy
YyApDQo+PiArICAgICAgICAgICAgICAgIHJldHVybiByYzsNCj4gLi4uIHRoaXMgdG8gYmUgbW92
ZWQgLi4uDQo+DQo+PiArICAgICAgICB9DQo+IC4uLiBoZXJlIHRvIHJlZHVjZSByZWR1bmRhbmN5
Lg0KPg0KPj4gQEAgLTU1MywxMSArNTgwLDEzIEBAIHN0YXRpYyBpbnQgaW5pdF9iYXJzKHN0cnVj
dCBwY2lfZGV2ICpwZGV2KQ0KPj4gICAgICAgICAgIHJvbS0+YWRkciA9IGFkZHI7DQo+PiAgICAg
ICAgICAgaGVhZGVyLT5yb21fZW5hYmxlZCA9IHBjaV9jb25mX3JlYWQzMihwZGV2LT5zYmRmLCBy
b21fcmVnKSAmDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFBDSV9ST01fQURE
UkVTU19FTkFCTEU7DQo+PiArICAgIH0NCj4+ICAgDQo+PiAtICAgICAgICByYyA9IHZwY2lfYWRk
X3JlZ2lzdGVyKHBkZXYtPnZwY2ksIHZwY2lfaHdfcmVhZDMyLCByb21fd3JpdGUsIHJvbV9yZWcs
DQo+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDQsIHJvbSk7DQo+PiAtICAgICAg
ICBpZiAoIHJjICkNCj4+IC0gICAgICAgICAgICByb20tPnR5cGUgPSBWUENJX0JBUl9FTVBUWTsN
Cj4+ICsgICAgcmMgPSBhZGRfYmFyX2hhbmRsZXJzKHBkZXYpOw0KPj4gKyAgICBpZiAoIHJjICkN
Cj4+ICsgICAgew0KPj4gKyAgICAgICAgcGNpX2NvbmZfd3JpdGUxNihwZGV2LT5zYmRmLCBQQ0lf
Q09NTUFORCwgY21kKTsNCj4+ICsgICAgICAgIHJldHVybiByYzsNCj4+ICAgICAgIH0NCj4gU2Vl
aW5nIHRoaXMgbW92ZWQgKGhlbmNlIHBlcmhhcHMgbW9yZSBhIHF1ZXN0aW9uIHRvIFJvZ2VyIHRo
YW4gdG8NCj4geW91KSByZXN0b3Jpbmcgb2YgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgLSB3aHkgaXMg
aXQgdGhhdCB0aGUgZXJyb3INCj4gcGF0aChzKSBoZXJlIGNhcmUocykgYWJvdXQgcmVzdG9yaW5n
IHRoaXMsIGJ1dCAuLi4NCj4NCj4+ICAgICAgIHJldHVybiAoY21kICYgUENJX0NPTU1BTkRfTUVN
T1JZKSA/IG1vZGlmeV9iYXJzKHBkZXYsIGNtZCwgZmFsc2UpIDogMDsNCj4gLi4uIG9uZXMgaW4g
bW9kaWZ5X2JhcnMoKSAoYW5kIGRvd253YXJkcykgZG9uJ3Q/IEkgd2FzIHdvbmRlcmluZw0KPiB3
aGV0aGVyIHRoZSByZXN0b3JlIGNvdWxkIGFjdHVhbGx5IGJlIGRvbmUgcHJpb3IgdG8gdGhlIHR3
byBjYWxscw0KPiAob3IsIGluIHRoZSBvcmlnaW5hbCBjb2RlLCB0aGUgb25lIGNhbGwpLCBvciBw
ZXJoYXBzIGV2ZW4gcmlnaHQNCj4gYWZ0ZXIgdGhlIGxhc3QgY2FsbCB0byBwY2lfc2l6ZV9tZW1f
YmFyKCkuIEF0IHRoZSB2ZXJ5IGxlYXN0IHRoZQ0KPiBjb21tZW50IGZ1cnRoZXIgdXAgc3VnZ2Vz
dHMgbWVtb3J5IGRlY29kZSBvbmx5IGdldHMgZGlzYWJsZWQgZm9yDQo+IHNpemluZyBCQVJzLCB3
aGljaCB3ZSdyZSBkb25lIHdpdGggYXQgdGhpcyBwb2ludC4NCg0KRm9yIGFsbCB0aGUgYWJvdmU6
IHdoYXQgdGhpcyBwYXRjaCBkb2VzIGlzIGEgcHVyZSBjb2RlIG1vdmUuDQoNCkkgaGFkIG5vIGlu
dGVudGlvbiB0byBhbHRlciBpdCBpbiBhbnkgb3RoZXIgd2F5IHJhdGhlciB0aGFuIHRoYXQuDQoN
CklmIHlvdSB0aGluayB0aGUgY29kZSBuZWVkcyB0byBiZSBmdW5jdGlvbmFsbHkgbW9kaWZpZWQg
SSB0aGluayB0aGlzDQoNCmRlc2VydmVzIGEgZGVkaWNhdGVkIHdvcmsgdG8gYmUgc3VibWl0dGVk
LCBidXQgSU1PIHRoaXMgcGF0Y2gNCg0Kc2hvdWxkbid0IHRvdWNoIGFueXRoaW5nLg0KDQo+DQo+
IEphbg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:06:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:06:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180734.327512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXzU-0001cL-CA; Tue, 07 Sep 2021 10:06:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180734.327512; Tue, 07 Sep 2021 10:06:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNXzU-0001cE-8q; Tue, 07 Sep 2021 10:06:08 +0000
Received: by outflank-mailman (input) for mailman id 180734;
 Tue, 07 Sep 2021 10:06:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNXzT-0001c5-GY
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:06:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ca338be1-7622-4c61-9c6a-b161f093615f;
 Tue, 07 Sep 2021 10:06:06 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-HT3K6R6bOXiOq4H7_uRxjw-1; Tue, 07 Sep 2021 12:06:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4191.eurprd04.prod.outlook.com (2603:10a6:803:45::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 10:06:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:06:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 10:06:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca338be1-7622-4c61-9c6a-b161f093615f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009165;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oRW46sjfDAt94IjC1ydronO3b2EUU8JpbrPq1APVDwg=;
	b=ClVIIcFNWKH11waog8tT+45stenI5dgIN7mSHxifl92Okq/7+RYNgQI1vtP4zP79huDQzM
	1lPNcNBor3MK0VbYVpaq8H695jEUzKMuoZOWV7vTO12TVTsy5KTt5l8HnZdV5kvqO1C/mT
	iKhftAjDURz1vkYy6EeCzKFYUrjZMpY=
X-MC-Unique: HT3K6R6bOXiOq4H7_uRxjw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=br/rr9eNNwv2wbxAiQGxT85pEuXCZ3v8ssI/cP3YDJbK9F2aEoPQ8HYTqWLdlSqfnzT57NlKWIm78mk+uGB6Yjw0HoypEwKt5jWl++Jj9Be7QanAHg5N92m5EzdzEyRfMvL7HOFQT0OdmaTR7sJq9a2z1ft4Txp6ljig/MCb+E6ThXRlcfh+9k7gKiybolcGRQzuZPVsB5J3sn74fz0uitodXqyxxLY+JM2dLf+bQU0daD47+pHV0mRpJAo9kJEI8wM2xUWKzyGVZeYihTJWEWemltqnO/W3XG9es0RxWIsWi3nTtFw7LkXz1aTFOk1YD5Q101nJjqcr8LlEXUQPaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oRW46sjfDAt94IjC1ydronO3b2EUU8JpbrPq1APVDwg=;
 b=QGw6EeBoZ4+pqvxfQMADsclCJvG13l2hvGE9XCyZsP1A+HGsJ78BFeA9nQjz1Lad0TdQLC+p3JCOcXBjKYgOcnbmC8BET3Tv/LGkgAH5zm39alUg1C4TdhzM7Oo9Mgx8ZMJ0CpkJMDvHHNnONZghdW0seruak8bPTB5jMlzoaI+1dBFh18k9qplq+lT/HqiGAGzjQ9N+tKc8a21ZMv1bGADLDszuNFG4dbJFbnLBi9MbqnkwhhFoYFTIae7n6L+oNeRAGcWtq/ls2GaxrH4Z23ef+ZlAyqMA60PdcrSRLVRDy2Ck0vSXpQiEm1oQol9IAMNd7/Dh5jDSlng1h8/9/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
Date: Tue, 7 Sep 2021 12:06:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1e0e0d22-6bd0-4490-fdd3-08d971e719ae
X-MS-TrafficTypeDiagnostic: VI1PR04MB4191:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4191DA9BB2E619D1C3490294B3D39@VI1PR04MB4191.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	bJtqs6GaIsi7awptKQzx8l+javLX0++t37nkHEChriL0CaScmgI7uYhMk222VJSh5DEsisZaX8mYisp0JI7Npq1LrrUBkLMSl0hnayaz1pOh+2QCBr7dZPvOIQ0P2JhYW20Lmk378rJCl5ecQG87uBX8KSzHTCa7W4O/R1dTXZw6oJTe7Ds8bhRKx3RAVQLBbW8JoqoLdtbm6/ejPzzGumXtGTiHsMKB+zIe6DeQDJLakEaZyBBctR/bUdSvhDd5JbUjPCDEpSBFg2xuBfyFUOwlvC2C7YrrLO17spg3Wh8pAeWNZ8xzIjtKUvGL/kaCjZOxOoho4jlCJYTBGAHjwArW8rxidm9M/RaZvOzVjdflZn5tbBS5TNeIcJWvinXEOPE72PAnMwR7JYUuoAtFX52PrNGEtdFbm+LOHwk2jhpWXNUKvicMwdMYyLvoGsraXsnvSYBdKnobOYr9wzidCeg+1Gn4XQH7m1I9GuEH9Tkz7CFAQmRy2WWqeRVhDl0avkYN9iblspbDQedjEeXExsaZX8F0eUSC2PV4/ao0LYooZmNOvYvvHfw1gQHIyHEseZ/4Sy7o8jSguKoX4dUaH4ngvJ7ILLtzU43EiSArFEfe9cmU9gpqSHv2c/wPBzKMOw1owe+GddsG3Hmn15uxJJFTNRqDgwL54mDhjin6iK/uT6y/thKxTGmeo2xWWusBmbbTDytCqyF1m+NvgAi8t+xPZxjf06Di7hXyyjJjUhNuDkw8sOWx5JceoA5m2ZZ1
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(86362001)(66946007)(26005)(31696002)(54906003)(186003)(5660300002)(8676002)(31686004)(2906002)(66556008)(66476007)(8936002)(7416002)(4326008)(83380400001)(53546011)(16576012)(2616005)(508600001)(6486002)(110136005)(956004)(316002)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UWZ2dEVOa3pUcmIrM3NkaUhZcDhHMzYwbHAzbEJyUkVUak5Fb1pHQWFBOFRZ?=
 =?utf-8?B?WnMyenJTVW13aVdXWVZqeUJ4VDJ6cEo0UytxQUFzSEtXeis0SnMxbHI5UkhI?=
 =?utf-8?B?dHJ4eVZuQXBCRTdSZHR0QSsvNEN4eStxQms1czV2R05Qd0d5eXNvandRQjly?=
 =?utf-8?B?Y2ZMcnFEcW5QWE02ZzE2N1p0TUVlV1hqUCsrdmUyY0RDZVJPdzU1b1RNcXJn?=
 =?utf-8?B?cS8wQlJWYW5VaCtESm0xQjJEQThXcmNkTGlYYzM4TnBCbllDamJhZlFYR3dz?=
 =?utf-8?B?eEhJWnhZSGtrMVBBYXpLUUVmQ0pOei9WS2hpU0xHdTRPQ2ljSlo3SU1KZkNK?=
 =?utf-8?B?NkpldFZPbkxIVndnMUg3S21zWEJIRFRxVTgxLzJOdTJVUCs4QXY4MWo1WVNR?=
 =?utf-8?B?czZKai9nOUhZR3pGeGtEMW5pRWVaZ1R3NjlPZXFyNTZWZUxQNVJWYURwZFBh?=
 =?utf-8?B?QWdkWTEyTDVYQkIvR1B0RVZXMmQrRDVMVHhGS2FnMmVCSkxOUDlsOC85VVlK?=
 =?utf-8?B?cW5vMTJBY0VCQ3dmMHdSdkFrLzA5Mm0rTG5sdS90dC8wMVlSMTVoZ2tHQ2N0?=
 =?utf-8?B?aXk4cks4U2x5Y1N0R1k2MnZHQ3pVOEw0WTRpV0VVT3Z1S0s5c1k5YUNqMERI?=
 =?utf-8?B?c0k1QVZIeDRENGhpb3dFK1dZaWVtdyt2NVp5VDBkanJqYm83SWNmbVk1M0tu?=
 =?utf-8?B?NDFDNmtmemhEYU1ZeEwzNjRZaE4yWGhBR0p5ZnlESWtwUmJ3b1RiTWFkMlcx?=
 =?utf-8?B?b1NkT3VyR0srUFpYdGdBNmJmUXplbml6VTVGdk91NGV2aVJ4amJSZlBkVVQ1?=
 =?utf-8?B?N0tOUVJWekxud2dTVm9qNzljMnl4NnphSWt3YUtrL3ZPYlRybEVZM3piejBu?=
 =?utf-8?B?cjE5a3QreXZNZzJYZWlONWEwZlB3ZlppdWJzV1lzdy9zaUlIMjAxUXRuWmcx?=
 =?utf-8?B?RG5WWEpNcEhxZ3RnbzUzY1hXRmo3ZlcrUFNkVy8vQk5QeW1rQ2h3cUgrYnBa?=
 =?utf-8?B?aWlkQWtEeG5VQlVFbVllOTVpTnhsKzZaWThEMUxSV0lWeGhnT1o2a211dlVM?=
 =?utf-8?B?NEtpSTVxYmlxN3oyWjZuMWRCZ3VvQUM2Vk0rVVFpWEFoWkxpSmtnSUl5Qms5?=
 =?utf-8?B?MDc4eDFzWkVranVWVFZZMlA0cjE0UmM2bi9zM2dCYzJLSGFDemZCSXBRdDhm?=
 =?utf-8?B?R2IrV3FBWkRXWVpOSmFLNmdJbWNrdXEzMnV3a0FZVDhjVkFuWEhzYXY1cXZP?=
 =?utf-8?B?WkhXUE9TYUVsUlVJeUpXa2FmTHNQeFFOcitrLzhSNXlBWVBvcEh0QmxOdmxM?=
 =?utf-8?B?alZNeFVhTEhMUFhuSkx2bGtqcWJLMnBxWCtQY05kamN3UmprdHRYV21KVEpT?=
 =?utf-8?B?S3VibGVEOVdGem41RVY2cUZGNUhuRXZPcUtoYm5Hb2tDOG9Rek5RcjEzcE01?=
 =?utf-8?B?Z2lHZm84VTNvdlM5TDFzZ3M4eVVZT29WdUNQcHA5MWJ5TVpjUERMeTVRNlRR?=
 =?utf-8?B?QnhKYlFBSzE1TG9GOHhPQnoreEhSWERLK0lNZVRLOHZ6VnI1Z0I4amxTREFw?=
 =?utf-8?B?UE85ZzJvTVZZSmxqRTE5K0xlOEU1ZjVaTTIzV2w1Y0p1U3g1MFlaa3dTMHRO?=
 =?utf-8?B?a0w3OEZwMmJyUnlwcEx4NnhFUHNYTzBkSDJERW5vMjBMTXpsOUZRMUUwMkxj?=
 =?utf-8?B?dGN5Z1FKNFVNaEZJNER4QkdqOW9QbUs1ZnROQVBNaXpDRFhncHZsTElmR2Mw?=
 =?utf-8?Q?vg+ptef31Ywl+m6kWFVSFhdgxnaeXy1CGy1fzqI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e0e0d22-6bd0-4490-fdd3-08d971e719ae
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:06:03.3007
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DslmiGuRLtSAxNjuMzS28bs5IhNvYVFg3nNJRxmECWWXq3PnzogdxmKSe5+rv+kx7Yw9gl9AW8V8f+c6b2jz1g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4191

On 07.09.2021 11:52, Oleksandr Andrushchenko wrote:
> 
> On 07.09.21 12:19, Jan Beulich wrote:
>> On 07.09.2021 11:07, Oleksandr Andrushchenko wrote:
>>> On 07.09.21 11:49, Jan Beulich wrote:
>>>> On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>>>>> So, if we have a hidden PCI device which can be assigned to a guest and it is literally untouched
>>>>> (not enabled in Dom0) then I think there will be no such reference as "host assigned values" as
>>>>> most probably the command register will remain in its after reset state.
>>>> What meaning of "hidden" do you imply here? Devices passed to
>>>> pci_{hide,ro}_device() may not be assigned to guests ...
>>> You are completely right here.
>>>> For any other meaning of "hidden", even if the device is completely
>>>> ignored by Dom0,
>>> Dom0less is such a case when a device is assigned to the guest
>>> without Dom0 at all?
>> In this case it is entirely unclear to me what entity it is to have
>> a global view on the PCI subsystem.
>>
>>>>    certain of the properties still cannot be allowed
>>>> to be DomU-controlled.
>>> The list is not that big, could you please name a few you think cannot
>>> be controlled by a guest? I can think of PCI_COMMAND_SPECIAL(?),
>>> PCI_COMMAND_INVALIDATE(?), PCI_COMMAND_PARITY, PCI_COMMAND_WAIT,
>>> PCI_COMMAND_SERR, PCI_COMMAND_INTX_DISABLE which we may want to
>>> be aligned with the "host reference" values, e.g. we only allow those bits
>>> to be set as they are in Dom0.
>> Well, you've compile a list already, and I did say so before as well:
>> Everything except I/O and memory decoding as well as bus mastering
>> needs at least closely looking at. INTX_DISABLE, for example, is
>> something I don't think a guest should be able to directly control.
>> It may still be the case that the host permits it control, but then
>> only indirectly, allowing the host to appropriately adjust its
>> internals.
>>
>> Note that even for I/O and memory decoding as well as bus mastering
>> it may be necessary to limit guest control: In case the host wants
>> to disable any of these (perhaps transiently) despite the guest
>> wanting them enabled.
> 
> Ok, so it is now clear that we need a yet another patch to add a proper
> command register emulation. What is your preference: drop the current
> patch, implement command register emulation and add a "reset patch"
> after that or we can have the patch as is now, but I'll only reset IO/mem and bus
> master bits, e.g. read the real value, mask the wanted bits and write back?

Either order is fine with me as long as the result will be claimed to
be complete until proper emulation is in place.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:07:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:07:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180740.327523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY0u-0002Hj-OQ; Tue, 07 Sep 2021 10:07:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180740.327523; Tue, 07 Sep 2021 10:07:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY0u-0002Hc-LS; Tue, 07 Sep 2021 10:07:36 +0000
Received: by outflank-mailman (input) for mailman id 180740;
 Tue, 07 Sep 2021 10:07:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY0t-0002HU-EM
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:07:35 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 43a92863-ce51-4304-ae2a-921d17555026;
 Tue, 07 Sep 2021 10:07:34 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-9-Cy3fR7TTMiqk8fr6zznO7w-2; Tue, 07 Sep 2021 12:07:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 10:07:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:07:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0027.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.21 via Frontend Transport; Tue, 7 Sep 2021 10:07:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43a92863-ce51-4304-ae2a-921d17555026
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009253;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VGr5KaKrPB+ECiV3mI7JH4ZubnK2Jw/FcM9+PIVCR9A=;
	b=hqRjfrkrRphKiA3n4lLVBEI4pTxvwCvUrlc7+mzk+krKLb60c4bY4mdJS1xathkrVrDikq
	6NbsCq7Lpk5eewybHy+osieBqobG/PR770sAgXbNzQO4l6osvoBz7aHfQJmL3D2lCFHhOt
	Fd2mFSwusz3B/UB0O0kgy7WHo7jo+Tw=
X-MC-Unique: Cy3fR7TTMiqk8fr6zznO7w-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E7w8LcHGWrntrIYAkeiSiGk55H9JXGxZWhzHMFppR/cRrQdSZv+/igXsqnHTmP0Db4BWfQ3SykDnf9DCNSlASWI78IYAYrGBxrUsI92qSlLC7WOiPER6Ot6kzhw90XiSweJmrg/Q5/zTEg9NkLFa6ZIskZhmAB/qwoykyhZvyZSNOxmDLzfvGmrTTZejD5lDZ0BpYAh/a1F6AEt6A8zJq/4c7vitO2JvcAZAc3Gei1wZCMmRbdnT6DHFc5xuEKUR1bPBy4EK8xK9Hh7P9qhyl7aFHqdZhmoj/YOkM2xOiHThcXDRxuwrPLlrxD5OwKJ1tzyQWIVkWznrpiCei+EsAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VGr5KaKrPB+ECiV3mI7JH4ZubnK2Jw/FcM9+PIVCR9A=;
 b=fkAvpHWiBc9h1bqrhaK5I963N+DD6VmjRJlVSZiYOy6sj/8wPjtRuPllCh0S1I58WDZ+0EamhtXYH4+lgXm6+X8D6Vwk5l2f+bvvJC1eGKvv0hWEvotTrer/ve6ZdCEUkJ4bT2GKn9U47on+ZDsb4zzdSW1dC877irj87ABIrhBOJdGkEfDC2Y6aE9fnFKbDodlk+6qeAPOoB6EiM3IJlCVnwxLP80+C61HtcTkPokC7kkprSxBQ75aR+phEqm/5nIEX3WnyF2Z2cZu1znJVmCmYQx5+VbzY+tKrSrPPmV451fQaCEdRwFQkGnqvz3T+6HFszpEShKMzIeBlGBxtnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 1/9] xen/x86: prevent PVH type from getting clobbered
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <46ec0dab-8b5e-f20f-225b-3f4c57f34a61@suse.com>
Date: Tue, 7 Sep 2021 12:07:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0027.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f341c0ab-13c6-4607-4725-08d971e74d4e
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB61761A562A0E0F796CF94FC6B3D39@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mSGEp41nFGKSmsQ1+MLYGHCoH7IgpiKvEoU6D4xsANOmdp6UCOt/3I+nOU1vHoWalA6KkS+wA7wqDtHUZIRQBIehvDnx2GlF1+ZewAcb0ozqIt7ieq++7xCeyMIx5pjW55O6XF+fFgqh9bK44aP5DV1P0ntT0AjhQYYnEolvzc1xCQWdv5EmGJ3smF+SPQDZXktIL3GBCrO9Uku3m/OkGizM7S13IW8Tl531sGroXAIWHZRe9TuSm3StKoZBbxglJyHkgfeNrk9vFou7DLmhXlTSksp3e8jo8AMT0+QghHGwEZZf/c8TVBb8lmr4str+sRIYCcnrHKG/7S8TE8zfaP7Hwr45iDEmXBNTO5F+6r2YQ8GgVwnMhEc22HxlMnQV+Ec3cH8yk9Nr/y5q+q/INT4JNLNvpDy0aybhmVZCX0WRM2asjWFG/Gdda4XJf136N8sv1u14ten8vwPTe5Idor9TmvFLOBqv19L4jvEGuzYVLxDIWL32pp2gkfMu77iTTirxGQaV0Rk2EgMa1KmJQhtwEWUshxm/QoJqAdOFj6WiIIdN5amGvyOnZpWrEzPrsxbUSPRrKeihhrcgNv2la0Hi9XYvpK/7RcHrCQUR68EmHJQsVIFdyc0a6gOWCzfXi1o8oVsbP+FUOSNX1EhtFtWXWPhBgMMxhWCDifgVj/bKGAXoOxah13OmLA8enrsR23bHWeay2HdW/YJ5sxh16jGt8PYBFPRy7SnXnmFaSqE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(346002)(396003)(376002)(54906003)(66476007)(66556008)(2616005)(86362001)(8676002)(66946007)(31696002)(956004)(26005)(38100700002)(6486002)(4326008)(8936002)(186003)(31686004)(36756003)(110136005)(316002)(478600001)(16576012)(2906002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?N2dzMlRpR3RYckx2TFFiNFhLWUd2OGZraEd0dmJpUExudTV6b0ZOTThkTXNu?=
 =?utf-8?B?MVZQMzdCMUFCU3hFTnlucTd3aFhvRmkxUW12VnduWkNneWdmSCsxS3lsZXdK?=
 =?utf-8?B?Ymo0TVBDbWhLZEhPRW42cVk5YlhTRkJDK3orODlSTjhVeExmNlBVSUl1TDhU?=
 =?utf-8?B?ZEk3Y0hPYnBCNUkwK3VYcVp5MnRVVjZUYUxpOGZpNFdpUlRzYmtJaWlJdkpl?=
 =?utf-8?B?NFlwZ0pGcmd1b1dKSWd6RlhnRnhMUXVHT2FoOEgxTWZTRzlCRERCR3FVUncv?=
 =?utf-8?B?NXhWcGZxUFJQMHNJMmlsSzhDdStHcndiRFFyVnluMVdWdTFabDVvN3ViTWNC?=
 =?utf-8?B?bnZoTXI0SlMydWs0eXE4ajZOcGFnU3RDZ0NoR0hhcjZoMitoVm0zNzRVamNs?=
 =?utf-8?B?RmVxYUMzUGxwWDNxQzBMdlJOem1IS3dTWXRseWs5RmdMTEFGbnJoQnJWNUhY?=
 =?utf-8?B?RHQ5WTdmQlV2Tlp2VjBQeUhTMmlJdmY4ZXRRV3VHQ2phdDBzbTFVUzFjQmQ3?=
 =?utf-8?B?bDBIVkNKMVJoVVpCM3kvT1dhQmgzZDNpbHlNNGN3L2l0OEVxNlBOYzFUTkVL?=
 =?utf-8?B?MW1HYzNpWDZJb0RYYlBTY0FIaXRoTzNUdHNjWXBnc2ZwSXVpazgvVkxQV2pi?=
 =?utf-8?B?N1ZmQi9pK3BIWllzREZsd001cWQyLzhwTjhQOCs4cWVnRjlQa1Q3NnJ6RnpF?=
 =?utf-8?B?a0Z5bEtXdjFCejI2NlpXWWJIckpuRjVPWGdraURZczM0SHBxQmMzelpMdWRQ?=
 =?utf-8?B?ekJtSUUyOXQ2cjlTM2VlakhGeTBWQkR1bDE0alI0N1BSNnh0aDBUZjNLaExm?=
 =?utf-8?B?Y1dCa1JOd0J6OXpXVERaa1F0ZFJRTExCUG1TOUZtRFIxL3pwRnBnbnpPdXV3?=
 =?utf-8?B?SC81cVhKMSt0NXNuVVlsaTg4akd6eXJTbVN1MnVQKzZOMHhEZUljRXlSSExO?=
 =?utf-8?B?SlQrVnVZYnlFclJid2tVTTRDckhpMjVMN3BKdXI3dEN2YVROc0Nsc01YN05a?=
 =?utf-8?B?M05QR0Q4eW16aHV0SWlLQ2FtMUdpMTZFTG9IcUUycGJqVkNmMnYzRE9hNFNE?=
 =?utf-8?B?UmRXSzYrTFphTnJIOGViNEJDd3pZT0E2SGRObVlPVGc2TnlaUEFQU3gyTVA0?=
 =?utf-8?B?NkZtdkx4dUtES1RUUGdUazJGWVloUFhUWi9BTmlGRGkrSEdNVDlRUktwb2hV?=
 =?utf-8?B?aCtqQW9NSzFobEdKRUlTMUNvYkZuQk1UbUZkUVo5YlFRMHVDc1RoRkh6eU5D?=
 =?utf-8?B?bU5QTDRtLzZVV2FjVnpmenUzYk5ZTnlQbTJlWU1OUG1EaWVKUjZmTTVZOVFG?=
 =?utf-8?B?YkNDYUp6MmVoWDJ6cytiazVtRUtLSlpKRStlSStOM1ZZOW43RmFRcFdRM2ZG?=
 =?utf-8?B?emswc054N1dpNnp0dFdvNFZCcXU2cE0yTlBZTmtXMEtRcG1lbGNiWEJkeVgv?=
 =?utf-8?B?WVZJVi91eTlRV2VFMS9XdktNbVFadE1hcXF6T29vU0xqTWIwSUZob0luTGdo?=
 =?utf-8?B?ckNFTjRVZVpOeU5XMFBJVVZxenc1T096VGpTSkxBWTRCK0V3YzJkM0JZUUlE?=
 =?utf-8?B?L0dkcUlINUxkYStoYWc1Umc5cFV5Q1pUMjVBRW94bGdzbGliZFJRSCtHV1lq?=
 =?utf-8?B?czVHalpSWWRDWXdTaHFIUExrdS85dVB6UFJpanFVNG9Sd2N3Y3UvcVZ6cWNO?=
 =?utf-8?B?QVk2MWhpZjFuMEEyWndoSjNTdHBHODJEWFlIMkVZbFk4K0NTNmpGQUZ0VkJU?=
 =?utf-8?Q?HDMt7eyMsTxeoDgm20BozDq9gHtYRZ43Ot0vBJe?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f341c0ab-13c6-4607-4725-08d971e74d4e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:07:29.9054
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QvS5/u6jIhA5YQ0631sjlbwiEn+yvGE77jAgeJqeSx60mKCJMyT7CZAKFxhZymyjUu4zRhQJG1MeuVMh54jebg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

Like xen_start_flags, xen_domain_type gets set before .bss gets cleared.
Hence this variable also needs to be prevented from getting put in .bss,
which is possible because XEN_NATIVE is an enumerator evaluating to
zero. Any use prior to init_hvm_pv_info() setting the variable again
would lead to wrong decisions; one such case is xenboot_console_setup()
when called as a result of "earlyprintk=xen".

Use __ro_after_init as more applicable than either __section(".data") or
__read_mostly.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -52,9 +52,6 @@ DEFINE_PER_CPU(struct vcpu_info, xen_vcp
 DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
 EXPORT_PER_CPU_SYMBOL(xen_vcpu_id);
 
-enum xen_domain_type xen_domain_type = XEN_NATIVE;
-EXPORT_SYMBOL_GPL(xen_domain_type);
-
 unsigned long *machine_to_phys_mapping = (void *)MACH2PHYS_VIRT_START;
 EXPORT_SYMBOL(machine_to_phys_mapping);
 unsigned long  machine_to_phys_nr;
@@ -69,9 +66,11 @@ __read_mostly int xen_have_vector_callba
 EXPORT_SYMBOL_GPL(xen_have_vector_callback);
 
 /*
- * NB: needs to live in .data because it's used by xen_prepare_pvh which runs
- * before clearing the bss.
+ * NB: These need to live in .data or alike because they're used by
+ * xen_prepare_pvh() which runs before clearing the bss.
  */
+enum xen_domain_type __ro_after_init xen_domain_type = XEN_NATIVE;
+EXPORT_SYMBOL_GPL(xen_domain_type);
 uint32_t xen_start_flags __section(".data") = 0;
 EXPORT_SYMBOL(xen_start_flags);
 



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:08:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180746.327534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY1f-0002rF-2S; Tue, 07 Sep 2021 10:08:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180746.327534; Tue, 07 Sep 2021 10:08:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY1e-0002r8-VF; Tue, 07 Sep 2021 10:08:22 +0000
Received: by outflank-mailman (input) for mailman id 180746;
 Tue, 07 Sep 2021 10:08:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY1d-0002qr-IW
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:08:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 86a685aa-0fc3-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:08:20 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2052.outbound.protection.outlook.com [104.47.8.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-anK_6fk6M3OBQzdA10Izag-1; Tue, 07 Sep 2021 12:08:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4191.eurprd04.prod.outlook.com (2603:10a6:803:45::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 10:08:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:08:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0013.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.20 via Frontend Transport; Tue, 7 Sep 2021 10:08:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86a685aa-0fc3-11ec-b0e1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009299;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LnFayoVCN7CZqe8/7arRmIOlirGbN3OOPeviniwqFXQ=;
	b=grRqzC2ACmkqAqP0HE+2Hzjiur/5+ZXIf+rak4xTixfrB1FNS11FwNhIi5IWqnPVIh0VCm
	XTTdsncwES21Qt3x00Jhr+Vvths6CiCCiL4NGea6BjZ03U3iuyp9FQQiEAw/PhgFfkSs3c
	CVN9hMjKOwAfRB0hOr4H9if86dypWC8=
X-MC-Unique: anK_6fk6M3OBQzdA10Izag-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=F0+xL0CiYwpR+SZ3j9oaPfsOv9j6cOlqC79ZwX5f5ElaKTPUueQPdyNQZFWX9tpCTZNcPk5TpnmeLbR1bVrF7Slv/815OzOS460uNu6j0dxJyrp18s4Tc+0yK9nvgABdL6j6aDOMjbYrsKPdkr7j5p3pesJCVndd7NjqoGSzPVno7PoX5P1ZjrUWI7PFAF+dRBj3NsipSLE394cM8t4IZOMsLhQuz30ZoAK1vmsAF2FLICfTRiJ/RmeevrpCCIsBLKGvgAl/mleMOvDkhaOTOyyNDISYxfzjxIIbx3XhwGXR798/Gz1Tww0UDEFTemRzrsBjjxQky4UcjE+wj/tq/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LnFayoVCN7CZqe8/7arRmIOlirGbN3OOPeviniwqFXQ=;
 b=Ea9pNlvlxPmJRrBJKuUKdhKz7cr4B2d9WKCAbvEzqcdycgkAA+AgPKKkwmQ82BbdjE9VZ9aFzcjjctVvtZwDZwMicEyTK8U/HMakHCYbqTil367YGYR4B2VyJytNhX2vKGgfOQF/hoMsLpRbuKaeNfE/OsyetA1T6MKCChYJz0fHBAHl0SSmKgjhXlY3c/4wSq4CBAm9tWJOYa/0tsv3EaaNSQ+uy/u6s5Q+lUdsNMEJIIdl10LTteMWXc6RpG5x0OJsqrThTeblibhVafjHn32Ne1anS/nZkUp9USHAPNh4NmZQwv7Xb4kKRBhbmsj25o03DF3apbqA1i/E9lGglw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/9] xen/x86: allow PVH Dom0 without XEN_PV=y
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Konrad Wilk <konrad.wilk@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <2254771e-68ff-e190-15b4-42ed92239712@suse.com>
Date: Tue, 7 Sep 2021 12:08:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0013.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8c2d98cc-5199-4b44-8a24-08d971e7694b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4191:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB419188A18F250F0BDA5412B9B3D39@VI1PR04MB4191.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gL9qw5b10QprkHYC3HkJ2vDJS3edQ1xwm1WhPxPxbAWJXJHUN9sTdnOlkkmx66Bm7FT8F0tsIrs22dFAnRl+6BPV4comZ2DnIcBlwTMXiadQtdvusUigfJhD1Y1IZfIO7LdTnaf7NTDD5hZwvOK5md6G9g2YbNlEE8Uhg+5ba8kh4jb4ftLH5Z2FyQnoBbZYbkX4ESOy2Ra4I5ukWBR7oHhUNd7qO+Px1Tz39FYgdSG0vaoA/UFjz7t0ctdHOmQR/Qmoc3zLIxmfeaA8H09j4hglcTAfCjAnL19MYeFdJaKJyMYVABmrOoUEVdr7FMMe9cZsRJJQ+1ZAl1erl/J6pVsUrAjEqKynLN6rsV1HeZj0nvV7aOOrUrL2IzsaLZu7bXFy2HRePjBJIJkOPGalRwx225fnLaiUYb8bbOKHvjMNewpmJbT8/SUIZOj1qtDiSibkkm8dBGFsk5WHukDLWCXrsopKzE8bi6LBit2RhBEcbQr9DoCgPqCwWPLsXlFdZL2AS9PXO6SDIpVB3s5m/JMFfYoyqC5zU9TPZIzZFLYsnG0Vib2TL2LfF9YFMPHiAh4ozNK8PnOyfRJ1vXuqUSJMRfVY+jyecdDVqKcdbs9B/WvmMRq20eTEei2jIoKFgb5C08IT/4UvF9vw3qxBjPHl1nAHrFGflVu+anKmltyQznuYiB1SVQxXuRCTX/U3CpvQf9AWaZb1mGwvZ7diODTeKH9iZrBk04xl0ScGjxo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(86362001)(66946007)(26005)(31696002)(54906003)(186003)(5660300002)(8676002)(31686004)(2906002)(66556008)(66476007)(8936002)(4326008)(83380400001)(16576012)(2616005)(508600001)(6486002)(110136005)(956004)(316002)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NkNYbzFPUnk4ZTc1S1RNMzljMlFkZ2xVaGpwTlpiV0ZYQVYzMU9haGkrdGhB?=
 =?utf-8?B?ZU1rSWxDWXRFWEhJcm03MFhCa0ZpYVExWWFVbk9yZUJocDU1MFUvZkxWM2dh?=
 =?utf-8?B?Mm50N3lqcTRYUnVsOEtmS2x3MFRqU29xd1VZQ2l4NWhIbU5sR0g5b1BjcGhG?=
 =?utf-8?B?RTRDM2ltakNmeTYwRlh2ZzBMYm1FUkUxL2o3WDVnSnp4aFJDYnRPMGFXQVFq?=
 =?utf-8?B?ZytOeGtsUFpiRnVSc0dGVzhIcDV2UlJ4LzBTcDNDa3pnQlloanllWGZ4ci9U?=
 =?utf-8?B?SVVSYXZyenFvN1hJa1NNRzJXS3JvNHlHR2JramtwZi9pUGdObHQwVHBhakhx?=
 =?utf-8?B?a3dXa2RueUV1Z25pYXllRDJXZVphUE8yQnV3NGh1YjBGUzNPQVl4bTBuOHVv?=
 =?utf-8?B?bitWcytEWHgreEN1dVF0L1ZmWFc1cE0wNmVmZXhzRGtJYmMwc2hMeGVBVDJn?=
 =?utf-8?B?M05SZ3drQlBQandKNDUwTUtIeE52L0tJQUJvcHkvWDNtL0xvSzZiM2gwQ2Vp?=
 =?utf-8?B?ZVVOS0RKTlQwQVpZQXg4U3NLbmJTa0EwaGNEdjFiNGFPQlNDQnNsS3Y1S0dq?=
 =?utf-8?B?QkJYbGhjWFI3R0FBUCsvbjNjWUlMQVBoOExhTWVNOEtYVnk2cTdUZktXem1T?=
 =?utf-8?B?cHZwU0dCKzVoZHhLSlFubGpRemlkMXNaNjFONmhnRlFLM0ZJU01QVXdKVHV1?=
 =?utf-8?B?cyt6V2p6eElwVHAyMUY0TEpIQkNBeEdqeEljYVFsenhka1Z4VGhaSWNnanZ5?=
 =?utf-8?B?S041cFF2UWorTFVReE1lRW9hSTlKQUhHenI1cUdCSnNRZ291eHhHM3NDb2lu?=
 =?utf-8?B?bG5YOFN2Qit0NHlNczEvVDRWbHJWckY5TTRtM1pNb1BHQUJjU2hpZUEzUFFZ?=
 =?utf-8?B?VjhNRUtuNzVNTURscU0rWlZDeVNzSVNrQjF2KzE5dXdCTTQ3bytHUTJDaHRF?=
 =?utf-8?B?TjhDandTT0pjTGUwWE5DblFtQ1VEY2FJQVF1RE1LaWtZNDZxZ3IyaTZZVWJx?=
 =?utf-8?B?N2J1Z3c4dEh2LzE2azVqVmg1YVRLQVJPcEY2eHZlc0hKTyt0aksxakZRbkpJ?=
 =?utf-8?B?ZndXU0xjNkFablVHZ2x0bVBnaGlZOGZTYk5DWTZaejdUallHYnlpcElueXBi?=
 =?utf-8?B?TSthK2tpNEFJUStjZVBkN2puRHhsUzMxK29HdWV6djhhdGpMb2VyNDhTU1hu?=
 =?utf-8?B?dHhUUmtHa3Nnd2kweXJqbjFteFBYdS9rcHkxUnd6ZGcvQnZRaUloYXN6TDBn?=
 =?utf-8?B?czhqa2VxSzhxWWE0OG15TytBdmdxcjZiN3JmWHZTNy9RMXlibnBJbWNYZTZM?=
 =?utf-8?B?WnBDSzYrVXpNcEpORlQyYldPaU1JeklkM0JaSmV6VFJDSUxMRkRZam9sZnN1?=
 =?utf-8?B?eUkxUGNhTDdrU1RERjB3alhZTCtIYkp0VDVOdS9QOXF0aXlrTXNnbGE5Zjkr?=
 =?utf-8?B?U0tqUG90aTd0bU5PSGk3VlNNczl0Wmw3UEh2WEFML3Zua1VHQkV0MTlYbEdI?=
 =?utf-8?B?YWVWeXM2RGR2dXZDVnZYZ2d6b0VaR0hoQTBCV2NFUlNEVjJxNk91ZUVjaTI1?=
 =?utf-8?B?SGx2ZHFZQ21ndHZPMFEzSk96N2xSUzN0MmUxU0JZamxLQ3FNN012Y3dxSTZz?=
 =?utf-8?B?dURXUDA0bHJub1RBRHVpYkN0WEVvQ3VuT3JsOUpJSDR6VjNGdHF3RGlkNld0?=
 =?utf-8?B?dlFxMzQ5UWxLWThPRXJWTmRLOEpMajN6bVFNRjBJMWNLWWlCM2N1bmREUGxh?=
 =?utf-8?Q?DVphMp1yCy/B2xaqgdE9cJtifskiwCC/hBZCNc8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c2d98cc-5199-4b44-8a24-08d971e7694b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:08:16.8807
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hOfTzLFt4eMzjhOo5QNWWlhVrkX29HqCvYHJBxSwYsoLR3PfEc8mYNGJUCUYEw2BuRA7cOeB2dAyYoOdA0nsJQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4191

Decouple XEN_DOM0 from XEN_PV, converting some existing uses of XEN_DOM0
to a new XEN_PV_DOM0. (I'm not convinced all are really / should really
be PV-specific, but for starters I've tried to be conservative.)

For PVH Dom0 the hypervisor populates MADT with only x2APIC entries, so
without x2APIC support enabled in the kernel things aren't going to work
very well. (As opposed, DomU-s would only ever see LAPIC entries in MADT
as of now.) Note that this then requires PVH Dom0 to be 64-bit, as
X86_X2APIC depends on X86_64.

In the course of this xen_running_on_version_or_later() needs to be
available more broadly. Move it from a PV-specific to a generic file,
considering that what it does isn't really PV-specific at all anyway.

Note that xen/interface/version.h cannot be included on its own; in
enlighten.c, which uses SCHEDOP_* anyway, include xen/interface/sched.h
first to resolve the apparently sole missing type (xen_ulong_t).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
For drivers/xen/pci.c it's not clear to me what the intentions are: On
one hand I would think this is needed in PVH as well (especially for
hotplugged devices), yet otoh the hypervisor's hvm_physdev_op() doesn't
let the respective physdev-ops through. As a result with how I have
things, a lot of "Failed to add - passthrough or MSI/MSI-X might fail!"
can be observed.

For arch/x86/xen/vga.c I think it really is needed for PVH Dom0 as well,
except that the needed data doesn't get supplied yet by the hypervisor
afaict.

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -10,6 +10,8 @@
 
 #include <xen/xen.h>
 #include <xen/features.h>
+#include <xen/interface/sched.h>
+#include <xen/interface/version.h>
 #include <xen/page.h>
 
 #include <asm/xen/hypercall.h>
@@ -257,6 +259,21 @@ int xen_vcpu_setup(int cpu)
 	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
 }
 
+/* Check if running on Xen version (major, minor) or later */
+bool xen_running_on_version_or_later(unsigned int major, unsigned int minor)
+{
+	unsigned int version;
+
+	if (!xen_domain())
+		return false;
+
+	version = HYPERVISOR_xen_version(XENVER_version, NULL);
+	if ((((version >> 16) == major) && ((version & 0xffff) >= minor)) ||
+		((version >> 16) > major))
+		return true;
+	return false;
+}
+
 void xen_reboot(int reason)
 {
 	struct sched_shutdown r = { .reason = reason };
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -142,22 +142,6 @@ static void __init xen_pv_guest_late_ini
 #endif
 }
 
-/* Check if running on Xen version (major, minor) or later */
-bool
-xen_running_on_version_or_later(unsigned int major, unsigned int minor)
-{
-	unsigned int version;
-
-	if (!xen_domain())
-		return false;
-
-	version = HYPERVISOR_xen_version(XENVER_version, NULL);
-	if ((((version >> 16) == major) && ((version & 0xffff) >= minor)) ||
-		((version >> 16) > major))
-		return true;
-	return false;
-}
-
 static __read_mostly unsigned int cpuid_leaf5_ecx_val;
 static __read_mostly unsigned int cpuid_leaf5_edx_val;
 
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -14,16 +14,19 @@ static inline int pci_xen_hvm_init(void)
 	return -1;
 }
 #endif
-#if defined(CONFIG_XEN_DOM0)
+#ifdef CONFIG_XEN_PV_DOM0
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
 	return -1;
 }
+#endif
+#ifdef CONFIG_XEN_DOM0
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
 static inline int xen_find_device_domain_owner(struct pci_dev *dev)
 {
 	return -1;
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -113,7 +113,7 @@ static int acpi_register_gsi_xen_hvm(str
 				 false /* no mapping of GSI to PIRQ */);
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 static int xen_register_gsi(u32 gsi, int triggering, int polarity)
 {
 	int rc, irq;
@@ -261,7 +261,7 @@ error:
 	return irq;
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 static bool __read_mostly pci_seg_supported = true;
 
 static int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
@@ -375,10 +375,10 @@ static void xen_initdom_restore_msi_irqs
 		WARN(ret && ret != -ENOSYS, "restore_msi -> %d\n", ret);
 	}
 }
-#else /* CONFIG_XEN_DOM0 */
+#else /* CONFIG_XEN_PV_DOM0 */
 #define xen_initdom_setup_msi_irqs	NULL
 #define xen_initdom_restore_msi_irqs	NULL
-#endif /* !CONFIG_XEN_DOM0 */
+#endif /* !CONFIG_XEN_PV_DOM0 */
 
 static void xen_teardown_msi_irqs(struct pci_dev *dev)
 {
@@ -555,7 +555,7 @@ int __init pci_xen_hvm_init(void)
 	return 0;
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 int __init pci_xen_initial_domain(void)
 {
 	int irq;
@@ -583,6 +583,9 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
+#endif
+
+#ifdef CONFIG_XEN_DOM0
 
 struct xen_device_domain_owner {
 	domid_t domain;
@@ -656,4 +659,4 @@ int xen_unregister_device_domain_owner(s
 	return 0;
 }
 EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
-#endif
+#endif /* CONFIG_XEN_DOM0 */
--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -43,13 +43,9 @@ config XEN_PV_SMP
 	def_bool y
 	depends on XEN_PV && SMP
 
-config XEN_DOM0
-	bool "Xen PV Dom0 support"
-	default y
-	depends on XEN_PV && PCI_XEN && SWIOTLB_XEN
-	depends on X86_IO_APIC && ACPI && PCI
-	help
-	  Support running as a Xen PV Dom0 guest.
+config XEN_PV_DOM0
+	def_bool y
+	depends on XEN_PV && XEN_DOM0
 
 config XEN_PVHVM
 	def_bool y
@@ -86,3 +82,12 @@ config XEN_PVH
 	def_bool n
 	help
 	  Support for running as a Xen PVH guest.
+
+config XEN_DOM0
+	bool "Xen Dom0 support"
+	default XEN_PV
+	depends on (XEN_PV && SWIOTLB_XEN) || (XEN_PVH && X86_64)
+	depends on X86_IO_APIC && ACPI && PCI
+	select X86_X2APIC if XEN_PVH && X86_64
+	help
+	  Support running as a Xen Dom0 guest.
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -45,7 +45,7 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinl
 
 obj-$(CONFIG_XEN_DEBUG_FS)	+= debugfs.o
 
-obj-$(CONFIG_XEN_DOM0)		+= vga.o
+obj-$(CONFIG_XEN_PV_DOM0)	+= vga.o
 
 obj-$(CONFIG_SWIOTLB_XEN)	+= pci-swiotlb-xen.o
 
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -109,7 +109,7 @@ static inline void xen_uninit_lock_cpu(i
 
 struct dom0_vga_console_info;
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 void __init xen_init_vga(const struct dom0_vga_console_info *, size_t size);
 #else
 static inline void __init xen_init_vga(const struct dom0_vga_console_info *info,
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -240,7 +240,7 @@ config XEN_PRIVCMD
 
 config XEN_ACPI_PROCESSOR
 	tristate "Xen ACPI processor"
-	depends on XEN && XEN_DOM0 && X86 && ACPI_PROCESSOR && CPU_FREQ
+	depends on XEN && XEN_PV_DOM0 && X86 && ACPI_PROCESSOR && CPU_FREQ
 	default m
 	help
 	  This ACPI processor uploads Power Management information to the Xen
@@ -258,7 +258,7 @@ config XEN_ACPI_PROCESSOR
 
 config XEN_MCE_LOG
 	bool "Xen platform mcelog"
-	depends on XEN_DOM0 && X86_MCE
+	depends on XEN_PV_DOM0 && X86_MCE
 	help
 	  Allow kernel fetching MCE error from Xen platform and
 	  converting it into Linux mcelog format for mcelog tools



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:09:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:09:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180753.327545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY2Z-0003X9-I7; Tue, 07 Sep 2021 10:09:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180753.327545; Tue, 07 Sep 2021 10:09:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY2Z-0003X2-DA; Tue, 07 Sep 2021 10:09:19 +0000
Received: by outflank-mailman (input) for mailman id 180753;
 Tue, 07 Sep 2021 10:09:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY2Y-0003Wu-Ag
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:09:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a8d6043e-0fc3-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:09:17 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-RKsAUFFINoOolmJZdNIdKQ-1; Tue, 07 Sep 2021 12:09:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 10:09:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:09:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0071.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:15::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 10:09:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8d6043e-0fc3-11ec-b0e1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009356;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0znofO36pU2FD4CJIYFn7aacsI8GwthcODXsjcIpWgA=;
	b=P+ppK0OrA74NFQM9G9e/qTuREoS7DdeGPF3fmZjyAegAcNtIs2c/yxiJn+K+j7S7RDvN6k
	9PSwcNeUyREwBnnxmrDaVo71kI+PvYqCk7YPmcmMYrEs74KRY5U9wOHK4QZ1AguXm048G+
	LOkEl8vfKY8eOpL3QLm4ielkSpUlIZY=
X-MC-Unique: RKsAUFFINoOolmJZdNIdKQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=m+Obh2I+xU/AOuzDzVsCa3QogoiyJWgpxdsjtcqpsayydXyYGSVdx53aa5oEFNnLQWTNr5LIf1Xf9cCMzLOm7jRD0ZqDXrd5FAp0NT1WUmLyQqnb8Dq6gOgP3tfJ7ftAkk6bL63PsxdKHPLGO4raT2qGpOr1xZNsQUh/fA/lnW76GjXV1xzyPbtz6cP+FKqbjFK2qVtkYzQixTKG6EWvRq2aojvD1DOJ4UE0x1SgG1aMt+hZkt8W5hiinkL05Zsm2ID5E92FimUK6/pd30RfgZe3m8d4nIj4auhkkAeGe7e4Ox7DCKqrFWCW4nRBhgvUX3OuX0fCUAA9EK4rLlY6YQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0znofO36pU2FD4CJIYFn7aacsI8GwthcODXsjcIpWgA=;
 b=bPEHIaEv9JYVoSbY+gvx5oyBVL1BqnjFPM4qZbZclx5+Qmy7GCDTKEjJDC3kHJ5L5uZ/7CbZ/rk7Q+4Ji9TPj9FWzGPjojIZf6gImmPwTAeTmx9HjcBVfwHHG06pnOeXBFTPDlD+MyodyOXnqEaVAF3ovQP99bcmmk6kyfktKgI70vS0DOnMTBWo+ijj1Si7UAWkCeXjIQyhsIDBxWGHRoEZHEJF7TfnOChTVwH2/NrneIsLzzH6ogEQdBsQbrmeKyqdmNYciOikikmk6ZMsqn6KCi3tzl32lFWXXG3Kiwxio066IUC5Op9h6tczPI7uUZ9bHTYmtafJZdLpG0xv+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.ozlabs.org; dkim=none (message not signed)
 header.d=none;lists.ozlabs.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/9] xen/x86: make "earlyprintk=xen" work better for PVH Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <ecf17c7b-09a4-29a7-6951-1e0b0dda3c67@suse.com>
Date: Tue, 7 Sep 2021 12:09:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0071.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:15::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2155bbff-726e-4d13-9c57-08d971e78b31
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB617617206747D72C7EC87873B3D39@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DqSxuuek7MRKqM9FPJxyC3RCob2LpV3j0q4Czc9nWvy5lx1gD2hV6b97MKRao45nN6Rf0ZGl5TIP7GFezJ5VWq6m7SPYwi4bUKd2x8bCZ2cHGspooh86kKm1T+4IW0rL1hySEYgu7V8ATig+QW+LtJLpMtzqHniQvOks5UpWVp4oKc0NyFFGD/v2J8D7HHU/SakxcltURskwnlgIOFAkoHB1Ag3xOHnsfUlkRKupmBtwuWzdjjQVy5uJlkX/V//bBRi/kUDQPXhIZp3I4tCIros3z6E5lpirYVhfIZdxBHDzhMGXrQ10mGK6uUu77sdtNaXSquCijFDf/61E4c9RvSq16xukd0CqtTqwnu7vzQ6aFneL5AKympo2pruHHd8uBbT1i3GVyqlToT6JbLj/VLJ7X1XTcFOJDDrCgR3wBqbAsZdToghOo6HtslSsAUtZ4PGgubRQEhLwsvrcJpTxamf7cCraU2DR3N2aomPbiObPlzpEGVckcl/9Fk3k6zWumLbKhH8bF5n1DS924BdxCfkO1YwZ5yDODuyOlSz8lnHpr5WRFq1WYdL7k5lb2FYfy4IX2AQNirM76DIPvqIdTRRzKsDMDdiVSzhe+Eek+wYIh59PrHyGmouTdfxXWrhFynm183fRx5GM45KeL0QU9RuU35LIFUlRxl3z7f8JFmPrV3Qhk3UCHODjnwH6gMXB+U7JF3aJPFjn8a6flYB63sFsl5+vvBK5MsHhBoMRRqE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(346002)(396003)(376002)(54906003)(66476007)(66556008)(2616005)(86362001)(8676002)(66946007)(4744005)(31696002)(956004)(26005)(38100700002)(6486002)(4326008)(8936002)(186003)(31686004)(36756003)(110136005)(316002)(478600001)(16576012)(2906002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MkFDS3NmcjJhR1kzSkJGL1dKZHNTUGFnclJLajhBZHo0R0lTOUpiaUxZZHQ3?=
 =?utf-8?B?b2FISDc0V28wdEx2ZmYxd3hNUGZvYmNJZVpIeHRWcGVMRVh6SGZGUCtMWXd3?=
 =?utf-8?B?alltUmdXU2hWOWlpQmpXYWVieEdpMkV4MHlDSWJDaEV2MThHbklSaVhTaHBl?=
 =?utf-8?B?Zlc1Tzg5SVErbzN5YVNkN2ZMNnRUZ3pyT1piUmhTbjZYVkRjOUtSYzJ6MHJR?=
 =?utf-8?B?aTFuL0FyNVlsSmRFS05SU2hhZmplTVM1NWpub09WNWpnSTBZTmZIMFVkajF5?=
 =?utf-8?B?MzZlQnJlVjVSZmkxQUFtN0d0S2tLcXNwVE1ib05BenVnV0pKNFBIbGkydm5B?=
 =?utf-8?B?OHRWSmlKNWJLaW5kRVBCYlJzNkYxZXBmUjdDL2s0Snk3c1V5Q3d6RnZlRThL?=
 =?utf-8?B?MmdwQ1VXN3V6VUxvQi9aeThhblhzcGlydHFxVkwzWVdiQzVUcE5UNnZIeUVT?=
 =?utf-8?B?djhQY0Y2dEh0V2JzTHNWcmk3aUpKV2U0M3E2NGRWNG9IY3Nva2ZKZGlGMCtQ?=
 =?utf-8?B?NHc1UWl0eGNOeG1YK0FUeU5HQWpVYzFPWHk4T3hWNG94N3ozNTBFM0Z6cTVG?=
 =?utf-8?B?TjdiQSt5cENnYVFEaDFWM0ovaW9kTUxnVktPRFArVnBaSzFzeEphRXIzRkVO?=
 =?utf-8?B?S2dPbk1ySCswOXk4c0tsMXU5ZHR6Z0U5WnJlQUZTWXRUOVA3SlVacWF4UGpi?=
 =?utf-8?B?RzBRY251NW1TN0toOTdTcTl1c2xNQzVkTTRscGE3ZkdENnJ0SGd5TTE2OTlK?=
 =?utf-8?B?bmNlM2lZV2FTVElkaGQySDVVcUIwUXZ0QzVaU2dMQ0NTcElIcHNmVmRyTnox?=
 =?utf-8?B?Z3BGYThRTmgvdC8rZDMwZ1ZEMXhwRmJxeS9MSk1TZ2ppQjZqd0t5a212MGJW?=
 =?utf-8?B?dUpKRkdzTmxZd1RrSGpId2tnZkF3ODVFZCtUdytPemx3SmZ3SlIrTzB2Wmc3?=
 =?utf-8?B?ZHByM0tHaWxkczB6bkl6elcwQjRFVGRCajBwMlVla1ZlaUpETlJhbE44eFhn?=
 =?utf-8?B?VUhEZVhjS1BKR2VERGRCSDBFMzdwdmEyWjNXdTN4a0xXcklHZ2NhNU5vM0h4?=
 =?utf-8?B?MDZtWDMvbWtaYU5aRkpHVFhpeVp4bjlteWJiNzUrdHVHbFZsN2RWb0tMaU1s?=
 =?utf-8?B?d2xEekhaWXI4K2t6MTdVMW5NT2o1VWJZUUxpTkpuQU9jS0NDdjBpUWo4RWRJ?=
 =?utf-8?B?VytEbzc0TXZ3TzI2ekMxK3pQUkhPNFNsb1QwcTNIQ25FSFJuQStUUUN4LzR4?=
 =?utf-8?B?eENJdTFnNWVndW1Za3hZdVpWeVczYlVFSzNiZUVJalZ6R21KKzRCYm5qZEpL?=
 =?utf-8?B?SHZoeHZYd2FlZ1NscjRUNlgwN3ljTnVBVG5sT29pRjhiMi9neVg0SzZTMmIz?=
 =?utf-8?B?cWRLOVhYZGRqZlFNSFJFN3RRRjl5ZHRHL3hNcWdKVTlJTk9mTThQL09VZE44?=
 =?utf-8?B?cnF3ZTNtZ3RpWlFJbkJWc3JmMDBjYnNPeFdTSXNOMVArSG0wV24wMnpScHlQ?=
 =?utf-8?B?bm9KUngvT0FHZStsb0lmZTFCeGZLcHY0UlRSYVd3ZWxqS3ExWTJTbW9BMEsr?=
 =?utf-8?B?cWlYYlRCSjFCQjhld0dPdkZTQ3NHaXVESVlHSUcwMythaFRuQlZPUWZhTnRu?=
 =?utf-8?B?V09GOFVPVXNiLzdtbG1OU2N2SmE1RVBTN1h6MUg0MHBxTUpKdHVreTRkYVpX?=
 =?utf-8?B?VTdyYm1oWlZFL3JKRUlvdEhNcjNhNFNmTTdncWNsbHRiQzN3aWt6dHFPSlpR?=
 =?utf-8?Q?DjIWeRVkRORIhdVaYHAohA9/jC3HKzFldnfQ1ev?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2155bbff-726e-4d13-9c57-08d971e78b31
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:09:13.7983
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jLBaJrGAjz0mRLQNZfUZ7RbXTndV1ys3SY0nFWniw1bduL2b6Kt5dDipo/9yF0nBg4Tk6gqy+zazSQGQxlYGZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

The xen_hvm_early_write() path better wouldn't be taken in this case;
while port 0xE9 can be used, the hypercall path is quite a bit more
efficient. Put that first, as it may also work for DomU-s (see also
xen_raw_console_write()).

While there also bail from the function when the first
domU_write_console() failed - later ones aren't going to succeed.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/tty/hvc/hvc_xen.c
+++ b/drivers/tty/hvc/hvc_xen.c
@@ -632,17 +632,16 @@ static void xenboot_write_console(struct
 	unsigned int linelen, off = 0;
 	const char *pos;
 
+	if (dom0_write_console(0, string, len) >= 0)
+		return;
+
 	if (!xen_pv_domain()) {
 		xen_hvm_early_write(0, string, len);
 		return;
 	}
 
-	dom0_write_console(0, string, len);
-
-	if (xen_initial_domain())
+	if (domU_write_console(0, "(early) ", 8) < 0)
 		return;
-
-	domU_write_console(0, "(early) ", 8);
 	while (off < len && NULL != (pos = strchr(string+off, '\n'))) {
 		linelen = pos-string+off;
 		if (off + linelen > len)



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:09:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:09:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180756.327555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY2x-00043L-Px; Tue, 07 Sep 2021 10:09:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180756.327555; Tue, 07 Sep 2021 10:09:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY2x-00043E-N1; Tue, 07 Sep 2021 10:09:43 +0000
Received: by outflank-mailman (input) for mailman id 180756;
 Tue, 07 Sep 2021 10:09:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY2w-00042w-PA
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:09:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7ff50b4c-2e67-42ba-8d4c-bbc4069e9dbe;
 Tue, 07 Sep 2021 10:09:42 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-kA07ANcXO0q6nVZZ70NSWA-1; Tue, 07 Sep 2021 12:09:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 10:09:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:09:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR06CA0079.eurprd06.prod.outlook.com (2603:10a6:208:fa::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 10:09:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ff50b4c-2e67-42ba-8d4c-bbc4069e9dbe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009381;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=QT2hWROh4+zHp6bv3m6ZK+pqrsQeBWMYacVPC/7XLFs=;
	b=MEib/EqnGV9tlL2S/+/2ehd9OhOQt50QhZGXI7Gb3aB5ZEmh8ykRlbjWJdBmY5fUHKUAb1
	jyNOfDHxvxF/k8B5ljx1x0ke9ZXgpGFhWDVzBNH6Br0pMp/AgOdIM3Q/GALnPU3omm6l8I
	OqDXenz9qMkkiot5ZO+JeKkPyCeq+rQ=
X-MC-Unique: kA07ANcXO0q6nVZZ70NSWA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=l98EoNAQvlKjJyr2A5jHqWKX7cUvES+NJe7QjVdKj1aTNx+hJoO4uudM1/uSvdUXsEvl9qMi1QTBsx0VpmBtrirB8PKcJWIPfIKquEpGoZ0O7RWtbcmMOt9VL25ybdrBA25tQfTyYw5HiiTzGN2Yp9JocdzcAi1KrbXZ0n+W0sfxnMWzXA2/BT9+TBSnr5PMfHAlWt0yXgLZFpJFIu1gEyMLrOg0bjOeuIVQHktgpQmEw6cpsnqC2o6POd0EMzPsaPGIjbRIfANBCRi+S7fCK4g9w3exPC7Hde4Y4j4VSMtjpAYFrZDDClF5K/1p0LceFJqPTw9t8Jej1Nelca7bMQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QT2hWROh4+zHp6bv3m6ZK+pqrsQeBWMYacVPC/7XLFs=;
 b=Qw4APWuQuUQGwr7Uyu252oUDDJAKww7agbTDd/KdF3C4aNvz8pDmhiX88po8S7yA+1G3yV/Esjk0vnKcNirsZ2glNxKzZQF1274mUUryrvV8C8GjvmpxZEmYSpGf8o6YLfXz3nyT6pPBiF6xhGSebtmRQyHblJq2rs5CQw9fYM7dX6BRGZMZyEyMsjQAdinYNT9hoNq96k+Hm7QegZjjyzsuALRBEDN9/ksqCBU7LI/HyHOfTDwqmfBEoaKajpuk+l1FTq+F9v/eIoU3KUXndlDzWtppzV6xD8NAP+US8nT/tw3Sp5188Rr8yUe7rwVw8ZVt/aDbA1eRNdbx1pJTyw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 4/9] xen/x86: allow "earlyprintk=xen" to work for PV Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <bbf5f984-d039-97b2-cfa2-23a8a7afcbed@suse.com>
Date: Tue, 7 Sep 2021 12:09:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR06CA0079.eurprd06.prod.outlook.com
 (2603:10a6:208:fa::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bff94427-b8c0-49da-b1af-08d971e79a59
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB617638AABEDD98B1FB96A081B3D39@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+N71l0PQEY3row9ZcGXcogF87GxEqAmHLNLg4DKPLKSDngbmLEgNg1jdqwvQLDB9xIJIUR8yxpOao7TiJwpS7CmeEQOOWxjjdAo5Nl29E3KaJp5by7R8EUQDe6LwPfEvRq30GRmK7pZZCgWiBdMDHhcyjakTqS0CmtqF86sogz3TybRqeVWHtMQW7IeKRGSzCGqj0W5Ch6if+/KC0+7tgVF0lRP/fmRz1BeG8jfA8nPO+fNCfVLVPAO0euxBWutCr64Mz4d2Q9LwBeTxgJvh7m2KTZJwn5wlvGq08Cjl0XwnDiXBMzXBMRw2kWiXYVg1vpjZAgVohrCwQqnYwPIYkqVv4YE8wnJGmioI9U2PCyFBLIK71vgJF6DLH7sN1UzQ0vBHPL/3lY8rrdVkE0Lb6KLsqjT6+MLrhytRfuyxXbFoNAZZ/lCdbLtgCMDCFyncI0X/2Ziz7rjTtDd1Ne/aYKbupOMIbKQBr7lyw7M1vakbxY6jih674Bf5Iy/MzcmAjpBSATnA7Wc8WEvJ7FxreXjg0wOJsDi25YI4zzE/C21P/L9toLpp1UPY3Ixn+30HiCJhCSyLrD+Lk+iJfYKxDw1qkvMVKcptREZ2YO6Od7WoKIZk5U7uiQ2k3JnL4t3XH4ZHeyBoCZGLS+dwnfyx0Qnu08A/NYqnrG2y3ybBFIvguuLkonQ2vn/RZ/2fDzOyLWHq1iUOT7Ask6WtzDgYsWpZtLy2NwsURdSJBn9A000=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(346002)(396003)(376002)(54906003)(66476007)(66556008)(2616005)(86362001)(8676002)(66946007)(4744005)(31696002)(956004)(26005)(38100700002)(6486002)(4326008)(8936002)(186003)(31686004)(36756003)(110136005)(316002)(478600001)(16576012)(2906002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZTkvbFdLU2o5RjJLU0kwelFSOEY2R2s5YkdOMy9YRGduVDM0U2tvZHFjdmZY?=
 =?utf-8?B?ajdJOXhHUmRNaW02VzM2eDhLVys5RW1ScFJZZnZ1MXE1K2o5YWZDNERGVkN4?=
 =?utf-8?B?K1BVWVN4M3A4dVd6K21EQS91L085d1lOQ3I5YnlzdEhkUzVMT0ZhdTFWUDd5?=
 =?utf-8?B?L3NwVHNCdTV0dy8xZU82UlFpQU9IcUYrUGE4MEF4Ynlsd2hwQXV1d2ZjeDln?=
 =?utf-8?B?dVgzR2FITWhRRlZSZmV1dmsvcU81eUo3U1QzS0w5NXI2L3RXZDdOeUJiMnJO?=
 =?utf-8?B?bTdZc21UcGdTdzBtd3lKSjR1WldXR25nY3JLeWlNbVdGZFh6Q2lDQit4R1lQ?=
 =?utf-8?B?ZkZ4RThLT0dhWmd2Z2Y3bjZESTVRN3RKZmE4MmUvcC94cjlBeFFlbGkwNE5p?=
 =?utf-8?B?TWlFQUxvaUUvWmlhcTZTdmJsVk5CRzV2eU9mb1VVNmp2c0ZYRVdLNFR0R2tY?=
 =?utf-8?B?UFhLdVUxTC92NkZWcTlWOUtWUjJja1RTb1BrWWdBaHpoazFkVFM0QUkyOHpE?=
 =?utf-8?B?ckdJYVI3UTZsem45WjV2WGE5Mkx5cUNUS0xEV0dYYmpGeHR5bnpYaHBwOUFT?=
 =?utf-8?B?czhjM3VxWXc1MDVpeEZXWmlqNEczN05SVWEydGFzQWNTdm9ncU4zQnN0dDlN?=
 =?utf-8?B?MklEd1lsaGVjTXJONGhlSVpKRU1scWJoSjRmVkdFTlRRTHcwM1NkS1dnNS9q?=
 =?utf-8?B?YVlML1BJdjIxVEd3MEVCTEVoc0thRXFmQWZZeEpsSm1xb1MrYVRLT3JhTDU4?=
 =?utf-8?B?KzUyTWhoUDdXeUxZVW40V0tnTW0xUzdzeWVXcXBQc3hJVnNBL1lDVHUwS0dB?=
 =?utf-8?B?cnoxSTRZR1c5QzVDQzBnQVhQcWtMZFlVNU5ZZE5pK1RQcHVCL2Q0Y2k4SnFt?=
 =?utf-8?B?eHpDQmsyRmxlcGdyQmlaWDlmbDh5Nkg5QkdLazFPc3h1MnBncTJJZ3dUcTFR?=
 =?utf-8?B?RDhlQzhhMkdxVU9vdlRqWUVWVjJTTDZFMmlRVlV3eXBxUTlKUlFOZ2JYV0Ja?=
 =?utf-8?B?WE9MK1YrS0tLbW0xSDhHb083MFNyNU9ELytCU3h5VlVTaFo5WXJ1MnZPaDRp?=
 =?utf-8?B?R0Rmd3VKSVRWUmNueDBnREphditXL01IREZPOGhncTJiaFVWUVNRdlJxMS9K?=
 =?utf-8?B?NkgxL25QTFVzbUczTVhQNGNKMjM3Nk1TbFpWcUFveE5YaThWNkhxRUd3TFQ0?=
 =?utf-8?B?KzA5RUt1bzJOakVtN1Vha2Y3S0pBNkxYOU9sL2dIMjcxaExKeE5FZ013SFVR?=
 =?utf-8?B?TmRpOXZlUGZKTzVlalRrcnRHNUNwSlNySUVVN2tNRkxoS29IWGIyVHhvcUNG?=
 =?utf-8?B?Vy9wVCtPMDVyQUZOQU16MTM5VHQvTkMzZW1qRFdJL2NGVE9VZU1nZVk5RWtJ?=
 =?utf-8?B?bDJ1RFZpZlB0V3M2Q25aL28rN0RlTmV2RCtieGE5bjVteUlrNzdXQTlPWG1n?=
 =?utf-8?B?WjNkY25PZTAzVHhkOVpacHJ0WlI3eTEvZDRoV0ZFS1g2MWFkUFpXODZIaUc0?=
 =?utf-8?B?OWFGUmtmOXhPbkFEcnZIM3lMN2ppSG1xSHFLOUhQMm8rdUk2KzlCU3NyL253?=
 =?utf-8?B?cHhSWC9WRHZiNk1UQklSc1ZQYTdPWk5jZHVvenlzS2h3MjdLRWlSRU5nYzN1?=
 =?utf-8?B?aE1uUVcycmVEa2ROb0JRbkV2T2UrY0R1cEpnbnJUZEl0ZnZoK25vMHZzQkk0?=
 =?utf-8?B?eHZvaGROd3doVEFsQ2Y4OXpSN1VNOVo2YVUxUmZJRkRBSTMwTTIwZjV0Wk1q?=
 =?utf-8?Q?uSGCiFyzaXKMfT/rsEMLHkiw794UOvco+WL125W?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bff94427-b8c0-49da-b1af-08d971e79a59
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:09:39.1818
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VBVGDkfwwg9LGWcuPiA+qVrOJqYU4F+PwRnvfb2cdy8J77FDERQMebOkkdpivuhmHJZ4ZGSsBkNBCZqWnNl1zQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

With preferred consoles "tty" and "hvc" announced as preferred,
registering "xenboot" early won't result in use of the console: It also
needs to be registered as preferred. Generalize this from being DomU-
only so far.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1340,7 +1340,6 @@ asmlinkage __visible void __init xen_sta
 	boot_params.hdr.hardware_subarch = X86_SUBARCH_XEN;
 
 	if (!xen_initial_domain()) {
-		add_preferred_console("xenboot", 0, NULL);
 		if (pci_xen)
 			x86_init.pci.arch_init = pci_xen_init;
 	} else {
@@ -1383,6 +1382,7 @@ asmlinkage __visible void __init xen_sta
 #endif
 	}
 
+	add_preferred_console("xenboot", 0, NULL);
 	if (!boot_params.screen_info.orig_video_isVGA)
 		add_preferred_console("tty", 0, NULL);
 	add_preferred_console("hvc", 0, NULL);



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:10:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:10:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180764.327567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY3c-0005Rm-5P; Tue, 07 Sep 2021 10:10:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180764.327567; Tue, 07 Sep 2021 10:10:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY3c-0005Rf-1s; Tue, 07 Sep 2021 10:10:24 +0000
Received: by outflank-mailman (input) for mailman id 180764;
 Tue, 07 Sep 2021 10:10:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY3a-0005RP-RL
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:10:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cf551e24-0fc3-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:10:22 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2053.outbound.protection.outlook.com [104.47.12.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-cUzVwyWJPzyir_PQMp-iAQ-1; Tue, 07 Sep 2021 12:10:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 10:10:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:10:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0116.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:e6::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 10:10:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf551e24-0fc3-11ec-b0e1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009421;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=o+hISsxpZ+6AAFYh011f4JZGbH+OhXY7HGys8HSC9dI=;
	b=Soz1egwhiiEi3TWcR/5fThZh2goFlKQ0M8cNj1W/z7/3LxqoH+1dXzwK/aiiYKiL2HE5O/
	8F+2pCKAj98CKb9e7Szj8DlBxuDDJiXGQFCSR4uhijZBbEPepIXcxv5gyFcwSb3kc1e1T+
	8EpGXoqoVs4NjVkvUW98aON2yOBRf70=
X-MC-Unique: cUzVwyWJPzyir_PQMp-iAQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ATFnLEMuoUG33dBh1EVkgXLpr+UCKxnv1vD02ZkkTC5KwfeB5CIuywm/qLzMqpgZKMV3+SA6XX6R7s+8uCdfhv5MyKLawL2/8I+R6KQn1kjRXyopwXVecRe8AYBLIFzzjmTWUBNVCqPFDIEHBsP23qGpV0Jxd4Tbd7jEou+r6Hols9IEiGOinpFEA6xPnoS4qrObp8CraAW3NpDFAofTV2/Jk7CS7bnfHcVDjf02yCM5AhmGPrNEor3TmPP2DIvb/rKamOFl73ZR8cDfBKvPdIAJ3M06Wsrmq2SpPZK0V5KC4T6ymnpKAZqMb+GcWHNThgZJpmASHvOPl+qJMbN13A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=o+hISsxpZ+6AAFYh011f4JZGbH+OhXY7HGys8HSC9dI=;
 b=gIRBzsqqtWU+N/GewJ1zbE/5dd1xgfelPJp2zbkITX8nu/GdEKkFppfiTWRE6GnR/Zuvpq43fgR1Ddcx2i5PrXAj0lXqzKEkyPnNOAdR1FcMwlUkvxdeWB/F/a7p6QQFpSWwoTa5A1KigGiuO7Wd7OQctW8tGZ7xjjUvPlFOnwfrWuT6c3GY1yWdeccK/Njzy2Rg54rtBFFwz/ExlzY7k4EOGWjuJv+qfFhKSLDzo5aA/lXFK8LCdwgKZl0ShB7vNJjqIMl4OVZxNsvKxVkrDJrhe7KhLyRhod1zpoCOFEyA0hE4dd7oVwtI0fTP7/oZH8zSNOdzvkZIDvkOMZB4Pw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.ozlabs.org; dkim=none (message not signed)
 header.d=none;lists.ozlabs.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 5/9] xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <5e9ff16e-85f0-50ea-f053-37e17351a0cc@suse.com>
Date: Tue, 7 Sep 2021 12:10:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0116.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:e6::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cd14d20f-e8ea-4892-7b34-08d971e7b1ea
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB61768E9D1E00D85C93BBDCC0B3D39@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	52TajtVU4F2tne9oH5kEP8osFd5bqWVTP7F8f3qxwejC1BxWxsF76FC++gQxUqHfO3hx2QQGN2RQW/KclcmZoXyC2fqx3ye/XH5U3QRiTh/C/odpud+qdyeyLDOZzib1Q9A0Hm96Ln4ehWK5hyR7SPQ33nxqKPaTx7bpHzLUO+tO3YBIVgxiwgJ35Sm3RWnTpvFBxFfpO++e8u1/MZziSUATICuYtDZ8arFn2mVF3KjSdEUM9z+Iji9iecXhN5WJ5rk01sgGeaE4phWf2vKAOzrrlrx+6CJpWmJr64AVwbwId12uU1RoyqXWSvnuPIISFwrcY+GNTR63Q6D/a9LG1dZj3XRk13XbiDtrzy4pn6kAB+goqDJ3BuMTyKO557TevmIKG7oRrTmVZs+2ZHeOdptHX3n2xA0atw09NP+OpVqMcIYuGcvxZwbGfZUx2azxv6wu2tVZFOiRPZAMuwqaM5rCQ9mDlI356bOVDc6MSatftE4yo8+W/w2TfJdYQU3Bl14DoOhp5x9C9BySPgJ5o6/8LCB1IUSo3eD1OZhdexHUlrndUUiIEMKBdi0v0ILS0dxVI5RDPs/lKHdg9WtsYgWLXDW96Qg+KA2qkewfp1hb1oYHh3Oe0rCzgygPTmDzdWrDn3D6c5Slfuhp7byXdRXQoIk3NmXRBSAG/v3BWvAzDrpJSzloA9BRxUduZaOKvpRiMmcS+Kn3uNPeaz4FFBrZ75wBwA7g//+/UKYETvM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(346002)(396003)(376002)(54906003)(66476007)(66556008)(2616005)(86362001)(8676002)(83380400001)(66946007)(4744005)(31696002)(956004)(26005)(38100700002)(6486002)(4326008)(8936002)(186003)(31686004)(36756003)(110136005)(316002)(478600001)(16576012)(2906002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MStORUI5eDFRY0lZWTRhT21VY3ZkMDM1d0RzaHRFUk8yaGRscEp0NW5TdzZJ?=
 =?utf-8?B?U2tQNk9UWHM3MFA1ZmNRL2J6S3kzdk16Y0tnamkxUnJWMkEvSGlNSzVNZVRj?=
 =?utf-8?B?Uy9PZDdndnJ5cktCYUQwODRraWN3WnlzYmVBaHl4TEhyWEJianBGMmxpd1ZJ?=
 =?utf-8?B?UlRUNUo5TllxSmRpYVlibzhINnRKeE5PMUE3RmRRTmpVUlB6a0VRMEo3WmtW?=
 =?utf-8?B?a2RIb3M4N294UEQ4QnBDRGx0T3VzbjVTeGp2MGI5MXBrU2svWXdSdCt0Nlg3?=
 =?utf-8?B?S01TV1NpaGQ1eVMxSkJEbjZCN25TVHRHbWY1ZVVFRnJCSWFaR3BJYTdlbEtK?=
 =?utf-8?B?bTdHb2ZUNTFNVkY1cmtGRFl5dGJqQ21XRlRTTGU5bnF2Q1VCSmcyM0JCcy8v?=
 =?utf-8?B?MEV2bW1CLzJLZmtCQVZ1ejVubVBsV3RHWDNOWDBISkM1TmZSSkdyRnpDSDNu?=
 =?utf-8?B?Njh6RnprK2tGdEIxT1FJY2xYb0VTa0lCVmxUYU9GMndSbmlyTEFaM29PVFNH?=
 =?utf-8?B?VTZDb2VteFk1bDFRclZNSENROHpHVWwxVHhmSStNK0RWOElXY3V0amF6OWYw?=
 =?utf-8?B?NVdmRWlQRGpPMWZ0YVgxRkUzOW43eTJLRERsdlNqNXp4dEVWM2hwVFQyZUps?=
 =?utf-8?B?R3d6UkpQWWM4U0pJbDhHSkkyTGZBOTJjWWNhdlYvS29yTUVDaHZhS2E3Q3Rs?=
 =?utf-8?B?eFlCM2phRHVqTVhZYzlPaWVsWTMvY21QL0hHc0dVSXpHT2ZTbS9wamJXMk4z?=
 =?utf-8?B?V2VsaDN4aUFWLzBNWW5Ic1pjMlN6QWpNZ1FPamxYU1RFMEgvSUxsMElQOFdR?=
 =?utf-8?B?aDhqdVZ3OGUxSHdHOThsQkRFKzFmdmRWOXNGZUtIZVZxM3FVVlR1a01wT25r?=
 =?utf-8?B?Q2QrQnovRERYQnpxMkFlRnplZ21QdVFHSEJsNmZINVBFSVhzSVJoaVRoVVdo?=
 =?utf-8?B?aVNSRlZBdXFXc1Y1bzh0SE5jNjF2UGtLUmJYeFJNWDFNK3U3b3QrTHNudWd5?=
 =?utf-8?B?eVZrWHFvVE1uNkxzcUZCalVzUjRBWVFXZXREN1NmRXRKeGdFNDRHdjBZOXdK?=
 =?utf-8?B?ZHhOZlJzV1dSVHNoVURQYkZQUG02aC9scGJwMWE5RkZvNGkyVVU1a24zQjUv?=
 =?utf-8?B?dXE2b2pCL01JKy96RytPbHJJcmZyVDB5a2lyRDRDeDNtL1JnMm9SRjlLWmdH?=
 =?utf-8?B?eEMwWXZ0UGYvTE8yRjN1QVhDTWp1TWYrNHMzRGhTTEsrSGZZUElXNks0eXJ0?=
 =?utf-8?B?UktUN1JzTWF1T1VBRk9udWhrbnhGZHN3U2IwZFBKbUdUMjE5RXFrRkpXSUN4?=
 =?utf-8?B?Z000b04waUsraHVtdjZ0eWUySGVTc3dYc2JFMUNLOGN1OEZIYm9PSUxFWFI2?=
 =?utf-8?B?VG5wZVFIdHFkLzExeHJzc1BQT25kc1BrbCszV3dwY1JGaWp4bFVaZDcyTmhD?=
 =?utf-8?B?NkZSV1BSMmh5Y0NGR1EvWlQybittbUhKdHc2RVdqT002dUhzbG12VWhpZlda?=
 =?utf-8?B?UEtYdUhVdXkwdlhzVkI4UEJ5eEpPYU1jcDRJOXlwZUhDbFhoaXRKZVlWWTVB?=
 =?utf-8?B?dmpYT1pINTlNajdhcm5Jc3djdU50M1V0ekwrU0h2d1dtNCsvUTZDN1RaOXlv?=
 =?utf-8?B?Z3ljSzBHM0RzTjh5S1dxL1ZweVphOGVBSmkrSXNEbFdsUE5jSG1rMS9McWY0?=
 =?utf-8?B?WW10cVBzZjRqTzA3SFY4T0lHVU1ZVngweFBmMDJ4Q2puYjhEVmhCQ0czNHdQ?=
 =?utf-8?Q?1tPC7FKAW3V1NHHKwVhQsdtEBaB7iwM+9hizwPn?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cd14d20f-e8ea-4892-7b34-08d971e7b1ea
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:10:18.7083
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dBWJ29EHAWW9qhp7K5EU/e3Iy/v4j9mjWX/wsY6QoUje96WZfw6EMlvcnIltHJy6I1tkkKx0FPt4ZaVC4ylsxw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

xenboot_write_console() is dealing with these quite fine so I don't see
why xenboot_console_setup() would return -ENOENT in this case.

Adjust documentation accordingly.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1266,7 +1266,7 @@
 			The VGA and EFI output is eventually overwritten by
 			the real console.
 
-			The xen output can only be used by Xen PV guests.
+			The xen option can only be used in Xen domains.
 
 			The sclp output can only be used on s390.
 
--- a/drivers/tty/hvc/hvc_xen.c
+++ b/drivers/tty/hvc/hvc_xen.c
@@ -618,10 +618,8 @@ static int __init xenboot_console_setup(
 {
 	static struct xencons_info xenboot;
 
-	if (xen_initial_domain())
+	if (xen_initial_domain() || !xen_pv_domain())
 		return 0;
-	if (!xen_pv_domain())
-		return -ENODEV;
 
 	return xencons_info_pv_init(&xenboot, 0);
 }



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:11:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:11:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180771.327577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4I-0006Cb-KI; Tue, 07 Sep 2021 10:11:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180771.327577; Tue, 07 Sep 2021 10:11:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4I-0006CU-Gi; Tue, 07 Sep 2021 10:11:06 +0000
Received: by outflank-mailman (input) for mailman id 180771;
 Tue, 07 Sep 2021 10:11:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY4H-0006CF-4x
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:11:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23d633f6-5023-4ef5-bcfe-4b00c1f66bfe;
 Tue, 07 Sep 2021 10:11:04 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2057.outbound.protection.outlook.com [104.47.13.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-5mPymMrOOya9GGoIeBAUAg-1; Tue, 07 Sep 2021 12:11:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 10:10:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:10:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0138.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 10:10:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23d633f6-5023-4ef5-bcfe-4b00c1f66bfe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009463;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Q6KEqQas+q/MIa0vYj9SJ6GpdI6JFmUKlBBr4SWA71o=;
	b=jaDsxA+kuuK1t5rROA2Q70Us8jenURt3WdCuUsBTHSIJFb5Q8Ee66w2oVvHXuG92ipdet5
	ALPMWr3as8dL4IoA2uCAlHvAqfyfR1tCyMqJSICdv609QO0eFFNArEevHJPck4oqeEANwe
	lcsvFYsqQ5IopjqCY8KW7CKRBzh13cg=
X-MC-Unique: 5mPymMrOOya9GGoIeBAUAg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cPRdsThc5Qow2i5RZl4qOO4M71ZtFK4yWRRzW+MlM9bDjsJ5KczruIAcS5Bd9d3hF93I7wIr1xeRDmHUwNFkYQBKML7CGCSdyMUMG5M3Z902VbQy3POgqT8iFbTtVBCCR9EmvHkwkM5R+jmg+M4QrVK4I8GbS+sFrw5pYPiYvCvP83jqw9wYT8XXaB6LBW7x1+ahl+2bLBrg06/M/ne6J3zoEQTTN4b1Bk4zG/kNdIH1XqECazIcnwHoaPyvX9NLI30pIt/0HqHm8QUKuAbCljTN62T83hkSwiIXoZha2fwiTB+inLVydbK8c3M6JoD6n5ZAcPql1YqwbvoeSFa9Qw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q6KEqQas+q/MIa0vYj9SJ6GpdI6JFmUKlBBr4SWA71o=;
 b=b7ryPUFJJDQTcdvn6o7WTcjmQtJz1NSR/wjHbiGhEnpnmgWKY2mWx6gD/EM2xH9tgLQpZVlpAhgUQPP+x2N+KW9r/MBD0UyMg0YD7KBNHIG2AT6cOyiNIKz6khIzV8uPYvL6TJc9OTzEPTVLJU+F4QCB9zXK5UiNZ90EVEu04I1mmSx+egtABP3bsqKu3X6H+ZKh9yLveeuie/LCAsrbUKb5gtVNCoJPFcUzZOkbUOaWv2Alqaujr+F+sbapBZ9DqZP5KrXOEWmF8Qh2/EcZPNr2uYF35rgi+EPN9uBcnlQhitgAjCgU8zBNNhLiwVMOHq88mN9tp38DM9q23Tv8aQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 6/9] xen/x86: generalize preferred console model from PV to
 PVH Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <0b7afef6-1c46-ed74-ca83-f1e29f763f4a@suse.com>
Date: Tue, 7 Sep 2021 12:10:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0138.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 72b07e47-2f27-44bf-f601-08d971e7ca3a
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24470E5CD42110F66B031A1AB3D39@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	C8aD99HGtdAYtLQp4+lfDc7M7B/T49DmW5VZ8Cd88iSkflaJxmNmXU/3R4dXHQwD7mIp/74b3dzv79QkyjzNL5ragb4RGJgn6aiWLmfbL2iw5hYiPhZJtbs1+Jk1QtZ7bWuBDllXU+uFo7ySGbimuVGS1XpcIxv6a2BB2I/OGElmwA/6Uu+J97Mt9jIMEzYjx1c8TMSNOeS5sPIlNir7QIG+L5y6k/fUcBaKPFeZJ6Y4KRiWcyUQIIvYyUkrg780gV/Z1XpqWmfoHUCTyIvHqbM8bIofTf8Um/RvvGrxlE3PFDXhp1eUstwXhS+YageytzDfDLfXtBvuVFdJcJCt9wez/e/aMPfTE1UInnhel5JOtf/1J8aQgmvzMMI5BEkKR1r2axdLEW4iQIvlo1SeRN6+kdJfjH21CV6kpBl2jwWR4ekZLu0PSSdt1ZvZr56CyMYe/DW9p4NYWp/y38cNwGrybI5GQssATqEolCWX1Ip4HEL1aN+khtgvSxM5SJFKxlaSDXugLqzxiamVIhvaGdgYFPWtqSLHLDKXhg30CYwtf868EgYHKmAbnQv97knLyWjcC6akuYDpkRleDMLE8k6FOf4hL6CdJ6+2bCZbxUblQ0Uv9sYkgNYDMZV4o/TraJGP9/wMCMY/WQQDqFCHRMPFQXI6IpujCXPCSqqOkQEc/sZ+XrJ7/svKUbOwCqpLipxkCmjVr6r0rG0rf+sh0kGFR+vlfVmhixl1z4dpFDQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(376002)(136003)(396003)(39860400002)(26005)(31686004)(5660300002)(186003)(2616005)(2906002)(31696002)(83380400001)(54906003)(956004)(4326008)(8676002)(86362001)(316002)(38100700002)(110136005)(6486002)(16576012)(478600001)(66476007)(66556008)(8936002)(66946007)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?emgxVUg0RWtEaE8zWWJ5NlBCdFhQVGRndlBGc1g1VHh4Q0laMllBQ2gxRnJ3?=
 =?utf-8?B?ZCsxK3d5SXZhQ2htWGVHbnJyRjZDQWl2NlRKcUsrMW9vTjlneno2Y2xOV204?=
 =?utf-8?B?dnF2M3JuZGRtSWcweXBmYTFhR0V4cHc5M2xRQThrTG9oMjc1d1Z3Nm8wczdD?=
 =?utf-8?B?dXNqSXNTNEp3YXJoMWt5M1VXa3JDd05ZNEtHREpheTlMZGhVT1ZpekhHakc2?=
 =?utf-8?B?MUltQWExUEtMSWg5TTJrdVVsdGFXU0FpL1lOYk9ZVXZWWjZQaTF0bjE4Ly8r?=
 =?utf-8?B?end4Y1lia2wzOEQrUUdQYnM0SHdEbktLWkZSV05QMWc0TGJ1NnpHZEhRbll0?=
 =?utf-8?B?Y3VwYmlUcEhWc1hPSDVJQkhvQzJqZzNjckZWMjNrNjM3emRYRWFmcjMzbitq?=
 =?utf-8?B?QTF3eWxOMDFqc2k1QTh4NjV1NlgzWW9jT2tpSkZpbjZldUs1dkUrbTVuRmNq?=
 =?utf-8?B?Q2V5U3U2aDBxNUVNVFdydWtRVm9RUHc2UHIvSEVHSUY4THNhNlVBZ3hYWHZN?=
 =?utf-8?B?RUFxNks5SnF5eWw0RmtYOFMxTkluZjFXTE5COWJMZFB3MFViRTJqVHQ0dmdy?=
 =?utf-8?B?NnRLY3d5c3dmRUg5UWV3bWRZcGlSdDMwb3lReHZoWFF4Smk5L1ROUnkxT0Qz?=
 =?utf-8?B?WVpRajRtaDM2UXBZQmdHTktFcXh3TDFaUW4rWCtoK3NGR0RYYndZWFovQ2hS?=
 =?utf-8?B?bWZ0c3M1ZnZjcDE0dHR1emUvRG1pRWlVZnp4SGdQRDZRM1hzR2xBOEQ5S3Y4?=
 =?utf-8?B?ME9TeElvK1UrU002QVQ1YTVuM21OOEtDMzBoTFdacS9jQUhEbkhaSHZrN3hY?=
 =?utf-8?B?c2Y5bXBuYU55VEprWWV2Q2ZlWjlvQWp5Tm9XMDlWdFF3VjhQKy8zRzBPbnhN?=
 =?utf-8?B?ZUV5enhlWlEycGV1cy9TU0dEbTBRTE55OHQrS3NyQ2hPbjRJYlJOQzN0bW1H?=
 =?utf-8?B?VE9YL2doYVZPYjVaYlhYc1VLN1hxdGRrclZoL0RZcThKL3JtK0xMdHczcStY?=
 =?utf-8?B?R2p1TnR3VmpxeUVzaFhTcEpJYkZYTW9EMnpIUWU1Tkt4ZFlwZ2lBRStxeDRr?=
 =?utf-8?B?bWFLdUFvVzFFOW11dHdMakQ1QWZLWDRGd2loQmdCbGZCQXFuMlE0eXJJcmFF?=
 =?utf-8?B?SVlCWElQakRpWXRmMXdzdFlPV3REZlhUQzcyMFlLY2VHNC9vWGxYVi92YUQr?=
 =?utf-8?B?NnF0Q2o3QlVid01OYXFEazRpMlArQzJzb21ON1F2UlVUNUZLZWJ4b2Z6LzV0?=
 =?utf-8?B?UkZ5NjMzUlh2L25iejVDdXRDbUtvNkU0NHRYalRUVG05VERsa0NLNHQ5YVBP?=
 =?utf-8?B?WEFCTUVnNFVhVCtZN1pDbTVKSmVwaXFFaVJPcjZ6VVhaWlhBbExvczNQZ1Ns?=
 =?utf-8?B?MWw0a1VFRVZyMTBadlE1bUxzakdJdUY5aWg3WC92emFqM3g5eEw2OUc5ckJ1?=
 =?utf-8?B?SFp1eHpyVWpLTzZFcEVlZ24vdGJQa0JWQW1wRDIvOTVhMzQzNEd0MkkxMkNI?=
 =?utf-8?B?NE81UGtrWkpSVkNQSVJRTlJWeWR6VThqK2ppWFpIVUgyVmJBQW8wZnZEREFV?=
 =?utf-8?B?NVpoK2s3NTlMS25VdWc2MzNUa215bFA5QnJJYXZ1c3NrUVdMTXR2WDJRT09T?=
 =?utf-8?B?UThlYmhaSWhiNkFtZHRBYXRXbTdjcmdSd1RuNytMZE9CaE15Nk5lbjZvb1o5?=
 =?utf-8?B?YmtqOTdDRDQ1Q0tmVzFFQkl3WUFEOEo0bm5aNmFEODhQTEY3dVQrUHFCVUIz?=
 =?utf-8?Q?NXZn1M9Ok0eRAwdR85VGn2MYRU7N4ZWoznLKxK4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 72b07e47-2f27-44bf-f601-08d971e7ca3a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:10:59.5000
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qzzuSfosyxO1v6cFLml8rZJvZXedtnyd/D0ny8EzQ1y2oUw78132/aj9kLoXedE8XVjo5YwQKiV0h8dr2NoXRQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

Without announcing hvc0 as preferred it won't get used as long as tty0
gets registered earlier. This is particularly problematic with there not
being any screen output for PVH Dom0 when the screen is in graphics
mode, as the necessary information doesn't get conveyed yet from the
hypervisor.

Follow PV's model, but be conservative and do this for Dom0 only for
now.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Prior to 418492ba40b2 ("x86/virt/xen: Use guest_late_init to detect Xen
PVH guest") x86_init.oem.arch_setup was already used by PVH, so I assume
the use of this hook is acceptable here.

Seeing that change, I wonder in how far setting xen_pvh to true only in
xen_hvm_guest_late_init() can really work: This hook, as its name says,
gets called pretty late; any decision taken earlier might have been
wrong. One such wrong decision is what gets added here - preferred
consoles won't be registered when taking that path. While adding a 2nd
call there might work, aiui they would better be registered prior to
parse_early_param(), i.e. before "earlyprintk=" gets evaluated.

I also consider tying "detecting" PVH mode to the no-VGA and no-CMOS-RTC
FADT flags as problematic looking forward: There may conceivably be
"legacy free" HVM guests down the road, yet they shouldn't be mistaken
for being PVH. Most of the XEN_X86_EMU_* controlled functionality would
seem unsuitable for the same reason; presence/absence of
XENFEAT_hvm_pirqs (tied to XEN_X86_EMU_USE_PIRQ) might be sufficiently
reliable an indicator. Question there is whether the separation
introduced by Xen commit b96b50004804 ("x86: remove XENFEAT_hvm_pirqs
for PVHv2 guests") came early enough in the process of enabling PVHv2.
Plus I'm not sure a HVM guest without pass-through enabled couldn't be
run with this off (i.e. by relaxing emulation_flags_ok() and having the
tool stack not request this emulation in such cases).

I think the approach here might be equally applicable for DomU, albeit
potentially pointless (i.e. dropping the conditional might make sense
even if simply benign there): A PVH DomU ought to never come with a VGA
console. Yet even then a dummy one may still get registered and would
take precedence over hvc?

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -3,6 +3,7 @@
 #ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG
 #include <linux/memblock.h>
 #endif
+#include <linux/console.h>
 #include <linux/cpu.h>
 #include <linux/kexec.h>
 #include <linux/slab.h>
@@ -18,6 +19,7 @@
 #include <asm/xen/hypervisor.h>
 #include <asm/cpu.h>
 #include <asm/e820/api.h> 
+#include <asm/setup.h>
 
 #include "xen-ops.h"
 #include "smp.h"
@@ -274,6 +276,16 @@ bool xen_running_on_version_or_later(uns
 	return false;
 }
 
+void __init xen_add_preferred_consoles(void)
+{
+	add_preferred_console("xenboot", 0, NULL);
+	if (!boot_params.screen_info.orig_video_isVGA)
+		add_preferred_console("tty", 0, NULL);
+	add_preferred_console("hvc", 0, NULL);
+	if (boot_params.screen_info.orig_video_isVGA)
+		add_preferred_console("tty", 0, NULL);
+}
+
 void xen_reboot(int reason)
 {
 	struct sched_shutdown r = { .reason = reason };
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -28,7 +28,6 @@
 #include <linux/mm.h>
 #include <linux/page-flags.h>
 #include <linux/highmem.h>
-#include <linux/console.h>
 #include <linux/pci.h>
 #include <linux/gfp.h>
 #include <linux/edd.h>
@@ -1382,12 +1381,7 @@ asmlinkage __visible void __init xen_sta
 #endif
 	}
 
-	add_preferred_console("xenboot", 0, NULL);
-	if (!boot_params.screen_info.orig_video_isVGA)
-		add_preferred_console("tty", 0, NULL);
-	add_preferred_console("hvc", 0, NULL);
-	if (boot_params.screen_info.orig_video_isVGA)
-		add_preferred_console("tty", 0, NULL);
+	xen_add_preferred_consoles();
 
 #ifdef CONFIG_PCI
 	/* PCI BIOS service won't work from a PV guest. */
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -36,6 +36,9 @@ void __init xen_pvh_init(struct boot_par
 	pfn = __pa(hypercall_page);
 	wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
 
+	if (xen_initial_domain())
+		x86_init.oem.arch_setup = xen_add_preferred_consoles;
+
 	xen_efi_init(boot_params);
 }
 
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -118,6 +118,8 @@ static inline void __init xen_init_vga(c
 }
 #endif
 
+void xen_add_preferred_consoles(void);
+
 void __init xen_init_apic(void);
 
 #ifdef CONFIG_XEN_EFI



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:11:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:11:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180772.327589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4J-0006Tb-St; Tue, 07 Sep 2021 10:11:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180772.327589; Tue, 07 Sep 2021 10:11:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4J-0006TN-P5; Tue, 07 Sep 2021 10:11:07 +0000
Received: by outflank-mailman (input) for mailman id 180772;
 Tue, 07 Sep 2021 10:11:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNY4I-0006CT-90
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:11:06 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e952aada-0fc3-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:11:05 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1879gqdr023812; 
 Tue, 7 Sep 2021 10:11:02 GMT
Received: from eur01-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2058.outbound.protection.outlook.com [104.47.1.58])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ax413rgxj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 10:11:02 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB6981.eurprd03.prod.outlook.com (2603:10a6:20b:299::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 10:11:00 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 10:11:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e952aada-0fc3-11ec-b0e1-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VVksJoTbHstAgAfeQGTuaTIT9aTYz4kxR0iSFSePZVbsJfR2Vbq6ZedSm3jACcBA9cWtR0PtZTFnJeEk7FNMzsMdjhCEkr9LaYXiXW+vfXDXZZlagIgwVuQX2RAsY73Oz/2go6EBx7cxAVzLQ7jeCQdm4jB3wAvBDkmmgcpFveIH8uNITWpteXdevt7yow5TMW/8ZZeC40PA2pO0Skx5vUYNjH0H1j069C4x6VKzPb5fMbYOon/71UklYWNh7NIjzy5G8gqVZn98Hk1X9xQEwQuaqAd3FEHbVgOnjUtHv/4IkwXCkVLwvhVxRMzxZ0NiVHYTDsSvLkrRcTAOyRnNSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kOiM5RsD5xxn92IuFKhIqw2Sf/NDbqDaoMJ/mC9j85M=;
 b=DQUtEpCZ0CF8enRDwPyWE+B0y3al16aWwMUO/fxnuogthTpmy/lUo9uYUeJCgUuA7SJGMkTtZSwNusvU56Xhtrkyc37OBeEHW21hZekDzr5me93vKwloiK3XF5GSa45klvbamHmSdna30VHbbOvwFlrCZHcZb45hkIH7VOlXtq7g4nm77lO361wTyXzj052mW7Y0ghXqudxtOqjYekcJ0jukxudb1/R+b5peTNnmjjcg8Q8/VN9xfvzNCoY6yfSHmLbMihOF7idwYFyJep4Lh1ShuwxKWIY+4YntO3ZUFZejsDDV1MZpq0Ec30RiRH37Qk3YB51vdT4RSEH1RW2thA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kOiM5RsD5xxn92IuFKhIqw2Sf/NDbqDaoMJ/mC9j85M=;
 b=YWo8luJZzET9nxoVRZBa/tpCzKxHw3LdiLSo1Cug/asWIWi5LIij4PzgC8PfSnMmvQRaIxVGruQCye2wBFCPjWR3TJP8fQTUZcshKiqesTdCdsU3r+uQ+K1YBP9tRF3Y4iJJoyIs8rmsYQ9XW7q5UEGNXWGTOiNBigfo9wHeWMeW2aQATCAeKxFPjmbaNgYjK/6PeS9mlEsZkjmwJrhGohuZTv4x6Fh7OZXGmsIzyXsXwJXl8joX1idvl+ZKfdiUbF8TWlI1BXgX2zPBH3k97A0zPC4DNp7MbTdB+jfXAXRfYxNer+e2WQMRWOVXTwOP7nHt8THgy8WBpPIZ2Be11w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Topic: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Index: AQHXoKusySJnLVveGkulaWKrBZdXpKuXEOQAgAFPE4A=
Date: Tue, 7 Sep 2021 10:11:00 +0000
Message-ID: <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
In-Reply-To: <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 28d3a7ab-ea58-4f82-915a-08d971e7cac7
x-ms-traffictypediagnostic: AS8PR03MB6981:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB69815EC925AFD3E8BC3B1420E7D39@AS8PR03MB6981.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 aZ7Eo3jg2n99D0rzDxiYTrVrpInJIUhV0EdvcGhVR4GZ9ZVDMh7FMCLk2aWahvzugwRiNi7QhFNDs6zY31PSpsIabhcIwl2a1EeEEUAHI6J5uJ7nui/dWqEILpDZm3p6zABOp3Ji3H4Hd8gxkC/H3haVxwP4+CVc3EymFSIMqw9ECuOB8natCm4361AsMySqHmym81PPslFhI7+DvLIZ6ysbPAuWiYY5Z6s7R7Dk5i/e/U5lK2pg2o2q0P49GjtYl1m0RhVyQv4KgUShAzjAN/bcvwaPBJqn3znJ2yqgL8PDhrKU23GMqEEk3cawklj4GxhbGYJLISlmjdpwdoIrLvThi2el1DSLwMwN+Gm7uY7eiIoxsS3pVSo9vFcOtNy/BXaklsMe/TtCYa4RvoeSQbw8yGY2laRw1Hi0g6wKy+na/oqw1Uc3Y4w0BT+dVgO2v+h/ygtKkz2pI+8Z8hnwtTTW63aZZV5pqZ+4u599sKrjMYPnr37Z2GaI2NnvdUcBojZVhReF0XyqxlQ3/GqVqn383j2jYM75Npw451iEj6FoaQCVVODTSMD9SvGFfSBcpwV08IdSXCtQ5Qc04S6/XoPEuv5C6+HnV/pcC/9tVTM7fhKEOll1I5fTDmhJaWsT2aFTezFiX/OB6rJPd26dO2PvMib/vJpgd8uV73oHbGzB26FOS7IFOb6CzuYE0tpcGC2olVKoPbm489MWQE5ppyRa7NPLLDFmCXrxzs0rG3iMXP1enbRt53wX8x2/El4U
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(366004)(136003)(346002)(376002)(39860400002)(2616005)(478600001)(6512007)(186003)(6486002)(122000001)(38100700002)(66556008)(64756008)(91956017)(4326008)(71200400001)(8676002)(26005)(8936002)(110136005)(6506007)(53546011)(66446008)(2906002)(76116006)(66946007)(36756003)(66476007)(5660300002)(31696002)(316002)(31686004)(86362001)(38070700005)(83380400001)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?YkhOR0o4MW5DUmg4NHhXNjh0YUcvekdBeTVUNDNyeUY1d3hUSmJiMzVvZnVl?=
 =?utf-8?B?aEU3VUpHMDNKMzlPS1lHVUNuKzV2SUdtcEVUUFZMUlBoY1NFUDF5UXZwU3Mw?=
 =?utf-8?B?dVNxamsxZUhKaGwrMUlLMGZHUVhzOUwwMW92TGtJNlhxVkVvOGpBazFIOUJs?=
 =?utf-8?B?OGIraStCL2hneVJwTk5qQUltazdjNHE4bkRiN0tDOVFFNWsvQVRFZVlWVVRn?=
 =?utf-8?B?cHpkY1lMNDBiYU8wLzJrcEpBUnNrMkxTZ1Nja3kzd1QxL0IzZ1ltNGNkUjgy?=
 =?utf-8?B?Q0hZanhoMEpsV0w4aTcrditwSVZwMmcyUFkxR1N4S2tIcGh3L0RlcWNtQzdF?=
 =?utf-8?B?Y09obUlIdXJHUjZWaWlsdXZuYVhRT2ZTaTZxbE9sdFhVdzhoN1lNVkw4WGVO?=
 =?utf-8?B?ZlAzZUQxQWE3UXQ0bmNpbUN1cFc3djlhL3U0VDkrU0paRUNGblZnNXBadkdH?=
 =?utf-8?B?b2JSQkpBTmdZVGNkQVNoRGJwS1pqWUZ1MGpwUHQ2cVY3ZTRIMVMxUEFhRjFv?=
 =?utf-8?B?d1lDQXNzekI3Zk9PbG4rOTUxaktrMVovWGpPL3ZoUlN2a2s3UjVVK09jZ2d3?=
 =?utf-8?B?YW95Q0p5NTNPNFJmYktNbDAzWGRiQjM3bjRRaU9YWEJtV1BDWEVRSEdEMDdW?=
 =?utf-8?B?dUg0N1dCUmhqbHRtS3l1dys1Ky9EQTlKTFU5VS9VWk80MzVMN0ZCR3RxWjVL?=
 =?utf-8?B?NDk2cWovTWliR3pQeWdJaHFRTitIV0JKVVRUdlRhd3JkeTNLMTJ5aG93NzdY?=
 =?utf-8?B?dDlsWTVQbXRnVUw1RmtMM21aYXJKYm84QUZUMXZEVEVBQWpzeWhIVkF2WXhX?=
 =?utf-8?B?ZUJiZFBJU1lweWlJZHRXcWlzRXN5NE1RYm9xV2JUSWdLbnlZaEk4K0dqcm9L?=
 =?utf-8?B?MmV4K1o3MHJ3cklWa1IxMXZOdThyL2dsclAvRnhkeXpzS3A0RG1FbnhTL01K?=
 =?utf-8?B?TUNFL2hsSi9MakZKZTBDSG4vcXR1K01mUWIzQlVHZ2tkcVdpRVQxb3JXOEMv?=
 =?utf-8?B?MEtkeXRUdUhZeGVGcS8yQzBaeEtmdmF2b1lzNlkyUGZsdmoxM0hDNHZuRi8r?=
 =?utf-8?B?bGJsVXBFZDl6VmlrVWZZUjBlbkZFaTc5S1hOai9KQmxjcjI2ZWN5bVpnZHNx?=
 =?utf-8?B?SGxyKzJxRGNUbFk2N0lKTGZFNnYvZldLNkNZRXRlMlFLU0V2aVl5NzVveTcx?=
 =?utf-8?B?elgrTk93aEJ4Znl6ZCtGenZ0NTlrYjVlemVITzArNWZMenhScG5nSmtOWFRS?=
 =?utf-8?B?dDU5eXVXUDdkeVpZc0p6VUMrMXhYQzFnc1BXVEt6U3huSlpmR0JlNG9ONjI1?=
 =?utf-8?B?a0FCQWpXc2FvRHUxOGoyN0RLQTk2TjlBd0NpR0ZrSU5GSHFwUUMxMlpGbmR5?=
 =?utf-8?B?L2s2NmJEcVV3OEV5VDUyZm54cHBGQkNqSUw2SEM5a01VRGlLZDdPMjVGT0l0?=
 =?utf-8?B?UVNQWGExV0R0eEgrZDR6NngweDJFRnZJZG00MFRtV2psTXFtaXFmUGZUZWo1?=
 =?utf-8?B?RFRVQkVEZlhoblF3dnkvNjQxVW5keVEveVQ2VEc4elJ6elE5bnhnRXZPSG1M?=
 =?utf-8?B?elEreEpxUDFzTllMYitEa0d2a2l5Rm5zTytTRWtqVkM1NmVlL0NKVWt3TFk1?=
 =?utf-8?B?aENZdm13ZytNVlVZZlNvRWNBQTdrbHVFNG5aY2d1VC9oTDcxcjBWOW1hQU5i?=
 =?utf-8?B?cWl5blI2UisrczdmbURoWVNya0lXekV2SnJDNEhQTTJYdFB1YlNWdWVpejVJ?=
 =?utf-8?Q?9c5eRYTF/gsayL65DAcStwdDZ3+lY+VzyImETO0?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E45B40E42AF73741B6C7E1792C5BC149@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 28d3a7ab-ea58-4f82-915a-08d971e7cac7
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 10:11:00.1254
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: BfcU329x42HJVXoj9TyAvhcvrLDGPJ5/YYmN9vzjUeK2ez2DTk8y66W/VNszGsWPEiQW9XWZ5dyg7wC1lqnAfH5mYWr4v2I4yHxK5BCffoZegCES5/ha1rW9s9zWFx8f
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB6981
X-Proofpoint-ORIG-GUID: JnOgF-wPfdCoEbzJHmP_aiLBHCItoqRA
X-Proofpoint-GUID: JnOgF-wPfdCoEbzJHmP_aiLBHCItoqRA
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_03,2021-09-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0
 priorityscore=1501 clxscore=1015 spamscore=0 malwarescore=0 suspectscore=0
 adultscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000
 definitions=main-2109070066

DQpPbiAwNi4wOS4yMSAxNzoxMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gQEAgLTU5Myw2ICs2MjUs
MzYgQEAgc3RhdGljIGludCBpbml0X2JhcnMoc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+PiAgIH0N
Cj4+ICAgUkVHSVNURVJfVlBDSV9JTklUKGluaXRfYmFycywgVlBDSV9QUklPUklUWV9NSURETEUp
Ow0KPj4gICANCj4+ICtpbnQgdnBjaV9iYXJfYWRkX2hhbmRsZXJzKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4gK3sNCj4+ICsgICAgaW50IHJjOw0KPj4g
Kw0KPj4gKyAgICAvKiBSZW1vdmUgcHJldmlvdXNseSBhZGRlZCByZWdpc3RlcnMuICovDQo+PiAr
ICAgIHZwY2lfcmVtb3ZlX2RldmljZV9yZWdpc3RlcnMocGRldik7DQo+PiArDQo+PiArICAgIC8q
IEl0IG9ubHkgbWFrZXMgc2Vuc2UgdG8gYWRkIHJlZ2lzdGVycyBmb3IgaHdkb20gb3IgZ3Vlc3Qg
ZG9tYWluLiAqLw0KPj4gKyAgICBpZiAoIGQtPmRvbWFpbl9pZCA+PSBET01JRF9GSVJTVF9SRVNF
UlZFRCApDQo+PiArICAgICAgICByZXR1cm4gMDsNCj4+ICsNCj4+ICsgICAgaWYgKCBpc19oYXJk
d2FyZV9kb21haW4oZCkgKQ0KPj4gKyAgICAgICAgcmMgPSBhZGRfYmFyX2hhbmRsZXJzKHBkZXYs
IHRydWUpOw0KPj4gKyAgICBlbHNlDQo+PiArICAgICAgICByYyA9IGFkZF9iYXJfaGFuZGxlcnMo
cGRldiwgZmFsc2UpOw0KPiAgICAgIHJjID0gYWRkX2Jhcl9oYW5kbGVycyhwZGV2LCBpc19oYXJk
d2FyZV9kb21haW4oZCkpOw0KSW5kZWVkLCB0aGFuayB5b3UgOykNCj4NCj4+ICsgICAgaWYgKCBy
YyApDQo+PiArICAgICAgICBncHJpbnRrKFhFTkxPR19FUlIsDQo+PiArICAgICAgICAgICAgIiVw
cDogZmFpbGVkIHRvIGFkZCBCQVIgaGFuZGxlcnMgZm9yIGRvbSVkXG4iLCAmcGRldi0+c2JkZiwN
Cj4+ICsgICAgICAgICAgICBkLT5kb21haW5faWQpOw0KPiBQbGVhc2UgdXNlICVwZCBhbmQgY29y
cmVjdCBpbmRlbnRhdGlvbi4gTG9nZ2luZyB0aGUgZXJyb3IgY29kZSBtaWdodA0KPiBhbHNvIGhl
bHAgc29tZSBpbiBkaWFnbm9zaW5nIGlzc3Vlcy4NClN1cmUsIEknbGwgY2hhbmdlIGl0IHRvICVw
ZA0KPiAgIEZ1cnRoZXIgSSdtIG5vdCBzdXJlIHRoaXMgaXMgYQ0KPiBtZXNzYWdlIHdlIHdhbnQg
aW4gcmVsZWFzZSBidWlsZHMNCldoeSBub3Q/DQo+ICAgLSBwZXJoYXBzIGdkcHJpbnRrKCk/DQpJ
J2xsIGNoYW5nZSBpZiB3ZSBkZWNpZGUgc28NCj4NCj4+ICsgICAgcmV0dXJuIHJjOw0KPj4gK30N
Cj4+ICsNCj4+ICtpbnQgdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4gK3sNCj4+ICsgICAgLyogUmVtb3ZlIHBy
ZXZpb3VzbHkgYWRkZWQgcmVnaXN0ZXJzLiAqLw0KPj4gKyAgICB2cGNpX3JlbW92ZV9kZXZpY2Vf
cmVnaXN0ZXJzKHBkZXYpOw0KPj4gKyAgICByZXR1cm4gMDsNCj4+ICt9DQo+IEFsc28gLSBpbiBo
b3cgZmFyIGlzIHRoZSBnb2FsIG9mIHlvdXIgd29yayB0byBhbHNvIG1ha2UgdlBDSSB3b3JrIGZv
cg0KPiB4ODYgRG9tVS1zPyBJZiB0aGF0J3Mgbm90IGEgZ29hbA0KSXQgaXMgbm90LCB1bmZvcnR1
bmF0ZWx5LiBUaGUgZ29hbCBpcyBub3QgdG8gYnJlYWsgeDg2IGFuZCB0byBlbmFibGUgQXJtDQo+
ICwgSSdkIGxpa2UgdG8gYXNrIHRoYXQgeW91IGxpbWl0IHRoZQ0KPiBpbnRyb2R1Y3Rpb24gb2Yg
Y29kZSB0aGF0IGVuZHMgdXAgZGVhZCB0aGVyZS4NCg0KV2hhdCdzIHdyb25nIHdpdGggdGhpcyBm
dW5jdGlvbiBldmVuIGlmIGl0IGlzIGEgb25lLWxpbmVyPw0KDQpUaGlzIHdheSB3ZSBoYXZlIGEg
cGFpciB2cGNpX2Jhcl9hZGRfaGFuZGxlcnMvdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzDQoNCmFu
ZCBpZiBJIHVuZGVyc3Rvb2QgY29ycmVjdGx5IHlvdSBzdWdnZXN0IHZwY2lfYmFyX2FkZF9oYW5k
bGVycy92cGNpX3JlbW92ZV9kZXZpY2VfcmVnaXN0ZXJzPw0KDQpXaGF0IHdvdWxkIHdlIGdhaW4g
ZnJvbSB0aGF0LCBidXQgeWV0IGFub3RoZXIgc2VjcmV0IGtub3dsZWRnZSB0aGF0IGluIG9yZGVy
DQoNCnRvIHJlbW92ZSBCQVIgaGFuZGxlcnMgb25lIG5lZWRzIHRvIGNhbGwgdnBjaV9yZW1vdmVf
ZGV2aWNlX3JlZ2lzdGVycw0KDQp3aGlsZSBJIHdvdWxkIHBlcnNvbmFsbHkgZXhwZWN0IHRvIGNh
bGwgdnBjaV9iYXJfYWRkX2hhbmRsZXJzJyBjb3VudGVycGFydCwNCg0KdnBjaV9yZW1vdmVfZGV2
aWNlX3JlZ2lzdGVycyBuYW1lbHkuDQoNCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5k
cg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:11:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:11:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180791.327600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4s-0007XR-8r; Tue, 07 Sep 2021 10:11:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180791.327600; Tue, 07 Sep 2021 10:11:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY4s-0007Ws-3f; Tue, 07 Sep 2021 10:11:42 +0000
Received: by outflank-mailman (input) for mailman id 180791;
 Tue, 07 Sep 2021 10:11:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY4r-0007VE-Ff
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:11:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ec5d5787-3209-4ed2-98a3-fcdd27580d25;
 Tue, 07 Sep 2021 10:11:40 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-Y5WVL8bmO0iESB8OdgIVFw-2;
 Tue, 07 Sep 2021 12:11:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 10:11:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:11:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0236.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 10:11:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec5d5787-3209-4ed2-98a3-fcdd27580d25
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009499;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=RRrXpYXcTwwQ057nr8ZyUKJTAVb8uMCfo1R65cMzIyc=;
	b=Zqfh4NKKZ/TdOyE5seIUSid8v/GFn51H5z+NAjHdeSpT9v7b1Mnmj5ikxJi1JKrSevU4vg
	UbxVqvXNp7UP0TiDGDCU2KS6xEYxo3AD24Sagzb1C3j5ewYZ6Sb5ndGwqkPPEt7r4dAH7Y
	eHfgm4TODmS5wEVK2Eve9Rfa4VQyGVk=
X-MC-Unique: Y5WVL8bmO0iESB8OdgIVFw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NAcFQj8m378SmU1rK/4XO+ZvKIIJDUJ7hqH+UezJQm4UrmFmCcXDws8WZs+N9dh6MyQ01SE3yEwMGGGnHkMTqihik2C+pGk0RUPQpSZP5k/X4KJYMCby3PGyE+Rw8Zd5cdAv3zT3Pf+QfBY7zkmy7yREHNQA+KEuZuhe5GFS6hR31h/wrEUp6nZaVCAooLBBIw614dJ9BabmdoF7RD1WCwS++HxImjrpuL8GauEtBUPMnMpRnZW+V7mUZlbgchMii4Z6SdUvhSEsppq2GGXJU1crEpCto/TjGjoHrYgNHl74nDmRChbQhT2na9Xo/NlhmMTBdNnmdYYn5jO6h1+S7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=RRrXpYXcTwwQ057nr8ZyUKJTAVb8uMCfo1R65cMzIyc=;
 b=DIql3QeGkH+9x2BGcrQqBIPaNPny4Q3YwckzAnQxO1zzO8gYDNoXTUhdfQtuqI/v+mIfBmQKFPCqBvc3m7zq8nV/dzOTuL/BQdEsv4DcgLbAYzxgDN6IhpOFtUCuX4ASt9vHCyO9105GpSGudwLXKNCQeGcg+JoyXGsbHutbITGodHUq+0lc+4y2JLdOAh/1BqMqqW9LlfHAf+zACU/xcnUG3PU2E1D0ecXjdrTpNwNOXdy5N0nY6MbUNaldtG4XNWLdtbz8RbL0jNLvwMR48t0HHPn6Upo25qUOnSK0gp0KSQgVqPdOTOdaMdSC9VZiBkYBR1spiq6/AbmV2bmuUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
Date: Tue, 7 Sep 2021 12:11:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0236.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ca94ea6-dec4-4502-a200-08d971e7e046
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2447E6ECE13AB2AD88462014B3D39@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:635;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tD+7OUJlsPyQgQWrjEg/VzzGUbE+OtklIEnab7QZzi6SHIjoiuaw6HyBjH9KPMKF2+BepxdfONPto1kdUfqVJrO7T41UKL38HnZf1oAmiQAX/tezGmXS0Q+oEsto7oDXyf5CXDQmDLVi4sztXKPUoreWVRRL/qdtMJATDUFqyP9XAFQugU1FZ1EDIizi3ZPVIsy3Hn+ykACa55inX16DW2zdJe58vUSP79PIjavHn5gwjDr+w6G25fbTH6dtNieIzUDjRQ7gmEKdlYC7s/f0Ki69HzqgTk2LNH0gzYmqQqYmWm0p8IHLQZJEO6fayJNSZ9uSCFTHedFOaEy5IEPWjYL8P4/AgXnNyROhjH2CxebONG0L117ruWSmaohNXxbhYY/9dSfmB9ctyM3myF2z/HWLQgLgPWH4O+xSSgSc+5ZsAFqAXD1kyKwaMq3dRtupYtpFIiO9wlmerkK3IwUMlXDNZer7DExvpAZ/2mRE2IBNVA2j2hO60w6xpmmr043N/WbPXwJ2+UfbV/oTIPYYhgfFDEWdnEXrEYWu+P61JotI+jqPjRUilCTIFBIE+Cdw2ndb8mWlonKFpyrevjKcUGVq6pvzz1ZWdFnHm29MaFGCJn2jUdOYCifV7CINu9/w68sncu/CoSwKe6bkrpIcTeWLJF9sKz1zp50HJ03BFmKlud6JCTrVgLJ8uzwZJO+KrXdGD4WppE9KxG3azkVNREpP+DjUnLt4pweRPPbzcPDj/2m/JmykSud1I1XHY6ChqZGxKX7yS6LGdl78vRHMPcmp12+YtqPS8wDd9vlrlMk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(376002)(136003)(396003)(39860400002)(26005)(31686004)(5660300002)(186003)(2616005)(2906002)(31696002)(83380400001)(54906003)(956004)(4326008)(8676002)(86362001)(316002)(38100700002)(110136005)(6486002)(16576012)(478600001)(66476007)(66556008)(8936002)(66946007)(36756003)(52103002)(158003001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TVNFSkM0V0I5T1pCRGt2WGdRWit5dmRIdHlmUm80cCtQcTBlY3dZT0hWMkkx?=
 =?utf-8?B?a0xEQ3NlKzM4OWh4VVpyZnFraGhpaUN5UkdTNU5XT2h4Qi8zRm9YQjR2aDBL?=
 =?utf-8?B?ejJ3cGFxM0wzcVZjc0RpeStSY2I3TFRielM1djZzeEhlVit0R09QVnlqUXVF?=
 =?utf-8?B?eXc4TTUxWTRNSWNYSHgzYVZLeGx4ck1wOGtXTGkwb2M3VVJoRjRNT0cvTFNo?=
 =?utf-8?B?MGEyMlcwb2ZrbVRueHJHN1ZqTFZGdUlLa1QwS09sVXJqQWFkZ3dBMVlvQ1dZ?=
 =?utf-8?B?MlpDejNRakdLK3BCZlY0eEVWVzhYeUxZNzI5WkFwWUsxWnJuM29jUVdTRFB5?=
 =?utf-8?B?WDdHR2M4NmVZcnNReU8yS3VSN3ZiZmROY3FMNXRud3ZzU2N5OGtyNE1PM3NM?=
 =?utf-8?B?WnRRZ3ZhV1Y1YWw4RXZZWUZpNU9PWGRpVXJkcFp2Y2syMUJGZENmdmJJVUto?=
 =?utf-8?B?QnpCNGhvUVkvVUxGM3dWbzdLQktRUEVEZTVSb2VXV0V4bVRhK1N0VlBEcS9R?=
 =?utf-8?B?RGYzMFZoazc3eUM3VFdoMEJPeGdIU2lJRGUwYWFoWC82Qk1wQkRUeEQ4TWlP?=
 =?utf-8?B?VmdaK2lIMzFuQTYydnZPOVFnNUdmKzB6VG9XTi81Z1BkcGhybFhvMXlqR3NP?=
 =?utf-8?B?dkNOdkRkMGZLTFh6V1dsQzJUTDZiZGVvM0NvbmlLeWI5NjhtSVdBanR4WVlJ?=
 =?utf-8?B?V2RsWEJoN0FSa2liNDJSVEh5MGNkak5VQm1ieUdoVDFZUU9SNS9zZW9jKy9Q?=
 =?utf-8?B?YU5zWDZQL1FpYlNhanZTT1pKWTZZTG9zZ2dYNFFYRHBjcW85WXQ5STNHVlBZ?=
 =?utf-8?B?bzl1cnRRQ0RMTGtzWngzUkFwZE5hZ1ZPWFFsMjhlSjkzakNXT3NJa0ZWaGRD?=
 =?utf-8?B?a0Y4UkZNRmRYK2pNVXc5L3R3b2xQd0k1T1ZkWCtsejIyWTRSei85dTZWZGNJ?=
 =?utf-8?B?Q3BOWUJxbncyalNRU2JXMU9LbzFYSXZlMDJWbU03ZGlzYm1vQStmWHpVbzZj?=
 =?utf-8?B?NmdmU2lxS2dYekJWVzBlQTVIaDVaaU1pNVV0cUJNOWV0K2xhQ2VrMkZua2x6?=
 =?utf-8?B?NnFuZGZpbVRMOUdMKzlqT1dYbzdZMXNjWkxNcmEwSU5KbG12RytzS3dSMEp4?=
 =?utf-8?B?bnBKYWNOZjBFNTBFTXNPTFNwNVg2SmFWWXpiaVRKdmdvUDJDWWNSYWdGMjNw?=
 =?utf-8?B?dldNL3BteVBLcGJxaU8xL1BxZkhoMTAyWHZzbmpZM0lOZlI4Q0pxdXloK1M5?=
 =?utf-8?B?U29kMVRDb1l5REhhaXhwcStxaGN4YUIxTEJRWVJ0TDdjOEZtYi9mY3VUR082?=
 =?utf-8?B?c1ZmanROV3Q2R0lDYnZqRDIzQ2poN0Z3b2VNYm5kMEF4QlBxaG43T3RjQ1o1?=
 =?utf-8?B?N2FaakVvUVorYWY5cEdWVldsV3FmZ3Q5bmROblpNYmJmbTNyd0oyVnAyWlVt?=
 =?utf-8?B?RXlWVWRub2NnNlMzeVhoNXowb213TEQ1MGMrZithT0s2bWl5bWJzUFgxaHBp?=
 =?utf-8?B?MWdIVG91TElnMnhVV2t4VE8rNHg4TURUVnJYSG1TY3dqZFpYUHlxRGxHVWtk?=
 =?utf-8?B?MkZmRVErYzJvRWJxRDF1UittbjJNNkwvdmpPUjJjV0p2OENIT0V4MFo0NEU0?=
 =?utf-8?B?REhKdHJ1RWRHUCtSNDFZY1RDVU9HanFmQkZXM2JXQndXMFRiTXgrYk5qS1FM?=
 =?utf-8?B?bEJjV3RJMlMxZ0FtS01BYmh6b3N1cXZraldIWWp0WGMvOHlKY25walUzT3pM?=
 =?utf-8?Q?QPh6akzVdk4BshLTZQSsUwys3qi5/0E34/gCsst?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ca94ea6-dec4-4502-a200-08d971e7e046
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:11:36.4774
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2reX+14OiwDhd1vrHhRx3IWOxAa/HEC9FnvM45xTLk3eSIrMKWurl/lKntmqephyKmY9Pb+kz7cXHCp9Z5nLSw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

This was effectively lost while dropping PVHv1 code. Move the function
and arrange for it to be called the same way as done in PV mode. Clearly
this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
check that was recently removed, as that's a PV-only feature.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
 	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
 }
 
+void __init xen_banner(void)
+{
+	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
+	struct xen_extraversion extra;
+	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
+
+	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
+	pr_info("Xen version: %u.%u%s%s\n",
+		version >> 16, version & 0xffff, extra.extraversion,
+		xen_feature(XENFEAT_mmu_pt_update_preserve_ad) ? " (preserve-AD)" : "");
+}
+
 /* Check if running on Xen version (major, minor) or later */
 bool xen_running_on_version_or_later(unsigned int major, unsigned int minor)
 {
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -108,17 +108,6 @@ struct tls_descs {
  */
 static DEFINE_PER_CPU(struct tls_descs, shadow_tls_desc);
 
-static void __init xen_banner(void)
-{
-	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
-	struct xen_extraversion extra;
-	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
-
-	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
-	pr_info("Xen version: %d.%d%s (preserve-AD)\n",
-		version >> 16, version & 0xffff, extra.extraversion);
-}
-
 static void __init xen_pv_init_platform(void)
 {
 	populate_extra_pte(fix_to_virt(FIX_PARAVIRT_BOOTMAP));
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -38,6 +38,7 @@ void __init xen_pvh_init(struct boot_par
 
 	if (xen_initial_domain())
 		x86_init.oem.arch_setup = xen_add_preferred_consoles;
+	x86_init.oem.banner = xen_banner;
 
 	xen_efi_init(boot_params);
 }
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -51,6 +51,7 @@ void __init xen_remap_memory(void);
 phys_addr_t __init xen_find_free_area(phys_addr_t size);
 char * __init xen_memory_setup(void);
 void __init xen_arch_setup(void);
+void xen_banner(void);
 void xen_enable_sysenter(void);
 void xen_enable_syscall(void);
 void xen_vcpu_restore(void);



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:12:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:12:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180800.327611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY5p-0008KA-MN; Tue, 07 Sep 2021 10:12:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180800.327611; Tue, 07 Sep 2021 10:12:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY5p-0008K3-J8; Tue, 07 Sep 2021 10:12:41 +0000
Received: by outflank-mailman (input) for mailman id 180800;
 Tue, 07 Sep 2021 10:12:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY5o-0008Jt-3Z
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:12:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 20fda39a-0fc4-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:12:39 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2051.outbound.protection.outlook.com [104.47.6.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32--O8Ql131P1-FXDx8m-4q6A-1; Tue, 07 Sep 2021 12:12:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 10:12:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:12:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0016.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 10:12:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20fda39a-0fc4-11ec-b0e1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009558;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NoqsE6RzeJ63b2XMCTslouLkpUVTdP62+7GdGZP4LU0=;
	b=i3CHbROn0nt1byhiqZugM7lG3rh7L2nDA/NFS7+5A1HuvYTBClaMvaAd+eKj9yx6sMO1wn
	qWIhKx7rqORLqf+Evvgm5EGS1Q3DaxzS8xhHePMR+JfuQOrNkr5+N1zY481k2/RQThZiUb
	pJ3MtXavHgUfjvoVxJAI1Fe9/81r+HY=
X-MC-Unique: -O8Ql131P1-FXDx8m-4q6A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LnMaN/3OzsXcZ3gXFpzLWzvagDaAfbNrTLv5rFJH+hLkrJJjjTQzG4yzs65RgVJUs0Kw8ZpDQkFKsf0y1Vej/7fcgXASh0HgxKWMOh0dF0kx3MyW2C72peug4iX1OJTNoH3d11gHh899NlWXzmDJm4O2aibvvc27dnTNRYfBhI0sYArEEmT+UgOsbY6+IObrrGuTuQFLrYf55n9kr5L+/P++lSxmrxdMYcfuY99z0O/iXUpDRIQCUHZUsbPLe5Vmkim61pWAeqMRiPFJ+R9dZzXu4VJwFdp490rzp0GmwXpszSskNweay3xsXYuPsIWio28a/6MdnzwY7C2BX4JE5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NoqsE6RzeJ63b2XMCTslouLkpUVTdP62+7GdGZP4LU0=;
 b=kge3izk4Fd0bUQuG8G+Snz+9pyivcl9ZW/sw4nSkHSnmV88faIA2+UglbW2bhYyrNQIvQ77m1b39Gckw1myUAZAk68xNRT+7ge/TRVeN8WUYOsL4PrMp9wzzM2bhA1VYQV6t055HNu8YwKv5jQSdMtxUZBnZ9Zdm5mUJgzziC7WdanslfaG5JGoPNoU3WEW04T7nEQmBXChqlxAUSuvHd0CsfO3DWhlw/UdDu2ffORiLfpJKKmW4coqxiCh0NV4+IrPnihyMTYvYeiI87zcG9K9/7u2IdrY32k2k94Z9xyyfe9UgmLDa1egWNyu/x6i0pACDjz/dipv7QLZ/WEBlJg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 8/9] x86/PVH: adjust function/data placement
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <cf3fd833-0379-aba5-421d-7f2588d2d881@suse.com>
Date: Tue, 7 Sep 2021 12:12:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0016.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 68249380-d5a4-4ca9-8f0e-08d971e8026e
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB27031AB253D6EB13DD2CA58BB3D39@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fG3g44J0l6hwlac5jI3G0mfTy/DXJoU+bFuTnoRWmaEf0fUH+NouvKFRCJqleNM3Yb3gjMsn/BozIZAbIZK/3cLVsMMewJC4KlWCjIXWX6IYo7xbUgU7yj9Z7AQFEhtlG3bmJUsr0aLhLj3BK913X/AnehzXCPL50ADvGhxuJCGFj9wj2UJcaMy7ToverXo3VP/SXjRwYjAn4gwFjvo6hrKEu6e4IP3XkRahrn/hzlUSaSf2DhNK2N0hL3FnMHXTQ0WqZ3/yUQES+MY7d9+ymH/rB0fhenJMEeiUnUWKnyt7k34hhJa7W60qg1sWDeiOY40GiktubBvGPNtJjzXR+EkDzeHQFsEuxE3NQxfKkFApH2F/ji2XjnRrzn7PJJl8kAA8qSvpi9gtbfZiFNzz38tYNfBUUG4hhB+7281pXeztqvnPAxe2B7PpYqfy8rLJAgf9I2N3KsjN98u33Aw7mgnLgAnZ1nvHs0P274ITf5K+lc+EGFTMZwIlkUJTVGmueHy5tkAtxibFWUEVYSqnIy4lztJyYnW4QLmufrh1B8n3rVSXI67DBW8ucjrkGvncANLYqpfIh0jZleiD4XotAGmph46ZUGqdgwfMZhlmaaCMW1I50HPPA+p1JgOS7TpdgaMnzYy5SrgV2iU0QrKafPx/P+t95+UBIrnGb1lKJnADd5dEFoXY776y4Xk+yPODEIBtYoQ7ws9EhuuzBp2356BXRhIsdsHtQ1SmdFa2BxI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(39860400002)(366004)(376002)(136003)(4326008)(186003)(86362001)(316002)(16576012)(8936002)(31696002)(478600001)(8676002)(36756003)(2616005)(956004)(31686004)(38100700002)(2906002)(26005)(66476007)(54906003)(66556008)(6486002)(5660300002)(110136005)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MXRxamZQT3A4WnRGMkRhVDlQYVlqVjFIdjNZUmVXMlQzTFdwRHBIOU5saGJW?=
 =?utf-8?B?cHprTnNybEVhRXFyY21PVXZOd3duRmJGTDJiUFQvLzErSzFiZ2d0VUJsRUNh?=
 =?utf-8?B?eGJNaDgrTkpDdnFaMlhBRmNwcVI1aXhCbWJTZlJPNzJRVWt2REl3QkVIK1BX?=
 =?utf-8?B?SWZPYTl5RUdlWW43TTlHL3Z6Mm5BYWphVVRkMkNsNDJWRmNKZXlWNDNPZGI5?=
 =?utf-8?B?TGl6dHcwV3ErSm93NDc5cjhQYzZPTklpSWdaeDBOUWZua2FSRlVGdUFCaHla?=
 =?utf-8?B?aG4zMGxmV2VSalpWOVRzQ013VDZMd0cyVXJHRERFejJ4L1U4Lyt6eitRTUcv?=
 =?utf-8?B?Wk9HZnBnZ1ZuaUlDUm0yNGJSMFZBNDhNQUtxN3RVTHh5RWlRZWxxTXVRdXFX?=
 =?utf-8?B?bWYyektmYWhXc3JQNE00TVQ5MjFpdzJiampaeERHM1krcUtKRVl0RlphTng1?=
 =?utf-8?B?OWtHRW5CYW0ySWJNRHBHVUZIZ21DcXNUV01ZaG1HTWFma0x5NHkrNWpuemJR?=
 =?utf-8?B?ZU13S2E5SDNSWTYwckVXdkNVaFd6MjJaUkQ0anp1UEFOQ0VzbVkvcjhMSjBp?=
 =?utf-8?B?VjlmaEFLb2s2V1dyQWs4cGFmcm5HdHJpb1hRL1hOcm1BWHBKMVZ5azRmemlU?=
 =?utf-8?B?NVd1SU5MVkpnbGl6VTFSMmRCQWFMb3RBZlY0TVkrd0lRSlFxSUl3USs2bndl?=
 =?utf-8?B?SlZ2U0VqMXNaS05CUFlrbnNtOTZUSXprb1pMbmdRNzVtMmhHZk5neS9HTFI0?=
 =?utf-8?B?U1N4QlhhTk1KT0pOb1ZqVVRWb0NBODdxZUdOelA4eG5jWXVXZ0pxWGNpSTdW?=
 =?utf-8?B?RWhrY1V6UlU1angxZWZJNmFNdU9zTmpldjZnSzRVcW14N1hiV2ZWZ3phVWhm?=
 =?utf-8?B?cEpScVo0M0Z4R01xTGxSS2t2eldJTC9Pb3ZrZGk0Wm8xbFQveko3Zk1Qb25L?=
 =?utf-8?B?dEs0SXJSODdQUWRES0VpcVdGQ2ZsU0tuT2FqaTQ3T3Bhb2NpTzZPTm02Ky83?=
 =?utf-8?B?TDB2eHppT1JzZGJ2eHM1YUJmTHplK0RudGIxdnNsWVAzc3Y3c1dleFYwOCtD?=
 =?utf-8?B?RFdoNk5pQkY1c2FjanRmY2VJUzUrbmpPdWVaeXlnTjd0aDQrNXUyWnFMOFhG?=
 =?utf-8?B?anpaYk9aRVhtaE80T3NUUnEybVVzYTFzMUFwSjJ5Y3MxZ1VDK0xvSktDcjZl?=
 =?utf-8?B?RnEyU1d1KzdvUWhRekx5d1RPRW13R25UMTZiOVR4RVNxRWxNUmFHaHg5MXlJ?=
 =?utf-8?B?cXRDOUs1cXBPMy9LWFE5RjluWWs0V0NwbnpSTFBGbVRadnFBV0k2YWVQS3dS?=
 =?utf-8?B?V2FESmMxMU9FSlBCTmdGODYwbkpia0c5MHdjWkRjeFZlYjdnZHZ5bVlBYjlx?=
 =?utf-8?B?aWdKaVkvbG1SYmJheTZxeTZWNUgzdVRrOHpXSzlXZ1hXMHVPK2tzYzBxRmcr?=
 =?utf-8?B?d09uYXdqamZQRXVOZmFIL0JiZTJkanZVTzhsWkVEZlh4VVladkY2MzlybFhW?=
 =?utf-8?B?bXhRd05pSUNxeEtnRjZzVVY2Y21jZDJ3aUUrVzN5OFY5bkFSMjU1Lzh3S2h6?=
 =?utf-8?B?NS90dUEzUkZPT0lRVGhXb3pqeHpHYk8xdGdMbEo3R3RlU2gzSGtnUmZIOFZC?=
 =?utf-8?B?cjVJNURqWEcvNGRISWlNTHFLVzFvQWVEdUtsZGR2VjFDUjRyVy9IZEx0RjNQ?=
 =?utf-8?B?aUVRSEo4by9uUjJObFpwRzYzUUlycXJHSjU3czMweWM1ZWhOS2VTc1ExRGZw?=
 =?utf-8?Q?+y4oKiIhmNMgYHsIJNYXUUj34aPo4GubsUT43LK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 68249380-d5a4-4ca9-8f0e-08d971e8026e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:12:33.8016
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HYL9YPnNvYfATx1Ct6kT41TdXHkvilXZ+D11gSzDaxvwQ2EmUjX48GJ5ylrJcqA/67UTDyKYc5b+KiXV+Fy+/g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

Two of the variables can live in .init.data, allowing the open-coded
placing in .data to go away. Another "variable" is used to communicate a
size value only to very early assembly code, which hence can be both
const and live in .init.*. Additionally two functions were lacking
__init annotations.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/platform/pvh/enlighten.c
+++ b/arch/x86/platform/pvh/enlighten.c
@@ -16,15 +16,15 @@
 /*
  * PVH variables.
  *
- * pvh_bootparams and pvh_start_info need to live in the data segment since
+ * pvh_bootparams and pvh_start_info need to live in a data segment since
  * they are used after startup_{32|64}, which clear .bss, are invoked.
  */
-struct boot_params pvh_bootparams __section(".data");
-struct hvm_start_info pvh_start_info __section(".data");
+struct boot_params __initdata pvh_bootparams;
+struct hvm_start_info __initdata pvh_start_info;
 
-unsigned int pvh_start_info_sz = sizeof(pvh_start_info);
+const unsigned int __initconst pvh_start_info_sz = sizeof(pvh_start_info);
 
-static u64 pvh_get_root_pointer(void)
+static u64 __init pvh_get_root_pointer(void)
 {
 	return pvh_start_info.rsdp_paddr;
 }
@@ -107,7 +107,7 @@ void __init __weak xen_pvh_init(struct b
 	BUG();
 }
 
-static void hypervisor_specific_init(bool xen_guest)
+static void __init hypervisor_specific_init(bool xen_guest)
 {
 	if (xen_guest)
 		xen_pvh_init(&pvh_bootparams);



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:13:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:13:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180806.327622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY6N-0000UB-0N; Tue, 07 Sep 2021 10:13:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180806.327622; Tue, 07 Sep 2021 10:13:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNY6M-0000U4-TT; Tue, 07 Sep 2021 10:13:14 +0000
Received: by outflank-mailman (input) for mailman id 180806;
 Tue, 07 Sep 2021 10:13:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNY6L-0000Tt-6T
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:13:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1fa92911-547c-4530-b653-2e4bcc89a973;
 Tue, 07 Sep 2021 10:13:12 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2055.outbound.protection.outlook.com [104.47.6.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-lAzMNj_cNImWjXxDWmK0UQ-1; Tue, 07 Sep 2021 12:13:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 10:13:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:13:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0075.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 10:13:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1fa92911-547c-4530-b653-2e4bcc89a973
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631009591;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uHbz7vDnLyWZn+Vx+UcHtoqyEcsa9/5Gh3CvlM0cqI4=;
	b=KN6HwClin63jXWvTBny/tu4g59X4oqvx6zcXsbmr3lsCenbIbAJEPtapBNLZGue+NLu6du
	ygPD2OTXBxtxaG+REhtO8u0hPVDWxebunneTgUCiiX5xFLNuiFhALFBGIwc9piF+U3zlfq
	qQV70aqbPT+ruBcD9irg6DbgRC0RF/8=
X-MC-Unique: lAzMNj_cNImWjXxDWmK0UQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CO9BcPUPSzL297plkLrfWFu7JBxYVt7I/sm2DxAIRgrmZVbvi88U8qguy6371yqm9kz8qhMwxhMFFLRlhZ5HY82bQayRNOVCOBIm24CVkrvKHw4FftUdWiMOmcgRDpXOHqTYgFxfUNaYU/yIdNnVvBOCkOwEu83JoWaGYeC3Lta0648dZZsaC9Jsu4+BzQRNV2aMnemE7SNIwIxb2Ocnevp/nvLQy2KaClYIll4E7TCpQVczB7ewPfwjulesVAp66T6xc4Qr73ObL8p1dRSa7pqfu0KJVsiwHi7oL0LWJ6hLCaTisVnRvsFdv8JzX5lYTEgGhpxOAFAH579+F0qz2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uHbz7vDnLyWZn+Vx+UcHtoqyEcsa9/5Gh3CvlM0cqI4=;
 b=SV1lgHXViEVBZwaxVWq5uFMF/sQPvP/wYSZD0ULY57z+Y/56InsHTil2/lY9qNm6K86QhRH9xPTfYdOL0l61THIgqSgt1OI2bods2dHS3917F/K3IG0PJZo49riBhXU6y0Pfi66Hu0ozpLtWJYRaQjWqib8uI2OUIyhz1ipxhA5woFhokQb2P6ZLdHsqOC5/hXvsqtt0gijogZNeqcdzoHQmK3QJd1PGi0H7oNQokQDLjjY+BxkWNfe9/qU7oA0XzkfOSDZKavbrMZfqUO0k11b9xnvJ72b1q4lJyqNuxzCKUoFuVhMpApnBY0XG3Lhm4ePisABDVehASkMDl+fvQg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 9/9] xen/x86: adjust data placement
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Message-ID: <a7d9936d-2fa8-507e-5b0a-a4930eb457dd@suse.com>
Date: Tue, 7 Sep 2021 12:13:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0075.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b22900b5-8fc6-421e-fa29-08d971e81776
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2703F05765650A72EA152E63B3D39@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uCMHuxX+xBvqr/iRMqO1XJwMYXmryJxG883EB/0M+NJsvlk+2wfaxDxFYjarYaf5aSVvzXW+pAZIIL/8tNo1cHdqWvzHsw3su7eudn8OT+5ARcRu/DqaHB9F4yuBy9UBAetQgm5eLw9OLeJDTe05/2xiVLHTe1KNFmasosWjJaXPWMIiGyHAnmiPD8t+VTIaoT4uHb8rZtG4xehUe6SIVE/b0HUkJM23XtwqLIgp0qX5HRiiyD82W7v814NwKhwZVckgq4umi3P6GukOc0RxMaJBqWY+m1LASLP24cCPPEApKFnso3KAxk9ly3+H3eVj/bhyT56R+1FpxD4elSqVwhTvsatXAUULBDJnBfsnKIbExC8SQ4q+4RNQIdojr5VXnuHnGwlTYWqRDGNX1nEWOiITI/CyBfubhULG6toIKJuyvIYuhl1bHzl3qkq/SVraeeShzupa1Wt4aV7A0rkSimRCzD4WLCVHP898z4I3stH3aJsd2Z9a8u4s7YNPZXyoq1Irnc1UcFDqUFn7u2qiUeN8te5H5pTGSiQDWEAVWSxuODhevEW6fhLW2dwIQaawjfYbEL8H2yCdKP9URsVNPA5QlZVwCWC5VLG1TGwGZHJxsMChbjPHewyU2s8jRijHNOGGNayhC/117DpchPBWi2dpZS0NEkAB7PzMw1pmvUAcUyEfODjNouPq27hACyNU+UbPpCAAIC7iOKqKxD2xXmkbjt5ySfNnYTtrbdclUxE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(39860400002)(366004)(376002)(136003)(4326008)(186003)(86362001)(316002)(16576012)(8936002)(31696002)(478600001)(8676002)(36756003)(2616005)(956004)(31686004)(38100700002)(2906002)(26005)(66476007)(54906003)(66556008)(6486002)(5660300002)(110136005)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WGNlRVk1NHd6cFpWVnIrb3F3bEYxSHJlTmF1ek9DcUYvblUzNWwwZWx2NWo0?=
 =?utf-8?B?Q3hDcnV5NHA5bW1uekFzNVZqRXlET01yUGJ1WjM0NktmQmJrbkFPQUx6SXhB?=
 =?utf-8?B?d2p2a2E1ZUdLZG1hUE90d3BaT2RHcm1SM29uM3N5Qjl4UFl0TDFDdmtDZGxW?=
 =?utf-8?B?bTZ5RmU3MkordlJXL0oxeXBSbmJ0ckNHUHdGU2Rnakg2RElGY0IvUDdGZW4y?=
 =?utf-8?B?SWMxc0JBVDNrMnMra3ozbkNtMkQyVkVyQXRGTDJwSzBBeXZHaUVBK0VWdFpp?=
 =?utf-8?B?TU5GTExoaXlCazc0NWhmeiszdDgzaXlUZTlnSzJSdEg4VDF3MzluVHVQa1F3?=
 =?utf-8?B?YXk1VlJxSUprR254ODBIOXBKcjRqOTducHp3NnBnZFBySzZ6SG9DeXI2clFI?=
 =?utf-8?B?R2VieUIvVnNMblZkNDNTVEJONEFZbTBEMHpNbmdIQ0FSMjlVOE9vK1B5R0lP?=
 =?utf-8?B?QkpKRW1MWUNjd28zUmhlV2Q1TWJxQjhzWlg4VWgyYzRPMFd3NlZ3ZS9DN3Yy?=
 =?utf-8?B?Qk1aZXJKcTQyQThtdnVXaVFid2dKSzUvdmczSlFCWHV3TmUvRlhaYThXODlm?=
 =?utf-8?B?TGJ4UzdoVEQ1d3B5aHJiNHZ4NmpOV2pOa254T3F0YnpFWlhGSk52OXM4NjNT?=
 =?utf-8?B?RWFZVFVORlNrbzBjMlovaDlDZWRCVDZWTDQwK0tFclpvaXQ2d21ZeFBWVDUr?=
 =?utf-8?B?TkRUT2xuaCszVmRlOTdXVUVQb0NGL3o0V2JzTGh5dWkzOWVqTERZRjVmeS9U?=
 =?utf-8?B?aEFZZWd2WVlIM2RlWmVtYjh5NGNXMlBMWUxvYzhVWU5vTDN0UERPaEozR3Zw?=
 =?utf-8?B?NHovNVlUS3F0R3NidUFhYTFOWDhJZnJ1ZS8wclArWnMwaUFmNE9YNm91bDcw?=
 =?utf-8?B?dHBHaUVVdnZOYWNoNGR1Qkd1L0ZHcDdScno1eTFSOE1HVSt5aW1GNU5PY3FD?=
 =?utf-8?B?Nm1aQS84cnpWQzdzR3VJTlNJcXRlVUhXQXJIN0ZqQnFEdnNmR2VCNnZBSFZa?=
 =?utf-8?B?bHlCRlBzVDN1eHdHS2VaTFVUa2VMQTg0VldJcXlZcFBRVEZBc2dldHNIa0ti?=
 =?utf-8?B?QUxHN3NBN0lVNXB0RkN5aVIyOHhMSzBMcG9yWG9QV1RkMk4yZUtrcGIvMStn?=
 =?utf-8?B?VE8wZk9FSVo1RTA3SmtoREVjSStGQmduTFJiUHdmMHE4MU1hVlVyLzZuV0Nn?=
 =?utf-8?B?NFJxVDBHeEtHNW1FNi9XTkcwd0JMMFNlTUpzUDdaUDlKSUpEWlg2Q1l0VTB1?=
 =?utf-8?B?bCtGTUM5cXZCTm9yMnQyTS9vNm1BMElqdE5jZ3RLSVRJUTN0OGg5VTVHWjd2?=
 =?utf-8?B?ODAzQ21OeFR2ZmdSL1ZRZE93WW1EbWMrNGZwUEk2SDJwejJtbjhrWWVnVlM2?=
 =?utf-8?B?YXNHNmhjTml3TXBibTlUK1AxSm9zNWhGamduK0V5WUZiakw5UDhxaHBBakcr?=
 =?utf-8?B?T3Nsd2s1b215SkY0NjFTOTRhRkwwYzg3b2tzQjM2RkdHUEZkMlJSTFBnWlNm?=
 =?utf-8?B?dlNJUzI1TGV1VEl3eDFxWXZnZlJvelZUVTI2OGM2V0s1VkxmU1EyZzVhYzk1?=
 =?utf-8?B?TjV5V0RqSEl6cjFGOGxnMXdRNllBRHJISFoyaHZXSWZJQk5oeDZvVmdidnJR?=
 =?utf-8?B?OE84cmxyNE8xVm0wWkkzVGM1cktvUzVDWVNiVlRnajhqcXBVQ0Z3R0NuNXEz?=
 =?utf-8?B?Wm1VWlF4U0ZOOFptVVhIMEQrNVR3bEZ5SVhaZHFEemZvblZWTDdZckdHcVhn?=
 =?utf-8?Q?cwyfKC2l3iJ7Px6KrYVYCJyr9ZtMFFFAVHYvlhg?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b22900b5-8fc6-421e-fa29-08d971e81776
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:13:09.0759
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +Uj30H/dxCjL3C4AsS8QjgM6Vtm1xcoiuIO9jw3ooI7NZY1d7mgUvG595wG/XmO773j5EXeiB/LXDbGTSMW65A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

Both xen_pvh and xen_start_flags get written just once aeryl during
init. Using the respective annotation then allows the open-coded placing
in .data to go away.

Additionally the former, like the latter, wants exporting, or else
xen_pvh_domain() can't be used from modules.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I have to admit that it is completely unclear to me which form of
exporting I should have used: xen_domain_type is GPL-only while
xen_start_flags is not, yet both are used in similar ways, extending to
xen_pvh.

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -75,7 +75,7 @@ EXPORT_SYMBOL_GPL(xen_have_vector_callba
  */
 enum xen_domain_type __ro_after_init xen_domain_type = XEN_NATIVE;
 EXPORT_SYMBOL_GPL(xen_domain_type);
-uint32_t xen_start_flags __section(".data") = 0;
+uint32_t __ro_after_init xen_start_flags;
 EXPORT_SYMBOL(xen_start_flags);
 
 /*
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/acpi.h>
+#include <linux/export.h>
 
 #include <xen/hvc-console.h>
 
@@ -18,10 +19,11 @@
 /*
  * PVH variables.
  *
- * The variable xen_pvh needs to live in the data segment since it is used
+ * The variable xen_pvh needs to live in a data segment since it is used
  * after startup_{32|64} is invoked, which will clear the .bss segment.
  */
-bool xen_pvh __section(".data") = 0;
+bool __ro_after_init xen_pvh;
+EXPORT_SYMBOL(xen_pvh);
 
 void __init xen_pvh_init(struct boot_params *boot_params)
 {



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 10:43:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 10:43:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180815.327633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYZj-0004Wb-Dv; Tue, 07 Sep 2021 10:43:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180815.327633; Tue, 07 Sep 2021 10:43:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYZj-0004WU-Af; Tue, 07 Sep 2021 10:43:35 +0000
Received: by outflank-mailman (input) for mailman id 180815;
 Tue, 07 Sep 2021 10:43:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNYZi-0004WO-SB
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 10:43:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7173c848-0fc8-11ec-b0e1-12813bfff9fa;
 Tue, 07 Sep 2021 10:43:33 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-sOfWlC9eM2-V1hDdeaL0tQ-1; Tue, 07 Sep 2021 12:43:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2448.eurprd04.prod.outlook.com (2603:10a6:800:52::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 10:43:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 10:43:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0221.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 10:43:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7173c848-0fc8-11ec-b0e1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631011412;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hAQn0uoWIlnVXTNyX1uoYabT8MPDY2xA0mxqdTElNQ0=;
	b=nf/EzMuQaZe/EuZ4y8PGSEtmg4UGNNXXm9jWVB1x+r9tY1HAkIc0R3TNFPMALiGEeK2dmN
	RLCs+A78JH1lGq0Mjb9ruxuHhLI7bCroTfdFNlh5cZ6QZmrnzXGpgFnE9LpUO77gIiYimw
	Q4XUpPWzyF0FZaBilcDX3yvExVwcHTA=
X-MC-Unique: sOfWlC9eM2-V1hDdeaL0tQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ebS6Ub0ovREellpTEBZnOSodFvX989VljNe8KiBejqk6xLcuJIvQhbaZ7byATllYVCGsVbVpp8GQcSEptt+CbxmzlxWlkboikksCemZC0t19D/UlGXoX23QWJUa7cXPfvDAVjdleLk7zCKoSNUXU4MJGvEb8NMpj37dEPQsv6uwb/maHREMKI/23BpATpwtVvj+3Is3/29SIwkz21e4rcHrtZx7j2jrR+qXOPkBhSyuCj8vJ5TCzaeKjv54OZHVyYRU4dXtFUD8MNdY9DQfjtRQNLSOtXBUC7wis57tegU4pKBI8EYYZTuHC7sbkt2jUkdQ2aLTiuNaKvShnXpi2Rw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hAQn0uoWIlnVXTNyX1uoYabT8MPDY2xA0mxqdTElNQ0=;
 b=LPwy3NKCnXoPJVoqLhawWWbs2OKaqWB9NjdrU9SznR60GbBaiYS9Qg7MBkuCN5Sn8hacZ8bvswg0EnXaXYRzBBYufjGRtj2QwBCmYDJfDsRK5xbHXEjqvOYJiD+IjGeM3prKfDlEY2n0kH9Ujue0sk59IZSe3/u23/eKzv7ttgwUG2eVuKdTTWkeHYEEjKlrZJVgmyz7LmdQQNv5E4MrFoo765Jqz4yZGwpNZrVH/Wh9kC8umaREyi5YDEa29Q41feWFhfMFdSeX9FzKaiHQdnwQllOEA5EC5zFhVDkcXivXkWxiKqVQWc+cb8n0Gn+JO7ZN3RRUOTXClHgby3NwKQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
Date: Tue, 7 Sep 2021 12:43:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0221.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e4ad2d90-1138-419b-24c4-08d971ec545f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2448:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2448BB0A1710996356969CB5B3D39@VI1PR0401MB2448.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Bl9Wt7VdahHDctMw47ZPujEJbZgzQqzqB8HD+02GhHy2Z2nWU76doOtv6y1gXhRsvpaW+4bIrFby2aZ9pgSoFJkXy6PiQZX5Zbpa0z0dZrKdbQix6s3rYCvnSxACDL4u7yqIURQARCGjgjJjZ+ch9IWK/guEveJR8BCBRQ0RtHpX+ppSj4YkozWPbaPvFlO1zLg7H22z2R3Nr+4jUUeGNcGkCiVlvZOPGCt1fNWWmI2BwKfk2R05EsIfGb7fKiK1KlfwvUkQ44DF/oAHOw23wRHbFTPXlfr+tHzkaHvNP2nbLLa7nwRDNV4DzcmMPzCTckK1J+IecrFATD8iNO+gxCGL0Ly5A5aAfu4zmeBCHKJ+tPcN9FpaLCSOCevN58V9VjVMmbxNhGSLroiWZzbvCtsjLuZT6YS6ldM90nRTZCce+kifVa3K4dfkarMombraFctVnep1GHVevTj3yectgwBt/trCUL6QOolVQyQXn9sgmh7tr3/RzNF/48cCXlHZNDBrBi/QN1ELASzpZPDSkRH1WfCTbm9fHDORSPdURjjE6jPZyake0u0tW7eIHueJ5ytWfLpyLNgGGhhxqRJwRExGU/5VL//trPhkXd2dfOAh5eWWEIsBvsbQfL6ddiayHFWMb1Baaa94UeTz6QDVymJm5Ybebq+XtdQIVrWapwsASgh048loPkMnPHAnnOwWvcOCWC7odghxPnEPXaUvLTzRVA5orilOXb2udm/6l4Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(136003)(366004)(396003)(376002)(39860400002)(316002)(478600001)(38100700002)(5660300002)(26005)(31696002)(86362001)(2906002)(53546011)(186003)(6486002)(8676002)(66476007)(66946007)(8936002)(66556008)(31686004)(83380400001)(2616005)(956004)(7416002)(16576012)(36756003)(4326008)(110136005)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YUtVUmxvMFJtRnhMdk9Zc2duVXhURmtkZkxaREI0ZW51NU9JZDdXYVhGbmRR?=
 =?utf-8?B?Vit6YnpPb1JQK0EwZXVTcjJxK1I5OWlBaFZlZHh3NkQzaDFFajd4L1g5aFhP?=
 =?utf-8?B?NWZYbHVIZk1UaDNmcFBzZjJ4RU1hWlF2Q3BBZ1JVV3dtNmtzVmI5NzVVbVNh?=
 =?utf-8?B?ZWw3ZGxCT2NXS0FNK1VFRm05K0pBRlpvWk5EeW9qa3R6ZjdXalNjTVdHSXVz?=
 =?utf-8?B?enR5VmVqczdydWRIYnBwMW1VQStYNU4xZ2h6T09scGZpZm4rVEtlQVZCTWd5?=
 =?utf-8?B?VTVPaHdwWWxwZUVsUmcvclB2UWx6S0ljN2I1TlY3TjRkZ1JpZDlyQSs2clRQ?=
 =?utf-8?B?QVQzVXZlNHpXUmdsekcrMEJQaXB5cGdpK0taaHRNaVVQWE50MXVadUtOYXZB?=
 =?utf-8?B?RlNONGJ6ZFVHOUpNbGwyeFp5T0RUcTEyblM3dHE2ZXBOdjN2S2xJanJzdnF4?=
 =?utf-8?B?WSswUEgyU2x4YkQ4Wlk1MGNLcFFNQm1TRVp6NnVTTGZaV3QrYmwvZ3FKcm1y?=
 =?utf-8?B?STd3YitWVEFFVXdIQURrTTNHL3dvSWQydXIycWtCUzJJYXNpZklRcmVYSFlh?=
 =?utf-8?B?eTV3QmNUd3IxbFV4Q2wyS3FSbHFhSmdyVVBzQ1l5a2N6Y1Y5dzhnVWErUlha?=
 =?utf-8?B?QllJSkdBS0lGemZIMjZuVEExMWdHa0k5TEIzR2h0UE8yQ2YzZmRrdVpPdnUy?=
 =?utf-8?B?c09mRjdVK3JuMUtCV2FnbG44b25nQWZYTkMxQzVYQlF1TE1UcTVaYTVsQk1Q?=
 =?utf-8?B?ZjhIS28zbW53NmxsWThtdnlNei9ja0VxMU1RM082ZnpsV2RKVHEyZGxOM2Zt?=
 =?utf-8?B?L0dUY0NWV2xsdFc4b0tpRWNnamp0ZWozT1BoYmxzOHQ0a1Rpc3U1dEI1RWpq?=
 =?utf-8?B?TlBWQmhZejg5c1NwTGVTOVRtQnE3ZEE2RXhPUS9IMUQ0YnBPaXhuR2tXZ0hr?=
 =?utf-8?B?UEdmSnVDS1hVY21PajB3Z2k0S1d6di9IZThMV2pDeml5WktJQVZOdUF5Qzdm?=
 =?utf-8?B?SDgzd0lkU2JRdHBLTEEwYm9YTHlxU1F6NTZ5UnNmWHAyYVAzOFk5N2F0U2oy?=
 =?utf-8?B?NFNDRTJxTDJWdHNuS1QrbHJTNUJaT1ZyWG8rYjFCSmhYaDhPR1JucS9qMHcr?=
 =?utf-8?B?V093UDdBZDluV1FaNlI5dlVzem5GRFFXdDZqbEpRTTVkQnpaSjRKTjZ4L0xJ?=
 =?utf-8?B?Nnp1U3ZYRUJzcnlwSmpGU0ErSklzTGJrUiswTE92U243UGN6RFFIV29Ta21U?=
 =?utf-8?B?TUwyck5NUDlYY0t6MWFob0oxbC9pMlAzdFdvUWU1cytBWVpVTUZRRVNEZ3Fh?=
 =?utf-8?B?eWhSRy9xZnJZanM3Vll4VVoxRkNBN0dPQUROalQ0TmlGbExmbmF0MlU5aXRK?=
 =?utf-8?B?SDF1S2wrR3poeENKT0xLanRQNlhSRkpacmhjNko2WXRzYmlaU3FtOEZQTDh4?=
 =?utf-8?B?VDliTEQ0czZ6SnplQnBzbjc0T1lZUTNvVGVjVWhQUDhoSXFkc3BQWTJ4ZFBO?=
 =?utf-8?B?bWxMdlRIYitSUDFsa2FlUjRNWGE3NmovZHVtTnprTWJvNWhaRTNQTlVpOFRw?=
 =?utf-8?B?K3BXK3plQ0hZeU5lMGNZTTVocVZuY2FYZks5c2dRQUZIa21wN2p3eVdZSlRZ?=
 =?utf-8?B?YzlOYkZSejRiT2Z0aFRHbWhHSkNMWjJSYWVYV1VGRyt2TlI5dXczMGl3SlN2?=
 =?utf-8?B?VWdSZXV3dGUyR1RaUXhKNFpqd1pqS1JaaFVja3NkWCtjRC9QVlBkelo5R0xv?=
 =?utf-8?Q?Mk6rSMYQSc5pc6DDhFSFgwwyB3vjbZFWgpgbpJ4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e4ad2d90-1138-419b-24c4-08d971ec545f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:43:29.2427
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: t5VD51ig4A8zNA0EXuPmX1BvK34PiVPm7+HnmoKxJ0i1iACLHUDlBnm9HXl1IW0N9VYwodJzjyj2kxUufLgU5g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2448

On 07.09.2021 12:11, Oleksandr Andrushchenko wrote:
> On 06.09.21 17:11, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
>>>   }
>>>   REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
>>>   
>>> +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
>>> +{
>>> +    int rc;
>>> +
>>> +    /* Remove previously added registers. */
>>> +    vpci_remove_device_registers(pdev);
>>> +
>>> +    /* It only makes sense to add registers for hwdom or guest domain. */
>>> +    if ( d->domain_id >= DOMID_FIRST_RESERVED )
>>> +        return 0;
>>> +
>>> +    if ( is_hardware_domain(d) )
>>> +        rc = add_bar_handlers(pdev, true);
>>> +    else
>>> +        rc = add_bar_handlers(pdev, false);
>>      rc = add_bar_handlers(pdev, is_hardware_domain(d));
> Indeed, thank you ;)
>>
>>> +    if ( rc )
>>> +        gprintk(XENLOG_ERR,
>>> +            "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
>>> +            d->domain_id);
>> Please use %pd and correct indentation. Logging the error code might
>> also help some in diagnosing issues.
> Sure, I'll change it to %pd
>>   Further I'm not sure this is a
>> message we want in release builds
> Why not?

Excess verbosity: If we have such here, why not elsewhere on error paths?
And I hope you agree things will get too verbose if we had such (about)
everywhere?

>>   - perhaps gdprintk()?
> I'll change if we decide so
>>
>>> +    return rc;
>>> +}
>>> +
>>> +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev)
>>> +{
>>> +    /* Remove previously added registers. */
>>> +    vpci_remove_device_registers(pdev);
>>> +    return 0;
>>> +}
>> Also - in how far is the goal of your work to also make vPCI work for
>> x86 DomU-s? If that's not a goal
> It is not, unfortunately. The goal is not to break x86 and to enable Arm
>> , I'd like to ask that you limit the
>> introduction of code that ends up dead there.
> 
> What's wrong with this function even if it is a one-liner?

The comment is primarily on the earlier function, and then extends to
this one.

> This way we have a pair vpci_bar_add_handlers/vpci_bar_remove_handlers
> and if I understood correctly you suggest vpci_bar_add_handlers/vpci_remove_device_registers?
> What would we gain from that, but yet another secret knowledge that in order
> to remove BAR handlers one needs to call vpci_remove_device_registers
> while I would personally expect to call vpci_bar_add_handlers' counterpart,
> vpci_remove_device_registers namely.

This is all fine. Yet vpci_bar_{add,remove}_handlers() will, aiui, be
dead code on x86. Hence there should be an arrangement allowing the
compiler to eliminate this dead code. Whether that's enclosing these
by "#ifdef" or adding early "if(!IS_ENABLED(CONFIG_*))" is secondary.
This has a knock-on effect on other functions as you certainly realize:
The compiler seeing e.g. the 2nd argument to the add-BARs function
always being true allows it to instantiate just a clone of the original
function with the respective conditionals removed.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:02:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:02:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180823.327644 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYrN-0007QB-UP; Tue, 07 Sep 2021 11:01:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180823.327644; Tue, 07 Sep 2021 11:01:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYrN-0007Q4-Qe; Tue, 07 Sep 2021 11:01:49 +0000
Received: by outflank-mailman (input) for mailman id 180823;
 Tue, 07 Sep 2021 11:01:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYrN-0007Pt-0x; Tue, 07 Sep 2021 11:01:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYrM-0002Hu-Ra; Tue, 07 Sep 2021 11:01:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYrM-0007zl-Iq; Tue, 07 Sep 2021 11:01:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYrM-00069J-IJ; Tue, 07 Sep 2021 11:01:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=U8Tws86ms3G3xPU9BlXML1AV03LC9htzmTEZjqahcwI=; b=BlKy9R1i7OlHVl54yRkzIf/eqA
	+BqO7xxTfNbidYoBaLifnHZZHMy6/nFfHvYfHEZob0qSANudps8s00hBuLCwzLpdkzho5yFtxV7Zz
	6ZkcDw4buxoj0uWk9JLjvv0gvsqyMciYYfAaZQyajXlDiONUHQnOws+rW4VV0bZQ2ZR8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164870-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164870: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=8f8eeb3750ae410e4b6fa96a2a559e0d9960716c
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 11:01:48 +0000

flight 164870 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164870/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              8f8eeb3750ae410e4b6fa96a2a559e0d9960716c
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  424 days
Failing since        151818  2020-07-11 04:18:52 Z  423 days  414 attempts
Testing same since   164870  2021-09-07 04:20:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71974 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:07:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:07:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180830.327658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYwX-00086V-Hg; Tue, 07 Sep 2021 11:07:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180830.327658; Tue, 07 Sep 2021 11:07:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYwX-00086O-ER; Tue, 07 Sep 2021 11:07:09 +0000
Received: by outflank-mailman (input) for mailman id 180830;
 Tue, 07 Sep 2021 11:07:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYwW-00085T-Lk; Tue, 07 Sep 2021 11:07:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYwW-0002NG-Gr; Tue, 07 Sep 2021 11:07:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYwW-000893-6F; Tue, 07 Sep 2021 11:07:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNYwW-0002KJ-5j; Tue, 07 Sep 2021 11:07:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=M63KrIA4eeoRfQIlmBDz/bKKgsssf+hstv0t3aEBzmM=; b=sZ/FfmFSS7NtzYqmsDccsoQzga
	W9pspFqdrivKlyl5WMYJ+9uszLBF2CVzYsuvWefZuvrx+mmid6MDgQzdwgLYhIxG7KB7j/QVaOyzX
	YgfvIncIoWOBN96r0+4YtOL2iFMJJnxu3d3bCkRhTwqF74q98ai9wOpI0G8e1y15gWCA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164866-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164866: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 11:07:08 +0000

flight 164866 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164866/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164848
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164848
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164848
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164848
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164848
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164848
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164848
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164848
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164848
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164848
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164848
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164848  2021-09-06 01:51:30 Z    1 days
Testing same since   164866  2021-09-06 23:07:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Orzel <michal.orzel@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2d4978ead4..6c27a8da8d  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17 -> master


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:10:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:10:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180838.327671 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYze-000168-6o; Tue, 07 Sep 2021 11:10:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180838.327671; Tue, 07 Sep 2021 11:10:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNYze-000161-2u; Tue, 07 Sep 2021 11:10:22 +0000
Received: by outflank-mailman (input) for mailman id 180838;
 Tue, 07 Sep 2021 11:10:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNYzc-00015t-37
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 11:10:20 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ba463846-66eb-43ae-8be8-af80615d07b8;
 Tue, 07 Sep 2021 11:10:19 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1879h71j030311; 
 Tue, 7 Sep 2021 11:10:16 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2050.outbound.protection.outlook.com [104.47.9.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ax3d2rtyj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 11:10:15 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM6PR03MB3959.eurprd03.prod.outlook.com (2603:10a6:20b:20::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 11:10:10 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 11:10:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba463846-66eb-43ae-8be8-af80615d07b8
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YR0VHE9EsYK0tB1QbfJwiv/Jjpug1suw4B/TxtXJXKaXrqyi2kh0zbxQoHvXcIE4nlFu68Ou7hK29HGBPl5Ro9/i/wiQ4F3AwNI2dFVXhPRQUEqNF0iT9qR+V2wGoBlwiMJ9rr810YuOShmOtBp8kPT5C8Xzt6wZ9C063Ap3jh1G2EttWwIyKpT41bPMqxNRDUDTU3vwsjGwAwg4YBfRYCmWX0jZDUq3GCQxeOWErVFVb8gmuHT18D9EdtKXv/faBdGIZuQ/WcA9/4hJMxRsykqv56RdUg9Oh9cxKX9LoRO66m3eAdqsp3erkoevrFgtG7gfSIe5Z3iyvZ8bb0AQFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PV70tGGgugQ0qJ9VluEfc8FUAV6SKKq+W2RYBlTHk04=;
 b=E/xFVVMyev/eaJxmc+SKnxw4W7mjlBXLl0KSnEY/BYgqbbxUgHsU1bbOTvfdVjjQxGrAaZOH5hkkyAqrToSJQ/NJ86e0FQELXiMqnRENYTm4T2JgFJEjf9gKEMc3Dwp1ArHBpwJ7XnO7DXgw5iG1t+R/K/fsKKxGpNIG0G35VEhQXLxrckb2/JJmCu/z9zYksTplz8d8n1Pe8fW2vQOAhRdaHhq7U284az3etEgVa/R7qGr0y0Qnjrjq+2Q8xQOY5PEJSWx2EPcnVk81BDEE7CMX5cZrO9Nm2js7C4wmHkzsgepylPMuiEnG27tMzlsojrxfUiTXUsOZzvBLYhhy8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PV70tGGgugQ0qJ9VluEfc8FUAV6SKKq+W2RYBlTHk04=;
 b=Q2FQuaUUFMlML6OpYGJbUspvS4ImPuAU47cLT9lzkPctJ5yA6SqxvVjTtPOJ1qiO7OztqwIErKE+vvG4S0ZO2/f1bLVeZQ8WgecXrb7CMk8lQgwzaJxrABS4BX9gIfssHrOMYMc00PPqmV/IRvSBc8MhsElYyfYfAgn9VsWU24F/AyR98+OjUrQOf/Ndnt/ebZQApNDKVdVPQrvWaNcf9VaRYyX0hXdPMgCcIQeNTwuHRyLA04uAvOmg2ywfdv8h5Y4bRJw+Gw8GjUEZiTK2qcmFDoCDqOaeVxOI2JqhWwG+uyDEuHVeKKsw9utoFbgV1vGqExyWSYFdFsNTES11yw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Topic: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Index: AQHXoKusySJnLVveGkulaWKrBZdXpKuXEOQAgAFPE4CAAAkRAIAAB3eA
Date: Tue, 7 Sep 2021 11:10:10 +0000
Message-ID: <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
 <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
In-Reply-To: <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 5376dc28-8c0c-4ec9-d32b-08d971f00eea
x-ms-traffictypediagnostic: AM6PR03MB3959:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM6PR03MB3959843F6417D3BC69019172E7D39@AM6PR03MB3959.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 zOvJSQ988Ul7tKRw1MAwisz3776VbU6uKa4WOkelhFCH/55bPWKoT8eaFUGyDqjNL0Fn9SmsEoQjsXyndChAYM4ilAz2yWYdr0sDZ1choqFJDxzJEbMO4JRDuo5GWbr8I94gCT/Uw1pVDV3o/uWJfwz5MOubJQxHQ381xJuFz8lEZHVipqmSCmhLjZG7ulRQdT+CbpciVYdnVA6DQXiEGlBCYuIKXJK+UKjGu8vArhAFYJLvT2HxsJ4KmHDcaLGJlbRF2UXNlPERQxr1feSeNVw4606TMSiyjPogtPmlFfg4z6v4EJiEL/jk9UlaUQa4m8+eUA1kOJrC4YPwbXiGgGoGFYmHCiQKZmR0fiozXZ9OEL1iOPK+pg/8DaVez/3G+iV9RmdIxktwpo5Fed1bWO5NM0VG7SaLuRRQqlfmgj6TSrWTCtDQGYztBf6SwTzVUD3XijMN4cADL93kQqVcYl/HXlFZSJDFJew0/P4QHtIrqmAWgC6Coow13ZdhZqSJQlQWYufieF07w0mMAQjMiP6983pnPTMm8t8IjCJdhDsHotWhDWrzF8OqCcPrBXHW8zchbJLselLaaP+YJRNAfgLAln64kflGOGNFjQLK+Tm9QXcQ7S9vD2iDJZ6U/Ju0UJ8luZ2EZgM1ODZEJeM1kWLTrq7lRVHCo3oPOAmyJT27tgHXxNaH6OtvUVUTgwiNVGelcPMlTs2IwB7mkr//D86RLlRB5t9QVGmcAI/Ci4mCiZ+9MstB77PFSnQMhDFr
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(366004)(39850400004)(346002)(136003)(396003)(110136005)(38100700002)(86362001)(91956017)(6512007)(66556008)(122000001)(64756008)(66446008)(71200400001)(76116006)(66476007)(66946007)(4326008)(83380400001)(2616005)(31696002)(53546011)(316002)(6506007)(6486002)(8936002)(5660300002)(8676002)(36756003)(26005)(186003)(31686004)(478600001)(54906003)(2906002)(38070700005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?L1hZV1Y4TWJXU25oRUNaeWJjL2RlVHdTMXVUcm4ySE5pWjlrZXlYQnp1Z0Rj?=
 =?utf-8?B?aW5jV1hlci82UElhNlN6Sy9SSU4rV1pTZ2FPOEp6T0JnRmxlVlhHTGpkUW9G?=
 =?utf-8?B?T251RS9EaHV1MHYydU11TFBObVNXQ21DS0F0di9NeVBKNU5OSUtud0Z6aGhk?=
 =?utf-8?B?aWlKaHBTbVdTYzR0cWFxVXBrbVJBblNIb1hjMzJpV3pabkh0RlRXTTE2MWVa?=
 =?utf-8?B?cThIM3VVTGVDaEpzUWtMSXJDREp0U09pM29tSU9OZEJNaEd3eHREOU5YbjZp?=
 =?utf-8?B?TXluV2c4U3lidFdEc0dwdDcrMm0yM1VESGF2NFY4Tm9rUXVUdzFyTHFXUTFM?=
 =?utf-8?B?bUo1c2J6SnhnekI0VDBLbEo5TlFmRkNualpUd1FxR090czRZYWtiWGVicDQv?=
 =?utf-8?B?UjNZb1FYZlh3L1pGTWlSK3VFeUdpYm5MQXlhdlI4UVZ6bExmV1BxNHhobDJG?=
 =?utf-8?B?bDFVRGZ1VjBGcjRsWWhCUEd1aTUvYjY5bEpqeE1IWGpTRDB2SmxpV1kxMm16?=
 =?utf-8?B?d0tKV3g5V0YwSUVMc0R4UEt5VnFSalBLQ2haZS9FNmxFYlFHWmpTTVo1bDhD?=
 =?utf-8?B?b3FwYzI4YTNMYmQ4RE1yK2sydTMxVEpOekI0eWdUUTdzQi9lSk84UXFqczUy?=
 =?utf-8?B?QTVhNytPdnlyK01Wd2lNTlVqdm1JZXpsOFNjdkdyR05xOVdlSGxPT1ltT3hR?=
 =?utf-8?B?R2hoTnB1Qk5UU2ZDSEdpUW5HV0Zod0tnblhoNkppOTd0a09YVG5yK2Nwajln?=
 =?utf-8?B?RXU3ZWttQTlrT0dsS1F1cTgrMXA3dlpJUW80WmlXaE9nNzRZWmwramNPT0pF?=
 =?utf-8?B?Skp1ZWhFcXRyUEN3WGcrbXU5eGFIYWxUWUZDMzBpcXBDcm1ZSmRlRnNET3Zn?=
 =?utf-8?B?d2xiUmVqaHMwQ2JpSUFhQ3IxNnFkalh5ZktQbGgrRWluaDRRdlRNS1UvTlNp?=
 =?utf-8?B?MkRNYnVSWVNYQndEZGtTZGZLRFRzUmlOM1RzdjFRUDJ2TmhCRzBkak5BY25X?=
 =?utf-8?B?Q1hQL3VJV1NIclVGVVVzcHB5eTN3d05KTVI0WHRTY1loVjZpRkxZZmc2RDFF?=
 =?utf-8?B?ZU83MmFBaERTd2RjQi8rV3FuK0ZSWjlEWnNKMVhGaityN29Fc3UvYlVxZXky?=
 =?utf-8?B?MmNaVlhBbXZ0bDZBaWRYKzZhQWFKRUFnVXJwQW9HWCtsekdCVWlvbGNyTXF1?=
 =?utf-8?B?b1ovRjg5WXd6VEtVSi93a2x5cElmRlBVeDFkeWQxSmZqYzMrZXdXaVcrSlJN?=
 =?utf-8?B?SkFlTkN1MXdtaHovdWFHTlhaVzdIcHJYQmVKdDR4V3Z0U25abmFRVTltWVZU?=
 =?utf-8?B?RU5MbWNlYTVJMGg0dDdkMGVFak95MjZ0OEh0UUM0TmxHZUR0TkhCSzNrempD?=
 =?utf-8?B?RFZseCtBbjF1ZUs3RWpBa1dXTVRPTjJoZGxXaVJSOXhtVnh4VStMNEF0Rktw?=
 =?utf-8?B?TVhLa2dlcGt6MXRWaTc2ZWYxZzc3NVlIc3F3MDB3TWtKRDNtY0tOdnRtc0JI?=
 =?utf-8?B?UHZhYy9NT0pMTk1ITkdRSjk3VndTcDAzL2dnUVgxVmdpSktpOXdBZDd4OUVo?=
 =?utf-8?B?NHZORld6RnBNRzR0aW1QOC83czJQM3psOFBwVVc0bUhCd3oyV0lwU3lkcUhN?=
 =?utf-8?B?c3dqdTVmS28zbFVYa09RY2JRZk55a09Oc3c2ZXRhd2F4Zmc0MGVieFpLNUhD?=
 =?utf-8?B?aGE5MXlkaHdKUFFxeVBQYXdCRDkvQmdBR0VSSVc0c21KQ2NKZmZNY2E5eGFz?=
 =?utf-8?Q?3WevrmrXVpPwkoahpIQ105CzsIWOjfe+8llZ8Ki?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F453448ADFB9BA4B976105F65D448495@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5376dc28-8c0c-4ec9-d32b-08d971f00eea
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 11:10:10.4263
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ThyXFsqRQ7QJ2JkPSwkye3Hx5DumtHH6JyD3ihjbNnbJinfWI8fzjHlkTH1Vu/SP+k4UvtgwP1Wh2/DPL7KfSiAQPbjzKaW6nlQ8RX/In/RsqWaLOnIH+OoHWoFIefxH
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB3959
X-Proofpoint-ORIG-GUID: pCZ4OBDrU4JOeyYAQz9IxTjVYs_UdpQZ
X-Proofpoint-GUID: pCZ4OBDrU4JOeyYAQz9IxTjVYs_UdpQZ
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_03,2021-09-07_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0
 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxlogscore=952
 suspectscore=0 mlxscore=0 bulkscore=0 priorityscore=1501 phishscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070074

DQpPbiAwNy4wOS4yMSAxMzo0MywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTI6MTEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDYuMDkuMjEgMTc6
MTEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBAQCAtNTkzLDYgKzYyNSwzNiBAQCBzdGF0aWMg
aW50IGluaXRfYmFycyhzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4gICAgfQ0KPj4+PiAgICBS
RUdJU1RFUl9WUENJX0lOSVQoaW5pdF9iYXJzLCBWUENJX1BSSU9SSVRZX01JRERMRSk7DQo+Pj4+
ICAgIA0KPj4+PiAraW50IHZwY2lfYmFyX2FkZF9oYW5kbGVycyhjb25zdCBzdHJ1Y3QgZG9tYWlu
ICpkLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4gK3sNCj4+Pj4gKyAgICBpbnQgcmM7DQo+
Pj4+ICsNCj4+Pj4gKyAgICAvKiBSZW1vdmUgcHJldmlvdXNseSBhZGRlZCByZWdpc3RlcnMuICov
DQo+Pj4+ICsgICAgdnBjaV9yZW1vdmVfZGV2aWNlX3JlZ2lzdGVycyhwZGV2KTsNCj4+Pj4gKw0K
Pj4+PiArICAgIC8qIEl0IG9ubHkgbWFrZXMgc2Vuc2UgdG8gYWRkIHJlZ2lzdGVycyBmb3IgaHdk
b20gb3IgZ3Vlc3QgZG9tYWluLiAqLw0KPj4+PiArICAgIGlmICggZC0+ZG9tYWluX2lkID49IERP
TUlEX0ZJUlNUX1JFU0VSVkVEICkNCj4+Pj4gKyAgICAgICAgcmV0dXJuIDA7DQo+Pj4+ICsNCj4+
Pj4gKyAgICBpZiAoIGlzX2hhcmR3YXJlX2RvbWFpbihkKSApDQo+Pj4+ICsgICAgICAgIHJjID0g
YWRkX2Jhcl9oYW5kbGVycyhwZGV2LCB0cnVlKTsNCj4+Pj4gKyAgICBlbHNlDQo+Pj4+ICsgICAg
ICAgIHJjID0gYWRkX2Jhcl9oYW5kbGVycyhwZGV2LCBmYWxzZSk7DQo+Pj4gICAgICAgcmMgPSBh
ZGRfYmFyX2hhbmRsZXJzKHBkZXYsIGlzX2hhcmR3YXJlX2RvbWFpbihkKSk7DQo+PiBJbmRlZWQs
IHRoYW5rIHlvdSA7KQ0KPj4+PiArICAgIGlmICggcmMgKQ0KPj4+PiArICAgICAgICBncHJpbnRr
KFhFTkxPR19FUlIsDQo+Pj4+ICsgICAgICAgICAgICAiJXBwOiBmYWlsZWQgdG8gYWRkIEJBUiBo
YW5kbGVycyBmb3IgZG9tJWRcbiIsICZwZGV2LT5zYmRmLA0KPj4+PiArICAgICAgICAgICAgZC0+
ZG9tYWluX2lkKTsNCj4+PiBQbGVhc2UgdXNlICVwZCBhbmQgY29ycmVjdCBpbmRlbnRhdGlvbi4g
TG9nZ2luZyB0aGUgZXJyb3IgY29kZSBtaWdodA0KPj4+IGFsc28gaGVscCBzb21lIGluIGRpYWdu
b3NpbmcgaXNzdWVzLg0KPj4gU3VyZSwgSSdsbCBjaGFuZ2UgaXQgdG8gJXBkDQo+Pj4gICAgRnVy
dGhlciBJJ20gbm90IHN1cmUgdGhpcyBpcyBhDQo+Pj4gbWVzc2FnZSB3ZSB3YW50IGluIHJlbGVh
c2UgYnVpbGRzDQo+PiBXaHkgbm90Pw0KPiBFeGNlc3MgdmVyYm9zaXR5OiBJZiB3ZSBoYXZlIHN1
Y2ggaGVyZSwgd2h5IG5vdCBlbHNld2hlcmUgb24gZXJyb3IgcGF0aHM/DQo+IEFuZCBJIGhvcGUg
eW91IGFncmVlIHRoaW5ncyB3aWxsIGdldCB0b28gdmVyYm9zZSBpZiB3ZSBoYWQgc3VjaCAoYWJv
dXQpDQo+IGV2ZXJ5d2hlcmU/DQpBZ3JlZSwgd2lsbCBjaGFuZ2UgaXQgdG8gZ2RwcmludGsNCj4N
Cj4+PiAgICAtIHBlcmhhcHMgZ2RwcmludGsoKT8NCj4+IEknbGwgY2hhbmdlIGlmIHdlIGRlY2lk
ZSBzbw0KPj4+PiArICAgIHJldHVybiByYzsNCj4+Pj4gK30NCj4+Pj4gKw0KPj4+PiAraW50IHZw
Y2lfYmFyX3JlbW92ZV9oYW5kbGVycyhjb25zdCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgcGNp
X2RldiAqcGRldikNCj4+Pj4gK3sNCj4+Pj4gKyAgICAvKiBSZW1vdmUgcHJldmlvdXNseSBhZGRl
ZCByZWdpc3RlcnMuICovDQo+Pj4+ICsgICAgdnBjaV9yZW1vdmVfZGV2aWNlX3JlZ2lzdGVycyhw
ZGV2KTsNCj4+Pj4gKyAgICByZXR1cm4gMDsNCj4+Pj4gK30NCj4+PiBBbHNvIC0gaW4gaG93IGZh
ciBpcyB0aGUgZ29hbCBvZiB5b3VyIHdvcmsgdG8gYWxzbyBtYWtlIHZQQ0kgd29yayBmb3INCj4+
PiB4ODYgRG9tVS1zPyBJZiB0aGF0J3Mgbm90IGEgZ29hbA0KPj4gSXQgaXMgbm90LCB1bmZvcnR1
bmF0ZWx5LiBUaGUgZ29hbCBpcyBub3QgdG8gYnJlYWsgeDg2IGFuZCB0byBlbmFibGUgQXJtDQo+
Pj4gLCBJJ2QgbGlrZSB0byBhc2sgdGhhdCB5b3UgbGltaXQgdGhlDQo+Pj4gaW50cm9kdWN0aW9u
IG9mIGNvZGUgdGhhdCBlbmRzIHVwIGRlYWQgdGhlcmUuDQo+PiBXaGF0J3Mgd3Jvbmcgd2l0aCB0
aGlzIGZ1bmN0aW9uIGV2ZW4gaWYgaXQgaXMgYSBvbmUtbGluZXI/DQo+IFRoZSBjb21tZW50IGlz
IHByaW1hcmlseSBvbiB0aGUgZWFybGllciBmdW5jdGlvbiwgYW5kIHRoZW4gZXh0ZW5kcyB0bw0K
PiB0aGlzIG9uZS4NCj4NCj4+IFRoaXMgd2F5IHdlIGhhdmUgYSBwYWlyIHZwY2lfYmFyX2FkZF9o
YW5kbGVycy92cGNpX2Jhcl9yZW1vdmVfaGFuZGxlcnMNCj4+IGFuZCBpZiBJIHVuZGVyc3Rvb2Qg
Y29ycmVjdGx5IHlvdSBzdWdnZXN0IHZwY2lfYmFyX2FkZF9oYW5kbGVycy92cGNpX3JlbW92ZV9k
ZXZpY2VfcmVnaXN0ZXJzPw0KPj4gV2hhdCB3b3VsZCB3ZSBnYWluIGZyb20gdGhhdCwgYnV0IHll
dCBhbm90aGVyIHNlY3JldCBrbm93bGVkZ2UgdGhhdCBpbiBvcmRlcg0KPj4gdG8gcmVtb3ZlIEJB
UiBoYW5kbGVycyBvbmUgbmVlZHMgdG8gY2FsbCB2cGNpX3JlbW92ZV9kZXZpY2VfcmVnaXN0ZXJz
DQo+PiB3aGlsZSBJIHdvdWxkIHBlcnNvbmFsbHkgZXhwZWN0IHRvIGNhbGwgdnBjaV9iYXJfYWRk
X2hhbmRsZXJzJyBjb3VudGVycGFydCwNCj4+IHZwY2lfcmVtb3ZlX2RldmljZV9yZWdpc3RlcnMg
bmFtZWx5Lg0KPiBUaGlzIGlzIGFsbCBmaW5lLiBZZXQgdnBjaV9iYXJfe2FkZCxyZW1vdmV9X2hh
bmRsZXJzKCkgd2lsbCwgYWl1aSwgYmUNCj4gZGVhZCBjb2RlIG9uIHg4Ni4NCnZwY2lfYmFyX2Fk
ZF9oYW5kbGVycyB3aWxsIGJlIHVzZWQgYnkgeDg2IFBWSCBEb20wDQo+ICAgSGVuY2UgdGhlcmUg
c2hvdWxkIGJlIGFuIGFycmFuZ2VtZW50IGFsbG93aW5nIHRoZQ0KPiBjb21waWxlciB0byBlbGlt
aW5hdGUgdGhpcyBkZWFkIGNvZGUuDQoNClNvLCB0aGUgb25seSBkZWFkIGNvZGUgZm9yIHg4NiBo
ZXJlIHdpbGwgYmUgdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzLiBZZXQuDQoNCkJlY2F1c2UgSSBo
b3BlIHg4NiB0byBnYWluIGd1ZXN0IHN1cHBvcnQgZm9yIFBWSCBEb20wIHNvb25lciBvciBsYXRl
ci4NCg0KPiAgIFdoZXRoZXIgdGhhdCdzIGVuY2xvc2luZyB0aGVzZQ0KPiBieSAiI2lmZGVmIiBv
ciBhZGRpbmcgZWFybHkgImlmKCFJU19FTkFCTEVEKENPTkZJR18qKSkiIGlzIHNlY29uZGFyeS4N
Cj4gVGhpcyBoYXMgYSBrbm9jay1vbiBlZmZlY3Qgb24gb3RoZXIgZnVuY3Rpb25zIGFzIHlvdSBj
ZXJ0YWlubHkgcmVhbGl6ZToNCj4gVGhlIGNvbXBpbGVyIHNlZWluZyBlLmcuIHRoZSAybmQgYXJn
dW1lbnQgdG8gdGhlIGFkZC1CQVJzIGZ1bmN0aW9uDQo+IGFsd2F5cyBiZWluZyB0cnVlIGFsbG93
cyBpdCB0byBpbnN0YW50aWF0ZSBqdXN0IGEgY2xvbmUgb2YgdGhlIG9yaWdpbmFsDQo+IGZ1bmN0
aW9uIHdpdGggdGhlIHJlc3BlY3RpdmUgY29uZGl0aW9uYWxzIHJlbW92ZWQuDQoNCldpdGggdGhl
IGFib3ZlIChlLmcuIGFkZCBpcyBnb2luZyB0byBiZSB1c2VkLCBidXQgbm90IHJlbW92ZSkgZG8g
eW91DQoNCnRoaW5rIGl0IGlzIHdvcnRoIHBsYXlpbmcgd2l0aCBpZmRlZidzIHRvIHN0cmlwIHRo
YXQgc2luZ2xlIGZ1bmN0aW9uIGFuZCBhZGQNCg0KYSBwaWVjZSBvZiBzcGFnaGV0dGkgY29kZSB0
byBzYXZlIGEgYml0PyBXaGF0IHdvdWxkIHRoYXQgaWZkZWYgbG9vayBsaWtlLA0KDQplLmcuICNp
ZmRlZiBDT05GSUdfQVJNIG9yICNpZm5kZWYgQ09ORklHX1g4NiAmJiBhbnkgb3RoZXIgcGxhdGZv
cm0sIGJ1dCBBUk0/DQoNCklNTywgaXQgaXMgY2xlYW5lciB0byBsZWF2ZSBpdCBhcyBpcy4gWWV0
IHdlIHdhc3RlIHNvbWUgYml0cyBmb3IgeDg2Lg0KDQo+DQo+IEphbg0KPg0KVGhhbmsgeW91LA0K
DQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:34:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:34:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180848.327687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZMw-0004Kb-8Q; Tue, 07 Sep 2021 11:34:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180848.327687; Tue, 07 Sep 2021 11:34:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZMw-0004KU-53; Tue, 07 Sep 2021 11:34:26 +0000
Received: by outflank-mailman (input) for mailman id 180848;
 Tue, 07 Sep 2021 11:34:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qx60=N5=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNZMu-0004KO-Lu
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 11:34:24 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ccfbc2d0-b984-4a6f-990a-a474e83ec3e9;
 Tue, 07 Sep 2021 11:34:21 +0000 (UTC)
Received: from PR0P264CA0054.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::18)
 by VE1PR08MB5775.eurprd08.prod.outlook.com (2603:10a6:800:1a0::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 11:34:07 +0000
Received: from VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1d:cafe::d4) by PR0P264CA0054.outlook.office365.com
 (2603:10a6:100:1d::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 11:34:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT007.mail.protection.outlook.com (10.152.18.114) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 11:34:07 +0000
Received: ("Tessian outbound 0ec886cb54dd:v105");
 Tue, 07 Sep 2021 11:34:06 +0000
Received: from eb989c4f2559.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8711F606-4838-4034-B687-C61EEADCF023.1; 
 Tue, 07 Sep 2021 11:33:56 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id eb989c4f2559.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 11:33:56 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6270.eurprd08.prod.outlook.com (2603:10a6:102:f3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 11:33:54 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 11:33:54 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0050.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 11:33:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ccfbc2d0-b984-4a6f-990a-a474e83ec3e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MCCMeL6Eftx+YuUav1pGAJzgOA0UT56Kr+WzuuB4RWM=;
 b=JJj7S3F0et2bfdOwiB5uOWCIPf/d6CAmCWTv9MYT3IrihaFF2lOTeMKYEbRwSmpVq05DLABkolHQn+hBpsSq03+onbb8o7bLDAjyffaCt/Z26xh5zu1DY/cDbjN1xvQsQRxzBu1AyjKrBtndX5SqawSaTq9FK9T28U9CwBqiLzU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 35c017e931b65242
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FctFGljTKHwoIn+RkMW29m+bJRD25ctBnAZGlAnZ8q4S0yrH9O3pCruloZuPa1PmiBxzIOwd7nGy4tQi9tsiTcH6CHLwxt3Kof2BYlYvOvTIRjiYHd+tD1T6BkukXuVOF8qVsEFgYU+gbnaz+kF7A7ov4+UEefdat21XGEYeCbV4CHtwlY/lf1iHWEz3crkB7A/v2E4kLFoTOz5NFyzepXNKD1P/OT642Jf7LHAqfVEDB0pY+2l+7SahLRTrYBJrhzdQ3+NTca7LYtpEUSHHSksUHVhMb2kjScypJ1MD0VeAaoiKdTsVMSK/QRuS0fd6IyqnngTujc8qfE8Ub+UtMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MCCMeL6Eftx+YuUav1pGAJzgOA0UT56Kr+WzuuB4RWM=;
 b=TxvVrqZSbtTtWbkjFK3Cnay1kB5MTmSqkFKu1BMz37EU4NebkEmeDvsxwY2f0oqRpqKk6lx98rxUOrh2NvEyj9h3L8sFobZTitwq7x24n1bbRA8zBH+t3RD20IJhCXJn5MQBEMCRMtSj0Aa6UnNH3OUgtv3hgjhd1Z3LXz3AwBBS5dQPIWytIHnfMiHu/FA+ZfA7ttDcTBT/DCG/cZAKoXIRfY6dXv1XM0UGwaeqxwYx0v0pa2yspK5vo73Vvf5Afwj3frug1rYZPddHANIGk+W0ncAN1WkOu+vHsJjPS65HXXuh6iMS0Iyq9EJo7QaXkRrOsGBeURwdIGJmCckdtQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MCCMeL6Eftx+YuUav1pGAJzgOA0UT56Kr+WzuuB4RWM=;
 b=JJj7S3F0et2bfdOwiB5uOWCIPf/d6CAmCWTv9MYT3IrihaFF2lOTeMKYEbRwSmpVq05DLABkolHQn+hBpsSq03+onbb8o7bLDAjyffaCt/Z26xh5zu1DY/cDbjN1xvQsQRxzBu1AyjKrBtndX5SqawSaTq9FK9T28U9CwBqiLzU=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1b4eef88-78cb-7201-c326-551261f9e9ad@suse.com>
Date: Tue, 7 Sep 2021 12:33:47 +0100
Cc: Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <1964F880-98C2-45BF-8CED-C34397F61F4C@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <ea736db5-e3cf-7eea-cffe-98ecb0290048@suse.com>
 <3222722f-b12e-3991-d0de-b455b9fca063@xen.org>
 <1b4eef88-78cb-7201-c326-551261f9e9ad@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0050.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:152::19) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 59b135d2-57d4-42c1-0948-08d971f36755
X-MS-TrafficTypeDiagnostic: PA4PR08MB6270:|VE1PR08MB5775:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB57757D3DA3474DDAA09E728DE4D39@VE1PR08MB5775.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 qLZHWkisKTiCF3UoirOvktChV1mbtN1n+e90qMT4ejv+8KMVwGAAHcDn9CmZJ/AB4cE43D2WK+b6VkPXafesx3Lc0ecTPBFOKZZlXAZszx362euaGd25CjYQsDSUtOgY/WIlqdDt3y3/GKuv+FvmbFuBzVzzeV78E26rTq5jfqkDFsA3PvA5kTXKCXuu3lzBFIeZjBP/9k52VFddOxLUGkgxm6iH43pyGvoXaTEDDBYuy37zA8Setxxo+VMEREfPpXR6OZVxG+A3CEf1kaZBbPI8M/84C/t7o0aOSsp7X/m4U3hFeGnzohOIlsc+vkEjQcQgcTMTujC+mUO3wVulSSuIpdNWtE8tyo1JrMcTKT7Mgtf+amI1OHQUiuOe+Px+xl/UZ9abtt/mJcQK1apYbIV7aKopBit8UjnaItomn3vdlU8VtDkHNHoUSn/ZVf8zwmBrgPTKpToNqbPGJfi9Q55Y4s+xgj262pEP4kiYwpb7eER9Y6FMECYryPy6yHXwC1IuMdeL7YLHl7heO3wKpimNZqLSBqvbxqDrtfgRKEccCp0yAb1e7UOuK6wmZwNsmiQTgeJtZMuYHl2tMGyFeW6wCgoBSdCusEM6YE3FuT76Kx7d21MdoLUmGOKqFimVGAQ6iq19qmD5m8xonDESq4Q71yTXYROYINWMRoGN+2AGlOY8QAG4BeMqST9pKAZDcCxVFfOld34PMA38ykYESGvRvF/6ShZ8SCM0gR7kpL/6GHJ8eaw/Pg6Soxm4CSEl
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(136003)(39850400004)(396003)(366004)(6512007)(44832011)(53546011)(26005)(186003)(66556008)(6666004)(6916009)(2906002)(2616005)(66476007)(66946007)(956004)(6486002)(83380400001)(36756003)(33656002)(54906003)(478600001)(8936002)(4326008)(5660300002)(316002)(86362001)(52116002)(38100700002)(6506007)(8676002)(38350700002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6270
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f82f01f4-376b-40c8-a761-08d971f35f8b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XBpNM0Z5UT5rHlSeTQ2/HllZHPA93gNWn+c1caRyE9qykexBfZ0476UQB3aSt8HCMBVARqqB/3ODpKX+hRgau6qNoSB/X8RgbvSbNvgskJMbD3YSfIA6I7GCyp08YEX8J6lm3JYFuk0t1Yk7ZKFLD9I4WR2og+kvrj6tSCUsrq6IztQ+6hN6bemuKpCogtrbzMYxrptKPGSxky3YRu+lBRnlrKSWlR7zTyORHKQvuAC+VCoHrdHah6QgFLjYTn9SYCTPd1GyanYegyXWzQQCpC48JABjFJ7QOe8Bga9qkjThHmI5JaX80Q4aLIYgA2/24ID+jiHA+8rld1i24+OAInhqEH6LHLB4FhqSfBKMPWJkJDNwWBjd8zXd7Fv4S2G96nlcumF2QlcvH5hjodzZWTO4sMWGd7B1e/FjUv4Gt0WfPYlsoGtXGHh5SC0QgoLurfSYRgcxjkklVQs4BzmsRjtCk03ptJeCqprf3ZODxbBPAUociOPmVbPMmYqaMgc78sGrZZBRfwH0EYOcS5c+DqJq63xjnFoBGtmZRayYbLF/wOF0jngk4sHhS4HPw/A1JT3Z7/odKPcBEmHHry/AfX9zv5ti+SCSC8+IrB7UndlvEe8MU8tPYwWk3faAi7pOexyIdvaLw39JMpUuHj1dqXn8mbXhaEpd2D/DE51z26k93Y+fKXrUCFBJzhU7Apwdn/EpDbQzcafOmWL1lL4rcoGxigjAdWTTmw7OWeBZhyc=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2616005)(6666004)(5660300002)(8676002)(54906003)(83380400001)(44832011)(6512007)(956004)(70206006)(70586007)(26005)(6506007)(316002)(33656002)(53546011)(6486002)(36756003)(8936002)(508600001)(336012)(4326008)(356005)(2906002)(186003)(82310400003)(6862004)(86362001)(36860700001)(47076005)(81166007)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 11:34:07.3144
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 59b135d2-57d4-42c1-0948-08d971f36755
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5775



> On 7 Sep 2021, at 10:24, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 07.09.2021 11:17, Julien Grall wrote:
>> On 07/09/2021 09:33, Jan Beulich wrote:
>>> I'd like to suggest a different scheme, not the least because I expect
>>> the individual domains being independent of e.g. hypervisor command
>>> line options or Dom0 kernel versions. Yet varying sets of these are,
>>> for example, a reason to have multiple sections in the current scheme.
>>> Every dom0less guest would then require spelling out in every such
>>> section. Hence I think we'd be better off having a section per guest:
>>>=20
>>> [guest1]
>>> kernel=3DImage-domu1.bin console=3DttyAMA0 root=3D/dev/ram0 rw
>>> property=3Dcpus=3D1
>>> property=3Dmemory=3D0xC0000
>>> dtb=3Ddomu.dtb
>>=20
>> I much prefer the idea of the section. This is going to be easier to=20
>> parse the configuration file as we would not have to look for "domuX_"=20
>> and then distinguishing X.
>>=20
>>>=20
>>> These sections would then be referenced by other sections, e.g. by a
>>> new "guests" (or "domus", but this ends up looking a little odd for
>>> its matching of an unrelated latin word) keyword:
>>>=20
>>> guests=3Dguest1,guest2
>>>=20
>>> If it is deemed necessary to make sure such a section can't be
>>> (mistakenly) used to create Dom0, such sections would need identifying
>>> in some way. Presence of property=3D (or, as per below, properties=3D)
>>> could be one means (allowing an empty setting would then be desirable).
>>=20
>> I would expect dom0 to be described in the similar fashion at some=20
>> point. So maybe we should name the property "domains=3D...".
>=20
> Not sure - the order above doesn't mandate domain IDs, yet Dom0 needs
> creating with ID 0. IOW I was deliberately suggesting "guests=3D".

Hi Jan, Julien,

Thanks for your feedbacks, really appreciated, I agree with you that sectio=
ns
are easier to read.
I stand with the =E2=80=9Cguests=3D=E2=80=9C or =E2=80=9Cdomus=3D=E2=80=9C =
or =E2=80=9Cdomu_guests=3D=E2=80=9C keywords where
the references to =E2=80=9Cguest sections=E2=80=9D can be listed, separated=
 by commas.=20

>=20
>>> As to the properties, is there anything wrong with having them all on
>>> one line:
>>>=20
>>> [guest1]
>>> kernel=3DImage-domu1.bin console=3DttyAMA0 root=3D/dev/ram0 rw
>>> dtb=3Ddomu.dtb
>>> properties=3Dcpus=3D1 memory=3D0xC0000
>>=20
>> It depends on the number of properties for the domain, this may become=20
>> quickly unreadable.
>>=20
>> But... if we use sections, then I think it would be better to have:
>>=20
>> kernel=3D..
>> dtb=3D...
>> cpu=3D1
>> memory=3D0xC0000
>>=20
>> This would also allow us to create more complex setup (such as for the=20
>> static memory allocation).
>=20
> If that's feasible parsing-wise - sure. I was first thinking to suggest
> separate keywords, but then decided there was a reason this wasn't done
> in the original proposal (with respective dom#_ prefixes).

Yes I think it is feasible and easy to read, so given your feedback we can =
agree on
something like that:

[global]
default=3Dxen

[xen]
options=3D[xen command line]
kernel=3D[dom0 kernel file] [domain 0 command line options]
ramdisk=3D[dom0 ramdisk]
xsm=3D[xsm filename]
dtb=3D[xen device tree blob]
domu_guests=3Dguest1,...

[guest1]
address-cells=3D2
size-cells=3D2
kernel=3D[domU kernel file] [domU command line options]
cpus=3D1
memory=3D0xC0000
dtb=3D[domU passthrough dtb]

This won=E2=80=99t break the compatibility because on older version the dom=
u_guests keyword
Is ignored, the user needs just to check that a guest section is not specif=
ied as default=3D
argument because in this case it will be loaded as dom0 kernel (and probabl=
y won=E2=80=99t work),
but I think we can just update the documentation to highlight this thing.

Finally this example should load a dom0less system:

[global]
default=3Dxen

[xen]
# Xen boot arguments
options=3Dnoreboot console=3Ddtuart dtuart=3Dserial0 bootscrub=3D0
# Xen device tree
dtb=3Ddevtree.dtb
# domus to be started
domu_guests=3Dguest1,guest2

[guest1]
address-cells=3D2
size-cells=3D2
kernel=3DImage-domu1.bin console=3DttyAMA0 root=3D/dev/ram0 rw
cpus=3D1
memory=3D0xC0000
dtb=3Ddomu.dtb

[guest2]
kernel=3DImage-domu2.bin console=3DttyAMA0 root=3D/dev/ram0 rw
cpus=3D2
memory=3D0x100000
vpl011


>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:49:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:49:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180856.327698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZbs-0006C5-Px; Tue, 07 Sep 2021 11:49:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180856.327698; Tue, 07 Sep 2021 11:49:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZbs-0006By-MI; Tue, 07 Sep 2021 11:49:52 +0000
Received: by outflank-mailman (input) for mailman id 180856;
 Tue, 07 Sep 2021 11:49:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZbr-0006Bc-6b
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 11:49:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1e9c2f4d-eb57-4274-a466-f4335d13ce44;
 Tue, 07 Sep 2021 11:49:50 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2053.outbound.protection.outlook.com [104.47.9.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-0EYdDkRHO7izB27Zj_Z-RA-1; Tue, 07 Sep 2021 13:49:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5325.eurprd04.prod.outlook.com (2603:10a6:803:60::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 11:49:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 11:49:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0501CA0053.eurprd05.prod.outlook.com (2603:10a6:200:68::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 7 Sep 2021 11:49:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e9c2f4d-eb57-4274-a466-f4335d13ce44
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631015389;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uHenRRTyf45+WJBF15XEKH7XjRi3lSRhsGAvltq2EWE=;
	b=AzoDJmxPKIgCko/Hq/KjLR9Vz4ZY3+jt/+I5yuHD8h3aEGCWnJKzA3/AhbqH3tJbR+ftQn
	UW/t6G7RA5Qnj2+ZTqyu8ACNzUKwvaVGFgOcV8b5GIdCxGFSOImJTGaZu97lSwFlzd9bSC
	BTIqlbFKd7XM2saWiAt4CraXKf8b8Gw=
X-MC-Unique: 0EYdDkRHO7izB27Zj_Z-RA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iYj0HIVfm48U+c49s1gTcyPG6K8u+GX8M50bbRawIaTf3nexRPn/W4zGLG0+a5viCkE8g7OZjoQFbGLT5Ez7SVGVT3afJVpJBKKUJvNUv22h+M8pNrzj50Veu5vU1ALhkMX9UZ6cIA0qDNqxI+RDdwto/wOnSxIKK97Uj6uoS+NF+Ht/JEEpd5o1H0OktqzDQywVKV8qsR/LZx4HCtpMREsCvBsq7x382seFYgXLMW3oP9/gKWR9r7rpUR/fn5IeuS7qkkzykUXcqF7eoHYRrWsLFUbUPj0L1mpT+da67yIpTL9AEDGPjq1P5yVuqKRs8CNbOarSI6Yu2RGhSm8kbw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uHenRRTyf45+WJBF15XEKH7XjRi3lSRhsGAvltq2EWE=;
 b=Er8VV61mSivAevJPngEKURkhnbMXPKJeZ955bhojAnW7QPjH1QfMOV8Tkyy1E0IgfGYE9zjct8FQXwHMjtGnEdE1orhv3S7Cyq3/cJR06BITXXRZoV5aTwuLmpuN0rnN4dOKI+qbGA0Va0Wd+SsG1ytJOQoRy6VBSwiYTwfetHWW3G82N1yuS+jMzCrWQ4Zf7zpowk/rakaNXFrpkGqF8m+LOTTQvXv+R2QmgliAE1VPGSAhQsqAn9z9T0x/Ufa22iqu+jb9B0fFqrTl3uv5/ypABNgRr0LvqyRCOi7xzw/N4O67e5esQwHnPTulasZe9oFy/CqtaV6EcPZAYrr6SQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
 <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
 <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <938156e7-cbd0-a19e-f09f-a41dce051367@suse.com>
Date: Tue, 7 Sep 2021 13:49:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0501CA0053.eurprd05.prod.outlook.com
 (2603:10a6:200:68::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d46988d-393e-466e-86a2-08d971f59657
X-MS-TrafficTypeDiagnostic: VI1PR04MB5325:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5325C7215AA53AD7B56DA9B6B3D39@VI1PR04MB5325.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YSB3lk8s/QoJO04og4CPCeAHRQLHRS4Qlr64DhwDaDZjbkzEJ+WmdqPGnmS/57/VbXzklXyjlEr5LvFqBEfmJ0bzwBTBNfx6RCp5R7GdyXEXVwCe1rIdA0mJMdUUS/cWz9W9rtcSo6EkvQr5qBjwmhiUFhKg+jZGzhpek5rwVTzVxrrOWcxypcqDByWDt/IgWlPHOGqI/XFVS9YDlbX6emnCD/FSKd7xCXkWPWlagZS+0TGOsSODo+Ea6ajHBXWp3nyiZ1egOQCOM+yMzljnXVfXYK1i6/PxwjmaevIq5lInEmYt7kFg61fxPj+T2uu3Xx00JyOM/yuzAXFHshezHigwOxZZN2bCNji2nhEucTELEWELTfx/4uVtrQE+ICtf4RmKDg9+9fOzrnzRglKJGxfA9iDrCnq28t+clDkLp783cZkHXqWRQwnZ1ZGCDK72rEL37F6Jq3gXJb2TgktSqzZ8aUBA1xg0HDgH1nOukv736yvo/cnlueGVpc5KFhMNKp7E/t+7qF4oXvRyoOLvW18EKehIT7LX2prtPdABnVtD2sQ2p8IRPtCK6nb+b95RxkE5BQQ5PBniEAsvM+hsgCVHFGi2rKNKZO2GbyUGI3sq6hTzX8syuiRWIv2yg6bT2uB7AZ6Xw3FiaDal7CB6rIGWMBInj13HqXkGATOd8blM5xMY/X7/cWxh39wRJua1a3zKvT9UxbqSFmRYb6N8i64jBRuDm6Ezx+WUBOcdvouxT14ajTY/k58iAhHFC6xE
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(39860400002)(136003)(366004)(376002)(316002)(2906002)(8936002)(110136005)(186003)(16576012)(31686004)(31696002)(478600001)(956004)(2616005)(5660300002)(4326008)(36756003)(86362001)(6486002)(38100700002)(8676002)(53546011)(83380400001)(26005)(54906003)(7416002)(66946007)(66476007)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YVlvZExCeTd5eFhsODhxdnFUWEF6M2ZrVitrOU1pZGVlY0hlVnFPdC9sQVVC?=
 =?utf-8?B?VEx6eG5vcytIYUVlN21lZVplWFNmc0lJYXpkb0Jwd0diVXJJMVBRRGR4N1ZB?=
 =?utf-8?B?SS9SR1Z0WjBUVHdrNG95YUxqVmtuZEx2RXVFZURFQjR5LzBaamNaQnM4VUhW?=
 =?utf-8?B?bHFDYjNhKzNtZld6anZVYXpMWkdlNnRUeXY3LzJISGxlekprOWo0NXR0ZjNG?=
 =?utf-8?B?QVkxVUx5L3FaYTBuWUwwMDNZWHc4R2FDRG96TkhRQnpJZkRIWUFJVkI0eitM?=
 =?utf-8?B?S01QQ1NXV29KMmcxSlBRWnBMeDZmL1NvdW5yQUtWUXBITVNLUWJ0dWEveXl2?=
 =?utf-8?B?NTRlMzg5eHBtd2E3WExCeFdTNzgxOWRTWEFDWFczYWxKbEtPOXRXVldYMVdv?=
 =?utf-8?B?cHBHOFZhN2JNV1R5ZmRPL1JLUGFScUJZc3FwRnJkL3pxWVA5WjNKd20xLzlF?=
 =?utf-8?B?ak9UZGN5S29KemVBKzVSRVpoYUtDY0Y5bjB1ZzlJdm1PaUhaL3FhZnVKTEk4?=
 =?utf-8?B?RGxnM0FIS201RGZBSUZhWWpaYWRQcnJqajJUUFVwcFNUZ01ldVYySW5QZFZB?=
 =?utf-8?B?RjdLZ3NPYUJTemRLU3c0T3MwemJGK1RJaXJCRDBDemFPZC9kWWEzdU1RTWJ6?=
 =?utf-8?B?UjdNdGVXYk54MzcyZndDcHVGWlJiZzk0eE9KSTJIdXVkZy9VTmQvdTU3cmE3?=
 =?utf-8?B?WUxvRmdadUFqQjdxU3YvbTl4a0hPb1cyVlN3VGdFRSt4Ny9EdlFrSDNIa0tL?=
 =?utf-8?B?emp1UEdZQW9nV3hBMVJTT3lLNk9rcnd4V1VHYzExcmplWUFGc1JEQXlaV1Fu?=
 =?utf-8?B?Ry95ZHZNQm93dlBlSXRMRmRqd2RmRStreU4vRkZyYkZaTDlIZldqekhCTENZ?=
 =?utf-8?B?MEI3OVNvRm4rY0JueERHek5PL1R1c0J2dTl6VnlBUjRzZ0lubFpzeDlYY1Bt?=
 =?utf-8?B?QXRyeEVwcXYwUGFjQlU3QzN6RGRUVGlIQnkxaXNZdkQ5dzRZQ0FqUUlTNzJX?=
 =?utf-8?B?NE56WlFldWgvRzdyNVF1Q0xKNU8yN3drV2RGZE5ZZ3RVZUxxVDJrSzdLRXRJ?=
 =?utf-8?B?MVhOR2F0Y2I1U0ZyQmhWRjN0d2w5TGxHYy9pOXY2SU5lN3dvTU5DSmtYUWpF?=
 =?utf-8?B?UjRQdUpCNlY3d3FVOXVVRXNQSm5Nbk5jMTJTc3ZPa2RKTXpxYUY4anM1U0Ru?=
 =?utf-8?B?VzY3RkRaZHVxT1A0dmQyUmkrSDRDYXhKTndLTzRVaTlSdlZDSi8xcURvQzRQ?=
 =?utf-8?B?UnpYMzBEaHpMaHJJMkVzNC9zUG5NdW1ObVB0TjJjaG5LdU5NTFR0UHQ4SGha?=
 =?utf-8?B?aTZ6UXNpdWhXaFlmcG5QbEl3ZTB2dDYvUWlvUnoxVkhPeHFYNE12eUJldDJM?=
 =?utf-8?B?azZHTWtnK2ZtSGNMTHNlWlAvNnJrUGk1YjVJbENyc2t4ZFpOWUw1eW1acEU5?=
 =?utf-8?B?ZEowcVo4LzRQaXk5MTJoVkgrZFVTMFIvQ0poODdMUTQwL1o0SENydXVCemo1?=
 =?utf-8?B?UW41Z0tsVytodmxHMlM0ZEdYUmNrUVorQjRIcDZEeS9DVnh4QnYwdGsydTJJ?=
 =?utf-8?B?N3NhK1JoMWFDUXUvYkhhbzYzb2JYNkZmdWZxUUpFNUplNnA3bTNsZU83TUVB?=
 =?utf-8?B?V0EvMmNQdEYxeGo2UkZZUGg5ZnlQTFFxbURYeTRwWjN4N0cxazJCd0lNTkRO?=
 =?utf-8?B?SzRMdVFwWXFoT0JLbnZlQ1NkWUFQSU1zZ1VsbklwdVhzS09ZU3Z6cFM3Smdu?=
 =?utf-8?Q?YJzdIFdFfnpKrxaa12Xo3FxnPR828PYMhI5jxm5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d46988d-393e-466e-86a2-08d971f59657
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 11:49:45.4129
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gkQZB7Wq/IXVB1IMrFP9tmvPahQMIIEcr5pWYIEDWIk6PQ5hCHsAnThhHI7M/vedM9B0V0wEZrlCcJiMaRjhxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5325

On 07.09.2021 13:10, Oleksandr Andrushchenko wrote:
> 
> On 07.09.21 13:43, Jan Beulich wrote:
>> On 07.09.2021 12:11, Oleksandr Andrushchenko wrote:
>>> On 06.09.21 17:11, Jan Beulich wrote:
>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>> @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
>>>>>    }
>>>>>    REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
>>>>>    
>>>>> +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
>>>>> +{
>>>>> +    int rc;
>>>>> +
>>>>> +    /* Remove previously added registers. */
>>>>> +    vpci_remove_device_registers(pdev);
>>>>> +
>>>>> +    /* It only makes sense to add registers for hwdom or guest domain. */
>>>>> +    if ( d->domain_id >= DOMID_FIRST_RESERVED )
>>>>> +        return 0;
>>>>> +
>>>>> +    if ( is_hardware_domain(d) )
>>>>> +        rc = add_bar_handlers(pdev, true);
>>>>> +    else
>>>>> +        rc = add_bar_handlers(pdev, false);
>>>>       rc = add_bar_handlers(pdev, is_hardware_domain(d));
>>> Indeed, thank you ;)
>>>>> +    if ( rc )
>>>>> +        gprintk(XENLOG_ERR,
>>>>> +            "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
>>>>> +            d->domain_id);
>>>> Please use %pd and correct indentation. Logging the error code might
>>>> also help some in diagnosing issues.
>>> Sure, I'll change it to %pd
>>>>    Further I'm not sure this is a
>>>> message we want in release builds
>>> Why not?
>> Excess verbosity: If we have such here, why not elsewhere on error paths?
>> And I hope you agree things will get too verbose if we had such (about)
>> everywhere?
> Agree, will change it to gdprintk
>>
>>>>    - perhaps gdprintk()?
>>> I'll change if we decide so
>>>>> +    return rc;
>>>>> +}
>>>>> +
>>>>> +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev)
>>>>> +{
>>>>> +    /* Remove previously added registers. */
>>>>> +    vpci_remove_device_registers(pdev);
>>>>> +    return 0;
>>>>> +}
>>>> Also - in how far is the goal of your work to also make vPCI work for
>>>> x86 DomU-s? If that's not a goal
>>> It is not, unfortunately. The goal is not to break x86 and to enable Arm
>>>> , I'd like to ask that you limit the
>>>> introduction of code that ends up dead there.
>>> What's wrong with this function even if it is a one-liner?
>> The comment is primarily on the earlier function, and then extends to
>> this one.
>>
>>> This way we have a pair vpci_bar_add_handlers/vpci_bar_remove_handlers
>>> and if I understood correctly you suggest vpci_bar_add_handlers/vpci_remove_device_registers?
>>> What would we gain from that, but yet another secret knowledge that in order
>>> to remove BAR handlers one needs to call vpci_remove_device_registers
>>> while I would personally expect to call vpci_bar_add_handlers' counterpart,
>>> vpci_remove_device_registers namely.
>> This is all fine. Yet vpci_bar_{add,remove}_handlers() will, aiui, be
>> dead code on x86.
> vpci_bar_add_handlers will be used by x86 PVH Dom0

Where / when? You add a call from vpci_assign_device(), but besides that
also being dead code on x86 (for now), you can't mean that because
vpci_deassign_device() also calls vpci_bar_remove_handlers().

>>   Hence there should be an arrangement allowing the
>> compiler to eliminate this dead code.
> 
> So, the only dead code for x86 here will be vpci_bar_remove_handlers. Yet.
> Because I hope x86 to gain guest support for PVH Dom0 sooner or later.
> 
>>   Whether that's enclosing these
>> by "#ifdef" or adding early "if(!IS_ENABLED(CONFIG_*))" is secondary.
>> This has a knock-on effect on other functions as you certainly realize:
>> The compiler seeing e.g. the 2nd argument to the add-BARs function
>> always being true allows it to instantiate just a clone of the original
>> function with the respective conditionals removed.
> 
> With the above (e.g. add is going to be used, but not remove) do you
> think it is worth playing with ifdef's to strip that single function and add
> a piece of spaghetti code to save a bit?

No, that I agree wouldn't be worth it.

> What would that ifdef look like,
> e.g. #ifdef CONFIG_ARM or #ifndef CONFIG_X86 && any other platform, but ARM?

A new setting, preferably; e.g. VCPU_UNPRIVILEGED, to be "select"ed by
architectures as functionality gets enabled.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:51:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:51:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180862.327709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZd1-0007Tj-3j; Tue, 07 Sep 2021 11:51:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180862.327709; Tue, 07 Sep 2021 11:51:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZd1-0007Tc-0p; Tue, 07 Sep 2021 11:51:03 +0000
Received: by outflank-mailman (input) for mailman id 180862;
 Tue, 07 Sep 2021 11:51:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNZcz-0007TG-GR; Tue, 07 Sep 2021 11:51:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNZcz-00036f-CE; Tue, 07 Sep 2021 11:51:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNZcz-0000o5-4S; Tue, 07 Sep 2021 11:51:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNZcz-0001nn-3v; Tue, 07 Sep 2021 11:51:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/mVGIVAAw7cJ3RmytizdZk2OG7wBCRYpWpE1VVNhkp8=; b=cv+VuZy/xo8erch8TpXaUgozsJ
	RRToRrNHrHuB0BFyu8eBkbH+IINVyM9+aneNUnMxu1T20dnO88i458KLv1BfU9tL2PsaHWiSOmGdC
	5lPf3T8p1A4b5caDIBmHa2Tpr+b69w/jqrBGeFxrhQmE9bbeyS21wzImVOgyJSS8M7z4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164872-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164872: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=5ae4120635ad3cbf6064268162b0fb2c44eb41e3
X-Osstest-Versions-That:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 11:51:01 +0000

flight 164872 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164872/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  5ae4120635ad3cbf6064268162b0fb2c44eb41e3
baseline version:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17

Last test of basis   164863  2021-09-06 18:02:44 Z    0 days
Testing same since   164872  2021-09-07 08:01:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>
  Wei Liu <wl@xen.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6c27a8da8d..5ae4120635  5ae4120635ad3cbf6064268162b0fb2c44eb41e3 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:51:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:51:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180869.327723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZdc-0008L9-FV; Tue, 07 Sep 2021 11:51:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180869.327723; Tue, 07 Sep 2021 11:51:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZdc-0008L2-Bt; Tue, 07 Sep 2021 11:51:40 +0000
Received: by outflank-mailman (input) for mailman id 180869;
 Tue, 07 Sep 2021 11:51:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qx60=N5=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNZdb-0008Kd-66
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 11:51:39 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.61]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f2959572-0fd1-11ec-b0ea-12813bfff9fa;
 Tue, 07 Sep 2021 11:51:33 +0000 (UTC)
Received: from PR0P264CA0251.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::23) by
 DB7PR08MB3676.eurprd08.prod.outlook.com (2603:10a6:10:4d::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.21; Tue, 7 Sep 2021 11:51:31 +0000
Received: from VE1EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:0:cafe::35) by PR0P264CA0251.outlook.office365.com
 (2603:10a6:100::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Tue, 7 Sep 2021 11:51:31 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT055.mail.protection.outlook.com (10.152.19.158) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 11:51:30 +0000
Received: ("Tessian outbound cc1d52552731:v105");
 Tue, 07 Sep 2021 11:51:30 +0000
Received: from b8a882c627c8.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 79EECCEF-83AE-41BC-B5C3-3921D7D635D9.1; 
 Tue, 07 Sep 2021 11:51:24 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b8a882c627c8.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 11:51:24 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6495.eurprd08.prod.outlook.com (2603:10a6:102:dd::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 11:51:23 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 11:51:23 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0360.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:d::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 11:51:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2959572-0fd1-11ec-b0ea-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YXbgVi4E/l2ZD/q5CGFs38xQTqeIroaeLOOfdmS8cYs=;
 b=7WdKlKsnRA7wl/zP/nDSoYJr3eIvmQ39bTIpktYzCbPwSPj5AHZFkvnxFCZMFxQTVph8ziZNLsujmsrdOz4cxe6kX8hBzoui3BEWfTdUIt2/iCKth4s7ZF2kLT0c6IlPxzTaQVdR8MPwRPjyGucGu4/xqdnLTM8/dkltyooXx08=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 1291435db9ea3fac
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CEtMWMnpKSDwS1L7TybOURvfYta4G4YJGWTL8GuCXdsaZaCMfj8l39w8bKGVmbfGybAy/y4h1qKK875ion+qsFOddbmcV/VghnW6YWSCaeENWE0jWhM5GZEhmChK6q2M1Q9Ls36d2UNICb5M3bHMhYaZtrRqmlaGHx5DPqrJXDDznhdJ6Qis03fYzDM00CwbdsXHucukIle6vogzxnsRQ1GWv7SDDVb3kvtr/yrS9Af4TdxMHxWfTCAs8oRRiRACpMYb/4+aZ7tValP3itIm3CHbqVVGb3lo0pIbONos8OO3z3TWgXWHhczCKQTewHUswsquhhzadhawbdJQD3Aipw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YXbgVi4E/l2ZD/q5CGFs38xQTqeIroaeLOOfdmS8cYs=;
 b=HNwMuNHGBr4f2073O/9E13JP+WrZwJgKCHrkjKlETechZbtL+obddVK04Scn8L/bCYcYUah6w0zXUXZtB8PlDHaXJ3B0RyaMMBmHKGNTQoagtxZEwW1rhDboqaakSVZW82Z1vVs0K1jyj0OtffJmHAeMVipSynGjr/5RMbH2+KQ0jCDDKtHaPmmj6HbYpm6nohjswyM5R2QLNTc7Zc1xfSlO+i863XUhChMM7EkkeLfZbfK+hLR/0eMj0WNYUQuGqSqbs4TaPNeP1W6C1ojNr0s+iGikPi0QOyKGJFi7TPG73RAnZNqW9n73Q7aHLGYaS7cuwmmuCB3NF6dbxdPKzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YXbgVi4E/l2ZD/q5CGFs38xQTqeIroaeLOOfdmS8cYs=;
 b=7WdKlKsnRA7wl/zP/nDSoYJr3eIvmQ39bTIpktYzCbPwSPj5AHZFkvnxFCZMFxQTVph8ziZNLsujmsrdOz4cxe6kX8hBzoui3BEWfTdUIt2/iCKth4s7ZF2kLT0c6IlPxzTaQVdR8MPwRPjyGucGu4/xqdnLTM8/dkltyooXx08=
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
Date: Tue, 7 Sep 2021 12:51:16 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
To: Julien Grall <julien@xen.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0360.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:d::36) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d0f3c566-314d-4f1e-0341-08d971f5d551
X-MS-TrafficTypeDiagnostic: PAXPR08MB6495:|DB7PR08MB3676:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB367639A2574DEBEFE656E296E4D39@DB7PR08MB3676.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 B6pcdUoi3zrAbyZl01n4VcVjdoctCxnIA6cQDfPj3nmQZiYC0JayfC9WERc1H7XAE14/vN1rL7db54c2TdKhxLbzVKYgBtNE7aY0RYgsNHP2yYOIgPEqYqc2KHbe0wCT84o00tHi2qvpHl6CgBhTGPpNZ1Qp+WfzCZUcYalnAJsOQzEtfDcgb7E/93krsrDHpyb4FRDRgs6ZtPK9LbT1uI5EpYmhNT5IYg6M8pmheYmdreVRV6RWuGzKecTIYdppvUtGWLdbJWOVtkf7+Xwn/C7N3wtEMWqQJRhnqz0lVmpssBm896rfhI6dIWd5z543DlgCFqtuf6qeCSIiZW/Iv66X6xyZWp1xOx4x+WMF65Bo3ATuntDM4yNIytPAD5nkzKzP+YWJWuQ097ZAOxaHTn6yKUQwVPvvLHYtHqNtqC7Y7aNX0YMqM773T/BYktzCxTfgcT9YHILhhn1WrTrysK6XL9+ulQ5+tq8nh4F3Dfr68vBLeq+9BrMbwL+E3t8SrHphtoyQQKCODgoEQGZcGEKpNhsYbkrxh4Jg5UJ3RVQ8RXVAp0UBGbaHLF+l5RSNBbbRML1E8pYolC720y3cbCeiAcVlPT+PfNQT8NsWp23/KczmK/i7/ZCxNUHuUCdPzgwZRJjoqGLCCk9DQYXc3IElaa0K5PztB+w4/hYXZccqzkophR2F+LL/G8WsTmh8Yzj5bwET32Ubjl1OmZA7PaOY9xclSNLeHDtP16i04iGzOIRnFok/qCUNajP3XeflD/c01W0UoWrotUL65p8gDg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(376002)(39860400002)(136003)(396003)(6666004)(26005)(186003)(6916009)(478600001)(36756003)(86362001)(6512007)(4326008)(33656002)(2616005)(956004)(5660300002)(316002)(54906003)(44832011)(2906002)(8676002)(38100700002)(38350700002)(66556008)(66476007)(8936002)(66946007)(53546011)(6506007)(52116002)(6486002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6495
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT055.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	88256cac-c7e1-4d16-4b29-08d971f5d03a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	08Q23tub5dfGLpkAAd8Lc+DPNN1iVXOdKI+u1nZ/oAk9kVuPVRVTBUQFU+Pseklt42FpWGE8Q2yya+tOPIKnKuti3n3h6Af2YMl9n3UMlNMz+Ym4E+M7KiBK3HRGqM6/ULuq4OpRqR6lqLrpXKsfN1wv//uAgpSCyEMwi9M2x05X+MQeFXVeYScHSaPPBE7729M3xkc5/T4RznvXkbmk8ryAyoGek5qrkb7pW6IbCeJTAKCaIm2dSt5ylUDOrD5hD78KU58yVwjm76okDPv2iapdoiUwBlqzXUnkQo9qJWL0avPYKW8t10qw/UgwQA5+QERNifOyoXQyQoYDTl68owIr8P/35Cffflj4NaCj7GzMtnUhIPMW5sPrqV7iCZ3GBzQ2PffMZAAuYMNyzV4xTRvaTm2x2tcatr5tfSxHFby8QFDfVj20GZsAoEuAgeEbK48QtPRi+cfuVblhWoH+YbYdZDQBJhPyLXztHzNLAufKPBnsf2KvwYMXfGfGfIZMFDAXfVhLqRwpM7RNFWvRum2t/yW6LO0146KxMzsJ+7j3p0R7CHctZ6A3OssgjvY2DOQOT/89EzYdfVfxURXvhBf8kwsP8ek8eLpI5FZhyt+fn1Pu3pApvJVhhYc9eD6xigiknftFOcBr1gLtn3rYSbVMFtuMwIXIWa2x9Joyniob2hSAdjF3AYWbFprqXv9cj4o4iuXxlUG2QCzydD0AGlFxzzdpXNaGkHY1obBNz30=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(346002)(136003)(39850400004)(396003)(46966006)(36840700001)(44832011)(956004)(478600001)(336012)(6512007)(33656002)(2616005)(6862004)(8676002)(186003)(6486002)(70586007)(70206006)(82310400003)(26005)(6666004)(53546011)(82740400003)(47076005)(6506007)(86362001)(36756003)(4326008)(54906003)(2906002)(356005)(81166007)(36860700001)(8936002)(5660300002)(316002)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 11:51:30.8328
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d0f3c566-314d-4f1e-0341-08d971f5d551
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT055.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3676



> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Luca,
>=20
> On 07/09/2021 07:52, Luca Fancellu wrote:
>> Add a design describing a proposal to improve the EFI
>> configuration file, adding keywords to describe domU
>> guests and allowing to start a dom0less system.
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>> ---
>>  docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>>  1 file changed, 105 insertions(+)
>>  create mode 100644 docs/designs/efi-arm-dom0less.md
>> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom=
0less.md
>> new file mode 100644
>> index 0000000000..8d8fa2243f
>> --- /dev/null
>> +++ b/docs/designs/efi-arm-dom0less.md
>> @@ -0,0 +1,105 @@
>> +# Xen EFI configuration file
>> +
>> +The current configuration file used by Xen when it is started as an EFI
>> +application is considering only the dom0 guest and doesn't have any
>> +property to describe and load in memory domU guests.
>=20
> From my understanding, the problem is less about properties (we already h=
ave them in the Device-Tree) but more about where are the binaries located =
in memory as we don't know in advance.

Hi Julien,

I think I used the wrong word there, I meant =E2=80=9Ckeyword=E2=80=9D inst=
ead of =E2=80=9Cproperty=E2=80=9D because I was referring about the
lack of keywords to describe a domu guest in the Xen EFI configuration file=
.

I agree with you that on systems with static allocation, the kernel and ram=
disk binaries must be at certain locations
that are out of control when we use the EFI boot services, the thing we can=
 do is provide a keyword to specify the
addresses and then use the CopyMem() function to relocate the kernel/ramdis=
k in the address we want.

>=20
> So I would like to propose something that build on top of the Device-Tree=
 work we did. Note this is early thoughts.
>=20
> The problematic nodes in the DT are:
>=20
>        module@0x4a000000 {
>            compatible =3D "multiboot,kernel", "multiboot,module";
>            reg =3D <0x0 0x4a000000 0xffffff>;
>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>        };
>=20
>        module@0x4b000000 {
>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>            reg =3D <0x0 0x4b000000 0xffffff>;
>        };
>=20
> In particular the property "reg" cannot be known in advance because the U=
EFI stub will be responsible to load the binaries in memory.

Yes that=E2=80=99s true, the UEFI stub is using from the UEFI boot service =
the AllocatePages function that is giving back an address out of our contro=
l,
then using another function the binary is read from the disk and copied at =
that address, finally the UEFI stub is writing the node in the device tree =
that
will be used by Xen later.

>=20
> What we could do is providing a list of binaries to load and associate a =
key for each of them. Something like:
>=20
> binary=3D<binary> <key>
> binary=3D<binary2> <key2>
> ....
>=20
> We can then replace the property "reg" with a new property "uefi,key" tha=
t will contain the name of the binary.
>=20
> What do you think?

Here I=E2=80=99m lost, because I don=E2=80=99t understand what we are going=
 to do with the name of the binary.

Cheers,

Luca

>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:55:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:55:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180876.327733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZh3-0000hT-VV; Tue, 07 Sep 2021 11:55:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180876.327733; Tue, 07 Sep 2021 11:55:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZh3-0000hM-Se; Tue, 07 Sep 2021 11:55:13 +0000
Received: by outflank-mailman (input) for mailman id 180876;
 Tue, 07 Sep 2021 11:55:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fKkb=N5=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mNZh3-0000hG-6G
 for xen-devel@lists.xen.org; Tue, 07 Sep 2021 11:55:13 +0000
Received: from mail-pg1-x52e.google.com (unknown [2607:f8b0:4864:20::52e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f96934d6-4c9f-4cf9-8d9a-8c1e3d079d13;
 Tue, 07 Sep 2021 11:55:11 +0000 (UTC)
Received: by mail-pg1-x52e.google.com with SMTP id 17so9680300pgp.4
 for <xen-devel@lists.xen.org>; Tue, 07 Sep 2021 04:55:11 -0700 (PDT)
Received: from laputa (p784a2304.tkyea130.ap.so-net.ne.jp. [120.74.35.4])
 by smtp.gmail.com with ESMTPSA id 73sm11108630pfu.92.2021.09.07.04.55.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 07 Sep 2021 04:55:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f96934d6-4c9f-4cf9-8d9a-8c1e3d079d13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=YGmEUWYy+3Y27wr4j+eiPDJQoM70U3zAZRfNASz1Nxs=;
        b=sixtt0LE5+rubYhgxQ3gxIafb8BStveV0zCkKP/TTVFMc+wR2FvKIgcx5BMnbVcgkF
         +kYmg5CpMon18Fyk1Uvz6jjVf0ydAFlhf12YPdTboaCmTCE84wSga+8TcQkeXAxPfBPB
         xgKEo0Dr5MSNm82QIbNhq6RVWIn7iSjrc46L6pkxQN22Qsu8GSaDDK5Tfl8v4LoOHOiL
         3UQZqfxnLQl9KaqXvwt5IF8QBycWnyMBVOmTEOC9sFa55uxxUXvjPFmiP2UYqrGaZ7ZV
         iTMwctPVRveEqOsMcSJVcZZNt4wJlzqkes0Nkzub72NOv+qqRhyJdjw9mNb4/TbP26LH
         SkkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=YGmEUWYy+3Y27wr4j+eiPDJQoM70U3zAZRfNASz1Nxs=;
        b=bMJ16mmDMPBBRs+ZpQiqVuYszyw04l6vpdMi32n/AOOI+nSgOHeZezbzRoHrVAbXsY
         R6Ms1nZKemnk2Jqr3R72ngo7o4N68jJGhehjsTW6wmor2vtiPPYEq9NQNQHXI9tRqD9F
         W9Ee5qdLQCdGT2Pu/gspOLPcnJjek2sYuMfJJAa2jYs+5AFuI+r1OFhy3Q+fQ+xQyHyo
         ilYXE7sLiszgU51k2uYMSKwjFF7WSnWUK8PdJue/EDY/PwKVijo+PsAp7AG0AnqPNEmP
         K0RZQaDEluvhGYDXQXR9VycZg1ctbZ+63Nah8/wVe9pyRWbST87/12vyihw/poJdyKjQ
         npJQ==
X-Gm-Message-State: AOAM531KtKE6zefo3NfgsCnXtxXsbLouxXUtr1unbibErXd/cgm/IeId
	wazeozlj79t3RkEhT1crV/lQsQ==
X-Google-Smtp-Source: ABdhPJz84h7MB9crcdLnGhrW7TXCZ3clzq/fXzbX64JcR9tazl28nxPVp/iHoLXGuJYQsyneDaM4pg==
X-Received: by 2002:a63:36c4:: with SMTP id d187mr12368353pga.205.1631015710887;
        Tue, 07 Sep 2021 04:55:10 -0700 (PDT)
Date: Tue, 7 Sep 2021 20:55:01 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: Wei Chen <Wei.Chen@arm.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin <Kaly.Xin@arm.com>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	"stefanha@redhat.com" <stefanha@redhat.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>,
	"pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
	Paul Durrant <paul@xen.org>, Xen Devel <xen-devel@lists.xen.org>,
	Rich Persaud <persaur@gmail.com>,
	Daniel Smith <dpsmith@apertussolutions.com>,
	James McKenzie <james@bromium.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210907115501.GC49004@laputa>
References: <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa>
 <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa>
 <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CACMJ4GbmNgbB5ponYt3NGEk3j6YCksot+kDy2qs8HMdFXWnQbw@mail.gmail.com>
 <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com>
 <20210902071902.GC71098@laputa>
 <CACMJ4GYY0wkm0vOzJfq8M9dCxPq1ZFwWT8Q73rMjqtrP7if-Zw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CACMJ4GYY0wkm0vOzJfq8M9dCxPq1ZFwWT8Q73rMjqtrP7if-Zw@mail.gmail.com>

Hi,

I have not covered all your comments below yet.
So just one comment:

On Mon, Sep 06, 2021 at 05:57:43PM -0700, Christopher Clark wrote:
> On Thu, Sep 2, 2021 at 12:19 AM AKASHI Takahiro <takahiro.akashi@linaro.org>
> wrote:

(snip)

> >    It appears that, on FE side, at least three hypervisor calls (and data
> >    copying) need to be invoked at every request, right?
> >
> 
> For a write, counting FE sendv ops:
> 1: the write data payload is sent via the "Argo ring for writes"
> 2: the descriptor is sent via a sync of the available/descriptor ring
>   -- is there a third one that I am missing?

In the picture, I can see
a) Data transmitted by Argo sendv
b) Descriptor written after data sendv
c) VirtIO ring sync'd to back-end via separate sendv

Oops, (b) is not a hypervisor call, is it?
(But I guess that you will have to have yet another call for notification
since there is no config register of QueueNotify?)

Thanks,
-Takahiro Akashi


> Christopher
> 
> 
> >
> > Thanks,
> > -Takahiro Akashi
> >
> >
> > > * Here are the design documents for building VirtIO-over-Argo, to
> > support a
> > >   hypervisor-agnostic frontend VirtIO transport driver using Argo.
> > >
> > > The Development Plan to build VirtIO virtual device support over Argo
> > > transport:
> > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> > >
> > > A design for using VirtIO over Argo, describing how VirtIO data
> > structures
> > > and communication is handled over the Argo transport:
> > > https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo
> > >
> > > Diagram (from the above document) showing how VirtIO rings are
> > synchronized
> > > between domains without using shared memory:
> > >
> > https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-blob-url=true&id=01f7d0e1-7686-4f0b-88e1-457c1d30df40&collection=contentId-1348763698&contextId=1348763698&mimeType=image%2Fpng&name=device-buffer-access-virtio-argo.png&size=243175&width=1106&height=1241
> > >
> > > Please note that the above design documents show that the existing VirtIO
> > > device drivers, and both vring and virtqueue data structures can be
> > > preserved
> > > while interdomain communication can be performed with no shared memory
> > > required
> > > for most drivers; (the exceptions where further design is required are
> > those
> > > such as virtual framebuffer devices where shared memory regions are
> > > intentionally
> > > added to the communication structure beyond the vrings and virtqueues).
> > >
> > > An analysis of VirtIO and Argo, informing the design:
> > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO
> > >
> > > * Argo can be used for a communication path for configuration between the
> > > backend
> > >   and the toolstack, avoiding the need for a dependency on XenStore,
> > which
> > > is an
> > >   advantage for any hypervisor-agnostic design. It is also amenable to a
> > > notification
> > >   mechanism that is not based on Xen event channels.
> > >
> > > * Argo does not use or require shared memory between VMs and provides an
> > > alternative
> > >   to the use of foreign shared memory mappings. It avoids some of the
> > > complexities
> > >   involved with using grants (eg. XSA-300).
> > >
> > > * Argo supports Mandatory Access Control by the hypervisor, satisfying a
> > > common
> > >   certification requirement.
> > >
> > > * The Argo headers are BSD-licensed and the Xen hypervisor implementation
> > > is GPLv2 but
> > >   accessible via the hypercall interface. The licensing should not
> > present
> > > an obstacle
> > >   to adoption of Argo in guest software or implementation by other
> > > hypervisors.
> > >
> > > * Since the interface that Argo presents to a guest VM is similar to
> > DMA, a
> > > VirtIO-Argo
> > >   frontend transport driver should be able to operate with a physical
> > > VirtIO-enabled
> > >   smart-NIC if the toolstack and an Argo-aware backend provide support.
> > >
> > > The next Xen Community Call is next week and I would be happy to answer
> > > questions
> > > about Argo and on this topic. I will also be following this thread.
> > >
> > > Christopher
> > > (Argo maintainer, Xen Community)
> > >
> > >
> > --------------------------------------------------------------------------------
> > > [1]
> > > An introduction to Argo:
> > >
> > https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> > > https://www.youtube.com/watch?v=cnC0Tg3jqJQ
> > > Xen Wiki page for Argo:
> > >
> > https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen
> > >
> > > [2]
> > > OpenXT Linux Argo driver and userspace library:
> > > https://github.com/openxt/linux-xen-argo
> > >
> > > Windows V4V at OpenXT wiki:
> > > https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> > > Windows v4v driver source:
> > > https://github.com/OpenXT/xc-windows/tree/master/xenv4v
> > >
> > > HP/Bromium uXen V4V driver:
> > > https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
> > >
> > > [3]
> > > v2 of the Argo test unikernel for XTF:
> > >
> > https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.html
> > >
> > > [4]
> > > Argo HMX Transport for VirtIO meeting minutes:
> > >
> > https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html
> > >
> > > VirtIO-Argo Development wiki page:
> > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> > >
> >
> >


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 11:55:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 11:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180882.327744 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZhi-0001Fx-8O; Tue, 07 Sep 2021 11:55:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180882.327744; Tue, 07 Sep 2021 11:55:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZhi-0001Fq-5R; Tue, 07 Sep 2021 11:55:54 +0000
Received: by outflank-mailman (input) for mailman id 180882;
 Tue, 07 Sep 2021 11:55:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZhh-0001Fe-2m
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 11:55:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5a85566e-690e-437c-8ed7-ea38d3ad4bf0;
 Tue, 07 Sep 2021 11:55:52 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2059.outbound.protection.outlook.com [104.47.0.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-Q2S6YrQtPACFdjbFEOxWlQ-2; Tue, 07 Sep 2021 13:55:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 11:55:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 11:55:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P251CA0008.EURP251.PROD.OUTLOOK.COM (2603:10a6:102:b5::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 11:55:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a85566e-690e-437c-8ed7-ea38d3ad4bf0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631015751;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=02K8jo5kY9f/H13Wdg3xSQKXDLNPqJIa9jSlnKCW1tU=;
	b=NYv0fS5he8IRkXNPWvA6fjvumZMr/MMmMZvGybOOHntDisUJgixvhBveDw/6KdfXpwGjtq
	2KnM+aSdsYcA2T9/+zvBXwq54U2ps5/g4P7iXMEnGY6qigmDHAE7c3iaLxS7LUQbPdCdBL
	CbyRKdPQfebLTOXIavtrBHfh9S58PB0=
X-MC-Unique: Q2S6YrQtPACFdjbFEOxWlQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SNtz/WSoQrRC2vFAqRm/r0p0w0pmeSD8daRRk8vzoeWxNrv7J7tsSCT2H5AtPrqJQMjyJltfYhSZap01pBZQg1WglzcgJofInP2LnWCGtOCbZT9co6YQQSCLzgHwZdBGhnnDWloBxm0U/70rb7oo+wkbs2BLG3qKUhHv272T4xrbAACMancs4SZR04xM6Mw7scLwdn5h+dRcmEs0QBBfLAwHJZPeVH+HxrFSV3Lg1GhOZfE1x5if3WDzyv08WYDiUfPwC1ZgBN5Nt/4KkKLiSAp9SzC6YvpGWBOz9HbCVqxzVusY6gu2PLT3PjL/BmpvfpqVQDVjsLSMt60ZyiiF9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=El/+PteE+bpQXJfC9/uVNmt0qBNzA8BbA4wrelc4lus=;
 b=ZTLQm2R56q9GPneKd1KXybi6fdmqRkxgmTxi3UJzLhRXBVJQxKj/wdF55BuNUfs/qKomxV6iRspV4ulSLh9hD0yUgI3IzQ91k6Eq5Ug4iWua26UP0CmFiJ7+V+hH5Sdu8/BkeYI0HrRe0U4LPwKsRz4HQBwWnxQ9J6Uy2kc/1K+OfZueC+I1xZaakpnT4mFM5YRDmgrdthJIbxktvLv2opkk+KD3uEkmQaIrr/y7h6utFoKzlLgMK/molJ2TNkrdp4LZZBJpOO9XIvUTCXak/h4uZuzVhoWZV8oNtFRD5trg0v/k3wfbG0axYKkzEbmADPEXIwfsnzK/HKNSEJKQ7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Luca Fancellu <luca.fancellu@arm.com>, Julien Grall <julien@xen.org>
CC: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <bbf8a6fd-0f7a-aadb-a419-3776e90b879d@suse.com>
Date: Tue, 7 Sep 2021 13:55:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P251CA0008.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:102:b5::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6c66c567-e819-4fdc-7aae-08d971f66e6e
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6176DAA416A1C8D6A7D90C38B3D39@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CdFi/uHpxDnHVJmp3Q1zeAmg8B6MH4G4Dvc99OlQfLqxIF0h3Jvykold5TBHhuVJxMax68yggwe+pxll52BC7/nZ/ZodtmscZYO/N/vWJT+GzN9FnJYlDjB3hDB9HKfDVznwIJ7mapElhU6s07ApKwf6Aow5hDFaSHdbJbqLuazMgKKUBjtwMJWHndIQ2ji798PQJ+fqwVYGjdIIeggt9g7DqX3VvycQOBYRnw+9vm1puPo4ZJiIYyhYCxl8QpULktJu2CY4M1b5o1AJds10kACb7Sw/Os+yFWNv/HVF29wJVuVVYWG+9mQK+AsYTA7c042ejHWSDUOD5/SVT3eQmBLiB1Tq258ROV/y4NO9iaYAvmO7wWVoCe2s7ReaSRlmKyDhjnO9hCUfsrlJgpo8ay/dbtcNDM927ORa4UpDlIJq6DrFWZoy3g92FWeuBVm9xUGfylOznRCY+jcs0FBq9qKPAwkDCpIIQheP7qD+ZFnxFoGr8685TJAtRFvwue4CC+5aO5ZRnDB2xTuwU8jcf/vaiE0L3yTv9pyyPoVRg1T9sV9VvXYYmmPk2yrY9oF7KFaKIPtbjr3HxhDz8L1ZoqpJ2J7M2qQlf3bpPPDWWN5hlOk4cp4S3NaZe02vnNzngTReC4/yDbiOF9ZqM0vmGL+GHmvaycMkTPMKoILQt4A2m3JMxiY8tqOQAVyhl2cbla/Q6xbG3zh4EnY4K6lWO16V3cuHjsRIWjobDxSBGWw7Qz/gFtLDRX5Fr9kCakSO77XoCyGdMfc50ohT03LRig==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(39860400002)(136003)(376002)(366004)(38100700002)(6486002)(8936002)(31696002)(4326008)(186003)(31686004)(66946007)(36756003)(956004)(26005)(5660300002)(2906002)(7416002)(316002)(110136005)(53546011)(16576012)(478600001)(54906003)(86362001)(2616005)(8676002)(66556008)(66476007)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?SsQ6jAed8bPbD8QH5WQVPr/p5FflCdooBjaFm9LJ7uDaBq//pWNKoGpPCplX?=
 =?us-ascii?Q?2Cw+hQ+Vy7QD8wC/15xbTZyvxIFDwUoHtQzixp0sNZAK8Y0ZZ29c+T9uA6iC?=
 =?us-ascii?Q?BFfnT2YrVOLDjGbjdUzKmMb20i5NCh/mNxYOFAItba1ak1qYXSVM7+Akdvxx?=
 =?us-ascii?Q?SY9d55DOx78Y4y/b2wWzg12I7i0LSf80+Cbqtr9oJvo9OuGpsiSXPK5RSk6b?=
 =?us-ascii?Q?/yJqGy45FyzO2BiVH/MXiq6ZOlzyMuksgfVWKqBaIw4Fwv1Yj7hZqo/O/wt8?=
 =?us-ascii?Q?vwI7wEf2JKk8Sm48/A4xEFn1R0WAFr0kzyH9GA4mvDaAKuGzLEUkP6yu/Su/?=
 =?us-ascii?Q?f5M55aQz9cwGTwl6ce/GGIqQNO8OpU6f+qYMjJBis4yhrTqG+NdUWJnApLVt?=
 =?us-ascii?Q?nSL2Bp88dKigcK/KXY1kzvH3vx99g0bSeCMnJfxhDs1Ose+7YOA/ZCCOzrfh?=
 =?us-ascii?Q?eHcIOAX+V13pf8SY5GTFfsuRqVqF47/cZbiM1d9pc7Wigbr/Mz5QmiZFicxN?=
 =?us-ascii?Q?UOchiaPGi1aEhY16eDDEZW+PatXrvlOPWQC6ue/PDqTXMH8MUD7ZQu4vrmT/?=
 =?us-ascii?Q?Tw7b62VDg6G5lIdy/D0mgNcS8nkRTRRMkTNiH7kX8IZzbpf9Lo6inr/cghk5?=
 =?us-ascii?Q?9pPtfZGUPojaY7gs1CFykyucpNlhYizbKLsUh6lVs9TyMJus/40EizPwVp+Q?=
 =?us-ascii?Q?xBfGswDdPpCEZEQ5mMRg/JEpABayhc9qG95mVpmS80wG8Y6zgn3MDXwlwboF?=
 =?us-ascii?Q?bXuAhjuiD/gyrnrB6VZfFo9MXWV1PyJbmUn3jPY4BBA963ZwYPn1R7uNyezZ?=
 =?us-ascii?Q?ZNIEUR23d7iQRkSx/Q2+y3kqDPeXsFVYnHh/5FapumCCPeUIGUsUEkfYUmdQ?=
 =?us-ascii?Q?nF4ixnK1sSfyprDK/DK8yZslFMTS8RiqhEhEMr7lG2lK0qdOl5INrXykLZ5c?=
 =?us-ascii?Q?8aDfuWdD7L6gkIERLhboFirDjBcNhM8wFr2hyuxmxItdROT+Q9ahnZXwkrfA?=
 =?us-ascii?Q?3HcszgszQKReSTH8580A8g+BXwv0wsH0N0fFagAwBPI7fX+3b5Yu2mYRAPtj?=
 =?us-ascii?Q?iLRquNTCaVczU6gzcvGKlw1zMNoC8IEKCQHGgMYFjDP32UBKADfDrb2FNObg?=
 =?us-ascii?Q?t623vCadsyjUd5cZhLEKTr5FuVaMPgAkOBu7ZB+OMOSvVENMppsLCXeshDBU?=
 =?us-ascii?Q?5SEsWvaBkaA/Lab6L9qAYTAYewIVCL/TvRDmjxzT3RIq3pTQXdbgrSqcKu4u?=
 =?us-ascii?Q?dP8HG7M6PiZua7pGu6qqYozJRsvjNnO5VFikMHNJ+JfNz2I78XSOw0Vf+tPk?=
 =?us-ascii?Q?xZUnbp9IxMuvqHBmsLsr6PiW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6c66c567-e819-4fdc-7aae-08d971f66e6e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 11:55:48.0282
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NSXzZXGr57trJ+4EkwnboSNC0e2kIkQyV3VcdtdTsGZ4QWjMaSP9c1/qM5i2RGWzjZSqhuy/ckxVYgvHpk5dKQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

On 07.09.2021 13:51, Luca Fancellu wrote:
>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>> On 07/09/2021 07:52, Luca Fancellu wrote:
>>> --- /dev/null
>>> +++ b/docs/designs/efi-arm-dom0less.md
>>> @@ -0,0 +1,105 @@
>>> +# Xen EFI configuration file
>>> +
>>> +The current configuration file used by Xen when it is started as an EF=
I
>>> +application is considering only the dom0 guest and doesn't have any
>>> +property to describe and load in memory domU guests.
>>
>> From my understanding, the problem is less about properties (we already =
have them in the Device-Tree) but more about where are the binaries located=
 in memory as we don't know in advance.
>=20
> I think I used the wrong word there, I meant =E2=80=9Ckeyword=E2=80=9D in=
stead of =E2=80=9Cproperty=E2=80=9D because I was referring about the
> lack of keywords to describe a domu guest in the Xen EFI configuration fi=
le.
>=20
> I agree with you that on systems with static allocation, the kernel and r=
amdisk binaries must be at certain locations
> that are out of control when we use the EFI boot services, the thing we c=
an do is provide a keyword to specify the
> addresses and then use the CopyMem() function to relocate the kernel/ramd=
isk in the address we want.
>=20
>>
>> So I would like to propose something that build on top of the Device-Tre=
e work we did. Note this is early thoughts.
>>
>> The problematic nodes in the DT are:
>>
>>        module@0x4a000000 {
>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>            reg =3D <0x0 0x4a000000 0xffffff>;
>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>        };
>>
>>        module@0x4b000000 {
>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>            reg =3D <0x0 0x4b000000 0xffffff>;
>>        };
>>
>> In particular the property "reg" cannot be known in advance because the =
UEFI stub will be responsible to load the binaries in memory.
>=20
> Yes that=E2=80=99s true, the UEFI stub is using from the UEFI boot servic=
e the AllocatePages function that is giving back an address out of our cont=
rol,
> then using another function the binary is read from the disk and copied a=
t that address, finally the UEFI stub is writing the node in the device tre=
e that
> will be used by Xen later.

If you know the intended address is available, AllocatePages() can very wel=
l
be given a pre-determined address via the AllocateAddress allocation type.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:03:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:03:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180894.327755 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZpC-0003Ig-Eb; Tue, 07 Sep 2021 12:03:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180894.327755; Tue, 07 Sep 2021 12:03:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZpC-0003IZ-BY; Tue, 07 Sep 2021 12:03:38 +0000
Received: by outflank-mailman (input) for mailman id 180894;
 Tue, 07 Sep 2021 12:03:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZpB-0003IT-IG
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:03:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a0f695b6-0fd3-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:03:36 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2052.outbound.protection.outlook.com [104.47.5.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-Y331f32DNwm2pXv9fgcBiQ-1;
 Tue, 07 Sep 2021 14:03:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Tue, 7 Sep
 2021 12:03:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:03:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0106.eurprd04.prod.outlook.com (2603:10a6:208:be::47) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:03:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0f695b6-0fd3-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016215;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=E2UE+ZRwTyLPZzFoXjrcRBIUg7Nc5I95rFYE+wbcJnA=;
	b=JMC2l+Lusc8oyAAImD41kO878TdsUUcp2RWjW9hmpJyOaZQEiOT2JORsPKsjl8hwoDXFSj
	lp422QirQ/wyyJUgQz9oFKiw13ABzy5hN81tlXoOT8CBTdHCgw7SSG0AmEIb1TyoIW4Hx9
	1hcEJh4nOhGE/pXtpMb0Se5IRqC/KV8=
X-MC-Unique: Y331f32DNwm2pXv9fgcBiQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=l28+79zKnHNhBBq9ANnTM17vSuBq/3+UeHInBi1Zso4NXl4KTFTzUVP/MELeiwnhjbI3DWfI9j/mVeFJXFwXfndaaC1XpQBydzAZCyAZpEqX79SB7cAXAZ1/05FoqnXmkG0rxiVvMLgQnaWNBUK0ae91Q436dgJI0ijkqoxjwjfzS4WX9a1HMmF5wDtw1W26qcmQKn9Zt8iTC27muSB2JiR4luMt6je/MciLK4zj7zHZDcExK4OTe4KL4NQ6m85RmmbXqXRvevxgV1UUk7feXHdbMfsy5PrK5n4Z10ceO4eZOEQfFuXgix7KyjYAMhLV9HKfpH1jBHNhrTOT3IAk8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=E2UE+ZRwTyLPZzFoXjrcRBIUg7Nc5I95rFYE+wbcJnA=;
 b=napd71XSbTlC44/bcrwYDtCTtjZQyUpwbh3kPX79Z7q+jcazdc0LR7YCkS/k/th5BBcayvri6ATOgL9ynHMA9XWLpfJ5rF0vyvaagoffUbwy6YNON/zB54+m+M1mFwf6SHao2Aix/avLapBlCLTZSwxIk7tMqK0exsfCSDwDBT+lTK0lqmY0VooDD2C1CgsgWmd/yhnFNs3Ht/AdOt1UzPTqg75rOmi+aGnGPTJdW2iq/JKfFdjzB11f0L2Y1f+0CIxwqCBqMg+3xwdRFH67LCVnPat2pGwMfarkbbtdtWJ+0U/o6R+c1AVXCsq5RamIAMfvKO/Jp6jqtXisxbsiPg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 00/12] swiotlb-xen: fixes and adjustments
Message-ID: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Date: Tue, 7 Sep 2021 14:03:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0106.eurprd04.prod.outlook.com
 (2603:10a6:208:be::47) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d4008ec7-e485-462c-2aa4-08d971f78366
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB312015890505818BA83EA4E4B3D39@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gXQ3lr0rQ/68oGTucfAwyap3gcxsTliw48mbk0/jbbax99JVMC0WlrbOTSBuEP8IwMrjMisLsrc8YbQ0ygnKPbDlZdThuAtpP1a5KmIiBJi8AXdX7TCeWd4pwfVIKbZgg3AfbdXHeWPXQ2U8xNF4y07Yz6TvAUvA/W5+IyEc9P0oSZmYUqWVLGSmbZMffTD6/0XA+qR4DLqCM6UztCZgCMAWUN4cLIVu0OmYJQ12Ic0t7vRN41WThHHhpaflHlQo99c//7DTCn9oxBdSSxNKd3wchE/FSZ8F0D0uysl7mQ4cvsB2clV4t41unGXtbDjRKmOoPyvu99dRmBJk6d6/AeAnBFh/J9dd2eXlfmhtEp8gQyHRQ6kquFm/aa2ESqLa20DFdKsR6SBkZMmbS0oXztPtFH+vAiTVnhf/DX+FuHt3C7QPj/TQXNidP9RnDSHk3oCMo7TlcbfhTYaSTofirEJf6n9qXdufjazMMbHU03ZU2Sxs1AjyBMYHMjClA0WN+D7ZqR7hFX1BVQ25KQmtNX0Vfv0mRylByI8KjPBKP+C8WyQHHOlXVWDVindEic83q6K7gOCgcHt6v0n4AAwBzI/6LDZOhW5RaTWKw+UBWhf2MQF62ImieNc+wxvLaBr7LgPAs3fqZMAJW3T2WDHbXcoh7Kwc2uTLZAkc27hzCuThiUsaAuCgjlalpR1U8t/UNADRUXX2UNvmhBAwL/i3ycvfYXhTinza3m0Rox4RQ07Y2ZLgEs0M4LYTYN0MI3iI
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(376002)(366004)(346002)(6486002)(956004)(38100700002)(2616005)(86362001)(316002)(16576012)(83380400001)(26005)(186003)(36756003)(4326008)(2906002)(66946007)(66476007)(478600001)(66556008)(5660300002)(8676002)(31686004)(110136005)(54906003)(8936002)(4744005)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aWtNeFdieU0xZUtVSitVTjBPMWRzcy9STTZDMjEyTVoraS95RUtzZEVnRVlm?=
 =?utf-8?B?UmkxY0ZMeE15NG90cnFISXBSOTUxMWhheVdTMjRraHRlM0ZRUkNCTytNeDFD?=
 =?utf-8?B?OHJVdmZRRDdtcFpVRS9xam16dTFQR3VxR29JNHlueG1rQjd1TGJhaVlKUTY3?=
 =?utf-8?B?eFNzbmhDZEwrNlUydFlBN1RnQng4L3dlNzN0UlVORncxZEJtSFp6Z1ZpRWVC?=
 =?utf-8?B?Um9WREZnUUZXRTJCWldDWjBkcFJYb3dnaTl1ZHVhYXQzbW8yNkVsZUxwbEpy?=
 =?utf-8?B?VjlVWUhGdHFIY3FGYXlsRFUvcy91MS90Z0VUeVNjd251VHpGVUJQdEk5NzBY?=
 =?utf-8?B?aWllSG5rZEVzcGNtM2hsTEtKWm9tZm1OZ3V3TkZlWHhDWE90b1VTUVkwZHYz?=
 =?utf-8?B?MWNuWGJMUVNUbWQwc2NTVnRtamRER1hrV1RYYXpuZ2JBdzJhTkVhMm9kblRx?=
 =?utf-8?B?VnJGWXJWWXlLdkZJMGJ2WDdGUWpsU2lMUE1VZittRTRhZWM4Y1BISUhkT1dT?=
 =?utf-8?B?YXkwRlA4Q2JJS0NGUXJ5R1Y3aUtnZW9Zcm12R3phakNFeEREcVRTNHNNNitJ?=
 =?utf-8?B?eExBdVV6dG1wbFBVRkMyL1VMQmdvd0Z3c2xSNGJnMWcxbHJXNXljQWRZdTFu?=
 =?utf-8?B?eDN0QXdYb1g2WFFKTXJncXlBc3Qrd3Z2OHNQRlE5RUo3WlJhUHU0bnUwNEJX?=
 =?utf-8?B?cWUwTnFxM3g0RWFDOTBYTmgySzhmNTc0MTgxeGY3SGFnZkd2U3BVSVJMTzhn?=
 =?utf-8?B?ZEZ5Q2hTM09EdjVCQ21tUmxXazN2VlBOV2ZJYmdIZUhZYlpoTmF3eGpSSVov?=
 =?utf-8?B?NHZGbkpJNzFsUi9iemlhZW1ZUXlwdFlBWGtkZWYxU3FEZnRnc2ppUjN4TXky?=
 =?utf-8?B?d243V0ZIVHY3cTM3QVJDcG9aRnJiRmpIRmRnR2J2NEd4SVBVbVB0U1QybmVP?=
 =?utf-8?B?alg3eTArUkFpenZsYmhSN1pwKzN4NDhMejRvRWhWYzQ4bTYyUlZQNUM4NnpV?=
 =?utf-8?B?T29icytlSEpVUW9Zc3h0TmtxZ0ZUS2V3ZkJ0SDVYd1RMdEpoZzQ1cWpzUjkx?=
 =?utf-8?B?dWZtdGRSUm5jMkU0aWxEdGtRQVNZa3o4RVhaOWhXMlU3WHhlU01oWUcxSzJl?=
 =?utf-8?B?Y05acGhQTFBaVFZaNXRJTEN4NkxrQkNNaUtXeGgrZERZWmg2ZUw4Mi9BZFQv?=
 =?utf-8?B?aTcrUHNZeUxuQWNMQ0c0dmZNTDlyRjVvWVRtOFZqQlRRQlprMCs3MXB3Q3ZL?=
 =?utf-8?B?UDdFZk11ZGJmZ3orSEE2NXppSVRvS3JjOEF6eWFYYjFSYTd5aEUwOUFVQWlu?=
 =?utf-8?B?MGdnOUR5aGRSOGFzYkZocDhSRFZ5VFhiN0tHSUxiMldaSXkwQlRibTBhS21u?=
 =?utf-8?B?Q2dyL3I1Q3U0WVVRK0ZYbGRDMms2OTJGazhSVzExdkV6bEFUOXpSZzNvejVD?=
 =?utf-8?B?UUFJc2hPVmRYVnVRNU4xKzZXNTk0dVI3WVJ2cmFUMWdmRWpFajBzanczcVEy?=
 =?utf-8?B?OFl5bmRRZlREUHJyMmVsL1hCNldLNVlGckxYYzV5ejhQSFg1a0YxcGZISVhO?=
 =?utf-8?B?SloweCtGMFdKdVlselEzWENEaDEvaU93UVVHRTJGUEtFVExZRzhCWHpuSFRO?=
 =?utf-8?B?KytDQXZ6WEtmV2dQMmhiR3doUGp1NTNGTFNQclhTUCt6QUJuWklkT3lBaG82?=
 =?utf-8?B?djJSTjBMbFdZZm8rS0I1MVNMS29EeDdPVGwyVVNpUEtwdHIwaXNmaE5YMGox?=
 =?utf-8?Q?X9VF6pViJXIf4J4I5WlrUnSUwWbFxSKHCPQvDvZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d4008ec7-e485-462c-2aa4-08d971f78366
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:03:32.6076
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vIovpZJGSULoeQGlwuD0njofzK9Yujr6F8vg8ppn8ZVmS5mQpzu+vXvTl6+0/lsjr+ysyD5eEtwKSWjQaqHK5g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

The primary intention really was the last patch, there you go ...

01: swiotlb-xen: avoid double free
02: swiotlb-xen: fix late init retry
03: swiotlb-xen: maintain slab count properly
04: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
05: swiotlb-xen: suppress certain init retries
06: swiotlb-xen: limit init retries
07: swiotlb-xen: drop leftover __ref
08: swiotlb-xen: arrange to have buffer info logged
09: swiotlb-xen: drop DEFAULT_NSLABS
10: xen-pcifront: this module is PV-only
11: xen/pci-swiotlb: reduce visibility of symbols
12: swiotlb-xen: this is PV-only on x86

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:04:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:04:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180902.327766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZq4-0003rM-PV; Tue, 07 Sep 2021 12:04:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180902.327766; Tue, 07 Sep 2021 12:04:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZq4-0003rF-MV; Tue, 07 Sep 2021 12:04:32 +0000
Received: by outflank-mailman (input) for mailman id 180902;
 Tue, 07 Sep 2021 12:04:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZq3-0003r7-AZ
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:04:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c166c816-0fd3-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:04:30 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2055.outbound.protection.outlook.com [104.47.8.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-1WTDrmR2MLyfRnbNaWU5jw-1; Tue, 07 Sep 2021 14:04:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Tue, 7 Sep
 2021 12:04:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:04:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0096.eurprd04.prod.outlook.com (2603:10a6:208:be::37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 12:04:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c166c816-0fd3-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016269;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YUjiZ+FcARWZPfFdze9B3FWeG4LoDQIbpj7UKsZHujs=;
	b=Qg/AlsS5lskTedWM7PBZtc1T9M3UnlHNgaXHI5xEZiSzMq4mgEMA/4Tfkd06ub5k7A1nym
	EzneuDDEenGhF9w9DpmDJBZcWb8yfhEUN98MKruMkhMVJIy3IRs/85Zohq4lhwj2fpqFCn
	qq0gYMMRPR/L0MyPW66A9eFqRyNkYRI=
X-MC-Unique: 1WTDrmR2MLyfRnbNaWU5jw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hWqVnRoSMtkkKoLxFIHeXIgiaJovVpi2T/C6/AsZB7qDEw+xtIrewTpygFBpQZZoseK8rv/oXPeqB/Kza9pJ90bRay4WbZJj7ouIFUIA8vcb/6Q9ZGxjOThcrIIGWqZGjuqEo7RoLMYwhGcZbyvUu28tXdCKByKfWxZMD8a0CTqjWYKz1pu1b4r42ZD4nZOJy9EdJv+9mAK18zwcpqqbMkYXrwHVoY90wnTudnjsEwyz7cCtferfkASLjI4MFjnMq6vxo7WfJr4FgGcQ+vZyfr0D1iJZv01FvTd2ko959+vhmP6sUAuO3C8hIc/CZB81u+pTlRYutm4XWuZzGmkDJw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YUjiZ+FcARWZPfFdze9B3FWeG4LoDQIbpj7UKsZHujs=;
 b=P/sUORgtT+v0e8U6tOn4IUxrHCYqaVhHrluOs2tQ+UpS42gjQBgVMNtpDnx9DGQHxpUC0onhjzDLvMQMEXKU4ytjm6SUeW8tcXIEF0azjIWi4sDVwQU67lYxbLsF+m0DrXo5T+8fltnBWNvJzwcnOwroomHg9EGY81/Ygjgb9tWq7BJiXssn3FFbywMjF9Jl7u+BBnrECVbegyPrEUE9vtXVBZzJqDvQmik+bI5pP4wcbvWeNdWKLYnyim7nJTGIbZq+I8lJI1vQ1pz27lm+AnxS6LJ5TO3IRV/dqLadTHtTLOjFwN/Ktxs0sBXMmLdH2IKAUmXxYSYzAmv1NyBeNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 01/12] swiotlb-xen: avoid double free
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <ce9c2adb-8a52-6293-982a-0d6ece943ac6@suse.com>
Date: Tue, 7 Sep 2021 14:04:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0096.eurprd04.prod.outlook.com
 (2603:10a6:208:be::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 921de9e7-79f6-4e70-1d37-08d971f7a410
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB312004B434F20DB2E967D513B3D39@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cTKXp63JMT+BOej5u6mmIeFwoZz5HLopoCNNDmsPAfAJMiZxsA7fw2XDuaefcIvO/z9fhrHpmzco/sPf+qbyEWQg2kNkrv2VEjOuVlrroyMfcvfE2I072EhTZ1sYwOAc1jeqdzuboiL5weYhMRsPIXWe+DIR2ju2+eV2gX4NzVHEYcxK8AkcxTiQh3EWAWkNO8R8vcTAQL/f2fMwEKP4lGofC0XzX/eIOLDHvw/pRPFI2ActBVdpN3zwUA82e26DNpRS8NrPCkA7DK00nB9WbruRgOrp51QMku8GZG1lx0MhjqfqrJKVaQ3b+rTLsh5EoKVGfmNYXAVg2gHNsrHNi0qshYakSNwtE8mIYZxNvmtXSIqEIFYN+/gP9sHzd5Ww0lctLsRQaCuKlXMqNpPC663xQC9eXQstM2YVJ+W4J/Z6n3UbLvecHEpc0gBz/2gOKgBt97MEmkrGnDeIXhlKq6QIKAwxWBJmE0apSNBNLGqGHbfkdTSfJHN9vBPz3LHEg4NhZ5vwOF1+SoOybfIeL4a9wXdbtQUJP7s0WxoU7yHZlpmA/KAvYbnxx4rOJDOh8QTU+NxyXwnN0xWQemY7TAslj8ZaxC+dVtzeDeRk8wBSeKHml5NqiBusXJpiUX5VjwvtW9/D2f01BHWq9iRGvuBywI4/e3nz4tikmbBjWksPB+xjrhKgnvo6msIQwlogf8IsKWSeMjRmMhRHvCFrj/JYibGZmYHfIFLdDz5BGHI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(376002)(366004)(346002)(6486002)(956004)(38100700002)(2616005)(86362001)(316002)(16576012)(83380400001)(26005)(186003)(36756003)(4326008)(2906002)(66946007)(66476007)(478600001)(66556008)(5660300002)(8676002)(31686004)(110136005)(54906003)(8936002)(4744005)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?d3FudDlzTVc5RUZZeWhhMWJyUEx6Rm5IaXAzWUZ0MmgrMUcrYjJNK1VMYUJS?=
 =?utf-8?B?U3paOGsxT2llZDNNMVBaakdUMEtNTzBzNmJzMTJKNlZmUm5oeXZhS05hWEhk?=
 =?utf-8?B?VnFZdGNHK1o3cmtIR3pIRzRFQVdjUzc2d0VkdWtIQlpLbDFWOE1WTENQMnRi?=
 =?utf-8?B?b2pUR3pQc1p1MUpGZ2lZRklKRjliRWdEYkZCZGhuQmpYdVc3SysvQ253aG9P?=
 =?utf-8?B?d1lHNDdTMXIxVGdOZXIxWUE2STluVjRPTVhoYnQxcnNtUWYzU0N5OWhqTE1a?=
 =?utf-8?B?VnNlMWNnL1hhT3lJLy94TW9jbzVnY1NnaVlpd1ZCbXJrV3k3ajIyTDRiOVZh?=
 =?utf-8?B?Q2E0QzJGL3dqVThtWE1UV2pYVTRQUmpqRGZsdkxMa1BDSDR6ZTA1RW1NWjBP?=
 =?utf-8?B?NWlaMzEvUjNidEpjekpMUmdQMFZkOVZWRnArSTlLNE1MSFdKWXkxekdIcWla?=
 =?utf-8?B?NElCck5iS0hudjhpcGoyL2FQS3hlVGZhQWEvWnI4L3lVOG5hRU12VVZBeHNN?=
 =?utf-8?B?RmhoVmlkZjViSUVyaUw1OC8rbGtCTGQzZHJxb3h0NDFTdzlRbCt6K1N5Ylhn?=
 =?utf-8?B?N1pFNE91ZHJZTGZnS1NqYkRvVWh0Ukx1WEg2M1Z4cC9FT0cycVJiNDFldDJF?=
 =?utf-8?B?dWJXY0VqVWxQbzF6STZHNHdvZ1U0NTB1dVZiT01sWHpBeXNYbnFpZE9qTXpZ?=
 =?utf-8?B?Tkk5OTZrVzhCY0JlM2xFVG4zM25xRjVnSncySXIzYjBtdGV6TUhxWFQweVJD?=
 =?utf-8?B?V0hrMEsxWmN2L3JPTjRIU3VIbldoZTRFT3ZnbW1PckZqZUcxbTBzbG15dWc0?=
 =?utf-8?B?cUx2T0h3WWQwdDFreUVGMzFMTE1wM3NtQjZ5ZkFuM2NvWWhXN2Q1VDFNV3ow?=
 =?utf-8?B?ZUp6R3BxOVJnUGVZY3JETnRya3d4WGZXNzRNK0I0VHpEQ1BLWWFiRE5KTkpY?=
 =?utf-8?B?VzUyZXV0MlBoY2M1VHlTaW8rRWxBbHZmSzhkTHloRFlSS05TQS8yMWxWY3J4?=
 =?utf-8?B?Y3FaZ1Y3WVZjbWJaOHBsdnlSaFRMTUtIeFd6T1g1WlVDOXRqWURoWGRZc2tt?=
 =?utf-8?B?a2hjOWtDTDBLaG5LaW1sUWdONnRvWkdSd1R2NVRhalhGMU9PM3ZkRTJoU1Vi?=
 =?utf-8?B?QWc3OUF6c04wZ1BwQ2EzYWgwRC9HZVY3cmhlOHV3YVJiYnZkcUY5VHhhV2sr?=
 =?utf-8?B?MFV0L213TXZncU52KzZJeTVncjZHQUtqbWFPdFN6TnVEVTRvQUdBQmh1MFo0?=
 =?utf-8?B?WmpQdEdOYmM1dE4xMy9OTW5FbVYwTU4yRllvakJvR1B4b29La2dya2FrSUJC?=
 =?utf-8?B?cjcyZlpVZnB1bVVaVzQ0Nm1rL3QrS0FxbFp6MEpWQ2pOWlA1Q2hkeHNoTVZu?=
 =?utf-8?B?QzJjQlNJZGZMMU41UUREdTZNalRlRnJycU1IdFVHZFl0MzR0d2ZReEdTTGRa?=
 =?utf-8?B?L0xpaUJjdDlhQTk3bkE2MkkwMUF2Zm9DMXR2eFZsVFNGVHN5K3dPdTdaQko3?=
 =?utf-8?B?WDFzTjQ5Z0JOMEZtc25KQnRzZ3BOZUdjTlp1WkZjM1FQSnpoSFhqUGUyVkRs?=
 =?utf-8?B?YVhObzRQMDV3eTZsRXdsTHE0R255SWtVdVRKbTVNYjQxa2VheTIybXpTM3Y3?=
 =?utf-8?B?VVNnOWNsRDY5OENWVU8yckxqVVBrY0E0ZUVQUXNDSzdCeU5sNVFwV1JrbjJH?=
 =?utf-8?B?NWE2TldNR21oVm8yV0R4SHdubTRqZ3RRVEtuR3FOelJBbnlHRkh5MFdFTmpW?=
 =?utf-8?Q?WAG8SzkZU1bnjyZ9mtOD9I9rNAZYcDqmM+yfsrP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 921de9e7-79f6-4e70-1d37-08d971f7a410
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:04:27.4045
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fxmyLLc2/qK2sEN83kbQC4/qg3WM3+6UGQu1CSP/kkah2+uudkIas1308gmuLjq4QwRWerOTYZ0iL0+NKi/ObQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

Of the two paths leading to the "error" label in xen_swiotlb_init() one
didn't allocate anything, while the other did already free what was
allocated.

Fixes: b82776005369 ("xen/swiotlb: Use the swiotlb_late_init_with_tbl to init Xen-SWIOTLB late when PV PCI is used")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Cc: stable@vger.kernel.org

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -216,7 +216,6 @@ error:
 		goto retry;
 	}
 	pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
-	free_pages((unsigned long)start, order);
 	return rc;
 }
 



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:04:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:04:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180904.327778 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZqR-0004MB-2K; Tue, 07 Sep 2021 12:04:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180904.327778; Tue, 07 Sep 2021 12:04:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZqQ-0004M2-VA; Tue, 07 Sep 2021 12:04:54 +0000
Received: by outflank-mailman (input) for mailman id 180904;
 Tue, 07 Sep 2021 12:04:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZqQ-0004LY-07
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:04:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ceef2456-0fd3-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:04:53 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2056.outbound.protection.outlook.com [104.47.5.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-OVm-SvxQOwytzB7hKxUMTg-1; Tue, 07 Sep 2021 14:04:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Tue, 7 Sep
 2021 12:04:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:04:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0025.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 12:04:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ceef2456-0fd3-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016292;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=SEuCaucOQlDfG0ryFHeKwBYGR4l7zA4FQQ51ni9uTXg=;
	b=mE8Q40nkht5RRzywnbwG8Em7tR5ztaY36D8bh8eDJx3ti+QNg/2jz3+5aWN90rNG2BWQQ4
	JnH2rneP9KzAM2A23f6DuS4njDyndwcSOFnWMijHXVD2IC8PNcmvTEQoERg4DwdDOHxSpo
	RQI2h8YxC7Lj+bs75fdOvDnrJaLsJaU=
X-MC-Unique: OVm-SvxQOwytzB7hKxUMTg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ab66C7ASTuno/bPzTb+Y8j3ccwUmyogXm6TzavZcrj3O1j2UW3/wnCZ2nP7GBM07+2VMSttopQsZL3M1bHVqu7tYUEc+WDliNoW3BEUNO1f3gSgOCS0+r4X8aR1pqBDedp6HXO5lezubR1Qv4MoVNJ6U5m66mAh9u5rwABNHjN97tTcsImXOvOGMxQaL0G6vppdK9UXiit4v+bsC+xZTNqamqSnLFR4NX24Sp1wQagy2O/N3o2z1rjcO/G9h81mXo6cqOaPKhdeaM9qWKHn+7FcGNJ4XIwWNYOOY4i9Cu7iIXKsoLRk2/feUE6vaX/DozkY0GVkKtbtO3ctu0PuZwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=SEuCaucOQlDfG0ryFHeKwBYGR4l7zA4FQQ51ni9uTXg=;
 b=SnzjV8laz/8YNs2cwweqzy/rW8YYs7YM4WCrzowBIoCakPCYY+0oeCSI7xq3kRikMLDC3MrN+iQMP1/pdCtPr68kh09M70foN3neAhMkreiYVR/1OFnmjH19h/svH8ThHFvZnlKn0YSAsKZUO5UZKV0Tg6pgOqY8uQJcg0Nwud0p1ODIupEDs0jXE9JVcJV9hWLf6CqgtfTn7euXlaaCN/apVUgPzGzAFdNQ47YtHGlfTRnTEYpttBBeWdJn/V2wdxmD4hKGUJersUgoS7PBpVV9JzP7GxyIgkGKUSyV0IAy2+rET1Mwh6R1diHaZTeBPLsRM2mCXAQPm8K3Of0uEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 02/12] swiotlb-xen: fix late init retry
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <778299d6-9cfd-1c13-026e-25ee5d14ecb3@suse.com>
Date: Tue, 7 Sep 2021 14:04:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P191CA0025.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3b9b8ee7-e96f-4f6f-ef01-08d971f7b176
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB312064CBD7DFF28BDD6FADD2B3D39@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:901;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gHKbMa/EKtBsFLeDT/7oxzcuXl2CWZ7i5ag6WT3vzfBLj/BZf/2wiynVdBsFfwBjFR8lQyxcbcVHC6zaa5fEomBY28TpP9BkKol0kSN2t05LNKCBlzRCQHnpkqdZNfUgrB92kN+ecLHTh0c/sKEB+zzLxg/SdGpLLNLzv05WUAH664MZ2+mjagQjxzpIqoT+ex3jeN4FoV4HI1iraFb+h+vkXDcQ0aAlK1qnOwLlINYy5m8bk0a+Rey0hL1ZCPNZsPOsF3FWqNsBegKdzu8In7CBbf9+4M/gw+qeHApI9IdCOxZEcvupHWonzSDqCt3Ux6tVNuHtsmfCwRqN9gPtBmRGhWf2YcXJU+o3mPkEqnDaQqC9HTQ4MWEo8RoEMNMT9e+E+TCC54PZ0F2oZS7n1WMOiwt1dja85SH0tgpW4H76/S5R7YmkzNrqWu7LJX9ELS6BELc576RDYpNThRd5/wPznTRkg31Nkfl8/H6VEm7yvcvzwuKAGJAjSzPMldSlV6Pook/oYzXJTgAciuqe12AVk2/OkTetVci1y4EYys/V31S2h3OCNkDIrSjuXS0OBgSAjhZfsFeHd6sLR60RQTGUBIW0CXsB0nq2gaNRKed11EBrqj+D9eI5LL5jw9Sqg5orY35MkRgJLb7thjlFxctCQr2sHBrOi2lstKm0ndzy693fT0/f5ccUvZ4kGTzsHI+mVYQQpcXzTT3CwbC7T+58N4IXA6aSlcEBVqXjYwE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(376002)(366004)(346002)(6486002)(956004)(38100700002)(2616005)(86362001)(316002)(16576012)(26005)(186003)(36756003)(4326008)(2906002)(66946007)(66476007)(478600001)(66556008)(5660300002)(8676002)(31686004)(110136005)(54906003)(8936002)(4744005)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VVBUeXI3d1htc1BlYVJ6dHlDdVZlT1hOeFVXODN5UUdhTDBQU3NlNVdRVE1G?=
 =?utf-8?B?c0hxbkxpclNmR2h0dkd1RlhDSkpVSXdPY3pLclRNZlJWWDI4MXAzM3pqK1d2?=
 =?utf-8?B?eU9jcVNVN2FGSjg0NTZ4b04zdXFEODIzeWxpbEQyd0JiVWx3Q1lRTnVGSlFW?=
 =?utf-8?B?N3h2TTRyM2R0ekFVc2RraTJxRjNybDZjRzhBejNyNFQzSmJtWEYzVUJCWHFn?=
 =?utf-8?B?NjVTUzg1WitVVFc3akRUVFU2dlBVMVlBdUtTcS9sT1Z2MWRwcnd1aHNFRW04?=
 =?utf-8?B?V2g1Z2FhdUxaRUpGa3JmZEc0bWNPMVp5MDhYWjZqYmxQT2FuTW5zVmRTTVp0?=
 =?utf-8?B?MXBEbzNVR0RCZFI3eFE1TWVzeUh5UU5IWW9MRzJtMzRBbHRkSjI4M0NLc3hQ?=
 =?utf-8?B?b2YzN24ydjUreTIrQVF0TGdhM2syQ0NWSWVCa0hMQlREQkl2dGZTOVJ4d1ov?=
 =?utf-8?B?NVJXNnNZRXdBTDQ5RExkdVp3bElDdEdOczdBU0xZT01jQ0JpN0JpYWxzdGpn?=
 =?utf-8?B?TkRVWXVabmJQVCs2a2hDbUk4Q0xFODViU0srbGpOVVNMUGJvRW5NazJ4NW5N?=
 =?utf-8?B?TVdEclFsYXhEMUtBaks4YXpnVCtqVmhuM3hzcDZwUHN3akRKNkFNTnBVbklK?=
 =?utf-8?B?YU1JdlRyNWc1cFNVRTBFSldhajl3cGJaeEVLRVhURElzK3FienF0NG1yWHRs?=
 =?utf-8?B?bEZFeGw2Zml5Z2QxUmVlQkhRclJBU0Jxa1dGNm41UWxZbXNVellWRzdOTjg2?=
 =?utf-8?B?UUZRMitxZHFrNWhrS3dJNDlWRCs3VDg5NS9wZTJXQ0hFTzgzWEljUlRtcTZy?=
 =?utf-8?B?Wm1qWjUra29zeGM4KzhEdHArMExYaGJMS0ZsbnF3RWtDUjVnNXJaVmRBNHFp?=
 =?utf-8?B?SXZIMk00MDFWMlRCYzdPczZQd083TVRiYWhHT1RjOG11MkdQU2UyYkpLc1lT?=
 =?utf-8?B?YlplWGx5bUNuTnhxYkk0YVNvKzJtSEZFeFlnSWhyRHNzNlNQdjVmMDJ6VVNQ?=
 =?utf-8?B?Y1lHSDF1ZytaWlR4TnlyTWNXd2xuTU5pT2VXTjNiMGRaSE1NbTJvM2lLWkhx?=
 =?utf-8?B?dml5TmE0TzR2bEM4cHpDNUwrTUZrSXVoSlFXd3B2QnpqY2NoVnM1dHMxa2pR?=
 =?utf-8?B?V0p3c05Hc3pidGhWZnFHVTBJT295enBSRm1XNVZlbytHSEo0SlI5a3VIZUNT?=
 =?utf-8?B?Z05BSkFlRThSTDJvb2V0d0RmRFNjc245dE1QSFRFSXlBZHBhSzNwYmRPUmg5?=
 =?utf-8?B?Y0N6d0g5RHVTUFFMeDEvNWhLc3kxNmhGZUhZNHJ2YWp3ZTdyd3VRL2tMZ3V3?=
 =?utf-8?B?OHRvMjJ2N1c5MENuYUtqNk4rL1NiTURNSWk3alBlTE9KbzZEK1dVMHY3Yy8y?=
 =?utf-8?B?T3ZLU2VWOEhxbGtHbk85ZnkwMTdUVngrQUg1cE5rbmM3aDhnMTVqdHpyN2JF?=
 =?utf-8?B?M0sxOFRYY3hpbWNLaVc4aUE2RmNJRHJ1ZFpvRlRYZG1KbzlYb2UzeG9Ic2VK?=
 =?utf-8?B?WTNNOGo2L3ovMmY3TWV5a2tBQitWd1ltZm5PSjd6N1lqeUVrMFI3RTVFNGhk?=
 =?utf-8?B?Qk9CODh1R000aitzaVRJK1BIeThKQ3V5NDZZaTQzek9oenNGdit6UnZOeGZ2?=
 =?utf-8?B?Tkc4QVpmUWVnODZYU2UyblkrQ1BjbndCNWdOY0U3Z29Fb0Yza1JockRxSVNK?=
 =?utf-8?B?b0h3Y1luMXZmdUlvajhaYzZHUE04UXJxUGY5Tkk0Nk9kRDBrekJjU1JuMTBJ?=
 =?utf-8?Q?C0FWkfpAGlLfHYZ6v8Q+PHZu5ejZzKVIfyDuDRj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3b9b8ee7-e96f-4f6f-ef01-08d971f7b176
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:04:49.8957
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5ekgPaTEnMDnoeOJ1xNV7KeNLMYISrN0W5UEYEoDo1ZcyumwM/eujCiVHt1UJcgqcmtKVMpXXX7Az4pqdMNEXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

The commit referenced below removed the assignment of "bytes" from
xen_swiotlb_init() without - like done for xen_swiotlb_init_early() -
adding an assignment on the retry path, thus leading to excessively
sized allocations upon retries.

Fixes: 2d29960af0be ("swiotlb: dynamically allocate io_tlb_default_mem")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Cc: stable@vger.kernel.org

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -211,8 +211,8 @@ error:
 	if (repeat--) {
 		/* Min is 2MB */
 		nslabs = max(1024UL, (nslabs >> 1));
-		pr_info("Lowering to %luMB\n",
-			(nslabs << IO_TLB_SHIFT) >> 20);
+		bytes = nslabs << IO_TLB_SHIFT;
+		pr_info("Lowering to %luMB\n", bytes >> 20);
 		goto retry;
 	}
 	pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:05:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:05:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180913.327788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZqt-00053R-Fk; Tue, 07 Sep 2021 12:05:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180913.327788; Tue, 07 Sep 2021 12:05:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZqt-00053I-CR; Tue, 07 Sep 2021 12:05:23 +0000
Received: by outflank-mailman (input) for mailman id 180913;
 Tue, 07 Sep 2021 12:05:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZqr-00052s-Lz
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:05:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1f79e998-0a83-4c1e-9023-9908260c0db0;
 Tue, 07 Sep 2021 12:05:20 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-YepnUYaDOWWKxsgoltrgHw-2; Tue, 07 Sep 2021 14:05:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 12:05:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:05:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0902CA0012.eurprd09.prod.outlook.com (2603:10a6:200:9b::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 7 Sep 2021 12:05:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f79e998-0a83-4c1e-9023-9908260c0db0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016319;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Um9DC69BNNgCMREa9h68z95lfy848yTwNtWb0Y+Y3VA=;
	b=BLW+c6Ug5Z+zwhNlk/YpH1Q5i3ASw5QY84r1mcs4V6bYiJNU3laJcKVtZIcl9oHP82ScLK
	D6QboYd6vhvPJmcv3SUaHQRdGWS9Re/r+c1fbB2PtjmXLTcpUhBY0Ybj5wAy3vYlms8A4s
	/nzD33dXTk8000D6soOGjq/zKYoyX1Q=
X-MC-Unique: YepnUYaDOWWKxsgoltrgHw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kmPC5szowPAPAcKQtrFxAUEPkIW1vldLPfpE1mITFMGlnxYUxjnG45p7jkouAGh+wah6/yS2nWVTiLx1AcU8Y8RiO7EOptLW3e7FAl4HxRQCc8W3qOn2P5VHL6z+yMvC3Fjh96JzeY2r7pTi3UrAM1Imi4IfPI/BhP48+N2rQ+XGf/aof+O4JoGzWaUL6Wr/50ZmSHR51GyxH4Z8NXmiJPdKnRlctIOc51w2stDm1Hbzpin6FtL9YpAIEllPB9hrUTpkaxxwbCD4vpMW2MNTpXh133deEewqx3wCAmNVt6nQwfeHweNYOrDhppT65RN3EoA+7L/l98bkFdAJAr0tlA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Um9DC69BNNgCMREa9h68z95lfy848yTwNtWb0Y+Y3VA=;
 b=B8vpRUnG/Yg1Rfe5G/aeQCNwpoFRYh3m/0Ei3R4L/33fYRBEtoZHj6vQGFmiZnmwCUHPXWkgZXNNKYNuRX07zeIdO+kVlaTLa1LwOfcPIseu4wsmPl1LNI1PFxBCCYstSXlUPQwQoRM9MkjwouML6cO4BB2khl1FyyycQyA6aZMXjC5dCd/HsuXfIxEsxcsPd7HrsPBJ1RuPNdE3HumwE7GXzqAVKqK/KNrqlEw5wFu/NowcBdBOCVH0hQZogcaeQsO6v+yQUioD0e92VgI5piJMLqMDdomrIzBXpi50ZMyUrIRP/ylR6STk2P9AE70SofFB5UaoL6bbxISXih0hUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 03/12] swiotlb-xen: maintain slab count properly
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <dc054cb0-bec4-4db0-fc06-c9fc957b6e66@suse.com>
Date: Tue, 7 Sep 2021 14:05:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0902CA0012.eurprd09.prod.outlook.com
 (2603:10a6:200:9b::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 79499601-472e-4206-729e-08d971f7c017
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2607B10DCB9FFCB2AE85281DB3D39@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vHNpV0PXcxqG0FHlMMXWRRspYP4aGcfX0qZtXuA4lZfbuknabNCkJ5tPA1iWVKHTuBhzCg1elOcIHInoUVlhvCh8TDG3yER7hDUDYLfh5B3Bu1B2ARpsO0ZDp5fDZwxQU+4bhDslrr2LVX/rvFvDQ8SXNb4HKjUgF0cvP/IQXq/olNTlFv+Hv4KAHlo03RRwTjx9w+ANl2ZhBZceAZWvqSW2p6IhXqW2o2pTFNERMUboGucFuljw0sMuf8oGrYC7QCsXEJ+i1Aa9lK+iUk6remTJ5x8XqxdyiBC2KF/8WaM1msnuAeYrcozBSFlGNcyxYgDsXHTN3lSGJp+KFGeXQ4wN1+xb7NdTpKaeqXP5kgTTnXnoU75Vw6HQz5ttFj5krhauYFnxmxvmscakJOXCX78IadnivoLvlHxZtqw7vNp3SXrx0swFVxKDOAdFHfJNBRDsd2+dTX+zXoZgKNWmioOCa9U5ySGPpAjkNZNRLijTvPPrSEAHc/0THG1lPeYSZzPKYW7aR79A0v66g/F7yXew239RTf3ev4/hDcdu5o3GCPLIWZOCKBhYj0EbhpoC62ycUUpUsyzcNMHSMopE/2x+jLgpD1CULyLbtlkrrkioe6SwU8S7Qqz75rZNmZkYtLZGkd05mzNYpfuipP6js0a+GQtmDgKnyBuTvcdkbv0wZg/z4un2Q3NTc+QPcYlBQsnHsp8xVWB+SA3gEBbNUdRRv5o2X/+jSRl5WXuIJCA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(346002)(136003)(376002)(396003)(186003)(83380400001)(66476007)(6486002)(31686004)(66946007)(66556008)(26005)(110136005)(54906003)(316002)(16576012)(36756003)(478600001)(2906002)(8936002)(4326008)(86362001)(2616005)(38100700002)(31696002)(956004)(5660300002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Y2V5UkdMRkZKNGxzWTNTMUtQazlybXdicE9DaHMxbHlVL0RYbnEzSmMrTnY1?=
 =?utf-8?B?ZTlpZWNzVmtiRWl0bSs5UlNndmhiVmp6b2czTkJ2alhtbDdPMVRudVFuTGdG?=
 =?utf-8?B?eHZ0bTJMVkhvTjE1VkFpQ2l4RXErRVJvbDZFUExKb3k5OVBsS0ZpVzYrenZ2?=
 =?utf-8?B?TkNlWXBWVUtKUjlZdVl6aHZMbldoR1hranYrWUFZNFMzWHFVNTdHY2JTMjln?=
 =?utf-8?B?RFE0cjlXTTEzZW45R204NWtsU24yaWh5ZGNVRjlIVkwxVms4Z0tCb3J0aCtY?=
 =?utf-8?B?d3Bad1F3Zk9JMjdRanczUjE5K1RiMTFIdHR1MldDRlBUV0hwZTRaNFVjSk5u?=
 =?utf-8?B?MjNJaWdGVVp4OFlLRkpmb2ptRm9CQ1VaMjdwaXhsV2lYWjJ1OHFjTXVMZytm?=
 =?utf-8?B?c1ZTc3lCaWVuSmFFYkUwWklvYldIeXYrSzlzWlQ2TTFSVlFFWGlURHVIQmNs?=
 =?utf-8?B?NzBCZU1RUVpXL0RLRjBmVSsxdk9zZE1BSUdITW5Fczl6Zkdac1NLZFlnbExB?=
 =?utf-8?B?WHVib1UxMnZWdnByOFl3SnVFUDlpMTdKS1BrdkIxNkVvaHExaXBtS2s4Qnlv?=
 =?utf-8?B?K0NpYVlJWGVGUlpNZldnY2c1SDF6VFdVSFcyVU02SWtIbU9MY2VZSnY0VVBt?=
 =?utf-8?B?VWJJdFcydHgvWXBlY3JDckc3Z0tEUldRQmZYTXZpdTlZSzZsTkowV3NKNmty?=
 =?utf-8?B?bEhiMjV6VW1XVSs1MW5LNmZKaFpjZ0xhK2RBdElRZXp0NFd0d05Ocm5QQVdk?=
 =?utf-8?B?UFVSOFZaalJyb3p2ZUhrWXRDbWhkMlR6bm1xcGF5dDZvT2FLYUNDV1Fidm5R?=
 =?utf-8?B?Z1JPUzdYVldDQ0wyTmt3NVArNDBCeFh6YWtKdklhK3VIU3dyamZ2UXVPdm13?=
 =?utf-8?B?NUQyNFdTbnZHUGZ6aE5QVHRxUTFoZ0ZrSmRhTlJ1R0pzN1FUM2NpYmNvZisw?=
 =?utf-8?B?T2w1YkNibXVKTkQ4RFRzbmw4eUFCNlltUm9BVFdiSHk0anlkc1JMS0g4aW5w?=
 =?utf-8?B?T3BBNlJrWUIzKzRyd1lwL1FyMU1pQ3ozU2RaSnpLMlltLzkzaFd1eUl2cWxU?=
 =?utf-8?B?OHY2T1J2SWkzTGErVXpnZ3NzSjlqMmhMZU02ak4vVCs1M0VySjNIUG9oV1pi?=
 =?utf-8?B?RERhUkZPNVB5R0ZJeis3b0krWlZ2NHpKc2h4c1J4M3NycHFuNWUyQU1QdjRV?=
 =?utf-8?B?SjQxN0RrSzdGWGlGRFZLRnZ4SlE4MHI1Qk1lb1JQb0l4bjF1UGRUQ1ZRcGFz?=
 =?utf-8?B?V2hEdnYxejFqallTUjdLTlQraHIybk5CMkg0U1pBUE5NSlorRlBuZTh6WnZS?=
 =?utf-8?B?OTNaS0g3WFFDYmJrRTNjYzgyS2xQbUlQcnU2QXIwcHZzUENzRVlKMjFac0FP?=
 =?utf-8?B?aWlKVHMzREZadXpka2doUFNOelN0Sm9PTEkxNHo1dUpOSEQzVXg3Uld2NDNY?=
 =?utf-8?B?MXJsUDN5dkJjVTBQSUlNZGZqeWtFUTZmUlo4THhKbk1pQnAwSGZGRWZXdTdH?=
 =?utf-8?B?WjFxRHUyKytvQnR3TzR6UzkyQytwSnFheFNmVWZVTldCcjk0YjVDYTQ1RmFI?=
 =?utf-8?B?bnlXWWZCM1FtTmNzdkx2RWswcjUzeElLalRZby92cStRSDRxMDl2RUxaVzVH?=
 =?utf-8?B?MGxDRUs0NXd0SGRnQ2UxbDltUi8ycHBCaFdrNG5DcUt6b0JxSkRkSEo0Z2Nr?=
 =?utf-8?B?Y2lINlRhdkdmb3BKUk1jcm5UL2UwT3UrVTBGVmRialhnbndkRFdWRFNyNDB1?=
 =?utf-8?Q?+Hjagl0OfzuLIvLEK6P338Gsiqgyig5RUhpYJUT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 79499601-472e-4206-729e-08d971f7c017
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:05:14.5097
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gc8lOrYo7lU2SoL0ccxfi4KFlLO0RxF1duDS9OWQ17m+Aeln5lzfSr5X8ZswULPYs9TvK/ffQPO5cctfDvWSmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

Generic swiotlb code makes sure to keep the slab count a multiple of the
number of slabs per segment. Yet even without checking whether any such
assumption is made elsewhere, it is easy to see that xen_swiotlb_fixup()
might alter unrelated memory when calling xen_create_contiguous_region()
for the last segment, when that's not a full one - the function acts on
full order-N regions, not individual pages.

Align the slab count suitably when halving it for a retry. Add a build
time check and a runtime one. Replace the no longer useful local
variable "slabs" by an "order" one calculated just once, outside of the
loop. Re-use "order" for calculating "dma_bits", and change the type of
the latter as well as the one of "i" while touching this anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -106,27 +106,26 @@ static int is_xen_swiotlb_buffer(struct
 
 static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
 {
-	int i, rc;
-	int dma_bits;
+	int rc;
+	unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
+	unsigned int i, dma_bits = order + PAGE_SHIFT;
 	dma_addr_t dma_handle;
 	phys_addr_t p = virt_to_phys(buf);
 
-	dma_bits = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT) + PAGE_SHIFT;
+	BUILD_BUG_ON(IO_TLB_SEGSIZE & (IO_TLB_SEGSIZE - 1));
+	BUG_ON(nslabs % IO_TLB_SEGSIZE);
 
 	i = 0;
 	do {
-		int slabs = min(nslabs - i, (unsigned long)IO_TLB_SEGSIZE);
-
 		do {
 			rc = xen_create_contiguous_region(
-				p + (i << IO_TLB_SHIFT),
-				get_order(slabs << IO_TLB_SHIFT),
+				p + (i << IO_TLB_SHIFT), order,
 				dma_bits, &dma_handle);
 		} while (rc && dma_bits++ < MAX_DMA_BITS);
 		if (rc)
 			return rc;
 
-		i += slabs;
+		i += IO_TLB_SEGSIZE;
 	} while (i < nslabs);
 	return 0;
 }
@@ -210,7 +209,7 @@ retry:
 error:
 	if (repeat--) {
 		/* Min is 2MB */
-		nslabs = max(1024UL, (nslabs >> 1));
+		nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
 		bytes = nslabs << IO_TLB_SHIFT;
 		pr_info("Lowering to %luMB\n", bytes >> 20);
 		goto retry;
@@ -245,7 +244,7 @@ retry:
 		memblock_free(__pa(start), PAGE_ALIGN(bytes));
 		if (repeat--) {
 			/* Min is 2MB */
-			nslabs = max(1024UL, (nslabs >> 1));
+			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
 			bytes = nslabs << IO_TLB_SHIFT;
 			pr_info("Lowering to %luMB\n", bytes >> 20);
 			goto retry;



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:05:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:05:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180916.327800 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZr8-0005UP-OX; Tue, 07 Sep 2021 12:05:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180916.327800; Tue, 07 Sep 2021 12:05:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZr8-0005Tw-LE; Tue, 07 Sep 2021 12:05:38 +0000
Received: by outflank-mailman (input) for mailman id 180916;
 Tue, 07 Sep 2021 12:05:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZr8-0005SV-3f
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:05:38 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e906bd04-0fd3-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:05:37 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-Yget3GjuOLGL9PuuNgD8PA-1; Tue, 07 Sep 2021 14:05:35 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 7 Sep
 2021 12:05:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:05:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0015.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 12:05:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e906bd04-0fd3-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016336;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DxJykJMCxYkJPMtlwaWjGcuTROCQKWVJPc/aqT8Eh6U=;
	b=Qqu5Oz1sA54M2K4Aae9GH0tQHO4H7xzdoYjjHAnCzWtw7/R4rMbAreS8CdJV6ETbUDhXaj
	l9w+bJ36CWazF/pfbxaFRz2Dk/noc5Z6iu/i7jHd0PAKnZnoYdddAryOmTzBbiFIFauijf
	Ghy0HL5p5/FiQKQqmvUi1Y81hwb21W8=
X-MC-Unique: Yget3GjuOLGL9PuuNgD8PA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cKFu6bKu5CqvQSsZeH90PTEjoTPDza5A2H7ptd3U9Ujy947lmUS7J7zarX/yebFT2r5weYEr8lIatS5LGhPg2zOjQwTIZQE8y8YaIGX1dZHIaM/J9joZTKEO9i3dNn/5nk+uzdaIKHEad0GzZmATYPIcpmBg+W2D4adXutXdMNq/2yh+YkWo/m9o0xiKBY00pVH27Oa4MoMze2m5zk9BPcqIwXpl8W40PSkqHrN4WxJcq7MvnjGB+Vl1BPJ6kf0uJwFAGoaUc5dxXC5vwYVnYteO7BLsG6Rk9Cf1XajP9ETMFT2jKq1qKZk9CLnVHuc+KYF2WQkqUsnjSr2agYETcA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DxJykJMCxYkJPMtlwaWjGcuTROCQKWVJPc/aqT8Eh6U=;
 b=CtXfxloLrJf41/XtEmZDN/7JY/k3ELuF3QgwEngr7bj2W61JXzXipKAJSXG+YqhWSeHRnO9yVVaELJUZ2AdRYhve5uT/0U9cr8Tf/tA+fd/N+WrL1JdEHSfoXd4X0S/64YD1J3/sAVKBOIGVTkHhrdWj0jCzTsj4CiYkqIss/c2XxK9pS2fuxCofiQfaxhOvR97+PK7QVUjaNlskqLyhdZ/nLmRRpRSMuhUfpwzmxYO0EUHjjZpp8sLsbKg+Jpx5PZh9LlmIzrOVfnydcmeHHkp7iVhKQfBrWVKsVQSkiJ7dNNgSE1pnSySVFgIHFYjlyEp1mcgYn+VoP9LygBUMxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
Date: Tue, 7 Sep 2021 14:05:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0015.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2721335c-c207-4d39-3699-08d971f7cbd0
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260769EFB5BFCBDD444DEC66B3D39@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:949;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1AXwpce2ZTGO0h99O41c6qeudTOS+SRO5wPXSIxeSQqn5xKaa/HDMhF7EzO7rOdJPRMv2vJE1sNwkZFL8qbuK8uNsuYMLyJdsfF87d0WYUjUjM9miel40AeQk+cbBAerKT0JKnXLIgPdRTsiipsSTcGPJV9lZN7N3Oe14dMHC/XLleob+6brqFCytjBJrW6A9/yycffL6tDeuxCBHAS0XBzDRKSNdePmn/kDCYYMvubYBHb+6AYctCdtlsC/UeIaC2TStMc37mPlDLv0juBwLbPW7OzIrWUNSJxUvzztahnzfzd43TTmAZaWACfgUSULu7MmBia4ca5AkAWk6M1zXvQ5fa8qbyFKpEM34Opcfh6vtboKa+BUHeUC4Irl7HQdCX++AU6rEie4b8aaSsvSGjUxcJq3dWzCmYf1oalVs61CHP6qcglBAghz4pDtdZdOisHlKtrfhUb+kA2TOzgipqZ19ThXMilurWRYOI4pj9lLVqB0CHYMQjtDKpHibxDoNdWc8zMT+LghZIYcqELm6I9OLL6GVH29IEZi1Bj15pMSxpO60L3aWOXB4bJN26W1xhuMBdWzufo6mGvbs7bdhe2SxpMh2Og0pHiVYPiWkxptBa2ZYSzWofFykcKWPuhnktvSf+obPEBLGBG68hKCiyK3Yb51hz2bRTFKIvYsPKX213vg+HAcIkRjP/7ZENrZaP7ydwuEKG7SMwuD+pYImGcDcZsbI+D0BSySFIZ0Ow8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(346002)(136003)(376002)(396003)(186003)(83380400001)(66476007)(6486002)(31686004)(66946007)(66556008)(26005)(110136005)(54906003)(316002)(16576012)(36756003)(478600001)(2906002)(8936002)(4326008)(86362001)(2616005)(38100700002)(31696002)(956004)(5660300002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UWpXVWZkanNhOERzNlNSV2oxdUJ1dEg5cEdmUm1IbTUzSHF0ckJjTkJXdnR6?=
 =?utf-8?B?MHRjY2FoTTRXK3ZxM1IxTnUyeXIwSTNtWjlFWjA2T3ErRmF0cEsrSW1rWnpC?=
 =?utf-8?B?R2c3QnVITitqdlppVkZHV1kzM1hIZjlGV2FZWWVkSlBjcWFPNHFIcEZFRExE?=
 =?utf-8?B?ODBXNlBqMEZDYnM4MFZSSGlndlZiemsxRVozRmFJb2NUeVF0TTAxeVJiNU9z?=
 =?utf-8?B?YWxYTE1BV2dBek9STGRMYzhOVjNHZFo5TXNFOGhzeHU4aVN0U0ZObzJNNGUv?=
 =?utf-8?B?T2pBUGpxaVU1WTArV1RUbkhzdFltdUczcXpPRnkyUlJaeWlnV1V3OHVVUnFT?=
 =?utf-8?B?aEQ5R2VBQmJaRDNTOXd0UnJFZlNIRVJRQWRVVDQrZkpTZFBUWkFodzFWRFFZ?=
 =?utf-8?B?Z2VrN3NjZUkwTjluRTlkUHUrKzQvWGVUalZTeFV4NjhXZ1pQbHp5SW1mSlZ1?=
 =?utf-8?B?S3FwSXdsYzlXMTc2MkdDNmh2cHFqVlNoeTNVeDJRcHNIUkwvRjY3bjM1NTBs?=
 =?utf-8?B?RXFEUmdrd3c5ckxLWUpPRXFLd2Zhc1EyOWoxbWlaQkIwS3lqdU9XR2N0UVNI?=
 =?utf-8?B?RmRvTUQ1aVp0WGI0alRZdGF5MUZCaS9RSnBIWmhKR3hPZXBLdjRMMFQ0MWlN?=
 =?utf-8?B?bnE1ODdqdlVUVzZkakJJSlpQQ0JyWGN1THY2eTVLK2d4dHBReTRrZjd6NWNh?=
 =?utf-8?B?TnVXTW5TRjRDSzVMeWZ4VjNwaHhHWFJjUTV0WWhwa1BkdzVMSTlxSGNMUVdG?=
 =?utf-8?B?bVU3NnI0NG01ME43ZU5JSFJQcHhNQzdjbHQ3THpHZlFuOE5IMmloR0J0WENx?=
 =?utf-8?B?YkVsM2dsbExQRVpaRmhPZnJiYkhxUGR6SFdtazg5QUM1SEY2ZWw5dWJNaWVR?=
 =?utf-8?B?WHY0NFJ6VC9wNFJxdW02SXM1ZmpzMlhkdWhISEtXT2JaSVRGMFB4U0I5WGZM?=
 =?utf-8?B?Ly9LRFhCMUVsSVhDOVMzMDgzM0FOY0xWSVUxQ2ZMdGE2cU1oZnE2NFNuNHBn?=
 =?utf-8?B?bHNNQlJRWWVVKzZuYlQ5YWpOSWVEMGRVV1UzZlcrSGFFL3A1K1FiQWVzU1lW?=
 =?utf-8?B?ekRyRmIyK0Q5MDBTRk1YYU91VnlMb3hmOGRDN1J2M094aHZSTnpUallwQzAz?=
 =?utf-8?B?bGNqLzZCODdyUTg0WGJwV1c3dmM0V0FIWlBSV092Ri9JRzZtcVJvbDNKbm1R?=
 =?utf-8?B?WFhmUXUvR1pBSklyVjIrd1VjOHNKN1RJVFhGVFA5cDZDM1N2WVRldm1ZU2JY?=
 =?utf-8?B?SHlNOEJ5dXpEcW1jM1lCVTlVamR2Sy9HWDNwNFpvM05Nc3FvVnRMc09DVmxp?=
 =?utf-8?B?S3Myd0tKNjhUL2hNSTRGdjd2UlRlYzk3TjlsOCtHVWJNT24zV3JINzlkem9X?=
 =?utf-8?B?QWpURm5CQkNBTVYwWm56SmFPN1B4d2hPdlFTWGcvWEZDWFIxcWVwY2xTNE8w?=
 =?utf-8?B?M3RoQnNJVnljZUI4QlRncWdIVkM1cHpCNjE4TnBFTWNqVGFSc1Y2VWZvUVMw?=
 =?utf-8?B?Z3F6NHZmT1pscklGRFA3WC9QbU1pVVhUb01EcWsxazkrTVpKRkNPVHMzRkhw?=
 =?utf-8?B?ODA1UjhGRlp6UXNwM0IrMUZEeWxQZVVwQndLYXpEdkFMcXNaS1ZkY3JZMXgr?=
 =?utf-8?B?ek9SZkVSUC9WUVdaSHI0WHNsMEF6Q2R6cmY1VlZ1WllqL0NScmhPeUZOUEdJ?=
 =?utf-8?B?ZHBxZE5Zdk1oOXBaYnM3QWlOSjZZTHpaUk01bEEvYmsxTnFNT2xBbHk2VDJO?=
 =?utf-8?Q?FfRMgDCFQ8QOyre1IYTAe5LHqbtM0mrGv1R0ZK6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2721335c-c207-4d39-3699-08d971f7cbd0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:05:34.1086
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9EeDANFU13xO+PQ/cQU5CxtYQasUAhqSGaUyX2pGNe+3fdUmKBirWBvRaL+rEloewYWIohN+bVIM7JdbyIpM1Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

While the hypervisor hasn't been enforcing this, we would still better
avoid issuing requests with GFNs not aligned to the requested order.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I wonder how useful it is to include the alignment in the panic()
message. I further wonder how useful it is to wrap "bytes" in
PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
least was supposed to be, prior to "swiotlb-xen: maintain slab count
properly").

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -231,10 +231,10 @@ retry:
 	/*
 	 * Get IO TLB memory from any location.
 	 */
-	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
+	start = memblock_alloc(PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
 	if (!start)
-		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
-		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
+		panic("%s: Failed to allocate %lu bytes align=%#x\n",
+		      __func__, PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
 
 	/*
 	 * And replace that memory with pages under 4GB.



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:06:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:06:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180921.327810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZrY-0006C9-27; Tue, 07 Sep 2021 12:06:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180921.327810; Tue, 07 Sep 2021 12:06:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZrX-0006C0-V9; Tue, 07 Sep 2021 12:06:03 +0000
Received: by outflank-mailman (input) for mailman id 180921;
 Tue, 07 Sep 2021 12:06:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZrW-0006BD-A7
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:06:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a7f0c699-b15d-4fbb-909d-2d8db2c1a257;
 Tue, 07 Sep 2021 12:06:01 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2059.outbound.protection.outlook.com [104.47.0.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-5mGp0lRPMUSPJGnZU8v3Vg-1;
 Tue, 07 Sep 2021 14:05:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 12:05:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:05:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4P190CA0023.EURP190.PROD.OUTLOOK.COM (2603:10a6:200:56::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:05:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7f0c699-b15d-4fbb-909d-2d8db2c1a257
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016360;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NjKaEaYjLVTnjJRE9LIByPoKIietul+QbtKOuyICMWI=;
	b=cUwoMCB0JoLefXLknZuXXBQIJi2/XEdAFclZm/Zcllba9pufqDSwOkqt7B6Z6tDzIFr9Wo
	lY8bLPEjAe8CE6xWUSU8qbz1dTLBNvcB7o7MgOAiwK40kEPiH7dRZ48efHrTxDjmoJBBrt
	NrUWPAdnctsWyXyBt+auDKifvhMtPbQ=
X-MC-Unique: 5mGp0lRPMUSPJGnZU8v3Vg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bNH5CqiSQp2WP6YKX8CS01lPsyYKYZT7W5c0oUP9S5nU8vKEtvfKQLRsQBoz7OMNWFWiz6mObGWAbr6J26UOBx5OrdEebihoh+kiAhdSz/NhDUK7SZS978VbN3Y1l+azXLJqgivXL2N/WcsiWyWH72DqZvj8AD2Xudo4VcPxQyVTsFTkEfN+zTfrjuZNmU/dfeWarcNsjWkjOzDozapymktu+CyTRKajKP4lXanUY2r4CjxTJV4G7XAFcyrhKBPs5zVr+QTyQVMi9yP6Mc7If81xBOqpk6e5kQ7VHiyZ+d0Q8cs1nyqVM9ia3jxcUwikI3uElc6m8bUOftFU6i4uuw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NjKaEaYjLVTnjJRE9LIByPoKIietul+QbtKOuyICMWI=;
 b=Nys53cUyRC4XUlRou/gwDBTvIY3i4CX8U2pwSSON9XCInlyMsBEGQj9YeSpo1MKluf09GQxJvzjxvbtcMCR/Zp6Yl+kOV0hHphSkcwySrlozoJvVq4g/gfQhjj5dC2iY1QgoUgM390KQG3oML5vbCHA4ix95aPI+U0+rLdwEDbspS9ZQleRwtKsDnpQ8sIwq9L8gGtKjY4JJddUn6HcOM+tzifnuOe8MoCUCUOeoDKBHKVtcE46N3yaJ0MKO5gUjXoJQcR3FVD4lqFa0PewzidXcmx9SpVq/M7fk/v1kcq+sMHnLds9ppVZHfI85nlj8boDrfGXDOn4SA2moeAMApg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 05/12] swiotlb-xen: suppress certain init retries
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <56477481-87da-4962-9661-5e1b277efde0@suse.com>
Date: Tue, 7 Sep 2021 14:05:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4P190CA0023.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:200:56::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f04244bf-cb78-4275-9e76-08d971f7d90e
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23351E1C52F90D85F485CDEEB3D39@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+XXOKh8+bG+9GNKJSyR8gXikRgeL++Ny28srIttNOVOeijUGZBfzPwMh+z0MLJBFgQQ7kgTuADnedHnaBZL5ahrRWrzf3Wp2r1paBEBVQutU9HOHLX6S/e6DBWJWKyrRPM5KFUY7ihst2GiruxreH0bHM9VWvZUM4QT+asbyJtaRUWZHNNSgrP3IYAEQVjVbUBXWUs4QyNLEU9OhF8i7fuUtQYK2ULLbpG95a0oAaow87Q1aeS1BmXDCB7CYwRN/tpT0fxYGN0jN/HiZ0EZqjF3F9k0PPacX7myFAglxZPcq7/mo7r1sdKTj9XgAHnaSoXewSUYkJOsxf7v184X+uJEhg2j5fNYsLxkBqhmMwpYvt5Xu1O07KxQn5imA6LAIFlQlNvcUqdF1HuPeJ3uZuEqP/VwIVi4glJ8kieRBKFxRxFohcLd/BUhkwE84aQ/AFy6gaAVWAniYnVhRAzvMLbbgLZLuNri2iGsf+pLcuLiXWtR3946qFQKidWdrjv/eD5SAyOCe8ihKlpLQR9cf8Mntt3ktVG7aOK+fH+AURt7gb2KNbFiUFGUEobDyDufRv0pUMbgQHa6sVfZtw7ZkjHQrlqLkn74guDwBDxjsJRn98l850b6F1Cghc9y4XmYCMuEmR+1qAMQPIMTJA+m7d/ANDqXy5tv447enrN1fn65Cdx+jZ1iyIPBOE8TXD3Q/ybzo/UbI11Yog7LitPVnhOtvoFcZtxEiNpmGCzPfnzE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(2616005)(8676002)(5660300002)(38100700002)(54906003)(110136005)(83380400001)(956004)(66476007)(66556008)(66946007)(4744005)(26005)(316002)(6486002)(31686004)(8936002)(16576012)(36756003)(4326008)(2906002)(478600001)(186003)(86362001)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RWJRN0wydFlEbjQ2T2tyOGpBTk4wQnBEOEc4dlBtTzIyZ3Vqa0szVDh3ODZz?=
 =?utf-8?B?ck1rVWpvTklNVXFPbWwrSUkvN29MeFB0WkVvNUg1MnFOc29VY2V2MGdjV1FZ?=
 =?utf-8?B?ZzJ1cmZ1RnBnR1lSTWNhNitwQ3p6c0hvRlJHTnBlK2JTV0g5VnhOaWxoZXp3?=
 =?utf-8?B?clNSSVdjQ3NZMS9MSDh1YzlwRFpQQkZpU2ZnT2xIS081cmlzSHllZjNjTUYv?=
 =?utf-8?B?VkZ0N21CUlBoamRHcDNzS0dtbHkyTTRDSmFjdHJBN29IM0VMMGR3ZE9kV1pE?=
 =?utf-8?B?bktYVDJneFBxQjNwbkZLR3EzSzNDOFArN2lLUnFoRHFyNXdJWGhKVEhrZzBR?=
 =?utf-8?B?Mjh2WDZjV05oWGVEWEJrcCtla1dFZ3VHL3htMDduVU5mcy82Wlc0MnNvUmJx?=
 =?utf-8?B?OS9UVWlOVHQvZy9Uek8xSXZYVUdQaHBTditXdGZjbTdIV3BZTTJJaHg5RVJG?=
 =?utf-8?B?UjhlOWRnRzlvcTNlOWt4Z2Z3OWdyVkVvNDBURnluWXVmeEhZSGc0NUdRdUhB?=
 =?utf-8?B?ekhFckxZZDNNeUpzRHo5RUx5NUFhNFN4djV6bmxiTDBXTmJHOXA5V1NDQkZE?=
 =?utf-8?B?NjVKUmF6QzBvMVZ4QzdITlBTRDg1UHRxcHpGNTl6QWR3bTk3Z29DM09rNTFU?=
 =?utf-8?B?V2ZhZlhwZG5lNmhDN3JndVFGY2lFelJGbW4rRFlraFlsbFRvdDRnOWVpbTVY?=
 =?utf-8?B?c1J4MmxzMm5OVzdxcE03K29kcEUvY0FBcGlIRjBZckQyVk92MGNDcDlLWTBL?=
 =?utf-8?B?Vjk3SWp0UThENExzby9uNVQzcXNhSTgxSnRDQUwrS0swR1Q3NzRpSU4zWk83?=
 =?utf-8?B?Mlkya1B1R3R1d3ZJdU5wQ0cvV1RHZENYZEhDeXBvc20yN1I0UkQ4bS9UM0Zk?=
 =?utf-8?B?YzFFZjV3NWNxbXBpVVd0Yi9CNWFhTW5PSU0wd0VLbmdXWEd6aXNzNEtwcGg3?=
 =?utf-8?B?YXhiRkc2KzZOZmxZeG9rT0J4WGZ1bENwRkd0SWFmTk9NMnJmcGxDRFdrTm5t?=
 =?utf-8?B?UVJUajZMMW03S3J1Nkw4VmozUmFlbnVRak54cmszc0Ria0NJYUZaS2pSbzNF?=
 =?utf-8?B?Uy9XeGxZVWlFQWNGNlVRQXJCK1U0cGNRR2JoTWZxU0xIcjZqS3AvUkozUFNu?=
 =?utf-8?B?ODk0YjZ0NkxrRmtha0hhNk9JeDQwOUpzWU1YcGxyYTZhMGY4TnZXSzBWcHVH?=
 =?utf-8?B?ejI5NXllUHZzNzZIWG5zSUZJTytRdjdnUHdiUEdmM2paU2JrcG5jcEJoZWlz?=
 =?utf-8?B?L2dQWjBPVDVJa2NWZHA5L2svY1pYZVdEcitTVEZzTzRHRHlnVGIza0E3bUM3?=
 =?utf-8?B?cHplZHVSZ2RIbHdzUmVXWm9xSDhwbmhoRjdnKzByL2U2WC94d0lGUWVXR2Zv?=
 =?utf-8?B?cVp2S3BUT05xUm9kQ08wSTBDTWw0V25VNFdlczN2dlU4OW5uRkJNRFREM0FX?=
 =?utf-8?B?OW9KaEc0eDgrOS9YNklCQlFOZVg1aDQxdCtMZG95WVMzVGVXYVphcUZZZGFa?=
 =?utf-8?B?eDZMRnJlb3djMzc5Rlp1RDZqNDVMSjk1UEdlYTQ2SVJsUzEzZHVDWGZzUFZM?=
 =?utf-8?B?MlFYRXhFcnYwZ1dPR3E5N3JSbGRtNENPL2p4aVZIVjNUTE42NUlPTXJzNU5z?=
 =?utf-8?B?Q2FLWXl4dHEzbCtBMjV0M2NwT1R3bzR0Zy94S1Rlbm9VMTNoZVR3MjVUbWRt?=
 =?utf-8?B?NUlvRTNMNDdVanV1RHQ1V01qMlYxcVBIU1BLckFoaHQ5bW5waDBxWWE4YURQ?=
 =?utf-8?Q?I3Wn8j4IwfjvVvww4WJllM1ljpGL55qNYjwmzUi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f04244bf-cb78-4275-9e76-08d971f7d90e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:05:56.3159
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aanrS8Dgx2smsMpI0qTYptKbqhrbXgSx90Dq+sCf5bekkXCEYkwfljdlYZqNb2Whgr2xX0DrgEcWMDg0QGCsiA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

Only on the 2nd of the paths leading to xen_swiotlb_init()'s "error"
label it is useful to retry the allocation; the first one did already
iterate through all possible order values.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm not convinced of the (lack of) indentation of the label, but I made
the new one matzch the existing one.

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -184,7 +184,7 @@ retry:
 		order--;
 	}
 	if (!start)
-		goto error;
+		goto exit;
 	if (order != get_order(bytes)) {
 		pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
 			(PAGE_SIZE << order) >> 20);
@@ -214,6 +214,7 @@ error:
 		pr_info("Lowering to %luMB\n", bytes >> 20);
 		goto retry;
 	}
+exit:
 	pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
 	return rc;
 }



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:06:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:06:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180934.327821 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsH-0006xY-Ei; Tue, 07 Sep 2021 12:06:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180934.327821; Tue, 07 Sep 2021 12:06:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsH-0006xR-Bm; Tue, 07 Sep 2021 12:06:49 +0000
Received: by outflank-mailman (input) for mailman id 180934;
 Tue, 07 Sep 2021 12:06:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZsF-0006wJ-Nn
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:06:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0f8a6625-0fd4-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:06:42 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2059.outbound.protection.outlook.com [104.47.0.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-Gwi4fgkDP9eCN2J82GTPhw-1; Tue, 07 Sep 2021 14:06:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 12:06:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:06:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0018.eurprd01.prod.exchangelabs.com (2603:10a6:102::31) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 7 Sep 2021 12:06:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f8a6625-0fd4-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016401;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YMBMI5yUqKk+WNG37WLHu6TzQ77qqIKUpYH+tOuAlHI=;
	b=LnltSzSpfkqB1ibig/iG80wXL18RO73FeO25lVZsLxex56VSyOmH4QOhOCVtUDNH6ScK+W
	SQyvcRT21WXI+N4QkR1CfJsqAa9Gfeo/mpZpB93u/WcGvxLrAe1/WVw07zH01AlTGkc7Ma
	MeCTxdk/dHUFpg6kdaO30u2aQPj2dIY=
X-MC-Unique: Gwi4fgkDP9eCN2J82GTPhw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R5vf+Dc/6u6YP3qATr/SVEHIz9bTHZ3VENpfiCtFFWXEH8JhbC3SvAzFgo2AsPHCnSqewi4aCG0TsaossKwDSp4vcwkPZiED6I60CRCdCnz8xe2+9fTkd9FGszqFOno38UG7/3kg5Po47bKu0MzzbDJk1nO+HTBbCYybDfPb8xwOdq1jsX+b2MBCgsMHvCDZH+izbK5C0wBzUshPCtqTlbONfFdfFslSVP++oOywbU8wga++rYbtRgHbVeKFNsR6S+tRH+FBsStCjK2glfGyPOUp7tlJd3ncF3jzDn6pvv5M8lKHYukNiMnPUaDZjg67Bb1+HaGB+4hUjNPW8DM0ww==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YMBMI5yUqKk+WNG37WLHu6TzQ77qqIKUpYH+tOuAlHI=;
 b=Da6k5Ye+k2Vi9VLypqWRuZSkQeyMNC+cSSE39zUAiLRkduFYRJ3Bx5HXYpzuxn6z/Tk7Gnf/UK4K2r9WKArxDnvYrRrqoMfgPGrPsTvDoRcchKHZCaphgXPD0w+fXVBrx/fgw2JQDHbp0MW5OY87SxLN5sFoiroofFTW7R1E0Vm8KrvWpfbyzjmRXGi2smY9IEx1eE3dDobPr75vbuDhKPLLDN3K1ehhTyeytv/IWfTFIcsnu0TKeDRWia52VdW7I5R8beUl4Tyc/+AucGSR3VPHqVio0yMVJEqMEnzSv7HD2CFLKfI+LCi9w5374GCATCSnxc71nuf/1v0RU2ifdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 06/12] swiotlb-xen: limit init retries
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <984fa426-2b7b-4b77-5ce8-766619575b7f@suse.com>
Date: Tue, 7 Sep 2021 14:06:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0018.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e87e0e46-93b6-442d-684d-08d971f7f265
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233587B1C1670D1B539CC167B3D39@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6p+jxvZNvlchQnDd2qOjDZwFxL66V8vytE1+ZIOz6IgenMe4qgcjUe2VdKNwxqWWpLmxNEO5itjH8OxfIp8DFLXER0IjypKgRhABvRwDn8jh42qUTzm1R+jdDvtBezSvx07I8YMpvzR3kXvZ5VLOu0QfEGNa21xb9gd8Y6brn7Kz1P5EKPdnj2xtXXvF35oHvaWdQV1iy2G94lY8JfW+PIGvRddoJqYS8LHMu+W50VF+IGO+CFdt4bkua/aDTBtnCYoiVrptQR6f8nBqnEP/xgL4ZyXPe2gQ6YaGvb+jhb5ha5eSH+3M9ctVX2ppJ+9twQGNwtEwSPFpWkl/1PDLmcT+ndTFg9Rp4osmZXpd0pF3SQbdp83JFPNST9s8Ik4Tr6xh+YktesFbg6qZNqrec9yBPDqT8PqTByhVzvko9xDAAEhzGtoW+uItXp1fiI6QKxMAaTC9x8C1WhA2DvRew+fNCR04p845+JOSXgpZDB+NmM4aBaKMuHH5lf1vLCkQom3BVDZNn/A3CoXiRLxIjEOQD1ctXk35BEBu2PeIpUxKp5fGXvJe2iw+zLl9DI360OHCsOI8EuubFxDe5bESM9RLe+1EC6xYEgROXhIW64IIcm0VxcfcoJq05iFOt7JSRAyS0GVoWxz+zyaAYe3M/wVcfkmhi6bMYMhxHOofggi0Ej/aKfwNi8cl1eunZJWXFAW4JXx/YSAd+7sxKdz6Kw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(2616005)(8676002)(5660300002)(38100700002)(54906003)(110136005)(83380400001)(956004)(66476007)(66556008)(66946007)(4744005)(26005)(316002)(6486002)(31686004)(8936002)(16576012)(36756003)(4326008)(2906002)(478600001)(186003)(86362001)(31696002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SVR3TXNZeUpyTWZVWGIzdUhYM1E2aUNtTUFSeHdLVy9WMEd5REdBcVM0bkRp?=
 =?utf-8?B?dW4wc2crNU1xTUFhOVIvcHgwdnJwMWVJSk1sTUJFTW5mY2NMUmZlUWxrYWh2?=
 =?utf-8?B?SHZtZzFqQndVcjRVWFlLOWY2MGtMUHRFTG05cGF5QTNlekR4U1drT3JycUtL?=
 =?utf-8?B?WW5ZMXEzekkrK09PNkRaWUZrdTFEb2NVaXR4emhlanV5TFBGYTUzWStuUnFv?=
 =?utf-8?B?RzgycExKUmgyZ01pZ1N1dW41VDlEQkN6UFlncUFXR3R5U1lkbEw1TThMRmc5?=
 =?utf-8?B?cGZRMEE3YzdjOGRyQ1VoMWxTQXR1VGtKckJra2dEKzdRNDNtOFJnTUJHTnZv?=
 =?utf-8?B?M0hBTldBRmdkNUVmbmNGRFZDZlpQQWoyOXR2SldLLytTVmd1WmVBYTRLYTdq?=
 =?utf-8?B?ZTdkYzhwOWQyOFQraEptdlpVcUNON1cwNFBYcDBXNXR2dGc4V3d5ckwrUDNE?=
 =?utf-8?B?cksyY040TFAyMnB0cm9BSllvVjl2bFNTVlg2cmx5bUxDUUwrYzAwQ1V6M0xq?=
 =?utf-8?B?cUVGdXdGS095d2hMUWtCazZJZGJIdWl6V1A1dHc3czBZTkpDb2p2MEVLdHhU?=
 =?utf-8?B?dndxcndZN1dKTjlnc2k0Vm1vOGFObzBld3BXR3phUDFzMFZzMk1lV1VaRllV?=
 =?utf-8?B?MmE2L2k5YitjRUNVSnFGc1paRTkwdVZERVQ5a200bzc0dmR4R2pTT3VBTkFn?=
 =?utf-8?B?K094MStTWjc0TGNUNS8rek5JL3JPdnAvSk1xaHhidHYyNjdQWW9LZ3M5dFRp?=
 =?utf-8?B?NDU1c3Jzb04zVk9Ud3ZneFZFUWpvdzRvZng3dE9tWVdXd0E3UENRVkFPVmdp?=
 =?utf-8?B?S0Jqam9zWnYxRUY1Y0hpU2p1THBOTmNKRWc0eHB1Z29NeFVKOWNsY2dieXl5?=
 =?utf-8?B?cDk0UVF4em9sZks2VGFZZ3pnZDE4bmVHM1o5VDY1b1FEdzMwUkI4VXp1TUhL?=
 =?utf-8?B?ejJOOUJ6QW9DNUYwSDF3VTNHNnBIcm1wbXlCbXQyQktqRkltZ3I2U0sxTHJT?=
 =?utf-8?B?aDZkTHVsVFpnMGl3eDhQTFhIS0ZqOHpuZm1TY0xuZDRxdEwxOCs4ZlN0Tkx5?=
 =?utf-8?B?NjBrSUtzc1h5eWQwSkpXbE5GZHZtekhhajNzWjhWeGFVSGN5emY0Z0tBMlNE?=
 =?utf-8?B?ZnNOSFVvS3hUdm1wZC9GOWV0MlFVU3dQcThWZ1YwTzVWOEdMWGRqS1BBQVRF?=
 =?utf-8?B?QzBYRUsxSzZpbERPQkZHdFptSjRWTFlsQ2JDRzdWNkZqZ2UvVllnZEtxbUY2?=
 =?utf-8?B?UTR1ZGs1RnlwL0FPeUJjeXA4L2ZMN0hoeTc4VFFjcDZ2clIvRHgvUHd1WmdS?=
 =?utf-8?B?Q1pIMStKRldZRFk0NHM0VHhta2R1R3hrNG1NUHlJbjhYRm9HaGFGczV3OE1T?=
 =?utf-8?B?UmhLQW1RN2wxeTdFV0VSZmJVNytBZFVTZjlNeVlSYktQWXNvbkFsVENtNzl4?=
 =?utf-8?B?cXB5NnVaRmJ0aU9lQlJtRUdoQlRYcTlualQ0YmpFVi9KSHhteU10Z040TVo4?=
 =?utf-8?B?d21PT3duWmp0cUtRUzZvQk95RG1jYmZ3VzZqcUExSjJzNjV4dFY4Qk94K0tt?=
 =?utf-8?B?T2FoQUQ5c3JjVTh1SWkrcWR2MWdjcFd6NUU5VUdWRW1xZWhjT1N2VFJWWGVN?=
 =?utf-8?B?MUJlVUxYaURCbUZJajV6SW5tTStLVE1BQ051aDBlSS96Yy9WeXVNN1BBZktq?=
 =?utf-8?B?Vm1zMEhJbnZqbVd3dCtxZjgyMlkxM2RQTmo1NmtucWJXSzYwUU9iNmQxOFd2?=
 =?utf-8?Q?ADFcBANdJhD8VY3lNsPOeRWZU7k3yVq0sCVDFF6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e87e0e46-93b6-442d-684d-08d971f7f265
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:06:38.8548
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Ph2AurSgsXAZiupuP2pvXbNulkvCbwBmtN+9q0Zh1ffPH8Wymh+4lbKcDL4jZkQDrkFawmyk3YKAt4Eu+kBM1g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

Due to the use of max(1024, ...) there's no point retrying (and issuing
bogus log messages) when the number of slabs is already no larger than
this minimum value.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -207,7 +207,7 @@ retry:
 	swiotlb_set_max_segment(PAGE_SIZE);
 	return 0;
 error:
-	if (repeat--) {
+	if (nslabs > 1024 && repeat--) {
 		/* Min is 2MB */
 		nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
 		bytes = nslabs << IO_TLB_SHIFT;
@@ -243,7 +243,7 @@ retry:
 	rc = xen_swiotlb_fixup(start, nslabs);
 	if (rc) {
 		memblock_free(__pa(start), PAGE_ALIGN(bytes));
-		if (repeat--) {
+		if (nslabs > 1024 && repeat--) {
 			/* Min is 2MB */
 			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
 			bytes = nslabs << IO_TLB_SHIFT;



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:07:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:07:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180935.327833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsV-0007MU-OB; Tue, 07 Sep 2021 12:07:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180935.327833; Tue, 07 Sep 2021 12:07:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsV-0007MN-Km; Tue, 07 Sep 2021 12:07:03 +0000
Received: by outflank-mailman (input) for mailman id 180935;
 Tue, 07 Sep 2021 12:07:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZsT-0007LE-GV
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:07:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c7167cc5-da60-41dd-964f-0f0b7ed46926;
 Tue, 07 Sep 2021 12:07:00 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2058.outbound.protection.outlook.com [104.47.0.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-sqoC9y0uOUmyMVzmjdz1SQ-2;
 Tue, 07 Sep 2021 14:06:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 12:06:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:06:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0063.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:06:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7167cc5-da60-41dd-964f-0f0b7ed46926
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016419;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=yfSrUDJsXrUgIa8eiGS6wO+C14mV3RvK021zMRjhrHg=;
	b=Zq8NCV1PhIRAR6RayonpC9+VvN4XjM8PkzIxG6ZLTq09Ggxv3aIn0GWpSISijyvN1/tV3l
	HmgsHSV/TaB8cXjkuQa49gI27c4OMmcMcp4XDy6wIa2EzsR1mQ6HOIfNsqpZmdRCqVWA6C
	v8w5wXZUFHxMnb2eLFFE8ubNwGIZBS0=
X-MC-Unique: sqoC9y0uOUmyMVzmjdz1SQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jWmrHmi+ZMHC3LYMtzJur7+Dw2pd9sVEpZFYIwzl1En8ZFIaMuvsyXDu46b6O63Xri3ktnl3NG4vEpo2X1NCj+edcilG4n7L8fqrFK3Lg5sPwvvpcEMleoMqsRQXf7gTN8ZR++aOFpoLKWzvFe9WvldIB01WEOVOJOdTLjOGdylsdinh2m88P3Tl9a/wlaN2YtVK2NODSaULwDrGCIaYxaio5NoXg2GOBP+MiTWza1oKNGCbWILjuodfrE0lZsuuVPuT52OOpILXMXlCIKvXLYjOe/8WHc1ruEmLs/SzzlU7GgGNdiOv5UQHlxhqVzNDXXf5vRl9yoShbaCeJwWAkQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=yfSrUDJsXrUgIa8eiGS6wO+C14mV3RvK021zMRjhrHg=;
 b=csNV5nHeNahFGwpU/LYDwy+AT9eP940y29k4YQXmrHVGKLzZG69WDnDPOUtKSLNFRJ45kWRZrH0F0tKhvqFWP9Gsrp4ICQq+I4bxESlQjOfXueGoE5RwmJYVqec3j7dn/11Dj0zuVRnhY4QFIkKGzmwn0A9lGuHrZ7XIwIeQROa9Bou8FpTAsbUaV6ofOsK7OJmI7XiCvHYaqMoQjaS1nHXO57rVItI0QfbN60php0A5+IxAU4M5U93Q6uQmwG8a5AVDmocy4GggdRzdWAF1JbOttbCBfe5hGS26QOEhdyL0HlDQHCwTaYetM3+wE5KmZB7f0hfHMq98EbjQmgqGpw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 07/12] swiotlb-xen: drop leftover __ref
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <7cd163e1-fe13-270b-384c-2708e8273d34@suse.com>
Date: Tue, 7 Sep 2021 14:06:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0063.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1d::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6d92d3fb-5473-4882-8cd6-08d971f7fd10
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23353A656619E82E11D01D10B3D39@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2887;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	X+TnSCKrSsUpHDy62Y4a7n1mHtJ9sHiN2ncLWbETlk/iczTyLciNrb5Q889LTvUlcTFTCatHa/e1EqPzGeQDQpSD+Y1AKhG6Hm24253horOclA2vhn884oXNKEvO9qcIIl8MPmtoatQAbIBjnu9cJRRtb0/ej/lKc5QCx+RcL6RZOQ7+ftwD762dJdvnoSSOeijFyC4Kk6njhHLs+2De3ZLT1GzhybjWv9hupmX9FDFWnRRm7x3D54pT57k3nQt9MqwL2td8mMaX8WPZ8Dpgc1XX2K8BaXZtRu0aMHtBzu8bxWFxTMZUPKCxTZUpqznBthXzUJQ36AdPzD21Jp0CSFz6S+ppSSinkIpVQkWttj+vYttUyddhDV23uNXZws1qy5f6FeLi8cuIrDaxOQwoUFgs4b+X2DYo05kJem9wJeERmpwUMo+GDleGFRqFDXVCbXDug2mEqhBXxifoahtCFOI286Ufw6W7W0+5zytY4VE0bRuHgrAz8LkVRydX3GYHdDIaCxURPWCuHzzbglGi1C5KDTxAXt+qd3q2Qdyp/3wcPAPQOFtloxJJnpeERFIcSVwbxX4oqVJnbddgS61tY+iUGPZZkgupsx4wQ00xAa4Od43WB168COO+sKbqmoa+Gglbu7X0GE2BBtbihFP17xWyzSH5cQFcpVdmVBehNxbioptTeRvXZggT+enSjf2d+31gIvnt7Mi1uN+fqZO0uDIs6NavEMlbVuKnX9ygZ+Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(2616005)(8676002)(5660300002)(38100700002)(54906003)(110136005)(956004)(66476007)(66556008)(66946007)(4744005)(26005)(316002)(6486002)(31686004)(8936002)(16576012)(36756003)(4326008)(2906002)(478600001)(186003)(86362001)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z1c2S2NncjAxVDlZdG9IcmphQXFJNUFlcmlBOW9MT0hCNUJ5YlhVOEJwTkc0?=
 =?utf-8?B?OTVXc1JUQ083RU5Dc1JTSEd1Z1Y2eG9sZVdEQ0VGTTkzajRlbFgrSXJOM0pt?=
 =?utf-8?B?dW13Nm4vWHp3R3NLZUl4L0taRTExSE51bXN6NWxQLzljV2d4Q2E3NWUwWVJY?=
 =?utf-8?B?aHFFQkVMNitXSzdXRWUvRE9RdSthSHNvSURNNXNmUjVDamM2aExKMTVSMUF0?=
 =?utf-8?B?WXVvUVFwd01pc3Fxa0ZYN3E4MG5ldUh5bFdMM2JqYmNSNmVZUVFRamNoaGJK?=
 =?utf-8?B?VVA1RXFNa0FlRmcySmVQeWg5YVlpQ2o3dFJRcENoUStsaVhCMXdaR2F0SW0z?=
 =?utf-8?B?b3FaK2Y3SWxGeGUwMk1zTkl1czNOcGFJZEN3YzNGclVTY0h2eHVONGVOQlE2?=
 =?utf-8?B?VTBWa0lRSjNKRFF0K3RNUnY5TytRMFFDYXF0c0JTTVd5UmZmK3JRL1IrQ1pZ?=
 =?utf-8?B?NkJMdVJVcFE0akFBaDNjcWFIazI5eXVEMDlEWnJwMU9kQisyMzNQdlZudGpC?=
 =?utf-8?B?OTlGZkdlWnRIRDRpdk5ZQlJMSUJid2VSNlhTd2M2RGJzVisvNWRDMUlrTlNR?=
 =?utf-8?B?ZTBYakxpNHdGUFR0cTB3QTVpMzFxRGF2NWJSK0pSb09aKzdKZG1ZakNjQXF3?=
 =?utf-8?B?SEFWV3BwTGdwd0Rjd3RsNzlsUFRHdlNSbjE1WUFRMFNpUzBhMXczY3V3aWo0?=
 =?utf-8?B?bk1KTUl4TGhPVGZ0b3MvTEF5Y1dDbHdYNFFraW9yeDRTMHVEZG96TWFUcnpH?=
 =?utf-8?B?OGpBTVFRb3VjWWQwQ2lyN2I0a1d3WnJFVGtqSnFRd1B6N0xVY1MzVDdrblVw?=
 =?utf-8?B?SWoyOS9VcG1kWUk0bHpvcmc3bDkrYTZxWis2bzcwZGU4dk9yQk0zbmFVQ256?=
 =?utf-8?B?Mnc1bmdZZkNJZzZuRVpvT3hWUnFUTkh1KzlxbGk5YXUreHlERDFWdVJpRUtX?=
 =?utf-8?B?VitiVlRCOXRuMkV1SU56ZTB2RnFkYnZHK0k4NUJCRkxmZHE3dkEvclVkcHpG?=
 =?utf-8?B?a1hqK1QrN0hiOGxXTjdqUU1TbUJmNzBwUjQwS3ErL2htLzd4ZnRZbmMxbkVV?=
 =?utf-8?B?SzZjR1NDb1Z5Uy9ZTm5PQzZDTmJXWmVtM0tvR3lFQWdFMDhXNEt2akl2ZmhO?=
 =?utf-8?B?RWVoNGFKY0lnMUdHN1VGUFBsV2FndUFTbzNINjk3NUdueStkVEdVUCtzdFhO?=
 =?utf-8?B?c2VPQ2xsbGFzV01xdURxM2lXaWVDVXNndWJlYWNaejdVR2N5RDRQYnVROGRJ?=
 =?utf-8?B?bXNGcUIrSU1ZbUxWazRWNkRlazFDbkduL2kyUmVYVGpGS1gxaXZpNlRqVEFy?=
 =?utf-8?B?SXZMS2tseHlDVnIxa0t0Z0R3ZWpUa21hZlJJVkIzekp4QnpYN1NVc09oWnk4?=
 =?utf-8?B?endjNlQ1RUNibWxIOEQxRXRoazhTSFh0QitSMGdnZlFmaXhudUNWZXlka1dT?=
 =?utf-8?B?ZlBVbUpEcmxsNzJ2eVA3T0YyNERpdUZtdnlzNEJKSURSYUFMNFU5SkNlSC9i?=
 =?utf-8?B?S1hhTW1OZER2QjlnemEyQkQ3NkN3MlltOHY1UnM1WmV3Z0Z2U3E3aEZXbjEy?=
 =?utf-8?B?ZG1ZZWJWR0gyek5wenM5V3h5YnQreUM3RmhnZ3hLY2hJZTVKZWwvK3hESWxG?=
 =?utf-8?B?dWVLR1BPc2RtQ21LMWFrSWRvMGl4WGlMWHJueTFWcDdWOXFWdmxqMkJ0Vitu?=
 =?utf-8?B?RzY4aE1vTGRmTG1zSXM1RXR4UGhqVHhBajNVaCtvRDFBdmcxejhWbkc2MXpU?=
 =?utf-8?Q?6zpueodSA9C4Hs9FhhGM1mnSgXSbmwbylLEBmaH?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6d92d3fb-5473-4882-8cd6-08d971f7fd10
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:06:56.7266
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: v3JjegjrSVa7Eyffa0vq6FD1jmgD626Xg9ETA8SXF6JHGDsF+T5u/SkYZ9MAPiV1vclz+c5K8UFXntF15X7vlg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

Commit a98f565462f0 ("xen-swiotlb: split xen_swiotlb_init") should not
only have added __init to the split off function, but also should have
dropped __ref from the one left.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -154,7 +154,7 @@ static const char *xen_swiotlb_error(enu
 
 #define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
 
-int __ref xen_swiotlb_init(void)
+int xen_swiotlb_init(void)
 {
 	enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
 	unsigned long bytes = swiotlb_size_or_default();



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:07:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180946.327844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsx-00088p-0s; Tue, 07 Sep 2021 12:07:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180946.327844; Tue, 07 Sep 2021 12:07:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZsw-00088i-U3; Tue, 07 Sep 2021 12:07:30 +0000
Received: by outflank-mailman (input) for mailman id 180946;
 Tue, 07 Sep 2021 12:07:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZsw-00086X-9p
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:07:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2a423eb0-0fd4-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:07:26 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-wO4pVYOwOfOPYqiRTM-_GQ-1; Tue, 07 Sep 2021 14:07:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 12:07:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:07:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0010.eurprd09.prod.outlook.com (2603:10a6:101:16::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:07:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a423eb0-0fd4-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016445;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=n7mzpWDSKQyeB1Y7K6PB6/ey7zolLxcOfxP5VRC4xpg=;
	b=SErut9zMOtkNGyzk/89jVwytfHcYzg+NPNOFJpwQ/WeP7lygv3MgD1ZTMW/C1KHGrllcPA
	k5baRR3ABtSG6pY6EBke79DMgymDIRXYPJnQUYCXlxyYK3JXOxuV9DYpzmaDhDV/Ayzfr3
	oIMGYDbPPqwMmWYC0dhhMeBtjakQw5o=
X-MC-Unique: wO4pVYOwOfOPYqiRTM-_GQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aTIxY8KEDICOriuKX5SuLtaLX8doPexdNM9CYmFxBEanU/cMmnEH74KVzYL39rSFD4cTK9jSwCf4ro+4qMbDkqqBLCOfMNGCux8PMyFvz+3NkZ/exWTw03IZyKMARPSGHmzocZ+YFv8HHvBAAwuxxZbO8mABTODP4EP5WWfk48N/foFbOGqUk0zrhDI3RhKPTgkV79ysNrU+2aYboDr/x9c27H5xTUK3hSOjV8HqIkjw6D2V2cLVL434+YGsBjElA/0ihK3DbZ4gBWofHjC4WAuufdkWFesQopamHg5dOqP/cJsKm6SJZVllpT/O+4hcO70T9szY3ZnX1siBGXbzUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=n7mzpWDSKQyeB1Y7K6PB6/ey7zolLxcOfxP5VRC4xpg=;
 b=bncueqn56HBLtcSH4vHgtaTxs9jKY+re9pppPYPvwbY4YDfvfW6CztYMvYRehgdwJm+YeuRjF0/y9b/kURFHgknYzLDHx1eYCTWmkrd9srrvXilKQIQI0hMH2Z0Yy1ANZo51f77VNnCNZS2p9CDgbP2m0ImH07hJFBtgxUnyOnpK6r3PhuXXV/WgCLAm5ettm1nm1vudbQ+GDSrBPOYlxALXuDiAYgxlcCNH3Q0cllH6xTyFwlCR66dG1heQVXIuOAt63n22daKJhdmYCaxZY3srkXpYSvqdCeym07RAkvGUMU77E6gjpmoiAUoIE5IFKdw7yYSRtAXn5hFNysynwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 08/12] swiotlb-xen: arrange to have buffer info logged
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <2e3c8e68-36b2-4ae9-b829-bf7f75d39d47@suse.com>
Date: Tue, 7 Sep 2021 14:07:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0010.eurprd09.prod.outlook.com
 (2603:10a6:101:16::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: aafa5b43-f735-4db3-d08a-08d971f80cd2
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2335A4D28E390E529E868D9EB3D39@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ALtpvWS75rPd7bTQjpiRwarOVxM9LjLzQJ6rf1JxZt1AVwxcvXpcn45dkxiQKB5ZUE+Dfd4K/QpZyDhhp7N+7Zt703ARx9ZdTLUl0hnpDQTsfpdZ8G97B5IP4PAkoN9Zazpzlxligg9aiCWyR4dJmmyvWppWuubxIxPnBHwiACKW+RGTiDLgTr2KBybIb7slboF4jAW+p9+usuzzYMw2fXMnu/835pKx69fVXcNpLFPpFoKrttRnK7WwGgB2G8fSPBZ7oErBw37nJpIiIk8xVCCDlIdPVHbzeiaSJACMLlBlzvM2yobefjCmEnXr+JaYzcAs4PqT5YIbSUGxa/d8LkTMbFNOZ7yhEACe+8oSkfSo0hfDJirqV7Cp4HoZ+xGPkg5rIjaVWJlNripn3uqwSak6XQVF5/MqlFl+nXwE6lXwpmdPPuQOI2FDKPnOYgyhji1HM8NSX+veqqZnFfPBSyZhKk1MEyGZ8dqzk+hJmyhW7MX6KmRSTs/2pRbbrgoJb8AYA9Bs5OjBo8RF1YovTwjE/hXZZb6f3FCwn7pPfa3dgfW8+me43jsnV27UdJCXt1hwCxVyLYIrxd0doxV1bL2eYL3JW/fmJL8knD6+JeSSk5wvmN8Bu+V49zfnznuSnmVTG75b1u/Hj9Gz21QFNvvt5xIUI38olM787kdrmOWXQKp+TyHZZDRHDxtQgbyYbygJHhIBvtPKzgiP6kEaax7UrrxC66jl5eJv1OJfcA0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(2616005)(8676002)(5660300002)(38100700002)(54906003)(110136005)(956004)(66476007)(66556008)(66946007)(4744005)(26005)(316002)(6486002)(31686004)(8936002)(16576012)(36756003)(4326008)(2906002)(478600001)(186003)(86362001)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M3FoUjNGR1NialF6c2owUENPTlFMM3B1d1psYkV6dmlpUU8xK1ZMc1ZJMGZw?=
 =?utf-8?B?dlZLMHZyWnp2bkpYeFp4STNsV0RPSnpqMFg2WE55MW5wZFZNZ3AyLzdlenpt?=
 =?utf-8?B?NHFvdFRkMnE2QnYzVkdEMGRhTGVwZWZqZk5MbTc5QVlHZEN5YlFYMnRwTWhs?=
 =?utf-8?B?Q3hDWXRZQk1MV2hCdmh6Qkw3TTlOMkZaQU93cDRCTzZBWVBFK0JFZm5NVGJL?=
 =?utf-8?B?ODVMMmF1Tk5DcnpUWHFRa284TzRNaFdiRG1TSEF5dDFWMmdaSVgyaXh6UER3?=
 =?utf-8?B?WVh2Yk9KbC91VXAyZzNDdFR1MjgrQUZTVGtxdWZsKzJncVdES3c4bkdONE54?=
 =?utf-8?B?Q0E0by84VWE5SkdhZkg4aG9scmFhc1NnSW9najRGbzlzaWtCdWkzY0toTkNR?=
 =?utf-8?B?eFRXNGhoMlJpL3hPekRRNCtWM3FoZnpWZklpa2ZLZVVBMUhqZ1V0NGxWd0Ny?=
 =?utf-8?B?a21OTVRaUEJvZHJFVUNRU0k5WGhFdU5MaDYrSEZzbTM0d1pvSGJSajFGcEVJ?=
 =?utf-8?B?SWJUWi9RVFBVSlFpNVQ5WlJMamkvQWNBUXRIQUFDUkRVZ1Q3ekZBYjBiakJa?=
 =?utf-8?B?ckdzRnI2ZXNDbi9td05OWEtZZlpvUDE0YVA5Vkc0bU1zdnlNNTBESDFBcWpi?=
 =?utf-8?B?eHE3a041Mnd3TXRZNTdyOTVKUTlCQndBRUtLWmpFZkhCU1haWFpwSUhFTkRC?=
 =?utf-8?B?L3YvMzU4eER2STc5alk5SmR3bTk3aUp0OWpSNTV2eW5hNlROL3dRbmZtczBU?=
 =?utf-8?B?ZFNkeUswL3JjOU9DSWY4REtDbHA3MCtOMzMrOFpyQVdkcnUrcnByTk5YcGNJ?=
 =?utf-8?B?bjlzVVcyUGx6UmxUWnFRQ3FVczByRUk3TEt4aHN2aUZlUFVuRXZFNzJhYmh0?=
 =?utf-8?B?amR4bGp3UTNVb2pNcW9LVWVOSSs2ejBYL09Rek5XRGtJT1NQR25YOXJuMmc3?=
 =?utf-8?B?MVhSMzVGSUF3SXJrNktuSWVwVUlFRzhCTGF3U3FOZERCTThuVVEvSWxsOTlk?=
 =?utf-8?B?NlpFYWtjTDdITVN3ZmJrNFk1bzcyOGt6MVRuREFraklYSGQxc1RvTXlZOTJm?=
 =?utf-8?B?UmNhK2JOTTlyTUFJRzk3aEFNaURITDNtd1VxMURSY0xaWm1JMXNCS3BLODQx?=
 =?utf-8?B?ck12N1NCcWwvSy9lWllBYzM1YjBjQWQyQVlLSTd2b1J4UFdzN1IrQUlvcnRU?=
 =?utf-8?B?SlFDdUFnVkR0azNQK2svTFZzZlJkR2xGMUp0RDl2MW9ick03cVpzeXVmS09D?=
 =?utf-8?B?ZGpFdUdDaC9rcEU5amVaK0VyalhGVDFrWFBmOUE2VVhzWWY5Tng5RTFGaWpl?=
 =?utf-8?B?d1ZueVl6eU9lZlZ0ZzhXcFQvRXpnbWdQNHJ6V0ZnUjVpZ3BCM0pPK2k5ZDNQ?=
 =?utf-8?B?Mk5Hdnd2cWhFL3JhMGs0S1QwMFRJSFBYZjUveUFMdzVJZy8yUEs5TElxRmU4?=
 =?utf-8?B?T1orZ09GZFgwRzZpWjgxYXJqeUR3TG1BYmVHRWZiMVc3cjkrdHI5dDVWc3Zq?=
 =?utf-8?B?eWFLV01aV1ZQWlIyTmp0L3VidjhYWC8yb0hlOTNHMWVvSGE5a2o3cEYreXRy?=
 =?utf-8?B?RUZjRnBTSEsyejdBS0phNW5Cc2IweDMxOUxtbjNtMFNoeTc2eXljVjYyTnhy?=
 =?utf-8?B?aWIzN1I4TXkxQ1FtNy9MMGRScXpIenJXRVRwdGp6Ymp2bUh4R2hWcDJOYmJW?=
 =?utf-8?B?SkE3anBGL0pUZTlPaGdFSnFUUnlOcFllREk0RkI4ZC92amxhZDJLM2g1VFJv?=
 =?utf-8?Q?orcBUZhClMFZ0qsoioEpPl5cbx/+29amJxrDzSd?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aafa5b43-f735-4db3-d08a-08d971f80cd2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:07:23.1625
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mZFC7s4D4ydJmgfgHq1eN5JHKDnRLeCTZaomrg8SF5gCbPA7ZyH/63UPSSxFvPLLe9wPZN3GKpUD02THWyXLHw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

I consider it unhelpful that address and size of the buffer aren't put
in the log file; it makes diagnosing issues needlessly harder. The
majority of callers of swiotlb_init() also passes 1 for the "verbose"
parameter. 

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -253,7 +253,7 @@ retry:
 		panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
 	}
 
-	if (swiotlb_init_with_tbl(start, nslabs, false))
+	if (swiotlb_init_with_tbl(start, nslabs, true))
 		panic("Cannot allocate SWIOTLB buffer");
 	swiotlb_set_max_segment(PAGE_SIZE);
 }



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:07:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:07:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180953.327855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZtP-0000PX-Eu; Tue, 07 Sep 2021 12:07:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180953.327855; Tue, 07 Sep 2021 12:07:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZtP-0000PQ-Aq; Tue, 07 Sep 2021 12:07:59 +0000
Received: by outflank-mailman (input) for mailman id 180953;
 Tue, 07 Sep 2021 12:07:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZtN-0000ML-Sa
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:07:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 38d5356a-0fd4-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:07:52 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-mTLXMExqNMKKhs3PsuV5vQ-1; Tue, 07 Sep 2021 14:07:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 12:07:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:07:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0247.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 12:07:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38d5356a-0fd4-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016471;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/5oNHcbLsqo9M2lUcfhAAuumf4VaC3DcQDQUojVuRYU=;
	b=G9E91MBOpVZjxwi18BVIxBH8gw+UsdRqt/pYQuVCDK2SKrJYE14LIndIDn+v9/t5oDx5NV
	rfJO8Mq6MFvU03WtBz4se15bUdbwPNxKsw66rMm15z4h5AJXHuNNcOEIAKA7MpIyPrSScW
	vUE4/n0++y/m5L6GfPYpls5wEFCAwWE=
X-MC-Unique: mTLXMExqNMKKhs3PsuV5vQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nbjHdUoBL+11rFy2gY9B7/5mScKhF6tDW8jzwh1MY89QgITtmyPuTtKBW0xg0Vu5cMuSraCSMBABO1E2hXH+30s+8L6rwE2hVLfRVeNDJx/I/zvguIlsa3RWLK1qaiZf43GUfrhsA+On0iuek9DjDTbnx2mL+TgUGnQPXjw2orNJ3zCLRz1ML/2WdmiyQ3uhws1+F1ngN0cWzsSwnk6uv4+opDLPzCW4OwKNmKTIP42kGvU4XDG5toyu7ES825sr59QPhB+buDuo9v8t8OWS8QbsAcUKg0KNL3ibMPAeblCDggQV3uRWL2ocXD7wyJb1BJOigXbJy5nPdRB3iQHxwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/5oNHcbLsqo9M2lUcfhAAuumf4VaC3DcQDQUojVuRYU=;
 b=OYZsiqHbMNHHvzMyUyXzyeazn7kelTM4uQq6MtiiWZxOX6jf2C5kbafG/gmmVLOt0qu2gDfVbiTdyDWPRsrfuBXYTEV1BuHWocO2tolkw2gvouu7XIdzHPaEuS0Vr8Mn3yrx3eQFA6fcNA5lZglT9DtdoZsg3t1gO04cdF/Hu+hjJg++yxteefVe98DrGI6Wm3U4fnVsL4/XjcLDI2HtzFaQ1KR0E9JUVnAISYO/5Y7dj35MUZMpsmnUlcSUrDpC0tvcd1GtzOBglkN0mR2firbjOfEct+BCrCfZR9XJgVCoOj+B1dUKIGIY/UxJtUg6GtvnJwwh0b9Yeqbe4OIYqQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 09/12] swiotlb-xen: drop DEFAULT_NSLABS
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <15259326-209a-1d11-338c-5018dc38abe8@suse.com>
Date: Tue, 7 Sep 2021 14:07:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0247.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::19)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ef7924f7-8733-4ea4-740e-08d971f81c50
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2335403C43EFB474C5E60388B3D39@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:626;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	P0wJiu3kiYlboYr2V/LGfu1A/YJF5KRBBcMhqlGVoTiAr0Vu5FfvfS8TSTyBnK1SyjAtdi3AIWl4NwOE3poSa+zGquSb/JwTvR3yeTkOKcRauRful66g22/Klc4GTLEfGu/VeZqi2Lnrpt04Bywau/h1f874QuFscuwt+0tsKwMeMmmGoBVW3MernkWTi98XvfS8BM0o1rcIWaVrNTZszjhlncLspkrs8fI4dGGy/wbhCdJIHchJN1Ql9o4SvjTX2DAELjMIUcg4euHK/PEMUpSM9b3oZrB8gkozpNyohgHb9HNklrIGGeEDDbwVNHfplXTD/lr1luNtFY9d8a8Pxfzlc9fYiQjcDp+83eZMqvaneoLypJM+WETb86IA6WWN2Rj6fmq+JI0sIPOTTKUbgb/cMKQRY3FFeNRBXj/jcYvaMraRbgQvTrUHpZ/xR3rPdx3LJWLkzzVMx3wy2KoarEzQtBu+oElfcSqxal+hc/HiPhetAFqC2zKml92pyFS6WYS7xEemWZl4GA7eF+AZ6XCOqoUtSzAvlbpNI3GBYb2vMEnal2ltc6IQWncRlaPTjMEUgSkFVhZMqk14pHSyylv5/oLGi86ZrmyqYjl4Bo2M7W/yPaDfhBAR7ILdSp1J21z1qoY0IuVyM1I5OJDsyCQCq0VcNsaIjBL70K7oZ/eqWC3J45us2gxG6eEiohB6YZJGylwRaKCOidfwVcqUAe5sv2Oj/sR2LZNjC1dKCLU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(2616005)(8676002)(5660300002)(38100700002)(54906003)(110136005)(956004)(66476007)(66556008)(66946007)(4744005)(26005)(316002)(6486002)(31686004)(8936002)(16576012)(36756003)(4326008)(2906002)(478600001)(186003)(86362001)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aTdRcll3RnQvVlhlWElSakw0S2NidjhxQXF5dkVHd1J6bm1KVnM3N3kxclM1?=
 =?utf-8?B?YzZ6NzNOTHNjYlA0MEMzTy9FK1ZSc1hhZkdJcWw2SG5sdks1cGsrRnlGYjlw?=
 =?utf-8?B?YzB3UXRrRTBqS3ZiMDIzZ1BNclp3UWo0TG5MS1dIRDBVbE5uNkxUWFpQbVBj?=
 =?utf-8?B?UG1OVml1NExEQ1JYL0hMdHBjU1UzU3N5RWVQay9yZURjOG4wc09JVG5pdFNa?=
 =?utf-8?B?VXRCeGxwUnBHYUhaRWNMTEFEWTk3U2hrSzNBZzhEQStVVXVqTk9kaW9MSkJP?=
 =?utf-8?B?aXN3UENzSHc4ckYvdEREbWdQM1lWVEVBaXI1eEdzRTdvTEQ2SG5MVmVEN2tW?=
 =?utf-8?B?YXBOYTZhQ1Byc3ZzOUpRU1FQMkZjYkwxaGptdHlpRXFQNDZiMjNSZnFFUVUw?=
 =?utf-8?B?NElNbHVmeFd0djdRY0tUaTZNdHBqbUxYUzhDaFJ5bmh0M3BnbW9ERXdya2RW?=
 =?utf-8?B?Y3FGQXFYZ0toM2N5ZlRadTJ4YWhsbXJDaHhrTlB0L3QyQlV5R29Ec0Z0aHVP?=
 =?utf-8?B?TDZ0SitoeFdZQjhhU0pxZUNRUjhUbEp2WmpIYjJhbEo3MTFOMSt6Nm9IVGxI?=
 =?utf-8?B?K2xSQ1hGL2trakVaUGNzYVVPSXNibGo0MUNUMDA0MldOa0U2Nmp1VWptMUFa?=
 =?utf-8?B?cnFwZzVSblEya1VGS3pCd0FtNGRUWUhyMVkrSXdYeFNob2FoNHB5NUlobCtF?=
 =?utf-8?B?SW90SndlNW4vNEJDcXNiTTR2RTQwMnRxTmpKT2pmRE0yOHBvL3B3cHRyQWI4?=
 =?utf-8?B?dGZNVUN1VzN5YmkxSU5lY2xmSDI5SEt0NlVmbmwwUjl1TXlvUjdlSHdzM2pU?=
 =?utf-8?B?Z3k2bkxELzBmZDY0bEVrVVh3bE1taVBYTDZOcExNVUsvaDU0Ti9YV3RhWnJl?=
 =?utf-8?B?R1ZtbThnSDNDWno0MXVvejFRdm1xN0JtN0Z0VW4rT2MzSlR0RWFTKzFhanlU?=
 =?utf-8?B?c1ZUeFc4T0Z6bG84eDdkdDV6ZVgxbFpXK2M5cGtJaHpYREg5bVFiYTFPemZj?=
 =?utf-8?B?TWFucjNXOUJPTkcveERORS9qQitYMHZZak5GMmJSNVpkRnZsYXNDajhDbUJQ?=
 =?utf-8?B?RlNLZ2loczUvbVdERVU3aHM0SnpwR09NaFBkVFJDbXZmQTMyRjJXT09pRU5o?=
 =?utf-8?B?Sk9NMDlHMytXMDBBVDVKdjM4U0tUM00rc1JuYVd4Tk5rdTl2Sm93M29icjh0?=
 =?utf-8?B?MnJmV1ZvY0syLzBnZHdXcG5ZQlBDUFR3V014SUZZMHRialczZFFwVy9NMU9W?=
 =?utf-8?B?dzVPdVYzS0tnVWE4NUxPVjFVK3B2alpIWWk4eXhkZldHQXBFaU0rVnZOVVZW?=
 =?utf-8?B?VnhDWDJ5eUtCZEFyU3ZGTE8rWENkcHdJZ1VkZ0JZMzJ5QUNManpSVTNzSTMv?=
 =?utf-8?B?dW5vNE5EamNyY0RKVXhWeEdVNC9ITkhqOEp3Sk16MkxSVytXamlrWmVjZzM0?=
 =?utf-8?B?TWVlbWk2ZDBhVzQwVS92RW9wMGtjV2FxVnZrS1UwK0k3dzFiaFNZcGorZ0tN?=
 =?utf-8?B?TG5oRHFKSS9ycFBiV1J3eDJ6eG5OUDNlVlFGNnlPY1BwejlxallMUU5lM3kv?=
 =?utf-8?B?MTlINWFvOGtWbWxsR2p3dDBCWXZsN1p5dWRIeU5FT1EzVElyYjlBZlZ0dU5u?=
 =?utf-8?B?VURUczRiSkRlaGFwVGovc1A3QVhFR3F5aDZjY3FaTHBUNUI2ODA3c2xkSDU3?=
 =?utf-8?B?QnZLQXBkTHl4L1dDeUI1UlZwY0c5Um5TSWJPSFYrb01TSStXNFoyRVN5S2o3?=
 =?utf-8?Q?iqC3YEIiT/tUSyh4haYpCigcaQNQuviBwl0CqEP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ef7924f7-8733-4ea4-740e-08d971f81c50
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:07:49.1837
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8BKWMzefbSY0LRJXWx2Zs2vQu7Is952TijJ4DX9+OI2nh3NvAJsJ3ZUUGA7RR2sYEu0sj7615QNzEj5OnFIiUA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

It was introduced by 4035b43da6da ("xen-swiotlb: remove xen_set_nslabs")
and then not removed by 2d29960af0be ("swiotlb: dynamically allocate
io_tlb_default_mem").

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -152,8 +152,6 @@ static const char *xen_swiotlb_error(enu
 	return "";
 }
 
-#define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
-
 int xen_swiotlb_init(void)
 {
 	enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:10:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:10:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180959.327866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZwA-0001rq-TF; Tue, 07 Sep 2021 12:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180959.327866; Tue, 07 Sep 2021 12:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZwA-0001rj-PF; Tue, 07 Sep 2021 12:10:50 +0000
Received: by outflank-mailman (input) for mailman id 180959;
 Tue, 07 Sep 2021 12:10:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZw9-0001rd-1V
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:10:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8d632e29-a0bc-4e46-a45d-add1b6fb1d7f;
 Tue, 07 Sep 2021 12:10:48 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2051.outbound.protection.outlook.com [104.47.6.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-31Ji2ZHLOkaVTUI_6HUsoA-1; Tue, 07 Sep 2021 14:10:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6670.eurprd04.prod.outlook.com (2603:10a6:803:120::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 12:10:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:10:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0034.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:10:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d632e29-a0bc-4e46-a45d-add1b6fb1d7f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016647;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=buzMPiRLo4X6mtdDWgXjttrreVWYxdvGTL8smO8qNkM=;
	b=fBP9BQnQq/B2VvsrqCbDJBR75WwH+Yb7QuCUbkRCnCZUBHrxpEKlNUr+xVT3eHQZa5okiX
	tFWVza7/7ASznenRlGQyjOa8bOUbRopuqgISsibDWZw15hB7MbLJ5qvyLZ3hPO4cMW4Sx+
	G8p00F6wdTEgyckR0aIM+/boXZKrWyk=
X-MC-Unique: 31Ji2ZHLOkaVTUI_6HUsoA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lGFfI5pOHudYsCXPNFmYDflsm9wZBoUuRnhxbfea8ryF/O+5IwR8zKXWSRGFWOwt+jw6mGeb/3FK7u7SyNedIIqp0XeK83N/3JWWnnOKlb7tfu0GMQF92SL4KU8XXEj9clG0A5xyBj6PJR21apusrWdeeLfpjh6TO++TkAM1HCL44GXusdnqIXiBAifFj5JF/VmlI2z+qeNLphnP36jIakOheXpdL6PDyNrOxVLBWe1SJeVgYBoqIhPhvLvmyB7c2GCTqp+BG1GQtnVpOzeMUJPjOK7zpbDRlYH/Ny7y5ag1Qo5ER9yWPCs1YRlLR70FALd/WFdiUuXx6kVt81wpVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=buzMPiRLo4X6mtdDWgXjttrreVWYxdvGTL8smO8qNkM=;
 b=ij9RxK0H9a9ZGFFPKH1dMfbKGuWzcdP5uWCEzVXHlVphkaKNR9maympd/yaX1aG7nP7qMMIetnnELaisC1V7ge/Uy0oBkLk0RNhA9wZKxJJPTK3quY8unFsMBDgJWHWxw5rSZ/iUxFKuXiGoZmVLS94ZHfmdo8WaiEjy9dRghuyH6wWhkG2YHN/iSZu/eU2i0H8GN6a66vZbq9TX0H4Qo19eXQRjGcNaheckmnp0FNCZhDN0yz0zQeFkt8KzzAwnlDpBKdwJ6M1/G3wdKkFOiMgQniG3lEt1MCBNfMTW9/EUe5vqqTKC7IikwPCPmLge9rXzGYQS3aiVZznu3eIXKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 10/12] xen-pcifront: this module is PV-only
From: Jan Beulich <jbeulich@suse.com>
To: Bjorn Helgaas <bhelgaas@google.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>, Boris Ostrovsky
 <boris.ostrovsky@oracle.com>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <bbfb4191-9e34-53da-f179-4549b10dcfb3@suse.com>
Date: Tue, 7 Sep 2021 14:10:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0034.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1227db84-abb6-4da7-c80d-08d971f88451
X-MS-TrafficTypeDiagnostic: VE1PR04MB6670:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6670E4AB564B1F5BDD9E0C35B3D39@VE1PR04MB6670.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1775;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	m7EAqm6GcEk0ur7TGJ1KZyyN8K3PN89uiz3gqAxJ9J1nOzPj2hc8dOnXprwzfy3gXqZqq5vHI5U4ZazQ5KwdXarQm6nASmvbcN6iTnjqzSOSwJIG5Gjif7Fqh0MuFMptCyiJYlNVci9h7ONjQZUeUkV8MIp4hRs1qAmKdPeoiJEvU6WFVQrvNI/y4aceIRe66/51smGTNj5QgCyu5ddAqGI+s4ocP4aySnEveTPusykjEU4j1khnETIBOt+77m1yVKsh+3FZBIPeQ8asEJZzu4vvDg5ShEdEy4eeJt9pJZcDJkavQih/ADDcO53BX0qJsydcvg1fizVKHSHuI8te0roO22eH68tPlZ/aaGpKhnes9p0IV+7js0K8p+Zoi7rYcbtLbWCN9LB9Cygyg+aCf6rR5Ket8izPO5S6DqaR+XXwZ8iFCQY/qQ/ogoE6Gpk2KQeT4XBeBI296RPD4hD6Q2keaGKze4Hmv2r5DzGgWIr76ofsaRrT8hc1MXfVNgG/SfWpZB1p8cJICfXGbyBQJFRzoVwbTMchqs/reHC8xO4CoqTPL63+DdYNiuQeQGbDxN8xezIRvx8PCngE77q1Y9jneDON/sQb+PtU9cc8/s7e2fLFOtqFdgyL7j/fvmsOBzu+lT0gQeyIxBhQ0tRgzfp0vN8WyP79S/Js8TibzmlICrFcvzxPIrLjn9CbHD3eqG4FHBEY2FX802K/bShGnNJH/EOcTuPiZf5Xg2iOj7g=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(376002)(396003)(136003)(366004)(39860400002)(31686004)(8676002)(2616005)(316002)(5660300002)(4326008)(6916009)(8936002)(478600001)(31696002)(66556008)(66476007)(38100700002)(66946007)(54906003)(186003)(4744005)(86362001)(2906002)(6486002)(36756003)(26005)(16576012)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M2dEUFBtcjI2ek1vVS83VkNHWXVQL0ZwMFRoRW9rcHBISUMxMXE2MFlVWU1q?=
 =?utf-8?B?YWpLdjNUcU9YcDJBaVFjZzdjMXREWDdGaUJ1T1h4UUxFNktPM2JybmRsdHZr?=
 =?utf-8?B?Z3A0bVJjKzdDTUpFVllhYkpoWDJweTlHaEJidzRWdzhYeHU0YXQvY3UrNnUz?=
 =?utf-8?B?K3haRUd5SVZYZndwaDN3eGRJcmNLd1RuZVFsVGFOTTBiQm81ZmdzOUt6L3ZC?=
 =?utf-8?B?MGprWU5qeVZZTlBISDd4NnRHMVhFWFZFNzc1eEJWejJqdVA5bjFMZ1VWeHJo?=
 =?utf-8?B?Tlloa2RQTS9DbzJGMmJDRkE2dWZxdWZ4b3dhRURzYm5oNElOZjlWeTBlMG5Y?=
 =?utf-8?B?ajlMeHJTeTZ1VFU0ekxIZSs5WXFlUkd1RTVjU012LzBYdTZpU1c4UFVRNFRW?=
 =?utf-8?B?ZFBVRWM1TW0wcjZPMG9XQzA1cmNMY3ZwNG9URm5JYVNHbzVua1RXOVgwbWpV?=
 =?utf-8?B?RHZTOTUrWkNzWE9uMzFDQnJad2piVWZoKzRnOUN6dGExWEJxd3JjaWJIMXA1?=
 =?utf-8?B?c3ZnWHNQcU5TUndTdXovdEJTV1d6UEpjNGR2bGV2Q3FhSVhwampBdERTaHlX?=
 =?utf-8?B?NnVBUC81QTRWK2s3ZTJWc2NjSUJDa3Q5QlgyTGlYQWdGSzJTdkVqYXk2WmZS?=
 =?utf-8?B?cUZaOGg1TkZVL2lTMURJNmhpUEQ2OFVwYXZ5a0QyQSt4VU9zOWx6c0pqTEFM?=
 =?utf-8?B?MmhnQ1FVRHByRHd3N09XMzlOTEcxQUNPU3ZxMXVwdGZXcFhqNTByVGJyVnhX?=
 =?utf-8?B?NDdVRlo3ZUhnSXFJUkUzSHlvNzZ2eHlFREhTMk5IdFRFYVVDNjhTR0ViWC9Y?=
 =?utf-8?B?YVo2SWVpTEdHQW0relU5VzNYVGgrRW4ydHdDb3N3V1R5RjhndExHQkUzeDJj?=
 =?utf-8?B?dmZ3aHZ6RlU3VElkV3RJWmRVNHlzQXU2bURDK2h4ZWNDSnQ4aFU1VnArYXRI?=
 =?utf-8?B?aElHUUV4KzBiWTBkQ0tkTXNSa2xBelc2U0lxM1YzVTA4OTZ2cHlULzdkYVc0?=
 =?utf-8?B?RWhhTm5LdWV5QmxsSlJ2aUZ1b28rR2Rja3pqVjRtWWp1K3FsTkVmZk5sQ29r?=
 =?utf-8?B?d2F0dkhjYzdON2xHMFlaYXA1MlRTSUpkYUpuSjdPUVVZRzB0YUJUdkNnZzc2?=
 =?utf-8?B?R2FhSk1MTTE1eFgzT0dDUEZ1OU82NGkxQTBNSG1JeFY1WkE0OTFpZTdTdWJW?=
 =?utf-8?B?a2labmQ2ZEtmY2FDMzBoR0Ztb2JCSWhDSXlGQVhORHNCL1Q3ZEZHc1REVlRi?=
 =?utf-8?B?bFNOY1luS2FObzVWMVZua3F5d1lSQ20zQkFYdVRhUTUzQzRod3RibzdwQTZs?=
 =?utf-8?B?bmlYbWN6ZU03K1Vzd2tFMTlSRTR0ZmtoZXVVQkVuNFEvWHFhU292OURuaTl2?=
 =?utf-8?B?OVJXckZ2MkllcEN1aW5ycWdHaXBZRG1aRThaMk1KVldSQXU5ZkhHamJOY3hJ?=
 =?utf-8?B?QzZBVHNhb24rdlMxTG1tYkZvWWhYRHhjQkhmTE42RDJKdnlkQ05ORHFBQ0tE?=
 =?utf-8?B?MTREU0lHWndSK1FldFBKeFR5RlVFVkliYkxNUnhqK1JndDcrWVQrL2VqUWJD?=
 =?utf-8?B?ejBTRkdHV3N0OHk2K1A5RVpSRHZrTCs3VHdKWlo3Z0RiUGo1TTFFUEM1SE9X?=
 =?utf-8?B?ejFJYS9zelNhS1lFRWdUQnRmejlwdHoxaXVEWXIvdTRBN0xvRFpsSjhLRDVi?=
 =?utf-8?B?Ujl0R1B2QUdFMmxKTE83b1BoRmNxR0hib0g2dTVSRnlkUjhMZ3J2b09OSkli?=
 =?utf-8?Q?Kz5RxVbryVNtDaBh6Im1Twyr3j3aEdy9lofUnsF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1227db84-abb6-4da7-c80d-08d971f88451
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:10:43.6715
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: F5oVfv4KKz7Sfl7MsVyluQn1Kj7eJyGUHCjIUpIYyXZBezIo2cWXP/iZ3IgskLgXkzW8iTWz8Tcmd7aQhGvW/w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6670

It's module init function does a xen_pv_domain() check first thing.
Hence there's no point building it in non-PV configurations.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -110,7 +110,7 @@ config PCI_PF_STUB
 
 config XEN_PCIDEV_FRONTEND
 	tristate "Xen PCI Frontend"
-	depends on X86 && XEN
+	depends on XEN_PV
 	select PCI_XEN
 	select XEN_XENBUS_FRONTEND
 	default y



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:11:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:11:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180963.327877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZwg-0002fk-5O; Tue, 07 Sep 2021 12:11:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180963.327877; Tue, 07 Sep 2021 12:11:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZwg-0002fd-26; Tue, 07 Sep 2021 12:11:22 +0000
Received: by outflank-mailman (input) for mailman id 180963;
 Tue, 07 Sep 2021 12:11:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZwe-0002fQ-18
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:11:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 87c36d05-5fb5-4f85-871f-79f0948e0ee3;
 Tue, 07 Sep 2021 12:11:19 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2055.outbound.protection.outlook.com [104.47.6.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-aRu3o9_mMI6z_lWB0iMeiQ-2; Tue, 07 Sep 2021 14:11:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6670.eurprd04.prod.outlook.com (2603:10a6:803:120::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 12:11:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:11:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0047.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:11:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87c36d05-5fb5-4f85-871f-79f0948e0ee3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016678;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w/4sgXZWL5TyOKqIEB0uUbPnSshL/83p3qgV1ftBS1M=;
	b=G5E0+El9twSdJ4JPdIdcA6T7uJI4xcnrZK5TsXLmAxVPKfRIQkejBObU2Nsl/R9ywrZ4gc
	9YIgDm/IXnIrq48RFo79xmthC05uop5obhm8MOVFToq4iYByQ2b13VxTVeSuEWc5sd3N2q
	growQoEhAbP33xGtkECN4tB1ibE5spg=
X-MC-Unique: aRu3o9_mMI6z_lWB0iMeiQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=By2MW81/o1kLuXrOUuM8q6oL3Ze9TE5xrmCWM8vczKO7aSWDhmJFL4M24jRQdjpV91rymu43Qyb6txMna7daDikJY5AKBxfzMV1G7lfA3un0sjyNuzyJLkOteTjvD9p4my5SAmsBMNJy3RTDF8Vwnv1CCAKcTd7Nxk13p/8d7sxUP8fb781Nnsuk7u3tFjAjNDv81uFKDQLy9pU3rIRAV8RfGyYO+nyjWHI0GMsEpVbvNGER4PnG6gCuM2DshNeq2+MT+eofI1D3tkdImsxMsngg8eMGasNmRC0oIK2L2t3b5fqJcl+s9fFp1rmbzxhbexhOzcvwb2I3c1y+o0BISQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=w/4sgXZWL5TyOKqIEB0uUbPnSshL/83p3qgV1ftBS1M=;
 b=lHmvz+NiNzSTlSrLOMxEGmfUSa6GiUMaikp6c9ZxF5uOa+1a8WReH4Xgfh1Ce8K68q4U3nLG+dQf6E9yy7lcD3D4tUMeF9bX2GN5KzgIXNyhx8kdljvnADorvItx1U/+/WxiYMrGNxsMyBVnF0YoHmsxkSlh06YnC1+m9cOmaXCFLiujSwxs7uTnhpytwdpUIHGkI4OroIlZgCq22RNf1h1kfWrF8OZbRGGo4RoTniRC7nhSOjnJfhsFp8ErY6s7Bp7Yizb45IEpc6uKGXLZ0yQOv5/fFB4ZqbY1HI0ZWXcaWBy6Kt351euSCasqdQ+Xzsjpsa9INTXZsDkhAFGSUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 11/12] xen/pci-swiotlb: reduce visibility of symbols
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <75591cef-95e8-c694-fd11-283800499e4e@suse.com>
Date: Tue, 7 Sep 2021 14:11:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0047.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8b96239a-338c-4c83-a97d-08d971f897c1
X-MS-TrafficTypeDiagnostic: VE1PR04MB6670:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66703BB89CF5EC4D02E926BDB3D39@VE1PR04MB6670.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	c6jtTYYeSeyA8Xs2NV5x4j7yK4F1HAeNdMSka5OVZXiMXj22vesIRzMd4/kz7xRK7qeOeP5IwQFI6EqYyXPILPJwqNnN9lvRsUYihsVtrICJ3zbM1CULB4bbn+ej5gugNzviDieOLUG10NPU525CzdrHB9Xgrwbo12Joo3JrBLxqB8gHVpmbQK0vxxb6cPjGhZ0urjrYSbApUmM7k7nFp5AMrcvFjIb+hlkDfaC2QxwjWRY09Rxt9fd9knGkaE4B6llaNKYoZ36tIFTvt/VH1OjictbFgOMoRLvPL8ItQacZ4fXVBaHkjk6/da31yp8PrArZi29sw5NmtH1Qv3fXg/5Jic9ogCLi0XwMwuCJ4HI9eded5XtvAEoFu5KyaIrQu5qouKAm9LxV2mmQ+lGUSimCnJZEjMuFu2XLFDR2edH+WLeJOfCvoZV8EOHUEK03r9ediEQSxhQKyffIlJAOJIMuVjgoR/w353zN3BtMKfwLK9x+XNASDwCsWwGL3ddIS8rCDvx/NtW1WFM1aiEvxw2+krqkwgO7Pn1CSXPxf8Gdkye/8CrPALf4gobyMbOYx8PydoDPu9QlLS/IYFae+iTqocbHnl1/8jW17dFiv9ccv7V48YjrvuBjLuUMjjXnM2YxeYcRHnR8XiqzKTr1fPU3PIVfTww9K0AE9Mk/af9ur36LKadfKkqxmlzaAu8DbsT4Wcm3ahbZ5WLgLEUZbxADiaUvu44Tgqi8c7OiSsI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(376002)(396003)(136003)(366004)(39860400002)(31686004)(8676002)(2616005)(316002)(5660300002)(4326008)(8936002)(478600001)(31696002)(66556008)(66476007)(38100700002)(66946007)(54906003)(83380400001)(186003)(86362001)(2906002)(110136005)(6486002)(36756003)(26005)(16576012)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YXdSMHNXREh2TFAwUWhyUzc3ZTVJbHFnc0NiMGNwV0RoVFMvTlN5emZUL3k3?=
 =?utf-8?B?VXdNd3JOeXRXdnIvd3FDYkZqclYyWTZ4VmtaYlhueEZ2bHE2Y1Nicm8yWFBZ?=
 =?utf-8?B?czQwNjhOK2plMEpLdUwzSURjVmFZK0JKd1VFOTJPNVliWjlzM1NVUjNsdng3?=
 =?utf-8?B?RVhOc21MeityM0x5WHA5b3NlT1hpanYrblVRNjgvemplbThhWkdDcVVXbjNx?=
 =?utf-8?B?aU1TajBrTjdSWitNVTZhTERCS2tqMVo3ME9jd2ZvQWUzUlVQQS9hVlRvYVhi?=
 =?utf-8?B?ZkkyaFNxOGJQckEyZjlCREQrUm02WnFhQnRLN3prT01NNDlPbWhVK0Nha2M1?=
 =?utf-8?B?aVZudVZMOUQ1TE5PQ0t0bWgxZVpyMjFldmY0OGNEU24weU90K1BBT1V3Wlln?=
 =?utf-8?B?QVhrRGtMZElVcklDVnpIMWJFVmRMK3lxK0JUWDZ4UnBSSUVHR0N1c1UzM2Nq?=
 =?utf-8?B?alFIdjB2VDVOOFpvb0J1ZXdNWFJUTGp4L1ptTXpPU2lEdnEvQkNIbmcxa0I4?=
 =?utf-8?B?VVRTYmM2OVltemhkQ2RXWFZMemtLM1ZnWXh4bm1zWnYwYk1uYzFuSVhsbTJk?=
 =?utf-8?B?OW02NTVXWVpoeTV3YTVRTUlCeTVBZTQ1U3FKMUU0WSs2ZmU1RjR3ZXhPNVpW?=
 =?utf-8?B?bWVzYlhRVUhaZU1lVkR6V1EvZG1KUFptc2tJb1pSaGVtM1ZVZHFtV1RSYkhy?=
 =?utf-8?B?dmlja3gvZENKeXVLTktaMzNMVGI0OU5wbHVnMyt6eHhpMUJjbldxb2M2VUNT?=
 =?utf-8?B?NHJWejIwN0dRcFlKQ0lCRFFMeldpcGZMMEM1Q3BGbE13V3lBTVBEUCt6OHdO?=
 =?utf-8?B?a2hWdm54QTJmdVhZcm4zdURaRjdSSnFkQzFoWm5PeHR3WnFwMG9Ub0Rtd3Rn?=
 =?utf-8?B?Z1ROZmQ1MkZ5SnJORUNuR1Jsb3B1M0thU2tueWRFUWZZQWlQRWU1RlVDUS95?=
 =?utf-8?B?Q0VITDRDRllGaVllNnc2UXVySVFYNStuRFVYa3I0OWkvOW5XT25wb20yU09k?=
 =?utf-8?B?YTd5emxwUHEvSndiL2E2VC9RL3ppZk5iMlBHVnpPRDlPVmMvbnBsekxtbWZB?=
 =?utf-8?B?YU45QjF1em5mSGpzTWVPL0NZSk5uZ1Q1WG80UTVDdEJrOEpCU0huVUs4U2FQ?=
 =?utf-8?B?ZWdadHU5eU5NcitJdnpGQnNVU0NyeE9YWnpzc0o5NjZ0Z2syK1J3ck1Ualh1?=
 =?utf-8?B?ZUV3bXZRamp0RUMwbDRmRUYwSy9XeTFiOFJUSEpkVlBOcEhYZHVQOWc1emdD?=
 =?utf-8?B?Ly8wR1k0YjRnZ2dpbDFodDh0dWc4TFlxNHN3aGx2RVRHOU5VVmF2OEJVZWZp?=
 =?utf-8?B?aXhkYUN4S3dmVkY0c2sxMWpiUUw1b25mbGJRU1Qvd3VZM2FxS0JFblE4Tm5i?=
 =?utf-8?B?RWt1THp6b0Z0Rit3VHZDMHJXUnlVNzZ1NG4zU1ZRbU5GTkk2S0NiaUNVN0Zu?=
 =?utf-8?B?TXVrNWp4dHJ1d2RmSldVMC8zNkZydU1nbEJ3bEtFNmg1ZGJOb2ZLdTlERWs0?=
 =?utf-8?B?UWU4cU15VGIzbjh0ZTJLMi9laXA1Q0dkam5adTFDRlpDTWhpckdCUW5nVjZh?=
 =?utf-8?B?Z3BtUnY3THlDbWVhRzlGRFFpWHd2ZDhXeTNhcjEyYlVMN01MRWVERXIwbmpK?=
 =?utf-8?B?MS9EMHg5MDluUVJQWXBmeDNSVmR5SldLNW5vNTdkenpKWWlRQ05rNlFTTFdE?=
 =?utf-8?B?U3dOSkZMTlM3amxxWDBVL29mMlROYXRSa0NMWHd3WEprak4wL2JwZjZHTEpn?=
 =?utf-8?Q?jztJsKC5v6UJGl1hsCBzl5tLH5sz3/K5EI9w1ma?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b96239a-338c-4c83-a97d-08d971f897c1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:11:16.2704
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qEfTjpzl4JcI18fHOhFc1nlSIL4QhKp3jrj0zwtijI0hb4Whbz4NLgrnIg4zOSGvgSYcNJGY8iOXwgUlRxDadg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6670

xen_swiotlb and pci_xen_swiotlb_init() are only used within the file
defining them, so make them static and remove the stubs. Otoh
pci_xen_swiotlb_detect() has a use (as function pointer) from the main
pci-swiotlb.c file - convert its stub to a #define to NULL.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,14 +3,10 @@
 #define _ASM_X86_SWIOTLB_XEN_H
 
 #ifdef CONFIG_SWIOTLB_XEN
-extern int xen_swiotlb;
 extern int __init pci_xen_swiotlb_detect(void);
-extern void __init pci_xen_swiotlb_init(void);
 extern int pci_xen_swiotlb_init_late(void);
 #else
-#define xen_swiotlb (0)
-static inline int __init pci_xen_swiotlb_detect(void) { return 0; }
-static inline void __init pci_xen_swiotlb_init(void) { }
+#define pci_xen_swiotlb_detect NULL
 static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
 #endif
 
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -18,7 +18,7 @@
 #endif
 #include <linux/export.h>
 
-int xen_swiotlb __read_mostly;
+static int xen_swiotlb __read_mostly;
 
 /*
  * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
@@ -56,7 +56,7 @@ int __init pci_xen_swiotlb_detect(void)
 	return xen_swiotlb;
 }
 
-void __init pci_xen_swiotlb_init(void)
+static void __init pci_xen_swiotlb_init(void)
 {
 	if (xen_swiotlb) {
 		xen_swiotlb_init_early();



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:13:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:13:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180970.327888 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZyi-0003Tx-L1; Tue, 07 Sep 2021 12:13:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180970.327888; Tue, 07 Sep 2021 12:13:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNZyi-0003Tq-I4; Tue, 07 Sep 2021 12:13:28 +0000
Received: by outflank-mailman (input) for mailman id 180970;
 Tue, 07 Sep 2021 12:13:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNZyi-0003Tk-30
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:13:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 014c1066-0fd5-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:13:27 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-hmssEa00Mkes3R_235iJ3A-1; Tue, 07 Sep 2021 14:13:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4943.eurprd04.prod.outlook.com (2603:10a6:803:51::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 12:13:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:13:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0039.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:13:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 014c1066-0fd5-11ec-b0eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631016806;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Oa3ZLU03Swba0gYvsSEjRU1XJu+IDpj/E8M5qxMYmhM=;
	b=m/81AvXjwZBlj8QjxuUQXwqF4l1qlxXQC9MUfuQyClDi11QVIQC0sr6VEvsZECn5zSNien
	8CR2EnqavAKyN8HSWQ0SzWuMHUwLZEBOCoEda58aoQ92gToAB73+WeBfnRhXI89W7G3zQN
	dIXuV48KDNbZEEfGxtlEl6jSEg9B230=
X-MC-Unique: hmssEa00Mkes3R_235iJ3A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fNrCjBtaCc9xrRgcZFD6TBodIymGQhEGSbmgii+q0J6PhiCvr9GbbJuWLbKnB3UY3AiS6gQ5Am15OL6bQ72vSLi2Rl1pS8ygiZUy5fMJMJVBNo2vBeKgD6GNJIz3oOyzOaErWXlx6+qgAOHM/xHdcCyoRl6IYlrUuChkpjOj0n5Rj6KfDry0tDdAKzdxfTNdOb6dKDkyJxySSyrIBoIfwxRYutrJtc3Q/aK+iAzkkavz7qe9K0/tLz0FRWkwNU9FG9wpXeT5QPqJdyzA5QyvjxQjYwE6qAKcmugli7NzsvX4mPCPxSOQc5d4354ZWCK25rcy1PWGw0+gBnOwlHiiTw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Oa3ZLU03Swba0gYvsSEjRU1XJu+IDpj/E8M5qxMYmhM=;
 b=JqLWdpRvPxsIWlGroRzDRKNQZLUS8rpC0Vt0vtXK9VxNpou54LoVMXTNImYpMRQsMha8Rnu0VWQu/QXCRXHObhYJf/5cVSuax64pcn4CdXJvC8tutebA/iyv3dHmt53PcQCAaU9jiVmnmLcTIxGdVKS4MVkkHCNFq/3WKbQqNGwFI2BkvLEbHty+BpXyG+1nyMntSXhfxiMfXt0jNTqAGgeUkzxX0zb8BZMQswUdfkIzmdJEQCuNjT/yjE1W2+Ww9jhD/V42Gy+RZ73oPjffsxbLGy1UoXVyvOK2fmCXIhRkZ9csNBnXLy4ilnn1SM8GGzLCc83VzFDfMlGMHABSwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: alien8.de; dkim=none (message not signed)
 header.d=none;alien8.de; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 the arch/x86 maintainers <x86@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Message-ID: <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com>
Date: Tue, 7 Sep 2021 14:13:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0039.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b7774d84-fc14-4235-a792-08d971f8e396
X-MS-TrafficTypeDiagnostic: VI1PR04MB4943:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4943E21069F40049E0492C83B3D39@VI1PR04MB4943.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8vr/cvpjrHj9i7Ab2anwybJQtqfCUkLxrZeWoopggiJjivFTbp2KsRgQ7A2tgPQ4E3dOZgyqMef+rxP0r5Pb/k/MdvSyLCv19bNVwzU3Wav/jqlnxpDcGNZf3oWI1zJ3821MhF2+GsIRFmZ/FRgc5TPAN+qcGGolJ296uFSTODOP4iBS41fSKYHrj0vKL2X+GH7hfLfocq4UG1yP75NCkcuUnP143Qeew0LxdElKLrcDQxoS/xWKZIbYj2Y4C+/R0nnB+NinOvshRjrymlEpD+/SpI3HUe5CU41rLje5eigEnujEG2jzV4ZHmh52GxX1FvrPUjf+I6wc+Ku1mf8MfzLHuT0W5aTkGHbGouuR8v3wfT3wWlsGfvDUwv3lRZv637OQuIIjQUvQHcItHz4iz+Ku+OOZ3QW1NSSBWI2u9rgrpiIMziD6/CXdNzDSSLOHvQ8nS9zDmWKra/zxTOeFyjw/xbxYT+vppVOHSpgnlFuWkvMKlsMUU5tTJ/xAJhPGttvOw4enrLTTEkhBRXyowPNlNQA6FsMcs0Tw1wm9/3QNA50+/aY1IMKRzau7FjZnj9WACrpOg633GLU2QDw/Qm521l1aI5Tyy/Maf0tW5F7gp/P1lPDMv90nJd4Bv6cdIK7hNEBlgIJCEFTYlPX/bkJRjCPs8p3zDNxcMeQ46w65dAX/NcfTtC5Vs+2X4NB9t9aHqIb8A4os50DK35X5H9nqR2plZG9dVJ5LvUOkGkM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(396003)(136003)(366004)(376002)(39860400002)(2906002)(110136005)(16576012)(956004)(4326008)(316002)(66556008)(66476007)(66946007)(5660300002)(83380400001)(2616005)(54906003)(478600001)(6486002)(31686004)(86362001)(31696002)(186003)(8936002)(8676002)(26005)(36756003)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?V2FGYmxETys3V1dJemlPY2s4OW5BSEFxcFE4M1N2dzNsdENSRlRES0ZRbEEr?=
 =?utf-8?B?Qk5ITnlXQ1V5U3RGUFRwbmVFUDdCQ3VsQjRZQ1BEandnRnVGOU5reitQMC9V?=
 =?utf-8?B?NEltOS9QU1pVTzdqMzlETE5ldVB5Wkk2SXpEZEhveXZKZUdPMDhtcUV4SVln?=
 =?utf-8?B?QUpjNG12QnFxUlhIem5Qck5NRFoxaGkzd2JldU5FTTNWVWJWNkwxNlVkSGVt?=
 =?utf-8?B?alpOd2R0dS9WeE1maW84c3lTSHRaY1dFbzFLL2xXdDI0Ri9UY3hGTEhjdWFU?=
 =?utf-8?B?cGNtWWlCU3hLUnVjcFVvU1lXb3FTaUdiNzlpTnRtRDR3YklnemZUbEdFcGJP?=
 =?utf-8?B?T3RQb0N4TzArRW5mQis4YzhRWlhiUGJZek5FZHZTaDl0Zm9qdUM4amZWYXls?=
 =?utf-8?B?N0J6dUJTTmNtOHpjZkNneTcvR2dYMkRBM3ltVzEvQlJVb2lnQjF5cEpHVEx4?=
 =?utf-8?B?ak43Y0Y2ajl3RmRQZjU3eVdMUGZXVVIwblQ2WDI2eEdKVUp0aXpYVVkzNkZH?=
 =?utf-8?B?bnU5NUxwOWtWcHVPZ1A4Uy81UWkxOGFJdEhuTklwWXVkVk90UjJHcUJsUlNi?=
 =?utf-8?B?K3pCNjluWUhYYVNybjJ2c3BhWVczU1N4ZXplanROc1RzU1FuRmYrTWVCSUdy?=
 =?utf-8?B?bTZaREhydEFaT0VnYUNZb1pHQ1ErQXQvVHh2VUFLWGROZzVMb1RFbjRPUFh1?=
 =?utf-8?B?TzJIOTZkRkF3WE5mVDVsS1FuMFZwSjJ6cjV1SmZhNjQ2R3Jsc1NpSms0NEUv?=
 =?utf-8?B?SEJTQ2hyNGhuSnJsQ1FmcXpJUnlZWWpoNThTOGNmWkZDSC8vN1pLUFlVZGlk?=
 =?utf-8?B?RkNqM0p4UTlscWROUmVyWkQ3Qy9LaUZ0U1huWlhjUS90RmNtOXV1c3M3NWI5?=
 =?utf-8?B?VWVuRndhbzdNaEYxM212MlV1MXRkQ3dIaTZqQmFEWUxtZnZVVGxydmJrNFIv?=
 =?utf-8?B?eU8xZEdGSWFyUDlPeEg2MmlGRFVDZk1MZzZBakRrNnN6ZFg5QWVvY3lDWFRq?=
 =?utf-8?B?VEtrVWIrZGVBU3FzaDV0WE1Qby9RVzdxY0c3SWptbVMvY3A5aG1VNHpxak9p?=
 =?utf-8?B?VFM5QWwzdVZ6KzdkWUp3SSttdHcwcEZxZVFlYzNnK0RacHRtOGZaTE5OYWNM?=
 =?utf-8?B?Y05GSFdhbFpWN2kwdGs0aDlCbW41dTNrL2FsSEIvSW0wWmcyekxZRTVrY3NI?=
 =?utf-8?B?Y01maHJLb3ZJMCtjRmdYbUtzNDdPdGlqc0ZoMjNmZDFuWmtwWVdCUjR6NWNL?=
 =?utf-8?B?ZjF5VjBaYWladFhtN2J0S3hGWHNqUUVOTHZsRGxKc2ZUUWlnNVVHVWcxVHFs?=
 =?utf-8?B?RG5JZ2lhK2J6bzhYSkRicm42cGdwQS9jRkJ2UDcvRXVHZ0VqVnlhc1lUdU1a?=
 =?utf-8?B?dlRtMVZtTUUyRTVFTEE0S1EzMmd0QWorSHE3NWRYVEJxdUh3VWV3alNCQ3Vv?=
 =?utf-8?B?S0xSN1VueXN4d1l1VDBYWFZJaWJIWjJTVnRRekVVYm83Q2tpbHFRNVhLR1Fa?=
 =?utf-8?B?Q2hPSWNMU2tLZXhZbnpqbmd0ejVBK2RrZXNtc2tLT1dEb3NaMHVZUU5Ga1ZC?=
 =?utf-8?B?SVFGZ0tuczh1MXk4T3QybkN1UnQweEtLU3JvVFNicTNJZTI4ekFjOWNrUE5D?=
 =?utf-8?B?amNwQmJpcnpJZDRvZHBRWnhIYW51eDZKemdJNitZMDdKQVVxL1dUMElsQmpx?=
 =?utf-8?B?dnZXdnlYaWtUNitkcEp0VmZDZTExKytYVVhJSE9mVUg2OXA2eUp6Slo5Y1pz?=
 =?utf-8?Q?DkglmFvVEmdbK0RxdTVeJSQptHiGlsOMnlgHSNX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b7774d84-fc14-4235-a792-08d971f8e396
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:13:23.5014
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XXdSfbCYvgXozurmPaARtxlnRGuVKssWDwgdeKOzKQHuirdoSSd5NUtLJsjbmk5WNeJhUNtDrNo0pK29j9rxJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4943

The code is unreachable for HVM or PVH, and it also makes little sense
in auto-translated environments. On Arm, with
xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
time seeing what good the Xen specific variant does - the generic one
ought to be fine for all purposes there. Still Arm code explicitly
references symbols here, so the code will continue to be included there.

Instead of making PCI_XEN's "select" conditional, simply drop it -
SWIOTLB_XEN will be available unconditionally in the PV case anyway, and
is - as explained above - dead code in non-PV environments.

This in turn allows dropping the stubs for
xen_{create,destroy}_contiguous_region(), the former of which was broken
anyway - it failed to set the DMA handle output.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2605,7 +2605,6 @@ config PCI_OLPC
 config PCI_XEN
 	def_bool y
 	depends on PCI && XEN
-	select SWIOTLB_XEN
 
 config MMCONF_FAM10H
 	def_bool y
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -177,6 +177,7 @@ config XEN_GRANT_DMA_ALLOC
 
 config SWIOTLB_XEN
 	def_bool y
+	depends on XEN_PV || ARM || ARM64
 	select DMA_OPS
 	select SWIOTLB
 
--- a/include/xen/xen-ops.h
+++ b/include/xen/xen-ops.h
@@ -46,19 +46,7 @@ extern unsigned long *xen_contiguous_bit
 int xen_create_contiguous_region(phys_addr_t pstart, unsigned int order,
 				unsigned int address_bits,
 				dma_addr_t *dma_handle);
-
 void xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order);
-#else
-static inline int xen_create_contiguous_region(phys_addr_t pstart,
-					       unsigned int order,
-					       unsigned int address_bits,
-					       dma_addr_t *dma_handle)
-{
-	return 0;
-}
-
-static inline void xen_destroy_contiguous_region(phys_addr_t pstart,
-						 unsigned int order) { }
 #endif
 
 #if defined(CONFIG_XEN_PV)



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:17:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:17:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180978.327898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa24-00047Q-56; Tue, 07 Sep 2021 12:16:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180978.327898; Tue, 07 Sep 2021 12:16:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa24-00047J-24; Tue, 07 Sep 2021 12:16:56 +0000
Received: by outflank-mailman (input) for mailman id 180978;
 Tue, 07 Sep 2021 12:16:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNa22-00047C-7A
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:16:54 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7c5f5902-0fd5-11ec-b0eb-12813bfff9fa;
 Tue, 07 Sep 2021 12:16:53 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 187BgonC004961; 
 Tue, 7 Sep 2021 12:16:50 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ax2jnh7sr-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 12:16:50 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM7PR03MB6450.eurprd03.prod.outlook.com (2603:10a6:20b:1b2::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 12:16:46 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 12:16:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c5f5902-0fd5-11ec-b0eb-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HaP2QAgMCTDubOczWTEKnaW5lqTzjcYi8TwhpbfZshnhrNitlrieg4079WGTO27jI5vfrZBLfk709ahprnQ4DbfpczejMVD85JnWOYXWEaYCSIYZMVLivaZFVTCQbi7ED4t8/NKf/Yec6NbNu62ozbw8SmB0emavGMcwmeN5MbC3+EbGolxvnMAa01s/B+GMwvji7WqPFJm7ZXoEiLwtIj52ImqsBIJkFCoVlXQ/d2f2phwP1PGmNsLNmCsbDC9B4IbwwjmcmjrmdrFE01gpEBL9g/+/T3DquUTZIpwrGUPiZbapJAPJBAPCvYYH5TqGzNcbMi9WWovKk7XrO7dHYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ommb9VYJ3nFEfnB9d/KRAz9te3a2pOZjcQ4N2qrkwOA=;
 b=isoC+O4Br4MGmrAB1wJbozl01Ni9zZ8m7SdnqSsbnS4LdnkFwjSKImIQrEbGQk7rGx+EJz6BWwPGNGhVx5WypTnGgJzFamZyvUordEF7UK7C/ou12hIILfDO0zn1Xhi3Rts4VpFaywZ08fliJxHdUi3428JJ1sSboyNHKxNJe3SWZTSZUrBPAFNZuDu01XjE6a6MX82A/GDx5L8IuNH1+/v1TzmZe7IB64SECadEGkyzgy65qd+7bWEP2vcKZTZqGfiMGzp4lbBObJDIJoEIe+xdGEdrmKAf52zM7AgpWnlRnx0k9jGhcSPdgzN1r+AGMoip21JkbXYqmLrGkPNqEQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ommb9VYJ3nFEfnB9d/KRAz9te3a2pOZjcQ4N2qrkwOA=;
 b=CBgam0yohhTe01AD5kNOU0JSUKfO8xLbxfX+tlR2C6jK1a086g8GIOLbP4x10tSVUQWMVpDjX237m8UZVKnQ7hq8SSeML5VPmBmRR+fYXLALZRPr3MhQ7VX5fY4JEGUD86qxe0uE9bH7WPF5g9n2n54rjcIoGuy9bE91w495FelY67I+TBbD+INMh/X7W0MMztLYoXmBv4KnVpQOvIDPAJJP1IfO446bO89x0pBxmvoBejxhnvkOn/1MLVT0pVQkbzmEVemragwohKKSHupNO3ONZ90QHI1ZBntHUu3Lz6bc3kbqL2MEkbIbWZy3wTowTLIVIFsogNQrRNOYsGS/aQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Topic: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Index: 
 AQHXoKusySJnLVveGkulaWKrBZdXpKuXEOQAgAFPE4CAAAkRAIAAB3eAgAALDQCAAAeQAA==
Date: Tue, 7 Sep 2021 12:16:46 +0000
Message-ID: <b4c94187-db81-d52f-0425-959fe1e9e83a@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
 <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
 <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
 <938156e7-cbd0-a19e-f09f-a41dce051367@suse.com>
In-Reply-To: <938156e7-cbd0-a19e-f09f-a41dce051367@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e21fe2b4-c39e-42c6-c929-08d971f95cd4
x-ms-traffictypediagnostic: AM7PR03MB6450:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM7PR03MB6450220C91BA3331A131E7CEE7D39@AM7PR03MB6450.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 tx42NoX7I0FjIqitNk9CCQyO3eUWeaSvuF+pkG6la+FGTviuA3hRvR0lo1TJV+Vnak3jg2waJyf6txgMNAsWK3O5KxcLqQhv+uMH28ihy01Rj3CuNs0FLZ8XXIB57+HvPUE7lbRJ2nsslggYdVbinhEkYUB63PbB/35GPU6CHMNtOdH1oZLxx+kQagdtaweanaQSPTgN6MxK+MbUFy/SrjDDlZJG7H0DtPNUaXntncxg4hIAxRnbX1d+lQ02spoj5IU6sAPty5ZMeBtJiqDaIMMbUUyTLvFF8lBibDfsO1DLDgGScM9vFwEYlQ3omfkrYx6r9OElVdVJ/LWsU44kqE018mGEAp40Vt/haYQjr6cqDTzdYEPLYxnVV1XX27UK7AqMJh8+yK8bdMlj2IMmuY1H4rfSVFUxdveHAXleL+9o6wreJ/ffxSMjz5I41+Ikj1t0v3ONSGaBbVqy8PEx8te2GG+is4k/0NMyajM/bdYrLDHpIidBDB2Wa7GCKfdMEKg3iRL5TBbNoramyF8etsE/GmKNgEHraRLQhnlolpyoNEVW4rpRO22hfAGFQO7o8wbpVc6T1lotLsWgHkjFOypF3fM3YsvAJaxwQ+Wb+sp23lRjTcV8ty+TeCXpeVH29RCqXd5G2iqibBezeGprMrkfK0I6BC5dA2v8Gf6Msjqic2FeLxrSaR58rekzHDW52hOX5TF1SS09VHK0+zbW7EGdM9Cq0tsuixxksRFGz79ceLK+4tMZoDwy0bcm9a1ZPn9buw/WursHEgYcBAZXJg==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(376002)(39860400002)(396003)(136003)(91956017)(316002)(122000001)(76116006)(4326008)(83380400001)(36756003)(5660300002)(38070700005)(54906003)(186003)(110136005)(66446008)(64756008)(66946007)(66556008)(38100700002)(6486002)(6506007)(2906002)(66476007)(31696002)(8676002)(6512007)(31686004)(8936002)(71200400001)(86362001)(2616005)(478600001)(26005)(53546011)(21314003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?K0VLLzJpd1h6bTVPaDRaQTNta1NUajhrTmcvUUFGZ2tLc1g4QWFZUlY5Qm9C?=
 =?utf-8?B?eklHT3RZVTRRQ2hTY1hzb3NBTHpUeGVTWkIyMkFOdTZ6MEl4c3QxcGhqWWNR?=
 =?utf-8?B?SHMwcE13RFh3MGw1MG9nVTIxNENjY1c0bEhGUHRoeERiM1grVzJLMUdPUHdt?=
 =?utf-8?B?dDhlN1ZvWnFGcTZDSFhOSk5vS01TZkVzSVlkTHQ4c3k3TTNheXRsTjREWERk?=
 =?utf-8?B?NTRUM0Z0aWlWSzBFc2doTDcwb2E2cmtuVmQzV1NQOGhZWW00Y1diU1Vid0x3?=
 =?utf-8?B?SURlamN6a0wyQlBqWTMvRndPSmVhellRemQyemZhcVJYYTNRcTNXeHlxN3RP?=
 =?utf-8?B?NVNjbUtYZVBQY1FMUDlzbGh3QnYwNXZ4ZjBGZzZiK0dRZjRtVGZrNWtQMEFy?=
 =?utf-8?B?WTRkTEtFbDhjb2x2ZGw5MUM4Sm11SXhKRzQvZlZFL3JXa3lPV242T2J2R3o2?=
 =?utf-8?B?Wjl4MDF3Sml6U3lUbzBFZEJKd2VwVjNYVjFSMHQ5eS9jTTFKRkY2eWEvd2x6?=
 =?utf-8?B?SmZxMVdqM1NnT0ZGbDRuMm5BWjgramFXZ1FheEx0dU5TMUlDQmhwemwvOUIr?=
 =?utf-8?B?UHpQVTBKdXg4UW9mU3lhUWMzMXpVaFd0QmVzMCtaZGErQXlFbEV1Z0Q3VzFq?=
 =?utf-8?B?S0lpQ2NsVTc0Y1c0ajEvbkpFK1J0NnorMStuK3VhNnBRZHFZNWxVQklHNFJa?=
 =?utf-8?B?TW9TRnk3YkRjYklYWFAvTmJ6ckt1enpKbmw2NXgyS3JUSEZrUzlyb1ArZFRy?=
 =?utf-8?B?anNZUWRPay83UjlWRGZsZVJ4VFFETDBnKzdxL0EzQXNMbDUvbDZzakllTHps?=
 =?utf-8?B?V0tpN3p5R2pBQlRROU1EUDNUSEpweVZRMmY2U2NzanVVdzRFYndpVjdYVnlK?=
 =?utf-8?B?L21BaHVIZGZIZUZxaXp2WjVNQUJra1psMVhwWC9QaDZPODJHWkFTcDlJbGFI?=
 =?utf-8?B?bjZja2dreGhodENWWUszOE5PdEpwRkFXcjJRdFlMYTc4Qml0c2U1elFWOTJz?=
 =?utf-8?B?dU9ZMU1qbjJsbkZqbHl6bTl1eWZDemVyYkJJa09UU1NBRHRSS2w4dUFRTXNL?=
 =?utf-8?B?WWtyakxGZlpQZTNpZzF6WmRVVkVZeDJHOGtqTVNvRmlzbnBFMk1mS2FFUlNR?=
 =?utf-8?B?emZiS0EzWXRXWDZ6NlhqczU4OWxFU2RCU2dIVlF0TjB1cEw2L0RsNDN1eTgv?=
 =?utf-8?B?bnV0SU9tRnJocEdvNmtjNFpRQ213NkFtUlZLM1BVdEcvTFhDN2I3ZlJhRys1?=
 =?utf-8?B?eXBHR2FRMko2TXhESVVHa2d1RCs2YXBVSXJWSnJEWG4weUFJbXdZcjB3TDJo?=
 =?utf-8?B?S1pGNGpaVjFFNmVOSVNnR1o0MCtEZDFQSnZJdmczVTF4dndEWmZKaTdxd2J5?=
 =?utf-8?B?VTQ5SjhQcnpCR1F5YWs1b0ZvQ0oxRFEzVEtLMkMySWYxSTZML3JWSkNPaUFz?=
 =?utf-8?B?TlNCVTBqS2ZNaW1Bbm9kMHd5M0pkTFZGbjIzMDlsUmpNaUV6cW1OQ2IvSWNI?=
 =?utf-8?B?TWxvR2Vxbk5BaDE5VUZSQUhLYlVFTEZDcDRUeW50dEJwRU5XOGFQQXFocGpV?=
 =?utf-8?B?WmptMGxuVzF3b3dqRkowWlNvVi9wVStrOFhKcmdQVWFkaEN4TlkrbEV1MVhK?=
 =?utf-8?B?NXBKNU5NUVpOWkZ0YjFBN0drbjJRZ1NYUTZibUJnWjNxSTlDZ3Z2aUJlc2do?=
 =?utf-8?B?cWJxaHQzUFByVzN2S2VPVGprK3hPSDVQVmdOU21na1ROWFBqdWtRaWZUV2VX?=
 =?utf-8?Q?/ANIPw8gohoGNQTNGcTeV92/oCJfZyOlVnbTY5o?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <11F4B582EC09F64BAB396447186E638B@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e21fe2b4-c39e-42c6-c929-08d971f95cd4
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 12:16:46.5998
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: EWcZsNY6nGUPyRO1Mr1KmrMl7MH49sFpNwOiRqHqpqp+Yy0zhlmRrGHgxtO7FizL59j6pvPTv5DwERvrpi5MzZ5INMrvV91z2U1KOPnGcZ+nwpSn3N221JHEeTZ/fFFN
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR03MB6450
X-Proofpoint-ORIG-GUID: uHPeS_bhTxsLI5KWF_woXqr_7g0YftxV
X-Proofpoint-GUID: uHPeS_bhTxsLI5KWF_woXqr_7g0YftxV
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_04,2021-09-07_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0
 clxscore=1015 priorityscore=1501 impostorscore=0 phishscore=0 spamscore=0
 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070081

DQpPbiAwNy4wOS4yMSAxNDo0OSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTM6MTAsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTM6
NDMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTI6MTEsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNi4wOS4yMSAxNzoxMSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBAQCAtNTkzLDYgKzYyNSwzNiBAQCBzdGF0aWMgaW50IGluaXRf
YmFycyhzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4+PiAgICAgfQ0KPj4+Pj4+ICAgICBSRUdJ
U1RFUl9WUENJX0lOSVQoaW5pdF9iYXJzLCBWUENJX1BSSU9SSVRZX01JRERMRSk7DQo+Pj4+Pj4g
ICAgIA0KPj4+Pj4+ICtpbnQgdnBjaV9iYXJfYWRkX2hhbmRsZXJzKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQsIHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4+Pj4+ICt7DQo+Pj4+Pj4gKyAgICBpbnQg
cmM7DQo+Pj4+Pj4gKw0KPj4+Pj4+ICsgICAgLyogUmVtb3ZlIHByZXZpb3VzbHkgYWRkZWQgcmVn
aXN0ZXJzLiAqLw0KPj4+Pj4+ICsgICAgdnBjaV9yZW1vdmVfZGV2aWNlX3JlZ2lzdGVycyhwZGV2
KTsNCj4+Pj4+PiArDQo+Pj4+Pj4gKyAgICAvKiBJdCBvbmx5IG1ha2VzIHNlbnNlIHRvIGFkZCBy
ZWdpc3RlcnMgZm9yIGh3ZG9tIG9yIGd1ZXN0IGRvbWFpbi4gKi8NCj4+Pj4+PiArICAgIGlmICgg
ZC0+ZG9tYWluX2lkID49IERPTUlEX0ZJUlNUX1JFU0VSVkVEICkNCj4+Pj4+PiArICAgICAgICBy
ZXR1cm4gMDsNCj4+Pj4+PiArDQo+Pj4+Pj4gKyAgICBpZiAoIGlzX2hhcmR3YXJlX2RvbWFpbihk
KSApDQo+Pj4+Pj4gKyAgICAgICAgcmMgPSBhZGRfYmFyX2hhbmRsZXJzKHBkZXYsIHRydWUpOw0K
Pj4+Pj4+ICsgICAgZWxzZQ0KPj4+Pj4+ICsgICAgICAgIHJjID0gYWRkX2Jhcl9oYW5kbGVycyhw
ZGV2LCBmYWxzZSk7DQo+Pj4+PiAgICAgICAgcmMgPSBhZGRfYmFyX2hhbmRsZXJzKHBkZXYsIGlz
X2hhcmR3YXJlX2RvbWFpbihkKSk7DQo+Pj4+IEluZGVlZCwgdGhhbmsgeW91IDspDQo+Pj4+Pj4g
KyAgICBpZiAoIHJjICkNCj4+Pj4+PiArICAgICAgICBncHJpbnRrKFhFTkxPR19FUlIsDQo+Pj4+
Pj4gKyAgICAgICAgICAgICIlcHA6IGZhaWxlZCB0byBhZGQgQkFSIGhhbmRsZXJzIGZvciBkb20l
ZFxuIiwgJnBkZXYtPnNiZGYsDQo+Pj4+Pj4gKyAgICAgICAgICAgIGQtPmRvbWFpbl9pZCk7DQo+
Pj4+PiBQbGVhc2UgdXNlICVwZCBhbmQgY29ycmVjdCBpbmRlbnRhdGlvbi4gTG9nZ2luZyB0aGUg
ZXJyb3IgY29kZSBtaWdodA0KPj4+Pj4gYWxzbyBoZWxwIHNvbWUgaW4gZGlhZ25vc2luZyBpc3N1
ZXMuDQo+Pj4+IFN1cmUsIEknbGwgY2hhbmdlIGl0IHRvICVwZA0KPj4+Pj4gICAgIEZ1cnRoZXIg
SSdtIG5vdCBzdXJlIHRoaXMgaXMgYQ0KPj4+Pj4gbWVzc2FnZSB3ZSB3YW50IGluIHJlbGVhc2Ug
YnVpbGRzDQo+Pj4+IFdoeSBub3Q/DQo+Pj4gRXhjZXNzIHZlcmJvc2l0eTogSWYgd2UgaGF2ZSBz
dWNoIGhlcmUsIHdoeSBub3QgZWxzZXdoZXJlIG9uIGVycm9yIHBhdGhzPw0KPj4+IEFuZCBJIGhv
cGUgeW91IGFncmVlIHRoaW5ncyB3aWxsIGdldCB0b28gdmVyYm9zZSBpZiB3ZSBoYWQgc3VjaCAo
YWJvdXQpDQo+Pj4gZXZlcnl3aGVyZT8NCj4+IEFncmVlLCB3aWxsIGNoYW5nZSBpdCB0byBnZHBy
aW50aw0KPj4+Pj4gICAgIC0gcGVyaGFwcyBnZHByaW50aygpPw0KPj4+PiBJJ2xsIGNoYW5nZSBp
ZiB3ZSBkZWNpZGUgc28NCj4+Pj4+PiArICAgIHJldHVybiByYzsNCj4+Pj4+PiArfQ0KPj4+Pj4+
ICsNCj4+Pj4+PiAraW50IHZwY2lfYmFyX3JlbW92ZV9oYW5kbGVycyhjb25zdCBzdHJ1Y3QgZG9t
YWluICpkLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4+PiArew0KPj4+Pj4+ICsgICAgLyog
UmVtb3ZlIHByZXZpb3VzbHkgYWRkZWQgcmVnaXN0ZXJzLiAqLw0KPj4+Pj4+ICsgICAgdnBjaV9y
ZW1vdmVfZGV2aWNlX3JlZ2lzdGVycyhwZGV2KTsNCj4+Pj4+PiArICAgIHJldHVybiAwOw0KPj4+
Pj4+ICt9DQo+Pj4+PiBBbHNvIC0gaW4gaG93IGZhciBpcyB0aGUgZ29hbCBvZiB5b3VyIHdvcmsg
dG8gYWxzbyBtYWtlIHZQQ0kgd29yayBmb3INCj4+Pj4+IHg4NiBEb21VLXM/IElmIHRoYXQncyBu
b3QgYSBnb2FsDQo+Pj4+IEl0IGlzIG5vdCwgdW5mb3J0dW5hdGVseS4gVGhlIGdvYWwgaXMgbm90
IHRvIGJyZWFrIHg4NiBhbmQgdG8gZW5hYmxlIEFybQ0KPj4+Pj4gLCBJJ2QgbGlrZSB0byBhc2sg
dGhhdCB5b3UgbGltaXQgdGhlDQo+Pj4+PiBpbnRyb2R1Y3Rpb24gb2YgY29kZSB0aGF0IGVuZHMg
dXAgZGVhZCB0aGVyZS4NCj4+Pj4gV2hhdCdzIHdyb25nIHdpdGggdGhpcyBmdW5jdGlvbiBldmVu
IGlmIGl0IGlzIGEgb25lLWxpbmVyPw0KPj4+IFRoZSBjb21tZW50IGlzIHByaW1hcmlseSBvbiB0
aGUgZWFybGllciBmdW5jdGlvbiwgYW5kIHRoZW4gZXh0ZW5kcyB0bw0KPj4+IHRoaXMgb25lLg0K
Pj4+DQo+Pj4+IFRoaXMgd2F5IHdlIGhhdmUgYSBwYWlyIHZwY2lfYmFyX2FkZF9oYW5kbGVycy92
cGNpX2Jhcl9yZW1vdmVfaGFuZGxlcnMNCj4+Pj4gYW5kIGlmIEkgdW5kZXJzdG9vZCBjb3JyZWN0
bHkgeW91IHN1Z2dlc3QgdnBjaV9iYXJfYWRkX2hhbmRsZXJzL3ZwY2lfcmVtb3ZlX2RldmljZV9y
ZWdpc3RlcnM/DQo+Pj4+IFdoYXQgd291bGQgd2UgZ2FpbiBmcm9tIHRoYXQsIGJ1dCB5ZXQgYW5v
dGhlciBzZWNyZXQga25vd2xlZGdlIHRoYXQgaW4gb3JkZXINCj4+Pj4gdG8gcmVtb3ZlIEJBUiBo
YW5kbGVycyBvbmUgbmVlZHMgdG8gY2FsbCB2cGNpX3JlbW92ZV9kZXZpY2VfcmVnaXN0ZXJzDQo+
Pj4+IHdoaWxlIEkgd291bGQgcGVyc29uYWxseSBleHBlY3QgdG8gY2FsbCB2cGNpX2Jhcl9hZGRf
aGFuZGxlcnMnIGNvdW50ZXJwYXJ0LA0KPj4+PiB2cGNpX3JlbW92ZV9kZXZpY2VfcmVnaXN0ZXJz
IG5hbWVseS4NCj4+PiBUaGlzIGlzIGFsbCBmaW5lLiBZZXQgdnBjaV9iYXJfe2FkZCxyZW1vdmV9
X2hhbmRsZXJzKCkgd2lsbCwgYWl1aSwgYmUNCj4+PiBkZWFkIGNvZGUgb24geDg2Lg0KPj4gdnBj
aV9iYXJfYWRkX2hhbmRsZXJzIHdpbGwgYmUgdXNlZCBieSB4ODYgUFZIIERvbTANCj4gV2hlcmUg
LyB3aGVuPyBZb3UgYWRkIGEgY2FsbCBmcm9tIHZwY2lfYXNzaWduX2RldmljZSgpLCBidXQgYmVz
aWRlcyB0aGF0DQo+IGFsc28gYmVpbmcgZGVhZCBjb2RlIG9uIHg4NiAoZm9yIG5vdyksIHlvdSBj
YW4ndCBtZWFuIHRoYXQgYmVjYXVzZQ0KPiB2cGNpX2RlYXNzaWduX2RldmljZSgpIGFsc28gY2Fs
bHMgdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzKCkuDQoNCllvdSBhcmUgcmlnaHQgaGVyZSBhbmQg
Ym90aCBhZGQvcmVtb3ZlIGFyZSBub3QgdXNlZCBvbiB4ODYgUFZIIERvbTAuDQoNCkkgYW0gc29y
cnkgZm9yIHdhc3RpbmcgeW91ciB0aW1lDQoNCj4NCj4+PiAgICBIZW5jZSB0aGVyZSBzaG91bGQg
YmUgYW4gYXJyYW5nZW1lbnQgYWxsb3dpbmcgdGhlDQo+Pj4gY29tcGlsZXIgdG8gZWxpbWluYXRl
IHRoaXMgZGVhZCBjb2RlLg0KPj4gU28sIHRoZSBvbmx5IGRlYWQgY29kZSBmb3IgeDg2IGhlcmUg
d2lsbCBiZSB2cGNpX2Jhcl9yZW1vdmVfaGFuZGxlcnMuIFlldC4NCj4+IEJlY2F1c2UgSSBob3Bl
IHg4NiB0byBnYWluIGd1ZXN0IHN1cHBvcnQgZm9yIFBWSCBEb20wIHNvb25lciBvciBsYXRlci4N
Cj4+DQo+Pj4gICAgV2hldGhlciB0aGF0J3MgZW5jbG9zaW5nIHRoZXNlDQo+Pj4gYnkgIiNpZmRl
ZiIgb3IgYWRkaW5nIGVhcmx5ICJpZighSVNfRU5BQkxFRChDT05GSUdfKikpIiBpcyBzZWNvbmRh
cnkuDQo+Pj4gVGhpcyBoYXMgYSBrbm9jay1vbiBlZmZlY3Qgb24gb3RoZXIgZnVuY3Rpb25zIGFz
IHlvdSBjZXJ0YWlubHkgcmVhbGl6ZToNCj4+PiBUaGUgY29tcGlsZXIgc2VlaW5nIGUuZy4gdGhl
IDJuZCBhcmd1bWVudCB0byB0aGUgYWRkLUJBUnMgZnVuY3Rpb24NCj4+PiBhbHdheXMgYmVpbmcg
dHJ1ZSBhbGxvd3MgaXQgdG8gaW5zdGFudGlhdGUganVzdCBhIGNsb25lIG9mIHRoZSBvcmlnaW5h
bA0KPj4+IGZ1bmN0aW9uIHdpdGggdGhlIHJlc3BlY3RpdmUgY29uZGl0aW9uYWxzIHJlbW92ZWQu
DQo+PiBXaXRoIHRoZSBhYm92ZSAoZS5nLiBhZGQgaXMgZ29pbmcgdG8gYmUgdXNlZCwgYnV0IG5v
dCByZW1vdmUpIGRvIHlvdQ0KPj4gdGhpbmsgaXQgaXMgd29ydGggcGxheWluZyB3aXRoIGlmZGVm
J3MgdG8gc3RyaXAgdGhhdCBzaW5nbGUgZnVuY3Rpb24gYW5kIGFkZA0KPj4gYSBwaWVjZSBvZiBz
cGFnaGV0dGkgY29kZSB0byBzYXZlIGEgYml0Pw0KPiBObywgdGhhdCBJIGFncmVlIHdvdWxkbid0
IGJlIHdvcnRoIGl0Lg0KPg0KPj4gV2hhdCB3b3VsZCB0aGF0IGlmZGVmIGxvb2sgbGlrZSwNCj4+
IGUuZy4gI2lmZGVmIENPTkZJR19BUk0gb3IgI2lmbmRlZiBDT05GSUdfWDg2ICYmIGFueSBvdGhl
ciBwbGF0Zm9ybSwgYnV0IEFSTT8NCj4gQSBuZXcgc2V0dGluZywgcHJlZmVyYWJseTsgZS5nLiBW
Q1BVX1VOUFJJVklMRUdFRCwgdG8gYmUgInNlbGVjdCJlZCBieQ0KPiBhcmNoaXRlY3R1cmVzIGFz
IGZ1bmN0aW9uYWxpdHkgZ2V0cyBlbmFibGVkLg0KDQpTbywgYXMgYWRkL3JlbW92ZSBhcmUgb25s
eSBuZWVkZWQgZm9yIEFybSBhdCB0aGUgbW9tZW50DQoNCnlvdSBzdWdnZXN0IEkgYWRkIFZDUFVf
VU5QUklWSUxFR0VEIHRvIEFybSdzIEtjb25maWcgdG8gZW5hYmxlDQoNCmNvbXBpbGluZyB2cGNp
X2Jhcl9hZGRfaGFuZGxlcnMvdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzPw0KDQpUbyBtZSB0aGlz
IGlzIG1vcmUgYWJvdXQgdlBDSSdzIHN1cHBvcnQgZm9yIGd1ZXN0cywgc28gc2hvdWxkIHdlIHBy
b2JhYmx5IGNhbGwgaXQNCg0KVlBDSV9YWFggaW5zdGVhZD8gRS5nLiBWUENJX0hBU19HVUVTVF9T
VVBQT1JUIG9yIHNvbWV0aGluZyB3aGljaA0KDQp3aWxsIHJlZmxlY3QgdGhlIG5hdHVyZSBvZiB0
aGUgY29kZSBiZWluZyBnYXRlZD8gVkNQVV9VTlBSSVZJTEVHRUQgc291bmRzDQoNCmxpa2Ugbm90
IGNvbm5lY3RlZCB0byB2cGNpIHRvIG1lDQoNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9s
ZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:17:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:17:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180980.327910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa2S-0004en-EJ; Tue, 07 Sep 2021 12:17:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180980.327910; Tue, 07 Sep 2021 12:17:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa2S-0004eg-Aa; Tue, 07 Sep 2021 12:17:20 +0000
Received: by outflank-mailman (input) for mailman id 180980;
 Tue, 07 Sep 2021 12:17:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNa2Q-0004eE-KV
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:17:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d943efb-468f-41e2-b01a-a931ddafbd61;
 Tue, 07 Sep 2021 12:17:17 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2059.outbound.protection.outlook.com [104.47.8.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-ynG92t5GMuyY6Ok5QxIYCQ-1;
 Tue, 07 Sep 2021 14:17:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5167.eurprd04.prod.outlook.com (2603:10a6:803:5b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 12:17:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:17:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FRYP281CA0016.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Tue, 7 Sep 2021 12:17:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d943efb-468f-41e2-b01a-a931ddafbd61
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631017037;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=3ipn8V6nm60M68/u9CKRAsm4Z70Kt+Rd/n32FD4uhi8=;
	b=PQvM20lBak2oM9lxT10OCWEacoyR54yuXM/tBcOUzxZ717k4qixkoyeL692Fohr/yVhufJ
	GnVksV4pyRmEGHEvShpeDX2eAEuKOEJll2R9GY1cMRHBAutP3M1dg0aRHDKv2KrZxF+Gkj
	hV0G1XnG7aYHnKeOae7oXem9CL4DpXQ=
X-MC-Unique: ynG92t5GMuyY6Ok5QxIYCQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k2/D0YK4Piymd4IYGSPPnaSuBPsLXM4g+bpydY5WVRFwRZTCR7LHm+1p2LylvxoaxlGKNwYyZn0z+MUI94e01r4pe90eRfBnKILaf3PQW8MRk3cYqhJTyRvv5ormM6lKP6AzXzBDl3RIt1BlEtvA8miVylPzFbttD/ammt1the9WOVCsz6KV/upY39/vvi7TYNPXQnmyGMtee3lGX1qKfOXQIk6ZdEZNDaLT3oTk81X+GeTVYGQzEGlD53+rOWN43kxhYnvLmDn5KGgL9UppBPL54ff7UFVk6IfodGUmL3xb/EniU93Cc5oo6cHrHcy4Tvgh92r/J0xhIHMzME3pog==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3ipn8V6nm60M68/u9CKRAsm4Z70Kt+Rd/n32FD4uhi8=;
 b=eJ2mb8DH+0p3F8SOxYXO5OJwCi4WWiSAEA6T3NdLKtnNOIzjFJ+1PQbZ96IWhUNWUHQrQTyzngeb5cflatDgoGXuptwj+kzwN2OzAhGtGG+vWIK4dYqtmEkmSwYZt6lkOCu3KRap8P8RoWvYUhkO7mC2pkSjaLvbJQndDlPsxX3Qhf33Nj3Yph2uOXKr2dm0QUDRw7xOigl5EooQH5Zl0bodEfiJ/zyJHeeU1b1wgSi/UXOf1d0UrObSQj0n8EHGTBb1P4veeedQ/6VdKMgt5Fs9ZrVdMtnUCjXG5Hu1cR9HfZh8h1BZ2L5mCg1TDd7vDpXL5wNiay9/w8E2LZV9+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] xen/pvcalls: backend can be a module
Message-ID: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
Date: Tue, 7 Sep 2021 14:17:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FRYP281CA0016.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::26)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c6d24a79-3689-461a-4955-08d971f96d81
X-MS-TrafficTypeDiagnostic: VI1PR04MB5167:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5167FB5D9536B52FA3A8EFBCB3D39@VI1PR04MB5167.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dKXCZSWhcx7GZDxjf8ctU81O3byUk595YyebVbpiEYxN2DQ/8P8TgUfZfWTLZVAAN1y9cXdW2spc0dLLDGIQE+9lY1RfBAtLeFK6mySywXuBXe34XQvnHmLZLvVGWv09c11npTA6XBrfsdwxVGE56OCANEE4O4s/WyGy8vJC0oiJOtkhcRnzr4vQLdgijsgKDrwxcrqZWiuDYTU8mnQr7/W8WRElBIxBAMtFSwJlKpki04x7Wi2QGOdCbSP+9iwkeX12u+hMdQfLccW66R6QFy3B9ickxSesa7pCNtWItDK7cX4FKWMR0o5vJrlXz+J4Z7VZhsyaz8xB+UNi1e7FuZqWmixsZG9rDag8uEApRlzjR7QF3EpMMe3c/P697CK5Ulx3lGEHStmPdLzzXfWAtDYyl4y7ezL9RlNYVgvOQoqunvtvQgbFQyIZV93JaOcIwlwscPSQM8Od5AdnonpXSMzQCNU6YFuOrEOhzjhCVHLEo4J/v9bADlzo7zTEDQYAkbBN0qV0eWXycDCNFONEnFvaCjLd727+VL+FzG5cY/KDvqLDTdUW1yGTXx7oibePhGmplrjAKBnoTj9HrTHXHPEiVv/F/tbrFl4TSpRk26i7gL+fVtyoCMnKJaKwPNYZ/OH5qb/FEYqZzZzDBQg2x+3jPTH3Qlo9taKxtti4wMaa5Bz/wRwm2aEnSl4AdtYGCNiaRhmRSFiqWZ5XlRyhuMcdLe2TxgfFK+ufUf769EU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(4744005)(66946007)(36756003)(26005)(186003)(31686004)(31696002)(8936002)(86362001)(2616005)(956004)(8676002)(5660300002)(110136005)(66476007)(66556008)(508600001)(16576012)(2906002)(38100700002)(316002)(4326008)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QzdMNjFBMHY2UzdqbmNuaXJEYXdMUzhXRk42ajBPYTV4cG9CUmlYTTRmNkVu?=
 =?utf-8?B?WlRSaWtIK3hDZFV0WWxhMjI3YWpiampzZ25NTUpCdmI5RndsUTJ6VENJMkE4?=
 =?utf-8?B?bzhqWlh5WHB6SDY4UkFYbXkvemxUQ3JKV2ttK1ZPOG4vN05rem1KaERVNk1x?=
 =?utf-8?B?RHdDYjczTk1TRk50UHZrTVFBT2NuMWZwS0k2WXV5ZzBsOUQ5T3ZLbnBiWExv?=
 =?utf-8?B?Wi90d241cFJydnNYbXEvNWxEMW12bmlYS0VBUVZhOEhTK0d0TzZ2NFEvN3hn?=
 =?utf-8?B?TTRxUHJuVFBpZzUrMkw2dkp1YzBZb3BPOUVISmNjcVFvTjFmTjBIUlJ2cEdq?=
 =?utf-8?B?d0RZS1dDajlUYlFQaThFTTVUKzB3bytzM1J5M1RIaldTeVBGZFprTTN2NlZ4?=
 =?utf-8?B?cW1IcGxtK1ZGZjhsSFUyek5yUiszampBaW5KaTAvTjJGcGsrYzlkdDA3c2cy?=
 =?utf-8?B?T2t5b1dxQ3pUbWdqSm1QMVdBbVdkZXZFTTB6UFZuaDBPVmM4dGluSlVrYmpw?=
 =?utf-8?B?OTVKV3EzdlhFcko0UW5hNkpjZnhKeXZMcEVKTnBCdG53bFUwYW1YV2sxWWNP?=
 =?utf-8?B?aC8zcGIrT0N2cldZSHJ4bzhZRDV5R3p3V1Q4TnY0aVZoNlJKZlg1bHg2cS9F?=
 =?utf-8?B?SElpMHlqVVlEVFpXNnlwOUxnVjJENmdDYmJNRFYrRDdYdEU0Q1BEMWNtUzVT?=
 =?utf-8?B?bUdQVi9FZlVycTJianpBbjE3YkxZeEJvV2pPZThBYXJGSjI5bTlGZ2lJRlgr?=
 =?utf-8?B?ZFFCaXhEUWx2VXQrL1NkL3NpOFREZkJmVkNUQnRCMTc3eDdXWmdFRmdoL0ht?=
 =?utf-8?B?a2tkU2RhS3cvVERxRTVEQ1JIUStmMlE1cDJNNVZpQ1BaUW9WQndvNU1TdHE0?=
 =?utf-8?B?cTA2UlkxbG1jSkt5alh4NFhRdnRGa2U1QVAyakNudlZWT0VzTWdBQ2VIanFC?=
 =?utf-8?B?bGhtakgwMzQvbXRtbHFRTGZDYW5zdHl4OXdiU2xMMTBjZHRVTm9BZGRpbjI2?=
 =?utf-8?B?UVpVOG9pcFQ4VDhRQzN5bzdqU2JQcXAxelFEVkJQYkNIeW90SzE2VDJFSEsr?=
 =?utf-8?B?bGExS01DV3FZR1Q1Mll0UWtwSFR0SXpvVld4enhZSTVvVUFSWGRIQ0Jlc1V1?=
 =?utf-8?B?RS9ZUWtOVmFTR0NhQlhkcTl3NHNqbThoQ2RVcTRSSm5MZGMzb1drc0MwZ2tT?=
 =?utf-8?B?d1JPMlRRU1N6S1hNSTB5ZHArNkxWUEhEdHJNcGVKQjJ0dHorTFhpbXh5WXpO?=
 =?utf-8?B?TVFGdWxzc3llZEhMYkRlZkJINHVXSzBPR1I5RGZZa1cwT2hxVG9UME4xQkF4?=
 =?utf-8?B?RjhLZ2NrWDFVZ0dDYWRWSjloSnpWTWlTT2EzZ0w1cTBxaXBxZkVVcTZnZ0hy?=
 =?utf-8?B?Mk5QemF1aVJvMkZ4UGhrQjRodUJiWm90cU1MK213WUJsNnhsVGphNUpYZG12?=
 =?utf-8?B?RUVVMjlUS05qNi9BU3Z3b3c5K3hMcEIreG1rcG9YUE9CTTFDUGE1eWZWb1Fn?=
 =?utf-8?B?NmlwOVptYmw0VTVJZE5oZHJjSkhwYWZHdTB6QWQvOStQUE1WSzNpTk9NV293?=
 =?utf-8?B?TVB0ZktSaE5vcEZEaWxRN0dTaHUyT3l0SUNLTCtrV3BJSFFZa0orZmFiWC91?=
 =?utf-8?B?YXVyYVpLdWxrUVl6dmJpYVpCNEhIdVlMaE1MNUxUVGV5NnF4S0xkWldrODgw?=
 =?utf-8?B?bVR1QWRtbDVybjlGTVl3NkRFcXdMUU1ZNlozc2dWb2Q2Q3pvZ3ZkWkFNMzVT?=
 =?utf-8?Q?dDqB7Jp0rT2zc2ec6e7OtK5PloxE22JlaUo06mW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c6d24a79-3689-461a-4955-08d971f96d81
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:17:14.8640
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qyYhDfgmVp5Amf3Ybl/g+1qZhBdykNGIiV3Ow3rZUsg7gAAXf4azQIB58EM4UtJrKMb8vmd0isW+P+BgiFTVuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5167

It's not clear to me why only the frontend has been tristate. Switch the
backend to be, too.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -214,7 +214,7 @@ config XEN_PVCALLS_FRONTEND
 	  implements them.
 
 config XEN_PVCALLS_BACKEND
-	bool "XEN PV Calls backend driver"
+	tristate "XEN PV Calls backend driver"
 	depends on INET && XEN && XEN_BACKEND
 	help
 	  Experimental backend for the Xen PV Calls protocol



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:20:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:20:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180991.327921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa5c-00069q-2X; Tue, 07 Sep 2021 12:20:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180991.327921; Tue, 07 Sep 2021 12:20:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa5b-00069j-Uf; Tue, 07 Sep 2021 12:20:35 +0000
Received: by outflank-mailman (input) for mailman id 180991;
 Tue, 07 Sep 2021 12:20:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNa5a-00069d-Ly
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:20:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff3fde82-0fd5-11ec-b0ec-12813bfff9fa;
 Tue, 07 Sep 2021 12:20:33 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2058.outbound.protection.outlook.com [104.47.8.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-6oMtLmXNOI67S5XH6MHvJg-1;
 Tue, 07 Sep 2021 14:20:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5167.eurprd04.prod.outlook.com (2603:10a6:803:5b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 12:20:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 12:20:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0074.eurprd04.prod.outlook.com (2603:10a6:208:be::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 12:20:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff3fde82-0fd5-11ec-b0ec-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631017232;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wOo934swFYPJYytLsLK7LxicndHY3S6qCxAIAYTwKao=;
	b=BdsFBVHImLm1iUAZU8KtAJYLKE1WiQzIcoUmXgiOKellqhiurPtxMdjMRGQ14a/V3SB3Bf
	nShNUhY5HMfXb1t+FnI1Uj3JaQUBpnEq4jHylOZ9caXUkgN0VGDtlEWooQG/diP54RfH3a
	OVYV+mVOqUNh77viXyciTy63SILyL/I=
X-MC-Unique: 6oMtLmXNOI67S5XH6MHvJg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H0XAC/1buMfHNfuTp6lcySLLCKJ26HRv19rypIzje4uceXbmcIq9nGbHoYuhXpTFJ7wyxxm4KJLmzHglO2f/Y50QSPFE2bbwy9eN2mfF4dS4icuXPesd7/b0EsWd+v4UraFJk8mqAutocwvbkUCwnVFXLON1KBAVdLbVw4j7G79fGPABkDIwR6CWTNI4mu4xLZscR1XYlBKV5kjib0lEOq2lT7sfOOM5BjMr6ZmBPZRQdCMgaDg5f+knGnc/lHxgrxAX7fIgu48FD+N+hdggeFWgDwdXrYoGuPoDHLDacZNGpjyvqhgT1ZspyurYz9p0+Qr6yCHvvMiSqaQFx6jPPg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wOo934swFYPJYytLsLK7LxicndHY3S6qCxAIAYTwKao=;
 b=oP+AuPA9M+n9XXLVIIxOuyaHCAkc/h+F2qy7316wfmO3jM50MSH6be8pp1MKuLLqyaPHYZ7MpRo5YT2h59Q4tMvIttVcEKbpcVQEKdoMocR8eIsWkKOSh8vrLUukGLO13XFqqnRbCSbjPEZWeShi83gG+JjMWk89g2smdb5I3f13cPRwumbMWesgMcZ/pxQArA7hcIC/A5/frM/pf2eqaV4eDsarbvYNX+FNnPzLPQOsb+sSMPum88VH+8BVm12su70/WiqNEASR+UBehtZgLUk8gqIKrSMd2ExX5FEjanh0V8t/zy3FzqZBt/GSvcKFB7hkvSFIWPs+4gsL5LUMzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
 <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
 <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
 <938156e7-cbd0-a19e-f09f-a41dce051367@suse.com>
 <b4c94187-db81-d52f-0425-959fe1e9e83a@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <de2165eb-0c81-d020-f76a-3a8f97fb8cfe@suse.com>
Date: Tue, 7 Sep 2021 14:20:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <b4c94187-db81-d52f-0425-959fe1e9e83a@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0074.eurprd04.prod.outlook.com
 (2603:10a6:208:be::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a8c0a58-6911-46b3-186b-08d971f9e1e4
X-MS-TrafficTypeDiagnostic: VI1PR04MB5167:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5167CC82093249586D7FD450B3D39@VI1PR04MB5167.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lt5rPZjmXJgizDDx6O66cYvY4i1eFNfCxwQ5sdT+zeuAvBub4ahGOnjUtHjWMJsmcUV33iU4aNVbjPQELzedL5T2Q2qIfQO8RGvDTRodeUoGghqmrx86P+Bia1y6WDdelxiGvDJebGF0vay0gg2cfznq7vO0Nu5RkdtU1Uv/+7mers+jsAHzUgRIz3SLLxg844Wl330hLtb8YjK4osA1M84ckkkROpMNwgOi/QBillx8POhl5MWTCSgXpHsThU7Q3btr1p7i/+egnRB7t2hk+8ZUjAb13OI2s/BrvPBfLywACBr63ofvPNOlg/0t9MkKsfp22WWTs8FVHcNS8yWM9sOefx7J9U8XSfd1rdcbt2OjSGj3LrFZh1NsVaohheP528V2piVRdnKJsqAXscU+UZ1VEshW30NchqUXYxMAj2T0X4LA0T1+Y8X4AoLMIuHLu03+Wf5NfpvqhXIF4swbYe14XSwI3vkdxJZDzHZfAzB2kV6DCdnwVkcEQCmD1dxT+1jegX+PaaE9kCWUkrGfCkDSo0SCVILogZL1q5OGCfwlMV7cZv6+Lh3Jz0dU9XJpTwxWslAmgQYlrcfxgZeYj0w5AvVBTMy5zPWfhtq4LhspLNtiitHb0366PJGPTE5jXWbJbg7BCLWclMoSjo1S/oNIgA9HYwQOvoabACxhbQdsnayUqM6CgnQ0spnas1EGnyfV5ApyXxXytun/rFJXVkf5ILLngfbXnEq4csaah7MRHt+/CqPqJ19ocZIlQJa4gmD1m501mDs6zHDbZrymaA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(66946007)(36756003)(26005)(186003)(31686004)(53546011)(31696002)(8936002)(86362001)(2616005)(956004)(8676002)(5660300002)(83380400001)(110136005)(66476007)(66556008)(7416002)(508600001)(16576012)(2906002)(38100700002)(316002)(4326008)(54906003)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cjEvcUZhR25NaTU1ekhiN2xFa296bUNoSTZFNWtKaHEyQUltUGgxZzZBL21M?=
 =?utf-8?B?ODZCNjhKZ2haWUxhSzdiUnIxODk4bVQrZE9iYXpTMlpRT0lKWE5NUXBLejJy?=
 =?utf-8?B?L3grWWhBK1JxRnBKQXV6WXpEZGNtZkxiN2tueUdBbExkTnFjcXRQcEJQWHNY?=
 =?utf-8?B?dGplNnBqVUJzWXd5TDhxL2dNQWFXOFRCTldZcnQzVGxOQWdGY3I5QkFYREpz?=
 =?utf-8?B?bHNaa2VFNWZMN0EwZm5DVkdHOHlnYjFxaWRoOEV3WE0wWUNZc21PbzMvLzh2?=
 =?utf-8?B?RE5qZlNiSzBjMEtXeEFDWnhiajQyczNFS2ZqUWlXaERHU1E4eTBtSHEvZzkr?=
 =?utf-8?B?aEJZRmZkUTZOb3lVUVBsZWxCdXNnV0VvL2lySjRjMUVrN25vSnFIZys4VHA0?=
 =?utf-8?B?R1ZMZERYZUc5OE43QVhuaTJpMGJUNUx3VkE2QTFPUHI3NHBnNjVMa2VPSXNS?=
 =?utf-8?B?NzBkeW9mWGZwbTdjNXNIS2VQOWNlWHN4Y0xTVXZSWU12ajJwNTlzRm1sWlN0?=
 =?utf-8?B?YTlGVHQ4WHE1YmVpRG1HSDZ2MCtmMGpLdWd4M3Ezc3c0Y29DTzNiUjk1MEVq?=
 =?utf-8?B?eVB6bHZ6alJidVptMWhDbXdBcDRrSVVKUFgvTGc0d3VhMnJjL2FqZ1VhcFhN?=
 =?utf-8?B?Zm1YZVhveExCcnh0d3YwMkpOOTlvaG9hVTRjb3BWeTJ0eS82OFQ3SFBCTU5p?=
 =?utf-8?B?ckJWVi9lMjV3b2RBL3BMUitpdTlzdWpUaUZ1TitFTkN0WjNoS2FtWWN4ZXVU?=
 =?utf-8?B?aDkrVldqM0tkT3U4bzZJLzh1T1Rxc09wWE05ZXNRZlI3YVpGTExZSGd4NmEr?=
 =?utf-8?B?ZUc4Znh3aGhvUUE4cDVzZXJtK1J6eGFsNUFyU1E3UThzV3lPSzNOWHZsVzQ2?=
 =?utf-8?B?RDBPUHQ3blJYTnZBV1c4RUk0SWpibVJtZVpGQ0ZIUmhaZTVlL3FmSlkxQlpq?=
 =?utf-8?B?OWl4L1RvVEJEOWV5VVhuanBLY2lldEdjVUI5RngxVGtwWUo3clRwbkVacndU?=
 =?utf-8?B?aGpmOXF5akliUVlVNXdXWUUwRXNXQS9NQnAzNnpUUkVla09YYmx0OXY2WExk?=
 =?utf-8?B?YXhnekhXTGd2Mmc3TUtRRVdpZm13TXN1U1poOGM3eEtETUFXb3Y2RlkzNjdU?=
 =?utf-8?B?eVBGOElYZ0hKV09LcXpaVjA1YWZWeSs5NGI5RjFOWTFrdUFDQUltNzFqK1pj?=
 =?utf-8?B?UURGVUh2K1lXUlFXeXhYQURZYkJvT1ZwU25CMkRGQnpSMzl6NE9tVXZ2UVFT?=
 =?utf-8?B?a3huWlhtYnpnZVpCWUxvY2lHSDloMVJBaEVoRFBaSkZBN2gvdmJwL0RkY1d1?=
 =?utf-8?B?ZWFZcFNneUV2RW9sRnQ1WWpiY3V0bXIreTFmN2x4UC8zd1pDdnhkK29HamZ1?=
 =?utf-8?B?R0IwS3lEaWVmRUUrYlBaZTJGTThMcWZCZmhxQUxGY3VHZTBjaXllS0JYaHBw?=
 =?utf-8?B?Vks4ZDdCajJQWjByeDNVNnlieGY0aUlMNWhleHZRRlk4eC9MR3U2QXlPQU1Y?=
 =?utf-8?B?aHlPbGFUTDlQdExBa2xwQUVTWVI2aURGOVNpRlpmcnZOQ3ZVdHJMVFBuOUhO?=
 =?utf-8?B?Snk4aTMvT0d4UUx5UkxLSHR1bzgwS210Z015bTdiaWJDWG0xQlV6YklBWUhv?=
 =?utf-8?B?Z21qZHU4RTZIT0VubE1wa1VhV2VhRVpoS3p2amM4a1lJVXlHVXBUcHUrakRJ?=
 =?utf-8?B?TGtrUU5aSU5Ja1ZVQ3Z1bWhadW93RDdWMENMaklqNk1UaGozNlZKWjV6Ynh3?=
 =?utf-8?Q?wkP5PargyR6utdG3vLZqxrGDqlyyt/oYzqIP2RQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a8c0a58-6911-46b3-186b-08d971f9e1e4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:20:30.1402
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: swnGz/9p8fsPUvGOyqrBc14O/PWj0wDu1T+seFWUgyc89qHb+UJuIGqzEUHbnyKxJypJf2RTXbC2OT+apmmd2Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5167

On 07.09.2021 14:16, Oleksandr Andrushchenko wrote:
> 
> On 07.09.21 14:49, Jan Beulich wrote:
>> On 07.09.2021 13:10, Oleksandr Andrushchenko wrote:
>>> On 07.09.21 13:43, Jan Beulich wrote:
>>>> On 07.09.2021 12:11, Oleksandr Andrushchenko wrote:
>>>>> On 06.09.21 17:11, Jan Beulich wrote:
>>>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>>>> @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
>>>>>>>     }
>>>>>>>     REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
>>>>>>>     
>>>>>>> +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
>>>>>>> +{
>>>>>>> +    int rc;
>>>>>>> +
>>>>>>> +    /* Remove previously added registers. */
>>>>>>> +    vpci_remove_device_registers(pdev);
>>>>>>> +
>>>>>>> +    /* It only makes sense to add registers for hwdom or guest domain. */
>>>>>>> +    if ( d->domain_id >= DOMID_FIRST_RESERVED )
>>>>>>> +        return 0;
>>>>>>> +
>>>>>>> +    if ( is_hardware_domain(d) )
>>>>>>> +        rc = add_bar_handlers(pdev, true);
>>>>>>> +    else
>>>>>>> +        rc = add_bar_handlers(pdev, false);
>>>>>>        rc = add_bar_handlers(pdev, is_hardware_domain(d));
>>>>> Indeed, thank you ;)
>>>>>>> +    if ( rc )
>>>>>>> +        gprintk(XENLOG_ERR,
>>>>>>> +            "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf,
>>>>>>> +            d->domain_id);
>>>>>> Please use %pd and correct indentation. Logging the error code might
>>>>>> also help some in diagnosing issues.
>>>>> Sure, I'll change it to %pd
>>>>>>     Further I'm not sure this is a
>>>>>> message we want in release builds
>>>>> Why not?
>>>> Excess verbosity: If we have such here, why not elsewhere on error paths?
>>>> And I hope you agree things will get too verbose if we had such (about)
>>>> everywhere?
>>> Agree, will change it to gdprintk
>>>>>>     - perhaps gdprintk()?
>>>>> I'll change if we decide so
>>>>>>> +    return rc;
>>>>>>> +}
>>>>>>> +
>>>>>>> +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev)
>>>>>>> +{
>>>>>>> +    /* Remove previously added registers. */
>>>>>>> +    vpci_remove_device_registers(pdev);
>>>>>>> +    return 0;
>>>>>>> +}
>>>>>> Also - in how far is the goal of your work to also make vPCI work for
>>>>>> x86 DomU-s? If that's not a goal
>>>>> It is not, unfortunately. The goal is not to break x86 and to enable Arm
>>>>>> , I'd like to ask that you limit the
>>>>>> introduction of code that ends up dead there.
>>>>> What's wrong with this function even if it is a one-liner?
>>>> The comment is primarily on the earlier function, and then extends to
>>>> this one.
>>>>
>>>>> This way we have a pair vpci_bar_add_handlers/vpci_bar_remove_handlers
>>>>> and if I understood correctly you suggest vpci_bar_add_handlers/vpci_remove_device_registers?
>>>>> What would we gain from that, but yet another secret knowledge that in order
>>>>> to remove BAR handlers one needs to call vpci_remove_device_registers
>>>>> while I would personally expect to call vpci_bar_add_handlers' counterpart,
>>>>> vpci_remove_device_registers namely.
>>>> This is all fine. Yet vpci_bar_{add,remove}_handlers() will, aiui, be
>>>> dead code on x86.
>>> vpci_bar_add_handlers will be used by x86 PVH Dom0
>> Where / when? You add a call from vpci_assign_device(), but besides that
>> also being dead code on x86 (for now), you can't mean that because
>> vpci_deassign_device() also calls vpci_bar_remove_handlers().
> 
> You are right here and both add/remove are not used on x86 PVH Dom0.
> 
> I am sorry for wasting your time
> 
>>
>>>>    Hence there should be an arrangement allowing the
>>>> compiler to eliminate this dead code.
>>> So, the only dead code for x86 here will be vpci_bar_remove_handlers. Yet.
>>> Because I hope x86 to gain guest support for PVH Dom0 sooner or later.
>>>
>>>>    Whether that's enclosing these
>>>> by "#ifdef" or adding early "if(!IS_ENABLED(CONFIG_*))" is secondary.
>>>> This has a knock-on effect on other functions as you certainly realize:
>>>> The compiler seeing e.g. the 2nd argument to the add-BARs function
>>>> always being true allows it to instantiate just a clone of the original
>>>> function with the respective conditionals removed.
>>> With the above (e.g. add is going to be used, but not remove) do you
>>> think it is worth playing with ifdef's to strip that single function and add
>>> a piece of spaghetti code to save a bit?
>> No, that I agree wouldn't be worth it.
>>
>>> What would that ifdef look like,
>>> e.g. #ifdef CONFIG_ARM or #ifndef CONFIG_X86 && any other platform, but ARM?
>> A new setting, preferably; e.g. VCPU_UNPRIVILEGED, to be "select"ed by
>> architectures as functionality gets enabled.
> 
> So, as add/remove are only needed for Arm at the moment
> you suggest I add VCPU_UNPRIVILEGED to Arm's Kconfig to enable
> compiling vpci_bar_add_handlers/vpci_bar_remove_handlers?
> To me this is more about vPCI's support for guests, so should we probably call it
> VPCI_XXX instead? E.g. VPCI_HAS_GUEST_SUPPORT or something which
> will reflect the nature of the code being gated? VCPU_UNPRIVILEGED sounds
> like not connected to vpci to me

And validly so - my fingers didn't type what the brain told them. I've
meant VPCI_UNPRIVILEGED. I would also be okay with HAS_VPCI_GUEST_SUPPORT
(i.e. not exactly what you've suggested), for example.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:23:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:23:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.180997.327931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa87-000772-Fz; Tue, 07 Sep 2021 12:23:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 180997.327931; Tue, 07 Sep 2021 12:23:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa87-00076v-Cn; Tue, 07 Sep 2021 12:23:11 +0000
Received: by outflank-mailman (input) for mailman id 180997;
 Tue, 07 Sep 2021 12:23:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNa86-00076p-Kl
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:23:10 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dce6d494-7b8c-4768-bfd0-12b0b54173a5;
 Tue, 07 Sep 2021 12:23:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dce6d494-7b8c-4768-bfd0-12b0b54173a5
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631017389;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=5SRy+GkaKSYNYkCp7c5Yrir90YVimONIfq6a0GmK36Y=;
  b=O3exGyuvoVxoIpeL+Y8IkZzVYAPaDz33DOqNotY5f01VOfClb8T7fiMK
   O9rggdneKS/4Hqt9sb4fx/Moyw2IPS2/Bi3flYFVy51Bhm+tIlXA61eTf
   xgBrMdW6gnpdBRx7Bh85HREhG3dn3Q0HJNRpfykI/eqgQvkjA6anMj5Bu
   8=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: jgCqq3dw9YpX5Uz6TmLwID2hGPT3wGsY+Pt9XkCfIKmBQxY58vc/UY1hXkCel8SlXLI+jyvYjr
 fDbl43hkqy8WusXCZv1e8B2uWDyAbKJY2xovtQa/iShlEybzgKCgbfOZl1lvVoDmhtcc5Ol7Dw
 4RS3dfKc73sbZnzxjmNGapy9gLedzhKh8v8j6w/ZAgFN8ivoII0DV/PCaN07HpT/Q3DyV+krS1
 toxYM7aL2XCogcS4mzA9iYmhhkABxUkUws9ItiHWlq9CWzUGJSP2CFIVM4y1+yVkbHTFaGtJdq
 2aHIxuc+i8PIxfecMaP6rIBZ
X-SBRS: 5.1
X-MesageID: 52134917
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:cl9QIq4XmHu+TJSn5gPXwAzXdLJyesId70hD6qkQc3Fom62j5q
 WTdZEgvyMc5wx/ZJhNo7690cq7MBHhHPxOgbX5VI3KNGXbUQOTR72KhrGSoAEIdReeygZcv5
 0QCZSXCrfLfCVHZRCR2njFLz4iquP3j5xBnY3lvhNQpZkBUdAZ0+9+YDzrdXFedU19KrcSMo
 GT3cZDryrIQwVtUizqbkN1OdQqvrfw5evbXSI=
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="52134917"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DbrTKVl9PF8CCdv0X86WEEyyYY2eoAwUGvtPdhggnbJcDTWb60btkCZJjq+/Hm5N0tPU6+F67ML7X3iZ+kTCA1A8WSz1zXFiyN0C9Nwp+gviBh3sh2KiLHwnN1CP6FRVKUYYl580W+rAkry7DJecJa7+0H9KtiedWZe92eCu4uGtC95ZK1YLAptv4TmBiJKEWWVMBC39yE25xvLy/9dhaW4kBtuqpwpXG2Xxgo+iPHoAzS39BHkW1wvn48jisU5BzYVCT9JTd9PrOM/2MyO/4/oXwfvXSlRZKF5fjYQFm91j6P+HbPr/HrdU5JYSR5NLmIOuuYjX+wRnsFncPZ/2jg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/89I8PoBYdPweDDXpJU0m/Tu0Qc273nHKnykOHY2riw=;
 b=N0p9uIBL7STgRCbOjYw/pHLAtSpUvn6pbcTopk8jytCsOGy9eaR06igKBPy49OcC4BOGi4v8s9+BzFSfQQO+hH5R+ZBHdVfsucUDWlTS/uM3wVZpoQ0a0N9JiiV9xJ1eaaLtv3plBYoinR22Oo36082Jcg3+n7PJ/Xm0fDQQqWruxff0Z2jN8GKcyOx8+R3yJkaXJZjr0NR040I64rq9Lc26qYiZP87q6MxjyioT8/k3SGPvKIianSC99Gg7Srcw/6q29mrqaSpy1w6jQFsWSA6PQrmAcx9NCFNBSKZrfdRXZoMMAGAVynR/roUSWBhxCjZILOMDAKwMlnzyTeVefg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/89I8PoBYdPweDDXpJU0m/Tu0Qc273nHKnykOHY2riw=;
 b=H4grYPRDrmf51qPqwBOf2Y4BH4XzlxHARHZh2pDAn2zYey4Flcvba5JmljPW3HejUTkd2MZIi2EBUhUwASW8Cxp2kSu75ll7pUCEuRgxMwzJ3L+nigF1xPecsaB93b1ZUa3KcXY6OwZcCriSka5yrP6vCKdypmIB02avs/3pW/Q=
Subject: Re: [PATCH] x86/p2m-pt: fix p2m_flags_to_access()
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <2ec3fb93-5b91-15d0-4769-adf414af86b9@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <5fb0ae14-3838-c0b6-3e1c-ef5b44129186@citrix.com>
Date: Tue, 7 Sep 2021 13:22:52 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2ec3fb93-5b91-15d0-4769-adf414af86b9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0427.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::31) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bd312749-5c94-40c6-f67b-08d971fa3a2e
X-MS-TrafficTypeDiagnostic: BYAPR03MB3494:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3494F42D32FB36868DDC5641BAD39@BYAPR03MB3494.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 8djyJaybhNVSV1A4J0PLX4ua64QGBCA9kfobzAKBGM86WvawdgU20+Pi+71xnaO0+RlC70rR/7yLWFOlxGqPDrEua+8w56jdZPHVUnvxTbM45rmZi9QLo3Gi9JG6pN5v3kSzlRrtE8dBXn8YtNwvl80ANQ5eUkiOYnBhrzlxs505nrsMsuwbZ5B/GBkuahtiG2YBRxIXl2usLQT4kL6iPbXwP5K3G4dql87YaGs7zlWRA1a6seS5S1sWq6NJN/8R4XCE2v00iGID9hQxvaKu9dG0wpbVJQLtMAEhVPbVa10NYhshQWlOoXDzp3CyzXIWX1YHyvAuB76pkVNLUDd9Ja6uwgoaUkbIDik/0WlJvW2+cIF9kT8sW0u+s0OLUwuqmt+13VJPAlIXocl37wjPboQ8oHPfD/kW/1s+WokgmSWOJRbzIiD52TD2d5FierZPOwnq13Gk4mfAqkwZ3q+fmuiTy+dEQNUURhCmQZu/XcprJmtr45rAyYUaEiRWNvk7T1B7WuUleodJrQpbME2JLJ79KKW1NJFvqHMYap7TF414ov+SYEyob9jyU6rWz04haEiqMM2pQo0p0vzf/RFQF4RKHy6fUjJEC8J7puQ9ySnzInfrZ2Gz1dMHrqTS9RGDU4qLBE2eeacYjtZ3Nq4FJwkQ+yqTaW+Ax4lmc5KC1YbRc7qGGaE/1A8QNyS4WoOpn+kmOT9mpbrifh7lxGR9Dx9O8+NuGxVmYdcKistDEjg=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(107886003)(2906002)(86362001)(36756003)(4326008)(53546011)(316002)(38100700002)(55236004)(54906003)(110136005)(956004)(2616005)(31686004)(6666004)(31696002)(8676002)(66476007)(6486002)(508600001)(66946007)(66556008)(16576012)(186003)(26005)(5660300002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dmNXbmVYY0VDbEhZUVJmeTZyQ2g0ajlid1RTYnJoNXU4Q0ZaTjhaV2wwYzFw?=
 =?utf-8?B?d1hqUWpRcjRRS05KZDJmb2FqaWhVWEZmMnJicXZiNk9uV1J5YmFpaVZOSUp0?=
 =?utf-8?B?VW51QkN5R040UjJuRTYrcUFXT05VWGZKdHQ4WU5JUXRqaGJGVW9yTEw0U2Jp?=
 =?utf-8?B?V0x2ejR2V29rMGliU2N3NElhYzRkUUk1ZGhpaWV1Q3FBUkxRSjhIa2diMFBq?=
 =?utf-8?B?R0JNSzE3YmhjY1NleFFWclp3VGpsV0REVjlLL1Vac0t6TnJVWHZPUHdhZ2pE?=
 =?utf-8?B?Z2luaVpVSDFXVkhHS2cwdSt3eEdVUGgxSFRsNDF1WEtQODBIZUgralpaNnpV?=
 =?utf-8?B?aU9BQjlWdDlCeVFQSDZIN3l3SHZ4ZEhnc1U0ZFRCNFcyc2lOQTFsQzR6Y0Fw?=
 =?utf-8?B?ODBnWjBlVHZvZ3k0TlVyVUZUSGM2MU9nNnc2TlJ0RUJtWGtadFNaZG45QW9I?=
 =?utf-8?B?dElqdXBZRHF4WjFlcGdLVlhITExtRWpnanBNQ1paVGpXMkY0d3R0YndSQ3dE?=
 =?utf-8?B?cVVzRmFseGxSZ2c1TEZVSkJibThFaXpVMld5T0l2NkMxUUxQVUtLWC9UYmhR?=
 =?utf-8?B?T3RzRlcvTTlNbHlHaHBIRXNWSHpJYS9ScFF4Qm50djNHc044OEJKcEpzcDFH?=
 =?utf-8?B?YlZFVEZFWHIzRnpMZi9sOC9HSXF2bldvcHVzQzZ6Vko2YnB0ZXR5YjlOUGpV?=
 =?utf-8?B?RENoUlFsNUVRUzRLR0NqWnNsbVVYTkFEZFVtREdMR0J4VXVVL1JmMVZabHpl?=
 =?utf-8?B?NVJwSjJudThOUTVXVVJSR2gyUXZEdUgwZ2hPaGYyVHdVbzJTWC9ZQlNIRW5h?=
 =?utf-8?B?ZEkreFd6YkVtenNTNlJWSDZtd2RNOVZVSDhFSU9KY2Z5NjZ1NTVYanJmeWF5?=
 =?utf-8?B?V0hlMVM4Y29sRlU3NnE5TGRQZjNBM256OXpmTXRFdlZKOTU2a1NqRmpERVBI?=
 =?utf-8?B?bU5USWpjb3RlZmJRWGo1VmVnYXRMSG9DZktCTlMwaUNNR3lmay95M1FVcGVu?=
 =?utf-8?B?RGllNXVtRWduNUZSZktqaW43SFAyNVNHakdSdXM5aTdHZU13OFBEYkkwaGhQ?=
 =?utf-8?B?bEtoMXNIZHl6Yk92bGxSN2JFR0FxY0pOK1poOU1nem15Q0RTK3MxV1NlRFVU?=
 =?utf-8?B?MWt2c2w5VjVMNVNoc3FKc1R0N3hIUW5zWE55WVBHWjZlV05aOWg3VHpzV3lm?=
 =?utf-8?B?S3c3d1BjcXV1K3BOVnRWNnZVNi8vYVVJOEJnc2VQdEI2Z0tGZ0k4QytkbnZv?=
 =?utf-8?B?eFlvSVdCNmxrY0NkMXlzaUxrM282RU1TeWs5Z2c0M2cwYVdUT0k5TE43azRX?=
 =?utf-8?B?NWkzanFSVDd0QXczaytsMWIyRGovY2IzU0RnREdScXF6REZVQVVLYWlyVFJl?=
 =?utf-8?B?T2Q1dnIyeVVrblB2emh3MUMrYjRucU5OZGNLcXk0RW9VWUllcFM0WnU5VGVI?=
 =?utf-8?B?VnhBTFNJMFpWNS9YRmEzUXVzSkZHSmJhSnl0T1NBc2xSS1BjbmpjZ1V4M1VN?=
 =?utf-8?B?bXlrVld5Mm81QkdTQXUzOTZwcU1NMlMyKzdFR1Y0aXVIM1U5UlQraWdCMDZt?=
 =?utf-8?B?TTBaTHZuNHJydS9kUUlYYVhlVkRZaTZtZGJqOTFlcW94RnF6V2VMR0tyUFBE?=
 =?utf-8?B?dFdZKy9yOWhwVEsvYmRMeHpjRkx5Y3Mwc0RHRGdJZStNNU1pOGovUkc1Si9s?=
 =?utf-8?B?RG45VmRKUXRaYVEra3hjUmh3enNuVThka3dyZnFKVk1TdEVJR0FmOGIvWkNX?=
 =?utf-8?Q?GlENTxOiqcn19kcMir2EVu+8PDAgxpqi9MU7V2A?=
X-MS-Exchange-CrossTenant-Network-Message-Id: bd312749-5c94-40c6-f67b-08d971fa3a2e
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 12:22:58.3113
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /eeY+2sRbbDASCxnzucwgBZCabBEWqIfDzNmjwbC0DzmiFBmXW7JLrfOafQiXfNQE7fHoixs+cJoHE3uBHb9sW3ym8o4lLEdIAovFLEtpD8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3494
X-OriginatorOrg: citrix.com

On 02/09/2021 08:01, Jan Beulich wrote:
> The initial if() was inverted, invalidating all output from this
> function. Which in turn means the mirroring of P2M mappings into the
> IOMMU didn't always work as intended: Mappings may have got updated when
> there was no need to. There would not have been too few (un)mappings;
> what saves us is that alongside the flags comparison MFNs also get
> compared, with non-present entries always having an MFN of 0 or
> INVALID_MFN while present entries always have MFNs different from these
> two (0 in the table also meant to cover INVALID_MFN):
>
> OLD					NEW
> P W	access	MFN			P W	access	MFN
> 0 0	r	0			0 0	n	0
> 0 1	rw	0			0 1	n	0
> 1 0	n	non-0			1 0	r	non-0
> 1 1	n	non-0			1 1	rw	non-0
>
> present <-> non-present transitions are fine because the MFNs differ.
> present -> present transitions as well as non-present -> non-present
> ones are potentially causing too many map/unmap operations, but never
> too few, because in that case old (bogus) and new access differ.
>
> Fixes: d1bb6c97c31e ("IOMMU: also pass p2m_access_t to p2m_get_iommu_flags())
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

>
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -548,7 +548,7 @@ int p2m_pt_handle_deferred_changes(uint6
>  /* Reconstruct a fake p2m_access_t from stored PTE flags. */
>  static p2m_access_t p2m_flags_to_access(unsigned int flags)
>  {
> -    if ( flags & _PAGE_PRESENT )
> +    if ( !(flags & _PAGE_PRESENT) )
>          return p2m_access_n;
>  
>      /* No need to look at _PAGE_NX for now. */
>



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:24:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:24:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181003.327943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa97-0007gn-RH; Tue, 07 Sep 2021 12:24:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181003.327943; Tue, 07 Sep 2021 12:24:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNa97-0007gg-Ni; Tue, 07 Sep 2021 12:24:13 +0000
Received: by outflank-mailman (input) for mailman id 181003;
 Tue, 07 Sep 2021 12:24:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNa96-0007gY-Uy
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:24:13 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8167cb72-0fd6-11ec-b0ec-12813bfff9fa;
 Tue, 07 Sep 2021 12:24:11 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 187Bh5dS023794; 
 Tue, 7 Sep 2021 12:24:08 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2058.outbound.protection.outlook.com [104.47.8.58])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ax0pd9gv1-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 12:24:08 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AM6PR03MB4949.eurprd03.prod.outlook.com (2603:10a6:20b:84::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 12:23:59 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 12:23:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8167cb72-0fd6-11ec-b0ec-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BQzOyFXyiwn7PkXjCS4i5+hsvLQPObhwIj8r21RVL6zrg4bn0EnkiqcJ7eWyoUik8Q25OqVF7MPB6X6FjG8pTRKF1PljTIdbI3BXItU/HozIMixWCI6BG0/Y+jhQCJebRD+UwF3qXFg1bfqCbErW2pIYv/8jcyQUcNJpk1j86LpRe947vqbJyv7I3f9uTquxug+fUaeOMt+5kSeZjV6/NDPjFKnr3WR7oTRITd7DRSR8JsOHjANGMCdVeJWldQxFK13l8hHzADax34TJsVg9tH0DGr0oMMCdyL/KYmUOUGUHtG4Iml3MkkQY1UTtW/2dFwZHX0il9fnFf/98hxLlhg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8qGrsR7JiIMdJ8qARnR6BCFPDRyxpVUqctH0MnLAb4A=;
 b=VvjL9WlzXozdtpeyRgI3lTABGgYuA9AmQK3c5xFt3jYu3GxvKt/RNC8luah0UHv0L9pBeE/MvoQFuFf5jgZuhjNatF/GCIDM6sXxfwGkRZ5DpBdN26adn8XaQTM2HHxvORSqQPki8OMfeWR9nMGuHA91PWDMyqQmG4/u6TgF5dEiZrH6Nc0qbMRjKX1QZGO3A4iLbR1DZZgiznDB/FJ9f5jIs+EvPCbV0x48vjlH5437CJ3HE+4D1nxVXpIqoQzoNIA3MijPLlE+6WGOtuDNgMIQP3poyGWEq8s6TnMmJQ/K+UZJbPcJetkNXupy9H7aOKKBUOYn7Oy56bmDP+X1LQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8qGrsR7JiIMdJ8qARnR6BCFPDRyxpVUqctH0MnLAb4A=;
 b=FyFbW+l0W3d5Bd+k9ybDUicMI13K95s09w5XXFirqrgcEGG8Dd89qGx0m48RcXL/zQs5SRJd5Fg4WzIilfUmMph8h7GiWFGcoU0vbCKvFnrD0NgpvIgpk0/zCcyip30WwHYqYywh3pQ3gVTR2FLfB16bQIfzBurWAsALh9KCwScyT2Tm8G0UCEfZ/q0q3HMcRZu/NLU+FHRE/9z3rVFZq7uj+ST/u0dgwlGKWnCLd7XJO9+iOMLDtsQHb95ZUqUqnidswuMQ0GDsG/hbi8UUYay9ZJR22Tmhml9F+zUiAJlF43InoGOBIDgsP0eYn+6nOU+v4FZGEgPLX/tBAGViNA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Topic: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
Thread-Index: 
 AQHXoKusySJnLVveGkulaWKrBZdXpKuXEOQAgAFPE4CAAAkRAIAAB3eAgAALDQCAAAeQAIAAAQeAgAAA/AA=
Date: Tue, 7 Sep 2021 12:23:58 +0000
Message-ID: <37b08bea-2e0a-1a9e-aa26-8e3f61ddce7f@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-5-andr2000@gmail.com>
 <8bb58d2d-255b-96e9-50ff-529985c655d5@suse.com>
 <16e396be-89f5-4d42-d57b-d91f09b70644@epam.com>
 <00bbe12b-6a79-e660-c5c1-57ccb200d2eb@suse.com>
 <242373cf-9854-74b7-4d54-bbcacc6cb7cb@epam.com>
 <938156e7-cbd0-a19e-f09f-a41dce051367@suse.com>
 <b4c94187-db81-d52f-0425-959fe1e9e83a@epam.com>
 <de2165eb-0c81-d020-f76a-3a8f97fb8cfe@suse.com>
In-Reply-To: <de2165eb-0c81-d020-f76a-3a8f97fb8cfe@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: debb9fc1-4a64-4f7c-81d1-08d971fa5e86
x-ms-traffictypediagnostic: AM6PR03MB4949:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM6PR03MB4949B32902410F0B635119F5E7D39@AM6PR03MB4949.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 BMbD0oZNp93Gkku0wuU9+6c9HEzB/R0f1rt6hAeRi3XPTpYVVzyqqvZvt1n6MVqfQuE7IGyg8L+wiCwP8ojCc7Bm3A99RcEMY0OV0mHX1bSGRvutFTfD1VIohjIOHFQUeoaTFi0SQNJpnRcbyzOL9fBu3hqvxj+0t59WBeL7wwgxQ1m/C7k5cSWCoIVWslBfNdOXn5mDU7zoftl53w9YvC4Hwj4F477yBVNYWHCujHAG7LnZDMohajS+ACIyeHeWSzxXX+cPYamRJQbesfYJ4W1itGSiuHN8/8QxptEfzxYtY1YyPRyRw5VuyevlBBX1YcrYxCDHQ/w5PA/i8mOA7zDNR7aDFPCzNIvpG122AYL2GwzZEUBzDST679mr6h+lrreyc6LWYWpBEYnrchci3kfYZ52l+tU2DrIpR/gfgUoRSOz8Hc7MYFt/50BjBLR8p3xT0c/1adlR/9LLJKJ7ZsmO3TGNa+vaLJvDo7dulqmBHPC/+rtvyb2KGGmXMtA6ak8Gon+iUyO+eDGT6Iv6gPo7QsZWbbpVvgzyV6A6sUbuJqmfL24K6csPI7JCw0fwxXB0t+JY+6YHGEqTEPXubPBn8t2GZLVtH/6MuztzwPml1N02lVHbvwXD/JLoN7PKVjKPHyZ88axnBpPeskpm9tFngVv7dHxFf0gRY366qss2gCVhHipRxtn+v3h9LJ7Hj7Zt5VDr8SpbRtwz8e+4L3lx/R9xInrzB4PDwPC+fZ7hTzxyKuQX4tELyukXiBPLasjknXobCeacD5gxJtwMcw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(83380400001)(86362001)(110136005)(54906003)(31696002)(6486002)(2906002)(38070700005)(122000001)(508600001)(36756003)(6512007)(31686004)(38100700002)(4326008)(71200400001)(26005)(186003)(66476007)(66556008)(8676002)(64756008)(66446008)(66946007)(53546011)(91956017)(76116006)(8936002)(6506007)(5660300002)(2616005)(21314003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?WTBIUnpNZ3hLNnZBdlBReWMyTENKTTJUeVFHT2J6UW5VWXhJSVVTbFdJeWw1?=
 =?utf-8?B?OXVaenpaSUQxMHorY2F1elJ5OUNicnJNTnArOVJ5RjZ5VVM0NElkY3dFNi9w?=
 =?utf-8?B?MVRFZHpLdDJRRUgrWkpLNkVuckVlOGdyZDZvRzROYXVYQnRwQzU5RmxWamFD?=
 =?utf-8?B?N3Ercko5ZW83YlhlUUVYWkVTSVByT0tkMGxhUncxWnlXTTNNZXVBL2w5THc0?=
 =?utf-8?B?ZUVoQVpRTk5oSkp4UmxzbUdwNjFsdFc3MFV2QSt4U0NuRzgvanRyMm1IU2xw?=
 =?utf-8?B?VUd2NXJCWHZmajZ5a0FXTUpjZkhYM0lURDA5T2lWWnBPNXNpTTNUWEVvRU1V?=
 =?utf-8?B?M3dDZGRMQll4UzhIbWRKWm05RzNibHMyZ3dBcUN4bnBNa0F0eG5RS2RLZGZ2?=
 =?utf-8?B?ZFFYR1B2VXB2RnI2L05OMnpsTWNzeU5nK0swVWdmWVZUckhMV0NOWlllaWcx?=
 =?utf-8?B?RlBxSm1nbGlSaDVNWUc1Y2hQTXl4QlUzZHY4YjFEWmtYem8rWHluQWhZUlNk?=
 =?utf-8?B?RHlicnpwd05Hbm5OTkNnc2F3L2xBTjRUaGRGL3NGclVnTWl1MWdLUWoxcXdo?=
 =?utf-8?B?QTVvQWRmQ2hqN2xPTC80L29vL0N1SkFOWURSZ1JGVUZNRmc1c09pdld1R1N6?=
 =?utf-8?B?MVp3STVhYlVWaVl1QnBkUWYxMkFjU2FtdG9SYXF3TzJKM25FM3ozdVd2MEVy?=
 =?utf-8?B?ZFpVSWpyYXhWRGdGVTh4VlpWb2FlbmcxMWhmTER2c3NxZWVhd3V4OUJXWlBy?=
 =?utf-8?B?aWxGbnJETWRJbENEU2hqRzcrdEZiRzNDaUk2KzBmRzN6YXB6a3lHemg5SThR?=
 =?utf-8?B?UW94T3d2T3RnaFlBM2Flc3Z6ZzF6emNkbVBwOGR6T09oUDlFYjJOL29UUG5x?=
 =?utf-8?B?YVZ4WUdEWkFPQkJ1bWVZR2M4QkRBaHVxRm9yRDBaK0YzWEZQU2Z0RzljMFM0?=
 =?utf-8?B?TFhRVG1JdmhMMVBkNjZxMzZTMGVlWnhwU2lsREhZUENtSnVVVzFrNEQ2MjV3?=
 =?utf-8?B?U3p4bTRQcTJGVGxERlRDaTgwMFVjeDYyVXlkNGhtYjRmZWEzVlAwWE1QSUtC?=
 =?utf-8?B?eVF5bGVPUDhYSGhyT0tHeExvT1pVVlBIekQzOFJUd2RpeVAralhyajZyRzE2?=
 =?utf-8?B?OS9GVHo0QmR2Qzl2ZGloRHZkU1NJSk5DZlpzUHBwc0VVZEJjZU0vVFlXME02?=
 =?utf-8?B?OU5HcTJnaXFPcWFQc0Y4ZlBzSGE4ZDhPZUhDS2x3TjdyV1RKR01sSkFCSktm?=
 =?utf-8?B?MGJUeGhkSHVPaUFkRWJ2OU5kVGtoMmErMGZnRlJReEJBREQxVGtoUVBjWWhL?=
 =?utf-8?B?VnJ1QWg1aGZUQ3lWNGw2OHJneDZjZWR0clROQlJET3RVUXRVZUp0YSttQWlS?=
 =?utf-8?B?djN5aUU2a2Q4cTlBanV4N3JmTElBNmdrRy9lYUdnQ1BJUmZnL3BRMUpORTYy?=
 =?utf-8?B?WGoxZjRrSzRxNlV6aWxIbDE4bHE1bkMrWm1lM2o2ZHdiWG02cjFjcXJTTm9N?=
 =?utf-8?B?bnkxaXNJVmFhZXhrM2k5bDNFQU10RFdFdE9MUFdrZHN3RFI2TXhCRDhheWNz?=
 =?utf-8?B?SjFDUCtSaHRCOGZTaUNMWjZ4MWNkYVovZG9zR25pSVpSMk80MCs4RFVTNkdK?=
 =?utf-8?B?alFMZ0VQVG9KSFlhZWM1M2dZUHZVMzd2am9udFpkSEo4T2xXeGp4bW9OeDBV?=
 =?utf-8?B?K0FZYnd0QTZWVXBHNGVIeCtaNW5iMVd3NE0zZkgwOWYxRnUzVXQzdmp0d3Rv?=
 =?utf-8?Q?x041ag8LH5w/2wCbTyxUgteZWcBAJIAhn7k1Tdr?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A080D9FE583C194995993088629FDEE9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: debb9fc1-4a64-4f7c-81d1-08d971fa5e86
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 12:23:58.9897
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: o8RguYEOB9GEeVx5ZJwBvfbeNpSNLEdNUTNTnDyH6im8ZH7S1eJZK5DICIeryHPDS1/sx+uFlDCAgB2u2M518SJ5SUhOW52W1BBTEOajasQYBmeWGCKox6BPdqKRLLlU
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB4949
X-Proofpoint-GUID: dIMHYvigZppWAJf-fJ20Y8gvWPBb3h9W
X-Proofpoint-ORIG-GUID: dIMHYvigZppWAJf-fJ20Y8gvWPBb3h9W
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_04,2021-09-07_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=999
 priorityscore=1501 suspectscore=0 bulkscore=0 mlxscore=0
 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 spamscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070081

DQpPbiAwNy4wOS4yMSAxNToyMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTQ6MTYsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTQ6
NDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTM6MTAsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxMzo0MywgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDEyOjExLCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNi4wOS4yMSAxNzoxMSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gQEAgLTU5Myw2ICs2MjUsMzYgQEAgc3RhdGljIGludCBpbml0X2JhcnMo
c3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+Pj4+Pj4+PiAgICAgIH0NCj4+Pj4+Pj4+ICAgICAgUkVH
SVNURVJfVlBDSV9JTklUKGluaXRfYmFycywgVlBDSV9QUklPUklUWV9NSURETEUpOw0KPj4+Pj4+
Pj4gICAgICANCj4+Pj4+Pj4+ICtpbnQgdnBjaV9iYXJfYWRkX2hhbmRsZXJzKGNvbnN0IHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4+Pj4+Pj4gK3sNCj4+Pj4+Pj4+
ICsgICAgaW50IHJjOw0KPj4+Pj4+Pj4gKw0KPj4+Pj4+Pj4gKyAgICAvKiBSZW1vdmUgcHJldmlv
dXNseSBhZGRlZCByZWdpc3RlcnMuICovDQo+Pj4+Pj4+PiArICAgIHZwY2lfcmVtb3ZlX2Rldmlj
ZV9yZWdpc3RlcnMocGRldik7DQo+Pj4+Pj4+PiArDQo+Pj4+Pj4+PiArICAgIC8qIEl0IG9ubHkg
bWFrZXMgc2Vuc2UgdG8gYWRkIHJlZ2lzdGVycyBmb3IgaHdkb20gb3IgZ3Vlc3QgZG9tYWluLiAq
Lw0KPj4+Pj4+Pj4gKyAgICBpZiAoIGQtPmRvbWFpbl9pZCA+PSBET01JRF9GSVJTVF9SRVNFUlZF
RCApDQo+Pj4+Pj4+PiArICAgICAgICByZXR1cm4gMDsNCj4+Pj4+Pj4+ICsNCj4+Pj4+Pj4+ICsg
ICAgaWYgKCBpc19oYXJkd2FyZV9kb21haW4oZCkgKQ0KPj4+Pj4+Pj4gKyAgICAgICAgcmMgPSBh
ZGRfYmFyX2hhbmRsZXJzKHBkZXYsIHRydWUpOw0KPj4+Pj4+Pj4gKyAgICBlbHNlDQo+Pj4+Pj4+
PiArICAgICAgICByYyA9IGFkZF9iYXJfaGFuZGxlcnMocGRldiwgZmFsc2UpOw0KPj4+Pj4+PiAg
ICAgICAgIHJjID0gYWRkX2Jhcl9oYW5kbGVycyhwZGV2LCBpc19oYXJkd2FyZV9kb21haW4oZCkp
Ow0KPj4+Pj4+IEluZGVlZCwgdGhhbmsgeW91IDspDQo+Pj4+Pj4+PiArICAgIGlmICggcmMgKQ0K
Pj4+Pj4+Pj4gKyAgICAgICAgZ3ByaW50ayhYRU5MT0dfRVJSLA0KPj4+Pj4+Pj4gKyAgICAgICAg
ICAgICIlcHA6IGZhaWxlZCB0byBhZGQgQkFSIGhhbmRsZXJzIGZvciBkb20lZFxuIiwgJnBkZXYt
PnNiZGYsDQo+Pj4+Pj4+PiArICAgICAgICAgICAgZC0+ZG9tYWluX2lkKTsNCj4+Pj4+Pj4gUGxl
YXNlIHVzZSAlcGQgYW5kIGNvcnJlY3QgaW5kZW50YXRpb24uIExvZ2dpbmcgdGhlIGVycm9yIGNv
ZGUgbWlnaHQNCj4+Pj4+Pj4gYWxzbyBoZWxwIHNvbWUgaW4gZGlhZ25vc2luZyBpc3N1ZXMuDQo+
Pj4+Pj4gU3VyZSwgSSdsbCBjaGFuZ2UgaXQgdG8gJXBkDQo+Pj4+Pj4+ICAgICAgRnVydGhlciBJ
J20gbm90IHN1cmUgdGhpcyBpcyBhDQo+Pj4+Pj4+IG1lc3NhZ2Ugd2Ugd2FudCBpbiByZWxlYXNl
IGJ1aWxkcw0KPj4+Pj4+IFdoeSBub3Q/DQo+Pj4+PiBFeGNlc3MgdmVyYm9zaXR5OiBJZiB3ZSBo
YXZlIHN1Y2ggaGVyZSwgd2h5IG5vdCBlbHNld2hlcmUgb24gZXJyb3IgcGF0aHM/DQo+Pj4+PiBB
bmQgSSBob3BlIHlvdSBhZ3JlZSB0aGluZ3Mgd2lsbCBnZXQgdG9vIHZlcmJvc2UgaWYgd2UgaGFk
IHN1Y2ggKGFib3V0KQ0KPj4+Pj4gZXZlcnl3aGVyZT8NCj4+Pj4gQWdyZWUsIHdpbGwgY2hhbmdl
IGl0IHRvIGdkcHJpbnRrDQo+Pj4+Pj4+ICAgICAgLSBwZXJoYXBzIGdkcHJpbnRrKCk/DQo+Pj4+
Pj4gSSdsbCBjaGFuZ2UgaWYgd2UgZGVjaWRlIHNvDQo+Pj4+Pj4+PiArICAgIHJldHVybiByYzsN
Cj4+Pj4+Pj4+ICt9DQo+Pj4+Pj4+PiArDQo+Pj4+Pj4+PiAraW50IHZwY2lfYmFyX3JlbW92ZV9o
YW5kbGVycyhjb25zdCBzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+
Pj4+Pj4+ICt7DQo+Pj4+Pj4+PiArICAgIC8qIFJlbW92ZSBwcmV2aW91c2x5IGFkZGVkIHJlZ2lz
dGVycy4gKi8NCj4+Pj4+Pj4+ICsgICAgdnBjaV9yZW1vdmVfZGV2aWNlX3JlZ2lzdGVycyhwZGV2
KTsNCj4+Pj4+Pj4+ICsgICAgcmV0dXJuIDA7DQo+Pj4+Pj4+PiArfQ0KPj4+Pj4+PiBBbHNvIC0g
aW4gaG93IGZhciBpcyB0aGUgZ29hbCBvZiB5b3VyIHdvcmsgdG8gYWxzbyBtYWtlIHZQQ0kgd29y
ayBmb3INCj4+Pj4+Pj4geDg2IERvbVUtcz8gSWYgdGhhdCdzIG5vdCBhIGdvYWwNCj4+Pj4+PiBJ
dCBpcyBub3QsIHVuZm9ydHVuYXRlbHkuIFRoZSBnb2FsIGlzIG5vdCB0byBicmVhayB4ODYgYW5k
IHRvIGVuYWJsZSBBcm0NCj4+Pj4+Pj4gLCBJJ2QgbGlrZSB0byBhc2sgdGhhdCB5b3UgbGltaXQg
dGhlDQo+Pj4+Pj4+IGludHJvZHVjdGlvbiBvZiBjb2RlIHRoYXQgZW5kcyB1cCBkZWFkIHRoZXJl
Lg0KPj4+Pj4+IFdoYXQncyB3cm9uZyB3aXRoIHRoaXMgZnVuY3Rpb24gZXZlbiBpZiBpdCBpcyBh
IG9uZS1saW5lcj8NCj4+Pj4+IFRoZSBjb21tZW50IGlzIHByaW1hcmlseSBvbiB0aGUgZWFybGll
ciBmdW5jdGlvbiwgYW5kIHRoZW4gZXh0ZW5kcyB0bw0KPj4+Pj4gdGhpcyBvbmUuDQo+Pj4+Pg0K
Pj4+Pj4+IFRoaXMgd2F5IHdlIGhhdmUgYSBwYWlyIHZwY2lfYmFyX2FkZF9oYW5kbGVycy92cGNp
X2Jhcl9yZW1vdmVfaGFuZGxlcnMNCj4+Pj4+PiBhbmQgaWYgSSB1bmRlcnN0b29kIGNvcnJlY3Rs
eSB5b3Ugc3VnZ2VzdCB2cGNpX2Jhcl9hZGRfaGFuZGxlcnMvdnBjaV9yZW1vdmVfZGV2aWNlX3Jl
Z2lzdGVycz8NCj4+Pj4+PiBXaGF0IHdvdWxkIHdlIGdhaW4gZnJvbSB0aGF0LCBidXQgeWV0IGFu
b3RoZXIgc2VjcmV0IGtub3dsZWRnZSB0aGF0IGluIG9yZGVyDQo+Pj4+Pj4gdG8gcmVtb3ZlIEJB
UiBoYW5kbGVycyBvbmUgbmVlZHMgdG8gY2FsbCB2cGNpX3JlbW92ZV9kZXZpY2VfcmVnaXN0ZXJz
DQo+Pj4+Pj4gd2hpbGUgSSB3b3VsZCBwZXJzb25hbGx5IGV4cGVjdCB0byBjYWxsIHZwY2lfYmFy
X2FkZF9oYW5kbGVycycgY291bnRlcnBhcnQsDQo+Pj4+Pj4gdnBjaV9yZW1vdmVfZGV2aWNlX3Jl
Z2lzdGVycyBuYW1lbHkuDQo+Pj4+PiBUaGlzIGlzIGFsbCBmaW5lLiBZZXQgdnBjaV9iYXJfe2Fk
ZCxyZW1vdmV9X2hhbmRsZXJzKCkgd2lsbCwgYWl1aSwgYmUNCj4+Pj4+IGRlYWQgY29kZSBvbiB4
ODYuDQo+Pj4+IHZwY2lfYmFyX2FkZF9oYW5kbGVycyB3aWxsIGJlIHVzZWQgYnkgeDg2IFBWSCBE
b20wDQo+Pj4gV2hlcmUgLyB3aGVuPyBZb3UgYWRkIGEgY2FsbCBmcm9tIHZwY2lfYXNzaWduX2Rl
dmljZSgpLCBidXQgYmVzaWRlcyB0aGF0DQo+Pj4gYWxzbyBiZWluZyBkZWFkIGNvZGUgb24geDg2
IChmb3Igbm93KSwgeW91IGNhbid0IG1lYW4gdGhhdCBiZWNhdXNlDQo+Pj4gdnBjaV9kZWFzc2ln
bl9kZXZpY2UoKSBhbHNvIGNhbGxzIHZwY2lfYmFyX3JlbW92ZV9oYW5kbGVycygpLg0KPj4gWW91
IGFyZSByaWdodCBoZXJlIGFuZCBib3RoIGFkZC9yZW1vdmUgYXJlIG5vdCB1c2VkIG9uIHg4NiBQ
VkggRG9tMC4NCj4+DQo+PiBJIGFtIHNvcnJ5IGZvciB3YXN0aW5nIHlvdXIgdGltZQ0KPj4NCj4+
Pj4+ICAgICBIZW5jZSB0aGVyZSBzaG91bGQgYmUgYW4gYXJyYW5nZW1lbnQgYWxsb3dpbmcgdGhl
DQo+Pj4+PiBjb21waWxlciB0byBlbGltaW5hdGUgdGhpcyBkZWFkIGNvZGUuDQo+Pj4+IFNvLCB0
aGUgb25seSBkZWFkIGNvZGUgZm9yIHg4NiBoZXJlIHdpbGwgYmUgdnBjaV9iYXJfcmVtb3ZlX2hh
bmRsZXJzLiBZZXQuDQo+Pj4+IEJlY2F1c2UgSSBob3BlIHg4NiB0byBnYWluIGd1ZXN0IHN1cHBv
cnQgZm9yIFBWSCBEb20wIHNvb25lciBvciBsYXRlci4NCj4+Pj4NCj4+Pj4+ICAgICBXaGV0aGVy
IHRoYXQncyBlbmNsb3NpbmcgdGhlc2UNCj4+Pj4+IGJ5ICIjaWZkZWYiIG9yIGFkZGluZyBlYXJs
eSAiaWYoIUlTX0VOQUJMRUQoQ09ORklHXyopKSIgaXMgc2Vjb25kYXJ5Lg0KPj4+Pj4gVGhpcyBo
YXMgYSBrbm9jay1vbiBlZmZlY3Qgb24gb3RoZXIgZnVuY3Rpb25zIGFzIHlvdSBjZXJ0YWlubHkg
cmVhbGl6ZToNCj4+Pj4+IFRoZSBjb21waWxlciBzZWVpbmcgZS5nLiB0aGUgMm5kIGFyZ3VtZW50
IHRvIHRoZSBhZGQtQkFScyBmdW5jdGlvbg0KPj4+Pj4gYWx3YXlzIGJlaW5nIHRydWUgYWxsb3dz
IGl0IHRvIGluc3RhbnRpYXRlIGp1c3QgYSBjbG9uZSBvZiB0aGUgb3JpZ2luYWwNCj4+Pj4+IGZ1
bmN0aW9uIHdpdGggdGhlIHJlc3BlY3RpdmUgY29uZGl0aW9uYWxzIHJlbW92ZWQuDQo+Pj4+IFdp
dGggdGhlIGFib3ZlIChlLmcuIGFkZCBpcyBnb2luZyB0byBiZSB1c2VkLCBidXQgbm90IHJlbW92
ZSkgZG8geW91DQo+Pj4+IHRoaW5rIGl0IGlzIHdvcnRoIHBsYXlpbmcgd2l0aCBpZmRlZidzIHRv
IHN0cmlwIHRoYXQgc2luZ2xlIGZ1bmN0aW9uIGFuZCBhZGQNCj4+Pj4gYSBwaWVjZSBvZiBzcGFn
aGV0dGkgY29kZSB0byBzYXZlIGEgYml0Pw0KPj4+IE5vLCB0aGF0IEkgYWdyZWUgd291bGRuJ3Qg
YmUgd29ydGggaXQuDQo+Pj4NCj4+Pj4gV2hhdCB3b3VsZCB0aGF0IGlmZGVmIGxvb2sgbGlrZSwN
Cj4+Pj4gZS5nLiAjaWZkZWYgQ09ORklHX0FSTSBvciAjaWZuZGVmIENPTkZJR19YODYgJiYgYW55
IG90aGVyIHBsYXRmb3JtLCBidXQgQVJNPw0KPj4+IEEgbmV3IHNldHRpbmcsIHByZWZlcmFibHk7
IGUuZy4gVkNQVV9VTlBSSVZJTEVHRUQsIHRvIGJlICJzZWxlY3QiZWQgYnkNCj4+PiBhcmNoaXRl
Y3R1cmVzIGFzIGZ1bmN0aW9uYWxpdHkgZ2V0cyBlbmFibGVkLg0KPj4gU28sIGFzIGFkZC9yZW1v
dmUgYXJlIG9ubHkgbmVlZGVkIGZvciBBcm0gYXQgdGhlIG1vbWVudA0KPj4geW91IHN1Z2dlc3Qg
SSBhZGQgVkNQVV9VTlBSSVZJTEVHRUQgdG8gQXJtJ3MgS2NvbmZpZyB0byBlbmFibGUNCj4+IGNv
bXBpbGluZyB2cGNpX2Jhcl9hZGRfaGFuZGxlcnMvdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzPw0K
Pj4gVG8gbWUgdGhpcyBpcyBtb3JlIGFib3V0IHZQQ0kncyBzdXBwb3J0IGZvciBndWVzdHMsIHNv
IHNob3VsZCB3ZSBwcm9iYWJseSBjYWxsIGl0DQo+PiBWUENJX1hYWCBpbnN0ZWFkPyBFLmcuIFZQ
Q0lfSEFTX0dVRVNUX1NVUFBPUlQgb3Igc29tZXRoaW5nIHdoaWNoDQo+PiB3aWxsIHJlZmxlY3Qg
dGhlIG5hdHVyZSBvZiB0aGUgY29kZSBiZWluZyBnYXRlZD8gVkNQVV9VTlBSSVZJTEVHRUQgc291
bmRzDQo+PiBsaWtlIG5vdCBjb25uZWN0ZWQgdG8gdnBjaSB0byBtZQ0KPiBBbmQgdmFsaWRseSBz
byAtIG15IGZpbmdlcnMgZGlkbid0IHR5cGUgd2hhdCB0aGUgYnJhaW4gdG9sZCB0aGVtLiBJJ3Zl
DQo+IG1lYW50IFZQQ0lfVU5QUklWSUxFR0VELiBJIHdvdWxkIGFsc28gYmUgb2theSB3aXRoIEhB
U19WUENJX0dVRVNUX1NVUFBPUlQNCj4gKGkuZS4gbm90IGV4YWN0bHkgd2hhdCB5b3UndmUgc3Vn
Z2VzdGVkKSwgZm9yIGV4YW1wbGUuDQpJJ2xsIHN0aWNrIHRvIEhBU19WUENJX0dVRVNUX1NVUFBP
UlQgYXMgaXQgc2VlbXMgdG8gYmUgbW9yZSBkZXNjcmlwdGl2ZQ0KPiBKYW4NCj4NClRoYW5rIHlv
dSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 12:31:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 12:31:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181010.327954 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNaFa-0000jx-MA; Tue, 07 Sep 2021 12:30:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181010.327954; Tue, 07 Sep 2021 12:30:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNaFa-0000jq-Iy; Tue, 07 Sep 2021 12:30:54 +0000
Received: by outflank-mailman (input) for mailman id 181010;
 Tue, 07 Sep 2021 12:30:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNaFZ-0000jk-CN
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 12:30:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNaFV-0003uY-RY; Tue, 07 Sep 2021 12:30:49 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNaFV-0003Kr-Kj; Tue, 07 Sep 2021 12:30:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=3WWgZmgWXQnQKchVGG3MzV2yIqBwyB+TS5pKQRzF5DY=; b=T8nMCBhO8U7R7fpQOm/3AmZiNy
	BSjLYkXc2v9mwDZdUqnP3mblMQESiVKe56E4oIqtn/Gqv/ITJYxV7IrWT3G/ypHI35g6Ew0FX8dBo
	ZTMIxkD3PSnNL2ZoUxaKUg3saHP1Noo+tV4TReHtvezUHSu+vX5X4YCDhL5T4eJQ0ydE=;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
Date: Tue, 7 Sep 2021 13:30:47 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 07/09/2021 12:51, Luca Fancellu wrote:
> 
> 
>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Luca,
>>
>> On 07/09/2021 07:52, Luca Fancellu wrote:
>>> Add a design describing a proposal to improve the EFI
>>> configuration file, adding keywords to describe domU
>>> guests and allowing to start a dom0less system.
>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>> ---
>>>   docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>>>   1 file changed, 105 insertions(+)
>>>   create mode 100644 docs/designs/efi-arm-dom0less.md
>>> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
>>> new file mode 100644
>>> index 0000000000..8d8fa2243f
>>> --- /dev/null
>>> +++ b/docs/designs/efi-arm-dom0less.md
>>> @@ -0,0 +1,105 @@
>>> +# Xen EFI configuration file
>>> +
>>> +The current configuration file used by Xen when it is started as an EFI
>>> +application is considering only the dom0 guest and doesn't have any
>>> +property to describe and load in memory domU guests.
>>
>>  From my understanding, the problem is less about properties (we already have them in the Device-Tree) but more about where are the binaries located in memory as we don't know in advance.
> 
> Hi Julien,
Hi Luca,

> I think I used the wrong word there, I meant “keyword” instead of “property” because I was referring about the
> lack of keywords to describe a domu guest in the Xen EFI configuration file.
> 
> I agree with you that on systems with static allocation, the kernel and ramdisk binaries must be at certain locations
> that are out of control when we use the EFI boot services, the thing we can do is provide a keyword to specify the
> addresses and then use the CopyMem() function to relocate the kernel/ramdisk in the address we want.

I wasn't specifically referring to static allocation here, sorry if this 
wasn't clear. I was pointing out that most of the information you create 
in the xen.cfg is going to be similar to what we already provide in the 
Device-Tree.

My main concern is everytime we add a new feature in Dom0less, a 
developer would need to write code for the DT and UEFI. This will 
increase the code size and maintenance.

The same can be said for the admin as if they want to switch from plain 
U-boot to UEFI, they would also need to fully rewrite the bindings.

AFAICT, most of the information provided in the Device-Tree are usable 
even when using UEFI. So I would prefer if we try to re-use what's 
existing. This is what my proposal below was about.

> 
>>
>> So I would like to propose something that build on top of the Device-Tree work we did. Note this is early thoughts.
>>
>> The problematic nodes in the DT are:
>>
>>         module@0x4a000000 {
>>             compatible = "multiboot,kernel", "multiboot,module";
>>             reg = <0x0 0x4a000000 0xffffff>;
>>             bootargs = "console=ttyAMA0 init=/bin/sh";
>>         };
>>
>>         module@0x4b000000 {
>>             compatible = "multiboot,ramdisk", "multiboot,module";
>>             reg = <0x0 0x4b000000 0xffffff>;
>>         };
>>
>> In particular the property "reg" cannot be known in advance because the UEFI stub will be responsible to load the binaries in memory.
> 
> Yes that’s true, the UEFI stub is using from the UEFI boot service the AllocatePages function that is giving back an address out of our control,
> then using another function the binary is read from the disk and copied at that address, finally the UEFI stub is writing the node in the device tree that
> will be used by Xen later.

I am not sure to follow. Are you saying the UEFI stub will create the 
dom0less node in the DT based on the xen.cfg?

> 
>>
>> What we could do is providing a list of binaries to load and associate a key for each of them. Something like:
>>
>> binary=<binary> <key>
>> binary=<binary2> <key2>
>> ....
>>
>> We can then replace the property "reg" with a new property "uefi,key" that will contain the name of the binary.
>>
>> What do you think?
> 
> Here I’m lost, because I don’t understand what we are going to do with the name of the binary.

<binaryX> would be used by the UEFI stub to load the binary in memory. 
Each binary will have a <keyX> which helps to refer them in the 
Device-Tree. To give a concrete example, let say we have two dom0less 
domains:
   - DomA: 2 vCPUs, 128MB
   - DomB: 3 vCPUs, 512MB

DomA and DomB will be using the same kernel but a different ramdisk. 
xen.cfg, would look like:

[global]
default=section1

[section1]
options=console=vga,com1 com1=57600 loglvl=all noreboot
kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
ramdisk=initrd-3.0.31-0.4-xen
xsm=<filename>
dtb=devtree.dtb
binary=vmlinuz-guest domu-kernel
binary=ramdisk-domA.img domA-ramdisk
binary=ramdisk-domB.img domB-ramdisk

The chosen node in the DT would look like:

chosen {
     domU1 {
         compatible = "xen,domain";
         #address-cells = <0x2>;
         #size-cells = <0x1>;
         memory = <0 0x8000000>;
         cpus = <2>;

         module@1 {
             compatible = "multiboot,kernel", "multiboot,module";
             uefi,binary = "domu-kernel";
             bootargs = "console=ttyAMA0 init=/bin/sh";
         };

         module@2 {
             compatible = "multiboot,ramdisk", "multiboot,module";
             uefi,binary = "domA-ramdisk";
         };
     };

     domU2 {
         compatible = "xen,domain";
         #address-cells = <0x3>;
         #size-cells = <0x1>;
         memory = <0 0x20000000>;
         cpus = <3>;

         module@1 {
             compatible = "multiboot,kernel", "multiboot,module";
             uefi,binary = "domu-kernel";
             bootargs = "console=ttyAMA0 init=/bin/sh";
         };

         module@2 {
             compatible = "multiboot,ramdisk", "multiboot,module";
             uefi,binary = "domA-ramdisk";
         };
     };
};

With this approach, the change is quite minimal to move between an 
classic U-boot boot and EFI boot.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:04:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:04:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181020.327968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNaln-0005HL-CW; Tue, 07 Sep 2021 13:04:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181020.327968; Tue, 07 Sep 2021 13:04:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNaln-0005HE-9a; Tue, 07 Sep 2021 13:04:11 +0000
Received: by outflank-mailman (input) for mailman id 181020;
 Tue, 07 Sep 2021 13:04:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GNGS=N5=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNall-0005H8-9C
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:04:09 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 14d68826-0fdc-11ec-b0ef-12813bfff9fa;
 Tue, 07 Sep 2021 13:04:06 +0000 (UTC)
Received: from DB6PR07CA0194.eurprd07.prod.outlook.com (2603:10a6:6:42::24) by
 AM9PR08MB6657.eurprd08.prod.outlook.com (2603:10a6:20b:308::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 13:04:01 +0000
Received: from DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:42:cafe::fa) by DB6PR07CA0194.outlook.office365.com
 (2603:10a6:6:42::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6 via Frontend
 Transport; Tue, 7 Sep 2021 13:04:01 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT046.mail.protection.outlook.com (10.152.21.230) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 13:04:01 +0000
Received: ("Tessian outbound cc1d52552731:v105");
 Tue, 07 Sep 2021 13:04:01 +0000
Received: from 23a0c5a7b7f6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C166E117-6975-49F0-AFC1-CC384A431CF3.1; 
 Tue, 07 Sep 2021 13:03:55 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 23a0c5a7b7f6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 13:03:55 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR0802MB2495.eurprd08.prod.outlook.com (2603:10a6:800:b9::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 13:03:44 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Tue, 7 Sep 2021
 13:03:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14d68826-0fdc-11ec-b0ef-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lt7nz3+gkmBFfx53RUPqJl18QGI12yP2CEb6pMBB8eY=;
 b=mMzhIMJfnGkg5sJZhXbqBqlZneJ8At/+5xIXkZJtpckK41qMqITXdV5B/rGlhUy8Lq8X8UobkUlXXkmANqYSO00xMxJsVjxQltpC+40u5pmAb4FJ8Ao6nbqm1bL3Aec4yIihYX0TfkK3E7N/by2NEt5fVyBmdj5iUQzP/6/YZ/E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=D72jRvCceZEB+y0SwSXVvcVohsoIZQAH6kJ5B05hVF9yiveiknPNMR2w0W5EqqwY09W0JbJIuW/RkLO/YhSjuWjVe8PBbSwOMMlyGgwTiEZcvLPJ3IMdEEi4YPRqlFZ6RUCySye+hNFOcfsASEGnAePOnUAt4T9DFf48zoZZ96Hqe3eADa4IbrDRqGLCxIAte8KeJLNiyv6tSu+nov9KH8SwuE/EAM4UJOgwYmFHxDYapbKH5zScw5y/vtEzSr7hSfjwqkiqmr9bxAn/g2e74ito3ZHgkHS1ynBDYTvnwJ9Lh57X1jkawFnCjjg7fiV4+60+WdQb2uLHY8m44cI2lw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lt7nz3+gkmBFfx53RUPqJl18QGI12yP2CEb6pMBB8eY=;
 b=TT+2GGFC9ifj9zzHRMx1B0cnEf1HQvrWg7svxwetC2h3nUQLl4C2rdgorlqfTIDmNi1k8o63/A+XwAg9cf+JQwCwvcu4YW5umCsQiLxJ6qV8sM6vqPvAkzL2mGq4KsCwi8JQFBNayGYPipwMHvKDdS9qemed5RmkUH7q0W7sv2wIJV1ZBUkx4jlIcq1A4+Y8bmJLWDaDlR1a95ImVcWRxDMjbeUdJUGvI1sNoUJfzADq6w5HIrLwVrDrSLlwhbODmspitB8w0t7/jY3CUPshs1ACgBApGo9QIWYYjr0gtQDv/aPBIMNxTMrumv5k5WL+n6lJdhOgdBMPGSOW1BiGEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lt7nz3+gkmBFfx53RUPqJl18QGI12yP2CEb6pMBB8eY=;
 b=mMzhIMJfnGkg5sJZhXbqBqlZneJ8At/+5xIXkZJtpckK41qMqITXdV5B/rGlhUy8Lq8X8UobkUlXXkmANqYSO00xMxJsVjxQltpC+40u5pmAb4FJ8Ao6nbqm1bL3Aec4yIihYX0TfkK3E7N/by2NEt5fVyBmdj5iUQzP/6/YZ/E=
From: Penny Zheng <Penny.Zheng@arm.com>
To: "julien@xen.org" <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Error around cc-option on AArch64
Thread-Topic: Error around cc-option on AArch64
Thread-Index: Adej51APmfJVGMySS/6ID827i8copQ==
Date: Tue, 7 Sep 2021 13:03:43 +0000
Message-ID:
 <VE1PR08MB5215CBA5B28A1FB90D7B161FF7D39@VE1PR08MB5215.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 97094123F4E8D840B972E8AF26637690.0
x-checkrecipientchecked: true
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 8611c997-becc-42b9-2221-08d971fff64f
x-ms-traffictypediagnostic: VI1PR0802MB2495:|AM9PR08MB6657:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB6657B4AD2E5495BADE621712F7D39@AM9PR08MB6657.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:143;OLM:143;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 mpXF8MSfoGP6DHGiNesV8ORQkS1Pa/sbHQ/G4cYO9YI7wfqBraxdmEkM8FFh5rL5gppWKoqy3bz9xpXMkI/YHvya1fX+INI56QAKBYFn0CKDY88202GL8D7kCGIs8Gegje7ua/wTKkFAxVMsiCnDdL+ORlKBpBJ7hIRHxqeTddElNoP+Ob3Ya0BBgPXufEIuLdDEDqITDytsT0FSsfro/gkN5/Vv0hMNT62l6w0DWp/sRGnBjm06MgAl8Rb2wDMezREFPx9LBDsPyW1u0kq3uKgRDxvZbtS9mZSsFME+a7YPgWRzTk5/sdwA9RHMQhdLginqkXlGXrCJbCFK6c5pWQlzCz56sPT3aSEsBTqYzZCqY0vbD6ctsbVY1xP3qrvhaCoDy27SQ+p9SZ6QJ+O8JSr3vYq8QOgUwr7XHA0fKoq73jXqFkdfVxAN5SBNN0y+V6gC3SJmsFYTLfT9jjkP9LXAAw/3Lcl3Ygt066HFONk/Q4t0cnisFOsgT6XkYBHgvkn+MGqQW/D8wbVXb9l/Tv5QZorqI4uA9CIwuHTOjaMJmaVTcp9AMYlMYnlsKaTKQbrS391T6g0phqCwwC4TeLQeyKqHAWluLBX5kgv0rOjUj15o24nZVSLuqYwQ4QCFb1ETDBoSzfrLj8+nR0VvsfdKgJGuwBtJzjtcs9DPe+zxhoP2PEKmmZyYFyXbX6ctq9yNbPWn5vgUb8sAacG0vRfg33gRnLVO4GVQsTXtkGdm7ifsZdfPU+40ItCd/uVYQSuE7vOGwKEwbUy/Aa3LisQ6NGFrROVlyYSpJxV37mg=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(136003)(39860400002)(346002)(366004)(8936002)(33656002)(71200400001)(5660300002)(966005)(52536014)(26005)(7696005)(83380400001)(186003)(166002)(76116006)(316002)(66446008)(9686003)(2906002)(86362001)(66946007)(6506007)(66556008)(8676002)(478600001)(38100700002)(4326008)(122000001)(54906003)(66476007)(110136005)(55016002)(64756008)(38070700005);DIR:OUT;SFP:1101;
Content-Type: multipart/alternative;
	boundary="_000_VE1PR08MB5215CBA5B28A1FB90D7B161FF7D39VE1PR08MB5215eurp_"
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2495
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9353f8bd-7fb8-4b60-9300-08d971ffec03
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eVFPnxr+toddTC4McmtC1MLNYEeRIFLYCOuziz9P+zcUTQbcHTfUKtmTco20LMnIyWR1GUX44RpSYrDAhnNnpWf5/4wBlNz0zJ6S8bwpaymKIZ2Or5TsIRpURij31pwtndTeIil4bug6wFkYT73qUqWxbkuCJICUIuJnU4lcRwqziWDc9aHLW/waiIwyAj4Uo4w3wiR67MwXuQtlBpamyZZERFKlDScEF/bG1kzaLgbCE/yYuZqtliMuVNrTeoFsLaDLTlpU/1hRLsQv19c+B7xuM+ZpunveNQ+rZ5oef5bkQXS9DiEeURYJD/vIPC7k2pz/UmpcpkPgpMcX3WJt1BDq8kRH5tWngPkpCS1yTWjfYbaxB5RW1n3owd0BrPPB3QWVj4l9v58dVRqh2yS9Ooqi8j7KqdPlImBkjXZQZmgyzHSbS8fxvkGLaFcXy0mbKFij0ZfbTlwD8dY208KLvqYVJ0bItU919ZkKyvT5pwm+7sVIKoBkC74jeZI6VjrDrBi8cjFiaOV3HIvHz0H3CN4l0s+0rqWUANtp4+CprvNU6A7+HtMmd0G1KFZEZsGunf4ARhgSbviHd3whvVSX7IU9TZI7n1b65ZMb8KSNrCRil5R16DLIq1AACTLb1/WMAd6JixZmC6sTxTq6fxT/0v8l8dusoIkz5IruCO1DbYQByc8u8GNXuB7XlNcq+sYummOQ7ARNpsTROh4j564uy5fS7gDVovPnjFOU2qJ4NJKP5lh2DfMxymlu3ePoQsL1YgE7z2jhbxZEWDzU59v4U0VXI+J0I2Eddoazay1ygRuQ47nBCko9a/cq2itiR1z2
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(396003)(346002)(376002)(136003)(36840700001)(46966006)(336012)(52536014)(36860700001)(5660300002)(54906003)(4326008)(966005)(9686003)(7696005)(82310400003)(55016002)(2906002)(8936002)(110136005)(70586007)(26005)(82740400003)(166002)(47076005)(356005)(70206006)(6506007)(8676002)(478600001)(186003)(86362001)(83380400001)(33656002)(316002)(81166007)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 13:04:01.2536
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8611c997-becc-42b9-2221-08d971fff64f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6657

--_000_VE1PR08MB5215CBA5B28A1FB90D7B161FF7D39VE1PR08MB5215eurp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Julien and Stefano

I found that the "cc-option/cc-option-add" is not working for "-march=3Dxxx=
" option on a few very common aarch64 compilers.
Here is what I got when trying to compile XEN on r82 platform.
```
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
index 11caec86ba..d2d71baef4 100644
--- a/xen/arch/arm/arch.mk
+++ b/xen/arch/arm/arch.mk
@@ -10,6 +10,10 @@ $(call cc-option-add,CFLAGS,CC,-Wnested-externs)
CFLAGS-$(CONFIG_ARM_32) +=3D -msoft-float
CFLAGS-$(CONFIG_ARM_32) +=3D -mcpu=3Dcortex-a15

+#ifeq ($(CONFIG_ARM64_V8R),y)
+    CFLAGS-$(CONFIG_ARM_64) +=3D $(call cc-option,$(CC),-march=3Darmv8-r,-=
march=3Darmv8.4-a)
+#endif
+
CFLAGS-$(CONFIG_ARM_64) +=3D -mcpu=3Dgeneric
CFLAGS-$(CONFIG_ARM_64) +=3D -mgeneral-regs-only # No fp registers etc
$(call cc-option-add,CFLAGS-$(CONFIG_ARM_64),CC,-mno-outline-atomics)
```
My gcc compiler version is as follows:
```
Using built-in specs.
COLLECT_GCC=3Daarch64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=3D/usr/lib/gcc-cross/aarch64-linux-gnu/9/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion=3D'Ubuntu 9.3.0-17ub=
untu1~20.04'
--with-bugurl=3Dfile:///usr/share/doc/gcc-9/README.Bugs --enable-languages=
=3Dc,ada,c++,
go,d,fortran,objc,obj-c++,gm2 --prefix=3D/usr --with-gcc-major-version-only=
 --program-suffix=3D-9
--enable-shared --enable-linker-build-id --libexecdir=3D/usr/lib --without-=
included-gettext
--enable-threads=3Dposix --libdir=3D/usr/lib --enable-nls --with-sysroot=3D=
/ --enable-clocale=3Dgnu
--enable-libstdcxx-debug --enable-libstdcxx-time=3Dyes --with-default-libst=
dcxx-abi=3Dnew
--enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-supp=
ort --enable-plugin
--enable-default-pie --with-system-zlib --without-target-system-zlib --enab=
le-libpth-m2 --enable-multiarch
--enable-fix-cortex-a53-843419 --disable-werror --enable-checking=3Drelease=
 --build=3Dx86_64-linux-gnu
--host=3Dx86_64-linux-gnu --target=3Daarch64-linux-gnu --program-prefix=3Da=
arch64-linux-gnu- --includedir=3D/usr/aarch64-linux-gnu/include
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
```
However. even if my gcc compiler is indeed not supporting "-march=3Darmv8-r=
", above check will not identify it
"unrecognized option" and seek to the "-march=3Darmv8.4-a" as expected.
See the resulting errors, it still uses "-march=3Darmv8-r":
```
make[3]: Entering directory '/home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6=
347_gcc6.4/xen/xen'
aarch64-linux-gnu-gcc -MMD -MP -MF ./.asm-offsets.s.d -DBUILD_ID -fno-stric=
t-aliasing -std=3Dgnu99 -Wall
-Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-varia=
ble -Wno-unused-local-typedefs
-O1 -fno-omit-frame-pointer -nostdinc -fno-builtin -fno-common -Werror -Wre=
dundant-decls -Wno-pointer-arith -Wvla
-pipe -D__XEN__ -include /home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_=
gcc6.4/xen/xen/include/xen/config.h
-g -march=3Darmv8-r -mgeneral-regs-only -mno-outline-atomics -I/home/penny/=
FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_gcc6.4
/xen/xen/include -fno-stack-protector -fno-exceptions -fno-asynchronous-unw=
ind-tables -fcf-protection=3Dnone
-Wnested-externs '-D__OBJECT_FILE__=3D"asm-offsets.s"' -S -g0 -o asm-offset=
s.s.new -MQ asm-offsets.s arch/arm/arm64/asm-offsets.c
cc1: error: unknown value 'armv8-r' for '-march'
cc1: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8=
.4-a armv8.5-a; did you mean 'armv8-a'?
make[3]: *** [Makefile:413: asm-offsets.s] Error 1
```
And the reason is that in the implementation of "cc-option": To handle this=
 we do a test compile, passing the option-under-test,
on a code fragment that will always produce a warning (integer assigned to =
pointer). We then grep for the option-under-test in
the compiler's output, the presence of which would indicate an "unrecognize=
d command-line option" warning/error.
```
cc-option =3D $(shell if test -z "`echo 'void*p=3D1;' | \
                     $(1) $(2) -S -o /dev/null -x c - 2>&1 | grep -- $(2) -=
`"; \
                     then echo "$(2)"; else echo "$(3)"; fi ;)
```
Here, that is, we are trying to grep "-march=3Darmv8-r" in output message o=
f the test compile, and since the output
message(unknown value 'armv8-r' for '-march') doesn't contain the COMPLETE =
string, it thought the compiler supported this
option, but it actually not...

Tracing back to the commit about the implementation of "cc-option", it disc=
ards the linux method(using exit code to tell) for dealing with
disable-warning options.
See https://github.com/xen-project/xen/commit/28f8fb7d2b3fde2f5cbe5526ac4f1=
c932e3f5d26 for more details.

To fix this issue, I could loose the "grep check", instead trying to grep t=
he COMPLETE option string("-march=3Darmv8-r"), only partial(armv8-r) is
enough. But, hmmm, the way of greping output message to check is kinds of r=
eliable, IMO, since there is no standard for that.
What do you suggest?

Cheers

--
Penny Zheng


--_000_VE1PR08MB5215CBA5B28A1FB90D7B161FF7D39VE1PR08MB5215eurp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:DengXian;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@DengXian";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:#0563C1;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi Julien and Stefano<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">I found that the &#8220;cc-option/cc-option-add&#822=
1; is not working for &#8220;-march=3Dxxx&#8221; option on a few very commo=
n aarch64 compilers.<o:p></o:p></p>
<p class=3D"MsoNormal">Here is what I got when trying to compile XEN on r82=
 platform.<o:p></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arc=
h.mk<o:p></o:p></p>
<p class=3D"MsoNormal">index 11caec86ba..d2d71baef4 100644<o:p></o:p></p>
<p class=3D"MsoNormal">--- a/xen/arch/arm/arch.mk<o:p></o:p></p>
<p class=3D"MsoNormal">+++ b/xen/arch/arm/arch.mk<o:p></o:p></p>
<p class=3D"MsoNormal">@@ -10,6 +10,10 @@ $(call cc-option-add,CFLAGS,CC,-W=
nested-externs)<o:p></o:p></p>
<p class=3D"MsoNormal">CFLAGS-$(CONFIG_ARM_32) +=3D -msoft-float<o:p></o:p>=
</p>
<p class=3D"MsoNormal">CFLAGS-$(CONFIG_ARM_32) +=3D -mcpu=3Dcortex-a15<o:p>=
</o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">+#ifeq ($(CONFIG_ARM64_V8R),y)<o:p></o:p></p>
<p class=3D"MsoNormal">+&nbsp;&nbsp;&nbsp; CFLAGS-$(CONFIG_ARM_64) +=3D $(c=
all cc-option,$(CC),-march=3Darmv8-r,-march=3Darmv8.4-a)<o:p></o:p></p>
<p class=3D"MsoNormal">+#endif<o:p></o:p></p>
<p class=3D"MsoNormal">+<o:p></o:p></p>
<p class=3D"MsoNormal">CFLAGS-$(CONFIG_ARM_64) +=3D -mcpu=3Dgeneric<o:p></o=
:p></p>
<p class=3D"MsoNormal">CFLAGS-$(CONFIG_ARM_64) +=3D -mgeneral-regs-only # N=
o fp registers etc<o:p></o:p></p>
<p class=3D"MsoNormal">$(call cc-option-add,CFLAGS-$(CONFIG_ARM_64),CC,-mno=
-outline-atomics)<o:p></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">My gcc compiler version is as follows:<o:p></o:p></p=
>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">Using built-in specs.<o:p></o:p></p>
<p class=3D"MsoNormal">COLLECT_GCC=3Daarch64-linux-gnu-gcc<o:p></o:p></p>
<p class=3D"MsoNormal">COLLECT_LTO_WRAPPER=3D/usr/lib/gcc-cross/aarch64-lin=
ux-gnu/9/lto-wrapper<o:p></o:p></p>
<p class=3D"MsoNormal">Target: aarch64-linux-gnu<o:p></o:p></p>
<p class=3D"MsoNormal">Configured with: ../src/configure -v --with-pkgversi=
on=3D'Ubuntu 9.3.0-17ubuntu1~20.04'<o:p></o:p></p>
<p class=3D"MsoNormal">--with-bugurl=3Dfile:///usr/share/doc/gcc-9/README.B=
ugs --enable-languages=3Dc,ada,c++,<o:p></o:p></p>
<p class=3D"MsoNormal">go,d,fortran,objc,obj-c++,gm2 --prefix=3D/usr --with=
-gcc-major-version-only --program-suffix=3D-9<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-shared --enable-linker-build-id --libexecdi=
r=3D/usr/lib --without-included-gettext<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-threads=3Dposix --libdir=3D/usr/lib --enabl=
e-nls --with-sysroot=3D/ --enable-clocale=3Dgnu<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-libstdcxx-debug --enable-libstdcxx-time=3Dy=
es --with-default-libstdcxx-abi=3Dnew<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-gnu-unique-object --disable-libquadmath --d=
isable-libquadmath-support --enable-plugin<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-default-pie --with-system-zlib --without-ta=
rget-system-zlib --enable-libpth-m2 --enable-multiarch<o:p></o:p></p>
<p class=3D"MsoNormal">--enable-fix-cortex-a53-843419 --disable-werror --en=
able-checking=3Drelease --build=3Dx86_64-linux-gnu<o:p></o:p></p>
<p class=3D"MsoNormal">--host=3Dx86_64-linux-gnu --target=3Daarch64-linux-g=
nu --program-prefix=3Daarch64-linux-gnu- --includedir=3D/usr/aarch64-linux-=
gnu/include<o:p></o:p></p>
<p class=3D"MsoNormal">Thread model: posix<o:p></o:p></p>
<p class=3D"MsoNormal">gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)<o:p=
></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">However. even if my gcc compiler is indeed not suppo=
rting &#8220;-march=3Darmv8-r&#8221;, above check will not identify it<o:p>=
</o:p></p>
<p class=3D"MsoNormal">&#8220;unrecognized option&#8221; and seek to the &#=
8220;-march=3Darmv8.4-a&#8221; as expected.
<o:p></o:p></p>
<p class=3D"MsoNormal">See the resulting errors, it still uses &#8220;-marc=
h=3Darmv8-r&#8221;:<o:p></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">make[3]: Entering directory '/home/penny/FVP/r82/fvp=
_aemv8r_cortexr82_0.0_6347_gcc6.4/xen/xen'<o:p></o:p></p>
<p class=3D"MsoNormal">aarch64-linux-gnu-gcc -MMD -MP -MF ./.asm-offsets.s.=
d -DBUILD_ID -fno-strict-aliasing -std=3Dgnu99 -Wall<o:p></o:p></p>
<p class=3D"MsoNormal">-Wstrict-prototypes -Wdeclaration-after-statement -W=
no-unused-but-set-variable -Wno-unused-local-typedefs<o:p></o:p></p>
<p class=3D"MsoNormal">-O1 -fno-omit-frame-pointer -nostdinc -fno-builtin -=
fno-common -Werror -Wredundant-decls -Wno-pointer-arith -Wvla<o:p></o:p></p=
>
<p class=3D"MsoNormal">-pipe -D__XEN__ -include /home/penny/FVP/r82/fvp_aem=
v8r_cortexr82_0.0_6347_gcc6.4/xen/xen/include/xen/config.h<o:p></o:p></p>
<p class=3D"MsoNormal">-g -march=3Darmv8-r -mgeneral-regs-only -mno-outline=
-atomics -I/home/penny/FVP/r82/fvp_aemv8r_cortexr82_0.0_6347_gcc6.4<o:p></o=
:p></p>
<p class=3D"MsoNormal">/xen/xen/include -fno-stack-protector -fno-exception=
s -fno-asynchronous-unwind-tables -fcf-protection=3Dnone<o:p></o:p></p>
<p class=3D"MsoNormal">-Wnested-externs '-D__OBJECT_FILE__=3D&quot;asm-offs=
ets.s&quot;' -S -g0 -o asm-offsets.s.new -MQ asm-offsets.s arch/arm/arm64/a=
sm-offsets.c<o:p></o:p></p>
<p class=3D"MsoNormal">cc1: error: unknown value &#8216;armv8-r&#8217; for =
&#8216;-march&#8217;<o:p></o:p></p>
<p class=3D"MsoNormal">cc1: note: valid arguments are: armv8-a armv8.1-a ar=
mv8.2-a armv8.3-a armv8.4-a armv8.5-a; did you mean &#8216;armv8-a&#8217;?<=
o:p></o:p></p>
<p class=3D"MsoNormal">make[3]: *** [Makefile:413: asm-offsets.s] Error 1<o=
:p></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">And the reason is that in the implementation of &#82=
20;cc-option&#8221;: To handle this we do a test compile, passing the optio=
n-under-test,<o:p></o:p></p>
<p class=3D"MsoNormal">on a code fragment that will always produce a warnin=
g (integer assigned to pointer). We then grep for the option-under-test in<=
o:p></o:p></p>
<p class=3D"MsoNormal">the compiler's output, the presence of which would i=
ndicate an &quot;unrecognized command-line option&quot; warning/error.<o:p>=
</o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">cc-option =3D $(shell if test -z &quot;`echo 'void*p=
=3D1;' | \<o:p></o:p></p>
<p class=3D"MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$(1) $=
(2) -S -o /dev/null -x c - 2&gt;&amp;1 | grep -- $(2) -`&quot;; \<o:p></o:p=
></p>
<p class=3D"MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;then e=
cho &quot;$(2)&quot;; else echo &quot;$(3)&quot;; fi ;)<o:p></o:p></p>
<p class=3D"MsoNormal">```<o:p></o:p></p>
<p class=3D"MsoNormal">Here, that is, we are trying to grep &#8220;-march=
=3Darmv8-r&#8221; in output message of the test compile, and since the outp=
ut<o:p></o:p></p>
<p class=3D"MsoNormal">message(unknown value &#8216;armv8-r&#8217; for &#82=
16;-march&#8217;) doesn&#8217;t contain the COMPLETE string, it thought the=
 compiler supported this<o:p></o:p></p>
<p class=3D"MsoNormal">option, but it actually not&#8230;<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Tracing back to the commit about the implementation =
of &#8220;cc-option&#8221;, it discards the linux method(using exit code to=
 tell) for dealing with<o:p></o:p></p>
<p class=3D"MsoNormal">disable-warning options.<o:p></o:p></p>
<p class=3D"MsoNormal">See <a href=3D"https://github.com/xen-project/xen/co=
mmit/28f8fb7d2b3fde2f5cbe5526ac4f1c932e3f5d26">
https://github.com/xen-project/xen/commit/28f8fb7d2b3fde2f5cbe5526ac4f1c932=
e3f5d26</a> for more details.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">To fix this issue, I could loose the &#8220;grep che=
ck&#8221;, instead trying to grep the COMPLETE option string(&#8220;-march=
=3Darmv8-r&#8221;), only partial(armv8-r) is<o:p></o:p></p>
<p class=3D"MsoNormal">enough. But, hmmm, the way of greping output message=
 to check is kinds of reliable, IMO, since there is no standard for that.<o=
:p></o:p></p>
<p class=3D"MsoNormal">What do you suggest? <o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Cheers<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">--<o:p></o:p></p>
<p class=3D"MsoNormal">Penny Zheng<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</body>
</html>

--_000_VE1PR08MB5215CBA5B28A1FB90D7B161FF7D39VE1PR08MB5215eurp_--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:09:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:09:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181027.327980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNarD-00060a-6O; Tue, 07 Sep 2021 13:09:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181027.327980; Tue, 07 Sep 2021 13:09:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNarD-00060T-3J; Tue, 07 Sep 2021 13:09:47 +0000
Received: by outflank-mailman (input) for mailman id 181027;
 Tue, 07 Sep 2021 13:09:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNarC-00060N-41
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:09:46 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6c39c5b0-5c72-4792-bbfc-2b90ee5247fc;
 Tue, 07 Sep 2021 13:09:44 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631020175056268.27192467139764;
 Tue, 7 Sep 2021 06:09:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c39c5b0-5c72-4792-bbfc-2b90ee5247fc
ARC-Seal: i=1; a=rsa-sha256; t=1631020180; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=Bpqq9silydKZDS3unZW4woy8Lpxr6mBVys9wZHwLDmGkSFBvcuz26I72Qa5w45tC2IbZ676RjubXimUoLx+0k9RS6xpUySVwtIKJwfklno6c+4AVRBEs38tKlS+Aj3ga1PX0JERzJBDyvhQT/ye1KjeIksyipXXARdYmYmo1kp0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631020180; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=D/hKG9aSJpSBbbyNnfJLMboaDfUK/6nKEu6OPyO+A24=; 
	b=IJklqAFis6wcZh2pny0SbFa7MHqePfvYMUHMa2KxPZF7cQYguPbHcb+gsuviFx7PBg/6dPVxN1eHwQZS2VdPgsRdPQVcoFsGHitXKo9MsnFremxjfqEDUdSJF2NcnnOE5HilbVjDwBf4ktlY71ZrO3IWV1G0aUoFBD8+8xo/prw=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631020180;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=D/hKG9aSJpSBbbyNnfJLMboaDfUK/6nKEu6OPyO+A24=;
	b=kNCDrjwqigiY0hOgCyTNR7okXsPNdvhQ1HLl4oaSyZDed8X7D/1TjQIgKpLyis8P
	syoixqwlifuPMOaA7SJZ7u59+ZxZFro3ykKzc/JTd9Qi226kSxHGwpLE9wFF+wksekS
	cPmFSB0CHYUILPVxxOv59D5r6eW37NJ2FlXr1apA=
Subject: Re: [PATCH v4 01/11] xen: Implement xen/alternative-call.h for use in
 common code
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-2-dpsmith@apertussolutions.com>
 <3a91e4ad-50c4-205f-3d90-f23c996b2938@suse.com>
 <8c3dfbf5-5c98-c923-ca81-28af3d13c4a4@citrix.com>
 <7e71ce74-dd81-21c3-db78-c37d95d5b239@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <da23a4b4-e2fb-7dfb-1791-39bd756ba18a@apertussolutions.com>
Date: Tue, 7 Sep 2021 09:07:50 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <7e71ce74-dd81-21c3-db78-c37d95d5b239@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External



On 9/7/21 2:00 AM, Jan Beulich wrote:
> On 06.09.2021 18:22, Andrew Cooper wrote:
>> On 06/09/2021 16:52, Jan Beulich wrote:
>>> On 03.09.2021 21:06, Daniel P. Smith wrote:
>>>> --- /dev/null
>>>> +++ b/xen/include/xen/alternative-call.h
>>>> @@ -0,0 +1,63 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>> +#ifndef XEN_ALTERNATIVE_CALL
>>>> +#define XEN_ALTERNATIVE_CALL
>>>> +
>>>> +/*
>>>> + * Some subsystems in Xen may have multiple implementions, which can be
>>>> + * resolved to a single implementation at boot time.  By default, this will
>>>> + * result in the use of function pointers.
>>>> + *
>>>> + * Some architectures may have mechanisms for dynamically modifying .text.
>>>> + * Using this mechnaism, function pointers can be converted to direct calls
>>>> + * which are typically more efficient at runtime.
>>>> + *
>>>> + * For architectures to support:
>>>> + *
>>>> + * - Implement alternative_{,v}call() in asm/alternative.h.  Code generation
>>>> + *   requirements are to emit a function pointer call at build time, and stash
>>>> + *   enough metadata to simplify the call at boot once the implementation has
>>>> + *   been resolved.
>>>> + * - Select ALTERNATIVE_CALL in Kconfig.
>>>> + *
>>>> + * To use:
>>>> + *
>>>> + * Consider the following simplified example.
>>>> + *
>>>> + *  1) struct foo_ops __alt_call_maybe_initdata ops;
>>>> + *
>>>> + *  2) const struct foo_ops __initconst foo_a_ops = { ... };
>>>> + *     const struct foo_ops __initconst foo_b_ops = { ... };
>>>> + *
>>>> + *     void foo_init(void)
>>>> + *     {
>>>> + *         ...
>>>> + *         if ( use_impl_a )
>>>> + *             ops = *foo_a_ops;
>>>> + *         else if ( use_impl_b )
>>>> + *             ops = *foo_b_ops;
>>>> + *         ...
>>>> + *     }
>>>> + *
>>>> + *  3) alternative_call(ops.bar, ...);
>>>> + *
>>>> + * There needs to a single ops object (1) which will eventually contain the
>>>> + * function pointers.  This should be populated in foo's init() function (2)
>>>> + * by one of the available implementations.  To call functions, use
>>>> + * alternative_{,v}call() referencing the main ops object (3).
>>>> + */
>>>> +
>>>> +#ifdef CONFIG_ALTERNATIVE_CALL
>>>> +
>>>> +#include <asm/alternative.h>
>>>> +
>>>> +#define __alt_call_maybe_initdata __initdata
>>> My v3 comment here was:
>>>
>>> "I think it wants (needs) clarifying that this may only be used if
>>>   the ops object is used exclusively in alternative_{,v}call()
>>>   instances (besides the original assignments to it, of course)."
>>>
>>> I realize this was slightly too strict, as other uses from .init.*
>>> are of course also okay, but I continue to think that - in
>>> particular with the example using it - there should be a warning
>>> about this possible pitfall. Or am I merely unable to spot the
>>> wording change somewhere in the comment?
>>
>> Such a comment is utterly pointless.  initdata has a well known meaning,
>> and a comment warning about the effects of it is just teaching
>> developers to suck eggs[1]
> 
> Well, okay then - at least the definition of __alt_call_maybe_initdata
> isn't far away from the comment. (What I'm not convinced of is that
> people knowing __initdata's meaning necessarily need to correctly
> infer __alt_call_maybe_initdata's.)
> 
> Two other observations about the comment though, which I'd like to be
> taken care of (perhaps while committing):
> 
> - __initconst wants to become __initconstrel.
> - foo_init(), seeing that there are section annotations elsewhere,
>    wants to be marked __init.
> 
> Then
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> Daniel, you having made changes (even if just minor ones) imo requires
> you S-o-b on the patch alongside Andrew's.

Ack, I realized after sending I didn't SoB it, my apologies on that.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:27:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:27:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181034.327990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNb8i-0000Sa-OM; Tue, 07 Sep 2021 13:27:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181034.327990; Tue, 07 Sep 2021 13:27:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNb8i-0000ST-LT; Tue, 07 Sep 2021 13:27:52 +0000
Received: by outflank-mailman (input) for mailman id 181034;
 Tue, 07 Sep 2021 13:27:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNb8h-0000SN-KM
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:27:51 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df0e0929-302e-4613-a234-1499f1b101e9;
 Tue, 07 Sep 2021 13:27:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df0e0929-302e-4613-a234-1499f1b101e9
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631021270;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=CmfnqghAQMqOoM1IPQVi098sHa9O0hBNYIPkFMm0Slg=;
  b=V39w2J+IFvtPtJDjCqry6ChXQAoHMqzKihZIM6px5bkBZzKIPB2JQrau
   pg8B2Ql8t0mR5iJfL1PDYke33r96lCNGewA16BQY8aN4rmzbupNRpP75d
   4UJUa1qo4QaHUoF/lHNuZMrhU6vv2CwCr8R5K5FekC91AR/9ttVp+SdQ3
   I=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: MBMbwhs+Qt/PsEDFcEedgW9L1WQkFHYIvRJ6XzlE5+EdqVBgdsQceh28n0zFOzZAzvR7PZkWvh
 wCmDq2qYH+uwhbGSrOZplbCaaHlJKjJRrfjDyKIglgUFoSGeY+/zU7Y/2eVaGYeJuwgb+tAyl5
 0hVa8zI+t2QFFB9Y6g25MVX8jWcGoXw1kIIw1aBJoxiI5/fepwoZCQB6mCotGIt7ogY0+6aH83
 r1TKVQW0EhI235a5VDUQTGCZxBBxulybwSAft5ezLa5lJsHsPCAdAfdovkt9PrOTq3fCVW9RO0
 60gMJrPgclpU5Gu3rJ4+Xnlx
X-SBRS: 5.1
X-MesageID: 52165139
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:qUSAQa0UlK89prBUkz3azQqjBIEkLtp133Aq2lEZdPUCSL3gqy
 nOpoV/6feX4Ax6ZJhEo7290ca7LU80maQb3WBzB8bBYOCFgguVxdpZnO3fKlTbckWUygc678
 ldmsNFeb7N5DZB7PoTT2ODYq4dKHXsytHNuQ9+pU0dKj1XVw==
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="52165139"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ReJT8obGyqBoZtAvo00onIB41y5N3PnJpvki1+Iferprxd5zgPdO66QS1QH47UhQP6eGVYxIKxCkn46sTNxmqHfegMxJD/+SkHWCgRlO4FAsCr9EfRgT7zFVbzeTdea85uoh1RqJ1h2Dpvvk5HgJHs54por3fLVPnV+SGiwK41ayQe8pqQ1tTwRY/J7WN0ZYT1mGXLYdIe3jgvYfhgowasX9UUyIQkvkix7o1MEs+mTjx05iCGEKH4F+puku18qjHPIsjqsMignaP80xIk9NHqPS5S7vzAHpJeDd54nAqzyeg6sh3RAS3uWZHHqBqQ2a+d7z4ROan/iR0AwQBw8fvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XZiBBNReedoay/JRiVqq7EKaNRkuz+H44YkjqwhFLpg=;
 b=TTxAnMzYVzcYhnw+diGSR0GrKhYPz6lTsR4ZSmTAAoZxUn7LRXyezDtIcD5jnG/pNh2cvzVSX/0hM16h+0DyS4NYnGU3yb6OGQbQnrniLUNRzH/VXBRAz2jA4NrnhNTH2+FVVhNp2UTrwdmq8PfSqQiYZ644jd0gZ7BXIlBbR6qUUI4wg/qN42IleflceiZVUA+LNEKCjhd3dbUb7BZdbqX2ZQdeqfGHZBoyv8625eixB8PLSngL0PHmXUA7+dUCEz3fJFFp3ZVs9ZmDgxfbBru2iTLstdSW5ehocDBVwVR4wQlo5lQ8Eg7g6JbFAvdO77ajj3N5JFPNT8hLyqcSHw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XZiBBNReedoay/JRiVqq7EKaNRkuz+H44YkjqwhFLpg=;
 b=fd7FJCaUPFkcZR25Gn1LUZtkvrb5vLu5EnSM25EbnxBwysypAmxGw+gA0XZgjGpDHC+n8lP90WcliXFsYG5oLZhizQxuymedbOxrgt5ZZiYfTnCjeUd10nYiYKJ3te+d7jcqXq47gj9Li8E0MQ1u60SjgexzfCZTdJ1kNiZwK80=
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Pu Wen <puwen@hygon.cn>, Andy Lutomirski
	<luto@kernel.org>, Xen-devel <xen-devel@lists.xenproject.org>, Jane Malalane
	<jane.malalane@citrix.com>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
 <2972c71a-5fed-0813-8ec0-d179eacff14d@citrix.com>
 <70f376c6-eef2-42ce-a8eb-67c1130a41e4@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
Message-ID: <dcdce06f-4a19-49c7-2225-a8e38d524dd8@citrix.com>
Date: Tue, 7 Sep 2021 14:27:41 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <70f376c6-eef2-42ce-a8eb-67c1130a41e4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0277.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:195::12) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c51edc4a-7dac-4f68-772c-08d972034851
X-MS-TrafficTypeDiagnostic: SJ0PR03MB6391:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB639166DBD1ED7E2ECEB972C4BAD39@SJ0PR03MB6391.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: T0Eo6sq9hIUbLzfZOrrAOCnHfqlMuWF9vFsyxmU0WrQ/3MtVmx9H7dNaT0KjfCAI4YSAo4OQ39gjQ7ZD7XVXo0ALDT/0uGGz4Ob0Oc87ge2SiejiH2JRip5bKPMmrkm8iUOKMhOIUz75sXTqbEsivfmoBzwa0oymxajpJuhfG1i+EIJraCekAsWrpn4mwJkf/crutFj3mjphr05K+aUI+vYnhA0P7gXFsBrKZDbBplIJV/bQ7ALgcTT0rDctZUZIOGvC4dEQOdcU5YCwDvwrlMavwrvpXiphqdmB7LRnsSvwYZRRwUmEc7LvhHbNExVO5m6h1lkOVvmMMG73dfEA8K2n+YHG5l9s5RgKYFXuLvOxIsiGx0itkPmUIJCvUL6a2JjKTnoY1blDtAMvo1rwyG45ILTqwO23RKzPVarP5mHF6h6Ysu9Ofa2v7S8z3ct+LFz2rSnJmHPKuRUr8VbauLQqLcympSkDwSLz7+QdAyhYjLT0G6wYMkJUQ9U8tHJNumt3xUIzxxGWnRNSeqsEQT81DNk2z5X5XYHPf/vQaHqj28lK72JObFguybuKts7Cr6h67lXiFTzh3LsQtPD3Bvbb8uw4echaS+rHljVlxxnxK6ROhMXbmxOGU289wAJW5b+Ji7HMkJA/VKANKANKDJxyYo2E341Mwkzrvilb7kL3CrlBBaUy6YYivUow5I2u0q3wZU7wcdzK8av0htPXORcpyQ1iMSYTYIwTJjb07SE=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(6486002)(55236004)(508600001)(316002)(83380400001)(26005)(16576012)(8676002)(6666004)(66556008)(31696002)(6916009)(66946007)(86362001)(956004)(2616005)(38100700002)(66476007)(5660300002)(8936002)(2906002)(4326008)(186003)(53546011)(107886003)(36756003)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?L3hUUnRCMkJLNSttSFZYVTFKSmZ3bmthUCtlOUF6ei9pS0xBTVpFdHFLU1FU?=
 =?utf-8?B?czVidm01aUZCUGx0enBsSCtlakxrbjl4U2tMaUt1MnRkSjFmSytSVXg4OVNl?=
 =?utf-8?B?ODdIYWExT01NeGVOTjcybDVPbVhHbG1TR0pDS1BvYlVUYURzR0VvSnZuZ25v?=
 =?utf-8?B?VmNnaVlDMENwK0JIeTdoL2F3L1hpdktucCtTMHdWV0xDMFp1Wk13NFgyZmlk?=
 =?utf-8?B?Yld0QnZGaXc3NnE1dytETEVMNDJ3U0dSU2VmUzZuRHdhVEQxeWo2cnA0c1Q0?=
 =?utf-8?B?QmtBU3M4MC9LWS9rUGk1aC9CQStSVjhYUk5nVmFJSzRVV3FzbjREMlBjZUlU?=
 =?utf-8?B?UFR2K3A4cnprMHhYUytTcUdZbklla1dVS2ErREpCS1FYc09URzczQnVjSTZk?=
 =?utf-8?B?RHFkbCtDQzcvTGQrYTlqcXNRWjdubGtlZ0hiSWptNDZ5ZVczcS9nekFJbXJM?=
 =?utf-8?B?Z1N3NER3UGVpTkVkckMvOHplWmt1clNqQVJXd2JlaWl1cEFUNmRiWDdWbWcr?=
 =?utf-8?B?SWc1THRFZEVNdFZVcFpuNHFwd09kM1BESzYyL1gxRVQ1MDQ1eXFTUHA5YUU5?=
 =?utf-8?B?ZStQYTNOd2NoMGF5SFB6TFliLzllU1FCRXpkdmc3bHNnR2ZvaFdVcERDbmov?=
 =?utf-8?B?dGNvYkNqYW9wNHk4ZER6dFp5UjVKbmlRd2t0bXNYRUt0am0xTzdBTFB0bVhr?=
 =?utf-8?B?ZnV1TWtueU5pV1lweU92RnB3UkZIak9KdWp0RzM1d0syclo0NXh1NHkzblBD?=
 =?utf-8?B?dkNBSlV0cFI5b1VtNzc4d2d5TFREWUMvUXJXTGx4WU1JMU5oU2tXMzFWdXJm?=
 =?utf-8?B?cHBiTy95MGZBWFVkbk00K0cxQkxWZzdVOGhaLzVWT012SFdYSnZzamYrNVNU?=
 =?utf-8?B?MDcvUVF2VXhVSjJCaDI5ZmVjRldnRExZY1MzbTJBZnhPcHNDQXhPTXF5ZVBk?=
 =?utf-8?B?bERCSUMyVkhsN3p6bkNScERZeWp2TnpnUitCZ3c1SDcxbU1IS1RuamJjNnl5?=
 =?utf-8?B?RzlQVVc5QkpUdGpEU21tVFBsRm5LWDJyMTZZVDA3SzQvU1BhYU1VSWo1a08x?=
 =?utf-8?B?OWF0UkM0RWYxSkdlMy8wVFlJcmRENmZyblNOY0RFUzFocDJDTE9BMnNrUlBk?=
 =?utf-8?B?RW1MQ2RBOXpUNitVRmRhOUlmNVppNTZOMGlnb0VCQkdGSDF1WGE3U3ZmNm04?=
 =?utf-8?B?QTJxS1I0MXhJdTBjUlg4RXMyczYxMXl3Ty9QUGhkVG1WZlg0dWVaL2FQMFlY?=
 =?utf-8?B?dSs4ZmlFSWxKcWtUVHFqdWw5VW5tL241RGFRRzBvS253ZHVHb3g1b2RXMTBy?=
 =?utf-8?B?WURoU0VpTUJoZnVQNWlycld2dEl3ZEZvVGE5bmlyZ3VKSHFYR29JdnJnNWJF?=
 =?utf-8?B?UnNxV0JsYVh6U0FIeGdYRnYzZXdXak9zQXMyWDU2Nks3QXNXY2dob3FPUEhS?=
 =?utf-8?B?c29lV2tKcnVBTk1BNEk3RzNWT2I4RWNQdkQ0RmNoVzdxWWVHVTlDOTBZTVVL?=
 =?utf-8?B?NkJmUUFyS0Z5c3Z0MG5GWm5keUlZZGYxT051VHdWa2xuNkJDWjQveloxeHlh?=
 =?utf-8?B?YmlQR1RHb3VUdTk1ejBGWjV2R1lId0J4eHUyeEhUV0ppWEZDbXhIcFVJRmYz?=
 =?utf-8?B?MVBTclJNdHhlUlNsWmhhaGkxMis0dmY2bGl4bzVXNkV2MTRmZmFNZEozM1I4?=
 =?utf-8?B?MnFFcWdqcEhNQW9WcU9PT0pIbXRSWUF0T1EvT2gyUStZSmkzL3p5MEMwOXlW?=
 =?utf-8?Q?JAz8Og6l7oflZGmFVvqs1ZJs7cazRq/tqcjrPQ/?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c51edc4a-7dac-4f68-772c-08d972034851
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 13:27:47.5091
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1qegWzUvuOVh20oKIBvHEmKo5Dpz8ZeHYDzaUgMp3v+LJ6hYaXF8tvDKXtSVhIi1LbzzuMz0PzHhAKt80bUvh5MzNWzEZv1u91cKrRcLeyk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6391
X-OriginatorOrg: citrix.com

On 07/09/2021 07:09, Jan Beulich wrote:
> On 06.09.2021 20:07, Andrew Cooper wrote:
>> On 06/09/2021 16:17, Jan Beulich wrote:
>>> On 06.09.2021 14:00, Jane Malalane wrote:
>>>> --- a/xen/arch/x86/cpu/amd.c
>>>> +++ b/xen/arch/x86/cpu/amd.c
>>>> @@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
>>>>  			  c->x86_capability);
>>>>  }
>>>> =20
>>>> +void detect_zen2_null_seg_behaviour(void)
>>> This can in principle be marked __init.
>>>
>>>> +{
>>>> +	uint64_t base;
>>>> +
>>>> +	wrmsrl(MSR_FS_BASE, 1);
>>>> +	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
>>> While I don't strictly mind the "m" part of the constraint to remain
>>> there (in the hope for compilers actually to support this), iirc it's
>>> not useful to have when the value is a constant: Last time I checked,
>>> the compiler would not instantiate an anonymous (stack) variable to
>>> fulfill this constraint (as can be seen when dropping the "r" part of
>>> the constraint).
>> This is "rm" because it is what we use elsewhere in Xen for selectors,
>> and because it is the correct constraints based on the legal instruction
>> encodings.
> grep-ing for "%%[defgs]s" reveals:
>
> efi_arch_post_exit_boot(), svm_ctxt_switch_to(),

These are writing multiple selectors in one go, and a register
constraint is the only sane option.

> and do_set_segment_base() all use just "r".

I had missed this one.

>  This grep has not produced
> any use of "rm". What are you talking about?

TRY_LOAD_SEG(), pv_emul_read_descriptor() for both lar and lsl,
do_double_fault() for another lsl, lldt(), ltr().

So ok - not everything, but most.

>
>> If you want to work around what you perceive to be bugs in compilers
>> then submit a independent change yourself.
> I don't perceive this as a bug; perhaps a desirable feature. I also
> did start my response with "While I don't strictly mind the "m"
> part ..." - was this not careful enough to indicate I'm not going
> to insist on the change, but I'd prefer it to be made?

No, because a maintainer saying "I'd prefer this to be changed" is still
an instruction to the submitter to make the change.

But the request is inappropriate.=C2=A0 "Last time I checked, the compiler
would" presumably means you've checked GCC and not Clang, and therefore
any conclusions about the behaviour are incomplete.

Unless there is a real concrete compiler bug to work around, "rm" is the
appropriate constraint to use, all other things being equal.=C2=A0 If the
complier is merely doing something dumb with the flexibility it has been
permitted, then fix the compiler and the problem will resolve itself the
proper way.

>
>>>> @@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
>>>>  	else /* Implicily "=3D=3D 0x10 || >=3D 0x12" by being 64bit. */
>>>>  		amd_init_lfence(c);
>>>> =20
>>>> +	/* Probe for NSCB on Zen2 CPUs when not virtualised */
>>>> +	if (!cpu_has_hypervisor && !cpu_has_nscb && c =3D=3D &boot_cpu_data =
&&
>>>> +	    c->x86 =3D=3D 0x17 && c->x86_model >=3D 30 && c->x86_model <=3D =
0x5f)
>>> DYM 0x30 here?
>> 0x30, although it turns out that some of the mobile Zen2 CPUs exceed
>> 0x60 in terms of model number.
>>
>> As Zen3 changes the family number to 0x19, I'd just drop the upper bound=
.
> Minor note: Even if it didn't, the !cpu_has_nscb would also be enough
> to avoid the probing there.

There is actually a problem.=C2=A0 From a non-AMD source, I've found the
Sucor Z+ CPU which is a Fam17h Model 0x50 Zen1.

So instead I'm going to recommend dropping all model checks and just
keeping the family checks.=C2=A0 This will extend the probe function to Zen=
1
too, but it is once on boot, trivial in terms of complexity, and really
not worth the time/effort it has taken to discover that the model list
wasn't correct to start with.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:31:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:31:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181041.328002 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbC3-00026t-8o; Tue, 07 Sep 2021 13:31:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181041.328002; Tue, 07 Sep 2021 13:31:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbC3-00026m-5l; Tue, 07 Sep 2021 13:31:19 +0000
Received: by outflank-mailman (input) for mailman id 181041;
 Tue, 07 Sep 2021 13:31:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qx60=N5=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNbC1-00026g-6z
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:31:17 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.52]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id dfe7ee80-0fdf-11ec-b0f3-12813bfff9fa;
 Tue, 07 Sep 2021 13:31:15 +0000 (UTC)
Received: from DB6PR1001CA0004.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:4:b7::14)
 by AS8PR08MB7141.eurprd08.prod.outlook.com (2603:10a6:20b:401::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 13:31:13 +0000
Received: from DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:b7:cafe::cd) by DB6PR1001CA0004.outlook.office365.com
 (2603:10a6:4:b7::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 7 Sep 2021 13:31:13 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT036.mail.protection.outlook.com (10.152.20.185) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 13:31:12 +0000
Received: ("Tessian outbound 620209b93b95:v105");
 Tue, 07 Sep 2021 13:31:12 +0000
Received: from a7dc6487bcbb.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EAF25DB4-9348-4ABC-A66F-8F4F675F57A6.1; 
 Tue, 07 Sep 2021 13:31:00 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a7dc6487bcbb.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 13:31:00 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 13:30:58 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 13:30:58 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0022.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:151::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 13:30:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dfe7ee80-0fdf-11ec-b0f3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AHo4LrTOFjlE7FFmnLbeXvdli6Hhb31JUEvOqsq4SnM=;
 b=IMvgmKvHQesTA4cEnkAVwOSqoE3fzUeK5rvWhcl1v5IFrLu0O2A2YeUxrJwEfhJYDfH4TO02Sy896Z+2OZpTEYUIV4YuRWpZlcvpUkpbBnp5mvAOduUsdczQm0V/uO6Csp6VaZ1MiOsOHVSembYqL/XvnAiucrSycmzsdgx1AF8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8183cf2115e268ed
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LGWHMWRjEmBWbTm+hPJqo+sjKJoakgpceunvtbJL1JQSz2XLpRJz5aLTgqgWG74cG/TiSt2Z8GbSdRaDmMyrq4cGZq1HMxVaxyraM2kRBX7jrjA/2L/4BnueRTyW3baBYZGCufIflpD+cUq5BrRxsyG59krre1Ziei3xoo3N3TBxJEI8+QtQ+YPjU4ou0OUvd77WvcUGwOH5JkbLIB4r5PCHsvM52IbwxD0EbbljPAnMx99BAfN94dwvtnpgUa3bV55aQAuDXZs6z68ZLCOKy0XP1Ax7JpnyxmwjTOA0k+kB2IWWsdfvL5605mos4UVSlquQZNjo5JSMjd62SmBsMQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AHo4LrTOFjlE7FFmnLbeXvdli6Hhb31JUEvOqsq4SnM=;
 b=j7GEbzytWfyuCjAqE8uV4AeCQ1fPS1d32c/d124f6kkdWxdlpFpVKPpoKcDEWBhFu4o/+EeKbSHCtlD7PDa1MtqNHrdYMChWShFU7Xss84VPAYCqQaUTCq4ygRwebuh8MqMGUZmsZeQn0FxqlK1xSO/6Tfr/zqkJdCgnlzFbgV4gmBS6dczns/XNl5miOKekDd6xBVnaceOnDmCiYhWTGgIog9Vr4jWobYJEJS7kVeZbUdJS/MnJf6Yz6LR5PNOSIJ00/nDPM/B6iYpDhtXU8mHkfGbAiZceSQaIbetPA1UdeRI3TSewLjVgT+E3b9XzB3+hA3x8W4PSdOtR8LsvPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AHo4LrTOFjlE7FFmnLbeXvdli6Hhb31JUEvOqsq4SnM=;
 b=IMvgmKvHQesTA4cEnkAVwOSqoE3fzUeK5rvWhcl1v5IFrLu0O2A2YeUxrJwEfhJYDfH4TO02Sy896Z+2OZpTEYUIV4YuRWpZlcvpUkpbBnp5mvAOduUsdczQm0V/uO6Csp6VaZ1MiOsOHVSembYqL/XvnAiucrSycmzsdgx1AF8=
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
Date: Tue, 7 Sep 2021 14:30:52 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <5DA71CEF-0ADE-4846-A7BC-B378B95A38AD@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
 <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
To: Julien Grall <julien@xen.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0022.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:151::9) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0643375f-98da-472d-57b7-08d97203c262
X-MS-TrafficTypeDiagnostic: PAXPR08MB6969:|AS8PR08MB7141:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB71419933E0E3D199A59025F4E4D39@AS8PR08MB7141.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 2vB0kVeDNCTk12dH9bO0Se7/fpANSRIrd7a4cU06ohk2M8sXVdqa2YUlT26J69Iov+FdKYuZjn2MitGwexLmjPV6sg/J6Nd8doK9xwX0vO9kZsKvUpw7VBiNlL/yfiOEt2HQAjYTPsBv3oHd01OT6ngdRDwmtpP1FH5Qnf5GZ+MNER6DphS4gXOkDjUPgJsKglppjME7lVhZ8+6jNyOitum1Y8/nLfz8rZ6UGLxFkPZJIniKxV8Ecn66JCR/dM7caDFt9LxfojQY2FvD3o0kUxaY2mzR4hJw4LKr6mii5O3xl6GC5p+gPlUtJGw9mFFNSaALLVhrGJQUU1tas3fbkHHI4cfkZWOD1jtMuJ+RAZwo5oak0E+6W9p+oPC25cJa8JUy7k+hSpY7JvSGCkXnfrpPnpAxIGnyB+RwVkv+UrTSfnGrjeoksCesCbDX64IZTDYH7gtiYBtAJgChFSJdr54z497C4bPk8MNzinuLsvQK72yT+KBL2+6GqtqZ2wLjgsvWlaS4eJG6p0zib6Fqa5ylFxML21grY/l5YHOHRq86yg1Y0B7cobSW0tJgDKHrE6hb0YaCuKpDYJTpopFA0b/teLa7OmiRA3bHCoLFNhUZLTIQIsZ1754GiKN+6RtFeX9QX2vksqg3KGFqit65ZPrmtNmTFc+XmBMIo+yIP2XtKl6H05po4Ayf0RQAziuwBKQ/sWpPrkCV/Q5EK3DlPQ/yumtq0duZn2pRWA1S4hxSNEK1LlpUaDrFp8Oj8SVG8DZlg0/NzbVWeCujd2HkpA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(136003)(346002)(396003)(376002)(39860400002)(956004)(2616005)(86362001)(5660300002)(66556008)(54906003)(52116002)(66476007)(44832011)(186003)(6916009)(66946007)(6486002)(4326008)(478600001)(26005)(6506007)(6512007)(53546011)(33656002)(83380400001)(36756003)(8936002)(2906002)(38350700002)(6666004)(8676002)(316002)(38100700002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6969
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	35f4f803-6d1a-4dfd-a445-08d97203ba32
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qkckQu7eZI4UEyntSBsy5Z9ym2TBShUutExOvcOFGUZwAAHTyhTB7w7RHpbmXzq2tJbk4IH7xbzoCwyADS4BYma7J4x1SB9P/gKzr3qQ+RRHUpqK00AL9wedGth+WJGZt/Nm8giLECCikAQ1PYdYtOdDLr8i8ACTjcHDpL/worj7I9peYz02TRK+23mIDp2hywW3oPDoYUvYr6GajNOJCwjrSDGj0XIXzvjTN11bPO+uQDBtd7LH09xvfS6l9RIC6+W1RiJ/z/EM1a5HLSbmcU17Y/JXu7brscp45eFciP4CWWEky9qxKGD8tqimSpy2egF3J6WRaZnMpshLTfk+wyFAEfn9qn1zRJVn6S/uOwNAWZaQFBAi++VC/5vUmfrEG0xc5+5IMFLXglrG/nURPV2HupkzGUmeqTuus5GuMozan2gDQ56q84R69P1VmVJ5VwewYrvob5caA/B7DW/rL2/VIvR6np5SqzeZsHJ2Om5zt2v3RUGGZR7rW54BjsJ4DWVBlGnjCE3i4nrrulJWiW0AvXa61OAVCr78AqZ0vyHGWuUbWVsAW1mXZ35TVmMuBwcXALKLXCFrWkCaYpCbHgMhojmdN3aMy2SX//fQEkj5JI4jw907m+xrRbItJp0NKePRurKvwn2jscCpyDNVz5c9qUVkNWaLWubExNrV+h3LIP0swZYv03zieALeOVsZJCpOIPmV7GHAEB2p7c5vpNlV2TY41iABsp6f5Ia/ZUI=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39850400004)(376002)(346002)(396003)(136003)(46966006)(36840700001)(6862004)(36860700001)(53546011)(186003)(4326008)(36756003)(8936002)(70586007)(83380400001)(478600001)(6486002)(2616005)(70206006)(6512007)(6666004)(47076005)(26005)(86362001)(6506007)(956004)(82310400003)(5660300002)(336012)(8676002)(316002)(2906002)(33656002)(82740400003)(356005)(81166007)(54906003)(44832011)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 13:31:12.1239
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0643375f-98da-472d-57b7-08d97203c262
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7141



> On 7 Sep 2021, at 13:30, Julien Grall <julien@xen.org> wrote:
>=20
>=20
>=20
> On 07/09/2021 12:51, Luca Fancellu wrote:
>>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>>=20
>>> Hi Luca,
>>>=20
>>> On 07/09/2021 07:52, Luca Fancellu wrote:
>>>> Add a design describing a proposal to improve the EFI
>>>> configuration file, adding keywords to describe domU
>>>> guests and allowing to start a dom0less system.
>>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>>> ---
>>>>  docs/designs/efi-arm-dom0less.md | 105 ++++++++++++++++++++++++++++++=
+
>>>>  1 file changed, 105 insertions(+)
>>>>  create mode 100644 docs/designs/efi-arm-dom0less.md
>>>> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-d=
om0less.md
>>>> new file mode 100644
>>>> index 0000000000..8d8fa2243f
>>>> --- /dev/null
>>>> +++ b/docs/designs/efi-arm-dom0less.md
>>>> @@ -0,0 +1,105 @@
>>>> +# Xen EFI configuration file
>>>> +
>>>> +The current configuration file used by Xen when it is started as an E=
FI
>>>> +application is considering only the dom0 guest and doesn't have any
>>>> +property to describe and load in memory domU guests.
>>>=20
>>> From my understanding, the problem is less about properties (we already=
 have them in the Device-Tree) but more about where are the binaries locate=
d in memory as we don't know in advance.
>> Hi Julien,
> Hi Luca,
>=20
>> I think I used the wrong word there, I meant =E2=80=9Ckeyword=E2=80=9D i=
nstead of =E2=80=9Cproperty=E2=80=9D because I was referring about the
>> lack of keywords to describe a domu guest in the Xen EFI configuration f=
ile.
>> I agree with you that on systems with static allocation, the kernel and =
ramdisk binaries must be at certain locations
>> that are out of control when we use the EFI boot services, the thing we =
can do is provide a keyword to specify the
>> addresses and then use the CopyMem() function to relocate the kernel/ram=
disk in the address we want.
>=20
> I wasn't specifically referring to static allocation here, sorry if this =
wasn't clear. I was pointing out that most of the information you create in=
 the xen.cfg is going to be similar to what we already provide in the Devic=
e-Tree.
>=20
> My main concern is everytime we add a new feature in Dom0less, a develope=
r would need to write code for the DT and UEFI. This will increase the code=
 size and maintenance.
>=20
> The same can be said for the admin as if they want to switch from plain U=
-boot to UEFI, they would also need to fully rewrite the bindings.
>=20
> AFAICT, most of the information provided in the Device-Tree are usable ev=
en when using UEFI. So I would prefer if we try to re-use what's existing. =
This is what my proposal below was about.
>=20
>>>=20
>>> So I would like to propose something that build on top of the Device-Tr=
ee work we did. Note this is early thoughts.
>>>=20
>>> The problematic nodes in the DT are:
>>>=20
>>>        module@0x4a000000 {
>>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>>            reg =3D <0x0 0x4a000000 0xffffff>;
>>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>>        };
>>>=20
>>>        module@0x4b000000 {
>>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>>            reg =3D <0x0 0x4b000000 0xffffff>;
>>>        };
>>>=20
>>> In particular the property "reg" cannot be known in advance because the=
 UEFI stub will be responsible to load the binaries in memory.
>> Yes that=E2=80=99s true, the UEFI stub is using from the UEFI boot servi=
ce the AllocatePages function that is giving back an address out of our con=
trol,
>> then using another function the binary is read from the disk and copied =
at that address, finally the UEFI stub is writing the node in the device tr=
ee that
>> will be used by Xen later.
>=20
> I am not sure to follow. Are you saying the UEFI stub will create the dom=
0less node in the DT based on the xen.cfg?

Yes, in my mind the UEFI stub would have created all necessary node into th=
e chosen {} to create the system, now the stub does that
but only for the dom0 modules.

>=20
>>>=20
>>> What we could do is providing a list of binaries to load and associate =
a key for each of them. Something like:
>>>=20
>>> binary=3D<binary> <key>
>>> binary=3D<binary2> <key2>
>>> ....
>>>=20
>>> We can then replace the property "reg" with a new property "uefi,key" t=
hat will contain the name of the binary.
>>>=20
>>> What do you think?
>> Here I=E2=80=99m lost, because I don=E2=80=99t understand what we are go=
ing to do with the name of the binary.
>=20
> <binaryX> would be used by the UEFI stub to load the binary in memory. Ea=
ch binary will have a <keyX> which helps to refer them in the Device-Tree. =
To give a concrete example, let say we have two dom0less domains:
>  - DomA: 2 vCPUs, 128MB
>  - DomB: 3 vCPUs, 512MB
>=20
> DomA and DomB will be using the same kernel but a different ramdisk. xen.=
cfg, would look like:
>=20
> [global]
> default=3Dsection1
>=20
> [section1]
> options=3Dconsole=3Dvga,com1 com1=3D57600 loglvl=3Dall noreboot
> kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> ramdisk=3Dinitrd-3.0.31-0.4-xen
> xsm=3D<filename>
> dtb=3Ddevtree.dtb
> binary=3Dvmlinuz-guest domu-kernel
> binary=3Dramdisk-domA.img domA-ramdisk
> binary=3Dramdisk-domB.img domB-ramdisk
>=20
> The chosen node in the DT would look like:
>=20
> chosen {
>    domU1 {
>        compatible =3D "xen,domain";
>        #address-cells =3D <0x2>;
>        #size-cells =3D <0x1>;
>        memory =3D <0 0x8000000>;
>        cpus =3D <2>;
>=20
>        module@1 {
>            compatible =3D "multiboot,kernel", "multiboot,module";
>            uefi,binary =3D "domu-kernel";
>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>        };
>=20
>        module@2 {
>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>            uefi,binary =3D "domA-ramdisk";
>        };
>    };
>=20
>    domU2 {
>        compatible =3D "xen,domain";
>        #address-cells =3D <0x3>;
>        #size-cells =3D <0x1>;
>        memory =3D <0 0x20000000>;
>        cpus =3D <3>;
>=20
>        module@1 {
>            compatible =3D "multiboot,kernel", "multiboot,module";
>            uefi,binary =3D "domu-kernel";
>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>        };
>=20
>        module@2 {
>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>            uefi,binary =3D "domA-ramdisk";
>        };
>    };
> };
>=20
> With this approach, the change is quite minimal to move between an classi=
c U-boot boot and EFI boot.

Ok now I see, yes this approach can work and can save some code, in the cur=
rent code we have that if
a "multiboot,module=E2=80=9D is found in the dtb, the Xen EFI configuration=
 file is skipped, but if we use the
module@XX {} without the compatible it can work, the UEFI stub will load th=
e binary and update all
the needed properties (compatible, reg).

Please let me know if we all agree on that

Cheers,

Luca


>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:34:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:34:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181048.328012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbEn-0002ol-Re; Tue, 07 Sep 2021 13:34:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181048.328012; Tue, 07 Sep 2021 13:34:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbEn-0002oe-Ol; Tue, 07 Sep 2021 13:34:09 +0000
Received: by outflank-mailman (input) for mailman id 181048;
 Tue, 07 Sep 2021 13:34:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNbEm-0002oW-QY
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:34:09 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4660a422-0fe0-11ec-b0f3-12813bfff9fa;
 Tue, 07 Sep 2021 13:34:07 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 187Bh1rU005659; 
 Tue, 7 Sep 2021 13:34:04 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ax5pm0sg4-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 13:34:04 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com (2603:10a6:20b:13c::18)
 by AS8PR03MB7159.eurprd03.prod.outlook.com (2603:10a6:20b:2ea::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20; Tue, 7 Sep
 2021 13:33:58 +0000
Received: from AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44]) by AM7PR03MB6325.eurprd03.prod.outlook.com
 ([fe80::d9b7:c0ab:2c2f:5b44%3]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 13:33:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4660a422-0fe0-11ec-b0f3-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=D6B/+dyscpm3VB+2Vgeqw1WFBxAhB8MYDLlNwN2qAjSohNb09W0AC7G3qPoKIem9V1iR3GeFM3HsjNrtMDL89Luykv6eu0TnWkFPspR5tLshkD9KCTOgsOD9jD5DI9vujdshVrE3MHA7Jhjw3QHLvHQEywqHztYl2N/xFfE5imCwFlNyx4x27HS3uJ89kuPah7IpYgnMe9jZ5vquqljxvez68GEykhQLxYexpDUDlM2QFkGNqcTTYUz56ZeqW/P7s0rLufXPN6gDtCc6Jn325Uwf4L2CuGd+RJ0PCIv4Bdu7LsGURA/koz/iuvD2F2w1TjmPgxGIO5uoyi/9FptItg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5BdAiv68cInWy+SgVNonoK3SC21/xHs/t4Ug12IeLdM=;
 b=gATdMFuKb0jXbZrtmMFtoQYTxVym/5H0DF14aka3BSFXzSjNme5KF5XUCW1/TQvC7MRkePA35/oM8JDL41/PihmDTorrErQ98nmga4xNXgQmPveqdKJcK4ShkB7Ajn4WygSnKmYjziX9sY0kkNyFS8Z4NGmq9Aztjl5z4Tj84L7+co/XglFII6dGr1q16YlpoAIVrCp3NVwqsVtNy2bQajZlJCVNHI4oXO/695usSijmfGvUz36A7zGvAPlne5SolMIi+4+9qw6Ml/cvcO2TRKkxOEZTZ2uJMfvV6jZ+WKGMePP3xvrBtisgy8Sbxo4rYV0cirhL7xY8NTxQR6KKwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5BdAiv68cInWy+SgVNonoK3SC21/xHs/t4Ug12IeLdM=;
 b=sVbJgih1zSDbeO54y1a1oAtJUR1rlz+8L/UUk1sgUZvKTAYFdaqI1+ljqQlpZ3zj+wiJRJ0LVFv7+16FFlD3OiPdABQh0tjgQp8SnlftKDHpjujylmBAe4ehvnb8ECb0Inaq3WMke+kvJpkPdmSvEaf2t8CvLnGyY+54Qu4RX7dSW4eUmioy0L/HY6LFc23xZM5L+C/IJOSVdGOkfm/SGfepdLMzDAkeNQRCAiIUamr+On6qTUhBsdS4stmCpGgWawSqjL79qsOMefXapMayWfEO7tpeIWgiYvvHdgby/UWgRH5E7MtLSARo7PBg31YhyOFvltwgOAHlm1K70Ijx6A==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLIA=
Date: Tue, 7 Sep 2021 13:33:57 +0000
Message-ID: <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
In-Reply-To: <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2af00d5c-e119-42a4-0ea1-08d972042545
x-ms-traffictypediagnostic: AS8PR03MB7159:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AS8PR03MB7159EE71E1FE6434E27E76FBE7D39@AS8PR03MB7159.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 hBFoENzXgo0ZlJHRFSRY7zsIgq3NkT5KL5ZlAG6Lvs8elEmvhLdth2C9YkR8ZiW+LeqTLakVsxXWn6DxbDyhJdz9e8hi8pheacg7bJmBrvtixakZnAjMmbY6y7EccHRDoj8BtgZ8l1n0LUABcg8yDwyzlmEvUotIU4aiGojU2ZZxidafN/6UQCzwDuQHTMnXWWa1GhAH+nE1Y2pzDQtNmcXSyj/ESR/43UOqIuy/KEpRurR0saDcIwtXoRoX+4U0gDhHLS8tAgFLQWgn3OoDXchoi8FRe0DS9vYvMmiuEYJD2n2UHa+cPIwvm4ej1fYVeooYTTMHT+FA80RrmTDbtp9qndXz/x5wsJieL1GMedpuw65fsbKz5Hu0m2lpQKgkq6iPgBoykYgF89BLUDsREEfpFT4CUzrzhXWkhbkMElUrEj6UGNmURIiloKfkNTgbwgcdKqFhcDzCgrjfzrJVttb8XpJZYknSOW1uCY8kKCQuND1Pl11VpKq2oo//La3qIyy8pawmmbXEgH+lId+jCsiWmWNRcInShyI+SWlTyHal3W3yc8jhzUYXJHsYSo/OmqNBKF2i6E6VG6pwwemnQlPXZ5Vh3txl9qsXZVbZEokvBszBdoCdY2YDqYmzHgLSQ/4jd/zgSfEECpjq8eRUhX4T2dve9ejScgmpqJd3vkJPp5n8WnUfQqIxFc0Bsm0irzFAkNnskGJ6i/rtS2IadSot6WuV2Fyh/ousTkupg08Ck5YiDL4N19jJOYnU1+PS
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6325.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(366004)(136003)(396003)(376002)(122000001)(316002)(4326008)(6512007)(83380400001)(38100700002)(86362001)(8936002)(53546011)(76116006)(6506007)(31686004)(36756003)(66946007)(91956017)(66446008)(66556008)(38070700005)(2616005)(66476007)(64756008)(5660300002)(2906002)(8676002)(6486002)(478600001)(54906003)(71200400001)(186003)(31696002)(26005)(110136005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?VVdYY1FiTmxmTFJRWWM2cWhFQXVJOHNHdlFyajBZS3pML292V0g4Z3NEVG9B?=
 =?utf-8?B?MDJhV3JMTHhJUjRPK0I5YTlIZEFmRzRtMG5rQ3FIRThkMzVOMjlIbVMrVVNH?=
 =?utf-8?B?VVoxSjc0b0k1dmszRkFmNCtiQmtlTkFJV3RHMm9rYTRnZUdDLzFUVEdKMTBH?=
 =?utf-8?B?N1B2SGg3bS81VS8razN4d0JUeW9vcGhaaUxQUjJ6WVZlbUdtKzJaeEoxRXpv?=
 =?utf-8?B?aFdTSmtxQmJ5MFQ3MzFrWnUxTUwzUWFRWHJEa1V2RmdOUkdHL3hOTHNmdmMx?=
 =?utf-8?B?cktWVTZtV2l6eFpwaXlEY1RzbzVYOHJMWUF2THFIc1JaUWFWQTRZWjJ0V0No?=
 =?utf-8?B?QlVCbWw5RlBBU3VVMHBlWEZIbmJVQkx4cnF2T2ZwRCtZU01KWlJ1T3huYWFE?=
 =?utf-8?B?amJVMkVGeUJoT045OE5tZkdrK3R3RjFRTDVMQVNvTERDeUs1bmxjaHlKM0xy?=
 =?utf-8?B?TEg4VFZzdXJFSVJmS2hnQ0lvdUhYS29Ram5ESkdMdnVDTE5ZbmliR1ZiMmY0?=
 =?utf-8?B?QVF6WkV0T2Z3eUo2OHBibXVSMTdZNDBFV2ZOWHk5b0JPWjJtOGRwdmFnUldj?=
 =?utf-8?B?R2srbTR4SEE0am5JUnA5UXFMYmUzaXBMTFhYM1pUNTRSUXk3NjNXMlpROHox?=
 =?utf-8?B?ZFk4b0NwTkFPdkVaZXBGNmwxYWJMTkM5eEhCWEIra05ocmhYMVpENjJ1NnJw?=
 =?utf-8?B?UXJsQXkwSkxqVUFoL3p1Z1VnSjhhd2hFeS80WDI3OVRxTWxSbVhpRG1sL29s?=
 =?utf-8?B?bnBYQU1QV2hvVFJKbVFvQWR6RlRBbmJmT1NjNjk5RkFiZStySFNCdWdDTkVH?=
 =?utf-8?B?aG1DSUZJc2x4TGJzVk9TTS9uRnQ3Nk1FQTJ4NTVYcWF1VVJ4ZWE4LzhNMytG?=
 =?utf-8?B?ZW0ybWZ0dy9Icmx5S0tkOEVuNitMdHlsNVV3Mit6Z2pvZ1k2alp2QkVtWmhP?=
 =?utf-8?B?dUxMT2dPdGZzelNlbXhZdGtrbHBlZlZRU2puakxvaUZ4blJkZE1LeGxXN1px?=
 =?utf-8?B?K0tPUmlGa3FiMFM1VEJ1aU94eE9VYnROY3pDbzN1OStORHpxak95SUhPbmJR?=
 =?utf-8?B?UCtzc2dtMGNtakdIMnFyck9Ddno0SDJ1WnNvdXhRNFc5c2dzYjJscU5VVFV1?=
 =?utf-8?B?Rm5pVFoxV0VPL0JuTFBTL0dsTCtZNjF4WnJ5WXdCWm1VS0dpMTBUVE1PR2lq?=
 =?utf-8?B?UWFjOElMMXRmbXQ1WVJxcEFNdk9wTit1KzBnZTRIZ1NMWGlUbEkxZTVLbTlh?=
 =?utf-8?B?VWtzMEt2TEtqRG1zSkhmNjY5Y1lLM0ZyNlJQWlh4WFF2dnY2UlZHalZ2bVF4?=
 =?utf-8?B?OFJYMDlPbU5Jc0ZpWUFraW9WZXhjNjNTcEUzVWYzNUFZZ0JpbXdEMCtVaFM0?=
 =?utf-8?B?c1g5ZC9nUFJCLzVuVlhMa0Q4L3BLemQ4VkljdWpoT2xIdlFSa0Z4S2xZaDFz?=
 =?utf-8?B?Wm5ncnA5c1A4QTZTK1AvcEpQL2NxZmhINUhtTnlpWlRTSFV3RFY1ZXc5MnNa?=
 =?utf-8?B?WFBzV1ltT1NWT2x4MVQ4T0dYWDc3M1VmSS9MTGtkbm5STldIOVpYeXBoV1Bz?=
 =?utf-8?B?WVFjenl0L0ZSaFlKaEdMQTg0SExQZ0dsRjF4WkpMUlV4SW94cDAxVVpMWmFq?=
 =?utf-8?B?Y1QrU2M0RjJ4WkZvcjNoSE9TR2d4K0NjWEg1LzBMU0lac0RRelp1bUNOckds?=
 =?utf-8?B?eTdGSHdPRnNWMUpOdW9QRytTVlhnMEV2bVlHeTlHbkpXQlFvMG45ZTUxc1Uz?=
 =?utf-8?Q?3U8VLTfm3zyM9ai5cT+4IR2bRjDPtqU3v5MF/HT?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <995FE7DEF132A749BF2A3236800428AA@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6325.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2af00d5c-e119-42a4-0ea1-08d972042545
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 13:33:57.9268
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tzsWDOHCWVIzVzS/8f8/vmIcxQ6idB4UBV1AVB4paQ9QcIPy0epgWWCr/AXIz+VrgJCTyJIEyKMM/fupY5A3LplkStKtx4jN93t2IcfuzAumZNZvaNIhq53Q+LqMqjS2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB7159
X-Proofpoint-ORIG-GUID: 91UDs_AjfZv1rEiZ5LKmJI1LQgo0nJbR
X-Proofpoint-GUID: 91UDs_AjfZv1rEiZ5LKmJI1LQgo0nJbR
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_04,2021-09-07_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0
 bulkscore=0 malwarescore=0 clxscore=1015 phishscore=0 mlxlogscore=999
 adultscore=0 priorityscore=1501 impostorscore=0 spamscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2108310000 definitions=main-2109070089

DQpPbiAwNi4wOS4yMSAxNzozMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEveGVuL2RyaXZl
cnMvdnBjaS9oZWFkZXIuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4g
QEAgLTQwMCwxMiArNDAwLDcyIEBAIHN0YXRpYyB2b2lkIGJhcl93cml0ZShjb25zdCBzdHJ1Y3Qg
cGNpX2RldiAqcGRldiwgdW5zaWduZWQgaW50IHJlZywNCj4+ICAgc3RhdGljIHZvaWQgZ3Vlc3Rf
YmFyX3dyaXRlKGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVnLA0K
Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgdmFsLCB2b2lkICpkYXRh
KQ0KPj4gICB7DQo+PiArICAgIHN0cnVjdCB2cGNpX2JhciAqYmFyID0gZGF0YTsNCj4+ICsgICAg
Ym9vbCBoaSA9IGZhbHNlOw0KPj4gKw0KPj4gKyAgICBpZiAoIGJhci0+dHlwZSA9PSBWUENJX0JB
Ul9NRU02NF9ISSApDQo+PiArICAgIHsNCj4+ICsgICAgICAgIEFTU0VSVChyZWcgPiBQQ0lfQkFT
RV9BRERSRVNTXzApOw0KPj4gKyAgICAgICAgYmFyLS07DQo+PiArICAgICAgICBoaSA9IHRydWU7
DQo+PiArICAgIH0NCj4+ICsgICAgZWxzZQ0KPj4gKyAgICAgICAgdmFsICY9IFBDSV9CQVNFX0FE
RFJFU1NfTUVNX01BU0s7DQo+PiArICAgIGJhci0+Z3Vlc3RfYWRkciAmPSB+KDB4ZmZmZmZmZmZ1
bGwgPDwgKGhpID8gMzIgOiAwKSk7DQo+PiArICAgIGJhci0+Z3Vlc3RfYWRkciB8PSAodWludDY0
X3QpdmFsIDw8IChoaSA/IDMyIDogMCk7DQo+IFdoYXQgeW91IHN0b3JlIGhlcmUgaXMgbm90IHRo
ZSBhZGRyZXNzIHRoYXQncyBnb2luZyB0byBiZSB1c2VkLA0KDQpiYXItPmd1ZXN0X2FkZHIgaXMg
bmV2ZXIgdXNlZCBkaXJlY3RseSB0byBiZSByZXBvcnRlZCB0byBhIGd1ZXN0Lg0KDQpUaGUgc2Ft
ZSBhcyBiYXItPmFkZHIgaXMgbmV2ZXIgdXNlZCB0byB3cml0ZSB0byByZWFsIEJBUi4NCg0KSXQg
aXMgYWx3YXlzIHVzZWQgYXMgYW4gaW5pdGlhbCB2YWx1ZSB3aGljaCBpcyB0aGVuIG1vZGlmaWVk
IHRvIHJlZmxlY3QNCg0KbG93ZXIgYml0cywgZS5nLiBCQVIgdHlwZSBhbmQgaWYgcHJlZmV0Y2hh
YmxlLCBzbyBJIHRoaW5rIHRoaXMgaXMgcGVyZmVjdGx5DQoNCmZpbmUgdG8gaGF2ZSBpdCB0aGlz
IHdheS4NCg0KPiAgIGFzDQo+IHlvdSBkb24ndCBtYXNrIG9mZiB0aGUgbG93IGJpdHMgKHRvIGFj
Y291bnQgZm9yIHRoZSBCQVIncyBzaXplKS4NCj4gV2hlbiBhIEJBUiBnZXRzIHdyaXR0ZW4gd2l0
aCBhbGwgb25lcywgYWxsIHdyaXRhYmxlIGJpdHMgZ2V0IHRoZXNlDQo+IG9uZXMgc3RvcmVkLiBU
aGUgYWRkcmVzcyBvZiB0aGUgQkFSLCBhaXVpLCByZWFsbHkgY2hhbmdlcyB0bw0KPiAodHlwaWNh
bGx5KSBjbG9zZSBiZWxvdyA0R2IgKGluIHRoZSBjYXNlIG9mIGEgMzItYml0IEJBUiksIHdoaWNo
DQo+IGlzIHdoeSBtZW1vcnkgLyBJL08gZGVjb2Rpbmcgc2hvdWxkIGJlIG9mZiB3aGlsZSBzaXpp
bmcgQkFScy4NCj4gVGhlcmVmb3JlIHlvdSBzaG91bGRuJ3QgbG9vayBmb3IgdGhlIHNwZWNpZmlj
ICJhbGwgd3JpdGFibGUgYml0cw0KPiBhcmUgb25lcyIgcGF0dGVybiAob3Igd29yc2UsIGFzIHlv
dSBwcmVzZW50bHkgZG8sIHRoZSAiYWxsIGJpdHMNCj4gb3V0c2lkZSBvZiB0aGUgdHlwZSBzcGVj
aWZpZXIgYXJlIG9uZXMiIG9uZSkgb24gdGhlIHJlYWQgcGF0aC4NCj4gSW5zdGVhZCBtYXNrIHRo
ZSB2YWx1ZSBhcHByb3ByaWF0ZWx5IGhlcmUsIGFuZCBzaW1wbHkgcmV0dXJuIGJhY2sNCj4gdGhl
IHN0b3JlZCB2YWx1ZSBmcm9tIHRoZSByZWFkIHBhdGguDQoiUENJIExPQ0FMIEJVUyBTUEVDSUZJ
Q0FUSU9OLCBSRVYuIDMuMCIsICJJTVBMRU1FTlRBVElPTiBOT1RFDQoNClNpemluZyBhIDMyLWJp
dCBCYXNlIEFkZHJlc3MgUmVnaXN0ZXIgRXhhbXBsZSIgc2F5cywgdGhhdA0KDQoiU29mdHdhcmUg
c2F2ZXMgdGhlIG9yaWdpbmFsIHZhbHVlIG9mIHRoZSBCYXNlIEFkZHJlc3MgcmVnaXN0ZXIsIHdy
aXRlcw0KMCBGRkZGIEZGRkZoIHRvIHRoZSByZWdpc3RlciwgdGhlbiByZWFkcyBpdCBiYWNrLiIN
Cg0KVGhlIHNhbWUgYXBwbGllcyBmb3IgNjQtYml0IEJBUnMuIFNvIHdoYXQncyB3cm9uZyBpZiBJ
IHRyeSB0byBjYXRjaCBzdWNoDQoNCmEgd3JpdGUgd2hlbiBhIGd1ZXN0IHRyaWVzIHRvIHNpemUg
dGhlIEJBUj8gVGhlIG9ubHkgZGlmZmVyZW5jZSBpcyB0aGF0DQoNCkkgY29tcGFyZSBhcw0KDQog
wqDCoMKgwqDCoMKgwqAgaWYgKCAodmFsICYgUENJX0JBU0VfQUREUkVTU19NRU1fTUFTS18zMikg
PT0gUENJX0JBU0VfQUREUkVTU19NRU1fTUFTS18zMiApDQp3aGljaCBpcyBiZWNhdXNlIHZhbCBp
biB0aGUgcXVlc3Rpb24gaGFzIGxvd2VyIGJpdHMgY2xlYXJlZC4NCg0KV2l0aCB0aGF0IHJlc3Bl
Y3QgSSBzZWUgbm8gb2J2aW91cyByZWFzb24gd2h5IHdlIGNhbid0IGNvbnN0cnVjdCBvdXIgY29k
ZQ0KDQphcyBpdCBpcy4NCg0KPg0KPj4gICB9DQo+PiAgIA0KPj4gICBzdGF0aWMgdWludDMyX3Qg
Z3Vlc3RfYmFyX3JlYWQoY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYsIHVuc2lnbmVkIGludCBy
ZWcsDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2b2lkICpkYXRhKQ0KPj4g
ICB7DQo+PiAtICAgIHJldHVybiAweGZmZmZmZmZmOw0KPj4gKyAgICBzdHJ1Y3QgdnBjaV9iYXIg
KmJhciA9IGRhdGE7DQo+PiArICAgIHVpbnQzMl90IHZhbDsNCj4+ICsgICAgYm9vbCBoaSA9IGZh
bHNlOw0KPj4gKw0KPj4gKyAgICBzd2l0Y2ggKCBiYXItPnR5cGUgKQ0KPj4gKyAgICB7DQo+PiAr
ICAgIGNhc2UgVlBDSV9CQVJfTUVNNjRfSEk6DQo+PiArICAgICAgICBBU1NFUlQocmVnID4gUENJ
X0JBU0VfQUREUkVTU18wKTsNCj4+ICsgICAgICAgIGJhci0tOw0KPj4gKyAgICAgICAgaGkgPSB0
cnVlOw0KPj4gKyAgICAgICAgLyogZmFsbHRocm91Z2ggKi8NCj4+ICsgICAgY2FzZSBWUENJX0JB
Ul9NRU02NF9MTzoNCj4+ICsgICAgew0KPiBQbGVhc2UgZG9uJ3QgYWRkIGJyYWNlcyB0byBjYXNl
IGJsb2NrcyB3aGVuIHRoZXkncmUgbm90IG5lZWRlZC4NClN1cmUNCj4NCj4+ICsgICAgICAgIGlm
ICggaGkgKQ0KPj4gKyAgICAgICAgICAgIHZhbCA9IGJhci0+Z3Vlc3RfYWRkciA+PiAzMjsNCj4+
ICsgICAgICAgIGVsc2UNCj4+ICsgICAgICAgICAgICB2YWwgPSBiYXItPmd1ZXN0X2FkZHIgJiAw
eGZmZmZmZmZmOw0KPj4gKyAgICAgICAgaWYgKCAodmFsICYgUENJX0JBU0VfQUREUkVTU19NRU1f
TUFTS18zMikgPT0gIFBDSV9CQVNFX0FERFJFU1NfTUVNX01BU0tfMzIgKQ0KPiBUaGlzIGlzIHdy
b25nIHdoZW4gZmFsbGluZyB0aHJvdWdoIHRvIGhlcmUgZnJvbSBWUENJX0JBUl9NRU02NF9ISToN
Cj4gQWxsIDMyIGJpdHMgbmVlZCB0byBiZSBsb29rZWQgYXQuDQpHb29kIGNhdGNoLCB3aWxsIGZp
eA0KPiAgIFlldCBhcyBwZXIgdGhlIGNvbW1lbnQgZnVydGhlcg0KPiB1cCBJIHRoaW5rIGl0IGlz
bid0IHJpZ2h0IGFueXdheSB0byBhcHBseSB0aGUgbWFzayBoZXJlLg0KPg0KPiBBbHNvOiBTdHJh
eSBkb3VibGUgYmxhbmtzLg0KPg0KPj4gKyAgICAgICAgew0KPj4gKyAgICAgICAgICAgIC8qIEd1
ZXN0cyBkZXRlY3RzIEJBUidzIHByb3BlcnRpZXMgYW5kIHNpemVzLiAqLw0KPj4gKyAgICAgICAg
ICAgIGlmICggaGkgKQ0KPj4gKyAgICAgICAgICAgICAgICB2YWwgPSBiYXItPnNpemUgPj4gMzI7
DQo+PiArICAgICAgICAgICAgZWxzZQ0KPj4gKyAgICAgICAgICAgICAgICB2YWwgPSAweGZmZmZm
ZmZmICYgfihiYXItPnNpemUgLSAxKTsNCj4+ICsgICAgICAgIH0NCj4+ICsgICAgICAgIGlmICgg
IWhpICkNCj4+ICsgICAgICAgIHsNCj4+ICsgICAgICAgICAgICB2YWwgfD0gUENJX0JBU0VfQURE
UkVTU19NRU1fVFlQRV82NDsNCj4+ICsgICAgICAgICAgICB2YWwgfD0gYmFyLT5wcmVmZXRjaGFi
bGUgPyBQQ0lfQkFTRV9BRERSRVNTX01FTV9QUkVGRVRDSCA6IDA7DQo+PiArICAgICAgICB9DQo+
PiArICAgICAgICBiYXItPmd1ZXN0X2FkZHIgJj0gfigweGZmZmZmZmZmdWxsIDw8IChoaSA/IDMy
IDogMCkpOw0KPj4gKyAgICAgICAgYmFyLT5ndWVzdF9hZGRyIHw9ICh1aW50NjRfdCl2YWwgPDwg
KGhpID8gMzIgOiAwKTsNCj4+ICsgICAgICAgIGJyZWFrOw0KPj4gKyAgICB9DQo+PiArICAgIGNh
c2UgVlBDSV9CQVJfTUVNMzI6DQo+IFBsZWFzZSBzZXBhcmF0ZSBub24tZmFsbC10aHJvdWdoIGNh
c2UgYmxvY2tzIGJ5IGEgYmxhbmsgbGluZS4NCldpbGwgZG8NCj4NCj4+IEBAIC01MjIsNiArNTgy
LDEzIEBAIHN0YXRpYyBpbnQgYWRkX2Jhcl9oYW5kbGVycyhzdHJ1Y3QgcGNpX2RldiAqcGRldiwg
Ym9vbCBpc19od2RvbSkNCj4+ICAgICAgICAgICAgICAgaWYgKCByYyApDQo+PiAgICAgICAgICAg
ICAgICAgICByZXR1cm4gcmM7DQo+PiAgICAgICAgICAgfQ0KPj4gKyAgICAgICAgLyoNCj4+ICsg
ICAgICAgICAqIEl0IGlzIG5laXRoZXIgc2FmZSBub3Igc2VjdXJlIHRvIGluaXRpYWxpemUgZ3Vl
c3QncyB2aWV3IG9mIHRoZSBCQVJzDQo+PiArICAgICAgICAgKiB3aXRoIHJlYWwgdmFsdWVzIHdo
aWNoIGFyZSB1c2VkIGJ5IHRoZSBoYXJkd2FyZSBkb21haW4sIHNvIGFzc2lnbg0KPj4gKyAgICAg
ICAgICogYWxsIHplcm9zIHRvIGd1ZXN0J3MgdmlldyBvZiB0aGUgQkFScywgc28gdGhlIGd1ZXN0
IGNhbiBwZXJmb3JtDQo+PiArICAgICAgICAgKiBwcm9wZXIgUENJIGRldmljZSBlbnVtZXJhdGlv
biBhbmQgYXNzaWduIEJBUnMgb24gaXRzIG93bi4NCj4+ICsgICAgICAgICAqLw0KPj4gKyAgICAg
ICAgYmFyc1tpXS5ndWVzdF9hZGRyID0gMDsNCj4gSSdtIGFmcmFpZCBJIGRvbid0IHVuZGVyc3Rh
bmQgdGhlIGNvbW1lbnQ6IFdpdGhvdXQgbWVtb3J5IGRlY29kaW5nDQo+IGVuYWJsZWQsIHRoZSBC
QVJzIGFyZSBzaW1wbGUgcmVnaXN0ZXJzICh3aXRoIGEgZmV3IHIvbyBiaXRzKS4NCg0KTXkgZmly
c3QgaW1wbGVtZW50YXRpb24gd2FzIHRoYXQgYmFyLT5ndWVzdF9hZGRyIHdhcyBpbml0aWFsaXpl
ZCB3aXRoDQoNCnRoZSB2YWx1ZSBvZiBiYXItPmFkZHIgKHBoeXNpY2FsIEJBUiB2YWx1ZSksIGJ1
dCB0YWxraW5nIG9uIElSQyB3aXRoDQoNClJvZ2VyIGhlIHN1Z2dlc3RlZCB0aGF0IHRoaXMgbWln
aHQgYmUgYSBzZWN1cml0eSBpc3N1ZSB0byBsZXQgZ3Vlc3QNCg0KYSBoaW50IGFib3V0IHBoeXNp
Y2FsIHZhbHVlcywgc28gdGhlbiBJIGNoYW5nZWQgdGhlIGFzc2lnbm1lbnQgdG8gYmUgMC4NCg0K
VGh1cyB0aGUgY29tbWVudA0KDQo+DQo+PiAtLS0gYS94ZW4vaW5jbHVkZS94ZW4vcGNpX3JlZ3Mu
aA0KPj4gKysrIGIveGVuL2luY2x1ZGUveGVuL3BjaV9yZWdzLmgNCj4+IEBAIC0xMDMsNiArMTAz
LDcgQEANCj4+ICAgI2RlZmluZSAgUENJX0JBU0VfQUREUkVTU19NRU1fVFlQRV82NAkweDA0CS8q
IDY0IGJpdCBhZGRyZXNzICovDQo+PiAgICNkZWZpbmUgIFBDSV9CQVNFX0FERFJFU1NfTUVNX1BS
RUZFVENICTB4MDgJLyogcHJlZmV0Y2hhYmxlPyAqLw0KPj4gICAjZGVmaW5lICBQQ0lfQkFTRV9B
RERSRVNTX01FTV9NQVNLCSh+MHgwZlVMKQ0KPj4gKyNkZWZpbmUgIFBDSV9CQVNFX0FERFJFU1Nf
TUVNX01BU0tfMzIJKH4weDBmVSkNCj4gUGxlYXNlIGRvbid0IGludHJvZHVjZSBhbiBpZGVudGlj
YWwgY29uc3RhbnQgdGhhdCdzIG1lcmVseSBvZg0KPiBkaWZmZXJlbnQgdHlwZS4gKHVpbnQzMl90
KVBDSV9CQVNFX0FERFJFU1NfTUVNX01BU0sgYXQgdGhlIHVzZQ0KPiBzaXRlIChpZiBhY3R1YWxs
eSBzdGlsbCBuZWVkZWQgYXMgcGVyIHRoZSBjb21tZW50IGFib3ZlKSB3b3VsZA0KPiBzZWVtIG1v
cmUgY2xlYXIgdG8gbWUuDQpPaywgSSB0aG91Z2h0IHR5cGUgY2FzdGluZyBpcyBhIGJpZ2dlciBl
dmlsIGhlcmUNCj4NCj4gSmFuDQoNClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:42:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:42:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181055.328023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbNE-0004Ym-Nu; Tue, 07 Sep 2021 13:42:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181055.328023; Tue, 07 Sep 2021 13:42:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbNE-0004Yf-Kr; Tue, 07 Sep 2021 13:42:52 +0000
Received: by outflank-mailman (input) for mailman id 181055;
 Tue, 07 Sep 2021 13:42:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNbND-0004YZ-BJ
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:42:51 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7ddadf84-0fe1-11ec-b0f3-12813bfff9fa;
 Tue, 07 Sep 2021 13:42:50 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631022164109295.00066750460735;
 Tue, 7 Sep 2021 06:42:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ddadf84-0fe1-11ec-b0f3-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631022167; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=EwXVjk7o0+dNW6BQyzIDtRSYN4NmBvRZLsUpPEsZNhVlT1DAgbWz+pViSNQTi4ASsluqS88rEakdUxr/EHLYUZwaYP9AtnT5YOtWsyMGFti+D9i2cLq4GKOJrkIn/cqKXBVHgjXefaKQ6sV2LKITuag2vkUo0cDMfne5LpeadHY=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631022167; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=CYEF02zRg0djSWApjMYgBANnz2rEyed+agc2MJGNgJ4=; 
	b=Wlgl57FKbX4InW3weDbgq8QTjou/oRB0zBjMlZPmUWuENbYe3kgGJAavDsERwnnOCe1Qu8Wa30dKo6LQnJ7gh/nDQH5ri7Y6D4NkEz677pYErByE/UNjXalqhikxcedvuP4uBIkCyZ5aTOA6W2X0EmQC8Wyj9o7G2VK/u3ZIsa0=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631022167;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=CYEF02zRg0djSWApjMYgBANnz2rEyed+agc2MJGNgJ4=;
	b=Z0vhHWYhejqxTL3qqWAPWN6v0Hzj/xcmLsISh4Y0oLXgwrIhTYN6OU10zLEAuYf4
	V0Eah+42XDGWEwRECKo40MbBD0haeKS6lC82QUm6BJkZ5gfXykXvsond1LfSezoUjul
	x7SQp7BOFUeE9Bf18jf0d6rEySnNg+9SUzORHZxI=
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
Date: Tue, 7 Sep 2021 09:41:00 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/6/21 2:17 PM, Andrew Cooper wrote:
> On 03/09/2021 20:06, Daniel P. Smith wrote:
>> Instead of intermixing coding style changes with code changes as they
>> are come upon in this patch set, moving all coding style changes
>> into a single commit. The focus of coding style changes here are,
>>
>>   - move trailing comments to line above
>>   - ensuring line length does not exceed 80 chars
>>   - ensuring proper indentation for 80 char wrapping
>>   - covert u32 type statements to  uint32_t
>>   - remove space between closing and opening parens
>>   - drop extern on function declarations
>>
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> ---
>>   xen/include/xsm/dummy.h | 173 +++++++++-----
>>   xen/include/xsm/xsm.h   | 494 ++++++++++++++++++++++------------------
>>   xen/xsm/xsm_core.c      |   4 +-
>>   3 files changed, 389 insertions(+), 282 deletions(-)
>>
>> diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
>> index 214b5408b1..deaf23035e 100644
>> --- a/xen/include/xsm/dummy.h
>> +++ b/xen/include/xsm/dummy.h
>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>   
>>   #endif /* CONFIG_XSM */
>>   
>> -static always_inline int xsm_default_action(
>> -    xsm_default_t action, struct domain *src, struct domain *target)
>> +static always_inline int xsm_default_action(xsm_default_t action,
>> +                                            struct domain *src,
>> +                                            struct domain *target)
> 
> The old code is correct.  We have plenty of examples of this in Xen, and
> I have been adding new ones when appropriate.
> 
> It avoids squashing everything on the RHS and ballooning the line count
> to compensate.  (This isn't a particularly bad example, but we've had
> worse cases in the past).

Based on the past discussions I understood either is acceptable and find 
this version much easier to visually parse myself. With that said, if
the "next line single indent" really is the preferred style by the 
maintainers/community, then I can convert all of these over.

>>   {
>>       switch ( action ) {
>>       case XSM_HOOK:
>> @@ -99,12 +100,13 @@ static always_inline int xsm_default_action(
>>   }
>>   
>>   static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
>> -                                    struct xen_domctl_getdomaininfo *info)
>> +    struct xen_domctl_getdomaininfo *info)
> 
> This doesn't match any styles I'm aware of.  Either struct domain on the
> new line, or the two structs vertically aligned.
> 
> It more obviously highlights why squashing all parameters on the RHS is
> a bad move.

Apologies I let one slip through, though going through over 80-some 
function defs trying to make sure they are all aligned and missing one
I would say is not a bad job.

>> @@ -291,37 +307,41 @@ static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
>>       return;
>>   }
>>   
>> -static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
>> +static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d,
>> +                                      struct evtchn *chn)
>>   {
>>       XSM_ASSERT_ACTION(XSM_HOOK);
>>       return xsm_default_action(action, d, NULL);
>>   }
>>   
>> -static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
>> +static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d,
>> +                                        struct evtchn *chn)
>>   {
>>       XSM_ASSERT_ACTION(XSM_TARGET);
>>       return xsm_default_action(action, current->domain, d);
>>   }
>>   
>> -static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
>> +static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1,
>> +                                       struct domain *d2)
>>   {
>>       XSM_ASSERT_ACTION(XSM_TARGET);
>>       return xsm_default_action(action, d1, d2);
>>   }
>>   
>> -static XSM_INLINE int xsm_alloc_security_evtchns(
>> -    struct evtchn chn[], unsigned int nr)
>> +static XSM_INLINE int xsm_alloc_security_evtchns(struct evtchn chn[],
>> +                                                 unsigned int nr)
> 
> I maintain that this was correct before.

Getting to this point I must say it would be helpful if this could be 
spelled out in CODING_STYLE. Specifically, so that I am clear, if a 
parameter overflows, than all the parameters overflow? Are there 
exceptions such as if overflow doesn't happen until the third of four or 
the fourth parameter. Having a rule set would be much more helpful than 
trying to look for examples elsewhere in code.

>> diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
>> index 9872bae502..8878281eae 100644
>> --- a/xen/include/xsm/xsm.h
>> +++ b/xen/include/xsm/xsm.h
>> @@ -19,7 +19,7 @@
>>   #include <xen/multiboot.h>
>>   
>>   /* policy magic number (defined by XSM_MAGIC) */
>> -typedef u32 xsm_magic_t;
>> +typedef uint32_t xsm_magic_t;
>>   
>>   #ifdef CONFIG_XSM_FLASK
>>   #define XSM_MAGIC 0xf97cff8c
>> @@ -31,158 +31,171 @@ typedef u32 xsm_magic_t;
>>    * default actions of XSM hooks. They should be compiled out otherwise.
>>    */
>>   enum xsm_default {
>> -    XSM_HOOK,     /* Guests can normally access the hypercall */
>> -    XSM_DM_PRIV,  /* Device model can perform on its target domain */
>> -    XSM_TARGET,   /* Can perform on self or your target domain */
>> -    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
>> -    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
>> -    XSM_OTHER     /* Something more complex */
>> +    /* Guests can normally access the hypercall */
>> +    XSM_HOOK,
>> +    /* Device model can perform on its target domain */
>> +    XSM_DM_PRIV,
>> +    /* Can perform on self or your target domain */
>> +    XSM_TARGET,
>> +    /* Privileged - normally restricted to dom0 */
>> +    XSM_PRIV,
>> +    /* Xenstore domain - can do some privileged operations */
>> +    XSM_XS_PRIV,
>> +    /* Something more complex */
>> +    XSM_OTHER
>>   };
> 
> Why?  This takes a table which was unambiguous to read, and makes it
> ambiguous at a glance.  You want either no change at all, or blank lines
> between comment/constant pairs so you don't need to read to either end
> to figure out how to parse the comments.

I went back to the comment that prompted me to do this and rereading now 
makes me think I took it to literal. Specifically, "...I'd like to 
encourage you to also address other style issues in the newly introduced 
file. Here I'm talking about comment style, requiring /* to be on its 
own line." I took that to include these as well since I am pretty sure I 
have seen elsewhere this kind of commenting. Regardless, looking back I 
  can see how the meaning could have only for other block quotes. 
Honestly I will gladly change it back as I think that is far clearer.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:46:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:46:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181062.328035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbQn-0005Cc-9s; Tue, 07 Sep 2021 13:46:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181062.328035; Tue, 07 Sep 2021 13:46:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbQn-0005CV-5U; Tue, 07 Sep 2021 13:46:33 +0000
Received: by outflank-mailman (input) for mailman id 181062;
 Tue, 07 Sep 2021 13:46:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNbQl-0005CP-Ej
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:46:31 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0136ce42-0fe2-11ec-b0f3-12813bfff9fa;
 Tue, 07 Sep 2021 13:46:30 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631022384849480.93473938892373;
 Tue, 7 Sep 2021 06:46:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0136ce42-0fe2-11ec-b0f3-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631022387; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=RXKB9dUWNPWkgmcdISqdTGb0F5yGEg6QznZcElAc8HE42k1NyujAz5oBU6OovQgwQHARHCz1ofQqU+IDSGJDUR9L90zXX/zhh3GHuBjkJv4Z5ujZbcbiSl3LAxcPuM/FcJAwgpCEYaelGBInQUJ7x+B0BuF5XtfZGQO+h2T25gc=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631022387; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=Eg118G40UsDAktVHH5I+xA97QrEkwUrl+xfBDFWQE6I=; 
	b=Z/GnHDIBMRsR+uzsWuAQhp7XNaMctaRpx2vpKOX0lPC4/w6lDla52HKkeR2kmJnQ4STfc0m9DaRxFOLhRUVNx9abrfflmwIEa6fBbQvZrKyxW8xVSpsKG1CF59Z1K2fdVJ89wf5WIC+CbhDN6XbjY1U/dLrrZ4Pz2UO2rLQaclM=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631022387;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=Eg118G40UsDAktVHH5I+xA97QrEkwUrl+xfBDFWQE6I=;
	b=YwBx7uc3aQS5R3OnGe61qLhP5pMEjDZFNJN1vPoWD321v0l/dqoUZI94l0Ppnkvi
	B9yGK94uo3rzfafP3yR33cMU2tRnOLIbM2Eyf68n7Az4lx3HqmoGmttzAzXjcspBpQv
	brPwbd4MwL5SqFT6C34EZKzKoQuBBkWPSVIYsLRE=
Subject: Re: [PATCH v4 05/11] xsm: refactor xsm_ops handling
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-6-dpsmith@apertussolutions.com>
 <6924c3ea-de14-60e5-d3dc-eaff6c228f1c@citrix.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <b5a2ff19-b7f5-cf2e-4c37-1663c69727c3@apertussolutions.com>
Date: Tue, 7 Sep 2021 09:44:41 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <6924c3ea-de14-60e5-d3dc-eaff6c228f1c@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External



On 9/6/21 2:31 PM, Andrew Cooper wrote:
> On 03/09/2021 20:06, Daniel P. Smith wrote:
>> This renames the `struct xsm_operations` to the shorter `struct xsm_ops` and
>> converts the global xsm_ops from being a pointer to an explicit instance. As
>> part of this conversion, it reworks the XSM modules init function to return
>> their xsm_ops struct which is copied in to the global xsm_ops instance.
>>
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> 
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> However, some suggestions...
> 
>> diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
>> index 55483292c5..859af3fe9a 100644
>> --- a/xen/xsm/xsm_core.c
>> +++ b/xen/xsm/xsm_core.c
>> @@ -28,9 +28,17 @@
>>   #include <asm/setup.h>
>>   #endif
>>   
>> -#define XSM_FRAMEWORK_VERSION    "1.0.0"
>> +#define XSM_FRAMEWORK_VERSION    "1.0.1"
>>   
>> -struct xsm_operations *xsm_ops;
>> +struct xsm_ops __read_mostly xsm_ops;
>> +
>> +enum xsm_ops_state {
>> +    XSM_OPS_UNREGISTERED,
>> +    XSM_OPS_REG_FAILED,
>> +    XSM_OPS_REGISTERED,
>> +};
>> +
>> +static enum xsm_ops_state xsm_ops_registered = XSM_OPS_UNREGISTERED;
> 
> __read_mostly, or can this be __initdata ?

Apologies, I think you may have suggested this before. It would be good 
to be able to check this later but currently since I just introduced 
this and it is only used during init, it could be made __initdata for 
now and later if it gets exposed, then it can be moved to __read_mostly.

Do you agree?

>> @@ -87,25 +88,35 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
>>       }
>>   #endif
>>   
>> -    if ( verify(&dummy_xsm_ops) )
>> +    if ( xsm_ops_registered != XSM_OPS_UNREGISTERED )
>>       {
>> -        printk(XENLOG_ERR "Could not verify dummy_xsm_ops structure\n");
>> +        printk(XENLOG_ERR
>> +               "Could not init XSM, xsm_ops register already attempted\n");
>>           return -EIO;
>>       }
>>   
>> -    xsm_ops = &dummy_xsm_ops;
>> -
>>       switch ( xsm_bootparam )
>>       {
>>       case XSM_BOOTPARAM_DUMMY:
>> +        xsm_ops_registered = XSM_OPS_REGISTERED;
>>           break;
>>   
>>       case XSM_BOOTPARAM_FLASK:
>> -        flask_init(policy_buffer, policy_size);
>> +        ops = flask_init(policy_buffer, policy_size);
>> +        if ( ops )
>> +        {
>> +            xsm_ops_registered = XSM_OPS_REGISTERED;
>> +            xsm_ops = *ops;
>> +        }
>>           break;
>>   
>>       case XSM_BOOTPARAM_SILO:
>> -        silo_init();
>> +        ops = silo_init();
>> +        if ( ops )
>> +        {
>> +            xsm_ops_registered = XSM_OPS_REGISTERED;
>> +            xsm_ops = *ops;
>> +        }
> 
> This if ( ops ) block can be deduplicated by moving after the switch()
> statement.  It's going to be common to all everything except dummy.

Good call. I will rework to remove the duplication.

v/r,
dps



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:51:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:51:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181070.328046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbVD-0006pp-VY; Tue, 07 Sep 2021 13:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181070.328046; Tue, 07 Sep 2021 13:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbVD-0006pi-SS; Tue, 07 Sep 2021 13:51:07 +0000
Received: by outflank-mailman (input) for mailman id 181070;
 Tue, 07 Sep 2021 13:51:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNbVB-0006pc-PY
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:51:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7256216b-c29c-4c48-85fd-64c86b5c82b2;
 Tue, 07 Sep 2021 13:51:04 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2055.outbound.protection.outlook.com [104.47.0.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5--Wrg1pWOO-eZLwVB7v2VBw-1;
 Tue, 07 Sep 2021 15:51:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 13:51:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 13:51:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0064.eurprd04.prod.outlook.com (2603:10a6:208:1::41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 13:51:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7256216b-c29c-4c48-85fd-64c86b5c82b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631022663;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oD9lEvFvJWfwl3ApFpXACR6rFLHSyvQV5QD0/g24X+M=;
	b=f3PpwEUzXbe7i0ifMyT1QoSD6FlOFZhxEXYtuZj1oZo0KuoUFgu8bdAEyMacqwhk7C0EEI
	7Pvqy0XpErEeZ9LE2GxmYhdybEug1GSLTIt7IPpCePoOb1Dk3NwRDaREmeGWh0H4+6BFuY
	4jaIckJbHyHyb/wYk9D6iQpnBugsNIs=
X-MC-Unique: -Wrg1pWOO-eZLwVB7v2VBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ned0cZd1dBZbzV6j7mc4luj8weeZVsOjB7ioXLV6QW8WAGVNxOa4SXoKB7MYRAVhfZeJIKDiYPIGxYe0OoRUtRoCRlSORf178gxbqgQLqU0zRlozhXsjXem3PEIaAzhBCCfLSkerAenGr1oxE0KpcqODIj5pBqC+fbX+Rv4aTmEf3kq6BBAgz0YWqxb0qzFoB2FYvKqy+99idZ4O677T15CCequP4qSlnmNWvsW6q1aoUsLR8DgYsoUaggEdVxUb8bV0YO6o2N9rJRAmt5abKvzJuNaCEw3SiIvQE+retMTe/UZ17r09o8HCLyq5UEzvJBmgslQDUIObVooQ/4/y5Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=12dn3pQY22t0OvIVVptgaOURbegA5BTrCgl17yzHBvQ=;
 b=Q2BxwsHdGemQzr/oBDtl+XiBZgjCpiXxAiQ7A6AdWR9ezJTp3tiEpr8I1hgsnyGwNFgRNocsxkOwfDoUOBwWSQr9cbpfgsZjr72kchN672kJ+AYp0abFihKUApkiLRsKpeeP3m/oUxWexZUlfc8kI/OrMxetayPrKxNS0INKy6a1IyyxP94Usl+J1ixVjbopDUXQX2M1e9qnFKXqYrFPijlAzfXRVBmBajWUL4+wHwcvNS6zoBXiJ1tSbQuADf2GwuqiibhbHLntMTKONv78NsDIYKqriITbq9m8ecVs2/Zfdyf4occ2VcNmGESDR3ZmqUczg8U7gzXEB43tqQHz5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
 <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
Date: Tue, 7 Sep 2021 15:50:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR04CA0064.eurprd04.prod.outlook.com
 (2603:10a6:208:1::41) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 50a728d6-2f4f-4ea6-097e-08d972068704
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686321243F67DD1C896C0DA3B3D39@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	BR23Jgn5+63Wq7KKhHVcnogmEGxKfIemIg7d3RknOCH7AB5Bd6qr/I5OiupuEQcf5M9CvxfvEbjtwP4ZNXt6S975hLHVtTMeq6nyWHEktGCz/D6h2ljw2M/rptDkgX12jdNXmxFg9VASnFouXgOMzgDQS6mZMp8nppkUpVmGwez3bpE236/mw5pFWmX7V4vBxq0eLigQiBqtK6YzzsNGYlyZrL1L4fU27XpcICa5AFC/Oe94HSArFdT2XvLPFN3psi0anXzFl6kpjV29JuJbSL0yyu9/pZENR7bJzeNzl5wcaXbpblYK4IoPZTzoPJZuRN/U81rF1PRSskTJ/qdrfsm/Kb9+A+Xi+NuJu75sdsmFncncTQfUf1Uuew7gOoRo4wrjf8uE2aYnzCYhla6PAMgoIb2DBzmCNCKStyma9lDyiuA+5oXtHweazMVhDcCiGaaX/CRah/hkAZa4HFfv3HeRiJvAhwGRe8KQ/ekb7/GTo8G5v6Yy5m8FZUg36/ixHjiYX28Bcn20/uqjrZC3hKID+yI3khB0t3+zq5LS06M3N+0cUTJHxfMVdDoAeHQMkdBrRmGQFAGFMnAdvchgRs3RuL06NTHhIbYMgK2jWR4QylHULCTh6Q95v1wwnNoIDuBxQ+917gHf5WSiroPQpvVPT/MFqadw3Q+RglBmf63fWvztoeGHuGfvsSVgBaDfkl1FG1Lrm/dOntzdYi6Wi4ou4KilabkYTeNNRwFkf+U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(396003)(346002)(366004)(39860400002)(66556008)(31686004)(5660300002)(66946007)(66476007)(38100700002)(478600001)(2906002)(6486002)(6916009)(186003)(8676002)(26005)(31696002)(83380400001)(36756003)(54906003)(8936002)(2616005)(316002)(956004)(86362001)(4326008)(16576012)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?VbgZAVlMZQXopuq0YOhPxSVGwvXueUBAt5SVmjJpVgRET6A5vhiXHEZYkTn4?=
 =?us-ascii?Q?YN+ZLMOhDDi3GyaCadVAuEmEkA6MYDZs0bdpcvwL7IZGGl0y82i1Imy3WcQE?=
 =?us-ascii?Q?ndPB6J+VUzyyVp+IoGNRlRQCjQnxcKZuZaj4g7JMWJAUErqjWgNTfBPCJ3R1?=
 =?us-ascii?Q?4kq6Begn2vEQDNpgXtmoIb3tqFLx8nHVp2IWZemwxXji3n05S3l9wiSD3t3X?=
 =?us-ascii?Q?nY+p0rrmsNJM7HEz4mhFIIDAn0rymxe9oBZP8gn8jBuKwnkXYLq514ShN3B9?=
 =?us-ascii?Q?+wNnYMtgbEBSPgEUdMgFhjcfXdpJ6zEVkFzx4EuUEGN1D0cZFLm/YnhbcwEP?=
 =?us-ascii?Q?s6JZ6HqkitphxjT3VatoVfsFnouDkHR9xWy1Iivtd3xzfxOjAiEOJmYxYf0P?=
 =?us-ascii?Q?p7VEkZvJ0gi3FgEnKimwfztOgUJpXnWt3XSR4QECvYhprAKIbjcId/6na3up?=
 =?us-ascii?Q?t34f++8G7ylbgSRYTiFcwC/lsu1cSaWcklak6RMolRy2pP5pkblCiHwZE3hP?=
 =?us-ascii?Q?GN6SYBhPewqeh14y7Xtd68j6K3GYW1K2PFpYQsKWZ1b+woGtpdDmR6Or3gIY?=
 =?us-ascii?Q?bUzz/DG//NoQ0W++xmVnWEgfb3nh9I/fxVbIRyHGoTKAEedkZRPcLzO5noQ9?=
 =?us-ascii?Q?PKXoc4ZSEvupPjnb2xRkHuQ/C+ZpjH6AcDjsPfRwor0f3bFgKYdpUQT/yn08?=
 =?us-ascii?Q?wkJm+6tddC7rHWCj5FdRwAqLTrSqEL3wlxmRLcR1T+iT/WmF5e8ZxnT0d7Bo?=
 =?us-ascii?Q?h31KDxkwsW3IXlOZavMYAq1ys7CTh9CtexCDWNvnb9lOmCi73WkQxpw7uwHd?=
 =?us-ascii?Q?aj/AXSg8/IAjncYRJqzdS103PSoDLLFjiOezYUUZFjasjMzJJpYsTCKKBVAM?=
 =?us-ascii?Q?iN7luGS3FQtfI7/Kb/F5HeLHDR2Q9/1qw38O7P8Lzc9FP3VekNQsbYPmBst5?=
 =?us-ascii?Q?CJkBFuuoYY1tRPGPUHVLBDBWYc9uK9z8I3OyJAwi5htVAGdsQ87GNBstqpM6?=
 =?us-ascii?Q?2e7zTCWDwRRQetK9r59V2/XdvZmWbPRLZkaOomiCo41ZWsUUEctx91bpaeKM?=
 =?us-ascii?Q?FcxSGMj+ML9SJbFxDs3GQoHD9OyUZiNmHVluOpUkunbOUizQu4ulD++EhzGg?=
 =?us-ascii?Q?zN8XoAUSumOkgC3nE7lw7IYl3fPAAoVKecsEyRQVQp6Wpl/iG/cmjrwuyEnU?=
 =?us-ascii?Q?T+9eQA1fw7N39h50tkf1bbTdAIKkCUnMifV5w0FBUNkBMF4L+rmdkYzj7O5k?=
 =?us-ascii?Q?ER/shWt6vaps/Aa6cdluLfT36ER6vJRDub3BL59CmdYNf1uT55OtthjVadeQ?=
 =?us-ascii?Q?o4NTRlcBfZ9U/jc4N5m1oTcH?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 50a728d6-2f4f-4ea6-097e-08d972068704
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 13:51:01.1773
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AVtYRxV+7RVobiCVcg3M4gzyYheJJ9EsKgDiDvnIgGH0S5G5oh/W5GBn8VCCkJM6fmMevcv9ekvSSnHYyfMh4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 07.09.2021 15:41, Daniel P. Smith wrote:
> On 9/6/21 2:17 PM, Andrew Cooper wrote:
>> On 03/09/2021 20:06, Daniel P. Smith wrote:
>>> --- a/xen/include/xsm/dummy.h
>>> +++ b/xen/include/xsm/dummy.h
>>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>>  =20
>>>   #endif /* CONFIG_XSM */
>>>  =20
>>> -static always_inline int xsm_default_action(
>>> -    xsm_default_t action, struct domain *src, struct domain *target)
>>> +static always_inline int xsm_default_action(xsm_default_t action,
>>> +                                            struct domain *src,
>>> +                                            struct domain *target)
>>
>> The old code is correct.=C2=A0 We have plenty of examples of this in Xen=
, and
>> I have been adding new ones when appropriate.
>>
>> It avoids squashing everything on the RHS and ballooning the line count
>> to compensate.=C2=A0 (This isn't a particularly bad example, but we've h=
ad
>> worse cases in the past).
>=20
> Based on the past discussions I understood either is acceptable and find=
=20
> this version much easier to visually parse myself. With that said, if
> the "next line single indent" really is the preferred style by the=20
> maintainers/community, then I can convert all of these over.

I guess neither is the "preferred" style; as Andrew says, both are
acceptable and both are in active use. I guess the rule of thumb is:
The longer what's left of the function name, the more you should
consider the style that you change away from.

Anyway, in the end I guess the request for style adjustments was
mainly to purge bad style, not to convert one acceptable form to
another. Converting the entire file to the same style is of course
fine (for producing a consistent result), but then - as per above -
here it would more likely be the one that in this case was already
there.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:53:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:53:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181076.328057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbX8-0007a0-Cm; Tue, 07 Sep 2021 13:53:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181076.328057; Tue, 07 Sep 2021 13:53:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbX8-0007Zt-9b; Tue, 07 Sep 2021 13:53:06 +0000
Received: by outflank-mailman (input) for mailman id 181076;
 Tue, 07 Sep 2021 13:53:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xlGX=N5=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNbX6-0007Zn-Sy
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:53:04 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 45fbc0fe-7e96-4ad0-b06a-85d1b067e582;
 Tue, 07 Sep 2021 13:53:03 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id ADE741FF95;
 Tue,  7 Sep 2021 13:53:02 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 9515513A24;
 Tue,  7 Sep 2021 13:53:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id QE3gIr5uN2GgfwAAGKfGzw
 (envelope-from <jgross@suse.com>); Tue, 07 Sep 2021 13:53:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45fbc0fe-7e96-4ad0-b06a-85d1b067e582
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631022782; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=Ni/NQ190x3Mlq9HWfhTgoCgr+slvDM1E7IewUCMMiMQ=;
	b=G4U3H2ifSiawaKDYI8HS5SMBvYKZEi1oT4Sc57S9YIfwSpXnxLuvPI7N7gf+3h772GZciD
	vNCuiRcUob/LGW63FuezNGjKRClJQLuTVn3/wfTDeLW+WPsiqsNL9zIpY6oZKtifZ1dByn
	NyUEHzcx3aaKgipXopsDDFEzZzv4eV8=
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: Sander Eikelenboom <linux@eikelenboom.it>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
From: Juergen Gross <jgross@suse.com>
Message-ID: <955da8ce-40f2-32c5-ee78-6801ce18afe5@suse.com>
Date: Tue, 7 Sep 2021 15:53:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Y0QXcXgTugGoqPx0Eq7pOX3EV4o9pfd7F"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Y0QXcXgTugGoqPx0Eq7pOX3EV4o9pfd7F
Content-Type: multipart/mixed; boundary="dUye6vfcouJSbb1w8PO2dLlk1jDNCtNt9";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Sander Eikelenboom <linux@eikelenboom.it>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <955da8ce-40f2-32c5-ee78-6801ce18afe5@suse.com>
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
In-Reply-To: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>

--dUye6vfcouJSbb1w8PO2dLlk1jDNCtNt9
Content-Type: multipart/mixed;
 boundary="------------72E11293AFB37D2C9018C87B"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------72E11293AFB37D2C9018C87B
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 06.09.21 23:35, Sander Eikelenboom wrote:
> L.S.,
>=20
> On my AMD box running:
>  =C2=A0=C2=A0=C2=A0 xen-unstable changeset: Fri Sep 3 15:10:43 2021 +02=
00 git:2d4978ead4
>  =C2=A0=C2=A0=C2=A0 linux kernel: 5.14.1
>=20
> With this setup I'm encountering some issues in dom0, see below.

Could you test whether the attached patch (only compile tested)
fixes the issue for you?


Juergen

--------------72E11293AFB37D2C9018C87B
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-xen-fix-usage-of-pmd-pud_poplulate-in-mremap-for-pv-.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename*0="0001-xen-fix-usage-of-pmd-pud_poplulate-in-mremap-for-pv-.pa";
 filename*1="tch"

=46rom c9e4ce85bcb9003ffa4877743e1792ac6ec24f19 Mon Sep 17 00:00:00 2001
From: Juergen Gross <jgross@suse.com>
To: x86@kernel.org
To: linux-kernel@vger.kernel.org
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: xen-devel@lists.xenproject.org
Date: Tue, 7 Sep 2021 15:17:39 +0200
Subject: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv gue=
sts

Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page
table entries") introduced a regression when running as Xen PV guest.

Today pmd/pud_poplulate() for Xen PV assumes that the PFN inserted is
referencing a not yet used page table. In case of move_normal_pmd/pud()
this is not true, resulting in WARN splats like:

[34321.304270] ------------[ cut here ]------------
[34321.304277] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:10=
2 xen_mc_flush+0x176/0x1a0
[34321.304288] Modules linked in:
[34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted 5.14.1-2021090=
6-doflr-mac80211debug+ #1
[34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V=
1.8B1 09/13/2010
[34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
[34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 0=
3 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00=
 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
[34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
[34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807=
d416398
[34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefd=
eadf00d
[34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff88800=
6160cc0
[34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12: 000000000=
0000000
[34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15: 000000000=
0000001
[34321.304320] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlG=
S:0000000000000000
[34321.304322] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 000000000=
0000660
[34321.304326] Call Trace:
[34321.304331]  xen_alloc_pte+0x294/0x320
[34321.304334]  move_pgt_entry+0x165/0x4b0
[34321.304339]  move_page_tables+0x6fa/0x8d0
[34321.304342]  move_vma.isra.44+0x138/0x500
[34321.304345]  __x64_sys_mremap+0x296/0x410
[34321.304348]  do_syscall_64+0x3a/0x80
[34321.304352]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[34321.304355] RIP: 0033:0x7f507196301a
[34321.304358] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c=
8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05=
 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
[34321.304360] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000=
000000000019
[34321.304362] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507=
196301a
[34321.304363] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506=
dc56000
[34321.304364] RBP: 0000000001a00000 R08: 0000000000000010 R09: 000000000=
0000004
[34321.304365] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506=
dc56060
[34321.304367] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205=
f950f30
[34321.304368] ---[ end trace a19885b78fe8f33e ]---
[34321.304370] 1 of 2 multicall(s) failed: cpu 0
[34321.304371]   call  2: op=3D12297829382473034410 arg=3D[aaaaaaaaaaaaaa=
aa] result=3D-22

Fix that by modifying xen_alloc_ptpage() to only pin the page table in
case it wasn't pinned already.

Fixes: 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page t=
able entries")
Cc: <stable@vger.kernel.org>
Reported-by: Sander Eikelenboom <linux@eikelenboom.it>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/mmu_pv.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 1df5f01529e5..8d751939c6f3 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1518,14 +1518,17 @@ static inline void xen_alloc_ptpage(struct mm_str=
uct *mm, unsigned long pfn,
 	if (pinned) {
 		struct page *page =3D pfn_to_page(pfn);
=20
-		if (static_branch_likely(&xen_struct_pages_ready))
+		pinned =3D false;
+		if (static_branch_likely(&xen_struct_pages_ready)) {
+			pinned =3D PagePinned(page);
 			SetPagePinned(page);
+		}
=20
 		xen_mc_batch();
=20
 		__set_pfn_prot(pfn, PAGE_KERNEL_RO);
=20
-		if (level =3D=3D PT_PTE && USE_SPLIT_PTE_PTLOCKS)
+		if (level =3D=3D PT_PTE && USE_SPLIT_PTE_PTLOCKS && !pinned)
 			__pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE, pfn);
=20
 		xen_mc_issue(PARAVIRT_LAZY_MMU);
--=20
2.26.2


--------------72E11293AFB37D2C9018C87B
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------72E11293AFB37D2C9018C87B--

--dUye6vfcouJSbb1w8PO2dLlk1jDNCtNt9--

--Y0QXcXgTugGoqPx0Eq7pOX3EV4o9pfd7F
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE3br4FAwAAAAAACgkQsN6d1ii/Ey+R
Ewf/VFek8aJAUMbjuWxGgqIccUOkBjyZN2fG8a1f1eQ8VdKJy+lTH7Z9pu+iyBBVTwutLHdYP1gU
7yDd040CmChljDWQz4ZdzvvJulVvlze6dcGrjhfUwfZADRS+mHc8iO/jDjjzYJB5xL81LgIR+lnR
QVEsK78QpWogPpAuTX2k1xiHLXAEfXiTjD0Dph+yLcWUU4IbLkYluzduR0L78C0n7gpFHaWG8Z/W
7fVKKhJf8/8KpSMMCOwaTESbpa2p3F6ENvBKCCf5lho3iqrnBq4SNkcHSwKGf1cCrNtPXJb3Pbd4
M6sWY+Dl6ksFIwn6gUfm90OKpcGodJQ/WSl2chYgkw==
=vCbt
-----END PGP SIGNATURE-----

--Y0QXcXgTugGoqPx0Eq7pOX3EV4o9pfd7F--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 13:54:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 13:54:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181082.328067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbYU-0008Az-O3; Tue, 07 Sep 2021 13:54:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181082.328067; Tue, 07 Sep 2021 13:54:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbYU-0008As-Kg; Tue, 07 Sep 2021 13:54:30 +0000
Received: by outflank-mailman (input) for mailman id 181082;
 Tue, 07 Sep 2021 13:54:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNbYT-0008Am-EY
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 13:54:29 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1df3285e-0fe3-11ec-b0f4-12813bfff9fa;
 Tue, 07 Sep 2021 13:54:28 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631022862921821.2351718715845;
 Tue, 7 Sep 2021 06:54:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1df3285e-0fe3-11ec-b0f4-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631022866; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=V+msD7NyuKPbikIqyuZrQjlfG9rqnXRv+UdGyKeDJcplSHc4nfcPrXg9YnrP0zSY3xG1ALWZkkjuJu6mWHMqR8KTILHXpePtu4CG8Mw80Gkj0H0egAzg8pIzD7WkJSo50PihBB7va5WTgceXaBw1ZKq0j44hb57xabUKLD014zI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631022866; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=a0yjKN4vFjwtQ5VFXF1O0n7vkyNeAOlsxBb/XzZzYrU=; 
	b=nwnrI5t0OWMYHJN3XD5wL4rfsmMKdxs8901CSG/lgUnqfsGJPj2J7PnTa8YjPg3UU8IFd2W5lfs7/b2HVbP5eac1J2k/M3aSGoV4Wb4JNDbhsrnlUQKqO8BX/Uxx/Z5K2JhgDknLXFeZNFMBL526sv3QXu6voz81mZbd8KMrDfQ=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631022866;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=a0yjKN4vFjwtQ5VFXF1O0n7vkyNeAOlsxBb/XzZzYrU=;
	b=Ak7OYVa89Vm+TLZuIxxkztPYzwSakyThdNyxbE+XkDO9LW/NPGk/V5BgKQUrvK6c
	JHm5FYxP6YBQfeXW7+s/EQBLHFwoX8/IbziLo1XYXi0hR4gPIwa8uOO9VVqg+5HNtta
	n4qd8D62m6pmfBnBIuuCAQC24CFhwPjQeMbddEnw=
Subject: Re: [PATCH v4 07/11] xsm: decouple xsm header inclusion selection
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-8-dpsmith@apertussolutions.com>
 <17155620-d091-3205-6af0-65e19dfa1bfb@citrix.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <786e7f2c-67ce-ee44-bf98-d94772f44921@apertussolutions.com>
Date: Tue, 7 Sep 2021 09:52:39 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <17155620-d091-3205-6af0-65e19dfa1bfb@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/6/21 2:47 PM, Andrew Cooper wrote:
> On 03/09/2021 20:06, Daniel P. Smith wrote:
>> diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
>> new file mode 100644
>> index 0000000000..4555e111dc
>> --- /dev/null
>> +++ b/xen/include/xsm/xsm-core.h
>> @@ -0,0 +1,274 @@
>> +/*
>> + *  This file contains the XSM hook definitions for Xen.
>> + *
>> + *  This work is based on the LSM implementation in Linux 2.6.13.4.
>> + *
>> + *  Author:  George Coker, <gscoker@alpha.ncsc.mil>
>> + *
>> + *  Contributors: Michael LeMay, <mdlemay@epoch.ncsc.mil>
>> + *
>> + *  This program is free software; you can redistribute it and/or modify
>> + *  it under the terms of the GNU General Public License version 2,
>> + *  as published by the Free Software Foundation.
>> + */
>> +
>> +#ifndef __XSM_CORE_H__
>> +#define __XSM_CORE_H__
>> +
>> +#include <xen/sched.h>
>> +#include <xen/multiboot.h>
>> +
>> +/* policy magic number (defined by XSM_MAGIC) */
>> +typedef uint32_t xsm_magic_t;
>> +
>> +#ifdef CONFIG_XSM_FLASK
>> +#define XSM_MAGIC 0xf97cff8c
>> +#else
>> +#define XSM_MAGIC 0x0
>> +#endif
> 
> Eww.  I know you're only moving code, but this construct is broken
> (right from XSM's introduction in c/s d046f361dc937), and creates a
> fairly-severe bug.
> 
> It causes xsm_multiboot_policy_init() to malfunction and accept a module
> which starts with 4 zeroes, rather than the flask magic number.  The one
> caller is suitably guarded so this is only a latent bug right now, but I
> don't think we could credibly security support the code without this
> being fixed.  (Again - fine to add to the todo list.  I know there's
> loads to do)

I cannot in good conscience leave a clearly latent bug. Let me see if 
can work a safer means to handling policy module loading.

>> +
>> +/* These annotations are used by callers and in dummy.h to document the
>> + * default actions of XSM hooks. They should be compiled out otherwise.
>> + */
> 
> For the coding style patch, this should be
> 
> /*
>   * These ...

Ack.

>> +#ifdef CONFIG_XSM
>> +
>> +#ifdef CONFIG_MULTIBOOT
>> +int xsm_multiboot_init(unsigned long *module_map,
>> +                       const multiboot_info_t *mbi);
>> +int xsm_multiboot_policy_init(unsigned long *module_map,
>> +                              const multiboot_info_t *mbi,
>> +                              void **policy_buffer,
>> +                              size_t *policy_size);
>> +#endif
>> +
>> +#ifdef CONFIG_HAS_DEVICE_TREE
>> +/*
>> + * Initialize XSM
>> + *
>> + * On success, return 1 if using SILO mode else 0.
>> + */
>> +int xsm_dt_init(void);
>> +int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
>> +bool has_xsm_magic(paddr_t);
>> +#endif
>> +
>> +#ifdef CONFIG_XSM_FLASK
>> +const struct xsm_ops *flask_init(const void *policy_buffer,
>> +                                 size_t policy_size);
>> +#else
>> +static inline const struct xsm_ops *flask_init(const void *policy_buffer,
>> +                                               size_t policy_size)
>> +{
>> +    return NULL;
>> +}
>> +#endif
>> +
>> +#ifdef CONFIG_XSM_SILO
>> +const struct xsm_ops *silo_init(void);
>> +#else
>> +static const inline struct xsm_ops *silo_init(void)
>> +{
>> +    return NULL;
>> +}
>> +#endif
>> +
>> +#else /* CONFIG_XSM */
>> +
>> +#ifdef CONFIG_MULTIBOOT
>> +static inline int xsm_multiboot_init(unsigned long *module_map,
>> +                                     const multiboot_info_t *mbi)
>> +{
>> +    return 0;
>> +}
>> +#endif
>> +
>> +#ifdef CONFIG_HAS_DEVICE_TREE
>> +static inline int xsm_dt_init(void)
>> +{
>> +    return 0;
>> +}
>> +
>> +static inline bool has_xsm_magic(paddr_t start)
>> +{
>> +    return false;
>> +}
>> +#endif /* CONFIG_HAS_DEVICE_TREE */
> 
> Shouldn't this be an #ifndef CONFIG_HAS_DEVICE_TREE ?
> 
> And the answer is no because of the #else /* CONFIG_XSM */ higher up,
> but it is incredibly deceptive to read.
> 
> 
> I think this logic would be far easier to follow as:
> 
> #if IS_ENABLED(CONFIG_XSM) && IS_ENABLED(CONFIG_MULTIBOOT)
> ...
> #else
> ...
> #endif
> 
> etc.
> 
> rather than having two separate #ifdef CONFIG_MULTIBOOT blocks doing
> opposite things due to the position of intermixed #ifdef CONFIG_XSM.

Ack.

v/r
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:02:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:02:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181090.328079 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbgN-0001ek-Mz; Tue, 07 Sep 2021 14:02:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181090.328079; Tue, 07 Sep 2021 14:02:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbgN-0001ed-Jg; Tue, 07 Sep 2021 14:02:39 +0000
Received: by outflank-mailman (input) for mailman id 181090;
 Tue, 07 Sep 2021 14:02:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNbgL-0001eX-JK
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:02:37 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4021a5a9-0fe4-11ec-b0f5-12813bfff9fa;
 Tue, 07 Sep 2021 14:02:36 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631023350916797.7635160563889;
 Tue, 7 Sep 2021 07:02:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4021a5a9-0fe4-11ec-b0f5-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631023353; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=guLKaRb7pEXs6IvRpXDUHthw78iVpWi9KMcWoCdAQtd0K27y9wSn2xwU+YtwYdaJYxegxbBaGtEuE8maOgIb9eF+CYzrKH5tF8czrIV74mNx7vYAiBl7rmLhES6XFlJuqRJHifsPcPFIK+PPrpb1WT4OequkCIf/oQvq/OpnJk0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631023353; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=9gom1ql8ae5NoV/h/dFHGpp6IOjAPFwXRoaCwPNyKHY=; 
	b=Ryir5TWDAdMIW6Hed3D6Tto3pPa2caSMeEuNlyhUgH/sUpM9RZnIAMUmY3e9DZbFTpYk6acRzQN5gWNJe0LgnIvlBmQO9bbmdj+w160fyFnbIrGaqflU9Mc9DMCF4/w8FG5TABY3/rnBRdLCAkwX1OqjeYkgrWuix3ycbjRMxA4=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631023353;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=9gom1ql8ae5NoV/h/dFHGpp6IOjAPFwXRoaCwPNyKHY=;
	b=SEcyi00ox7hQx1Q6b539rpNUYrC+0QIR9R324xlHYNdvffgW7J0Wdc3UYXptjHlh
	fhGe+Jx8+rpkcptvj+N3vm/wCbf9kqkxo4+xz3zzuJNmLKBn5ZkBBFr6KBnteuW99Gj
	cWGu0dD26swHy4U0OnKa2aLoAlLbkBpyx2zZ006c=
Subject: Re: [PATCH v4 09/11] silo: remove circular xsm hook call
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-10-dpsmith@apertussolutions.com>
 <d415ab82-ca83-98e8-0b07-0e093a1fa7e1@citrix.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <997db7f6-266d-b488-4024-37d9adeb4ceb@apertussolutions.com>
Date: Tue, 7 Sep 2021 10:00:47 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <d415ab82-ca83-98e8-0b07-0e093a1fa7e1@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/6/21 2:55 PM, Andrew Cooper wrote:
> On 03/09/2021 20:06, Daniel P. Smith wrote:
>> SILO implements a few XSM hooks to extended the decision logic beyond
>> what is defined in the dummy/default policy. For each of the hooks, it
>> falls back to the dummy/default policy. The fall back is done a slight
>> round-about way.
> 
> "done in a slightly" ?

Ack.

>>   This commit makes the direct call to the default policy's
>> logic, xsm_default_action().
>>
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> ---
>>   xen/xsm/silo.c | 15 ++++++++-------
>>   1 file changed, 8 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
>> index 6db793f35c..56a330a831 100644
>> --- a/xen/xsm/silo.c
>> +++ b/xen/xsm/silo.c
>> @@ -17,6 +17,7 @@
>>    * You should have received a copy of the GNU General Public License along with
>>    * this program; If not, see <http://www.gnu.org/licenses/>.
>>    */
>> +#include <xsm/xsm-core.h>
>>   #include <xsm/dummy.h>
>>   
>>   /*
>> @@ -43,7 +44,7 @@ static int silo_evtchn_unbound(struct domain *d1, struct evtchn *chn,
>>       else
>>       {
>>           if ( silo_mode_dom_check(d1, d2) )
>> -            rc = xsm_evtchn_unbound(d1, chn, id2);
>> +            rc = xsm_default_action(XSM_TARGET, current->domain, d1);
>>           rcu_unlock_domain(d2);
>>       }
>>   
>> @@ -54,7 +55,7 @@ static int silo_evtchn_interdomain(struct domain *d1, struct evtchn *chan1,
>>                                      struct domain *d2, struct evtchn *chan2)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_evtchn_interdomain(d1, chan1, d2, chan2);
>> +        return xsm_default_action(XSM_HOOK, d1, d2);
>>       return -EPERM;
>>   }
>>   
>> @@ -62,21 +63,21 @@ static int silo_grant_mapref(struct domain *d1, struct domain *d2,
>>                                uint32_t flags)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_grant_mapref(d1, d2, flags);
>> +        return xsm_default_action(XSM_HOOK, d1, d2);
>>       return -EPERM;
>>   }
>>   
>>   static int silo_grant_transfer(struct domain *d1, struct domain *d2)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_grant_transfer(d1, d2);
>> +        return xsm_default_action(XSM_HOOK, d1, d2);
>>       return -EPERM;
>>   }
>>   
>>   static int silo_grant_copy(struct domain *d1, struct domain *d2)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_grant_copy(d1, d2);
>> +        return xsm_default_action(XSM_HOOK, d1, d2);
>>       return -EPERM;
>>   }
>>   
>> @@ -86,14 +87,14 @@ static int silo_argo_register_single_source(const struct domain *d1,
>>                                               const struct domain *d2)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_argo_register_single_source(d1, d2);
>> +        return 0;
>>       return -EPERM;
>>   }
>>   
>>   static int silo_argo_send(const struct domain *d1, const struct domain *d2)
>>   {
>>       if ( silo_mode_dom_check(d1, d2) )
>> -        return xsm_argo_send(d1, d2);
>> +        return 0;
> 
> Shouldn't these be XSM_HOOK too?  Or should all other XSM_HOOK's be
> short-circuted to 0?
> 
> The asymmetry here seems weird.

It makes more sense when you follow the approach, which was to duplicate 
the body of the dummy hook instead of making a call to the hook which 
would then call the function pointer to the dummy hook. The definition 
for the argo dummy hooks is to return 0. In the future these other calls 
may well have XSM_HOOk replaced with the proper role expected. Since all 
argo checks just return 0, this reflects there is no logic rules in 
xsm_default_action to determine argo accesses. Of course this is on my 
list todo and when the dummy hook is fixed, these would be synchronized.
With that said, converting over to XSM_HOOK does provide the equivalent 
and would provide consistency within the context of this file. Basically 
a long winded way of saying, ack.

v/r,
dps




From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:05:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:05:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181096.328090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbit-0002GB-6w; Tue, 07 Sep 2021 14:05:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181096.328090; Tue, 07 Sep 2021 14:05:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbit-0002G4-2F; Tue, 07 Sep 2021 14:05:15 +0000
Received: by outflank-mailman (input) for mailman id 181096;
 Tue, 07 Sep 2021 14:05:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNbis-0002Fw-CW
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:05:14 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3dfa05ba-0966-4b85-aa69-e27de524392b;
 Tue, 07 Sep 2021 14:05:13 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631023508010397.42168302385335;
 Tue, 7 Sep 2021 07:05:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dfa05ba-0966-4b85-aa69-e27de524392b
ARC-Seal: i=1; a=rsa-sha256; t=1631023510; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=DwVlQ8X2gwzBmofhZa0/mLTHPEFlZjcMnPjguUG6rAl52TJYQ1DLpmpEN4Ye3zovYIVKjdpMMPeR/11Jooh2ZCky0kTwuOA7fLDtb0UxS0OvMNfhl3ZEKekUxESMEy2Hcw3UbKGp2A/N3JCQFEPGps51Pdc8c47dltQNpke+uNQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631023510; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=GDLLgnfggnonVh83Z26KWmFNzxC2njX3v4Ulq7RK37o=; 
	b=El0mEU8QogFCmZvWlx5baQjQITtDhMYQ7QHP4OE9zDsttTahbauYGL8F4CaYbCGmib63Ls8TT2zIth7ENcepYPUuE3ikClAet8G3ExM51JFJuse8/ko6RnXdpG6oYo6RX+shc8qIdRAhgsUzy6b9ZOepqW2t+RUhYkq0IhtVYPY=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631023510;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=GDLLgnfggnonVh83Z26KWmFNzxC2njX3v4Ulq7RK37o=;
	b=A33lDEdZgYvISMA48FhQDSKaPy+eJmF6V4PbzsRsIYW/o2EO+iEOiYzbdhCSUSPB
	9GQiVlBCUxKTqpOi5+J1DC9VY28YrsVNaN4ayE0Mnn2dYuqgwIO4o89sqPA/aZXtVpQ
	DwPWvlaHIciSfItuSpYk55W6BJRLUlMNJ/OlYZMA=
Subject: Re: [PATCH v4 11/11] xsm: remove alternate xsm hook interface
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-12-dpsmith@apertussolutions.com>
 <b35d8b3a-c398-a101-8893-6a3ca6906e94@citrix.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <c4c6a02e-a7d2-b225-050d-ea3312924d06@apertussolutions.com>
Date: Tue, 7 Sep 2021 10:03:24 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <b35d8b3a-c398-a101-8893-6a3ca6906e94@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/6/21 3:18 PM, Andrew Cooper wrote:
> On 03/09/2021 20:06, Daniel P. Smith wrote:
>> -static inline int xsm_memtype(xsm_default_t def, uint32_t access)
>> +#if 0
>> +/* Could not find any usages */
>> +static inline int xsm_memtype(xsm_default_t action, uint32_t access)
>>   {
>>       return alternative_call(xsm_ops.memtype, access);
>>   }
>> +#endif
> 
> There wants to be an earlier patch deleting dead code.  We don't want to
> retain this if-0'd out.

I can transition this to a standalone patch at the beginning to drop the 
benign code.

>> diff --git a/xen/xsm/dummy.h b/xen/xsm/dummy.h
>> new file mode 100644
>> index 0000000000..b9a7e8c40f
>> --- /dev/null
>> +++ b/xen/xsm/dummy.h
>> @@ -0,0 +1,739 @@
>> +/*
>> + *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
>> + *
>> + *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
>> + *
>> + *  This program is free software; you can redistribute it and/or modify
>> + *  it under the terms of the GNU General Public License version 2,
>> + *  as published by the Free Software Foundation.
>> + *
>> + *
>> + *  Dummy XSM hooks implementing the default access check. Each hook should
>> + *  have as its first line XSM_DEFAULT_ACTION declaring the privilege level
>> + *  required for this access.
>> + */
>> +
>> +#ifndef __XSM_DUMMY_H__
>> +#define __XSM_DUMMY_H__
>> +
>> +#include <xen/sched.h>
>> +#include <xsm/xsm-core.h>
>> +#include <public/hvm/params.h>
>> +
>> +#define XSM_DEFAULT_ACTION(def) xsm_default_t action = def; (void)action
>> +
>> +static always_inline int xsm_default_action(
>> +    xsm_default_t action, struct domain *src, struct domain *target)
>> +{
>> +    switch ( action ) {
> 
> Either here (because you're moving code), or in the style fix, the brace
> wants to be on the next line.

I will promote to the style patch and carry it through the move.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:11:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:11:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181102.328100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNboX-0003yv-Op; Tue, 07 Sep 2021 14:11:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181102.328100; Tue, 07 Sep 2021 14:11:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNboX-0003yo-Lz; Tue, 07 Sep 2021 14:11:05 +0000
Received: by outflank-mailman (input) for mailman id 181102;
 Tue, 07 Sep 2021 14:11:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNboX-0003yB-0N
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:11:05 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6f5131f8-0fe5-11ec-b0f7-12813bfff9fa;
 Tue, 07 Sep 2021 14:11:03 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631023856310111.27960753270997;
 Tue, 7 Sep 2021 07:10:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f5131f8-0fe5-11ec-b0f7-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631023862; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=ZIL3ngtVoMCpENgJBqG6qOYrPmBTvfSmYGYT8jlTyBVe8U2ePx8VpgTasGI+kphTyo4ro3Il8mVA8m6ez/s80TvKrK614WL/ZUI/HKh2S85c4HvHwoMOzUSpPQhNzABkd/WhlXwatH46GTnzC3eWVUO6e/GsThtIq1msUG070ZI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631023862; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=NGknd/JHTSfyOh9tzQSO+4h9Q2/QFArTITRMaNAFEqs=; 
	b=juQWA93l5PqfG068ROI0otQkMXrRjgB8gNMUnU4plSFOI4dXmW56aGY0hvNbzE0wS/nrJFbdyX7Vt6+Aw8zmt2TYypHCDpn4ObYFj+Ix1GgQNuSZl8E+NErq8UMTHcGgThucIUq71++r2ukwk8JUaj6RvLeiYbcTCTewYAug3Ng=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631023862;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=NGknd/JHTSfyOh9tzQSO+4h9Q2/QFArTITRMaNAFEqs=;
	b=YN/QLhcL3Nc6dl9z2F4ist/Z89bUslpmyQsNTr9ThNnXJWdTCnwGoegR7X53Vldg
	D0il3JmNEi24FeIUgCNzRkkhAkhS3prydd1OidjJ9P3HtUpkXWUHdG3PEFgbUo98fFY
	V9D8alPvzuhb8oiNQdHzhdqo0IqkVgB4p17VhR4w=
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: Jan Beulich <jbeulich@suse.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
 <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
 <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <3b3762a7-5034-9094-84f9-cd8ab24e24a0@apertussolutions.com>
Date: Tue, 7 Sep 2021 10:09:12 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/7/21 9:50 AM, Jan Beulich wrote:
> On 07.09.2021 15:41, Daniel P. Smith wrote:
>> On 9/6/21 2:17 PM, Andrew Cooper wrote:
>>> On 03/09/2021 20:06, Daniel P. Smith wrote:
>>>> --- a/xen/include/xsm/dummy.h
>>>> +++ b/xen/include/xsm/dummy.h
>>>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>>>    
>>>>    #endif /* CONFIG_XSM */
>>>>    
>>>> -static always_inline int xsm_default_action(
>>>> -    xsm_default_t action, struct domain *src, struct domain *target)
>>>> +static always_inline int xsm_default_action(xsm_default_t action,
>>>> +                                            struct domain *src,
>>>> +                                            struct domain *target)
>>>
>>> The old code is correct.  We have plenty of examples of this in Xen, and
>>> I have been adding new ones when appropriate.
>>>
>>> It avoids squashing everything on the RHS and ballooning the line count
>>> to compensate.  (This isn't a particularly bad example, but we've had
>>> worse cases in the past).
>>
>> Based on the past discussions I understood either is acceptable and find
>> this version much easier to visually parse myself. With that said, if
>> the "next line single indent" really is the preferred style by the
>> maintainers/community, then I can convert all of these over.
> 
> I guess neither is the "preferred" style; as Andrew says, both are
> acceptable and both are in active use. I guess the rule of thumb is:
> The longer what's left of the function name, the more you should
> consider the style that you change away from.
> 
> Anyway, in the end I guess the request for style adjustments was
> mainly to purge bad style, not to convert one acceptable form to
> another. Converting the entire file to the same style is of course
> fine (for producing a consistent result), but then - as per above -
> here it would more likely be the one that in this case was already
> there.

Understood, I will respin with all the function defs to align with the 
"next line single indent" style, though it would be helpful for 
clarification on this style exactly. Do you always wrap all args if one 
extends past 80 col or is there a rule for when some should remain on 
the first line (function def line)?

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:18:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:18:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181109.328112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbvv-0004gM-Ia; Tue, 07 Sep 2021 14:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181109.328112; Tue, 07 Sep 2021 14:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbvv-0004gF-FU; Tue, 07 Sep 2021 14:18:43 +0000
Received: by outflank-mailman (input) for mailman id 181109;
 Tue, 07 Sep 2021 14:18:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mNbvu-0004g9-3O
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:18:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNbvq-0005oi-JY; Tue, 07 Sep 2021 14:18:38 +0000
Received: from [54.239.6.187] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mNbvq-0004tO-BP; Tue, 07 Sep 2021 14:18:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=BgwjwoRffSORjuYOIjYU6JiyDgimHWexiXbdlk6/CCE=; b=Pb8RniQnpbLpN97SzuDt6ciO/l
	lubmC3unY3XmmqFTaEZxpWMyJGXMpli5q7YLSDM9oGZE+plZljbzEOSkoNes38vSyMu2sGA0eYmRt
	LeQHFZI22xfQhcj42iSW/Tt7WQuyDU4tSDANhYqRDmbtVZGtxX17SYOCqckme8LApoCE=;
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
 <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
 <5DA71CEF-0ADE-4846-A7BC-B378B95A38AD@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <910f2758-cbce-53b0-4e57-4b2a966ce2b3@xen.org>
Date: Tue, 7 Sep 2021 15:18:35 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <5DA71CEF-0ADE-4846-A7BC-B378B95A38AD@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Luca,

On 07/09/2021 14:30, Luca Fancellu wrote:
>> On 7 Sep 2021, at 13:30, Julien Grall <julien@xen.org> wrote:
>> On 07/09/2021 12:51, Luca Fancellu wrote:
>>>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>>> What we could do is providing a list of binaries to load and associate a key for each of them. Something like:
>>>>
>>>> binary=<binary> <key>
>>>> binary=<binary2> <key2>
>>>> ....
>>>>
>>>> We can then replace the property "reg" with a new property "uefi,key" that will contain the name of the binary.
>>>>
>>>> What do you think?
>>> Here I’m lost, because I don’t understand what we are going to do with the name of the binary.
>>
>> <binaryX> would be used by the UEFI stub to load the binary in memory. Each binary will have a <keyX> which helps to refer them in the Device-Tree. To give a concrete example, let say we have two dom0less domains:
>>   - DomA: 2 vCPUs, 128MB
>>   - DomB: 3 vCPUs, 512MB
>>
>> DomA and DomB will be using the same kernel but a different ramdisk. xen.cfg, would look like:
>>
>> [global]
>> default=section1
>>
>> [section1]
>> options=console=vga,com1 com1=57600 loglvl=all noreboot
>> kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>> ramdisk=initrd-3.0.31-0.4-xen
>> xsm=<filename>
>> dtb=devtree.dtb
>> binary=vmlinuz-guest domu-kernel
>> binary=ramdisk-domA.img domA-ramdisk
>> binary=ramdisk-domB.img domB-ramdisk
>>
>> The chosen node in the DT would look like:
>>
>> chosen {
>>     domU1 {
>>         compatible = "xen,domain";
>>         #address-cells = <0x2>;
>>         #size-cells = <0x1>;
>>         memory = <0 0x8000000>;
>>         cpus = <2>;
>>
>>         module@1 {
>>             compatible = "multiboot,kernel", "multiboot,module";
>>             uefi,binary = "domu-kernel";
>>             bootargs = "console=ttyAMA0 init=/bin/sh";
>>         };
>>
>>         module@2 {
>>             compatible = "multiboot,ramdisk", "multiboot,module";
>>             uefi,binary = "domA-ramdisk";
>>         };
>>     };
>>
>>     domU2 {
>>         compatible = "xen,domain";
>>         #address-cells = <0x3>;
>>         #size-cells = <0x1>;
>>         memory = <0 0x20000000>;
>>         cpus = <3>;
>>
>>         module@1 {
>>             compatible = "multiboot,kernel", "multiboot,module";
>>             uefi,binary = "domu-kernel";
>>             bootargs = "console=ttyAMA0 init=/bin/sh";
>>         };
>>
>>         module@2 {
>>             compatible = "multiboot,ramdisk", "multiboot,module";
>>             uefi,binary = "domA-ramdisk";
>>         };
>>     };
>> };
>>
>> With this approach, the change is quite minimal to move between an classic U-boot boot and EFI boot.
> 
> Ok now I see, yes this approach can work and can save some code, in the current code we have that if
> a "multiboot,module” is found in the dtb, the Xen EFI configuration file is skipped, but if we use the
> module@XX {} without the compatible it can work, the UEFI stub will load the binary and update all
> the needed properties (compatible, reg).
With my proposal, you don't know whether the binary is a kernel, 
ramdisk... So you wouldn't be able to recreate the compatible properly.

But the behavior of the UEFI stub can be modified. We could say that if 
there is a "xen,domain" then use the configuration file to fetch the 
binaries.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:21:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:21:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181115.328123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbyL-0006LY-Vk; Tue, 07 Sep 2021 14:21:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181115.328123; Tue, 07 Sep 2021 14:21:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbyL-0006LR-Sf; Tue, 07 Sep 2021 14:21:13 +0000
Received: by outflank-mailman (input) for mailman id 181115;
 Tue, 07 Sep 2021 14:21:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNbyK-0006LD-Mt
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:21:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d977dc7a-0fe6-11ec-b0fa-12813bfff9fa;
 Tue, 07 Sep 2021 14:21:11 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2051.outbound.protection.outlook.com [104.47.8.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-o8JIptTZNTWFw-2IS7wz3Q-1; Tue, 07 Sep 2021 16:21:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 7 Sep
 2021 14:21:08 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 14:21:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0001.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 14:21:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d977dc7a-0fe6-11ec-b0fa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631024470;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=A6mkUxjwYk+rRPjoLLQldea3GJY3g6bYT4edMm8ZTHk=;
	b=HmEthlFO9fAqdmLBBWlIMEgUJ0OvYjVpC9SGryDq9eDLbY8Fbyps8YnWgaTqDUl6D6Hbw8
	ygBmnkCo5JpcFc0k9odQHpd5v+POmcs7XObWC5yms+CJ7PgeddipeajHs/fzs4SP9tWCtn
	OAsTVNVf5ZRg19QR3wn+QxyfGLzJYnQ=
X-MC-Unique: o8JIptTZNTWFw-2IS7wz3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Yv87STg/suGEq/VSd+mw1aoRMgB1MsR7j8xQf/aEsaa4wkVX+Tu6E3+aS+1qzEop+F5OijtmNGu+DyMABjdctWjpzS4hZ7go5JqhQKzIYJbcOScHxf0hsUsrLOpQtVEEw9HDnukPQ9ODyicHxEU7ZKJGjT4lkivFUm1YS9Ixtlr/jMZpGd4U/7cUw+HOjHZ+k16l8BGb2ix40IfRXWRlMAS2V+NzK8oSrJOg9icHKIWoe8+6pxIOdyFlY4Z9L2E5i7RWqmESOzua+naTGX6k1JCEEc/sZNUzm5l2Zy/PGKOrPsU9qElIEXFqSQbp1AOwrSvwa8idA7o2mBuCcJswPg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5P2vnTU15/bPw9O+/l2UDZyNPfbD2qOdV5IDyfmhzB0=;
 b=RXaLOf249NjrhE4i9FmD1QcV7EpsN/iFl+wgTEA52wFwQu9nrPqHpkwFvstUs5UgDVKI6DDmScHemMEhPyepcM5IGbPnJkQvlAw+q5k8H/XQXr3ja8QZWk557sFGsisgbQsfPG3505MhYVjx04NXOvlXKk3Ybpst+ACDhvpUfWyimEc6/xFW+qcvGYPGf858JNz93MAIGtY+2aRzV8r/i7Gcnn6yxNEl0uYtgTWZuaFKt1O2RLO/P26E8YUDpgOf0EYVfuMCqIFSFwtumguYzgazuixGSf9uRthVYpgz/bpzk+ary/QwX3n7fmrlDyTSY3wMuiwdtKnd0POH8D5RZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v1 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Pu Wen <puwen@hygon.cn>,
 Andy Lutomirski <luto@kernel.org>, Xen-devel
 <xen-devel@lists.xenproject.org>, Jane Malalane <jane.malalane@citrix.com>
References: <cover.1630929059.git.jane.malalane@citrix.com>
 <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <26b88d5a-ae6f-0065-7c3f-5eeb6dd80078@suse.com>
 <2972c71a-5fed-0813-8ec0-d179eacff14d@citrix.com>
 <70f376c6-eef2-42ce-a8eb-67c1130a41e4@suse.com>
 <dcdce06f-4a19-49c7-2225-a8e38d524dd8@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1076b12b-dbbd-f1cc-4d78-511122f02b99@suse.com>
Date: Tue, 7 Sep 2021 16:21:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <dcdce06f-4a19-49c7-2225-a8e38d524dd8@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P195CA0001.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b6f31afb-8ae4-439e-2067-08d9720abbf2
X-MS-TrafficTypeDiagnostic: VI1PR04MB5600:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5600466A1464C880FED76934B3D39@VI1PR04MB5600.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kNEg1fcaAdxyWoNstaE4kHU27kb2AgLmx7yQFVOgj2Eqdxfck4ZgC9bpzpq42dACwkWohwzJcSj5InGc0nV0KOSXZHTp5FY3wnoeeDfJDkkteNrgLaDefMIvyaVuU2RSYX35FCQJk6UmpP6rzWZzas0I05AFpHGbP4zaulN8wIRE3vSM3wV2UNd0Ue2VHU1I3hh+uBRjJPfFAs9dP/udjviLvGtfqSeh4jr6z5nubbvOKMdi1TW7f/BQUJLXX8Ds4EGB627M9n4J0uX+KWObyZOUgxtiT8ohnqLcz3m5gsmVgPlKEYIFFvzmoIIqhzWOlrsAl2VKeXbO1XWtaKvFSS4dv6lXIBXCPva/FVJzOP7QUVuo11lZyUhsAwAOJRS4fkEvf/ur4Cb1He0H0bkmf6Svcnoi6ZxRYfXdHuRwbb2XKRjgrTPP80T1lWHXmiJ+JS20jDI5607DuasTbwyfWOGc8vmo8tECdZcK/oObe8S+lu0ixvelF4GqPf8qVCz/Ky+nml7XjJf2V2+jOOM3kw4N3HWYX1GgHUbGunrN/Ks2M4ZUp33tqX5gmWrp1i0C17/ZPrcayuinjDdxGAr4JForMTBonTQbY/TzdGaTIYjz2ZFANBQlhEBc7tml5BVCElhYhF/GSrsiv+QEj2JtSU2vjJfoZAIg/VZg4gLpI4IzfWUVQ4mNDMWIdObk0fKv/WZg/IexEF0XkniDsOxAfA0YiuPOz2hC3Q6vLmX52u/xiv+xZ4Mzy1Qn9c7XsAPU
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(54906003)(316002)(8936002)(2616005)(16576012)(4326008)(53546011)(86362001)(66476007)(38100700002)(66556008)(31686004)(5660300002)(66946007)(26005)(6486002)(6916009)(186003)(8676002)(31696002)(83380400001)(36756003)(508600001)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?lt0+ED7xobip3EtLHRqaIHZvtFbkefATb5/TGfMrNgTeFIRYj2nVY71y93qn?=
 =?us-ascii?Q?f9uS6LL5+N5d1GK624li3mhv7qGKt6ddb9IwEtSsyRfxVVtSZ37qcRuacRVB?=
 =?us-ascii?Q?eldBc4LMazq1+j8CQ5+puuiJ20awtHn7TqDV+ZFvTw6LFQA19Adj54TVPp3j?=
 =?us-ascii?Q?o+21J5Q61EqEs7n0Y4+UvrLCGLwq9JIcduJp8Sd1ujCoBr1wA9gBRWvx/B26?=
 =?us-ascii?Q?1hoOe/V9fsAF3wR5N40hyZCNA9laijSNejYTziME34/nyJSAsDNtAhjhKXfZ?=
 =?us-ascii?Q?ILtLp8L7XWnFXkRATy9bNmn7YCVcKCj0e7pyGWeQKk7TmwhntUBEGth9jRgx?=
 =?us-ascii?Q?H+CZWjx1GLjIibGAcc2W61p3j+MvO0a6JrvtbRyE4KMzbKoQHnxfS68cLF85?=
 =?us-ascii?Q?oODc++IckvK5A/pYxjjKWrDLrdczkJMz2jjWhibGWUbDXjAcaBytgT3kFZ8c?=
 =?us-ascii?Q?o0LVNeYLdVfeslqf6NXnh0ph7XOc/kO4GB7iZLvoE/l+ZScXhMUj3BtZ882u?=
 =?us-ascii?Q?DDg+UH31RVVg9yDvh+aO93ChtoTsmHN32NvIyOMls4rlS1F0oSlikNyenV5H?=
 =?us-ascii?Q?+1GvlMacXMcJIPjw9f6fRKaG7hQTC9Y7s0lahOqhHhd7yU+Ih7xuORB9voxh?=
 =?us-ascii?Q?MB6nzziUvlSNPNTK+AMCOGKL2ZB2hrKW0NR9uMUhkJerJxgQ6eTLJ3FjcWxK?=
 =?us-ascii?Q?pKiICT/urkeZAFY7s/NMMNMZEaNhug3RYN4oGffF9aFlu0ljFsDowfKSIl5p?=
 =?us-ascii?Q?huaPpfR1baiwTRtqTRjrfb83AbNxSqUPzV0iox7HaLTbHBMBWw1Y72E/pZUZ?=
 =?us-ascii?Q?6LigD56qdWDl78+UzDYTpJaMgqIkQAuodH49bCxFkeY01kc0r71bIcszGfit?=
 =?us-ascii?Q?7/GU3CeDNE/r1Okegwvf0wgDZ512IpCnWKmaYuVt7jCIFdpMHQT3uxdYXhGF?=
 =?us-ascii?Q?eCAJT8iXL07Wco1L9k8v1h+3QxHHoeyc58b6HiVsaotLr6KsKKGYJsGUQAvn?=
 =?us-ascii?Q?7L3m8oicMTlM21EuxY+JFm2GDGCmjj5oKu9lRUotD0l/YXWx1JbHQBNP9ed8?=
 =?us-ascii?Q?UR+cMTx72Ad2VVI1e4K48b/FxI72HETo6zn+yyFfXJbsT5Dr0yDrvasCGLb6?=
 =?us-ascii?Q?qdINQh9wvjSY96mtaATe0cYPtIn08GEcvyLMGlBoRCq274fOijDh7O/n1565?=
 =?us-ascii?Q?AWJMs3tX3ZHU02iu9RGvzvRexBHojKOQryQmkX3KDmj1Nkfg7MUXj8nWzbpA?=
 =?us-ascii?Q?ZM3B5SHU74KtCzc0jSqUDZTgRCoiwOAmltreFWaPIkrtfngYp8npD7+lqRbC?=
 =?us-ascii?Q?uSwDGr7Td8u2zkuBWyioeIhI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b6f31afb-8ae4-439e-2067-08d9720abbf2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 14:21:07.9731
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FKQZTtM7zK6vRRgi0XH7IYsVJkmeDsz94TFoej2/yp2//XooVP4YoClJhYQ3p3uEKY3RQi08oCDohgzoSopQ9Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5600

On 07.09.2021 15:27, Andrew Cooper wrote:
> On 07/09/2021 07:09, Jan Beulich wrote:
>> On 06.09.2021 20:07, Andrew Cooper wrote:
>>> On 06/09/2021 16:17, Jan Beulich wrote:
>>>> On 06.09.2021 14:00, Jane Malalane wrote:
>>>>> --- a/xen/arch/x86/cpu/amd.c
>>>>> +++ b/xen/arch/x86/cpu/amd.c
>>>>> @@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
>>>>>  			  c->x86_capability);
>>>>>  }
>>>>> =20
>>>>> +void detect_zen2_null_seg_behaviour(void)
>>>> This can in principle be marked __init.
>>>>
>>>>> +{
>>>>> +	uint64_t base;
>>>>> +
>>>>> +	wrmsrl(MSR_FS_BASE, 1);
>>>>> +	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
>>>> While I don't strictly mind the "m" part of the constraint to remain
>>>> there (in the hope for compilers actually to support this), iirc it's
>>>> not useful to have when the value is a constant: Last time I checked,
>>>> the compiler would not instantiate an anonymous (stack) variable to
>>>> fulfill this constraint (as can be seen when dropping the "r" part of
>>>> the constraint).
>>> This is "rm" because it is what we use elsewhere in Xen for selectors,
>>> and because it is the correct constraints based on the legal instructio=
n
>>> encodings.
>> grep-ing for "%%[defgs]s" reveals:
>>
>> efi_arch_post_exit_boot(), svm_ctxt_switch_to(),
>=20
> These are writing multiple selectors in one go, and a register
> constraint is the only sane option.
>=20
>> and do_set_segment_base() all use just "r".
>=20
> I had missed this one.
>=20
>>  This grep has not produced
>> any use of "rm". What are you talking about?
>=20
> TRY_LOAD_SEG(), pv_emul_read_descriptor() for both lar and lsl,
> do_double_fault() for another lsl, lldt(), ltr().

TRY_LOAD_SEG() and pv_emul_read_descriptor() don't pass constants
as asm() argument values. do_double_fault()'s use of lsl is indeed
an example matching the pattern here. lldt() and ltr() are generic
inline helpers, so validly allow for both because they should not
make assumptions on what the caller passes. Plus "m" there is okay,
because if the caller passes a constant there will be a named
variable (the function parameter), i.e. the compiler does not need
to instantiate any anonymous one.

> So ok - not everything, but most.
>=20
>>
>>> If you want to work around what you perceive to be bugs in compilers
>>> then submit a independent change yourself.
>> I don't perceive this as a bug; perhaps a desirable feature. I also
>> did start my response with "While I don't strictly mind the "m"
>> part ..." - was this not careful enough to indicate I'm not going
>> to insist on the change, but I'd prefer it to be made?
>=20
> No, because a maintainer saying "I'd prefer this to be changed" is still
> an instruction to the submitter to make the change.

It was a request to _consider_ dropping the m part, yes. But (see
below) now that you've forced me to re-check (I presume you didn't
check yourself, or else I would expect you would have drawn the same
conclusion as I did), I actually feel stronger about this needing
adjustment.

> But the request is inappropriate.=C2=A0 "Last time I checked, the compile=
r
> would" presumably means you've checked GCC and not Clang, and therefore
> any conclusions about the behaviour are incomplete.

Not really, no. IIRC I did check the version of clang that I have easy
access to. (For gcc I've just now re-checked with 10.x and 11.x.)

> Unless there is a real concrete compiler bug to work around, "rm" is the
> appropriate constraint to use, all other things being equal.=C2=A0 If the
> complier is merely doing something dumb with the flexibility it has been
> permitted, then fix the compiler and the problem will resolve itself the
> proper way.

I disagree. When an asm() constraint permits multiple kinds of values,
dropping one or more of the alternatives should IMO still yield valid
(perhaps sub-optimal) code (IOW every one of the supplied kinds should
be valid). The issue here is that it is not spelled out clearly
whether something like "m" (0) is actually legal. The error messages,
however, suggest this is intended to be illegal: gcc says "memory
input ... is not directly addressable", whereas clang says "invalid
lvalue in asm input for constraint 'm'".

IOW I do think the one case of LSL in do_double_fault() does need
adjusting.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:22:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:22:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181122.328134 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbzd-00072N-G2; Tue, 07 Sep 2021 14:22:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181122.328134; Tue, 07 Sep 2021 14:22:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNbzd-00072G-CI; Tue, 07 Sep 2021 14:22:33 +0000
Received: by outflank-mailman (input) for mailman id 181122;
 Tue, 07 Sep 2021 14:22:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tZpi=N5=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNbzb-000720-I5
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:22:31 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3002f295-b139-440b-a5a1-88b580df7c90;
 Tue, 07 Sep 2021 14:22:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3002f295-b139-440b-a5a1-88b580df7c90
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631024550;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=4JdChk7sqL2oApxqAlGiteoD3KNkhWtQhRpFCQ+N8VY=;
  b=H+mQnH5zu+/L9vKU+6lfbId3r2y53eivyzp6AbYJKTWgr8Ussxld8ED4
   6eCCW2BBB5xNXbz/Fcy4FnG2YS6I5a0ZBJECpAV93mXy0M0PbFjPX9JRC
   FPwcsWv6soKPWoe4jmOQc0wXQUWMYYFFOYuLt7Z8L3Yb11FPpaszRRik3
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: LA5RXG2VTDD8kaiTVULn1g4JGu2Bryc+aJ6sVvCuW0cZfV2HVkIJMXEF2ovF84au77QyqyCTXi
 2w+8JQoFDrYi+if5XpC3Ir3oTRiUNC0u/GRVm4vyE2aaMWJApUSbiEVgAxPRMWQzWmF4QcIEMi
 Heh1Hv8Ivfq0eifXuVngLVTxGrOc10YYy6V8kPem6zJxnVYeuLVQF/zlvZ3vJ1jRoRNoNZ8JV2
 Lq6vKpA2ZAfgESnfdZrR8eMfV6dWfWGmH/X+Ncc1fmByG7dCxMibFJIUDvGgxkhoHkr4rqE3Kj
 Dn58NISckNv3LcyM//YrhEfv
X-SBRS: 5.1
X-MesageID: 52563678
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:hhJAtKxiDUUl0/HmY+nLKrPw6L1zdoMgy1knxilNoHxuH/Bw9v
 re+cjzsCWftN9/Yh4dcLy7VpVoIkmsl6Kdg7NwAV7KZmCP1FdARLsI0WKI+UyCJ8SRzI9gPa
 cLSdkFNDXzZ2IK8PoTNmODYqodKNrsytHWuQ/HpU0dKT2D88tbnn9E4gDwKDwQeCB2QaAXOb
 C7/cR9qz+paR0sH7+G7ilsZZmkmzXT/qiWGCI7Ow==
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="52563678"
Date: Tue, 7 Sep 2021 15:22:25 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>, Julien Grall <julien@xen.org>
CC: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>
Subject: Re: [XEN PATCH v7 04/51] build: factorise generation of the linker
 scripts
Message-ID: <YTd1oV8kzz0MJ3L7@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-5-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20210824105038.1257926-5-anthony.perard@citrix.com>

(dropping most CC)

On Tue, Aug 24, 2021 at 11:49:51AM +0100, Anthony PERARD wrote:
> In Arm and X86 makefile, generating the linker script is the same, so
> we can simply have both call the same macro.
> 
> We need to add *.lds files into extra-y so that Rules.mk can find the
> .*.cmd dependency file and load it.
> 
> Change made to the command line:
> - Use cpp_flags macro which simply filter -Wa,% options from $(a_flags).
> - Added -D__LINKER__ even it is only used by Arm's lds.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Hi Julian,

Could you give a Ack to this patch? I think it only misses an ack from
Arm maintainers.

And if you don't mind, could you commit it with the two other arm patch
that are ready?

  - build,arm: move LDFLAGS change to arch.mk
    ( you proposed to edit the commit message before committing )
  - build: avoid building arm/arm/*/head.o twice

Cheers,

> ---
>  xen/Rules.mk          | 4 ++++
>  xen/arch/arm/Makefile | 6 ++++--
>  xen/arch/x86/Makefile | 6 ++++--
>  3 files changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/Rules.mk b/xen/Rules.mk
> index eaf770414305..3f61682ceab7 100644
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -255,6 +255,10 @@ cmd_cpp_s_S = $(CPP) $(call cpp_flags,$(a_flags)) -MQ $@ -o $@ $<
>  %.s: %.S FORCE
>  	$(call if_changed,cpp_s_S)
>  
> +# Linker scripts, .lds.S -> .lds
> +quiet_cmd_cpp_lds_S = LDS     $@
> +cmd_cpp_lds_S = $(CPP) -P $(call cpp_flags,$(a_flags)) -D__LINKER__ -MQ $@ -o $@ $<
> +
>  # Add intermediate targets:
>  # When building objects with specific suffix patterns, add intermediate
>  # targets that the final targets are derived from.
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b440..0edd9dee6f49 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -66,6 +66,8 @@ obj-y += vpsci.o
>  obj-y += vuart.o
>  extra-y += $(TARGET_SUBARCH)/head.o
>  
> +extra-y += xen.lds
> +
>  #obj-bin-y += ....o
>  
>  ifneq ($(CONFIG_DTB_FILE),"")
> @@ -133,8 +135,8 @@ $(TARGET)-syms: prelink.o xen.lds
>  .PHONY: include
>  include:
>  
> -xen.lds: xen.lds.S
> -	$(CPP) -P $(a_flags) -D__LINKER__ -MQ $@ -o $@ $<
> +xen.lds: xen.lds.S FORCE
> +	$(call if_changed,cpp_lds_S)
>  
>  dtb.o: $(patsubst "%",%,$(CONFIG_DTB_FILE))
>  

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:27:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:27:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181128.328144 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNc48-0007im-2U; Tue, 07 Sep 2021 14:27:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181128.328144; Tue, 07 Sep 2021 14:27:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNc47-0007if-Vg; Tue, 07 Sep 2021 14:27:11 +0000
Received: by outflank-mailman (input) for mailman id 181128;
 Tue, 07 Sep 2021 14:27:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNc47-0007iY-4M
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:27:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id af340802-0fe7-11ec-b0fa-12813bfff9fa;
 Tue, 07 Sep 2021 14:27:10 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-Kvl8OVOJPsO3QdgAF5zH2Q-1; Tue, 07 Sep 2021 16:27:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5902.eurprd04.prod.outlook.com (2603:10a6:803:ed::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 14:27:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 14:27:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0010.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 14:27:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af340802-0fe7-11ec-b0fa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631024829;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/cnaseTopJc0M3gFnFozGKtr9ArBNZkh7NEJ4U8bQHY=;
	b=RZxTwzoK+lW3/WvN1gd/XAzCHSvXMY6ejiJ9FBFrvzz1OTlbXe6hvW1at1MHVrB4W6QQfa
	3ayCvUgOPIOprUkTTmsGeuA+VLem67x337UOlmUUit7eTc437Fjat2hSpJ1IqOskDyPkwa
	c3sxNx5B2vu++pjRDH40QTgUORx5SlE=
X-MC-Unique: Kvl8OVOJPsO3QdgAF5zH2Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UD29ztSBhiSO4Zn2b98F5Wm0ibcuYUAJdcXChpqY2WKA8DFeBkdJ6TQIS86QvLSXTEF0eBt3vuvm25LEgxVu8S7f0OrSIvOK8Ztqn/cNvVKSWSXbcAPI3lQEYekm77rUhuD9iE5609dCDHhtPNPBILWQBaejvJGPR5ZIaWfHFkJNOKIXkhKuqEDqe3jyhWLL6Mxa969BVhQPBf95vfedu/uOra2r5Y8UaRLYw6drTQMxArLARVsVHeV/uil+mF8GrN7H4n8pzGPwWGZ0bfpj6sjublabtZGgx7+8Ue8fhkfXGDt/cFszQqkGEKVNy6dEH+sOb+BUu79CNYQG8skaRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q1oeAetJ19oaQfdWFwMctncqRNBSqpmyqIffsfr/z08=;
 b=O7+qhpaaGHOKIpSuxev6l4M+etsTAwpO+rZwaHXkEVgd3V4CEBPjYYuyZu7g8LZdkyN37Sa8GxDhXlTX1bMPkaxQAmGOlB4yvYr9Ul5dxPHfhP6WFPiCNuBlKRqay0b4HTxehQFzLmsSkGatJpGnj29rVquFRyXhCNsHSP9oIoFC3xh3kDNHb1hkbkzDBlCvsgvGI2saRJflbEBaukpgcl2G87zhsxsCJgxNc7mKyJnWJDR98LDURoVB4XZ/Z9mQup1NLAhEi3zKjJpeOVbgcAD8rZCB0Wu/zcCVexZe924WdEYXYQouuftiBOMxRWnJzStd3PxL0uHMM12b29Yvwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
 <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
 <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
 <3b3762a7-5034-9094-84f9-cd8ab24e24a0@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e006dc60-7745-43b7-f725-2390e174bfa7@suse.com>
Date: Tue, 7 Sep 2021 16:27:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <3b3762a7-5034-9094-84f9-cd8ab24e24a0@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM8P191CA0010.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 304e346f-66ac-4ce0-d10a-08d9720b917f
X-MS-TrafficTypeDiagnostic: VI1PR04MB5902:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB590251198FD0B6555A217027B3D39@VI1PR04MB5902.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lJ3mye5uRFfqlaCzrlHp+dwV/+/2hR3nJXlrzF93E4wnRhO9amZ713IL+kYU2bYCQg0r6aTjE07BbZoacR0fF4rF+cjlaS4/VoGC6HzR1G9ryXMeKQmzwLG+8s+oJrf/AtyitqgoPA2DZovdB6uU3BGyJYesZSz7pzSuiCrIoC1JQtVU9pPMSI7NbVW+s0rWZj0OrTT0voBfrjJ3BR+zBWO6wWMjLYqfg1OnQUB5sEiXPYko94u8/mbUQ0DirharEEmvCC18HmAijXiEal5JpyzO4Zr+zCTb7TzubkeOrAeHkc3JErltfLQkplGjJ7lmOvM3eRxXuoDm0RA//9G/9WPspko9vwOgcerpjLeUpaex2hfacEhIU32oyEm34q2k6R393NOac4209aJjZizAa5RmlgQYpiDv9gUfN8rEe+DHQN8/7KmFR0ddtNhJEESyULP7IxgeILaHJM46w1zrAU5oGmohlE3HZkP/Lvj2UzX4upl3yfoKCrVGhW/CRI3KaPEAh7KeM2bXY7X2HJFAq+pMs+FFXANXI6vNDrdPA2f4s6WRJaDnOf7j8gTKjGtbkYvfJ4hYD+2VJyj+pmKfG64i8iwii/viWzgCESCTcUcX95UjG7bbtpbQ+DilZ9uiLMbkXlTMv00ZAqFO752tr4IRQAZVFdxuDrMYBL9EbjISjIYTc7jkj2q9MqW5Xq1EijlN3VKGhpS8cNgWgG8Q+nMMJBMiKAnmKZARwZLOwPdUZsEjspRlM5Ivg+bs8QvT
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(136003)(376002)(396003)(39860400002)(31686004)(66476007)(66556008)(66946007)(54906003)(8936002)(6486002)(956004)(2616005)(16576012)(316002)(478600001)(5660300002)(2906002)(36756003)(31696002)(8676002)(83380400001)(38100700002)(86362001)(4326008)(186003)(53546011)(6916009)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?W9/SUaQaResgfhQ45BmHstxWNKBE9V2WKrRh8s79NN5uYCN21ZLqi/9JafwV?=
 =?us-ascii?Q?bBGn9zJjqnOmzrilEhehcQKUAgC5EOj39HOmr2N8wXIMgNISMHJrpvfa54Ck?=
 =?us-ascii?Q?AJl9T99GAOy11UdF3j/aaWAxrEWGH5p0SDgUA6woL4s2nRfRitsJ6RlcftoV?=
 =?us-ascii?Q?lZ6zhRKWAog13VNOo497qSbAbYOB+lKXGF9TPb0slhVVsLpIB9grxrq+d1fP?=
 =?us-ascii?Q?6WzOdGJgqIFH0Nxc0miiuso0+Km/gH/IlWEm3neGqVo0uRKz4iTKsGGqqL4Q?=
 =?us-ascii?Q?wp3wmvPHGHAGotUGkUjmNbJOEmAGBbeM4U8kVCiJ+c8pvN6dCw9mVLjQ6GaO?=
 =?us-ascii?Q?erzDGTXgJT5B7feLAwMoRHuDnH82CMoIvMM0rbA8+S2bnRG+Utr+E69X+zoQ?=
 =?us-ascii?Q?o8dGAR3fdImIycwFBwb4MYcThg+xd4c2gVc7JAwtd9hBNntJDgrahy0hBv5Z?=
 =?us-ascii?Q?7XxOP+tX6WbBluEVR/GhjwctaHPkNYVPLsqXKKHS7lsWiIqCsh8dopHNHaA1?=
 =?us-ascii?Q?QKmX9vbjt5W7qxeIjf4JK5jJxm0p1vB3zmpB5jDaengS8bMyclSSIIRhmEF3?=
 =?us-ascii?Q?3KDQkkA1QeKFjX4tqsONlqRMr2LYaACPQkurGwg5x2Sb4AWD8B7J9Vrnwv1p?=
 =?us-ascii?Q?TXXxzNr4apoMk0UbG35ADBAJI7p79fJhcfOE/0uYh6ThxJMjs6obzTCbxPHq?=
 =?us-ascii?Q?xVmXSzNTB6aqcHrOh0CuOLjEAr6gJBRgPOqK1zhse7+/B0qjssKBSfS7DUIZ?=
 =?us-ascii?Q?gzGCsJyJxLUn/NdHKd0ILXJIFNOYSYlEK1qczFvsIb51h7dvTmUrwFo/gvLM?=
 =?us-ascii?Q?JkPl0r8OwR2rAJjP+SGABEJ+OO6pWqF2MAZg4xzqJmgWMpuMzIkMvdg2URUl?=
 =?us-ascii?Q?Evbf/7bpbzg6c9HW1hoUGXIGJiQpajpmtHBrEIUzZuCPHGNOL4hgjPsvqFMc?=
 =?us-ascii?Q?29zc4aDLi6kQAwRHUJ4TrhlGOLUm3kffrGfVsTC8k8BdVZICBT98tncyHChn?=
 =?us-ascii?Q?i7BiI+W8qJ6RPZIAt4x3hQf1MSjsm3S3SJlfzCeDETfGbBqMYEaImtWp2wwH?=
 =?us-ascii?Q?vfBOHK2egV1lTCS5mOfHTrMWgafFV/5dZevF2861M0UYDuZ/HQkvGOl2ZgUp?=
 =?us-ascii?Q?CZH0m1q7i1xpsnAazrcocT38ST4GT8Pk0L9Eio1KUhQ9CYjIGBWslf3gP4e/?=
 =?us-ascii?Q?16Qt+h8Z19TUzmuP6miaWIHViRVkqINjf+X5BA95SEzLwaySKdY6Jbs3JqrM?=
 =?us-ascii?Q?EgFA7m6hX/iM69WoKkSfq0VJuunBxWEWVpLT9+qWKdoqMqcPcq5Wz75fdKE1?=
 =?us-ascii?Q?jhtbhr6h9OWTEorJuTirx+IJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 304e346f-66ac-4ce0-d10a-08d9720b917f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 14:27:06.1904
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: spKIiJ+n5MePkrYN3RMGCDcbAcqeZL/5BvUoFR5XidiCj6OxWZ63qCk1+3Zt8PDH3yWoWD3zC5AnE0MBCIkYIA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5902

On 07.09.2021 16:09, Daniel P. Smith wrote:
> On 9/7/21 9:50 AM, Jan Beulich wrote:
>> On 07.09.2021 15:41, Daniel P. Smith wrote:
>>> On 9/6/21 2:17 PM, Andrew Cooper wrote:
>>>> On 03/09/2021 20:06, Daniel P. Smith wrote:
>>>>> --- a/xen/include/xsm/dummy.h
>>>>> +++ b/xen/include/xsm/dummy.h
>>>>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>>>>   =20
>>>>>    #endif /* CONFIG_XSM */
>>>>>   =20
>>>>> -static always_inline int xsm_default_action(
>>>>> -    xsm_default_t action, struct domain *src, struct domain *target)
>>>>> +static always_inline int xsm_default_action(xsm_default_t action,
>>>>> +                                            struct domain *src,
>>>>> +                                            struct domain *target)
>>>>
>>>> The old code is correct.=C2=A0 We have plenty of examples of this in X=
en, and
>>>> I have been adding new ones when appropriate.
>>>>
>>>> It avoids squashing everything on the RHS and ballooning the line coun=
t
>>>> to compensate.=C2=A0 (This isn't a particularly bad example, but we've=
 had
>>>> worse cases in the past).
>>>
>>> Based on the past discussions I understood either is acceptable and fin=
d
>>> this version much easier to visually parse myself. With that said, if
>>> the "next line single indent" really is the preferred style by the
>>> maintainers/community, then I can convert all of these over.
>>
>> I guess neither is the "preferred" style; as Andrew says, both are
>> acceptable and both are in active use. I guess the rule of thumb is:
>> The longer what's left of the function name, the more you should
>> consider the style that you change away from.
>>
>> Anyway, in the end I guess the request for style adjustments was
>> mainly to purge bad style, not to convert one acceptable form to
>> another. Converting the entire file to the same style is of course
>> fine (for producing a consistent result), but then - as per above -
>> here it would more likely be the one that in this case was already
>> there.
>=20
> Understood, I will respin with all the function defs to align with the=20
> "next line single indent" style, though it would be helpful for=20
> clarification on this style exactly. Do you always wrap all args if one=20
> extends past 80 col or is there a rule for when some should remain on=20
> the first line (function def line)?

I don't think that aspect has been discussed. I would say

void sufficiently_long_attribute test(unsigned int x, unsigned int y,
                                      unsigned int z, void *p);

is as acceptable as

void sufficiently_long_attribute test(unsigned int x,
                                      unsigned int y,
                                      unsigned int z,
                                      void *p);

with a slight preference to the former.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:29:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:29:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181134.328156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNc6M-0008Kl-Go; Tue, 07 Sep 2021 14:29:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181134.328156; Tue, 07 Sep 2021 14:29:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNc6M-0008Ke-Cc; Tue, 07 Sep 2021 14:29:30 +0000
Received: by outflank-mailman (input) for mailman id 181134;
 Tue, 07 Sep 2021 14:29:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNc6L-0008KY-D3
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:29:29 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 01bc4f6c-0fe8-11ec-b0fb-12813bfff9fa;
 Tue, 07 Sep 2021 14:29:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01bc4f6c-0fe8-11ec-b0fb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631024968;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=lGl+w9RZI0yp86LvWEJpnlZkmHmp3WZZhe19+vSleZA=;
  b=bFsar9cQCXdcdd2pvyq+aRSAZ2IOClsuFqKLVj1Dfkkq2TPYdER44FNz
   sy/TmdCT9c3gAOmpTEaRByO3VeG4yv/t0BQMuzZojWslQEoxb56HU00MN
   ACFtgBpR961QiALUr1wq0Wq991TZ1hf+SrhSU7IN6gGWBs52gLgzXTUrZ
   U=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 5G8ZHOZDQ3UqttNSD/nRZIat0DCwuD8jlbOvjimFqOVhYrrHEuzc+DivD1AzVvCyeukkGzEz5P
 DS8sg3h8jVBnB+fWxASj0PB5PLcqpbNSLjlEWt9AKfMAb2E9ydNHGpndmO9VeSYx5lxJIteQbI
 pWhtVqRf1wlursJasycsPg5DNpb20RRDlSsgKa5bO5TGDGqnWvL8REC+cpX0Ob/HDcKrpWLStM
 A4aEY+e5EEv/hH0LpQUv7gZ4uwg1twkQx7XF9ZiVQnsv/UXMZsaUoFhUiugj0yBSHmC2TfOl3c
 9okHltYbV+aigI0QcNG34S6o
X-SBRS: 5.1
X-MesageID: 51764707
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:zQzsKqC7ICIJpFTlHeglsceALOsnbusQ8zAXPh9KJyC9I/b2qy
 nxppgmPH/P6Ar4WBkb6LS90dq7MA3hHPlOkPYs1NaZLXXbUQ6TTb2KgrGSuAEIdxeOkNK1kJ
 0QDpSWa+eAfmSS7/yKmDVQeuxIqLLsndHK9IWuvUuFDzsaDJ2Ihz0JejpzeXcGPTWua6BJca
 Z0qvA33QZJLh8sH7WG7zQ+Lqf+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+SemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lkdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNxN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wiJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABCnhkjizy1SKeGXLzMO9k/seDlFhiXV6UkXoJlB9Tpc+CRF9U1wra7USPF/lq
 /52+pT5elzpmJ/V9MKOA47e7rCNoX6e2OFDIujGyWTKEg5AQO7l3fW2sR52Aj4Qu1F8HMN8K
 6xGW+w81RCIH7TNQ==
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="51764707"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e2FAZXfta6HueaFIwDivZSRLlKn79/z9UJaRxDffmo4v20ClQkQ260H1LcfwUF0DK7ErXvMuF6cULkgZp4lSCYA0GtmjIrCaV98eYK5bFRsQ3eFjwkiv9cnYxY/6ocytfJJrKcEt3CoqJQEOZHBfMqgujmrky0p5IvUd3fFHvqKVbDnP4U+gro/Zi01bXusXcekZG/G4u0DjAi+/Td8Fw8R8sWZQQpfg8KT3yEqDl62zwdOeqNPM1vYUJyuaiuzEjq3wLm6t5PUQ6cQUuNT+PLdwYxjlDQSEcOz+XRt2/glbGa4/27LjSFesyDMRifffdxkwBmLEqDs1NvQnGFA2Ig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=E9qaqhkQxxacAGQ126F758E9SkuBmd5PlZ0+MFHQQXw=;
 b=J1+vnW1lZEcnDh7lOaMgxcjuiZ8/1oZrw4ScVKoSPftnoTxTkrignCQzlXYDNL5UY5Nr0txiQwiXjm+myw1TeEBhxBN+N5p0Icp5jfCTth0YdoEBLKnWBti5z4lmL/jlYEbRanJ5cQe75RausrJp9/h5run9PfSKFtH/jTW0/oTWaIxjW7kHNGqtIcoWAeRAJWW98aiJrRfKkcAWbng5YIM/yxMPDQgfX5tFYoocNw9HRUzpJMhhePuTAk8GU3g8cpuG8k39jkMnLBcEQyfKAk+c6vsHVqAk1yh4C9GFKygpeXQ6nHsa3k8Kf+mtcbADnByICZBM6iCqaC1HaI5TfQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=E9qaqhkQxxacAGQ126F758E9SkuBmd5PlZ0+MFHQQXw=;
 b=CeUbcAcbVb9KVc7GJt9BPKIw8X8tibn9RvFBQpwgVGpUY4Q1Csy6819EMrtrxre7/rSF85Cn1li8BNcvH7+IPNSR8of3F9JGpPxXB7IL+mLzf0xWEKDVczLUZgPYIU+Itc8ZT9RDzj+m5aYrTWukYW8s0ZlgSkxBk7YPxfCylaM=
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210817143006.2821-1-andrew.cooper3@citrix.com>
 <20210817143006.2821-2-andrew.cooper3@citrix.com>
 <1ba952b1-ad26-79e1-7aa9-af7df28774f6@suse.com>
 <34905dc6-f8e1-afdd-7f05-1c8e93eab3ff@citrix.com>
 <709c7d54-2088-415a-6460-bf44002d1d76@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/3] x86/spec-ctrl: Split the "Hardware features"
 diagnostic line
Message-ID: <074cf055-6e8f-4ec8-692b-2cd0e3f11b7f@citrix.com>
Date: Tue, 7 Sep 2021 15:29:13 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <709c7d54-2088-415a-6460-bf44002d1d76@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0428.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18b::19) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ba221d1e-303e-46d7-46b7-08d9720be056
X-MS-TrafficTypeDiagnostic: BYAPR03MB4247:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB42476B2A2781514419977F48BAD39@BYAPR03MB4247.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: ZNMGI+dEwEkTNjGF3iyVqsxgiqC8PMdJcvip1GuYXvgeXDBGml0b4SsUxOUpk9reqH4CYhUaodMy5EWo1PwGVEUYl9zckICXwRkVIivE5VA4G7IgFUv79qoUYn4oDzqjKQO/86i9D09wWO4lh/3cfrjff4ptCaNS9LGgilx+4v/2QPXswq1LdrsV98uyGAo1FdPA3zJLqwEiWJ025zCQFg+YRwtITpQNmU/tiFyHYmgya1e9g1n5GG1iGpOpFFraFflsr3g8DHkDNG8I8jhUIY2vB7ZTq9pYEB78H8uwL4LuHwF4Qd2JPggpjP7I5dDui2KYR6cF/ZnOwxIyKSAdi5Q0KMz5p3IWrbF5wCwKK7vF31SAISTtkZQba68OFsHDLlMse4V9AqQ+PSFWO62q7zHMAM8Cu/28rjyEyWjmX4336xU5KnYf+1cHcX89AvsWrsjdIqrZhFcpVClR28sU9+Yzyqw1H8D9MLZxkU77veU/h0BJOx9NUv0Qvi7KcCeyxVgW6AehK6PoC2UyEh/JS6Ini40+bbYWl7GDM23EsAXCAkypdmVGoJ56Oosgd4j4+0h8m+MEsynYhREC+RF6V7UsrXH04ot8OC4g/+0OZYY+DPbjgT7kl9wG4QhRqVjqmLXVgZ4O5ECRTdZoKTNm4jra0w4TXWuYBCYddsvZCW9YiyeTwr4e1d/4l2tSWFkez7kazia/zm0HCkh0U1/I0r5liXw87aWFXS8OgjztYDk=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(396003)(376002)(366004)(136003)(39860400002)(8936002)(83380400001)(38100700002)(6916009)(186003)(36756003)(4326008)(86362001)(2906002)(956004)(54906003)(26005)(6666004)(53546011)(2616005)(8676002)(31696002)(66556008)(66476007)(55236004)(6486002)(16576012)(31686004)(478600001)(316002)(66946007)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U3lFY0NmeUMxOGZzYXp4OXZwRnIyQ3h4VFJFWjE0MTUzVDJvbWxMMVJtdC9h?=
 =?utf-8?B?VVFnRDJhQmZMRU5PTmtpSVZBQVQzUi9BTkNMNWZPNWRlWWxpQWxTc040LzVk?=
 =?utf-8?B?VFZDU3hhS1IwNDNZRnZ1RmY3YnhVS0c3ZVVqOXhUSldZYTA1SHk5UjVDWk93?=
 =?utf-8?B?TmN1R1RySEg1Z29ZdEhtalRRSDZIUThIOGJocDVCOWZFakVTWmFySDFiVzg2?=
 =?utf-8?B?a1c2eW1UbnZuMDhOTW5rbXh0bVUvcnMwdVd5Zy9WTURzd05kbTNYbG01UGdl?=
 =?utf-8?B?WE5MaUJXMEF4ak9wOExQdHZtdThMOWdVVVRVN21abCtrZkcycnVsaWFTak5x?=
 =?utf-8?B?TC9BK3BUdjlFQVdZRDFSL28wWUxwV3FRQXZhc2N1L2ljMUJTSGdLOXlmeFMx?=
 =?utf-8?B?djRkUGxyTmlVZngxOC9sbU00RkdkRVIwb0swRWJSQ0FWd3R3VzhsU0M2ODVP?=
 =?utf-8?B?dFlsVXN0VEEwanNoWGVCUngvMWZxZm5BUy9oaHFGcHJlcUxTeTg1d3FlVDNr?=
 =?utf-8?B?bW1jamVDV09FTUpLV2JLaDhDMzBETnN4ZVlzS2JEVkVuNGZHSUdxNk1BbFJn?=
 =?utf-8?B?Sm1aVGpleVpoRHhLYmhyWUh5STF2blVISi9yMHZkL1BCb3BmcEpSVTBxaDlR?=
 =?utf-8?B?dU90Tkk2R1YxSkFOMndQK21ZZ0JtNmZ0VUdybHF3SnJUOGtETG4xTGI4dEla?=
 =?utf-8?B?dHUxRnIwTyt2MEVHWGVNeTdIVGhwSDNIMTFOVXZxdjRSQk95aThFRWtoODVt?=
 =?utf-8?B?ODNvbmd4NytrWlpXUnlYMTZMQXlKVllGNCtYakZqSHhwWEFGSC91NVFicVRZ?=
 =?utf-8?B?MS9NV3p1S1E1eXRuOWNLOGFWb2NKZWRtSHdIdHFzcC9LYXhZayt3NzRzMzBH?=
 =?utf-8?B?MVN3a3FtMG1LelFMdEsyeTdCYkJMOGFRNzdTVDJzSVozd0pVQkNMazJPQkhy?=
 =?utf-8?B?bWw3Tm0ybEhvMFdzcHhHdFhjQ2l3MGFianNJODZ0UHhia1JNMi9wM2YxR3Vs?=
 =?utf-8?B?R1RZTFIrVjRaOG5UVHhNLzVRemlxN0NKeGEyZDg2dmpNYXJrQkRseEY5aFZI?=
 =?utf-8?B?Q3VxVHFVWUF5QnhET0ZtMnNVNmUzakg1a3lSc2NDaEUwWTFuaXcwNmZZeVFq?=
 =?utf-8?B?VlBlQmorTEFYVCtranRMa0JRcHZRNVNQQXlsWVUrUXZmSmJTR2cyclFNRTlW?=
 =?utf-8?B?M2IrbUZtNzFiZWhMOHBNczJvUHF4aTA5dXhubEhhTmQwaEw3NS9Ja29zN1FE?=
 =?utf-8?B?TFZPbHlJV0U0bFJqWjdmakFRQkVTWkpaZjQvK3RBaEcrZzZRRSt6MkRnNG91?=
 =?utf-8?B?VGxLN21KSVZtRXc0dXF0bStrcWdHUlloK2p5VWdZWEU5NitJZkU1WmlvUWJM?=
 =?utf-8?B?YU1DNU16R1RvWkFaaGVIMDg2Zk5NMnY4cTBVZG1FUDhzYTRidXFsbDViQ1hB?=
 =?utf-8?B?bDZ1dHlVVHFOTGhOM1M0SG1BeklEWGhOMjhZbGl2NnAwS2Zid2lzUEV6U1pU?=
 =?utf-8?B?YjJJSEFrUDk5cUI4OHRURVpuRXBTdE1mMm9MRzNTaFZPYTJMTHZURDVVdExz?=
 =?utf-8?B?em4yR1luT2IvZXprMzhUMkthaGFaK0hsNmgzeTJoRThtWm00VE5GQ05ycTNC?=
 =?utf-8?B?YTJjMDM0ZGt3cDFtZTJDajVHTlhNWTFkU0hLdE92NkxjK2ptYkFHVmQ3dndh?=
 =?utf-8?B?NkVHYkZFYUxLaEJqM1lmRFRWR0xhcGlEV28rc0JpOUp6RWhyaUJQMGtvYmpj?=
 =?utf-8?Q?hge0gBZx6oYQhBaIUXxfnKoXyHaTvZQV1HET81r?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ba221d1e-303e-46d7-46b7-08d9720be056
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 14:29:18.4742
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 21uEQhV3ijmmgJ78MfIb81i+8Od03MNexR5P9dbq1eLRsidAZNJ/aANG52JupMfKkXt/D+H5k/7YbI5liqFCvKZ8R9O7emp50tLEdBsadnA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4247
X-OriginatorOrg: citrix.com

On 24/08/2021 14:15, Jan Beulich wrote:
> On 24.08.2021 14:57, Andrew Cooper wrote:
>> On 19/08/2021 15:38, Jan Beulich wrote:
>>> On 17.08.2021 16:30, Andrew Cooper wrote:
>>>> --- a/xen/arch/x86/spec_ctrl.c
>>>> +++ b/xen/arch/x86/spec_ctrl.c
>>>> @@ -317,23 +317,30 @@ static void __init print_details(enum ind_thunk =
thunk, uint64_t caps)
>>>> =20
>>>>      printk("Speculative mitigation facilities:\n");
>>>> =20
>>>> -    /* Hardware features which pertain to speculative mitigations. */
>>>> -    printk("  Hardware features:%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : =
"",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP"     : =
"",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH)) ? " L1D_FLUSH=
" : "",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_SSBD))  ? " SSBD"      : =
"",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR)) ? " MD_CLEAR" =
: "",
>>>> -           (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL)) ? " SRBDS_CT=
RL" : "",
>>>> -           (e8b  & cpufeat_mask(X86_FEATURE_IBPB))  ? " IBPB"      : =
"",
>>>> -           (caps & ARCH_CAPS_IBRS_ALL)              ? " IBRS_ALL"  : =
"",
>>>> -           (caps & ARCH_CAPS_RDCL_NO)               ? " RDCL_NO"   : =
"",
>>>> -           (caps & ARCH_CAPS_RSBA)                  ? " RSBA"      : =
"",
>>>> -           (caps & ARCH_CAPS_SKIP_L1DFL)            ? " SKIP_L1DFL": =
"",
>>>> -           (caps & ARCH_CAPS_SSB_NO)                ? " SSB_NO"    : =
"",
>>>> -           (caps & ARCH_CAPS_MDS_NO)                ? " MDS_NO"    : =
"",
>>>> -           (caps & ARCH_CAPS_TSX_CTRL)              ? " TSX_CTRL"  : =
"",
>>>> -           (caps & ARCH_CAPS_TAA_NO)                ? " TAA_NO"    : =
"");
>>>> +    /*
>>>> +     * Hardware read-only information, stating immunity to certain is=
sues, or
>>>> +     * suggestions of which mitigation to use.
>>>> +     */
>>>> +    printk("  Hardware hints:%s%s%s%s%s%s%s\n",
>>>> +           (caps & ARCH_CAPS_RDCL_NO)                        ? " RDCL=
_NO"        : "",
>>>> +           (caps & ARCH_CAPS_IBRS_ALL)                       ? " IBRS=
_ALL"       : "",
>>> I take it you flipped the order of these two to match the ordering
>>> of their bit numbers?
>> Yes.=C2=A0 IIRC, the first draft spec had the bits in the opposite order=
, and
>> I presumably forgot to flip the printk() when correcting msr-index.h
>>
>>>  I'm slightly inclined to ask whether we
>>> wouldn't better stay with what we had, as I could imagine users
>>> having not sufficiently flexible text matching in place somewhere.
>>> But I'm not going to insist. It only occurred to me and is, unlike
>>> for the IBRS/IBPB re-arrangement of the other part, easily possible
>>> here.
>> dmesg is not and never can will be an ABI.
> Well, sure, I understand this. I said "slightly" not because I would
> use the log this way, but because I know of at least similar (ab)uses
> of logs.

Lots of details which are currently only available in `xl dmesg` should
be available via a real hypercall.

The domain creation improvement work is making headway with retrofitting
proper details to virt_caps/etc, but there is loads more data needing
exposing.

>
>> Amongst other things, `xl dmesg | grep` fails at boot on large systems
>> (because you keep on refusing to let in patches which bump the size of
>> the pre-dynamic console),
> And instead I've taken the time to reduce boot time verbosity. Plus -
> the last attempt must have been years ago. Given good enough arguments
> and little enough undesirable (side) effects, I'm sure I could be
> convinced. (But yes, especially the "good enough" aspect is definitely
> pretty subjective. Yet then I could be easily outvoted if others agree
> with the provided reasoning.)
>
>> or after sufficient uptime when the contents has wrapped.
> The boot log can easily be made persistent on disk before it can wrap.

All failures we've had with customers are the fixed initdata buffer
wrapping before it gets dynamically allocated.=C2=A0 As a consequence, we
unilaterally up _CONRING_SIZE to 64k.

And yes - reducing the verbosity of the ACPI tables is a good thing, but
there's plenty more in need of shrinking.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 14:55:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 14:55:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181143.328167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcVm-00044Y-M6; Tue, 07 Sep 2021 14:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181143.328167; Tue, 07 Sep 2021 14:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcVm-00044R-J1; Tue, 07 Sep 2021 14:55:46 +0000
Received: by outflank-mailman (input) for mailman id 181143;
 Tue, 07 Sep 2021 14:55:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WTX3=N5=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mNcVl-00044L-JE
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 14:55:45 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ad36f8f8-0feb-11ec-b104-12813bfff9fa;
 Tue, 07 Sep 2021 14:55:44 +0000 (UTC)
Received: from [10.10.1.24] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631026537886258.94692952877904;
 Tue, 7 Sep 2021 07:55:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad36f8f8-0feb-11ec-b104-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631026541; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=dEuYpSadwO6BWXXzFGESW3zKszBfn9BDjg7PbivaKnzxLZl/ObSB9DcqOqVidWDdDCAcmHH2VERSm4nbSneNS4EhKWzNa+cSdRI72JggfBjIUGsl0jZ/QleeX2z678y89rgwpexpvkp3k1J21L7m/9RpPCEnAWPLl1UF+/YDgyQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631026541; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=1S9g/DAWEke9fgbmngX/mer48f8cYnTvTwR4FI8XbP8=; 
	b=ZZYPoDEO2umi+nMxueIX3gVflBRtsPUhFJWtVtoNFqxDJEjQBfaySLrsD1O1RaMwW/TTFzVzSAdCW+5fR7Mj3yVPiU+sPUwYFOzyd/1sRK+Lzq78h3RrfUWaRQVGPS0ppFUR1fpw0UHf6C3SAIVBTdTnIUk9AbF7N9RLNfHuBCE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631026541;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=1S9g/DAWEke9fgbmngX/mer48f8cYnTvTwR4FI8XbP8=;
	b=P1acJ6j4P1gKWNaDrgPeeqX4PLdqzbbu5JLUFK+E0QwcfdSrMP8vCRACvBwvzm1H
	Hhb3ypw/bFvSqw9d4DhrsAvqij4DYIqvSXDvEuz0qd4PI/tSOVfxA5ZxnrPJVVCYmtW
	BcCDVVnZjVRB0/dq/WEbk4Xc9DEpOecpM2hsDvwU=
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: Jan Beulich <jbeulich@suse.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
 <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
 <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
 <3b3762a7-5034-9094-84f9-cd8ab24e24a0@apertussolutions.com>
 <e006dc60-7745-43b7-f725-2390e174bfa7@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <bf876f84-aa0f-faff-da63-0f0525087883@apertussolutions.com>
Date: Tue, 7 Sep 2021 10:55:36 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <e006dc60-7745-43b7-f725-2390e174bfa7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/7/21 10:27 AM, Jan Beulich wrote:
> On 07.09.2021 16:09, Daniel P. Smith wrote:
>> On 9/7/21 9:50 AM, Jan Beulich wrote:
>>> On 07.09.2021 15:41, Daniel P. Smith wrote:
>>>> On 9/6/21 2:17 PM, Andrew Cooper wrote:
>>>>> On 03/09/2021 20:06, Daniel P. Smith wrote:
>>>>>> --- a/xen/include/xsm/dummy.h
>>>>>> +++ b/xen/include/xsm/dummy.h
>>>>>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>>>>>    
>>>>>>    #endif /* CONFIG_XSM */
>>>>>>    
>>>>>> -static always_inline int xsm_default_action(
>>>>>> -    xsm_default_t action, struct domain *src, struct domain *target)
>>>>>> +static always_inline int xsm_default_action(xsm_default_t action,
>>>>>> +                                            struct domain *src,
>>>>>> +                                            struct domain *target)
>>>>>
>>>>> The old code is correct.  We have plenty of examples of this in Xen, and
>>>>> I have been adding new ones when appropriate.
>>>>>
>>>>> It avoids squashing everything on the RHS and ballooning the line count
>>>>> to compensate.  (This isn't a particularly bad example, but we've had
>>>>> worse cases in the past).
>>>>
>>>> Based on the past discussions I understood either is acceptable and find
>>>> this version much easier to visually parse myself. With that said, if
>>>> the "next line single indent" really is the preferred style by the
>>>> maintainers/community, then I can convert all of these over.
>>>
>>> I guess neither is the "preferred" style; as Andrew says, both are
>>> acceptable and both are in active use. I guess the rule of thumb is:
>>> The longer what's left of the function name, the more you should
>>> consider the style that you change away from.
>>>
>>> Anyway, in the end I guess the request for style adjustments was
>>> mainly to purge bad style, not to convert one acceptable form to
>>> another. Converting the entire file to the same style is of course
>>> fine (for producing a consistent result), but then - as per above -
>>> here it would more likely be the one that in this case was already
>>> there.
>>
>> Understood, I will respin with all the function defs to align with the 
>> "next line single indent" style, though it would be helpful for 
>> clarification on this style exactly. Do you always wrap all args if one 
>> extends past 80 col or is there a rule for when some should remain on 
>> the first line (function def line)?
> 
> I don't think that aspect has been discussed. I would say
> 
> void sufficiently_long_attribute test(unsigned int x, unsigned int y,
>                                       unsigned int z, void *p);
> 
> is as acceptable as
> 
> void sufficiently_long_attribute test(unsigned int x,
>                                       unsigned int y,
>                                       unsigned int z,
>                                       void *p);
> 
> with a slight preference to the former.
> 
> Jan
> 

Apologies, I was referring to this style which I am understanding is a
little more preferred

void short_function_name(
    struct really_long__struct_name *x,
    struct really_long__struct_name *y, unsigned int z, void *p);

vs

void short_function_name(struct really_long__struct_name *x,
    struct really_long__struct_name *y, unsigned int z, void *p);


NB: I don't recall it off the top of my head, but there is one function
def in here that is similar to this situation

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 15:00:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 15:00:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181149.328178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNca5-0005UB-8y; Tue, 07 Sep 2021 15:00:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181149.328178; Tue, 07 Sep 2021 15:00:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNca5-0005U4-5o; Tue, 07 Sep 2021 15:00:13 +0000
Received: by outflank-mailman (input) for mailman id 181149;
 Tue, 07 Sep 2021 15:00:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qx60=N5=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNca3-0005Ty-BP
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 15:00:11 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4b78e2ed-0fec-11ec-b105-12813bfff9fa;
 Tue, 07 Sep 2021 15:00:10 +0000 (UTC)
Received: from DB8PR03CA0013.eurprd03.prod.outlook.com (2603:10a6:10:be::26)
 by DB9PR08MB6892.eurprd08.prod.outlook.com (2603:10a6:10:2a7::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Tue, 7 Sep
 2021 15:00:00 +0000
Received: from DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:be:cafe::80) by DB8PR03CA0013.outlook.office365.com
 (2603:10a6:10:be::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Tue, 7 Sep 2021 15:00:00 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT008.mail.protection.outlook.com (10.152.20.98) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 15:00:00 +0000
Received: ("Tessian outbound 7a08471b1ef6:v105");
 Tue, 07 Sep 2021 15:00:00 +0000
Received: from 2a10722e183a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 016CF1FD-286E-4099-B9C3-8602872EF9F8.1; 
 Tue, 07 Sep 2021 14:59:47 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2a10722e183a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 07 Sep 2021 14:59:47 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR3PR08MB5612.eurprd08.prod.outlook.com (2603:10a6:102:8f::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 14:59:45 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 14:59:45 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0113.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:192::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 14:59:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b78e2ed-0fec-11ec-b105-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g0ut8YQV8nIB9fjekl4Z/u2KCM0ldFTLssCyvuThFQ8=;
 b=GPAmWs8VKqm3+AsG3QWR6fGs0fDfKjJhtqQgCy+z9ot277xmSBH2JbYs465YO3U8mTtiTFqkLIt/Cz1fcs1XtMNLIQhepLUCKgQo4EEll2UMGXsqxPuYft9TSBp3MKV96306+znATtWT9w5hzIyFOmovw9sMhFG6h+06VqzZc0w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 389f84a7de519024
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GHjeyp1Tk8Bt44uKrDdrSON+4CmkXk3u9jS490qUmrPEYBftMGFIA6yD1nwkaemjy/hWIm1vey4CgoButen+4P89EEQ6qlKVayETBVecbFY+zF0PiE3odWayNPQQNcMRlHs8j1XsaP2guBQnT1IiOCaCj9LqHmosjXmaN+Nr2INgQNd8d7zdc7WmuDVzDMYyxi/DjPb0THrYGa9XNYi43+zjbFZJvgmF/aRUzqdZ8tmJ4vLCbd49iEsQ1ja6XmUZMsgVtnaDt5suEFkywJJ8FoGFKQwM38bSXK01cq7JC5gr9tUBfbkSEqvCk1+p/5atDhAUXmI3MuFseSqiThpRSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g0ut8YQV8nIB9fjekl4Z/u2KCM0ldFTLssCyvuThFQ8=;
 b=BG3RXNkExaN28DR+y9MminQvCXEta2BYknxeJBTeBsCthS0nXowWwBn01xdjaIPWHFANmDM7WpM1jC9gkg0khDR0bkP3Y/Q+I6mJtb8w5HdoUGhkmfUydBAroq9EPJzdppAv/+n2vj1a98vbnDMSHVZeP8Mnag41i9rdr7bc6BThCAzONPHDDGxeyJrg8eGsCKI7jWa9bnvUOPzbVrMlIFxtQrwcTbVj0Ft1ME8TR9Wsr5DxTe5wl+Hgj5ejjtduUScRln79cgloTt1kdyvW+/efUJcQz6BFVcouTR38NVyXYR6I92gHz2RBb/LgnZXcfZA5RWJjPyWYArNbn8xI1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g0ut8YQV8nIB9fjekl4Z/u2KCM0ldFTLssCyvuThFQ8=;
 b=GPAmWs8VKqm3+AsG3QWR6fGs0fDfKjJhtqQgCy+z9ot277xmSBH2JbYs465YO3U8mTtiTFqkLIt/Cz1fcs1XtMNLIQhepLUCKgQo4EEll2UMGXsqxPuYft9TSBp3MKV96306+znATtWT9w5hzIyFOmovw9sMhFG6h+06VqzZc0w=
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <910f2758-cbce-53b0-4e57-4b2a966ce2b3@xen.org>
Date: Tue, 7 Sep 2021 15:59:39 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <A2ED24B7-CED0-4064-B75F-F42115774C51@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
 <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
 <5DA71CEF-0ADE-4846-A7BC-B378B95A38AD@arm.com>
 <910f2758-cbce-53b0-4e57-4b2a966ce2b3@xen.org>
To: Julien Grall <julien@xen.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0113.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:192::10) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7c8a4209-cda9-445f-79f0-08d972102a22
X-MS-TrafficTypeDiagnostic: PR3PR08MB5612:|DB9PR08MB6892:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB68922FF241B24B933F8F4C97E4D39@DB9PR08MB6892.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 PojKJmOLpCAW7HE2c56gpfJ5iIGiqeoGCOggi10p9BRleMAWXW3Nuphre9GN1jGCdNhE7LuJNZLsHl45Hqnu5+n7axcYYDKAFE0Gk/RqoZmTQR7azOZh1kjP5X9iv5qY5MASBzF8ptEBK1nk0BCYxx/axbM5x076eqHJEj7jaj1vSS4KsA7zU2fF7upwcTuf1jjJpqre3LaIC7y98Qavl99TRsWU8KzcAIUt+LrsafFhjsiVqC7EBgzT9sXTULh2klqkUi88zCVR9UQGTcLEgB5wobz9YUB6O+fpNYCJ4mDt1vpubBJ/h3nVWQqQq7pHPbV3LzRXfvn1oe6O/vbalKGgYNMxKOjX078RcaoZipkj6ZPR/sjIloFwtoHPVip5eP1OKGG5R2b9+s2Yw0xOFd3pyqsgAaAJvO0L8XdtsHwwvIOlEQHiaEbl33jnAL7REuRi/gM62WDGNBw/sn+pLm5/9aOzvaX2dJY0BwB2zJn9BJH6IoFp+gZOmk/L6EfZ6svz0z/WkISnaEyVDuVqBggQEAt52rjabOvQKsxjvgCqX/eRSfWgvd5p4PKvazZtWFSXwIY3FvCoutriGKoozMTWaOBG0PZlRWjdkfz9wSFob79nfnnSHhX9sxzlraMFIIPF9Ir3BTd+YrDyqI69lvfrYx4s4jIzUCPbqtWBdaJgWs4SOYkF7Cuvu+gqC8WKsgoh5a3hEBfNpbKaPnYsdajSjypOVsng7oDKibGRfmj7DL/2Kv8SEmDtBdeyx9zB/5AXXzsLfYNz1LD8CyK9wQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(346002)(376002)(396003)(39860400002)(86362001)(26005)(186003)(2906002)(83380400001)(478600001)(6506007)(2616005)(956004)(8936002)(316002)(54906003)(6486002)(6666004)(66556008)(66476007)(52116002)(33656002)(5660300002)(53546011)(6916009)(44832011)(66946007)(38100700002)(8676002)(36756003)(38350700002)(6512007)(4326008)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5612
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	63729969-2c5c-4764-9aa0-08d97210212a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GUdGWtaEljibbUCSdnY4z9Hv26jglvwoL7N5JxADmVF6/NreGWkytIQG9zjZ8m8tC8T/jRMpCMF3ByC8m3QJP6Yq4BOsOtWi8u4EjFNRy05exlNwbZ66xS1nOgnIsxTXav3eN+DSSTD1OWVLsYYmqYTBRJKiQj1uZxpDB3cOvCSY62y+ahyOrIOgNBUVBIEwwZ1WCdCSEGzATO1sGUdoZ3jszgaYoP3sCKlkRNfZTdcK13DFBSYplw+AqHNprpbJlnBQTKNoGzZR3ArhhhO84UiYAPZhf0PItqGQInsEkSsY5VHzuvow2O9EzEe1BM3YdHGoGpg5aTH6u2tEwMEancu/UXHJlshApJbm41s8JDeubAts82+EhRxNd+uU2p6J5G0heH7+IDRbBYNinkdbiJW8RZNsSdP8QpnDuI2sz9RP2sarZft6QrF0d5oqjb9CNBh2qV+XuRlonsFVj1gK3kLUANczOC7G8FE6bbKufe963THDNidGufuJxx8InklBEuPXu2/N9kMpWyT9cvv88YTl+zaEhBvpaU2NBy/qSG2JIvX/te2LTxZjAzGZiB6sGnmbHQLhKsLol0ABFjieE8yqekZg42BYAUG0FB/ZAYs+YCeYVUyp3NuanYoPrOx83YBsLs3YP+XyRP5ONYJ8wWNDfUHGrvEk74ZtpCP35u4hq5v3JPVK+LnO24tbmGLwNTnaTcXigCzRZyWLZmWEAvg0kWR+VDIfH4scX9WkHFY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(396003)(39860400002)(136003)(346002)(46966006)(36840700001)(316002)(70586007)(6512007)(4326008)(53546011)(33656002)(336012)(6506007)(83380400001)(47076005)(2906002)(8676002)(70206006)(478600001)(26005)(5660300002)(81166007)(956004)(82310400003)(36756003)(6666004)(8936002)(356005)(2616005)(54906003)(86362001)(6486002)(82740400003)(186003)(36860700001)(44832011)(6862004)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 15:00:00.1516
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c8a4209-cda9-445f-79f0-08d972102a22
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6892



> On 7 Sep 2021, at 15:18, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Luca,
>=20
> On 07/09/2021 14:30, Luca Fancellu wrote:
>>> On 7 Sep 2021, at 13:30, Julien Grall <julien@xen.org> wrote:
>>> On 07/09/2021 12:51, Luca Fancellu wrote:
>>>>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>>>> What we could do is providing a list of binaries to load and associat=
e a key for each of them. Something like:
>>>>>=20
>>>>> binary=3D<binary> <key>
>>>>> binary=3D<binary2> <key2>
>>>>> ....
>>>>>=20
>>>>> We can then replace the property "reg" with a new property "uefi,key"=
 that will contain the name of the binary.
>>>>>=20
>>>>> What do you think?
>>>> Here I=E2=80=99m lost, because I don=E2=80=99t understand what we are =
going to do with the name of the binary.
>>>=20
>>> <binaryX> would be used by the UEFI stub to load the binary in memory. =
Each binary will have a <keyX> which helps to refer them in the Device-Tree=
. To give a concrete example, let say we have two dom0less domains:
>>>  - DomA: 2 vCPUs, 128MB
>>>  - DomB: 3 vCPUs, 512MB
>>>=20
>>> DomA and DomB will be using the same kernel but a different ramdisk. xe=
n.cfg, would look like:
>>>=20
>>> [global]
>>> default=3Dsection1
>>>=20
>>> [section1]
>>> options=3Dconsole=3Dvga,com1 com1=3D57600 loglvl=3Dall noreboot
>>> kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>> ramdisk=3Dinitrd-3.0.31-0.4-xen
>>> xsm=3D<filename>
>>> dtb=3Ddevtree.dtb
>>> binary=3Dvmlinuz-guest domu-kernel
>>> binary=3Dramdisk-domA.img domA-ramdisk
>>> binary=3Dramdisk-domB.img domB-ramdisk
>>>=20
>>> The chosen node in the DT would look like:
>>>=20
>>> chosen {
>>>    domU1 {
>>>        compatible =3D "xen,domain";
>>>        #address-cells =3D <0x2>;
>>>        #size-cells =3D <0x1>;
>>>        memory =3D <0 0x8000000>;
>>>        cpus =3D <2>;
>>>=20
>>>        module@1 {
>>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>>            uefi,binary =3D "domu-kernel";
>>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>>        };
>>>=20
>>>        module@2 {
>>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>>            uefi,binary =3D "domA-ramdisk";
>>>        };
>>>    };
>>>=20
>>>    domU2 {
>>>        compatible =3D "xen,domain";
>>>        #address-cells =3D <0x3>;
>>>        #size-cells =3D <0x1>;
>>>        memory =3D <0 0x20000000>;
>>>        cpus =3D <3>;
>>>=20
>>>        module@1 {
>>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>>            uefi,binary =3D "domu-kernel";
>>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>>        };
>>>=20
>>>        module@2 {
>>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>>            uefi,binary =3D "domA-ramdisk";
>>>        };
>>>    };
>>> };
>>>=20
>>> With this approach, the change is quite minimal to move between an clas=
sic U-boot boot and EFI boot.
>> Ok now I see, yes this approach can work and can save some code, in the =
current code we have that if
>> a "multiboot,module=E2=80=9D is found in the dtb, the Xen EFI configurat=
ion file is skipped, but if we use the
>> module@XX {} without the compatible it can work, the UEFI stub will load=
 the binary and update all
>> the needed properties (compatible, reg).
> With my proposal, you don't know whether the binary is a kernel, ramdisk.=
.. So you wouldn't be able to recreate the compatible properly.
>=20
> But the behavior of the UEFI stub can be modified. We could say that if t=
here is a "xen,domain" then use the configuration file to fetch the binarie=
s.

Yes you are right, or we can introduce domu_kernel, domu_ramdisk, domu_dtb =
with the same syntax of your binary=3D keyword, that would be enough
to select the right compatible, instead the right module is identified by t=
he string.

>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 15:01:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 15:01:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181155.328189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcbP-0006Om-Km; Tue, 07 Sep 2021 15:01:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181155.328189; Tue, 07 Sep 2021 15:01:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcbP-0006Of-Gx; Tue, 07 Sep 2021 15:01:35 +0000
Received: by outflank-mailman (input) for mailman id 181155;
 Tue, 07 Sep 2021 15:01:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNcbO-0006OX-E7
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 15:01:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e9534e9b-dccb-4cdb-ab3a-9eee082daf8a;
 Tue, 07 Sep 2021 15:01:33 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-4-Weu6wJ30MmKhLA8foV0Wjw-1; Tue, 07 Sep 2021 17:01:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2605.eurprd04.prod.outlook.com (2603:10a6:800:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 15:01:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 15:01:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0051.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Tue, 7 Sep 2021 15:01:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9534e9b-dccb-4cdb-ab3a-9eee082daf8a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631026892;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3gOJT+pE51xlYTei+0Zv/UygilQrt+3sdPAMFW0ydco=;
	b=ZYPec0pZ2H8QsE2u/yGIh3VeQz+DivH4/Peit5lPqvP34yKZpNK/FR/59cZHbBR3r3hMWb
	kJgbnJBq6Nw1FQN3a5+XFVS3qWo6fHQWSd56RMTz2qtrvfKvWXMMJEWawpMXdRiKK2hYQL
	ehl7HfU76+EBQadFri6g3z2z8YCvFSw=
X-MC-Unique: Weu6wJ30MmKhLA8foV0Wjw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IOA/6gS4pY6AnCYtILAANLrTVUfP0X+rxkVrL7ehjnxI5881dchcqvIsVqieEqCLr2xwUfd/X0e42p0u0E0E/YOl+frFa6wd5CpAWm05eS4wDEV5sxGyFG3YWp5xtehHFok8UkicEHsFYGxBzw30odhX9E4E+tHAwP7ljIy3DzFclwZ8sUsTSc5TcRTa4Isc2bCbNG5/1UufvrU56Jh+qpevb8zMa+wctRv+6AGUFYpXo+XUj5xJ1ocF8q4II6bWBs4MzKmWAnXjE8Itg6cpHUNqHOn3ubVtslrsw7oecB5Xhf3JBGlqREmCiMxo88jwc7Z21PIpG8zuWFoJ05vTKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/yJIGk4ydRvmXHfQr1S7Kfi9GZbDReLS7yzFgjiLH+0=;
 b=I9Zge/jXxu5qsMi//mDK8p844hiS8Ypw5zIwzoxdvDtSs9ZsWwD9SjG8fNQz8I7TSs3sHBs1RwDuenq2y0eAd5W6z4HMgvplMzby5UfirsYL2Ktv1p0bvd1LAWPiDwg8Q8TYKvB7WQACo5Ey2Swg6XBGUueicIQysTp3iAW1y6WuZuVfnsnIUq/BSKB38Ko09u/euEjbCg98wCaOo/HmozVpkKFjbh3IjHVAMppbfR8Pu29YZUzuwETPOovORpF98B/Kv15vxE84eTSBJZKlFgMzA5uHJTASl7WA3RvtC0xOsVqA2larNfkAxaTYzaUpY1rGEO+mtEubcU88tNqWHA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 04/11] xsm: apply coding style
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-5-dpsmith@apertussolutions.com>
 <01c81885-9ea0-3ecf-66b0-009b9e7ba39b@citrix.com>
 <ef0d627e-cc1f-4b9a-7695-daa646968c42@apertussolutions.com>
 <a3a9207e-7ab9-5eb9-7488-74bc4203e7a3@suse.com>
 <3b3762a7-5034-9094-84f9-cd8ab24e24a0@apertussolutions.com>
 <e006dc60-7745-43b7-f725-2390e174bfa7@suse.com>
 <bf876f84-aa0f-faff-da63-0f0525087883@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6c042f71-80a1-c6ee-f171-fa1ac4c3dcef@suse.com>
Date: Tue, 7 Sep 2021 17:01:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <bf876f84-aa0f-faff-da63-0f0525087883@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0051.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b2284eca-7d06-4f1d-3bb1-08d972105e35
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26057DFFF89225A813432EAAB3D39@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JrFF53dtO2jI54Ecl/NGU8KqOxWYurUwTn0xxHndtsx3rBUTA/9JeHnHt5qY6D2OMHNAzAeEw598wqj6l5aXK5/6a8m3BosG9rRuo0lrsmOnwveWncoCpt5+rrgMk/MEPf1lOCLmkF0+55zs/WyfR+EBHCHVLYzgKjXytnVfw0cy+g7Z4nOwwBNxyJIOZcZZDs0QvNkK2kvqodQKCxnCZG+uU1R5YKRYYb0F1ZQhinHR5jpg6mvFSAnznn1CZXMm2CiUCdOpl5JgifeXoLMXc3YZWKPZ4vuidNbNGKWWT/66URafH1gqSQqgioLM7WqaLnVM79/lT31bfSB904XYkrnpk0omeoaIdfsf1BsYdX1HHtSBb+PPSruhJO5HOF8YRJQVm96hXyfCgL1COReg1hb/cWM/XgB8GRumTH2+Q1slMR7WyUYy2KtcJwaSRsYu2FVSbO8AjuiCwPR4FA5G9nu5rU4CKj4SHpv25bdhlbAs8IL8QH+z5MtuLFTDbkZ876dBoITw1LmW9n9l/yqNqHzGLtKtF92+qC3+EGB/shHKCysAt9I6wy6HiqsCxylA1RT1Z0JDC8Q4yAFSBpX1CanHEDTyFlYAtcOToyls+UoDdS900aRqF2QfmjeLB2TH0ugI6ttg5Zz7ZoB9eMNlVDHlFllq7oAVqDH3ClGVA3GWRAUwACnBrWN+L0YM6LfgN2WI2dCI4rXqr/aZfwgMNqUnUut21BXDTjZMlQE+A3E=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(39850400004)(376002)(366004)(136003)(6916009)(53546011)(5660300002)(36756003)(316002)(16576012)(83380400001)(66946007)(6486002)(66476007)(66556008)(2906002)(186003)(31686004)(956004)(38100700002)(31696002)(26005)(2616005)(478600001)(8936002)(8676002)(4326008)(86362001)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?8qyMkGVN2guD2B3QnRHxgFg+Io2MhXpBXl24SjvMRRVTabOZMVx1cmHvNKcI?=
 =?us-ascii?Q?V7pBJkWXboZf2Vv3O9/E33UXZu/vDFHllWa3jHRtX4pETgBDlhyJsjG3cNay?=
 =?us-ascii?Q?cHOMbLpRWK3OZhSLypwpJZCk01pvtF2qikbUr1zubYzHgg4JKMYQrzG2SNwn?=
 =?us-ascii?Q?vk0ZOyIF6M0SlCMAFMCtmeI6rF5tb4l9cKpGKxpazbTgamqZWAGR+7TX/mXc?=
 =?us-ascii?Q?QBzLSgzR72TUsSonRF6JJXRgNxLwgnkUtMRRH5hNclXEVxq1kSlmJH5lcLiI?=
 =?us-ascii?Q?sm7Js0Bw2VMR2+Ism06xC0KW6CGryD5T/h9RqFi2XsLL62XBLWHIJU5H8zoh?=
 =?us-ascii?Q?CBAzzbGylZA8A/KTw1ClIux/BeYe5Vr8HnnFE+0sRrc+FxNbnbhk8X0Gr1o6?=
 =?us-ascii?Q?iUB8SE7LgeMbJSMzPRwGtmtddHYPrS4dYx2pPD8mW/Z5X0uvkua4b+7GkKm1?=
 =?us-ascii?Q?sqdmDmIvh3T/3sFVJABEgEf0oT8sy0uG0sRCndGy7aeArgCfap90fj+ZXnic?=
 =?us-ascii?Q?i6GUvIofYZ/xApQ8mG8bDtFv1hG19SW7sPttxjW+2QqkVYSbcCYVmSmzP0CY?=
 =?us-ascii?Q?d7V13VwyUflfoIqoyMIOEHQOebTfjZnQS3BfNxrue0O9a5/pCWX1du3Y6Ao1?=
 =?us-ascii?Q?cbwdYwFSkD1NiPB4EzEVu0+MqIPnwwyycYbG1q/RUmUfcueO3W31wHI5mD8O?=
 =?us-ascii?Q?QE1EEkUGQr1EtPcVCDlVw2G3nIKR+HFpr7/J5IdnTSNIlMShKIBSOz5sNKd8?=
 =?us-ascii?Q?O252/twMHi3uInfUPuksi/Qtfcc7xm8vwsT5+ySagpXkFgJTXEVlhxNbxPe4?=
 =?us-ascii?Q?8JavYhwcfhG2mxv0YRWW39RQLYcudvKxlscQrxu0fqrzyrPc9QO5aAmQGPOG?=
 =?us-ascii?Q?q+YNUt8xYPxsPkefsF2N0svk84Dt2Mb3BoGnjxC190kpOhAUjCyufuWAxDhz?=
 =?us-ascii?Q?C1146R4v7x+kMK+nG9C/0haeopB7vP4b7cCrGBVhSkpPviTDJgaduVmeR5hq?=
 =?us-ascii?Q?uMaXCZRdSkRd19qeKqc90D5npjOOr5Pedxlh56xkDmQ+SDB+LyBX/ywPl+Zu?=
 =?us-ascii?Q?uZQvmLCj60avQWZGkQV+c/x7GRtjYx5od1NtQAnvTxoU80Lzwwa0fktrdZI4?=
 =?us-ascii?Q?lcbOWZ6Ez6Di65LamSBWx7cd600csTpOSzecD+zZwYhZOk6DfGaqOgEyTYRc?=
 =?us-ascii?Q?RK2Z1FkgmafyOFtE+D3/NSKTe4qAZSxwxrxhMkcNNgU44YTbxC1j4mPr013L?=
 =?us-ascii?Q?scKCRYyLJ9Y6v4gdXBS6Pwv5BPPXfeWMXmepdKcUB3aXPsQXHzkpH0pSEFhN?=
 =?us-ascii?Q?b6bmD+2d2TZOqF6JOISDtiRE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2284eca-7d06-4f1d-3bb1-08d972105e35
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 15:01:27.6315
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DkM6VkY+qvb4VFNQ8K32XycFTwC7TTRep3MY1iAebIi/izZSjZyVgoN5txO+CeLnUdK5CT3Q2yzbU3XP1JZyoQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 07.09.2021 16:55, Daniel P. Smith wrote:
> On 9/7/21 10:27 AM, Jan Beulich wrote:
>> On 07.09.2021 16:09, Daniel P. Smith wrote:
>>> On 9/7/21 9:50 AM, Jan Beulich wrote:
>>>> On 07.09.2021 15:41, Daniel P. Smith wrote:
>>>>> On 9/6/21 2:17 PM, Andrew Cooper wrote:
>>>>>> On 03/09/2021 20:06, Daniel P. Smith wrote:
>>>>>>> --- a/xen/include/xsm/dummy.h
>>>>>>> +++ b/xen/include/xsm/dummy.h
>>>>>>> @@ -69,8 +69,9 @@ void __xsm_action_mismatch_detected(void);
>>>>>>>   =20
>>>>>>>    #endif /* CONFIG_XSM */
>>>>>>>   =20
>>>>>>> -static always_inline int xsm_default_action(
>>>>>>> -    xsm_default_t action, struct domain *src, struct domain *targe=
t)
>>>>>>> +static always_inline int xsm_default_action(xsm_default_t action,
>>>>>>> +                                            struct domain *src,
>>>>>>> +                                            struct domain *target)
>>>>>>
>>>>>> The old code is correct.=C2=A0 We have plenty of examples of this in=
 Xen, and
>>>>>> I have been adding new ones when appropriate.
>>>>>>
>>>>>> It avoids squashing everything on the RHS and ballooning the line co=
unt
>>>>>> to compensate.=C2=A0 (This isn't a particularly bad example, but we'=
ve had
>>>>>> worse cases in the past).
>>>>>
>>>>> Based on the past discussions I understood either is acceptable and f=
ind
>>>>> this version much easier to visually parse myself. With that said, if
>>>>> the "next line single indent" really is the preferred style by the
>>>>> maintainers/community, then I can convert all of these over.
>>>>
>>>> I guess neither is the "preferred" style; as Andrew says, both are
>>>> acceptable and both are in active use. I guess the rule of thumb is:
>>>> The longer what's left of the function name, the more you should
>>>> consider the style that you change away from.
>>>>
>>>> Anyway, in the end I guess the request for style adjustments was
>>>> mainly to purge bad style, not to convert one acceptable form to
>>>> another. Converting the entire file to the same style is of course
>>>> fine (for producing a consistent result), but then - as per above -
>>>> here it would more likely be the one that in this case was already
>>>> there.
>>>
>>> Understood, I will respin with all the function defs to align with the=
=20
>>> "next line single indent" style, though it would be helpful for=20
>>> clarification on this style exactly. Do you always wrap all args if one=
=20
>>> extends past 80 col or is there a rule for when some should remain on=20
>>> the first line (function def line)?
>>
>> I don't think that aspect has been discussed. I would say
>>
>> void sufficiently_long_attribute test(unsigned int x, unsigned int y,
>>                                       unsigned int z, void *p);
>>
>> is as acceptable as
>>
>> void sufficiently_long_attribute test(unsigned int x,
>>                                       unsigned int y,
>>                                       unsigned int z,
>>                                       void *p);
>>
>> with a slight preference to the former.
>=20
> Apologies, I was referring to this style which I am understanding is a
> little more preferred
>=20
> void short_function_name(
>     struct really_long__struct_name *x,
>     struct really_long__struct_name *y, unsigned int z, void *p);
>=20
> vs
>=20
> void short_function_name(struct really_long__struct_name *x,
>     struct really_long__struct_name *y, unsigned int z, void *p);

This latter style is not supposed to be used.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 15:03:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 15:03:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181162.328200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcdZ-00077q-7K; Tue, 07 Sep 2021 15:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181162.328200; Tue, 07 Sep 2021 15:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNcdZ-00077j-3A; Tue, 07 Sep 2021 15:03:49 +0000
Received: by outflank-mailman (input) for mailman id 181162;
 Tue, 07 Sep 2021 15:03:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zymF=N5=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mNcdX-00077d-VY
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 15:03:48 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 31753929-94cc-4732-8916-55f8d079c637;
 Tue, 07 Sep 2021 15:03:46 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 187F3a2o070616
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 7 Sep 2021 11:03:42 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 187F3asv070615;
 Tue, 7 Sep 2021 08:03:36 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31753929-94cc-4732-8916-55f8d079c637
Date: Tue, 7 Sep 2021 08:03:36 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YTd/SFtvuzejeiik@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Tue, Sep 07, 2021 at 10:03:51AM +0200, Jan Beulich wrote:
> On 06.09.2021 22:47, Elliott Mitchell wrote:
> > On Mon, Sep 06, 2021 at 09:52:17AM +0200, Jan Beulich wrote:
> >> On 06.09.2021 00:10, Elliott Mitchell wrote:
> >>> I brought this up a while back, but it still appears to be present and
> >>> the latest observations appear rather serious.
> >>>
> >>> I'm unsure of the entire set of conditions for reproduction.
> >>>
> >>> Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
> >>> this is an older AMD IOMMU).
> >>>
> >>> This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
> >>> Debian's patches, but those are mostly backports or environment
> >>> adjustments.
> >>>
> >>> Domain 0 is presently using a 4.19 kernel.
> >>>
> >>> The trigger is creating a HVM or PVH domain where memory does not equal
> >>> maxmem.
> >>
> >> I take it you refer to "[PATCH] x86/pod: Do not fragment PoD memory
> >> allocations" submitted very early this year? There you said the issue
> >> was with a guest's maxmem exceeding host memory size. Here you seem to
> >> be talking of PoD in its normal form of use. Personally I uses this
> >> all the time (unless enabling PCI pass-through for a guest, for being
> >> incompatible). I've not observed any badness as severe as you've
> >> described.
> > 
> > I've got very little idea what is occurring as I'm expecting to be doing
> > ARM debugging, not x86 debugging.
> > 
> > I was starting to wonder whether this was widespread or not.  As such I
> > was reporting the factors which might be different in my environment.
> > 
> > The one which sticks out is the computer has an older AMD processor (you
> > a 100% Intel shop?).
> 
> No, AMD is as relevant to us as is Intel.
> 
> >  The processor has the AMD NPT feature, but a very
> > early/limited IOMMU (according to Linux "AMD IOMMUv2 functionality not
> > available").
> > 
> > Xen 4.14 refused to load the Domain 0 kernel as PVH (not enough of an
> > IOMMU).
> 
> That sounds odd at the first glance - PVH simply requires that there be
> an (enabled) IOMMU. Hence the only thing I could imagine is that Xen
> doesn't enable the IOMMU in the first place for some reason.

Doesn't seem that odd to me.  I don't know the differences between the
first and second versions of the AMD IOMMU, but could well be v1 was
judged not to have enough functionality to bother with.

What this does make me wonder is, how much testing was done on systems
with functioning NPT, but disabled IOMMU?  Could be this system is in an
intergenerational hole, and some spot in the PVH/HVM code makes an
assumption of the presence of NPT guarantees presence of an operational
IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
code, some portion of the IOMMU code might be checking for presence of
NPT instead of presence of IOMMU.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Tue Sep 07 15:30:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 15:30:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181180.328211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNd3G-0002Qp-Ah; Tue, 07 Sep 2021 15:30:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181180.328211; Tue, 07 Sep 2021 15:30:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNd3G-0002Qi-6p; Tue, 07 Sep 2021 15:30:22 +0000
Received: by outflank-mailman (input) for mailman id 181180;
 Tue, 07 Sep 2021 15:30:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bt8x=N5=kernel.org=helgaas@srs-us1.protection.inumbo.net>)
 id 1mNd3E-0002Qc-Qs
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 15:30:20 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 823dd0b8-0ff0-11ec-b109-12813bfff9fa;
 Tue, 07 Sep 2021 15:30:19 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A9E9B610D0;
 Tue,  7 Sep 2021 15:30:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 823dd0b8-0ff0-11ec-b109-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631028619;
	bh=kp8Q8kJRz0WoYnZlxjaDlmj4GyxqVupLYpX0r3kh5eI=;
	h=Date:From:To:Cc:Subject:In-Reply-To:From;
	b=FE6OgWm/fxpyj4FLQwu9LoQuMjKnA94XCfxVDByukq4rictBgJBARDgnsuKZZf51p
	 WXTFndXhgPK25xj8rW4dWKHHBd93KkRJMISbXYQeHJ0AMHoyWkddZYiARPkgRcGwBJ
	 AT6niJGOnCXNPNvY3lbnqpZz36S29V/LYwGZE4/xaCHHf6512nYdVpZOaJSLMuZUoR
	 sM+3SiKS+cOzVNn/u7KymTC8iwDRxTh24dDklEeX8z7qtgA+ZJj3kLfOVv0bv4A6b7
	 7vbe+oIWcT0eDqgNzqf+WJuR4rDw+5kM7nUIpdutwsL0sFBHvcbEN7fiD4Vb+EWH86
	 LY7D7/lwqLanA==
Date: Tue, 7 Sep 2021 10:30:17 -0500
From: Bjorn Helgaas <helgaas@kernel.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH 10/12] xen-pcifront: this module is PV-only
Message-ID: <20210907153017.GA741607@bjorn-Precision-5520>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <bbfb4191-9e34-53da-f179-4549b10dcfb3@suse.com>

Update subject to follow conventions (use "git log --oneline
drivers/pci/Kconfig").  Should say what this patch does.

Commit log below should also say what this patch does.  Currently it's
part of the rationale for the change, but doesn't say what the patch
does.

On Tue, Sep 07, 2021 at 02:10:41PM +0200, Jan Beulich wrote:
> It's module init function does a xen_pv_domain() check first thing.
> Hence there's no point building it in non-PV configurations.

s/It's/<name of function that calls xen_pv_domain()/   # pcifront_init()?
s/building it/building <name of module>/               # xen-pcifront.o?

I see that CONFIG_XEN_PV is only mentioned in arch/x86, so
CONFIG_XEN_PV=y cannot be set on other arches.  Is the current
"depends on X86" just a reflection of that, or is it because of some
other x86 dependency in the code?

The connection between xen_pv_domain() and CONFIG_XEN_PV is not
completely obvious.

If you only build xen-pcifront.o when CONFIG_XEN_PV=y, and
xen_pv_domain() is true if and only if CONFIG_XEN_PV=y, why bother
calling xen_pv_domain() at all?

> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -110,7 +110,7 @@ config PCI_PF_STUB
>  
>  config XEN_PCIDEV_FRONTEND
>  	tristate "Xen PCI Frontend"
> -	depends on X86 && XEN
> +	depends on XEN_PV
>  	select PCI_XEN
>  	select XEN_XENBUS_FRONTEND
>  	default y
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 15:57:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 15:57:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181187.328222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdTL-0005nr-It; Tue, 07 Sep 2021 15:57:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181187.328222; Tue, 07 Sep 2021 15:57:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdTL-0005nk-Dd; Tue, 07 Sep 2021 15:57:19 +0000
Received: by outflank-mailman (input) for mailman id 181187;
 Tue, 07 Sep 2021 15:57:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNdTK-0005nb-Bs
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 15:57:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 45e2d060-0ff4-11ec-b10b-12813bfff9fa;
 Tue, 07 Sep 2021 15:57:16 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2050.outbound.protection.outlook.com [104.47.4.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-a6zWnPUFMfahP0fn_KPTRQ-1; Tue, 07 Sep 2021 17:57:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3773.eurprd04.prod.outlook.com (2603:10a6:803:1c::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 15:57:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 15:57:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0028.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22 via Frontend Transport; Tue, 7 Sep 2021 15:57:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45e2d060-0ff4-11ec-b10b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631030235;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3em18/5I60ro58fWzw7m4WZD8kA9dd11ewhblexpUAc=;
	b=W7JkEt7mqQxUofae3Gvi0oKNuAFHeAPAcEuLlTFjWb4f3C9mrUfC8fhX2x/ldJPN60UFMS
	7uHvVCZbtjBGwaNFMDq++Qzq8yNo3m27I7+DALfszTksKRYOl6FcLX4/y44+T35HsJHd4c
	lQGP7U6Owhq47rqrkeOOw6jz5gUt+n4=
X-MC-Unique: a6zWnPUFMfahP0fn_KPTRQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QpoW4jOkG792Rsx+dZiz6nck9Pva6+FaiX24JBxU1GgOSIQ+oYipDalrXbT68eQPadgNVLnn1wpN/DD3aMJsy+r7wXbSU3unAwuLyBA/eZ+Gr/cc63I8noTm4kfb5jUzJHa6nAH9nT76KWESGwlE/kdboRCBJx7XaB+PqrAnAoYZ9SS7mPpz5emJRwUtTEzcLVgsQwmc5Lw7wtm6swmWFPgYjdMrk+DdVY6O/CXDoyOYRSgLq3P26cLk2T47mMz+UWKwUQc/6I+wO0gThWZeiMBLqiKH9swvW/Qb/xWXOs/WmaMFWT8FhMrxX6M4WpZ5j7vtjenJUzw1Q0tIH9yckg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3em18/5I60ro58fWzw7m4WZD8kA9dd11ewhblexpUAc=;
 b=if4/rzahEFo09d7dOQ7Io9ZebCIXM9vZmBMPeMIaoLCenDifVp+lo1CV3nD79TUbNQk/pTjVwaY0tX3LPmpodz0L+57PMwm2Gc3vWOba3RvJFZimvP10aB7LcA2bUa7AJ7q1CR5RvRAxQ60wPq2PZsviHKfe6Pu8FkI2mXMEXIznSpDqV6xQEYe5RWK7bpLwKN/JfidFE5np+3P+dvuivZoMboGqfV+auVaOUpWRidAwMsXhESk4LLHmTA6Wb5UWs0X0r5E8ICnZyhqA47z1qa3mTiqmyMQiEBDRlZhj/npqFbwFTWNLbNrtVbq6w0y8LEHnbsHeockrzK76c6COYQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
Date: Tue, 7 Sep 2021 17:57:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTd/SFtvuzejeiik@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0028.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 846c4bf9-f7fc-474c-a1d5-08d972182823
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3773:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB37738A57BC3B9D40243D4FB9B3D39@VI1PR0402MB3773.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	d0mRHbZmgAh5qW+PhDiGMMNv946vWuZ0u1m+5/CyUf9pAuHSvPe/VQcfPQhAcwrKLTgnpZ07Tpac015kwc0vb8lDBrGETZZyqFXPbxTEveAYlt67g1M7te55uxKrgjIwL09V5w0cLbCb851fWcxndHpxi42/+HptRwZNe8N40G0YCQqZrO5wOqg1MVuM5sz5V8iq5nPAcCRq6dmixmep4kmIPDeLdLZu6TLIY1x2qc0XetUSdS9exaE7o1bvwfXrlS+lkrmVVwRUgK9beg0tgDR9pQHf36YsoGYSu178xlJAneGQL3X+BWSWSdlc3zbpE8v8B89PQJRQRCT7iZOK33OSSvydCs2YHHrpfOw+1iWOE/c3TOKtyEvHxVf1K3o7B1hV3KCkep4kdFe5QXEdByk0CCwzyz0voPuC+J2rteFaWOdSWPqEyOL5dU6dM4D8SIXeFeyKRPv87oETPOJ2t60lzfHHKaowkfzx6ibXMEuGSd92rWv9vFqNGbdQgnJwUsbb+Yw73WtYBqAnmdVDa3DIk9XdWiBCcZHBA4yW56wmx62URA+bqmxrt6kHg6Uc30fpyusKiwTQE45EBAu5iqpWRU/CjFb1qY8Gmj0mL5MSd60eDjVLs2QaQiNj9Nm2516ozVg4lgfn665Pma0iIOorx78qyqdEpHQr7IRDIZM2DrrcQcPOMulU9vetwnBE8cC8uHlFfkSnI9X8ko5uUW4TEx46KcvmRONPzkwEhOcBjtjM02CLQMpslHpNwIMj
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(2906002)(26005)(8676002)(508600001)(38100700002)(3480700007)(31696002)(86362001)(6486002)(4326008)(83380400001)(5660300002)(53546011)(8936002)(31686004)(36756003)(186003)(66476007)(316002)(16576012)(66946007)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aWFCNld3M1R1VUdObVc0ZVAwbkFXYUJxd21zd01YQ2xoUHNkWForUERUZkY5?=
 =?utf-8?B?ZFdvbUZpQXFiVGd2TEswTTdvSmFvME5PZFJRZWhBT2VWR0lLM2NMd0pWUTg3?=
 =?utf-8?B?QzdUMENjbXhPN05PN1NXUUVFUjRFWldqZU5QVEtiVW5LN1ZxVTc5MHRQSVY5?=
 =?utf-8?B?Z1N4amRoKzE0WjlKcEhheEUxQWc4VzU3Zm9DZjRRR24zRHA5c2JsdGtZbGF6?=
 =?utf-8?B?aWxTbEI4WllYbnJoTkZsQ3NMMUJ3Rm5mMWhGcEJETGhlN0ZJR2VxdHlkRFlH?=
 =?utf-8?B?MW8wS1ltS0VHWXVKK2F2QWhySThjbmxlaGZxVy91TjhzcmxMeUNRSUkwUy9T?=
 =?utf-8?B?NEQwaEwvOUlXMVY0V0VDR2MrNFJpWnh3TzJpZnpPUGlEdkZCZXd0SmQ3U3Rs?=
 =?utf-8?B?T213aUIveFVCU0EwWExSeEpIQ2dKRGthdFdOb1F5NEh1eTZwM1JLMDd6RkNw?=
 =?utf-8?B?Y0tta3JuY1FkOHFmcCtZbmFLVjZEMytFYlorZmRaSjdlSDQ0ckJJbUZWakRw?=
 =?utf-8?B?QWJ1SGtFUUIrUi95SklCVFg0bUNjSThYZWY3bXFwYjN2NjdvRXBqUmN4VTIx?=
 =?utf-8?B?ZzdjckFpY2E1SkowdWt2ZkpSRG9JeSt2S2thNzRhSUdpTGpJOGM1aThGajR5?=
 =?utf-8?B?RDM4bktKQmt6VjNxT2QzZ0xISjlxR2RuMHNCWUFJd2phQUY4amFiMmE2TGw3?=
 =?utf-8?B?Yit0b3ZlMVE4aGZZemxIQklVVzN1eGx4N09qMm9Rc1h2anpGcTFZNzllZjI5?=
 =?utf-8?B?NmViQno1L2h0T3VMQkUrZ0Q5ODVnSlNoUGdEc2ExdHBnWjVJaHRaN3ZNc2RV?=
 =?utf-8?B?QWdacEhFV2NaaWpvMzRyMGRIVm1FOXdOUmlpVEMwOGRKbXl1NGFsYnc1RUky?=
 =?utf-8?B?RnJXQ0xIOG83L2Q5K1M0RitZMEVtS3dsMExCbWNwQ05xZ1B5NHhaSko5cVA3?=
 =?utf-8?B?S21POG1kTXJoVnJNTksxR052MFdzdlRxTlEwbVFMWEg4QWV2eHJWeGQ2TmRE?=
 =?utf-8?B?bDdoU3pRYzNrRnBieTlLNkN1bFluYlUwazVlS1dtWndQUmlQNk9qcEhWNlJJ?=
 =?utf-8?B?a2ttV3Q0b1lzT0paQWdrczd3aWN4Q0JpdTIxcHJkZ3ZvVzlyRkR5aTBENENt?=
 =?utf-8?B?VzhmcFVHN2MvSmp1ZG5sVTZ2M0tGcTYxQ2dvaVNwZVdKMWo2Wm1hdm1jb24w?=
 =?utf-8?B?STJORXRmamlsbHdLZFJBdWZJUTR1Nm5OYW12ZnM1Zm1jenRtYjBFLzA2SmUw?=
 =?utf-8?B?Y1JKbWc2MHpkTXd1QS94eEk0R0djWUhGeVRpbjh3dW0rajczZ05TS1hIMXhF?=
 =?utf-8?B?b2hQOVdZbjJ5WFZobEJ2ekVQQjhnZUlGVEdlS1RCaGF0bzF6UmxRU0xFekdl?=
 =?utf-8?B?RG5BajE0V01JNThDYURMb1hmODA2aEVPUzN0Vi9Ib0JCSHdlc2xEdktOczNh?=
 =?utf-8?B?ZGVxR0FHTmw0S1pPM0s3ZVg3TlN5Rk1NM1ZkVjdKN1pGOU81bXFVb2dCbFIw?=
 =?utf-8?B?WXpTSGs2UzNoc05DR09ybTBWNmtBRFpCdGZNQW1ESW93N1lHaFJDWklFTDRt?=
 =?utf-8?B?N05HbmtIckoxa1RZUVZwQWpnWjRDcnhXZFVkK2hGWUhtYmhtM2MwZXBEN2ww?=
 =?utf-8?B?ME10dXZVYXVEM3lCM1pOM0lmOStuOWY1RVBDbWgwY29FN2xyNTZRZXMwV2x6?=
 =?utf-8?B?ZWdyNE5ERFdmNlpxS1JoU3VPZGFtNVI5K3pVQVJUV1NoRUNLd1Q2MHJPdE1m?=
 =?utf-8?Q?eATeJTxVDjgAQOx3Nm5p4wcSNIo7+K4XVT8xE4W?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 846c4bf9-f7fc-474c-a1d5-08d972182823
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 15:57:13.0059
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OrwKDx90gigXjgsT5XnkrEZ1dzmNZNpHrJi2GedM66wGUuT6+m963HpPLdLY9p9zMzeYFGrnNC2ugznAiky7Yw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3773

On 07.09.2021 17:03, Elliott Mitchell wrote:
> On Tue, Sep 07, 2021 at 10:03:51AM +0200, Jan Beulich wrote:
>> On 06.09.2021 22:47, Elliott Mitchell wrote:
>>> On Mon, Sep 06, 2021 at 09:52:17AM +0200, Jan Beulich wrote:
>>>> On 06.09.2021 00:10, Elliott Mitchell wrote:
>>>>> I brought this up a while back, but it still appears to be present and
>>>>> the latest observations appear rather serious.
>>>>>
>>>>> I'm unsure of the entire set of conditions for reproduction.
>>>>>
>>>>> Domain 0 on this machine is PV (I think the BIOS enables the IOMMU, but
>>>>> this is an older AMD IOMMU).
>>>>>
>>>>> This has been confirmed with Xen 4.11 and Xen 4.14.  This includes
>>>>> Debian's patches, but those are mostly backports or environment
>>>>> adjustments.
>>>>>
>>>>> Domain 0 is presently using a 4.19 kernel.
>>>>>
>>>>> The trigger is creating a HVM or PVH domain where memory does not equal
>>>>> maxmem.
>>>>
>>>> I take it you refer to "[PATCH] x86/pod: Do not fragment PoD memory
>>>> allocations" submitted very early this year? There you said the issue
>>>> was with a guest's maxmem exceeding host memory size. Here you seem to
>>>> be talking of PoD in its normal form of use. Personally I uses this
>>>> all the time (unless enabling PCI pass-through for a guest, for being
>>>> incompatible). I've not observed any badness as severe as you've
>>>> described.
>>>
>>> I've got very little idea what is occurring as I'm expecting to be doing
>>> ARM debugging, not x86 debugging.
>>>
>>> I was starting to wonder whether this was widespread or not.  As such I
>>> was reporting the factors which might be different in my environment.
>>>
>>> The one which sticks out is the computer has an older AMD processor (you
>>> a 100% Intel shop?).
>>
>> No, AMD is as relevant to us as is Intel.
>>
>>>  The processor has the AMD NPT feature, but a very
>>> early/limited IOMMU (according to Linux "AMD IOMMUv2 functionality not
>>> available").
>>>
>>> Xen 4.14 refused to load the Domain 0 kernel as PVH (not enough of an
>>> IOMMU).
>>
>> That sounds odd at the first glance - PVH simply requires that there be
>> an (enabled) IOMMU. Hence the only thing I could imagine is that Xen
>> doesn't enable the IOMMU in the first place for some reason.
> 
> Doesn't seem that odd to me.  I don't know the differences between the
> first and second versions of the AMD IOMMU, but could well be v1 was
> judged not to have enough functionality to bother with.
> 
> What this does make me wonder is, how much testing was done on systems
> with functioning NPT, but disabled IOMMU?

No idea. During development is may happen (rarely) that one disables
the IOMMU on purpose. Beyond that - can't tell.

>  Could be this system is in an
> intergenerational hole, and some spot in the PVH/HVM code makes an
> assumption of the presence of NPT guarantees presence of an operational
> IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
> code, some portion of the IOMMU code might be checking for presence of
> NPT instead of presence of IOMMU.

This is all very speculative; I consider what you suspect not very likely,
but also not entirely impossible. This is not the least because for a
long time we've been running without shared page tables on AMD.

I'm afraid without technical data and without knowing how to repro, I
don't see a way forward here.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:13:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:13:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181193.328233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdiq-0000lz-V1; Tue, 07 Sep 2021 16:13:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181193.328233; Tue, 07 Sep 2021 16:13:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdiq-0000ls-QD; Tue, 07 Sep 2021 16:13:20 +0000
Received: by outflank-mailman (input) for mailman id 181193;
 Tue, 07 Sep 2021 16:13:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNdip-0000lm-JN
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 16:13:19 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 831eba64-0ff6-11ec-b110-12813bfff9fa;
 Tue, 07 Sep 2021 16:13:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 831eba64-0ff6-11ec-b110-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631031198;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=ZjBGaPM2bSvNwBtVv/unW1+T+h+jUN3bJ5+Rg602PtM=;
  b=Suq7z8Hhrh6HmFyY+pCiQOvazipPLEjieCpI7alDsemkrVcLTnSWD9DP
   +EseKAm2DOvWRsHf94RJsktjFQfgeQTTGwoc6H2M17vyQyQ8g/SjueQgI
   ft42fCAjvTF6VXHdh5qD2oQGwNyPDGOqLfHeQ+sZQjc9HHJPsjwFN31HG
   M=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: WeDhx3aZTsNwSrUt37wthJ203zdY78m4pOa6SDNNYr5t0GRjpqseDuASX5YpLqgC4jl2ewWWch
 d8f2CBraCCkGMPkrasv8pOtI/JaxV6DduaNkzHVV6oVwNft8VKpC/Ht70MEAjdtT64c5D5yTql
 PTKx/gi37ftDkVUPh4VeAG/tRPFJYv8duy8Vc0QrV//oEz/uoPhTEicKk/lzO0XMRpJqdmwdkl
 Lsm2QNq4NFP9Ib7ku743FRepIPgknZWsy2uZZ4L1KPi+dceRzV7vHQvYlQiEQB8FcnBuia80Fc
 lR5UXWG7Em49TbEUJ+6tgPbK
X-SBRS: 5.1
X-MesageID: 52577414
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Diz7g6NVyot3dcBcT1b155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/OxoS5PwPk80kqQFnbX5XI3SITUO3VHHEGgM1/qb/9SNIVyYygcZ79
 YbT0EcMqyBMbEZt7eC3ODQKb9Jq7PmgcPY9ts2jU0dKT2CA5sQnjuRYTzrdHGeKjM2Z6bRWK
 Dsnfau8FGbCAoqh4mAdzU4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kLEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 bxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72zeBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlJXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbZrmGuhHjXkV1RUsZiRtixZJGbAfqFCgL3V79FupgE686NCr/Zv2Evp9/oGOtF5Dq
 r/Q/1VfBwndL5gUUtHPpZ1fSKAMB2Fffv9ChPhHb3ZLtByB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="52577414"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=K7SlWlGwaXyjVtbqpkeq/EzSChgvLRV7UjG0eOT04HZDkCqujU+wBlpWeVuRjFpwMtyBVsxB5ODz944gHYHp2JP24QJ3aKw2k3+HYr/19u/2XX0bkzDE85mvTZFbXEWGNbWViqr3OCX/1H4QIfyIoLgP4sH3lyAJhk4ZuEnfU7EwB8Ypf55URoleztL2ds+BeeVZaBdKJ3yFrzP+uYMCjIjSgjN6twb/pfPGB9NETYyageS2+5j5niPIqtQNA3r6ZprJeJVGkqVasWhfvesrdZlmYqkEcIgduWVJcMy2xjhwpATwZFQlsLFpb9+g/GbyEX+dDRuktKtOoSoM9N1czA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8OsbXCrpG5uZi/MCeMp64W912DDWz+WDnlhKCE3efRQ=;
 b=H0cSJ4A8CTdxL5zQNVo07fwWmN5V+yy5yAT3R/DCY86JaE78upTgbLcAMGhDHvMMppiKcYbVCETyiblKgaJ98CGLqOubYcIgVtPPMqMf+dXz1Xp5f7RDFOnjm2LsZdo4v834Dj0IqhtrPB7HqO3cgzFRBYry/7l512fPZWJchpEVINbz0PdyLpzEjtdnQBs5g+vyoeb+xg3t6cecPA6ya2vRAQC6dgrk9rj7jQwJ6VfTl+YfiyHOLjWnd/8lAX775cwWbsYUQcHlOOdzvgk/am8CJLXS5fEdVMpioN/xM7U4Lc1vf8cibOCLV3PKOJEJDadwbdouKvWsmWUnXJ2yWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8OsbXCrpG5uZi/MCeMp64W912DDWz+WDnlhKCE3efRQ=;
 b=lob7k9njJbxxKgxlsS8UXQYaDZfWtmybmGy7fmIrpXBSVycyLd8Qr1tW7U1pz5XM0GrakzPrLJOozaU8ihMg5lCjCIv83q9bLldb5Omq4/n88ZiZG5akNwkiD0+if++A3bEY7nL46FtZ8FaOfsB3M/5TNT3XltVnhTCRtpmTocI=
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210817143006.2821-1-andrew.cooper3@citrix.com>
 <20210817143006.2821-3-andrew.cooper3@citrix.com>
 <606ce242-65d9-2180-8330-525fe5ec5562@suse.com>
 <351b2066-d01f-16d2-f1a0-bd7e2fe0965a@citrix.com>
 <968441f7-cf76-c98d-b256-18416ceca3bc@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/3] x86/amd: Enumeration for speculative features/hints
Message-ID: <dbcf485a-657c-6b27-8e69-31a998eec81e@citrix.com>
Date: Tue, 7 Sep 2021 17:12:52 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <968441f7-cf76-c98d-b256-18416ceca3bc@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0481.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a8::18) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5bbbcd21-5fbb-4d72-079c-08d9721a5c03
X-MS-TrafficTypeDiagnostic: BYAPR03MB3991:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3991CE002CB9C309C2DC0740BAD39@BYAPR03MB3991.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: SiMMn9rSfPUqZ3BZ1FYuZoG/04MkPPSORo24oa2zUKk1hCGcA9I459LTu4HbMMLRaFFT8fuyqV73wVvrrBEYf5unGDlbYAo5cJo3th/xz48G7g2IIyD5cO76KRg/I+iKtQ/ZVgoic00p4GlOs5K3mHyFEEYELV0h1EgiGDlZBhrUSv5Dn528we3bIbyi6Y7kq0T75TIVVhao7iomumjNbH2qWepka5loYMOb6xz4FwjyAIPD12PVk9J4yki+tLL7lnkiixwC41IZGOTSwOQhQMOVL16GahQiJqEyF5zUtQctwofFqGjH7bkvrdANxywHoEwTO8wmSYiiqAU0ItfR+WYS9ED4YCnRHu/XaYhw9131a11YjnU6BstVSuoakwFwSnRfKRfELmKYwi64QXAtJpJadnPFZMpwEAdIxX0vKSan6mJ0Mjy/VYnjQYpzRMEaKPo7klLO3Z1Ov2LSX8khH4/OYg7Y1hJk7iTpMrHnzjd2tA6FpsOfiW6hmlU1hJqqwnw5Dt0/rHUe7wKm9kW5kM4WecHzHVQYZY6iVClP+wYYzlTWBcxRnVHqvyzk5aLkqDrCT9aLrmOK6AN4jP9lYPPLhUgrKY5JGTYslBleQuFFM4xwkY8IcnlMxmbSCa3+eghykdbjUXL5ImsbSSKSSLQt/vwataEVykIKqEsC1kVbn2cFX5A88vf1rBG2nPXd52E6mGsLQinKteUANXhttsI8r+woK+sueCTjEoHgA/Y=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39860400002)(376002)(366004)(396003)(136003)(4326008)(86362001)(31696002)(66476007)(66556008)(66946007)(38100700002)(6916009)(16576012)(316002)(54906003)(2616005)(2906002)(5660300002)(31686004)(36756003)(478600001)(6666004)(6486002)(26005)(8936002)(8676002)(186003)(83380400001)(55236004)(956004)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YjRVYVE2QkZTSThzV1FpYkROK0VVMnFodmVzV0xSQ1NxOGFoWGxBM2dLOVBx?=
 =?utf-8?B?cEhqRlBveXVXMHVhaXZiVmhvTmEvalNtS1AvR1pwTG1Lck40Y0w3eE5oVHE5?=
 =?utf-8?B?QnNrakNkeGhsNUhLa0FkeXZudGR4NDJWTzQvR1psanZPcksvMnFzT3QzWVFk?=
 =?utf-8?B?dldKSW80TUNzMkFTZGt5K0MxWktrNlFlTjNXTGpvZzFTTG5ZNjNCanF3ZDdO?=
 =?utf-8?B?R2I0aWtyZW9xakJCa05nMEIyaXJyRFNudW5GTE9kS1FVRGxFN3ByS2U0SE1M?=
 =?utf-8?B?NVhhL3BFUXRvZ1ZIQUs5NVVCUC9PRFM5eXpUWS9oQUNkMnlJM2RvSVhhaFky?=
 =?utf-8?B?R3hwZ2NrWWMyaXQ3aDBZWkpqYWN4ZFRLeGlSN1MxZVpDcGY0dTVxS1RDSXJa?=
 =?utf-8?B?bnZqaGt3Zlo1K3g0SDhkWTBrWmhEdnFKa3lSWEtnS2NINWxUZlNVcW9nN3dy?=
 =?utf-8?B?dGhocU45bXEwZHR5cWxtMFFCYmRGTHBjaVltMnI1WGRmVWZzNXo5dUMxekNT?=
 =?utf-8?B?R0F6UDdtaWZqY2ZCRjFTeHo3MDJRNUFQMkg0ZUpIT0lnMnlmM2ZWVGlsREZD?=
 =?utf-8?B?aDhGNVBIMnBoM0ZFVnlNNjlYNmwwWlR0ZEtqRng3UU5BYk5SRzBRVUVZQTZR?=
 =?utf-8?B?M2hJZ0NUL0RYM3ExYjN6R1hYcDU4bWgxU0tYWmJpVHlRK0tqc2wwNHVxZzJJ?=
 =?utf-8?B?YnVXT1ZSdDlLTzE2UTAyc0EzSU1zRDd1SEVxekN2dVJGU1lDVlJCdGNKRm5l?=
 =?utf-8?B?V1lYejAzT1N2dC9zT2Z6b0s5endhTHMvYTBtdDAxcFJKMVA5OVFOWWRKZ0tO?=
 =?utf-8?B?MjFtS2t2SG1zNHNxY09BT3FjUkZyNFpEdXFOQTRkNlVKdFBqNUZidC9vZlk4?=
 =?utf-8?B?YnVTZW5zdXExbG90UGVNTE1mWmRVOEI2eWJZYXBlM3UxdmdISWZCaFdaVmNa?=
 =?utf-8?B?aVFwSk5uQ29DaFhJdTlqNVY2alpjVExSQ3VVVjF0SUlUTVViOFBKMG5DZncw?=
 =?utf-8?B?M0ZiQlZveEgrT1ZjSkwyTFFxRytlZ3psZXF1K0VxTCt6MFBmaFJndVNyVDZ6?=
 =?utf-8?B?QTI4NkdCeTQ5RVdwTkZNN3V0KzJ2blpZdEJJeVo2MzFFM1ZWOU5RSVVSdHpq?=
 =?utf-8?B?SmtkYThiY3BKcDEweTY5aU1RN0VucTg4bDZzdkhMZ0tXRXVVWEZxYVgrb2Yx?=
 =?utf-8?B?UFRpa1pVZXdSdk5UN3VkeCsvSFBSYW14TmZsbWM0OWIrbXVneDdLM3FvRytS?=
 =?utf-8?B?WWg3Z0RpYWhBZkc0Y1JRVmNvNVRNdC8rczNQcFpaRTBZN1l3Sjc1ai9UUzRq?=
 =?utf-8?B?dFphZkNySkxndXNaYTNlNjRqVTJqd3oyeUc0K0h5cURjSUdERDNYN0IyOVo0?=
 =?utf-8?B?ZFFmSXRDM2lTZXp0Z0RtYjFoMy9UVkdpS3BOZm1pejRqOXM5NTdEMzMrd1Fx?=
 =?utf-8?B?bStjVjlVMmc2U2xsTlkwcnRVNXBrUGtVRFF3YUdIWnVnblh1b1Rwd3VBcU1p?=
 =?utf-8?B?cmc2Q1ZLZWkrTEJiTG16NFA0MVZIZG5XQkQ2NjFPQ082TkdiYmJzZmFZNHk4?=
 =?utf-8?B?NVRyMVFjWE9lT0dxNEZqOVZpTlN2V0oya3RvMFYzdUpvdU9kUUdmdkRMNHpX?=
 =?utf-8?B?dmtaNEdNTHR1aVJrdHgrem9TSml1SEJSTjRXL2QwOGpDcjdMekUzb2MyMFVF?=
 =?utf-8?B?dytiQU9mdlpqbmh1ZExOdzZZckQ5enA5ZVB0Um11eTd4M1ZJK1RmOExTVllS?=
 =?utf-8?Q?Pc6AYEOC/nKIyXbEwTvnQW46ugiEZdQH2omQENJ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5bbbcd21-5fbb-4d72-079c-08d9721a5c03
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 16:12:58.9333
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bmBz/ID+jSTgP4p0zJ0TZdweuND9APX95qKzOphWDRv5jvEdK74LGcSzIt2KhIUbGJSczHWNbjNM85GWZGchk0I1K/CSMvvf+q1b3ahSHkI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3991
X-OriginatorOrg: citrix.com

On 24/08/2021 16:15, Jan Beulich wrote:
> On 24.08.2021 15:26, Andrew Cooper wrote:
>> On 19/08/2021 15:47, Jan Beulich wrote:
>>> On 17.08.2021 16:30, Andrew Cooper wrote:
>>>> There is a step change in speculation protections between the Zen1 and=
 Zen2
>>>> microarchitectures.
>>>>
>>>> Zen1 and older have no special support.  Control bits in non-architect=
ural
>>>> MSRs are used to make lfence be dispatch-serialising (Spectre v1), and=
 to
>>>> disable Memory Disambiguation (Speculative Store Bypass).  IBPB was
>>>> retrofitted in a microcode update, and software methods are required f=
or
>>>> Spectre v2 protections.
>>>>
>>>> Because the bit controlling Memory Disambiguation is model specific,
>>>> hypervisors are expected to expose a MSR_VIRT_SPEC_CTRL interface whic=
h
>>>> abstracts the model specific details.
>>>>
>>>> Zen2 and later implement the MSR_SPEC_CTRL interface in hardware, and
>>>> virtualise the interface for HVM guests to use.  A number of hint bits=
 are
>>>> specified too to help guide OS software to the most efficient mitigati=
on
>>>> strategy.
>>>>
>>>> Zen3 introduced a new feature, Predictive Store Forwarding, along with=
 a
>>>> control to disable it in sensitive code.
>>>>
>>>> Add CPUID and VMCB details for all the new functionality.
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>> with one suggestion:
>>>
>>>> --- a/tools/libs/light/libxl_cpuid.c
>>>> +++ b/tools/libs/light/libxl_cpuid.c
>>>> @@ -274,8 +274,18 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_l=
ist *cpuid, const char* str)
>>>>          {"rstr-fp-err-ptrs", 0x80000008, NA, CPUID_REG_EBX, 2, 1},
>>>>          {"wbnoinvd",     0x80000008, NA, CPUID_REG_EBX,  9,  1},
>>>>          {"ibpb",         0x80000008, NA, CPUID_REG_EBX, 12,  1},
>>>> +        {"ibrs",         0x80000008, NA, CPUID_REG_EBX, 14,  1},
>>>> +        {"amd-stibp",    0x80000008, NA, CPUID_REG_EBX, 15,  1},
>>>> +        {"ibrs-always",  0x80000008, NA, CPUID_REG_EBX, 16,  1},
>>>> +        {"stibp-always", 0x80000008, NA, CPUID_REG_EBX, 17,  1},
>>>> +        {"ibrs-fast",    0x80000008, NA, CPUID_REG_EBX, 18,  1},
>>>> +        {"ibrs-same-mode", 0x80000008, NA, CPUID_REG_EBX, 19,  1},
>>> Here and below, how about dropping the "mode" part of the name?
>>> I can't seem to be able to think of any other "same" that could
>>> possibly apply here.
>> ibrs-same is very ambiguous.
> I'm curious as to why you think so.

Same what?=C2=A0 There are load of plausible options, e.g. "privilege".

"mode" is the second most important piece of info, behind ibrs.

>
>> =C2=A0 The only other reasonable reasonable
>> alternative I can think of is ibrs-psmp as an abbreviation for
>> ProvideSameModeProtection.=C2=A0 Obviously, the "Provides" bit of that c=
an't
>> be dropped.
> Then better stay with what you have I would say - for me "psmp"
> immediately raises the question "What strange kind of SMP?"
> While not tied to any formal naming, I could see "ibrs-sm" as
> an option ...

That's an initialisation of a shortening, and too far removed from the
original context IMO.

Given nothing better, I'll stick with ibrs-same-mode.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:14:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:14:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181200.328243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdjv-0001Ok-B4; Tue, 07 Sep 2021 16:14:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181200.328243; Tue, 07 Sep 2021 16:14:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdjv-0001Od-89; Tue, 07 Sep 2021 16:14:27 +0000
Received: by outflank-mailman (input) for mailman id 181200;
 Tue, 07 Sep 2021 16:14:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNdjt-0001OL-Ld
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 16:14:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 30c8a34e-73ef-4329-8462-3bcbdaadde8d;
 Tue, 07 Sep 2021 16:14:24 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2053.outbound.protection.outlook.com [104.47.8.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-YfMEg0chOuetOUYREYE7pg-1; Tue, 07 Sep 2021 18:14:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2448.eurprd04.prod.outlook.com (2603:10a6:800:52::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep
 2021 16:14:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 16:14:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0027.eurprd09.prod.outlook.com (2603:10a6:102:b7::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 16:14:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30c8a34e-73ef-4329-8462-3bcbdaadde8d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631031263;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LcbSHEuJ/n74+zmRLZEo33hTo1f5VMgflTIa8T0fZDo=;
	b=awOxYUlK1ZNufIwFjdxTMmqQAM3jdP9MYKCQ52OAXAic1RTyE7d3Haiv/dRnuXZfK+OptC
	nmi/xn7aUPerdG+kqbRnC6bh/LB++13qBa+S87eaPLFE/xd5F0p2Ft1cBpCJNOF2sKRYBw
	yU0tfPIG8oFBNnRIF3GLnofmTjhj/J0=
X-MC-Unique: YfMEg0chOuetOUYREYE7pg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=O3x2M6ZNPBorl0v/ebyO/SUSa6mf94oeLXF8VSaWjisitGFKx+VnZOSNKsjdeK+OC8gssgIH3Ttl1UQcogKgGCqqQ55wppPCifo4NAZnBe0QovFu1L4bnMkC8+jjgaSK4uedNlZxj/ZJvcXPadD99Vwb0BMvUhtYksdm4XWeb2QaVHHDKq5GZcvT1/r+yV8YUw+nd7lokoJZQRl6TU5JKZ1zQ61NrPIMct5rp+oiHMqRODHvkrnG/Na2W6D/5Nm4RXfA6urjYvf5+wLnNMVOw6l1Q05eqGnagKOVCza2VWG8wmSWLsMbxMfVUEanRhaGuIvQcYQPogs5UpES9zJszA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LcbSHEuJ/n74+zmRLZEo33hTo1f5VMgflTIa8T0fZDo=;
 b=hlECaybXcC4+Gqa0ZwV4tIXbyv1K0tcfdO0905bXclPl03sBWJcmqDkXofYLTXOiRZCev3CIIUjEWvaCGMgUx5itykjLU1t4uj3MH88agT3otTrO5HMq/hdDybDiIXoUQxsff0qlygti68A/W6WFnfr9lsEIZi49N9RTKQBnGzzAf9I49ITsZt6TEfTSz2Iqg6KeazRxE908KkzLP3V5XHt4K9ftHh2t1d2UfM3okwqUjBWq75LEtJ5GD2rneVlrOrX1il9gcbm9f9+sd9QVY0VI83KkeC4CMydYFbKyXn+pwb6uSJt/Eav8rwAixcpaJidFN+DKIEBIGHS7D2amOA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 10/12] xen-pcifront: this module is PV-only
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>, Boris Ostrovsky
 <boris.ostrovsky@oracle.com>,
 "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
References: <20210907153017.GA741607@bjorn-Precision-5520>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <82a99bb4-85bd-61cc-85de-f8dd9d1f98ed@suse.com>
Date: Tue, 7 Sep 2021 18:14:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210907153017.GA741607@bjorn-Precision-5520>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0027.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5aaa3d38-b7b0-47bc-14cd-08d9721a8b62
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2448:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24488025797F5C4F35645645B3D39@VI1PR0401MB2448.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2733;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Y7vEub8SuX+2Mjk/NCi6siHY+3n/JlgiU+KUsKWvwv0WFWAz7osXDXOUwYpnA2VEZppgAB7SxdGkSWpGfJkjhLccjlp2S8zMfr2WS9ybDVvyMUZTyaATFXy4dr2JGV9OJTu0//fR098gSXBeXNkMpyCgVm15Y3XwSXoc+kVt/meF+kaQHIM/dljJll7pel0H+dEpgsrOav7gjhCSb/15B03cYQxvOHkiOn8ziNP0t+fy5m5Ymjr/J2+9xvJEM7GGf9y+T+lKnyBsw2/7KLkF8jLUdtHlWR3TAsh+NJAGNCD4UGOcBzIs/379IGogu+RBPXHi2eUahtSrdz/jX2rHkSEQQeoTHcBZ+d8GpbwX3ZCJ0NSv8QjhcZmAyAGp8LpCFQ9XAuAdfgAfBfMD3g4xZHZuTKcG29bm4BMMTynwmbN6cH/VWauVfQwm5lUwtrfdY9YoCkCkbQkcDnaW0md0O+sKwza/lEn7YSwDO86eIfgnV006xwd4D449TT8VqXB40AAsDYO3uFhm0EJPtk2JPiMm+rF/j3YoMjslrFhfID0BgIhuPaIVZqystPZ1DufuemK0dd2X/P3buPdYNZ/qQ76c3GFF/UV7tGx2rcFXniTHAQZN03I3INVB1AeyHoZZCqMeglk8KC/xG1zI1t6ntSjDO7jlNjje6YEtGNXy4aLOx2RNkg4In1HNhqaIRNr4RC5BM/4rTrHIPWA0f6V/drfSZNS/8/zOCFN2H4g7CP6CK1X1aIMm/r6X4raCLYu2
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(6916009)(316002)(38100700002)(478600001)(5660300002)(86362001)(26005)(53546011)(31696002)(186003)(2906002)(6486002)(8676002)(66946007)(31686004)(8936002)(66556008)(66476007)(83380400001)(2616005)(956004)(4326008)(36756003)(54906003)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NjBNdHRlVUI4VEZiYTJGUTg1dVcrRjMzalZ3WlNIN1FReStpTm5LbnQ1U1Nn?=
 =?utf-8?B?NlhjVENFM3EvOEhyenhiU05xaTI2eTlYUTl3WkFwTnF1VmNQenBMdS9UZU9u?=
 =?utf-8?B?VzFLNEJuZkhNTERWM1B1QWtyanBFa0dUT2Iyemx2aC9xS21YeFovcjRPTk5p?=
 =?utf-8?B?ZGRwNEpiY0ZzMFNxS1VxQ3J0WXMydm1OSUlaUFczWFZuUXo1Y2dkRkh0NVB5?=
 =?utf-8?B?emc1Q0ZuMmY3dmVqZmVPS3BVYUpOWGtKRmh4TzlrMmlhZ1d1RjFCL0xWMnpE?=
 =?utf-8?B?NUVGalU0OWx2clhveTFuQVptS0J5ZXNrYXpqajF6cmtpQlNPa1o4Rk4zTkNx?=
 =?utf-8?B?QlZwcjljLzZsQTJUb3NGaEw5Z0d4bjNvdjJMTWlLQ0Z4ai9ycG8xa240OVBF?=
 =?utf-8?B?Tmh4ZTBySktZYTJNVVJrMmFSdzdHaGhEZHY0UWZ2SXdFaVZ5aXd4QmU3Rndr?=
 =?utf-8?B?Tms5VStGbVFsTUNjTCt5bUw3SmwvcHRILzdKRjVxZXBrZHNxZDc5SlN0eDRR?=
 =?utf-8?B?Z28rTG1oWnlHUVhsQ1dhVHc0U1J3bjZYQVZ2a1ZNL3AxaEJybUZOcW5DQXZz?=
 =?utf-8?B?VldqWXBINjltVlZrYlJySFROb29wRjBZeThKangrWG9ZSE9ZSXB4RFU0cGxh?=
 =?utf-8?B?ZW1RM1RQelM3N3JLREt3WTFKdkNGL0NRWlYwbHMrK3o1cUtZM2NEdERVbDBD?=
 =?utf-8?B?QmJEZ1FxaGphVUgwaFE5NnJ1eUxScmtMekgzM3BtMFBpUkxhd3c3VjNaMGMw?=
 =?utf-8?B?ckhpdzF2Ui9TQnJQRFF6ZU9WdzRKaVRWMVIxMDY3UXlxOHpoQUk0YmhZWkp2?=
 =?utf-8?B?TDUwejBHbi9Lb0tIOE5QZDRjV3o3UEJvT3dHS1RJLzVXRFJyUnN6OUJyNXNB?=
 =?utf-8?B?VVBFRFpnc0FsLzhhK3EyMi8veG44TkJJUXloMVhTY0FocUtwdnpTSTROWVJN?=
 =?utf-8?B?SENGNSs2OEVWYUVaNFRSYVlLSW9qS051cU5qckJQTnZjZVZYWlhCZ2JGUDZr?=
 =?utf-8?B?MWVMdERYWk1rTTh0RE5KNzNuU2E0d1VnRjRrK1hkSkIra1lyNm1zN2xkUThU?=
 =?utf-8?B?U0Jtd1FhRGxGR1ZLeDNtKzNEcTI3SkNMZWRjTGE2bnkyaXZPa2FFc0U2N1lz?=
 =?utf-8?B?QTN6Rjk2VTRzVzh5WWNYUmI1WDkrckpkQ1hpM1FXZUcxRHRCUHNPQmVyYnFE?=
 =?utf-8?B?Y0gzVzVBUThVVnpUL1JyMitPYUVnTlZjQXZwWWg5c2VjZ2gwVnJkcU1pT2ZL?=
 =?utf-8?B?alZYaFJaa2JhcmpiR2ljMnRNU1hXejJRSlo5ang3M0dhdXZKcE5lUE01b3Zn?=
 =?utf-8?B?SGVWK2pWQk84bFE4Ukd4cXdDZXBiSUpxc0llSzRCb0FoQlpLZTlvTE95eGlW?=
 =?utf-8?B?aGxnSkNGRzRCSTVQc1lEeEdQM2FLNXNXakw1TWZPdjdMQktHeDFlN1JtQTN2?=
 =?utf-8?B?RzNXSG9Halp1MGNzNUVZd2lqeGxCRldTZXJpRXB1Y3FQN1ExUHBvV3ViMUVV?=
 =?utf-8?B?QlFGdHc2cTV0blkwb3pxemNCdW1XSGxuMGFXUmRZODJuQkJMemZDOUI4Y0ll?=
 =?utf-8?B?cWdOQmpRY1Jyc0E4Y1JCVjdES0E5bWE1dXV6K0tDajRLQ1d6ZENxOHJPSkZW?=
 =?utf-8?B?SER0by9Bd1ZKSjY1aktWQmpOUDdONjdpZDFTaXcwQVlmWmdpSFpiMXY4YVFK?=
 =?utf-8?B?MTh5Z25zM3EwcDYwV3ZVd25MVVpaT3c5Q055L085UDd3a3ZyWE82NWwxTFJJ?=
 =?utf-8?Q?uVv0HZ/5AMd9M47xGdt8+xVYzo0SEtzUVzXw9ST?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5aaa3d38-b7b0-47bc-14cd-08d9721a8b62
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 16:14:18.3962
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Pso7JENlOr8XTGDxqLt62uiVP6Bzb2jYhXKVqkYKWnLwQSThe9r8WdSzfRPA5JEpJJyUnTREAqf8EKvPoTNP6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2448

On 07.09.2021 17:30, Bjorn Helgaas wrote:
> Update subject to follow conventions (use "git log --oneline
> drivers/pci/Kconfig").  Should say what this patch does.

I can change that; I don't think it'll carry any different information.

> Commit log below should also say what this patch does.  Currently it's
> part of the rationale for the change, but doesn't say what the patch
> does.

"There's no point building ..." to me is as good as "Don't build ...".
But oh well, I can adjust ...

> On Tue, Sep 07, 2021 at 02:10:41PM +0200, Jan Beulich wrote:
>> It's module init function does a xen_pv_domain() check first thing.
>> Hence there's no point building it in non-PV configurations.
> 
> s/It's/<name of function that calls xen_pv_domain()/   # pcifront_init()?

I don't understand this - how is "module init function" not clear
enough?

> s/building it/building <name of module>/               # xen-pcifront.o?

The driver name is already part of the subject; I didn't think I
need to repeat that one here.

> I see that CONFIG_XEN_PV is only mentioned in arch/x86, so
> CONFIG_XEN_PV=y cannot be set on other arches.  Is the current
> "depends on X86" just a reflection of that, or is it because of some
> other x86 dependency in the code?
> 
> The connection between xen_pv_domain() and CONFIG_XEN_PV is not
> completely obvious.
> 
> If you only build xen-pcifront.o when CONFIG_XEN_PV=y, and
> xen_pv_domain() is true if and only if CONFIG_XEN_PV=y, why bother
> calling xen_pv_domain() at all?

Because XEN_PV=y only _enables_ the kernel to run in PV mode. It
may be enabled to also run in HVM and/or PVH modes. And it may
then _run_ in any of the enabled modes. IOW xen_pv_domain() will
always return false when !XEN_PV; no other implication is valid.
I don't think this basic concept needs explaining in a simple
patch like this. Instead I think the config option in question,
despite living in drivers/pci/Kconfig, should be under "XEN
HYPERVISOR INTERFACE" maintainership. I realize that's not even
expressable in ./MAINTAINERS. I wonder why the option was put
there in the first place, rather than in drivers/xen/Kconfig.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:19:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:19:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181207.328255 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdoe-00029W-Ve; Tue, 07 Sep 2021 16:19:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181207.328255; Tue, 07 Sep 2021 16:19:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdoe-00029P-SU; Tue, 07 Sep 2021 16:19:20 +0000
Received: by outflank-mailman (input) for mailman id 181207;
 Tue, 07 Sep 2021 16:19:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNdod-00029H-Dl
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 16:19:19 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 59a4fa94-0ff7-11ec-b110-12813bfff9fa;
 Tue, 07 Sep 2021 16:19:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59a4fa94-0ff7-11ec-b110-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631031557;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=bHhckMg4yqxP26r9jo3gv+1xllUSkTSk4o7gtf2dNXg=;
  b=I69MU90t1NVuh+/UcjLzymYbVwJdGWCnWaEx8qa+NEbRrxBTY3X8A/Gc
   ALc0bfRlkHOer5v8EWyhcJ3m+rz1/KaypHc7HNtSwUellrlL0mTfc9zN/
   Yf8y3vOSkKEadFs22Gv7CH75zzDYw9Qjo4eBVfzYB2wjB10vWc5rKAr1H
   4=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 02+NUHQjLy+r/Kth1Kj2Zv6IvEIOeDUblDTTcOILZr3KkXssJY5oQY7q9+dyRxNfkj3Sdn6FXI
 QvbS1QIO4eWY5zQB7N6bm+ZaZcDs5apycj50VoNc41GgvdMr6gXJc2pZtNIYGS5yxR8iFQhNBq
 3iNUdh+GskCs/rkYdjZxanqsSt7L7oCphq8lY3PPoOYentAC0nOP44xzBqyD7P2xI57V7dsTVU
 RiP3JGaNic5+nm/yLld7USxCH34rAPzxTEFWOrpsyIYLS+/T1An5eG/a06UcyJv3W2IdVQ4fmy
 Nhb6IwibL1yWYonkAQe3XS7R
X-SBRS: 5.1
X-MesageID: 51777373
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:R1n+Dq7uy66rA6ji8wPXwPDXdLJyesId70hD6qhwISY6TiX+rb
 HWoB17726TtN9/YhEdcLy7VJVoBEmskKKdgrNhWotKPjOW21dARbsKheCJrgEIWReOktK1vZ
 0QC5SWY+eQMbEVt6nHCXGDYrQd/OU=
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="51777373"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, Wei Liu <wei.liu2@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 3/3] x86/amd: Use newer SSBD mechanisms if they exist
Date: Tue, 7 Sep 2021 17:19:08 +0100
Message-ID: <20210907161908.17402-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210817143006.2821-4-andrew.cooper3@citrix.com>
References: <20210817143006.2821-4-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

The opencoded legacy Memory Disambiguation logic in init_amd() neglected
Fam19h for the Zen3 microarchitecture.  In practice, all Zen2 based system
have the architectural MSR_SPEC_CTRL and the SSBD bit within it.

Implement the algorithm given in AMD's SSBD whitepaper, and leave a
printk_once() behind in the case that no controls can be found.

This now means that a user explicitly choosing `spec-ctrl=ssbd` will properly
turn off Memory Disambiguation on Fam19h/Zen3 systems.

This still remains a single system-wide setting (for now), and is not context
switched between vCPUs.  As such, it doesn't interact with Intel's use of
MSR_SPEC_CTRL and default_xen_spec_ctrl (yet).

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v2:
 * Fix whitespace style in amd_init_ssbd()
 * Rewrite commit message with the correct command line syntax
---
 xen/arch/x86/cpu/amd.c   | 69 +++++++++++++++++++++++++++++++++++-------------
 xen/arch/x86/cpu/cpu.h   |  1 +
 xen/arch/x86/cpu/hygon.c | 10 +------
 xen/arch/x86/spec_ctrl.c |  5 +++-
 4 files changed, 57 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 2260eef3aab5..3f6a8e6aa3b9 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -681,6 +681,56 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
 			  c->x86_capability);
 }
 
+/*
+ * Refer to the AMD Speculative Store Bypass whitepaper:
+ * https://developer.amd.com/wp-content/resources/124441_AMD64_SpeculativeStoreBypassDisable_Whitepaper_final.pdf
+ */
+void amd_init_ssbd(const struct cpuinfo_x86 *c)
+{
+	int bit = -1;
+
+	if (cpu_has_ssb_no)
+		return;
+
+	if (cpu_has_amd_ssbd) {
+		wrmsrl(MSR_SPEC_CTRL, opt_ssbd ? SPEC_CTRL_SSBD : 0);
+		return;
+	}
+
+	if (cpu_has_virt_ssbd) {
+		wrmsrl(MSR_VIRT_SPEC_CTRL, opt_ssbd ? SPEC_CTRL_SSBD : 0);
+		return;
+	}
+
+	switch (c->x86) {
+	case 0x15: bit = 54; break;
+	case 0x16: bit = 33; break;
+	case 0x17:
+	case 0x18: bit = 10; break;
+	}
+
+	if (bit >= 0) {
+		uint64_t val, mask = 1ull << bit;
+
+		if (rdmsr_safe(MSR_AMD64_LS_CFG, val) ||
+		    ({
+			    val &= ~mask;
+			    if (opt_ssbd)
+				    val |= mask;
+			    false;
+		    }) ||
+		    wrmsr_safe(MSR_AMD64_LS_CFG, val) ||
+		    ({
+			    rdmsrl(MSR_AMD64_LS_CFG, val);
+			    (val & mask) != (opt_ssbd * mask);
+		    }))
+			bit = -1;
+	}
+
+	if (bit < 0)
+		printk_once(XENLOG_ERR "No SSBD controls available\n");
+}
+
 static void init_amd(struct cpuinfo_x86 *c)
 {
 	u32 l, h;
@@ -731,24 +781,7 @@ static void init_amd(struct cpuinfo_x86 *c)
 	else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */
 		amd_init_lfence(c);
 
-	/*
-	 * If the user has explicitly chosen to disable Memory Disambiguation
-	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
-	 */
-	if (opt_ssbd) {
-		int bit = -1;
-
-		switch (c->x86) {
-		case 0x15: bit = 54; break;
-		case 0x16: bit = 33; break;
-		case 0x17: bit = 10; break;
-		}
-
-		if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
-			value |= 1ull << bit;
-			wrmsr_safe(MSR_AMD64_LS_CFG, value);
-		}
-	}
+	amd_init_ssbd(c);
 
 	/* MFENCE stops RDTSC speculation */
 	if (!cpu_has_lfence_dispatch)
diff --git a/xen/arch/x86/cpu/cpu.h b/xen/arch/x86/cpu/cpu.h
index 1ac3b2867a04..1a5b3918b37e 100644
--- a/xen/arch/x86/cpu/cpu.h
+++ b/xen/arch/x86/cpu/cpu.h
@@ -21,3 +21,4 @@ extern bool detect_extended_topology(struct cpuinfo_x86 *c);
 void early_init_amd(struct cpuinfo_x86 *c);
 void amd_log_freq(const struct cpuinfo_x86 *c);
 void amd_init_lfence(struct cpuinfo_x86 *c);
+void amd_init_ssbd(const struct cpuinfo_x86 *c);
diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c
index 67e23c5df9e3..56792146739e 100644
--- a/xen/arch/x86/cpu/hygon.c
+++ b/xen/arch/x86/cpu/hygon.c
@@ -33,15 +33,7 @@ static void init_hygon(struct cpuinfo_x86 *c)
 	unsigned long long value;
 
 	amd_init_lfence(c);
-
-	/*
-	 * If the user has explicitly chosen to disable Memory Disambiguation
-	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
-	 */
-	if (opt_ssbd && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
-		value |= 1ull << 10;
-		wrmsr_safe(MSR_AMD64_LS_CFG, value);
-	}
+	amd_init_ssbd(c);
 
 	/* MFENCE stops RDTSC speculation */
 	if (!cpu_has_lfence_dispatch)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index c310a7f6ac96..f0c67d41b85f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -326,20 +326,23 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
            (caps & ARCH_CAPS_IBRS_ALL)                       ? " IBRS_ALL"       : "",
            (caps & ARCH_CAPS_RSBA)                           ? " RSBA"           : "",
            (caps & ARCH_CAPS_SKIP_L1DFL)                     ? " SKIP_L1DFL"     : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_SSB_NO)) ||
            (caps & ARCH_CAPS_SSB_NO)                         ? " SSB_NO"         : "",
            (caps & ARCH_CAPS_MDS_NO)                         ? " MDS_NO"         : "",
            (caps & ARCH_CAPS_TAA_NO)                         ? " TAA_NO"         : "");
 
     /* Hardware features which need driving to mitigate issues. */
-    printk("  Hardware features:%s%s%s%s%s%s%s%s\n",
+    printk("  Hardware features:%s%s%s%s%s%s%s%s%s\n",
            (e8b  & cpufeat_mask(X86_FEATURE_IBPB)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB))          ? " IBPB"           : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB))          ? " IBRS"           : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_STIBP))          ? " STIBP"          : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_AMD_SSBD)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_SSBD))           ? " SSBD"           : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH))      ? " L1D_FLUSH"      : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR))       ? " MD_CLEAR"       : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL))     ? " SRBDS_CTRL"     : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_VIRT_SSBD))      ? " VIRT_SSBD"      : "",
            (caps & ARCH_CAPS_TSX_CTRL)                       ? " TSX_CTRL"       : "");
 
     /* Compiled-in support which pertains to mitigations. */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:22:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:22:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181214.328266 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNds0-0003rR-FL; Tue, 07 Sep 2021 16:22:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181214.328266; Tue, 07 Sep 2021 16:22:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNds0-0003rK-C2; Tue, 07 Sep 2021 16:22:48 +0000
Received: by outflank-mailman (input) for mailman id 181214;
 Tue, 07 Sep 2021 16:22:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNdrz-0003rA-AW; Tue, 07 Sep 2021 16:22:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNdrz-0008W9-6e; Tue, 07 Sep 2021 16:22:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNdry-0004w4-UO; Tue, 07 Sep 2021 16:22:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNdry-00061t-Tq; Tue, 07 Sep 2021 16:22:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OgiXK6N41G6Zmzj0XHu4XJlmQS9TcqMu8fQl7XWvshE=; b=668qqaShDEKMy5nwmpQa0ef0IK
	ndJU05J670Cw6+eDx7nVPtyuB55xDHBPcqQfQuc8MDV5wBoOds4PNYzKDHZBZ6uQFyyLWqqSHDvTO
	j1sp15ZcxXjI0BuG93UZ8p4kcJ8xZy0/B3hZIuEwRMOOL3N8+uTaS8R4VuVYDYq0UREA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164875-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164875: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
X-Osstest-Versions-That:
    xen=5ae4120635ad3cbf6064268162b0fb2c44eb41e3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 16:22:46 +0000

flight 164875 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164875/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1
baseline version:
 xen                  5ae4120635ad3cbf6064268162b0fb2c44eb41e3

Last test of basis   164872  2021-09-07 08:01:48 Z    0 days
Testing same since   164875  2021-09-07 13:00:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   5ae4120635..e70a9a043a  e70a9a043a5ce6d4025420f729bc473f711bf5d1 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:31:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:31:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181221.328280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdzw-0005HW-Bv; Tue, 07 Sep 2021 16:31:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181221.328280; Tue, 07 Sep 2021 16:31:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNdzw-0005HP-8N; Tue, 07 Sep 2021 16:31:00 +0000
Received: by outflank-mailman (input) for mailman id 181221;
 Tue, 07 Sep 2021 16:30:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GyZL=N5=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNdzv-0005HJ-2T
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 16:30:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fa587f1e-0ff8-11ec-b112-12813bfff9fa;
 Tue, 07 Sep 2021 16:30:57 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2057.outbound.protection.outlook.com [104.47.10.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-xzGIXKmmPOWIzO3Ef1a4fQ-1; Tue, 07 Sep 2021 18:30:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2333.eurprd04.prod.outlook.com (2603:10a6:800:28::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 16:30:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 16:30:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0024.eurprd09.prod.outlook.com (2603:10a6:101:16::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Tue, 7 Sep 2021 16:30:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa587f1e-0ff8-11ec-b112-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631032256;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aH3OuqSwJzb+8ZU3KTnwJsKyxX6bcxjI+WZ19P4EHUY=;
	b=jBB5J3VIGAxV3Njdha9p/JEV6kiYzbLQb580lnYG1SSkk4+nXs46EmLDQr4Qz9IrnaYbPg
	mIuaLG+S81GQ8PjwCA5nj4Vo3sdUCjRPhmiMaM01PfFcYSzSU9zFtB7zq/aLw8mW3S6qax
	kwuxw89Cw4yUWmmq3uXA8Sh0WhwIIT0=
X-MC-Unique: xzGIXKmmPOWIzO3Ef1a4fQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gjO9pJob/cDyZZx2niWHexw75N8srREp7wcjiaC6YJN3hqv6+IKGBy5w56amWYyPA/f5W+zQdB+ykd5UObIx62+fpVw9W4UEPRLbj7AvU/pKuL8xmloUI+9zIzWxNys2Jwpky+PGX2APMsnisnibmGytdadcwi7gKWGDsNiMR3gEX71j8W/9cXF4bMrJetTVGm4dWLA0p0hgRP6oOkQjJo9MPcT2E4wI/e9dUre/8ZNz8SixX8aFwuTdHURd42MK/uZ860tOpTs3Pwn1sYbV41nQ1/j932wv91atoScseE4rTze4qVo2uKWQfAat92/kQcnQXsdM14jSRltJUsOXxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aci6m3pjwbaz5O1KJStmBiwl09NYg9Epzo3SDRuPRc0=;
 b=GzohOCJce+guttojPssOmQoSdKjE6SgftLQQ2pOBCw74eyOVAbDlIwxD4JRNK2A3csjf8zcYbSlp84pSAGeSzSTwRu1vjy9glEM6UGfR5SSwU61bWilHLVQZN10ShwZuBOySZXN5tLDrxeG4DJjDaIYvk3VUeF3ifpNFmkjMKQPVskNzlpAqbVxdmwsasc0wSu3mOMyw/hr+F4vJWht3Z5PHAKdLthkWGsACpx5MK7LYvOYw5pinyINtn6g8IVg6VuSGEiX6a3C1/5XY4WJRwloGGtTutwMgDSfBTCXNL00OmSvLt9olKThUbjVixHNlomDJe8gL2CyvriN6Os0bAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
Date: Tue, 7 Sep 2021 18:30:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2PR09CA0024.eurprd09.prod.outlook.com
 (2603:10a6:101:16::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 93f00116-d657-4b80-1b97-08d9721cdca3
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2333782CC72C9E1E0E2ED536B3D39@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jVTVDauqvRrfUM0HTpofQdmj/lYdS+QszUt1QBpjfrmKBLd4LLyvvCkuQIfyK2/n/G31KUrN3fsGCMYLXX4+3Fz6Hhjt25QmF/cXh/ePffuXdPhrjLMK5rNY98oIWfJjIR4ftJ//koE9RfiW8nGFaixot2mkLkRe2G1irGB8hCjUxAoxLfco8fTAXWZqNF5Y+PIPCTl+i71Z6x/ODQk5x6R+V9c2kORmYoIvB+DKEp8jCgJcyk7pQtH+X7XYKp+p0OKmcAPHpVA8MstvoM8Rg9lL6Mbnryq5m7oHJ4414wrO/hIUSBpXDAtKo9Fq6/jiPeMqsXMBwYr2ecRsAsvTppyxnP/bDxX4WjfnPYscHjECsJdQnpMJW9MeKwyG/xLy8svcKxXKtX9nOQbeJn5rt0pESuC+ldUNo82Gq86KPzOKBmC6j/6R1ZNneCtY9oMYmIlt0iZOZAgVhePRcu/9dvBsQ2P3wVgj/F1SkOXRa8XChTMR6R9Qp2ruOl+cHPEkr3slE/napYV/5ROEDNfuAbPe3mk39IeV1bizZa+SF06sHbKHXuxkhAteLzz+3CCG+mluGUpNOJPjvdnStYIdP8pTcDD06SGLhG9JhcHTlEsRAcl8noypRmdhKrHlatc+u+3aUhMKiu0AY5TvBXfdYvsP3QHUGgJNfcfiK/sCg1RNHmNbpRtopKGZXiHyuzDO/pInAWwo+N3p9zlFqM5fCylx8AFU/kkHEwCfxYrTCjtBBvqhjYVnptD9ceX8q8Kz
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(39860400002)(376002)(396003)(366004)(38100700002)(8936002)(54906003)(110136005)(478600001)(8676002)(53546011)(2906002)(26005)(31686004)(7416002)(31696002)(86362001)(36756003)(66946007)(66556008)(66476007)(2616005)(83380400001)(956004)(4326008)(5660300002)(6486002)(16576012)(186003)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?EGzw3YmtUrxGtDEMC9ih/8Uc2Sv3ptGg5W+h0HpEvUYAE0jIr6DV9oPLcI5q?=
 =?us-ascii?Q?lPjmfGYRF+iEpD1PS21EqZpe8l7UqZs1aV4RRxBatFZi5OQ5w1v7gvvPd4y8?=
 =?us-ascii?Q?tKBy7g1pXqRoBOM456y/XUtNI5fr/QHjFMwert7GjFXU5MCMCxTTLQJ0ynTe?=
 =?us-ascii?Q?SlapRhkxiaWEKJ+ZNyOex8zguDtActtT6rKpeyfIjqL4zjbGf8B9pQffJRgy?=
 =?us-ascii?Q?WsPdHTl3IjGBq+03hiD0rqO8G2KOdJq5m3/rmYK/yXKnoSVnqQhJ/LhNICeH?=
 =?us-ascii?Q?DL82b9hxydJhbkHD8YpVabiQNfc3Yk1E4tcflTrL5Hpv6EeSy2355GoG4mJ9?=
 =?us-ascii?Q?GC3+0ZdizbJYfEbqt9GXwyVgXzXWGAlEX43TuEXZ4G5prkLwi8yhQrlvVfe5?=
 =?us-ascii?Q?cXgE379pvtWG2RIPu89MuAjwdIqfHf+QbgIBat6RuxXz1AVDEWJRGy6LFOdS?=
 =?us-ascii?Q?bGSK+djH51kg5kEBeL6kXGFgOleO1k5cF7PzbwQf0CVXEDOvHArR76IA1cN6?=
 =?us-ascii?Q?VEZvgrlFA7aBsjlqf/cJLhsqweSkQ4XICSYfj3Sftg0N2xjWOvqlPBMYHsnL?=
 =?us-ascii?Q?L/8DUNqXVroVJU+2tv5T00lqYw9piR6gAoeMuAkpoBoHEP8AOrsOFZaWESau?=
 =?us-ascii?Q?r8tEguD9pLreTNFsq7vIejc2Q1wlVDARPpdfTEAJGwgzXAQ98r2h9swMpm10?=
 =?us-ascii?Q?mNOup8nrf3MZBCNCNRC5Pcmapa727QnEhxeTPoFClJlQ0PFiEiIpC1M6WnYJ?=
 =?us-ascii?Q?OHj4ssCZ5aEX1d1peuXA4IaZ4aPGlexe3dEE6TnSlliRwSq/bJigckfnAoyZ?=
 =?us-ascii?Q?ysz0hoKHJii+8NzgOsOXJMAG5Ht9TmDJqcGV8Cmi7ROD5f1YpMKNBeXaENRc?=
 =?us-ascii?Q?Xytt2x/cNYSuFV8YnACnCiQatiGNZe1FJpaWGIBDvle87CleDqG2XNvrsDbN?=
 =?us-ascii?Q?KkEwQt9BFlRF2RjzBPXjqPOa4ChlZOfr4VIAwr/WP6nH0hi7OpCIa6rDJgUU?=
 =?us-ascii?Q?TpWahRSYMdLm9cFOx/xRqHUmk4uL0KEGCxH6XIXTM+Cbx/FIaU9WjXfxUvD1?=
 =?us-ascii?Q?WGGA8Xp7w+Ln6i4u6hUbTKsif4Y13owDxPl68tbvy6l4X9vxCGg5Gbx+WwKg?=
 =?us-ascii?Q?75h5YqkThz1+khmvpeOT1r1kHwDwrJ/6xdC2OTFXguxrDXHpQhXmwLTfWkWJ?=
 =?us-ascii?Q?XwVI1CPiD2SG/gdljcbMylJU/vD9l1oy/oOq9EUhEXEWXqC95RZkc9aKD5NX?=
 =?us-ascii?Q?tKG7uEoKUvkckRp3mWwvs2sp96CTJBs7ck77PEnUIV97WSGPwy3t3CjwI3aG?=
 =?us-ascii?Q?cp+FcWCR8OSYFKVJtrBQtePV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 93f00116-d657-4b80-1b97-08d9721cdca3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 16:30:53.7082
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8ekXdIyJsPFy2+2hcjoAmYOeIZzl+PKvQ5EM6ZlcN08Rj7JzcEA0TJ1/PcX7C5eDRg4qLviMtPcDyVDedVHw7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

On 07.09.2021 15:33, Oleksandr Andrushchenko wrote:
> On 06.09.21 17:31, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/vpci/header.c
>>> +++ b/xen/drivers/vpci/header.c
>>> @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pdev,=
 unsigned int reg,
>>>   static void guest_bar_write(const struct pci_dev *pdev, unsigned int =
reg,
>>>                               uint32_t val, void *data)
>>>   {
>>> +    struct vpci_bar *bar =3D data;
>>> +    bool hi =3D false;
>>> +
>>> +    if ( bar->type =3D=3D VPCI_BAR_MEM64_HI )
>>> +    {
>>> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
>>> +        bar--;
>>> +        hi =3D true;
>>> +    }
>>> +    else
>>> +        val &=3D PCI_BASE_ADDRESS_MEM_MASK;
>>> +    bar->guest_addr &=3D ~(0xffffffffull << (hi ? 32 : 0));
>>> +    bar->guest_addr |=3D (uint64_t)val << (hi ? 32 : 0);
>> What you store here is not the address that's going to be used,
>=20
> bar->guest_addr is never used directly to be reported to a guest.

And this is what I question, as an approach. Your model _may_ work,
but its needlessly deviating from how people like me would expect
this to work. And if it's intended to be this way, how would I
have known?

> It is always used as an initial value which is then modified to reflect
> lower bits, e.g. BAR type and if prefetchable, so I think this is perfect=
ly
> fine to have it this way.

And it is also perfectly fine to store the value to be handed
back to guests on the next read. Keeps the read path simple,
which I think can be assumed to be taken more frequently than
the write one. Plus stored values reflect reality.

Plus - if what you say was really the case, why do you mask off
PCI_BASE_ADDRESS_MEM_MASK here? You should then simply store
the written value and do _all_ the processing in the read path.
No point having partial logic in two places.

>>   as
>> you don't mask off the low bits (to account for the BAR's size).
>> When a BAR gets written with all ones, all writable bits get these
>> ones stored. The address of the BAR, aiui, really changes to
>> (typically) close below 4Gb (in the case of a 32-bit BAR), which
>> is why memory / I/O decoding should be off while sizing BARs.
>> Therefore you shouldn't look for the specific "all writable bits
>> are ones" pattern (or worse, as you presently do, the "all bits
>> outside of the type specifier are ones" one) on the read path.
>> Instead mask the value appropriately here, and simply return back
>> the stored value from the read path.
> "PCI LOCAL BUS SPECIFICATION, REV. 3.0", "IMPLEMENTATION NOTE
>=20
> Sizing a 32-bit Base Address Register Example" says, that
>=20
> "Software saves the original value of the Base Address register, writes
> 0 FFFF FFFFh to the register, then reads it back."
>=20
> The same applies for 64-bit BARs. So what's wrong if I try to catch such
> a write when a guest tries to size the BAR? The only difference is that
> I compare as
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( (val & PCI_BASE_ADDRESS_=
MEM_MASK_32) =3D=3D PCI_BASE_ADDRESS_MEM_MASK_32 )
> which is because val in the question has lower bits cleared.

Because while this matches what the spec says, it's not enough to
match how hardware behaves. Yet you want to mimic hardware behavior
as closely as possible here. There is (iirc) at least one other
place in the source tree were we had to adjust a similar initial
implementation to be closer to one expected by guests, no matter
that they may not be following the spec to the letter. Don't
forget that there may be bugs in kernels which don't surface until
the kernel gets run on an overly simplistic emulation.

>>> @@ -522,6 +582,13 @@ static int add_bar_handlers(struct pci_dev *pdev, =
bool is_hwdom)
>>>               if ( rc )
>>>                   return rc;
>>>           }
>>> +        /*
>>> +         * It is neither safe nor secure to initialize guest's view of=
 the BARs
>>> +         * with real values which are used by the hardware domain, so =
assign
>>> +         * all zeros to guest's view of the BARs, so the guest can per=
form
>>> +         * proper PCI device enumeration and assign BARs on its own.
>>> +         */
>>> +        bars[i].guest_addr =3D 0;
>> I'm afraid I don't understand the comment: Without memory decoding
>> enabled, the BARs are simple registers (with a few r/o bits).
>=20
> My first implementation was that bar->guest_addr was initialized with
> the value of bar->addr (physical BAR value), but talking on IRC with
> Roger he suggested that this might be a security issue to let guest
> a hint about physical values, so then I changed the assignment to be 0.

Well, yes, that's certainly correct. It's perhaps too unobvious to me
why one may want to use the host value here in the first place. It
simply has no meaning here.

>>> --- a/xen/include/xen/pci_regs.h
>>> +++ b/xen/include/xen/pci_regs.h
>>> @@ -103,6 +103,7 @@
>>>   #define  PCI_BASE_ADDRESS_MEM_TYPE_64	0x04	/* 64 bit address */
>>>   #define  PCI_BASE_ADDRESS_MEM_PREFETCH	0x08	/* prefetchable? */
>>>   #define  PCI_BASE_ADDRESS_MEM_MASK	(~0x0fUL)
>>> +#define  PCI_BASE_ADDRESS_MEM_MASK_32	(~0x0fU)
>> Please don't introduce an identical constant that's merely of
>> different type. (uint32_t)PCI_BASE_ADDRESS_MEM_MASK at the use
>> site (if actually still needed as per the comment above) would
>> seem more clear to me.
> Ok, I thought type casting is a bigger evil here

Often it is, but imo here it is not. I hope you realize that ~0x0fU
if not necessarily 0xfffffff0? We make certain assumptions on type
widths. For unsigned int we assume it to be at least 32 bits wide.
We should avoid assumptions of it being exactly 32 bits wide. Just
like we cannot (more obviously) assume the width of unsigned long.
(Which tells us that for 32-bit arches PCI_BASE_ADDRESS_MEM_MASK is
actually of the wrong type. This constant should be the same no
matter the bitness.)

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:38:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:38:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181229.328290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNe7O-0006OX-7Z; Tue, 07 Sep 2021 16:38:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181229.328290; Tue, 07 Sep 2021 16:38:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNe7O-0006OQ-4g; Tue, 07 Sep 2021 16:38:42 +0000
Received: by outflank-mailman (input) for mailman id 181229;
 Tue, 07 Sep 2021 16:38:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bt8x=N5=kernel.org=helgaas@srs-us1.protection.inumbo.net>)
 id 1mNe7M-0006OK-Q5
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 16:38:40 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 555f382a-462a-4eb3-ac83-329dd8215b30;
 Tue, 07 Sep 2021 16:38:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D418760E77;
 Tue,  7 Sep 2021 16:38:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 555f382a-462a-4eb3-ac83-329dd8215b30
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631032719;
	bh=kM4x6R7FRoHUCOnGXVz/Y0FnaHvMZjmGIfLn42Hpaso=;
	h=Date:From:To:Cc:Subject:In-Reply-To:From;
	b=kgIy3BUhv2UT6NbQOChJDvdO/ddqyb+6TdoPWIQSGyG0EZbk/qLuWzTimoMpC+HH5
	 bwX+/ZfHBVLxuGEChqx5IyxN0z46QnZOPUp0zMXmb0lpcl062y0vMY+JXDu48u3XCx
	 0wd2ByqQdRofvst0nS4V54iGJnDo1Es/FBimcg/RmdCrHGULjBs2TexrBHb7prHuCl
	 RjtZGpzzbLyiQNYxTeWFm6kyFImNgbTk60ye10VyP3P+pcpI+/LKQCYbSnB4Uujxef
	 X0ABVPjXHtLiE3QFE0p7R5MJimNBUekSXsZduzlGc5FfYHtSGdHkH/nxec/lHTrRHM
	 OpweP7+y/xQIw==
Date: Tue, 7 Sep 2021 11:38:37 -0500
From: Bjorn Helgaas <helgaas@kernel.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH 10/12] xen-pcifront: this module is PV-only
Message-ID: <20210907163837.GA747361@bjorn-Precision-5520>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <82a99bb4-85bd-61cc-85de-f8dd9d1f98ed@suse.com>

On Tue, Sep 07, 2021 at 06:14:16PM +0200, Jan Beulich wrote:
> On 07.09.2021 17:30, Bjorn Helgaas wrote:
> > Update subject to follow conventions (use "git log --oneline
> > drivers/pci/Kconfig").  Should say what this patch does.
> 
> I can change that; I don't think it'll carry any different information.

It might not be different information, but if you use the same
sentence structure and formatting as all the other subject lines,
they're easier to read as a group.

> > Commit log below should also say what this patch does.  Currently it's
> > part of the rationale for the change, but doesn't say what the patch
> > does.
> 
> "There's no point building ..." to me is as good as "Don't build ...".
> But oh well, I can adjust ...
> 
> > On Tue, Sep 07, 2021 at 02:10:41PM +0200, Jan Beulich wrote:
> >> It's module init function does a xen_pv_domain() check first thing.
> >> Hence there's no point building it in non-PV configurations.
> > 
> > s/It's/<name of function that calls xen_pv_domain()/   # pcifront_init()?
> 
> I don't understand this - how is "module init function" not clear
> enough?

Saying "module init function" makes the reader do extra work to
figure out what function you are referring to.  I had to look
at drivers/pci/Makefile to find the module name, then look at
drivers/pci/xen-pcifront.c to look for the init function.
Saying pcifront_init() makes it trivial to look *there*.

> > s/building it/building <name of module>/               # xen-pcifront.o?
> 
> The driver name is already part of the subject; I didn't think I
> need to repeat that one here.

Why be vague when it's so easy to be explicit and save everybody else
the effort?  Part of the disconnect here is that the subject line is
not *part* of the commit log, so the commit log should make sense even
if you can't see the subject line.  It's like an essay that should
make sense without its title.

Most of this *is* trivial, I agree.  Just minor hiccups in the process
of reading.

> > I see that CONFIG_XEN_PV is only mentioned in arch/x86, so
> > CONFIG_XEN_PV=y cannot be set on other arches.  Is the current
> > "depends on X86" just a reflection of that, or is it because of some
> > other x86 dependency in the code?
> > 
> > The connection between xen_pv_domain() and CONFIG_XEN_PV is not
> > completely obvious.
> > 
> > If you only build xen-pcifront.o when CONFIG_XEN_PV=y, and
> > xen_pv_domain() is true if and only if CONFIG_XEN_PV=y, why bother
> > calling xen_pv_domain() at all?
> 
> Because XEN_PV=y only _enables_ the kernel to run in PV mode. It
> may be enabled to also run in HVM and/or PVH modes. And it may
> then _run_ in any of the enabled modes. IOW xen_pv_domain() will
> always return false when !XEN_PV; no other implication is valid.
> I don't think this basic concept needs explaining in a simple
> patch like this. Instead I think the config option in question,
> despite living in drivers/pci/Kconfig, should be under "XEN
> HYPERVISOR INTERFACE" maintainership. I realize that's not even
> expressable in ./MAINTAINERS. I wonder why the option was put
> there in the first place, rather than in drivers/xen/Kconfig.

No doubt it's obvious to Xen experts.  Unfortunately I am not one.

Evidently there's no real dependency on the X86 arch, which makes me
wonder why the "depends on X86" was there in the first place.

Bjorn


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 16:44:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 16:44:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181236.328302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNeD4-00086x-Sa; Tue, 07 Sep 2021 16:44:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181236.328302; Tue, 07 Sep 2021 16:44:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNeD4-00086q-P6; Tue, 07 Sep 2021 16:44:34 +0000
Received: by outflank-mailman (input) for mailman id 181236;
 Tue, 07 Sep 2021 16:44:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNeD3-00086g-43; Tue, 07 Sep 2021 16:44:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNeD2-0000S3-OH; Tue, 07 Sep 2021 16:44:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNeD2-0005jL-9O; Tue, 07 Sep 2021 16:44:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNeD2-0005qX-8u; Tue, 07 Sep 2021 16:44:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XdHOkZ0Sl7HcdmNgdy0n0SQacTbETZEE7X+q0N0/gyA=; b=pYR6n7VweqU2/OR7fd3AgtDvxz
	kfdxUMp4fz9uMzImma+oNI5yRiE4yh//ogxvC9buGQgfAhqU+eQNu0IQxMjwrRek6rlhFZV5gxdxa
	tPU6sFwoanJVLGGIVQ5bepsWbdHxVQSx7zCtQIZ7JBbwowRyz+3hKLV1n/OoL6XvOtmk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164871-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164871: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b93c544e90e2b28326182d31ee008eb80e02074
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 16:44:32 +0000

flight 164871 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164871/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                4b93c544e90e2b28326182d31ee008eb80e02074
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  402 days
Failing since        152366  2020-08-01 20:49:34 Z  401 days  699 attempts
Testing same since   164865  2021-09-06 21:40:45 Z    0 days    2 attempts

------------------------------------------------------------
7221 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2221825 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:10:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:10:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181246.328316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebb-0002iA-Uk; Tue, 07 Sep 2021 17:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181246.328316; Tue, 07 Sep 2021 17:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebb-0002i3-RN; Tue, 07 Sep 2021 17:09:55 +0000
Received: by outflank-mailman (input) for mailman id 181246;
 Tue, 07 Sep 2021 17:09:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNebb-0002hx-9x
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:09:55 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c53e1a52-308e-47af-b219-550dd6532ed3;
 Tue, 07 Sep 2021 17:09:54 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id m28so20870217lfj.6
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 10:09:53 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id t20sm1059533lfk.157.2021.09.07.10.09.51
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 07 Sep 2021 10:09:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c53e1a52-308e-47af-b219-550dd6532ed3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=Eb3nTuViuxkM2u25OPGYlL8g/e8tL46q5jZZVa4g28I=;
        b=KP6THAg3j2h3laFCwGL5cczIeTLwtChwkQPOmjuYWuwXVJm4iCDbMNWljxQJsS51V9
         U3QfumpMjey7dx9fzCA33igvERXlRiqcS5hQOMLSdRW1inlq9p+2Jdroz4Ysv1y5uw/d
         IgE4zEBsFvGfq4iDslJ/TuWV1T72ct9WtyozmWBWwyxwO7XLhULP92KcQa0ZvFs8/G4y
         R8w3VoT3fmYcQYXhiL4Pab9DpIxC8Ie5CnTuPBVViXdWfklFFZ3JTp9AjifpBphL33YA
         bwG+qQs+po0ALxhyPPzAYPU9apCnC0AI2bBO+oNmNMsyDXu1FG7KbY50ERp5NdOKkYzX
         PuOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=Eb3nTuViuxkM2u25OPGYlL8g/e8tL46q5jZZVa4g28I=;
        b=rCbKOLEQfKWv88OJ5NCLMptP3XtGvZmm+Q4VrWI2eWNsVmX+xE60PFXXaegJkMKV2l
         6fQCDqeAa0esEXcdfihoGAiZU8a/o8wFcJdGvkGWESIeCa+ytHKH9daER6KRnVWwSg/v
         +GS6KfUqaQto6W2XNA7I68iTNamfOuyDgkJ92OI0vgsBFg0mxfbz8HnlBtABBfXkfs1G
         PQz7i2NAKbFJ3f0w7Mstcg+0tLYHIbXtp2B3Sd1ch8tcPZQfBwktHr9XlcYwOtzY5WEP
         oSlzUjqd0Y7Pi+bEDPcu1Xzw3y8h535vY91RehsUgcvUcZFoqRcyv9guKlBuig+GxBkS
         ik2Q==
X-Gm-Message-State: AOAM531X23uxe2uClcDe7vWmU0+xElO2HGQe7g6wYy173CivB8EilEPz
	71SwsOQPOPCiEfrja29rhXjQRBL7Ewg=
X-Google-Smtp-Source: ABdhPJzrrQuBHf/px86brxNUtIXtsL3f4TvChuTwFU0eFe8bl8AjmS+1W0LXx6/jcCJ7EOw5P8BGsA==
X-Received: by 2002:ac2:551a:: with SMTP id j26mr13864021lfk.354.1631034592707;
        Tue, 07 Sep 2021 10:09:52 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [RFC PATCH 0/3] Add handling of extended regions (safe ranges) on Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
Date: Tue,  7 Sep 2021 20:09:35 +0300
Message-Id: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

You can find an initial discussion at [1].

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree.

The extended regions are calculated differently for direct mapped
Dom0 (with and without IOMMU) and non-direct mapped DomUs.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

Also please note that we haven't figured out yet how to properly 
extend the Xen hypervisor device-tree bindings on Arm (either via new
compatible or via new property). I decided to go with new property
for now, but this can be changed. This uncertainty is the main reason
why this series is marked as RFC.

Patch series is also available at [2].

The corresponding Linux changes is not in a good shape now (require
some cleanup and refactoring), I will publish them once put them
in order (I hope, it will be in a few days).

[1] https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
[2] https://github.com/otyshchenko1/xen/commits/map_opt_ml2

Oleksandr Tyshchenko (3):
  xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
  xen/arm: Add handling of extended regions for Dom0
  toolstack/arm: Add handling of extended regions for DomU

 tools/include/libxl.h            |   7 ++
 tools/libs/light/libxl.c         |   2 +
 tools/libs/light/libxl_arm.c     |  92 +++++++++++++++-
 tools/libs/light/libxl_types.idl |   2 +
 xen/arch/arm/domain_build.c      | 233 ++++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/sysctl.c            |   2 +
 xen/arch/x86/sysctl.c            |   2 +
 xen/include/public/sysctl.h      |   3 +-
 8 files changed, 338 insertions(+), 5 deletions(-)

-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:10:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:10:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181247.328327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebh-00030A-Ay; Tue, 07 Sep 2021 17:10:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181247.328327; Tue, 07 Sep 2021 17:10:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebh-000303-7C; Tue, 07 Sep 2021 17:10:01 +0000
Received: by outflank-mailman (input) for mailman id 181247;
 Tue, 07 Sep 2021 17:10:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNebg-0002hx-3O
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:10:00 +0000
Received: from mail-lj1-x236.google.com (unknown [2a00:1450:4864:20::236])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 28928d9d-15b1-49c5-85cf-f083d39f0e82;
 Tue, 07 Sep 2021 17:09:55 +0000 (UTC)
Received: by mail-lj1-x236.google.com with SMTP id s3so17675394ljp.11
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 10:09:54 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id t20sm1059533lfk.157.2021.09.07.10.09.52
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 07 Sep 2021 10:09:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28928d9d-15b1-49c5-85cf-f083d39f0e82
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=r8OO/kJJTJV6IKHPTsMlElMm0wsP5l7Pgo7wl/ZjD8Y=;
        b=ZXLY3dMPIAwjgvH2Nnqi9I0Lo/ZLY6Y58wMChCZhiAaXZnfyvFIw3Zp5oguMdqXR5D
         RhQLqzfXzZY1vYvMF0pW/JIWOfp0OYQOsERKgVZklfd3uG+wX4bz7zi2Sdt+Y8Yx6dAV
         xAkZDRo1lpVEsdFrUnsb4JeQbNOh3ZtltvmfRHxWfEVR3T1Gcz6Kt7FtOWKeq0z3T8gr
         wd5vtKsCokgIvlg962mBNgh4lNnbCnOM3c0Xoww1pCfvOEuZELf7U4JUsLWZcyxd2/yt
         6EEUpcJ+D1W892JPHWkj/hvIF5eO2DpNx11fMEttxneRNOsS/848n41kgnIJjy8Mhks2
         pbGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=r8OO/kJJTJV6IKHPTsMlElMm0wsP5l7Pgo7wl/ZjD8Y=;
        b=YDgO4Jfbq++Y6z8hs61UA6PJ+5WsqFbiv4qQSNxeiMHuu4N9jA3LKW4ozqxscVRLjt
         l/deBhZ8gZQPdCN9jkV1PWeCAXI0BAIIaDWLWBg7nLCNHTGvL9eg5IXzFw6RF/QZqp1q
         QkFDfwioL+kGQjLDZZDtTzyBpHk9eIXKWBqGr7ro0PEMz1tKljfv++HJxNRhOBH6VoDy
         n4McCzT9BxYhnYa2pxUVbvqm6farpZICMxLCk2HFXV1Db/tOHpqLfMhkUT03KrEtZsgc
         8VG5t5qhp3JKxEwFy0/I686eGrJJhU+sWJz63mkYzt67abUgWwkylQ6TjETugBU1pkKh
         xGyA==
X-Gm-Message-State: AOAM532e2eUfa8F+TiulsfNUznec7D/pWJGFh/VT16//PuQT9mLLhGqh
	NFCuH3HkQvE1+Op6BTimVw0vemd8vZ8=
X-Google-Smtp-Source: ABdhPJxC8qf79t1ZIF2AN8Zknfh5cN0xbkLmLI52Ll/jKPS4wGQHEL3CESZZSeVt5iwva25PNfeEqQ==
X-Received: by 2002:a05:651c:893:: with SMTP id d19mr8632562ljq.268.1631034593860;
        Tue, 07 Sep 2021 10:09:53 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>,
	Henry Wang <Henry.Wang@arm.com>
Subject: [RFC PATCH 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
Date: Tue,  7 Sep 2021 20:09:36 +0300
Message-Id: <1631034578-12598-2-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

We need to pass info about maximum supported guest address
space size to the toolstack on Arm in order to properly
calculate the base and size of the extended region (safe range)
for the guest. The extended region is unused address space which
could be safely used by domain for foreign/grant mappings on Arm.
The extended region itself will be handled by the subsequents
patch.

Use p2m_ipa_bits variable on Arm, the x86 equivalent is
hap_paddr_bits.

As we change the size of structure bump the interface version.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Changes since RFC:
   - update patch subject/description
   - replace arch-specific sub-struct with common gpaddr_bits
     field and update code to reflect that
---
 tools/include/libxl.h            | 7 +++++++
 tools/libs/light/libxl.c         | 2 ++
 tools/libs/light/libxl_types.idl | 2 ++
 xen/arch/arm/sysctl.c            | 2 ++
 xen/arch/x86/sysctl.c            | 2 ++
 xen/include/public/sysctl.h      | 3 ++-
 6 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d..da44944 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -855,6 +855,13 @@ typedef struct libxl__ctx libxl_ctx;
  */
 #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
 
+ /*
+  * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
+  *
+  * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
+  */
+ #define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
+
 /*
  * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1
  *
diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c
index 204eb0b..c86624f 100644
--- a/tools/libs/light/libxl.c
+++ b/tools/libs/light/libxl.c
@@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
     physinfo->cap_vmtrace =
         !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace);
 
+    physinfo->gpaddr_bits = xcphysinfo.gpaddr_bits;
+
     GC_FREE;
     return 0;
 }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff6..f7437e4 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -1061,6 +1061,8 @@ libxl_physinfo = Struct("physinfo", [
     ("cap_shadow", bool),
     ("cap_iommu_hap_pt_share", bool),
     ("cap_vmtrace", bool),
+
+    ("gpaddr_bits", uint32),
     ], dir=DIR_OUT)
 
 libxl_connectorinfo = Struct("connectorinfo", [
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index f87944e..91dca4f 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -15,6 +15,8 @@
 void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
 {
     pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
+
+    pi->gpaddr_bits = p2m_ipa_bits;
 }
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a1..7b14865 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
     if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow;
+
+    pi->gpaddr_bits = hap_paddr_bits;
 }
 
 long arch_do_sysctl(
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 039ccf8..f53b42e 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -35,7 +35,7 @@
 #include "domctl.h"
 #include "physdev.h"
 
-#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014
 
 /*
  * Read console content from Xen buffer ring.
@@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
     uint64_aligned_t outstanding_pages;
     uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
     uint32_t hw_cap[8];
+    uint32_t gpaddr_bits;
 };
 
 /*
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:10:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:10:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181248.328338 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebm-00040L-K7; Tue, 07 Sep 2021 17:10:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181248.328338; Tue, 07 Sep 2021 17:10:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebm-0003za-Go; Tue, 07 Sep 2021 17:10:06 +0000
Received: by outflank-mailman (input) for mailman id 181248;
 Tue, 07 Sep 2021 17:10:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNebl-0002hx-3R
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:10:05 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3feea662-c75e-43f0-859a-9afa379c3c52;
 Tue, 07 Sep 2021 17:09:55 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id h16so20825989lfk.10
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 10:09:55 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id t20sm1059533lfk.157.2021.09.07.10.09.53
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 07 Sep 2021 10:09:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3feea662-c75e-43f0-859a-9afa379c3c52
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=j1FVXxbL655dDi6m/uiQMBsY6jfRKFwUr0FLdTNzkcs=;
        b=qLChzUpurCwBuStVjudwSNRgfGuirpgjxZ5kOD55jqGSXT/hsJpD/Tqc5KLcFLygGu
         Sjy0+ampswXA6LxrAwchG46XYYubP4eMeG1o8LCh/pzuMPS+Cr3RfU7mLoaph8d8i7Df
         RNumeLv0rMY8nLH1yFgQZ9s/mlMY6va1551KPvWZQTz5QIK3AqqbKErijiuIYd7AY2Wa
         xOP1+r8fMKH7SOoGeakQm5KHUZ3qt4yjWhQ7n9Of31VrYCiJ6YZgoYFqkZIQLrmGlPuL
         2fO2ro64uvg9KpCW60+gEn4cjbWoMwGVR+s18zqVdbc9WwhBEbcUc/b5RR9lSf6CddES
         WuQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=j1FVXxbL655dDi6m/uiQMBsY6jfRKFwUr0FLdTNzkcs=;
        b=HRszFJP8OX/Qcuete0kbPRVog7wOg2gqtHe+omL+NKWDyCagUQQfir1pkJJ5JN4mzs
         ghl8mFMiiW3c5GJW24N6epx5Xai/YIHljTEbF1Gx7cVpncalQegzmAlah1DT7xei8DQm
         TCLla8NmmGybDj4g2aaSpiF/cTw1ZkzChAaj4TIZ9xE3C2+7K+atpN0zAxxrW1kjKthL
         /+afbm44R/Cv4SL1tq2jyLQF08Rx2+/OMD/HhxNnbAby5p5cCbBISgYj70+z5rIKED7I
         bRDT3M2QQqkLKDK3SkyCp8Cpw1jDc8a8BbZDH8QQ3vz6usNs+KwePPlL4ZEN/XGKR+9U
         vjCw==
X-Gm-Message-State: AOAM532+hDOjgrUTTWoBzRTQqBjEC19d3j/X2Dbebx7ZSKIPYARRXb7t
	hlNlBpfmznH+GxsdVy62ff2s8W23rfg=
X-Google-Smtp-Source: ABdhPJwL3Ib/sovQSSWBncUPiMNcQtMgEHP9AlJf4cx4aSB0kd4rFfaZP6wjgO/HtJrBTyIh1OT18A==
X-Received: by 2002:a19:c394:: with SMTP id t142mr13504035lff.313.1631034594744;
        Tue, 07 Sep 2021 10:09:54 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>,
	Henry Wang <Henry.Wang@arm.com>
Subject: [RFC PATCH 2/3] xen/arm: Add handling of extended regions for Dom0
Date: Tue,  7 Sep 2021 20:09:37 +0300
Message-Id: <1631034578-12598-3-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space, introduce new "extended-region" property to inform
domain about the presence of additional regions (1...N).
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

***

As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
the algorithm to choose extended regions for it is different
in comparison with the algorithm for non-direct mapped DomU.
What is more, that extended regions should be chosen differently
whether IOMMU is enabled or not.

Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
holes found in host device-tree if otherwise. Make sure that
extended regions are 2MB-aligned and located within maximum possible
addressable physical memory range. The maximum number of extended
regions is 128.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
 xen/arch/arm/domain_build.c | 233 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 231 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d..f9453de 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domain *d,
     return res;
 }
 
+static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
+{
+    struct meminfo *ext_regions = data;
+    paddr_t start, size;
+
+    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
+        return 0;
+
+    /* Both start and size of the extended region should be 2MB aligned */
+    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
+    if ( start > e )
+        return 0;
+
+    size = (e - start + 1) & ~(SZ_2M - 1);
+    if ( !size )
+        return 0;
+
+    ext_regions->bank[ext_regions->nr_banks].start = start;
+    ext_regions->bank[ext_regions->nr_banks].size = size;
+    ext_regions->nr_banks ++;
+
+    return 0;
+}
+
+/*
+ * The extended regions will be prevalidated by the memory hotplug path
+ * in Linux which requires for any added address range to be within maximum
+ * possible addressable physical memory range for which the linear mapping
+ * could be created.
+ * For 48-bit VA space size the maximum addressable range are:
+ * 0x40000000 - 0x80003fffffff
+ */
+#define EXT_REGION_START   0x40000000ULL
+#define EXT_REGION_END     0x80003fffffffULL
+
+static int __init find_unallocated_memory(const struct kernel_info *kinfo,
+                                          struct meminfo *ext_regions)
+{
+    const struct meminfo *assign_mem = &kinfo->mem;
+    struct rangeset *unalloc_mem;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find unallocated memory for extended regions\n");
+
+    unalloc_mem = rangeset_new(NULL, NULL, 0);
+    if ( !unalloc_mem )
+        return -ENOMEM;
+
+    /* Start with all available RAM */
+    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
+    {
+        start = bootinfo.mem.bank[i].start;
+        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
+        res = rangeset_add_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove RAM assigned to Dom0 */
+    for ( i = 0; i < assign_mem->nr_banks; i++ )
+    {
+        start = assign_mem->bank[i].start;
+        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
+        res = rangeset_remove_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove reserved-memory regions */
+    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
+    {
+        start = bootinfo.reserved_mem.bank[i].start;
+        end = bootinfo.reserved_mem.bank[i].start +
+            bootinfo.reserved_mem.bank[i].size - 1;
+        res = rangeset_remove_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove grant table region */
+    start = kinfo->gnttab_start;
+    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
+    res = rangeset_remove_range(unalloc_mem, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_report_ranges(unalloc_mem, start, end,
+                                 add_ext_regions, ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(unalloc_mem);
+
+    return res;
+}
+
+static int __init find_memory_holes(const struct kernel_info *kinfo,
+                                    struct meminfo *ext_regions)
+{
+    struct dt_device_node *np;
+    struct rangeset *mem_holes;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find memory holes for extended regions\n");
+
+    mem_holes = rangeset_new(NULL, NULL, 0);
+    if ( !mem_holes )
+        return -ENOMEM;
+
+    /* Start with maximum possible addressable physical memory range */
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_add_range(mem_holes, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
+    dt_for_each_device_node( dt_host, np )
+    {
+        unsigned int naddr;
+        u64 addr, size;
+
+        naddr = dt_number_of_address(np);
+
+        for ( i = 0; i < naddr; i++ )
+        {
+            res = dt_device_get_address(np, i, &addr, &size);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                       i, dt_node_full_name(np));
+                goto out;
+            }
+
+            start = addr & PAGE_MASK;
+            end = PAGE_ALIGN(addr + size) - 1;
+            res = rangeset_remove_range(mem_holes, start, end);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                       start, end);
+                goto out;
+            }
+        }
+    }
+
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_report_ranges(mem_holes, start, end,
+                                 add_ext_regions,  ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(mem_holes);
+
+    return res;
+}
+
 static int __init make_hypervisor_node(struct domain *d,
                                        const struct kernel_info *kinfo,
                                        int addrcells, int sizecells)
@@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain *d,
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
         "xen,xen";
-    __be32 reg[4];
+    __be32 reg[(NR_MEM_BANKS + 1) * 4];
     gic_interrupt_t intr;
     __be32 *cells;
     int res;
     void *fdt = kinfo->fdt;
+    struct meminfo *ext_regions;
+    unsigned int i;
 
     dt_dprintk("Create hypervisor node\n");
 
@@ -757,15 +949,52 @@ static int __init make_hypervisor_node(struct domain *d,
     if ( res )
         return res;
 
+    ext_regions = xzalloc(struct meminfo);
+    if ( !ext_regions )
+        return -ENOMEM;
+
+    if ( is_32bit_domain(d) )
+        printk(XENLOG_WARNING "The extended region is only supported for 64-bit guest\n");
+    else
+    {
+        if ( !is_iommu_enabled(d) )
+            res = find_unallocated_memory(kinfo, ext_regions);
+        else
+            res = find_memory_holes(kinfo, ext_regions);
+
+        if ( res )
+            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
+    }
+
     /* reg 0 is grant table space */
     cells = &reg[0];
     dt_child_set_range(&cells, addrcells, sizecells,
                        kinfo->gnttab_start, kinfo->gnttab_size);
+    /* reg 1...N are extended regions */
+    for ( i = 0; i < ext_regions->nr_banks; i++ )
+    {
+        u64 start = ext_regions->bank[i].start;
+        u64 size = ext_regions->bank[i].size;
+
+        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
+                   i, start, start + size);
+
+        dt_child_set_range(&cells, addrcells, sizecells, start, size);
+    }
+    xfree(ext_regions);
+
     res = fdt_property(fdt, "reg", reg,
-                       dt_cells_to_size(addrcells + sizecells));
+                       dt_cells_to_size(addrcells + sizecells) * (i + 1));
     if ( res )
         return res;
 
+    if ( i > 0 )
+    {
+        res = fdt_property(fdt, "extended-region", NULL, 0);
+        if ( res )
+            return res;
+    }
+
     BUG_ON(d->arch.evtchn_irq == 0);
 
     /*
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:10:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:10:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181249.328349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebr-0004Rb-V0; Tue, 07 Sep 2021 17:10:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181249.328349; Tue, 07 Sep 2021 17:10:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNebr-0004R7-Qm; Tue, 07 Sep 2021 17:10:11 +0000
Received: by outflank-mailman (input) for mailman id 181249;
 Tue, 07 Sep 2021 17:10:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNebq-0002hx-3l
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:10:10 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bb19d5b4-180d-4792-9daf-20ca2f636c64;
 Tue, 07 Sep 2021 17:09:57 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id z2so20891905lft.1
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 10:09:56 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id t20sm1059533lfk.157.2021.09.07.10.09.54
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 07 Sep 2021 10:09:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb19d5b4-180d-4792-9daf-20ca2f636c64
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=Ya0iv+xjtgk+KbWZdbpF+3/p23xC0vgD9QxwNHC4wzo=;
        b=ZYoki/JEJRYk82IEIOvaWK4Wj0RDoDRnDs+qMeGITnAE5fQOGPxG4YvHb7M1c0I7OH
         oPVly8GrZynKZArYCRdjo8IPiBJjR+CLFeOD09+d2NUboOq/CfC1wuJt9ppejFLN8UWm
         bsdZAwRr6DgLcNnOTa7ZrKfqpKfUQgu0FKuCnJg0JWXWVH6lwEgOTxzDIx77Zg9mOikR
         K6qGqqnWAHGg78fzStMheeIFFiOk2ckTMzbPXFCDWEnvRC7/sgZM13IWqiro3WFoRAM/
         kOLFWUEmQBxWND74h9uOKAMYYQC9WjSbbIzOWVYyXybx9w4HWmcRGxCLwwPt0Q8rpEzF
         qaCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=Ya0iv+xjtgk+KbWZdbpF+3/p23xC0vgD9QxwNHC4wzo=;
        b=oRr+OmD5AGR8cJCunGAF3kO2Pjha8sWEd6+NODrX4aEEG1GfhkeiiPoFBZzuP7FN/U
         RrmsIL0xtf3Dxsf91JalZAiLrRzwO3Wfg2k5wEc86TVEKnGHP8QvqcmzV5CTyJS7HH+5
         QQVVLM6ip00j0g15voXyqNNmVFtMKPJgh/I5uV/j/srCY8GUNHJoVfPA/53CFxIYmOuI
         TfWU5BeVIKhLzxbmxzxkPNIEDl4AJOYluI3bS86NP4612Sx6aKaxc2d4h9AfLb0x01BO
         QY8sYjhX8ENPddfF/pRHSOYCF7ru3VzurjwwctJT84ZW1TPI9LgG1E5o7DtOC18oWvuS
         T72Q==
X-Gm-Message-State: AOAM530a5YI0YF41g+mrgWtUqw70YIX2TZRmOBAKOTZZidHPdsfkVtNq
	T1aPz82WvCNhKgiUjCu7rN1V+oIPPT8=
X-Google-Smtp-Source: ABdhPJxT0Q6VmD3LuyWEIxjpy/iZPkXyU7wuzlg78/XTigbVbBnOWhav/35Lhnf8aIy10QA2uNHfMg==
X-Received: by 2002:a05:6512:32c6:: with SMTP id f6mr13316789lfg.292.1631034595700;
        Tue, 07 Sep 2021 10:09:55 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>,
	Henry Wang <Henry.Wang@arm.com>
Subject: [RFC PATCH 3/3] toolstack/arm: Add handling of extended regions for DomU
Date: Tue,  7 Sep 2021 20:09:38 +0300
Message-Id: <1631034578-12598-4-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space, introduce new "extended-region" property to inform
domain about the presence of additional regions (1...N).
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

***

The algorithm to choose extended regions for non-direct mapped
DomU is simpler in comparison with the algorithm for direct mapped
Dom0. As we have a lot of unused space above 4GB, provide single
1GB-aligned region from the second RAM bank taking into the account
the maximum supported guest address space size and the amount of
memory assigned to the guest. The maximum size of the region is 128GB.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
 tools/libs/light/libxl_arm.c | 92 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 90 insertions(+), 2 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6..41cf017 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -615,9 +615,23 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
                               "xen,xen");
     if (res) return res;
 
-    /* reg 0 is grant table space */
+    /*
+     * reg 0 is a placeholder for grant table space, reg 1 is a placeholder
+     * for the extended region.
+     */
     res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+                            2, 0, 0, 0, 0);
+    if (res) return res;
+
+    /*
+     * TODO: It would be correct to advertise that we provide the extended
+     * region after allocating it in finalise_ext_region(). The reason why we do
+     * this right now (in advance) is because we cannot insert a properly in
+     * finalise_ext_region(), but we have a mechanism to remove it there.
+     * So, if we are not able to allocate the region later on, we will remove
+     * the property.
+     */
+    res = fdt_property(fdt, "extended-region", NULL, 0);
     if (res) return res;
 
     /*
@@ -1069,6 +1083,78 @@ static void finalise_one_node(libxl__gc *gc, void *fdt, const char *uname,
     }
 }
 
+#define ALIGN_UP_TO_GB(x)   (((x) + GB(1) - 1) & (~(GB(1) - 1)))
+
+#define EXT_REGION_SIZE   GB(128)
+
+static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image *dom)
+{
+    void *fdt = dom->devicetree_blob;
+    uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
+    be32 *cells = &regs[0];
+    uint64_t region_size = 0, region_base, bank1end_align, bank1end_max;
+    uint32_t gpaddr_bits;
+    libxl_physinfo info;
+    int offset, rc;
+
+    offset = fdt_path_offset(fdt, "/hypervisor");
+    assert(offset > 0);
+
+    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
+        LOG(WARN, "The extended region is only supported for 64-bit guest");
+        goto out;
+    }
+
+    rc = libxl_get_physinfo(CTX, &info);
+    assert(!rc);
+
+    gpaddr_bits = info.gpaddr_bits;
+    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
+
+    /*
+     * Try to allocate single 1GB-aligned extended region from the second RAM
+     * bank (above 4GB) taking into the account the maximum supported guest
+     * address space size and the amount of memory assigned to the guest.
+     * The maximum size of the region is 128GB.
+     */
+    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
+    bank1end_align = GUEST_RAM1_BASE +
+        ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << XC_PAGE_SHIFT);
+
+    if (bank1end_max <= bank1end_align) {
+        LOG(WARN, "The extended region cannot be allocated, not enough space");
+        goto out;
+    }
+
+    if (bank1end_max - bank1end_align > EXT_REGION_SIZE) {
+        region_base = bank1end_max - EXT_REGION_SIZE;
+        region_size = EXT_REGION_SIZE;
+    } else {
+        region_base = bank1end_align;
+        region_size = bank1end_max - bank1end_align;
+    }
+
+out:
+    /*
+     * The first region for grant table space must be always present.
+     * If we managed to allocate the extended region then insert it as
+     * a second region. Otherwise, remove the property that advertises it.
+     */
+    set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+              GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+    if (region_size > 0) {
+        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
+            region_base, region_base + region_size);
+
+        set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                  region_base, region_size);
+    } else
+        fdt_nop_property(fdt, offset, "extended-region");
+
+    rc = fdt_setprop_inplace(fdt, offset, "reg", regs, sizeof(regs));
+    assert(!rc);
+}
+
 int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
                                                uint32_t domid,
                                                libxl_domain_config *d_config,
@@ -1109,6 +1195,8 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
 
     }
 
+    finalise_ext_region(gc, dom);
+
     for (i = 0; i < GUEST_RAM_BANKS; i++) {
         const uint64_t size = (uint64_t)dom->rambank_size[i] << XC_PAGE_SHIFT;
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:13:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:13:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181273.328360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNefL-0006MN-N2; Tue, 07 Sep 2021 17:13:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181273.328360; Tue, 07 Sep 2021 17:13:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNefL-0006MG-Jw; Tue, 07 Sep 2021 17:13:47 +0000
Received: by outflank-mailman (input) for mailman id 181273;
 Tue, 07 Sep 2021 17:13:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNefK-0006MA-4J
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:13:46 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b996f2e-ce1e-40d3-8f9c-8ad88b944c99;
 Tue, 07 Sep 2021 17:13:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b996f2e-ce1e-40d3-8f9c-8ad88b944c99
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631034824;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=GJ3bNmTS1SCgQmc3iJNrcycJ8ksUb33jC2GsqzoyOMY=;
  b=TmkorK7brbNkvBlntW1C2GUuCVh/IyIe9ctxluPM0qFpKD8cpMSsmWXh
   qhqhghQHelnW2HXPm5GOHs6RplKStIYgda2C1Es0Llf0EIGpUNYOZTsv5
   CiAi9fnXu+8wB/UAm31ZJvcl/YaCy5PxHmGR7LnW/1zLGzRYdfaaLs2w7
   c=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: xthbV/wYKObH9XYXDBoscF+DLVj3fg2Le+AN78iur3lWnjc+uWj2fX8qjGAKO/1qWBW7TGjJc0
 fetO4bcq12gxg+rBPwLHlIxiyowWVrPBO3IodEBFFjSVgxMrZfAuY1r0uA0+OvOu0MWORoRrCZ
 WjKMsuceHcFgC9tSTgsg8t7jVf5S7uM9a1WgINK7kRDfO3OxwmAd4euiC3JAByTLH4YzLRSaV3
 b/pdb17PzBAgATKHZwXbwr+If3sO9Q2dRLgS0Gzx8bX4tMIF+XGl6ubL71xLkW33JdJoI+HKrf
 3iCfjTT4JCWedPBo7mKaz15Z
X-SBRS: 5.1
X-MesageID: 51782389
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:QrjXcqAsKD89qOTlHemQ55DYdb4zR+YMi2TDsHoBLSC9E/bo8v
 xG885rtiMc5AxxZJhCo7690cu7MBThHPdOiOF6UItKNDOW3ldAR7sSj7cKrQeBJ8TWzJ8l6U
 8+GJIUNDSLNzdHZGzBkXGF+q0brOW6zA==
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="51782389"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cAZ7dx2t6MPKD/Cwl7e2n6jSgZObv3CFTvGCVph/DjqGibmqqEAjl7VakRqZSyslN0Xk/ZuBKjCpnI4j8RGoCUogvHK7X51uYn6dbKHdgWspvhYJBMed/rT4YuYtVrvecWgg0i5+iZb5EPIi7NpSl2R6P/wYce+s38hF0x14gKn8UiTUYbRacAHPQhj6ZSfT3m3HVEwohInn2bP8UjjIV6wKCybqyLlmNKvRQ/A5bBaSbktSdOjxfiekVMY/o4IoYdqo0f8gSj4z8Ld7K2eNuKUxgh3S88VD/OEoAVyv5GpNfAy+3whGI77zTea42niBygFiNKB6HGfZH9JwGq+T9w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GJ3bNmTS1SCgQmc3iJNrcycJ8ksUb33jC2GsqzoyOMY=;
 b=EA6KHDfd9OxBSSOH3w4COol4f9i/QwkChFKl87s4Cq0nTYlUkC8nA8t7CzUKUWVaYqdtBE0CloExRRxriXkw65PITgo/ULbWSJRZmN4tEyzo3lKKsU4hiYxbR4N3gDfAX0GbVQT6YYjYYJt0LPKbs0jRbFCiOqOj39XGyQm9E6gvprMnpqc+HYBRyZeq9O6BnhBmfn/fp+CYPJ9HSUeWM4XNsDPrhZ+AYDiCx97PRTeFktub+LAYlo9PeVACV+irB+0BMatFr7om8UHck60Xs7ZMpdDsTigV6qE0cLuqrdxUSSAYLhyJDdb881bqAw1KDX/jwv57FnpZzCm5faGCYA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GJ3bNmTS1SCgQmc3iJNrcycJ8ksUb33jC2GsqzoyOMY=;
 b=DolPnkG8UAcZGy6OjHGOedRs3Z2a/fEqQMQu4MFO9bcRlAdo+qxqc7VCcAtjVEaqBhQhBwMjS+z5w2KEKHSIsWxyAM/ky4qrOYv92sK8zTvpyvtY+7zK+SgkMHZTwiLG+usE7qsAG6Grv4lQbzrnOmUu8G1QEAPnAa40vsF4HWc=
To: Jan Beulich <jbeulich@suse.com>
CC: Paul Durrant <paul@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <5d6e8360-7ba8-487e-0ac8-7f517e560520@suse.com>
 <42aa8c24-7f23-deec-6103-f2d556e4a9f9@citrix.com>
 <b3db56ae-7a53-ee22-d821-d711b0d0bf3b@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 07/17] IOMMU/x86: restrict IO-APIC mappings for PV Dom0
Message-ID: <3f9722e4-704b-8f68-553d-04a3a3189d99@citrix.com>
Date: Tue, 7 Sep 2021 18:13:36 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b3db56ae-7a53-ee22-d821-d711b0d0bf3b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0069.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:60::33) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 455a7645-da4e-4d03-7d72-08d97222d774
X-MS-TrafficTypeDiagnostic: BY5PR03MB4965:
X-Microsoft-Antispam-PRVS: <BY5PR03MB4965DBB5D3A483048B0ACD7DBAD39@BY5PR03MB4965.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: mAhLbOA+3AX5whahOBH0gJdWUAh8V+z+SVBJkvaU206exP1awKVvRVkC/D0Q/PD+/G3G/cnP5WJPIYOqROun7nTvwwKf0CQLXHJOhJIDJ679QBdTlbpYX65FJIXsa9WXvEn9aESK8NWzK1jv/YbJopYqrBiCtn3LmXWJ58fk/veZgtsvWx4OjQbPRx+th9UcQ+9Jc8AVKS1J7hAEN26ThivD679kXZLi03JQJ+KIIO20EWOZidTc3BWb8l3546cfGLH9SwrDJmndJIH6nzhQZcgbZO9gPUS24GXY+rlKjDUuOaRH2jha+JyCIEWegGkjI4WRXCNv/yqKi4rHH6+Rkdxb9ua+sjZzT8i9tgAOKwtv3Pvl01OzG74omgFR1xSn6x3zGPa6RwHV9OZRMBIEWHHpOc369f94gioZ6eo+82zI8SY+BxLxChAdj7SiezfVGPOBnlyrHDFZryZTRQmXWZ6rlDc4HegQK7LRoPoaWD0X8q41fl/txzGqTWW4/dPCKyWoHJlnkigVvWkSvOkM0e1qTUdYD37CxT9hY3lOsYNB1gMLi2e8k1iYGJ2ksfuqUfXpOWmPB9uT8OjjbV2r4Lyd6KbF+Zq0NUW9Qlr/+2wgE/a2BUuoVoxwqHWpA1nfBvLGE4Py2QWIeiyJwgq6xzIMu1Om+8uyVD1AGzXpODoVZprEHyeTFGIkEViBc30qDuBTCtUu8iS9MLucFzVcu8Xef8UzZH/Y2eX23ZzzP0U=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(366004)(39860400002)(136003)(396003)(186003)(53546011)(86362001)(478600001)(36756003)(8936002)(316002)(5660300002)(6486002)(38100700002)(31686004)(66946007)(6916009)(55236004)(66476007)(4326008)(2906002)(16576012)(31696002)(83380400001)(26005)(6666004)(54906003)(2616005)(8676002)(956004)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bEpnWnVmSGl4cG9QdU1aT0F4K0J1bXVPUyt6Zk9FR1gwd0dXUm9DMTF2dXow?=
 =?utf-8?B?RytYbTJDWjFURnZQVnVFdWQ4YU83eXpUNHQrdm5Ib2NISEQyS3ZyeVc4VVFV?=
 =?utf-8?B?Z2tYdWltSDh4Q2k1cEQ1Y0pabFBnWFlEbmxlUC9CMkxvQ1BRVlB2WE5KNTFo?=
 =?utf-8?B?SG53dmVkZGlVU1J5bllibW83V24yM2V5d1BHTGhrY2xIQnNLRkhZYklqKzVm?=
 =?utf-8?B?UW8rN0U0K0NuZ2orK2RWb0hYWHdmSHkrRWRJZlhOVVNTWEtLZkxHcWxwaXRP?=
 =?utf-8?B?VCtwdkpSV2dYQml5d0kwRll4Q082RG4yZmlQV1ZMalZxTlRNSG1oNHNWSVRQ?=
 =?utf-8?B?TEthQjFicWZ4RlBKa0RrMGZMeXVjN2tsV25CNTV2TDFyalhCVm5zYVVtRkJD?=
 =?utf-8?B?LzNNTE44RHhiZmt6T1ZXbU9yZTBUSEU2L2FIZWVHeUs0QkVPcE5BUEorSTEv?=
 =?utf-8?B?YzgvNi9WOGlyeWhTUDVEa0I5RGRScWN2QVlHaGJKY3A2UTBjVDMydDZHSGVW?=
 =?utf-8?B?dWM2aWlxMkRXZjFRTDNPcnV2Y1hzbGJjRUxHd0VYbVRiTENzSldOS0RUM3FY?=
 =?utf-8?B?STZEVUYrM1lvMnd3OUp0eExsU0MreDVPenZaUGllWXNDaHdGMThEMGl4cklk?=
 =?utf-8?B?TDJ3b21EdjkvMmMxZTNoZ1NmQkFET1plRWRteXNYdG5jcmg4Uk9GTklpU1Mz?=
 =?utf-8?B?YjZwNjdKcWNURkJTcHhSOXIxaGlIUFZXOGh4SFNURlpzNTc5NXduSVVrK3Qz?=
 =?utf-8?B?ZXFwQ1RWVVRvRjRNcit6YnZvTzZvL0lyOUEvaTdpSWZqbXR0bTBSOGFEdEtG?=
 =?utf-8?B?bVpNOEx4L1VEcXExclhiQVg0MGFzeVF6OFZMeC81bHhjMVpOWUFiYXlkWGo0?=
 =?utf-8?B?Tkl5NXMzdHpjbU1JV2dHZGRaeHUxTU5vNll1dCtLZWxaMjliemRlNnBtV3p0?=
 =?utf-8?B?cDd4NUJRWXp3cWQ0NkpyU3dlS0pIL2pIVkVnWmE4ZUVWZW1VRWRjZi9EVFdB?=
 =?utf-8?B?WlNxTzBGT1FTV2IxdVlvVitXaFlDZkVjOVdZSjh1RHpzMHJQWGwvUmY1UnhN?=
 =?utf-8?B?SFNkYklZMjlGNDlDN1NPZnI3YW9oSzEwTmJ6aFhNN1dOempXajMzWUUwam0w?=
 =?utf-8?B?bitScWdpck10MjhEdVh4Y0hOdWJPdFU3NU1pRGxUbm43V2hncHFhUUxNSTZo?=
 =?utf-8?B?OEdEc09tcER2UkNlQjZkZ0c5TzZwUzEwNzk5TWFqdGJKamhlVWI2Nm9xSTcr?=
 =?utf-8?B?ZnlrZW9HZEkrbWtDSEdlOFVacUJCd05CU3RzLzJFRnFxMXRrZFdiR2grdEh5?=
 =?utf-8?B?bnNVN1dwcU14NUNhL1hYMllwVWtxUFNIYjV0Ym8rM0tBblNvdStLdnFGbmFx?=
 =?utf-8?B?V0JwSTZpc1I5cjlQOG1jYy8xY21sWm5ncGpUbFNEZ3pXR0h5NjZ5SW56b1Mx?=
 =?utf-8?B?enRGWUdXaTF0S29DOTB5NXVXaG5kQ2p3TnhMYnlrdXhhTnFFaG1OdnJyNVZH?=
 =?utf-8?B?U0hOeVlCV2M0bTNZRlc5OVpCL2s1UDhOakVvRUF4dm1PZjYwNE1vUEJPMDhE?=
 =?utf-8?B?NFAvcUljdGtSNEtuRFBBQmhjZVJaN2Y0dEhLQ3I0ZnBDanFqNWVwWGI2bDlS?=
 =?utf-8?B?Qm1pZmVGSGRjeHlMRVpCMHk3akdDMGV3eSsrbWxTY0VjbXZyMXhCdUVEdkRJ?=
 =?utf-8?B?VWVjRjdoV3o1T0hIbFZjOWJ4YkFUUlhmMVNvUGJhckpRSStXcnBsNW9zU3Fq?=
 =?utf-8?Q?HkHQbPhbGSbu/AI3tCujzYfl6V5Fl75IsxkPNms?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 455a7645-da4e-4d03-7d72-08d97222d774
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 17:13:42.1794
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oFZG24g9Ki2IkBLBZLaFaoglD+PQvcDLK6SuRREk6pqGzMChZNnwFwEecJ4uJ3bGjxrjMxtKYLd+hLnqV1vUmasRXqsoFQMtpf9Nw1rY5lI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4965
X-OriginatorOrg: citrix.com

On 26/08/2021 13:55, Jan Beulich wrote:
> On 26.08.2021 13:57, Andrew Cooper wrote:
>> On 24/08/2021 15:21, Jan Beulich wrote:
>>> While already the case for PVH, there's no reason to treat PV
>>> differently here, though of course the addresses get taken from another
>>> source in this case. Except that, to match CPU side mappings, by defaul=
t
>>> we permit r/o ones. This then also means we now deal consistently with
>>> IO-APICs whose MMIO is or is not covered by E820 reserved regions.
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> Why do we give PV dom0 a mapping of the IO-APIC?=C2=A0 Having thought ab=
out
>> it, it cannot possibly be usable.
>>
>> IO-APICs use a indirect window, and Xen doesn't perform any
>> write-emulation (that I can see), so the guest can't even read the data
>> register and work out which register it represents.=C2=A0 It also can't =
do an
>> atomic 64bit read across the index and data registers, as that is
>> explicitly undefined behaviour in the IO-APIC spec.
>>
>> On the other hand, we do have PHYSDEVOP_apic_{read,write} which, despite
>> the name, is for the IO-APIC not the LAPIC, and I bet these hypercalls
>> where introduced upon discovering that a read-only mapping of the
>> IO-APIC it totally useless.
>>
>> I think we can safely not expose the IO-APICs to PV dom0 at all, which
>> simplifies the memory handling further.
> The reason we do expose it r/o is that some ACPI implementations access
> (read and write) some RTEs from AML. If we don't allow Dom0 to map the
> IO-APIC, Dom0 will typically fail to boot on such systems.

I think more details are needed.

If AML is reading the RTEs, it's is also writing to the index register.

Irrespective of Xen, doing this behind the back of the OS cannot work
safely, because at a minimum the ACPI interpreter would need to take the
ioapic lock, and I see no evidence of workarounds like this in Linux.


In Xen, we appear to swallow writes to mmio_ro ranges which is rude, and
causes the associated reads to read garbage.=C2=A0 This is Xen-induced memo=
ry
corruption inside dom0.


I don't think any of this is legitimate behaviour.=C2=A0 ACPI needs disabli=
ng
on such systems, or interlocking suitably, and its not just IO-APICs
which are problematic - any windowed I/O (e.g. RTC) as well as any other
device with read side effects.

I think we should seriously consider not mapping the IO-APIC at all.=C2=A0
That said, I would be surprised if Linux is cleanly avoiding the
IO-APIC, so a slightly less bad alternative is to redirect to an "MMIO"
frame of ~0's which we still write-discard on top of.

That at least makes the Xen-induced memory corruption behave
deterministically.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:36:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:36:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181283.328371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNf0s-0000uZ-Ho; Tue, 07 Sep 2021 17:36:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181283.328371; Tue, 07 Sep 2021 17:36:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNf0s-0000uS-EB; Tue, 07 Sep 2021 17:36:02 +0000
Received: by outflank-mailman (input) for mailman id 181283;
 Tue, 07 Sep 2021 17:36:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JcTc=N5=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNf0q-0000uG-Ml
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:36:00 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 10316374-1002-11ec-b118-12813bfff9fa;
 Tue, 07 Sep 2021 17:35:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10316374-1002-11ec-b118-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631036159;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=GsjaAJfgrCNlIL34Yd1ZnN8XhjsnG6kdA+YSA8g09jA=;
  b=WAQi6ntvivcCU3IKdj+sXl9/mFNwrqOYo9Y7wk9wHWu6bcBQqnwn5cnY
   UuYtp0X82tN7vUS8MZ4m84gIMb3gGFeR7eo9+tohZM47/W80+qTQycbGN
   nDKGWMplvb9gDNB/9YoxaPpIf6h1DWQ0IAOcNH/E208PgOnt1tnmlVL4q
   8=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: zB0VZmyGTwWt9Gp2YY0ZTcUaN4I7s/9xxEWc7AgJRZaxRmpIWhQMJuigtR8tFbt7NGDlCQix9V
 Yl9sg4DS7rpdvBt2YyErcykB2PvKaQO5Ip5x4TwO05QXEwkcwam6akdwwmhqUMa0Bszih1rhJn
 T3e0IbaYBEu13hOeI/RzW1IC0aBLed6LM7MXX9Bt/tQYVORXhTmjMmMdK1P+AacTL/eASdr6vA
 0QzFo9cBMqu1B7kuLXXNm5gIAharup2oiII8tEDWzCrW4j392qA7GZ4pfrtM00LDy04Ge0z0vs
 3SVpeL6365z04GXdqiR9lzP+
X-SBRS: 5.1
X-MesageID: 53939861
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:IhhBFaoFt/emnMtespdSRMIaV5u8L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBbhHO1OkPYs1NCZLXXbUQqTXfxfBO7ZrQEIdBeOjtK1uZ
 0QFJSWTeeAd2SS7vyKkDVQcexQueVvmZrA7Yy1rwYPPHJXguNbnmNE426gYzxLrWJ9dPwE/f
 Snl6h6TnabCA8qhpPRPAh6YwGPnayGqLvWJTo9QzI34giHij2lrJb8Dhijxx8bFxdC260r/2
 TpmxHwovzLiYD09jbsk0voq7hGktrozdVOQOSKl8guMz3pziKlfp5oVbGutC085Muv9FEput
 /RpApIBbU911rhOkWO5Tf90Qjp1zgjr1fk1F+jmHPm5ff0QTorYvAxzr5xQ1/80Q4Nrdt82K
 VE0yayrJxMFy7Nmyz7+pzhSwxqvlDcmwtgrccjy1hkFacOYr5YqoISuGlPFo0bIS784Ic7VM
 FzEcDn4upMe1/yVQGYgoBW+q3oYp0PJGbDfqBb0fbllAS+3UoJjnfw/fZv3Evpr/kGOt95D+
 etCNUhqFgBdL5OUUrRbN1xNvdfMVa9NC4kBljiaGgPJJt3SU4llKSHlIndxNvaMqDgn6FC1a
 gobjtjxBgPkgTVeJWz4KE=
X-IronPort-AV: E=Sophos;i="5.85,274,1624334400"; 
   d="scan'208";a="53939861"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=h2kVk7IR4CL6JHvSHeBqgj7uXWQIFar8/3EJ4/kHNDSFAePmw3Oj4bI/4MH/ufqBRRFrsZWxWJBksQuRau3ZappG/N9KkG8XnSkJORDiCE2ldilCu+JD9VHyaq+HY8t4EEERDVyVYDwdmjbSE/wFh/b54vJKMvydBrCQP0hXl8+eut2d+FERIrWnukC5OJd8lZsXL/0911FS640w34VYFFX3ExSQzoJRK1kJlD6LSXk1k+218E3Jth2WYKf0wirGR8JQiTR6KKehLwjGxWliZTBY4lJJ4Cpe8uKsLP0JzB81c9jyATCWbzqwOBnK+xlrYkL+W/Lwk1EdE8Z5trOZDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GsjaAJfgrCNlIL34Yd1ZnN8XhjsnG6kdA+YSA8g09jA=;
 b=X51eO0bU1D5ZcbYB2zRrRkfg6zzGrgHoM9AyyCVWlNpFbRT9qtBIqyDk1JhfZuDH8nMrqpjornwS4KwjMwofGrBAELdvC1ywcwhwjFTptq7MCyb8U/afX7DnuHFiY8YzNQCrctBTzzWMsvzet85RngdlOW2wMG1PMCL2xEZJGyYFXbXQhDxbNS7dNqi9JA9oOs/yAhx9KQK6j1fhJ/OC/KU12EXqenZ4ZZ90qR3hUYQiUZf9f59B7RH5API2ck8PI1IuesWxH6O0XAm6m7hFKtGJ5y0HdqwbOymzehiPo0zPm7AUzxh6tl+ZoGsgKb484cJP/3mMkwKiboxyjznT1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GsjaAJfgrCNlIL34Yd1ZnN8XhjsnG6kdA+YSA8g09jA=;
 b=jE7LUW1ixHrrRoSkONH1AZ4W0O5FXvXESxNZROtkmuDbK+DaZ2zjJmv9JWuxD27/u7M4SQO5sHNpXDlns+oxe96bBo9xTN68h8/5J6xAhOEsBBdXNvJPUHFlhaAzaYa6K4USu4N6Yfyy5UeMYXu4dVxfnPymaRlje2jjVx7F8KU=
To: Oleksandr Tyshchenko <olekstysh@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, Henry Wang
	<Henry.Wang@arm.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
 <1631034578-12598-2-git-send-email-olekstysh@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [RFC PATCH 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
Message-ID: <973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com>
Date: Tue, 7 Sep 2021 18:35:47 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1631034578-12598-2-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0428.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18b::19) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e5916e5-4c9a-4f04-6581-08d97225f265
X-MS-TrafficTypeDiagnostic: BY5PR03MB4998:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB49989FC05D74565F4B512858BAD39@BY5PR03MB4998.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: rcN3zlvDev2j8AMCOBLN/pXs3Lo7SRWefXqY9EFbyd4gOIVx2v9mn7UCFfyF4i2SmAyZ4/aAemJ6sclTEG3C4nDaImq0goLM9/A74Uzh9Iu3FZiwx37Pc/LS+XTodyRSHr8L42hYqQYWyH9pmWWUHzNl7ya21uRYk4b25nLIihdMwtlA4ApB3RkikrZGaAaYxDa3q27+9tvk3eqPViiFUKxGMUp0fORnw3co8OIBsAgJAQy8ktIwLNB1q7g1SRa146gQVBQ4v944UY1HQa4HdlaH0WdBGy/wmrhvPUoJ9zz6U7SJdp7DG7C078Wfyf33U2vIAVOSMUj7Je4zhbqAXXcSoH2hMCaL2revHGPdVeBxGYD+pksMfHyq2Vg9zGyCLpW1WBf5vEd1YYMaODPyOC0aPftCFDZXPLRkz9ZGPbHFKhSx+AV3nwmBJl8S5C2W0kuB1zPfsUidqnDuSWDxp/UofMG4Gx5np1wHBAXuZqdyt6oWwqXHXgPv4j6qooF7o5BnuhACYBzZFbaljLzvvHa900+c0FBzRynGIH0jd9TBSga+2vRjZcb5YUv8bve1QEgsBAd0Nf8xYgnRMNAJmQ6MWlYm7ydwUz5C0/gUF5F9Pk/iEfhJw8fnWC/oyYGvzp7VqFDcbsU6SDVBixXRSmoZN0MJvPJnLr5aLE3fUCHG82VdO9rWkEsUxOZ/J8ju5LKcpKyCG9yF26YpJQEk+wJLKFWMmM24acTjPuJQ3jQ=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(376002)(39860400002)(136003)(396003)(36756003)(4326008)(66946007)(186003)(66556008)(16576012)(478600001)(86362001)(8676002)(31696002)(6666004)(2906002)(55236004)(7416002)(5660300002)(31686004)(2616005)(8936002)(316002)(53546011)(38100700002)(83380400001)(956004)(26005)(66476007)(6486002)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NmJaL0JCclZPL1R1RVFZVlhFb3Rsby9mWG9BVUhDemxmZUIvVlhzRFJlSXRG?=
 =?utf-8?B?Q3c5T0N1UnJxMGZDMEpRcWhMMmVPWTltQnppMjh0LzhyY203dXNlUUorL1pF?=
 =?utf-8?B?V0I0Mk1BTE1VUTVMWmlCLzZiM2trcy85NVVINW9jNEJ3c21rNVpWZExxUEFH?=
 =?utf-8?B?ZzJiMHlqUExlbzR2WXpUVXpUKzJDNEo0bEhuM280MHAyMzJ2cjhSeU1KODFw?=
 =?utf-8?B?SlNSdXBUQksyNGErYzFrTE1ibWVoTnlZd0JIeFZRMVlTWXRtOCt6WTMzdlZp?=
 =?utf-8?B?T1kzOEt1b1JnOVVsTDQ2NXVpMmJsNUZjazA3a3NwZjNUMHhSNmFVcWt3d1I0?=
 =?utf-8?B?dXB4MzFTbHVvQTRPNWtWRmdFVFIyMHpzMjB2TXp1clBUV01LVzBTQzhCQmJp?=
 =?utf-8?B?RFNIQk41N25EWEkxNGtBQkdjOGE4aEt2U3BmaFRBVWlNVkJFUXdQVnNTcVZr?=
 =?utf-8?B?VURNWVZMbGFpd3VMNkZRRGlPWExXWmF4ZkpiVGFmb3ExV1JnejZ2N3BJMHVG?=
 =?utf-8?B?TktLTDYyOHdqSHFFOFZsU3psYjhKUmF5SkNRZ3RpeHVJVWlhUVo1MHAzTDhO?=
 =?utf-8?B?d0w0aWYwWE9NZFR5ZTVjMlJWSXE4VEduVFlWN1Y5QlNmdnhyaGNNKzVMaTA4?=
 =?utf-8?B?bHR4ZzVVOVYveWpLWkZlSkQraUpoYjFWdUk5Sm01dklxVlhWb0hnbk5kaTBB?=
 =?utf-8?B?NiswQUt4YXhQZENCU2h3d1QzS2JnTkdIMFVMcnNXVTc0ZVl3WW01WmlFVVBN?=
 =?utf-8?B?SDJRTDJwOFg4KzNoUlhrR3hBdXpoOUo2cUlJSUhMQ2RHUDFKUk9GOC9UbXhF?=
 =?utf-8?B?empieVRZbFB6V0dQVEtaOEpCVE5BNFNPMzZVRUpab3JGUFh0YTJuU2N4OGNP?=
 =?utf-8?B?VTFXMDhIb0xudjArWElEaDNoUmxTM3h0VXp2aTF5ZDFWOXMxcXllWGFsSk1E?=
 =?utf-8?B?WVN0elNWdVAxdEs3dXRvZGpLOGlMNjZoZ0NvSVFubVFteWZlMmt6SDJMQndQ?=
 =?utf-8?B?WUwwakNLQ3l2ODZSa2xDb3RObFc2T0FkSDc4OUpTSmlMeGNjY2tZK1JlWHhI?=
 =?utf-8?B?clVnRW14eUpReXkyZkFyNlMvOHVsenNJZURaVjBOZXVZV0xLTFVuL1JhVFBo?=
 =?utf-8?B?bk1KeXVYZEJScXFzUHNsN1paVzdpQmpFU1NhbFdRWDJBZWF4Wm5uNXkrSlQ3?=
 =?utf-8?B?UkxxeWEyZlBSRUVrWkw2M3pmc3ZHcEpzN0Y2eGpvME9ZT2NzUEdnaXc4TkZY?=
 =?utf-8?B?R283VnVtcTdMSUgvNVVWd04zRlhaaGt4emNTK3plNGtUdURPTm1LUmZ4c09F?=
 =?utf-8?B?OFRNaUF5bzQ1RE9VK1BOeFN3dVhUYkJBTDN1UGhZc05Jb3hRNzhPbmE0NzFs?=
 =?utf-8?B?azl1TlAxT1ZvTTBXVTRVRXpzcFF2eTRNNTFOa2EwSUN2M0FYcnhMcFhldTh2?=
 =?utf-8?B?L0VLMThqTWJldmFwTGhqV3d2ejVjSjBEVy82T0lvTzJTTEdvY01VWmxLM0Jm?=
 =?utf-8?B?SElhTTBmVnRZRUNkM3NBMnFwUnl0UWttV1pwbGltbGZ3TmNwdUdpc2YvOC9y?=
 =?utf-8?B?dnVOLzdyUVNaYU1vTTR0by9HMjk2WHVYUDdaYWdzRDRyQ3hIdGNnNHVUc1py?=
 =?utf-8?B?RGFtZWF3K0JjaVFUbmQ2cEdNR2RQakUrU0hLMEw1b2dadkllQkRaTDZDRUpP?=
 =?utf-8?B?c0p2RWVoeENtdHZSUFRKZGVRVm41UlJiSnZZN0lDUEw5Y3RUN25aNmJLZnlk?=
 =?utf-8?Q?x+PyrvLZ9dB5Y0b5QxvhrzSNzmcEbM3RBhIUl1v?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e5916e5-4c9a-4f04-6581-08d97225f265
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 17:35:55.9591
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bKfelRUTQgcj/sY8qnpXk+x267fov9x5Q77tzuRe73MReXT4u9XIWkCwXGHPPR+5G9NC/Pkin230/B/f9AIGUUWmvrqtdONgaEyuQIJNpz8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4998
X-OriginatorOrg: citrix.com

On 07/09/2021 18:09, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> We need to pass info about maximum supported guest address
> space size to the toolstack on Arm in order to properly
> calculate the base and size of the extended region (safe range)
> for the guest. The extended region is unused address space which
> could be safely used by domain for foreign/grant mappings on Arm.
> The extended region itself will be handled by the subsequents
> patch.
>
> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
> hap_paddr_bits.
>
> As we change the size of structure bump the interface version.
>
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

So while I think this is a suitable way forward, you're painting
yourself into a corner WRT migration.

On x86, the correct value is d->arch.cpuid->extd.maxphysaddr and this
value is under toolstack control, not Xen control.=C2=A0 In particular, it
needs to be min(hostA, hostB) to make migration safe, and yes - this is
currently a hole in x86's migration logic that will cause large VMs to
explode.

The same will be true on ARM as/when you gain migration support.

I think this would be better as a domctl.=C2=A0 On ARM, it can reference
p2m_ipa_bits for now along with a /* TODO - make per-domain for
migration support */, while on x86 it can take the appropriate value
(which will soon actually be safe in migration scenarios).

However, that does change the ordering requirements in the toolstack -
this hypercall would need to be made after the domain is created, and
has been levelled, and before its main memory layout is decided.

Alternatively, the abstraction could be hidden in libxl itself in arch
specific code, with x86 referring to the local cpu policy (as libxl has
the copy it is/has worked on), and ARM making a hypercall.=C2=A0 This does
make the ordering more obvious.

(As a note on the x86 specifics of this patch, hap_paddr_bits is
actually unused in practice.=C2=A0 It was a proposal from AMD for the
hypervisor to fill in those details, which wasn't implemented by anyone,
not even Xen, because the important field to modify is maxphysaddr if
you don't want to rewrite every kernel to behave differently when
virtualised.)

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 17:40:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 17:40:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181290.328382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNf4m-0001zR-04; Tue, 07 Sep 2021 17:40:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181290.328382; Tue, 07 Sep 2021 17:40:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNf4l-0001yt-Sz; Tue, 07 Sep 2021 17:40:03 +0000
Received: by outflank-mailman (input) for mailman id 181290;
 Tue, 07 Sep 2021 17:40:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hq6o=N5=epam.com=prvs=98842be756=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNf4k-0001hT-6i
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 17:40:02 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a012b290-1002-11ec-b118-12813bfff9fa;
 Tue, 07 Sep 2021 17:40:00 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 187HdHfK008599; 
 Tue, 7 Sep 2021 17:39:58 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3axcp0801p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 07 Sep 2021 17:39:57 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5698.eurprd03.prod.outlook.com (2603:10a6:208:171::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Tue, 7 Sep
 2021 17:39:54 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.014; Tue, 7 Sep 2021
 17:39:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a012b290-1002-11ec-b118-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L/MK8hwg9nGOd8gb5Ws2edbdyv778uQwgZPVMD8b+MKRAhCje0TQO+6XghwEA/CMt4CL7T8OP25/0yBU1JJK9LwJbX42boTbze8fAs1LBuuIC9ve+VPx+fBlLcFEtKiuuE3Wq55kgOD/Fcsdf9l9nAAsCiin/fXvmWjutQzWdtmWdJLZZygWeE2OMAIK0rV50LNDhLv6lgQz4M19/OeAJr3QXoNCpx/xGtCJsY5779MElfqOAamLM1p0YpcSA3TwXy+0xRBPNJt1pVYKvqThanG+4wHPdSuCGvjlNs1U1v4fxQChYwZt9k1foSd6jjt6Y8IJ1y8NU2FijK/zDJ9yLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q5ycq2nXR5FZX0BWbi7FjueKCXrfpTy2STsvy440gaU=;
 b=m3PQuJB9rfpJ7tobfCoYCGotDjtY0ZSRCuIPwWb0H3YUZlpqPW7PQHb1UrEsz38j/tFd5SGHKEmioCbAEU2gIy/JeCQBwupJ9JPxo7XknQXuNZoqS7fKruOtFnreDaEiLS16Ifte/GYOjudqFcfxWEn2wl8DMZsgOliS3O5uC1T5Es1vLC81IejbkBG8AXDRMeAzBjEwfVOET3aMMI4GmC6r5AYlNsf8T783PVSWr8Y15HCVQSXmVtohnDUQ2Lr53xTAmagjEOMo1cu9peX3/1ZGifMBqRfAo/urD5v+I77lN/M2kHAnaf8irF78vYsUXjnOpQST4JC+xN2hLEErWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q5ycq2nXR5FZX0BWbi7FjueKCXrfpTy2STsvy440gaU=;
 b=tksrpfuOZn1tf3LjUt99XqNea3nx3XZMXcP3hSp6uUx3lSRSDiz56nJ1Blp2YNLSWTc9wj/Ij5d0RggXqKTGJGIXWqDhW89090QmxC3qaeEkVWguAfLwFGykY9CvHmHUIPF5E4PF6j2CEBu/7/UvNzB3K7LBmaaqJuw9mr/FY7Rd93CztytyZ1B0ngRyfNYPu6m6EC6QKlUXQiIscHnraSB2KLoH1+xlUqLoftKLaoyRyQlPVQUBtE63rLj0EKbzWNNak+bwZEm6E0iZLPvpv10Y6DPqhtqr6cxvKa5RwgEDF36grslOVDNeUpEmiYjxmRK44UhrmaWtTekr3sKtYA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLICAADFtgIAAE0qA
Date: Tue, 7 Sep 2021 17:39:53 +0000
Message-ID: <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
In-Reply-To: <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d665de14-4035-41cd-b75d-08d97226808c
x-ms-traffictypediagnostic: AM0PR03MB5698:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5698D597C0C97051001595D2E7D39@AM0PR03MB5698.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 6vuZpvrPAynKg2di+1hAuLDCecb9fCfpWYcYFaKhzenTyjgH7XHdpMreUOVeFagECEenqt4xbKzBmczAaBUtObPYKoR3/wnQ8rkvq3Nxk7G3lqGEWm6H0hNkv4wKWxQ2JUddWO1RtlsBK927krBJqDK1mgkVlQW1Wwc6CVt26yiY4FA8auOKStotRpK2tb2YYsLvMHXMOe1PrYZL9h2zLR7AG36dgGm6bPEp3O2ib+U7vjzviEg+HGkWc3YTIHursq7ftM+BjlSxO5tM1gZZhITZCyCj6ttbPe0Xg0Y6Auub71mGCNLMgDaQQ2wfpj9zlxkRIAss1QVWDuypuTznZSOnlVPiEdJe7Z6Q1Nv2D/oQtgJOnV2OPaVc9u7+WYcUyyIsGEPTUzOGI6OZlwoT/hHy7O0/AAwjnmDUiPKULzjjzjmIFrma2/zsklkgotXlPpMLQeHtyqcyEK21y9Bgo+5MylyalULZW9zWC2R0cX8eXvPzavhhBZPikNmy9Lu9U506Y4McI4anbCP6DBKqIJNC8hVd8GYjTJ8jtyrvSIv5HlTxdDM2htQe+n9hVzRPM7Fa3gM1Ni5delgukedj2GcofsVSr6jutodgtoufLU9zS57vcJDmrtRHypeWuPc8YnMESjqbz15YNaUUMkup28lJ6gZyeN1mVgwZk+Evpl6Ufgihi7bgC2/mank2hdwA8dHbFryJUOhWslAae0HDLV/TkFFyFOWzvbnB5NKDrShi4PKAtmjFvU7PzlWz0MJg
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(376002)(136003)(346002)(39860400002)(66946007)(31686004)(64756008)(38070700005)(66446008)(66556008)(36756003)(316002)(86362001)(66476007)(53546011)(76116006)(8936002)(6486002)(6506007)(2906002)(478600001)(31696002)(8676002)(71200400001)(83380400001)(122000001)(110136005)(26005)(38100700002)(54906003)(186003)(4326008)(2616005)(5660300002)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SUM4VW5WSHA0VXhTalBLSlFyZFdHanlOdEV4a2J2czVTbGg1UHNRbnNXbUwr?=
 =?utf-8?B?d1pOdDVacGF1emdYQ09UL3lrcUtZaWZrVW5wNDJpbXkwOVJjMHVMNWx3TVZZ?=
 =?utf-8?B?cy82TnF2MmVKa1luclBQV1lzMVBJRGJOZXJIOHRuNnpQZHFUdjdxb2pHckl6?=
 =?utf-8?B?ZFZYZWdxSmNZY0dvQXRZc3oyRm1wM2NRZW9MbDhyaXBWZWw4Z0xaZjJnWWpS?=
 =?utf-8?B?VmtOZmxOZHQ2NUtoM0hGWnR5dzVydE5EcFRudFEvaUxYeVV6c0phUVJNUFA0?=
 =?utf-8?B?aUFSaXlQQjlYUndQbWNvTXhoU3V1cGJnYWZ0dUhwOERjclAvWGRtYWNybDEx?=
 =?utf-8?B?S2dRYWVxellmZW5rN3Q5WC9JVjNLUEVjQ3poVkcvbkFOQTFvNHRUMElWZnNy?=
 =?utf-8?B?NzlMOENqRkt3SkJsUVpWQTZIcXRDaDVUejlxMVJKSDROQjlyVENodE9aUE9B?=
 =?utf-8?B?UTYwNFpiTXJuSlJSaFVtanVBNW4wb2J5L05mNmk2Yk1mczZKZWYzRTFnVW1R?=
 =?utf-8?B?d1ZnaWc2bi95VDNzejhrZWc1RzdzaFZqVTVZMTZLUy9pQVZmeWNwcjlYQ1RE?=
 =?utf-8?B?amVJcnFSMnpxZDhUTzhIclRsUHYvQng3c1YwK1EvUW5hTzhnMTBWckkvN3hT?=
 =?utf-8?B?VjRZcGRLMGNVckZpQTlkKzBVNFJLK1Q5KzJmck45ZXJQalNtRXZJOERSYVpa?=
 =?utf-8?B?UFFrazdpdXg1VnU2aUlkZ0dBWmwwQ0lmL0tFb3g4MDJUem9IcWk5Yi9Zdndy?=
 =?utf-8?B?bHhpMEVoMVFDaHRZR0JSUjRZeG1HSmZrWmJBbXlKYnZwZk5haitqcnpiQ3lo?=
 =?utf-8?B?S0NWMkxMZWlmU1BKNVQvSXdveWxqN2RwdDh6VXNwRXJHN0p2M2NsZHMyVFRL?=
 =?utf-8?B?WkhOSWMwV1Q1WUZ2NStvdUI4WUFieUNnd3o4SFhvS0JJV2YvcCsxS0ZXZXlV?=
 =?utf-8?B?Wlltak1rR2lIcXRCTTIva3c2aFFHQmFLRTBET2dIUE4xM3d6bWRYWnQwNGhn?=
 =?utf-8?B?Y0ZMS3JubEUzY2oxa1ZnZnVSQkl6ZlJOVE9DU0dKZSt6T054b2lXSHVSNzQy?=
 =?utf-8?B?NjM1Q1RkdUFpeURjRTFtbTdiUzRQRldVTjdmVDRpamtkZ3JxbjgxRUpGYm13?=
 =?utf-8?B?TDRIVWlrMlRIUTNaVHNzRzVYdWxoM3FqUytHaHdsMkhKaXpqTGpMY2FiRnhj?=
 =?utf-8?B?Q0x4UFAwYWR2SmdvVjBkM3BCc01HWXlBd3N1SWpCSzZnUW9zWmVDaXRna21w?=
 =?utf-8?B?dnp5RXZKa0tRYjdWaXoxeVNnWFljb3ZXZyttN0ZvL3RDWXRLOFRrdDhweU9V?=
 =?utf-8?B?b1VXcWp3bTlqMGlBcDg4Yks4ZmV1L1RMYUNTeEZxQVFxOTMzdEpQY1ZoTjEw?=
 =?utf-8?B?RUQybVpORDhTWk5aeFF6YVFCemxhcmNwTVBGT2t0eWZ5M1NjTVU1dWdITERi?=
 =?utf-8?B?ZXErbGF2ZFJsV0pQWUNHRmYwcWpXalVkWmxlb3BGNkdUK05iVGtWWkZKUUIz?=
 =?utf-8?B?TW05ZWZNcCtyRStDcFhjeUJZRlQ2NG1yUkw1cGtDTUVTcnJXU0V4M2g2QTRX?=
 =?utf-8?B?ZmtkY3phbTVPTDZEbmw5MEtxNzJZeXh0RzJUR0NuYnVWWE5nT0N1T0g0aHZ4?=
 =?utf-8?B?Z0Mxa3BPTlRTRXBPK3hxbCtHNlBUc3ZUMmdWSmNncnE4MThSaEJLb2VKU1Ux?=
 =?utf-8?B?c0FselI3WnNZQjR1VVJiNkxOc1RwMkxEV2w0TEd3NlBsYW11b1NRTCtad1NQ?=
 =?utf-8?Q?HJ4PaUIuYRSMejkel5OzfnL3728WaB5MTuqEvUg?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <7D12DE7B3F3E954297B1A39DBC5196F3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d665de14-4035-41cd-b75d-08d97226808c
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 17:39:53.9639
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: GsgvCCCFxy/Zfb0OtDOxEVWxGSJnts0euCdkHfWPRsRSjR2CZbFM7KnRrGeblQyV7mavMN0uogZnPIYM1uKjRvZ+Da07Za362kDQ37uh7Gjzo/TxLCZ4RDEShwgSRjH/
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5698
X-Proofpoint-ORIG-GUID: DjBeJHtbMsIyhGXSYA36Cy7vhdlgs_tP
X-Proofpoint-GUID: DjBeJHtbMsIyhGXSYA36Cy7vhdlgs_tP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-07_06,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0
 spamscore=0 phishscore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999
 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109070114

DQpPbiAwNy4wOS4yMSAxOTozMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTU6MzMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDYuMDkuMjEgMTc6
MzEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hlYWRl
ci5jDQo+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+Pj4gQEAgLTQwMCwx
MiArNDAwLDcyIEBAIHN0YXRpYyB2b2lkIGJhcl93cml0ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAq
cGRldiwgdW5zaWduZWQgaW50IHJlZywNCj4+Pj4gICAgc3RhdGljIHZvaWQgZ3Vlc3RfYmFyX3dy
aXRlKGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVnLA0KPj4+PiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgdmFsLCB2b2lkICpkYXRhKQ0K
Pj4+PiAgICB7DQo+Pj4+ICsgICAgc3RydWN0IHZwY2lfYmFyICpiYXIgPSBkYXRhOw0KPj4+PiAr
ICAgIGJvb2wgaGkgPSBmYWxzZTsNCj4+Pj4gKw0KPj4+PiArICAgIGlmICggYmFyLT50eXBlID09
IFZQQ0lfQkFSX01FTTY0X0hJICkNCj4+Pj4gKyAgICB7DQo+Pj4+ICsgICAgICAgIEFTU0VSVChy
ZWcgPiBQQ0lfQkFTRV9BRERSRVNTXzApOw0KPj4+PiArICAgICAgICBiYXItLTsNCj4+Pj4gKyAg
ICAgICAgaGkgPSB0cnVlOw0KPj4+PiArICAgIH0NCj4+Pj4gKyAgICBlbHNlDQo+Pj4+ICsgICAg
ICAgIHZhbCAmPSBQQ0lfQkFTRV9BRERSRVNTX01FTV9NQVNLOw0KPj4+PiArICAgIGJhci0+Z3Vl
c3RfYWRkciAmPSB+KDB4ZmZmZmZmZmZ1bGwgPDwgKGhpID8gMzIgOiAwKSk7DQo+Pj4+ICsgICAg
YmFyLT5ndWVzdF9hZGRyIHw9ICh1aW50NjRfdCl2YWwgPDwgKGhpID8gMzIgOiAwKTsNCj4+PiBX
aGF0IHlvdSBzdG9yZSBoZXJlIGlzIG5vdCB0aGUgYWRkcmVzcyB0aGF0J3MgZ29pbmcgdG8gYmUg
dXNlZCwNCj4+IGJhci0+Z3Vlc3RfYWRkciBpcyBuZXZlciB1c2VkIGRpcmVjdGx5IHRvIGJlIHJl
cG9ydGVkIHRvIGEgZ3Vlc3QuDQo+IEFuZCB0aGlzIGlzIHdoYXQgSSBxdWVzdGlvbiwgYXMgYW4g
YXBwcm9hY2guIFlvdXIgbW9kZWwgX21heV8gd29yaywNCj4gYnV0IGl0cyBuZWVkbGVzc2x5IGRl
dmlhdGluZyBmcm9tIGhvdyBwZW9wbGUgbGlrZSBtZSB3b3VsZCBleHBlY3QNCj4gdGhpcyB0byB3
b3JrLiBBbmQgaWYgaXQncyBpbnRlbmRlZCB0byBiZSB0aGlzIHdheSwgaG93IHdvdWxkIEkNCj4g
aGF2ZSBrbm93bj8NCldlbGwsIEkganVzdCB0cmllZCB0byBmb2xsb3cgdGhlIG1vZGVsIHdlIGFs
cmVhZHkgaGF2ZSBpbiB0aGUgZXhpc3RpbmcgY29kZS4uLg0KPg0KPj4gSXQgaXMgYWx3YXlzIHVz
ZWQgYXMgYW4gaW5pdGlhbCB2YWx1ZSB3aGljaCBpcyB0aGVuIG1vZGlmaWVkIHRvIHJlZmxlY3QN
Cj4+IGxvd2VyIGJpdHMsIGUuZy4gQkFSIHR5cGUgYW5kIGlmIHByZWZldGNoYWJsZSwgc28gSSB0
aGluayB0aGlzIGlzIHBlcmZlY3RseQ0KPj4gZmluZSB0byBoYXZlIGl0IHRoaXMgd2F5Lg0KPiBB
bmQgaXQgaXMgYWxzbyBwZXJmZWN0bHkgZmluZSB0byBzdG9yZSB0aGUgdmFsdWUgdG8gYmUgaGFu
ZGVkDQo+IGJhY2sgdG8gZ3Vlc3RzIG9uIHRoZSBuZXh0IHJlYWQuIEtlZXBzIHRoZSByZWFkIHBh
dGggc2ltcGxlLA0KPiB3aGljaCBJIHRoaW5rIGNhbiBiZSBhc3N1bWVkIHRvIGJlIHRha2VuIG1v
cmUgZnJlcXVlbnRseSB0aGFuDQo+IHRoZSB3cml0ZSBvbmUuIFBsdXMgc3RvcmVkIHZhbHVlcyBy
ZWZsZWN0IHJlYWxpdHkuDQo+DQo+IFBsdXMgLSBpZiB3aGF0IHlvdSBzYXkgd2FzIHJlYWxseSB0
aGUgY2FzZSwgd2h5IGRvIHlvdSBtYXNrIG9mZg0KPiBQQ0lfQkFTRV9BRERSRVNTX01FTV9NQVNL
IGhlcmU/IFlvdSBzaG91bGQgdGhlbiBzaW1wbHkgc3RvcmUNCj4gdGhlIHdyaXR0ZW4gdmFsdWUg
YW5kIGRvIF9hbGxfIHRoZSBwcm9jZXNzaW5nIGluIHRoZSByZWFkIHBhdGguDQo+IE5vIHBvaW50
IGhhdmluZyBwYXJ0aWFsIGxvZ2ljIGluIHR3byBwbGFjZXMuDQoNCkkgd2lsbCBtb3ZlIHRoZSBs
b2dpYyB0byB0aGUgd3JpdGUgaGFuZGxlciwgaW5kZWVkIHdlIGNhbiBzYXZlIHNvbWUNCg0KQ1BV
IGN5Y2xlcyBoZXJlDQoNCj4NCj4+PiAgICBhcw0KPj4+IHlvdSBkb24ndCBtYXNrIG9mZiB0aGUg
bG93IGJpdHMgKHRvIGFjY291bnQgZm9yIHRoZSBCQVIncyBzaXplKS4NCj4+PiBXaGVuIGEgQkFS
IGdldHMgd3JpdHRlbiB3aXRoIGFsbCBvbmVzLCBhbGwgd3JpdGFibGUgYml0cyBnZXQgdGhlc2UN
Cj4+PiBvbmVzIHN0b3JlZC4gVGhlIGFkZHJlc3Mgb2YgdGhlIEJBUiwgYWl1aSwgcmVhbGx5IGNo
YW5nZXMgdG8NCj4+PiAodHlwaWNhbGx5KSBjbG9zZSBiZWxvdyA0R2IgKGluIHRoZSBjYXNlIG9m
IGEgMzItYml0IEJBUiksIHdoaWNoDQo+Pj4gaXMgd2h5IG1lbW9yeSAvIEkvTyBkZWNvZGluZyBz
aG91bGQgYmUgb2ZmIHdoaWxlIHNpemluZyBCQVJzLg0KPj4+IFRoZXJlZm9yZSB5b3Ugc2hvdWxk
bid0IGxvb2sgZm9yIHRoZSBzcGVjaWZpYyAiYWxsIHdyaXRhYmxlIGJpdHMNCj4+PiBhcmUgb25l
cyIgcGF0dGVybiAob3Igd29yc2UsIGFzIHlvdSBwcmVzZW50bHkgZG8sIHRoZSAiYWxsIGJpdHMN
Cj4+PiBvdXRzaWRlIG9mIHRoZSB0eXBlIHNwZWNpZmllciBhcmUgb25lcyIgb25lKSBvbiB0aGUg
cmVhZCBwYXRoLg0KPj4+IEluc3RlYWQgbWFzayB0aGUgdmFsdWUgYXBwcm9wcmlhdGVseSBoZXJl
LCBhbmQgc2ltcGx5IHJldHVybiBiYWNrDQo+Pj4gdGhlIHN0b3JlZCB2YWx1ZSBmcm9tIHRoZSBy
ZWFkIHBhdGguDQo+PiAiUENJIExPQ0FMIEJVUyBTUEVDSUZJQ0FUSU9OLCBSRVYuIDMuMCIsICJJ
TVBMRU1FTlRBVElPTiBOT1RFDQo+Pg0KPj4gU2l6aW5nIGEgMzItYml0IEJhc2UgQWRkcmVzcyBS
ZWdpc3RlciBFeGFtcGxlIiBzYXlzLCB0aGF0DQo+Pg0KPj4gIlNvZnR3YXJlIHNhdmVzIHRoZSBv
cmlnaW5hbCB2YWx1ZSBvZiB0aGUgQmFzZSBBZGRyZXNzIHJlZ2lzdGVyLCB3cml0ZXMNCj4+IDAg
RkZGRiBGRkZGaCB0byB0aGUgcmVnaXN0ZXIsIHRoZW4gcmVhZHMgaXQgYmFjay4iDQo+Pg0KPj4g
VGhlIHNhbWUgYXBwbGllcyBmb3IgNjQtYml0IEJBUnMuIFNvIHdoYXQncyB3cm9uZyBpZiBJIHRy
eSB0byBjYXRjaCBzdWNoDQo+PiBhIHdyaXRlIHdoZW4gYSBndWVzdCB0cmllcyB0byBzaXplIHRo
ZSBCQVI/IFRoZSBvbmx5IGRpZmZlcmVuY2UgaXMgdGhhdA0KPj4gSSBjb21wYXJlIGFzDQo+PiAg
IMKgwqDCoMKgwqDCoMKgIGlmICggKHZhbCAmIFBDSV9CQVNFX0FERFJFU1NfTUVNX01BU0tfMzIp
ID09IFBDSV9CQVNFX0FERFJFU1NfTUVNX01BU0tfMzIgKQ0KPj4gd2hpY2ggaXMgYmVjYXVzZSB2
YWwgaW4gdGhlIHF1ZXN0aW9uIGhhcyBsb3dlciBiaXRzIGNsZWFyZWQuDQo+IEJlY2F1c2Ugd2hp
bGUgdGhpcyBtYXRjaGVzIHdoYXQgdGhlIHNwZWMgc2F5cywgaXQncyBub3QgZW5vdWdoIHRvDQo+
IG1hdGNoIGhvdyBoYXJkd2FyZSBiZWhhdmVzLg0KQnV0IHdlIHNob3VsZCBpbXBsZW1lbnQgYXMg
dGhlIHNwZWMgc2F5cywgbm90IGxpa2UgYnVnZ3kgaGFyZHdhcmUgYmVoYXZlcw0KPiAgIFlldCB5
b3Ugd2FudCB0byBtaW1pYyBoYXJkd2FyZSBiZWhhdmlvcg0KPiBhcyBjbG9zZWx5IGFzIHBvc3Np
YmxlIGhlcmUuIFRoZXJlIGlzIChpaXJjKSBhdCBsZWFzdCBvbmUgb3RoZXINCj4gcGxhY2UgaW4g
dGhlIHNvdXJjZSB0cmVlIHdlcmUgd2UgaGFkIHRvIGFkanVzdCBhIHNpbWlsYXIgaW5pdGlhbA0K
PiBpbXBsZW1lbnRhdGlvbiB0byBiZSBjbG9zZXIgdG8gb25lIGV4cGVjdGVkIGJ5IGd1ZXN0cywN
Cg0KQ291bGQgeW91IHBsZWFzZSBwb2ludCBtZSB0byB0aGF0IGNvZGUgc28gSSBjYW4gY29uc3Vs
dCBhbmQgcG9zc2libHkNCg0KcmUtdXNlIHRoZSBhcHByb2FjaD8NCg0KPiAgIG5vIG1hdHRlcg0K
PiB0aGF0IHRoZXkgbWF5IG5vdCBiZSBmb2xsb3dpbmcgdGhlIHNwZWMgdG8gdGhlIGxldHRlci4g
RG9uJ3QNCj4gZm9yZ2V0IHRoYXQgdGhlcmUgbWF5IGJlIGJ1Z3MgaW4ga2VybmVscyB3aGljaCBk
b24ndCBzdXJmYWNlIHVudGlsDQo+IHRoZSBrZXJuZWwgZ2V0cyBydW4gb24gYW4gb3Zlcmx5IHNp
bXBsaXN0aWMgZW11bGF0aW9uLg0KVGhpcyBpcyBzYWQuIEFuZCB0aGUga2VybmVsIG5lZWRzIHRv
IGJlIGZpeGVkIHRoZW4sIG5vdCBYZW4NCj4NCj4+Pj4gQEAgLTUyMiw2ICs1ODIsMTMgQEAgc3Rh
dGljIGludCBhZGRfYmFyX2hhbmRsZXJzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBib29sIGlzX2h3
ZG9tKQ0KPj4+PiAgICAgICAgICAgICAgICBpZiAoIHJjICkNCj4+Pj4gICAgICAgICAgICAgICAg
ICAgIHJldHVybiByYzsNCj4+Pj4gICAgICAgICAgICB9DQo+Pj4+ICsgICAgICAgIC8qDQo+Pj4+
ICsgICAgICAgICAqIEl0IGlzIG5laXRoZXIgc2FmZSBub3Igc2VjdXJlIHRvIGluaXRpYWxpemUg
Z3Vlc3QncyB2aWV3IG9mIHRoZSBCQVJzDQo+Pj4+ICsgICAgICAgICAqIHdpdGggcmVhbCB2YWx1
ZXMgd2hpY2ggYXJlIHVzZWQgYnkgdGhlIGhhcmR3YXJlIGRvbWFpbiwgc28gYXNzaWduDQo+Pj4+
ICsgICAgICAgICAqIGFsbCB6ZXJvcyB0byBndWVzdCdzIHZpZXcgb2YgdGhlIEJBUnMsIHNvIHRo
ZSBndWVzdCBjYW4gcGVyZm9ybQ0KPj4+PiArICAgICAgICAgKiBwcm9wZXIgUENJIGRldmljZSBl
bnVtZXJhdGlvbiBhbmQgYXNzaWduIEJBUnMgb24gaXRzIG93bi4NCj4+Pj4gKyAgICAgICAgICov
DQo+Pj4+ICsgICAgICAgIGJhcnNbaV0uZ3Vlc3RfYWRkciA9IDA7DQo+Pj4gSSdtIGFmcmFpZCBJ
IGRvbid0IHVuZGVyc3RhbmQgdGhlIGNvbW1lbnQ6IFdpdGhvdXQgbWVtb3J5IGRlY29kaW5nDQo+
Pj4gZW5hYmxlZCwgdGhlIEJBUnMgYXJlIHNpbXBsZSByZWdpc3RlcnMgKHdpdGggYSBmZXcgci9v
IGJpdHMpLg0KPj4gTXkgZmlyc3QgaW1wbGVtZW50YXRpb24gd2FzIHRoYXQgYmFyLT5ndWVzdF9h
ZGRyIHdhcyBpbml0aWFsaXplZCB3aXRoDQo+PiB0aGUgdmFsdWUgb2YgYmFyLT5hZGRyIChwaHlz
aWNhbCBCQVIgdmFsdWUpLCBidXQgdGFsa2luZyBvbiBJUkMgd2l0aA0KPj4gUm9nZXIgaGUgc3Vn
Z2VzdGVkIHRoYXQgdGhpcyBtaWdodCBiZSBhIHNlY3VyaXR5IGlzc3VlIHRvIGxldCBndWVzdA0K
Pj4gYSBoaW50IGFib3V0IHBoeXNpY2FsIHZhbHVlcywgc28gdGhlbiBJIGNoYW5nZWQgdGhlIGFz
c2lnbm1lbnQgdG8gYmUgMC4NCj4gV2VsbCwgeWVzLCB0aGF0J3MgY2VydGFpbmx5IGNvcnJlY3Qu
IEl0J3MgcGVyaGFwcyB0b28gdW5vYnZpb3VzIHRvIG1lDQo+IHdoeSBvbmUgbWF5IHdhbnQgdG8g
dXNlIHRoZSBob3N0IHZhbHVlIGhlcmUgaW4gdGhlIGZpcnN0IHBsYWNlLiBJdA0KPiBzaW1wbHkg
aGFzIG5vIG1lYW5pbmcgaGVyZS4NCkRvIHlvdSB3YW50IG1lIHRvIHJlbW92ZSB0aGUgY29tbWVu
dD8NCj4NCj4+Pj4gLS0tIGEveGVuL2luY2x1ZGUveGVuL3BjaV9yZWdzLmgNCj4+Pj4gKysrIGIv
eGVuL2luY2x1ZGUveGVuL3BjaV9yZWdzLmgNCj4+Pj4gQEAgLTEwMyw2ICsxMDMsNyBAQA0KPj4+
PiAgICAjZGVmaW5lICBQQ0lfQkFTRV9BRERSRVNTX01FTV9UWVBFXzY0CTB4MDQJLyogNjQgYml0
IGFkZHJlc3MgKi8NCj4+Pj4gICAgI2RlZmluZSAgUENJX0JBU0VfQUREUkVTU19NRU1fUFJFRkVU
Q0gJMHgwOAkvKiBwcmVmZXRjaGFibGU/ICovDQo+Pj4+ICAgICNkZWZpbmUgIFBDSV9CQVNFX0FE
RFJFU1NfTUVNX01BU0sJKH4weDBmVUwpDQo+Pj4+ICsjZGVmaW5lICBQQ0lfQkFTRV9BRERSRVNT
X01FTV9NQVNLXzMyCSh+MHgwZlUpDQo+Pj4gUGxlYXNlIGRvbid0IGludHJvZHVjZSBhbiBpZGVu
dGljYWwgY29uc3RhbnQgdGhhdCdzIG1lcmVseSBvZg0KPj4+IGRpZmZlcmVudCB0eXBlLiAodWlu
dDMyX3QpUENJX0JBU0VfQUREUkVTU19NRU1fTUFTSyBhdCB0aGUgdXNlDQo+Pj4gc2l0ZSAoaWYg
YWN0dWFsbHkgc3RpbGwgbmVlZGVkIGFzIHBlciB0aGUgY29tbWVudCBhYm92ZSkgd291bGQNCj4+
PiBzZWVtIG1vcmUgY2xlYXIgdG8gbWUuDQo+PiBPaywgSSB0aG91Z2h0IHR5cGUgY2FzdGluZyBp
cyBhIGJpZ2dlciBldmlsIGhlcmUNCj4gT2Z0ZW4gaXQgaXMsIGJ1dCBpbW8gaGVyZSBpdCBpcyBu
b3QuIEkgaG9wZSB5b3UgcmVhbGl6ZSB0aGF0IH4weDBmVQ0KPiBpZiBub3QgbmVjZXNzYXJpbHkg
MHhmZmZmZmZmMD8gV2UgbWFrZSBjZXJ0YWluIGFzc3VtcHRpb25zIG9uIHR5cGUNCj4gd2lkdGhz
LiBGb3IgdW5zaWduZWQgaW50IHdlIGFzc3VtZSBpdCB0byBiZSBhdCBsZWFzdCAzMiBiaXRzIHdp
ZGUuDQo+IFdlIHNob3VsZCBhdm9pZCBhc3N1bXB0aW9ucyBvZiBpdCBiZWluZyBleGFjdGx5IDMy
IGJpdHMgd2lkZS4gSnVzdA0KPiBsaWtlIHdlIGNhbm5vdCAobW9yZSBvYnZpb3VzbHkpIGFzc3Vt
ZSB0aGUgd2lkdGggb2YgdW5zaWduZWQgbG9uZy4NCj4gKFdoaWNoIHRlbGxzIHVzIHRoYXQgZm9y
IDMyLWJpdCBhcmNoZXMgUENJX0JBU0VfQUREUkVTU19NRU1fTUFTSyBpcw0KPiBhY3R1YWxseSBv
ZiB0aGUgd3JvbmcgdHlwZS4gVGhpcyBjb25zdGFudCBzaG91bGQgYmUgdGhlIHNhbWUgbm8NCj4g
bWF0dGVyIHRoZSBiaXRuZXNzLikNCk9rLCBJIHdpbGwgbm90IGludHJvZHVjZSBhIG5ldyBkZWZp
bmUgYW5kIHVzZSAodWludDMyX3QpDQo+DQo+IEphbg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2Fu
ZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 18:09:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 18:09:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181298.328393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNfXa-0005eo-Ce; Tue, 07 Sep 2021 18:09:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181298.328393; Tue, 07 Sep 2021 18:09:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNfXa-0005eh-9c; Tue, 07 Sep 2021 18:09:50 +0000
Received: by outflank-mailman (input) for mailman id 181298;
 Tue, 07 Sep 2021 18:09:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M3oQ=N5=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mNfXZ-0005eb-J0
 for xen-devel@lists.xen.org; Tue, 07 Sep 2021 18:09:49 +0000
Received: from mail-oi1-x22f.google.com (unknown [2607:f8b0:4864:20::22f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e4df1ca-9116-47f3-8ca0-a63691a40011;
 Tue, 07 Sep 2021 18:09:47 +0000 (UTC)
Received: by mail-oi1-x22f.google.com with SMTP id bi4so13838516oib.9
 for <xen-devel@lists.xen.org>; Tue, 07 Sep 2021 11:09:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e4df1ca-9116-47f3-8ca0-a63691a40011
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=aVw++g1itE5W1KXRa2qBfqmZ00VcJA2TbGtfWD/TAkI=;
        b=Pno1ppX3t5c+a8nm871i+2J1n2BTXpQiYb+0xVgI98VEyKggmXDlkBqbsM/rLw+apF
         TObj4WW/6sqIscc58HqCalPz9bfinKhEMT/xR6A9uHnvn5W8HRiL4L4PPh0r79kRZffG
         peNo0DzLXFSzrt79CrL4LhA+KbPYfrI0h4RpVYpHY6/wQINkaK9Z1kQHCsok6L8MUFAW
         galdV+YBy69bYskaJcQ6lemREOUgxrQjQi3bxGh5wC/ElN38IEE5HO2bvX0sDJL05qoH
         2y3wuFdRC7P/JSAvrSCE1BkKkq0PBtH+fA6bXBO2OMGgoqoE+smATTEkQrIdHBBD1Ab/
         bXJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=aVw++g1itE5W1KXRa2qBfqmZ00VcJA2TbGtfWD/TAkI=;
        b=RRF+4gVwezgI/4GVvxrQvJmeTVdlS+S3jHYg3TjHKeIZ9czYYgMMgid8dmponYWFjv
         QRtcuI/f6eLwdqDODkh36H2E/NUVKXwZg1qchQG3td8F/qH3xXVKnA+bmLnQbiUqv02z
         BqXCyuzOLjsleHpARhSucWmaWTEYr3AqFvVzZMgwK8qnK1wsLEmG2eyhBbvBYdEL0VeC
         rx0jGufZCLQtNQR2KGw4KlATCsOTUj82gmb3QPu8who8fPNvew/PpKQUQDKyVk1WyMrn
         myzrVWpdFeiv0+SBDr5hYmCTBr99/qVjQbnW364uJvE9qyIvdIybUEVObI8wM5BERmAy
         zh/g==
X-Gm-Message-State: AOAM5338EUjSgINvSEeJ38EHSH/ZnK2YPac68JNa+3Ptor0wrrMr2NBi
	yc28TedQjq/YHQLsXzedQgCUenNYtM3Bcsn4GhE=
X-Google-Smtp-Source: ABdhPJy3qil9pHk//YzKgBCdmYRr9vXF2WvPB4AvwKW/VBzOF3cQYTL8LpdSl6GW6VODwOTT7axdkrNAu6sm37ATyNc=
X-Received: by 2002:a05:6808:690:: with SMTP id k16mr4024290oig.152.1631038186568;
 Tue, 07 Sep 2021 11:09:46 -0700 (PDT)
MIME-Version: 1.0
References: <DB9PR08MB6857C656472153A42FB438C49EFE9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210818053840.GE39588@laputa> <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa> <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CACMJ4GbmNgbB5ponYt3NGEk3j6YCksot+kDy2qs8HMdFXWnQbw@mail.gmail.com>
 <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com>
 <20210902071902.GC71098@laputa> <CACMJ4GYY0wkm0vOzJfq8M9dCxPq1ZFwWT8Q73rMjqtrP7if-Zw@mail.gmail.com>
 <20210907115501.GC49004@laputa>
In-Reply-To: <20210907115501.GC49004@laputa>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Tue, 7 Sep 2021 11:09:34 -0700
Message-ID: <CACMJ4GYvndK6Cat-cZxM3rJ+6Ys2-hT=0_QivQ4md1MHAGbN3Q@mail.gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Wei Chen <Wei.Chen@arm.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, "Alex Benn??e" <alex.bennee@linaro.org>, 
	Kaly Xin <Kaly.Xin@arm.com>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	Viresh Kumar <viresh.kumar@linaro.org>, 
	Stefano Stabellini <stefano.stabellini@xilinx.com>, "stefanha@redhat.com" <stefanha@redhat.com>, 
	Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>, 
	"pratikp@quicinc.com" <pratikp@quicinc.com>, Srivatsa Vaddagiri <vatsa@codeaurora.org>, 
	Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>, 
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Bertrand Marquis <Bertrand.Marquis@arm.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>, 
	Paul Durrant <paul@xen.org>, Xen Devel <xen-devel@lists.xen.org>, Rich Persaud <persaur@gmail.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, James McKenzie <james@bromium.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000b9c09e05cb6bab7a"

--000000000000b9c09e05cb6bab7a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Sep 7, 2021 at 4:55 AM AKASHI Takahiro <takahiro.akashi@linaro.org>
wrote:

> Hi,
>
> I have not covered all your comments below yet.
> So just one comment:
>
> On Mon, Sep 06, 2021 at 05:57:43PM -0700, Christopher Clark wrote:
> > On Thu, Sep 2, 2021 at 12:19 AM AKASHI Takahiro <
> takahiro.akashi@linaro.org>
> > wrote:
>
> (snip)
>
> > >    It appears that, on FE side, at least three hypervisor calls (and
> data
> > >    copying) need to be invoked at every request, right?
> > >
> >
> > For a write, counting FE sendv ops:
> > 1: the write data payload is sent via the "Argo ring for writes"
> > 2: the descriptor is sent via a sync of the available/descriptor ring
> >   -- is there a third one that I am missing?
>
> In the picture, I can see
> a) Data transmitted by Argo sendv
> b) Descriptor written after data sendv
> c) VirtIO ring sync'd to back-end via separate sendv
>
> Oops, (b) is not a hypervisor call, is it?
>

That's correct, it is not - the blue arrows in the diagram are not
hypercalls, they are intended to show data movement or action in the flow
of performing the operation, and (b) is a data write within the guest's
address space into the descriptor ring.



> (But I guess that you will have to have yet another call for notification
> since there is no config register of QueueNotify?)
>

Reasoning about hypercalls necessary for data movement:

VirtIO transport drivers are responsible for instantiating virtqueues
(setup_vq) and are able to populate the notify function pointer in the
virtqueue that they supply. The virtio-argo transport driver can provide a
suitable notify function implementation that will issue the Argo hypercall
sendv hypercall(s) for sending data from the guest frontend to the backend.
By issuing the sendv at the time of the queuenotify, rather than as each
buffer is added to the virtqueue, the cost of the sendv hypercall can be
amortized over multiple buffer additions to the virtqueue.

I also understand that there has been some recent work in the Linaro
Project Stratos on "Fat Virtqueues", where the data to be transmitted is
included within an expanded virtqueue, which could further reduce the
number of hypercalls required, since the data can be transmitted inline
with the descriptors.
Reference here:
https://linaro.atlassian.net/wiki/spaces/STR/pages/25626313982/2021-01-21+P=
roject+Stratos+Sync+Meeting+notes
https://linaro.atlassian.net/browse/STR-25

As a result of the above, I think that a single hypercall could be
sufficient for communicating data for multiple requests, and that a
two-hypercall-per-request (worst case) upper bound could also be
established.

Christopher



>
> Thanks,
> -Takahiro Akashi
>
>
> > Christopher
> >
> >
> > >
> > > Thanks,
> > > -Takahiro Akashi
> > >
> > >
> > > > * Here are the design documents for building VirtIO-over-Argo, to
> > > support a
> > > >   hypervisor-agnostic frontend VirtIO transport driver using Argo.
> > > >
> > > > The Development Plan to build VirtIO virtual device support over Ar=
go
> > > > transport:
> > > >
> > >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1
> > > >
> > > > A design for using VirtIO over Argo, describing how VirtIO data
> > > structures
> > > > and communication is handled over the Argo transport:
> > > >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo
> > > >
> > > > Diagram (from the above document) showing how VirtIO rings are
> > > synchronized
> > > > between domains without using shared memory:
> > > >
> > >
> https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-b=
lob-url=3Dtrue&id=3D01f7d0e1-7686-4f0b-88e1-457c1d30df40&collection=3Dconte=
ntId-1348763698&contextId=3D1348763698&mimeType=3Dimage%2Fpng&name=3Ddevice=
-buffer-access-virtio-argo.png&size=3D243175&width=3D1106&height=3D1241
> > > >
> > > > Please note that the above design documents show that the existing
> VirtIO
> > > > device drivers, and both vring and virtqueue data structures can be
> > > > preserved
> > > > while interdomain communication can be performed with no shared
> memory
> > > > required
> > > > for most drivers; (the exceptions where further design is required
> are
> > > those
> > > > such as virtual framebuffer devices where shared memory regions are
> > > > intentionally
> > > > added to the communication structure beyond the vrings and
> virtqueues).
> > > >
> > > > An analysis of VirtIO and Argo, informing the design:
> > > >
> > >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+=
Argo+as+a+transport+medium+for+VirtIO
> > > >
> > > > * Argo can be used for a communication path for configuration
> between the
> > > > backend
> > > >   and the toolstack, avoiding the need for a dependency on XenStore=
,
> > > which
> > > > is an
> > > >   advantage for any hypervisor-agnostic design. It is also amenable
> to a
> > > > notification
> > > >   mechanism that is not based on Xen event channels.
> > > >
> > > > * Argo does not use or require shared memory between VMs and
> provides an
> > > > alternative
> > > >   to the use of foreign shared memory mappings. It avoids some of t=
he
> > > > complexities
> > > >   involved with using grants (eg. XSA-300).
> > > >
> > > > * Argo supports Mandatory Access Control by the hypervisor,
> satisfying a
> > > > common
> > > >   certification requirement.
> > > >
> > > > * The Argo headers are BSD-licensed and the Xen hypervisor
> implementation
> > > > is GPLv2 but
> > > >   accessible via the hypercall interface. The licensing should not
> > > present
> > > > an obstacle
> > > >   to adoption of Argo in guest software or implementation by other
> > > > hypervisors.
> > > >
> > > > * Since the interface that Argo presents to a guest VM is similar t=
o
> > > DMA, a
> > > > VirtIO-Argo
> > > >   frontend transport driver should be able to operate with a physic=
al
> > > > VirtIO-enabled
> > > >   smart-NIC if the toolstack and an Argo-aware backend provide
> support.
> > > >
> > > > The next Xen Community Call is next week and I would be happy to
> answer
> > > > questions
> > > > about Argo and on this topic. I will also be following this thread.
> > > >
> > > > Christopher
> > > > (Argo maintainer, Xen Community)
> > > >
> > > >
> > >
> -------------------------------------------------------------------------=
-------
> > > > [1]
> > > > An introduction to Argo:
> > > >
> > >
> https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-=
%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> > > > https://www.youtube.com/watch?v=3DcnC0Tg3jqJQ
> > > > Xen Wiki page for Argo:
> > > >
> > >
> https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)=
_for_Xen
> > > >
> > > > [2]
> > > > OpenXT Linux Argo driver and userspace library:
> > > > https://github.com/openxt/linux-xen-argo
> > > >
> > > > Windows V4V at OpenXT wiki:
> > > > https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> > > > Windows v4v driver source:
> > > > https://github.com/OpenXT/xc-windows/tree/master/xenv4v
> > > >
> > > > HP/Bromium uXen V4V driver:
> > > > https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
> > > >
> > > > [3]
> > > > v2 of the Argo test unikernel for XTF:
> > > >
> > >
> https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.htm=
l
> > > >
> > > > [4]
> > > > Argo HMX Transport for VirtIO meeting minutes:
> > > >
> > >
> https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.htm=
l
> > > >
> > > > VirtIO-Argo Development wiki page:
> > > >
> > >
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1
> > > >
> > >
> > >
>

--000000000000b9c09e05cb6bab7a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Tue, Sep 7, 2021 at 4:55 AM AKASHI=
 Takahiro &lt;<a href=3D"mailto:takahiro.akashi@linaro.org">takahiro.akashi=
@linaro.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">Hi,<br>
<br>
I have not covered all your comments below yet.<br>
So just one comment:<br>
<br>
On Mon, Sep 06, 2021 at 05:57:43PM -0700, Christopher Clark wrote:<br>
&gt; On Thu, Sep 2, 2021 at 12:19 AM AKASHI Takahiro &lt;<a href=3D"mailto:=
takahiro.akashi@linaro.org" target=3D"_blank">takahiro.akashi@linaro.org</a=
>&gt;<br>
&gt; wrote:<br>
<br>
(snip)<br>
<br>
&gt; &gt;=C2=A0 =C2=A0 It appears that, on FE side, at least three hypervis=
or calls (and data<br>
&gt; &gt;=C2=A0 =C2=A0 copying) need to be invoked at every request, right?=
<br>
&gt; &gt;<br>
&gt; <br>
&gt; For a write, counting FE sendv ops:<br>
&gt; 1: the write data payload is sent via the &quot;Argo ring for writes&q=
uot;<br>
&gt; 2: the descriptor is sent via a sync of the available/descriptor ring<=
br>
&gt;=C2=A0 =C2=A0-- is there a third one that I am missing?<br>
<br>
In the picture, I can see<br>
a) Data transmitted by Argo sendv<br>
b) Descriptor written after data sendv<br>
c) VirtIO ring sync&#39;d to back-end via separate sendv<br>
<br>
Oops, (b) is not a hypervisor call, is it?<br></blockquote><div><br></div><=
div>That&#39;s correct, it is not - the blue arrows in the diagram are not =
hypercalls, they are intended to show data movement or action in the flow o=
f performing the operation, and (b) is a data write within the guest&#39;s =
address space into the descriptor ring.</div><div><br></div><div>=C2=A0<br>=
</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b=
order-left:1px solid rgb(204,204,204);padding-left:1ex">
(But I guess that you will have to have yet another call for notification<b=
r>
since there is no config register of QueueNotify?)<br></blockquote><div><br=
></div><div>Reasoning about hypercalls necessary for data movement:</div><d=
iv><br></div><div>VirtIO transport drivers are responsible for instantiatin=
g virtqueues (setup_vq) and are able to populate the notify function pointe=
r in the virtqueue that they supply. The virtio-argo transport driver can p=
rovide a suitable notify function implementation that will issue the Argo h=
ypercall sendv hypercall(s) for sending data from the guest frontend to the=
 backend. By issuing the sendv at the time of the queuenotify, rather than =
as each buffer is added to the virtqueue, the cost of the sendv hypercall c=
an be amortized over multiple buffer additions to the virtqueue.</div><div>=
<br></div><div><div>I also understand that there has been some recent work =
in the Linaro Project Stratos on &quot;Fat Virtqueues&quot;, where the data=
 to be transmitted is included within an expanded virtqueue, which could fu=
rther reduce the number of hypercalls required, since the data can be trans=
mitted inline with the descriptors.</div><div>Reference here:<br></div><div=
><a href=3D"https://linaro.atlassian.net/wiki/spaces/STR/pages/25626313982/=
2021-01-21+Project+Stratos+Sync+Meeting+notes">https://linaro.atlassian.net=
/wiki/spaces/STR/pages/25626313982/2021-01-21+Project+Stratos+Sync+Meeting+=
notes</a><br></div><div><a href=3D"https://linaro.atlassian.net/browse/STR-=
25">https://linaro.atlassian.net/browse/STR-25</a><br></div><br class=3D"gm=
ail-Apple-interchange-newline"></div><div>As a result of the above, I think=
 that a single hypercall could be sufficient for communicating data for mul=
tiple requests, and that a two-hypercall-per-request (worst case) upper bou=
nd could also be established.</div><div><br></div><div>Christopher</div><di=
v><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">
<br>
Thanks,<br>
-Takahiro Akashi<br>
<br>
<br>
&gt; Christopher<br>
&gt; <br>
&gt; <br>
&gt; &gt;<br>
&gt; &gt; Thanks,<br>
&gt; &gt; -Takahiro Akashi<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; &gt; * Here are the design documents for building VirtIO-over-Arg=
o, to<br>
&gt; &gt; support a<br>
&gt; &gt; &gt;=C2=A0 =C2=A0hypervisor-agnostic frontend VirtIO transport dr=
iver using Argo.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; The Development Plan to build VirtIO virtual device support =
over Argo<br>
&gt; &gt; &gt; transport:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/1696=
169985/VirtIO-Argo+Development+Phase+1" rel=3D"noreferrer" target=3D"_blank=
">https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; A design for using VirtIO over Argo, describing how VirtIO d=
ata<br>
&gt; &gt; structures<br>
&gt; &gt; &gt; and communication is handled over the Argo transport:<br>
&gt; &gt; &gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages=
/1348763698/VirtIO+Argo" rel=3D"noreferrer" target=3D"_blank">https://openx=
t.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Diagram (from the above document) showing how VirtIO rings a=
re<br>
&gt; &gt; synchronized<br>
&gt; &gt; &gt; between domains without using shared memory:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-a=
bd4377a1194#media-blob-url=3Dtrue&amp;id=3D01f7d0e1-7686-4f0b-88e1-457c1d30=
df40&amp;collection=3DcontentId-1348763698&amp;contextId=3D1348763698&amp;m=
imeType=3Dimage%2Fpng&amp;name=3Ddevice-buffer-access-virtio-argo.png&amp;s=
ize=3D243175&amp;width=3D1106&amp;height=3D1241" rel=3D"noreferrer" target=
=3D"_blank">https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a11=
94#media-blob-url=3Dtrue&amp;id=3D01f7d0e1-7686-4f0b-88e1-457c1d30df40&amp;=
collection=3DcontentId-1348763698&amp;contextId=3D1348763698&amp;mimeType=
=3Dimage%2Fpng&amp;name=3Ddevice-buffer-access-virtio-argo.png&amp;size=3D2=
43175&amp;width=3D1106&amp;height=3D1241</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Please note that the above design documents show that the ex=
isting VirtIO<br>
&gt; &gt; &gt; device drivers, and both vring and virtqueue data structures=
 can be<br>
&gt; &gt; &gt; preserved<br>
&gt; &gt; &gt; while interdomain communication can be performed with no sha=
red memory<br>
&gt; &gt; &gt; required<br>
&gt; &gt; &gt; for most drivers; (the exceptions where further design is re=
quired are<br>
&gt; &gt; those<br>
&gt; &gt; &gt; such as virtual framebuffer devices where shared memory regi=
ons are<br>
&gt; &gt; &gt; intentionally<br>
&gt; &gt; &gt; added to the communication structure beyond the vrings and v=
irtqueues).<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; An analysis of VirtIO and Argo, informing the design:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/1333=
428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO" rel=3D"noreferrer=
" target=3D"_blank">https://openxt.atlassian.net/wiki/spaces/DC/pages/13334=
28225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; * Argo can be used for a communication path for configuratio=
n between the<br>
&gt; &gt; &gt; backend<br>
&gt; &gt; &gt;=C2=A0 =C2=A0and the toolstack, avoiding the need for a depen=
dency on XenStore,<br>
&gt; &gt; which<br>
&gt; &gt; &gt; is an<br>
&gt; &gt; &gt;=C2=A0 =C2=A0advantage for any hypervisor-agnostic design. It=
 is also amenable to a<br>
&gt; &gt; &gt; notification<br>
&gt; &gt; &gt;=C2=A0 =C2=A0mechanism that is not based on Xen event channel=
s.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; * Argo does not use or require shared memory between VMs and=
 provides an<br>
&gt; &gt; &gt; alternative<br>
&gt; &gt; &gt;=C2=A0 =C2=A0to the use of foreign shared memory mappings. It=
 avoids some of the<br>
&gt; &gt; &gt; complexities<br>
&gt; &gt; &gt;=C2=A0 =C2=A0involved with using grants (eg. XSA-300).<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; * Argo supports Mandatory Access Control by the hypervisor, =
satisfying a<br>
&gt; &gt; &gt; common<br>
&gt; &gt; &gt;=C2=A0 =C2=A0certification requirement.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; * The Argo headers are BSD-licensed and the Xen hypervisor i=
mplementation<br>
&gt; &gt; &gt; is GPLv2 but<br>
&gt; &gt; &gt;=C2=A0 =C2=A0accessible via the hypercall interface. The lice=
nsing should not<br>
&gt; &gt; present<br>
&gt; &gt; &gt; an obstacle<br>
&gt; &gt; &gt;=C2=A0 =C2=A0to adoption of Argo in guest software or impleme=
ntation by other<br>
&gt; &gt; &gt; hypervisors.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; * Since the interface that Argo presents to a guest VM is si=
milar to<br>
&gt; &gt; DMA, a<br>
&gt; &gt; &gt; VirtIO-Argo<br>
&gt; &gt; &gt;=C2=A0 =C2=A0frontend transport driver should be able to oper=
ate with a physical<br>
&gt; &gt; &gt; VirtIO-enabled<br>
&gt; &gt; &gt;=C2=A0 =C2=A0smart-NIC if the toolstack and an Argo-aware bac=
kend provide support.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; The next Xen Community Call is next week and I would be happ=
y to answer<br>
&gt; &gt; &gt; questions<br>
&gt; &gt; &gt; about Argo and on this topic. I will also be following this =
thread.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Christopher<br>
&gt; &gt; &gt; (Argo maintainer, Xen Community)<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; -----------------------------------------------------------------=
---------------<br>
&gt; &gt; &gt; [1]<br>
&gt; &gt; &gt; An introduction to Argo:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://static.sched.com/hosted_files/xensummit19/92/A=
rgo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%2=
02019.pdf" rel=3D"noreferrer" target=3D"_blank">https://static.sched.com/ho=
sted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%2=
0Clark%20-%20Xen%20Summit%202019.pdf</a><br>
&gt; &gt; &gt; <a href=3D"https://www.youtube.com/watch?v=3DcnC0Tg3jqJQ" re=
l=3D"noreferrer" target=3D"_blank">https://www.youtube.com/watch?v=3DcnC0Tg=
3jqJQ</a><br>
&gt; &gt; &gt; Xen Wiki page for Argo:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Medi=
ated_Exchange_(HMX)_for_Xen" rel=3D"noreferrer" target=3D"_blank">https://w=
iki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen</a=
><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; [2]<br>
&gt; &gt; &gt; OpenXT Linux Argo driver and userspace library:<br>
&gt; &gt; &gt; <a href=3D"https://github.com/openxt/linux-xen-argo" rel=3D"=
noreferrer" target=3D"_blank">https://github.com/openxt/linux-xen-argo</a><=
br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Windows V4V at OpenXT wiki:<br>
&gt; &gt; &gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages=
/14844007/V4V" rel=3D"noreferrer" target=3D"_blank">https://openxt.atlassia=
n.net/wiki/spaces/DC/pages/14844007/V4V</a><br>
&gt; &gt; &gt; Windows v4v driver source:<br>
&gt; &gt; &gt; <a href=3D"https://github.com/OpenXT/xc-windows/tree/master/=
xenv4v" rel=3D"noreferrer" target=3D"_blank">https://github.com/OpenXT/xc-w=
indows/tree/master/xenv4v</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; HP/Bromium uXen V4V driver:<br>
&gt; &gt; &gt; <a href=3D"https://github.com/uxen-virt/uxen/tree/ascara/win=
dows/uxenv4vlib" rel=3D"noreferrer" target=3D"_blank">https://github.com/ux=
en-virt/uxen/tree/ascara/windows/uxenv4vlib</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; [3]<br>
&gt; &gt; &gt; v2 of the Argo test unikernel for XTF:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-devel/2=
021-01/msg02234.html" rel=3D"noreferrer" target=3D"_blank">https://lists.xe=
nproject.org/archives/html/xen-devel/2021-01/msg02234.html</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; [4]<br>
&gt; &gt; &gt; Argo HMX Transport for VirtIO meeting minutes:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-devel/2=
021-02/msg01422.html" rel=3D"noreferrer" target=3D"_blank">https://lists.xe=
nproject.org/archives/html/xen-devel/2021-02/msg01422.html</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; VirtIO-Argo Development wiki page:<br>
&gt; &gt; &gt;<br>
&gt; &gt; <a href=3D"https://openxt.atlassian.net/wiki/spaces/DC/pages/1696=
169985/VirtIO-Argo+Development+Phase+1" rel=3D"noreferrer" target=3D"_blank=
">https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+=
Development+Phase+1</a><br>
&gt; &gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
</blockquote></div></div>

--000000000000b9c09e05cb6bab7a--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 19:53:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 19:53:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181312.328404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNh9N-0000sI-3l; Tue, 07 Sep 2021 19:52:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181312.328404; Tue, 07 Sep 2021 19:52:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNh9M-0000sB-Vg; Tue, 07 Sep 2021 19:52:56 +0000
Received: by outflank-mailman (input) for mailman id 181312;
 Tue, 07 Sep 2021 19:52:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g753=N5=eikelenboom.it=linux@srs-us1.protection.inumbo.net>)
 id 1mNh9L-0000s5-TG
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 19:52:56 +0000
Received: from server.eikelenboom.it (unknown [91.121.65.215])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3051de83-1015-11ec-b121-12813bfff9fa;
 Tue, 07 Sep 2021 19:52:54 +0000 (UTC)
Received: from 76-24-144-85.ftth.glasoperator.nl ([85.144.24.76]:40046
 helo=[172.16.1.50]) by server.eikelenboom.it with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <linux@eikelenboom.it>)
 id 1mNh99-0000xc-7f; Tue, 07 Sep 2021 21:52:43 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3051de83-1015-11ec-b121-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=eikelenboom.it; s=20180706; h=Content-Transfer-Encoding:Content-Type:
	In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject:Sender:
	Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From:
	Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:
	List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=trC0etVSAP2kxLlL5V959uspM4Sc5FjGHeuAcplcm2s=; b=ALpw8XegMQbrkpYL4zIl8scRe2
	6XREaewve79+K+LHR+y3Mj01MyAfYv9yhV8KzAwlcygO/zC9U+xlglFnDPFgC8e0Iw2sSuuEEpTIm
	jg4SfhgBGLAiWgC42QlkK3fOjZATF0MW2bOaMFON/7Fa6CiqZbO+jK5l3BPe/h8/gQSQ=;
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
To: Juergen Gross <jgross@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
 <955da8ce-40f2-32c5-ee78-6801ce18afe5@suse.com>
From: Sander Eikelenboom <linux@eikelenboom.it>
Message-ID: <b0854555-01fd-cbd8-6e6a-c793109b2972@eikelenboom.it>
Date: Tue, 7 Sep 2021 21:52:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <955da8ce-40f2-32c5-ee78-6801ce18afe5@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 07/09/2021 15:53, Juergen Gross wrote:
> On 06.09.21 23:35, Sander Eikelenboom wrote:
>> L.S.,
>>
>> On my AMD box running:
>>       xen-unstable changeset: Fri Sep 3 15:10:43 2021 +0200 git:2d4978ead4
>>       linux kernel: 5.14.1
>>
>> With this setup I'm encountering some issues in dom0, see below.
> 
> Could you test whether the attached patch (only compile tested)
> fixes the issue for you?
> 
> 
> Juergen
> 

Sure, running it now.
Since I haven't got a clear trigger, will report tomorrow if the logs stay clean :)

Thanks so far !

--
Sander


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 20:53:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 20:53:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181347.328415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNi5D-0007nF-LI; Tue, 07 Sep 2021 20:52:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181347.328415; Tue, 07 Sep 2021 20:52:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNi5D-0007n8-I7; Tue, 07 Sep 2021 20:52:43 +0000
Received: by outflank-mailman (input) for mailman id 181347;
 Tue, 07 Sep 2021 20:52:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNi5C-0007my-0e; Tue, 07 Sep 2021 20:52:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNi5B-0004tT-N8; Tue, 07 Sep 2021 20:52:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNi5B-0000vW-6A; Tue, 07 Sep 2021 20:52:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNi5B-0005wE-5h; Tue, 07 Sep 2021 20:52:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TJa5hmGyOGfHEV4UaYO8IKURO8WopCgVJdKU7xNNWTk=; b=OcCzJnkTQHVURKUzjE+jW01IWE
	H8tf5sX8eK8ecDKjQFEYv5gwNjjOvkiV1nD0Ui9fHXK+q26CbU/OWBIE9TDpr86HqCC17YlGnIP9h
	svOJbDIIhL6fKrKBIJiVtK1t6C/0QCtUUZV00xhfQHbUYQrIhyD7cTByUJqb5o8eyCD4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164873-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164873: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
X-Osstest-Versions-That:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 20:52:41 +0000

flight 164873 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164873/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164866
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164866
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164866
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164866
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164866
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164866
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164866
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164866
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164866
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164866
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164866
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
baseline version:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17

Last test of basis   164873  2021-09-07 11:09:27 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 21:28:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 21:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181356.328429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNidZ-0003Bh-MN; Tue, 07 Sep 2021 21:28:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181356.328429; Tue, 07 Sep 2021 21:28:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNidZ-0003Ba-If; Tue, 07 Sep 2021 21:28:13 +0000
Received: by outflank-mailman (input) for mailman id 181356;
 Tue, 07 Sep 2021 21:28:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNidY-0003BU-H9
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 21:28:12 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0a400bb9-9177-4b0f-82b0-8f51a1c42618;
 Tue, 07 Sep 2021 21:28:11 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id t19so19813lfe.13
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 14:28:11 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id k10sm10473lfo.57.2021.09.07.14.28.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 Sep 2021 14:28:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a400bb9-9177-4b0f-82b0-8f51a1c42618
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=rCoPbmqBwB8B6I+y0U69hfFC7ri8VBmXepajnA2jkcI=;
        b=GawO2P5zMyZpdXRAbTM0ddAxKXijkkN9Jx5LugaBUviT+HxZvxx6ZKDVOj9LlxziWh
         QHPns/U+Rk4YQ6hfCqfnVUzye95HEiIfOcKF7xHKsj6EjHkRmfeU8vUxgN3O3wylY4Am
         xi4kJJk+p6t7hPoHz1cKytAR9AyLckoLKsVeWORVzhkhNKXhcl58HLw58QnfX5iYdLtu
         HnbLqBPURowicpYM4/CWrBdQNAggrlzSWfIbWpsPJX/YgSIxs8xVUMFIs6Unr6MlhJU1
         dk+ecjgpgFSE387Vkq7E0He7fb62laxZsKvA0oAdqII2JoosEpc+ZyQt003AV7SH6QY4
         Dngg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=rCoPbmqBwB8B6I+y0U69hfFC7ri8VBmXepajnA2jkcI=;
        b=ruE+f628Bab5cADQ/+NBEq3h6UhEVHGsLVEYGW8SMZpcUTmyTqr3H3femHh4eqBt5/
         LK09FKqs5eKR6UBhL/R8P4CkIh3oWBSyyU1qfX60Nrt//TGXlpx50u+JEGuThE0aw/qh
         dNL6/TQASDWPczsC7bHyMiELWswU164HdSlr44mRFsQ7mV2TfMBL7v3fztctWbxVz597
         JSIFfZIOCCwoohBkBAY+yXBaFzLSXDN6hzqS8AlCIFfxukDfsdZD0UHzYeMxPNpOG2N9
         0UiNNkkDWnivMrwroLu3bOiaYE23mHthOPAD4Jaa2ratWsvStjHKrPwGXEDJqpvIWW0Y
         NPoQ==
X-Gm-Message-State: AOAM533NLo+40lQF6MIikmx3TDvy6jGI+ksF1F3PHYuOdu/k61G2BQcU
	lahQxxhhiMZ+Mg1HUD6lpMg=
X-Google-Smtp-Source: ABdhPJxXV9/nB6x2dRD0ZzUdN29UvvrG5mslv6tFU2OOS4Dhy/0KZj3CI6rKj1YVVrNXmsBp9TTj2A==
X-Received: by 2002:a05:6512:21a8:: with SMTP id c8mr321662lft.514.1631050090020;
        Tue, 07 Sep 2021 14:28:10 -0700 (PDT)
Subject: Re: [RFC PATCH 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 Henry Wang <Henry.Wang@arm.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
 <1631034578-12598-2-git-send-email-olekstysh@gmail.com>
 <973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <0994251e-5e02-009a-851d-4e4ee760fc66@gmail.com>
Date: Wed, 8 Sep 2021 00:28:08 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 07.09.21 20:35, Andrew Cooper wrote:

Hi Andrew

> On 07/09/2021 18:09, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> We need to pass info about maximum supported guest address
>> space size to the toolstack on Arm in order to properly
>> calculate the base and size of the extended region (safe range)
>> for the guest. The extended region is unused address space which
>> could be safely used by domain for foreign/grant mappings on Arm.
>> The extended region itself will be handled by the subsequents
>> patch.
>>
>> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
>> hap_paddr_bits.
>>
>> As we change the size of structure bump the interface version.
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> So while I think this is a suitable way forward, you're painting
> yourself into a corner WRT migration.
>
> On x86, the correct value is d->arch.cpuid->extd.maxphysaddr and this
> value is under toolstack control, not Xen control.  In particular, it
> needs to be min(hostA, hostB) to make migration safe, and yes - this is
> currently a hole in x86's migration logic that will cause large VMs to
> explode.
>
> The same will be true on ARM as/when you gain migration support.

Oh, I admit, I didn't keep in mind migration support while creating this 
patch.


>
> I think this would be better as a domctl.  On ARM, it can reference
> p2m_ipa_bits for now along with a /* TODO - make per-domain for
> migration support */, while on x86 it can take the appropriate value
> (which will soon actually be safe in migration scenarios).

OK, could you please clarify, did you mean to introduce new domctl 
(something like get_maxphysaddr) or reuse some existing?

And ...

>
> However, that does change the ordering requirements in the toolstack -
> this hypercall would need to be made after the domain is created, and
> has been levelled, and before its main memory layout is decided.

... I am not sure I totally understand the ordering requirements in the 
toolstack.

On Arm this information (gpaddr_bits) should be obtained by the time we 
call libxl__arch_domain_finalise_hw_description()
where we actually calculate extended region and insert it in domain's 
device-tree. At that time, the domain memory is already populated, so 
it's layout is known.
Please see the last patch of this series, which demonstrates the usage:

https://lore.kernel.org/xen-devel/1631034578-12598-4-git-send-email-olekstysh@gmail.com/


> Alternatively, the abstraction could be hidden in libxl itself in arch
> specific code, with x86 referring to the local cpu policy (as libxl has
> the copy it is/has worked on), and ARM making a hypercall.  This does
> make the ordering more obvious.

May I please ask what would be the preferred option to move forward? I 
am fine with both proposed options, with a little preference for the 
former (common domctl, abstraction is hidden in Xen itself in arch 
specific code). It is highly appreciated if we could choose the option 
which would satisfy all parties, this would save me some time.

> (As a note on the x86 specifics of this patch, hap_paddr_bits is
> actually unused in practice.  It was a proposal from AMD for the
> hypervisor to fill in those details, which wasn't implemented by anyone,
> not even Xen, because the important field to modify is maxphysaddr if
> you don't want to rewrite every kernel to behave differently when
> virtualised.)

Thank you for the clarification.


>
> ~Andrew
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 21:34:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 21:34:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181364.328439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNijn-0004l9-CY; Tue, 07 Sep 2021 21:34:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181364.328439; Tue, 07 Sep 2021 21:34:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNijn-0004l2-9d; Tue, 07 Sep 2021 21:34:39 +0000
Received: by outflank-mailman (input) for mailman id 181364;
 Tue, 07 Sep 2021 21:34:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NjNB=N5=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mNijl-0004kw-To
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 21:34:37 +0000
Received: from mail-lj1-x231.google.com (unknown [2a00:1450:4864:20::231])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c141cd11-d8c8-4d3a-adaf-8e5ec596e468;
 Tue, 07 Sep 2021 21:34:36 +0000 (UTC)
Received: by mail-lj1-x231.google.com with SMTP id y6so848208lje.2
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 14:34:36 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id g10sm11246lfb.87.2021.09.07.14.34.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 Sep 2021 14:34:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c141cd11-d8c8-4d3a-adaf-8e5ec596e468
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=X3YkShdpCjM0j8ZJigoaLHyM/keq1t8ObtbGWzImwWs=;
        b=V7Z2S2HGFliTqXR5z+T5SKrmTBIfsYQA/s7U71bLMtmcWh0w18IZXZzNwo0ebUiRxk
         XgeZNNf7lFV4m1cCZ/zo691i89CXPU2ELG1qlRtwHVUiud+rMuibDSUsqAjZqehzC3ig
         xg2cGzo1yAwq6NGWwO4LNWEkdgpoGsvw3OD0ymhb5vWvbTNWNStnTV2YTG/B0Xpv/uce
         hjqiQ3nzhxNnTKBpK83GrNZoXscSZ5Qq/iBj/8BQZcF7C+urcDpxVhMidzOlZfvDOFev
         vilsdOgVU8a0ZZHFFbMr3TJCjSLm//rtDFpNLMQxHwOQV03o36fUR+uvESynoOvKfGQi
         47lg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=X3YkShdpCjM0j8ZJigoaLHyM/keq1t8ObtbGWzImwWs=;
        b=Zm7P3cK00zBmm1b5ilRBP24oFY8ONANMJzbg3njQ3dMx0BozKnODTUTLz6PQ0YqJwq
         cZJ3YQDBwuouLq3RcMQo2IX72Qc1SfZbA0WoCK3JNL4wdHhGAm3jCkgk8td7vnhfiLc7
         E9TqWVVFDeBwFO/r7FGj2EYxhe9yyivmQkKyG7DmXZHeX8BmgO8t9E/TA3IesHVgWzAo
         jm/XdpmOrq10jlBbuOgU+rU96HyUbmzn6SKsxBiW3oTOKj5C7a6DEFUTQnZsDTywh3Ba
         YqHZ+OmKXA7I5VK8xfajHxfM2GJr/ljtSf39LM9G6uPA32P+w+I46sDeWBMxC/Iglt/O
         Q2Gw==
X-Gm-Message-State: AOAM532o0Lcqv+Hzf7XhTB587SLSbpGC+wc6By2WOtNYvM442W8WnFXI
	21AxJ6LEvZeiDYJmrodZGd8=
X-Google-Smtp-Source: ABdhPJwLmj7DVmaZk+g6qRV0XlrRzgLrUEGqmkKXpjdOLi1nQnMrPUtIUmK6+j1KR5qFqcM9O4F1uA==
X-Received: by 2002:a2e:90ca:: with SMTP id o10mr232555ljg.67.1631050475852;
        Tue, 07 Sep 2021 14:34:35 -0700 (PDT)
Subject: Re: [RFC PATCH] xen/memory: Introduce a hypercall to provide
 unallocated space
To: Henry Wang <Henry.Wang@arm.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1627489110-25633-1-git-send-email-olekstysh@gmail.com>
 <PA4PR08MB6253F59F4C63FFE50CC6ADBC92D39@PA4PR08MB6253.eurprd08.prod.outlook.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <b420b0ab-588b-f1c3-cf5b-edfefe67ba08@gmail.com>
Date: Wed, 8 Sep 2021 00:34:34 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <PA4PR08MB6253F59F4C63FFE50CC6ADBC92D39@PA4PR08MB6253.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 07.09.21 11:53, Henry Wang wrote:
> Hi Oleksandr,

Hi Henry


>
>> -----Original Message-----
>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
>> Oleksandr Tyshchenko
>> Sent: Thursday, July 29, 2021 12:19 AM
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Daniel De
>> Graaf <dgdegra@tycho.nsa.gov>; Daniel P. Smith
>> <dpsmith@apertussolutions.com>; Ian Jackson <iwj@xenproject.org>; Wei
>> Liu <wl@xen.org>; Andrew Cooper <andrew.cooper3@citrix.com>; George
>> Dunlap <george.dunlap@citrix.com>; Jan Beulich <jbeulich@suse.com>;
>> Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>;
>> Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Roger Pau Monné
>> <roger.pau@citrix.com>; Bertrand Marquis <Bertrand.Marquis@arm.com>;
>> Wei Chen <Wei.Chen@arm.com>
>> Subject: [RFC PATCH] xen/memory: Introduce a hypercall to provide
>> unallocated space
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> Add XENMEM_get_unallocated_space hypercall which purpose is to
>> query hypervisor to find regions of guest physical address space
>> which are unused and can be used to create grant/foreign mappings
>> instead of wasting real pages from the domain memory for
>> establishing these mappings. The problem with the current Linux
>> on Xen on Arm behaviour is if we want to map some guest memory
>> regions in advance or to perform cache mappings in the backend
>> we might run out of memory in the host (see XSA-300).
>> This of course, depends on the both host and guest memory sizes.
>>
>> The "unallocated space" can't be figured out precisely by
>> the domain on Arm without hypervisor involvement:
>> - not all device I/O regions are known by the time domain starts
>>    creating grant/foreign mappings
>> - the Dom0 is not aware of memory regions used for the identity
>>    mappings needed for the PV drivers to work
>> In both cases we might end up re-using these regions by
>> a mistake. So, the hypervisor which maintains the P2M for the domain
>> is in the best position to provide "unallocated space".
>>
>> The arch code is in charge of finding these regions and filling
>> in corresponding array in new helper arch_get_unallocated_space().
>>
>> This patch implements common and Arm parts, the x86 specific bits
>> are left uniplemented for now.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Tested-by: Henry Wang <Henry.Wang@arm.com>

Thank you!


Please note, I have pushed new version (which is notably different) 
based on the discussion in current thread:

https://lore.kernel.org/xen-devel/1631034578-12598-1-git-send-email-olekstysh@gmail.com/



>
> Kind regards,
>
> Henry
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 07 21:40:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 21:40:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181370.328451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNipe-00069N-2S; Tue, 07 Sep 2021 21:40:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181370.328451; Tue, 07 Sep 2021 21:40:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNipd-00069G-Vf; Tue, 07 Sep 2021 21:40:41 +0000
Received: by outflank-mailman (input) for mailman id 181370;
 Tue, 07 Sep 2021 21:40:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zymF=N5=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mNipc-00069A-N9
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 21:40:40 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c0f389cb-a8c4-4acf-b99a-78afcad85853;
 Tue, 07 Sep 2021 21:40:39 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 187LeWHK073357
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 7 Sep 2021 17:40:37 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 187LeW1h073356;
 Tue, 7 Sep 2021 14:40:32 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0f389cb-a8c4-4acf-b99a-78afcad85853
Date: Tue, 7 Sep 2021 14:40:31 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YTfcTztxXdGmM2Ym@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Tue, Sep 07, 2021 at 05:57:10PM +0200, Jan Beulich wrote:
> On 07.09.2021 17:03, Elliott Mitchell wrote:
> > On Tue, Sep 07, 2021 at 10:03:51AM +0200, Jan Beulich wrote:
> >>
> >> That sounds odd at the first glance - PVH simply requires that there be
> >> an (enabled) IOMMU. Hence the only thing I could imagine is that Xen
> >> doesn't enable the IOMMU in the first place for some reason.
> > 
> > Doesn't seem that odd to me.  I don't know the differences between the
> > first and second versions of the AMD IOMMU, but could well be v1 was
> > judged not to have enough functionality to bother with.
> > 
> > What this does make me wonder is, how much testing was done on systems
> > with functioning NPT, but disabled IOMMU?
> 
> No idea. During development is may happen (rarely) that one disables
> the IOMMU on purpose. Beyond that - can't tell.

Thus this processor having an early and not too capable IOMMU seems
worthy of note.

> >  Could be this system is in an
> > intergenerational hole, and some spot in the PVH/HVM code makes an
> > assumption of the presence of NPT guarantees presence of an operational
> > IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
> > code, some portion of the IOMMU code might be checking for presence of
> > NPT instead of presence of IOMMU.
> 
> This is all very speculative; I consider what you suspect not very likely,
> but also not entirely impossible. This is not the least because for a
> long time we've been running without shared page tables on AMD.
> 
> I'm afraid without technical data and without knowing how to repro, I
> don't see a way forward here.

I cannot report things which do not exist.  This occurs very quickly and
no warning or error messages have ever been observed on the main console
(VGA).

Happens during user domain kernel boot.  The configuration:
builder = "hvm"
name = "kr45h"
memory = 1024
maxmem = 16384
vcpus = 2
vif = [ '' ]
disk = [ 'vdev=sdc,format=raw,access=r,devtype=cdrom,target=/tmp/boot.iso', ]
sdl = 1

has been tested and confirmed to reproduce.  Looks like this was last
examined with a FreeBSD 12.2 AMD64 ISO, but Linux ISOs (un)happily work
too.  It is less than 40 seconds from `xl create` to indications of
hardware boot process starting.

Since there don't appear to be too many reports, the one factor which
now stands out is this machine has an AMD processor.  Xen confirms
presence of NPT support, but reports "I/O virtualisation disabled"
(older, less capable IOMMU).


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Tue Sep 07 23:17:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 23:17:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181386.328473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkLV-0008N0-GN; Tue, 07 Sep 2021 23:17:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181386.328473; Tue, 07 Sep 2021 23:17:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkLV-0008Mt-DD; Tue, 07 Sep 2021 23:17:41 +0000
Received: by outflank-mailman (input) for mailman id 181386;
 Tue, 07 Sep 2021 23:17:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iDhR=N5=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mNkLT-0008MB-SU
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 23:17:39 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id caacbc92-1031-11ec-b132-12813bfff9fa;
 Tue, 07 Sep 2021 23:17:38 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B8F1B60F14;
 Tue,  7 Sep 2021 23:17:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: caacbc92-1031-11ec-b132-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631056657;
	bh=XA9IMkep5CyFBW7EaXh7yuXdS92Q2DxXSqumrb+GRfg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=prsFy9ld4u6dcUI5qwFjPyEYdvofujezsY1Tbg7zSxFIpBKmFB/WPgxIhDpUSquKT
	 f1O1/chMuzqiB3UH9x0WmUeDqtscIB1+2zDWKReVKk0bbGGdt7mbf2z3XwwFyw4dvR
	 CkNuESdDmAvzrhGrX9dfYFVr9p1Pk3NXCRUBCwSndEThCN6vcZ33pKHuvw6Wi4qyEU
	 kyTm+l16HtEx6taf8vWgQm/xw8XvoVLgsQ0QboMGyjFBIe5qAt2wCoGwn0iRXNpk5j
	 T6vOPWDSTi7Ieq2m8eQe2rL1mSd8JlimJXGLcaKeihXNhHl/pbYFGTkNi75TxySDA7
	 1AWB1402BJrbw==
Date: Tue, 7 Sep 2021 16:17:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 7/7] xen/arm: Sanitize CTR_EL0 and emulate it if
 needed
In-Reply-To: <F675C327-DA08-478D-ADA8-51D9DA63DE6D@arm.com>
Message-ID: <alpine.DEB.2.21.2109071616320.14059@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <a7482653a0c29d68854846f25553f4938a6279b9.1629897306.git.bertrand.marquis@arm.com> <14b1639e-c9e4-f8fb-e921-73d7f28681ac@xen.org> <98E8FB9D-3372-47D2-92F7-8AD401AC1092@arm.com>
 <6705127a-bde2-36cd-0fed-900eb1eace03@xen.org> <CA9E8DFA-E0D8-4C33-A277-E19EFFCAFDC4@arm.com> <alpine.DEB.2.21.2109031540470.26072@sstabellini-ThinkPad-T480s> <D0CCB87F-1CC1-450B-A612-E4B505B36B5B@arm.com> <7fc6e6b6-464a-a305-2693-42cba40ad869@xen.org>
 <F675C327-DA08-478D-ADA8-51D9DA63DE6D@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1027929738-1631056639=:14059"
Content-ID: <alpine.DEB.2.21.2109071617300.14059@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1027929738-1631056639=:14059
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109071617301.14059@sstabellini-ThinkPad-T480s>

On Tue, 7 Sep 2021, Bertrand Marquis wrote:
> Hi Julien,
> 
> > On 6 Sep 2021, at 18:36, Julien Grall <julien@xen.org> wrote:
> > 
> > Hi Bertrand,
> > 
> > On 06/09/2021 09:29, Bertrand Marquis wrote:
> >>> On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>> 
> >>> On Tue, 31 Aug 2021, Bertrand Marquis wrote:
> >>>> Hi Julien,
> >>>> 
> >>>>> On 31 Aug 2021, at 15:47, Julien Grall <julien@xen.org> wrote:
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> On 31/08/2021 14:17, Bertrand Marquis wrote:
> >>>>>> Hi Julien,
> >>>>> 
> >>>>> Hi Bertrand,
> >>>>> 
> >>>>>>> On 27 Aug 2021, at 16:05, Julien Grall <julien@xen.org> wrote:
> >>>>>>> 
> >>>>>>> Hi Bertrand,
> >>>>>>> 
> >>>>>>> On 25/08/2021 14:18, Bertrand Marquis wrote:
> >>>>>>>> Sanitize CTR_EL0 value between cores.
> >>>>>>>> In most cases different values will taint Xen but if different
> >>>>>>>> i-cache policies are found, we choose the one which will be compatible
> >>>>>>>> between all cores in terms of invalidation/data cache flushing strategy.
> >>>>>>> 
> >>>>>>> I understand that all the CPUs in Xen needs to agree on the cache flush strategy. However...
> >>>>>>> 
> >>>>>>>> In this case we need to activate the TID2 bit in HCR to emulate the
> >>>>>>>> TCR_EL0 register for guests. This patch is not activating TID2 bit all
> >>>>>>>> the time to limit the overhead when possible.
> >>>>>>> 
> >>>>>>> as we discussed in an earlier version, a vCPU is unlikely (at least in short/medium) to be able move across pCPU of different type. So the vCPU would be pinned to a set of pCPUs. IOW, the guest would have to be big.LITTLE aware and therefore would be able to do its own strategy decision.
> >>>>>>> 
> >>>>>>> So I think we should be able to get away from trappings the registers.
> >>>>>> I do agree that we should be able to get away from that in the long term once
> >>>>>> we have cpupools properly set but right now this is the only way to have
> >>>>>> something useable (I will not say right).
> >>>>>> I will work on finding a way to setup properly cpupools (or something else as
> >>>>>> we discussed earlier) but in the short term I think this is the best we can do.
> >>>>> 
> >>>>> My concern is you are making look like Xen will be able to deal nicely with big.LITTLE when in fact there are a lot more potential issue by allow a vCPU moving accross pCPU of different type (the errata is one example).
> >>>> 
> >>>> I agree and this is why Xen is tainted.
> >>>> 
> >>>>> 
> >>>>>> An other solution would be to discard this patch from the serie for now until
> >>>>>> I have worked a proper solution for this case.
> >>>>>> Should we discard or merge or do you have an other idea ?
> >>>>> Please correct me if I am wrong, at the moment, it doesn't look like this patch will be part of the longer plan. If so, then I think it should be parked for now.
> >>>> 
> >>>> Not sure it depends on what the final solution would be but this is highly possible I agree.
> >>>> 
> >>>>> 
> >>>>> This would also have the advantage to avoid spending too much time on resolving the emulation issue I mentioned in my previous answer.
> >>>>> 
> >>>>> No need to resend a new version of this series yet. You can wait until the rest of the series get more feedback.
> >>>> 
> >>>> Ok, I will wait for feedback and next serie will not include this patch anymore.
> >>> 
> >>> Would it be worth keeping just the part that sanitizes ctr, without any
> >>> of the emulation stuff? That way we could still detect cases where there
> >>> is a mismatch between CPUs, print a useful message and taint Xen.
> >> That’s a good idea, it means removing the emulation part and just keep the sanitization.
> >> @Julien: would you be ok with that ?
> > 
> > I actually thought about suggesting it before Stefano did it. So I am OK with that.
> > 
> >> Should I send a v4 or should we use Stefano’s patch directly instead ?
> > 
> > I would suggest to send a v4. This needs a signed-off-by from Stefano and a new commit message.
> 
> Ok I will do that beginning of next week.

Of course you can add my signed-off-by and even my reviewed-by (although
it will look weird as it seems I reviewed my own patch)
--8323329-1027929738-1631056639=:14059--


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 23:17:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 23:17:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181385.328461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkLN-000856-8b; Tue, 07 Sep 2021 23:17:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181385.328461; Tue, 07 Sep 2021 23:17:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkLN-00084z-5M; Tue, 07 Sep 2021 23:17:33 +0000
Received: by outflank-mailman (input) for mailman id 181385;
 Tue, 07 Sep 2021 23:17:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iDhR=N5=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mNkLL-00084t-CG
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 23:17:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1aa5a533-8819-49de-b45c-841c709c772c;
 Tue, 07 Sep 2021 23:17:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3582160F14;
 Tue,  7 Sep 2021 23:17:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1aa5a533-8819-49de-b45c-841c709c772c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631056649;
	bh=YjPWq2Wq+vSUdQyVVhKv+XleADcbhg8WXnCbpZrIKik=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UX+twxd8x6POOzQBrVSlknvMWUwFTnsr+QwKOHztzsb58c5X7R40gpqX0ZlZ1thzd
	 /wjZxl5MWTENvH6LvfGUqP7/5XDZpPU5D5mDPi/O1HwnJYJLwg3V/SfpysZ/vapLkn
	 0bzdJyy/RUkgJtsrSrq1QTlE8iRh303UrJsxyYFJsk19CNBS+MHIrBTCF5Noh1G1Xl
	 Rd5byoNSdfNOlC14YYQelo1qkLpYp3P38XHKZjprIUsKnYR3txhLDAYbL7/yJ+EJYg
	 c44hPhwh2mCyfclt7pqkBw2ZnmGAChzYfBlqzOfP7MF/T88Wm4qy20spag00Dd0y/y
	 zJvhtgnnhxokA==
Date: Tue, 7 Sep 2021 16:17:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 6/7] xen/arm: Taint Xen on incompatible DCZID values
In-Reply-To: <DDB381B4-1519-43B5-9DA5-DE0C0E11916B@arm.com>
Message-ID: <alpine.DEB.2.21.2109071615170.14059@sstabellini-ThinkPad-T480s>
References: <cover.1629897306.git.bertrand.marquis@arm.com> <c1868fce1bcb0bcddf7bc786be166007f91f2f67.1629897306.git.bertrand.marquis@arm.com> <alpine.DEB.2.21.2109031529400.26072@sstabellini-ThinkPad-T480s> <DDB381B4-1519-43B5-9DA5-DE0C0E11916B@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 6 Sep 2021, Bertrand Marquis wrote:
> > On 3 Sep 2021, at 23:49, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Wed, 25 Aug 2021, Bertrand Marquis wrote:
> >> Use arm64 cpu feature sanitization to TAIN Xen if different DCZID values
> >                                        ^  TAINT
> > 
> > 
> >> are found (ftr_dczid is using only STRICT method).
> >> In this case actual memory being cleaned by DC ZVA operations would be
> >> different depending on the cores which could make a guest zeroing too
> >> much or too little memory if it is merged between CPUs.
> >> 
> >> We could, on processor supporting it, trap access to DCZID_EL0 register
> >               ^ processors
> 
> Could those typos be fixed during commit ?

Yes they can

 
> >> using HFGRTR_EL2 register but this would not solve the case where a
> >> process is being migrated during a copy or if it cached the value of the
> >> register.
> >> 
> >> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> >> ---
> >> Change in v3: none
> >> Change in v2: Patch introduced in v2
> >> ---
> >> xen/arch/arm/arm64/cpufeature.c  | 14 +++++++++++---
> >> xen/arch/arm/cpufeature.c        |  2 ++
> >> xen/include/asm-arm/cpufeature.h |  8 ++++++++
> >> 3 files changed, 21 insertions(+), 3 deletions(-)
> >> 
> >> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> >> index 61f629ebaa..b1936ef1d6 100644
> >> --- a/xen/arch/arm/arm64/cpufeature.c
> >> +++ b/xen/arch/arm/arm64/cpufeature.c
> >> @@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
> >> 	ARM64_FTR_END,
> >> };
> >> 
> >> -#if 0
> >> -/* TODO: handle this when sanitizing cache related registers */
> >> static const struct arm64_ftr_bits ftr_dczid[] = {
> >> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> >> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> >> 	ARM64_FTR_END,
> >> };
> >> -#endif
> >> 
> >> static const struct arm64_ftr_bits ftr_id_isar0[] = {
> >> 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> >> @@ -592,6 +589,17 @@ void update_system_features(const struct cpuinfo_arm *new)
> >> 
> >> 	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
> >> 
> >> +	/*
> >> +	 * Comment from Linux:
> > 
> > I don't know if I would keep or remove "Comment from Linux"
> 
> I added that because the comment itself does not really apply to Xen.
> I could have rephrased the comment/
> Anyway I have no objection to remove that statement.
> 
> Do I need to send a v2 for that ?

No you don't need to resend just for that or the typos. However if you
are going to resend, then an update would be nice :-)


From xen-devel-bounces@lists.xenproject.org Tue Sep 07 23:32:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 07 Sep 2021 23:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181403.328484 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkZd-0002p0-TE; Tue, 07 Sep 2021 23:32:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181403.328484; Tue, 07 Sep 2021 23:32:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNkZd-0002ot-QC; Tue, 07 Sep 2021 23:32:17 +0000
Received: by outflank-mailman (input) for mailman id 181403;
 Tue, 07 Sep 2021 23:32:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNkZd-0002oj-2H; Tue, 07 Sep 2021 23:32:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNkZc-0007fe-PX; Tue, 07 Sep 2021 23:32:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNkZc-0001QX-D0; Tue, 07 Sep 2021 23:32:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNkZc-00039a-CY; Tue, 07 Sep 2021 23:32:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=F9k0EF5cS0WsEXI+wXRHk8idvPFgzt/INONcaPSr6QE=; b=S4pIqQLgLDTa8sALgT3UwPlGhC
	rL7b59lXxYZbVguQvGAmN5h7nOe3PAepsIZ6zTFvEWXrX4VG9LCf2AB6tORguYhTSH5bMEfe/Qs/r
	e91NaMfQ3vC+r1Y40lSZ/Yhqd6W+DWvkjON4iV/9MaQEAh58CxpwPppSMdUZW7bdfjm4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164874-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164874: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=a61c30b8c8c3c8619847cfaa289233cc696f5689
X-Osstest-Versions-That:
    qemuu=935efca6c246c108253b0e4e51cc87648fc7ca10
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 07 Sep 2021 23:32:16 +0000

flight 164874 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164874/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164864
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164864
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164864
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164864
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164864
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164864
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164864
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                a61c30b8c8c3c8619847cfaa289233cc696f5689
baseline version:
 qemuu                935efca6c246c108253b0e4e51cc87648fc7ca10

Last test of basis   164864  2021-09-06 18:07:55 Z    1 days
Testing same since   164874  2021-09-07 12:37:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Tokarev <mjt@tls.msk.ru>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   935efca6c2..a61c30b8c8  a61c30b8c8c3c8619847cfaa289233cc696f5689 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 00:49:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 00:49:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181412.328498 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNlmI-00032w-Kh; Wed, 08 Sep 2021 00:49:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181412.328498; Wed, 08 Sep 2021 00:49:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNlmI-00032p-Hl; Wed, 08 Sep 2021 00:49:26 +0000
Received: by outflank-mailman (input) for mailman id 181412;
 Wed, 08 Sep 2021 00:49:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mNlmG-00032j-Ra
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 00:49:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9c0d77ac-103e-11ec-b138-12813bfff9fa;
 Wed, 08 Sep 2021 00:49:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C8EFC60F92;
 Wed,  8 Sep 2021 00:49:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c0d77ac-103e-11ec-b138-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631062163;
	bh=7Wrz7jbq4ZQpZbwmyQgxeFbh9DIedMdFxmZ4tupnR8g=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=loxhqFx2E6wvRf5U+77Vgq/G4KD4QYGjNxv/XXwguJR7h+SGcMNdeCalOVlWBVmlv
	 XpxOnyIjCYt9iNy/57iu/a55l7C51KbUdnvtZqrf8owUP8ymQHDG5/8v6I9y/iqKNY
	 2Rf/pUzL1rOqvX0AVjQX0rz4UvQ4jlhnBPXO5zEVnwSZezdW8uLbTh9tv9pXTXtqam
	 5CbGn/QNfcrdITpjLzz71QHXWDUfIXt98Ln6hHjz9PJdrlzA8yokvcxNB1M37+gZzx
	 YdyMRfDizU/B1oCc1Vg/7S3x3J97fGgH9rBTDpNXNERBLNINInx5C4/DFrzXpKGjTK
	 yMstLYGXAQWIA==
Date: Tue, 7 Sep 2021 17:49:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] xen/pvcalls: backend can be a module
In-Reply-To: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
Message-ID: <alpine.DEB.2.21.2109071745270.14059@sstabellini-ThinkPad-T480s>
References: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> It's not clear to me why only the frontend has been tristate. Switch the
> backend to be, too.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Actually although the frontend is a tristate, it cannot really work as a
module. Specifically see pvcalls_stream_ops in the enabling patch (still
not upstream):

https://github.com/lf-edge/runx/blob/master/kernel/patches/0001-patch-pvcalls_enable.patch


So I had to change tristate to bool for XEN_PVCALLS_FRONTEND in our
internal kernel tree.

That said, the PVCalls backend could very well be a module and
technically I don't see any reasons why not. So:


Acked-by: Stefano Stabellini <sstabellini@kernel.org>



> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -214,7 +214,7 @@ config XEN_PVCALLS_FRONTEND
>  	  implements them.
>  
>  config XEN_PVCALLS_BACKEND
> -	bool "XEN PV Calls backend driver"
> +	tristate "XEN PV Calls backend driver"
>  	depends on INET && XEN && XEN_BACKEND
>  	help
>  	  Experimental backend for the Xen PV Calls protocol
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 01:09:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 01:09:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181419.328509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNm5b-0003wJ-7s; Wed, 08 Sep 2021 01:09:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181419.328509; Wed, 08 Sep 2021 01:09:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNm5b-0003wC-4j; Wed, 08 Sep 2021 01:09:23 +0000
Received: by outflank-mailman (input) for mailman id 181419;
 Wed, 08 Sep 2021 01:09:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mNm5Z-0003w6-VT
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 01:09:21 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 65802e8e-1041-11ec-b138-12813bfff9fa;
 Wed, 08 Sep 2021 01:09:20 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 84E906113B;
 Wed,  8 Sep 2021 01:09:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65802e8e-1041-11ec-b138-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631063360;
	bh=pGXBU6C0v1ImgWBSjk60VtuyKReuk2qQAlQteZR1lUo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=CJHu7UaR1Kr3UbMYACb0j3DTxNpyeGwKxPfmfXL2LssgXd/P3ZlKM5ZvndqceczU1
	 nAXypYKoIyvI8PEUdOpKtZceQGicsw7jAYx8GVvvoTPiflgO/OliLcQn1UexFwoEoh
	 XxCnObIK7BYhvhBA53pbBOaJJN4E0ztqmglRF8xpFl7i01AFI1tTmyF1pGzU4nSbtY
	 rxwONC465Pvz5sNTUcSp7DujBW01nckbkYz9Pda0J2C+HNyVySH4hSET04PJWrrToc
	 huWctUAvQwKvrwrgPjGntWqag7YZJeb+E2LwXEKr64iz27fjXKmuLoOXc/RKhkm9fG
	 5eqOGeEB1YIUA==
Date: Tue, 7 Sep 2021 18:09:18 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Luca Fancellu <luca.fancellu@arm.com>, xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system
 start
In-Reply-To: <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
Message-ID: <alpine.DEB.2.21.2109071750262.14059@sstabellini-ThinkPad-T480s>
References: <20210907065228.21794-1-luca.fancellu@arm.com> <4bab7902-0268-5705-5462-fcd7571d7492@xen.org> <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com> <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-792418537-1631062279=:14059"
Content-ID: <alpine.DEB.2.21.2109071752210.14059@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-792418537-1631062279=:14059
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109071752211.14059@sstabellini-ThinkPad-T480s>

On Tue, 7 Sep 2021, Julien Grall wrote:
> On 07/09/2021 12:51, Luca Fancellu wrote:
> > > On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
> > > 
> > > Hi Luca,
> > > 
> > > On 07/09/2021 07:52, Luca Fancellu wrote:
> > > > Add a design describing a proposal to improve the EFI
> > > > configuration file, adding keywords to describe domU
> > > > guests and allowing to start a dom0less system.
> > > > Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> > > > ---
> > > >   docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
> > > >   1 file changed, 105 insertions(+)
> > > >   create mode 100644 docs/designs/efi-arm-dom0less.md
> > > > diff --git a/docs/designs/efi-arm-dom0less.md
> > > > b/docs/designs/efi-arm-dom0less.md
> > > > new file mode 100644
> > > > index 0000000000..8d8fa2243f
> > > > --- /dev/null
> > > > +++ b/docs/designs/efi-arm-dom0less.md
> > > > @@ -0,0 +1,105 @@
> > > > +# Xen EFI configuration file
> > > > +
> > > > +The current configuration file used by Xen when it is started as an EFI
> > > > +application is considering only the dom0 guest and doesn't have any
> > > > +property to describe and load in memory domU guests.
> > > 
> > >  From my understanding, the problem is less about properties (we already
> > > have them in the Device-Tree) but more about where are the binaries
> > > located in memory as we don't know in advance.
> > 
> > Hi Julien,
> Hi Luca,
> 
> > I think I used the wrong word there, I meant “keyword” instead of “property”
> > because I was referring about the
> > lack of keywords to describe a domu guest in the Xen EFI configuration file.
> > 
> > I agree with you that on systems with static allocation, the kernel and
> > ramdisk binaries must be at certain locations
> > that are out of control when we use the EFI boot services, the thing we can
> > do is provide a keyword to specify the
> > addresses and then use the CopyMem() function to relocate the kernel/ramdisk
> > in the address we want.
> 
> I wasn't specifically referring to static allocation here, sorry if this
> wasn't clear. I was pointing out that most of the information you create in
> the xen.cfg is going to be similar to what we already provide in the
> Device-Tree.
> 
> My main concern is everytime we add a new feature in Dom0less, a developer
> would need to write code for the DT and UEFI. This will increase the code size
> and maintenance.
> 
> The same can be said for the admin as if they want to switch from plain U-boot
> to UEFI, they would also need to fully rewrite the bindings.
> 
> AFAICT, most of the information provided in the Device-Tree are usable even
> when using UEFI. So I would prefer if we try to re-use what's existing. This
> is what my proposal below was about.
> 
> > 
> > > 
> > > So I would like to propose something that build on top of the Device-Tree
> > > work we did. Note this is early thoughts.
> > > 
> > > The problematic nodes in the DT are:
> > > 
> > >         module@0x4a000000 {
> > >             compatible = "multiboot,kernel", "multiboot,module";
> > >             reg = <0x0 0x4a000000 0xffffff>;
> > >             bootargs = "console=ttyAMA0 init=/bin/sh";
> > >         };
> > > 
> > >         module@0x4b000000 {
> > >             compatible = "multiboot,ramdisk", "multiboot,module";
> > >             reg = <0x0 0x4b000000 0xffffff>;
> > >         };
> > > 
> > > In particular the property "reg" cannot be known in advance because the
> > > UEFI stub will be responsible to load the binaries in memory.
> > 
> > Yes that’s true, the UEFI stub is using from the UEFI boot service the
> > AllocatePages function that is giving back an address out of our control,
> > then using another function the binary is read from the disk and copied at
> > that address, finally the UEFI stub is writing the node in the device tree
> > that
> > will be used by Xen later.
> 
> I am not sure to follow. Are you saying the UEFI stub will create the dom0less
> node in the DT based on the xen.cfg?
> 
> > 
> > > 
> > > What we could do is providing a list of binaries to load and associate a
> > > key for each of them. Something like:
> > > 
> > > binary=<binary> <key>
> > > binary=<binary2> <key2>
> > > ....
> > > 
> > > We can then replace the property "reg" with a new property "uefi,key" that
> > > will contain the name of the binary.
> > > 
> > > What do you think?
> > 
> > Here I’m lost, because I don’t understand what we are going to do with the
> > name of the binary.
> 
> <binaryX> would be used by the UEFI stub to load the binary in memory. Each
> binary will have a <keyX> which helps to refer them in the Device-Tree. To
> give a concrete example, let say we have two dom0less domains:
>   - DomA: 2 vCPUs, 128MB
>   - DomB: 3 vCPUs, 512MB
> 
> DomA and DomB will be using the same kernel but a different ramdisk. xen.cfg,
> would look like:
> 
> [global]
> default=section1
> 
> [section1]
> options=console=vga,com1 com1=57600 loglvl=all noreboot
> kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> ramdisk=initrd-3.0.31-0.4-xen
> xsm=<filename>
> dtb=devtree.dtb
> binary=vmlinuz-guest domu-kernel
> binary=ramdisk-domA.img domA-ramdisk
> binary=ramdisk-domB.img domB-ramdisk
> 
> The chosen node in the DT would look like:
> 
> chosen {
>     domU1 {
>         compatible = "xen,domain";
>         #address-cells = <0x2>;
>         #size-cells = <0x1>;
>         memory = <0 0x8000000>;
>         cpus = <2>;
> 
>         module@1 {
>             compatible = "multiboot,kernel", "multiboot,module";
>             uefi,binary = "domu-kernel";
>             bootargs = "console=ttyAMA0 init=/bin/sh";
>         };
> 
>         module@2 {
>             compatible = "multiboot,ramdisk", "multiboot,module";
>             uefi,binary = "domA-ramdisk";
>         };
>     };
> 
>     domU2 {
>         compatible = "xen,domain";
>         #address-cells = <0x3>;
>         #size-cells = <0x1>;
>         memory = <0 0x20000000>;
>         cpus = <3>;
> 
>         module@1 {
>             compatible = "multiboot,kernel", "multiboot,module";
>             uefi,binary = "domu-kernel";
>             bootargs = "console=ttyAMA0 init=/bin/sh";
>         };
> 
>         module@2 {
>             compatible = "multiboot,ramdisk", "multiboot,module";
>             uefi,binary = "domA-ramdisk";
>         };
>     };
> };
> 
> With this approach, the change is quite minimal to move between an classic
> U-boot boot and EFI boot.

Great idea! I think it is good to try to reuse Device Tree, and using it
as configuration is well aligned with other projects too (e.g.  Zephyr).


There are a few options for the bindings. These are some ideas.

If we are just going to specify a filename and a reference with the
"binary" key=value pair, then we could get rid of it entirely and just
write the filename directly in device tree:

    domU1 {
        compatible = "xen,domain";
        #address-cells = <0x2>;
        #size-cells = <0x1>;
        memory = <0 0x8000000>;
        cpus = <2>;

        module@1 {
            compatible = "multiboot,kernel", "multiboot,module";
            uefi,binary = "vmlinuz-guest";
            bootargs = "console=ttyAMA0 init=/bin/sh";
        };
    };

Then we don't even need the binary key=value pair, and especially the
device tree becomes "self contained" as in we don't need to resolve the
reference to "binary=value ref" to understand it.


(Also worth noting as inspiration the HyperLaunch binding that is using
the "mb-index" property as index in the multiboot module chain. Not
directly applicable, but interesting.)
--8323329-792418537-1631062279=:14059--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 04:02:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 04:02:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181428.328520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNonL-0006ep-2F; Wed, 08 Sep 2021 04:02:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181428.328520; Wed, 08 Sep 2021 04:02:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNonK-0006ei-U1; Wed, 08 Sep 2021 04:02:42 +0000
Received: by outflank-mailman (input) for mailman id 181428;
 Wed, 08 Sep 2021 04:02:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNonJ-0006eY-P4; Wed, 08 Sep 2021 04:02:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNonJ-0002lK-K6; Wed, 08 Sep 2021 04:02:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNonJ-0005vE-6X; Wed, 08 Sep 2021 04:02:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNonJ-0003do-5j; Wed, 08 Sep 2021 04:02:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=31yRpftZznTCZf6mr6Q6bH05CMy10lku+SaZLPWF7OI=; b=4ZG8erYYquhmBgiCaibvBoSuXi
	TjlTypTQeZSkUDCNdf9DyAmT/qGL2elhXesvvt4f2XD1qOkqnGsTfA+sulU0zRG9/4qjK8SvZPVtM
	xPfm3N67HP/Rbh1gKQFvQD7anQ0IcJVzSnYxNjPKL3DZ72jzxWl9CpNC0MHl5z1b+m3A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164876-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164876: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4b93c544e90e2b28326182d31ee008eb80e02074
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 04:02:41 +0000

flight 164876 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164876/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                4b93c544e90e2b28326182d31ee008eb80e02074
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  403 days
Failing since        152366  2020-08-01 20:49:34 Z  402 days  700 attempts
Testing same since   164865  2021-09-06 21:40:45 Z    1 days    3 attempts

------------------------------------------------------------
7221 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2221825 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 04:48:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 04:48:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181377.328534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNpVw-0002y0-MS; Wed, 08 Sep 2021 04:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181377.328534; Wed, 08 Sep 2021 04:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNpVw-0002xt-JB; Wed, 08 Sep 2021 04:48:48 +0000
Received: by outflank-mailman (input) for mailman id 181377;
 Tue, 07 Sep 2021 22:41:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XuZq=N5=seco.com=sean.anderson@srs-us1.protection.inumbo.net>)
 id 1mNjm1-0004gR-8e
 for xen-devel@lists.xenproject.org; Tue, 07 Sep 2021 22:41:01 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.66]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a8f1d8b2-102c-11ec-b131-12813bfff9fa;
 Tue, 07 Sep 2021 22:40:54 +0000 (UTC)
Received: from DB7PR03MB4523.eurprd03.prod.outlook.com (2603:10a6:10:19::27)
 by DB9PR03MB7514.eurprd03.prod.outlook.com (2603:10a6:10:22a::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Tue, 7 Sep
 2021 22:40:52 +0000
Received: from DB7PR03MB4523.eurprd03.prod.outlook.com
 ([fe80::dc6c:815b:2062:d1f1]) by DB7PR03MB4523.eurprd03.prod.outlook.com
 ([fe80::dc6c:815b:2062:d1f1%7]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021
 22:40:52 +0000
Received: from [172.27.1.65] (50.195.82.171) by
 BL1PR13CA0065.namprd13.prod.outlook.com (2603:10b6:208:2b8::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.4 via Frontend
 Transport; Tue, 7 Sep 2021 22:40:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8f1d8b2-102c-11ec-b131-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DAqhCa6P0makIMugGEUYGLBCWQpqrwKqcSWDZXWR5Zv1MJ+rnA6epSTlzihwMcQgTa6NhIKAeXPDLPRnMAuyhWZ2cVG0pECvQuhadCYVB9ZFMAyimacuq6u1595NK1ehS3UpNTF7ipaAF75scZvI9WPfjRWxD/rSiqW3XxIOtMQjfeDggZFpTOy/tISGPugkfmeOhWJUau2wmFqhzXCkNc3ibtw00+BfbZztU5Ojj27jGHRxCYnbP28S7h80ap9UwSnn7TDuMME49cXvu4bJi0i6YOsVGiDXS6ssHg4/LDtHrUlmOHMeAWfilMNBcp2KUIuwAipqYwlkkEzbtDHVVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xZGokp59XTygdvR/QbsSyJCnfBZE6dBkB3sePL70M8Y=;
 b=gEjK7eH/kgWoJX5b3YysxljI8unsHWGGy4UnqYvBSdHoq/aeXA1Tb8j7eiWATfYnj4ZQIxsEtm1yoMgdQm4vDlRfkeP0Anj3MIYJI/MGDR3wCDhBEAlYzVr9pUIwRyG8xzZavGj4w2kYhkcjPzr5eplXVRAaMiHU1f6/ffjuBOJDsg9p5PjUtDlVLndKSPkoF2KK/wOO6yzB9IfkuFhZZ4AIIOa2/lt+QgL3AXV3FjRGe0kNPOeqDPuw4iDPkTmJDutHvi6jEmBjakvhq4LwgoyIVYuih7znCaHbdzQ9o9D7EuxyA8VPW099mt1AXJ8NFXJ4NUVcAECUC0E+ob1W6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=seco.com; dmarc=pass action=none header.from=seco.com;
 dkim=pass header.d=seco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=secospa.onmicrosoft.com; s=selector2-secospa-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xZGokp59XTygdvR/QbsSyJCnfBZE6dBkB3sePL70M8Y=;
 b=ntqrV9aXJaPc48uCaAaZEd54r7EUT1nc8DTiLlvYYTIAstDY3v3rGdBX3ntOQDqk/qLNgh3kOKwOIZYxsvDZ4qSAE/HPUM9rualziEDS5Mj9CkSRVqJR2Lqn9tRoRXgmQLm4OzbFHyvbgmnmMKl/Avsr2KX+RDnMOJTBheEusnQ=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=seco.com;
From: Sean Anderson <sean.anderson@seco.com>
Subject: Re: [PATCH v2 4/5] mm/page_alloc: place pages to tail in
 __free_pages_core()
To: david@redhat.com
Cc: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com,
 dave.hansen@intel.com, haiyangz@microsoft.com, kys@microsoft.com,
 linux-acpi@vger.kernel.org, linux-hyperv@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mm@kvack.org,
 mgorman@techsingularity.net, mhocko@kernel.org, mhocko@suse.com,
 osalvador@suse.de, pankaj.gupta.linux@gmail.com,
 richard.weiyang@linux.alibaba.com, rppt@kernel.org, sthemmin@microsoft.com,
 vbabka@suse.cz, wei.liu@kernel.org, willy@infradead.org,
 xen-devel@lists.xenproject.org
References: <20201005121534.15649-5-david@redhat.com>
Message-ID: <a52dacbe-5649-7245-866f-ceaba44975b5@seco.com>
Date: Tue, 7 Sep 2021 18:40:45 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
In-Reply-To: <20201005121534.15649-5-david@redhat.com>
Content-Type: multipart/mixed;
 boundary="------------6DD34436CA52906803239DD3"
Content-Language: en-US
X-ClientProxiedBy: BL1PR13CA0065.namprd13.prod.outlook.com
 (2603:10b6:208:2b8::10) To DB7PR03MB4523.eurprd03.prod.outlook.com
 (2603:10a6:10:19::27)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 50993f61-2ff4-4447-4c9d-08d972508b91
X-MS-TrafficTypeDiagnostic: DB9PR03MB7514:
X-Microsoft-Antispam-PRVS:
	<DB9PR03MB7514FE3C7254E1FA878EF6D996D39@DB9PR03MB7514.eurprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6bmNaIYFblconuOojrGI+fgqGKtzuEcfNh1P7ZYwwHPhWTQpDkdq2BKUqUiq+wv9r4uJFNHtp/hQ2EYXMkP+JmXQgBy3/smNRZpBFCRIArncvtgNeSyjBV2lj6ymaHY59CSU1PNFIwFtLzk0lCRU1hCSCoTXRJk0tHBs57GF+dExsf3LnQG+PS1Ylsl5au/sm7BDSOdkdwkivwSR90a+W0D9zKSVPTDfUdL0Yad6dPGvjKw36qvzlMqvCmpmr3eXf9yFZzznUHs+NIzfc1fKcHlahFmoJpUZpqgfkLDNXT1DJ919ZghAAgYgJoJ3VvsaOBPxqhIMJwm5iqHgjJuHm6FRsIqG8Ot5MCPbV5Hso/uaG3OYRAIGl8n5q/If9MGhX1Ilsojs6c+xajqdx0YXgd2LCqHl4ukxM72+YmCom9RVECq9K8N0vp5a722MbfBqxx9Flr6B679pr6HuKjJnlrA/3ZyxS0yyh3saGqJe8eWRvGAccZmG03f3VO0cmHhcHDTEvSBamzoaArOtvOMVplU79/fUcGk4Zm3oFB794f4yfrB1WcB4dMFK8gtPn1jeNnC6kGQauYr7f1WBcDjoSVadIWW9DOk4MEvuXR9zv6J7dxMX0q7P4YtKgZ4qcGm19OwkLCi4YJpPSo+mANAuojC/nyIcWpHprM8fyk9D9LCmWOzDpZsfPEw9PL0YOKFGjzknUldqnLL5BGKpPloS16IcqMoS/hXpiqgUd8txqbN7r1hkmsL0vXjpI9dyoltHSuzeXorCHvddK7uyBfgWBg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB7PR03MB4523.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(396003)(346002)(39850400004)(136003)(36756003)(86362001)(316002)(16576012)(38100700002)(38350700002)(66556008)(66476007)(66616009)(66946007)(478600001)(6666004)(235185007)(2906002)(5660300002)(6486002)(31686004)(4326008)(31696002)(52116002)(33964004)(44832011)(186003)(8936002)(8676002)(26005)(7416002)(2616005)(956004)(6916009)(83380400001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QjBwNldVVEh1SkNLQ2tGMFJxK3hhZU9MYzljWkptUXhWUVJvNCtLM21IYU56?=
 =?utf-8?B?a2xocGtNc1RrRk9ieHRTVzlBL3ovd3hhV2g2RjUzOW03Nnp4SHJURytjRlY5?=
 =?utf-8?B?aFlSRXBaQmZoVm11WG8rSmovdTZyV3NkZ2RWbVRDRW5xMWNTaTRTSFd1eUZI?=
 =?utf-8?B?K1lSam1nOENuem1GZGUvRWx6NnNyTVE4TGxOWitpOXJvRTFOY1A1Uzc3bHVr?=
 =?utf-8?B?OUhPcG5KZzNEeS9HVXJxL1ZBTjNKZ3pFUW1hRUtHdjFwQkk0dUd6NmNyYUR0?=
 =?utf-8?B?UGNOb1QzWXNlTHo3QzNUcklqU0dFRHBWbytnNlVhei9jcUYvcGhSSk1rNXNu?=
 =?utf-8?B?dHQ5dm54TFRmWG1rb1VyUitoK1JmbzB3Qk1WOWlSNVlFc01SUlJ4Yy9lUEh5?=
 =?utf-8?B?SFAwdFh5eXNxdGJCTDlLak5Bd25keHBHR2t0ZmtiR0dpNTQ1THE0STF6QTUv?=
 =?utf-8?B?ZGM3UG5VL3FDWWdJZzVtOC9GSHFHS0dMbWxXY3I4M1FFNjlIMDB1eUxrZ1VI?=
 =?utf-8?B?YkdYcGZmbXZFVXJFNFFZL1RMQmtWRkxMbmlld0s1dFo5NjF6Y1IxaUpKZ3pp?=
 =?utf-8?B?djdKSE9hVEpFaGNnenR6c1Vjd1kyTDYrYndCaVYxSi9RWmx0c3YyMlF0MmU4?=
 =?utf-8?B?cUZKeGVrdWlDb21CYVVGOVA5cTdLWXNHVzZKb1dVR0t2aUE5bGI3WnhJZk1N?=
 =?utf-8?B?aTBid2tCZlBpMDJ5eCsrN2hCQkVrNjBXZHNDSGdKZDFOMkgySE9qMUlXQTNj?=
 =?utf-8?B?ejY0alJxMHhERWJnYVZYWTJoRllDSTFRbGgvY3F3WEgrd1Nrd3VQTkZJV0NM?=
 =?utf-8?B?SUl0eGFnb3Bsd09EUktDa1pCQXB1ZzZ4cGtxOHpKQkR1OFRRNnlwTjZQTVBi?=
 =?utf-8?B?UE9tRzdFL2NxV0FlVFhiTTE0eE5vcjNTTUFXSmp1aWVxdEVYK3JIMERQcDRY?=
 =?utf-8?B?RldteVIxVkJMRWJpMHh6WFh0YlhMSzVKNEdBZVBZWTFnSTVLSWxMTlN0dlg5?=
 =?utf-8?B?eFdmd1Qrd3pxUTdHQmwxYnlVZFlMZWdMOWgxaWJsVnFFbnozWXBhL2c0d1RY?=
 =?utf-8?B?S0lsN09ZbzZkM05EWUpWTENyRng3SUtFN0l2MVpkRFFpTU5VWXpTZ0xFbFhU?=
 =?utf-8?B?OWVySDhDRm5IYU9Fend4dmR6bktzYW9zbDhGbHVWMEtNRlp2NS9Pd1dMbzBZ?=
 =?utf-8?B?VGh6Z2xFdWRTWEhkWk9URWVSL1FwTWNvb3o3VGZKaU9hUXJncEhITlAxNGV1?=
 =?utf-8?B?VWdLdElyVFM2Y3lzS05TWCtoSytvMnQrdmlxN3kya2R4alZLS3duc3BUS2xS?=
 =?utf-8?B?ZDZvTXp5cW5pSmZzUFBTRTM2UkFSc3AyUGp6MW1XR2Y3QVRDdDlSTE5aY0Vl?=
 =?utf-8?B?ZHhDSWtQaExKeWJQdkJhRzB0dWdsSDllcnZuMldmWWF1VUp6U09Keno3L092?=
 =?utf-8?B?ampzVksvZU93NFBQVjRYVDZFaHpYTDBiWmlsK1NqenZYOUhkMENaRVM4NkNF?=
 =?utf-8?B?eFRSM2huUHBmRkU3OGgycFcrZzZoU1JnanhBWnMvRVQ3QUYxVFY1MzdxREhW?=
 =?utf-8?B?RDN6cjlHN3YydTBFanlaU3lFZGlRc25hS0UxWUJmS0gzR1grTnRWOEZydWZJ?=
 =?utf-8?B?aFJ5bVYydWo4MThJWFRRRmNjSkc0UjVSWWlLdXVSaGRPbXVNekxvS3lWV2xt?=
 =?utf-8?B?N0ZXU3d0VG4rb2pjeEUzVXdSK1VQU1NjMXhHVFRISHhZcUxtYkZpNUQ5YXJR?=
 =?utf-8?Q?/2p/1xiGT+YjllGNPNhQJYGXiglkgHfUWQpcOju?=
X-OriginatorOrg: seco.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 50993f61-2ff4-4447-4c9d-08d972508b91
X-MS-Exchange-CrossTenant-AuthSource: DB7PR03MB4523.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 22:40:51.8348
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: bebe97c3-6438-442e-ade3-ff17aa50e733
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 963kxJCamLyZL3NcMwTTnV7hkDQKBTU6kkIK+XGo4TyiE7UZ+7PV4AUwlMxM6/sHYcgpOwKDIvdnc2VpsLrFkA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR03MB7514

--------------6DD34436CA52906803239DD3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi David,

This patch breaks booting on my custom Xilinx ZynqMP board. Booting
fails just after/during GIC initialization:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.14.0 (sean@plantagenet) (aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #251 SMP Tue Sep 7 18:11:50 EDT 2021
[    0.000000] Machine model: xlnx,zynqmp
[    0.000000] earlycon: cdns0 at MMIO 0x00000000ff010000 (options '115200n8')
[    0.000000] printk: bootconsole [cdns0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000087fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000007fefffff]
[    0.000000]   node   0: [mem 0x0000000800000000-0x000000087fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000087fffffff]
[    0.000000] On node 0, zone Normal: 256 pages in unavailable ranges
[    0.000000] cma: Reserved 1000 MiB at 0x0000000041400000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 19 pages/cpu s46752 r0 d31072 u77824
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1033987
[    0.000000] Kernel command line: earlycon clk_ignore_unused root=/dev/mmcblk0p2 rootwait rw cma=1000M
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x000000003d400000-0x0000000041400000] (64MB)
[    0.000000] Memory: 3023384K/4193280K available (4288K kernel code, 514K rwdata, 1200K rodata, 896K init, 187K bss, 145896K reserved, 1024000K cma-reserved)
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Adjusting CPU interface base to 0x00000000f902f000
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GIC: Using split EOI/Deactivate mode

and I bisected it to this patch. Applying the following patch (for 5.14)
fixes booting again:

---
  mm/page_alloc.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index eeb3a9cb36bb..d4317392cadb 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1688,7 +1688,7 @@ void __free_pages_core(struct page *page, unsigned int order)
  	 * Bypass PCP and place fresh pages right to the tail, primarily
  	 * relevant for memory onlining.
  	 */
-	__free_pages_ok(page, order, FPI_TO_TAIL | FPI_SKIP_KASAN_POISON);
+	__free_pages_ok(page, order, FPI_NONE);
  }

  #ifdef CONFIG_NUMA
-- 
2.25.1

I have attached my config; but note that it lacks any storage drivers to
make bisecting easier.

--Sean


--------------6DD34436CA52906803239DD3
Content-Type: text/plain; charset=UTF-8;
 name=".config"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename=".config"

IwojIEF1dG9tYXRpY2FsbHkgZ2VuZXJhdGVkIGZpbGU7IERPIE5PVCBFRElULgojIExpbnV4L2Fy
bTY0IDUuMTQuMCBLZXJuZWwgQ29uZmlndXJhdGlvbgojCkNPTkZJR19DQ19WRVJTSU9OX1RFWFQ9
ImFhcmNoNjQtbGludXgtZ251LWdjYyAoVWJ1bnR1IDkuMy4wLTE3dWJ1bnR1MX4yMC4wNCkgOS4z
LjAiCkNPTkZJR19DQ19JU19HQ0M9eQpDT05GSUdfR0NDX1ZFUlNJT049OTAzMDAKQ09ORklHX0NM
QU5HX1ZFUlNJT049MApDT05GSUdfQVNfSVNfR05VPXkKQ09ORklHX0FTX1ZFUlNJT049MjM0MDAK
Q09ORklHX0xEX0lTX0JGRD15CkNPTkZJR19MRF9WRVJTSU9OPTIzNDAwCkNPTkZJR19MTERfVkVS
U0lPTj0wCkNPTkZJR19DQ19DQU5fTElOSz15CkNPTkZJR19DQ19DQU5fTElOS19TVEFUSUM9eQpD
T05GSUdfQ0NfSEFTX0FTTV9HT1RPPXkKQ09ORklHX0NDX0hBU19BU01fSU5MSU5FPXkKQ09ORklH
X0NDX0hBU19OT19QUk9GSUxFX0ZOX0FUVFI9eQpDT05GSUdfSVJRX1dPUks9eQpDT05GSUdfQlVJ
TERUSU1FX1RBQkxFX1NPUlQ9eQpDT05GSUdfVEhSRUFEX0lORk9fSU5fVEFTSz15CgojCiMgR2Vu
ZXJhbCBzZXR1cAojCkNPTkZJR19JTklUX0VOVl9BUkdfTElNSVQ9MzIKIyBDT05GSUdfQ09NUElM
RV9URVNUIGlzIG5vdCBzZXQKQ09ORklHX0xPQ0FMVkVSU0lPTj0iIgpDT05GSUdfTE9DQUxWRVJT
SU9OX0FVVE89eQpDT05GSUdfQlVJTERfU0FMVD0iIgpDT05GSUdfREVGQVVMVF9JTklUPSIiCkNP
TkZJR19ERUZBVUxUX0hPU1ROQU1FPSIobm9uZSkiCkNPTkZJR19TWVNWSVBDPXkKQ09ORklHX1NZ
U1ZJUENfU1lTQ1RMPXkKIyBDT05GSUdfV0FUQ0hfUVVFVUUgaXMgbm90IHNldApDT05GSUdfQ1JP
U1NfTUVNT1JZX0FUVEFDSD15CiMgQ09ORklHX1VTRUxJQiBpcyBub3Qgc2V0CkNPTkZJR19IQVZF
X0FSQ0hfQVVESVRTWVNDQUxMPXkKCiMKIyBJUlEgc3Vic3lzdGVtCiMKQ09ORklHX0dFTkVSSUNf
SVJRX1BST0JFPXkKQ09ORklHX0dFTkVSSUNfSVJRX1NIT1c9eQpDT05GSUdfR0VORVJJQ19JUlFf
U0hPV19MRVZFTD15CkNPTkZJR19HRU5FUklDX0lSUV9FRkZFQ1RJVkVfQUZGX01BU0s9eQpDT05G
SUdfR0VORVJJQ19JUlFfTUlHUkFUSU9OPXkKQ09ORklHX0hBUkRJUlFTX1NXX1JFU0VORD15CkNP
TkZJR19JUlFfRE9NQUlOPXkKQ09ORklHX0lSUV9ET01BSU5fSElFUkFSQ0hZPXkKQ09ORklHX0dF
TkVSSUNfSVJRX0lQST15CkNPTkZJR19HRU5FUklDX01TSV9JUlE9eQpDT05GSUdfR0VORVJJQ19N
U0lfSVJRX0RPTUFJTj15CkNPTkZJR19JUlFfTVNJX0lPTU1VPXkKQ09ORklHX0hBTkRMRV9ET01B
SU5fSVJRPXkKQ09ORklHX0lSUV9GT1JDRURfVEhSRUFESU5HPXkKQ09ORklHX1NQQVJTRV9JUlE9
eQojIENPTkZJR19HRU5FUklDX0lSUV9ERUJVR0ZTIGlzIG5vdCBzZXQKIyBlbmQgb2YgSVJRIHN1
YnN5c3RlbQoKQ09ORklHX0dFTkVSSUNfVElNRV9WU1lTQ0FMTD15CkNPTkZJR19HRU5FUklDX0NM
T0NLRVZFTlRTPXkKQ09ORklHX0FSQ0hfSEFTX1RJQ0tfQlJPQURDQVNUPXkKQ09ORklHX0dFTkVS
SUNfQ0xPQ0tFVkVOVFNfQlJPQURDQVNUPXkKCiMKIyBUaW1lcnMgc3Vic3lzdGVtCiMKQ09ORklH
X1RJQ0tfT05FU0hPVD15CkNPTkZJR19OT19IWl9DT01NT049eQojIENPTkZJR19IWl9QRVJJT0RJ
QyBpcyBub3Qgc2V0CkNPTkZJR19OT19IWl9JRExFPXkKIyBDT05GSUdfTk9fSFpfRlVMTCBpcyBu
b3Qgc2V0CkNPTkZJR19OT19IWj15CkNPTkZJR19ISUdIX1JFU19USU1FUlM9eQojIGVuZCBvZiBU
aW1lcnMgc3Vic3lzdGVtCgpDT05GSUdfSEFWRV9FQlBGX0pJVD15CkNPTkZJR19BUkNIX1dBTlRf
REVGQVVMVF9CUEZfSklUPXkKCiMKIyBCUEYgc3Vic3lzdGVtCiMKIyBDT05GSUdfQlBGX1NZU0NB
TEwgaXMgbm90IHNldAojIGVuZCBvZiBCUEYgc3Vic3lzdGVtCgpDT05GSUdfUFJFRU1QVF9OT05F
PXkKIyBDT05GSUdfUFJFRU1QVF9WT0xVTlRBUlkgaXMgbm90IHNldAojIENPTkZJR19QUkVFTVBU
IGlzIG5vdCBzZXQKCiMKIyBDUFUvVGFzayB0aW1lIGFuZCBzdGF0cyBhY2NvdW50aW5nCiMKQ09O
RklHX1RJQ0tfQ1BVX0FDQ09VTlRJTkc9eQojIENPTkZJR19WSVJUX0NQVV9BQ0NPVU5USU5HX0dF
TiBpcyBub3Qgc2V0CiMgQ09ORklHX0lSUV9USU1FX0FDQ09VTlRJTkcgaXMgbm90IHNldApDT05G
SUdfQlNEX1BST0NFU1NfQUNDVD15CiMgQ09ORklHX0JTRF9QUk9DRVNTX0FDQ1RfVjMgaXMgbm90
IHNldAojIENPTkZJR19QU0kgaXMgbm90IHNldAojIGVuZCBvZiBDUFUvVGFzayB0aW1lIGFuZCBz
dGF0cyBhY2NvdW50aW5nCgpDT05GSUdfQ1BVX0lTT0xBVElPTj15CgojCiMgUkNVIFN1YnN5c3Rl
bQojCkNPTkZJR19UUkVFX1JDVT15CiMgQ09ORklHX1JDVV9FWFBFUlQgaXMgbm90IHNldApDT05G
SUdfU1JDVT15CkNPTkZJR19UUkVFX1NSQ1U9eQpDT05GSUdfUkNVX1NUQUxMX0NPTU1PTj15CkNP
TkZJR19SQ1VfTkVFRF9TRUdDQkxJU1Q9eQojIGVuZCBvZiBSQ1UgU3Vic3lzdGVtCgpDT05GSUdf
SUtDT05GSUc9eQpDT05GSUdfSUtDT05GSUdfUFJPQz15CiMgQ09ORklHX0lLSEVBREVSUyBpcyBu
b3Qgc2V0CkNPTkZJR19MT0dfQlVGX1NISUZUPTE2CkNPTkZJR19MT0dfQ1BVX01BWF9CVUZfU0hJ
RlQ9MTIKQ09ORklHX1BSSU5US19TQUZFX0xPR19CVUZfU0hJRlQ9MTMKQ09ORklHX0dFTkVSSUNf
U0NIRURfQ0xPQ0s9eQoKIwojIFNjaGVkdWxlciBmZWF0dXJlcwojCiMgZW5kIG9mIFNjaGVkdWxl
ciBmZWF0dXJlcwoKQ09ORklHX0FSQ0hfU1VQUE9SVFNfTlVNQV9CQUxBTkNJTkc9eQpDT05GSUdf
Q0NfSEFTX0lOVDEyOD15CkNPTkZJR19BUkNIX1NVUFBPUlRTX0lOVDEyOD15CkNPTkZJR19DR1JP
VVBTPXkKIyBDT05GSUdfTUVNQ0cgaXMgbm90IHNldAojIENPTkZJR19DR1JPVVBfU0NIRUQgaXMg
bm90IHNldAojIENPTkZJR19DR1JPVVBfUElEUyBpcyBub3Qgc2V0CiMgQ09ORklHX0NHUk9VUF9S
RE1BIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0dST1VQX0ZSRUVaRVIgaXMgbm90IHNldAojIENPTkZJ
R19DR1JPVVBfSFVHRVRMQiBpcyBub3Qgc2V0CiMgQ09ORklHX0NQVVNFVFMgaXMgbm90IHNldAoj
IENPTkZJR19DR1JPVVBfREVWSUNFIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0dST1VQX0NQVUFDQ1Qg
aXMgbm90IHNldAojIENPTkZJR19DR1JPVVBfUEVSRiBpcyBub3Qgc2V0CiMgQ09ORklHX0NHUk9V
UF9NSVNDIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0dST1VQX0RFQlVHIGlzIG5vdCBzZXQKIyBDT05G
SUdfTkFNRVNQQUNFUyBpcyBub3Qgc2V0CiMgQ09ORklHX0NIRUNLUE9JTlRfUkVTVE9SRSBpcyBu
b3Qgc2V0CiMgQ09ORklHX1NDSEVEX0FVVE9HUk9VUCBpcyBub3Qgc2V0CiMgQ09ORklHX1NZU0ZT
X0RFUFJFQ0FURUQgaXMgbm90IHNldAojIENPTkZJR19SRUxBWSBpcyBub3Qgc2V0CkNPTkZJR19C
TEtfREVWX0lOSVRSRD15CkNPTkZJR19JTklUUkFNRlNfU09VUkNFPSIiCkNPTkZJR19SRF9HWklQ
PXkKQ09ORklHX1JEX0JaSVAyPXkKQ09ORklHX1JEX0xaTUE9eQpDT05GSUdfUkRfWFo9eQpDT05G
SUdfUkRfTFpPPXkKQ09ORklHX1JEX0xaND15CkNPTkZJR19SRF9aU1REPXkKIyBDT05GSUdfQk9P
VF9DT05GSUcgaXMgbm90IHNldApDT05GSUdfQ0NfT1BUSU1JWkVfRk9SX1BFUkZPUk1BTkNFPXkK
IyBDT05GSUdfQ0NfT1BUSU1JWkVfRk9SX1NJWkUgaXMgbm90IHNldApDT05GSUdfTERfT1JQSEFO
X1dBUk49eQpDT05GSUdfU1lTQ1RMPXkKQ09ORklHX0hBVkVfVUlEMTY9eQpDT05GSUdfU1lTQ1RM
X0VYQ0VQVElPTl9UUkFDRT15CkNPTkZJR19FWFBFUlQ9eQpDT05GSUdfVUlEMTY9eQpDT05GSUdf
TVVMVElVU0VSPXkKIyBDT05GSUdfU0dFVE1BU0tfU1lTQ0FMTCBpcyBub3Qgc2V0CkNPTkZJR19T
WVNGU19TWVNDQUxMPXkKQ09ORklHX0ZIQU5ETEU9eQpDT05GSUdfUE9TSVhfVElNRVJTPXkKQ09O
RklHX1BSSU5USz15CkNPTkZJR19QUklOVEtfTk1JPXkKQ09ORklHX0JVRz15CkNPTkZJR19FTEZf
Q09SRT15CkNPTkZJR19CQVNFX0ZVTEw9eQpDT05GSUdfRlVURVg9eQpDT05GSUdfRlVURVhfUEk9
eQpDT05GSUdfSEFWRV9GVVRFWF9DTVBYQ0hHPXkKQ09ORklHX0VQT0xMPXkKQ09ORklHX1NJR05B
TEZEPXkKQ09ORklHX1RJTUVSRkQ9eQpDT05GSUdfRVZFTlRGRD15CkNPTkZJR19TSE1FTT15CkNP
TkZJR19BSU89eQpDT05GSUdfSU9fVVJJTkc9eQpDT05GSUdfQURWSVNFX1NZU0NBTExTPXkKQ09O
RklHX01FTUJBUlJJRVI9eQpDT05GSUdfS0FMTFNZTVM9eQojIENPTkZJR19LQUxMU1lNU19BTEwg
aXMgbm90IHNldApDT05GSUdfS0FMTFNZTVNfQkFTRV9SRUxBVElWRT15CiMgQ09ORklHX1VTRVJG
QVVMVEZEIGlzIG5vdCBzZXQKQ09ORklHX0FSQ0hfSEFTX01FTUJBUlJJRVJfU1lOQ19DT1JFPXkK
IyBDT05GSUdfS0NNUCBpcyBub3Qgc2V0CkNPTkZJR19SU0VRPXkKIyBDT05GSUdfREVCVUdfUlNF
USBpcyBub3Qgc2V0CkNPTkZJR19FTUJFRERFRD15CkNPTkZJR19IQVZFX1BFUkZfRVZFTlRTPXkK
IyBDT05GSUdfUEMxMDQgaXMgbm90IHNldAoKIwojIEtlcm5lbCBQZXJmb3JtYW5jZSBFdmVudHMg
QW5kIENvdW50ZXJzCiMKQ09ORklHX1BFUkZfRVZFTlRTPXkKIyBDT05GSUdfREVCVUdfUEVSRl9V
U0VfVk1BTExPQyBpcyBub3Qgc2V0CiMgZW5kIG9mIEtlcm5lbCBQZXJmb3JtYW5jZSBFdmVudHMg
QW5kIENvdW50ZXJzCgpDT05GSUdfVk1fRVZFTlRfQ09VTlRFUlM9eQojIENPTkZJR19DT01QQVRf
QlJLIGlzIG5vdCBzZXQKQ09ORklHX1NMQUI9eQojIENPTkZJR19TTFVCIGlzIG5vdCBzZXQKIyBD
T05GSUdfU0xPQiBpcyBub3Qgc2V0CkNPTkZJR19TTEFCX01FUkdFX0RFRkFVTFQ9eQojIENPTkZJ
R19TTEFCX0ZSRUVMSVNUX1JBTkRPTSBpcyBub3Qgc2V0CiMgQ09ORklHX1NMQUJfRlJFRUxJU1Rf
SEFSREVORUQgaXMgbm90IHNldAojIENPTkZJR19TSFVGRkxFX1BBR0VfQUxMT0NBVE9SIGlzIG5v
dCBzZXQKQ09ORklHX1BST0ZJTElORz15CiMgZW5kIG9mIEdlbmVyYWwgc2V0dXAKCkNPTkZJR19B
Uk02ND15CkNPTkZJR182NEJJVD15CkNPTkZJR19NTVU9eQpDT05GSUdfQVJNNjRfUEFHRV9TSElG
VD0xMgpDT05GSUdfQVJNNjRfQ09OVF9QVEVfU0hJRlQ9NApDT05GSUdfQVJNNjRfQ09OVF9QTURf
U0hJRlQ9NApDT05GSUdfQVJDSF9NTUFQX1JORF9CSVRTX01JTj0xOApDT05GSUdfQVJDSF9NTUFQ
X1JORF9CSVRTX01BWD0yNApDT05GSUdfQVJDSF9NTUFQX1JORF9DT01QQVRfQklUU19NSU49MTEK
Q09ORklHX0FSQ0hfTU1BUF9STkRfQ09NUEFUX0JJVFNfTUFYPTE2CkNPTkZJR19OT19JT1BPUlRf
TUFQPXkKQ09ORklHX1NUQUNLVFJBQ0VfU1VQUE9SVD15CkNPTkZJR19JTExFR0FMX1BPSU5URVJf
VkFMVUU9MHhkZWFkMDAwMDAwMDAwMDAwCkNPTkZJR19MT0NLREVQX1NVUFBPUlQ9eQpDT05GSUdf
VFJBQ0VfSVJRRkxBR1NfU1VQUE9SVD15CkNPTkZJR19HRU5FUklDX0JVRz15CkNPTkZJR19HRU5F
UklDX0JVR19SRUxBVElWRV9QT0lOVEVSUz15CkNPTkZJR19HRU5FUklDX0hXRUlHSFQ9eQpDT05G
SUdfR0VORVJJQ19DU1VNPXkKQ09ORklHX0dFTkVSSUNfQ0FMSUJSQVRFX0RFTEFZPXkKQ09ORklH
X0FSQ0hfTUhQX01FTU1BUF9PTl9NRU1PUllfRU5BQkxFPXkKQ09ORklHX1NNUD15CkNPTkZJR19L
RVJORUxfTU9ERV9ORU9OPXkKQ09ORklHX0ZJWF9FQVJMWUNPTl9NRU09eQpDT05GSUdfUEdUQUJM
RV9MRVZFTFM9MwpDT05GSUdfQVJDSF9TVVBQT1JUU19VUFJPQkVTPXkKQ09ORklHX0FSQ0hfUFJP
Q19LQ09SRV9URVhUPXkKCiMKIyBQbGF0Zm9ybSBzZWxlY3Rpb24KIwojIENPTkZJR19BUkNIX0FD
VElPTlMgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1NVTlhJIGlzIG5vdCBzZXQKIyBDT05GSUdf
QVJDSF9BTFBJTkUgaXMgbm90IHNldAojIENPTkZJR19BUkNIX0FQUExFIGlzIG5vdCBzZXQKIyBD
T05GSUdfQVJDSF9CQ00yODM1IGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9CQ000OTA4IGlzIG5v
dCBzZXQKIyBDT05GSUdfQVJDSF9CQ01fSVBST0MgaXMgbm90IHNldAojIENPTkZJR19BUkNIX0JF
UkxJTiBpcyBub3Qgc2V0CiMgQ09ORklHX0FSQ0hfQklUTUFJTiBpcyBub3Qgc2V0CiMgQ09ORklH
X0FSQ0hfQlJDTVNUQiBpcyBub3Qgc2V0CiMgQ09ORklHX0FSQ0hfRVhZTk9TIGlzIG5vdCBzZXQK
IyBDT05GSUdfQVJDSF9TUEFSWDUgaXMgbm90IHNldAojIENPTkZJR19BUkNIX0szIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQVJDSF9MQVlFUlNDQVBFIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9MRzFL
IGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9ISVNJIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9L
RUVNQkFZIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9NRURJQVRFSyBpcyBub3Qgc2V0CiMgQ09O
RklHX0FSQ0hfTUVTT04gaXMgbm90IHNldAojIENPTkZJR19BUkNIX01WRUJVIGlzIG5vdCBzZXQK
IyBDT05GSUdfQVJDSF9NWEMgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1FDT00gaXMgbm90IHNl
dAojIENPTkZJR19BUkNIX1JFQUxURUsgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1JFTkVTQVMg
aXMgbm90IHNldAojIENPTkZJR19BUkNIX1JPQ0tDSElQIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJD
SF9TMzIgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1NFQVRUTEUgaXMgbm90IHNldAojIENPTkZJ
R19BUkNIX0lOVEVMX1NPQ0ZQR0EgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1NZTlFVQUNFUiBp
cyBub3Qgc2V0CiMgQ09ORklHX0FSQ0hfVEVHUkEgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1NQ
UkQgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1RIVU5ERVIgaXMgbm90IHNldAojIENPTkZJR19B
UkNIX1RIVU5ERVIyIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9VTklQSElFUiBpcyBub3Qgc2V0
CiMgQ09ORklHX0FSQ0hfVkVYUFJFU1MgaXMgbm90IHNldAojIENPTkZJR19BUkNIX1ZJU0NPTlRJ
IGlzIG5vdCBzZXQKIyBDT05GSUdfQVJDSF9YR0VORSBpcyBub3Qgc2V0CkNPTkZJR19BUkNIX1pZ
TlFNUD15CiMgZW5kIG9mIFBsYXRmb3JtIHNlbGVjdGlvbgoKIwojIEtlcm5lbCBGZWF0dXJlcwoj
CgojCiMgQVJNIGVycmF0YSB3b3JrYXJvdW5kcyB2aWEgdGhlIGFsdGVybmF0aXZlcyBmcmFtZXdv
cmsKIwpDT05GSUdfQVJNNjRfV09SS0FST1VORF9DTEVBTl9DQUNIRT15CkNPTkZJR19BUk02NF9F
UlJBVFVNXzgyNjMxOT15CkNPTkZJR19BUk02NF9FUlJBVFVNXzgyNzMxOT15CkNPTkZJR19BUk02
NF9FUlJBVFVNXzgyNDA2OT15CkNPTkZJR19BUk02NF9FUlJBVFVNXzgxOTQ3Mj15CkNPTkZJR19B
Uk02NF9FUlJBVFVNXzgzMjA3NT15CkNPTkZJR19BUk02NF9FUlJBVFVNXzg0NTcxOT15CkNPTkZJ
R19BUk02NF9FUlJBVFVNXzg0MzQxOT15CkNPTkZJR19BUk02NF9MRF9IQVNfRklYX0VSUkFUVU1f
ODQzNDE5PXkKQ09ORklHX0FSTTY0X0VSUkFUVU1fMTAyNDcxOD15CkNPTkZJR19BUk02NF9FUlJB
VFVNXzE0MTgwNDA9eQpDT05GSUdfQVJNNjRfV09SS0FST1VORF9TUEVDVUxBVElWRV9BVD15CkNP
TkZJR19BUk02NF9FUlJBVFVNXzExNjU1MjI9eQpDT05GSUdfQVJNNjRfRVJSQVRVTV8xMzE5MzY3
PXkKQ09ORklHX0FSTTY0X0VSUkFUVU1fMTUzMDkyMz15CkNPTkZJR19BUk02NF9XT1JLQVJPVU5E
X1JFUEVBVF9UTEJJPXkKQ09ORklHX0FSTTY0X0VSUkFUVU1fMTI4NjgwNz15CkNPTkZJR19BUk02
NF9FUlJBVFVNXzE0NjMyMjU9eQpDT05GSUdfQVJNNjRfRVJSQVRVTV8xNTQyNDE5PXkKQ09ORklH
X0FSTTY0X0VSUkFUVU1fMTUwODQxMj15CkNPTkZJR19DQVZJVU1fRVJSQVRVTV8yMjM3NT15CkNP
TkZJR19DQVZJVU1fRVJSQVRVTV8yMzE1ND15CkNPTkZJR19DQVZJVU1fRVJSQVRVTV8yNzQ1Nj15
CkNPTkZJR19DQVZJVU1fRVJSQVRVTV8zMDExNT15CkNPTkZJR19DQVZJVU1fVFgyX0VSUkFUVU1f
MjE5PXkKQ09ORklHX0ZVSklUU1VfRVJSQVRVTV8wMTAwMDE9eQpDT05GSUdfSElTSUxJQ09OX0VS
UkFUVU1fMTYxNjAwODAyPXkKQ09ORklHX1FDT01fRkFMS09SX0VSUkFUVU1fMTAwMz15CkNPTkZJ
R19RQ09NX0ZBTEtPUl9FUlJBVFVNXzEwMDk9eQpDT05GSUdfUUNPTV9RREYyNDAwX0VSUkFUVU1f
MDA2NT15CkNPTkZJR19RQ09NX0ZBTEtPUl9FUlJBVFVNX0UxMDQxPXkKQ09ORklHX05WSURJQV9D
QVJNRUxfQ05QX0VSUkFUVU09eQpDT05GSUdfU09DSU9ORVhUX1NZTlFVQUNFUl9QUkVJVFM9eQoj
IGVuZCBvZiBBUk0gZXJyYXRhIHdvcmthcm91bmRzIHZpYSB0aGUgYWx0ZXJuYXRpdmVzIGZyYW1l
d29yawoKQ09ORklHX0FSTTY0XzRLX1BBR0VTPXkKIyBDT05GSUdfQVJNNjRfMTZLX1BBR0VTIGlz
IG5vdCBzZXQKIyBDT05GSUdfQVJNNjRfNjRLX1BBR0VTIGlzIG5vdCBzZXQKQ09ORklHX0FSTTY0
X1ZBX0JJVFNfMzk9eQojIENPTkZJR19BUk02NF9WQV9CSVRTXzQ4IGlzIG5vdCBzZXQKQ09ORklH
X0FSTTY0X1ZBX0JJVFM9MzkKQ09ORklHX0FSTTY0X1BBX0JJVFNfNDg9eQpDT05GSUdfQVJNNjRf
UEFfQklUUz00OAojIENPTkZJR19DUFVfQklHX0VORElBTiBpcyBub3Qgc2V0CkNPTkZJR19DUFVf
TElUVExFX0VORElBTj15CiMgQ09ORklHX1NDSEVEX01DIGlzIG5vdCBzZXQKIyBDT05GSUdfU0NI
RURfU01UIGlzIG5vdCBzZXQKQ09ORklHX05SX0NQVVM9NApDT05GSUdfSE9UUExVR19DUFU9eQoj
IENPTkZJR19OVU1BIGlzIG5vdCBzZXQKIyBDT05GSUdfSFpfMTAwIGlzIG5vdCBzZXQKQ09ORklH
X0haXzI1MD15CiMgQ09ORklHX0haXzMwMCBpcyBub3Qgc2V0CiMgQ09ORklHX0haXzEwMDAgaXMg
bm90IHNldApDT05GSUdfSFo9MjUwCkNPTkZJR19TQ0hFRF9IUlRJQ0s9eQpDT05GSUdfQVJDSF9T
UEFSU0VNRU1fRU5BQkxFPXkKQ09ORklHX0hXX1BFUkZfRVZFTlRTPXkKQ09ORklHX0FSQ0hfSEFT
X0ZJTFRFUl9QR1BST1Q9eQojIENPTkZJR19QQVJBVklSVCBpcyBub3Qgc2V0CiMgQ09ORklHX1BB
UkFWSVJUX1RJTUVfQUNDT1VOVElORyBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWEVDIGlzIG5vdCBz
ZXQKIyBDT05GSUdfS0VYRUNfRklMRSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSQVNIX0RVTVAgaXMg
bm90IHNldAojIENPTkZJR19YRU4gaXMgbm90IHNldApDT05GSUdfRk9SQ0VfTUFYX1pPTkVPUkRF
Uj0xMQpDT05GSUdfVU5NQVBfS0VSTkVMX0FUX0VMMD15CkNPTkZJR19ST0RBVEFfRlVMTF9ERUZB
VUxUX0VOQUJMRUQ9eQojIENPTkZJR19BUk02NF9TV19UVEJSMF9QQU4gaXMgbm90IHNldApDT05G
SUdfQVJNNjRfVEFHR0VEX0FERFJfQUJJPXkKQ09ORklHX0NPTVBBVD15CkNPTkZJR19LVVNFUl9I
RUxQRVJTPXkKIyBDT05GSUdfQVJNVjhfREVQUkVDQVRFRCBpcyBub3Qgc2V0CgojCiMgQVJNdjgu
MSBhcmNoaXRlY3R1cmFsIGZlYXR1cmVzCiMKQ09ORklHX0FSTTY0X0hXX0FGREJNPXkKQ09ORklH
X0FSTTY0X1BBTj15CkNPTkZJR19BU19IQVNfTERBUFI9eQpDT05GSUdfQVNfSEFTX0xTRV9BVE9N
SUNTPXkKIyBlbmQgb2YgQVJNdjguMSBhcmNoaXRlY3R1cmFsIGZlYXR1cmVzCgojCiMgQVJNdjgu
MiBhcmNoaXRlY3R1cmFsIGZlYXR1cmVzCiMKIyBDT05GSUdfQVJNNjRfUE1FTSBpcyBub3Qgc2V0
CkNPTkZJR19BUk02NF9SQVNfRVhUTj15CkNPTkZJR19BUk02NF9DTlA9eQojIGVuZCBvZiBBUk12
OC4yIGFyY2hpdGVjdHVyYWwgZmVhdHVyZXMKCiMKIyBBUk12OC4zIGFyY2hpdGVjdHVyYWwgZmVh
dHVyZXMKIwpDT05GSUdfQVJNNjRfUFRSX0FVVEg9eQpDT05GSUdfQVJNNjRfUFRSX0FVVEhfS0VS
TkVMPXkKQ09ORklHX0NDX0hBU19CUkFOQ0hfUFJPVF9QQUNfUkVUPXkKQ09ORklHX0NDX0hBU19T
SUdOX1JFVFVSTl9BRERSRVNTPXkKQ09ORklHX0FTX0hBU19QQUM9eQpDT05GSUdfQVNfSEFTX0NG
SV9ORUdBVEVfUkFfU1RBVEU9eQojIGVuZCBvZiBBUk12OC4zIGFyY2hpdGVjdHVyYWwgZmVhdHVy
ZXMKCiMKIyBBUk12OC40IGFyY2hpdGVjdHVyYWwgZmVhdHVyZXMKIwpDT05GSUdfQVJNNjRfQU1V
X0VYVE49eQpDT05GSUdfQVNfSEFTX0FSTVY4XzQ9eQpDT05GSUdfQVJNNjRfVExCX1JBTkdFPXkK
IyBlbmQgb2YgQVJNdjguNCBhcmNoaXRlY3R1cmFsIGZlYXR1cmVzCgojCiMgQVJNdjguNSBhcmNo
aXRlY3R1cmFsIGZlYXR1cmVzCiMKQ09ORklHX0FTX0hBU19BUk1WOF81PXkKQ09ORklHX0FSTTY0
X0JUST15CkNPTkZJR19DQ19IQVNfQlJBTkNIX1BST1RfUEFDX1JFVF9CVEk9eQpDT05GSUdfQVJN
NjRfRTBQRD15CkNPTkZJR19BUkNIX1JBTkRPTT15CkNPTkZJR19BUk02NF9BU19IQVNfTVRFPXkK
Q09ORklHX0FSTTY0X01URT15CiMgZW5kIG9mIEFSTXY4LjUgYXJjaGl0ZWN0dXJhbCBmZWF0dXJl
cwoKIwojIEFSTXY4LjcgYXJjaGl0ZWN0dXJhbCBmZWF0dXJlcwojCkNPTkZJR19BUk02NF9FUEFO
PXkKIyBlbmQgb2YgQVJNdjguNyBhcmNoaXRlY3R1cmFsIGZlYXR1cmVzCgpDT05GSUdfQVJNNjRf
U1ZFPXkKIyBDT05GSUdfQVJNNjRfUFNFVURPX05NSSBpcyBub3Qgc2V0CkNPTkZJR19SRUxPQ0FU
QUJMRT15CiMgQ09ORklHX1JBTkRPTUlaRV9CQVNFIGlzIG5vdCBzZXQKQ09ORklHX0NDX0hBVkVf
U1RBQ0tQUk9URUNUT1JfU1lTUkVHPXkKQ09ORklHX1NUQUNLUFJPVEVDVE9SX1BFUl9UQVNLPXkK
IyBlbmQgb2YgS2VybmVsIEZlYXR1cmVzCgojCiMgQm9vdCBvcHRpb25zCiMKQ09ORklHX0NNRExJ
TkU9IiIKQ09ORklHX0VGSV9TVFVCPXkKQ09ORklHX0VGST15CiMgQ09ORklHX0RNSSBpcyBub3Qg
c2V0CiMgZW5kIG9mIEJvb3Qgb3B0aW9ucwoKQ09ORklHX1NZU1ZJUENfQ09NUEFUPXkKCiMKIyBQ
b3dlciBtYW5hZ2VtZW50IG9wdGlvbnMKIwpDT05GSUdfU1VTUEVORD15CkNPTkZJR19TVVNQRU5E
X0ZSRUVaRVI9eQojIENPTkZJR19TVVNQRU5EX1NLSVBfU1lOQyBpcyBub3Qgc2V0CkNPTkZJR19Q
TV9TTEVFUD15CkNPTkZJR19QTV9TTEVFUF9TTVA9eQojIENPTkZJR19QTV9BVVRPU0xFRVAgaXMg
bm90IHNldAojIENPTkZJR19QTV9XQUtFTE9DS1MgaXMgbm90IHNldApDT05GSUdfUE09eQojIENP
TkZJR19QTV9ERUJVRyBpcyBub3Qgc2V0CkNPTkZJR19QTV9DTEs9eQpDT05GSUdfUE1fR0VORVJJ
Q19ET01BSU5TPXkKIyBDT05GSUdfV1FfUE9XRVJfRUZGSUNJRU5UX0RFRkFVTFQgaXMgbm90IHNl
dApDT05GSUdfUE1fR0VORVJJQ19ET01BSU5TX1NMRUVQPXkKQ09ORklHX1BNX0dFTkVSSUNfRE9N
QUlOU19PRj15CkNPTkZJR19DUFVfUE09eQojIENPTkZJR19FTkVSR1lfTU9ERUwgaXMgbm90IHNl
dApDT05GSUdfQVJDSF9ISUJFUk5BVElPTl9QT1NTSUJMRT15CkNPTkZJR19BUkNIX1NVU1BFTkRf
UE9TU0lCTEU9eQojIGVuZCBvZiBQb3dlciBtYW5hZ2VtZW50IG9wdGlvbnMKCiMKIyBDUFUgUG93
ZXIgTWFuYWdlbWVudAojCgojCiMgQ1BVIElkbGUKIwpDT05GSUdfQ1BVX0lETEU9eQpDT05GSUdf
Q1BVX0lETEVfTVVMVElQTEVfRFJJVkVSUz15CiMgQ09ORklHX0NQVV9JRExFX0dPVl9MQURERVIg
aXMgbm90IHNldApDT05GSUdfQ1BVX0lETEVfR09WX01FTlU9eQojIENPTkZJR19DUFVfSURMRV9H
T1ZfVEVPIGlzIG5vdCBzZXQKQ09ORklHX0RUX0lETEVfU1RBVEVTPXkKCiMKIyBBUk0gQ1BVIElk
bGUgRHJpdmVycwojCiMgQ09ORklHX0FSTV9DUFVJRExFIGlzIG5vdCBzZXQKQ09ORklHX0FSTV9Q
U0NJX0NQVUlETEU9eQpDT05GSUdfQVJNX1BTQ0lfQ1BVSURMRV9ET01BSU49eQojIGVuZCBvZiBB
Uk0gQ1BVIElkbGUgRHJpdmVycwojIGVuZCBvZiBDUFUgSWRsZQoKIwojIENQVSBGcmVxdWVuY3kg
c2NhbGluZwojCkNPTkZJR19DUFVfRlJFUT15CiMgQ09ORklHX0NQVV9GUkVRX1NUQVQgaXMgbm90
IHNldAojIENPTkZJR19DUFVfRlJFUV9ERUZBVUxUX0dPVl9QRVJGT1JNQU5DRSBpcyBub3Qgc2V0
CiMgQ09ORklHX0NQVV9GUkVRX0RFRkFVTFRfR09WX1BPV0VSU0FWRSBpcyBub3Qgc2V0CkNPTkZJ
R19DUFVfRlJFUV9ERUZBVUxUX0dPVl9VU0VSU1BBQ0U9eQojIENPTkZJR19DUFVfRlJFUV9ERUZB
VUxUX0dPVl9PTkRFTUFORCBpcyBub3Qgc2V0CiMgQ09ORklHX0NQVV9GUkVRX0RFRkFVTFRfR09W
X0NPTlNFUlZBVElWRSBpcyBub3Qgc2V0CiMgQ09ORklHX0NQVV9GUkVRX0RFRkFVTFRfR09WX1ND
SEVEVVRJTCBpcyBub3Qgc2V0CiMgQ09ORklHX0NQVV9GUkVRX0dPVl9QRVJGT1JNQU5DRSBpcyBu
b3Qgc2V0CiMgQ09ORklHX0NQVV9GUkVRX0dPVl9QT1dFUlNBVkUgaXMgbm90IHNldApDT05GSUdf
Q1BVX0ZSRVFfR09WX1VTRVJTUEFDRT15CiMgQ09ORklHX0NQVV9GUkVRX0dPVl9PTkRFTUFORCBp
cyBub3Qgc2V0CiMgQ09ORklHX0NQVV9GUkVRX0dPVl9DT05TRVJWQVRJVkUgaXMgbm90IHNldAoj
IENPTkZJR19DUFVfRlJFUV9HT1ZfU0NIRURVVElMIGlzIG5vdCBzZXQKCiMKIyBDUFUgZnJlcXVl
bmN5IHNjYWxpbmcgZHJpdmVycwojCkNPTkZJR19DUFVGUkVRX0RUPXkKQ09ORklHX0NQVUZSRVFf
RFRfUExBVERFVj15CiMgZW5kIG9mIENQVSBGcmVxdWVuY3kgc2NhbGluZwojIGVuZCBvZiBDUFUg
UG93ZXIgTWFuYWdlbWVudAoKIwojIEZpcm13YXJlIERyaXZlcnMKIwojIENPTkZJR19BUk1fU0NN
SV9QUk9UT0NPTCBpcyBub3Qgc2V0CiMgQ09ORklHX0FSTV9TQ1BJX1BST1RPQ09MIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQVJNX1NERV9JTlRFUkZBQ0UgaXMgbm90IHNldAojIENPTkZJR19GSVJNV0FS
RV9NRU1NQVAgaXMgbm90IHNldAojIENPTkZJR19BUk1fRkZBX1RSQU5TUE9SVCBpcyBub3Qgc2V0
CiMgQ09ORklHX0dPT0dMRV9GSVJNV0FSRSBpcyBub3Qgc2V0CgojCiMgRUZJIChFeHRlbnNpYmxl
IEZpcm13YXJlIEludGVyZmFjZSkgU3VwcG9ydAojCkNPTkZJR19FRklfRVNSVD15CkNPTkZJR19F
RklfUEFSQU1TX0ZST01fRkRUPXkKQ09ORklHX0VGSV9SVU5USU1FX1dSQVBQRVJTPXkKQ09ORklH
X0VGSV9HRU5FUklDX1NUVUI9eQpDT05GSUdfRUZJX0FSTVNUVUJfRFRCX0xPQURFUj15CiMgQ09O
RklHX0VGSV9HRU5FUklDX1NUVUJfSU5JVFJEX0NNRExJTkVfTE9BREVSIGlzIG5vdCBzZXQKIyBD
T05GSUdfRUZJX0JPT1RMT0FERVJfQ09OVFJPTCBpcyBub3Qgc2V0CiMgQ09ORklHX0VGSV9DQVBT
VUxFX0xPQURFUiBpcyBub3Qgc2V0CiMgQ09ORklHX0VGSV9URVNUIGlzIG5vdCBzZXQKIyBDT05G
SUdfUkVTRVRfQVRUQUNLX01JVElHQVRJT04gaXMgbm90IHNldAojIENPTkZJR19FRklfRElTQUJM
RV9QQ0lfRE1BIGlzIG5vdCBzZXQKIyBlbmQgb2YgRUZJIChFeHRlbnNpYmxlIEZpcm13YXJlIElu
dGVyZmFjZSkgU3VwcG9ydAoKQ09ORklHX0VGSV9FQVJMWUNPTj15CkNPTkZJR19BUk1fUFNDSV9G
Vz15CiMgQ09ORklHX0FSTV9QU0NJX0NIRUNLRVIgaXMgbm90IHNldApDT05GSUdfSEFWRV9BUk1f
U01DQ0M9eQpDT05GSUdfSEFWRV9BUk1fU01DQ0NfRElTQ09WRVJZPXkKQ09ORklHX0FSTV9TTUND
Q19TT0NfSUQ9eQoKIwojIFRlZ3JhIGZpcm13YXJlIGRyaXZlcgojCiMgZW5kIG9mIFRlZ3JhIGZp
cm13YXJlIGRyaXZlcgoKIwojIFp5bnEgTVBTb0MgRmlybXdhcmUgRHJpdmVycwojCkNPTkZJR19a
WU5RTVBfRklSTVdBUkU9eQojIENPTkZJR19aWU5RTVBfRklSTVdBUkVfREVCVUcgaXMgbm90IHNl
dAojIGVuZCBvZiBaeW5xIE1QU29DIEZpcm13YXJlIERyaXZlcnMKIyBlbmQgb2YgRmlybXdhcmUg
RHJpdmVycwoKQ09ORklHX0FSQ0hfU1VQUE9SVFNfQUNQST15CiMgQ09ORklHX0FDUEkgaXMgbm90
IHNldAojIENPTkZJR19WSVJUVUFMSVpBVElPTiBpcyBub3Qgc2V0CiMgQ09ORklHX0FSTTY0X0NS
WVBUTyBpcyBub3Qgc2V0CgojCiMgR2VuZXJhbCBhcmNoaXRlY3R1cmUtZGVwZW5kZW50IG9wdGlv
bnMKIwojIENPTkZJR19KVU1QX0xBQkVMIGlzIG5vdCBzZXQKQ09ORklHX0hBVkVfRUZGSUNJRU5U
X1VOQUxJR05FRF9BQ0NFU1M9eQpDT05GSUdfSEFWRV9LUFJPQkVTPXkKQ09ORklHX0hBVkVfS1JF
VFBST0JFUz15CkNPTkZJR19IQVZFX0ZVTkNUSU9OX0VSUk9SX0lOSkVDVElPTj15CkNPTkZJR19I
QVZFX05NST15CkNPTkZJR19IQVZFX0FSQ0hfVFJBQ0VIT09LPXkKQ09ORklHX0hBVkVfRE1BX0NP
TlRJR1VPVVM9eQpDT05GSUdfR0VORVJJQ19TTVBfSURMRV9USFJFQUQ9eQpDT05GSUdfR0VORVJJ
Q19JRExFX1BPTExfU0VUVVA9eQpDT05GSUdfQVJDSF9IQVNfRk9SVElGWV9TT1VSQ0U9eQpDT05G
SUdfQVJDSF9IQVNfS0VFUElOSVRSRD15CkNPTkZJR19BUkNIX0hBU19TRVRfTUVNT1JZPXkKQ09O
RklHX0FSQ0hfSEFTX1NFVF9ESVJFQ1RfTUFQPXkKQ09ORklHX0hBVkVfQVJDSF9USFJFQURfU1RS
VUNUX1dISVRFTElTVD15CkNPTkZJR19BUkNIX1dBTlRTX05PX0lOU1RSPXkKQ09ORklHX0hBVkVf
QVNNX01PRFZFUlNJT05TPXkKQ09ORklHX0hBVkVfUkVHU19BTkRfU1RBQ0tfQUNDRVNTX0FQST15
CkNPTkZJR19IQVZFX1JTRVE9eQpDT05GSUdfSEFWRV9GVU5DVElPTl9BUkdfQUNDRVNTX0FQST15
CkNPTkZJR19IQVZFX0hXX0JSRUFLUE9JTlQ9eQpDT05GSUdfSEFWRV9QRVJGX1JFR1M9eQpDT05G
SUdfSEFWRV9QRVJGX1VTRVJfU1RBQ0tfRFVNUD15CkNPTkZJR19IQVZFX0FSQ0hfSlVNUF9MQUJF
TD15CkNPTkZJR19IQVZFX0FSQ0hfSlVNUF9MQUJFTF9SRUxBVElWRT15CkNPTkZJR19NTVVfR0FU
SEVSX1RBQkxFX0ZSRUU9eQpDT05GSUdfTU1VX0dBVEhFUl9SQ1VfVEFCTEVfRlJFRT15CkNPTkZJ
R19BUkNIX0hBVkVfTk1JX1NBRkVfQ01QWENIRz15CkNPTkZJR19IQVZFX0NNUFhDSEdfTE9DQUw9
eQpDT05GSUdfSEFWRV9DTVBYQ0hHX0RPVUJMRT15CkNPTkZJR19BUkNIX1dBTlRfQ09NUEFUX0lQ
Q19QQVJTRV9WRVJTSU9OPXkKQ09ORklHX0hBVkVfQVJDSF9TRUNDT01QPXkKQ09ORklHX0hBVkVf
QVJDSF9TRUNDT01QX0ZJTFRFUj15CiMgQ09ORklHX1NFQ0NPTVAgaXMgbm90IHNldApDT05GSUdf
SEFWRV9BUkNIX1NUQUNLTEVBSz15CkNPTkZJR19IQVZFX1NUQUNLUFJPVEVDVE9SPXkKQ09ORklH
X1NUQUNLUFJPVEVDVE9SPXkKQ09ORklHX1NUQUNLUFJPVEVDVE9SX1NUUk9ORz15CkNPTkZJR19B
UkNIX1NVUFBPUlRTX0xUT19DTEFORz15CkNPTkZJR19BUkNIX1NVUFBPUlRTX0xUT19DTEFOR19U
SElOPXkKQ09ORklHX0xUT19OT05FPXkKQ09ORklHX0FSQ0hfU1VQUE9SVFNfQ0ZJX0NMQU5HPXkK
Q09ORklHX0hBVkVfQ09OVEVYVF9UUkFDS0lORz15CkNPTkZJR19IQVZFX1ZJUlRfQ1BVX0FDQ09V
TlRJTkdfR0VOPXkKQ09ORklHX0hBVkVfSVJRX1RJTUVfQUNDT1VOVElORz15CkNPTkZJR19IQVZF
X01PVkVfUFVEPXkKQ09ORklHX0hBVkVfTU9WRV9QTUQ9eQpDT05GSUdfSEFWRV9BUkNIX1RSQU5T
UEFSRU5UX0hVR0VQQUdFPXkKQ09ORklHX0hBVkVfQVJDSF9IVUdFX1ZNQVA9eQpDT05GSUdfQVJD
SF9XQU5UX0hVR0VfUE1EX1NIQVJFPXkKQ09ORklHX01PRFVMRVNfVVNFX0VMRl9SRUxBPXkKQ09O
RklHX0FSQ0hfSEFTX0VMRl9SQU5ET01JWkU9eQpDT05GSUdfSEFWRV9BUkNIX01NQVBfUk5EX0JJ
VFM9eQpDT05GSUdfQVJDSF9NTUFQX1JORF9CSVRTPTE4CkNPTkZJR19IQVZFX0FSQ0hfTU1BUF9S
TkRfQ09NUEFUX0JJVFM9eQpDT05GSUdfQVJDSF9NTUFQX1JORF9DT01QQVRfQklUUz0xMQpDT05G
SUdfQVJDSF9XQU5UX0RFRkFVTFRfVE9QRE9XTl9NTUFQX0xBWU9VVD15CkNPTkZJR19DTE9ORV9C
QUNLV0FSRFM9eQpDT05GSUdfT0xEX1NJR1NVU1BFTkQzPXkKQ09ORklHX0NPTVBBVF9PTERfU0lH
QUNUSU9OPXkKQ09ORklHX0NPTVBBVF8zMkJJVF9USU1FPXkKQ09ORklHX0hBVkVfQVJDSF9WTUFQ
X1NUQUNLPXkKQ09ORklHX1ZNQVBfU1RBQ0s9eQpDT05GSUdfSEFWRV9BUkNIX1JBTkRPTUlaRV9L
U1RBQ0tfT0ZGU0VUPXkKIyBDT05GSUdfUkFORE9NSVpFX0tTVEFDS19PRkZTRVRfREVGQVVMVCBp
cyBub3Qgc2V0CkNPTkZJR19BUkNIX0hBU19TVFJJQ1RfS0VSTkVMX1JXWD15CkNPTkZJR19TVFJJ
Q1RfS0VSTkVMX1JXWD15CkNPTkZJR19BUkNIX0hBU19TVFJJQ1RfTU9EVUxFX1JXWD15CkNPTkZJ
R19IQVZFX0FSQ0hfQ09NUElMRVJfSD15CkNPTkZJR19IQVZFX0FSQ0hfUFJFTDMyX1JFTE9DQVRJ
T05TPXkKQ09ORklHX0FSQ0hfVVNFX01FTVJFTUFQX1BST1Q9eQojIENPTkZJR19MT0NLX0VWRU5U
X0NPVU5UUyBpcyBub3Qgc2V0CkNPTkZJR19BUkNIX0hBU19SRUxSPXkKQ09ORklHX0FSQ0hfV0FO
VF9MRF9PUlBIQU5fV0FSTj15CkNPTkZJR19IQVZFX0FSQ0hfUEZOX1ZBTElEPXkKQ09ORklHX0FS
Q0hfU1VQUE9SVFNfREVCVUdfUEFHRUFMTE9DPXkKCiMKIyBHQ09WLWJhc2VkIGtlcm5lbCBwcm9m
aWxpbmcKIwojIENPTkZJR19HQ09WX0tFUk5FTCBpcyBub3Qgc2V0CkNPTkZJR19BUkNIX0hBU19H
Q09WX1BST0ZJTEVfQUxMPXkKIyBlbmQgb2YgR0NPVi1iYXNlZCBrZXJuZWwgcHJvZmlsaW5nCgpD
T05GSUdfSEFWRV9HQ0NfUExVR0lOUz15CiMgZW5kIG9mIEdlbmVyYWwgYXJjaGl0ZWN0dXJlLWRl
cGVuZGVudCBvcHRpb25zCgpDT05GSUdfUlRfTVVURVhFUz15CkNPTkZJR19CQVNFX1NNQUxMPTAK
IyBDT05GSUdfTU9EVUxFUyBpcyBub3Qgc2V0CkNPTkZJR19NT0RVTEVTX1RSRUVfTE9PS1VQPXkK
IyBDT05GSUdfQkxPQ0sgaXMgbm90IHNldApDT05GSUdfQVNOMT15CkNPTkZJR19BUkNIX0lOTElO
RV9TUElOX1RSWUxPQ0s9eQpDT05GSUdfQVJDSF9JTkxJTkVfU1BJTl9UUllMT0NLX0JIPXkKQ09O
RklHX0FSQ0hfSU5MSU5FX1NQSU5fTE9DSz15CkNPTkZJR19BUkNIX0lOTElORV9TUElOX0xPQ0tf
Qkg9eQpDT05GSUdfQVJDSF9JTkxJTkVfU1BJTl9MT0NLX0lSUT15CkNPTkZJR19BUkNIX0lOTElO
RV9TUElOX0xPQ0tfSVJRU0FWRT15CkNPTkZJR19BUkNIX0lOTElORV9TUElOX1VOTE9DSz15CkNP
TkZJR19BUkNIX0lOTElORV9TUElOX1VOTE9DS19CSD15CkNPTkZJR19BUkNIX0lOTElORV9TUElO
X1VOTE9DS19JUlE9eQpDT05GSUdfQVJDSF9JTkxJTkVfU1BJTl9VTkxPQ0tfSVJRUkVTVE9SRT15
CkNPTkZJR19BUkNIX0lOTElORV9SRUFEX0xPQ0s9eQpDT05GSUdfQVJDSF9JTkxJTkVfUkVBRF9M
T0NLX0JIPXkKQ09ORklHX0FSQ0hfSU5MSU5FX1JFQURfTE9DS19JUlE9eQpDT05GSUdfQVJDSF9J
TkxJTkVfUkVBRF9MT0NLX0lSUVNBVkU9eQpDT05GSUdfQVJDSF9JTkxJTkVfUkVBRF9VTkxPQ0s9
eQpDT05GSUdfQVJDSF9JTkxJTkVfUkVBRF9VTkxPQ0tfQkg9eQpDT05GSUdfQVJDSF9JTkxJTkVf
UkVBRF9VTkxPQ0tfSVJRPXkKQ09ORklHX0FSQ0hfSU5MSU5FX1JFQURfVU5MT0NLX0lSUVJFU1RP
UkU9eQpDT05GSUdfQVJDSF9JTkxJTkVfV1JJVEVfTE9DSz15CkNPTkZJR19BUkNIX0lOTElORV9X
UklURV9MT0NLX0JIPXkKQ09ORklHX0FSQ0hfSU5MSU5FX1dSSVRFX0xPQ0tfSVJRPXkKQ09ORklH
X0FSQ0hfSU5MSU5FX1dSSVRFX0xPQ0tfSVJRU0FWRT15CkNPTkZJR19BUkNIX0lOTElORV9XUklU
RV9VTkxPQ0s9eQpDT05GSUdfQVJDSF9JTkxJTkVfV1JJVEVfVU5MT0NLX0JIPXkKQ09ORklHX0FS
Q0hfSU5MSU5FX1dSSVRFX1VOTE9DS19JUlE9eQpDT05GSUdfQVJDSF9JTkxJTkVfV1JJVEVfVU5M
T0NLX0lSUVJFU1RPUkU9eQpDT05GSUdfSU5MSU5FX1NQSU5fVFJZTE9DSz15CkNPTkZJR19JTkxJ
TkVfU1BJTl9UUllMT0NLX0JIPXkKQ09ORklHX0lOTElORV9TUElOX0xPQ0s9eQpDT05GSUdfSU5M
SU5FX1NQSU5fTE9DS19CSD15CkNPTkZJR19JTkxJTkVfU1BJTl9MT0NLX0lSUT15CkNPTkZJR19J
TkxJTkVfU1BJTl9MT0NLX0lSUVNBVkU9eQpDT05GSUdfSU5MSU5FX1NQSU5fVU5MT0NLX0JIPXkK
Q09ORklHX0lOTElORV9TUElOX1VOTE9DS19JUlE9eQpDT05GSUdfSU5MSU5FX1NQSU5fVU5MT0NL
X0lSUVJFU1RPUkU9eQpDT05GSUdfSU5MSU5FX1JFQURfTE9DSz15CkNPTkZJR19JTkxJTkVfUkVB
RF9MT0NLX0JIPXkKQ09ORklHX0lOTElORV9SRUFEX0xPQ0tfSVJRPXkKQ09ORklHX0lOTElORV9S
RUFEX0xPQ0tfSVJRU0FWRT15CkNPTkZJR19JTkxJTkVfUkVBRF9VTkxPQ0s9eQpDT05GSUdfSU5M
SU5FX1JFQURfVU5MT0NLX0JIPXkKQ09ORklHX0lOTElORV9SRUFEX1VOTE9DS19JUlE9eQpDT05G
SUdfSU5MSU5FX1JFQURfVU5MT0NLX0lSUVJFU1RPUkU9eQpDT05GSUdfSU5MSU5FX1dSSVRFX0xP
Q0s9eQpDT05GSUdfSU5MSU5FX1dSSVRFX0xPQ0tfQkg9eQpDT05GSUdfSU5MSU5FX1dSSVRFX0xP
Q0tfSVJRPXkKQ09ORklHX0lOTElORV9XUklURV9MT0NLX0lSUVNBVkU9eQpDT05GSUdfSU5MSU5F
X1dSSVRFX1VOTE9DSz15CkNPTkZJR19JTkxJTkVfV1JJVEVfVU5MT0NLX0JIPXkKQ09ORklHX0lO
TElORV9XUklURV9VTkxPQ0tfSVJRPXkKQ09ORklHX0lOTElORV9XUklURV9VTkxPQ0tfSVJRUkVT
VE9SRT15CkNPTkZJR19BUkNIX1NVUFBPUlRTX0FUT01JQ19STVc9eQpDT05GSUdfTVVURVhfU1BJ
Tl9PTl9PV05FUj15CkNPTkZJR19SV1NFTV9TUElOX09OX09XTkVSPXkKQ09ORklHX0xPQ0tfU1BJ
Tl9PTl9PV05FUj15CkNPTkZJR19BUkNIX1VTRV9RVUVVRURfU1BJTkxPQ0tTPXkKQ09ORklHX1FV
RVVFRF9TUElOTE9DS1M9eQpDT05GSUdfQVJDSF9VU0VfUVVFVUVEX1JXTE9DS1M9eQpDT05GSUdf
UVVFVUVEX1JXTE9DS1M9eQpDT05GSUdfQVJDSF9IQVNfTk9OX09WRVJMQVBQSU5HX0FERFJFU1Nf
U1BBQ0U9eQpDT05GSUdfQVJDSF9IQVNfU1lTQ0FMTF9XUkFQUEVSPXkKQ09ORklHX0ZSRUVaRVI9
eQoKIwojIEV4ZWN1dGFibGUgZmlsZSBmb3JtYXRzCiMKQ09ORklHX0JJTkZNVF9FTEY9eQpDT05G
SUdfQ09NUEFUX0JJTkZNVF9FTEY9eQpDT05GSUdfQVJDSF9CSU5GTVRfRUxGX1NUQVRFPXkKQ09O
RklHX0FSQ0hfSEFWRV9FTEZfUFJPVD15CkNPTkZJR19BUkNIX1VTRV9HTlVfUFJPUEVSVFk9eQpD
T05GSUdfRUxGQ09SRT15CkNPTkZJR19DT1JFX0RVTVBfREVGQVVMVF9FTEZfSEVBREVSUz15CkNP
TkZJR19CSU5GTVRfU0NSSVBUPXkKIyBDT05GSUdfQklORk1UX01JU0MgaXMgbm90IHNldApDT05G
SUdfQ09SRURVTVA9eQojIGVuZCBvZiBFeGVjdXRhYmxlIGZpbGUgZm9ybWF0cwoKIwojIE1lbW9y
eSBNYW5hZ2VtZW50IG9wdGlvbnMKIwpDT05GSUdfU1BBUlNFTUVNPXkKQ09ORklHX1NQQVJTRU1F
TV9FWFRSRU1FPXkKQ09ORklHX1NQQVJTRU1FTV9WTUVNTUFQX0VOQUJMRT15CkNPTkZJR19TUEFS
U0VNRU1fVk1FTU1BUD15CkNPTkZJR19IQVZFX0ZBU1RfR1VQPXkKQ09ORklHX0FSQ0hfS0VFUF9N
RU1CTE9DSz15CkNPTkZJR19NRU1PUllfSVNPTEFUSU9OPXkKQ09ORklHX0FSQ0hfRU5BQkxFX01F
TU9SWV9IT1RQTFVHPXkKQ09ORklHX01FTU9SWV9IT1RQTFVHPXkKQ09ORklHX01FTU9SWV9IT1RQ
TFVHX1NQQVJTRT15CiMgQ09ORklHX01FTU9SWV9IT1RQTFVHX0RFRkFVTFRfT05MSU5FIGlzIG5v
dCBzZXQKQ09ORklHX0FSQ0hfRU5BQkxFX01FTU9SWV9IT1RSRU1PVkU9eQojIENPTkZJR19NRU1P
UllfSE9UUkVNT1ZFIGlzIG5vdCBzZXQKQ09ORklHX01IUF9NRU1NQVBfT05fTUVNT1JZPXkKQ09O
RklHX1NQTElUX1BUTE9DS19DUFVTPTQKQ09ORklHX0FSQ0hfRU5BQkxFX1NQTElUX1BNRF9QVExP
Q0s9eQpDT05GSUdfQ09NUEFDVElPTj15CiMgQ09ORklHX1BBR0VfUkVQT1JUSU5HIGlzIG5vdCBz
ZXQKQ09ORklHX01JR1JBVElPTj15CkNPTkZJR19BUkNIX0VOQUJMRV9IVUdFUEFHRV9NSUdSQVRJ
T049eQpDT05GSUdfQVJDSF9FTkFCTEVfVEhQX01JR1JBVElPTj15CkNPTkZJR19DT05USUdfQUxM
T0M9eQpDT05GSUdfUEhZU19BRERSX1RfNjRCSVQ9eQojIENPTkZJR19LU00gaXMgbm90IHNldApD
T05GSUdfREVGQVVMVF9NTUFQX01JTl9BRERSPTMyNzY4CkNPTkZJR19BUkNIX1NVUFBPUlRTX01F
TU9SWV9GQUlMVVJFPXkKIyBDT05GSUdfTUVNT1JZX0ZBSUxVUkUgaXMgbm90IHNldApDT05GSUdf
VFJBTlNQQVJFTlRfSFVHRVBBR0U9eQojIENPTkZJR19UUkFOU1BBUkVOVF9IVUdFUEFHRV9BTFdB
WVMgaXMgbm90IHNldApDT05GSUdfVFJBTlNQQVJFTlRfSFVHRVBBR0VfTUFEVklTRT15CiMgQ09O
RklHX0NMRUFOQ0FDSEUgaXMgbm90IHNldApDT05GSUdfQ01BPXkKIyBDT05GSUdfQ01BX0RFQlVH
IGlzIG5vdCBzZXQKIyBDT05GSUdfQ01BX0RFQlVHRlMgaXMgbm90IHNldAojIENPTkZJR19DTUFf
U1lTRlMgaXMgbm90IHNldApDT05GSUdfQ01BX0FSRUFTPTcKIyBDT05GSUdfWlBPT0wgaXMgbm90
IHNldAojIENPTkZJR19aU01BTExPQyBpcyBub3Qgc2V0CkNPTkZJR19HRU5FUklDX0VBUkxZX0lP
UkVNQVA9eQojIENPTkZJR19ERUZFUlJFRF9TVFJVQ1RfUEFHRV9JTklUIGlzIG5vdCBzZXQKIyBD
T05GSUdfSURMRV9QQUdFX1RSQUNLSU5HIGlzIG5vdCBzZXQKQ09ORklHX0FSQ0hfSEFTX0NBQ0hF
X0xJTkVfU0laRT15CkNPTkZJR19BUkNIX0hBU19QVEVfREVWTUFQPXkKQ09ORklHX0FSQ0hfSEFT
X1pPTkVfRE1BX1NFVD15CiMgQ09ORklHX1pPTkVfRE1BIGlzIG5vdCBzZXQKQ09ORklHX1pPTkVf
RE1BMzI9eQpDT05GSUdfQVJDSF9VU0VTX0hJR0hfVk1BX0ZMQUdTPXkKIyBDT05GSUdfUEVSQ1BV
X1NUQVRTIGlzIG5vdCBzZXQKIyBDT05GSUdfR1VQX1RFU1QgaXMgbm90IHNldAojIENPTkZJR19S
RUFEX09OTFlfVEhQX0ZPUl9GUyBpcyBub3Qgc2V0CkNPTkZJR19BUkNIX0hBU19QVEVfU1BFQ0lB
TD15CiMgZW5kIG9mIE1lbW9yeSBNYW5hZ2VtZW50IG9wdGlvbnMKCiMgQ09ORklHX05FVCBpcyBu
b3Qgc2V0CgojCiMgRGV2aWNlIERyaXZlcnMKIwpDT05GSUdfQVJNX0FNQkE9eQpDT05GSUdfSEFW
RV9QQ0k9eQojIENPTkZJR19QQ0kgaXMgbm90IHNldAojIENPTkZJR19QQ0NBUkQgaXMgbm90IHNl
dAoKIwojIEdlbmVyaWMgRHJpdmVyIE9wdGlvbnMKIwojIENPTkZJR19VRVZFTlRfSEVMUEVSIGlz
IG5vdCBzZXQKQ09ORklHX0RFVlRNUEZTPXkKQ09ORklHX0RFVlRNUEZTX01PVU5UPXkKQ09ORklH
X1NUQU5EQUxPTkU9eQpDT05GSUdfUFJFVkVOVF9GSVJNV0FSRV9CVUlMRD15CgojCiMgRmlybXdh
cmUgbG9hZGVyCiMKQ09ORklHX0ZXX0xPQURFUj15CkNPTkZJR19FWFRSQV9GSVJNV0FSRT0iIgoj
IENPTkZJR19GV19MT0FERVJfVVNFUl9IRUxQRVIgaXMgbm90IHNldAojIENPTkZJR19GV19MT0FE
RVJfQ09NUFJFU1MgaXMgbm90IHNldApDT05GSUdfRldfQ0FDSEU9eQojIGVuZCBvZiBGaXJtd2Fy
ZSBsb2FkZXIKCkNPTkZJR19XQU5UX0RFVl9DT1JFRFVNUD15CkNPTkZJR19BTExPV19ERVZfQ09S
RURVTVA9eQpDT05GSUdfREVWX0NPUkVEVU1QPXkKIyBDT05GSUdfREVCVUdfRFJJVkVSIGlzIG5v
dCBzZXQKIyBDT05GSUdfREVCVUdfREVWUkVTIGlzIG5vdCBzZXQKIyBDT05GSUdfREVCVUdfVEVT
VF9EUklWRVJfUkVNT1ZFIGlzIG5vdCBzZXQKQ09ORklHX0dFTkVSSUNfQ1BVX0FVVE9QUk9CRT15
CkNPTkZJR19HRU5FUklDX0NQVV9WVUxORVJBQklMSVRJRVM9eQpDT05GSUdfU09DX0JVUz15CkNP
TkZJR19SRUdNQVA9eQpDT05GSUdfUkVHTUFQX0kyQz15CkNPTkZJR19SRUdNQVBfTU1JTz15CkNP
TkZJR19SRUdNQVBfSVJRPXkKQ09ORklHX0RNQV9TSEFSRURfQlVGRkVSPXkKIyBDT05GSUdfRE1B
X0ZFTkNFX1RSQUNFIGlzIG5vdCBzZXQKQ09ORklHX0dFTkVSSUNfQVJDSF9UT1BPTE9HWT15CiMg
ZW5kIG9mIEdlbmVyaWMgRHJpdmVyIE9wdGlvbnMKCiMKIyBCdXMgZGV2aWNlcwojCiMgQ09ORklH
X0JSQ01TVEJfR0lTQl9BUkIgaXMgbm90IHNldAojIENPTkZJR19TSU1QTEVfUE1fQlVTIGlzIG5v
dCBzZXQKIyBDT05GSUdfVkVYUFJFU1NfQ09ORklHIGlzIG5vdCBzZXQKIyBDT05GSUdfTUhJX0JV
UyBpcyBub3Qgc2V0CiMgZW5kIG9mIEJ1cyBkZXZpY2VzCgojIENPTkZJR19HTlNTIGlzIG5vdCBz
ZXQKIyBDT05GSUdfTVREIGlzIG5vdCBzZXQKQ09ORklHX0RUQz15CkNPTkZJR19PRj15CiMgQ09O
RklHX09GX1VOSVRURVNUIGlzIG5vdCBzZXQKQ09ORklHX09GX0ZMQVRUUkVFPXkKQ09ORklHX09G
X0VBUkxZX0ZMQVRUUkVFPXkKQ09ORklHX09GX0tPQko9eQpDT05GSUdfT0ZfRFlOQU1JQz15CkNP
TkZJR19PRl9BRERSRVNTPXkKQ09ORklHX09GX0lSUT15CkNPTkZJR19PRl9SRVNFUlZFRF9NRU09
eQpDT05GSUdfT0ZfUkVTT0xWRT15CkNPTkZJR19PRl9PVkVSTEFZPXkKIyBDT05GSUdfUEFSUE9S
VCBpcyBub3Qgc2V0CgojCiMgTlZNRSBTdXBwb3J0CiMKIyBlbmQgb2YgTlZNRSBTdXBwb3J0Cgoj
CiMgTWlzYyBkZXZpY2VzCiMKIyBDT05GSUdfQUQ1MjVYX0RQT1QgaXMgbm90IHNldAojIENPTkZJ
R19EVU1NWV9JUlEgaXMgbm90IHNldAojIENPTkZJR19JQ1M5MzJTNDAxIGlzIG5vdCBzZXQKIyBD
T05GSUdfRU5DTE9TVVJFX1NFUlZJQ0VTIGlzIG5vdCBzZXQKIyBDT05GSUdfQVBEUzk4MDJBTFMg
aXMgbm90IHNldAojIENPTkZJR19JU0wyOTAwMyBpcyBub3Qgc2V0CiMgQ09ORklHX0lTTDI5MDIw
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19UU0wyNTUwIGlzIG5vdCBzZXQKIyBDT05GSUdf
U0VOU09SU19CSDE3NzAgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0FQRFM5OTBYIGlzIG5v
dCBzZXQKIyBDT05GSUdfSE1DNjM1MiBpcyBub3Qgc2V0CiMgQ09ORklHX0RTMTY4MiBpcyBub3Qg
c2V0CiMgQ09ORklHX1NSQU0gaXMgbm90IHNldApDT05GSUdfWElMSU5YX1NERkVDPXkKIyBDT05G
SUdfQzJQT1JUIGlzIG5vdCBzZXQKCiMKIyBFRVBST00gc3VwcG9ydAojCkNPTkZJR19FRVBST01f
QVQyND15CiMgQ09ORklHX0VFUFJPTV9MRUdBQ1kgaXMgbm90IHNldAojIENPTkZJR19FRVBST01f
TUFYNjg3NSBpcyBub3Qgc2V0CiMgQ09ORklHX0VFUFJPTV85M0NYNiBpcyBub3Qgc2V0CiMgQ09O
RklHX0VFUFJPTV9JRFRfODlIUEVTWCBpcyBub3Qgc2V0CiMgQ09ORklHX0VFUFJPTV9FRTEwMDQg
aXMgbm90IHNldAojIGVuZCBvZiBFRVBST00gc3VwcG9ydAoKIwojIFRleGFzIEluc3RydW1lbnRz
IHNoYXJlZCB0cmFuc3BvcnQgbGluZSBkaXNjaXBsaW5lCiMKIyBlbmQgb2YgVGV4YXMgSW5zdHJ1
bWVudHMgc2hhcmVkIHRyYW5zcG9ydCBsaW5lIGRpc2NpcGxpbmUKCiMgQ09ORklHX1NFTlNPUlNf
TElTM19JMkMgaXMgbm90IHNldAojIENPTkZJR19BTFRFUkFfU1RBUEwgaXMgbm90IHNldAojIENP
TkZJR19FQ0hPIGlzIG5vdCBzZXQKIyBDT05GSUdfVUFDQ0UgaXMgbm90IHNldAojIENPTkZJR19Q
VlBBTklDIGlzIG5vdCBzZXQKIyBlbmQgb2YgTWlzYyBkZXZpY2VzCgojCiMgU0NTSSBkZXZpY2Ug
c3VwcG9ydAojCkNPTkZJR19TQ1NJX01PRD15CiMgZW5kIG9mIFNDU0kgZGV2aWNlIHN1cHBvcnQK
CkNPTkZJR19IQVZFX1BBVEFfUExBVEZPUk09eQoKIwojIElucHV0IGRldmljZSBzdXBwb3J0CiMK
Q09ORklHX0lOUFVUPXkKQ09ORklHX0lOUFVUX0xFRFM9eQojIENPTkZJR19JTlBVVF9GRl9NRU1M
RVNTIGlzIG5vdCBzZXQKIyBDT05GSUdfSU5QVVRfU1BBUlNFS01BUCBpcyBub3Qgc2V0CiMgQ09O
RklHX0lOUFVUX01BVFJJWEtNQVAgaXMgbm90IHNldAoKIwojIFVzZXJsYW5kIGludGVyZmFjZXMK
IwojIENPTkZJR19JTlBVVF9NT1VTRURFViBpcyBub3Qgc2V0CiMgQ09ORklHX0lOUFVUX0pPWURF
ViBpcyBub3Qgc2V0CkNPTkZJR19JTlBVVF9FVkRFVj15CiMgQ09ORklHX0lOUFVUX0VWQlVHIGlz
IG5vdCBzZXQKCiMKIyBJbnB1dCBEZXZpY2UgRHJpdmVycwojCkNPTkZJR19JTlBVVF9LRVlCT0FS
RD15CiMgQ09ORklHX0tFWUJPQVJEX0FEQyBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWUJPQVJEX0FE
UDU1ODggaXMgbm90IHNldAojIENPTkZJR19LRVlCT0FSRF9BRFA1NTg5IGlzIG5vdCBzZXQKQ09O
RklHX0tFWUJPQVJEX0FUS0JEPXkKIyBDT05GSUdfS0VZQk9BUkRfUVQxMDUwIGlzIG5vdCBzZXQK
IyBDT05GSUdfS0VZQk9BUkRfUVQxMDcwIGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfUVQy
MTYwIGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfRExJTktfRElSNjg1IGlzIG5vdCBzZXQK
IyBDT05GSUdfS0VZQk9BUkRfTEtLQkQgaXMgbm90IHNldApDT05GSUdfS0VZQk9BUkRfR1BJTz15
CkNPTkZJR19LRVlCT0FSRF9HUElPX1BPTExFRD15CiMgQ09ORklHX0tFWUJPQVJEX1RDQTY0MTYg
aXMgbm90IHNldAojIENPTkZJR19LRVlCT0FSRF9UQ0E4NDE4IGlzIG5vdCBzZXQKIyBDT05GSUdf
S0VZQk9BUkRfTUFUUklYIGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfTE04MzIzIGlzIG5v
dCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfTE04MzMzIGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9B
UkRfTUFYNzM1OSBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWUJPQVJEX01DUyBpcyBub3Qgc2V0CiMg
Q09ORklHX0tFWUJPQVJEX01QUjEyMSBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWUJPQVJEX05FV1RP
TiBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWUJPQVJEX09QRU5DT1JFUyBpcyBub3Qgc2V0CiMgQ09O
RklHX0tFWUJPQVJEX1NBTVNVTkcgaXMgbm90IHNldAojIENPTkZJR19LRVlCT0FSRF9TVE9XQVdB
WSBpcyBub3Qgc2V0CiMgQ09ORklHX0tFWUJPQVJEX1NVTktCRCBpcyBub3Qgc2V0CiMgQ09ORklH
X0tFWUJPQVJEX09NQVA0IGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfVE0yX1RPVUNIS0VZ
IGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfWFRLQkQgaXMgbm90IHNldAojIENPTkZJR19L
RVlCT0FSRF9DQVAxMVhYIGlzIG5vdCBzZXQKIyBDT05GSUdfS0VZQk9BUkRfQkNNIGlzIG5vdCBz
ZXQKQ09ORklHX0lOUFVUX01PVVNFPXkKQ09ORklHX01PVVNFX1BTMj15CkNPTkZJR19NT1VTRV9Q
UzJfQUxQUz15CkNPTkZJR19NT1VTRV9QUzJfQllEPXkKQ09ORklHX01PVVNFX1BTMl9MT0dJUFMy
UFA9eQpDT05GSUdfTU9VU0VfUFMyX1NZTkFQVElDUz15CkNPTkZJR19NT1VTRV9QUzJfU1lOQVBU
SUNTX1NNQlVTPXkKQ09ORklHX01PVVNFX1BTMl9DWVBSRVNTPXkKQ09ORklHX01PVVNFX1BTMl9U
UkFDS1BPSU5UPXkKIyBDT05GSUdfTU9VU0VfUFMyX0VMQU5URUNIIGlzIG5vdCBzZXQKIyBDT05G
SUdfTU9VU0VfUFMyX1NFTlRFTElDIGlzIG5vdCBzZXQKIyBDT05GSUdfTU9VU0VfUFMyX1RPVUNI
S0lUIGlzIG5vdCBzZXQKQ09ORklHX01PVVNFX1BTMl9GT0NBTFRFQ0g9eQpDT05GSUdfTU9VU0Vf
UFMyX1NNQlVTPXkKIyBDT05GSUdfTU9VU0VfU0VSSUFMIGlzIG5vdCBzZXQKIyBDT05GSUdfTU9V
U0VfQ1lBUEEgaXMgbm90IHNldAojIENPTkZJR19NT1VTRV9FTEFOX0kyQyBpcyBub3Qgc2V0CiMg
Q09ORklHX01PVVNFX1ZTWFhYQUEgaXMgbm90IHNldAojIENPTkZJR19NT1VTRV9HUElPIGlzIG5v
dCBzZXQKIyBDT05GSUdfTU9VU0VfU1lOQVBUSUNTX0kyQyBpcyBub3Qgc2V0CiMgQ09ORklHX0lO
UFVUX0pPWVNUSUNLIGlzIG5vdCBzZXQKIyBDT05GSUdfSU5QVVRfVEFCTEVUIGlzIG5vdCBzZXQK
IyBDT05GSUdfSU5QVVRfVE9VQ0hTQ1JFRU4gaXMgbm90IHNldAojIENPTkZJR19JTlBVVF9NSVND
IGlzIG5vdCBzZXQKIyBDT05GSUdfUk1JNF9DT1JFIGlzIG5vdCBzZXQKCiMKIyBIYXJkd2FyZSBJ
L08gcG9ydHMKIwpDT05GSUdfU0VSSU89eQpDT05GSUdfU0VSSU9fU0VSUE9SVD15CiMgQ09ORklH
X1NFUklPX0FNQkFLTUkgaXMgbm90IHNldApDT05GSUdfU0VSSU9fTElCUFMyPXkKIyBDT05GSUdf
U0VSSU9fUkFXIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VSSU9fQUxURVJBX1BTMiBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFUklPX1BTMk1VTFQgaXMgbm90IHNldAojIENPTkZJR19TRVJJT19BUkNfUFMy
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VSSU9fQVBCUFMyIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VS
SU9fR1BJT19QUzIgaXMgbm90IHNldAojIENPTkZJR19VU0VSSU8gaXMgbm90IHNldAojIENPTkZJ
R19HQU1FUE9SVCBpcyBub3Qgc2V0CiMgZW5kIG9mIEhhcmR3YXJlIEkvTyBwb3J0cwojIGVuZCBv
ZiBJbnB1dCBkZXZpY2Ugc3VwcG9ydAoKIwojIENoYXJhY3RlciBkZXZpY2VzCiMKQ09ORklHX1RU
WT15CkNPTkZJR19WVD15CkNPTkZJR19DT05TT0xFX1RSQU5TTEFUSU9OUz15CkNPTkZJR19WVF9D
T05TT0xFPXkKQ09ORklHX1ZUX0NPTlNPTEVfU0xFRVA9eQpDT05GSUdfSFdfQ09OU09MRT15CkNP
TkZJR19WVF9IV19DT05TT0xFX0JJTkRJTkc9eQpDT05GSUdfVU5JWDk4X1BUWVM9eQojIENPTkZJ
R19MRUdBQ1lfUFRZUyBpcyBub3Qgc2V0CkNPTkZJR19MRElTQ19BVVRPTE9BRD15CgojCiMgU2Vy
aWFsIGRyaXZlcnMKIwpDT05GSUdfU0VSSUFMX0VBUkxZQ09OPXkKQ09ORklHX1NFUklBTF84MjUw
PXkKQ09ORklHX1NFUklBTF84MjUwX0RFUFJFQ0FURURfT1BUSU9OUz15CkNPTkZJR19TRVJJQUxf
ODI1MF8xNjU1MEFfVkFSSUFOVFM9eQojIENPTkZJR19TRVJJQUxfODI1MF9GSU5URUsgaXMgbm90
IHNldApDT05GSUdfU0VSSUFMXzgyNTBfQ09OU09MRT15CkNPTkZJR19TRVJJQUxfODI1MF9ETUE9
eQpDT05GSUdfU0VSSUFMXzgyNTBfTlJfVUFSVFM9NApDT05GSUdfU0VSSUFMXzgyNTBfUlVOVElN
RV9VQVJUUz00CkNPTkZJR19TRVJJQUxfODI1MF9FWFRFTkRFRD15CiMgQ09ORklHX1NFUklBTF84
MjUwX01BTllfUE9SVFMgaXMgbm90IHNldApDT05GSUdfU0VSSUFMXzgyNTBfU0hBUkVfSVJRPXkK
IyBDT05GSUdfU0VSSUFMXzgyNTBfREVURUNUX0lSUSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFUklB
TF84MjUwX1JTQSBpcyBub3Qgc2V0CkNPTkZJR19TRVJJQUxfODI1MF9EV0xJQj15CkNPTkZJR19T
RVJJQUxfODI1MF9GU0w9eQpDT05GSUdfU0VSSUFMXzgyNTBfRFc9eQojIENPTkZJR19TRVJJQUxf
ODI1MF9SVDI4OFggaXMgbm90IHNldApDT05GSUdfU0VSSUFMX09GX1BMQVRGT1JNPXkKCiMKIyBO
b24tODI1MCBzZXJpYWwgcG9ydCBzdXBwb3J0CiMKIyBDT05GSUdfU0VSSUFMX0FNQkFfUEwwMTAg
aXMgbm90IHNldAojIENPTkZJR19TRVJJQUxfQU1CQV9QTDAxMSBpcyBub3Qgc2V0CiMgQ09ORklH
X1NFUklBTF9FQVJMWUNPTl9BUk1fU0VNSUhPU1QgaXMgbm90IHNldApDT05GSUdfU0VSSUFMX1VB
UlRMSVRFPXkKQ09ORklHX1NFUklBTF9VQVJUTElURV9DT05TT0xFPXkKQ09ORklHX1NFUklBTF9V
QVJUTElURV9OUl9VQVJUUz04CkNPTkZJR19TRVJJQUxfQ09SRT15CkNPTkZJR19TRVJJQUxfQ09S
RV9DT05TT0xFPXkKIyBDT05GSUdfU0VSSUFMX1NJRklWRSBpcyBub3Qgc2V0CiMgQ09ORklHX1NF
UklBTF9TQ0NOWFAgaXMgbm90IHNldAojIENPTkZJR19TRVJJQUxfU0MxNklTN1hYIGlzIG5vdCBz
ZXQKIyBDT05GSUdfU0VSSUFMX0JDTTYzWFggaXMgbm90IHNldAojIENPTkZJR19TRVJJQUxfQUxU
RVJBX0pUQUdVQVJUIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VSSUFMX0FMVEVSQV9VQVJUIGlzIG5v
dCBzZXQKQ09ORklHX1NFUklBTF9YSUxJTlhfUFNfVUFSVD15CkNPTkZJR19TRVJJQUxfWElMSU5Y
X1BTX1VBUlRfQ09OU09MRT15CiMgQ09ORklHX1NFUklBTF9BUkMgaXMgbm90IHNldAojIENPTkZJ
R19TRVJJQUxfRlNMX0xQVUFSVCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFUklBTF9GU0xfTElORkxF
WFVBUlQgaXMgbm90IHNldAojIENPTkZJR19TRVJJQUxfQ09ORVhBTlRfRElHSUNPTE9SIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VSSUFMX1NQUkQgaXMgbm90IHNldAojIGVuZCBvZiBTZXJpYWwgZHJp
dmVycwoKQ09ORklHX1NFUklBTF9NQ1RSTF9HUElPPXkKIyBDT05GSUdfU0VSSUFMX05PTlNUQU5E
QVJEIGlzIG5vdCBzZXQKIyBDT05GSUdfTlVMTF9UVFkgaXMgbm90IHNldAojIENPTkZJR19IVkNf
RENDIGlzIG5vdCBzZXQKQ09ORklHX1NFUklBTF9ERVZfQlVTPXkKQ09ORklHX1NFUklBTF9ERVZf
Q1RSTF9UVFlQT1JUPXkKIyBDT05GSUdfVFRZX1BSSU5USyBpcyBub3Qgc2V0CiMgQ09ORklHX1ZJ
UlRJT19DT05TT0xFIGlzIG5vdCBzZXQKIyBDT05GSUdfSVBNSV9IQU5ETEVSIGlzIG5vdCBzZXQK
IyBDT05GSUdfSVBNQl9ERVZJQ0VfSU5URVJGQUNFIGlzIG5vdCBzZXQKIyBDT05GSUdfSFdfUkFO
RE9NIGlzIG5vdCBzZXQKQ09ORklHX0RFVk1FTT15CkNPTkZJR19UQ0dfVFBNPXkKIyBDT05GSUdf
VENHX1RJUyBpcyBub3Qgc2V0CiMgQ09ORklHX1RDR19USVNfSTJDX0NSNTAgaXMgbm90IHNldAoj
IENPTkZJR19UQ0dfVElTX0kyQ19BVE1FTCBpcyBub3Qgc2V0CiMgQ09ORklHX1RDR19USVNfSTJD
X0lORklORU9OIGlzIG5vdCBzZXQKIyBDT05GSUdfVENHX1RJU19JMkNfTlVWT1RPTiBpcyBub3Qg
c2V0CiMgQ09ORklHX1RDR19WVFBNX1BST1hZIGlzIG5vdCBzZXQKIyBDT05GSUdfVENHX1RJU19T
VDMzWlAyNF9JMkMgaXMgbm90IHNldAojIENPTkZJR19YSUxMWUJVUyBpcyBub3Qgc2V0CiMgZW5k
IG9mIENoYXJhY3RlciBkZXZpY2VzCgojIENPTkZJR19SQU5ET01fVFJVU1RfQ1BVIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUkFORE9NX1RSVVNUX0JPT1RMT0FERVIgaXMgbm90IHNldAoKIwojIEkyQyBz
dXBwb3J0CiMKQ09ORklHX0kyQz15CkNPTkZJR19JMkNfQk9BUkRJTkZPPXkKQ09ORklHX0kyQ19D
T01QQVQ9eQpDT05GSUdfSTJDX0NIQVJERVY9eQpDT05GSUdfSTJDX01VWD15CgojCiMgTXVsdGlw
bGV4ZXIgSTJDIENoaXAgc3VwcG9ydAojCiMgQ09ORklHX0kyQ19BUkJfR1BJT19DSEFMTEVOR0Ug
aXMgbm90IHNldAojIENPTkZJR19JMkNfTVVYX0dQSU8gaXMgbm90IHNldAojIENPTkZJR19JMkNf
TVVYX0dQTVVYIGlzIG5vdCBzZXQKIyBDT05GSUdfSTJDX01VWF9MVEM0MzA2IGlzIG5vdCBzZXQK
IyBDT05GSUdfSTJDX01VWF9QQ0E5NTQxIGlzIG5vdCBzZXQKQ09ORklHX0kyQ19NVVhfUENBOTU0
eD15CiMgQ09ORklHX0kyQ19NVVhfUkVHIGlzIG5vdCBzZXQKIyBDT05GSUdfSTJDX01VWF9NTFhD
UExEIGlzIG5vdCBzZXQKIyBlbmQgb2YgTXVsdGlwbGV4ZXIgSTJDIENoaXAgc3VwcG9ydAoKQ09O
RklHX0kyQ19IRUxQRVJfQVVUTz15CgojCiMgSTJDIEhhcmR3YXJlIEJ1cyBzdXBwb3J0CiMKCiMK
IyBJMkMgc3lzdGVtIGJ1cyBkcml2ZXJzIChtb3N0bHkgZW1iZWRkZWQgLyBzeXN0ZW0tb24tY2hp
cCkKIwpDT05GSUdfSTJDX0NBREVOQ0U9eQojIENPTkZJR19JMkNfQ0JVU19HUElPIGlzIG5vdCBz
ZXQKIyBDT05GSUdfSTJDX0RFU0lHTldBUkVfUExBVEZPUk0gaXMgbm90IHNldAojIENPTkZJR19J
MkNfRU1FVjIgaXMgbm90IHNldAojIENPTkZJR19JMkNfR1BJTyBpcyBub3Qgc2V0CiMgQ09ORklH
X0kyQ19OT01BRElLIGlzIG5vdCBzZXQKIyBDT05GSUdfSTJDX09DT1JFUyBpcyBub3Qgc2V0CiMg
Q09ORklHX0kyQ19QQ0FfUExBVEZPUk0gaXMgbm90IHNldAojIENPTkZJR19JMkNfUkszWCBpcyBu
b3Qgc2V0CiMgQ09ORklHX0kyQ19TSU1URUMgaXMgbm90IHNldApDT05GSUdfSTJDX1hJTElOWD15
CgojCiMgRXh0ZXJuYWwgSTJDL1NNQnVzIGFkYXB0ZXIgZHJpdmVycwojCiMgQ09ORklHX0kyQ19U
QU9TX0VWTSBpcyBub3Qgc2V0CgojCiMgT3RoZXIgSTJDL1NNQnVzIGJ1cyBkcml2ZXJzCiMKIyBl
bmQgb2YgSTJDIEhhcmR3YXJlIEJ1cyBzdXBwb3J0CgpDT05GSUdfSTJDX1NMQVZFPXkKQ09ORklH
X0kyQ19TTEFWRV9FRVBST009eQojIENPTkZJR19JMkNfU0xBVkVfVEVTVFVOSVQgaXMgbm90IHNl
dAojIENPTkZJR19JMkNfREVCVUdfQ09SRSBpcyBub3Qgc2V0CiMgQ09ORklHX0kyQ19ERUJVR19B
TEdPIGlzIG5vdCBzZXQKIyBDT05GSUdfSTJDX0RFQlVHX0JVUyBpcyBub3Qgc2V0CiMgZW5kIG9m
IEkyQyBzdXBwb3J0CgojIENPTkZJR19JM0MgaXMgbm90IHNldAojIENPTkZJR19TUEkgaXMgbm90
IHNldAojIENPTkZJR19TUE1JIGlzIG5vdCBzZXQKIyBDT05GSUdfSFNJIGlzIG5vdCBzZXQKIyBD
T05GSUdfUFBTIGlzIG5vdCBzZXQKCiMKIyBQVFAgY2xvY2sgc3VwcG9ydAojCgojCiMgRW5hYmxl
IFBIWUxJQiBhbmQgTkVUV09SS19QSFlfVElNRVNUQU1QSU5HIHRvIHNlZSB0aGUgYWRkaXRpb25h
bCBjbG9ja3MuCiMKIyBlbmQgb2YgUFRQIGNsb2NrIHN1cHBvcnQKCiMgQ09ORklHX1BJTkNUUkwg
aXMgbm90IHNldApDT05GSUdfR1BJT0xJQj15CkNPTkZJR19HUElPTElCX0ZBU1RQQVRIX0xJTUlU
PTUxMgpDT05GSUdfT0ZfR1BJTz15CkNPTkZJR19HUElPTElCX0lSUUNISVA9eQojIENPTkZJR19E
RUJVR19HUElPIGlzIG5vdCBzZXQKQ09ORklHX0dQSU9fU1lTRlM9eQpDT05GSUdfR1BJT19DREVW
PXkKQ09ORklHX0dQSU9fQ0RFVl9WMT15CgojCiMgTWVtb3J5IG1hcHBlZCBHUElPIGRyaXZlcnMK
IwojIENPTkZJR19HUElPXzc0WFhfTU1JTyBpcyBub3Qgc2V0CiMgQ09ORklHX0dQSU9fQUxURVJB
IGlzIG5vdCBzZXQKIyBDT05GSUdfR1BJT19DQURFTkNFIGlzIG5vdCBzZXQKIyBDT05GSUdfR1BJ
T19EV0FQQiBpcyBub3Qgc2V0CiMgQ09ORklHX0dQSU9fRlRHUElPMDEwIGlzIG5vdCBzZXQKIyBD
T05GSUdfR1BJT19HRU5FUklDX1BMQVRGT1JNIGlzIG5vdCBzZXQKIyBDT05GSUdfR1BJT19HUkdQ
SU8gaXMgbm90IHNldAojIENPTkZJR19HUElPX0hMV0QgaXMgbm90IHNldAojIENPTkZJR19HUElP
X01CODZTN1ggaXMgbm90IHNldAojIENPTkZJR19HUElPX1BMMDYxIGlzIG5vdCBzZXQKIyBDT05G
SUdfR1BJT19TSUZJVkUgaXMgbm90IHNldAojIENPTkZJR19HUElPX1hHRU5FIGlzIG5vdCBzZXQK
Q09ORklHX0dQSU9fWElMSU5YPXkKQ09ORklHX0dQSU9fWllOUT15CiMgQ09ORklHX0dQSU9fQU1E
X0ZDSCBpcyBub3Qgc2V0CiMgZW5kIG9mIE1lbW9yeSBtYXBwZWQgR1BJTyBkcml2ZXJzCgojCiMg
STJDIEdQSU8gZXhwYW5kZXJzCiMKIyBDT05GSUdfR1BJT19BRFA1NTg4IGlzIG5vdCBzZXQKIyBD
T05GSUdfR1BJT19BRE5QIGlzIG5vdCBzZXQKIyBDT05GSUdfR1BJT19HV19QTEQgaXMgbm90IHNl
dAojIENPTkZJR19HUElPX01BWDczMDAgaXMgbm90IHNldAojIENPTkZJR19HUElPX01BWDczMlgg
aXMgbm90IHNldApDT05GSUdfR1BJT19QQ0E5NTNYPXkKIyBDT05GSUdfR1BJT19QQ0E5NTNYX0lS
USBpcyBub3Qgc2V0CiMgQ09ORklHX0dQSU9fUENBOTU3MCBpcyBub3Qgc2V0CiMgQ09ORklHX0dQ
SU9fUENGODU3WCBpcyBub3Qgc2V0CiMgQ09ORklHX0dQSU9fVFBJQzI4MTAgaXMgbm90IHNldAoj
IGVuZCBvZiBJMkMgR1BJTyBleHBhbmRlcnMKCiMKIyBNRkQgR1BJTyBleHBhbmRlcnMKIwpDT05G
SUdfR1BJT19UUFM2NTA4Nj15CiMgZW5kIG9mIE1GRCBHUElPIGV4cGFuZGVycwoKIwojIFZpcnR1
YWwgR1BJTyBkcml2ZXJzCiMKIyBDT05GSUdfR1BJT19BR0dSRUdBVE9SIGlzIG5vdCBzZXQKIyBD
T05GSUdfR1BJT19NT0NLVVAgaXMgbm90IHNldAojIGVuZCBvZiBWaXJ0dWFsIEdQSU8gZHJpdmVy
cwoKIyBDT05GSUdfVzEgaXMgbm90IHNldApDT05GSUdfUE9XRVJfUkVTRVQ9eQojIENPTkZJR19Q
T1dFUl9SRVNFVF9HUElPIGlzIG5vdCBzZXQKIyBDT05GSUdfUE9XRVJfUkVTRVRfR1BJT19SRVNU
QVJUIGlzIG5vdCBzZXQKQ09ORklHX1BPV0VSX1JFU0VUX0xUQzI5NTI9eQojIENPTkZJR19QT1dF
Ul9SRVNFVF9SRUdVTEFUT1IgaXMgbm90IHNldAojIENPTkZJR19QT1dFUl9SRVNFVF9SRVNUQVJU
IGlzIG5vdCBzZXQKIyBDT05GSUdfUE9XRVJfUkVTRVRfWEdFTkUgaXMgbm90IHNldAojIENPTkZJ
R19QT1dFUl9SRVNFVF9TWVNDT04gaXMgbm90IHNldAojIENPTkZJR19QT1dFUl9SRVNFVF9TWVND
T05fUE9XRVJPRkYgaXMgbm90IHNldAojIENPTkZJR19OVk1FTV9SRUJPT1RfTU9ERSBpcyBub3Qg
c2V0CkNPTkZJR19QT1dFUl9TVVBQTFk9eQojIENPTkZJR19QT1dFUl9TVVBQTFlfREVCVUcgaXMg
bm90IHNldApDT05GSUdfUE9XRVJfU1VQUExZX0hXTU9OPXkKIyBDT05GSUdfUERBX1BPV0VSIGlz
IG5vdCBzZXQKIyBDT05GSUdfR0VORVJJQ19BRENfQkFUVEVSWSBpcyBub3Qgc2V0CiMgQ09ORklH
X1RFU1RfUE9XRVIgaXMgbm90IHNldAojIENPTkZJR19DSEFSR0VSX0FEUDUwNjEgaXMgbm90IHNl
dAojIENPTkZJR19CQVRURVJZX0NXMjAxNSBpcyBub3Qgc2V0CiMgQ09ORklHX0JBVFRFUllfRFMy
NzgwIGlzIG5vdCBzZXQKIyBDT05GSUdfQkFUVEVSWV9EUzI3ODEgaXMgbm90IHNldAojIENPTkZJ
R19CQVRURVJZX0RTMjc4MiBpcyBub3Qgc2V0CiMgQ09ORklHX0JBVFRFUllfU0JTIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQ0hBUkdFUl9TQlMgaXMgbm90IHNldAojIENPTkZJR19NQU5BR0VSX1NCUyBp
cyBub3Qgc2V0CiMgQ09ORklHX0JBVFRFUllfQlEyN1hYWCBpcyBub3Qgc2V0CiMgQ09ORklHX0JB
VFRFUllfTUFYMTcwNDAgaXMgbm90IHNldAojIENPTkZJR19CQVRURVJZX01BWDE3MDQyIGlzIG5v
dCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9NQVg4OTAzIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdF
Ul9MUDg3MjcgaXMgbm90IHNldAojIENPTkZJR19DSEFSR0VSX0dQSU8gaXMgbm90IHNldAojIENP
TkZJR19DSEFSR0VSX01BTkFHRVIgaXMgbm90IHNldAojIENPTkZJR19DSEFSR0VSX0xUMzY1MSBp
cyBub3Qgc2V0CiMgQ09ORklHX0NIQVJHRVJfTFRDNDE2MkwgaXMgbm90IHNldAojIENPTkZJR19D
SEFSR0VSX0RFVEVDVE9SX01BWDE0NjU2IGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI0
MTVYIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI0MTkwIGlzIG5vdCBzZXQKIyBDT05G
SUdfQ0hBUkdFUl9CUTI0MjU3IGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI0NzM1IGlz
IG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI1MTVYIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hB
UkdFUl9CUTI1ODkwIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI1OTgwIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQ0hBUkdFUl9CUTI1NlhYIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hBUkdFUl9T
TUIzNDcgaXMgbm90IHNldAojIENPTkZJR19CQVRURVJZX0dBVUdFX0xUQzI5NDEgaXMgbm90IHNl
dAojIENPTkZJR19CQVRURVJZX0dPTERGSVNIIGlzIG5vdCBzZXQKIyBDT05GSUdfQkFUVEVSWV9S
VDUwMzMgaXMgbm90IHNldAojIENPTkZJR19DSEFSR0VSX1JUOTQ1NSBpcyBub3Qgc2V0CiMgQ09O
RklHX0NIQVJHRVJfVUNTMTAwMiBpcyBub3Qgc2V0CiMgQ09ORklHX0NIQVJHRVJfQkQ5OTk1NCBp
cyBub3Qgc2V0CkNPTkZJR19IV01PTj15CiMgQ09ORklHX0hXTU9OX0RFQlVHX0NISVAgaXMgbm90
IHNldAoKIwojIE5hdGl2ZSBkcml2ZXJzCiMKIyBDT05GSUdfU0VOU09SU19BRDc0MTQgaXMgbm90
IHNldAojIENPTkZJR19TRU5TT1JTX0FENzQxOCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNf
QURNMTAyMSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURNMTAyNSBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfQURNMTAyNiBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURNMTAy
OSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURNMTAzMSBpcyBub3Qgc2V0CiMgQ09ORklH
X1NFTlNPUlNfQURNMTE3NyBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURNOTI0MCBpcyBu
b3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURUNzQxMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNP
UlNfQURUNzQxMSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURUNzQ2MiBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFTlNPUlNfQURUNzQ3MCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURU
NzQ3NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQUhUMTAgaXMgbm90IHNldAojIENPTkZJ
R19TRU5TT1JTX0FTMzcwIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19BU0M3NjIxIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19BWElfRkFOX0NPTlRST0wgaXMgbm90IHNldAojIENPTkZJ
R19TRU5TT1JTX0FTUEVFRCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQVRYUDEgaXMgbm90
IHNldAojIENPTkZJR19TRU5TT1JTX0NPUlNBSVJfQ1BSTyBpcyBub3Qgc2V0CiMgQ09ORklHX1NF
TlNPUlNfQ09SU0FJUl9QU1UgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0RTNjIwIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19EUzE2MjEgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JT
X0Y3MTgwNUYgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0Y3MTg4MkZHIGlzIG5vdCBzZXQK
IyBDT05GSUdfU0VOU09SU19GNzUzNzVTIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19GVFNU
RVVUQVRFUyBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfR0w1MThTTSBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfR0w1MjBTTSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfRzc2MEEg
aXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0c3NjIgaXMgbm90IHNldAojIENPTkZJR19TRU5T
T1JTX0dQSU9fRkFOIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19ISUg2MTMwIGlzIG5vdCBz
ZXQKQ09ORklHX1NFTlNPUlNfSUlPX0hXTU9OPXkKIyBDT05GSUdfU0VOU09SU19JVDg3IGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19KQzQyIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19Q
T1dSMTIyMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTElORUFHRSBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfTFRDMjk0NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTFRDMjk0
N19JMkMgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0xUQzI5OTAgaXMgbm90IHNldAojIENP
TkZJR19TRU5TT1JTX0xUQzI5OTIgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0xUQzQxNTEg
aXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0xUQzQyMTUgaXMgbm90IHNldAojIENPTkZJR19T
RU5TT1JTX0xUQzQyMjIgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0xUQzQyNDUgaXMgbm90
IHNldAojIENPTkZJR19TRU5TT1JTX0xUQzQyNjAgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JT
X0xUQzQyNjEgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDEyNyBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfTUFYMTYwNjUgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDE2
MTkgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDE2NjggaXMgbm90IHNldAojIENPTkZJ
R19TRU5TT1JTX01BWDE5NyBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTUFYMzE3MzAgaXMg
bm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDY2MjEgaXMgbm90IHNldAojIENPTkZJR19TRU5T
T1JTX01BWDY2MzkgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDY2NDIgaXMgbm90IHNl
dAojIENPTkZJR19TRU5TT1JTX01BWDY2NTAgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01B
WDY2OTcgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01BWDMxNzkwIGlzIG5vdCBzZXQKIyBD
T05GSUdfU0VOU09SU19NQ1AzMDIxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19UQzY1NCBp
cyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVFBTMjM4NjEgaXMgbm90IHNldAojIENPTkZJR19T
RU5TT1JTX01SNzUyMDMgaXMgbm90IHNldApDT05GSUdfU0VOU09SU19MTTYzPXkKIyBDT05GSUdf
U0VOU09SU19MTTczIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTc1IGlzIG5vdCBzZXQK
IyBDT05GSUdfU0VOU09SU19MTTc3IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTc4IGlz
IG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTgwIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09S
U19MTTgzIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTg1IGlzIG5vdCBzZXQKIyBDT05G
SUdfU0VOU09SU19MTTg3IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTkwIGlzIG5vdCBz
ZXQKIyBDT05GSUdfU0VOU09SU19MTTkyIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTkz
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTk1MjM0IGlzIG5vdCBzZXQKIyBDT05GSUdf
U0VOU09SU19MTTk1MjQxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTk1MjQ1IGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19QQzg3MzYwIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09S
U19QQzg3NDI3IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19OVENfVEhFUk1JU1RPUiBpcyBu
b3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTkNUNjY4MyBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNP
UlNfTkNUNjc3NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTkNUNzgwMiBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFTlNPUlNfTkNUNzkwNCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfTlBD
TTdYWCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfT0NDX1A4X0kyQyBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfUENGODU5MSBpcyBub3Qgc2V0CkNPTkZJR19QTUJVUz15CkNPTkZJR19T
RU5TT1JTX1BNQlVTPXkKIyBDT05GSUdfU0VOU09SU19BRE0xMjY2IGlzIG5vdCBzZXQKIyBDT05G
SUdfU0VOU09SU19BRE0xMjc1IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19CRUxfUEZFIGlz
IG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19CUEFfUlM2MDAgaXMgbm90IHNldAojIENPTkZJR19T
RU5TT1JTX0ZTUF8zWSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfSUJNX0NGRlBTIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19EUFM5MjBBQiBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNP
UlNfSU5TUFVSX0lQU1BTIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19JUjM1MjIxIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19JUjM2MDIxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09S
U19JUjM4MDY0IGlzIG5vdCBzZXQKQ09ORklHX1NFTlNPUlNfSVJQUzU0MDE9eQojIENPTkZJR19T
RU5TT1JTX0lTTDY4MTM3IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19MTTI1MDY2IGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19MVEMyOTc4IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09S
U19MVEMzODE1IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19NQVgxNTMwMSBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFTlNPUlNfTUFYMTYwNjQgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01B
WDE2NjAxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19NQVgyMDczMCBpcyBub3Qgc2V0CkNP
TkZJR19TRU5TT1JTX01BWDIwNzUxPXkKIyBDT05GSUdfU0VOU09SU19NQVgzMTc4NSBpcyBub3Qg
c2V0CiMgQ09ORklHX1NFTlNPUlNfTUFYMzQ0NDAgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JT
X01BWDg2ODggaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX01QMjg4OCBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfTVAyOTc1IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19QSU00MzI4
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19QTTY3NjRUUiBpcyBub3Qgc2V0CiMgQ09ORklH
X1NFTlNPUlNfUFhFMTYxMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfUTU0U0oxMDhBMiBp
cyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfU1RQRERDNjAgaXMgbm90IHNldAojIENPTkZJR19T
RU5TT1JTX1RQUzQwNDIyIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19UUFM1MzY3OSBpcyBu
b3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVUNEOTAwMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNP
UlNfVUNEOTIwMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfWERQRTEyMiBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFTlNPUlNfWkw2MTAwIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19TQlRT
SSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfU0hUMTUgaXMgbm90IHNldAojIENPTkZJR19T
RU5TT1JTX1NIVDIxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19TSFQzeCBpcyBub3Qgc2V0
CiMgQ09ORklHX1NFTlNPUlNfU0hUNHggaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1NIVEMx
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19ETUUxNzM3IGlzIG5vdCBzZXQKIyBDT05GSUdf
U0VOU09SU19FTUMxNDAzIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19FTUMyMTAzIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19FTUM2VzIwMSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNP
UlNfU01TQzQ3TTEgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1NNU0M0N00xOTIgaXMgbm90
IHNldAojIENPTkZJR19TRU5TT1JTX1NNU0M0N0IzOTcgaXMgbm90IHNldAojIENPTkZJR19TRU5T
T1JTX1NDSDU2MjcgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1NDSDU2MzYgaXMgbm90IHNl
dAojIENPTkZJR19TRU5TT1JTX1NUVFM3NTEgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1NN
TTY2NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQURDMTI4RDgxOCBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfQURTNzgyOCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfQU1DNjgy
MSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfSU5BMjA5IGlzIG5vdCBzZXQKIyBDT05GSUdf
U0VOU09SU19JTkEyWFggaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX0lOQTMyMjEgaXMgbm90
IHNldAojIENPTkZJR19TRU5TT1JTX1RDNzQgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1RI
TUM1MCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVE1QMTAyIGlzIG5vdCBzZXQKIyBDT05G
SUdfU0VOU09SU19UTVAxMDMgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1RNUDEwOCBpcyBu
b3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVE1QNDAxIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09S
U19UTVA0MjEgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1RNUDUxMyBpcyBub3Qgc2V0CiMg
Q09ORklHX1NFTlNPUlNfVlQxMjExIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19XODM3NzNH
IGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19XODM3ODFEIGlzIG5vdCBzZXQKIyBDT05GSUdf
U0VOU09SU19XODM3OTFEIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19XODM3OTJEIGlzIG5v
dCBzZXQKIyBDT05GSUdfU0VOU09SU19XODM3OTMgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JT
X1c4Mzc5NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVzgzTDc4NVRTIGlzIG5vdCBzZXQK
IyBDT05GSUdfU0VOU09SU19XODNMNzg2TkcgaXMgbm90IHNldAojIENPTkZJR19TRU5TT1JTX1c4
MzYyN0hGIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VOU09SU19XODM2MjdFSEYgaXMgbm90IHNldAoj
IENPTkZJR19USEVSTUFMIGlzIG5vdCBzZXQKQ09ORklHX1dBVENIRE9HPXkKQ09ORklHX1dBVENI
RE9HX0NPUkU9eQojIENPTkZJR19XQVRDSERPR19OT1dBWU9VVCBpcyBub3Qgc2V0CkNPTkZJR19X
QVRDSERPR19IQU5ETEVfQk9PVF9FTkFCTEVEPXkKQ09ORklHX1dBVENIRE9HX09QRU5fVElNRU9V
VD0wCiMgQ09ORklHX1dBVENIRE9HX1NZU0ZTIGlzIG5vdCBzZXQKIyBDT05GSUdfV0FUQ0hET0df
SFJUSU1FUl9QUkVUSU1FT1VUIGlzIG5vdCBzZXQKCiMKIyBXYXRjaGRvZyBQcmV0aW1lb3V0IEdv
dmVybm9ycwojCiMgQ09ORklHX1dBVENIRE9HX1BSRVRJTUVPVVRfR09WIGlzIG5vdCBzZXQKCiMK
IyBXYXRjaGRvZyBEZXZpY2UgRHJpdmVycwojCiMgQ09ORklHX1NPRlRfV0FUQ0hET0cgaXMgbm90
IHNldAojIENPTkZJR19HUElPX1dBVENIRE9HIGlzIG5vdCBzZXQKQ09ORklHX1hJTElOWF9XQVRD
SERPRz15CiMgQ09ORklHX1pJSVJBVkVfV0FUQ0hET0cgaXMgbm90IHNldAojIENPTkZJR19BUk1f
U1A4MDVfV0FUQ0hET0cgaXMgbm90IHNldAojIENPTkZJR19BUk1fU0JTQV9XQVRDSERPRyBpcyBu
b3Qgc2V0CkNPTkZJR19DQURFTkNFX1dBVENIRE9HPXkKIyBDT05GSUdfRFdfV0FUQ0hET0cgaXMg
bm90IHNldAojIENPTkZJR19NQVg2M1hYX1dBVENIRE9HIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJN
X1NNQ19XQVRDSERPRyBpcyBub3Qgc2V0CiMgQ09ORklHX01FTl9BMjFfV0RUIGlzIG5vdCBzZXQK
Q09ORklHX1NTQl9QT1NTSUJMRT15CiMgQ09ORklHX1NTQiBpcyBub3Qgc2V0CkNPTkZJR19CQ01B
X1BPU1NJQkxFPXkKIyBDT05GSUdfQkNNQSBpcyBub3Qgc2V0CgojCiMgTXVsdGlmdW5jdGlvbiBk
ZXZpY2UgZHJpdmVycwojCkNPTkZJR19NRkRfQ09SRT15CiMgQ09ORklHX01GRF9BQ1Q4OTQ1QSBp
cyBub3Qgc2V0CiMgQ09ORklHX01GRF9BUzM3MTEgaXMgbm90IHNldAojIENPTkZJR19NRkRfQVMz
NzIyIGlzIG5vdCBzZXQKIyBDT05GSUdfUE1JQ19BRFA1NTIwIGlzIG5vdCBzZXQKIyBDT05GSUdf
TUZEX0FBVDI4NzBfQ09SRSBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9BVE1FTF9GTEVYQ09NIGlz
IG5vdCBzZXQKIyBDT05GSUdfTUZEX0FUTUVMX0hMQ0RDIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZE
X0JDTTU5MFhYIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0JEOTU3MU1XViBpcyBub3Qgc2V0CiMg
Q09ORklHX01GRF9BWFAyMFhfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX01BREVSQSBpcyBu
b3Qgc2V0CiMgQ09ORklHX1BNSUNfREE5MDNYIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0RBOTA1
Ml9JMkMgaXMgbm90IHNldAojIENPTkZJR19NRkRfREE5MDU1IGlzIG5vdCBzZXQKIyBDT05GSUdf
TUZEX0RBOTA2MiBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9EQTkwNjMgaXMgbm90IHNldAojIENP
TkZJR19NRkRfREE5MTUwIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0dBVEVXT1JLU19HU0MgaXMg
bm90IHNldAojIENPTkZJR19NRkRfTUMxM1hYWF9JMkMgaXMgbm90IHNldAojIENPTkZJR19NRkRf
TVAyNjI5IGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0hJNjQyMV9QTUlDIGlzIG5vdCBzZXQKIyBD
T05GSUdfSFRDX1BBU0lDMyBpcyBub3Qgc2V0CiMgQ09ORklHX0hUQ19JMkNQTEQgaXMgbm90IHNl
dAojIENPTkZJR19NRkRfSVFTNjJYIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0tFTVBMRCBpcyBu
b3Qgc2V0CiMgQ09ORklHX01GRF84OFBNODAwIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEXzg4UE04
MDUgaXMgbm90IHNldAojIENPTkZJR19NRkRfODhQTTg2MFggaXMgbm90IHNldAojIENPTkZJR19N
RkRfTUFYMTQ1NzcgaXMgbm90IHNldAojIENPTkZJR19NRkRfTUFYNzc2MjAgaXMgbm90IHNldAoj
IENPTkZJR19NRkRfTUFYNzc2NTAgaXMgbm90IHNldAojIENPTkZJR19NRkRfTUFYNzc2ODYgaXMg
bm90IHNldAojIENPTkZJR19NRkRfTUFYNzc2OTMgaXMgbm90IHNldAojIENPTkZJR19NRkRfTUFY
Nzc4NDMgaXMgbm90IHNldAojIENPTkZJR19NRkRfTUFYODkwNyBpcyBub3Qgc2V0CiMgQ09ORklH
X01GRF9NQVg4OTI1IGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX01BWDg5OTcgaXMgbm90IHNldAoj
IENPTkZJR19NRkRfTUFYODk5OCBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9NVDYzNjAgaXMgbm90
IHNldAojIENPTkZJR19NRkRfTVQ2Mzk3IGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX01FTkYyMUJN
QyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9OVFhFQyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9S
RVRVIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1BDRjUwNjMzIGlzIG5vdCBzZXQKIyBDT05GSUdf
TUZEX1JUNDgzMSBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9SVDUwMzMgaXMgbm90IHNldAojIENP
TkZJR19NRkRfUkM1VDU4MyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9SSzgwOCBpcyBub3Qgc2V0
CiMgQ09ORklHX01GRF9STjVUNjE4IGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1NFQ19DT1JFIGlz
IG5vdCBzZXQKIyBDT05GSUdfTUZEX1NJNDc2WF9DT1JFIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZE
X1NNNTAxIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1NLWTgxNDUyIGlzIG5vdCBzZXQKIyBDT05G
SUdfTUZEX1NUTVBFIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1NZU0NPTiBpcyBub3Qgc2V0CiMg
Q09ORklHX01GRF9USV9BTTMzNVhfVFNDQURDIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX0xQMzk0
MyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9MUDg3ODggaXMgbm90IHNldAojIENPTkZJR19NRkRf
VElfTE1VIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1BBTE1BUyBpcyBub3Qgc2V0CiMgQ09ORklH
X1RQUzYxMDVYIGlzIG5vdCBzZXQKIyBDT05GSUdfVFBTNjUwMTAgaXMgbm90IHNldAojIENPTkZJ
R19UUFM2NTA3WCBpcyBub3Qgc2V0CkNPTkZJR19NRkRfVFBTNjUwODY9eQojIENPTkZJR19NRkRf
VFBTNjUwOTAgaXMgbm90IHNldAojIENPTkZJR19NRkRfVFBTNjUyMTcgaXMgbm90IHNldAojIENP
TkZJR19NRkRfVElfTFA4NzNYIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1RJX0xQODc1NjUgaXMg
bm90IHNldAojIENPTkZJR19NRkRfVFBTNjUyMTggaXMgbm90IHNldAojIENPTkZJR19NRkRfVFBT
NjU4NlggaXMgbm90IHNldAojIENPTkZJR19NRkRfVFBTNjU5MTAgaXMgbm90IHNldAojIENPTkZJ
R19NRkRfVFBTNjU5MTJfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1RQUzgwMDMxIGlzIG5v
dCBzZXQKIyBDT05GSUdfVFdMNDAzMF9DT1JFIGlzIG5vdCBzZXQKIyBDT05GSUdfVFdMNjA0MF9D
T1JFIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1dMMTI3M19DT1JFIGlzIG5vdCBzZXQKIyBDT05G
SUdfTUZEX0xNMzUzMyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9UQzM1ODlYIGlzIG5vdCBzZXQK
IyBDT05GSUdfTUZEX1RRTVg4NiBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9MT0NITkFHQVIgaXMg
bm90IHNldAojIENPTkZJR19NRkRfQVJJWk9OQV9JMkMgaXMgbm90IHNldAojIENPTkZJR19NRkRf
V004NDAwIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1dNODMxWF9JMkMgaXMgbm90IHNldAojIENP
TkZJR19NRkRfV004MzUwX0kyQyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9XTTg5OTQgaXMgbm90
IHNldAojIENPTkZJR19NRkRfUk9ITV9CRDcxOFhYIGlzIG5vdCBzZXQKIyBDT05GSUdfTUZEX1JP
SE1fQkQ3MDUyOCBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9ST0hNX0JENzE4MjggaXMgbm90IHNl
dAojIENPTkZJR19NRkRfUk9ITV9CRDk1N1hNVUYgaXMgbm90IHNldAojIENPTkZJR19NRkRfU1RQ
TUlDMSBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9TVE1GWCBpcyBub3Qgc2V0CiMgQ09ORklHX01G
RF9BVEMyNjBYX0kyQyBpcyBub3Qgc2V0CiMgQ09ORklHX01GRF9RQ09NX1BNODAwOCBpcyBub3Qg
c2V0CiMgQ09ORklHX1JBVkVfU1BfQ09SRSBpcyBub3Qgc2V0CiMgZW5kIG9mIE11bHRpZnVuY3Rp
b24gZGV2aWNlIGRyaXZlcnMKCkNPTkZJR19SRUdVTEFUT1I9eQojIENPTkZJR19SRUdVTEFUT1Jf
REVCVUcgaXMgbm90IHNldApDT05GSUdfUkVHVUxBVE9SX0ZJWEVEX1ZPTFRBR0U9eQojIENPTkZJ
R19SRUdVTEFUT1JfVklSVFVBTF9DT05TVU1FUiBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRP
Ul9VU0VSU1BBQ0VfQ09OU1VNRVIgaXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfODhQRzg2
WCBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9BQ1Q4ODY1IGlzIG5vdCBzZXQKIyBDT05G
SUdfUkVHVUxBVE9SX0FENTM5OCBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9EQTkxMjEg
aXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfREE5MjEwIGlzIG5vdCBzZXQKIyBDT05GSUdf
UkVHVUxBVE9SX0RBOTIxMSBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9GQU41MzU1NSBp
cyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9GQU41Mzg4MCBpcyBub3Qgc2V0CkNPTkZJR19S
RUdVTEFUT1JfR1BJTz15CiMgQ09ORklHX1JFR1VMQVRPUl9JU0w5MzA1IGlzIG5vdCBzZXQKIyBD
T05GSUdfUkVHVUxBVE9SX0lTTDYyNzFBIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX0xQ
Mzk3MSBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9MUDM5NzIgaXMgbm90IHNldAojIENP
TkZJR19SRUdVTEFUT1JfTFA4NzJYIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX0xQODc1
NSBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9MVEMzNTg5IGlzIG5vdCBzZXQKIyBDT05G
SUdfUkVHVUxBVE9SX0xUQzM2NzYgaXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfTUFYMTU4
NiBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9NQVg4NjQ5IGlzIG5vdCBzZXQKIyBDT05G
SUdfUkVHVUxBVE9SX01BWDg2NjAgaXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfTUFYODg5
MyBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9NQVg4OTUyIGlzIG5vdCBzZXQKIyBDT05G
SUdfUkVHVUxBVE9SX01BWDc3ODI2IGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX01DUDE2
NTAyIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX01QNTQxNiBpcyBub3Qgc2V0CiMgQ09O
RklHX1JFR1VMQVRPUl9NUDg4NTkgaXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfTVA4ODZY
IGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX01QUTc5MjAgaXMgbm90IHNldAojIENPTkZJ
R19SRUdVTEFUT1JfTVQ2MzExIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX1BDQTk0NTAg
aXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfUEY4WDAwIGlzIG5vdCBzZXQKIyBDT05GSUdf
UkVHVUxBVE9SX1BGVVpFMTAwIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX1BWODgwNjAg
aXMgbm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfUFY4ODA4MCBpcyBub3Qgc2V0CiMgQ09ORklH
X1JFR1VMQVRPUl9QVjg4MDkwIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX1JUNDgwMSBp
cyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9SVDYxNjAgaXMgbm90IHNldAojIENPTkZJR19S
RUdVTEFUT1JfUlQ2MjQ1IGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX1JUTVYyMCBpcyBu
b3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9TTEc1MTAwMCBpcyBub3Qgc2V0CiMgQ09ORklHX1JF
R1VMQVRPUl9TWTgxMDZBIGlzIG5vdCBzZXQKIyBDT05GSUdfUkVHVUxBVE9SX1NZODgyNFggaXMg
bm90IHNldAojIENPTkZJR19SRUdVTEFUT1JfU1k4ODI3TiBpcyBub3Qgc2V0CiMgQ09ORklHX1JF
R1VMQVRPUl9UUFM1MTYzMiBpcyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9UUFM2MjM2MCBp
cyBub3Qgc2V0CiMgQ09ORklHX1JFR1VMQVRPUl9UUFM2NTAyMyBpcyBub3Qgc2V0CiMgQ09ORklH
X1JFR1VMQVRPUl9UUFM2NTA3WCBpcyBub3Qgc2V0CkNPTkZJR19SRUdVTEFUT1JfVFBTNjUwODY9
eQojIENPTkZJR19SRUdVTEFUT1JfVFBTNjUxMzIgaXMgbm90IHNldAojIENPTkZJR19SRUdVTEFU
T1JfVkNUUkwgaXMgbm90IHNldAojIENPTkZJR19SQ19DT1JFIGlzIG5vdCBzZXQKIyBDT05GSUdf
TUVESUFfQ0VDX1NVUFBPUlQgaXMgbm90IHNldAojIENPTkZJR19NRURJQV9TVVBQT1JUIGlzIG5v
dCBzZXQKCiMKIyBHcmFwaGljcyBzdXBwb3J0CiMKIyBDT05GSUdfRFJNIGlzIG5vdCBzZXQKCiMK
IyBBUk0gZGV2aWNlcwojCiMgZW5kIG9mIEFSTSBkZXZpY2VzCgojCiMgRnJhbWUgYnVmZmVyIERl
dmljZXMKIwpDT05GSUdfRkJfQ01ETElORT15CkNPTkZJR19GQl9OT1RJRlk9eQpDT05GSUdfRkI9
eQojIENPTkZJR19GSVJNV0FSRV9FRElEIGlzIG5vdCBzZXQKQ09ORklHX0ZCX0NGQl9GSUxMUkVD
VD15CkNPTkZJR19GQl9DRkJfQ09QWUFSRUE9eQpDT05GSUdfRkJfQ0ZCX0lNQUdFQkxJVD15CiMg
Q09ORklHX0ZCX0ZPUkVJR05fRU5ESUFOIGlzIG5vdCBzZXQKIyBDT05GSUdfRkJfTU9ERV9IRUxQ
RVJTIGlzIG5vdCBzZXQKIyBDT05GSUdfRkJfVElMRUJMSVRUSU5HIGlzIG5vdCBzZXQKCiMKIyBG
cmFtZSBidWZmZXIgaGFyZHdhcmUgZHJpdmVycwojCiMgQ09ORklHX0ZCX0FSTUNMQ0QgaXMgbm90
IHNldAojIENPTkZJR19GQl9FRkkgaXMgbm90IHNldAojIENPTkZJR19GQl9PUEVOQ09SRVMgaXMg
bm90IHNldAojIENPTkZJR19GQl9TMUQxM1hYWCBpcyBub3Qgc2V0CiMgQ09ORklHX0ZCX0lCTV9H
WFQ0NTAwIGlzIG5vdCBzZXQKQ09ORklHX0ZCX1hJTElOWD15CiMgQ09ORklHX0ZCX1ZJUlRVQUwg
aXMgbm90IHNldAojIENPTkZJR19GQl9NRVRST05PTUUgaXMgbm90IHNldAojIENPTkZJR19GQl9T
SU1QTEUgaXMgbm90IHNldAojIENPTkZJR19GQl9TU0QxMzA3IGlzIG5vdCBzZXQKIyBlbmQgb2Yg
RnJhbWUgYnVmZmVyIERldmljZXMKCiMKIyBCYWNrbGlnaHQgJiBMQ0QgZGV2aWNlIHN1cHBvcnQK
IwojIENPTkZJR19MQ0RfQ0xBU1NfREVWSUNFIGlzIG5vdCBzZXQKIyBDT05GSUdfQkFDS0xJR0hU
X0NMQVNTX0RFVklDRSBpcyBub3Qgc2V0CiMgZW5kIG9mIEJhY2tsaWdodCAmIExDRCBkZXZpY2Ug
c3VwcG9ydAoKIwojIENvbnNvbGUgZGlzcGxheSBkcml2ZXIgc3VwcG9ydAojCkNPTkZJR19EVU1N
WV9DT05TT0xFPXkKQ09ORklHX0RVTU1ZX0NPTlNPTEVfQ09MVU1OUz04MApDT05GSUdfRFVNTVlf
Q09OU09MRV9ST1dTPTI1CkNPTkZJR19GUkFNRUJVRkZFUl9DT05TT0xFPXkKQ09ORklHX0ZSQU1F
QlVGRkVSX0NPTlNPTEVfREVURUNUX1BSSU1BUlk9eQojIENPTkZJR19GUkFNRUJVRkZFUl9DT05T
T0xFX1JPVEFUSU9OIGlzIG5vdCBzZXQKIyBDT05GSUdfRlJBTUVCVUZGRVJfQ09OU09MRV9ERUZF
UlJFRF9UQUtFT1ZFUiBpcyBub3Qgc2V0CiMgZW5kIG9mIENvbnNvbGUgZGlzcGxheSBkcml2ZXIg
c3VwcG9ydAoKIyBDT05GSUdfTE9HTyBpcyBub3Qgc2V0CiMgZW5kIG9mIEdyYXBoaWNzIHN1cHBv
cnQKCiMgQ09ORklHX1NPVU5EIGlzIG5vdCBzZXQKCiMKIyBISUQgc3VwcG9ydAojCkNPTkZJR19I
SUQ9eQojIENPTkZJR19ISURfQkFUVEVSWV9TVFJFTkdUSCBpcyBub3Qgc2V0CiMgQ09ORklHX0hJ
RFJBVyBpcyBub3Qgc2V0CiMgQ09ORklHX1VISUQgaXMgbm90IHNldApDT05GSUdfSElEX0dFTkVS
SUM9eQoKIwojIFNwZWNpYWwgSElEIGRyaXZlcnMKIwojIENPTkZJR19ISURfQTRURUNIIGlzIG5v
dCBzZXQKIyBDT05GSUdfSElEX0FDUlVYIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX0FQUExFIGlz
IG5vdCBzZXQKIyBDT05GSUdfSElEX0FVUkVBTCBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9CRUxL
SU4gaXMgbm90IHNldAojIENPTkZJR19ISURfQ0hFUlJZIGlzIG5vdCBzZXQKIyBDT05GSUdfSElE
X0NISUNPTlkgaXMgbm90IHNldAojIENPTkZJR19ISURfQ09VR0FSIGlzIG5vdCBzZXQKIyBDT05G
SUdfSElEX01BQ0FMTFkgaXMgbm90IHNldAojIENPTkZJR19ISURfQ01FRElBIGlzIG5vdCBzZXQK
IyBDT05GSUdfSElEX0NZUFJFU1MgaXMgbm90IHNldAojIENPTkZJR19ISURfRFJBR09OUklTRSBp
cyBub3Qgc2V0CiMgQ09ORklHX0hJRF9FTVNfRkYgaXMgbm90IHNldAojIENPTkZJR19ISURfRUxF
Q09NIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX0VaS0VZIGlzIG5vdCBzZXQKIyBDT05GSUdfSElE
X0dFTUJJUkQgaXMgbm90IHNldAojIENPTkZJR19ISURfR0ZSTSBpcyBub3Qgc2V0CiMgQ09ORklH
X0hJRF9HTE9SSU9VUyBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9WSVZBTERJIGlzIG5vdCBzZXQK
IyBDT05GSUdfSElEX0tFWVRPVUNIIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX0tZRSBpcyBub3Qg
c2V0CiMgQ09ORklHX0hJRF9XQUxUT1AgaXMgbm90IHNldAojIENPTkZJR19ISURfVklFV1NPTklD
IGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX0dZUkFUSU9OIGlzIG5vdCBzZXQKIyBDT05GSUdfSElE
X0lDQURFIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX0lURSBpcyBub3Qgc2V0CiMgQ09ORklHX0hJ
RF9KQUJSQSBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9UV0lOSEFOIGlzIG5vdCBzZXQKIyBDT05G
SUdfSElEX0tFTlNJTkdUT04gaXMgbm90IHNldAojIENPTkZJR19ISURfTENQT1dFUiBpcyBub3Qg
c2V0CiMgQ09ORklHX0hJRF9MRUQgaXMgbm90IHNldAojIENPTkZJR19ISURfTEVOT1ZPIGlzIG5v
dCBzZXQKIyBDT05GSUdfSElEX0xPR0lURUNIIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX01BR0lD
TU9VU0UgaXMgbm90IHNldAojIENPTkZJR19ISURfTUFMVFJPTiBpcyBub3Qgc2V0CiMgQ09ORklH
X0hJRF9NQVlGTEFTSCBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9SRURSQUdPTiBpcyBub3Qgc2V0
CiMgQ09ORklHX0hJRF9NSUNST1NPRlQgaXMgbm90IHNldAojIENPTkZJR19ISURfTU9OVEVSRVkg
aXMgbm90IHNldAojIENPTkZJR19ISURfTVVMVElUT1VDSCBpcyBub3Qgc2V0CiMgQ09ORklHX0hJ
RF9OVEkgaXMgbm90IHNldAojIENPTkZJR19ISURfT1JURUsgaXMgbm90IHNldAojIENPTkZJR19I
SURfUEFOVEhFUkxPUkQgaXMgbm90IHNldAojIENPTkZJR19ISURfUEVUQUxZTlggaXMgbm90IHNl
dAojIENPTkZJR19ISURfUElDT0xDRCBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9QTEFOVFJPTklD
UyBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9QTEFZU1RBVElPTiBpcyBub3Qgc2V0CiMgQ09ORklH
X0hJRF9QUklNQVggaXMgbm90IHNldAojIENPTkZJR19ISURfU0FJVEVLIGlzIG5vdCBzZXQKIyBD
T05GSUdfSElEX1NBTVNVTkcgaXMgbm90IHNldAojIENPTkZJR19ISURfU0VNSVRFSyBpcyBub3Qg
c2V0CiMgQ09ORklHX0hJRF9TUEVFRExJTksgaXMgbm90IHNldAojIENPTkZJR19ISURfU1RFQU0g
aXMgbm90IHNldAojIENPTkZJR19ISURfU1RFRUxTRVJJRVMgaXMgbm90IHNldAojIENPTkZJR19I
SURfU1VOUExVUyBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9STUkgaXMgbm90IHNldAojIENPTkZJ
R19ISURfR1JFRU5BU0lBIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX1NNQVJUSk9ZUExVUyBpcyBu
b3Qgc2V0CiMgQ09ORklHX0hJRF9USVZPIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX1RPUFNFRUQg
aXMgbm90IHNldAojIENPTkZJR19ISURfVEhJTkdNIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX1VE
UkFXX1BTMyBpcyBub3Qgc2V0CiMgQ09ORklHX0hJRF9XSUlNT1RFIGlzIG5vdCBzZXQKIyBDT05G
SUdfSElEX1hJTk1PIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX1pFUk9QTFVTIGlzIG5vdCBzZXQK
IyBDT05GSUdfSElEX1pZREFDUk9OIGlzIG5vdCBzZXQKIyBDT05GSUdfSElEX1NFTlNPUl9IVUIg
aXMgbm90IHNldAojIENPTkZJR19ISURfQUxQUyBpcyBub3Qgc2V0CiMgZW5kIG9mIFNwZWNpYWwg
SElEIGRyaXZlcnMKCiMKIyBJMkMgSElEIHN1cHBvcnQKIwojIENPTkZJR19JMkNfSElEX09GIGlz
IG5vdCBzZXQKIyBDT05GSUdfSTJDX0hJRF9PRl9HT09ESVggaXMgbm90IHNldAojIGVuZCBvZiBJ
MkMgSElEIHN1cHBvcnQKIyBlbmQgb2YgSElEIHN1cHBvcnQKCkNPTkZJR19VU0JfT0hDSV9MSVRU
TEVfRU5ESUFOPXkKIyBDT05GSUdfVVNCX1NVUFBPUlQgaXMgbm90IHNldAojIENPTkZJR19NTUMg
aXMgbm90IHNldAojIENPTkZJR19NRU1TVElDSyBpcyBub3Qgc2V0CkNPTkZJR19ORVdfTEVEUz15
CkNPTkZJR19MRURTX0NMQVNTPXkKIyBDT05GSUdfTEVEU19DTEFTU19GTEFTSCBpcyBub3Qgc2V0
CiMgQ09ORklHX0xFRFNfQ0xBU1NfTVVMVElDT0xPUiBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNf
QlJJR0hUTkVTU19IV19DSEFOR0VEIGlzIG5vdCBzZXQKCiMKIyBMRUQgZHJpdmVycwojCiMgQ09O
RklHX0xFRFNfQU4zMDI1OUEgaXMgbm90IHNldAojIENPTkZJR19MRURTX0FXMjAxMyBpcyBub3Qg
c2V0CiMgQ09ORklHX0xFRFNfQkNNNjMyOCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfQkNNNjM1
OCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfTE0zNTMwIGlzIG5vdCBzZXQKIyBDT05GSUdfTEVE
U19MTTM1MzIgaXMgbm90IHNldAojIENPTkZJR19MRURTX0xNMzY0MiBpcyBub3Qgc2V0CiMgQ09O
RklHX0xFRFNfTE0zNjkyWCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfUENBOTUzMiBpcyBub3Qg
c2V0CkNPTkZJR19MRURTX0dQSU89eQojIENPTkZJR19MRURTX0xQMzk0NCBpcyBub3Qgc2V0CiMg
Q09ORklHX0xFRFNfTFAzOTUyIGlzIG5vdCBzZXQKIyBDT05GSUdfTEVEU19MUDUwWFggaXMgbm90
IHNldAojIENPTkZJR19MRURTX0xQNTVYWF9DT01NT04gaXMgbm90IHNldAojIENPTkZJR19MRURT
X0xQODg2MCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfUENBOTU1WCBpcyBub3Qgc2V0CiMgQ09O
RklHX0xFRFNfUENBOTYzWCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfUkVHVUxBVE9SIGlzIG5v
dCBzZXQKIyBDT05GSUdfTEVEU19CRDI4MDIgaXMgbm90IHNldAojIENPTkZJR19MRURTX0xUMzU5
MyBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfVENBNjUwNyBpcyBub3Qgc2V0CiMgQ09ORklHX0xF
RFNfVExDNTkxWFggaXMgbm90IHNldAojIENPTkZJR19MRURTX0xNMzU1eCBpcyBub3Qgc2V0CiMg
Q09ORklHX0xFRFNfSVMzMUZMMzE5WCBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfSVMzMUZMMzJY
WCBpcyBub3Qgc2V0CgojCiMgTEVEIGRyaXZlciBmb3IgYmxpbmsoMSkgVVNCIFJHQiBMRUQgaXMg
dW5kZXIgU3BlY2lhbCBISUQgZHJpdmVycyAoSElEX1RISU5HTSkKIwojIENPTkZJR19MRURTX0JM
SU5LTSBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfTUxYUkVHIGlzIG5vdCBzZXQKIyBDT05GSUdf
TEVEU19VU0VSIGlzIG5vdCBzZXQKIyBDT05GSUdfTEVEU19USV9MTVVfQ09NTU9OIGlzIG5vdCBz
ZXQKCiMKIyBGbGFzaCBhbmQgVG9yY2ggTEVEIGRyaXZlcnMKIwoKIwojIExFRCBUcmlnZ2Vycwoj
CkNPTkZJR19MRURTX1RSSUdHRVJTPXkKQ09ORklHX0xFRFNfVFJJR0dFUl9USU1FUj15CkNPTkZJ
R19MRURTX1RSSUdHRVJfT05FU0hPVD15CkNPTkZJR19MRURTX1RSSUdHRVJfSEVBUlRCRUFUPXkK
Q09ORklHX0xFRFNfVFJJR0dFUl9CQUNLTElHSFQ9eQpDT05GSUdfTEVEU19UUklHR0VSX0NQVT15
CiMgQ09ORklHX0xFRFNfVFJJR0dFUl9BQ1RJVklUWSBpcyBub3Qgc2V0CkNPTkZJR19MRURTX1RS
SUdHRVJfR1BJTz15CkNPTkZJR19MRURTX1RSSUdHRVJfREVGQVVMVF9PTj15CgojCiMgaXB0YWJs
ZXMgdHJpZ2dlciBpcyB1bmRlciBOZXRmaWx0ZXIgY29uZmlnIChMRUQgdGFyZ2V0KQojCkNPTkZJ
R19MRURTX1RSSUdHRVJfVFJBTlNJRU5UPXkKQ09ORklHX0xFRFNfVFJJR0dFUl9DQU1FUkE9eQoj
IENPTkZJR19MRURTX1RSSUdHRVJfUEFOSUMgaXMgbm90IHNldAojIENPTkZJR19MRURTX1RSSUdH
RVJfUEFUVEVSTiBpcyBub3Qgc2V0CiMgQ09ORklHX0xFRFNfVFJJR0dFUl9BVURJTyBpcyBub3Qg
c2V0CiMgQ09ORklHX0xFRFNfVFJJR0dFUl9UVFkgaXMgbm90IHNldAojIENPTkZJR19BQ0NFU1NJ
QklMSVRZIGlzIG5vdCBzZXQKQ09ORklHX0VEQUNfU1VQUE9SVD15CkNPTkZJR19FREFDPXkKQ09O
RklHX0VEQUNfTEVHQUNZX1NZU0ZTPXkKIyBDT05GSUdfRURBQ19ERUJVRyBpcyBub3Qgc2V0CkNP
TkZJR19FREFDX1NZTk9QU1lTPXkKIyBDT05GSUdfRURBQ19YR0VORSBpcyBub3Qgc2V0CiMgQ09O
RklHX0VEQUNfRE1DNTIwIGlzIG5vdCBzZXQKQ09ORklHX1JUQ19MSUI9eQpDT05GSUdfUlRDX0NM
QVNTPXkKQ09ORklHX1JUQ19IQ1RPU1lTPXkKQ09ORklHX1JUQ19IQ1RPU1lTX0RFVklDRT0icnRj
MCIKQ09ORklHX1JUQ19TWVNUT0hDPXkKQ09ORklHX1JUQ19TWVNUT0hDX0RFVklDRT0icnRjMCIK
IyBDT05GSUdfUlRDX0RFQlVHIGlzIG5vdCBzZXQKQ09ORklHX1JUQ19OVk1FTT15CgojCiMgUlRD
IGludGVyZmFjZXMKIwpDT05GSUdfUlRDX0lOVEZfU1lTRlM9eQpDT05GSUdfUlRDX0lOVEZfUFJP
Qz15CkNPTkZJR19SVENfSU5URl9ERVY9eQojIENPTkZJR19SVENfSU5URl9ERVZfVUlFX0VNVUwg
aXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX1RFU1QgaXMgbm90IHNldAoKIwojIEkyQyBSVEMg
ZHJpdmVycwojCiMgQ09ORklHX1JUQ19EUlZfQUJCNVpFUzMgaXMgbm90IHNldAojIENPTkZJR19S
VENfRFJWX0FCRU9aOSBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfQUJYODBYIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUlRDX0RSVl9EUzEzMDcgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX0RT
MTM3NCBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfRFMxNjcyIGlzIG5vdCBzZXQKIyBDT05G
SUdfUlRDX0RSVl9IWU04NTYzIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9NQVg2OTAwIGlz
IG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9SUzVDMzcyIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRD
X0RSVl9JU0wxMjA4IGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9JU0wxMjAyMiBpcyBub3Qg
c2V0CiMgQ09ORklHX1JUQ19EUlZfSVNMMTIwMjYgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJW
X1gxMjA1IGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9QQ0Y4NTIzIGlzIG5vdCBzZXQKIyBD
T05GSUdfUlRDX0RSVl9QQ0Y4NTA2MyBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfUENGODUz
NjMgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX1BDRjg1NjMgaXMgbm90IHNldAojIENPTkZJ
R19SVENfRFJWX1BDRjg1ODMgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX000MVQ4MCBpcyBu
b3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfQlEzMksgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJW
X1MzNTM5MEEgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX0ZNMzEzMCBpcyBub3Qgc2V0CiMg
Q09ORklHX1JUQ19EUlZfUlg4MDEwIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9SWDg1ODEg
aXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX1JYODAyNSBpcyBub3Qgc2V0CiMgQ09ORklHX1JU
Q19EUlZfRU0zMDI3IGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9SVjMwMjggaXMgbm90IHNl
dAojIENPTkZJR19SVENfRFJWX1JWMzAzMiBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfUlY4
ODAzIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9TRDMwNzggaXMgbm90IHNldAoKIwojIFNQ
SSBSVEMgZHJpdmVycwojCkNPTkZJR19SVENfSTJDX0FORF9TUEk9eQoKIwojIFNQSSBhbmQgSTJD
IFJUQyBkcml2ZXJzCiMKIyBDT05GSUdfUlRDX0RSVl9EUzMyMzIgaXMgbm90IHNldAojIENPTkZJ
R19SVENfRFJWX1BDRjIxMjcgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX1JWMzAyOUMyIGlz
IG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9SWDYxMTAgaXMgbm90IHNldAoKIwojIFBsYXRmb3Jt
IFJUQyBkcml2ZXJzCiMKIyBDT05GSUdfUlRDX0RSVl9EUzEyODYgaXMgbm90IHNldAojIENPTkZJ
R19SVENfRFJWX0RTMTUxMSBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfRFMxNTUzIGlzIG5v
dCBzZXQKIyBDT05GSUdfUlRDX0RSVl9EUzE2ODVfRkFNSUxZIGlzIG5vdCBzZXQKIyBDT05GSUdf
UlRDX0RSVl9EUzE3NDIgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX0RTMjQwNCBpcyBub3Qg
c2V0CiMgQ09ORklHX1JUQ19EUlZfRUZJIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9TVEsx
N1RBOCBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfTTQ4VDg2IGlzIG5vdCBzZXQKIyBDT05G
SUdfUlRDX0RSVl9NNDhUMzUgaXMgbm90IHNldAojIENPTkZJR19SVENfRFJWX000OFQ1OSBpcyBu
b3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfTVNNNjI0MiBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19E
UlZfQlE0ODAyIGlzIG5vdCBzZXQKIyBDT05GSUdfUlRDX0RSVl9SUDVDMDEgaXMgbm90IHNldAoj
IENPTkZJR19SVENfRFJWX1YzMDIwIGlzIG5vdCBzZXQKQ09ORklHX1JUQ19EUlZfWllOUU1QPXkK
CiMKIyBvbi1DUFUgUlRDIGRyaXZlcnMKIwojIENPTkZJR19SVENfRFJWX1BMMDMwIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUlRDX0RSVl9QTDAzMSBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfQ0FE
RU5DRSBpcyBub3Qgc2V0CiMgQ09ORklHX1JUQ19EUlZfRlRSVEMwMTAgaXMgbm90IHNldAojIENP
TkZJR19SVENfRFJWX1I3MzAxIGlzIG5vdCBzZXQKCiMKIyBISUQgU2Vuc29yIFJUQyBkcml2ZXJz
CiMKIyBDT05GSUdfUlRDX0RSVl9HT0xERklTSCBpcyBub3Qgc2V0CkNPTkZJR19ETUFERVZJQ0VT
PXkKIyBDT05GSUdfRE1BREVWSUNFU19ERUJVRyBpcyBub3Qgc2V0CgojCiMgRE1BIERldmljZXMK
IwpDT05GSUdfRE1BX0VOR0lORT15CkNPTkZJR19ETUFfT0Y9eQojIENPTkZJR19BTFRFUkFfTVNH
RE1BIGlzIG5vdCBzZXQKIyBDT05GSUdfQU1CQV9QTDA4WCBpcyBub3Qgc2V0CiMgQ09ORklHX0FY
SV9ETUFDIGlzIG5vdCBzZXQKIyBDT05GSUdfRFdfQVhJX0RNQUMgaXMgbm90IHNldAojIENPTkZJ
R19GU0xfRURNQSBpcyBub3Qgc2V0CiMgQ09ORklHX0ZTTF9RRE1BIGlzIG5vdCBzZXQKIyBDT05G
SUdfSU5URUxfSURNQTY0IGlzIG5vdCBzZXQKIyBDT05GSUdfTVZfWE9SX1YyIGlzIG5vdCBzZXQK
IyBDT05GSUdfUEwzMzBfRE1BIGlzIG5vdCBzZXQKQ09ORklHX1hJTElOWF9ETUE9eQpDT05GSUdf
WElMSU5YX1pZTlFNUF9ETUE9eQojIENPTkZJR19YSUxJTlhfWllOUU1QX0RQRE1BIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUUNPTV9ISURNQV9NR01UIGlzIG5vdCBzZXQKIyBDT05GSUdfUUNPTV9ISURN
QSBpcyBub3Qgc2V0CiMgQ09ORklHX0RXX0RNQUMgaXMgbm90IHNldAojIENPTkZJR19TRl9QRE1B
IGlzIG5vdCBzZXQKCiMKIyBETUEgQ2xpZW50cwojCiMgQ09ORklHX0FTWU5DX1RYX0RNQSBpcyBu
b3Qgc2V0CkNPTkZJR19ETUFURVNUPXkKQ09ORklHX0RNQV9FTkdJTkVfUkFJRD15CgojCiMgRE1B
QlVGIG9wdGlvbnMKIwpDT05GSUdfU1lOQ19GSUxFPXkKIyBDT05GSUdfU1dfU1lOQyBpcyBub3Qg
c2V0CiMgQ09ORklHX1VETUFCVUYgaXMgbm90IHNldAojIENPTkZJR19ETUFCVUZfTU9WRV9OT1RJ
RlkgaXMgbm90IHNldAojIENPTkZJR19ETUFCVUZfREVCVUcgaXMgbm90IHNldAojIENPTkZJR19E
TUFCVUZfU0VMRlRFU1RTIGlzIG5vdCBzZXQKIyBDT05GSUdfRE1BQlVGX0hFQVBTIGlzIG5vdCBz
ZXQKIyBlbmQgb2YgRE1BQlVGIG9wdGlvbnMKCiMgQ09ORklHX0FVWERJU1BMQVkgaXMgbm90IHNl
dApDT05GSUdfVUlPPXkKQ09ORklHX1VJT19QRFJWX0dFTklSUT15CkNPTkZJR19VSU9fRE1FTV9H
RU5JUlE9eQojIENPTkZJR19VSU9fUFJVU1MgaXMgbm90IHNldAojIENPTkZJR19WRklPIGlzIG5v
dCBzZXQKIyBDT05GSUdfVklSVF9EUklWRVJTIGlzIG5vdCBzZXQKQ09ORklHX1ZJUlRJTz15CkNP
TkZJR19WSVJUSU9fTUVOVT15CiMgQ09ORklHX1ZJUlRJT19CQUxMT09OIGlzIG5vdCBzZXQKIyBD
T05GSUdfVklSVElPX0lOUFVUIGlzIG5vdCBzZXQKIyBDT05GSUdfVklSVElPX01NSU8gaXMgbm90
IHNldApDT05GSUdfVkhPU1RfTUVOVT15CiMgQ09ORklHX1ZIT1NUX0NST1NTX0VORElBTl9MRUdB
Q1kgaXMgbm90IHNldAoKIwojIE1pY3Jvc29mdCBIeXBlci1WIGd1ZXN0IHN1cHBvcnQKIwojIGVu
ZCBvZiBNaWNyb3NvZnQgSHlwZXItViBndWVzdCBzdXBwb3J0CgojIENPTkZJR19HUkVZQlVTIGlz
IG5vdCBzZXQKIyBDT05GSUdfQ09NRURJIGlzIG5vdCBzZXQKIyBDT05GSUdfU1RBR0lORyBpcyBu
b3Qgc2V0CiMgQ09ORklHX0dPTERGSVNIIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0hST01FX1BMQVRG
T1JNUyBpcyBub3Qgc2V0CiMgQ09ORklHX01FTExBTk9YX1BMQVRGT1JNIGlzIG5vdCBzZXQKQ09O
RklHX0hBVkVfQ0xLPXkKQ09ORklHX0hBVkVfQ0xLX1BSRVBBUkU9eQpDT05GSUdfQ09NTU9OX0NM
Sz15CgojCiMgQ2xvY2sgZHJpdmVyIGZvciBBUk0gUmVmZXJlbmNlIGRlc2lnbnMKIwojIENPTkZJ
R19JQ1NUIGlzIG5vdCBzZXQKIyBDT05GSUdfQ0xLX1NQODEwIGlzIG5vdCBzZXQKIyBlbmQgb2Yg
Q2xvY2sgZHJpdmVyIGZvciBBUk0gUmVmZXJlbmNlIGRlc2lnbnMKCiMgQ09ORklHX0NPTU1PTl9D
TEtfTUFYOTQ4NSBpcyBub3Qgc2V0CkNPTkZJR19DT01NT05fQ0xLX1NJNTM0MT15CiMgQ09ORklH
X0NPTU1PTl9DTEtfU0k1MzUxIGlzIG5vdCBzZXQKIyBDT05GSUdfQ09NTU9OX0NMS19TSTUxNCBp
cyBub3Qgc2V0CiMgQ09ORklHX0NPTU1PTl9DTEtfU0k1NDQgaXMgbm90IHNldApDT05GSUdfQ09N
TU9OX0NMS19TSTU3MD15CiMgQ09ORklHX0NPTU1PTl9DTEtfQ0RDRTcwNiBpcyBub3Qgc2V0CiMg
Q09ORklHX0NPTU1PTl9DTEtfQ0RDRTkyNSBpcyBub3Qgc2V0CiMgQ09ORklHX0NPTU1PTl9DTEtf
Q1MyMDAwX0NQIGlzIG5vdCBzZXQKIyBDT05GSUdfQ09NTU9OX0NMS19BWElfQ0xLR0VOIGlzIG5v
dCBzZXQKIyBDT05GSUdfQ09NTU9OX0NMS19YR0VORSBpcyBub3Qgc2V0CkNPTkZJR19DT01NT05f
Q0xLX1ZDNT15CiMgQ09ORklHX0NPTU1PTl9DTEtfRklYRURfTU1JTyBpcyBub3Qgc2V0CkNPTkZJ
R19YSUxJTlhfVkNVPXkKQ09ORklHX0NPTU1PTl9DTEtfWllOUU1QPXkKIyBDT05GSUdfSFdTUElO
TE9DSyBpcyBub3Qgc2V0CgojCiMgQ2xvY2sgU291cmNlIGRyaXZlcnMKIwpDT05GSUdfVElNRVJf
T0Y9eQpDT05GSUdfVElNRVJfUFJPQkU9eQpDT05GSUdfQVJNX0FSQ0hfVElNRVI9eQpDT05GSUdf
QVJNX0FSQ0hfVElNRVJfRVZUU1RSRUFNPXkKQ09ORklHX0FSTV9BUkNIX1RJTUVSX09PTF9XT1JL
QVJPVU5EPXkKQ09ORklHX0ZTTF9FUlJBVFVNX0EwMDg1ODU9eQpDT05GSUdfSElTSUxJQ09OX0VS
UkFUVU1fMTYxMDEwMTAxPXkKQ09ORklHX0FSTTY0X0VSUkFUVU1fODU4OTIxPXkKIyBDT05GSUdf
TUlDUk9DSElQX1BJVDY0QiBpcyBub3Qgc2V0CiMgZW5kIG9mIENsb2NrIFNvdXJjZSBkcml2ZXJz
CgpDT05GSUdfTUFJTEJPWD15CiMgQ09ORklHX0FSTV9NSFUgaXMgbm90IHNldAojIENPTkZJR19B
Uk1fTUhVX1YyIGlzIG5vdCBzZXQKIyBDT05GSUdfUExBVEZPUk1fTUhVIGlzIG5vdCBzZXQKIyBD
T05GSUdfUEwzMjBfTUJPWCBpcyBub3Qgc2V0CiMgQ09ORklHX0FMVEVSQV9NQk9YIGlzIG5vdCBz
ZXQKIyBDT05GSUdfTUFJTEJPWF9URVNUIGlzIG5vdCBzZXQKQ09ORklHX1pZTlFNUF9JUElfTUJP
WD15CkNPTkZJR19JT01NVV9JT1ZBPXkKQ09ORklHX0lPTU1VX0FQST15CkNPTkZJR19JT01NVV9T
VVBQT1JUPXkKCiMKIyBHZW5lcmljIElPTU1VIFBhZ2V0YWJsZSBTdXBwb3J0CiMKIyBDT05GSUdf
SU9NTVVfSU9fUEdUQUJMRV9MUEFFIGlzIG5vdCBzZXQKIyBDT05GSUdfSU9NTVVfSU9fUEdUQUJM
RV9BUk1WN1MgaXMgbm90IHNldAojIGVuZCBvZiBHZW5lcmljIElPTU1VIFBhZ2V0YWJsZSBTdXBw
b3J0CgojIENPTkZJR19JT01NVV9ERUJVR0ZTIGlzIG5vdCBzZXQKIyBDT05GSUdfSU9NTVVfREVG
QVVMVF9QQVNTVEhST1VHSCBpcyBub3Qgc2V0CkNPTkZJR19PRl9JT01NVT15CkNPTkZJR19JT01N
VV9ETUE9eQojIENPTkZJR19BUk1fU01NVSBpcyBub3Qgc2V0CiMgQ09ORklHX0FSTV9TTU1VX1Yz
IGlzIG5vdCBzZXQKIyBDT05GSUdfVklSVElPX0lPTU1VIGlzIG5vdCBzZXQKCiMKIyBSZW1vdGVw
cm9jIGRyaXZlcnMKIwpDT05GSUdfUkVNT1RFUFJPQz15CiMgQ09ORklHX1JFTU9URVBST0NfQ0RF
ViBpcyBub3Qgc2V0CiMgZW5kIG9mIFJlbW90ZXByb2MgZHJpdmVycwoKIwojIFJwbXNnIGRyaXZl
cnMKIwojIENPTkZJR19SUE1TR19RQ09NX0dMSU5LX1JQTSBpcyBub3Qgc2V0CiMgQ09ORklHX1JQ
TVNHX1ZJUlRJTyBpcyBub3Qgc2V0CiMgZW5kIG9mIFJwbXNnIGRyaXZlcnMKCiMgQ09ORklHX1NP
VU5EV0lSRSBpcyBub3Qgc2V0CgojCiMgU09DIChTeXN0ZW0gT24gQ2hpcCkgc3BlY2lmaWMgRHJp
dmVycwojCgojCiMgQW1sb2dpYyBTb0MgZHJpdmVycwojCiMgZW5kIG9mIEFtbG9naWMgU29DIGRy
aXZlcnMKCiMKIyBCcm9hZGNvbSBTb0MgZHJpdmVycwojCiMgQ09ORklHX1NPQ19CUkNNU1RCIGlz
IG5vdCBzZXQKIyBlbmQgb2YgQnJvYWRjb20gU29DIGRyaXZlcnMKCiMKIyBOWFAvRnJlZXNjYWxl
IFFvcklRIFNvQyBkcml2ZXJzCiMKIyBDT05GSUdfUVVJQ0NfRU5HSU5FIGlzIG5vdCBzZXQKIyBD
T05GSUdfRlNMX1JDUE0gaXMgbm90IHNldAojIGVuZCBvZiBOWFAvRnJlZXNjYWxlIFFvcklRIFNv
QyBkcml2ZXJzCgojCiMgaS5NWCBTb0MgZHJpdmVycwojCiMgZW5kIG9mIGkuTVggU29DIGRyaXZl
cnMKCiMKIyBFbmFibGUgTGl0ZVggU29DIEJ1aWxkZXIgc3BlY2lmaWMgZHJpdmVycwojCiMgQ09O
RklHX0xJVEVYX1NPQ19DT05UUk9MTEVSIGlzIG5vdCBzZXQKIyBlbmQgb2YgRW5hYmxlIExpdGVY
IFNvQyBCdWlsZGVyIHNwZWNpZmljIGRyaXZlcnMKCiMKIyBRdWFsY29tbSBTb0MgZHJpdmVycwoj
CiMgZW5kIG9mIFF1YWxjb21tIFNvQyBkcml2ZXJzCgojIENPTkZJR19TT0NfVEkgaXMgbm90IHNl
dAoKIwojIFhpbGlueCBTb0MgZHJpdmVycwojCkNPTkZJR19aWU5RTVBfUE9XRVI9eQpDT05GSUdf
WllOUU1QX1BNX0RPTUFJTlM9eQojIGVuZCBvZiBYaWxpbnggU29DIGRyaXZlcnMKIyBlbmQgb2Yg
U09DIChTeXN0ZW0gT24gQ2hpcCkgc3BlY2lmaWMgRHJpdmVycwoKIyBDT05GSUdfUE1fREVWRlJF
USBpcyBub3Qgc2V0CkNPTkZJR19FWFRDT049eQoKIwojIEV4dGNvbiBEZXZpY2UgRHJpdmVycwoj
CiMgQ09ORklHX0VYVENPTl9BRENfSkFDSyBpcyBub3Qgc2V0CiMgQ09ORklHX0VYVENPTl9GU0E5
NDgwIGlzIG5vdCBzZXQKIyBDT05GSUdfRVhUQ09OX0dQSU8gaXMgbm90IHNldAojIENPTkZJR19F
WFRDT05fTUFYMzM1NSBpcyBub3Qgc2V0CiMgQ09ORklHX0VYVENPTl9QVE41MTUwIGlzIG5vdCBz
ZXQKIyBDT05GSUdfRVhUQ09OX1JUODk3M0EgaXMgbm90IHNldAojIENPTkZJR19FWFRDT05fU001
NTAyIGlzIG5vdCBzZXQKIyBDT05GSUdfRVhUQ09OX1VTQl9HUElPIGlzIG5vdCBzZXQKIyBDT05G
SUdfRVhUQ09OX1VTQkNfVFVTQjMyMCBpcyBub3Qgc2V0CiMgQ09ORklHX01FTU9SWSBpcyBub3Qg
c2V0CkNPTkZJR19JSU89eQpDT05GSUdfSUlPX0JVRkZFUj15CiMgQ09ORklHX0lJT19CVUZGRVJf
Q0IgaXMgbm90IHNldAojIENPTkZJR19JSU9fQlVGRkVSX0RNQSBpcyBub3Qgc2V0CiMgQ09ORklH
X0lJT19CVUZGRVJfRE1BRU5HSU5FIGlzIG5vdCBzZXQKIyBDT05GSUdfSUlPX0JVRkZFUl9IV19D
T05TVU1FUiBpcyBub3Qgc2V0CkNPTkZJR19JSU9fS0ZJRk9fQlVGPXkKQ09ORklHX0lJT19UUklH
R0VSRURfQlVGRkVSPXkKIyBDT05GSUdfSUlPX0NPTkZJR0ZTIGlzIG5vdCBzZXQKQ09ORklHX0lJ
T19UUklHR0VSPXkKQ09ORklHX0lJT19DT05TVU1FUlNfUEVSX1RSSUdHRVI9MgojIENPTkZJR19J
SU9fU1dfREVWSUNFIGlzIG5vdCBzZXQKIyBDT05GSUdfSUlPX1NXX1RSSUdHRVIgaXMgbm90IHNl
dAojIENPTkZJR19JSU9fVFJJR0dFUkVEX0VWRU5UIGlzIG5vdCBzZXQKCiMKIyBBY2NlbGVyb21l
dGVycwojCiMgQ09ORklHX0FEWEwzNDVfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfQURYTDM3Ml9J
MkMgaXMgbm90IHNldAojIENPTkZJR19CTUExODAgaXMgbm90IHNldAojIENPTkZJR19CTUE0MDAg
aXMgbm90IHNldAojIENPTkZJR19CTUMxNTBfQUNDRUwgaXMgbm90IHNldAojIENPTkZJR19EQTI4
MCBpcyBub3Qgc2V0CiMgQ09ORklHX0RBMzExIGlzIG5vdCBzZXQKIyBDT05GSUdfRE1BUkQwNiBp
cyBub3Qgc2V0CiMgQ09ORklHX0RNQVJEMDkgaXMgbm90IHNldAojIENPTkZJR19ETUFSRDEwIGlz
IG5vdCBzZXQKIyBDT05GSUdfRlhMUzg5NjJBRl9JMkMgaXMgbm90IHNldAojIENPTkZJR19JSU9f
U1RfQUNDRUxfM0FYSVMgaXMgbm90IHNldAojIENPTkZJR19LWFNEOSBpcyBub3Qgc2V0CiMgQ09O
RklHX0tYQ0pLMTAxMyBpcyBub3Qgc2V0CiMgQ09ORklHX01DMzIzMCBpcyBub3Qgc2V0CiMgQ09O
RklHX01NQTc0NTVfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfTU1BNzY2MCBpcyBub3Qgc2V0CiMg
Q09ORklHX01NQTg0NTIgaXMgbm90IHNldAojIENPTkZJR19NTUE5NTUxIGlzIG5vdCBzZXQKIyBD
T05GSUdfTU1BOTU1MyBpcyBub3Qgc2V0CiMgQ09ORklHX01YQzQwMDUgaXMgbm90IHNldAojIENP
TkZJR19NWEM2MjU1IGlzIG5vdCBzZXQKIyBDT05GSUdfU1RLODMxMiBpcyBub3Qgc2V0CiMgQ09O
RklHX1NUSzhCQTUwIGlzIG5vdCBzZXQKIyBlbmQgb2YgQWNjZWxlcm9tZXRlcnMKCiMKIyBBbmFs
b2cgdG8gZGlnaXRhbCBjb252ZXJ0ZXJzCiMKIyBDT05GSUdfQUQ3MDkxUjUgaXMgbm90IHNldAoj
IENPTkZJR19BRDcyOTEgaXMgbm90IHNldAojIENPTkZJR19BRDc2MDZfSUZBQ0VfUEFSQUxMRUwg
aXMgbm90IHNldAojIENPTkZJR19BRDc5OVggaXMgbm90IHNldAojIENPTkZJR19BRElfQVhJX0FE
QyBpcyBub3Qgc2V0CiMgQ09ORklHX0NDMTAwMDFfQURDIGlzIG5vdCBzZXQKIyBDT05GSUdfRU5W
RUxPUEVfREVURUNUT1IgaXMgbm90IHNldAojIENPTkZJR19IWDcxMSBpcyBub3Qgc2V0CkNPTkZJ
R19JTkEyWFhfQURDPXkKIyBDT05GSUdfTFRDMjQ3MSBpcyBub3Qgc2V0CiMgQ09ORklHX0xUQzI0
ODUgaXMgbm90IHNldAojIENPTkZJR19MVEMyNDk3IGlzIG5vdCBzZXQKIyBDT05GSUdfTUFYMTM2
MyBpcyBub3Qgc2V0CiMgQ09ORklHX01BWDk2MTEgaXMgbm90IHNldAojIENPTkZJR19NQ1AzNDIy
IGlzIG5vdCBzZXQKIyBDT05GSUdfTkFVNzgwMiBpcyBub3Qgc2V0CiMgQ09ORklHX1NEX0FEQ19N
T0RVTEFUT1IgaXMgbm90IHNldAojIENPTkZJR19USV9BREMwODFDIGlzIG5vdCBzZXQKIyBDT05G
SUdfVElfQURTMTAxNSBpcyBub3Qgc2V0CiMgQ09ORklHX1ZGNjEwX0FEQyBpcyBub3Qgc2V0CkNP
TkZJR19YSUxJTlhfWEFEQz15CiMgZW5kIG9mIEFuYWxvZyB0byBkaWdpdGFsIGNvbnZlcnRlcnMK
CiMKIyBBbmFsb2cgRnJvbnQgRW5kcwojCiMgQ09ORklHX0lJT19SRVNDQUxFIGlzIG5vdCBzZXQK
IyBlbmQgb2YgQW5hbG9nIEZyb250IEVuZHMKCiMKIyBBbXBsaWZpZXJzCiMKIyBDT05GSUdfSE1D
NDI1IGlzIG5vdCBzZXQKIyBlbmQgb2YgQW1wbGlmaWVycwoKIwojIENhcGFjaXRhbmNlIHRvIGRp
Z2l0YWwgY29udmVydGVycwojCiMgQ09ORklHX0FENzE1MCBpcyBub3Qgc2V0CiMgZW5kIG9mIENh
cGFjaXRhbmNlIHRvIGRpZ2l0YWwgY29udmVydGVycwoKIwojIENoZW1pY2FsIFNlbnNvcnMKIwoj
IENPTkZJR19BVExBU19QSF9TRU5TT1IgaXMgbm90IHNldAojIENPTkZJR19BVExBU19FWk9fU0VO
U09SIGlzIG5vdCBzZXQKQ09ORklHX0JNRTY4MD15CkNPTkZJR19CTUU2ODBfSTJDPXkKIyBDT05G
SUdfQ0NTODExIGlzIG5vdCBzZXQKIyBDT05GSUdfSUFRQ09SRSBpcyBub3Qgc2V0CiMgQ09ORklH
X1BNUzcwMDMgaXMgbm90IHNldAojIENPTkZJR19TQ0QzMF9DT1JFIGlzIG5vdCBzZXQKIyBDT05G
SUdfU0VOU0lSSU9OX1NHUDMwIGlzIG5vdCBzZXQKIyBDT05GSUdfU1BTMzBfSTJDIGlzIG5vdCBz
ZXQKIyBDT05GSUdfU1BTMzBfU0VSSUFMIGlzIG5vdCBzZXQKIyBDT05GSUdfVlo4OVggaXMgbm90
IHNldAojIGVuZCBvZiBDaGVtaWNhbCBTZW5zb3JzCgojCiMgSGlkIFNlbnNvciBJSU8gQ29tbW9u
CiMKIyBlbmQgb2YgSGlkIFNlbnNvciBJSU8gQ29tbW9uCgojCiMgSUlPIFNDTUkgU2Vuc29ycwoj
CiMgZW5kIG9mIElJTyBTQ01JIFNlbnNvcnMKCiMKIyBTU1AgU2Vuc29yIENvbW1vbgojCiMgZW5k
IG9mIFNTUCBTZW5zb3IgQ29tbW9uCgojCiMgRGlnaXRhbCB0byBhbmFsb2cgY29udmVydGVycwoj
CiMgQ09ORklHX0FENTA2NCBpcyBub3Qgc2V0CiMgQ09ORklHX0FENTM4MCBpcyBub3Qgc2V0CiMg
Q09ORklHX0FENTQ0NiBpcyBub3Qgc2V0CiMgQ09ORklHX0FENTU5M1IgaXMgbm90IHNldAojIENP
TkZJR19BRDU2OTZfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfRFBPVF9EQUMgaXMgbm90IHNldAoj
IENPTkZJR19EUzQ0MjQgaXMgbm90IHNldAojIENPTkZJR19NNjIzMzIgaXMgbm90IHNldAojIENP
TkZJR19NQVg1MTcgaXMgbm90IHNldAojIENPTkZJR19NQVg1ODIxIGlzIG5vdCBzZXQKIyBDT05G
SUdfTUNQNDcyNSBpcyBub3Qgc2V0CiMgQ09ORklHX1RJX0RBQzU1NzEgaXMgbm90IHNldAojIENP
TkZJR19WRjYxMF9EQUMgaXMgbm90IHNldAojIGVuZCBvZiBEaWdpdGFsIHRvIGFuYWxvZyBjb252
ZXJ0ZXJzCgojCiMgSUlPIGR1bW15IGRyaXZlcgojCiMgZW5kIG9mIElJTyBkdW1teSBkcml2ZXIK
CiMKIyBGcmVxdWVuY3kgU3ludGhlc2l6ZXJzIEREUy9QTEwKIwoKIwojIENsb2NrIEdlbmVyYXRv
ci9EaXN0cmlidXRpb24KIwojIGVuZCBvZiBDbG9jayBHZW5lcmF0b3IvRGlzdHJpYnV0aW9uCgoj
CiMgUGhhc2UtTG9ja2VkIExvb3AgKFBMTCkgZnJlcXVlbmN5IHN5bnRoZXNpemVycwojCiMgZW5k
IG9mIFBoYXNlLUxvY2tlZCBMb29wIChQTEwpIGZyZXF1ZW5jeSBzeW50aGVzaXplcnMKIyBlbmQg
b2YgRnJlcXVlbmN5IFN5bnRoZXNpemVycyBERFMvUExMCgojCiMgRGlnaXRhbCBneXJvc2NvcGUg
c2Vuc29ycwojCkNPTkZJR19CTUcxNjA9eQpDT05GSUdfQk1HMTYwX0kyQz15CiMgQ09ORklHX0ZY
QVMyMTAwMkMgaXMgbm90IHNldAojIENPTkZJR19NUFUzMDUwX0kyQyBpcyBub3Qgc2V0CiMgQ09O
RklHX0lJT19TVF9HWVJPXzNBWElTIGlzIG5vdCBzZXQKIyBDT05GSUdfSVRHMzIwMCBpcyBub3Qg
c2V0CiMgZW5kIG9mIERpZ2l0YWwgZ3lyb3Njb3BlIHNlbnNvcnMKCiMKIyBIZWFsdGggU2Vuc29y
cwojCgojCiMgSGVhcnQgUmF0ZSBNb25pdG9ycwojCiMgQ09ORklHX0FGRTQ0MDQgaXMgbm90IHNl
dAojIENPTkZJR19NQVgzMDEwMCBpcyBub3Qgc2V0CiMgQ09ORklHX01BWDMwMTAyIGlzIG5vdCBz
ZXQKIyBlbmQgb2YgSGVhcnQgUmF0ZSBNb25pdG9ycwojIGVuZCBvZiBIZWFsdGggU2Vuc29ycwoK
IwojIEh1bWlkaXR5IHNlbnNvcnMKIwojIENPTkZJR19BTTIzMTUgaXMgbm90IHNldAojIENPTkZJ
R19ESFQxMSBpcyBub3Qgc2V0CiMgQ09ORklHX0hEQzEwMFggaXMgbm90IHNldAojIENPTkZJR19I
REMyMDEwIGlzIG5vdCBzZXQKIyBDT05GSUdfSFRTMjIxIGlzIG5vdCBzZXQKIyBDT05GSUdfSFRV
MjEgaXMgbm90IHNldAojIENPTkZJR19TSTcwMDUgaXMgbm90IHNldAojIENPTkZJR19TSTcwMjAg
aXMgbm90IHNldAojIGVuZCBvZiBIdW1pZGl0eSBzZW5zb3JzCgojCiMgSW5lcnRpYWwgbWVhc3Vy
ZW1lbnQgdW5pdHMKIwojIENPTkZJR19CTUkxNjBfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfRlhP
Uzg3MDBfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfS01YNjEgaXMgbm90IHNldAojIENPTkZJR19J
TlZfSUNNNDI2MDBfSTJDIGlzIG5vdCBzZXQKIyBDT05GSUdfSU5WX01QVTYwNTBfSTJDIGlzIG5v
dCBzZXQKIyBDT05GSUdfSUlPX1NUX0xTTTZEU1ggaXMgbm90IHNldAojIENPTkZJR19JSU9fU1Rf
TFNNOURTMCBpcyBub3Qgc2V0CiMgZW5kIG9mIEluZXJ0aWFsIG1lYXN1cmVtZW50IHVuaXRzCgoj
CiMgTGlnaHQgc2Vuc29ycwojCiMgQ09ORklHX0FESkRfUzMxMSBpcyBub3Qgc2V0CiMgQ09ORklH
X0FEVVgxMDIwIGlzIG5vdCBzZXQKIyBDT05GSUdfQUwzMDEwIGlzIG5vdCBzZXQKIyBDT05GSUdf
QUwzMzIwQSBpcyBub3Qgc2V0CiMgQ09ORklHX0FQRFM5MzAwIGlzIG5vdCBzZXQKIyBDT05GSUdf
QVBEUzk5NjAgaXMgbm90IHNldAojIENPTkZJR19BUzczMjExIGlzIG5vdCBzZXQKIyBDT05GSUdf
QkgxNzUwIGlzIG5vdCBzZXQKIyBDT05GSUdfQkgxNzgwIGlzIG5vdCBzZXQKIyBDT05GSUdfQ00z
MjE4MSBpcyBub3Qgc2V0CiMgQ09ORklHX0NNMzIzMiBpcyBub3Qgc2V0CiMgQ09ORklHX0NNMzMy
MyBpcyBub3Qgc2V0CiMgQ09ORklHX0NNMzYwNSBpcyBub3Qgc2V0CiMgQ09ORklHX0NNMzY2NTEg
aXMgbm90IHNldAojIENPTkZJR19HUDJBUDAwMiBpcyBub3Qgc2V0CiMgQ09ORklHX0dQMkFQMDIw
QTAwRiBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfSVNMMjkwMTggaXMgbm90IHNldAojIENP
TkZJR19TRU5TT1JTX0lTTDI5MDI4IGlzIG5vdCBzZXQKIyBDT05GSUdfSVNMMjkxMjUgaXMgbm90
IHNldAojIENPTkZJR19KU0ExMjEyIGlzIG5vdCBzZXQKIyBDT05GSUdfUlBSMDUyMSBpcyBub3Qg
c2V0CiMgQ09ORklHX0xUUjUwMSBpcyBub3Qgc2V0CiMgQ09ORklHX0xWMDEwNENTIGlzIG5vdCBz
ZXQKIyBDT05GSUdfTUFYNDQwMDAgaXMgbm90IHNldAojIENPTkZJR19NQVg0NDAwOSBpcyBub3Qg
c2V0CiMgQ09ORklHX05PQTEzMDUgaXMgbm90IHNldAojIENPTkZJR19PUFQzMDAxIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUEExMjIwMzAwMSBpcyBub3Qgc2V0CiMgQ09ORklHX1NJMTEzMyBpcyBub3Qg
c2V0CiMgQ09ORklHX1NJMTE0NSBpcyBub3Qgc2V0CiMgQ09ORklHX1NUSzMzMTAgaXMgbm90IHNl
dAojIENPTkZJR19TVF9VVklTMjUgaXMgbm90IHNldAojIENPTkZJR19UQ1MzNDE0IGlzIG5vdCBz
ZXQKIyBDT05GSUdfVENTMzQ3MiBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfVFNMMjU2MyBp
cyBub3Qgc2V0CiMgQ09ORklHX1RTTDI1ODMgaXMgbm90IHNldAojIENPTkZJR19UU0wyNTkxIGlz
IG5vdCBzZXQKIyBDT05GSUdfVFNMMjc3MiBpcyBub3Qgc2V0CiMgQ09ORklHX1RTTDQ1MzEgaXMg
bm90IHNldAojIENPTkZJR19VUzUxODJEIGlzIG5vdCBzZXQKIyBDT05GSUdfVkNOTDQwMDAgaXMg
bm90IHNldAojIENPTkZJR19WQ05MNDAzNSBpcyBub3Qgc2V0CiMgQ09ORklHX1ZFTUw2MDMwIGlz
IG5vdCBzZXQKIyBDT05GSUdfVkVNTDYwNzAgaXMgbm90IHNldAojIENPTkZJR19WTDYxODAgaXMg
bm90IHNldAojIENPTkZJR19aT1BUMjIwMSBpcyBub3Qgc2V0CiMgZW5kIG9mIExpZ2h0IHNlbnNv
cnMKCiMKIyBNYWduZXRvbWV0ZXIgc2Vuc29ycwojCiMgQ09ORklHX0FLODk3NCBpcyBub3Qgc2V0
CiMgQ09ORklHX0FLODk3NSBpcyBub3Qgc2V0CiMgQ09ORklHX0FLMDk5MTEgaXMgbm90IHNldAoj
IENPTkZJR19CTUMxNTBfTUFHTl9JMkMgaXMgbm90IHNldAojIENPTkZJR19NQUczMTEwIGlzIG5v
dCBzZXQKIyBDT05GSUdfTU1DMzUyNDAgaXMgbm90IHNldAojIENPTkZJR19JSU9fU1RfTUFHTl8z
QVhJUyBpcyBub3Qgc2V0CiMgQ09ORklHX1NFTlNPUlNfSE1DNTg0M19JMkMgaXMgbm90IHNldAoj
IENPTkZJR19TRU5TT1JTX1JNMzEwMF9JMkMgaXMgbm90IHNldAojIENPTkZJR19ZQU1BSEFfWUFT
NTMwIGlzIG5vdCBzZXQKIyBlbmQgb2YgTWFnbmV0b21ldGVyIHNlbnNvcnMKCiMKIyBNdWx0aXBs
ZXhlcnMKIwojIENPTkZJR19JSU9fTVVYIGlzIG5vdCBzZXQKIyBlbmQgb2YgTXVsdGlwbGV4ZXJz
CgojCiMgSW5jbGlub21ldGVyIHNlbnNvcnMKIwojIGVuZCBvZiBJbmNsaW5vbWV0ZXIgc2Vuc29y
cwoKIwojIFRyaWdnZXJzIC0gc3RhbmRhbG9uZQojCiMgQ09ORklHX0lJT19JTlRFUlJVUFRfVFJJ
R0dFUiBpcyBub3Qgc2V0CiMgQ09ORklHX0lJT19TWVNGU19UUklHR0VSIGlzIG5vdCBzZXQKIyBl
bmQgb2YgVHJpZ2dlcnMgLSBzdGFuZGFsb25lCgojCiMgTGluZWFyIGFuZCBhbmd1bGFyIHBvc2l0
aW9uIHNlbnNvcnMKIwojIGVuZCBvZiBMaW5lYXIgYW5kIGFuZ3VsYXIgcG9zaXRpb24gc2Vuc29y
cwoKIwojIERpZ2l0YWwgcG90ZW50aW9tZXRlcnMKIwojIENPTkZJR19BRDUyNzIgaXMgbm90IHNl
dAojIENPTkZJR19EUzE4MDMgaXMgbm90IHNldAojIENPTkZJR19NQVg1NDMyIGlzIG5vdCBzZXQK
IyBDT05GSUdfTUNQNDAxOCBpcyBub3Qgc2V0CiMgQ09ORklHX01DUDQ1MzEgaXMgbm90IHNldAoj
IENPTkZJR19UUEwwMTAyIGlzIG5vdCBzZXQKIyBlbmQgb2YgRGlnaXRhbCBwb3RlbnRpb21ldGVy
cwoKIwojIERpZ2l0YWwgcG90ZW50aW9zdGF0cwojCiMgQ09ORklHX0xNUDkxMDAwIGlzIG5vdCBz
ZXQKIyBlbmQgb2YgRGlnaXRhbCBwb3RlbnRpb3N0YXRzCgojCiMgUHJlc3N1cmUgc2Vuc29ycwoj
CiMgQ09ORklHX0FCUDA2ME1HIGlzIG5vdCBzZXQKIyBDT05GSUdfQk1QMjgwIGlzIG5vdCBzZXQK
IyBDT05GSUdfRExITDYwRCBpcyBub3Qgc2V0CiMgQ09ORklHX0RQUzMxMCBpcyBub3Qgc2V0CiMg
Q09ORklHX0hQMDMgaXMgbm90IHNldAojIENPTkZJR19JQ1AxMDEwMCBpcyBub3Qgc2V0CiMgQ09O
RklHX01QTDExNV9JMkMgaXMgbm90IHNldAojIENPTkZJR19NUEwzMTE1IGlzIG5vdCBzZXQKIyBD
T05GSUdfTVM1NjExIGlzIG5vdCBzZXQKIyBDT05GSUdfTVM1NjM3IGlzIG5vdCBzZXQKIyBDT05G
SUdfSUlPX1NUX1BSRVNTIGlzIG5vdCBzZXQKIyBDT05GSUdfVDU0MDMgaXMgbm90IHNldAojIENP
TkZJR19IUDIwNkMgaXMgbm90IHNldAojIENPTkZJR19aUEEyMzI2IGlzIG5vdCBzZXQKIyBlbmQg
b2YgUHJlc3N1cmUgc2Vuc29ycwoKIwojIExpZ2h0bmluZyBzZW5zb3JzCiMKIyBlbmQgb2YgTGln
aHRuaW5nIHNlbnNvcnMKCiMKIyBQcm94aW1pdHkgYW5kIGRpc3RhbmNlIHNlbnNvcnMKIwojIENP
TkZJR19JU0wyOTUwMSBpcyBub3Qgc2V0CiMgQ09ORklHX0xJREFSX0xJVEVfVjIgaXMgbm90IHNl
dAojIENPTkZJR19NQjEyMzIgaXMgbm90IHNldAojIENPTkZJR19QSU5HIGlzIG5vdCBzZXQKIyBD
T05GSUdfUkZENzc0MDIgaXMgbm90IHNldAojIENPTkZJR19TUkYwNCBpcyBub3Qgc2V0CiMgQ09O
RklHX1NYOTMxMCBpcyBub3Qgc2V0CiMgQ09ORklHX1NYOTUwMCBpcyBub3Qgc2V0CiMgQ09ORklH
X1NSRjA4IGlzIG5vdCBzZXQKIyBDT05GSUdfVkNOTDMwMjAgaXMgbm90IHNldAojIENPTkZJR19W
TDUzTDBYX0kyQyBpcyBub3Qgc2V0CiMgZW5kIG9mIFByb3hpbWl0eSBhbmQgZGlzdGFuY2Ugc2Vu
c29ycwoKIwojIFJlc29sdmVyIHRvIGRpZ2l0YWwgY29udmVydGVycwojCiMgZW5kIG9mIFJlc29s
dmVyIHRvIGRpZ2l0YWwgY29udmVydGVycwoKIwojIFRlbXBlcmF0dXJlIHNlbnNvcnMKIwojIENP
TkZJR19NTFg5MDYxNCBpcyBub3Qgc2V0CiMgQ09ORklHX01MWDkwNjMyIGlzIG5vdCBzZXQKIyBD
T05GSUdfVE1QMDA2IGlzIG5vdCBzZXQKIyBDT05GSUdfVE1QMDA3IGlzIG5vdCBzZXQKIyBDT05G
SUdfVE1QMTE3IGlzIG5vdCBzZXQKIyBDT05GSUdfVFNZUzAxIGlzIG5vdCBzZXQKIyBDT05GSUdf
VFNZUzAyRCBpcyBub3Qgc2V0CiMgZW5kIG9mIFRlbXBlcmF0dXJlIHNlbnNvcnMKCiMgQ09ORklH
X1BXTSBpcyBub3Qgc2V0CgojCiMgSVJRIGNoaXAgc3VwcG9ydAojCkNPTkZJR19JUlFDSElQPXkK
Q09ORklHX0FSTV9HSUM9eQpDT05GSUdfQVJNX0dJQ19NQVhfTlI9MQpDT05GSUdfQVJNX0dJQ19W
Mz15CkNPTkZJR19BUk1fR0lDX1YzX0lUUz15CiMgQ09ORklHX0FMX0ZJQyBpcyBub3Qgc2V0CiMg
Q09ORklHX1hJTElOWF9JTlRDIGlzIG5vdCBzZXQKQ09ORklHX1BBUlRJVElPTl9QRVJDUFU9eQoj
IGVuZCBvZiBJUlEgY2hpcCBzdXBwb3J0CgojIENPTkZJR19JUEFDS19CVVMgaXMgbm90IHNldApD
T05GSUdfUkVTRVRfQ09OVFJPTExFUj15CiMgQ09ORklHX1JFU0VUX1RJX1NZU0NPTiBpcyBub3Qg
c2V0CgojCiMgUEhZIFN1YnN5c3RlbQojCkNPTkZJR19HRU5FUklDX1BIWT15CiMgQ09ORklHX1BI
WV9YR0VORSBpcyBub3Qgc2V0CiMgQ09ORklHX1BIWV9DQU5fVFJBTlNDRUlWRVIgaXMgbm90IHNl
dAojIENPTkZJR19CQ01fS09OQV9VU0IyX1BIWSBpcyBub3Qgc2V0CiMgQ09ORklHX1BIWV9DQURF
TkNFX1RPUlJFTlQgaXMgbm90IHNldAojIENPTkZJR19QSFlfQ0FERU5DRV9EUEhZIGlzIG5vdCBz
ZXQKIyBDT05GSUdfUEhZX0NBREVOQ0VfU0lFUlJBIGlzIG5vdCBzZXQKIyBDT05GSUdfUEhZX0NB
REVOQ0VfU0FMVk8gaXMgbm90IHNldAojIENPTkZJR19QSFlfRlNMX0lNWDhNUV9VU0IgaXMgbm90
IHNldAojIENPTkZJR19QSFlfTUlYRUxfTUlQSV9EUEhZIGlzIG5vdCBzZXQKIyBDT05GSUdfUEhZ
X1BYQV8yOE5NX0hTSUMgaXMgbm90IHNldAojIENPTkZJR19QSFlfUFhBXzI4Tk1fVVNCMiBpcyBu
b3Qgc2V0CiMgQ09ORklHX1BIWV9YSUxJTlhfWllOUU1QIGlzIG5vdCBzZXQKIyBlbmQgb2YgUEhZ
IFN1YnN5c3RlbQoKIyBDT05GSUdfUE9XRVJDQVAgaXMgbm90IHNldAojIENPTkZJR19NQ0IgaXMg
bm90IHNldAoKIwojIFBlcmZvcm1hbmNlIG1vbml0b3Igc3VwcG9ydAojCiMgQ09ORklHX0FSTV9D
Q0lfUE1VIGlzIG5vdCBzZXQKIyBDT05GSUdfQVJNX0NDTiBpcyBub3Qgc2V0CiMgQ09ORklHX0FS
TV9DTU4gaXMgbm90IHNldApDT05GSUdfQVJNX1BNVT15CiMgQ09ORklHX0FSTV9EU1VfUE1VIGlz
IG5vdCBzZXQKIyBDT05GSUdfQVJNX1NQRV9QTVUgaXMgbm90IHNldAojIGVuZCBvZiBQZXJmb3Jt
YW5jZSBtb25pdG9yIHN1cHBvcnQKCkNPTkZJR19SQVM9eQoKIwojIEFuZHJvaWQKIwpDT05GSUdf
QU5EUk9JRD15CiMgQ09ORklHX0FORFJPSURfQklOREVSX0lQQyBpcyBub3Qgc2V0CiMgZW5kIG9m
IEFuZHJvaWQKCiMgQ09ORklHX0RBWCBpcyBub3Qgc2V0CkNPTkZJR19OVk1FTT15CkNPTkZJR19O
Vk1FTV9TWVNGUz15CkNPTkZJR19OVk1FTV9aWU5RTVA9eQojIENPTkZJR19OVk1FTV9STUVNIGlz
IG5vdCBzZXQKCiMKIyBIVyB0cmFjaW5nIHN1cHBvcnQKIwojIENPTkZJR19TVE0gaXMgbm90IHNl
dAojIENPTkZJR19JTlRFTF9USCBpcyBub3Qgc2V0CiMgZW5kIG9mIEhXIHRyYWNpbmcgc3VwcG9y
dAoKQ09ORklHX0ZQR0E9eQojIENPTkZJR19BTFRFUkFfUFJfSVBfQ09SRSBpcyBub3Qgc2V0CkNP
TkZJR19GUEdBX0JSSURHRT15CiMgQ09ORklHX0FMVEVSQV9GUkVFWkVfQlJJREdFIGlzIG5vdCBz
ZXQKQ09ORklHX1hJTElOWF9QUl9ERUNPVVBMRVI9eQpDT05GSUdfRlBHQV9SRUdJT049eQpDT05G
SUdfT0ZfRlBHQV9SRUdJT049eQojIENPTkZJR19GUEdBX0RGTCBpcyBub3Qgc2V0CkNPTkZJR19G
UEdBX01HUl9aWU5RTVBfRlBHQT15CiMgQ09ORklHX0ZTSSBpcyBub3Qgc2V0CiMgQ09ORklHX1RF
RSBpcyBub3Qgc2V0CkNPTkZJR19QTV9PUFA9eQojIENPTkZJR19TSU9YIGlzIG5vdCBzZXQKIyBD
T05GSUdfU0xJTUJVUyBpcyBub3Qgc2V0CiMgQ09ORklHX0lOVEVSQ09OTkVDVCBpcyBub3Qgc2V0
CiMgQ09ORklHX0NPVU5URVIgaXMgbm90IHNldAojIENPTkZJR19NT1NUIGlzIG5vdCBzZXQKIyBl
bmQgb2YgRGV2aWNlIERyaXZlcnMKCiMKIyBGaWxlIHN5c3RlbXMKIwpDT05GSUdfRENBQ0hFX1dP
UkRfQUNDRVNTPXkKIyBDT05GSUdfVkFMSURBVEVfRlNfUEFSU0VSIGlzIG5vdCBzZXQKQ09ORklH
X0ZTX1BPU0lYX0FDTD15CkNPTkZJR19FWFBPUlRGUz15CiMgQ09ORklHX0VYUE9SVEZTX0JMT0NL
X09QUyBpcyBub3Qgc2V0CiMgQ09ORklHX0ZJTEVfTE9DS0lORyBpcyBub3Qgc2V0CiMgQ09ORklH
X0ZTX0VOQ1JZUFRJT04gaXMgbm90IHNldAojIENPTkZJR19GU19WRVJJVFkgaXMgbm90IHNldAoj
IENPTkZJR19ETk9USUZZIGlzIG5vdCBzZXQKIyBDT05GSUdfSU5PVElGWV9VU0VSIGlzIG5vdCBz
ZXQKIyBDT05GSUdfRkFOT1RJRlkgaXMgbm90IHNldAojIENPTkZJR19RVU9UQSBpcyBub3Qgc2V0
CiMgQ09ORklHX0FVVE9GUzRfRlMgaXMgbm90IHNldAojIENPTkZJR19BVVRPRlNfRlMgaXMgbm90
IHNldAojIENPTkZJR19GVVNFX0ZTIGlzIG5vdCBzZXQKIyBDT05GSUdfT1ZFUkxBWV9GUyBpcyBu
b3Qgc2V0CgojCiMgQ2FjaGVzCiMKIyBDT05GSUdfRlNDQUNIRSBpcyBub3Qgc2V0CiMgZW5kIG9m
IENhY2hlcwoKIwojIFBzZXVkbyBmaWxlc3lzdGVtcwojCkNPTkZJR19QUk9DX0ZTPXkKIyBDT05G
SUdfUFJPQ19LQ09SRSBpcyBub3Qgc2V0CkNPTkZJR19QUk9DX1NZU0NUTD15CkNPTkZJR19QUk9D
X1BBR0VfTU9OSVRPUj15CiMgQ09ORklHX1BST0NfQ0hJTERSRU4gaXMgbm90IHNldApDT05GSUdf
S0VSTkZTPXkKQ09ORklHX1NZU0ZTPXkKQ09ORklHX1RNUEZTPXkKQ09ORklHX1RNUEZTX1BPU0lY
X0FDTD15CkNPTkZJR19UTVBGU19YQVRUUj15CiMgQ09ORklHX1RNUEZTX0lOT0RFNjQgaXMgbm90
IHNldApDT05GSUdfQVJDSF9TVVBQT1JUU19IVUdFVExCRlM9eQpDT05GSUdfSFVHRVRMQkZTPXkK
Q09ORklHX0hVR0VUTEJfUEFHRT15CkNPTkZJR19NRU1GRF9DUkVBVEU9eQpDT05GSUdfQVJDSF9I
QVNfR0lHQU5USUNfUEFHRT15CkNPTkZJR19DT05GSUdGU19GUz15CkNPTkZJR19FRklWQVJfRlM9
eQojIGVuZCBvZiBQc2V1ZG8gZmlsZXN5c3RlbXMKCiMgQ09ORklHX01JU0NfRklMRVNZU1RFTVMg
aXMgbm90IHNldAojIENPTkZJR19OTFMgaXMgbm90IHNldAojIENPTkZJR19VTklDT0RFIGlzIG5v
dCBzZXQKQ09ORklHX0lPX1dRPXkKIyBlbmQgb2YgRmlsZSBzeXN0ZW1zCgojCiMgU2VjdXJpdHkg
b3B0aW9ucwojCkNPTkZJR19LRVlTPXkKIyBDT05GSUdfS0VZU19SRVFVRVNUX0NBQ0hFIGlzIG5v
dCBzZXQKIyBDT05GSUdfUEVSU0lTVEVOVF9LRVlSSU5HUyBpcyBub3Qgc2V0CiMgQ09ORklHX1RS
VVNURURfS0VZUyBpcyBub3Qgc2V0CiMgQ09ORklHX0VOQ1JZUFRFRF9LRVlTIGlzIG5vdCBzZXQK
IyBDT05GSUdfS0VZX0RIX09QRVJBVElPTlMgaXMgbm90IHNldAojIENPTkZJR19TRUNVUklUWV9E
TUVTR19SRVNUUklDVCBpcyBub3Qgc2V0CiMgQ09ORklHX1NFQ1VSSVRZIGlzIG5vdCBzZXQKQ09O
RklHX1NFQ1VSSVRZRlM9eQpDT05GSUdfSEFWRV9IQVJERU5FRF9VU0VSQ09QWV9BTExPQ0FUT1I9
eQojIENPTkZJR19IQVJERU5FRF9VU0VSQ09QWSBpcyBub3Qgc2V0CiMgQ09ORklHX0ZPUlRJRllf
U09VUkNFIGlzIG5vdCBzZXQKIyBDT05GSUdfU1RBVElDX1VTRVJNT0RFSEVMUEVSIGlzIG5vdCBz
ZXQKIyBDT05GSUdfSU1BX1NFQ1VSRV9BTkRfT1JfVFJVU1RFRF9CT09UIGlzIG5vdCBzZXQKQ09O
RklHX0RFRkFVTFRfU0VDVVJJVFlfREFDPXkKQ09ORklHX0xTTT0ibGFuZGxvY2ssbG9ja2Rvd24s
eWFtYSxsb2FkcGluLHNhZmVzZXRpZCxpbnRlZ3JpdHksYnBmIgoKIwojIEtlcm5lbCBoYXJkZW5p
bmcgb3B0aW9ucwojCgojCiMgTWVtb3J5IGluaXRpYWxpemF0aW9uCiMKQ09ORklHX0lOSVRfU1RB
Q0tfTk9ORT15CiMgQ09ORklHX0lOSVRfT05fQUxMT0NfREVGQVVMVF9PTiBpcyBub3Qgc2V0CiMg
Q09ORklHX0lOSVRfT05fRlJFRV9ERUZBVUxUX09OIGlzIG5vdCBzZXQKIyBlbmQgb2YgTWVtb3J5
IGluaXRpYWxpemF0aW9uCiMgZW5kIG9mIEtlcm5lbCBoYXJkZW5pbmcgb3B0aW9ucwojIGVuZCBv
ZiBTZWN1cml0eSBvcHRpb25zCgpDT05GSUdfQ1JZUFRPPXkKCiMKIyBDcnlwdG8gY29yZSBvciBo
ZWxwZXIKIwojIENPTkZJR19DUllQVE9fRklQUyBpcyBub3Qgc2V0CkNPTkZJR19DUllQVE9fQUxH
QVBJPXkKQ09ORklHX0NSWVBUT19BTEdBUEkyPXkKQ09ORklHX0NSWVBUT19BRUFEPXkKQ09ORklH
X0NSWVBUT19BRUFEMj15CkNPTkZJR19DUllQVE9fU0tDSVBIRVI9eQpDT05GSUdfQ1JZUFRPX1NL
Q0lQSEVSMj15CkNPTkZJR19DUllQVE9fSEFTSD15CkNPTkZJR19DUllQVE9fSEFTSDI9eQpDT05G
SUdfQ1JZUFRPX1JORz15CkNPTkZJR19DUllQVE9fUk5HMj15CkNPTkZJR19DUllQVE9fUk5HX0RF
RkFVTFQ9eQpDT05GSUdfQ1JZUFRPX0FLQ0lQSEVSMj15CkNPTkZJR19DUllQVE9fQUtDSVBIRVI9
eQpDT05GSUdfQ1JZUFRPX0tQUDI9eQpDT05GSUdfQ1JZUFRPX0tQUD15CkNPTkZJR19DUllQVE9f
QUNPTVAyPXkKQ09ORklHX0NSWVBUT19NQU5BR0VSPXkKQ09ORklHX0NSWVBUT19NQU5BR0VSMj15
CiMgQ09ORklHX0NSWVBUT19NQU5BR0VSX0RJU0FCTEVfVEVTVFMgaXMgbm90IHNldAojIENPTkZJ
R19DUllQVE9fTUFOQUdFUl9FWFRSQV9URVNUUyBpcyBub3Qgc2V0CkNPTkZJR19DUllQVE9fR0Yx
MjhNVUw9eQpDT05GSUdfQ1JZUFRPX05VTEw9eQpDT05GSUdfQ1JZUFRPX05VTEwyPXkKIyBDT05G
SUdfQ1JZUFRPX1BDUllQVCBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19DUllQVEQgaXMgbm90
IHNldAojIENPTkZJR19DUllQVE9fQVVUSEVOQyBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19U
RVNUIGlzIG5vdCBzZXQKQ09ORklHX0NSWVBUT19FTkdJTkU9eQoKIwojIFB1YmxpYy1rZXkgY3J5
cHRvZ3JhcGh5CiMKQ09ORklHX0NSWVBUT19SU0E9eQojIENPTkZJR19DUllQVE9fREggaXMgbm90
IHNldApDT05GSUdfQ1JZUFRPX0VDQz15CkNPTkZJR19DUllQVE9fRUNESD15CiMgQ09ORklHX0NS
WVBUT19FQ0RTQSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19FQ1JEU0EgaXMgbm90IHNldAoj
IENPTkZJR19DUllQVE9fU00yIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JZUFRPX0NVUlZFMjU1MTkg
aXMgbm90IHNldAoKIwojIEF1dGhlbnRpY2F0ZWQgRW5jcnlwdGlvbiB3aXRoIEFzc29jaWF0ZWQg
RGF0YQojCkNPTkZJR19DUllQVE9fQ0NNPXkKQ09ORklHX0NSWVBUT19HQ009eQojIENPTkZJR19D
UllQVE9fQ0hBQ0hBMjBQT0xZMTMwNSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19BRUdJUzEy
OCBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19TRVFJViBpcyBub3Qgc2V0CiMgQ09ORklHX0NS
WVBUT19FQ0hBSU5JViBpcyBub3Qgc2V0CgojCiMgQmxvY2sgbW9kZXMKIwpDT05GSUdfQ1JZUFRP
X0NCQz15CiMgQ09ORklHX0NSWVBUT19DRkIgaXMgbm90IHNldApDT05GSUdfQ1JZUFRPX0NUUj15
CiMgQ09ORklHX0NSWVBUT19DVFMgaXMgbm90IHNldApDT05GSUdfQ1JZUFRPX0VDQj15CiMgQ09O
RklHX0NSWVBUT19MUlcgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fT0ZCIGlzIG5vdCBzZXQK
IyBDT05GSUdfQ1JZUFRPX1BDQkMgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fWFRTIGlzIG5v
dCBzZXQKIyBDT05GSUdfQ1JZUFRPX0tFWVdSQVAgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9f
QURJQU5UVU0gaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fRVNTSVYgaXMgbm90IHNldAoKIwoj
IEhhc2ggbW9kZXMKIwpDT05GSUdfQ1JZUFRPX0NNQUM9eQpDT05GSUdfQ1JZUFRPX0hNQUM9eQoj
IENPTkZJR19DUllQVE9fWENCQyBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19WTUFDIGlzIG5v
dCBzZXQKCiMKIyBEaWdlc3QKIwpDT05GSUdfQ1JZUFRPX0NSQzMyQz15CiMgQ09ORklHX0NSWVBU
T19DUkMzMiBpcyBub3Qgc2V0CkNPTkZJR19DUllQVE9fWFhIQVNIPXkKQ09ORklHX0NSWVBUT19C
TEFLRTJCPXkKIyBDT05GSUdfQ1JZUFRPX0JMQUtFMlMgaXMgbm90IHNldApDT05GSUdfQ1JZUFRP
X0NSQ1QxMERJRj15CkNPTkZJR19DUllQVE9fR0hBU0g9eQojIENPTkZJR19DUllQVE9fUE9MWTEz
MDUgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fTUQ0IGlzIG5vdCBzZXQKQ09ORklHX0NSWVBU
T19NRDU9eQojIENPTkZJR19DUllQVE9fTUlDSEFFTF9NSUMgaXMgbm90IHNldAojIENPTkZJR19D
UllQVE9fUk1EMTYwIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JZUFRPX1NIQTEgaXMgbm90IHNldAoj
IENPTkZJR19DUllQVE9fU0hBMjU2IGlzIG5vdCBzZXQKQ09ORklHX0NSWVBUT19TSEE1MTI9eQoj
IENPTkZJR19DUllQVE9fU0hBMyBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19TTTMgaXMgbm90
IHNldAojIENPTkZJR19DUllQVE9fU1RSRUVCT0cgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9f
V1A1MTIgaXMgbm90IHNldAoKIwojIENpcGhlcnMKIwpDT05GSUdfQ1JZUFRPX0FFUz15CiMgQ09O
RklHX0NSWVBUT19BRVNfVEkgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fQkxPV0ZJU0ggaXMg
bm90IHNldAojIENPTkZJR19DUllQVE9fQ0FNRUxMSUEgaXMgbm90IHNldAojIENPTkZJR19DUllQ
VE9fQ0FTVDUgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fQ0FTVDYgaXMgbm90IHNldAojIENP
TkZJR19DUllQVE9fREVTIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JZUFRPX0ZDUllQVCBpcyBub3Qg
c2V0CiMgQ09ORklHX0NSWVBUT19DSEFDSEEyMCBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19T
RVJQRU5UIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JZUFRPX1NNNCBpcyBub3Qgc2V0CiMgQ09ORklH
X0NSWVBUT19UV09GSVNIIGlzIG5vdCBzZXQKCiMKIyBDb21wcmVzc2lvbgojCkNPTkZJR19DUllQ
VE9fREVGTEFURT15CkNPTkZJR19DUllQVE9fTFpPPXkKIyBDT05GSUdfQ1JZUFRPXzg0MiBpcyBu
b3Qgc2V0CiMgQ09ORklHX0NSWVBUT19MWjQgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fTFo0
SEMgaXMgbm90IHNldApDT05GSUdfQ1JZUFRPX1pTVEQ9eQoKIwojIFJhbmRvbSBOdW1iZXIgR2Vu
ZXJhdGlvbgojCiMgQ09ORklHX0NSWVBUT19BTlNJX0NQUk5HIGlzIG5vdCBzZXQKQ09ORklHX0NS
WVBUT19EUkJHX01FTlU9eQpDT05GSUdfQ1JZUFRPX0RSQkdfSE1BQz15CiMgQ09ORklHX0NSWVBU
T19EUkJHX0hBU0ggaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fRFJCR19DVFIgaXMgbm90IHNl
dApDT05GSUdfQ1JZUFRPX0RSQkc9eQpDT05GSUdfQ1JZUFRPX0pJVFRFUkVOVFJPUFk9eQpDT05G
SUdfQ1JZUFRPX0hBU0hfSU5GTz15CgojCiMgQ3J5cHRvIGxpYnJhcnkgcm91dGluZXMKIwpDT05G
SUdfQ1JZUFRPX0xJQl9BRVM9eQojIENPTkZJR19DUllQVE9fTElCX0JMQUtFMlMgaXMgbm90IHNl
dAojIENPTkZJR19DUllQVE9fTElCX0NIQUNIQSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19M
SUJfQ1VSVkUyNTUxOSBpcyBub3Qgc2V0CkNPTkZJR19DUllQVE9fTElCX1BPTFkxMzA1X1JTSVpF
PTkKIyBDT05GSUdfQ1JZUFRPX0xJQl9QT0xZMTMwNSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBU
T19MSUJfQ0hBQ0hBMjBQT0xZMTMwNSBpcyBub3Qgc2V0CkNPTkZJR19DUllQVE9fSFc9eQojIENP
TkZJR19DUllQVE9fREVWX0FUTUVMX0VDQyBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19ERVZf
QVRNRUxfU0hBMjA0QSBpcyBub3Qgc2V0CiMgQ09ORklHX0NSWVBUT19ERVZfQ0NQIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQ1JZUFRPX0RFVl9aWU5RTVBfQUVTIGlzIG5vdCBzZXQKQ09ORklHX0NSWVBU
T19ERVZfVklSVElPPXkKIyBDT05GSUdfQ1JZUFRPX0RFVl9TQUZFWENFTCBpcyBub3Qgc2V0CiMg
Q09ORklHX0NSWVBUT19ERVZfQ0NSRUUgaXMgbm90IHNldAojIENPTkZJR19DUllQVE9fREVWX0hJ
U0lfU0VDIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JZUFRPX0RFVl9BTUxPR0lDX0dYTCBpcyBub3Qg
c2V0CkNPTkZJR19BU1lNTUVUUklDX0tFWV9UWVBFPXkKQ09ORklHX0FTWU1NRVRSSUNfUFVCTElD
X0tFWV9TVUJUWVBFPXkKQ09ORklHX1g1MDlfQ0VSVElGSUNBVEVfUEFSU0VSPXkKIyBDT05GSUdf
UEtDUzhfUFJJVkFURV9LRVlfUEFSU0VSIGlzIG5vdCBzZXQKQ09ORklHX1BLQ1M3X01FU1NBR0Vf
UEFSU0VSPXkKCiMKIyBDZXJ0aWZpY2F0ZXMgZm9yIHNpZ25hdHVyZSBjaGVja2luZwojCkNPTkZJ
R19TWVNURU1fVFJVU1RFRF9LRVlSSU5HPXkKQ09ORklHX1NZU1RFTV9UUlVTVEVEX0tFWVM9IiIK
IyBDT05GSUdfU1lTVEVNX0VYVFJBX0NFUlRJRklDQVRFIGlzIG5vdCBzZXQKIyBDT05GSUdfU0VD
T05EQVJZX1RSVVNURURfS0VZUklORyBpcyBub3Qgc2V0CiMgQ09ORklHX1NZU1RFTV9CTEFDS0xJ
U1RfS0VZUklORyBpcyBub3Qgc2V0CiMgZW5kIG9mIENlcnRpZmljYXRlcyBmb3Igc2lnbmF0dXJl
IGNoZWNraW5nCgojCiMgTGlicmFyeSByb3V0aW5lcwojCkNPTkZJR19MSU5FQVJfUkFOR0VTPXkK
IyBDT05GSUdfUEFDS0lORyBpcyBub3Qgc2V0CkNPTkZJR19CSVRSRVZFUlNFPXkKQ09ORklHX0hB
VkVfQVJDSF9CSVRSRVZFUlNFPXkKQ09ORklHX0dFTkVSSUNfU1RSTkNQWV9GUk9NX1VTRVI9eQpD
T05GSUdfR0VORVJJQ19TVFJOTEVOX1VTRVI9eQpDT05GSUdfR0VORVJJQ19GSU5EX0ZJUlNUX0JJ
VD15CiMgQ09ORklHX0NPUkRJQyBpcyBub3Qgc2V0CiMgQ09ORklHX1BSSU1FX05VTUJFUlMgaXMg
bm90IHNldApDT05GSUdfUkFUSU9OQUw9eQpDT05GSUdfR0VORVJJQ19QQ0lfSU9NQVA9eQpDT05G
SUdfQVJDSF9VU0VfQ01QWENIR19MT0NLUkVGPXkKQ09ORklHX0FSQ0hfSEFTX0ZBU1RfTVVMVElQ
TElFUj15CkNPTkZJR19BUkNIX1VTRV9TWU1fQU5OT1RBVElPTlM9eQojIENPTkZJR19JTkRJUkVD
VF9QSU8gaXMgbm90IHNldAojIENPTkZJR19DUkNfQ0NJVFQgaXMgbm90IHNldApDT05GSUdfQ1JD
MTY9eQojIENPTkZJR19DUkNfVDEwRElGIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1JDX0lUVV9UIGlz
IG5vdCBzZXQKQ09ORklHX0NSQzMyPXkKIyBDT05GSUdfQ1JDMzJfU0VMRlRFU1QgaXMgbm90IHNl
dApDT05GSUdfQ1JDMzJfU0xJQ0VCWTg9eQojIENPTkZJR19DUkMzMl9TTElDRUJZNCBpcyBub3Qg
c2V0CiMgQ09ORklHX0NSQzMyX1NBUldBVEUgaXMgbm90IHNldAojIENPTkZJR19DUkMzMl9CSVQg
aXMgbm90IHNldAojIENPTkZJR19DUkM2NCBpcyBub3Qgc2V0CiMgQ09ORklHX0NSQzQgaXMgbm90
IHNldApDT05GSUdfQ1JDNz15CkNPTkZJR19MSUJDUkMzMkM9eQojIENPTkZJR19DUkM4IGlzIG5v
dCBzZXQKQ09ORklHX1hYSEFTSD15CkNPTkZJR19BVURJVF9BUkNIX0NPTVBBVF9HRU5FUklDPXkK
IyBDT05GSUdfUkFORE9NMzJfU0VMRlRFU1QgaXMgbm90IHNldApDT05GSUdfWkxJQl9JTkZMQVRF
PXkKQ09ORklHX1pMSUJfREVGTEFURT15CkNPTkZJR19MWk9fQ09NUFJFU1M9eQpDT05GSUdfTFpP
X0RFQ09NUFJFU1M9eQpDT05GSUdfTFo0X0RFQ09NUFJFU1M9eQpDT05GSUdfWlNURF9DT01QUkVT
Uz15CkNPTkZJR19aU1REX0RFQ09NUFJFU1M9eQpDT05GSUdfWFpfREVDPXkKQ09ORklHX1haX0RF
Q19YODY9eQpDT05GSUdfWFpfREVDX1BPV0VSUEM9eQpDT05GSUdfWFpfREVDX0lBNjQ9eQpDT05G
SUdfWFpfREVDX0FSTT15CkNPTkZJR19YWl9ERUNfQVJNVEhVTUI9eQpDT05GSUdfWFpfREVDX1NQ
QVJDPXkKQ09ORklHX1haX0RFQ19CQ0o9eQojIENPTkZJR19YWl9ERUNfVEVTVCBpcyBub3Qgc2V0
CkNPTkZJR19ERUNPTVBSRVNTX0daSVA9eQpDT05GSUdfREVDT01QUkVTU19CWklQMj15CkNPTkZJ
R19ERUNPTVBSRVNTX0xaTUE9eQpDT05GSUdfREVDT01QUkVTU19YWj15CkNPTkZJR19ERUNPTVBS
RVNTX0xaTz15CkNPTkZJR19ERUNPTVBSRVNTX0xaND15CkNPTkZJR19ERUNPTVBSRVNTX1pTVEQ9
eQpDT05GSUdfR0VORVJJQ19BTExPQ0FUT1I9eQpDT05GSUdfWEFSUkFZX01VTFRJPXkKQ09ORklH
X0FTU09DSUFUSVZFX0FSUkFZPXkKQ09ORklHX0hBU19JT01FTT15CkNPTkZJR19IQVNfRE1BPXkK
Q09ORklHX0RNQV9PUFM9eQpDT05GSUdfTkVFRF9TR19ETUFfTEVOR1RIPXkKQ09ORklHX05FRURf
RE1BX01BUF9TVEFURT15CkNPTkZJR19BUkNIX0RNQV9BRERSX1RfNjRCSVQ9eQpDT05GSUdfRE1B
X0RFQ0xBUkVfQ09IRVJFTlQ9eQpDT05GSUdfQVJDSF9IQVNfU0VUVVBfRE1BX09QUz15CkNPTkZJ
R19BUkNIX0hBU19URUFSRE9XTl9ETUFfT1BTPXkKQ09ORklHX0FSQ0hfSEFTX1NZTkNfRE1BX0ZP
Ul9ERVZJQ0U9eQpDT05GSUdfQVJDSF9IQVNfU1lOQ19ETUFfRk9SX0NQVT15CkNPTkZJR19BUkNI
X0hBU19ETUFfUFJFUF9DT0hFUkVOVD15CkNPTkZJR19TV0lPVExCPXkKQ09ORklHX0RNQV9OT05D
T0hFUkVOVF9NTUFQPXkKQ09ORklHX0RNQV9DT0hFUkVOVF9QT09MPXkKQ09ORklHX0RNQV9SRU1B
UD15CkNPTkZJR19ETUFfRElSRUNUX1JFTUFQPXkKQ09ORklHX0RNQV9DTUE9eQojIENPTkZJR19E
TUFfUEVSTlVNQV9DTUEgaXMgbm90IHNldAoKIwojIERlZmF1bHQgY29udGlndW91cyBtZW1vcnkg
YXJlYSBzaXplOgojCkNPTkZJR19DTUFfU0laRV9NQllURVM9MjU2CkNPTkZJR19DTUFfU0laRV9T
RUxfTUJZVEVTPXkKIyBDT05GSUdfQ01BX1NJWkVfU0VMX1BFUkNFTlRBR0UgaXMgbm90IHNldAoj
IENPTkZJR19DTUFfU0laRV9TRUxfTUlOIGlzIG5vdCBzZXQKIyBDT05GSUdfQ01BX1NJWkVfU0VM
X01BWCBpcyBub3Qgc2V0CkNPTkZJR19DTUFfQUxJR05NRU5UPTgKIyBDT05GSUdfRE1BX0FQSV9E
RUJVRyBpcyBub3Qgc2V0CiMgQ09ORklHX0RNQV9NQVBfQkVOQ0hNQVJLIGlzIG5vdCBzZXQKQ09O
RklHX1NHTF9BTExPQz15CkNPTkZJR19HTE9CPXkKIyBDT05GSUdfR0xPQl9TRUxGVEVTVCBpcyBu
b3Qgc2V0CkNPTkZJR19DTFpfVEFCPXkKIyBDT05GSUdfSVJRX1BPTEwgaXMgbm90IHNldApDT05G
SUdfTVBJTElCPXkKQ09ORklHX0xJQkZEVD15CkNPTkZJR19PSURfUkVHSVNUUlk9eQpDT05GSUdf
VUNTMl9TVFJJTkc9eQpDT05GSUdfSEFWRV9HRU5FUklDX1ZEU089eQpDT05GSUdfR0VORVJJQ19H
RVRUSU1FT0ZEQVk9eQpDT05GSUdfR0VORVJJQ19WRFNPX1RJTUVfTlM9eQpDT05GSUdfRk9OVF9T
VVBQT1JUPXkKIyBDT05GSUdfRk9OVFMgaXMgbm90IHNldApDT05GSUdfRk9OVF84eDg9eQpDT05G
SUdfRk9OVF84eDE2PXkKQ09ORklHX0FSQ0hfU1RBQ0tXQUxLPXkKIyBlbmQgb2YgTGlicmFyeSBy
b3V0aW5lcwoKQ09ORklHX0dFTkVSSUNfTElCX0RFVk1FTV9JU19BTExPV0VEPXkKCiMKIyBLZXJu
ZWwgaGFja2luZwojCgojCiMgcHJpbnRrIGFuZCBkbWVzZyBvcHRpb25zCiMKQ09ORklHX1BSSU5U
S19USU1FPXkKIyBDT05GSUdfUFJJTlRLX0NBTExFUiBpcyBub3Qgc2V0CiMgQ09ORklHX1NUQUNL
VFJBQ0VfQlVJTERfSUQgaXMgbm90IHNldApDT05GSUdfQ09OU09MRV9MT0dMRVZFTF9ERUZBVUxU
PTcKQ09ORklHX0NPTlNPTEVfTE9HTEVWRUxfUVVJRVQ9NApDT05GSUdfTUVTU0FHRV9MT0dMRVZF
TF9ERUZBVUxUPTQKIyBDT05GSUdfQk9PVF9QUklOVEtfREVMQVkgaXMgbm90IHNldAojIENPTkZJ
R19EWU5BTUlDX0RFQlVHIGlzIG5vdCBzZXQKIyBDT05GSUdfRFlOQU1JQ19ERUJVR19DT1JFIGlz
IG5vdCBzZXQKQ09ORklHX1NZTUJPTElDX0VSUk5BTUU9eQpDT05GSUdfREVCVUdfQlVHVkVSQk9T
RT15CiMgZW5kIG9mIHByaW50ayBhbmQgZG1lc2cgb3B0aW9ucwoKIwojIENvbXBpbGUtdGltZSBj
aGVja3MgYW5kIGNvbXBpbGVyIG9wdGlvbnMKIwpDT05GSUdfREVCVUdfSU5GTz15CiMgQ09ORklH
X0RFQlVHX0lORk9fUkVEVUNFRCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX0lORk9fQ09NUFJF
U1NFRCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX0lORk9fU1BMSVQgaXMgbm90IHNldApDT05G
SUdfREVCVUdfSU5GT19EV0FSRl9UT09MQ0hBSU5fREVGQVVMVD15CiMgQ09ORklHX0RFQlVHX0lO
Rk9fRFdBUkY0IGlzIG5vdCBzZXQKIyBDT05GSUdfREVCVUdfSU5GT19EV0FSRjUgaXMgbm90IHNl
dAojIENPTkZJR19ERUJVR19JTkZPX0JURiBpcyBub3Qgc2V0CiMgQ09ORklHX0dEQl9TQ1JJUFRT
IGlzIG5vdCBzZXQKQ09ORklHX0ZSQU1FX1dBUk49MjA0OAojIENPTkZJR19TVFJJUF9BU01fU1lN
UyBpcyBub3Qgc2V0CiMgQ09ORklHX1JFQURBQkxFX0FTTSBpcyBub3Qgc2V0CiMgQ09ORklHX0hF
QURFUlNfSU5TVEFMTCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX1NFQ1RJT05fTUlTTUFUQ0gg
aXMgbm90IHNldApDT05GSUdfU0VDVElPTl9NSVNNQVRDSF9XQVJOX09OTFk9eQojIENPTkZJR19E
RUJVR19GT1JDRV9GVU5DVElPTl9BTElHTl82NEIgaXMgbm90IHNldApDT05GSUdfQVJDSF9XQU5U
X0ZSQU1FX1BPSU5URVJTPXkKQ09ORklHX0ZSQU1FX1BPSU5URVI9eQojIENPTkZJR19WTUxJTlVY
X01BUCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX0ZPUkNFX1dFQUtfUEVSX0NQVSBpcyBub3Qg
c2V0CiMgZW5kIG9mIENvbXBpbGUtdGltZSBjaGVja3MgYW5kIGNvbXBpbGVyIG9wdGlvbnMKCiMK
IyBHZW5lcmljIEtlcm5lbCBEZWJ1Z2dpbmcgSW5zdHJ1bWVudHMKIwpDT05GSUdfTUFHSUNfU1lT
UlE9eQpDT05GSUdfTUFHSUNfU1lTUlFfREVGQVVMVF9FTkFCTEU9MHgxCkNPTkZJR19NQUdJQ19T
WVNSUV9TRVJJQUw9eQpDT05GSUdfTUFHSUNfU1lTUlFfU0VSSUFMX1NFUVVFTkNFPSIiCkNPTkZJ
R19ERUJVR19GUz15CkNPTkZJR19ERUJVR19GU19BTExPV19BTEw9eQojIENPTkZJR19ERUJVR19G
U19ESVNBTExPV19NT1VOVCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX0ZTX0FMTE9XX05PTkUg
aXMgbm90IHNldApDT05GSUdfSEFWRV9BUkNIX0tHREI9eQojIENPTkZJR19LR0RCIGlzIG5vdCBz
ZXQKQ09ORklHX0FSQ0hfSEFTX1VCU0FOX1NBTklUSVpFX0FMTD15CiMgQ09ORklHX1VCU0FOIGlz
IG5vdCBzZXQKIyBlbmQgb2YgR2VuZXJpYyBLZXJuZWwgRGVidWdnaW5nIEluc3RydW1lbnRzCgpD
T05GSUdfREVCVUdfS0VSTkVMPXkKQ09ORklHX0RFQlVHX01JU0M9eQoKIwojIE1lbW9yeSBEZWJ1
Z2dpbmcKIwojIENPTkZJR19QQUdFX0VYVEVOU0lPTiBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVH
X1BBR0VBTExPQyBpcyBub3Qgc2V0CiMgQ09ORklHX1BBR0VfT1dORVIgaXMgbm90IHNldAojIENP
TkZJR19QQUdFX1BPSVNPTklORyBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX1JPREFUQV9URVNU
IGlzIG5vdCBzZXQKQ09ORklHX0FSQ0hfSEFTX0RFQlVHX1dYPXkKIyBDT05GSUdfREVCVUdfV1gg
aXMgbm90IHNldApDT05GSUdfR0VORVJJQ19QVERVTVA9eQojIENPTkZJR19QVERVTVBfREVCVUdG
UyBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX09CSkVDVFMgaXMgbm90IHNldAojIENPTkZJR19E
RUJVR19TTEFCIGlzIG5vdCBzZXQKQ09ORklHX0hBVkVfREVCVUdfS01FTUxFQUs9eQojIENPTkZJ
R19ERUJVR19LTUVNTEVBSyBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX1NUQUNLX1VTQUdFIGlz
IG5vdCBzZXQKIyBDT05GSUdfU0NIRURfU1RBQ0tfRU5EX0NIRUNLIGlzIG5vdCBzZXQKQ09ORklH
X0FSQ0hfSEFTX0RFQlVHX1ZNX1BHVEFCTEU9eQojIENPTkZJR19ERUJVR19WTSBpcyBub3Qgc2V0
CiMgQ09ORklHX0RFQlVHX1ZNX1BHVEFCTEUgaXMgbm90IHNldApDT05GSUdfQVJDSF9IQVNfREVC
VUdfVklSVFVBTD15CiMgQ09ORklHX0RFQlVHX1ZJUlRVQUwgaXMgbm90IHNldAojIENPTkZJR19E
RUJVR19NRU1PUllfSU5JVCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX1BFUl9DUFVfTUFQUyBp
cyBub3Qgc2V0CkNPTkZJR19IQVZFX0FSQ0hfS0FTQU49eQpDT05GSUdfSEFWRV9BUkNIX0tBU0FO
X1NXX1RBR1M9eQpDT05GSUdfSEFWRV9BUkNIX0tBU0FOX0hXX1RBR1M9eQpDT05GSUdfSEFWRV9B
UkNIX0tBU0FOX1ZNQUxMT0M9eQpDT05GSUdfQ0NfSEFTX0tBU0FOX0dFTkVSSUM9eQpDT05GSUdf
Q0NfSEFTX1dPUktJTkdfTk9TQU5JVElaRV9BRERSRVNTPXkKIyBDT05GSUdfS0FTQU4gaXMgbm90
IHNldApDT05GSUdfSEFWRV9BUkNIX0tGRU5DRT15CiMgQ09ORklHX0tGRU5DRSBpcyBub3Qgc2V0
CiMgZW5kIG9mIE1lbW9yeSBEZWJ1Z2dpbmcKCiMgQ09ORklHX0RFQlVHX1NISVJRIGlzIG5vdCBz
ZXQKCiMKIyBEZWJ1ZyBPb3BzLCBMb2NrdXBzIGFuZCBIYW5ncwojCiMgQ09ORklHX1BBTklDX09O
X09PUFMgaXMgbm90IHNldApDT05GSUdfUEFOSUNfT05fT09QU19WQUxVRT0wCkNPTkZJR19QQU5J
Q19USU1FT1VUPTAKIyBDT05GSUdfU09GVExPQ0tVUF9ERVRFQ1RPUiBpcyBub3Qgc2V0CiMgQ09O
RklHX0RFVEVDVF9IVU5HX1RBU0sgaXMgbm90IHNldAojIENPTkZJR19XUV9XQVRDSERPRyBpcyBu
b3Qgc2V0CiMgZW5kIG9mIERlYnVnIE9vcHMsIExvY2t1cHMgYW5kIEhhbmdzCgojCiMgU2NoZWR1
bGVyIERlYnVnZ2luZwojCiMgQ09ORklHX1NDSEVEX0RFQlVHIGlzIG5vdCBzZXQKIyBDT05GSUdf
U0NIRURTVEFUUyBpcyBub3Qgc2V0CiMgZW5kIG9mIFNjaGVkdWxlciBEZWJ1Z2dpbmcKCiMgQ09O
RklHX0RFQlVHX1RJTUVLRUVQSU5HIGlzIG5vdCBzZXQKCiMKIyBMb2NrIERlYnVnZ2luZyAoc3Bp
bmxvY2tzLCBtdXRleGVzLCBldGMuLi4pCiMKQ09ORklHX0xPQ0tfREVCVUdHSU5HX1NVUFBPUlQ9
eQojIENPTkZJR19QUk9WRV9MT0NLSU5HIGlzIG5vdCBzZXQKIyBDT05GSUdfTE9DS19TVEFUIGlz
IG5vdCBzZXQKIyBDT05GSUdfREVCVUdfUlRfTVVURVhFUyBpcyBub3Qgc2V0CiMgQ09ORklHX0RF
QlVHX1NQSU5MT0NLIGlzIG5vdCBzZXQKIyBDT05GSUdfREVCVUdfTVVURVhFUyBpcyBub3Qgc2V0
CiMgQ09ORklHX0RFQlVHX1dXX01VVEVYX1NMT1dQQVRIIGlzIG5vdCBzZXQKIyBDT05GSUdfREVC
VUdfUldTRU1TIGlzIG5vdCBzZXQKIyBDT05GSUdfREVCVUdfTE9DS19BTExPQyBpcyBub3Qgc2V0
CiMgQ09ORklHX0RFQlVHX0FUT01JQ19TTEVFUCBpcyBub3Qgc2V0CiMgQ09ORklHX0RFQlVHX0xP
Q0tJTkdfQVBJX1NFTEZURVNUUyBpcyBub3Qgc2V0CiMgQ09ORklHX0xPQ0tfVE9SVFVSRV9URVNU
IGlzIG5vdCBzZXQKIyBDT05GSUdfV1dfTVVURVhfU0VMRlRFU1QgaXMgbm90IHNldAojIENPTkZJ
R19TQ0ZfVE9SVFVSRV9URVNUIGlzIG5vdCBzZXQKIyBDT05GSUdfQ1NEX0xPQ0tfV0FJVF9ERUJV
RyBpcyBub3Qgc2V0CiMgZW5kIG9mIExvY2sgRGVidWdnaW5nIChzcGlubG9ja3MsIG11dGV4ZXMs
IGV0Yy4uLikKCiMgQ09ORklHX0RFQlVHX0lSUUZMQUdTIGlzIG5vdCBzZXQKIyBDT05GSUdfU1RB
Q0tUUkFDRSBpcyBub3Qgc2V0CiMgQ09ORklHX1dBUk5fQUxMX1VOU0VFREVEX1JBTkRPTSBpcyBu
b3Qgc2V0CiMgQ09ORklHX0RFQlVHX0tPQkpFQ1QgaXMgbm90IHNldAoKIwojIERlYnVnIGtlcm5l
bCBkYXRhIHN0cnVjdHVyZXMKIwojIENPTkZJR19ERUJVR19MSVNUIGlzIG5vdCBzZXQKIyBDT05G
SUdfREVCVUdfUExJU1QgaXMgbm90IHNldAojIENPTkZJR19ERUJVR19TRyBpcyBub3Qgc2V0CiMg
Q09ORklHX0RFQlVHX05PVElGSUVSUyBpcyBub3Qgc2V0CiMgQ09ORklHX0JVR19PTl9EQVRBX0NP
UlJVUFRJT04gaXMgbm90IHNldAojIGVuZCBvZiBEZWJ1ZyBrZXJuZWwgZGF0YSBzdHJ1Y3R1cmVz
CgojIENPTkZJR19ERUJVR19DUkVERU5USUFMUyBpcyBub3Qgc2V0CgojCiMgUkNVIERlYnVnZ2lu
ZwojCiMgQ09ORklHX1JDVV9TQ0FMRV9URVNUIGlzIG5vdCBzZXQKIyBDT05GSUdfUkNVX1RPUlRV
UkVfVEVTVCBpcyBub3Qgc2V0CiMgQ09ORklHX1JDVV9SRUZfU0NBTEVfVEVTVCBpcyBub3Qgc2V0
CkNPTkZJR19SQ1VfQ1BVX1NUQUxMX1RJTUVPVVQ9MjEKQ09ORklHX1JDVV9UUkFDRT15CiMgQ09O
RklHX1JDVV9FUVNfREVCVUcgaXMgbm90IHNldAojIGVuZCBvZiBSQ1UgRGVidWdnaW5nCgojIENP
TkZJR19ERUJVR19XUV9GT1JDRV9SUl9DUFUgaXMgbm90IHNldAojIENPTkZJR19DUFVfSE9UUExV
R19TVEFURV9DT05UUk9MIGlzIG5vdCBzZXQKIyBDT05GSUdfTEFURU5DWVRPUCBpcyBub3Qgc2V0
CkNPTkZJR19IQVZFX0ZVTkNUSU9OX1RSQUNFUj15CkNPTkZJR19IQVZFX0ZVTkNUSU9OX0dSQVBI
X1RSQUNFUj15CkNPTkZJR19IQVZFX0RZTkFNSUNfRlRSQUNFPXkKQ09ORklHX0hBVkVfRFlOQU1J
Q19GVFJBQ0VfV0lUSF9SRUdTPXkKQ09ORklHX0hBVkVfRlRSQUNFX01DT1VOVF9SRUNPUkQ9eQpD
T05GSUdfSEFWRV9TWVNDQUxMX1RSQUNFUE9JTlRTPXkKQ09ORklHX0hBVkVfQ19SRUNPUkRNQ09V
TlQ9eQpDT05GSUdfVFJBQ0VfQ0xPQ0s9eQpDT05GSUdfVFJBQ0lOR19TVVBQT1JUPXkKIyBDT05G
SUdfRlRSQUNFIGlzIG5vdCBzZXQKIyBDT05GSUdfU0FNUExFUyBpcyBub3Qgc2V0CiMgQ09ORklH
X1NUUklDVF9ERVZNRU0gaXMgbm90IHNldAoKIwojIGFybTY0IERlYnVnZ2luZwojCiMgQ09ORklH
X1BJRF9JTl9DT05URVhUSURSIGlzIG5vdCBzZXQKIyBDT05GSUdfREVCVUdfRUZJIGlzIG5vdCBz
ZXQKIyBDT05GSUdfQ09SRVNJR0hUIGlzIG5vdCBzZXQKIyBlbmQgb2YgYXJtNjQgRGVidWdnaW5n
CgojCiMgS2VybmVsIFRlc3RpbmcgYW5kIENvdmVyYWdlCiMKIyBDT05GSUdfS1VOSVQgaXMgbm90
IHNldAojIENPTkZJR19OT1RJRklFUl9FUlJPUl9JTkpFQ1RJT04gaXMgbm90IHNldAojIENPTkZJ
R19GQVVMVF9JTkpFQ1RJT04gaXMgbm90IHNldApDT05GSUdfQVJDSF9IQVNfS0NPVj15CkNPTkZJ
R19DQ19IQVNfU0FOQ09WX1RSQUNFX1BDPXkKIyBDT05GSUdfS0NPViBpcyBub3Qgc2V0CkNPTkZJ
R19SVU5USU1FX1RFU1RJTkdfTUVOVT15CiMgQ09ORklHX0xLRFRNIGlzIG5vdCBzZXQKIyBDT05G
SUdfVEVTVF9NSU5fSEVBUCBpcyBub3Qgc2V0CiMgQ09ORklHX1RFU1RfU09SVCBpcyBub3Qgc2V0
CiMgQ09ORklHX1RFU1RfRElWNjQgaXMgbm90IHNldAojIENPTkZJR19CQUNLVFJBQ0VfU0VMRl9U
RVNUIGlzIG5vdCBzZXQKIyBDT05GSUdfUkJUUkVFX1RFU1QgaXMgbm90IHNldAojIENPTkZJR19S
RUVEX1NPTE9NT05fVEVTVCBpcyBub3Qgc2V0CiMgQ09ORklHX0lOVEVSVkFMX1RSRUVfVEVTVCBp
cyBub3Qgc2V0CiMgQ09ORklHX0FUT01JQzY0X1NFTEZURVNUIGlzIG5vdCBzZXQKIyBDT05GSUdf
VEVTVF9IRVhEVU1QIGlzIG5vdCBzZXQKIyBDT05GSUdfU1RSSU5HX1NFTEZURVNUIGlzIG5vdCBz
ZXQKIyBDT05GSUdfVEVTVF9TVFJJTkdfSEVMUEVSUyBpcyBub3Qgc2V0CiMgQ09ORklHX1RFU1Rf
U1RSU0NQWSBpcyBub3Qgc2V0CiMgQ09ORklHX1RFU1RfS1NUUlRPWCBpcyBub3Qgc2V0CiMgQ09O
RklHX1RFU1RfUFJJTlRGIGlzIG5vdCBzZXQKIyBDT05GSUdfVEVTVF9TQ0FORiBpcyBub3Qgc2V0
CiMgQ09ORklHX1RFU1RfQklUTUFQIGlzIG5vdCBzZXQKIyBDT05GSUdfVEVTVF9VVUlEIGlzIG5v
dCBzZXQKIyBDT05GSUdfVEVTVF9YQVJSQVkgaXMgbm90IHNldAojIENPTkZJR19URVNUX09WRVJG
TE9XIGlzIG5vdCBzZXQKIyBDT05GSUdfVEVTVF9SSEFTSFRBQkxFIGlzIG5vdCBzZXQKIyBDT05G
SUdfVEVTVF9IQVNIIGlzIG5vdCBzZXQKIyBDT05GSUdfVEVTVF9JREEgaXMgbm90IHNldAojIENP
TkZJR19GSU5EX0JJVF9CRU5DSE1BUksgaXMgbm90IHNldAojIENPTkZJR19URVNUX0ZJUk1XQVJF
IGlzIG5vdCBzZXQKIyBDT05GSUdfVEVTVF9TWVNDVEwgaXMgbm90IHNldAojIENPTkZJR19URVNU
X1VERUxBWSBpcyBub3Qgc2V0CiMgQ09ORklHX1RFU1RfTUVNQ0FUX1AgaXMgbm90IHNldAojIENP
TkZJR19URVNUX1NUQUNLSU5JVCBpcyBub3Qgc2V0CiMgQ09ORklHX1RFU1RfTUVNSU5JVCBpcyBu
b3Qgc2V0CiMgQ09ORklHX1RFU1RfRlJFRV9QQUdFUyBpcyBub3Qgc2V0CkNPTkZJR19BUkNIX1VT
RV9NRU1URVNUPXkKIyBDT05GSUdfTUVNVEVTVCBpcyBub3Qgc2V0CiMgZW5kIG9mIEtlcm5lbCBU
ZXN0aW5nIGFuZCBDb3ZlcmFnZQojIGVuZCBvZiBLZXJuZWwgaGFja2luZwo=

--------------6DD34436CA52906803239DD3--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:21:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:21:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181442.328545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNqxp-0005oR-Q9; Wed, 08 Sep 2021 06:21:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181442.328545; Wed, 08 Sep 2021 06:21:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNqxp-0005oK-My; Wed, 08 Sep 2021 06:21:41 +0000
Received: by outflank-mailman (input) for mailman id 181442;
 Wed, 08 Sep 2021 06:21:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Bx+7=N6=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mNqxn-0005oC-Qy
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:21:39 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 062b4a3d-106d-11ec-b144-12813bfff9fa;
 Wed, 08 Sep 2021 06:21:38 +0000 (UTC)
Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com
 [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-327-8vDZ1ZkhO9mA5aQ2DxWJLw-1; Wed, 08 Sep 2021 02:21:36 -0400
Received: by mail-wm1-f71.google.com with SMTP id
 j33-20020a05600c48a100b002e879427915so526154wmp.5
 for <xen-devel@lists.xenproject.org>; Tue, 07 Sep 2021 23:21:35 -0700 (PDT)
Received: from [192.168.3.132] (p5b0c6125.dip0.t-ipconnect.de. [91.12.97.37])
 by smtp.gmail.com with ESMTPSA id
 l21sm1014143wmh.31.2021.09.07.23.21.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 07 Sep 2021 23:21:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 062b4a3d-106d-11ec-b144-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631082098;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=z34UVAiI8Q78/349kHm5g+91ow7Iz9fZ37lSWz6rrwY=;
	b=axfZ8DDCs3RCvOLyr4CFX8PtrudeMAvhZE25+JxlU9p32pwRCGAx7t6+yheogb1+9jP+6h
	Ky4uP9Mbx/16mr0pMSILJrLTDimd9Tajm001koLUdZOK803RAL735x5G72KmC0KpSY3Tx0
	jm7L855GCNCra006HzyAU3azuLWh5z0=
X-MC-Unique: 8vDZ1ZkhO9mA5aQ2DxWJLw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:to:cc:references:from:organization:subject
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=z34UVAiI8Q78/349kHm5g+91ow7Iz9fZ37lSWz6rrwY=;
        b=GbJiajyEh5oi9ZmT43yMvCo/ppfDABdzosNron7RU1Tc53Km5oA40K4Vw9esey/ktC
         9tHoKG5W/dO0Dm2ud8+rBzdJ26czgE63SZS6C/AvxtZxCSbxncsgvNDWOIvH3Mhoq1Df
         pHmxefoLFBH0MBFT3cxg0nCjHKCgHeCUKYvdngmIDU1b3jpwkab3RXdRhXptlT76e0do
         t1YLfp4WEotI4re6JBHjzJL/C18qF4i1ubXDfTMuaOUDuO6tBqed++8zjQbtxcp+b57C
         8SNjjc89UhruNwrF+D83/9p82XFsEgFYGtVIt6b+exIlhrzF2FPCn1xifmByRnw9gMrn
         7lFw==
X-Gm-Message-State: AOAM532xz1f7JgxljUJkQ7p4MFZOsqATFxa4Gt7xB7UpeX67aQe6kiCT
	nNXV2beHIy6vzsZ4H3KIXlr8M22J7v+550k8sBkPEJ+ku6N8mBjQGyJYximl0HErEupQLVlYPpm
	rPD2GyAoiattecxZmuHdv7i66BI4GDUtCZCqYg3tEawgb17apZuvGxRBeFSM+0bEvUtizjpnQGA
	0=
X-Received: by 2002:adf:9bdb:: with SMTP id e27mr1991910wrc.162.1631082094839;
        Tue, 07 Sep 2021 23:21:34 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJwVePYl3oJzEgmfpZnrDZETUp0uOCSKVg5L73nfFg5SJFFaAVj5u3fATe4qqvZ9JKp+koI50g==
X-Received: by 2002:adf:9bdb:: with SMTP id e27mr1991865wrc.162.1631082094550;
        Tue, 07 Sep 2021 23:21:34 -0700 (PDT)
To: Sean Anderson <sean.anderson@seco.com>
Cc: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com,
 dave.hansen@intel.com, haiyangz@microsoft.com, kys@microsoft.com,
 linux-acpi@vger.kernel.org, linux-hyperv@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mm@kvack.org,
 mgorman@techsingularity.net, mhocko@kernel.org, mhocko@suse.com,
 osalvador@suse.de, pankaj.gupta.linux@gmail.com,
 richard.weiyang@linux.alibaba.com, rppt@kernel.org, sthemmin@microsoft.com,
 vbabka@suse.cz, wei.liu@kernel.org, willy@infradead.org,
 xen-devel@lists.xenproject.org
References: <20201005121534.15649-5-david@redhat.com>
 <a52dacbe-5649-7245-866f-ceaba44975b5@seco.com>
From: David Hildenbrand <david@redhat.com>
Organization: Red Hat
Subject: Re: [PATCH v2 4/5] mm/page_alloc: place pages to tail in
 __free_pages_core()
Message-ID: <528e8d9c-b148-30ec-d8cc-3dd072eaa7f2@redhat.com>
Date: Wed, 8 Sep 2021 08:21:32 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <a52dacbe-5649-7245-866f-ceaba44975b5@seco.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 08.09.21 00:40, Sean Anderson wrote:
> Hi David,
> 
> This patch breaks booting on my custom Xilinx ZynqMP board. Booting
> fails just after/during GIC initialization:
> 
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
> [    0.000000] Linux version 5.14.0 (sean@plantagenet) (aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #251 SMP Tue Sep 7 18:11:50 EDT 2021
> [    0.000000] Machine model: xlnx,zynqmp
> [    0.000000] earlycon: cdns0 at MMIO 0x00000000ff010000 (options '115200n8')
> [    0.000000] printk: bootconsole [cdns0] enabled
> [    0.000000] efi: UEFI not found.
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
> [    0.000000]   Normal   [mem 0x0000000100000000-0x000000087fffffff]
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000000000000-0x000000007fefffff]
> [    0.000000]   node   0: [mem 0x0000000800000000-0x000000087fffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000087fffffff]
> [    0.000000] On node 0, zone Normal: 256 pages in unavailable ranges
> [    0.000000] cma: Reserved 1000 MiB at 0x0000000041400000
> [    0.000000] psci: probing for conduit method from DT.
> [    0.000000] psci: PSCIv1.1 detected in firmware.
> [    0.000000] psci: Using standard PSCI v0.2 function IDs
> [    0.000000] psci: MIGRATE_INFO_TYPE not supported.
> [    0.000000] psci: SMC Calling Convention v1.1
> [    0.000000] percpu: Embedded 19 pages/cpu s46752 r0 d31072 u77824
> [    0.000000] Detected VIPT I-cache on CPU0
> [    0.000000] CPU features: detected: ARM erratum 845719
> [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1033987
> [    0.000000] Kernel command line: earlycon clk_ignore_unused root=/dev/mmcblk0p2 rootwait rw cma=1000M
> [    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] software IO TLB: mapped [mem 0x000000003d400000-0x0000000041400000] (64MB)
> [    0.000000] Memory: 3023384K/4193280K available (4288K kernel code, 514K rwdata, 1200K rodata, 896K init, 187K bss, 145896K reserved, 1024000K cma-reserved)
> [    0.000000] rcu: Hierarchical RCU implementation.
> [    0.000000] rcu: 	RCU event tracing is enabled.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
> [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
> [    0.000000] GIC: Adjusting CPU interface base to 0x00000000f902f000
> [    0.000000] Root IRQ handler: gic_handle_irq
> [    0.000000] GIC: Using split EOI/Deactivate mode
> 
> and I bisected it to this patch. Applying the following patch (for 5.14)
> fixes booting again:

Hi Sean,

unfortunately that patch most likely (with 99.9999% confidence) revealed 
another latent BUG in your setup.

Some memory that shouldn't be handed to the buddy as free memory is 
getting now allocated earlier than later, resulting in that issue.


I had all different kinds of reports, but they were mostly

a) Firmware bugs that result in uncached memory getting exposed to the 
buddy, resulting in severe performance degradation such that the system 
will no longer boot. [3]

I wrote kstream [1] to be run under the old kernel, to identify these.

b) BUGs that result in unsuitable memory getting exposed to either the 
buddy or devices, resulting in errors during device initialization. [6]

c) Use after free BUGs.

Exposing memory, such as used for ACPI tables, to the buddy as free 
memory although it's still in use. [4]

d) Hypervisor BUGs

The last report (heavy performance degradation) was due to a BUG in 
dpdk. [2]


What the exact symptoms you're experiencing? Really slow boot/stall? 
Then it could be a) and kstream might help.


[1] https://github.com/davidhildenbrand/kstream
[2] 
https://lore.kernel.org/dpdk-dev/20210827161231.579968-1-eperezma@redhat.com/T/#u
[3] 
https://lore.kernel.org/r/MW3PR12MB4537C3C6EFD9CA3A4B32084DF36B9@MW3PR12MB4537.namprd12.prod.outlook.com
[4] https://lkml.kernel.org/r/4650320.31r3eYUQgx@kreacher
[5] https://lkml.kernel.org/r/87361onphy.fsf_-_@codeaurora.org
[6] 
https://lore.kernel.org/r/20201213225517.3838501-1-linus.walleij@linaro.org


-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:49:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:49:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181449.328555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrOr-0000Cr-Sg; Wed, 08 Sep 2021 06:49:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181449.328555; Wed, 08 Sep 2021 06:49:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrOr-0000Ck-Ph; Wed, 08 Sep 2021 06:49:37 +0000
Received: by outflank-mailman (input) for mailman id 181449;
 Wed, 08 Sep 2021 06:49:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrOp-0000Ce-Dg
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:49:36 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 71ccbeff-fed4-4b6c-9170-620336930e35;
 Wed, 08 Sep 2021 06:49:32 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrNu-008ZoP-Gv; Wed, 08 Sep 2021 06:48:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71ccbeff-fed4-4b6c-9170-620336930e35
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=OrzGM9PLHfbyfeiU5N8A74VHnUlyeQDz1745rBQSmpE=; b=JvLbZFDB8IlVYMmqnbPjeH0zpT
	RS6TMNs3DqCPi4wnv+3RN+BAWH5wMMsH5OIbUJGjC9c+FFCSx0qVUnY9l1pDwbtkGF0v1rdZSeqkv
	8XX7xKR06ZiLWC8KJX9Wc5cBfed6lBQRH62JfWIeV4NJjQStpPIefHWmFJFFjudwengFWX7O9+UKm
	JnPsIv35qqOOxV/brv/0P5TTjFwkO3aGrqtXxjiSYg3zvBlHIJhqDusKynHPkF+9LwnkgJSAZfb8p
	N72r4r40Aj7RwC/fkxH0CgbUafSPL2d56ee5y/Rj+rPHP+4/dH3Ce027JiYI5/vdK/Awy8PLopLbN
	sv0vmElA==;
Date: Wed, 8 Sep 2021 07:48:38 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 01/12] swiotlb-xen: avoid double free
Message-ID: <YThcxvBhQqOVX2X4@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <ce9c2adb-8a52-6293-982a-0d6ece943ac6@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ce9c2adb-8a52-6293-982a-0d6ece943ac6@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:04:25PM +0200, Jan Beulich wrote:
> Of the two paths leading to the "error" label in xen_swiotlb_init() one
> didn't allocate anything, while the other did already free what was
> allocated.
> 
> Fixes: b82776005369 ("xen/swiotlb: Use the swiotlb_late_init_with_tbl to init Xen-SWIOTLB late when PV PCI is used")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Cc: stable@vger.kernel.org

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:50:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:50:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181455.328567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrPt-0001UL-81; Wed, 08 Sep 2021 06:50:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181455.328567; Wed, 08 Sep 2021 06:50:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrPt-0001UE-4m; Wed, 08 Sep 2021 06:50:41 +0000
Received: by outflank-mailman (input) for mailman id 181455;
 Wed, 08 Sep 2021 06:50:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uOOR=N6=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNrPr-0001U4-Kt
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:50:39 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.45]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 124eb228-1071-11ec-b147-12813bfff9fa;
 Wed, 08 Sep 2021 06:50:37 +0000 (UTC)
Received: from AM5PR04CA0018.eurprd04.prod.outlook.com (2603:10a6:206:1::31)
 by VI1PR08MB4414.eurprd08.prod.outlook.com (2603:10a6:803:f3::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 06:50:33 +0000
Received: from VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:1:cafe::32) by AM5PR04CA0018.outlook.office365.com
 (2603:10a6:206:1::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 06:50:33 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT036.mail.protection.outlook.com (10.152.19.204) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 06:50:32 +0000
Received: ("Tessian outbound cc1d52552731:v105");
 Wed, 08 Sep 2021 06:50:32 +0000
Received: from a7d1b221e695.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D3104BFB-56FC-497E-9F9F-9471EC98E033.1; 
 Wed, 08 Sep 2021 06:50:26 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a7d1b221e695.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 06:50:26 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6621.eurprd08.prod.outlook.com (2603:10a6:102:dc::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 06:50:24 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 06:50:24 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P123CA0009.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:a6::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 06:50:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 124eb228-1071-11ec-b147-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pXFdvcbYFekiJeuPsAux0l5olVsrjIqfTBNNqjHOfbo=;
 b=fix5C2bTXUIgPc5p3I+HPsrA4GOwFPO+ZNZiaiWz0Ib/YCb/OHXEGOUCBDdKCQktsbB3CR5lxo74n4cCXtYr9Jrxrp23r2ZDjZ9dk3YgNme1ch4jiQH0GAb6GYJxrEj3Tq+MQ8VsliDEvWBBompqmxqXtDy/SWwGLd8ohVneOwc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0497a7ae381cb32d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YOLtCutZwnefg5y1wFxjnwKeCZxqtPZRX8/QF5fe/5Kd1gsnsHdkckU1icNMPO+N0S5D4RiY5nGukObg07vhpJXWDORF70Z8Lels7osGzYCSvr6wDlGueiuFtIEA8Y5OaeIUyFIOwOYGmuzljd5SpixZpFlGeYbRkcqsUHj+hg7R6ypg3lTxKNS9puFlqvFfUHV6tna28m1qnJxfo15MI+OfI3Z9+xXuX60c+rndtpQP+SBJjgsXTeXvVDpevJM6gL5IApQmbInUDJNZlim01t5gY+JggDyLdh6lGDU23hbLoft5nPudxq+BhcUQB50OUj6jfO2aDgEHoQIx8vMYwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pXFdvcbYFekiJeuPsAux0l5olVsrjIqfTBNNqjHOfbo=;
 b=ZY6fRNACWAPRxQX4wQ13qct5b6bbZ4fnbRuRzEEkC0V6NIAk3Epvet61/t79CkE9S924ftv92xubPfDKe1OqKqnUdbTR56wQWAZ5d1Vg5wpgcNuHaJxicpiPKbUb8CZ0ftmXhHXY/gbYMn5TCzKdhagZE364kc5emrc446QAuTRjbcVWcME4I8HCsckje5aQB6VBF/xmu09CpZYwLcvd5v2NgrSNBPxEJncdqj1PXVmg3bnNvpQgBYh5ewUb0b1GD+vhR5xwmcvTNg97/F9FTyPgQveKHfB7fEsDkyQTA5SVn6GwzovUzPnF2NBLsiG3OIvyr7/XBOijmUSRUXEMxg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pXFdvcbYFekiJeuPsAux0l5olVsrjIqfTBNNqjHOfbo=;
 b=fix5C2bTXUIgPc5p3I+HPsrA4GOwFPO+ZNZiaiWz0Ib/YCb/OHXEGOUCBDdKCQktsbB3CR5lxo74n4cCXtYr9Jrxrp23r2ZDjZ9dk3YgNme1ch4jiQH0GAb6GYJxrEj3Tq+MQ8VsliDEvWBBompqmxqXtDy/SWwGLd8ohVneOwc=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109071750262.14059@sstabellini-ThinkPad-T480s>
Date: Wed, 8 Sep 2021 07:50:13 +0100
Cc: Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <1D635A64-25A8-44A9-924D-2E609D0D0BEB@arm.com>
References: <20210907065228.21794-1-luca.fancellu@arm.com>
 <4bab7902-0268-5705-5462-fcd7571d7492@xen.org>
 <7BD59287-75ED-4D4F-B892-F6B04583A986@arm.com>
 <1efbd9a8-232b-b267-fa8e-7f5f1af7cfed@xen.org>
 <alpine.DEB.2.21.2109071750262.14059@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P123CA0009.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:a6::21) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0480f88d-9590-45f8-d8df-08d97294f46f
X-MS-TrafficTypeDiagnostic: PAXPR08MB6621:|VI1PR08MB4414:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB4414B6124AC5165674E36306E4D49@VI1PR08MB4414.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 VVTiIj9Ld3eFBUHdd80sdaijWJLmo6TWX67UpipDodKTDM4s1x6aXMINR56tk0sCMv8PjaygFWAkmXJrpfJna4Dewms2aE3C9XX8rSrG2gyFffwrP19wmtxPvmXRQ98jsq6oMhNKwX9aognpFGMWhkW0KS/fihZHScEYBwaeBPRv31KB4j7t/rogAu0HOXzdyArZvh0M9GaZjaCIxBQEF0gcK0j/0x3n81VEGi29OBM5R7Q8qUC7Qzi7aqHpqLOOgALzxNTFYvvleg4n2mCfUuC2PLyxA05HYNkEgZQLAoLBYw0MhB6KcaKnhK6j/5UY84McgP662PFBmM3g32KfHt8TnTDRz/Du0hEzptbKzEkHY+rTARTx2Kdn1yfgN4r+u+4BxRsFPyKpDZNkB52lCwn+FE4OPS58MMROFcuo2XyTiFcESa2ho6M2yWOVJ7/3ARRz9bm20a6PlcnuOI7gdXElkLqVQh6DsYWwahpMbw5QEwRB4iom7aUKupUbCp5sZNpzG4lQj385ThwoverUsHOmGtFC2wA8c+aRVj7oPBCYwhFVhxV9AcXThQ3+egGtjJP7zasl5GYXELGIXxd4p4PaeyrPNMO3V+VTXcqmAaJRBgmByk+sfg2u+5EFdHYtF6DDOq9+huvMRmbcRaCW/PcBB2rPBmgWGhmmMuPECNOIihmU2W/dQSnT0B9/veh09QmW21fMDWu9Kh9+CN39HGeyeKOewdWTb8Fy9nqiXGcySZdQTMcJH2aM0dJaQDfFCl85kQ+sjjvLAM8E88xiRA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(39860400002)(396003)(366004)(136003)(6486002)(66556008)(66946007)(478600001)(66476007)(6506007)(8676002)(8936002)(44832011)(5660300002)(26005)(186003)(33656002)(2906002)(53546011)(4326008)(83380400001)(52116002)(6512007)(6916009)(316002)(36756003)(86362001)(956004)(2616005)(54906003)(6666004)(38100700002)(38350700002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6621
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	49838ec5-12cf-41f5-d53b-08d97294ef5a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yxGs9tVKGSi6SH3jcncPYjrswgX0IKJoLV2eZHfEx5soAvxJr7rAGVOCFvocNLzDdYsfdjpV3Fw7KHTw4uc7sACHEI2Ew7asC/NxtRqWVzrnMjlKb/6NX+m7KA+k0BYh8xygMJacbEoCAufrC7oEJPcfK/Y8QSVTzjoH0t/Kq0Ff2jzPn5H893R4bTgteb1IZfS20r6U1L4tWa7K/jSGtjiiPwHbvecS7t4/+CV8FZtAoPqFBWoacpKgbnSUtZ7sn6KlFGAQQLQxJXnEj3E1J502EkwMfEq7DKrKLVM64zhQuQQbm5yCCoXthKJqmzBLoBb6bvpJoVWzzvV0muwKAohjnNJ1+eHpCMGBVXaKqlKR/5/9iHIqFowrTR8AUCCDr2ZZgOsiMvdlOcy3FODOhaTM4nVFopZhcbOwVKNqkQuZTQhRoOQRY18ZiLWjRHsedolFOBOkNVhslvNFH0fLOEgFTeTQ7KmpaZcgzgWHRhG8WEAdlI1ETu0f7hLmbB4xtO4vIGKZL93uZYjGRXWdBvC6Vtn03TRyrthV3y+MSmejk0zdJAd912/TF+JGk2iwBVDsnrZ7ghzJy5XJdt27ztNwXjN2TwlQFgz5Mn0mf62UpgnI9HKer6alJmPMCOf7AqacIaGtkyv5cgjUDNR9CrCS6Cv+QDiIKjMnl4G5LYMuSdND2Sx62Eeb9f0sD1jdyCkkgZ0dQT3zFiL0O9mTKhqacpOxapzQ/t/7wcJH9cU=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(376002)(136003)(396003)(346002)(46966006)(36840700001)(186003)(6666004)(81166007)(356005)(33656002)(478600001)(83380400001)(54906003)(6862004)(956004)(86362001)(36756003)(2906002)(82310400003)(6486002)(82740400003)(336012)(44832011)(2616005)(316002)(5660300002)(26005)(6512007)(36860700001)(8676002)(47076005)(70586007)(70206006)(6506007)(8936002)(53546011)(4326008)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 06:50:32.9893
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0480f88d-9590-45f8-d8df-08d97294f46f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4414



> On 8 Sep 2021, at 02:09, Stefano Stabellini <sstabellini@kernel.org> wrot=
e:
>=20
> On Tue, 7 Sep 2021, Julien Grall wrote:
>> On 07/09/2021 12:51, Luca Fancellu wrote:
>>>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>>>=20
>>>> Hi Luca,
>>>>=20
>>>> On 07/09/2021 07:52, Luca Fancellu wrote:
>>>>> Add a design describing a proposal to improve the EFI
>>>>> configuration file, adding keywords to describe domU
>>>>> guests and allowing to start a dom0less system.
>>>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>>>> ---
>>>>>  docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++=
++
>>>>>  1 file changed, 105 insertions(+)
>>>>>  create mode 100644 docs/designs/efi-arm-dom0less.md
>>>>> diff --git a/docs/designs/efi-arm-dom0less.md
>>>>> b/docs/designs/efi-arm-dom0less.md
>>>>> new file mode 100644
>>>>> index 0000000000..8d8fa2243f
>>>>> --- /dev/null
>>>>> +++ b/docs/designs/efi-arm-dom0less.md
>>>>> @@ -0,0 +1,105 @@
>>>>> +# Xen EFI configuration file
>>>>> +
>>>>> +The current configuration file used by Xen when it is started as an =
EFI
>>>>> +application is considering only the dom0 guest and doesn't have any
>>>>> +property to describe and load in memory domU guests.
>>>>=20
>>>> From my understanding, the problem is less about properties (we alread=
y
>>>> have them in the Device-Tree) but more about where are the binaries
>>>> located in memory as we don't know in advance.
>>>=20
>>> Hi Julien,
>> Hi Luca,
>>=20
>>> I think I used the wrong word there, I meant =E2=80=9Ckeyword=E2=80=9D =
instead of =E2=80=9Cproperty=E2=80=9D
>>> because I was referring about the
>>> lack of keywords to describe a domu guest in the Xen EFI configuration =
file.
>>>=20
>>> I agree with you that on systems with static allocation, the kernel and
>>> ramdisk binaries must be at certain locations
>>> that are out of control when we use the EFI boot services, the thing we=
 can
>>> do is provide a keyword to specify the
>>> addresses and then use the CopyMem() function to relocate the kernel/ra=
mdisk
>>> in the address we want.
>>=20
>> I wasn't specifically referring to static allocation here, sorry if this
>> wasn't clear. I was pointing out that most of the information you create=
 in
>> the xen.cfg is going to be similar to what we already provide in the
>> Device-Tree.
>>=20
>> My main concern is everytime we add a new feature in Dom0less, a develop=
er
>> would need to write code for the DT and UEFI. This will increase the cod=
e size
>> and maintenance.
>>=20
>> The same can be said for the admin as if they want to switch from plain =
U-boot
>> to UEFI, they would also need to fully rewrite the bindings.
>>=20
>> AFAICT, most of the information provided in the Device-Tree are usable e=
ven
>> when using UEFI. So I would prefer if we try to re-use what's existing. =
This
>> is what my proposal below was about.
>>=20
>>>=20
>>>>=20
>>>> So I would like to propose something that build on top of the Device-T=
ree
>>>> work we did. Note this is early thoughts.
>>>>=20
>>>> The problematic nodes in the DT are:
>>>>=20
>>>>        module@0x4a000000 {
>>>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>>>            reg =3D <0x0 0x4a000000 0xffffff>;
>>>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>>>        };
>>>>=20
>>>>        module@0x4b000000 {
>>>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>>>            reg =3D <0x0 0x4b000000 0xffffff>;
>>>>        };
>>>>=20
>>>> In particular the property "reg" cannot be known in advance because th=
e
>>>> UEFI stub will be responsible to load the binaries in memory.
>>>=20
>>> Yes that=E2=80=99s true, the UEFI stub is using from the UEFI boot serv=
ice the
>>> AllocatePages function that is giving back an address out of our contro=
l,
>>> then using another function the binary is read from the disk and copied=
 at
>>> that address, finally the UEFI stub is writing the node in the device t=
ree
>>> that
>>> will be used by Xen later.
>>=20
>> I am not sure to follow. Are you saying the UEFI stub will create the do=
m0less
>> node in the DT based on the xen.cfg?
>>=20
>>>=20
>>>>=20
>>>> What we could do is providing a list of binaries to load and associate=
 a
>>>> key for each of them. Something like:
>>>>=20
>>>> binary=3D<binary> <key>
>>>> binary=3D<binary2> <key2>
>>>> ....
>>>>=20
>>>> We can then replace the property "reg" with a new property "uefi,key" =
that
>>>> will contain the name of the binary.
>>>>=20
>>>> What do you think?
>>>=20
>>> Here I=E2=80=99m lost, because I don=E2=80=99t understand what we are g=
oing to do with the
>>> name of the binary.
>>=20
>> <binaryX> would be used by the UEFI stub to load the binary in memory. E=
ach
>> binary will have a <keyX> which helps to refer them in the Device-Tree. =
To
>> give a concrete example, let say we have two dom0less domains:
>>  - DomA: 2 vCPUs, 128MB
>>  - DomB: 3 vCPUs, 512MB
>>=20
>> DomA and DomB will be using the same kernel but a different ramdisk. xen=
.cfg,
>> would look like:
>>=20
>> [global]
>> default=3Dsection1
>>=20
>> [section1]
>> options=3Dconsole=3Dvga,com1 com1=3D57600 loglvl=3Dall noreboot
>> kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>> ramdisk=3Dinitrd-3.0.31-0.4-xen
>> xsm=3D<filename>
>> dtb=3Ddevtree.dtb
>> binary=3Dvmlinuz-guest domu-kernel
>> binary=3Dramdisk-domA.img domA-ramdisk
>> binary=3Dramdisk-domB.img domB-ramdisk
>>=20
>> The chosen node in the DT would look like:
>>=20
>> chosen {
>>    domU1 {
>>        compatible =3D "xen,domain";
>>        #address-cells =3D <0x2>;
>>        #size-cells =3D <0x1>;
>>        memory =3D <0 0x8000000>;
>>        cpus =3D <2>;
>>=20
>>        module@1 {
>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>            uefi,binary =3D "domu-kernel";
>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>        };
>>=20
>>        module@2 {
>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>            uefi,binary =3D "domA-ramdisk";
>>        };
>>    };
>>=20
>>    domU2 {
>>        compatible =3D "xen,domain";
>>        #address-cells =3D <0x3>;
>>        #size-cells =3D <0x1>;
>>        memory =3D <0 0x20000000>;
>>        cpus =3D <3>;
>>=20
>>        module@1 {
>>            compatible =3D "multiboot,kernel", "multiboot,module";
>>            uefi,binary =3D "domu-kernel";
>>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>>        };
>>=20
>>        module@2 {
>>            compatible =3D "multiboot,ramdisk", "multiboot,module";
>>            uefi,binary =3D "domA-ramdisk";
>>        };
>>    };
>> };
>>=20
>> With this approach, the change is quite minimal to move between an class=
ic
>> U-boot boot and EFI boot.
>=20
> Great idea! I think it is good to try to reuse Device Tree, and using it
> as configuration is well aligned with other projects too (e.g.  Zephyr).
>=20
>=20
> There are a few options for the bindings. These are some ideas.
>=20
> If we are just going to specify a filename and a reference with the
> "binary" key=3Dvalue pair, then we could get rid of it entirely and just
> write the filename directly in device tree:
>=20
>    domU1 {
>        compatible =3D "xen,domain";
>        #address-cells =3D <0x2>;
>        #size-cells =3D <0x1>;
>        memory =3D <0 0x8000000>;
>        cpus =3D <2>;
>=20
>        module@1 {
>            compatible =3D "multiboot,kernel", "multiboot,module";
>            uefi,binary =3D "vmlinuz-guest";
>            bootargs =3D "console=3DttyAMA0 init=3D/bin/sh";
>        };
>    };
>=20
> Then we don't even need the binary key=3Dvalue pair, and especially the
> device tree becomes "self contained" as in we don't need to resolve the
> reference to "binary=3Dvalue ref" to understand it.
>=20
>=20
> (Also worth noting as inspiration the HyperLaunch binding that is using
> the "mb-index" property as index in the multiboot module chain. Not
> directly applicable, but interesting.)

Hi Stefano,

Yes this is an amazing idea, it will solve every issue without introducing =
new keywords in the config file
and it maintains the compatibility.
Now in the current code there is a check for modules that are =E2=80=9Cmult=
iboot,module=E2=80=9D, if any is found the configuration
file is skipped, but if we change this bit to check only for modules whose =
parent is the chosen node, we can
distinguish very well between the dom0 and domus kernel.

If we all agree with this, I will update the design.

Cheers,

Luca
=20




From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:52:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:52:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181463.328578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrRR-0002PG-NA; Wed, 08 Sep 2021 06:52:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181463.328578; Wed, 08 Sep 2021 06:52:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrRR-0002P9-KA; Wed, 08 Sep 2021 06:52:17 +0000
Received: by outflank-mailman (input) for mailman id 181463;
 Wed, 08 Sep 2021 06:52:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrRQ-0002P3-Cl
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:52:16 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2f72cfd4-adc1-47cb-8b8a-010e1f42024e;
 Wed, 08 Sep 2021 06:52:15 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrPl-008Ztv-5x; Wed, 08 Sep 2021 06:50:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f72cfd4-adc1-47cb-8b8a-010e1f42024e
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=dBfrdwNivWqqlCKmJ0ASZqHL/mKJ+iqsG8FTGZNie28=; b=D6unVWGGt0nhN9vNLt5bPN4XJ6
	H5YeTJ6wPEjXfqN4OQvde23ebXYK4ScG4iV7RuMd/m2BDecE+805+QIOyDDJIjDMyQpo4eYaj4QmT
	F5gpCSBn3mbFzriE/4gNmiFAEKCw8EFJTlqjfosMVn+/7NyuTmPOcaj/pSI3twWgrK3D3gLCgPrMe
	ka5oWI1awXDsLFEZBWksAMt1lCG3nYEk5JDPx/M4rxdsFFQs1qr52K4DUKb7c+e381aB2rZA5g8eZ
	E72vmwrF3bYOH4V0FUjxdhgtQPxBvK92pTAIHZbOja/YjfQKbYEWK4Z0FzwpkGP7d/t2qsuf7If0X
	4CAEntbw==;
Date: Wed, 8 Sep 2021 07:50:33 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 02/12] swiotlb-xen: fix late init retry
Message-ID: <YThdOdg3VoPAl7xI@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <778299d6-9cfd-1c13-026e-25ee5d14ecb3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <778299d6-9cfd-1c13-026e-25ee5d14ecb3@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:04:47PM +0200, Jan Beulich wrote:
> The commit referenced below removed the assignment of "bytes" from
> xen_swiotlb_init() without - like done for xen_swiotlb_init_early() -
> adding an assignment on the retry path, thus leading to excessively
> sized allocations upon retries.
> 
> Fixes: 2d29960af0be ("swiotlb: dynamically allocate io_tlb_default_mem")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Cc: stable@vger.kernel.org

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:55:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:55:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181470.328589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrUA-00032v-6J; Wed, 08 Sep 2021 06:55:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181470.328589; Wed, 08 Sep 2021 06:55:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrUA-00032o-2U; Wed, 08 Sep 2021 06:55:06 +0000
Received: by outflank-mailman (input) for mailman id 181470;
 Wed, 08 Sep 2021 06:55:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrU8-00032N-AL
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:55:04 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4ee8d88b-6697-42d9-a1a9-435fb2597190;
 Wed, 08 Sep 2021 06:54:59 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrSS-008a0O-5n; Wed, 08 Sep 2021 06:53:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ee8d88b-6697-42d9-a1a9-435fb2597190
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=mNv6jBhL0dwmk4rDDYf4ikhCU0kC8phJg4t0M4uQRDc=; b=aE5y0hGlp2Xj4xb952dvEVdScM
	KnAkeo8HeySFYAkHcdp37IMVTE1i0gtREK8+yHbyQ5UCZahHa1/ncIMqtviCOSYTzL/AmK9KAmrfa
	GbkJD7rJALGnUYk7MMG+5GPjSi3u2vcTySZWbKcHBhIaTZfX4KhIV1b8fYvgn6mid2qUVy57Zv+xj
	bT3JLbdCLI2NEQBhgA864/cN0ICBGxedybXpxHkTrhaXYJUYyO3bprAOlD7SGZz90vpWEEFcJpiki
	keuEbK6oYViBMYnqTQotJTIhoHqzVhJ17AvLP/tLzLc46VGBhDiVNvg6j04I7IECrcc9WRWQ0W5gg
	EL39LFfg==;
Date: Wed, 8 Sep 2021 07:53:20 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 03/12] swiotlb-xen: maintain slab count properly
Message-ID: <YThd4HDvB3pBD2/F@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <dc054cb0-bec4-4db0-fc06-c9fc957b6e66@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <dc054cb0-bec4-4db0-fc06-c9fc957b6e66@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:05:12PM +0200, Jan Beulich wrote:
> Generic swiotlb code makes sure to keep the slab count a multiple of the
> number of slabs per segment. Yet even without checking whether any such
> assumption is made elsewhere, it is easy to see that xen_swiotlb_fixup()
> might alter unrelated memory when calling xen_create_contiguous_region()
> for the last segment, when that's not a full one - the function acts on
> full order-N regions, not individual pages.
> 
> Align the slab count suitably when halving it for a retry. Add a build
> time check and a runtime one. Replace the no longer useful local
> variable "slabs" by an "order" one calculated just once, outside of the
> loop. Re-use "order" for calculating "dma_bits", and change the type of
> the latter as well as the one of "i" while touching this anyway.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 06:58:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 06:58:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181476.328600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrXI-0003hH-Me; Wed, 08 Sep 2021 06:58:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181476.328600; Wed, 08 Sep 2021 06:58:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrXI-0003hA-J9; Wed, 08 Sep 2021 06:58:20 +0000
Received: by outflank-mailman (input) for mailman id 181476;
 Wed, 08 Sep 2021 06:58:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrXG-0003h4-TA
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 06:58:18 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4e0cca89-1a5b-425a-8854-31c4169a11bb;
 Wed, 08 Sep 2021 06:58:18 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrWF-008a9l-56; Wed, 08 Sep 2021 06:57:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e0cca89-1a5b-425a-8854-31c4169a11bb
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=yqhbwzwdJH8qRS1MarFWp8lRnwDdQKIjHrVncEEHPh8=; b=fgBDwHw7l63tdQPsH7I2iUXyMZ
	u6GLvgy1FAu0zukPtZrZWZAGYTkNv93qLQ0SzlM4o/dSNEZZB34rJlFBTv+g/qg2PSEBxfH0qrtEq
	40iF+eI7p8r9HBX4e+qdIj0FiX5kR8x0Cn7qtaYsW6Hvb2GMjQzPi+kaNS7GQYVkm5RdE447u5hGc
	0550qzIoVi9MtzwZPjO8uku14gwMEt+yVYO9XZqzB0c2z092Fp/J7HiOH1tOmYfikBtfWZVXUaXy0
	tBjOyYM79QvEb39a4jsRkH0T1KaIT2OpzIi4U5yUtMw5Yt8jo0gL7SIhFqjH9BrADxVv9XMb5RgYq
	euwSfHxg==;
Date: Wed, 8 Sep 2021 07:57:15 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
Message-ID: <YThey/iyCxi5NUwC@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:05:32PM +0200, Jan Beulich wrote:
> While the hypervisor hasn't been enforcing this, we would still better
> avoid issuing requests with GFNs not aligned to the requested order.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I wonder how useful it is to include the alignment in the panic()
> message. I further wonder how useful it is to wrap "bytes" in
> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
> least was supposed to be, prior to "swiotlb-xen: maintain slab count
> properly").
> 
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -231,10 +231,10 @@ retry:
>  	/*
>  	 * Get IO TLB memory from any location.
>  	 */
> -	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
> +	start = memblock_alloc(PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
>  	if (!start)
> -		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
> -		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
> +		panic("%s: Failed to allocate %lu bytes align=%#x\n",
> +		      __func__, PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);

CAn you avoid the overly long lines here?  A good way to make it more
readable would be a variable to hold the byte count.


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:00:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:00:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181482.328610 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrYx-0004Wz-1U; Wed, 08 Sep 2021 07:00:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181482.328610; Wed, 08 Sep 2021 07:00:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrYw-0004WI-Ug; Wed, 08 Sep 2021 07:00:02 +0000
Received: by outflank-mailman (input) for mailman id 181482;
 Wed, 08 Sep 2021 07:00:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrYv-0004Hv-Hk
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:00:01 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc851688-09a0-43ef-9428-e4d35a423485;
 Wed, 08 Sep 2021 07:00:00 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrXr-008aF4-Do; Wed, 08 Sep 2021 06:59:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc851688-09a0-43ef-9428-e4d35a423485
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=7gmr+v/UYd69hqQff5Fcu4B0asSjyasIdDD4iqilMBs=; b=BhJhq2e3Ic/6vaQ2X9A7y70QlA
	LN7tsn+RSNKPWUCjSCIDNsjviepIb4CgpXAMqNPNl0hCUOuRIE5CY+CkQLja8SX75eR6DryDm1ntu
	3lscohH7+1ssTfUDNi/P/N7uBNXacnQBZA8Hwdqtv+q8NMyvtpoAp4qXNq9o5P6fAZ2/dkCjoEriy
	9tKDLpzsK8JdUsp5eCc433G7x3oabL/dTqC0I0T/PLIzJFdqeztX9uUHC6V51OBay7pb4m35Oya0f
	MqyWaHEDdXo8bVRyW4we+Z42VMXBxFTjies7EWwuwlP/xoOr+NeRaW4DyrX4oGeCcpwEA6IVe1Gbq
	8C+Nm+Sg==;
Date: Wed, 8 Sep 2021 07:58:55 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 05/12] swiotlb-xen: suppress certain init retries
Message-ID: <YThfL9RChhtBA0Oc@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <56477481-87da-4962-9661-5e1b277efde0@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <56477481-87da-4962-9661-5e1b277efde0@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:05:54PM +0200, Jan Beulich wrote:
> Only on the 2nd of the paths leading to xen_swiotlb_init()'s "error"
> label it is useful to retry the allocation; the first one did already
> iterate through all possible order values.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:01:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:01:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181489.328622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNraD-0005rE-Da; Wed, 08 Sep 2021 07:01:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181489.328622; Wed, 08 Sep 2021 07:01:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNraD-0005r7-9X; Wed, 08 Sep 2021 07:01:21 +0000
Received: by outflank-mailman (input) for mailman id 181489;
 Wed, 08 Sep 2021 07:01:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNraB-0005r1-VT
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:01:19 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ce99a267-d7c9-403e-8b14-e9dad578ed15;
 Wed, 08 Sep 2021 07:01:19 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrZH-008aHN-76; Wed, 08 Sep 2021 07:00:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce99a267-d7c9-403e-8b14-e9dad578ed15
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=R8OKvw4S2MzNKNmIXpBxq9HTvvmNUrQ/SvUk9VnsHUY=; b=rpo3HkOJxy7GQnr44u+Kw8/GWw
	b8U0qMtuai8fWaIBPF65LKePgrrYtAvsRonr7HKZ/kABbXwFaD3Mpb9NLGgv57kVtXFmpclCY72AW
	bRhylceyEaHc9mZwksibH7RRQWka7q8ZoZO0inYa0QUo14guNPVn4vgylPrf7IJVRKoj/ZEvI6QNn
	BnqFevB+Z2JckxFutk9LRF6NHmlUhurLK75V0CumKbej0KWAQMeYjX0SWkuA0cQAqZ5knLN3VjNDg
	0Q71U9TW6LUgg1Y/e5lIBYVvFbraJ7etFrdulT7G7nM5y7yJQdeATwHB+s08UVof9jVe9hk8cGZ1V
	nCpsr0OQ==;
Date: Wed, 8 Sep 2021 08:00:23 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 06/12] swiotlb-xen: limit init retries
Message-ID: <YThfh2aIlAIkdrXA@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <984fa426-2b7b-4b77-5ce8-766619575b7f@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <984fa426-2b7b-4b77-5ce8-766619575b7f@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:06:37PM +0200, Jan Beulich wrote:
> Due to the use of max(1024, ...) there's no point retrying (and issuing
> bogus log messages) when the number of slabs is already no larger than
> this minimum value.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:03:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181496.328633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrck-0006Wy-Ph; Wed, 08 Sep 2021 07:03:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181496.328633; Wed, 08 Sep 2021 07:03:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrck-0006Wr-MY; Wed, 08 Sep 2021 07:03:58 +0000
Received: by outflank-mailman (input) for mailman id 181496;
 Wed, 08 Sep 2021 07:03:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrcj-0006Wl-Il
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:03:57 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 80301a52-5a33-41df-be3c-c5a0d125574b;
 Wed, 08 Sep 2021 07:03:55 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrbE-008aLp-HO; Wed, 08 Sep 2021 07:03:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80301a52-5a33-41df-be3c-c5a0d125574b
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=gatW1BubQnKb2kduQto9Cf0sWBkeir64mglmf+pQf2Q=; b=k7G3Dc9dTikD/msbbkUkrK6bH/
	rcD7O+3KeLMyQ1++oVllKlyi8fr3OzWCFSPglsQEwTuZYSOeBs1r73L3/drk5ceobljVtDfydSKFB
	j/domY+VTpfr9Pk9f5rNEQqFYIDkHP5jTSDi9iVT3CSZOC35ka5bQlKxSolhcWPXaq/1QfK21G+ih
	COU4O1kTW2U6YdQXei4QY2KiZAUUKvTqMlzEA/VevxT4QnIBgJt2vBtZgAO4rpNo8bQHCz2QyPe5C
	fe2/HGVSMq4ipo4jC45iFAjbTS29+lrORLywMIyJG16unTn6UIJAoJaoEX8vMVXeRptUArk8CdQ0M
	IkU/sUdQ==;
Date: Wed, 8 Sep 2021 08:02:24 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 07/12] swiotlb-xen: drop leftover __ref
Message-ID: <YThgAClbSGQCBv5n@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <7cd163e1-fe13-270b-384c-2708e8273d34@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <7cd163e1-fe13-270b-384c-2708e8273d34@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:06:55PM +0200, Jan Beulich wrote:
> Commit a98f565462f0 ("xen-swiotlb: split xen_swiotlb_init") should not
> only have added __init to the split off function, but also should have
> dropped __ref from the one left.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:06:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:06:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181502.328644 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNres-00079G-6j; Wed, 08 Sep 2021 07:06:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181502.328644; Wed, 08 Sep 2021 07:06:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNres-000799-3c; Wed, 08 Sep 2021 07:06:10 +0000
Received: by outflank-mailman (input) for mailman id 181502;
 Wed, 08 Sep 2021 07:06:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrer-000793-0h
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:06:09 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 47745604-fee7-4bdb-b441-22ad7585a358;
 Wed, 08 Sep 2021 07:06:03 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrdA-008aSO-K1; Wed, 08 Sep 2021 07:04:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47745604-fee7-4bdb-b441-22ad7585a358
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=u0jDPhGZwBkm+JMr6omQzYQizJyYgqZlwoU/jhm0G5s=; b=MlbsyzCdFKeIEDRfx1McK2amJc
	sZxrTxkScG/1WgXYGHD4V+bhWBRDZcajEMfSC3Rwh7B9lDj5xGPybecP4NGCxlvEnCQ5dAOw/mvQC
	Ne+YzN2biFxXb0cqjqdD5AOk5OTnSO86XUY8fGrf5F9OLSg1zVss0ee990dy8gbMeXRwV80gqaQDr
	AB/je8uC8iu7fwxs1wpvfFnQHp9bbN0RhMh1Yv+PeKByy9sQeiTX39GFwbTWNr1IIAGhJtTj1RjEi
	nMjgdiOKSed2JDSqDn7hI5nd+A9lN6PF4SsxMBk0Yvlq7ucqDs2VuYNkj6v52X1Qm7lpLLWu3Xn5N
	JDT+ugtg==;
Date: Wed, 8 Sep 2021 08:04:24 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 08/12] swiotlb-xen: arrange to have buffer info logged
Message-ID: <YThgeOiR0PU+iNlC@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <2e3c8e68-36b2-4ae9-b829-bf7f75d39d47@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2e3c8e68-36b2-4ae9-b829-bf7f75d39d47@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:07:21PM +0200, Jan Beulich wrote:
> I consider it unhelpful that address and size of the buffer aren't put
> in the log file; it makes diagnosing issues needlessly harder. The
> majority of callers of swiotlb_init() also passes 1 for the "verbose"
> parameter. 
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:08:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:08:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181508.328655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrh7-0007ml-Ko; Wed, 08 Sep 2021 07:08:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181508.328655; Wed, 08 Sep 2021 07:08:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrh7-0007me-Ga; Wed, 08 Sep 2021 07:08:29 +0000
Received: by outflank-mailman (input) for mailman id 181508;
 Wed, 08 Sep 2021 07:08:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrh5-0007mY-V4
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:08:27 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 68e862cc-57a1-443a-963a-8cfc54df5453;
 Wed, 08 Sep 2021 07:08:26 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrfF-008aXl-2g; Wed, 08 Sep 2021 07:06:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68e862cc-57a1-443a-963a-8cfc54df5453
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=xTrVWx4u6aj0HJxIiwK6w63wBv8UFL6W0kryTgqQOBk=; b=UZnSHfGOxpfl5k2QHa5deLLAej
	zonJbx4n51ec8iDwWq3JYz2OPPMiVjGyAUWGv4ls4dA1usel5U3fT/1gvezBu704ucCFd3+IoU+Xx
	/3suk5AMMFVT85WrMAbyKyeKyDc8jQpU1ubdS0X6vYrUL91adfQ/uEwVJtCOmDDvt/xQGeNw5r7aT
	yWtkmphfLXByz/kFQQDjCEIQq8TuIrs78mKspznFajfvCQvCX6gPMzESAMk0sibuyeVlB3FQIOYdo
	CHiaZ+u6qSC1/p9oaZJ8UOV2EehqAHMkshll3Ux7bbkQN51H9mmfHonVGwsJSgxPP910Gd4H8k47p
	w6qvbcTw==;
Date: Wed, 8 Sep 2021 08:06:33 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 09/12] swiotlb-xen: drop DEFAULT_NSLABS
Message-ID: <YThg+e+5lDoSFuqO@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <15259326-209a-1d11-338c-5018dc38abe8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <15259326-209a-1d11-338c-5018dc38abe8@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:07:47PM +0200, Jan Beulich wrote:
> It was introduced by 4035b43da6da ("xen-swiotlb: remove xen_set_nslabs")
> and then not removed by 2d29960af0be ("swiotlb: dynamically allocate
> io_tlb_default_mem").
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:10:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:10:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181514.328666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrj4-0000ga-WD; Wed, 08 Sep 2021 07:10:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181514.328666; Wed, 08 Sep 2021 07:10:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrj4-0000gT-TH; Wed, 08 Sep 2021 07:10:30 +0000
Received: by outflank-mailman (input) for mailman id 181514;
 Wed, 08 Sep 2021 07:10:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrj4-0000gN-8d
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:10:30 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e2bd219-9118-4494-9a0d-590a504c9997;
 Wed, 08 Sep 2021 07:10:29 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrha-008aj6-H4; Wed, 08 Sep 2021 07:09:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e2bd219-9118-4494-9a0d-590a504c9997
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=BIE/LIdFO8jY5B6XQFOTWQMi8dbTNeZ9iRgAJzUQFJU=; b=sj3hwvVEXJG7RXFgHJKiWrl2DW
	PSMKQHTTMqu0a23Z7httVaAjEPT4AM/xssoAfAMFxEVJ6rBlvytzg+s4FehCm7M2Zxirf64YczMY7
	12aopbnZx89CwC6n3lwGa5O0523gUpp+NyNoIRD2nC/aGJ3FvqwZHevdybiDOFcOFgaGILaXKVRNe
	Guo/1qOs4FIXJKtxXEZwQhH4ONEiYUG6xp7qxMxYlUPnI8paKE4dBt5vDzj/NFUNVslm6Yp2u200T
	CuTrCqt17eJo9fP2yxyZHTHqo9QXfN+KhClS4uoWVTEBVxSLfZlemZN6pP2UsIeLh8g8G0WXBkRNM
	kRaojG4w==;
Date: Wed, 8 Sep 2021 08:08:58 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 11/12] xen/pci-swiotlb: reduce visibility of symbols
Message-ID: <YThhih963WfY/3/W@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <75591cef-95e8-c694-fd11-283800499e4e@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <75591cef-95e8-c694-fd11-283800499e4e@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:11:14PM +0200, Jan Beulich wrote:
> xen_swiotlb and pci_xen_swiotlb_init() are only used within the file
> defining them, so make them static and remove the stubs. Otoh
> pci_xen_swiotlb_detect() has a use (as function pointer) from the main
> pci-swiotlb.c file - convert its stub to a #define to NULL.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:15:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:15:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181526.328676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNro5-0001d2-Mq; Wed, 08 Sep 2021 07:15:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181526.328676; Wed, 08 Sep 2021 07:15:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNro5-0001cv-Js; Wed, 08 Sep 2021 07:15:41 +0000
Received: by outflank-mailman (input) for mailman id 181526;
 Wed, 08 Sep 2021 07:15:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bk/O=N6=eikelenboom.it=linux@srs-us1.protection.inumbo.net>)
 id 1mNro4-0001cp-0O
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:15:40 +0000
Received: from server.eikelenboom.it (unknown [91.121.65.215])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5270bb18-20b0-4a0e-84d0-487559756e76;
 Wed, 08 Sep 2021 07:15:37 +0000 (UTC)
Received: from 76-24-144-85.ftth.glasoperator.nl ([85.144.24.76]:59650
 helo=[172.16.1.50]) by server.eikelenboom.it with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <linux@eikelenboom.it>)
 id 1mNrnq-0003D7-Vf; Wed, 08 Sep 2021 09:15:27 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5270bb18-20b0-4a0e-84d0-487559756e76
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=eikelenboom.it; s=20180706; h=Content-Transfer-Encoding:Content-Type:
	In-Reply-To:MIME-Version:Date:Message-ID:References:To:From:Subject:Sender:
	Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From:
	Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:
	List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=bqLBZYZf+OGSwmIWDrkm11xoKqUhXHfSvj8/ddCAhHY=; b=f+l7brMEK28eKiSLveWpl+GucP
	KwotBj40F1WcwLc34Kyaj+9/tr19Gu+v1hWtzJpIpA9ffFj9P+fivlaGcF4cBrxIndrefuz2FvspB
	ZOI5n0APGJAAfHPLkTij9pEJJ+zUHLgJWGv+kLPG8VjzCrmqFoP9nYikM7eSRR/RfBos=;
Subject: Re: xen-unstable linux-5.14: 1 of 2 multicall(s) failed: cpu 0
From: Sander Eikelenboom <linux@eikelenboom.it>
To: Juergen Gross <jgross@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9c36205e-cc37-ed9a-3408-0cccc4a83115@eikelenboom.it>
 <955da8ce-40f2-32c5-ee78-6801ce18afe5@suse.com>
 <b0854555-01fd-cbd8-6e6a-c793109b2972@eikelenboom.it>
Message-ID: <f38766b1-9b6b-e301-93e4-12f3cce618b0@eikelenboom.it>
Date: Wed, 8 Sep 2021 09:15:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <b0854555-01fd-cbd8-6e6a-c793109b2972@eikelenboom.it>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: nl-NL
Content-Transfer-Encoding: 8bit

On 07/09/2021 21:52, Sander Eikelenboom wrote:
> On 07/09/2021 15:53, Juergen Gross wrote:
>> On 06.09.21 23:35, Sander Eikelenboom wrote:
>>> L.S.,
>>>
>>> On my AMD box running:
>>>        xen-unstable changeset: Fri Sep 3 15:10:43 2021 +0200 git:2d4978ead4
>>>        linux kernel: 5.14.1
>>>
>>> With this setup I'm encountering some issues in dom0, see below.
>>
>> Could you test whether the attached patch (only compile tested)
>> fixes the issue for you?
>>
>>
>> Juergen
>>
> 
> Sure, running it now.
> Since I haven't got a clear trigger, will report tomorrow if the logs stay clean :)
> 
> Thanks so far !
> 
> --
> Sander
> 
> 

Hi Juergen,

So far so good, it survived the night without issues.
So I think you can add a Tested-By.

Thanks for the swift fix !

--
Sander


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:16:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:16:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181533.328688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrp6-0002Dd-1H; Wed, 08 Sep 2021 07:16:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181533.328688; Wed, 08 Sep 2021 07:16:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNrp5-0002DW-Tw; Wed, 08 Sep 2021 07:16:43 +0000
Received: by outflank-mailman (input) for mailman id 181533;
 Wed, 08 Sep 2021 07:16:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EHX5=N6=casper.srs.infradead.org=BATV+2e9f3f7219a4df6f1e93+6590+infradead.org+hch@srs-us1.protection.inumbo.net>)
 id 1mNrp4-0002DM-4b
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:16:42 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f81b569-4fed-42fa-9d05-4bfd16929154;
 Wed, 08 Sep 2021 07:16:40 +0000 (UTC)
Received: from hch by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat
 Linux)) id 1mNrml-008b2V-QY; Wed, 08 Sep 2021 07:14:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f81b569-4fed-42fa-9d05-4bfd16929154
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=MnjsixqmosBihT6Q0qVYXXqBZ2K/JeOxHPpv883b1xQ=; b=XbM8Xg3PjSFGTYmv3zW07WfyaY
	yWEdMq1eEqx0RRQqf2r7e6pYKpvEx0oG4KVrQaoCagb/MmI7o0acywnXwpXSPJTo/1MkSPkPOEFJy
	mKVPyz7XoUQE44YQmLjxnlC/1P/RLwrxrspgdaqdkFsWsJpB2cRQfFSd4r+Bz7k7bOYyGJ9XCEdOU
	VV701Kva713ECwwrgTRZcHhYdgohcpF+ORONW/M/BNUXYF7JoqGFO0tJVrTc+1JqjXQEOCwt3RRqO
	IbwKTqnS0QnoY/SeWesV1xBSinYJz2jyaANLtuvExQupl4FPr9q6HdJd1s3vCXjHopyF3JjAZ7vVe
	7+fwQCwQ==;
Date: Wed, 8 Sep 2021 08:14:19 +0100
From: Christoph Hellwig <hch@infradead.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	the arch/x86 maintainers <x86@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
Message-ID: <YThiyxG0d2tmCtb+@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
> The code is unreachable for HVM or PVH, and it also makes little sense
> in auto-translated environments. On Arm, with
> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
> time seeing what good the Xen specific variant does - the generic one
> ought to be fine for all purposes there. Still Arm code explicitly
> references symbols here, so the code will continue to be included there.

Can the Xen/arm folks look into that?  Getting ARM out of using
swiotlb-xen would be a huge step forward cleaning up some DMA APIs.

> 
> Instead of making PCI_XEN's "select" conditional, simply drop it -
> SWIOTLB_XEN will be available unconditionally in the PV case anyway, and
> is - as explained above - dead code in non-PV environments.
> 
> This in turn allows dropping the stubs for
> xen_{create,destroy}_contiguous_region(), the former of which was broken
> anyway - it failed to set the DMA handle output.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:34:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:34:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181543.328699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNs6I-0004uC-JA; Wed, 08 Sep 2021 07:34:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181543.328699; Wed, 08 Sep 2021 07:34:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNs6I-0004u5-Et; Wed, 08 Sep 2021 07:34:30 +0000
Received: by outflank-mailman (input) for mailman id 181543;
 Wed, 08 Sep 2021 07:34:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=GobK=N6=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mNs6H-0004tz-AA
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:34:29 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.81]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 31fdb5b4-1077-11ec-b147-12813bfff9fa;
 Wed, 08 Sep 2021 07:34:27 +0000 (UTC)
Received: from AM6P191CA0072.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::49)
 by GV1PR08MB7314.eurprd08.prod.outlook.com (2603:10a6:150:1d::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Wed, 8 Sep
 2021 07:34:18 +0000
Received: from VE1EUR03FT012.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:7f:cafe::b6) by AM6P191CA0072.outlook.office365.com
 (2603:10a6:209:7f::49) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 07:34:18 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT012.mail.protection.outlook.com (10.152.18.211) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 07:34:18 +0000
Received: ("Tessian outbound 0ec886cb54dd:v105");
 Wed, 08 Sep 2021 07:34:17 +0000
Received: from e0baeef2aed7.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 85FF79B3-9BA3-4458-AEF8-EC0CB6DB9A29.1; 
 Wed, 08 Sep 2021 07:34:08 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e0baeef2aed7.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 07:34:08 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB6281.eurprd08.prod.outlook.com (2603:10a6:10:1f6::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 07:34:05 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4478.025; Wed, 8 Sep 2021
 07:34:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31fdb5b4-1077-11ec-b147-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3Z+hSZ9cAhQo5fyXkwP2E3uRS7GHreh4U+7Gx87EBjs=;
 b=2BOFqqVmTJNw0HYG2y+nk+x7tus10L5yedUbvS7Iyla+tade+y1M4TPr+9jYyhFEt/PpBTUPKKAgOhCyqMnrfoP8hW23dK02ANgbZmb0LZYVa+pLLsobfdrEEo45Vvh++I8lKeWrmgN6nz9VspbsLi0KudX8QgG7gr6kLwoIksA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KSmxRL6x+q0cZxShWhZZ3qnp+XpEh6YPSINKOBd+6Szccvfs3HWHffP+h/z0QV1D1HBUHhM7P50a1SJOhQ3UdOxqaYk9urOZu3pqH9WSkWcb/bUgTnyoQtNQ/IBEyCiq8F/9NwUwIuf3zZ+EwoJJ+7u3SY2JEydo5YsElYUNhxSdZio2XGMjRTYbIYEsPZiI6TGRXmulxeMjgiiOwY9FUHwFmxEExaT7dpXSRlmYu6fqpnJIliqYeaRDer+A/Q5HWTSlWSRo1jpIpjMjk41BrO0CEeuy+kZtCGBTMvaQeBjGU2yuPqOGnLs8LWcHrQBEJ3dKLl8yR8XjjCRLuJNVdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3Z+hSZ9cAhQo5fyXkwP2E3uRS7GHreh4U+7Gx87EBjs=;
 b=HKsK2fVoX30nujNdv6rfMgMAZpRd7mwaoClPT1cXVVms3N1B9CE5jRjZ/fuVVdWwlyGjaMu9S4wjOB78Uzu6M/PLA825nOFGtPf7jxVa51HfbE/N8Nlq7f906lqmmMOBBEFOYWEto1jH4KkoHhbq4L1J+dwO10qO1sHdA25knqo77WR/+4pBahzfcsvN8/cfUqQDz74JC9jRcSkf1JoKaU+xqL++h51nMFZdPRBHCe6Bjam9OGqu2ftxGIWtWkD3UYAAhF9P67Vh867xUzjwm9ImAjnRs1eIlLAtCeckMYXGxcRq7g5F9OxbK5DG/dZSCCPQ5NZtKvIV1fEXW1m0OQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3Z+hSZ9cAhQo5fyXkwP2E3uRS7GHreh4U+7Gx87EBjs=;
 b=2BOFqqVmTJNw0HYG2y+nk+x7tus10L5yedUbvS7Iyla+tade+y1M4TPr+9jYyhFEt/PpBTUPKKAgOhCyqMnrfoP8hW23dK02ANgbZmb0LZYVa+pLLsobfdrEEo45Vvh++I8lKeWrmgN6nz9VspbsLi0KudX8QgG7gr6kLwoIksA=
From: Wei Chen <Wei.Chen@arm.com>
To: Wei Chen <Wei.Chen@arm.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>
Subject: RE: [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse device
 tree memory node
Thread-Topic: [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse
 device tree memory node
Thread-Index: AQHXjps6uVXfLZKD7EKSvvhEptOzS6uINJ4AgAAs9PCAAHGGgIAAOFuQgBDWjrA=
Date: Wed, 8 Sep 2021 07:34:04 +0000
Message-ID:
 <DB9PR08MB68572B980E3E688F3E946C329ED49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-24-wei.chen@arm.com>
 <alpine.DEB.2.21.2108271757250.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857618112DC045293A449699EC99@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <5c9a196a-8ac3-1173-f402-01d99ccc23e4@xen.org>
 <AS8PR08MB68531741D1035B35A19EF34F9EC99@AS8PR08MB6853.eurprd08.prod.outlook.com>
In-Reply-To:
 <AS8PR08MB68531741D1035B35A19EF34F9EC99@AS8PR08MB6853.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: B42AF9527E707E4F828EBE67411CFEAB.0
x-checkrecipientchecked: true
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c9698f87-b599-4132-5fc8-08d9729b1154
x-ms-traffictypediagnostic: DBBPR08MB6281:|GV1PR08MB7314:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<GV1PR08MB73142CD81206F45972F4CF499ED49@GV1PR08MB7314.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 U/DkNesgfc40VW7sjTPexyjjxeOVhyVtJwcu4O1SQV8iNfs+5ySPIUCtY5SpirDjMI1f3znBODsat65VCmeIoSP7/jRId6hWgx5IWiKw7txeZo6GPvcXNgyPZGswSAPz6qbf0+MtVu/I7AD0lBBOJ6RvmSrCnJKBsqyLC/NSQM93T0mUrEMztX5NYsmj84QxMDCxIfhCeRoRZkyAOePkwN7H+KrZvVDShS1KLGnKRKgtD6f5Goo1jgIAXIQ1sVu1+GzSQaQhLnx5BsCDChjw2sZdlNTYaxoHHj4QnH8sTieOlqp36XC29dZQTENvZyayGRfMHxFqYMxLwlxPN8bpyg+CcHUB6r/D/1xlxg+ua4vOvAeHhtwDfpHPrYp9Ebin14RoajlX2lidVkbN0GKxJCXajlX69epR8c3gSpuGHOAKBcYWU7eoP6reWHJ/vAmN/6wWd3am9fIS391ATtnMZN8V9PAjFuiZJxU0jCzLEOvjpCArFGNPlKoOgyJDPpsXtNq4obzgMaCmg0t5xCl/2bAkKK9PAHjQ5xxWBzq6d2ajiqXZron+HlNdqiXjlRdvKmy1kAjf/UyYPx9WAKPQYSGgVq02J3CQpKBKvhXOXxlBVPhV2SYPeCaXuJ2V7T1NyeF57VmGjHPNm+WajLFtFxONByq+BKqnpn45J0z25Jx+ZyVrISrQSQUcEtk/UQVpM2DEH1w0+/ZYaXpA/RF8W0zwsxm//s0dFNXEXm3AJY3zB7ylZYDpR6/dzK+yfCfnAVYuznhavO26drP5031EvxaIJfyHRcXklU6wtlBwWZ9RQGybpPqiOJVH+CrPYEcM8XRMf6XYR6VJ065oAq2D7Q==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(86362001)(2906002)(110136005)(316002)(5660300002)(9686003)(4326008)(55016002)(54906003)(33656002)(38070700005)(71200400001)(966005)(53546011)(6506007)(52536014)(186003)(66946007)(66446008)(66556008)(66476007)(64756008)(76116006)(8936002)(83380400001)(478600001)(26005)(8676002)(38100700002)(122000001)(7696005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6281
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b6b65d38-cc08-4940-3c67-08d9729b093c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	u3fd9pt+5wWhVfuSJ/JSKWcjID+5fgu8Dk4BZOMbIsskenmu3h9RNuWNhonZP1BaTeEL2sHgqVZrpK8i6IsvJGnuNZ+xzl2Kd5KQGLjSGpyNPm0W5b5cVUp4wpNFw9mSn7jq3G0jsi8/aGx2775HS1lrlfhx6nY9p+wQ/xW2GX+uIyhSuVf89+Miqkbm8miIuwVc/drpyCUCfQ5IUOQ7CJ9s/w0y9y5+V+npY0DNJSBdpt+IhJo8LR/FgM9SObdEFE4a91AhxEYZVaBXhDTSOyfaa1ZJAPqRZ/Rfm38HAh9Za/WN1YIYSLvtT8a4wMauR2pHSSydijHTL4mpkrKjCu6n55oY+c5w+Ofsm6vTSQHZd2+xsg72sIg+ytE5UovbSO5Sj3RitYkUIjhWaoEm8sNqOxPUzoQSasJXju/CwdHopoO3P8E6qzWNGJDmuuynIxWWA9pFLke74WkUvERccFxBzY5hQe4zqAhI43FaZqVCAsIcbKd/N361wVFB6+HgdwX1kssxK1mON8Jlw3uXEeM1wREDBdFkVjWs5ty7Jt9Fl10RwUIT+NVjhQIzbDZWT/iF45+rsZkJ3GKzGWVDI0H80DUBx05elB7BZR8YbL9J5jBuS3gS+DauUSeFpBaAq9XZhlz7K1OSIF9L2kGEfy2HCzBqxDUAoHoZokVBmsYQJVXKJv8UwyfG4vg4G6MLT9Yxqsbkx8yXaCMWgOciy/jfZZbAhDbaY0HjxtsbbGnfpZp4iPkH6Xe4pFleiUilMlCs0GMzqrWBebMXcLgpG+QvZVfca/MUAsb/yeOOox68w802HBrf4VSH3A1j9Z3I7H7pW4T/ZeEu3M6NTrY6mg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(346002)(376002)(396003)(39860400002)(46966006)(36840700001)(52536014)(7696005)(336012)(9686003)(8936002)(54906003)(107886003)(82740400003)(186003)(966005)(26005)(33656002)(86362001)(55016002)(6506007)(5660300002)(478600001)(53546011)(8676002)(2906002)(47076005)(356005)(70206006)(70586007)(82310400003)(36860700001)(316002)(83380400001)(81166007)(110136005)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 07:34:18.3466
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c9698f87-b599-4132-5fc8-08d9729b1154
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7314

SGkgSnVsaWVuLCBTdGVmYW5vLCBKYW4NCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0K
PiBGcm9tOiBYZW4tZGV2ZWwgPHhlbi1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3Jn
PiBPbiBCZWhhbGYgT2YgV2VpDQo+IENoZW4NCj4gU2VudDogMjAyMeW5tDjmnIgyOOaXpSAyMTo1
OA0KPiBUbzogSnVsaWVuIEdyYWxsIDxqdWxpZW5AeGVuLm9yZz47IFN0ZWZhbm8gU3RhYmVsbGlu
aQ0KPiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0cy54ZW5w
cm9qZWN0Lm9yZzsgQmVydHJhbmQgTWFycXVpcw0KPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
PjsgSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KPiBTdWJqZWN0OiBSRTogW1hFTiBS
RkMgUEFUQ0ggMjMvNDBdIHhlbi9hcm06IGludHJvZHVjZSBhIGhlbHBlciB0byBwYXJzZQ0KPiBk
ZXZpY2UgdHJlZSBtZW1vcnkgbm9kZQ0KPiANCj4gSGkgSnVsaWVuLA0KPiANCj4gPiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IEp1bGllbiBHcmFsbCA8anVsaWVuQHhlbi5v
cmc+DQo+ID4gU2VudDogMjAyMeW5tDjmnIgyOOaXpSAxODozNA0KPiA+IFRvOiBXZWkgQ2hlbiA8
V2VpLkNoZW5AYXJtLmNvbT47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA+IDxzc3RhYmVsbGluaUBr
ZXJuZWwub3JnPg0KPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IEJlcnRy
YW5kIE1hcnF1aXMNCj4gPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPg0KPiA+IFN1YmplY3Q6IFJlOiBbWEVOIFJGQyBQQVRDSCAyMy80
MF0geGVuL2FybTogaW50cm9kdWNlIGEgaGVscGVyIHRvIHBhcnNlDQo+ID4gZGV2aWNlIHRyZWUg
bWVtb3J5IG5vZGUNCj4gPg0KPiA+IEhpIFdlaSwNCj4gPg0KPiA+IE9uIDI4LzA4LzIwMjEgMDQ6
NTYsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+
ID4gPj4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0K
PiA+ID4+IFNlbnQ6IDIwMjHvv73vv70477+977+9Mjjvv73vv70gOTowNg0KPiA+ID4+IFRvOiBX
ZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiA+PiBDYzogeGVuLWRldmVsQGxpc3RzLnhl
bnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOw0KPiA+IGp1bGllbkB4ZW4ub3Jn
Ow0KPiA+ID4+IGpiZXVsaWNoQHN1c2UuY29tOyBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5N
YXJxdWlzQGFybS5jb20+DQo+ID4gPj4gU3ViamVjdDogUmU6IFtYRU4gUkZDIFBBVENIIDIzLzQw
XSB4ZW4vYXJtOiBpbnRyb2R1Y2UgYSBoZWxwZXIgdG8NCj4gcGFyc2UNCj4gPiA+PiBkZXZpY2Ug
dHJlZSBtZW1vcnkgbm9kZQ0KPiA+ID4+DQo+ID4gPj4gT24gV2VkLCAxMSBBdWcgMjAyMSwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gPj4+IE1lbW9yeSBibG9ja3MnIE5VTUEgSUQgaW5mb3JtYXRpb24g
aXMgc3RvcmVkIGluIGRldmljZSB0cmVlJ3MNCj4gPiA+Pj4gbWVtb3J5IG5vZGVzIGFzICJudW1h
LW5vZGUtaWQiLiBXZSBuZWVkIGEgbmV3IGhlbHBlciB0byBwYXJzZQ0KPiA+ID4+PiBhbmQgdmVy
aWZ5IHRoaXMgSUQgZnJvbSBtZW1vcnkgbm9kZXMuDQo+ID4gPj4+DQo+ID4gPj4+IEluIG9yZGVy
IHRvIHN1cHBvcnQgbWVtb3J5IGFmZmluaXR5IGluIGxhdGVyIHVzZSwgdGhlIHZhbGlkDQo+ID4g
Pj4+IG1lbW9yeSByYW5nZXMgYW5kIE5VTUEgSUQgd2lsbCBiZSBzYXZlZCB0byB0YWJsZXMuDQo+
ID4gPj4+DQo+ID4gPj4+IFNpZ25lZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29t
Pg0KPiA+ID4+PiAtLS0NCj4gPiA+Pj4gICB4ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5j
IHwgMTMwDQo+ID4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4gPiA+Pj4gICAx
IGZpbGUgY2hhbmdlZCwgMTMwIGluc2VydGlvbnMoKykNCj4gPiA+Pj4NCj4gPiA+Pj4gZGlmZiAt
LWdpdCBhL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+PiBiL3hlbi9hcmNo
L2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gPiA+Pj4gaW5kZXggMzdjYzU2YWNmMy4uYmJlMDgx
ZGNkMSAxMDA2NDQNCj4gPiA+Pj4gLS0tIGEveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUu
Yw0KPiA+ID4+PiArKysgYi94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+ID4gPj4+
IEBAIC0yMCwxMSArMjAsMTMgQEANCj4gPiA+Pj4gICAjaW5jbHVkZSA8eGVuL2luaXQuaD4NCj4g
PiA+Pj4gICAjaW5jbHVkZSA8eGVuL25vZGVtYXNrLmg+DQo+ID4gPj4+ICAgI2luY2x1ZGUgPHhl
bi9udW1hLmg+DQo+ID4gPj4+ICsjaW5jbHVkZSA8eGVuL2xpYmZkdC9saWJmZHQuaD4NCj4gPiA+
Pj4gICAjaW5jbHVkZSA8eGVuL2RldmljZV90cmVlLmg+DQo+ID4gPj4+ICAgI2luY2x1ZGUgPGFz
bS9zZXR1cC5oPg0KPiA+ID4+Pg0KPiA+ID4+PiAgIHM4IGRldmljZV90cmVlX251bWEgPSAwOw0K
PiA+ID4+PiAgIHN0YXRpYyBub2RlbWFza190IHByb2Nlc3Nvcl9ub2Rlc19wYXJzZWQgX19pbml0
ZGF0YTsNCj4gPiA+Pj4gK3N0YXRpYyBub2RlbWFza190IG1lbW9yeV9ub2Rlc19wYXJzZWQgX19p
bml0ZGF0YTsNCj4gPiA+Pj4NCj4gPiA+Pj4gICBzdGF0aWMgaW50IHNyYXRfZGlzYWJsZWQodm9p
ZCkNCj4gPiA+Pj4gICB7DQo+ID4gPj4+IEBAIC01NSw2ICs1Nyw3OSBAQCBzdGF0aWMgaW50IF9f
aW5pdA0KPiA+ID4+IGR0Yl9udW1hX3Byb2Nlc3Nvcl9hZmZpbml0eV9pbml0KG5vZGVpZF90IG5v
ZGUpDQo+ID4gPj4+ICAgICAgIHJldHVybiAwOw0KPiA+ID4+PiAgIH0NCj4gPiA+Pj4NCj4gPiA+
Pj4gKy8qIENhbGxiYWNrIGZvciBwYXJzaW5nIG9mIHRoZSBtZW1vcnkgcmVnaW9ucyBhZmZpbml0
eSAqLw0KPiA+ID4+PiArc3RhdGljIGludCBfX2luaXQgZHRiX251bWFfbWVtb3J5X2FmZmluaXR5
X2luaXQobm9kZWlkX3Qgbm9kZSwNCj4gPiA+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcGFkZHJfdCBzdGFydCwgcGFkZHJfdCBzaXplKQ0KPiA+ID4+PiArew0KPiA+ID4+PiAr
ICAgIHN0cnVjdCBub2RlICpuZDsNCj4gPiA+Pj4gKyAgICBwYWRkcl90IGVuZDsNCj4gPiA+Pj4g
KyAgICBpbnQgaTsNCj4gPiA+Pj4gKw0KPiA+ID4+PiArICAgIGlmICggc3JhdF9kaXNhYmxlZCgp
ICkNCj4gPiA+Pj4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gPj4+ICsNCj4gPiA+Pj4g
KyAgICBlbmQgPSBzdGFydCArIHNpemU7DQo+ID4gPj4+ICsgICAgaWYgKCBudW1fbm9kZV9tZW1i
bGtzID49IE5SX05PREVfTUVNQkxLUyApDQo+ID4gPj4+ICsgICAgew0KPiA+ID4+PiArICAgICAg
ICBkcHJpbnRrKFhFTkxPR19XQVJOSU5HLA0KPiA+ID4+PiArICAgICAgICAgICAgICAgICJUb28g
bWFueSBudW1hIGVudHJ5LCB0cnkgYmlnZ2VyIE5SX05PREVfTUVNQkxLUw0KPiBcbiIpOw0KPiA+
ID4+PiArICAgICAgICBiYWRfc3JhdCgpOw0KPiA+ID4+PiArICAgICAgICByZXR1cm4gLUVJTlZB
TDsNCj4gPiA+Pj4gKyAgICB9DQo+ID4gPj4+ICsNCj4gPiA+Pj4gKyAgICAvKiBJdCBpcyBmaW5l
IHRvIGFkZCB0aGlzIGFyZWEgdG8gdGhlIG5vZGVzIGRhdGEgaXQgd2lsbCBiZQ0KPiB1c2VkDQo+
ID4gPj4gbGF0ZXIgKi8NCj4gPiA+Pj4gKyAgICBpID0gY29uZmxpY3RpbmdfbWVtYmxrcyhzdGFy
dCwgZW5kKTsNCj4gPiA+Pj4gKyAgICAvKiBObyBjb25mbGljdGluZyBtZW1vcnkgYmxvY2ssIHdl
IGNhbiBzYXZlIGl0IGZvciBsYXRlciB1c2FnZQ0KPiAqLzsNCj4gPiA+Pj4gKyAgICBpZiAoIGkg
PCAwICkNCj4gPiA+Pj4gKyAgICAgICAgZ290byBzYXZlX21lbWJsazsNCj4gPiA+Pj4gKw0KPiA+
ID4+PiArICAgIGlmICggbWVtYmxrX25vZGVpZFtpXSA9PSBub2RlICkgew0KPiA+ID4+PiArICAg
ICAgICAvKg0KPiA+ID4+PiArICAgICAgICAgKiBPdmVybGFwcyB3aXRoIG90aGVyIG1lbWJsayBp
biB0aGUgc2FtZSBub2RlLCB3YXJuaW5nDQo+IGhlcmUuDQo+ID4gPj4+ICsgICAgICAgICAqIFRo
aXMgbWVtYmxrIHdpbGwgYmUgbWVyZ2VkIHdpdGggY29uZmxpY3RlZCBtZW1ibGsgbGF0ZXIuDQo+
ID4gPj4+ICsgICAgICAgICAqLw0KPiA+ID4+PiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5J
TkcNCj4gPiA+Pj4gKyAgICAgICAgICAgICAgICJEVDogTlVNQSBOT0RFICV1ICglIlBSSXg2NA0K
PiA+ID4+PiArICAgICAgICAgICAgICAgIi0lIlBSSXg2NCIpIG92ZXJsYXBzIHdpdGggaXRzZWxm
ICglIlBSSXg2NCItDQo+ID4gPj4gJSJQUkl4NjQiKVxuIiwNCj4gPiA+Pj4gKyAgICAgICAgICAg
ICAgIG5vZGUsIHN0YXJ0LCBlbmQsDQo+ID4gPj4+ICsgICAgICAgICAgICAgICBub2RlX21lbWJs
a19yYW5nZVtpXS5zdGFydCwNCj4gbm9kZV9tZW1ibGtfcmFuZ2VbaV0uZW5kKTsNCj4gPiA+Pj4g
KyAgICB9IGVsc2Ugew0KPiA+ID4+PiArICAgICAgICAvKg0KPiA+ID4+PiArICAgICAgICAgKiBD
b25mbGljdCB3aXRoIG1lbWJsayBpbiBvdGhlciBub2RlLCB0aGlzIGlzIGFuIGVycm9yLg0KPiA+
ID4+PiArICAgICAgICAgKiBUaGUgTlVNQSBpbmZvcm1hdGlvbiBpcyBpbnZhbGlkLCBOVU1BIHdp
bGwgYmUgdHVybiBvZmYuDQo+ID4gPj4+ICsgICAgICAgICAqLw0KPiA+ID4+PiArICAgICAgICBw
cmludGsoWEVOTE9HX0VSUg0KPiA+ID4+PiArICAgICAgICAgICAgICAgIkRUOiBOVU1BIE5PREUg
JXUgKCUiUFJJeDY0Ii0lIg0KPiA+ID4+PiArICAgICAgICAgICAgICAgUFJJeDY0Iikgb3Zlcmxh
cHMgd2l0aCBOT0RFICV1ICglIlBSSXg2NCItDQo+ID4gJSJQUkl4NjQiKVxuIiwNCj4gPiA+Pj4g
KyAgICAgICAgICAgICAgIG5vZGUsIHN0YXJ0LCBlbmQsIG1lbWJsa19ub2RlaWRbaV0sDQo+ID4g
Pj4+ICsgICAgICAgICAgICAgICBub2RlX21lbWJsa19yYW5nZVtpXS5zdGFydCwNCj4gbm9kZV9t
ZW1ibGtfcmFuZ2VbaV0uZW5kKTsNCj4gPiA+Pj4gKyAgICAgICAgYmFkX3NyYXQoKTsNCj4gPiA+
Pj4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gPj4+ICsgICAgfQ0KPiA+ID4+PiArDQo+
ID4gPj4+ICtzYXZlX21lbWJsazoNCj4gPiA+Pj4gKyAgICBuZCA9ICZub2Rlc1tub2RlXTsNCj4g
PiA+Pj4gKyAgICBpZiAoICFub2RlX3Rlc3RfYW5kX3NldChub2RlLCBtZW1vcnlfbm9kZXNfcGFy
c2VkKSApIHsNCj4gPiA+Pj4gKyAgICAgICAgbmQtPnN0YXJ0ID0gc3RhcnQ7DQo+ID4gPj4+ICsg
ICAgICAgIG5kLT5lbmQgPSBlbmQ7DQo+ID4gPj4+ICsgICAgfSBlbHNlIHsNCj4gPiA+Pj4gKyAg
ICAgICAgaWYgKCBzdGFydCA8IG5kLT5zdGFydCApDQo+ID4gPj4+ICsgICAgICAgICAgICBuZC0+
c3RhcnQgPSBzdGFydDsNCj4gPiA+Pj4gKyAgICAgICAgaWYgKCBuZC0+ZW5kIDwgZW5kICkNCj4g
PiA+Pj4gKyAgICAgICAgICAgIG5kLT5lbmQgPSBlbmQ7DQo+ID4gPj4+ICsgICAgfQ0KPiA+ID4+
PiArDQo+ID4gPj4+ICsgICAgcHJpbnRrKFhFTkxPR19JTkZPICJEVDogTlVNQSBub2RlICV1ICUi
UFJJeDY0Ii0lIlBSSXg2NCJcbiIsDQo+ID4gPj4+ICsgICAgICAgICAgIG5vZGUsIHN0YXJ0LCBl
bmQpOw0KPiA+ID4+PiArDQo+ID4gPj4+ICsgICAgbm9kZV9tZW1ibGtfcmFuZ2VbbnVtX25vZGVf
bWVtYmxrc10uc3RhcnQgPSBzdGFydDsNCj4gPiA+Pj4gKyAgICBub2RlX21lbWJsa19yYW5nZVtu
dW1fbm9kZV9tZW1ibGtzXS5lbmQgPSBlbmQ7DQo+ID4gPj4+ICsgICAgbWVtYmxrX25vZGVpZFtu
dW1fbm9kZV9tZW1ibGtzXSA9IG5vZGU7DQo+ID4gPj4+ICsgICAgbnVtX25vZGVfbWVtYmxrcysr
Ow0KPiA+ID4+DQo+ID4gPj4NCj4gPiA+PiBJcyBpdCBwb3NzaWJsZSB0byBoYXZlIG5vbi1jb250
aWdvdXMgcmFuZ2VzIG9mIG1lbW9yeSBmb3IgYSBzaW5nbGUNCj4gTlVNQQ0KPiA+ID4+IG5vZGU/
DQo+ID4gPj4NCj4gPiA+PiBMb29raW5nIGF0IHRoZSBEVCBiaW5kaW5ncyBhbmQgTGludXggaW1w
bGVtZW50YXRpb24sIGl0IHNlZW1zDQo+IHBvc3NpYmxlLg0KPiA+ID4+IEhlcmUsIGl0IHNlZW1z
IHRoYXQgbm9kZV9tZW1ibGtfcmFuZ2UvbWVtYmxrX25vZGVpZCBjb3VsZCBoYW5kbGUgaXQsDQo+
ID4gPj4gYnV0IG5vZGVzIGNvdWxkbid0Lg0KPiA+ID4NCj4gPiA+IFllcywgeW91J3JlIHJpZ2h0
LiBJIGNvcGllZCB0aGlzIGNvZGUgZm9yIHg4NiBBQ1BJIE5VTUEuIERvZXMgQUNQSQ0KPiBhbGxv
dw0KPiA+ID4gbm9uLWNvbnRpZ3VvdXMgcmFuZ2VzIG9mIG1lbW9yeSBmb3IgYSBzaW5nbGUgTlVN
QSBub2RlIHRvbz8NCj4gPg0KPiA+IEkgY291bGRuJ3QgZmluZCBhbnkgcmVzdHJpY3Rpb24gZm9y
IEFDUEkuIEFsdGhvdWdoLCBJIG9ubHkgYnJpZWZseQ0KPiA+IGxvb2tlZCBhdCB0aGUgc3BlYy4N
Cj4gPg0KPiA+ID4gSWYgeWVzLCBJIHRoaW5rDQo+ID4gPiB0aGlzIHdpbGwgYWZmZWN0IHg4NiBB
Q1BJIE5VTUEgdG9vLiBJbiBuZXh0IHZlcnNpb24sIHdlIHBsYW4gdG8gbWVyZ2UNCj4gPiA+IGR0
Yl9udW1hX21lbW9yeV9hZmZpbml0eV9pbml0IGFuZCBhY3BpX251bWFfbWVtb3J5X2FmZmluaXR5
X2luaXQgaW50bw0KPiBhDQo+ID4gPiBuZXV0cmFsIGZ1bmN0aW9uLiBTbyB3ZSBjYW4gZml4IHRo
ZW0gYXQgdGhlIHNhbWUgdGltZS4NCj4gPiA+DQo+ID4gPiBJZiBub3QsIG1heWJlIHdlIGhhdmUg
dG8ga2VlcCB0aGUgZGl2ZXJzaXR5IGZvciBkdGIgYW5kIEFDUEkgaGVyZS4NCj4gPg0KPiA+IEkg
YW0gbm90IGVudGlyZWx5IHN1cmUgd2hhdCB5b3UgbWVhbi4gQXJlIHlvdSBzYXlpbmcgaWYgQUNQ
SSBkb2Vzbid0DQo+ID4gYWxsb3cgbm9uLWNvbnRpZ3VvdXMgcmFuZ2VzIG9mIG1lbW9yeSwgdGhl
biB3ZSBzaG91bGQga2VlcCB0aGUNCj4gPiBpbXBsZW1lbnRhdGlvbiBzZXBhcmF0ZWQ/DQo+ID4N
Cj4gPiBJZiBzbywgdGhlbiBJIGRpc2FncmVlIHdpdGggdGhhdC4gSXQgaXMgZmluZSB0byBoYXZl
IGNvZGUgdGhhdCBzdXBwb3J0cw0KPiA+IG1vcmUgdGhhbiB3aGF0IGEgZmlybXdhcmUgdGFibGUg
c3VwcG9ydHMuIFRoZSBtYWluIGJlbmVmaXQgaXMgbGVzcyBjb2RlDQo+ID4gYW5kIHRoZXJlZm9y
ZSBsZXNzIGxvbmcgdGVybSBtYWludGVuYW5jZSAod2l0aCB0aGUgY3VycmVudCBzb2x1dGlvbiB3
ZQ0KPiA+IHdvdWxkIG5lZWQgdG8gY2hlY2sgYm90aCB0aGUgQUNQSSBhbmQgRFQgaW1wbGVtZW50
YXRpb24gaWYgdGhlcmUgaXMgYQ0KPiA+IGJ1ZyBpbiBvbmUpLg0KPiA+DQo+IA0KPiBZZXMsIEkg
YWdyZWUuDQo+IA0KDQpJIGFtIGxvb2tpbmcgZm9yIHNvbWUgbWV0aG9kcyB0byBhZGRyZXNzIHRo
aXMgY29tbWVudC4gQ3VycmVudCAibm9kZXMiDQpoYXMgbm90IGNvbnNpZGVyZWQgdGhlIHNpdHVh
dGlvbiBvZiBtZW1vcnkgYWRkcmVzc2VzIG9mIGRpZmZlcmVudCBOVU1BDQpub2RlcyBjYW4gYmUg
aW50ZXJsZWF2ZWQuDQoNClRoaXMgY29kZSBleGlzdHMgaW4geDg2IE5VTUEgaW1wbGVtZW50YXRp
b24uIEkgdGhpbmsgaXQgbWF5IGJlIGJhc2VkIG9uDQpvbmUgZWFybHkgdmVyc2lvbiBvZiBMaW51
eCB4ODYgTlVNQSBpbXBsZW1lbnRhdGlvbi4gSW4gcmVjZW50IExpbnV4DQpjb2RlLCBib3RoIEFD
UEkvbnVtYS9zcmF0LmNbMV0gYW5kIHg4NiBOVU1BIGNvZGVbMl0gYXJlIG5vdCB1c2luZw0KIm5v
ZGVzIiB0byByZWNvcmQgTlVNQSBtZW1vcnkgYWRkcmVzcyBib3VuZGFyeS4gVGhleSBkb24ndCBk
ZXBlbmQNCm9uICJub2RlcyIgdG8gZG8gc2FuaXR5IGNoZWNrLg0KDQpUbyBmaXggaXQsIHdlJ2Qg
YmV0dGVyIHRvIHVwZ3JhZGUgdGhlIHg4NiBOVU1BIGRyaXZlci4gSXQgd2lsbCBtYWtlDQphIGdy
ZWF0IGFmZmVjdCBmb3IgWGVuLXg4Ni4gQW5kIEkgdGhpbmsgaXQgbWlnaHQgb3V0IG9mIHRoaXMg
c2VyaWVzDQpzY29wZS4gQ2FuIHdlIGNyZWF0ZSBhbm90aGVyIHRocmVhZCB0byBkaXNjdXNzIGFi
b3V0IGl0Pw0KDQpPciBjb3VsZCB5b3UgZ2l2ZSBtZSBzdWdnZXN0aW9ucyB0aGF0IHdlIGNhbiB1
c2Ugc29tZSBzaW1wbGUgd2F5cw0KdG8gZml4IGl0Pw0KDQpBbHNvLCBvbiBMaW51eCwgTlVNQSBp
bXBsZW1lbnRhdGlvbnMgZm9yIHg4NiBhcmUgZGlmZmVyZW50IGZyb20gQXJtNjQNCmFuZCBSSVND
LVYgaW1wbGVtZW50YXRpb25zLlszXQ0KDQpbMV0gaHR0cHM6Ly9naXRodWIuY29tL3RvcnZhbGRz
L2xpbnV4L2Jsb2IvbWFzdGVyL2RyaXZlcnMvYWNwaS9udW1hL3NyYXQuYw0KWzJdIGh0dHBzOi8v
Z2l0aHViLmNvbS90b3J2YWxkcy9saW51eC9ibG9iL21hc3Rlci9hcmNoL3g4Ni9tbS9udW1hLmMN
ClszXSBodHRwczovL2dpdGh1Yi5jb20vdG9ydmFsZHMvbGludXgvYmxvYi9tYXN0ZXIvZHJpdmVy
cy9iYXNlL2FyY2hfbnVtYS5jDQoNCj4gPiBDaGVlcnMsDQo+ID4NCj4gPiAtLQ0KPiA+IEp1bGll
biBHcmFsbA0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 07:36:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 07:36:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181551.328710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNs8V-0005ZL-4Q; Wed, 08 Sep 2021 07:36:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181551.328710; Wed, 08 Sep 2021 07:36:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNs8V-0005ZE-1G; Wed, 08 Sep 2021 07:36:47 +0000
Received: by outflank-mailman (input) for mailman id 181551;
 Wed, 08 Sep 2021 07:36:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNs8T-0005Z4-Ms
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 07:36:45 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f760927-54e6-4e74-bf2a-f9169fa9ec97;
 Wed, 08 Sep 2021 07:36:44 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2C5C420002;
 Wed,  8 Sep 2021 07:36:43 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id C194613721;
 Wed,  8 Sep 2021 07:36:42 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id OfMuLQpoOGElRgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 07:36:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f760927-54e6-4e74-bf2a-f9169fa9ec97
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631086603; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Z+WndBSU3C8fZ88DpLGMyx1R2eYI/u6zm2djOHlAuhI=;
	b=Gxv+cR175RWwcar+tgco/LV9K7gZvM79EFqyhBr+f3UXoCEbLE/QUAkRS6fNiZ7lBQkSJu
	zF92+gHGNw1F6qIchk8eh/7NlH3reksuw4VWG2W/5DQu0IkjV76wlvPNl12TKkDJUr4LqG
	HjIAEFm6aGO7BCxqzOKpy69FZalwFK8=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	stable@vger.kernel.org,
	Sander Eikelenboom <linux@eikelenboom.it>
Subject: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv guests
Date: Wed,  8 Sep 2021 09:36:40 +0200
Message-Id: <20210908073640.11299-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page
table entries") introduced a regression when running as Xen PV guest.

Today pmd/pud_poplulate() for Xen PV assumes that the PFN inserted is
referencing a not yet used page table. In case of move_normal_pmd/pud()
this is not true, resulting in WARN splats like:

[34321.304270] ------------[ cut here ]------------
[34321.304277] WARNING: CPU: 0 PID: 23628 at arch/x86/xen/multicalls.c:102 xen_mc_flush+0x176/0x1a0
[34321.304288] Modules linked in:
[34321.304291] CPU: 0 PID: 23628 Comm: apt-get Not tainted 5.14.1-20210906-doflr-mac80211debug+ #1
[34321.304294] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1 09/13/2010
[34321.304296] RIP: e030:xen_mc_flush+0x176/0x1a0
[34321.304300] Code: 89 45 18 48 c1 e9 3f 48 89 ce e9 20 ff ff ff e8 60 03 00 00 66 90 5b 5d 41 5c 41 5d c3 48 c7 45 18 ea ff ff ff be 01 00 00 00 <0f> 0b 8b 55 00 48 c7 c7 10 97 aa 82 31 db 49 c7 c5 38 97 aa 82 65
[34321.304303] RSP: e02b:ffffc90000a97c90 EFLAGS: 00010002
[34321.304305] RAX: ffff88807d416398 RBX: ffff88807d416350 RCX: ffff88807d416398
[34321.304306] RDX: 0000000000000001 RSI: 0000000000000001 RDI: deadbeefdeadf00d
[34321.304308] RBP: ffff88807d416300 R08: aaaaaaaaaaaaaaaa R09: ffff888006160cc0
[34321.304309] R10: deadbeefdeadf00d R11: ffffea000026a600 R12: 0000000000000000
[34321.304310] R13: ffff888012f6b000 R14: 0000000012f6b000 R15: 0000000000000001
[34321.304320] FS:  00007f5071177800(0000) GS:ffff88807d400000(0000) knlGS:0000000000000000
[34321.304322] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
[34321.304323] CR2: 00007f506f542000 CR3: 00000000160cc000 CR4: 0000000000000660
[34321.304326] Call Trace:
[34321.304331]  xen_alloc_pte+0x294/0x320
[34321.304334]  move_pgt_entry+0x165/0x4b0
[34321.304339]  move_page_tables+0x6fa/0x8d0
[34321.304342]  move_vma.isra.44+0x138/0x500
[34321.304345]  __x64_sys_mremap+0x296/0x410
[34321.304348]  do_syscall_64+0x3a/0x80
[34321.304352]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[34321.304355] RIP: 0033:0x7f507196301a
[34321.304358] Code: 73 01 c3 48 8b 0d 76 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 19 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 46 0e 0c 00 f7 d8 64 89 01 48
[34321.304360] RSP: 002b:00007ffda1eecd38 EFLAGS: 00000246 ORIG_RAX: 0000000000000019
[34321.304362] RAX: ffffffffffffffda RBX: 000056205f950f30 RCX: 00007f507196301a
[34321.304363] RDX: 0000000001a00000 RSI: 0000000001900000 RDI: 00007f506dc56000
[34321.304364] RBP: 0000000001a00000 R08: 0000000000000010 R09: 0000000000000004
[34321.304365] R10: 0000000000000001 R11: 0000000000000246 R12: 00007f506dc56060
[34321.304367] R13: 00007f506dc56000 R14: 00007f506dc56060 R15: 000056205f950f30
[34321.304368] ---[ end trace a19885b78fe8f33e ]---
[34321.304370] 1 of 2 multicall(s) failed: cpu 0
[34321.304371]   call  2: op=12297829382473034410 arg=[aaaaaaaaaaaaaaaa] result=-22

Fix that by modifying xen_alloc_ptpage() to only pin the page table in
case it wasn't pinned already.

Fixes: 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page table entries")
Cc: <stable@vger.kernel.org>
Reported-by: Sander Eikelenboom <linux@eikelenboom.it>
Tested-by: Sander Eikelenboom <linux@eikelenboom.it>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/mmu_pv.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 1df5f01529e5..8d751939c6f3 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1518,14 +1518,17 @@ static inline void xen_alloc_ptpage(struct mm_struct *mm, unsigned long pfn,
 	if (pinned) {
 		struct page *page = pfn_to_page(pfn);
 
-		if (static_branch_likely(&xen_struct_pages_ready))
+		pinned = false;
+		if (static_branch_likely(&xen_struct_pages_ready)) {
+			pinned = PagePinned(page);
 			SetPagePinned(page);
+		}
 
 		xen_mc_batch();
 
 		__set_pfn_prot(pfn, PAGE_KERNEL_RO);
 
-		if (level == PT_PTE && USE_SPLIT_PTE_PTLOCKS)
+		if (level == PT_PTE && USE_SPLIT_PTE_PTLOCKS && !pinned)
 			__pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE, pfn);
 
 		xen_mc_issue(PARAVIRT_LAZY_MMU);
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 08:01:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 08:01:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181564.328721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNsWg-0001Se-Ew; Wed, 08 Sep 2021 08:01:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181564.328721; Wed, 08 Sep 2021 08:01:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNsWg-0001SX-Bq; Wed, 08 Sep 2021 08:01:46 +0000
Received: by outflank-mailman (input) for mailman id 181564;
 Wed, 08 Sep 2021 08:01:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6z62=N6=amazon.co.uk=prvs=8782002cc=ahmeddan@srs-us1.protection.inumbo.net>)
 id 1mNsWe-0001SR-DO
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 08:01:44 +0000
Received: from smtp-fw-9102.amazon.com (unknown [207.171.184.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 011f09b2-107b-11ec-b147-12813bfff9fa;
 Wed, 08 Sep 2021 08:01:43 +0000 (UTC)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 email-inbound-relay-1a-e34f1ddc.us-east-1.amazon.com) ([10.25.36.214])
 by smtp-border-fw-9102.sea19.amazon.com with ESMTP; 08 Sep 2021 08:01:36 +0000
Received: from EX13D37EUA002.ant.amazon.com
 (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38])
 by email-inbound-relay-1a-e34f1ddc.us-east-1.amazon.com (Postfix) with ESMTPS
 id 14C1CA1A39; Wed,  8 Sep 2021 08:01:33 +0000 (UTC)
Received: from EX13D24EUA001.ant.amazon.com (10.43.165.233) by
 EX13D37EUA002.ant.amazon.com (10.43.165.200) with Microsoft SMTP Server (TLS)
 id 15.0.1497.23; Wed, 8 Sep 2021 08:01:32 +0000
Received: from EX13D24EUA001.ant.amazon.com ([10.43.165.233]) by
 EX13D24EUA001.ant.amazon.com ([10.43.165.233]) with mapi id 15.00.1497.023;
 Wed, 8 Sep 2021 08:01:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 011f09b2-107b-11ec-b147-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.co.uk; i=@amazon.co.uk; q=dns/txt;
  s=amazon201209; t=1631088104; x=1662624104;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-id:content-transfer-encoding:
   mime-version;
  bh=NNiwhOsw2/lA3gYN93p8XpwsNp5oh1ZqFV8csTQzejw=;
  b=mu1DWzIvAsz7313KTLrdbLv4lb6nRAS4ffnSYp8E6Zkp4IsRzaxeoyKP
   mLbtIIVlQrjKnPf6LHER497CvSoIYVm4DrP5l1XyILeHFN42RJsgtMQwk
   Y1ntYdr5qLeyUmkuqZfh4iIId6PbMiPXsqy1zJoMjEN1PYojPfi5C08iK
   4=;
X-IronPort-AV: E=Sophos;i="5.85,277,1624320000"; 
   d="scan'208";a="158343262"
From: "Ahmed, Daniele" <ahmeddan@amazon.co.uk>
To: Dario Faggioli <dfaggioli@suse.com>, "julien@xen.org" <julien@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>, "Pohlack, Martin"
	<mpohlack@amazon.de>, "Grall, Julien" <jgrall@amazon.co.uk>, "Doebel, Bjoern"
	<doebel@amazon.de>
Subject: Re: NULL scheduler DoS
Thread-Topic: NULL scheduler DoS
Thread-Index: AQHXpIe8OXmLgpZp00yyIO6D9XnksA==
Date: Wed, 8 Sep 2021 08:01:32 +0000
Message-ID: <562980DD-ABAB-4082-A565-2E07F71292E5@amazon.com>
References: <ED25BE5E-D695-4763-B97A-78D6040E2341@amazon.com>
 <8193a685-3ab1-9407-75f3-e335ea4406e4@xen.org>
 <f3082b8c-1388-c92b-6175-e6d8b9d4cab4@xen.org>
 <6b5e28f427ccd89505a4448f2b9e5be992a08292.camel@suse.com>
In-Reply-To: <6b5e28f427ccd89505a4448f2b9e5be992a08292.camel@suse.com>
Accept-Language: en-US
Content-Language: en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.43.166.178]
Content-Type: text/plain; charset="utf-8"
Content-ID: <C1E2687B92B3324CB802991D4E093699@amazon.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Precedence: Bulk

SGksIHNvcnJ5IGZvciB0aGUgbGF0ZSByZXBseS4NCg0KSGVyZSBpcyB0aGUgZnVsbCBsb2cgZnJv
bSB0aGUgc2VyaWFsIGNvbnNvbGU6IGh0dHBzOi8vcGFzdGViaW4uY29tL3Jhdy9nOUVGdGszeQ0K
DQpUaGlzIHN0aWxsIGhhcHBlbnMgb24gdGhlIGN1cnJlbnQgdW5zdGFibGUgYXMgb2YgdG9kYXku
DQoNClRoYW5rIHlvdQ0KRGFuaWVsZQ0KDQrvu79PbiAxMS8wOC8yMDIxLCAxMjowOSwgIkRhcmlv
IEZhZ2dpb2xpIiA8ZGZhZ2dpb2xpQHN1c2UuY29tPiB3cm90ZToNCg0KICAgIE9uIE1vbiwgMjAy
MS0wOC0wOSBhdCAyMTozOCArMDEwMCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KICAgID4gT24gMDkv
MDgvMjAyMSAxODozNSwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KICAgID4gPiANCiAgICA+ID4gVGhp
cyBpbXBsaWVzIHRoYXQgYSBwQ1BVIG1heSB0ZW1wb3JhcmlseSBiZSBhc3NpZ25lZCB0byB0d28g
dkNQVXMNCiAgICA+ID4gYW5kIHdlIA0KICAgID4gPiBleHBlY3QgdG8gYmUgZml4ZWQgdXAgYWZ0
ZXJ3YXJkcy4gSG93ZXZlciwgYSBkb21haW4gbWF5IGJlDQogICAgPiA+IGRlc3Ryb3llZCANCiAg
ICA+ID4gYmVmb3JlIHRoaXMgaXMgaGFwcGVuaW5nLg0KICAgID4gPiANCiAgICA+ID4gU28gaXQg
bG9va3MgbGlrZSB0aGF0IHVuaXRfZGVhc3NpZ24oKSBpcyBub3QgYWJsZSB0byBjb3BlIHdpdGgg
dGhpcw0KICAgID4gPiBjYXNlLiBGcm9tIGEgYnJpZWYgbG9vaywgSSB0aGluayB3ZSBtYXkgd2Fu
dCB0byBjaGVjayBpZiB0aGUgcENQVQ0KICAgID4gPiBpcyBpbiANCiAgICA+ID4gdGhlIHdhaXQg
bGlzdC4gSWYgaXQgaXMsIHRoZW4gd2Ugc2hvdWxkIGJhaWwgb3V0Lg0KICAgID4gDQogICAgPiBB
Y3R1YWxseSwgSSB3YXMgd3JvbmcuIEl0IGxvb2tzIGxpa2UgbnVsbF91bml0X3JlbW92ZSgpIGlz
IGFscmVhZHkgDQogICAgPiBjaGVja2luZyB0aGlzIGNvbmRpdGlvbi4gQWxzbywgdGhlIHZDUFUg
c2hvdWxkIGJlIG9mZmxpbmUgKGFuZCB0aGUNCiAgICA+IHVuaXQgDQogICAgPiBhcyB3ZWxsKSBi
ZWNhdXNlIHRoZXkgaGF2ZW4ndCBjb21lIG9ubGluZSB5ZXQ6DQogICAgPiANCiAgICBUaGF0J3Mg
d2hhdCBpcyBjdXJyZW50bHkgcHV6emxpbmcgbWUuDQoNCiAgICBJIG1lYW4sIHRoZSBjcmFzaCBo
YXBwZW5zIGluIHVuaXRfZGVhc3NpZ24oKSwgY2FsbGVkIGJ5DQogICAgbnVsbF91bml0X3JlbW92
ZSgpLCBjYWxsZWQgYnkgc2NoZWRfZGVzdHJveV92Y3B1KCkuDQoNCiAgICBBbmQgSSBhZ3JlZSB0
aGF0IHRoZSB1bml0IHNob3VsZCBiZSBvZmZsaW5lLCBidXQgbnVsbF91bml0X3JlbW92ZSgpDQog
ICAgY2FsbHMgdW5pdF9kZWFzc2lnbigpIG9ubHkgaWYgdGhlIHVuaXQgKmlzKiBvbmxpbmUsIHNv
Li4uIFdoYXQncyBnb2luZw0KICAgIG9uPyA6LS8NCg0KICAgIFJlZ2FyZHMNCiAgICAtLSANCiAg
ICBEYXJpbyBGYWdnaW9saSwgUGguRA0KICAgIGh0dHA6Ly9hYm91dC5tZS9kYXJpby5mYWdnaW9s
aQ0KICAgIFZpcnR1YWxpemF0aW9uIFNvZnR3YXJlIEVuZ2luZWVyDQogICAgU1VTRSBMYWJzLCBT
VVNFIGh0dHBzOi8vd3d3LnN1c2UuY29tLw0KICAgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgICA8PFRoaXMgaGFw
cGVucyBiZWNhdXNlIF9JXyBjaG9vc2UgaXQgdG8gaGFwcGVuIT4+IChSYWlzdGxpbiBNYWplcmUp
DQoNCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 08:20:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 08:20:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181569.328731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNsoL-0003IY-1b; Wed, 08 Sep 2021 08:20:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181569.328731; Wed, 08 Sep 2021 08:20:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNsoK-0003IR-Ur; Wed, 08 Sep 2021 08:20:00 +0000
Received: by outflank-mailman (input) for mailman id 181569;
 Wed, 08 Sep 2021 08:19:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=azY0=N6=citrix.com=Jane.Malalane@srs-us1.protection.inumbo.net>)
 id 1mNsoJ-0003IL-N0
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 08:19:59 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16fe695d-0a62-4a99-9eaa-9aa78ffc0697;
 Wed, 08 Sep 2021 08:19:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16fe695d-0a62-4a99-9eaa-9aa78ffc0697
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631089198;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=EsMNMCKTezC7j6gNcKVFe/+UQsY5QrZ/q/UgvOYDVhU=;
  b=ZteyLYJUmPErJ/9eyWCp3Pyag95/+BAdB1WTSKma1pZZzbmqXp5TDeug
   ohCdBl2xAh6ZVi5b/PLoKGdHtYlOKYCWnuv+UEF9rlWayBfy3kwS7RfJV
   h3r4Q0xLakEhboSP35TeuduLYtujBBwarCTvRSn2uhtdH/AS5gi19trQU
   w=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: WkLE+G5BwbTYK+RdRMVIg/26UTHktzIcSXsqOe2TAMVOPeO7OuOEUSFZ9nhssQ5V1W0LLfqKcf
 g5g0mhBoHtPv0t7o4ZJ0vrZPSjdoZm/r2Q6Cfb/hH/9wAP+5jG95P4Bzv56qQHGZWVmeG6dSwt
 BmIA4A4qvuyBTg5Seysuwazfmw0vp9cKedHis1wQpmvme7S6+4NrgZcrSUlHpS/ttzDdRC75Mp
 uihwgdioZSmlh9dRWEDIGPwgc+wAflGAqKW2s+DvE7b//UHM53zBcZKeCQ4qn2vd0sS4Dl3wct
 /X+KJkSAR9hxvw3WxFZ506+o
X-SBRS: 5.1
X-MesageID: 51829769
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:1J0Cu6wGeGnLjTrmK2RrKrPwFL1zdoMgy1knxilNoRw8SKKlfq
 eV7Y0mPH7P+VAssR4b+exoVJPtfZqYz+8R3WBzB8bEYOCFghrKEGgK1+KLqFeMJ8S9zJ846U
 4JSdkHNDSaNzlHZKjBjzVQa+xQouW6zA==
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="51829769"
From: Jane Malalane <jane.malalane@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Jane Malalane <jane.malalane@citrix.com>, Wei Liu <wl@xen.org>, "Jan
 Beulich" <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Pu Wen
	<puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v2 2/2] x86/cpuid: Detect null segment behaviour on Zen2 CPUs
Date: Wed, 8 Sep 2021 09:19:45 +0100
Message-ID: <20210908081945.11047-1-jane.malalane@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
References: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Zen2 CPUs actually have this behaviour, but the CPUID bit couldn't be
introduced into Zen2 due to a lack of leaves. So, it was added in a
new leaf in Zen3. Nonetheless, hypervisors can synthesize the CPUID
bit in software.

So, Xen probes for NSCB (NullSelectorClearsBit) and
synthesizes the bit, if the behaviour is present.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jane Malalane <jane.malalane@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Pu Wen <puwen@hygon.cn>
CC: Andy Lutomirski <luto@kernel.org>

v2:
 * Mark detect_zen2_null_seg_behaviour() with __init
 * Remove model checks
---
 xen/arch/x86/cpu/amd.c           | 18 ++++++++++++++++++
 xen/arch/x86/cpu/cpu.h           |  1 +
 xen/arch/x86/cpu/hygon.c         |  5 +++++
 xen/include/asm-x86/cpufeature.h |  1 +
 4 files changed, 25 insertions(+)

diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 2260eef3aa..cb12861481 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -681,6 +681,19 @@ void amd_init_lfence(struct cpuinfo_x86 *c)
 			  c->x86_capability);
 }
 
+void __init detect_zen2_null_seg_behaviour(void)
+{
+	uint64_t base;
+
+	wrmsrl(MSR_FS_BASE, 1);
+	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
+	rdmsrl(MSR_FS_BASE, base);
+
+	if (base == 0)
+		setup_force_cpu_cap(X86_FEATURE_NSCB);
+
+}
+
 static void init_amd(struct cpuinfo_x86 *c)
 {
 	u32 l, h;
@@ -731,6 +744,11 @@ static void init_amd(struct cpuinfo_x86 *c)
 	else /* Implicily "== 0x10 || >= 0x12" by being 64bit. */
 		amd_init_lfence(c);
 
+	/* Probe for NSCB on Zen2 CPUs when not virtualised */
+	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
+	    c->x86 == 0x17)
+		detect_zen2_null_seg_behaviour();
+
 	/*
 	 * If the user has explicitly chosen to disable Memory Disambiguation
 	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
diff --git a/xen/arch/x86/cpu/cpu.h b/xen/arch/x86/cpu/cpu.h
index 1ac3b2867a..0dd1b762ff 100644
--- a/xen/arch/x86/cpu/cpu.h
+++ b/xen/arch/x86/cpu/cpu.h
@@ -21,3 +21,4 @@ extern bool detect_extended_topology(struct cpuinfo_x86 *c);
 void early_init_amd(struct cpuinfo_x86 *c);
 void amd_log_freq(const struct cpuinfo_x86 *c);
 void amd_init_lfence(struct cpuinfo_x86 *c);
+void detect_zen2_null_seg_behaviour(void);
diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c
index 67e23c5df9..d7a04af2bb 100644
--- a/xen/arch/x86/cpu/hygon.c
+++ b/xen/arch/x86/cpu/hygon.c
@@ -34,6 +34,11 @@ static void init_hygon(struct cpuinfo_x86 *c)
 
 	amd_init_lfence(c);
 
+	/* Probe for NSCB on Zen2 CPUs when not virtualised */
+	if (!cpu_has_hypervisor && !cpu_has_nscb && c == &boot_cpu_data &&
+	    c->x86 == 0x18)
+		detect_zen2_null_seg_behaviour();
+
 	/*
 	 * If the user has explicitly chosen to disable Memory Disambiguation
 	 * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 5f6b83f71c..4faf9bff29 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -146,6 +146,7 @@
 #define cpu_has_cpuid_faulting  boot_cpu_has(X86_FEATURE_CPUID_FAULTING)
 #define cpu_has_aperfmperf      boot_cpu_has(X86_FEATURE_APERFMPERF)
 #define cpu_has_lfence_dispatch boot_cpu_has(X86_FEATURE_LFENCE_DISPATCH)
+#define cpu_has_nscb            boot_cpu_has(X86_FEATURE_NSCB)
 #define cpu_has_xen_lbr         boot_cpu_has(X86_FEATURE_XEN_LBR)
 #define cpu_has_xen_shstk       boot_cpu_has(X86_FEATURE_XEN_SHSTK)
 
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:17:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181575.328743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNthS-0001jY-GG; Wed, 08 Sep 2021 09:16:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181575.328743; Wed, 08 Sep 2021 09:16:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNthS-0001jR-CE; Wed, 08 Sep 2021 09:16:58 +0000
Received: by outflank-mailman (input) for mailman id 181575;
 Wed, 08 Sep 2021 09:16:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNthR-0001jL-76
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:16:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 764b3c30-6b44-47e3-94a6-5f12b8efb4e7;
 Wed, 08 Sep 2021 09:16:54 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-wJcEkFcVP1ebfVIA_sWHvw-2; Wed, 08 Sep 2021 11:16:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 09:16:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 09:16:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0035.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 09:16:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 764b3c30-6b44-47e3-94a6-5f12b8efb4e7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631092613;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=5o3MjP4Xyup2z4+VruHsu4pa+Ez3/ZYNop7+qUPcPWU=;
	b=YR7GJ6xA45Dl+RPi0d5ZvAJoLG8SizbcE0G15r7QMeKdvv7Y0P42s0A7jNDBrrpnrZUSzR
	RB3YIAhpG9ZddfLFmVKg7s/FyxLHvUvOfyZfpAmHL92Yj54kyVPi862R9CaZoGQzv0EiQ/
	KKjzDL6ZUSjYq60AZsyLDJRdt1dF8hY=
X-MC-Unique: wJcEkFcVP1ebfVIA_sWHvw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iENznPPznq4lOx7kKxSZ0R6DH6baKoSPmiBe14QQaJiQmCetzQTroOsgkpRXwnRpAHg4nRG97mJ78KAMzexlyIefbfVB2GVAk0k6/Ng/L51Hqy2IU4aMrhoi0wcpo9Qbjd2COm4Ux4vrEBgEwMkrfhtr8WPe75bZzvt5bQ17axpOel+xKKf1rSROgCgtr8C0dhdYQlWLysEo2Wy8Jqc78JeCkmR0bAkNGE2W3r5W+h6Q2NWq//q6sf7m+/kwrbLwuV+Vy3YlIVGjsid0ioJxpK0h0B/aRs0FpfVHYkKrwFJC0bz0xI3sMWZMlgIBXLEOy2AdRqFPX+pjj4Xd1bfQgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5o3MjP4Xyup2z4+VruHsu4pa+Ez3/ZYNop7+qUPcPWU=;
 b=Gtvc26UtBl36xK/xqvboQlK6LugjpJx+nlHVpZw/VkesP0uaQdZ3GXbspTLilQxP2eTPgXvF3vLo2GKTMGGcpBzI8T7Tig9M8SXYRuaUEpRtEqe38bCyzyXr2JoVxWTNH5iTyNjWtBgfPW8Hdm5/OgVQHqBHLr8PGzDcuRj98rFwdDrjJCxL6WPDvX+UuZtLa9ICUBrKogAJIkpX2txNlTxW1kR/5Am+0qnwsOWa/zjn872jdC07metqq8evut/WdspAgNgCc/Kxxo9aTfhdNH0aSFj4SDBhkKMTo2JXX0w6NFsI+lKh/0lAIpWBJqvLW1qXX/Y4ZSWT3m5SNDaqCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
To: Christoph Hellwig <hch@infradead.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
 <YThey/iyCxi5NUwC@infradead.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0082b78b-3821-9908-d365-3f66cf1e92ce@suse.com>
Date: Wed, 8 Sep 2021 11:16:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YThey/iyCxi5NUwC@infradead.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0035.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 58c068e7-bd38-449e-2459-08d972a963dc
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6383BD57F9654B7F487D365DB3D49@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1122;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Guxq+V8Is6BKCmeCsESNqL3+mxl1B1a2wWpJVzr1gl8atZjNJBvUBvXAivqypvQEZxHP/lrHbmhlay8GPHwGY6Uvj20ZI4z8qbRr+xMijHzK1s0yYkArZVk/9kgM+c88j26L1dTgqtXICM9b4KsuwXcMZD4AncWX0iVWRiv07LVadBBvxjUJ/GyjX66RytpI1/JedJwLcuWlJfsisEP/aYdKOXysZnU+Neq/pTBQD70rifQhPwUQXB3zYqatUnpqiVh635kx8n4hyI1LHlh6gYViK+s28uz+/Z5VmMtcGZcoRIDImxw4gawMgfm1qYXRH1xZD5RCbUCx6l446EW32j0nZMCVtJhzlPwkGGv4hjV7tkvIlgA/Jid8CUIYWV4HFay5qcxWBFOyBY2wF61L1gHSkOrewZj4UCrxbUWbOIcyc/lAsXp16Z6h9LCiCv5AyDJs509Aixmkzl53YhRak4qcz9zlhxuSRD8nTAxNmBt7qgJ9Y+gklh1Wb9D7M/m9N6PGozx2ewast5JggbZPUBgP8VMPv2CsnErpbMa+afwHMIEck30zHp7Ky4Ij4kWwSrf61wk33aNWnSej3TtzyYAqrnBPXlkQPa44s5QqatqVu2naF5voVirFeIJgiHuDTiJefvSqvJ/Ijq7NAprepagdz2NyB57ZZJ0ZAjRKuzD8y1IGRxfp2Ly1EnhbltrL56WgbCnDKvwaU8w64R8tTH88zVhu5nFmdV9TXZGkw+ceyOyvsHuPimA6hCYIBhIC
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(136003)(346002)(396003)(366004)(316002)(16576012)(2906002)(6486002)(478600001)(66556008)(53546011)(38100700002)(83380400001)(186003)(66476007)(66946007)(5660300002)(54906003)(31686004)(6916009)(36756003)(2616005)(4326008)(956004)(86362001)(8676002)(26005)(31696002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NUdKYUlFbHN3NUVNWGRUaGh3YUl5b1ZYRGN3M2pXWmY1bEliYkJUMzRZN29O?=
 =?utf-8?B?MjNUTGx3TEpEcW9ZalVhbmpQVHFKeHgvRHgwTmJaZktnMEp6TDFVTURzT3M2?=
 =?utf-8?B?LzVhbW9nUTRLNzZGUUNyRWdqZnJGaU9GM2FMWWZSeUE1Rlc3RHp3NzBkZDU4?=
 =?utf-8?B?VlFWcGJDQWlraFRXbGhhUDNMSGFJMzUxQVN3MmFQODhOR2tqcHBoMlluNCs4?=
 =?utf-8?B?NXBSRkMrM1B5US9McUpjYUJ6YXQ0VVJCakJVMXZ2RmhGQ3BlOUV1ZnkrUTBr?=
 =?utf-8?B?bDBERG5xaFpaTHdsOHRPV2FONEVubUt2MXF2dWlXTVN3NmR4OTdYTld6dFJh?=
 =?utf-8?B?UHdOeUJ5ZCtLclV5bEY1RmRTQ2JuQlJzRWZaODNvaEFQTHhKb3g5c00vanRW?=
 =?utf-8?B?Z282eUVJbTQ1N3RzTUFxYWdMVmFpVVdzRnppUkxmTlBFL01aWjUxQ0tYSVo2?=
 =?utf-8?B?UDVWOTFTNFcrZ2V0RU82bm9abU0yeVpzKzhkeURFaEh2bmNRTStiT0NJUEZ5?=
 =?utf-8?B?c3VBQlJPdEJQdTd0Q0ZJb0VhYVMxTWZoaEFpQlhZTDVHblpmaUd0Z0VadlJl?=
 =?utf-8?B?bEx5QXZOalhoUWg3WWRHSXZtT1pXbUY5ZzViczhhRkZlQ050T1hISEtPS3Qy?=
 =?utf-8?B?UkxHNVc5Y3JFZStnSUZNM2ZSdTIzOXA1RVFtL29sZ2hmSHkxbi9CU1FGMTA5?=
 =?utf-8?B?cDFBVS9LT0VzNVk2OXp0RktDdFROOVpZa3pXVU11OEsrcmE1MHNSd3Y3eTdG?=
 =?utf-8?B?NS9iMzZsZW91Nk1HcXRTcTFGZnk5dWlkb3BTNXVpNXYvM3kzTFpMSllKWjBw?=
 =?utf-8?B?K0NuVVBEaEFWU2hFYzFINEgvWnlsNjFvRHdPdXZHMlBqMmJ2dThuaHh0enVI?=
 =?utf-8?B?M1VJdDB5bGxpQk94cTJudXhaWnZTa0tWTldnMXAxYXJ5aGpsSGg2MDZwd1lu?=
 =?utf-8?B?cSsvSHA4c1lmTlBmRjkzaThPUmlyVzVXbXphZEx0MVB4YVE5Mk9EZ2FJaEsy?=
 =?utf-8?B?eFhBNWxQOFZlSCtpLzljeFdnOTl1UEtVNEo4UzNjbnQ4VnJoQlUrcVhhZ1lB?=
 =?utf-8?B?ekVFR21FeG4zYUxKQVY2S2VKMnBqYnZVeXVjRlliU1FQZUViajBENk02cDN5?=
 =?utf-8?B?UWM1NGZ2SG1pTGM5cmtGM3U3Q2lJajh6eldCdk50TXdVdXdPalVlcFJHZnJ5?=
 =?utf-8?B?VVVrb1ZVak04WTcvQkZjckJCb3dtTlZTMXpaVjVGb3dBTnNWenpsb2dMMVhl?=
 =?utf-8?B?SXZ4cUdiSFNDVlVVTHQ5QkM0UldZaTQ2b3djTDE4ZUl6QStmeU5hVVgwVnZl?=
 =?utf-8?B?L21PMmc1RWt6WWpOQlZyaFlTMXpHQTl1UDR0aFZ2Zm5nZW1BZC92SHk0WnpC?=
 =?utf-8?B?UG5uVkQ2SkM4MGJVWUNmdzRoaU84N1FOMU9sZlFhWkViS1Z0UFdVdGkxcnNo?=
 =?utf-8?B?VU5KR2VjK2ZENHpiTForaEhMWHpreVZxaHRGMURsY3M0VUVVbk14Q3pxUlAz?=
 =?utf-8?B?Ynpsd3BxV1JGK2tIS1cyWk5PcXd2eDcrQzVrVTBWNjNmT2RhM3pXVm5zY0pO?=
 =?utf-8?B?VXd3c2Vvam1zdkVaeEdWVkpmWWhuYktDRkhuSDN1cDYwUFd3ai8yMUd6d2c0?=
 =?utf-8?B?RGdUWFdXa3NGSmhCN0N0UWZzdmZDbFRoTDlmU0MvYmVudDQyaGk4Zzh3RzRM?=
 =?utf-8?B?dE9tZ1N1ekxGZU5DcU91U0FGYWNRc1dVWGF3UlRkVXBPNnAzR2FUYWxRZDZ6?=
 =?utf-8?Q?qbZVQZz9At29VAsS2iO2s4Jy08h6Ek2uyO5oZvQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 58c068e7-bd38-449e-2459-08d972a963dc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:16:50.1985
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oxiRe2KcUtPOjSQt5W89hd1K48UU4j81/tBa+grj82mrAEvna7RhDDNcFvfBWZtOxr9eFEGZyOzcKB3JKWQ0+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

On 08.09.2021 08:57, Christoph Hellwig wrote:
> On Tue, Sep 07, 2021 at 02:05:32PM +0200, Jan Beulich wrote:
>> While the hypervisor hasn't been enforcing this, we would still better
>> avoid issuing requests with GFNs not aligned to the requested order.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I wonder how useful it is to include the alignment in the panic()
>> message. I further wonder how useful it is to wrap "bytes" in
>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
>> properly").
>>
>> --- a/drivers/xen/swiotlb-xen.c
>> +++ b/drivers/xen/swiotlb-xen.c
>> @@ -231,10 +231,10 @@ retry:
>>  	/*
>>  	 * Get IO TLB memory from any location.
>>  	 */
>> -	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
>> +	start = memblock_alloc(PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
>>  	if (!start)
>> -		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
>> -		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
>> +		panic("%s: Failed to allocate %lu bytes align=%#x\n",
>> +		      __func__, PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
> 
> CAn you avoid the overly long lines here?  A good way to make it more
> readable would be a variable to hold the byte count.

There already is a variable for the byte count - bytes. Did you read
the post-commit-message remark? _That's_ what I'd prefer to shorten
things. Meanwhile I can of course wrap lines; I will admit that I
failed to pay attention to line length here.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:27:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:27:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181582.328753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNtrM-0003TL-Hn; Wed, 08 Sep 2021 09:27:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181582.328753; Wed, 08 Sep 2021 09:27:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNtrM-0003TE-En; Wed, 08 Sep 2021 09:27:12 +0000
Received: by outflank-mailman (input) for mailman id 181582;
 Wed, 08 Sep 2021 09:27:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNtrL-0003T7-9N
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:27:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f08a8b74-1086-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:27:09 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-t_TL7G9YOfGi_mOJw49UfQ-1; Wed, 08 Sep 2021 11:27:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5471.eurprd04.prod.outlook.com (2603:10a6:803:d0::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 09:27:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 09:27:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0259.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 09:27:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f08a8b74-1086-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631093228;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=BGsWH3DlKX8CGcHvdtIStd5DnQ/LPtxITcKt1B0HTMw=;
	b=i7FHzqfAHQEIfoFXdYBq0l4Cwrd4LBe2MCW12YGzGRZzSYTOQLuhK62fEsqLMmU4V9fDys
	F//5NqIM65+QlR5hDqa1agQDbct6/7ja1n/GE7MxblI/VppT32Jy8ZnscI8wcDPUWy4XgN
	PeV17OytrqzsKv5OcPQyjEv6Alo5VOI=
X-MC-Unique: t_TL7G9YOfGi_mOJw49UfQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZTf+i/a8bwz3ctpaateDObBE9D3gnoJA9W+tUTXWQZ5nU6jQXZ/VfDGrZ60RAFZGkOTH3/P+rzg36rmjK5e/B5RmXUw/PB8FFPelt/4OSccTINKHs3beoWm2hdCz03g3a3oeMg1erZNRefbWiGzHDbIm6VB4fiVpCrraUmPxvEFVDylnW+5/eLja0kz9d0WU3VNtntGXq9kyd4LKcJ/0URiCAuHbkT72/EsThLG3CaDnGwyaFusFgO0Ena9wifon3oHIjee0NQ9jq3uNsuS0r6ConH0kjBGwxb3ejqVdwKl8eG85SQeJImfD1qz0Pg8gp7+XNpR0L7U+GrhAInddWw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1zhjAwC6vyGmgkrRKbJ1KOOZfLGy0u9qqYmJFK21YRk=;
 b=cKk4nP7hC3JDOfIL0dBMkjY9dd/xkC+L6gFX7M4sGsZIm/31ozwgW96HpbLopXcAZPKddY9xTZDGc70d1Rqg/G7Xl/2cFjudq5zKzG4636mEDo+97qB8OhBhcSyIs+VfajAkxGEZpk4YJP1owC5meNcdiPJpSzgzGjC6JImPHIW6PBfOf+MqyYK/33sm5AjoeHJ3PwUbh2LAnJ5bud6DVJ7YEWYRlMFby1076imZhQf8mJWAPoCTvoCt9u7uQYO23NoKsSV3LS6jxzTb/kF4rXjAkVtpjHyIfRlvPdP9s37TY6YBDy+sayXMigPEbOe/KPS4tCmNaiHgGmkJwhO/Fg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
Date: Wed, 8 Sep 2021 11:27:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0259.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::31)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 700032e4-a12a-479a-facc-08d972aad2ce
X-MS-TrafficTypeDiagnostic: VI1PR04MB5471:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5471D5F931ABD6189A3F010FB3D49@VI1PR04MB5471.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FGDjygKQMoUD429p3r+tWx84GNwJVZGnnN9bZbHt8hJ+UtZS2x+9ql4wZ5zS5KqYoE6E8Aad22cTxsaylR20FI7e4/BAZmbKqtBxu+/k1qZBQeYqRgvShrC80FCAOSZQIHX1H6PDoazwUPw8tSBheH3dDzipjhGfCWneMUlfsHkS0mfXZp2ic1RSy+wtTtOs/m+7FgGrgYmo5UTgyncE6I491CbbgI2x5Mx7JirC4ZeF3Ojbeq537VQFq3tjzP02m1BflREM6ndQepes43vL54o0awoPwTMBszJj/bXaZl53Tw4UBanwduEOAnjDz0T9pGLrMcmMdK0cf6fWVyCaDB5bsRhzcCYY9dXHCd9LLzsz6C1q7IfOCBOaoEnlV5hnqYPUxFRNfk4Vsudv3bCG+eQ3M+b33tI9Oc+PDF8YMA65xpmu8eMXUJSmC4N70WaSQ/FIdqI9IebmRgmOKDgRvuEWtGRgGpcEmsRtaZMVvSAW8LwH9GYyZ2pigWOl+boLfI1oYh+jquSWRHKw/KDVKvoULA6X2enpd5HNSOp04TIwdNBw2mfEL5cxLC60m5FxBO9SGmxtecFA2BzEIiKIZTXi1p1jdBnUjKnJw3hA23UAbvsD9oxWXaQDeA7n738cyEpM5xnw0vauAVf4r9R2IW4BvDtvH3KLaMbUWTsdnOKW+B8LPz963bBrk+oYo+HzTnNGB+92Epne71l9o3s4k7UznFsEP1WnIF36pLcw9Pd1htX+rG6jwX/3C5tI4Fep
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(39860400002)(346002)(396003)(366004)(53546011)(16576012)(2906002)(186003)(36756003)(66556008)(66946007)(26005)(31696002)(4326008)(54906003)(478600001)(316002)(86362001)(2616005)(8676002)(956004)(8936002)(5660300002)(31686004)(110136005)(38100700002)(6486002)(66476007)(83380400001)(7416002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?9pFIHmupBTgZQnJNxGolZZGpQLRTZ5wSh9KSvGK0/QZvgopmAjWp6vT3mypH?=
 =?us-ascii?Q?3Ua+TK298yxLL2qm/lfM9eWkG0RoGeip6uLowgtWxJvxd2Ci6pl+Lsxv9GXB?=
 =?us-ascii?Q?b47lpZ8Bv2pI38o0TNLJIPxIDnRGAU/APH9YArpIKJriRdLnFhouo8BJYf0C?=
 =?us-ascii?Q?KFxwGOzSfyBbRHZMPVXI8tTb0Zn1F8ExTB8otUHh8E/oRFYue4SzylohZdSl?=
 =?us-ascii?Q?WPA9GeO+OaG4Sm3ir6XIRi9T6ajW2jrseihcxTonvhi8/PP+5QfKjstNcM4s?=
 =?us-ascii?Q?dlUMSQPy20y3w2BT/LWgq6We40g+qMFoWsMBGd1XMQdPXKZ4MuUWInf2MWqt?=
 =?us-ascii?Q?pcby/meM7PNmjAzraJexvJfOyqgHMqHuNruAjkE1f2VPdhj56lKijQXcDuRY?=
 =?us-ascii?Q?7UccfD7ViqqMVzdBeYJE/QS+MEOcYw9Kt6fkOPesEKi7IeMDC30N5OdtMBVQ?=
 =?us-ascii?Q?QKdiUWKLGUzPdqZQ8uROHVKUOsWVRaj12Kvw4cL8fCIpF7/zX3PtEWYn6Vsd?=
 =?us-ascii?Q?GMYuDwsqge7NoGHCRowCqLUiUgZi0Omh0ubiEOZyWrQxYxvkpRb5oVXySh7U?=
 =?us-ascii?Q?4XPLacWCviF0WfEx8VdYiBmF6NY5IkYhuvVRWmKMwpcpccuB+Fkf2ScYZGnz?=
 =?us-ascii?Q?AZ2umt7VaOelZyLZqjUO6GuKcek5sZZ2jZoLSrEoBByI16Zs0kxUyo7R6ND7?=
 =?us-ascii?Q?Q91rRcK5XED/4wIfEE4n8clOuSbmA8Xd4MX4HuXAS/Yi5CECxlqkxv25GlAZ?=
 =?us-ascii?Q?BKQBkrKAPMmwoiNPXIe7/mctbHMPaYXx10LmQqllbM7TcOjPzS5cWCGnP0Cd?=
 =?us-ascii?Q?hsFk0MKryEOLSm43wXXKfPSpiTCZdvqnhmP0x6tVqEe5+7W+C11q8sZiOwDH?=
 =?us-ascii?Q?2QtfOsscER0QNoiBOv5iPuE4ShMixA2UeHbstW7v7rcEPuF0Bg3lMvmXWAol?=
 =?us-ascii?Q?XZ0NoAoQgKZV3vgL4W93QQuO42FK0mrMW8ennwfaig5KBsr8J+wOK1QzXsWO?=
 =?us-ascii?Q?Y6+syZuVp2xKbWNHWPMkImKniifXYXktJGmee0nqC0RbjwHofukxp48n3CcV?=
 =?us-ascii?Q?mSMIFQxspYzekadPEbx9aQqtlov1DEZGmdqxfeo8laLw6/LjrDsNlp2dAl0g?=
 =?us-ascii?Q?7a1WfNCXaAwxgRKLmVfWUFCOQQRkvErgOsOi9MRztEiSkbUNpG+gPRvWiCoB?=
 =?us-ascii?Q?0uDW0LtPTEfVzBeoIG7gSj97J7XEHM7OGGJ5uZgJnH5gdZPM2M8Smp/P2CJU?=
 =?us-ascii?Q?eqYhqZz6x6sL5BCIiZ3pNhTpf0h7GKCA/rINuB4QUy5zgdnCYdR6dDHY2Vpr?=
 =?us-ascii?Q?CZnk/BJILLQmoK7vBSZK27HP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 700032e4-a12a-479a-facc-08d972aad2ce
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:27:05.7841
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: h+Pebd6H3BP+bhhMcrV4OxuHtCxL6f2eEOD9d5ySi29qntH8LYtTfae8UnIz7OjUdVWhymzMFJ/pvO3N3QXPiQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5471

On 07.09.2021 19:39, Oleksandr Andrushchenko wrote:
> On 07.09.21 19:30, Jan Beulich wrote:
>> On 07.09.2021 15:33, Oleksandr Andrushchenko wrote:
>>> On 06.09.21 17:31, Jan Beulich wrote:
>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>> --- a/xen/drivers/vpci/header.c
>>>>> +++ b/xen/drivers/vpci/header.c
>>>>> @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pde=
v, unsigned int reg,
>>>>>    static void guest_bar_write(const struct pci_dev *pdev, unsigned i=
nt reg,
>>>>>                                uint32_t val, void *data)
>>>>>    {
>>>>> +    struct vpci_bar *bar =3D data;
>>>>> +    bool hi =3D false;
>>>>> +
>>>>> +    if ( bar->type =3D=3D VPCI_BAR_MEM64_HI )
>>>>> +    {
>>>>> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
>>>>> +        bar--;
>>>>> +        hi =3D true;
>>>>> +    }
>>>>> +    else
>>>>> +        val &=3D PCI_BASE_ADDRESS_MEM_MASK;
>>>>> +    bar->guest_addr &=3D ~(0xffffffffull << (hi ? 32 : 0));
>>>>> +    bar->guest_addr |=3D (uint64_t)val << (hi ? 32 : 0);
>>>> What you store here is not the address that's going to be used,
>>>>    as
>>>> you don't mask off the low bits (to account for the BAR's size).
>>>> When a BAR gets written with all ones, all writable bits get these
>>>> ones stored. The address of the BAR, aiui, really changes to
>>>> (typically) close below 4Gb (in the case of a 32-bit BAR), which
>>>> is why memory / I/O decoding should be off while sizing BARs.
>>>> Therefore you shouldn't look for the specific "all writable bits
>>>> are ones" pattern (or worse, as you presently do, the "all bits
>>>> outside of the type specifier are ones" one) on the read path.
>>>> Instead mask the value appropriately here, and simply return back
>>>> the stored value from the read path.
>>> "PCI LOCAL BUS SPECIFICATION, REV. 3.0", "IMPLEMENTATION NOTE
>>>
>>> Sizing a 32-bit Base Address Register Example" says, that
>>>
>>> "Software saves the original value of the Base Address register, writes
>>> 0 FFFF FFFFh to the register, then reads it back."
>>>
>>> The same applies for 64-bit BARs. So what's wrong if I try to catch suc=
h
>>> a write when a guest tries to size the BAR? The only difference is that
>>> I compare as
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if ( (val & PCI_BASE_ADDRE=
SS_MEM_MASK_32) =3D=3D PCI_BASE_ADDRESS_MEM_MASK_32 )
>>> which is because val in the question has lower bits cleared.
>> Because while this matches what the spec says, it's not enough to
>> match how hardware behaves.
> But we should implement as the spec says, not like buggy hardware behaves

The behavior I'm describing doesn't violate the spec. There merely is
more to it than what the spec says, or one might also view it the way
that the spec doesn't use the best way of expressing things.

>>   Yet you want to mimic hardware behavior
>> as closely as possible here. There is (iirc) at least one other
>> place in the source tree were we had to adjust a similar initial
>> implementation to be closer to one expected by guests,
>=20
> Could you please point me to that code so I can consult and possibly
> re-use the approach?

I only recall the fact; this might have been hvmloader, vpci, or yet
somewhere else. I'm sorry.

>>>>> @@ -522,6 +582,13 @@ static int add_bar_handlers(struct pci_dev *pdev=
, bool is_hwdom)
>>>>>                if ( rc )
>>>>>                    return rc;
>>>>>            }
>>>>> +        /*
>>>>> +         * It is neither safe nor secure to initialize guest's view =
of the BARs
>>>>> +         * with real values which are used by the hardware domain, s=
o assign
>>>>> +         * all zeros to guest's view of the BARs, so the guest can p=
erform
>>>>> +         * proper PCI device enumeration and assign BARs on its own.
>>>>> +         */
>>>>> +        bars[i].guest_addr =3D 0;
>>>> I'm afraid I don't understand the comment: Without memory decoding
>>>> enabled, the BARs are simple registers (with a few r/o bits).
>>> My first implementation was that bar->guest_addr was initialized with
>>> the value of bar->addr (physical BAR value), but talking on IRC with
>>> Roger he suggested that this might be a security issue to let guest
>>> a hint about physical values, so then I changed the assignment to be 0.
>> Well, yes, that's certainly correct. It's perhaps too unobvious to me
>> why one may want to use the host value here in the first place. It
>> simply has no meaning here.
> Do you want me to remove the comment?

Yes. I wonder whether the assignment is necessary in the first place:
I'd somehow expect the structure to come from xzalloc(). Albeit I
guess this function can be run through more than once without freeing
the structure and then allocating is anew?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:44:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:44:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181590.328776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNu7o-0006Md-By; Wed, 08 Sep 2021 09:44:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181590.328776; Wed, 08 Sep 2021 09:44:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNu7o-0006MS-7i; Wed, 08 Sep 2021 09:44:12 +0000
Received: by outflank-mailman (input) for mailman id 181590;
 Wed, 08 Sep 2021 09:44:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0iEI=N6=epam.com=prvs=98850bb71f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNu7m-0006JA-Ss
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:44:11 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c69fc1e8-ed8b-40ae-be59-91f4e4c4704e;
 Wed, 08 Sep 2021 09:44:09 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1889FxoT024637; 
 Wed, 8 Sep 2021 09:44:06 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2050.outbound.protection.outlook.com [104.47.8.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3axtd083nt-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 08 Sep 2021 09:44:06 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5700.eurprd03.prod.outlook.com (2603:10a6:208:16f::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 09:43:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.015; Wed, 8 Sep 2021
 09:43:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c69fc1e8-ed8b-40ae-be59-91f4e4c4704e
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VSFa/X+79PDEBINK7vW5RYS39EJ8wso4+uI1RozkiTUHmKBcySqoxW89V8sJti/sef79MWdSMCzdxWJHgx0ZM+iPPiePEhzJEaOXyytdv9zsUeL8tsgADslosXT5szef2IT4qEjNTDo1xTyzZpNEGibCeDwPcyHNlA0nZx36/jNopxP6dy/hqAKo0Tnawz9PkNh/x+fCMi+VX7EO7HLy/qc3Grajgp4OEeCV971s8+Gks+g0WjktZp+fFySltxUscARmMaAwOCGpWm/KyHg3iW7WRgmjxfWir4uxQxqhpb7V/VCUgIU50cwyQ7xzljFh3JilYSJi+3opTykpGHDHgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=X2aKd0k+XEncRABb6WT/koWSa4++HRFGx8CZSLC/ryw=;
 b=dbGjh8a2Aa9nmAJX9Uo2tAWU2fkRocCXSVj8ZGyqsXpLRSZ5MpJyHeMJwJVAQm28TeYWKnSxzgNrDQbaEl7TI8JFZxOgZiBEF5t59tMl6hLKoCqpxzYCD67FskBR3AtAGLNfn929vlqOnjVwyHynKNeRFrgGkz4V0yA7VEx5zkhXP3OwC6wBuTmR4jsxRfoBwEtLffxnhJDad+7HLWRa18/oImX1PrlOejbD0DYtCU/M+7c08ggseOyqIbHzQcuNZFPkk17tvJZ5OZTkR+nntKrtolUBG7DRiGsNq6PkOBEpDCeL65oBBdo7KzeSdvXwmao+LrgzZxHVXNUPBnRQ4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=X2aKd0k+XEncRABb6WT/koWSa4++HRFGx8CZSLC/ryw=;
 b=072+yUaJttkbl1Pp+sWpvbUZUZUrzAxK+OvzF1/mRNK+c8W0hgVcfXgicyESRpXL8U+StK+n7qQLtEINgbRwj+aQA1Jtp4NkYO6akWmAeVlgEFzCbIo3Vfsb4f0gcZWfEIOSqNc3rE49rxKnDE7gk8NGEE7xPpPgBVL9SJ+Mz53DCw3ZL/DObkopY1/dcnPP6J2c/PfiaxRtapKjZsyGRbAmF+Cuna8EYx83wvCxE1k6DzKaADZJBGVdM+xJfroNBQn7ySuotgxRAQ8ci6Fz/hHufMtrx0Vw8y+XUXSchsDW3eXxZtzAfqiezg3QLjZk9/TBTX+MMwsYm2bxHZo7dw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: 
 AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLICAADFtgIAAE0qAgAEIooCAAAS6AA==
Date: Wed, 8 Sep 2021 09:43:59 +0000
Message-ID: <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
In-Reply-To: <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b5f5beeb-a20a-4741-bfb7-08d972ad2f1a
x-ms-traffictypediagnostic: AM0PR03MB5700:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB570043DC38F1A70C0BA8230AE7D49@AM0PR03MB5700.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 8b3+MPJ8fTW8GNQJ28JO1xIeYrvduCPcdyHuOjJbNZjMvdmgMma1Xrc4CwuPnHoHJoby7RxsiZemVLZUz2MtHg+D+4b9cqHHfeodEhW88CwyuKHNeJQn+sAEPjutwj2l6e0W3bxKZ3QJEb2Lwtj0pIG+tjM7TNrnJ5I3pH7LlViuQSl1VRDw0R8b1KSgkMY8dak6N0KtptkHO0ALecY6tO42W2U017K8YsblTtWXk/9j7/JySvEwoGhOFlGBMSxLBCdXyjpwTo1eGdZ5aKQPp2xiIDbXg7NBKOnZ0cu1VGtUcVzkOjJs1ZOwYuQHPIM9sn2BlFKe6AG43FzkCvJyOPUABnItV7OR9rCmdrSFwWvHyHWiPQgOnay7c9T5sYwfwnagnOEsS1a18HVd4HU2Vf18vm6lQhEyWahSD1zhfkLAikOaFvvEWdFIo54fV9tXfeRkja40LCmZGGeMX2wVwaYiXTMJ3QSwdLDmBXslOLYNE+JlkhtpOWBQanZ3q015JPdVOQyi9J9i+wJNgFWLPaGXCOaHiHAWRG/tuuQk89vp2qdHHzaTnGnOJ3yOALLc+d6uQTK6QagWVyYwaiqLTpvAom50b8rVRgcLJKZ7QIfBB9Gjric8K6Y+jtgl6sQXtwiTLVXerKJa0s/M7Ln2lMsPeS8op60fcOPv1Mm7sONRRprWSkEPzy2YvPGLqY6g3+xYSgMB8c1S+bySXkXpDphmJyFsHvZpz/qVmSp3Idk2QbkxcZk9Z3F649yUvJpQ
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(346002)(136003)(366004)(376002)(39860400002)(8936002)(38100700002)(6486002)(4326008)(122000001)(186003)(31686004)(64756008)(66946007)(91956017)(76116006)(83380400001)(31696002)(6506007)(5660300002)(26005)(110136005)(316002)(478600001)(53546011)(2906002)(54906003)(36756003)(71200400001)(86362001)(2616005)(8676002)(66476007)(38070700005)(66556008)(66446008)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eUNMOTlrSWhCNzI4SmEvS2FqY1hiQ3VhWUNkTURRdDlUSytyOUxxcVpVOExE?=
 =?utf-8?B?RmdQeHVPWVBmN0xFVHZCYXVZbTBhVlowUE10MGd1VDBLai9GY1cwdnpkWjZ6?=
 =?utf-8?B?RnhBTHk4eVYvUGRFWm9LYTRnRGFJZkFleWNyMHphdk9neU5IQVVDYXhzclJF?=
 =?utf-8?B?eXd5Yksrb2l5WnVJSmFESmpzbzcwa2RibXdKMU4zL1JJbDBjU1pzY1pubDcy?=
 =?utf-8?B?TXM5bE9FYnp3dDlIbWpIOHNxbGEwNGxlSVBNd2dUeFlhaGVlVk52MnRiK0xo?=
 =?utf-8?B?WjE4Q3Rja25BRWNxSjd6L3ROczJCWVpJK3k4MnU1QnFReDFZeG1JTHFtd1ZP?=
 =?utf-8?B?QXNqUnkxVE12dGExT0wvUUZIY0V1WHdCRUdaRXcwYlNoREwvOEtOZWMrRzg5?=
 =?utf-8?B?eitaeXZXdVhqbDdUSm9oaGRRQVZzcStFb2poSzR3aFBQUDhKTEVOTCtvWlBP?=
 =?utf-8?B?Rm90NXpQMWFWeWxSVWtITW4wMFIvV0VaTy9vRG84WlVKK0tLQ3VqMVlld1V1?=
 =?utf-8?B?VUM2anM0QUg2My9PM1ExUGFTYzlOc1BMODYvVEJmSHh4NTR5L0RUWDRBTFVM?=
 =?utf-8?B?dWoxZXFsVzJ3ZlFFUFk0bXhlWUI0YjhONXJzRDdaYkg1ZkxRZDJWNjdmOFRL?=
 =?utf-8?B?a2EzN3VWT1d6OUdvTkRta3QwK0dDbFZGVGh4QTI5WGxydXNSaHVsZGdZVS8w?=
 =?utf-8?B?Yk8wOHQwY3ZYc0ZiMWV3TWphbm5iM3FveWV0UHVvbEwrTUNySkgvWVlFeStE?=
 =?utf-8?B?VzFNSkRZaDFNZVowc0NXUFNLZ3BJUXd4RDFQQlhHS0s3ZWgya1dHODYwL2xT?=
 =?utf-8?B?bERnVmUyT1grc3Y2emNaOVY2Yzh5VWhsRk0vRXVvOWVMS1ZycDZKV0c2clY0?=
 =?utf-8?B?b0tEN0Zyek5Jb3FXZ2ZnNmtUK0k4V2M1K3JESEtHelhYd0VvQUZUbnZib3Fo?=
 =?utf-8?B?TGx1RTdNZmRUY2pxUE1IblZDN1RJUklNMGwxQUhac2hwT0t0WW1JRGpXeElQ?=
 =?utf-8?B?MklTWXJwNEhIM3h6ckxUdE5ieHJFZW5hQ2hnMzB0MkZGYkxJdU9VdUpUUnJs?=
 =?utf-8?B?b1k3VUVTRStJVG5YK0NWZU5QTFMyejN3US95TDVMcldYc0xNS3ZXRVV5ajJ6?=
 =?utf-8?B?dDZRSE1mcDFNbGlHWXVFcUFnekVpcFNNNWpiZk9mVHQrMnBtVkJLZTdNWDhJ?=
 =?utf-8?B?Zk5oZXRBTy9QNmMrL3hEVCswTUFyRlBuOU9FcnNXa1Bob09RRlFvZW50TjdP?=
 =?utf-8?B?QTdFNHUyRHZtcVo2bjU0N2NjWlZ2bFFhRWpWWjZEOXM4OEhVZUtaQ3RGT1BJ?=
 =?utf-8?B?dGFJbFpHaEVPVW1ObWs0TVE4TzZOZXRWUTQ4SERoRkJKbEhBOW9YQXdvbVBs?=
 =?utf-8?B?dFlKVzRSVU8vMXMwZ3dwY2F5ZTFmZVg3UGh2SXhTRjlJNVJRRzVMaFlKY3N2?=
 =?utf-8?B?d1A4N1hLVjNLRTJuaGE0MnV5aUI1VzQvQzdqQVNqUTZBSVpER2U2cWlRd0N6?=
 =?utf-8?B?ZTV3ZnFHS0M0WmtON3BOTURzYnc1UWYxR1VyRW5rRTlodFExTmRUNGdqMHJW?=
 =?utf-8?B?TEJOMEdjaTZIWmJpSk94WjdFdHNQVU90WElibHdrMEg3THRWZFdkZEZUTndG?=
 =?utf-8?B?U1Vvb21BM2psS3RXeVRycDZOUVNyeWIrVE1KVG1jWWFrS09WYm9rcU5Ebys5?=
 =?utf-8?B?UDkwa3pISzRvczR1QkN4QVFWdW9FTGw3NTVFd1dUSUdRRmRBbG9XZVVFRldW?=
 =?utf-8?Q?nsW1o0OzCzRtWbWymcFQL9AHqljHJIRiTygZCuZ?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <2DDCAB43ABD7A6409F62548FDE514006@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5f5beeb-a20a-4741-bfb7-08d972ad2f1a
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 09:43:59.3233
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dwC6Zi1yfh/yRvDbGIT66ZuOMihftHyQrsoR7rBSfcaCZwgzf/GtiRIGJRjF3PefeR/BSvsimNYWKaNH213+Zow1SifDmDU+H9hj+PGxJSffM1rc+qbYSVj/KYGRdAMg
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5700
X-Proofpoint-GUID: bBojQ6-moUQjMXAwKoCL4aBJnyOo2AqR
X-Proofpoint-ORIG-GUID: bBojQ6-moUQjMXAwKoCL4aBJnyOo2AqR
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-08_03,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 clxscore=1015
 lowpriorityscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999
 priorityscore=1501 adultscore=0 bulkscore=0 phishscore=0 impostorscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080062

DQpPbiAwOC4wOS4yMSAxMjoyNywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTk6MzksIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTk6
MzAsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTU6MzMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNi4wOS4yMSAxNzozMSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hlYWRlci5jDQo+Pj4+
Pj4gKysrIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4+Pj4+IEBAIC00MDAsMTIgKzQw
MCw3MiBAQCBzdGF0aWMgdm9pZCBiYXJfd3JpdGUoY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYs
IHVuc2lnbmVkIGludCByZWcsDQo+Pj4+Pj4gICAgIHN0YXRpYyB2b2lkIGd1ZXN0X2Jhcl93cml0
ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldiwgdW5zaWduZWQgaW50IHJlZywNCj4+Pj4+PiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHZhbCwgdm9pZCAqZGF0YSkN
Cj4+Pj4+PiAgICAgew0KPj4+Pj4+ICsgICAgc3RydWN0IHZwY2lfYmFyICpiYXIgPSBkYXRhOw0K
Pj4+Pj4+ICsgICAgYm9vbCBoaSA9IGZhbHNlOw0KPj4+Pj4+ICsNCj4+Pj4+PiArICAgIGlmICgg
YmFyLT50eXBlID09IFZQQ0lfQkFSX01FTTY0X0hJICkNCj4+Pj4+PiArICAgIHsNCj4+Pj4+PiAr
ICAgICAgICBBU1NFUlQocmVnID4gUENJX0JBU0VfQUREUkVTU18wKTsNCj4+Pj4+PiArICAgICAg
ICBiYXItLTsNCj4+Pj4+PiArICAgICAgICBoaSA9IHRydWU7DQo+Pj4+Pj4gKyAgICB9DQo+Pj4+
Pj4gKyAgICBlbHNlDQo+Pj4+Pj4gKyAgICAgICAgdmFsICY9IFBDSV9CQVNFX0FERFJFU1NfTUVN
X01BU0s7DQo+Pj4+Pj4gKyAgICBiYXItPmd1ZXN0X2FkZHIgJj0gfigweGZmZmZmZmZmdWxsIDw8
IChoaSA/IDMyIDogMCkpOw0KPj4+Pj4+ICsgICAgYmFyLT5ndWVzdF9hZGRyIHw9ICh1aW50NjRf
dCl2YWwgPDwgKGhpID8gMzIgOiAwKTsNCj4+Pj4+IFdoYXQgeW91IHN0b3JlIGhlcmUgaXMgbm90
IHRoZSBhZGRyZXNzIHRoYXQncyBnb2luZyB0byBiZSB1c2VkLA0KPj4+Pj4gICAgIGFzDQo+Pj4+
PiB5b3UgZG9uJ3QgbWFzayBvZmYgdGhlIGxvdyBiaXRzICh0byBhY2NvdW50IGZvciB0aGUgQkFS
J3Mgc2l6ZSkuDQo+Pj4+PiBXaGVuIGEgQkFSIGdldHMgd3JpdHRlbiB3aXRoIGFsbCBvbmVzLCBh
bGwgd3JpdGFibGUgYml0cyBnZXQgdGhlc2UNCj4+Pj4+IG9uZXMgc3RvcmVkLiBUaGUgYWRkcmVz
cyBvZiB0aGUgQkFSLCBhaXVpLCByZWFsbHkgY2hhbmdlcyB0bw0KPj4+Pj4gKHR5cGljYWxseSkg
Y2xvc2UgYmVsb3cgNEdiIChpbiB0aGUgY2FzZSBvZiBhIDMyLWJpdCBCQVIpLCB3aGljaA0KPj4+
Pj4gaXMgd2h5IG1lbW9yeSAvIEkvTyBkZWNvZGluZyBzaG91bGQgYmUgb2ZmIHdoaWxlIHNpemlu
ZyBCQVJzLg0KPj4+Pj4gVGhlcmVmb3JlIHlvdSBzaG91bGRuJ3QgbG9vayBmb3IgdGhlIHNwZWNp
ZmljICJhbGwgd3JpdGFibGUgYml0cw0KPj4+Pj4gYXJlIG9uZXMiIHBhdHRlcm4gKG9yIHdvcnNl
LCBhcyB5b3UgcHJlc2VudGx5IGRvLCB0aGUgImFsbCBiaXRzDQo+Pj4+PiBvdXRzaWRlIG9mIHRo
ZSB0eXBlIHNwZWNpZmllciBhcmUgb25lcyIgb25lKSBvbiB0aGUgcmVhZCBwYXRoLg0KPj4+Pj4g
SW5zdGVhZCBtYXNrIHRoZSB2YWx1ZSBhcHByb3ByaWF0ZWx5IGhlcmUsIGFuZCBzaW1wbHkgcmV0
dXJuIGJhY2sNCj4+Pj4+IHRoZSBzdG9yZWQgdmFsdWUgZnJvbSB0aGUgcmVhZCBwYXRoLg0KDQpC
dXQgaW4gY2FzZSBvZiBCQVIgc2l6aW5nIEkgbmVlZCB0byBhY3R1YWxseSByZXR1cm4gQkFSIHNp
emUuDQoNClNvLCB0aGUgY29tcGFyaXNvbiBpcyB0aGUgd2F5IHRvIHRlbGwgaWYgdGhlIGd1ZXN0
IHdhbnRzIHRvIHJlYWQNCg0KYWN0dWFsIChjb25maWd1cmVkKSBCQVIgdmFsdWUgb3IgaXQgdHJp
ZXMgdG8gZGV0ZXJtaW5lIEJBUidzIHNpemUuDQoNClRoaXMgaXMgd2h5IEkgY29tcGFyZSBhbmQg
dXNlIHRoZSByZXN1bHQgYXMgdGhlIGFuc3dlciB0byB3aGF0IG5lZWRzDQoNCnRvIGJlIHN1cHBs
aWVkIHRvIHRoZSBndWVzdC4gU28sIGlmIEkgZG9uJ3QgY29tcGFyZSB3aXRoIDB4ZmZmZmZmZmYg
Zm9yIHRoZQ0KDQpoaSBwYXJ0IGFuZCAweGZmZmZmZmYwIGZvciB0aGUgbG93IHRoZW4gaG93IGRv
IEkga25vdyB3aGVuIHRvIHJldHVybg0KDQpjb25maWd1cmVkIEJBUiBvciByZXR1cm4gdGhlIHNp
emU/DQoNCj4+Pj4gIlBDSSBMT0NBTCBCVVMgU1BFQ0lGSUNBVElPTiwgUkVWLiAzLjAiLCAiSU1Q
TEVNRU5UQVRJT04gTk9URQ0KPj4+Pg0KPj4+PiBTaXppbmcgYSAzMi1iaXQgQmFzZSBBZGRyZXNz
IFJlZ2lzdGVyIEV4YW1wbGUiIHNheXMsIHRoYXQNCj4+Pj4NCj4+Pj4gIlNvZnR3YXJlIHNhdmVz
IHRoZSBvcmlnaW5hbCB2YWx1ZSBvZiB0aGUgQmFzZSBBZGRyZXNzIHJlZ2lzdGVyLCB3cml0ZXMN
Cj4+Pj4gMCBGRkZGIEZGRkZoIHRvIHRoZSByZWdpc3RlciwgdGhlbiByZWFkcyBpdCBiYWNrLiIN
Cj4+Pj4NCj4+Pj4gVGhlIHNhbWUgYXBwbGllcyBmb3IgNjQtYml0IEJBUnMuIFNvIHdoYXQncyB3
cm9uZyBpZiBJIHRyeSB0byBjYXRjaCBzdWNoDQo+Pj4+IGEgd3JpdGUgd2hlbiBhIGd1ZXN0IHRy
aWVzIHRvIHNpemUgdGhlIEJBUj8gVGhlIG9ubHkgZGlmZmVyZW5jZSBpcyB0aGF0DQo+Pj4+IEkg
Y29tcGFyZSBhcw0KPj4+PiAgICDCoMKgwqDCoMKgwqDCoCBpZiAoICh2YWwgJiBQQ0lfQkFTRV9B
RERSRVNTX01FTV9NQVNLXzMyKSA9PSBQQ0lfQkFTRV9BRERSRVNTX01FTV9NQVNLXzMyICkNCj4+
Pj4gd2hpY2ggaXMgYmVjYXVzZSB2YWwgaW4gdGhlIHF1ZXN0aW9uIGhhcyBsb3dlciBiaXRzIGNs
ZWFyZWQuDQo+Pj4gQmVjYXVzZSB3aGlsZSB0aGlzIG1hdGNoZXMgd2hhdCB0aGUgc3BlYyBzYXlz
LCBpdCdzIG5vdCBlbm91Z2ggdG8NCj4+PiBtYXRjaCBob3cgaGFyZHdhcmUgYmVoYXZlcy4NCj4+
IEJ1dCB3ZSBzaG91bGQgaW1wbGVtZW50IGFzIHRoZSBzcGVjIHNheXMsIG5vdCBsaWtlIGJ1Z2d5
IGhhcmR3YXJlIGJlaGF2ZXMNCj4gVGhlIGJlaGF2aW9yIEknbSBkZXNjcmliaW5nIGRvZXNuJ3Qg
dmlvbGF0ZSB0aGUgc3BlYy4gVGhlcmUgbWVyZWx5IGlzDQo+IG1vcmUgdG8gaXQgdGhhbiB3aGF0
IHRoZSBzcGVjIHNheXMsIG9yIG9uZSBtaWdodCBhbHNvIHZpZXcgaXQgdGhlIHdheQ0KPiB0aGF0
IHRoZSBzcGVjIGRvZXNuJ3QgdXNlIHRoZSBiZXN0IHdheSBvZiBleHByZXNzaW5nIHRoaW5ncy4N
Cg0KV2VsbCwgdGhlIHNwZWMgZXhwbGljaXRseSBzYXlzICJ3cml0ZSAweGZmZmZmZmZmIGFuZCBy
ZWFkIGJhY2siDQoNCkkgY2FuJ3Qgc2VlIGFueSB3YXkgaXQgY2FuIGJlIHJlYWQgZGlmZmVyZW50
bHkNCg0KPg0KPj4+ICAgIFlldCB5b3Ugd2FudCB0byBtaW1pYyBoYXJkd2FyZSBiZWhhdmlvcg0K
Pj4+IGFzIGNsb3NlbHkgYXMgcG9zc2libGUgaGVyZS4gVGhlcmUgaXMgKGlpcmMpIGF0IGxlYXN0
IG9uZSBvdGhlcg0KPj4+IHBsYWNlIGluIHRoZSBzb3VyY2UgdHJlZSB3ZXJlIHdlIGhhZCB0byBh
ZGp1c3QgYSBzaW1pbGFyIGluaXRpYWwNCj4+PiBpbXBsZW1lbnRhdGlvbiB0byBiZSBjbG9zZXIg
dG8gb25lIGV4cGVjdGVkIGJ5IGd1ZXN0cywNCj4+IENvdWxkIHlvdSBwbGVhc2UgcG9pbnQgbWUg
dG8gdGhhdCBjb2RlIHNvIEkgY2FuIGNvbnN1bHQgYW5kIHBvc3NpYmx5DQo+PiByZS11c2UgdGhl
IGFwcHJvYWNoPw0KPiBJIG9ubHkgcmVjYWxsIHRoZSBmYWN0OyB0aGlzIG1pZ2h0IGhhdmUgYmVl
biBodm1sb2FkZXIsIHZwY2ksIG9yIHlldA0KPiBzb21ld2hlcmUgZWxzZS4gSSdtIHNvcnJ5Lg0K
Tm8gcHJvYmxlbQ0KPg0KPj4+Pj4+IEBAIC01MjIsNiArNTgyLDEzIEBAIHN0YXRpYyBpbnQgYWRk
X2Jhcl9oYW5kbGVycyhzdHJ1Y3QgcGNpX2RldiAqcGRldiwgYm9vbCBpc19od2RvbSkNCj4+Pj4+
PiAgICAgICAgICAgICAgICAgaWYgKCByYyApDQo+Pj4+Pj4gICAgICAgICAgICAgICAgICAgICBy
ZXR1cm4gcmM7DQo+Pj4+Pj4gICAgICAgICAgICAgfQ0KPj4+Pj4+ICsgICAgICAgIC8qDQo+Pj4+
Pj4gKyAgICAgICAgICogSXQgaXMgbmVpdGhlciBzYWZlIG5vciBzZWN1cmUgdG8gaW5pdGlhbGl6
ZSBndWVzdCdzIHZpZXcgb2YgdGhlIEJBUnMNCj4+Pj4+PiArICAgICAgICAgKiB3aXRoIHJlYWwg
dmFsdWVzIHdoaWNoIGFyZSB1c2VkIGJ5IHRoZSBoYXJkd2FyZSBkb21haW4sIHNvIGFzc2lnbg0K
Pj4+Pj4+ICsgICAgICAgICAqIGFsbCB6ZXJvcyB0byBndWVzdCdzIHZpZXcgb2YgdGhlIEJBUnMs
IHNvIHRoZSBndWVzdCBjYW4gcGVyZm9ybQ0KPj4+Pj4+ICsgICAgICAgICAqIHByb3BlciBQQ0kg
ZGV2aWNlIGVudW1lcmF0aW9uIGFuZCBhc3NpZ24gQkFScyBvbiBpdHMgb3duLg0KPj4+Pj4+ICsg
ICAgICAgICAqLw0KPj4+Pj4+ICsgICAgICAgIGJhcnNbaV0uZ3Vlc3RfYWRkciA9IDA7DQo+Pj4+
PiBJJ20gYWZyYWlkIEkgZG9uJ3QgdW5kZXJzdGFuZCB0aGUgY29tbWVudDogV2l0aG91dCBtZW1v
cnkgZGVjb2RpbmcNCj4+Pj4+IGVuYWJsZWQsIHRoZSBCQVJzIGFyZSBzaW1wbGUgcmVnaXN0ZXJz
ICh3aXRoIGEgZmV3IHIvbyBiaXRzKS4NCj4+Pj4gTXkgZmlyc3QgaW1wbGVtZW50YXRpb24gd2Fz
IHRoYXQgYmFyLT5ndWVzdF9hZGRyIHdhcyBpbml0aWFsaXplZCB3aXRoDQo+Pj4+IHRoZSB2YWx1
ZSBvZiBiYXItPmFkZHIgKHBoeXNpY2FsIEJBUiB2YWx1ZSksIGJ1dCB0YWxraW5nIG9uIElSQyB3
aXRoDQo+Pj4+IFJvZ2VyIGhlIHN1Z2dlc3RlZCB0aGF0IHRoaXMgbWlnaHQgYmUgYSBzZWN1cml0
eSBpc3N1ZSB0byBsZXQgZ3Vlc3QNCj4+Pj4gYSBoaW50IGFib3V0IHBoeXNpY2FsIHZhbHVlcywg
c28gdGhlbiBJIGNoYW5nZWQgdGhlIGFzc2lnbm1lbnQgdG8gYmUgMC4NCj4+PiBXZWxsLCB5ZXMs
IHRoYXQncyBjZXJ0YWlubHkgY29ycmVjdC4gSXQncyBwZXJoYXBzIHRvbyB1bm9idmlvdXMgdG8g
bWUNCj4+PiB3aHkgb25lIG1heSB3YW50IHRvIHVzZSB0aGUgaG9zdCB2YWx1ZSBoZXJlIGluIHRo
ZSBmaXJzdCBwbGFjZS4gSXQNCj4+PiBzaW1wbHkgaGFzIG5vIG1lYW5pbmcgaGVyZS4NCj4+IERv
IHlvdSB3YW50IG1lIHRvIHJlbW92ZSB0aGUgY29tbWVudD8NCj4gWWVzLiBJIHdvbmRlciB3aGV0
aGVyIHRoZSBhc3NpZ25tZW50IGlzIG5lY2Vzc2FyeSBpbiB0aGUgZmlyc3QgcGxhY2U6DQo+IEkn
ZCBzb21laG93IGV4cGVjdCB0aGUgc3RydWN0dXJlIHRvIGNvbWUgZnJvbSB4emFsbG9jKCkuIEFs
YmVpdCBJDQo+IGd1ZXNzIHRoaXMgZnVuY3Rpb24gY2FuIGJlIHJ1biB0aHJvdWdoIG1vcmUgdGhh
biBvbmNlIHdpdGhvdXQgZnJlZWluZw0KPiB0aGUgc3RydWN0dXJlIGFuZCB0aGVuIGFsbG9jYXRp
bmcgaXMgYW5ldz8NCg0KSSdsbCBjaGVjayB0aGF0IGFuZCBpZiB0aGUgc3RydWN0dXJlIGlzIGFs
cmVhZHkgemVyb2VkIHRoZW4gSSdsbCByZW1vdmUgYm90aCB0aGUNCg0KYXNzaWdubWVudCBhbmQg
dGhlIGNvbW1lbnQNCg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:44:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:44:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181589.328765 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNu7m-00066K-2q; Wed, 08 Sep 2021 09:44:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181589.328765; Wed, 08 Sep 2021 09:44:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNu7l-00066D-VW; Wed, 08 Sep 2021 09:44:09 +0000
Received: by outflank-mailman (input) for mailman id 181589;
 Wed, 08 Sep 2021 09:44:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNu7j-000667-UA
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:44:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4ecf7292-1089-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:44:06 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2058.outbound.protection.outlook.com [104.47.12.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-p9xAwkobOOqALHL89s1eNQ-1; Wed, 08 Sep 2021 11:44:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3295.eurprd04.prod.outlook.com (2603:10a6:802:f::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 09:44:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 09:44:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0068.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Wed, 8 Sep 2021 09:44:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ecf7292-1089-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631094245;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kYrUvTDOaeHXOQl4F+JcjOpNIyGdYKlDiKPxbowcl2Q=;
	b=feUUomu7lNjCrsNgveMuY2HAId1KSTSKsvN18F6z3MPJQv9RtHP2w9RkVvv7cp6VTCKyw8
	NTRTEupg6LIkaKhZn+w8B75Xn3dFosjASbMPkeBFb6h6LB9UIBHUQRuR9PUbstKLIkAyS+
	ixBKx5BKzR46tWvUrjQuoX0lulsaLNo=
X-MC-Unique: p9xAwkobOOqALHL89s1eNQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YJzrp2WC8FcsRVS8SkRMENAVVyAVepLuQEYpi0uJq/6kBn/A58CkoIJG3FU2Ih1QGci+m2N7d9E39kblkCVpAhbIwc78e1V6odyOSR8mwNrPzko7DGr0Fpo3rM9lnBNQqp30e2sS9XT2iwQiHuh1wPXeX6cz2XGaWhlElchc1xMW707wazzOT0PqfZlqME9YcFfJWiJVKKJ2GYfuho4LVpoIzOreS8sft4RHzhDoxp2lbyPwe8ExAj4zeQqKC03oFuu0guFfG+Uz+Ir55gEUGPF5yEKeDE0pxKvN2AqpVu2jnwo7cFOHEGC4aU4+W9UvQFh8rzay2sovj90Zh1F1MA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4/2nMHtwUkwOJeNn+c/1PSvVLkll4mq/pLnUB8brkF0=;
 b=TK4azVDcKoNqk8E2/AF/Nqek1lbq6kfowYhzQaSDjciL7rGFCp8jaZDCmWpuF5hA25QCGlNiZL8qEm3pYXc2EK581BKcp4bhFGPyxQHX+9rHNCLaXJ7jFS3QfB9X0quKEenvL2ZZmjDmf/4EmEjdeTumUIdj/G2WdXN+mgwyuAm0mBONrRj4kJXXVtBWCeFxhjp2LqqXl+tncTkKMs4GwYm7rkOWzMB51QFJMVZFnjb3oKyROu91xmrebvmZIO1iX6IvNIggbvheChJSJmAwmyOEC4joASdjndhHN4+MJxMhtv0w2HH4sSewcG9RLDMO+iqPhkC9SozLgWnRpaGByQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 07/17] IOMMU/x86: restrict IO-APIC mappings for PV Dom0
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Paul Durrant <paul@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <5d6e8360-7ba8-487e-0ac8-7f517e560520@suse.com>
 <42aa8c24-7f23-deec-6103-f2d556e4a9f9@citrix.com>
 <b3db56ae-7a53-ee22-d821-d711b0d0bf3b@suse.com>
 <3f9722e4-704b-8f68-553d-04a3a3189d99@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <48f43552-d657-2600-4503-2eb8626c114e@suse.com>
Date: Wed, 8 Sep 2021 11:44:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <3f9722e4-704b-8f68-553d-04a3a3189d99@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0068.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1d::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9c9c3f30-2682-4ba7-965c-08d972ad30fb
X-MS-TrafficTypeDiagnostic: VI1PR04MB3295:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB329527837FF767C36362E4B9B3D49@VI1PR04MB3295.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z6f2bVMSSQty+iXoyRJThmGihcrlm+t8aA2pMwzhLEQe8jONW1ZtdA6n7p8I1+sbcK7BTkzhgLcyKT+E2qHRxBH90PyHMHOOMVeRpm1Q9Ilm57lR6hst9MoGRD08cnzwg1fuqKbFSwDSo3QwSQXVPlTX9FV2p7Fzi/zlQSSZG7Bs0tWrkI2z7aIOhuGXr7TQ6DOiNfLajqz8MWFpnfYBO1hGifZiu2ghk7mlOI5WEIadgfio3116XPV6ebiEuj+wekxpIKM0C445ivi8RsuPNCiVxYg4NDqDPXknoIE4AKbhQopwP6+tW+jpvCCb0orL37k/mqzSrWHV6KH1g579fToUoW3QnTFseKjPgL4ZlyvZmpcIobz4vPSgpGnj1DUpqmlTmeol3ggimm7EqMBjgOgq59oHSMJ7xMu53UANHHyri8zCo8JKwkijrjcnwNQjUiddF7kT3xXcYeGVI8/l7hVWC/VOSfdXkYKrEVB9LPMWoZ4uG3uG0M+5BF9tgid5mTLx1q1dVkx+a4WcEjdv5YjE+Z0byyW4v1kwfzn+PdBN4AE7lfDIJkK+KeBa/18JQIqMFpLs9HBPuaXiRv+E1hzB/0EeAK+8NCErCmywvcGwBuuWJzAYq1FfL5a5xE6AoHl3wxQzUBOOqybqqDqIjKkJEOjRW14No/jsisiNcYsFU3AFkJObwcDJx3wrLtUbbuTowYnp5BNJ29yFQbEQUnmMydstocqWpSmDKg2MMK3dx7v6z4eMJwFq/V3aXJu0
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(346002)(136003)(366004)(396003)(6486002)(66946007)(66556008)(8676002)(478600001)(66476007)(8936002)(16576012)(186003)(5660300002)(26005)(53546011)(6916009)(316002)(83380400001)(2906002)(36756003)(86362001)(4326008)(956004)(2616005)(54906003)(31696002)(31686004)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?kWy3ul4G50CaOH1coO17a3tayFqEtOE5213/ciXaCzmjtHnhWoo4fS3fDnAh?=
 =?us-ascii?Q?4xi5EpAkw89Npwu9KiIkRNty55IQHvKL2N2dizfCY50klTryLaoQRAeX2xqC?=
 =?us-ascii?Q?6SOMRypl1OC8V+eipSBeXl5tm0Y4dy0HsWEKKjQVCVTiA2AyMtloRoLIyjd1?=
 =?us-ascii?Q?bgwXjZma1YG/3fHfMOOYSaMM+ysgZcux+bQ2xTYcyhJkvww0ILYE7yVBNaSh?=
 =?us-ascii?Q?FFupOutZV8O37vy3IutbSxvKxX+LicCgqnhgkqoXD7N4vpZmuDVgZna+q094?=
 =?us-ascii?Q?4kXMpScei1kIeVkgWTJCvDwBWuKHjxoNX7o7CWxSk112JziKJb582B+YdbPt?=
 =?us-ascii?Q?aoLQQokwopjy19f0l1FNR28et/ZQXmDFlwKjQMXuGKEcd4BK7i2pzhr61JjC?=
 =?us-ascii?Q?rz8oEyrdAEkNyjlPXyfBPBgqUuPSCx1qhFSfjSPSy2XyMfAMQ0k5okz2TZ9U?=
 =?us-ascii?Q?Ecm4jvrRxa/c98Am49kfdtWTCk+8M1m2YXiAC8aWfXnOF5lw5N4wA5s0EK/X?=
 =?us-ascii?Q?nx8rAj5tHRy7L/HON+ne6aPkTgMidTK2UdAuOQ5IuqkJRmHrnvA/6E+/KUJH?=
 =?us-ascii?Q?FNBDdvGWzdjdZDKTLYqlV/3yr5IO0hsz8L7l1Aq+0AbhrV1W1vUmyEPNZysP?=
 =?us-ascii?Q?mx0SZPFqJqkznH9lg9yBHZw1xgu3OtrFgEOgdnDoKb8rvtr4ui0EY4pjWKEd?=
 =?us-ascii?Q?TrbOd4fby2aAsEpBSVIz1CrYgx1jshvovgbQruDT+R/7iwciL9x7aVEeIP40?=
 =?us-ascii?Q?QOQeJ7DKULoA6PU79Zn8ycOZjzLQ4+d2TKJUR4xgH+o0KNtdZelLvRR5XmPS?=
 =?us-ascii?Q?PKzkVyVoXUAdq8rT2DC0K4cNL3bPjYPG6ptlm63yTLYEx+fcHCddpqH5UYkS?=
 =?us-ascii?Q?a+4W5EVFNCVWcR1Tnn9zSQ7SunjWOyuLtIt8JI/i4d8dfn64ArwJG2yJveTK?=
 =?us-ascii?Q?n6DZUobRtznEN9In4160nicoshpSWTLsZrrWkh46C9SdpYLVPDZ2QsmOQWeZ?=
 =?us-ascii?Q?0pAgj78To40BHVkzbRFFpZSOQwSdofk2cWTTEjj+nLGh38swEb5gCmy/H9oK?=
 =?us-ascii?Q?k5S+Rlh7/hNMtQJD0YNBxx3xvPhpvFpkeTWYfJSeIPtrx8F4+FEMzxqDxtkO?=
 =?us-ascii?Q?iB+Of2q1MfV2qkxGGmX8yt1KpWbqIZ7YY1lciCPiP2yHxKlWM7v1OspDuw3M?=
 =?us-ascii?Q?v60VivGNEqY7Z4tgD68jrbiuLxIWYW3RwFXKx6AdqFpixCOsL6imrFujfk4Y?=
 =?us-ascii?Q?nUCDgtTPT57jqIq66/GOTFkxK3ilLKbgOp5ojrURUlIDyWKRn0PmGQPQJHUg?=
 =?us-ascii?Q?xcfF1TiQrbdNRzxNseMt5VMK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c9c3f30-2682-4ba7-965c-08d972ad30fb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:44:02.7618
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KgGpkXU43xWi461PK6q93CfMyXXMXdtuRVt9vSnjyDIlnywiyC2gfmS70q0m0dbD3tj3jHgPnure0OebRW3O8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3295

On 07.09.2021 19:13, Andrew Cooper wrote:
> On 26/08/2021 13:55, Jan Beulich wrote:
>> On 26.08.2021 13:57, Andrew Cooper wrote:
>>> On 24/08/2021 15:21, Jan Beulich wrote:
>>>> While already the case for PVH, there's no reason to treat PV
>>>> differently here, though of course the addresses get taken from anothe=
r
>>>> source in this case. Except that, to match CPU side mappings, by defau=
lt
>>>> we permit r/o ones. This then also means we now deal consistently with
>>>> IO-APICs whose MMIO is or is not covered by E820 reserved regions.
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> Why do we give PV dom0 a mapping of the IO-APIC?=C2=A0 Having thought a=
bout
>>> it, it cannot possibly be usable.
>>>
>>> IO-APICs use a indirect window, and Xen doesn't perform any
>>> write-emulation (that I can see), so the guest can't even read the data
>>> register and work out which register it represents.=C2=A0 It also can't=
 do an
>>> atomic 64bit read across the index and data registers, as that is
>>> explicitly undefined behaviour in the IO-APIC spec.
>>>
>>> On the other hand, we do have PHYSDEVOP_apic_{read,write} which, despit=
e
>>> the name, is for the IO-APIC not the LAPIC, and I bet these hypercalls
>>> where introduced upon discovering that a read-only mapping of the
>>> IO-APIC it totally useless.
>>>
>>> I think we can safely not expose the IO-APICs to PV dom0 at all, which
>>> simplifies the memory handling further.
>> The reason we do expose it r/o is that some ACPI implementations access
>> (read and write) some RTEs from AML. If we don't allow Dom0 to map the
>> IO-APIC, Dom0 will typically fail to boot on such systems.
>=20
> I think more details are needed.

How do you expect to collect the necessary info without having an affected
system to test? I see close to zero chance to locate the old reports (and
possible discussion) via web search.

> If AML is reading the RTEs, it's is also writing to the index register.

Quite likely, yes. Albeit this being broken to a fair degree in the
first place, ...

> Irrespective of Xen, doing this behind the back of the OS cannot work
> safely, because at a minimum the ACPI interpreter would need to take the
> ioapic lock, and I see no evidence of workarounds like this in Linux.

... as you indicate you think (as much as I do), leaves room for the
actual accesses to also be flawed (and hence meaningless in the first
place). I do recall looking at the disassembled AML back at the time,
but I do not recall any details for sure. What I seem to vaguely recall
is that their whole purpose was to set the mask bit in an RTE (I think
to work around the dual routing issue, and I assume in turn to work
around missing workarounds in certain OSes). For that the current
approach as well as the alternative one you suggest below would be
equally "good enough".

> In Xen, we appear to swallow writes to mmio_ro ranges which is rude, and
> causes the associated reads to read garbage.=C2=A0 This is Xen-induced me=
mory
> corruption inside dom0.
>=20
>=20
> I don't think any of this is legitimate behaviour.=C2=A0 ACPI needs disab=
ling
> on such systems, or interlocking suitably, and its not just IO-APICs
> which are problematic - any windowed I/O (e.g. RTC) as well as any other
> device with read side effects.

I don't think disabling ACPI on such systems would be a viable option.
Things tend to not work very well that way ... Plus iirc these issues
weren't exclusively on some random no-name systems, but in at least one
of the cases on ones of a pretty large vendor.

> I think we should seriously consider not mapping the IO-APIC at all.=C2=
=A0

We can easily do so on the IOMMU side, if you agree to have CPU and
IOMMU mappings diverge for this case. Since the behavior is PV-
specific anyway, there are also no concerns as to differing behavior
with vs without shared page tables (on PVH).

> That said, I would be surprised if Linux is cleanly avoiding the
> IO-APIC, so a slightly less bad alternative is to redirect to an "MMIO"
> frame of ~0's which we still write-discard on top of.
>=20
> That at least makes the Xen-induced memory corruption behave
> deterministically.

It would work more deterministically, yes, but without such a system
available to test we wouldn't know whether that approach would actually
make things work (sufficiently). Whereas for the current approach we do
know (from the testing done back at the time).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:50:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:50:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181603.328786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuEE-00089y-6l; Wed, 08 Sep 2021 09:50:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181603.328786; Wed, 08 Sep 2021 09:50:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuEE-00089r-3o; Wed, 08 Sep 2021 09:50:50 +0000
Received: by outflank-mailman (input) for mailman id 181603;
 Wed, 08 Sep 2021 09:50:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuED-00089h-9O; Wed, 08 Sep 2021 09:50:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuEC-0001TN-VT; Wed, 08 Sep 2021 09:50:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuEC-0005e0-Ly; Wed, 08 Sep 2021 09:50:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuEC-0006Sj-LQ; Wed, 08 Sep 2021 09:50:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jxv4/VF0rSRpr8nxrdNXoK4ReTaCqegR0vpgtk6im0g=; b=zV9wbBfYnpvqYSHYsZDN9z2xcF
	xF2/9Tl+BHjk4z23mVzBV7w3c3G3Se4H7ewE6DEfxnv6AdKgyqF+R8Rn0GZYJLs+7YapdbNVb6TUX
	BRLz4Ne9OpZ0jZ15LsrqePbf37+wff4GW3qF+K/kIj/sKL0YyX/JzcunlpIdMUjnqQRg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164882-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 164882: all pass - PUSHED
X-Osstest-Versions-This:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
X-Osstest-Versions-That:
    xen=2d4978ead4b98452bc45ebe47952e524b12e2b41
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 09:50:48 +0000

flight 164882 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164882/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1
baseline version:
 xen                  2d4978ead4b98452bc45ebe47952e524b12e2b41

Last test of basis   164827  2021-09-05 09:18:31 Z    3 days
Testing same since   164882  2021-09-08 09:18:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>
  Michal Orzel <michal.orzel@arm.com>
  Wei Liu <wl@xen.org>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2d4978ead4..e70a9a043a  e70a9a043a5ce6d4025420f729bc473f711bf5d1 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181610.328801 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHa-0000bp-Nh; Wed, 08 Sep 2021 09:54:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181610.328801; Wed, 08 Sep 2021 09:54:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHa-0000bi-KL; Wed, 08 Sep 2021 09:54:18 +0000
Received: by outflank-mailman (input) for mailman id 181610;
 Wed, 08 Sep 2021 09:54:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuHZ-0000bc-I7
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:17 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b7b15bec-2067-4a9e-95f2-30d0d984b939;
 Wed, 08 Sep 2021 09:54:14 +0000 (UTC)
Received: from AM5PR0701CA0057.eurprd07.prod.outlook.com (2603:10a6:203:2::19)
 by AM4PR08MB2884.eurprd08.prod.outlook.com (2603:10a6:205:e::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 09:54:13 +0000
Received: from VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:2:cafe::f6) by AM5PR0701CA0057.outlook.office365.com
 (2603:10a6:203:2::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.8 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:13 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT014.mail.protection.outlook.com (10.152.19.38) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:12 +0000
Received: ("Tessian outbound 7a08471b1ef6:v105");
 Wed, 08 Sep 2021 09:54:12 +0000
Received: from f9cb34ce38a9.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E3D35D93-00F2-4BFD-944C-AECC6C0EEA11.1; 
 Wed, 08 Sep 2021 09:54:06 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f9cb34ce38a9.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:06 +0000
Received: from AM0PR10CA0063.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:15::16)
 by VI1PR0801MB2110.eurprd08.prod.outlook.com (2603:10a6:800:88::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Wed, 8 Sep
 2021 09:54:05 +0000
Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:15:cafe::2b) by AM0PR10CA0063.outlook.office365.com
 (2603:10a6:208:15::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:05 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:05 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:08 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7b15bec-2067-4a9e-95f2-30d0d984b939
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=atD+jn23OE3UfPIAVyiiv0CU9p/Xc0KpqFQDOZmE+ro=;
 b=xDzMJJMeRGJQCDwVvReAfHVMVZWV5kkt1k/IaREBFsU0uWfazkma23ZCFC+VLoAvBjVEFTHThicE7LUMUoSNItl+eKZSMVAe9OBlpcz1iAJ91ZNiTIWzPv7OdBuBwHCKsD7z4IVhaJuc5kr0kj03Dxbl+1uL+FaghgWwjGdKwOE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d66a6ad96b4563a5
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cs5964ruU/zXdVU2Oti8ZONkQoqxdfDsQ+Ad4dLFvz2eAtl4pgTmcOLaso7sj9c3sbBOFcGrie54bNiuF2TJ5pD1vRikS3RlmZEUBMjuz2POHlb8YHXryV4E5UkcUUq9wwIG5ylvNC/biNvCacVz5aqtFrQiGEbxrjc80xFjeBNE+JmlGPjGzDBGUsIcWGijeJE9dQFoZkADAhTdOQMzHNDOOULJmOiuQuGdagP1koNzi4X/sUt41+iy8Dv0u7YBbtDNROsm4DZim33kY0IeMu2nq+CXC7saPIzkuRunhZgg8wY75B/bNsfW6Mpu6H49YtcVGdsoUZv+d0VfMXcfIw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=atD+jn23OE3UfPIAVyiiv0CU9p/Xc0KpqFQDOZmE+ro=;
 b=UNrFUXPtXXZnq8HhBCkoJokrXzibv+rbk8wPPYZXDvDTQh+SUQOhstHSKVNshg5L+um69/b+DMelqE92erxWyl70ukq7jK3mfO6BRb3lTasxk/F8yg96ph/HHHgjP+Ky8y4YsF2P1QgqpbAZSnMOr9dMr7n1qhBjWOtt/1F2P+vqeylPDG8jQgfgy16QlGOQiBf/4P7f0vBgfPUT77gV2TtUWxFx+K3EN+5Rnoty0BYM/82His+M9No2oq+TPcX2NhnMI0p+gtE89/K6P6AG41LPXKOtkmmR30JM8rpBLLUw8UrgWDGaZDWs3oziWXs7bosBGAP5xdffFhVwl/N9yA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=atD+jn23OE3UfPIAVyiiv0CU9p/Xc0KpqFQDOZmE+ro=;
 b=xDzMJJMeRGJQCDwVvReAfHVMVZWV5kkt1k/IaREBFsU0uWfazkma23ZCFC+VLoAvBjVEFTHThicE7LUMUoSNItl+eKZSMVAe9OBlpcz1iAJ91ZNiTIWzPv7OdBuBwHCKsD7z4IVhaJuc5kr0kj03Dxbl+1uL+FaghgWwjGdKwOE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 1/7] xen/arm: introduce new helper device_tree_get_meminfo
Date: Wed, 8 Sep 2021 09:52:42 +0000
Message-ID: <20210908095248.545981-2-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8ea29876-98fb-4e87-885c-08d972ae9cae
X-MS-TrafficTypeDiagnostic: VI1PR0801MB2110:|AM4PR08MB2884:
X-Microsoft-Antispam-PRVS:
	<AM4PR08MB28841439D86E26D4DB9337A5F7D49@AM4PR08MB2884.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:41;OLM:41;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GKdMi4ctUd5Xo6YEnZLv4p3v89P/yqnvqJ6+XsyQ6aBjkV3uPeh9BZe4vMYtc5Naum4aPMkRtLp5JoIGUlRFvEG/1poEW1Jwwjlj+e6pOUbajb5zoLKlztI+gxxdph58fZkcG8yUyhFqhNainvmMNExqUpWarbA/3cXBgo7zLE8z49xvQ4QzLlzIqbxNwCy0HtQ/SoHL3RxOC4XgXBrZ6p46Vg78RoXywPOHzvsJA/IiOPu8S5YVYwjHjrSh+GRLgA51Jsn3WkdiV0g3fH9Hv6S4zLpBVg8eg6Czvokwd46t+cmtBzFlucV8W+i6JJeq4hB8zC2oLoK2q3rmviSbzOprJ6LlBTFoMhIlmi5CwUDmOAPf1lIX3QlkZ5u/IuSznqukeFW80wXfQ4lOe2UDEnw4RmbTtARUhFo3KhkRmir4zDdVXSKbL9jztG0YVdQdwkfIwkP1ngJNGxHz/2Ahv87osJA9jmgQNOxV70P4N1ZDD55aZcyyRjN3jVtRUIQzjYu+FE8GQ/6rIdP+AgOkmkU2aofwXcBMrjdFNQOfJDQdl6c1BM/IoVqBmoh9KvI60d4qc266hcU7zli7WKKsjVD438ZGWVNVxKLLzC6nd6Tjp7mf8NWolpRHsSUBbrexKO8c4x0lSvhFj0pxii9ZjA5eqauWsJvH8Azhxg+mK+Ww9TWxz5oslGTbuBEDgwKTKB7Psr7dP/Ah5aBTzjpvZILETyYvxTz9PrFTCK9gnyA=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(36840700001)(46966006)(5660300002)(2906002)(336012)(478600001)(2616005)(7696005)(316002)(86362001)(47076005)(83380400001)(36756003)(6666004)(26005)(81166007)(82310400003)(1076003)(44832011)(8676002)(82740400003)(4326008)(426003)(110136005)(36860700001)(70206006)(70586007)(8936002)(54906003)(356005)(186003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2110
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a22bcb71-7aec-41fa-0708-08d972ae981d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4YCmS1+/kYlhRhk0d0qPdxTIYUZkNOq+LhvbJQPjFpoP/PK3JqfOqQ5X/A0anWOmlhiXB8FgeuT+AxOd9Nxh3Iv5jpbx/NEl3ay8wcehhyWpx6CZ8tPYnvqNZCapMEebc6IzCSpy6enV03j+tWKEbx0fQiSf1kKKBu2uRhMb9DILiYK5RZrvuFmek4JK2YOEHet2gib97doolQWaU3pT/jf6ah+XnWPZRUvxHunu2phfNW8W5n8KqHvNMgSQFxgT4XY8gqDg1aSUviCV0K/X1mSF3MhWOeQxei8YvIniEE0d7H8KAnMgQ75RiiGswYZI2PPAPZwuARwwMkWWb1QgpvsCYunitHgZjhCoqi0BnYgOWSktVjhLgGnHwa4v8xtCZykzNPDkmz/b8KW9X2LNfM3p0fPI+Ou4EjGSAkxd1bC07hNvbY6j5biyDQt95QvuJeipeEbtoxChYItvyg0R7JdS53cdAnVy90vQISVRDP5xFwfWmBA+N2WIkQWqm6udpRvK6oaaOraKusANoQ4mWqODhVrr50qWD79yk90B4cMtpWp3AStgGGlDprzzOdTJoY7URX73WQZfYjGLCK7I2xxzo+9LgIS2W1rQVJOvxMt51U8599bi0HmsyALmF6WetZrQJSBi6zqmtSV13lHeHiRlAI/OyiF9iAPGvLwUgBKSOwXCoEprhimzBmJRso5e2yGtsyUUyVkJ2FSyKFxCzw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(376002)(346002)(396003)(39860400002)(46966006)(36840700001)(7696005)(2906002)(82310400003)(86362001)(5660300002)(8676002)(478600001)(70206006)(70586007)(26005)(186003)(47076005)(36860700001)(336012)(8936002)(54906003)(1076003)(316002)(6666004)(2616005)(44832011)(82740400003)(36756003)(83380400001)(4326008)(110136005)(107886003)(81166007)(426003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:12.7096
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8ea29876-98fb-4e87-885c-08d972ae9cae
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR08MB2884

This commit creates a new helper device_tree_get_meminfo to iterate over a
device tree property to get memory info, like "reg".

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/bootfdt.c | 83 ++++++++++++++++++++++++------------------
 1 file changed, 47 insertions(+), 36 deletions(-)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 476e32e0f5..b01badda3e 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -63,6 +63,52 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
     *size = dt_next_cell(size_cells, cell);
 }
 
+static int __init device_tree_get_meminfo(const void *fdt, int node,
+                                          const char *prop_name,
+                                          u32 address_cells, u32 size_cells,
+                                          void *data)
+{
+    const struct fdt_property *prop;
+    unsigned int i, banks;
+    const __be32 *cell;
+    u32 reg_cells = address_cells + size_cells;
+    paddr_t start, size;
+    struct meminfo *mem = data;
+
+    if ( address_cells < 1 || size_cells < 1 )
+    {
+        printk("fdt: property `%s': invalid #address-cells or #size-cells",
+               prop_name);
+        return -EINVAL;
+    }
+
+    prop = fdt_get_property(fdt, node, prop_name, NULL);
+    if ( !prop )
+        return -ENOENT;
+
+    cell = (const __be32 *)prop->data;
+    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
+
+    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
+    {
+        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
+        /* Some DT may describe empty bank, ignore them */
+        if ( !size )
+            continue;
+        mem->bank[mem->nr_banks].start = start;
+        mem->bank[mem->nr_banks].size = size;
+        mem->nr_banks++;
+    }
+
+    if ( i < banks )
+    {
+        printk("Warning: Max number of supported memory regions reached.\n");
+        return -ENOSPC;
+    }
+
+    return 0;
+}
+
 u32 __init device_tree_get_u32(const void *fdt, int node,
                                const char *prop_name, u32 dflt)
 {
@@ -139,42 +185,7 @@ static int __init process_memory_node(const void *fdt, int node,
                                       u32 address_cells, u32 size_cells,
                                       void *data)
 {
-    const struct fdt_property *prop;
-    int i;
-    int banks;
-    const __be32 *cell;
-    paddr_t start, size;
-    u32 reg_cells = address_cells + size_cells;
-    struct meminfo *mem = data;
-
-    if ( address_cells < 1 || size_cells < 1 )
-    {
-        printk("fdt: node `%s': invalid #address-cells or #size-cells",
-               name);
-        return -EINVAL;
-    }
-
-    prop = fdt_get_property(fdt, node, "reg", NULL);
-    if ( !prop )
-        return -ENOENT;
-
-    cell = (const __be32 *)prop->data;
-    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
-
-    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
-    {
-        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
-        /* Some DT may describe empty bank, ignore them */
-        if ( !size )
-            continue;
-        mem->bank[mem->nr_banks].start = start;
-        mem->bank[mem->nr_banks].size = size;
-        mem->nr_banks++;
-    }
-
-    if ( i < banks )
-        return -ENOSPC;
-    return 0;
+    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
 }
 
 static int __init process_reserved_memory_node(const void *fdt, int node,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181611.328812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHg-0000to-0r; Wed, 08 Sep 2021 09:54:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181611.328812; Wed, 08 Sep 2021 09:54:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHf-0000tf-TX; Wed, 08 Sep 2021 09:54:23 +0000
Received: by outflank-mailman (input) for mailman id 181611;
 Wed, 08 Sep 2021 09:54:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuHe-0000bc-Ge
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:22 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.79]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e8f14c5b-4e24-4dad-9847-be994e9b4b79;
 Wed, 08 Sep 2021 09:54:15 +0000 (UTC)
Received: from PR2P264CA0033.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::21) by
 DB9PR08MB6764.eurprd08.prod.outlook.com (2603:10a6:10:2ac::24) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22; Wed, 8 Sep 2021 09:54:13 +0000
Received: from VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:101:1:cafe::dd) by PR2P264CA0033.outlook.office365.com
 (2603:10a6:101:1::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:13 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT039.mail.protection.outlook.com (10.152.19.196) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:12 +0000
Received: ("Tessian outbound 610b9df843d5:v105");
 Wed, 08 Sep 2021 09:54:12 +0000
Received: from 056c89f01e9d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B14FAD12-741A-4612-B01D-4FCB12450E7A.1; 
 Wed, 08 Sep 2021 09:54:05 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 056c89f01e9d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:05 +0000
Received: from AM0PR10CA0068.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:15::21)
 by AM6PR08MB3320.eurprd08.prod.outlook.com (2603:10a6:209:48::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:03 +0000
Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:15:cafe::77) by AM0PR10CA0068.outlook.office365.com
 (2603:10a6:208:15::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:03 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:02 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:06 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Wed, 8
 Sep 2021 09:53:59 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8f14c5b-4e24-4dad-9847-be994e9b4b79
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CBX0PwkTMiWyE/sx14ufn9c6BIC7mzVQZNMew7fmruw=;
 b=dHHcxkEsvmGks+u0JkP1kgCfQlkBHAJ8kkcLl+b8SEW+z+36LAq4PdpLdvXi02nQjyNn/+LnIZ/1eD+TWaiDYuQaZL5w3TqDy2cNBDGHCUYTn3UTbUARFzpCXi1qS4sUNtLoPBGGKO9WAeWvOhfR0tkDhDpxv04wC8gZ8Nuw0zA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6b671d2233b8a19d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Six+12KXp1VJDlKtrW44SuymfQWgEU0qInacgt+tj1d3qoytge2Hmq1cECUcfnmMtrBgh3Og3PEcNCoY9MAlI3OQJgDRO+tCjqFX/Jt1iORS1Uj2Am9NrwoS8h0ktQ3cxCkbpWaZP/Dr3YGJtgLMwDMB4CF8/WlBSL6FMsf6Z1vuHfaSRn/QOsmwcd3EuuXJZF3ryJh+VJ/zq7sejM4hBXrQgazzy48VkXD3V1PJOR8Tsdhfd+vgFHb1QoURSSmtfw69QZVXUqY4Buew1hPRdMmczs1kvw1Z/1nHs8WXGPnQ42cIIc+Q30z0g2T0DPQi5DC/FXEm4nAYCXhBxC4wxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CBX0PwkTMiWyE/sx14ufn9c6BIC7mzVQZNMew7fmruw=;
 b=XgE+mXKkiSptOrvI33nM+Q7uHd+Cc1TWGLTwMretKSUi9wKK9dG1+UKQnVxILXtzLGfwh+Zk7aCToAVOvzSye4f7B6EtTP//1F2cFWm/X/nzfce0njI1XvKAK2/h1pxbXpy3pn/6wGVuH7bj+/u4BmZpveFYXD42oscMxo7J01DOI+NamLekCwJhDOiL353ly3yuCy2BqMIHeHPQPVRNNqYpLnXpqF4jQOnRdW0REFQfH/r7OpQfYZYf/AnyL7k+2/jjd9SSabKO9WunoX2oH1gwjjXT6vfnE+jon96xs9s+jUr5JYeVPmWmSx2HYs3yEFTH42xDGH+U6y7TnEy8yA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CBX0PwkTMiWyE/sx14ufn9c6BIC7mzVQZNMew7fmruw=;
 b=dHHcxkEsvmGks+u0JkP1kgCfQlkBHAJ8kkcLl+b8SEW+z+36LAq4PdpLdvXi02nQjyNn/+LnIZ/1eD+TWaiDYuQaZL5w3TqDy2cNBDGHCUYTn3UTbUARFzpCXi1qS4sUNtLoPBGGKO9WAeWvOhfR0tkDhDpxv04wC8gZ8Nuw0zA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 0/7] Domain on Static Allocation
Date: Wed, 8 Sep 2021 09:52:41 +0000
Message-ID: <20210908095248.545981-1-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8c073a9a-1d55-4a52-4e90-08d972ae9cb3
X-MS-TrafficTypeDiagnostic: AM6PR08MB3320:|DB9PR08MB6764:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB676425EB9B27AE682F3CEE8FF7D49@DB9PR08MB6764.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 qTqc/3a0/Vhag0ux6FztrgpkVCRp2tU8xpeBnXGqESCqDP2sQgL2iRt4c0aYDBOXbpsnVedMawd6PVFAIlsvCHS4zOhWAcFL7VQ0VZujJllM9Sw05m7cEV+UfXKGsjaUysOwRhiyVRh1huDZxrACy9RGxoQH0blcZFO8Bket4XEBwZyYE3t6iJMva5GRVUPwU/ixPYwHAot4+/lh83k6vBOEp5E6QRkcY7ujYdflSq+oNc8l7wB1tEgTgvKe0nne5Uu1/kQ5dEPao/MRL7Ov7Bwcfh4XWHW2itAyWY68wMk6qnAa07QFFwYFtczXU17OsFr2ziy0QB+ZSd0hIEv9OF7mApQDHnH6ff1yTc3mJnleTA3SFMuJqCTQkrsDgXrXol/2/BX/8Hso9lOngqJmngxf8ND+x/FmDxqxG4YcLp7y37K2/YRCy/5C/Ssle8Pw2QpNtnHXUSvUmNglwA9bXAhn71dk1jRdcbJni3n9r9OJTvDePdRTafIWNSq3pAGuB3q1BO8cYSpAK4QEJka0CSogryMjJxL2FKbVHCno/mOhgAZNUDYsrsNJRKSuOvUXwNTWvWFvTUfYDkoeDZFNiW6T3AWal4RS/Z982hgWISleczqmSuQsewPccg4+H1dQ8vtRnB5LUMhsc46/6QZJgP3OBXTkxtGqZ97HAiQA4uy7uCLTryZjYbUJNehYlnMxuZtUFfuWnzE1EJ2tc1ak8vx2Ovw2xjYAas/Mudidxry9n77t89I1Y+uerM3n1JacblQSlvzN14x61RQ/MLf+elE7voeIykDshHV0IXTa5e3ATvfeWkusaHZJqu4P48Ec
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(396003)(136003)(36840700001)(46966006)(82740400003)(110136005)(316002)(36860700001)(1076003)(47076005)(83380400001)(86362001)(44832011)(82310400003)(8676002)(4326008)(2906002)(2616005)(26005)(6666004)(36756003)(426003)(356005)(478600001)(8936002)(336012)(5660300002)(81166007)(70206006)(186003)(966005)(7696005)(54906003)(70586007)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3320
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2a4a8db5-15e9-46d1-4ada-08d972ae96dc
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+UceenRdIteWWyKgdXjzXFzrESqBDbVvbbaOqs55uhGqTqEFO6HuQRzMqcJPsA31eArXJSE8xIqkOdn5Wc9+RFxFR/P2xFkj2gzKbW6rxTYelEVBMBiQxXTVuIkmd9pI5iBK+B7LKAleuu9Aq22f0RowjY/HRdd4H5g3sVV3KvObIB45QXDZo6iTd21Bbf7/u1AUcePCJY7d1sJDTlD0EEPI7XgyAFjzTdKFuTwnSDmHC1IvfYQqB+RCjEsCQe6uTM28LWTnePxoKSbeA9NyklhLhD9G2Tq2DFhOGoqN2MDMzvjykysD2vJTaXTIFJZ4bh/DQF4uSjPkXmKdaxNgG0issV/lvmDOVsCin2c9fpY/wJy4RflbVJUmjR1VWGlUjD5hI8oViYujf+bEXKBX0Thf/rdDzUXhIr2kYUFa5cKzULDcvHDCeulASboXzaKliHInHxjv132KLM4U5ddAM5rLTssTCVh7dwGXzHlivOXpy30+cQzO99ytl6P9iVsMCbobdXGI+zdyjwqIZQrxBHBXjpQpJZFgJN3Jeo2l+CeE+MAao5IDRSoyMeMaEmDcGm1a2OIGYlmrcRXbCb3Um2DTEMR0COqfcU45rmi0HdZPn7ZONr79OQ9+keedb1THAF7ZpxQyYxgomAjDlyRjMBCulRprP4kp62WFlD7lMuXGGyfOSk5ep686BqkioNW80AAlbv17mpLIIgjFbweHq4QmV3VCTiAvdMT3Ek+ByTZv+RYWNl85l0SGHmYfF4RauwEzxUKL95A8wvNynQdxavMs/DSgZL4qpmDvhPdkywY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(39860400002)(136003)(36840700001)(46966006)(44832011)(82740400003)(26005)(4326008)(36756003)(426003)(36860700001)(186003)(70206006)(70586007)(110136005)(6666004)(54906003)(336012)(316002)(86362001)(107886003)(8676002)(5660300002)(7696005)(81166007)(2616005)(2906002)(47076005)(83380400001)(478600001)(82310400003)(8936002)(966005)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:12.7089
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c073a9a-1d55-4a52-4e90-08d972ae9cb3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6764

Static Allocation refers to system or sub-system(domains) for which memory
areas are pre-defined by configuration using physical address ranges.

Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
beginning, shall never go to heap allocator or boot allocator for any use.

Memory can be statically allocated to a domain using the property "xen,static-
mem" defined in the domain configuration. The number of cells for the address
and the size must be defined using respectively the properties
"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".

The property 'memory' is still needed and should match the amount of memory
given to the guest. Currently, it either comes from static memory or lets Xen
allocate from heap. *Mixing* is not supported.

The static memory will be mapped in the guest at the usual guest memory
addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

This Patch Serie only talks about Domain on Static Allocation.

Looking into related [design link](
https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg00882.html)
for more details.

The whole design is about Static Allocation and 1:1 direct-map, and this
Patch Serie only covers parts of it, which are Domain on Static Allocation.
Other features will be delievered through different patch series.
---
changes in v2:
 - consider reserved-memory binding and use phandle to refer(still in
discussion)
 - remove unused reserved field in struct page_info, unused helper
page_get_reserved_owner and page_set_reserved_owner
 - introduce CONFIG_STATIC_ALLOCATION to avoid bringing dead codes in
other archs.
 - extract common code from free_heap_pages and free_staticmem_pages
 - introduce new interface assign_pages_nr
 - change pfn-named to mfn-named
 - remove meaningless MEMF_no_owner case
 - leave zone concept out of DMA limitation check
 - take care of concurrency issue on static memory allocation
 - ensure the consistency when `xen,static-mem` and `memory` are both defined
 - fix the scalability issue in allocate_static_memory
 - allocate static memory when parse
---
changes in v3:
- use "xen,static-mem" property to be compatible with System Sevice Tree
in the future
- introduce new helper device_tree_get_meminfo
- extract common codes for dealing with reserved memory stored in
bootinfo
- rename from "free_page" to "mark_page_free"
- remove non-trivial page_to_mfn conversion in "mark_page_free" due to
pdx compression, and let the MFN passed in
- let all switch-cases shared in "mark_page_free"
- change CONFIG_STATIC_ALLOCATION to CONFIG_STATIC_MEMORY
- put init_staticmem_pages in setup_mm
- rename assign_pages_nr to assign_pages
- alter the order of assign_pages parameters to help backporting
- change name from alloc_staticmem_pages/alloc_domstatic_pages to
acquire_staticmem_pages and acquire_domstatic_pages.
- remove hunks' #ifdef-ary by introducing PGC_reserved = 0
- remove DMA restriction
- "memory" property shall be mandatory
- rename allocate_static_bank_memory to append_static_memory_to_bank
- infer the next GFN from the bank information in append_static_memory_to_bank
- simplify the code of double loop in allocate_static_memory
---
changes in v4:
- move the option CONFIG_STATIC_MEMORY to common code, and with Arm
"select"ing it
- replace round_pg{down,up}() with PFN_DOWN()/PFN_UP()
- in all cases where order-0 pages get passed, prefer using new assign_pages
to pass literal 1
- reconstruct the order of assign_pages parameters
- moving tlb/cache flush outside of the locked region, considering XSA-364
and reducing the amount of work happening with the heap_lock held
- remove MEMF_no_refcount case
- make acquire_staticmem_pages/acquire_domstatic_pages being __init
---
changes in v5:
- check the node using the Xen domain binding whether contains the property
"xen,static-mem", not the property itself
- add "rc = ..." to get the error propagated
- introduce new field "bool xen_domain", then static memory shall be also stored
as reserved memory(bootinfo.reserved_mem), but being bind to one
specific Xen domain node.
- make CONFIG_STATIC_MEMORY user selectable and gated by UNSUPPORTED.
- wrap all static-allocation-related codes with CONFIG_STATIC_MEMORY
even in arm-specific file.
- make bank_start/bank_end type of mfn_t, and rename bank_size to
bank_pages.
- Having both functions assign_pages/assign_page with similar parameter
arrangement
- bundle all the functions for static allocation in a single place
- return an error and revert the changes, when the page is not free
and reserved.
- check the MFN is valid for every page and also add a comment to warn
that this function needs to be reworked if used outside of boot.
- use less of mfn_to_page/page_to_mfn
- use ASSERT_UNREACHABLE() to also check that the two flags are clear
- pass the start MFN first and then the number of pages in both
acquire_staticmem_pages and acquire_domstatic_pages
- make acquire_domstatic_pages() to return an errno
- don't split comment over multi-line (even they are more than 80 characters)
- simply use dt_find_property(node, "xen,static-mem", NULL) to tell
whether using allocate_static_memory, and add error comment when
"xen,static-mem" is used but CONFIG_STATIC_MEMORY is not enabled.
- exporting p2m_insert_mapping() and introduce guest_physmap_add_pages
to cope with adding guest RAM p2m mapping with nr_pages.
- check both pbase and psize are page aligned
- simplify the code in the loops by moving append_static_memory_to_bank()
outside of the if/else.
---
change v6:
- move address_cells and size_cells check into device_tree_get_meminfo
- doc and comment refinement
- indentation, and the kconfig message fix
- let assign_page have a different order for the arguments, not assign_pages
- fix variable declaration and some format
- use PFN_DOWN(...)
- provide a stub allocate_static_memory implementation for
!CONFIG_STATIC_MEMORY
- add a statement to clarify the binding between the "memory" property
and "xen,static-mem"
---
TODO:
- reboot domain on static allocation
- Implement all memory-ops(hypercalls) regarding domain on static allocation
to balloon in/out memory
- asynchronously scrubbing PGC_reserved pages

Penny Zheng (7):
  xen/arm: introduce new helper device_tree_get_meminfo
  xen/arm: introduce domain on Static Allocation
  xen: introduce mark_page_free
  xen/arm: static memory initialization
  xen: re-define assign_pages and introduce a new function assign_page
  xen/arm: introduce acquire_staticmem_pages and acquire_domstatic_pages
  xen/arm: introduce allocate_static_memory

 docs/misc/arm/device-tree/booting.txt |  39 ++++
 xen/arch/arm/bootfdt.c                | 108 +++++++----
 xen/arch/arm/domain_build.c           | 160 ++++++++++++++++-
 xen/arch/arm/p2m.c                    |   7 +-
 xen/arch/arm/setup.c                  |  27 +++
 xen/arch/x86/pv/dom0_build.c          |   2 +-
 xen/common/Kconfig                    |  13 ++
 xen/common/grant_table.c              |   2 +-
 xen/common/memory.c                   |   6 +-
 xen/common/page_alloc.c               | 249 ++++++++++++++++++++------
 xen/include/asm-arm/mm.h              |   3 +
 xen/include/asm-arm/p2m.h             |  11 ++
 xen/include/asm-arm/setup.h           |   1 +
 xen/include/xen/mm.h                  |  14 ++
 14 files changed, 544 insertions(+), 98 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181612.328822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHk-0001Ep-EE; Wed, 08 Sep 2021 09:54:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181612.328822; Wed, 08 Sep 2021 09:54:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHk-0001Eg-Ae; Wed, 08 Sep 2021 09:54:28 +0000
Received: by outflank-mailman (input) for mailman id 181612;
 Wed, 08 Sep 2021 09:54:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuHj-0000bc-Gg
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:27 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.66]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 710e629f-b153-4d48-945e-6e5213475b1e;
 Wed, 08 Sep 2021 09:54:19 +0000 (UTC)
Received: from AM5PR0701CA0002.eurprd07.prod.outlook.com
 (2603:10a6:203:51::12) by AM6PR08MB4504.eurprd08.prod.outlook.com
 (2603:10a6:20b:bd::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.20; Wed, 8 Sep
 2021 09:54:16 +0000
Received: from VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:51:cafe::80) by AM5PR0701CA0002.outlook.office365.com
 (2603:10a6:203:51::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.6 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT052.mail.protection.outlook.com (10.152.19.173) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:16 +0000
Received: ("Tessian outbound b48d04bf8afd:v105");
 Wed, 08 Sep 2021 09:54:15 +0000
Received: from c44f5e52554f.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 51AD65E1-3B5B-43CE-846D-6B3460ACD56B.1; 
 Wed, 08 Sep 2021 09:54:09 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c44f5e52554f.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:09 +0000
Received: from AM6PR01CA0048.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::25) by AM8PR08MB6611.eurprd08.prod.outlook.com
 (2603:10a6:20b:36b::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:08 +0000
Received: from VE1EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:e0:cafe::32) by AM6PR01CA0048.outlook.office365.com
 (2603:10a6:20b:e0::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:08 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT055.mail.protection.outlook.com (10.152.19.158) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:08 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:12 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 710e629f-b153-4d48-945e-6e5213475b1e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=YapclBF9JHWSw1oLOXUWmwDXkDDlWaTR49bkTFpbeJwIZ1rBBqJNxXhnDAI/JAoo8p9yWtJd66enr4DcSMlOe/QFb5pEMqp33UshvsdOIhkkqTzO8ki0m//TZ7jD1rQjgWzo9TBbIQhYKEsI5/dYjTT9YzCtOmGXVnmamuSVO7g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3fd3d9752e95c7b8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AKK9ez2QUc/A0Qvnnjmi4Qm9SDSLvrtWPe+seb4l/zxrcXrG30OjJQ64i6vJYHeaC2iatcvOOJ3CDqaz0euepEl1TNfRh8h9Sx655n9qSm2Lh8dvfO6zVJ5nS6pp+/tgsBpL/qPjSVV/QPK83luX6+cwK2M5Jdx62//a4kzBodXult0+CVdsHrxaOZqIhOFdqRyWoNMmtJBei61npWbHIG03wyzriWoUS//pM/hmVUS93Do5ThJb0yKQBMOZqBWjacmanM04ojRf+kHpNGby75j64IeO9hVZkolOpee3sgsqcoMxNzLp7UpONC3xD+EHAu8DnW1ayQhBB1fC7s+nsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=jQetPhEu9LIt/pyh59/vFu+WVq1RQJpUb6ZN9XplNmNPv124XFLeAka7LjQjrye0q0bvB2wSufie64zreRzVaGGdPFlUCKTXrMowvssfCru2uJZd+ex0IaQXBE7FgamfsQC8XjiQFj4bqD5iBUAELP7UZ9i7fYZdu/9MFCPIdWTR4KSVgBnweH+n+oY9ehERfWoH4GYptXG/9sX3IqX+S+ArFsfvE2bfEoU/cMqH6+Pq1Z5CPeI4ctev072j9EZUY8DH95w8HN1fQjkNNWhZkqqyqeHGTlm0Is201fIvv+ZVD5cUBBkviuk172rufyZbKW8Ni4s0xFAX138pir+fXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=YapclBF9JHWSw1oLOXUWmwDXkDDlWaTR49bkTFpbeJwIZ1rBBqJNxXhnDAI/JAoo8p9yWtJd66enr4DcSMlOe/QFb5pEMqp33UshvsdOIhkkqTzO8ki0m//TZ7jD1rQjgWzo9TBbIQhYKEsI5/dYjTT9YzCtOmGXVnmamuSVO7g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 3/7] xen: introduce mark_page_free
Date: Wed, 8 Sep 2021 09:52:44 +0000
Message-ID: <20210908095248.545981-4-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9ee270c3-d84b-46e8-5064-08d972ae9eda
X-MS-TrafficTypeDiagnostic: AM8PR08MB6611:|AM6PR08MB4504:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB45041B7AF0D322F8B3508F8CF7D49@AM6PR08MB4504.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1051;OLM:1051;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Pj+BKWgtu/2rQ4dGzigUQfEgqbZ3it7nn01d8j16dP4g9B4AlucmsxNu1vKm0s2lXEk/gazxU3iY+lNYRgf2VrRwvl+5MRUjmnQuZ9+cLpddFf5scxs1xINxL9MF/YIv2kO3ApIElTz+yyVqWcan2kw3f7NU1t2REedGknDeCh/BL5B8ANkUiIro/wXyyuADOhmFjseYOTBkU8wOV51f1PEoouQyCHdSBIRUE+nAfcsBDCrWWxPKeh4PNJLYPL3WYkfQEBXkZ543wZEBKMMzpHCa6xqWpP510jVerubGYxbmnlaJSbPGBizoHtvALRsgflAhPlATYMso6R30dHddZQAwtXWQh6pvgcTiL89xfX+Q/bq83MW0QtCSoUnrJotPZ0E/r6X02HIjtHBP//5V9cREo6vZYXjP011aNPajxQ+M62iWh698aNs8l9o4dbINpXYzZZb+uEB4Ff4zrjVTuwLe5/B8GSv/swJs9vMu1E14hjbD5/vNO3oh/zQ+0CRhS9SBdEgJxJ+Hz4NR3gaBYZ0M+c5nZttQ3iirE/BuqqTAKKdpdQhN9EPECERIAH6MjMzodp8Zo5DUCaCsFg0aMmUM/mdRUw+GtXB4VwC0oiAlj6nEa6yXsk+xOcV2+meQ9wIsTJprIRXA1ez+vAhH1sMuCEIjEV2jx09Tv4KIJJ27o/trv1NLNgl1I7E+icJVA56jUx2PKa0T2cFJMXktzTm8sTatW75UbHCs1IMHgpU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(81166007)(8936002)(426003)(4326008)(316002)(5660300002)(336012)(1076003)(8676002)(47076005)(2906002)(356005)(70586007)(70206006)(83380400001)(36756003)(26005)(110136005)(186003)(86362001)(36860700001)(7696005)(44832011)(82310400003)(6666004)(54906003)(508600001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6611
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e34ca84a-a1f7-46e3-1735-08d972ae99f2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Txja8jyXeoLQrPOBX1Lv9eB7y+rdXz+wonQN2hXo8IO68DniA/prBGZfUxZ7mkeaSdXOX4krcOhVNmJwIRcYO7u996eLlgrGasxbVOBc8mnYtgJAf+t0h8B+86TLi4iMXfq5BR7YBt4+s1Jmd9xPbtTiyxODkovfbwYBsp00FH+vSL0zndO4JUihmgR60dTNdThuj2qJpPjiMOQcizzYFd+YC1WOrDsoIJhzcyQRciZwlmBE1NeXfg8HBUd6YgcvzsVfq5M5jWjDUmz776NFN6Ga2nQnYUT4H7/9NidFMABnvHQ+aurF9smuBuztpHYRHUy/Vg47P7QSkIuP8fIsr0h6Rb3YcbHcimvmDOWh4iWfvRcxu0IJffhDP43sHtYvHTZBLeRDIfjs42pkFYEZAHgzjn4dflVY5S29Supew0cYBafgFSS5DyNRBF0xjVJe9haBDB3BAdHSOtpt96mEWzW9GYgD4xxZLOIjkaQx2P7pmXPaggoSyoptM7Mxu8QXqr4XoZAwc9yZQMnNXgD+P9+2xc6P3gCuD0zGPPLPe+9Ean4xxQc+MHa98F6yqqQL+RqzRHR27C9YKJfNxv/n5zWTHNC9wdLx7czrYvCbZl06t/1RJIUWMpXT3GdkUwOcBKAekuPPk3LAT/3XWscmxKufuno9G4Kchq/Doh83V50bbc9n7dSMWzkOZnSDkJuhw1veMW4nCSMXelqPcjSV5w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(136003)(376002)(39860400002)(396003)(36840700001)(46966006)(186003)(5660300002)(2616005)(107886003)(83380400001)(86362001)(36860700001)(1076003)(82310400003)(47076005)(6666004)(36756003)(81166007)(478600001)(82740400003)(336012)(70586007)(426003)(8936002)(2906002)(4326008)(316002)(26005)(7696005)(44832011)(54906003)(70206006)(8676002)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:16.3489
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ee270c3-d84b-46e8-5064-08d972ae9eda
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4504

This commit defines a new helper mark_page_free to extract common code,
like following the same cache/TLB coherency policy, between free_heap_pages
and the new function free_staticmem_pages, which will be introduced later.

The PDX compression makes that conversion between the MFN and the page can
be potentially non-trivial. As the function is internal, pass the MFN and
the page. They are both expected to match.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 xen/common/page_alloc.c | 89 ++++++++++++++++++++++-------------------
 1 file changed, 48 insertions(+), 41 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 958ba0cd92..a3ee5eca9e 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1376,6 +1376,53 @@ bool scrub_free_pages(void)
     return node_to_scrub(false) != NUMA_NO_NODE;
 }
 
+static void mark_page_free(struct page_info *pg, mfn_t mfn)
+{
+    ASSERT(mfn_x(mfn) == mfn_x(page_to_mfn(pg)));
+
+    /*
+     * Cannot assume that count_info == 0, as there are some corner cases
+     * where it isn't the case and yet it isn't a bug:
+     *  1. page_get_owner() is NULL
+     *  2. page_get_owner() is a domain that was never accessible by
+     *     its domid (e.g., failed to fully construct the domain).
+     *  3. page was never addressable by the guest (e.g., it's an
+     *     auto-translate-physmap guest and the page was never included
+     *     in its pseudophysical address space).
+     * In all the above cases there can be no guest mappings of this page.
+     */
+    switch ( pg->count_info & PGC_state )
+    {
+    case PGC_state_inuse:
+        BUG_ON(pg->count_info & PGC_broken);
+        pg->count_info = PGC_state_free;
+        break;
+
+    case PGC_state_offlining:
+        pg->count_info = (pg->count_info & PGC_broken) |
+                         PGC_state_offlined;
+        tainted = 1;
+        break;
+
+    default:
+        printk(XENLOG_ERR
+               "pg MFN %"PRI_mfn" c=%#lx o=%u v=%#lx t=%#x\n",
+               mfn_x(mfn),
+               pg->count_info, pg->v.free.order,
+               pg->u.free.val, pg->tlbflush_timestamp);
+        BUG();
+    }
+
+    /* If a page has no owner it will need no safety TLB flush. */
+    pg->u.free.need_tlbflush = (page_get_owner(pg) != NULL);
+    if ( pg->u.free.need_tlbflush )
+        page_set_tlbflush_timestamp(pg);
+
+    /* This page is not a guest frame any more. */
+    page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
+    set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
+}
+
 /* Free 2^@order set of pages. */
 static void free_heap_pages(
     struct page_info *pg, unsigned int order, bool need_scrub)
@@ -1392,47 +1439,7 @@ static void free_heap_pages(
 
     for ( i = 0; i < (1 << order); i++ )
     {
-        /*
-         * Cannot assume that count_info == 0, as there are some corner cases
-         * where it isn't the case and yet it isn't a bug:
-         *  1. page_get_owner() is NULL
-         *  2. page_get_owner() is a domain that was never accessible by
-         *     its domid (e.g., failed to fully construct the domain).
-         *  3. page was never addressable by the guest (e.g., it's an
-         *     auto-translate-physmap guest and the page was never included
-         *     in its pseudophysical address space).
-         * In all the above cases there can be no guest mappings of this page.
-         */
-        switch ( pg[i].count_info & PGC_state )
-        {
-        case PGC_state_inuse:
-            BUG_ON(pg[i].count_info & PGC_broken);
-            pg[i].count_info = PGC_state_free;
-            break;
-
-        case PGC_state_offlining:
-            pg[i].count_info = (pg[i].count_info & PGC_broken) |
-                               PGC_state_offlined;
-            tainted = 1;
-            break;
-
-        default:
-            printk(XENLOG_ERR
-                   "pg[%u] MFN %"PRI_mfn" c=%#lx o=%u v=%#lx t=%#x\n",
-                   i, mfn_x(mfn) + i,
-                   pg[i].count_info, pg[i].v.free.order,
-                   pg[i].u.free.val, pg[i].tlbflush_timestamp);
-            BUG();
-        }
-
-        /* If a page has no owner it will need no safety TLB flush. */
-        pg[i].u.free.need_tlbflush = (page_get_owner(&pg[i]) != NULL);
-        if ( pg[i].u.free.need_tlbflush )
-            page_set_tlbflush_timestamp(&pg[i]);
-
-        /* This page is not a guest frame any more. */
-        page_set_owner(&pg[i], NULL); /* set_gpfn_from_mfn snoops pg owner */
-        set_gpfn_from_mfn(mfn_x(mfn) + i, INVALID_M2P_ENTRY);
+        mark_page_free(&pg[i], mfn_add(mfn, i));
 
         if ( need_scrub )
         {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181613.328834 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHl-0001WP-QP; Wed, 08 Sep 2021 09:54:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181613.328834; Wed, 08 Sep 2021 09:54:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHl-0001VS-LG; Wed, 08 Sep 2021 09:54:29 +0000
Received: by outflank-mailman (input) for mailman id 181613;
 Wed, 08 Sep 2021 09:54:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FUO+=N6=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNuHk-0001EI-5d
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:28 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c0d5fdd8-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0d5fdd8-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631094867;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=PDBj/mA8Lu6e2IhZjHrH6EN0R60DBQb8n23okxDMJvY=;
  b=adEy6LxjdRHGd/wTBJNudWVup26YVrWmBZcWXd6oJiERwIWhdqDGepq7
   /C0YnIrNj7OvhkQ4fDvKD8cA1OxC7xIiUqEf2XNEYOMQE+9b+7Lw4rOQR
   yOuksf02F2I0vlOnOg5RLkN8ZrimwzzdDBwCGDaMdr1t3j0lIp9cICD++
   8=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 9hJzvvNN+HDUTSRgGrqP7csVUq9njft5zD/0wcM2reORepBNmRYH54XkhC/IaQPSZO3KjwEKA2
 5kNFrhbq5ILf0VgImqSQ/Obt3cVy+WVtoRoOleKRsAWimzGSgEBsYhkhgrLUb1DTIjjihxpNSt
 6czpK3hgivc+TlAPAc29kRZo43ouizWsK/ILM8OC8Sk3+QAZeqRXIOXWMAYUzajGxChxSv409z
 RS02wwNBDbnz9gH4a3hy/fG1KEyssh65v97oYE9U45Q1MbgQTcR++tAV5oU7mJZO7ehvA6QWvi
 6W4m2GMga8UsZS4QRhjkVjpn
X-SBRS: 5.1
X-MesageID: 53993474
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:opGXYqACNyXpAYXlHemm55DYdb4zR+YMi2TC1yhKJiC9Ffbo8v
 xG/c5rsiMc5wxxZJhNo7290cq7MBHhHPxOgbX5VI3KNGKNhILBFvAH0WKI+VPd8kPFmtK1rZ
 0QEJRDNA==
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="53993474"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Anthony PERARD <anthony.perard@citrix.com>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, "Daniel De
 Graaf" <dgdegra@tycho.nsa.gov>, George Dunlap <george.dunlap@citrix.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [XEN PATCH 0/2] build: attempt to "fix" GitLab CI job "debian-unstable-gcc-arm64"
Date: Wed, 8 Sep 2021 10:54:20 +0100
Message-ID: <20210908095422.438324-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.fix-flask-policy.mk-v1

Well, that job fails with weird failure:
    make[6]: Entering directory '/builds/xen-project/xen/tools/flask/policy'
    make[6]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
    /builds/xen-project/xen/tools/flask/policy/Makefile.common:115: *** target pattern contains no '%'.  Stop.
    make[6]: Leaving directory '/builds/xen-project/xen/tools/flask/policy'

This is because we do $(shell $(MAKE) xenversion) and that sometime produce
more than just "4.16-unstable", which lead to makefile parsing failure.
Maybe `make -j80` is too much.

But you can have the same kind of failure when running `make --debug=a` or
`remake --trace` with flask policy generation enabled.

This patch series fixes build of "xen" (first patch), but only workaround with
a probably not good enough patch for build of the "tools".

Anthony PERARD (2):
  build: set policy filename on make command line
  build: add --full to version.sh to guess $(XEN_FULLVERSION)

 tools/flask/policy/Makefile.common |  2 +-
 version.sh                         | 15 ++++++++++++++-
 xen/xsm/flask/Makefile             |  2 +-
 3 files changed, 16 insertions(+), 3 deletions(-)

-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181615.328844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHr-00021Y-4y; Wed, 08 Sep 2021 09:54:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181615.328844; Wed, 08 Sep 2021 09:54:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHr-00021L-0l; Wed, 08 Sep 2021 09:54:35 +0000
Received: by outflank-mailman (input) for mailman id 181615;
 Wed, 08 Sep 2021 09:54:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FUO+=N6=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNuHp-0001EI-4K
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:33 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c177d7de-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c177d7de-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631094868;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=AsZBJR8DqTXnM0J0C60Tfl9G1/1Dykb1qhx3aXLPQKA=;
  b=XFhnteq3B7TK12Iq7u47AlagKCEKuLPmZeTIteKciX2sJ2F7XbkOaaho
   0mkFKRdswyzlbxRj3WRpXHLhUFzFkKylXWqhw7dbWaVdPqkWVDHA85Ti4
   tHUd5RGfaSEg26wuJdVDayBm4O/t97R0PLvHgsf2+45G81oR0SlKJ5QlV
   g=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 5+r2TLm/n1r96Gjv2CZirI56HcpkfJGxmXSw1x1oAV2t8202ESGWCBYciVSTu1jDtTh5ZnXAhd
 +RuMZxzI+C06I7+2UGajzNuSPGRA+Ewv8wqQMEG5P76gdLwg7CC3azHyEdGu1Th6t+44fDdVcz
 56LHITxzjQbHwqXB6f/PZnQZvyHuwqoPJlNBnDlCJu46gL0HAKQa0AVc/RCA/YK9ZEjxHBqaS4
 8dbD83KwdFvvMv3AHliFmoBdH83FODY0MWG8NIQvM5M15SGv52813EGVj7pPFhhKSkRtQUfhTO
 BpwP/FEuN3G5HPNICzwNvt6i
X-SBRS: 5.1
X-MesageID: 51835828
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:KULzgKmXcDch0cVoHxuZlxF/ccrpDfLW3DAbv31ZSRFFG/Fw9/
 rCoB3U73/JYVcqKRUdcLW7UpVoLkmyyXcY2+cs1NSZLWzbUQmTXeJfBOLZqlWNJ8SXzIVgPM
 xbAspD4bPLbGSTjazBkXSF+9RL+qj6zEh/792usEuETmtRGt9dBx8SMHf9LqXvLjM2fqbQEv
 Cnl6x6jgvlQ1s7ROKhCEIIWuDSzue77q4PMXY9dmcaABDlt0LR1ILH
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="51835828"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Anthony PERARD <anthony.perard@citrix.com>, "Daniel De
 Graaf" <dgdegra@tycho.nsa.gov>, "Daniel P. Smith"
	<dpsmith@apertussolutions.com>
Subject: [XEN PATCH 1/2] build: set policy filename on make command line
Date: Wed, 8 Sep 2021 10:54:21 +0100
Message-ID: <20210908095422.438324-2-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210908095422.438324-1-anthony.perard@citrix.com>
References: <20210908095422.438324-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

In order to avoid flask/Makefile.common calling `make xenversion`, we
override POLICY_FILENAME with the value we are going to use anyway.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/xsm/flask/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 637159ad8276..093ee50bf1c8 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -42,7 +42,7 @@ FLASK_BUILD_DIR := $(CURDIR)
 POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
 
 policy.bin: FORCE
-	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C $(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
+	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C $(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)
 	cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
 
 .PHONY: clean
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181616.328856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHu-0002Sq-Gj; Wed, 08 Sep 2021 09:54:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181616.328856; Wed, 08 Sep 2021 09:54:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHu-0002SN-C5; Wed, 08 Sep 2021 09:54:38 +0000
Received: by outflank-mailman (input) for mailman id 181616;
 Wed, 08 Sep 2021 09:54:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuHt-0000bc-H8
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:37 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.63]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eece8e26-0e47-4435-b533-0948220731fa;
 Wed, 08 Sep 2021 09:54:28 +0000 (UTC)
Received: from AM5PR0602CA0007.eurprd06.prod.outlook.com
 (2603:10a6:203:a3::17) by DB9PR08MB6762.eurprd08.prod.outlook.com
 (2603:10a6:10:2a8::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:26 +0000
Received: from AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:a3:cafe::f9) by AM5PR0602CA0007.outlook.office365.com
 (2603:10a6:203:a3::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT037.mail.protection.outlook.com (10.152.17.241) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:26 +0000
Received: ("Tessian outbound b9872a30149f:v105");
 Wed, 08 Sep 2021 09:54:25 +0000
Received: from f70b2a647f00.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 75238AC5-D489-4D20-9685-0146AADA106F.1; 
 Wed, 08 Sep 2021 09:54:20 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f70b2a647f00.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:20 +0000
Received: from PR0P264CA0191.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::35)
 by AM6PR08MB3623.eurprd08.prod.outlook.com (2603:10a6:20b:48::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Wed, 8 Sep
 2021 09:54:18 +0000
Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::f) by PR0P264CA0191.outlook.office365.com
 (2603:10a6:100:1c::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:18 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:18 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:20 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eece8e26-0e47-4435-b533-0948220731fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ulrPrQnKNKHeTjBqMvFiBcaLOSg+pt3ZsckmpTE4xGg=;
 b=NhfkiD374f1cloiY9O5ZepaoDxfL6UQl6/Cc71E02QObnBD9V2Eud+a9vo394hrzw9y1mjjMK+/87UDILLF3KNy+EmfSTNDbgXBKds8IQyK2xz9/oLpD2PqZ+QSbvygGs1eRnPjOtDEZIgftBJIW8IH+CPrIOpe26AreZZrAbdU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 23d4e0f8689de261
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HjW+45NasmLGFrCPkiko7oX/Z2tpz6BEGB4dIph9QbnzcK08vthBamEcvApL2p6IommH16cBY71+TNrvIBRfaUbFyoc5BmH94NGUKr9yBvttnkBRz7+UPEDjy3+nw3dHAhg3kl3ZNv27gA3KX8Kz0fzompH8PDcXxdxa2duEnd20VBA8od9J++PHAdhFFM6qCvNQ5D38w1pVtSuMLllMEBcS5KfHRqIDV6tpFpWYRZAUkYccUC9RzLB0XVNLX8h/o/76EDowAuFA/kFVx+9ecadpd/TA3YA4AkE2KpYTRa1ueLfevT8+Yoc78nRWyGMPMXMswLOlGhdP0c7iT2l3MA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ulrPrQnKNKHeTjBqMvFiBcaLOSg+pt3ZsckmpTE4xGg=;
 b=goRlHzMSp/PDdpVqfzCXTO5VSoUCXPWfS2tdz+Cyb3IKzkSkwIF3c5p1eTqUVpglY9EkHBgMjqK0fisXUq2j5StfC2wcfTQAKulVZSkhus0Tso5oInPVs042ESnZ4qZVrKPYl9E4hAV9MyXgyXe7xsC+A1qKGJwZPrby+F9gr2g6xmSYN4giKawfGWn7sL0wCZnToZCChsHUKWbcXUJg2esM8s383+puoHho2xXH/TOt3kcTP8xj07SUIJ9UYVPKYaC4C8gJmyQKfI/MuAEQyqIGVDUmzzqyFzuq13jg6+lR/Rex2RpCTYIJk/lsuHFO0lOvH7aUuG2yJc8OtKD37w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ulrPrQnKNKHeTjBqMvFiBcaLOSg+pt3ZsckmpTE4xGg=;
 b=NhfkiD374f1cloiY9O5ZepaoDxfL6UQl6/Cc71E02QObnBD9V2Eud+a9vo394hrzw9y1mjjMK+/87UDILLF3KNy+EmfSTNDbgXBKds8IQyK2xz9/oLpD2PqZ+QSbvygGs1eRnPjOtDEZIgftBJIW8IH+CPrIOpe26AreZZrAbdU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 6/7] xen/arm: introduce acquire_staticmem_pages and acquire_domstatic_pages
Date: Wed, 8 Sep 2021 09:52:47 +0000
Message-ID: <20210908095248.545981-7-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ce590b9e-63d8-459c-867d-08d972aea4a7
X-MS-TrafficTypeDiagnostic: AM6PR08MB3623:|DB9PR08MB6762:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB67629BF90E7DEE5B7827FB56F7D49@DB9PR08MB6762.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 nl8jk9LRhIMcN6l2ckGpEixtbCGp8JTWoblfBW+orT5kCMwhRPKY2Rv2p+FIhdNObOFK19X+99rgk8ntJ9Tj1YwVuVwBOzVo98HnEjvMueevO7+jofecUZHcpAxRerkHUkRrOotJIHHODe+RxkJ8HVNI5DTpZifHBgbkHw3lqu2IXO15Z9xPqG4ijWUStS02dIKCNA7c8BNNHYnUBG2iS98CVtvCKZopYORRXH1OWhKbPS1lD2zysEs0xN4ncYvf8xCuy3QmxDGehoPH7zf4Fd0yRVb8KJOUe+8So6Heh/EdpYmvcG7OdRR15wtLnHZwzszX6cF6Kqtk23hMtZHnFvW5EfinQlyNYO63bRuMikS64gZgXzXeu4f83rzrBZC9Lmu/zsL2bY+23cg4vTUGHkd67GstNeDYPYoNsn+b6kqi0Qz3HGfzf1sHV5fFZmToIB/z+vcXDhOxKjCTdxxkna8tzbL0QelSTfNeQ/jbyCgT1r2VkMxIbfDOIvgTZoxCRFqhd4Z2/1vkBfLflYiD3w3/s/E7NrMebORUjsqq+Rzq9jQdNAmgLaO2jARzAt5ZjrdyXm8+GOgEbzuBXvI6Zt6eGUKRp8pwkgimjnBBght92fiYHou9+P8DosWgWFm2Ak7/lOTBoN+Yha/upgE5urHDMPSP2PVg+V732U64M34Ma7aJ1MLo+j/+g4sEYkd7JAEZDx5GuDSKKDHQ0i2mVbWmXwxEmtKQ3MkT3XNrdgE=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(110136005)(316002)(81166007)(356005)(54906003)(7696005)(6666004)(70586007)(2616005)(1076003)(508600001)(336012)(426003)(70206006)(186003)(4326008)(8936002)(2906002)(26005)(5660300002)(8676002)(36860700001)(86362001)(36756003)(44832011)(47076005)(82310400003)(83380400001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3623
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d56d9f29-9e71-4677-8119-08d972ae9fe7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Nv23PaB9KOLZkqXBH6qg7uco6r2/L5UmqnXw9D6ODSpa+0csl+0MpfWUz0nqvVfdNrd6qJ7RidszSiEgTd4CVUApqz0bPdoLzc54LK3jvTpic9iKd45Wiq1gv/ZO9HuiJXYmXyK5MO6hJgBvKHcgwcyhfsCWWVmQkcLUDG7SkLjeAZ009jiEIzDkZB7uHncY5Zy2Y2TpXbNe6JlinNekQQj8tFmThfLoMfYlMmIXN5OqPvOaoS6HQYzXaZVRwtoG6yGmp2UW7UyiHXgMNGRgORBnbugjUCqkQP7CjAvKMgm+3u5ELKopZmFnMc46P4GZ1fSgyr+TM0kaD3RVmiuddsUICRtHcCVgpY1oQJ08t+92quvBDqxjGZZ597k9mnVMeO7yMf4rqj36HBKtAMC2Tk6WFPKSuROlQE51IdWYkBOtme9CK0JLkNBcroAGBeOEh9d7vwD3BksolJBUfV7yYxlmBguwvirLGC7uh21R8Pg2NKD87WqffIuYA4RN1LQVeknz7o/OXs0MBePYer4bHp9z1Tgr+xaWz/2BvJQzFxcUDYEj94iWL6t3uw01wcAjPrJmzcC08ostH/3mk+MWpO1ADgovKn7U1eI6Tc5D58A/+3aXRNRnACcGY2b9X64mO9g+UoWXO2s3BhBjjn2WfIvXL7Z7ctxPEut2SJVhDHDUo4CPxQ9f/FmqiEBLOME1DCnLfLvlzDU3FAExh9VSxw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(346002)(39860400002)(376002)(396003)(46966006)(36840700001)(81166007)(8936002)(8676002)(70206006)(70586007)(2616005)(7696005)(36756003)(26005)(336012)(426003)(316002)(2906002)(4326008)(47076005)(54906003)(5660300002)(36860700001)(82740400003)(6666004)(110136005)(478600001)(186003)(86362001)(1076003)(83380400001)(44832011)(107886003)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:26.1435
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ce590b9e-63d8-459c-867d-08d972aea4a7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6762

New function acquire_staticmem_pages aims to acquire nr_mfns contiguous pages
of static memory, starting at #smfn. And it is the equivalent of
alloc_heap_pages for static memory.

For each page, it shall check if the page is reserved(PGC_reserved)
and free. It shall also do a set of necessary initialization, which are
mostly the same ones in alloc_heap_pages, like, following the same
cache-coherency policy and turning page status into PGC_state_inuse, etc.

New function acquire_domstatic_pages is the equivalent of alloc_domheap_pages
for static memory, and it is to acquire nr_mfns contiguous pages of
static memory and assign them to one specific domain.

It uses acquire_staticmem_pages to acquire nr_mfns pages of static memory.
Then on success, it will use assign_pages to assign those pages to one
specific domain.

In order to differentiate pages of static memory from those allocated from
heap, this patch introduces a new page flag PGC_reserved, then mark pages of
static memory PGC_reserved when initializing them.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/common/page_alloc.c  | 118 ++++++++++++++++++++++++++++++++++++++-
 xen/include/asm-arm/mm.h |   3 +
 xen/include/xen/mm.h     |   2 +
 3 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index bb19bb10ff..59dffcfa1d 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -151,6 +151,10 @@
 #define p2m_pod_offline_or_broken_replace(pg) BUG_ON(pg != NULL)
 #endif
 
+#ifndef PGC_reserved
+#define PGC_reserved 0
+#endif
+
 /*
  * Comma-separated list of hexadecimal page numbers containing bad bytes.
  * e.g. 'badpage=0x3f45,0x8a321'.
@@ -2283,7 +2287,7 @@ int assign_pages(
 
         for ( i = 0; i < nr; i++ )
         {
-            ASSERT(!(pg[i].count_info & ~PGC_extra));
+            ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
             if ( pg[i].count_info & PGC_extra )
                 extra_pages++;
         }
@@ -2322,7 +2326,8 @@ int assign_pages(
         page_set_owner(&pg[i], d);
         smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
         pg[i].count_info =
-            (pg[i].count_info & PGC_extra) | PGC_allocated | 1;
+            (pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated | 1;
+
         page_list_add_tail(&pg[i], page_to_list(d, &pg[i]));
     }
 
@@ -2626,8 +2631,117 @@ void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
             /* TODO: asynchronous scrubbing for pages of static memory. */
             scrub_one_page(pg);
         }
+
+        /* In case initializing page of static memory, mark it PGC_reserved. */
+        pg[i].count_info |= PGC_reserved;
     }
 }
+
+/*
+ * Acquire nr_mfns contiguous reserved pages, starting at #smfn, of
+ * static memory.
+ * This function needs to be reworked if used outside of boot.
+ */
+static struct page_info * __init acquire_staticmem_pages(mfn_t smfn,
+                                                         unsigned long nr_mfns,
+                                                         unsigned int memflags)
+{
+    bool need_tlbflush = false;
+    uint32_t tlbflush_timestamp = 0;
+    unsigned long i;
+    struct page_info *pg;
+
+    ASSERT(nr_mfns);
+    for ( i = 0; i < nr_mfns; i++ )
+        if ( !mfn_valid(mfn_add(smfn, i)) )
+            return NULL;
+
+    pg = mfn_to_page(smfn);
+
+    spin_lock(&heap_lock);
+
+    for ( i = 0; i < nr_mfns; i++ )
+    {
+        /* The page should be reserved and not yet allocated. */
+        if ( pg[i].count_info != (PGC_state_free | PGC_reserved) )
+        {
+            printk(XENLOG_ERR
+                   "pg[%lu] Static MFN %"PRI_mfn" c=%#lx t=%#x\n",
+                   i, mfn_x(smfn) + i,
+                   pg[i].count_info, pg[i].tlbflush_timestamp);
+            goto out_err;
+        }
+
+        if ( !(memflags & MEMF_no_tlbflush) )
+            accumulate_tlbflush(&need_tlbflush, &pg[i],
+                                &tlbflush_timestamp);
+
+        /*
+         * Preserve flag PGC_reserved and change page state
+         * to PGC_state_inuse.
+         */
+        pg[i].count_info = PGC_reserved | PGC_state_inuse;
+        /* Initialise fields which have other uses for free pages. */
+        pg[i].u.inuse.type_info = 0;
+        page_set_owner(&pg[i], NULL);
+    }
+
+    spin_unlock(&heap_lock);
+
+    if ( need_tlbflush )
+        filtered_flush_tlb_mask(tlbflush_timestamp);
+
+    /*
+     * Ensure cache and RAM are consistent for platforms where the guest
+     * can control its own visibility of/through the cache.
+     */
+    for ( i = 0; i < nr_mfns; i++ )
+        flush_page_to_ram(mfn_x(smfn) + i, !(memflags & MEMF_no_icache_flush));
+
+    return pg;
+
+ out_err:
+    while ( i-- )
+        pg[i].count_info = PGC_reserved | PGC_state_free;
+
+    spin_unlock(&heap_lock);
+
+    return NULL;
+}
+
+/*
+ * Acquire nr_mfns contiguous pages, starting at #smfn, of static memory,
+ * then assign them to one specific domain #d.
+ */
+int __init acquire_domstatic_pages(struct domain *d, mfn_t smfn,
+                                   unsigned long nr_mfns, unsigned int memflags)
+{
+    struct page_info *pg;
+
+    ASSERT(!in_irq());
+
+    pg = acquire_staticmem_pages(smfn, nr_mfns, memflags);
+    if ( !pg )
+        return -ENOENT;
+
+    if ( !d || (memflags & (MEMF_no_owner | MEMF_no_refcount)) )
+    {
+        /*
+         * Respective handling omitted here because right now
+         * acquired static memory is only for guest RAM.
+         */
+        ASSERT_UNREACHABLE();
+        return -EINVAL;
+    }
+
+    if ( assign_pages(d, pg, nr_mfns, memflags) )
+    {
+        free_staticmem_pages(pg, nr_mfns, memflags & MEMF_no_scrub);
+        return -EINVAL;
+    }
+
+    return 0;
+}
 #endif
 
 /*
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index ded74d29da..7b5e7b7f69 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -108,6 +108,9 @@ struct page_info
   /* Page is Xen heap? */
 #define _PGC_xen_heap     PG_shift(2)
 #define PGC_xen_heap      PG_mask(1, 2)
+  /* Page is reserved */
+#define _PGC_reserved     PG_shift(3)
+#define PGC_reserved      PG_mask(1, 3)
 /* ... */
 /* Page is broken? */
 #define _PGC_broken       PG_shift(7)
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index a74e93eba8..da1b158693 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -89,6 +89,8 @@ bool scrub_free_pages(void);
 /* These functions are for static memory */
 void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
                           bool need_scrub);
+int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned long nr_mfns,
+                            unsigned int memflags);
 #endif
 
 /* Map machine page range in Xen virtual address space. */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181617.328867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHw-0002pC-7Q; Wed, 08 Sep 2021 09:54:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181617.328867; Wed, 08 Sep 2021 09:54:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuHv-0002oH-Vl; Wed, 08 Sep 2021 09:54:39 +0000
Received: by outflank-mailman (input) for mailman id 181617;
 Wed, 08 Sep 2021 09:54:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FUO+=N6=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNuHu-0001EI-4X
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:38 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c2f1fdc5-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2f1fdc5-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631094870;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=qNr9KSzgnELykt88dPBpEyZ6wsbID+Fk75Z7xVFv0NI=;
  b=GiSqzthqW2ZRwbgm0rqK1TNl9aGSMCUyp+bAcz64F9IRDl2O+bmq1D4C
   zaSmVPfwOUGreT7FsDj04QTg7+qy+fsKb6uQMBSl/sThbqPiADFu4QyNl
   7MvldZgq9geH+0hYMz75UhGXsR+2OkEeXgINWnOUDa99NQ3TpNPXbQenA
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: TEp3lFiQCKdFcOnbcMELEiY57LjMHq9nWflMVXnA4QGUeCG7MaikXmLDWaPjXq2buvsKTXYVer
 aTROEza9lzQXnohAeQlKtiLW4qaVa5YFycHGuyLG0hN6rQVJhx5a91c/q/hjEA3sWNlIcW45HU
 FBb5/D0Lm53u1QtiT6x+NShp/0hXY2IHtSy96aeQ9Hlus/a9Y87qTBQ9tLV8afymE71uouGrSk
 NgvsXUdS+FTqlspecNKSweRyiTWAa+linCCxiUx+2pn+e6WXX/NCo10gl88TcO8Iaov3sl30yg
 qRRcOUicFyIvj89y2jp73PpM
X-SBRS: 5.1
X-MesageID: 53993479
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:DyHrDaywYbFr/qAmlg9xKrPwKr1zdoMgy1knxilNoRw8SKOlfq
 eV7ZMmPH7P+U8ssR4b+OxoVJPsfZqYz+8W3WBzB8bHYOCFgguVxehZhOOIqQEIWReOk9K1vZ
 0QFZSWY+efMbEVt6rHCXGDYrUd/OU=
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="53993479"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Anthony PERARD <anthony.perard@citrix.com>, "Daniel De
 Graaf" <dgdegra@tycho.nsa.gov>, "Daniel P. Smith"
	<dpsmith@apertussolutions.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH 2/2] build: add --full to version.sh to guess $(XEN_FULLVERSION)
Date: Wed, 8 Sep 2021 10:54:22 +0100
Message-ID: <20210908095422.438324-3-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210908095422.438324-1-anthony.perard@citrix.com>
References: <20210908095422.438324-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Running $(MAKE) like that in a $(shell ) while parsing the Makefile
doesn't work reliably. In some case, make will complain with
"jobserver unavailable: using -j1.  Add '+' to parent make rule.",
which will be part of "$(make xenversion)" output.

It isn't possible to distinguish between the output produced by
the target "xenversion" and `make`'s own output.

Instead of running make, this patch "improve" `version.sh` to try to
guess the output of `make xenversion`. This only works as long as
$(XEN_VENDORVERSION) isn't set, or $(XEN_EXTRAVERSION) isn't
overridden.

This fix GitLab CI's build job "debian-unstable-gcc-arm64" on which
$(shell $(MAKE) xenversion) in "flask/policy/Makefile.common" reliably
produce more output than just the xenversion.

This patch also allows to run for example `make --debug=a` without
breaking the build when flask policy is enabled.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/flask/policy/Makefile.common |  2 +-
 version.sh                         | 15 ++++++++++++++-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/tools/flask/policy/Makefile.common b/tools/flask/policy/Makefile.common
index bea5ba4b6a40..7f470cd02861 100644
--- a/tools/flask/policy/Makefile.common
+++ b/tools/flask/policy/Makefile.common
@@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER)
 #
 ########################################
 
-POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory)
+POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)
 POLICY_LOADPATH = /boot
 
 # List of policy versions supported by the hypervisor
diff --git a/version.sh b/version.sh
index e894ee7e0469..abd19ce79ac8 100755
--- a/version.sh
+++ b/version.sh
@@ -1,5 +1,18 @@
 #!/bin/sh
 
+opt_full=false
+while [ $# -gt 1 ]; do
+    case "$1" in
+        --full) opt_full=true ;;
+        *) break ;;
+    esac
+    shift
+done
+
 MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "`
 MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "`
-printf "%d.%d" $MAJOR $MINOR
+
+if $opt_full; then
+    EXTRAVERSION=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
+fi
+printf "%d.%d%s" $MAJOR $MINOR $EXTRAVERSION
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181622.328878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI0-0003VV-HE; Wed, 08 Sep 2021 09:54:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181622.328878; Wed, 08 Sep 2021 09:54:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI0-0003VE-DC; Wed, 08 Sep 2021 09:54:44 +0000
Received: by outflank-mailman (input) for mailman id 181622;
 Wed, 08 Sep 2021 09:54:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuHz-0001EI-4x
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:43 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.45]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c2f1fdc4-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:31 +0000 (UTC)
Received: from AS8PR04CA0170.eurprd04.prod.outlook.com (2603:10a6:20b:331::25)
 by AM5PR0801MB1745.eurprd08.prod.outlook.com (2603:10a6:203:3a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.23; Wed, 8 Sep
 2021 09:54:20 +0000
Received: from VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:331:cafe::10) by AS8PR04CA0170.outlook.office365.com
 (2603:10a6:20b:331::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:20 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT044.mail.protection.outlook.com (10.152.19.106) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:20 +0000
Received: ("Tessian outbound 0ec886cb54dd:v105");
 Wed, 08 Sep 2021 09:54:19 +0000
Received: from a14e68f2d9c3.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 DD5C5540-293F-4767-9609-8CF34A47C456.1; 
 Wed, 08 Sep 2021 09:54:14 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a14e68f2d9c3.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:14 +0000
Received: from PR0P264CA0185.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::29)
 by PAXPR08MB7033.eurprd08.prod.outlook.com (2603:10a6:102:205::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Wed, 8 Sep
 2021 09:54:12 +0000
Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::7) by PR0P264CA0185.outlook.office365.com
 (2603:10a6:100:1c::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:12 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:12 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:16 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Wed, 8
 Sep 2021 09:54:08 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2f1fdc4-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VWaJ31sy4Cs8NaQhMRjrM5i+mYxMfjvIRjfogEWDwok=;
 b=ADdZ4P/coQ35CGkuuOyodRvhsQCgHL8tdCdcvXXTx/6Gxirwxr7u47BZk8+OQL8/CifC/xmi3e+1QRvl6kNR5TfC9LweWA7PFX0bZE11Vc8aUPsyJg1LKochit52pNyyv+qkCVQm1QuTDQlGz/kA5LarnRMuw4Czu74K43o9928=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: b74c3c38bd22e03a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OCGByYaeyKtAb8BiUooJCHGJJ8zpbkoev5qp1lgNYUHAKVyVKqcm9hkr6YVfp+fjkR66G19atRNAUiGq6ftwWCpbUioHJc601UD3+gDByhKbYco9PxARP75fFKwYYyoTZlU9mXiUrqtq8pUSzDaFe6a105BDVjV1vOX6XMe+BcaT1Sv0wUhiLcYg3Y+0GuD5kdq39fHIs+ZtmNtfadQ1ceu3HyQV/dkJk7GJQquAOwNcyP8ew44ZmM2ZBdu58Fg8J6AKqkCehSQVAlKiCZzKscvC8qR46dC4wsNbqb0Gdgjm4Isrs1DNNG9TugICsbmfbHI7w+cdnkQHlGc8xEAsnQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VWaJ31sy4Cs8NaQhMRjrM5i+mYxMfjvIRjfogEWDwok=;
 b=S2Kk/A4P1zrQSJjFJf8qln9Uur3sCwvsh1b8EeSGnd3kP5F3J4e0jR6wi7BitSaVrx/Y/Jjv6iZ7/to6uNBOMT1OuV9l9ZSu3Sd1QrzqwlA4NW0suItxdLXmBc3mED2vmndVoHd//2CMpxiJwgEHn8l5o21mbLLy/8NR7Fx7UZGOe3xl09YISM2tUQ5HPm0o4FpbrWBEUMbAy0y7NI5rBwTstujfERMpA7uZjxD+G5SiAiXB8jbFunFn4KrHvcsjnZrVtQ5cgjwzYSqBXbA5zKUDxGB7PVuMmS3O+9vq7SZWF/MsmFDKLdAzLh9dDD+kRlrCrrKhkOP3wZJ9xD5acw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VWaJ31sy4Cs8NaQhMRjrM5i+mYxMfjvIRjfogEWDwok=;
 b=ADdZ4P/coQ35CGkuuOyodRvhsQCgHL8tdCdcvXXTx/6Gxirwxr7u47BZk8+OQL8/CifC/xmi3e+1QRvl6kNR5TfC9LweWA7PFX0bZE11Vc8aUPsyJg1LKochit52pNyyv+qkCVQm1QuTDQlGz/kA5LarnRMuw4Czu74K43o9928=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 4/7] xen/arm: static memory initialization
Date: Wed, 8 Sep 2021 09:52:45 +0000
Message-ID: <20210908095248.545981-5-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d308a56e-e3b3-49a4-f2c7-08d972aea11b
X-MS-TrafficTypeDiagnostic: PAXPR08MB7033:|AM5PR0801MB1745:
X-Microsoft-Antispam-PRVS:
	<AM5PR0801MB1745BD025EB3A49EAC432360F7D49@AM5PR0801MB1745.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:207;OLM:207;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ld7uyxPh7vBV7epAQhqr2GgiIKpuSfE7ghweRdu4B9JGWtPhndYkAchVe2lwHxGHNEitVqcvkwTqqv7bJz5pg7qxJ6gWEBjZpe6DoMDGzKqCaDOKpwDOiz58QvKJDtRTz22v6JyeDhal1AHMLiOV+yrXM3PiXQottMnBfxyDXYzhBk1vctIOgPdTjTaafqzM/d56Yl48GHplqCYdZtg+PiF7cRDWTRUOkAvWZmEVQmY/enZowlU8J33YcEzH+UD5XRn8BoRBlxc3Vrm3jFPEivIW4ziJQ6gpfO5zOLUa7FkjqMCYPdF32d7dJwjezJO16XHtEQPx5lva+9IJEWorDSIbxg+0zWuyaZya2P9fXuVM0yelBj/bqM6lEf27oLldbmTRRVeL5GQMNdzUlMntKXE7wCeI9loGmCoS8kriq7vXGCjlogPuUmqMRqbcyFpxzE+lo62mA/F8/Zz5elWJLTXlRSD3u7gA5j9vjzKV1bKoloJ2Z0Ronrwbu+j4urm0K14hYY0366wlsiio/14mpaiVTiAEsqNC8NdV7hlFZV16I9+WO2y1vsbffhI9lCom2CDACYilbZ4pWtQhRIbBJmZ1o7bPW6AoSjlvWMGoM/QEtR+EJmTEJv8ehNkjSBnxJ5L/zycBT1jcReiytxPeOilbSsi7srCDNBKFprAB35/vQeI6zYA4ikPe0fFs0u14wKgue8bznfX12p7YbM4DiAbTvkMcy4hdT3DNTPoy620=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(376002)(346002)(396003)(39860400002)(136003)(46966006)(36840700001)(86362001)(316002)(82310400003)(26005)(70586007)(36860700001)(4326008)(336012)(8676002)(110136005)(82740400003)(8936002)(54906003)(186003)(6666004)(70206006)(426003)(1076003)(7696005)(356005)(47076005)(2906002)(36756003)(2616005)(83380400001)(5660300002)(478600001)(44832011)(81166007)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7033
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ea79c35b-fd2b-4ef4-cd5c-08d972ae9c9e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HY/1ydUxKveCzmZWY9DTLrajh29OLKuYP2XD/0JadVmUXkfVc4V/AiP0qsKhEvHCWXv0Y6ICjb/JEtOpyj0xOqBJgY1NWTQzNBAHfGWSGV3uPSKlvyhaLGaFLVV90BUrYxC6wUVqYq480td5vnRYWErSHjcT9Bnq8nHTceJ6lpyM/x4ZAa5FiqHKAxnPidc2mlzApJNr08ox4Jny51L9RE+B3di8T6s5X/cuGICNQ7N5MZGGGzFsSGyiIIY72WEJKb2ikafjY7WZTBCREhLKLrnhi2FpkBmt76CLpKiLHIXLGOphWuLkx9um+F0q3OdrN6vGlsQSBPSXRNzEPQ/ruCIdK0inYRoMRtIs9+97E2jv8fYY4+EJWBz7TYQZo89lqHkBs4IhI1ttBhZVAYVVJoLQHZIZ9SXn2RIuOOvdv/KCeJgWiWXG2VJqZTv43pHL+9SvIGwLGw+UBuXXyCavU0tAlOxt7Qzu30dYV4xzAXiMrrygo23eyRe9VsvxWp78cMJCguRvTv4gJBgznofhuIerARyGG5PM6dFIIXch37Mdb3VQHTmrUSYuLYrpezmfe6AFHFlEMzN56e/1lRob+aQr/bNtCmiKqRrp7wsgldVFrjj0RDKT1skec1nQlD4RymVF+680z8on7GJddNBkS05CQ9b609ZdSqYmDtSaheGz+HScSOQQwmxx+bGwEaVVzKvaq4hNLnZ1ey3WRSVJqw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(346002)(136003)(376002)(36840700001)(46966006)(26005)(2616005)(36860700001)(81166007)(82740400003)(6666004)(107886003)(336012)(36756003)(1076003)(83380400001)(47076005)(82310400003)(86362001)(426003)(7696005)(186003)(110136005)(8676002)(316002)(8936002)(478600001)(44832011)(70206006)(5660300002)(70586007)(2906002)(54906003)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:20.1347
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d308a56e-e3b3-49a4-f2c7-08d972aea11b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1745

This patch introduces static memory initialization, during system boot-up.

The new function init_staticmem_pages is responsible for static memory
initialization.

Helper free_staticmem_pages is the equivalent of free_heap_pages, to free
nr_mfns pages of static memory.

This commit also introduces a new CONFIG_STATIC_MEMORY option to wrap all
static-allocation-related code.

Put asynchronously scrubbing pages of static memory in TODO list.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/setup.c    | 27 +++++++++++++++++++++++++++
 xen/common/Kconfig      | 13 +++++++++++++
 xen/common/page_alloc.c | 21 +++++++++++++++++++++
 xen/include/xen/mm.h    |  6 ++++++
 4 files changed, 67 insertions(+)

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 63a908e325..5be7f2b0c2 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -609,6 +609,29 @@ static void __init init_pdx(void)
     }
 }
 
+/* Static memory initialization */
+static void __init init_staticmem_pages(void)
+{
+#ifdef CONFIG_STATIC_MEMORY
+    unsigned int bank;
+
+    for ( bank = 0 ; bank < bootinfo.reserved_mem.nr_banks; bank++ )
+    {
+        if ( bootinfo.reserved_mem.bank[bank].xen_domain )
+        {
+            mfn_t bank_start = _mfn(PFN_UP(bootinfo.reserved_mem.bank[bank].start));
+            unsigned long bank_pages = PFN_DOWN(bootinfo.reserved_mem.bank[bank].size);
+            mfn_t bank_end = mfn_add(bank_start, bank_pages);
+
+            if ( mfn_x(bank_end) <= mfn_x(bank_start) )
+                return;
+
+            free_staticmem_pages(mfn_to_page(bank_start), bank_pages, false);
+        }
+    }
+#endif
+}
+
 #ifdef CONFIG_ARM_32
 static void __init setup_mm(void)
 {
@@ -736,6 +759,8 @@ static void __init setup_mm(void)
     /* Add xenheap memory that was not already added to the boot allocator. */
     init_xenheap_pages(mfn_to_maddr(xenheap_mfn_start),
                        mfn_to_maddr(xenheap_mfn_end));
+
+    init_staticmem_pages();
 }
 #else /* CONFIG_ARM_64 */
 static void __init setup_mm(void)
@@ -789,6 +814,8 @@ static void __init setup_mm(void)
 
     setup_frametable_mappings(ram_start, ram_end);
     max_page = PFN_DOWN(ram_end);
+
+    init_staticmem_pages();
 }
 #endif
 
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11a..3558be0dbc 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -67,6 +67,19 @@ config MEM_ACCESS
 config NEEDS_LIBELF
 	bool
 
+config STATIC_MEMORY
+	bool "Static Allocation Support (UNSUPPORTED)" if UNSUPPORTED
+	depends on ARM
+	help
+	  Static Allocation refers to system or sub-system(domains) for
+	  which memory areas are pre-defined by configuration using physical
+	  address ranges.
+
+	  When enabled, memory can be statically allocated to a domain using
+	  the property "xen,static-mem" defined in the domain configuration.
+
+	  If unsure, say N.
+
 menu "Speculative hardening"
 
 config SPECULATIVE_HARDEN_ARRAY
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index a3ee5eca9e..ba7adc80db 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2604,6 +2604,27 @@ struct domain *get_pg_owner(domid_t domid)
     return pg_owner;
 }
 
+#ifdef CONFIG_STATIC_MEMORY
+/* Equivalent of free_heap_pages to free nr_mfns pages of static memory. */
+void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
+                                 bool need_scrub)
+{
+    mfn_t mfn = page_to_mfn(pg);
+    unsigned long i;
+
+    for ( i = 0; i < nr_mfns; i++ )
+    {
+        mark_page_free(&pg[i], mfn_add(mfn, i));
+
+        if ( need_scrub )
+        {
+            /* TODO: asynchronous scrubbing for pages of static memory. */
+            scrub_one_page(pg);
+        }
+    }
+}
+#endif
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 667f9dac83..8e8fb5a615 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -85,6 +85,12 @@ bool scrub_free_pages(void);
 } while ( false )
 #define FREE_XENHEAP_PAGE(p) FREE_XENHEAP_PAGES(p, 0)
 
+#ifdef CONFIG_STATIC_MEMORY
+/* These functions are for static memory */
+void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
+                          bool need_scrub);
+#endif
+
 /* Map machine page range in Xen virtual address space. */
 int map_pages_to_xen(
     unsigned long virt,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181626.328889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI4-0004Gm-VM; Wed, 08 Sep 2021 09:54:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181626.328889; Wed, 08 Sep 2021 09:54:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI4-0004GM-RB; Wed, 08 Sep 2021 09:54:48 +0000
Received: by outflank-mailman (input) for mailman id 181626;
 Wed, 08 Sep 2021 09:54:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuI3-0000bc-HI
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:47 +0000
Received: from FRA01-PR2-obe.outbound.protection.outlook.com (unknown
 [40.107.12.78]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78803600-1be9-494b-9e7c-8c98769411dc;
 Wed, 08 Sep 2021 09:54:31 +0000 (UTC)
Received: from AM7PR02CA0014.eurprd02.prod.outlook.com (2603:10a6:20b:100::24)
 by PR2PR08MB4715.eurprd08.prod.outlook.com (2603:10a6:101:1d::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:30 +0000
Received: from AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:100:cafe::c6) by AM7PR02CA0014.outlook.office365.com
 (2603:10a6:20b:100::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.20 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:29 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT050.mail.protection.outlook.com (10.152.17.47) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:29 +0000
Received: ("Tessian outbound 7a08471b1ef6:v105");
 Wed, 08 Sep 2021 09:54:29 +0000
Received: from 3afc8c350cb9.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7BF624EC-5049-42E5-91B8-05FD44A495A4.1; 
 Wed, 08 Sep 2021 09:54:23 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3afc8c350cb9.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:23 +0000
Received: from PR0P264CA0184.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::28)
 by VI1PR08MB4447.eurprd08.prod.outlook.com (2603:10a6:803:f3::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:19 +0000
Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::6) by PR0P264CA0184.outlook.office365.com
 (2603:10a6:100:1c::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:19 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:19 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:22 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78803600-1be9-494b-9e7c-8c98769411dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q9Xvps1nERdmkV37FMndb6oyhfDMDttEJab3tc9qnbU=;
 b=1H8cRACXBw54AoXOv2bne/IOQrNzoRv0sd/p/MJbAF88AnpkiVAXBPMUNkNf6C90X/04q6vQwpnX9Xg4ng8RMCtZiRR0U5Y/65h85Z5Eu4M71O9kmSmFUk011TbeEa5OyJPaz5fEO/ahpBd3heSrh6fz5JRpV1Kl1ZwD8Fz1eN4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0110ddf03a380d5f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G0twraB4bld5XI72+NKgCux4wCa47nDs3V13g+pTOdVQWU7vtAr2CXB9+S+rvaiOx40Jh9lRAVnfjmA9vha7fD+KFaTKBfR9eFFFOyNYTOhkbMkcCLe5kjT30IZwkJQaBF3S9gyOD5T0i30MfXw3Ij0pW0wljqE0mOikHrhtNO4F/7rilhGRw9jB1VsuFntWLhi70WuvifT+cesezQZm0+Gtf21QCkpbM41moDP299W3hTu+WNcqz8XcPjjDx7aovx/GaINPbVe1NGHjYFy5j5WHjdto8GbMMywE6e2I7Lp9LSRpcHrxDrIymJ0YAYrRf/NJqvrG0hJf1Ncw1kFptA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q9Xvps1nERdmkV37FMndb6oyhfDMDttEJab3tc9qnbU=;
 b=X6dadGPLbA9nOHxzFsAjgNIUYzTwFTBWy9uXhLMA6vrznhcui89/S+m6zWW8bsvhN+3SwJ5nweeCNtkjIgG0SC34A1vbRITqGfyKxFpwrfsLzbcAppxmKRrHpDRfXDhXPP+uEiSHfBEI1HWX/oKrW/ICCdu1/oZN2bE+24UXfaNSMxWRgdUdiYtotC905KEhiyoj9i88P0xY3Vz3NrWarPqQme7OR6ZFIucis8Sqzct/uK3DzEpJDQt/DO5HA5O+546qTBpDBRNjcAyWzGOXEXqXOKWIzQA2MXSBU1q1v7IEuo4fM38bTF/o5P2nKySLYPRVcNY8nbBwAk1ZRK8VtQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q9Xvps1nERdmkV37FMndb6oyhfDMDttEJab3tc9qnbU=;
 b=1H8cRACXBw54AoXOv2bne/IOQrNzoRv0sd/p/MJbAF88AnpkiVAXBPMUNkNf6C90X/04q6vQwpnX9Xg4ng8RMCtZiRR0U5Y/65h85Z5Eu4M71O9kmSmFUk011TbeEa5OyJPaz5fEO/ahpBd3heSrh6fz5JRpV1Kl1ZwD8Fz1eN4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 7/7] xen/arm: introduce allocate_static_memory
Date: Wed, 8 Sep 2021 09:52:48 +0000
Message-ID: <20210908095248.545981-8-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ecfed280-6e87-429f-927b-08d972aea6ec
X-MS-TrafficTypeDiagnostic: VI1PR08MB4447:|PR2PR08MB4715:
X-Microsoft-Antispam-PRVS:
	<PR2PR08MB47156F4E6E7ED9018B5365F4F7D49@PR2PR08MB4715.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:644;OLM:644;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 1TxyTFqHUG3FWwPS6fkevvNE/gmvhc90TJ7+rEFAbKJ+HEy08xGdIwmdR9SwLc7eRazGAB8HveJVf5dh1UlzErAT+aniC/lek62ZqNEBDNEFrv4lKSGHxB8bJ7et/V2xDP9yKlfB5UwG7eGTNt/tUfPXJs51iwnP1J7jAPO88YHiDogaiqGMPZpARDQpvc3/SbOcDYBBmd+jlpvaoCeR1PLz7KgAUcr0kIZOdRJFDgBm1c07j8fhKMYgckNDdNnhT14DoxD0XTXVtVb9xIQ+s/gkR0uaeJktuGogs/LpBIb0e87WI2cpHIuQDAXC4Gyc6A2+ya8tFJ6VVjRApw1I/263PbWib/GT7AAuhJIfg8iobmN8D1JD5BoUaxBUYFzOhr7/8HhEyb/fIV3WZYT4pntE9++IlZMOJrBcC1ojGw205/RZ21UJqdBPKueR4x6TvVAja6eO63MI4+IKC5g6J1HXTTggbwmgQHy47DbouUrXKkdm7Ecpf8V/wR1JNzO/9VWSNBmaBFWcgZbkorYNGCyB9623jaPSIw0yh8AguY5ycoNSRVux85rza/qRFkjMSnPXwPSXpRr3wdUcFuvNSTyZsgTe6HP8U/ON6qWlCKzb8LgBb1dSPhLcR7ych3op0jjrwVFCo3ZzXZYR3c/CmycnkeOalTwGwoSMXiIVe5qExTu8HFJiGlLeNfnqcoSA9ETL+1P+Y/KYJbV6RHkXsw8GvlBZvrjNj6JiZY2bn/g=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(46966006)(36840700001)(26005)(8936002)(70206006)(70586007)(110136005)(54906003)(4326008)(186003)(2906002)(316002)(81166007)(82740400003)(7696005)(5660300002)(336012)(83380400001)(47076005)(6666004)(356005)(82310400003)(478600001)(36860700001)(8676002)(1076003)(2616005)(36756003)(426003)(44832011)(86362001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4447
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1a1e78b6-238c-4d93-88ed-08d972aea083
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	V3mU7NCkvMGrOPE7LXMqB7IInLn2g5kcbrhyRcuq3riN0jdwqdveK5w73qKRwz4rWyqrULvVVzjt1m9vNtAwC4F3ncCeeuJWSV5c0W3/ZsEpGide4yCQV7PL1Om33T6AaCOgX2l400zJw6h8fBZtoxOTXjBUpknXtHJpFLXqtfTtoWMVew+7+JHMVNIfFTqj0GmdccMnhrt2EIC1JORlXqAowMLNdVwAtyxHh48GbAT/PCOBL/4PfAEAqsuNeSAUL6xHKG1d2IHPCWnZqyd1XlFjuLIG0YJa7al6MDQHkzYZ9vdhZdCTXhp2lWI8nwBLaVEVlaa3KbGpXjPYILnGEuqOmoKipclnnev0n0zxZrwBpsXQ5ExB5U9Sa56dhtqefWu45g2sIgf/s7R0Q6jmF6UH0T8SYpCrvQbTooHbkeRjjAoCVxyZ54RfvwhtikZl7kcrDDaxHQNmjeUhImqj1nlzownv0qqKyY7ndAMvhQT5qIOEOEUWXGlLES46SRIZB5cT5ZQ+aFisMRUtc3goIg5CF9lJywIRGj514xa2yYNA7n5NpfmSclL9kRWbFBPVyCEYxy8KPOEnOcVLsONge94pHIUR+BbEDt3t+V0E3bEo+iTJR4kd+e7+Z/e5gMM12ikT4O3KrNXnE42kJEVPvQzG2QbGNo4yVB97hCWYf4KqTsms98JHsyMWjdn9Y3DIVvKmq9NkDKDSum4TuO6bsw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(46966006)(36840700001)(2616005)(186003)(336012)(82740400003)(426003)(83380400001)(107886003)(82310400003)(36756003)(47076005)(8676002)(70206006)(6666004)(26005)(70586007)(8936002)(478600001)(81166007)(36860700001)(2906002)(5660300002)(7696005)(44832011)(4326008)(54906003)(1076003)(316002)(110136005)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:29.9214
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ecfed280-6e87-429f-927b-08d972aea6ec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4715

This commit introduces a new function allocate_static_memory to allocate
static memory as guest RAM for Domain on Static Allocation.

It uses acquire_domstatic_pages to acquire pre-configured static memory
for this domain, and uses guest_physmap_add_pages to set up P2M table.
These pre-defined static memory banks shall be mapped to the usual guest
memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

In order to deal with the trouble of count-to-order conversion when page number
is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
a new function guest_physmap_add_pages to cope with adding guest RAM p2m
mapping with nr_pages.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/domain_build.c | 161 +++++++++++++++++++++++++++++++++++-
 xen/arch/arm/p2m.c          |   7 +-
 xen/include/asm-arm/p2m.h   |  11 +++
 3 files changed, 173 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d1c0..b011cc4789 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -480,6 +480,162 @@ fail:
           (unsigned long)kinfo->unassigned_mem >> 10);
 }
 
+#ifdef CONFIG_STATIC_MEMORY
+static bool __init append_static_memory_to_bank(struct domain *d,
+                                                struct membank *bank,
+                                                mfn_t smfn,
+                                                paddr_t size)
+{
+    int res;
+    unsigned int nr_pages = PFN_DOWN(size);
+    /* Infer next GFN. */
+    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
+
+    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
+        return false;
+    }
+
+    bank->size = bank->size + size;
+
+    return true;
+}
+
+/* Allocate memory from static memory as RAM for one specific domain d. */
+static void __init allocate_static_memory(struct domain *d,
+                                          struct kernel_info *kinfo,
+                                          const struct dt_device_node *node)
+{
+    const struct dt_property *prop;
+    u32 addr_cells, size_cells, reg_cells;
+    unsigned int nr_banks, gbank, bank = 0;
+    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
+    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
+    const __be32 *cell;
+    u64 tot_size = 0;
+    paddr_t pbase, psize, gsize;
+    mfn_t smfn;
+    int res;
+
+    prop = dt_find_property(node, "xen,static-mem", NULL);
+    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
+                               &addr_cells) )
+    {
+        printk(XENLOG_ERR
+               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
+        goto fail;
+    }
+
+    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
+                               &size_cells) )
+    {
+        printk(XENLOG_ERR
+               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
+        goto fail;
+    }
+    reg_cells = addr_cells + size_cells;
+
+    /*
+     * The static memory will be mapped in the guest at the usual guest memory
+     * addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
+     * xen/include/public/arch-arm.h.
+     */
+    gbank = 0;
+    gsize = ramsize[gbank];
+    kinfo->mem.bank[gbank].start = rambase[gbank];
+
+    cell = (const __be32 *)prop->value;
+    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
+
+    for ( ; bank < nr_banks; bank++ )
+    {
+        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
+        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
+
+        smfn = maddr_to_mfn(pbase);
+        res = acquire_domstatic_pages(d, smfn, PFN_DOWN(psize), 0);
+        if ( res )
+        {
+            printk(XENLOG_ERR
+                   "%pd: failed to acquire static memory: %d.\n", d, res);
+            goto fail;
+        }
+
+        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
+               d, bank, pbase, pbase + psize);
+
+        while ( 1 )
+        {
+            /* Map as much as possible the static range to the guest bank */
+            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
+                                               min(psize, gsize)) )
+                goto fail;
+
+            /*
+             * The current physical bank is fully mapped.
+             * Handle the next physical bank.
+             */
+            if ( gsize >= psize )
+            {
+                gsize = gsize - psize;
+                break;
+            }
+            /*
+             * When current guest bank is not enough to map, exhaust
+             * the current one and seek to the next.
+             * Before seeking to the next, check if we still have available
+             * guest bank.
+             */
+            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
+            {
+                printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
+                goto fail;
+            }
+            else
+            {
+                psize = psize - gsize;
+                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
+                /* Update to the next guest bank. */
+                gbank++;
+                gsize = ramsize[gbank];
+                kinfo->mem.bank[gbank].start = rambase[gbank];
+            }
+        }
+
+        tot_size += psize;
+    }
+
+    kinfo->mem.nr_banks = ++gbank;
+
+    kinfo->unassigned_mem -= tot_size;
+    /*
+     * The property 'memory' should match the amount of memory given to the
+     * guest.
+     * Currently, it is only possible to either acquire static memory or let
+     * Xen allocate. *Mixing* is not supported'.
+     */
+    if ( kinfo->unassigned_mem )
+    {
+        printk(XENLOG_ERR
+               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\". Unsupported configuration.\n");
+        goto fail;
+    }
+
+    return;
+
+ fail:
+    panic("Failed to allocate requested static memory for domain %pd.", d);
+}
+#else
+static void __init allocate_static_memory(struct domain *d,
+                                          struct kernel_info *kinfo,
+                                          const struct dt_device_node *node)
+{
+}
+#endif
+
 static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                                    const struct dt_device_node *node)
 {
@@ -2453,7 +2609,10 @@ static int __init construct_domU(struct domain *d,
     /* type must be set before allocate memory */
     d->arch.type = kinfo.type;
 #endif
-    allocate_memory(d, &kinfo);
+    if ( !dt_find_property(node, "xen,static-mem", NULL) )
+        allocate_memory(d, &kinfo);
+    else
+        allocate_static_memory(d, &kinfo, node);
 
     rc = prepare_dtb_domU(d, &kinfo);
     if ( rc < 0 )
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index eff9a105e7..6e01e83967 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1293,11 +1293,8 @@ out:
     return resolved;
 }
 
-static inline int p2m_insert_mapping(struct domain *d,
-                                     gfn_t start_gfn,
-                                     unsigned long nr,
-                                     mfn_t mfn,
-                                     p2m_type_t t)
+int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
+                       mfn_t mfn, p2m_type_t t)
 {
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     int rc;
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 6a2108398f..f885cc522b 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -300,6 +300,9 @@ int map_dev_mmio_region(struct domain *d,
                         unsigned long nr,
                         mfn_t mfn);
 
+int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
+                       mfn_t mfn, p2m_type_t t);
+
 int guest_physmap_add_entry(struct domain *d,
                             gfn_t gfn,
                             mfn_t mfn,
@@ -315,6 +318,14 @@ static inline int guest_physmap_add_page(struct domain *d,
     return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
 }
 
+static inline int guest_physmap_add_pages(struct domain *d,
+                                          gfn_t gfn,
+                                          mfn_t mfn,
+                                          unsigned int nr_pages)
+{
+    return p2m_insert_mapping(d, gfn, nr_pages, mfn, p2m_ram_rw);
+}
+
 mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn);
 
 /* Look up a GFN and take a reference count on the backing page. */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181628.328900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI6-0004Zm-JX; Wed, 08 Sep 2021 09:54:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181628.328900; Wed, 08 Sep 2021 09:54:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI6-0004Z9-Eq; Wed, 08 Sep 2021 09:54:50 +0000
Received: by outflank-mailman (input) for mailman id 181628;
 Wed, 08 Sep 2021 09:54:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuI4-0001EI-4h
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:48 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c4067fc8-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:32 +0000 (UTC)
Received: from DBBPR09CA0022.eurprd09.prod.outlook.com (2603:10a6:10:c0::34)
 by DBAPR08MB5717.eurprd08.prod.outlook.com (2603:10a6:10:1ae::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 09:54:29 +0000
Received: from DB5EUR03FT045.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:c0:cafe::ee) by DBBPR09CA0022.outlook.office365.com
 (2603:10a6:10:c0::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:29 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT045.mail.protection.outlook.com (10.152.21.164) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:29 +0000
Received: ("Tessian outbound b48d04bf8afd:v105");
 Wed, 08 Sep 2021 09:54:29 +0000
Received: from 0a083f3b0cee.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 9541EF8D-BDB2-4C5C-B993-98BB700987A7.1; 
 Wed, 08 Sep 2021 09:54:08 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0a083f3b0cee.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:08 +0000
Received: from DU2PR04CA0019.eurprd04.prod.outlook.com (2603:10a6:10:3b::24)
 by VI1PR08MB2814.eurprd08.prod.outlook.com (2603:10a6:802:1f::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 09:54:06 +0000
Received: from DB5EUR03FT054.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:3b:cafe::b8) by DU2PR04CA0019.outlook.office365.com
 (2603:10a6:10:3b::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:06 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT054.mail.protection.outlook.com (10.152.20.248) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:06 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:11 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Wed, 8
 Sep 2021 09:54:04 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c4067fc8-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MXkdwEjVyQ82/D1j1eSxQZQfJtE5JUO8wUvXNctlD2M=;
 b=jVs9JpcbPL22+ynUZG+fFCTyljcDWMetWM2ISdh4k8/+bhThLAQkxVZPJDB1XHOFI5j6nFG03l/T4bOyv5xx3PpMKLKJ3cL9bBRVytLcw7mn3spvuJEpxlGN5Ef/a9pRWOy0bSBo6Y6/aJRse1gyQyFxyqom0QVM6AGewRGlSyg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8f4fa1b2d57f1ca3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Zas51da3cn0wrB+jPtbEhng/oTJ5BGF73p0uZSLmSekv3DJv0LSZca7XA8LXyWL79GI7l6Sa9Ixg289KyhipO5k2UmK3k7IpVqjn3QoR5wAN6XuppM9ZkzDXJkTBjaNnCsSrmYjg6Dr8EITYHLurqupr844BBzImDOqy8PvrnsHlobqNL1RGXsjyfT2Up2QD0HZTJ8V5S36bRZHyNI2tKyYd2kjPAMSLpC9YuU1o5EMrL+SZxzUlM/PO8wohVrU7C3Lb+DHRvA8XkkZ3atZH24rqVlKCug8/AtC1AeQgbTi4ePJGs9CSxibezV00pFKLwW45AL2p4wzVC7dn7ZO4KA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MXkdwEjVyQ82/D1j1eSxQZQfJtE5JUO8wUvXNctlD2M=;
 b=VODPylCXW9DRdpWek+PUtrzAicCTB/ZR0jOGe3By8rltTkEGwh4uIergf3dURf+zThfWiNjEjit8RA1ZUG/GGvocTp1hGfNsYMvJyGDfPD/CCpI/RN5vqukhsHhdMj1VtEEaf0OEvM2Kh3rG7HBZyXWzG18DVnLaU3wSOu2pA+tHSMNMpp/WWTLbl+BJCYyulpSZCUo+ac0PFZO1tw89XSLnRI+CI4Op7rCVy4xjqh8rHRILH8vMx34YOpKB2GHyFxEbRv6tW/eM40N9BZGcUeU5C3ZcHzjizQ86mtDNSJ6HsNbV68uAFK3cHhisWKSzSj+ql051s70vUMKNXSSepw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MXkdwEjVyQ82/D1j1eSxQZQfJtE5JUO8wUvXNctlD2M=;
 b=jVs9JpcbPL22+ynUZG+fFCTyljcDWMetWM2ISdh4k8/+bhThLAQkxVZPJDB1XHOFI5j6nFG03l/T4bOyv5xx3PpMKLKJ3cL9bBRVytLcw7mn3spvuJEpxlGN5Ef/a9pRWOy0bSBo6Y6/aJRse1gyQyFxyqom0QVM6AGewRGlSyg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 2/7] xen/arm: introduce domain on Static Allocation
Date: Wed, 8 Sep 2021 09:52:43 +0000
Message-ID: <20210908095248.545981-3-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 14296cbc-908d-4340-a8a5-08d972aea66a
X-MS-TrafficTypeDiagnostic: VI1PR08MB2814:|DBAPR08MB5717:
X-Microsoft-Antispam-PRVS:
	<DBAPR08MB57173A9728FC9D35EE2B3E5DF7D49@DBAPR08MB5717.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 LuIu0+pxr56YjVQdUDpyJ+gR2ldmA70kBn2fBEG8oVQJFLdkO/Ul42sid/6FWDHmUtq9gB8w/C2X5evd3wkF141bNGz56MwVH5FsWrH4RO6gD2yvVOxij7+St3XMlUsMY1PRv3VIKPFuqnIgb/ozHdVsH0Yp+/QXn+ikny5wRYCwJjBunpBGS+8wKdnh5gbAzXneE2HsA4VnfHdpMnirLZi+RjbYwCvAaovMCZ4vf9qD3Egl7qS8eMgp+g2AQDW8sKzEDBwVFxBZMijVJUMsLEjM1qiI6RctvfAELYmeN+zamsb2KYTjbnWiT1mgtqmB9fsgm/dFU5zkI0RAkqy1lBczrGvs7RXn8/JiG6CaBX7esACf+OlFeoCJgom04NCXc9ePcyP/pRsYrh3ms3BSeciAl8qox4k0n2WN9V5PTShOuecDIT+Kly0gL0lRUeXOW1DjxJC1PEQ0KDRYe34pVtdALrscJWQLFa7+xWcLvuUqgLxVSD10d3qivcj5NipjEZI/0aKmHh7EKCXk+vtfc3BzSJbXKgX7ZD4xhp8gAUfThRe5R+LZrKo7wXyqS4t2h1VfBZOgFBa8lZAkLZXV2v0YU6HHxEmqctjkRASVxkauRznlMPIEELJCqcrmrj13mR0fyiiC31XS1icdZYkIdMfMtLPbKiyNC4ejuG6Q3VoVPP08ORAHqy+f0p/E9BfUMaTqlBMDcBN+26YdDFeD3ikoR0+6JuPTLKTmkpYPLvo=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(396003)(346002)(136003)(376002)(39860400002)(36840700001)(46966006)(1076003)(82740400003)(6666004)(83380400001)(2906002)(36756003)(110136005)(5660300002)(86362001)(54906003)(7696005)(82310400003)(44832011)(4326008)(478600001)(47076005)(186003)(36860700001)(81166007)(8936002)(70206006)(356005)(70586007)(336012)(26005)(8676002)(2616005)(316002)(426003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB2814
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT045.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ca927daa-0fb8-49a1-4a09-08d972ae98bb
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PeQjW98bbKWCZUD1BXubt1SIO2xgvOs+wnBCilTNnJNJmSN5HtEqwHQWR0l0hxf1tmG1EJZ1xr71Z4F2vxxCbeN4QopyIovKq8b4PeVZiv29fdbKdWDaTS1+uFNzxA29Wvt8BUMgXpBSY5m00SrSacSSQzq2NaWTeO6bsqGAjMHIYmeDgTP86UA2ymW4pwYvdXXASDwHbg44kTia43lZPK1CpAScjmZqid4mVPqMrsOzU9SUEf7YVUwV9uteLu9w51bzNhji57Naoy2HDVt6zm1lVILYkbfU4cO+EZiMd/NE/IRoh1EbFtYRdhFwkSBL4KvQbPUBG1o8nqMw6EMgrda//8hz97RncAmfg8p3y6yXUyKxOyl88XBEYk35dBsgPv3LXlhmftkdZlOd/AVKKrKmvZbwe9MBHrMUN80HpGHGkoDkzYQ6W82TKx5KOjGuzvQ+kcZSmsOoVsUv35A0+I4PLvJMPkk4aMgA04uMfjCF5KjvdJQUC08a5YXB1KU49ueldg+IbXstcq1yl4Xq5tcU9pRj+9Pyxtj2khf5bkWgTlHVuHNTJ1bnzasz1N99JbqzQUA1ZGtRpC9pef3X1tmI/GDAPSUXLYSg/BejjClF2baK5CNR5aWrQzFxEU2BfbHB+rluZ+PFgaZR8k5d8GzrLXAi3/pBxM6Aaiyar8MgxgLwbJ5j4ZCNG90bAiq9y9sn1vfO2nTUgzkZaNW9MA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(46966006)(36840700001)(36860700001)(426003)(81166007)(107886003)(186003)(36756003)(478600001)(5660300002)(316002)(86362001)(336012)(82310400003)(8676002)(70206006)(54906003)(70586007)(26005)(1076003)(82740400003)(110136005)(2616005)(47076005)(2906002)(6666004)(44832011)(7696005)(8936002)(4326008)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:29.1439
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 14296cbc-908d-4340-a8a5-08d972aea66a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT045.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5717

Static Allocation refers to system or sub-system(domains) for which memory
areas are pre-defined by configuration using physical address ranges.

Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
beginning, shall never go to heap allocator or boot allocator for any use.

Memory can be statically allocated to a domain using the property "xen,static-
mem" defined in the domain configuration. The number of cells for the address
and the size must be defined using respectively the properties
"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".

The property 'memory' is still needed and should match the amount of memory
given to the guest. Currently, it either comes from static memory or lets Xen
allocate from heap. *Mixing* is not supported.

The static memory will be mapped in the guest at the usual guest memory
addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

This patch introduces this new `xen,static-mem` feature, and also documents
and parses this new attribute at boot time.

This patch also introduces a new field "bool xen_domain" in "struct membank"
to tell whether the memory bank is reserved as the whole hardware resource,
or bind to a xen domain node, through "xen,static-mem"

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 docs/misc/arm/device-tree/booting.txt | 42 +++++++++++++++++++++++++++
 xen/arch/arm/bootfdt.c                | 30 +++++++++++++++++--
 xen/include/asm-arm/setup.h           |  1 +
 3 files changed, 71 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 5243bc7fd3..44cd9e1a9a 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -268,3 +268,45 @@ The DTB fragment is loaded at 0xc000000 in the example above. It should
 follow the convention explained in docs/misc/arm/passthrough.txt. The
 DTB fragment will be added to the guest device tree, so that the guest
 kernel will be able to discover the device.
+
+
+Static Allocation
+=============
+
+Static Allocation refers to system or sub-system(domains) for which memory
+areas are pre-defined by configuration using physical address ranges.
+
+Memory can be statically allocated to a domain using the property "xen,static-
+mem" defined in the domain configuration. The number of cells for the address
+and the size must be defined using respectively the properties
+"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
+
+The property 'memory' is still needed and should match the amount of memory
+given to the guest. Currently, it either comes from static memory or lets Xen
+allocate from heap. *Mixing* is not supported.
+
+The static memory will be mapped in the guest at the usual guest memory
+addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
+xen/include/public/arch-arm.h.
+
+Below is an example on how to specify the static memory region in the
+device-tree:
+
+    / {
+        chosen {
+            domU1 {
+                compatible = "xen,domain";
+                #address-cells = <0x2>;
+                #size-cells = <0x2>;
+                cpus = <2>;
+                memory = <0x0 0x80000>;
+                #xen,static-mem-address-cells = <0x1>;
+                #xen,static-mem-size-cells = <0x1>;
+                xen,static-mem = <0x30000000 0x20000000>;
+                ...
+            };
+        };
+    };
+
+This will reserve a 512MB region starting at the host physical address
+0x30000000 to be exclusively used by DomU1.
diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index b01badda3e..afaa0e249b 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -66,7 +66,7 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
 static int __init device_tree_get_meminfo(const void *fdt, int node,
                                           const char *prop_name,
                                           u32 address_cells, u32 size_cells,
-                                          void *data)
+                                          void *data, bool xen_domain)
 {
     const struct fdt_property *prop;
     unsigned int i, banks;
@@ -97,6 +97,7 @@ static int __init device_tree_get_meminfo(const void *fdt, int node,
             continue;
         mem->bank[mem->nr_banks].start = start;
         mem->bank[mem->nr_banks].size = size;
+        mem->bank[mem->nr_banks].xen_domain = xen_domain;
         mem->nr_banks++;
     }
 
@@ -185,7 +186,8 @@ static int __init process_memory_node(const void *fdt, int node,
                                       u32 address_cells, u32 size_cells,
                                       void *data)
 {
-    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
+    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells,
+                                   data, false);
 }
 
 static int __init process_reserved_memory_node(const void *fdt, int node,
@@ -339,6 +341,28 @@ static void __init process_chosen_node(const void *fdt, int node,
     add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
 }
 
+static int __init process_domain_node(const void *fdt, int node,
+                                      const char *name,
+                                      u32 address_cells, u32 size_cells)
+{
+    const struct fdt_property *prop;
+
+    printk("Checking for \"xen,static-mem\" in domain node\n");
+
+    prop = fdt_get_property(fdt, node, "xen,static-mem", NULL);
+    if ( !prop )
+        /* No "xen,static-mem" present. */
+        return 0;
+
+    address_cells = device_tree_get_u32(fdt, node,
+                                        "#xen,static-mem-address-cells", 0);
+    size_cells = device_tree_get_u32(fdt, node,
+                                     "#xen,static-mem-size-cells", 0);
+
+    return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells,
+                                   size_cells, &bootinfo.reserved_mem, true);
+}
+
 static int __init early_scan_node(const void *fdt,
                                   int node, const char *name, int depth,
                                   u32 address_cells, u32 size_cells,
@@ -357,6 +381,8 @@ static int __init early_scan_node(const void *fdt,
         process_multiboot_node(fdt, node, name, address_cells, size_cells);
     else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") )
         process_chosen_node(fdt, node, name, address_cells, size_cells);
+    else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") )
+        rc = process_domain_node(fdt, node, name, address_cells, size_cells);
 
     if ( rc < 0 )
         printk("fdt: node `%s': parsing failed\n", name);
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index c4b6af6029..95da0b7ab9 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -24,6 +24,7 @@ typedef enum {
 struct membank {
     paddr_t start;
     paddr_t size;
+    bool xen_domain; /* whether the memory bank is bound to a Xen domain. */
 };
 
 struct meminfo {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 09:54:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 09:54:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181634.328911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuIA-000597-2V; Wed, 08 Sep 2021 09:54:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181634.328911; Wed, 08 Sep 2021 09:54:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuI9-00058c-RH; Wed, 08 Sep 2021 09:54:53 +0000
Received: by outflank-mailman (input) for mailman id 181634;
 Wed, 08 Sep 2021 09:54:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SEYe=N6=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mNuI9-0001EI-4v
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 09:54:53 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.77]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c32e3227-108a-11ec-b148-12813bfff9fa;
 Wed, 08 Sep 2021 09:54:33 +0000 (UTC)
Received: from AM7PR02CA0003.eurprd02.prod.outlook.com (2603:10a6:20b:100::13)
 by HE1PR0802MB2507.eurprd08.prod.outlook.com (2603:10a6:3:e1::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 09:54:28 +0000
Received: from AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:100:cafe::35) by AM7PR02CA0003.outlook.office365.com
 (2603:10a6:20b:100::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:28 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT050.mail.protection.outlook.com (10.152.17.47) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:26 +0000
Received: ("Tessian outbound 7a08471b1ef6:v105");
 Wed, 08 Sep 2021 09:54:26 +0000
Received: from 74c9ed8f84e5.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 308EF0D1-ED56-442B-A907-3A37F47D11CC.1; 
 Wed, 08 Sep 2021 09:54:20 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 74c9ed8f84e5.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 09:54:20 +0000
Received: from PR0P264CA0172.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::16)
 by VI1PR0801MB1662.eurprd08.prod.outlook.com (2603:10a6:800:52::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 09:54:16 +0000
Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::60) by PR0P264CA0172.outlook.office365.com
 (2603:10a6:100:1c::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:16 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 09:54:16 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 8 Sep
 2021 09:54:17 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 8 Sep 2021 09:54:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c32e3227-108a-11ec-b148-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GZ922YnGKts8gghHRklqg+V1lm7yq6BNVhkDhoqkHMI=;
 b=oZaIAM+gZAQkg+BJPFVHM36wkQ/p/jcSkuM7oLy3d9SyhjNzPgoHCje0/gEfQQEixz0Ut0Kfhfvm+R2nIgSRDDT62rf08YcIkBuhueU0cPko15zhqOJPsyWRUa1E8+Ui0MHBp9G25VFtUDeMIa2bdzNsWhji6vf1QOeBy2MK3xM=
X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is
 63.35.35.123) smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass
 (signature was verified) header.d=armh.onmicrosoft.com;lists.xenproject.org;
 dmarc=temperror action=none header.from=arm.com;
Received-SPF: TempError (protection.outlook.com: error in processing during
 lookup of arm.com: DNS Timeout)
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3a3057876d07db5c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R1UHluNxa+R6hIjroPCKQEv618n0gJrKd/ItShm0P+PdD4MeCghE9u90DATF9G2gLWmnoTObK+BvZasCqoIcqWSk6EAIbWYMm7OMCSSArlN+GsJGom6hNpxRrWMC+vBhlqvZYCYgk6GxKUN45tBhyrSOBQwJ4zjNCqcNtw0wZ3VUSbEMO48PpBKeijnZ+i/KQGR2Se7Mmj89SLADwfIuGGMan2JQTB2q/K4yRwJhA4yEyVPpXKliB7NiOrMWy43C5Gw9YU8QtyFPCd0V/Hs/SWxN2TPiwRYv87wqGYmQ0gX7piW8IYtifTmAIJx8Xa82I5A94dtqeEFtFH7hjP2PHA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GZ922YnGKts8gghHRklqg+V1lm7yq6BNVhkDhoqkHMI=;
 b=MB0KTYSaheV22jOk2JpsTHfILbwu0HG4f+6q62WBzM0bkKdPsj1JXmPZAFaVJnw6Sq08NuYgJtCGPeCJbEy8aqv/xfzGCI6ehbCKGrFe43/l+UBTNzDqGCpQZHFh7Oi7EWK12oqlRoGmX8X7vtq349Hvenb7jXppLD5QKaCWTN0mBOIzX59uM0E6rm9d4bYEtACYZwVLZlEUhQwXF6OcltcomOOh6Glw5gz8mE/8atGoXQemSCGFpmrWpRekUuwOLF0vWKq0tKvPxjwTrBkN9KxrJaf1D1OG/XE7RGGfNcf2aoaPbGkdPG31k2S7nrZfa/2Jshv/Ohdg0c5hbpDc4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GZ922YnGKts8gghHRklqg+V1lm7yq6BNVhkDhoqkHMI=;
 b=oZaIAM+gZAQkg+BJPFVHM36wkQ/p/jcSkuM7oLy3d9SyhjNzPgoHCje0/gEfQQEixz0Ut0Kfhfvm+R2nIgSRDDT62rf08YcIkBuhueU0cPko15zhqOJPsyWRUa1E8+Ui0MHBp9G25VFtUDeMIa2bdzNsWhji6vf1QOeBy2MK3xM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new function assign_page
Date: Wed, 8 Sep 2021 09:52:46 +0000
Message-ID: <20210908095248.545981-6-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210908095248.545981-1-penny.zheng@arm.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0467dcbc-9d1c-46db-730d-08d972aea527
X-MS-TrafficTypeDiagnostic: VI1PR0801MB1662:|HE1PR0802MB2507:
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB25073865A11DAF94B89B0693F7D49@HE1PR0802MB2507.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:288;OLM:288;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 uFgXCUUMY1VZxuDKCZ7RPJldeqgtMKzm8TK/lpz8Yu1v/Kb/+lgI6z81WLrkfDJTGjwAHrxgYNjIFlLgKynw6B9t7w/zjBRFFCxSd8LdnFhNPHKoO69gOIEpndt1w2GPYD5ri9F0+dxm0pR8O4nQrBY9n64RpWmYPvqee6dL0xy1IpjK+xcL/OFcaTE+1XxEHVQFAIr0JvQarhhtCSEPUDtRlGtn2G+2AutencUc7qnDkRIChKWlvKC+T90jQhHmP7SZ27qcvBCz7uCciNlyEjRy7bfs5mRbFoUCZNxVhvT1/NRfZdHE3jnR+illQZ41I+JthmZvDZweiyDDkOvoVr56zeo/u2rVP0wtvgLSL1Nn3dqRxO3yEN42jR5wxN6O+CKQJMxG0pKmu2YYF9F84YIXxPPmxmPgX8gw1aNTHqyKOnHzvkPz6/L/V/CZt/rNWj7L4WuYzsCD0aEEC+60RGcri6JtiKn7rQcDFPifq1DyyrVf0YXSaJKxeeNX+G3TBhGDZioWxulPwAj9GzZ7rivaMewFZFKVlmf8EtXY6o1fL9mYGmzLie0aO+PxU4ELgp6pQlUb0OfCkK3LL1ioo5HVgkL7xAIID8G1yFJi0GBimIM0GwKQcR1N8QqqKyfgTAVR1SgC7Hp3kTCNQByLzO7JtiqcpcWBOUvi+fVVhcxuLEfUh1ADYZpZnwJatqEGtx3Xt7Ix1TMLRDOuNAoFHMX/3jF5dIDbJmX2voWPy3k=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39860400002)(136003)(376002)(346002)(396003)(36840700001)(46966006)(2906002)(1076003)(44832011)(82740400003)(356005)(2616005)(54906003)(83380400001)(26005)(81166007)(70206006)(82310400003)(8936002)(7696005)(5660300002)(70586007)(110136005)(4326008)(36860700001)(86362001)(47076005)(316002)(186003)(426003)(6666004)(336012)(36756003)(478600001)(8676002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1662
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0a3a3f3c-2d03-4abb-3fa9-08d972ae9ef2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	bfdLnWz4F/l5rOi4ojEeHFBdQMWWDHcVK2Kc+aKrLBzcWM5ooI9jOSCPRYVAn8VqdjKlkH0U45M6/yI+ERxZ5b6pwHV6XOdgXJD6tkcU3rbjClQ3MPltJTxJs2V0bLrO/4nPT9ec5L/NeHWz+DWPzrt8fDctZCaSlHG0sdkvdVZl3nzvQrkDjgti53lHwnUQpuFySeMpRlete10tn8YyAQ6dSKvZCttyZjyWPQfDqTZN3Lgraol3jVVfbxQna6eJIJ4tqMIY7/PYxR6rR8O50lPxs0eS/M6aMaBbp3JrrknuSN5oZAe4C1uj2jiCgs7LOVGYh5G+EysFhPha1I3AhHL24lEclbHbTuPU6f7/69xIh1miuTo0APE7R0qpvWeXLMg4/wHhYCFxlFsSgXo+aIlML0HEYRo8V4MCUPFH95ROMNA5wvo79NJmn+7DGxlVCwCvIqTHQj5dcU7l9b1ueoX2BYutD7qTE+rArBFh04I5b4zeI5JE4jQYVxsXhm64xcTESwBhLb2YhA3SSLANKyO9KcHvyx4gfUdOGAAOyqaQ13eApvIyTUcHFHG2sMSCiUWLp/ovI/piwXO55miBKuq+p7EzlV5Y6kjAGKqn285j5I/tCJepIkJVxq4v7Nc5WtFM+en5vsLT/81JJ3fyqsgb3lSavtwiAzHIzlGKJOr3ORSQFZc99FaLfDCaV1dxBOr3V8+sK9ihBkL1BtIncQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(376002)(346002)(396003)(46966006)(36840700001)(316002)(47076005)(2616005)(63370400001)(478600001)(70586007)(26005)(54906003)(2906002)(110136005)(4326008)(36860700001)(1076003)(82740400003)(5660300002)(44832011)(107886003)(82310400003)(6666004)(36756003)(186003)(81166007)(83380400001)(86362001)(426003)(63350400001)(336012)(8936002)(7696005)(8676002)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 09:54:26.9728
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0467dcbc-9d1c-46db-730d-08d972aea527
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2507

In order to deal with the trouble of count-to-order conversion when page number
is not in a power-of-two, this commit re-define assign_pages for nr pages and
introduces a new helper assign_page for original page with a single order.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/x86/pv/dom0_build.c |  2 +-
 xen/common/grant_table.c     |  2 +-
 xen/common/memory.c          |  6 +++---
 xen/common/page_alloc.c      | 21 +++++++++++++--------
 xen/include/xen/mm.h         |  6 ++++++
 5 files changed, 24 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index d7f9e04b28..7787cc8fca 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -568,7 +568,7 @@ int __init dom0_construct_pv(struct domain *d,
         else
         {
             while ( count-- )
-                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 0, 0) )
+                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 1, 0) )
                     BUG();
         }
         initrd->mod_end = 0;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index ee61603a97..50f5f83023 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -2358,7 +2358,7 @@ gnttab_transfer(
          * is respected and speculative execution is blocked accordingly
          */
         if ( unlikely(!evaluate_nospec(okay)) ||
-            unlikely(assign_pages(e, page, 0, MEMF_no_refcount)) )
+            unlikely(assign_pages(e, page, 1, MEMF_no_refcount)) )
         {
             bool drop_dom_ref;
 
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 74babb0bd7..9cef8790ff 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -728,8 +728,8 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
         /* Assign each output page to the domain. */
         for ( j = 0; (page = page_list_remove_head(&out_chunk_list)); ++j )
         {
-            if ( assign_pages(d, page, exch.out.extent_order,
-                              MEMF_no_refcount) )
+            if ( assign_page(page, exch.out.extent_order, d,
+                             MEMF_no_refcount) )
             {
                 unsigned long dec_count;
                 bool_t drop_dom_ref;
@@ -797,7 +797,7 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
      * cleared PGC_allocated.
      */
     while ( (page = page_list_remove_head(&in_chunk_list)) )
-        if ( assign_pages(d, page, 0, MEMF_no_refcount) )
+        if ( assign_pages(d, page, 1, MEMF_no_refcount) )
         {
             BUG_ON(!d->is_dying);
             free_domheap_page(page);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index ba7adc80db..bb19bb10ff 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2261,7 +2261,7 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
 int assign_pages(
     struct domain *d,
     struct page_info *pg,
-    unsigned int order,
+    unsigned long nr,
     unsigned int memflags)
 {
     int rc = 0;
@@ -2281,7 +2281,7 @@ int assign_pages(
     {
         unsigned int extra_pages = 0;
 
-        for ( i = 0; i < (1ul << order); i++ )
+        for ( i = 0; i < nr; i++ )
         {
             ASSERT(!(pg[i].count_info & ~PGC_extra));
             if ( pg[i].count_info & PGC_extra )
@@ -2290,18 +2290,18 @@ int assign_pages(
 
         ASSERT(!extra_pages ||
                ((memflags & MEMF_no_refcount) &&
-                extra_pages == 1u << order));
+                extra_pages == nr));
     }
 #endif
 
     if ( pg[0].count_info & PGC_extra )
     {
-        d->extra_pages += 1u << order;
+        d->extra_pages += nr;
         memflags &= ~MEMF_no_refcount;
     }
     else if ( !(memflags & MEMF_no_refcount) )
     {
-        unsigned int tot_pages = domain_tot_pages(d) + (1 << order);
+        unsigned int tot_pages = domain_tot_pages(d) + nr;
 
         if ( unlikely(tot_pages > d->max_pages) )
         {
@@ -2313,10 +2313,10 @@ int assign_pages(
     }
 
     if ( !(memflags & MEMF_no_refcount) &&
-         unlikely(domain_adjust_tot_pages(d, 1 << order) == (1 << order)) )
+         unlikely(domain_adjust_tot_pages(d, nr) == nr) )
         get_knownalive_domain(d);
 
-    for ( i = 0; i < (1 << order); i++ )
+    for ( i = 0; i < nr; i++ )
     {
         ASSERT(page_get_owner(&pg[i]) == NULL);
         page_set_owner(&pg[i], d);
@@ -2331,6 +2331,11 @@ int assign_pages(
     return rc;
 }
 
+int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
+                unsigned int memflags)
+{
+    return assign_pages(d, pg, 1UL << order, memflags);
+}
 
 struct page_info *alloc_domheap_pages(
     struct domain *d, unsigned int order, unsigned int memflags)
@@ -2373,7 +2378,7 @@ struct page_info *alloc_domheap_pages(
                 pg[i].count_info = PGC_extra;
             }
         }
-        if ( assign_pages(d, pg, order, memflags) )
+        if ( assign_page(pg, order, d, memflags) )
         {
             free_heap_pages(pg, order, memflags & MEMF_no_scrub);
             return NULL;
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 8e8fb5a615..a74e93eba8 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -133,8 +133,14 @@ void heap_init_late(void);
 
 int assign_pages(
     struct domain *d,
+    struct page_info *pg,
+    unsigned long nr,
+    unsigned int memflags);
+
+int assign_page(
     struct page_info *pg,
     unsigned int order,
+    struct domain *d,
     unsigned int memflags);
 
 /* Dump info to serial console */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 10:03:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 10:03:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181689.328922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuQK-0000El-4f; Wed, 08 Sep 2021 10:03:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181689.328922; Wed, 08 Sep 2021 10:03:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuQK-0000Ee-1F; Wed, 08 Sep 2021 10:03:20 +0000
Received: by outflank-mailman (input) for mailman id 181689;
 Wed, 08 Sep 2021 10:03:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNuQI-0000EY-Bu
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 10:03:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fc76c11f-108b-11ec-b149-12813bfff9fa;
 Wed, 08 Sep 2021 10:03:17 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-Vcc3AT0SMDaAsVXbK_HbKg-1;
 Wed, 08 Sep 2021 12:03:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 10:03:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 10:03:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P250CA0006.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 10:03:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc76c11f-108b-11ec-b149-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631095396;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=U4+CNeQJy/jr2vUjG+fVEAweTSeM8l6WQo+Ob8voX2E=;
	b=BCnS/sJ8Nk7cHbGSubSoAnQRiXbcCEcds17esOeiyiT+5hO58zMCDYdQM9kS8Mm7S7jBGF
	3OQZL3T55Di6PqVcEtkuZQ/ldj0Esz24d4SENu+LZWD907RMyMgjKS/b5q9thqxgs/DP10
	TNWXguYnNBWnNTGcu/CsziNFP/DsdQI=
X-MC-Unique: Vcc3AT0SMDaAsVXbK_HbKg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZdWt6YQtvw/0XHTi4ZGH5s2Xq5undt7XLvvO/C6kNxDGBSeiMR1blqr/pPvOsECVGk8erBt6+9hVZJuEvDl11bzsBYyxaR0Wai+8zhQpncop6oH0zg77Y1AftoR9FY2/x1my6VILoXGri7Tt3Afxe0s1cW159GwYGzIiOAltDwDJ/YprT9UbaXWB0X5J5TmSClMUUkeBKCLK9t7K6oxMTlFXucOJT+2ZpB78B6hiCCex3M2pBAHZi2wMbqKRv2Sb3v6kogae+Z7kkHC9rl8M/z6GcwEyX6UJvSLXrZgSmZ6EKffmw3VbBzCLYk158oK2e4Hn7wqaOuoXt3TEryi5vw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=U4+CNeQJy/jr2vUjG+fVEAweTSeM8l6WQo+Ob8voX2E=;
 b=ATC657B0Ld5qd3kG2jULmvSeGjSihW2k+MdI/U588suRfNPb5RU3ZAfmyvtlFSXP5bCJ692ea90Jd/GutDGUEKy3yfbi68cUtzv6Q5YK69VHrJW7Xh83wu8oZ53Zg/WEqkzsL2y+yLgFvE5GTKgG8ch5aJK19B+I6Dm8V1DL604lzysaqEFDdzysl/sEIHOLg5CKVTuO7v9CWLOYSoORo2TiIjAf9fwGd0X2XozWRY0AKi86Jm3u7nWlQTdfBRX3uNUrmMt9cPDKEwYit7sme4DxOaQds4rJVCKcdy1TR5/bk7OFiptKaZXxmKnKMVkz34yyVWYiLQHNT8ZBJ7OJfg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
Date: Wed, 8 Sep 2021 12:03:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P250CA0006.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:102:57::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dc32e481-8487-4579-7252-08d972afdeee
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB244577376877BBEAFC0C492AB3D49@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sQ9FiJwfSC8bHu5C0Md487CHI0IQFldMmpP0zzZ1x1nTlJkD09GPgDEJ1Ldx8B5b6jJICTIoWgv5xV4mR+KUB3Zq0cI7YKw3eMkyaGe41t7bPKXsmiShefqraD+0GoOSkQ5DbWL9YjMOHWbMzByp6djhhEulL117f39bJtfcOg/yKs0Vls3fjYTHHQ8JvvmZ3yJbQwTZKiR2riTRr6+q7IUXj7g8APxnSxFCi17A3aPoZ9M1gtUkWbaU6X9sU8BdcuPJhuUcFqzeYm4JxjnFARL/wwM21vbXyuvhOFgBzDKIG1mXXuiDHm130U1FtZbSUnniG8nesYlB8UIPGZpkYWOkuFXoGsbPpERo6NyUhLmKuKhN0bbh8C/S20KVkCmm0t+ye/Y9PX95dofB6JXtVbgIRHpC/itZBbd0nzIYXe3yU7mFNJGRKRxmgDYqeU7dZ0WIsr+Pv7+gdUwi1UMZsvC/nSXK40Y5rPrTrz2OBuaBsFRjH5zep9NVLmrCLtZGlYD7e6lKxW4bn3afG1xUGUNLsOBn/w6btQfFmWS9VarV6otvAJ53bm/pXJWmMKxKuZzpzJB6xfdFK312459iXZKDWUb3tx0S9VUv2Xnh6Hv4I/h9dHVITML6vYlVo9dpUgQZlOQkaWCZzndXjALnEWecERzWw9k+HDSAxGK6dhBMBKyqIQMlbQ8SojTeUL1UkbPGr36pCofzUN/9qOeLH1WGQELWFNZVb5Btga8YdRRRgu0iUzxe5bpI5N95seng
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(136003)(376002)(346002)(396003)(86362001)(83380400001)(4326008)(5660300002)(956004)(2616005)(478600001)(2906002)(26005)(36756003)(8676002)(31696002)(53546011)(8936002)(31686004)(66946007)(66556008)(66476007)(7416002)(110136005)(38100700002)(16576012)(6486002)(54906003)(316002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UGZNQVAwVGxwcGk4VFZpSmMzRmhxMlV3ZXFuU1pNaTBQOUZCdjJPb0syOVZS?=
 =?utf-8?B?ZFBxaFVCQmRNQXpVTDdOUFFSLy9rL3FEbjJYbTE4TytnUVNZbXA5UGRNYUVU?=
 =?utf-8?B?c1NGOG91T1dPaEhaL2JMTC9jcGVDY3ZVTDBmZC95bWhuQXJhUkNqZnRwdzNE?=
 =?utf-8?B?bHRrRWNrL1NCTS9VbHlCM09FU1VTZndvTVVWbTdZY0V5QzdRQkdaRG5RWWQz?=
 =?utf-8?B?VzRhU0Q1N0tLbm9sd1BteXJmWjFldkRyYjlmRTJmK3ZsNXBwY1VuL0ZNNmxL?=
 =?utf-8?B?djRxaG1PSnozVDRPL0xHL1dRRS9mR1VXMEVuczAzeVhHRFg1bHBNQzBHbUZx?=
 =?utf-8?B?alRFOERpSXJPZ1lTdFN3UnM0NDFkMm5JMHdmUzBad3UzU0lDY04wYjNtOU9n?=
 =?utf-8?B?V2syb1NXeVkyZ0hrUVdxTk1SS0VQVkpzOHN4eWdOSThYL2loOVhxbkladWI0?=
 =?utf-8?B?R1lKQkd2QXhwcWtLRThoMTVtTU5CSGNmbTk2Z0VJOUx6dytoakNlWWtRdmdw?=
 =?utf-8?B?V2pqZE1QbEQrcFB6RkpkcVJpUWRLMXNDbm00cWdHVVdndGVWZFJXb2k1UnhP?=
 =?utf-8?B?c0cwOVhsNVhjcjF0QUl2OEdOem52dEtpRzR4NGVpOXJCR3pHZ2RMMkl1NVgy?=
 =?utf-8?B?MkdBcEZnanpJUXV1dDRvMzdubUR2YXBzeXRraE1INGlIcCs3aUVtS1VSVzdw?=
 =?utf-8?B?QldLRE93bmF5UTBuSVlPcm8wZDNzNUpXRW5GYnJmVUZVVzJFVjZnUm1jTGY0?=
 =?utf-8?B?VVBlWlZLVExsSGtBZmpoR3hPMGNZMW0zOXdQejV3MHNXLzNGRk5NSENuRUlX?=
 =?utf-8?B?aVJFeEhoSmg3eG5ta01YRUl4cHNFcFZic2o5aGV0cUIyaHluWURhU0RBeW5X?=
 =?utf-8?B?ZXNBTk9CTlRCeSs5a3pyOVlHU2w3RWZTTVZIalRnQVZ2c0cvNVc0L09LTGNN?=
 =?utf-8?B?ck9sWVBwMWEzay9WZlJuYndyQ2NIZkR2MkJMYVdwM21BMmkrUnlTZzdXY2xs?=
 =?utf-8?B?YnJ3bGtLL1JIbnZ1NmVLaFV3K3B3V1FVUXlzdnBmRzlGdzZPUmdWVC94ekIx?=
 =?utf-8?B?K1BZb3IzOGswUys3ZFZJT0YwL1drdEl1WGRNZ2ZvYy9zN1NBbmpvcnRUaVEy?=
 =?utf-8?B?cDYwRUViWWI2Y0hpRngxWEdubzRwUFpXblB5MDc0UzE4QmdRdjRSOGo3UFdC?=
 =?utf-8?B?R3lNdndxTE9jRktBNFhHVkR2WXN2bmVuL2I5TUVRTXNLM090em5SSTZqTlc2?=
 =?utf-8?B?V2pxM2p1S1NrRS9kMXlac0ljaWVUd1AvMG42RFJPcWc4dFBldk1rOWVmdUQ1?=
 =?utf-8?B?aHhvYzkzNUo4QzJwWVlWalVUdWYrYVF2TzdCYldjdS95QUp0WnpFT0dUNTIr?=
 =?utf-8?B?L0RkVlRvSE5ZVkdHd2NFTERRVnpYdUdreGR5TkpwUkpBVzJqY2JXMFI5L3FJ?=
 =?utf-8?B?WjVvNkp0eFFpalRsSG1hTHIzTTNqS0l6S0lJT0hYQTBhSzU2TUNCd2tYUll0?=
 =?utf-8?B?RUZPR2dySVcyMXYwQnZCcmo0K3JTbTZWTXRFVmcrMER1cW52YjZSTXJUNmZ2?=
 =?utf-8?B?SU5VQTdCWFpnUjlXTVNGU1YxUWRoSzRJV0M0WFFEVFpYdFRCZUFpRk1wSHBT?=
 =?utf-8?B?cmdxdW1RZjR1b2U1YlF3TmoyRGpldU05WEtyYS9uVVA5NWpWdTJoeGF3VzVp?=
 =?utf-8?B?bUpUQXZOMVJJWmxYd2FEaGtPT1QyODBEeWRBUzYrQXFkd2sxQXFOYTFmZTBM?=
 =?utf-8?Q?lCN387pjh1IT/3oh9Ux6TIEuTzhvxMvaBVU1/Fm?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dc32e481-8487-4579-7252-08d972afdeee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 10:03:13.5947
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: t3DwaNHHUgiHaX1/e/PvoowUtG6f1Z64r1YsIl9Va7/p94UpJIeVuic8tPJrzGAT+VMDHq4d6EnJWNTRlQ7qEQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

On 08.09.2021 11:43, Oleksandr Andrushchenko wrote:
> 
> On 08.09.21 12:27, Jan Beulich wrote:
>> On 07.09.2021 19:39, Oleksandr Andrushchenko wrote:
>>> On 07.09.21 19:30, Jan Beulich wrote:
>>>> On 07.09.2021 15:33, Oleksandr Andrushchenko wrote:
>>>>> On 06.09.21 17:31, Jan Beulich wrote:
>>>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>>>> --- a/xen/drivers/vpci/header.c
>>>>>>> +++ b/xen/drivers/vpci/header.c
>>>>>>> @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
>>>>>>>     static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
>>>>>>>                                 uint32_t val, void *data)
>>>>>>>     {
>>>>>>> +    struct vpci_bar *bar = data;
>>>>>>> +    bool hi = false;
>>>>>>> +
>>>>>>> +    if ( bar->type == VPCI_BAR_MEM64_HI )
>>>>>>> +    {
>>>>>>> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
>>>>>>> +        bar--;
>>>>>>> +        hi = true;
>>>>>>> +    }
>>>>>>> +    else
>>>>>>> +        val &= PCI_BASE_ADDRESS_MEM_MASK;
>>>>>>> +    bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
>>>>>>> +    bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
>>>>>> What you store here is not the address that's going to be used,
>>>>>>     as
>>>>>> you don't mask off the low bits (to account for the BAR's size).
>>>>>> When a BAR gets written with all ones, all writable bits get these
>>>>>> ones stored. The address of the BAR, aiui, really changes to
>>>>>> (typically) close below 4Gb (in the case of a 32-bit BAR), which
>>>>>> is why memory / I/O decoding should be off while sizing BARs.
>>>>>> Therefore you shouldn't look for the specific "all writable bits
>>>>>> are ones" pattern (or worse, as you presently do, the "all bits
>>>>>> outside of the type specifier are ones" one) on the read path.
>>>>>> Instead mask the value appropriately here, and simply return back
>>>>>> the stored value from the read path.
> 
> But in case of BAR sizing I need to actually return BAR size.
> So, the comparison is the way to tell if the guest wants to read
> actual (configured) BAR value or it tries to determine BAR's size.
> This is why I compare and use the result as the answer to what needs
> to be supplied to the guest. So, if I don't compare with 0xffffffff for the
> hi part and 0xfffffff0 for the low then how do I know when to return
> configured BAR or return the size?

Well, but that's the common misunderstanding that I've been trying
to point out: There's no difference between these two forms of
reads. The BARs are simply registers with some r/o bits. There's
no hidden 2nd register recording what was last written. When you
write 0xffffffff, all you do is set all writable bits to 1. When
you read back from the register, you will find all r/o bits
unchanged (which in particular means all lower address bits are
zero, thus allowing you to determine the size).

When the spec says to write 0xffffffff for sizing purposes, OSes
should follow that, yes. This doesn't preclude them to use other
forms of writes for whichever purpose. Hence you do not want to
special case sizing, but instead you want to emulate correctly
all forms of writes, including subsequent reads to uniformly
return the intended / expected values.

Just to give an example (perhaps a little contrived): To size a
64-bit BAR, in principle you'd first need to write 0xffffffff to
both halves. But there's nothing wrong with doing this in a
different order: Act on the low half alone first, and then act
on the high half. The acting on the high half could even be
skipped if the low half sizing produced at least bit 31 set. Now
if you were to special case seeing ffffffff:fffffff? as the
last written pair of values, you'd break that (imo legitimate)
alternative process of sizing.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 10:09:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 10:09:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181695.328932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuWJ-0000v7-Qm; Wed, 08 Sep 2021 10:09:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181695.328932; Wed, 08 Sep 2021 10:09:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNuWJ-0000v0-Nq; Wed, 08 Sep 2021 10:09:31 +0000
Received: by outflank-mailman (input) for mailman id 181695;
 Wed, 08 Sep 2021 10:09:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuWI-0000um-Mk; Wed, 08 Sep 2021 10:09:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuWI-0001sX-9S; Wed, 08 Sep 2021 10:09:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuWH-0006hN-Rs; Wed, 08 Sep 2021 10:09:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNuWH-0002xQ-RM; Wed, 08 Sep 2021 10:09:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=U00WU4ZXiiBY+nPWZILah4Mk/Lkfq7uGOh5TPoyiqB0=; b=VchLHLeQclUKMWp9C/oBRTNFRm
	GGoKuNsnZCHNoxY1bSN3lSiiraji8uTm0ZNeVAcn5Ks/PNBt6ys/VZCl/pGuDjTOp4nYOQ168zDi0
	a+tIJQGFgBjfLs4o8WGWBBEwBab9HU7+83ujrq9Ouf+4sgI3K5SjFJCk+VAVcVTfrqNE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164877-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164877: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:guest-start/debianhvm.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
X-Osstest-Versions-That:
    xen=6c27a8da8d2024aa78a9ec67bcfb18c123e84a17
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 10:09:29 +0000

flight 164877 xen-unstable real [real]
flight 164881 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164877/
http://logs.test-lab.xenproject.org/osstest/logs/164881/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 20 guest-start/debianhvm.repeat fail pass in 164881-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164873
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164873
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164873
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164873
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164873
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164873
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164873
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164873
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164873
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164873
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164873
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1
baseline version:
 xen                  6c27a8da8d2024aa78a9ec67bcfb18c123e84a17

Last test of basis   164873  2021-09-07 11:09:27 Z    0 days
Testing same since   164877  2021-09-07 21:09:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>
  Wei Liu <wl@xen.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6c27a8da8d..e70a9a043a  e70a9a043a5ce6d4025420f729bc473f711bf5d1 -> master


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 10:33:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 10:33:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181743.329019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNut1-0005bd-Ln; Wed, 08 Sep 2021 10:32:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181743.329019; Wed, 08 Sep 2021 10:32:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNut1-0005bW-IY; Wed, 08 Sep 2021 10:32:59 +0000
Received: by outflank-mailman (input) for mailman id 181743;
 Wed, 08 Sep 2021 10:32:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNut1-0005bQ-4v
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 10:32:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ee7236d0-6d58-4690-88e8-3a712a7c7473;
 Wed, 08 Sep 2021 10:32:57 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-_AzjD38gOF24_PKpt7JfWg-1; Wed, 08 Sep 2021 12:32:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4846.eurprd04.prod.outlook.com (2603:10a6:803:5b::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 10:32:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 10:32:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0010.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Wed, 8 Sep 2021 10:32:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee7236d0-6d58-4690-88e8-3a712a7c7473
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631097176;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1D/OwiwFXsedKdK1yUIbJBrg4OkseZXV1iXwOcwtd38=;
	b=XoOjjHZo4tDBh5rq2Oa8B3JyOTB5Dsjchc9iMEpBAmjbNqct3cgG4/GQfEK6SlnFx84sqn
	JTBGXqm8KkwwxJRmZ27zdLk6XqzF9Vs1gHclZBoqjSmAYnCMf+RGchm643iPATZB4ZentU
	zmye5IXOM/28aUGrBT0TalsTB2ltJH0=
X-MC-Unique: _AzjD38gOF24_PKpt7JfWg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f6XlWOjzltmSlPLlSdBRHV0LnqHFx+4WW4rWZWnapkqGPJ76MvoBcG3ZiNOwGNsgGxmO+hTdGO0CIbadcLZtElJ0SLfvoBBc21yBGltytrf9Jb+XZ6bIeT/oiFws70VmDTxt1wUQrGcbSDyIVpZh9lhZnWN3HmTsEs7MMquHKY+cXtUl+rDSEuz3JKwM+QLcIXkPiMVgv6J4HoPvh3c0aHsUKrYpUyT6MmOClM5Ydyu9zsxM0KZfjB4oRXbWlbJScVcDWulxpQ6eqVZHlNVHpNEe4OEqlou46JwpDrjCJLZ9hCJPX/jO2DNMu+3OowvHzv1XyIu+i+0mquIk0T7kxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1D/OwiwFXsedKdK1yUIbJBrg4OkseZXV1iXwOcwtd38=;
 b=C+K4p6kg1fgMS6/HiZKKsqh04Lbfc9Ghcon3YYVixJORZCMxjEI8Y7thrcG8XqmaKCw/GdPdCGbrpvirS14kRqHeP2qKFIcG64so0vZFvk3yktOqfUUSToaGPWGEOw9gjyV5q782CLHuUf5XgTlj3Hv+N0VhfiIo2CmU0wnCi8A0rJNqDHOwJHPFbvY2Xca1UGTlJffj/M1eamXNzIgBkmx1o0t++nx7oDDf/mmWsGBSag+nlaQ4e3JjNEdmNnU+a468pJW5AL93BpNCR5Cwb2zKt/QaycSSryQp6MSGxRRscZJA8wZhRcEdiFCThBULR1bfJp2VjU2s1MQWEXwjQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
Message-ID: <518201c2-1250-26fd-2105-f44a60d40dba@suse.com>
Date: Wed, 8 Sep 2021 12:32:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0010.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 381d111f-36ef-4869-78b3-08d972b4038d
X-MS-TrafficTypeDiagnostic: VI1PR04MB4846:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB48460E1E43A0D062232351C4B3D49@VI1PR04MB4846.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JCs3H9LOjrVxSPuAi03RN8XUTN6ya1XWZmL56itQoHRVYQKAXVFHcnTCIbNyOC3pZjO2siYyl8U4gUMrC9UrSkI1PsBQ7gjS4xE+g6+bZsxJWF8cNKDSjUQTJBgzrCN+tVOYVwej+nxOvilkOPLiGWySh4OQjc2BTNSyemduRoqv6tL/AKemF62aqQnmSt7fJn4hG4dUdfsg61f/5yhSsrXPmfHOeCVZrBJOAnskjE1VhJst7wjfSf5O1kTU5pn1+4EmPohTDceC/f1dHS2YpZEkwI3xuVoXtVwEsO/TwFzLdGevxA/w/kkpahg92bCb7Q2Udw4Q/bJj2dgCmNdIDsnapgoEbtfE/VKqKFGfm9UqLmmlBxP/HogF67oLFf5w/oP78o5lPgp4ZuHYU9CqkjJdG2wxWHuYAPEuleE0Ev629n1MKQfI6YOHK9PZpdmUIAZjmqL831D7XyLuwrLufvAT2/lfUjMvZs6Y2LdrhnhZzCRpBZELO1MjvcWwJIiwmJZyIG6kXWCa9K/nvR7TibyQrbP1VYntJnKEz9H6A+GbJrwfrwU+De0om/3uSFhzDiAYsqryUBruDWMXLXE3YMQA06ufxrGF17fQKwZlnXyyeAlCnbX0Oqg6OgsgiZKMutEDdKkiNkBVam5neCTUSaianVCB5wP0fV5gYwARbGMRHhs+KSgfdtqxpRp+fPw20c7305P83+v3QHhjO7QpyhRddfmrKwYE0HhBtik4OuJuydjPMMK39TsTd5ZRmvKO
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(39860400002)(396003)(366004)(136003)(31696002)(31686004)(66946007)(53546011)(54906003)(86362001)(4326008)(16576012)(186003)(38100700002)(2906002)(6486002)(2616005)(26005)(956004)(8676002)(8936002)(66476007)(36756003)(6916009)(316002)(66556008)(83380400001)(478600001)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K1FwTTFHYVR1dlhGS1FwUUl2OWtHWXVtLzNoaEFjTXM5VFl3UnhRamdtWGVV?=
 =?utf-8?B?QU1jRnloUk9Qck9kYXFjZ1kwMU9UV1lLTk9lcHB3U09Jck5HYXRaVFUvTFBj?=
 =?utf-8?B?QU02anNPWm1BU1hicDJ2RThKZFRBYm5maHdBY3JuMHhJZUJRRm5BcWlYNyt2?=
 =?utf-8?B?dmNrUzdHa2sxSjhlZDA3bm1QMElzMGtCbzZVUUhtMnMzVU12QzNPQ2tGQmpy?=
 =?utf-8?B?VVJGR1JMaEYwd0NJd0VOQ21RVXNoWnFMTUhYWDR6c1I0OS9WMGVxdmgzd0dr?=
 =?utf-8?B?c1FkVkxiWHJxKzNpQWNRSFNseXNQZng3cjRSUVRDZ0lCbHBxZGNnU3hYQjJQ?=
 =?utf-8?B?bzArY3VwM21GcC94ODdZREJCVlA0MEZNM1VNZ3JIVVE3YjYwYkQvR2RlQXVk?=
 =?utf-8?B?cE1QUGNpekxyUldUSXUyZE9vY3VhczhINGR3ek03dnV5VDl6elNPNGwxQ1Zh?=
 =?utf-8?B?RkMvdXZ5R0tJN0M0TTh5YUh3S0NhbHRzdjdaQWY2QjVSS3lvSTZHVnVUZVlk?=
 =?utf-8?B?alNkN1FZVjhTd2ROc2ExalQ2ckNCcnd6RkVxd0Uxalc4aXIrd005aE1FVXN4?=
 =?utf-8?B?dno3R0MxYlZsMkFDeW9QMEwvajZvZkRURWx3RmdSYVVJcnFGVllWcmNOVU9t?=
 =?utf-8?B?ajdqZ0dYVHlwV0xrT0xQRElmUUJyMDM5Y3lqZ0l1clBIREVPd0REQndtZVgz?=
 =?utf-8?B?RkFJZnUzQ05TNDFNdHN6Zk9ndndxUzBHUUROd240b2pUdmZQY3ZSLzhPUUJn?=
 =?utf-8?B?WSsyNFRKUldER3pLdXo2bEszM3o4R2ZOdDR2ckVyZVk1djcwZFVBVzEvektO?=
 =?utf-8?B?eUZ4OWNNVnhUZmg0OWNZdHQ3SHZyN1dZY05STTl2MTFWSk9mYUtnUmtKZnN6?=
 =?utf-8?B?RG5vUHJRbzRMRzZXQUpUa0FQTjlxNEUvSDdnZzlmWFpKaVFCZ0tDWURTcDRu?=
 =?utf-8?B?NlRocTJuT243K3ZUZ1NLVy9aN2VRREttbkxJcXdFcHI1OVF1RGQzN0oyM2JD?=
 =?utf-8?B?aGFxVTBYenhSSFlxZFN1dS9HY1ZCeGc4b0sycXU5MlBpOFRia1REUmY1OHZj?=
 =?utf-8?B?RFl0MmpCVnBPVDFucFc5R3piTjFWejhqcnNKYlpETlpmcnNYT09lT2JFRXZh?=
 =?utf-8?B?bWpETnBpQjZPSW1SbjU5K0pxYjVOejBNWlJvOTRGUThwSnVFRHB6YUxwemdW?=
 =?utf-8?B?WDJsdFpBaGt1YVVWSlgrcXd3NittTTViaUd5OStzZm1DaTdlSUZtSnh6YW5L?=
 =?utf-8?B?SjRJeWx6eTJuSTlvSzZoYmxWQTNxeXA4SEtqTkdqWmRROTA5bkwyVGxmbTc5?=
 =?utf-8?B?MWRpVU5hZVZlNlRYc0QyK0FhU0J4Q0grcEFlMnA1ZjFlMVFpanZNYTZsbXBo?=
 =?utf-8?B?OFFIQW95SGNYRTM2c0I4OVptZlRGMGV2Ujhxakt3NjZNSk45VGoyZ0tsUlVM?=
 =?utf-8?B?ZWtqQ2tBd083TVJQaDc2ZDQ4Sm5yL0Z6emFwNUo3Z2JLNjY4S3JOenJ4Vjhr?=
 =?utf-8?B?S0dUczNGdkxSRU9tcjEwL3dlZ3FuTkkraGk5ZkgxR2JOZGpBeWFkbkFjNi9W?=
 =?utf-8?B?dWZ2KzhBeFJESnJxZGhaZ0JYdUM2NWVCSlNtVEh4VEV2ZjRhOUVwZWRBVzFY?=
 =?utf-8?B?cHVxSHBOdzIrTE5jS2tEekNjOEk2WWtiMjNPZlFuYWJuYTY0Wm83SHhMWXhl?=
 =?utf-8?B?dXhCMVJ1VzV4RXROUytmYlpUTlQ4dVE2U1k2UEdZSDh6Nk42azl3UExvSFU0?=
 =?utf-8?Q?kLjFFHDdpKlPZEsEc7FvyLwtzt6dZHnm78JQ6q+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 381d111f-36ef-4869-78b3-08d972b4038d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 10:32:53.0317
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /Yr4UdhJ9rjzKm8sYfiLx9r0xIMit5dcLlNmnqjSOhovjlbK4sLCuek0l4xt4+AgmucismCqqUM7nmQ2TyLdsA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4846

On 01.09.2021 10:45, Jan Beulich wrote:
> Commit 806448806264 ("xen/domain: Fix label position in
> domain_teardown()" has caused Coverity to report a _new_ supposedly
> un-annotated fall-through in a switch(). I find this (once again)
> puzzling; I'm having an increasingly hard time figuring what patterns
> the tool is actually after. I would have expected that the tool would
> either have spotted an issue also before this change, or not at all. Yet
> if it had spotted one before, the statistics report should have included
> an eliminated instance alongside the new one (because then the issue
> would simply have moved by a few lines).
> 
> Hence the only thing I could guess is that the treatment of comments in
> macro expansions might be subtly different. Therefore try whether
> switching the comments to the still relatively new "fallthrough" pseudo
> keyword actually helps.
> 
> Coverity-ID: 1490865
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> If this doesn't help, I'm afraid I'm lost as to what Coverity means us
> to do to silence the reporting.

According to the most recent report this did not help. Shall I
revert the change? Or do we consider it a step towards using the
pseudo keyword more uniformly?

Jan

> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -401,13 +401,13 @@ static int domain_teardown(struct domain
>           */
>  #define PROGRESS(x)                             \
>          d->teardown.val = PROG_ ## x;           \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>      case PROG_ ## x
>  
>  #define PROGRESS_VCPU(x)                        \
>          d->teardown.val = PROG_vcpu_ ## x;      \
>          d->teardown.vcpu = v;                   \
> -        /* Fallthrough */                       \
> +        fallthrough;                            \
>      case PROG_vcpu_ ## x:                       \
>          v = d->teardown.vcpu
>  
> 
> 



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 10:43:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 10:43:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181749.329029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNv3B-0007Hb-NZ; Wed, 08 Sep 2021 10:43:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181749.329029; Wed, 08 Sep 2021 10:43:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNv3B-0007HU-Kf; Wed, 08 Sep 2021 10:43:29 +0000
Received: by outflank-mailman (input) for mailman id 181749;
 Wed, 08 Sep 2021 10:43:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNv3A-0007HO-J4
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 10:43:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4c7040d2-cd60-40e5-8b52-ae17bc609e4e;
 Wed, 08 Sep 2021 10:43:27 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-2m6IG6VnNLCd58WGwDgg8g-1; Wed, 08 Sep 2021 12:43:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4384.eurprd04.prod.outlook.com (2603:10a6:803:6f::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 10:43:24 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 10:43:24 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0005.eurprd04.prod.outlook.com (2603:10a6:208:122::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 10:43:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c7040d2-cd60-40e5-8b52-ae17bc609e4e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631097806;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6/VVPUxhJU9k+lxB/ME4qx3Y4zC/7I21qhXyIqu0Ua4=;
	b=Qv/jZFFVWtzBUb37FERgOJP3t9Zcmu9+QlQLkLTb7sMgMDD3swi1mb2itkFBftX2kvMXmq
	D2qxAY2pt66tqiGYH1v7DP9sKuEq6fdZVg4P9EMlDE0nF4vb+/J9tu9zg8Gr6QXY4XZqv5
	wpaqdTnyKthpP5Sk8Cte8HDCcv4jqSk=
X-MC-Unique: 2m6IG6VnNLCd58WGwDgg8g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EtTJgrsJpdDKa96B5JuDxjoLX1wcsGZDK+9PWBGOvYrkxjyzFRrxzxJYWgVtxOowi5CYTTijJGJfuehsgeSbpvQYoalIQFZiLa+rYin7wTtkLndARr7ZFt2MVupxhBPtA839WRgR6IJ+jkKbQL/ObU9b90wf75vy8YNeEw580D8HVPoPDrC7xJWpgInhqeilHkrkzN6Ob0rsj4P9P3GK8yWSGYCipLR7nMGY06IpQjvPMCdU/jqUg2Bxu7QLqIqFzC42J8xbARMsA19tZsp7eGxNyO19uAEsQj8H65Kef8iwd9YYwz03XyKsSJb9dd8TBJuUdlut0AYt7JU5pGx9Bw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6/VVPUxhJU9k+lxB/ME4qx3Y4zC/7I21qhXyIqu0Ua4=;
 b=TT4MbePL+EEW4g458+wRgUgLp/ObCRunsri7wpKn4uBPcNhn+EZW76tCuqQmy1d0n9HMUPk0ovA+QM8RczW7yhM+AlUaq5736UirYpNlvW3llyZyHdOyQuKEvtnPUjffDjdwkKIqMVdT5J4kLVwRZxvUoZ90Em6Ny7IKhnt7VmULC4033YZ1CB8jZsnCbYmCE6u22o8h3jWlgTN121j4pFwX4raLWGPdxJtRgx36RzR8679BK/MrrZPXdECizC+2NIO2jS52uR5g4LrmI9XPdIOm66VpuTmuhUKXyStYNFuHTiRmhZZojDPZ+l+bN4hhQESfrTlmO3IOXNLnkXgxdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 3/3] x86/amd: Use newer SSBD mechanisms if they exist
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210817143006.2821-4-andrew.cooper3@citrix.com>
 <20210907161908.17402-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1fa45e94-ecd8-8920-1f08-027b348a71e1@suse.com>
Date: Wed, 8 Sep 2021 12:43:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210907161908.17402-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0005.eurprd04.prod.outlook.com
 (2603:10a6:208:122::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2974fa39-e0e2-4986-1473-08d972b57bbb
X-MS-TrafficTypeDiagnostic: VI1PR04MB4384:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4384C4F47EC4720E3A915DBAB3D49@VI1PR04MB4384.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ytJvn9LIvS/BYip1OR527o0y+Dx980pEnCK7xsQMETl5yJM1/7GTGlcGqPP7WEypUNAT2sVK+KelGUKqQ4dSLvFaTb64OsXCYJAruYIzoGuVGHmlRAZOcEEPoLtHQfJq0/OeQZZsjFzI51rm4DiFw+XqQqGOuZ/8JA9cYDPkn3e6k/rm0/dolfwIZG2rWGHkGrzQtA4fkPQGn/mLf8TIPQ98nPTlICTk46Gc8Qg/lukt3OfakMDw1FpxLofbbOIXDpYd28/qBivrc5xopFLHNsvv3ePllrNf3NcxFZjPLQBP/13sQrCPPwib58WmYaeyKuHsKnHLFin5kbphHA2/qET3tethmGZRY4R1oMcaigZkIhLflFbSBXvnjNuPI02GInE3T/yVxXnknpXZBDaGfFEG/3SdOdasPVX1KtKsleVex9gD/fYamV5dIr1gn2+mCu2S+cFVnWp71RBJtu5JyPzaiOuzR30c605kovdxUN6DXeCrD2ngCFLcMVfa9k/L8boxAI3a2RuXTH8CuNNs0XGbWGykaP15f7549gk86dYyITfDA5tpxlIPrHVCnuLK/0QqOromtLs/Kkx3wJHEijUjd5z3SIsTkWDBS6lNPcnSuG96aWf77xAdQrUWwUtpWRTN67Qap5I2jbmJ3fx7kfkHTzZntVt9T0uzZrtPgmqJu9l/p0OQ+OpmTpPpqn0YAY9F0CoGOMQqiRE+G1UF7iAkbYnjc4C28hYd58wzPcWlVJ+kq/4NR4VKuAMhiUoH
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(39860400002)(136003)(346002)(366004)(16576012)(36756003)(53546011)(6916009)(8676002)(4744005)(956004)(186003)(8936002)(38100700002)(478600001)(5660300002)(86362001)(316002)(6486002)(4326008)(31696002)(54906003)(66946007)(66476007)(66556008)(2906002)(2616005)(26005)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dlNPS2tqU0VVTFlnaEh1Q2hmWWtpRmFVUW9wTWRiaEY4dUhFTXNkSUgrbGZm?=
 =?utf-8?B?QWszZzRNbTBvcG44R0pZUDQ3dG1qWU1kaCtkNlRGQzNWWUVVMmlJZGlEYlhJ?=
 =?utf-8?B?dzRIbVNGN3c4YlFKazg5K0p2c1FSeUliNUEvR3g0QTlHQ0dUMTh4RXRtdnhl?=
 =?utf-8?B?NTFsOFg1TUEyUEdHS09qMDdPL05JUUtxVGFrelNid0N3bTMrWW84aU1weEps?=
 =?utf-8?B?QXp3azNkMldWbGVPRlI4ZUl0aEhBVmx2MWlrQ2ZjaVBqVTg0cE81MDdkb1JC?=
 =?utf-8?B?QlpKalRtRHhlS3YyeDRIMnJmNEpGZGpaa0VKaDhuUnJUZ0UwYU1mZkZtRjA1?=
 =?utf-8?B?ZzJWbHVyNTk4TlA5RFN6YWx3NXQwZVVXRWJRUkNoSGtVbjdJK1JiS0ZpNytL?=
 =?utf-8?B?Um1yWkgzR2NaVDU5blU2WmxzbTM5K1dldzUxWGFiUk4vYXNQOXNwd3hOdm85?=
 =?utf-8?B?NUR4Q29jWkhXREtIcytDazRXaXd3VHR5QVRPem1ZNUVOMU9WbVBvWTRTdXUv?=
 =?utf-8?B?Qk1wYW15WjU1NkdhZ0hTZFh4clhBYkhndzBrL1ROcldjYVZYT3FTalMxTTcv?=
 =?utf-8?B?VDVwai9sQUE0TFE5UEI0NmhDV29kYjZHeUxvUGdKbTV0Y2hHYzdXS3MrdHRr?=
 =?utf-8?B?clpPZ2xJUk9YNVYzLy92cElranRGcm8rRDl3TW83NG55L3hFdndMV0lBNFFG?=
 =?utf-8?B?MjAxL0o3MmdhRDJqODFmR0ZieUw3YmtJK3EwcHBzMVZoWkdLejVwU3VsMFlz?=
 =?utf-8?B?YjBQTWtRS2ptQmNPWURmcThRNERJN1VnWjByRDhZTXoyL2RTS0xIdExnbUFi?=
 =?utf-8?B?VXhwdU1XbGszL1B6d0pYUlplOVJOdzcweE5GZGhBS3E0WS9xTVRYMCtiazFO?=
 =?utf-8?B?SE1DSkE0bUJaNzZYNmNQaUxCbHBMd3o5bGFXQ3ZoMEF4ZHNKU2JlSFF0TXhK?=
 =?utf-8?B?NVRBU1VWUDN5V0NUTWRRMnZDNFlaK3MxRGc1NzRUdDFtWU1FejdpRWt0ejcv?=
 =?utf-8?B?cjhVbVNMbEFFcUJWNWpMdld5ZmJpRHZyRFFBL3F5ZDNWclh4THdlOVhqczYz?=
 =?utf-8?B?d1g3R3RkVmkxZWxGL3JHL3FRdU5LUUlPM2hZN3ZXWTkzeG9xbDEvKzEvQTla?=
 =?utf-8?B?Y1ZBZVFVSzBNaVkrbmM5YTZJNWpSbXlGY1NCcGF0cERJV2VkM2ZzMnl4U0M2?=
 =?utf-8?B?NlJManlHbGliTDB2K1MyMlhRWGZyUHdScWtSZE1QTzR4L2dQcFZLRmZOMHp1?=
 =?utf-8?B?RnhNMmZqdXdScDNFYmNwWjFGNzAzMEg0blQ1QXliVUlhamczS1VpS0ErRDdo?=
 =?utf-8?B?Y09Xd3g0OEVHN1hwY3BzVXFSUEFIQW9Gbkd6MlZDZVNoNnZwY0FoR3A3MjF5?=
 =?utf-8?B?NnpHbk1aL25hOEpERTRTUlMwUlJpSUJPQUJiMEVJbVVxM2NjM1g3dE1zR1ly?=
 =?utf-8?B?N3d6dncyVnpUeSt0b2xDZHcrdkZ0eDNlZ05UbmVWaDlCVlJqZUNrTHgxTzNh?=
 =?utf-8?B?dlpFWE9VdXJqWVFXak1hWTBpd3RyZytsYTRJYlZiSVUyVmlmQ0M3a2tsbEpB?=
 =?utf-8?B?SEVPMEZ2SmhQUHB6a3BvampBUTJLcnhPNnorUVBMaitNRHBQdGFrY2gvOWdN?=
 =?utf-8?B?dWtrS1VsVmtTRnRMSkxXbzZMZmRKaUJJYTNPR0VZWUprZEo3UUlPQUpFSjlR?=
 =?utf-8?B?ZThkRGZzUFNsQnptZnIvdERuWHM4S0VMYzZBQ3EzbjltejEvUUJXdUJGMDUx?=
 =?utf-8?Q?ET4nFEcWO1w0NdhY8SDp3Lqa/iqYPl6AmgTq3W2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2974fa39-e0e2-4986-1473-08d972b57bbb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 10:43:24.1290
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7EpAq56XH6YiOn2fBn2mlCNzgB14T2NVhkTTqJV2vo8m5HCko4W+aOj4u63PDnbfYZ4ZPXgx6LsJaYyA4gvKYA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4384

On 07.09.2021 18:19, Andrew Cooper wrote:
> The opencoded legacy Memory Disambiguation logic in init_amd() neglected
> Fam19h for the Zen3 microarchitecture.  In practice, all Zen2 based system
> have the architectural MSR_SPEC_CTRL and the SSBD bit within it.

Don't you mean Zen3 in the 2nd sentence? Otherwise there's a missing
connect between both sentences.

> Implement the algorithm given in AMD's SSBD whitepaper, and leave a
> printk_once() behind in the case that no controls can be found.
> 
> This now means that a user explicitly choosing `spec-ctrl=ssbd` will properly
> turn off Memory Disambiguation on Fam19h/Zen3 systems.
> 
> This still remains a single system-wide setting (for now), and is not context
> switched between vCPUs.  As such, it doesn't interact with Intel's use of
> MSR_SPEC_CTRL and default_xen_spec_ctrl (yet).
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 10:47:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 10:47:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181758.329040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNv7F-0007xL-E1; Wed, 08 Sep 2021 10:47:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181758.329040; Wed, 08 Sep 2021 10:47:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNv7F-0007xE-Av; Wed, 08 Sep 2021 10:47:41 +0000
Received: by outflank-mailman (input) for mailman id 181758;
 Wed, 08 Sep 2021 10:46:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6l89=N6=gmail.com=ysaikiran1997@srs-us1.protection.inumbo.net>)
 id 1mNv6F-0007uQ-Gq
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 10:46:39 +0000
Received: from mail-pl1-x633.google.com (unknown [2607:f8b0:4864:20::633])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5db95af6-931d-4862-bf9b-120a9091ffaa;
 Wed, 08 Sep 2021 10:46:38 +0000 (UTC)
Received: by mail-pl1-x633.google.com with SMTP id bb10so1054611plb.2
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 03:46:38 -0700 (PDT)
Received: from Sai-Kiran-Y ([106.51.80.86])
 by smtp.gmail.com with ESMTPSA id b5sm1869785pjq.2.2021.09.08.03.46.35
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 08 Sep 2021 03:46:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5db95af6-931d-4862-bf9b-120a9091ffaa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:date:from:subject:thread-topic:message-id:to;
        bh=58hJPrkGl6/diKWfZw8+65HdLLM506Up/fI+9XRh7vc=;
        b=MCfwu5pzVrinp4wQEmfrQX/x9GK9TFCSP7pNxv1G/h3jOejGhCsZUGGokEjkyC+h8M
         vRYGlwwvALB7CS/A7w8+IleaY0azcE12bH4zQ1sbXVbtVJf+ujX+eDwbRscHE4eOwAEi
         3vvSnTY2TPTtSofQNrrUXW705nA5MO4ia06M5ZuhbGyXB0NLu78vSRipp0TM86WnBE2f
         +4WebKz8+ed+b43NcImIRS4HFLKLTsqHqApWYUe8Mz12f5Z60BvjDy2diBJhpQVrvJaj
         Kv5MB0vX3gEdqgCvSownCBfQ1PTrW1Dmne17VWykk07zRS2eZSlQC00qyeohEha38jUz
         s1fw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:date:from:subject:thread-topic
         :message-id:to;
        bh=58hJPrkGl6/diKWfZw8+65HdLLM506Up/fI+9XRh7vc=;
        b=caRUN0KZwAncDWVkQNQIIkikTUBfJdrNpEZhyDSq4/DeGn7oJa+1eG5Nco9JEaTjyr
         T9S6awnUkKKP0DJbwugvnYb+CrwpyIZXCb+WFNbS7EbdqlFWf88Lm/QCeUBfMQkbJXQV
         IQy38ARZC8rgBnyudPQ045F9ckqf/7X4CxHDvgHBMvYKXb3t7p4Tw3WLg8uu9X7MWEzA
         GkfXrBXgWJJCNFyPsLQzbu4d544oFOR92bAqzumddtjiS8irbB5R4B4TEFQijFuMhFdF
         wNLI29xybZNHq7kb/Ed03wG2nuz7g1Tc7Ohgts/MK+ebVRvnMJbZQc6wQ1RkEBc1qPlm
         vZpQ==
X-Gm-Message-State: AOAM533wVxOaoxKbmaEv2Eqmw6DxXIqzd2EF/QpSMtKjvdJ6i8DN5LDO
	MHJj9tSWbciDa+ae5du2ozkxtiJJtBI=
X-Google-Smtp-Source: ABdhPJy/8ww9rv+dnSDtuG4VFlvKDuydQk07awo5BaK9YZH+Wp4oQ6eZa/QsVYbg7U48CswZx8mRiw==
X-Received: by 2002:a17:90b:3110:: with SMTP id gc16mr3461111pjb.199.1631097997376;
        Wed, 08 Sep 2021 03:46:37 -0700 (PDT)
MIME-Version: 1.0
Date: Wed, 8 Sep 2021 16:16:31 +0530
From: Sai Kiran <ysaikiran1997@gmail.com>
Subject: Xen Booting Problem on ARM Machine
Thread-Topic: Xen Booting Problem on ARM Machine
Message-ID: <1B61076F-D44A-41CC-89CC-A8E0E65E64EB@hxcore.ol>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: multipart/related;
	boundary="_30A821E4-34E6-412A-BD80-6EC1FA5235BE_"

--_30A821E4-34E6-412A-BD80-6EC1FA5235BE_
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="utf-8"

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content=
=3D"text/html; charset=3Dutf-8"><meta name=3DGenerator content=3D"Microsoft=
 Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#defaul=
t#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
.MsoChpDefault
	{mso-style-type:export-only;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style></head><body lang=3DEN-IN link=3Dblue vlink=3D"#954F72" style=3D=
'word-wrap:break-word'><div class=3DWordSection1><p class=3DMsoNormal><span=
 lang=3DEN-US>Hello,<o:p></o:p></span></p><p class=3DMsoNormal><span lang=
=3DEN-US><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-U=
S>I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it. I am=
 able to do =E2=80=9Cmake world=E2=80=9D and =E2=80=9Cmake install=E2=80=9D=
, after =E2=80=9C./configure=E2=80=9D, as specified in README file. When I =
reboot the system, I get the following message:<o:p></o:p></span></p><p cla=
ss=3DMsoNormal><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p><p class=3DMs=
oNormal><span lang=3DEN-US>Warning: All 128 bootinfo mem banks exhausted. <=
o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>Warning: All 1=
28 bootinfo mem banks exhausted. <o:p></o:p></span></p><p class=3DMsoNormal=
><span lang=3DEN-US>Cannot exit boot services: ErrCode: 0x8000000000000002 =
<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US><o:p>&nbsp;</=
o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>Here is the snapshot=
 of the error.<o:p></o:p></span></p><p class=3DMsoNormal><img width=3D516 h=
eight=3D387 style=3D'width:5.375in;height:4.0333in' id=3D"Picture_x0020_3" =
src=3D"cid:image002.jpg@01D7A4CC.E2C899C0"></p><p class=3DMsoNormal><o:p>&n=
bsp;</o:p></p><p class=3DMsoNormal>Can someone tell me how to fix this erro=
r?<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNo=
rmal>Thanks &amp; Regards,<o:p></o:p></p><p class=3DMsoNormal>Sai Kiran.<o:=
p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>=
<o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Sent from <a href=3D"https://go.m=
icrosoft.com/fwlink/?LinkId=3D550986">Mail</a> for Windows</p><p class=3DMs=
oNormal><o:p>&nbsp;</o:p></p></div></body></html>=

--_30A821E4-34E6-412A-BD80-6EC1FA5235BE_
Content-Type: image/jpeg; name="D00C86F8093B4937B59AA7450B930686.jpg"
Content-ID: <image002.jpg@01D7A4CC.E2C899C0>
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="D00C86F8093B4937B59AA7450B930686.jpg"

/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsK
CwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAHkAoUDASIA
AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA
AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3
ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm
p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEA
AwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSEx
BhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElK
U1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3
uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDBj8Ta
iGEn2uPI55WrS+OdQMp2y20rkcnb6VxUSyKxCyqcjPLCqojutxKOxU+/FeS5O4Hdf8J9qnnAhbds
cnnj6Gp5PHl4twjC3t2XqY1J/LNcDHDNHjDFePrmnia7VslWff8A7JGazcitTvJPiBdcM1nHgHJO
7GR6UD4iOUybHKZ4/efpXCNNc7F3gfL/AAsgz/8AXqLzJJOQqgZ+lLmZSVj0ZviNaxqS1jJ0GMMP
vU61+JFqrAvbysh69DXne9mQkE4XhqjW4yQqg4+vFONyb3PTW+Iek+YzK0wHbMeP0BqSPx/pskYD
SSDvuKmvLmBVtwJU0RbpMjI/GtNblWPU7fxrpskgVr3aPU5FW7PxhpxkZkvURgeMjr+leQzQqrAr
ySeacqBQGyAe1O4z2I+N7KaRSb+M7Txnjn8qvyeOonVW+1257Z3A5+teIR25Y5HC9cilNuONilvW
q3Ee4Q+PHhPE0Az3XGPx5qebx2G2r5kEgPU7ua8K+YxhdzVF5cnIyQeuae4z6Bs/GVvbyl1jiJK/
e3cmpP8AhPA94sr20Zb+Eb8DivnyKSZDnzGH402SS438TyE59TS16MZ9Ex+NM3Bne3ZwTk4fj8KU
+MoVUiSOYBmB+8K+ekvbxUwbqRv9nJxQurXx/wCXqXAx1Y0tSuY+kF8b2izA+TK7DoQARU9j8QrN
WYPJLu7nZ0r5tTXtWj4+1thRxzSr4m1aPO26YD+7xRqh3sfUf/CwNLbyxE0m/GCzxmnf8JZokk3n
NcswUEsrIw4r5eTxjrHI+0bx6Mo4pf8AhNtY4y6nB4G0DNGouY+mW8XafOygXICqeGIPAHfpVtfF
1j5gP2+EY6Et2r5gXx5qaZJ2AHtinN8RNQKgfZ4eDkHB/Ks3cq59Tr4ttLqJoDeq0b92IAFMhure
7PkrfQr82Ad4wfxr5i/4WfdsI1ktI2KjGQSCalg+KEq4zZru9d3SlqB9NTTmGVoFnWYY/wBYXBrS
tZEs7dJVaKScD+9nP1r5cj+KShQPsjZz13VZj+KcIkDNbzg98N1pNsZ9Q3GsTspSWOPY3Tbj5asL
eRrY5kKoey45NfMC/FazKqA11GxOSQf/AK9WofjFbx4H2m6Xjb82Wx79aV2B9IzXkJsXQxZkPzCQ
VShUwxgHc7ueIw/SvB7T4xW0bHOo3Sp/eO7j8K0Lb4tWT4J1d1YcBiCD9TxRdiPcGupre4KupjG3
5R/CP8am0lY5Lgh5S+Qc/LgKfUV4tb/F5GfcNcQN0G8Dp+IrWsfimkcJRdZgMrnkMF247c1XNIZ6
tq0zWYkSDbMrD5sCsALc3FxyiwpkYU+h71xcPxGkaZj9vtWB4LBl5/WtL/hNPt0cKq8Fw5J3EEDH
p3qlJiOjvkaORI450Kt09q0Y1kig2RsyYOSx+6c1xcniaHfEyrDcEDBVjhVb8+ee9aVr4yknhcPG
FIHyqjZ//VV8zJ0udf8AY/LCoctn0Gc1l3nmWs6Rm3C5Oc5qlYeLp1hfYG2qQzbm5GfwqS+8QJdI
hmiwG+6VkBP5dqrmE9TRsZn01irwLOJMkJJ0P41MoW+kytmEmbp5hJUe1Yf9pRsY/N8xNpLLscDc
PSrq6kGjMlsZIdowdzcE1XMSS3kLwTFJAFHJIQZFELhhFFJGF3JgfL1PrVH7ZO6q0h3bWwZABznn
k1dhuJZo2L/vMD7wP3RmndgZ7abLDNuxlffrU13MkkxUrtPAOAcVsJNFNH5ce55MdGFZlzMzTAgZ
TPCkUXYDmWKztGdidn3vMPUe1S2N9DqEYUyMgzxvGOPWobiOa9+VGRVUfdxx9aIQ0expYI5Jl7r9
0j6dqdwLtxpf2d3kWU7jwi9mH+NRwmdNyzR7P4vmxmpY7oXEqlXZGIIdWIGPTBqTUFlYobfaz4yd
7ZzQ2Mrw6WY1BiKS7m3Fi3P0xUzsBcBp0jfsO1U7gFrQpIyLNuzsX/GrDrGbO3Rdpkxna1SBPNCO
Lhy3lKPl24b8OKyY40vJZ2y3zHC44Aq5uljjdbm4EcX8OD37AD/CsWOeTzgYoZGxwW255+lZO5ob
trHttzaCYpKT/EOCP6VbS1udOhkGfK7knnj1rHt9Qm3SMlrJKQMyMQMn6VqTXzxQKNqwl13Fc5NK
7Gc9qEyT3B2lt68E84PvUa280qrvX5QcgNzVy8jmaYFCvPPyY5FV42hmbayMzg42k4//AFVaegiS
G3hsQZmXcN2TEBk/XNT395aLbrcKWR2+XbtBUZ/Gs8QuxcB2jC9SvJqK4sg7BYjJdLndwuF/HmjU
V0MRZGUrDChZj95jhavRxBVH2yNdw6bP51Te4jjh2SbgOvXFWLF7e4besYY9D85JH4YovqLfUmbF
xPH9nRpQByrD9Sax73TUW+IjPOMs3ofStnz4oZxJbedFgbWBxj8BSrFBNbTMdyHqGwOfqc8U43Cy
sc1cNbbDglZFOD3FZ+7y52iYNLE2CMsMitCSONmYW0TZwQW6/lVc6K6gSuT5mBjd1+lUQUp1fc3l
qNinIGRwP61YaS1+xkB2aVhnYwxj8qkmcW3LpxnBJGKhmQXACghB1wg+9RcBIP8ASAvlsqSjj95n
FZ+pb7WZmldZX6bM8CtJYbaGxG4GN1OQzDP61jXSxXmW3bh1PHWncB1syXMqmUsFx90dKSbLXHlx
q+ScAgcCm6asixk+UpGeM5H61pxfabhRESidv3Zz+tNSsBVS1eNmSedWb0zmoJ7QNGwhkw/vnFT/
AGeJZWzjzF5JJz+tZ1zZzTCUCZV3dNhOarmJZXubtliWJ+T61Vt7Z7q4yjsjLyMjtV+0dxbiJ492
w4LNVqFEhfzFYMWODjsKnmZOhQuLMXK+SQzM3V8/yqK1szAdjcL0POSK1plLM0igYA7VSbHyv/eP
ajmYPQWO1g52BmHqwxRVryw33ufxoq7sR4UlxAqHJQ89MH/CnR6nb7iA4Qjsen4VK1wI1ClGYA9E
fqafHtaQsflLD+Jga5JIZXF3axt5gCyjvzxUttdLNgl0XHOCQP680yORWZwyKApwOFPP5VKsPmLh
Y8sP+mYP58Vny6FISa48wOwbHue9RxszYKhgqjr1xU0ccUsR8wBWUZCrH1pBbxFQR8v1UgYpcrHu
RzOZWwDkdxUSrIy8jGOR0FWFs1MgZSMfjSyW+WIB49d5rVC0K4jdTyQxqRW8yQghVPqBgfzq1JYh
FcFt3oQetRrCjfLgjHemHUhbC5xkj1zTTuwpIJUmpGt2AGFIFSLFuwMEbeKQyNF+YhDwaYGeOTOe
nTirPkquPm6/kab5QbqcD6VSE0RrcBt3GB9KUSNyQxx2pfKUlsHKj3pFRjnONtVZFAbjKjlTjio2
kORznNSNCo+7z9aj2fNg4OaXKAbmDFcg4pvnbWxgHPrg0rJlxz+tN8krzjmqsMXzBgDFMaUN/BxS
NGeOMCmtGQ2BmkytxPNB7Yo3rxkZpnklT0OfWnKCueufTtS5ewWY5im3pUbbWxjA9M0rb93U01wz
LkdKloFoIyxjGDn1pnyL1Jz2wKQqVHtUbHa3NSUT7U7nNN+63GKhViD0wPrTt3+zk9zTaQrjtmWO
0g4PNK0XANNU+gyfrTxknpSGN8ts+9KE6DqR3p4GOTn61KuGpiYwR7W9Kd15HU07aOxp6oAvXFAK
5Hu2jB7U9ZG2cMw9ADUnlnaO9KI1XGBn1q0kNix3E8bDErg9chjVldQvF5W6mU+u80xYhj0p4jDc
Aj0609HoZlmPXNRXG2/uF7AhzVyDxRqsJUG/nIXn5nzis+GHaw4BHpU72oGMdfTrTsI2YPHWu+YH
/tCU4IK9OPpWnb+Pte+ZW1CQkj7pA/DPFctb2x6nIPvVhYTxxz7UWQHZwfFDXlTyxPDjjOYhyRV2
H4ra3DJuV4t/Q/uuDXCxxb/fuSauQxjjovvRYVz0S1+K2rxxkG2t/mABO0/pzV1finfrgCygHTJU
kHHcda4KNQuOc8cVYyADg8k9etAz0aH4uXDQiKWwRk3ZG1iPwJ7ir1r8UIFjdX0tmfoGSTA/XtXm
NvJuGDkgdRvqwJ2bIGVHUDOSKYHpcPxIsYo5BNZSbWPRWGDxVi38f6YfLWO2nVm5Pc15qzyGMLHI
5Ve7YH9KnguJuCFJJ6tgHP6VLGen/wDCa6Mw24ljTkh2QZP61Kvi3SN2/e/HGSg49+vNeax6g0pS
NBjnBOwVYecbizRx5xjBTI+vWouxpHpv/CQaDMpZp1Q4BUMhO4+3pVy31TRZYgyXVvs3HIV+n515
RHqC7STHG7n/AGSo/DBqa3u4mjO9IRnouWz+I9KRXU9be7014x5V3AqsvGGwRSW9ro06h3uYcY27
nkwc15hHfRSxbRErR8biMj8MkVpLcWjuUMQRQM5kmJx+lIZ6DbaBZ+ZiOUNk7ciQAGpW8JRTKJRv
Pou4Hj6V559stkhMUbMSw3DbKhGPbOKm+1QeTCokmjIJB+dDn3OGyKYeR3Y8KwR5TEjFhnbt6e3F
QHwmiyOzq+0DJypIHtXFnUIY/mju7tfLOARwx/HNWl1S7kjd0vbzYT3ZsMff1NNXFZGtN4Ljy5Ab
Y3IDIfy56Uh8GvtVY5AJCeFwRWPFr2oIPKj1S8Zv7hDlTn69Kf8A8JDrVm2f7WZFxnfKADx9R+FU
2wsaMnhG7s8L9qQndzkdP8aj/sGWOOSGfGxx1U4NZc3jDVh8v9pwS+WwZiEQ5PY9P0qK68e6vcEs
slqwA2jcqc/p/wDqoUiXY2bfw3OsICouw8KOM/hUV5pGpOnl/ZiFUglsjLegrKtvH2rj5g1kHX/l
n8pH6YqK4+I2tvI5SG3ZFYAEIQfww3Si7Hp1H3/hfU7hiVjG/r5fJqt/Yd+o4spWn6eWi5zUs/xJ
1iVzI2l2sYGMurNyfTrUK/EzUoZGWXTY3LNk7JCoxTuT1H3OkTvZhBFKHzgoq85rAvNPu7GZBFDK
vPPydDW9D8UhAyu2ktjcSfLmyRx9KoXPxiiVQw0mVS3G5Tkn65AoTuBRtbW6GDNuaFx0K0+KyG0k
3L28S8AIp59ulOb41aPHGVbTbzdjHO01Vm+M2heaT5F2kaoPlaMHnuBzVCNDZbw2siwxBXxje/es
NpvOOySPavT5RjP41cm+LvhxljJeT5xlsQYI9qgk+LXhCRVaV23k4wUyFoAJrNbhT5EZRiMBd1UD
EbCYQscMRzuPAq+3xM8Hwoz/AG0K2MjEbE/oKik+IHg+4VJDqEDs3JUo3HsciglpC2s2/IEi4I6+
tT29rui6cZyCar/8Jr4MaNdupW8cmfuqSOPerH/CW+HJABFqtqS3QeaBj86NSSZUto12u5Lewoqt
/wAJBoqEj+1LVvpKDRV2EfCMfj65jJ2axMpz03kVYT4iXvG7WGbH95811X/CI2LYHkxn6oKhl8D6
e/3rSE++wVu6UuwXMSP4kagu5V1KNt3XIU5q2nxO1UYYXVuSvT92tWv+Fe6W/W0jGfRcUh+GekyL
gWyj/dY1n7JroMfb/FbVI3DFbZ8D+7Vr/hbV8xBazt29QpIzWc3wp0tvuxyKf9mU0L8JbLcdslyn
0lNT7PyHc2B8YJ+B/ZyIB2WU4/WrMfxgiXBfTmGfvAOP8K53/hUUbZ2X13GP98GlPwff+DVrkf7y
g1Ps12GtdTql+L+nMMyWM6kdgQRUifFbR2JLJcRD18sH+tcg3wdvHI26w4+sQph+EGr4JXVYivbd
Cf8AGjlQX1O+X4oeH5cBriRCBgFojj9Km/4WNoEmF+3qFxjJRhXnDfCHxAQNl9aP9UIpB8HfE5Y4
ezb/AIEw/pS5UPQ9MXxtoTY/4mMPtkkVL/wlWjsrFNSg/wC+68tPwh8XZ+W3s3AP/PUj+lI3wl8Z
5H/Erhfv8s4pcq7jPU4vEGnSDAvrYf8AbUCpRrNiwG29gI74lH+NeRf8Kp8ac48Ps+Ou2dD/AFqM
/DDxmiknw3dH2RlP9afIu4HtMd/bSt8s8bDvtcGns0btnI/Bq8QbwH4uhYf8UzqmT/diJ/lUMnhn
xXbkl9C1mMr6W8n9KOUZ7uUC8ZznpTDnOPmz24rwdl8TW7BWs9aiOMhTBKP6U0eIPEFrjfLqURP9
+Nx/MU+VjR74U3cE8fSmFcNgnI+leDJ481q3b/kI3Kn/AGgf6ipU+J2sbh/xM8n0ZV/wo5Gx2Pcm
yGx1+lC7jzivF4/iprC9L2F/+ACrKfFjWAeXtm/4AP8AGo5ZXKPXuinNMZmxg4C+leVp8WtTCgtB
aufoR/Wp0+Lt1n5rCFvo5FLlkB6Xtx1GKjlTpjn8K89/4W1Jxv06PPbbIanj+LEOAGsHz7OKnlkM
7gL6rS7QwHHFcYPitZNw1jOPcEGpY/inpZzuguF/AUuWXYWx2LBeCq44pmdvbmuaX4laK6ndJMv1
Snx/ELQ263LZ/wBqMinysLnSq3vTo1zk54FYEfjjQ5SMXyr9Qf8ACrSeM9GbgahDx6nFLlaGbirk
e59qd5bdTWZH4r0iTaVv7c+28Vci1/TJOFvYCenEg4/WjlYiztOOmakjj8xs9+1Rx31s+f8ASIm/
4GKswzR/wyKc+hotYBSh3Z61Kq4420scatn296m8scHP61dzMaF9uKeMt/e+tJsKZA+tSJlWzk9c
0wsPXGPlzn3GKspwF54I61AsnzDbwPzqf5l4J6npmlcCaNSM4bNXIVGcMePrVVVGAOmKt25G7jp7
invqIurjaApViTgLuyR79KsrC20t8rEccYqlGobcx454watRxjcuPrnP/wBai6KSuWo1IUgBSOp+
UHpTlhcc+Xtx6f8A66RYFPAyPXaam8lY8bt6LnPykdfpU3FYuWyu0igRlQ/3WOfT61ZaFg6/umU5
5bkZHp1qmsMu0fOwQ/dPXP61Z8l4YwEk6jAPl9/Q03rsNMtrH9nYKRkHqeeP0pfLZ4wzK2B0+cDP
/jtRxpLbr80pY4z/AKs/l0qdmuFjX/SVDE7vnDLkZ96ge5Lb24mBZxIrc8hwO3BPFS+TGqjHysDg
M20j6kEjNQfaJwpH7uVumGfnrU8d1PJDGsn2RSowD5yrkfnSLFjjEyFQzOqjO8Io/H71SeWishw6
fLjf5H3j+BOKf9ufcF+yxu3T93Mhz79KtLNOsYSXTTEV/j3Ln9BzQvMBkenxNC5S42ux4zG4I9s7
OfwqOPSxCiDzljO3LBVkBB9/l/lVqO+lE7SSafcTFSAjRjjP4NUs1+6ymRtOukB6RmMljnvwc1XK
Iq3FvFLGUMy5BOBuYY/A1VbTfMEebmExMMA7l47ZOT0rSXxJbCFnnguI5AQu1o2BbrnkHio5NdsZ
IyfMkhy24IWdh+YBoWm4myBdOkXJWeNwCQvlyxnIH4mq11HqCsgifa6hcASFic89hzV6bxFYxnmY
x7gApYnCf3jyo9qg1LVtPmRQ80BjYc4UFm+vvTeuoepk3DX8zPEssyEj5t4YHPfvVZrbUVmZnhZ0
TGGaJgOe+M+1X01LSYyrebCr7T8xgiI4GADmnfbrK8XMc0ET7OfLCDp3AVuetNIZlXExVgkq+S7D
PzQ46e1Y8jzW8rAxxopPeLn/ANB4rrLcWi+XJPK8khGA6vIAD68Z9qpakhng3yXTGReAGnkOV7Zz
7VQrHPfbrmZwkcMMyL/Gm5ST69Bmoby8uk/eShvOZeRuIxz1HNbEdnMwV/tD+Wwwvv8ATI6VXNgD
KxE/mcZCEJz78rTIOd/tK63o6mUgffYsePphqoXWtXU0fm5kT+H7+fx69a3rqxEcwjkkUBuflERz
+QrIm097dirEkAnjORn2xxSEYkmrPt3O8qHOCwOf6VTuNTlZhtZmjZSQHQE8d+lbV5YxK7EgSOq8
RshyW+oNZE1uLqTAjXfjmNUbj6c0x2drsyrjVPLYqrBt2Du2dPbkVQuNQ+YKEUH1YDArXuITuLMq
hV4OBjGPwrNubdWUhUXn7vPI/SkRvqZk1wVYnCv744qvLd/LkovHpxitWS3jWFg28vn+98v5YrPk
tRJlQfyq+YGig10pOccDrUUk6P06+lWprJY14XPvmqU0QVun14p3JI2mHZiv40VH5QopiNuMA84q
YRluDTVXFWkHQV69mY31IWtv7ozU0FnI+Qqlj6AZqeOLJFfTv7I9jK3hz4hSWUml2mqR2cMlve6t
GjQxfvMHcXBABHFTLQ2jqfMH2Nk6ghvQipVhbgkYNfYPjX4P2nxC1rwidVubKLUnsry71bUdDjQ2
9xFB8wWHaAhfbx7HrXmmufBfw7r+j6frXgLVL67sZtRTS5l1iJYpEmflcbSQVx1NRoyrHh6xkdua
tLHt6CvfNb/Zv0SBNd0zRfFjal4s0OCS4vtMks2ijVYwDLtlJwcdvWtDV/2dVs/2ftP8T272MmsL
O9xczLfIV+zGMFUAzjeD1Uc1Olx2sj55SMvz39qmjXauMZr1r9nDwTbeMvHckF1bWN+kFnNKtjeT
GLz22HG0gHJU/N+FVfAfwL174jWGpajpzWcFjY3Cwzz3lwsKR5ydxJ7cdfcUE2PNYl3AnGKuRYXt
XojfAbxXH4vHhyGziubpovtCXMUq/Zmi7SCU/Ltzxn14qXxp8C/EPw/8NyaxrJtIIlu1tFhjmWRp
GZS25SpIKjGDzwanlQHCWuNx3CryqCw447Yqpb4HBGa77VvEOheIPC9kr6cbHxBaBYfNtVCwzxjo
zDs4Hcde9Q4IPI5i3bYR8vPate0k+WvRtH/Z5vtS8K6Xq6azpqX2pwNcWWkyORcThT0XIwTXDW+g
6jNdT20dlO08JxLGsTFkwcYIxxWbgh6odb3Hl9Oa0bW6IkG48Ve8J/D+88SWev3KSLbDR7RruaOV
TvfBA2gevPesYI0JAdWQ4/iXFR7Nbl3N2O6HBOCKtx3UTIQ0UZ7AkCq2ueHbvw3Bpct08TLqFqt3
EInDEISQN3oeOlZkcjMc9qhwHdmjdWemz8NYWjj/AGoVP9Kxrvw/4eud3naFp0n+9ap/hWkFVxyO
aq3ChelZ8iHzHP3nw98HXZJfw3pZ3dc2qf4ViXnwZ8DTBv8AimtOGf7kQX+VdZIryNtQFj6Lyary
+ZCxDKyEf3hiqUH0E2cBdfAPwKyk/wBgwof9iRx/WsK8/Z78GnJGnSxn0S4cf1r1eSZmHOc1SkLc
56VXs2HNc8buv2d/DDNmP7dEM/w3BOPzFZVx+zzo/wDyy1DUYwP+mgP9K9vkTvjBqjLCV5/Gjkfc
dzwm4/Z/hVj5Ws3aj/bRTWdc/Am5jGI9bYjPO+AV768HzdcGq01qOoFX7N9yOY+fJfgpq8f+r1a3
b/ehI/rVGb4Sa/H0ubNvqGFfQstv83A/Sq0lsp/hyafspPZi5j56f4Z+JIMALav7CQ/4VVk8B+JI
+fsUL/7stfRElmvPy5/CqzWCnjbz1o9nIrmPndvB/iGME/2Ux/3ZFqL+wdaj66TP74wa+iGsEHVa
jaxVmxtwKXs5ApHzs1pqkOSdNvEx/sGk+1XkP3re9T/tm1fRsOlCa4VNuSeprYTwlav1D/mK1hh5
zV0jGdaFPdny5H4gv7ZiUmvIfT74p6+OdVQ4/tS8VR/00evpPVPDNhZrym5m6ZArEm0CwlyGtYiP
dBUSpODs0aRkprmR4jD8RtXRdo1u4C+hep1+KmtRjCa1IMepB/pXrMnhTS5M5sLcgHvGKqSeBdFk
OTptuQf9jmo5fIv0PPIfi/4gTBXU43P+1Gh/pVyL41eIlYMLu3f/AHoVIrrm+Gfh9zk6dD74yM1A
3wl8OzNlrMoP9mQilyrsBjQ/HjX1IDpZS98+Xg/zq3H+0HrKcHT7Fz6gMv8AWrbfBrw6wOIZwf8A
ZmNQSfBDRGUFJL1T6LKD/ShwiBYh/aO1FFIfR7Zgeu2VhWlb/tMFeZNABOfm8u5xx+K1z8nwJ05v
ualexn6g1G3wAj3EprN0g/2kB/rS9nEDs7f9qC1XPm6HOoPTy7hT/StSP9qTQWC+bo1+jgfeUo39
RXmP/CgLlmbZrrYH96H/AOvVeT4DaqGAXXISB/eiIodOIXsey2n7U3hiJjvsdQwf70S8fk1aVv8A
tQeDZpP3ovrfHQ/ZyR/OvAJPgX4gGdmqWbj3DD+lVpPgr4pVjtuLOQAdd5H9KPZRXUfMfTEf7Tng
mbGb+4jYdN1u/wDhWjbftIeB5mLnWfLcj+KKTJ/8dr5Ll+EXiyPOIbaQL/dk61Xb4YeLoz/yDo37
grIP8an2Ue4J2PsW2+P3g1wVj8QWqnr8zFf5itK0+NHhW4UKniTTFC8jdPGP5mvh+TwB4rhb59IY
n2YH+tQSeDfEkeRJos+fYUOitri5tT7vj+K2gTt8uuaazbdvyzRHv/vVrWvjPRLqRIvtulSN1Hzq
fwzX54TeHdchxv0e5/74qvJpuqx9dKulP/XM0vY+Y+Y/SmPWLGYRpaixlUMGZowSemCODVqPUrXz
FZrYHnuXAA9+OlfmVjUYTn7LeJ/wFhinx61qlt9yXUIf90uKfsmuo+Y/TK61i0jUvtnx1EcVy6Ip
zx3qnc6lHzL/AKRJuHKtdklQPqTmvzjh8d6/aghNZ1aEd8XEg/rWhH8X/F1vyvijVlx0LXTn+ZqP
YyFzH6B3WrySxyNulTAwfmQn2wcZ/Gs430bYLmYsAMZjR/zytfDUXx68bRk7fFl8zH/npJu/mKsL
+0J40UENr7S56+ZHG2fzFP2bFc+2FMDKwzgHs9qhH44FQCWxjZjIYScHH+jEfyIr43j/AGivF6nJ
v7WX2a1jxx9BWjB+0x4sVNrDTpO//Hvt/karkYcx9gz2NhcQGaVrNWAH3Y3P4fe4qnPb26q3kPZu
rD55BvBB/OvleD9p7xDHIrS6fYXGAflbeBk9+G61N/w09qu0b9HtARzuSRwc/nRyMOY+lZcMyIbi
Jl6Iy3LDYPTlTVS4tpBHMIpHi2EA+Xe/oOBmvnqH9pqdf9boqtuOXKz8n2GQan/4aTtpFbzdIlUn
urpkf+O0nF3Ee4zKYVjYXN47HIKidW69uTxVWe2nYiUy3ETHnLeW+Pfr/OvGY/2hNMeF0ewuBuOd
yhc1Lb/HLw4ZlcxXMbdyYQ34jkUcrC56VdQ3LeYZL66kkX5QGhBBHYZ3cCqN1BeRqw8wIi/MC0GC
fTpXEyfGrw9cXErC4uIg33S0Rz+jcVDH8WNGmkJOpEHqCwb8qOULnXSTTyRkm4jMjDOwxP8A4VWa
a9wgEiqynORuyPesO4+J2i32W/tKFTjBwSP6VGnjrSmYhNThCdQrv/M45pcojY865dihkifqeTj+
feq000kbEIkW0H1GazZPF2nSKnl30QHU5cc/SmN4mtZBxdxsT1+Ze340+UNixJfOScRqf+BCq80r
NyY/rUEmsQTZ2zRkn3H+NMe9Q4GVY+xqjMdI/wDsYoqnJcJnqv50UFadjqI/m4/WrMS/nUcSD8as
xrzXu8rOclhXa1exfA/4ieHfBtr4o0rxNDfy6Xrlj9kdtP2mSM7gwbDEDtXkUK96sx57cU3C5Sdj
6Y0349+D/Cd14Y0fRbbUpvDenQ3sEt3fKq3A+1LtchFJBCdRzznFZ9x8RvAngHwrY6J4V1K+8Rq2
tQ6tNLc2v2YwiMY2AEncTn8MV8/xrlTmkkhK8io9lbY1Uj6uk8deAdL13xn4+g8Vx3d74g065gHh
9bZxPFJMgXBYjadp5PP0rgvD+raN4s/Z7u/DE+s2mkarpepPqqx3jbRcoYtojjPd8jpXh6DnrUq/
Wp9mPmPoT9kXQ4f+E8i1+61jS9Os7JZInjvLpYpX3xsAVU9Rk1o3ump4J+DHxG8P3Gr2E96NWs5I
xZ3SyCZTknbg/MAOvpivm6EFT6VYjYqwIrP2bvcLn2t4Untdau/B+nTywv4fuvBiw61OJQPs0Syk
jLZ+Ulto/GuG/aU0NbXwh4di8OizufB2kSyQx3FrefaXSWU7sSMAAM44H1rwjwR4+1fwDrA1HSJV
jm2lHjlQPFKp7Op4Yex9K6bx38bvEfxG0u307UTZ2lhFJ5rW+n2ywJJJjAZwOpA6fWp5XcG7nER/
eq/btjtWep5HNWYnocRI+rX0fUdWs/gfNZ2U11FDCGlkiQsqATAksegGAT+Fd9ceJtE8O6HrOspq
V/Yi+8Q3SfbtDiSV5sAYDZByo5xXyXYfGDxfYeHP7Bt/EF5DpHlmL7KrDAQ9VBxkDk9DVnwT8XvE
/wAPoZotE1M20ExDPFJGsqk+uGBwfpWXKyrn1HoviiyvvEHizVtE0q6MsfhxZW/tK18t7yUOT5pT
+LOBz3xXmnxQ1KTxP8MvB/iXUY4W1q5ubiCW4jiWPeiH5VIXjj+tee6b8ePGGn+IL3WV1Pzb+9jW
GZpo1ZTGDkIFxhRx0Hqam8RfGTW/F2paXcalFZSwac++GxjgCW+c5JZB1z0PtSsB3HxrWO7HgW4S
1hiku9GheRbZBGHYsewr1m18G6LfafeaDeaf4bsZrewaT7DbFn1KBlQMC8nRjnBJx3rwrxn8eL/x
xpFtZ3GhaNZy2+37PdWkLLLCFOQq5YgD2rctf2k7qGOad/DmmvrF3bm3u9TyVlnym3J7A9Dx6VI9
T0FtJ8Iabqngvw8/hiC6udcsoWnvXlZTHvBG9R/eyCfyr598ZWKaH4i1TT43Z0tbmSFWbqQrEA11
tx8YZLjxV4U1t9PQf2FBDbiFZOJQhJznHBOa4LxRrR17XNQ1Fk8r7ZO85TOdu5icZ/GlYD0r4KTN
pPhfxxr9nDC2s6bbQ/Y5pY1k8su+1sA8cjivTPD+m3mvfEHwk3jbwtFBqjWV4bi4liiMV2qrlMIv
AKg9+ea8Q+FvjzSPDdrrmja9BdSaPrMKxTy2OPPjKNuXaG45PWvTLX4++D9J1jwutjaaoNM0e2vL
ctPiSVvNACN97vg59M8U0NmhefCzRNKXxre29lb3WmX0Nrc6dIF3CFHnw8anJ5HQ49cVhP4F0C8+
PXjHS5NMgGm2WnXE0Nsq4RHWFcEAehOfrWZ4U+Omn2Pwv1bQNS8572O4WSxwgIaLzldo8j7uCCef
WugvPHHgWz8Q+JPHNt4la8vtWsJIF0UWrLJG8iKuC5+U7cc/pTJseL/Df4dz+PtdaBZI7ewtIzdX
11KTtigXl2wOScdhXVeOPCHhO5+GuteKdAs5reL+3VsrLzHPywCIEgjPdsnPXpUuv+N9J8K/D+Dw
r4TuBdDVII7jVtQO4SGTvCBgYC4x3zmtnwbotl46+BEvh2PxBpWlakmtG6aPUbgRZj8oAY/HP5UW
A+f9H0c6xrlnYiWGFriZYhJcPsjXJAyzdh716b8WvhW0fxasPCugaLDpVxcwQJHbR3Rljd2By+9u
gOP0qhD8I76P4jW/hyDU9JvLhQlwbmK8UQbepAc8bsdutfQXjfwzcR/tLeEfEpltX0aeSG0ilSdC
xkWNzgrnP41S0EfNniz9nnxl4R0f+0b/AE6NoBII5Ft5llkiJOAXUcqD6n1FcrdfC7xLD4wbwsdI
mk19SAbGMqz8qG6g4+6c9a+gvAerLfaP8W727vPMEl3bbpZXz8huzzn0wK9D0X4c63fftS3XjBbI
jw9s8yK+LjZNmBVXZz83+ArSLte5J8Tat4L1jSLC1u7zTLi2tbuWSGCWRMCR0O1wvrg8Vma1oN94
fvpbHUrSaxvY8b4J0KuuRkZB9QQfxr7H8Cx6WnhyBfH8lhJYXmtTy+GIbvcWEgkYMWK9Ii+3IPf8
K8A8eeFfGnj/AOL2t2F3YSah4reZjPb2wXACqANvONoXbj2x3qo6geRtGT/KlW33NwuSeBWlf6bP
p99Pa3ETQ3ELtFJGwwVYHBB9wRXc/APTVvPjD4QjkQOn9oxEhhkHBz/StJU7K4k7nF6JpLNMXcYC
8EMK7zwr4BuvFS6o1o8EaabYyahM00mzMaYyB6nmvrXwn8P9A/4SzRNTbSrZ7681nWHuJJEDeaqm
QKCDxgcY4ryLw3Pbf2H8ZbU2tsYU0+edWMK74m8wqoU9hz0HoK1jNRg7dDzJx9pXSfW/4W/zPmtd
FvvFVxdf2dZTXYtoWuJBChYxxL1c+gHrTfD/AMP9a8Uafq97ptg11baVB9ovJA6r5Mf945Iz36Z6
V9B/su6n4WtfB/jx9T0W4nvLXSZ3vLyO6K+basVHlKuPlbP8VWv2fbzw7Y+EfjDq0mm3DeG1hiUW
LTZlMLM22MvxnsCa4XeTuz1klBWR8nzWoXjbjAqD7OeABX0R8XtG8La98HdC8caJ4ei8MXM+oy6e
9nbTNJFIqrned3O7Ixx2NeC2tx5NwjAB9pDAMMg4PQilyjWjsUltwGwQc+hp5tCOCMV9t3HgHQdf
k0f41XdkItNtdMW6vNOEBRZb6NQqBYtv+qJAyc8jmuS8E/s86P428K23jDXbTxBdT6/czTxW/hq3
jaO0XzCMOGz1OcY6AUuUo+V4bMbSB6elTi1Xb1w1fR11+zL4f8OSeL7jxF4tk0vTdDv4rVZY7fzG
mSSPeABkZfBUYHHBrg/ix8K9M8HeHdC8SeHtZl1rQNWMsUUtzb+RKJIzhgVz933o5QPL/JHTikZS
vpitvwHYLrXjTQ7CRPMjuL6CJlIyCGkUHP4Gvof4u/s3x+IfiF4hj8Pa1oUGpFnmtvC9tlZvLVRw
MDapIG7BI680uXoM+XFDEdcUuCRXYax8LdX8P6H4b1O4MBTX2kW0hRyXBSQId3GBkniuvj/Za8dX
F/q1qkFj/wASu5W1up2vFSJHaMSA7mx8uCBn1IFLlJueOrmjc3p+ld9rnwZ8WeH/ABVaeHp9LafU
LwKbb7GfNinBHVHHykDue3erXjD4BeOPA8NpNq2hSQx3cy28LxSpKGkPRfkY4J7Zo5RnnH3ieMZ5
PpTD64/Ku/1j4K+N/Duj3Wq6n4bv7LTrY7ZbiWL5VIbbz+PGa5e/0DULCysru4sJ7e1vFZraaSMq
kwBwShP3ufSnyEt9jDY7uMVEAVY8c102j+Er7UvFFnoVxbT2V1POkTrLC2+IMR8xXrgA7vpS+NPD
cHhrxZqmj2d6uqQ2dw0CXcabRNg4yB2o5BanKMoIxtzTGjXuoIrVubKW3UGWF09NykVSa1dvmKYX
t2pcnkF2VGt4m/gXP0qGSygPWJD/AMBFXvK7Yp1nZyX91FbxrmSV1RV9ycUco22Y0mkWchy1tCT7
oKqzaDp0mS1pAc9f3Yr0X4lfD25+GnjLUPDt9cQ3V1ZFQ8lvnYSVDY554ziuQe3HORS5LEHPt4V0
eQHdp9uf+2YFU5PAOhTddNt8f7tdR9lX6U0W+5jzT5Bp2ORk+GPh58n+zYh9OKrSfCXw8xLfYtvp
tcj+tdwtvk9al+z/AC4pcgXPPJPg3oLHIhmT6Smq7fBXSGB2z3Sf9tM16asJ9cYHpTWiP40ezEeU
yfBOy3HZqF0v4g4qtJ8Exzs1aX23xg1655ZPbNNMJ4p+z8xnjUnwXu0Y7NWUj/aiqrJ8HtVVvk1G
3P8AvRkV7a8J475qFrcgn5c1XIwPD5vhPrqE7bm0cfUj+lVZPhr4hVThbZ/TEle6vbdDjioJIT2F
R7NiPCW+H/iOPj7JGx9pBVaTwZ4gj66aSO+1wa96aE+lRNCCc4OafsmO54E/hnW4+ulzY9sVA2k6
rHwdNuVPoEr6AaJT1GahMC+ho9nLsO58/Na6jH1s7tQP9g0hub6NfuXS/g1e+m3T05qCSxXuo/Kl
7OQ9DwpdV1CNRhrgD3Bor2t9LiZslFJ+lFP2b7EnaRqGwc/hVqOPv1qGNM4/nXZL8OdVj+HieMSY
f7Ka/wD7P27v3nmbN2cY+7jvmvd5bWOVnORDnNWUXviltLOSc7Yo2kf+6oJNWfsskb7HRkcfwsMG
iyLV7CRqBU6p8uO1OW3aNvmGDXR6D4J1XxFpOrajYWpmtNKiWe8k3AbEY7QcHrz6VNkPU5drcjJA
4pqjax5rprDwnqepaLqWr2tm82nab5Yu5xjEW9tqA/U1jS2h6gcd6XKi0ysgLdalXpil5EYXA474
5pUXoaXLcokj9O/rU6fWoUGc+9TqvSs+VASKxx0qeJ++c1AM8U+MbeMUclwLaN37VJu3YwMVAvy5
zUiNngVDpgWI5D0zVuOY9c1RXluuKmjbnB6VHsxo0UmBGc1YhlPfn3rNViWxVmNtvFZSgPY0lkPr
UczHuc4qJZccUrNnINZ8g9Bu7d9aB79aY3y8UD5mFLlBMWRS2TUXzg9asnOMYpmzJp8tgbBc7R6U
1vvU/BWhhuGK0sSVZGKtnpUM08rY+duOetWJVPIAwaiZRjpzVKAit9sljDKsjKj/AHlDHDfUd61I
/GOuQWsEUWtahHFB/qo1upAI+MfKM8fhWVIgOSOKiyE9xXRTjZ6iZPeeINTuEs4pb2eSKyJNsjSE
iEk7iUB6ZPPHeug8E/FC88L+Pv8AhL7+Fte1ZVdke6nZT5xXasjEctj+73rl5It/I5qGSH3rq9jG
SsY8zWwzWtQuNa1S7v7khrm6laaQqMAsxycD6mtfwK13Y69aahZyvb3FpIJY5ozgow6EGskQ7mA6
k8V6DoOkDTbEbgCWG5jXTTpKW55mMxP1enpuz0zxB+0z4qtNJnw9hBK6FYpobVVeFj95kPYnvXA+
CP2kJfB/h280qbwpomrrebluri6jfzrlWbdtkYN8wz7V514y1gahfNFCf3UXy+mTXO7SwwMknoBX
n1qcU+WC0OzBxn7NTqO7Z23w/wDi0fAniHWLl9IttR0bV4ngvtIZjHG8TNuCKw5XBxj2FN0j4yf8
I74N8ceHLXR4YrTxK8bB1lbNoqOWCKDncMccntXn0y4OQMmq0qk9RXP7PW56FzvNd+KMerfCHQfB
i2RjfTr6e9kut+RLvGFG3sQM/pXFaDcWNvrFhLqML3Ngk6Ncwo2GeMMNyg9iRms9vXPC01lI6Vfs
tBX1ufTdl+07aXvxQmur+3nj8CvYyaSmmoo8xLMoQq8H72QPmznFb/hT4v8Ag7VPAPh7Sr7xVrXg
ubQzNEIdMjeQXUbPvDFlIwe3NfI6zMuBU8UzdMkD61EqPYq+h794q+KGga18L9f0izlvWv8AUPEQ
v41vnMspt1jwGeTGCSe1ZXxM8ZaLq3wb+HOgafei41DTRdyXsIUgws7gqDng5A7V5BDMdwzUkmev
el7PYm513wdurOx+KXha4v7mOzsodSgkmuJiFSNVYElieg4r2r4W+PNHf9pLxx4juNUtrawaLUZb
a4mkCrJknbtJ6kjp618yOCqkjgnrUPnENgcUnT1ZVz6v8AXnhn4v6P4GttWvrjRdQ8M3ggit0hMi
6gjyBxsOMB92N2TgCvRPiZ4P1r4geC/GNj4fcPf3XiqZxa+aEa5SKNEIGSAcHnB9K+WPBX7QXjPw
F4ZOg6Nq32TT8uyr5KMylupDFSQfxrA1D4peIL7QLbR5tRkawt7mS7Rf4jLJ99mf7xz7moVOV7hc
+x9D1LT9J+JHgbw81xb3WsaD4Wntp7BZVLfajGB5GR/GeRgc1RtbvVIbjw7ok3w8g8FWOqeIbTzj
Les8s7RneNkbc7QOrDgdK+J7fWrm2vFuoJ5IrpDuWZHIYH1BHOau3XjjXbrULW9n1e+mvLfmCeS6
dniPXKsT8v4U3TZNz174h/HbxFpnxY8bSw3hvYLh7rTEhuizRRQklMqmcBgBwfevSPA/hex+NPwy
+H93qkqRW/hm7lt79reVUS2s1UyB5A3QsUAz6dq+QGvHuJ5JJnaSSRizOxySTyST610uj/EfXtA8
O6lomm6pNaaZqQAu7aMgCXAxz/8AWpum+gI9k+HXjC/+In7V+n+IYVFo11qOCICSFhVduMnsUXn6
1V8H+LNN0j9oDxNr2raNNqdp9qumFxDb+cdPYyHbcbMYO339a85+Gvxm8TfCp7x/Dt5HaPdhfOLQ
RybtucfeBx1PStCD9oDxfH44l8VPeQvqU0P2edPs6LBNGB9x41AVh9e9TyvXQeh7745/tz4haPos
Eusab478J6nq1pbpftbrDe6azMAVdEAClwSOc8DtUh8Qw+KPGnxE8D3Ok6SvhDQNNvfIhiskWVPJ
AWL96PmyGwc55rwnxR+0f4t8SNp2JLLSILK5W8jh0q1W3R5lI2yOB94jHGemTWh4q/ae1rxDo+o2
droWh6LdakhjvdS0+22XNwhOWVmJPDHr3qVF9QuehfEjxZ4d+CuleB9Ms/A+galqtxotvPe32p2/
mkhhkjZx82cndnPavBPiVr3h1viTf6r4Nt5LTSPNSe1jkTbskABYhT0XfnA9Kj+J3xK1L4o6za6j
qMFvbPa2kVnHHaghQiLgHknk1xjKcDj6VbjZaCNXxN4p1Txrr15rOr3JvNRvJDJNMVC7jjHQAAfh
WU0W8YHSnLGVx2qQLk8VPKIreT2zz9KaI9vvV1lBXmo/LC8+tHKBXZcdqUZbk8dqlZP1pY49wxjF
VygMUetQt39KsONrYqNuoFPlAYMLikdgWyBinbcdetI0fy0coEbR5IORSYG3gc1Mq9cijyy3tT5Q
KzR561C0foKvMvUYqB87cfnS5WBRdQOSPpUTqDxVySP5agdRtHrVcoFR4xUTRcdKtMuetRNnuKvl
AqeWG7YxUUkZXOR9Kt/QVDIAw5FPlAqYop7L6UU+VAdRCApAI+uK+iPDem3ev/sm6vb2VtNdz2/i
SKTy4ULtgwkE4H0r52hPQmu98C/Ejxn4AtLp/DWsX+l2jsrXH2Ynyy3QFuMZr0alNuzXQyi97nr3
wFj1Dwh8P/iVq9lamz8S6fb2rW00sAaSNGkw4CsDwR7V6Eugz/EBvhN4jm0zTdQ8SakbgXkl9Fsi
l8okqzogAJUDOMc4wa+ebP49eO7fxV/wkZ16ebVzF5BnlVWDR/3SuMEfUVNq/wAdPG2veJdP1281
mRr+wP8AoxiQRJH64RQBz39a5pUJuTZSmuh9T6l4Z8J/FLw/ZXU91p+uNa69aWb3Gn6WdOMccj7X
jYfx5457VSj1/T9U0X4xeH7bw5p2jvpVrJBHNp8fliWFJcAOO7DAOfrXh2s/tQeM9b0v7BjTdPt/
Piuh9gs1hIljbcHGO+QM/StfUP2pNc1TQ9W09NA0OzbVoGhvrq2tyss5PWRjnluvtzWCoVFYrmRY
+HFnaal8A/iPC1uUurVrW4FwkrAuDIBtZc7SBgkZHeuw0fwb8O9G8MfDuLVPDdxqmq+KI/LkuFu2
jSImTZ5nHU8jjpgVw3w6+OWkeDPBd74fufBNnqiXybLy4e6eNrhQdyhsZxj2xWbf/GRr4eB410lI
I/DDsyBZSTMplEgXkcYxjvVOnUbegXR6bo/7OvhTQ7XXLzUrPVvFLW+sPpkVjpjBJIUU8Sv1yDXl
n7R/hvQPB/xBGgaBZfZINNtIoZ3LZM8pG4uffDAH3Fe//Bv4vaLqTeL9WgudP0jxFrGoefJa61eG
GAQc7QkgGS+Sc/hXhX7SVl4e/wCE88/w/c29zLPbrLqAs5jNAlyfvCOQ/eGMH65pU+Z1OWRVzI+B
Mvh//hO4rLxHZwXOn6jC9j50/ItWfhZQMcsD0+teo+KPgTZeAPC8fhy7hgvPF3iLVkg02ZnANvbq
2BIcE8PnBHavJvgr4p0LwP44h1zXoZriKyieW1jiQMDcY/d7hn7ua9P0r9oDTtd0WK58VmWbXNL1
yLUtP8kMxeNnzLGWb7qgD5RRVjNO8dixfFv7LcGjaLqc+la++o6lpYVr21nsngQLkBijnhsH06ij
UP2T3sdSGkHxjo7a/NHvtdMJYSzfLkDpxnnr9a674jfFbRdU03WLu2+J9/f21+Qbfw/Da7PK3MDs
dmH3AM5xWJc/Efw5eftU6b4ih1WE6IvlB75srGMQ7TnI9eKzg6jEfPF9YzaXqFxZ3KbJ4JGikT0Z
Tgj8634/h/qzeCE8ViJW0hrs2W8NlxIBnlew96r+Orq3vvGuu3FtIs1vNfTSRSL0ZTISCPYivZPg
X4q0n4Z+BdR8Q63crq9neXa2aeHQyNllwxndWzgAdCB1rqneMU0iThIfgd4xk1bSdOGks11qdt9s
twHXHk/32OcKPr61JrHwP8ZaLrlppNxosr3t0jSQJCRIJFUZbBBxwByK+kfDcjQ/EzW7ptZt/E6+
IdCabSrO6mVSwLBhbMoOFwOw7A1r+G77XLPxV4V0vU/Dmm+GI41vZreztbvzpc+UcsV7KT3z1FcT
qyuWtD5H8SfDfxN4Ntba51rR7rTYLjiKSaPAY4zj2Psa59GPevoTQdav/E3wD+Jc+q3c2oTLexFX
uHL7DkHK56fhXzyMjgV0xjzp3Ie5YVhinrJuOfwq9p/hLWNQ0G81q206ebSrNgtxdqmY4yccE/iP
zqHSNGvtcuhbafaTXtwQW8q3jMjYHU4ApciERKoI60qqN2e1TQ2U00wgjiZ5ScCNVJYn0x60s0Ml
tIY5UaN1OGVhgg+4qOUBqrupdtKpG2nrx15qeUCJlz1qLkHirLgNkgYqu3U04wGRsuT/ADqB1BzU
/K5pu3JxXTGOginIv/66ryJmtCSMdDzUDx5ye1bKIFONtpwelSyQdCOaa8W1s1reHdO/ta8WHOAO
WPtXVTjfQ5K8lSi5vYueF9A+0TrPKvyLyPrWv401NdF0tgh/fS/Kq5/Wupjs4dNs2dhtijXJP0rx
zxRqp1jVJZR/qVOEGe1dc0qcPM+WwnNmWK9o/hiYLZaRmbktyaDHnnoae646daauSp9a8nldz7Qq
zKMnj61Tn+VfWtGTkHgf41QuOeMVapgUmXniopARxVkKd3tUcmCeeTVqmBAM5NKjfN9KVVLZz0o+
7n0o9mBZ8wDO3+dWo5NyjdyKzl+bpUkMmzOan2SA0JBx/OqUq7GFSrP1yailYSHPQip9kA6OJsZz
xTdpOQauWjI0RB61BcMd3HSocAImjKruH4VF95DnrUm7HeoZOM46U/ZgNyd2M1Mudo4waSOP05p7
H/ZpezAkjxxmn7u3aoVfOD2z0qb7w6VHswI5BwM0wVLtJHSmrjnPNS6YDVBel2hmFS4wePSl8vvU
+zAg2Gnqu3r1qUR7j0pzJtyR1qeQCHZnjp7Ux12nFSevGKTbnmh0wGFc4x0xTlXGO9KB+VOxjrS5
bgQSL83PSo9uOcZqxIA2ewqPjnjFPkAj9gKGU+nFSbfUe9Ob7vFCiBWYHtzSbhUrfKeKjY8ZFLl6
ANZhz9KrsvHWpNx9OKY3JpqmBA2e9RFQasmHcvPAqFl/EVXKBFIoC9KglX2qeTdxgYHrUbLkZquX
UCnIMdOlQtH6VcZdo96gkwc+lPlQFUe5oqRlBPWilygb8ajt0r3/AOFEKXX7OPxXh25eN7Gccekh
H9a+f4Ador1n4PfF61+HemeINL1Pw/H4h0jW4o4rm2knaLhGLAgj3r0KsG4px3VjCPU0/gL4B0Xx
hqXiC819Li403Q9Ll1J7O2fy3uNmBt39uufwrsfEHwp8H+IPC/hzxf4fN/4f0bUNSOm3Npdf6XLG
R/y0Xbyf92sXQPjd4U8NeJpLnRPBT6Vot9YS6fqdiNQeV7iN8ZKOw+QjHvW7J+0Xomg2fhvTPCHh
uez0nSdSGqFNTufNd5P7oIAwuPxrklGq5XSZfuo6rxp+y9pSeFrnUtBt9e0ie0khVv7ejQR3Cu4T
KbeQQSGwe1dN4K+EXhDwfdeOtKtdVfVtdsdEngvba8tVVQ20NvhOTkAjGevIrmrz9ojwPa6H4mTS
7bxDc6nrTpcFdQlVoYZBKH2IAeFznnGelalv8ZfhlHr3ibxX9r1ca5r+nSW8ll9mBht3dACAwPzc
gc1zWr2tJMd0ed/Cfwzaan8O/iRdbbOe/t9PUpDcQlnjQOCZEborZAFWvCv7O7a54R0nxFf+KdK0
O11N2it1vN25pA20KMdScHp0rX+BWpeDNJ8I+Ko9b8Wx6Tf63ZvYfZpLSR/KXIIk3Dg/SoPEPjXQ
I/AvgDR7HVY7250fU7lpmWJlAhMoKScjjIGcda1lKopNR7/oNWsjL0f9nDxJf3msxX1xp2iWulXH
2Sa81G4CQmTAIVT3yCD+NYXxU+Fs3wtvtM0+61C3vby6tvtE0duc+QSxABPfIwQeOtfV3hnVNF8Z
ah8QL+8ng1vwTLfwtDFDC8zG4CJl9qDcV6D8DXgP7Tek3q+OrfWbi4jubLVbVXsmjgaEpEnybGRv
mBHv1pUqs51OWRMlZaHDfCr4Xr8TvFE2lSXjWIjs5rkSqgY5RchcZ7mvO72T+z5ZI2BJVyv5GvoL
9lJk/wCFqeUzqnm6fcxrvYAEleBUbfs46no99rlz4p0oJY/2bez2ki3CsPPSMuh+Vj6E4NPEVHT5
knrpY0g72PEp9Q0T/hHLQwPeHXTO/wBoDqv2cQ4Gzaeu7Oc9qzFvo9xBzj6V6H4f+EY8XfDfw9d6
TCW1/UNamsWZ5CEESRB+R2x8xrK8WfCZvDulyalYeItI8Q28LbbhdPlbfDnoWVwD+VeIsZiI9TZx
Ry630R7496lW+j4+bmunTwZbxfBOTxI1kst0+rpbLex3IJhTy2JjeLOQWOCD6CrEfwD8ayaUmoDS
V+zy2yXcQNxGHljZdwKKWyxx2AzWyzCt2QuVHNWuqeVIrxTNG6nhlYgj8a0V8QXTTLP9vuDOilRL
5zbgD1AOelQeGfhp4o8XQXs+j6LdX0NpkTOicJ1456ng8CoPFPhXU/BOsHStUjWC8WKOYqjhgFdQ
y5I74PTtTeOnu4oVjQg1m6htZbaO8mjt5iDJCspCSY6bhnB/GoVfLDBzWVp2nXeqT+VZ2013P18u
FC7Y9cCp59NvbNSZ7WeDaxjPmxsuGHVee/tVxzB/yC5TttL8f61o/hm/8PWuovFo9+d1xa4BVzxz
kjI6D8qteAPiHq/w51ltS0SeOG6kiaFjLGHUqSCRg+4H5V5/tmhwHR07jcCK2vCPh/UfGXiKy0bT
drXt4+yISNtXOCeT24Bp/XYapwDlOs8K+Nr3wn4tg8Q2awy30MrSgTJlCWznI/E1B4q8RXHjDX7/
AFi7WNLm8lM0ixDCgn0FckWmhkeNmIZWKn6g1d0+C+1BzFaW813Ljd5cMZdseuAKPrlO9+UXKWF4
Y5pfMJJHas+SaeKRlcbWU4KsMEU37a/f8ar65Se6DlNPeNvWomOTVH7aw7CnrfDB3DB7VpHGUerD
lZO/ao87W61XbUOvycfWoW1AE/cNbxxlD+YVmaDKGHeomXjkVXTUkX7ytilbU4Qufm+mK6Y4mg/t
Eyixxj8whcc/StPTlfTZFliJDjqaq2up2KrmSQq59VNNutaswpMc+T24NdUK1Ja8yOKrCVT3LaG9
4s8aG70dbSL5Jn4kI9K88dSvNTTXqSMSWGM1D50Z/jB/GqnWjUd+ZGmGwsMLDkpoayZwSM4phX0q
VnXjDDH1qNhnOORjrSikzqK0mNxH9KqTKvNXHXdnB7Zqs2eQRxWyiBSMXOAD+NR/ZWLc/lWgUDNS
+Xt781pyoCr9h+UHOPUVFJb9K0GB21XkjOCT17UcgFELt470xmIbpU8mRzUe3dzmq9mA0tuGe9Cn
1HNOaPauQaPTjNL2YFi24696lkh3Kf4aghl2sBtAPWrbFmQt0rJ09QKEibenNQtluO1Tyyduh9Kb
tBHXj/ZNP2YCKdvA605z+FA9AeaXaT2zU+zAYuQ1Tx9s9+tQhTu9qswx8c9OgqJU2A9lOOKjWIqx
PQVaXjjFDoMZqPZgRKgKg55p20UKewFSr06VPIBCU9KNo7VI+FPpTOT0NTyARnr60hA246UbucdT
QWHIxnPGaXIwI8fMDTmIHv60n3VJ96ikkwORU+zAk9e9G32qNG/Kn7zz6UnEBfemjBzRuOOe1L/D
7VHIAnl8H0qAr7Y+lS7vypgbPPajlYFZ46jyRwMVZl+lVivtinygN+pqFs9qsFfyphj4yKfKBXf7
nzfhUTZWrJXjkVFJGGx2osBUkUsPU1XdMdjV5htzVaYjNHKBW9aKMH8PrRS5QNy3OQv05r2P4KeB
9I8YeHviFNqFsbi40zQpL2zZXK+XIrL83HXjPWvIIoWWONuoYZr2z9m/xdoHhvUfFFj4k1E6Tp2s
6NNpwu/KaQI7kYJVeccGvQrX9lzR8jmg02cn8Nfh3qHxJ8UQaLp7xQyOrSyTzNiOKNV3O7ewHpXa
eKvgHqWg6Zb6no2r6b4t0yS4W0e50iQsIpmI2owYDk5rrPhmfA3wg+IWl6hb+ObXxDY3kFxZ3Bgt
JYjbeZGVV33DBXJ5xz7V0EWpeDfh18KrrwxdeKtO8SNqWsQXm3SC7bIVK79+QCpwOK46lapz+4tP
Q15VY4DxF+zf4o8M6JPqUk2mX/2VQ93Z2F2stxbL3LoOQFPB9K7P4a/ss6xql0lz4os5LTSpbCW4
QQXCCeOQJuj8xOSoPuPSvZZPEWjW+h+L5INU8I22gXGlSLpUViUS9ddoIWU5zn2PJJ9qk8P6bJqn
xcuPHFnrdgvhfUNFWEf6YqtK4gC7TGTnhgTyO1cf1qq1Z6Fcseh8xfDj4ax+LLbxbc3P2wW+jadJ
cJJax7v3w+4H/wBk4bP0rK8P/DXxN4otY7nStCvdQgcsFkghLKSuNwB9sj869k/Zp8J6zqVx40vb
S1eXTrjTLuxWRWG152HypjPX3q1dapr/AIC+Amh2EV1daNfR+IZrW7ijbZIMLu2kjnrXRKtKM3GO
5KiuVXPIvBN54z8Na1dQeGm1Sz1NQUnhskYyAA8hlA6A+oo8fDxdf3Vpq3iz7dJPeIwgnvgQXVDt
IA7YPGMV9j2FnLq3jjx1omnSTaJfXVvZ3f8AbdrGNyfIg2Z7k8nr618+/tWahqVx47sbG8jvEtLG
yjihkulAE7fxyrgn7xxnvkUUayq1kuVImfux1PE49yuNrEH1FeraFBN8NrfSNa8S36/2L4n067so
po5TK9qHTZvdOvGc4HWvK4c7hgcZr2z9pSKTXPgx8MtT+yRnUJo5RK1tAFJVVHUKOgApZg3GlZdR
UHd3OOuvEfhPw34V8OeFdM8XzX/k6y+py6xY2jx/ZwUCBdr4JORk44xn6Va+Kd14Z1LwZcXOra/o
Wu+K9ym1v9AR1luPX7QpVVAAxjaB05r5/aN2BYK2zON2OPpSqCGFfIOb2sd3ofQGg+E1uv2bdSsk
17Qvt1zqUWqpZtqEayrEkTqyspI+fOML3zW7P4htLr43/CNl1GGS0t9K05XYTApC4U7gTnCnpmvm
VfvZ6Uu7DGiM7CPsH4d29l4h8P6r/bt0mlaVpXiSS80m6F2lut7OWJeNnf5SqhVPHrXz18YG1ST4
ka9caxaCxvbq5a4MMbl02udylG7qQQQe4qLwT8Xde8D6fNptsljqGmyP5gs9UtEuYkfuyK33Se5H
WsjxX4u1Hxtr1xquqzCa8mAX5V2qigYVFHZQMADsKtzUo2Eev/spXM1nr/i24t5XhuI9AuXSSM4Z
SACCD2INe8eGY18WNpFjrSrqdlLoukag8N0A6tcyXO15Tn+JhwT3FfEGk61faLJK9hezWbTIYpGh
kKFkPVTjqD6VuQ/EHxHDC0Sa3eLG0EdsVEpx5UbbkT/dU8gU4z5VYZ7s2vaj8RND+INr4iK6jBpF
zB/Z4aBEMGbnyztKqD9zjmuw0jxINB/aHtvBVpo+nDRLOXFsIrRFmgIhLbzKBubv1POea+cfEXxm
8YeK9LXT9T1ua5s1YOUCIhYjpuKqC3rzW7pf7RXjPTI7cxXVk1xCNn2uSyja4dcY2vJjcwx6ntT5
7iOt8Q+GdN+3fCmNbGEDUwTe7Rg3Gbsqd3r8vFXvh/bp4d/alOn6WWsbNNXmt1hhYqvlBm+T3GB0
NcR4Z+OmueHbSwiFnpl/Np7lrK7vrUSTWwLbikbZ4XOePc1Z8H/GZ/DvjS/8U3GgadqmrXVy10sk
7SIbd2JJ8vDdDnuDS5lcDqfBXgTTfG/xC8cz6jAt5FprTXKWs10LWOZjOVAeU/dHOfrir2vfCPwq
3i7wxE8sOhwakk4n0221JLz95Hjy1WYcAyZA54FcfB8XrGHxFq10vhe1j0fWLfyNQ0yK5kxMd+/e
JDllO7HTjiqF1458Kvrlg8Pg2GHRII3jmtTcs88u/q/mn+Jf4eMD3o5kBo/Erwr4U0KzuIrXT9e8
O6zbsALTVClxFcg9dkqAAY69815fs9/wr1fxh8WNC1LwC/hnSbHV5oHaMxtrl4tx9jCHgQYUbc9D
7V5Mv3hzxUyt0Kjue2P8A9CXVNM0I+LiviLVbWGe0s/sRZAzpuCyOD8vPQ88c1g6f8C1Hh9tX17x
NYeHrWPUJtNk8+N5T5sePuheWBOeR0xXqnijxJ4G8N/ErQ/El5qt5Hq2kWFqTZW9uJorrEA24kB+
TqAc56V5V8QfiHpfij4c2NlbO6aidcvb+a3ZT+7jkxs+boT9K2tFIkrj4BatFres2d/qum6Tp+lm
IPqt5KUgbzRuiA4Jyw56cd6oXfwN8RL4kt9ItGtNRFzaPfW17bzD7PcQoCWZWPpgj6it34d6pYeJ
/h14m8Lahr9rpWp3t3a3UN1q0rCFkiDArvwSDyMDpXReIviNpXhHxZ8OLax1RdUh8P2qWmpS6c+Y
Z1L5dFY4DKVJB7dqSsM8Rj8J6lc+GrvX44QdMtbiO1ll3DIkcEqMdTwp5rY8WeCYfD/w38MatJY3
MWoatLO/2mR1aCSFduwIAeGGTnPtXovx1l0rwro+n+D9AvLa6jubybVLuSxCeTIJHP2dCQT80akj
0ANW/id8O9e0n4H+BkubJfK0qW6mv2jnjdYBLInlk7WOcj0zTSsxpnmvgH4F+JPF2qaHLd6RqFp4
fvruO3k1IQ8IrMF3DP171z998M9buta1q20PSNR1az0+8ktjcW9s0nCsQN20YyQM19c/2L4m1f8A
aE8Ja/oTXDeEBZWyPfQy4tJVEZDKDnBJOBjrWVD4gj0nwd4cktNH8Taq815fySSeGbxoQjm5b5Zg
qkEkY+9jirUUSfGsOl3l1M0MFvNNMv3o40LMuOuQB61v+JvAN54Y8IeG9dubhSutfaNlqVKvD5Th
Tuz619Ia94k1TSvAfjvxR4fspvDXiG48TLbuluFa4gjEY3RsVBGM8nHGa88/aU1PVdS8FfDGXXYT
Dq0+n3E86tH5bMWl4dl/vMACT3zmtNhHi/hfw1e+NPEmm6Hp7oL2/nW3hMjEIGJ4yewrJ1axudF1
K6sZ2/fW0rQvtbIypwcH04r0/wDZpiWb45+EAV37bsyFf91GP9K9F0/xRZ/FKL4k6RqfhbRtOsNI
0u7v7ZrS0EdzFMjgJul+83J5z1zVxlLuU7HzAtxMuPnbNS/a5MZL896+mY/hTo2p/tEeE9Mj0KKf
w5Jo9tdXKwofImxbEvIWHq/U5618zalsjvrgINqCRgq+gycCtPa1I7SYtC1a2erahp97f2trNcWV
iFN1PGmUhDHapY9snisubVJ8kYX24r6A+B8mhab8EviRfeIra4vdKa50+BrW1cI87B2YJv8A4QSB
k9quXvwM8H6tq9hrsKahpnheXw1J4juNKWcSTqscmzyklI/i65I46VpHEVraTYmkfNT6jK6kYXd3
GKg/tKUDGxa9/wBW+CHhXxt4N0vxR4Em1TS7e41mHRJLTWmSRjJIQA6Mg+6M8g807xF8A/BksPjD
S/D/AIg1afxR4VtpLm+F9bIlpKImCyCNlJbkn5c/jWixmI/mEkjwBdScYDJx61PDqijGUJPsa981
T4YfDK2/Zps/EK69cJrFzfuIr6TTn8ySZIQWtAA+Am453159+zX4W0jxt8YNE0TXdO/tPTL4yRyQ
+a0ezEbNvBXnI29Kr6/iEr8xXKtjiTqUY52tmrMeqoy8ggV1nhH4Pz+PtS8RXFtqVh4f8O6RNtn1
TVpikEO5ysSEgEknHpU+t/ATxFpPiDR9K06ay8RxaxEZtP1HTJw1tOqnDkO2MBSOc1X1+v3J5Uef
XOqQ5IyQfemW+qQtxu4+ldx48/Z48TeEtDk1gT6VrllE6xztol+l2YCxwpcLyoJ4HvxXWfDH9k3X
bjWILnxlpz2egmxubmZYLqPz4mSFnQSqCWjyQOGA9K0/tKt2Q+VHksd5D5gGePerBuIdufMUfjXQ
+FPgX478baQNS0XwrqF9p+8qlyiYVsemcZ+orFs/hx4r1e4uILHw9qV5Lbz/AGWVYLZ3KS4J2HA+
9gE49qf9pVOsULlKqzR7uHUn61bhwy7tw259qyNY0PUPD+oS2Op2Vxp97CdslvcRlHU+hB5FQxth
R2HTNP8AtJ9YhynTxyI0ZA6/3s0My7SDzTpfCL2vgOPxJJqFqPNvzYrp+/8A0jhNxkx/c7Z9a5uS
4fZ94ihZl1cQ5TcjcU4N6HmubW8kyAJD+da2i6TrfiS4eHSbC+1WaNd7x2cDysq9MkKDgUf2nTen
Kw5WXGbr3qPecYNRatofiHQVU6jpeoWIdtq/arV48n0G4DmqFxJf2u0XEMkJPTzYyp/Wr/tGkt0x
cpptg1Gx6jrWadQuFTcRgfTFTw3TSx7jx9K6qOIp4iXLDcTVidmODzULSdjwKbJONvTn9KgebC5r
v9mSWt4/yaVZM96oLN8w71OJRuwDU+zAtCT2p1VY23d6mV/lHesnTGP/ABpm7axNBbuDSL81LlJu
KfnXjpTPLz2zUir+VBX5QBUcpRXaPDH0o+72xUxB/CmN81KwEDLVaT3q24AyaqTVXKBCxyp5qpJn
pVmQHHFQSd8UcoiILiiloqbDOp02MTWMJYZBXvU0lo0fKj5ai0GTfptsc/wV1/hzwtqnii4a20nT
7jUZ1Xe0dtEXYL6kDtXp03FU1JnhKpNSsjmI8rjmrULFfaum1r4b69olu9xfaJf2MSfekmtnVRz6
kVznkPGw3Lj2rZcstUdkZ3J4m+YE9avRyMOrHn3qlGORVqFc5JHFHKirmlY6xfafkWt3PbA9RDKy
fyNTtqV3cIEluZpUDGQK8hI3Hq2CevvWbjdzU0Waj2cb3sTzs9S+H3xw1Xwcmo2t/bJ4i07Ugv2m
1v5GyzKAFO/7wwBjHSq/xM+KsvxGOl28el2+jaXpkbpbWcDtJgucsSzcnJxx2rzzOamjYiuf6tBT
9olqKVVtWZd0+8k02+t7qEKZoZFkTeoYZByMg8H8a9b0/wDaG8X+JfEPh3TtRgt9Vs2u1gNlaWka
SzJIDG6IQOCVYjivHV61o6L42n+HmqW+uWtpBd3UAdYluAcIzIQJBjkMpOQfUVzY6lF0ZTa1SLoz
tNRueseKfhtpdvrXgz4YWMgOk6tqk17c6xCodZJASoijbPBjX5WHqQao/ED4A6FYeDdZ1WysH8N3
OlgMnm6zBfJegttKqq4ZD35rzDw58eNV0HTdIha3j1C/0vVTqdrfXTFnj3A+ZEB/dcncfepPE3xD
8E6lpd4ujeCZdP1W7bdJd3motOIyTkmNcDBPvmvieaL3PUPQ9a+AngmPxRceDtO1fV28UNp/2+GS
WOP7KmIPNKOeGJIBwVHcehr502lWIJxivZofj1p0vxkHjObTLmO1/s02JtldTJu+zGHdnpjPP0rx
l5Nzkjuc1hO0tgPUfhP8J4fEqW+veJJzp/hQXUdqGU/vb2ZmAEEQ9eeT0ArC+Lnh6y8KfE7xNpGn
RGGwsr+SGGMsWKoDwMnrW18MfjprngW30/Rd9jP4fS9W5khvLKOcxkkB3UkZBwO1X/F2s+Gvix8e
tRvdQ1WPRPDV9dt/xMobfaREFO12X+8cDJPrmqsuUOp51oel/wBs6ta2Rubey85wn2i6k8uKPPdm
7CvYda+B+lWHxO8E+GLPWZbyx162t5pL9UA5dmBaMf3fl4zXmEmg2Nz40l0nTtXtm083TQ2+qXZM
UTJnCyNx8oI9q9x+Kmmiy1r4eanpfjPRIW06xtNKe/srwTNazpuJlKKCdgz1x+FOPUDgfHfhfwDo
drfRaLruuT6tbTGJbW+09EjYhsN84bjp6V53H1r6R8eHxB/wq3xN/wALD1nStXui0H9hzW9xBI5k
3/vCvljIynr2NeQf8Kp1mP4bjxx5lmdH84QFRODMGLbclPr+PenJX2A2vDPwZ1bxFoMN/wDa7Wwu
bwMdM0+6YrNqO0Zbyu2PQtgMeBk1ws0MtrO8MsbxSoxV0cFWUjqCOxr6L8P6XN4uk+EGt6VsutH0
C3ih1W6DhVsmilMj+Zn7o28g9+1eHfEDUoNU8c+IL21kWe2uNRuJYpF6OjSMVYfUEGk46aAZdlay
XlxHbxjfLI4RFHdicAVu+OfBs/gTxBLo11cwXN5AiG4FuSRFIVyYySOSM4OOK3P2f5oE+MHhZrl4
44ReLuMpAXocZz74rrvhjp5l+Jnja61W0FzqGlaff3sK36byk6H5WZW64z0PFEYpoDxNsr9Kms9N
u76O5ktraa4S3jM0zRoWEaA4LNjoMkcmu38VfEi98e6C1vqWgacbyOZTDqljai3eP1jIQbW3cdeR
jiui8RKnwi+HtlokQEXiPxFbmXV2K/vIbQ42W+D90nGTxzxzTUdR3PH8knDHA/vVCzbeM19JXXh/
wdeeFxdaH4Pt/Enh1bYfadQ066kGq2bbAS0qMdud2egIwK+arhlWeQRljHuOzdjOM8Z98U2rCJ4L
S5uIZ5ooJJYoFDSuiErGCcAse3PHNQNIeua9N/Zz1KUfEq00Vgsthr0b6bdxOTtMbqfmx0LAjIyD
g1594gsotM1zULOEl4red4lZupCsRn68UW0uMoSTM3U5/GkkuJfL2F2Kf3cnFei/s9+G9O8V/FrQ
9O1W2W9s5XcvbyDKSYjY4P4gV3HxD8MvZXOlaZqvwv03wv8A2lfxwQalaXTsWXeNwVQ5XlT39a0i
tBHg0Gu6lZRrFb391BErblSOZlUN6gA8Gp9H8f8AiXwzHNDpWuX+mwytukjtbhkVm9Tg9a+kPi98
OfCvgG71dE+FOrXOk2cQK61BfyrDkqPmOVIADHHWvN/hD4O8BeJvBvii+8RWWsG80O2N9LNZXSKs
kZYKqKpXhsnqTV8obnmnhf4m+J/A95cXOg65eaZNcf64wycSc9WB4PU9aoeMPiBr/jia2k1zU59T
ktlZIWuGyVVmLEfmTWVqjW39oXJsxItmZGMKzEFwmflDEcE464p3hvT7XWvEWnWF7cvZ2tzcJDJc
Rx+Y0YYgbguRnr0qkO1yx4T8ban4D8SWmuaRMtvqVqWMUjIGAJUqTg8dCa7TxF+0h4p8S6JfafLb
6TZzagnlX2oWNikV1dxnkrJIOoJAJ9cVhePvhZe+F/ilqfgvSxPrt3a3P2eEwxHfMSoPCjPPP6Vm
+J/hh4r8D28Uuv6BqGkxTMUjkuoGRWPXAJ71VmLc92+Fnxy0H4d+BbXf4w1TUbi2tpSfDraaEXzX
Rl2LcZJ2Atux0z2r5hubg3Fw7kYLMTxWzfeDdd0vTlvrzSb61spAClxNbuqMD0IYjHNS+D/BOoeL
vEOl6XbxtEdRuEt47iVSIgzHGSaetrCO4+GPxc0Xwj4H1rwrr/hv+39J1a6iuJzHcmGVAikDYcHD
ZIOfTIroYP2jtHudduLK50C4tfBsuh/8I+ltb3Ia8ht92/cJGG0sW65GMV5D448I3Pgnxdq+gTyx
3U2m3D2zzQ52OVOMjPaubClWJPIFXaxNz3y6+P8A4X8H6J4b8P8Ag/RtSl0TTNbi1ydtYmTzpJEP
+rGwY2+5Gam8S/GH4eaLp/j7U/DV1rmo6/4wgktpbbULWOGK0Esgkdg6sS2MYAxXz/bxrNcgMA4w
etST2MB/5Z8e1ezhMrrYyn7SDSQua2h6RL418L+Iv2ebHwvd6rLpGuaFf3F9BD9maVNQMuAAGH+r
2gEc5rc/ZJuvCvh3xovifxH4rsdCewSaOCzuopC0rSQsocMqkAKW7814wul239w+3NSDTYBx8wH1
reWQ4rumPnWp754Lj8PXHgX4h+AI/G2iQX1/qFrdQateO8NlcxR5LBXK7t2WHBHY11ul+IPhzJ4x
+HvhPVta0/VoPDeiSW63MchNhLqDuXQGTgFOeSeOxr5TfTYtpAZjxVX+zFVjiRhjtWUskxvZfeTz
H23qHiK10H4Z6lF4mvfBOn3tzrWmrJF4ZWIEW6zBy8hj+/wpOAOMe9Vf+EZ1nw546+Mnji91Cztd
F1TSb97GNNQjdrxXwEcIrHgAj72DlhxXxRcaaWAIlOfpSNY3DRgGdsYwRk1n/ZONj9j8iuZM+8tc
+16hpXgVtB+G0njzTbfR7WFNastSmt4lcD5lJQ7cKc5JHrXC/E/4ma9p3wb8Sa9bTroep6/4xeGf
+y7kOoSGAAqko+8MgfMOtfKlrqOr6fZm2tdSnto+crFM6hgRjBAOMf41RkfUp7OO0e6ke1jcukDS
Exqx4JC9AT61i8sxkf8Al2wudX4/+JmofEi40271NIEuLGyjsVkiDb5lTPzyEklnOeTXLxzFjjPW
qgsp167evHPFWobe8srqOeMAyROHQrgjIOQea55YHFR3pv7i1JH0F4682f8AZw+FdrdLuuZ7+8aP
5AHEQZVVSQM4ySfxr1LXvFy2H7Qmh/DfSvD+gxaVILOzv/M02OWSfMSlzuYfKcEjj69a+d/E37Rv
xH8XaOulatqq3lijpII2s4AQVYMMMEyOQOARnvWBb/F/xfbfExfHss6XHiRZvPE80ClC23aPkGBj
HasXha0VrB/cxK1yr8VrXTtE+Jniew0mHyNMtdRnhgiyTtRXIAyfpXtn7Mc9npHwu+J2s3+vXXhm
3ZLKyGqWUTSTQs0hb5VUg84x1r5k1jVL7WtWu9QvMvc3UzzSsFx8zHJP5muk034palo3w11nwTDa
25stUvYbye6YN5oMYIVRzjbznpmsJUpKV3Fr5FL4bHuvgXxFN4x+P3gTQ7f4gav450RNSiuj/aUc
sSxsmW+47HPA6+9XNQ8aX/xg8HfF278VXj6rZ+H3FzpEsp/eWsjzlFRG7IV6r3wOa8E+DnxQHwj+
IGn+KG0xdX+xiQC1aYxbt6Fc7gDggHPStfxx8b7HUfDd74d8GeHB4T0TUpFn1NZLs3U95IpyoaQg
YRTyFA6k1HXXyJ2PafB/xo8TWP7O3iTX7uTT7u5sNRs9N097jS7Z/KTYS4x5fzcBRk5PFeAeIPFV
14s1OfVL1bdbqfBb7LAkMf4IgCj8BUjfFO1/4UfH4FispkvG1g6nPeFxsZRGEVAvXPU1zELbYVGO
QK+gyWnzSbfRfqTPcutKT3prPuXHaqpkJ709XwOK+t9kjAkVju4H41Ir/NnpUCsd1PV9xqHSAtK+
Vx/WplcFdtUgw+n0qxDJjvWMqVkMsL+hp3O7A6UyNjUijnmud07CJR2pCR3pwXJprLWPIPUay+9M
bjNSH8xUbNtFLkEVbiQrmqzPVqZe9VmjDGnyjuyGQZ5qA+1WWXHTpULLT5REODk4OKKkEYPvRWfK
F2TeGdaNvZRRycoBwfSvX/hb8Wtb+GuoTah4cvFtbiePypN0auGXIOCD7ivC9HZfsaYrYs7qSFgY
2K16FCjGrh4qSumjGph9eaDsz6g8TftSeN/Gvhm80LU57OSyvE8ubbaqrFcg8Ht0ry3yxNwy5rmN
K8QK2EnGD/erp7OVJlDKwPpzV06EKKtFWPGre1jK89yFtLbrFz7GolVlYqQVxWxFntVhraO4A3IC
emcVtYUMU4/EY0bDt2qZWPANW5NHKsNjce9V3t5IGw6Ee9HKdMa0Z6pj1bDfpUoOahTrn9alX3qO
UmUiaNvWr+neCNR+IWpQaLphhW7mDyKZ32rhEZ25+gNZ6+3SvT/2eFST4vaFEzbUkW4jLf3QYHBP
4VxY2P8As8/Q1oP94jxCDwNqs3g278ULHGdKtbxLCRt43iVlLAbfTCnmsWa3mtiDLFJHkcblIr7M
On+FfEXw+t4PBC2FlbyeL7GzkjmBnh81A6id1Y8iQEHb7Vd1h9P8cfDv4j297ceINbOjWrlDrVnF
HbwToxAaB0AOeDgH+Gvzx0z3bnxDnC5qaNSw4PJr6g+LXxO07wL4u0/w5e+G9NuvDFxoVtJe28Fn
GtxPJJbgK4lIyhBC/dI6etW4rjQNNuvhL4Zi8I6JcReJNNtjf311ah5mV3ZfkPG1hyd3U5HpU8nQ
o+ZNC0HUPE2sW2l6ZayXmo3L7IbeMfM7dcD8qgkhktZ5IpVKSxsUdW6gg4Ir274X+H9N8P8A7Uke
gC1N3YjUJrOLzZXWSIYO11dSDuGOtHgTwl4RHhj4h+IfEul3Orf2Jfwx28MVy0RbfI67WYdiQMnk
8UlAR4mrN3p+/HJGK+kLD4DeFfE/ibQbyA3Ok6XqmhPrC6KkwklZ0YqYI5TyS2M5wSPSue+LfgPw
t4S+G+jXen6Bqel61rF28iRapNme1iiG1kYYGQ5IYHHbrV8jsB4n5pOcHFPW6k2+WHbYeq7vlP4V
1Hj7RfCmkWOhSeGtdm1ie4tQ+oxTQ7Ps02BlAcDIzn16Vx0Lcn1qLBubNrql5aWk1tDdTQ284Amh
SQqkgHTcAcH8aRZN2K968E/BTw/ceFtA07Wra6fxN4otpLqzvIXYx2a4/wBHGANpMjAg7j8vtXL/
AAL+HOj+MPiTdeFfE9vdrIscyK1rOEMMsed2eDuHBFXyt6i62PNLO8ms7qO4hcxzRuHR16qwOQR+
NdlN8XvEEnjQ+KoZYbLWHRUmkgiGyfC7WMiHIbd3zwfSqHhXw/p2qfEDT9F1GS5XT7i+Fo8ltt80
AvtBGeOuKq/Ejw1F4L8da5odvO9xBp929uksgAZgpwCcd6OVhc6fWPj94s1h7AzT2cEVncpdrb2l
nHDHJKhyrOqgbse9cR4h8T3nibXrzV75/MvbuZp5GHTcTngentWWu6SRUTlmOAB613/jr4J678P9
H8OXWpmIXWss6LZKcyQsNuFc9MncOO1GrGdJpH7R02k7dQtfCWjReJlh8r+2kV1cnbtLmMEITj2x
Xj91cPdXUs7ndJI5diOOScmvW7r9mnXYUurW11nRdQ8QWsXmz6Bb3JN4mACy7SMEgEdDWP4X/Z/8
V+KtFt9TtxYWsd0SLSG9vEhluADjKKx5GePrV8rsLQi+EPjHRfAF9qevX6Sz65a25XSIFjygnYY8
1mzxsHOCDnNcHfXkl/dTTytulkcyO3qxOSfzNN1DT7rSNQuLK9he2ureQxywyrtZGBwQR2Na/h/w
Pq3ijRdb1WwiR7PRYBcXbM4UqpOOB3NFug7nS/A3xppfgf4j6XqusPJDYQrKsksMZkZd0ZUHb6An
tW5fW/gTQNa0bWdN8eXGt+RqUMslnPp0kTJHvBZgSSOAOnevHGJBqCRjVxDc9W+PHxTn8XfEDxDL
pGuXs/hy7kXyofOkWJ1Cr/yzJxjI7ius+Cfhuwn+FvjVJfFXh+wvvEFkLO3s72+EMsZWTJLgjgED
jGa+djI3cE00sSGJwMDP1qwtoRapD9lu5od6v5bsm6M5U4PUHuK6b4M+C9U8dfEbR7LSbYXcsNxH
czKXVdsSOpc8kZwOwrlGRpmwqMzdcAZqg1xLbzB4ZGikHRlJBplI+0rfw5q2i/tO/Eea+tWt7vV9
Jv5NGQuPMusgKnlYOc8HjrxWJ4R8MeLNL+Euu6b41bUNMvtQ17S7fT21zdIUzL8zxrIcHHfH418k
Lrl/FfRXQvLgXMZzHOJW3p9GzkfhVzWfF+teII4F1TV77UVhz5Qu7l5dmeu3cTjoK0Mz7z1xnvvC
/wAULS/tfE08GnafNC1xrcytYTy7gFeCLaAp4yu3hRxVOw8SeKNJ/aQ0XwXotqZfBdta2wlsBaA2
1uPJVmkUgfKQxBznqa+MdU+M3jPVtLbT73xRql3ZtB9mMM10zKYsg7MZ6fKPyrW0n9pj4kaJpllp
9p4uvobSzCiCP5WKgdFJIyR7E4pk6n1jea14b8E6Hc6zearH4fvte8Qakz3f9jJqJuUjmKKjBvug
dsdc1x/iDWvB3hbwf8RvGeieFNO1FP7ctLaxj1exKrFIYcyP5bDIBYsdnA5HpXgXhL9prxx4Mtbq
3s761u7e4uHumj1G0juVWRjl2TcPlyeTjvWHr3xl8TeJvDWo6Jqd6t5a6hqI1W5kkQGWScLtB3eg
HamlrcRvfEzxt4U8ePoeoaLoS6JrwtmXWFtYVhtZJMjaYkBOOM5rg5vmYgVS0mRFmcswU4wNxq7L
PHk4dT+NfpWRqP1SOvVmMr3Gsvy5AzxTN59KXzd3B9expOA2O9fQ8i3Idw3+1QSHaTVhk+XJ4+lU
5V2tVKmh8w18sR+tO3dB1FRTE7geadu4AFJ0kHMEh2KD/FVdpG45FSyLuzk/hVZk2H2FEaYcw/d0
yc1bhl3R89M9azuvsfSrNu21SM8UOkPmLG/HzE8ZpGkLNjPFQseB7dKTJrF0g5tRZmxwO561Dgbi
etLP2PIxVcygcZqPZXWpRM23pgY+lQtHHjlF/KkdvlwDUe/rWTw8HukPmYLHExOUX8qkZyq7RxUD
OQc/yp4YYBrL2EYapWFdki/dPc1IrYXOORUXO0dqf9DScAF8ztUkbfjVfad3Ix71Og4NYuAEyc/W
rEeQw5zVePJx61Zj9awlERaRTUqe/Wmx/dBp+4dQK5XEZIvrjihgDmmBsdDxS+vesXAA2/L1qJ1w
eakZjn0FMPzZrPkAgkAwfXtVZhtYZHNW5FHSq7jFNRAgZuuahappBntUbLg+1Jx7iK+8p6Zop7Ly
aKjlC5k6K261A7g1rI+OO1YOgyZtTjk5raRuOeld2A97DU35G5djrSsdSns8GN8D061krxjnNTrJ
0ru5e5EoqSs0dvp/iiOQATrsJ7iumtLqO4UFJFb0wa8rjb3x3q7a38tq4MblCPeodM8utgISV4aM
9XjI781P5auuCM/UVwmn+MpYyBMgcf3u9dPpviWzu9o8wIe4alys8Ktha9N3t9xptpEEn3RtPtUE
mguAfLcH2rRhlD4ZTuFWlbDcUuU4vrNSGlznP7PnjwCmfpWX4iubzRdPNzazTWlwjgLNC5R1zwcE
Hjiu/jX/AOvW34P8H6V468XaLoerxtLp15c7JlR9jEBGPUdOQK4MdD/ZqnoztweNlOvCEluz5ut/
E2p2djJYwX1xFaSSrO8MchCNIv3XI9Rk8+9dbqXx68fatYSWV54p1C4tJbf7LJDJINrxd1IxzXVw
fAO4h+GfiLU9Q0y+sddi1Oyt9LW5PlRzQzs67vmGCDgYbOBWdrn7Oes6bo9/f6Zrmh+JH06Pzb6z
0m6Mk9ugOGZlKgYU8Eg1+Y8kj7g4TxF4w1XxjfJeazdte3UcMdskj4BWNF2ovA7AVvx/FrxAdW8L
6i9zC9x4Zhjg07dENqojFlDAfe5PetyT9mDx5b6THqBs7JlmtVvLe1W9j+0XETIHzHHnc5APIAPI
xVDTP2e/iDrGiwarZeHZ57GeAXMUgkQb0JI4BOSflPy9enHIppMDR8FfHa+8IePL/wAWzaNpetav
dS+cJLxGAgckktHtYYJzjvxT9a+NzalpHi3TbLw7p+j2niKW2mmitXkKxNExYldxPLE5Pp2rmfhj
8P5vHnj7T/D832u2SSQ/apLe3MssEaj5n2Dk4qiPDGoapf3sGlWF5qK203kmS3tnYkliEyADtLYP
HsaPe6DPb/hX8a9Ck8UeC28RQf2UnhvTWs7C9Ds8XnliyyTADIT5sECrX7Q3iDwv4k0Oz1L+2dK1
Pxg1yRv0GaeSA2xGT5nmk7W3dAvGCa+ebjSb/Tb42F3aXFteAgG3liZZAew2kZq3NoOpW+mvfy6f
cxWSSCBrh4WVBIRkJkjrjnFaa2JOq+IPxAsPGljoUFr4csdCk021FvLLZDBuyABvfgc8e/U1xiSc
+hqtu4PrSq/Y+tTa+4z6t0H9pKLSde+Gdlp3iC4svDWn6fbW2swGEiMSKW38EZbjHK1F8BNV8N2/
xc1rxdqPirTNMtvtF0sVtdMySTLLu2upxjHPrmvmu2028msZbyK1nks4SFkuFjYxoT0DNjANOhjl
kDbEd9oydoJwPU1psI9n8IeD4IfjhaQL4k0GW1s7yPUG1D7cqW8iCRWKq7Yy+D932rM/aJ0l9P8A
ihrd2LyyvLXU7iS8gksblJhsZjgNt6H2NeVtI3emeY2cDigktQB5Zo40BZ2YKoHUk9BX0n8S/hz4
oh+Dvw0s7nTrq2ubWedLuaQZFu8syiMuRnGcivmJZNpGDg1p/wDCSaqLfyf7SvDFkERtO5Xg5HGe
1KKsyz6X8RDxB8D572Dw94U1DVdbSBkvfGV/bSykuwG7yeq7AOMtknrW54eutNt9K+DtvqXhqXxL
c3ESvb6ks7xCwLTjjCKVbB5+b0r5oj+LXjRLd4V8V60IXUq0f2+XaQeoxuqbwz8YPGXg/Sm07RvE
V7p9gxLfZ4mG3J6kZBx+FadSGS/GV5W+Kvi3zpVuJf7TnDyqu0Md55x2r1H9nHUI9B+GfxN1WTTo
9US3tYM2k2Skh38bgOoHBI74r55munuJXkldndiWZmOSSe5Ndn8OvjF4h+GMV9Doslv9nvin2mG4
hWRZAp+6c9iCQR3BotrcEe/6X4H8OeOIPCfivUdBsYbifRtS1G5sLFPJtZmt2AiBQHocnODzXPaL
8PfDfxo8N+GdXTQrfwrcXeu/2VLHpBYxyReXvLEOThhjjHHNefP+0p4mbxRZ6qbbTo7S2t5LMaPB
b+VZtDJ/rEKA5+YgEkHtT9Q/aU1qN9EbQdF0rwzY6Xdm9SzsYSYpZyNpZt2T93jANUOzOz1rwH4D
8QXHxD0HSvDc2iX3hS1muE1Zb95vtBiYLtaNhgbvXPFQeNG8CRfsx+HbhNAvrS6u7y4Fq8dwjOLl
UCs8jlQWjJ/hHSsDxJ+0dpF9pHiMaJ4Lt9E1zxBCYNR1L7Y8wkRzukxGRgEnoc8elcjN8U9G1b4O
Q+DtW0u5kvtLnln0q9tZgiIZCC/nKQd3fGPWqVrlanUfsn6XaXnjHWrg3M1tqlppVxNbsIElhK7C
HDhuc4IxisPwH8E/D3ib4cX/AIz8S+KH8Oafa34stqWnntJlA3yDcCWyRx6Amtf9nz4m/Dr4dWWo
XWux6/8A23e2s9hK9mkTwLE4xlQSDux61h+JviJ4VT4Pz+C9BfUJiviNtQimvYVUta+VsUsVJG/P
UVSXQHvodJ4T/Zah/wCF4aZoF7qljf8Ah947fUA004tZ7u1kBICRk7t/ByoJIFO+P3wz1LxF8YtH
8F+H9A8P6YsjSQ6cNJdA0kIPL3JDHDKFPXB61fHxo8JXX7SHgzxQ99JF4e0zTre2lnaFsxukDKfl
AzgMQOK5X4afFHQtN/ajuvFl9dmDRrq+vGjumU4QSh1R2HUD5hn0FMz12OW8ffs/3vg7wvP4hs/E
GjeJtMtZlt7uTR52kNszZ2+YCBjJBFeN3DbZMKa+n/H0ugfDX4F+J/Ddr4t0vxTqfiDVLe7jOkMz
rFHGSW8wkfLyePWvltsM2SauysJHruj/ALLPxM1nQtK1ay8P/abPU41lttlzHuKMCQzLuyoIHU1m
aJ8AfiH4gk1BNN8KaldtYTG3udkX3JB1XnqfpX17/bWmN4duraDUbVnh8IaBYBo51wGknBIGD1/l
XffFK7bXmtE0TwS3xA+y6jdJctZaq1obGUOuFYoeSRg89hSuwPzps/hj4t1TWNQ0uz8Pand6hp5/
0u1gtXeSDnHzgDI5rmfEvhfWPD6QSajpl3YJcAtE9xC0YkHQlSRyPpX6EaL4o1bXvEXxJ1Se0t9A
1Fr7QtJMOmX63BTM2CDMmMkqSCPwNefftnakdQ+Fepvcv9oZPG1za2jyHcYIY4gDGhP3V3DOBxW0
ZSbsB8WeGdJu/FHiLTtItX23V9cJbRF2IAZmCgn866T4neBtU+FfjXVfDN7eLd3Gmy+U9xbljGxw
Dxn61q/s46ampfG/wXAF3k6pC35Nk/yr688TeJNU+IHjf42eGvEWmWqeG9I0y4uLOzayWNjMHURz
eZjeWJO7rg56YrT2tSLsm/vGfAbahcleJZM/U1BJqV7185x6V+kXgnwX4aj1jT/h1rUnhiK4h03/
AE7w3a6E8sgzCXLi/b5gxyG3HGDwO1eXeL7fwR8Gvgf8P9Ti+H2i6/rutT3KLPqSFwEEh5cAgudp
UDnirjjK/Sb+9k2R8XQ3GsXEM9xBHPPbW4BmmSIssYY4BYgYGTwM08a5exjGVI915FfonD8BfD/2
P4u+GNKH9j6XqFxpjXMNsu+RAIjcSw26dz2VT0z3xXzP8P8Awx8Mvid8apbC6spfCegQ2zrZaZdX
ZR724TgJNM3+qZ+SccA8Ct45jikv4j+8LI8DbxFcK2CE9qrzeJJ9xzHH+Ga+hf2ivhv4N8E+HHH/
AAhWs+BtfEqGylW9Op6fqUZGX2z4UKVBU5Gc5xXzLChmkCKPvHAGOa6FmuOirqowUVc1V8SOrcxK
SR2arsPigbTugHoPmr6rsf2D9IsY9G0/WG8aXmr30MMkmoaJpUcumxNIMhS7MGAXI3E+9eZa5+yb
qdroOuSaJqA8SeIdH19tGutH06Le0cfRJyc5wzcfdwMHmrjnmO/5+fgv8hcqPKv7ehkXLKye+c0R
+JLaQH5XU9K+r/AP7Meg+HfC/wAVdL1jW9Evb220yyjbVrpcRaVM7FpUyCfnQLj5eScDFeJ+Iv2U
9dsY/Dd/4Z1vTPF+ja9dNZWuoWbNbxiZQSVYShSBgE56cVpHPsalq19wcsTzyTXLTyyCzA/7tVf7
YtZODJtPuK9N8S/steIdN0NtT0jW/D/i2GK5itbgaFemZrd5G2p5gKjaC3Ga5NP2e/G0/wAUb34f
2umx3nii0DGa2huEKDaoY4ckDoRWi4ixiWqX3f8ABGoo59tWtN2PtAJo/tC17Tpj610vhX9mf4je
N7Ca/wBF8L3N7axytAZDJGgd14YLuYbseoyKtfCX9lvxv8XfEl9pdnp02mwWBljur65hYxQzIpby
jj+I4wB71a4jrr4qa/EfIjkVuoGbiVT6cip45EY8Op79ax9a8D674b8QS6DqWlXlprKOI/sUkDLM
WPQBMZ57euah1fwxrXh2RF1XTL7TXf7guoHiz9NwGa0/1kle0qX4/wDAFyHSFt3OQfxpc5+tW/hv
8Hdf+JTa59kkj09dJ0yXVJXvd6B40x8q8dTkY7VwonlXOZWH4mn/AKxQerpv7xcp2Kr83PBqZV4z
msjQpXktNzMWy3c1rJ8w5r6ahNYilGqla5BNGT0/KrUfPNVY+D0q3H3olGxKuWIztAzUi89KhTJy
KmjA+lc8og7jlJHFKf0pBzml78c1g4lA3v1puKd97tSj/wDXWcok9Su2ahkywANTvn8Khb0qeUor
smfqKif61YdqqyfTFLlJZHuOeBmikZWY5BwKKylHXYPkc14dY+TIO2a3kYLiud8OSFo5R1APGa3U
bPXkVvlfvYSH9dTcuxtx1x6VZjI/Gs+Nwp5qzGx6V6/KK5cVunoanjYcdzVNWC/Sp1bpjmjlFzF1
ZAvepI5Cpz3qmjfjUyPS5SDYs9Yu7VgY5mGO2a27Hxxd2+BNtlH05rkA+PrUm48VPIc1TD0qvxxu
em2XxBtJABNG0Td+9dj4A+KHh/wr4w0TWdQu2jsLK7WScohZguCDgDr1rwZZMGotSWS4s2jiDO7E
AKoJJOegrix0P9lqejOOGWUYVY1IXVmfStz+1F4f8beCddsPFUYbGvWlxaWSIzGWwSXc0I4wFRQc
An+M13cPxf8ADK+H/HaXXj7w7dWOpabPHpGl2VisEsKkHZFI4QEsBhdvI4JzXxJrng/X/C8cT6zo
moaUkpxG97avEH+hYDNY+8qa/KOdnv2PoH42fE1rbxx4O1nwrrim7sPDtjCLi1kD/Z5hEVdecgNg
4IrsLvxcl5r/AOz1CurRywQ29u11EtwCsUv2k5LjOFbGOtfKPmE5xUizEcCkpa3DofWPgXRdU1v9
s++v9Ige6sbHWWmvLi3YbI4mJGWIPIJq1oX/AAkXwv8AAvxsmK3Whaql5ZujsNkixyTSYZT2yDww
r5OtdSubKTfb3EsDf3oXKn9DVuXX9Qu1mE1/dSrNt80STMwkx03ZPOO2elWLofcXgjUo9S8TfDC6
1C1bUde1zwxJEmsSoJZoLjzG23DseTtAxnORxXE/tWX+u6H4V8N+F9Tvb7U5lkluL7VprPyIr6QH
EeM9Sikj6Eda8J+Gvxs8Q/DfWobyKb+0rRbVrGSwvXZomtmOWiHOUB9VxW58Rvj1H418J2/hzS/D
Nr4b0pbr7bLHFcSXDSTbdoIZzlRjsOtWtUKxpfHeXxM+n+DP+Ej8M6f4fQaYosprELm7iwuHfBPz
dODg814+sg3elWdU8RalrUdsl9f3N6lsnlwrcSs4iX+6uTwPYVmmTnPakkFz9BfAfhm007RfAnhO
31rR7bRdV0cyX2j3kpF5ePcpuaRVxtYrj5M9MGvKv2UI5NJ+MviPRI5DLp32O7SSORQRKI2whbjr
z+teRWfx+1+Hxf4V8RzW9lNeeHbWKztY/LZUeNFKjfhsk4Y8jFb/AMJ/2h7b4a6tqWqP4Ss9V1W+
eVnvmuZI3VJDlowBlce+M1Yiv8E5mt/jf4cCxwyJLqawOk0SuhR2KkYIx0P4Vz/xYhhs/iV4pgto
lhhj1S5VY41CqiiVsAAdAK6Dwh8VPCfhr4m/8JQ3hGf7HC6T2en2+osPs8qkEsXZSXBOeD61zXxU
8WaJ4w8YX2s6Hp95pkN67TzQ3c6zHzmYs5UgDC89KAeyOes1ikvIVnlaK3LqJJFXcVUnkgd8DtXu
Hxc8E+B9Dsfhtc6KLq10fVYGe8vphmeWMShTKU5AOMkAdsCvCtPe3e+gW7keO0Z1ErwqGZUz8xA7
kDPFfQ3jDxh8L9c0v4f2kWsapdWfh8paz2t3YbPtFuZN0jFlbggZ4HWkmPQ6HTfhb8P/AIh6F4ik
0rw3qPh3StMtpZbLxZcXbmG8aMgANG6gAtnoDUPhv4AeD7jTvCmn3lt4h1HUdfs0uP7b07Z9is3f
ICSAg/dxzzk+1Y3xa1Xwn46+13lh8UreGxt4ith4f/sy5gijiXlIgFBUn/aI68mu88K/FLQpm8F6
xb+PrXQtA0jTI7fU/DU00kclxIgYOREo2PuyMZ60xbnyV4i04aLrmoaf5y3H2S4kg81RgPsYrn6H
FeqfDP4Ij4kfCvxFrGnx3V14htbyC2s7eJlEbBuX3Z9u+QBivJfFGo22oeINSurVfLtZrmWSFcbc
IXJUY7cEV9FfBGyufEn7Ofi3QtH1e1stdvNRjEVvJdrDJOoUExjJH3gCOeOtPqNbHi998JfGFt4r
Pho6DePrYXeLSNN7FcZ3AjgjHfNZHibwZ4i8J30OlazpN5p13J88VvcREM+TgFR3544r7Jkll/sy
/wDCWn3aTeMbbwTFZCytrhTOLgSlmRWB5YAhuD0NaPw1t30q6+E2k+L2VfEkY1KWSDVHVrhSeI/v
EnnPy/Tim0CZ8Na14N17QFjfUtHvtPWRtqG6tnj3HHT5gOam8VfDrxB4R0PRtU1SxNtZatG0tq5I
3FVO07h1Xnsa+l9W1Txi/wAG/ind+M21TzLS/tH0f+11YGL9+3zQlx1xjkdq5/8AaU+I3ijUPhH8
PLS51GS6tdW0sXeoO0anzpRJ8hLY4P0xVqI7ngXg/wCH+qeNNP1+60424j0azN9c+dKEPlg4O0dz
XKsrf3T65r6R+AN5NL8DPizHcQQS21vpyyQSSQIXV2ypIfGcYA4zW/4e1LRPAPwh+FTDwfoWs6v4
jvJ4JbrVLQSt5fnhenUn5hgk8Y96fQpny7ougah4o1S20zS7S4vb+4bZHDBGXZj7AU/xf4H1zwDr
b6T4g0+bS79UWTyZsbtrdDwe9fa/wT0Dwd4N/aa8daLpulXUd7p8c0lvcLcgwxQFELx7MZzuY4Oe
mBivF/BPhnwR8SviR431uS21W98NaBpM2qfYdRuh5txIhAMZkQcISTjHPAqjNnzZdXEiW8kSsQjf
eAPBx0rHkfgkdetfQfxv8H+EJ/hb4Y8feFdGn8Mxapez2EmlzXbXIPl8+YHYAj0x0rw3RdLXWNas
bLcV+03EcO5RnG5gM/rWqV1ckz4L5rdi2T7YOMVftvFWp2G42upXdtu5byZ3TcfU4PNfZWqfsQ+A
JtQnsNP8R63BdR339jL9ohjdHvTD5qnIxiPGc8Z9K5ux/YNtvJ07TNT8T6ha+I7+IPGLTRpZ7BS2
SgafgKcDnOMZqoyj1A+U7Lxdq2lrILTUry2EkqzsIZ2QNIpyrkA8sDyD1FN1zxxrfiCxFpqGrXt7
bC4e68m4nZ185/vyYJ+82OT1NfQ/g/8AYf1PxLpd8bzxPYaXq8Wq3OlQWTxPItxJAu5yJBwBgMeR
2ryf41/BW7+DupabHJqVrrWnalbfarLULPIjmTOCQp5GGyOfStk03ZAcX4X8Wal4O1yz1nSLp7LU
rKQTQXEeMow6Hn+teheJP2oPiT4w8MPoGq+I5JtPkIMu2JEllAOQHkUBmGexOOB6VpfsjfCvR/i5
8WE0fXbWS801LG4uXgikZGdlT5ORz94iofiJ+y7428Era3IsYdStry8+xxLpNwLt45iflikCfdY9
MH0NT7rdnuBsaf8Ats/EjTdPigjudMluFt/s0l9LYIbmePZsAkk+85AwcnuAa4Txp+0B4j8caJ4R
0rU47M2nhgEWYhiKs+SCTJ8xyflHTFafjb9lv4kfD3QX1nW/DzQ6erKjyQXEc3lk9NwRiRzxz3qh
N+zL8T47Se4fwRrCwW8QmlkNucKhXIY/hzSSgtgOk8Wftg+LvFFvqqpb2uj3moaxb619tsWdZYZY
YhGiJz93AB55qW1/aK0TVPHVz4i1/wCHmj6pFqVmttqdrGxi8+bHzXEbY/cuxwTtH881wWtfBHxD
pHgLwv4o2Ldw+Irma1s7G2Vnud0RAOUA5yegGTxXJ2Xh/VLqeS1gsbqW4jJDwxwsWXBwcgDjBocV
ugPZ/jJ+0Roviz4cQeCfC/ha40HSPta3sv8AaGpPfujKu1UhZgNiYJyO9eCWM/2O8iuNu/y3D7ex
wc4q1qWn3OnzeVdwSW7gfdlQqR+BrNk+Yg5wtTaysB9neIP2mvhv451ODxPqPib4heH9UFtEs2ha
JOIrRnjQDy1kD5UMR97bxnOK5TwB+014a+D+lWOp+E7W4n8VaxqEr+IZ9YLTNHZ+ZlIY3/iJU5Zs
bsjqK+WmhYLu7daaY2wCM/U1HIrAfWnjr4p/DLSfhR8TPDngvV7i6Ova5bXNnDcQSBjAq7nO5gMA
MSADzwK7T4d/Fb4Xf8IB8GvDXiDU7G8SxnvLi+tbhCYra4ZSIGmyMbdxBzyOK+NbjwXr+meGdP8A
EF3pdxb6LqDvFaXzpiOZk++FPfFYTZTPPHvUvYD9KLj4uaJofgNbfxb488I6vqE3iHT3mXw1BGqx
2qS7yT5agycLzxx070/wP4XtPDfx++IfxO1XxV4bFjfWt5LpsMGpxSyyqwyrYB+XCgDB55r80pJC
qqRxUnnblz3qNlYD79+GvhXXfiV4B8NWHiPSbXX/AAmDJPp/inQtSSC80LdLuZJN2BhSCzYUn3ri
v2cby/X4y/F37P4mm8V6tFot9HY3Uc7NLqUxwqvGucu2BnjOBXxwuq3cakRTywg8MI5CuR74pmn6
tfaNfJd2F5PY3cf3Li1kMbrn0YHIqL9x2Prv9jO2ltfiJ8Qta8VW99e6zo2jzv5EjE6h52QuIt2W
Eg5wRyDivYPD+taZ8TNZ8B+HdS8JeMJ7CfxA922oePpVuW2xQMzwRFvm2nCkjGOO9fnzoPjbX/DO
uLrel6zfWGsKWYX9vcMk+WyGO8HPOTnmtrV/jT4517xFp2u6l4q1S+1fTiptby4uGkeLByME+9Q3
qx3Psrw/8TfF/iH4R/HHVfECSXGkW0TWmlXFxbbGiWSXaYY3wMqFC8fSvhg6LqD6W+opY3LafG3l
vdCJjErHsXxgGu88TftJ/En4gaXdaPr/AIuur3TLzYtxbyJGkbBSSMhFHc9uvFVR8b/E1j8MLn4c
Wuor/wAInPc/aXt2tkDs+Qc78bsZA4z2qrp2XmHmZGips06LjrzWkh+bHUfWqmnr5VrEOuFq9Go9
MHvX7LhqXs8PCPZL8jnbZPGPm561ahOOCeKrR/L1NWVwelROIalhevHWplwy+9QLjHSphkc1zOID
j8uMfnSZ79DRtHc02sHEOo/dR+NRbqer8YxxWbiMSUYU459KrsD6VM/Q+lRN9eKmwFdvm681WfAO
DkVZbgHAqFvUjJpWIvYg59qKduIzxRXNP4izi/DkwHmgHkgVuxsWIHeuZ8NH97J9M10anGa2yX3s
FD5/mXJl2NvmAP51YjkFUom3YJNWI2HUmvdUSC2rZbjr3qdH96po3qasR9PSq5LAWlkzyBUyybqq
rwODUiydu1LlAtLJ61IH7VWVjtBHWpFOFPPNTygWF+X3r0b9nzy5vjX4ISRFkRtVhBVhkHmvM1Yk
c1Zsdav/AA7eW2paVPJbajayrNbzRHDI6nIIrlxlJzw1SMVq0/yGtz03VviB4y8SXnjTwpe2l94q
0K41uOKSWZnley23LKiwuxKxF87cn2r2a/8AgD4I8VaP4l05NE0Lw9qmi2EtyG0rW3u9QjkjAPl3
ERG3PZivGelfPniL9o7xxremy2cWnaToZuLiO6u59H05LeW6kjbehkYfew3zfWuht/2u9fsI9aeH
wV4fg1LWrV7fUdRjt5RPdMy4MjHf15JwAASfpX5N9WrLeD+5m99DvIfgx8MG1rwN4Pk03WG8Q+K9
It7oalHegQ2cjxk7wmPnBYH5eMetZdr8B/hro+meCLXX9S8QPrnihWigXTxGIoZBM0QkYuOVzt+U
c8GvNrX9om+j+IngbxVPosbN4VsobFLVJGUTrGrAMzEHBO7t6VLrX7QK69q/w6u30ZoV8JyGSRUn
ybkG484hcj5eOOc1k4OLs0I66P8AZs0PwnpviTWPG/im4s9H0vVm0aKTSbPzpZJgM72ViMLj8c1t
/Dv9nPwnF8cIPDHiLXnvNOurSO/023W3dHvoZImcB2U/umUYJGSD0rIvv2jvCPxCsPFWheK9L1fT
dC1TWf7atptLaOW4jkI27HD4UjHcVTH7R2iw/tB+G/GMGnXZ0DR7SHTQshUXEkSQmLzWA4DYOdoP
bGaWwkeWePtF0nw74nvLHRNaTXtOjbCXqQPDu45G1+cg8fhVXwh4Z1Lxt4isNE0iD7VqV5KIoYgQ
Mn3J6ADJ/Co/Hv8AYsPii8Hh7VZtY0tm8yO6uLY27sW5IKEnGDxXTfs8eNtO8B/GHwxresSPFptr
dZmdF3FAysu7HoCwJ9hTKTOn8R/s1+K9Dt7e4trvR/EFvNdR2TSaPfLOIJXO1BL02ZbjJ71y7fCX
xSvjDUfCy6VJNrmnpJJcWsbqdqou5iDnBGOeOte9/ELxhdapoFzo0/xH8ErZ6xqcCImg2QWTZ5mf
NnkVV2BBg89eeRXT+Ivid4a8VWev+BdC1xbfxLZ6QYj44meIDVFiUs8BcchWBIBBJOMUEnzn4V+G
aav8LfFXi26N9AulvDFatDDvhkkZgHV2/hwpB/GtLwb+zp418beCtR8Uadp7rY2qLJDHIjiS9Ukg
mAAHfgjmvSPgx8PfEGrfs5fEFbK0W5fWvszafElzHumEch8w7d2Vxj+IDNaX7Neva3dfBP4naPpF
9ezeIYFh/s6wtZmaePBIYwqDkcnkrTsNny9cRTWNzJBPG8U0bFHjkBVlYHBBB6HNdf4T+G+peMPC
PibxFZzW6WugJC88MpbfIJGKjZgEcY5ziuP1aW6/tW7N+sy3xlbz/tBPmeZk7t2ec5zmvpX9lvVd
V0H4SfFTU9Es1vtUtY7RoY2h87GGcl9vfaMt+FHUOlz5yZGjbDLtPoRikMr7AOWQHIFfdHhbQdO+
JmjfDLxL4r0q0v8AXJTfPcStAsYkEUcjorooAIDBTgiuU8IaXpXx58K6PqPiTQtMguofE8Gnb9Lt
ha74HTc6uF+9nA57dqLCPkIO+31B9ae3mrD5pRhHnaH2nBPpmvrCPR/CnxS0X4pWU3g7R9AbwrFL
JZXujxGGZyhkA38kEEIM/WsrxR4n8NSfsm6NJ/wh9rEbq/mtoFiu5B5NwsYBueepOOV6VXKO+lj5
WeRmYn1pPtDLzk59ajmkCsRXu/wR+Cvhbxx4C1LxFrE+savPazeVJo/hwRG6towCfOcSfeU4xhRm
i2o1seIQ6pc2siyQTSRSLyHRirD8RT5vEuozXkd299cPdRkFJ2lYuuORhs5GK+hPCf7M/hzxrrHi
W50zXtUvPC+lLDtS109pNQ8yUH900eBkpg7iB9Ktn9iSa/8AGS6daeIWtdKuNKbU7e41G0MU67WV
THLHn5OTnOenar06juj588RfEbxL4ss4bXWde1HVLeFt8cV5dPIqNjGQCTg0jfEnxFJ4QHhRtWuX
8PeaJhp7EGMODnI4yOa9fm/ZPGstpEnhPxppviOxvNS/sme7jheJLW42lu/LrgNyvoPWvOIvgvr1
1o/jPVbee0Ft4UkSK8LOVZy0hQbOOfuk844q0Gh0Pgz9qXxz4M0SDRrW/t5NHgj8lbWSxgfKdgSV
+b8c1zOrfGjX9Wi8NpPLbmHw7cPcadH5CqEZpBIQ23GRuA/Diuo8P/CWwf8AZx8ReNLq1+2al9sh
gsrm2vVItRvAcSxZzkjpweoPFZ3h79l34i+KvCsWv6fofmWU0TTQrJOkc0qAZ3LGxDEHHGBz2qkk
BH4Z/aV8TeGfiR4j8bR21hdavrkMkNwksbCJA+3lQGyCNoxkmsj4U/GKb4Y6vq0kumQa1pWsWj2W
o6fO5jE8Tc4Drypz6VF4S+Avj3xxZT3uieFtQ1C1gkMUkixhcOOq/MRkjvjpWSnwt8Wah4kvtAt/
D+oS6zZBjcWMduzSx465UA+o/Or0Jub3xh+Nlt8QfD+heHdF8PQ+GfDukNJNDZRztOxlkPzsXbnH
TA+tec+E9Yi8P+KNJ1O4iaeGzu4riSJDhnVXDED3IFSaz4X1TQtWfTNR0+5sdQUhTaXETJKCenyk
Z5qjqGl3Gl3Bhu7eW2mXkxzIVYfgea2UdLEn2JpP7avhM+Ikv7vStVt4m8ST6y4VI3PlNbGKJfvf
eBPPb3qzqn7Ungvxbd2WtXHxA8ceFJIbNIZvDujLtid0UgbZclQWODkrXxVY28t9eQ20ULSzSuEj
jT7zMTgAe+TXpOqfst/FXS7Z7ibwPrHlom9mjty+BjJ6ZqfZwvqB9DfCT9pXwRoPh/w4NS1m8S9g
v9Yvrpr2N5pFeaNlh3uq4ZmzyR3z0rxT9pz4iaD410/4d22g6imoLpegx292EVh5U5Ys6HIHI9uK
8Skjmh3odw5wynjkeo9aijjaTsK2UFF3JtqfSX7C+v6F4c+IPiC41rXLTQEk0aaCC4u32DexUcc8
kdcdeK9g8J6f4U+HHhTR/Bnijxlpk9xrXixNRe50jUPMPkImFdpU5jJfA5Oefxr4RgjfzNq5Le1K
7SLIQRhgfSsnC7bRR+mOpWq6b8N/iAdQ0Pwz4be+khtLb+y70Sz3cTTgCSX52HOeOc53ZrF8WeMN
U1L9urwn4Zh1e4/sTTbQO1nFMRFn7O7tuAODnC9fQV+c9xdy+WNzttxwM02z1S9sbqO6trqaC5X7
s0chVxxjqDnpUql0uLqfpFqGpK3wx8MaT4au7LTviLf6de32gTXajkSXDNJHAScJMy42sR2OMV5H
+zl4s1nwD4L12DXdB8R6Suo6r5cnjDRLdJruGcH54p1cFtgI3EAc5NfHreJNY+1WUzajeGaxwbSQ
zNmDByPL5+XB54711/hj42ePPB95fXmieKtT0+5vpPNu5Irg5uH/ALz5+8eTyfWhwaVhnpH7ZFv4
mXVvC0niLVbDxLDPYNNYa9b2Zt7m8hLnidTggoeACBxXHfsp/D3SviN8bvDej6zAt7przNJPasSB
IiIznJHbgfnXFeNvHfiD4ha0+reI9VuNW1F1Cma4bJwBgADoBj0pPBPjHU/APiKw17Qb2Sx1Wzfe
ky9B2II/iBHBHcHFEovlSBaH2z48174Qat4d8Yadqes+D9RWGzuBY2mk+GpbG+huFz5Q87GCARg9
Aa53S/gf4H1rwn4Z+L19oD6X4V0nSlGo+F47VjLqE8fyiUEEZjkYgtIeBj8a8M8UftYeMvEWi3ml
vZeHtOS8wl1daVpUdvLcRg5KMwHKk4yPanW/7XnjFPiRP4wuILC6mm046Y2lsrrZ+T5ewAR7u33s
dM84rPlaQH0F4417wFoPwN+Eul6/4JfxCuqm4urDTRetbRWqyzHksvzMwBUDjGAayNU/ZB8H2fxn
8Q6dBoGua3pVtY295Fplndx20MMkwJMb3crDG0cqOS34V88+Mv2g9S8cXXgSS60q0t4PCdvHbW8M
LttnCOGy+ehOO1eral+2pYeNNS8VxeLPB8l34f15rWZ7HT7/AMmWKSFQq/vSp3KcZxgUmrbAdj4m
/Yf8IR6sJydV8P6efDt3q8umPdRXM1vLCQqgzAFWViQeB+NeLfDn9mO08d/BPWPG7642n3tvqsWm
WltIqi3feyLvdzyAN/b0r1Fv21vDN/fx2D+ELzSfCreHpNAlt7G4V7iFGYtviLfL6A7s9KyfEH7R
nwoh+DVp8OfDvhnxDFpX9qRXl8b24jV7mMHMh8xWJVzxjC4GKXRXFqYXxK/ZT8F/D+11DTbj4gXF
l4ssrQ3KwappptrG9ZQNy285Pz5yQvHOK09L/Zr+Gkn7Mc/jG58Zwx6zcXipFqc9rOkUDrHue18s
Z3En+PGK1PGf7Qvw2j+E3iHw1BrGv+OI7yPZpOm69Yqr6U2cq4udxZgmANuQDXDW/wASPCPib9lO
28FXHiBdA17SNSm1E2txbSSLqJYYVUZBhSOnzelQt0Vc5T4d/szax488LS+JrrXNG8K+HfPNrBqW
tztHHczAZKIFVjwOckAVZm/ZB8ep4wv9CVNPFtZW8d5LrVxdCLT/ACJB+6k81sYDdACATXufwb+N
li37P2g+F/D3jjw74K1/T72Z74eKYVeK4RuVMeUcH3JweK0bn4gWfxe8I/FHwr/wsHRdV8S6hLYQ
QahfhNLtZrWE/MIwTjAbOMdRg1C13Bs8X+NP7Odl8F/gj4V1TU1ZvF+o6hcJNLBciS2kt1AMbR44
IIOdw65r55j+eaMerYr6o/ayuNLs/Afwj8D6V4l0/wAQy6bYyLcXNnOJIxK8mOTngDpz2FeP/Fj4
G6p8FfE2j6bqmq6Xqst9brdo+lXBlRFJI2scDB4rejT9pOnDq7fmLXUpW42qF6YGKsJwarKMCpol
OeelfuDhZWOa7LcbbenI6VajY7aqxDdx1q0q7VxXHOJSJo2+XFPVveov4R2p69B6VyOJRKGJpN2P
amghevFNZsfSsJRAczAmjJUYpGG0Kegbkc0nvjNZOICEk5zmmNnpUjGombcamzAifIaopPm6VMe4
4JqFuCcmpFYjDDvRQuOeRRXNJLmGedeGT++kz/drqFO4c1zHh9w11jOcKR0rpd4Kg+npU5D72CXq
y57k6t8vXNTxNVVGHpip4/lavp0iC0rZIxUyt0xVaNj1/KpQearlYFlWPtUob8arLnHJ71IrY69P
WnygW1kxin7/AMqqh6eG54oUALStnnrUgbc3XjrVXd0p6sBzmq5ALQfk5OaXdt7VXV6l4b2FaqJL
2Jty9xmnAI38C4I/u5qHP/1qkVgR71fs1LdEEptoXUbo0P4CkbT7U9YE/KgNn2p247qiWFpSXvQT
+SHzMYuj2Tf8sAB7E07+w7L+4w+jVOr1KrDINZyy7CS+KlH7kHMyunhu0YAb5B+NE3heHfxM/p0B
xWgsm3Bx05qR5hk8ZJ71zyyXAS3pIOZmbDoMkaER3ciZ7Dip9P0rUNKuhcWWqS2VxgjzYWZGweoy
DVxXxjnmplkzwRWL4ey9/wDLu3zY/aMx5PDt3PI8jzrK7cs7sSSfUmuh8HeJ/GPw/wDO/wCEd1u5
0oTMryrazFRIVzt3DvjJ6+tRiToKepO4YrnlwzgJdGvmL2rNq++LPxI1DX9P1y68Q3lzqennNtIz
rtjz1GwDaQe+Rz3q/r3x7+JGuz6ZLJeJbHT7gXcKafbRwKZR0dggG44457E1zgcgdMVLu2jPUelc
8uFcJupSX3f5B7RnU69+0X4y1jQ9V0yDR9H0Qaqnl6hdaXp4hmuVOcqzZPXJz9TXOaf8Ur61+Gt1
4J1LQrbVrHzXuLG4nDpLYSsMM6FSA2fRs1Ep+bnGPQ04c8ViuEqMtVVa+SLVQ84uI3Xkow/CvWPg
78WvCfgexWLXNA1FNSt7j7Ra63oN39nuxkEFH3ZVl9OKoSKm3BQdPSqUkMUnBjXHptFZ/wCp9/hr
fh/wSvaHstr+1d4d17VvGNvqVpq/hrS9c+zGPUdEkU3yNBwGkJIBZx94irXhX9pXwR4Y8Ra/Ktz4
n1G0m0JtOgvNWl+0zTTk5Lbd37pe2AT0zXhv9n23eCIsP9gVDJpNg2d1tHk+2Kl8G1/s1V+I/aI5
jwv4wvfC2vadfQTy+XaXUd15CysqsVYHseuOM+9fRv7Tnxm8K694Nj0XwXqH2r+39QOsarNFuiEb
7VH2dgVXcAeec9K8Vk8PaYyZ+ygH1UmqjeF9PaQkxuo9nOaylwfjb3jKL+b/AMh+0jazPofRPCOl
Wf7Leu+Gbfx34UudVvbyPUgseohMRooJjIZQd+RjGMe9WPGnhO4+KWn/AA78Q+E/GekaNpuh6Bb2
txfz6iIJrOaLcXJQHfkcdBz2r5yXwdZzMVWSRD15YVUbwnbLuCyOAc8VyvhLMo9E/mHtEfZ3g/UL
rxB8EPBcPh/w3D8SLu1e6W/eHV2s5YJTKSWyGVjuyDyKjsfGXivVviH42hPhzz7IR2FlqEPhPVvN
1GzZF+WSN1AZwuSHXHUAZr40j8MzWqBbbUp4Rjnb8v8AI0aXpus6HdNdabrU9jckYM1vK8bkHnkg
5Nc0uF8yjf8Ad3+a/wAxc63PbvjF4f8AF9v+0D4BsbfxDHq+s+VbPpV1rMCRXEOXLKl4MZLg5+9k
4xXmH7Wt54xk+LlzD45l02XWoLWKMPpI/cmPBK++eTnPNcrqmj61qt897d6k13ductPNKzSMexLH
msi88I6pdSedPIsxb7zNLub9az/sHMI70n9w+ZdzS+DOltqPxV8KWhA3S6nbKAfeRa+j/wBqHx54
UtfGHi+LTdc8b2HilLlovKgu1XTi4wCMBgwUgV8waFa6/wCEddstV0omO9sZ1uLaddp2upyrYPHU
dDXqviP9pr4keI9NvLTVdH0G9N3G0clxJo0BlO4YLBscN79q5pZTjozu6MvuYc0e56l4i1b4Y/CT
Qfhfp/iDwDpepnXdGt7vV9UmjZp443zl0C9ZPmJznsPSqOpeHfgv8M/hR4T8R33gybxKviC/vFt5
bi8khYWqzEK7hT95U24AHds184/EXx14o+JcmivrFoo/sjT4tMtlt4SgEMf3c+repqx4y+JmueNf
Afg/wrd6clrY+GYZYoJIlbdLvbcS+eM9uKwll+Ip254SXfRhfsfT/gvRfAfgHwj8XvHvw++z61Hp
i26aNqF/bCQ2zSH541WQc4yBkjOK5DQJPDn7V19Z3PiDTn0S98NaTc6h4g1LS4I4mv4kcbEjQYVW
wT8xFedfBv44w/D7wnr3hHxF4cbxH4T1llkuLWOdraZZFIwwcA5HA49q7rSf2r/D1jrj6evgs2Hg
iTRH0L7LaTj7asLNuZ/NIwzbvUdK4/Y1I3vFjHXX7J/hn4l6T4U1n4a6vqVtp+salJpjx+IkTzEZ
FLmQeWMFQFIxXefDn4B/DrwanxKOl+IP+Er1bR9GurG7sdS05U+zzHAEkROcjcrDPUZFcjov7W3h
L4dXHgvS/CXhrUn8MaJfTXs/9pXCm5kaVWVtu35flDEjPXFasPx5+EXhXSvH15oM3iHUdf8AFI8x
/wC0LZI41zKHMQ2scDlju5rOUZ7MZzN1+wpPHB9gPi+3PjAWhuv7E+wTeVnZv2C5xs+739eK82+E
P7PGr/FuPxA9jqek6d/ZEe6ZNRnMbOcEkIADnG08njkV9JeOP2lvDniGLVNf074r6/pcVxZbYfCd
lYhXjm8sKF81wV27skkYzzivnj4Wan8LYfDPic+N01l/EckbnS5LFiIt204DYPXdjOeMULmadwPG
NRt/IkZcglSRxXcfBX4L6j8XdWvYYry30rSdNh+16lql02I7WHPLEdWPoBXG3hWRyQOvrX0R+yb4
i0lvD/xB8GTajb6VrfiTT1t9OuLxgkBZSSY2f+EnPGeKqYHM+M/2Yn/sfSda+HerN4+0TULr+zvM
ht/Iniu+cRtGSSNw5BzXltr8LfEl/e6nbW2i3s8ultsvljhJ+zHfs/eEcL83HNfZnhfQrvwj4H8K
fCjS/E2m2nxAn1065NNaXIeKxjSMkb5B8pfAztGc5rq/E2s6J4z8N3OofD69s9J0ybxBb2Xi+/EA
tprotIoWVWbIWMtuO3sTmsnJ7CPmtf2JfErTR6W3iTwxF4oMQk/4RuS/23oJXcE27cFsehx71xXh
H9l74heOm1MaXopjGnytbStdyLAGmGcxIWI3vwflFff2ki/uv2jNU+1eBILGys4ZDB42mtpVuJAk
QUN5jHy2PvjBrzrxZp2reONP+CKeHWudUsRqc9xf3lkfkSXz97mTb91gu4nOOM1CkwPh7wp8H/GP
jTxHcaDoug3d7q1tu8+3Vdph2nB3lsBefU1ka74H1rw74guNC1PTri11eGTyXs5EPmb/AEx3/Cvv
z9ozWHvPhR4wu/CT/aLjVfGP2e4uNJO5mREVVVynJBYcA9TXoGraZYWOvaZqt3a2954m8PeBJL1r
mWNXuVucKqM2Ru3DDYz0zT7Bc/LzxZ8PfEHgm8jtPEGj3ui3UqCVIb6FomZD0YA9qzb/AMNalpEE
E99YXVtBcLuhkmhZFkHqpI5H0r9G/gv4dg+LXw2+HV147t/+Eju5fEF5PHNqS73FvHG7smepQuAc
HiuP0TVp/jr8Jfi/L4wZdV0/SL+JNEQoE+wuzFFEW0Dau0KNvShK4XPgMq+0siHaOpA4FQvIdoJ4
PY5r9KNN0fSfC3xv0f4HaZomnr4Iu9FMmswSQh5riRofMaVpT829cDHTGa4SPwx4a+Ba/DbRNH8K
6brt745uSdSuNaiE8n2ZpvLSKPI/dja2dyjOR1rK2jfYNz4NLNuzt61c0+9NrdxyPukCDpmvtfXv
2c/h/wDB2z+Ivjy70+TxRYaBq407TNEvnC27SNt5lI5cKW+7xkCotS/Zr8HePtS8O+NWt28M6Hfe
GpfEmraHpoAVfJIUpCzfcEh55zt5q6c5UqinDdPT5MNGfJ0fiaFsfunHryKtw+JLViSyyKPpX0NL
+z74b+O3hvwX4j8C6avgeTVtYk0KfTZpXnhLIpk89WJLfd4I9RxQ3wB+HHib/hMvCHhN9XTxb4Vt
JL2XWNSkH2a98o4mTy1/1fJG05Oe9fRLiLHaXafy+QuVXPBofEFm3R2X6irsOtWkmB5mT9DXs2h/
steGo7rR/B2ta9fR/EnX7EX2n21rEDY2wZN8Uc7EbizAHlcBe9c5of7Pum6H4ft9f+I3iJvCulXl
xJa2CWNuLu4uXRyjvjcAsakHknJ7Cn/rHi95Ri/v/wAxcqOGXUrU4HnoB6E1NHdwScLKn/fVd1ef
ss69Z+NNS064vrW18Pafbx6hN4kmbFstm/McuOpLDog5J4rN8T/s66/p/iXRtP8ADzL4p07WoftG
mapZrsinjH32bJ/d7D97ceO9P/WKt1pr8SuU5hp4ycB1P0NNZu2c10Pjv4Baz4L8Px65a6rpfibS
xIYbq50SZpUtJR/BJlQRzxkcGo7r9n/xvpvgl/E9zYpDapGs72RmH2xIW4WZofvLGT0Y0/8AWJ/a
pfj/AMAXKYi54qRm/E1v+GP2d/iN4q0OHVtO0hmtLhd8Imu4oZJV/vKjsGIPYgYNctBYX2mzXNrq
EUsF3DIUkimGGRh1Fehg82jjans1BoTjYmZtwpm7v1oZutMZsivdsyBGYdutQnliT1p+4A1Ex/8A
11NhjGYZ64opjce9Fc8lrsLU858OyYvSOp25rqkzwetcdoMm2+XjGQcV1qsdtc3DrvhGvN/oaSZZ
VualEnbFQR+/WpF6+9fXIzLStjjtUyt7ZqurcDFSK2e+KpK4y0rBafuyOelQK3HWnbqvlQFlHHzZ
GTjj2pVfvVdW45p27jI5p8orloMcehpysGyT1qtu6ZNPX5fpVqIXRZWTjHenq2ODzVYSe1KrDvWi
iZlsSfN7Gnh8NVXeeaekh4ya1USblxWP3s0/ee3Sqat6cGplk+Uc1ooDLqsWA9alSTa2OtVY2+mK
f5gFWogXPMHGTmn7stVZWBNSqw4Ga05CdyyDtHPNSqy4ODVZWDY7CpVYM2OlVyklpG+UVPHllyRx
VRGyAKsw724AJP8AdA5oce4XJlYY65p64zjvUC+p/KpE+ZhnrRyCJ/5U4Nx61Hv/ABppbHPenGPQ
pMfNIZB6VCcBSRz65pskmeOhPoKi8xRwc12U6aiVdE0fLEbsDGeaRpBtIHWoPMUdM0PJuxgV1KAE
of5TzSK5kHPUdO1Vmky/FSxyFuDzWqhoLmJHyufbrTWY9+fTinq+7vTSwyeKfKhNixltuDgZ596Z
vCsVxzjrSFtuCOmac+D3pcorjdm5Tu6ConQj+Kng7eAM0rfMozxT5REX+rxuG4fyqN1+b3xxUqru
/wBrFS/I0ZGMPu4PtirsNFVY8rkDNSxx8DgCnL6DmnBSe3PalYQnkrkAhT9RUbWcD4/dI3/ARVkL
yOKcVKiuedOL3RabsZsukWbAg28Wf9wVXk8P2DH/AI9Y/TpitWTczEnknnJpNtccsJQl8UF9yFzM
x28Mad/z7rn2JqtJ4SsG+7G6/RjW+F+bHf1pGj3f41wyyzBSfvUY/ch8zOZbwRaTMTvkT/gVMbwH
brwJ5PyGK6xEyvNL5eCBiuGWSZdLeig52cj/AMIa0eDFdsjDocc/zpn/AAi99bxvHFfsqSEF0UsF
YjpkZ5rsvLDKBUZi/KuaXDuWy/5d2+b/AMylNmPda545k0mTS38V6pLpzLsa0e+lMRX0Kk4x7VT0
fxJ478K6LqGj6Pr15YaXqAIubS3uCqS5GDkfSt2SIGqzx+lckuF8ul0a+ZXOzK8A/EL4gfCW4vZf
C+pSacbsATqAkiORyGKuCMjsetZ2g/Ejx94T8eJ4wtL+7l14OzSXFy3m+cCMMkgJ+ZSOMHituZfQ
VRkjLL0rF8IYJrScl93+RXOyzr37QXxN8QeNtI8Ri7ksr7SSPsFpp8Pk21uAOVSJeMEdR3zS/Ez9
p7xr4601dKg0+z8I6YZjdXNnocDQLc3BbcZJCSSTnt0FZrRlTkH/APXVSbLZzz61zS4NoX92q18k
HtPI7u5/bS15dDW5HhyxX4gGzGmyeMmyblrcAj7mNokIO0v1IFL4d/a8tLfw/wCHpfE/hFPEfjHw
ujR6HrEk5jiiXOUE0QHzlDypBHvXnLKn93PccVDLDHPITIiuzck4GTXFLgyytGt+H/BHznpXhP8A
aqtPFHh3xD4S+Kltd6p4f1q8Op/bdHRI7q3u853AEhWQ8jB6cYroNL/a+0rUfFV5o+paRJpPw9ud
DPhqFbNjLd2dqDlZQT9992Cw4yOBXh39l2jNhreMD/cGaik0Wy3cWyc+1cM+EMQneNVfc/UOddT6
B0L9q7wx8KdY8DaF4S0261jwn4auprqW/v8A91c3k0yFXkVBwgUE7VOfc1NdfFD4d/CuHx1rPhjx
FdeMtX8ZwPbJAbZrYWEDsHcysw5kJyAF4wOtfO/9gWbHmLH/AAI0L4csm5CuPo1cNThXGR2lF/N/
5D51ufXl18WvAcHxGsfjX/wk0Exi0hLW18JohN9DdCExbG7CMfe3984xWFrzad+0N8M/AkeleItI
0ZPDqSRava6xdrBLEWcu8yqTmRTnouTXzL/wjdtjO+QCn/8ACMxNgLMw7cgVwS4bzCP2U/mVzI+z
dY+IWj/HbRfiV4V8E3SzaxeQ6dbafb3WIPtVrarhyhJxndk7Tjjn2rS8KeKdD06bTPh7pmq2uo+J
tI8HXmlRR27gxTahM+6SJJDwxxn6ngV8SR+H5I2DR3TRsP4gMH+dLHot3DOJ4bspOrbhIrFWB9QR
zXHLJMwg7+y2815efkPmTPq/4Y6bF8IfhXd6L42lh8O614k12wAsrxwsy2scoLu6dUX/AHsVuXVv
qWk/Fz41eM9Yt5LLTZNPubDTZLogRXMswVYkj5+fK8jGetfG2o2uqardG4vbx724IwZriVnfjpye
a09W8TeLdYtdNtdQ1u9v7fTAFs4p7lnWADGAoPToK5XlmNh/y6fcrmR7nrWoW/we1jQrz4hSah4t
8b6Xbwiz022mENvpkSDdEkj7T5h5GVHAHcmvJvFvjS7+IHibU/EV+kcN1qM7TvHCMIuew9q31/ab
+Lv2U29x4mkvYSNuLu3gmOMYxlkJrhFZ3XdK26RiWbgAZPJr3sjwlWlVlKpBrTqiJMexyPSolOc+
tKW3KT0qJmwcV9nymem4pbGT3qJmP0odyG46VG0n94VHKFxCxFFJu9Bn8aK55XTsXdnmmijytQjD
DJJ9a69Olcdo/F/Dg98V18fy45xXk8NP/Zprz/RDkiZW7mp15UVVXlqnjbFfaEFiNhtyTUitUCt3
FSK3XBwa0WgrkysKdu61ArY+tPVs1aVybk6kY60u72qNWGcZOKUdflNapXETBsYzUoY/WoAfl55N
Lu6itBFjOBmnBsmq6yE4Bp4Y9vlrRRAsBhjrTw3AAqupxT0l59a1URaFldvHzc1IrfKKqn5WxkN7
r0qRW6AHmtVEhlyOT3xUsbk4qkrHPJ4qZX44rXlY7l3aYyQeCODT45O559KrCX5cVJGwOMVooiZc
ifjIqdD3zniqSPgdc1OrYxVcoi3G3PPTtViJvmzk/WqatuarUbY4zScLgWFw3GalXgjHSq6tjn8q
l3ce9PlGPZvSmMxxzRuqNicEdK2jC2iEJndwD9Sahkb5evT9aWQ4+lQtw2DXXGNtQHp65+lKztnB
HXvUJYdetM3fMecfWuiMQJt554qRG3YwKj4YAfhUkUe1uO1VawEir05zU6hcDI470yPCkA8qfSpf
usQoIUjv1qWgGnYFIGdvUe1RMoP0+tTSLs6jDUzbmhRAYIsgljgdhRgdhxUvl556GlEOOetUBCv7
vcAoO7g5GaQQ55xU+0lh29qkCbcE0gINm7HGGHpRt/i5yatRR5bkY9KHjC4/SpYEKpgZI5oboARU
jZ6EcCmlhWTiBCymkEeOKczHB7/0pyrkg9ayaFdAI+/FJ5PzZB4qTgcUobpxWLQyNVwDml8v2qXb
z04pdo/KswIfL4GKYy9qsN0AqGT73rxU8oytIh2mqzRFe2TV1toU1Wly3U/hS5TQoTR5aqc68Vfk
wc54NUph19KsDOnXoapyKNp7H6VdmHrVKX72Ogq+UCqw45qI7W47ip5MdutQY29BzUuIpCN2JFM5
J9qcQTkmnbPlyDzWMkQKvzdf0qVFC8Co1HtzUsZ56e1cU0PmJU/zipAexHSos8jJzjoO1Sq52lMD
BrkkikPLfLjNOVsetRDgdcUFj05HpXO0Mfux0PNN3d6bvz14pjMaxcbgSbuT7Uwtk01WGfb0prvl
s1i4hcGYjpUZk46UrN2zUbOB2zWbiIR5fWmO34U2o2YBjms3DsUBfng0VERuORk0UvZpged6e3k3
0AK4IYZFdkrK2M9a4izbbcwkjqwrtU9D2r5Thj+FUXn+h0VCRTzjoKk9OTimL19venY249K+6juY
7osb/lApwbr6VCvc05VPUCtUQTK3OQM1IDVcZHXipBWqQEw4xjk1Jk/SolbHNBPfvWkUBPuxwR+V
OV88dDUO4nr+dKrd+hrVICxu6Ubju5FRh+eeafuz1rZICb+HIOM+9Ckg8H8aiDVIGx0NaxiRIlEn
Jp6sePWoVPenq/txXRGPQksqTke9So3qeBVVG9+lSxsc5rRRAsq+7npU0b8jA4qsvzD2qaM45rVR
Asq25uDVlJOnfiqcJw2amRj2570+UC5GfrViF+nb2qnHlqsJ3HpS5UBcViwzUqt/+uqytuPBzU27
GOhz70+UBxbdUcjdqNw7Uwvye3tWqQCM3QAnimFd3U5NKuG7nml3dRjJrpigI9uM+vWmhdrcjg1I
wHXPOaGet0gFT5fr2xVmPDMxbknue1V1XueTV6IcZIxQ13ARowOnB71IoJ4xmhlDdaeqlV6jFQA3
btwD+HFIyD8aduB4pSvpxVoA2+uCOtHDNzwvtS9VI5xjvTliyQd2TS0WoBHGvPNOc7j/AEpdu0e1
MfOOagB6thfbPXFQzS8c5/nT1bsTUbdf5UhDFJbAwSKe2CuAMY96bywzzS/hnFJonmEj47Aj3o+7
3peepp34ZrJq4bjVX160/byCDkULQfTNZtDQ5cikGPpR0GO9NZuoHHvWTiUJ/KmN70biGIPSkdu/
ap5QIJc89sVXk+7U0jd6qzMdwGePalygV5Mc8kVTmO2rU0mcjrVOZhyelNI1RTmX5icVSmz1zVqV
s5HOKqScj1q7MTdiA8ds1G3X0P5VK2cUzbuB7c1EiG7jGA6Uix46dKdt+bBpB3rGQDlXB5HHpTtw
3ZxtFIo+XOc0p7dxXHJAO5bkU8NlQBz6Go27Y7UbuOK5ZoCTd82aCe/X60zhue/pTQ554rDlKUu5
JuG6hlbyy207c43Y4qFmJxSM3GMk1i4g2O3Go2YenNDTErtPT8KjZunftUNED920k43VC3NLuPPO
DTN4XOOc/pWbiO41mI4PAqKRqdI55qFm7dqOQLhuUdTg0VC7ZPA6UUciC7PPrX/XR54wwruo13c1
wsDbZFz613ds4CL34r4Dhl6VV6fqdkvMfjGQKXkdRS47gcU9VyOK+9iyBFwOfWpMnimbTnHanqOP
euiJmOXvnrUi8r1pqAdzkmpFraIhT2o60YNG30rWIC8fj2oX5m60DHejt1raJDY9W+Y80/8AhzUQ
xxnmnZ9+K2iJaEintUg98+1Qhjt46U9W/GumMR6tkw+XjNKrcg1EGI9qkVh17Vuokk2cdO9SLIRg
8+9QL9cU/d0x1rdRAtK3ocVNHL6GqatySamjPQDirUQLqtyM81PGffFU4yVx61OjfMT39KrlAvRM
PxqxG2BiqatnAHFWFP4YpcoFxZBtAxin7tuAe9VkkHepdx+tPlYDpGA4xzTN3XFMYjPTmmbuCBWs
YgT56c4obGTgfjUG4il3dhXSogTlRtAPPem4yOlM3BuOnrTwxbBH41qo9RE0aH2NX4lyOTVW3wMe
tXY12/SoluMVYx3pGUcAU/pUbHLVIBx1xk0v3mA6+1AXn/ClHXjp3p2YDxnbnvQqnkDI9cUisdxI
6ZqVW9O3epYCSY44x2qvJuYcVZly3zY3c/eqBmbbtLHH90VKAYrAg8YpWyevFJ3Jo3evQVVib6Dl
bbyDn3pOMnBzThGphLGRQwYAR87u/P04/Wotx6fyrMlMXaeMVIq7eO3c0i8qPWpVIAct1xwPWoki
kupHtP4UoH5dqGy30obGPeosMjY8570wtQcZzSMC+TU8oxrP2zUbnuOaGOKZKwCrg896XKBFIx6V
Wkz3qWY/LVYvjrijlL0IJm46dapzN1I6VZmY9gcVRlO0+9LlGyvMc1Wk6VLJl2wPWoJs8+1DViPU
Yynk9h780zdk9KVmI680xvTODmspIduqBj0xilX1xmom4brnntTixJrCSJH7zz60uc/Wol5705cb
sDmueUR6sfgntijaeKRpBSM25STxXJKIh3TqaViu081Gp980D5uc/gTWLQBTH69aVmy3X2pjtyB+
NZNXARm7dPemFsdqcfu9KjLZJ/Ws2gD73Womb04pWbcq+wxxUMjHoKmwDmbGT1qFmI56e9OZxt96
jL9eM8d6XKAzdzzRUTEsetFKxXKziRGVYEH3rurNM26MeflGa41IdrDFdnZL/o8ff5RX5jw3P95U
XkjtkT46c0uzPvShflwelSKvHNfoEZGQjL07UoBBqRQMcinAA5Oa6osUtiMLge9OVT0xQv5inbWx
7VtEzGnO0880c0qqdwpT3FdCExMEjNJ3wad06Ck4I9K6IiSsAOKcANw9KOPb60gw2Qa2irhbuP6Z
7jtTk9DTFHHv70obv0rriStCbtntQrEdKi3e9LvK4reIiyPWlVxx7elQbu/3aUN3UZrcC1uzU0f3
ulVUbv2qeOTaM1qBcjbJxnAq0nBwO9UI5NvX161ajky2aAL1uCzBcgZ/iPQVOpIY5OaqRsBnvU6s
WXjimBZVhtOQMdqd5gUDFQbsAUhbPXpVqIEjN2zyaj8w/wB79KRn9fSmMw6/pWyQE28dqcrdABxU
SZ27RjaTnpzSr7dK6IruIm3DJqaNdvJ59qhGOO1SqcqDnmtRX1LEbcZ6Grkcg7/lVCNvXrVhOfas
pLqUXBJ2IzSbtv8A9aoFbOfUVJ14rPlAlUHAPf0p3vmo1k4AAp6tngcc0AODdRmjds5pm4qenFBb
PXpStcEOZi31qJjtalZtvQU1mIbpSsA7dTDluQKRiOnWjk0jNi7ivPGTQHG4EdaAvBJ4pm4c0gJt
2080/llz1qLcOMDntT2b5V2nnHIqGjQXleg5prDbyRR5nUU12+UDNKwCblZfSoioZsFtg9etIzDn
NRs3PBpND2CRgpyDuweOKhdmyc0rPyahYnHFFg9RsjfLVZ3A4NSM3XJqrI/4ik4l3QyWZt2Q2D6m
qcnfnJ71JI2W981BNhTwagZWlJyD3z+VV3kBzuzjFWC+GBCg45571VmbcxJG3PPHSkZDNwPGeKY2
GbgkU3NN8z27VDQwbjGP0pVY4x1pm7jGeaUnHTpXO0If/OnLjr3qMtlaFbmuaSHcdu655xS8c5Pa
mZC5pPvVzSXQQ8tnoOKa5O73pDnGBzTdx6GudrsAMSvWm7i3NJuyDmmlvl9DWTQAXJ9qazDpSN8v
Oc0zPeswFZtv0qFmJPFPfrjpiopHPrS5SraCMx6dDUTEg8GnbvXnFQs3NTYkOdxwaKbuPaikVzHN
rHnpn3rq9O/eWsRJ5xzXLjC5Oa6jS2/0GIgdq/JOHXavNeX6ndU2LqgnH9Km2/mOxqFW6VMuOp61
+hoyHKtLgelGRTxyK6IsRGycUZ3e/FKzY+lIGCrxzXZFmbFC96T2Apd3rxRwK6IiGY2nFJuHHY0p
xn2qNvlPWumJOzHZGcdSKUErUZz1A5pVaumIIkLZ9qUt0Heo/XJwPem7s+2K6okE27mnjJIwagHN
SKx989q6EBPu9fypVYVHu5PPNL7it47gWFOVBzUkfzfSqysDU0eBzmtBFyNueORVqNg3TrVKPj6d
qsxnb0NAy7C3WrKt7c+1UlbHWpQ+1TiqsBYLe1J5mOlQiQ+uKUyVrECbzMYJ4pm7cc4qLeG5/Sgy
EcCtogWN3cVJG21fbvUANPyO341rGxOzLCtwD1FSq3T3qvG3QVYXjkitR2uSjcMHvVhTgAfnVcNu
Uc1IrbcZHFST1sWVbANKr/jUasaVc/SsSyxuPI6mnVGvQU5WPA6UAP3c80N9z3+lN4Gc5pS5x7Ug
E5288mm87uTSrj1+opGb5fbvTEI3yn29qaueg/Cl3EKwHAPB96TfgcDnsaixMhWJ49abwM9c0m7g
8U3cR1p8pS2JFb5f8adu28/hUORt6UbvX8KVhkjNnOOPSmM2T6UhZdvvUTSevAqQCSTnk1GzfL6U
yVsnimbjtNK3UBWb8aiabapXpTif3ZY8DNV2bLE5pND9Rsj7qrSNtJ4OKc0nXr+VQyMe547VBehF
I26q0jGpWbb93jmq8rfe3dKCWyGRvmOORUMhz05pXk2ngVH/ABEkjHWs5EjZmC8jjPWod2cmnyN1
BH496gZ9vGeKiWwD+MAnrT/uqOhB5GDUBPccU7cFXPU1zyAeX/On7uxxUG7nPWntxyDXNIB7EU36
1GScc05ZNq4xn61zyAX3pvmU0tjOabvFYMBGYims272oZs+9RllAFYvQBxfGRjNRs3ShmJUmos56
1kwHMx9cimvjbuz83pTW6dahbhcGkMRpNtNZ/wAajZvmpG96dhrUXzMUVCxOfWipJM4KOPSul0v/
AI8kHpxXNBsj+ldFo7FrVOeM1+M5BK2Ja8v8jvm7ov8AGafuK4yPrTOnAp6ncvTiv0aLMHpsPyWH
NKrdOM+1Rrt6UoPvXVEVyQ4cnAwBTfumm7vfFM3g49a7IEkzEe+aY0h6Cmbjg5ppI9K6ogPLUjMO
OMVGZPWmeZnHNdUUZslyfxpATuznFMB+bJPFIzAD3rqiIl3H0waTduqLdnjvTieldEdgHq1Sqx4y
cCoBhu9O5210xAn3ggZ/OnBz3ORUGe1Oj+tarci9i0rDsM1KjH0wKqqx2nFSxtitB3L0TDGQMVYV
jnr161RjzjFWY2Pc0yi6rdMVMsh6HrVRW3DOaduweKtAWN23PelV+hNVy4/CgMRxnPpWyVgJ/M7i
m7/m9qZux1IFKvUc1sgLUcnFSpgZ5981VjIZvap87eD+dMCyjdOOKsI2eKqIwqYN0IouItKB3qRC
d3Tmqyvn2NTLJ1H50XYn3LCsNwANP5DYPFVgfxqRWAPNIZYDdOQak37eMVXyBjBo3cYzQK/cn3HN
G45wKiDGjeeOffFBRLk/rSN93rmmbyw9OKaxAoAR2PA7UokPPeoGct17Upbj39KYibfn6UhbPeox
JnHOKTcB3pASq3HPHpTd+cA9KYcNmkZgOc0nsMkYDscetQykq2DxTN43c5A+tRu3NRYAZ8YGaYzZ
zUcjc01mGMk8+lIBWbdkVAz+tO3npnNQyE+nNADJMZNQvJ1Bp0nzNk/XrUEjg896AIWbOewqvJJ2
zx70+ZzuPNVpAWqeUBJmMeBVczg5zmiRiTjP0qvJJtPvUuIEskuVz+lQFxTd9NZtoBPAIrJoCXcM
Z6+lN3YznnNQ7+AOopfvHk8VhJATo+0A9T705m3ZY1B056j1o3Ht0rlkgJj0JHSm7tv1pjMex470
0sK5pbAPZunrSbhu68VH5nvSbvlNYSAczDHoai3ZYjpSNjaSaaW7ZrB7gOz701vTNN985prPyTUM
BCQO9Mkf35prN1FR7vxNZjEzzjoaOg5NM6t1pGbHegBwPviioJJDu46fWig0M3dnOOldFocg+xgY
zg9a5ePB46f1roNIlH2coDhQeDzzX4rkjti0vJnXJaGx5lPWTtnIqrHJt6805ZQM56elfpCMSwzY
6c00TbWHaqzSHGRyKXzBuGBxXbBWMy2s+FPQn3FMaQbjVdn6kGmtMTXXFCLDS9PWmMxPtUO4NjNB
l7d67IisS7jt5PFMaT2qIye5xTQ3INdMSH2LHmZ4pGYcVD5mOccUbstnNdUEIm37jz2pdw781XVu
TShucZrqihFqNsGnbveoA2MU4c8961W4yzkFQM5p69vSqq/XipR0yDmtUK1yfJUdeKmT7vXNQRuv
brU6f5xWgyzG3yVOpPGKqodoqZXHUUAW1YLipFbPPWqYk3fSpfMO3rxWqAsMwXrQslVfMLdKerHt
xWwFoNnjr2pV5qBXx1FP3jpVp9ALMbFV4ODVhW3AVQDndkdamWY+9UBcXjGDU27jmqaycehqVZqA
LcbDOByRUm4t7VVRgz46E+3WpuVbB5xQL1LSt1zUiEt3/OqvmZxzgVOJNsfGCc5/+tQMn3Be+fSk
WT5snkZ59arGUtzmnKxx70EvXQtSvGGPllivuKFkC81WZvfn0zShh60FFjzB0Peo2cq2KYzenSms
3OaABm754p+4MvrUG7kilVtvAqo7gT7uDxmml9wAzUW/bmk3DmhqwEpbaPSoy3zcmm7ty5zTWbcc
ng1IAx3fzqIyFuRzQzHNQk8+lJ7AEjH8KjLZB55prvyajLH1rMB+49c1G2W5P6UhfANMZuKAGSEr
VeT73pU8kv7sLtA5znvVV2PPrQBHK2fpVSRivUkCp52CE4O73qlLjrzQA12aNuTiq747mntJnr83
vUDNu+lTIAbqabwWwaYx3HjmkbP41jIBd3oKd/D15qPd2PHHalDbqwmMcrheDUit1xzUXAXk0qsF
7cVyyD1HdO+B6UbvxFMbqe9G7rk1zPYLDmI5xTCc9Ka1M3Felc8hDy1M498UhbaMimbi1YyAc+PW
omyOlOZs/hUTNuwc81DAZuO054pm40pJNJWQDGbbx3prNnGaVsZNM60AIzegoqNsnviigvmRjmTv
+netvQph9ncY+YNXNeZ0JOK19Dn+WQE856V+J5M7YyPzOuWx0Pm0hk3N19qp+cT3o80nHNfp0Uc5
eE3HJ49KUzZPuaorL8vBqWOXam7POehrtiiXctbRlSz7OM5POajMvfrVZpstx35pvmc9c11xEmi3
5hakaX3wag80UFvfiuiImyypDHqBSM2GI4OO4qDd+lIW2qMkk10xJJhJ8vvR5nqOKj3Dkjk0GTc3
JxXTF9AJd45pFbDcVFuP0FLuPPPB611REWN2OQacr9M1WMnQU9W6c81tFikWQwDZzUqt0P8AWqgb
pUkbc89a0Fsi7HzzVq3kRWw2dpGMjrVCNsYHU1PG3tTLL+zKl1+aMHGcdKVTz+tVVk2nHapVk/Oq
QFoEtinGQc/WqnnHv+lHmZ61tHYWjLayZ6c/SneYOneqXmdMcmniU/hW6BlxZSB1pyydutU92acj
jd6jvWlgReWTBz3AqRZPw71SWT5vapUfd3qhl5ZOBzg1MjfnVFZOnNSLJz7UhF9ZMkHODU4fceTk
4qik2cdqnWbmgZb3EL1zT1kB2g9aqiX8qesh4IpX1sBbDdaN+37pyai84c84J4pu4ZwetMCz16d6
cG25B69KrKxY+9BkJz3pdQJ9244zxQW3ck81FvPBxTN5GcjHvTFbW5KZMHil8z5j+lVyfmpdwHQ0
DJ9wK5pvmc1C0nFIrdu9VfSwEzPweaarFuByfSmsu4Dn8Ki3FeetSAryHBHJK8YPaoGk75p00u7n
HOKqvJ09fap1sA7zM0oPBNVg25jzQJip3K3PWoAkkkIX1qNnyB+VRs/BPXuKYH7UASHZtYsx3duO
D9arSN6c0kjY/rTCwCtk89qQEch3Kfzqqxz05qWRht4NVS/4VNwGyNt6VCzndgdaWVvSq5zknrSd
9wBmKim7zjLd/WmbjzSZ4NZyAdnjil6VHu7dBSGTGOSKwkxXJtxbilyRwRUG/bjineYc84Fcshkv
mYphY01m6+tNYnGM1zSYE24bRzz700N8xzjAqNmz0pM7q5pAKZMqR+VJmmlx0xSHGMk8/WsWAO3X
NJJgKNv86a3Q5qI1DAczfKcVHvobOPambh071mA1mHam7hQ365pjcc96ABv3hyOKKYzHscUV206U
ZRTaGcx5nc8VpaJIN0mOfWudkm659a0tDuAJJPpX4Hk7tjYHbLY6b7R6GkM3XJ4qmsx5pQ2Wziv1
WJzF5JRu4NSMzDvVDcMnripRcBto2gADHFdkQLRVvvclaarBhkHnNQLcOMqCdrdhQX28Y5710xRm
XFU+WHIypOKHbAHeqyyNtAzx9aXdhsiuiIiwGPrT1bpVfduxilSQDJPXsB3roQE+4DnvTRITx1qH
zNzUpYKeDk10wAs7h69KaGFRBuM+vNG4d66IgTq2c8c+9O3fjUCtjvT0Ytwa1As7umKljbpjpVUE
r9aljb1NbAXFb5h61MsmMnvVNWG4Y61KsmMZpgW1k61Irde1Vlb86VWPTNWmKxZVvXrQGNQmT3pd
34VrFhbqTrJ+HanKwbv9ar76Fk6VsmD1LStz1p+7b0quueMelOV/yqg8i5GRj1FPDYUc1W3YwOxq
RT61XMLZFpW6elTCT8Kp+YTjHFSLIafMPoXI344qdZBVFW4PNSoxXuDUt3GXhJu68U/zCoAzVFZK
kEnvQBeWXPGOaes3zcnFUN5xmlWT5jVgaPnbaRZOv61U84sP0pVfbTAueZ7U0tu71X3+ppGkPrjN
AEpk+Y85pd2BVTzMt1yaf5nvS6agT7j1NG7PFV/M/OkL+hpXAmWRuVHFMaYjv09KhaQrURlpPUCa
Scmq7yfnTGkK9OlRl6QD92AcHik3BuKhZivFIxP1xSAk3HPqKjkPz8HjtUbyDPy8CmMx+p9qTdgF
kY1A0hx7UO5zyaikkPHcVPMA1jk8nAqGZh03ZHr60SNu5qrI+DjrUgPeTt+lQM3Wms2KjZivvRcB
7EcntUbOORmmN7mms23tmsmwJcjgY5pF61CJPxpdwH41zyET7getBcc1CZNwA6UeYOnNc0hku89+
TSM38qbuLe9JnuO9c0gHZG0UlNUFmA6DPU03ca5nuBI7e4qNn4x0pCeuaZuGec4/rUMB24c5NIzY
qPPvSM4bjpWbtsApYn6VFuxRmmFscVmANuY0xmpGfPHSmMwXpzQNClveiombng0V6VH4EVocM0mR
1q5o0h+0NzxisrNXdJbbcHHXGK/nvK/dxdN+Z1y2OkWQr0OKkExAGGqqGPc07OCMdK/WInMXFm3Z
XtT1k2+9VA5wAO1Shvl5rsiBZWT1ODUnnBlzk57VX3ALkn8KcjFfpXRFhZFlSemacrZwOtVy2ena
n7s+xreLI2LDbkIDcZGRQuUGT0JqAN0zk0b8966IskmaTDnA460JJyar7+vrUirwSfSumLSAl3Gn
bumKi3elLnpXRFgT5p6yYquJOABTlcgYrZMC0shIqZG9Kqq3TFSB8VYFlWPapVk6VXjanhh1q0wL
KydR2pyycZ7VX3YWk3H14q0wLnm85zSeaaq+YaesnetEQ99C75ny59qFbd9aqeZThIenetUUi55n
bPtT45CvvVRXxjnJNPVuwq0DLqv6nFTCT5ev4VSDdM1L5hJ61Qywjn0/WpkkK9elVg2e/vUq84Pa
lzAWVf0qVM9KqbwO9SJJhgQaoC39TxmneZu6VU8ws1S7uPagCdX9Ker/ADVXWT3p+4UAWfM5wad5
h2+uaqqx4OKeG/AUDSuWGb1qJpM96Qn5cZqNpO3ancB6SHd1qQt0NQRsO1I0m0tzQIm8w460jSfj
UDSE49aN25sdqQE3mHaecmoHbtSNJjIHWoHY8jjNAEnmdqjZsLxUe4jjNRyNuXrQBI0nTnNDSHua
r7ugpjSkVL1QFhm9OtM389eag83PWgN8pJPSoAczE1FJIBxmk8wZ5HPrVeR9zcUrgDyDJqu0gx05
olbd1qAv1J6UXQCsxPI61Czk9e1LuY5K9BUMkhZsk5NQ3cBWk+bNNdy3SomYihmIxis5APz2IoX7
vJ6VHuJo8zpxXPJgTrIKNwZqh3bWJpynvmueQE/mY4703djiombvnNOLblHP41yy3Ae8hYY9KYr7
mprE/Wk3VgArN3prNz1ppbPsaRmGR61mA4sF6VGfQcijcOaYZDkgDPHeodgDdjvxUeWYnApdxPBq
Pzev5VAxzMOpqFvmORxQzde1RFjyahsqw7cMnIoqIzbeKK6IYhQiosWhwRfaeuKu6TIftAA5FZu/
d7mrelsVulr8GwLtiYNdztlsdMrb6lYKsakMCT+lVFb9ak3c4r9Wic1iyp4yeKcrccVDuDdelORu
4rtg9BFmM5+tSK2ODzVeNgT/ADqXcS3pW6YEjSZ470ob5sfzqL+LJNLz2reLFYmEn40u7PP5UyPG
OTQWG7+VbxkBMuOnfvT/ADAevSq+duOPxp241vFi3RNu3dKd1FQhulOaT0rrjIgk3HtUm7pUKt0q
ThsVtcvlJg3PHFPViKh3EYp/mfKe1aKQmrFlXG35hz+lO3jqOKrK2R149ak4I681YrFgSErQJPwF
QB8Nilzk5rRCJg23rzT1cdarZ45p69K0iBYVqVZDxg81Bu7nrT9wq07AWA2DT0Y5JqsrEqamjyav
mAtLJuBqSNhyetQx4zzT92D6U7gWFk71KshAyTxVZfm6U7npimBaWT3p68c55qqrbeKkVuxGaq4F
rzR+NP8AMPQVTEm1uelTK47GnzAWVm596crgciqobnrxSh6OZDLolpFkO7rVbzOh6UqyAtS9CrFo
yUjSDt+tVzhqRm9Km5BOspXvS7+Mk81VEny+vNKz/N1p3AsK3XNIZAq88+9Qxye/40wtuyDSbAnZ
s429aiLdu9MMm3oKY7Bj1pcwxXk3Hnio5HGBzxTWk6kmoWbFHMIm+8M5qOQhelM8ykdh1zS5gHL9
3IprMec80zzMDgnNI0vTJqLjsIWxnmomYjPpSNIGyO/0qJpPwoCwyRvlzULOGJ5pzSYzkcGoHYKS
B0pCGsx554qN22j1FI5BpjHHek3YBGkJUc803jJzTZGNIG+X1rJsB7NjoetOjY1EzdOelOV2bhRn
1rCQEp4bA5FKrelReZSjPc1hJgSmTb160bvTgVEzbsDvSqT9ea5JDHZxzSbh9aY2cnNNyKxbEOZu
4pofuaYGI60m4tWbdikK0m1uOQabv5POKRmHr0pssm5Rx0rMOoM3cnrTC3GQaa0ny81H5ny4zmpY
9NkPZ+OTz3qFm568UM3X0qMv6Vk3YWwMBnrRUZk20Vzyl7wfI4IN61b09wt0hqnnj2qaz/4+I/rX
4rhXavB+aO92sdPHlue1Tx7f4qpxNtGAamZumK/VkzAsI3XFOWTbn3qDcNvGc05W7V1wkTYsR5DZ
/GpQ3XJxmqu4888VJvU4IGK6VIkl3ZHXipFJycnFQbWkViASF64HFOWQj61vFgWFJ5NKrAHpzUSy
diTilzlvbNbphp0J9/UZGDTly3Q5NQ479M1IrY6VspCJB705evpTNwY46Y/OhWOa6IyCxOq4wacp
4qIOe9PVicDPAreMgH7jn3p6+9Rn5sc04ZX3rZSJtqTbjj09KVWO3BNRfzpY3/AVakD7Ew78ZpVk
21CH/KnbuRitUwaJN53dcU9SexxUG7tinb8d62TJJs/jT1JPaqysSalVWjba2QavmEW4+KkVx3qD
zCzc/TipB096E+5VtCdZM9+KeG6Gq8ZHepgR2qiSeL5m4xmplcFTzioFYKoweaXcOvemBYVtw96U
7t2OlQLKPXn0p3mE55xT5mBP6U7zB0z9Kr+Zu9RTt27GKfMBZ69T+dODnvzVbcVJzzzUm4d8570+
ZFpE7MPxpGbA61Dv+Ydqd0b0p3QWuShuOKGY7CO/tUasTzmmluuKlsnUk3HBpd3HNQ7t3tRuz1pX
0HuyXzMfX1oLEKD0qJmwDTC2AfSpKsS7s9DUTSe+DSF9tRM27nrQMfu4NRNJ+dHmbc5HGKiaTkig
ViUyDbzyKZ5m3OeKZuPTPFRNIfqc0rognRiT9ajZgW6mo2kJ9qZuB60Nmg8sOpFQu/FJJIaj5kyF
5OCevpUXZL7DS2DiombbnmhmwvWomJ69anmIE9cnmmM3PXNBk+b1qFmqXIBzNnimk4FMyTgd6JHb
aOMA9PesnLsOw7IYU5SQo7VAM9vzp7NwD3rKUg1JFYr71Kk3riqysaUN83HPtXLKQic9803cVxg8
U0kqoOabu5rmcgJGfdg5prNn3pnDZ5phbrWTY7Em4dM0wvknFMaTPtSbsL7+tQ33AcWI5pjcdRgn
tTCxHQ0jSHcSTmob7FoGY456VGWpzSDBXFQZ61m2MViBgd6azelJuHemM3WsnIVriMc+h+tFRnNF
ckpe8TqcMDlamtGP2hO/NQVJbtiZfrX4zRdqkX5noyOnXBp6+marxZI68VID3r9UhLQwZOp5qRfl
xUW7Ip654rojIknDZ6dKcrAcVHHj8adXTGQyUSHGAeD2py9RUa7Vwe9O3c+ldMZCJVwtPXnqfyqJ
R707pgYrdSFyosc+nFKMD2qHzCBipPMIU8VtGQWH8HIpyqQoxTFZdo/vHtT9w28Hmt1IWhIFOQDw
akXGar+YWOWNPTPB9q1UibE+egHelHueKj3Et1pd35VsphYduNP3d8n2pgxj1FG7qcZraMriJCfy
oB9aZu47Ypd341vGQkh4anKxzyKj3fL709WXb1IbtWykMk7Ej+dKshznuahDe2KXOTzVpi9C2sh6
d6eJM+1VFl7VJGxqhlxW69hT0cc81DG2V2HaBnO7HP5+lH8WAeKrmFyllZMLxTjIVaqzse3UUgdj
yeapMmxbWQc1MGBGd2PaqSsQOeTTom6kcUyki75me2PSneZtqDzM8flR5g6HkVPMJFpXyRk09XDS
KCwUE4LenvVXeOPSn7gKOYosthWYBtwzwfX3pQwCjJ5Paq+/OKXcT0NLmAsq/Az0pGY4B6VAJN3G
KTzO3QUuYZNn34o3fMPSot35UMxXpS5hEzPyecVGzAqB6d6iaT8KY0h5xUcwEjSeh/GovM603dTW
Y96fMArSfhUe49QPrSM/+c1F5h6EZpcwEhl7D86b5nzZNMLe9RtJvYnoaOYCVpM8n60xmBx1pnmb
uMU1nzx0pczAez5UjHPaoyuKYz4xTGb5utK4DZOp5qPd8w7jvSqw55z7GoWk7djUcxDQm7DHnGaa
cE4JprEt1GKYzYqb3HbQk+WNcDnNMZsdaj3d6aWG7196xchJdx+70604cjriox3xTlBIP93vWMpD
RJxTh+XvUe7nAXH40iseRXPJlWHs2TzSN0yDzTV6805sba5nInlEGecmkU7c5pNxHXkUxm5rJyKW
gm75uelKZBz3qPo3FNZu1ZuQhzNnrTWbA4pp+tMY4FS2MVnB6mmFuOlNLfNSM4FZ8wDS3PTimknv
QzduppjSFvasXLUBfMKniim53ds0VyykrlWZxAOfanw8SLj1plOQ4cYODmvxqm7NHcdFGx2g+1Tb
gFqrC2UXPWp93y4r9PpyvFGRMrAJ709ST71CnIqRW6DvXZGRPKTLj8akVqgXOak5rpjIVmTKehFO
V6iXPHFOXoO1dCaJJ93TNCt6VD+NO3Hb6cV0KQE6tnHrTw3y9c1AkhHOcUob0NbKQEysN2akyOea
rl8dKC/PtW6ZL0Le4DpyKcJumKqiTjj86dv4yTmtUx9CdpaVXJXk+1V/Mzn1pRJ74rVMhlxW+br2
608tx1qoGx3/AApyyVqpCLGc4HWl3HvzUO/KjFOWRc8nBrZSsImRvlIpS4wTioTJjvzQrBupwa3i
xkyye+KcrD5evPXNV1kC5Bp/mD0zWqkIm4X2p6yelVw+TT1YLWlwLiyEHrUiyY981TDehxT/ADAe
vWr5gLfm84pdw65qop696cJOgzRzIZbVuM54qRZPSqqtTw3FHMBYEgJ4/GpFbGDmqySenSpA2V9K
nmESiTdx+lSBg3FQtjjvQGOOtLmGWd/40KxX6VDup6vnio5irMmVj3pdwz1qPzPek3cgd6lzDlJQ
3XsKXJZTz+NRb93I4o37VPNLnKtYCcHFIxwvqKQvu5pjSflUuQA7bun603cMjmmliaYzcUubsS0L
IwyRmotxPUUjN600SED/ABpcxNhWPy1Fu4J6UMwIxmms2VxVcwgaQmmhw3U4P0qNmIPsKRm6dqfM
A8sVwp6VG3ANNZuOeTUbMc8c0cyAduGOmKiZ+cdqa3HXimM35VLkA5m6nJGKiLcZxQzDb15pm4HO
TWbn0AUOaGboRSBl5GOT056Uwn2xWDlcCQZ5waeG+U81XU4XOeaXd2zzWEpAWF+Vc0uQAMHNQDOM
E8VIG7VhKQx/mNs8vPy5z0H86buwwFNB+tNz1Oa55SHZjywbvUZoHrTNx3Vg5CtYXPXimM3NGetN
3Drms+cQF8Uxm3UjN60it+FZuoApNRd6X2qJm+bg1m5oAZhu4FNZvfmkY8elN3DqRUc9xh5hoptF
ckpK5qch2xTY+XUds0UV+Qx3Oo3YWLKMntVkdqKK/RqLfs0ZEsf3acrc0UV6EW7AP3Gpk9aKK3i2
BKG3daG4BI4ooroi3cmWwoahmPWiiuyO5AmelPi+ZSfaiiumIDo+VJJp1FFbLYBFP88UZoorYTJF
+7nvTm4bFFFbxIluOTsM96evIb2ooqwW4isdx+uKWiit6YSFYkNt7U5WIxRRXShxEZyMfXFSLzto
oq0KW47+LFPUmiitiSdfvU+iigqIR8yY7VLRRQOQ6IbsZ9M1JnEee9FFALYdH0FTxsStFFZscdh3
WhOWooqSXuOLEYpQciiis/slodTGY7utFFIZJmkzlqKKADsKYzHJoorOQDHYjAzUeT1oopfaJe6G
t1P0qFCWzmiitGkUMJ5IprHH5UUVitzEgLFsE0p5xn1oorZbARnio9xXkHkUUUwIGY4Bzkmmt/Si
ispbICPcd2KT+LHaiioAMbVYimBjkd6KKxkBIygYwKCxXkHnFFFYvYBV+7nvQuf1oorlew1uJvO4
0rfKoPtRRXNI1EU/KTSMoA6UUVzSE9iJx3qEMcmiisjISTuaae1FFZvcpDVO4nNRM23OPWiipJEZ
j0prdqKKiW5pEj3HceaKKK45N3KP/9k=

--_30A821E4-34E6-412A-BD80-6EC1FA5235BE_--



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:06:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:06:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181767.329052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvPM-0002KO-6c; Wed, 08 Sep 2021 11:06:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181767.329052; Wed, 08 Sep 2021 11:06:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvPM-0002KH-3V; Wed, 08 Sep 2021 11:06:24 +0000
Received: by outflank-mailman (input) for mailman id 181767;
 Wed, 08 Sep 2021 11:06:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uOOR=N6=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mNvPJ-0002KB-VF
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:06:22 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::62e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 51402be4-325a-4eef-ae23-0b6b4ab43f8f;
 Wed, 08 Sep 2021 11:06:19 +0000 (UTC)
Received: from AS9PR06CA0245.eurprd06.prod.outlook.com (2603:10a6:20b:45f::13)
 by AS8PR08MB6200.eurprd08.prod.outlook.com (2603:10a6:20b:292::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 11:06:15 +0000
Received: from VE1EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::4f) by AS9PR06CA0245.outlook.office365.com
 (2603:10a6:20b:45f::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 11:06:15 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT023.mail.protection.outlook.com (10.152.18.133) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Wed, 8 Sep 2021 11:06:15 +0000
Received: ("Tessian outbound cc1d52552731:v105");
 Wed, 08 Sep 2021 11:06:14 +0000
Received: from f5378b50b472.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 43942ADA-248F-4FBA-BFFF-1EE7A8106600.1; 
 Wed, 08 Sep 2021 11:06:05 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f5378b50b472.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 08 Sep 2021 11:06:05 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR2PR08MB5225.eurprd08.prod.outlook.com (2603:10a6:101:1c::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 11:06:04 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 11:06:03 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0504.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13b::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 11:06:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51402be4-325a-4eef-ae23-0b6b4ab43f8f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rjVEPgleUufWG7FxRAJ8OgjKsZxLoAtFwjrmB8PICUc=;
 b=RTFG2ZswWbQynh2C8SVAULRBQXiLD6vQ6KYVnn9xpIuxjQBC4HTGr1AYeVIrXyYoIWSh8q3VwbEuIy7b5ZbCME/4lxaKc2F10P4xLpD1lfA7LZX6BJ30OovO8USWZL74UovcNAV9Ad4ZGZ0ZGZGv/7PDgWdqyEUtMItvW9S1RxI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=fail (body hash did not
 verify) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 77d5b0b807d761fd
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P8kWVOiBSP8qLaAqd09llyYPIw6gMG4/PteTBs+oJ3uhI2cC0UPuFGV3p6wNVdGmvvGvxPGigFF66mcVcNcsvjrKfwGHNiFxHeRAHHonnEAs7N7D4GozY++CdSfjOVzoJvSIEAoLpln6arqUGTZAEm2v8RAI5D8rFAGpUSCbuj9o1tx94h0gtC1ngOil76t2x+MUVRZ4NKmKpExMEq6AJ2Cc+nRxj917b3vim92S0zLwcr6+p9pfokOOD/YE4Twa//Rzp0V8g9rNxVGC19xN7T/TNRGgLy3WXDEgtwAHlGoY8MnRWqcrEZiAd/ovoZWGpeweBcGfyDakuPHFk1IHAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YqptrvRxS37Q3pgOV+agaxEVAKQ847xwENGiT+K4Hdw=;
 b=J1FPWc9IGmAZyP69FSlXEdf0IQFMvFe74gXV6VottOJ5ZcktG9aV26JL5fWvVpXyn7MBrksLIE03A0a1tMZJa0pNu9QLQwtB6oNu2nmAqhdN4tWSLimyDz27Ha6nX5oGEPetpCItoccLKvcRvJoJ8ryqCo7pZcXfO/7isziX1ue7PLJwGrP5XGy5xb61Cx2LZt5nNoWF9yq2b8IiAUjV1fl8GpmqZYdvSFNf4pMLaEK7fO5Cv8KvO2JkEt9+1lC5UJgMIcguBa2FYUFi2KURVGt4epRnwRkvLf3MWFlbshHbf90PilFP+1aexbRRLajA4GQ8wiB5pLGY9esBbL+6Ng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YqptrvRxS37Q3pgOV+agaxEVAKQ847xwENGiT+K4Hdw=;
 b=8BVntMYWvuAlycfQT1hoJ/PYT7qcR1VMxXElvwr0PRW9oFVKVkM3sVL6MfW42AeGWHyqBWnVuyAiq+5eCGx+YR9pOGdCl2D2E8p2WwkbgVM1UKl9e7en2urzo8HiAmniLXaGHY3CzJd/L4gIURTyCRZLePDLfNz66gvDII08fbs=
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: Xen Booting Problem on ARM Machine
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1B61076F-D44A-41CC-89CC-A8E0E65E64EB@hxcore.ol>
Date: Wed, 8 Sep 2021 12:05:56 +0100
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <A3F0FA99-E434-43D2-8F93-64E6C6423DEC@arm.com>
References: <1B61076F-D44A-41CC-89CC-A8E0E65E64EB@hxcore.ol>
To: Sai Kiran <ysaikiran1997@gmail.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0504.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:13b::11) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 625394a9-6ded-4d33-9648-08d972b8ad23
X-MS-TrafficTypeDiagnostic: PR2PR08MB5225:|AS8PR08MB6200:
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB6200584C5062CD2BB978D7C4E4D49@AS8PR08MB6200.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 jO60Phj85dBvXxQlvGAE90E8AwrJFPdK+7hYcBi80bYXdrQGAAddUy8ghvMMTKapIR7ZHOhv07ljherh8w+S419/TwycyToou2P/X83ZwEc4zUUY2tV1/ICy+l1KFlWXRHanY8btK6ZFHQV6dWgbYEhnjzs4yduj4VF5MgZz6Mc0vm58mWQJtWY5HMC5bc/SIjDwYB33p13hj9O3+IrjMKuysFK6dwrGsZfgkdUGyaEaL7krA6qpAAjGzspJg6mZupZgz+qLfD1/KDJMc1kZoYxghxvCMtjs9ojPdjOjnlUj8bIZS5NEvkNaYDZVr+jyq0q+OzVNkWCvlZpIBCQwelTpB8UaVg8x9suCWZ8YpvxAN0pu5y35e2HRLI6InvZMpNEBcmnPIIWq5xeA2jduNtn8UWoxCRwyM4vwft5RhXjJ/Qd/S1vQIRbQ7M/e/Ugzfs/NaPp4Uw+d+yQuaHg21VdFmDWR9UXtx2H3R3Q72nBVYrfw3GJEPwtLg+S09Dl+TZ9oT2lVTy6fHdoViShEGrKM1yHeUMJ36TP00Iobnyc6fAHWh5ol36nGhqBx2Na6D/dkDzuIYrAonKEkMjFyd7FHIpUR1yArIp1m7oWcGSfqCdtVLRk0ohjPkE/lxxdvzCUPmMDe+TkEU0y2sC35ienc/R5/6c1DqW+8i0MeiB3oJRO2U106cIPQWTM+JddNXiqhE5lixaI/yUmd/b59udehcV88DaUbNLu/QWUt5rA=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(376002)(346002)(396003)(366004)(316002)(2906002)(6666004)(4326008)(66556008)(66476007)(38350700002)(83380400001)(33656002)(6916009)(44832011)(36756003)(38100700002)(26005)(6512007)(478600001)(8936002)(5660300002)(8676002)(186003)(52116002)(6486002)(53546011)(6506007)(66946007)(2616005)(956004)(86362001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB5225
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	206d7893-cccc-4f11-b113-08d972b8a5c5
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	klPi255n2epLo+YUD/DCQJHA96P7JzcJodveVdqgXKt+eY4tAYxArRL81ZvcvShjAQrbDvmmnG3g2UMWXVZd7ozu2Rtf9yz0P8rPylJMzNrfL5lVmN7N5HpKCOgKrUub8mrDek0JXIiD0LczyXobREoC6FQh60Ij6TOSGK8GTi2743rYkeQ9xBshVIcLplO7wwieVUoLlvnSbPbH1es/C+yq8umzRr9YW32fa+PMAar+56qJepdU1NVIZCmj/X8tPqOpHSjDK1I2LdUTXt0sXrvdQZFs7QidDNZTjwzBZ94mpHp/gQsrCCBQZv6a7wqMeREMgOJT2v4pCR56O/O9dL1gJzuohyCUhwSaWiXOK1q6sMwLmJsEt8r+j8j9rvb2hLHC5QbFUbppA/90q7UkBwfZV0EVS2AfQJCxBqvmr5W0aVh16e50/n5oK86kPRjTuMzXTkiXa2MoIElqimr4NDe/zMev6St2GH7NAe+XOw4M90/dJEw+ZpBBO5GgW385ft575AyEolBsKPgNKZN+ex1u0owKeGbKl9dlF+l7v1qnYOKKrsc6QrAnlLIixjcmhLHyv+ksS0dzWVNJWBN1C0Yt3C7rEqcULLq2EWPXREntAj3+9miuUsCiv2U7ILkEBS/eHZAG/U4iZkLgyz+uL5k3h5+kMXuSXOmP1UWcd4pHLtEw6YAL3+3Oa7NUAXhq4W5LvKJuVWUoIszadX+Uaw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(346002)(396003)(376002)(136003)(46966006)(36840700001)(6512007)(82740400003)(6666004)(2616005)(8676002)(70586007)(4326008)(8936002)(70206006)(53546011)(6506007)(26005)(36756003)(86362001)(47076005)(4744005)(44832011)(83380400001)(36860700001)(336012)(186003)(33656002)(2906002)(6862004)(356005)(6486002)(5660300002)(956004)(316002)(82310400003)(81166007)(478600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 11:06:15.2875
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 625394a9-6ded-4d33-9648-08d972b8ad23
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6200



>=20On=208=20Sep=202021,=20at=2011:46,=20Sai=20Kiran=20<ysaikiran1997@gmail=
.com>=20wrote:
>=20
>=20
>=20
>=20Hello,
>=20=20
>=20I=20have=20Xen-4.15.0=20on=20an=20ARM=20Machine,=20with=20Debian=2011=
=20installed=20on=20it.=20I=20am=20able=20to=20do=20=E2=80=9Cmake=20world=
=E2=80=9D=20and=20=E2=80=9Cmake=20install=E2=80=9D,=20after=20=E2=80=9C./co=
nfigure=E2=80=9D,=20as=20specified=20in=20README=20file.=20When=20I=20reboo=
t=20the=20system,=20I=20get=20the=20following=20message:
>=20=20
>=20Warning:=20All=20128=20bootinfo=20mem=20banks=20exhausted.=20
>=20Warning:=20All=20128=20bootinfo=20mem=20banks=20exhausted.=20
>=20Cannot=20exit=20boot=20services:=20ErrCode:=200x8000000000000002=20
>=20=20
>=20Here=20is=20the=20snapshot=20of=20the=20error.
>=20<D00C86F8093B4937B59AA7450B930686.jpg>
>=20=20
>=20Can=20someone=20tell=20me=20how=20to=20fix=20this=20error?

Hi,

I=20think=20this=20is=20more=20a=20question=20for=20xen-users@lists.xenproj=
ect.org.

Cheers,
Luca

>=20=20
>=20Thanks=20&=20Regards,
>=20Sai=20Kiran.
>=20=20
>=20=20
>=20Sent=20from=20Mail=20for=20Windows




From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:08:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:08:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181773.329063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvR3-0002vn-JH; Wed, 08 Sep 2021 11:08:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181773.329063; Wed, 08 Sep 2021 11:08:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvR3-0002vg-G9; Wed, 08 Sep 2021 11:08:09 +0000
Received: by outflank-mailman (input) for mailman id 181773;
 Wed, 08 Sep 2021 11:08:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNvR2-0002vY-Jn
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:08:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0b5ca974-1095-11ec-b14b-12813bfff9fa;
 Wed, 08 Sep 2021 11:08:07 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-cB24rCK_O2mHUJsw7tIBxA-2; Wed, 08 Sep 2021 13:08:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 11:08:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 11:08:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0052.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.22 via Frontend Transport; Wed, 8 Sep 2021 11:08:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b5ca974-1095-11ec-b14b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631099286;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ibZZSZDjrGpS0aAvSHO8S3jjjnBSNdFfyqK0ghkhpSU=;
	b=NsaByE/xXWWLyN0oNmwuqurFRmQXrMND+8O0yM5VqCLzBL83pLztT3V/eqruQxnP2Rn5zE
	NgcpdDKkO0FQ7p9QnDN5mT+OzdYfb0nuuc4329DROVQoVk/tKAlBOK6dB0rVQVcXhdcI22
	6dO0SJl7sdNwrVC8CY4JYukynDDFLew=
X-MC-Unique: cB24rCK_O2mHUJsw7tIBxA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kojxzyUv9/2zTUos7IKE2PwBZBeC1HlpRWqMgCscqEGqwe9gqo9u9UcE3l8ihbkaLQkmIILrOtp4QUsoDQauqiEQOcXIwgkB2NOYPt9gXsxirqbLD2aXhtzNU8/m5Q0mfYTdG2O77qNaI6KyBg9N+TIroLI4gENyN8pWk/iJlyMsCxQa4q604fSpGIGsBkvEwyvKaSI0jinFMvHG1DFui9xX/BBDeAkQunSFay5EPxvrIy6erJU6jRVSq1tTkZR7YsvDteG7UmQOBRikyzyEBZPesINpud4IW+jn4vJ2KWBuWHwKK4Tnj9z1inzI48q+DMo0eYHvW8u1wNfwtw+c9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ibZZSZDjrGpS0aAvSHO8S3jjjnBSNdFfyqK0ghkhpSU=;
 b=n89IR/s7EENoYKOP7zqcqikiBQa/6U6Pw4yVEdVkTg8cbR9YPGOSs0HnnBtKfbcgehC1QJqFZNZV3QKsTcV33Lh+nxu2A7HS4KKOv5CDcAox8Wvx+zLTdfdSdEeTs2DCBbmfslPOzNS6uJ4PlVZQST8ZBDAcyjsoeVp05mpI74zJxz3fZMFLP/pTgrzbFhKuKu1C0gWPHlVGDtkX3DUHE9SkK1gEt5/u9nDdhiuOsX6gBTaRUvl08uEZX/sHnBjhohkHsn3Y/5gcNA09zkxX+snyF7OnLfTNsUAs5+mGKv2zFmc25Jv92/CIngsaw2urYRSLshpaHPeEOLPyU69sXg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
To: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210908073640.11299-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
Date: Wed, 8 Sep 2021 13:07:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908073640.11299-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0052.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2f41b41d-b43a-4b7a-6790-08d972b8ec75
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7039F6D45AE04AF5D442EA02B3D49@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	utxgzYs6dqQkvYSIi4H5UoBlLIk0CWWEwwADTjpOH1Bd4dSJ/KZAF+ADIOmDpS+OyNgPEG0fnWoR7mlPq3D4KkZiLy/VrZY73jYm9tNVlMGn6nDk5xLqaFdyaZwARtC7G/s8JzwtHKC6E3i0/BW8xHxIZRv11Dgr1YTYeGB6cwx0v48HtMkAUty8hmjNqZ9RLjTlwGwWVuI6twro9ekZIdYzr9SIeHRzMwNWP3wnXBkD10X2HMVY0htijEs6N4uTDUllyZc9uD6euEXBdf0yW9XN9OgpTlv28ZblCNp0syphMiSmh70JBlHINPsOmQ0+f1fp0bptpWKxEie/uj5SIuAfqySJ7WmpcVDv+v3e+f7bp/sjxUa/449KrxR7Pmmwg1RL0bqQQKu1SF0Jo6xhLqwp/0GaQtA3HTm8tQTIbLk5kiZoeZwBv3cYcGeXEQvijTjoOUgC7HTLSWi9FwxIEHLOPtR+TAS12FVJgHhyzprcwExcMnXuaa/T+1cB0P0OmGNc9U5SuwIDE4ewTSyL7JxEsyPPkEH5OHwJgkbFcfSbL5eJ5tQii7FVvugq1VUhEn44/0maQlZG/GU9Vfg1+2Nt4JDgPfTuFAxnjfxu0SdQ108Yse5cpn5brNp5pwYl0qLa5iDUOGRQeF0Z1s300M45ADdxWCW2vWjtrKvaJBp4qrcMQ9Ew+Ed7MUZzvpXtDLFATgywzL73GD/6Qc1x2aSJtv+cGOUrdP62Su1s7qg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(396003)(39860400002)(136003)(376002)(8936002)(186003)(5660300002)(8676002)(478600001)(26005)(956004)(2616005)(86362001)(4744005)(6486002)(53546011)(66946007)(38100700002)(66556008)(4326008)(66476007)(16576012)(6862004)(6636002)(83380400001)(37006003)(31696002)(7416002)(2906002)(316002)(54906003)(31686004)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ODQwS2pkZitMMHA5dm1lM1NqNHRSYiswTGU3SzhkRkp2ZysvZFFob3BtelF1?=
 =?utf-8?B?dnB5M3Z1ZHV0ZGRrNmRiVjk3c2hab3FybTRvd044Rzhhb0h5ZnhhYlhZMjYy?=
 =?utf-8?B?YXprZEdYN2hnRG9ZbUV1SHNtbTZSa3FRVjR3T0tYRng3VjcramZVMXhQOWF1?=
 =?utf-8?B?TjZSZjZHcG13enYwbGxPRE9Xem9HUFJjYlFuaG01UGJlK042MEtiQTQ3VWFl?=
 =?utf-8?B?aGpLSGNwQVRQUExvTlcySzRYekJCRUlmampYOWJvV29taThXSmEyVGdrZWhS?=
 =?utf-8?B?bXY4dmdxYUQrUDNPL0hhc0prcmsyVTQ5dm5sdlVOV3RmTkpZYU4vVnlaZGlw?=
 =?utf-8?B?U2tOMStqcWN5c3RPdWVFa3IzL2tod1cyZXBpNk13SjRGVVNJQWhycTBSY2Q4?=
 =?utf-8?B?V1d5S0Q2QTdSUU4yMVVKaFBXZ0dEM0Y4eDBqR2VtSG1KTk9zT2lxa3VNeTVs?=
 =?utf-8?B?dXkxbFdKcWcwV2tUMG4wOG80S1BzTTdLQ3o4R1lVeDVrU1VZdkNsZEpLMGZ1?=
 =?utf-8?B?WE84YmgyU2JjalhhVTl0SXhiT3BSQm5nRm52Uk9xaUJYSG9tMDB5Q1NkMk1H?=
 =?utf-8?B?b05McUpTaUlEUWtnQWQrQnJDanBqaWhqK3p5WHdTYi9ZQjFnSUNaWEEyWUp3?=
 =?utf-8?B?eHpxMml2d1hFZkw1YjJwekttaVVHcU5Db21qengwUDZTdEswamxwT3lzTTFL?=
 =?utf-8?B?d2NkV2tEOEVHbzhPakdnMHgzK2p6SmozVnFxUkhvUFcwc0pseHNDTlBoMzJB?=
 =?utf-8?B?NVFqSzBudzJLeEZCdTd1WmJZd2RPbXRBU1hCUFNTT3djOE10U2NzdjUvaStP?=
 =?utf-8?B?YnRVRDZnUEZJUUh5bmVSMmZ6dVh5TEsyRGRqd1NsWGowendvbHA5bDlHc3A4?=
 =?utf-8?B?bjFJdjhTc0k1LzFBVjk1MStGQW5rbDdveDVqUEJTdUxRcElZdDVqV0x4STZN?=
 =?utf-8?B?RzBtODJEQ01jdzhOSS9Va3Rqa25sb0hzYU93OGI4Q0QrR3BZUWl6NHY3Nzli?=
 =?utf-8?B?N3hlaXQyZk1TVStLdlN6YjZEU0ZkRlZNQzBDUWhvdkJCekg5K3F3M0VoMEZY?=
 =?utf-8?B?ejdDSW9ySW81YnAvRURWaXU4VDNrWEJSSGpiakNKU3RlcmcxYmxoQXVPRU9Y?=
 =?utf-8?B?WjZGUmVTbzdaRWRRVGtqM0RoR3ZGQzNPMTJWTlArd1d4M29ENDNSQXJ3V1Q4?=
 =?utf-8?B?N001ZDBVa0pMUVhYSW85MFFlZkJrdmtpUkxkcEVtc0pXdElwNGVXaURrazdy?=
 =?utf-8?B?d1hlUFFqRitBSFdpNEtoMGlYNEdoQU9hYmhyd3Nmd3VoWnJCLzdmSlQ3dWpj?=
 =?utf-8?B?TzZQczdmVVREYi8zWTAyM1FHc0VPUUNaUHF6Lzk5Y3h4M3BPT0ExajdKc0lB?=
 =?utf-8?B?Y2lvNGpsdURnb1lJckNvOThoTXIvcE5HMTgwdnJjM2xwS2p4UHNxeURCdE11?=
 =?utf-8?B?S2FxRHNURjlkRmRjYVFyamFENUJGSnp1aHpaU2lralMyWnJCZDM5alo4RmFJ?=
 =?utf-8?B?c2xaeDY2bmN1WjhiWG41ZnRjVitHOVdWbmxFcS95Tnd1ODZKZjVJa2U1bXBp?=
 =?utf-8?B?NGJrYUVpYlp5WGRuS2tJYTBBVlNZczQ5NnZydUpvazR4aHo5NVZpL1VLWFZS?=
 =?utf-8?B?Z0ZlcEZKdElpUGt5UVlzMElJalcyR0lWYnJkaXJ0MWF1UHJ3SC9pKytORXdQ?=
 =?utf-8?B?WGdkTVpkZVZFYVM5MDluQ2M1WVFOMUdTcHdUNEI1SFlYcldmR2ZLVklZcklZ?=
 =?utf-8?Q?9bkFrGvPxGZxWkNbXvaqTo/b/suyhuSN2+mUZNK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f41b41d-b43a-4b7a-6790-08d972b8ec75
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 11:08:01.8090
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZLHvnujt1kr3En3TS+/V2UrsYzpUsJ8YohOMkTEo7Rf3vUlrfLp7XX8XECNWVT4CLf14MmIj3x+hZt1iKFEiag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 08.09.2021 09:36, Juergen Gross wrote:
> Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page
> table entries") introduced a regression when running as Xen PV guest.

The description of that change starts with "pmd/pud_populate is the
right interface to be used to set the respective page table entries."
If this is deemed true, I don't think pmd_populate() should call
paravirt_alloc_pte(): The latter function, as its name says, is
supposed to be called for newly allocated page tables only (aiui).

> Today pmd/pud_poplulate() for Xen PV assumes that the PFN inserted is
> referencing a not yet used page table. In case of move_normal_pmd/pud()
> this is not true, resulting in WARN splats like:

I agree for the PMD part, but is including PUD here really correct?
While I don't know why that is, xen_alloc_ptpage() pins L1 tables
only. Hence a PUD update shouldn't be able to find a pinned L2
table.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:15:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:15:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181779.329074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvXl-0004a1-DO; Wed, 08 Sep 2021 11:15:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181779.329074; Wed, 08 Sep 2021 11:15:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvXl-0004Zu-9W; Wed, 08 Sep 2021 11:15:05 +0000
Received: by outflank-mailman (input) for mailman id 181779;
 Wed, 08 Sep 2021 11:15:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FUO+=N6=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNvXj-0004Zo-OK
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:15:03 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c82d6b71-a64b-4306-b480-d3964c41194f;
 Wed, 08 Sep 2021 11:15:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c82d6b71-a64b-4306-b480-d3964c41194f
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631099702;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=ofIWEMY5+EK+JKgsmrzt8FZLnZlMtaCP0HWNgorTbrs=;
  b=iMQ/ITT9nEUILEFnucIHsxOsDvIPaG2IkBujcXG/oRl8JGGAzkITyTDr
   N/xcFgGQlcXFclwt4YD0uZB4Evsowq0vIqXc0kTsOYSduR7D0rV57Ltml
   GOzLLyXbRNA86R2uqaFW4DiKPS+GDrDlU7dqkJdLQbMk4Z7CM3mJYwyWh
   c=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: yu7wkN9+I7fVB+84sgevwm/Fcn1zI3OrEoXq/wC/7jI6xgO1bfOpqdAEb204briH+nN7MPbXEA
 Ayj3yk1pjMBDs1Qbvk/8Mdq9O8AxNu1Ftgv1oMCgprFWZI8RPkhFnbou0CHCJt2cbk04xnChtW
 b0Os76oZbM00MnPZFLyFKC/U4IJSJctxEvCkjNqK1hAr4/j9svpOHS2F9gam3sUiwVFYQNF2an
 YkUdl008MgR7OXXxu22u6yKlq10r6AEEqeE2cfVBIqr1x1hSSWdQUVUjRgcbPtbGkETki7ZX2c
 jo7GIGMHCJhozx+J8Cck7q09
X-SBRS: 5.1
X-MesageID: 52251320
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:mZRoAak58A2M27MqqKZ5KJyQSMDpDfLW3DAbv31ZSRFFG/Fw9/
 rCoB3U73/JYVcqKRUdcLW7UpVoLkmyyXcY2+cs1NSZLWzbUQmTXeJfBOLZqlWNJ8SXzIVgPM
 xbAspD4bPLbGSTjazBkXSF+9RL+qj6zEh/792usEuETmtRGt9dBx8SMHf9LqXvLjM2fqbQEv
 Cnl6x6jgvlQ1s7ROKhCEIIWuDSzue77q4PMXY9dmcaABDlt0LR1ILH
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="52251320"
Date: Wed, 8 Sep 2021 12:14:57 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, "Tim
 Deegan" <tim@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a
 single .c file
Message-ID: <YTibMaayiWlJwcZ/@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-6-anthony.perard@citrix.com>
 <e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com>

On Tue, Sep 07, 2021 at 08:14:14AM +0200, Jan Beulich wrote:
> On 24.08.2021 12:49, Anthony PERARD wrote:
> > This replace the use of a single .c file use for multiple .o file by
> > creating multiple .c file including the first one.
> > 
> > There's quite a few issues with trying to build more than one object
> > file from a single source file: there's is a duplication of the make
> > rules to generate those targets; there is an additional ".file" symbol
> > added in order to differentiate between the object files; and the
> > tools/symbols have an heuristic to try to pick up the right ".file".
> > 
> > This patch adds new .c source file which avoid the need to add a
> > second ".file" symbol and thus avoid the need to deal with those
> > issues.
> > 
> > Also remove __OBJECT_FILE__ from $(CC) command line as it isn't used
> > anywhere anymore. And remove the macro "build-intermediate" since the
> > generic rules for single targets can be used.
> > 
> > And rename the objects in mm/hap/ to remove the extra "level".
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Hmm, when replying to 00/51 I didn't recall I gave an R-b for this one
> already. I'd like to restrict it some: It should be taken to stand for
> the technical correctness of the change. Nevertheless I'm not really
> happy with the introduction of the various tiny source files. I've
> meanwhile been wondering: Can't these be generated (in the build tree,
> as soon as that's possible to be separate) rather than getting put in
> the repo?

Do we really need to generated those never to be change tiny source
file? Do we really need to make the build system a lot more complicated?

With those tiny source file in a different directory to the main source
file, we need to add "-I" to CFLAGS. (source tree vs build tree.)

I don't like preparation phase, I'd rather do just-in-time generation of
those tiny file (if we really have too...) as to let make organize
itself on when to do things. That mean, extra targets on how to generate
the files, and telling make that those would be intermediate target
shouldn't be deleted to avoid the final object from been regenerated
over and over again (I'm not sure why the rebuild of tiny source file
happen when they are intermediate, maybe because FORCE prerequisite on
%.o).

Can't we commit this patch as is? What kind of issue is there with those
tiny source files? Should we add a warning in those tiny source files,
something like "No modification of this file allowed, it's part of the
build system." ?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:17:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:17:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181784.329084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvaO-0005Dy-RC; Wed, 08 Sep 2021 11:17:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181784.329084; Wed, 08 Sep 2021 11:17:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvaO-0005Dr-O2; Wed, 08 Sep 2021 11:17:48 +0000
Received: by outflank-mailman (input) for mailman id 181784;
 Wed, 08 Sep 2021 11:17:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FUO+=N6=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mNvaN-0005Dk-Dm
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:17:47 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e15f394f-9ca9-4501-8de3-bfa322d43296;
 Wed, 08 Sep 2021 11:17:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e15f394f-9ca9-4501-8de3-bfa322d43296
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631099866;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=yFoFUnDtDv1Bx6JlM1glTN7fv8M7gM83RxpG34T+cRY=;
  b=ZbaEUlHZTA0s9AMBBVmOnBzNbWxZMAa6Uri1khm+QVM3NdTEX4mAuarx
   rYQc5n1pqBJUcb+xm3cAw9I4+0Su94Ug2IiN0/91IYDczmHek5GdrxL4q
   qbhJ+zzaBdH2WtEts/ZY+QaWDjfbaV1+ndCgwNSM3/LBZEpNkRjOFDIEH
   g=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: eJup9ZqHw10h4D/ZS5pZyXcOzlKqTsplKgaSzOy3lgSUgNnC0xL2D1DEtmDOsi+MyZy5D3z7iu
 EJo4Te53bMEeEYuZE1XZOyXMdxozW+HGQS+ymxbmOHjjgcFC+hVXiIVFpeL+GpuuTW9fjU8IID
 cvjRSX5Kb00xT3kDIiH8rYtZMCNKl1axeZiAeW/Yu2Beqz06s/IFEZ3ijsVxvMUU62aNOfmUxC
 rvKgSOaPvFGeg14yGV0msqMqFRPFkdfvew6uj6C7/lvOdsnu3q9ZozPPKTElLAWZLneRp/i3p3
 xCY2pPOryyrlwzmj5kVEnd6T
X-SBRS: 5.1
X-MesageID: 53998744
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Z2m7M6wW8HYwdS+Q8kRnKrPwIL1zdoMgy1knxilNoRw8SKKlfq
 eV7ZAmPH7P+VAssR4b+exoVJPtfZq+z+8R3WByB8bAYOCOggLBR+sO0WKL+UyGJ8SUzI9gPM
 lbHJSWcOeAb2RHsQ==
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="53998744"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Ian
 Jackson" <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: [XEN PATCH v3] xen: rework `checkpolicy` detection when using "randconfig"
Date: Wed, 8 Sep 2021 12:17:27 +0100
Message-ID: <20210908111727.440265-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

This will help prevent the CI loop from having build failures when
`checkpolicy` isn't available when doing "randconfig" jobs.

To prevent "randconfig" from selecting XSM_FLASK_POLICY when
`checkpolicy` isn't available, we will actually override the config
output with the use of KCONFIG_ALLCONFIG.

Doing this way still allow a user/developer to set XSM_FLASK_POLICY
even when "checkpolicy" isn't available. It also prevent the build
system from reset the config when "checkpolicy" isn't available
anymore. And XSM_FLASK_POLICY is still selected automatically when
`checkpolicy` is available.
But this also work well for "randconfig", as it will not select
XSM_FLASK_POLICY when "checkpolicy" is missing.

This patch allows to easily add more override which depends on the
environment.

Also, move the check out of Config.mk and into xen/ build system.
Nothing in tools/ is using that information as it's done by
./configure.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
v3:
- use KCONFIG_ALLCONFIG
- don't override XSM_FLASK_POLICY value unless we do randconfig.
- no more changes to the current behavior of kconfig, only to
  randconfig.

v2 was "[XEN PATCH v2] xen: allow XSM_FLASK_POLICY only if checkpolicy binary is available"
---
 Config.mk          |  6 ------
 xen/Makefile       | 18 ++++++++++++++++--
 xen/common/Kconfig |  2 +-
 3 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/Config.mk b/Config.mk
index d0712724f8e4..144411133f38 100644
--- a/Config.mk
+++ b/Config.mk
@@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=y
 build_id_linker := --build-id=sha1
 endif
 
-ifndef XEN_HAS_CHECKPOLICY
-    CHECKPOLICY ?= checkpolicy
-    XEN_HAS_CHECKPOLICY := $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xen && echo y || echo n)
-    export XEN_HAS_CHECKPOLICY
-endif
-
 define buildmakevars2shellvars
     export PREFIX="$(prefix)";                                            \
     export XEN_SCRIPT_DIR="$(XEN_SCRIPT_DIR)";                            \
diff --git a/xen/Makefile b/xen/Makefile
index f47423dacd9a..89804aefe385 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?= $(shell hostname)
 PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
 export PYTHON		?= $(PYTHON_INTERPRETER)
 
+export CHECKPOLICY	?= checkpolicy
+
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
 
@@ -178,6 +180,8 @@ CFLAGS += $(CLANG_FLAGS)
 export CLANG_FLAGS
 endif
 
+export HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)
+
 export root-make-done := y
 endif # root-make-done
 
@@ -189,14 +193,24 @@ ifeq ($(config-build),y)
 # *config targets only - make sure prerequisites are updated, and descend
 # in tools/kconfig to make the *config target
 
+# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
+# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
+filechk_kconfig_allconfig = \
+    $(if $(findstring n,$(HAS_CHECKPOLICY)),echo 'CONFIG_XSM_FLASK_POLICY=n';) \
+    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG), :)
+
+
+.allconfig.tmp: FORCE
+	set -e; { $(call filechk_kconfig_allconfig); } > $@
+
 config: FORCE
 	$(MAKE) $(kconfig) $@
 
 # Config.mk tries to include .config file, don't try to remake it
 %/.config: ;
 
-%config: FORCE
-	$(MAKE) $(kconfig) $@
+%config: .allconfig.tmp FORCE
+	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=$< $@
 
 else # !config-build
 
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11af3..73d8afb7bcbd 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -235,7 +235,7 @@ config XSM_FLASK_AVC_STATS
 
 config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
-	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
+	default y if "$(HAS_CHECKPOLICY)"
 	depends on XSM_FLASK
 	---help---
 	  This includes a default XSM policy in the hypervisor so that the
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:19:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:19:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181791.329096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvcL-0005tI-Bv; Wed, 08 Sep 2021 11:19:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181791.329096; Wed, 08 Sep 2021 11:19:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNvcL-0005tB-91; Wed, 08 Sep 2021 11:19:49 +0000
Received: by outflank-mailman (input) for mailman id 181791;
 Wed, 08 Sep 2021 11:19:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNvcK-0005t3-1E
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:19:48 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ac53af8e-1096-11ec-b14b-12813bfff9fa;
 Wed, 08 Sep 2021 11:19:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac53af8e-1096-11ec-b14b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631099986;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=aDBidDCC9WuT7tkVMIEbQgAn3Qkxl/NpiZcnYOTXLnY=;
  b=hQ2lYKccmG115exS3eWhAEAa4nDu7aOHzdpRqIw5KrcquQyx8qoOBhD7
   ng4glZaQq+TX9l6iMF89fg7ri5hZleuvqHR2y0sJNloBcB+2LlcHNcI/w
   b/LIqodg8bFlCoCbWohBuvcj1+84liw9yw7f4jngeB9dYqAWMDktnk2A0
   A=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: JpeoATnmWEi63cmMlurSE6BrQFH4lnB6Wf7cykLDMh8c3eDTMrnyd9PZs1tlkiznOcu6kDZ1wA
 iaaD9l3mcOaoIjAz4wsVPkufAHcp1dWTXprbvt+UKEvj54FIckXg3r7iH0Zq3knp3RaoELItoY
 r8rRT1/s1EKXayTp6x6v2AMzG5oZBWK3ShKGeVcwsp9KKcN3HmRTQ2V72hnWHw/3vaSRsG3tZ0
 L3XY6SDCwx7NSwtrPDOrsG1ol/6Fpdo1QB6aD3u9F6zxhyTSTRA6Ksiw3f0ZovA63iZHoTgbKs
 BiHWSv5j8Y0EucHEzOuqCyvT
X-SBRS: 5.1
X-MesageID: 52642355
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:SRh5IazutSZA/iSoI4cJKrPxu+skLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjkfZquz+8L3WB3B8bfYOCGghrUEGgG1+XfKlLbalXDH4JmpM
 Bdmu1FeafN5DtB/LbHCWuDYq8dKbC8mcjC74eurAYfcegpUdAF0+4QMHfrLqQcfnghOXNWLu
 v/2iMKnUvaRZxBBf7LeEXtEtKz6+HjpdbDW1orFhQn4A6BgXeB76P7KQGR2lM7XylUybkv3G
 DZm0ihj5/T/c2T+1v57Sv+/p5WkNzuxp9qA9GNsNEcLnHJhhyzbIpsdrWetHQeof2p6nwtjN
 7Qyi1QcPhb2jf0RCWYsBHt0w7v3HIH7GLj80aRhT/ZrcnwVFsBeoF8rLMcViGcx1srvdl63q
 4O9XmerYBrARTJmzm4z8TUVjlx/3DE4kYKoKo2tThyQIEeYLheocg050VOCqoNGyr89cQODP
 RuNsfB//xbGGnqL0wxhlMfheBEY05DWitvGiM5y4uoOnlt7TFEJnIjtY4idixqzuN6d3FGj9
 60epiA2os+F/P/wMpGdZA8qPCMexnwqCT3QSuvyGTcZdM60k322urKCZUOlauXkc8zvdYPcK
 qoaiIviYd1QTO3NfGz
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="52642355"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aUgw24THEqNdg6LUGQw6mkalO/20JrQU52VSIghftCoVZ0iTkVo3Y4U7ql5gft/4yA1ROA27USR6eNgaVtgZ3LXZOCzLFVrtAW1Jim9och2Aol9Z/Pu8aFBBFqbgmgZX7iJmKvhGl7VxpVMTkcIgzKA30YlHzlsNTVqD+vPbYEdr0mLEOJPgjtbNG1vtdyfgIYwVcKW5LMUPuM8veo8SkNAXq8Ci3KNkCsGJZey4LQOQKsMK4pLqWvhceP4zeZBsigkz11TTZyPc1m40+kMLZ+ohKGrZUfx6yaT9Ee/mFuNiVOy3HcwU95vEi10/ADvZIQTPKUgLyr/vUPfFcpGfOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eub7v+WpEf0gP0zssA1hcLQX+YwwrFfIgLHvmvV0DNM=;
 b=XyHB77aSplRQbQx46al6tmq6Zju3qimfmdamozmWTRJk4k38yw1/SCBID708n1W5+RIuEO3Ny0We35fOZKW52AOj5Ck3F5mBDJasYy9eAYZO0HWW/JpKS6b/CUpAfZj0/1r5Q8+KQU/SfYPVpE0qLU3h+Suf7KTi8Ck/OM9DyrwbU4cokx6zCQoTJB6LmK1OD3EpKQ2GsYkH9I33yU376JUyaqnn2OVYZRN493Jtg3KY+Y4HAMOLwglgFYOSoQPP8vi4S350N2OjBRt+LvuNRU9oy6vQXtCJSpCTaukygasp+ixnA3D5UW70n4I8zX4coQT9HxoSxesmYoLq3wXYoQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eub7v+WpEf0gP0zssA1hcLQX+YwwrFfIgLHvmvV0DNM=;
 b=sZ4sGona6ueRIROEiVffo5qXz7RyN/qMEBIQ7M/XIcosgQKio67M743xuJoUR625UezF3St9g5EE8KomoyfrDWWOAXw6hB3KjH9MNVGg73NfGYha+zAH+gilpb+Q/ZmNEqTCWJ438jSl1Is2RjIGwrWPr/pgiFFBv6RbWOgB60Y=
To: Jan Beulich <jbeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210817143006.2821-4-andrew.cooper3@citrix.com>
 <20210907161908.17402-1-andrew.cooper3@citrix.com>
 <1fa45e94-ecd8-8920-1f08-027b348a71e1@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 3/3] x86/amd: Use newer SSBD mechanisms if they exist
Message-ID: <3895db85-d5cf-dff3-0877-684b4bc3d718@citrix.com>
Date: Wed, 8 Sep 2021 12:19:38 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1fa45e94-ecd8-8920-1f08-027b348a71e1@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0349.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:d::25) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 509c5b08-113c-4e30-cba3-08d972ba8eba
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5663:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB566373431EB090F4A9896EE9BAD49@SJ0PR03MB5663.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: XV7vKGv66v1UQZf6jXeTrGYs4hxcdKJXJHfPF8xc5wBkBDY158yUk3lTze0pzpShDJ2vwznCPSwl/CMYF39nD6DVL18aKOhayd4aWnSkmBgXu9sy5hxhN/3dDsBHcT9GNbZWss1+Hki5UcqesLdoSkykwhQTYo1SDY6XiJCy7Kl200Qq/Tbwuo+3FAPuPCQaEoHcsnHXHl8javxMxTW9d7wCaN2j1Ah5PiSSUO/GQVALq+yniiSnax/NGTrPoda7GlXCbQmOkKs6TCYiwlyxkTrxm8BXzIzVZ3wieXi70M1Foe9lXDYBGWnoShHvMzXWcWqsgfxtokF2CVhmAx1Xwi618TN7m6Npatx8AkP50PFbtre8XjPg4IN+EjBzuLilI5PyWC7hLTub7WSb5vEesBMSd+SipNAyiNeczoFgis8uRStgHXklSb+z51Hf1brQzdY/HgBbyXXuaGP8sbyk0eZaVuaCS54IcYjMeZ3rqq6Z5rT02vkZkxgdEgvmoJRwCbWtwPlemZ5+ky2P1k/guKTZK+djvnRWxgQGEg4VlTnMxynGUdsgGofy2BgaEz1sp5dt9JrTYMO15mdjb/4cond/FscPbZ21op1e8RzsLmBVUJBZVnc4ZW1zmurcn2djeyJcX96/t46hIS8yEWLFOztLVNDvmm/4fxivb1vlTq596aRPWIT/CZw4+VJBg/gdBeXgoNmbNNjzYMyw9osJdCDP/4jiBsA+mpD/MZD6hhQ=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(376002)(39860400002)(136003)(346002)(5660300002)(2906002)(31686004)(53546011)(6666004)(8936002)(36756003)(38100700002)(956004)(4326008)(2616005)(8676002)(6916009)(16576012)(316002)(54906003)(186003)(478600001)(26005)(86362001)(55236004)(66476007)(66556008)(66946007)(31696002)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aVdGTDBpNWRjVDdXM01naTVwa3c1cmp5bmYrM2ZMUTBLL2FVMWxkQUs3NFdy?=
 =?utf-8?B?OVVSTmFPWEd3Y3FxcnduOTdtcHduMEZaN0hOVGFJcTRuUUxqTlVILzRlL1Zm?=
 =?utf-8?B?Zkt5MG1WcXc4b1dEeHoyRU1MRmt1L0k0WlR1N1hEWTNZYmVJd2RDVnpYVG5E?=
 =?utf-8?B?eE1kOGdzS1VGT0k0eUY4cHNOZCtQbDJNQTFDUkIrcTBKR1BjTDc2OEU4b1gx?=
 =?utf-8?B?d3FmRVhBNW9LTXdhbERBbzFtbnRhNWR5MmRpTkIvZlpEU0tqVzBmVWpiKzNv?=
 =?utf-8?B?clVRaWgrSDZsdjVHWXZtRFFPRjZRVkh3cE9KKzIxKzhMUkpEVDdNbWpjb2xv?=
 =?utf-8?B?RC9mVTc2NmpDMXpUeDMwRS8rQjBKYXRJQmRJbWtyWnV0VzQvazB5cVY0VnpQ?=
 =?utf-8?B?RkRUUVJFWkRHZ3JRRWIwbnhYNXJwU2JVM093TlFZTHBmUnVkOVZ3Wkd6cTFk?=
 =?utf-8?B?WUN3Rjk2aTdtdUZEZXFQem9MK3RyQVVIYzVMYlcyK2RjZDRlYXlDOVZSL3lU?=
 =?utf-8?B?Wk1CakZjQ1RpU1d4VmJ4TWpuRW9hUzlrUDdhYk1LZzVJZmVkSlMrODUwYktY?=
 =?utf-8?B?OVoyQUlLWDNOQzNMM1cycFZZMjhKM0p4OHVITXZYYWJPMW5FWHRpa29reVpX?=
 =?utf-8?B?eUxOOHJsSTFsT1NlZkxkb245cnpkQVRUaW1GRUhjbnpRM3NhbHpEMDA0cHlO?=
 =?utf-8?B?MFpJZUJBY1J2UWVoU015ckxsT3dISjhuOTZXMTVnNTI1a1dQVGxQSklXcVVE?=
 =?utf-8?B?YlNYaFV5SGpxVnVMeGE4d2FTWEFyYzVvUlhUbFdCNGY3akNqNlRoWXptZlFp?=
 =?utf-8?B?Y2FuU3lmZ0drejMyUTlad1dReVFpN09WLzdxdkN4WFBDdDJMMFhJUUx3YWx0?=
 =?utf-8?B?aVlEeEovMmJyYlJJSnVyU2NvTGoveGtsc3F0ckYxWXpHWDVDNE5VUURaTTV0?=
 =?utf-8?B?SzJXVVdxRllTUFlXeGRNYjZTcjMwek5OazFaQjdabVBQNFJVKzRGcUVsUGV0?=
 =?utf-8?B?aGZSL0ZYR3JpYnhqRGxHK1pFRXRnZE1HYTlweE9XME9EcWF6VVUwb1UxU0tW?=
 =?utf-8?B?MFdxUEJNWmd0VWkvUWNSUXFic2NFck50bDZJcXJGV3BSK054RGtMSlovWTg5?=
 =?utf-8?B?YWtTenhRaURrT3pqYTFNd2lQZHE2RGpTZXVHMHdEUERNcVVoeWNTUjFkdnZX?=
 =?utf-8?B?NVNYQ3pNcFJJV2UvM0lDSGFpVUhKYjgvNXh3dGIwWWFRYTVvcnZrNW9kcWxJ?=
 =?utf-8?B?NnFrS0VSSVdPUnlJUnNDMk8xVWc3OW05TTh0Q2lHQlZLZVE1cHEwT3lBUUlo?=
 =?utf-8?B?eHJkSG9oZnErWjIveWY1K3JyOXBicjZ6ZmRpMGlReXhwdVdWSVhTa0hMSjFa?=
 =?utf-8?B?TzZ4MDB0WlNLZWtSaythR3ZNVFdOanRnSWNqWFlvYUswbTZacEhMdGxuTEx4?=
 =?utf-8?B?a3FuaG8zVUdpYWYyZGE1Z0R3bXNldVdldEQ4dk1GMWVqSTY4QklGL2JJREhY?=
 =?utf-8?B?R202T01KRkVIbUtoZnJqeTZ4WXFyb2ZkbGJFMCtSOTdPZ1praFlRV0hwamVC?=
 =?utf-8?B?RktlVGV6ZytSYVZ1Y1JMcWZpY1VFaEoxRDlzK2tSWGFXelZoVFJ0NnZSUXVo?=
 =?utf-8?B?SzVUQSs5Zm5zaEhiS2JYWFp4YWxLVldDQjdQMnJBWUFxc2VvNXBJRHh4Y1dj?=
 =?utf-8?B?ZVd5N1pzL2hRSGdSSGlMKzZLbWJZM0VTTTZvK0JacHhRNnR1WU55R0hLMmJF?=
 =?utf-8?Q?S1RT0hO1zSf6tcDczLzCn6VCiiVJSRmErOX72ky?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 509c5b08-113c-4e30-cba3-08d972ba8eba
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 11:19:43.4875
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hgAZm0I3ZKpnUbdJOncACtXe1FUan7ZUNRPpYSNDTTSOKgF2r2+dhjJ1ye7stszNXjVvdOjTi3l8Zu6uSoLlMwSfV4blHupozss/ffBJfCc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5663
X-OriginatorOrg: citrix.com

On 08/09/2021 11:43, Jan Beulich wrote:
> On 07.09.2021 18:19, Andrew Cooper wrote:
>> The opencoded legacy Memory Disambiguation logic in init_amd() neglected
>> Fam19h for the Zen3 microarchitecture.  In practice, all Zen2 based syst=
em
>> have the architectural MSR_SPEC_CTRL and the SSBD bit within it.
> Don't you mean Zen3 in the 2nd sentence? Otherwise there's a missing
> connect between both sentences.

No.=C2=A0 Zen2/Rome has MSR_SPEC_CTRL.

The point is that Zen2 and later shouldn't be using MSR_AMD64_LS_CFG in
the first place.=C2=A0 I'll tweak the wording.

>
>> Implement the algorithm given in AMD's SSBD whitepaper, and leave a
>> printk_once() behind in the case that no controls can be found.
>>
>> This now means that a user explicitly choosing `spec-ctrl=3Dssbd` will p=
roperly
>> turn off Memory Disambiguation on Fam19h/Zen3 systems.
>>
>> This still remains a single system-wide setting (for now), and is not co=
ntext
>> switched between vCPUs.  As such, it doesn't interact with Intel's use o=
f
>> MSR_SPEC_CTRL and default_xen_spec_ctrl (yet).
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:44:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:44:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181798.329107 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNw0D-0001Ed-CU; Wed, 08 Sep 2021 11:44:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181798.329107; Wed, 08 Sep 2021 11:44:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNw0D-0001EW-9I; Wed, 08 Sep 2021 11:44:29 +0000
Received: by outflank-mailman (input) for mailman id 181798;
 Wed, 08 Sep 2021 11:44:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mNw0C-0001EQ-7C
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:44:28 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mNw0C-0003dY-3k
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:44:28 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mNw0C-00027L-2m
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 11:44:28 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mNvzw-00032p-8M; Wed, 08 Sep 2021 12:44:12 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=JDArQN2Fi8ETRRDga4y+qn+1GcKXiOotD1ShURS9fiU=; b=IzCMJz7hDCG1qM3Wf7K4wE09pV
	GFw8W0j9XPOgV0PdRYRbB6PpYsuirC7+cmon39IsfsU8csIQPcbqo7tVpHMQAcaCzyfxlY0g+iSTZ
	IKVIK5KKb+vNGLVO9lM5l7bRlDqbep3kmSZjVyPddTZWUloheapGEWqqZNdECfdg81Is=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24888.41483.762554.171248@mariner.uk.xensource.com>
Date: Wed, 8 Sep 2021 12:44:11 +0100
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
    "Tim  Deegan" <tim@xen.org>,
    <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a
 single .c file
In-Reply-To: <YTibMaayiWlJwcZ/@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
	<20210824105038.1257926-6-anthony.perard@citrix.com>
	<e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com>
	<YTibMaayiWlJwcZ/@perard>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Anthony PERARD writes ("Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a single .c file"):
> On Tue, Sep 07, 2021 at 08:14:14AM +0200, Jan Beulich wrote:
> > Hmm, when replying to 00/51 I didn't recall I gave an R-b for this one
> > already. I'd like to restrict it some: It should be taken to stand for
> > the technical correctness of the change. Nevertheless I'm not really
> > happy with the introduction of the various tiny source files. I've
> > meanwhile been wondering: Can't these be generated (in the build tree,
> > as soon as that's possible to be separate) rather than getting put in
> > the repo?
> 
> Do we really need to generated those never to be change tiny source
> file? Do we really need to make the build system a lot more complicated?

I'm not an x86 maintainer, but my 2p anyway:

I think the handful of tiny source files is probably better than some
contraption in the build system.  Much less risk of something funny
and confusing going on.

We could reduce the number of copies of the same text by making the
copies of guest_walk*.c in hap/ be symlinks to ../guest_walk*.c.

> Can't we commit this patch as is? What kind of issue is there with those
> tiny source files? Should we add a warning in those tiny source files,
> something like "No modification of this file allowed, it's part of the
> build system." ?

I don't think we need any such warning.  No-one is going to take that
tiny file and try to edit it to put functionality in it, and if they
do it will be spotted on review.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 11:59:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 11:59:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181803.329118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwEF-0002xc-Ly; Wed, 08 Sep 2021 11:58:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181803.329118; Wed, 08 Sep 2021 11:58:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwEF-0002xV-J1; Wed, 08 Sep 2021 11:58:59 +0000
Received: by outflank-mailman (input) for mailman id 181803;
 Wed, 08 Sep 2021 11:58:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNwEE-0002xL-Bn; Wed, 08 Sep 2021 11:58:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNwEE-0003rv-3a; Wed, 08 Sep 2021 11:58:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNwED-0003xD-Nl; Wed, 08 Sep 2021 11:58:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNwED-0004ut-NG; Wed, 08 Sep 2021 11:58:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=bqj3Pxw7uKK0Dpen6uKCu2Qw66bfenQpSvzRcA97Vgk=; b=y0rRAw/lEBWDfukNoDZ1/5Uvd+
	w8Mseuc7FZQyeKhM275N/mfjZ2PqyaAqQqkpNXHMEwbP/L/dGnfQNCiFJz1MJ6FTwYr60fx8ziA46
	M+ng0sF6o+rPN/ryeJzF3knpZ2SKXooR36h/wyRZjqnB6WvyWFUHDXGAFaXdYwmg0a44=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164878-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164878: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=abf7aee72ea66944a62962603e4c2381f5e473e7
X-Osstest-Versions-That:
    qemuu=a61c30b8c8c3c8619847cfaa289233cc696f5689
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 11:58:57 +0000

flight 164878 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164878/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164874

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164874
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164874
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164874
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164874
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164874
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164874
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164874
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                abf7aee72ea66944a62962603e4c2381f5e473e7
baseline version:
 qemuu                a61c30b8c8c3c8619847cfaa289233cc696f5689

Last test of basis   164874  2021-09-07 12:37:56 Z    0 days
Testing same since   164878  2021-09-07 23:39:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Bennée <alex.bennee@linaro.org>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  David Hildenbrand <david@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jared Rossi <jrossi@linux.ibm.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   a61c30b8c8..abf7aee72e  abf7aee72ea66944a62962603e4c2381f5e473e7 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:01:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:01:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181810.329131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwH2-0004dt-Dr; Wed, 08 Sep 2021 12:01:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181810.329131; Wed, 08 Sep 2021 12:01:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwH2-0004dm-Az; Wed, 08 Sep 2021 12:01:52 +0000
Received: by outflank-mailman (input) for mailman id 181810;
 Wed, 08 Sep 2021 12:01:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNwH0-0004dg-RY
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:01:50 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8bcd4f9e-109c-11ec-b14d-12813bfff9fa;
 Wed, 08 Sep 2021 12:01:49 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-EpyZuIrHNUGasRVrBYmQiA-1;
 Wed, 08 Sep 2021 14:01:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3934.eurprd04.prod.outlook.com (2603:10a6:803:1b::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Wed, 8 Sep
 2021 12:01:46 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 12:01:46 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0074.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Wed, 8 Sep 2021 12:01:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bcd4f9e-109c-11ec-b14d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631102508;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UG2AjnDcsmumIq1uOUgi3tNiDykLsZsmE9tn4WXrbnY=;
	b=Tj9qRJJTwa7nXMRN/PsTOh+HYlGHXcPQIH3gZ8q2wODdGy/P+CEz/E0NQjfObd7ndmwf+l
	QA977aeGspxNviOfN/bE0xiE/aZFqbMs2lDAMnIEZxyyGQhAMpAc5+bepZgB1FLRlkNDKw
	duUQExqJ0z9wrS4Wsa2U4IVvIaq6TmU=
X-MC-Unique: EpyZuIrHNUGasRVrBYmQiA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CMplft3RF/qv2zfzrZG9BISwNk9psfoiMDxyjQNgzd32ZmLLxBRFhQFHcfuytoCw7m8qNsOICjVc/P1D5fOzOks8uU2xXhWG39EqwAVZnZtQ0/X5sZAyCLcfhS6HwSu8qpAydQCLb/iAL3w0fKcPYMAOxy46M6+c0Bk0YwMaBIcTC8a/A8EsnmNseZR4pgu8WVcKi2B5RAjtViYl91l8VcdwHFEer/Ki9WkQRFlLb/T6JAJentpEXVhWASbt2LvvQKYDOYuY7jK36qiKs4JxoowxPrRkKBzvWynplWE0Fb+/sWfYqKZUsfZFUF2NFD7nd/qlNR+Lti+H2GyvjNaeeQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UG2AjnDcsmumIq1uOUgi3tNiDykLsZsmE9tn4WXrbnY=;
 b=dGhhtLCGDj55ld7Vf4r4w8y7UgzkepZhKMdjesP94Wxg3pyZjIEFvPJZyDq8pWxVbdSZ6iI2D/AviePDJdCmtL3N2KKLa1JLnRriTP4CVzS1imlhWgqDJdYU56H0Pd4mKBTyZmk8nTG1i3IWOtaiXmf479Kp/34H5209YFVpTaIL2lcgkOS4ZZlu5zMZJVBP7DxN0NAPDYU6DI2ewa4z6YL3Vng6kTS58XLFYdAQ1XtVmN6pSmrQ5nnI4r+QFJi1p+v8/HAli2it7qFyL0xC72j7LkgJZ2e5M5M4A+gP/F7CxQ/ZUp4JROyaPb4kDh/o0ReSq23CzM6kYhwmSwro/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a
 single .c file
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-6-anthony.perard@citrix.com>
 <e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com> <YTibMaayiWlJwcZ/@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <27874b1a-70cb-e647-d271-93ef12dc40dc@suse.com>
Date: Wed, 8 Sep 2021 14:01:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <YTibMaayiWlJwcZ/@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0074.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 43cc3789-7638-45b5-2273-08d972c06e3e
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3934:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB39349BCF6F41B1D3B00EB2B0B3D49@VI1PR0402MB3934.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aEem4ObOgpR51fzyBDOoMX0gVHcgBbWvsMJuiqlg3xDjRwAiFieylvQoC5wPeI2Yhjl0GHckSKgkbOKPYYKSuFxS7R2Gy+O4LoxsI0jQjmibhVV1xxDQZqf8T4x6+MVrFM6b52TF8PsxTOcta1+mvXX5UGJ1i1TT5Gd6gZetDwWC0yDwR7v2xKMd7CAQuF4qsuyhwxegyW6Nqb+1kDIUL22vLNdeO5VI7sQB5pblMpyemRMTwSKFYmRCLm2l0rPFn6uIR8u3yVKgKeX4ccs8fDm0mrkprMv0eeQVeDAHyoovaO+woa8FXEYNbFLNEbjmwLfWB1Drj0LCi1gUylETWG6unlDrQZvZo/WfOy/54R8hfTBuslpwntOjWxWQfci1M8rkWhXH93mTZnuBlr755tIbala7f/b2cZvorCBJ94dqRrH4UppPoRQ+aV8M8lAqeGFpeLaNJM+WSo8A2qtr6bk2O6FS47Z0icCTm1rXLYBCEZoIG0Pqg0ICIg6Gl/QXZw8I+v6JOHNhGa1IhnCmT34Qin6pXQAXmCpWA5/4dkTCRYBpK0r/9o3X0/SM+y/Vds1Y2zTPsrsOUIn3fMURPcGoa5MKQun13ZShJUaDtanGu3umyJPE57FBI5j6Q0scLVRni3jdxPEBQTXc7Rt6r28NH6wDzlQ47ziUpbdPOTlUXmWERjVk2PxRd+Vka+TVPRDZ1dN9HxutJ2fFmBOk6oxEL8/YqWekTfT0DcODqypo/hMmD0Ka/mJyyQ7b4SCQxsX2RFaGoU0z8JWceQFJJQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(396003)(346002)(376002)(366004)(31696002)(31686004)(316002)(38100700002)(16576012)(2906002)(8936002)(956004)(2616005)(6916009)(66476007)(478600001)(66946007)(53546011)(66556008)(186003)(26005)(83380400001)(7416002)(36756003)(54906003)(5660300002)(4326008)(8676002)(86362001)(6486002)(142923001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OHB4ZnpDUEQ0ZTA5dG1kVWRqQlAzV3ZkdWdFUkwxWGdoT215cm95dk5naDVm?=
 =?utf-8?B?dGtnMHdndkNZTUs0Yi9wTEJ5VjR5VXA2K0lGdm1OWWZrakY4RnRGT3FFSUZV?=
 =?utf-8?B?YzFWdWNERGcrU3ZaVFBPcHR0RVNIOGRUTW5TN3cwbVM0cnV5QnBSY0FMMXB1?=
 =?utf-8?B?MmJEWnFtSlFsSm0xVDZ5L2E3SEViNmZVc0Q5Q0Z1Z3RiVmFiWDgwRVlnZjNL?=
 =?utf-8?B?YlNIcm5YTzF0cElZU0lrZ2s1QWNpWXNsK1IrL2c2SlN1QWswRzNtdzk4TElC?=
 =?utf-8?B?THM4MElkbyt6eSsxcVZVNFRTQ04rRU9vdGVZSHFlZDBvdDVyNVJrU2VocjdC?=
 =?utf-8?B?SUtsbWhRQ20wdlFxYXpDODZNSER3ZlRmSFhKeGFiejMzaExpNUtGVU9GVFE1?=
 =?utf-8?B?TGpiQjFtTVZLSFRIdHNTbXRQWG5oUnM5cU9MVTF6Wi9Jc0x5TU51SWdnSXNX?=
 =?utf-8?B?djJ1QjJrcmlhMmZodzRlSm5YSFRwdU0wT0NYNUJIQmVzT0Q0NUJ6OHpBbnRo?=
 =?utf-8?B?NFpvVnRvUCtEYWdVTTJIK0FoNTdUbDNNbkQxdTNnc3ptdW0xWTdKQVhXdWZp?=
 =?utf-8?B?VHlEYmh3NnhsWndjZDB3NS9SWURtYzBweEs2dHhNWVd4NmFNZmY0cU5BTlZK?=
 =?utf-8?B?SC9qbmp3Y2U0ZWRwU2I2SHh5ZjRxYlk2VGJpRzZSS1ZoNkVaMi9GYWV3MjFH?=
 =?utf-8?B?ZWUyMElMc2dxS2ZhTmtmL2krRnIxL0dzbjc5SnBSQjVncVM2WEE0NFFHQjVt?=
 =?utf-8?B?OVRrVkkyOFlublZxb1dsTHJveVBYbzBPV3hMSGtMQ0RIaVdWWXl4eWJnNWZj?=
 =?utf-8?B?WU5zMnpjLzBGUGV4OVhsRXl5UGJZcytCNmE5SVdlck5CMTNuRzA3UnVheG1M?=
 =?utf-8?B?RGVEdDlRSG1UeVR4WVR2RkZtTStIWGE4VG5JcjZrMDl6bmtLdGcwenpPbElZ?=
 =?utf-8?B?Ui81R2FpKytRbU1Nbm1qanBYck1yRXFoaDVJVVEzTnRxU25yMHdkdHFIMWpS?=
 =?utf-8?B?eGt3ZDNKR1U2ODA1bWlURlJmSFhqMEMxM2RMSVJkOVl5SkJJeTIyOFg2N2t0?=
 =?utf-8?B?QzFJcENsSjZvdjZaYWYrMHhTUDAvSGJmOWtUWTlyQ0FodFB2WlFYNHQ4NU9t?=
 =?utf-8?B?enZuWmN6SXVwTHlENUl0ZUl1TnVYaXNNODBGMEJFQ0JvTWJ5b29qUTRTa3c2?=
 =?utf-8?B?bFZzamlvdzFvUnc4VzM3NEorL3l4S3ZReUtaUU5HUkROSHF5S1B0YUFoaEhJ?=
 =?utf-8?B?anJpcEEyS0pnaXh6VExuS0VzYmg0QW8xRTVOVWsxUnVkMWM0K1MxYjV5Ymlx?=
 =?utf-8?B?dDZnWkRBQ1ZmWEp3Y205SWhZS05OK1Q0V3cwUnJQMDFqT04xbDhNUUFmdkFk?=
 =?utf-8?B?WWtVMzVQc1ZhZ1k3SHNtWTNnM3pUVTNKWHkyUU1hekNqYUovbS9MN1lGZ29j?=
 =?utf-8?B?MTBXRTJCZ0VpcEVTQVUwSTNyWUw0SWE2SXBFRHRHSTdHV2dmSlZQWDNtcENP?=
 =?utf-8?B?YXNyQXN0N3lQR0FEMjRITzhDRDlVWXZPWUFsRUVoR2o2aWJ3bU1jTmJZcUhh?=
 =?utf-8?B?eURaeXIrdHA4SXA2UTNyMGlNT0dlOHNZMUk5TXdQRG9kOWJteHBCTy9wSUl4?=
 =?utf-8?B?WnVwRnVVY2hkVHkwVEp0VXFEb2cyK0dHM0x4RW5yb1ZFU21hS3gzVUxrMUJB?=
 =?utf-8?B?M0JrYnJYNlI3b0FrditnVDExY00zTk1FWlRXNy9HZklha1Z1L2RHSXhCYXFQ?=
 =?utf-8?Q?sfSW/8k20o+Yuf4Mo4HZlSp6k2R09QzXqapGITP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43cc3789-7638-45b5-2273-08d972c06e3e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:01:46.0088
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 02L2npVOEtfFZOxAA1NTzSPDMnv1Z/GWW7qlQZHRXQkb73BYvoMlYPttfOC0OyB7esV+EmkNI7MmYE/1qumlpg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3934

On 08.09.2021 13:14, Anthony PERARD wrote:
> On Tue, Sep 07, 2021 at 08:14:14AM +0200, Jan Beulich wrote:
>> On 24.08.2021 12:49, Anthony PERARD wrote:
>>> This replace the use of a single .c file use for multiple .o file by
>>> creating multiple .c file including the first one.
>>>
>>> There's quite a few issues with trying to build more than one object
>>> file from a single source file: there's is a duplication of the make
>>> rules to generate those targets; there is an additional ".file" symbol
>>> added in order to differentiate between the object files; and the
>>> tools/symbols have an heuristic to try to pick up the right ".file".
>>>
>>> This patch adds new .c source file which avoid the need to add a
>>> second ".file" symbol and thus avoid the need to deal with those
>>> issues.
>>>
>>> Also remove __OBJECT_FILE__ from $(CC) command line as it isn't used
>>> anywhere anymore. And remove the macro "build-intermediate" since the
>>> generic rules for single targets can be used.
>>>
>>> And rename the objects in mm/hap/ to remove the extra "level".
>>>
>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>
>> Hmm, when replying to 00/51 I didn't recall I gave an R-b for this one
>> already. I'd like to restrict it some: It should be taken to stand for
>> the technical correctness of the change. Nevertheless I'm not really
>> happy with the introduction of the various tiny source files. I've
>> meanwhile been wondering: Can't these be generated (in the build tree,
>> as soon as that's possible to be separate) rather than getting put in
>> the repo?
> 
> Do we really need to generated those never to be change tiny source
> file? Do we really need to make the build system a lot more complicated?
> 
> With those tiny source file in a different directory to the main source
> file, we need to add "-I" to CFLAGS. (source tree vs build tree.)
> 
> I don't like preparation phase, I'd rather do just-in-time generation of
> those tiny file (if we really have too...) as to let make organize
> itself on when to do things. That mean, extra targets on how to generate
> the files, and telling make that those would be intermediate target
> shouldn't be deleted to avoid the final object from been regenerated
> over and over again (I'm not sure why the rebuild of tiny source file
> happen when they are intermediate, maybe because FORCE prerequisite on
> %.o).
> 
> Can't we commit this patch as is?

I'll consider another time.

> What kind of issue is there with those tiny source files?

To me they're ugly and their presence is at least mildly confusing.
And apparently I'm not the only one thinking that way, or else such
tiny stubs would have been put there right when introducing these
multiply built sources.

> Should we add a warning in those tiny source files,
> something like "No modification of this file allowed, it's part of the
> build system." ?

Not sure this would make much of a difference. Them getting touched
later on is not a primary concern of mine. In fact once they're
there, I don't see why they couldn't be extended. At least in shadow
code there might be functions which could live there, reducing
#ifdef-ary.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:09:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:09:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181819.329143 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwNt-0005Lv-6f; Wed, 08 Sep 2021 12:08:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181819.329143; Wed, 08 Sep 2021 12:08:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwNt-0005Lo-3C; Wed, 08 Sep 2021 12:08:57 +0000
Received: by outflank-mailman (input) for mailman id 181819;
 Wed, 08 Sep 2021 12:08:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNwNs-0005Li-K9
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:08:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 89de3fb2-109d-11ec-b14f-12813bfff9fa;
 Wed, 08 Sep 2021 12:08:55 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-KSshkRI8OLC-HnKJ5vqB0g-1; Wed, 08 Sep 2021 14:08:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3391.eurprd04.prod.outlook.com (2603:10a6:803:3::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 12:08:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 12:08:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0118.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 12:08:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89de3fb2-109d-11ec-b14f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631102934;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LsHY1ULNA9PA1Ool8eII2TD7Ugxn9D41f/uPUTi33jM=;
	b=YWftrmIP4di7d4Fd536QsXelGACMctDixAInHCegIy7jMd91JkPh+PVdASfzlXtq7v+luS
	OGgKZNRe+JsQtzcj1HrxWI7Syma5Oo16FzHgtl4tZA4GoYfEfDz99dHoOwiV+YMWnq703O
	bp/A3vIxy/qkxxbaOQP4jl4dbZFyD80=
X-MC-Unique: KSshkRI8OLC-HnKJ5vqB0g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nJihRneq+s3lDAE3y6DluchPZzJYlXMOZIrjvgorEk0u6zLxjS/CDatI+zgIMnfS9NGXIrwee6TD++eftth2SiHIOjxPKtqnHIzfji+zJl/humVISBWlo3bVwe1ZwWBXo1dsAO2Cgb1RoXcGp9+4Fj1Rl5c/UM/EMp8Fs/M4KiJ722RBjUq4aFSFU+wo40Gz7H03KGSYTySIydFF2SZnuNDi1mnUBo2TRYt2oROoXvXE1FFo/t2yLx7fSQLwWAeWjjLRh+PbIZ/UuaHZ5mAGJ3kjpKU+aymvMlqXIfXYPUTpxuAVihNyUr698jOlzCklXLu97Q6+vtfNhqSzu2KG/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LsHY1ULNA9PA1Ool8eII2TD7Ugxn9D41f/uPUTi33jM=;
 b=PN6I8pMpZYuM7TTVEfrbTomjKU82d7f+3MxVB5y6yTyA0NJHNMcr4ASap5op0MW/d0YwZKUeGokwfB+lJekbbTv8L8ENbeQBVcgCQeMcQ/5RjpxEvO5gxHHEeyHxO9ItSsl1W5XsUm6lVzEDF2jOyibd5QhRaotsE4PuFrf4VpTaOh7ZskKBPRXR/NswEE9mt/isLd07ed1/BLj43knaxqMVwvhySMk/QXqmCSWHJ5TsHe00wuQMdbdkG3+4Ctlr/NPFXjind6k66T0J/4v1TtZSNqNLw1XdPaMU6qQJWwQucPZ+h0La9E/8+qMRlE/TXSTUgHci1vi0NcFSqPJMEQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
To: Jane Malalane <jane.malalane@citrix.com>
Cc: Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Pu Wen <puwen@hygon.cn>, Andy Lutomirski <luto@kernel.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <20210908081945.11047-1-jane.malalane@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a27d3631-86a3-cacc-efd6-97b02e3f8b94@suse.com>
Date: Wed, 8 Sep 2021 14:08:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908081945.11047-1-jane.malalane@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0118.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 85a39ec5-6f67-48b0-f35d-08d972c16c16
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3391:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB339110BE9FE200C9D583122CB3D49@VI1PR0402MB3391.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NeNUTTPQApBE+kcj+NU3afJ7RF1TftV3M0cyLgdLNj/Jd/WGAZufSvOfS531O4o+xdymlO77+fZlfxS1aeEd01eIYJDDhZYRoGfSv3Ni/yb0z6M5MIpFpmX8QnB7oYtyOOl/3o1+3U7KQOdf5871GYfxsdelOXOnxOtqh93mj4Qxd+wEvS6kq0RNGwKZ0ZhVEZ4DgEHCZoVDPbl6VEnsNLn3rOejULRZorf7rGUNCDXS1X3CsipHBO3NS9+J6tKpy5QSZUHe1ZJfYFHm4i1pyNBk2gWyT9Y29WPbHZCUhKjSJEfFQUdbcHM7EdrDICkUMSoXcf3qjPBA2mfllghHXAx19hYaAK/UePIAQpSvsbdYc4fNrbMJCfeIDwEOTOKGidyXk6BPoLcLKUh9Qj4qBhgiAF3ZdivzgJnFOl91NfR90jfPUP0SR7Wakv6pEV69ddX3N1D65k2pOg8jxWU4HKJf8vl5H3g8zwAflimYUGgLCSI2/RLUSztkbcmfEIhl0XYW8vHxhf71sWqTi/yf+IHLRxFgjtsNTdTi9YPtayfIT6jV0q/8NslRXXh4gHDqm/XUCIsCAcyaQYM/gxPz765Kf7019MeCMUgO7mfs6HnnAEP/ZdQb3pjX67NlqqWE7vwVlUDvHqE/cBkkEmRClE2w2GBwok6rd/JE/iXVFA6SGc1ogvHiR7vlFCRuFay5dOAPS0lrOYC7aY76ODXaaKdaYww5fme5N+QA3D4ba6W7qQOGNahmD2bmShlQubfQ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4744005)(54906003)(36756003)(186003)(6486002)(66476007)(8936002)(38100700002)(5660300002)(53546011)(31696002)(26005)(66946007)(4326008)(31686004)(8676002)(66556008)(86362001)(2906002)(2616005)(956004)(316002)(6916009)(508600001)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WFNtVkxUeTBFZjVsQWpuaCt6Z2Noa3dWdThNQVNldlorb0Q5ZmZmNTlITDMz?=
 =?utf-8?B?cEEwREp0ejlkQVU0d1hsL3l6dWhVcmsrUWV6bGtrc1VieHNhK3ppNW1jNVdY?=
 =?utf-8?B?VXNWY2IzZDU3SHNUQXpELzdKcVhUTnRUb2RLc1JpaGtLdGNyWTdDd0tqemdp?=
 =?utf-8?B?MnFQTGZEZHk0T1cyYjVYWFR3RTh5Y1NZZkhWVzhJK2toaEZaSFlkU0hjRCt6?=
 =?utf-8?B?WUw5Q0tkeFZrYWk1VGo4Zi9lbkViMWFnaTAwZXRBWTFPeWd6MnFkeE1jTFFT?=
 =?utf-8?B?VG91NFlLTzQyQTBlYlNjM3NOVDhsVXl1ZExGS1BMQmtsZWRidEZieWVMZS8y?=
 =?utf-8?B?OXNLTkxua3RJZWtPd2Mwa21YZ2VJZUtxRmhpNnc4VTVxRURvTklPek5IWHM2?=
 =?utf-8?B?L0poaXNFVmVucXlTR0Z3THYzTlAySnJIOEYxcU40STBqMG1lUzNDZ3ZmK25Z?=
 =?utf-8?B?QlBaU25CQkJXbUgxR01zSS9CZlVBQVhOWmRzdmdzTWprSXVUVkphUXoxYlBD?=
 =?utf-8?B?cGt5SFlRNWJORXVJL2ZGc1E0Y1dnME1vVXRYQ3dBMEtkL2IrenFRYnR0Zjc4?=
 =?utf-8?B?QWJ6T3prRk52c3oyTHFhbzA4U3NNRzFhNGhGQ3ZjaFNWRkhieS84Y3ZTOGxU?=
 =?utf-8?B?UGtIVE5ENzdOUWZSZHk3REJscnEycVFpbUk5R2xpM0NOSmhKa0VseXFoYlFS?=
 =?utf-8?B?YTI3dEVNSmhSZFE1RGRRTmZxdEdJZ0xER3hCR0VSdlF5cEJRTkJ5NkRLeFNr?=
 =?utf-8?B?UEJla21OUDU4VTRGT0I2MEo2ZTBHUkVUcEJFWFpCMEtZdE4xUmhIR0UzYXhG?=
 =?utf-8?B?UHBqakMxb3diNE1FcjMwWnZxSVNmUnVlVGgyUUZpTGh6WTQ4YnRZbG5QL0xj?=
 =?utf-8?B?N1QzUENMK3pWeEZpb21BOHlxbE5WOVJ2TWhvdnh2dHNQUU9jdUVjUUQzSVpt?=
 =?utf-8?B?K1lRT2pqbTR0QUxHd2NVU1BhZEVMMWoraEl6b2psU0JMVHF4UEloWnJ2SUhS?=
 =?utf-8?B?c0o0Z21SbnBjOW9TY2h0a1prT3Z2dDdTMHNFeEhTRDhlTjA4ODJsNld5MXl6?=
 =?utf-8?B?cEtyb0hxOW01Nk8xYlVGZDFyUWxYbnprYk8yVS9oTDJqeE5ZMUdJRUlpZ24z?=
 =?utf-8?B?UFd2ejB4VkNqSXlFaTBZa212MUVYcUVlVElzNDlFSjVwL1VSclNOOE8reFQv?=
 =?utf-8?B?WUNqdWkxT3NldDd0MmVYTk1SQnEzcXBuc0FyWk55SVovdHFQMmxoY1FUZUtP?=
 =?utf-8?B?RUhhWWpyS2l3eTdra3I2dlBPZG1BOXc2TFMzWDlGaXdLWkl0bWJ1YzlDRXcz?=
 =?utf-8?B?alVwVHN6MTFJaUxDS3FpcFdsd2V2bGxOTy9kNC9JOXVPSmtTV0FaS3BCY1li?=
 =?utf-8?B?TTMrQ0x6Z0pPcFVwYmJGZ08rQmlxQnRuSDBXRXZ0a001Vkk5Z0NFdEhkTE9j?=
 =?utf-8?B?VDl4TnQ1bmVlY0dBYVpMV1ppUUZGNElMRGdjNUtlL240T3pUdTlSWTU3VVVt?=
 =?utf-8?B?L1J3L1M3Y0lDQWhPZXJQWUZXcUdFbTVwbkxucDA5Yld1a0ViUDJnOXJNdzRF?=
 =?utf-8?B?ek1SRlNmaGY2RUFzRXBtY3UwSlpGUFBMc3BOaW9TeW01Qk85Skp1VjNOTVN5?=
 =?utf-8?B?Rk0yVHVFbmFZOUswamV2d3ZLRDhMRFBPL2Nkd0NLd2Ixci9kTW1IanErbWRv?=
 =?utf-8?B?R0xSejZjTVpHSWhHNkVyaTlZR2RiL1BYMS9QZHo2UFo5M0VIN3hKSysrMDRa?=
 =?utf-8?Q?EStC87hSkb9gtXxkH32cpU6IO6ZLL1r8tOznUJV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 85a39ec5-6f67-48b0-f35d-08d972c16c16
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:08:51.8513
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1UY2KpoZxkmIJvWDigDehxM364KpxmMNd2zOvvDRn7I9+GH5kjFESdJhhWyWI8LW9N1bdPBta3zFtRVIVqvSXg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3391

On 08.09.2021 10:19, Jane Malalane wrote:
> Zen2 CPUs actually have this behaviour, but the CPUID bit couldn't be
> introduced into Zen2 due to a lack of leaves. So, it was added in a
> new leaf in Zen3. Nonetheless, hypervisors can synthesize the CPUID
> bit in software.

Considering the prior model checks, I understand this isn't all Zen2s?
No matter what the answer, I'd like to ask that the first sentence
start with either "All" or "Some" (or something along these lines).
Which is of course fine to insert while committing, so not need to
send a v3.

> So, Xen probes for NSCB (NullSelectorClearsBit) and
> synthesizes the bit, if the behaviour is present.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jane Malalane <jane.malalane@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:20:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:20:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181828.329153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwYr-0007n2-4N; Wed, 08 Sep 2021 12:20:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181828.329153; Wed, 08 Sep 2021 12:20:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwYr-0007mv-1H; Wed, 08 Sep 2021 12:20:17 +0000
Received: by outflank-mailman (input) for mailman id 181828;
 Wed, 08 Sep 2021 12:20:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNwYq-0007mp-0w
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:20:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d2a7abbf-a19d-4ad8-9e76-39e7701f3a82;
 Wed, 08 Sep 2021 12:20:14 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-vTq9rxmwPEuZ5Swfl3pAtw-1; Wed, 08 Sep 2021 14:20:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3535.eurprd04.prod.outlook.com (2603:10a6:803:11::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.18; Wed, 8 Sep
 2021 12:20:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 12:20:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0065.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.6 via Frontend Transport; Wed, 8 Sep 2021 12:20:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2a7abbf-a19d-4ad8-9e76-39e7701f3a82
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631103613;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CJNVwzYym6eiXmkVEWFw5XalOdBRLFERy32pUmqqaKU=;
	b=Q1/uisIDEIJuNKmPnulTL8/2K7tKUZsoUFajanfSCRpMhtJqwnJGzBCcZueUBicoMPb77b
	7VMNMJH+LIGC2iX4tndT8zDytwHnK/MjClnckU1DZwembFZL1gSCyWoo6INyTfYCg/uXDj
	rPViR7DNUe4RdOoVSaN5eqmLlbklKKo=
X-MC-Unique: vTq9rxmwPEuZ5Swfl3pAtw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dGcJX8fOZTqIRSx688p3aB7ULkkl6DZ7fJbWlSj1zyODk99N2TzCcwka5Mnd0XN7uasKr+76r58shHYSgTY9Hqszs7SrnkYW8XWzy9WFkRkLQdLesNbeIGnGJgYnGp/DLbfbusBO8HfqaCfn6pTuNsuMaVlPnLkpfIIZFdOlnNQsjUypJptwcOKMrG4ywh1kA7NguD6OmA1L2hRCcFAKtcE99XnLB/MrFwBQSat4/hFXV3FQopkIItNl3V03UO+/VKUFT5mVsKnBhGdarVYNhrEd3zL53inxi08MCDwb5zrhAnirr3ILd767pRJJoZt3rT7lZ/P1V24RHUA9gtF45w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CJNVwzYym6eiXmkVEWFw5XalOdBRLFERy32pUmqqaKU=;
 b=IYHghuzRF7ohVhDMS6BzBOVovru3ucv3V2iGjNGyfoK/fK07Yhx+yn+b3nMMdZvIM1X/1yPYkrLjUrCmYnBv6HW5ed/fEfPo59UKliCV1jSEBIbtCJMO+8DSGg4YVfOR7WyTJjBHbPebT4XzIrxHxv1rSqWWPnohfq8PyU24yzWi/Yb3bGkV1yHH0PTRfHVV1DYfebTO2bfw9afqI5OxD9u7pqI9MFu1NzB++8eTPje0Ao75to1MgGvunoh6wpRhECFnZ+AGgZWNeq4EJnAOgXmNqwd0bIuyJFwyr76N9NkTuv5mzNMP2wXYEwNlcIa4sG2aLd+VBnxmxl3DPtQB6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH 1/2] build: set policy filename on make command line
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
References: <20210908095422.438324-1-anthony.perard@citrix.com>
 <20210908095422.438324-2-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c6f7e54a-36ac-aef1-98af-e8644cf34487@suse.com>
Date: Wed, 8 Sep 2021 14:20:08 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908095422.438324-2-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0065.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 38c8422d-c949-4027-078c-08d972c300b9
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3535:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3535C75DF67CDA498EC90DE2B3D49@VI1PR0402MB3535.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:962;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	G6C0Bl6MOu1H/zU0qz7Qikd4dfeQAwzv9Jf3NYDvUsxIycgJXYbsKmeizho2V1EVTL/AIxYRooO+GDznx5tBWdrrLI0RsmKM0marQTiW4vCjn3OC0w0M4y9NtkhJeekd3Ii1ZFYhLYtyqqcgWsPBmW1MqmMpm81tq58UBebOHNreAlDFRC5GeJ4BPTwH2thorigH1QtO4d06puKfyp9zcLpQUsupTQ02/TM229FfmEw7J/9KW7A4545ELuMxB6uuGcKY40HsMwRVSayHdNZaZl89TaQugBE8m7L1o0iEq/9tnA+vB6WudjSrBMUTLHmPK9WuLnpmTnbrUBJGX12bAqtvGku7sNTblzNSKYq4jFnjnVmiF+DSi4HIjhEE9+tTxaDiTuDrofWeS86EzYRO4oxKgMRdZ7WlPm7rBotm6I64tyIf9tw2e6DAuh8qUOv37bz5Ov7DkiyIpYobkL6rDMdog0ma9Aru8X9fvXj5uGkrGS1Exkco9ELNmLGewh21/1qlAruIwsipC27qHQDTWptHjCtKl4BduQ3TqIcnKVkKeatF9BUlQ7eTSdJ5idEC58IwCJP596NayLiQAPYTDEjDjhb61IyTH7QFE5bFik/rFbzm+rH6rdsLoDxs7X6EaJnwWEpa+IvhM7rXkfjeSx/tyMQWLjej7nJGv/7DSFHK092drSWkhbNjJx77ksI9q+tiIxKiTTLynbVSnCDgTGx/0RJyxExFV6z8pBXBodE8yREuwtEvBqZFw9w36jDz
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(39860400002)(346002)(136003)(366004)(5660300002)(4744005)(26005)(66556008)(31686004)(186003)(31696002)(6486002)(86362001)(8936002)(2616005)(956004)(53546011)(36756003)(4326008)(2906002)(478600001)(6916009)(316002)(38100700002)(8676002)(54906003)(16576012)(83380400001)(66946007)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QlZjTEErblNUd1dxcGVlRWpqUnBWQ3kxOXB6ZHFSaktPYlphU2wxUDIwTXJj?=
 =?utf-8?B?czVaUSt1RU9VNER4dUl6VGZlakRRTjBWRXFpRUdaWTA4Vy83K1Zhb3MyK21k?=
 =?utf-8?B?bGlrT2l0em9EOGZPd0pxcnhXOWpPSFczaGwySGN6YVkzbkJsVXZDWTk1ZHVu?=
 =?utf-8?B?dERERGl3T29OTGZCaTl1cWd1ZGZkN2UxRW1YZTdvejUwUEhMUGdBQlNXYmY1?=
 =?utf-8?B?bDcrQmpXVHJtcVNldGt1YjY1U1orSzVJM1FyQjU3U2x3eTVRTVhBZzR3UWIw?=
 =?utf-8?B?dXdnRDJwRTI0V0dRbS9NN0E4SStNaTZaazd1cmM1REFkYXJxOGptVDFSWmZR?=
 =?utf-8?B?KzRGZGVtR2VWN1QwT05IblBkYTNRQ1FuRjNUR1JqSzNVQlpoanZOdzdoZ3RO?=
 =?utf-8?B?Z0hHOUpBZUU3a1dCYzNucjZxWGtXYk9GTm1Oc3NneTlGeEx2SnVBRURQY1RT?=
 =?utf-8?B?MlBlV3J2RkIrTllod0Fld05pWXVWWjJ2a2E5REdhY0JHK29HZ00rMy9hYjh0?=
 =?utf-8?B?amJlZmdmSlRtU1RocXlMSzhmRUdZeFZKUDE0U3liaDlnV3BISDBubnozMVJG?=
 =?utf-8?B?Q0xVcUh5ZlRDMUJ6VWpIYWdYbHl6MUpsVzF2SmhIcnJRK3JQVzI2KzZnYXRS?=
 =?utf-8?B?b1pVUjFYM1ZONHRDaXJMRUtZWmhTOWV1WWc0OWlIUHpNUy9yOWxacUhIVDM5?=
 =?utf-8?B?b0dDYUxrblpaelp2WGNCRGloOHRzWGtydHMvdiszcFVsMTVCUURaY2tBTmhy?=
 =?utf-8?B?K0ZpcFhZK3ZpYWlsK05WbmwzcW12SFF6eEFuTnZsYzN4USt0V2F5RUNIVHNL?=
 =?utf-8?B?Z0FPMU5aODlTUCt5RHFGeXFtS3RLTDg5RTJKb3hlUlVqUXZRSWNQcmYxWnp4?=
 =?utf-8?B?S3lEcjh3bVROQld4STBuYmFmNUNxa3VPbFhTenIyNVZBcXdGdE1sZWhlN2lV?=
 =?utf-8?B?T3AzWmV2MnZlNUxyWCs4czBPbVBGSHplQXFoRTR6Z2VBN2wwczdiRjY0dUlz?=
 =?utf-8?B?eS9HNWFuaGVHS2Z3OVlYaE1KNWxRbzNhamI4bzRhWmkyVXVHWGlWdys1TEVo?=
 =?utf-8?B?dk5KNmJzNEpzUExSd1BFTXZXRVlFeUxEMEU5azJyWmtVcnV0cmpJY2Y1MjVI?=
 =?utf-8?B?OEd2K3FxKzdMUVV0N2Y3RGtmYThqdjJUa3B4WURQTFpjejhUK2EwWlpza3FQ?=
 =?utf-8?B?blRpdFZGNGF1ZnlDWVd2SG9HOTZUZ1lVYlp6Q05JSE1ING11Y1dTTGM4REk1?=
 =?utf-8?B?U243bTlBeUpKRTJJdkdWUkVyTFQ2eFBjKzBjMGRNd2k0NDZ2aGxaQnFsUTh0?=
 =?utf-8?B?MlNjWStGRE5sbEJQaXFsL0pweEorcFZRcC9YTHF0cWhkZVBoOWdwY3NJREJM?=
 =?utf-8?B?NXhjOHlSNHkwWEF0L3kvT1lkWndwQ3VaQjhoTVU4YWtkYyswMGVDL0ZxQjhj?=
 =?utf-8?B?ZTQ3V25oUS94NzFUVzRmQko5Wk0rWEVOVGsxRnI2SWNYUnRzdUFNMUhNRGVO?=
 =?utf-8?B?YjZsM1k2dEJSTFdZOEdHQ2lPcGRCMWNwSWRyeFhFTEZvUXpCUzIxVlVnQWZU?=
 =?utf-8?B?OFp6STBucE1pTzhqd0YrS1dYUkZGQUo0Mm5yUWpRRU9aWWpieDVoYWdzNXY3?=
 =?utf-8?B?RnQyRHg4WGtBNDVxVVR6bGJOeVAxMktMK1Y0TElRd1JHcml1OTMxbUpBQ2N5?=
 =?utf-8?B?RHhZNHJMOGpOMnl2QjNvTUNpU25SNHlDbHg4MERpZXhhRks3Qml2SWZyV0Y1?=
 =?utf-8?Q?k3M2gvS9xJVoyNs/DlsIZGcPQHajroeCvhdrun4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 38c8422d-c949-4027-078c-08d972c300b9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:20:10.7430
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HYY08AN9YVu8leTHukp62OED9PF/ZJIMhiDHVI7yF3n3AbjYElwvlI2v0YYs/LiKjUgbMzE9tK41k+0Xc4f9Xw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3535

On 08.09.2021 11:54, Anthony PERARD wrote:
> In order to avoid flask/Makefile.common calling `make xenversion`, we
> override POLICY_FILENAME with the value we are going to use anyway.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -42,7 +42,7 @@ FLASK_BUILD_DIR := $(CURDIR)
>  POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
>  
>  policy.bin: FORCE
> -	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C $(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
> +	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C $(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)

I expect that you wouldn't mind me wrapping this overly long line
some, while committing?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:27:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:27:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181836.329164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwfm-0000N3-0O; Wed, 08 Sep 2021 12:27:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181836.329164; Wed, 08 Sep 2021 12:27:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwfl-0000Mw-TX; Wed, 08 Sep 2021 12:27:25 +0000
Received: by outflank-mailman (input) for mailman id 181836;
 Wed, 08 Sep 2021 12:27:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNwfl-0000Mq-2A
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:27:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1dbb416b-10a0-11ec-b150-12813bfff9fa;
 Wed, 08 Sep 2021 12:27:24 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-Kh1UnnC1OVWmZKqCrQgLOw-1;
 Wed, 08 Sep 2021 14:27:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4382.eurprd04.prod.outlook.com (2603:10a6:803:73::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 12:27:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 12:27:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0001.eurprd09.prod.outlook.com (2603:10a6:102:b7::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 12:27:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1dbb416b-10a0-11ec-b150-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631104043;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=qbvvTZjgc6qM9fIxlneSsnWKvnclSGDEy1xCwQbgaNc=;
	b=DWhwdKFnW7t8Bn8C+IDVsG6R51giB0RA54MA0g4zIiVGHexe6Omdmof4Mauc4ubwA6SxFM
	Mr8GQ03h4mLs5slIkP2KPWmOdnmDxQTseqSj6V2U4B4sz1cJLvvyCIgTpBQW78UVJCc38T
	uJPEKAedLEIoWe83f7nt8NIiWsVYfIU=
X-MC-Unique: Kh1UnnC1OVWmZKqCrQgLOw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y7kjz/k9/vojd9aAtZROHUCRXy4eGpsE2V1LpBQyHrONJy3Hzv99c7BrAEyJgfXOnoQCDX86k9cBw8nzTk4DPZzphDI9g6vGEC4p6KdWGmSTF3JZHQqSx5Lusicy/0KCCHdb9EZVfL2o7HZ/NjQvrd7B384BX+kdonxLs4b8wpQTePwDl2WglukD7f/2CM8mr2WztrArHPj/xvEyT71WAPjx1DRcFifqld1NWtmRmwJwd55Sq+/PS57RdCIUUL3cPeyu4KymFbj449NonxQr4RDCJHP2h8T46ihDk4F1iJ9FZJuirsiBVxJyRPAZGSrzKpuXb345DgaPDlh/CATnvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qbvvTZjgc6qM9fIxlneSsnWKvnclSGDEy1xCwQbgaNc=;
 b=czu3gQ1Bg2gV/6QZ5LGsTGiVrR9fQZKtQ7U88Q9awiqB9d/i8JXUB+kEZO2N2p3zc+fBtwfFchjEkIHQXq+Dle2jh7O+SFRjXRGB/3PxQzprs1b1NtyH8Wf5CiEwf8S9CV0E3btlQn2yrAD6X3czydT2wIsBUjTvypxHWFv0S3HczOA0EOPtPHqw+5vSyuD/jvJH4JtVCkc4hq4066I/HRH7LmMT+BpW9LPUgUctD7oCs7QD299Rf3yH0zL46sJHtm2L3xyyy3Gzq2A7ysyRGEX/JyXn6biEnkZmigvfPX+HamI1bQjZibSS1j/4iAjQZIVdESjxWKggujxv81D6Yg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH 2/2] build: add --full to version.sh to guess
 $(XEN_FULLVERSION)
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20210908095422.438324-1-anthony.perard@citrix.com>
 <20210908095422.438324-3-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <be89ae8c-99cf-5385-00a7-d9433e0fc50b@suse.com>
Date: Wed, 8 Sep 2021 14:27:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908095422.438324-3-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0001.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 336202a8-151b-42e1-0f1c-08d972c4006b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4382:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4382F527A677FA604EAFF252B3D49@VI1PR04MB4382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qu/IjHFTD5aPZyr578Yt9XUCJVKvwz1pVsicF2lH6LGJyE65WJJElI5elr1AMgbHGYHQLanTcW4Cluo5iCHnOWWfMX9E2gpNC6+KpMRGroWwqVXmZNDNMDmlzMuHkB+qijVVFYasfwcc/osjy2DDEmsKpUkh4OBywzufAvXYNMOuq+IkjPagFnMGMHEXHybPVJSbQqAJ+v6+5vKVCKrn4668OV+Fgguc9TmaV1WDiTWjKHJkR70Xg+dVOGsfETbjB4FNOy/GaWWSMCBpr1iVdhwHT+OVAN4GtJIPowIrho8XWClwG/rdQMG+DyaMCkQhqbHiW0frAmWXO238EQTlnIZaZzOCTQg8VxlMrPIUDa0jC7RMOb93XNSwwfKkhLKErA//RfnGQRQYmjsV3v26x+jNcut6UMEMwmA8fujG8K0xKKAK96NSL9XWcTU8I4csHVE3GTqyYFtcUss1/6rMQuRXfow5gqtaKOTMKTosPTZrE6UGPB0Nwwv1mmEDcQUmTFEyet6uINvfdeqqyIsW3d1kDpzY9jzcU3PaSM6hZLNNrW6w71ktswsKcAaz+VVVvyf9ucSGRfme0+cC12o+Z2bEVLueuJfT4dgSEzIXVM1FZoZdEUkncmbBFohQ3EyDnXDDqNAqJV1c5G6rYAle75gq6oiB0HtZL7NvvX22H3naf5eIRu2wFciGo6Xcns7DCyk2mVkAL5wmY6hPOouWAI95JcSJmDk+ZDbLyfRdZDjPxDDHTWblfcMST+CK99LU
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(39860400002)(346002)(376002)(5660300002)(38100700002)(66946007)(26005)(956004)(7416002)(8936002)(8676002)(31686004)(66476007)(66556008)(31696002)(6486002)(86362001)(2906002)(54906003)(36756003)(16576012)(6916009)(316002)(4326008)(53546011)(186003)(478600001)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aXhndE1hRnVzNktkZ09qbTJUb1JRTHBuRklKbEV5Rnh6K2lnWExDMnNUdG1o?=
 =?utf-8?B?eSswN215TlRvaXNXdTMwRjJDRnZhUFpRWGVwOCsxVVhJL0lLbWNmWTNxYUlD?=
 =?utf-8?B?VE9IT0xoZ21aYTk2RkYrK25RM2ZWZG0zRzhxTWtGZVBXM21FeUdMb3pSSkw5?=
 =?utf-8?B?VGtyOWpGZHlHNnlCRGsxQmthY01PTXFJY2FsTUI4b3hxNSt3a0xMaENlakZs?=
 =?utf-8?B?L2hURnJrV1Q5blpHZGNVNkQ2ZDFmSThleUVxbzFDZldIUWJORnhiUWR6aW4v?=
 =?utf-8?B?V3Z2dUVMUGVBVEw0azVHUmV6M2VMWVMzbUdLZW1Ra1pFUWk0Tnc1T2wreFhE?=
 =?utf-8?B?Qk1qQVMzSURZcXQ1Nm1iSWVsSDc4UGgrdjFXK2F2SENKVzd3T3owL2RTZVNz?=
 =?utf-8?B?TldLSE52WWJwRTJXTGFjd25JYkVpSGhrMWxjc2hPdko0a2JEMnBFU3BEblZw?=
 =?utf-8?B?YjcxQlErcmJaeGRSQzFENFhXdjR0M0ZkMnRzdnJOell2a2xuL0JPdk9aMW1p?=
 =?utf-8?B?QWZXSHhvVU85Z0I3dFFOSWxmT2xBMko0T090VUtTaG82cklkWWZ1N0tiY2JU?=
 =?utf-8?B?UWVKY2JmWG5sRkI4b2d6MW1QRVROUE13bDNtZFZRSXpzNHB1ZS9CeDhzWkQ1?=
 =?utf-8?B?WXJzaXNpNFM3M1NGSStsUVBoQ0JsRWNvSVMybGhjUVNRYjlSUkhJSWw3Mk1O?=
 =?utf-8?B?b2FnMEk5UnlLRDIwTlF2RmpVZjAyUVJZMVpRcGJwa0R4emd4a2swZUMxS2tw?=
 =?utf-8?B?UkMzM2lONVhFdzFvbXdlbVhlYW96Sm9wajJ4TkhKSTc2dlJKMkVrK0xpR2to?=
 =?utf-8?B?RjVmL01NUGRTcTdJaVo2cjJHUE5uTWpvd3RnamNrOXpIT0NGUGU5RC80Si9i?=
 =?utf-8?B?eDFOaCt3NkVic1JyZVVhdHpBOUJhVzlKTmhob24zamRORG9Gd2xyaHpxL3dk?=
 =?utf-8?B?UFNzcDdxZ1FHZ3lKUEd2SVJkTDFWSGRLMFZUVml6dmNvYjZHYVE3UXZmMUFD?=
 =?utf-8?B?SSthWTNMZWlkeWJwSHR1bnpUVGxUT3dPb1BtMlVjdlFrRHpnMmgyelBTQmxF?=
 =?utf-8?B?dW0xYk8wSkNiTE45VWdjOHM3Rys1L3Fqb0FhSDhaZk5aWjA5WkN6RzFtQjhp?=
 =?utf-8?B?dzZGTFFRLzZJQmVvb3dzT3VqWlY3SGttMlJkUm5TY3h3VW1YVzhXVXBKYWty?=
 =?utf-8?B?MkVLanFDQm1yaEpoNGZJOHlVMy9rSVlOei9MVTNWcTdsMmJaVEVneGIvQUtS?=
 =?utf-8?B?WjRHcmRKZlNia2JUbm0xZW41RzYxL0xKNnZncUZ0cVBKaml1S0lBRlRuNzYy?=
 =?utf-8?B?UnJ5bUZYbDhmOW9jM1c0emxGcW9iVFBleGRrTEF5L1NDRGpxS0c3VDk5cENa?=
 =?utf-8?B?OGVFQWZNNU1Sa0hUUG1DNjZ5aEVZRlRFcnhqMjd6cU92VHo0Z0JFNU5BUmxX?=
 =?utf-8?B?bmxVOExpWnA1VTBvbDdtbjVNb3BRQzQvMXNlZk4rc29XOHRSdHljWmp5dmZI?=
 =?utf-8?B?WWQ1R3BvVkhFeXJuRFo3dklIVTZSd0pFaXFlSDNkNmFPakdleFU5Qkp1akdt?=
 =?utf-8?B?S21ZQStyeGgwdWxoU1N5RG9RK1duc3ViQURCaHFGR2cwVEZzRjB0aHpOQ3VQ?=
 =?utf-8?B?SUxiMDBZYjZ1cDM3ZUNRbm5pWWVxdUE3VU5RTmxESm03eHdSYTZDMVlEQXBE?=
 =?utf-8?B?RHpyT2xyOHBNMnlqYytXbW1CMzNJbk9iaVdjekR2bTRyc2VnYmswSks5azNv?=
 =?utf-8?Q?nE8IShgj3hLleFOWlmx5uTvn6AVx8N6sQ4DPZoX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 336202a8-151b-42e1-0f1c-08d972c4006b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:27:19.6947
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2BeABeOqFPw4yVoALlTKj5icqiHdIsbRFI/sliTitETLgCFl2lq2T5jloEwIM4sCVZ6+oibgCjyEI5/8eTD4NQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4382

On 08.09.2021 11:54, Anthony PERARD wrote:
> --- a/tools/flask/policy/Makefile.common
> +++ b/tools/flask/policy/Makefile.common
> @@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER)
>  #
>  ########################################
>  
> -POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory)
> +POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)

Shell scripts better get invoked by "$(SHELL) <script>", to avoid
depending on the script actually being marked as executable (which is
impossible on some file systems).

> --- a/version.sh
> +++ b/version.sh
> @@ -1,5 +1,18 @@
>  #!/bin/sh
>  
> +opt_full=false
> +while [ $# -gt 1 ]; do
> +    case "$1" in
> +        --full) opt_full=true ;;
> +        *) break ;;
> +    esac
> +    shift
> +done
> +
>  MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "`
>  MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "`
> -printf "%d.%d" $MAJOR $MINOR
> +
> +if $opt_full; then
> +    EXTRAVERSION=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
> +fi
> +printf "%d.%d%s" $MAJOR $MINOR $EXTRAVERSION

I guess you want to clear EXTRAVERSION either prior to the "if" or
in an "else".

With these addressed:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:28:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:28:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181843.329179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwgp-0000y4-CZ; Wed, 08 Sep 2021 12:28:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181843.329179; Wed, 08 Sep 2021 12:28:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwgp-0000xx-8b; Wed, 08 Sep 2021 12:28:31 +0000
Received: by outflank-mailman (input) for mailman id 181843;
 Wed, 08 Sep 2021 12:28:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aiV0=N6=xenbits.xen.org=iwj@srs-us1.protection.inumbo.net>)
 id 1mNwgn-0000wB-KA
 for xen-devel@lists.xen.org; Wed, 08 Sep 2021 12:28:29 +0000
Received: from mail.xenproject.org (unknown [104.130.215.37])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89e63179-d4a1-4c8a-a7ae-19edf7ed1ee0;
 Wed, 08 Sep 2021 12:28:23 +0000 (UTC)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mNwgb-0004Q2-Do; Wed, 08 Sep 2021 12:28:17 +0000
Received: from iwj by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mNwgb-0005t1-CE; Wed, 08 Sep 2021 12:28:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89e63179-d4a1-4c8a-a7ae-19edf7ed1ee0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=+eTtHa9X8dbPGooWfB6LFRjJ0dBQugzbxCtJcYazIaI=; b=CALfCEjYZv13z/LZ0e48Yvcjbv
	fG0cfbLQApmAroNd5XRz8KxlLnPyDswqBy8fmomWKoy4NWuCPCwaQXeBMZGybPoM/TWahCkwObP03
	L/IBJT9nHOl8zwWk8exDd8Zdlg6aYtG0syBXPrBASc+VN/NAoe7QD+nMxT6Kmi4J0mm4=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 384 v3 (CVE-2021-28701) - Another race in
 XENMAPSPACE_grant_table handling
Message-Id: <E1mNwgb-0005t1-CE@xenbits.xenproject.org>
Date: Wed, 08 Sep 2021 12:28:17 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

            Xen Security Advisory CVE-2021-28701 / XSA-384
                               version 3

            Another race in XENMAPSPACE_grant_table handling

UPDATES IN VERSION 3
====================

Public release.

ISSUE DESCRIPTION
=================

Guests are permitted access to certain Xen-owned pages of memory.  The
majority of such pages remain allocated / associated with a guest for
its entire lifetime.  Grant table v2 status pages, however, are
de-allocated when a guest switches (back) from v2 to v1.  Freeing
such pages requires that the hypervisor enforce that no parallel request
can result in the addition of a mapping of such a page to a guest.  That
enforcement was missing, allowing guests to retain access to pages
that were freed and perhaps re-used for other purposes.

Unfortunately, when XSA-379 was being prepared, this similar issue was
not noticed.

IMPACT
======

A malicious guest may be able to elevate its privileges to that of the
host, cause host or guest Denial of Service (DoS), or cause information
leaks.

VULNERABLE SYSTEMS
==================

All Xen versions from 4.0 onwards are affected.  Xen versions 3.4 and
older are not affected.

Only x86 HVM and PVH guests permitted to use grant table version 2
interfaces can leverage this vulnerability.  x86 PV guests cannot
leverage this vulnerability.  On Arm, grant table v2 use is explicitly
unsupported.

MITIGATION
==========

Running only PV guests will avoid this vulnerability.

Suppressing use of grant table v2 interfaces for HVM or PVH guests will
also avoid this vulnerability.

NOTE REGARDING EMBARGO
======================

Please note that the public embargo time for this advisory is
2021-09-08, two weeks later than XSA-378,379,380,382,383.

CREDITS
=======

This issue was discovered by Julien Grall of Amazon.

RESOLUTION
==========

Applying the appropriate attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa384.patch           xen-unstable - Xen 4.15.x
xsa384-4.14.patch      Xen 4.14.x - 4.12.x
xsa384-4.11.patch      Xen 4.11.x

$ sha256sum xsa384*
3bf555e8b2a37dec361b86c0b6a3f59af2e1a24e3457ed92e0cfeaa662f1663a  xsa384.patch
435b431dc77d255031832dc265a8d5aa2f13f3b1a7de497b62ac2df5ad61da90  xsa384-4.11.patch
f98ec4c25fb122de6353eb7d9f5678dd09982f887bf201d6f178e9b647618c9a  xsa384-4.14.patch
$

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or PV-guest-only mitigations described
above (or others which are substantially similar) is permitted during
the embargo, even on public-facing systems with untrusted guest users
and administrators.

HOWEVER, distribution of updated software is prohibited (except to other
members of the predisclosure list).

ADDITIONALLY, deployment of the grant table v2 disabling mitigation
described above *is* now (following the public release of XSA-379)
permitted during the embargo on public-facing systems with untrusted
guest users and administrators.  This is because (although such a
configuration change is recognizable by the affected guests) it is a
mitigation recommended in XSA-379, so such a change would not reveal
the existence of a further problem.

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----

iQE/BAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmE4rD4MHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZ8S8H+ITq1u5AaK+N1wgsgztLJRh1cjFTDaLZdI0mypwV
+wPhNdw1FFH19wYDZhOJIzv3h5352YVUYs8P6HvhFWrsUaq5n4cz17wxUHch74r3
MUUU9WUdTffQrAAOSSK65yWTUewv/FglEWP55YFBDPqBJHpVWt2MMidmP6My6azK
GZAHSWU7+qNXbs4/OM+dQJyUJm6yZtAltVtVmiJdJ5bSZyqe+82zRMnS39jlkZEh
VwFnw6rdlPcYO/fNYi25bQSlXbFeruSJYK+omrrFsyd65Z4D5LyZc5CQkfRJgEt6
vMDsQR+5hrE/KXKr2mfyTx6nh0RdR8kcI2Wh017BYuLqhA==
=AEo5
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa384.patch"
Content-Disposition: attachment; filename="xsa384.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGRlYWwgd2l0aCBzdGF0dXMgZnJhbWUgbWFwcGluZyByYWNl
CgpPbmNlIGdudHRhYl9tYXBfZnJhbWUoKSBkcm9wcyB0aGUgZ3JhbnQgdGFi
bGUgbG9jaywgdGhlIE1GTiBpdCByZXBvcnRzCmJhY2sgdG8gaXRzIGNhbGxl
ciBpcyBmcmVlIHRvIG90aGVyIG1hbmlwdWxhdGlvbi4gSW4gcGFydGljdWxh
cgpnbnR0YWJfdW5wb3B1bGF0ZV9zdGF0dXNfZnJhbWVzKCkgbWlnaHQgZnJl
ZSBpdCwgYnkgYSByYWNpbmcgcmVxdWVzdCBvbgphbm90aGVyIENQVSwgdGh1
cyByZXN1bHRpbmcgaW4gYSByZWZlcmVuY2UgdG8gYSBkZWFsbG9jYXRlZCBw
YWdlIGdldHRpbmcKYWRkZWQgdG8gYSBkb21haW4ncyBQMk0uCgpPYnRhaW4g
YSBwYWdlIHJlZmVyZW5jZSBpbiBnbnR0YWJfbWFwX2ZyYW1lKCkgdG8gcHJl
dmVudCBmcmVlaW5nIG9mIHRoZQpwYWdlIHVudGlsIHhlbm1lbV9hZGRfdG9f
cGh5c21hcF9vbmUoKSBoYXMgYWN0dWFsbHkgY29tcGxldGVkIGl0cyBhY3Rp
bmcKb24gdGhlIHBhZ2UuIERvIHNvIHVuaWZvcm1seSwgZXZlbiBpZiBvbmx5
IHN0cmljdGx5IHJlcXVpcmVkIGZvciB2MgpzdGF0dXMgcGFnZXMsIHRvIGF2
b2lkIGV4dHJhIGNvbmRpdGlvbmFscyAod2hpY2ggdGhlbiB3b3VsZCBhbGwg
bmVlZCB0bwpiZSBrZXB0IGluIHN5bmMgZ29pbmcgZm9yd2FyZCkuCgpUaGlz
IGlzIENWRS0yMDIxLTI4NzAxIC8gWFNBLTM4NC4KClJlcG9ydGVkLWJ5OiBK
dWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgpTaWduZWQtb2ZmLWJ5
OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2VkLWJ5
OiBKdWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgotLS0KdjI6IFB1
bGwgZ2V0X3BhZ2UoKSBlYXJsaWVyIGFuZCBmb2xkIGlmKClzLgoKLS0tIGEv
eGVuL2FyY2gvYXJtL21tLmMKKysrIGIveGVuL2FyY2gvYXJtL21tLmMKQEAg
LTE0MjAsNiArMTQyMCw4IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBf
b25lKAogICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgIHJldHVybiBy
YzsKIAorICAgICAgICAvKiBOZWVkIHRvIHRha2UgY2FyZSBvZiB0aGUgcmVm
ZXJlbmNlIG9idGFpbmVkIGluIGdudHRhYl9tYXBfZnJhbWUoKS4gKi8KKyAg
ICAgICAgcGFnZSA9IG1mbl90b19wYWdlKG1mbik7CiAgICAgICAgIHQgPSBw
Mm1fcmFtX3J3OwogCiAgICAgICAgIGJyZWFrOwpAQCAtMTQ4Nyw5ICsxNDg5
LDEyIEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogICAgIC8q
IE1hcCBhdCBuZXcgbG9jYXRpb24uICovCiAgICAgcmMgPSBndWVzdF9waHlz
bWFwX2FkZF9lbnRyeShkLCBnZm4sIG1mbiwgMCwgdCk7CiAKLSAgICAvKiBJ
ZiB3ZSBmYWlsIHRvIGFkZCB0aGUgbWFwcGluZywgd2UgbmVlZCB0byBkcm9w
IHRoZSByZWZlcmVuY2Ugd2UKLSAgICAgKiB0b29rIGVhcmxpZXIgb24gZm9y
ZWlnbiBwYWdlcyAqLwotICAgIGlmICggcmMgJiYgc3BhY2UgPT0gWEVOTUFQ
U1BBQ0VfZ21mbl9mb3JlaWduICkKKyAgICAvKgorICAgICAqIEZvciBYRU5N
QVBTUEFDRV9nbWZuX2ZvcmVpZ24gaWYgd2UgZmFpbGVkIHRvIGFkZCB0aGUg
bWFwcGluZywgd2UgbmVlZAorICAgICAqIHRvIGRyb3AgdGhlIHJlZmVyZW5j
ZSB3ZSB0b29rIGVhcmxpZXIuIEluIGFsbCBvdGhlciBjYXNlcyB3ZSBuZWVk
IHRvCisgICAgICogZHJvcCBhbnkgcmVmZXJlbmNlIHdlIHRvb2sgZWFybGll
ciAocGVyaGFwcyBpbmRpcmVjdGx5KS4KKyAgICAgKi8KKyAgICBpZiAoIHNw
YWNlID09IFhFTk1BUFNQQUNFX2dtZm5fZm9yZWlnbiA/IHJjIDogcGFnZSAh
PSBOVUxMICkKICAgICB7CiAgICAgICAgIEFTU0VSVChwYWdlICE9IE5VTEwp
OwogICAgICAgICBwdXRfcGFnZShwYWdlKTsKLS0tIGEveGVuL2FyY2gveDg2
L21tL3AybS5jCisrKyBiL3hlbi9hcmNoL3g4Ni9tbS9wMm0uYwpAQCAtMjcy
Niw2ICsyNzI2LDggQEAgaW50IHhlbm1lbV9hZGRfdG9fcGh5c21hcF9vbmUo
CiAgICAgICAgIHJjID0gZ250dGFiX21hcF9mcmFtZShkLCBpZHgsIGdwZm4s
ICZtZm4pOwogICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgIHJldHVy
biByYzsKKyAgICAgICAgLyogTmVlZCB0byB0YWtlIGNhcmUgb2YgdGhlIHJl
ZmVyZW5jZSBvYnRhaW5lZCBpbiBnbnR0YWJfbWFwX2ZyYW1lKCkuICovCisg
ICAgICAgIHBhZ2UgPSBtZm5fdG9fcGFnZShtZm4pOwogICAgICAgICBicmVh
azsKIAogICAgIGNhc2UgWEVOTUFQU1BBQ0VfZ21mbjoKLS0tIGEveGVuL2Nv
bW1vbi9ncmFudF90YWJsZS5jCisrKyBiL3hlbi9jb21tb24vZ3JhbnRfdGFi
bGUuYwpAQCAtNDA5Nyw3ICs0MDk3LDE2IEBAIGludCBnbnR0YWJfbWFwX2Zy
YW1lKHN0cnVjdCBkb21haW4gKmQsIHUKICAgICB9CiAKICAgICBpZiAoICFy
YyApCi0gICAgICAgIGdudHRhYl9zZXRfZnJhbWVfZ2ZuKGd0LCBzdGF0dXMs
IGlkeCwgZ2ZuKTsKKyAgICB7CisgICAgICAgIC8qCisgICAgICAgICAqIE1h
a2Ugc3VyZSBnbnR0YWJfdW5wb3B1bGF0ZV9zdGF0dXNfZnJhbWVzKCkgd29u
J3QgKHN1Y2Nlc3NmdWxseSkKKyAgICAgICAgICogZnJlZSB0aGUgcGFnZSB1
bnRpbCBvdXIgY2FsbGVyIGhhcyBjb21wbGV0ZWQgaXRzIG9wZXJhdGlvbi4K
KyAgICAgICAgICovCisgICAgICAgIGlmICggZ2V0X3BhZ2UobWZuX3RvX3Bh
Z2UoKm1mbiksIGQpICkKKyAgICAgICAgICAgIGdudHRhYl9zZXRfZnJhbWVf
Z2ZuKGd0LCBzdGF0dXMsIGlkeCwgZ2ZuKTsKKyAgICAgICAgZWxzZQorICAg
ICAgICAgICAgcmMgPSAtRUJVU1k7CisgICAgfQogCiAgICAgZ3JhbnRfd3Jp
dGVfdW5sb2NrKGd0KTsKIAo=

--=separator
Content-Type: application/octet-stream; name="xsa384-4.11.patch"
Content-Disposition: attachment; filename="xsa384-4.11.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGRlYWwgd2l0aCBzdGF0dXMgZnJhbWUgbWFwcGluZyByYWNl
CgpPbmNlIGdudHRhYl9tYXBfZnJhbWUoKSBkcm9wcyB0aGUgZ3JhbnQgdGFi
bGUgbG9jaywgdGhlIE1GTiBpdCByZXBvcnRzCmJhY2sgdG8gaXRzIGNhbGxl
ciBpcyBmcmVlIHRvIG90aGVyIG1hbmlwdWxhdGlvbi4gSW4gcGFydGljdWxh
cgpnbnR0YWJfdW5wb3B1bGF0ZV9zdGF0dXNfZnJhbWVzKCkgbWlnaHQgZnJl
ZSBpdCwgYnkgYSByYWNpbmcgcmVxdWVzdCBvbgphbm90aGVyIENQVSwgdGh1
cyByZXN1bHRpbmcgaW4gYSByZWZlcmVuY2UgdG8gYSBkZWFsbG9jYXRlZCBw
YWdlIGdldHRpbmcKYWRkZWQgdG8gYSBkb21haW4ncyBQMk0uCgpPYnRhaW4g
YSBwYWdlIHJlZmVyZW5jZSBpbiBnbnR0YWJfbWFwX2ZyYW1lKCkgdG8gcHJl
dmVudCBmcmVlaW5nIG9mIHRoZQpwYWdlIHVudGlsIHhlbm1lbV9hZGRfdG9f
cGh5c21hcF9vbmUoKSBoYXMgYWN0dWFsbHkgY29tcGxldGVkIGl0cyBhY3Rp
bmcKb24gdGhlIHBhZ2UuIERvIHNvIHVuaWZvcm1seSwgZXZlbiBpZiBvbmx5
IHN0cmljdGx5IHJlcXVpcmVkIGZvciB2MgpzdGF0dXMgcGFnZXMsIHRvIGF2
b2lkIGV4dHJhIGNvbmRpdGlvbmFscyAod2hpY2ggdGhlbiB3b3VsZCBhbGwg
bmVlZCB0bwpiZSBrZXB0IGluIHN5bmMgZ29pbmcgZm9yd2FyZCkuCgpUaGlz
IGlzIENWRS0yMDIxLTI4NzAxIC8gWFNBLTM4NC4KClJlcG9ydGVkLWJ5OiBK
dWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgpTaWduZWQtb2ZmLWJ5
OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2VkLWJ5
OiBKdWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgoKLS0tIGEveGVu
L2FyY2gvYXJtL21tLmMKKysrIGIveGVuL2FyY2gvYXJtL21tLmMKQEAgLTEy
MzgsNiArMTIzOCw4IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25l
KAogICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgIHJldHVybiByYzsK
IAorICAgICAgICAvKiBOZWVkIHRvIHRha2UgY2FyZSBvZiB0aGUgcmVmZXJl
bmNlIG9idGFpbmVkIGluIGdudHRhYl9tYXBfZnJhbWUoKS4gKi8KKyAgICAg
ICAgcGFnZSA9IG1mbl90b19wYWdlKG1mbik7CiAgICAgICAgIHQgPSBwMm1f
cmFtX3J3OwogCiAgICAgICAgIGJyZWFrOwpAQCAtMTMwNCw5ICsxMzA2LDEy
IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogICAgIC8qIE1h
cCBhdCBuZXcgbG9jYXRpb24uICovCiAgICAgcmMgPSBndWVzdF9waHlzbWFw
X2FkZF9lbnRyeShkLCBnZm4sIG1mbiwgMCwgdCk7CiAKLSAgICAvKiBJZiB3
ZSBmYWlsIHRvIGFkZCB0aGUgbWFwcGluZywgd2UgbmVlZCB0byBkcm9wIHRo
ZSByZWZlcmVuY2Ugd2UKLSAgICAgKiB0b29rIGVhcmxpZXIgb24gZm9yZWln
biBwYWdlcyAqLwotICAgIGlmICggcmMgJiYgc3BhY2UgPT0gWEVOTUFQU1BB
Q0VfZ21mbl9mb3JlaWduICkKKyAgICAvKgorICAgICAqIEZvciBYRU5NQVBT
UEFDRV9nbWZuX2ZvcmVpZ24gaWYgd2UgZmFpbGVkIHRvIGFkZCB0aGUgbWFw
cGluZywgd2UgbmVlZAorICAgICAqIHRvIGRyb3AgdGhlIHJlZmVyZW5jZSB3
ZSB0b29rIGVhcmxpZXIuIEluIGFsbCBvdGhlciBjYXNlcyB3ZSBuZWVkIHRv
CisgICAgICogZHJvcCBhbnkgcmVmZXJlbmNlIHdlIHRvb2sgZWFybGllciAo
cGVyaGFwcyBpbmRpcmVjdGx5KS4KKyAgICAgKi8KKyAgICBpZiAoIHNwYWNl
ID09IFhFTk1BUFNQQUNFX2dtZm5fZm9yZWlnbiA/IHJjIDogcGFnZSAhPSBO
VUxMICkKICAgICB7CiAgICAgICAgIEFTU0VSVChwYWdlICE9IE5VTEwpOwog
ICAgICAgICBwdXRfcGFnZShwYWdlKTsKLS0tIGEveGVuL2FyY2gveDg2L21t
LmMKKysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ3NTEsNiArNDc1MSw4
IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogICAgICAgICAg
ICAgcmMgPSBnbnR0YWJfbWFwX2ZyYW1lKGQsIGlkeCwgZ3BmbiwgJm1mbik7
CiAgICAgICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgICAgICByZXR1
cm4gcmM7CisgICAgICAgICAgICAvKiBOZWVkIHRvIHRha2UgY2FyZSBvZiB0
aGUgcmVmIG9idGFpbmVkIGluIGdudHRhYl9tYXBfZnJhbWUoKS4gKi8KKyAg
ICAgICAgICAgIHBhZ2UgPSBtZm5fdG9fcGFnZShtZm4pOwogICAgICAgICAg
ICAgYnJlYWs7CiAgICAgICAgIGNhc2UgWEVOTUFQU1BBQ0VfZ21mbjoKICAg
ICAgICAgewotLS0gYS94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKKysrIGIv
eGVuL2NvbW1vbi9ncmFudF90YWJsZS5jCkBAIC0zOTY0LDcgKzM5NjQsMTYg
QEAgaW50IGdudHRhYl9tYXBfZnJhbWUoc3RydWN0IGRvbWFpbiAqZCwgdQog
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKm1mbiwg
MCk7CiAKICAgICBpZiAoICFyYyApCi0gICAgICAgIGdudHRhYl9zZXRfZnJh
bWVfZ2ZuKGd0LCBzdGF0dXMsIGlkeCwgZ2ZuKTsKKyAgICB7CisgICAgICAg
IC8qCisgICAgICAgICAqIE1ha2Ugc3VyZSBnbnR0YWJfdW5wb3B1bGF0ZV9z
dGF0dXNfZnJhbWVzKCkgd29uJ3QgKHN1Y2Nlc3NmdWxseSkKKyAgICAgICAg
ICogZnJlZSB0aGUgcGFnZSB1bnRpbCBvdXIgY2FsbGVyIGhhcyBjb21wbGV0
ZWQgaXRzIG9wZXJhdGlvbi4KKyAgICAgICAgICovCisgICAgICAgIGlmICgg
Z2V0X3BhZ2UobWZuX3RvX3BhZ2UoKm1mbiksIGQpICkKKyAgICAgICAgICAg
IGdudHRhYl9zZXRfZnJhbWVfZ2ZuKGd0LCBzdGF0dXMsIGlkeCwgZ2ZuKTsK
KyAgICAgICAgZWxzZQorICAgICAgICAgICAgcmMgPSAtRUJVU1k7CisgICAg
fQogCiAgICAgZ3JhbnRfd3JpdGVfdW5sb2NrKGd0KTsKIAo=

--=separator
Content-Type: application/octet-stream; name="xsa384-4.14.patch"
Content-Disposition: attachment; filename="xsa384-4.14.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiBnbnR0YWI6IGRlYWwgd2l0aCBzdGF0dXMgZnJhbWUgbWFwcGluZyByYWNl
CgpPbmNlIGdudHRhYl9tYXBfZnJhbWUoKSBkcm9wcyB0aGUgZ3JhbnQgdGFi
bGUgbG9jaywgdGhlIE1GTiBpdCByZXBvcnRzCmJhY2sgdG8gaXRzIGNhbGxl
ciBpcyBmcmVlIHRvIG90aGVyIG1hbmlwdWxhdGlvbi4gSW4gcGFydGljdWxh
cgpnbnR0YWJfdW5wb3B1bGF0ZV9zdGF0dXNfZnJhbWVzKCkgbWlnaHQgZnJl
ZSBpdCwgYnkgYSByYWNpbmcgcmVxdWVzdCBvbgphbm90aGVyIENQVSwgdGh1
cyByZXN1bHRpbmcgaW4gYSByZWZlcmVuY2UgdG8gYSBkZWFsbG9jYXRlZCBw
YWdlIGdldHRpbmcKYWRkZWQgdG8gYSBkb21haW4ncyBQMk0uCgpPYnRhaW4g
YSBwYWdlIHJlZmVyZW5jZSBpbiBnbnR0YWJfbWFwX2ZyYW1lKCkgdG8gcHJl
dmVudCBmcmVlaW5nIG9mIHRoZQpwYWdlIHVudGlsIHhlbm1lbV9hZGRfdG9f
cGh5c21hcF9vbmUoKSBoYXMgYWN0dWFsbHkgY29tcGxldGVkIGl0cyBhY3Rp
bmcKb24gdGhlIHBhZ2UuIERvIHNvIHVuaWZvcm1seSwgZXZlbiBpZiBvbmx5
IHN0cmljdGx5IHJlcXVpcmVkIGZvciB2MgpzdGF0dXMgcGFnZXMsIHRvIGF2
b2lkIGV4dHJhIGNvbmRpdGlvbmFscyAod2hpY2ggdGhlbiB3b3VsZCBhbGwg
bmVlZCB0bwpiZSBrZXB0IGluIHN5bmMgZ29pbmcgZm9yd2FyZCkuCgpUaGlz
IGlzIENWRS0yMDIxLTI4NzAxIC8gWFNBLTM4NC4KClJlcG9ydGVkLWJ5OiBK
dWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgpTaWduZWQtb2ZmLWJ5
OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+ClJldmlld2VkLWJ5
OiBKdWxpZW4gR3JhbGwgPGpncmFsbEBhbWF6b24uY29tPgoKLS0tIGEveGVu
L2FyY2gvYXJtL21tLmMKKysrIGIveGVuL2FyY2gvYXJtL21tLmMKQEAgLTE0
MjMsNiArMTQyMyw4IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25l
KAogICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgIHJldHVybiByYzsK
IAorICAgICAgICAvKiBOZWVkIHRvIHRha2UgY2FyZSBvZiB0aGUgcmVmZXJl
bmNlIG9idGFpbmVkIGluIGdudHRhYl9tYXBfZnJhbWUoKS4gKi8KKyAgICAg
ICAgcGFnZSA9IG1mbl90b19wYWdlKG1mbik7CiAgICAgICAgIHQgPSBwMm1f
cmFtX3J3OwogCiAgICAgICAgIGJyZWFrOwpAQCAtMTQ5MCw5ICsxNDkyLDEy
IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogICAgIC8qIE1h
cCBhdCBuZXcgbG9jYXRpb24uICovCiAgICAgcmMgPSBndWVzdF9waHlzbWFw
X2FkZF9lbnRyeShkLCBnZm4sIG1mbiwgMCwgdCk7CiAKLSAgICAvKiBJZiB3
ZSBmYWlsIHRvIGFkZCB0aGUgbWFwcGluZywgd2UgbmVlZCB0byBkcm9wIHRo
ZSByZWZlcmVuY2Ugd2UKLSAgICAgKiB0b29rIGVhcmxpZXIgb24gZm9yZWln
biBwYWdlcyAqLwotICAgIGlmICggcmMgJiYgc3BhY2UgPT0gWEVOTUFQU1BB
Q0VfZ21mbl9mb3JlaWduICkKKyAgICAvKgorICAgICAqIEZvciBYRU5NQVBT
UEFDRV9nbWZuX2ZvcmVpZ24gaWYgd2UgZmFpbGVkIHRvIGFkZCB0aGUgbWFw
cGluZywgd2UgbmVlZAorICAgICAqIHRvIGRyb3AgdGhlIHJlZmVyZW5jZSB3
ZSB0b29rIGVhcmxpZXIuIEluIGFsbCBvdGhlciBjYXNlcyB3ZSBuZWVkIHRv
CisgICAgICogZHJvcCBhbnkgcmVmZXJlbmNlIHdlIHRvb2sgZWFybGllciAo
cGVyaGFwcyBpbmRpcmVjdGx5KS4KKyAgICAgKi8KKyAgICBpZiAoIHNwYWNl
ID09IFhFTk1BUFNQQUNFX2dtZm5fZm9yZWlnbiA/IHJjIDogcGFnZSAhPSBO
VUxMICkKICAgICB7CiAgICAgICAgIEFTU0VSVChwYWdlICE9IE5VTEwpOwog
ICAgICAgICBwdXRfcGFnZShwYWdlKTsKLS0tIGEveGVuL2FyY2gveDg2L21t
LmMKKysrIGIveGVuL2FyY2gveDg2L21tLmMKQEAgLTQ2MjAsNiArNDYyMCw4
IEBAIGludCB4ZW5tZW1fYWRkX3RvX3BoeXNtYXBfb25lKAogICAgICAgICAg
ICAgcmMgPSBnbnR0YWJfbWFwX2ZyYW1lKGQsIGlkeCwgZ3BmbiwgJm1mbik7
CiAgICAgICAgICAgICBpZiAoIHJjICkKICAgICAgICAgICAgICAgICByZXR1
cm4gcmM7CisgICAgICAgICAgICAvKiBOZWVkIHRvIHRha2UgY2FyZSBvZiB0
aGUgcmVmIG9idGFpbmVkIGluIGdudHRhYl9tYXBfZnJhbWUoKS4gKi8KKyAg
ICAgICAgICAgIHBhZ2UgPSBtZm5fdG9fcGFnZShtZm4pOwogICAgICAgICAg
ICAgYnJlYWs7CiAgICAgICAgIGNhc2UgWEVOTUFQU1BBQ0VfZ21mbjoKICAg
ICAgICAgewotLS0gYS94ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMKKysrIGIv
eGVuL2NvbW1vbi9ncmFudF90YWJsZS5jCkBAIC00MTAyLDcgKzQxMDIsMTYg
QEAgaW50IGdudHRhYl9tYXBfZnJhbWUoc3RydWN0IGRvbWFpbiAqZCwgdQog
ICAgIH0KIAogICAgIGlmICggIXJjICkKLSAgICAgICAgZ250dGFiX3NldF9m
cmFtZV9nZm4oZ3QsIHN0YXR1cywgaWR4LCBnZm4pOworICAgIHsKKyAgICAg
ICAgLyoKKyAgICAgICAgICogTWFrZSBzdXJlIGdudHRhYl91bnBvcHVsYXRl
X3N0YXR1c19mcmFtZXMoKSB3b24ndCAoc3VjY2Vzc2Z1bGx5KQorICAgICAg
ICAgKiBmcmVlIHRoZSBwYWdlIHVudGlsIG91ciBjYWxsZXIgaGFzIGNvbXBs
ZXRlZCBpdHMgb3BlcmF0aW9uLgorICAgICAgICAgKi8KKyAgICAgICAgaWYg
KCBnZXRfcGFnZShtZm5fdG9fcGFnZSgqbWZuKSwgZCkgKQorICAgICAgICAg
ICAgZ250dGFiX3NldF9mcmFtZV9nZm4oZ3QsIHN0YXR1cywgaWR4LCBnZm4p
OworICAgICAgICBlbHNlCisgICAgICAgICAgICByYyA9IC1FQlVTWTsKKyAg
ICB9CiAKICAgICBncmFudF93cml0ZV91bmxvY2soZ3QpOwogCg==

--=separator--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:28:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:28:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181847.329222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwh8-0002Ip-Ik; Wed, 08 Sep 2021 12:28:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181847.329222; Wed, 08 Sep 2021 12:28:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwh8-0002Ig-F6; Wed, 08 Sep 2021 12:28:50 +0000
Received: by outflank-mailman (input) for mailman id 181847;
 Wed, 08 Sep 2021 12:28:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNwh6-0002Ek-Pq
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:28:48 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4fbf02bf-10a0-11ec-b150-12813bfff9fa;
 Wed, 08 Sep 2021 12:28:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fbf02bf-10a0-11ec-b150-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631104126;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=wodbCM6Hlgl8wSUIOmYxcU89STUKdXw0inBRJN0XIaY=;
  b=GixLMEgHqc7pR3jicxjBh4hhMzEDJ9Bx/3QMDyglYuDZKxCpc0tkiGZS
   ivy4/M8wPwNuQF1qjIpaK1Ua4x34RB+TlpYyU029Sb+T6zIyu5WGNx2Rf
   K+EQ2T8NkV0U6BsLTy80i0eLG2Xf8Y9iSYQXV4cSgIDCmoRo1/00MU6NF
   k=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: +2yKxdrHQOgrG/B7HtjD4CdK4uJgisO4g59s9MzKG6PQ2IhseGISKbmcw/id7eeHyg9U6m324T
 usqTFc+zvpEYXG+cmxAHdSyaDBGXMZk/FD1EYRAtD7Y59ca42oHiqj8laRl1DHT2RUSiJDO/nx
 YQgPx9nUMHsD/6LBmRfAcmf/QL4sobbgewF2dU3CnLvdcFySWhulvThbD5DaQfPYCacVD0Swb7
 BlxWXA3pCRDLZ/OKn3JFUkAiOfwEJG9uZaeJ9f/WXSi63oFS76Ib4Hb+07AMnJR4AS3A0Ya4gD
 m46zqp3+9txWT0JWYbpIY1jv
X-SBRS: 5.1
X-MesageID: 52242045
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Iv4csa16fFa1ejDCseQl4QqjBSVyeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5OEtApTiBUJPwJE80hqQFnrX5Wo3SIDUO2VHYUb2KiLGN/9SOIVyHygcw79
 YGT0E6MqyLMbEYt7eL3ODbKadY/DDvysnB7o2/vhQdPj2CKZsQizuRYjzrY3GeLzM2Y6bReq
 DshPav6wDQAkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,277,1624334400"; 
   d="scan'208";a="52242045"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PkGp3tN2R5FU6E1FVv6PBAFe2ZYf7BZfC5u+O7xJyglCuSj55LYrzHuhgaJsZMeiz6Futly4xW5kYU7DI1Et8B1g8O9/3lVuxiG7AhYcm2bp9n7/hukfrk81Q4t2iIhM6WumBFA4ICnBDZsVTPF9EaYIxjE/HdJ46+0dAum4aUpK9Nj/HHw6iEZ1dfkCsF6gcYB5Bgot/ueFmzpqk6rIAKK11Mhh7bIpjjEpylbVWNU2XdRsV2/OQITaVZMvs6CZ2VumdOzA+XJysVlRj9e1NJSbeCIL1YrcnQxmSEnUWxS5GO7HaK0uA5HqSExHRl0hD3GEdW41J36FxjjPFPP9DA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wodbCM6Hlgl8wSUIOmYxcU89STUKdXw0inBRJN0XIaY=;
 b=XvFVhAKr0lz4XOZfE0Gac6kBBV3AE3Feu3LK4fopXIoL1N0ku9OAGDQyOq5Jv5mXnwLn0vQk242RXxn7r2q2+haSKhW030iDxkuO+X6UNHqqEPYXzy3fMtvurcF5GbiWUZ/bAgER+Z/1LPzZtbwzD42yFeU8STCjLaC/Q8lG6Cxdx+MXPqC/wTp+OF+PC+3UbboIB+c3huLDhHbhw2u2tEjH4g19SBCsuEJJgZdD36WoaQBW7aKoOf/T5v38F+eiwsHEkiuUuBKRB1krSkotEnQSMQbgef3NNMmRUcE4r5+QXgPFUfHqEPtxV5fITV4wEmvXkrKQjbLISn+fLaxhzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wodbCM6Hlgl8wSUIOmYxcU89STUKdXw0inBRJN0XIaY=;
 b=JppvLS4ZroM083ToHrDLK/8z+kegAPmZ7AAiq60Eax/CxVvakg1ltc4al2bD3MckVTQ93yuigukKF/bjiwUS4syb85xc4Pwr551ZFMUSqXRAXrtXVEu468069kN7EcGS7ubWGGpFwfex2NHqxHlKw7E2IaOxrxF4j53t3y7Ex14=
To: Jan Beulich <jbeulich@suse.com>, Jane Malalane <jane.malalane@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Pu Wen <puwen@hygon.cn>, Andy Lutomirski
	<luto@kernel.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <c9c3228982cc81c79cab4ced983f80296107124a.1630929059.git.jane.malalane@citrix.com>
 <20210908081945.11047-1-jane.malalane@citrix.com>
 <a27d3631-86a3-cacc-efd6-97b02e3f8b94@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 2/2] x86/cpuid: Detect null segment behaviour on Zen2
 CPUs
Message-ID: <e89a7269-351c-6bdc-60e7-5b58399f1e31@citrix.com>
Date: Wed, 8 Sep 2021 13:28:35 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <a27d3631-86a3-cacc-efd6-97b02e3f8b94@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0238.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:b::34) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4f51b033-aa7f-4f6d-1680-08d972c43151
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5664:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5664E42D2EB201EEB95FB72CBAD49@SJ0PR03MB5664.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +817jgOoCCpbp073jZUvq2dU3zNpMjEa8ghp7BUq6lX58mfC5QZePbeYTZGN6NqTAPKvTXL3478rLWyKdA6OnIJsw2ihbcSCtCji0q8Yb+zNTQgvbtK7Ip+fXkPDRQolQWNOZi7DI27Bv15oABmBqR3j5hgClgScJ5g8UrQR00AlJLjlOI8F4sBDOhoPVQl9/EmB+X0lKjxrNMFvnqB2CRBGdwz1+yYWz62mnbFgC6b1dnfl9UFsQouDMwQOKJyBJUhQW5Ntb/1gxWCwgpn0Ba82aRmVXWDe6ZCBqiyk8U5y/n6pg8AY7piEbQ6ACOQI9hzkEuyzN9nnc/IamDSS+6fPIQu13aGfWuCDJzbyUYO5jTPbbyyFHTks1OpqA9Uq2UwnDjNM192Ij7x99PYiIkqY87eeYFgSvYZ9rDXFgOaM5maCwks2ejPthhO+V5avajXbnwUK1wA0+vpvyMyWJyriK7zOZz8BlqO0n2GSIG2ngTyvgc8arj9HG9X0K5OiJc8VN/f+6fzwgEjMvewU+6k7VRNvYZxZ89QLd270ybvw1rmA/85prJcXyt45qNK+5arNFfGc2tZyxt1zhzjpqvaBs0WSRC2WOHWWroukwamnIb1PfC1/1oJNz0n+nmXBOAyUCAItM2X/3AyuW4ho6Vl8u7gUxw5oC9DcxmwyEMNDAhpni7mS09x/hp7K/YC0w8zyQkNpV9PM/R+q0wSA5Wuef5KlSSMADOk1T0eOUM4=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(6636002)(38100700002)(956004)(6666004)(2616005)(5660300002)(66476007)(66556008)(8676002)(186003)(26005)(66946007)(8936002)(53546011)(55236004)(31696002)(31686004)(316002)(16576012)(6486002)(86362001)(110136005)(4744005)(54906003)(36756003)(508600001)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZHBya1VhSy9rZkhkakxSeFdodHhtTWt6elFNMXgzQ2ZETEZGdHhLcHdQdkk1?=
 =?utf-8?B?ZVd5Z3pxeWZjc1IwdFJ4eUVhWVMybTBvUlJ1bTZHM0oyMElvajNaYkZpcUhO?=
 =?utf-8?B?MTgrWng5YjVySnlsUEsxUHBjb2syWTlVME52b0dJWWxkMksxMzRiRlNPUURT?=
 =?utf-8?B?RGlPaWpVUEJ2dlpBTURZOTdYZjNhVXFuQVZ6ZHBZZnk5aGhKQmFBWWtIc0lN?=
 =?utf-8?B?UVMyZW9ETHhuMWNEb1lTWnQ3dEVrUXlpUnBuUUZBYmpDZWtlMVRPTkM4SFIz?=
 =?utf-8?B?TmNOMmljdDhnNHMrNXZheWxjOE1pUC9qckk3Tk1pTzhUK3VaVEN1V3RRYTV6?=
 =?utf-8?B?ZVROK1piY1RZNmoyTkJoejlHZ0xTQ3BmWXFtM2ZIU0ZiRzhiMFF6WS8ralhC?=
 =?utf-8?B?VEM1RUdURTRyNXRZeUI0Rk1sakhoYUQ5OGhxNnFNNVBxRUxEMFMzd0lhMW1S?=
 =?utf-8?B?cjZOVzFObkcyWjZ6R01Fc2prS2p1MkRwZ0J4TDB3TEoyUVlSS2c1YitWOElj?=
 =?utf-8?B?Y1dlREVzTHh5UUhMWHMyTUEwOU50RElFT2l3VFozOTk2eVlBZGMxTjlKWStQ?=
 =?utf-8?B?TnlkWjNnMTcwNmxYazdGOElNajZyQ3locDZ5SWRyVzdaWUNQcVAwQlRadWlE?=
 =?utf-8?B?VVhGV0t2cW9YNk1Sc2VyZThuby9NZVd5L0g3MkZCQ0pkNDl5SFlxcVB5ZTVq?=
 =?utf-8?B?MFJoYWxqcnYvMXFtMTNzcStwb3dtY25IRnordkNDYVJ4VDNEZFI2Q2VlWnl2?=
 =?utf-8?B?TVlnTHRLSEdxYWNTcWNOQXhlUytCWlZNQ3RLc0VEaXNPQk41RjRxVTFzRjg2?=
 =?utf-8?B?YnNLQVAxV1dZRVUwSm0vamZzd3MydGdrMGZvSTV1VnowVnkwL3ZzOUU4VCth?=
 =?utf-8?B?WThCc21sRk5Mak4vY2FWdlBxaXZMZXVGRnQrRW1uakwyeXhIWjh0ZEEzd1Rl?=
 =?utf-8?B?Z2ZKNkE0QmZ4aHBYY3JvVUlCdTFmdUJLN3M3MlRVelhsVGF3dmhyNldJZEpH?=
 =?utf-8?B?ZmdnYSt6NHhsSHh6WVhYclJOaUpYc3VYWW93T0VBVnZTcjcyMVpiU3dWb1hi?=
 =?utf-8?B?QkRyZmlXekZ1Wnd3SWh6YkJGUzVUaGtiV3lHRW5CQ2Zkd1JzQ2hNUlA5V3Q4?=
 =?utf-8?B?ZmJaNEp1UzM4czEyWTV2Tm1aM0FpeEtlQXdneTFoRWUxb2xxb2V0M1RFUlVo?=
 =?utf-8?B?RU9OcjVPVmx2b0hBZndoWHA5ZGV5ZTlOMkQzZVU0NGtqZ3c1L2hhL2pQUC9z?=
 =?utf-8?B?R3BRZys2YVM4QW5OcjV5ZGxpRFJXNnhrVjRZbTc0UXM2bGgxK2syTmhBS3FQ?=
 =?utf-8?B?MXVUR3BmNTEvWFlDUDJuQjFWK3hUb2hFTWpCajZaNGRZVzhSQ3B5WG45UFIr?=
 =?utf-8?B?VTNoZmtoZWI1VEhoZldIRFM0UVpvTys2aDdTSHpGTFNXY0lCeFl4anNJdnM5?=
 =?utf-8?B?SjBwTDJPREhhUW16NUcvNWNhc0FrTTBiUHZHcnJCYlpSdUFGREtyeWZUMUN6?=
 =?utf-8?B?c0RUd2NjVjRWVGo0dVI0d2EzWGFnQTBrTCtPTVJ6enRicjRTL1NuNzB6L0xk?=
 =?utf-8?B?R2xydVNkTkZxRGlsVysvOFZ6V1BBUHFIbEZlTTBhOGJzNEFYbktiOHZGRnBB?=
 =?utf-8?B?TFF4Z1hISDFlQU1MaFErV25TNjgrMjBIU3NSMTdYN3AwK2x2UXAyVVVUMzBD?=
 =?utf-8?B?dHFJR3B3b2F4eG5QOWFkcnIwcXRNY2lwTUZQOUNObEFTeXdVZFhGYS9Hb3Aw?=
 =?utf-8?Q?0TuJVi9cD+oUcoXw4V6cq6cWEK7UrK/IwK7jWV0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f51b033-aa7f-4f6d-1680-08d972c43151
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:28:41.7719
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: C8/24zCFPSWIVAbV3t9vihkvpUnksG9Gwis4sXUsKZHg7AqNl8MRLe//i0a7dFKgQ5al3hza8CzCThcL85l6rG3vJM1wFoyncPedCQEThO8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5664
X-OriginatorOrg: citrix.com

On 08/09/2021 13:08, Jan Beulich wrote:
> On 08.09.2021 10:19, Jane Malalane wrote:
>> Zen2 CPUs actually have this behaviour, but the CPUID bit couldn't be
>> introduced into Zen2 due to a lack of leaves. So, it was added in a
>> new leaf in Zen3. Nonetheless, hypervisors can synthesize the CPUID
>> bit in software.
> Considering the prior model checks, I understand this isn't all Zen2s?
> No matter what the answer, I'd like to ask that the first sentence
> start with either "All" or "Some" (or something along these lines).
> Which is of course fine to insert while committing, so not need to
> send a v3.

All Zen2's have this behaviour.

The model checks were trying to avoid running the probe on Zen1, but
"model >=3D 0x30 && mode !=3D 0x50" is error prone.

There are no faults in the probe, and its fast, so running on all Zen1/2
is better than hoping that we got the model list correct.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:34:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:34:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181899.329232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwmb-0005GO-At; Wed, 08 Sep 2021 12:34:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181899.329232; Wed, 08 Sep 2021 12:34:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwmb-0005GH-7z; Wed, 08 Sep 2021 12:34:29 +0000
Received: by outflank-mailman (input) for mailman id 181899;
 Wed, 08 Sep 2021 12:34:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNwma-0005GB-3K
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:34:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4505d12b-1626-4893-a4e1-7b4fdfb5baa0;
 Wed, 08 Sep 2021 12:34:26 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2052.outbound.protection.outlook.com [104.47.8.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-jTNbYORbOGaFlpPb-sRA1A-1; Wed, 08 Sep 2021 14:34:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4846.eurprd04.prod.outlook.com (2603:10a6:803:5b::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 12:34:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 12:34:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0195.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 12:34:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4505d12b-1626-4893-a4e1-7b4fdfb5baa0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631104466;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PpEv5+mHfqiqUFwb1j4lcLdt4/wSHF+p1OudjtufR1Q=;
	b=Nt/nD4KMhSlbeOwg3BgYO3jurGF/K2v8qY5IHOFXZ6PnCvs7DUlz9/pfJR4tacpl4WkeKJ
	viZS7YsoJW4+jWvnYPX+0TaImfQXPkm/TJPWfmrAKrLhz8EPhbZfqQNnwdhJX0a6n93dIO
	hDTbEGji/LryxGqn/vVeCJ1fx5Txess=
X-MC-Unique: jTNbYORbOGaFlpPb-sRA1A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VqwT+9zpWiYbBPurU0aVLrTeIAyrCgCwv/H4MgaTrpwg8DxRh5fS1hBr/3o7A5PXG9Zik1+o0mf2bGORgAD/Km6D9q6ONPJeoH7UAWstkfhekrwhxGYR9VvmNteN0U536EbbafQ8o2TtcmaBU7kKFVoc7AG1bWM3P7i8QtB6FHMTfxPj6NlgIvZgLSFZHp3u95mG7wnaV+g73lCczo0LspYlnlQfwMAoZktQfOc/SK+kchGxRZjkFVGfPBOzI0M3+oMJD4fU1Ch8BmrnoEHO968AQ5T3nZayOaSRhGdIdkMUU7AqCR2+YJtMptyQhfacNfiR0a1++LinKIkur5C6VQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PpEv5+mHfqiqUFwb1j4lcLdt4/wSHF+p1OudjtufR1Q=;
 b=m1NlxbkWqPAk+DLJHFALHwqVr8l/YwRYzXcNH7u9/k47rfnr8v5PD9Y/heHIg7JpczWwCw1iP92Rp/Q3gm33Lyr/kRjt9IMC4Oa2FB29t4MmEOwhKEPHTRQ6cyMPoZJNwFxbtGLRrl76U0ATvcQwGeIh4mkbT16SNjElxHfzuH3e2+qD1fwF1hhiS5GSx7b+ttMMGwwAYJB0uK7luu39pShPdhO402wH6WF+J/E+diucjfo+PeN6UUWnLmubOXx5ML71yJ/ZZ3m+xoA0OJi1tKX4ZjURC418S41KqFyHjdM4aIMKqWijDrjKHhr9P5EtExJ4dRe9P8Rh89w/nRgbqQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] domain: try to address Coverity pointing out a missing
 "break" in domain_teardown()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20e28e4b-c794-7435-5f5c-332e9a6c0433@suse.com>
 <518201c2-1250-26fd-2105-f44a60d40dba@suse.com>
Message-ID: <bb9abe14-22ae-a459-03ca-8895e9f23a61@suse.com>
Date: Wed, 8 Sep 2021 14:34:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <518201c2-1250-26fd-2105-f44a60d40dba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0195.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d32e9f87-4a7c-4552-e9ae-08d972c4fd26
X-MS-TrafficTypeDiagnostic: VI1PR04MB4846:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4846F2CF92CDC49F8AB63B91B3D49@VI1PR04MB4846.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FHjJjBl8JgLzZnT3cI7kEJHktjqI6rtQoiRfMti3dSUved4X7zRHED91fwXNoa8IThmz11jcmJ8WytMSHcjPKLPlZG4QrnqjSW3zvEJpTJlMkqNuwrCnWFmD5Kwl+BbYaDHauCVDIlCBNgbjATBQvW8wAF0+lSFStda08aa5UaX4IzhSYgCZGJu8Kqy+Z3S50AR72Xe6QADLtn5rIQT0YeuWdljK6fYWRvDLqZFyCU8SBM3QjrDaljmjJspjEKDYl2QQDzh/e/HI/kvSROMq6jwlzPhaf84C22lOXQ0hYtFu5oI7l834d6vGA7sC19FKKX1FBFvmjsltAGsZlmyatQLKhlChc/pebmXSY42h7GX/5UaO/x54Jhsku6SJc4kCuS2rnT/cuYak/+/I+9MgC1NcFjiICM0UnimvMmc73IJsuwij90Q2aX9wSYXzRQREOt3Ro74McWCxkvuL0yxziPodpwkZIFCFBZgNbpd06SJSQaKdD6a2QVakkm6VuEAtmOJ/OjBfLP/DFKvmdvs0bOmoERenNOoMmUHUPukwZM9lJtL06aQ2ccrpRP5VGOhX/iS3hDxi3fRZ3LyYwiehjurcFl1ymENKPX9xuz1aZd8yxdePPGPYpnF+ZSpNOmmti1DcyYR6v83IjXGVGj9IrHKVL0HwDcmZS+inkensu/BAzOpYlzphLu2QDaVw7PpV/gegqU9FxmJakfgKFJ5llr/BTqiQPhqRr/XmGzkK9Zm5boj6LMVoyX8iTnbPXoig
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(396003)(366004)(39860400002)(31686004)(66946007)(31696002)(53546011)(54906003)(16576012)(4326008)(86362001)(186003)(2906002)(6486002)(2616005)(26005)(38100700002)(956004)(8936002)(8676002)(66476007)(36756003)(6916009)(316002)(66556008)(83380400001)(5660300002)(478600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cDJ0OXBxMGI5eW5SVnFxZVhVSG1FSXhRWEFlVFBpaFRQMk4rRzd4Z1lQcEUr?=
 =?utf-8?B?Y1BVeVBOU0h0U1lSM3IzUHZGQVQ1MkpYK2tHNEpMZDM5TW51V0V2OVphazk4?=
 =?utf-8?B?OWYzL1JUSHkxcUNFUUFUNTBDM04xS0pxNVRucTkvNUR2R3Bnc3RvdzB1Uktp?=
 =?utf-8?B?SHZOWEZ3WG9oTlV4VG5QaFhwUmhmaEhpMldTTzdPdlNvMFFBaWdhL0t6MmUy?=
 =?utf-8?B?bFZWMUthOElHbEFiTytqNDJ4VkplQlFpWHBNTlNvNC9vaFd3cWQ1L205Z0tD?=
 =?utf-8?B?aDR0VWZLT0NJT0tVMTZRbXMvL0NIamJHMzgrNVY3N3dEYklSUVpOQS9YVkEz?=
 =?utf-8?B?N1lWVXozV2NPeGxDbnVDZ2V2ZG5LdnRDYm1MbWFwZ2NLSDl6UDMrM2N4UENK?=
 =?utf-8?B?VVFNdSs3eVFCbDdBVUVaZEYyV3VNTjlEYWhYdm96T2RYRDFJRlp2OU9lQzNP?=
 =?utf-8?B?M3RvNll2Smd0aFlVQWhlVHorNm5SUmZud2RSS1Z6M0xucjRaWVRSbzZSMjc0?=
 =?utf-8?B?anZyMGZxajAzRzlISGsycnhjY1M3cG1wNDdUNm1zazVrV2RIREtyWEpIYWdW?=
 =?utf-8?B?NHZRUlU3bURGZGZSMmZYRTdxc0FHWTZGNUVLZ2ZYYTlCWkphSlZYTzFZWk5R?=
 =?utf-8?B?Q3hzUHF5YXJlM1JwclhCemVtVkppb3g4a1ZxeUpHR2dlQXYxQ0d3Z0ZzOE96?=
 =?utf-8?B?c2p0NWJGOTJwNjJHclUvRDlrTlF3bTdQT05xVnBVZkR2Tm95SEVvRStkRWtM?=
 =?utf-8?B?UWJrcmhiNUlIanBaVWVsd3BsbHYySmVWdTJ2UG5FUCtMc2xjWmRSR0ZVaHN4?=
 =?utf-8?B?MlN0c0FkdmtVamxXcENub3JHMi9ha1ZNQ0pDRkJoNkU5S3h4bkhZKzVsQ0RD?=
 =?utf-8?B?ek9XY2IrNzk2QjhNMG9RYzg3TkhVL1VqYlpIQU50ZklULzdxaDkydWxCK1pR?=
 =?utf-8?B?ZjdhamhsV2FBQ2xIeFpYQ2FKcndnMktyeGpzQW9lMTY5SnNCanFTMndwUXdR?=
 =?utf-8?B?VGFPZVhCL2d0cGdJV1Y4Q2EzWTVmcW5Mc3Rnb3FRY2s3WkRoOFRLMnE1bjVp?=
 =?utf-8?B?WFpMOHFQQnFhVUI5eTdsaUxpdHV0Z3gwZTdWL25rVmF4c2hBUmF4ZjBDZlM0?=
 =?utf-8?B?a0tkdXBRZnpLeUtCanNOZVBZdmN0N0ZLMGhDb0krK28yVWZGS25sWVJBdHRj?=
 =?utf-8?B?Ym01aHZ5aktGaC9XTmZTRzdBL3NsaCtTSFNKK1hWRWZYVEgrajIzZVNOUlNR?=
 =?utf-8?B?NnpHSGtqS0o0YkZDbGdYaUhWT1dJemZHZnhFZ0JZS2FqMEFjOXFqVkJQRW9Z?=
 =?utf-8?B?L2ZrSFNCV2gyYXpGUkpWbGlpaHdrcnA1RGFqaDlCQzZkM2xWQ2tid3ppc05m?=
 =?utf-8?B?MzJlMXJnT25zUFd1WWxxc2lKZXpKc09tQ2dxdzNRVW1qWXFhK0x6eWN3RmN0?=
 =?utf-8?B?d0owLzc1YkVkVnJBOThuc0hHdEk3ai8rMUgwRS9qYkpTa3FSQlQvNDNlNTNT?=
 =?utf-8?B?NUZjaEpqMFNkRnhNTEIxNjZML05Vd2RnSTllUjFHMlZKTHh6VmpRMjhiN21C?=
 =?utf-8?B?Q2paOHcvN3pYcXQ4UnRZK1h1VDlkNi9VY25RV1o2VFJMTVdvQ1BlU1EyTGRV?=
 =?utf-8?B?Z2x0bytDQU9XM2wrZXkrV0Z5NjNqUkc4bk1LY2tZRTMrcllEbHV1dWNWQzlF?=
 =?utf-8?B?R2MrTHBUa083QjZsMEdpR0U4ZlYrak94b3dNbGlkbm9wT1pRQ05MOUdpZk4x?=
 =?utf-8?Q?7QjyzUnlnMF2XScD5gqqmrveWTYhJ8HZxOYik/J?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d32e9f87-4a7c-4552-e9ae-08d972c4fd26
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:34:23.7447
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: d4asWIh4nAKvQbKrhJoJrZrnjEjAvjKAfRE0bfbizWV7Tk+3/aKbIBVHdhweBgFygUWt9pvOkVkQKfDFQ3+phA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4846

On 08.09.2021 12:32, Jan Beulich wrote:
> On 01.09.2021 10:45, Jan Beulich wrote:
>> Commit 806448806264 ("xen/domain: Fix label position in
>> domain_teardown()" has caused Coverity to report a _new_ supposedly
>> un-annotated fall-through in a switch(). I find this (once again)
>> puzzling; I'm having an increasingly hard time figuring what patterns
>> the tool is actually after. I would have expected that the tool would
>> either have spotted an issue also before this change, or not at all. Yet
>> if it had spotted one before, the statistics report should have included
>> an eliminated instance alongside the new one (because then the issue
>> would simply have moved by a few lines).
>>
>> Hence the only thing I could guess is that the treatment of comments in
>> macro expansions might be subtly different. Therefore try whether
>> switching the comments to the still relatively new "fallthrough" pseudo
>> keyword actually helps.
>>
>> Coverity-ID: 1490865
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> If this doesn't help, I'm afraid I'm lost as to what Coverity means us
>> to do to silence the reporting.
> 
> According to the most recent report this did not help.

Just noticed that this was rubbish: I thought I had committed the change,
but I actually didn't. Now I'm about to - let's see what happens.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:43:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:43:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181906.329244 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwv2-0006xW-7Q; Wed, 08 Sep 2021 12:43:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181906.329244; Wed, 08 Sep 2021 12:43:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNwv2-0006xP-40; Wed, 08 Sep 2021 12:43:12 +0000
Received: by outflank-mailman (input) for mailman id 181906;
 Wed, 08 Sep 2021 12:43:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNwv0-0006xJ-TG
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:43:10 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 52653518-10a2-11ec-b151-12813bfff9fa;
 Wed, 08 Sep 2021 12:43:10 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 3E8C020141;
 Wed,  8 Sep 2021 12:43:09 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 1593013A8B;
 Wed,  8 Sep 2021 12:43:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 6EPBA92vOGEgGgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 12:43:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52653518-10a2-11ec-b151-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631104989; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=NHBNiheUYg9gphq65/36gve6J5qXevlzv8FdaBzRsBQ=;
	b=I37rPJW9BXtjr2SyvCrJuOncYKspk84WiuuqRVj5TGx2gZhW9+T4ZTLEsWuzurWRh3Zoy0
	Lqjp5wDSVZZzzYrC265DOyKBnCVYVFAfBB+K4GTKThTL4lgtqQa3ZquxfqFDF0tVp9ZOvU
	vTMM0UAnCa+I6P6Ku4LZ6n5XFBaJGzY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] tools/libs: fix build of stubdoms
Date: Wed,  8 Sep 2021 14:43:03 +0200
Message-Id: <20210908124303.20680-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In case abi-dumper is available the stubdom builds will fail due to a
false dependency on dynamic loadable libraries. Fix that.

Fixes: d7c9f7a7a3959913b4 ("tools/libs: Write out an ABI analysis when abi-dumper is available")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/libs/libs.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/libs/libs.mk b/tools/libs/libs.mk
index 84130ccbaf..ebdb2a4782 100644
--- a/tools/libs/libs.mk
+++ b/tools/libs/libs.mk
@@ -98,10 +98,12 @@ lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) libxen$(LIBNAME).map
 
 # If abi-dumper is available, write out the ABI analysis
 ifneq ($(ABI_DUMPER),)
+ifneq ($(nosharedlibs),y)
 libs: $(PKG_ABI)
 $(PKG_ABI): lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR) headers.lst
 	$(ABI_DUMPER) $< -o $@ -public-headers headers.lst -lver $(MAJOR).$(MINOR)
 endif
+endif
 
 .PHONY: install
 install: build
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:52:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:52:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181922.329270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNx4A-0000MU-9M; Wed, 08 Sep 2021 12:52:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181922.329270; Wed, 08 Sep 2021 12:52:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNx4A-0000MN-69; Wed, 08 Sep 2021 12:52:38 +0000
Received: by outflank-mailman (input) for mailman id 181922;
 Wed, 08 Sep 2021 12:52:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNx48-0000MH-Hr
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:52:36 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f0680f4d-5440-4476-8297-788d5e60eb95;
 Wed, 08 Sep 2021 12:52:35 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id D9ED120141;
 Wed,  8 Sep 2021 12:52:34 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 83A8D13A8B;
 Wed,  8 Sep 2021 12:52:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 6TCYHhKyOGGZHAAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 12:52:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0680f4d-5440-4476-8297-788d5e60eb95
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631105554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=M6GQ+molqFLn27l3OHP4cZYUO31klpYjR5c6dJdii4s=;
	b=R8j1yemUKEISpGYp/d7VcLx4SM/987wt5penAju4SVQ9v5byrbupbWSfemGKGRoCgX/Rp/
	+YvfBgz/6oRCX0To38wSv+xFad0yRrDus9wI+5qJvBGXeivS0mkwqvKyOjMaEZ/kgSVeUb
	F0zTVwqDhzer1IR2VSuo58SCW2PTcyQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2] Config: use Mini-OS commit 9f09744aa3e5982 for xen-unstable
Date: Wed,  8 Sep 2021 14:52:32 +0200
Message-Id: <20210908125232.20993-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Switch the used Mini-OS commit to 9f09744aa3e5982 in xen-unstable.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- use a specific commit instead of master (Ian Jackson, Jan Beulich)
---
 Config.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Config.mk b/Config.mk
index 4d723eec1d..e85bf18654 100644
--- a/Config.mk
+++ b/Config.mk
@@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?= git://xenbits.xen.org/mini-os.git
 endif
 OVMF_UPSTREAM_REVISION ?= b37cfdd2807181aed2fee1e17bd7ec1190db266a
 QEMU_UPSTREAM_REVISION ?= master
-MINIOS_UPSTREAM_REVISION ?= 051b87bb9c19609976fb038f386920e1ce5454c5
+MINIOS_UPSTREAM_REVISION ?= 9f09744aa3e5982a083ecf8e9cd2123f477081f9
 
 SEABIOS_UPSTREAM_REVISION ?= rel-1.14.0
 
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 12:53:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 12:53:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181929.329282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNx4m-0000zc-Of; Wed, 08 Sep 2021 12:53:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181929.329282; Wed, 08 Sep 2021 12:53:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNx4m-0000zT-L5; Wed, 08 Sep 2021 12:53:16 +0000
Received: by outflank-mailman (input) for mailman id 181929;
 Wed, 08 Sep 2021 12:53:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNx4l-0000ye-Pj
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 12:53:16 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ddae5de8-3375-48e0-a374-03ac561259fb;
 Wed, 08 Sep 2021 12:53:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddae5de8-3375-48e0-a374-03ac561259fb
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631105594;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=10cDCX+HBcAjUdhksN9InxpkGJbG3lqZ9TVZ6Tx08yo=;
  b=hTSnYTVYXw0cz1leKgnKJOCJ0L5vDslk0okEjrS7HsuvPH1s6afL6A0N
   g/77+4dUvuFW1Z+oAyqmh2zT3g5pm+3zTU5gyGIFvHZCdJrrk4wdkA2oC
   pBH/IgQf+cTDiezkLPrwiiKxwF3dbT0WvxI+7Pek8BwpE/mUNQiDW8olF
   I=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: XBuDn1eYi80kGcS6067kbRkSTyfxto9v0neY4S6gvc4MbjAoh1m5jQa00RftLGIafilhAsLeK0
 Ejfy6E7E1M+ulFSKgxgfuwPMIsO2D1vtH760vRnBBHb6hWZz6xfCIWq0osY03W/5/xr3UGa3Mr
 gj7XlQ9laJQ6d4t9AKb6APauv8qpVGf2T+mSFiyUbWCrTWwav/8aLbaStwgl2irtoeXHL45nqI
 rJ413/s9uL3f6DOpP7BWCodx2hOpsQYyC3mAgoerNbkyNFaRFGJ58u1vvqsg3tq0AxbiIQNLDr
 BnYlYPqV8aOYOXsM6Kgp8az8
X-SBRS: 5.1
X-MesageID: 52235485
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:QAnm6K3q8GGcYwnbYlACHwqjBSNxeYIsimQD101hICG9Lfbo7P
 xGzc5rrSMc1gxhIU3IwerwcZVpQRvnhO9ICPoqTP+ftWjdyTqVxeRZgvqZ/9SOIVyKygcw79
 YeT0E6MqyHMbF2t62V3ODaKadx/DDkytHduQ629R4EIG0aCdAEnmMJbXf8YzAGODWuR6BJY6
 Z0jvA3jQZIEk5nHvhTc0N1LdQr0Oe7564ODyR2eSLPDjP+7g9A74SbL/Gf5H1+bxp/hZMZtU
 TVmQ3w4auu99uhzAXH6mPV55NK3PP819pqHqW3+5soAwSprjztSJVqWrWEsjxwivqo8kwWnN
 7FpAplF9hv6knWYnq+rXLWqlvdOXcVmirfIG2j8CzeSP/CNWQH4g169Ktkmy7imh4dVRdHoY
 p2NiyixrFq5Fj77W/ADpDzJltXfwyP0FcfeWJ4tQ1ybWJWUs4TkaUT+VoQFJ8GACr+rIwiGO
 lqSMvY7OxNcVybZXjFvm9pqebcEUgbExKeBkwLvdCc1nxbmHh4yg8TwcQEhHcO9Jg0VplD4K
 DAOr500LVJQ88Ybah0GecGRo+7AnHAKCi8fV66LFH/UKUOPG7Ertr65L004aWhfZQT1ps8mJ
 7ITVNfsio2c1jyTcmJ0Z1F8hzBBG6sNA6dhv23P/VCy/3BrGGHC1zDdLj1/vHQ7ck3E4nVXe
 2+N4lRBOKmJWzyGZxR1wm7QJVKL2ICOfdl5urTdmj++P4jELea69AzscyjYobFAHIhQCfyE3
 EDVD/8KIFJ6V2qQGbxhFzUV2n2ckLy8JpsGOyCltJjlrQlJ8lJqEwYmF657saEJXlLtbE3Zl
 J3JPfinrmgrWe78G7U5yFiOwZbDExS/LL8Okk6+jMiIgfxa/IOqt+fcWdd0D+OIQJ+VdrfFE
 pFq1F+6cuMXsetLOAZeoqa20eh/jwuTV6xPtIhc5y4lJ7Yk8kDf9QbsYRKZEH2KyA=
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="52235485"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MDpphsaQEXAJuP3od3+JzX9XowCXn5PcKEKgJh2o+ZvYNxT+I1DyXIIUE51RAPJjL4mRgdLNfrPf0EW10E4WsDQVxlTzGFK8Q97sGo9TZcRaGrEBbL/cOfK2GTZz50nUw1WMHCy9XKUFVDrcdoAKMcOlb6YK1y67fF3i2jsJhDQYcnJEM3KvHnlRv6Jdn0OW6veZUztNBVuOatBmUjYYMXPxmFEbn9kIOBHCx15GHzih3CjUUVkGQKrchfmRf+C701X27uSZxT32qftbhOmCITVc0qDG5/bgs7u8jLs4C3ilJRV0nyc6mBII4mpXNgmF3/gTHVgBY+kFtLa24DB/9w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=10cDCX+HBcAjUdhksN9InxpkGJbG3lqZ9TVZ6Tx08yo=;
 b=cFXVXMzAlZB0saTHYpNq80EOlGjGygkIFcGuPOkz6A4Vi3da2feI/fKy9QU6/jWxpuP1jKRabAJOS+USO2+cF04SGehSfH7sR5EIWwzDDN0ksm3QpnAGURJ0l3t3FfbsNUfyP0skW2iFj+vMLAXvcjK/0aehEsMeAONkJi4XDOD7K4kVNbCIHyTrXKD29iQDAnJiVXb6QyAVquTJ/F9Pxa/9igQSAsNvmQohFrntx45zHyN8YozcKC1H7s6s6JzVWGhw2rLNBFTjgf4HIsilA5NN1f/4BD4JGlVWI9z13N7crWp7CIZSmnihCMTqtFGrXcCNWu33n6esemNtN3jG5g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=10cDCX+HBcAjUdhksN9InxpkGJbG3lqZ9TVZ6Tx08yo=;
 b=OxsO/h0F+gPBvhGTOlDcpsQd2TaJEJeVOLsYd1Wsz9yUTnaM4Ol5I5ATp0hHbevrM0fvr98yaHPma8OtjNAQ0p16gP0PPhfS6xjR38og9n/t0Ah0GiKeZpG2hrv487oO85dExZTpXl3cIEKSfr1EuzUQ+dMeOYjs8G6KT5Ug9Gs=
Subject: Re: [PATCH] tools/libs: fix build of stubdoms
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210908124303.20680-1-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>
Date: Wed, 8 Sep 2021 13:53:03 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210908124303.20680-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0446.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:e::26) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 352de6fb-deb9-49fa-90cb-08d972c79c8d
X-MS-TrafficTypeDiagnostic: BYAPR03MB4421:
X-Microsoft-Antispam-PRVS: <BYAPR03MB4421B334A5392CBD95CF61F2BAD49@BYAPR03MB4421.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1079;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: FyNPTVhvv7OsvdcV4JebYmkyTiDxmlo+JKiBe+FNa5n2l2cAfGzZ/B/4i4j6efHqNhf5IZjoiY62Z/jj6h7M8kqhahsLmZUMSjLSNVoUVkgko01N0qOrUkQ558jobL+AA6x7TkrDiM2Z6Uh8CeMyGK0Gsg6o+ExkBtWRi+7G+5sb12fMXiLYNz4R6s6V9Si0JQ3rI74hL5VLmMFyVy5WADME8/eadQ2NfvNDAOjSorkxahkbA5fQ5fpttNgacWgn2wRSz6sFDSMcXOch0ErAhxQmRzWRxSyqZK3w1jasEzu+xzRfuyYYNPBw8Jk5RUENnL6jxS4/BELQZnBpdYOcEhI00kjCnUNpnFuDCzyYu97EFKifzkOZ9U7/QCAaaW4BDNk/t8rbiXIxwxLR1EDFow5NyWRMpS8OB1AwEV45sLthL98Uh88S1tfUeBSc/QSYE4uR9nkJ9b3fP8UY1sVeU0s1lpOjHHlJ6GSJ4jbLS/xARDYfScKCVYOK947Rfrj01IswdoXAvq+31T5idMHe5nUwsK0FqIhi9GlLEIXNhaV4+Kqhr9/NivNTVPefPI0kJr854Hs6/fURWbUh7Tml93EZSE68JABjkqhW2g/tyu0TuEdvwWvyrK9uZGJ/UpHTN5oNSspAkD+DazVj7JzMZVoSrS22HkPl+e3mNj6Jw4IxRr/sUwUzEmZUQiXYm4C9GHR1xKuycs+op7KroFVgDp4xRMJ1hrVOuJMZI1p8mD8=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(86362001)(4326008)(186003)(53546011)(31696002)(8676002)(6666004)(55236004)(66946007)(26005)(4744005)(66476007)(66556008)(31686004)(316002)(16576012)(5660300002)(2906002)(36756003)(956004)(2616005)(8936002)(54906003)(6486002)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VFk4aUFEOU9NbkdkcUJrbDdPcS91bHNKemtyeWxYN1h3eEk1cWVTZlVqRm5s?=
 =?utf-8?B?OWNGZVpmQWI5cUxaZXJ4ejdPWE1nR3dDVStZUUJPRnp6TmF6Y1FTb2E3UkYr?=
 =?utf-8?B?enVhYUNNc3lVYTVBcGNJNVdZM2xJajNTTG9lN1pDSEhhUXBmN2xNQWJSYWR3?=
 =?utf-8?B?TjFtSXluNTJKY0VrcnRyZEJrOXh6SlVJTDR4UUpkcmJYTnFValAwdnp1b2tO?=
 =?utf-8?B?U2V5bWVYU1J4VHZiOFEveGtkdHJTREQ5Q2d0MjR1MEZDa1BUQ2llenV5OHNp?=
 =?utf-8?B?dGZnamtkdEU0a3JRNWVidnZqR2VRYXV4R1YxY1JqazBMMDBjT2t0Qzc2Wi9G?=
 =?utf-8?B?QzF6eDlmMHFxN09lVHZibE5XY29oY00zRWQzTk1UbnJMNTNBOHpHdUJBVmRC?=
 =?utf-8?B?QzJ3R0dCd2R3T3JoZzVmSmdKc3lJQmRVTVZia3BkRFNBemo1Ym16R3owSDZ1?=
 =?utf-8?B?UWVxQ1RwVytkZDdiVGo0SHB4dEhBV2lyWlZwVlBDeVR2Z3orSSs3b3NCK2tj?=
 =?utf-8?B?UnYyZ00ydjJkQ0UrTDlzSXJqWHRFVzNCa3JqS0VObmJUbTB0UHFyM2xKMW1Z?=
 =?utf-8?B?QnpDeXNWV1ZDV1NrZmF2VUJ2bWJsYnBDK09nNDhyamJhc09vQkJyUWltT0tw?=
 =?utf-8?B?MFJkWGYvcmRwRURkejNPMnRQWWs4dFFGTld4MUJ5OWcvTlF1R3pXSTZBdW9r?=
 =?utf-8?B?aWs1dVRHOW9NUktud2FWREdWN2tHdCtUZ2w0eWhJbkRFMEUxY3FjKzc1VCtm?=
 =?utf-8?B?NU1SL1FNQm10MnNQZ0VNbGg3M0lXTC80bUJ4aG16ZUVZL1FWNGQ1dkt1UGZl?=
 =?utf-8?B?WWFMMDR2Znk5OGtJd2EveEtkbk1rQzRramNlTGtYS1hYTGZLTUR4alhDUnhJ?=
 =?utf-8?B?b0huSzlhbUhNZytxRnJWTUpkcUhjWGJrWnpSK3E1c3N4a00zMlhMY1I5UHA2?=
 =?utf-8?B?a1cvS1ZoRGFvZXRrWGFoTndNc2E0MXF1LzhQcjN0emtISkZDcXo3aEpjcE8x?=
 =?utf-8?B?NytZd016TWdrRVFiaHJzdkVuUi9MRFdhSGduWTV1MWRLbzBvMW95cDFNK3Z6?=
 =?utf-8?B?K0txZ0lXdmRjbnYrTXExTEhXa2dWaDIzWEpsWWtnTzFUaFZQUEx3eTR5cElL?=
 =?utf-8?B?ZEtuMWNtd3BvZVFLaEFUKzI5SThocUgyTk9xcGNqS0VaRFVNaUEyR2o4Um5H?=
 =?utf-8?B?dGxEZmFtZk5IeUY3c0d5ZE4vV09qZlYwUUlCQ2hqdU0xdUtYVzRaNVBoQ3Qw?=
 =?utf-8?B?bkFJVWU2cUFkMFp2Uy95RmtTRmt0NnZ1Vkx6QndLay8vN3dnTlB6dU51Vndk?=
 =?utf-8?B?T21QMGF0VG5Hc3RqbDdSdTI3YmN5T1UrUEhGRTZyQU1ESklkNEwzVFp3WVd5?=
 =?utf-8?B?UTRTbFM2SW16SlovK3BwUE9MN0RGb2IxSHBVZWJqZHI5dDNlYlZpWjRValZD?=
 =?utf-8?B?WGxQRUJrKzBSK1hiSGVCY2N6L1A3VWVzQWdmVkwwSGx6ZDg5VDhMQXFNZnJC?=
 =?utf-8?B?WVg1dHVoZW90OFA2ZTZQZGVHelJ5LzZrbXNldkNHeFYwSUFrU24wRW1UUjlN?=
 =?utf-8?B?NTlHWk9ZZXlhWFdoUXIvRzVvTk10ZUIwSFp2T0NBL0ZWbFRWQ01XTCtnVmpK?=
 =?utf-8?B?dUlEbm11TWswclMyYmg3UGhxYytkVkEwVnQ5UDRLYWlpTGM4WU9BRzhBZXZM?=
 =?utf-8?B?QW9FK3pXWERvVWxTK09BaU9pR1IvVkNzR200bm40MVlSRkNTMTU5ZE5CNXJr?=
 =?utf-8?Q?DNwg8m9f0a6vUh5v+W8TqWV67Tg6o4yu7Ps7s35?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 352de6fb-deb9-49fa-90cb-08d972c79c8d
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 12:53:10.2136
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PIGfB+IqY0oP9oIl63VglqSKIMnc8rRBeRie7PhE9junrPhRLvfUdtA/FsOdSx8DRi4jMI/YNCGLwAVcvAFHDHd0efpqnflShakf29ZFbmw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4421
X-OriginatorOrg: citrix.com

On 08/09/2021 13:43, Juergen Gross wrote:
> In case abi-dumper is available the stubdom builds will fail due to a
> false dependency on dynamic loadable libraries. Fix that.
>
> Fixes: d7c9f7a7a3959913b4 ("tools/libs: Write out an ABI analysis when abi-dumper is available")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:04:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:04:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.181983.329375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxFs-0004YC-T9; Wed, 08 Sep 2021 13:04:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 181983.329375; Wed, 08 Sep 2021 13:04:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxFs-0004Y5-Pe; Wed, 08 Sep 2021 13:04:44 +0000
Received: by outflank-mailman (input) for mailman id 181983;
 Wed, 08 Sep 2021 13:04:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNxFr-0004Xt-Sv
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:04:43 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0c989046-50a8-4166-8785-446a842621e3;
 Wed, 08 Sep 2021 13:04:43 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2982B2218A;
 Wed,  8 Sep 2021 13:04:42 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id EAF4E13A8B;
 Wed,  8 Sep 2021 13:04:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id NiHqN+m0OGG4HwAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 13:04:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c989046-50a8-4166-8785-446a842621e3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631106282; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=fASvCnULJeUIRs9+t4Wk0OtjCFkWaouH82TpoiiwbKE=;
	b=I/u0pmQjYKnoOStNBLNJaroYMBAMzjdeULBS1z3Yzb+oU7l/hu30+gaGjCCizTj1wIjshq
	6B7n7my7lDK1zSUXA6q4VqWE6579/zWMud9i3DTMcaFvFjx5s9rWTeij6U6FC3ghnByDGT
	TBaWSlb3DD9GoMCEyedi7x6tJZKELSA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>
Subject: [PATCH v2] xen/sched: fix sched_move_domain() for domain without vcpus
Date: Wed,  8 Sep 2021 15:04:40 +0200
Message-Id: <20210908130440.24605-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In case a domain is created with a cpupool other than Pool-0 specified
it will be moved to that cpupool before any vcpus are allocated.

This will lead to a NULL pointer dereference in sched_move_domain().

Fix that by tolerating vcpus not being allocated yet.

Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpupools with different granularity")
Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- Only test for vcpu[0] not being present (Jan Beulich)
---
 xen/common/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8d178baf3d..8f4b1ca10d 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -651,7 +651,7 @@ int sched_move_domain(struct domain *d, struct cpupool *c)
     struct scheduler *old_ops = dom_scheduler(d);
     void *old_domdata;
     unsigned int gran = cpupool_get_granularity(c);
-    unsigned int n_units = DIV_ROUND_UP(d->max_vcpus, gran);
+    unsigned int n_units = d->vcpu[0] ? DIV_ROUND_UP(d->max_vcpus, gran) : 0;
     int ret = 0;
 
     for_each_vcpu ( d, v )
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:21:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:21:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182003.329389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxWI-0007EZ-CM; Wed, 08 Sep 2021 13:21:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182003.329389; Wed, 08 Sep 2021 13:21:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxWI-0007ES-9T; Wed, 08 Sep 2021 13:21:42 +0000
Received: by outflank-mailman (input) for mailman id 182003;
 Wed, 08 Sep 2021 13:21:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNxWG-0007EM-Mu
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:21:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b2cd0142-10a7-11ec-b152-12813bfff9fa;
 Wed, 08 Sep 2021 13:21:39 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-gP9nn5wLPLyUtBLryOhemw-1; Wed, 08 Sep 2021 15:21:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 13:21:35 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 13:21:35 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0047.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.21 via Frontend Transport; Wed, 8 Sep 2021 13:21:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2cd0142-10a7-11ec-b152-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631107298;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=Lw50COFzGHyI/j6BIzWLnNNGnqltLBGeG4UmZVagX/k=;
	b=CuL3zne2BzkQN7Y1aExRlnvFX7tdePk/eA91/EHWJjrWZDFk6+XynGP2qIZUdummXvwzrR
	RkCdJm018IOlE0/XMPpwjumdv8/D6iqUkjz7zI/JLavt7Z8FPAx645gvB+IfJCQ5g6idJG
	613UWkxa8mL0XjRarQngrO7yfIvIEUI=
X-MC-Unique: gP9nn5wLPLyUtBLryOhemw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=arh588hgXUy/sBThSJ14GzgSXAhs4jUSLo0vj0KIvysv0Xip5g4Le3EKIC36QAUcvIig4XgeQSu4aTmtRXZoYgvM7xEXnUEBYr6hLM/AnMoaUp4H0b/M1OfcUYts0SM8NWZUsJAaLKEA8kZpeBvO38E++IUP828Iq5QbJkFXSbMJivZvCz2BOKv+mE8oOBZQqVj1xhV6AZ7y+3zCzBhoRDlbY+jPSQQ6gZNzoIbLRlyfIY8MlQZkKFez02RmXLPFx2bP51syr+DztJdZK7JqBZ7Y1h/Aq8+K+q/XDtjLQP2inZfupH4hlRMP4tqUpUU5va9O1Naw65EvtP8sAEijbw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Lw50COFzGHyI/j6BIzWLnNNGnqltLBGeG4UmZVagX/k=;
 b=NUgdvMOv2vhxETz5nAM7KJp8UMVcUXRINznhFPXu2JVSJ6imcI28GlR8lAl54uw+14LX9wNXcnynxsKhfvnSpbv6TtUylsbzgvCmLFYu8tkot1LlE0HO2mBQZve4LAjR6fnXKumI8GmSXqp8HjnTLGnK9hAy+DwgBj2eiDiKFDBMl8x9hQnj9+Jpb8q0kzclfJMyTtlPDCqOBi3JRxSAOs/+lQ+KeNl/XpNgdWXlt93wVaPiR7OZ/bE3Sfi4GMQA8bq04UHzrk+uaFLloeUF1laWsJW+63sNpR/Ow4J5d1B+pM1rBhPfSwasxNS688NEojyah/xg8qeU/m6KpPBBWA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/3] x86/boot: small video mode handling adjustments
Message-ID: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Date: Wed, 8 Sep 2021 15:21:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0047.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b281a9a-b32e-4018-0c74-08d972cb9468
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2607891FF878894149A5008FB3D49@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Vh/fzuQCpfMkCtUE3DCyDnQAN2zenonAJ8ErY9PVE/qSAnsyI9qaXABBRbF8YYme7cD2HkQs+2Ywb8/W3ysnQNeGYauLLNE4hlScrZJYoNRgop2FUxkPaB245uOKhnOFyxis0TZdOgCxariDcxTL6HPBLZqTSJJbnpJANjCJ33oLFwJOphZdeiedjlDA+7QgQG3umoKKQ8RF+/Wojdudiu0ffTLtKw2iGC3B5wMZqmz1uSyM015H/d7PnoZ2VI2Q451wAxU80EUr+R5Xwk93EEqLyD9kQLnnlLz6jbZnjZQaGVjrozXoySU7wfXMlzgab1KNBeFOC1U6aT9TWvmgsqIumynYvDi5e5ZEo1r604wPxJQPnWqhjUr4dRQoPFXZyl2j4c+J1apg6s+Se1vV1woAOYSDvuzDp0cZsYHi3s/zJDnjKSdcR59PeZojPP38pU+uufJAbs1L8uHT6b/rgDWPMlyuLUqD0nhA9k2iT1qFH1SEsQIkvoDSoY3XM0jlTf3D/nXiTSKIAqNaSWYpzMV7pvg1iBKPu9rknfCilcrIfoBhqqfdXtXGRIL4TfQRs0EyajuZ9thn90nPjQ5wffjFb0VWlY3faeGxKJz0K4J1lSKTjwvht2imFR/+7GNUDUowi5kLgnI8jbmzQYnwsEwdMCzups8gNLQfH5qZubHGrm1qGiwu6lealWJgHWr7qfl/88GRQxr01GXlER4dsc1oC7IPR+MbJr72Tl6ttD5BAnuQHPWwCz9LushiObY+
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(376002)(396003)(136003)(346002)(2906002)(6916009)(31686004)(86362001)(38100700002)(16576012)(956004)(4326008)(316002)(6486002)(66476007)(26005)(54906003)(5660300002)(66946007)(478600001)(66556008)(36756003)(8676002)(558084003)(8936002)(186003)(31696002)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TUtIK1NIS05MVGs0Qzd6bGdSOHlpdjUxNU44VE1vTkxyaFBCc3VvaGI0SEJK?=
 =?utf-8?B?VzBWb2xCOUJCNytmcmw5UFlTOFJGL28zRVFGQkNRajZLQzNwVkJVZ3dibnlZ?=
 =?utf-8?B?bUNGWEQ5WEJoRXljdi8xZDlzK3dZYlV1RjBKTjB1KzduVm1BQWU1ZkpaNWFM?=
 =?utf-8?B?ZDFMbnpNM2dYRGFyMDVPWHBYdjBBSmpURkdmUWRXNEFiS3IyOUd1VHAxelJq?=
 =?utf-8?B?MXg2bndVYkRyOHd2TlhIUmtjQ0YwRXhHWTZOcTdUK3lSMHVrNkFndWJTV3g5?=
 =?utf-8?B?YkJuQ3VnTlJMN3BsQm5jckxEM1RpdXg1QXdEeTRSRURXc09Uc2JDR3JyUTcr?=
 =?utf-8?B?UHZ5TG5CRGJwakJhV2V5M25MK3ZzYm1GNm5VNlhsZ2YzNFZnRDZrOU9oV0lx?=
 =?utf-8?B?RXlFZmp4VVBOOW0xL1Z0eUdxS3dNZnVQdndNZURQYmtjdEprazgzU2FySDc3?=
 =?utf-8?B?elpxVTJJbng0bEZYRU04UGZDTFFmTlZZZmZXdFJtcGJLeTJhMnoyaHZnajlV?=
 =?utf-8?B?VUx0bnl0YXd5R3ROYmlNTCtCa3FPY0d4ckhENGFHd2g3allJV0RpQkZnbzBn?=
 =?utf-8?B?SUU4ODRaTkpQQlV1Nm1lTUVGajNCRzRoYWY0ZkNDaXg4N1daenU3b1cvZFpC?=
 =?utf-8?B?WTh5d3FrMVNpb1VYckNWRmlPU25VYWVUQXlMbGRtaUw0RVY5K3JvQ0dvd2Vw?=
 =?utf-8?B?SzRVY0lvYUFxMGN4eks3WDQ1TktWOTVCNG14WGszbDlJakZCdDNnY2cvMGow?=
 =?utf-8?B?WFJXQkRJZ2VDZGRhLy8ycVVvWUNNMSsvZGhlbGdtR0hwYWNCcUt6MjVEdzMw?=
 =?utf-8?B?U3VLTlNQb2FNSHVGbHFTTzB1bEZnNExzOU5SZWNLZUNQYThDR0JCc1N6K3ZE?=
 =?utf-8?B?UGRIOWtVODRyWi9JZUtYb2lYcERqdVVIb1pqWmthc0xwWjU5bk44R2Q0NVZU?=
 =?utf-8?B?d1k1VnExUXBLcTIvUlhYd3dONERnM0JxMkp1YlRRbG9kZC90V1VKMmJWQ0ZD?=
 =?utf-8?B?WFVYUFJRU2ZjNnJmQXM1RGh0MGdJU0oyMXVlVXRSYzJwbjZBTURWSkpHTE5m?=
 =?utf-8?B?bUQvQXp0QVI5M2VPVUtVRDRIays5bGZvc3MrcGVvVWdXTFVzZiszMCthbDdn?=
 =?utf-8?B?NXJHSGVtYmp5MDE4aHFkN09VdzRBTGFlVzhXMXA0M1IvQm5ST0czOGpLbVJh?=
 =?utf-8?B?b0tITStMSU4wTGpmK0lnVkJxWENmNitzWElVUDFWM2ljUE1MaGQ5TC9wWkUz?=
 =?utf-8?B?YTBud0E2ZmJEbzRNY3RuS05PTnpUUVRmR1MwbXpTTTltS2VuL2sra3hqK3Z1?=
 =?utf-8?B?NVpLaWxNcFo0K1pEbFJFMDVhWFFQVzc5bWRMQVcySm5uL0laWDNGcit0ZkFs?=
 =?utf-8?B?SmFPdUt0ZWpnYUs4Q0xzVXpRZ29PWmV4RTZNS0RBRWRLZXNiSjE4VjlZb1Rk?=
 =?utf-8?B?em9zR2crcjJTOGMrbXdHYjZLcFF5Z2RBUnlSUmkzTElSbWhUcjIzUHN5OE1j?=
 =?utf-8?B?TmRkWUVtV2lycjBwc0xmSG5aZFlHcEM5cGNjSTZveEVNYXV6cWRVUys0RHJy?=
 =?utf-8?B?d3J4M1FpUlhKK01OYlNoVnFONUQwWk1JaytsbUlwT2lidmFvTW9MQUYyUmlO?=
 =?utf-8?B?YnA1c0JNd0JOcTJuN3FoZjFxSEc2YzBTZi9HemtBYWRFMkFuVnBNWXZmNGJ1?=
 =?utf-8?B?RDFsRTZXUEJoMUh5ajEvSUNUVzFuWmJiWThGMHZOUThkcHJncTNKVWZ2Mkhn?=
 =?utf-8?Q?/C03EEOP18iP4AuNAxo7Cw7ZJli+qBJ3wHFkknc?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b281a9a-b32e-4018-0c74-08d972cb9468
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 13:21:35.3444
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: X5FAK0Z/k8fi5W+AxBop8FBPEHMszk1uTKIlDpTHvqO9fe+IrNl99SJ1vMOWvXixZFYmphNfSHWzWIyv8ATN7A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

1: make "vga=current" work with graphics modes
2: simplify mode_table
3: fold branches in video handling code

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:23:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:23:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182011.329401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxY0-00085g-P6; Wed, 08 Sep 2021 13:23:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182011.329401; Wed, 08 Sep 2021 13:23:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxY0-00085Z-Li; Wed, 08 Sep 2021 13:23:28 +0000
Received: by outflank-mailman (input) for mailman id 182011;
 Wed, 08 Sep 2021 13:23:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNxXy-00085T-Lr
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:23:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f220a6dc-10a7-11ec-b152-12813bfff9fa;
 Wed, 08 Sep 2021 13:23:25 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2056.outbound.protection.outlook.com [104.47.5.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-k5LQGFm3N8CDHAhoG7Mksw-1; Wed, 08 Sep 2021 15:23:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4350.eurprd04.prod.outlook.com (2603:10a6:803:3e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 13:23:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 13:23:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0086.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.8 via Frontend Transport; Wed, 8 Sep 2021 13:23:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f220a6dc-10a7-11ec-b152-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631107404;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ce7dFYXhAhhqvbLlY4RJBNyNUGpZQ938LDFfj9KUbSw=;
	b=SlkQ5wwGEQBHUVdeQbox1I3K3lrHnow0UrdUWGY5+irl5Z1f5yFC7h3E5XgYl1WdpjjWNv
	5RVwNhHuvvPCesUmwNWWGF6Pp3r+8ec8O/QX/A9JnH+rUtXqI5uv4H1oXKLwzdkNw7eAwL
	8pTlAbSYV5C00ES/DbltK9y4oImjASE=
X-MC-Unique: k5LQGFm3N8CDHAhoG7Mksw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bHyy9P5qsEeujhaNHiyaRkY+YNhmJ4Af0vq14OimSn9WyiVMFLrDvYLKMPDYNvXz9N4nqyRdUVSL3GUh3JXQW3WUHBqXENOBtYLtQhWqIN5pJ1DU9/xdItJS4mlscHcyV2IWdoszaUd1DRnjKWyUklbNQ7yS9c2hXoZq6N/QjGkZSqel9XlIoEMt14rvf9ZIIte36zYno0BUQyl4BdQr/rTrrLiqAHhfNj/nZP4nLpbQL67FH6WUsXcuH/nQ1kLDqNnQZBbJm31jczt0kTmXYkavrIXpd5LGn9azOzk3gz2ksngQ64pOkawuYsDUmkd0dCotl6i+VFyiq4qQ4xPVGg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ce7dFYXhAhhqvbLlY4RJBNyNUGpZQ938LDFfj9KUbSw=;
 b=DJJLNs97/ESU6hTYdrs8w5axNNnvKDleYCSlnJjA5awRfqJOHafhagqzyK3eFqze6xdXLTLWdzmWhp3YU7iJZGcpyjmc5Qy9ONEb7lmrSWtv0MOrJbD7p8yBqMH2ftyB+kxsOZL3sdAi4Y/I+cRLBCgUXCq85cQnPh3PU0PXHAVar4mzQyFChvlYQg1jsXVHL6TRoGbuvLh8Dauo446WjgY3V9dg/+ybrgAptnX+qZNIzoekPSbjKSLQp06HWNgdgAIt23+idMfnbZtWOVCu1QnX7yxky4Y7ueGiYa5Qnwqpvwiz2geFTdG1S425JSBuwGGQ8I3Js//CdbCamQ7hAQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/3] x86/boot: make "vga=current" work with graphics modes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Message-ID: <0603984c-36aa-8964-630d-2afc632738d9@suse.com>
Date: Wed, 8 Sep 2021 15:23:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0086.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 868a2554-3579-476c-5edd-08d972cbd4b5
X-MS-TrafficTypeDiagnostic: VI1PR04MB4350:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43502A7C0A57417C85E587F3B3D49@VI1PR04MB4350.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cHf4GGo4K157HRFscRfs080e9eun85zF1ynp3Sd+H5jGL/scIVROoEqOUZx7o/W709enSkHsHbhSccqmMJKWoyZMh4kWk1PMVdrA753IURDZtTB6yz3vCQinjmy8AMQU3QN1sdpYxCBsnjAvgj/2Ev7hsFCqw4RV6mevvrXhp+c1vLwP4qeIrVPvwHCBLgy86x4WZja8RqJS91g1G3jzU+X6tvcb+Lkq+rkyMn0EID73NW5ABj6MTYfiKN1QvWkcKS1yO6UktNUByj3wBqTh7Gi5PVeIGsRWf2szD8g5aW3t+uo4ZK/TeuvpEfISWOvJzS4682+VBsVGtneF9/zfUWShSTMtpoz86muDlu7e8cmAHYCb71T2uFlv8HYEoOWU4y789RMBf6GfR1uUoT38juXck6KuAfWv+v+XfZGWhL9RF85lRt426L1L5Aw52Zx6wx+Fc1x1uo28vMmW0XeMrtdtktcEhcsipcrUICTTvH009njjzhzQDri+po4cLfG7TzwuS1TVvZ3/mVS/Zr5Sqpaj6PY197tXTiCYTKvTWHP+pS9FQm4dxzjX1zIazjjXi9Ir0Sfiulz5DJoAfuHW+HWi3bqoc8lWOkZAiYIClj5a4atZdw6JfypERBhnR8XLDTvFGp/BkeoZerYnU/7RsRbhGw7tLaPkJgB5TDaGs9l4CiZoirH5GA17avpWPPGZ4SNUlBZf4/H7mPke+BuA3/zIIqg007/VQbuPGPe1swlJSKkGnCPw0SgxKGY0zGim
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(31686004)(66476007)(26005)(66946007)(54906003)(2616005)(8936002)(508600001)(6486002)(956004)(16576012)(316002)(2906002)(36756003)(5660300002)(6666004)(31696002)(86362001)(83380400001)(8676002)(38100700002)(4326008)(186003)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VmlpbDl1M1M4dE5FVlQ3VklCU1pUNnkxU1RHN09La0xKLzJnTUFNaXRXSmd2?=
 =?utf-8?B?QlcxNHNNekhyc2JlNEdCSnAxTU9pYmcwTHZYMnZyam1rRXQ0VVhSVGRvS1Jt?=
 =?utf-8?B?Q3VUNGFINDNuRlVuekNBelJzbHJyRjRacGVIVnBKckpxSzliOTZlelNuakw1?=
 =?utf-8?B?RW5RMnIzd2o2T2ZKTDFSVndSQ2U3NjFNSGs0UCtaZ3Azc3hxR3J2Sk9IeVhM?=
 =?utf-8?B?VVc5aGtEMkxKN1BVN3pJRXFzOUhCUEttc0laOGlNYVBIVW1zMCtDdnlyTW4w?=
 =?utf-8?B?ZFlvRVE1WkgzSVdSOWdieUNBQkg4VWVWSGdPTmRMZzRvS1FWS2dEM3JqL1hX?=
 =?utf-8?B?bGY3bVEyR2hlM3QrUEpWNXI3UzlodXAzbTFrUGtOQytBMjlqNlhxcmZadW1m?=
 =?utf-8?B?OG85VzFEcEo3RFFOSjJKWDQ3RkE3RG5rTkhhVWlZUlArOVVDYTBObnlXKzFp?=
 =?utf-8?B?d0JqWVpuU256OHZMWkVQbzY0bHl4cmZaalFwOXBxUDduZVNEOFNVc3NXc3VP?=
 =?utf-8?B?ZTFnRWs1QXkxSkJoYVpYTTZEMWtNREtVc1BpaTBjVjF1eVRkZnF2c0t0NVB1?=
 =?utf-8?B?Y1pSanFMUXpsTHBOaGlyNjROSFpYYWNGSDZUL2EzVXhwRHViYTBCSW9pcGtN?=
 =?utf-8?B?OUoxVUFPSDZZWU5uOUdVWW1lNnU2a1R6MkVGTWtQT0ZrVzNWMmxYSk9ybjNR?=
 =?utf-8?B?aS9jWG9aWkY5WlBmMExhcmtLak5JYlA3YWFiU0Y5aXNLY0hyK092SVoza2k4?=
 =?utf-8?B?eE9EeFN6ejY1bnJBZXFaOWphRHJSb1B6ZTR0NXJVTjNIMmVkMlFUdHdKRFJn?=
 =?utf-8?B?YmxVQkM3d0ZCSEMwYUdzMDBWaWFWbjVCMjR2REorejFpMW5QdVE2Tm1MMlNT?=
 =?utf-8?B?UTdqU3N2LzVHQkJyVkp6eVBnY0dmblRLNDdlWU9GN0lDQzhrSlZuaGVJMXRN?=
 =?utf-8?B?OXJ4dTJQazZ4WjJwalE4VzY4Zm5jSkIzSWkyaHgyNGJlZmF5RlM4K2ZWSSsr?=
 =?utf-8?B?K3lTdDZhYUNYelZoU2FMYkhWWkpreUFkVGo0ZUVFSVhSQ01WaFl2TkZTYjBa?=
 =?utf-8?B?T3VjZ0JvRmV2aTdlbHFkUndzeStOYUlxSjEvUXNNR0pjU2NsMHFBTDhxcXpr?=
 =?utf-8?B?L010Ukc1T1g2RTV5K0lrdks2R0hMNzJTejZUN3pMRjBWdDRVeTlHNW1vMVdG?=
 =?utf-8?B?MmRjNEp3S1RSZG16alZoYTl1MTVyZVRsN3c1VEJTL2IvakVXSXVkb214eXMz?=
 =?utf-8?B?SjZOeURibUIrc1hDc2RZanhHSE1QOEZaaDZOSlJwSDU2SGdqcE82QTd3cXRt?=
 =?utf-8?B?ajNtMGVYUnUyMm53T3h4bm05Skg1a0ttdWFJWGpuWk9scnV5YzJFVGhKZGhX?=
 =?utf-8?B?ZE5ZcVc3OUhFaXBCUW85dUl5OHVLdm03NmxWS25IV2RidG5xZjNMeGNRV0Ux?=
 =?utf-8?B?eERnaW1GMWNWdjM0aElTZ3I4c2dmTVRPMllWZ3QxS3EzNUJtWEZYaFVrTHFO?=
 =?utf-8?B?SDc5TDVxSmNnQ0hNL1Bsa3c3b2xnb2lEZGdUY1oxM1lZV1c5Z08xczA0VEt4?=
 =?utf-8?B?bWxVMXcxTkgvYlhzNFFWejJDb2VDdjVzaVhLaURsN3NuQ20vd0FUMTdTTzZv?=
 =?utf-8?B?cGh0TmxFenlXcjBCS0FKdTh3UUJWeGlhYUJqWUMyZXZ4Rkl0RVZ5Tk9nMzNR?=
 =?utf-8?B?SkN3YmxhU3RrSCtMeXN3Vk1tN01Yb0FOZitPeXpXZWZueFFWZmFiTWovRDdP?=
 =?utf-8?Q?Jp2Jg8+6kUlNLqPqp/U/cdZvwg/SrmuW0ypaQGq?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 868a2554-3579-476c-5edd-08d972cbd4b5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 13:23:22.3278
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: smT9nnczqjjekncBFjwGgTqKsUngducdjmoTPiITfSDOqhSg5B2dOgJX5qpC1zC3Y86QyJHHoY3wiWlxfhPzVQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4350

GrUB2 can be told to leave the screen in the graphics mode it has been
using (or any other one), via "set gfxpayload=keep" (or suitable
variants thereof). In this case we can avoid doing another mode switch
ourselves. This in particular avoids possibly setting the screen to a
less desirable mode: On one of my test systems the set of modes
reported available by the VESA BIOS depends on whether the interposed
KVM switch has that machine set as the active one. If it's not active,
only modes up to 1024x768 get reported, while when active 1280x1024
modes are also included. For things to always work with an explicitly
specified mode (via the "vga=" option), that mode therefore needs be a
1024x768 one.

For some reason this only works for me with "multiboot2" (and
"module2"); "multiboot" (and "module") still forces the screen into text
mode, despite my reading of the sources suggesting otherwise.

For starters I'm limiting this to graphics modes; I do think this ought
to also work for text modes, but
- I can't tell whether GrUB2 can set any text mode other than 80x25
  (I've only found plain "text" to be valid as a "gfxpayload" setting),
- I'm uncertain whether supporting that is worth it, since I'm uncertain
  how many people would be running their systems/screens in text mode,
- we're pretty close to needing a 4th trampoline page, so I'd like to
  limit the amount of added code.

For starters I'm also limiting mode information retrieval to raw BIOS
accesses. This will allow things to work (in principle) also with other
boot environments where a graphics mode can be left in place. The
downside is that this then still is dependent upon switching back to
real mode, so retrieving the needed information from multiboot info is
likely going to be desirable down the road.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm not convinced boot_vid_mode really needs setting here; I'm doing so
mainly because setvesabysize also does.

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -575,7 +575,6 @@ set14:  movw    $0x1111, %ax
         movb    $0x01, %ah              # Define cursor scan lines 11-12
         movw    $0x0b0c, %cx
         int     $0x10
-set_current:
         stc
         ret
 
@@ -693,6 +692,39 @@ vga_modes:
         .word   VIDEO_80x60, 0x50,0x3c,0        # 80x60
 vga_modes_end:
 
+# If the current mode is a VESA graphics one, obtain its parameters.
+set_current:
+        leaw    vesa_glob_info, %di
+        movw    $0x4f00, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        movw    $0x4f03, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        leaw    vesa_mode_info, %di     # Get mode information structure
+        movw    %bx, %cx
+        movw    $0x4f01, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        movb    (%di), %al              # Check mode attributes
+        andb    $0x99, %al
+        cmpb    $0x99, %al
+        jne     .Lsetc_done             # Doh! No linear frame buffer
+
+        movb    $1, bootsym(graphic_mode)
+        movw    %bx, bootsym(boot_vid_mode)
+        movw    %bx, bootsym(video_mode)
+
+.Lsetc_done:
+        stc
+        ret
+
 # Detect VESA modes.
 vesa_modes:
         movw    %di, %bp                # BP=original mode table end



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:23:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:23:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182017.329412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxYP-0000Co-53; Wed, 08 Sep 2021 13:23:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182017.329412; Wed, 08 Sep 2021 13:23:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxYP-0000Ch-22; Wed, 08 Sep 2021 13:23:53 +0000
Received: by outflank-mailman (input) for mailman id 182017;
 Wed, 08 Sep 2021 13:23:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNxYO-00007S-0O
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:23:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1861d1b7-16ed-4190-ae91-5a0a250ac00d;
 Wed, 08 Sep 2021 13:23:51 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-I29Mg1gZMG6aBADZ8qyFjg-1;
 Wed, 08 Sep 2021 15:23:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4350.eurprd04.prod.outlook.com (2603:10a6:803:3e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 13:23:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 13:23:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0083.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.4 via Frontend Transport; Wed, 8 Sep 2021 13:23:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1861d1b7-16ed-4190-ae91-5a0a250ac00d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631107430;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+iOZotKkLcJY4CXebIky0sg+x0ejVE7dgfMqo24AelE=;
	b=UgG38ulEC2trvHJ2tw1OqAo/VNWjSjeKDIv++qqAFuZpLEg48mnBUWQDsFeP57zkKIMsda
	2QFAN6xLmg/W48HecRHyPZEOh7entoK58R+Hb4ol6lsxlUjEhjXB4PgIsx2esXNJdsJNW1
	ec1EaDybCcAMV2fLJFiOPs3JAuu7fC8=
X-MC-Unique: I29Mg1gZMG6aBADZ8qyFjg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TBClx6+dHssJUsF+DMg4mZgZ5RgYHpSiSQuQtBJK5aayQtD5cK+iaBxaMMBVznBi9d39YM8aXqYkgewS1yk/4J2FUN7YlFwb9msoxARHp7cqcZuOqQ7xg9FUuYGQjusQpP8qsEQTUv3AoEuKNihoaJYiTTLywEMfeXkw76bkDDOEUyHEHZEzExMyxBwUaNlLyQTIg6h5puGitNfsFEfmD7eGE+xHf4To2UI5eo+Pj2um9OyiSPPij5PWs9FCQfccHAnr5dEk0WCZ9p/9YNAkqN33fPZOE/kNpziyL8cbwXJg1BzjEcChY875IX9fkczMqouzs6ANnaoYdGinSRi6Mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+iOZotKkLcJY4CXebIky0sg+x0ejVE7dgfMqo24AelE=;
 b=WBYGlDUPGe63F7i17EzUGyVBfTQXoFhDmoqlfhUwiMUq7FF9dxHSSNF1jRDLQXs9Cp7Igv69/y1BHeq/tlFUVifhp34+QbgMr7owlRPEHRoRqfNFAbysc7Qnt+xd0SRRqxg95Liw+BMINDVjxRG/4MRfnGypeb1Ud4QTFNJVVPgUiHL9nr9un13Qw0P0/zdSsRoeZ+PhJ4pdvO8cclXLJhYW2xpjixzQPZJk6BMrJGKPTmJN0jzxVHyUJGvVm5eC43MdVTpX1m/tPWUfiky6Sddss2N4wGWBjGlODXyRDOr6gzpYbPRODhRMB32yyJmUYKjWUtYtROyGv3jcUEJy4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/3] x86/boot: simplify mode_table
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Message-ID: <30f0c220-8d19-e8ff-1a8f-dc4e5db890bb@suse.com>
Date: Wed, 8 Sep 2021 15:23:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0083.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 70c691f6-7e6b-44c1-c3d3-08d972cbe3f8
X-MS-TrafficTypeDiagnostic: VI1PR04MB4350:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43507280AEAA842A0A3E12D0B3D49@VI1PR04MB4350.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	x33esPvjYEUxqgnmH0K4KiUKgwC5rfIyeAdOahu3nN++PnhoyJ1mmOc+byF6xr2w4Ssx/JnfEhzSTwzWLGWY5BAyBIHvIXrmbJ8kCXsWycg4cRGHAwT9lZGeeZFeWned1deBJJDo1KJw2ODTQg8Z28fMNbXShGsEpQmt5IJqBmO3mPcbYx85LwasppbjlUTppoy9QjQM2SainYdIrH+hTCdwG4wXNM5gqzygSNopU/zUloDkc+oQRtgnV7SCANYuPxlLrWCwiJsVDx7UsU4gB99HWelLeoPRgXY9v0oiqeXVZwoHgCpm7OC9smHw8zW1PZCb90znDbxn0I6yRHFwgpqNulit17tiNOI6Rw5cMX++lG8bt3YTSj4KxRQKR4rrV3yvi2zwKw0ElaVzIiMfNqGe2kzpFOycby3tExhU7cI6Exnm11UXxulKID7QTnXzdJcAskvW1cdSAWtuYa7IR6J4ktAmTQ+MNf0mbk5V8wiDTY0vYVTWopWjvevnr0QhaFJX0o07cFFxTmSEu0yiXJpMKCUO6EIXVT8oxY6BhlUtWcoQFk47W5naSGumU5p5qOFsgHD0KKdYwFnhM7lv6Kwt8f/2p+GLWJbVpb0kFoLhjtFHKbyvmA/wOUDGH1MoPV7oNFHkEH2d+mafs9B6bHO63402Byb3PPohRelhnIVzyRceU+b4gFonjNbfSYGrTV5ziVm4+IuabNhLED03o0m/UwymGxxKhOIME/H//mg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(31686004)(66476007)(26005)(66946007)(54906003)(2616005)(8936002)(508600001)(6486002)(956004)(16576012)(316002)(2906002)(36756003)(5660300002)(31696002)(86362001)(8676002)(38100700002)(4326008)(186003)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VGk5QnBFT0c1TDFQbVZvQ1hoeHNsQXRScTAvT2JJSXN0andWaGJZR29QRFZj?=
 =?utf-8?B?LzdRTi8vbjk3KzdCZnJyMnNJbVlBTmRBcVROc0JqWVdOV1NSRXZvRVlubHZB?=
 =?utf-8?B?ekJLeFJDUWpYOWF0clducDkxRzhNWTU2UngyOGYvWldSY2p6bG5oN0x1QWw3?=
 =?utf-8?B?bU5TQlplU0JUelhxb0tIb0EvZGVaaGZKZ0R0SHVzbXhab1V1eDU4Z0FzZW5V?=
 =?utf-8?B?cVVuTTZ4dTJJV0VXbWdHUWVpSFAxK2lVUlU3YzhXaWZyWG5kYmpOeVR2Nndm?=
 =?utf-8?B?ZDZ6eFVRdTh4ZXRHMU5OajhLMFNkcDJqMWtiazFDTEcrV0pmZzlPQys4eTY2?=
 =?utf-8?B?dElqMC9KQjgwZHB3Nm0zWlk5L00xVDI0eUgrdnZ5UlFpT0FCcTZ2U1RLdmsx?=
 =?utf-8?B?bkh3QTd6K3lqZ09NMHNKK1dTNmZTbDRIa3RYNUp4RDhUcTlKQjFLQ05neFVz?=
 =?utf-8?B?Y2lwRkhZT3E3MWNhanVkTG1kMWtXZy9WZmEwVVJwdXorSUtGQkhsVlVEMHJH?=
 =?utf-8?B?eG8xVVV2Wmt3OWx4WS9BOG5kRyt0enlVbVlpSUIrd3c4YWpkN1NRMmUwZmw2?=
 =?utf-8?B?RmVyU084TmRvaFUvaElYVFRWM0FDaHk0STN2cHQ3enBBcFlIaVRhR1JmRzJw?=
 =?utf-8?B?SmdxWkQxZytDMmFaY25jUVMrQjdrck9ubjd6T1JvZmpMNUdPRVZydGNxNTZS?=
 =?utf-8?B?RG4rbitWeGF5T2JnZzUweTZnQW5pcUNJSjI3UTRRYzRtNGNwdUR1VUdVSGJp?=
 =?utf-8?B?YU1pNUFETlptZlpPUkVaTCtIdzZNMFpTM1UvNW9XbWZtZlFRUzdkWUM0dVVw?=
 =?utf-8?B?YWVLYlBmQmIwSllYVm1qeHJtZUNTWkpvbnYwS0dnWkFqZDgzbHRrL1doNlQ4?=
 =?utf-8?B?YjQzSEpCK2N4VEVHZzgzWjEzWWVNN0hhL3dZNkx3NitCQVpycVZkZWJzT3Z6?=
 =?utf-8?B?R1QxUjIwTWxLbXJDcE91cm1oS2NNV1hYMDVLQVFWUS9SeDB5MVAzNS92dURj?=
 =?utf-8?B?RmZVYlZyWGk0bWVGczRDdWdzblE1MEdIZUlMNWN1NzdLci95TDVmRExhbHdm?=
 =?utf-8?B?ODZ6S2NTRUMrQTUyeGdkN1FJaFYrbnMrbThQZ1Q4TVBXdmUwVHBuUHhCL09K?=
 =?utf-8?B?QVdvdTlFTlJUSXlUa1hJV1g2Slg4WDBiZUcwRUpJV09McHZJOHljbWN5dXFM?=
 =?utf-8?B?UC9pcXJkUmNsMXJQN2NXSERqUWo2Q1ZMWEx6TWNtVnRxVzNmUEhBak45eFcw?=
 =?utf-8?B?V0l6ZDZPelZlQ0R0MUo3d0JaVm45bjB5S05YRTQxS1dmdGNRN2VxU1BNYlRh?=
 =?utf-8?B?NG9JQngvLytsb2VtYXFLb2VDY1FoYTVDUnpYSGRvUjllOEVhK0V1TVB0ZFli?=
 =?utf-8?B?c0hCSGVFQmlkdVBDbmlPVjBxWnJCS000aUZVR0ZKOWQxV2Z5Sll0cTV2K1dM?=
 =?utf-8?B?bGNkRThXUmg3NkxhdFBxd0owdW5DckVNZ205Ujl0R1M4SnVwQ2RyUHd6RkJI?=
 =?utf-8?B?cDNLVmNsM0Y2dGhoaDFFZ0ozcUxZcUVOeEtDczVCKzdTTnRLbVJiUzlaSmVC?=
 =?utf-8?B?UzBHc3ZRMlB6SmhyVGZYU2EwSU12ZThTdTFJTlIyTnlmVzhxdlJCZHd0ZzVk?=
 =?utf-8?B?M1E2dWVuaXJkMHdzQ2ltdExnTWRtRENPZzZxbkxqL2N6QWZCOUVicjF2U3BR?=
 =?utf-8?B?dnR6cGhkcmdLMStuVFU2ZVJxMXd5MUhmRXpveUxmNDhNcFkveFFYNmJtTDlM?=
 =?utf-8?Q?RL0Zv52yxqjsFzLGyJ1/w7fsYouxuKU8SZeXzD0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 70c691f6-7e6b-44c1-c3d3-08d972cbe3f8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 13:23:47.9363
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MLZcz5k95xbQ0WQLZsq2UJp9gcZbkGZx/g8UVPoxo1p5yOuwFGmzK8//hSb5uoccjcWtW0p8DC+XCT1VPMjSWA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4350

There's no point in writing 80x25 text mode information via multiple
insns all storing immediate values. The data can simply be included
first thing in the vga_modes table, allowing the already present
REP MOVSB to take care of everything in one go.

While touching this also correct a related but stale comment.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -655,8 +655,9 @@ outidx: outb    %al, %dx
 # Build the table of video modes (stored after the setup.S code at the
 # `modelist' label. Each video mode record looks like:
 #        .word        MODE-ID             (our special mode ID (see above))
-#        .byte        rows                (number of rows)
-#        .byte        columns             (number of columns)
+#        .word        rows                (number of rows)
+#        .word        columns             (number of columns)
+#        .word        0                   (color depth; gfx modes only)
 # Returns address of the end of the table in DI, the end is marked
 # with a ASK_VGA ID.
 mode_table:
@@ -665,12 +666,6 @@ mode_table:
         jnz     mtab1
 
         leaw    modelist, %di           # Store standard modes:
-        movw    $VIDEO_80x25,(%di)      # The 80x25 mode (ALL)
-        movw    $0x50,2(%di)
-        movw    $0x19,4(%di)
-        movw    $0x00,6(%di)
-        addw    $8,%di
-
         leaw    bootsym(vga_modes), %si # All modes for std VGA
         movw    $vga_modes_end-vga_modes, %cx
         rep     movsb
@@ -684,6 +679,7 @@ ret0:   ret
 
 # Modes usable on all standard VGAs
 vga_modes:
+        .word   VIDEO_80x25, 0x50,0x19,0        # 80x25
         .word   VIDEO_80x50, 0x50,0x32,0        # 80x50
         .word   VIDEO_80x43, 0x50,0x2b,0        # 80x43
         .word   VIDEO_80x28, 0x50,0x1c,0        # 80x28



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:24:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:24:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182026.329422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxYq-0000r7-FL; Wed, 08 Sep 2021 13:24:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182026.329422; Wed, 08 Sep 2021 13:24:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxYq-0000r0-CO; Wed, 08 Sep 2021 13:24:20 +0000
Received: by outflank-mailman (input) for mailman id 182026;
 Wed, 08 Sep 2021 13:24:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNxYp-0000oA-IN
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:24:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ad47221e-1c2a-4667-93a8-040045733c19;
 Wed, 08 Sep 2021 13:24:18 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-wwpW5kiPO_GK1N_zeZd9Yw-1; Wed, 08 Sep 2021 15:24:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 13:24:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 13:24:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0015.eurprd09.prod.outlook.com (2603:10a6:102:b7::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 13:24:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad47221e-1c2a-4667-93a8-040045733c19
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631107457;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=J7205OxPodD00LxNwAZNGY+4ROFN3Gj6YfOBp01MwvY=;
	b=B7Qk1rwKzQrJUuLsdzJ8HI3ujMUSYD2p+z8Z1ZAPq5wzqFsiwYFqLmRfw8pbgTg7E0F8/x
	pEY2GQqTB6cOXVgSz0f4Mt9jBgbFYet+mUbTmdDbZ/hfhWtoJuuBK5pakE067DMzo2dZjR
	1csXkdI+J/2vtnFYdeeMBP+5+O1kAcw=
X-MC-Unique: wwpW5kiPO_GK1N_zeZd9Yw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VIP/5zXDu66XxYAvSMmlJnEGwVejDrOAIIVPvE7yTQLlg+b9bpEhWm6S23eYH/AbEU3U+UAlfpw33Hv0ocvGWPx5n+uGDkUUeB/8R+apkkwNzAdPVYx/w4XwBdZZxQyDuePkL3tQzta8Mgicgz2Dmak55mKDRvvJPysAbh/nTTImp4Ma2rSnJNdvViuI0iEWOgTwIZUj+b8w5S3cEj2Dbi7gLfm2Vg3G2T7MfXOeh1qsO4JRMUP24EZj4+EWrS6VZuz0gdtboXR7HouSJd0ffNS/9rreDMOiXTnqR093akGDgpxc9moEitBs/rsVEzh10p8VdjO/nnvm8zqb+YoVDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=J7205OxPodD00LxNwAZNGY+4ROFN3Gj6YfOBp01MwvY=;
 b=iHAbpLehg3Bgaj2ZoRxL1VyG2NTeRLgGeVT5BUVofh7yUWW6J3aD4Z3E5xCa+idoXUGTSgVV18h6CEkBnxAwBDHQQzNtFY/n2L2O1UpYZkXWysmENsBzXjhRXQ/ZIM0YTjJE9IJKFxTG2VfBsbJQYs18+oG+JTFELNo+1C3iKrInItHhFpmObB7th1qBf4jty5/BDDKNkw8f4ocqhIDeTxa2OQx7u77Dz7c+lMbaSV5JbktFbD1yH5ZhJZt7bGfN3tsancPHiGsyUdBzFHVqIZOP1TFCJcyVqYWoWq8GYKzFkLwDB1+mQyd92cvTbDM1K+JD2WY0Hdq+DhXv5lZhJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/3] x86/boot: fold branches in video handling code
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Message-ID: <33dc91cf-1c80-b386-f9e0-6cdb8e32d5d0@suse.com>
Date: Wed, 8 Sep 2021 15:24:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0015.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a8993c5b-e677-44ad-01a6-08d972cbf439
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB63043ABFDCC3B6B1F54F648AB3D49@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FKDFgccG75koD33zy5xgGuLGH60kMjSbkr3Y1cTXm3nTFnSkRWz66FWmPwxNGPvHEAAQuShyn3hslcXct5GP8sS46c7oT2VvjHBL1RvOustCw/WL489JubmHjUQKVg7z0eM0VzLdr0IiSgTqopsmyVnZf1suPpOI+tm/Hrtpih2T1rHn31aYo3EwzGO6TYLFY0h6L75CtaBHiuFb07bkJQt0clLPrm21ogQWMb/9nHj0es3N1tqwjwpJRdy8fJdJiY+1WFgIe0eywdZF1BNjNnrH/KfApWpL9VyNb12gIu3FTfA/wZl72XRY6RYHu92WMV5DpHsPyPIItQip3O91mq+GB4g6oFFX00J13ZZ3wyf2VxWI10DD7MFfQWoQKDeKu2pP3CYRyCL2yAxjcru8tT2bKI4rYaI7MleBJ1T4qGwk3jt+MiDSJvesmDETqo16igcSYepdEUe3z3xwTDwRdQ4CdLzSBhnyYSYP1hvERymqMC0u8fRyqbQ72L2P9GLuDx270ncoqcAVaguD1RAgnAHozt0b0Kbo4IJAv+zgXtxIrh/067VtKvs2pvASCV7JLvI1vQGP5dGZSkZubf88sH3P1T7CUTKftBLGATN92djXiwu4fIyjSnuz8rCuN6OCcR1inNiLZ4SUOsGI7bRNMUwcT3qzE1rWmKLA6JdYbyAJzUGhteGHKyqrl21zJMCQoJ7kQjxQGt0ByWDbB7LRa/OcqXrNGJh0DphPbOOGDYQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(366004)(39860400002)(346002)(376002)(8676002)(66476007)(54906003)(4326008)(66556008)(31686004)(478600001)(66946007)(31696002)(83380400001)(186003)(956004)(6916009)(26005)(38100700002)(16576012)(86362001)(316002)(4744005)(2616005)(6486002)(2906002)(5660300002)(36756003)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c3h1b3N3OE5uZDVRS3hJV2FYMWg5TFlYOGFrTDAzQVR2ZE1xNVFNc2Z4UVl0?=
 =?utf-8?B?L0VMZURGbXpBV0JSMDBEekRPblhIZE9LUXN6cVYzYlVlVm9FcVQzVUlKWm51?=
 =?utf-8?B?Q2lSUkZmaVBpYmUyNzRaWitNTTZtcERTekZxV3Zab1RWWm5qdUszbGMveHVx?=
 =?utf-8?B?b3pOMkJiWTVyOWxpZ2xOc1JVSnNaNE1KQit2UFA3RmZ5bHRxaHV2YTl2QlIx?=
 =?utf-8?B?NWlUSm81dFRZN1RwazhMN2FLYXNDbm1yMHBscTJSektYczRCaUNrMThMemg3?=
 =?utf-8?B?eXl2WWI1QWNFNFFEamdNL2kxTDdlbjU5ZnhFVnYyNEhqcGxMaFdqNThCQ1Fi?=
 =?utf-8?B?ckdJYm5VdndRU0RHaGhXYm51eFoxMEJBU3o5TEFKZXhqekdEYS9GUE5LSDdi?=
 =?utf-8?B?R3lCdm9xakdmYjlCYWRlN2VIL3VDcjZycGRNN1phQUVhQThuRDR4T2Y3dUdk?=
 =?utf-8?B?SHk5cmd6YlprR3N2OUtvTWRxaXVHODdra2lJL3BsTjdSVVJWRER4dFgwWUJy?=
 =?utf-8?B?UzA1bXdHZkFQQzlFRFFWNVlNYmJFWjZ6YVZLT2JpV1UraktOYUd4NVBobmUr?=
 =?utf-8?B?RDFxOHVVWS82bGNPSjhqWDhKTVBBZS9leUNQS0dNTUVDQzFWekJUVVNNUTJr?=
 =?utf-8?B?VTFhbEMyZDVsc21QSVVNSitWK2V3Q2FObjJ1SzZYbk5DS2tzc2VxVkFZZ1FZ?=
 =?utf-8?B?eXpXMWtpaUY2WGdCc3dLQ0hidnRoR0F1UlVPRWQ1OVhDZ0JqcFVqUDkwdW9i?=
 =?utf-8?B?UTNEdEMrUkc1WkUyc0ZOdlZ3aElLZTdrWnpYOEZPcTZrL1hHWU9JSFJvbTRo?=
 =?utf-8?B?ZWRZY2cwRW5SSTQ1OFF2NThQenBjUFFFMHFlUENnU0s0U0xkMUJkdG9qSWU5?=
 =?utf-8?B?K1R0clpCRkpaczk0Y281ajlUQzdNUXlpT2sxUGhWdnFKVjUwWWtpRzNoenpX?=
 =?utf-8?B?MnBkS1lSR1AwN1NYMzBWdzkxNjlIZnZjYkZCWnJTb2FmWTNjVHMxNzdLQjhG?=
 =?utf-8?B?bGpIVFFudll0U1hKZnZIdi9BczZpeU1nMWtIaGFZRW5tU3BjYXFZaVJnSUlu?=
 =?utf-8?B?Kzc4anozNHZIVzFCdlVCYllmOWc4TjM1ZGNPVExZS3RHcmlPcUhDSHZhM2w2?=
 =?utf-8?B?S04yVGtwYVRQV1Jja2VKUFFueG9ubkt4eUZVdlpQYVJDVkE2SGp2ZzNqdUNF?=
 =?utf-8?B?ZE40WWl4TzBsWUpQcXdUdGRHaHM1TFBSYkR6cEk0MzYwakhSMi9Da0lOSncx?=
 =?utf-8?B?VmhwNHB1eXB3OXZkVHB0V1NaQ0ozTXdVaC9SL2R1WG9aaFlhVnI3amxZdnpz?=
 =?utf-8?B?UWxBUCtUZnU5SWF6M1pYSC9uQkN2TGNzK1V0bEdiWHVIdTJFSGV5aWhGa3pR?=
 =?utf-8?B?Rm1GK0ZQSCtkTndnM1loaXA2aHFBZUFBaitIQW5ncmduNERWbTBsT0xHL3Ns?=
 =?utf-8?B?Z1IreGdyLzNIOFpWMXZXY2hET2FlczlKRFZ6clRvaDQ5b1VRMlNwenE0ZHVx?=
 =?utf-8?B?RFNXY2xWTnNBUkZuTWsyMVNPM2JwRTh5ZnE3RTZ0RC9GZlU2R0tCRktpcGgv?=
 =?utf-8?B?SGxKNUlaRXZFaCtGUWljcDc4a2pZbHlXOXFTdTlEbnFObkk4QUVJL3ZweHBU?=
 =?utf-8?B?WXl2aG9BZW9mWTRETjRONDA5L0JseUJJS2M1MlBsaE9rbVQweFdoUmpVMUR1?=
 =?utf-8?B?TSthMHhXSDNURTVxSTlHODZQQ3R2TS9DbTIzaUdTblZYaUd0THMvYUIwODE2?=
 =?utf-8?Q?Z7n9MgOfJxTavl24HijFShvLFFHae/9lItJWKSW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a8993c5b-e677-44ad-01a6-08d972cbf439
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 13:24:15.1799
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IbfrS1LMEkPJ6ZgBkIZGOo0awUX2ws4DRB5M6V9AwVPH3CAXZaO5IuB8/XVJ+ikNsa1O3vMHbxQQBXltMNuqJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

Using Jcc to branch around a JMP is necessary only in pre-386 code,
where Jcc is limited to disp8. Use the opposite Jcc directly in two
places. Since it's adjacent, also convert an ORB to TESTB.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
This is an easy change reducing the overall trampoline size a little.
We're pretty close to needing a 4th page, which I'd prefer to avoid for
as long as we can.

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -332,8 +332,7 @@ lment:  movb    $0, (%di)
         cmpw    $0x656d, (%si)          # 'me'
         jnz     lmhx
         cmpw    $0x756e, 2(%si)         # 'nu'
-        jnz     lmhx
-        jmp     listm
+        jz      listm
 
 lmhx:   xorw    %bx, %bx                # Else => mode ID in hex
 lmhex:  lodsb
@@ -401,10 +400,8 @@ mode_set:
         cmpb    $VIDEO_FIRST_VESA>>8, %ah
         jnc     check_vesa
 
-        orb     %ah, %ah
-        jnz     setbad
-
-        jmp     setmenu
+        testb   %ah, %ah
+        jz      setmenu
 
 setbad: clc
         ret



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:32:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:32:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182032.329434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxh1-0002dz-Bq; Wed, 08 Sep 2021 13:32:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182032.329434; Wed, 08 Sep 2021 13:32:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxh1-0002ds-8Q; Wed, 08 Sep 2021 13:32:47 +0000
Received: by outflank-mailman (input) for mailman id 182032;
 Wed, 08 Sep 2021 13:32:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNxh0-0002dm-8T
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:32:46 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 92ca8384-bca1-4eb1-a131-ccc8ccc47595;
 Wed, 08 Sep 2021 13:32:45 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 12AAD1FD84;
 Wed,  8 Sep 2021 13:32:44 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id AE3B313A8F;
 Wed,  8 Sep 2021 13:32:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id s+biKHu7OGF3JgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 13:32:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92ca8384-bca1-4eb1-a131-ccc8ccc47595
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631107964; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=F/Ui6vmGrOMpzxKSJA2nBowgoVyDEDRCsNzbizaCewY=;
	b=sBa2NeLXe//BoVtuNhFB+2kAt6omI0Pss7VxJGa9WMI20GVCH1+KOeu4fa5yiByRaHNHZJ
	AMBZH99scIdEm9bpFqMT4ZWL6Ky4sx8plPcK7VPL75q0iT56DEaJ+wD1LM4keC0SkCnZsq
	GH8bu5+yOScwZDK5/5xiQTx6yxzvZOY=
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210908073640.11299-1-jgross@suse.com>
 <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
Date: Wed, 8 Sep 2021 15:32:43 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="mhQytEXpU9tCTCQw6oBk6yPm459sptWLZ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--mhQytEXpU9tCTCQw6oBk6yPm459sptWLZ
Content-Type: multipart/mixed; boundary="oMRl6C4CSjiTQpTvwN6zP8AdTyooWJhjq";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
References: <20210908073640.11299-1-jgross@suse.com>
 <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
In-Reply-To: <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>

--oMRl6C4CSjiTQpTvwN6zP8AdTyooWJhjq
Content-Type: multipart/mixed;
 boundary="------------9FF5BF531CFAD049E5474674"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------9FF5BF531CFAD049E5474674
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 08.09.21 13:07, Jan Beulich wrote:
> On 08.09.2021 09:36, Juergen Gross wrote:
>> Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update pag=
e
>> table entries") introduced a regression when running as Xen PV guest.
>=20
> The description of that change starts with "pmd/pud_populate is the
> right interface to be used to set the respective page table entries."
> If this is deemed true, I don't think pmd_populate() should call
> paravirt_alloc_pte(): The latter function, as its name says, is
> supposed to be called for newly allocated page tables only (aiui).

In theory you are correct, but my experience with reality tells me that
another set of macros for this case will not be appreciated.

>=20
>> Today pmd/pud_poplulate() for Xen PV assumes that the PFN inserted is
>> referencing a not yet used page table. In case of move_normal_pmd/pud(=
)
>> this is not true, resulting in WARN splats like:
>=20
> I agree for the PMD part, but is including PUD here really correct?
> While I don't know why that is, xen_alloc_ptpage() pins L1 tables
> only. Hence a PUD update shouldn't be able to find a pinned L2
> table.

I agree that I should drop mentioning PUD here.

I will do that change when committing in case no other changes are
required.


Juergen

--------------9FF5BF531CFAD049E5474674
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------9FF5BF531CFAD049E5474674--

--oMRl6C4CSjiTQpTvwN6zP8AdTyooWJhjq--

--mhQytEXpU9tCTCQw6oBk6yPm459sptWLZ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE4u3sFAwAAAAAACgkQsN6d1ii/Ey80
Hgf/f8asQXvDRoEpJZ+a4PWY1tqLCbOIKQQg0umYtg4m+BhRu9vTv/3cRga8vRQaW4Sr2eCccOuS
yfiX/2GfX0QRtl25kB4javG4Sl0Vt044CtkXW1Cq+OtrLngf+F3VpByoONFaMEk17IDsaAL11jak
26TGrdjCD5PT3BmXObcYv5wSvr4dv0DTN5FP8x2Mol2OQJEsD8COUZwj7SfnDADDNUsv4l4q0K/o
M9sDGm7dk42bZwpexON7r9ZUaO9xgCjeUVhXVMDu1sjb8nLCgB969PdUiwFHKW0inXpWGYpV4Gg1
cdLZGYndrihUI69KGSI61JDdPsp3YOoBCt4s9deFSw==
=BJw2
-----END PGP SIGNATURE-----

--mhQytEXpU9tCTCQw6oBk6yPm459sptWLZ--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:33:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:33:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182039.329445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxhz-0003HD-Q3; Wed, 08 Sep 2021 13:33:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182039.329445; Wed, 08 Sep 2021 13:33:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxhz-0003H6-MJ; Wed, 08 Sep 2021 13:33:47 +0000
Received: by outflank-mailman (input) for mailman id 182039;
 Wed, 08 Sep 2021 13:33:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0iEI=N6=epam.com=prvs=98850bb71f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNxhx-0003Gu-C0
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:33:45 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 62e394a0-10a9-11ec-b153-12813bfff9fa;
 Wed, 08 Sep 2021 13:33:44 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188DSOk0002191; 
 Wed, 8 Sep 2021 13:33:40 GMT
Received: from eur01-db5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2052.outbound.protection.outlook.com [104.47.2.52])
 by mx0a-0039f301.pphosted.com with ESMTP id 3axwvm02ac-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 08 Sep 2021 13:33:40 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM4PR0301MB2195.eurprd03.prod.outlook.com (2603:10a6:200:4f::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Wed, 8 Sep
 2021 13:33:34 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.015; Wed, 8 Sep 2021
 13:33:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62e394a0-10a9-11ec-b153-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bOydCZ5mxGwjF7xuFQOdCSAPA4edjgKUmOsFvvtN0mC/olA/H6/UCkEDN/4CRR7lIA7P7TYKU8rEdQf+RRf8F0e1FWbms7RXq1DfBm92X7wWF6GjEKjJRLwgr+5j6pGGEhTVZsMnInDW4tRMLvH84AZbK1pxISSIb7QOF42QJoO4NAfgQhriTL2xEpAF1wt3aRS1oxN9Dy0vLcvjD0NvY0dW5NvOMxPxhljplDVvUFOHB/rAeSA8WxOgrEypVAFGsxp/ljjz+rgOyKnOGaAQCf2fHcZZCJrHF7gSqkdMBMnweY0kM5XfNwf+JlVN0kaem/ZIHxUmH7sBMUEWd8bpnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=L3ub5nCEgz8/BPoFSGoHx/RyAVnfVYyGxyt7KNEG+Ik=;
 b=X/yzr41xWXDQ2IEDS1MJRR75MFFCiIm2FX2pBRrKzuEeLC1i3NPbmHid6KqcupZvlCsf3eEj2/F51i0DVYlN4mVcEO1DqNAv7L6vv5s6xlH4f2kyhWTthYN5kCPbdzM1bBYDlAhk9ZdkcmaBj5t69U/THg3i+mtdq6z4qtHZyorWeh2ukfGq3mncZ1q3vGTICbZMdeWljnjJF6FbFf4ucoxMul8TkYTYMacJs20TCZYpj/kfGinAaqjKvNG4gvYECmpvXSnKDEyvt8yNT/jtRwOF8h+yb1fb45X7UZvIgXvdFvi4mCOkmij2EqWMZeNKD9QRL5c3x5OLEWNB68wwiA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L3ub5nCEgz8/BPoFSGoHx/RyAVnfVYyGxyt7KNEG+Ik=;
 b=p/EfBf2BQh6qUxGgmB5jJd3O1a2WAwzsLwlzzyg5gSCi3co3hqw4l8bZTZg8JLRQXF1JAUU2exGrQd3kDx+6tLw2EMqeGTmJwi9Zu8nJiXmn4BQUInj0oGwWox6WuO4vHszQ7R2P7GxQbLrG5eztwXZlL85TXsht5+Ol2fO8as7gIWXxzPgOiV4y1leddDSmHgpE3RPOeUehq67v9hrWZRszEQQTpSHhZeX+VZIJTDoWDIZ0Y8bN7wrkXvqmDMW5JVoZGISZ3ilnGIvToDvAd8KAzw6A/7P9YNBlp29f2XIAoPJqgadyp0IO6fedkbC9vWtWUg8B48YKPgNWXjZUyg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: 
 AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLICAADFtgIAAE0qAgAEIooCAAAS6AIAABV4AgAA6yAA=
Date: Wed, 8 Sep 2021 13:33:34 +0000
Message-ID: <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
 <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
In-Reply-To: <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 114635a2-f0fd-457a-4269-08d972cd41c7
x-ms-traffictypediagnostic: AM4PR0301MB2195:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM4PR0301MB219563194611A0E9508E5A79E7D49@AM4PR0301MB2195.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 IoY5qpN4GPz8HayI0DywAkj/2Fv35AyaDFTHxjgVADG5KaMwrEWMTPMz3y13RgH16nG/e/UgSae6nkz1z63S2trDXCj2B1sfErPqVEv2ZsZ9pgvbZcwMlcwWSWIMwi4Zdo8VXptX9UtZuoN18XJNCnHPD4Lcra+aRD63wX73wJ90eUeD+HFdjPANEK33IjJK2/x/Sll6k/+NGOQrfa325sPF8wPOSzQisY4NK+KchCWez15TaPiFtDY+g7Jmsuqv5te1cGUXcxtCLAh1QDWbUyRacfr1U2bb07h5q/C5sL/LrlHuVRL865oAnJasWeHTCL9xvd4OC24vxrMnIHK9SDYHtv+uVyqJPqWK0B7z2Gy+yKRD3Ri5WIdBzpDuKxZL8RdtpC22daDaCzRo6vHHktOFn9MEtZEeVct97YQcFruWQJHUXxckm5L6asdYQ7h749bHFuPfLVlUUrWJaIcq3cix0S6DSUbJ8Owq5mqOVhHIZ5eLpVRDZmE6Oic9gyD80PKCF7hUUk5dAL4ug9W0/S/Hvz0Sv1awemrRsJ1H+QeMsFO3LVQ9mrk0akJd7uuLEamJs53Gx4NlOlEEAePx/QHpR9OI08B1QAR77d++OyKonhoDjLgnkBgNkpJwGoEZz8Y9bA08kw+9pn+tyODG+OwBxR/sDQxjOvUkGbCmK4TI3MwvhgVfmfghcOBleEC/uhZxJXg+URz2rgmNWE2AKPL7Pa3iMb+x4j2Gcjy7Xb4fAyes5a7kEOFcEY5zqryg
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(366004)(39860400002)(346002)(136003)(376002)(26005)(110136005)(8936002)(122000001)(86362001)(5660300002)(6506007)(83380400001)(76116006)(31686004)(66946007)(6486002)(66556008)(66476007)(64756008)(66446008)(2906002)(71200400001)(316002)(53546011)(38100700002)(478600001)(2616005)(186003)(6512007)(36756003)(4326008)(8676002)(38070700005)(31696002)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aFhrQlRCOGZLUW1oTHZ4d0pKdHRmazJ1ZkphdlJVUGFEeHRFWjByYU53L0Zp?=
 =?utf-8?B?Ykc1T0xrYXErT3dFZGR1UXF5L1gxcXBycVJKUXVoSERHMmxzbjBJWFpBbUpq?=
 =?utf-8?B?ak1EdUoveXVIRWdVdG9pTzEwaGJIZUU3OGJLRzJFYWFhOUJlcFBXSTV0YVUr?=
 =?utf-8?B?QktDYU5qdUdwM0pDc1E4TldGOTRrK2FvQ2dmTEVzd25HYVlNdGJpSnpZU3FC?=
 =?utf-8?B?Y2RRR3gxMW4xNXVBRVdFZjB6eEZ3aG84QVZwTTN3M1Z6VVNTK0I2RXN2ODJo?=
 =?utf-8?B?WFNhNlNOTnpmQVZsWVdVaVBWMjNzekorbEdDVUdzcm1zR29CbnRNZE5lOXNW?=
 =?utf-8?B?TnhvY3VKTXFjbmNTOTNycXFXaVpPcUFUaldFTDdSOFhsWnhwbjFUZ2MrbTU0?=
 =?utf-8?B?UnhKSk9xSXFFSlYrK1MwbmpHbWRMUTBqZ3h5Z2IvQ3FqaXA2ZWU2UzRBRUdn?=
 =?utf-8?B?QmY5ck9qcDUzWnlNc3gxbnBBbFAvaEIyRC82TkVHVmIxMWQrZkxjVDdkK0tj?=
 =?utf-8?B?NjdvRmxmVmdnN2VLVXBSeXF2Y2dxa1M3OEpyZW0rOXJDa1U1dEFQNmJpYmVZ?=
 =?utf-8?B?dDBQdmdoTVEwNkFhYVVxMW0xekFsSkhtck5qUjhlTVk2M0hSdWMvRzU4WjNO?=
 =?utf-8?B?ZHczd2pRUlhXa2VFOHFWMGIxYlpwMFh4UDdEVVJKTGpwNk82UGYzMk53Wnpi?=
 =?utf-8?B?L2tvdnNGQTZUcURhMXVIem11OVhQVFQwWE1HS3lUYUdVK3l2MDdMTUNYcml6?=
 =?utf-8?B?S2RKTzZVeE10WDVpa25jd1hUL1JlWllSOTZEcTRPb3lDK1JzMTN6bVJWdkpx?=
 =?utf-8?B?RFdrVVphWXRuMkpWeGNaZEZadGU0b0FMOElKbDhSVUwzUDFnNzBBanFUTVYw?=
 =?utf-8?B?QzllZGFCa0tDNFg2cnV5cTlEY1FWa0p4bXNOOVg3TVE0cjBQcEtjVVBnandk?=
 =?utf-8?B?Ym9sdDVVdG5qR0duRlNpSkhyRE9EMXZnQ3gxL1ZNZ2grekVkTnNQb3VRWVBT?=
 =?utf-8?B?NVJTOHh4My9PZW12K2JxekRORkVxK2drQTl3TXZLNkhHOXMxMWJiT3Fic3RI?=
 =?utf-8?B?emJmeVlDU2FLeDlqRmlDdEsxZFhDOElScjJvSzBOWFpoMnNINTYvYm5ZTHYz?=
 =?utf-8?B?VENSUFJYVm5XUHR5RFJuVnQyRDNBU2o2OFRJWExuWVdIcnlTbS93bWhyc1Y2?=
 =?utf-8?B?emUvaC94UW9zaVREQVo1SWNlMnVISm0xeUFIWFQwZmkxWmlpM2tWY3NRdWg5?=
 =?utf-8?B?b2pZa2lNU21oNHNxZzlUc2FtYlFXbmVKYVU0WU1iOWZnaE00RGN2ckRhL2tx?=
 =?utf-8?B?UUVBVjhLZTJ3c3d1UzN4WmxtOVdmTVZIYnpyOXhvbEE1TzFQTFkzYVA5S2J0?=
 =?utf-8?B?bWwveU11VWU5K085WTJJbzRiUXJjeVE4bDlWNEUyYWNFNlFQMDVVdDZveXQ3?=
 =?utf-8?B?UVpjQ3VOcXpKd21uaHBZcVRLYzRZUFBQV2xGT3A1T2JxVWRHT0IwYXJCcFlv?=
 =?utf-8?B?S3JZc2k2RWQ3U0UvL0sybC9Ic0FwYlBORzZmdTBLTU9TYlVkckVQbjR1YlZL?=
 =?utf-8?B?NjlNMUF6N3BiRXJDc0tCY1dER0dqUjVQZjBXNlRJTlJxeGk3Y1F6Ri94bFNP?=
 =?utf-8?B?eEltY1JibjV5YnNjZ1I2RUUwU0lmWXNqcVliRWs1TlJtek4rY3lJU3pxQWZB?=
 =?utf-8?B?VC8yWkxLbGNZdTFURkRtVnl2OEpQYks3Mjlha25Hc3lub0FuOHNEYURTMHdO?=
 =?utf-8?Q?NLQfinxg4NQHGgEee4lsw1pyEktIeEAHeuWhgH6?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <78697C31968CE74B8F4148B7A11CB667@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 114635a2-f0fd-457a-4269-08d972cd41c7
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 13:33:34.6418
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: suqtWY7mdh7x0nqSIBCMVKZSW7rH+4gYQDNpGF4OoYyAsfZLqO+nY6A96QxpqKLs7zuFOG8cLpJVSFwsPDq/6kexq9NgK3SbArAiq3FOFLYHsUcNyyi4Mv9VbrQDqWCV
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0301MB2195
X-Proofpoint-ORIG-GUID: i6oOndyb55rzE_GPRB9OPj3F8kVmHtCi
X-Proofpoint-GUID: i6oOndyb55rzE_GPRB9OPj3F8kVmHtCi
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-08_06,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 mlxlogscore=999 bulkscore=0 mlxscore=0 priorityscore=1501 suspectscore=0
 malwarescore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080086

DQpPbiAwOC4wOS4yMSAxMzowMywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA4LjA5LjIwMjEg
MTE6NDMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDguMDkuMjEgMTI6
MjcsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTk6MzksIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxOTozMCwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDE1OjMzLCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNi4wOS4yMSAxNzozMSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4+Pj4+Pj4g
KysrIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4+Pj4+Pj4gQEAgLTQwMCwxMiArNDAw
LDcyIEBAIHN0YXRpYyB2b2lkIGJhcl93cml0ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldiwg
dW5zaWduZWQgaW50IHJlZywNCj4+Pj4+Pj4+ICAgICAgc3RhdGljIHZvaWQgZ3Vlc3RfYmFyX3dy
aXRlKGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVnLA0KPj4+Pj4+
Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgdmFsLCB2b2lkICpk
YXRhKQ0KPj4+Pj4+Pj4gICAgICB7DQo+Pj4+Pj4+PiArICAgIHN0cnVjdCB2cGNpX2JhciAqYmFy
ID0gZGF0YTsNCj4+Pj4+Pj4+ICsgICAgYm9vbCBoaSA9IGZhbHNlOw0KPj4+Pj4+Pj4gKw0KPj4+
Pj4+Pj4gKyAgICBpZiAoIGJhci0+dHlwZSA9PSBWUENJX0JBUl9NRU02NF9ISSApDQo+Pj4+Pj4+
PiArICAgIHsNCj4+Pj4+Pj4+ICsgICAgICAgIEFTU0VSVChyZWcgPiBQQ0lfQkFTRV9BRERSRVNT
XzApOw0KPj4+Pj4+Pj4gKyAgICAgICAgYmFyLS07DQo+Pj4+Pj4+PiArICAgICAgICBoaSA9IHRy
dWU7DQo+Pj4+Pj4+PiArICAgIH0NCj4+Pj4+Pj4+ICsgICAgZWxzZQ0KPj4+Pj4+Pj4gKyAgICAg
ICAgdmFsICY9IFBDSV9CQVNFX0FERFJFU1NfTUVNX01BU0s7DQo+Pj4+Pj4+PiArICAgIGJhci0+
Z3Vlc3RfYWRkciAmPSB+KDB4ZmZmZmZmZmZ1bGwgPDwgKGhpID8gMzIgOiAwKSk7DQo+Pj4+Pj4+
PiArICAgIGJhci0+Z3Vlc3RfYWRkciB8PSAodWludDY0X3QpdmFsIDw8IChoaSA/IDMyIDogMCk7
DQo+Pj4+Pj4+IFdoYXQgeW91IHN0b3JlIGhlcmUgaXMgbm90IHRoZSBhZGRyZXNzIHRoYXQncyBn
b2luZyB0byBiZSB1c2VkLA0KPj4+Pj4+PiAgICAgIGFzDQo+Pj4+Pj4+IHlvdSBkb24ndCBtYXNr
IG9mZiB0aGUgbG93IGJpdHMgKHRvIGFjY291bnQgZm9yIHRoZSBCQVIncyBzaXplKS4NCj4+Pj4+
Pj4gV2hlbiBhIEJBUiBnZXRzIHdyaXR0ZW4gd2l0aCBhbGwgb25lcywgYWxsIHdyaXRhYmxlIGJp
dHMgZ2V0IHRoZXNlDQo+Pj4+Pj4+IG9uZXMgc3RvcmVkLiBUaGUgYWRkcmVzcyBvZiB0aGUgQkFS
LCBhaXVpLCByZWFsbHkgY2hhbmdlcyB0bw0KPj4+Pj4+PiAodHlwaWNhbGx5KSBjbG9zZSBiZWxv
dyA0R2IgKGluIHRoZSBjYXNlIG9mIGEgMzItYml0IEJBUiksIHdoaWNoDQo+Pj4+Pj4+IGlzIHdo
eSBtZW1vcnkgLyBJL08gZGVjb2Rpbmcgc2hvdWxkIGJlIG9mZiB3aGlsZSBzaXppbmcgQkFScy4N
Cj4+Pj4+Pj4gVGhlcmVmb3JlIHlvdSBzaG91bGRuJ3QgbG9vayBmb3IgdGhlIHNwZWNpZmljICJh
bGwgd3JpdGFibGUgYml0cw0KPj4+Pj4+PiBhcmUgb25lcyIgcGF0dGVybiAob3Igd29yc2UsIGFz
IHlvdSBwcmVzZW50bHkgZG8sIHRoZSAiYWxsIGJpdHMNCj4+Pj4+Pj4gb3V0c2lkZSBvZiB0aGUg
dHlwZSBzcGVjaWZpZXIgYXJlIG9uZXMiIG9uZSkgb24gdGhlIHJlYWQgcGF0aC4NCj4+Pj4+Pj4g
SW5zdGVhZCBtYXNrIHRoZSB2YWx1ZSBhcHByb3ByaWF0ZWx5IGhlcmUsIGFuZCBzaW1wbHkgcmV0
dXJuIGJhY2sNCj4+Pj4+Pj4gdGhlIHN0b3JlZCB2YWx1ZSBmcm9tIHRoZSByZWFkIHBhdGguDQo+
PiBCdXQgaW4gY2FzZSBvZiBCQVIgc2l6aW5nIEkgbmVlZCB0byBhY3R1YWxseSByZXR1cm4gQkFS
IHNpemUuDQo+PiBTbywgdGhlIGNvbXBhcmlzb24gaXMgdGhlIHdheSB0byB0ZWxsIGlmIHRoZSBn
dWVzdCB3YW50cyB0byByZWFkDQo+PiBhY3R1YWwgKGNvbmZpZ3VyZWQpIEJBUiB2YWx1ZSBvciBp
dCB0cmllcyB0byBkZXRlcm1pbmUgQkFSJ3Mgc2l6ZS4NCj4+IFRoaXMgaXMgd2h5IEkgY29tcGFy
ZSBhbmQgdXNlIHRoZSByZXN1bHQgYXMgdGhlIGFuc3dlciB0byB3aGF0IG5lZWRzDQo+PiB0byBi
ZSBzdXBwbGllZCB0byB0aGUgZ3Vlc3QuIFNvLCBpZiBJIGRvbid0IGNvbXBhcmUgd2l0aCAweGZm
ZmZmZmZmIGZvciB0aGUNCj4+IGhpIHBhcnQgYW5kIDB4ZmZmZmZmZjAgZm9yIHRoZSBsb3cgdGhl
biBob3cgZG8gSSBrbm93IHdoZW4gdG8gcmV0dXJuDQo+PiBjb25maWd1cmVkIEJBUiBvciByZXR1
cm4gdGhlIHNpemU/DQo+IFdlbGwsIGJ1dCB0aGF0J3MgdGhlIGNvbW1vbiBtaXN1bmRlcnN0YW5k
aW5nIHRoYXQgSSd2ZSBiZWVuIHRyeWluZw0KPiB0byBwb2ludCBvdXQ6IFRoZXJlJ3Mgbm8gZGlm
ZmVyZW5jZSBiZXR3ZWVuIHRoZXNlIHR3byBmb3JtcyBvZg0KPiByZWFkcy4gVGhlIEJBUnMgYXJl
IHNpbXBseSByZWdpc3RlcnMgd2l0aCBzb21lIHIvbyBiaXRzLiBUaGVyZSdzDQo+IG5vIGhpZGRl
biAybmQgcmVnaXN0ZXIgcmVjb3JkaW5nIHdoYXQgd2FzIGxhc3Qgd3JpdHRlbi4gV2hlbiB5b3UN
Cj4gd3JpdGUgMHhmZmZmZmZmZiwgYWxsIHlvdSBkbyBpcyBzZXQgYWxsIHdyaXRhYmxlIGJpdHMg
dG8gMS4gV2hlbg0KPiB5b3UgcmVhZCBiYWNrIGZyb20gdGhlIHJlZ2lzdGVyLCB5b3Ugd2lsbCBm
aW5kIGFsbCByL28gYml0cw0KPiB1bmNoYW5nZWQgKHdoaWNoIGluIHBhcnRpY3VsYXIgbWVhbnMg
YWxsIGxvd2VyIGFkZHJlc3MgYml0cyBhcmUNCj4gemVybywgdGh1cyBhbGxvd2luZyB5b3UgdG8g
ZGV0ZXJtaW5lIHRoZSBzaXplKS4NCj4NCj4gV2hlbiB0aGUgc3BlYyBzYXlzIHRvIHdyaXRlIDB4
ZmZmZmZmZmYgZm9yIHNpemluZyBwdXJwb3NlcywgT1Nlcw0KPiBzaG91bGQgZm9sbG93IHRoYXQs
IHllcy4gVGhpcyBkb2Vzbid0IHByZWNsdWRlIHRoZW0gdG8gdXNlIG90aGVyDQo+IGZvcm1zIG9m
IHdyaXRlcyBmb3Igd2hpY2hldmVyIHB1cnBvc2UuIEhlbmNlIHlvdSBkbyBub3Qgd2FudCB0bw0K
PiBzcGVjaWFsIGNhc2Ugc2l6aW5nLCBidXQgaW5zdGVhZCB5b3Ugd2FudCB0byBlbXVsYXRlIGNv
cnJlY3RseQ0KPiBhbGwgZm9ybXMgb2Ygd3JpdGVzLCBpbmNsdWRpbmcgc3Vic2VxdWVudCByZWFk
cyB0byB1bmlmb3JtbHkNCj4gcmV0dXJuIHRoZSBpbnRlbmRlZCAvIGV4cGVjdGVkIHZhbHVlcy4N
Cj4NCj4gSnVzdCB0byBnaXZlIGFuIGV4YW1wbGUgKHBlcmhhcHMgYSBsaXR0bGUgY29udHJpdmVk
KTogVG8gc2l6ZSBhDQo+IDY0LWJpdCBCQVIsIGluIHByaW5jaXBsZSB5b3UnZCBmaXJzdCBuZWVk
IHRvIHdyaXRlIDB4ZmZmZmZmZmYgdG8NCj4gYm90aCBoYWx2ZXMuIEJ1dCB0aGVyZSdzIG5vdGhp
bmcgd3Jvbmcgd2l0aCBkb2luZyB0aGlzIGluIGENCj4gZGlmZmVyZW50IG9yZGVyOiBBY3Qgb24g
dGhlIGxvdyBoYWxmIGFsb25lIGZpcnN0LCBhbmQgdGhlbiBhY3QNCj4gb24gdGhlIGhpZ2ggaGFs
Zi4gVGhlIGFjdGluZyBvbiB0aGUgaGlnaCBoYWxmIGNvdWxkIGV2ZW4gYmUNCj4gc2tpcHBlZCBp
ZiB0aGUgbG93IGhhbGYgc2l6aW5nIHByb2R1Y2VkIGF0IGxlYXN0IGJpdCAzMSBzZXQuIE5vdw0K
PiBpZiB5b3Ugd2VyZSB0byBzcGVjaWFsIGNhc2Ugc2VlaW5nIGZmZmZmZmZmOmZmZmZmZmY/IGFz
IHRoZQ0KPiBsYXN0IHdyaXR0ZW4gcGFpciBvZiB2YWx1ZXMsIHlvdSdkIGJyZWFrIHRoYXQgKGlt
byBsZWdpdGltYXRlKQ0KPiBhbHRlcm5hdGl2ZSBwcm9jZXNzIG9mIHNpemluZy4NCg0KSG93IGFi
b3V0Og0KDQpzdGF0aWMgdm9pZCBndWVzdF9iYXJfd3JpdGUoY29uc3Qgc3RydWN0IHBjaV9kZXYg
KnBkZXYsIHVuc2lnbmVkIGludCByZWcsDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVpbnQzMl90IHZhbCwgdm9pZCAqZGF0YSkNCnsNCiDC
oMKgwqAgc3RydWN0IHZwY2lfYmFyICpiYXIgPSBkYXRhOw0KIMKgwqDCoCBib29sIGhpID0gZmFs
c2U7DQoNCiDCoMKgwqAgaWYgKCBiYXItPnR5cGUgPT0gVlBDSV9CQVJfTUVNNjRfSEkgKQ0KIMKg
wqDCoCB7DQogwqDCoMKgwqDCoMKgwqAgQVNTRVJUKHJlZyA+IFBDSV9CQVNFX0FERFJFU1NfMCk7
DQogwqDCoMKgwqDCoMKgwqAgYmFyLS07DQogwqDCoMKgwqDCoMKgwqAgaGkgPSB0cnVlOw0KIMKg
wqDCoCB9DQogwqDCoMKgIGVsc2UNCiDCoMKgwqAgew0KIMKgwqDCoMKgwqDCoMKgIHZhbCAmPSBQ
Q0lfQkFTRV9BRERSRVNTX01FTV9NQVNLOw0KIMKgwqDCoMKgwqDCoMKgIHZhbCB8PSBiYXItPnR5
cGUgPT0gVlBDSV9CQVJfTUVNMzIgPyBQQ0lfQkFTRV9BRERSRVNTX01FTV9UWVBFXzMyDQogwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDogUENJX0JBU0VfQUREUkVTU19NRU1fVFlQRV82NDsN
CiDCoMKgwqDCoMKgwqDCoCB2YWwgfD0gYmFyLT5wcmVmZXRjaGFibGUgPyBQQ0lfQkFTRV9BRERS
RVNTX01FTV9QUkVGRVRDSCA6IDA7DQogwqDCoMKgIH0NCg0KIMKgwqDCoCBiYXItPmd1ZXN0X2Fk
ZHIgJj0gfigweGZmZmZmZmZmdWxsIDw8IChoaSA/IDMyIDogMCkpOw0KIMKgwqDCoCBiYXItPmd1
ZXN0X2FkZHIgfD0gKHVpbnQ2NF90KXZhbCA8PCAoaGkgPyAzMiA6IDApOw0KDQogwqDCoMKgIGJh
ci0+Z3Vlc3RfYWRkciAmPSB+KGJhci0+c2l6ZSAtIDEpIHwgflBDSV9CQVNFX0FERFJFU1NfTUVN
X01BU0s7DQp9DQoNCnN0YXRpYyB1aW50MzJfdCBndWVzdF9iYXJfcmVhZChjb25zdCBzdHJ1Y3Qg
cGNpX2RldiAqcGRldiwgdW5zaWduZWQgaW50IHJlZywNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdm9pZCAqZGF0YSkNCnsNCiDC
oMKgwqAgc3RydWN0IHZwY2lfYmFyICpiYXIgPSBkYXRhOw0KDQogwqDCoMKgIGlmICggYmFyLT50
eXBlID09IFZQQ0lfQkFSX01FTTY0X0hJICkNCiDCoMKgwqDCoMKgwqDCoCByZXR1cm4gYmFyLT5n
dWVzdF9hZGRyID4+IDMyOw0KDQogwqDCoMKgIHJldHVybiBiYXItPmd1ZXN0X2FkZHI7DQp9DQoN
Ckl0IHNlZW1zIHRvIHNvbHZlIGFsbCB0aGUgcXVlc3Rpb25zIHdlIGhhdmU6IG1vcmUgd29yayBv
biB3cml0ZSBwYXRoLA0KDQpubyBjb21wYXJpc29uIHdpdGggMHhmZmZmZmZmZjogQkFSJ3Mgc2l6
ZSBpcyB1c2VkIHRvIG1hc2sgdW53YW50ZWQgYml0cy4NCg0KQlRXLCBiYXJzW2ldLmd1ZXN0X2Fk
ZHIgPSAwOyBpcyBuZWVkZWQgYXMgdGhpcyBmaWVsZCBjYW4gYmUgcmUtdXNlZC4NCj4NCj4gSmFu
DQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:45:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:45:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182053.329467 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxsr-0005Ac-0z; Wed, 08 Sep 2021 13:45:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182053.329467; Wed, 08 Sep 2021 13:45:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxsq-0005AV-UH; Wed, 08 Sep 2021 13:45:00 +0000
Received: by outflank-mailman (input) for mailman id 182053;
 Wed, 08 Sep 2021 13:45:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNxsq-0005AL-9U; Wed, 08 Sep 2021 13:45:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNxsp-0005ua-VN; Wed, 08 Sep 2021 13:44:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNxsp-0008HF-Lo; Wed, 08 Sep 2021 13:44:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNxsp-00038c-LH; Wed, 08 Sep 2021 13:44:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=slWShBUiNUHpEsuLD9dGfutj2xYhFePNHcZB9yoeGng=; b=C3h9fza/ewBMrrRVc1RdIdDyMK
	iXIXs5QwGC5ryv0ujOBbmx82U/swDnpeTsexgRt9mMZo6mnhVgTqCZXgqBng2st4xyQs0PW6fjt/V
	DYhlZMFzqgtRW9VTBDkC21mQYe8DRa8eZq5JgIvCfnlk22B3J+qACb0W5grA0fPqoNSs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164879-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164879: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ac08b1c68d1b1ed3cebb218fc3ea2c07484eb07d
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 13:44:59 +0000

flight 164879 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164879/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-libvirt      8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                ac08b1c68d1b1ed3cebb218fc3ea2c07484eb07d
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  403 days
Failing since        152366  2020-08-01 20:49:34 Z  402 days  701 attempts
Testing same since   164879  2021-09-08 04:07:16 Z    0 days    1 attempts

------------------------------------------------------------
7251 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2234642 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 13:49:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 13:49:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182061.329481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxwy-0005to-NY; Wed, 08 Sep 2021 13:49:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182061.329481; Wed, 08 Sep 2021 13:49:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNxwy-0005th-KW; Wed, 08 Sep 2021 13:49:16 +0000
Received: by outflank-mailman (input) for mailman id 182061;
 Wed, 08 Sep 2021 13:49:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNxwx-0005tb-Hc
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 13:49:15 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8d7dedc6-10ab-11ec-b156-12813bfff9fa;
 Wed, 08 Sep 2021 13:49:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d7dedc6-10ab-11ec-b156-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631108954;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=DtULsXlCgXIOQLWWWQwNgu4W3fzzOpTBB/X0vAO+RWA=;
  b=RYuELsEBS7K1NFG5mDgul2FA+Fn56+b8yo77dzwGELMi+8O//RSo1pqq
   bAfFxqEJoJXUKBHZnHqThFGo38WNOTchy/iw9GhZdAuEAtAr/QMM9FKRX
   2fw14NOi0r9H+/SkdMJPmXawXV1g/a9UyvAJAE82CmgknOHP4pPljuP3l
   0=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: uNySJenAmPay8mybxv87qnmwJEb+pvmhv0Z5N242NqaN/k9Lz1h4VQDs8XhHdv+57jNqGy54oj
 gYOjpEDGh5afllPVCHokReWVkKdGafRP//HxKdr0VcjT1Df0Xz21N8O7FAIpkEqpY4JkUvgnkh
 APBWceMIi1kK91lfXpKo4+tlIZux/bLAC3OOFXJdssBOg9eBNKVGIhpHpFfOluDigi1Mf4LzeI
 vE4wrshJjgrHhkHLuyoKfPNDiUAUoc5Ycbp2fIH8I6fVyfFYyoikxPC5+XxPILNK4TSKguYsdC
 Sr9Zr2fgtZzgSDkEQEEHIDAm
X-SBRS: 5.1
X-MesageID: 54017415
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:rj4KqKkTx9ffw4fos5fCsg2DgG/pDfP4imdD5ihNYBxZY6Wkfp
 +V8cjzhCWftN9OYhodcIi7SdC9qADnhOZICOgqTMCftWbdyQ6Vxe1Zg7cKhgeQfREWldQto5
 uIEZIObOEYZGIS5aqU3OD7KadG/DDtysCVbJLlvhVQpHZRGsJdBmlCazqzIwlTfk1rFJA5HJ
 2T6o5svDy7Y0kaacy9Gz0sQ/XDj8ejruOvXTc2QzocrCWehzKh77D3VzKC2A0Fbj9JybA+tU
 DYjg3C4Lm5uf3T8G6W64aT1eUapDOtouEzTvBkuPJlZQkEuTzYKriJbofy8Azc+ImUmRcXeZ
 f30lcd1o9ImgjslymO0GHQMk/boX8TAlLZuC+laEHY0L/EbSN/D9AEiZNScxPf5UZltNZg0L
 hT12bcs5ZMFxvPkCn0+tCNDnhR5z+JiGtnlfRWg21UUIMYZrMUpYsD/FlNGJNFGC7h8ogoHO
 RnEcmZ7vdLdlGRaWzfowBUsa6Rtl9aJGb6fqEmgL3g79F7pgEG86JD/r1hop4pzuNNd3Flj9
 60TphVqA==
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="54017415"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QqXHydLbR7CT3aaSXvofUG6GsZhmgH2gJK1HP523f6z2jSx/9zGRJRx8rTM8C9tp00zu1nNgDIFiqrSW1IP6u00GL+G2xs3rcv4iemX6p0PltvYqiwaIsqC0vPTO11ahIxiyZtORtrKL8Ws3JglABS2xxN/PFJRLoMlKumlliSLgIv/TZLAnzvT1ck+qIrO2rwgfS8Ujh/u1IthN95T7GxNRjpwPULNki59l3qrvYFwYR8R+KbrwkyuNfCMomvuzWlMBfnEHCnX8t1bds/uIzN1N4AZ98do67UenZI5RnuN7/OU6Db2Siw1NubsOHy+L+gaxjrrhoDSmlargtSveaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DtULsXlCgXIOQLWWWQwNgu4W3fzzOpTBB/X0vAO+RWA=;
 b=RYj0K05T0Jg1CHWaGlx+kJLXOAw7CRi09t8QV6w2fko/yOHzX+UO9w6cVaV2ZsEHx5loNp/oIumnyUaehnYtglZk833iUUtP5Mx/yXRGEB96zeddtSD0SA44YQ13TPmgcvT76pzPuSPdsOr+ShLjJTt07l+gv7mw04XVMW2lMTX/8R2Dzye22nRi2bqbzsxTOsYvvgmNXBPlroLGczxZ0/xPVVVqQkNL6G5XUCxr4vpDLURoo0YKZ/OLIuaR3sWgU1QUDh/DQrq7a3hM30DIWJcE+enllrXOEeyUAst/3WFbuKm0V0Udc2cORNKlmTyWRlu1B3Iukfg/SaFxRtToyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DtULsXlCgXIOQLWWWQwNgu4W3fzzOpTBB/X0vAO+RWA=;
 b=YsS7LAbLVef5J/1LWPh+ZM2bX+dv0mvoV4RbZX0MzLcDULDagUjk6J5zgS8S1pkaMdnnNDy1IDsDm1dxhs9YActIP89lA4rfZnuXHusbPYVtuXEluR9gfoA65zyy3nZPI89/1TfehbOY04snrVWLeC3JVy3iEMCDZu6RsuSW0ts=
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>
References: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
 <33dc91cf-1c80-b386-f9e0-6cdb8e32d5d0@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 3/3] x86/boot: fold branches in video handling code
Message-ID: <6d74fb5a-d7ca-8fab-7f6e-09f6a0a9a700@citrix.com>
Date: Wed, 8 Sep 2021 14:49:04 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <33dc91cf-1c80-b386-f9e0-6cdb8e32d5d0@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LNXP265CA0083.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:76::23) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f61a4b8f-1c68-403b-4bf1-08d972cf6f3e
X-MS-TrafficTypeDiagnostic: BYAPR03MB4551:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4551744A3375AD74B9B1FF4ABAD49@BYAPR03MB4551.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: XEStdw1XPUVsMryYnb7YpViuRCWw0OK0gO/jiTu2oHlA2LUdKVeP8KfpTDgW4Lbu+pj5LrHRhvHlFKPSmDskXadCRAQncTMJk70H1BPHpO02kwKPg41vJkMS9qEnF4zwkM6vLcRWB9HWr53vEb/LXcy7DdkZ5wXsuCaOzHQWztdX8+VWIyahKKI7CGTgrMtm8taWfGdsGhcYeOq3SrVsTh1uN1V7p7BSInu4ookwZdG/fhWEhSF1lDprbSGfnkE6FbRfvFxnpCG0hsyAzTearVAb+3P9x10kzV9wEvV0eQ7uYZshWcxjUNyI+NYrRjiKUMv3bMD1fubWJME9OBGyODP1b33dPwodg9rTtA541R4MxTU27eaL4z8J83Ay4bQhUtbfXqN8zgz4p6EO7oAREOdiPjpetlqrRlxvkTIvECfgBVm/wUmSXJ3UCdZroqw70Ox5jf9vqUR48qtGWtmd0H4vdJrIIqfN59sGFhVRAWKkHHoSrf4sAVAs1UtjerO3b7VJJQJRJnv2t1B60VFRr2F1k7e8ViZPIriUa1XCVKu+TVc3kHjdqfMmctGYo41Qqxpyl917HN8Bys/1nOyUMZh2FE+C5ywvTrBeGNmA+GTbfTilQs6VnRn/42WFDp80b5+Fqt8lKGXuLfVUMV75twE0V7Xb+CBt3EVW71LtwAC3tdLz2KA6sLzzot2a2lMTMrSAGNpuUioQwYe5M1Vckg+sGz3I7Apo/Ts6t6SJmUE=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(366004)(396003)(346002)(136003)(376002)(8936002)(107886003)(6486002)(478600001)(4326008)(5660300002)(66946007)(2616005)(66476007)(6666004)(66556008)(956004)(8676002)(38100700002)(16576012)(31696002)(83380400001)(31686004)(26005)(110136005)(55236004)(2906002)(36756003)(53546011)(86362001)(186003)(316002)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZEpyT2p6aVd2YjlkSnhwa2I2SGR0OFQ3L2E1ME9zeGRzZVBIcWRuNDVERVVC?=
 =?utf-8?B?MjdTQitKR1hNeVNWdVBGaXFzTlduSWsvZ2ljMHJ3VVJJYVI4d3ZKeXJ2am5o?=
 =?utf-8?B?TktsM2RTK1ZxVzBmQURURUphdnZJN1VxRzUxL0VSdzRPN2hHYmh6US91aGJV?=
 =?utf-8?B?d3l6MUJHREprSmlrVmNVZWtFbHZsNyszNmNNUUdGUE82eDBVNUZkdFZjL2Rm?=
 =?utf-8?B?R0VzVUNBNnU1N0ZWN0dreXhlNFgvT1JmazQ2dlMrUVB0aEoxa1NVczBtSDV1?=
 =?utf-8?B?UTAxMS9lemhYY3lkUGEzTUVjWWRKUFVJZ2pGV0xINXlhV1F5Tm0vV3IrakUz?=
 =?utf-8?B?bEV1YnJmS1JGTlZLWmVSdG1ZSHdHOVhjWXJOV1Z0ZWVLME5raFRmcEwrSyta?=
 =?utf-8?B?RXpyT083b3YyR21KN1VWUWdNbThOOVM2TXdJbzd2UXplaExkUE1YMWJCaXZG?=
 =?utf-8?B?Q0dKQ1Q5Z2ZHcTFscktFaVRKdWpid3Fwei9hOGNjT2lwOVhhSm14b3dSOWhv?=
 =?utf-8?B?WjZTN3A2YmhEb2tqZUlqVWwydmVwYUwxeVNIOWN5NFZvajZJbmtuV3BSTTNB?=
 =?utf-8?B?dUFBSXFNNjJYRCttQ3NndzNQRE84T0RWMUtEYUlJZnQ3TzhOeXRDdU1rNU1j?=
 =?utf-8?B?ekNYaHowY0hkNVpHL2t3c21yVVdQcHhUZWlRcWJ4bVJvaTcwTEswUm1CTDNO?=
 =?utf-8?B?ZGlrelFRSmdZTjdkdDBZUVZFY0pkK3dLcjFYUzVHa0ZqMlNYaE14dnVnakxk?=
 =?utf-8?B?QmhJU1pTQmU3Q1phNURub0VQcjZNQkdUdkRHdVhibjJtRS9OUFNTRUR4RnAz?=
 =?utf-8?B?SjNMT0YxeXMydG1tTUdWU2R6NWtaWHY2QUZyRnExOURKMDZ6SFp5VE44QmRr?=
 =?utf-8?B?ZlpRYzdKaXFnb0tOOVliVVdnSnpOV2lyTFBjcUZkdm42cUtBazZCM3p1SnYr?=
 =?utf-8?B?YlYrNCthZS9TckhaeFVTQXNXbEozSG02eE9BVW9pcUlpL1N2YW5ybGZMNWUy?=
 =?utf-8?B?dXNqdDQ0NVlKQVNIeXFMTnJQQUJ4U1pUU2Q0TUZSV2prOVBHV2REN0dWUHp6?=
 =?utf-8?B?ZjFkQXNEck03c2ZVR2Z0N1JZWlpwaTczZ29nQVcyenhuS3dNQ1pLdzlEVkp3?=
 =?utf-8?B?QUc4THFGVnFpdHQzVTdCVVpDVGZjTzNXZGp2SmVXUE1mR2llRWRVY2hCb1RE?=
 =?utf-8?B?MG9LYjBDaEhTU1B3bmZtUXhYQitJaDhpN1ozTytQc0k4U0lFc0lXeFVzWDJP?=
 =?utf-8?B?bmkzNE9haytvdDRqSXBTVXBWUnNyL25HaW5QSGtndE9mOE1BRjlBZW5wMTNx?=
 =?utf-8?B?TEZUbEdHSGtsRjZJbVRSbUxXQnd1a2tBRUYzdG5pbno5QUJDRDdMZDQ2UWdQ?=
 =?utf-8?B?MFgzY1Z4NVlkT1JQL1VYNE1rUWRrb0JDVmdQUlpBSi9XUGNQSWkyM2EyWFdE?=
 =?utf-8?B?dURxU1ZscER4emhZdHN4emxJVzUvUTFhMU9rV0J0SjBaYWZEVUZaZGNVTWZT?=
 =?utf-8?B?U25BMDVya0VnNVNvMmJhY3R2ZWpHYW1IMTFSMU1oQzV2YnBVZzRUMGpYaFY5?=
 =?utf-8?B?Y2MvRTNTOHpvSXEzdzRuSHNkcHZKNms3NG5KeGl0ejVZR0VGM1BvT0pHTUFC?=
 =?utf-8?B?OG9BUU9ES3RUZlk5NjdFVDgwcTFqOWFMZkFQN1p2elFPWE1kTXBDYUxoUzQ2?=
 =?utf-8?B?NmY0VXZkY28ra3UvczJucUt3UHR4S0lyNi9lOWhMZjdPMHFCMy8xTkJXUE1Z?=
 =?utf-8?Q?zJ6l1kR8j0S25eWQsKOQIXW3A1loQLbvH3w6sn3?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f61a4b8f-1c68-403b-4bf1-08d972cf6f3e
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 13:49:10.1305
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fu6SE5zKFcb7aj/SOzeL0cVvvhaRvc/3Uxo9isp/WfdBP80QPijx8ez4UjdOexCFvy8XrOnAfMX1EGlH8Olx2UY5QtwzSDWUEzyPAOHub1E=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4551
X-OriginatorOrg: citrix.com

On 08/09/2021 14:24, Jan Beulich wrote:
> Using Jcc to branch around a JMP is necessary only in pre-386 code,
> where Jcc is limited to disp8. Use the opposite Jcc directly in two
> places. Since it's adjacent, also convert an ORB to TESTB.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> This is an easy change reducing the overall trampoline size a little.
> We're pretty close to needing a 4th page, which I'd prefer to avoid for
> as long as we can.

Honestly, it is high time we ditch all of this.

The interactive video mode menu only appears to have confused users
who've encountered it, and it is a weird thing to have in the first place.

Furthermore, the Multiboot1/2 specs have supported passing
video/edid/etc information for longer than Xen has been around.

I am not aware of anything we currently obtain via BIOS INT calls which
we can't rely on the bootloader for.=C2=A0 Furthermore, if the bootloader
can't provide it, we've got 0 chance of our 16bit assembly being able to
do something useful...

Dropping all of that should remove the bsp-only path from the
trampoline, taking it down to just a single page and removing a lot of
fragile code.=C2=A0 It gets us closer to being able to support Coreboot
(where the current logic says "not EFI?=C2=A0 I can make BIOS INT calls
then", and is wrong).

I know its a larger piece of work, but the wins are substantially larger.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:00:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:00:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182067.329492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNy7U-0008PM-OV; Wed, 08 Sep 2021 14:00:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182067.329492; Wed, 08 Sep 2021 14:00:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNy7U-0008PI-Ku; Wed, 08 Sep 2021 14:00:08 +0000
Received: by outflank-mailman (input) for mailman id 182067;
 Wed, 08 Sep 2021 14:00:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNy7S-0008PC-Kd
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:00:06 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0613db5b-a467-40f5-863c-e88f1e595989;
 Wed, 08 Sep 2021 14:00:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0613db5b-a467-40f5-863c-e88f1e595989
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631109605;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=+YYk7/na8ZKjyFRiJk5MwAM2qU7Mj4JY+X17r6lSsPo=;
  b=HCq4xdLVMIoWP+pUo/6NWuFFVW6CTz7Y93rlT0Kg3Zvkw7ZmsKifVb2G
   Npi1VN6nn2Z3MdVRjJgskcDHIgBpF/7bcBaZEVXlpIYb0pgYfM/0Hmgcq
   kzBRukxPYSOD/uVz0HtLEY84akay+qmGRNMcHHperawOevAstt4/7BXvB
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: PljX3lcNJM/9o00BU7fNr85cO5dz7EjkwuLsH8Qt3mGGGm7WU6BNyWI+6EGqAKLZJY/zw06sLO
 CpJoELOi2qd0OqJiZ2YG0bmjHptKxxemfI+hf+k508+AVteRVP5NzFocCp5lbToJ4JZaSHTTAk
 IcE3vaP9dZqCKQ3dqxUSBSsX6p4GI3bK+wYRvUiUsfLMq608WGL/6X74jqAjxTC5sJulPp6KSC
 P0jH+U4sAd9IoMyUVbm6QOuq0S0dOG7l7PwxZiSXY7RoMDbRoau54bi3PwNSBQH13NzV4FA1yL
 rrLe5WOk9ehhTibK9pgDz7Jc
X-SBRS: 5.1
X-MesageID: 52271030
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:k72ItajlNszCTjTcBoe8ZjHSv3BQXtYji2hC6mlwRA09TySZ//
 rBoB19726StN9xYgBFpTnuAsm9qB/nmaKdgrNhWItKPjOW21dARbsKheCJrgEIcxeOkNK1vp
 0AT0ERMrLN5CBB/KTH3DU=
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="52271030"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Pu Wen
	<puwen@hygon.cn>
Subject: [PATCH] x86/amd: Introduce and use X86_BUG_NULL_SEG
Date: Wed, 8 Sep 2021 14:59:31 +0100
Message-ID: <20210908135931.26275-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

AMD/Hygon processors before the Zen2 microarchitecture don't clear the base or
limit fields when loading a NULL segment.

Express the logic in terms of cpu_bug_null_seg, and adjust the workaround in
do_set_segment_base().

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wl@xen.org>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Pu Wen <puwen@hygon.cn>

Since originally posted, most logic dependent on this workaround has been
reworked, but one still remains.  This form of the workaround is more flexible
if other impacted CPUs are discovered.
---
 xen/arch/x86/cpu/amd.c            | 7 +++++++
 xen/arch/x86/cpu/hygon.c          | 7 +++++++
 xen/arch/x86/pv/misc-hypercalls.c | 3 +--
 xen/include/asm-x86/cpufeature.h  | 1 +
 xen/include/asm-x86/cpufeatures.h | 1 +
 5 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index c4d84373a710..379ea1638ca6 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -801,6 +801,13 @@ static void init_amd(struct cpuinfo_x86 *c)
 	    c->x86 == 0x17)
 		detect_zen2_null_seg_behaviour();
 
+	/*
+	 * AMD CPUs before Zen2 don't clear segment bases/limits when loading
+	 * a NULL selector.
+	 */
+        if (c == &boot_cpu_data && !cpu_has_nscb)
+		setup_force_cpu_cap(X86_BUG_NULL_SEG);
+
 	/* MFENCE stops RDTSC speculation */
 	if (!cpu_has_lfence_dispatch)
 		__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c
index 429d6601fc13..a2963036b389 100644
--- a/xen/arch/x86/cpu/hygon.c
+++ b/xen/arch/x86/cpu/hygon.c
@@ -40,6 +40,13 @@ static void init_hygon(struct cpuinfo_x86 *c)
 	    c->x86 == 0x18)
 		detect_zen2_null_seg_behaviour();
 
+	/*
+	 * Hygon CPUs before Zen2 don't clear segment bases/limits when
+	 * loading a NULL selector.
+	 */
+        if (c == &boot_cpu_data && !cpu_has_nscb)
+		setup_force_cpu_cap(X86_BUG_NULL_SEG);
+
 	/* MFENCE stops RDTSC speculation */
 	if (!cpu_has_lfence_dispatch)
 		__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/pv/misc-hypercalls.c b/xen/arch/x86/pv/misc-hypercalls.c
index 3a4e4aa4603e..5dade2472687 100644
--- a/xen/arch/x86/pv/misc-hypercalls.c
+++ b/xen/arch/x86/pv/misc-hypercalls.c
@@ -227,8 +227,7 @@ long do_set_segment_base(unsigned int which, unsigned long base)
         if ( sel > 3 )
             /* Fix up RPL for non-NUL selectors. */
             sel |= 3;
-        else if ( boot_cpu_data.x86_vendor &
-                  (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+        else if ( cpu_bug_null_seg )
             /* Work around NUL segment behaviour on AMD hardware. */
             asm volatile ( "mov %[sel], %%gs"
                            :: [sel] "r" (FLAT_USER_DS32) );
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 94a485f99c22..802d9257b0bf 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -159,6 +159,7 @@
 
 /* Bugs. */
 #define cpu_bug_fpu_ptrs        boot_cpu_has(X86_BUG_FPU_PTRS)
+#define cpu_bug_null_seg        boot_cpu_has(X86_BUG_NULL_SEG)
 
 enum _cache_type {
     CACHE_TYPE_NULL = 0,
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index 6c8f432aee4f..72beb7babcce 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -45,6 +45,7 @@ XEN_CPUFEATURE(XEN_SHSTK,         X86_SYNTH(26)) /* Xen uses CET Shadow Stacks *
 #define X86_BUG(x) ((FSCAPINTS + X86_NR_SYNTH) * 32 + (x))
 
 #define X86_BUG_FPU_PTRS          X86_BUG( 0) /* (F)X{SAVE,RSTOR} doesn't save/restore FOP/FIP/FDP. */
+#define X86_BUG_NULL_SEG          X86_BUG( 1) /* NULL-ing a selector preserves the base and limit. */
 
 /* Total number of capability words, inc synth and bug words. */
 #define NCAPINTS (FSCAPINTS + X86_NR_SYNTH + X86_NR_BUG) /* N 32-bit words worth of info */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:06:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:06:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182094.329522 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyDm-0001s0-QK; Wed, 08 Sep 2021 14:06:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182094.329522; Wed, 08 Sep 2021 14:06:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyDm-0001rt-N3; Wed, 08 Sep 2021 14:06:38 +0000
Received: by outflank-mailman (input) for mailman id 182094;
 Wed, 08 Sep 2021 14:06:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNyDl-0001re-A2
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:06:37 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f9b77c8a-10ad-11ec-b15c-12813bfff9fa;
 Wed, 08 Sep 2021 14:06:35 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6717220011;
 Wed,  8 Sep 2021 14:06:34 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 3BBAE13A94;
 Wed,  8 Sep 2021 14:06:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id G0ATDWrDOGFyLgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 14:06:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9b77c8a-10ad-11ec-b15c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631109994; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=1XIi7BIee86vqFQ4dDJAQB/5cN+dbnwh/K7yGPhc9zI=;
	b=HCiQWm5qM6saWpbD71HNYeHk9J9iarVaFQJOH1anDB2pOMgrH3HqF/fs4+gxbZm5unFPau
	TzRmKiZwm5RcZH/pebQU72xPh1ITA+KItsL1BGwxr1yOP+sVbPz1Oeq9JDPip2RjULthCF
	khKIayUkxPhcBRtTbgW8iEI7BbY07+Q=
From: Juergen Gross <jgross@suse.com>
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>
References: <osstest-152672-mainreport@xen.org>
 <93d94ca3-1a87-a11a-daef-11ec183d9a2f@suse.com>
Subject: Re: [linux-linus test] 152672: regressions - FAIL
Message-ID: <baff0583-b616-18f5-1e18-080d41a4931d@suse.com>
Date: Wed, 8 Sep 2021 16:06:33 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <93d94ca3-1a87-a11a-daef-11ec183d9a2f@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="md2H7nhgLOdtbEeggTBqW5yPh5e8d3DnQ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--md2H7nhgLOdtbEeggTBqW5yPh5e8d3DnQ
Content-Type: multipart/mixed; boundary="n8GQsBiNdeWF7GOhCyXqDnnu3jos1BsEn";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>
Message-ID: <baff0583-b616-18f5-1e18-080d41a4931d@suse.com>
Subject: Re: [linux-linus test] 152672: regressions - FAIL
References: <osstest-152672-mainreport@xen.org>
 <93d94ca3-1a87-a11a-daef-11ec183d9a2f@suse.com>
In-Reply-To: <93d94ca3-1a87-a11a-daef-11ec183d9a2f@suse.com>

--n8GQsBiNdeWF7GOhCyXqDnnu3jos1BsEn
Content-Type: multipart/mixed;
 boundary="------------0D479D592C96ABD0B535A90E"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------0D479D592C96ABD0B535A90E
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.08.20 07:52, J=C3=BCrgen Gro=C3=9F wrote:
> On 23.08.20 07:24, osstest service owner wrote:
>> flight 152672 linux-linus real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/152672/
>>
>> Regressions :-(
>=20
> With 32-bit pv support now removed from the kernel the associated tests=

> should be removed for the upstream kernel, too.

Those tests are failing for more than a year now, due to non-existing PV
support in upstream 32-bit kernels.

Can we please disable the tests needing a 32-bit PV Linux system with a
kernel 5.9 or younger?

I have brought this up several times now (via mail and on IRC), and
nothing happened. This huge number of failures renders the linux-linus
tests completely worthless while trying to run the tests is consuming
test capacity.

Yes, I know that currently the testing of 32-bit tools is relying on
those tests, but without support in the kernel they can't work at all.
So I suggest to switch those tests either to a pre-5.9 kernel, or to use
BSD for that purpose (PVH can't work, because x86 dom0 needs x2apic for
PVH, and that support 64-bit only in the kernel).


Juergen

>=20
>=20
> Juergen
>=20
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>> =C2=A0 test-amd64-i386-qemuu-rhel6hvm-intel=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-qemut-rhel6hvm-intel=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 7 xen-boot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemut-debianhvm-amd64=C2=A0 7 xen-boot=C2=A0=
=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot=
 fail=20
>> REGR. vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-ws16-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail=
=20
>> REGR. vs. 152332
>> =C2=A0 test-amd64-i386-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 10 xen-boot/src_host=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail R=
EGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 11 xen-boot/dst_host=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail R=
EGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm=C2=A0 7 xen-boot=C2=
=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-examine=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 8 r=
eboot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7 x=
en-boot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemut-ws16-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fa=
il=20
>> REGR. vs. 152332
>> =C2=A0 test-amd64-i386-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-qemut-rhel6hvm-amd=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-coresched-i386-xl=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-amd64=C2=A0 7 xen-boot=C2=A0=
=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-libvirt-xsm=C2=A0=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-qemuu-rhel6hvm-amd=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-freebsd10-i386=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail R=
EGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-raw=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 7 xen-boot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-freebsd10-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR=
=2E=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemut-debianhvm-i386-xsm=C2=A0 7 xen-boot=C2=
=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-win7-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-pvshim=C2=A0=C2=A0=C2=A0=C2=A0 7 xen-boot=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-shadow=C2=A0=C2=A0=C2=A0=C2=A0 7 xen-boot=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemut-win7-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemuu-ovmf-amd64=C2=A0 7 xen-boot=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-libvirt-pair 10 xen-boot/src_host=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-libvirt-pair 11 xen-boot/dst_host=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot=
 fail=20
>> REGR. vs. 152332
>> =C2=A0 test-armhf-armhf-libvirt-raw=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-armhf-armhf-xl-credit2=C2=A0=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-armhf-armhf-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-armhf-armhf-xl-arndale=C2=A0=C2=A0 7 xen-boot=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail REGR.=20
>> vs. 152332
>> =C2=A0 test-armhf-armhf-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7 xen-bo=
ot=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 fail REGR.=20
>> vs. 152332
>>
>> Tests which did not succeed, but are not blocking:
>> =C2=A0 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail=20
>> like 152332
>> =C2=A0 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail=20
>> like 152332
>> =C2=A0 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail=20
>> like 152332
>> =C2=A0 test-armhf-armhf-xl-rtds=C2=A0=C2=A0=C2=A0=C2=A0 16 guest-start=
/debian.repeat=C2=A0=C2=A0=C2=A0 fail =20
>> like 152332
>> =C2=A0 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail=20
>> like 152332
>> =C2=A0 test-arm64-arm64-xl-seattle=C2=A0 13 migrate-support-check=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-seattle=C2=A0 14 saverestore-support-check=C2=
=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-amd64-amd64-libvirt-xsm 13 migrate-support-check=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-amd64-amd64-libvirt=C2=A0=C2=A0=C2=A0=C2=A0 13 migrate-sup=
port-check=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-thunderx 13 migrate-support-check=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-thunderx 14 saverestore-support-check=C2=A0=
=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 13 migrate-support-check=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-credit2=C2=A0 13 migrate-support-check=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 14 saverestore-support-check=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-credit2=C2=A0 14 saverestore-support-check=C2=
=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 13 migrat=
e-support-check=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 14 savere=
store-support-check=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-credit1=C2=A0 13 migrate-support-check=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-xl-credit1=C2=A0 14 saverestore-support-check=C2=
=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11=20
>> migrate-support-check fail never pass
>> =C2=A0 test-arm64-arm64-libvirt-xsm 13 migrate-support-check=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-arm64-arm64-libvirt-xsm 14 saverestore-support-check=C2=A0=
=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-amd64-amd64-libvirt-vhd 12 migrate-support-check=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-credit1=C2=A0 13 migrate-support-check=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-credit1=C2=A0 14 saverestore-support-check=C2=
=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-rtds=C2=A0=C2=A0=C2=A0=C2=A0 13 migrate-sup=
port-check=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-rtds=C2=A0=C2=A0=C2=A0=C2=A0 14 saverestore=
-support-check=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-multivcpu 13 migrate-support-check=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-multivcpu 14 saverestore-support-check=C2=A0=
=C2=A0=C2=A0 fail =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-cubietruck 13 migrate-support-check=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail=20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-cubietruck 14 saverestore-support-check=C2=A0=
=C2=A0=C2=A0 fail=20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-vhd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 12 migrat=
e-support-check=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-armhf-armhf-xl-vhd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 13 savere=
store-support-check=C2=A0=C2=A0=C2=A0 fail  =20
>> never pass
>> =C2=A0 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2=C2=
=A0 fail=20
>> never pass
>>
>> version targeted for testing:
>> =C2=A0 linux=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 f873db9acd3c92d4741bc3676c9eb511b2f9a6f6
>> baseline version:
>> =C2=A0 linux=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 deacdb3e3979979016fcd0ffd518c320a62ad166
>>
>> Last test of basis=C2=A0=C2=A0 152332=C2=A0 2020-07-31 19:41:23 Z=C2=A0=
=C2=A0 22 days
>> Failing since=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 152366=C2=A0 2=
020-08-01 20:49:34 Z=C2=A0=C2=A0 21 days=C2=A0=C2=A0 21=20
>> attempts
>> Testing same since=C2=A0=C2=A0 152672=C2=A0 2020-08-22 12:17:18 Z=C2=A0=
=C2=A0=C2=A0 0 days=C2=A0=C2=A0=C2=A0 1=20
>> attempts
>>
>> ------------------------------------------------------------
>> 2189 people touched revisions under test,
>> not listing them all
>>
>> jobs:
>> =C2=A0 build-amd64-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
pass
>> =C2=A0 build-arm64-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
pass
>> =C2=A0 build-i386-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 build-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-arm64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-armhf=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-i386=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-amd64-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-arm64-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-armhf-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-i386-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-amd64-pvops=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-arm64-pvops=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-armhf-pvops=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 build-i386-pvops=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-coresched-amd64-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-arm64-arm64-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-coresched-i386-xl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail
>> =C2=A0 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-i386-xl-qemut-debianhvm-i386-xsm=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail
>> =C2=A0 test-amd64-amd64-libvirt-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pass
>> =C2=A0 test-arm64-arm64-libvirt-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pass
>> =C2=A0 test-amd64-i386-libvirt-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-arm64-arm64-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-xsm=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-qemuu-nested-amd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-pvhv2-amd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-i386-qemut-rhel6hvm-amd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-qemuu-rhel6hvm-amd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-dom0pvh-xl-amd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-qemut-debianhvm-amd64=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemut-debianhvm-amd64=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemuu-debianhvm-amd64=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-amd64=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-freebsd10-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-qemuu-freebsd11-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-qemuu-freebsd12-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-qemuu-ovmf-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemuu-ovmf-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemut-win7-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl-qemut-win7-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemuu-win7-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl-qemuu-win7-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemut-ws16-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl-qemut-ws16-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-qemuu-ws16-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl-qemuu-ws16-amd64=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-armhf-armhf-xl-arndale=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-credit1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-arm64-arm64-xl-credit1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-xl-credit1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-credit2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-arm64-arm64-xl-credit2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-xl-credit2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 fail
>> =C2=A0 test-armhf-armhf-xl-cubietruck=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
pass
>> =C2=A0 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-examine=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-arm64-arm64-examine=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-examine=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-examine=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-freebsd10-i386=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
fail
>> =C2=A0 test-amd64-amd64-qemuu-nested-intel=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-pvhv2-intel=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-qemut-rhel6hvm-intel=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-qemuu-rhel6hvm-intel=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-dom0pvh-xl-intel=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-i386-libvirt=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-multivcpu=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-armhf-armhf-xl-multivcpu=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-amd64-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-libvirt-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-i386-libvirt-pair=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail
>> =C2=A0 test-amd64-amd64-amd64-pvgrub=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 pass
>> =C2=A0 test-amd64-amd64-i386-pvgrub=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-pvshim=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-pvshim=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-pygrub=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-qcow2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-libvirt-raw=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fail
>> =C2=A0 test-amd64-i386-xl-raw=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-rtds=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-armhf-armhf-xl-rtds=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-arm64-arm64-xl-seattle=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-amd64-amd64-xl-shadow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 pass
>> =C2=A0 test-amd64-i386-xl-shadow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 fail
>> =C2=A0 test-arm64-arm64-xl-thunderx=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pass
>> =C2=A0 test-amd64-amd64-libvirt-vhd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pass
>> =C2=A0 test-armhf-armhf-xl-vhd=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pass
>>
>>
>> ------------------------------------------------------------
>> sg-report-flight on osstest.test-lab.xenproject.org
>> logs: /home/logs/logs
>> images: /home/logs/images
>>
>> Logs, config files, etc. are available at
>> =C2=A0=C2=A0=C2=A0=C2=A0 http://logs.test-lab.xenproject.org/osstest/l=
ogs
>>
>> Explanation of these reports, and of osstest in general, is at
>>     =20
>> http://xenbits.xen.org/gitweb/?p=3Dosstest.git;a=3Dblob;f=3DREADME.ema=
il;hb=3Dmaster=20
>>
>>     =20
>> http://xenbits.xen.org/gitweb/?p=3Dosstest.git;a=3Dblob;f=3DREADME;hb=3D=
master
>>
>> Test harness code can be found at
>> =C2=A0=C2=A0=C2=A0=C2=A0 http://xenbits.xen.org/gitweb?p=3Dosstest.git=
;a=3Dsummary
>>
>>
>> Not pushing.
>>
>> (No revision log; it would be 287986 lines long.)
>>
>=20
>=20
>=20


--------------0D479D592C96ABD0B535A90E
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------0D479D592C96ABD0B535A90E--

--n8GQsBiNdeWF7GOhCyXqDnnu3jos1BsEn--

--md2H7nhgLOdtbEeggTBqW5yPh5e8d3DnQ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE4w2kFAwAAAAAACgkQsN6d1ii/Ey/a
twf/cU6IXxDY4MbbW2V+WmGO2yAyKsZalJMLrp4ykNCqey/31J2RNubySTQKl+HVnUjRvoqCHFPm
Moh4sQNi6j425wyZQFbLcrXL5cqo/fXiS1TQrbZ1rLVP8w15tZEk+ZG4fQcertlv/Up7DYOIpsic
idGtQv0FhQj4LYVEBmKyHDPr1qoZtd7QFARZdnUWa3gp98i25UL/Fa7sQILNOCnbv9Bzy0+EEYZe
U+jYXiH6AA/ykQga6POR0RdVkm3vq77kFOs0+IZeCq+PJFejXRDjgjUanL5DZAO39mfQ4AsmFzVd
/+OogfMHc6Cz6CuSu8/AXraT6XJs2wsfxTr2nG+K2Q==
=OnSd
-----END PGP SIGNATURE-----

--md2H7nhgLOdtbEeggTBqW5yPh5e8d3DnQ--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:21:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:21:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182105.329532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNySN-0004lE-4B; Wed, 08 Sep 2021 14:21:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182105.329532; Wed, 08 Sep 2021 14:21:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNySN-0004l7-1H; Wed, 08 Sep 2021 14:21:43 +0000
Received: by outflank-mailman (input) for mailman id 182105;
 Wed, 08 Sep 2021 14:21:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ljEy=N6=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mNySL-0004l0-FE
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:21:41 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 15521872-10b0-11ec-b15e-12813bfff9fa;
 Wed, 08 Sep 2021 14:21:40 +0000 (UTC)
Received: from pps.filterd (m0246629.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188DxvcP012550; 
 Wed, 8 Sep 2021 14:21:29 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3axcs1aud1-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:21:29 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 188EKqcc107138;
 Wed, 8 Sep 2021 14:21:28 GMT
Received: from nam04-bn8-obe.outbound.protection.outlook.com
 (mail-bn8nam08lp2045.outbound.protection.outlook.com [104.47.74.45])
 by userp3030.oracle.com with ESMTP id 3axcq1f8cb-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:21:28 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB4914.namprd10.prod.outlook.com (2603:10b6:208:30d::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 14:21:26 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4478.026; Wed, 8 Sep 2021
 14:21:26 +0000
Received: from [10.74.115.194] (138.3.201.2) by
 SN6PR04CA0099.namprd04.prod.outlook.com (2603:10b6:805:f2::40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 14:21:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15521872-10b0-11ec-b15e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=r5DkWAm23nvRRUg/jAQr5y90goAn0GgNJDof9SZXWYw=;
 b=a8GOx0ZSEoT9cxdc6ymbmx/DDIcFVf/luxDG3Tbj/1Ux0J/IJQFZtL5BR0affL3I3bio
 rhFSWXC8u9jv2O4pJlNjxdVLruPCTX7cU7lRGLQBgWRCy+z2NSA/YMVkD9Tv1eDZvoE5
 nF0YwlToCV3caUVNx2/8CioJUmkwnJ424tDkuN/xo5vGGRrZRxMtDIOknihfgXjpv2sQ
 Ze6uoerKZJPULF1K7SQzFkjGENEJPMYLQ/8I9cxiGxi9PjTI82oPOg6wm3czgT3Vkmw4
 TIj+c9Ufvy7ime/rofwasUoPMN+5GEiy0O1bugzptFJbR4bAmxKrBkcRTYYEb6Xa/4T5 qQ== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=r5DkWAm23nvRRUg/jAQr5y90goAn0GgNJDof9SZXWYw=;
 b=PXJDs84wWcpUFzLVKNydmNoEzffkJPyl8uUB7fYO8NRlE9tIqNVozyqAWJfHnvf1OBOF
 srTGAz0Xf6n2DLvSk/+Hqiu3mz3Cx5fLCRs68cNRAuUaF0udOZ2mQrOBF2aPTNh20rG0
 OeOKNfdOQuS6GGRhEjobQX2ywMBLsNlh7UupljH2otMoX+R5hoDL5EuFflmSj5dac/XY
 aSZhiwf5/q+itaFbiZqVpurxNduewMd+2w1aU0IgwDGT2lkc1/rNssCa/1DAIdd/PGBJ
 8a2J5WzFYKyMRXLKyLFvEqI3mzjaY0put65GKuqr/tj/soh5tqSdR2O6rCgxAxujHZy2 cw== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hBj2ZtJP2tO/f1n9xwbmC/R2uqwvKW1U/xXLR8Es850XznSCfLrTOnys1N7ab2GtMJ87XVAdzOkDTy5svmkho7B7oI57jMfF0eKuEWqcCsRVkTsJ4u5x+7s41iKDdZUYGeZD3IHp2VrV8GBZhzjTwoEMn9MEk5lHYpzdR2pOUsEZmVI9OyesaIJihTUz+cEMVusJE4Uf0QBbWS0r5ir4WPWcBTkPB79mlnPtCxvGEZvnw8nXUqwZwDmt6pnMN7092Aggzd5g0mWaUP6ItyMuo/p02ACHRVmCXUaoOXC9O3HsXKeiYPmucgZ8SpYDnM3FqYI1t2n2H3tPiqdL1LepDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=r5DkWAm23nvRRUg/jAQr5y90goAn0GgNJDof9SZXWYw=;
 b=GBemeKD5g5a1txp/YX8m23XfbyV6UJkk9XrusQBLsZ0UUZAKSIHOoBTXn/t2c70tMRxIOq/V+pesCg+rdKsFyQOLUaVR5jdpt4sDLbVkg6p/k060qO/CNGeM+hO6cb4vdx67OcEWsrkB13qW4aVR0zsPVeyGJkjt/X7HB38KZzXoVrJ4bRJ/l0f5qsyZnPQt78nUwX8P8hWZ582We9LiW60WYrHhFUecyIQBhhWFXLaqDz6pzNRssIfu7PEMBV+ddZTLYdQBEFPzgsknSzEfi+5BreS6AlhMU+kDJr2fDKGX7n5m9KWUsztO3VCff/yeCm47uOTGeNakz38VxPIjSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r5DkWAm23nvRRUg/jAQr5y90goAn0GgNJDof9SZXWYw=;
 b=MM95ZO32D5e3kREJDDAOYpJEsgURZKa4avPypcMLWFybsLLCRCIkdm1SZekV0LZJtMYJa8vWTV01i0NFIvrNfTP1DmYCanU+G5YNjdM73M0farE2QBtMuIF6pJLpsvgt0e6BNxP2CwW1jsSLF7zMyn00LxF5xC0IIrZQz/zHwcQ=
Authentication-Results: zytor.com; dkim=none (message not signed)
 header.d=none;zytor.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH] xen: remove stray preempt_disable() from PV AP startup
 code
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210825113158.11716-1-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <df9f93ae-a833-8bf0-9df6-91cf2c4e007b@oracle.com>
Date: Wed, 8 Sep 2021 10:21:20 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210825113158.11716-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN6PR04CA0099.namprd04.prod.outlook.com
 (2603:10b6:805:f2::40) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f0209f4d-e276-4bbd-ca16-08d972d3f12a
X-MS-TrafficTypeDiagnostic: BLAPR10MB4914:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB49144AA1A328EF771A9F9B298AD49@BLAPR10MB4914.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:451;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	fbgEZ/aZUMVO2QfBb4NXTZe3nkm2Qeq0h+e/PY0dUHkPuEGokMlNm1xOlGQCxxdJArSJrf+gkGxq2Zkpug6uwRIHAskhBgiWE/VKZDMe/ghDq2hidEGeumwJnzrvBtd6jIWNClBUa1apJNfpK5bIDRCJBPlhmDSr2ej3lk9s8SsZsKhGLUX+nUNyMJ2BZSX9xQWdJ7K6QMGnh9hZa1TZ5Wo5rTBSS+Of4N4woXI7UxG2loUUxDBdRtuWDNEE1gkFGvGLnA5TGaEzVTJ5P0TKOLnKrNRjYwULdfWv2vOV7AJZbVmn4lx9adUXK40GwcMt7LgjRzwjnSUJSkhNnP9wxRO5YeYt1/F1sPCX3ZERCiniEehQCCRcSfR3LwoWl3N9KmHcUuU9RhE/1Zrqt0SIkGNL4QPec5QxIWDhumQhNgs1cvf2MNeD8mYDLWtTl2TChveaoxxraxwgqKXBIFrOWoYFNhRcQ8DmVNBYkaGIXA5Ska/DUy17vNqyIrLX/8YH8YH1YFAGIUfjrTlh/0iXkeSrxQGdOUf2VBKD45aOb91DOkDa750tY9E4214VR8CS2D4UyKi+tLxiUmPirMfnhHQ28gDdvqY58xryYmpqRumHifzPV6OTL4h4fUSvKvZT9ghsmQq4eZ5lK11S8tQ29J0y1Ljl2unWZsNE3Hz6+X38HJqM/7daDm2XRGVv2UIU81oYwkKcIazsxianKMTkjcgZJHxIQUWaur96aWQcIC0=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(136003)(396003)(366004)(39860400002)(4326008)(26005)(38100700002)(31696002)(4744005)(8676002)(8936002)(478600001)(2906002)(53546011)(36756003)(44832011)(54906003)(66556008)(86362001)(83380400001)(66946007)(66476007)(186003)(6486002)(956004)(5660300002)(2616005)(16576012)(316002)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UTJOYThrQnhiOVZmN0tHR256d3Y5V2JJN0R3dndDbW55UTI1OHRsK1AyVWFq?=
 =?utf-8?B?OVBOMno4QzRKcGhDNWkvUjM0UHNYQXVaUHA3NVRqcmZodzVkWjJUb1o1Y1gy?=
 =?utf-8?B?eTFydnlDdnd2cUxDczE5bnJJVExDbnA4aUhIZTFQOUVkM0FmZmE2Q3huZndi?=
 =?utf-8?B?c3VvZEs0Z0hxVGZIMmExM203bm1tQ0NDLzR1WFhFVlJ2Q01XbFhBVWRKYy83?=
 =?utf-8?B?OWluY09RS213Q0dhMVpES3NHOU5yUlB6TjVReUNveFdVS3YxU0hkNm1ueHpP?=
 =?utf-8?B?c29IM1NOSzBnVURnenhYL2pDL0MzMVFyektyN1BRVXljdnQ0WHZCbDNyaEtP?=
 =?utf-8?B?b3ZheE1XQWFUNk5WcndURmJ4a1BQdDRQbk53RVBZa1VmUXRWd05Ha1dxM2tT?=
 =?utf-8?B?QVNJTlFnaEdrZ3FvdGRHSEtaaFF2djUxcGgyTWQwWWNScjNiZStSd0RWbUx4?=
 =?utf-8?B?OERTblJSQUduOWNOc2NPZ3Vxb0JSZnUrT3h5OFFSUmNWNnZHTU5pYTgwNXFs?=
 =?utf-8?B?RzhhSUxlaGdBamNhck1GVlBOa3lsRTlHcVlwRjBtYlc4Qm9HNko4T3YweWdF?=
 =?utf-8?B?eFhJVkV6TUJhK0o3SFlFMjVxeWdqK2JLZC9Sd0xoWDBIMlcyTkd0ak5NVUdz?=
 =?utf-8?B?L1pINC9xUi9FWGd4WUo5V2paSHBYUUNuZGtwL2lkWm5LVzl0dDNYSGpGOG9F?=
 =?utf-8?B?ekhCYjR0bFBiQUczakhpTGdHZG1SbUduejlsNUd4RVdpd3IxKzk0TnVXQ29Y?=
 =?utf-8?B?aTgrTURZbWZJdkZWa1BaOGo0Ly9NWW9ad2F1ZnZpc3lpYjZnQk1SRDl3dG9Q?=
 =?utf-8?B?bXNBRkFNMUx3RDN0Uy9LTVh0bEJjRDBKU25BZTRLTzQvRG80YjlCNVVORUkx?=
 =?utf-8?B?Z1RVNUQvQTlGMzNBWi96Vm5QSlEyeDNMdHhhelVnNStOSDVkeGlJckJRLzJh?=
 =?utf-8?B?KysxeWtqOU54Y0Ewb0dIaXhQK2h6em1paTQ3STZLY1dkNGhpWjRXM2VoZktM?=
 =?utf-8?B?T0grTzAxTWxsSmZ1Z3ZlVGU0azFPbmlmODZkRWJIT2diSnl6K0ZaVlpVclZL?=
 =?utf-8?B?SVBJSVpoRldMSzVsVEl4UTFJa1BEQWtaVTRZNXF5d3lLZEp6Y1NqeHZJcmha?=
 =?utf-8?B?eHVTL3JUQk92dll6NGIxZTJrcUY2dGhiWks1ZndHQVBHVHpFV1JZaGRBYWor?=
 =?utf-8?B?TmluUmdwWE8wMEs2STY0anZuenBicmVNWDVBdFB3d1RqUzYwb0ZGR2dKNDlB?=
 =?utf-8?B?dmhqVU4rQmtja0tYNmN1OWxJeVJsZ1BTMTJVSHpkSG0wL0RBS2ZBZUlUNk1n?=
 =?utf-8?B?dlE2Y3BFdlZnWVhFN1BISGdXSjNFQmZ5L05pZWptaHhzMVhITXk5YzZSQmQ2?=
 =?utf-8?B?NTBoRnJmMVZiK0N4MVJJRm0xb3pQektTM0dXQ2ZDcElRR3haSXBpbWw1VWxo?=
 =?utf-8?B?a3IxYzRrc1lXY3V5OWgwazBERDQ1MUdzVDBaNFg3UjFoVlY2T0k5cnB6VERp?=
 =?utf-8?B?S3gveW16WHluRk1OMDBjS2R6TWdQNjBjdUJQa0drMFZoVllZSVdzMFRvNklt?=
 =?utf-8?B?UktXenZDbWViUjltRWFrRWc5b1h2QUZNL3NrTkNtbDlkZy95L3dKWldwcW9G?=
 =?utf-8?B?M0VCV2VwWEJwTGFDY1UyS0NYK09BcW9qQk0xVS9kajJNdHZkUnZLb2pZTit0?=
 =?utf-8?B?YXNwdXNPNm5wcm15MnZkVVBlM2lVWm54ZFZjd1JYd1ErdXkydU9rOWVvcEFs?=
 =?utf-8?Q?tIepf5V7Py4rDVoZK67FPYBFtUeslFiTP37vqtt?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f0209f4d-e276-4bbd-ca16-08d972d3f12a
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:21:26.1774
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4IgoZUo51MAMg8/bdZOLcLtS3BQSPlRkZCPGHHqMSWJySsTTFrzJPr7MK6rg/YVHj0HiMjwOZayxoqjdilFLB9+pyw9h+IPiyU8UkQbq5q0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB4914
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10100 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0
 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 mlxlogscore=999
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080092
X-Proofpoint-ORIG-GUID: 05yXRHHkwXBDUmyT8k9WE8Fd0hhC5v1L
X-Proofpoint-GUID: 05yXRHHkwXBDUmyT8k9WE8Fd0hhC5v1L


On 8/25/21 7:31 AM, Juergen Gross wrote:
> In cpu_bringup() there is a call of preempt_disable() without a paired
> preempt_enable(). This is not needed as interrupts are off initially.
> Additionally this will result in early boot messages like:
>
> BUG: scheduling while atomic: swapper/1/0/0x00000002
>
> Signed-off-by: Juergen Gross <jgross@suse.com>


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:28:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182111.329543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyYp-0005WH-OP; Wed, 08 Sep 2021 14:28:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182111.329543; Wed, 08 Sep 2021 14:28:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyYp-0005WA-LG; Wed, 08 Sep 2021 14:28:23 +0000
Received: by outflank-mailman (input) for mailman id 182111;
 Wed, 08 Sep 2021 14:28:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNyYo-0005W4-2Q
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:28:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 17d0a313-c03e-4a50-aebb-9c505797c71f;
 Wed, 08 Sep 2021 14:28:21 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-rM9ZSgQDPbG-VEFVfUScTA-1; Wed, 08 Sep 2021 16:28:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 14:28:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 14:28:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0020.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 14:28:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17d0a313-c03e-4a50-aebb-9c505797c71f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631111300;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=j6d9ZMxRtBBb6+QUt6hwEKx14KSM3R+s8pNB7AtZApw=;
	b=i4L+HTWOLsBdeoUMs1vUJHtyCG6yF1IEDR8q5DpcKtCqkzGyEmrO0huf4obSDNaGsBlLh4
	knYufTN7ME3bnozoBVgW4xj3wUhVBxV1AT/p9b82bxaQ7QW3XjdBETkrlGc5FB8t6zVCsR
	PeFK+/QeptokyvCyxu/b8FvHVDOkLqo=
X-MC-Unique: rM9ZSgQDPbG-VEFVfUScTA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=K1f64Qoh67yKL/yyrd/J7B6Wv+f6AuZw1zvesc4pj+aVTmxpIrEgvPfoxOJeCCub6iUPXa5ha6VE7fAz3Qirgc0qs77HDEMnAsBdKZhzhZyyLFGLQh9U+4eCjUIROcWeZ2+wIFMBdMgm7vKW5levxk1cPcGCV7YeL3AQ8WUeOskXvGl10WqJTrnVVNVepsR5PgVlT9jkvhEixagRDKEddTX0gJBOSOYbS8jvUIxsU01GST+YAhZqLHlIyacfqIxBepw7276eMmRn+YK1R981tAiSrPUKZ1E9BmK3yLylWGFI3LSnF5/gUhHNrKcl2BYDAJphVBZpfH4iAtV7ZxGhOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=j6d9ZMxRtBBb6+QUt6hwEKx14KSM3R+s8pNB7AtZApw=;
 b=npA9YXS37OZ9eCnFZevG1Oe9sCiM53wc7KNME7aoBBxslib9tjgw8wUF1cUlCTJ3ku53+0dil6g3nvXPModqWf7WJE9QztQ+vFYHaFMEA9egf3nSCIrbm4EtvfrR4ZeiBJ++1TNerw7OosPCyFKcwYmsZVaoqGjsKAnlXjAVnoxM6mARgh5vpYa7t6K4wfBzJXnYkM+onOmACC31jERPaJn5z1vlnc8NmOFF51y9shoIhTZkQBFYZf/cx6Bf9GY5FervdZl/K1fnmXQ7jmTzSW6N/eXTDK99tz5h+f969GmERKBS4bHq6pwHXPUZgsjziL2eX0gcTzXUQ0WckOhsuw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
To: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210908073640.11299-1-jgross@suse.com>
 <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
 <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6862566d-4d55-dc5c-082a-da4fbcafcfce@suse.com>
Date: Wed, 8 Sep 2021 16:28:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P195CA0020.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 258b01ef-e0ae-46eb-7640-08d972d4e62e
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2959EDA1DDB7FA1ECD996FE3B3D49@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pKOgsU6+d/BzhFdtn8fh0taiwakbEhUB4cGrV4YJGeZx6uJheMl0QFP1qCVz/Y9rVXjg0c4ZWho3SFYoXhLCk6IuUAODrS1AHgiM3hXoWF9j8wEbVg/CNEqffJkhmMFh3h3m4qPsWBJUnBafY/fMgOoPyAgjP90JK3Dxr1bmGlUTkD/eh14I8aKxAdppUnAHUM3sRGXneDCqHu4JMvEBBf/tqdJC10TVqD4t+dJzrh5+TAY71YX5+hWD0IKJha/5mLg6C9WuzOLMi4wOcCcthufurIDHHzAFZr5J5TY0TNTqLsuiBjme28ycGGsnc6ZHJWM2qDBNxjPnTSdkM1z600TX2nobFm+wctOdmQH4IQpqJN8y/fjfn9fANUfW6u9RhJcdyBYmGxdSjl3kbCQtRxh3otu1hpOBwlaPW+8qxG1VfnziZMx54ujwFwDUQq9L9vp7h7S8bDdASgsTSxcFx6/3HWFzIV/+x9OcppjZCdBYFpMkHOwTr7B2z9NqQxSZMOWDz/ufAmk9tMWHijcCNWueP2WHzCbd9UZYl6Ukouzje16hHH0x7QDoJoiIL2JlXDFptUdOE+tuSsgyWNMg8K0Y8QVs9KYW2J1z63A48b0Rhy0+944PbAaSuzvl42amq8x0g34xIVRPLg2RdEQjAzqBwCgs9p5U39rw7p2DQf1sK3f4mWQaZlRj7byxYZRrAmtaY6ltOpFzFawL/L85GXyTVZh+4osq82na8lyc+60=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(53546011)(26005)(186003)(16576012)(2906002)(37006003)(508600001)(66556008)(66946007)(31696002)(316002)(4326008)(54906003)(86362001)(2616005)(956004)(8936002)(8676002)(5660300002)(6636002)(31686004)(38100700002)(4744005)(6486002)(6862004)(66476007)(83380400001)(7416002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eHBYWWRFK3dNeERISXZVOTlPVENqSlA5ZmRXTmV5NEkzN2hIRHFsMkZISG8y?=
 =?utf-8?B?KzRJY1Q5Yk5sRExraldkNW9Db1NPalMyV0p3VndwL2NEZlBLQW5acmtISGNy?=
 =?utf-8?B?THY5STdVWEIzeGRPMm5NMmZaSlpyTVNQdmUzZHJPKzBRR3E3QU5ycGs5VUxz?=
 =?utf-8?B?ZjgxUFNYZjFpK2sxRDQzVDdpblExa25ubCtYUis3K1ZvaTM0RTIxc0s4N2xC?=
 =?utf-8?B?enczaW4vVU1BN3VkTENWRURLQjUwaDNSdzFYS2dIZ29KMDhseG92aVk1UUtt?=
 =?utf-8?B?N2pVd0RiaVM1K2JqZ3JSTzZEeXJibVpHanlpRnpLbUo5SWVjM2lPZzFLa1hm?=
 =?utf-8?B?eUQ3ZWFoUUdYdWNBdFV1NGM1L01hVW1Dd0w3SnVnTytPdjFOT2V0Yy9wMEZa?=
 =?utf-8?B?VERiV2ptMmd3WWhWTmVIT0RUUGRrdXJOZEZ0U2h1aUIxYlE1Z3JBaElJWHN5?=
 =?utf-8?B?em5EK3pzL3JCTFQvN3hEVFRiWG5QR0JXckY2K3kreUpMVjBDM1ZleVZMRzFQ?=
 =?utf-8?B?eUdIeHcrQVROYnUrNURCK3I1MTR5NGZmYVZqRmZqeWYrNlVBYUxaekc2cng5?=
 =?utf-8?B?dFdmSTdRZktrQ1YxbWx6bFc4OVJ5ZkFoSDFzT0ZpUkVvU2o3SDd5ekZpT3Qx?=
 =?utf-8?B?aXVEeVd5Slc1VWpqZVlPVGpKeSt6TzVCQklLSjUxd25TOFh0VlJNcWYvOU9q?=
 =?utf-8?B?QXpOKzdCQ2RqSVJ4Z1VoR2hkNEpQdzZ4RDV4NVZrcWh4U1ZwWWRoNVNsZzYy?=
 =?utf-8?B?eG1XUmF3cUtaR3VNSDNZZ1BrTW13bTdwMDM0WFRGN05INkVNN2xmWS9uRkl5?=
 =?utf-8?B?Tm5NM25iNE9acUNCcHU1ZFl0Yzh1SVBxTUlhcHdFY09HRVRwSmYvMkxmaVRs?=
 =?utf-8?B?WnlwM1JGTGFzSkxDbmJSdnBtc01GcFRnaDVxV3lxc0RpZGEzV2JtLzBqWlU0?=
 =?utf-8?B?ODg4RmMwOW1jWDlQZ3ZwV0dIUk42VUdidWtEQlFvMVhBR3lFcndmMjg2cGRy?=
 =?utf-8?B?Y0c1R1RpTGhUR0ljZkYycnV1bmhWZFhvS0p6cWxGWXZaRkllWTI3YVg1Snpk?=
 =?utf-8?B?LzFYenhqSnFSenZTMmdGSjlGS1huNVhrSXh1NnpaUTd1SnU1b0lzYjM2by94?=
 =?utf-8?B?cC8rV091Tmx1L0p1N2xkMUlvSklmNXl1QWM4dkRncWt1UzVMMWt4KytkVW5M?=
 =?utf-8?B?dExHSHhTZ0c0WEQrUWNSZDhacnRuSWxLdURIM3kvL2FvMTVQVlNEY2hsaTBY?=
 =?utf-8?B?YkF3SEp2NlJ2d25tdG1xcWpmL01YbGtMTGZEdVZTaGRlSkx4a2RxTVlxTE41?=
 =?utf-8?B?OWFDQVpUaTZCSlM0elpUWU55M0w5cTZGVzJwTktsT1VSSFFiOTN6UW1zNDFj?=
 =?utf-8?B?OVlTYjJmTmZJRFhPTThQc0ttMWw5MEd3dkp0UG5PcDdDa0ZZTHRWODlaaUVV?=
 =?utf-8?B?WkpqRlpZMWkxV2txdmJtSHB6VzQ5RnVZQmVibVNqd0VUQUZoc1NENlhCbTRo?=
 =?utf-8?B?MC9adjI0eERYeloyVmpBUUtmbnZRUzNZK1JnSkhxYTI1a3Y3eFF0YUVXeGpN?=
 =?utf-8?B?aWlXM1RDb3o0OE1OR1hpK05yVXJUM1hTK1JVTzFIbmRhVDB0Um9lZ2FxS1k4?=
 =?utf-8?B?YjJNcUlJVXR2MmVpWk4zZDA5WmVEaUI2Q2MwRE5BN0dkV2pNak9lUE1aQVAz?=
 =?utf-8?B?ZDFoUE94R3dBeTcrQmcwZmJQTm14bzNGTC9meFBnSml0eVdibnlleFlzcFNI?=
 =?utf-8?Q?w4hJH51wQZ2r4e8i5xaXpdtXBY1J6m9XlJy/gm4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 258b01ef-e0ae-46eb-7640-08d972d4e62e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:28:17.2422
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rmo54oSoWhokqMYSYw/bukK2rNAcW4qnvYcwxpAZO8m7VQPetOe1DN2n4eJXfNCnPZibptN2wZOv6TFZKKN1VQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

On 08.09.2021 15:32, Juergen Gross wrote:
> On 08.09.21 13:07, Jan Beulich wrote:
>> On 08.09.2021 09:36, Juergen Gross wrote:
>>> Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update page
>>> table entries") introduced a regression when running as Xen PV guest.
>>
>> The description of that change starts with "pmd/pud_populate is the
>> right interface to be used to set the respective page table entries."
>> If this is deemed true, I don't think pmd_populate() should call
>> paravirt_alloc_pte(): The latter function, as its name says, is
>> supposed to be called for newly allocated page tables only (aiui).
> 
> In theory you are correct, but my experience with reality tells me that
> another set of macros for this case will not be appreciated.

Perhaps a new parameter to the macros / inlines identifying fresh
vs moved? Or perhaps the offending change wasn't really correct in
what its description said?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:31:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:31:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182117.329555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNybm-0006wH-7W; Wed, 08 Sep 2021 14:31:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182117.329555; Wed, 08 Sep 2021 14:31:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNybm-0006w9-3w; Wed, 08 Sep 2021 14:31:26 +0000
Received: by outflank-mailman (input) for mailman id 182117;
 Wed, 08 Sep 2021 14:31:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0iEI=N6=epam.com=prvs=98850bb71f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNybk-0006vr-58
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:31:24 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 70f75e52-10b1-11ec-b15f-12813bfff9fa;
 Wed, 08 Sep 2021 14:31:23 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 188E42Gu010308;
 Wed, 8 Sep 2021 14:31:20 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2054.outbound.protection.outlook.com [104.47.4.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3axxky841j-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 08 Sep 2021 14:31:19 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6689.eurprd03.prod.outlook.com (2603:10a6:20b:2db::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 14:31:17 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.015; Wed, 8 Sep 2021
 14:31:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70f75e52-10b1-11ec-b15f-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kCq4+k/jbsTJofAybHa3ZA1NyoNoHykUhwtfRHe9b3+MRoJuYQ5D9VPs3yibzUs8aC+jS+G69Q9QE9xqhUVJ5J52MVaLdLskg/AC5BzYTiyiJcWb3IIyxUzgoqGUTq3OwjcRiLHElOis+XrN5x+a8s7GYsBIFaBTGlGDrWfP+a51h6w6tD0z6+ALvaopYO4otWaiRBqfufhJei1SJdN9nC+6/yU1CBilC7j9NzG8wElNTqtvJ11tuIjXPy4PYCpkytZaRFkY/LpjjTjoM+slXc+SvLmH6gXmJLhEfygbUQQQO5aqgg6Z6hfJ8cxphgW5P740reQ0IsTu317KgzYszw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=s+J8jXTMhPTDJuhVNF/qQX5QGCTBsoadplJY/1/I5rg=;
 b=HxtEkXgGQprarekDnNpTj8imXSdqg3u47DrUrwesMibRT2bK1aAHI33At8d8bcEseIrvldGnOMMbqX84PCpYWLnjSoV9kKB6P7ccsUsSvCdIlrfgCgURZHrky1W/c1CB0pH7tOq9TFGe8b0DgGdIdpZZGQeYkQWa+UWWL3hDjCNU2OskAkzyFdoVcY+dKYc1uNpFPhV6tsa5ARLyUkW3KQ6S1keCYI9GUc+atWgSS86hdwdIl4i9j52fDINacGwLfoSmHkualHwIW/m2FKeRO93oVfegOWQd+LomPxO2cC4LWTK94jUKHRpPt98YEOdzxftlZjCz4ogcL0XuEjb1Mg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=s+J8jXTMhPTDJuhVNF/qQX5QGCTBsoadplJY/1/I5rg=;
 b=GXeY/ZIDd5Z1gUTslVSrZjDfFA9D0PODmCqXlPuqgVxuxUVjFHWNkdxgrUosyBj3PPXssweR1gC46Hr7YX+8wWBMhQL/gsr05GM9UHHHOxDL4FoIYV/ER8qCo9SiKWt11rovnDNKfX9cVFR2PKfYQweRbuqZn2VFgCIcFKZZMHSb8UBVNg4Xszu97BlV7hfNz5aBg8UfZSv5+rn7eAbzKCc28Ls3GizeaiuWpeiKYije8IjqOwunE9xZd04RSa28BjOEQ67e0q/jJ73kCxbMAZ5nbTg1rX1AsildzAu0XUGze/YQwwq4ZLETDlgPA2PL27z8hAlN8gDCQlj2vYDzdg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Topic: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Index: AQHXoKxjXfke7B20QEeo6YjlfUMhlquXGvYAgAMgDgA=
Date: Wed, 8 Sep 2021 14:31:17 +0000
Message-ID: <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
In-Reply-To: <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 88493270-bdbb-4d6d-a6aa-08d972d551c3
x-ms-traffictypediagnostic: AM9PR03MB6689:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB668990F75822E25E80BB00D1E7D49@AM9PR03MB6689.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 NyDj4D1q5llDhKckMulHrIJJ78iqnz0yZlZPj2s/Oz9/LbeiYCQNO9Y+dcDmj+1MOxd0f3wh2s+o0PISkGWUo5pgKv3px3LgO3csIvWaLUQX/vLfho/z23D3T9ZanQL/J/2OerOFuw3+9bj7LdyKi7rtNSoT7NMBp9Rbzhvhg5t92vU/ZZ+PLmgSo2gFQdasnIR9jGhf5goklpRHp+78mtuibVpo4zYWBz3wVBtCxYvrriAApIqzmekdWa8AmofJPefOgjBCAyaxYQ9KZtY+WJZtY9ILXtjpNwQiZKbsaEJAWfM1T/1ToiozJ3tDImrQ5G8jTzlmckYyQ9Pvm3YGHeaXhreJtgdsrhkPoxosZ0y9Qr2phqsIDFocid83hJHdpZ/511tc4yc592G+gs3gsdg/s2aj4XhHfiiRdt+FLneg+7yhNzaHHdi3NO3Zqj5MQzK/desbZk9rKqqWt1ZvGwo0G4+dV6wQf+6jSTBeMKVqgmdHaSwCc1Vf69Ki3D7axTm6u1XZEIDxpuJAv8n2xD6jPEnKp0hpJGy9Yk8H4Fjs8RxH9UaGrcwVrj5ugHck53swbMMM9bRifbXuwuEVkx3cktQ4shTa7gaNJh8hf0HeYXeR3bSfln4GTeQ9ynDVHtToVj65BR4ROQpW+o47zWeLBzuU+MhMNGlmwpl9wcN7eroeA3R+sFap/X+ZDAZT03iwp0VFCHBN1O5vbm8tUC2wkTrW36bIEtHemJfZ2FAONhNbHLNYDdrA0juLLpon
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(39860400002)(136003)(396003)(346002)(366004)(186003)(4326008)(110136005)(26005)(71200400001)(54906003)(2906002)(316002)(53546011)(6506007)(38070700005)(122000001)(8676002)(5660300002)(31686004)(86362001)(38100700002)(66946007)(8936002)(76116006)(36756003)(66446008)(64756008)(66556008)(66476007)(83380400001)(31696002)(6486002)(2616005)(6512007)(478600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?cisyaGtrS2VCS0RPdittSW1nN1dXSitKYUxPbkFzNVpyMFNLT3ExSndwd0Jt?=
 =?utf-8?B?Ni9WcW95NlAvQmhhNWNPdGtPajVLdWtKRjhVeTJBdlRMS2FFK3g2c2dIVUFV?=
 =?utf-8?B?VDMyclNKeUFjZVdIY25qTFBjOXhNYlQ2QmJ5QzRxUndUcVdHekFjcXRCeStk?=
 =?utf-8?B?S294QTE3WDNvNjJ4M1hlajllMTVwRHIzdFFEZlk0QXBBVytnRzA0dXp2UHhB?=
 =?utf-8?B?UWRSTU1lQlNsY294Qnl2UTBZVjZkdnRGZSt4cll0cEhnMG0yckRCNlhVcW1a?=
 =?utf-8?B?SElLNnlIWkduNndEZ0pkTnhDVzFReDduSUx0dWJRWUw5Ry9iTkEvSWNhTzB0?=
 =?utf-8?B?UTExTVNjMjdRZSsyOU8vaFRVZ1JWSDdLdkJmaFA3bTdUa3VISGl3SjRUSXE2?=
 =?utf-8?B?c1J0Yzh6VG8wWExDbmNhNUdGZjB1eGI5OHhFbGNVZmRKbDFJNjd1V1NKd3B5?=
 =?utf-8?B?alo4VjY5dGo4Z3owQVNWa1RHb1lIbGNoMDE2OWlENlpxYU9wRjU3cm95T2NH?=
 =?utf-8?B?UEhYZS9pWTJWb3NIZVg5UmhNSVBmamhleVcxTmlqREFvWlBZVTBYNGJzQzJq?=
 =?utf-8?B?ckdRRFZYb1lmUTRva1lhZGJvamw2SjUyVE9jUHBOWHBWZlV6cnB0dlMzaDhm?=
 =?utf-8?B?RlhPVjk4ZWpPOGFqTExJU2tBR3JjUk9PSlhiQ0ZLZUU1M1l4bzNYc3hDS1U1?=
 =?utf-8?B?OXdlVU9NNU5Fc01sbk5RM0xQb251dVhxdjcwUWFDTWJsL1ZoYnhidStSS2Z0?=
 =?utf-8?B?YVJQL0w5dXpXY1RuZW0xYktaM29JejNNTm0ySWYwMkl4WHNLMWVoWmRwVnVw?=
 =?utf-8?B?R0thTnBLbWJDKyswTit1V2FGbjdYelNLZlBoU1FrOE1NT01MaWdSYnd3clhk?=
 =?utf-8?B?RjRnVmRYRVFxa3dhN0FJdGJHSXU4UHZHbGdMRHZqNnhiSWJtSXpNYjFGMTRk?=
 =?utf-8?B?UnhuUnI0UkZGWGdsVkVqdDlBaGpZSU5Wcjl4UjM5dWJSd0hzL1FWN3J6aXFK?=
 =?utf-8?B?NGlpbFdGRG1GL1ROd2RIUWdlY3Y5SjRBR2xnMXQ4ZmtXZ1JnTUduN29QVjBI?=
 =?utf-8?B?L0Z2dEVpQkxqalBBekg0akk0ZkFGS0RKMUE1UzJMNitXZFB1eEpMQXA2eDM1?=
 =?utf-8?B?UXpNN3JGeVl6UGZMc3pSM3lHUUVQdklwMmFsM1Y4VE5tRjhtcFRzQVV3Ymt5?=
 =?utf-8?B?UGtjekljcm1HU0RWempVcERQUzZYWUZGMGFjZ1RSdU9HVUNibzR6VWR1dmRE?=
 =?utf-8?B?NENlMjQ3ZW1sZzVQZkNWbXQ4TFBKam03SmQ5ZVRSTFVVNUlCT0xTd2RYcnFy?=
 =?utf-8?B?TXRhSFVRaG1vUXhjWVE0RzR5dXIvMjVjLzZhS25tUFFsRmd2R0tSOGdyQU1z?=
 =?utf-8?B?bitySW5kWTc4K2w5b2g4NE9BSVRBNURXeHRxTE1EaTUrYmZFa3VMVDZxU3Y5?=
 =?utf-8?B?S0lFS2I5RkNhRkhWZDc5NGN4aEtVdzYzYU8zMm5tYXdZckQrMU5tUUloK0FO?=
 =?utf-8?B?N3dHSDdPYWZoSlI2THVCVWF5eityU01Nd3hkSkJHY3RldGFCVHhnZzVWR2Fa?=
 =?utf-8?B?WHVXd1kxbnAvWkJoNzJ5djhSNllYbjAvSjhsNjVTN3o1dkFvMXhQdHgrNyt0?=
 =?utf-8?B?VERqVTlqQzNtcUlYb0ZVR3gva2dxMHJSeGg0M0RnY3IrbC9wM3grcGFTT25w?=
 =?utf-8?B?OG84QWhUSnhwWXJrV3JaRmNPdmsvdGl2bHNVNVRVUXpaZjBvZkZ6SUdKUmJk?=
 =?utf-8?Q?hBSmnfFLk+tkpGp1BXbhMrOVuREfDdh3tE4H1wi?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C204A37F9BFE61479255EC5544474337@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88493270-bdbb-4d6d-a6aa-08d972d551c3
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 14:31:17.3748
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Oo0ipHk4Q44Jfb89JSsDqhyt3AnU8XG63UYdbVO7G71fjWrhfpmJo4pZ9SmjPhnXBe3SDW8l2vWTznCiACEQhxNyk5ITC+ofsZswL3k6awXfAxIMYZSWW65GlZzcuww/
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6689
X-Proofpoint-GUID: yVyAGTPpKiaiegS4wgjflsqDfMdwZlcy
X-Proofpoint-ORIG-GUID: yVyAGTPpKiaiegS4wgjflsqDfMdwZlcy
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-08_06,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015
 phishscore=0 mlxlogscore=975 impostorscore=0 priorityscore=1501
 malwarescore=0 mlxscore=0 suspectscore=0 spamscore=0 adultscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080092

DQpPbiAwNi4wOS4yMSAxNzo0NywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+
IEluc3RlYWQgb2YgaGFuZGxpbmcgYSBzaW5nbGUgcmFuZ2Ugc2V0LCB0aGF0IGNvbnRhaW5zIGFs
bCB0aGUgbWVtb3J5DQo+PiByZWdpb25zIG9mIGFsbCB0aGUgQkFScyBhbmQgUk9NLCBoYXZlIHRo
ZW0gcGVyIEJBUi4NCj4gV2l0aG91dCBsb29raW5nIGF0IGhvdyB5b3UgY2Fycnkgb3V0IHRoaXMg
Y2hhbmdlIC0gdGhpcyBsb29rIHdyb25nIChhcw0KPiBpbjogd2FzdGVmdWwpIHRvIG1lLiBEZXNw
aXRlIC4uLg0KPg0KPj4gVGhpcyBpcyBpbiBwcmVwYXJhdGlvbiBvZiBtYWtpbmcgbm9uLWlkZW50
aXR5IG1hcHBpbmdzIGluIHAybSBmb3IgdGhlDQo+PiBNTUlPcy9ST00uDQo+IC4uLiB0aGUgbmVl
ZCBmb3IgdGhpcywgZXZlcnkgaW5kaXZpZHVhbCBCQVIgaXMgc3RpbGwgY29udGlndW91cyBpbiBi
b3RoDQo+IGhvc3QgYW5kIGd1ZXN0IGFkZHJlc3Mgc3BhY2VzLCBzbyBjYW4gYmUgcmVwcmVzZW50
ZWQgYXMgYSBzaW5nbGUNCj4gKHN0YXJ0LGVuZCkgdHVwbGUgKG9yIGEgcGFpciB0aGVyZW9mLCB0
byBhY2NvdW50IGZvciBib3RoIGhvc3QgYW5kIGd1ZXN0DQo+IHZhbHVlcykuIE5vIG5lZWQgdG8g
dXNlIGEgcmFuZ2VzZXQgZm9yIHRoaXMuDQoNCkZpcnN0IG9mIGFsbCB0aGlzIGNoYW5nZSBpcyBp
biBwcmVwYXJhdGlvbiBmb3Igbm9uLWlkZW50aXR5IG1hcHBpbmdzLA0KDQplLmcuIGN1cnJlbnRs
eSB3ZSBjb2xsZWN0IGFsbCB0aGUgbWVtb3J5IHJhbmdlcyB3aGljaCByZXF1aXJlIG1hcHBpbmdz
DQoNCmludG8gYSBzaW5nbGUgcmFuZ2Ugc2V0LCB0aGVuIHdlIGN1dCBvZmYgTVNJLVggcmVnaW9u
cyBhbmQgdGhlbiB1c2UgcmFuZ2Ugc2V0DQoNCmZ1bmN0aW9uYWxpdHkgdG8gY2FsbCBhIGNhbGxi
YWNrIGZvciBldmVyeSBtZW1vcnkgcmFuZ2UgbGVmdCBhZnRlciBNU0ktWC4NCg0KVGhpcyB3b3Jr
cyBwZXJmZWN0bHkgZmluZSBmb3IgMToxIG1hcHBpbmdzLCBlLmcuIHdoYXQgd2UgaGF2ZSBhcyB0
aGUgcmFuZ2UNCg0Kc2V0J3Mgc3RhcnRpbmcgYWRkcmVzcyBpcyB3aGF0IHdlIHdhbnQgdG8gYmUg
bWFwcGVkL3VubWFwcGVkLg0KDQpXaHkgcmFuZ2Ugc2V0cz8gQmVjYXVzZSB0aGV5IGFsbG93IHBh
cnRpYWwgbWFwcGluZ3MsIGUuZy4geW91IGNhbiBtYXAgcGFydCBvZg0KDQp0aGUgcmFuZ2UgYW5k
IHJldHVybiBiYWNrIGFuZCBjb250aW51ZSBmcm9tIHdoZXJlIHlvdSBzdG9wcGVkLiBBbmQgaWYg
SQ0KDQp1bmRlcnN0YW5kIHRoYXQgY29ycmVjdGx5IHRoYXQgd2FzIHRoZSBpbml0aWFsIGludGVu
dGlvbiBvZiBpbnRyb2R1Y2luZyByYW5nZSBzZXRzIGhlcmUuDQoNCg0KRm9yIG5vbi1pZGVudGl0
eSBtYXBwaW5ncyB0aGlzIGJlY29tZXMgbm90IHRoYXQgZWFzeS4gRWFjaCBpbmRpdmlkdWFsIEJB
UiBtYXkgYmUNCg0KbWFwcGVkIGRpZmZlcmVudGx5IGFjY29yZGluZyB0byB3aGF0IGd1ZXN0IE9T
IGhhcyBwcm9ncmFtbWVkIGFzIGJhci0+Z3Vlc3RfYWRkcg0KDQooZ3Vlc3QgdmlldyBvZiB0aGUg
QkFSIHN0YXJ0KS4gVGh1cyB3ZSBuZWVkIHRvIGNvbGxlY3QgYWxsIHRob3NlIG5vbi1pZGVudGl0
eSBtYXBwaW5ncw0KDQpwZXIgQkFSIG5vdyAoc28gd2UgaGF2ZSBhIG1hcHBpbmcgImd1ZXN0IHZp
ZXciIDogInBoeXNpY2FsIEJBUiIgYW5kIGFnYWluIGN1dCBvZmYNCg0KTVNJLVggcmVnaW9ucyBh
cyBiZWZvcmUuwqAgU28sIHllcywgaXQgbWF5IGJlIGEgYml0IHdhc3RlZnVsIHRvIHVzZSBtYW55
IHJhbmdlIHNldHMsDQoNCmJ1dCBtYWtlcyB2UENJIGxpZmUgbXVjaC1tdWNoIGVhc2llci4gVGh1
cywgSSB0aGluayB0aGF0IGV2ZW4gcGVyLUJBUiByYW5nZSBzZXRzIGFyZQ0KDQpnb29kIHRvIGdv
IGFzIHRoZXkgaGF2ZSBtb3JlIHByb3MgdGhhbiBjb25zLiBJTU8NCg0KRXZlbiBpZiB3ZSBnbyB3
aXRoICJjYW4gYmUgcmVwcmVzZW50ZWQgYXMgYSBzaW5nbGUgKHN0YXJ0LGVuZCkgdHVwbGUiIGl0
IGRvZXNuJ3QgYW5zd2VyDQoNCnRoZSBxdWVzdGlvbiB3aGF0IG5lZWRzIHRvIGJlIGRvbmUgaWYg
YSByYW5nZSBnZXRzIHBhcnRpYWxseSBtYXBwZWQvdW5tYXBwZWQuDQoNCldlJ2xsIG5lZWQgdG8g
cHV0IHNvbWUgbG9naWMgdG8gcmUtdHJ5IHRoZSBvcGVyYXRpb24gbGF0ZXIgYW5kIHJlbWVtYmVy
IHdoZXJlIGRpZCB3ZSBzdG9wLg0KDQpBdCB0aGUgZW5kIG9mIHRoZSBkYXkgd2UnbGwgaW52ZW50
IG9uZSBtb3JlIHJhbmdlIHNldCwgYnV0IG5vdyB2UENJIG93bi4NCg0KPg0KPiBKYW4NCj4NClRo
YW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:36:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:36:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182144.329584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNygT-0000X2-AK; Wed, 08 Sep 2021 14:36:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182144.329584; Wed, 08 Sep 2021 14:36:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNygT-0000Wv-7H; Wed, 08 Sep 2021 14:36:17 +0000
Received: by outflank-mailman (input) for mailman id 182144;
 Wed, 08 Sep 2021 14:36:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNygS-0000Wp-6j
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:36:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1e7118fc-10b2-11ec-b160-12813bfff9fa;
 Wed, 08 Sep 2021 14:36:15 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-7td--1hNMv2e53sBoxk4yw-1; Wed, 08 Sep 2021 16:36:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5598.eurprd04.prod.outlook.com (2603:10a6:803:e9::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 14:36:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 14:36:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0025.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 14:36:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e7118fc-10b2-11ec-b160-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631111774;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZbpT9o6qcRjHxsKKMwBw8BXxu4vaTaXGYPkuG4Lg9uE=;
	b=Y5cmscKmLR/CGI+zwiVKgF23G8YSgoq+TgQYhD6T1Rz2WClMqiF9CkJA6AlzWJapBzOJ1S
	2+2oPQgk8sekreF+nQT3D4p1MP/Ge6G/3D+/Ra4DUaRzRxxIWIHFNYjrTmeBuuAd0gIQgq
	5HMDilOnHuwvFYD5xEgrDS/K1OaD6S4=
X-MC-Unique: 7td--1hNMv2e53sBoxk4yw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TerP9x9BmiCHfpDlH0Sh4Idd/x2veCKc/65Jroi2thxHwagBmHIkh1aKPGXyuMhzOKXfyOzM/ZTNq4yzRehYW194lYbKCAjTjVZ5+1gZ94+Y8ol0KRiuVuQD2kFE92trog9lzX2NRN2dj5+Prz7lO37fWcXdMnlxS6kcDbTGiqoiEw3+6fOwhepNX23PSWfv0HC9u8AmVWlDRCwKSDWjZ0TlZ1X6lbfZy1cX8xjanR1PjZZuyxS/TDzeBAb1/vbov3nOs7/ehgnosFva58Pj9ACeQ4vF1xPMfXn3Vm96yY2rcLdGH1Z2kRdhST6nd6OzhsVRi6enknYOw3Lyh57ZIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IDcu2Y264VS1Qq2YwjagZznV1KTSt9QTfw5Vn+imeYY=;
 b=lT7Ft7+/G+uqvRlXwGz0V/xB+gCHE0uuGPpAFTyhveK70oNX+JrsI7hxCd+gZ3gumfX/ZKzXrmFU4NXZgnx9Q5jguqjb+PmmtnOkJc0fO9GRXpuR7qRux8D5qgFfTOzH6XsEVIcdWQPOSsIRO3K6Sf7Z71UYHv9yuvvAHj6LCEu/8WNXgG1F+3TudShS9yCY/g0hP8P7qU8PoptC8k7E/Lyt/Es0yWjOdzwZ2yv5VFiFlSsnoEzv4I5V2G2Hehj3eFW8YFTDpxV1NZuJusgx26suA+R11mBepHxQfK1kbmrZrI65oYZ0uEokEODcyuk35wM0d1/tx+PlvIP5u+ZXuw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 3/3] x86/boot: fold branches in video handling code
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20bf2b1c-9c4b-bf9e-bf57-d0ba18e6458c@suse.com>
 <33dc91cf-1c80-b386-f9e0-6cdb8e32d5d0@suse.com>
 <6d74fb5a-d7ca-8fab-7f6e-09f6a0a9a700@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <704e971f-5e52-60fc-0a54-6dcca9157745@suse.com>
Date: Wed, 8 Sep 2021 16:36:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <6d74fb5a-d7ca-8fab-7f6e-09f6a0a9a700@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2P264CA0025.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a414b58a-0f09-45b4-5972-08d972d600fd
X-MS-TrafficTypeDiagnostic: VI1PR04MB5598:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB55985266FAD0E2FBB227FCFCB3D49@VI1PR04MB5598.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eLDWtceeFD0jUiNIH5rJRxXUtNWs5W2+sWz1HdaVLHwErxRvKsZgQyl2MVsEQvhDyWZR7KOlu5UlrYRrSXy390ug5OaeLuoO4MXUYprVsqTC06vWVjaxIS1z9FotuaVtFRmQqsl26QbZojFb++nt4IdIG51u63/+msEwl1HHkvOFpWfqKHEwTZslsKcoL9krWXai/MkKuYmyPyoudT80x0lHdSGLYuj3oQMz92i+xyuMQcrLpikofGtTvwLk1lezsMC87qdJ+1/tvZzQFBlj+7aaM4NUprl8YGTSi2ZOICAuzH1z4R4ocFd0H2zedrn6HDK0QkOfEuzFaCHzy4dItEEa3++FQh4fHD9LBo4FMJ2q1Qk/w2RdG3m4AbcOnKmx/r+VbkvNdEkgfri35ozix7ARX+zHUaXxS6b5KS18nv+3Ap4PK1I2J9jxaEmxlM7UaFv0OfMxzy/qAgxO4mnzSN9Df/QSv1KvT+7myxmw/2VxbPEMiFz/04GWo+/1U/DDUwoTBB8pqSzwQyDsIl047CCKnEfx+buGBkXZiMQs3XHZCEjZoVvTd09GKcEz74O66JIBGJoNdYwGnf1rcL1r6eEFyVrmuevMEkifYkFxWqRbbd0hGj60Xfm8/DQakamp4gaeYgeEXBX2WUADHdjO8zyX7FQYdmNy9AbCa46C5WI+aNWGbuek+QoG3IjZx6QGNDy65TOsHQ5ViD5YT3+bq2LD9IJ1m5H3QLBeh4rIUzGYrjmtMuckjKUfChnNna3c
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(396003)(39860400002)(136003)(956004)(26005)(2616005)(6486002)(54906003)(38100700002)(478600001)(186003)(5660300002)(31696002)(316002)(83380400001)(6916009)(86362001)(2906002)(16576012)(31686004)(36756003)(8676002)(53546011)(8936002)(66946007)(66476007)(66556008)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?m3KkL1kx8Hw6DjkCbKbH5+vtgLd22g4QSw2Gwy5or2Qq8r43Q68as+emytnl?=
 =?us-ascii?Q?odiMRTe/YHtvanL3eo+0zQXsLcByka3QY7W/0L6phHeCWdLM+a6Jrd3DH+Ew?=
 =?us-ascii?Q?XjQkVBdutoSQnyl6sQxe2MrO0bLOnvAAEMT1u3YMG1yjzgn6+yk6jf5AYhxV?=
 =?us-ascii?Q?9s2sFg8MtfMihdYuwi7lAuYPCnxpDmsIr3lwiJPaORXfXpl66yvlZ3Epwz/9?=
 =?us-ascii?Q?2h0+KwFrhrcgFPqoAyx210fr8EecU2Jr2bBvMWZNJ74n6HULeboeuq6IuRJl?=
 =?us-ascii?Q?BScrly4OVQaZmUqZIoydy84fPklqMcal/LxrEFGVhGO+rHPLLN2ZnthBZ5Y9?=
 =?us-ascii?Q?lBmb44vyJhwBvMhZ1VEkXg3f9gRca7I41zCq+7I9QL2AlgHBbyibQMLvV82e?=
 =?us-ascii?Q?wIa1RykiEZ1qgu4HbwE1yAMVxeXg8KcQPihyvOJnNy8GvpxmsI4bysRD3ybi?=
 =?us-ascii?Q?DAd63xHxPNFJLLo7zX7aB6ikKDdFjp4fnRbujTTFmQT7SdpwIdKfFl5Cv5rQ?=
 =?us-ascii?Q?by5c0YBLZW8AbNrrzBgsKC3jNupOreaG+rMqDu1oDae5T0iqtBP2dgVnKfBI?=
 =?us-ascii?Q?KT+4bcMYnDeEqinWvwlS20BZCLX+RfAaj5rjsg1F/yjov+RmAVJ52fg7ypjW?=
 =?us-ascii?Q?Lbaru3cLgMf5+12NeuHub7toSCVKEt1NyIZgeLSW7JwfI2L3/QVv0BsG1CBm?=
 =?us-ascii?Q?sL6feznvCgij0hB63Hg3ZSCXSWwpSwLWRknB6VtL7kIBounOyjLzO2edC9x2?=
 =?us-ascii?Q?xpUB+wJYHGEP1NpUomN/PDiGyvKAnI9kgm38DZGiSJrQrZheMazh3nuawJWb?=
 =?us-ascii?Q?UvS6VC8OyUeiozZ0BohCNUqqCUuJd/12YkzbS4nOXZP0sFBqRSigmQIqy6Kh?=
 =?us-ascii?Q?oI6Ed3AYpF973d+1FKWtPergovik0+6sUF1fMcqBzv36Uz2rIx9wCB6ao1/u?=
 =?us-ascii?Q?SJob6B+rCmXf0fHQLb1SWee6i1BRqMVbeQR7A2n31BiLZ4tIoDCIhcR0uKvL?=
 =?us-ascii?Q?83Vbh8fKNWyf5eqoXSrr6gKrhqiLSgkkQtMMxFTXQgYYvcqbH2fRo37mOVJ/?=
 =?us-ascii?Q?Eb7kFbfvqFcf4Bpl6SMIDTZ8XOHCxTMCwm9LdoTzWq2uDWFnfWMKTlVs7oNM?=
 =?us-ascii?Q?3Xrru7NkIwdHl9AlUG5WZ1LSBF9UxXwRVHl3iWGLCekqd3H4v6pfDUDRg7cC?=
 =?us-ascii?Q?1lPLIL42ShFsn6W5bS+50HyRTR+aejpyAotWd5U/w70nJVPF8S+URH6gvoYa?=
 =?us-ascii?Q?sJsn5rRbGubISbed/3K0tAbtwHHrAzK+YXPi2kZ3ZatyXB19h2iQ7m0URqUy?=
 =?us-ascii?Q?3mVW8vrvyJanSUAmlm0NuHP7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a414b58a-0f09-45b4-5972-08d972d600fd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:36:11.6216
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Z7SeZ0VVO6ISPNSt4cOjZ7h2zOuVp31DB7ZY7nWhp61EW8Fi/zfICwG4SUqFwt58dpqovWqgK/gA07INJcltJw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5598

On 08.09.2021 15:49, Andrew Cooper wrote:
> On 08/09/2021 14:24, Jan Beulich wrote:
>> Using Jcc to branch around a JMP is necessary only in pre-386 code,
>> where Jcc is limited to disp8. Use the opposite Jcc directly in two
>> places. Since it's adjacent, also convert an ORB to TESTB.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> This is an easy change reducing the overall trampoline size a little.
>> We're pretty close to needing a 4th page, which I'd prefer to avoid for
>> as long as we can.
>=20
> Honestly, it is high time we ditch all of this.
>=20
> The interactive video mode menu only appears to have confused users
> who've encountered it, and it is a weird thing to have in the first place=
.

Iirc that's not something we've invented, but logic we've inherited
from Linux.

> Furthermore, the Multiboot1/2 specs have supported passing
> video/edid/etc information for longer than Xen has been around.

As per patch 1 you can see how successful I was with using mb1 with
grub2 when it came to graphics modes. (Just as a data point: Afaics
the tools configuring grub in our distro still use "multiboot", not
"multiboot2", and hence we'd be hosed there when gfx bits only work
in the latter case.)

> I am not aware of anything we currently obtain via BIOS INT calls which
> we can't rely on the bootloader for.

I'm unaware of replacements for what edd.S does. Which means ...

>=C2=A0 Furthermore, if the bootloader
> can't provide it, we've got 0 chance of our 16bit assembly being able to
> do something useful...

... I don't really agree here.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:46:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182152.329595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyqA-0002G9-9v; Wed, 08 Sep 2021 14:46:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182152.329595; Wed, 08 Sep 2021 14:46:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyqA-0002G2-5y; Wed, 08 Sep 2021 14:46:18 +0000
Received: by outflank-mailman (input) for mailman id 182152;
 Wed, 08 Sep 2021 14:46:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNyq8-0002Fw-3r
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:46:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1135ad44-261c-4d06-b578-ad89a789e08f;
 Wed, 08 Sep 2021 14:46:14 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-Tx8vfN78OI-4-bqSio0OXw-1; Wed, 08 Sep 2021 16:46:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 14:46:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 14:46:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0024.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Wed, 8 Sep 2021 14:46:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1135ad44-261c-4d06-b578-ad89a789e08f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631112373;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=v8DtTdJYa/Tw7LBS0jWqH87B2qObWyBGzYqUMWDx7yU=;
	b=n2OPgli/jutmrIBRfTjO+U9OwoOtJNrebfikbZ7+k+h9hznHvL/yPNIOLp7K/323UtTdPK
	sU1wAgI6m4zKS6xuJ2bLTZbCJptFyy2h9vbskm/Tt/wgdUqab/oY4TPiP8ygsd5p9honFT
	mMAHMd9NwlIT8caPEEZjIK/XLWpxBx0=
X-MC-Unique: Tx8vfN78OI-4-bqSio0OXw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Oi9fkcmKjaTYTOev3W+Ci6hQscx+v9RBneU5VRSh7WBasEic+TbBUtrfXbidiYC855OHHIxhfY13WCXIHheLJWoXr2cuVU6tHgT2TZ7W2g11cadaZQdsw/39x48+bscrnx1LJTyAYRdauKM+3po1+Mw4pzjvumKnXEy+jh6iINLoCr+j+psd1lF+KzTU/+flBkyoiYxDE2+AkACywqma2NjE2QLyMqZx/bNeQqfx0P0gvX6hYqLhtZtJWeBzWChl1PHcRXZ7cH+zyz8IPp5Z3OygmoO1RpTn81OyWP+prEd1aE3d0V7vkzniZDA5Xlx+YhBXCmiL/zBCeFFDUqb1mA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LY85W1wE8icZ5Xp3oQXxJul0FLdst9GEKEWgt33NLQk=;
 b=ZEcav5W7WXlzxenujNzmbDZ9DwplOa1tkrB/Y9NtpSmgZZvofppy6XVe35M4l6Cfv2+QFFhvJN43SY3OpBBo1NKltggHRZRLbNoc+WVs4t0BeopeXjNOkLcXGsv84QHwz9dEs9kKLUNSyw+eQRmGf5yUtuRaRBa7Iv5UDLs1r1LhBtw6AWPPLE3iygbb7fv2ecxrim7yOE875TuoxUUCIRuq1J7IFUjSjcpY+yVUwViLKHnWq8OApjLH148Cf50w5YTZAmJgdgamyhFu6moAwCsCLEHBX3G836Bv7rQX1L3PXpbkzySuy++LH46CtkMImlBI9bWF/tB6UThTDhzzig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
 <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
 <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2fd6b81e-e5fc-f6d8-b24f-9fbb1db034d7@suse.com>
Date: Wed, 8 Sep 2021 16:46:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P195CA0024.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2c320985-ed5d-416b-b584-08d972d76567
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB736052E3EBA1161240E78653B3D49@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	88id3ZCw1FVGiuxgAA1YIgl9a6CxAoQ69uhX6QUx7cLwjweFw1cP39EPLButsV96xxIL71sa4ParzwTNN/ZxQHpZ74H8mObv+eQCV/WS4X1DlTOTjib0rfny3iNlUUulQMwZf/j9+NtSrK8sdGMszthZmIU7ScmCxZQ5McZUPlGylVWuF36iRUmQpf9dcqS988I3PubJvfU/7OyIKEk4KNjOhbwZqm2UwKVO4pYcIyBk+xdWBFNl+s14PYeoFOwwWgfbAzUecGqKP4MYTR9VsDFycLvr2ZuCCSNb9j8McnMpBdvYamDY8n/RCZehKxcbOyn0cLlXbyXcUkbdQvZK3Y8500e1QndW3YvVml0+DOjEgabnt5+/ebsl2p0Bg+48E4SJOw7DgEsTaKVLuL3QrJER5B8TNmQuBmMRrhbAr1Wfe42SgX0dQQSmU5BuflvxK1i+xNS9NgsnKB9IYIUXGwRedRWftoTuMZVFEctL+a1reaMGTcfxaFIrES2IEUBwyH5YyOp3nXi+ocJpX6bJKbj7pazL/UImWFTKEvkYpQaCk8yoNCuFOKvepzXlNHNGdYyM+Uket93v4/Xkr44ZIGKaHHalvaTgYZ3DZ0W9ephl3/ku55BKFbUV3UD1vAd6EnkU9XR7Di4vWWAcvynfCZqjrCH+Wjy+1FG44E+ePujqCyFp556X30GtpvyxhR0msVaKUJOio0mLhY4bw1QFaZ3tzTZ3/MRvlavvoQ/kZEDcvAgpvOuppz+l/GSAKB0V
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(346002)(366004)(136003)(396003)(31696002)(8936002)(53546011)(83380400001)(316002)(110136005)(956004)(54906003)(16576012)(5660300002)(2906002)(6486002)(66476007)(7416002)(66946007)(66556008)(6666004)(31686004)(36756003)(2616005)(26005)(186003)(4326008)(8676002)(86362001)(478600001)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?U/urUua2N+dzmmhf4WaLOJ3SBm3kk7oPWeJi69yuJ6EGpg0s/vl5Wgu4AAGv?=
 =?us-ascii?Q?9NvwwYjI8OgLaXfSsyuFIbtcqftW0G7luC12YidvYb3NZRMMy7EOJpAbWQmG?=
 =?us-ascii?Q?oRDwUCmR1zhNLuKCp6jJCzbhhK+UJja6QWA1lD6UQngSswkv+V7Z2lCwojAH?=
 =?us-ascii?Q?pcCn2TxGPi7P64pSqrFS4RlF+ePyL2uI0NUzILA7Bk1+mUWBUkERgoI6S256?=
 =?us-ascii?Q?dkAjyxhQvJ5O8qH6Gkgw9qXUsv0k0CaMKjitTAeGciTKFx03pOOb7BYaWHNY?=
 =?us-ascii?Q?fTPV4rwEozUGquoetFjOsZYp0hDBQRg0BEjrDQ4rqFWleQuht5YF4IeF+JkU?=
 =?us-ascii?Q?UIeEstIjjZaBtmpUnUcWqTtTJG5HbWyF/62IDyRo7JgpaEZPfgaM/f7WsS2Z?=
 =?us-ascii?Q?Zay9CjHtPiVMqeEBW69y6iXr/SUoi0cBNwUNBC2AXnYqWGW0IdSqiBBlhsmT?=
 =?us-ascii?Q?i6mfbm8ov2tnWtcCefvnnyGUwuz3IWGe6RxvtD82bu8Klbw2mlu1vpzPZjyy?=
 =?us-ascii?Q?E+KMhN+YGY+FtPRzZrkNegmDVAwtRyYvHX7U4rTSzOGmY01rS4T2pB9Soz4P?=
 =?us-ascii?Q?DkoalZd972C/a1Q3LmH8hJAdxCqQm2eVd+1PgUSuBYMF9WxuHqKng3COC73c?=
 =?us-ascii?Q?UIHaih+vofTiGuId3c4y/nTY5gJU12JeRh4FhPRsB84QWtFZLLpdYVo49WEG?=
 =?us-ascii?Q?GA2dY2JnT5MoAQcW30s5lATbwhFi/A5MYzyxWM/884RKSxDZp7Ra3iRNpfEB?=
 =?us-ascii?Q?GyoZ+pEKHS5eeOUcrf2I06Ix97Ram4E2fgPOwb0EkRUXo6yfdt/7NVN8khfc?=
 =?us-ascii?Q?wTw5EdkmuxKTA7ul9eVHXBPtdtQGW80nLYjyjfSzV2iLPU2aqmHSe2OKPynb?=
 =?us-ascii?Q?jFTZkB9AIfxLm7vmcvaXG0PIRJHYi10bGyjKrrYTNqk2SF4Jmr/W3ciUa6fX?=
 =?us-ascii?Q?Zpey03AMNlTGJehJ6lIFzKRX4YRWEmPhfeP6Te7P3mKrvbWypUzcL/ejEM1h?=
 =?us-ascii?Q?ABUYYeKewptj9guPVW/OjuUKjxfmtePrJDJZt05C7bt5gOwQJBYQFIMRbmxi?=
 =?us-ascii?Q?7X8NbSpl7yzZGYvuJJIoD+dQEjx65EFUBb3Kz2mzxzkr1ujdi9hiajwTIJKW?=
 =?us-ascii?Q?JF/UK+FdhcFgTtJRpymWLT07JkFDoiXerxyzOmf0F+75dZEEQMyGKrVcUwYr?=
 =?us-ascii?Q?TFkQubJCWKNgH2frPpyBIHAoFwGG8vCSEvKfPBRCiRbiFBWNeqJdARROVnNB?=
 =?us-ascii?Q?jAe0/54ujJvznxKlG90SoKZ1PUbnem36ETqmCk5EgHPVJ2HuImqTc0kakay/?=
 =?us-ascii?Q?tAOE6iQWXUT4fngGPQxo44T/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c320985-ed5d-416b-b584-08d972d76567
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:46:09.5825
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YiIAW1CJ5A6Oj+WAVT8dQOA3dGba7XoybLublHNN52+MkaotUM4BPNwfDJs3e+OmOdLEjTsgeCAKemdwSfnLmw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 08.09.2021 15:33, Oleksandr Andrushchenko wrote:
>=20
> On 08.09.21 13:03, Jan Beulich wrote:
>> On 08.09.2021 11:43, Oleksandr Andrushchenko wrote:
>>> On 08.09.21 12:27, Jan Beulich wrote:
>>>> On 07.09.2021 19:39, Oleksandr Andrushchenko wrote:
>>>>> On 07.09.21 19:30, Jan Beulich wrote:
>>>>>> On 07.09.2021 15:33, Oleksandr Andrushchenko wrote:
>>>>>>> On 06.09.21 17:31, Jan Beulich wrote:
>>>>>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>>>>>> --- a/xen/drivers/vpci/header.c
>>>>>>>>> +++ b/xen/drivers/vpci/header.c
>>>>>>>>> @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev =
*pdev, unsigned int reg,
>>>>>>>>>      static void guest_bar_write(const struct pci_dev *pdev, unsi=
gned int reg,
>>>>>>>>>                                  uint32_t val, void *data)
>>>>>>>>>      {
>>>>>>>>> +    struct vpci_bar *bar =3D data;
>>>>>>>>> +    bool hi =3D false;
>>>>>>>>> +
>>>>>>>>> +    if ( bar->type =3D=3D VPCI_BAR_MEM64_HI )
>>>>>>>>> +    {
>>>>>>>>> +        ASSERT(reg > PCI_BASE_ADDRESS_0);
>>>>>>>>> +        bar--;
>>>>>>>>> +        hi =3D true;
>>>>>>>>> +    }
>>>>>>>>> +    else
>>>>>>>>> +        val &=3D PCI_BASE_ADDRESS_MEM_MASK;
>>>>>>>>> +    bar->guest_addr &=3D ~(0xffffffffull << (hi ? 32 : 0));
>>>>>>>>> +    bar->guest_addr |=3D (uint64_t)val << (hi ? 32 : 0);
>>>>>>>> What you store here is not the address that's going to be used,
>>>>>>>>      as
>>>>>>>> you don't mask off the low bits (to account for the BAR's size).
>>>>>>>> When a BAR gets written with all ones, all writable bits get these
>>>>>>>> ones stored. The address of the BAR, aiui, really changes to
>>>>>>>> (typically) close below 4Gb (in the case of a 32-bit BAR), which
>>>>>>>> is why memory / I/O decoding should be off while sizing BARs.
>>>>>>>> Therefore you shouldn't look for the specific "all writable bits
>>>>>>>> are ones" pattern (or worse, as you presently do, the "all bits
>>>>>>>> outside of the type specifier are ones" one) on the read path.
>>>>>>>> Instead mask the value appropriately here, and simply return back
>>>>>>>> the stored value from the read path.
>>> But in case of BAR sizing I need to actually return BAR size.
>>> So, the comparison is the way to tell if the guest wants to read
>>> actual (configured) BAR value or it tries to determine BAR's size.
>>> This is why I compare and use the result as the answer to what needs
>>> to be supplied to the guest. So, if I don't compare with 0xffffffff for=
 the
>>> hi part and 0xfffffff0 for the low then how do I know when to return
>>> configured BAR or return the size?
>> Well, but that's the common misunderstanding that I've been trying
>> to point out: There's no difference between these two forms of
>> reads. The BARs are simply registers with some r/o bits. There's
>> no hidden 2nd register recording what was last written. When you
>> write 0xffffffff, all you do is set all writable bits to 1. When
>> you read back from the register, you will find all r/o bits
>> unchanged (which in particular means all lower address bits are
>> zero, thus allowing you to determine the size).
>>
>> When the spec says to write 0xffffffff for sizing purposes, OSes
>> should follow that, yes. This doesn't preclude them to use other
>> forms of writes for whichever purpose. Hence you do not want to
>> special case sizing, but instead you want to emulate correctly
>> all forms of writes, including subsequent reads to uniformly
>> return the intended / expected values.
>>
>> Just to give an example (perhaps a little contrived): To size a
>> 64-bit BAR, in principle you'd first need to write 0xffffffff to
>> both halves. But there's nothing wrong with doing this in a
>> different order: Act on the low half alone first, and then act
>> on the high half. The acting on the high half could even be
>> skipped if the low half sizing produced at least bit 31 set. Now
>> if you were to special case seeing ffffffff:fffffff? as the
>> last written pair of values, you'd break that (imo legitimate)
>> alternative process of sizing.
>=20
> How about:

Yes, that's what I was after. Just one nit right away:

> static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 uint32_t val, void *data)
> {
>  =C2=A0=C2=A0=C2=A0 struct vpci_bar *bar =3D data;
>  =C2=A0=C2=A0=C2=A0 bool hi =3D false;
>=20
>  =C2=A0=C2=A0=C2=A0 if ( bar->type =3D=3D VPCI_BAR_MEM64_HI )
>  =C2=A0=C2=A0=C2=A0 {
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ASSERT(reg > PCI_BASE_ADDRESS=
_0);
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bar--;
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hi =3D true;
>  =C2=A0=C2=A0=C2=A0 }
>  =C2=A0=C2=A0=C2=A0 else
>  =C2=A0=C2=A0=C2=A0 {
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val &=3D PCI_BASE_ADDRESS_MEM=
_MASK;
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val |=3D bar->type =3D=3D VPC=
I_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : PCI_BASE_ADDRESS_MEM_TYPE_64;
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val |=3D bar->prefetchable ? =
PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
>  =C2=A0=C2=A0=C2=A0 }
>=20
>  =C2=A0=C2=A0=C2=A0 bar->guest_addr &=3D ~(0xffffffffull << (hi ? 32 : 0)=
);
>  =C2=A0=C2=A0=C2=A0 bar->guest_addr |=3D (uint64_t)val << (hi ? 32 : 0);
>=20
>  =C2=A0=C2=A0=C2=A0 bar->guest_addr &=3D ~(bar->size - 1) | ~PCI_BASE_ADD=
RESS_MEM_MASK;
> }
>=20
> static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int r=
eg,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 void *data)
> {
>  =C2=A0=C2=A0=C2=A0 struct vpci_bar *bar =3D data;

const please.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:47:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:47:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182158.329606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyrG-0002r4-KP; Wed, 08 Sep 2021 14:47:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182158.329606; Wed, 08 Sep 2021 14:47:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyrG-0002qx-H1; Wed, 08 Sep 2021 14:47:26 +0000
Received: by outflank-mailman (input) for mailman id 182158;
 Wed, 08 Sep 2021 14:47:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ljEy=N6=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mNyrF-0002qn-Tt
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:47:25 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dd098093-113d-4c9e-bfdf-cd34d9a9e1b2;
 Wed, 08 Sep 2021 14:47:25 +0000 (UTC)
Received: from pps.filterd (m0246631.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188DfEbY030422; 
 Wed, 8 Sep 2021 14:47:23 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3axd44txky-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:47:23 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 188EaY3x103681;
 Wed, 8 Sep 2021 14:47:22 GMT
Received: from nam04-mw2-obe.outbound.protection.outlook.com
 (mail-mw2nam08lp2172.outbound.protection.outlook.com [104.47.73.172])
 by userp3020.oracle.com with ESMTP id 3axst3x6cm-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:47:22 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4399.namprd10.prod.outlook.com (2603:10b6:208:1d9::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 14:47:20 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4478.026; Wed, 8 Sep 2021
 14:47:20 +0000
Received: from [10.74.115.194] (138.3.201.2) by
 SN2PR01CA0084.prod.exchangelabs.com (2603:10b6:800::52) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 14:47:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd098093-113d-4c9e-bfdf-cd34d9a9e1b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=YxqmVh6FQ9ZKrX+i4+s2IIrwqHoUq4z1e+qAUQnB6WY=;
 b=V8iZEmW5csJHBBKS4cmrr5rOhlvGH6DmkwJ4Tq3ZI6cvigsVccRtjNNyGh3KeQzlYSSt
 A1ikX144WMnpzA9lCqKAoE+Fo/SVIaWs63Hnnttqq6zxzjwjvPWBVzPp3uyHUXnaTnOg
 GAPw4V7xNZNPznDyUsJkLMvZ1vbuziEP1TehTWOPYlP93jj/P9MS60vTEGfpQb8dTNxk
 t4m2EWmwI1PV1FCBLUc3KyiYAun+EYAdViCqF4JcJlqXPOobpJ98+d2Tf2IZLWESmof0
 tbfOoznfWKsyvPVi4Pn0GVUhWMRAi54sdivgAOGLL31ug1L4nLJwp8AEWsTEWRsm+ITi pA== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=YxqmVh6FQ9ZKrX+i4+s2IIrwqHoUq4z1e+qAUQnB6WY=;
 b=S0B14uExGMVKbuhFJ/cVYxOjAs8mwfcXSsQQUxXCJOohar+oEQpv5VlqYwrwLcJBzUHL
 9PdZvVKoQY8k4OgIIMz09EbRii8kA75SMeMxJMGDjvb8nQq/MKqQRnMl1s29vwnlywTP
 L7eDC2+vfFAvCCApHy2yrulyWv2KLgV5KXR9Q/79YrTJl6U7MhVqh5MLUxHO/woNf7CW
 QQxfTIB6PHtqmIer1dMk0L1Lq+oZg9BLZ4KHXKDUVnmll+ympfGqHGA4Rk3Jqpzx9cEx
 1lzxdwos/r9kWAtyRGNUXpHotXDuLrcCE/Ab4QPBPGduXX0suE4KiJATPqRUgoDENPj9 zQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BiCenKX6nm3MJVWqYptB0kwZ2SAMAqbOTE4GzayzS7tp1S67nfnQunElxzNcb+haz+6aMe0FaookqVjR9l8oxO7hknKSQicfVLXLAtv+aZtQSE9AQIRabVDL/SLuU9NN02NoH+X/ZQ45jBFSDygyFynTONGnbzTwkB6UptUXKM0tkRULawOt418gzrVmSwqt8Z6obh2iOgidFl8rWmnjIOWyADfyJXfq22UGyp6sE+4OF7h6GUGk2TiSBZm5N/bha9qGHPj+dT8WoEexdllD9aQXnAuoNhH+Zka4kcJdAyKXj0tqpw0GVcRLk3nJcdG2gUoku6bfoArVujB6Umg/UA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YxqmVh6FQ9ZKrX+i4+s2IIrwqHoUq4z1e+qAUQnB6WY=;
 b=llBxwXCpLhbZhUG9Tztiz4toFd4IYOrTISb9PojCS+/VkXAEv/GVOpv4m9RidTWTq8a8bd6LnMJxv3vpcWHUxbY5QzhDcUt7CqWjHuOXGSLagcqS3MGTIImOpHLMJl59r9CTzjYC9XaKosGVZ/GdaNwNYMFufu2RG0p83Tve9NohOWhZrWq9QVe7bCY6K4thcwV0kCLMq3wdfujXw3AHnWgJMpEptg+OP4LicBa0Y1hWi6prTe7mTDzvarO9Mmf7nz+WLQbXM9ECOhvckCTmq+WmiwlqG/D0e6Su4M5vIXwLyuZJBTgyBqh8qdBK94m82qcF2ITj3gwDS42ZuC9pjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YxqmVh6FQ9ZKrX+i4+s2IIrwqHoUq4z1e+qAUQnB6WY=;
 b=fY6YIuKmwx5V8bAK3w3gQ1YNBOU4zrKbad6sXGZGYPhTEyA1+K0/0vckwBvsRe91HmY2MGB/aT8Sumo6myTAVeJE80wMJQC6TnCU5psp1yF968io73NmtTiCOF/io4KvWxAmEz9Vd2bcGWkgaGQcCBqmJR+FvAeQc5uMQ6bfOs4=
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH] xen/balloon: use a kernel thread instead a workqueue
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        Jan Beulich <jbeulich@suse.com>
References: <20210827123206.15429-1-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>
Date: Wed, 8 Sep 2021 10:47:16 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210827123206.15429-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN2PR01CA0084.prod.exchangelabs.com (2603:10b6:800::52) To
 BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a4b69cbd-c1db-4cf1-f73f-08d972d78f72
X-MS-TrafficTypeDiagnostic: MN2PR10MB4399:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB43999D980A9E93E91795AE4A8AD49@MN2PR10MB4399.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1284;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	FyfiyvVLby/neppyX9lOwFvKyPJIiR6S+9HMmTtt2Dg+22DAk0Hc9agbR6vmby3wnlbPoik9CAcazvIlZLyDPtJEplZCJbE7VWpQ+nbBN1stG6rwYGFeqVELfTDyrmn/KY7tuplLZhYTJiz0VgW5q4C1EQuD88BRzkradyjKxL7KAhAVs3POklnxA6UitvKlaVQmx0aJlYO+pIxoXhGp2CpA6aU8V8GekAv6Fqu3Q7g+mfnm/S48YT1JaVEILXc296VKW36FDxJrmYlG3VcZsPXaX79M4fowIMqxQP9rpc3HBomBKUBE6f3F6rFUpYFTok1YHgDJMerNzgqCBzoNKZG8uMLSDtTqFknwI4m+stWo/OSTx/W5bVCgSihPxXntFj5IbJQ9H2a0AUYwwtVqfIF7uYhQfARAp4SglGrRIZ0WZKW5GJSsPnItKsQn+orSxduobcM1G+ONzri2Q3sGTjZYyJL590xpYS7ECAiz+WOnUbC3WaFuDrtuoeDK2w9k3a/YK2HTfCWdaT/DjcyWuQ+JVermDzrTDocAA58qj7N7iqznHJ7roRnU4fZxAMdmanhX34HaA7x7smpWvV0lZS/9QnFGIR8gYAwlrL6TIdyCmFT6SVSeToZfGNMgCpsEhLQitu7ZEqcHr55gAsThgIr9pzCk2eQc6eXCtgAmAi6Z5cMtL6w6+lpPye9X6lzSs1jMr2NX2hrUGOCGMeyqJw==
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(83380400001)(956004)(36756003)(6486002)(2906002)(54906003)(86362001)(508600001)(316002)(16576012)(44832011)(38100700002)(2616005)(66476007)(66946007)(66556008)(186003)(31686004)(4326008)(5660300002)(53546011)(8936002)(26005)(8676002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?MkJ5L3ZWQlVGNEg3UHNNMmlPSUZPVFArNHMrVnNPODJGK1JhVVIwZVF1d0V3?=
 =?utf-8?B?MkN0clJFcGo5RE96dUp4MUpSUEpqY2gvUjI1SUF3Sy9FV0RxV3BGNHZTbnBt?=
 =?utf-8?B?RUxlei9NbjE5Snl4dkdaQ2hCVnplK2M0aEE0ZVV3TGxwaWVCbk50Y0xWM01P?=
 =?utf-8?B?cjlLL245a0RKQm0yK2J4TDVmOWNENnpiR3JBQzBreVRUTXNBRFRhNTlDa0xK?=
 =?utf-8?B?aWdwN29sSXZwOUJSWW1MUFcxZEFTK3V2Zi93NEpCazN2NVhKQ1dkSWRPNUc0?=
 =?utf-8?B?d2dYdW9GNkZuUi9mYzZsUmVrZTN3aWlmdGFQSU9qRzVFWmRETHE5Q3liY2k1?=
 =?utf-8?B?ajNZTEdQWTZTWG8rZys0Sis1ME5NNUhaR3oxeDhROGd2cEl2cnkycjhDdjUy?=
 =?utf-8?B?RGRCTmtiNzF6VDIrdkVobTF4ZFFRTExGQmlQdHZjNlczam8vbUgzeU55c2dH?=
 =?utf-8?B?NjRZU2N5aFcvTjJWOU5TMlU3TSt4bXlLQnF0SnI3eEY2ZWYxakNLKzc0UnBz?=
 =?utf-8?B?V0VWWXZuQkxwaEpQRnlyZURUZzQxa1pVQVNPRnRsNTFPek5mOVdBUWRtSXZZ?=
 =?utf-8?B?dzdIUjZsZXdSQXlyU1I5NDZZazVOVnc4NVZ2Z0lRaksvNjJWb1VGVW53dWRl?=
 =?utf-8?B?ZnA3ZFo3MnErNmJJWkkzZUF6cGI0TjVVNEEyM1FtemtIdVlCb1c3dHhGL05E?=
 =?utf-8?B?NnRuVERvYkFQV0U0QjhYVllDaVREektRQTNyVU9VdUliYS82ckM0cm8rWStG?=
 =?utf-8?B?R2JjRmgvbzhLeEg5anlkbjFTT0pFVUw0bWViMWtjQVhCdFQxV1JJR1hRUE4z?=
 =?utf-8?B?MDU1V25rS2JHYWdMenlNN2Njd2ZwOWtVd0JydmNLMVlnSXhOcE9EbmhiNlNZ?=
 =?utf-8?B?WjBPbXdwajNrSHZKUEtNRnI5ZUtVRk1RNzduNTJNYmduNHFzOWl0V2txcmh4?=
 =?utf-8?B?RGErWHJuRXVpUG92ZTZ4YWxCWktZTHRFQ0Q3UnBrMHhKWDljNm5ZSFhYSE4r?=
 =?utf-8?B?T2F0SHZKdWNCT0kwMnBWTkhOaTNCeXFZN1E4YkM4aUU4enFoNmx5NTRtcHpQ?=
 =?utf-8?B?eE1uYlllTTEyUHl4eXlNWDFqUE9hOS9Zb3U1L29mc3JxeGdJMHkwem9ETE9I?=
 =?utf-8?B?TkxMaXBMcWJxUTRweXFkNHllSVZwRGh6QWJRUXBDUEYzdnpXcFFqZVBHUE9I?=
 =?utf-8?B?Z1JHTWZ6YTRIZmhvd0R4Q2wwRDRreXFwNG0waUl1WDRBc2hwVXA1OWFYc3Zx?=
 =?utf-8?B?QUQxZ3IrYzJhYnJxOGlJUjhUeEdqZWMwdVBDVXJzaHh0c1pvY050Z0c5UGdm?=
 =?utf-8?B?UFBaSEJEQ2IySjRPaE40VEJHLzN3amdkMy95SytjQ3FkOGNKOVZXR0VyMnlS?=
 =?utf-8?B?VXlRUFJaVlJUMVp1Qzh4eWlva0NkbWc5NU9ITW9kTVQwL0RSRERCMWNRRkxk?=
 =?utf-8?B?UDMraG5EdDd0Y0RveEtCK2dTdzlybzkwLzlVOTBJTTZCMmVqWmRGTEVHK1NX?=
 =?utf-8?B?ZDdmdzJRamdxVHJkdDdIdEl6a2lxNXdGWDZsbW11SEI2cU9Ja1BsbjloMkVu?=
 =?utf-8?B?cTI5R1h5eDFhNmRCMXZZaTU0aFgrOEh4ODdXdWFmUWEvdTNhSEFXS0lzaDBo?=
 =?utf-8?B?SE5DT3p4TzJxcFJ6eE0xYis4QmlSdExObDArWTZpZStDamNCU0dpc2d6MDFk?=
 =?utf-8?B?d0tMSTlRUlRBaUIrR0J6eVg5NkRlS0hFbWwvYkNjcG4xSzBrOVhscTd4WVpL?=
 =?utf-8?Q?FBF1L+Ot1A+yPuUk3olo65lVPmwiEpKXE0yTSqC?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a4b69cbd-c1db-4cf1-f73f-08d972d78f72
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:47:20.1350
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: khy0cw4NWpFnEzRq0OsL64g6kqQsGAjnGhdF4JjTDWB+fF0ExqenXFanSbWAgjWDwvgU80bTT04zobNjFrzYKh1Jbz3Fuk16QK3OwNH9JcQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4399
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10100 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0
 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080093
X-Proofpoint-GUID: PHKTu_8mP6F-M0kgjuqQT4dIkupBdrip
X-Proofpoint-ORIG-GUID: PHKTu_8mP6F-M0kgjuqQT4dIkupBdrip


On 8/27/21 8:32 AM, Juergen Gross wrote:
> +static bool balloon_thread_cond(enum bp_state state, long credit)
> +{
> +	if (state != BP_EAGAIN)
> +		credit = 0;
> +
> +	return current_credit() != credit || kthread_should_stop();
> +}
> +
> +/*
> + * As this is a kthread it is guaranteed to run as a single instance only.
>   * We may of course race updates of the target counts (which are protected
>   * by the balloon lock), or with changes to the Xen hard limit, but we will
>   * recover from these in time.
>   */
> -static void balloon_process(struct work_struct *work)
> +static int balloon_thread(void *unused)
>  {
>  	enum bp_state state = BP_DONE;
>  	long credit;
> +	unsigned long timeout;
> +
> +	set_freezable();
> +	for (;;) {
> +		if (state == BP_EAGAIN)
> +			timeout = balloon_stats.schedule_delay * HZ;
> +		else
> +			timeout = 3600 * HZ;
> +		credit = current_credit();
>  
> +		wait_event_interruptible_timeout(balloon_thread_wq,
> +				 balloon_thread_cond(state, credit), timeout);


Given that wait_event_interruptible_timeout() is a bunch of nested macros do we need to worry here about overly aggressive compiler optimizing out 'credit = current_credit()'?


-boris




From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:47:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:47:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182160.329617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyrW-0003HL-0k; Wed, 08 Sep 2021 14:47:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182160.329617; Wed, 08 Sep 2021 14:47:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyrV-0003HE-Tb; Wed, 08 Sep 2021 14:47:41 +0000
Received: by outflank-mailman (input) for mailman id 182160;
 Wed, 08 Sep 2021 14:47:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNyrU-0003Ge-Tj
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:47:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a2c40b9-ccba-4694-9d0f-edbae8295add;
 Wed, 08 Sep 2021 14:47:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 52D7022256;
 Wed,  8 Sep 2021 14:47:39 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id E608C13A96;
 Wed,  8 Sep 2021 14:47:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id Kxm0NgrNOGFWNwAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 14:47:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a2c40b9-ccba-4694-9d0f-edbae8295add
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631112459; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=jEFNWRIePbeIPt9GmDov9/wA6u8O4zqeE8y8N3lFb/0=;
	b=R7t45CL/vN0fqwIkKaTAptvlWWf5gcWvgVuoFeE8stPitbx+wMt2erghnki2rfI1oLiRal
	wySquU2/gkjDEWjjC/y+en3qzsYcI3d8UpNpxOHZ8/P2XcT+jtCtj+m9xtfmFOMJM65BAm
	5nu0K6rkLjboJJnJRO4INfze4CY3N6A=
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210908073640.11299-1-jgross@suse.com>
 <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
 <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
 <6862566d-4d55-dc5c-082a-da4fbcafcfce@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
Message-ID: <21bad77f-90fb-1288-be52-291b56ccdd40@suse.com>
Date: Wed, 8 Sep 2021 16:47:38 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <6862566d-4d55-dc5c-082a-da4fbcafcfce@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="3GzrgL9tJGXXoMnm3mnp4eEQIR0B4Ilr6"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--3GzrgL9tJGXXoMnm3mnp4eEQIR0B4Ilr6
Content-Type: multipart/mixed; boundary="BAS2MNyQNemPES6JplNDoPiAx8GLYmZex";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org, Sander Eikelenboom <linux@eikelenboom.it>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <21bad77f-90fb-1288-be52-291b56ccdd40@suse.com>
Subject: Re: [PATCH] xen: fix usage of pmd/pud_poplulate in mremap for pv
 guests
References: <20210908073640.11299-1-jgross@suse.com>
 <5a4859db-d173-88dd-5ea9-dd5fd893d934@suse.com>
 <34afed98-5072-c563-5d29-97e09a0b4ebd@suse.com>
 <6862566d-4d55-dc5c-082a-da4fbcafcfce@suse.com>
In-Reply-To: <6862566d-4d55-dc5c-082a-da4fbcafcfce@suse.com>

--BAS2MNyQNemPES6JplNDoPiAx8GLYmZex
Content-Type: multipart/mixed;
 boundary="------------57ADF394A2EC90BAF7B799FF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------57ADF394A2EC90BAF7B799FF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 08.09.21 16:28, Jan Beulich wrote:
> On 08.09.2021 15:32, Juergen Gross wrote:
>> On 08.09.21 13:07, Jan Beulich wrote:
>>> On 08.09.2021 09:36, Juergen Gross wrote:
>>>> Commit 0881ace292b662 ("mm/mremap: use pmd/pud_poplulate to update p=
age
>>>> table entries") introduced a regression when running as Xen PV guest=
=2E
>>>
>>> The description of that change starts with "pmd/pud_populate is the
>>> right interface to be used to set the respective page table entries."=

>>> If this is deemed true, I don't think pmd_populate() should call
>>> paravirt_alloc_pte(): The latter function, as its name says, is
>>> supposed to be called for newly allocated page tables only (aiui).
>>
>> In theory you are correct, but my experience with reality tells me tha=
t
>> another set of macros for this case will not be appreciated.
>=20
> Perhaps a new parameter to the macros / inlines identifying fresh
> vs moved? Or perhaps the offending change wasn't really correct in
> what its description said?

The problem is that those macros are spread over all architectures with
each architecture defining them separately. Changing all those will not
be really welcomed.

And the change was correct IMO, as the replaced pmd_set() should be used
for leaf entries only (at least in arch independent code).
pmd_populate() is the correct one for non-leaf entries.


Juergen

--------------57ADF394A2EC90BAF7B799FF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------57ADF394A2EC90BAF7B799FF--

--BAS2MNyQNemPES6JplNDoPiAx8GLYmZex--

--3GzrgL9tJGXXoMnm3mnp4eEQIR0B4Ilr6
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE4zQoFAwAAAAAACgkQsN6d1ii/Ey+r
dgf6Ake5fyapfF3DOH1qLAFqjRubIjOR2e/8QUAFI5HTwoeZXQhCgyPXBCLL6RRk/EdXEoiq8S9M
cjQmylBZGpE3ow72fI2tGPV1OVasY/3aowSKd8Ib+a7lcOr/y7Kt0kBo7IpWahER4GX0nmBjZHve
9izQnBRiAgGFoBHwq/1SKegbW0bKk9RSUnhboEb106+64kYG3Aaa4OGvSt9XMAUEYllq3VX2m62x
GQjcimcHVViw2CVSCJUu7gDr9qmY3O6aIctZln0G2TVsc9SpPm4ZctEq1psKvFz+ikW8JT68MWPd
EckUq8qIkYDKV+GltlE3Xz4GRcjWY99z5IJ9WS75sA==
=zv9R
-----END PGP SIGNATURE-----

--3GzrgL9tJGXXoMnm3mnp4eEQIR0B4Ilr6--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 14:54:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 14:54:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182186.329645 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyyB-0006Au-7Q; Wed, 08 Sep 2021 14:54:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182186.329645; Wed, 08 Sep 2021 14:54:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNyyB-0006An-3q; Wed, 08 Sep 2021 14:54:35 +0000
Received: by outflank-mailman (input) for mailman id 182186;
 Wed, 08 Sep 2021 14:54:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ljEy=N6=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mNyy9-0006Af-UP
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 14:54:33 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ad5aad60-10b4-11ec-b161-12813bfff9fa;
 Wed, 08 Sep 2021 14:54:33 +0000 (UTC)
Received: from pps.filterd (m0246632.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188EqxO2013120; 
 Wed, 8 Sep 2021 14:54:16 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3axcw6b1d5-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:54:15 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 188EoMJC159962;
 Wed, 8 Sep 2021 14:54:14 GMT
Received: from nam02-dm3-obe.outbound.protection.outlook.com
 (mail-dm3nam07lp2040.outbound.protection.outlook.com [104.47.56.40])
 by userp3020.oracle.com with ESMTP id 3axst3xexg-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 14:54:14 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BL0PR10MB3444.namprd10.prod.outlook.com (2603:10b6:208:73::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Wed, 8 Sep
 2021 14:54:11 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4478.026; Wed, 8 Sep 2021
 14:54:11 +0000
Received: from [10.74.115.194] (138.3.201.2) by
 SN7P220CA0014.NAMP220.PROD.OUTLOOK.COM (2603:10b6:806:123::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 14:54:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad5aad60-10b4-11ec-b161-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=R9CyFc89EI4mmm0TQEqBL19rGkf/JvNABOb3GOn/r5A=;
 b=RJWapuPcRb7dK7JLYWYhl+SPp3Bs9fPoGaJeNETVCxU2fULDBt+Mzfe8DlAkt63CuuWF
 dIOv24wtv1ZBjU5Yt/n8CqYCkT1fFq1tOi/KmfwoaY7aoCzFlKtDCGsNCxMDNpw6gSEG
 vFhLJa8kzvtEkO5y6ql9aMpIA1EqjXWTpQ+dUtPITyCodZfx2UhEfqmob7RYD1r1n8fZ
 gONIKku9MPmtNdhAODHWJv9MQk9ohQ4RIXnK0mPNTdyHEuK1bZBQ4ZgO36sYQx2VVGMM
 ipKbk9GSXuY/tVHi/uTBJUf8e7k5V88eC6C+7AOeTkC+lRFDXQdrt8UkV/RWeU0PU+ZQ EA== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=R9CyFc89EI4mmm0TQEqBL19rGkf/JvNABOb3GOn/r5A=;
 b=OoHcIVPIz+jqJ3N99214gI4DWgHUBJ5Q+8hdPsFwNRQueEDBYZp6yeV/tbmhT2LpqtAU
 4qy3NhCi+aw/8IzTnU7nIA3BY4KWd11wrM5zJGARv6l2Y8WEj3wuYA65BNeR7MmBz4PK
 Bh36crNjnqWv1iDYxeC8qpMAsX1XqeuOIP+BQwUQmjfphO5wJGGOq7pDwJ0KFVTSWFVq
 KMn18yj0f7U77+5F6YiFG2Qmu9M45TSKYrPSS8E0JxvfVBgAme/G1Y1tmrS32poiuOFG
 scdnu+cQ02IsMxt7Q9NLrN1Yfhmlo5B8FoZiMAgLfpqHZWop2Rh+ldBVkVI9vhvsvfQd BQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JatBAAa8Al7H/EPuRuKb1/ulkUmIOM7Tsvj/wePVUSrNTjsM36mNZ6UbEGCRGQVK7036H2BZj3b2K9mp+Izs2W8jV/AYxwVEQJyDyuiJI7pjDY2KAYAgC2wWUbKr252+WQAInail8b4o1KrJZ5ctPItIcuf65BdG6UQ2p7NunwOuSS3x8HmBBDKKmYh9SK2aRXXhnRSAIKTKcMBYj0yvS4fMLJwMcySLHQVnPR2fcpJUY3PnxqRlWM4zOrpxnToOgz/0PCYoerIGklP6sYfWsiRMUunA4cSlL7xWSnfQ4XX7FdhGkGa3i3XJ0l1GIZ9uo49YtZ+bNCY4eP7Y4/2sMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R9CyFc89EI4mmm0TQEqBL19rGkf/JvNABOb3GOn/r5A=;
 b=CGuRI04im/M3G8H0gfR5UmtJhSugMLh7k5TFxcZeIdJAHRIfgNTMy3b130D8RRM5fzjLl2YlreXYkH2IoXCvuJIpC3YlSzGLlHNbvsC141OmwWL+iNJ+pywpPca+RZ2+zgUjg0UGmVXtnlKiZEeSA+qmJcgRnz+vT3HCi1nnxbLSKvKWeOgQyMOqDmInWmHjVgdTmwGeDzhIPyYJiCJivCRDCpEXwEWOZ6tfJY5+0n1J5S5y71fkBb+8b9MYVvXGxXXFWJYZriWLdgXJ/bilFUdwwnl/SEHenNK15Q+1pJQJhr3A82zUesvWQHFrGTyrjXRxfGSPGEoHX0rtHUuMVg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=R9CyFc89EI4mmm0TQEqBL19rGkf/JvNABOb3GOn/r5A=;
 b=sdIvNrp4+X4ATfxFezfasklSsId+ofFvV+w4gMNmyFAO97GLn4xYN1pd4WGSO8gZ1IxxawE+FwhLiASoLe6g9EZ0brBQT52JWmeDCL3WcidgAQFm77aBxpHHMvC2yl6cgFlnu5NwzXcVCXhlDJVaMLO3x+U15oluBpKA9MILXd8=
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH 2/2] xen: reset legacy rtc flag for PV domU
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
        stable@vger.kernel.org
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-3-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <b2cd2936-7ec4-e0b2-458e-51c12a3f56aa@oracle.com>
Date: Wed, 8 Sep 2021 10:54:07 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210903084937.19392-3-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN7P220CA0014.NAMP220.PROD.OUTLOOK.COM
 (2603:10b6:806:123::19) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9c67d3e7-3179-4284-7f7a-08d972d88477
X-MS-TrafficTypeDiagnostic: BL0PR10MB3444:
X-Microsoft-Antispam-PRVS: 
	<BL0PR10MB3444ED12E1A5C04A32E24A178AD49@BL0PR10MB3444.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:923;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	Gl7oYKj8lVelmHxOmcoOcSFbYLyhAOlcd8nxrPtF4ibotx8wNSjAEZ/6sqKlp+5XBPBTekMBa45MlcBZyTzPxoIr6BP+ObepOgRZFpm/fZOE/RbReOqzM2WgQ/amW2c+J2RXhNQgTlbuHCPinG7MYr6Bm8r1r56pB1L2/BQeGLqLwc0pvfuMTRxC2P75V4umdKSIBKbTtt7slnn21zy/SJ73M3owM6iyeeI334xKZ5JBbOXu9dNhyK0YYwZJx+7tkr34GgPDLC8p9Br76SeOEHEyvnf9IVqRpw97Di9dEa46iTrrLuN7Pi4LdjR8WfjEECQOabNPVB4mbErdOHZDDJbCK/J3+hTDTwiHSGJ+BK3IOfTZnpkkurQM7iHj2iqcJNqejxFSRuJGeqdVf3Qt7Cw19b9xVKcNlIxkr4f5DG6Dt30vnLb+ZeAKeiaMNVwSArHfaPCQFrJcow4lNyeuMZNvK9NX66seRn5q5smuMal5u9QE4iQaKg9op5Bje8/m9nMEZl9R0tyagSmEMxsCzcctC1+a3yJhfhkz0ykvSsGcBZicvXh+KwVmvH+28RuoZVG6z0b6CEOtidfI6Oh9CzPw8P3cxIgv+o/w0OTz4w2082jSrLjPYUeda0JksCSvaogmaLPSLjsAeT111munmDVPAHEYPLmaz8V5k/VZZvBcWJq8sAzB4uTa/MjtR6U6Ius69aMTbukBPY8buHvo8YG33dPmE/FUfs57q+7hAak=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(45080400002)(66476007)(6486002)(54906003)(31686004)(44832011)(66556008)(16576012)(8936002)(83380400001)(26005)(2906002)(31696002)(8676002)(186003)(2616005)(4326008)(38100700002)(86362001)(66946007)(36756003)(7416002)(508600001)(5660300002)(53546011)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?NmJ3WmE2ZjVaUE15TUFiVFFQV2dvUTFiY3RIc3M2RGNlTEcxNlBjMDdwenkv?=
 =?utf-8?B?aXl6enFldUJ3YTVXRWVCVFo1aUNCNzNuU1Y0ZFBqakxoRWVETG9paEFKdGZU?=
 =?utf-8?B?QUF3T1F2SVNKVy81YnlLSEV4SG9OK1U0czZBYlQxS3hKNElZTlNhdTVORVMv?=
 =?utf-8?B?ZDJpVjdaamJWL1F3R0VmdzRJQVVjeno2K2Y4bzVuZ0lRcXJQajB5dUw1MktW?=
 =?utf-8?B?ZkxRenc1TlVtRGc1RWk3TTFXS2c2L0hTVC82MDRnR3hjTG5tVFI4ZndjeWx2?=
 =?utf-8?B?Z0VGV1IveGVRYnZrL1R1UzIwb1MzQ0FVMTh4QldaNHhzdlh5dnh5Si81OVk1?=
 =?utf-8?B?Zk91NzIxVkd0VC9GTXMveXNxQjZ2cVdTeGpiTWxweFF0REZ5MjNXWmEvNUZ1?=
 =?utf-8?B?VHNMZjdMOUs1Uk12MDVBUTZrRnRRQ2F1R0diS1g3Y2lXMk1wV0Nta1ZtNkJW?=
 =?utf-8?B?VlJ6cEFiaGppdWFHcWNHbWlxTlhnUk1EZm8vYWh6eVlEcFFVZTJKUis3RlZn?=
 =?utf-8?B?K3gvbkdNYUV6MWFEcWZtdlRsRks0dTgzejBncGg5WC80VkJ4bm82WHFrc1BK?=
 =?utf-8?B?bkdRSDZaWkdScVJSeVAyL3lmOUpMYmlwVnI0Q2V0YjdGdGhnMzFnc2YvQ0FD?=
 =?utf-8?B?ODNmQlJFWjI1Wm5MeDhnaWxMZzltY1UrVVNWQVpLMnh5akN1K3gwZFU4ZmZI?=
 =?utf-8?B?QWlIRWt4amFyU1dXd0pRRjFIb2E4TXd4ZHl4NGlaQXpOTCt0bysrTk1LVzBG?=
 =?utf-8?B?S3FSTWN1QW1wWWZTWmFOVGhpbzFxejYzOHp3U2pLS3g3MUZHejlGTzllWFky?=
 =?utf-8?B?ekozKzhBcllQUzhzT29EeGM1YTdTdlhRSVFUR3FwZm9BQ2JicUxQbm9xa05R?=
 =?utf-8?B?dGZreElZWVBkY1BiRGZ6RC9zb2dORjlwYmVHc1R6N1RId3pMVVpYL1ZmNEdW?=
 =?utf-8?B?eFlEZFJ5ZnpNaXJPMUc2VUkwUUxaZWZZM1dYcjgwcU5GeUFWM3hwOTVsTFJh?=
 =?utf-8?B?dzZSU1NwdlNPbWFxZWJCTkl2anBMV2FueGxSUnREZWdqM1VUQXFUWXVBdzBW?=
 =?utf-8?B?ZU56bnJjT0dtajlmazRuSXlXQ2NWMkhWQk9lVU10eDV6T05OdlREN284Q0VR?=
 =?utf-8?B?b2NKNWtFMFFUc2FiYVRDa2ZwMmgzaVM5WDNaYmhlc3BNQTlJam1QTFc1TEdB?=
 =?utf-8?B?UE5ZODI2Qy9WeVNBM1dOTEJwZ01XUDVrakM3TGhXVzlTYlViOE11RjV0SjRY?=
 =?utf-8?B?ZGdsTHpPTEx3QXl6dEdkRWNQcUl6VXNBRG1reUtKVHZpRWlKek5iNmJFN0dV?=
 =?utf-8?B?bngySEZQVVVzNmw2UnpkSFg2RUVCejEzSDdPazBPQTdpS25zVkV3bzRMU1RF?=
 =?utf-8?B?NjRlbGpSUWVOcG82dXJMYWQrT01UYTBPbDlDaDFVcHhCWXhmZFlES2tlWUVC?=
 =?utf-8?B?YkYxTjl3QmlLLzBLaHFRbGxFc0g3d1NwS1RMR3RWZWlEb2VmelRCZlVydjBo?=
 =?utf-8?B?eDEwR29jQUZWZHR5NmttdE5UVDRIUEk2bWNSREEwWFRGeWFyOEloczdNNUpp?=
 =?utf-8?B?NXY4VVl6WnJwbytvRzdTLzlzcENiZU5xRkhqS3pRUXRLeGhoVXZyaVRTK1Ux?=
 =?utf-8?B?dnpLS1BBQkNvOWdrVnNDd295T1VXcnJ0WEd1VXNHQWZVc3E2TGtibjFpclY1?=
 =?utf-8?B?NnJXNmI1bFVmU29VREl1aUdpSWRBNnp3endIN0RJNU8yN1FoLyswS2xJdWVK?=
 =?utf-8?Q?RolcjRs08WzEtLzjiX//bpizM7Ep4Jk4NIYKF2y?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c67d3e7-3179-4284-7f7a-08d972d88477
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 14:54:11.2224
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bRLXF+EPkNc+8N4+wrckUmzNzw6wmUKX47qx1iM1qj71KHEXpzkBfkpVqixhoI4FZdeCcSKvzyMkI1AsP0rVnUWo1EHuAJarqO9Fle6pkXQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB3444
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10101 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0
 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080093
X-Proofpoint-GUID: 7U7S6iHGs1PIrquqq2ApWZHCvSLEFWlN
X-Proofpoint-ORIG-GUID: 7U7S6iHGs1PIrquqq2ApWZHCvSLEFWlN


On 9/3/21 4:49 AM, Juergen Gross wrote:
> A Xen PV guest doesn't have a legacy RTC device, so reset the legacy
> RTC flag. Otherwise the following WARN splat will occur at boot:
>
> [    1.333404] WARNING: CPU: 1 PID: 1 at /home/gross/linux/head/drivers/rtc/rtc-mc146818-lib.c:25 mc146818_get_time+0x1be/0x210
> [    1.333404] Modules linked in:
> [    1.333404] CPU: 1 PID: 1 Comm: swapper/0 Tainted: G        W         5.14.0-rc7-default+ #282
> [    1.333404] RIP: e030:mc146818_get_time+0x1be/0x210
> [    1.333404] Code: c0 64 01 c5 83 fd 45 89 6b 14 7f 06 83 c5 64 89 6b 14 41 83 ec 01 b8 02 00 00 00 44 89 63 10 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b 48 c7 c7 30 0e ef 82 4c 89 e6 e8 71 2a 24 00 48 c7 c0 ff ff
> [    1.333404] RSP: e02b:ffffc90040093df8 EFLAGS: 00010002
> [    1.333404] RAX: 00000000000000ff RBX: ffffc90040093e34 RCX: 0000000000000000
> [    1.333404] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 000000000000000d
> [    1.333404] RBP: ffffffff82ef0e30 R08: ffff888005013e60 R09: 0000000000000000
> [    1.333404] R10: ffffffff82373e9b R11: 0000000000033080 R12: 0000000000000200
> [    1.333404] R13: 0000000000000000 R14: 0000000000000002 R15: ffffffff82cdc6d4
> [    1.333404] FS:  0000000000000000(0000) GS:ffff88807d440000(0000) knlGS:0000000000000000
> [    1.333404] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    1.333404] CR2: 0000000000000000 CR3: 000000000260a000 CR4: 0000000000050660
> [    1.333404] Call Trace:
> [    1.333404]  ? wakeup_sources_sysfs_init+0x30/0x30
> [    1.333404]  ? rdinit_setup+0x2b/0x2b
> [    1.333404]  early_resume_init+0x23/0xa4
> [    1.333404]  ? cn_proc_init+0x36/0x36
> [    1.333404]  do_one_initcall+0x3e/0x200
> [    1.333404]  kernel_init_freeable+0x232/0x28e
> [    1.333404]  ? rest_init+0xd0/0xd0
> [    1.333404]  kernel_init+0x16/0x120
> [    1.333404]  ret_from_fork+0x1f/0x30
>
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Juergen Gross <jgross@suse.com>


Fixes: 211e5db19d15 ("rtc: mc146818: Detect and handle broken RTCs") ?


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:01:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:01:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182195.329658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNz4E-0007gV-WA; Wed, 08 Sep 2021 15:00:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182195.329658; Wed, 08 Sep 2021 15:00:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNz4E-0007gO-QQ; Wed, 08 Sep 2021 15:00:50 +0000
Received: by outflank-mailman (input) for mailman id 182195;
 Wed, 08 Sep 2021 15:00:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNz4E-0007gI-5i
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:00:50 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 24fc7253-5a5b-4127-9c37-88fe82a9b5f3;
 Wed, 08 Sep 2021 15:00:49 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-by4afzcJOgu_iYBmySoebQ-1; Wed, 08 Sep 2021 17:00:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 15:00:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 15:00:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0004.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 15:00:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24fc7253-5a5b-4127-9c37-88fe82a9b5f3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631113248;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sZVR7O+OObN/g1PdMt9hlvKFvOQM8eMajPFVhoANg4w=;
	b=SFz3jopk2l7SW6BWOFWWues8urYKQpVGMyJc9+SoT1wS/wY8VYO0J27NsiLD8JJGkXY/g0
	nqlbXJ/n9DudmkKRxBJ2QOLqx1sKJimYsIozC/27hqA2+BVcrig32LV43OPW4dK9TAGxe2
	qJSLyWLa9gujBVI0gvBmx1CLqnrM3bM=
X-MC-Unique: by4afzcJOgu_iYBmySoebQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MpW6YhlN5dULv583Is0DuYwbX/id5+W8fPkm0FesN5yrV7bafHvbubpAmnJDsgPhLx3KmOu+FmK4aWc9GhPtSpwk0zHvoQqruTtKLIOZHnnQa2jqCsAn+jnbqpbdEAWvYvSdZ5onoJ3N/dKTdN3FYTfT4L4uRBpDVAYiULcwgc+ASSUaoxxWLJmnIrtCtes/BnrlY65/N9f6MtQHynH11GWxwAt/y7xyumIi//cZXA7s1j9afJewkTvuwEFLQSaOT9BYpdbCzHEG6sUVWK2U64ewBJ0Vt+YxJGKzqE3R1typNnq/PaGW3XJ8T9uvE+Y/zHJYjpJCc/9gsyCAfYY6Ow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ocepLHku+35R11tzlchjRw8Kfo4yHRPs8pdkaxCwP0I=;
 b=BAxyavjBCH9P0peYCJjVbdH9VAXpJTTUND05+oq0kppQwMsSK72h7zobWvJkKoMJX45ZlGXYJvJee2d7TEyjX2jpA+kCu8DjJtuisgC9k1jpTus1kIEPUjoP6YJZ2PWiqY5UPkvKt5Iovr3t3pbtxJLFaQ3qStTq10wLcv1OOUDlVt6FLIipnyIWVjkl300bsrZndjTVUYUFYss3G2II/t/7njCDntWOAnSCcrXjo9x4ySBLXK9Auz9a+hog5y3QZkJKLkkBhDbP4jrCOU0bpgYK4Uy8m/GQPFp1gy+a7qjhaSlVmkUCArsDLYMmpai2kJ3jgD+cQUSq8uAEYRelpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
Date: Wed, 8 Sep 2021 17:00:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR1P264CA0004.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fcd64fc5-276c-41f3-bdd4-08d972d96f12
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68620724EF963914135B0694B3D49@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7mr8rUIcg7Ga+mQjHd/7VD0NMwUi0hLNXPLicl7quU+CDsB1/UnrXeujDDPl/Px7uTqs73BLarQasNsjvwWm+qsUqdLDnlMwKVDLioXm+UG4Ztn6K3D4vUyJB7uBypK4U/FOISfWVQEgFLqgGCc2kfOqOecol0TcYzZFhsRZduDoeriuB1SPCZLN/kZJTcPZkT9gE31qxYJkyd5sh/YsRu7czi+Jd+TP+MxzoJUYcL9CTfDFqCSj+TErECW2RsSQUbuSMKZad1ykhnUtblYm/Cw+u1L9AJW7XiOMbBmFdCHlC0+6LMwcpJu21hSI3AStL0a2Iyv9OmYda1rw9WHQ3rP6UDlyei9HdE1uIMnDbjdeGyRLenyvKRhykHAWuB18A1oZnPbVWyWePt/YJsC7iXMKiEhokUrqjm7RqJT4D8i2hCMOSlLXzDbqQvaWP0j1+oltP+xRSMLtGBazBwAo+kFKjSDc4rasG6Frm3McpG/QYhrnXGD5XRno5qyD1pKDVrkwrXbv1G8dbciDgqJTLdmeimc7+jaKErEajFb9MAdn820lnkw3rbD+NU30R5fikDq0JT/rYf1oke2vGTWRHhcJ+wsN2LSCqJGY2RPg6Pv8++F8EXmoEOtS1LIuSaUHQJXBEyn5JtLPEWMhGVUu3VumlcRGPvNaeey74x5hHgZky+FsmOUYk1Qte5umVv/kHZUyBu6BJcZWG/FgyS12KOy36XD9Re+gnpe3yU4hjFwtgyz+iDzjHGi3XUgK/t4J
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(136003)(396003)(346002)(31696002)(8936002)(53546011)(83380400001)(316002)(110136005)(956004)(54906003)(16576012)(5660300002)(2906002)(6486002)(66476007)(7416002)(66946007)(66556008)(31686004)(36756003)(2616005)(186003)(26005)(4326008)(8676002)(86362001)(478600001)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?WWCYzLbCjPaWGeRaW7VmbtCa4SnOYDScMcIf5+W/shhgBygq6PvBPSg1F0O2?=
 =?us-ascii?Q?b9cl/W92ZTHgkweLH8aEwPThZ8QP5akTfS8c37kOMMCTpnF7m8qt8zknIEz+?=
 =?us-ascii?Q?R+2X7lJ2zuiYWZfe4RoWA86x+MCzYWzBwwXlE7ygz0W8SATPnKCbV4eg9pLu?=
 =?us-ascii?Q?mbemKYtFeko4vuQNWke1Ox/wRWXgc9xknMHD9tOPTRQLYbiSDM6XSTmK2RWL?=
 =?us-ascii?Q?erYER/pQA00kZ94eSsndtp4IYO3jTYlKFBgjqtssXKjNyJfAABjDm0E5z5Pt?=
 =?us-ascii?Q?+XBmOl3ukK3DElFtSoVFs6XyEqyDkxDVEHKZdQJRvcO1HIfMpPOvzkBAn7H0?=
 =?us-ascii?Q?+SRAhGW2/3cmPBjVKcPqcC0uwzdWofuUQczzuFZseIpoaLoy9jk96z+7T2ER?=
 =?us-ascii?Q?ihBhL66emvkGKgANFs9aT9w9SwirasLzalJae5+9SQjyPSJsDioZibG9lVZi?=
 =?us-ascii?Q?mbWC+BE4ccbVrEfHYPWex/d9uNqINBTsyMLTqXFmaO/t4iE+cDrq+WVS2rKu?=
 =?us-ascii?Q?MUtBhX5+/cG4Kp5fvdPVqeciUCVkPs2t28GAf/1J9CBYw5LFiV3/04IHyxtZ?=
 =?us-ascii?Q?4m4fvDNlO53Wmx8lCgVBARYpM5VEKuYEBpofplN6nTpTvFJm0nV5ujVpjvUy?=
 =?us-ascii?Q?XMnZ/AghGGi6JzLMORTrfy3FVPHe8ZzPTG124hPzTSEuauZzRqX3fdBb0b3a?=
 =?us-ascii?Q?WaKlJ6Ii/v/EFxrIw25+tSmt6vBh/ACAACqTJPkTwMOA/Vk7rUSb4HYnMH30?=
 =?us-ascii?Q?xvhaeKFzDFojP+QM4jjpdd96vUdwle/n7XtAYcxqOPByLui+GcboxSe8oLCz?=
 =?us-ascii?Q?eINg4NZl4Qc94TwdDF6mRW+wfv/frOnbbVRjhBsY58P9GGmvW8u9ZheoIk24?=
 =?us-ascii?Q?dN0oelsNMXnyV2IzN74NPBj7Ubi7pEsBJ1IC47ge8xegZWg5XTuI84ZfqdrB?=
 =?us-ascii?Q?tNpg5RBEKpiw3lXhWnbpsFr0WknfSsqYchWbPjrf2oD7FgUvF82fnkF21sIa?=
 =?us-ascii?Q?EEpzn9rucSBVME3cqWww41bJ1SOvQFp6Kn2STwpMe9L5R+MtQG1tbIBkBxGZ?=
 =?us-ascii?Q?e9wN20wCqraVX/VepZaIeV3V2sqtMTDIoRXg2Vh2D2HJGvTzzEyxtyZyh1oM?=
 =?us-ascii?Q?O6rR+OqC/gfuLGDel4RNRUIryTapeehGV35FlnN8pW+p2/nNhpoQ2hfm1Bp+?=
 =?us-ascii?Q?tcj7/WLfgzAbl7Xoqw8LjqgnYM8LB6aRvTKLsL4IX1qXyQs1zYdqEhi+DCIH?=
 =?us-ascii?Q?cUPx7ROT7yYDTo7D3Be40CE8H3RVAI2Z8TawFLM3DMKCs/EDRUp3wVzAQutH?=
 =?us-ascii?Q?g69GKW8AIayft0Bq+W33pGEJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fcd64fc5-276c-41f3-bdd4-08d972d96f12
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 15:00:44.7714
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0AbORJILUFe4ynWKoS5CUpdeQT7leT7TkwOa/VXVCkqUIEWhiCcQ49ueqy0DV9B4wRYSK7384YLMzGreCVW3/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

On 08.09.2021 16:31, Oleksandr Andrushchenko wrote:
>=20
> On 06.09.21 17:47, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> Instead of handling a single range set, that contains all the memory
>>> regions of all the BARs and ROM, have them per BAR.
>> Without looking at how you carry out this change - this look wrong (as
>> in: wasteful) to me. Despite ...
>>
>>> This is in preparation of making non-identity mappings in p2m for the
>>> MMIOs/ROM.
>> ... the need for this, every individual BAR is still contiguous in both
>> host and guest address spaces, so can be represented as a single
>> (start,end) tuple (or a pair thereof, to account for both host and guest
>> values). No need to use a rangeset for this.
>=20
> First of all this change is in preparation for non-identity mappings,

I'm afraid I continue to not see how this matters in the discussion at
hand. I'm fully aware that this is the goal.

> e.g. currently we collect all the memory ranges which require mappings
> into a single range set, then we cut off MSI-X regions and then use range=
 set
> functionality to call a callback for every memory range left after MSI-X.
> This works perfectly fine for 1:1 mappings, e.g. what we have as the rang=
e
> set's starting address is what we want to be mapped/unmapped.
> Why range sets? Because they allow partial mappings, e.g. you can map par=
t of
> the range and return back and continue from where you stopped. And if I
> understand that correctly that was the initial intention of introducing r=
ange sets here.
>=20
> For non-identity mappings this becomes not that easy. Each individual BAR=
 may be
> mapped differently according to what guest OS has programmed as bar->gues=
t_addr
> (guest view of the BAR start).

I don't see how the rangeset helps here. You have a guest and a host pair
of values for every BAR. Pages with e.g. the MSI-X table may not be mapped
to their host counterpart address, yes, but you need to special cases
these anyway: Accesses to them need to be handled. Hence I'm having a hard
time seeing how a per-BAR rangeset (which will cover at most three distinct
ranges afaict, which is way too little for this kind of data organization
imo) can gain you all this much.

Overall the 6 BARs of a device will cover up to 8 non-adjacent ranges. IOW
the majority (4 or more) of the rangesets will indeed merely represent a
plain (start,end) pair (or be entirely empty).

> Thus we need to collect all those non-identity mappings
> per BAR now (so we have a mapping "guest view" : "physical BAR" and again=
 cut off
> MSI-X regions as before.=C2=A0 So, yes, it may be a bit wasteful to use m=
any range sets,
> but makes vPCI life much-much easier.

Which I'm yet to be convinced of. Then again I'm not the maintainer of
this code, so if you can convince Roger you'll be all good.

> Thus, I think that even per-BAR range sets are
> good to go as they have more pros than cons. IMO
> Even if we go with "can be represented as a single (start,end) tuple" it =
doesn't answer
> the question what needs to be done if a range gets partially mapped/unmap=
ped.

This question also isn't answered when you use rangesets.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:07:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:07:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182204.329668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzAH-0000Hc-O3; Wed, 08 Sep 2021 15:07:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182204.329668; Wed, 08 Sep 2021 15:07:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzAH-0000HV-JL; Wed, 08 Sep 2021 15:07:05 +0000
Received: by outflank-mailman (input) for mailman id 182204;
 Wed, 08 Sep 2021 15:07:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNzAG-0000HO-Al
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:07:04 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6b44e665-10b6-11ec-b165-12813bfff9fa;
 Wed, 08 Sep 2021 15:07:03 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-MG6RLw6aOzqsZi2-xEwclw-1;
 Wed, 08 Sep 2021 17:07:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5471.eurprd04.prod.outlook.com (2603:10a6:803:d0::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Wed, 8 Sep
 2021 15:07:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 15:07:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0022.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:17c::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 15:06:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b44e665-10b6-11ec-b165-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631113622;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Wzbd6DFR/SngT3RP4KdhWBcf+ymOUKJls4cDe7LsDGU=;
	b=N2wcSHX7Re3VuioILRzhvdKK1yeTcmgpt15HdYT3MNnu7AxBWo6aSsdRl44mm9/ab9P69b
	DvbJPHtXG3VCHPyWpr396jJBUnjRhObd2VZSFN02fIRMe1o/xpHIBa0LWklD19QPmCdVoo
	hOeb31tiGwn4PJx5+VUaYCOPkt2fPRk=
X-MC-Unique: MG6RLw6aOzqsZi2-xEwclw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AWLp0uL7uz/CKhn+2MJkjQb5yNRzpDE+36HUOyAj7+aRosFdQIhr3Le1k4I3XoaL3E5W2te9t0NB6uiMRmMjH3SHtbU77nfwM+mlvD3K3lYGBHwvgklHZRu/5Tzsrg4EUXjaipyC3f0CvxEbVw/nUNVzvi6e1zEHr1wywrAVjv83E/x4rCjxBeckZ47BxF7BLwi2CxLq4CFo0E1M1gwzh35iv7UTE+jolK520tTKjqIdHWLtIpwke80EarrzpbN8srzwtvG11GmGOrgCwACEFXkZ5yRH39MvpDVSDhTcECfbaNwL2J3ZP01w+WSv0NGdy1ehZ4R8FAlQj5910wEKsQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Wzbd6DFR/SngT3RP4KdhWBcf+ymOUKJls4cDe7LsDGU=;
 b=MrdDDubrQ4khSVypUwGaR76SfuMT+Qj0IVQoCPjQRJVPPEqwjdCr2+nG8ptvMpG1U5ysWtrEt+9sjgTe0vYkybC/1Of4zVE68DwJIMMsbwmdzVwUMJwjZ9ys96OtIVK7Oet5Q16jfSmMKF8jd+3S9+WlxLANT8gnO1Ih0H4AXTBD89vuOSHTpxTwWqle3/VR7RHkYc9eyXGxEPn/7LiVhHwrAMwYT1zuCK1p/NkTjnHtaOlxRvHhW/5Vk7890eEcxXD8/8MEavH1sa7sOLgyv42tMA+Z51z82JZwELTRETOuoZeTZw3xUKB7v6TPDHwmYS4lolSAGEN1d9icCyeBdQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/amd: Introduce and use X86_BUG_NULL_SEG
To: Andrew Cooper <andrew.cooper3@citrix.com>, Pu Wen <puwen@hygon.cn>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908135931.26275-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2289cadf-0dd6-0b5e-39f2-99ce6f522bfa@suse.com>
Date: Wed, 8 Sep 2021 17:06:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908135931.26275-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0022.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:17c::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b699a53-ba73-41f5-240d-08d972da4ecd
X-MS-TrafficTypeDiagnostic: VI1PR04MB5471:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB54719D64D0A4BB19DE03B0AFB3D49@VI1PR04MB5471.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2rFQQeZL1888TeqckB3sHYZp0+jhLV9ctvmFFg7O5xaDy+YeyW6ydS138piziMvXcHrgr1V8l1P9IbG3conEQHSL0mFnED605MpNovXubnitioTlo8/u5w63IqifVZdG89ykJOez3J2NM7bfPocPDf5Ea31CMtSBVs/yNAJJjEXh+IE5JcBZkVGctbr9ghRBy1O+fBIa8eWwM0yA73g7hjdFrNjHB7EEdDpwJOtF5WIE0PHO4r0IAckD88U+r66fqF3fu2YZu65cHzJFCB5qtpKNGDMWG8XgvTpu2Qw08K75ps+JXw2/RsQfxxBTXX7S4G1dp7qdI9g6cOStkw6iExEnJyJYF312adHk/1J9exmI/XS9w445I8puIfvkSBb9XsnknQ1q77JNzXAv7HArrOKmT2xqFWEKBiQ8hYmys0TaaHgyBoUC8TTsyx4zYIq6ZdU99tkFYcVrkE27aKwq2AEsbFjM6sr5PIq7KVloh+2VC1jPU6e9PxS10ONLs1/N5TKMpvIpi2GmVvr4a/dw+Zbg7dhPw0x3TAY4mxSqWEZol6x6cLWBZ2GcMjNw3HllhCBD2ECBS3QsWwgvOvSJDFohKONzuPXtDkCYEANb4Yy5/7nb7sjflgyyRe+PpyWZYDNmkzqWPUz21QjOzqAFCPWWrdz+Pc/EMiUJBTzX/BICWyIQDf7dBbN829FFFpD1wfnGAFqKxn0gHts8tUrPO0Ecn+10L0o/WWPo8d1Vf/sW3vPmE6jJp+oE3SITX+HL
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(396003)(366004)(39860400002)(376002)(53546011)(26005)(16576012)(2906002)(66556008)(66946007)(186003)(36756003)(31696002)(4326008)(316002)(54906003)(86362001)(2616005)(8676002)(956004)(8936002)(66476007)(478600001)(110136005)(5660300002)(31686004)(38100700002)(4744005)(6486002)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SDBZY2pUbXVzZkxLWktGRmkrVWtSUXRrQXgzVG8zS09vc3Rha2E3VFB2Zks5?=
 =?utf-8?B?bHVCbFJobHlmNng4Z1kzUGFENktBQmkyQ1VCUTlIaE9ERkJyRHlqTm5hK3NB?=
 =?utf-8?B?VXBuNjdYUFU4dVJSbXJuY2Fsa3pyaFYrR2ljazNHVy9PUkcvUm1PcU9FblV3?=
 =?utf-8?B?K0ljMWZENnpZeFpRTWFSTG9zMlh4RFJZcE13SFpuOWtySThHdm55a1JDZllq?=
 =?utf-8?B?dUd0RUlyZHd6bUVXWVB4ZUZ2ak9EWEZuNGRkNGdTQjg2Vk1TUnltUTQzWGpM?=
 =?utf-8?B?OFVRVnNURW1zYmxpRTRQR2xmcWp1bU9UU0hSYUpjR3UxeTg2NEovb3pxRWQw?=
 =?utf-8?B?Nnh2a1pRUlNHZUxnQjhtNnNGbHlKdThXWGY2VXRpd0RaTnp5WWJHeUc2MmdJ?=
 =?utf-8?B?R1d6aGRhZzE1dkQycE5iaEhqeVA5S3VrTG9KV3NzLzFwUThFZHVtUmRDcTNt?=
 =?utf-8?B?SGZtRHA3ZnFEdXRhc1FRTHBBT3JyZEx6ZWZZWUEwdlk4VjdlWWl4L0Zicjhw?=
 =?utf-8?B?NnJGanQzZVMyOWJqaU5Bei9ndERETXlzWGZHMG1QbnNRSFZRVGZ0dE9GaDIx?=
 =?utf-8?B?Vk92RnRJc05pWHhqclFid3M2eGZPNE44aFlyY3ZEdDA5UUxvY1EwU3RnZ01n?=
 =?utf-8?B?aSsza3lrOFlmaUdSZUphYW96NnBiMzFseThMMzF6d0ZQNTI5UndRL1RETVFL?=
 =?utf-8?B?djh0bVVabW1RMUc0NEsvMDNnNlAyemN0UkFDL2JPbzI0UDR3NHFtMlhubjF2?=
 =?utf-8?B?NUFUL2xNZ3NCZS8xa1pXL2s3cXl0UnA5b1BWQ0o2K3lqd2JSQ1dGdGJ0Mjlp?=
 =?utf-8?B?K1ozay9uVldVNjlKdzlFU1MvdEMwdkFMMUtObHV4QXRXT3o5aVFra1Y1TUNa?=
 =?utf-8?B?OFY3dldjLzdwanZmeWYxYy9DZThGK1IrQ0VRYittWEtNaFNSZ2Y1aXZnbkdP?=
 =?utf-8?B?eE4rakNyeVZwTXQ3UWFmbE96WldDeEJ2Rk9MdE84Wk81YnEzUDJmQ0xvTmtm?=
 =?utf-8?B?R0g5bUZTY0RxYmVHSmc5b2VlZ2pTWElSRjZoNnJQd2Q1VnZGbEFXcW5BSHRu?=
 =?utf-8?B?VzNUc2lRY1BiVDhNYXVEY3FYd2F2aHk5OFJZVzdrZDBEWHdrWE04akU2bTB2?=
 =?utf-8?B?cWNNYmJad2N4NEE0K09nM2t6S3pGMS9wUVVUaHdubkgrcGVNRHFabXg4YnU3?=
 =?utf-8?B?RWRtalRSZ2FMYlVQeXJzWmxzaHc3T1JnU0VzVkdmUFRIaW1kQVhhREkxSDNP?=
 =?utf-8?B?RjYwLzBWcnhUSVlFd2JiQUROb3FrUXVrZzFrc1pOd0hQZnVIL3dJTW1BdVYr?=
 =?utf-8?B?NzVOQnFiRTU2YTN6MCtnbCt5bDJKK2VXN29HV2dYbE9pUWRuc0JhVC80TW5p?=
 =?utf-8?B?Mk1ETWNOYmkxbHlxMzhJd2ZvckdiUlZha1pDdFRTNlYrOHo5bWxveTg2L3pz?=
 =?utf-8?B?MHE4MkQ5SEs0Mk1laW5xQ3UxVUkxdER6blMrVkVZZHZ3R0hUK2JhVGdlaGN6?=
 =?utf-8?B?eVp0NENQWWlYckVjR241NWxRc2EwMEJtUU5iVGhSQlVmcHM0b3RqNmNxSlhM?=
 =?utf-8?B?c3FsRVpZR01UaDRPOVFVOHJsbDRtR2JXNnExbzhSaWhlSi9PcDV6eGFDNjlo?=
 =?utf-8?B?byt3Q2JQRnI2M1k0UklCWXNXUU5JaHVOSHdhVlFjVWZVbmhlb1ltcm9mSmh0?=
 =?utf-8?B?WCtSQmtMK29Ld3gxZ0dOQUVrR3VSbVhvYkp1RldBY3FRTkFMRktCV0xVbXVC?=
 =?utf-8?Q?lflWO6UA0tEcOW2j/vEtWe3elbw/GZFeW0iKPhF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b699a53-ba73-41f5-240d-08d972da4ecd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 15:07:00.0819
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AIaU1nv4XwqpFyM2wP/xRbJyUSeWRi55aGtH/s/okYCT5A4+xFsROMKGZdy5r9WLHccxpMvhSJdJoRYZtG5E7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5471

On 08.09.2021 15:59, Andrew Cooper wrote:
> AMD/Hygon processors before the Zen2 microarchitecture don't clear the base or
> limit fields when loading a NULL segment.
> 
> Express the logic in terms of cpu_bug_null_seg, and adjust the workaround in
> do_set_segment_base().
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit with a question (perhaps to Pu):

> --- a/xen/arch/x86/cpu/hygon.c
> +++ b/xen/arch/x86/cpu/hygon.c
> @@ -40,6 +40,13 @@ static void init_hygon(struct cpuinfo_x86 *c)
>  	    c->x86 == 0x18)
>  		detect_zen2_null_seg_behaviour();
>  
> +	/*
> +	 * Hygon CPUs before Zen2 don't clear segment bases/limits when
> +	 * loading a NULL selector.
> +	 */
> +        if (c == &boot_cpu_data && !cpu_has_nscb)
> +		setup_force_cpu_cap(X86_BUG_NULL_SEG);

Are there any affected CPUs in reality?

And actually (only noticing when seeing the malformed context
above) there look to be indentation issues in the changes to
both this file and amd.c.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:11:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:11:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182211.329679 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzEj-0001i8-AQ; Wed, 08 Sep 2021 15:11:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182211.329679; Wed, 08 Sep 2021 15:11:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzEj-0001i1-6X; Wed, 08 Sep 2021 15:11:41 +0000
Received: by outflank-mailman (input) for mailman id 182211;
 Wed, 08 Sep 2021 15:11:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNzEh-0001hv-8v
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:11:39 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f766ff67-992c-4f03-b9ba-e4f476c1f390;
 Wed, 08 Sep 2021 15:11:38 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 89C1520166;
 Wed,  8 Sep 2021 15:11:37 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 5C43013A9C;
 Wed,  8 Sep 2021 15:11:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id DhjbFKnSOGHXPAAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 15:11:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f766ff67-992c-4f03-b9ba-e4f476c1f390
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631113897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=svmGTRjN6JH/kF3lOs1T+B7jEWqgdg10jT2MIlW7N3o=;
	b=t86y6dqiOlOPYgOEgsGOcXk768O/AZM9LSpVbEY9GQwfUHt4mto7AemKgZDJEEEasMxYdY
	A76WQct/8Zs0WZeLw7uL4rteh5SkcqYwJo8SEMRaBWCrcrgXo0AiU/VFedtQRWk3STGC8P
	ClKgcTsw0XLv2gP4f86KO6TKcx7aVmQ=
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Jan Beulich <jbeulich@suse.com>
References: <20210827123206.15429-1-jgross@suse.com>
 <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen/balloon: use a kernel thread instead a workqueue
Message-ID: <9d1edc4e-b2b5-c620-17c6-31d238c2c417@suse.com>
Date: Wed, 8 Sep 2021 17:11:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="jV7W9DT5QK8tnenkioUhL7o7Tsq4LqbAR"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--jV7W9DT5QK8tnenkioUhL7o7Tsq4LqbAR
Content-Type: multipart/mixed; boundary="UuDL2UNeUp3H7W26SmDKcsotYQMTcxYay";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Jan Beulich <jbeulich@suse.com>
Message-ID: <9d1edc4e-b2b5-c620-17c6-31d238c2c417@suse.com>
Subject: Re: [PATCH] xen/balloon: use a kernel thread instead a workqueue
References: <20210827123206.15429-1-jgross@suse.com>
 <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>
In-Reply-To: <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>

--UuDL2UNeUp3H7W26SmDKcsotYQMTcxYay
Content-Type: multipart/mixed;
 boundary="------------2BAC314933443E7B316CE07F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2BAC314933443E7B316CE07F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 08.09.21 16:47, Boris Ostrovsky wrote:
>=20
> On 8/27/21 8:32 AM, Juergen Gross wrote:
>> +static bool balloon_thread_cond(enum bp_state state, long credit)
>> +{
>> +	if (state !=3D BP_EAGAIN)
>> +		credit =3D 0;
>> +
>> +	return current_credit() !=3D credit || kthread_should_stop();
>> +}
>> +
>> +/*
>> + * As this is a kthread it is guaranteed to run as a single instance =
only.
>>    * We may of course race updates of the target counts (which are pro=
tected
>>    * by the balloon lock), or with changes to the Xen hard limit, but =
we will
>>    * recover from these in time.
>>    */
>> -static void balloon_process(struct work_struct *work)
>> +static int balloon_thread(void *unused)
>>   {
>>   	enum bp_state state =3D BP_DONE;
>>   	long credit;
>> +	unsigned long timeout;
>> +
>> +	set_freezable();
>> +	for (;;) {
>> +		if (state =3D=3D BP_EAGAIN)
>> +			timeout =3D balloon_stats.schedule_delay * HZ;
>> +		else
>> +			timeout =3D 3600 * HZ;
>> +		credit =3D current_credit();
>>  =20
>> +		wait_event_interruptible_timeout(balloon_thread_wq,
>> +				 balloon_thread_cond(state, credit), timeout);
>=20
>=20
> Given that wait_event_interruptible_timeout() is a bunch of nested macr=
os do we need to worry here about overly aggressive compiler optimizing o=
ut 'credit =3D current_credit()'?

I don't think so. current_credit() is reading from balloon_stats, which
is a global variable. So the compiler shouldn't assume the contents
won't change.

But I can add a barrier() after 'credit =3D current_credit()' in case
you'd feel uneasy without it.


Juergen

--------------2BAC314933443E7B316CE07F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2BAC314933443E7B316CE07F--

--UuDL2UNeUp3H7W26SmDKcsotYQMTcxYay--

--jV7W9DT5QK8tnenkioUhL7o7Tsq4LqbAR
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE40qgFAwAAAAAACgkQsN6d1ii/Ey9I
mAf/e2A3onHFreFek+dmRlccdjx/ynL2gG2nJMUl0V6fDmDmTSkYX5e0aKnhF0om9yBWYBooKDko
QM2WJ0N+qO2F1Ej0pS9FoF2ImxrL+sM2/NjsteOXO6S7GJ8HSohqXSWww1vt6i8DkX+ff87diU8e
Mi/3vnEUcKZ30h6C2LRyUXxxBDLQ+Wox9KdRPHj6J0DYNq3x6DdFW9svwk860ttfAp+YpFzHy94q
RHBumUpgx3b70kQba8q+kLdj+q6lmqCc5Fp4PeNpOPSZSAtDzd/HuScCAzoBLqnfJ3ozlx9zLS5D
qSjFM3BRQooWoV+/ZW2vTiLuwAdWS9rrrx/LrClrNg==
=Pj2e
-----END PGP SIGNATURE-----

--jV7W9DT5QK8tnenkioUhL7o7Tsq4LqbAR--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:15:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:15:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182218.329689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzHt-0002bu-Ox; Wed, 08 Sep 2021 15:14:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182218.329689; Wed, 08 Sep 2021 15:14:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzHt-0002bn-Ln; Wed, 08 Sep 2021 15:14:57 +0000
Received: by outflank-mailman (input) for mailman id 182218;
 Wed, 08 Sep 2021 15:14:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0iEI=N6=epam.com=prvs=98850bb71f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNzHs-0002bh-PX
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:14:56 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f28711e2-70cd-4fdf-ab5a-038c19e4878d;
 Wed, 08 Sep 2021 15:14:55 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188EwNox030430; 
 Wed, 8 Sep 2021 15:14:51 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3axwna8mh3-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 08 Sep 2021 15:14:51 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB2766.eurprd03.prod.outlook.com (2603:10a6:800:dd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 15:14:47 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.015; Wed, 8 Sep 2021
 15:14:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f28711e2-70cd-4fdf-ab5a-038c19e4878d
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cezgYwNokEuP8IU1WrNp/SpeWHd0ADJdfhLuqqRHvEwFdGsda/MotUEgKYQhbCHzV9WdMbj4ns8W26X3dcvAM7CwN0naeOx+LUjHKHOg08uDyDFhIlFjSuEBHGoiY96Iq6OqDQwtqtf0AwqzdHhp+7mAxwRFRWKOxZ7t52T2Pxhlb5CqXFt/TGXMNe23i8P90bkstwq0b3DdCfK6bJkA04LQsUTdQA9e3AVUK9hYTThmftnuXVuSXqMy7r553X+MSJtRQWS8+gmJLHt+6m9fu3Q/VSaC6iY1pUIphEOVLMOgUz8i4TcvgNN12ocXOIrM3Xyo7CxvCRikL55dFaGJYA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jLpf7T2GS3Z0c/y8shA8fr/NAWe8QNcklPFtyAfEp40=;
 b=oWlz2kiRzhTOWEUfQhjRy6uvSd6kpe97o4a4n/oTx2B6j2BkjwbDCP2/ZuGehOKbS7sF2WCtsc38hnQI07k+jckRb9JkmEFBqyE6rf/J9BA7Gmy7PPeH2h4NwJHD5sS1NScyCAvjksfNrTs4zF9M4gSi6qY5HOQ99e0R0hH/7wiO5XZMzuaWkBtMGU1QuP+/Ezg2SmoC9kpjBtmoAeVg+fy4Iit0C0IgE5WkYZA5SaSVV6IB54g6HPeOGoMFMfuwDIeWIVTw20X4vi9snBQmuI+1HJRwTfgqzZ1HnNJQZ6LPBgFuNuUBOAo+jlhmqi1lM4DVT9y94s3/uUxOxk4Bhw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jLpf7T2GS3Z0c/y8shA8fr/NAWe8QNcklPFtyAfEp40=;
 b=qOS/UN7oN24JtK3M+3VYnw8oaOEjkl21qsL7fInFcRYqz18VxDo6kuQQDjhiEvcROs447j2QYKqe9WXpcFf9Lex+O37022l1cXuTCM/PZ0+q3frbmBIJsrjWzk15lxqrNRRJGzuhLiXqTrT+ptaOVkauFsQHNyvxevXL/XNDliFvi+3lBfTR68j+q8RAz6MbooA41fsxqJfUqct/HhXOAKC++ynKmdHwc+15qlYOGIf6VpcVLe0KWqw4ycmyOpLWcLbWi297u+Kcb/LkwVzSJ7wH0KJ33to/I8mY+Rvg/1jRsQqmM/bRXWhR03y0pYV0HWbk6HL7myShhtpjEV88Jg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: 
 AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLICAADFtgIAAE0qAgAEIooCAAAS6AIAABV4AgAA6yACAABRDgIAACAOA
Date: Wed, 8 Sep 2021 15:14:46 +0000
Message-ID: <d5ef78d1-253e-26cb-8d24-667d1ce7a671@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
 <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
 <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
 <2fd6b81e-e5fc-f6d8-b24f-9fbb1db034d7@suse.com>
In-Reply-To: <2fd6b81e-e5fc-f6d8-b24f-9fbb1db034d7@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2c051a72-ec4a-4d4e-09e1-08d972db6520
x-ms-traffictypediagnostic: VI1PR0302MB2766:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB2766DD7BDA0B8D54BD582CECE7D49@VI1PR0302MB2766.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 u0ZUfZslA5SnZCE4Up8jl4fNc1Enp+RVtT9Lmp7HPjOF3VSX8bDwhpMOmu/rZoQ4JQuEl3oQaxubxI0SLwtuwoc1YDjij+cBWN12GCcxiCz5Z6u4nB5ckcv7jC/evE1TfrlFX6qK9jG4cNtSdzlYqO8QFlnSFCtqzv2KhactVgiCMXm4/tKMDag3j1jnx2YE11vGGYaHeLZFBtoibbY3oc6f/KW5GlcmuBSRKke8XUYcUhox29Fc+cfs5seRDyAk+T++9vde+aZ7jiCcGgAx1zXkSSz5t4tei08yc0AX0YiZGObuZlnZm4K4Ffgt5ajemlc3CbltC11u3HmgTTmKgKOLi9RxGLxgsAplNnZN2t8BP5KtQBGgDyMmhwtIKjpnuaaJkShsF4apDQVVzN9hwtYTCqyakI0SauXE1cNoeZelItO6th/J1TsvttcBETUPagJjPUG/BEHnbbyDG2M7US7RxqTnMszZKR7CrjH0EQ3RiMHfqyTM5WX2TQI+EoAGMcrBBIV63fhm469K95X/t2NIDEvQmOr9xfj/XhRHIUe/hlShrYFjJf0uclABQPs/2gOiVcHcA8S8G8rGH0WMr/EBiOEeK2Lgiod2p4zK4o6lCFuo8vK6gaRwhv6hRGvlfSS/MTzbNCfQO0PbIp77PecYWdb9B8SbdECU9H1ZDhvTdE2qUvLQLEMY+DMGobU3XXp/RC6MnDP+Qk2/fgHTMoyKhIUwGQMFR9l/kRkbTzB9SkulRw0P6rdqjjt1bJSl
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(136003)(376002)(346002)(39860400002)(2616005)(53546011)(122000001)(66946007)(6506007)(66476007)(71200400001)(38100700002)(64756008)(66556008)(36756003)(6486002)(86362001)(91956017)(4326008)(76116006)(31696002)(83380400001)(26005)(478600001)(5660300002)(6512007)(66446008)(186003)(38070700005)(8676002)(2906002)(110136005)(316002)(8936002)(31686004)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eTNQN1BJM0RwdjlBTDJxNitSR2wyZmlJbkVUUW9mVk5vaktPSkcyS1ZDTXg1?=
 =?utf-8?B?S0c1WXM4SjN3NHBHTkprNTdxVC9pZ2grSWh5bGw3NDQ0WHA5aEpMa2k2V1VM?=
 =?utf-8?B?cVorVnJ5NEVEQllKdHVuZ1dKakxUQklhTWRnQWp1UUJMaGZFOHdjRFdQd2Zr?=
 =?utf-8?B?c21jSDVJOXNveitQU0Uva1hsNFVWc2RKZUE0OURLNmZ6dHdOY0JGTzlPdFZF?=
 =?utf-8?B?SzVxZmtYbnhYNmlOMklFQmJ2NkloMEYydHNBTXVMNEpzdUpSUS8xQTVQU2Vw?=
 =?utf-8?B?N0MzOGdObjU5Z0VVQllQdUc1VWg0RDd4ays1UTcxQStPQ0orRm52My82Sk9J?=
 =?utf-8?B?TWxRT0grMkc1NzAxOVQ2MzdrZ3YzU2xGN3BabHM4VG9VTC8yYWxxRTlqSjlY?=
 =?utf-8?B?d3c1cDhzNFRTNFBrK2lNKzlEYW9aSjMwVWdvKzNEV1pMYTdaU3VJNlBQNVhW?=
 =?utf-8?B?OUd5SEQrMnJGbGVva1FUbVYwVHFMbWl5TW5nS3ZrUHR1OUwwUzhlK2NuRExj?=
 =?utf-8?B?T1ZKN2xETTB3cVdOKy9vTkZTeXM1MllUQVNMVXp4ZWdTYVNIc0llOGF1TjB1?=
 =?utf-8?B?UEY3TlB4V3NXN29MVEdUcXo4TzFnL1B2Ulg5TFBJZW5FNTdGR0FKdXQybkNr?=
 =?utf-8?B?blRQMXA1dUJBT0NoaXhKWnlGTTBYbkh3MHowcjVlSGdKZ2RseDdrZDJyWCtw?=
 =?utf-8?B?S2p4enhvN0hqQmVVSTdaT3A4MnBoZUNVMDBidmJZN1lDVDZBUFZldmw4eEU0?=
 =?utf-8?B?Wnhnak1raFFjRzVyZGphYTBRT0lIeHJ3bWlad295ek9qWEs4cTR6N1Z2Qk8y?=
 =?utf-8?B?R042amNHWjJyM1NWSjRlZXJ0S0ErV0c1TTZvZ1NvaTJrL1VWcEZEQW0zd0NX?=
 =?utf-8?B?YVJ3NHd5L2JDRmJPTk0rMjErS0lETEU5SHZCcS9EMHJ3WWpFRit1TWlWN3FD?=
 =?utf-8?B?MjFwK1BMRVJCTHBJOGkwWVNvRk1wWXpaZENRVGFHOEJPUUk4dmREY2x2eXNq?=
 =?utf-8?B?akw0WWFxWGJYTHJsbW5UcnRVaW43dXRUcDFpNCt2YnBWenFUVU9QWUZLMjVN?=
 =?utf-8?B?NC92VjBENXhubWVMWVJ4aTR6WURmQkJ5Y3J6d21VSHhnK3cyWnNYRGRjRXlP?=
 =?utf-8?B?Ly8yV3pLT3FKdVRTbzNjWEtjbXBTTzUvQm5lZllNaGlGT0l6SjRqRE0yVG82?=
 =?utf-8?B?d0Uxelhrcm9xc0FDWS94MTRpWTNHOVZQcDYzMWZ0VGZ4MFFXaTlmVHVQMU50?=
 =?utf-8?B?ZXA5Z3VuVmk5M1R4SkNEdmhxWHlzeklHbXJxMnRNakVtQUlmbFhKSDl0TXBT?=
 =?utf-8?B?OGQvdWVUcmc0VnZaQkNXZFFBNDhBUjlaL2dCS1hWck1mSHEwTXBJYnNtc3p4?=
 =?utf-8?B?cXRrcEFPbWRqTGZsd2h0eTAwcTBDS3NySER6WVoyUUIvY3YzN0crbjk4ZjFJ?=
 =?utf-8?B?QTNLN0xnOU5KSnFYZ0NpUXc4NmsrMXpZZmVlajIrYjdDYTlDUTgyVlRqZXFY?=
 =?utf-8?B?Sjl1V1Uvd2F6R0NXNDdtQjVkQTVhaDFUekRNL1FRN3l1Z1ZLclA2dy9aMEE3?=
 =?utf-8?B?clRRYUo5UXFhQXRNN0szUmczZ09MZGhMOExsV21nK0NDM3ludWlkaXNqUVBt?=
 =?utf-8?B?ZXZLS0wzMThvSzRzZkZFWk1lS0piWFVnYlRGL1Boc1VGc0gyODdEM1E4eDZw?=
 =?utf-8?B?ZFFuZDNPY29WckNLNjJjUVhBRTJGVFp0bzhJMytpdk5ZNnFnYTViV1g4K0pW?=
 =?utf-8?Q?ayX7M4+0hM29POdF2/nftJfNwXe3Dbj5kXHt45+?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <FCA610E42E623B42B957260569C831D0@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c051a72-ec4a-4d4e-09e1-08d972db6520
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 15:14:46.8394
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ZgmO5Bh7Ka8cgqh6aKakNjcI2+leaXTTWhnBLeyZnlf+SR552fD0r/yFxwTK0j1gHiBziyXvcCtR2Kg1KCvWNpo12Yn9eLdQkmAEiCyvWC0zft4ROj2ZqfXewni3PqBD
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2766
X-Proofpoint-ORIG-GUID: 33t4pPL0pMMOIaiUq4upRmil8h2nh5Hx
X-Proofpoint-GUID: 33t4pPL0pMMOIaiUq4upRmil8h2nh5Hx
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-08_06,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0
 phishscore=0 priorityscore=1501 mlxlogscore=999 suspectscore=0
 lowpriorityscore=0 mlxscore=0 adultscore=0 impostorscore=0 clxscore=1015
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080096

DQpPbiAwOC4wOS4yMSAxNzo0NiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA4LjA5LjIwMjEg
MTU6MzMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDguMDkuMjEgMTM6
MDMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA4LjA5LjIwMjEgMTE6NDMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwOC4wOS4yMSAxMjoyNywgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDE5OjM5LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNy4wOS4yMSAxOTozMCwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTU6MzMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gT24gMDYuMDkuMjEgMTc6MzEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+
Pj4+Pj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hlYWRlci5jDQo+Pj4+Pj4+Pj4+
ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+Pj4+Pj4+Pj4gQEAgLTQwMCwxMiAr
NDAwLDcyIEBAIHN0YXRpYyB2b2lkIGJhcl93cml0ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRl
diwgdW5zaWduZWQgaW50IHJlZywNCj4+Pj4+Pj4+Pj4gICAgICAgc3RhdGljIHZvaWQgZ3Vlc3Rf
YmFyX3dyaXRlKGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVnLA0K
Pj4+Pj4+Pj4+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgdmFs
LCB2b2lkICpkYXRhKQ0KPj4+Pj4+Pj4+PiAgICAgICB7DQo+Pj4+Pj4+Pj4+ICsgICAgc3RydWN0
IHZwY2lfYmFyICpiYXIgPSBkYXRhOw0KPj4+Pj4+Pj4+PiArICAgIGJvb2wgaGkgPSBmYWxzZTsN
Cj4+Pj4+Pj4+Pj4gKw0KPj4+Pj4+Pj4+PiArICAgIGlmICggYmFyLT50eXBlID09IFZQQ0lfQkFS
X01FTTY0X0hJICkNCj4+Pj4+Pj4+Pj4gKyAgICB7DQo+Pj4+Pj4+Pj4+ICsgICAgICAgIEFTU0VS
VChyZWcgPiBQQ0lfQkFTRV9BRERSRVNTXzApOw0KPj4+Pj4+Pj4+PiArICAgICAgICBiYXItLTsN
Cj4+Pj4+Pj4+Pj4gKyAgICAgICAgaGkgPSB0cnVlOw0KPj4+Pj4+Pj4+PiArICAgIH0NCj4+Pj4+
Pj4+Pj4gKyAgICBlbHNlDQo+Pj4+Pj4+Pj4+ICsgICAgICAgIHZhbCAmPSBQQ0lfQkFTRV9BRERS
RVNTX01FTV9NQVNLOw0KPj4+Pj4+Pj4+PiArICAgIGJhci0+Z3Vlc3RfYWRkciAmPSB+KDB4ZmZm
ZmZmZmZ1bGwgPDwgKGhpID8gMzIgOiAwKSk7DQo+Pj4+Pj4+Pj4+ICsgICAgYmFyLT5ndWVzdF9h
ZGRyIHw9ICh1aW50NjRfdCl2YWwgPDwgKGhpID8gMzIgOiAwKTsNCj4+Pj4+Pj4+PiBXaGF0IHlv
dSBzdG9yZSBoZXJlIGlzIG5vdCB0aGUgYWRkcmVzcyB0aGF0J3MgZ29pbmcgdG8gYmUgdXNlZCwN
Cj4+Pj4+Pj4+PiAgICAgICBhcw0KPj4+Pj4+Pj4+IHlvdSBkb24ndCBtYXNrIG9mZiB0aGUgbG93
IGJpdHMgKHRvIGFjY291bnQgZm9yIHRoZSBCQVIncyBzaXplKS4NCj4+Pj4+Pj4+PiBXaGVuIGEg
QkFSIGdldHMgd3JpdHRlbiB3aXRoIGFsbCBvbmVzLCBhbGwgd3JpdGFibGUgYml0cyBnZXQgdGhl
c2UNCj4+Pj4+Pj4+PiBvbmVzIHN0b3JlZC4gVGhlIGFkZHJlc3Mgb2YgdGhlIEJBUiwgYWl1aSwg
cmVhbGx5IGNoYW5nZXMgdG8NCj4+Pj4+Pj4+PiAodHlwaWNhbGx5KSBjbG9zZSBiZWxvdyA0R2Ig
KGluIHRoZSBjYXNlIG9mIGEgMzItYml0IEJBUiksIHdoaWNoDQo+Pj4+Pj4+Pj4gaXMgd2h5IG1l
bW9yeSAvIEkvTyBkZWNvZGluZyBzaG91bGQgYmUgb2ZmIHdoaWxlIHNpemluZyBCQVJzLg0KPj4+
Pj4+Pj4+IFRoZXJlZm9yZSB5b3Ugc2hvdWxkbid0IGxvb2sgZm9yIHRoZSBzcGVjaWZpYyAiYWxs
IHdyaXRhYmxlIGJpdHMNCj4+Pj4+Pj4+PiBhcmUgb25lcyIgcGF0dGVybiAob3Igd29yc2UsIGFz
IHlvdSBwcmVzZW50bHkgZG8sIHRoZSAiYWxsIGJpdHMNCj4+Pj4+Pj4+PiBvdXRzaWRlIG9mIHRo
ZSB0eXBlIHNwZWNpZmllciBhcmUgb25lcyIgb25lKSBvbiB0aGUgcmVhZCBwYXRoLg0KPj4+Pj4+
Pj4+IEluc3RlYWQgbWFzayB0aGUgdmFsdWUgYXBwcm9wcmlhdGVseSBoZXJlLCBhbmQgc2ltcGx5
IHJldHVybiBiYWNrDQo+Pj4+Pj4+Pj4gdGhlIHN0b3JlZCB2YWx1ZSBmcm9tIHRoZSByZWFkIHBh
dGguDQo+Pj4+IEJ1dCBpbiBjYXNlIG9mIEJBUiBzaXppbmcgSSBuZWVkIHRvIGFjdHVhbGx5IHJl
dHVybiBCQVIgc2l6ZS4NCj4+Pj4gU28sIHRoZSBjb21wYXJpc29uIGlzIHRoZSB3YXkgdG8gdGVs
bCBpZiB0aGUgZ3Vlc3Qgd2FudHMgdG8gcmVhZA0KPj4+PiBhY3R1YWwgKGNvbmZpZ3VyZWQpIEJB
UiB2YWx1ZSBvciBpdCB0cmllcyB0byBkZXRlcm1pbmUgQkFSJ3Mgc2l6ZS4NCj4+Pj4gVGhpcyBp
cyB3aHkgSSBjb21wYXJlIGFuZCB1c2UgdGhlIHJlc3VsdCBhcyB0aGUgYW5zd2VyIHRvIHdoYXQg
bmVlZHMNCj4+Pj4gdG8gYmUgc3VwcGxpZWQgdG8gdGhlIGd1ZXN0LiBTbywgaWYgSSBkb24ndCBj
b21wYXJlIHdpdGggMHhmZmZmZmZmZiBmb3IgdGhlDQo+Pj4+IGhpIHBhcnQgYW5kIDB4ZmZmZmZm
ZjAgZm9yIHRoZSBsb3cgdGhlbiBob3cgZG8gSSBrbm93IHdoZW4gdG8gcmV0dXJuDQo+Pj4+IGNv
bmZpZ3VyZWQgQkFSIG9yIHJldHVybiB0aGUgc2l6ZT8NCj4+PiBXZWxsLCBidXQgdGhhdCdzIHRo
ZSBjb21tb24gbWlzdW5kZXJzdGFuZGluZyB0aGF0IEkndmUgYmVlbiB0cnlpbmcNCj4+PiB0byBw
b2ludCBvdXQ6IFRoZXJlJ3Mgbm8gZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZXNlIHR3byBmb3JtcyBv
Zg0KPj4+IHJlYWRzLiBUaGUgQkFScyBhcmUgc2ltcGx5IHJlZ2lzdGVycyB3aXRoIHNvbWUgci9v
IGJpdHMuIFRoZXJlJ3MNCj4+PiBubyBoaWRkZW4gMm5kIHJlZ2lzdGVyIHJlY29yZGluZyB3aGF0
IHdhcyBsYXN0IHdyaXR0ZW4uIFdoZW4geW91DQo+Pj4gd3JpdGUgMHhmZmZmZmZmZiwgYWxsIHlv
dSBkbyBpcyBzZXQgYWxsIHdyaXRhYmxlIGJpdHMgdG8gMS4gV2hlbg0KPj4+IHlvdSByZWFkIGJh
Y2sgZnJvbSB0aGUgcmVnaXN0ZXIsIHlvdSB3aWxsIGZpbmQgYWxsIHIvbyBiaXRzDQo+Pj4gdW5j
aGFuZ2VkICh3aGljaCBpbiBwYXJ0aWN1bGFyIG1lYW5zIGFsbCBsb3dlciBhZGRyZXNzIGJpdHMg
YXJlDQo+Pj4gemVybywgdGh1cyBhbGxvd2luZyB5b3UgdG8gZGV0ZXJtaW5lIHRoZSBzaXplKS4N
Cj4+Pg0KPj4+IFdoZW4gdGhlIHNwZWMgc2F5cyB0byB3cml0ZSAweGZmZmZmZmZmIGZvciBzaXpp
bmcgcHVycG9zZXMsIE9TZXMNCj4+PiBzaG91bGQgZm9sbG93IHRoYXQsIHllcy4gVGhpcyBkb2Vz
bid0IHByZWNsdWRlIHRoZW0gdG8gdXNlIG90aGVyDQo+Pj4gZm9ybXMgb2Ygd3JpdGVzIGZvciB3
aGljaGV2ZXIgcHVycG9zZS4gSGVuY2UgeW91IGRvIG5vdCB3YW50IHRvDQo+Pj4gc3BlY2lhbCBj
YXNlIHNpemluZywgYnV0IGluc3RlYWQgeW91IHdhbnQgdG8gZW11bGF0ZSBjb3JyZWN0bHkNCj4+
PiBhbGwgZm9ybXMgb2Ygd3JpdGVzLCBpbmNsdWRpbmcgc3Vic2VxdWVudCByZWFkcyB0byB1bmlm
b3JtbHkNCj4+PiByZXR1cm4gdGhlIGludGVuZGVkIC8gZXhwZWN0ZWQgdmFsdWVzLg0KPj4+DQo+
Pj4gSnVzdCB0byBnaXZlIGFuIGV4YW1wbGUgKHBlcmhhcHMgYSBsaXR0bGUgY29udHJpdmVkKTog
VG8gc2l6ZSBhDQo+Pj4gNjQtYml0IEJBUiwgaW4gcHJpbmNpcGxlIHlvdSdkIGZpcnN0IG5lZWQg
dG8gd3JpdGUgMHhmZmZmZmZmZiB0bw0KPj4+IGJvdGggaGFsdmVzLiBCdXQgdGhlcmUncyBub3Ro
aW5nIHdyb25nIHdpdGggZG9pbmcgdGhpcyBpbiBhDQo+Pj4gZGlmZmVyZW50IG9yZGVyOiBBY3Qg
b24gdGhlIGxvdyBoYWxmIGFsb25lIGZpcnN0LCBhbmQgdGhlbiBhY3QNCj4+PiBvbiB0aGUgaGln
aCBoYWxmLiBUaGUgYWN0aW5nIG9uIHRoZSBoaWdoIGhhbGYgY291bGQgZXZlbiBiZQ0KPj4+IHNr
aXBwZWQgaWYgdGhlIGxvdyBoYWxmIHNpemluZyBwcm9kdWNlZCBhdCBsZWFzdCBiaXQgMzEgc2V0
LiBOb3cNCj4+PiBpZiB5b3Ugd2VyZSB0byBzcGVjaWFsIGNhc2Ugc2VlaW5nIGZmZmZmZmZmOmZm
ZmZmZmY/IGFzIHRoZQ0KPj4+IGxhc3Qgd3JpdHRlbiBwYWlyIG9mIHZhbHVlcywgeW91J2QgYnJl
YWsgdGhhdCAoaW1vIGxlZ2l0aW1hdGUpDQo+Pj4gYWx0ZXJuYXRpdmUgcHJvY2VzcyBvZiBzaXpp
bmcuDQo+PiBIb3cgYWJvdXQ6DQo+IFllcywgdGhhdCdzIHdoYXQgSSB3YXMgYWZ0ZXIuIEp1c3Qg
b25lIG5pdCByaWdodCBhd2F5Og0KPg0KPj4gc3RhdGljIHZvaWQgZ3Vlc3RfYmFyX3dyaXRlKGNv
bnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVnLA0KPj4gICDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDMyX3Qg
dmFsLCB2b2lkICpkYXRhKQ0KPj4gew0KPj4gICDCoMKgwqAgc3RydWN0IHZwY2lfYmFyICpiYXIg
PSBkYXRhOw0KPj4gICDCoMKgwqAgYm9vbCBoaSA9IGZhbHNlOw0KPj4NCj4+ICAgwqDCoMKgIGlm
ICggYmFyLT50eXBlID09IFZQQ0lfQkFSX01FTTY0X0hJICkNCj4+ICAgwqDCoMKgIHsNCj4+ICAg
wqDCoMKgwqDCoMKgwqAgQVNTRVJUKHJlZyA+IFBDSV9CQVNFX0FERFJFU1NfMCk7DQo+PiAgIMKg
wqDCoMKgwqDCoMKgIGJhci0tOw0KPj4gICDCoMKgwqDCoMKgwqDCoCBoaSA9IHRydWU7DQo+PiAg
IMKgwqDCoCB9DQo+PiAgIMKgwqDCoCBlbHNlDQo+PiAgIMKgwqDCoCB7DQo+PiAgIMKgwqDCoMKg
wqDCoMKgIHZhbCAmPSBQQ0lfQkFTRV9BRERSRVNTX01FTV9NQVNLOw0KPj4gICDCoMKgwqDCoMKg
wqDCoCB2YWwgfD0gYmFyLT50eXBlID09IFZQQ0lfQkFSX01FTTMyID8gUENJX0JBU0VfQUREUkVT
U19NRU1fVFlQRV8zMg0KPj4gICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOiBQQ0lfQkFT
RV9BRERSRVNTX01FTV9UWVBFXzY0Ow0KPj4gICDCoMKgwqDCoMKgwqDCoCB2YWwgfD0gYmFyLT5w
cmVmZXRjaGFibGUgPyBQQ0lfQkFTRV9BRERSRVNTX01FTV9QUkVGRVRDSCA6IDA7DQo+PiAgIMKg
wqDCoCB9DQo+Pg0KPj4gICDCoMKgwqAgYmFyLT5ndWVzdF9hZGRyICY9IH4oMHhmZmZmZmZmZnVs
bCA8PCAoaGkgPyAzMiA6IDApKTsNCg0KRG8geW91IHRoaW5rIHRoaXMgbmVlZHMgdG8gYmUgMHhm
ZmZmZmZmZnVsLCBub3QgMHhmZmZmZmZmZnVsbD8NCg0KZS5nLiBzL3VsbC91bA0KDQo+PiAgIMKg
wqDCoCBiYXItPmd1ZXN0X2FkZHIgfD0gKHVpbnQ2NF90KXZhbCA8PCAoaGkgPyAzMiA6IDApOw0K
Pj4NCj4+ICAgwqDCoMKgIGJhci0+Z3Vlc3RfYWRkciAmPSB+KGJhci0+c2l6ZSAtIDEpIHwgflBD
SV9CQVNFX0FERFJFU1NfTUVNX01BU0s7DQo+PiB9DQo+Pg0KPj4gc3RhdGljIHVpbnQzMl90IGd1
ZXN0X2Jhcl9yZWFkKGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB1bnNpZ25lZCBpbnQgcmVn
LA0KPj4gICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgdm9pZCAqZGF0YSkNCj4+IHsNCj4+ICAgwqDCoMKgIHN0cnVjdCB2cGNpX2Jh
ciAqYmFyID0gZGF0YTsNCj4gY29uc3QgcGxlYXNlLg0KU3VyZQ0KPg0KPiBKYW4NCj4NClRoYW5r
IHlvdSBmb3IgaGVscGluZyB3aXRoIHRoaXMhIQ0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:18:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:18:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182225.329701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzL5-0003KO-Dn; Wed, 08 Sep 2021 15:18:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182225.329701; Wed, 08 Sep 2021 15:18:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzL5-0003KH-Aa; Wed, 08 Sep 2021 15:18:15 +0000
Received: by outflank-mailman (input) for mailman id 182225;
 Wed, 08 Sep 2021 15:18:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNzL4-0003KB-0v
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:18:14 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fb8a7c60-10b7-11ec-b166-12813bfff9fa;
 Wed, 08 Sep 2021 15:18:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb8a7c60-10b7-11ec-b166-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631114293;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=vgbADa/BeAfDLA5gMI6dcJmJzTZlZecMN4BBfA2o+do=;
  b=HyZw9NFBdPvPM0/VSxa7UVFPERa4Q0F1taC1/E1YXE8j4m1KJcqzyfbs
   WeYM+26Sb2+BU8nZxObb8RwTtqdtYGSVG0NVO9tQo/49C1+B0OALK3YFp
   UDHKRY7v3fwjWizu5Qg9epflxPgu3qv1N4pJahIwBc2D+o5pgycgq55p5
   o=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: ownEoTcax3yoAnjo4Fg+7GcwxJRSwkBrSCKNQS14/hryjFi9waoALThHEU97k4SAemyRvltmDr
 UrQB+4iRhj8RS9C6XN1GY7nrleoYc6hesBCMpi2KQ0G+10qWmZHYOO/02XRP1tnNiBGGgMxcrb
 6i0L1uli1UprUyiw9Yk7wfnoHkhyy9r+8rgFjhjOpuq1yROk/BmQ9FmXIaSBT0UuzYrpLj9aaT
 nxA0XApjpC0vx+umPjefposb5zDSUdgbiCDBYMqaa37cGMIqbUCXZz3GkjJjmVrQD7guI9RLKu
 e49VuMhW3tovnAB0OJQRhinZ
X-SBRS: 5.1
X-MesageID: 52268475
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:0xa1Makx2GgY7NzTBpYDLHeniR3pDfK33DAbv31ZSRFFG/Fw9v
 rEoB1173HJYVoqN03I+urrBEDoexq1yXcf2+Qs1NmZMDUOw1HIEKhSqbH6xiHmGWnX7/c179
 YDT4FOTOfoCER8j4LQ/Am8Hr8bsb+62ZHtv/7a03BuCSZjb60I1WdEIzfeKFRyWA5FQbwmE5
 aGj/A3xAaISDAsd8GmAXtAZcjvzue78a7OUFo9Hhs67wvLtz+y97LgNBCd0n4lIldy6IZny3
 HBjwv6opquqOu6zBi07R6106hr
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="52268475"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH for-4.13] x86/PVH: Fix build following XSA-378 bugfix
Date: Wed, 8 Sep 2021 16:17:54 +0100
Message-ID: <20210908151754.9593-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Fixes: 8d8b4bde3e1c ("x86/PVH: de-duplicate mappings for first Mb of Dom0 memory")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/hvm/dom0_build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index 62958bf82cbe..c6eb86bfb7ee 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -467,7 +467,7 @@ static int __init pvh_populate_p2m(struct domain *d)
         p2m_type_t p2mt;
 
         if ( mfn_eq(get_gfn_query(d, i, &p2mt), INVALID_MFN) )
-            rc = set_mmio_p2m_entry(d, _gfn(i), _mfn(i), PAGE_ORDER_4K,
+            rc = set_mmio_p2m_entry(d, i, _mfn(i), PAGE_ORDER_4K,
                                     p2m_get_hostp2m(d)->default_access);
         else
             ASSERT(p2mt == p2m_ram_rw);
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:18:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:18:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182226.329712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzLB-0003cy-Ld; Wed, 08 Sep 2021 15:18:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182226.329712; Wed, 08 Sep 2021 15:18:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzLB-0003cr-IH; Wed, 08 Sep 2021 15:18:21 +0000
Received: by outflank-mailman (input) for mailman id 182226;
 Wed, 08 Sep 2021 15:18:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RTps=N6=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mNzL9-0003c7-RG
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:18:19 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fef71a2a-10b7-11ec-b166-12813bfff9fa;
 Wed, 08 Sep 2021 15:18:19 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 331631FD91;
 Wed,  8 Sep 2021 15:18:18 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id EADAB13A9C;
 Wed,  8 Sep 2021 15:18:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id h4pwNznUOGGUPgAAGKfGzw
 (envelope-from <jgross@suse.com>); Wed, 08 Sep 2021 15:18:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fef71a2a-10b7-11ec-b166-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631114298; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=G5Zk68YGYcr0HzofW2ihLUNZ50TcUo8lhAlDEbDeNHc=;
	b=OgxKOrp8tBRVvF91WSKRQmViYjeRK41KBpRkyRFBWM4il6wUKa4R8V9PbM27VSZXBjXGCi
	oGttmiI/4MBgI8EQOWhMdEaMISRMyG3NC+dH1DddL5xlMQho7ecCd0GBUM6w8ZrmCwkRre
	O3vNVtzH9Acs8lGqVg3oYqHL5x7w4G8=
Subject: Re: [PATCH 00/12] swiotlb-xen: fixes and adjustments
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 iommu@lists.linux-foundation.org
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <5623df3b-b810-6965-2fe3-fd4250998b49@suse.com>
Date: Wed, 8 Sep 2021 17:18:17 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="kiBXlrHUO2vcijL3q3ORdzz0LLTT33OUG"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--kiBXlrHUO2vcijL3q3ORdzz0LLTT33OUG
Content-Type: multipart/mixed; boundary="E0GLwH60fzdpZMY3XoyJe2hrn4wyGCoe0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 iommu@lists.linux-foundation.org
Message-ID: <5623df3b-b810-6965-2fe3-fd4250998b49@suse.com>
Subject: Re: [PATCH 00/12] swiotlb-xen: fixes and adjustments
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>

--E0GLwH60fzdpZMY3XoyJe2hrn4wyGCoe0
Content-Type: multipart/mixed;
 boundary="------------2F25B44C78E3383CE5184EBE"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2F25B44C78E3383CE5184EBE
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 14:03, Jan Beulich wrote:
> The primary intention really was the last patch, there you go ...
>=20
> 01: swiotlb-xen: avoid double free
> 02: swiotlb-xen: fix late init retry
> 03: swiotlb-xen: maintain slab count properly
> 04: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
> 05: swiotlb-xen: suppress certain init retries
> 06: swiotlb-xen: limit init retries
> 07: swiotlb-xen: drop leftover __ref
> 08: swiotlb-xen: arrange to have buffer info logged
> 09: swiotlb-xen: drop DEFAULT_NSLABS
> 10: xen-pcifront: this module is PV-only
> 11: xen/pci-swiotlb: reduce visibility of symbols
> 12: swiotlb-xen: this is PV-only on x86
>=20
> Jan
>=20

You should have Cc-ed Konrad being the maintainer and=20
iommu@lists.linux-foundation.org. Doing so now.


Juergen

--------------2F25B44C78E3383CE5184EBE
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2F25B44C78E3383CE5184EBE--

--E0GLwH60fzdpZMY3XoyJe2hrn4wyGCoe0--

--kiBXlrHUO2vcijL3q3ORdzz0LLTT33OUG
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE41DkFAwAAAAAACgkQsN6d1ii/Ey9i
6Af/XUtL8n34y7NXXB4YvyjbovHeyY1G1M0np4qTnOj/58mDCf4JhggM5cO78YnFGF+0juIvBonE
Fbs3k6V0s+szi/SCPVome8Gt/N9vNiJUBnmBy5loxO7EFiGNrd0mAhNW8kQ6dDGD/OZMhUQFQSZw
6R5dNamDm54KXtdVwT5eN+aCPzhMSDm2wJAxMpFzwQQ83GwlYw6HXgq21w3LZSOruqIh/68sNbUz
kQgQWwQ6mBI34ZJdmGSAdc0AuvJQXeYzh3iLo8/b2CAk8e/mYgtm2TFk1pw9Je7Eeqd9zIm5TMQC
DgcLRFSPQhbV/uhWpyzkBjrFW9lXtsaLK4aQdgCNfQ==
=6aie
-----END PGP SIGNATURE-----

--kiBXlrHUO2vcijL3q3ORdzz0LLTT33OUG--


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:30:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:30:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182242.329723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzWV-0005xX-PA; Wed, 08 Sep 2021 15:30:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182242.329723; Wed, 08 Sep 2021 15:30:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzWV-0005wv-L1; Wed, 08 Sep 2021 15:30:03 +0000
Received: by outflank-mailman (input) for mailman id 182242;
 Wed, 08 Sep 2021 15:30:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNzWT-0005gb-Oh
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:30:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a12ab90e-10b9-11ec-b167-12813bfff9fa;
 Wed, 08 Sep 2021 15:30:00 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2057.outbound.protection.outlook.com [104.47.13.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-K4IFlJ-UMW-fLb5KrPljBw-1; Wed, 08 Sep 2021 17:29:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 15:29:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 15:29:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0158.eurprd02.prod.outlook.com (2603:10a6:20b:28d::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 8 Sep 2021 15:29:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a12ab90e-10b9-11ec-b167-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631114999;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WShhmBomMdXUvoPL+ZxDbQGjovzQP4FsvPEmvg/RKY0=;
	b=lvvTS4DLJ+JoGs46NyheD7RZY7kuDb9QelJWQP5Y78SfRux26aR3wnRBzPOPWqjHGBM6BX
	kCPNlE9We344jYsDQj69hPjzHDycagY4kbvkoNPfBoM7cWic992fA+HKpeLGDG5A4BZwPG
	xURU0TNsmpNxNIXu0TEgJ+RlgInb9gc=
X-MC-Unique: K4IFlJ-UMW-fLb5KrPljBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XClmFNKyrvyT7I0uCs5oeem2vZ3fCar7EDkD2LPoe6J1cRmBPFOsJL9EKvy4QNx/ofdE3SUQTluDbyPtX+pATJ1kMvAHyS2vFkqZ2e35hnAG9cbe3v1fespU9IoVER++qP94330ucDRhJeodL2PA42o2OLjc5AWWBJYnt+YVVKYI7X1YD3e/iQN+Iw/DoymCxi3O8lsuQkFG6yJ8TMHXHMX3nMWq/pRivaE2IQNYfGEmY4sTnzKS6Vdjz2psHYqwcnsK8gYHNU4dRsL/EVoQVNMbtvFcGdk1D8E0GZNEEC1zJOF+Dxsi5DwuprqduXnInbK6e/kVBlzJQwU0/Ma3kQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DvYrDPdby1PLKL0r55nnyfge/U0QTcsM7hcPE5QmApE=;
 b=THUAS6Z9hmP17VlQgFafqczekRVQN74FatCWz25Qd8Bc/AYVUV/nWu4kD/DuEdPzPzA9PMD/elESRTK6LiMS8Q300wRAQyVF7usJs5RXI+RJjv/lT4QqH59X8wBycXsTzCkb2r/zvYeHnZ0XIxhm2P3bsKUGiJfQSgh0KKsMxRnRk6Sqf+hoWPBy6kFGvsS9d84K0SaA01R4f58ARstfeuxqCNB0FeqXXhaZiM2LIy2bS2XPLs+sy6El3lwpGNvO5XhlNv3XbQCSh6hHi3xfjME8+IhtMdoSvzBp1qAjhoyRI8fjl6qEgdvH6JEuKwAgeI5Se78RtbzFUXbvrWOc7w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
 <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
 <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
 <2fd6b81e-e5fc-f6d8-b24f-9fbb1db034d7@suse.com>
 <d5ef78d1-253e-26cb-8d24-667d1ce7a671@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <47173d64-781f-ae37-5eee-f483fd74e828@suse.com>
Date: Wed, 8 Sep 2021 17:29:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <d5ef78d1-253e-26cb-8d24-667d1ce7a671@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0158.eurprd02.prod.outlook.com
 (2603:10a6:20b:28d::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7b15e9ed-7c20-4178-f8a7-08d972dd829c
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686392597ADB5C1EC978BAFCB3D49@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2657;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QmtNS9YUKif7KNXwfKaQAr8EGt6odx1DwW681jTfvBsDPpBoQVDy6FjB2doaatgRvHMWxdw4oo90X+8Tsvo/F5uRwEAZWdt2PXdxwLuF2fqxWNpPSJilyhZ3066iu6lHLTM7668U1WSVqYzQMi/idzVWCOdUDXBEazqpHlQbbCDRongBYESHdZjgUfB4jEE+pg/SodqOUIPLrLfdcYTpomBwgOmEXsTt+YFSq8AqS0LO+c37kQMvUj/o86PVi3IHu/lXjfjgZuAggyIrdiC5O2P5e6J82VFM/Lvin+4LjzVzHDxMaU3hJXkOOja/SKgb9mCqThn409AwAdm3gSSj4Farh4vGCDvcXMO4+3WuziMqKmMrH7iwG9jvJ+rPGDdOY7P3BtR7Mr9Bcj0t17Hba7Z46Iv3+pKJoEjDQIw0rzrLmVpQQH0n8OUxV2NjrB3Z35uP5lSnjc3ASr9HakZARxd8lOYBD7Uzr6X69Soa4PQGy1qi1sUO/kazpaUTosbyqJHRSaY8j825md/6LajdvsgACdyObqnw92R1QVlzoKG1mvQ3EBOQxGeMf8M57q8pUsB7HrSomSd++0wUlOR0iTNaoNcpgNw/zm54+euaqFf8kq+detKe+jikRXQlLzqCtiCKlm141P3svtdOcSI+FJWceNnh+WTXtV8PZtHgNR51xgna3a1mbsW5lNKHu0TF/pe3qL2/BTxj47fIYM+IskaI9k5CIZHsHSQ8ts2SSmmPc9G6pa6GXSYmP277SaSm
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(136003)(39860400002)(346002)(2616005)(66946007)(53546011)(66476007)(38100700002)(66556008)(36756003)(6486002)(86362001)(956004)(4326008)(31696002)(26005)(478600001)(5660300002)(7416002)(4744005)(186003)(8676002)(2906002)(316002)(8936002)(110136005)(31686004)(16576012)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?EIViW5IwzmhWX6ILq1sNqYu5lAB0Ycz7xOyf9FC+/gMM9YK0vtHRMqfwPbP0?=
 =?us-ascii?Q?UERqsLgojZRfC15yYUeaHcgxcEF7Qk/9zODn+cywwYWhL5zXi2r/6RZVsCAt?=
 =?us-ascii?Q?XuVLbPbSHdXXcR6tYGbgOko62BTOYMPB9bZhng32qyrl09EDPdUbGLfrmqWf?=
 =?us-ascii?Q?V4KUIMKlB2US4oFZEgRTU8GFPyeJKLmeedX77cOBuAkRbajI38Mtk3c9e9oQ?=
 =?us-ascii?Q?5cHe7PLHPoVy29ynjf9fP3M5oTJizP/THVykQakjDr0Q/mwyDBw7LqwNRUh4?=
 =?us-ascii?Q?dVFFXfH4l3ukz4nL4VpwF1tIg3+5IDTXRArttLlY5+v6eE3gBpJCTicXY0bT?=
 =?us-ascii?Q?PoiBcAlHt/JuJtElm/sBh/19u/ixzqUWz0u2E+nCQDSt3yGbuvZADSoYUYnN?=
 =?us-ascii?Q?WVern6q1cA2J/Ukdrrsh/Gk9lbj/POfBZZPmtFCy6gPgVDulqi2h9qfycktz?=
 =?us-ascii?Q?9/1NMyGhp8aienJkaz4NvvzMzyJLIdEjGQkSlqzJuF56NDlqfl1lvebnq3a2?=
 =?us-ascii?Q?Jo4sHkRQZwkzRVcnyNHbu4+4lvIbg4Dr+6M6mSTU4gxGj8/0IxwPaGsOoVQD?=
 =?us-ascii?Q?44BUDRbgOn94nv2qIuXB8iFsB/K1J3NVJEWuLUyjjClgJXSA539ohxsyIt+d?=
 =?us-ascii?Q?dqTpIzAIf8b9eW0Ue4oExqzWURmTOEbOlOmInPCCSw5bSqdL7ytIZFpGUPXr?=
 =?us-ascii?Q?kE/+JfTboPVVGguj+n15dcCNKBJYbclQXKe5A5MTWB3HY3CPIsgROGPuO4+P?=
 =?us-ascii?Q?Sz61xJikKzRVUaEq4ibnFkejSMyYj4wzMEly3G+ItqYEm9+lRIc0+36+gceN?=
 =?us-ascii?Q?3O05Usm9hjOobxSB+C0NpLD8V2oTtvaJZV2hQx8obv0B2RZC57IWG8AiaABw?=
 =?us-ascii?Q?6BRF3e5fnOY/aDHgLafBjAJ8cmuM+PFMIgQEFfKh34k14VvcPyOnFzXrhcfA?=
 =?us-ascii?Q?UyB4xU6vSgH6U4aWtxLVgnmob8VGl31E0e3gkTWzfFnqu9xDbAbeKyBTMT5U?=
 =?us-ascii?Q?LYx25DIJ311R+ZrHKhhrTF3zih+tyYwCUW4cCRznLVNHFRJQPVqS2meGasal?=
 =?us-ascii?Q?A5ojEnKbnFAeNURwq6+gN3Pp2CmqMe3ycsrq9WNjxvJUP+xIAPTH4SRQ/LA5?=
 =?us-ascii?Q?Dtd0hZZjweqwFZLdo2CziI0A/k/tf1kHd4a0L3uSauEHiYr01b7x/jEdjAH6?=
 =?us-ascii?Q?OxM14jcYmpU0whfDffBKkxYQhWnTSyF52QgxeMvBSq2dL2MHVKP58Qt2qSGp?=
 =?us-ascii?Q?9m0VbTi4OwTcyQdnsNv5fjbWSa9ytzrgiyhY3d8eK3ZGt3j3u8IraX+GyxXz?=
 =?us-ascii?Q?EG0PHxPInbk126ylS+JfjEUw?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b15e9ed-7c20-4178-f8a7-08d972dd829c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 15:29:55.5169
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uWAd/g4mvIj+x6n99KPcMfE+jICw41/TLVyd/FdP/AV1SrCjd5Tj69coEOB47nzDA5AgFBPWEc/Jv1DA72i9lw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 08.09.2021 17:14, Oleksandr Andrushchenko wrote:
> On 08.09.21 17:46, Jan Beulich wrote:
>> On 08.09.2021 15:33, Oleksandr Andrushchenko wrote:
>>> static void guest_bar_write(const struct pci_dev *pdev, unsigned int re=
g,
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 uint32_t val, void *data)
>>> {
>>>   =C2=A0=C2=A0=C2=A0 struct vpci_bar *bar =3D data;
>>>   =C2=A0=C2=A0=C2=A0 bool hi =3D false;
>>>
>>>   =C2=A0=C2=A0=C2=A0 if ( bar->type =3D=3D VPCI_BAR_MEM64_HI )
>>>   =C2=A0=C2=A0=C2=A0 {
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ASSERT(reg > PCI_BASE_ADDR=
ESS_0);
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bar--;
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hi =3D true;
>>>   =C2=A0=C2=A0=C2=A0 }
>>>   =C2=A0=C2=A0=C2=A0 else
>>>   =C2=A0=C2=A0=C2=A0 {
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val &=3D PCI_BASE_ADDRESS_=
MEM_MASK;
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val |=3D bar->type =3D=3D =
VPCI_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : PCI_BASE_ADDRESS_MEM_TYPE_64;
>>>   =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 val |=3D bar->prefetchable=
 ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
>>>   =C2=A0=C2=A0=C2=A0 }
>>>
>>>   =C2=A0=C2=A0=C2=A0 bar->guest_addr &=3D ~(0xffffffffull << (hi ? 32 :=
 0));
>=20
> Do you think this needs to be 0xfffffffful, not 0xffffffffull?
>=20
> e.g. s/ull/ul

If guest_addr is uint64_t then ull would seem more correct to me,
especially when considering (hypothetical?) 32-bit architectures
potentially wanting to use this code.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:35:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:35:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182249.329733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzbt-0007Ni-DU; Wed, 08 Sep 2021 15:35:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182249.329733; Wed, 08 Sep 2021 15:35:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzbt-0007Nb-AT; Wed, 08 Sep 2021 15:35:37 +0000
Received: by outflank-mailman (input) for mailman id 182249;
 Wed, 08 Sep 2021 15:35:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0iEI=N6=epam.com=prvs=98850bb71f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mNzbs-0007NV-JV
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:35:36 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 690beb28-10ba-11ec-b168-12813bfff9fa;
 Wed, 08 Sep 2021 15:35:35 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188FSiPQ004668; 
 Wed, 8 Sep 2021 15:35:32 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 by mx0b-0039f301.pphosted.com with ESMTP id 3axyusr0wg-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 08 Sep 2021 15:35:32 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3716.eurprd03.prod.outlook.com (2603:10a6:208:4a::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 15:35:30 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4500.015; Wed, 8 Sep 2021
 15:35:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 690beb28-10ba-11ec-b168-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lJOKCfFNsfO6eyg8HXQr2MEpj6QsQu3kY8a3pNpivqMBixsFS4WmKzSY54+faDCJkEHJJ2UtuNgkGhDVof0BycXil8ti7zj4+anLImbaQ6EZ7d8IHIQj8tJEcNd1R1oN54MKH7G+d3QbfCLHnZyE3ZBBxm3Uo2oeZN+khcXofoIUQejGy1NVZp2uEccPzLuzoLGndYS/VnxeNXYKg57bfF/kgyxlqGSuJLhjRirY9AwghdUF4xl3LZVdE0hUfo91kDDzIEiNtkPJnKo2tDP/Fp14ohL1zAa0n++qI081aFDC01sAWcdGZvMBaQVFkatW4E39m1i6mK0UKuqxweY37w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6pkHIxDai02+KJhR4h/In0HYY9J2Svl375q3OsrMWnc=;
 b=Zap9DTGZnzafuJ2yOJ+MTdLUVY1O9Q/Eecp6RXDQ37u/VfAZDrEtvLSTc/cs/TmR3w1bE3eYq6U9vMuzE1Tw5NrdCzBohHJcCMunNloc8Z5rxciZW3QAotJI8SfT5oeFcecQq/+ICrsANo4ayG4e4kwRp8oCKbzV9H7kFk27r8P3eX+yGoU4oZS81b+9jLjwyLI22o5VVrTVZmPbyJRRggZELBnebouU0IdedGXmwgUU75k/hTxc7XK6IPezbSE4APo9V44R080aC3+8wR11OGaR7+wFCWxL/6ZCyfJGTVHLhtwTDKTvmBQUWjBSRLTGmofSNwanP6fzcwOOMUdq2A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6pkHIxDai02+KJhR4h/In0HYY9J2Svl375q3OsrMWnc=;
 b=xiFcLDqSMXMzQ+Y8mxnHRZBH5NdwbZtsvyvCV3cQbetHbbqBe9MCoNPEOdRCcw3KpIEm2IiJaLs+kNiOWL+ZwDD91ECWMqTa1mR/Gv8oLCrmolOYLPZJP4TZB00tTqvkHdqbaVTcXcRyeVhdAVJY4HEZDbA2ktNZcWb0qls78Mh+q7BuXBaxDe0eMZcDggGFOsdPIRJhVcY6ZlV+tlRBimiOV50UPvWqyNXRf5Ihe5DVyO0I0+7QvP8juFhk7tHuvI89oJG4KECvhQH1UfSlWqKlSYmvhG8biCMpS2TEs9NasfFqTjo71ov0aNzsFHiElBMFemYfbse1vZyzovr9qw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Topic: [PATCH 5/9] vpci/header: Implement guest BAR register handlers
Thread-Index: 
 AQHXoKxmxnIdZosa7ESbvTmPlG6KHauXFn+AgAGCLICAADFtgIAAE0qAgAEIooCAAAS6AIAABV4AgAA6yACAABRDgIAACAOAgAAEOQCAAAGQAA==
Date: Wed, 8 Sep 2021 15:35:29 +0000
Message-ID: <d6979910-271a-a37a-886c-9fb90223a1e9@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-6-andr2000@gmail.com>
 <1848521d-4179-f5ee-e3af-f4e6738f60e6@suse.com>
 <eda075b5-eafd-2367-2f1e-7f6b9730beb0@epam.com>
 <2e0c2ff5-7228-a439-c8a6-50f7a022e77b@suse.com>
 <24624eca-766d-6841-dc90-43371f5a39ba@epam.com>
 <983f79a9-d72d-0704-9b04-950f858fbd7e@suse.com>
 <be31cb44-fda1-0413-dae9-412affd6e3f7@epam.com>
 <80de7569-522d-656a-e9e2-9e741511963d@suse.com>
 <dafd3135-ca9f-cd4e-c966-eb51c2a6e9e4@epam.com>
 <2fd6b81e-e5fc-f6d8-b24f-9fbb1db034d7@suse.com>
 <d5ef78d1-253e-26cb-8d24-667d1ce7a671@epam.com>
 <47173d64-781f-ae37-5eee-f483fd74e828@suse.com>
In-Reply-To: <47173d64-781f-ae37-5eee-f483fd74e828@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7a40f487-d9a4-4703-fbe0-08d972de4a01
x-ms-traffictypediagnostic: AM0PR03MB3716:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB3716DEBBAC433E8B4DC6980EE7D49@AM0PR03MB3716.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6430;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 kGeHLzZUjQxDcIXDCoBLk5AJWH2EHcqZoO26ZaU6Te2JzR8ZTeHCb6N8jllo9Rix2BCBCo4kHcrKjh0ZqjTTXeqP+Z34pdK4aZu6zu/M67mK6EOW9UMURrFANEujnihQf4UutyYQyU35V+YV5f+giFRgqJIvdJvJb+hosmn3TuPTY5pf/Fecomfie5rbEHnLGWHE8SOFDhcUzsW469VIbHkVnesdBrZtqjxTQy1NjL57Els8mQmyj3c9MnjwI9WgPIIx3m12Run5J2vglRHq2apNQeMbGA2zme338JvJk+0+WWHBFlDzK2husRAcrkjBD/640gYeHGBYH/375CtZGBqU3BjBR9ugOGLAaL+PyJzgiiH3jgicxkWAV7pcEaUmKFNsgS7htJV/JHBwVGPERRtI8HXL0lj1R5Cq04f+RSukLnsk1x2uQ6DcWlWXYy9Q+PAcTp9WTXHwncpQqfk3mZSU4q7fPDB4qdoAkj72oIbq0HFlHT0PwtUh8cu2PJa3+iUHt1K0Z7Q5HEfUhEehTYF+i34Xany42mloqJugOgpiJO+wdYJ9ZN+8Fq0YVkSjnKiNk7r01+7W87OObW49nP6HRTovouSYzVtimhDmDY98FuDrmi3PA1Lhs0ZH1KZ6FWOOBeec18r2koLhhMu38SGwEhLoUb85sN+YR5AYOCpegYmb0g3ZL7zan0vYwZBCJSWYlDHgm+xjvDLYPrUQLh8gpN8BZWEHL4dFHueejvzyWhYFrs9q2FA2zSrtQvSi
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(366004)(346002)(136003)(396003)(38070700005)(86362001)(71200400001)(2906002)(6486002)(31696002)(5660300002)(8676002)(478600001)(66476007)(66556008)(64756008)(6512007)(76116006)(66946007)(66446008)(26005)(186003)(122000001)(53546011)(6506007)(8936002)(31686004)(54906003)(316002)(2616005)(36756003)(110136005)(4326008)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?N3ZZK1JHelZoemZkZWNnZWVVbitaSWJEdmFMUGo0SDU0Z1hWVlVhYlpqTUlM?=
 =?utf-8?B?Q2ZNdUViUGZZU0hlNCtYS2l3U2hIY2U3Q1laYzc4VDZTMHVDWEMzR1MyZ1JH?=
 =?utf-8?B?N2dBOHphNXlVMUNPQzJqT0hIUnJqWlZ5TjNUTnROYjBySkRqQUhTWXE5cGRk?=
 =?utf-8?B?dllUM09GdDROakNBKzZibXRqdHQ0cXNkWCs1YmFyK2c3c2lQczRWSk41SXcw?=
 =?utf-8?B?bG5Vc1dtaFRpcUlMZTNTOU45MW84cmpYL1J2dGFHaHZwdFB2WlEzenlEeExT?=
 =?utf-8?B?UWRwN2JabVJPZWsxM2xvSXpsckZpZzZxQTlFdCtSSkRyS24reXoxbkg2ZHdV?=
 =?utf-8?B?eU9SaE5Zbk5zNFZxOC81cDQxYmoxMUJKU1BhRW9BaDFuMHI4dVJ4UldvM3BS?=
 =?utf-8?B?bDZwMEpnOTNDQ21uUzdYMVJlUzhxb1AwZzNsVlVOMGpTT1RxbzREbGk5OU9L?=
 =?utf-8?B?WFhLWkw3cUorUkNsRW92enA3akdKa3JzMFQxcEMydFFyd2UxWGJRME1xVnNr?=
 =?utf-8?B?cE1SVDJVOGo1WU56VVpxbHJON09ySks2RjU5bXZBNkI4Q2xkTmcwNHNFaVFS?=
 =?utf-8?B?VjI3bVI3SERuLzFxN0hhdkRwamQrWkd3VnI0NVBLK3AxZjR1TTY0WmxlbG5k?=
 =?utf-8?B?eUpydlZFcktNYS9uOVpXRGlOc0U3WmVsK2lCRWhTNUxCaHFESXRTMDNFSm81?=
 =?utf-8?B?NWExQ1AvVmZ4T3h4d2w2MXI5Zm5FSDMzbTlhVG82aUdLdGFQaVpnQ3pvdHZR?=
 =?utf-8?B?U0dNNEEzUFJtMWZrME9TWEdteFpReGRIODZRWWg4RkUzdzZHdHppTGFESkFU?=
 =?utf-8?B?R2NVTHJWRmRLcGJ0bHA3a3V3c2UrSUtHMmlYQmFpSERHQTlRa3FSYzhVUi9V?=
 =?utf-8?B?dGllaDJpNzFxQ0Q3L2NMQmtVWDR1QXdiTWVmZUNYbmRldW4zYzBuN0VHUUE4?=
 =?utf-8?B?QVhRK0pFTGxVeWhaQTZFczdPN2pSdmdiSWJORW81NXc4SGxqVDVqS2FPS3Br?=
 =?utf-8?B?eTdpWmYrb2lVT2RSMWlyK1ZxYzB0VmpTaGlkTzlUbWtzRld3MGlJeDVOaTJk?=
 =?utf-8?B?cVU4bERYODVFc013QkxvWjFFTWJYMU95NnhGZFRxOCtTSlM3VTY2di8zb0Y1?=
 =?utf-8?B?dHFWb3R6WENjdVhoSWF1R1NuTmpSYkNmVndGVm5SUHpYMXBaclc0UFUwOWZL?=
 =?utf-8?B?TU9BTkFXQms2OG5NOGNleklKMlA5ZlRXd1lkRFJmVWNCMndaWjF4QWVSK3gv?=
 =?utf-8?B?RThjNFlHWHlTQUxYQmtqZ21ISTdWbjkxZzN3a3d1emZSS2ZORWxqY1g5QXVZ?=
 =?utf-8?B?U3pVSjh0K0wrZDJCeTlWV3pqNmlyaStVbkRZcVhkY2RYdTZib2FuNjArL1or?=
 =?utf-8?B?V2JtSGxnWC85MHZTMlAwYmhtRjQ0eVA4dkJPVXJNYWxSbStrdHJDUGR6UndB?=
 =?utf-8?B?c2xLRU1TYkkvcWg5V3pDR0F5SmUrYUZYcFNSZVl3eDBDczJaUW1uOTdhNVIv?=
 =?utf-8?B?aU96ZjBuQk5YSVphaXUvVU45MFlNMlJiMlpTVEFGb01KOEZqYUhRVXk1Y0t6?=
 =?utf-8?B?eUpzbzlrWVpvREQ2M1FNNEhLbWZDc0VYaGhiZmJ1bXBUT01xL29nNkpDUUtn?=
 =?utf-8?B?MjdOSGZ3cXZqUzR5QkEzVTJGckwzSUZxckZ2Rmc4em9JV0tISFY3dXpVV2hK?=
 =?utf-8?B?dXNuM3A1M3hiVHF5L2w0UTgxVlVWS3BNbUx6UFFsTUVGNDR1ZmtSeWRtRzI2?=
 =?utf-8?Q?hDDj5Cg227hz/xmD55Kx/ZzvcMyW0boOtywp53h?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <49696A9F0476AE479B033744B7C1B174@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7a40f487-d9a4-4703-fbe0-08d972de4a01
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 15:35:29.9197
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: a3atHkNmq8fArDerBUB+qPgrEUNrsoxkuI6HhPeW1FdeLIaoGQ1jvJz3xWbb7OkTer6c9IPfiooB0OSvwO12jBK6UOtDqhZDMUExEoYARZ12+ydKugQ2tp98cxqkyM2m
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3716
X-Proofpoint-GUID: syrIWgGfckKHZFzvPBYhXTIyEgoCQ7AO
X-Proofpoint-ORIG-GUID: syrIWgGfckKHZFzvPBYhXTIyEgoCQ7AO
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-08_06,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0
 suspectscore=0 mlxlogscore=884 mlxscore=0 spamscore=0 priorityscore=1501
 phishscore=0 bulkscore=0 clxscore=1015 malwarescore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109080097

DQpPbiAwOC4wOS4yMSAxODoyOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA4LjA5LjIwMjEg
MTc6MTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDguMDkuMjEgMTc6
NDYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA4LjA5LjIwMjEgMTU6MzMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBzdGF0aWMgdm9pZCBndWVzdF9iYXJfd3JpdGUo
Y29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYsIHVuc2lnbmVkIGludCByZWcsDQo+Pj4+ICAgIMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1aW50
MzJfdCB2YWwsIHZvaWQgKmRhdGEpDQo+Pj4+IHsNCj4+Pj4gICAgwqDCoMKgIHN0cnVjdCB2cGNp
X2JhciAqYmFyID0gZGF0YTsNCj4+Pj4gICAgwqDCoMKgIGJvb2wgaGkgPSBmYWxzZTsNCj4+Pj4N
Cj4+Pj4gICAgwqDCoMKgIGlmICggYmFyLT50eXBlID09IFZQQ0lfQkFSX01FTTY0X0hJICkNCj4+
Pj4gICAgwqDCoMKgIHsNCj4+Pj4gICAgwqDCoMKgwqDCoMKgwqAgQVNTRVJUKHJlZyA+IFBDSV9C
QVNFX0FERFJFU1NfMCk7DQo+Pj4+ICAgIMKgwqDCoMKgwqDCoMKgIGJhci0tOw0KPj4+PiAgICDC
oMKgwqDCoMKgwqDCoCBoaSA9IHRydWU7DQo+Pj4+ICAgIMKgwqDCoCB9DQo+Pj4+ICAgIMKgwqDC
oCBlbHNlDQo+Pj4+ICAgIMKgwqDCoCB7DQo+Pj4+ICAgIMKgwqDCoMKgwqDCoMKgIHZhbCAmPSBQ
Q0lfQkFTRV9BRERSRVNTX01FTV9NQVNLOw0KPj4+PiAgICDCoMKgwqDCoMKgwqDCoCB2YWwgfD0g
YmFyLT50eXBlID09IFZQQ0lfQkFSX01FTTMyID8gUENJX0JBU0VfQUREUkVTU19NRU1fVFlQRV8z
Mg0KPj4+PiAgICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgOiBQQ0lfQkFTRV9BRERSRVNT
X01FTV9UWVBFXzY0Ow0KPj4+PiAgICDCoMKgwqDCoMKgwqDCoCB2YWwgfD0gYmFyLT5wcmVmZXRj
aGFibGUgPyBQQ0lfQkFTRV9BRERSRVNTX01FTV9QUkVGRVRDSCA6IDA7DQo+Pj4+ICAgIMKgwqDC
oCB9DQo+Pj4+DQo+Pj4+ICAgIMKgwqDCoCBiYXItPmd1ZXN0X2FkZHIgJj0gfigweGZmZmZmZmZm
dWxsIDw8IChoaSA/IDMyIDogMCkpOw0KPj4gRG8geW91IHRoaW5rIHRoaXMgbmVlZHMgdG8gYmUg
MHhmZmZmZmZmZnVsLCBub3QgMHhmZmZmZmZmZnVsbD8NCj4+DQo+PiBlLmcuIHMvdWxsL3VsDQo+
IElmIGd1ZXN0X2FkZHIgaXMgdWludDY0X3QgdGhlbiB1bGwgd291bGQgc2VlbSBtb3JlIGNvcnJl
Y3QgdG8gbWUsDQo+IGVzcGVjaWFsbHkgd2hlbiBjb25zaWRlcmluZyAoaHlwb3RoZXRpY2FsPykg
MzItYml0IGFyY2hpdGVjdHVyZXMNCj4gcG90ZW50aWFsbHkgd2FudGluZyB0byB1c2UgdGhpcyBj
b2RlLg0KT2ssIHRoZW4gSSdsbCBrZWVwIHVsbA0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0K
T2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:37:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:37:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182257.329745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzdl-00084h-VA; Wed, 08 Sep 2021 15:37:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182257.329745; Wed, 08 Sep 2021 15:37:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzdl-00084a-Rb; Wed, 08 Sep 2021 15:37:33 +0000
Received: by outflank-mailman (input) for mailman id 182257;
 Wed, 08 Sep 2021 15:37:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NNqu=N6=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mNzdk-00084U-QJ
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:37:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ad887fd2-10ba-11ec-b168-12813bfff9fa;
 Wed, 08 Sep 2021 15:37:31 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-sFYyAxDZN2KxqbFpHhxKpQ-1; Wed, 08 Sep 2021 17:37:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4384.eurprd04.prod.outlook.com (2603:10a6:803:6f::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 8 Sep
 2021 15:37:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.014; Wed, 8 Sep 2021
 15:37:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P250CA0002.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 15:37:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad887fd2-10ba-11ec-b168-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631115450;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DukexvpXsqgxZHDUYNhgpzEDBnXDoedSFIcVjon77Aw=;
	b=XjmyGvQOJZhXKfQtdOLoVhPifQD3BGhNhhkAwp74vkKc6b7hyJGg2h8AjOgcKIgirEL83Y
	84HCBpynexmrqQu2xGxz8Q1WRgPDGFhKuquUWE5dRxiiONsHXdBZotI1LliqCckGAS4qvS
	7DisL9jOfMdr8/Tq/fkHPzMqJfm4jAk=
X-MC-Unique: sFYyAxDZN2KxqbFpHhxKpQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e+ggfs0XmjxwOcUSC+vyhA58Smp+zTzpEsuzkx9HeDIef73P6S6598eBDjncXaZdM2Lq0/cOWbWoK1mA+Iw+upll4attGjpKHJuxyk6l+Qc756vyGDMCVJSM08pzMIm3xCny8gNWgnmeSN94rl6zUlqZN1dkeSY7U4e8xm/YmPIQMjMXbuhzrwnMtPPiI+mIRgKAf9p0hfIRtQ0AjN0I2Jp7VQypx3VwyRQ2UQ4V8ccpJXi1bKNKdYnIrBvmSEaxsTQWsjHz/BEJw8koo1muTpOEf6EstCrWOBZ4R/VC1c/9nF9nIdWxIR3QPlffEaIBfALnVh6ekNjcZVoYrq6fkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DukexvpXsqgxZHDUYNhgpzEDBnXDoedSFIcVjon77Aw=;
 b=cVZdtev/k3tydGRL9RmhAQKTK38ys3MWgEB0A7Z+1rUp1ShGeI/+/9b0EvoRfdXG+aoJLo2T4zvgv6VlweDVXSXCH3QMYd7MJZ9+tcDPWR3naUCSulytDi0jfGXsv3tX0gXM8nIF2Y7GuRSV8x8R6d1SWveYdqaYuNEtsXeOagsn+A4jdXGZ+pakZoZ0x93buFPhOWYVFK/8SgAjOw2sNZ0phEfwsksPDvYUwn+j7uiOPbEqsfE8XLMXsw0Ev6sWuTVgAvc/c7/snBCTOkQxNxuba8tPZMjyisoQo8EEMDMNIFd1PS4XcZQRV9h7TthXeYdutreSikiq+BQvcxbWuA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH for-4.13] x86/PVH: Fix build following XSA-378 bugfix
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908151754.9593-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5b9c4130-4115-a08a-4c25-2014d021a2b9@suse.com>
Date: Wed, 8 Sep 2021 17:37:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210908151754.9593-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P250CA0002.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:102:57::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2ce1b94b-a057-47f3-7a89-08d972de8ff6
X-MS-TrafficTypeDiagnostic: VI1PR04MB4384:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4384831F28AADF0F60C0D0AFB3D49@VI1PR04MB4384.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2449;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	einaST4aclNrGQMTOhIEzXgSwfYGspo/M5yZ4i9O5B43S+fpfnIsHZEbj3yB7hk2PulnxlS2m/AmEu4CqHuOFwMDLvOSU48xNOVcQkKz9TDH+qjMA0HWGNE9nQLVQz9qTKSDqcRaoqm5nD6P6UobQNvYMcB9OfiyySNTwlnhK+ArMLmXrFRGBNdoKQgyAWqSTZQy2XKglkurPwUcR3w/zu4ixM/CUgYaEvlGkerXtGVGwQd4xVTFLTEZXd95oWNQSgek5K49AvdE/ewLE4Cm8ZGGkny5fUuMxoVwMV0DgFg+SvcKCj97KDOaFyzkiC60814BQ/tXkq/EHBnuOMzOU4Gea6oh7PjzSJ4uDolgwqvlDw0waXhD5Q+39e6fzTtG0F8kKLkyMK4SQ2z1lNDtnH80QEoOty3V2ut6yp4hhnDAj11BA1Kx2D+/rN5YBKcXBf+xQLmb6E2COLl6rQnQHax7JNLRGKm/EHKpcYRmAOSJHKrCtos3u9OVrgNt/Mo1lJu7NdLmnuqjYO2oDTfvbOyeqBjPYz7jlgTPIb2CGnsvE0iCKtmoylU2rGcXO2Ye3uhMLepgms7jwdx9g/tqoecWcOgRcsIdr1ffbTM8o69jTNfIehZo/oGWV2j+8hxSu/ubwviMGyoleoRl7MHLXKVUo66DF+SQfWNsTsthUu3JJkmRV4vADcZDdIEY4olNtLMEFFbYfuqrVu7kxOBIF+6LNWRjil4mGeSYA5IvSWgZl4F6ZomHseF0VBBn+OSW
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(396003)(366004)(136003)(376002)(16576012)(53546011)(6916009)(558084003)(36756003)(8676002)(956004)(186003)(8936002)(38100700002)(478600001)(5660300002)(86362001)(316002)(6486002)(4326008)(31696002)(54906003)(66946007)(66556008)(66476007)(2906002)(2616005)(26005)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YlYxRm1BQkFNRzlWZWJVUk9kdXVFRThsSEc0OWVSS2VhRXFPamhvVHlHSUUw?=
 =?utf-8?B?RkMvVlJxdGRvTnY3bTNjc3I5RWtqTXI2aTJhcHFkMzYwRzJoN3FLTTNaSnhJ?=
 =?utf-8?B?WUQ4eGxwRWNrenVUN0J0b0NKL3FJVTZ3Yjl3emUzMlRUSWpwSGo4cU1BZ1Jy?=
 =?utf-8?B?V2pzS29ISUMxb2lQUUUxYUN3NzI2cmxYK2sxZGpuMUhhMU41Q1FRWVpBYVdu?=
 =?utf-8?B?OFJBQSthTHRIdi9Helo5NWF1Ry9nT1FkL0djd09WTzJLN1NsR2VBd0g1OWRS?=
 =?utf-8?B?U0FGUzlPUFN1bGgwMVRvL0tFajR0ZFRIZHFWTkNDSENLMWlHS2VmV0w3dlp1?=
 =?utf-8?B?NUlzb2U0KzF0My9qWVQ1OTYyK0k3ZGozUHBNMEdYb2FkWUtEbWs5WkVzSS9x?=
 =?utf-8?B?enhDU0wveFYvV0dSRW4ydkxPVVBkbloxcjh5S2NUMlh6eVNTd0pFazNjeDh0?=
 =?utf-8?B?elRDUDdURzdFeEJWNlQvQVVHY0lvelRrRS93RkFIKzRuN2Nqdk9iKzMzb3p4?=
 =?utf-8?B?eFRuMUtEU3BNV1NJMURJM1haenNCd3JWYlB1V3VzaHYveUxDNi9idENTYmoz?=
 =?utf-8?B?bXhVazRIMHJSem5FOE1pWXhtYzBUcWJ2VTF3aDQzc3BybG03YWppMTdwdUtG?=
 =?utf-8?B?T3gxeVVHSC9XZ2xwc1hjZkV3OGhNQTZxVVhFelBRb2hhcmt1RGxqc3lMWm1n?=
 =?utf-8?B?Ymkxa1drY055TzlKdE9ISFRtb2hML2NpT1RLT2ZDcCtjbnM4eHJpcWo4Y0NJ?=
 =?utf-8?B?Q0h0N0NDVzQ2YUMxTStqajIzc0Rnb0Zxa2hLbjJodzJIVFJvUW43R0JJM3gr?=
 =?utf-8?B?RmxDRTBmZmd5aDdoWUZ6NFNjcWRYeGZJOU1Db2NRd0ZPMGU0ZVdjQm12WURO?=
 =?utf-8?B?UW5LTWo0WnVqSEJOSGxMT3gwdjNGV2pHK1dHTDFmTXFxeEJYTHJtWGFnOC84?=
 =?utf-8?B?bW56dSt1R2o5SlNva1JBVkNZZmsrSGtjZ25TVnBYSHpJVHZqYTlPMk1TNXlC?=
 =?utf-8?B?NElPSFBjMDJzNUtJUXFtYzZiZVRqVlJ4ZVlWb3pYclk0QzVkMzdHUTNPRU9E?=
 =?utf-8?B?ckREL0dRc0RvYytsekttWVNFZzU4Vm9LbXBHcWQ3U1lnMWUvSFoydVpCdjl1?=
 =?utf-8?B?em5mUTRNTjcwOHdDVWZpZXFRVDZlNnFwVDRlS0NvektqeTBPQWVKeWk4TEd0?=
 =?utf-8?B?cm1aSElMZ0Y0dTFIZmc2Z3A1QlkyeEovbHIrWGJlU0p4SzdLZDhubXZtVXpL?=
 =?utf-8?B?U0J6U3ovQVhwZjdQL3FrdnRMbUcxcG85YXd4VnVkTVRZYmNJdThBczNzVmRV?=
 =?utf-8?B?bk1kR0dxQVdjU1NtVGlmaWtMTm1nQ2tUUGNwVUNEVk0zcXNFbDVXejlrR05W?=
 =?utf-8?B?YnNwYmVwV2FDSXh6M29GZHVxY29RRkNBc1pZMGtINUlneGlNTkRRdzJSMEE3?=
 =?utf-8?B?U2ViNnFEY2psSjF2aHVHNHNtTWFQUXZWQ2hST2o1MW5GQlliSXFPQUUybFh5?=
 =?utf-8?B?amhqUWZrSGF1MmtVSVpuZGxwNWc4dzRlb0xsMVFSQzEyeUsvK1VqSGxOM3dx?=
 =?utf-8?B?bHlMNWVSRFN3NjZ4Q3Niayt1OTNsdTVIQzB4VGtPcG9MS0o1elNnbVZnZ3gr?=
 =?utf-8?B?Z25KRExSQzlZZ0lOUHM1OFl0SjlGUGxuSGN1UmVtT0lBTlFjNi90ak9maTR0?=
 =?utf-8?B?eXp2MzVsalY1ZW5JSmV4QzNvSS8wcXo4MTdZbDFUd3htdFZvWEVydi9XSDI2?=
 =?utf-8?Q?pHQ0VIcAFXwL4oEqU59t+XIrZ7uvvxEpJYCoq8a?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ce1b94b-a057-47f3-7a89-08d972de8ff6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 15:37:27.5136
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: E0mrMWVi13H9tXoSUS06GOMp7ml3iCazm6hTL5la7U/HXVgUS7hEY3lWL/1lNn1ZMegnOhbS/n5rsB9J53Padw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4384

On 08.09.2021 17:17, Andrew Cooper wrote:
> Fixes: 8d8b4bde3e1c ("x86/PVH: de-duplicate mappings for first Mb of Dom0 memory")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

And in it goes.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:44:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:44:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182265.329760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzkG-0001Nk-M8; Wed, 08 Sep 2021 15:44:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182265.329760; Wed, 08 Sep 2021 15:44:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzkG-0001Nd-J5; Wed, 08 Sep 2021 15:44:16 +0000
Received: by outflank-mailman (input) for mailman id 182265;
 Wed, 08 Sep 2021 15:44:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNzkF-0001NP-JT; Wed, 08 Sep 2021 15:44:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNzkF-00087z-BG; Wed, 08 Sep 2021 15:44:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mNzkF-0004xZ-24; Wed, 08 Sep 2021 15:44:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mNzkF-0001Vd-1H; Wed, 08 Sep 2021 15:44:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ZEqHeE7NkdSpRi97zH5IIY2qXIDIeIhcKvzi2dNlN2w=; b=sTQWLBX91fwLN4HmI1jF8KKs9H
	f4olU1fhDXztRVEOQcC23/OXAeklHwetBx5/YPrI6fOc4ffz8TZ2Z9apqx1Qa4v2brgYAEqsgdaS2
	1B6eMmlt1CcK1UcXzbf8tnzpG3/MDeS5RH07HDRMtwaVaMNOKcnbBL0PiYL57+6XBPU8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164880-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164880: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=8f8eeb3750ae410e4b6fa96a2a559e0d9960716c
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 15:44:15 +0000

flight 164880 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164880/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              8f8eeb3750ae410e4b6fa96a2a559e0d9960716c
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  425 days
Failing since        151818  2020-07-11 04:18:52 Z  424 days  415 attempts
Testing same since   164870  2021-09-07 04:20:08 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 71974 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 15:52:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 15:52:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182291.329791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzsP-0004Aq-UF; Wed, 08 Sep 2021 15:52:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182291.329791; Wed, 08 Sep 2021 15:52:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mNzsP-0004Aj-RE; Wed, 08 Sep 2021 15:52:41 +0000
Received: by outflank-mailman (input) for mailman id 182291;
 Wed, 08 Sep 2021 15:52:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mNzsO-0004Ad-Gg
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 15:52:40 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 32b3c195-d4b8-4e1e-962d-46fe59da3bb4;
 Wed, 08 Sep 2021 15:52:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32b3c195-d4b8-4e1e-962d-46fe59da3bb4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631116359;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=fjAUA8/SCaXIMH4joHqSzWUdjZCpXtr4qcdp6nCSKTs=;
  b=DGsPWCO9YinwRIcL9RgY1ObaZqV/TMcV6N9HJ2RzuHquNvioTvJCGnuL
   vxi8ihvLQLKlwOUxflRZspxDbT6GPbvpwStNh1NKZFCFXAJywH3nQXI2d
   uQHgMCc54WhcrWt6bOWFmiblJNTSfmIj7rnDMIOqjkOrM6cRKj6xaQW67
   0=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: dyUNGmK3EyFGR/8cEav43aTX3GNY1ZUNZ5FobnGGBq3NfBFiED8ViNb/416gxE+Sf1LW75pwAO
 esmZow9MpEHL4r7pzhPKUvup9q2J1Xlww+HKnONjqX0tCtogzCW7i/pIb5jSPU10QIHwnS1+8m
 bRVIbM1F/P6THVfQnLKpcVKZgV4uiGiYS6oup3fbihjBRSRulp3Z9EfiLSCP4k6TTPO4RrQxrO
 B1e6tYDxMpNzVGFOqb+ZCgKPzQZUvJhkjTPKQCXPL1MFrK28bqTV3OogM3TsjabipvbQ+ORpwN
 oSYpwaQsAYllfUQDMhYhw/i1
X-SBRS: 5.1
X-MesageID: 52272560
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:3fe38a7LKKbwfIBpKwPXwKvXdLJyesId70hD6qkXc3xom62j9v
 xG885x6faZslwssRIb9+xoWpPtfZq0z/cc3WB7B9iftWfd1VeVEA==
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="52272560"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mKwTmdsHdGRC6HSo8ExStgLb3rLxfm/41Z6FehcPI+OajPCBikiq3QMlNogZ9Zai8eCE9bNj/tQ/yGM3jKvfIJXxVxMERzcXHp0mojErmM69RGYjLdSWr+a29xH6HXNyOGMpo4rACVxCF597KzkCJsKWler9zycLTbXFRngBZ+yq93atI5FXdMKi1hsXl4p8qXM44S6Zz99S0gAg4dC8jgGHxrsk4wBxsoqpoQtmSuiopFaVUA/kBjDtsAufBJhCeBOsdfwJjxq0hGBZ4QNJyhYmGa5m6Fsu1pWsao4flyJGrVV17B0ecxVLm6jBTkttG2wc608iaUsmjis3I7NLCA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cqjBOhqeeDex+h7447RBggMUvcd6fvUwsbvwQzNF4rI=;
 b=NeBl3B70lWpQgYMHd/9POp6o2reksvto2E1vOYKjDyKVJ5XUH6ipqY6+39Rez+2J8v3UpUqXwMCy0BBFrymM0RgmPaRXFyM9VntItV1DRFal2DEtMUaC657ME6MOvxLOBDd8+jFXlBvOaODVw32aNFWxSYa51fYP6Z/JugjmLgkrPGxPjd8dwg/hfojxJyUsPNe4NKMyg11obYMJAb/rzU0Afz4U2zXXR+pmSy5wQ3gtGJZm+JaCG66USLLsDnhR+LpUV2qJSck7sUk3FZiEmhyZLYSnqoQI0yTv2njhPAYxqqPp4w3YosbXNlucq/7ZD8MgIHVj1rOY9F9pyMP5xg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cqjBOhqeeDex+h7447RBggMUvcd6fvUwsbvwQzNF4rI=;
 b=lxKCRwWgeOzZyi29Gd5hK8PPJxVptGk5cVuR/fVdeZc+DQTJ6lROWdJMAj5smPwsFtT6Lz1nCjoV/+KoEi8BQvI9QREnQlimHrSO/YJNXJv8OyVbWdArX8WWTTRwpX6YobG65XUW7lsEuoRwFsLSfS59/qo3IT6v/L3yAjzVXF0=
Subject: Re: [PATCH] x86/amd: Introduce and use X86_BUG_NULL_SEG
To: Jan Beulich <jbeulich@suse.com>, Pu Wen <puwen@hygon.cn>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908135931.26275-1-andrew.cooper3@citrix.com>
 <2289cadf-0dd6-0b5e-39f2-99ce6f522bfa@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <0b5c23d8-374a-27d8-76b7-e5a77951fdd6@citrix.com>
Date: Wed, 8 Sep 2021 16:52:27 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2289cadf-0dd6-0b5e-39f2-99ce6f522bfa@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0255.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:194::8) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ca70addc-f84d-4584-8d59-08d972e0ac7b
X-MS-TrafficTypeDiagnostic: BYAPR03MB4296:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4296D94D9AA353DEA7678B51BAD49@BYAPR03MB4296.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2657;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: UDKOJpyqHRCKa73Qur8P4K7HkCVzOPqUpc0Vj/RmDnXo/V9ZDtdtL/CtDr/Zuam8WHS7TPz5iVHAowhWU00mvVYyPoKAr8s+8VHuwkWdKMnE0X/9GeqOcsR+iHDarhfE5S0osJl4jOfFqPpggexSNWC0pB64H8BlUmfuUIh7aqe42VfDM8rVy4+zXmEZuzC7aNdYK8vLFyhQ0P5AhfkcKotXWzuhjJvNftfLQySyeuJOEG0e1h3aAWU4ssC2dDKBMj/RbKXN+0fhOfToSym6t/itkspopXcOOhpwPuyzzO90HOYdJMZdTpg676zy6GPhdTDB2YEf22RN6I3Vss9iPid/ck1jmDUmVjDOJcmWYSNLaPyd19tZN8/a2PT4mPnnkfQNwchr+lMRnpgRnqJla7YeMpf5MyIiWhizVZdK06zPWih6olUGliVQsUq2zS6g/YDkn5mqNFi69dxJd4BBe3CqJzcnNOdu/qQM0P/evASkOTRLzZjuyZC0nBs9rIt4TXZ2PGHN12unVrAn0AIgGf9O9YXxxxpt33YOKEdrNBC0xGRy61MWsa9aZnbHCHXME46dcmNMRRL364nhc5C7O3BbMv7nCixQjQ4OT7ppP/8lr8d81ILghxtw99qw24jdy1ijeJ0a5U+EyxZLhB59Skf1MQXcLtZzOVBz9S7m24mJiCvF2gOdV1eXnKgFq/sxAoCJzJV5JKfxB2SBrIS78c84xR2CVH7E2Mkm6m2aTiw=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(396003)(39860400002)(366004)(54906003)(110136005)(26005)(31686004)(478600001)(6486002)(4326008)(66946007)(2906002)(6666004)(186003)(2616005)(956004)(86362001)(31696002)(53546011)(55236004)(16576012)(36756003)(66556008)(316002)(66476007)(38100700002)(8676002)(8936002)(5660300002)(83380400001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QWZ6V2N0bTUwV2FFZmZvU1M0UWRQYlY4djlQODA2TDJIQnluTjFya3JrclFP?=
 =?utf-8?B?TWxseEhLTXRBQ1JlTkdRSjVVTllYRGJUTEZpamtRV3FPZ2lBcFBTMnZUVUdK?=
 =?utf-8?B?Z3VTM0RUdUN6YXZvdHc2TEtIU1ZsM1BXT1h0Q2ovaERNMTBRd3Bwc1dkeGsr?=
 =?utf-8?B?UjM1ODVETmxrQnpRRlF1dFZKcjRGc3RQalJNcTRvQ2FWaXpDNWVBdlZSZTVK?=
 =?utf-8?B?b1h5dmV2cXVDVzRxNXZDV3ZvNmUyNm1RSFh0WVdzM1JmTmpHMlRQMTBJSzJN?=
 =?utf-8?B?MXNBYjRRWmZzdVAwYnpETTBodURPYnFSdFFCMzM3Qk1rcU01WnNiUXpHbUNl?=
 =?utf-8?B?WjJWcGJia203TEtNellBczRFWm5MN0Rka3dCUGd4RlFVMDJhbzE2WHBIU216?=
 =?utf-8?B?K2xDZ0pTY3MvTndUVUhpYjR5bVdwVTBLWFJEUVFYUHlMWWVtdkI1UUlqejZB?=
 =?utf-8?B?K0JNUUhrWGR6U0JsdXBFdm9Bc3ZzUTU1T1JjL2paS1huS1lEY09oblVERk0z?=
 =?utf-8?B?T01YeFlMamw2TmtmZTBUaXNXWC8vTUZFL0p1cWVrcXc2TjIrU1ErWGptaHQw?=
 =?utf-8?B?b3owbDl6VFZXQjVTVy9PWHFwTjlpM2hscnFxVWFReW0rcjFHRmpxOHlFVjFp?=
 =?utf-8?B?eW5HejR6UU1kUTh1S1pjNWVIaHllR0g1aFd3V2tvdHdHNGp0a2QzNmY1aFBo?=
 =?utf-8?B?TzE1U25SVmpSVEFzOFpydHNRS1Q5YWxNRGM4WUZYeG5heURTdG9SUVVHS3Rm?=
 =?utf-8?B?enM2MFRtZVYwVjJsOEhRQ0F4ME1aNDg0ZWh0ejlQVHovczZmYWhaTzVoYTBB?=
 =?utf-8?B?OXNRNGRNaFZQMi82dmQ4cVl3ZVJvNDc3YnltSlpwVVE1VnRXeFlpRHJncVE1?=
 =?utf-8?B?cDhpOE5Hazg0L3VEcU1US2doRmNGYVFTSjViaGRJVDBKa1FlKzhJZFFiSzAx?=
 =?utf-8?B?aHc0blRvVHVKc2tiZG5DLytyTk5rdmZwZDhyRkJMNmsxSm55QnZ1VzdqN0JK?=
 =?utf-8?B?d0d0UWl4aDBRV3Q2YU9HanNRQjNXRFVTSnJOQ0V2cEtVc1ZMNmVPMDU3dTJh?=
 =?utf-8?B?bDdQT3QwTDhxaDJQcTIwV3ZoM2VHVUNnQlJpZndsL0xCSlBXdnoyNENpU2Jk?=
 =?utf-8?B?MWJWVjdnR2lrSnY2RjJMcGlHUDM5MzZLdXltRjdEbFNYUUtmQzUwNGpvQ1Yy?=
 =?utf-8?B?Zk5BQlUzNzZDeThrODh6cFMrU3gwODNsU0JoVnQwREJMZmlMRWV5cDdJaU9p?=
 =?utf-8?B?U2xFQk03QkpmZlZneHo5cGZvc0JNeitKYmhwd2RDRVY4MGhWbTM2eEJUWUhq?=
 =?utf-8?B?c3FHR2E1K3I4VlF0Y2dHckxlaHd0TVA1VWFCOXY4bnBzNDVCK2h6dENHZC9B?=
 =?utf-8?B?bXpqWkF2NFhhY0JyaUZCTmlxOVRYa1hSWkRDTXMzcENhMVB2c0NrU08zNGw1?=
 =?utf-8?B?MlVwWTNHMVorU1NDa0lubERRS3IyWjlIajJoOWZKOVZYTnZ4S2VIanZUQ0Ri?=
 =?utf-8?B?M0xSN2FucG1qRVZKYk9TNkFtVmgrUHVQL0MweDNwYXR5TFJlT2xxczduRGlt?=
 =?utf-8?B?aW5jNjNzejlPaXA3UnkwUmRCdlp2UEFFRktSWUNyam1aZzgzT2ZLQ1NpU3c5?=
 =?utf-8?B?UHdKL3MvbFcyemNEb1FoSUxNa3JSVlIzUXBLcG8wMnlTNEFubmo1SzFhL2hP?=
 =?utf-8?B?N3VDbGpPYzZwV3FQT0ZxcGtVNGJtVFREQmNRUTY5UW9oUjhtdDB1WVVmekRV?=
 =?utf-8?Q?6ddkmnqkP3KsRxcnfxWMFRYHQ46udi1+GKZtI1h?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ca70addc-f84d-4584-8d59-08d972e0ac7b
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 15:52:34.2854
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7oXdQb41MckH1fYw5amRVPda3FPtVuXff02h98+HxvUG6dIGyTGEMf9JPip23O2rv/T6e7DJwEjfLfz2yQPdGXIcM1IVPd3jee8PUAJxit4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4296
X-OriginatorOrg: citrix.com

On 08/09/2021 16:06, Jan Beulich wrote:
> On 08.09.2021 15:59, Andrew Cooper wrote:
>> AMD/Hygon processors before the Zen2 microarchitecture don't clear the base or
>> limit fields when loading a NULL segment.
>>
>> Express the logic in terms of cpu_bug_null_seg, and adjust the workaround in
>> do_set_segment_base().
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> albeit with a question (perhaps to Pu):
>
>> --- a/xen/arch/x86/cpu/hygon.c
>> +++ b/xen/arch/x86/cpu/hygon.c
>> @@ -40,6 +40,13 @@ static void init_hygon(struct cpuinfo_x86 *c)
>>  	    c->x86 == 0x18)
>>  		detect_zen2_null_seg_behaviour();
>>  
>> +	/*
>> +	 * Hygon CPUs before Zen2 don't clear segment bases/limits when
>> +	 * loading a NULL selector.
>> +	 */
>> +        if (c == &boot_cpu_data && !cpu_has_nscb)
>> +		setup_force_cpu_cap(X86_BUG_NULL_SEG);
> Are there any affected CPUs in reality?

Yes - the Hygon Zen1 uarch CPUs are affected.

>
> And actually (only noticing when seeing the malformed context
> above) there look to be indentation issues in the changes to
> both this file and amd.c.

Ah yes - I'll fix on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 16:05:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 16:05:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182300.329803 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO04w-0006YD-8f; Wed, 08 Sep 2021 16:05:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182300.329803; Wed, 08 Sep 2021 16:05:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO04w-0006Y6-4t; Wed, 08 Sep 2021 16:05:38 +0000
Received: by outflank-mailman (input) for mailman id 182300;
 Wed, 08 Sep 2021 16:05:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ljEy=N6=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mO04u-0006Y0-7A
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 16:05:36 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 992bc338-10be-11ec-b16a-12813bfff9fa;
 Wed, 08 Sep 2021 16:05:34 +0000 (UTC)
Received: from pps.filterd (m0246627.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 188ExKN3023705; 
 Wed, 8 Sep 2021 16:05:33 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3axd7tb7y0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 16:05:32 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 188G0LxD057687;
 Wed, 8 Sep 2021 16:05:31 GMT
Received: from nam12-mw2-obe.outbound.protection.outlook.com
 (mail-mw2nam12lp2044.outbound.protection.outlook.com [104.47.66.44])
 by userp3020.oracle.com with ESMTP id 3axst41p1y-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 08 Sep 2021 16:05:31 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB5331.namprd10.prod.outlook.com (2603:10b6:208:334::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 8 Sep
 2021 16:05:29 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4478.026; Wed, 8 Sep 2021
 16:05:29 +0000
Received: from [10.74.115.194] (138.3.201.2) by
 BYAPR04CA0009.namprd04.prod.outlook.com (2603:10b6:a03:40::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 8 Sep 2021 16:05:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 992bc338-10be-11ec-b16a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=WOotRj4nQjDKasrtz0snA9uu+YSNII+Jq0+VtpkJHbw=;
 b=Jpo9oY4Q9n1JPkT8Ldflw013e3cOnnPZkyXnY6QLRTmhpV1kafqUw0HFEPFmgQp1OOI+
 1b0lLnTJp4bseEeXSbr7y7NXok/FELfrR3xdjA0i6xwpPcbadt7ZhP1cDg1JukvRyXQO
 bQWWAwgqiRjv4bqxQ5usU147mvINmNIKCFdOw/viBQ7oNlUmZqpTRcaUquPW0TWW0u9C
 CyDYL70KUt+4BAu+u81/qUj4Eb1+K51mGLITQZ9G3vvDkC+F/SDBWzZO+LZ2Av4f4r5p
 C0wmCpOL7VStwGGNs3mS5jT05JHxAKf7GUZ2Q8vTlhdQb9vPcA0n6tTxojZUg0+uUtAp iw== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=WOotRj4nQjDKasrtz0snA9uu+YSNII+Jq0+VtpkJHbw=;
 b=TtF5RrTSGSZ6BWMHeYM1CFI8YmNAwi1QzayBlAkWFi+U0l8r5QYPPgN1PLaAnBmhLsJu
 1fOScIRwA8ZxiMFCwJT+5EMhMGSDDDhtohy9LTRAr0KZipsuK78AM+0z+Kkbn3q7al7g
 JFBuPQ4LMgaNGX+3LjmeoEu5JdqmvPZtArPCLkWl6Tm0/QUNUkhosJYaEmiE7JQWN/oZ
 Xw1wVQuDbSJU6/jojSTNaFcfuA13ecmqwqmsSn7aVhxHY01IO67w1CDlopFTcuCz24+y
 VVvRjlKk796pv70e5Fam3XaLjUleEf/Lu2dChRMQW9rM9e8KqQG4esDxEwQf/03rXPGg HQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eBtMP37QRI4HPrJ/rhg3UCgFtzCCQIaS2fn3vf7IMyokCjw74oYxHhOTxIIulA83HADjFj7GkmZ4M0tC7NXZR9iuwL5c4+JZGCvCFCj1PGaRuETgkLjFKdj5QiiRwGsYlhH5n0DPtCAKUqbxCuudV9ODuwV6biOLsD5ZDYKCyq76NgZ4moB+iOco/khDjxFcVLrDCQ0ETy73t/OW2ry2YVwVevmC4MZ/rhWn0PiA4Mf8wIk7vkh+nm2vn0gAvGZ853E/vZ8WJTaKumNXUzISPGGYugOYxPUjnq+0T1s64GP7Unac2ZE+01qWOVHujv6XDRuq1nVno9CuGoOJG8JABw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WOotRj4nQjDKasrtz0snA9uu+YSNII+Jq0+VtpkJHbw=;
 b=ct5aYIbohUwIxYkHIesKQPTqfXxKM5LjHSNn8TWV/ggHXUEBNwa8KxEtyoKGm8wdGYdh6Y0cMiQ8jWESLAVY0mbem9nITqmo1hfY43LJjePn+C8xRDBS7YYuPlADH9cPDBHmHPbaP5EC0nST0oy5HLykPKizGcP1gYlBDzRn9mPt8yNgjsIwj9xajxU8kmRoICMvrV4MltyVz3PYcW/RdnGPt//+Ce8yt5GxAKAwMsieA/mDLWHU45siezvx1KhYSxIiZqHBPdkgkekocyahH2BqpRg4n+A2LUaJtC1xGEzLbEjUnw643+PFRSzbWyVh1ebhkURKfRvZlalY9CuJMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WOotRj4nQjDKasrtz0snA9uu+YSNII+Jq0+VtpkJHbw=;
 b=QPfRNzHA40Q4NKktFC9zdeRrqopY7XIjVTQAFjlF20/jsFhp3NylDsL23Uga2WO9hQx64/uxhGAU1bUuq68rW7LG/ZbVbKeDYuftHRAm5zaFsBvfnqPWyhJeNL4KFE1axOqDzZq3kqnWrfcfSybWA0hxTXl335nYqAznyndyDoY=
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH] xen/balloon: use a kernel thread instead a workqueue
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        Jan Beulich <jbeulich@suse.com>
References: <20210827123206.15429-1-jgross@suse.com>
 <8b13525b-0075-f4c1-8900-1be151e85e3e@oracle.com>
 <9d1edc4e-b2b5-c620-17c6-31d238c2c417@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <97f394d4-6538-a931-6531-063998c6c5e2@oracle.com>
Date: Wed, 8 Sep 2021 12:05:24 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <9d1edc4e-b2b5-c620-17c6-31d238c2c417@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: BYAPR04CA0009.namprd04.prod.outlook.com
 (2603:10b6:a03:40::22) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0378668d-bfc5-4e46-2c71-08d972e27a53
X-MS-TrafficTypeDiagnostic: BLAPR10MB5331:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB5331057E2A96DA0C126CE12C8AD49@BLAPR10MB5331.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1091;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	q5eCZWhkSbC6YHbPM7FC/DDsEs5zlBlgrgn/eRctLc2rRez+PyE6z3HT6+VKXseBUscEuPnADsUW0fURDYm1mYNHNX7v1EM0q6AWHdErt1fJVD1ZlZLJM1j9+Fx/Ot6ZsRShi6al1Jxx7mwIeRHL5rJEvVsuyleZZHt1SlJtdwoFIMe18qn9auJmAhis1MjWfZEXbekUcfggjH/44glZ44gEwHL61xiXecXo0V07ICdnlO1H9Z502xvt2kaxcAv+/dfPz5WDw3KZIxEn2PJxoUgMb1WoVZC6+KdkNy8HoLGPd6S+l18OTw2fF8+jQLR2OBI9puYAxt8mZ6tf8wuQvIu5i7eouo+eiy9mzFERYadBeAQhIOCq66dT3x0iSKX3xN74rJogEucnXz2fgeV+dyx0TLIaj4eTfR5fw8wJx+nu74c9JqTQ3igWkzXnJeI7ASl5XaReXB3AweE+3J8cLvxw8pMlfM7GbLxp/gNRNW1KfwKifsPQ+EXOWBojHx4O2ZY7uJhy3Jbu2kSJ6sr16B2Prf0MA/BLIebLXv/Xe8EAubbH8fQGd/RJMkDB7+W+rIL7v6Cm76labYsp12vhi37LysZDZV4lZ0xUS0rimKV1KA7zhXe0ih8k8uL0/lUDx+upCBLJhDX7t3AiBBwvFI2uXfYLeCOEc3h2SADmG3oVmbpjVyf5VtlTnM/lYB1GXkZxA7T0z1LTzFTpe8+M8cBW470aRk8XCP6o3wz/sQs=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(396003)(366004)(39860400002)(376002)(5660300002)(956004)(2616005)(4326008)(316002)(16576012)(66946007)(66556008)(54906003)(31696002)(66476007)(44832011)(8676002)(36756003)(86362001)(4744005)(53546011)(186003)(6486002)(478600001)(2906002)(26005)(8936002)(31686004)(6666004)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ek1hSnJzU21LMGFoSk0yeHJYNm5peUFzcVF5dytIMHlBWmJMVmdFMGs5OWZa?=
 =?utf-8?B?K00rUXN0ZnNrREh6UEswYWw5a1JPZGYyQmt2bHhwUUpMWWpFMEJnUmFDcmlj?=
 =?utf-8?B?bnc2M05XR2dTSHcvYkthM2NEZzE5MzZjQjRzSnc1aTZLNFBqWlFlcElGTlFT?=
 =?utf-8?B?SE52R2svU1doMHZ2bTlGOW84bjEyNFBvTVhMbzk1d0hpNUovdDJDV2N5N1A5?=
 =?utf-8?B?cGZVMlYvcjI1VHpzZWZNSSs2ekFyS0tNSmVSNkI4RFhPN0NDZ0trOCt6RU1l?=
 =?utf-8?B?WXhOTU9HR1Y2bFBwc1ZTejJHYlJTMmd1cFYvdVFkZk5BeVJmUEJaK0lJc2Fk?=
 =?utf-8?B?THJmK3JSTHFNZXRJajF6TmQ4KzJQd3UyeDBac2dPOXczUWdwSldxa1lSNThS?=
 =?utf-8?B?Ulc0WkFZR1RLMFBCSCtKV2hQelBVTTFmZDJ4OC8vUWpsOStRMjRIajU5N0hE?=
 =?utf-8?B?enpCUHRIdlVQMEZZZEd0di85UUFuQlRmVnF0ZmpWV1ZHS2dIVk80R3JVRHha?=
 =?utf-8?B?RGQvYWU3dlVYNnJPYjZzNHVtNENpTkh0M0dUR3psRlB2VlpJSjYzWk1DS3BX?=
 =?utf-8?B?VlM5VnlCNno4UHFiZzVPRjdNc3BxSXViWTZrbXNIL01mRDl3aitJbnMrbXR0?=
 =?utf-8?B?K3lYSEJDc2ljZllNdWlwdUowM3RLVjhBaG9qVW96ZHJHbGJyY0pqSDZVbVp0?=
 =?utf-8?B?OG4vUXUxMFVQcU1ZdG4vVnlJRzlWanNGOTZkQ3ZPanF4TnQ0akV1STJQbG01?=
 =?utf-8?B?VFZkdHQraWd1ZzYvbVFYM085MEQ1M3poZEIzNUl5YXJ5YlIvVytScm5UNXpL?=
 =?utf-8?B?dVg4YzlwaTR2NnM3aFBrNWdidGpkdmpHTHBkMjNvaEExZk91RDYvbFQyWmpP?=
 =?utf-8?B?alVvZGFaL2Yvd0Y4RElyVVFDN1pmWUY3cWxKTnZmdEJFd1AxZnFULy9OUnNi?=
 =?utf-8?B?TWRvVk05QlVvVS9mcUlYMEc1b0tJSkdPVjFUWGRSdUxidHZ4LzR3dXlBcnRn?=
 =?utf-8?B?akQ5aHBWQStGMkYvMlVzUGJLR1l2WUxJWW5IZ0lIK0d1a1dBS3E0anVYdUdX?=
 =?utf-8?B?WlFpdmx1elZnTlhjMEhJRWRpRThmYmJNaGh1c2NBVVpWK0hrcndlZE5BdU0v?=
 =?utf-8?B?ekdTeWsrSmI3ZTgxdG93clhzZ0VRWlc3clJGRTVtVHFNbXdJRmNxNTZ6MGtI?=
 =?utf-8?B?QktOZVVpT2ZuWWtTdDcydy9CWjNJM2J2cDNvbDY3SVZSUkxKRUhRV0w1dDVm?=
 =?utf-8?B?eDBLMWxva09HQjNUajFHMXlJR2F1VUFTUHg0czdmb25aYUw3cnBKc0VJUlk5?=
 =?utf-8?B?a2dmeklvY3AyRXdQSUtUNjdjdGtOQ2lVbVQrVDVyVVl5citEVTZPVGhmZ3Fp?=
 =?utf-8?B?Z3YrNTYrMEIwUldsbEJ4djh6OUFQWkN2THVMZDRBNFZEdjBxMGk0YmRDa21R?=
 =?utf-8?B?VHRuNjBuTDQzWEI3OC8yc1FBTFNzQTFjKzM3L0xxR3hVN0R3UExBNVdjbzFY?=
 =?utf-8?B?VkFydVYvYnZWTVZ2MDdpYUNvTEhWRXlkdDQ1Vk1DRlVtSGtVeEJqZW1WOHdE?=
 =?utf-8?B?SnlHTU4zRHdvRFhKUk9wSUJOVFc5VTVIK0FUV3NITDJ6M0kvbFVRdjBaeVpJ?=
 =?utf-8?B?SFF3Qzk0YVZHUTg3WDNnclVsdkdsMkVZTnVBaWtVcE5jeG1ZamEzaStVd1k3?=
 =?utf-8?B?SFIxUWhqeDhuVUExRHVSNHZxN21lVVJrUUsrUWpsTGZvWHlnR2NGbGtSZXVl?=
 =?utf-8?Q?UYH570yQo2328mqNq7qGdWS2kNoIlIf11RqSnuq?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0378668d-bfc5-4e46-2c71-08d972e27a53
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2021 16:05:29.1137
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0yBje2jKz4kx2VE4rwXMeQ1DDsT5NjlruM6Y12QVcctF71VT1NeYodt4FySEq6oWb5AC1Wl8cKsMyX1VTCp1ma7Rm6Wo0c1P3l/83zX5S2Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5331
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10101 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0
 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080100
X-Proofpoint-GUID: LTVQLA2FxSYhyNO8pqma16wmNm95498a
X-Proofpoint-ORIG-GUID: LTVQLA2FxSYhyNO8pqma16wmNm95498a


On 9/8/21 11:11 AM, Juergen Gross wrote:
> On 08.09.21 16:47, Boris Ostrovsky wrote:
>>
>>
>> Given that wait_event_interruptible_timeout() is a bunch of nested macros do we need to worry here about overly aggressive compiler optimizing out 'credit = current_credit()'?
>
> I don't think so. current_credit() is reading from balloon_stats, which
> is a global variable. So the compiler shouldn't assume the contents
> won't change.


Ah, ok --- good point. Then I guess we should be fine.


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>


>
> But I can add a barrier() after 'credit = current_credit()' in case
> you'd feel uneasy without it.
>
>
> Juergen


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 16:19:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 16:19:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182309.329814 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO0IN-0008Ld-Cn; Wed, 08 Sep 2021 16:19:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182309.329814; Wed, 08 Sep 2021 16:19:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO0IN-0008LW-9R; Wed, 08 Sep 2021 16:19:31 +0000
Received: by outflank-mailman (input) for mailman id 182309;
 Wed, 08 Sep 2021 16:19:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mO0IL-0008LO-It
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 16:19:29 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8a063f44-10c0-11ec-b16a-12813bfff9fa;
 Wed, 08 Sep 2021 16:19:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a063f44-10c0-11ec-b16a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631117968;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=1PNTtirelvdOFT8pkqCJx/nKsOd1/I276VVJU/Bt6vI=;
  b=IWc1XbUwloA1pHE9V19cusa0TBEUUkfCsg+My5vnvqWyAHll9vZeDpCR
   d5S2g7hXNIeDHgICVJPJ0LnyY6ZSe4mjzHKmsLv7mOXIsB7Gh5NE/kZEo
   ecNbrAfbBV8oqhl4GFkLXFaog9cc00OGfyGLumuwOqn0RR+u67UtMEA2G
   U=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: bkjV2uRkmrFZZolAqDluZKT+JnhJV18b6j/tit/bmUnJH67XO66aCLtKWco9+WH5V9C9TirUWZ
 n3wrfb6POp03jg8U5v8CUMBeoL3Nr1LOm1EazIxlIM4esLwYSk0txYBLkrsf5p9z7B5qq3qnLO
 GCJV6Oe5F4kSZiOuqOFf0BrHgW5GJMF6VGeQpXMbiuQiELa2BBDktCDaJGeQO1aqp8PiFydgl5
 J+59XL6UAyWPNNv586YhcND1rDzyEtVOeDBD6JDR+hyuMmJCjgxGsdVjbZPdGnnD1zz6cZsklU
 9EB5skYzI59s1POMZFmRGrzq
X-SBRS: 5.1
X-MesageID: 52289560
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:c/rKd6EwjPGy2QajpLqE5MeALOsnbusQ8zAXP0AYc3Jom6uj5q
 eTdZUgpHvJYVkqOE3I9ertBEDiewK4yXcW2/hzAV7KZmCP0wHEEGgL1/qF/9SKIUzDH4Bmup
 uIC5IOauHNMQ==
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="52289560"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH] x86/svm: Intercept and terminate RDPRU with #UD
Date: Wed, 8 Sep 2021 17:19:18 +0100
Message-ID: <20210908161918.25911-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

The RDPRU instruction isn't supported at all (and it is unclear how this can
ever be offered safely to guests).  However, a guest which ignores CPUID and
blindly executes RDPRU will find that it functions.

Use the intercept and terminate with #UD.  While at it, fold SKINIT into the
same "unconditionally disabled" path.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

I could have sworn that I'd posted this before, but I can't locate any
evidence of it.  I've got a separate patch adding the CPUID infrastructure for
rdpru, but that is better left until we've got more libx86 levelling logic in
place.
---
 xen/arch/x86/hvm/svm/svm.c         | 5 ++---
 xen/arch/x86/hvm/svm/vmcb.c        | 3 ++-
 xen/include/asm-x86/hvm/svm/vmcb.h | 4 +++-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 8dc92c8b9f96..18c4831f98ad 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2945,6 +2945,8 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
 
     case VMEXIT_MONITOR:
     case VMEXIT_MWAIT:
+    case VMEXIT_SKINIT:
+    case VMEXIT_RDPRU:
         hvm_inject_hw_exception(TRAP_invalid_op, X86_EVENT_NO_EC);
         break;
 
@@ -2963,9 +2965,6 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     case VMEXIT_CLGI:
         svm_vmexit_do_clgi(regs, v);
         break;
-    case VMEXIT_SKINIT:
-        hvm_inject_hw_exception(TRAP_invalid_op, X86_EVENT_NO_EC);
-        break;
 
     case VMEXIT_XSETBV:
         if ( vmcb_get_cpl(vmcb) )
diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c
index 55da9302e5d7..565e997155f2 100644
--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -70,7 +70,8 @@ static int construct_vmcb(struct vcpu *v)
         GENERAL2_INTERCEPT_STGI        | GENERAL2_INTERCEPT_CLGI        |
         GENERAL2_INTERCEPT_SKINIT      | GENERAL2_INTERCEPT_MWAIT       |
         GENERAL2_INTERCEPT_WBINVD      | GENERAL2_INTERCEPT_MONITOR     |
-        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP;
+        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP       |
+        GENERAL2_INTERCEPT_RDPRU;
 
     /* Intercept all debug-register writes. */
     vmcb->_dr_intercepts = ~0u;
diff --git a/xen/include/asm-x86/hvm/svm/vmcb.h b/xen/include/asm-x86/hvm/svm/vmcb.h
index 4fa2ddfb2ff2..ed7cebea7174 100644
--- a/xen/include/asm-x86/hvm/svm/vmcb.h
+++ b/xen/include/asm-x86/hvm/svm/vmcb.h
@@ -74,7 +74,8 @@ enum GenericIntercept2bits
     GENERAL2_INTERCEPT_MONITOR = 1 << 10,
     GENERAL2_INTERCEPT_MWAIT   = 1 << 11,
     GENERAL2_INTERCEPT_MWAIT_CONDITIONAL = 1 << 12,
-    GENERAL2_INTERCEPT_XSETBV  = 1 << 13
+    GENERAL2_INTERCEPT_XSETBV  = 1 << 13,
+    GENERAL2_INTERCEPT_RDPRU   = 1 << 14,
 };
 
 
@@ -300,6 +301,7 @@ enum VMEXIT_EXITCODE
     VMEXIT_MWAIT            = 139, /* 0x8b */
     VMEXIT_MWAIT_CONDITIONAL= 140, /* 0x8c */
     VMEXIT_XSETBV           = 141, /* 0x8d */
+    VMEXIT_RDPRU            = 142, /* 0x8e */
     VMEXIT_NPF              = 1024, /* 0x400, nested paging fault */
     VMEXIT_INVALID          =  -1
 };
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 16:34:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 16:34:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182315.329824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO0Wo-0002kY-NG; Wed, 08 Sep 2021 16:34:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182315.329824; Wed, 08 Sep 2021 16:34:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO0Wo-0002kR-KG; Wed, 08 Sep 2021 16:34:26 +0000
Received: by outflank-mailman (input) for mailman id 182315;
 Wed, 08 Sep 2021 16:34:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO0Wm-0002kH-Fz; Wed, 08 Sep 2021 16:34:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO0Wm-00018r-B2; Wed, 08 Sep 2021 16:34:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO0Wm-000804-2Y; Wed, 08 Sep 2021 16:34:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mO0Wm-0002w1-25; Wed, 08 Sep 2021 16:34:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uMQZC91fEGXfIPhUpFNKewu0rEdcYQDDhq2YhohVfxY=; b=Imu9KoghtSerGp2FwFTjPYJzgK
	WdzlMhlEfbejxYapadzwr8dzGFPEEb0xsIjb1ZFW0cWJ3nvSbAFlUSmJ9x9U5kKBb7y2s6osB/rD+
	t61P3xQIYSIXJTCv7lSdzDUszSna0uQ1N+TIhEjMklGFBYVA77R020Qae+yTygLiBhXI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164885-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164885: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c81e7efe2146c8f381fbdbb037b9d46866a6451e
X-Osstest-Versions-That:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 16:34:24 +0000

flight 164885 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164885/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c81e7efe2146c8f381fbdbb037b9d46866a6451e
baseline version:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1

Last test of basis   164875  2021-09-07 13:00:24 Z    1 days
Testing same since   164885  2021-09-08 13:01:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e70a9a043a..c81e7efe21  c81e7efe2146c8f381fbdbb037b9d46866a6451e -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 17:34:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 17:34:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182328.329846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO1Sd-00024f-EH; Wed, 08 Sep 2021 17:34:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182328.329846; Wed, 08 Sep 2021 17:34:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO1Sd-00024Y-BQ; Wed, 08 Sep 2021 17:34:11 +0000
Received: by outflank-mailman (input) for mailman id 182328;
 Wed, 08 Sep 2021 17:34:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hF9z=N6=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mO1Sb-00024S-Jv
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 17:34:09 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f8724946-10ca-11ec-b174-12813bfff9fa;
 Wed, 08 Sep 2021 17:34:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8724946-10ca-11ec-b174-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631122448;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=nqVi/a8THNB+9s/Zs2ox9A9xaFtj4KKqUP7zrMtJP/A=;
  b=CscAQ84fHrPZMPe2ba4gSUeTeA3/sL7nkt4EDfosNyltRVgqEDmEIUNV
   ceOnfRIUKqp8DlFXo8b2jxsOEQ0Qp8x+lz/sruuQ2ErgXb9LRgzr3mIHG
   KTG0GJzkZlpYsW3mKCXdj/Xzil5aO896+wvyxmGDerkV7zjYENamDm5Xd
   U=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: +DeGpNH8Ob149aRf5A3cX5+j0LIDBQtMr5ter6u3fEXv8XTipAfKDsAiRodQnNK/kv1Ww1LnjH
 78dC3q5MlUUnqyWGu8Od1e9vKerZc1/EvvfRrl1i3bwUfeD3Zfc9O2KNV/B3HYsorUaApYEdeA
 4q17Qa7zbP6MzB/jnUORKW2DQjX7mUT2qtqoW5tp20koHkhIwiJHzNmsRXANNCR1pIwJhR59ks
 I6//L8ws00dDUCSwgupNTxVU9M4S397ERm6xm6DNmDnFKgThn/Gi/gRoSlU6qNBDbqKMCYaH08
 kh3kZQF2O+YaLA4V9J30jOzq
X-SBRS: 5.1
X-MesageID: 54047157
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:sxhkfqujuGN39VpBLDREI11u7skDeNV00zEX/kB9WHVpmszxra
 GTdZMgpHnJYVcqKRYdcL+7Scq9qB/nmqKdgrNhWYtKPjOW2ldARbsKheCJrlHd8kXFh5dgPM
 xbE5SWZuefMbEDt7ee3DWF
X-IronPort-AV: E=Sophos;i="5.85,278,1624334400"; 
   d="scan'208";a="54047157"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH] x86/spec-ctrl: Print all AMD speculative hints/features
Date: Wed, 8 Sep 2021 18:33:52 +0100
Message-ID: <20210908173352.3627-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

We already print Intel features that aren't yet implemented/used, so be
consistent on AMD too.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

For reference, here are the bit layouts on different generations of AMD CPUs.

Fam15, and Zen1:
  (XEN)   Hardware hints:
  (XEN)   Hardware features: IBPB

Zen2:
  (XEN)   Hardware hints: IBRS_FAST IBRS_SAME_MODE
  (XEN)   Hardware features: IBPB IBRS STIBP SSBD

Zen3:
  (XEN)   Hardware hints: STIBP_ALWAYS IBRS_FAST IBRS_SAME_MODE
  (XEN)   Hardware features: IBPB IBRS STIBP SSBD PSFD
---
 xen/arch/x86/spec_ctrl.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index f0c67d41b85f..16d2a1d172b5 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -321,7 +321,7 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
      * Hardware read-only information, stating immunity to certain issues, or
      * suggestions of which mitigation to use.
      */
-    printk("  Hardware hints:%s%s%s%s%s%s%s\n",
+    printk("  Hardware hints:%s%s%s%s%s%s%s%s%s%s%s\n",
            (caps & ARCH_CAPS_RDCL_NO)                        ? " RDCL_NO"        : "",
            (caps & ARCH_CAPS_IBRS_ALL)                       ? " IBRS_ALL"       : "",
            (caps & ARCH_CAPS_RSBA)                           ? " RSBA"           : "",
@@ -329,16 +329,23 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
            (e8b  & cpufeat_mask(X86_FEATURE_SSB_NO)) ||
            (caps & ARCH_CAPS_SSB_NO)                         ? " SSB_NO"         : "",
            (caps & ARCH_CAPS_MDS_NO)                         ? " MDS_NO"         : "",
-           (caps & ARCH_CAPS_TAA_NO)                         ? " TAA_NO"         : "");
+           (caps & ARCH_CAPS_TAA_NO)                         ? " TAA_NO"         : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_IBRS_ALWAYS))    ? " IBRS_ALWAYS"    : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_STIBP_ALWAYS))   ? " STIBP_ALWAYS"   : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_IBRS_FAST))      ? " IBRS_FAST"      : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_IBRS_SAME_MODE)) ? " IBRS_SAME_MODE" : "");
 
     /* Hardware features which need driving to mitigate issues. */
-    printk("  Hardware features:%s%s%s%s%s%s%s%s%s\n",
+    printk("  Hardware features:%s%s%s%s%s%s%s%s%s%s\n",
            (e8b  & cpufeat_mask(X86_FEATURE_IBPB)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB))          ? " IBPB"           : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_IBRS)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_IBRSB))          ? " IBRS"           : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_AMD_STIBP)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_STIBP))          ? " STIBP"          : "",
            (e8b  & cpufeat_mask(X86_FEATURE_AMD_SSBD)) ||
            (_7d0 & cpufeat_mask(X86_FEATURE_SSBD))           ? " SSBD"           : "",
+           (e8b  & cpufeat_mask(X86_FEATURE_PSFD))           ? " PSFD"           : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_L1D_FLUSH))      ? " L1D_FLUSH"      : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_MD_CLEAR))       ? " MD_CLEAR"       : "",
            (_7d0 & cpufeat_mask(X86_FEATURE_SRBDS_CTRL))     ? " SRBDS_CTRL"     : "",
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 20:32:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 20:32:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182378.329894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO4F0-0000ZN-2h; Wed, 08 Sep 2021 20:32:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182378.329894; Wed, 08 Sep 2021 20:32:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO4Ez-0000ZG-VQ; Wed, 08 Sep 2021 20:32:17 +0000
Received: by outflank-mailman (input) for mailman id 182378;
 Wed, 08 Sep 2021 20:32:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4Ey-0000Z6-0u; Wed, 08 Sep 2021 20:32:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4Ex-0005Py-NN; Wed, 08 Sep 2021 20:32:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4Ex-0003TD-Dm; Wed, 08 Sep 2021 20:32:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4Ex-000258-C1; Wed, 08 Sep 2021 20:32:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=GiT0jLLSl2QSQgEU7EJ7AAHfE0sVmSOWoHk0F2r9wAs=; b=unz96WvKJsvVyfAE3/dCZVhJ5A
	Fay270IMrCChAgHUHbLggS5PbL6b1yMafOQy1bxTUWFjZcco+J45hxB8KPCFFS1ikf6l+rXW1vhaS
	YRJdL8HoljRJWfSnEqB+XtP6tayIb4A0tzuW9H5JExVJbxgmBOpECRMggRvaCE2opbPk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164891-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164891: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=56abcf1a58bdaef18579cf2ce8645c8c72a2b749
X-Osstest-Versions-That:
    xen=c81e7efe2146c8f381fbdbb037b9d46866a6451e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 20:32:15 +0000

flight 164891 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164891/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  56abcf1a58bdaef18579cf2ce8645c8c72a2b749
baseline version:
 xen                  c81e7efe2146c8f381fbdbb037b9d46866a6451e

Last test of basis   164885  2021-09-08 13:01:33 Z    0 days
Testing same since   164891  2021-09-08 17:01:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c81e7efe21..56abcf1a58  56abcf1a58bdaef18579cf2ce8645c8c72a2b749 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 20:44:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 20:44:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182385.329908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO4QR-0002LH-6p; Wed, 08 Sep 2021 20:44:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182385.329908; Wed, 08 Sep 2021 20:44:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO4QR-0002L9-36; Wed, 08 Sep 2021 20:44:07 +0000
Received: by outflank-mailman (input) for mailman id 182385;
 Wed, 08 Sep 2021 20:44:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4QQ-0002L0-OY; Wed, 08 Sep 2021 20:44:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4QQ-0005bu-GA; Wed, 08 Sep 2021 20:44:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4QQ-0004Tu-5n; Wed, 08 Sep 2021 20:44:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mO4QQ-00019Q-5K; Wed, 08 Sep 2021 20:44:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=D4edtT3c6HEJKSAyOBYS3vl9v+EwijGKjIpENMEz+CE=; b=WXxi5pM52mmSwzxWrkibSeDOnU
	cvlAcAnZLBZZFWmbci7QGYOMtqn0n/Ho2fcVl/kefxoDkZ/b1M5wzDEC9+P7zFhIs+jDkFdMR8dlx
	NHsMHe4oLaLZPyA1jYNCpZ09jLD4iptokXYPeAkmYJSyEA4CgMrkZjpOgMGPul0iGDBI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164883-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164883: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
X-Osstest-Versions-That:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 08 Sep 2021 20:44:06 +0000

flight 164883 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164883/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164877
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164877
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164877
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164877
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164877
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164877
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164877
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164877
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164877
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164877
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164877
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1
baseline version:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1

Last test of basis   164883  2021-09-08 10:10:28 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 22:32:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 22:32:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182396.329921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO66u-0007GI-BZ; Wed, 08 Sep 2021 22:32:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182396.329921; Wed, 08 Sep 2021 22:32:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO66u-0007GB-8U; Wed, 08 Sep 2021 22:32:04 +0000
Received: by outflank-mailman (input) for mailman id 182396;
 Wed, 08 Sep 2021 22:32:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO66s-0007G5-DN
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 22:32:02 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 954d0cc8-10f4-11ec-b18a-12813bfff9fa;
 Wed, 08 Sep 2021 22:32:00 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E519060F5B;
 Wed,  8 Sep 2021 22:31:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 954d0cc8-10f4-11ec-b18a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631140320;
	bh=dWob8UzTP/GdFG9k8M2M0wr2Pyydbr6toPp0fjLQU3k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OpNHIu7F+1sUxUIWNOeJM1LzdOH8Xv/CP6ChtErTxekbmpFXx+FnXvsi0ij/tJmqS
	 c3M7JVgaIjGfzRltTgMWvHqzlss/gJScVNLY3oGke0NMxE70SsfaUVhP1haqYbM7Hl
	 9Qky0SQrubycuODG8LfzZL+MvL9bZEybcTS439KYEqJZ/vIajBlA+uW9n1EYuSLmmn
	 VK2IotCUIJIhs99RekzVV3W9+VmwaI14uBmbTL4K7C0NwKmADBajAzzrk713xvqO2W
	 Cldfel+FeR7CMdiyuZQIAJGNV/cXfj8Nh7TgOrH2a/2NtzzIqHaNNiT3GSgaBZ4rIx
	 3N1I73IYhVBoQ==
Date: Wed, 8 Sep 2021 15:31:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Jan Beulich <jbeulich@suse.com>
Subject: RE: [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse device
 tree memory node
In-Reply-To: <DB9PR08MB68572B980E3E688F3E946C329ED49@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109081458300.10523@sstabellini-ThinkPad-T480s>
References: <20210811102423.28908-1-wei.chen@arm.com> <20210811102423.28908-24-wei.chen@arm.com> <alpine.DEB.2.21.2108271757250.17851@sstabellini-ThinkPad-T480s> <DB9PR08MB6857618112DC045293A449699EC99@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <5c9a196a-8ac3-1173-f402-01d99ccc23e4@xen.org> <AS8PR08MB68531741D1035B35A19EF34F9EC99@AS8PR08MB6853.eurprd08.prod.outlook.com> <DB9PR08MB68572B980E3E688F3E946C329ED49@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Wei Chen wrote:
> > > >>> @@ -55,6 +57,79 @@ static int __init
> > > >> dtb_numa_processor_affinity_init(nodeid_t node)
> > > >>>       return 0;
> > > >>>   }
> > > >>>
> > > >>> +/* Callback for parsing of the memory regions affinity */
> > > >>> +static int __init dtb_numa_memory_affinity_init(nodeid_t node,
> > > >>> +                                paddr_t start, paddr_t size)
> > > >>> +{
> > > >>> +    struct node *nd;
> > > >>> +    paddr_t end;
> > > >>> +    int i;
> > > >>> +
> > > >>> +    if ( srat_disabled() )
> > > >>> +        return -EINVAL;
> > > >>> +
> > > >>> +    end = start + size;
> > > >>> +    if ( num_node_memblks >= NR_NODE_MEMBLKS )
> > > >>> +    {
> > > >>> +        dprintk(XENLOG_WARNING,
> > > >>> +                "Too many numa entry, try bigger NR_NODE_MEMBLKS
> > \n");
> > > >>> +        bad_srat();
> > > >>> +        return -EINVAL;
> > > >>> +    }
> > > >>> +
> > > >>> +    /* It is fine to add this area to the nodes data it will be
> > used
> > > >> later */
> > > >>> +    i = conflicting_memblks(start, end);
> > > >>> +    /* No conflicting memory block, we can save it for later usage
> > */;
> > > >>> +    if ( i < 0 )
> > > >>> +        goto save_memblk;
> > > >>> +
> > > >>> +    if ( memblk_nodeid[i] == node ) {
> > > >>> +        /*
> > > >>> +         * Overlaps with other memblk in the same node, warning
> > here.
> > > >>> +         * This memblk will be merged with conflicted memblk later.
> > > >>> +         */
> > > >>> +        printk(XENLOG_WARNING
> > > >>> +               "DT: NUMA NODE %u (%"PRIx64
> > > >>> +               "-%"PRIx64") overlaps with itself (%"PRIx64"-
> > > >> %"PRIx64")\n",
> > > >>> +               node, start, end,
> > > >>> +               node_memblk_range[i].start,
> > node_memblk_range[i].end);
> > > >>> +    } else {
> > > >>> +        /*
> > > >>> +         * Conflict with memblk in other node, this is an error.
> > > >>> +         * The NUMA information is invalid, NUMA will be turn off.
> > > >>> +         */
> > > >>> +        printk(XENLOG_ERR
> > > >>> +               "DT: NUMA NODE %u (%"PRIx64"-%"
> > > >>> +               PRIx64") overlaps with NODE %u (%"PRIx64"-
> > > %"PRIx64")\n",
> > > >>> +               node, start, end, memblk_nodeid[i],
> > > >>> +               node_memblk_range[i].start,
> > node_memblk_range[i].end);
> > > >>> +        bad_srat();
> > > >>> +        return -EINVAL;
> > > >>> +    }
> > > >>> +
> > > >>> +save_memblk:
> > > >>> +    nd = &nodes[node];
> > > >>> +    if ( !node_test_and_set(node, memory_nodes_parsed) ) {
> > > >>> +        nd->start = start;
> > > >>> +        nd->end = end;
> > > >>> +    } else {
> > > >>> +        if ( start < nd->start )
> > > >>> +            nd->start = start;
> > > >>> +        if ( nd->end < end )
> > > >>> +            nd->end = end;
> > > >>> +    }
> > > >>> +
> > > >>> +    printk(XENLOG_INFO "DT: NUMA node %u %"PRIx64"-%"PRIx64"\n",
> > > >>> +           node, start, end);
> > > >>> +
> > > >>> +    node_memblk_range[num_node_memblks].start = start;
> > > >>> +    node_memblk_range[num_node_memblks].end = end;
> > > >>> +    memblk_nodeid[num_node_memblks] = node;
> > > >>> +    num_node_memblks++;
> > > >>
> > > >>
> > > >> Is it possible to have non-contigous ranges of memory for a single
> > NUMA
> > > >> node?
> > > >>
> > > >> Looking at the DT bindings and Linux implementation, it seems
> > possible.
> > > >> Here, it seems that node_memblk_range/memblk_nodeid could handle it,
> > > >> but nodes couldn't.
> > > >
> > > > Yes, you're right. I copied this code for x86 ACPI NUMA. Does ACPI
> > allow
> > > > non-contiguous ranges of memory for a single NUMA node too?
> > >
> > > I couldn't find any restriction for ACPI. Although, I only briefly
> > > looked at the spec.
> > >
> > > > If yes, I think
> > > > this will affect x86 ACPI NUMA too. In next version, we plan to merge
> > > > dtb_numa_memory_affinity_init and acpi_numa_memory_affinity_init into
> > a
> > > > neutral function. So we can fix them at the same time.
> > > >
> > > > If not, maybe we have to keep the diversity for dtb and ACPI here.
> > >
> > > I am not entirely sure what you mean. Are you saying if ACPI doesn't
> > > allow non-contiguous ranges of memory, then we should keep the
> > > implementation separated?
> > >
> > > If so, then I disagree with that. It is fine to have code that supports
> > > more than what a firmware table supports. The main benefit is less code
> > > and therefore less long term maintenance (with the current solution we
> > > would need to check both the ACPI and DT implementation if there is a
> > > bug in one).
> > >
> > 
> > Yes, I agree.
> > 
> 
> I am looking for some methods to address this comment. Current "nodes"
> has not considered the situation of memory addresses of different NUMA
> nodes can be interleaved.
> 
> This code exists in x86 NUMA implementation. I think it may be based on
> one early version of Linux x86 NUMA implementation. In recent Linux
> code, both ACPI/numa/srat.c[1] and x86 NUMA code[2] are not using
> "nodes" to record NUMA memory address boundary. They don't depend
> on "nodes" to do sanity check.
> 
> To fix it, we'd better to upgrade the x86 NUMA driver. It will make
> a great affect for Xen-x86. And I think it might out of this series
> scope. Can we create another thread to discuss about it?
> 
> Or could you give me suggestions that we can use some simple ways
> to fix it?

It looks like that we would have to replace all the node->start /
node->end checks with node_memblk_range checks. There are a few of them
in valid_numa_range, conflicting_memblks, cutoff_node,
nodes_cover_memory. It wouldn't be trivial.

Although I do think that non-contiguous memory for NUMA nodes is
important to support, the patch series is already 40 patches. I don't
think it is a good idea to add other significant changes to it. I
wouldn't upgrade the x86 NUMA driver now. If we can't find a better way,
we can proceed as you are doing in this version, with the known gap that
we can't deal with non-contigious memory for NUMA nodes, and fix it with
a follow-up series later. In that case we would want to have an explicit
check for non-contiguous memory for NUMA nodes in
dtb_numa_memory_affinity_init and error out if found.


> Also, on Linux, NUMA implementations for x86 are different from Arm64
> and RISC-V implementations.[3]
> 
> [1] https://github.com/torvalds/linux/blob/master/drivers/acpi/numa/srat.c
> [2] https://github.com/torvalds/linux/blob/master/arch/x86/mm/numa.c
> [3] https://github.com/torvalds/linux/blob/master/drivers/base/arch_numa.c


In general, I like the idea of sharing code as much as possible between
architectures (x86, ARM, etc.) and between DT/ACPI because it makes the
code maintainance easier and one might even gain certain features for
free.

However, the excercise of sharing code shouldn't take significant
additional efforts. In fact, it should decrease the overall effort:
instead of writing new code one just take existing code and move it to
common.

In this instance, I think it would be good to be able to share the NUMA
initialization code between x86/ARM and ACPI/DT if it doesn't cause
extra efforts.

Here the extra effort that my previous comment might cause doesn't
derive from x86/ARM or DT/ACPI code sharing. It derives from the fact
that our existing code doesn't deal with non-contigous memory for NUMA
nodes unfortunately. That is something we need to find a way to cope
with anyway.


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182404.329932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6rp-00050j-63; Wed, 08 Sep 2021 23:20:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182404.329932; Wed, 08 Sep 2021 23:20:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6rp-00050c-2z; Wed, 08 Sep 2021 23:20:33 +0000
Received: by outflank-mailman (input) for mailman id 182404;
 Wed, 08 Sep 2021 23:20:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6rn-00050U-Vj
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:32 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 5b4f9728-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:20:29 +0000 (UTC)
Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com
 [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-313-k9Fk0EiOM2OnB9yNGWAc7Q-1; Wed, 08 Sep 2021 19:20:27 -0400
Received: by mail-wm1-f71.google.com with SMTP id
 r4-20020a1c4404000000b002e728beb9fbso30514wma.9
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:27 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id s7sm16447wra.75.2021.09.08.16.20.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b4f9728-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143229;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=vRoi5WrTbIXmM4WjAcv/SppS1KuoM6iQfhbpZ457xYQ=;
	b=ZgpUEbRQi5VpwPemN4cpd8DV4KvE0Qt0ufwHkVeS8WMItrjXojAB+Lt1b+6uESLbvqbxUv
	fvDuhgmgQyVRDCceaJ90JNkn2hSlHRZz1ibd1ZYtAkYJ3q8Ns72anq9kGrPzY6Cg/PNX19
	tzeKJoOKQ3w85uH3sCirjGThqrgnc/M=
X-MC-Unique: k9Fk0EiOM2OnB9yNGWAc7Q-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=1ByN3Of2ZgwJxeRnuvE2fFlKqpGdp9A80ERs6ZdUaDo=;
        b=Qos5RLMvNCe+978ZrzCEkh/589yrXoJ7/OaBEZYQ5vLC2Z7HEGfcdzUnvOP60NIVvH
         RcQuwC61xTMx+84jBqvI1mzJwtqUO5MFqtyqnhJFHFMzs0f9sDjjlnDIrcCpfYCEfC9f
         bIKXjnrWOeM2gEEiBdgPYCC/Fo9xzC2IIqeJ3bLZlVSv1zIdnPdOAJhtwNfDthmAIbsC
         ntCOCLmWS9TWOQzQiXpxoOvNG4rUocGwJZl3im5ZBKp2Psuo05CEFeh6VRRd8qYGLVOJ
         Kg2cmSIPWYQm7dmH2cXaejI/Hp9dM7pgXQC2uKX6e5QyA1vYjLCz7UBn9+CCODruWWYo
         y+ng==
X-Gm-Message-State: AOAM530Oa0fhUgqDSyS8lCeRFXYSS/kiFPp0WB3qYTRwX1L5daLoVUa8
	zFowBQj8G2MpVNuYVTvnJMYT4LqiJ+Ew1Lx7oESVqSBP+1Y6URXpIFDb95lihgdcL7GtUJDVt/h
	Eo2eOGKG+LEwr3+U1rdTNtd+eRUM=
X-Received: by 2002:adf:cf0b:: with SMTP id o11mr116905wrj.72.1631143226207;
        Wed, 08 Sep 2021 16:20:26 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJwJzCKhor1xK77PrKMQ04S9HdElVG3wG7IduY7aZzH1YZcWGdPgvZWSbOwy0GcAZECXsMDD+g==
X-Received: by 2002:adf:cf0b:: with SMTP id o11mr116900wrj.72.1631143226045;
        Wed, 08 Sep 2021 16:20:26 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 00/10] security: Introduce qemu_security_policy_taint() API
Date: Thu,  9 Sep 2021 01:20:14 +0200
Message-Id: <20210908232024.2399215-1-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hi,=0D
=0D
This series is experimental! The goal is to better limit the=0D
boundary of what code is considerated security critical, and=0D
what is less critical (but still important!).=0D
=0D
This approach was quickly discussed few months ago with Markus=0D
then Daniel. Instead of classifying the code on a file path=0D
basis (see [1]), we insert (runtime) hints into the code=0D
(which survive code movement). Offending unsafe code can=0D
taint the global security policy. By default this policy=0D
is 'none': the current behavior. It can be changed on the=0D
command line to 'warn' to display warnings, and to 'strict'=0D
to prohibit QEMU running with a tainted policy.=0D
=0D
As examples I started implementing unsafe code taint from=0D
3 different pieces of code:=0D
- accelerators (KVM and Xen in allow-list)=0D
- block drivers (vvfat and parcial null-co in deny-list)=0D
- qdev (hobbyist devices regularly hit by fuzzer)=0D
=0D
I don't want the security researchers to not fuzz QEMU unsafe=0D
areas, but I'd like to make it clearer what the community=0D
priority is (currently 47 opened issues on [3]).=0D
=0D
Regards,=0D
=0D
Phil.=0D
=0D
[1] https://lore.kernel.org/qemu-devel/20200714083631.888605-2-ppandit@redh=
at.com/=0D
[2] https://www.qemu.org/contribute/security-process/=0D
[3] https://gitlab.com/qemu-project/qemu/-/issues?label_name[]=3DFuzzer=0D
=0D
Philippe Mathieu-Daud=C3=A9 (10):=0D
  sysemu: Introduce qemu_security_policy_taint() API=0D
  accel: Use qemu_security_policy_taint(), mark KVM and Xen as safe=0D
  block: Use qemu_security_policy_taint() API=0D
  block/vvfat: Mark the driver as unsafe=0D
  block/null: Mark 'read-zeroes=3Doff' option as unsafe=0D
  qdev: Use qemu_security_policy_taint() API=0D
  hw/display: Mark ATI and Artist devices as unsafe=0D
  hw/misc: Mark testdev devices as unsafe=0D
  hw/net: Mark Tulip device as unsafe=0D
  hw/sd: Mark sdhci-pci device as unsafe=0D
=0D
 qapi/run-state.json        | 16 +++++++++=0D
 include/block/block_int.h  |  6 +++-=0D
 include/hw/qdev-core.h     |  6 ++++=0D
 include/qemu-common.h      | 19 +++++++++++=0D
 include/qemu/accel.h       |  5 +++=0D
 accel/kvm/kvm-all.c        |  1 +=0D
 accel/xen/xen-all.c        |  1 +=0D
 block.c                    |  6 ++++=0D
 block/null.c               |  8 +++++=0D
 block/vvfat.c              |  6 ++++=0D
 hw/core/qdev.c             | 11 ++++++=0D
 hw/display/artist.c        |  1 +=0D
 hw/display/ati.c           |  1 +=0D
 hw/hyperv/hyperv_testdev.c |  1 +=0D
 hw/misc/pc-testdev.c       |  1 +=0D
 hw/misc/pci-testdev.c      |  1 +=0D
 hw/net/tulip.c             |  1 +=0D
 hw/sd/sdhci-pci.c          |  1 +=0D
 softmmu/vl.c               | 70 ++++++++++++++++++++++++++++++++++++++=0D
 qemu-options.hx            | 17 +++++++++=0D
 20 files changed, 178 insertions(+), 1 deletion(-)=0D
=0D
--=20=0D
2.31.1=0D
=0D



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182405.329943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6ru-0005Hm-Ex; Wed, 08 Sep 2021 23:20:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182405.329943; Wed, 08 Sep 2021 23:20:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6ru-0005Hd-BZ; Wed, 08 Sep 2021 23:20:38 +0000
Received: by outflank-mailman (input) for mailman id 182405;
 Wed, 08 Sep 2021 23:20:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6rt-0005H5-57
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:37 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 29ae4109-0146-4d0d-9aab-02903dff8a94;
 Wed, 08 Sep 2021 23:20:36 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-188-xr-uShr_OCCOqUsQg9aBEA-1; Wed, 08 Sep 2021 19:20:32 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 h1-20020a05600c350100b002e751bf6733so27556wmq.8
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:32 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id d24sm351621wmb.35.2021.09.08.16.20.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29ae4109-0146-4d0d-9aab-02903dff8a94
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143235;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=24r6QApw3vBXmuZyf+4hmvAM+x9sjvcUeNKNebXV07M=;
	b=cHMSEsH74t2LvVloVlfq4EvD+YK4jqLIsSY4YFJusw8TgkB2ciE6ChM/g1pmYKdE0f9ZOJ
	XPBjIoKkp3S6WWLeZg8vBtc3kD6MGqMCDuJziJOhsqagrbrF1YyCwNW8JW8rlivQ5In0B6
	LSsJq/esbeEV0randd9PzKnI4N0eeac=
X-MC-Unique: xr-uShr_OCCOqUsQg9aBEA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=24r6QApw3vBXmuZyf+4hmvAM+x9sjvcUeNKNebXV07M=;
        b=OSEQ+1btCzwIP0Fjw06aUhpWT7CZ/JirzP7Qh7uRHA81iDx+/TKmGKriDqYzsIU3Qt
         0QpPU6aL5Ceb0AFD2H7U3GG315dR29AZjzFvZlDb4G4wweNG95prTSd5nqwMFHCvpGdX
         OEceVtNGpKlh8VUdH8y0Ul1xHwmkQi1TxaMhGf03/a9BMS2WvpjTeHwjjUsS9Ao3AnTO
         DPCyQSdViIRS78NO+mbdtYjKdfsrGr0jplBiABg3nBLWH//w5p1kt4G6bZJkatONQYD4
         dzFNmseoLFC4tclXtDcDK8CQHWcoTcNve/6hEIoGziRoazq+S5imVpqZpJpz3lHJExxm
         USXw==
X-Gm-Message-State: AOAM533OX6TrTHtxLJU8dl6515fkYRDl0SSQ795hNvUAN8nD0pdiiKMJ
	RMOcjOb3p5nCgmS4XFuVYV46vOwhvCRkXsuaszn9eruDL9V8+hUObr4uuQ6kZvA74jskDRQqDTI
	G+aAlbPxWLMtK66MxyDzQmfsWQtM=
X-Received: by 2002:a1c:a505:: with SMTP id o5mr581205wme.32.1631143231080;
        Wed, 08 Sep 2021 16:20:31 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxASJCusoNVcbOBdnL042tIJLhajKX+g6Wmf6M6mn99T6nfkEBgAxNrSyEjSzJ0g2TtZ/mq2Q==
X-Received: by 2002:a1c:a505:: with SMTP id o5mr581183wme.32.1631143230848;
        Wed, 08 Sep 2021 16:20:30 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 01/10] sysemu: Introduce qemu_security_policy_taint() API
Date: Thu,  9 Sep 2021 01:20:15 +0200
Message-Id: <20210908232024.2399215-2-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce qemu_security_policy_taint() which allows unsafe (read
"not very maintained") code to 'taint' QEMU security policy.

The "security policy" is the @SecurityPolicy QAPI enum, composed of:
- "none"   (no policy, current behavior)
- "warn"   (display a warning when the policy is tainted, keep going)
- "strict" (once tainted, exit QEMU before starting the VM)

The qemu_security_policy_is_strict() helper is also provided, which
will be proved useful once a VM is started (example we do not want
to kill a running VM if an unsafe device is hot-added).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 qapi/run-state.json   | 16 +++++++++++
 include/qemu-common.h | 19 ++++++++++++
 softmmu/vl.c          | 67 +++++++++++++++++++++++++++++++++++++++++++
 qemu-options.hx       | 17 +++++++++++
 4 files changed, 119 insertions(+)

diff --git a/qapi/run-state.json b/qapi/run-state.json
index 43d66d700fc..b15a107fa01 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -638,3 +638,19 @@
 { 'struct': 'MemoryFailureFlags',
   'data': { 'action-required': 'bool',
             'recursive': 'bool'} }
+
+##
+# @SecurityPolicy:
+#
+# An enumeration of the actions taken when the security policy is tainted.
+#
+# @none: do nothing.
+#
+# @warn: display a warning.
+#
+# @strict: prohibit QEMU to start a VM.
+#
+# Since: 6.2
+##
+{ 'enum': 'SecurityPolicy',
+  'data': [ 'none', 'warn', 'strict' ] }
diff --git a/include/qemu-common.h b/include/qemu-common.h
index 73bcf763ed8..bf0b054bb66 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -139,4 +139,23 @@ void page_size_init(void);
  * returned. */
 bool dump_in_progress(void);
 
+/**
+ * qemu_security_policy_taint:
+ * @tainting whether any security policy is tainted (compromised).
+ * @fmt: taint reason format string
+ * ...: list of arguments to interpolate into @fmt, like printf().
+ *
+ * Allow unsafe code path to taint the global security policy.
+ * See #SecurityPolicy.
+ */
+void qemu_security_policy_taint(bool tainting, const char *fmt, ...)
+        GCC_FMT_ATTR(2, 3);
+
+/**
+ * qemu_security_policy_is_strict:
+ *
+ * Return %true if the global security policy is 'strict', %false otherwise.
+ */
+bool qemu_security_policy_is_strict(void);
+
 #endif
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 55ab70eb97f..92c05ac97ee 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -489,6 +489,20 @@ static QemuOptsList qemu_action_opts = {
     },
 };
 
+static QemuOptsList qemu_security_policy_opts = {
+    .name = "security-policy",
+    .implied_opt_name = "policy",
+    .merge_lists = true,
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_security_policy_opts.head),
+    .desc = {
+        {
+            .name = "policy",
+            .type = QEMU_OPT_STRING,
+        },
+        { /* end of list */ }
+    },
+};
+
 const char *qemu_get_vm_name(void)
 {
     return qemu_name;
@@ -600,6 +614,52 @@ static int cleanup_add_fd(void *opaque, QemuOpts *opts, Error **errp)
 }
 #endif
 
+static SecurityPolicy security_policy = SECURITY_POLICY_NONE;
+
+bool qemu_security_policy_is_strict(void)
+{
+    return security_policy == SECURITY_POLICY_STRICT;
+}
+
+static int select_security_policy(const char *p)
+{
+    int policy;
+    char *qapi_value;
+
+    qapi_value = g_ascii_strdown(p, -1);
+    policy = qapi_enum_parse(&SecurityPolicy_lookup, qapi_value, -1, NULL);
+    g_free(qapi_value);
+    if (policy < 0) {
+        return -1;
+    }
+    security_policy = policy;
+
+    return 0;
+}
+
+void qemu_security_policy_taint(bool tainting, const char *fmt, ...)
+{
+    va_list ap;
+    g_autofree char *efmt = NULL;
+
+    if (security_policy == SECURITY_POLICY_NONE || !tainting) {
+        return;
+    }
+
+    va_start(ap, fmt);
+    if (security_policy == SECURITY_POLICY_STRICT) {
+        efmt = g_strdup_printf("%s taints QEMU security policy, exiting.", fmt);
+        error_vreport(efmt, ap);
+        exit(EXIT_FAILURE);
+    } else if (security_policy == SECURITY_POLICY_WARN) {
+        efmt = g_strdup_printf("%s taints QEMU security policy.", fmt);
+        warn_vreport(efmt, ap);
+    } else {
+        g_assert_not_reached();
+    }
+    va_end(ap);
+}
+
 /***********************************************************/
 /* QEMU Block devices */
 
@@ -2764,6 +2824,7 @@ void qemu_init(int argc, char **argv, char **envp)
     qemu_add_opts(&qemu_semihosting_config_opts);
     qemu_add_opts(&qemu_fw_cfg_opts);
     qemu_add_opts(&qemu_action_opts);
+    qemu_add_opts(&qemu_security_policy_opts);
     module_call_init(MODULE_INIT_OPTS);
 
     error_init(argv[0]);
@@ -3230,6 +3291,12 @@ void qemu_init(int argc, char **argv, char **envp)
                     exit(1);
                 }
                 break;
+            case QEMU_OPTION_security_policy:
+                if (select_security_policy(optarg) == -1) {
+                    error_report("unknown -security-policy parameter");
+                    exit(1);
+                }
+                break;
             case QEMU_OPTION_parallel:
                 add_device_config(DEV_PARALLEL, optarg);
                 default_parallel = 0;
diff --git a/qemu-options.hx b/qemu-options.hx
index 8f603cc7e65..d9939f7ae1d 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4298,6 +4298,23 @@ SRST
 
 ERST
 
+DEF("security-policy", HAS_ARG, QEMU_OPTION_security_policy, \
+    "-security-policy none|warn|strict\n" \
+    "                 action when security policy is tainted [default=none]\n",
+    QEMU_ARCH_ALL)
+SRST
+``-security-policy policy``
+    The policy controls what QEMU will do when an unsecure feature is
+    used, tainting the process security. The default is ``none`` (do
+    nothing). Other possible actions are: ``warn`` (display a warning
+    and keep going) or ``strict`` (exits QEMU before launching a VM).
+
+    Examples:
+
+    ``-security-policy warn``; \ ``-security-policy strict``
+
+ERST
+
 DEF("echr", HAS_ARG, QEMU_OPTION_echr, \
     "-echr chr       set terminal escape character instead of ctrl-a\n",
     QEMU_ARCH_ALL)
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182406.329955 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6rw-0005Zq-PB; Wed, 08 Sep 2021 23:20:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182406.329955; Wed, 08 Sep 2021 23:20:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6rw-0005Zf-Ld; Wed, 08 Sep 2021 23:20:40 +0000
Received: by outflank-mailman (input) for mailman id 182406;
 Wed, 08 Sep 2021 23:20:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6rv-0005Yv-Pb
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:39 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 60e9c8ca-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:20:39 +0000 (UTC)
Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com
 [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-352-HPvg9sFIMMGFOrpSD9GX4g-1; Wed, 08 Sep 2021 19:20:37 -0400
Received: by mail-wm1-f71.google.com with SMTP id
 v2-20020a7bcb420000b02902e6b108fcf1so32012wmj.8
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:37 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id j14sm30789wrp.21.2021.09.08.16.20.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60e9c8ca-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143238;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VWnIa1NnZmGclGfmWQAc+byFrHYlEtTXg6KMBnsWc1s=;
	b=CNr7uEMA9kce1mHm8iJRelPNGfg/Oj7Dn6LovQY5TTLblc8kbJOcFtS22RRLTtSaNaEl4E
	VLYd9KZkuGraLYmHaL3r2Jw+BHe28d/3lc1D6u88BLC7NtYOvuGSOhyDdvS8coF4RZ+KJH
	4/0/Kg2IXtekjoQnjG/sd5TxZZtASz4=
X-MC-Unique: HPvg9sFIMMGFOrpSD9GX4g-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=VWnIa1NnZmGclGfmWQAc+byFrHYlEtTXg6KMBnsWc1s=;
        b=RH7+oWKce2XP0YN2dFIErhB/jA8c+W2j/w03VzJZmANWkMrz2FWInpTq0m0Z0Aq73x
         oXb7vaIHFMzmGgbD/kL+t7yTaipEiQWYtcNDnuRvY95QtciR2wZWHpomshwxsSGTW24O
         EVEn6q5Bi0bxk1mBXfjwN4mRHEaoNSqRd97Kii1F9RlYFIilNXYLRUQ4NqGv1Fx485NF
         YYRQCdJ1p8atR+cX9p3Hj42lTEl5dNOeHOLJGQjYupxAtPxhbL1Et1reuj9h9uS7YeTr
         GcElue3bYxB6D8PFFAXeJeYvw5n53tC1dtZ9uuJVVeI0dB4uVHdIztZX9PF35UvWXaAG
         2ACg==
X-Gm-Message-State: AOAM532BP1yKPsT69m06v3Ty57MrFE5memNP3fIQWyUTWtu3n8rLWYW2
	5TV1ldXz+xk1onfmn+H7yRuheQh5PspEIk+wXw8+3JTBIhXnhqqU8jhiDVRGfSQZmZy/G0Ul5ce
	UlRpD6DVt3RODQy5mjMIOpiVsl8M=
X-Received: by 2002:a1c:448a:: with SMTP id r132mr557411wma.105.1631143236078;
        Wed, 08 Sep 2021 16:20:36 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJw6WkilBgiH6FuwGNniEzqjgdLM6GzbGQfLFFCepoIH7M5nhemfkyQL//fRVRjKLyNs63E1PQ==
X-Received: by 2002:a1c:448a:: with SMTP id r132mr557389wma.105.1631143235880;
        Wed, 08 Sep 2021 16:20:35 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 02/10] accel: Use qemu_security_policy_taint(), mark KVM and Xen as safe
Date: Thu,  9 Sep 2021 01:20:16 +0200
Message-Id: <20210908232024.2399215-3-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add the AccelClass::secure_policy_supported field to classify
safe (within security boundary) vs unsafe accelerators.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/qemu/accel.h | 5 +++++
 accel/kvm/kvm-all.c  | 1 +
 accel/xen/xen-all.c  | 1 +
 softmmu/vl.c         | 3 +++
 4 files changed, 10 insertions(+)

diff --git a/include/qemu/accel.h b/include/qemu/accel.h
index 4f4c283f6fc..895e30be0de 100644
--- a/include/qemu/accel.h
+++ b/include/qemu/accel.h
@@ -44,6 +44,11 @@ typedef struct AccelClass {
                        hwaddr start_addr, hwaddr size);
 #endif
     bool *allowed;
+    /*
+     * Whether the accelerator is withing QEMU security policy boundary.
+     * See: https://www.qemu.org/contribute/security-process/
+     */
+    bool secure_policy_supported;
     /*
      * Array of global properties that would be applied when specific
      * accelerator is chosen. It works like MachineClass.compat_props
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 0125c17edb8..eb6b9e44df2 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -3623,6 +3623,7 @@ static void kvm_accel_class_init(ObjectClass *oc, void *data)
     ac->init_machine = kvm_init;
     ac->has_memory = kvm_accel_has_memory;
     ac->allowed = &kvm_allowed;
+    ac->secure_policy_supported = true;
 
     object_class_property_add(oc, "kernel-irqchip", "on|off|split",
         NULL, kvm_set_kernel_irqchip,
diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index 69aa7d018b2..57867af5faf 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -198,6 +198,7 @@ static void xen_accel_class_init(ObjectClass *oc, void *data)
     ac->setup_post = xen_setup_post;
     ac->allowed = &xen_allowed;
     ac->compat_props = g_ptr_array_new();
+    ac->secure_policy_supported = true;
 
     compat_props_add(ac->compat_props, compat, G_N_ELEMENTS(compat));
 
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 92c05ac97ee..e4f94e159c3 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2388,6 +2388,9 @@ static int do_configure_accelerator(void *opaque, QemuOpts *opts, Error **errp)
         return 0;
     }
 
+    qemu_security_policy_taint(!ac->secure_policy_supported,
+                               "%s accelerator", acc);
+
     return 1;
 }
 
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182408.329966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6s5-00061U-3h; Wed, 08 Sep 2021 23:20:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182408.329966; Wed, 08 Sep 2021 23:20:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6s4-00061L-WF; Wed, 08 Sep 2021 23:20:49 +0000
Received: by outflank-mailman (input) for mailman id 182408;
 Wed, 08 Sep 2021 23:20:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6s3-0005H5-3u
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:47 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 8209b610-f9ec-45d7-bcba-940fde5c21dd;
 Wed, 08 Sep 2021 23:20:43 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-542-sjj3yeEOOUaRxEH5KLTLSw-1; Wed, 08 Sep 2021 19:20:42 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 f17-20020a05600c155100b002f05f30ff03so43144wmg.3
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:42 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id u8sm3407763wmq.45.2021.09.08.16.20.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8209b610-f9ec-45d7-bcba-940fde5c21dd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143243;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AztXDO9JQH0V78yza1WhT6PPCcwDZClDQJU1AbRxT1E=;
	b=DzNscKHD3B+rraynLTiG6CcFw3GBOY0qxjYVo/6Orqx2lRKrTmHUdjS8qLxwmpQcj7StRN
	Mykf0lwDYaebs39zHs5SnQpcb9VgCmEOSsSQ8VtDZoP1M16PR3lxxWqBDQNoTCfYQwlhAW
	wdLaQ24zI0jb7CYEshHXMM+oVlnWEZo=
X-MC-Unique: sjj3yeEOOUaRxEH5KLTLSw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=AztXDO9JQH0V78yza1WhT6PPCcwDZClDQJU1AbRxT1E=;
        b=MOOYlBQB7VUncjPvHrY9A58y11LIRsd18zsn5d/ixPQSNRlXNlXq+5+a6hox/755LX
         inmYWNWYojUpOLganr1IVzHI62giFbvp2OTJuPZCViPNIsR2wMxIOBtu1GYL/lW4AqN7
         QyolSkjUQMHUYy1VVeJexdx9O3eg0+meXSz6UjIBEMyrzLpDpn8Wn6QGJmnsEUYwmErZ
         9zMwJcAbXcp6ahn1lJXu1CrcNydWCj+ySTf0KVzJGVDyOOBhajrIkVi6iX5sUzXtq+28
         HpMIrvFxooe5Z5GNgZ/dFKYfOyZHWo/gpI1WrYyEm8hFe4wmptkMyFjyHnsycjXVJl0I
         iksw==
X-Gm-Message-State: AOAM532nFPnv2t/J71ziI8G0D0p0guTYtbAOpUdizLYOjfjU+KaHfjPM
	1VqhK+PXyPsI2/aIYfpQkdi7EKXZWDiLNNcJxdhujiMPwZJmDHlzy0aSxBRWuSa2uuvc2FDeoGF
	XtMI34D59poA8m/mREwgJf7ppTJ0=
X-Received: by 2002:adf:cd92:: with SMTP id q18mr84501wrj.211.1631143241058;
        Wed, 08 Sep 2021 16:20:41 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyXGBcGkm4TBFmSADncXUcjFYrWCY3XIJ3gTrnp0jPkA2D2pCoYXECv5ulk3hjXuxgjKt0f9A==
X-Received: by 2002:adf:cd92:: with SMTP id q18mr84482wrj.211.1631143240917;
        Wed, 08 Sep 2021 16:20:40 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 03/10] block: Use qemu_security_policy_taint() API
Date: Thu,  9 Sep 2021 01:20:17 +0200
Message-Id: <20210908232024.2399215-4-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add the BlockDriver::bdrv_taints_security_policy() handler.
Drivers implementing it might taint the global QEMU security
policy.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/block/block_int.h | 6 +++++-
 block.c                   | 6 ++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index f1a54db0f8c..0ec0a5c06e9 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -169,7 +169,11 @@ struct BlockDriver {
     int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
                           Error **errp);
     void (*bdrv_close)(BlockDriverState *bs);
-
+    /*
+     * Return %true if the driver is withing QEMU security policy boundary,
+     * %false otherwise. See: https://www.qemu.org/contribute/security-process/
+     */
+    bool (*bdrv_taints_security_policy)(BlockDriverState *bs);
 
     int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
                                        Error **errp);
diff --git a/block.c b/block.c
index b2b66263f9a..696ba486001 100644
--- a/block.c
+++ b/block.c
@@ -49,6 +49,7 @@
 #include "qemu/timer.h"
 #include "qemu/cutils.h"
 #include "qemu/id.h"
+#include "qemu-common.h"
 #include "block/coroutines.h"
 
 #ifdef CONFIG_BSD
@@ -1587,6 +1588,11 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv,
         }
     }
 
+    if (drv->bdrv_taints_security_policy) {
+        qemu_security_policy_taint(drv->bdrv_taints_security_policy(bs),
+                                   "Block protocol '%s'", drv->format_name);
+    }
+
     return 0;
 open_failed:
     bs->drv = NULL;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182409.329977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6s6-0006Km-Kw; Wed, 08 Sep 2021 23:20:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182409.329977; Wed, 08 Sep 2021 23:20:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6s6-0006Ka-Fn; Wed, 08 Sep 2021 23:20:50 +0000
Received: by outflank-mailman (input) for mailman id 182409;
 Wed, 08 Sep 2021 23:20:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6s5-00061E-13
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:49 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 667b4552-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:20:48 +0000 (UTC)
Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com
 [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-501-bkO3WsiWMvqQSzdibUOoCQ-1; Wed, 08 Sep 2021 19:20:47 -0400
Received: by mail-wm1-f71.google.com with SMTP id
 r126-20020a1c4484000000b002e8858850abso23995wma.0
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:47 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id p13sm36976wro.8.2021.09.08.16.20.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 667b4552-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143248;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AWN2pZpGs6sHyN/PrFeo6PGJIp1YJY+EJRzHbkMHcxk=;
	b=eFitySEzpqjh8S+blF/SNzqVLEpUBdOl9Tl+W/tlo6SNOaMsUbPVLtr9QFRC+SKfXNh1ce
	+4MjvYeIqHgWHSLNxR21Zvm1DaZoMu/qYaFBul8PzCVbFxxXaN3/yTADnb7KInf2kDt3Mk
	StBfjU7YSg7SugwQnNyiteCvgiUVdHI=
X-MC-Unique: bkO3WsiWMvqQSzdibUOoCQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=AWN2pZpGs6sHyN/PrFeo6PGJIp1YJY+EJRzHbkMHcxk=;
        b=xcGqj8lDBFIGP7CUP6lMfOGltp1vO2rKCH09ZyGWJ+B9tebSoXeLcQS1AMTLw1UEJ9
         qvp79D88CfGChEF6o/67kvIQ1EQlFkjae8v8Nqln717wm0blO8wmVFzUUrPPrAKPQZ4f
         yNUeMCgAa4Zv5TY55Pj+s9AHrsAJEa8KQlrOnMLJ/9Ib8TCcS3e3zHys6hHbBzUZhoOe
         yrP1csqmSd6i5gAWgbU+HkU7ZmfAZqB96w3lyBYKz9+gyU0x++BQXJa1Gz+ebBm1PSY2
         5HQb8HkFr9pHK4D8yutCkTYbNzXJwwX9TJ0D7ZzjuPzKpHZdzDySOie51vQ0ULfh4rSl
         twnw==
X-Gm-Message-State: AOAM532LAW26Qy2VTTn1KFe59mltfZQEqRmTkSJ6fesBngeBtVHa5Jhj
	Gn7oXTIahvWQUiJtFxt/KC1mXxH4WkHPOfhJFI6fE0MMZZd7xAt8E0y2imCSOrW7/bwVoR2SH9r
	4/Kone+40YC3hPDQe/mBWw4Wleac=
X-Received: by 2002:a05:600c:3641:: with SMTP id y1mr5746027wmq.181.1631143245828;
        Wed, 08 Sep 2021 16:20:45 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyRx5vddYT6ifvfXuEUZzDhSVwlkW5T8HDOd0oz2QEHRH1Qx3AckLgtuI1lDwOwP+2e9s6Xjw==
X-Received: by 2002:a05:600c:3641:: with SMTP id y1mr5746017wmq.181.1631143245705;
        Wed, 08 Sep 2021 16:20:45 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 04/10] block/vvfat: Mark the driver as unsafe
Date: Thu,  9 Sep 2021 01:20:18 +0200
Message-Id: <20210908232024.2399215-5-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While being listed as 'supported' in MAINTAINERS, this driver
does not have many reviewers and contains various /* TODO */
unattended since various years. Not safe enough for production
environment, so have it taint the global security policy.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 block/vvfat.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/block/vvfat.c b/block/vvfat.c
index 34bf1e3a86e..993e40727d6 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -3199,6 +3199,11 @@ static void vvfat_close(BlockDriverState *bs)
     }
 }
 
+static bool vvfat_taints_security_policy(BlockDriverState *bs)
+{
+    return true;
+}
+
 static const char *const vvfat_strong_runtime_opts[] = {
     "dir",
     "fat-type",
@@ -3219,6 +3224,7 @@ static BlockDriver bdrv_vvfat = {
     .bdrv_refresh_limits    = vvfat_refresh_limits,
     .bdrv_close             = vvfat_close,
     .bdrv_child_perm        = vvfat_child_perm,
+    .bdrv_taints_security_policy = vvfat_taints_security_policy,
 
     .bdrv_co_preadv         = vvfat_co_preadv,
     .bdrv_co_pwritev        = vvfat_co_pwritev,
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:20:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:20:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182411.329988 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sE-0006yb-1O; Wed, 08 Sep 2021 23:20:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182411.329988; Wed, 08 Sep 2021 23:20:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sD-0006yQ-Su; Wed, 08 Sep 2021 23:20:57 +0000
Received: by outflank-mailman (input) for mailman id 182411;
 Wed, 08 Sep 2021 23:20:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sC-0006qz-06
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:20:56 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 6aa80e62-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:20:55 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-120-R6zlE6tdNA-tMDGhi21A2g-1; Wed, 08 Sep 2021 19:20:51 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 5-20020a1c00050000b02902e67111d9f0so4241wma.4
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:51 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id c3sm30713wrd.34.2021.09.08.16.20.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6aa80e62-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143255;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=p6u3sclI6JBi/0aQ8m/8uYmNWK6u/Efc+CTX33mwcn4=;
	b=UAbntxhNMDLfhVBJUanQ/nbNFX8SsAgORd8PabEo0ooHKj4cGHZLtZJV1Ap4C+4U+v4W1m
	9JXqOZLnbflNIhvAsLodB2QDw6saRyC5xk4oXokL4grrM0ug0kgsIlLCH43ermB775L6GW
	OxaKR8ypL7bHx7uyTyNf2AXNhuX1iS8=
X-MC-Unique: R6zlE6tdNA-tMDGhi21A2g-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=p6u3sclI6JBi/0aQ8m/8uYmNWK6u/Efc+CTX33mwcn4=;
        b=sT9ulG+ZLywii1VxtIiObwE6Cfz6u6/yBn70EPuwbVCw6/ZfKA7qU6gdGHNv+JLu4O
         xlNFhV2PQQmkq3Ag8ta/I/6EmwPQXwbfwYBp4TwLJzPK4Ya5gMyGCHwxZcdADwRphnPo
         ZMz0xm+EBplM2vappXNzHx52zdCcBrGNbpKrcbsOLUPBmfbPnmudlQ8a/FudIlvcfhoE
         9uAoRPeUjJTuaNaFnRG/XrJBd/pjvFcFQwa8bdVkI8v9Vbmqhrkf7dIADSo/nSKlawVI
         wsNSYxohSKxP/6aKtAx2qd6p37SER7F1rBI67QD/mQrHWPtaD/5iLGcAjXST6f5Bs6KP
         YBlA==
X-Gm-Message-State: AOAM533FLPUp4ly/iDks+MJ/+x1m+bKmxFUHVLy/dT6Ux8hPMnJStnIi
	MakxHz1GfiQpUmSh/Tj+/OZTLTNAXNjqcEYY6kZeLBdqbnkenuQsMrM8bKjPAoWJPpC+zj1xuL8
	mwPT2FIrR3Klpf1KaWlgBJty5t08=
X-Received: by 2002:adf:fb8d:: with SMTP id a13mr108073wrr.164.1631143250631;
        Wed, 08 Sep 2021 16:20:50 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzrh/n5wSnZqfaTQvrzl6IJ2DKh1pSXkkHEuOdKtVrOvCH+TWDpEiwg1GoyEgBtyhNoco5QeA==
X-Received: by 2002:adf:fb8d:: with SMTP id a13mr108055wrr.164.1631143250497;
        Wed, 08 Sep 2021 16:20:50 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 05/10] block/null: Mark 'read-zeroes=off' option as unsafe
Date: Thu,  9 Sep 2021 01:20:19 +0200
Message-Id: <20210908232024.2399215-6-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

See commit b317006a3f1 ("docs/secure-coding-practices: Describe how
to use 'null-co' block driver") for rationale.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 block/null.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/block/null.c b/block/null.c
index cc9b1d4ea72..11e428f3cc2 100644
--- a/block/null.c
+++ b/block/null.c
@@ -99,6 +99,13 @@ static int null_file_open(BlockDriverState *bs, QDict *options, int flags,
     return ret;
 }
 
+static bool null_taints_security_policy(BlockDriverState *bs)
+{
+    BDRVNullState *s = bs->opaque;
+
+    return !s->read_zeroes;
+}
+
 static int64_t null_getlength(BlockDriverState *bs)
 {
     BDRVNullState *s = bs->opaque;
@@ -283,6 +290,7 @@ static BlockDriver bdrv_null_co = {
     .bdrv_parse_filename    = null_co_parse_filename,
     .bdrv_getlength         = null_getlength,
     .bdrv_get_allocated_file_size = null_allocated_file_size,
+    .bdrv_taints_security_policy = null_taints_security_policy,
 
     .bdrv_co_preadv         = null_co_preadv,
     .bdrv_co_pwritev        = null_co_pwritev,
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:21:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:21:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182420.329998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sK-0007am-Au; Wed, 08 Sep 2021 23:21:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182420.329998; Wed, 08 Sep 2021 23:21:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sK-0007ab-6v; Wed, 08 Sep 2021 23:21:04 +0000
Received: by outflank-mailman (input) for mailman id 182420;
 Wed, 08 Sep 2021 23:21:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sI-0006qz-W6
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:21:03 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 6ed5206a-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:21:02 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-34-BNSDJFlcOr6a8RcCnT7r_Q-1; Wed, 08 Sep 2021 19:21:01 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 y188-20020a1c7dc5000000b002e80e0b2f87so46767wmc.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:21:01 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id e8sm14000wrc.96.2021.09.08.16.20.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ed5206a-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143262;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Qceza/WB3CtMew9GJDkTkDK3S8IfTB0xLv9ykQRmWEY=;
	b=BZf5flKneby/HH/TUn+2En4hcuRn0nmqUQlhObaQRtHRTS85+MH3RYybtdnDIP/nlNwvYl
	OqT0y4+C9/70yYBTF9ExiumHax9KbbMvWoqxYzsKno14Lq6ti5oo0k+SYsOzxIrHtDInIZ
	Ay6kf2TslFss/rF3NzONLuSpIgUsdmU=
X-MC-Unique: BNSDJFlcOr6a8RcCnT7r_Q-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=Qceza/WB3CtMew9GJDkTkDK3S8IfTB0xLv9ykQRmWEY=;
        b=wOCRkgEQuiqwpczRegApAqmrcK2g7v20+z3jfm/JbUOQq41fbLpyBc4SHbYcfil34H
         3arakN1/uPO+uBzBLSbmVNjBnDthriJf5w5n0ZPHM3eEq5j3dJpeDZlCUE2YOOvhnchp
         x+5GIXvDXQqTLsikSUG11TMSvdPicBMZKQHx/uGqY7TbQVsNI58sJvoLhIK1rjrMy2gB
         jLQh1pFYhQtEmM+zfJ3UVOkGQjSGo0uZveJ2aDmcTmd4YNXQ5KInPjOpMoMRayG0hES3
         RMyJNWRZS5GnPZzr21xI1BUIMbOxNZfYQhUNFXDJaNOt2zYCUc6R0XoCTHIUEmV3nG1H
         9awA==
X-Gm-Message-State: AOAM5302j9ezc/2MHgqp6LObCdCJTe7aRS1bSPUe1Sdag9UGtIfQGzUU
	FU3TQmPpe2NPBA2nP0bCI5N60ydooLTKVrzAlIFyPZP1IukCO/ifAiZnmAjKXvt6Rf3PaXHp+SF
	s9CblitN6aNxGu+QettFmU7bBfKo=
X-Received: by 2002:adf:fd51:: with SMTP id h17mr109105wrs.178.1631143260116;
        Wed, 08 Sep 2021 16:21:00 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzGCdGOUWAMH7ZAThbx+xTDgHh6zdyfVzIsc9P7NbgQbnC+meNZTqgQ/wi8QkKk51AhWel3JA==
X-Received: by 2002:adf:fd51:: with SMTP id h17mr109093wrs.178.1631143259991;
        Wed, 08 Sep 2021 16:20:59 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 07/10] hw/display: Mark ATI and Artist devices as unsafe
Date: Thu,  9 Sep 2021 01:20:21 +0200
Message-Id: <20210908232024.2399215-8-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/display/artist.c | 1 +
 hw/display/ati.c    | 1 +
 2 files changed, 2 insertions(+)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 21b7fd1b440..067a4b2cb59 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -1482,6 +1482,7 @@ static void artist_class_init(ObjectClass *klass, void *data)
     dc->vmsd = &vmstate_artist;
     dc->reset = artist_reset;
     device_class_set_props(dc, artist_properties);
+    dc->taints_security_policy = true;
 }
 
 static const TypeInfo artist_info = {
diff --git a/hw/display/ati.c b/hw/display/ati.c
index 31f22754dce..2f27ab69a87 100644
--- a/hw/display/ati.c
+++ b/hw/display/ati.c
@@ -1024,6 +1024,7 @@ static void ati_vga_class_init(ObjectClass *klass, void *data)
     device_class_set_props(dc, ati_vga_properties);
     dc->hotpluggable = false;
     set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories);
+    dc->taints_security_policy = true;
 
     k->class_id = PCI_CLASS_DISPLAY_VGA;
     k->vendor_id = PCI_VENDOR_ID_ATI;
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:21:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:21:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182429.330010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sT-0000HH-Jn; Wed, 08 Sep 2021 23:21:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182429.330010; Wed, 08 Sep 2021 23:21:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sT-0000H3-Fh; Wed, 08 Sep 2021 23:21:13 +0000
Received: by outflank-mailman (input) for mailman id 182429;
 Wed, 08 Sep 2021 23:21:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sS-0005H5-4e
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:21:12 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id a74451f7-6faf-4990-9bed-d74110fca1f2;
 Wed, 08 Sep 2021 23:20:58 +0000 (UTC)
Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com
 [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-347-PIP6fDH6OSiC69KndTpGiA-1; Wed, 08 Sep 2021 19:20:56 -0400
Received: by mail-wm1-f71.google.com with SMTP id
 v21-20020a05600c215500b002fa7eb53754so32601wml.4
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:20:56 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id b22sm390180wmn.18.2021.09.08.16.20.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:20:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a74451f7-6faf-4990-9bed-d74110fca1f2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143257;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=l2rov07jCDHA26r/9GcgxSj1NkWhTs7Eb1vPCWPgxnA=;
	b=RozFAsnnwYFkaB0pfKzK9DQMb1u2uHJ5zI8M5bmUzi/8hI+7Pz8yqiGnYTuGIZcu00cbVd
	JmeiTD2j6hg3mLMbDNIkcel7sy77u4AYx80QLjruhpnw+Lg4gp+vWK9gnX7w3kR66fr74F
	62kzflqAg8uWRDuspmlsA44ZS6bwqdo=
X-MC-Unique: PIP6fDH6OSiC69KndTpGiA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=l2rov07jCDHA26r/9GcgxSj1NkWhTs7Eb1vPCWPgxnA=;
        b=kR8hMkI2S4iwZtUou3lLCHFrDw+8RpsgTXEtGs/RPHjiBFbCcyGmyNQUbTA3N/txfS
         BdNSN51WTj49n+IqZ1uL/u2/W7oi+L+dza9e0mWZeHsWVyQMYEQr/gOUV+qkUv6dj7qS
         BDyMPADUNpZgEfbSO2uCZekSN3XHW4R+p7dEZs+8Uwhw3e2gdkKELDEZ7YRJ6sx+CQ35
         T7h0toaQYBmIXxJpLwrXVJwPT5+TEt6fpqkqkC3jrFrsCP+6NK4uRkcj03V+v8pZ2uzP
         fLdnvrdHBxrfxgkmTACjpu7Y/CxDwLu15eCTzDPeIb4uqbc/6i89YNbx8JtpAz4VGJiu
         MkJA==
X-Gm-Message-State: AOAM531v68+ynw+Inc1TgGyog7K9WC8J3U9nXR0E69/CvdZSr6XlVIoB
	It9pXAaNinQBpkLWkBJ5qr301TFjRsceulaLWMJcQr8XuS5bjrzl2VL5xm/18N/kHFOW9BxuXmA
	WAJk2GUcTgd5A9hI479KtGBpSKTo=
X-Received: by 2002:a5d:6781:: with SMTP id v1mr82997wru.249.1631143255221;
        Wed, 08 Sep 2021 16:20:55 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJx0YS7l2ozSVHS0MrlXrrjEkKOBRZTLJQiA3uLjalXryjpOEItjin5eop39X0TrxKIC57agAQ==
X-Received: by 2002:a5d:6781:: with SMTP id v1mr82983wru.249.1631143255072;
        Wed, 08 Sep 2021 16:20:55 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 06/10] qdev: Use qemu_security_policy_taint() API
Date: Thu,  9 Sep 2021 01:20:20 +0200
Message-Id: <20210908232024.2399215-7-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add DeviceClass::taints_security_policy field to allow an
unsafe device to eventually taint the global security policy
in DeviceRealize().

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/qdev-core.h |  6 ++++++
 hw/core/qdev.c         | 11 +++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index bafc311bfa1..ff9ce6671be 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -122,6 +122,12 @@ struct DeviceClass {
      */
     bool user_creatable;
     bool hotpluggable;
+    /*
+     * %false if the device is within the QEMU security policy boundary,
+     * %true if there is no guarantee this device can be used safely.
+     * See: https://www.qemu.org/contribute/security-process/
+     */
+    bool taints_security_policy;
 
     /* callbacks */
     /*
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index cefc5eaa0a9..a5a00f3564c 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -31,6 +31,7 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/visitor.h"
 #include "qemu/error-report.h"
+#include "qemu-common.h"
 #include "qemu/option.h"
 #include "hw/hotplug.h"
 #include "hw/irq.h"
@@ -257,6 +258,13 @@ bool qdev_hotplug_allowed(DeviceState *dev, Error **errp)
     MachineClass *mc;
     Object *m_obj = qdev_get_machine();
 
+    if (qemu_security_policy_is_strict()
+            && DEVICE_GET_CLASS(dev)->taints_security_policy) {
+        error_setg(errp, "Device '%s' can not be hotplugged when"
+                         " 'strict' security policy is in place",
+                   object_get_typename(OBJECT(dev)));
+    }
+
     if (object_dynamic_cast(m_obj, TYPE_MACHINE)) {
         machine = MACHINE(m_obj);
         mc = MACHINE_GET_CLASS(machine);
@@ -385,6 +393,9 @@ bool qdev_realize(DeviceState *dev, BusState *bus, Error **errp)
     } else {
         assert(!DEVICE_GET_CLASS(dev)->bus_type);
     }
+    qemu_security_policy_taint(DEVICE_GET_CLASS(dev)->taints_security_policy,
+                               "device type %s",
+                               object_get_typename(OBJECT(dev)));
 
     return object_property_set_bool(OBJECT(dev), "realized", true, errp);
 }
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:21:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:21:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182436.330021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sb-0000xG-TT; Wed, 08 Sep 2021 23:21:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182436.330021; Wed, 08 Sep 2021 23:21:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sb-0000x7-QC; Wed, 08 Sep 2021 23:21:21 +0000
Received: by outflank-mailman (input) for mailman id 182436;
 Wed, 08 Sep 2021 23:21:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sa-0006qz-Vv
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:21:21 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 728db82a-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:21:08 +0000 (UTC)
Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com
 [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-217-gyJtz0OhPtunCPLAYwdHHw-1; Wed, 08 Sep 2021 19:21:07 -0400
Received: by mail-wm1-f69.google.com with SMTP id
 y188-20020a1c7dc5000000b002e80e0b2f87so46882wmc.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:21:07 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id t18sm13191wrp.97.2021.09.08.16.21.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:21:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 728db82a-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143268;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=geCiN4saZ2frlXmhHZT44DYe1b4J1aCN7Xj/FRob9pQ=;
	b=CaYMj3N+mpDrdOYIZRw3OMS8+U1HPeApFqVvZrJ0UT+LcSu3TEg3HvtNM9os+t679YTvpe
	genSjISt4UB146LgJlsDk0k2YHFL0f2Z8fLX1oikJ83kr9o53TPnbkwsDKwwymEYFBSKJQ
	Q8XQTJmqzMkpuKzx1Lw+oAvxx0bhd+Y=
X-MC-Unique: gyJtz0OhPtunCPLAYwdHHw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=geCiN4saZ2frlXmhHZT44DYe1b4J1aCN7Xj/FRob9pQ=;
        b=l0oHGb9+yotHZR3/8j2KfSySbXpxrTJiLylNY39n3gmROqKuvnW2mZkuXwZy3WNLH+
         ap8YoiZU/Mf2g1altnKyZ55NTeEcu5vd48lFV3xilxot6Nh1JoiJqylvXIF8FQdO+TGg
         6sPTLq2uyzZJSzBwIrPUhvx9D46HzwO7pYF8HyXLFFXgW16zljBtztzSLfOLOUeCtAaz
         P3YLVU4gWCOoogcUstsI6wJ20OAYs5/SmZs6YYPZiixgWoWfw6XIUJrcrwIKaX/eScO6
         7wj9w+LLNxUbj46gigygl4CBTDOG7VI7EoWqJCIdM5O5R1GhU7ycQUDCZHFDzlSB8DNM
         OeYg==
X-Gm-Message-State: AOAM530xxyJ/5fZ/k9FmIb925hpg3nJG9GZ0O1/7rGqZn0hK5dL9MDlN
	r5wsCxq3kv+mCRQb3UIrRc/0CU+lr9Tsy6gTH/y+Wb65aRABGkrAGVUvZkru7NPpZ4zoHwHvFbJ
	CNuDFYJuSDqlPnlGrwPI1FJjI9mE=
X-Received: by 2002:a05:600c:3b84:: with SMTP id n4mr5902095wms.50.1631143265001;
        Wed, 08 Sep 2021 16:21:05 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxCzGiWOShaEshb6oqBDiWr1/WLhbErNDpTZCYYbU0bD2w0oJKgL2kMezD9hKtqCektWiLpSA==
X-Received: by 2002:a05:600c:3b84:: with SMTP id n4mr5902087wms.50.1631143264855;
        Wed, 08 Sep 2021 16:21:04 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 08/10] hw/misc: Mark testdev devices as unsafe
Date: Thu,  9 Sep 2021 01:20:22 +0200
Message-Id: <20210908232024.2399215-9-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/hyperv/hyperv_testdev.c | 1 +
 hw/misc/pc-testdev.c       | 1 +
 hw/misc/pci-testdev.c      | 1 +
 3 files changed, 3 insertions(+)

diff --git a/hw/hyperv/hyperv_testdev.c b/hw/hyperv/hyperv_testdev.c
index 9a56ddf83fe..6a75c350389 100644
--- a/hw/hyperv/hyperv_testdev.c
+++ b/hw/hyperv/hyperv_testdev.c
@@ -310,6 +310,7 @@ static void hv_test_dev_class_init(ObjectClass *klass, void *data)
 
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
     dc->realize = hv_test_dev_realizefn;
+    dc->taints_security_policy = true;
 }
 
 static const TypeInfo hv_test_dev_info = {
diff --git a/hw/misc/pc-testdev.c b/hw/misc/pc-testdev.c
index e3896518694..6294b80ec1b 100644
--- a/hw/misc/pc-testdev.c
+++ b/hw/misc/pc-testdev.c
@@ -199,6 +199,7 @@ static void testdev_class_init(ObjectClass *klass, void *data)
 
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
     dc->realize = testdev_realizefn;
+    dc->taints_security_policy = true;
 }
 
 static const TypeInfo testdev_info = {
diff --git a/hw/misc/pci-testdev.c b/hw/misc/pci-testdev.c
index 03845c8de34..189eb9bf1bb 100644
--- a/hw/misc/pci-testdev.c
+++ b/hw/misc/pci-testdev.c
@@ -340,6 +340,7 @@ static void pci_testdev_class_init(ObjectClass *klass, void *data)
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
     dc->reset = qdev_pci_testdev_reset;
     device_class_set_props(dc, pci_testdev_properties);
+    dc->taints_security_policy = true;
 }
 
 static const TypeInfo pci_testdev_info = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:21:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:21:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182439.330032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sh-0001Yn-Bb; Wed, 08 Sep 2021 23:21:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182439.330032; Wed, 08 Sep 2021 23:21:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sh-0001YZ-6O; Wed, 08 Sep 2021 23:21:27 +0000
Received: by outflank-mailman (input) for mailman id 182439;
 Wed, 08 Sep 2021 23:21:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sf-0006qz-W6
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:21:26 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 75dabf6e-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:21:14 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-436-rMRz529aM0mahUiIOwrOvA-1; Wed, 08 Sep 2021 19:21:11 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 m22-20020a7bcb96000000b002f7b840d9dcso39609wmi.1
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:21:10 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id e2sm29478wra.40.2021.09.08.16.21.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:21:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75dabf6e-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143274;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/j1jkBDfrokHKQKoDNuT1x0UKxyqYOGBFYU13ViaHTo=;
	b=T1Y/2Ag6QNLFp+5TDQD6naC0EfJRjM1DIO0yKMS+yi1kJLp8RQZ8Q8ySHDA0LubfeN5wqt
	Ct6Xp0aJPCRCg6EVMi8vptt0ZaeeGkf3llMwD7D9/4m7aQIwYrUcvKe8IcaF+OodnT/q+9
	Ui2KNrt8EYk0Gn/ZJ4vGhACGMtwi8bQ=
X-MC-Unique: rMRz529aM0mahUiIOwrOvA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=/j1jkBDfrokHKQKoDNuT1x0UKxyqYOGBFYU13ViaHTo=;
        b=TglR5A+LlYq/bZe6lkoCh2XLmH8W5q2887ulhXHC20+i3wcWnx2q5E8eT9w8uWX4+x
         sie6sX27guZiUnc0RUBNbia4olFhCxBw4xceAW3ddW2ZYzVyiMUmw7Vu1oz9518skLAo
         ROFIg691e3kllE/TTTxrb4sPLxs4zXJTUh3Yol4XQpAEdpchHnAVJi7+UNaoti5wlIrB
         1b/N6PdvgWOYTMB9zFQju8vaB7oDLF3ytPpgHhdHOEa5WYpnCVLXqd53vPNPotGYninI
         dr+g4kgmigO1V79bvIJ0YZDdjhS5zW8HYBnDz5s61etlB/V6IscGP45NelQ1CSD5QNCr
         AG7w==
X-Gm-Message-State: AOAM5326pTzA0gKZXJGzRLVBYp0QGWK8re0DeBErzQPBuLe69HelN2Ja
	qUOAJCtht0IFuNo3PQwMAauAFXiu7Hrh9LZlziDg5D/NZ44hAquhdQZHYL2KwPzpzu/vi6cX4hA
	Z7E4TVzgelkHoVxCgQPXkDDT7/Yk=
X-Received: by 2002:a1c:f709:: with SMTP id v9mr5800239wmh.124.1631143269932;
        Wed, 08 Sep 2021 16:21:09 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzsU5M2MRY44dKoBP/lcBIPQuM1MTJ8pmbiG93tEMNWtoFxyRNm7Nzd/6gGAEAcV0FSbIvT1A==
X-Received: by 2002:a1c:f709:: with SMTP id v9mr5800221wmh.124.1631143269808;
        Wed, 08 Sep 2021 16:21:09 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 09/10] hw/net: Mark Tulip device as unsafe
Date: Thu,  9 Sep 2021 01:20:23 +0200
Message-Id: <20210908232024.2399215-10-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/net/tulip.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/net/tulip.c b/hw/net/tulip.c
index ca69f7ea5e1..eaad3266212 100644
--- a/hw/net/tulip.c
+++ b/hw/net/tulip.c
@@ -1025,6 +1025,7 @@ static void tulip_class_init(ObjectClass *klass, void *data)
     device_class_set_props(dc, tulip_properties);
     dc->reset = tulip_qdev_reset;
     set_bit(DEVICE_CATEGORY_NETWORK, dc->categories);
+    dc->taints_security_policy = true;
 }
 
 static const TypeInfo tulip_info = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:21:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:21:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182443.330043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sl-0001zv-Ls; Wed, 08 Sep 2021 23:21:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182443.330043; Wed, 08 Sep 2021 23:21:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO6sl-0001yo-GJ; Wed, 08 Sep 2021 23:21:31 +0000
Received: by outflank-mailman (input) for mailman id 182443;
 Wed, 08 Sep 2021 23:21:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=39dd=N6=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mO6sl-0006qz-04
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:21:31 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 77d9edc6-10fb-11ec-b18c-12813bfff9fa;
 Wed, 08 Sep 2021 23:21:17 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-317-NgJ5RylbONODu41M2ITodQ-1; Wed, 08 Sep 2021 19:21:16 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 u14-20020a7bcb0e0000b0290248831d46e4so30001wmj.6
 for <xen-devel@lists.xenproject.org>; Wed, 08 Sep 2021 16:21:16 -0700 (PDT)
Received: from x1w.. (21.red-83-52-55.dynamicip.rima-tde.net. [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id t64sm344074wma.48.2021.09.08.16.21.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 08 Sep 2021 16:21:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77d9edc6-10fb-11ec-b18c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631143277;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZkBFAMCpq6n9y+VH8JjTuNHo5ej2dLMGMd9K3LlaeP8=;
	b=MbajT/kNKsg4SXBKvHTR6tDLMIIK9gQUeckCC931eQZSpF/3DAT0PItZYHI/N0H3/BXSkH
	mgVNMFrCiSG+J/8pX7Eo5vNPU5izZBGZXmdUjY39Rzc5qYqdr1iwlNITyuZWSRp48lgDbc
	BYG9oZIHlsPUzJa+vTOzD1n6xajIZpI=
X-MC-Unique: NgJ5RylbONODu41M2ITodQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ZkBFAMCpq6n9y+VH8JjTuNHo5ej2dLMGMd9K3LlaeP8=;
        b=kIG9GwTh0qeSOI/bbSX68+/c00cIiyJgEW4DPhZtRPs+XYLojOQVOLx+XsPw3sm+5p
         6g9+fcSsCqDtSLnu5BLfagZ91hMd8wkC6HuVRD6waavbK0tBxtxXGgHLvrUWY78x7hH9
         ZjJLosWWWiHB/bFK5WodzZ1El/2FOMGg7s6QvgPAUvitBEXLON5SEE99Jkdzlf1rzJLw
         NzXxKqOE9j5Cp0iQvkSxNHZQJhkRyNlGLK69ORDkp/CWSfhWw+te6Bq/goAaHLYn2FQ3
         Y9p5xeMKK+etVYZLjOKNWVKs/JmbThi6quAHDXvNpafrVVoBxMEICd1zr5RISKI/kY00
         +iwA==
X-Gm-Message-State: AOAM531hMa7IJdvVUFWDH9mCuPpCaf+JdghZlNgVZ/4QnykVPkZhAhyD
	vgkbKYeOPawhafgboEAZpQ/v4ws2vVyGJCu/hd8qYBqtbyVr4PeYzGaWzFwjH2waVCs15ELLEDU
	hm24l+E7JyfzqMJjcmWM/aS/tSUI=
X-Received: by 2002:a7b:c4cd:: with SMTP id g13mr5796244wmk.91.1631143275278;
        Wed, 08 Sep 2021 16:21:15 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJybmoGBTuN4rPAmVNoHLJ613PnCRxyUI7oFmyfrpx3hP5AYKM4updonQnfJItVUJwu02OE/7Q==
X-Received: by 2002:a7b:c4cd:: with SMTP id g13mr5796223wmk.91.1631143275096;
        Wed, 08 Sep 2021 16:21:15 -0700 (PDT)
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
To: qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org,
	Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Subject: [RFC PATCH 10/10] hw/sd: Mark sdhci-pci device as unsafe
Date: Thu,  9 Sep 2021 01:20:24 +0200
Message-Id: <20210908232024.2399215-11-philmd@redhat.com>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/sd/sdhci-pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/sd/sdhci-pci.c b/hw/sd/sdhci-pci.c
index c737c8b930e..7a36f88fd87 100644
--- a/hw/sd/sdhci-pci.c
+++ b/hw/sd/sdhci-pci.c
@@ -64,6 +64,7 @@ static void sdhci_pci_class_init(ObjectClass *klass, void *data)
     k->device_id = PCI_DEVICE_ID_REDHAT_SDHCI;
     k->class_id = PCI_CLASS_SYSTEM_SDHCI;
     device_class_set_props(dc, sdhci_pci_properties);
+    dc->taints_security_policy = true;
 
     sdhci_common_class_init(klass, data);
 }
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:29:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:29:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182455.330054 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO70M-0003nf-Ft; Wed, 08 Sep 2021 23:29:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182455.330054; Wed, 08 Sep 2021 23:29:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO70M-0003nY-Ca; Wed, 08 Sep 2021 23:29:22 +0000
Received: by outflank-mailman (input) for mailman id 182455;
 Wed, 08 Sep 2021 23:29:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO70L-0003nQ-HO
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:29:21 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9777562c-10fc-11ec-b18d-12813bfff9fa;
 Wed, 08 Sep 2021 23:29:20 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E96260FDA;
 Wed,  8 Sep 2021 23:29:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9777562c-10fc-11ec-b18d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631143759;
	bh=mGTIHxtsH3AVhxFFacA7WTV5cD6baIb/RYElMqmSSqw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KH1aFE/VHS6C4uJUtoa7uQapTlPEBmzNHCpuPKtry0l7gpIsDADDMBOQOKH/dyWOr
	 MBFiGZQ+bAT2mAGTq7GMPf57h/QuseXR0LA5SpJBKUYi0n0VaXHBjKUkEF4fldnIQb
	 kQQ0MHsKMF9ylJXzIUDPMEF4qm0mARTz+Xn4Qzvz2ysjqHvEOSdbGJteIGaa4fz92k
	 cEsyb4KIXx0+MaZp6esBlCDAD6ebtAvVTTBDT/uixwKqk4S5PinRTS9EkmiJog6Ted
	 hmooHvYyuE8GeFrF95Sb0JoYSfWjwoX55S763bydNc4QtjLURD7CX1dBZxMGZGflPP
	 pvcAjBMUZAwJw==
Date: Wed, 8 Sep 2021 16:29:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 1/7] xen/arm: introduce new helper
 device_tree_get_meminfo
In-Reply-To: <20210908095248.545981-2-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081628160.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-2-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> This commit creates a new helper device_tree_get_meminfo to iterate over a
> device tree property to get memory info, like "reg".
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/bootfdt.c | 83 ++++++++++++++++++++++++------------------
>  1 file changed, 47 insertions(+), 36 deletions(-)
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 476e32e0f5..b01badda3e 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -63,6 +63,52 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
>      *size = dt_next_cell(size_cells, cell);
>  }
>  
> +static int __init device_tree_get_meminfo(const void *fdt, int node,
> +                                          const char *prop_name,
> +                                          u32 address_cells, u32 size_cells,
> +                                          void *data)
> +{
> +    const struct fdt_property *prop;
> +    unsigned int i, banks;
> +    const __be32 *cell;
> +    u32 reg_cells = address_cells + size_cells;
> +    paddr_t start, size;
> +    struct meminfo *mem = data;
> +
> +    if ( address_cells < 1 || size_cells < 1 )
> +    {
> +        printk("fdt: property `%s': invalid #address-cells or #size-cells",
> +               prop_name);
> +        return -EINVAL;
> +    }
> +
> +    prop = fdt_get_property(fdt, node, prop_name, NULL);
> +    if ( !prop )
> +        return -ENOENT;
> +
> +    cell = (const __be32 *)prop->data;
> +    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
> +
> +    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
> +    {
> +        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
> +        /* Some DT may describe empty bank, ignore them */
> +        if ( !size )
> +            continue;
> +        mem->bank[mem->nr_banks].start = start;
> +        mem->bank[mem->nr_banks].size = size;
> +        mem->nr_banks++;
> +    }
> +
> +    if ( i < banks )
> +    {
> +        printk("Warning: Max number of supported memory regions reached.\n");
> +        return -ENOSPC;
> +    }
> +
> +    return 0;
> +}
> +
>  u32 __init device_tree_get_u32(const void *fdt, int node,
>                                 const char *prop_name, u32 dflt)
>  {
> @@ -139,42 +185,7 @@ static int __init process_memory_node(const void *fdt, int node,
>                                        u32 address_cells, u32 size_cells,
>                                        void *data)
>  {
> -    const struct fdt_property *prop;
> -    int i;
> -    int banks;
> -    const __be32 *cell;
> -    paddr_t start, size;
> -    u32 reg_cells = address_cells + size_cells;
> -    struct meminfo *mem = data;
> -
> -    if ( address_cells < 1 || size_cells < 1 )
> -    {
> -        printk("fdt: node `%s': invalid #address-cells or #size-cells",
> -               name);
> -        return -EINVAL;
> -    }
> -
> -    prop = fdt_get_property(fdt, node, "reg", NULL);
> -    if ( !prop )
> -        return -ENOENT;
> -
> -    cell = (const __be32 *)prop->data;
> -    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
> -
> -    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
> -    {
> -        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
> -        /* Some DT may describe empty bank, ignore them */
> -        if ( !size )
> -            continue;
> -        mem->bank[mem->nr_banks].start = start;
> -        mem->bank[mem->nr_banks].size = size;
> -        mem->nr_banks++;
> -    }
> -
> -    if ( i < banks )
> -        return -ENOSPC;
> -    return 0;
> +    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
>  }
>  
>  static int __init process_reserved_memory_node(const void *fdt, int node,
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:54:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:54:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182486.330076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7Oa-0008DJ-OD; Wed, 08 Sep 2021 23:54:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182486.330076; Wed, 08 Sep 2021 23:54:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7Oa-0008DC-Kv; Wed, 08 Sep 2021 23:54:24 +0000
Received: by outflank-mailman (input) for mailman id 182486;
 Wed, 08 Sep 2021 23:54:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO7OY-0008Cm-Pc
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:54:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8adf7f8b-5cbd-4a9d-bb93-1b65e90c00c3;
 Wed, 08 Sep 2021 23:54:21 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E10A661059;
 Wed,  8 Sep 2021 23:54:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8adf7f8b-5cbd-4a9d-bb93-1b65e90c00c3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631145261;
	bh=vGPUkqiSUhixdY4PTm7cRj0Zcx/J+QLHfFfbGVsUer0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bneSrwGacrTy8qW9CARX3W4mF/2XOZKHJhiGfc3nQNXSBH+Iwc9R+FDwx6OGb95RV
	 I0TnpMwA0SmOr73D+juvhMHfFoHSKiPCE+KesvPrJd/pQrrCafJvgB5qwbSRIhIcRf
	 8wuiNWl28J49EuhtWA1zs5wW079h0rJEmykrWWUVk7f/7+3Pfhb9UP5/SfeHNZCfdZ
	 Kfce7cAIsJWMGNnYnrWgBuPismc1ipzJBWvXDUOvOG1moRmv2C35of+JAuNn+81Z0l
	 1JJhtYZo7LvTI5sBQP3JpO2LGw07wkbde55L3BkqRtkvV7gNohsCQA6A6FMYO1yDtf
	 nv9Xvg6oHHjTw==
Date: Wed, 8 Sep 2021 16:54:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 4/7] xen/arm: static memory initialization
In-Reply-To: <20210908095248.545981-5-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081643340.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-5-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> This patch introduces static memory initialization, during system boot-up.
> 
> The new function init_staticmem_pages is responsible for static memory
> initialization.
> 
> Helper free_staticmem_pages is the equivalent of free_heap_pages, to free
> nr_mfns pages of static memory.
> 
> This commit also introduces a new CONFIG_STATIC_MEMORY option to wrap all
> static-allocation-related code.
> 
> Put asynchronously scrubbing pages of static memory in TODO list.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/setup.c    | 27 +++++++++++++++++++++++++++
>  xen/common/Kconfig      | 13 +++++++++++++
>  xen/common/page_alloc.c | 21 +++++++++++++++++++++
>  xen/include/xen/mm.h    |  6 ++++++
>  4 files changed, 67 insertions(+)
> 
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 63a908e325..5be7f2b0c2 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -609,6 +609,29 @@ static void __init init_pdx(void)
>      }
>  }
>  
> +/* Static memory initialization */
> +static void __init init_staticmem_pages(void)
> +{
> +#ifdef CONFIG_STATIC_MEMORY
> +    unsigned int bank;
> +
> +    for ( bank = 0 ; bank < bootinfo.reserved_mem.nr_banks; bank++ )
> +    {
> +        if ( bootinfo.reserved_mem.bank[bank].xen_domain )
> +        {
> +            mfn_t bank_start = _mfn(PFN_UP(bootinfo.reserved_mem.bank[bank].start));
> +            unsigned long bank_pages = PFN_DOWN(bootinfo.reserved_mem.bank[bank].size);
> +            mfn_t bank_end = mfn_add(bank_start, bank_pages);
> +
> +            if ( mfn_x(bank_end) <= mfn_x(bank_start) )
> +                return;
> +
> +            free_staticmem_pages(mfn_to_page(bank_start), bank_pages, false);
> +        }
> +    }
> +#endif
> +}
> +
>  #ifdef CONFIG_ARM_32
>  static void __init setup_mm(void)
>  {
> @@ -736,6 +759,8 @@ static void __init setup_mm(void)
>      /* Add xenheap memory that was not already added to the boot allocator. */
>      init_xenheap_pages(mfn_to_maddr(xenheap_mfn_start),
>                         mfn_to_maddr(xenheap_mfn_end));
> +
> +    init_staticmem_pages();
>  }
>  #else /* CONFIG_ARM_64 */
>  static void __init setup_mm(void)
> @@ -789,6 +814,8 @@ static void __init setup_mm(void)
>  
>      setup_frametable_mappings(ram_start, ram_end);
>      max_page = PFN_DOWN(ram_end);
> +
> +    init_staticmem_pages();
>  }
>  #endif
>  
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 0ddd18e11a..3558be0dbc 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -67,6 +67,19 @@ config MEM_ACCESS
>  config NEEDS_LIBELF
>  	bool
>  
> +config STATIC_MEMORY
> +	bool "Static Allocation Support (UNSUPPORTED)" if UNSUPPORTED
> +	depends on ARM
> +	help
> +	  Static Allocation refers to system or sub-system(domains) for
> +	  which memory areas are pre-defined by configuration using physical
> +	  address ranges.
> +
> +	  When enabled, memory can be statically allocated to a domain using
> +	  the property "xen,static-mem" defined in the domain configuration.
> +
> +	  If unsure, say N.
> +
>  menu "Speculative hardening"
>  
>  config SPECULATIVE_HARDEN_ARRAY
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index a3ee5eca9e..ba7adc80db 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2604,6 +2604,27 @@ struct domain *get_pg_owner(domid_t domid)
>      return pg_owner;
>  }
>  
> +#ifdef CONFIG_STATIC_MEMORY
> +/* Equivalent of free_heap_pages to free nr_mfns pages of static memory. */
> +void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
> +                                 bool need_scrub)
> +{
> +    mfn_t mfn = page_to_mfn(pg);
> +    unsigned long i;
> +
> +    for ( i = 0; i < nr_mfns; i++ )
> +    {
> +        mark_page_free(&pg[i], mfn_add(mfn, i));
> +
> +        if ( need_scrub )
> +        {
> +            /* TODO: asynchronous scrubbing for pages of static memory. */
> +            scrub_one_page(pg);
> +        }
> +    }
> +}
> +#endif
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 667f9dac83..8e8fb5a615 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -85,6 +85,12 @@ bool scrub_free_pages(void);
>  } while ( false )
>  #define FREE_XENHEAP_PAGE(p) FREE_XENHEAP_PAGES(p, 0)
>  
> +#ifdef CONFIG_STATIC_MEMORY
> +/* These functions are for static memory */
> +void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
> +                          bool need_scrub);
> +#endif
> +
>  /* Map machine page range in Xen virtual address space. */
>  int map_pages_to_xen(
>      unsigned long virt,
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:54:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:54:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182485.330065 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7OW-0007wv-Fr; Wed, 08 Sep 2021 23:54:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182485.330065; Wed, 08 Sep 2021 23:54:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7OW-0007wo-Ch; Wed, 08 Sep 2021 23:54:20 +0000
Received: by outflank-mailman (input) for mailman id 182485;
 Wed, 08 Sep 2021 23:54:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO7OV-0007wi-6k
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:54:19 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 13fbed2c-1100-11ec-b18f-12813bfff9fa;
 Wed, 08 Sep 2021 23:54:17 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D58FE60F93;
 Wed,  8 Sep 2021 23:54:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13fbed2c-1100-11ec-b18f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631145257;
	bh=9nUyLz8jYEkDv/4J+XH04a+GDFKO8RY50xUdH5tN5Pw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Nb1pZZyQrX1qFh3lL9zIXsuxNfNkWkvVFfXr2GYQFpHqI+7SBJH2kXqcoahJIqRm+
	 ku3caOLCMMcTjxEGugErVTVyl8L9Os1HEt2CJ21eshwFQeHq1Sm7MKt27/V/7d89cQ
	 050DtswCRK8hoVbj4t/6Ngb0a78xnuIXLhYDXHxDQLFwFDW2H6pS4pmxH/s10Z/X4q
	 wY8O+Ru5EKak54Kcdqp85gtEntqGseryMD0o+a7rGfyJyE3D1iESmpj/pMEfE1BTT9
	 w7LFRxcBzuBIrkkK+JlCPa9R4Ea530o8VK37te7V9l9JnH1YtwSpFDoSftCr+kfFC3
	 BxEJpZrihKXqQ==
Date: Wed, 8 Sep 2021 16:54:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 2/7] xen/arm: introduce domain on Static Allocation
In-Reply-To: <20210908095248.545981-3-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081631400.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-3-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> Static Allocation refers to system or sub-system(domains) for which memory
> areas are pre-defined by configuration using physical address ranges.
> 
> Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
> beginning, shall never go to heap allocator or boot allocator for any use.
> 
> Memory can be statically allocated to a domain using the property "xen,static-
> mem" defined in the domain configuration. The number of cells for the address
> and the size must be defined using respectively the properties
> "#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
> 
> The property 'memory' is still needed and should match the amount of memory
> given to the guest. Currently, it either comes from static memory or lets Xen
> allocate from heap. *Mixing* is not supported.
> 
> The static memory will be mapped in the guest at the usual guest memory
> addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> xen/include/public/arch-arm.h.
> 
> This patch introduces this new `xen,static-mem` feature, and also documents
> and parses this new attribute at boot time.
> 
> This patch also introduces a new field "bool xen_domain" in "struct membank"
> to tell whether the memory bank is reserved as the whole hardware resource,
> or bind to a xen domain node, through "xen,static-mem"
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  docs/misc/arm/device-tree/booting.txt | 42 +++++++++++++++++++++++++++
>  xen/arch/arm/bootfdt.c                | 30 +++++++++++++++++--
>  xen/include/asm-arm/setup.h           |  1 +
>  3 files changed, 71 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 5243bc7fd3..44cd9e1a9a 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -268,3 +268,45 @@ The DTB fragment is loaded at 0xc000000 in the example above. It should
>  follow the convention explained in docs/misc/arm/passthrough.txt. The
>  DTB fragment will be added to the guest device tree, so that the guest
>  kernel will be able to discover the device.
> +
> +
> +Static Allocation
> +=============
> +
> +Static Allocation refers to system or sub-system(domains) for which memory
> +areas are pre-defined by configuration using physical address ranges.
> +
> +Memory can be statically allocated to a domain using the property "xen,static-
> +mem" defined in the domain configuration. The number of cells for the address
> +and the size must be defined using respectively the properties
> +"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
> +
> +The property 'memory' is still needed and should match the amount of memory
> +given to the guest. Currently, it either comes from static memory or lets Xen
> +allocate from heap. *Mixing* is not supported.
> +
> +The static memory will be mapped in the guest at the usual guest memory
> +addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> +xen/include/public/arch-arm.h.
> +
> +Below is an example on how to specify the static memory region in the
> +device-tree:
> +
> +    / {
> +        chosen {
> +            domU1 {
> +                compatible = "xen,domain";
> +                #address-cells = <0x2>;
> +                #size-cells = <0x2>;
> +                cpus = <2>;
> +                memory = <0x0 0x80000>;
> +                #xen,static-mem-address-cells = <0x1>;
> +                #xen,static-mem-size-cells = <0x1>;
> +                xen,static-mem = <0x30000000 0x20000000>;
> +                ...
> +            };
> +        };
> +    };
> +
> +This will reserve a 512MB region starting at the host physical address
> +0x30000000 to be exclusively used by DomU1.
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index b01badda3e..afaa0e249b 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -66,7 +66,7 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
>  static int __init device_tree_get_meminfo(const void *fdt, int node,
>                                            const char *prop_name,
>                                            u32 address_cells, u32 size_cells,
> -                                          void *data)
> +                                          void *data, bool xen_domain)
>  {
>      const struct fdt_property *prop;
>      unsigned int i, banks;
> @@ -97,6 +97,7 @@ static int __init device_tree_get_meminfo(const void *fdt, int node,
>              continue;
>          mem->bank[mem->nr_banks].start = start;
>          mem->bank[mem->nr_banks].size = size;
> +        mem->bank[mem->nr_banks].xen_domain = xen_domain;
>          mem->nr_banks++;
>      }
>  
> @@ -185,7 +186,8 @@ static int __init process_memory_node(const void *fdt, int node,
>                                        u32 address_cells, u32 size_cells,
>                                        void *data)
>  {
> -    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
> +    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells,
> +                                   data, false);
>  }
>  
>  static int __init process_reserved_memory_node(const void *fdt, int node,
> @@ -339,6 +341,28 @@ static void __init process_chosen_node(const void *fdt, int node,
>      add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
>  }
>  
> +static int __init process_domain_node(const void *fdt, int node,
> +                                      const char *name,
> +                                      u32 address_cells, u32 size_cells)
> +{
> +    const struct fdt_property *prop;
> +
> +    printk("Checking for \"xen,static-mem\" in domain node\n");
> +
> +    prop = fdt_get_property(fdt, node, "xen,static-mem", NULL);
> +    if ( !prop )
> +        /* No "xen,static-mem" present. */
> +        return 0;
> +
> +    address_cells = device_tree_get_u32(fdt, node,
> +                                        "#xen,static-mem-address-cells", 0);
> +    size_cells = device_tree_get_u32(fdt, node,
> +                                     "#xen,static-mem-size-cells", 0);
> +
> +    return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells,
> +                                   size_cells, &bootinfo.reserved_mem, true);
> +}
> +
>  static int __init early_scan_node(const void *fdt,
>                                    int node, const char *name, int depth,
>                                    u32 address_cells, u32 size_cells,
> @@ -357,6 +381,8 @@ static int __init early_scan_node(const void *fdt,
>          process_multiboot_node(fdt, node, name, address_cells, size_cells);
>      else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") )
>          process_chosen_node(fdt, node, name, address_cells, size_cells);
> +    else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") )
> +        rc = process_domain_node(fdt, node, name, address_cells, size_cells);
>  
>      if ( rc < 0 )
>          printk("fdt: node `%s': parsing failed\n", name);
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index c4b6af6029..95da0b7ab9 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -24,6 +24,7 @@ typedef enum {
>  struct membank {
>      paddr_t start;
>      paddr_t size;
> +    bool xen_domain; /* whether the memory bank is bound to a Xen domain. */
>  };
>  
>  struct meminfo {
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 08 23:57:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 08 Sep 2021 23:57:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182499.330087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7Rm-0000q0-Av; Wed, 08 Sep 2021 23:57:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182499.330087; Wed, 08 Sep 2021 23:57:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7Rm-0000pt-7z; Wed, 08 Sep 2021 23:57:42 +0000
Received: by outflank-mailman (input) for mailman id 182499;
 Wed, 08 Sep 2021 23:57:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IK88=N6=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO7Rl-0000pn-0P
 for xen-devel@lists.xenproject.org; Wed, 08 Sep 2021 23:57:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e7eb6c99-57b2-41dc-8324-868a8e5f3e28;
 Wed, 08 Sep 2021 23:57:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E3CBE61078;
 Wed,  8 Sep 2021 23:57:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7eb6c99-57b2-41dc-8324-868a8e5f3e28
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631145459;
	bh=m4x1Nut6of6QS3jxMRWIT4+lBYGXEaekkeZtJr0tZ4I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KuVx1yRXtaDH1dsUKiXmAeQGDTfN4//4+iJOmvbxk36NG/jP1isDQeA7Wj7ffDy4t
	 gtjGV9XfpCK2GIoYkO5ZRZxGJQjLZPx5GLahtD1R1R39U/JK+PeovKfTMN+jNI6Hnh
	 lsFTW0bpB8VeQL1guEi65ocaggdNsPS0Qkw0VDKu4HMXxA4EhmRGY59gUS942yj+11
	 3K4M0ED1LHj399naElpq7m9cjzB6fC0sPS6F5+0WDVrR21KhA8pLPjeKsmaKvto8/m
	 /fxnEZ/xbkmX7B6wPFZAH9MSXzo27jH2HDWazcVVALfGe44r/9Z9u7m+ctoQto76f8
	 phc8Bvgpbb3tQ==
Date: Wed, 8 Sep 2021 16:57:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
In-Reply-To: <20210908095248.545981-6-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081654290.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-6-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> In order to deal with the trouble of count-to-order conversion when page number
> is not in a power-of-two, this commit re-define assign_pages for nr pages and
> introduces a new helper assign_page for original page with a single order.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

The patch looks correct to me, however I don't think I understood Jan's
request to the previous version of this patch, so I can't tell if you
addressed his concerns.


> ---
>  xen/arch/x86/pv/dom0_build.c |  2 +-
>  xen/common/grant_table.c     |  2 +-
>  xen/common/memory.c          |  6 +++---
>  xen/common/page_alloc.c      | 21 +++++++++++++--------
>  xen/include/xen/mm.h         |  6 ++++++
>  5 files changed, 24 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
> index d7f9e04b28..7787cc8fca 100644
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -568,7 +568,7 @@ int __init dom0_construct_pv(struct domain *d,
>          else
>          {
>              while ( count-- )
> -                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 0, 0) )
> +                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 1, 0) )
>                      BUG();
>          }
>          initrd->mod_end = 0;
> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> index ee61603a97..50f5f83023 100644
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -2358,7 +2358,7 @@ gnttab_transfer(
>           * is respected and speculative execution is blocked accordingly
>           */
>          if ( unlikely(!evaluate_nospec(okay)) ||
> -            unlikely(assign_pages(e, page, 0, MEMF_no_refcount)) )
> +            unlikely(assign_pages(e, page, 1, MEMF_no_refcount)) )
>          {
>              bool drop_dom_ref;
>  
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index 74babb0bd7..9cef8790ff 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -728,8 +728,8 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
>          /* Assign each output page to the domain. */
>          for ( j = 0; (page = page_list_remove_head(&out_chunk_list)); ++j )
>          {
> -            if ( assign_pages(d, page, exch.out.extent_order,
> -                              MEMF_no_refcount) )
> +            if ( assign_page(page, exch.out.extent_order, d,
> +                             MEMF_no_refcount) )
>              {
>                  unsigned long dec_count;
>                  bool_t drop_dom_ref;
> @@ -797,7 +797,7 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
>       * cleared PGC_allocated.
>       */
>      while ( (page = page_list_remove_head(&in_chunk_list)) )
> -        if ( assign_pages(d, page, 0, MEMF_no_refcount) )
> +        if ( assign_pages(d, page, 1, MEMF_no_refcount) )
>          {
>              BUG_ON(!d->is_dying);
>              free_domheap_page(page);
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index ba7adc80db..bb19bb10ff 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2261,7 +2261,7 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
>  int assign_pages(
>      struct domain *d,
>      struct page_info *pg,
> -    unsigned int order,
> +    unsigned long nr,
>      unsigned int memflags)
>  {
>      int rc = 0;
> @@ -2281,7 +2281,7 @@ int assign_pages(
>      {
>          unsigned int extra_pages = 0;
>  
> -        for ( i = 0; i < (1ul << order); i++ )
> +        for ( i = 0; i < nr; i++ )
>          {
>              ASSERT(!(pg[i].count_info & ~PGC_extra));
>              if ( pg[i].count_info & PGC_extra )
> @@ -2290,18 +2290,18 @@ int assign_pages(
>  
>          ASSERT(!extra_pages ||
>                 ((memflags & MEMF_no_refcount) &&
> -                extra_pages == 1u << order));
> +                extra_pages == nr));
>      }
>  #endif
>  
>      if ( pg[0].count_info & PGC_extra )
>      {
> -        d->extra_pages += 1u << order;
> +        d->extra_pages += nr;
>          memflags &= ~MEMF_no_refcount;
>      }
>      else if ( !(memflags & MEMF_no_refcount) )
>      {
> -        unsigned int tot_pages = domain_tot_pages(d) + (1 << order);
> +        unsigned int tot_pages = domain_tot_pages(d) + nr;
>  
>          if ( unlikely(tot_pages > d->max_pages) )
>          {
> @@ -2313,10 +2313,10 @@ int assign_pages(
>      }
>  
>      if ( !(memflags & MEMF_no_refcount) &&
> -         unlikely(domain_adjust_tot_pages(d, 1 << order) == (1 << order)) )
> +         unlikely(domain_adjust_tot_pages(d, nr) == nr) )
>          get_knownalive_domain(d);
>  
> -    for ( i = 0; i < (1 << order); i++ )
> +    for ( i = 0; i < nr; i++ )
>      {
>          ASSERT(page_get_owner(&pg[i]) == NULL);
>          page_set_owner(&pg[i], d);
> @@ -2331,6 +2331,11 @@ int assign_pages(
>      return rc;
>  }
>  
> +int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
> +                unsigned int memflags)
> +{
> +    return assign_pages(d, pg, 1UL << order, memflags);
> +}
>  
>  struct page_info *alloc_domheap_pages(
>      struct domain *d, unsigned int order, unsigned int memflags)
> @@ -2373,7 +2378,7 @@ struct page_info *alloc_domheap_pages(
>                  pg[i].count_info = PGC_extra;
>              }
>          }
> -        if ( assign_pages(d, pg, order, memflags) )
> +        if ( assign_page(pg, order, d, memflags) )
>          {
>              free_heap_pages(pg, order, memflags & MEMF_no_scrub);
>              return NULL;
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 8e8fb5a615..a74e93eba8 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -133,8 +133,14 @@ void heap_init_late(void);
>  
>  int assign_pages(
>      struct domain *d,
> +    struct page_info *pg,
> +    unsigned long nr,
> +    unsigned int memflags);
> +
> +int assign_page(
>      struct page_info *pg,
>      unsigned int order,
> +    struct domain *d,
>      unsigned int memflags);
>  
>  /* Dump info to serial console */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 00:03:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 00:03:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182505.330098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7XQ-0003AL-GX; Thu, 09 Sep 2021 00:03:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182505.330098; Thu, 09 Sep 2021 00:03:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7XQ-0003AE-DU; Thu, 09 Sep 2021 00:03:32 +0000
Received: by outflank-mailman (input) for mailman id 182505;
 Thu, 09 Sep 2021 00:03:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO7XO-0003A8-9X
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 00:03:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 68c6bc64-aebf-4700-b3d0-84ca59c6351b;
 Thu, 09 Sep 2021 00:03:28 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1633060F6C;
 Thu,  9 Sep 2021 00:03:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68c6bc64-aebf-4700-b3d0-84ca59c6351b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631145808;
	bh=O5g3IT1QAlttxYH6hZmifAew4o0sOSkg0hnR7Y7Iwa4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uAYBDyb4fhs4fS/FwHkp1mD/q33Do2tnSbZ3saXmwJNtCuPO+bwJalzqtL+dmLsIh
	 myzqHPVRZ2ICmS/cZjz3MPH5UetPbCgfNNDnJ7gQYvjvtSWkkwi2n3fwcUk45B6HtG
	 GViqaSkfG7NgzAXJUE4DPrHIqU1D5rWXraA7IUPSf6kDysVge343Aoi/ntF3bjGN46
	 1DALdapGlFWotUXYH/YQ/X2YEJIIC1x8vEJKYnfaD3UWo3/pzG13Qbhl97Fx2vu9rg
	 s3q39D542lWhrITGLQj1E0nChAnlFQA1nV5NUUI5BnexS5AHjKtL2nCdVfNbILJDVH
	 HiRZlWFPuFMsw==
Date: Wed, 8 Sep 2021 17:03:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 6/7] xen/arm: introduce acquire_staticmem_pages and
 acquire_domstatic_pages
In-Reply-To: <20210908095248.545981-7-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081703180.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-7-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> New function acquire_staticmem_pages aims to acquire nr_mfns contiguous pages
> of static memory, starting at #smfn. And it is the equivalent of
> alloc_heap_pages for static memory.
> 
> For each page, it shall check if the page is reserved(PGC_reserved)
> and free. It shall also do a set of necessary initialization, which are
> mostly the same ones in alloc_heap_pages, like, following the same
> cache-coherency policy and turning page status into PGC_state_inuse, etc.
> 
> New function acquire_domstatic_pages is the equivalent of alloc_domheap_pages
> for static memory, and it is to acquire nr_mfns contiguous pages of
> static memory and assign them to one specific domain.
> 
> It uses acquire_staticmem_pages to acquire nr_mfns pages of static memory.
> Then on success, it will use assign_pages to assign those pages to one
> specific domain.
> 
> In order to differentiate pages of static memory from those allocated from
> heap, this patch introduces a new page flag PGC_reserved, then mark pages of
> static memory PGC_reserved when initializing them.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/common/page_alloc.c  | 118 ++++++++++++++++++++++++++++++++++++++-
>  xen/include/asm-arm/mm.h |   3 +
>  xen/include/xen/mm.h     |   2 +
>  3 files changed, 121 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index bb19bb10ff..59dffcfa1d 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -151,6 +151,10 @@
>  #define p2m_pod_offline_or_broken_replace(pg) BUG_ON(pg != NULL)
>  #endif
>  
> +#ifndef PGC_reserved
> +#define PGC_reserved 0
> +#endif
> +
>  /*
>   * Comma-separated list of hexadecimal page numbers containing bad bytes.
>   * e.g. 'badpage=0x3f45,0x8a321'.
> @@ -2283,7 +2287,7 @@ int assign_pages(
>  
>          for ( i = 0; i < nr; i++ )
>          {
> -            ASSERT(!(pg[i].count_info & ~PGC_extra));
> +            ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
>              if ( pg[i].count_info & PGC_extra )
>                  extra_pages++;
>          }
> @@ -2322,7 +2326,8 @@ int assign_pages(
>          page_set_owner(&pg[i], d);
>          smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
>          pg[i].count_info =
> -            (pg[i].count_info & PGC_extra) | PGC_allocated | 1;
> +            (pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated | 1;
> +
>          page_list_add_tail(&pg[i], page_to_list(d, &pg[i]));
>      }
>  
> @@ -2626,8 +2631,117 @@ void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
>              /* TODO: asynchronous scrubbing for pages of static memory. */
>              scrub_one_page(pg);
>          }
> +
> +        /* In case initializing page of static memory, mark it PGC_reserved. */
> +        pg[i].count_info |= PGC_reserved;
>      }
>  }
> +
> +/*
> + * Acquire nr_mfns contiguous reserved pages, starting at #smfn, of
> + * static memory.
> + * This function needs to be reworked if used outside of boot.
> + */
> +static struct page_info * __init acquire_staticmem_pages(mfn_t smfn,
> +                                                         unsigned long nr_mfns,
> +                                                         unsigned int memflags)
> +{
> +    bool need_tlbflush = false;
> +    uint32_t tlbflush_timestamp = 0;
> +    unsigned long i;
> +    struct page_info *pg;
> +
> +    ASSERT(nr_mfns);
> +    for ( i = 0; i < nr_mfns; i++ )
> +        if ( !mfn_valid(mfn_add(smfn, i)) )
> +            return NULL;
> +
> +    pg = mfn_to_page(smfn);
> +
> +    spin_lock(&heap_lock);
> +
> +    for ( i = 0; i < nr_mfns; i++ )
> +    {
> +        /* The page should be reserved and not yet allocated. */
> +        if ( pg[i].count_info != (PGC_state_free | PGC_reserved) )
> +        {
> +            printk(XENLOG_ERR
> +                   "pg[%lu] Static MFN %"PRI_mfn" c=%#lx t=%#x\n",
> +                   i, mfn_x(smfn) + i,
> +                   pg[i].count_info, pg[i].tlbflush_timestamp);
> +            goto out_err;
> +        }
> +
> +        if ( !(memflags & MEMF_no_tlbflush) )
> +            accumulate_tlbflush(&need_tlbflush, &pg[i],
> +                                &tlbflush_timestamp);
> +
> +        /*
> +         * Preserve flag PGC_reserved and change page state
> +         * to PGC_state_inuse.
> +         */
> +        pg[i].count_info = PGC_reserved | PGC_state_inuse;
> +        /* Initialise fields which have other uses for free pages. */
> +        pg[i].u.inuse.type_info = 0;
> +        page_set_owner(&pg[i], NULL);
> +    }
> +
> +    spin_unlock(&heap_lock);
> +
> +    if ( need_tlbflush )
> +        filtered_flush_tlb_mask(tlbflush_timestamp);
> +
> +    /*
> +     * Ensure cache and RAM are consistent for platforms where the guest
> +     * can control its own visibility of/through the cache.
> +     */
> +    for ( i = 0; i < nr_mfns; i++ )
> +        flush_page_to_ram(mfn_x(smfn) + i, !(memflags & MEMF_no_icache_flush));
> +
> +    return pg;
> +
> + out_err:
> +    while ( i-- )
> +        pg[i].count_info = PGC_reserved | PGC_state_free;
> +
> +    spin_unlock(&heap_lock);
> +
> +    return NULL;
> +}
> +
> +/*
> + * Acquire nr_mfns contiguous pages, starting at #smfn, of static memory,
> + * then assign them to one specific domain #d.
> + */
> +int __init acquire_domstatic_pages(struct domain *d, mfn_t smfn,
> +                                   unsigned long nr_mfns, unsigned int memflags)
> +{
> +    struct page_info *pg;
> +
> +    ASSERT(!in_irq());
> +
> +    pg = acquire_staticmem_pages(smfn, nr_mfns, memflags);
> +    if ( !pg )
> +        return -ENOENT;
> +
> +    if ( !d || (memflags & (MEMF_no_owner | MEMF_no_refcount)) )
> +    {
> +        /*
> +         * Respective handling omitted here because right now
> +         * acquired static memory is only for guest RAM.
> +         */
> +        ASSERT_UNREACHABLE();
> +        return -EINVAL;
> +    }
> +
> +    if ( assign_pages(d, pg, nr_mfns, memflags) )
> +    {
> +        free_staticmem_pages(pg, nr_mfns, memflags & MEMF_no_scrub);
> +        return -EINVAL;
> +    }
> +
> +    return 0;
> +}
>  #endif
>  
>  /*
> diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
> index ded74d29da..7b5e7b7f69 100644
> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -108,6 +108,9 @@ struct page_info
>    /* Page is Xen heap? */
>  #define _PGC_xen_heap     PG_shift(2)
>  #define PGC_xen_heap      PG_mask(1, 2)
> +  /* Page is reserved */
> +#define _PGC_reserved     PG_shift(3)
> +#define PGC_reserved      PG_mask(1, 3)
>  /* ... */
>  /* Page is broken? */
>  #define _PGC_broken       PG_shift(7)
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index a74e93eba8..da1b158693 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -89,6 +89,8 @@ bool scrub_free_pages(void);
>  /* These functions are for static memory */
>  void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
>                            bool need_scrub);
> +int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned long nr_mfns,
> +                            unsigned int memflags);
>  #endif
>  
>  /* Map machine page range in Xen virtual address space. */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 00:15:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 00:15:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182530.330126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7ip-0005yr-UZ; Thu, 09 Sep 2021 00:15:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182530.330126; Thu, 09 Sep 2021 00:15:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO7ip-0005yk-RU; Thu, 09 Sep 2021 00:15:19 +0000
Received: by outflank-mailman (input) for mailman id 182530;
 Thu, 09 Sep 2021 00:15:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO7io-0005ye-QG
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 00:15:18 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 461a176c-f192-40a3-8404-a8636a37d0c2;
 Thu, 09 Sep 2021 00:15:17 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 928BC610C8;
 Thu,  9 Sep 2021 00:15:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 461a176c-f192-40a3-8404-a8636a37d0c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631146516;
	bh=pPBd4WRGm7Ra0PUJuOXWjZjygWQ5emFJWMiCDbGxX9U=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dnUHbrxZtU3w3icH/gSKVZ63FWAuigVRhSdIcLnLEZHT1Y6nQUxgi3wxAYfDSIH61
	 e7ydnOqfhdBFqcG9uxpasJ0/HrUpgjpPZNbU7Osd4LewWnM1i0ESsFGU9+GAJU0x2Z
	 8Id4tRvzjjALCN8X6hWQTeK72DSlwxMx8q8EV47gfNM8lcenSiHhL3+ObSyEc9YQYZ
	 YFtdZAvypO71XpWAx8DNm4C9HdOnu7jpFskyBJbt9vBdIEzc+5W5DNyX12PyKKAqeT
	 XpPEIQKKXrNxbjKETatt3x5gJVE4Rry7r8cKirM+lCydCpDglXbm6douugrYMB0zaj
	 UCm00HlbZdwZg==
Date: Wed, 8 Sep 2021 17:15:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Penny Zheng <penny.zheng@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, Wei.Chen@arm.com, jbeulich@suse.com
Subject: Re: [PATCH v6 7/7] xen/arm: introduce allocate_static_memory
In-Reply-To: <20210908095248.545981-8-penny.zheng@arm.com>
Message-ID: <alpine.DEB.2.21.2109081709370.10523@sstabellini-ThinkPad-T480s>
References: <20210908095248.545981-1-penny.zheng@arm.com> <20210908095248.545981-8-penny.zheng@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Penny Zheng wrote:
> This commit introduces a new function allocate_static_memory to allocate
> static memory as guest RAM for Domain on Static Allocation.
                             ^ for domains.

> It uses acquire_domstatic_pages to acquire pre-configured static memory
> for this domain, and uses guest_physmap_add_pages to set up P2M table.
      ^ the                                                  ^the 

> These pre-defined static memory banks shall be mapped to the usual guest
> memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> xen/include/public/arch-arm.h.
> 
> In order to deal with the trouble of count-to-order conversion when page number
> is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
> a new function guest_physmap_add_pages to cope with adding guest RAM p2m
> mapping with nr_pages.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>  xen/arch/arm/domain_build.c | 161 +++++++++++++++++++++++++++++++++++-
>  xen/arch/arm/p2m.c          |   7 +-
>  xen/include/asm-arm/p2m.h   |  11 +++
>  3 files changed, 173 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 206038d1c0..b011cc4789 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -480,6 +480,162 @@ fail:
>            (unsigned long)kinfo->unassigned_mem >> 10);
>  }
>  
> +#ifdef CONFIG_STATIC_MEMORY
> +static bool __init append_static_memory_to_bank(struct domain *d,
> +                                                struct membank *bank,
> +                                                mfn_t smfn,
> +                                                paddr_t size)
> +{
> +    int res;
> +    unsigned int nr_pages = PFN_DOWN(size);
> +    /* Infer next GFN. */
> +    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
> +
> +    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
> +    if ( res )
> +    {
> +        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
> +        return false;
> +    }
> +
> +    bank->size = bank->size + size;
> +
> +    return true;
> +}
> +
> +/* Allocate memory from static memory as RAM for one specific domain d. */
> +static void __init allocate_static_memory(struct domain *d,
> +                                          struct kernel_info *kinfo,
> +                                          const struct dt_device_node *node)
> +{
> +    const struct dt_property *prop;
> +    u32 addr_cells, size_cells, reg_cells;
> +    unsigned int nr_banks, gbank, bank = 0;
> +    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
> +    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
> +    const __be32 *cell;
> +    u64 tot_size = 0;
> +    paddr_t pbase, psize, gsize;
> +    mfn_t smfn;
> +    int res;
> +
> +    prop = dt_find_property(node, "xen,static-mem", NULL);
> +    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
> +                               &addr_cells) )
> +    {
> +        printk(XENLOG_ERR
> +               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
> +        goto fail;
> +    }
> +
> +    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
> +                               &size_cells) )
> +    {
> +        printk(XENLOG_ERR
> +               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
> +        goto fail;
> +    }
> +    reg_cells = addr_cells + size_cells;
> +
> +    /*
> +     * The static memory will be mapped in the guest at the usual guest memory
> +     * addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> +     * xen/include/public/arch-arm.h.
> +     */
> +    gbank = 0;
> +    gsize = ramsize[gbank];
> +    kinfo->mem.bank[gbank].start = rambase[gbank];
> +
> +    cell = (const __be32 *)prop->value;
> +    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
> +
> +    for ( ; bank < nr_banks; bank++ )
> +    {
> +        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
> +        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
> +
> +        smfn = maddr_to_mfn(pbase);
> +        res = acquire_domstatic_pages(d, smfn, PFN_DOWN(psize), 0);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR
> +                   "%pd: failed to acquire static memory: %d.\n", d, res);
> +            goto fail;
> +        }
> +
> +        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
> +               d, bank, pbase, pbase + psize);
> +
> +        while ( 1 )
> +        {
> +            /* Map as much as possible the static range to the guest bank */
> +            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
> +                                               min(psize, gsize)) )
> +                goto fail;
> +
> +            /*
> +             * The current physical bank is fully mapped.
> +             * Handle the next physical bank.
> +             */
> +            if ( gsize >= psize )
> +            {
> +                gsize = gsize - psize;
> +                break;
> +            }
> +            /*
> +             * When current guest bank is not enough to map, exhaust
> +             * the current one and seek to the next.
> +             * Before seeking to the next, check if we still have available
> +             * guest bank.
> +             */
> +            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
> +            {
> +                printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
> +                goto fail;
> +            }
> +            else
> +            {
> +                psize = psize - gsize;
> +                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
> +                /* Update to the next guest bank. */
> +                gbank++;
> +                gsize = ramsize[gbank];
> +                kinfo->mem.bank[gbank].start = rambase[gbank];
> +            }
> +        }
> +
> +        tot_size += psize;
> +    }
> +
> +    kinfo->mem.nr_banks = ++gbank;
> +
> +    kinfo->unassigned_mem -= tot_size;
> +    /*
> +     * The property 'memory' should match the amount of memory given to the
> +     * guest.
> +     * Currently, it is only possible to either acquire static memory or let
> +     * Xen allocate. *Mixing* is not supported'.
                                                 ^ stray '

These are all NITs that I'd be happy to fix on commit if the series
doesn't need another update.

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>




> +     */
> +    if ( kinfo->unassigned_mem )
> +    {
> +        printk(XENLOG_ERR
> +               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\". Unsupported configuration.\n");
> +        goto fail;
> +    }
> +
> +    return;
> +
> + fail:
> +    panic("Failed to allocate requested static memory for domain %pd.", d);
> +}
> +#else
> +static void __init allocate_static_memory(struct domain *d,
> +                                          struct kernel_info *kinfo,
> +                                          const struct dt_device_node *node)
> +{
> +}
> +#endif
> +
>  static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                                     const struct dt_device_node *node)
>  {
> @@ -2453,7 +2609,10 @@ static int __init construct_domU(struct domain *d,
>      /* type must be set before allocate memory */
>      d->arch.type = kinfo.type;
>  #endif
> -    allocate_memory(d, &kinfo);
> +    if ( !dt_find_property(node, "xen,static-mem", NULL) )
> +        allocate_memory(d, &kinfo);
> +    else
> +        allocate_static_memory(d, &kinfo, node);
>  
>      rc = prepare_dtb_domU(d, &kinfo);
>      if ( rc < 0 )
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index eff9a105e7..6e01e83967 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -1293,11 +1293,8 @@ out:
>      return resolved;
>  }
>  
> -static inline int p2m_insert_mapping(struct domain *d,
> -                                     gfn_t start_gfn,
> -                                     unsigned long nr,
> -                                     mfn_t mfn,
> -                                     p2m_type_t t)
> +int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
> +                       mfn_t mfn, p2m_type_t t)
>  {
>      struct p2m_domain *p2m = p2m_get_hostp2m(d);
>      int rc;
> diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
> index 6a2108398f..f885cc522b 100644
> --- a/xen/include/asm-arm/p2m.h
> +++ b/xen/include/asm-arm/p2m.h
> @@ -300,6 +300,9 @@ int map_dev_mmio_region(struct domain *d,
>                          unsigned long nr,
>                          mfn_t mfn);
>  
> +int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
> +                       mfn_t mfn, p2m_type_t t);
> +
>  int guest_physmap_add_entry(struct domain *d,
>                              gfn_t gfn,
>                              mfn_t mfn,
> @@ -315,6 +318,14 @@ static inline int guest_physmap_add_page(struct domain *d,
>      return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
>  }
>  
> +static inline int guest_physmap_add_pages(struct domain *d,
> +                                          gfn_t gfn,
> +                                          mfn_t mfn,
> +                                          unsigned int nr_pages)
> +{
> +    return p2m_insert_mapping(d, gfn, nr_pages, mfn, p2m_ram_rw);
> +}
> +
>  mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn);
>  
>  /* Look up a GFN and take a reference count on the backing page. */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 01:11:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 01:11:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182539.330138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8b7-0003V4-9H; Thu, 09 Sep 2021 01:11:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182539.330138; Thu, 09 Sep 2021 01:11:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8b7-0003Ux-5T; Thu, 09 Sep 2021 01:11:25 +0000
Received: by outflank-mailman (input) for mailman id 182539;
 Thu, 09 Sep 2021 01:11:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO8b5-0003Ur-Gg
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 01:11:23 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1ede8905-734b-4846-9312-87f74a23a52c;
 Thu, 09 Sep 2021 01:11:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 84DB361158;
 Thu,  9 Sep 2021 01:11:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ede8905-734b-4846-9312-87f74a23a52c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631149881;
	bh=/1kkwotCVNZUHTkoMefJIcmv7eGp0QtMuywZS/5fL3s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Dz2Uv7gvY675/x8u6HUrvP3sjIc/utMM/+dfUoUN+qTbLbCmMzecmi0ycsvxTKA3v
	 INzIMH9ZpdW4tJuMvJJ5WaLVXra77AtaO3uZGK4PraQxNTs9xh+jv1ksvd+ad3cg6W
	 sRQFRK6rwEu+B30XnPnn21IlBrapY4BBHYO2/PhU177T/w+NBHODCLHg7czf8m+Rcg
	 8Bgn082nruikUnq2foWOTxJibZ+TrnrT1jp5Du3Zb1tzzVa4UP91FMaR62d/VXmLNO
	 PAuu0eTgllxgJwWJ8Z+NyrWy2pg/09cFv82BX8p1DS4EwJ5jqV4YG1WbV2Z+Rj4xJN
	 Q4KttPAHv28DA==
Date: Wed, 8 Sep 2021 18:11:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Rahul Singh <Rahul.Singh@arm.com>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Paul Durrant <paul@xen.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v1 03/14] xen/pci: solve compilation error on ARM with
 ACPI && HAS_PCI
In-Reply-To: <a8982dae-0928-3230-f10d-ada277bc8397@suse.com>
Message-ID: <alpine.DEB.2.21.2109081802190.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <97d39d3ee398d6018bdcaf745f00d039df6a92ef.1629366665.git.rahul.singh@arm.com> <5fad114f-06c4-120e-0a48-da7045fb85c5@suse.com> <76520AA8-D02C-4207-A722-33D935E76776@arm.com>
 <a8982dae-0928-3230-f10d-ada277bc8397@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-554672202-1631149778=:10523"
Content-ID: <alpine.DEB.2.21.2109081809450.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-554672202-1631149778=:10523
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109081809451.10523@sstabellini-ThinkPad-T480s>

On Fri, 20 Aug 2021, Jan Beulich wrote:
> On 20.08.2021 13:41, Rahul Singh wrote:
> > Hi Jan
> > 
> >> On 20 Aug 2021, at 8:06 am, Jan Beulich <jbeulich@suse.com> wrote:
> >>
> >> On 19.08.2021 14:02, Rahul Singh wrote:
> >>> Compilation error is observed when ACPI and HAS_PCI is enabled for ARM
> >>> architecture. Move the code under CONFIG_X86 flag to gate the code for
> >>> ARM.
> >>
> >> Please give at least one example of what it is that fails to compile.
> >> What an appropriate action is depends on the nature of the error(s),
> >> and from looking at the enclosed code I cannot easily see what it
> >> might be that breaks for Arm.
> > 
> > I am observing below error when enabled ACPI && HAS_PCI for ARM.
> > 
> > prelink.o: In function `pcie_aer_get_firmware_first’:
> > /xen/drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'
> > aarch64-linux-gnu-ld: /home/rahsin01/work/xen/pci-passthrough-upstream/xen/xen/.xen-syms.0: hidden symbol `apei_hest_parse' isn't defined
> > 
> > I found that apei/ is only enabled for x86 and pcie_aer_get_firmware_first() is only called from x86 code.
> > obj-$(CONFIG_X86) += apei/
> > 
> > I am not sure whether we need this code for ARM architecture 
> > that is why I gate the code for ARM via CONFIG_X86
> 
> So you Arm folks will probably want to settle on that aspect first. What
> is wanted to keep things building depends on that.

Reading the APEI description, it looks like there might be some use for
it on ARM but it would work a bit differently from x86 as there are no
NMIs on ARM. So enabling APEI on ARM is not just a matter of enabling
the build of apei/, it is not going to be straightforward.

For the scope of this series (which is actually about PCI), I would
leave it alone, and keep apei/ x86 only, which means #ifdefing
pcie_aer_get_firmware_first.

I would just add an in-code comment saying "APEI not supported on ARM
yet". Another option would be to introduce a symbol like HAS_ACPI_APEI
but it is a bit overkill for this.
--8323329-554672202-1631149778=:10523--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 01:16:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 01:16:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182545.330149 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8fp-00049o-SJ; Thu, 09 Sep 2021 01:16:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182545.330149; Thu, 09 Sep 2021 01:16:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8fp-00049R-PN; Thu, 09 Sep 2021 01:16:17 +0000
Received: by outflank-mailman (input) for mailman id 182545;
 Thu, 09 Sep 2021 01:16:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mO8fo-00049L-Ez
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 01:16:16 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 87243880-110b-11ec-b192-12813bfff9fa;
 Thu, 09 Sep 2021 01:16:15 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9FB9C61041;
 Thu,  9 Sep 2021 01:16:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87243880-110b-11ec-b192-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631150174;
	bh=5hoIVC8+6r3rj3BgWbn+kgdUpMZoWk4zh27+6RX9yrk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fmwEPgFzw+ajEBOXu/nlBmQvzwv+6cLJGjGT85w34gBxXGeR2VOenNx+IE2psodFL
	 LVBB1wuDXgHC2CYgxDcN5WrNUO/K28ZJZiJtDBV8ZVqA2/Ngg7SFDZorK0GPLad0Nm
	 sbhZrY1FH8fDZdxrbG5Xk2l6uRlikBuD8Ii8IDmOfXsT/TR1tYtHgyTpH/EysaClZ1
	 MZmUBY1xdFO97kg4NtkVkOjrtR1nH4mplgpyk+JMKp2Krd4hw8J+yfq3Z3mNEUSMze
	 KWjulgKM14TfTXgQCC4EOlBObbW1tvFTlBRE+rtrQaOiJ0b1x/vuDzxYHja/bcoQj9
	 d8bwGG6xleisg==
Date: Wed, 8 Sep 2021 18:16:13 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
In-Reply-To: <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109081814120.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> pci_init(..) will be called during xen startup to initialize and probe
> the PCI host-bridge driver.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/device.h |  1 +
>  2 files changed, 55 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index dc55d23778..d1c9cf997d 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -14,13 +14,67 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>  
> +#include <xen/acpi.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
>  #include <xen/pci.h>
> +#include <xen/param.h>
>  
>  int arch_pci_clean_pirqs(struct domain *d)
>  {
>      return 0;
>  }
>  
> +static int __init dt_pci_init(void)
> +{
> +    struct dt_device_node *np;
> +    int rc;
> +
> +    dt_for_each_device_node(dt_host, np)
> +    {
> +        rc = device_init(np, DEVICE_PCI, NULL);
> +        if( !rc )
> +            continue;
> +        /*
> +         * Ignore the following error codes:
> +         *   - EBADF: Indicate the current is not an pci
                                                     ^ a

> +         *   - ENODEV: The pci device is not present or cannot be used by
> +         *     Xen.
> +         */
> +        else if ( rc != -EBADF && rc != -ENODEV )
> +        {
> +            printk(XENLOG_ERR "No driver found in XEN or driver init error.\n");
> +            return rc;
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +#ifdef CONFIG_ACPI
> +static void __init acpi_pci_init(void)

If the DT init function returns int, then it would make sense for the
ACPI init function to return int as well?


> +{
> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
> +    return;
> +}
> +#else
> +static inline void __init acpi_pci_init(void) { }
> +#endif
> +
> +static int __init pci_init(void)
> +{
> +    if ( acpi_disabled )
> +        dt_pci_init();
> +    else
> +        acpi_pci_init();
> +
> +    pci_segments_init();
> +
> +    return 0;
> +}
> +__initcall(pci_init);
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index ee7cff2d44..5ecd5e7bd1 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -34,6 +34,7 @@ enum device_class
>      DEVICE_SERIAL,
>      DEVICE_IOMMU,
>      DEVICE_GIC,
> +    DEVICE_PCI,
>      /* Use for error */
>      DEVICE_UNKNOWN,
>  };
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 01:23:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 01:23:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182552.330160 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8mb-0005ug-KS; Thu, 09 Sep 2021 01:23:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182552.330160; Thu, 09 Sep 2021 01:23:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO8mb-0005uZ-Gp; Thu, 09 Sep 2021 01:23:17 +0000
Received: by outflank-mailman (input) for mailman id 182552;
 Thu, 09 Sep 2021 01:23:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO8ma-0005uP-Ri; Thu, 09 Sep 2021 01:23:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO8ma-0000X7-Hn; Thu, 09 Sep 2021 01:23:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mO8ma-0001uo-7u; Thu, 09 Sep 2021 01:23:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mO8ma-0005AQ-7A; Thu, 09 Sep 2021 01:23:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pTSH6SvP77Bncj2kDk8ZvcPvCeWofmQEO9vuAJZ+gbk=; b=Xb0IEbUP08EmAfvAP+AkS0tkNt
	afC24cXRBoknOVU7WzCJrhzi5E1n7wjtmDIJ8YUk+m6j8zUUm6pDhj28kBMbvN9gfETpnuR3nVjYz
	5IGPZhfZKZ5CWKZHDGknPYcqtJuaEM9axVrQXfFTxmJa0tcnLciShkPOHmRyi3/mSUMA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164884-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164884: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start:fail:allowable
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=bd662023e683850c085e98c8ff8297142c2dd9f2
X-Osstest-Versions-That:
    qemuu=abf7aee72ea66944a62962603e4c2381f5e473e7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 01:23:16 +0000

flight 164884 qemu-mainline real [real]
flight 164893 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164884/
http://logs.test-lab.xenproject.org/osstest/logs/164893/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl           8 xen-boot            fail pass in 164893-retest

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 164878
 test-armhf-armhf-xl-rtds     14 guest-start              fail REGR. vs. 164878

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 164893 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 164893 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164878
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164878
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164878
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164878
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164878
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164878
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164878
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                bd662023e683850c085e98c8ff8297142c2dd9f2
baseline version:
 qemuu                abf7aee72ea66944a62962603e4c2381f5e473e7

Last test of basis   164878  2021-09-07 23:39:10 Z    1 days
Testing same since   164884  2021-09-08 12:38:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   abf7aee72e..bd662023e6  bd662023e683850c085e98c8ff8297142c2dd9f2 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 02:21:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 02:21:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182560.330174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO9gM-00059I-20; Thu, 09 Sep 2021 02:20:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182560.330174; Thu, 09 Sep 2021 02:20:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mO9gL-00059B-U7; Thu, 09 Sep 2021 02:20:53 +0000
Received: by outflank-mailman (input) for mailman id 182560;
 Thu, 09 Sep 2021 02:20:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fJWR=N7=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mO9gJ-000595-UZ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 02:20:52 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.55]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 83b89824-7c2a-48c8-91a5-33aa97d6b8e5;
 Thu, 09 Sep 2021 02:20:49 +0000 (UTC)
Received: from AS9PR06CA0240.eurprd06.prod.outlook.com (2603:10a6:20b:45e::9)
 by AM6PR08MB3320.eurprd08.prod.outlook.com (2603:10a6:209:48::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Thu, 9 Sep
 2021 02:20:37 +0000
Received: from AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45e:cafe::ec) by AS9PR06CA0240.outlook.office365.com
 (2603:10a6:20b:45e::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Thu, 9 Sep 2021 02:20:37 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT016.mail.protection.outlook.com (10.152.16.142) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 9 Sep 2021 02:20:36 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Thu, 09 Sep 2021 02:20:36 +0000
Received: from 889472a1c476.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 04B3ABDE-7AEE-486F-B949-01532FF30FFB.1; 
 Thu, 09 Sep 2021 02:20:25 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 889472a1c476.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 09 Sep 2021 02:20:25 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB4510.eurprd08.prod.outlook.com (2603:10a6:803:fc::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 02:20:23 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Thu, 9 Sep 2021
 02:20:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83b89824-7c2a-48c8-91a5-33aa97d6b8e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nynvBABngUwOW1F6uuWXjRTaGIbbj4LSXRhrLUHPG6o=;
 b=GF1FNwGhLpiZyRcfYE6ToFJFUohl2pSnfULbBpXllZNx8LyValGNMob1YNDaL7l+TSVUkt71RbFSXfNEhMYxdXvNBh4xaYDTpu7ZEiHqKI1OwA2zbV9GXAVegsN6Jo7/3hXuTVUC0iaZhlEZo0EftOGUL5TyYbHbCgPAnnG3i1Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dmEKAMHB4Jt+BpvIveYKcR0Kg61UQ3KWWlHjhxIVE4neNwSMM+pesTH8gBrhmbp8/rtYo7uA50EHQ3QJuxNE9b1euBbdt7JK5yUoNdU1lrfsvK9Nxpu7jJ2/SXyKp2OnKNXfsNm5KTBK+aqFG987nE9ucp87h1sWzVR7JZqnMjP6BW2RXYm0OBhaAbnfJiNJzXnzgFhzqRQ5+5ZM+cOJwO8LgpFbtfRbOcyxeVZ3V6sl6IUkiwIbs64xYTj/YOvy4y61OlyDip6o9RkV0erdTFfDkUEsasj6ldUt5YmYDf3HUuUxIHfDFhRxaZiBe6T4Fe0x7mbPIb+dyUKH5IEd3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nynvBABngUwOW1F6uuWXjRTaGIbbj4LSXRhrLUHPG6o=;
 b=jWLZextee5GhuNRz/Xwxifg1M78o7GQovja4uSmlpFd6gazee9I/OukQIrOAHNu5wJ7vY1ulORooudlIb6FAI1ejHVPK8BbICRC1Ol0TalCyehUxaql28IgNYzl2SqO0/m19zFkSZk6wCjZ+lvMoZW7ZVSQAEotJxWr7zR+yYst6Bp3l9cTpdgehtRgZ/UKHIjl11FOJBqX3JabISMmdPm7VB3nNRVjti8tNs9xVqY+fukkMLVm/UetkS1z0S90VisnOA019DHsMONzE3bgabit5LwRwrTjT9rU5Rz4BQl66xQ/55U+md5ZIpFvu2XeUhqzPe4PsJn149lMoXFtbSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nynvBABngUwOW1F6uuWXjRTaGIbbj4LSXRhrLUHPG6o=;
 b=GF1FNwGhLpiZyRcfYE6ToFJFUohl2pSnfULbBpXllZNx8LyValGNMob1YNDaL7l+TSVUkt71RbFSXfNEhMYxdXvNBh4xaYDTpu7ZEiHqKI1OwA2zbV9GXAVegsN6Jo7/3hXuTVUC0iaZhlEZo0EftOGUL5TyYbHbCgPAnnG3i1Y=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>, "jbeulich@suse.com"
	<jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: RE: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
Thread-Topic: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
Thread-Index: AQHXpQ1TrWpL2DewC0y7cp371PcYX6ua9x7w
Date: Thu, 9 Sep 2021 02:20:21 +0000
Message-ID:
 <VE1PR08MB521580BE54919E53282AE691F7D59@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
 <20210908095248.545981-6-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109081654290.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109081654290.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 00451682A6507C4CBB9A8C033ED51150.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: e74e92b9-5c60-46b7-175d-08d97338693d
x-ms-traffictypediagnostic: VI1PR08MB4510:|AM6PR08MB3320:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB3320111A92A6B3C5E577DC00F7D59@AM6PR08MB3320.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:142;OLM:142;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 5B91eR3oVmGWyYk4hIpBpc+gJkLqupHZd/QbU0eNDAD8d2o/cg/6HQnSyRba6iEgTd0IEk/tchWXEakbSnXz2myBFZfY8De5Bu4ein5l6WL5nTJKNpdyXypVcyStAUm/VCLmz8BFpTstoIuGS1L6NHbHeeXEgQACOf//Sw9JQFoaBUvXztjH9m98qN6Bs9qHm8wMU8cmtjoc0i2eBGjTD37XboatqKnmUo0O500RtbOeBdXjSUKuxYHFMIvjJPVI6shQwJ3oI+FJIOlp31N9EyFE/00AtVQBLikU9RmlcbmDDlHT5v1fdHVPXBqKlGIK/oLeQGmRAh9wUXMQwM5sDL//1IX33QDWgGafFzWMUkZtAJ9mjLaCIzH5UgVhzaMCd9y683o5pPQPZukgSvbCQ05r3OnUxXqAlXoZG+/K9RvhRaXs6zCc8IJ4UABGNj8w1emfRB8WJKzejg0cCbTZi0CyIteuQHmrkTaAWmHvk6ZAmAJGiV89t7vtSbEVgitw6gg0q4McpYtRESXw8JriEQ6ysVUbEiGaIiMJMls3mCQGhu6xEYjeXvluM1dcvz7UsAqWnBuKExxSw9DqjvUgCyFc1KMbQV4nupxiKxvz7HJmz/aKEShWTNeHXSyjnafvTDzTUD3W4pPxFR4h24rJMCZVaGENLPzvHuubX/19Dplfo316nycBK1XYI90vkgfvEI44RLDR4D9AWX0PtDXl4A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(376002)(346002)(396003)(39860400002)(9686003)(86362001)(52536014)(83380400001)(5660300002)(4326008)(33656002)(55016002)(478600001)(71200400001)(2906002)(66446008)(26005)(53546011)(6506007)(8676002)(7696005)(64756008)(66946007)(76116006)(66556008)(66476007)(8936002)(110136005)(122000001)(38100700002)(38070700005)(316002)(186003)(54906003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4510
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9f3231dd-d2af-4e70-2964-08d973386100
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kR3QifmV0V/JqrFbzsxsZLdw3TfOyc4j9vmi/rwKPU6E7glTLb0lXwzDM2evZDAiNi/4uYEXNBUdB3fevctrJ0c5qlgff64w+0Z/vCMYVOcb6hu2SmY1T00KXg8Yfk2oxZ5MAHMa1VeKLS11OIoUZg7HjJhCiB1v15Bq2igHYwfm7UTebdNYM+ccJJG0HIQnHAfwRVZucGvVIfqBPPkOXWRJf1GQG+cYmXCUMyl9vfOOVAoRR5ABI+QuWsX8J0KrahrMfwI/XA0mk/Hfx8jDSfsI5t/06iy38ue0bE0YKyyIghCbDRSQXYW9T8/CBqv3tuWm9hQGtsT9GznrtOe82ga+yeFzOjyQBrzJwDWX0iyVErhr7g54w4TMk9WsjoCZjZOVgDODWFv8GZohoNmdiavVELu4+13mOZt8FCF/gQs0xX2e0rjuZPkHaMi/IHjca4nYtro+FNkGpGn6+9P6UvJ1sJou/BN/RzSEUkSr3o4jTnLQZR5bctnvkNo42DrRWY97Z7JjD/T2n3Ajt48X3Zt2WusEvDn3OfAl/QvJXWULW5eG32+SOSyTPFSCYjQt0OBXEvHdkRwyh2Jzxg/K+lVhH2clpnSP8JghHgNyFTcK2qIAxyPUV+gd6VP+7+E+Utqix0k2CTsJY/MtQaDKSYhTpMklLHx036mQhFtViixj70J69ZwcTSkeXhk0MXdhPj5vTsppHjtg8VRKopBIqg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(396003)(39860400002)(36840700001)(46966006)(82310400003)(26005)(8676002)(2906002)(4326008)(70586007)(186003)(5660300002)(70206006)(81166007)(54906003)(478600001)(6506007)(8936002)(55016002)(52536014)(356005)(336012)(53546011)(9686003)(82740400003)(110136005)(47076005)(86362001)(83380400001)(316002)(36860700001)(33656002)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 02:20:36.9880
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e74e92b9-5c60-46b7-175d-08d97338693d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3320

Hi Jan

> -----Original Message-----
> From: Stefano Stabellini <sstabellini@kernel.org>
> Sent: Thursday, September 9, 2021 7:58 AM
> To: Penny Zheng <Penny.Zheng@arm.com>
> Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.or=
g;
> Bertrand Marquis <Bertrand.Marquis@arm.com>; Wei Chen
> <Wei.Chen@arm.com>; jbeulich@suse.com
> Subject: Re: [PATCH v6 5/7] xen: re-define assign_pages and introduce a n=
ew
> function assign_page
>=20
> On Wed, 8 Sep 2021, Penny Zheng wrote:
> > In order to deal with the trouble of count-to-order conversion when
> > page number is not in a power-of-two, this commit re-define
> > assign_pages for nr pages and introduces a new helper assign_page for
> original page with a single order.
> >
> > Signed-off-by: Penny Zheng <penny.zheng@arm.com>
>=20
> The patch looks correct to me, however I don't think I understood Jan's
> request to the previous version of this patch, so I can't tell if you add=
ressed his
> concerns.
>=20

Would you like to take a look at whether I address your concerns in this ve=
rsion? Thx.

>=20
> > ---
> >  xen/arch/x86/pv/dom0_build.c |  2 +-
> >  xen/common/grant_table.c     |  2 +-
> >  xen/common/memory.c          |  6 +++---
> >  xen/common/page_alloc.c      | 21 +++++++++++++--------
> >  xen/include/xen/mm.h         |  6 ++++++
> >  5 files changed, 24 insertions(+), 13 deletions(-)
> >
> > diff --git a/xen/arch/x86/pv/dom0_build.c
> > b/xen/arch/x86/pv/dom0_build.c index d7f9e04b28..7787cc8fca 100644
> > --- a/xen/arch/x86/pv/dom0_build.c
> > +++ b/xen/arch/x86/pv/dom0_build.c
> > @@ -568,7 +568,7 @@ int __init dom0_construct_pv(struct domain *d,
> >          else
> >          {
> >              while ( count-- )
> > -                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 0, 0) )
> > +                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 1, 0)
> > + )
> >                      BUG();
> >          }
> >          initrd->mod_end =3D 0;
> > diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index
> > ee61603a97..50f5f83023 100644
> > --- a/xen/common/grant_table.c
> > +++ b/xen/common/grant_table.c
> > @@ -2358,7 +2358,7 @@ gnttab_transfer(
> >           * is respected and speculative execution is blocked according=
ly
> >           */
> >          if ( unlikely(!evaluate_nospec(okay)) ||
> > -            unlikely(assign_pages(e, page, 0, MEMF_no_refcount)) )
> > +            unlikely(assign_pages(e, page, 1, MEMF_no_refcount)) )
> >          {
> >              bool drop_dom_ref;
> >
> > diff --git a/xen/common/memory.c b/xen/common/memory.c index
> > 74babb0bd7..9cef8790ff 100644
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -728,8 +728,8 @@ static long
> memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t)
> arg)
> >          /* Assign each output page to the domain. */
> >          for ( j =3D 0; (page =3D page_list_remove_head(&out_chunk_list=
)); ++j )
> >          {
> > -            if ( assign_pages(d, page, exch.out.extent_order,
> > -                              MEMF_no_refcount) )
> > +            if ( assign_page(page, exch.out.extent_order, d,
> > +                             MEMF_no_refcount) )
> >              {
> >                  unsigned long dec_count;
> >                  bool_t drop_dom_ref;
> > @@ -797,7 +797,7 @@ static long
> memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t)
> arg)
> >       * cleared PGC_allocated.
> >       */
> >      while ( (page =3D page_list_remove_head(&in_chunk_list)) )
> > -        if ( assign_pages(d, page, 0, MEMF_no_refcount) )
> > +        if ( assign_pages(d, page, 1, MEMF_no_refcount) )
> >          {
> >              BUG_ON(!d->is_dying);
> >              free_domheap_page(page);
> > diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index
> > ba7adc80db..bb19bb10ff 100644
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -2261,7 +2261,7 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
> > int assign_pages(
> >      struct domain *d,
> >      struct page_info *pg,
> > -    unsigned int order,
> > +    unsigned long nr,
> >      unsigned int memflags)
> >  {
> >      int rc =3D 0;
> > @@ -2281,7 +2281,7 @@ int assign_pages(
> >      {
> >          unsigned int extra_pages =3D 0;
> >
> > -        for ( i =3D 0; i < (1ul << order); i++ )
> > +        for ( i =3D 0; i < nr; i++ )
> >          {
> >              ASSERT(!(pg[i].count_info & ~PGC_extra));
> >              if ( pg[i].count_info & PGC_extra ) @@ -2290,18 +2290,18
> > @@ int assign_pages(
> >
> >          ASSERT(!extra_pages ||
> >                 ((memflags & MEMF_no_refcount) &&
> > -                extra_pages =3D=3D 1u << order));
> > +                extra_pages =3D=3D nr));
> >      }
> >  #endif
> >
> >      if ( pg[0].count_info & PGC_extra )
> >      {
> > -        d->extra_pages +=3D 1u << order;
> > +        d->extra_pages +=3D nr;
> >          memflags &=3D ~MEMF_no_refcount;
> >      }
> >      else if ( !(memflags & MEMF_no_refcount) )
> >      {
> > -        unsigned int tot_pages =3D domain_tot_pages(d) + (1 << order);
> > +        unsigned int tot_pages =3D domain_tot_pages(d) + nr;
> >
> >          if ( unlikely(tot_pages > d->max_pages) )
> >          {
> > @@ -2313,10 +2313,10 @@ int assign_pages(
> >      }
> >
> >      if ( !(memflags & MEMF_no_refcount) &&
> > -         unlikely(domain_adjust_tot_pages(d, 1 << order) =3D=3D (1 << =
order)) )
> > +         unlikely(domain_adjust_tot_pages(d, nr) =3D=3D nr) )
> >          get_knownalive_domain(d);
> >
> > -    for ( i =3D 0; i < (1 << order); i++ )
> > +    for ( i =3D 0; i < nr; i++ )
> >      {
> >          ASSERT(page_get_owner(&pg[i]) =3D=3D NULL);
> >          page_set_owner(&pg[i], d);
> > @@ -2331,6 +2331,11 @@ int assign_pages(
> >      return rc;
> >  }
> >
> > +int assign_page(struct page_info *pg, unsigned int order, struct domai=
n *d,
> > +                unsigned int memflags) {
> > +    return assign_pages(d, pg, 1UL << order, memflags); }
> >
> >  struct page_info *alloc_domheap_pages(
> >      struct domain *d, unsigned int order, unsigned int memflags) @@
> > -2373,7 +2378,7 @@ struct page_info *alloc_domheap_pages(
> >                  pg[i].count_info =3D PGC_extra;
> >              }
> >          }
> > -        if ( assign_pages(d, pg, order, memflags) )
> > +        if ( assign_page(pg, order, d, memflags) )
> >          {
> >              free_heap_pages(pg, order, memflags & MEMF_no_scrub);
> >              return NULL;
> > diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index
> > 8e8fb5a615..a74e93eba8 100644
> > --- a/xen/include/xen/mm.h
> > +++ b/xen/include/xen/mm.h
> > @@ -133,8 +133,14 @@ void heap_init_late(void);
> >
> >  int assign_pages(
> >      struct domain *d,
> > +    struct page_info *pg,
> > +    unsigned long nr,
> > +    unsigned int memflags);
> > +
> > +int assign_page(
> >      struct page_info *pg,
> >      unsigned int order,
> > +    struct domain *d,
> >      unsigned int memflags);
> >
> >  /* Dump info to serial console */
> > --
> > 2.25.1
> >


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 03:54:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 03:54:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182574.330185 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOB8o-000093-2W; Thu, 09 Sep 2021 03:54:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182574.330185; Thu, 09 Sep 2021 03:54:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOB8n-00008w-VF; Thu, 09 Sep 2021 03:54:21 +0000
Received: by outflank-mailman (input) for mailman id 182574;
 Thu, 09 Sep 2021 03:54:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/ytW=N7=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mOB8m-00008q-5S
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 03:54:20 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.46]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9b09cd18-1121-11ec-b199-12813bfff9fa;
 Thu, 09 Sep 2021 03:54:17 +0000 (UTC)
Received: from DU2PR04CA0180.eurprd04.prod.outlook.com (2603:10a6:10:2b0::35)
 by DB6PR0802MB2311.eurprd08.prod.outlook.com (2603:10a6:4:87::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 03:54:16 +0000
Received: from DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::e) by DU2PR04CA0180.outlook.office365.com
 (2603:10a6:10:2b0::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Thu, 9 Sep 2021 03:54:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT011.mail.protection.outlook.com (10.152.20.95) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 9 Sep 2021 03:54:15 +0000
Received: ("Tessian outbound 16951d3c485e:v103");
 Thu, 09 Sep 2021 03:54:15 +0000
Received: from d037be74e984.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 04888784-824D-475B-A304-45662293B985.1; 
 Thu, 09 Sep 2021 03:54:10 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d037be74e984.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 09 Sep 2021 03:54:10 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB5339.eurprd08.prod.outlook.com (2603:10a6:10:114::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Thu, 9 Sep
 2021 03:54:08 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4478.025; Thu, 9 Sep 2021
 03:54:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b09cd18-1121-11ec-b199-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=T8oybjAyomrzS4Eifsd/X85wSsOcjifh+5Oma4Ki+yg=;
 b=ccYRDVfF9A0DR0oeFXcEnMmH+FeJWHHTLP/t6li/tnywPlKAkoYgPUzWYmcE2vvelKZgZq157WZ1GCsi8Jldzo0EQDJlMyKbMxtbX4tNEfwvV2Y5P4QuMopMPeBynkVTTnP02JW35CMpaKSHJ8mIqIWvE7qZFKYBPPPdDNx1+U4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CkmHvKqwZfh6bMkI8v44beoyuR9zEvzWbumigAVSi9VKSZ2KBU7NCJ/K/EZYqsptDi3oJCxAmlZyhcF3vLICh0NejGCfoVsKGUNtHwAsVWHLwOVbBqIdj6D8lYF9AWWszxukYXLu8H5Ky8gk8Dni3roiXYY/msAbeJXZ3hAf+bKHGtPx2mNDUzJGBM1b3gFieSBDdxrpKRA3NN4GN2Pm6GmhUh24Enr6RCthQh9ksIWnxaELGDUZcAQGb///F2JyldOgb6UWWjTWDfe41ycgLr62HDqx+72ydsgxQ0arwIA077bHmr7cIhDxHKw93uK9WjMMqL/f6v9P5lonsWOPvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=T8oybjAyomrzS4Eifsd/X85wSsOcjifh+5Oma4Ki+yg=;
 b=EyJj2WXnfamE+zzN1x+2gu52UMWo7BiPGHoquAlg0wc4mquH9Ph98svaGtMb7tsVwjEi8OmXIe+Ju7h2GaJiBCZXQ84Mdoahin+6pmr+Djzbm7WCbZJbcXSWBiAl0Zq+Se5E3/jP9lU22nmMWK/CbobtH153f4QFxDRuc2iw7sK0lcxc9KYzV/ne8aby+CLO/PXtS7Bk+1m+8+HwvhKQ3jqxe1rFfL6EV+Io3GKiyfhUvM2rCcvE3lLK4jIJSRxe0EFOb+jIfpuRqFmwMy/G7TPwxv8qF19lGrw5mQ22MW92188LKTGanTZ61ukMlZ8rkXgkIZ5+xY1h2yWuri42fA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=T8oybjAyomrzS4Eifsd/X85wSsOcjifh+5Oma4Ki+yg=;
 b=ccYRDVfF9A0DR0oeFXcEnMmH+FeJWHHTLP/t6li/tnywPlKAkoYgPUzWYmcE2vvelKZgZq157WZ1GCsi8Jldzo0EQDJlMyKbMxtbX4tNEfwvV2Y5P4QuMopMPeBynkVTTnP02JW35CMpaKSHJ8mIqIWvE7qZFKYBPPPdDNx1+U4=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>
Subject: RE: [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse device
 tree memory node
Thread-Topic: [XEN RFC PATCH 23/40] xen/arm: introduce a helper to parse
 device tree memory node
Thread-Index:
 AQHXjps6uVXfLZKD7EKSvvhEptOzS6uINJ4AgAAs9PCAAHGGgIAAOFuQgBDWjrCAAQNtgIAAS/rA
Date: Thu, 9 Sep 2021 03:54:08 +0000
Message-ID:
 <DB9PR08MB6857B389B9B67FFE43C48C059ED59@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210811102423.28908-1-wei.chen@arm.com>
 <20210811102423.28908-24-wei.chen@arm.com>
 <alpine.DEB.2.21.2108271757250.17851@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857618112DC045293A449699EC99@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <5c9a196a-8ac3-1173-f402-01d99ccc23e4@xen.org>
 <AS8PR08MB68531741D1035B35A19EF34F9EC99@AS8PR08MB6853.eurprd08.prod.outlook.com>
 <DB9PR08MB68572B980E3E688F3E946C329ED49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109081458300.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109081458300.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 369D10C5F7DA6B468C6858BD0BBCA7DC.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 78f0b843-6e52-412f-5a30-08d973457e60
x-ms-traffictypediagnostic: DB8PR08MB5339:|DB6PR0802MB2311:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB231120E9CCA644D46F22B8979ED59@DB6PR0802MB2311.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HpfBWotXCbuOLWKUCqEsfPofsfIotW0JM8NewCwy71CxS/wOMF23NRIYkPPKajsInqdBeBACDl6cdpK1pABCkNHDOw3NhjjdyCWbZ+sVfEKekwKUJpVcTGOS2ewtptGDCsjYAEPrvBuXm1NrPPo4b1C5lp4rHPYwYwbgoCFr/YoqQTyLOAcsXgW/6G7cQ6opI7iOx2GHOUo2iPS+5Sqh54yqec+bZHnLEDYdWHQfMLAdASo2F6K9gAXBu+AP2r/4cKj3tDH1lojpH9UW4aSSNBRsyaNXMUe8vfmGv3QQ/XiW2TBfHAsxcHXuRrB6x1VtqGMmIOKKxz/OdW48VTeJx0DZ8kNYsPUFM3yjGXQvGeIzBV1lYoxaWRB9bDgP2wiCLGVowszWvnN2ycCb67un9fLps8R9b9FmWX2IQls4qUgeQqD21yeK9SBQvrxM+xQTV7XP6qmGFo2Yd9mQGNmPwG//J+6SKH3sOQhUddEzoV3cXXXeheh3+ef+R0To1CwG0VJI30++6NqyHDoCwkxUVnptCl3bIFp064/yIQlADtIpV/XHvXITa0x/ZqC/jeoSUYGLJ5mXrf+tSvkjvgs13gxuQU9N7Cu3nhZCZGSl+1GSyflNxwCjEH2eKZcu7z430F3LyICMcy3IgO872kCfe24RTIUr0knq3Xyjq9/wzJ+3C/bGIROL0ImfKfpwmJ2IiW10RBwftXUVDg6UWKlSOWLZ3NfGm0CKsdtcJ+Qahd4NBhMTN6gmrAsRvbMdJWgMd3uu094Z7JCBghbJohRs9a9wNZ0nDpo7RPT1J2KSR0pUdJfwn07OV3ZtFLWe87dwvcMEeGa/yms5+GezLteYCA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(376002)(346002)(39860400002)(136003)(396003)(33656002)(186003)(86362001)(7696005)(71200400001)(5660300002)(9686003)(122000001)(66476007)(38100700002)(26005)(76116006)(66946007)(6916009)(38070700005)(53546011)(316002)(2906002)(4326008)(8676002)(52536014)(66446008)(66556008)(64756008)(478600001)(83380400001)(55016002)(8936002)(6506007)(966005)(54906003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5339
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d287b816-f325-4212-c771-08d9734579ed
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	P5I644LU7xapFwFHyDLpHwokpOcmrFB+9epCGPqNi5RkqQQJWxU0yTVmYGqlUGQXYPqyYpootb/hMdOAhcvMAP6Ea6LSmpkmivje0hQHFVQgl6nzrL+73Xi9OgbusorZJJG+SnXoTUl9PdJwhaSk/RzSIY3aA4pFuV1zN7bNmvaBmpcVXFSIXVFU2fzzHyTXy4wAGIu7NX8+/wFHi8Q/J5bHT5NkH5mD7W3YzIu8sBwomGd7el7PwDo8tJ1qhMsWRvoLvMpGibNhbWPtQAz2KmXlDAcFUMNbi5psbEPnxBhIkR2lnXduXS5tTzbfpeKWA1+WALC/dHSfktK3hEO9faGDmEWRbO0fSDIpXtxdIfV+reeSC/QyXslv2NmRvgNgIaWqwi9ab249kivDcAy9UgcuO2oLepeFmpiccNLCwmcFaIMGGb7k3b0yW6w0cFeAH1+QWCu1vCSoZCFflrCx69FjoJz8Jy8Es7D/ZJ0yFQAbdf0LYbikCj4ShrtI1RMg6pf8Z76IhhOdQBlAX4MFnsfVULIsNbJNP7i3GncWKZdzbNFii5JUWa01q2VMeRC+BBZXf697pzACs7NrS06rOqVxp+oJBQDZ5po9CLY90E+qKyT6cHj3vw4UwdcPdvZKMhZXsu12ADXmEWNdsM+28uimbvobYLnidVWtD3HBx9Sm/Y/9DGZfXCZP1uk7jQZa0RpyEXii502tLkQyehsrURZAfE4arvtZlTlMiehzmQs9wLcFYDd40VC4mVlgwcfdXGe+7tXVRv/+sMyWHzCGb+YAo3YxZF9UKN7235F0qUPx9llROGyiRh+9shlv+U9yuR34LlpqIS0gOF+EZbrayw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(376002)(136003)(396003)(36840700001)(46966006)(26005)(8676002)(4326008)(83380400001)(966005)(8936002)(86362001)(33656002)(107886003)(186003)(36860700001)(81166007)(82740400003)(9686003)(55016002)(336012)(356005)(5660300002)(6862004)(47076005)(54906003)(478600001)(6506007)(52536014)(82310400003)(70206006)(316002)(7696005)(53546011)(70586007)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 03:54:15.9737
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 78f0b843-6e52-412f-5a30-08d973457e60
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2311

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjnI1SA2OjMyDQo+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IEp1bGll
biBHcmFsbCA8anVsaWVuQHhlbi5vcmc+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPHNzdGFiZWxs
aW5pQGtlcm5lbC5vcmc+OyB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IEJlcnRyYW5k
IE1hcnF1aXMNCj4gPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IEphbiBCZXVsaWNoIDxqYmV1
bGljaEBzdXNlLmNvbT4NCj4gU3ViamVjdDogUkU6IFtYRU4gUkZDIFBBVENIIDIzLzQwXSB4ZW4v
YXJtOiBpbnRyb2R1Y2UgYSBoZWxwZXIgdG8gcGFyc2UNCj4gZGV2aWNlIHRyZWUgbWVtb3J5IG5v
ZGUNCj4gDQo+IE9uIFdlZCwgOCBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiA+ID4+
PiBAQCAtNTUsNiArNTcsNzkgQEAgc3RhdGljIGludCBfX2luaXQNCj4gPiA+ID4gPj4gZHRiX251
bWFfcHJvY2Vzc29yX2FmZmluaXR5X2luaXQobm9kZWlkX3Qgbm9kZSkNCj4gPiA+ID4gPj4+ICAg
ICAgIHJldHVybiAwOw0KPiA+ID4gPiA+Pj4gICB9DQo+ID4gPiA+ID4+Pg0KPiA+ID4gPiA+Pj4g
Ky8qIENhbGxiYWNrIGZvciBwYXJzaW5nIG9mIHRoZSBtZW1vcnkgcmVnaW9ucyBhZmZpbml0eSAq
Lw0KPiA+ID4gPiA+Pj4gK3N0YXRpYyBpbnQgX19pbml0IGR0Yl9udW1hX21lbW9yeV9hZmZpbml0
eV9pbml0KG5vZGVpZF90IG5vZGUsDQo+ID4gPiA+ID4+PiArICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBwYWRkcl90IHN0YXJ0LCBwYWRkcl90IHNpemUpDQo+ID4gPiA+ID4+PiArew0K
PiA+ID4gPiA+Pj4gKyAgICBzdHJ1Y3Qgbm9kZSAqbmQ7DQo+ID4gPiA+ID4+PiArICAgIHBhZGRy
X3QgZW5kOw0KPiA+ID4gPiA+Pj4gKyAgICBpbnQgaTsNCj4gPiA+ID4gPj4+ICsNCj4gPiA+ID4g
Pj4+ICsgICAgaWYgKCBzcmF0X2Rpc2FibGVkKCkgKQ0KPiA+ID4gPiA+Pj4gKyAgICAgICAgcmV0
dXJuIC1FSU5WQUw7DQo+ID4gPiA+ID4+PiArDQo+ID4gPiA+ID4+PiArICAgIGVuZCA9IHN0YXJ0
ICsgc2l6ZTsNCj4gPiA+ID4gPj4+ICsgICAgaWYgKCBudW1fbm9kZV9tZW1ibGtzID49IE5SX05P
REVfTUVNQkxLUyApDQo+ID4gPiA+ID4+PiArICAgIHsNCj4gPiA+ID4gPj4+ICsgICAgICAgIGRw
cmludGsoWEVOTE9HX1dBUk5JTkcsDQo+ID4gPiA+ID4+PiArICAgICAgICAgICAgICAgICJUb28g
bWFueSBudW1hIGVudHJ5LCB0cnkgYmlnZ2VyDQo+IE5SX05PREVfTUVNQkxLUw0KPiA+ID4gXG4i
KTsNCj4gPiA+ID4gPj4+ICsgICAgICAgIGJhZF9zcmF0KCk7DQo+ID4gPiA+ID4+PiArICAgICAg
ICByZXR1cm4gLUVJTlZBTDsNCj4gPiA+ID4gPj4+ICsgICAgfQ0KPiA+ID4gPiA+Pj4gKw0KPiA+
ID4gPiA+Pj4gKyAgICAvKiBJdCBpcyBmaW5lIHRvIGFkZCB0aGlzIGFyZWEgdG8gdGhlIG5vZGVz
IGRhdGEgaXQgd2lsbCBiZQ0KPiA+ID4gdXNlZA0KPiA+ID4gPiA+PiBsYXRlciAqLw0KPiA+ID4g
PiA+Pj4gKyAgICBpID0gY29uZmxpY3RpbmdfbWVtYmxrcyhzdGFydCwgZW5kKTsNCj4gPiA+ID4g
Pj4+ICsgICAgLyogTm8gY29uZmxpY3RpbmcgbWVtb3J5IGJsb2NrLCB3ZSBjYW4gc2F2ZSBpdCBm
b3IgbGF0ZXINCj4gdXNhZ2UNCj4gPiA+ICovOw0KPiA+ID4gPiA+Pj4gKyAgICBpZiAoIGkgPCAw
ICkNCj4gPiA+ID4gPj4+ICsgICAgICAgIGdvdG8gc2F2ZV9tZW1ibGs7DQo+ID4gPiA+ID4+PiAr
DQo+ID4gPiA+ID4+PiArICAgIGlmICggbWVtYmxrX25vZGVpZFtpXSA9PSBub2RlICkgew0KPiA+
ID4gPiA+Pj4gKyAgICAgICAgLyoNCj4gPiA+ID4gPj4+ICsgICAgICAgICAqIE92ZXJsYXBzIHdp
dGggb3RoZXIgbWVtYmxrIGluIHRoZSBzYW1lIG5vZGUsIHdhcm5pbmcNCj4gPiA+IGhlcmUuDQo+
ID4gPiA+ID4+PiArICAgICAgICAgKiBUaGlzIG1lbWJsayB3aWxsIGJlIG1lcmdlZCB3aXRoIGNv
bmZsaWN0ZWQgbWVtYmxrDQo+IGxhdGVyLg0KPiA+ID4gPiA+Pj4gKyAgICAgICAgICovDQo+ID4g
PiA+ID4+PiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcNCj4gPiA+ID4gPj4+ICsgICAg
ICAgICAgICAgICAiRFQ6IE5VTUEgTk9ERSAldSAoJSJQUkl4NjQNCj4gPiA+ID4gPj4+ICsgICAg
ICAgICAgICAgICAiLSUiUFJJeDY0Iikgb3ZlcmxhcHMgd2l0aCBpdHNlbGYgKCUiUFJJeDY0Ii0N
Cj4gPiA+ID4gPj4gJSJQUkl4NjQiKVxuIiwNCj4gPiA+ID4gPj4+ICsgICAgICAgICAgICAgICBu
b2RlLCBzdGFydCwgZW5kLA0KPiA+ID4gPiA+Pj4gKyAgICAgICAgICAgICAgIG5vZGVfbWVtYmxr
X3JhbmdlW2ldLnN0YXJ0LA0KPiA+ID4gbm9kZV9tZW1ibGtfcmFuZ2VbaV0uZW5kKTsNCj4gPiA+
ID4gPj4+ICsgICAgfSBlbHNlIHsNCj4gPiA+ID4gPj4+ICsgICAgICAgIC8qDQo+ID4gPiA+ID4+
PiArICAgICAgICAgKiBDb25mbGljdCB3aXRoIG1lbWJsayBpbiBvdGhlciBub2RlLCB0aGlzIGlz
IGFuIGVycm9yLg0KPiA+ID4gPiA+Pj4gKyAgICAgICAgICogVGhlIE5VTUEgaW5mb3JtYXRpb24g
aXMgaW52YWxpZCwgTlVNQSB3aWxsIGJlIHR1cm4NCj4gb2ZmLg0KPiA+ID4gPiA+Pj4gKyAgICAg
ICAgICovDQo+ID4gPiA+ID4+PiArICAgICAgICBwcmludGsoWEVOTE9HX0VSUg0KPiA+ID4gPiA+
Pj4gKyAgICAgICAgICAgICAgICJEVDogTlVNQSBOT0RFICV1ICglIlBSSXg2NCItJSINCj4gPiA+
ID4gPj4+ICsgICAgICAgICAgICAgICBQUkl4NjQiKSBvdmVybGFwcyB3aXRoIE5PREUgJXUgKCUi
UFJJeDY0Ii0NCj4gPiA+ID4gJSJQUkl4NjQiKVxuIiwNCj4gPiA+ID4gPj4+ICsgICAgICAgICAg
ICAgICBub2RlLCBzdGFydCwgZW5kLCBtZW1ibGtfbm9kZWlkW2ldLA0KPiA+ID4gPiA+Pj4gKyAg
ICAgICAgICAgICAgIG5vZGVfbWVtYmxrX3JhbmdlW2ldLnN0YXJ0LA0KPiA+ID4gbm9kZV9tZW1i
bGtfcmFuZ2VbaV0uZW5kKTsNCj4gPiA+ID4gPj4+ICsgICAgICAgIGJhZF9zcmF0KCk7DQo+ID4g
PiA+ID4+PiArICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4gPiA+ID4gPj4+ICsgICAgfQ0KPiA+
ID4gPiA+Pj4gKw0KPiA+ID4gPiA+Pj4gK3NhdmVfbWVtYmxrOg0KPiA+ID4gPiA+Pj4gKyAgICBu
ZCA9ICZub2Rlc1tub2RlXTsNCj4gPiA+ID4gPj4+ICsgICAgaWYgKCAhbm9kZV90ZXN0X2FuZF9z
ZXQobm9kZSwgbWVtb3J5X25vZGVzX3BhcnNlZCkgKSB7DQo+ID4gPiA+ID4+PiArICAgICAgICBu
ZC0+c3RhcnQgPSBzdGFydDsNCj4gPiA+ID4gPj4+ICsgICAgICAgIG5kLT5lbmQgPSBlbmQ7DQo+
ID4gPiA+ID4+PiArICAgIH0gZWxzZSB7DQo+ID4gPiA+ID4+PiArICAgICAgICBpZiAoIHN0YXJ0
IDwgbmQtPnN0YXJ0ICkNCj4gPiA+ID4gPj4+ICsgICAgICAgICAgICBuZC0+c3RhcnQgPSBzdGFy
dDsNCj4gPiA+ID4gPj4+ICsgICAgICAgIGlmICggbmQtPmVuZCA8IGVuZCApDQo+ID4gPiA+ID4+
PiArICAgICAgICAgICAgbmQtPmVuZCA9IGVuZDsNCj4gPiA+ID4gPj4+ICsgICAgfQ0KPiA+ID4g
PiA+Pj4gKw0KPiA+ID4gPiA+Pj4gKyAgICBwcmludGsoWEVOTE9HX0lORk8gIkRUOiBOVU1BIG5v
ZGUgJXUgJSJQUkl4NjQiLSUiUFJJeDY0IlxuIiwNCj4gPiA+ID4gPj4+ICsgICAgICAgICAgIG5v
ZGUsIHN0YXJ0LCBlbmQpOw0KPiA+ID4gPiA+Pj4gKw0KPiA+ID4gPiA+Pj4gKyAgICBub2RlX21l
bWJsa19yYW5nZVtudW1fbm9kZV9tZW1ibGtzXS5zdGFydCA9IHN0YXJ0Ow0KPiA+ID4gPiA+Pj4g
KyAgICBub2RlX21lbWJsa19yYW5nZVtudW1fbm9kZV9tZW1ibGtzXS5lbmQgPSBlbmQ7DQo+ID4g
PiA+ID4+PiArICAgIG1lbWJsa19ub2RlaWRbbnVtX25vZGVfbWVtYmxrc10gPSBub2RlOw0KPiA+
ID4gPiA+Pj4gKyAgICBudW1fbm9kZV9tZW1ibGtzKys7DQo+ID4gPiA+ID4+DQo+ID4gPiA+ID4+
DQo+ID4gPiA+ID4+IElzIGl0IHBvc3NpYmxlIHRvIGhhdmUgbm9uLWNvbnRpZ291cyByYW5nZXMg
b2YgbWVtb3J5IGZvciBhDQo+IHNpbmdsZQ0KPiA+ID4gTlVNQQ0KPiA+ID4gPiA+PiBub2RlPw0K
PiA+ID4gPiA+Pg0KPiA+ID4gPiA+PiBMb29raW5nIGF0IHRoZSBEVCBiaW5kaW5ncyBhbmQgTGlu
dXggaW1wbGVtZW50YXRpb24sIGl0IHNlZW1zDQo+ID4gPiBwb3NzaWJsZS4NCj4gPiA+ID4gPj4g
SGVyZSwgaXQgc2VlbXMgdGhhdCBub2RlX21lbWJsa19yYW5nZS9tZW1ibGtfbm9kZWlkIGNvdWxk
IGhhbmRsZQ0KPiBpdCwNCj4gPiA+ID4gPj4gYnV0IG5vZGVzIGNvdWxkbid0Lg0KPiA+ID4gPiA+
DQo+ID4gPiA+ID4gWWVzLCB5b3UncmUgcmlnaHQuIEkgY29waWVkIHRoaXMgY29kZSBmb3IgeDg2
IEFDUEkgTlVNQS4gRG9lcyBBQ1BJDQo+ID4gPiBhbGxvdw0KPiA+ID4gPiA+IG5vbi1jb250aWd1
b3VzIHJhbmdlcyBvZiBtZW1vcnkgZm9yIGEgc2luZ2xlIE5VTUEgbm9kZSB0b28/DQo+ID4gPiA+
DQo+ID4gPiA+IEkgY291bGRuJ3QgZmluZCBhbnkgcmVzdHJpY3Rpb24gZm9yIEFDUEkuIEFsdGhv
dWdoLCBJIG9ubHkgYnJpZWZseQ0KPiA+ID4gPiBsb29rZWQgYXQgdGhlIHNwZWMuDQo+ID4gPiA+
DQo+ID4gPiA+ID4gSWYgeWVzLCBJIHRoaW5rDQo+ID4gPiA+ID4gdGhpcyB3aWxsIGFmZmVjdCB4
ODYgQUNQSSBOVU1BIHRvby4gSW4gbmV4dCB2ZXJzaW9uLCB3ZSBwbGFuIHRvDQo+IG1lcmdlDQo+
ID4gPiA+ID4gZHRiX251bWFfbWVtb3J5X2FmZmluaXR5X2luaXQgYW5kIGFjcGlfbnVtYV9tZW1v
cnlfYWZmaW5pdHlfaW5pdA0KPiBpbnRvDQo+ID4gPiBhDQo+ID4gPiA+ID4gbmV1dHJhbCBmdW5j
dGlvbi4gU28gd2UgY2FuIGZpeCB0aGVtIGF0IHRoZSBzYW1lIHRpbWUuDQo+ID4gPiA+ID4NCj4g
PiA+ID4gPiBJZiBub3QsIG1heWJlIHdlIGhhdmUgdG8ga2VlcCB0aGUgZGl2ZXJzaXR5IGZvciBk
dGIgYW5kIEFDUEkgaGVyZS4NCj4gPiA+ID4NCj4gPiA+ID4gSSBhbSBub3QgZW50aXJlbHkgc3Vy
ZSB3aGF0IHlvdSBtZWFuLiBBcmUgeW91IHNheWluZyBpZiBBQ1BJIGRvZXNuJ3QNCj4gPiA+ID4g
YWxsb3cgbm9uLWNvbnRpZ3VvdXMgcmFuZ2VzIG9mIG1lbW9yeSwgdGhlbiB3ZSBzaG91bGQga2Vl
cCB0aGUNCj4gPiA+ID4gaW1wbGVtZW50YXRpb24gc2VwYXJhdGVkPw0KPiA+ID4gPg0KPiA+ID4g
PiBJZiBzbywgdGhlbiBJIGRpc2FncmVlIHdpdGggdGhhdC4gSXQgaXMgZmluZSB0byBoYXZlIGNv
ZGUgdGhhdA0KPiBzdXBwb3J0cw0KPiA+ID4gPiBtb3JlIHRoYW4gd2hhdCBhIGZpcm13YXJlIHRh
YmxlIHN1cHBvcnRzLiBUaGUgbWFpbiBiZW5lZml0IGlzIGxlc3MNCj4gY29kZQ0KPiA+ID4gPiBh
bmQgdGhlcmVmb3JlIGxlc3MgbG9uZyB0ZXJtIG1haW50ZW5hbmNlICh3aXRoIHRoZSBjdXJyZW50
IHNvbHV0aW9uDQo+IHdlDQo+ID4gPiA+IHdvdWxkIG5lZWQgdG8gY2hlY2sgYm90aCB0aGUgQUNQ
SSBhbmQgRFQgaW1wbGVtZW50YXRpb24gaWYgdGhlcmUgaXMNCj4gYQ0KPiA+ID4gPiBidWcgaW4g
b25lKS4NCj4gPiA+ID4NCj4gPiA+DQo+ID4gPiBZZXMsIEkgYWdyZWUuDQo+ID4gPg0KPiA+DQo+
ID4gSSBhbSBsb29raW5nIGZvciBzb21lIG1ldGhvZHMgdG8gYWRkcmVzcyB0aGlzIGNvbW1lbnQu
IEN1cnJlbnQgIm5vZGVzIg0KPiA+IGhhcyBub3QgY29uc2lkZXJlZCB0aGUgc2l0dWF0aW9uIG9m
IG1lbW9yeSBhZGRyZXNzZXMgb2YgZGlmZmVyZW50IE5VTUENCj4gPiBub2RlcyBjYW4gYmUgaW50
ZXJsZWF2ZWQuDQo+ID4NCj4gPiBUaGlzIGNvZGUgZXhpc3RzIGluIHg4NiBOVU1BIGltcGxlbWVu
dGF0aW9uLiBJIHRoaW5rIGl0IG1heSBiZSBiYXNlZCBvbg0KPiA+IG9uZSBlYXJseSB2ZXJzaW9u
IG9mIExpbnV4IHg4NiBOVU1BIGltcGxlbWVudGF0aW9uLiBJbiByZWNlbnQgTGludXgNCj4gPiBj
b2RlLCBib3RoIEFDUEkvbnVtYS9zcmF0LmNbMV0gYW5kIHg4NiBOVU1BIGNvZGVbMl0gYXJlIG5v
dCB1c2luZw0KPiA+ICJub2RlcyIgdG8gcmVjb3JkIE5VTUEgbWVtb3J5IGFkZHJlc3MgYm91bmRh
cnkuIFRoZXkgZG9uJ3QgZGVwZW5kDQo+ID4gb24gIm5vZGVzIiB0byBkbyBzYW5pdHkgY2hlY2su
DQo+ID4NCj4gPiBUbyBmaXggaXQsIHdlJ2QgYmV0dGVyIHRvIHVwZ3JhZGUgdGhlIHg4NiBOVU1B
IGRyaXZlci4gSXQgd2lsbCBtYWtlDQo+ID4gYSBncmVhdCBhZmZlY3QgZm9yIFhlbi14ODYuIEFu
ZCBJIHRoaW5rIGl0IG1pZ2h0IG91dCBvZiB0aGlzIHNlcmllcw0KPiA+IHNjb3BlLiBDYW4gd2Ug
Y3JlYXRlIGFub3RoZXIgdGhyZWFkIHRvIGRpc2N1c3MgYWJvdXQgaXQ/DQo+ID4NCj4gPiBPciBj
b3VsZCB5b3UgZ2l2ZSBtZSBzdWdnZXN0aW9ucyB0aGF0IHdlIGNhbiB1c2Ugc29tZSBzaW1wbGUg
d2F5cw0KPiA+IHRvIGZpeCBpdD8NCj4gDQo+IEl0IGxvb2tzIGxpa2UgdGhhdCB3ZSB3b3VsZCBo
YXZlIHRvIHJlcGxhY2UgYWxsIHRoZSBub2RlLT5zdGFydCAvDQo+IG5vZGUtPmVuZCBjaGVja3Mg
d2l0aCBub2RlX21lbWJsa19yYW5nZSBjaGVja3MuIFRoZXJlIGFyZSBhIGZldyBvZiB0aGVtDQo+
IGluIHZhbGlkX251bWFfcmFuZ2UsIGNvbmZsaWN0aW5nX21lbWJsa3MsIGN1dG9mZl9ub2RlLA0K
PiBub2Rlc19jb3Zlcl9tZW1vcnkuIEl0IHdvdWxkbid0IGJlIHRyaXZpYWwuDQo+IA0KPiBBbHRo
b3VnaCBJIGRvIHRoaW5rIHRoYXQgbm9uLWNvbnRpZ3VvdXMgbWVtb3J5IGZvciBOVU1BIG5vZGVz
IGlzDQo+IGltcG9ydGFudCB0byBzdXBwb3J0LCB0aGUgcGF0Y2ggc2VyaWVzIGlzIGFscmVhZHkg
NDAgcGF0Y2hlcy4gSSBkb24ndA0KPiB0aGluayBpdCBpcyBhIGdvb2QgaWRlYSB0byBhZGQgb3Ro
ZXIgc2lnbmlmaWNhbnQgY2hhbmdlcyB0byBpdC4gSQ0KPiB3b3VsZG4ndCB1cGdyYWRlIHRoZSB4
ODYgTlVNQSBkcml2ZXIgbm93LiBJZiB3ZSBjYW4ndCBmaW5kIGEgYmV0dGVyIHdheSwNCj4gd2Ug
Y2FuIHByb2NlZWQgYXMgeW91IGFyZSBkb2luZyBpbiB0aGlzIHZlcnNpb24sIHdpdGggdGhlIGtu
b3duIGdhcCB0aGF0DQo+IHdlIGNhbid0IGRlYWwgd2l0aCBub24tY29udGlnaW91cyBtZW1vcnkg
Zm9yIE5VTUEgbm9kZXMsIGFuZCBmaXggaXQgd2l0aA0KPiBhIGZvbGxvdy11cCBzZXJpZXMgbGF0
ZXIuIEluIHRoYXQgY2FzZSB3ZSB3b3VsZCB3YW50IHRvIGhhdmUgYW4gZXhwbGljaXQNCj4gY2hl
Y2sgZm9yIG5vbi1jb250aWd1b3VzIG1lbW9yeSBmb3IgTlVNQSBub2RlcyBpbg0KPiBkdGJfbnVt
YV9tZW1vcnlfYWZmaW5pdHlfaW5pdCBhbmQgZXJyb3Igb3V0IGlmIGZvdW5kLg0KPiANCg0KWWVz
LCBJIHRoaW5rIHRoaXMgbWF5IGJlIGEgbW9yZSBhcHByb3ByaWF0ZSBtZXRob2QgYXQgcHJlc2Vu
dC4NCkkgd291bGQgYWRkIHNvbWUgY29kZSB0byBkbyBleHBsaWNpdCBjaGVjayBhbmQgZ2l2ZSB3
YXJuaW5nL2Vycm9yLg0KIA0KPiANCj4gPiBBbHNvLCBvbiBMaW51eCwgTlVNQSBpbXBsZW1lbnRh
dGlvbnMgZm9yIHg4NiBhcmUgZGlmZmVyZW50IGZyb20gQXJtNjQNCj4gPiBhbmQgUklTQy1WIGlt
cGxlbWVudGF0aW9ucy5bM10NCj4gPg0KPiA+IFsxXQ0KPiBodHRwczovL2dpdGh1Yi5jb20vdG9y
dmFsZHMvbGludXgvYmxvYi9tYXN0ZXIvZHJpdmVycy9hY3BpL251bWEvc3JhdC5jDQo+ID4gWzJd
IGh0dHBzOi8vZ2l0aHViLmNvbS90b3J2YWxkcy9saW51eC9ibG9iL21hc3Rlci9hcmNoL3g4Ni9t
bS9udW1hLmMNCj4gPiBbM10NCj4gaHR0cHM6Ly9naXRodWIuY29tL3RvcnZhbGRzL2xpbnV4L2Js
b2IvbWFzdGVyL2RyaXZlcnMvYmFzZS9hcmNoX251bWEuYw0KPiANCj4gDQo+IEluIGdlbmVyYWws
IEkgbGlrZSB0aGUgaWRlYSBvZiBzaGFyaW5nIGNvZGUgYXMgbXVjaCBhcyBwb3NzaWJsZSBiZXR3
ZWVuDQo+IGFyY2hpdGVjdHVyZXMgKHg4NiwgQVJNLCBldGMuKSBhbmQgYmV0d2VlbiBEVC9BQ1BJ
IGJlY2F1c2UgaXQgbWFrZXMgdGhlDQo+IGNvZGUgbWFpbnRhaW5hbmNlIGVhc2llciBhbmQgb25l
IG1pZ2h0IGV2ZW4gZ2FpbiBjZXJ0YWluIGZlYXR1cmVzIGZvcg0KPiBmcmVlLg0KPiANCj4gSG93
ZXZlciwgdGhlIGV4Y2VyY2lzZSBvZiBzaGFyaW5nIGNvZGUgc2hvdWxkbid0IHRha2Ugc2lnbmlm
aWNhbnQNCj4gYWRkaXRpb25hbCBlZmZvcnRzLiBJbiBmYWN0LCBpdCBzaG91bGQgZGVjcmVhc2Ug
dGhlIG92ZXJhbGwgZWZmb3J0Og0KPiBpbnN0ZWFkIG9mIHdyaXRpbmcgbmV3IGNvZGUgb25lIGp1
c3QgdGFrZSBleGlzdGluZyBjb2RlIGFuZCBtb3ZlIGl0IHRvDQo+IGNvbW1vbi4NCj4gDQo+IElu
IHRoaXMgaW5zdGFuY2UsIEkgdGhpbmsgaXQgd291bGQgYmUgZ29vZCB0byBiZSBhYmxlIHRvIHNo
YXJlIHRoZSBOVU1BDQo+IGluaXRpYWxpemF0aW9uIGNvZGUgYmV0d2VlbiB4ODYvQVJNIGFuZCBB
Q1BJL0RUIGlmIGl0IGRvZXNuJ3QgY2F1c2UNCj4gZXh0cmEgZWZmb3J0cy4NCj4gDQo+IEhlcmUg
dGhlIGV4dHJhIGVmZm9ydCB0aGF0IG15IHByZXZpb3VzIGNvbW1lbnQgbWlnaHQgY2F1c2UgZG9l
c24ndA0KPiBkZXJpdmUgZnJvbSB4ODYvQVJNIG9yIERUL0FDUEkgY29kZSBzaGFyaW5nLiBJdCBk
ZXJpdmVzIGZyb20gdGhlIGZhY3QNCj4gdGhhdCBvdXIgZXhpc3RpbmcgY29kZSBkb2Vzbid0IGRl
YWwgd2l0aCBub24tY29udGlnb3VzIG1lbW9yeSBmb3IgTlVNQQ0KPiBub2RlcyB1bmZvcnR1bmF0
ZWx5LiBUaGF0IGlzIHNvbWV0aGluZyB3ZSBuZWVkIHRvIGZpbmQgYSB3YXkgdG8gY29wZQ0KPiB3
aXRoIGFueXdheS4NCg0KWWVzLiBJIHBvc3RlZCBhYm92ZSBsaW5rcyBkaWRuJ3QgbWVhbiB0byBj
cmVhdGUgZGl2ZXJzaXR5IGZvciBBcm0veDg2Lg0KVGhvdWdoLCBJJ20gbm90IHN1cmUgZXhhY3Rs
eSB3aHkgTGludXggZG9lcyB0aGlzLiBCdXQgSSB0aGluayBmb3IgWGVuLA0Kd2Ugc3RpbGwgY2Fu
IHRyeSB0byBzaGFyZSBjb2RlIGZvciBBcm0veDg2IGFuZCBEVC9BQ1BJLg0KDQpDaGVlcnMsDQpX
ZWkgQ2hlbg0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 04:23:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 04:23:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182581.330196 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOBat-0004QJ-8y; Thu, 09 Sep 2021 04:23:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182581.330196; Thu, 09 Sep 2021 04:23:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOBat-0004QC-5t; Thu, 09 Sep 2021 04:23:23 +0000
Received: by outflank-mailman (input) for mailman id 182581;
 Thu, 09 Sep 2021 04:23:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOBar-0004Q6-UW
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 04:23:22 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5a970af7-1b8b-4c40-a9db-0730e868be01;
 Thu, 09 Sep 2021 04:23:20 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1894NG15017834; 
 Thu, 9 Sep 2021 04:23:17 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayayu0126-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 04:23:15 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4147.eurprd03.prod.outlook.com (2603:10a6:208:cf::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Thu, 9 Sep
 2021 04:23:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 04:23:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a970af7-1b8b-4c40-a9db-0730e868be01
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Trt2hdYIxmmzOI/aMsJVP5fb4Wmq6mGazEMGLBSY5Bfa8TXeXl+CV9AJXB+G4IjWk1OSH4AcXYa1yrlUuo7T3XzoyejdsdchtBa/x8ZBPeuhpgTitwxOSBkf+IXa6dULFkOyx2UZMWLWO5GC7ttpFMSlgCSIQgx4mCgAViLtgwPzPy3YKZfTkpWi4FyjhprZBoR8UzneMuvSLYWffYavHPswlzVes37DZm6S+NmBvio8WDFLfLH/PGMgY5WYA43U+HsVEhX2IzGY8j/12itzzzn4fJT3GXnM/uC0yJ+D/yiJLgC4K2xUcjO5osvd/5CriAv5L3yXn6LTNyO7mhLcsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5/nJC3Ejo4dEXeyJ7Q7cuDsRvlFO+tjpz0+gtHrFn0g=;
 b=CgGFyoZgXM8FcF7Q+fTOd8rUuri/BzuiaIFnfTxPpueP6+sy/xyA4sqwr6Umh2aknWNu1jRb26r0R4Trj8Oxy0ot6KhKa2x6wsSSs7DQeVeKFMVWzzGobioCNlUhbIDjuJLkJLHzNYePiaQaKxYNRnXk+CPmOrIFGh7o1T9yYRkOPGH5IkY4TlSZEwKeYAXL/XizXdvrdtXrXp0mxwldWKydMERkTqgvF2QqCO0PiGGbXDlW9r/NifvF46G+lcvogj+MPeV78fKAZvPbG8mZdl6Ha84IorNlIwqLDSrM8YUyu/1KDd1fM7Ah3gUpwUhHCU9jqk/cZ6iQXnXzBNbZKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5/nJC3Ejo4dEXeyJ7Q7cuDsRvlFO+tjpz0+gtHrFn0g=;
 b=3AYlwfPT+Tkq0ERUMc2thENEq8EbiTxEmtL2jZpZYLiPXQT1rqNR4E8HGyHW7i13HRmbtRcFM81z74NwPM+UeYjqAjh8LAH1AIjYBd4wLcEr9LhyJx354HjuvW/0B4asGw3DoV2oKh2As9nq4oK9sNlfX5pZKvMoZyD3foiQXzutvao+bPPzQDYfOSoGPqtyFtBqyk7e+UKPiDdNrNwfBa61ZWr6PVZzfgLUJJ32l8alg2WAqjPfhbJ1W3dkIVcgyZFLtt7q7AhgeeXIuDkaSwEDDqWMHmKqAs4VekawQMEIxeuelXennUZ6IiwIhQlRqqfTMYTxr3pEL0hkSTpYLA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        Rahul Singh <rahul.singh@arm.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 9/9] vpci/header: Use pdev's domain instead of vCPU
Thread-Topic: [PATCH 9/9] vpci/header: Use pdev's domain instead of vCPU
Thread-Index: AQHXoKux1/ZjebFtnUG1WufptJXen6uXHZ8AgAQFzQA=
Date: Thu, 9 Sep 2021 04:23:05 +0000
Message-ID: <63217fb0-c9cb-3777-7cd7-1046a27b62ba@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-10-andr2000@gmail.com>
 <214a8300-e818-321c-7618-185c56684db0@suse.com>
In-Reply-To: <214a8300-e818-321c-7618-185c56684db0@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3f014c78-d658-486a-9502-08d97349851d
x-ms-traffictypediagnostic: AM0PR03MB4147:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB41472FED748A8D24B4496B17E7D59@AM0PR03MB4147.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2657;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 qiiqYDyZCRxXKr9cFGCz6D60FnoQhHWOogbk4CEc0UsgiPcHqRUFY57Q7npW22e37NmMnyTrODXfwZeL11FZuvPrc7PoDbpfCR5TFR/RI7ROt2vgDXOOup3l8CRi/v742m8X7a5BNLjsTj1iaU1EZ26udPVR/6H7KkzJ+imOzhyrIKMViYBHITDH/j6YTjR0lNfHtoyRx0uQ4OOP06PpexPvbJgk71wpYk2RD68/7ntIVztZs7cdSIEv6sSsDxxdRgh3XdE1hJGxCnhxukQyLjEvxMXlZUDsYyiP5KOZMDj9riv/AKkLPRRn+8hmX7a+erdXKAJ118CdGn5X4ifYjTYxspd8NVOmrXxa83qYz3aQAyhN8MSFjByx2KYX73cXAJw7qbRSrqLVtA+qP/1+hIzpdpAE7t/2jSCYSvDO02NaVNbmr3ALJIotQ9I1/0pJa/uB40s6OFwFgucCu+1OSaHifWQWDL+2one8tkZApCl4h5X/C9nZFfthJAkeUzShf/5eOUN1ICui8v80Hb69OuDtfgoBlz6WkjoFCPshEOVRAgIMNN9ch4QwvMbb6daGMcAxa+j7JNJep8YFp+Ln/FJFP6M2xHDjitDJM3E9U9FxPXbsbVRSgqDoCaUO3SIjpbKltsKtZeae7sYRllBxVogA3i0rEKPe2gEkW4a6oY9vm+xXsxu+7YLVEFjFOfUs9oySyuDbylXIVkN4wmM+Z1unco15adMpp1bPAXQg+amjhv/ngSsQftLeVa+W3OPG
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(122000001)(38100700002)(26005)(6506007)(53546011)(6512007)(478600001)(5660300002)(4744005)(31686004)(186003)(2906002)(316002)(66946007)(66446008)(66476007)(66556008)(8676002)(8936002)(31696002)(38070700005)(64756008)(86362001)(36756003)(2616005)(4326008)(6486002)(76116006)(110136005)(91956017)(54906003)(71200400001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NU9ET3JCOUFSTTluUVFqN1RIdzdGdloxL05kaVFzc09MNThaVk01dVVGNHJm?=
 =?utf-8?B?ZkRxc3BIQUNBSjBIZzFNR2tXT1ZJam42N1daSldwZVNPVW5iU2pRWTJvNldJ?=
 =?utf-8?B?OWU5VWlQM0JWa0lVc0Z1WGZXUTZxSXpPRUpWYStsK0Vsb0NWUWFxSTF2MzdI?=
 =?utf-8?B?R0tpWXBlbFJkeXFWNjdEWnE1UFJVMUJLaVRFQUVHNzVETXlndEt5KzBYTm56?=
 =?utf-8?B?TnprazVCUElmTVJ3czJRcHd5bzk2bitBUnRMd1Q1ZDhkSHMrdHdFaG9scEQ1?=
 =?utf-8?B?Vm9Kelo0L0hKdWJRSHVXNTZoekRnWU9UR1A2VVhibFk0YTExQy81YVBqeU9Z?=
 =?utf-8?B?RzVqTmJKNk9FSUhYYjgyWVBidTJ5S1BOejMvcEZmU3VJZnV1b0tFNGs0U3lO?=
 =?utf-8?B?VFNrUEhmblBacURDRjVVYkwySzEzLzNwdkExT01oWENqcjEzcmVwaXhVZ2tp?=
 =?utf-8?B?aU5EVVVrYWxJcFp0ZmYvZ05PdHY5U0s1cnRjZVhZdzB3emNtL1JhYnJ1eVNQ?=
 =?utf-8?B?Z0tOWDZ0TmVtcWFUaFdFb1htTGVTY01rRnZMUGluZkZvWFc5aHNFWm1BbmdV?=
 =?utf-8?B?RE5icHV2czVod3d2aHA4c2JrOW9jOERxNFdWZy9EOFhKcDQzY1hoZHFRMlIx?=
 =?utf-8?B?aGVubUEvaXhVQS9VQVBUaU5obWNJNHlaU0tETjJodmduQVFQZVB0cm5udnJP?=
 =?utf-8?B?Yy9Ld3VHcjdsZXZCQmFobU5rYkhPV1dQWXRQOUYreXlqdkFKMjE2QmF3eWx4?=
 =?utf-8?B?bUZGVWg2QWlmQk9iVnB6bEtJVmIwN3JEbkNFd1pKbzVLQ2RYcU9TZmJBbDlT?=
 =?utf-8?B?RElEVEpWTWdqVURSMUVpWkkrMjVIUEo4ZnJ0bDd0bXpoQS81WVRhZnRST0Nu?=
 =?utf-8?B?SUl1dTFkZ2dkcVZIWmcyUlEzOVl2bHJDQTBQL0ovamZpK1BWSmdUenJKUUxY?=
 =?utf-8?B?T292TnR5ZFc4QlBSN3QycGU5azM5aXhObGdXaGtVWmFRaXZxeE5xaFBTeVhO?=
 =?utf-8?B?MEtQWXFxRS9TcHNneVFiSENmRDZmTlBhaGtkd1BZV2JpTkdnSXczZnErREQy?=
 =?utf-8?B?YTA4TFBvSkV4dzI5K2wxL1VlUk9aZng5ZkNkeW4xM2EwdWRPcmF6VVJoS1Aw?=
 =?utf-8?B?Vm55Z0x0b3ZqazB6OERVVXEvTDBXTlA0UDVQWHRCK200WEY3dXBTWTBGQ21w?=
 =?utf-8?B?N1BZSDRaMytTV1NZV2tiaFhGTDkxcG9UenBicXJXajg5aTA0NXQwWXNDSUNV?=
 =?utf-8?B?UnNqU3lra2Y5WGk5bWZYUE1hS3lkQXRhRDVCbmFpbmdIV3BXS0lrVDZyK0NE?=
 =?utf-8?B?SzVadEc3cVAvNW1PTXpPTUErRTFKSVQ2TXNVSnN5WlNLQnJOaUlYdFlEaU4y?=
 =?utf-8?B?TGc1eVMyTXNxZWlsSlVMalBJWHlydlZZdXVpL2RNZjRyVzhLTzRuanJKcFNi?=
 =?utf-8?B?RHdMNnpXZUtqTVlQU05RT29XSVVGeGRLaTdiZDhodmVtSzVORWZRS0cwT0c5?=
 =?utf-8?B?clJCWW1HNkl5cWZRa3dFcEE1MjJzSVN1N3JXL0lRWFk0ekRuakppTTRZb2hX?=
 =?utf-8?B?Q2ZGVDhsMjlNOEhNckw2VmJ0RUcwS2tYdWVTMVgrRHpubmw4L1Myb2VZTTV2?=
 =?utf-8?B?V1Nkem51cndTM3ZrSEVLYiswWDUwTWIyWTdIdGgvNEJNRktnckg5S0RNYTZn?=
 =?utf-8?B?a200UGZPaUQySW5CWUFLL3pETWRnaXkvTStTTkY0SEY1ZjdaWnhHZ0hoNUVD?=
 =?utf-8?Q?fUOHWII6csd/9eTTqlUVh0Ss18wP4m4lNcpJHhg?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F67783780276D846B62198497CE4A277@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f014c78-d658-486a-9502-08d97349851d
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 04:23:05.1088
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Hm3tR0oe8C/cInXbBJ4vO3ezK9zu9hQmNbsNz2KFYj9Ld5FXf9AgR6VFeP1QGGnHKOR/2MbhSVVrGtUqskeVNYAzRBnZxkOVzqtkIlJql6rWtrR2rdFoKhZavIPKsUuL
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4147
X-Proofpoint-ORIG-GUID: JTWAHk33C3hnkyeG3x1VWUH21qC019FK
X-Proofpoint-GUID: JTWAHk33C3hnkyeG3x1VWUH21qC019FK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_01,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 clxscore=1015
 suspectscore=0 malwarescore=0 mlxlogscore=805 phishscore=0 adultscore=0
 impostorscore=0 bulkscore=0 lowpriorityscore=0 priorityscore=1501
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090026

DQpPbiAwNi4wOS4yMSAxNzo1NywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogUmFodWwgU2lu
Z2ggPHJhaHVsLnNpbmdoQGFybS5jb20+DQo+Pg0KPj4gRml4ZXM6IDljMjQ0ZmRlZjdlNyAoInZw
Y2k6IGFkZCBoZWFkZXIgaGFuZGxlcnMiKQ0KPiBJbiB3aGljaCB3YXkgaXMgdGhhdCBvcmlnaW5h
bCBjaGFuZ2UgYnJva2VuPw0KDQpBZnRlciBjb25zdWx0aW5nIHdpdGggQXJtIHdlIGRlY2lkZWQg
dGhhdCB0aGlzIHBhdGNoIGNhbiBiZSBkcm9wcGVkLg0KDQpJZiB3ZSBmYWNlIHNvbWUgaXNzdWUg
YW5kIG5lZWQgYmUgaXQgd2lsbCBiZSBzdWJtaXR0ZWQgc2VwYXJhdGVseQ0KDQo+IFRoZSB0aXRs
ZSBkb2Vzbid0DQo+IGNsYXJpZnkgdGhpcywgYW5kIHRoZSBkZXNjcmlwdGlvbiBpcyBlbXB0eSAu
Li4NCj4NCj4gSmFuDQo+


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 04:30:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 04:30:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182589.330207 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOBhd-0005s7-3P; Thu, 09 Sep 2021 04:30:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182589.330207; Thu, 09 Sep 2021 04:30:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOBhd-0005s0-0C; Thu, 09 Sep 2021 04:30:21 +0000
Received: by outflank-mailman (input) for mailman id 182589;
 Thu, 09 Sep 2021 04:30:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOBhc-0005rq-8n; Thu, 09 Sep 2021 04:30:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOBhc-0004P3-22; Thu, 09 Sep 2021 04:30:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOBhb-0004sB-N6; Thu, 09 Sep 2021 04:30:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOBhb-0004kR-Kx; Thu, 09 Sep 2021 04:30:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TTQ0ELuW9EkEqfCodcaItx5uQKeeUa4HyCzrxLgxxaE=; b=2LtFcg/ZW4BmXXcVCKDAlSNRtH
	Yj1nDXeVKHrMPapHgPCI5wkZ3mku0I628X69RKRdgz01hA3Zy8sMYtymzycL37nzG1XJdJcqGZWY6
	6mzRd17V5XXEJsRF5xUReoNIfbkq8NbvyeRHMdUwNOC4silT9WUgK09n3M6bpFYwq6rs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164886-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.12-testing test] 164886: regressions - FAIL
X-Osstest-Failures:
    xen-4.12-testing:test-amd64-amd64-xl-qcow2:guest-saverestore.2:fail:regression
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=47193c78887734c2f95a50125684bf81419f1565
X-Osstest-Versions-That:
    xen=35ba323378d05509f2e0dc049520e140be183003
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 04:30:19 +0000

flight 164886 xen-4.12-testing real [real]
flight 164894 xen-4.12-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164886/
http://logs.test-lab.xenproject.org/osstest/logs/164894/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qcow2    18 guest-saverestore.2      fail REGR. vs. 164489

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164489
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164489
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164489
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164489
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164489
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164489
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164489
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  47193c78887734c2f95a50125684bf81419f1565
baseline version:
 xen                  35ba323378d05509f2e0dc049520e140be183003

Last test of basis   164489  2021-08-25 17:57:16 Z   14 days
Testing same since   164886  2021-09-08 13:06:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 47193c78887734c2f95a50125684bf81419f1565
Author: Jan Beulich <jbeulich@suse.com>
Date:   Wed Sep 8 15:03:11 2021 +0200

    gnttab: deal with status frame mapping race
    
    Once gnttab_map_frame() drops the grant table lock, the MFN it reports
    back to its caller is free to other manipulation. In particular
    gnttab_unpopulate_status_frames() might free it, by a racing request on
    another CPU, thus resulting in a reference to a deallocated page getting
    added to a domain's P2M.
    
    Obtain a page reference in gnttab_map_frame() to prevent freeing of the
    page until xenmem_add_to_physmap_one() has actually completed its acting
    on the page. Do so uniformly, even if only strictly required for v2
    status pages, to avoid extra conditionals (which then would all need to
    be kept in sync going forward).
    
    This is CVE-2021-28701 / XSA-384.
    
    Reported-by: Julien Grall <jgrall@amazon.com>
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>
    master commit: eb6bbf7b30da5bae87932514d54d0e3c68b23757
    master date: 2021-09-08 14:37:45 +0200
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 05:01:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 05:01:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182599.330224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOCB5-00028H-Ma; Thu, 09 Sep 2021 05:00:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182599.330224; Thu, 09 Sep 2021 05:00:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOCB5-00028A-JO; Thu, 09 Sep 2021 05:00:47 +0000
Received: by outflank-mailman (input) for mailman id 182599;
 Thu, 09 Sep 2021 05:00:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOCB4-000280-Hw; Thu, 09 Sep 2021 05:00:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOCB4-0005FS-9A; Thu, 09 Sep 2021 05:00:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOCB3-0007PB-Sl; Thu, 09 Sep 2021 05:00:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOCB3-0006wj-SG; Thu, 09 Sep 2021 05:00:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YhJWhFB6A+KyLoZIk4Ah03vHMtARLyCoqlF74MJqWvs=; b=ejWxvGhW+qSYv2db4yDyy5zPSc
	49oEZZe4GBl2nXM/ozGKDguoiQI8a1LiBqHQxjZB0Tt8dvGDoZdqKeguepvSvKFJZ1sofzIKj+jZX
	lBA7B+z67vFhiaNElWbUp7w2rB5bjozuXz+SBfGIoVhpNEQKImGwQPJNbuvDA9f8Foy4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164887-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.13-testing test] 164887: regressions - FAIL
X-Osstest-Failures:
    xen-4.13-testing:build-amd64:xen-build:fail:regression
    xen-4.13-testing:build-amd64-xsm:xen-build:fail:regression
    xen-4.13-testing:test-amd64-i386-xl-pvshim:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-raw:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-shadow:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-coresched-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-shadow:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-rtds:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-ovmf-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:build-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qcow2:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-pvshim:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-pvhv2-intel:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-pvhv2-amd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-dom0pvh-xl-amd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-dom0pvh-xl-intel:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-credit2:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-credit1:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-livepatch:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-nested-intel:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-pair:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-nested-amd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-pygrub:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-freebsd11-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-freebsd12-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-ws16-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-win7-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-coresched-i386-xl:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-freebsd10-amd64:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-freebsd10-i386:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-livepatch:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-migrupgrade:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-pair:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-qemut-rhel6hvm-amd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-qemut-rhel6hvm-intel:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-qemuu-rhel6hvm-amd:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-qemuu-rhel6hvm-intel:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-amd64-i386-xl:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-xtf-amd64-amd64-1:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-xtf-amd64-amd64-2:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-xtf-amd64-amd64-3:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-xtf-amd64-amd64-4:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-xtf-amd64-amd64-5:build-check(1):blocked:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
X-Osstest-Versions-That:
    xen=32d580902b959000d79d51dff03a3560653c4fcb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 05:00:45 +0000

flight 164887 xen-4.13-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164887/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 163761
 build-amd64-xsm               6 xen-build                fail REGR. vs. 163761

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-pvshim     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1)         blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)      blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-raw        1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-shadow     1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-xsm        1 build-check(1)               blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-shadow    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1)             blocked n/a
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1)             blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1)     blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-amd64-amd64-xl-qcow2     1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvshim    1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-livepatch    1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)              blocked n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pair         1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-pygrub       1 build-check(1)               blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)           blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)           blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)              blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)              blocked n/a
 test-amd64-coresched-i386-xl  1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)               blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-amd64-i386-livepatch     1 build-check(1)               blocked  n/a
 test-amd64-i386-migrupgrade   1 build-check(1)               blocked  n/a
 test-amd64-i386-pair          1 build-check(1)               blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)               blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1)             blocked n/a
 test-amd64-i386-xl            1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-1        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-2        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-3        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-4        1 build-check(1)               blocked  n/a
 test-xtf-amd64-amd64-5        1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 163761
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 163761
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
baseline version:
 xen                  32d580902b959000d79d51dff03a3560653c4fcb

Last test of basis   163761  2021-07-17 07:57:22 Z   53 days
Failing since        164260  2021-08-19 17:07:29 Z   20 days    9 attempts
Testing same since   164887  2021-09-08 14:08:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Dario Faggioli <dfaggioli@suse.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              fail    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  fail    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          blocked 
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       blocked 
 test-xtf-amd64-amd64-2                                       blocked 
 test-xtf-amd64-amd64-3                                       blocked 
 test-xtf-amd64-amd64-4                                       blocked 
 test-xtf-amd64-amd64-5                                       blocked 
 test-amd64-amd64-xl                                          blocked 
 test-amd64-coresched-amd64-xl                                blocked 
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           blocked 
 test-amd64-coresched-i386-xl                                 blocked 
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        blocked 
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         blocked 
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 blocked 
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-xl-xsm                                      blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       blocked 
 test-amd64-amd64-qemuu-nested-amd                            blocked 
 test-amd64-amd64-xl-pvhv2-amd                                blocked 
 test-amd64-i386-qemut-rhel6hvm-amd                           blocked 
 test-amd64-i386-qemuu-rhel6hvm-amd                           blocked 
 test-amd64-amd64-dom0pvh-xl-amd                              blocked 
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemut-debianhvm-amd64                     blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     blocked 
 test-amd64-i386-freebsd10-amd64                              blocked 
 test-amd64-amd64-qemuu-freebsd11-amd64                       blocked 
 test-amd64-amd64-qemuu-freebsd12-amd64                       blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64                          blocked 
 test-amd64-amd64-xl-qemut-win7-amd64                         blocked 
 test-amd64-i386-xl-qemut-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64                         blocked 
 test-amd64-i386-xl-qemuu-win7-amd64                          blocked 
 test-amd64-amd64-xl-qemut-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemut-ws16-amd64                          blocked 
 test-amd64-amd64-xl-qemuu-ws16-amd64                         blocked 
 test-amd64-i386-xl-qemuu-ws16-amd64                          blocked 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  blocked 
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  blocked 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        blocked 
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         blocked 
 test-amd64-i386-freebsd10-i386                               blocked 
 test-amd64-amd64-qemuu-nested-intel                          blocked 
 test-amd64-amd64-xl-pvhv2-intel                              blocked 
 test-amd64-i386-qemut-rhel6hvm-intel                         blocked 
 test-amd64-i386-qemuu-rhel6hvm-intel                         blocked 
 test-amd64-amd64-dom0pvh-xl-intel                            blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-livepatch                                   blocked 
 test-amd64-i386-livepatch                                    blocked 
 test-amd64-amd64-migrupgrade                                 blocked 
 test-amd64-i386-migrupgrade                                  blocked 
 test-amd64-amd64-xl-multivcpu                                blocked 
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        blocked 
 test-amd64-i386-pair                                         blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-amd64-amd64-xl-pvshim                                   blocked 
 test-amd64-i386-xl-pvshim                                    blocked 
 test-amd64-amd64-pygrub                                      blocked 
 test-amd64-amd64-xl-qcow2                                    blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       blocked 
 test-amd64-amd64-xl-rtds                                     blocked 
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              blocked 
 test-amd64-amd64-xl-shadow                                   blocked 
 test-amd64-i386-xl-shadow                                    blocked 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 796 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 05:22:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 05:22:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182607.330241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOCW6-0005YK-PY; Thu, 09 Sep 2021 05:22:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182607.330241; Thu, 09 Sep 2021 05:22:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOCW6-0005YD-Lw; Thu, 09 Sep 2021 05:22:30 +0000
Received: by outflank-mailman (input) for mailman id 182607;
 Thu, 09 Sep 2021 05:22:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOCW5-0005Y7-LW
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 05:22:29 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ebe9952d-112d-11ec-b19e-12813bfff9fa;
 Thu, 09 Sep 2021 05:22:28 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1894nBsA001003; 
 Thu, 9 Sep 2021 05:22:25 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3aybjtr3u6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 05:22:24 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6323.eurprd03.prod.outlook.com (2603:10a6:20b:159::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 05:22:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 05:22:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebe9952d-112d-11ec-b19e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G3b0cn+IQ58ZtPw9WRhT9ASaEC4BGzJbhCWHERtHLjH6oPDFaoRg3vNsD5leiFcHVjBTA3hBfVSqK2T8uDja04YSXqsyNrakAuYOa6w5mY8szEs4aAweFLhJVej7Bo1kFXqnpnz79PdL4DQBWuTbkDwRLbNlRz3QzkNg6cTXYbXnt16uypgVYzZGkK7YDx+nS7Wnx9dFVBb6eFFssECs/W0TBZkJzv9h+xgJTOTICN6hLS8lGXkF3FigKk+yB20x0P70gnOoRlfG9J0bpResa2g+QK0Tv3tJb+dvMSHRgTrPmAOV7yKlI+ZUfjF8DUECl5i/vyETFON7vu7i8CGm4Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+CwpRiGSJLLP1t+op9L5h6wdjH0oQcj9ympwW45/GcM=;
 b=Fu/zpACr55ERa+RnrV9oq206mNQHrRv1IXYgaIW861ni5QKFUq10UeOkGoP1m2pnPMfmC2TyqOlKtSZiICSa3iFLri+/QU8/MFn4R86SdbXoh8CiC5FRhsIJBtPCgwcMVDDtO608A7y8Ing2y/JYYBAA4kn/HaQsHepknh14Q2zd67QrVpwTc+kE3LhUy9aiNcT8rzN8DMdXrSWV9S9jB5gis+E6ZE4CmSdxOkaeS5gkTwWglhpSQBMt+7Bj+tCW5IPGIYQt2A/pGIsmHUIwBIq+Lst6FM/GpfX5c3SHFdsbfubYv06BR/P7L3DAJFWVCEkOfe8R2w0yA1PvmpVWZg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+CwpRiGSJLLP1t+op9L5h6wdjH0oQcj9ympwW45/GcM=;
 b=bbVXaonU0AZMeYXY2BVftx+O9W2SoL4DHL4ChUFLGyl8nSzPcNTg64yCxy87QSTg8Tw/PnxPNZWhK1hVzncF00NDFHjKyP0jUCU1PK1vVMu563xT2mx4ChqInOGYrsVqnYjkQkyMPmLRu9WwcjMyPOocwR36iW9Dix9HBdR0Q6zzZVsWRbV+8ywpAccLr4PPaQQQJlfGFlNMqM7vu3qG6pZW47CO4ya9a1nO4OZQqO1s2XZjkDcMptRAkrtrcy3hjXGUTRdG7FbBRDBTFHDp/93ragrbmnOS2Xj8I6bNDzf6CQmtCeIxYJ8Ng6+Krhe8xZmmV98t+Rj7guVyjh2BHg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Topic: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Index: AQHXoKxjXfke7B20QEeo6YjlfUMhlquXGvYAgAMgDgCAAAg4gIAA8L+A
Date: Thu, 9 Sep 2021 05:22:21 +0000
Message-ID: <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
In-Reply-To: <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9c14ad79-4399-4da2-41c7-08d97351cd25
x-ms-traffictypediagnostic: AM0PR03MB6323:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB6323BADB023AE517F7747D60E7D59@AM0PR03MB6323.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 BM8U3GqiPCxeDoYG7dphId1HpHcM0ThNJT8YsQE21gY2agboMVWKbyASP6CfgeY8BFBQ3tePXH7C6TB7CTD5TkxAKLbccTh6adpzMblSEyGXmRleZWXe0e/FteRJXim7whc0kqcF7SfgGN1601yf9CJ8aEMA5xcuNmwbPd4veObc04V87j/Y07R8MqmwCjShHq9dY29lwEQEEIRTf5Ooi9fcZBnZmVn6V09SKfVknOp1KRxZNTS/xTRYsvvPaUUVHZcpSyR6a52TvQC6Gg0kMjmE6I1pRFPbpHpkLHxl9L6TtK6r9wRA/HuhxPbxysY3EpD5gA/xJeo5OQVHgjNjT4iufES9hgr2bdWNCMvUOKi9+wT+OdTi7m1T0iW8p6k3GtKBnX7e9Of1cdafrRCUc1Bwfz4jLinV5iN+rn8BYbZckhOogf4LtFOvTzZYQgNPtCYHA2C7dTRR8UAQjucAMNtBSRRUBBlMKNtvQ3WKbLiVf27EgttC+q7NUnB0gjytgrhZBttFy2ykaafnth3+IkHIc/ows0g4IyLT9zwwh2vUfm3POv4n8ZlvpJxeipdL4Jdn8fcMWkPoEbezhHRTQ4eu9uVCoIhg/L9BKh840fIpaVoiDwfLKukDqmdap9mBHyTbkawkDyD5e1chkIN19OmdwV3Sinas+RkkXaXRozcX7GgvXG5KbvuKwyE+sJ+dNDEEGSUfHYKoTP//f5uugDmZqF+u3jm4xfnsTt7BbxidcvNcApuLbJ73IHGOCt/Z
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(346002)(366004)(376002)(136003)(39860400002)(38100700002)(6512007)(4326008)(316002)(110136005)(54906003)(31696002)(83380400001)(26005)(122000001)(38070700005)(186003)(66446008)(76116006)(64756008)(86362001)(53546011)(66476007)(66556008)(6506007)(91956017)(2906002)(36756003)(2616005)(66946007)(31686004)(8936002)(6486002)(71200400001)(5660300002)(8676002)(478600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?YTE4YzhxNW03d3lKN3g5UUtaTTUrTUFUcDBrUm0rc2V1UFl5eERPQVdwVzd5?=
 =?utf-8?B?bXVIRm8vK1I1c295UUdHTjNxdDI0Yll4L1huMlpPQ0FpUUtOUC95N3ZYdzRp?=
 =?utf-8?B?SzVjNWtRV1hhajRGdkIwZXpXckZEczFvOGh5Y1VWc2UyalpMVmtodlJIem1r?=
 =?utf-8?B?V0ZFWkNSc3BVOTkvNkFKaW9qZ3lTODdJYzVXazVlNUNpanFVK3ZRdmV5cG50?=
 =?utf-8?B?amxyUG8wZEpTUk1OSUNMVERJd0hIeTJOd2lGakJIK2IzNGhoRTR2dElPcysw?=
 =?utf-8?B?ZHVpM2FwbFdCZDRja0o3cXV4NGtPd2dIQVZ4UDZSMjFPbzdXaG9BbFZkYUI5?=
 =?utf-8?B?S3lSR29mZk14U005dXkrOURuUzlvMXFJTkxiVlRhbTJ3Q2cvYW9KWUVjcm8x?=
 =?utf-8?B?U3AvZDZWeVEwMHJaUjBoazRYWlpOOFl0RlV3QTNjQjZTM2RnTTV6bDRjMXJI?=
 =?utf-8?B?WjlPdUNCQkhtSnRvTEN4dy9KT0JhSVR1U25LWFVYWkowY1ZjY3h0dEx5M2Yw?=
 =?utf-8?B?dW5oM0J2ZUtNSFVhZU94LzJzZXFjc1RGYUJjbS9IQnEzVEduWXprSzNqNitM?=
 =?utf-8?B?bWFIMGdSVTZIQ0Ftd09FNVdtZ1dDSnBPTERPaXIzWGhOOURublVMT3FOd1Qy?=
 =?utf-8?B?NnhQUzZ1YXNiNHVnelkybFVKWUt6NHc0OGxGTjhaTURwNmt6MGJFbExIb1Z3?=
 =?utf-8?B?MFlMajJOKy9QYzVsdEx2QndOQ0NtUVNCQXYwTXJVd0dPSVR5Q1FpUDlEN2p2?=
 =?utf-8?B?R01uR1NOMVdhd3VYbHR4RGNaK2ZjT0VVb3MySUk2YTllNlFheElIcVQrdUpt?=
 =?utf-8?B?ZWd0aFFVTUhrWi9jdzN0enZLS3RTbE5rRE1qeXBOa0gxUlM4SmozTFVyekVl?=
 =?utf-8?B?TUhRZHBGWU04UHBjV0FnNGZYcFpHZEY5NGtvN2w5TmNKK1VndDBCSzNpWVJu?=
 =?utf-8?B?Uk0rQzVhVUlabEIwa24xeXpVclpCUUtmMEwremduVllIRlRua1hFS1FMbk1a?=
 =?utf-8?B?NmhYcStZbzFjM2xaREdQNWc4YklsNFdGcUdkVG5pY2Z5V0RjOUxoeWE3c0tv?=
 =?utf-8?B?NW1QTUtlWVVPN0RFZisya2k3cTNCeDhXUnh2NmpWOTlOampCWERycHZCVnRJ?=
 =?utf-8?B?NExXYzhNbjFCdDlnR0txeUZTK0hTS3VIVVRjZjBjR2NBRmxVTU5VSm1peDRi?=
 =?utf-8?B?RUQzQ3BLMFl1SjNuVDVjb3pFa1NuWjFPS2JzMDNNL1dYbE1wY1ZMMVBKVjNV?=
 =?utf-8?B?ckFDcUkwU3gyRzdZaENVQldXay82SnZ5MnFOSEM2SjlXcTVDcXloNitiTXBt?=
 =?utf-8?B?Rkl3NC8wNHRIdmFzbUI4aStkZDZTd05ranFVcnNhQW5wQ3h2ejZWaEVQd0ZB?=
 =?utf-8?B?SkFiWktqWGlsTWRhVXZRYlZRcFllaVhTbC81cllZV3VJWG1pZjdYWFZKTzBX?=
 =?utf-8?B?aTJkcHBPcnBDQU1TTlRyMFp4ZmxlcHlDYlVIRGltNzZGQVI2Q21XR3V6RERT?=
 =?utf-8?B?YllPT1VGWE50QlhxYW5RaWVlcndSVVdRS1d1OUtSTlNyQ1JOY0xFZlNDbzJJ?=
 =?utf-8?B?TUhhZFNFUEd3cHBEQ20yU1Vjb1BrL2w2MGVRTEdjNG5lS1RwOUtnOGZIUzdE?=
 =?utf-8?B?TUNjR2VieFJGOHFtVWtJNk9qOHFwazFYaWVrSHBEaXdwSWs4c3ZvbzNtbzJP?=
 =?utf-8?B?clR0UGttcEQva1NrSHRJemUzT1V1c0prcmlQd3pNTTBQQVRCSit5NGJuVXZJ?=
 =?utf-8?Q?YLIsgjEH/rhU4wvl3InRY8P6T6i7ghanxc8wwHp?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <FFB3DF1AD81BA844BE0528E9BCE5566D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c14ad79-4399-4da2-41c7-08d97351cd25
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 05:22:21.9896
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yKOTKrDmpiaYpVL42LjV1gmPNMF6FpS9a4W+ONIOmUuPxE2kRMgkw13QBggwQWYPTZLal5RqvbNEHdaNREivUWnGh8tNHWriLxMoqx03We3pjnMv5DaD0+HiviVFIJlB
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6323
X-Proofpoint-ORIG-GUID: YsPPrfSn1cU-W4GmPEL2V-Dt_sOLGQWs
X-Proofpoint-GUID: YsPPrfSn1cU-W4GmPEL2V-Dt_sOLGQWs
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_01,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 impostorscore=0 clxscore=1015 priorityscore=1501 phishscore=0 spamscore=0
 mlxscore=0 bulkscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090032

DQpPbiAwOC4wOS4yMSAxODowMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA4LjA5LjIwMjEg
MTY6MzEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDYuMDkuMjEgMTc6
NDcsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+DQo+Pj4+IEluc3RlYWQg
b2YgaGFuZGxpbmcgYSBzaW5nbGUgcmFuZ2Ugc2V0LCB0aGF0IGNvbnRhaW5zIGFsbCB0aGUgbWVt
b3J5DQo+Pj4+IHJlZ2lvbnMgb2YgYWxsIHRoZSBCQVJzIGFuZCBST00sIGhhdmUgdGhlbSBwZXIg
QkFSLg0KPj4+IFdpdGhvdXQgbG9va2luZyBhdCBob3cgeW91IGNhcnJ5IG91dCB0aGlzIGNoYW5n
ZSAtIHRoaXMgbG9vayB3cm9uZyAoYXMNCj4+PiBpbjogd2FzdGVmdWwpIHRvIG1lLiBEZXNwaXRl
IC4uLg0KPj4+DQo+Pj4+IFRoaXMgaXMgaW4gcHJlcGFyYXRpb24gb2YgbWFraW5nIG5vbi1pZGVu
dGl0eSBtYXBwaW5ncyBpbiBwMm0gZm9yIHRoZQ0KPj4+PiBNTUlPcy9ST00uDQo+Pj4gLi4uIHRo
ZSBuZWVkIGZvciB0aGlzLCBldmVyeSBpbmRpdmlkdWFsIEJBUiBpcyBzdGlsbCBjb250aWd1b3Vz
IGluIGJvdGgNCj4+PiBob3N0IGFuZCBndWVzdCBhZGRyZXNzIHNwYWNlcywgc28gY2FuIGJlIHJl
cHJlc2VudGVkIGFzIGEgc2luZ2xlDQo+Pj4gKHN0YXJ0LGVuZCkgdHVwbGUgKG9yIGEgcGFpciB0
aGVyZW9mLCB0byBhY2NvdW50IGZvciBib3RoIGhvc3QgYW5kIGd1ZXN0DQo+Pj4gdmFsdWVzKS4g
Tm8gbmVlZCB0byB1c2UgYSByYW5nZXNldCBmb3IgdGhpcy4NCj4+IEZpcnN0IG9mIGFsbCB0aGlz
IGNoYW5nZSBpcyBpbiBwcmVwYXJhdGlvbiBmb3Igbm9uLWlkZW50aXR5IG1hcHBpbmdzLA0KPiBJ
J20gYWZyYWlkIEkgY29udGludWUgdG8gbm90IHNlZSBob3cgdGhpcyBtYXR0ZXJzIGluIHRoZSBk
aXNjdXNzaW9uIGF0DQo+IGhhbmQuIEknbSBmdWxseSBhd2FyZSB0aGF0IHRoaXMgaXMgdGhlIGdv
YWwuDQo+DQo+PiBlLmcuIGN1cnJlbnRseSB3ZSBjb2xsZWN0IGFsbCB0aGUgbWVtb3J5IHJhbmdl
cyB3aGljaCByZXF1aXJlIG1hcHBpbmdzDQo+PiBpbnRvIGEgc2luZ2xlIHJhbmdlIHNldCwgdGhl
biB3ZSBjdXQgb2ZmIE1TSS1YIHJlZ2lvbnMgYW5kIHRoZW4gdXNlIHJhbmdlIHNldA0KPj4gZnVu
Y3Rpb25hbGl0eSB0byBjYWxsIGEgY2FsbGJhY2sgZm9yIGV2ZXJ5IG1lbW9yeSByYW5nZSBsZWZ0
IGFmdGVyIE1TSS1YLg0KPj4gVGhpcyB3b3JrcyBwZXJmZWN0bHkgZmluZSBmb3IgMToxIG1hcHBp
bmdzLCBlLmcuIHdoYXQgd2UgaGF2ZSBhcyB0aGUgcmFuZ2UNCj4+IHNldCdzIHN0YXJ0aW5nIGFk
ZHJlc3MgaXMgd2hhdCB3ZSB3YW50IHRvIGJlIG1hcHBlZC91bm1hcHBlZC4NCj4+IFdoeSByYW5n
ZSBzZXRzPyBCZWNhdXNlIHRoZXkgYWxsb3cgcGFydGlhbCBtYXBwaW5ncywgZS5nLiB5b3UgY2Fu
IG1hcCBwYXJ0IG9mDQo+PiB0aGUgcmFuZ2UgYW5kIHJldHVybiBiYWNrIGFuZCBjb250aW51ZSBm
cm9tIHdoZXJlIHlvdSBzdG9wcGVkLiBBbmQgaWYgSQ0KPj4gdW5kZXJzdGFuZCB0aGF0IGNvcnJl
Y3RseSB0aGF0IHdhcyB0aGUgaW5pdGlhbCBpbnRlbnRpb24gb2YgaW50cm9kdWNpbmcgcmFuZ2Ug
c2V0cyBoZXJlLg0KPj4NCj4+IEZvciBub24taWRlbnRpdHkgbWFwcGluZ3MgdGhpcyBiZWNvbWVz
IG5vdCB0aGF0IGVhc3kuIEVhY2ggaW5kaXZpZHVhbCBCQVIgbWF5IGJlDQo+PiBtYXBwZWQgZGlm
ZmVyZW50bHkgYWNjb3JkaW5nIHRvIHdoYXQgZ3Vlc3QgT1MgaGFzIHByb2dyYW1tZWQgYXMgYmFy
LT5ndWVzdF9hZGRyDQo+PiAoZ3Vlc3QgdmlldyBvZiB0aGUgQkFSIHN0YXJ0KS4NCj4gSSBkb24n
dCBzZWUgaG93IHRoZSByYW5nZXNldCBoZWxwcyBoZXJlLiBZb3UgaGF2ZSBhIGd1ZXN0IGFuZCBh
IGhvc3QgcGFpcg0KPiBvZiB2YWx1ZXMgZm9yIGV2ZXJ5IEJBUi4gUGFnZXMgd2l0aCBlLmcuIHRo
ZSBNU0ktWCB0YWJsZSBtYXkgbm90IGJlIG1hcHBlZA0KPiB0byB0aGVpciBob3N0IGNvdW50ZXJw
YXJ0IGFkZHJlc3MsIHllcywgYnV0IHlvdSBuZWVkIHRvIHNwZWNpYWwgY2FzZXMNCj4gdGhlc2Ug
YW55d2F5OiBBY2Nlc3NlcyB0byB0aGVtIG5lZWQgdG8gYmUgaGFuZGxlZC4gSGVuY2UgSSdtIGhh
dmluZyBhIGhhcmQNCj4gdGltZSBzZWVpbmcgaG93IGEgcGVyLUJBUiByYW5nZXNldCAod2hpY2gg
d2lsbCBjb3ZlciBhdCBtb3N0IHRocmVlIGRpc3RpbmN0DQo+IHJhbmdlcyBhZmFpY3QsIHdoaWNo
IGlzIHdheSB0b28gbGl0dGxlIGZvciB0aGlzIGtpbmQgb2YgZGF0YSBvcmdhbml6YXRpb24NCj4g
aW1vKSBjYW4gZ2FpbiB5b3UgYWxsIHRoaXMgbXVjaC4NCj4NCj4gT3ZlcmFsbCB0aGUgNiBCQVJz
IG9mIGEgZGV2aWNlIHdpbGwgY292ZXIgdXAgdG8gOCBub24tYWRqYWNlbnQgcmFuZ2VzLiBJT1cN
Cj4gdGhlIG1ham9yaXR5ICg0IG9yIG1vcmUpIG9mIHRoZSByYW5nZXNldHMgd2lsbCBpbmRlZWQg
bWVyZWx5IHJlcHJlc2VudCBhDQo+IHBsYWluIChzdGFydCxlbmQpIHBhaXIgKG9yIGJlIGVudGly
ZWx5IGVtcHR5KS4NCkZpcnN0IG9mIGFsbCwgbGV0IG1lIGV4cGxhaW4gd2h5IEkgZGVjaWRlZCB0
byBtb3ZlIHRvIHBlci1CQVINCnJhbmdlIHNldHMuDQpCZWZvcmUgdGhpcyBjaGFuZ2UgYWxsIHRo
ZSBNTUlPIHJlZ2lvbnMgYW5kIE1TSS1YIGhvbGVzIHdlcmUNCmFjY291bnRlZCBieSBhIHNpbmds
ZSByYW5nZSBzZXQsIGUuZy4gd2UgZ28gb3ZlciBhbGwgQkFScyBhbmQNCmFkZCBNTUlPcyBhbmQg
dGhlbiBzdWJ0cmFjdCBNU0ktWCBmcm9tIHRoZXJlLiBXaGVuIGl0IGNvbWVzIHRvDQptYXBwaW5n
L3VubWFwcGluZyB3ZSBoYXZlIGFuIGFzc3VtdGlvbiB0aGF0IHRoZSBzdGFydGluZyBhZGRyZXNz
IG9mDQplYWNoIGVsZW1lbnQgaW4gdGhlIHJhbmdlIHNldCBpcyBlcXVhbCB0byBtYXAvdW5tYXAg
YWRkcmVzcywgZS5nLg0Kd2UgaGF2ZSBpZGVudGl0eSBtYXBwaW5nLiBQbGVhc2Ugbm90ZSwgdGhh
dCB0aGUgcmFuZ2Ugc2V0IGFjY2VwdHMNCmEgc2luZ2xlIHByaXZhdGUgZGF0YSBwYXJhbWV0ZXIg
d2hpY2ggaXMgZW5vdWdoIHRvIGhvbGQgYWxsDQpyZXF1aXJlZCBkYXRhIGFib3V0IHRoZSBwZGV2
IGluIGNvbW1vbiwgYnV0IHRoZXJlIGlzIG5vIHdheSB0byBwcm92aWRlDQphbnkgcGVyLUJBUiBk
YXRhLg0KDQpOb3csIHRoYXQgd2Ugd2FudCBub24taWRlbnRpdHkgbWFwcGluZ3MsIHdlIGNhbiBu
byBsb25nZXIgYXNzdW1lDQp0aGF0IHN0YXJ0aW5nIGFkZHJlc3MgPT0gbWFwcGluZyBhZGRyZXNz
IGFuZCB3ZSBuZWVkIHRvIHByb3ZpZGUNCmFkZGl0aW9uYWwgaW5mb3JtYXRpb24gb24gaG93IHRv
IG1hcCBhbmQgd2hpY2ggaXMgbm93IHBlci1CQVIuDQpUaGlzIGlzIHdoeSBJIGRlY2lkZWQgdG8g
dXNlIHBlci1CQVIgcmFuZ2Ugc2V0cy4NCg0KT25lIG9mIHRoZSBzb2x1dGlvbnMgbWF5IGJlIHRo
YXQgd2UgZm9ybSBhbiBhZGRpdGlvbmFsIGxpc3Qgb2YNCnN0cnVjdHVyZXMgaW4gYSBmb3JtIChJ
IG9tbWl0IHNvbWUgb2YgdGhlIGZpZWxkcyk6DQpzdHJ1Y3Qgbm9uX2lkZW50aXR5IHsNCiDCoMKg
wqAgdW5zaWduZWQgbG9uZyBzdGFydF9tZm47DQogwqDCoMKgIHVuc2lnbmVkIGxvbmcgc3RhcnRf
Z2ZuOw0KIMKgwqDCoCB1bnNpZ25lZCBsb25nIHNpemU7DQp9Ow0KU28gdGhpcyB3YXkgd2hlbiB0
aGUgcmFuZ2Ugc2V0IGdldHMgcHJvY2Vzc2VkIHdlIGdvIG92ZXIgdGhlIGxpc3QNCmFuZCBmaW5k
IG91dCB0aGUgY29ycmVzcG9uZGluZyBsaXN0J3MgZWxlbWVudCB3aGljaCBkZXNjcmliZXMgdGhl
DQpyYW5nZSBzZXQgZW50cnkgYmVpbmcgcHJvY2Vzc2VkIChzLCBlLCBkYXRhKToNCg0Kc3RhdGlj
IGludCBtYXBfcmFuZ2UodW5zaWduZWQgbG9uZyBzLCB1bnNpZ25lZCBsb25nIGUsIHZvaWQgKmRh
dGEsDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1bnNpZ25lZCBs
b25nICpjKQ0Kew0KW3NuaXBdDQogwqDCoMKgIGdvIG92ZXIgdGhlIGxpc3QgZWxlbWVudHMNCiDC
oMKgwqAgwqDCoMKgIGlmICggbGlzdC0+c3RhcnRfbWZuID09IHMgKQ0KIMKgwqDCoCDCoMKgwqAg
wqDCoMKgIGZvdW5kLCBjYW4gdXNlIGxpc3QtPnN0YXJ0X2dmbiBmb3IgbWFwcGluZw0KW3NuaXBd
DQp9DQpUaGlzIGhhcyBzb21lIGNvbXBsaWNhdGlvbnMgYXMgbWFwX3JhbmdlIG1heSBiZSBjYWxs
ZWQgbXVsdGlwbGUgdGltZXMNCmZvciB0aGUgc2FtZSByYW5nZTogaWYge3VubWFwfG1hcH1fbW1p
b19yZWdpb25zIHdhcyBub3QgYWJsZSB0byBjb21wbGV0ZQ0KdGhlIG9wZXJhdGlvbiBpdCByZXR1
cm5zIHRoZSBudW1iZXIgb2YgcGFnZXMgaXQgd2FzIGFibGUgdG8gcHJvY2VzczoNCiDCoMKgwqDC
oMKgwqDCoCByYyA9IG1hcC0+bWFwID8gbWFwX21taW9fcmVnaW9ucyhtYXAtPmQsIHN0YXJ0X2dm
biwNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzaXplLCBfbWZuKHMpKQ0KIMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA6IHVubWFwX21taW9fcmVnaW9ucyhtYXAt
PmQsIHN0YXJ0X2dmbiwNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2l6ZSwgX21mbihz
KSk7DQpJbiB0aGlzIGNhc2Ugd2UgbmVlZCB0byB1cGRhdGUgdGhlIGxpc3QgaXRlbToNCiDCoMKg
wqAgbGlzdC0+c3RhcnRfbWZuICs9IHJjOw0KIMKgwqDCoCBsaXN0LT5zdGFydF9nZm4gKz0gcmM7
DQogwqDCoMKgIGxpc3QtPnNpemUgLT0gcmM7DQphbmQgaWYgYWxsIHRoZSBwYWdlcyBvZiB0aGUg
cmFuZ2Ugd2VyZSBwcm9jZXNzZWQgZGVsZXRlIHRoZSBsaXN0IGVudHJ5Lg0KDQpXaXRoIHJlc3Bl
Y3Qgb2YgY3JlYXRpbmcgdGhlIGxpc3QgZXZlcnl0aGluZyBhbHNvIG5vdCBzbyBjb21wbGljYXRl
ZDoNCndoaWxlIHByb2Nlc3NpbmcgZWFjaCBCQVIgY3JlYXRlIGEgbGlzdCBlbnRyeSBhbmQgZmls
bCBpdCB3aXRoIG1mbiwgZ2ZuDQphbmQgc2l6ZS4gVGhlbiwgaWYgTVNJLVggcmVnaW9uIGlzIHBy
ZXNlbnQgd2l0aGluIHRoaXMgQkFSLCBicmVhayB0aGUNCmxpc3QgaXRlbSBpbnRvIG11bHRpcGxl
IG9uZXMgd2l0aCByZXNwZWN0IHRvIHRoZSBob2xlcywgZm9yIGV4YW1wbGU6DQoNCk1NSU8gMCBs
aXN0IGl0ZW0NCk1TSS1YIGhvbGUgMA0KTU1JTyAxIGxpc3QgaXRlbQ0KTVNJLVggaG9sZSAxDQoN
CkhlcmUgaW5zdGVhZCBvZiBhIHNpbmdsZSBCQVIgZGVzY3JpcHRpb24gd2Ugbm93IGhhdmUgMiBs
aXN0IGVsZW1lbnRzDQpkZXNjcmliaW5nIHRoZSBCQVIgd2l0aG91dCBNU0ktWCByZWdpb25zLg0K
DQpBbGwgdGhlIGFib3ZlIHN0aWxsIHJlbGllcyBvbiBhIHNpbmdsZSByYW5nZSBzZXQgcGVyIHBk
ZXYgYXMgaXQgaXMgaW4gdGhlDQpvcmlnaW5hbCBjb2RlLiBXZSBjYW4gZ28gdGhpcyByb3V0ZSBp
ZiB3ZSBhZ3JlZSB0aGlzIGlzIG1vcmUgYWNjZXB0YWJsZQ0KdGhhbiB0aGUgcmFuZ2Ugc2V0cyBw
ZXIgQkFSDQo+DQo+PiBUaHVzIHdlIG5lZWQgdG8gY29sbGVjdCBhbGwgdGhvc2Ugbm9uLWlkZW50
aXR5IG1hcHBpbmdzDQo+PiBwZXIgQkFSIG5vdyAoc28gd2UgaGF2ZSBhIG1hcHBpbmcgImd1ZXN0
IHZpZXciIDogInBoeXNpY2FsIEJBUiIgYW5kIGFnYWluIGN1dCBvZmYNCj4+IE1TSS1YIHJlZ2lv
bnMgYXMgYmVmb3JlLsKgIFNvLCB5ZXMsIGl0IG1heSBiZSBhIGJpdCB3YXN0ZWZ1bCB0byB1c2Ug
bWFueSByYW5nZSBzZXRzLA0KPj4gYnV0IG1ha2VzIHZQQ0kgbGlmZSBtdWNoLW11Y2ggZWFzaWVy
Lg0KPiBXaGljaCBJJ20geWV0IHRvIGJlIGNvbnZpbmNlZCBvZi4gVGhlbiBhZ2FpbiBJJ20gbm90
IHRoZSBtYWludGFpbmVyIG9mDQo+IHRoaXMgY29kZSwgc28gaWYgeW91IGNhbiBjb252aW5jZSBS
b2dlciB5b3UnbGwgYmUgYWxsIGdvb2QuDQoNClBlci1CQVIgcmFuZ2Ugc2V0cyBsb29rIG1vcmUg
Y2xlYXIgdG8gbWUgYW5kIGFkZCByZWxhdGl2ZWx5IGxlc3MgY29kZSB3aGljaA0KDQpzZWVtcyB0
byBiZSBnb29kLg0KDQo+DQo+PiBUaHVzLCBJIHRoaW5rIHRoYXQgZXZlbiBwZXItQkFSIHJhbmdl
IHNldHMgYXJlDQo+PiBnb29kIHRvIGdvIGFzIHRoZXkgaGF2ZSBtb3JlIHByb3MgdGhhbiBjb25z
LiBJTU8NCj4+IEV2ZW4gaWYgd2UgZ28gd2l0aCAiY2FuIGJlIHJlcHJlc2VudGVkIGFzIGEgc2lu
Z2xlIChzdGFydCxlbmQpIHR1cGxlIiBpdCBkb2Vzbid0IGFuc3dlcg0KPj4gdGhlIHF1ZXN0aW9u
IHdoYXQgbmVlZHMgdG8gYmUgZG9uZSBpZiBhIHJhbmdlIGdldHMgcGFydGlhbGx5IG1hcHBlZC91
bm1hcHBlZC4NCj4gVGhpcyBxdWVzdGlvbiBhbHNvIGlzbid0IGFuc3dlcmVkIHdoZW4geW91IHVz
ZSByYW5nZXNldHMuDQpib29sIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQ0K
ew0KDQpbc25pcF0NCg0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmMgPSByYW5nZXNldF9jb25z
dW1lX3JhbmdlcyhiYXItPm1lbSwgbWFwX3JhbmdlLCAmZGF0YSk7DQoNCiDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIGlmICggcmMgPT0gLUVSRVNUQVJUICkNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgcmV0dXJuIHRydWU7DQoNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtz
YW5kcg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 06:13:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 06:13:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182615.330257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mODJE-0003o3-Ep; Thu, 09 Sep 2021 06:13:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182615.330257; Thu, 09 Sep 2021 06:13:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mODJE-0003nw-Bm; Thu, 09 Sep 2021 06:13:16 +0000
Received: by outflank-mailman (input) for mailman id 182615;
 Thu, 09 Sep 2021 06:13:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mODJC-0003nq-Gf
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 06:13:14 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c88fba47-1f0d-4c4a-936c-3fdd3aacba72;
 Thu, 09 Sep 2021 06:13:13 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1895FJ8f031477; 
 Thu, 9 Sep 2021 06:13:09 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 by mx0a-0039f301.pphosted.com with ESMTP id 3aybxm860b-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 06:13:09 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3217.eurprd03.prod.outlook.com (2603:10a6:208:2::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 06:13:06 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 06:13:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c88fba47-1f0d-4c4a-936c-3fdd3aacba72
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XWDJRi7F6o3sao9f/vU/EO3Gsp57Z6dTKNYBpcmPgjfAUac1sBCJqVXlH+cTGthrK9aI2wVJj/IJmx2AH7m+FNswVV0aUPHyA7X6F9LRTNTWHPiXuggsIgySOZY/ARBTW5dn8AoPCI+NH4eHoi/Xd4DaS2o9i6oMQQo5loc4f3BYXVlI+qBl/WdCWSPfePIR+yMELNeFEZ4jFXYewPSx17Dp9CQCnupCwyF7dQy6xsqL7INGv0Audu5IPcky8l4nR/JCPt/q89QQz/a5vYQ1c/r2O6tsAzQ8WHJF3stmK5l3hxCUWDfvXt6Fs/p+/xwLZ1Ar6iU/Z0wRaJnlb757nA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LC4+rTwvOahb/P0W0i7gTem2KvP4a2j9bmnJIM+0E68=;
 b=Ms5y0cDgFnHp6VKr2pm6KxmD5IrnkhQkiF1VEeO5x6J6dO1MFk8tgsGAmg5kE2Q9V4SQ2u/xnCa6wdDmNLbfJPLQAaCy9YuMR50UVu444AdY8NoWuzJVR7Urlxd6ex0dC+1YEuLVnJCHxFUCzwmypNhJ0IdX08v2ym1zL5u6uYlt3esota1hDZvqg/zHD1Bx2PRIriynSX1oKBCsiciDV7cUhJk3fUkyDEA2W76BEoAicOR1cTGhxHozB3mUdyTQUGLtanjgXU6k1RnvDHKugXkhfeoIzg0v6JQq+dxJe+FuyeHI4U3Vyx2MFXCtSyWGubs2sEDwxil5Io8M1Ar+jQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LC4+rTwvOahb/P0W0i7gTem2KvP4a2j9bmnJIM+0E68=;
 b=NeqAyeQPxnm3Pcx2JwxrqWS2+cXHAdO8tQBBU+/CQ8XG1fABvuktgYxcia9PNl+mgoXUAU2HgEiRrAXO4ESEkkUJLcCI9N7bDGsE0UqFl9QtGVdMW+q789QCsNEwF15GlB14V7cBrNR0yNp5r9d7iQ3KSjD00hKN42OHeLQEVah1ChtzfBPQtzkS1WXSpF8ahDY59ozW2UfTQU/Ut3MuMCZoPvPuMKw690NqcIPPu08DcYPWebSs84SvUUlLBrK3A0bHZCPSKDZhmBZWshlXw2bTpm9X+2ihz3STz9smdFoKdDWFD8bLlB/ob1qYctnlKZlrgt9WV4+Ow8Xbl3VHQg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Index: AQHXoKxowwaOW1OR4UKei+2TOBJan6uXHAEAgAQmJoA=
Date: Thu, 9 Sep 2021 06:13:05 +0000
Message-ID: <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
 <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
In-Reply-To: <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 35ec013c-eb7b-4640-acbd-08d97358e380
x-ms-traffictypediagnostic: AM0PR0302MB3217:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR0302MB3217A85C653FBBDA32FBEA6FE7D59@AM0PR0302MB3217.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:4303;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 bbaUUaGclTzZAz+RwBUpMOiy0BQiIhDek0emtZoa+yPtll2rNg733gRyojr9wTMdHYYVv//3mPU5fbEAJauZUKrufEbq3+ZDMJSmOlqSZAj4H6KBneSlgdpTTVy4J6DdSLKKii/ROkST7gW7ESMafkvBnyXB0Xge8+u3Trzn3eB1u3xI6lFNcwGK8Ccx8BdZQnRsWxklEP6vDbpErdBLAnPyto1OZdFjeVNzftzGoZf16fOokdh5FxHv6oDTzQUR4jBNRahzY4Cevrjaw/mNiRs0DIHOU9NrCm2kyAJDcZ8m9gugaiqQOJOF8lZGmNQRudO3ZbudQ+it3NHvPPRhuf8IwDESGXXhlxQnPDvTlrCPu5DdBWY3wl0isaliolauychAKSh1HeB8OloXfQ0dBNYG2edQyugokHXBW2ZsyTyGE2/3zXpv97eU5SAAZVV+8FBth1fwg559QbyHqBDya5LxkKaXhslTbhhXfNCgX6QPIBGIItM9YIIqt0QiHQBwBuvYGQFwjc3nuQ8ghZXOGDQVyPpic0VIbxEmGWiDEPZbsQDHgZThuWqEXzXRgTiTDflTXpgnK4ry3oMVjLb/faIo/JCJRgP2sUTpF5eweYhxFOSP73T2EJg/CWOO19+jhd3NcGwqLyZIGLBG+w9bKG8JNJAUnEflrayeLJhIHdeyf8z+VDZutVuZ5NuCVDhYO+bfwVW5yI10pIcZr1X7MJm+O8oGkXuKSR6nkJBrVenMwsTyhv+UQzY1U+HTW/dH
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(366004)(136003)(346002)(376002)(83380400001)(2616005)(8936002)(478600001)(4326008)(36756003)(5660300002)(66946007)(66476007)(54906003)(38070700005)(26005)(2906002)(6486002)(186003)(66556008)(316002)(122000001)(71200400001)(8676002)(31696002)(53546011)(6506007)(76116006)(110136005)(86362001)(66446008)(31686004)(64756008)(38100700002)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?YWtWaVpHY0NJYmhvM3V1YURYOXpFaDUvY09idytxRS9SMHA5ZUVkbm5QL0k4?=
 =?utf-8?B?Yk1ZblZuUHJUTlZ0QlJLNHB0TUtZcWwvbUxncFdNdlJwdUthVlJIU3JzSGhv?=
 =?utf-8?B?NFlVY29IbGkzemhFZm9PS3FIN2pyUlFYZmVOVUhaYlBiNzVtMGNIR0lDaXp1?=
 =?utf-8?B?UFBDdFRiR3dRMEd1RU52eUJYc3hPVE01VXp2blpXWEx2dHRIVURvQURBSjk5?=
 =?utf-8?B?aHFVNVlIQzduMDI1TXl0c0lvak9LWHZ0KzdJSW1qalZzekxxblVCeHFkMnVP?=
 =?utf-8?B?NmtHdWFRSUJXQVpUZGRtTDYrbnphMDlnODdqSDI3bnhQdVNJTGdDcDBMbTYy?=
 =?utf-8?B?NkVoYXpqaTNqWWVpMUptZVducTdtajVSODhKZmJST2pSayswZ1NzVXNQeEJx?=
 =?utf-8?B?UitmQlJxelRzL05ZTkdIcktVMkNDOFpCcUpYVzJZekxkOVdHOFZCNE5tZTVN?=
 =?utf-8?B?Wkt1OWxpSG5VZExYMXRDMHRzMVVkbzA4NHZWVHliQnhHNUlNcG1vN1BtUVJB?=
 =?utf-8?B?b1pCdDZsMHZTVEFpV2Q2MXNjNWNTVzhWWGp6andmSUdqaHZJWXFHSXkvUENk?=
 =?utf-8?B?NWYxQ254NGNwbjNRZVZ5dkd2OUtLN0w4RnBCZ1hpdTM1NGlsWTlmeUlXMVpD?=
 =?utf-8?B?aEg2MlBzeXAyM2NJZ1R6OHIwc2g2cmlXeUJDQjN6WTdyUVBVTFllZFZJeHdU?=
 =?utf-8?B?aDhLU1pwWml4MXd5b1Z4TjZoYlRpQS9NSk16Y3EreFQwdXNjOVNLOWJYcC9U?=
 =?utf-8?B?ZnhxZUNRZHBoS2xmbnExVHdzWUVBYWMwSEZTdFlGVUhSN2xYNzUvS3B6Yzc3?=
 =?utf-8?B?M1Njd2NLQVQ4dkZzZmFLa2EzYldBUUh2LzFpblQyeXhLMUljT3JIWXJnZVVq?=
 =?utf-8?B?bkdxQ1pqOUlBa0kwYnorMzAwMkREYlVLclU2Vno3ZitGbkNsR2ZKWnlyVmRQ?=
 =?utf-8?B?NW1qL2RIVXR1VGkyYVl1Um43U0I4VEVRT0tPR3JYbk5GSTJPdzF1WmZmU2Zk?=
 =?utf-8?B?YlZEZGFKTUV5azF3ZzZabGR6MGxBRGhWWEVsZm9yWGNUYWxqd29ITzZlS2hB?=
 =?utf-8?B?S2xHZ0dOOXlJeXdsM1JnUEFXZnJBMm8yazdpVlhmM1JmQzFxTUdoK1dMazNs?=
 =?utf-8?B?LzIvUWsrQlllTDA2a0tjQlo2TDcwNGIyQUYvOWNJczFaNHVvYWZwcU83YlhI?=
 =?utf-8?B?L0RiOTFSZUJsd3llODIvMGpWTXc4YUJkVk1wY2p1YkphNnh3N2tJRi9zVWw1?=
 =?utf-8?B?QStJOTVGYUFBY0NVenBmUGFlTCtOOFNNTHhWTnF4SlNsak9IZ3NSQXJvdkxS?=
 =?utf-8?B?dnlKSjNDemZhcVFYMDdDMWFKTkM2c3l2c3gwb0diOHVhKzNaVHBVZ2NaLzF2?=
 =?utf-8?B?TmZKOG4xOE9acUxzLzFFSlRrU21rMzFJTit1aTlGN2MvS3ltWEdEcTNJRVJU?=
 =?utf-8?B?TVpDV0p6Wld1UER4NUEwbmNucVBtR0hpVThTUEVYOGlPVDRMSnFtaTBJNUNM?=
 =?utf-8?B?WDk4YU80T2pRZmtEMHBVRlI3em5JTFJXU3k2WmFGS09GeTc3M2R2TGJhc3k1?=
 =?utf-8?B?M0FGSm5KWldJMDYvYUVuYnRrMGVwWFhZTncrdTcxczdSRDVpRUh0ZXhtcDZG?=
 =?utf-8?B?RlRmYlovRnJqWnVwL2xMeHZQNU1IeUYzQmZLOG5UN0JnVFRSSERadkNjWVV6?=
 =?utf-8?B?OTZ5a2VVVnpidEp5dUh6SC9DMDVqdXY0ckVVM2RTU3RUYy9zbWJIdVNTZEhr?=
 =?utf-8?Q?b4tIBi8eh6HodixyDNhxAW+7hmEqj15Cn9tq676?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E4F3646E79C0254D869B266C87003108@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 35ec013c-eb7b-4640-acbd-08d97358e380
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 06:13:05.9448
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: G2b8NzdW718EqPAxqpfwLmNL94hyOlPSaU1y6GO6fmkAVuWnQm6GNGg3SrXZ9SPcm3/9qo6fQO5HXeKum8VfYScLs6tYfxrldmHEnV0gkgqakwzSKpluUkCzfq7EyoSh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3217
X-Proofpoint-GUID: vnd7x1L48kq7i73rGbxtM6UrRlk9RU-O
X-Proofpoint-ORIG-GUID: vnd7x1L48kq7i73rGbxtM6UrRlk9RU-O
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_01,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=957
 priorityscore=1501 lowpriorityscore=0 impostorscore=0 mlxscore=0
 spamscore=0 adultscore=0 clxscore=1015 phishscore=0 bulkscore=0
 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109090035

DQpPbiAwNi4wOS4yMSAxNzo1MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDAzLjA5LjIwMjEg
MTI6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gQEAgLTM3LDEyICs0MSwy
OCBAQCBzdGF0aWMgaW50IG1hcF9yYW5nZSh1bnNpZ25lZCBsb25nIHMsIHVuc2lnbmVkIGxvbmcg
ZSwgdm9pZCAqZGF0YSwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyAq
YykNCj4+ICAgew0KPj4gICAgICAgY29uc3Qgc3RydWN0IG1hcF9kYXRhICptYXAgPSBkYXRhOw0K
Pj4gKyAgICBnZm5fdCBzdGFydF9nZm47DQo+PiAgICAgICBpbnQgcmM7DQo+PiAgIA0KPj4gICAg
ICAgZm9yICggOyA7ICkNCj4+ICAgICAgIHsNCj4+ICAgICAgICAgICB1bnNpZ25lZCBsb25nIHNp
emUgPSBlIC0gcyArIDE7DQo+PiAgIA0KPj4gKyAgICAgICAgLyoNCj4+ICsgICAgICAgICAqIEFu
eSBCQVIgbWF5IGhhdmUgaG9sZXMgaW4gaXRzIG1lbW9yeSB3ZSB3YW50IHRvIG1hcCwgZS5nLg0K
Pj4gKyAgICAgICAgICogd2UgZG9uJ3Qgd2FudCB0byBtYXAgTVNJIHJlZ2lvbnMgd2hpY2ggbWF5
IGJlIGEgcGFydCBvZiB0aGF0IEJBUiwNCj4+ICsgICAgICAgICAqIGUuZy4gd2hlbiBhIHNpbmds
ZSBCQVIgaXMgdXNlZCBmb3IgYm90aCBNTUlPIGFuZCBNU0kuDQo+PiArICAgICAgICAgKiBJbiB0
aGlzIGNhc2UgTVNJIHJlZ2lvbnMgYXJlIHN1YnRyYWN0ZWQgZnJvbSB0aGUgbWFwcGluZywgYnV0
DQo+PiArICAgICAgICAgKiBtYXAtPnN0YXJ0X2dmbiBzdGlsbCBwb2ludHMgdG8gdGhlIHZlcnkg
YmVnaW5uaW5nIG9mIHRoZSBCQVIuDQo+PiArICAgICAgICAgKiBTbyBpZiB0aGVyZSBpcyBhIGhv
bGUgcHJlc2VudCB0aGVuIHdlIG5lZWQgdG8gYWRqdXN0IHN0YXJ0X2dmbg0KPj4gKyAgICAgICAg
ICogdG8gcmVmbGVjdCB0aGUgZmFjdCBvZiB0aGF0IHN1YnN0cmFjdGlvbi4NCj4+ICsgICAgICAg
ICAqLw0KPj4gKyAgICAgICAgc3RhcnRfZ2ZuID0gZ2ZuX2FkZChtYXAtPnN0YXJ0X2dmbiwgcyAt
IG1mbl94KG1hcC0+c3RhcnRfbWZuKSk7DQo+IEkgbWF5IGJlIG1pc3Npbmcgc29tZXRoaW5nLCBi
dXQgZG9uJ3QgeW91IG5lZWQgdG8gYWRqdXN0ICJzaXplIiB0aGVuDQo+IGFzIHdlbGw/DQoNCk5v
LCBhcyByYW5nZSBzZXRzIGdldCBjb25zdW1lZCB3ZSBoYXZlIGUgYW5kIHMgdXBkYXRlZCBhY2Nv
cmRpbmdseSwNCg0Kc28gZWFjaCB0aW1lIHNpemUgcmVwcmVzZW50cyB0aGUgcmlnaHQgdmFsdWUu
DQoNCj4gICBBbmQgZG9uJ3QgeW91IG5lZWQgdG8gYWNjb3VudCBmb3IgdGhlICJob2xlIiBub3Qg
YmVpbmcgYXQNCj4gdGhlIHN0YXJ0Pw0KDQpXZSBvbmx5IGhhdmUgTU1JTyByYW5nZXMgaGVyZSBh
bmQgYWxsIHRoZSByYW5nZXMgaGF2ZSB0aGVpciBzdGFydCBzZXQNCg0KYXBwcm9wcmlhdGVseQ0K
DQo+ICAgKEFzIGFuIGFzaWRlIC0gZG8geW91IG1lYW4gIk1TSS1YIHJlZ2lvbnMiIGV2ZXJ5d2hl
cmUgeW91DQo+IHNheSBqdXN0ICJNU0kiIGFib3ZlPykNClllcywgSSBtZWFuIE1TSS1YOiB3aWxs
IHVwZGF0ZQ0KPg0KPiBKYW4NCj4=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 06:28:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 06:28:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182624.330275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mODXU-0005j7-02; Thu, 09 Sep 2021 06:28:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182624.330275; Thu, 09 Sep 2021 06:27:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mODXT-0005j0-TE; Thu, 09 Sep 2021 06:27:59 +0000
Received: by outflank-mailman (input) for mailman id 182624;
 Thu, 09 Sep 2021 06:27:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mODXS-0005iu-Es
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 06:27:58 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 121b110e-1137-11ec-b1a4-12813bfff9fa;
 Thu, 09 Sep 2021 06:27:57 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4659B2230C;
 Thu,  9 Sep 2021 06:27:56 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 0B9CE13A61;
 Thu,  9 Sep 2021 06:27:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id e7xIAWypOWGHXAAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 06:27:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 121b110e-1137-11ec-b1a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631168876; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type;
	bh=KVvxTR/WFnUlWyEGIAHkydWApcA2cjZ3/tqAIpuxxZ0=;
	b=Do3yIESp8p9pE8E7SBDcL3+qKBkVvQy+5XukpRBbqj5dJp02oC3bcnJLTJ21FICl4D5Mjj
	6jdCQeMExmhIZ2qqPe7pvI+GCv40ap1wcXfTTERczYaYxMVJXKlCWp8htEwRlfNWQXV/Hq
	4B9InQrrhkoxrAN379njG8rCiBti04w=
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>
Subject: Introduction of stable interface between Xenstore and hypervisor
Message-ID: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
Date: Thu, 9 Sep 2021 08:27:55 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="fzSR67JHUtEPKSIwAP8omhVzYar2VS9yj"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--fzSR67JHUtEPKSIwAP8omhVzYar2VS9yj
Content-Type: multipart/mixed; boundary="EjUglcClrXlVesDqQOeguf8gQKBuNrQ7l";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>
Message-ID: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
Subject: Introduction of stable interface between Xenstore and hypervisor

--EjUglcClrXlVesDqQOeguf8gQKBuNrQ7l
Content-Type: multipart/mixed;
 boundary="------------125A8F04F6718D3FDD1681A0"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------125A8F04F6718D3FDD1681A0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

Today Xenstore is using one unstable interface to retrieve the state
of a domain (existing, dying).

In order to decouple Xenstore further from the hypervisor, a stable
interface should be established. This new interface should cover the
following functionality:

- get the needed state of a domain (existing, dying)
- get a unique identifier per domain telling Xenstore that a domid
   has been reused (today Xenstore could in theory miss this)
- provide a way to avoid having to get the state of all domains when
   a state change of a domain has been signaled (there is only an event
   "some domain has changed state" today: VIRQ_DOM_EXC)

I'm suggesting the following hypervisor modifications:

- Addition of a global 64 bit domain sequence counter. This counter
   will be incremented each time a new domain is created.
- Addition of a sequence count to struct domain. It will be set to
   the value of the new global domain sequence counter when the domain
   is being created.
- Addition of a global bitmap with one bit for each possible domid
   (resulting in a 4 kB page). Each time the state of a domain is changed=

   (domain creation, shutdown, deletion) the related bit is set.
   When the VIRQ_DOM_EXC event is being registered the bitmap is reset to=

   have the bits set for all existing domains.
- Addition of a new stable hypercall ("get domain state") returning the
   following information:
   + domid of a domain having the bit set in above bitmap
   + state of that domain (existing, dying)
   + sequence count of that domain
   The related bit is reset in the bitmap as a side effect of the call.

Xenstore should be changed as follows:

- At startup of Xenstore the state of all domains is being queried via
   the new hypercall. This will cover all existing domains as the bitmap
   in the hypervisor has been reset to flag all existing domains to have
   a changed state when the VIRQ_DOM_EXC has been registered by Xenstore.=

- When receiving a VIRQ_DOM_EXC event the changed states of domains are
   queried via the new hypercall.

I think those modifications will cover all requirements while being
rather limited in the needed effort.

Thoughts?


Juergen

--------------125A8F04F6718D3FDD1681A0
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------125A8F04F6718D3FDD1681A0--

--EjUglcClrXlVesDqQOeguf8gQKBuNrQ7l--

--fzSR67JHUtEPKSIwAP8omhVzYar2VS9yj
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE5qWsFAwAAAAAACgkQsN6d1ii/Ey+h
vgf/er3BLPxaJ84BzwaUcU+a/0Za4yLpJDjkbahAzhbyTSAh7AdOM2xwaFJLalJnOv4p/D5BrobC
KWDx2vFZBS5tXiI7HSrEiTfo2fIQZFpQ0JbSYK5Er50EaADoIHnVG1HfV+NVRdeMf3HEoz31j/fs
BZupoGfSw3a7Vq4QRKhX3LxpQAnQ4AsqR0ivwN/OxkbtK5Urf29tW9/SYMb66quefqWEDCSo8l60
4FEfIjSO9TSUf6O+uXFughHl2CefOjzKj7K6FJ3ezbT3V7T5zuHXm8GP1+dEf/U7ZzwULw7xA4y3
jsGKJDc3PwJkiyUXz8//BLSCIe0Fp7xeuvYaqllZkA==
=Khw1
-----END PGP SIGNATURE-----

--fzSR67JHUtEPKSIwAP8omhVzYar2VS9yj--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 06:59:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 06:59:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182632.330292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOE1Q-0001Sr-CJ; Thu, 09 Sep 2021 06:58:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182632.330292; Thu, 09 Sep 2021 06:58:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOE1Q-0001Sk-8f; Thu, 09 Sep 2021 06:58:56 +0000
Received: by outflank-mailman (input) for mailman id 182632;
 Thu, 09 Sep 2021 06:58:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOE1P-0001Sa-7L; Thu, 09 Sep 2021 06:58:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOE1O-0007HL-DP; Thu, 09 Sep 2021 06:58:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOE1O-0004XL-3w; Thu, 09 Sep 2021 06:58:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOE1O-0000eq-3Q; Thu, 09 Sep 2021 06:58:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=VPKhUank3DuQX0tFkXbIXOHp2cLzL3uKZVKiyBEmnsg=; b=iGFi/xQ+m3rouQ6SUC8mj/b7kL
	RFPoCWD8XjAaYJJUIs6EhB8WeproU2WIOiqJeqC/0KAsuWz682mkjSM/77gsLmlDCvqI6+SMmMxO1
	SHK7rGjZEzVdCbe3oB+18Cgbw96eb7UNKzlYbMR9NW2YHqwzvCBmuThBrApS136b+cWc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164888-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.14-testing test] 164888: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.14-testing:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    xen-4.14-testing:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    xen-4.14-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ef6455a3707cf1c7c61ad8af12558811eeee4ba8
X-Osstest-Versions-That:
    xen=49299c4813b7847d29df07bf790f5489060f2a9c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 06:58:54 +0000

flight 164888 xen-4.14-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164888/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 163750
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 163750

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 163750
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 163750
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163750
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 163750
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 163750
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 163750
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 163750
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 163750
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 163750
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 163750
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 163750
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  ef6455a3707cf1c7c61ad8af12558811eeee4ba8
baseline version:
 xen                  49299c4813b7847d29df07bf790f5489060f2a9c

Last test of basis   163750  2021-07-16 21:07:40 Z   54 days
Failing since        164261  2021-08-19 17:07:29 Z   20 days    9 attempts
Testing same since   164888  2021-09-08 14:08:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Dario Faggioli <dfaggioli@suse.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   49299c4813..ef6455a370  ef6455a3707cf1c7c61ad8af12558811eeee4ba8 -> stable-4.14


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 07:06:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 07:06:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182697.330418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOE8K-0004gj-QW; Thu, 09 Sep 2021 07:06:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182697.330418; Thu, 09 Sep 2021 07:06:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOE8K-0004gc-MF; Thu, 09 Sep 2021 07:06:04 +0000
Received: by outflank-mailman (input) for mailman id 182697;
 Thu, 09 Sep 2021 07:06:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOE8J-0004gW-KR
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 07:06:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6439aa04-113c-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 07:06:02 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-73G9n_BsPaC1GMD3wqt7PQ-1; Thu, 09 Sep 2021 09:06:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2336.eurprd04.prod.outlook.com (2603:10a6:800:27::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 07:05:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 07:05:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0029.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19f::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 07:05:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6439aa04-113c-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631171161;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=fcipelzzzmJgRBZUJpaz+bt87OOxbuC/mcjaojGC0uA=;
	b=U7YJ/CeFK4dsHOzMApr04d/sekIm9u/n61xgvsrt0n+zMNfNf3aE48W60JEZ+sPuY5cHWe
	SfxtDCG1apiWdSqBGkRmzUm0iZ3wOXIUAhmlk2WZ5JBgp7FyDo5SJSosGPB1x8vvmnSutM
	qAx0m5A+KQBfus4p89MmCaOLIIeinNg=
X-MC-Unique: 73G9n_BsPaC1GMD3wqt7PQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oWca3Siq9mE4YcuD2+MIpVyNpKtSZdzgdYBE+3vdqfTKCG21fZiqPcKCn4j2xj7qvhJMsTY7vIrUgWtfZQ9Sh02VBmaXDEGFnI2FFJOm3twcBvU1ilbpl+wqnq9u4TJh514vUuSpD2suMUHkvzFfwS0e0KA4Lb9vic4w91wicEvMJEQ4ciG5nU9KJcpyZzSqMwRhpKrJYz3qmJ6h4HopOC5vcnKLMBCheE4WSHYC2PoYe/j1nrkWAMwX+3ImpKtrDjY72/NdJhy8YnKUfbvbM477uFEKNfzuyas121fWm4YhDPVR4Yjlly996GfwFLvtopcvl1/M2+U6pVQ4QZaDwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fcipelzzzmJgRBZUJpaz+bt87OOxbuC/mcjaojGC0uA=;
 b=BYw/NtoQH1Bi4gtLlVsvnVqClUvgbXPzWOewExEhr4pT6t9Nhb/VhcFBXpKbK9B+rchbn4lYKyn4pyao4sg4WHsg/ynikpna+2Ga6Hi8pdBqTxK2dVKc+Mg3G4hGQ7jPNDv0LqzPDcT/jowKtC+OsevPNIWF5P/c1NlcGf9k/iuDJOI7F+t/W1z9yr3wdXNXFdhIYt7SMQBiGibSk0Qwt+3ZH4hrJ+DF5dCPoAUBOCjjWNeEuaKpIYT81xTE7w6vDKID1ZDUuzU6b/clB2pKcABcfPdLtqaHwtzD8CAoUBfpYNFNholhayWOyAyuQvv4K49W+JHOF+5BT6xoRnuJrQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
To: Penny Zheng <Penny.Zheng@arm.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "julien@xen.org" <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20210908095248.545981-1-penny.zheng@arm.com>
 <20210908095248.545981-6-penny.zheng@arm.com>
 <alpine.DEB.2.21.2109081654290.10523@sstabellini-ThinkPad-T480s>
 <VE1PR08MB521580BE54919E53282AE691F7D59@VE1PR08MB5215.eurprd08.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c56961c6-0733-2245-ced2-61a33ddcd1c7@suse.com>
Date: Thu, 9 Sep 2021 09:05:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <VE1PR08MB521580BE54919E53282AE691F7D59@VE1PR08MB5215.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0029.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19f::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8869094c-bdea-4b83-e6dd-08d973604671
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2336:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233640B35BF823F4FC75F63BB3D59@VI1PR0401MB2336.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:514;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nX6WsE7Uu6iMOt8GAVYUmArep2xMhywO6re5zxaAF2dy7XXZrngE2yZivkIuJhilA3/6sWnT8vEw/ORRAIX/0RRwdJgONGfel8Qf61TcOrmmO8omZN63sR4BvOpvZj/CAnfa+ILI8E1wm9FwXnP3Y9HIxFmkhEqGzkai9pLCDozxT40xlCkhjsES4DWGAkTo6odhKQvIl3YJgKlnxNaeB9X7U9N4hX7p49w/Hgdr8bsKiWiaW2N9/5zjmB+byvg9m2e5/AuXEIDvXYPpARSnRlaxQYdjlfUOwlAH0/e5SW/AWOutrdebGI9xeTAlnNpoxgRZIiEFwHpiX/H9+UAGD2Wg7KSv7JcL2N73N8k1i65M2j7uzGVeBl0AL05CI4+uo/v8Vxby+MVG+qXI0I3P0pz5H2zgDj9TIyxXMTKbiz6uXl03tWwBp5nUntdFtusaTkC3tZCEgbPuSuXA/3kaJmcMg8LYNRvDYMkPaYcV5wG+xRa1y0x+NRgzxbXlcxH2EmCCl3qJkgF9YPlkyBvQsykUhn9o5eUOpJSTlof8hAl2+SHWz6jnXed9RYuFPITrJS+T8+YFTccFquO1MxvxchqUFDjjodJraVVSQij7lV8YZZX8Em0ZevD58HQ+K4BykF78yrFyO/qju6/zCIAehftxvYle2/acJo4oQ8lTrvV/vDLbykhwn1vhFv58YbZ5enx0U47voky88YUuE5OYC5pM8m2hY/0byII0H4LgSB0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(346002)(366004)(39860400002)(136003)(53546011)(8936002)(8676002)(4326008)(16576012)(6916009)(38100700002)(186003)(83380400001)(4744005)(2616005)(956004)(54906003)(31686004)(316002)(6666004)(36756003)(86362001)(2906002)(66946007)(66476007)(478600001)(66556008)(26005)(5660300002)(6486002)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c0ZIQ3dCdTdIRDJUS2dMVWZ0NHY5cEI3MC9HZnhwb1RmM0RtN1dYRHNZcHky?=
 =?utf-8?B?ZDJuZHQwSkk3NTJseHFTem1QY25ZR3lUWCtvK0pBSnFETGIwYndCTFl2UE1x?=
 =?utf-8?B?SElZYlZDQi9rc0RIenpweVExVXgvcERjYk5tbFdtWkpsNkRmRW1QWGxHeU5O?=
 =?utf-8?B?dlEvUnM3L3B2eXdIOUlteDhvRUMwT3o0ZCtUS1p1VFo2dDJmOGdDRS9QeVpD?=
 =?utf-8?B?ZGFZVDlBRW5RZUVvb2xJblc1RTdTbk80anhvS29KcTV0dU9MUmFXWi90czAy?=
 =?utf-8?B?dHd3RlUyYlhRK1V2NDM1T1lmcktkRnozVFR6R1FmclA0dGZxNzJDMEtZTzds?=
 =?utf-8?B?dTlPQkg0YjJFUFRGYWppTGdqZlRnZzhxb3Q4U1Yvb1dVcm41UEUzWDFrUjA5?=
 =?utf-8?B?SkxxTnlSR2ZMc2lIaENkYXl6QzRUVnFhTDRFMzI0bkp5ZHBuVkMydnVKQk4z?=
 =?utf-8?B?My9rQlh4cElZcHByOTBmSHlhbVByb043dmNib2pIdDg5cmlrTW5jVFZUY1Zl?=
 =?utf-8?B?WE1rOUZSVVoyaFkzUGxESGoyUW9LL2Q1Umg4K0Q3Tm55NE5GdHpJeDVSc1I5?=
 =?utf-8?B?L1pBMGFlYXl3NFlQY0psNWJhZ1JwTGFjYkkyNWxhdEVWMTRMbTU5OGx1aFJJ?=
 =?utf-8?B?a1QzbmNONmdLenhlc3I5YTlQZUpxN3Q3SjlBUUNzSmViYmh3THlaZG5zK1N2?=
 =?utf-8?B?U1pDbVQ5N2pvaGNPa1lTTUlEQnoyTkN1cmRmWURLY3loMGY4dFFXeVVhYnNr?=
 =?utf-8?B?SXpYa1JSSDRhMVlHMG1wcVNmSnF4S1U4a2oxZ2ZVTW5Lcnpna0ZxaVRUVzh4?=
 =?utf-8?B?V0hkejg1YlR6Zm9RVU9XSWVGM1BRakcxMFhZVHhHWW8rRXlqdU15RTJSeVdN?=
 =?utf-8?B?RC9oZzBuQ2ZUS0JYTUI1c3RXMmNmVWwvRE1ncHBzVlpKdXZDWFRuTDBWK2ZS?=
 =?utf-8?B?Z1Q0NEVFU2lNMDlvS0FTY2kzTHNVWGVyOVcyZW1DekNlby93cGJURHhsVnZ6?=
 =?utf-8?B?ZC93NlBiNG9GVExSY21xY3BmNUZsKzZYOTJISDErWnpKZlB3K1kzZFl3d2NB?=
 =?utf-8?B?U05DRE15T29ZcHNYYnBVa29CbTRzNko5K3BoRGFWN3dYUUV2bUp3ZTFWYkY1?=
 =?utf-8?B?QzJEUStyNWhuU1RWN3ZsM0gzN1NwdFhETzVEUG1TN2k2WkZzUjB4V3BaNCtD?=
 =?utf-8?B?QVZoeGtoS3lsaG8zVnJ4cTRPTHlVdUw0SlpwTGJ4Z0dQR3BHVXcyVUVzcEtH?=
 =?utf-8?B?dnZjWHpDQmY1b0FTR0VPeDVCRnBoRmlGYlQraGdhNW5GQ3EvZ05qeXhMQ2xI?=
 =?utf-8?B?OHZDS3YxNWNmWE9MR3J4YmhTWEd1RHdiU1p3ejkwemUrTHNJbEhpck5jZ3Qv?=
 =?utf-8?B?dWVTTFgwTlEwUzIrek9HSGk1M3E4aEtFTWtxUmJpakdPRkZvaTllQVN3d2Rz?=
 =?utf-8?B?TGNRNXh4S0Rrclhudmk1Rm82bmQ0eHBJc2ZXTkg5U1VNaTNJenBUcys2YjE2?=
 =?utf-8?B?N2gva1JhZHhYNXRRcXNzcVhHR2NsanRwOVFxbTYvaW9LenhWaUE2UzlMYmoz?=
 =?utf-8?B?UU1PRFJMSVE2UCtrTTRxTnZwT1ZhQ3dDNzdmSk9SWDRoY0VyWTF2ZkJhR01o?=
 =?utf-8?B?UDhuMTJETHMvUFZXZEgzdTVxTm1TMW9mbmJ6SUhlRDhHd3NKQnhwWkpwWUhM?=
 =?utf-8?B?cnA2d0Uvc1pwM1lSVnFZMU14U3JCd3BjN2hYRmU2L041MHNVbDZmUVJsMy9Z?=
 =?utf-8?Q?h0bqU9Ik9XYw3eqXoa4XPd4wMD6K+1QHNB3+xFI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8869094c-bdea-4b83-e6dd-08d973604671
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 07:05:59.0957
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RhvkbFOyuz52TyT4veLNV+MP7c5aq5csnXEq6HJ02EH5PGkpuw5oKJhXGbqz7pkMe23d18wx2Rj85GDbWg+4Eg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2336

On 09.09.2021 04:20, Penny Zheng wrote:
>> From: Stefano Stabellini <sstabellini@kernel.org>
>> Sent: Thursday, September 9, 2021 7:58 AM
>>
>> On Wed, 8 Sep 2021, Penny Zheng wrote:
>>> In order to deal with the trouble of count-to-order conversion when
>>> page number is not in a power-of-two, this commit re-define
>>> assign_pages for nr pages and introduces a new helper assign_page for
>> original page with a single order.
>>>
>>> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
>>
>> The patch looks correct to me, however I don't think I understood Jan's
>> request to the previous version of this patch, so I can't tell if you addressed his
>> concerns.
> 
> Would you like to take a look at whether I address your concerns in this version? Thx.

I will take a look, independent of Stefano's reply to you. It's been just
yesterday that you've submitted the new version, so I'm afraid I have to
say that I consider it a little early to ping for a response. I'm sorry.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:24:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:24:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182740.330471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFMB-0007fG-BE; Thu, 09 Sep 2021 08:24:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182740.330471; Thu, 09 Sep 2021 08:24:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFMB-0007f9-7w; Thu, 09 Sep 2021 08:24:27 +0000
Received: by outflank-mailman (input) for mailman id 182740;
 Thu, 09 Sep 2021 08:24:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOFMA-0007f3-63
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:24:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5683202f-1147-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:24:24 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-1cdfYwQHP_SSh7j15tiBtg-1; Thu, 09 Sep 2021 10:24:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4847.eurprd04.prod.outlook.com (2603:10a6:803:54::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Thu, 9 Sep
 2021 08:24:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 08:24:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.8 via Frontend Transport; Thu, 9 Sep 2021 08:24:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5683202f-1147-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631175863;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OmuYhMgA15CUwW4jq3ZtxwJqvsWj0orTDK1C3zPblsc=;
	b=MCWD/0FhJyeZOP378ydIoWPb2KkLvUBH1s8wk2ZoWdTZcczyOC1uK15ckeNQj6a1UOC8sA
	RGhXIVh6J9frK7jI4P9Umof7RdmNplqCeeDu2sRqMSvtAWjHTcPZNCHZK8Fes2IqROAXPU
	x6zwEpfXe01Oqo32ZbFtbK17PA4zn5I=
X-MC-Unique: 1cdfYwQHP_SSh7j15tiBtg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JAJ4sdL3iCzA3UiFuoPjw1U33/Wp33xxvBdmQI3RZa/l0ocFA5o4qbGld49Vz8XViW0FFT4Z4NLU1NqsMjZ0TqFjFkkYKr6aMkP7lXWU8O8Te4ib1duCF9AhdaV6aU0o+yVUkHZfSnxrqbbJL7b3LNih924lvW3/6Mmb9TKWuYY/XwE2DDZb5bPdig+VLcCOFdbPdF6TBkgpfXt1KRCfPxdhwLGXoN4kEy0rH7XRq0nho8/KJ2XY+OsTh5ztQF+2eGBCbDBAg+7YFbU4pLoFt+2Qj1Mley2DoEKRnqKaMJ89l6Qp7MhIyxz8BDhvwQThQA4jivNoYfNGU+f0Xl6Alw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ARrB8G9oG7epXdTmcH7phUZElpm3lxbUVHX5MKp8shw=;
 b=P5mCDlvs/wJvh+fRB0KTOnXPKNQGE+0ztRvEz5F/ZVliwvKGRYhGPofuSJvebMlhB/AvNH8nPz1iG2mqBnUVCxFpFHch88BexTFISQCGhvAxQzyN/YlogPcmhDFLHgJjBi9FsCmn4SkTYVtrfuzEMQSM7rSjSR2bgFPmYUvpmWcD13sMzV3lPP4pU1kDRd3aDWWYqnY9QPjB5IOdcn6c+2k+qgtCcgGDSViY4Plcgl50uvqgUzbVczsrVAoWdIC1ztWJR5nQKaGkHXzLQbjfCc/GJzmgH4V5oc5s5xoITrLAjMFAT89hpH1Io2otqwltlBpCN1IIQHSeZ2WePN5g+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
Date: Thu, 9 Sep 2021 10:24:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: db8b9c11-c8df-42e1-29b7-08d9736b3857
X-MS-TrafficTypeDiagnostic: VI1PR04MB4847:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4847823555BFAC2B2D509251B3D59@VI1PR04MB4847.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Hv8E8zqs1pbLYiPSDvNuZgd3IgEjZ6WXMExIOtcyZGBWScYKrmc3pDxj1sT/AqOZGOpf8iEJ2gm61yxtqAYxxcEkgdBWRFl7Ho5/aqd7ZjzIIfWwchosJ7eAh6M+Eo1C4nZry2tTmGuKTtHRxNtOCJdKOgjzk+qOjTQnzxz9SCQnkA960zSzza9v6cecKGejk1rWX7Lw9f8kHKmVhT5SFkDLo1gGYqWYDrCjoMi4FY3ZzRe/LKZudHvYr67pSja6qEMbRliI8ZNtzvMY5GDoAm8OMFJSBii7IkaGPWN+CjA+GTbCJo3YMs1X5aZ72dVsWzTNJa/7IivsTxywJM5JBu70vhohumsF8vWhjLuew6qUvZp0AB1yIiJODfu/z0qyKHMtrABwH7PTvxuVmq8Fuaa7xmFvjhqcfZKIK/d4hTU8mdlepHGO2gMWaTEv46E3WLXCmmojjDbZFY8Q2v9uqOatkCbs/GbdTRcONXgcXegObg4YErIzwQff4Lur3FCYAnBv3DmuaJPpUGkc9I/n4Ks5JbBkWFsxoSAZ/tfJCPrvOGAP1yYeLEKltcvGlu6gMt5DgfAx9CVTAITT/C1q89NdCSLrXQMHG2m6h25sXPtvKj9YAp8L+5whUe4ANDjwptxVWMB4ouyg3dIVzRHD5sNf+NFah2W5tBoh++KMtd+E5vFL+2iY8cWQBxMU9vTSnaNIJ477rt8N55Utq+IFO1KIfl2A/IBeeyuO2L9WauI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(366004)(136003)(39860400002)(346002)(5660300002)(36756003)(2906002)(8676002)(7416002)(478600001)(110136005)(53546011)(66476007)(66556008)(16576012)(6486002)(8936002)(316002)(4326008)(86362001)(956004)(31686004)(2616005)(83380400001)(186003)(38100700002)(31696002)(54906003)(26005)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?aUDk4TZoKLfn0ALDxYJD4jyMGavfz4GMUVCzamb/Yg9BwF4JIMGH4njmHNa6?=
 =?us-ascii?Q?KKifFMhVffdZJIev9s0iF3mda1p5BWJ6j2eHZorI5LHYjN2Ij8wQNv4XZcPI?=
 =?us-ascii?Q?iFh5xM5RFf3rSR4V+ZZGBVXKV5O3CVJTHcJWT3cKPPIpZj+0ZVCpZn24DGBo?=
 =?us-ascii?Q?QDuQP0pNoDi4ygu6sc8vTPXYLJfFPwElFslzxg454VfzmCHMmGWrUxyTaT4B?=
 =?us-ascii?Q?/KZmIV5oBFY0AsWZCx53qACkDADDSZ0KOg/HlDelHlHAUqeef7yLE2rxvnQj?=
 =?us-ascii?Q?3DSpy1EFsvx7uJAKelAu4vs7N0/RbLx374eRYUejJ9O8Rw4Z3ag1CaaU9VEI?=
 =?us-ascii?Q?Fjn4/FQSU/FBF79thGI7wBynLwoG69Q5RaQXnIxf5MXJQtW+xtQZ3TLklliI?=
 =?us-ascii?Q?9ZbDALGsQNeJzcN9SqU98qF7nXCY8/Q441KkOK1olEZanxq3gQd7rYmrhgzF?=
 =?us-ascii?Q?MQ2YP+ANrVpXexRwWmyTGzc6lwFYh1Lo7lcm2zIFSjzmNvPjHe4GTgviYpJ3?=
 =?us-ascii?Q?PhrX3j7IesciPFINTDSKH76+Ee51p8XJ2NiHAGgcdwG6MPCZlsD/7KJfBsvo?=
 =?us-ascii?Q?XZA0IFEwkHkQgx06SXY3Zu+jI4sEwsCgVAv5bomhn8dMStzDpKa3ObDkm92W?=
 =?us-ascii?Q?2c5fR+7c3hOArtDRkTPDe2tdtGqPwc6edGSnY+BmSIXDAMXzDkvPe2+x7l24?=
 =?us-ascii?Q?VWZ/c9YfAb2t+QAUBbvBvuxl8XXWJJrOkjF7YRqPcIGleytdX4ShGpT0swpA?=
 =?us-ascii?Q?byjNyVd33T1DSmVOr9ufLX0i/l4hmUAIaOtptnkSXfr9sLeeY17X8aODZ6UT?=
 =?us-ascii?Q?QLAxKRP5r6q/fo7su/c/KBx5ug0IXP4ivZM9vGGayvOk5HNs5aJQFpWHhxWY?=
 =?us-ascii?Q?xbQpxv04EkpwH+GIbT1iKyl793SHmc368qg7kj3vwKcg46yHaY/8Bj1mLUuL?=
 =?us-ascii?Q?y0P1VvH5s00hLv2vIWY/+gSj5lPJMf9yOHqghCfUMajODtpsLGYN8b/YUxAt?=
 =?us-ascii?Q?80xx/ZyzPUq0w9I5Y57Qrhy7ksgo9yIfx05iZE5PQFmi96QdX8WGwFSYwC2b?=
 =?us-ascii?Q?z3JpysgSd7JRpJzMFbzAZY93V2jsgs6Acj/5CtJAZg5+VT38ji+Qv5U1awTR?=
 =?us-ascii?Q?3DW+q61II3aXU4qhl6LnJRSienLwR1T19IGcfV9f6eWiJ26Xos8Sfw4mCSL7?=
 =?us-ascii?Q?+esBs1jpnPwinrdmk25OklGHKkNb3gGPq7MBB2SVPCV+g/h9TORcCsPvb9Nv?=
 =?us-ascii?Q?fa+23E7CE3wiwddIDGT+uWZQ60gRUikpjmzhVua9cz0tkw8pERPH1BgJnf/m?=
 =?us-ascii?Q?89t9KivVOyBIaosQRmLsmahj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: db8b9c11-c8df-42e1-29b7-08d9736b3857
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 08:24:19.5046
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EM/p8trsCQ1vckOK0S8wPBpGmgZV3d1xO8MTutzOudWBNG+wzddg+roNa36Q3tA9wabNUdM1hjlwizYrlsmQ3w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4847

On 09.09.2021 07:22, Oleksandr Andrushchenko wrote:
>=20
> On 08.09.21 18:00, Jan Beulich wrote:
>> On 08.09.2021 16:31, Oleksandr Andrushchenko wrote:
>>> On 06.09.21 17:47, Jan Beulich wrote:
>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> Instead of handling a single range set, that contains all the memory
>>>>> regions of all the BARs and ROM, have them per BAR.
>>>> Without looking at how you carry out this change - this look wrong (as
>>>> in: wasteful) to me. Despite ...
>>>>
>>>>> This is in preparation of making non-identity mappings in p2m for the
>>>>> MMIOs/ROM.
>>>> ... the need for this, every individual BAR is still contiguous in bot=
h
>>>> host and guest address spaces, so can be represented as a single
>>>> (start,end) tuple (or a pair thereof, to account for both host and gue=
st
>>>> values). No need to use a rangeset for this.
>>> First of all this change is in preparation for non-identity mappings,
>> I'm afraid I continue to not see how this matters in the discussion at
>> hand. I'm fully aware that this is the goal.
>>
>>> e.g. currently we collect all the memory ranges which require mappings
>>> into a single range set, then we cut off MSI-X regions and then use ran=
ge set
>>> functionality to call a callback for every memory range left after MSI-=
X.
>>> This works perfectly fine for 1:1 mappings, e.g. what we have as the ra=
nge
>>> set's starting address is what we want to be mapped/unmapped.
>>> Why range sets? Because they allow partial mappings, e.g. you can map p=
art of
>>> the range and return back and continue from where you stopped. And if I
>>> understand that correctly that was the initial intention of introducing=
 range sets here.
>>>
>>> For non-identity mappings this becomes not that easy. Each individual B=
AR may be
>>> mapped differently according to what guest OS has programmed as bar->gu=
est_addr
>>> (guest view of the BAR start).
>> I don't see how the rangeset helps here. You have a guest and a host pai=
r
>> of values for every BAR. Pages with e.g. the MSI-X table may not be mapp=
ed
>> to their host counterpart address, yes, but you need to special cases
>> these anyway: Accesses to them need to be handled. Hence I'm having a ha=
rd
>> time seeing how a per-BAR rangeset (which will cover at most three disti=
nct
>> ranges afaict, which is way too little for this kind of data organizatio=
n
>> imo) can gain you all this much.
>>
>> Overall the 6 BARs of a device will cover up to 8 non-adjacent ranges. I=
OW
>> the majority (4 or more) of the rangesets will indeed merely represent a
>> plain (start,end) pair (or be entirely empty).
> First of all, let me explain why I decided to move to per-BAR
> range sets.
> Before this change all the MMIO regions and MSI-X holes were
> accounted by a single range set, e.g. we go over all BARs and
> add MMIOs and then subtract MSI-X from there. When it comes to
> mapping/unmapping we have an assumtion that the starting address of
> each element in the range set is equal to map/unmap address, e.g.
> we have identity mapping. Please note, that the range set accepts
> a single private data parameter which is enough to hold all
> required data about the pdev in common, but there is no way to provide
> any per-BAR data.
>=20
> Now, that we want non-identity mappings, we can no longer assume
> that starting address =3D=3D mapping address and we need to provide
> additional information on how to map and which is now per-BAR.
> This is why I decided to use per-BAR range sets.
>=20
> One of the solutions may be that we form an additional list of
> structures in a form (I ommit some of the fields):
> struct non_identity {
>  =C2=A0=C2=A0=C2=A0 unsigned long start_mfn;
>  =C2=A0=C2=A0=C2=A0 unsigned long start_gfn;
>  =C2=A0=C2=A0=C2=A0 unsigned long size;
> };
> So this way when the range set gets processed we go over the list
> and find out the corresponding list's element which describes the
> range set entry being processed (s, e, data):
>=20
> static int map_range(unsigned long s, unsigned long e, void *data,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long *c)
> {
> [snip]
>  =C2=A0=C2=A0=C2=A0 go over the list elements
>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 if ( list->start_mfn =3D=3D s )
>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 found, can use =
list->start_gfn for mapping
> [snip]
> }
> This has some complications as map_range may be called multiple times
> for the same range: if {unmap|map}_mmio_regions was not able to complete
> the operation it returns the number of pages it was able to process:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rc =3D map->map ? map_mmio_re=
gions(map->d, start_gfn,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 size, _mfn(s))
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : unmap_mmio_regions=
(map->d, start_gfn,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size, _mfn(s));
> In this case we need to update the list item:
>  =C2=A0=C2=A0=C2=A0 list->start_mfn +=3D rc;
>  =C2=A0=C2=A0=C2=A0 list->start_gfn +=3D rc;
>  =C2=A0=C2=A0=C2=A0 list->size -=3D rc;
> and if all the pages of the range were processed delete the list entry.
>=20
> With respect of creating the list everything also not so complicated:
> while processing each BAR create a list entry and fill it with mfn, gfn
> and size. Then, if MSI-X region is present within this BAR, break the
> list item into multiple ones with respect to the holes, for example:
>=20
> MMIO 0 list item
> MSI-X hole 0
> MMIO 1 list item
> MSI-X hole 1
>=20
> Here instead of a single BAR description we now have 2 list elements
> describing the BAR without MSI-X regions.
>=20
> All the above still relies on a single range set per pdev as it is in the
> original code. We can go this route if we agree this is more acceptable
> than the range sets per BAR

I guess I am now even more confused: I can't spot any "rangeset per pdev"
either. The rangeset I see being used doesn't get associated with anything
that's device-related; it gets accumulated as a transient data structure,
but _all_ devices owned by a domain influence its final content.

If you associate rangesets with either a device or a BAR, I'm failing to
see how you'd deal with multiple BARs living in the same page (see also
below).

Considering that a rangeset really is a compressed representation of a
bitmap, I wonder whether this data structure is suitable at all for what
you want to express. You have two pieces of information to carry / manage,
after all: Which ranges need mapping, and what their GFN <-> MFN
relationship is. Maybe the latter needs expressing differently in the
first place? And then in a way that's ensuring by its organization that
no conflicting GFN <-> MFN mappings will be possible? Isn't this
precisely what is already getting recorded in the P2M?

I'm also curious what your plan is to deal with BARs overlapping in MFN
space: In such a case, the guest cannot independently change the GFNs of
any of the involved BARs. (Same the other way around: overlaps in GFN
space are only permitted when the same overlap exists in MFN space.) Are
you excluding (forbidding) this case? If so, did I miss you saying so
somewhere? Yet if no overlaps are allowed in the first place, what
modify_bars() does would be far more complicated than necessary in the
DomU case, so it may be worthwhile considering to deviate more from how
Dom0 gets taken care of. In the end a guest writing a BAR is merely a
request to change its P2M. That's very different from Dom0 writing a BAR,
which means the physical BAR also changes, and hence the P2M changes in
quite different a way.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:27:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:27:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182749.330488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOk-0008Hz-Ri; Thu, 09 Sep 2021 08:27:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182749.330488; Thu, 09 Sep 2021 08:27:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOk-0008Hs-Nx; Thu, 09 Sep 2021 08:27:06 +0000
Received: by outflank-mailman (input) for mailman id 182749;
 Thu, 09 Sep 2021 08:27:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOFOj-0008Hm-Ih
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:27:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b6359178-1147-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:27:04 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2050.outbound.protection.outlook.com [104.47.9.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-4kHK6nxZMYq4p-MIH3RgNQ-1; Thu, 09 Sep 2021 10:27:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 08:27:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 08:27:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM9P193CA0023.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:21e::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Thu, 9 Sep 2021 08:27:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6359178-1147-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631176023;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XbfbMu4AIYc02eVv1Yja5SXVJWTZIYj33ovk8QlUmAQ=;
	b=l8LM+9ASYUA4rVMAZNSYDW7bwc26RQhR8yiAVO6YYx402/vkt9Bj9X1bMak6GWoKKOatDW
	gJkTOmUj7OtSx8iu8cBc2AsebHIBVYYNZjqkcdcNDq0+wqHEQvqMWbt4ZIxcW6uBUF5axb
	H/FH3o+r2kwqGy6zLd4VP+yL5dMM4ro=
X-MC-Unique: 4kHK6nxZMYq4p-MIH3RgNQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RKe/xMwgeBuP3DkXwGX/vys0yJksINQWcHdm/M/B/geQHF4NHE3P9AXf2F6tU6ML/UOHg9H5Aty4OtX0P0QHGzA6mWXPSWYWQ7NJHTwViR8JUWsj9CROy9JKYME0N2HzBVWkpfbSPY1Gsw/9oPza3E9NiT0X3X9w7RTKNWfp2ozMt/tla29XaNZy/HRjcjI6/L+AHlF2cFD3vnR0XF0n5dWEEtBDkhJo/dXro63EmKbx9VL288FukXkbKLBsD2UrYKwfAmwhSlyWVVloxGdtJA995tOxFHFBAF1EcgOHzNdn85MorXDter+lWgFJITPFd/eT0dxHNMx1BGzoNc3fdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XbfbMu4AIYc02eVv1Yja5SXVJWTZIYj33ovk8QlUmAQ=;
 b=OSWUzDzK9XPc7vTUnEJpMr4mHGdHQkFdZnDFCTdmf4wEh0s7lVIxPxi3LqRhnp0fD/br7PRKkzcuv4sxtD0xuINJFyXf1P4mSf8FCz9AwlQwZ5S3wq6++lotzVu6yMyvEEhNE80W9Lt65X4dsAfIN3911JzxckmM8Pfa9FHiXlyowdwYXtw5Rgrag25KKJ1PIGYJUnnzc1cHzdrkJHsuaRIUIi/0rVI78UATMuEwtQeaQzrWCjNaqjs7v+eAFJVUmRYkWIpONZeSQTONWrecSiyEvbQsc3Wgdt5z9+rOVQIp/MZGgp8iEd8yRyg22Qf0+fZ0p/b/JTr/rib1b2lxxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
 <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
 <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ee9a69f1-6b47-a944-7255-4ffaa3e1c601@suse.com>
Date: Thu, 9 Sep 2021 10:26:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM9P193CA0023.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:20b:21e::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d6066d8-7b46-4b24-c31a-08d9736b98d4
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233479609F72980453625B35B3D59@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eKcOEkxeu0OQzvZaXXsdLEDN1wmx0nevxJXeX4LKWErxXHHdH0sz9Oscf1aMRAd7Jo29rZ9KB1lvqwg+J8FvmSwXb4cLk67nhE4vrnFSDj3FOPGTjbcdU4FZHAw/WnmeOizsofTkPx2Zyh3dbaZI2j/VfQcOFY7XI1/kTjX6ZX/WpxpLaD+5idWrQFlwXB1iZrNvDDRcTj6cmDx+lwjYebzdUtK4OvjA9tjI4sOVxWQhHfguru9n3Z1b/QvQKE555S+4TWr0MvsbIqFgiW8oI8nXpnRe/u7xxOLUMhiNmw+FMOSpiTCDYSWH2zqqF54wGUoBghAfOKdqEePHDzJtBSsMax+CQJe/OO7j6om0HdxlrPouKaVKvcPpuMn1wme6knmn6gGNERVLkMY7XstTInE6aw5DlrUPB2j5z+63OUAy0uiz7I0X8Chkr4hLpRlXGofJArGizSBydXnIdPIFIxFMkAVwMPgttncoNN39EmQvcihU0Llv0ztRLImmZp1PetStgHxdSMaJCzq5WDfzEO1+zOlsfPuaYadErM35vgaq8zoFmFgAOqNbznJp+3HXvqzEPksJsrKBQCw9VxInrqj4CV0JL69BMQCKI/TBx2MEcY+HG+N5Pt4Zz+ZbBRAvxv2gUoDtW+miS9A5sMQ/oobUNABOKvYm/5vP0kr6H4bA+LlU9M+nz+XHylGGEDM228tOgZ5VNc+DwFp0AZoR5Cx+hoFhMOskNpoSUyC1AEHejd7A72u3newqFnrDOKwM
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(366004)(39860400002)(136003)(376002)(36756003)(86362001)(316002)(6486002)(16576012)(110136005)(4326008)(53546011)(54906003)(2906002)(66476007)(478600001)(66556008)(83380400001)(66946007)(7416002)(26005)(186003)(31696002)(8676002)(31686004)(8936002)(2616005)(956004)(38100700002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WmZoVjFsK01NdDFJcXdoRUg0NHIrQUdPdzhKZnY3N1lFa0E4SnpwNEZleXlS?=
 =?utf-8?B?WmlqeFNJZVZnL09vdm9iaks5eS8zOE5Jd2dYbHZ1MXdsdjR0aGNtTTJOTWZY?=
 =?utf-8?B?K2NEZWhMNHQ0cmR6dE1yeFRUNXFJc3Z3V1M1NWM4QzhOMEU4UC9za2tuZGpq?=
 =?utf-8?B?RmFwdFc0cWlPVHVHbVRsN0t5TVpDUVBtTDM2UDVXTFFHN0Y5Z1c0NHFBTnha?=
 =?utf-8?B?WXk4U1hMbzdnTFJ4UW9Jdk50dG52cStPbjl1OEFzSjRCOWtYNUtFRUFIS254?=
 =?utf-8?B?M2E2NEhNVjFVL0c3QmNQTGM5bTFsaUt6NWx0UjhocEdIdWJ5NmllUVJteVBm?=
 =?utf-8?B?bjFNR2VsSkErc2RIZERJSkdLVTBzSHYzWDRNUnhmVncrTHEyYlMrQy9kMDdV?=
 =?utf-8?B?SjJybDZVYnNNNTdHa082OHNiemtSMkNTSkIrUUxsZGg3b3ZHNEE5ejQ1Z25M?=
 =?utf-8?B?cXlCbEM2SWNqcWtQc1FvdzhzZU9saVhYR3dNcHJ6aG9tRzAzdzNOZFd2NSsy?=
 =?utf-8?B?bi9LTGJUdE1CVm1IZUdFcXFMOGJmQ0FtSGIxemxKNzgvNUtjUW16Rjl2clhp?=
 =?utf-8?B?elUzU0JFdXVzanZhckk2blRVWXpjbWZBWGxReXhhQmM3di94c2tvSm1QaSti?=
 =?utf-8?B?MUtiRUljVHlhZG9yS3FmQ0NGY3dZN2F0UkZuVktic0dMTTZQL0xVaTd5RG04?=
 =?utf-8?B?OEpveWxhblZ6eVFjbDFJQVhnZVNKSFZHRGM0UFpRWSt5NEVoSVB1Y1oySXFG?=
 =?utf-8?B?NEdWK1F3SWI3VmpDNjV5aGFnQWhuVjZKRGVvbkk4UUlyZ1haZytBdm9BeFcx?=
 =?utf-8?B?M0Z4ZThycVZyUWJyZmcvOXJSQjNlbGhzb1FaRVI5TmNnQmN1SG9nV1Z4U3Fv?=
 =?utf-8?B?REtRSDFaM1IzM3JMKzgwMS8xWWZNVmVTMDBGYjJNeTRZSVhhOGdQcml0bzAr?=
 =?utf-8?B?enJObHJkdHZYSnl5R1FhekpFeU9XWUphVEl0djBxQW1MUHZGcFhSaGlscDN2?=
 =?utf-8?B?dVJwUXEvOC9nZVpGOWJXd0dHeUFCVTRxMEtNVFRKb3RSZFU4L3Z5NTNnYjgz?=
 =?utf-8?B?WWl4MGdrZUVHcUpCaDVxQmJKc3JlbFdjTEFWTm5DRHdhN01vemY2N29LOXRr?=
 =?utf-8?B?YVZ4blZEM0RNK0pIUlBvUGorRU9ZdHNLVjdGNVllWU5RazZnZFFsM1M0aW1h?=
 =?utf-8?B?Qi9BU2tyS2NmaU9UQlZaUlVQQkhtbjBtaTExUUt2b29jaDJTMU5vb2pMZlFR?=
 =?utf-8?B?dkw5WU5HR2QyNmRrUzVzbDdIQlJUcWtQaU1wMmhURS9kNHROd0plMkwwdUNM?=
 =?utf-8?B?SHQ1SHlXYmNMcFdpOXRHSmF0ZWVZNlVCcnFvM0lUUnc1N3UvdFJmVXBvczRC?=
 =?utf-8?B?QXhkTm84MUVuYzdBL0lWSlhRTSttWlhwWFZhb09GSm9CQjJnYjZKTmd4L3JL?=
 =?utf-8?B?UDN5YTBUSkpTMmVUTEc5aVVnR1VtMW5qUDNSeU5LU0FobTB3VTBmODBhNmhn?=
 =?utf-8?B?YjJSSzhiVHdGQmMrT0pjSWxXZTZIR1I5VkdETUNUckVvZ3d2RktpRjZDb0Q5?=
 =?utf-8?B?c0IwaGkweHo5SlJXWmR0MVFhRFMwUXg3emh2eW5qQmg4UnlncDZKNTEzVVVU?=
 =?utf-8?B?aUd3cG4yalFRTSs3Z0E0eDBGWldybGNWZm1OUkFBbm1xbFZ4THkvb0IrSks3?=
 =?utf-8?B?VExsRkNmTFUvSG9Sc216OC9JMkREMTJJOHBnZjNINktEWTFlOUx4U1lDRmdX?=
 =?utf-8?Q?uLwLHbelrY7sxMC2Q5FYl6WeXE1Gixay6aujkuN?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d6066d8-7b46-4b24-c31a-08d9736b98d4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 08:27:01.3670
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: v+cabYR0ShczcxyglfXr4ptQ/4EuNrwsupn8dQaeFGkoFWQHtdfcfmqQPrqWFGe8G3wVDO8JBZUQaiPDmhChug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

On 09.09.2021 08:13, Oleksandr Andrushchenko wrote:
> 
> On 06.09.21 17:51, Jan Beulich wrote:
>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>> @@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>>>                        unsigned long *c)
>>>   {
>>>       const struct map_data *map = data;
>>> +    gfn_t start_gfn;
>>>       int rc;
>>>   
>>>       for ( ; ; )
>>>       {
>>>           unsigned long size = e - s + 1;
>>>   
>>> +        /*
>>> +         * Any BAR may have holes in its memory we want to map, e.g.
>>> +         * we don't want to map MSI regions which may be a part of that BAR,
>>> +         * e.g. when a single BAR is used for both MMIO and MSI.
>>> +         * In this case MSI regions are subtracted from the mapping, but
>>> +         * map->start_gfn still points to the very beginning of the BAR.
>>> +         * So if there is a hole present then we need to adjust start_gfn
>>> +         * to reflect the fact of that substraction.
>>> +         */
>>> +        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
>> I may be missing something, but don't you need to adjust "size" then
>> as well?
> 
> No, as range sets get consumed we have e and s updated accordingly,
> so each time size represents the right value.

It feels like something's wrong with the rangeset construction then:
Either it represents _all_ holes (including degenerate ones at the
start of end of a range), or none of them.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:27:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182751.330499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOz-0000JR-7n; Thu, 09 Sep 2021 08:27:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182751.330499; Thu, 09 Sep 2021 08:27:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOz-0000IY-4J; Thu, 09 Sep 2021 08:27:21 +0000
Received: by outflank-mailman (input) for mailman id 182751;
 Thu, 09 Sep 2021 08:27:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOFOx-0000Fc-9b
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:27:19 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bd7e9cea-1147-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:27:16 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id D9C8A1FDD8;
 Thu,  9 Sep 2021 08:27:13 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 7F3841365E;
 Thu,  9 Sep 2021 08:27:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 19N2HWHFOWHPEAAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 08:27:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd7e9cea-1147-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631176033; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=m6etc4M/6aJ6ZeXC29srC0CnM+jaYzvg6C/U2YvOZrg=;
	b=bdK7ODzHl98S6s38rnRPwJHhUVB3WTcGkzKvLhhHtB8ZIN1FQvDA1uAOXpRrtKyACQIwjN
	mRNMjX2HN8TSrCAbgHiUiRfTj/BgiSqFHONiAg5FtovP9vwyQ4bOSrF8U084xCnikUNRXe
	khXJCYkHKmJUEidnM7ervUdDl3IHaWU=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: [PATCH 0/2] stubdom: disable building of pv-grub per default
Date: Thu,  9 Sep 2021 10:27:03 +0200
Message-Id: <20210909082705.9434-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is a first step of deprecating pv-grub. Switch the default to not
building it.

NOTE: This should be mentioned in the release notes!

Juergen Gross (2):
  stubdom: fix build with disabled pv-grub
  stubdom: disable building pv-grub

 Makefile             |  4 ++--
 stubdom/Makefile     | 13 +++++++++++++
 stubdom/configure    |  8 ++------
 stubdom/configure.ac |  2 +-
 4 files changed, 18 insertions(+), 9 deletions(-)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:27:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182752.330504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOz-0000Lx-Jt; Thu, 09 Sep 2021 08:27:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182752.330504; Thu, 09 Sep 2021 08:27:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFOz-0000LA-CO; Thu, 09 Sep 2021 08:27:21 +0000
Received: by outflank-mailman (input) for mailman id 182752;
 Thu, 09 Sep 2021 08:27:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOFOx-0000Fc-TD
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:27:19 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bd7e9ceb-1147-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:27:17 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7AE3722323;
 Thu,  9 Sep 2021 08:27:15 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 44BED13AB4;
 Thu,  9 Sep 2021 08:27:14 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id SGNPD2LFOWHPEAAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 08:27:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd7e9ceb-1147-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631176035; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hKgds8sS3qW4uVOZtuYU/xKRL4VfjAB9MmtS4lgtd1s=;
	b=pBaQHqm4l76QobGJBt0AG1Bqj3JJHypVnlW2cOHLAXSLKwOQvbFoxR7mxYiZkfIxZA1tpg
	2j7VmjQgJKONoDWVziTvopkWYNOitkSyqmxZi9OlPTtwxOQ6tR5MhrrFN5zrFuo+h/aT0d
	KWYOka5qw6xRuPV7CVLOxT/s2AGynqw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 2/2] stubdom: disable building pv-grub
Date: Thu,  9 Sep 2021 10:27:05 +0200
Message-Id: <20210909082705.9434-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210909082705.9434-1-jgross@suse.com>
References: <20210909082705.9434-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The stubdom based pv-grub is using a very outdated version of grub
(0.97) and should not be used any longer. Mainline grub has support for
PV guests for a long time now, so that should be used as a boot loader
of a PV domain.

So disable building pv-grub per default. In case someone really wants
to continue using it he/she can still use a pv-grub binary from an older
Xen version or manually enable building it via:

  configure --enable-pv-grub

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 stubdom/configure    | 8 ++------
 stubdom/configure.ac | 2 +-
 2 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/stubdom/configure b/stubdom/configure
index aa48df986d..df31532abb 100755
--- a/stubdom/configure
+++ b/stubdom/configure
@@ -1342,7 +1342,7 @@ Optional Features:
   --enable-ioemu-stubdom  Build and install ioemu-stubdom
   --enable-c-stubdom      Build and install c-stubdom (default is DISABLED)
   --enable-caml-stubdom   Build and install caml-stubdom (default is DISABLED)
-  --disable-pv-grub       Build and install pv-grub (default is ENABLED)
+  --enable-pv-grub        Build and install pv-grub (default is DISABLED)
   --disable-xenstore-stubdom
                           Build and install xenstore-stubdom (default is
                           ENABLED)
@@ -2129,11 +2129,7 @@ fi
 else
 
 
-grub=y
-STUBDOM_TARGETS="$STUBDOM_TARGETS grub"
-STUBDOM_BUILD="$STUBDOM_BUILD pv-grub"
-STUBDOM_INSTALL="$STUBDOM_INSTALL install-grub"
-STUBDOM_UNINSTALL="$STUBDOM_UNINSTALL install-grub"
+grub=n
 
 
 fi
diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index bd6f765929..a07a1edae5 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -21,7 +21,7 @@ m4_include([../m4/fetcher.m4])
 AX_STUBDOM_CONDITIONAL([ioemu-stubdom], [ioemu])
 AX_STUBDOM_DEFAULT_DISABLE([c-stubdom], [c])
 AX_STUBDOM_DEFAULT_DISABLE([caml-stubdom], [caml])
-AX_STUBDOM_DEFAULT_ENABLE([pv-grub], [grub])
+AX_STUBDOM_DEFAULT_DISABLE([pv-grub], [grub])
 AX_STUBDOM_DEFAULT_ENABLE([xenstore-stubdom], [xenstore])
 AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
 AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:27:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:27:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182753.330521 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFP1-0000vV-Qs; Thu, 09 Sep 2021 08:27:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182753.330521; Thu, 09 Sep 2021 08:27:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFP1-0000vI-Mt; Thu, 09 Sep 2021 08:27:23 +0000
Received: by outflank-mailman (input) for mailman id 182753;
 Thu, 09 Sep 2021 08:27:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOFP0-0000r2-Ra
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:27:22 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 777175a9-2056-4fba-8ab6-2f31ab8818d8;
 Thu, 09 Sep 2021 08:27:22 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3E2D722326;
 Thu,  9 Sep 2021 08:27:18 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id E1D861365E;
 Thu,  9 Sep 2021 08:27:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id kMS7NWHFOWHPEAAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 08:27:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 777175a9-2056-4fba-8ab6-2f31ab8818d8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631176038; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EGsf4GAR2GL00EaPTSPTi/UURLGcI6rfW30asCxzTy0=;
	b=sLdwAApthP3myFJPzarWHeUIkkgefvZlSErS1ox+2gBFb9+P04jJ7THt5EXeJz85zbnXIi
	Y4KhhfrrDr88FPeY1mmCcWFz2gW7kshFC+e2qTXpRzrQVxxm/OXl8QFs33jbpu/SyhNXVs
	FT/Q0HaHFW9/pxEKtEaWQnoz7yVy7Yc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: [PATCH 1/2] stubdom: fix build with disabled pv-grub
Date: Thu,  9 Sep 2021 10:27:04 +0200
Message-Id: <20210909082705.9434-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210909082705.9434-1-jgross@suse.com>
References: <20210909082705.9434-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Today the build will fail if --disable-pv-grub as a parameter of
configure, as the main Makefile will unconditionally try to build a
32-bit pv-grub stubdom.

Fix that by introducing a pv-grub32 target in stubdom/Makefile taking
care of this situation.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 Makefile         |  4 ++--
 stubdom/Makefile | 13 +++++++++++++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 96d32cfd50..5b5cef3e49 100644
--- a/Makefile
+++ b/Makefile
@@ -72,7 +72,7 @@ build-tools-oxenstored: build-tools-public-headers
 build-stubdom: mini-os-dir build-tools-public-headers
 	$(MAKE) -C stubdom build
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub32
 endif
 
 .PHONY: build-docs
@@ -143,7 +143,7 @@ install-tools: install-tools-public-headers
 install-stubdom: mini-os-dir install-tools
 	$(MAKE) -C stubdom install
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub32
 endif
 
 .PHONY: tools/firmware/seabios-dir-force-update
diff --git a/stubdom/Makefile b/stubdom/Makefile
index 06aa69d8bc..b339ae701c 100644
--- a/stubdom/Makefile
+++ b/stubdom/Makefile
@@ -531,6 +531,13 @@ vtpmmgr-stubdom: mini-os-$(XEN_TARGET_ARCH)-vtpmmgr vtpmmgr
 pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxenguest grub
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
 
+.PHONY: pv-grub32
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+pv-grub32: pv-grub
+else
+pv-grub32:
+endif
+
 .PHONY: xenstore-stubdom
 xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxenguest xenstore
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
@@ -560,6 +567,12 @@ install-grub: pv-grub
 	$(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)"
 	$(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz"
 
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+install-grub32: install-grub
+else
+install-grub32:
+endif
+
 install-c: c-stubdom
 
 install-caml: caml-stubdom
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:40:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:40:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182776.330532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFbK-0004GB-Vk; Thu, 09 Sep 2021 08:40:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182776.330532; Thu, 09 Sep 2021 08:40:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFbK-0004G4-SV; Thu, 09 Sep 2021 08:40:06 +0000
Received: by outflank-mailman (input) for mailman id 182776;
 Thu, 09 Sep 2021 08:40:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOFbI-00047y-Ry
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:40:05 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 862f073c-1149-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:40:02 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1898cb9u007312; 
 Thu, 9 Sep 2021 08:39:59 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2054.outbound.protection.outlook.com [104.47.12.54])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayek8g6qw-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 08:39:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4755.eurprd03.prod.outlook.com (2603:10a6:208:c2::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 08:39:56 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 08:39:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 862f073c-1149-11ec-b1a7-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Mrh8NkgqkSYE4M4TBb0KfeRb9n7co1/dyL8GBIjiX8BWJxX/kOH/4bDWzW6Jq/xTiXUUbckE9ZZ+Q47CXqA2XVoBvTwRX812VUmz6zp9Rp9Dfnsq28V4dtoiRI6oWfE1ICa/ESoSBDXpOhzHQT1FiqWvnBpkAShk+LB5hNcD8STWQ8qQvwtCq0cJrWH9zNA3XjNgVhVzzDa5SRIbfUHnZAQJLdAm5o3m30d4G9djFz9jO/1V5RguaQpgyPwDQb29iMAuCarlVFALIpp0l6IkZTrz99FnVQ1j/qmnXRYsHGEzFVXz5F/fZo70LLFj6vFeKkALBlCAsnpawMIb7LYnaQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=W8Jhqkd6vtBgYYxy7gzk21vNw27Kmce0dz8EbMBVHds=;
 b=MJfb3K4D37iVwOAalWZ2CJbQh7ePcozTDjnqRg9Vm6HbCIJ+3+GQQi5jMZRNZWQJwLmojn+G/Oe4W8syTVt9CYFIvtH3BCiLnVgRlo1/o9go/RgKI0+IXd5EVmcQAyclfTNwU1CWd9H1EoeenDDtoh9QpAzoeoDkQ8FwIuqmnAchZyv0GoYLjEZgsHzNg9yQN1S3FAHxOr5xFaTF2DY6LRWVdx+4aw19PlJk5oav1iaslaIj0OYdDc3Dw19uUCTsR/VjjNL81RB4moe4diW25kkxMwRugJMkpwqAyvXpugUVR4FY/wCaTrQOSjwF55q4DziX1yWwNPFRNDzg+wgrOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=W8Jhqkd6vtBgYYxy7gzk21vNw27Kmce0dz8EbMBVHds=;
 b=C0fmE6fuDxU378Qwzyci3NT7W9jCWrq3VnooDKmuLZxkSXA1ncKPqe6a+5CJqEYtMdLwQDLdshTCO5fp2ODABmHKeXd6i0Fh3eNv6NZL5oDCh5aBzUvrL8/akyuGtOQcP4opC0RFUSmuyh1vKrB+GJfaFPPH8C6rSCH3UroxVgAULCKHTdfFvmJ0HXsuZzFTwFd9ecpvRVWqIdjjCuuGjSboIs0EMoeoZ6+AQDjkdjHUeMiIphM8+/5kZVIdDcK034RDRm/wnF6ZfL51afaYpsbSzGE6Vm1Sa+y9KHbvHxOtelf6hUg5pLPOi/xOuqhqNyiBzeu4tbjWT69WQx3kcA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuA
Date: Thu, 9 Sep 2021 08:39:55 +0000
Message-ID: <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
In-Reply-To: <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ae7c8e1c-5260-4133-487e-08d9736d669a
x-ms-traffictypediagnostic: AM0PR03MB4755:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4755F698577669BD9580B290E7D59@AM0PR03MB4755.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 hYqSZ4tqEcNuNHB5ktpkdsL8iAV06sagZA2wSn5V93SgmlAdmpQlMJGfgo7YdRtuo2xkh3NrZDjnSuh8rvNEjO6Re11rhf94cXqjMXIoA7D5a2s9P9+KQqjqv+CanLIZEx+wu7Tafa7hJ5B5kfGBKXIh+PuqXeq0it89YQtea4zB5pON03oAGstRILFvSRopK5DCPjXW9yc6WRrESUCgrN18vgJ6IyMnUNHuohNyeCqskiP++VdeZ3RSRCxA1RkWQCeNlECK+DKMDbGUJ5+0FFyKX2aXXI3TuftOZjBlwaP+TqrL4FBUowkCk+KvUIe2iTwHK7/jasofbfd76qceZB6XQ6oWafnspfzsPy/eOaOZ6KUQ+XAXJZtD+y8PnXp294Qp6G5Ay+HzHT3LADNvO/XWuDhb+3UHowHIsKBPIQVTkSBEAdCAqYia9lLOgUDoczhqnlH5+8y2WvquGFLt4ZgkNvlUcjeiuOYf2GWn+CsLvc/c0Z0wmzeRN1FWmU7IQJVbLOxY1HPNBnw4wE96Crt7fdvXiGbmLc5zFUupj1k4XLrZNiJkcxE8pFIJUHOaAcOBmpb3gIys3arn7JhQqgwWrjmJ7JA6XMJO3Av3f65K2n92YNJnU3UtuGrMTverFob1CcEAvDyJ5PaDyeQfizLDGi0LLhrs5n3rN9aec3V/QJrcpiIPCUGQfdKkfkeExJL/EUql3Q1/h3YRgzzBFCl/xGbvOGvH5qBAQbX0FwPhuUX6F/qvoJHp7TqSL9Aao1LXGQaCFBs8M7Wch4yc3BxsoJihyFaQbB8vObtRj4Od301Vyx/vEhqHYiYbG1mJgsAI+b9Qla6eBJxt6NjKyKHk5JlSxppeeOsqFfDmwtE=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(136003)(396003)(39860400002)(376002)(6512007)(71200400001)(54906003)(86362001)(6486002)(26005)(76116006)(53546011)(6506007)(66946007)(91956017)(8936002)(8676002)(2616005)(64756008)(66476007)(66446008)(66556008)(38070700005)(478600001)(5660300002)(2906002)(186003)(83380400001)(36756003)(31686004)(31696002)(122000001)(316002)(966005)(110136005)(4326008)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NU1SN2hHUkpGT2UyUlFuVE1pM1BvaWVnZXFVazQyVUlIMENibU1PU3p2Zk9X?=
 =?utf-8?B?eUsyNmZyL3UybHo5RGkwQVBxRjRWUURaUTQzQkRJQ2dabVBqT3pRcndVRXRj?=
 =?utf-8?B?VFZ0QVIxSkpNYjRHMUdES0hWRS9IblEwekF0c1lUTit2NHh3MGNRM1FvZmha?=
 =?utf-8?B?TlJKQS9td2lIaStERkF6eHZXdEFUL09VVXRzU1JPY1FiN3l0MjFGS1RiYVBj?=
 =?utf-8?B?RUthZ0JwMmUxRkRsd09SaWQrbXYyeUlzUTJMQlEwbEJQb0thY3pqMFJqR2lv?=
 =?utf-8?B?TmtYVkd3azYyMFc4cGUwU2xYUHR1MFFKUFhIdDV5MWtHdnNVR2g3ODZGK3Zq?=
 =?utf-8?B?dVdsSk4zM2hDSXV6MUN6ZDZNUVVjT1JVRisxckx3aCttY1pmK2V4VEd1ZC9s?=
 =?utf-8?B?dHlKTUF5SzdXeE9OalFxeHBTeXhlVnZvcFVrM294a1FXL0NFMmtVUG1hYzdR?=
 =?utf-8?B?YVNUN0dqT3dYWnZoZE9kdU1oQXRmbmVVL0hqenhXYmJFbHJqR0xIWkVlcnVR?=
 =?utf-8?B?WUN1VUxWZ0ZzTTMwejYwR29XS1RnbmQ5QmhwQklITEc2SWYvbTRPYkx0OHhx?=
 =?utf-8?B?TnYzOVJRREN4bU1BNEttNFBlQXdTUVJXNzNpRHplczNrdmU0YmZzcXZoWk1K?=
 =?utf-8?B?YzIzTGNkNnIrQlIra1gvMzNlcXJNR2tVeUJ4c1VzVFR2S1c1Z1JXdVcwcVZh?=
 =?utf-8?B?TVFuaVpndXlSSWN4Z2xNa0EvNTFwSkVpRFUrYXZwNHJpMEdtaW1WdytGZUtQ?=
 =?utf-8?B?eTZWWWxnOGo2OTRQYjFuMUFENVJKMk9xd1JobVNRSHJTT1VvRGhaekI4TlE4?=
 =?utf-8?B?ZzViRlJTVTd6amdOWXZmQkdOeWdvdUJ3a1lsY1EydjJzTlZKampXOWJPdzl3?=
 =?utf-8?B?cXJEMVBxK1VtS2MrRDdXRXZDeGNZNVRxTUZEdWZCR3NYRnFPMmwzOXF4Yloz?=
 =?utf-8?B?TkNleEYxalg3NmgvVHJIaGF6ZnM2dlBncmtuOExVTGo2S21YNjMzYUlrTXRm?=
 =?utf-8?B?dUpxZWk0U05ZSEtXS3k4ZUhsZEt2YVdoNE9zejgwVTVwVXV2TW5ieVArc2lX?=
 =?utf-8?B?YWtZRGRKRzFkUjBvaGZpajVZTEdvVDE0ZVNzWmd5UmV3ZjlHRUxoamhuMjJw?=
 =?utf-8?B?ZzEwN3JTS3labnJPTHRSczlXZzFmeGFWU3lKdDFNTE9rQ2NoWnZ6djJzZ2R2?=
 =?utf-8?B?eTJla0lHMDhXZGR3b05SdElTbmJUZCs5NlJ5alRnSHVlbysrWkQyNkhEL3hm?=
 =?utf-8?B?eEplQXVSRzVpWDNKSURZSGlWazE4WWkxNUVnTDlHcXRlRHJkbkRPT1MrejE4?=
 =?utf-8?B?Q0d1WjlDL1FsUnJlVkZ5R1EvbVBXTVdNTFZTQzU4L3FYN3lIZlhZcUFMRDBa?=
 =?utf-8?B?eTl1aHlIUTBZU0ZoU2haRXk4VjhPYkR6YTg1ekhqTEE4ZU4vK25lNWJ2SStp?=
 =?utf-8?B?TGRQcjV4d2ZYYkpJQ0dtc2FGR3Q3d1BKUmxMbmZwKyt6YkJVeHowMjU5WElx?=
 =?utf-8?B?enk4czd2YjZMR1hhWUlIcWlLa1NxdVYweEJhVWJPTjIvd2dNRDRSOUVXT3U5?=
 =?utf-8?B?VGhDNTJtRDJsL1JDV1FZMDVZSlRxTHJ5cTNpcEV5L2NONklqTnpZa21IdXZM?=
 =?utf-8?B?eHFQU2RhZURDWDdpVnRHWXdadE9IMDBETWZSUlZhMXAxU0pCM2JQeTA2VnFZ?=
 =?utf-8?B?MjljTnlOcFVBbGtiQ1hZNncwRTJ5M3RqNHBaOXhnNkY2bXZna1BoKzV5RFJp?=
 =?utf-8?Q?Lpvj3Pm/LtUoO9i8JuR1/4iLByDUxytz5kMZZQh?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <88AE7DBE0D19CB47ADBED3F487FB2254@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae7c8e1c-5260-4133-487e-08d9736d669a
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 08:39:55.8278
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6ACHRFWyrZ7s6UEhlVJbgkbhv+XBIdGoVGuPR5zoDKP9UHjFXySvE+1hLqL0yHntzKINYqGIg9Ngu2UUXC0DKr0R5hS+AhotDAUUN+vIflpg4ZqAkrLDEZC1MlIo5orc
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4755
X-Proofpoint-ORIG-GUID: jclkPhnSXVKAu-xqI44l551K8YNViC8f
X-Proofpoint-GUID: jclkPhnSXVKAu-xqI44l551K8YNViC8f
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_02,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0
 impostorscore=0 priorityscore=1501 spamscore=0 mlxlogscore=999
 clxscore=1015 suspectscore=0 mlxscore=0 malwarescore=0 lowpriorityscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090051

DQpPbiAwNy4wOS4yMSAxMzowNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA3LjA5LjIwMjEg
MTE6NTIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTI6
MTksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTE6MDcsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxMTo0OSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDEwOjE4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBTbywgaWYgd2UgaGF2ZSBhIGhpZGRlbiBQQ0kgZGV2aWNlIHdo
aWNoIGNhbiBiZSBhc3NpZ25lZCB0byBhIGd1ZXN0IGFuZCBpdCBpcyBsaXRlcmFsbHkgdW50b3Vj
aGVkDQo+Pj4+Pj4gKG5vdCBlbmFibGVkIGluIERvbTApIHRoZW4gSSB0aGluayB0aGVyZSB3aWxs
IGJlIG5vIHN1Y2ggcmVmZXJlbmNlIGFzICJob3N0IGFzc2lnbmVkIHZhbHVlcyIgYXMNCj4+Pj4+
PiBtb3N0IHByb2JhYmx5IHRoZSBjb21tYW5kIHJlZ2lzdGVyIHdpbGwgcmVtYWluIGluIGl0cyBh
ZnRlciByZXNldCBzdGF0ZS4NCj4+Pj4+IFdoYXQgbWVhbmluZyBvZiAiaGlkZGVuIiBkbyB5b3Ug
aW1wbHkgaGVyZT8gRGV2aWNlcyBwYXNzZWQgdG8NCj4+Pj4+IHBjaV97aGlkZSxyb31fZGV2aWNl
KCkgbWF5IG5vdCBiZSBhc3NpZ25lZCB0byBndWVzdHMgLi4uDQo+Pj4+IFlvdSBhcmUgY29tcGxl
dGVseSByaWdodCBoZXJlLg0KPj4+Pj4gRm9yIGFueSBvdGhlciBtZWFuaW5nIG9mICJoaWRkZW4i
LCBldmVuIGlmIHRoZSBkZXZpY2UgaXMgY29tcGxldGVseQ0KPj4+Pj4gaWdub3JlZCBieSBEb20w
LA0KPj4+PiBEb20wbGVzcyBpcyBzdWNoIGEgY2FzZSB3aGVuIGEgZGV2aWNlIGlzIGFzc2lnbmVk
IHRvIHRoZSBndWVzdA0KPj4+PiB3aXRob3V0IERvbTAgYXQgYWxsPw0KPj4+IEluIHRoaXMgY2Fz
ZSBpdCBpcyBlbnRpcmVseSB1bmNsZWFyIHRvIG1lIHdoYXQgZW50aXR5IGl0IGlzIHRvIGhhdmUN
Cj4+PiBhIGdsb2JhbCB2aWV3IG9uIHRoZSBQQ0kgc3Vic3lzdGVtLg0KPj4+DQo+Pj4+PiAgICAg
Y2VydGFpbiBvZiB0aGUgcHJvcGVydGllcyBzdGlsbCBjYW5ub3QgYmUgYWxsb3dlZA0KPj4+Pj4g
dG8gYmUgRG9tVS1jb250cm9sbGVkLg0KPj4+PiBUaGUgbGlzdCBpcyBub3QgdGhhdCBiaWcsIGNv
dWxkIHlvdSBwbGVhc2UgbmFtZSBhIGZldyB5b3UgdGhpbmsgY2Fubm90DQo+Pj4+IGJlIGNvbnRy
b2xsZWQgYnkgYSBndWVzdD8gSSBjYW4gdGhpbmsgb2YgUENJX0NPTU1BTkRfU1BFQ0lBTCg/KSwN
Cj4+Pj4gUENJX0NPTU1BTkRfSU5WQUxJREFURSg/KSwgUENJX0NPTU1BTkRfUEFSSVRZLCBQQ0lf
Q09NTUFORF9XQUlULA0KPj4+PiBQQ0lfQ09NTUFORF9TRVJSLCBQQ0lfQ09NTUFORF9JTlRYX0RJ
U0FCTEUgd2hpY2ggd2UgbWF5IHdhbnQgdG8NCj4+Pj4gYmUgYWxpZ25lZCB3aXRoIHRoZSAiaG9z
dCByZWZlcmVuY2UiIHZhbHVlcywgZS5nLiB3ZSBvbmx5IGFsbG93IHRob3NlIGJpdHMNCj4+Pj4g
dG8gYmUgc2V0IGFzIHRoZXkgYXJlIGluIERvbTAuDQo+Pj4gV2VsbCwgeW91J3ZlIGNvbXBpbGUg
YSBsaXN0IGFscmVhZHksIGFuZCBJIGRpZCBzYXkgc28gYmVmb3JlIGFzIHdlbGw6DQo+Pj4gRXZl
cnl0aGluZyBleGNlcHQgSS9PIGFuZCBtZW1vcnkgZGVjb2RpbmcgYXMgd2VsbCBhcyBidXMgbWFz
dGVyaW5nDQo+Pj4gbmVlZHMgYXQgbGVhc3QgY2xvc2VseSBsb29raW5nIGF0LiBJTlRYX0RJU0FC
TEUsIGZvciBleGFtcGxlLCBpcw0KPj4+IHNvbWV0aGluZyBJIGRvbid0IHRoaW5rIGEgZ3Vlc3Qg
c2hvdWxkIGJlIGFibGUgdG8gZGlyZWN0bHkgY29udHJvbC4NCj4+PiBJdCBtYXkgc3RpbGwgYmUg
dGhlIGNhc2UgdGhhdCB0aGUgaG9zdCBwZXJtaXRzIGl0IGNvbnRyb2wsIGJ1dCB0aGVuDQo+Pj4g
b25seSBpbmRpcmVjdGx5LCBhbGxvd2luZyB0aGUgaG9zdCB0byBhcHByb3ByaWF0ZWx5IGFkanVz
dCBpdHMNCj4+PiBpbnRlcm5hbHMuDQo+Pj4NCj4+PiBOb3RlIHRoYXQgZXZlbiBmb3IgSS9PIGFu
ZCBtZW1vcnkgZGVjb2RpbmcgYXMgd2VsbCBhcyBidXMgbWFzdGVyaW5nDQo+Pj4gaXQgbWF5IGJl
IG5lY2Vzc2FyeSB0byBsaW1pdCBndWVzdCBjb250cm9sOiBJbiBjYXNlIHRoZSBob3N0IHdhbnRz
DQo+Pj4gdG8gZGlzYWJsZSBhbnkgb2YgdGhlc2UgKHBlcmhhcHMgdHJhbnNpZW50bHkpIGRlc3Bp
dGUgdGhlIGd1ZXN0DQo+Pj4gd2FudGluZyB0aGVtIGVuYWJsZWQuDQo+PiBPaywgc28gaXQgaXMg
bm93IGNsZWFyIHRoYXQgd2UgbmVlZCBhIHlldCBhbm90aGVyIHBhdGNoIHRvIGFkZCBhIHByb3Bl
cg0KPj4gY29tbWFuZCByZWdpc3RlciBlbXVsYXRpb24uIFdoYXQgaXMgeW91ciBwcmVmZXJlbmNl
OiBkcm9wIHRoZSBjdXJyZW50DQo+PiBwYXRjaCwgaW1wbGVtZW50IGNvbW1hbmQgcmVnaXN0ZXIg
ZW11bGF0aW9uIGFuZCBhZGQgYSAicmVzZXQgcGF0Y2giDQo+PiBhZnRlciB0aGF0IG9yIHdlIGNh
biBoYXZlIHRoZSBwYXRjaCBhcyBpcyBub3csIGJ1dCBJJ2xsIG9ubHkgcmVzZXQgSU8vbWVtIGFu
ZCBidXMNCj4+IG1hc3RlciBiaXRzLCBlLmcuIHJlYWQgdGhlIHJlYWwgdmFsdWUsIG1hc2sgdGhl
IHdhbnRlZCBiaXRzIGFuZCB3cml0ZSBiYWNrPw0KPiBFaXRoZXIgb3JkZXIgaXMgZmluZSB3aXRo
IG1lIGFzIGxvbmcgYXMgdGhlIHJlc3VsdCB3aWxsIGJlIGNsYWltZWQgdG8NCj4gYmUgY29tcGxl
dGUgdW50aWwgcHJvcGVyIGVtdWxhdGlvbiBpcyBpbiBwbGFjZS4NCkkgdHJpZWQgdG8gc2VlIHdo
YXQgb3RoZXJzIGRvIGluIG9yZGVyIHRvIGVtdWxhdGUgUENJX0NPTU1BTkQgcmVnaXN0ZXINCmFu
ZCBpdCBzZWVtcyB0aGF0IGF0IG1vc3QgdGhleSBjYXJlIGFib3V0IHRoZSBvbmx5IElOVFggYml0
IChiZXNpZGVzDQpJTy9tZW1vcnkgZW5hYmxlIGFuZCBidXMgbXVzdGVyIHdoaWNoIGFyZSB3cml0
ZSB0aHJvdWdoKS4gUGxlYXNlIHNlZQ0KWzFdIGFuZCBbMl0uIFByb2JhYmx5IEkgbWlzcyBzb21l
dGhpbmcsIGJ1dCBpdCBjb3VsZCBiZSBiZWNhdXNlIGluIG9yZGVyDQp0byBwcm9wZXJseSBlbXVs
YXRlIHRoZSBDT01NQU5EIHJlZ2lzdGVyIHdlIG5lZWQgdG8ga25vdyBhYm91dCB0aGUNCndob2xl
IFBDSSB0b3BvbG9neSwgZS5nLiBpZiBhbnkgc2V0dGluZyBpbiBkZXZpY2UncyBjb21tYW5kIHJl
Z2lzdGVyDQppcyBhbGlnbmVkIHdpdGggdGhlIHVwc3RyZWFtIHBvcnQgZXRjLiBUaGlzIG1ha2Vz
IG1lIHRoaW5rIHRoYXQgYmVjYXVzZQ0Kb2YgdGhpcyBjb21wbGV4aXR5IG90aGVycyBqdXN0IGln
bm9yZSB0aGF0LiBOZWl0aGVyIEkgdGhpbmsgdGhpcyBjYW4gYmUNCmVhc2lseSBkb25lIGluIG91
ciBjYXNlLiBTbyBJIHdvdWxkIHN1Z2dlc3Qgd2UganVzdCBhZGQgdGhlIGZvbGxvd2luZw0Kc2lt
cGxlIGxvZ2ljIHRvIG9ubHkgZW11bGF0ZSBQQ0lfQ09NTUFORF9JTlRYX0RJU0FCTEU6IGFsbG93
IGd1ZXN0IHRvDQpkaXNhYmxlIHRoZSBpbnRlcnJ1cHRzLCBidXQgZG9uJ3QgYWxsb3cgdG8gZW5h
YmxlIGlmIGhvc3QgaGFzIGRpc2FibGVkDQp0aGVtLiBUaGlzIGlzIGFsc28gY291bGQgYmUgdHJp
Y2t5IGEgYml0IGZvciB0aGUgZGV2aWNlcyB3aGljaCBhcmUgbm90DQplbmFibGVkIGFuZCB0aHVz
IG5vdCBjb25maWd1cmVkIGluIERvbTAsIGUuZy4gd2UgZG8gbm90IGtub3cgZm9yIHN1cmUNCmlm
IHRoZSB2YWx1ZSBpbiB0aGUgUENJX0NPTU1BTkQgcmVnaXN0ZXIgKGluIHBhcnRpY3VsYXINClBD
SV9DT01NQU5EX0lOVFhfRElTQUJMRSBiaXQpIGNhbiBiZSB1c2VkIGFzIHRoZSByZWZlcmVuY2Ug
aG9zdCB2YWx1ZSBvcg0Kbm90LiBJdCBjYW4gYmUgdGhhdCB0aGUgdmFsdWUgdGhlcmUgaXMganVz
dCB0aGUgb25lIGFmdGVyIHJlc2V0IG9yIHNvLg0KVGhlIHJlc3Qgb2YgdGhlIGNvbW1hbmQgcmVn
aXN0ZXIgYml0cyB3aWxsIGdvIGRpcmVjdGx5IHRvIHRoZSBjb21tYW5kDQpyZWdpc3RlciB1bnRv
dWNoZWQuDQpTbywgYXQgdGhlIGVuZCBvZiB0aGUgZGF5IHRoZSBxdWVzdGlvbiBpcyBpZiBQQ0lf
Q09NTUFORF9JTlRYX0RJU0FCTEUNCmlzIGVub3VnaCBhbmQgaG93IHRvIGdldCBpdHMgcmVmZXJl
bmNlIGhvc3QgdmFsdWUuDQoNCj4gSmFuDQoNClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQoNClsx
XSBodHRwczovL2dpdGh1Yi5jb20vcWVtdS9xZW11L2Jsb2IvbWFzdGVyL2h3L3hlbi94ZW5fcHRf
Y29uZmlnX2luaXQuYyNMMzEwDQpbMl0gaHR0cHM6Ly9naXRodWIuY29tL3Byb2plY3RhY3JuL2Fj
cm4taHlwZXJ2aXNvci9ibG9iL21hc3Rlci9oeXBlcnZpc29yL2h3L3BjaS5jI0wzMzY=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:43:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:43:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182783.330543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFef-0005Gl-L2; Thu, 09 Sep 2021 08:43:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182783.330543; Thu, 09 Sep 2021 08:43:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFef-0005Ge-Gt; Thu, 09 Sep 2021 08:43:33 +0000
Received: by outflank-mailman (input) for mailman id 182783;
 Thu, 09 Sep 2021 08:43:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOFee-0005GY-EQ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:43:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 023f6a6a-114a-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:43:31 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-dyiNiURvNA6DQogx8WGF4g-1; Thu, 09 Sep 2021 10:43:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 08:43:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 08:43:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0036.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Thu, 9 Sep 2021 08:43:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 023f6a6a-114a-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631177010;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wqtc6eD4w9cLj344mz9Jg3QTuzoqp5l+GWqOi2EnBUA=;
	b=DCUlz4a1/HBRQ5s0j2Uh16eFxYIoKTks+Ru9JAe2ricv/uwl6+25cDaHBO89Nw74IVaAJZ
	trrPxJjH+Dzc7Did5F7ZqWnPaogZN3DCHGFstK0P+mNae5MM+fURJqhW0z0wn3kbKDFb4A
	yYA+F2vUopOv2W2yDg0iCe4R88dPHB4=
X-MC-Unique: dyiNiURvNA6DQogx8WGF4g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VWNFVcBJe3nea4vmQWnolX79t7Zkh68+es/QHFtPZkkmbHuGeCbUDFlRZLyZ9ui4UlWAJLfUeX2DIu6YqJ8LDUuqITMLGDLniy4vP8tkwXvKeKcyJI3V2pdqT2YPX6vwSdWTx7NTbnx6xPm6HvNo2AGKzMVw07/JS7PJXm6cAVinSZWLiBrF2ZMTIvM2KQCjsiqtwSJrcV3p48u8bee/QAMRAldZDaqodlbKIApCkQy1IQTSsSvJqsLhWgOxvPMfSjraiBTws/ejb4cYGGfXNiD96qXRdOefodr8g0oqfM2JjAdIScYZPCYCVPHE6Mls4xJPlUsDPSt7kP2z97MI0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wqtc6eD4w9cLj344mz9Jg3QTuzoqp5l+GWqOi2EnBUA=;
 b=JbZSFRj5I6jMOfen6RRu+EXPUP+cE1VRtApcYrE5s0LeV7cRDfbJIqUrmUn1Bixl3WIXgDBhnFZuNAL3pNVa4ObGz1TD9zaXS6OxHDOoYqjxeZxAn4N6uaMXmQlXHNVTry/ovLM/UuoQAivAbv2njY6+XRmV/uY3U4vohE93ZH4s58dElaFGpxysbroxD4E+JyTSsW+RW5/KWEvBvzGve4sZ6by9HX2laX4CCHfHQFVF7ysE6ZEQfoyOwjrF4Myb0iqHdclB/awYx2XBTdoRKQS2evhZIcH/3klc/XACKmjuG83Jn0Y8gGisZSxiIl21vUg5zWo5Ry/sd5QJUmxseg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
Date: Thu, 9 Sep 2021 10:43:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0036.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9d26ea7b-859a-4047-1c11-08d9736de41b
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7039C19FCFEA4B9C3DE23C8FB3D59@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ayl/HTiWgyk+tu3FG2XKiLRLTRAeXjby5pXB4Vgn8I3pzDUNq3OR0CsNQl5zMSXvvsLra5FWZ2Nj2K+iQqxSNZMxadbCgJ+n/gXpDRQfC2KpsWxzoMMpd0Kvw/eR1g540w+U5E0ToEOKw9Lgi48h0uDtiwgvrFyHLmJZ30HuTFIiddBKZARCbIjKZAhzHLGIcvsn9E7fCCqKUfS1f03Yz+0EA8IMCORFkw439sVrG1aE9+tcMwC48eAQXtAhyUU2ouEsbpHpy2IjxpvFZ3tnLZ65jOV86DF5ZADdPsE4voirbp/ovsKvI+p9e5FCeVhVqkMFG0m+6aZzzVoJFJYCw/BLz2ZxLrjGDwQI8e2aBAr9Wsgd5oUSnMGDeAPNTyFHLzdu1lnUYS1yEr0fLb3h3yJ1FU1Su0GwNK5HBBRMHTeSg7UyA0nqJTqfPVfrmgbLoSXRLwL2kQ13gdegzyn5OsEE2lWe5Hb1iU6Jnu1Y6NS6Lhr1sAiCpwvjPWWXMzMDWIDVBxXRu5JcrKTJoCEa32EKZCzGMuO4u6GDsknXJ3MEiNQnyXSh8OgCmwiZtZklgOxVDrw4mZdrl6P13y3E1Ydplpy/VTC+lVHUqxkKv09KkQhlaDMJSduq9Y9K71ZQHglnTOwhn/TCJ4poRwTgHi94quSI/HpvcUhuljcDHe1EdeF6l9UHETxolhjztasaH95whsq+x9OfcJdAH4cxKsTgtwMLfrHVCSnJ41T3Q0EgLtHnliaW+XMwJbVr2SyF
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(39860400002)(376002)(346002)(136003)(66946007)(6486002)(66556008)(38100700002)(66476007)(26005)(7416002)(478600001)(8936002)(2906002)(53546011)(31686004)(316002)(5660300002)(16576012)(54906003)(36756003)(110136005)(86362001)(4326008)(8676002)(186003)(83380400001)(2616005)(31696002)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SFVvRmJuR2hvUE1JQ2dqVUhTZU9tL1RSU2p1eWQ5TXJxbUtsSzR5dnRXZHdm?=
 =?utf-8?B?SUZhNFpEazdXY2YzTHdPYUZ6YVlpSmQweG44dnBDb0dudmdWclgvRlB6eTdX?=
 =?utf-8?B?MDBCWkxwZWd0a05Ua3BILytFVHMwaGlvWHlrc2I1TTFObUpJM05EczRoSDJ4?=
 =?utf-8?B?aDl1MmpCTXU4MGNWYVdNVnVKeDNjQ2U0MW1qSDY2QytWNVh5aFg3RTZYODBi?=
 =?utf-8?B?L2J1dGxHb0F3cmJKRGhVNm9LYnd2UXdRTXcxVWlXRFJuUDhPYVcyc1U4Z2l6?=
 =?utf-8?B?WDBtdGdYdEVleTR4enNITHBxeFhXNFVheG1MWFNuWllZNzVTRVcvdzI3SHNZ?=
 =?utf-8?B?YnFQMDFtMjJYalVwcEpnWEdGRmZJNW11Z09HRWcwNytmQ294ZUtseGVldlo0?=
 =?utf-8?B?SFVidWd1KzdxQUhIc3p4OUgvcXQ1V0lKVUZXNDA4Qk94TENpZUZqSlNMMGNu?=
 =?utf-8?B?cGJmOGJPN0ZsNlRFb0NYeHQ1MWVaK2MycldzY2lxV0UxcjVteXVBQnQ5M2c3?=
 =?utf-8?B?MDcxanhzSWY4M2Nnayt4SElBd0M3S01vWmRKS2dPaElaQVBDOHpMY3NldHhk?=
 =?utf-8?B?eTA1eG1xaXJyRGF6a3U2c1VLNUhJKzdPR004Y1d5OEJiVkhpVEZVdE1sY0d4?=
 =?utf-8?B?eWFjMktvTnowcldGQW9ibDhqRFJ3Q1BwaGdySnl5UTA4UHFzaHRiK3JUTm5n?=
 =?utf-8?B?THFmUkFkdVZ0ajJmU1BRUEQ1YzZaUHVSL0h4b1N0czFtbUt6S0tVMzRQNGw0?=
 =?utf-8?B?VEVCa24ydnRHaDhSN1NNZXVHVUNGV2JtNEhkK3RqU280SzRYZ3Q5M01wbmxq?=
 =?utf-8?B?dmNWYkJ1YmFZb2dUaDdhZi9STDR4TjUxRVpmOVlkRUpvdjdtYU1wVlRWS2JI?=
 =?utf-8?B?amx2VVVHbGdFSDhWbGttMUJUOVV3ZGJJUGdTeW1UTmxVcW16T2pNU05YbURD?=
 =?utf-8?B?Tlg2d2pNV21QbWlkK0NTeFhCWmpIdFV5Z3dYcTV4dHhlK1I0dzBKb0J1Z015?=
 =?utf-8?B?OUE0bC9xUUMwU3k3RWFkVERWRVVKQjRhZ0kzMFRhSWZmMmhRelMwQnBtU2U3?=
 =?utf-8?B?NFB4NjFYV3RiWWNWdk1SOHhGbmFPcVRueHVjM2RZRlViV2JtT0FoUGhSRE8z?=
 =?utf-8?B?cTNmZWkzSXF6R002VkhvYVJvek1tVFY1WGplcWpuSjdQRXFYNjU1aHdUSTVu?=
 =?utf-8?B?WE9IUDlSbGRlejJWazhqNUt2a29NYStiZlNHZjhVQVhheWJiVmIyZ3RqZHZl?=
 =?utf-8?B?MElpNUtERDNjdytRMmNPVHNmNTB6aStkbXpkZ2EzUXdaZVhPZGRqUytxUTd6?=
 =?utf-8?B?MnBkYmRSbXh3MmNTR3N5M09iRVRvRStZR3pXZEFtVnlVNmFNZDkzUFprS0tk?=
 =?utf-8?B?MC9telNEQzhRcmh4c2FQdU5oQytrVTk5eGVTeVZvUWJKclJrSnRFSXNOd0RE?=
 =?utf-8?B?NjQ1bzJiTk5uK0VTODZNbzBLZzFGaU5jcjJuTjg2Y0UxSWRJQk5ZTlVsT3BG?=
 =?utf-8?B?aWZGdFpRYVN2Y0RWQWhQSWx1emFsSWQwWVViK21pM1J1alFwTGhJV2RhSjQ4?=
 =?utf-8?B?WnYya2t5VTgwcVhXdDlGNDZpV2Q4RlBFb2xiTE9lN1FROEV6c1V0ZHg1TUdZ?=
 =?utf-8?B?T09heXFwVUhpZHpObWV6M2YzTDA4eHhNc0tpUEVVZ1o2TXl4QkVoQTF3QWZn?=
 =?utf-8?B?cDlvNkxNWUtLU1NEdjBRZUQrNFhNanJBd0tVUXN3eWVlL2pLUjRndkJ5cmxP?=
 =?utf-8?Q?2Vcz8fYzYCpcUxuVJrHJmwXIXkM6CgycfkrHZkv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9d26ea7b-859a-4047-1c11-08d9736de41b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 08:43:26.6396
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WG6cwc/aN/v08dsMnG/OcyrhQApcwSUBGJQLaxUn+sBdnE2iAnrq1XBJ9Wpdv2y516IclVEFWgNTWH6xD2YJ3A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 09.09.2021 10:39, Oleksandr Andrushchenko wrote:
> 
> On 07.09.21 13:06, Jan Beulich wrote:
>> On 07.09.2021 11:52, Oleksandr Andrushchenko wrote:
>>> On 07.09.21 12:19, Jan Beulich wrote:
>>>> On 07.09.2021 11:07, Oleksandr Andrushchenko wrote:
>>>>> On 07.09.21 11:49, Jan Beulich wrote:
>>>>>> On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>>>>>>> So, if we have a hidden PCI device which can be assigned to a guest and it is literally untouched
>>>>>>> (not enabled in Dom0) then I think there will be no such reference as "host assigned values" as
>>>>>>> most probably the command register will remain in its after reset state.
>>>>>> What meaning of "hidden" do you imply here? Devices passed to
>>>>>> pci_{hide,ro}_device() may not be assigned to guests ...
>>>>> You are completely right here.
>>>>>> For any other meaning of "hidden", even if the device is completely
>>>>>> ignored by Dom0,
>>>>> Dom0less is such a case when a device is assigned to the guest
>>>>> without Dom0 at all?
>>>> In this case it is entirely unclear to me what entity it is to have
>>>> a global view on the PCI subsystem.
>>>>
>>>>>>     certain of the properties still cannot be allowed
>>>>>> to be DomU-controlled.
>>>>> The list is not that big, could you please name a few you think cannot
>>>>> be controlled by a guest? I can think of PCI_COMMAND_SPECIAL(?),
>>>>> PCI_COMMAND_INVALIDATE(?), PCI_COMMAND_PARITY, PCI_COMMAND_WAIT,
>>>>> PCI_COMMAND_SERR, PCI_COMMAND_INTX_DISABLE which we may want to
>>>>> be aligned with the "host reference" values, e.g. we only allow those bits
>>>>> to be set as they are in Dom0.
>>>> Well, you've compile a list already, and I did say so before as well:
>>>> Everything except I/O and memory decoding as well as bus mastering
>>>> needs at least closely looking at. INTX_DISABLE, for example, is
>>>> something I don't think a guest should be able to directly control.
>>>> It may still be the case that the host permits it control, but then
>>>> only indirectly, allowing the host to appropriately adjust its
>>>> internals.
>>>>
>>>> Note that even for I/O and memory decoding as well as bus mastering
>>>> it may be necessary to limit guest control: In case the host wants
>>>> to disable any of these (perhaps transiently) despite the guest
>>>> wanting them enabled.
>>> Ok, so it is now clear that we need a yet another patch to add a proper
>>> command register emulation. What is your preference: drop the current
>>> patch, implement command register emulation and add a "reset patch"
>>> after that or we can have the patch as is now, but I'll only reset IO/mem and bus
>>> master bits, e.g. read the real value, mask the wanted bits and write back?
>> Either order is fine with me as long as the result will be claimed to
>> be complete until proper emulation is in place.
> I tried to see what others do in order to emulate PCI_COMMAND register
> and it seems that at most they care about the only INTX bit (besides
> IO/memory enable and bus muster which are write through). Please see
> [1] and [2]. Probably I miss something, but it could be because in order
> to properly emulate the COMMAND register we need to know about the
> whole PCI topology, e.g. if any setting in device's command register
> is aligned with the upstream port etc. This makes me think that because
> of this complexity others just ignore that. Neither I think this can be
> easily done in our case. So I would suggest we just add the following
> simple logic to only emulate PCI_COMMAND_INTX_DISABLE: allow guest to
> disable the interrupts, but don't allow to enable if host has disabled
> them. This is also could be tricky a bit for the devices which are not
> enabled and thus not configured in Dom0, e.g. we do not know for sure
> if the value in the PCI_COMMAND register (in particular
> PCI_COMMAND_INTX_DISABLE bit) can be used as the reference host value or
> not. It can be that the value there is just the one after reset or so.
> The rest of the command register bits will go directly to the command
> register untouched.
> So, at the end of the day the question is if PCI_COMMAND_INTX_DISABLE
> is enough and how to get its reference host value.

Well, in order for the whole thing to be security supported it needs to
be explained for every bit why it is safe to allow the guest to drive it.
Until you mean vPCI to reach that state, leaving TODO notes in the code
for anything not investigated may indeed be good enough.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:44:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:44:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182790.330553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFff-0005qs-UA; Thu, 09 Sep 2021 08:44:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182790.330553; Thu, 09 Sep 2021 08:44:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFff-0005ql-RA; Thu, 09 Sep 2021 08:44:35 +0000
Received: by outflank-mailman (input) for mailman id 182790;
 Thu, 09 Sep 2021 08:44:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOFfe-0005qd-St
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:44:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 27a6a387-114a-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 08:44:34 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2051.outbound.protection.outlook.com [104.47.2.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-Qjzmz6lLOMefy1tmOGCJiQ-1; Thu, 09 Sep 2021 10:44:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 08:44:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 08:44:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0020.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 08:44:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27a6a387-114a-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631177073;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hJsEPTI7iAhv1VfCdSFgbNeaxTnEKnCS7Sh+KJxfHyo=;
	b=P52SJynOvqkMPsAN75S0nL10W5GJ1V66gOA9y+MV/x3OD++pZc/o5yIKDHJ02ekkawMB4c
	h1eM1LCkJ7KotmVzfIQ++uTj9sw+upRN7+r+SKE8Kh2rPGY/kS+DGvXBxPoc39zuimlpT/
	ZoAvvUo51gXvEgsX3UdIYtUYJK9rFmM=
X-MC-Unique: Qjzmz6lLOMefy1tmOGCJiQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KLApFy64OC6qbj+9Bd9/INk7JrUPO6TssvC4O9+xb84VapVH9ATkU/17r6805byqMVdvCd2OxSN+6MpPcDhkd6VULM8yQkE0XjAg4WCkwCTVhTMiDNUoVi4K5gLsrPXX+JVCxtFXJGA8h3p0V5QvgJe0VlUGRJ9AQPu8n6mUJRfb1UwBHEA06T5aDi4b/CDRMw2b95PJrRiUw66SJ0W52QXJbdusaft1c5FBJFIakD8tXLudvaAE1R6EaYDRyXKNzKFfMyZyozsIVD6Ajz/vzNgfHKs/6P7ke4vx/6UmTy5RoH/kkcUnxc946duOTCGUTpLOiJ4q28qWf9wKPEMblA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hJsEPTI7iAhv1VfCdSFgbNeaxTnEKnCS7Sh+KJxfHyo=;
 b=L4pM4KFKcF2WWtcF4UixT49QhBEiERiQ3JiRQgZJrIT3EXHNUTP2iLo/z2afSIQTLpAeEqiO+A4Im9Zq6KZ0QUtWlvWyMzvWaGkyBwovoZ6xz8OfW/AFd0sLlMRief+C0EMDF9zH/of1lNh7pbFIQfBSUrBe9fg0r98favmEvlLD3ckC8WZIZdVelprERVNYpPkDxo+rIWVSaO3RubaK+ZQW+R7X+rFhF2QUjJEKYD7uTUqwuOCFcRP53DuwjlzbfLC7NWH25HTtqKAz71FRNuYSrpVzrp6XCW8ID5/Y1S7O1nS1xJGiY+sBO8WtBD/H1vPioGrOQ8t7PNpm3lUJaQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 0/2] stubdom: disable building of pv-grub per default
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Samuel Thibault <samuel.thibault@ens-lyon.org>,
 xen-devel@lists.xenproject.org
References: <20210909082705.9434-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e1004466-5d12-efd6-8729-27e72ad6f7d8@suse.com>
Date: Thu, 9 Sep 2021 10:44:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210909082705.9434-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0020.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::32)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2d30fef9-2e5b-4102-5196-08d9736e09e9
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23346D5E7B7F10EF5F4426FCB3D59@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HZEZIxvgbXHsfcitgKS2b1bFQvFsrrGlZ1Pv0fc0LNWpo0LcJMfUwXL93f1vz/O+piLZExSlxG39Pogypg31sfZz2fV2EDUIJCtvk0I9jeAKT1m8wYEwy8ZG1WglKpJd8yve/vSukugVta368GrRV/U7jXe6jOzc0FfZkL0FVKj5L8Ex47a7Yval9xbEtjj7+rNsX7CZ/j6IV5RyUWD2uAtcfzOywZzMjlQ+Q79kSTxwM/+s5RlrQUk691R8QXY72WstlquYsy1ZVCP5mcKjZPB6w4OGUJZRIVJCjsAre4UFHOkHIzEWzSJlspqSw4nlQ0A4rm6uk5WhDCm7IOHoCtajIeiw7YpH6Mvtj2ddDCkWkAUpL1GafxCcXIo26noG5WlSyJiTz+vWHh8AqJNd84h2S7CD14aapLmdNHg9NlybP+WPdQnxn/FmCutwgm4TQacp56PnqvjpaQfSqLfApR6nfPgfsaeRwk194JWykR1p0lXAvqAdk2NupEyDTD9lDlCs5qna/eL7dvCT9H5TNoOZDfad8Dc8cz4WvxBIakiJmBJk+aGHj1NqXBN+9i0B78DdF2UgLPVtEbsDPIA1LRVuEPM0QiCb3Kqd6mNcVxD5sveXmZQfqLOEE2yGGU+UEVhW4ZRJdSx/HfqMLmvbja9qLvPEw8r9TNeeYhMRU/xHXQolt4gGGmgOAZdAoAHmsPA6sLBpW7Bz4GXPq35XhF4VWRqQ2HMCwsVuMx8RKGJhp/JNIehrbErJ2kYlLa5y
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(366004)(346002)(396003)(26005)(186003)(66946007)(31696002)(37006003)(2906002)(53546011)(54906003)(66556008)(66476007)(478600001)(5660300002)(8676002)(31686004)(8936002)(558084003)(2616005)(956004)(38100700002)(316002)(16576012)(6486002)(86362001)(36756003)(6862004)(6636002)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c2lEVkcycThKTU5tMWhXNS8rZG1nNzk5NHgwS1lOSFNuRkRBV1RWUzgrNnpK?=
 =?utf-8?B?RFZoTkJ4aHpZa0sxdUFtYkFYQURueHBOMUMyelhxN2F2WkUralVxVlhVOTFP?=
 =?utf-8?B?bjlHN3dkVnpuWFpiaXhWZVVDQTZody8rRnBMaVBaQjRzOXBTUEUveGJTSk5U?=
 =?utf-8?B?Y29JdTlSb2lVYTN5Y0tGa3JQSkt3M0tiK3o1ZDFJQ0ZFSEExcEIvQTlOam9Q?=
 =?utf-8?B?dzltbXcrZERyNGY4eHF2LzJWdi9jVndBRUdvamh3UkRDelFZVWhGWkV1Ykha?=
 =?utf-8?B?dlcyUEJoZHFFZDJIU0xiVFlLK3Q3d3NHN1EvYnZTZG0xbWEwWm1rY3lIWS9P?=
 =?utf-8?B?clNodnY1OGwrMG16L0wwMEcyVlFDanl3RnBxUW1WWHdKRFZ0V3RqQ2xkSk1D?=
 =?utf-8?B?OTlWTnRkZEdjMmxwbFlCWS9yamdnclh6akxYVGhKcCtCdTFRcmlsQ0FxVEo4?=
 =?utf-8?B?TENNejROMkZBQVVmc2xwakZsa2tFMWY1M1RLZDU2eG5mc0xzNTNRQkVCUVhw?=
 =?utf-8?B?bi9VUFlFdWl3MlhQZGowMkFHQzFiUjV0WWt5R2Z4YmFGM2pxbWN0M2ZncnpM?=
 =?utf-8?B?Z0RKYXN2VzNHOGdlYUxETUNHaHdMc1lYS0h2U1RYd0hPWjdsUlBkLzRaT2Yz?=
 =?utf-8?B?MDJvZmsxUjZ6VC9nSkRXTlh6UXRsL2RtSS81NW1HWUE3TkJFcFZDMVhuR2Zu?=
 =?utf-8?B?VVVmaENlRWJpb1JZQzM1eWhaYVdYQjd6MWhuQVNuQjEwb0J4dkFCR0JET25u?=
 =?utf-8?B?TGF6WWFwaGlldDl5dURieWY1SVBibjQxQ05DSXZSbWFUcVNKSkxsN1VTckRq?=
 =?utf-8?B?eE9ZdHg2N3pocmJ3WS9JQkVIcldpQ3M0bVZFbm5aYmtwS0RkaXpiV1JHY2lS?=
 =?utf-8?B?blZUdjZ5bnJKUE1SZFlqREpWanZPUzN4eGplOThLc3didHNqOHF3UVRjTDQ3?=
 =?utf-8?B?STF5dWp5bmpzelBjUk43cnM2THZMMGE1dlVxcXRndng2Ly9kRWYzdDRsTERz?=
 =?utf-8?B?UHhRSXRHd1dWZWJwUHRhWXVwRll3cFhjUmdSTm1kb0VESGk2WlBDZHROY3VQ?=
 =?utf-8?B?K1FKYVBZUXY3ekpmbXlSTTIyVFdtMWJBZWpTT0s4SU1UWWIyYUtUM251dThm?=
 =?utf-8?B?amlLUW8yenpTaTZRWVBGQlRLRmc2aWJqMjdsa3EyR0VLbk1ocEQ1RzJObXcx?=
 =?utf-8?B?dE02L3RvWVQxZkZyRTA2TzFaODAwQkhBNzFCdUEzNVVNeUM4Smc1NTVGY3lG?=
 =?utf-8?B?Mzl0N3g3S1haaUd6cUZMK1ZUVER4RXIzVmExRFVaWnNyTHlnOVFLdDBzdTNz?=
 =?utf-8?B?LzljbEFQQldVeTZFZmV0eXIxUnEzUzJjTXYybXNLSE9pWjFVSUZFejNQRXE1?=
 =?utf-8?B?a2dZdVNEYUlNbThDNnNBb0N5Wks5bGE5Q3gvUENYdFYwdFJ6b0ZsNEdjSW1Q?=
 =?utf-8?B?eGd2blFJQnV4OC9rTk1xYXZSU2lhcjhSZG5IYU5LU1JGaVBiWHgxblBiMFQz?=
 =?utf-8?B?WWkyQytQL2tYUlZEcTEwL1NIbkpiTnNzaDVWR3VKRHNXcXFVU3hKOU9UTGM2?=
 =?utf-8?B?NTVpQlU0ZlFGM1d2T3dxbDlGVHcxMU8xMjJYcG5BQmRCc21JYmFOcUQ5Wm1P?=
 =?utf-8?B?cGx3S2w5U1JCbmVLbkc1WXhVejRJZGxsR3J3amFXbmdNay9CQ3RkSmVjeDNm?=
 =?utf-8?B?RkNkZUNzbW0rS1VIbTJlZndGOGdNeHJvbWF3b1czQlFwZW9iaTZFc00rS2xi?=
 =?utf-8?Q?5wjzGrCgkD8RZMsyQyInrIvy+Gx3bEn/J8b9xJ4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2d30fef9-2e5b-4102-5196-08d9736e09e9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 08:44:30.0567
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ncnAQwe2OCF/G1f/+8nl4JZSA2ZCY6h21VV7q16XUvhlpBc3aGoibR4yvdmhZNKw9khLz7PI4v//DgO4IBN6xg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

On 09.09.2021 10:27, Juergen Gross wrote:
> This is a first step of deprecating pv-grub. Switch the default to not
> building it.
> 
> NOTE: This should be mentioned in the release notes!

And/or perhaps in ./CHANGELOG.md?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:50:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:50:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182799.330571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFlb-0007HQ-Ly; Thu, 09 Sep 2021 08:50:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182799.330571; Thu, 09 Sep 2021 08:50:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFlb-0007HJ-IZ; Thu, 09 Sep 2021 08:50:43 +0000
Received: by outflank-mailman (input) for mailman id 182799;
 Thu, 09 Sep 2021 08:50:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOFla-0007HD-43
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:50:42 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 18203bc2-7ebc-4f9f-aee2-8a044f7ba926;
 Thu, 09 Sep 2021 08:50:41 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1898nQs3031071; 
 Thu, 9 Sep 2021 08:50:38 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayf3bg03p-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 08:50:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3281.eurprd03.prod.outlook.com (2603:10a6:208:a::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 08:50:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 08:50:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18203bc2-7ebc-4f9f-aee2-8a044f7ba926
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bvIoCU1C4d9RnGGqwH8u1CwTSHxaVbB8PB53MTqF67Ix3+Cet1D0Rb2EOwPGolMOvdwmq+oEjTxbm/zH++Gda/Cx/6mpt51A2Kn9/eBhtLc9IfqUClI8fAkoKf5Jrd7c3eLml/ixebDY+0hffHD/m6F45YfF5xljOZW/2K2Sb5M2IDQe3Dg3xLgwA5Q+g3JdVXeK0I6tetYFvGTW/sQGlCav324cNn9enKC6toq/i+vUqFeDYKwzvvXqz1IG+2NBd+spK3O/+Xmugfa99ujdYaolC46o5isWUuTwO6ZhkyxqFOx8dQULLFIbQ7By3oPupUBbtcHeWI+Rq9fRELVi8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=iqLuCiWGA7vzMyytONa+qXpS50N6PCZ5JTuyAmXc0Ec=;
 b=dGWeeWs6sxQrJDVqqSfbgTRoprhCZVJZwDAuRZ2Bvvz31EiMY+JmSt14sdmi6LovG0NoQBNGRF1t92825T3DOW4H/7RoW+PAAplpwLJjw+37OtPTj9fF788Ozxt4jzmChxErt38A1xYe4XFqfgxOAuBK2lRurqwbVzowLkxsOC0K8WK2L4l3ENlUzAFvNWoS/ii4m5lOcfShVACL0C7qq0MsBVUIrf/e/djby9HAKGVs+mgPfjS2nino98oeXgtLQccwJU5dxafdV35Ys1DfArMj3mS0hkJ/9eUoE1UMHyIBIlInvSMJRSNjqzv1lRCpqUEU0jvF1yGgZ9hFUa5CGA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iqLuCiWGA7vzMyytONa+qXpS50N6PCZ5JTuyAmXc0Ec=;
 b=kFYklUI0p2yyUBCt8CLPWIAZLtmUaXhGbQldOXJ5Gy4gRyeF0z2GkxANTQF95NZMqDxM3HDyUcFm0yUcRi32E8RwX5KEtdEpCbheoXpXZKfESsauPcD61Di5eLv91IHWZU4LBaFmMJRGRREcUGq68tWEuugzWVKAyjdz4qS7o7XRVzr6AWm6ph/x0Z93NH/pBpK8P2b1wTP+RpyPcrV508gqYr3ykOl5Amf1jIei6VunjLwC/KOgiDPrjGTvCMSXHOsFlMc2vO1eqzB3lN8NrfE+jAR0bF2LNIVkA6Oem4iohxewujW0QI1wd6SAxs4IzpNiQQSK5OCBvFcxAHa97Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AA==
Date: Thu, 9 Sep 2021 08:50:33 +0000
Message-ID: <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
In-Reply-To: <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1eab9de8-f72b-4729-970d-08d9736ee2d3
x-ms-traffictypediagnostic: AM0PR0302MB3281:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR0302MB32812DD6C6D739040B38F4F9E7D59@AM0PR0302MB3281.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 2dd/pFpK9FIm+2GyWs5bYh7bkPLTkaeMO0tA8kAkG5cYwGuxaU6dbZwQe4xAVKnUb5Ixe6wc+VX2XEUWf/WJ7tuCabOdOdztqfyHMDkiQCjzPqC+YfBty3B+7WNmyPTr01aPw5MECgHoZ/DXZrm8s+w2UQ5o0z0XuDXTYR5lgZNM1hNKa1WhSQfyFuZqRmTQVISNDA9WhOwSEWzNxftZZUgh6UJDrri4RsJpftcvZZXdKj6THxz80fFYhcpwVo9A2gGjGenGYiBkZI0cXoNEsNyN4LlAQB31UXr3MiaD8XGp4AoaunGHTmZ279co10Mij3KHIqJHAw3b1MjME7bvBYv7eyBjgQ0eeLWPjqL4/jaLn7RFuTUAVqFKtf4ckvu2GyqCu77qkzjbp3qyzu1k1ymiUbgyPHqZThND0HWhjv51+KhW6+VaMRBgNpC39o9t8qUJvzcILOdqspNbGNxF/JmR6OuhVowXVdToPectTfRTsMwxEQ6fo9YV0H7HbxIatzOLEeYeHtbSbm9B9xOOQTxeR5Phz70E5mewLffZNsXfFbfqffmXmbh6haEmfHgrNzRsEjqgySeuGBI4u+ra4cARXtgEpEReNbgg8yp4JHzVyxIzsnpo6a3uehDqqR5O3KqaY88EW1oNP+aMIfrXTGWAYBkc3UmARGgduzkcNyknyedb5i+YwScg/zv6YfRI7L2NH6nHHePiD3MTdni8H/Th9PWLQACxVfJL/9I4vtkGWFDwyoSao6DJwwXZVfWI
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(366004)(396003)(136003)(376002)(39860400002)(316002)(26005)(31686004)(76116006)(36756003)(5660300002)(83380400001)(110136005)(6506007)(54906003)(2906002)(4326008)(38070700005)(6512007)(122000001)(31696002)(66556008)(6486002)(8676002)(86362001)(64756008)(38100700002)(2616005)(66476007)(8936002)(91956017)(66946007)(71200400001)(186003)(478600001)(66446008)(53546011)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?K1JnQ21jT1RKSTJDWGFhelBXUGNuaUFTMjgrQVdlYzhzZG43ZDRDU0NpQUM2?=
 =?utf-8?B?ZnBKSUUzTmtOWEZjeHA2SzJWcE03SE5Ud2p2cWZNSXFVQUV4alhtNlBsdHNX?=
 =?utf-8?B?TEw0c1RKRzJwTTNHT1QvOHBMdCtxRXVFTjNSekQyOVNucy81cUNGdFRXL08x?=
 =?utf-8?B?MXZBYklrcFFLaThpOTJ6cXlPV1BsK0l3em43WU4zZ05FdFRoclRDZlByOUNM?=
 =?utf-8?B?TUFaeThuMnptY1drMGgxWmhBekhmNnJpV3BUVkVSekxaeVJ0YmxZWDFpelI1?=
 =?utf-8?B?WXcyUjV2Nmt1WjlMWTZhdmRJWjZmeHVjSmxCY0J0VE45LzdIYTB5cG12S29P?=
 =?utf-8?B?bDRYcDlBR1luLzg4U0t4TXJTUEh2c0Y0bjFXaDlXRmZzS05uUW1Xc3F2aUdN?=
 =?utf-8?B?Tm9VVWFRUWNMdmpOaHRsdDFQQmJGZXhlb3JJelF5eTNpdlNOT1FTaC9jeDhm?=
 =?utf-8?B?cnA5MHlKMnYzZkorSURkak11S29XRk5kSm9sNDNpVElscitDY0lleElNNS9B?=
 =?utf-8?B?UWgyUW15YVBKaEt0TTRyVEhYdmNzS1l5TXBvT0ROVjE2MXcvdGQ2KzB2NFRz?=
 =?utf-8?B?MEFQajk3elhLTEE4Q24ySHFxRGx5K3NWUEwzdFdUUUdSSkl4TzBlL0NxV1Fl?=
 =?utf-8?B?ZHRDcnNrTm8wWi9TUE85MGRRWktaS3o2T2VSUTJCT2ljTUxUMkRwbG5nSWl4?=
 =?utf-8?B?Z3M5K3d0dUZNSE9LckZXdHJac0xUcFNpTnhTbmp6YUo3bTRpTXMrQ2NGeWRo?=
 =?utf-8?B?dVk4NThZaDVlRmJKU2RQdkd3MWdvUmM1QnJVVnBCUERxTG4rY0J2OUxkV1Fp?=
 =?utf-8?B?SytUM1BBd1BnNTZFV2JOTWgyUGVVYkVnRFV1U2ZkUHpaYzRhMDFiaHRlbHMz?=
 =?utf-8?B?RXdjR3VucFJ5cXRjYWxXTlM0d2h0c0FXUUVSbkJ2SEpiSzdoeTVWa3RRa3oy?=
 =?utf-8?B?YUlza1p2R0ovNStXNzh0UzQ0ZUZ2c2pXZDk1OFBzUldzeXovYkh3d0tZaFVi?=
 =?utf-8?B?RXNQRmZEU3JOSHpreEpNZFRFMDVUb1VZeFdqLzRJS1pKZUZGdld0TjlobUpK?=
 =?utf-8?B?cUZsWTdOS1JxS1RLZEdmQ2JMK0p2c1FRdVhNTVBtcXRQMHhHSmlQUFFUOUJF?=
 =?utf-8?B?alByS2c4QmNrYUpPd1k0bitKVkMwODZINVpUaktpY0lGRUJZbkpDUVh2a3ZG?=
 =?utf-8?B?Z0h2UTNpZkZ0OUpFMlFHamllNHBVaUVnOWxEZGNmQW9MVE1sTHZrWnJsWWti?=
 =?utf-8?B?clF4S3dmVUNrR1BvemYvcW9TK0VVMkJOOHRpTFV4UmRvcmlaR2ZsSFJHRWZV?=
 =?utf-8?B?bzZUSlBBRzVoMkhBWDh2N283MG1oWXdZN1hqb0pyZE83MUFGdW1takthYTQx?=
 =?utf-8?B?blpTY2tqMklGeGJ0VUxGMmlRbHd0WnlUUHlxQVNNTXd0T0pRK0VxRmxFTVRR?=
 =?utf-8?B?SlZOdDAzNnhwcjZHTjcrKytTclYrWXd1U0IwM1BWMWRTbDlqeVJkQzd5aS9W?=
 =?utf-8?B?a0xWVU5HcVk1eERhc2pSV29VSTBDVE5RM1F0Y3plYjl0ZVlyRUNON0FuVVpu?=
 =?utf-8?B?bTBqQjB2dXZobUFnVjZQemNCLzlyeVIwckNHSkdjZC9GNk1Zd2dyKzJJZzVW?=
 =?utf-8?B?SmhlMmFEUmZ2MXE1YU5FcVVhTUIrUlFrOGt1OXhHWngxQTc1SllOZjJDbFBX?=
 =?utf-8?B?ZitRc3BKV3o0N2wveVB6MTFnOEdqMFMwUWxmcURveWFCNjIwK1U1TU5aSFky?=
 =?utf-8?Q?9wxI1fDGykd9UHD+lgDYuJkeAWFOw5Gv0JNW6gj?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <9B64E875B08D33458C3D2AA62E84616D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1eab9de8-f72b-4729-970d-08d9736ee2d3
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 08:50:33.7645
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: nLPZqDczDRkq5ENj+UUHPLKbc0cOYNM/36lOB94kbj4mcvdB+iC9A04vwGbt8OgKQPcauolVBQ1fbeuq4h9C/jcjzWI8dlZr9/+mLy4qosUInogIKno8ZpUHY6mxNkwY
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3281
X-Proofpoint-GUID: JqSHGRpxHxgnqX7tm9a4Ydnj7Dew-9IE
X-Proofpoint-ORIG-GUID: JqSHGRpxHxgnqX7tm9a4Ydnj7Dew-9IE
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_02,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 priorityscore=1501 bulkscore=0 spamscore=0 clxscore=1015 mlxscore=0
 mlxlogscore=999 phishscore=0 lowpriorityscore=0 malwarescore=0
 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109090053

DQpPbiAwOS4wOS4yMSAxMTo0MywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTA6MzksIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDcuMDkuMjEgMTM6
MDYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA3LjA5LjIwMjEgMTE6NTIsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDExOjA3LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNy4wOS4yMSAxMTo0OSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTA6MTgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gU28sIGlmIHdlIGhhdmUgYSBoaWRkZW4gUENJIGRldmljZSB3aGljaCBj
YW4gYmUgYXNzaWduZWQgdG8gYSBndWVzdCBhbmQgaXQgaXMgbGl0ZXJhbGx5IHVudG91Y2hlZA0K
Pj4+Pj4+Pj4gKG5vdCBlbmFibGVkIGluIERvbTApIHRoZW4gSSB0aGluayB0aGVyZSB3aWxsIGJl
IG5vIHN1Y2ggcmVmZXJlbmNlIGFzICJob3N0IGFzc2lnbmVkIHZhbHVlcyIgYXMNCj4+Pj4+Pj4+
IG1vc3QgcHJvYmFibHkgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgd2lsbCByZW1haW4gaW4gaXRzIGFm
dGVyIHJlc2V0IHN0YXRlLg0KPj4+Pj4+PiBXaGF0IG1lYW5pbmcgb2YgImhpZGRlbiIgZG8geW91
IGltcGx5IGhlcmU/IERldmljZXMgcGFzc2VkIHRvDQo+Pj4+Pj4+IHBjaV97aGlkZSxyb31fZGV2
aWNlKCkgbWF5IG5vdCBiZSBhc3NpZ25lZCB0byBndWVzdHMgLi4uDQo+Pj4+Pj4gWW91IGFyZSBj
b21wbGV0ZWx5IHJpZ2h0IGhlcmUuDQo+Pj4+Pj4+IEZvciBhbnkgb3RoZXIgbWVhbmluZyBvZiAi
aGlkZGVuIiwgZXZlbiBpZiB0aGUgZGV2aWNlIGlzIGNvbXBsZXRlbHkNCj4+Pj4+Pj4gaWdub3Jl
ZCBieSBEb20wLA0KPj4+Pj4+IERvbTBsZXNzIGlzIHN1Y2ggYSBjYXNlIHdoZW4gYSBkZXZpY2Ug
aXMgYXNzaWduZWQgdG8gdGhlIGd1ZXN0DQo+Pj4+Pj4gd2l0aG91dCBEb20wIGF0IGFsbD8NCj4+
Pj4+IEluIHRoaXMgY2FzZSBpdCBpcyBlbnRpcmVseSB1bmNsZWFyIHRvIG1lIHdoYXQgZW50aXR5
IGl0IGlzIHRvIGhhdmUNCj4+Pj4+IGEgZ2xvYmFsIHZpZXcgb24gdGhlIFBDSSBzdWJzeXN0ZW0u
DQo+Pj4+Pg0KPj4+Pj4+PiAgICAgIGNlcnRhaW4gb2YgdGhlIHByb3BlcnRpZXMgc3RpbGwgY2Fu
bm90IGJlIGFsbG93ZWQNCj4+Pj4+Pj4gdG8gYmUgRG9tVS1jb250cm9sbGVkLg0KPj4+Pj4+IFRo
ZSBsaXN0IGlzIG5vdCB0aGF0IGJpZywgY291bGQgeW91IHBsZWFzZSBuYW1lIGEgZmV3IHlvdSB0
aGluayBjYW5ub3QNCj4+Pj4+PiBiZSBjb250cm9sbGVkIGJ5IGEgZ3Vlc3Q/IEkgY2FuIHRoaW5r
IG9mIFBDSV9DT01NQU5EX1NQRUNJQUwoPyksDQo+Pj4+Pj4gUENJX0NPTU1BTkRfSU5WQUxJREFU
RSg/KSwgUENJX0NPTU1BTkRfUEFSSVRZLCBQQ0lfQ09NTUFORF9XQUlULA0KPj4+Pj4+IFBDSV9D
T01NQU5EX1NFUlIsIFBDSV9DT01NQU5EX0lOVFhfRElTQUJMRSB3aGljaCB3ZSBtYXkgd2FudCB0
bw0KPj4+Pj4+IGJlIGFsaWduZWQgd2l0aCB0aGUgImhvc3QgcmVmZXJlbmNlIiB2YWx1ZXMsIGUu
Zy4gd2Ugb25seSBhbGxvdyB0aG9zZSBiaXRzDQo+Pj4+Pj4gdG8gYmUgc2V0IGFzIHRoZXkgYXJl
IGluIERvbTAuDQo+Pj4+PiBXZWxsLCB5b3UndmUgY29tcGlsZSBhIGxpc3QgYWxyZWFkeSwgYW5k
IEkgZGlkIHNheSBzbyBiZWZvcmUgYXMgd2VsbDoNCj4+Pj4+IEV2ZXJ5dGhpbmcgZXhjZXB0IEkv
TyBhbmQgbWVtb3J5IGRlY29kaW5nIGFzIHdlbGwgYXMgYnVzIG1hc3RlcmluZw0KPj4+Pj4gbmVl
ZHMgYXQgbGVhc3QgY2xvc2VseSBsb29raW5nIGF0LiBJTlRYX0RJU0FCTEUsIGZvciBleGFtcGxl
LCBpcw0KPj4+Pj4gc29tZXRoaW5nIEkgZG9uJ3QgdGhpbmsgYSBndWVzdCBzaG91bGQgYmUgYWJs
ZSB0byBkaXJlY3RseSBjb250cm9sLg0KPj4+Pj4gSXQgbWF5IHN0aWxsIGJlIHRoZSBjYXNlIHRo
YXQgdGhlIGhvc3QgcGVybWl0cyBpdCBjb250cm9sLCBidXQgdGhlbg0KPj4+Pj4gb25seSBpbmRp
cmVjdGx5LCBhbGxvd2luZyB0aGUgaG9zdCB0byBhcHByb3ByaWF0ZWx5IGFkanVzdCBpdHMNCj4+
Pj4+IGludGVybmFscy4NCj4+Pj4+DQo+Pj4+PiBOb3RlIHRoYXQgZXZlbiBmb3IgSS9PIGFuZCBt
ZW1vcnkgZGVjb2RpbmcgYXMgd2VsbCBhcyBidXMgbWFzdGVyaW5nDQo+Pj4+PiBpdCBtYXkgYmUg
bmVjZXNzYXJ5IHRvIGxpbWl0IGd1ZXN0IGNvbnRyb2w6IEluIGNhc2UgdGhlIGhvc3Qgd2FudHMN
Cj4+Pj4+IHRvIGRpc2FibGUgYW55IG9mIHRoZXNlIChwZXJoYXBzIHRyYW5zaWVudGx5KSBkZXNw
aXRlIHRoZSBndWVzdA0KPj4+Pj4gd2FudGluZyB0aGVtIGVuYWJsZWQuDQo+Pj4+IE9rLCBzbyBp
dCBpcyBub3cgY2xlYXIgdGhhdCB3ZSBuZWVkIGEgeWV0IGFub3RoZXIgcGF0Y2ggdG8gYWRkIGEg
cHJvcGVyDQo+Pj4+IGNvbW1hbmQgcmVnaXN0ZXIgZW11bGF0aW9uLiBXaGF0IGlzIHlvdXIgcHJl
ZmVyZW5jZTogZHJvcCB0aGUgY3VycmVudA0KPj4+PiBwYXRjaCwgaW1wbGVtZW50IGNvbW1hbmQg
cmVnaXN0ZXIgZW11bGF0aW9uIGFuZCBhZGQgYSAicmVzZXQgcGF0Y2giDQo+Pj4+IGFmdGVyIHRo
YXQgb3Igd2UgY2FuIGhhdmUgdGhlIHBhdGNoIGFzIGlzIG5vdywgYnV0IEknbGwgb25seSByZXNl
dCBJTy9tZW0gYW5kIGJ1cw0KPj4+PiBtYXN0ZXIgYml0cywgZS5nLiByZWFkIHRoZSByZWFsIHZh
bHVlLCBtYXNrIHRoZSB3YW50ZWQgYml0cyBhbmQgd3JpdGUgYmFjaz8NCj4+PiBFaXRoZXIgb3Jk
ZXIgaXMgZmluZSB3aXRoIG1lIGFzIGxvbmcgYXMgdGhlIHJlc3VsdCB3aWxsIGJlIGNsYWltZWQg
dG8NCj4+PiBiZSBjb21wbGV0ZSB1bnRpbCBwcm9wZXIgZW11bGF0aW9uIGlzIGluIHBsYWNlLg0K
Pj4gSSB0cmllZCB0byBzZWUgd2hhdCBvdGhlcnMgZG8gaW4gb3JkZXIgdG8gZW11bGF0ZSBQQ0lf
Q09NTUFORCByZWdpc3Rlcg0KPj4gYW5kIGl0IHNlZW1zIHRoYXQgYXQgbW9zdCB0aGV5IGNhcmUg
YWJvdXQgdGhlIG9ubHkgSU5UWCBiaXQgKGJlc2lkZXMNCj4+IElPL21lbW9yeSBlbmFibGUgYW5k
IGJ1cyBtdXN0ZXIgd2hpY2ggYXJlIHdyaXRlIHRocm91Z2gpLiBQbGVhc2Ugc2VlDQo+PiBbMV0g
YW5kIFsyXS4gUHJvYmFibHkgSSBtaXNzIHNvbWV0aGluZywgYnV0IGl0IGNvdWxkIGJlIGJlY2F1
c2UgaW4gb3JkZXINCj4+IHRvIHByb3Blcmx5IGVtdWxhdGUgdGhlIENPTU1BTkQgcmVnaXN0ZXIg
d2UgbmVlZCB0byBrbm93IGFib3V0IHRoZQ0KPj4gd2hvbGUgUENJIHRvcG9sb2d5LCBlLmcuIGlm
IGFueSBzZXR0aW5nIGluIGRldmljZSdzIGNvbW1hbmQgcmVnaXN0ZXINCj4+IGlzIGFsaWduZWQg
d2l0aCB0aGUgdXBzdHJlYW0gcG9ydCBldGMuIFRoaXMgbWFrZXMgbWUgdGhpbmsgdGhhdCBiZWNh
dXNlDQo+PiBvZiB0aGlzIGNvbXBsZXhpdHkgb3RoZXJzIGp1c3QgaWdub3JlIHRoYXQuIE5laXRo
ZXIgSSB0aGluayB0aGlzIGNhbiBiZQ0KPj4gZWFzaWx5IGRvbmUgaW4gb3VyIGNhc2UuIFNvIEkg
d291bGQgc3VnZ2VzdCB3ZSBqdXN0IGFkZCB0aGUgZm9sbG93aW5nDQo+PiBzaW1wbGUgbG9naWMg
dG8gb25seSBlbXVsYXRlIFBDSV9DT01NQU5EX0lOVFhfRElTQUJMRTogYWxsb3cgZ3Vlc3QgdG8N
Cj4+IGRpc2FibGUgdGhlIGludGVycnVwdHMsIGJ1dCBkb24ndCBhbGxvdyB0byBlbmFibGUgaWYg
aG9zdCBoYXMgZGlzYWJsZWQNCj4+IHRoZW0uIFRoaXMgaXMgYWxzbyBjb3VsZCBiZSB0cmlja3kg
YSBiaXQgZm9yIHRoZSBkZXZpY2VzIHdoaWNoIGFyZSBub3QNCj4+IGVuYWJsZWQgYW5kIHRodXMg
bm90IGNvbmZpZ3VyZWQgaW4gRG9tMCwgZS5nLiB3ZSBkbyBub3Qga25vdyBmb3Igc3VyZQ0KPj4g
aWYgdGhlIHZhbHVlIGluIHRoZSBQQ0lfQ09NTUFORCByZWdpc3RlciAoaW4gcGFydGljdWxhcg0K
Pj4gUENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFIGJpdCkgY2FuIGJlIHVzZWQgYXMgdGhlIHJlZmVy
ZW5jZSBob3N0IHZhbHVlIG9yDQo+PiBub3QuIEl0IGNhbiBiZSB0aGF0IHRoZSB2YWx1ZSB0aGVy
ZSBpcyBqdXN0IHRoZSBvbmUgYWZ0ZXIgcmVzZXQgb3Igc28uDQo+PiBUaGUgcmVzdCBvZiB0aGUg
Y29tbWFuZCByZWdpc3RlciBiaXRzIHdpbGwgZ28gZGlyZWN0bHkgdG8gdGhlIGNvbW1hbmQNCj4+
IHJlZ2lzdGVyIHVudG91Y2hlZC4NCj4+IFNvLCBhdCB0aGUgZW5kIG9mIHRoZSBkYXkgdGhlIHF1
ZXN0aW9uIGlzIGlmIFBDSV9DT01NQU5EX0lOVFhfRElTQUJMRQ0KPj4gaXMgZW5vdWdoIGFuZCBo
b3cgdG8gZ2V0IGl0cyByZWZlcmVuY2UgaG9zdCB2YWx1ZS4NCj4gV2VsbCwgaW4gb3JkZXIgZm9y
IHRoZSB3aG9sZSB0aGluZyB0byBiZSBzZWN1cml0eSBzdXBwb3J0ZWQgaXQgbmVlZHMgdG8NCj4g
YmUgZXhwbGFpbmVkIGZvciBldmVyeSBiaXQgd2h5IGl0IGlzIHNhZmUgdG8gYWxsb3cgdGhlIGd1
ZXN0IHRvIGRyaXZlIGl0Lg0KDQpTbywgZG8gd2Ugd2FudCBhdCBsZWFzdCBQQ0lfQ09NTUFORF9J
TlRYX0RJU0FCTEUgYml0IGFsaWduZWQNCg0KYmV0d2VlbiB0aGUgaG9zdCBhbmQgZ3Vlc3Q/IElm
IHNvLCB3aGF0IGRvIHlvdSB5b3UgdGhpbmsgYWJvdXQNCg0KdGhlIHJlZmVyZW5jZSB2YWx1ZSBm
b3IgaXQgKHBsZWFzZSBzZWUgYWJvdmUpLg0KDQo+IFVudGlsIHlvdSBtZWFuIHZQQ0kgdG8gcmVh
Y2ggdGhhdCBzdGF0ZSwgbGVhdmluZyBUT0RPIG5vdGVzIGluIHRoZSBjb2RlDQo+IGZvciBhbnl0
aGluZyBub3QgaW52ZXN0aWdhdGVkIG1heSBpbmRlZWQgYmUgZ29vZCBlbm91Z2guDQpPaywgSSds
bCBhZGQgVE9ETyB0aGVuLg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 08:52:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 08:52:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182806.330582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFnH-0008GY-4K; Thu, 09 Sep 2021 08:52:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182806.330582; Thu, 09 Sep 2021 08:52:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFnH-0008GP-16; Thu, 09 Sep 2021 08:52:27 +0000
Received: by outflank-mailman (input) for mailman id 182806;
 Thu, 09 Sep 2021 08:52:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOFnF-0008GH-32
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 08:52:25 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6b05beb0-d3c3-4c89-88f6-291e1a1de93d;
 Thu, 09 Sep 2021 08:52:23 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 163F22003C;
 Thu,  9 Sep 2021 08:52:23 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id C1F9113727;
 Thu,  9 Sep 2021 08:52:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id yXpaLUbLOWFEGwAAGKfGzw
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 08:52:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b05beb0-d3c3-4c89-88f6-291e1a1de93d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631177543; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=LTz2n8rnEh9zuVBxWj0uzzqkPGN4y4TSI1dOQATsEQs=;
	b=TYvxsvWxf5tIj+R4LO2q5sZaKnFYXStc6hD54eSWWbkYP2y3XztWJrWXcc6h6QLZOpcRP4
	aKiSJ9cWZhjqAFv03CUepgYO8CUuhr6OW90FWAvSScebcCRl0H7RwrLk47WdmBzW0ULVRH
	I2KJgXVeip2YNknifrqo0cS+dvmDrSg=
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Samuel Thibault <samuel.thibault@ens-lyon.org>,
 xen-devel@lists.xenproject.org
References: <20210909082705.9434-1-jgross@suse.com>
 <e1004466-5d12-efd6-8729-27e72ad6f7d8@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 0/2] stubdom: disable building of pv-grub per default
Message-ID: <164af20b-62f1-d565-968f-560f42c27442@suse.com>
Date: Thu, 9 Sep 2021 10:52:22 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <e1004466-5d12-efd6-8729-27e72ad6f7d8@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="1RBr44I0AbkDVGT7RJ7ZP2oaRteQkoT53"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--1RBr44I0AbkDVGT7RJ7ZP2oaRteQkoT53
Content-Type: multipart/mixed; boundary="KQJolj9nUp2sXr3agwboF2pY2kBCe1f08";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Samuel Thibault <samuel.thibault@ens-lyon.org>,
 xen-devel@lists.xenproject.org
Message-ID: <164af20b-62f1-d565-968f-560f42c27442@suse.com>
Subject: Re: [PATCH 0/2] stubdom: disable building of pv-grub per default
References: <20210909082705.9434-1-jgross@suse.com>
 <e1004466-5d12-efd6-8729-27e72ad6f7d8@suse.com>
In-Reply-To: <e1004466-5d12-efd6-8729-27e72ad6f7d8@suse.com>

--KQJolj9nUp2sXr3agwboF2pY2kBCe1f08
Content-Type: multipart/mixed;
 boundary="------------D6E94ECF7A5B74444CDA3231"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------D6E94ECF7A5B74444CDA3231
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 09.09.21 10:44, Jan Beulich wrote:
> On 09.09.2021 10:27, Juergen Gross wrote:
>> This is a first step of deprecating pv-grub. Switch the default to not=

>> building it.
>>
>> NOTE: This should be mentioned in the release notes!
>=20
> And/or perhaps in ./CHANGELOG.md?

Oh yes, of course!

I'll add this to patch 2.


Juergen

--------------D6E94ECF7A5B74444CDA3231
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------D6E94ECF7A5B74444CDA3231--

--KQJolj9nUp2sXr3agwboF2pY2kBCe1f08--

--1RBr44I0AbkDVGT7RJ7ZP2oaRteQkoT53
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE5y0YFAwAAAAAACgkQsN6d1ii/Ey/K
mQf+NqfWgv5CTyFOy1jwHZ9RGkqKpwRSmqpXxK8BCzEnIl4UtwxV9RRbk0nPjy+hflArtOdsggeH
HUsPW47N4CIkYXVDNdVIlass2/kF9MyKSPBDsbe3STIDJiimU6N/5gtjJ6M83oMqec029eInrAmo
ioHdIDnQpSlKxuLiataH69IHdeYiulrIvodivLM41jw2b1QXySsin+l6cEf3xaoH65Ixg0nKyWNm
2EalViOGSKIerJdMqN4ULxykKC3YLZOOAkllZY0uvzS0CpY4Dbg0l7L6dhl6BMz1/vWxa9bYCAoE
MK4HEhu6NTotD2DfA1hqAcAXQLfBDnmBkHlTDxYAUg==
=Jp5+
-----END PGP SIGNATURE-----

--1RBr44I0AbkDVGT7RJ7ZP2oaRteQkoT53--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:03:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182815.330598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFyH-0001eD-86; Thu, 09 Sep 2021 09:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182815.330598; Thu, 09 Sep 2021 09:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFyH-0001e6-56; Thu, 09 Sep 2021 09:03:49 +0000
Received: by outflank-mailman (input) for mailman id 182815;
 Thu, 09 Sep 2021 09:03:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+l3J=N7=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mOFyG-0001e0-7u
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:03:48 +0000
Received: from mail2-relais-roc.national.inria.fr (unknown [192.134.164.83])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2d28ec08-0786-4d54-8c32-180c343af04a;
 Thu, 09 Sep 2021 09:03:47 +0000 (UTC)
Received: from unknown (HELO begin) ([193.50.111.40])
 by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 09 Sep 2021 11:03:43 +0200
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mOFyB-00FcqH-Gr;
 Thu, 09 Sep 2021 11:03:43 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d28ec08-0786-4d54-8c32-180c343af04a
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AdESmtaCmo3oSFmLlHemu55DYdb4zR+YMi2TD?=
 =?us-ascii?q?GXocdfUnSL3+qynOpoV+6faaslYssR0b9uxofZPwJE80lqQFg7X5X43CYOCOgg?=
 =?us-ascii?q?LBEGgh1+HfKlbbcBEWmNQx6U4tSdkcNDWsYGIVsfrH?=
X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; 
   d="scan'208";a="527510806"
Date: Thu, 9 Sep 2021 11:03:43 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 1/2] stubdom: fix build with disabled pv-grub
Message-ID: <20210909090343.fqlc5vt35kfkqru6@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210909082705.9434-1-jgross@suse.com>
 <20210909082705.9434-2-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210909082705.9434-2-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le jeu. 09 sept. 2021 10:27:04 +0200, a ecrit:
> Today the build will fail if --disable-pv-grub as a parameter of
> configure, as the main Makefile will unconditionally try to build a
> 32-bit pv-grub stubdom.
> 
> Fix that by introducing a pv-grub32 target in stubdom/Makefile taking
> care of this situation.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
>  Makefile         |  4 ++--
>  stubdom/Makefile | 13 +++++++++++++
>  2 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 96d32cfd50..5b5cef3e49 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -72,7 +72,7 @@ build-tools-oxenstored: build-tools-public-headers
>  build-stubdom: mini-os-dir build-tools-public-headers
>  	$(MAKE) -C stubdom build
>  ifeq (x86_64,$(XEN_TARGET_ARCH))
> -	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub
> +	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub32
>  endif
>  
>  .PHONY: build-docs
> @@ -143,7 +143,7 @@ install-tools: install-tools-public-headers
>  install-stubdom: mini-os-dir install-tools
>  	$(MAKE) -C stubdom install
>  ifeq (x86_64,$(XEN_TARGET_ARCH))
> -	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
> +	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub32
>  endif
>  
>  .PHONY: tools/firmware/seabios-dir-force-update
> diff --git a/stubdom/Makefile b/stubdom/Makefile
> index 06aa69d8bc..b339ae701c 100644
> --- a/stubdom/Makefile
> +++ b/stubdom/Makefile
> @@ -531,6 +531,13 @@ vtpmmgr-stubdom: mini-os-$(XEN_TARGET_ARCH)-vtpmmgr vtpmmgr
>  pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxenguest grub
>  	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
>  
> +.PHONY: pv-grub32
> +ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
> +pv-grub32: pv-grub
> +else
> +pv-grub32:
> +endif
> +
>  .PHONY: xenstore-stubdom
>  xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxenguest xenstore
>  	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
> @@ -560,6 +567,12 @@ install-grub: pv-grub
>  	$(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)"
>  	$(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz"
>  
> +ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
> +install-grub32: install-grub
> +else
> +install-grub32:
> +endif
> +
>  install-c: c-stubdom
>  
>  install-caml: caml-stubdom
> -- 
> 2.26.2
> 

-- 
Samuel
The nice thing about Windows is - It does not just crash, it displays a
dialog box and lets you press 'OK' first.
(Arno Schaefer's .sig)


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:04:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:04:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182821.330610 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFys-0002CS-H4; Thu, 09 Sep 2021 09:04:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182821.330610; Thu, 09 Sep 2021 09:04:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOFys-0002CL-E1; Thu, 09 Sep 2021 09:04:26 +0000
Received: by outflank-mailman (input) for mailman id 182821;
 Thu, 09 Sep 2021 09:04:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+l3J=N7=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mOFyr-0002CD-My
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:04:25 +0000
Received: from mail3-relais-sop.national.inria.fr (unknown [192.134.164.104])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 656a4802-07c2-424d-b89c-cc84c02c5f78;
 Thu, 09 Sep 2021 09:04:23 +0000 (UTC)
Received: from unknown (HELO begin) ([193.50.111.40])
 by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 09 Sep 2021 11:04:22 +0200
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mOFyn-00FcqY-VV;
 Thu, 09 Sep 2021 11:04:22 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 656a4802-07c2-424d-b89c-cc84c02c5f78
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AArLU5K5mlMIo9WYeCAPXwOHXdLJyesId70hD?=
 =?us-ascii?q?6qm+c3Nom+ij5qSTdZUgpHrJYVkqMk3I9ersBEDEewK6yXcX2/hyAZ6HVAni/E?=
 =?us-ascii?q?SvKYRl85f6hwDnchefygc+78ZdmnhFebvN5DFB5K6Q3OD7KadC/DDtys6VbInl?=
 =?us-ascii?q?o0uFATsaEp1d0w=3D=3D?=
X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; 
   d="scan'208";a="392481966"
Date: Thu, 9 Sep 2021 11:04:21 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH 2/2] stubdom: disable building pv-grub
Message-ID: <20210909090421.yn222xbmci425box@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
	Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210909082705.9434-1-jgross@suse.com>
 <20210909082705.9434-3-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210909082705.9434-3-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le jeu. 09 sept. 2021 10:27:05 +0200, a ecrit:
> The stubdom based pv-grub is using a very outdated version of grub
> (0.97) and should not be used any longer. Mainline grub has support for
> PV guests for a long time now, so that should be used as a boot loader
> of a PV domain.
> 
> So disable building pv-grub per default. In case someone really wants
> to continue using it he/she can still use a pv-grub binary from an older
> Xen version or manually enable building it via:
> 
>   configure --enable-pv-grub
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
>  stubdom/configure    | 8 ++------
>  stubdom/configure.ac | 2 +-
>  2 files changed, 3 insertions(+), 7 deletions(-)
> 
> diff --git a/stubdom/configure b/stubdom/configure
> index aa48df986d..df31532abb 100755
> --- a/stubdom/configure
> +++ b/stubdom/configure
> @@ -1342,7 +1342,7 @@ Optional Features:
>    --enable-ioemu-stubdom  Build and install ioemu-stubdom
>    --enable-c-stubdom      Build and install c-stubdom (default is DISABLED)
>    --enable-caml-stubdom   Build and install caml-stubdom (default is DISABLED)
> -  --disable-pv-grub       Build and install pv-grub (default is ENABLED)
> +  --enable-pv-grub        Build and install pv-grub (default is DISABLED)
>    --disable-xenstore-stubdom
>                            Build and install xenstore-stubdom (default is
>                            ENABLED)
> @@ -2129,11 +2129,7 @@ fi
>  else
>  
>  
> -grub=y
> -STUBDOM_TARGETS="$STUBDOM_TARGETS grub"
> -STUBDOM_BUILD="$STUBDOM_BUILD pv-grub"
> -STUBDOM_INSTALL="$STUBDOM_INSTALL install-grub"
> -STUBDOM_UNINSTALL="$STUBDOM_UNINSTALL install-grub"
> +grub=n
>  
>  
>  fi
> diff --git a/stubdom/configure.ac b/stubdom/configure.ac
> index bd6f765929..a07a1edae5 100644
> --- a/stubdom/configure.ac
> +++ b/stubdom/configure.ac
> @@ -21,7 +21,7 @@ m4_include([../m4/fetcher.m4])
>  AX_STUBDOM_CONDITIONAL([ioemu-stubdom], [ioemu])
>  AX_STUBDOM_DEFAULT_DISABLE([c-stubdom], [c])
>  AX_STUBDOM_DEFAULT_DISABLE([caml-stubdom], [caml])
> -AX_STUBDOM_DEFAULT_ENABLE([pv-grub], [grub])
> +AX_STUBDOM_DEFAULT_DISABLE([pv-grub], [grub])
>  AX_STUBDOM_DEFAULT_ENABLE([xenstore-stubdom], [xenstore])
>  AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
>  AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
> -- 
> 2.26.2
> 

-- 
Samuel
<Raize> can you guys see what I type?
<vecna> no, raize
<Raize> How do I set it up so you can see it?


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:06:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:06:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182827.330621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG13-0002re-Uo; Thu, 09 Sep 2021 09:06:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182827.330621; Thu, 09 Sep 2021 09:06:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG13-0002rX-Q5; Thu, 09 Sep 2021 09:06:41 +0000
Received: by outflank-mailman (input) for mailman id 182827;
 Thu, 09 Sep 2021 09:06:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOG12-0002rR-Gw
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:06:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3dc8cc4a-114d-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 09:06:39 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-4-zgW_6RbmMhSL5WgH_llk-w-1; Thu, 09 Sep 2021 11:06:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 09:06:35 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:06:35 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0081.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Thu, 9 Sep 2021 09:06:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dc8cc4a-114d-11ec-b1a7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631178398;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=luE6vr4B2Em6B7HB3xH6vsWXRYWQjWo9rnvd6ZwmLXM=;
	b=RXchymbA8tqoraEaQpqFC9kw+G3twF5R+4nAY2KNHtujyxxN01fBcDSCZobIYlmB4GZSNw
	SBbHh10bxlPymOmEcYu34j+YkrPnTsKqfUBNPENKuM+sKnALnafXLcvf4Aei9Y+gEKWMiB
	QVuXpOKw2qDSjFV4Ri0+YkDXp8jxGMc=
X-MC-Unique: zgW_6RbmMhSL5WgH_llk-w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T+/SGkWEdRYrSUOOilA2ADlXJ3RdabHx4b3S2gRAa9NnoQvkHZlHn8QynjsVE773UNW1HacaOFtlvJnwbJCsdTAz6rOg8a7nGAVQERsS+WxoHoA9QVnFSMkkBAGc2+zmwGwQQp4+edw7ujD9nLNeQVST0yxOybdMLZvzAiMH80/DvigDiviBME/Zf9s5kEzQscmaEB6K1jcvz5UGLaKpO6Ngy3ga76xycNZyDrWcKqUhGvzEWP31W1bR/qeWonaZUYGrjw2gVquEdCf3pgPF0AnIv/IjelJ20lPmZ8uGGfmYGClApgpnelt7x369fqkxjhbUuqtmfbRyhLqBNqDh5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=luE6vr4B2Em6B7HB3xH6vsWXRYWQjWo9rnvd6ZwmLXM=;
 b=VPFp1S46PMqMCWiwjgs6WIEwi+bQxgHg3mJTosBu+4xdbc3uFHoxBkVqUm92NY1A4B1akXMxPLBhiqLOv03CBdUXJ6ZyWRJ5tMjbvFQR8Otj5ZP42WaFxCZZVBavDwY69diqjO5R//DlOODTGq1svx43WFSAk93DwbteOpJCAMn4ax7Ga5SyRk5R5SABlliFJ4wbsbxdyN/KLaS0/fPAqBH8oP/bwIfzfkD3PmYedkNgn7rf93g9Id+ZOnMsNBey2TyTE7J46NlLo0gpddOcuJvnn6McEdu8uQ3EoW4YWckKsyw9HoEXA7ze+bZJHIpjgqXiLuJqhu9VP/GeZWEgFw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
To: Penny Zheng <penny.zheng@arm.com>
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com,
 xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org
References: <20210908095248.545981-1-penny.zheng@arm.com>
 <20210908095248.545981-6-penny.zheng@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <da3dafd3-9e61-a7d5-b5cf-897d73d0fd16@suse.com>
Date: Thu, 9 Sep 2021 11:06:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210908095248.545981-6-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0081.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a3670a37-31d6-4c8b-abe2-08d973711fd3
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686176E7CFCA8BF203C1235CB3D59@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FBgUTYTyyavtVar2YNqd7nHvbbJQA5kvGoKZpEIbRg77E9OFdaDO19gSmiUY6IXtkf8M8qRP6X+fzv5whXoyB+ZSOWS3/bHJtdLXC55tZ0KEKn7aWOa/nbYAz+ND+shAsTmzCqrxDSanuSSM9q/D+5KhFVc1tyr7DDFtH0VnwqLP7xYMWoj9FyxjwM+F/zrgJaGDl7vY6BwyT+4uvlW+ZAt+sWCZ6kdFP0agj2IoksWBt+MB/v6wVp5pmP6fgMWPzs9Y17lDT6UYhXDvHmOd5F68Z+hKBq7v3/Pzbcy90OrtP4CwMjXq8eM4ss3rsjLj0CAeu2ki/4m5zkEZKCqIew/N96/YNpImGtka1SKxou6hUGRE9kjCKZsPQbYGOyvcINfjkq2a5zFzwUGEGzlcKXeXW9WIFpFfC/llZXJX/nAGhD+Kzg0AdqAp4XxV/hyfGABCYBpSw6lgEwyJ28kQNRAimeD3OXcERP0wjcHMqaGO1GVdLyB396Ii7/LpBr3ScjHD9xM1Q6Ue34Kiw3+38zCD9N65za8OXQzcygR52eZK7bK3dHoR5UfqzjKKMw5XDXfDXIcq9Q1Sl9ijCGbmddK6/ahPxofqDayN7S/+zklsduy30ymodIOFDPs3Zwzf6E4fwicRsOUBLmVs4IRUWGYIUzzrW7mnsMVVu8EccR38mwzVDqBPIq3lb2aNTioLrVVI7FafQ3XJ6lDilExiz0XLY7QDXXvfmHURasriY78=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(136003)(396003)(366004)(36756003)(2906002)(2616005)(53546011)(8936002)(16576012)(6486002)(478600001)(956004)(38100700002)(5660300002)(31686004)(86362001)(26005)(6916009)(186003)(31696002)(4326008)(316002)(8676002)(66476007)(83380400001)(66946007)(6666004)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SW1sdHRBdVBaQXRNNmpISVhiY3NhektDQ3FEQkNqRjgrZzVhWWg3Q242MkZ6?=
 =?utf-8?B?ODRNYnh0V3h1YUJDelFtUEFVandSNjVnallWb2U1UHhDUzA5QzZ0dkI1WXdy?=
 =?utf-8?B?U0k0bXJ1b0l0dTcvWVhBZzBZcFJtVjh5bEp1RHMxLy9WeVRiLytjSEJLUWlS?=
 =?utf-8?B?Y1FEcDZ3UnNZYm5lUldiemYwOGQ2dzNNOFkwL3dEQ0dGcHFjQmhkcmZFR3c0?=
 =?utf-8?B?dUpQTzZaK29sNlhjaUFUOGlXWTZhRkZ3WlNUZG1YNXJQYnNlVjlxTmV3VWUx?=
 =?utf-8?B?cTZvVDB2S1M4UEhqL3UrUWxSd1l2MFNGT3RMK09tMXRKOXd1Y0xzNm9KS2E2?=
 =?utf-8?B?NUFIQ1lQOUVPTVR1T05XbjE1ODNtVCtLU1VSNXFPaXIxYll3LzVkZUh6NmR1?=
 =?utf-8?B?R0pSc0l3b0RTbDcrRWVqajMxVHhpMHdvazVZc1ZvMjBDVnR2L200K05VUzFj?=
 =?utf-8?B?dDVYb2htV3haQkd4djdDYnF1R3A3NU9TK1B3STQ4UitIOVRJRGVyc3lvZmp3?=
 =?utf-8?B?eTJQaklEMEZ6eTd4VU5KQXlrazhCWnhORWNkalFTMW1iekdkVXpvQUdHU3V1?=
 =?utf-8?B?Z0k0TXJMNmpxSDczaWg1dllmeFVJY1RDaFdpMTlLbHRTTkJiYXBrYnU5MW4v?=
 =?utf-8?B?MTh0SEx1QzZrbzFNNGZSRzFsZFdvKzJkOEhzYTRpdTdGYmZ0NE5lZFlDOFlL?=
 =?utf-8?B?eE0xb3hUQXdhamJuMWxWV3VTTW5RbkcxMzhyZDJ6T1R4Y1dLZ25LNTVsTXZK?=
 =?utf-8?B?UlpHbGk4MkZ0cyt6eVlOYlIrS09DaGFmODJScU1wbzFQd05oWVRQNElwcXQv?=
 =?utf-8?B?STh2Q3BYMlRVWTdDMGdFSW5UUXJPOWlVNjlkOUdCYjZqREF5VTN1UWN3TEFV?=
 =?utf-8?B?K0RsankxRXM0Z1JoSVdIRXR1SGo4UGRzU2Jmem5VL3VBWVVOT05TbjFmTmU2?=
 =?utf-8?B?L0ZtOUhXOEpOaDdEcXY4ZVkraTdibUo5Z0hNbHJibzVuWDFmVFk5czByVGRX?=
 =?utf-8?B?YTR0ZlRzTTkzQTY5ZmhidllHeVMyRWpMVllMR1M0UTNwZE44Q09VT1RxQ2Ri?=
 =?utf-8?B?cSt2ZEZLLzRRb08rNlNZcGI0aWVhMWpWRlJxRmorQ0Y3S0JMUTZFOHNybVox?=
 =?utf-8?B?Q3VCMXRQaFJJSERQWFpIOWN3eEpycDZQNGNxNkwyVXJXU2JrYjBnTXRkTWFW?=
 =?utf-8?B?SzlPcm9LZUVlTTJ4anBnaU96ZksvT0dWRDM1dExoTmFYeGdSOUl5UVNJakEz?=
 =?utf-8?B?UXdjb0R3NHFQV1lWaHdTRjFmZnpZZVZhdU1KRlpxNXBzbVF6M2VwcW5qNGhJ?=
 =?utf-8?B?U1lNdk9pNFFzNGxjL01sUXdTeG1kNXdxREhac1hVWCtxanQyZ2FMUnpFQTg4?=
 =?utf-8?B?eElDVXQ1TkRwNGZqZGVDSldHRVdjV1NtSUU4WDN2Njd2MWdaNGhJYk4xRkor?=
 =?utf-8?B?TVBIb0pnRXZjbW8xR0RveHFzcitCalZhU3Y5NFhJaU5USHVqcVlIaFY0VHRC?=
 =?utf-8?B?djhXRUdLNXNvZU1RZk4vd2VFTEt0MmVzWnVMeE11RGdoMGFCUVpqZkJhYmJC?=
 =?utf-8?B?VEFIOUh0R0VYMXhrQ0dBVEFpMzVEV3p3YXJMOUZjOXJMSWFzUlpnMDIwS3lH?=
 =?utf-8?B?cmltL2I0TjlxYVRaamdpQmI2NFFHVnpJOEpJWllGOWZraXgwdlhIUHNuOWJW?=
 =?utf-8?B?eXprWEhvcVR5eWJFcGd0R0VXQUtSd3lUbzhES3I3WExqZVhqbURtQUIxTkx1?=
 =?utf-8?Q?9K1n5RMTobtL+WwUPVZnbsY92WbCHJVheY2cev/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a3670a37-31d6-4c8b-abe2-08d973711fd3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:06:35.3179
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EqiQadXNCsUMRIz4B3auIlfRWO5PqFN0mfsM1UiMCkpM9/RKuEQ2z2oeozPWqGCStnnlOgkAYoGTqETYKnVbvQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 08.09.2021 11:52, Penny Zheng wrote:
> In order to deal with the trouble of count-to-order conversion when page number
> is not in a power-of-two, this commit re-define assign_pages for nr pages and
> introduces a new helper assign_page for original page with a single order.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>

I have to admit that I'm now very puzzled: Instead of restoring the
long agreed upon ordering of parameters (and then keeping my A-b),
you've dropped the ack.

> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -568,7 +568,7 @@ int __init dom0_construct_pv(struct domain *d,
>          else
>          {
>              while ( count-- )
> -                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 0, 0) )
> +                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 1, 0) )

This change alone demonstrates the problem when it comes to backporting
future changes: If the original patch contained a code addition similar
to what you change to, without the person doing the backporting paying
close attention, the result will be an order-1 request when an order-0
one is wanted. It was explained to you that in order to make people
doing backports aware of this semantic change, the order of parameters
of the function ought to be altered. That way the compiler will
complain, and the person will know to look closely what adjustments are
needed.

In this context I find it further puzzling ...

> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -133,8 +133,14 @@ void heap_init_late(void);
>  
>  int assign_pages(
>      struct domain *d,
> +    struct page_info *pg,
> +    unsigned long nr,
> +    unsigned int memflags);
> +
> +int assign_page(
>      struct page_info *pg,
>      unsigned int order,
> +    struct domain *d,
>      unsigned int memflags);

... that you also neglected the request to harmonize the argument order
of both functions. What we want (and what I thought has long been agreed
upon) is e.g.

int assign_pages(
    struct page_info *pg,
    unsigned long nr,
    struct domain *d,
    unsigned int memflags);

int assign_page(
    struct page_info *pg,
    unsigned int order,
    struct domain *d,
    unsigned int memflags);

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:08:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:08:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182834.330632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG39-0003Zx-FL; Thu, 09 Sep 2021 09:08:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182834.330632; Thu, 09 Sep 2021 09:08:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG39-0003Zq-B5; Thu, 09 Sep 2021 09:08:51 +0000
Received: by outflank-mailman (input) for mailman id 182834;
 Thu, 09 Sep 2021 09:08:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOG37-0003Zk-EM
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:08:49 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOG37-0001of-B1; Thu, 09 Sep 2021 09:08:49 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOG37-0006uy-5d; Thu, 09 Sep 2021 09:08:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=mhOfLDnuYzrnywWhoV3AZ8H8cwhqEJE5ro+NGfTKMGI=; b=FBld4M8HJQAomyeXuFLu4wEKKA
	eXyLJ13/JPiWH7PNtCeV5njVcOwDbDwmABu2UN3+heQ1zXpZU3sZEhBzoTeFpWhepWe7uRevC/WqR
	Kasns+92hfZS2Bd9QVOVBAe2bhQuxP5bir6HT/KrrHX9fkHMPSNUi63n5GAe2MN8LfmI=;
Subject: Re: About Arm guests accessing to GICD_ICPENR
To: Hongda Deng <Hongda.Deng@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>, nd <nd@arm.com>
References: <VE1PR08MB567729BD416FCCD32FCBD6CCE6D39@VE1PR08MB5677.eurprd08.prod.outlook.com>
From: Julien Grall <julien@xen.org>
Message-ID: <d30242ce-7e67-aad2-4b67-6a038dfd3e93@xen.org>
Date: Thu, 9 Sep 2021 10:08:47 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <VE1PR08MB567729BD416FCCD32FCBD6CCE6D39@VE1PR08MB5677.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

On 07/09/2021 09:05, Hongda Deng wrote:
> Hi Julien,

Hi,

(+ Stefano)

>> On 06/09/2021 10:04, Hongda Deng wrote:
>>> Hi Julien,
>>> If Xen wants a complete GICD ICPENDRn emulation to fix it, do you have any
>>> suggestions.
>>
>> Emulating ICPENDR is not easy with the existing vGIC. It would be great
>> to finally have a vGIC spec compliant, but I also appreciate this is
>> going to take some time.
> 
> Yes, I agree with that.
> 
>>
>> Ignoring the access is probably the best we can do. However, this is
>> also a risky approach because Zephyr (or another OS) may genuinely want
>> to clear an already pending interrupt. So I would suggest to walk the
>> interrupts that are meant to be modified and check whether they are
>> already pending. If they are then print a message.
> 
> OK, I would add this check and warning message to my current patch to fix it
> in current stage. Is it OK?

I would be fine with the warning. I have added Stefano in CC so he is 
aware of the discussion.

Cheers,

> 
>>
>> This would make clear to the developper that something may go wrong
>> later on.
>>
>> Cheers,
>>
>> --
>> Julien Grall
> 
> [1] https://github.com/zephyrproject-rtos/zephyr/blob/eaf6cf745df3807e6e
> cc941c3a30de6c179ae359/drivers/interrupt_controller/intc_gicv3.c#L274
> 
> Cheers
> Hongda
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:12:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:12:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182841.330643 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG6f-0005FK-UD; Thu, 09 Sep 2021 09:12:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182841.330643; Thu, 09 Sep 2021 09:12:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOG6f-0005FD-Qp; Thu, 09 Sep 2021 09:12:29 +0000
Received: by outflank-mailman (input) for mailman id 182841;
 Thu, 09 Sep 2021 09:12:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOG6d-0005F7-MM
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:12:27 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0cd10836-114e-11ec-b1a7-12813bfff9fa;
 Thu, 09 Sep 2021 09:12:26 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1898hYTK014860; 
 Thu, 9 Sep 2021 09:12:23 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayex9g604-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 09:12:23 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6323.eurprd03.prod.outlook.com (2603:10a6:20b:159::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 09:12:21 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 09:12:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cd10836-114e-11ec-b1a7-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RRF3yGb5oiuK0r0gfCJcOtNwbv1AaMw7jRUtGhjEZBsvjQzlqEKWXFuoRKReY/N4uc5u3SK/pvLJgp55CCJQQQQHfEydq7d+Er648n8fmdbpfdGSaKQFdEkXjsT4MXU2mhyUsQip9JCKD/YnlS/ZqxCj+aeDAX3C6N9Ldoo+fSds/RTRpQ/TCxA9hq3E9iRpSu8P2zMcFz+shdIjQyzEdTE2U4yf0rK2mrjdeJazfHy5NVV3MR/1sctOR/vfyB6fVPU9xluh4BvlRagJz43vdkE6hnH+z4XTbOxJP8RPEFTB815IzrL7sIsvuD0Fly0ET02VurvjgpM2uoRA4b1ftg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=t8unG2SVBWLUsd/bfD1mgceFjbQrhNUQlU2bjVcxXqM=;
 b=jMe7kTs7HxpSodE6bSv/I2rpzjeHGKIEPJy+oKZ/lFTGVEHuW7bVFpBUfPcHDgHzcCGJOEEpteZEzO4C1N8qBMSG8XbQ88Aru56zp0TemQmtOS/65uPtgcOKc5gYvPnY/2jpbJ23IJlJvqzO98fNAuIrCdq86uWG5k4BaRL5FXEjOj7xcnzfFc9FKhVUDADtRogm7rlEMlxGWby+hFezNyUusouMyHuWNHCyh8+vzqkj3VR4UU/79UeGYiUNtFOhRpDIHsQZxUF2kzKXGo/wRUlQrXdPAlgYGTlzpuPX+w2kvQhUYaeCrnqK0xyRwHnmOXjko75KL9zUjGfSEiDJhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=t8unG2SVBWLUsd/bfD1mgceFjbQrhNUQlU2bjVcxXqM=;
 b=sSBlJxY4Aqus1bwJop/UbfpqwA0utCnP23i1tr/83fcrN5sw+T1WVq9gbL6KyyR+UqLlhzFv7gCaOhZE1DVImNR0zf/KKXvL1RotQVpS8M7BOO8v7goJU11es+BwJNldfM6t9krPJTRejprwtj2KgHsO8mBch2XS3n28HiM2Ixw6f44vPTY3745li8lsdsIlfrpKEYMuHxXjJmukSJ8D8VJYdQD5BhOodWF2wVhNnvBaNmgTbWhDzZ9jh8eAQUN1f7AUTl2uvZhSdET6JZ1utAhsRAykSiEazTD0YZ3axefcT5TxXMtbFcwFB68Bk17T5fXiBHC362qoiF4rdLbFGQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Topic: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Index: 
 AQHXoKxjXfke7B20QEeo6YjlfUMhlquXGvYAgAMgDgCAAAg4gIAA8L+AgAAy0wCAAA1uAA==
Date: Thu, 9 Sep 2021 09:12:21 +0000
Message-ID: <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
In-Reply-To: <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e0206cca-a8b7-4ff7-ea3b-08d97371ee06
x-ms-traffictypediagnostic: AM0PR03MB6323:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB632343C46CEDA5BD22D4E5BBE7D59@AM0PR03MB6323.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 XjNgKFdnPFj3+ZWlGiYS7Hyz1EimevseJUj6xsGhF81qCi8bOCrjn6IuGjxhfo/Fwqwk70YD8uoY8B2FhSKb8Du3PjCXA1ViRrdE4kHO9G/YyOFUL78+gTZQfNflHfjE+VFaFT1v6ZFOaOLQnbv5AFkNfgxTqmu1GEWgeo+9z9Tn4yfX78WLUM3qK2ErxB/I4ONWfrcUteuk/lfnzUNY9DRz1IZZ4pBrE2WJ+goiM8nVskFZmypyzURy7YZ9vPV+hfPj9sdJf53Zw2k0enkbLBNQgfcLWpB+K5M1n1yR5Mv3d8RFuRqVjRq5r4ynEwJvK7s2zETwNZ+SvCqFMEnzsgotLTSE6rj3dMmj4D7xr7VM8IMD6OG9U7fO8EpZq+9DwzXlibQb0L28z6tAIDbdD4FM6M0ucRjR07reIak6lDMV/GoWZ+WhH0Z1MoXLK2b4J0KXQGmeRuV1T3PkTh/WFrv0Ss6W3GU7j7SlNQNRun3WxsatYfEIQFoB6X4DN5gdivvFQS2yDa5oSwsrHUcEU7ef2NXDPewClvosbYbfKTxDjvyvD3ULEhMsYrG5twAs6K2hJIcqOp9ZMRg95aTWjp3iic85RqG9KKb7RzXhaltvQindcLozVd9NM34dATCxWUR7xVSE8TI896TLu2Q22nPwCooXhZ98uUrXCxEf0KG8ln8YuMnnxUDGHMWdcCI06pQ4O7mycp6vdjwPl4xQJgNqpceBkbNQqnQGyT+vnDlhKrZM9eLZaBRjZHq3HWBSygpSa1nRzFiyTEtgkVTj+leUX/Qa4JrGUsvArQG8nCly4kJCJT7IfZg+rzSpPtA2
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(366004)(346002)(376002)(396003)(66556008)(2616005)(53546011)(66476007)(6506007)(91956017)(2906002)(36756003)(64756008)(86362001)(66446008)(76116006)(478600001)(31686004)(66946007)(8936002)(8676002)(71200400001)(6486002)(5660300002)(966005)(6512007)(38100700002)(38070700005)(186003)(316002)(54906003)(31696002)(4326008)(110136005)(122000001)(83380400001)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Z2t4M2dxQ2M2RlR6NExNYkp0WDJaRTlwMzEweEpZOXhSL2xIdDIvZkR4bk9I?=
 =?utf-8?B?aVFMR1ZoTVlsNjN5K3Mxcnk0OGt1NDJkY0xuVStJT3BKNng2d01seEN3V2NP?=
 =?utf-8?B?bk1SeXpvNW10bmZkK3VwSXpLWm5KeTluUkhOdE9zUTBWdmxrMXFISHQ5M1hC?=
 =?utf-8?B?ZzIxUzlxUFYralRIREZEb2U3TmpDNnBjR0xKU2hBTFpxRFRwQ2VNTGphSUI3?=
 =?utf-8?B?dFJiOG1TNEk1eEFCYjdYNDVPeGdGWFpJWWtTNVVKSXhsdyt1dUNwM0tWck9k?=
 =?utf-8?B?R0NRQ3BzN3NkTk91dE11MVBFellGTFJaSFprY1RwN3NXRFpkQkdpZUw0UUxW?=
 =?utf-8?B?dWZETTlwbWtMY2tIOXE3WU41a3BUZHEzSS9OdGRDbEJtODB1ZU1mSU5uTnlO?=
 =?utf-8?B?QTkweEN0R29KOHBCWDN6NHhmZHZUOFhpVkhVT1B2V2V6dFFtblNpR05rUk1J?=
 =?utf-8?B?TWhMZDFqK1JCK3lkS0VWVXAySURaNTYzbExWcEh2WDl1bVZFOUZLM1JCdjNt?=
 =?utf-8?B?WG5lY2VKTjJOZEpsSVBJZUpDdlJhU243MUlVMkthdTdJQTJoWURoTmR3Ylpl?=
 =?utf-8?B?RktZL2l6cjMxdmhCQUFvWjZCRExRUTFzbDJvMXZXNzRzTEdrQlRhSEc4MU5Q?=
 =?utf-8?B?TzZwVUlNTko2Yitteks5NVBUcTdSUmtVYitRZHJiMG1rWjljQ3ZzUXpHYWlj?=
 =?utf-8?B?TCtqSXI3Wi92TkFoWkFyLzdBMDNKeFhoVklmdjhUK2UwTTJzbUw3S2RrRzBE?=
 =?utf-8?B?clBXSWxzbzlPOG1TamlqYmZpMm92UGFQSEhZQkJPWFdMVHAvWXN6bXowZUFG?=
 =?utf-8?B?cjJDeThXaml3QUhCSnA1TXpDMnBKM2tXVnVpV3E1aDl4VytRb1pLRnJEaW85?=
 =?utf-8?B?UkRiSkVyc28zclpYTkVLVGtIUEtxSnQvVjlTeWlJZTVsQ2VzYVRQcWY3RitX?=
 =?utf-8?B?eElnSHpaMXhXSEQ2MmwzRkpVblBxMEpYQWlCZnpEeFoyNUZHeVZITnM4azFm?=
 =?utf-8?B?NWxxTzBsbVltaUtGRnlzdE9NRzFoQzNHR2NUczIrVzdPamtORmdpdG5lQmhD?=
 =?utf-8?B?YTdIZ1EvazhKNUZydUlKTGxFbCtSdFFUSTg0U1JIT2UwNXU5OWVQck1GREJP?=
 =?utf-8?B?U25VbjZMRFB2OUhNWEp2SG41UTdvMG5OTXc5WUNOZ3h3UHFYczhTdnhvTVdv?=
 =?utf-8?B?c2RHdTgyM0dRanUvaU5YTXJTZlNtUEpnM3JYdUJSMFBxTmJXSUhLUlpiMzEv?=
 =?utf-8?B?SC9MYkRFc3JhWE53N3RVMEtLS1hra0wrakV0a3lLQWR4clFsSkNMWEluT2ll?=
 =?utf-8?B?LzNXWnJob2VRR25uYldGSTFVdTgxY01yVDRYUndvVkVXdHByV09VdTJ6N2NE?=
 =?utf-8?B?TEJiaWhFMlZ6Y2Qrdll1RzRXMG9IYUFMVUxSanJCZmk2d2ZSWEloTGtJWFZw?=
 =?utf-8?B?bzN4NGdJWlRNZ0tCUUliUEhWSWYyRDlxVWJnZUdrb3BaTHRnRmlEL1F0OE5p?=
 =?utf-8?B?dURScUVFeGJYcklzUkhvNm1HVG5nM0NpOWxsK0J6UndpSkdzSDd5SmNsVnFW?=
 =?utf-8?B?OW5jaGdXUWVvVnNybzJuc2tPU09TUzBlR2xtb3I4T1h3b1JpcVJyVkgrbmtx?=
 =?utf-8?B?Qmx3VnlDRHVGeEQ3Vk8xajk4Yk0vOWhRS0VwbGZMQ28wSGZ6aVhFWnBScDdU?=
 =?utf-8?B?V1hoQURmdXBEbnhGbFlkKzFmdzJKYXNDa2ZLS0Y0amRpdXJsZ3l6NHNJc2Zl?=
 =?utf-8?Q?zzXD9TmWRiZhcg19jPvkfFPcsROkbPUp9eKWefx?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <DCDA4C994775FD4F855A99C6EDCE9507@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e0206cca-a8b7-4ff7-ea3b-08d97371ee06
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 09:12:21.0380
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: TMYusrRp8SIi8ZJ8NYYoYCiqe30z1VSVGqYVN/gmuzFIdWDsHEXxQP6HnQFVphJFqjhN4unXnDR7X888AEPJO+yovpWsblqBC/fOblVI9c8fRdTOad/6L+Rn56X1H1Ps
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6323
X-Proofpoint-GUID: vFm9OBJnJOd4tjp0LtLCqmPIS7im0LKW
X-Proofpoint-ORIG-GUID: vFm9OBJnJOd4tjp0LtLCqmPIS7im0LKW
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_03,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxscore=0
 phishscore=0 bulkscore=0 spamscore=0 priorityscore=1501 adultscore=0
 clxscore=1015 lowpriorityscore=0 malwarescore=0 mlxlogscore=999
 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090055

DQpPbiAwOS4wOS4yMSAxMToyNCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MDc6MjIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDguMDkuMjEgMTg6
MDAsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA4LjA5LjIwMjEgMTY6MzEsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNi4wOS4yMSAxNzo0NywgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3Nh
bmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pj4NCj4+Pj4+PiBJbnN0ZWFkIG9mIGhh
bmRsaW5nIGEgc2luZ2xlIHJhbmdlIHNldCwgdGhhdCBjb250YWlucyBhbGwgdGhlIG1lbW9yeQ0K
Pj4+Pj4+IHJlZ2lvbnMgb2YgYWxsIHRoZSBCQVJzIGFuZCBST00sIGhhdmUgdGhlbSBwZXIgQkFS
Lg0KPj4+Pj4gV2l0aG91dCBsb29raW5nIGF0IGhvdyB5b3UgY2Fycnkgb3V0IHRoaXMgY2hhbmdl
IC0gdGhpcyBsb29rIHdyb25nIChhcw0KPj4+Pj4gaW46IHdhc3RlZnVsKSB0byBtZS4gRGVzcGl0
ZSAuLi4NCj4+Pj4+DQo+Pj4+Pj4gVGhpcyBpcyBpbiBwcmVwYXJhdGlvbiBvZiBtYWtpbmcgbm9u
LWlkZW50aXR5IG1hcHBpbmdzIGluIHAybSBmb3IgdGhlDQo+Pj4+Pj4gTU1JT3MvUk9NLg0KPj4+
Pj4gLi4uIHRoZSBuZWVkIGZvciB0aGlzLCBldmVyeSBpbmRpdmlkdWFsIEJBUiBpcyBzdGlsbCBj
b250aWd1b3VzIGluIGJvdGgNCj4+Pj4+IGhvc3QgYW5kIGd1ZXN0IGFkZHJlc3Mgc3BhY2VzLCBz
byBjYW4gYmUgcmVwcmVzZW50ZWQgYXMgYSBzaW5nbGUNCj4+Pj4+IChzdGFydCxlbmQpIHR1cGxl
IChvciBhIHBhaXIgdGhlcmVvZiwgdG8gYWNjb3VudCBmb3IgYm90aCBob3N0IGFuZCBndWVzdA0K
Pj4+Pj4gdmFsdWVzKS4gTm8gbmVlZCB0byB1c2UgYSByYW5nZXNldCBmb3IgdGhpcy4NCj4+Pj4g
Rmlyc3Qgb2YgYWxsIHRoaXMgY2hhbmdlIGlzIGluIHByZXBhcmF0aW9uIGZvciBub24taWRlbnRp
dHkgbWFwcGluZ3MsDQo+Pj4gSSdtIGFmcmFpZCBJIGNvbnRpbnVlIHRvIG5vdCBzZWUgaG93IHRo
aXMgbWF0dGVycyBpbiB0aGUgZGlzY3Vzc2lvbiBhdA0KPj4+IGhhbmQuIEknbSBmdWxseSBhd2Fy
ZSB0aGF0IHRoaXMgaXMgdGhlIGdvYWwuDQo+Pj4NCj4+Pj4gZS5nLiBjdXJyZW50bHkgd2UgY29s
bGVjdCBhbGwgdGhlIG1lbW9yeSByYW5nZXMgd2hpY2ggcmVxdWlyZSBtYXBwaW5ncw0KPj4+PiBp
bnRvIGEgc2luZ2xlIHJhbmdlIHNldCwgdGhlbiB3ZSBjdXQgb2ZmIE1TSS1YIHJlZ2lvbnMgYW5k
IHRoZW4gdXNlIHJhbmdlIHNldA0KPj4+PiBmdW5jdGlvbmFsaXR5IHRvIGNhbGwgYSBjYWxsYmFj
ayBmb3IgZXZlcnkgbWVtb3J5IHJhbmdlIGxlZnQgYWZ0ZXIgTVNJLVguDQo+Pj4+IFRoaXMgd29y
a3MgcGVyZmVjdGx5IGZpbmUgZm9yIDE6MSBtYXBwaW5ncywgZS5nLiB3aGF0IHdlIGhhdmUgYXMg
dGhlIHJhbmdlDQo+Pj4+IHNldCdzIHN0YXJ0aW5nIGFkZHJlc3MgaXMgd2hhdCB3ZSB3YW50IHRv
IGJlIG1hcHBlZC91bm1hcHBlZC4NCj4+Pj4gV2h5IHJhbmdlIHNldHM/IEJlY2F1c2UgdGhleSBh
bGxvdyBwYXJ0aWFsIG1hcHBpbmdzLCBlLmcuIHlvdSBjYW4gbWFwIHBhcnQgb2YNCj4+Pj4gdGhl
IHJhbmdlIGFuZCByZXR1cm4gYmFjayBhbmQgY29udGludWUgZnJvbSB3aGVyZSB5b3Ugc3RvcHBl
ZC4gQW5kIGlmIEkNCj4+Pj4gdW5kZXJzdGFuZCB0aGF0IGNvcnJlY3RseSB0aGF0IHdhcyB0aGUg
aW5pdGlhbCBpbnRlbnRpb24gb2YgaW50cm9kdWNpbmcgcmFuZ2Ugc2V0cyBoZXJlLg0KPj4+Pg0K
Pj4+PiBGb3Igbm9uLWlkZW50aXR5IG1hcHBpbmdzIHRoaXMgYmVjb21lcyBub3QgdGhhdCBlYXN5
LiBFYWNoIGluZGl2aWR1YWwgQkFSIG1heSBiZQ0KPj4+PiBtYXBwZWQgZGlmZmVyZW50bHkgYWNj
b3JkaW5nIHRvIHdoYXQgZ3Vlc3QgT1MgaGFzIHByb2dyYW1tZWQgYXMgYmFyLT5ndWVzdF9hZGRy
DQo+Pj4+IChndWVzdCB2aWV3IG9mIHRoZSBCQVIgc3RhcnQpLg0KPj4+IEkgZG9uJ3Qgc2VlIGhv
dyB0aGUgcmFuZ2VzZXQgaGVscHMgaGVyZS4gWW91IGhhdmUgYSBndWVzdCBhbmQgYSBob3N0IHBh
aXINCj4+PiBvZiB2YWx1ZXMgZm9yIGV2ZXJ5IEJBUi4gUGFnZXMgd2l0aCBlLmcuIHRoZSBNU0kt
WCB0YWJsZSBtYXkgbm90IGJlIG1hcHBlZA0KPj4+IHRvIHRoZWlyIGhvc3QgY291bnRlcnBhcnQg
YWRkcmVzcywgeWVzLCBidXQgeW91IG5lZWQgdG8gc3BlY2lhbCBjYXNlcw0KPj4+IHRoZXNlIGFu
eXdheTogQWNjZXNzZXMgdG8gdGhlbSBuZWVkIHRvIGJlIGhhbmRsZWQuIEhlbmNlIEknbSBoYXZp
bmcgYSBoYXJkDQo+Pj4gdGltZSBzZWVpbmcgaG93IGEgcGVyLUJBUiByYW5nZXNldCAod2hpY2gg
d2lsbCBjb3ZlciBhdCBtb3N0IHRocmVlIGRpc3RpbmN0DQo+Pj4gcmFuZ2VzIGFmYWljdCwgd2hp
Y2ggaXMgd2F5IHRvbyBsaXR0bGUgZm9yIHRoaXMga2luZCBvZiBkYXRhIG9yZ2FuaXphdGlvbg0K
Pj4+IGltbykgY2FuIGdhaW4geW91IGFsbCB0aGlzIG11Y2guDQo+Pj4NCj4+PiBPdmVyYWxsIHRo
ZSA2IEJBUnMgb2YgYSBkZXZpY2Ugd2lsbCBjb3ZlciB1cCB0byA4IG5vbi1hZGphY2VudCByYW5n
ZXMuIElPVw0KPj4+IHRoZSBtYWpvcml0eSAoNCBvciBtb3JlKSBvZiB0aGUgcmFuZ2VzZXRzIHdp
bGwgaW5kZWVkIG1lcmVseSByZXByZXNlbnQgYQ0KPj4+IHBsYWluIChzdGFydCxlbmQpIHBhaXIg
KG9yIGJlIGVudGlyZWx5IGVtcHR5KS4NCj4+IEZpcnN0IG9mIGFsbCwgbGV0IG1lIGV4cGxhaW4g
d2h5IEkgZGVjaWRlZCB0byBtb3ZlIHRvIHBlci1CQVINCj4+IHJhbmdlIHNldHMuDQo+PiBCZWZv
cmUgdGhpcyBjaGFuZ2UgYWxsIHRoZSBNTUlPIHJlZ2lvbnMgYW5kIE1TSS1YIGhvbGVzIHdlcmUN
Cj4+IGFjY291bnRlZCBieSBhIHNpbmdsZSByYW5nZSBzZXQsIGUuZy4gd2UgZ28gb3ZlciBhbGwg
QkFScyBhbmQNCj4+IGFkZCBNTUlPcyBhbmQgdGhlbiBzdWJ0cmFjdCBNU0ktWCBmcm9tIHRoZXJl
LiBXaGVuIGl0IGNvbWVzIHRvDQo+PiBtYXBwaW5nL3VubWFwcGluZyB3ZSBoYXZlIGFuIGFzc3Vt
dGlvbiB0aGF0IHRoZSBzdGFydGluZyBhZGRyZXNzIG9mDQo+PiBlYWNoIGVsZW1lbnQgaW4gdGhl
IHJhbmdlIHNldCBpcyBlcXVhbCB0byBtYXAvdW5tYXAgYWRkcmVzcywgZS5nLg0KPj4gd2UgaGF2
ZSBpZGVudGl0eSBtYXBwaW5nLiBQbGVhc2Ugbm90ZSwgdGhhdCB0aGUgcmFuZ2Ugc2V0IGFjY2Vw
dHMNCj4+IGEgc2luZ2xlIHByaXZhdGUgZGF0YSBwYXJhbWV0ZXIgd2hpY2ggaXMgZW5vdWdoIHRv
IGhvbGQgYWxsDQo+PiByZXF1aXJlZCBkYXRhIGFib3V0IHRoZSBwZGV2IGluIGNvbW1vbiwgYnV0
IHRoZXJlIGlzIG5vIHdheSB0byBwcm92aWRlDQo+PiBhbnkgcGVyLUJBUiBkYXRhLg0KPj4NCj4+
IE5vdywgdGhhdCB3ZSB3YW50IG5vbi1pZGVudGl0eSBtYXBwaW5ncywgd2UgY2FuIG5vIGxvbmdl
ciBhc3N1bWUNCj4+IHRoYXQgc3RhcnRpbmcgYWRkcmVzcyA9PSBtYXBwaW5nIGFkZHJlc3MgYW5k
IHdlIG5lZWQgdG8gcHJvdmlkZQ0KPj4gYWRkaXRpb25hbCBpbmZvcm1hdGlvbiBvbiBob3cgdG8g
bWFwIGFuZCB3aGljaCBpcyBub3cgcGVyLUJBUi4NCj4+IFRoaXMgaXMgd2h5IEkgZGVjaWRlZCB0
byB1c2UgcGVyLUJBUiByYW5nZSBzZXRzLg0KPj4NCj4+IE9uZSBvZiB0aGUgc29sdXRpb25zIG1h
eSBiZSB0aGF0IHdlIGZvcm0gYW4gYWRkaXRpb25hbCBsaXN0IG9mDQo+PiBzdHJ1Y3R1cmVzIGlu
IGEgZm9ybSAoSSBvbW1pdCBzb21lIG9mIHRoZSBmaWVsZHMpOg0KPj4gc3RydWN0IG5vbl9pZGVu
dGl0eSB7DQo+PiAgIMKgwqDCoCB1bnNpZ25lZCBsb25nIHN0YXJ0X21mbjsNCj4+ICAgwqDCoMKg
IHVuc2lnbmVkIGxvbmcgc3RhcnRfZ2ZuOw0KPj4gICDCoMKgwqAgdW5zaWduZWQgbG9uZyBzaXpl
Ow0KPj4gfTsNCj4+IFNvIHRoaXMgd2F5IHdoZW4gdGhlIHJhbmdlIHNldCBnZXRzIHByb2Nlc3Nl
ZCB3ZSBnbyBvdmVyIHRoZSBsaXN0DQo+PiBhbmQgZmluZCBvdXQgdGhlIGNvcnJlc3BvbmRpbmcg
bGlzdCdzIGVsZW1lbnQgd2hpY2ggZGVzY3JpYmVzIHRoZQ0KPj4gcmFuZ2Ugc2V0IGVudHJ5IGJl
aW5nIHByb2Nlc3NlZCAocywgZSwgZGF0YSk6DQo+Pg0KPj4gc3RhdGljIGludCBtYXBfcmFuZ2Uo
dW5zaWduZWQgbG9uZyBzLCB1bnNpZ25lZCBsb25nIGUsIHZvaWQgKmRhdGEsDQo+PiAgIMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyAqYykNCj4+
IHsNCj4+IFtzbmlwXQ0KPj4gICDCoMKgwqAgZ28gb3ZlciB0aGUgbGlzdCBlbGVtZW50cw0KPj4g
ICDCoMKgwqAgwqDCoMKgIGlmICggbGlzdC0+c3RhcnRfbWZuID09IHMgKQ0KPj4gICDCoMKgwqAg
wqDCoMKgIMKgwqDCoCBmb3VuZCwgY2FuIHVzZSBsaXN0LT5zdGFydF9nZm4gZm9yIG1hcHBpbmcN
Cj4+IFtzbmlwXQ0KPj4gfQ0KPj4gVGhpcyBoYXMgc29tZSBjb21wbGljYXRpb25zIGFzIG1hcF9y
YW5nZSBtYXkgYmUgY2FsbGVkIG11bHRpcGxlIHRpbWVzDQo+PiBmb3IgdGhlIHNhbWUgcmFuZ2U6
IGlmIHt1bm1hcHxtYXB9X21taW9fcmVnaW9ucyB3YXMgbm90IGFibGUgdG8gY29tcGxldGUNCj4+
IHRoZSBvcGVyYXRpb24gaXQgcmV0dXJucyB0aGUgbnVtYmVyIG9mIHBhZ2VzIGl0IHdhcyBhYmxl
IHRvIHByb2Nlc3M6DQo+PiAgIMKgwqDCoMKgwqDCoMKgIHJjID0gbWFwLT5tYXAgPyBtYXBfbW1p
b19yZWdpb25zKG1hcC0+ZCwgc3RhcnRfZ2ZuLA0KPj4gICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCBzaXplLCBfbWZuKHMpKQ0KPj4gICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgOiB1bm1hcF9tbWlvX3JlZ2lvbnMobWFwLT5kLCBzdGFydF9nZm4sDQo+PiAgIMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzaXplLCBfbWZuKHMpKTsNCj4+IEluIHRoaXMgY2Fz
ZSB3ZSBuZWVkIHRvIHVwZGF0ZSB0aGUgbGlzdCBpdGVtOg0KPj4gICDCoMKgwqAgbGlzdC0+c3Rh
cnRfbWZuICs9IHJjOw0KPj4gICDCoMKgwqAgbGlzdC0+c3RhcnRfZ2ZuICs9IHJjOw0KPj4gICDC
oMKgwqAgbGlzdC0+c2l6ZSAtPSByYzsNCj4+IGFuZCBpZiBhbGwgdGhlIHBhZ2VzIG9mIHRoZSBy
YW5nZSB3ZXJlIHByb2Nlc3NlZCBkZWxldGUgdGhlIGxpc3QgZW50cnkuDQo+Pg0KPj4gV2l0aCBy
ZXNwZWN0IG9mIGNyZWF0aW5nIHRoZSBsaXN0IGV2ZXJ5dGhpbmcgYWxzbyBub3Qgc28gY29tcGxp
Y2F0ZWQ6DQo+PiB3aGlsZSBwcm9jZXNzaW5nIGVhY2ggQkFSIGNyZWF0ZSBhIGxpc3QgZW50cnkg
YW5kIGZpbGwgaXQgd2l0aCBtZm4sIGdmbg0KPj4gYW5kIHNpemUuIFRoZW4sIGlmIE1TSS1YIHJl
Z2lvbiBpcyBwcmVzZW50IHdpdGhpbiB0aGlzIEJBUiwgYnJlYWsgdGhlDQo+PiBsaXN0IGl0ZW0g
aW50byBtdWx0aXBsZSBvbmVzIHdpdGggcmVzcGVjdCB0byB0aGUgaG9sZXMsIGZvciBleGFtcGxl
Og0KPj4NCj4+IE1NSU8gMCBsaXN0IGl0ZW0NCj4+IE1TSS1YIGhvbGUgMA0KPj4gTU1JTyAxIGxp
c3QgaXRlbQ0KPj4gTVNJLVggaG9sZSAxDQo+Pg0KPj4gSGVyZSBpbnN0ZWFkIG9mIGEgc2luZ2xl
IEJBUiBkZXNjcmlwdGlvbiB3ZSBub3cgaGF2ZSAyIGxpc3QgZWxlbWVudHMNCj4+IGRlc2NyaWJp
bmcgdGhlIEJBUiB3aXRob3V0IE1TSS1YIHJlZ2lvbnMuDQo+Pg0KPj4gQWxsIHRoZSBhYm92ZSBz
dGlsbCByZWxpZXMgb24gYSBzaW5nbGUgcmFuZ2Ugc2V0IHBlciBwZGV2IGFzIGl0IGlzIGluIHRo
ZQ0KPj4gb3JpZ2luYWwgY29kZS4gV2UgY2FuIGdvIHRoaXMgcm91dGUgaWYgd2UgYWdyZWUgdGhp
cyBpcyBtb3JlIGFjY2VwdGFibGUNCj4+IHRoYW4gdGhlIHJhbmdlIHNldHMgcGVyIEJBUg0KPiBJ
IGd1ZXNzIEkgYW0gbm93IGV2ZW4gbW9yZSBjb25mdXNlZDogSSBjYW4ndCBzcG90IGFueSAicmFu
Z2VzZXQgcGVyIHBkZXYiDQo+IGVpdGhlci4gVGhlIHJhbmdlc2V0IEkgc2VlIGJlaW5nIHVzZWQg
ZG9lc24ndCBnZXQgYXNzb2NpYXRlZCB3aXRoIGFueXRoaW5nDQo+IHRoYXQncyBkZXZpY2UtcmVs
YXRlZDsgaXQgZ2V0cyBhY2N1bXVsYXRlZCBhcyBhIHRyYW5zaWVudCBkYXRhIHN0cnVjdHVyZSwN
Cj4gYnV0IF9hbGxfIGRldmljZXMgb3duZWQgYnkgYSBkb21haW4gaW5mbHVlbmNlIGl0cyBmaW5h
bCBjb250ZW50Lg0KDQpZb3UgYXJlIGFic29sdXRlbHkgcmlnaHQgaGVyZSwgc29ycnkgZm9yIHRo
ZSBjb25mdXNpb246IGluIHRoZSBjdXJyZW50DQoNCmNvZGUgdGhlIHJhbmdlIHNldCBiZWxvbmdz
IHRvIHN0cnVjdCB2cGNpX3ZjcHUsIGUuZy4NCg0KLyogUGVyLXZjcHUgc3RydWN0dXJlIHRvIHN0
b3JlIHN0YXRlIHdoaWxlIHt1bn1tYXBwaW5nIG9mIFBDSSBCQVJzLiAqLw0KDQo+DQo+IElmIHlv
dSBhc3NvY2lhdGUgcmFuZ2VzZXRzIHdpdGggZWl0aGVyIGEgZGV2aWNlIG9yIGEgQkFSLCBJJ20g
ZmFpbGluZyB0bw0KPiBzZWUgaG93IHlvdSdkIGRlYWwgd2l0aCBtdWx0aXBsZSBCQVJzIGxpdmlu
ZyBpbiB0aGUgc2FtZSBwYWdlIChzZWUgYWxzbw0KPiBiZWxvdykuDQoNClRoaXMgd2FzIGV4YWN0
bHkgdGhlIGlzc3VlIEkgcmFuIGludG8gd2hpbGUgZW11bGF0aW5nIFJUTDgxMzkgb24gUUVNVToN
Cg0KVGhlIE1NSU9zIGFyZSAxMjggYnl0ZXMgbG9uZyBhbmQgTGludXggcHV0IHRoZW0gb24gdGhl
IHNhbWUgcGFnZS4NCg0KU28sIGl0IGlzIGEga25vd24gbGltaXRhdGlvbiB0aGF0IHdlIGNhbid0
IGRlYWwgd2l0aCBbMV0NCg0KPg0KPiBDb25zaWRlcmluZyB0aGF0IGEgcmFuZ2VzZXQgcmVhbGx5
IGlzIGEgY29tcHJlc3NlZCByZXByZXNlbnRhdGlvbiBvZiBhDQo+IGJpdG1hcCwgSSB3b25kZXIg
d2hldGhlciB0aGlzIGRhdGEgc3RydWN0dXJlIGlzIHN1aXRhYmxlIGF0IGFsbCBmb3Igd2hhdA0K
PiB5b3Ugd2FudCB0byBleHByZXNzLiBZb3UgaGF2ZSB0d28gcGllY2VzIG9mIGluZm9ybWF0aW9u
IHRvIGNhcnJ5IC8gbWFuYWdlLA0KPiBhZnRlciBhbGw6IFdoaWNoIHJhbmdlcyBuZWVkIG1hcHBp
bmcsIGFuZCB3aGF0IHRoZWlyIEdGTiA8LT4gTUZODQo+IHJlbGF0aW9uc2hpcCBpcy4gTWF5YmUg
dGhlIGxhdHRlciBuZWVkcyBleHByZXNzaW5nIGRpZmZlcmVudGx5IGluIHRoZQ0KPiBmaXJzdCBw
bGFjZT8NCg0KSSBwcm9wb3NlZCBhIGxpc3Qgd2hpY2ggY2FuIGJlIGV4dGVuZGVkIHRvIGhvbGQg
YWxsIHRoZSByZXF1aXJlZCBpbmZvcm1hdGlvbg0KDQp0aGVyZSwgZS5nLiBNRk4sIEdGTiwgc2l6
ZSBldGMuDQoNCj4gICBBbmQgdGhlbiBpbiBhIHdheSB0aGF0J3MgZW5zdXJpbmcgYnkgaXRzIG9y
Z2FuaXphdGlvbiB0aGF0DQo+IG5vIGNvbmZsaWN0aW5nIEdGTiA8LT4gTUZOIG1hcHBpbmdzIHdp
bGwgYmUgcG9zc2libGU/DQoNCklmIHlvdSBtZWFuIHRoZSB1c2UtY2FzZSBhYm92ZSB3aXRoIGRp
ZmZlcmVudCBkZXZpY2UgTU1JT3MgbGl2aW5nDQoNCmluIHRoZSBzYW1lIHBhZ2UgdGhlbiBteSB1
bmRlcnN0YW5kaW5nIGlzIHRoYXQgc3VjaCBhIHVzZS1jYXNlIGlzDQoNCm5vdCBzdXBwb3J0ZWQg
WzFdDQoNCj4gICBJc24ndCB0aGlzDQo+IHByZWNpc2VseSB3aGF0IGlzIGFscmVhZHkgZ2V0dGlu
ZyByZWNvcmRlZCBpbiB0aGUgUDJNPw0KPg0KPiBJJ20gYWxzbyBjdXJpb3VzIHdoYXQgeW91ciBw
bGFuIGlzIHRvIGRlYWwgd2l0aCBCQVJzIG92ZXJsYXBwaW5nIGluIE1GTg0KPiBzcGFjZTogSW4g
c3VjaCBhIGNhc2UsIHRoZSBndWVzdCBjYW5ub3QgaW5kZXBlbmRlbnRseSBjaGFuZ2UgdGhlIEdG
TnMgb2YNCj4gYW55IG9mIHRoZSBpbnZvbHZlZCBCQVJzLiAoU2FtZSB0aGUgb3RoZXIgd2F5IGFy
b3VuZDogb3ZlcmxhcHMgaW4gR0ZODQo+IHNwYWNlIGFyZSBvbmx5IHBlcm1pdHRlZCB3aGVuIHRo
ZSBzYW1lIG92ZXJsYXAgZXhpc3RzIGluIE1GTiBzcGFjZS4pIEFyZQ0KPiB5b3UgZXhjbHVkaW5n
IChmb3JiaWRkaW5nKSB0aGlzIGNhc2U/IElmIHNvLCBkaWQgSSBtaXNzIHlvdSBzYXlpbmcgc28N
Cj4gc29tZXdoZXJlPw0KQWdhaW4gWzFdDQo+ICAgWWV0IGlmIG5vIG92ZXJsYXBzIGFyZSBhbGxv
d2VkIGluIHRoZSBmaXJzdCBwbGFjZSwgd2hhdA0KPiBtb2RpZnlfYmFycygpIGRvZXMgd291bGQg
YmUgZmFyIG1vcmUgY29tcGxpY2F0ZWQgdGhhbiBuZWNlc3NhcnkgaW4gdGhlDQo+IERvbVUgY2Fz
ZSwgc28gaXQgbWF5IGJlIHdvcnRod2hpbGUgY29uc2lkZXJpbmcgdG8gZGV2aWF0ZSBtb3JlIGZy
b20gaG93DQo+IERvbTAgZ2V0cyB0YWtlbiBjYXJlIG9mLiBJbiB0aGUgZW5kIGEgZ3Vlc3Qgd3Jp
dGluZyBhIEJBUiBpcyBtZXJlbHkgYQ0KPiByZXF1ZXN0IHRvIGNoYW5nZSBpdHMgUDJNLiBUaGF0
J3MgdmVyeSBkaWZmZXJlbnQgZnJvbSBEb20wIHdyaXRpbmcgYSBCQVIsDQo+IHdoaWNoIG1lYW5z
IHRoZSBwaHlzaWNhbCBCQVIgYWxzbyBjaGFuZ2VzLCBhbmQgaGVuY2UgdGhlIFAyTSBjaGFuZ2Vz
IGluDQo+IHF1aXRlIGRpZmZlcmVudCBhIHdheS4NCg0KU28sIHdoYXQgaXMgdGhlIGRpZmZlcmVu
Y2UgdGhlbiBiZXNpZGVzIGh3ZG9tIHJlYWxseSB3cml0ZXMgdG8gYSBCQVI/DQoNClRvIG1lIG1v
c3Qgb2YgdGhlIGxvZ2ljIHJlbWFpbnMgdGhlIHNhbWU6IHdlIG5lZWQgdG8gbWFwL3VubWFwLg0K
DQpUaGUgb25seSBkaWZmZXJlbmNlIEkgc2VlIGhlcmUgaXMgdGhhdCBmb3IgRG9tMCB3ZSBoYXZl
IDE6MSBhdCB0aGUgbW9tZW50DQoNCmFuZCBmb3IgZ3Vlc3Qgd2UgbmVlZCBHRk4gPC0+IE1GTi4N
Cg0KDQpBbnl3YXlzLCBJIGFtIG9wZW4gdG8gYW55IGRlY2lzaW9uIG9uIHdoYXQgd291bGQgYmUg
dGhlIHJpZ2h0IGFwcHJvYWNoIGhlcmU6DQoNCjEuIFVzZSByYW5nZSBzZXRzIHBlciBCQVIgYXMg
aW4gdGhlIHBhdGNoDQoNCjIuIFJlbW92ZSByYW5nZSBzZXRzIGNvbXBsZXRlbHkgYW5kIGhhdmUg
YSBwZXItdkNQVSBsaXN0IHdpdGggbWFwcGluZw0KDQpkYXRhIGFzIEkgZGVzY3JpYmVkIGFib3Zl
DQoNCjMuIEFueXRoaW5nIGVsc2U/DQoNCj4NCj4gSmFuDQoNClRoYW5rIHlvdSwNCg0KT2xla3Nh
bmRyDQoNClsxXSBodHRwczovL3dpa2kueGVucHJvamVjdC5vcmcvd2lraS9YZW5fUENJX1Bhc3N0
aHJvdWdoI0lfZ2V0Xy4yMm5vbi1wYWdlLWFsaWduZWRfTU1JT19CQVIuMjJfZXJyb3Jfd2hlbl90
cnlpbmdfdG9fc3RhcnRfdGhlX2d1ZXN0


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:17:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:17:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182848.330653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGB9-0005sD-Hv; Thu, 09 Sep 2021 09:17:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182848.330653; Thu, 09 Sep 2021 09:17:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGB9-0005s6-Ex; Thu, 09 Sep 2021 09:17:07 +0000
Received: by outflank-mailman (input) for mailman id 182848;
 Thu, 09 Sep 2021 09:17:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOGB8-0005s0-Lp
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:17:06 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4bb0cd0c-d34c-4926-aa54-0237910ac66b;
 Thu, 09 Sep 2021 09:17:05 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1899E0OG012082; 
 Thu, 9 Sep 2021 09:17:02 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayddm8w9w-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 09:17:02 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6963.eurprd03.prod.outlook.com (2603:10a6:20b:2d5::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 09:16:58 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 09:16:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bb0cd0c-d34c-4926-aa54-0237910ac66b
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aU5Te7fkqva71CCFT0t3QnPQmPSblMH9kO5g1Ni+Ed752q/w7Za82oRGnWn6QS6hblUIO9mtRU9c/rbr9Y2nGKjfdCto1DSaUMOaYT+o0cQZnIJOZwos0pRR/hplg6Q1EQoeVdsbPUoYEtxqTaxjwtXwFV6o/xJGpDZYRXYzE5ZKrvwWvpcMYycf7UiZqYg8/vAHAv9NLbRV2OsuMNQWOl5SXuiTJuQpcp+lPjdaEhg2eUhpCbMvyf0oqaRGv9YZSX7AK0IS02V5Ps5MorUUaHltTks6C+dv7VogVnmmUCtAzjJdj1S9tdxysS2A7s7jZXYWuXsQz9jknKcIzVoqLg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=chX/GOJFxoeWG/qEndEZOLb8ztbNtxyQ22IOr+l1lS4=;
 b=TQqqeGEMsSteV2ycJ0HDCxtby0sNBxFFV14eN/lpS7ngEJsrR/GQSmNF8qKmOba0BKMGv+2rp1c07XwHfajydXpjbCsAzIBGgc9VXbmxu4/Lmr4zRo1QR3Z53JOwvgTOMlPZwPj5PkoZReI6OjLYa0zrqS0c05/rk0v9SOoBLdDm4vGV7SCQze0DUk3ASmDOODQMdgWKgXxEJ/sw4gOkz49sU6h4YUxugxIitok21CpZSySXerLOxiRanfVZFkQZ3o3SXRcyJrOvX9/siwpE3xxLRzrUZw/54GemJogxJfB55jyyaNaPG5SZIpRTojKHHLIiikXUkOx8XE2XXC6Mfw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=chX/GOJFxoeWG/qEndEZOLb8ztbNtxyQ22IOr+l1lS4=;
 b=t3I1hAAURy6sFAac7ULkRzQvy3QERGzPXdUd7/DKKl36We6njwXn+c/V0sgXR6BTL04rfVxsIM4naIb2yvXNn8E43+IrTWvySKe1HFkJST5UG/lW4+cAWV4vUZ30oRA9ocPtNnc80xa+BmP0bsZuD4imRktuPCE5WOh0oaHzTbkv7lFj9vyWpIwFCd8S/VT0tCZlynI7J5T8O29g4f24qM+ENVcxRoXc4VVhaZ1My1asnAy7OBvWsbBhHsarA/EtTtSvZTB2xiM0XJ3ZkrkbCHXWN6Ghp8abx5UiRMS4L/+vsmEpQXZmPrYPFEN7nC9+Lk7aM4BmIjIxzZBhiI5peg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Index: AQHXoKxowwaOW1OR4UKei+2TOBJan6uXHAEAgAQmJoCAACVpAIAADfgA
Date: Thu, 9 Sep 2021 09:16:58 +0000
Message-ID: <b9fe88da-3bdb-0b37-f2f4-9a575806e6de@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
 <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
 <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
 <ee9a69f1-6b47-a944-7255-4ffaa3e1c601@suse.com>
In-Reply-To: <ee9a69f1-6b47-a944-7255-4ffaa3e1c601@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e73ba585-155e-4a6b-15be-08d973729392
x-ms-traffictypediagnostic: AM9PR03MB6963:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB696345396B0F8FDAC631D108E7D59@AM9PR03MB6963.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 i4DGXvuKlOrWm2wDFzQYSZztG1vKwGDCB39is9htYHhRQY1jpddalbj3Jt5+ULHSDYUV0yKipf60D0iSE2VqK+/vn6WMjODCWZH5ClutlYS/fMzSVW5aWYJC6wNtp/g+eNNdYFzzdQxsfJORbuuhoqSoQ4gQlXWXgyimrRhUhhJ9+NG9ZFAHvf4/BfWOaj20ep19JDIWTRIcYgFUEfN+VCDUSf03kauLDeVigV+pus98m9LiEzJDhJYCOA8pVXY6JpX5BoURgbeZMqKh9s/fcFU197xeK3jWpoEzuOF9N1kCJOYfI8lwAk1Rl7PAYLY9y2NMxp0eL3dJQMiad6U0MvJuP1L+gOl0R8a2t23Ww/LWsuej0amNVZQXYvIanzvyQxxWZ/gDaaC9wsBb2J+6rMeUJyIS5vYDXKVLUW/Yo4GHwVxLmuwnxcnV4CLWCLVh45g51bH8kLRN90Z+S2B0qjJ3Y3lLO5L6xqrBktfPQ911e4mm+/uusx//naWEG16q4cvD+IX3ZP9t3DUk5ZG+6uz6JqFohNceOf6KuVKel33BzY0PKt34YuniUludEbs5zbcwynYboQ8VQi1V4mIw6UyD446ZyN3cQdSVR1sLynR3IT1xYlOw8ZZHY7UaMDpfBkKaGvtS337bOBpZFDNEvBg35z95XXys5Iy/yIQrkKNlv7G8Jhh3kyX0+aTCikAYelH6DJYB7BhS66Pf0OeDLxnoaT4+4rgeMOxFlpsM72vPBpR0JDCOlnHmpajyxrs+
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(71200400001)(6506007)(6512007)(53546011)(186003)(38070700005)(6486002)(122000001)(4326008)(316002)(5660300002)(86362001)(38100700002)(66556008)(26005)(83380400001)(8936002)(66446008)(64756008)(66476007)(76116006)(91956017)(31696002)(36756003)(508600001)(2616005)(2906002)(54906003)(31686004)(8676002)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?WlJkTHR0MVR1c1IwQjBpR2VPNW1UREd4K2lEL0tHSHpvdUVLcnY5NTRTSng4?=
 =?utf-8?B?NXhpN1U1eW9xa2tDOWZIUVRlNXFWaUFGMU9ZbE8zOXJxd3R1NHY4dFQvNUFq?=
 =?utf-8?B?UWlIV3h2ZUNVRnhPMmlBdzlTREJWMytYYTYzME1KZDNlWHBUMStPS0Fxc3FU?=
 =?utf-8?B?QVc3cVQ2RkpFdW8rL2V2TVQ2Z2FLdEJ3SDJncExTc05ZM2paK1NoU2YvYzFB?=
 =?utf-8?B?VENjUHJiS09BWUh6VE5DeVVKd3VDVmgvMkV6REN0QzUwd1FCTU9ydXVQdkRh?=
 =?utf-8?B?UmNXQkVHajZpQ3FVSUlEcE9EU0c1TGphSE5vUHplVjRGNnpsUmtKNWJRMm1N?=
 =?utf-8?B?LzI2RXp5cU05VkpreDVKMkhtWkdkVzl5NE5mVXRuUGY2TGU0UDhFNTJTUWhL?=
 =?utf-8?B?alB0cUxEdERENGdkVjEyektjdThjM1N1RmljU0VsWnQ4RWJmd1Y2aDNWMEZy?=
 =?utf-8?B?SWp4MlVHaFJUUVdtWkpWMmVXb1JmWE1xcndxWFFjTHhsYVUvNUJYdU5mMTZO?=
 =?utf-8?B?cG1KUzNlb09WeTA4czl5U2p6RFBta0tDVVd5c1Z1YjhwdEhURlVHUDNOZVdq?=
 =?utf-8?B?RWtvOWxxQTdKOVFPam45UG9CTFU1TEFnMXBJRUNHb1RTazVxWExCcGdVR0Jl?=
 =?utf-8?B?QlVxL3pzdnZlbVRFbUFRclVrRjdkWHBDV3pQQkdCWXRuRDJjSFg0NEtSdU5x?=
 =?utf-8?B?RmNmQmhDVUJlY1pvR2wvUmx2Zjd4KzZMUHJjTVdjaU10VDNyQ25GS21BV0Fo?=
 =?utf-8?B?RnpkM3M1YSs1RnlzNlM0ZUc5cmhwdDhLUlRvYmV6TDBXa1RxUVIzc2xadmJw?=
 =?utf-8?B?YUM5Vndqa2x3dElnM0VSdUVRd2hpN2tLMmJoQnFRbWVPMFhVZ0d6SGovZllE?=
 =?utf-8?B?b3paN3puWkVCOHpKL2QzZDV1UlVpdDRGdStlbVJETW9tMUIwclRBZHQzNStr?=
 =?utf-8?B?VS9aOU42QVQyTFhLRGRXSG96d2RFTkN2eE13VkVhQU1hVGVhKytzVHZPWTBD?=
 =?utf-8?B?WXBKc3RCUHByV3RQV2NuR05EMmtqMjRFcFpHYm1ZSTl4OXJkeE5MVitJb0Z2?=
 =?utf-8?B?SXNtL01SWjFqVHhSMUVSUjBkK0c5a3FCQmlSODNjMjdRcCtmK0tDWjNUeTdB?=
 =?utf-8?B?STRpMHd4a29aanYyaUNUOGNRWUhhZTZEaUI4N29pWE5UVWNWNld6MG1MV01K?=
 =?utf-8?B?MVBYN0R0NUE0dVRUY0RUdjNrMWdncHQwejZFQWdtZjVTdkZXMG9qbC9Gd3l0?=
 =?utf-8?B?R2RNZDBkRytYUHI4TEtKaWpXK2xiQVkvTGtVc1dsbWxGbjZ3QnRnc0hZbDhP?=
 =?utf-8?B?dTYrTjhESGpUVmppaW1rREI0RXVXNHpnUUZSQzZ5SU5NdFd1ajlFaWhHWlRw?=
 =?utf-8?B?KzFGaDM2YUZydkdBeFpYa3Bqb0lCZnEySlJRQ2Y0VVR2WGJtb2ovMUp6Qkhm?=
 =?utf-8?B?bGx3bDR6Rnd5aGw1Wm5QZElsbWJram5SbFJQL2VwNmJ6OG1iblI4cDQ5eStw?=
 =?utf-8?B?Rmx0T1daQ09ib3VRVFVKMCtiV2ttUktnQVV5aksyNVQrQUlIVU1JUnd5cXlt?=
 =?utf-8?B?R2VuU0RBZHFXMUJwekdPR21FTHlBMngxa2hxM2ZxMmRyQ0pkMjI5UnY4eG9h?=
 =?utf-8?B?OTFicWRqb0JLYnVLSy9qL09hMGhZUHNFdmhWQ2RJQjZRbGpSN01BN3dmZ1dV?=
 =?utf-8?B?M3Q3Y0E3MU41ekNlcWRrK1dFb3FUUkZ1YVJFM2Y3ME5MZkFWSkFnUXBuOUFl?=
 =?utf-8?Q?UayEYM5MYqJlus2KfOacektxmWXHh69EZYUF7TA?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <CAE81B8B40FE1048B16615AF98F72ADC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e73ba585-155e-4a6b-15be-08d973729392
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 09:16:58.7482
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: o6kvI8FesfDkzlcNLqb3wXmtclu4JXVjcW5pGoWFlPBM585LG6mWoTIS3pf+o22b2mdUTub6uMbE3KqwBBSDb4LwTw1TD9yjZ1Nr+FaNXKTPqZQO4COZsJO0/6f6csFW
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6963
X-Proofpoint-GUID: xk3LBsux2er1dBVXKsXmlN4ZXT_FJch1
X-Proofpoint-ORIG-GUID: xk3LBsux2er1dBVXKsXmlN4ZXT_FJch1
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_03,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0
 bulkscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0
 impostorscore=0 spamscore=0 mlxlogscore=993 priorityscore=1501
 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090056

DQpPbiAwOS4wOS4yMSAxMToyNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MDg6MTMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDYuMDkuMjEgMTc6
NTEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDAzLjA5LjIwMjEgMTI6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBAQCAtMzcsMTIgKzQxLDI4IEBAIHN0YXRpYyBp
bnQgbWFwX3JhbmdlKHVuc2lnbmVkIGxvbmcgcywgdW5zaWduZWQgbG9uZyBlLCB2b2lkICpkYXRh
LA0KPj4+PiAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nICpjKQ0KPj4+PiAg
ICB7DQo+Pj4+ICAgICAgICBjb25zdCBzdHJ1Y3QgbWFwX2RhdGEgKm1hcCA9IGRhdGE7DQo+Pj4+
ICsgICAgZ2ZuX3Qgc3RhcnRfZ2ZuOw0KPj4+PiAgICAgICAgaW50IHJjOw0KPj4+PiAgICANCj4+
Pj4gICAgICAgIGZvciAoIDsgOyApDQo+Pj4+ICAgICAgICB7DQo+Pj4+ICAgICAgICAgICAgdW5z
aWduZWQgbG9uZyBzaXplID0gZSAtIHMgKyAxOw0KPj4+PiAgICANCj4+Pj4gKyAgICAgICAgLyoN
Cj4+Pj4gKyAgICAgICAgICogQW55IEJBUiBtYXkgaGF2ZSBob2xlcyBpbiBpdHMgbWVtb3J5IHdl
IHdhbnQgdG8gbWFwLCBlLmcuDQo+Pj4+ICsgICAgICAgICAqIHdlIGRvbid0IHdhbnQgdG8gbWFw
IE1TSSByZWdpb25zIHdoaWNoIG1heSBiZSBhIHBhcnQgb2YgdGhhdCBCQVIsDQo+Pj4+ICsgICAg
ICAgICAqIGUuZy4gd2hlbiBhIHNpbmdsZSBCQVIgaXMgdXNlZCBmb3IgYm90aCBNTUlPIGFuZCBN
U0kuDQo+Pj4+ICsgICAgICAgICAqIEluIHRoaXMgY2FzZSBNU0kgcmVnaW9ucyBhcmUgc3VidHJh
Y3RlZCBmcm9tIHRoZSBtYXBwaW5nLCBidXQNCj4+Pj4gKyAgICAgICAgICogbWFwLT5zdGFydF9n
Zm4gc3RpbGwgcG9pbnRzIHRvIHRoZSB2ZXJ5IGJlZ2lubmluZyBvZiB0aGUgQkFSLg0KPj4+PiAr
ICAgICAgICAgKiBTbyBpZiB0aGVyZSBpcyBhIGhvbGUgcHJlc2VudCB0aGVuIHdlIG5lZWQgdG8g
YWRqdXN0IHN0YXJ0X2dmbg0KPj4+PiArICAgICAgICAgKiB0byByZWZsZWN0IHRoZSBmYWN0IG9m
IHRoYXQgc3Vic3RyYWN0aW9uLg0KPj4+PiArICAgICAgICAgKi8NCj4+Pj4gKyAgICAgICAgc3Rh
cnRfZ2ZuID0gZ2ZuX2FkZChtYXAtPnN0YXJ0X2dmbiwgcyAtIG1mbl94KG1hcC0+c3RhcnRfbWZu
KSk7DQo+Pj4gSSBtYXkgYmUgbWlzc2luZyBzb21ldGhpbmcsIGJ1dCBkb24ndCB5b3UgbmVlZCB0
byBhZGp1c3QgInNpemUiIHRoZW4NCj4+PiBhcyB3ZWxsPw0KPj4gTm8sIGFzIHJhbmdlIHNldHMg
Z2V0IGNvbnN1bWVkIHdlIGhhdmUgZSBhbmQgcyB1cGRhdGVkIGFjY29yZGluZ2x5LA0KPj4gc28g
ZWFjaCB0aW1lIHNpemUgcmVwcmVzZW50cyB0aGUgcmlnaHQgdmFsdWUuDQo+IEl0IGZlZWxzIGxp
a2Ugc29tZXRoaW5nJ3Mgd3Jvbmcgd2l0aCB0aGUgcmFuZ2VzZXQgY29uc3RydWN0aW9uIHRoZW46
DQo+IEVpdGhlciBpdCByZXByZXNlbnRzIF9hbGxfIGhvbGVzIChpbmNsdWRpbmcgZGVnZW5lcmF0
ZSBvbmVzIGF0IHRoZQ0KPiBzdGFydCBvZiBlbmQgb2YgYSByYW5nZSksIG9yIG5vbmUgb2YgdGhl
bS4NCg0KVGhlIHJlc3VsdGluZyByYW5nZSBzZXQgb25seSBoYXMgdGhlIE1NSU9zIGluIGl0LiBX
aGlsZSBjb25zdHJ1Y3RpbmcgdGhlIHJhbmdlIHNldA0KDQp3ZSBjdXQgb2ZmIE1TSS1YIG91dCBv
ZiBpdCAobWFrZSBob2xlcykuIEJ1dCBmaW5hbGx5IGl0IG9ubHkgaGFzIHRoZSByYW5nZXMgdGhh
dCB3ZQ0KDQpuZWVkIHRvIG1hcC91bm1hcC4NCg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0K
T2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:21:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:21:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182855.330664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGFT-0007b6-8m; Thu, 09 Sep 2021 09:21:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182855.330664; Thu, 09 Sep 2021 09:21:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGFT-0007az-5q; Thu, 09 Sep 2021 09:21:35 +0000
Received: by outflank-mailman (input) for mailman id 182855;
 Thu, 09 Sep 2021 09:21:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGFS-0007at-0p
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:21:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 521d2da6-114f-11ec-b1a8-12813bfff9fa;
 Thu, 09 Sep 2021 09:21:32 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-Y-NZ8-HwOUGsaMPaZ45nWg-1; Thu, 09 Sep 2021 11:21:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Thu, 9 Sep
 2021 09:21:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:21:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0033.eurprd03.prod.outlook.com (2603:10a6:208:14::46) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 09:21:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 521d2da6-114f-11ec-b1a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631179291;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=qCf+5Yd/fZvb/dblph5Mi3U37xAhwaBmRYC/0Jhk3P4=;
	b=NSWlC/Cwso2yk2dvORFHUlfeFIvVZjl6w4PB9EpRQg3Y0vfM5+Y1ruyFhLCXbHifzzg7UD
	DP0duIm4CVKqx4kQZykbVOsXtpU7XAMDfPJtUBug2UbwSEKvlpAkybkkJMCSuY5+Pgvggk
	+SpRvxS+5GR1oJyyEkUuFCQ452XJpDg=
X-MC-Unique: Y-NZ8-HwOUGsaMPaZ45nWg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T3++X5Ywxv7Nm67jYvHXB0VSViN1gt4IMaPrXnl3tnrVUFfmpnk25hlh9fjCSeoj5jDbBxYsy3bRb0jOWFfG/ik1Uee2qkNI/Jl9emb4J0/RhyOb9z+l3Ha85WkvH3t4H9tSd1Y98ziFxH1H2VlQDC2bkBfSGCffnjH+ePOOmrhZXqn9V+AICabsBb7rWjRgQrz3Fuyjgz/TWs6DPdnPNB4821s0q2zO7N6kk0YCVyJ596J+lkx2wFzlmhYKLbFUuI0kMAxaO8aQWOzlPHBRhaTXc6xTnhjdL4S7gJXoIRB/w337DopEOzIzgfONjyTUWRQrzoXGg4ESNjPN32pJMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qCf+5Yd/fZvb/dblph5Mi3U37xAhwaBmRYC/0Jhk3P4=;
 b=Tmsg3dQ6fGbWIEKWkpHRZpPnGRe7M7LslhLeYbodZODBqa6nGo/ZqkbVvLRGC5tF8y2alI71YLaaGuzaKuUd5AOGrB7JLBU8DKy5/48YIb/ruxH4OViMtujjIvYxlMEz+z2IAb7L1k/MZ+TDtQ4cM3F9qkJt+SbuE7PylUJ6qJiDfLOAihjx7yoJoApB4LT24FCKwWnZ2aT0B+gd8NYTweolrs8B0L1p74lWnur91YuL5KYmdTjPwTmNLtywEQExgnl3tDJdgTFEmiQwKH2QkYeIgMN57BpZzBZb25G2p4WAAPEa+6vQs/IgFX8VZ5yEV3sR4OZigO1p2gabeRZOGA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
Date: Thu, 9 Sep 2021 11:21:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0033.eurprd03.prod.outlook.com
 (2603:10a6:208:14::46) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1b992711-1d68-4be6-98fb-08d973733448
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB393336D915BBDA82117490C3B3D59@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dpT5mUI6Z0I9ebyzosrVNxaSjDk/fC1LkVz7ErS4J4/WiVwwVCAKrWPLUw1+zdE1Wq/RzLaamgoa1M/VP7Tk870RjdbLAac5dojvu8m+6Ew9xKb5pUDmXQKNqosMTDTpAQawCcilu1BIRBaYfZ4ZKvzV8OCphSFLLR9h5BE8UyOwOgiK4lfi1EyTqRrrfIwh5l9jhz7wZUNUOTE92E0xn7dwr5jACkBljOqlKMmukr/3WKK9KwdgapeN00XOOKLgUUSRFAayD4Szm/ehHG1WI3AKZGxrA4c/8nB+pTrzpETWo4CEyAp1hdqHvRi0zoaw82w9g2j7aRIXSexBha+i8zZps3+GCu84op4s0Jft/OzxGVgxNjsuZWCeJiX2Kyxm7cb4eZ8ZfU6S2mDKBR9sBFcbcV2H026HgYfEmr4pNJWWFQWc7SzNFe452FepdhodkdOfkCcWMGZK7CcCXSjdfRopTieEdWqJgle5KebL1BrjxHZT4/1vy4nZRUSeB/p9qtHApZ63tNEkzgSrSieRDP7R3ibUfenyXZHgAh+3qstNrBtUJHPamRmv8ZDyGLTGBKmS5xIrezHoWm8xQiq2Yvwx4k8wQfDLfI4pfnNInBWnhIIl1X5tNZUZoh+ElFlcM69dEFcgBLLJKW+Vlxk2wB7Zgb88/WVDYfnKQZvpwmfS8Vsca5F/ebc6nWGfAfyjLhSM7IZSw71/s5adNEYs8Rr5YDqnUY1HYzwuLAOB/Lk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(39860400002)(376002)(366004)(346002)(6486002)(956004)(2906002)(2616005)(31696002)(38100700002)(26005)(86362001)(7416002)(83380400001)(36756003)(186003)(4326008)(478600001)(316002)(66476007)(16576012)(8676002)(31686004)(66946007)(53546011)(5660300002)(54906003)(110136005)(8936002)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OFdGMlJPaFdWZXlCTVEwRUM5aFZDK2loVG83TkVIaUZvckduR09WbGFrZzlP?=
 =?utf-8?B?dkxYV0R4M24wcFlub0pzaXQ3YXVoSms5YXd5dk5HTVpqMXc3L3l2MytUeDlh?=
 =?utf-8?B?Mk5PdzlVSjRxWnFiN2xjenFvcStJcmE4Y1BXdktuMVE4NGlKa0huYU9VeEN0?=
 =?utf-8?B?N2F6enVZYml2N3JQS1BBeTlFSHJ3SnRvYll0VlpQbmFhbDc3aFovYkFmWStv?=
 =?utf-8?B?KzdDQ1BweGRXemhDOGVwS0t0OThhL0FzNUpVc3hiMnJpcmdyUWZkaFJsWmVw?=
 =?utf-8?B?dHRVUlRnMWhtNWd4eUwxWm92aGp6YlZxMnAzeHVuR3NaU2hIdzFrdUhJL1JB?=
 =?utf-8?B?bVo4YzIyaFpOSFp2Mm45TjhmMEsvWnVrYUNBNWlkVFV1eEpwZHp5T1JCTlJ3?=
 =?utf-8?B?bVUxQ04xT21CbXVHTXBaZFljVDBMY0RSbjBtVnpMK1ZqWTFFSDBGM29Mbko4?=
 =?utf-8?B?OGtoaTNDQ21VdUFOQ0YxQzMwQmMyQ0dHNjQvV0xpVHRneWZTZ2piaDlwWTR4?=
 =?utf-8?B?RmlDSWlzVTVKb3BVV2Zna21kYkViZlEzdXlBd2RUbjFGN2diVzhKU3h0eHJw?=
 =?utf-8?B?N3hRa1lYZ3Exb3E4eHk0eEsrWTJReVRPR00yeG5EYUV3NGU3NXg5WHVOSndZ?=
 =?utf-8?B?KzJ6VVJtTksxUE5EQjJmSHZqQ3FGUkVPT2IyeXdFYmhvbHRzR0RtaWZZNUYy?=
 =?utf-8?B?WUR2M1ZyWTlpNERLQjRldlFBeXZKM1A0ejRwejdHTjVreEFjVitKeExjMFhF?=
 =?utf-8?B?eUpNUThsODAvb0VMUHdldnRyNW81Ynhhb1ZRS0JsblpKK3kxUTNCZEpNNUEz?=
 =?utf-8?B?K2VaVWVZMXVzRkFaQ3pxT3lkQ2U1TU94eW9lbWFjQnpXV0RvcFRQTVpkbzZy?=
 =?utf-8?B?VVVHQ3BqT2pQVGxQSHZPSWR2RXVZWXJhYzFUdkZ3UXFwTTlCbGFIajdNMzRC?=
 =?utf-8?B?L1pGVi81b2pXVklWMFRLREs4YUVGeEZpYjkyM2dWNHFsYW5jTjViOFNFbVZa?=
 =?utf-8?B?U1lNK1R3TlpWMTh2aFU1NWNmUFJjZitpT2NyT2thakw1bjc0TTFiWkhjRXA1?=
 =?utf-8?B?QkdCVkhjc3VONjV3SHAxWUkvSndvS0JiRlhQRWs1Z1AwdW9Qa3ZmNzVYc2pw?=
 =?utf-8?B?S1FlWDVNNlJtcElVaUdJeVZKd3dJSGlDd2dJbGdpQkpKSjU5TzFvUmdqY1k5?=
 =?utf-8?B?SUltdmpIcDE3Z0FtTDB0K1ZscE56OFJzSmFtWjhXVk1oUWQ0UWhPeVpkeWJJ?=
 =?utf-8?B?eVJOay8vUHNWS3ZxL3BYZi83NGsvdUZKQnYvZ3BNT2JpRk5DWmt4REpCOTQy?=
 =?utf-8?B?M2U0ZDZMRmMzYkk4Q29pUm4waFJCU0VZWmNtTUpRYmUyVmpGVDlleVJicnhM?=
 =?utf-8?B?djAwc1d2TFBRQkVpTi9PZ2gzZkVJMkdubjRCQmtUc29UamRBOFF3SUZGelpY?=
 =?utf-8?B?NVR2LytEMUZhcHNZSzZZTVdoOUp2ZGpoWmhwYkx5TlNvM3FzRE9YNWdYQmdr?=
 =?utf-8?B?K1ByQys1Zk5TcjBBUU1QcHlFRkZ2b2J3L0g1QzFYQ1o0V01xODNwQlBxdFR6?=
 =?utf-8?B?blA4NHJ5VGN4SlQzSzRVN1FQRjRrbk9OaEdSTXJrZk9Jc1hDb2lraDJ6QU5o?=
 =?utf-8?B?M3d2UC9LVVB0aXFKckMvUjBXYlN0Q0pvQVNoT1FKT09hTTBqWEI5YWRoWmgr?=
 =?utf-8?B?dHRFdVlYQVhZbktvOFRlbkFmQm5icDVVZ3F3eTkzbXhub1JUQ3pNYVllQkJo?=
 =?utf-8?Q?20DiCY5TrPjOD7zG5oVAgJqhYAFJvv59Q0VMyZA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1b992711-1d68-4be6-98fb-08d973733448
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:21:28.6441
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A/NlnfQu71CUvDUwREDhu4LSXBHWBqqsGLGafyaWWmmsnQvNIfgxOb6E5TrC4H6nblXgwHjVUTdqIk3AipXN4Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 09.09.2021 10:50, Oleksandr Andrushchenko wrote:
> 
> On 09.09.21 11:43, Jan Beulich wrote:
>> On 09.09.2021 10:39, Oleksandr Andrushchenko wrote:
>>> On 07.09.21 13:06, Jan Beulich wrote:
>>>> On 07.09.2021 11:52, Oleksandr Andrushchenko wrote:
>>>>> On 07.09.21 12:19, Jan Beulich wrote:
>>>>>> On 07.09.2021 11:07, Oleksandr Andrushchenko wrote:
>>>>>>> On 07.09.21 11:49, Jan Beulich wrote:
>>>>>>>> On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>>>>>>>>> So, if we have a hidden PCI device which can be assigned to a guest and it is literally untouched
>>>>>>>>> (not enabled in Dom0) then I think there will be no such reference as "host assigned values" as
>>>>>>>>> most probably the command register will remain in its after reset state.
>>>>>>>> What meaning of "hidden" do you imply here? Devices passed to
>>>>>>>> pci_{hide,ro}_device() may not be assigned to guests ...
>>>>>>> You are completely right here.
>>>>>>>> For any other meaning of "hidden", even if the device is completely
>>>>>>>> ignored by Dom0,
>>>>>>> Dom0less is such a case when a device is assigned to the guest
>>>>>>> without Dom0 at all?
>>>>>> In this case it is entirely unclear to me what entity it is to have
>>>>>> a global view on the PCI subsystem.
>>>>>>
>>>>>>>>      certain of the properties still cannot be allowed
>>>>>>>> to be DomU-controlled.
>>>>>>> The list is not that big, could you please name a few you think cannot
>>>>>>> be controlled by a guest? I can think of PCI_COMMAND_SPECIAL(?),
>>>>>>> PCI_COMMAND_INVALIDATE(?), PCI_COMMAND_PARITY, PCI_COMMAND_WAIT,
>>>>>>> PCI_COMMAND_SERR, PCI_COMMAND_INTX_DISABLE which we may want to
>>>>>>> be aligned with the "host reference" values, e.g. we only allow those bits
>>>>>>> to be set as they are in Dom0.
>>>>>> Well, you've compile a list already, and I did say so before as well:
>>>>>> Everything except I/O and memory decoding as well as bus mastering
>>>>>> needs at least closely looking at. INTX_DISABLE, for example, is
>>>>>> something I don't think a guest should be able to directly control.
>>>>>> It may still be the case that the host permits it control, but then
>>>>>> only indirectly, allowing the host to appropriately adjust its
>>>>>> internals.
>>>>>>
>>>>>> Note that even for I/O and memory decoding as well as bus mastering
>>>>>> it may be necessary to limit guest control: In case the host wants
>>>>>> to disable any of these (perhaps transiently) despite the guest
>>>>>> wanting them enabled.
>>>>> Ok, so it is now clear that we need a yet another patch to add a proper
>>>>> command register emulation. What is your preference: drop the current
>>>>> patch, implement command register emulation and add a "reset patch"
>>>>> after that or we can have the patch as is now, but I'll only reset IO/mem and bus
>>>>> master bits, e.g. read the real value, mask the wanted bits and write back?
>>>> Either order is fine with me as long as the result will be claimed to
>>>> be complete until proper emulation is in place.
>>> I tried to see what others do in order to emulate PCI_COMMAND register
>>> and it seems that at most they care about the only INTX bit (besides
>>> IO/memory enable and bus muster which are write through). Please see
>>> [1] and [2]. Probably I miss something, but it could be because in order
>>> to properly emulate the COMMAND register we need to know about the
>>> whole PCI topology, e.g. if any setting in device's command register
>>> is aligned with the upstream port etc. This makes me think that because
>>> of this complexity others just ignore that. Neither I think this can be
>>> easily done in our case. So I would suggest we just add the following
>>> simple logic to only emulate PCI_COMMAND_INTX_DISABLE: allow guest to
>>> disable the interrupts, but don't allow to enable if host has disabled
>>> them. This is also could be tricky a bit for the devices which are not
>>> enabled and thus not configured in Dom0, e.g. we do not know for sure
>>> if the value in the PCI_COMMAND register (in particular
>>> PCI_COMMAND_INTX_DISABLE bit) can be used as the reference host value or
>>> not. It can be that the value there is just the one after reset or so.
>>> The rest of the command register bits will go directly to the command
>>> register untouched.
>>> So, at the end of the day the question is if PCI_COMMAND_INTX_DISABLE
>>> is enough and how to get its reference host value.
>> Well, in order for the whole thing to be security supported it needs to
>> be explained for every bit why it is safe to allow the guest to drive it.
> 
> So, do we want at least PCI_COMMAND_INTX_DISABLE bit aligned
> between the host and guest? If so, what do you you think about
> the reference value for it (please see above).

Please may I ask that you come up with a proposal? I don't think I've
said you need to emulate this or any of the other bits. All I've asked
for is that for every bit you allow the guest to control directly, you
justify why that's safe and secure. If no justification can be given,
emulation is going to be necessary. How to solve that is first and
foremost part of your undertaking.

For the bit in question, where the goal appears to be to have hardware
hold the OR of guest and host values, an approach similar to that used
for some of the MSI / MSI-X bits might be chosen: Maintain guest and
host bits in software, and update hardware (at least) when the
effective resulting value changes. A complicating fact here is, though,
that unlike for the MSI / MSI-X bits here Dom0 (pciback or its PCI
susbstem) may also have a view on what the setting ought to be.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:34:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:34:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182861.330676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGRs-0000y1-G7; Thu, 09 Sep 2021 09:34:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182861.330676; Thu, 09 Sep 2021 09:34:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGRs-0000xu-D6; Thu, 09 Sep 2021 09:34:24 +0000
Received: by outflank-mailman (input) for mailman id 182861;
 Thu, 09 Sep 2021 09:34:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fJWR=N7=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOGRq-0000xX-Fa
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:34:22 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.59]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1b9d27b6-1151-11ec-b1a8-12813bfff9fa;
 Thu, 09 Sep 2021 09:34:20 +0000 (UTC)
Received: from AM6P193CA0048.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:8e::25)
 by VI1PR0802MB2333.eurprd08.prod.outlook.com (2603:10a6:800:9f::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.24; Thu, 9 Sep
 2021 09:34:17 +0000
Received: from VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8e:cafe::e5) by AM6P193CA0048.outlook.office365.com
 (2603:10a6:209:8e::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Thu, 9 Sep 2021 09:34:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT015.mail.protection.outlook.com (10.152.18.176) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Thu, 9 Sep 2021 09:34:17 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Thu, 09 Sep 2021 09:34:16 +0000
Received: from 07792fdaa4da.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5DEF0512-3D12-4BB4-9E1D-B7648F044389.1; 
 Thu, 09 Sep 2021 09:34:06 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 07792fdaa4da.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 09 Sep 2021 09:34:06 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB2672.eurprd08.prod.outlook.com (2603:10a6:802:1c::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Thu, 9 Sep
 2021 09:34:04 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4478.026; Thu, 9 Sep 2021
 09:34:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b9d27b6-1151-11ec-b1a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pVLiyRJmeubHdsZ4iEmGaXE0d3O/Gu+hmo+4GLSGD0g=;
 b=8nxUEsgresHP2b9vnaqoDDY1rAVHHG8l+jI4PaPW5MXZc1e+q60eQjbY6AD42V/gkfYeIw+PMbMcYMqTiFu4DzhuvUaRXFaNLEgx4X6umHKn/1s/1TH3XbAjmSLSvXKdBELq4C9AxRkZcvl/+cgULlWOg9E3uYuxYrvEpFhyq7E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nEWd5ZiraaixogvwoLR5qxYYJUVXsVz71PIiUgIhDcFBeSShgUDtDWS6nYo+dNoQP6vJTncbO4WpT4qBLEj3nDsOy13aB/+xhYEXrbROwQtsWJcpRnHbJfHdSKHr9X/7/cFUMLSHn8mBA9TNRNtVv19PdDzMqiT2yWL6LT9JxE0DVkbi3OSghnc0ORTS5PawOGtUJS+x4mkLhrAhzBJNGnEleX64/gG/vvR8UlI4n1R+/djstjUbcf1rVqDslBdkj1x/g9iM2hXLPpZTNVdJHoAyg0RPn9ct49v077zjYvHi1DfjK7HrVVkrzoUC1IHY4DPGOcKhiOKjPMifRtfJsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pVLiyRJmeubHdsZ4iEmGaXE0d3O/Gu+hmo+4GLSGD0g=;
 b=HJzKBDuLPlWFLp+4EHTOq5Gurus4l4l6nrmDUXJN9Ny1yRZylj3WrZNiIvBjMrjysyzfHIhPNQsJ4jvJBaIITOJ1HDFfu1aAbgmmnXOEWjcjulF+Kzs9+PBzbUGAsfNM578CdIhiokGaq7hV43VmBV2O6JMcUaqGGewnOgS3ge3IbKxQkBYNnE7qS56wofNoGuWUAcspjt01F/d7UzbKpD3FyVQEG6X9Lt3h1CvUXD3zGx4k0KXrtf4zzq0xkOzWqYfjZQ89KGwxZf8ZgUMtTMXx5AW/4JpgzzrPJyiDX6alit9x5LMfn8/+DbVuSENS/X04ULMiiYgr08y3ffHy+w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pVLiyRJmeubHdsZ4iEmGaXE0d3O/Gu+hmo+4GLSGD0g=;
 b=8nxUEsgresHP2b9vnaqoDDY1rAVHHG8l+jI4PaPW5MXZc1e+q60eQjbY6AD42V/gkfYeIw+PMbMcYMqTiFu4DzhuvUaRXFaNLEgx4X6umHKn/1s/1TH3XbAjmSLSvXKdBELq4C9AxRkZcvl/+cgULlWOg9E3uYuxYrvEpFhyq7E=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>
Subject: RE: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
Thread-Topic: [PATCH v6 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
Thread-Index: AQHXpVn2rWpL2DewC0y7cp371PcYX6ubbG4Q
Date: Thu, 9 Sep 2021 09:34:04 +0000
Message-ID:
 <VE1PR08MB5215F0A9EAB39A15E15B73F9F7D59@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210908095248.545981-1-penny.zheng@arm.com>
 <20210908095248.545981-6-penny.zheng@arm.com>
 <da3dafd3-9e61-a7d5-b5cf-897d73d0fd16@suse.com>
In-Reply-To: <da3dafd3-9e61-a7d5-b5cf-897d73d0fd16@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: E4FF70204E49514B90295B8ECEC4633D.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ef253a11-8249-487b-2583-08d97374fe69
x-ms-traffictypediagnostic: VI1PR08MB2672:|VI1PR0802MB2333:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0802MB2333559DB933A4A9F5F8A69DF7D59@VI1PR0802MB2333.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2399;OLM:2399;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 TqN+Lslwi+IpA4PxksHGxk2bxMLX3x3RmSk4ZqfoC09JtKH5laD4Thnj56ipZh62Ep+fCKCfRffXaJWBjh5APp3X4+MZruNg+WHHM1s76jN+Nrp8B4s/iwE/MaChIyhz7pthV7ldumGaSpd7ffUWdComQEK/3SIBdHKp64jaLry9Yp/PE4hIgBUHRNGDH8g1yjtqERH638b09XsyqCHkqDI76I8eH8TuV6ZmqncgcY/GTj5A+5ihrXswSoc+e59oM3XUkI2NDnzCANrkX/z5xqcJJZkKOv0YGqop8g488Egy5d01jaqk05h95f8a8wztMAM8RiyeCU9c8OPS6WZsRUDx5zoPXosX/hHO7PlYYDW77QX6mQT8Y45f+cgm96Yqri+9kDTGB+oAP+nj3pLF2XCH0nRFP220lsTNPlxvXZe3wkm66m9Ci2qJsTPKb7dDiy/Sn6TaFYKSqes5Yp9Wj+10POAAcSblpWTFbZ4EOgFOwcg3G7qxlcC8pXUGRfrk7F7jFWRYqsUfRWqxyNbAtbx/+M4R5lmLIgJ1UWYfYUj5B5Uq4If9PtWLiRGRQ1gXJvpjBlVDowjHlZFwdmtXXtca+G0OlAVrgTrPXBfCh4CYVFKynIp6iEa+UAEZ4GoJ5tOHa8kl4PD78r9AUJHpmsyUZMbFk0Nxqcx0AkEcBhIOHiPhRrVfFAaG8u1rfxV4Fua9ti0RV1bN7is6jbgp2A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(136003)(396003)(366004)(86362001)(53546011)(6506007)(71200400001)(38070700005)(186003)(26005)(9686003)(55016002)(33656002)(478600001)(316002)(8676002)(64756008)(76116006)(54906003)(66946007)(66476007)(66556008)(5660300002)(66446008)(4326008)(7696005)(52536014)(83380400001)(8936002)(38100700002)(122000001)(110136005)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB2672
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	17c90321-21aa-4146-5349-08d97374f6d7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nHUtbYUNZk/wwk21B0JZLPBxFIFEUYt6Um3ApQbLerF7DUUUyR63NqFcxprAP9296a5hVzdCFZlUPxzkhOLUxi+eSQKH/Za/sAEVkwb2y3Ii8bjRtpN5rJmHW7cAtFFSkfbQy6Zi99OWBY4yqfLroeZfpl7ej02FqGjdS5yvGT9dzODM1Z0ePk1yi0FM7NtPyPOM05x+uZinho/XDuQGNhrCkIThHu+ZbemicTvIVNKUfChdfshMVwiiO8S6MBwOe4PsMyTlyBV/lTHjxVhPoM3KCdTdiJvhehtLrAu6eR9m+BScr1tR+wo+Us2pd2QfCmgTRDEIv7R9IyRniuQPvh7IHguMcQgi5KZLLoGXN6t7arNvrN2xxvbcJBBpE77dIc7EMM+PwC4MqIQJRj2rU0cOYFf3vGyLGEMYcmiaB7KQ7e5YeWViafCvnA8gKRImnfAgd8u2Z6oM28w3t8Dyy+KA8da15Henmfi0sV1+BGjgCTnwqVbeid67wSCNhzObBKi4EFd+BM0PuoRJW29hvzBa6hTc4MZINQh5MwD/q+BoJiZczjsmxz/YGFnoLD10SPe07gWRzWhwQh8POi4Z+5JQHy2G1SxARhiTsSITpmKVyX5iIQ26gJxEoVHdqgmL3kexnkcIAWg4iRI3DPza7iShGQN/6TfkhShKOLbZZU+u8H7ULCROKs9w7bdDwNHGk1ftpLYQL7gfLaTEFM0TsA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(346002)(39860400002)(396003)(376002)(36840700001)(46966006)(7696005)(86362001)(54906003)(52536014)(478600001)(81166007)(82740400003)(47076005)(83380400001)(110136005)(6506007)(70206006)(70586007)(82310400003)(53546011)(336012)(26005)(186003)(8936002)(33656002)(4326008)(8676002)(5660300002)(2906002)(316002)(36860700001)(9686003)(55016002)(356005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:34:17.0170
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ef253a11-8249-487b-2583-08d97374fe69
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2333

SGkgSmFuIGFuZCBTdGVmYW5vDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJv
bTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KPiBTZW50OiBUaHVyc2RheSwgU2Vw
dGVtYmVyIDksIDIwMjEgNTowNiBQTQ0KPiBUbzogUGVubnkgWmhlbmcgPFBlbm55LlpoZW5nQGFy
bS5jb20+DQo+IENjOiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
OyBXZWkgQ2hlbg0KPiA8V2VpLkNoZW5AYXJtLmNvbT47IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9q
ZWN0Lm9yZzsNCj4gc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVuQHhlbi5vcmcNCj4gU3Vi
amVjdDogUmU6IFtQQVRDSCB2NiA1LzddIHhlbjogcmUtZGVmaW5lIGFzc2lnbl9wYWdlcyBhbmQg
aW50cm9kdWNlIGEgbmV3DQo+IGZ1bmN0aW9uIGFzc2lnbl9wYWdlDQo+IA0KPiBPbiAwOC4wOS4y
MDIxIDExOjUyLCBQZW5ueSBaaGVuZyB3cm90ZToNCj4gPiBJbiBvcmRlciB0byBkZWFsIHdpdGgg
dGhlIHRyb3VibGUgb2YgY291bnQtdG8tb3JkZXIgY29udmVyc2lvbiB3aGVuDQo+ID4gcGFnZSBu
dW1iZXIgaXMgbm90IGluIGEgcG93ZXItb2YtdHdvLCB0aGlzIGNvbW1pdCByZS1kZWZpbmUNCj4g
PiBhc3NpZ25fcGFnZXMgZm9yIG5yIHBhZ2VzIGFuZCBpbnRyb2R1Y2VzIGEgbmV3IGhlbHBlciBh
c3NpZ25fcGFnZSBmb3INCj4gb3JpZ2luYWwgcGFnZSB3aXRoIGEgc2luZ2xlIG9yZGVyLg0KPiA+
DQo+ID4gU2lnbmVkLW9mZi1ieTogUGVubnkgWmhlbmcgPHBlbm55LnpoZW5nQGFybS5jb20+DQo+
IA0KDQpTdGVmYW5vLCBTaW5jZSBJIG5lZWQgdG8gcmUtY29tbWl0IHRoaXMgb25lLCBJJ2xsIGFk
ZC1pbiB5b3VyIE5JVCBzdWdnZXN0aW9uIGluIGNvbW1pdCA3KCINCnhlbi9hcm06IGludHJvZHVj
ZSBhbGxvY2F0ZV9zdGF0aWNfbWVtb3J5IiksIGFuZCBwdXNoIGEgbmV3IFNlcmllIGFzYXAuIDsp
DQoNCj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQgSSdtIG5vdyB2ZXJ5IHB1enpsZWQ6IEluc3RlYWQg
b2YgcmVzdG9yaW5nIHRoZSBsb25nIGFncmVlZA0KPiB1cG9uIG9yZGVyaW5nIG9mIHBhcmFtZXRl
cnMgKGFuZCB0aGVuIGtlZXBpbmcgbXkgQS1iKSwgeW91J3ZlIGRyb3BwZWQgdGhlDQo+IGFjay4N
Cj4gDQo+ID4gLS0tIGEveGVuL2FyY2gveDg2L3B2L2RvbTBfYnVpbGQuYw0KPiA+ICsrKyBiL3hl
bi9hcmNoL3g4Ni9wdi9kb20wX2J1aWxkLmMNCj4gPiBAQCAtNTY4LDcgKzU2OCw3IEBAIGludCBf
X2luaXQgZG9tMF9jb25zdHJ1Y3RfcHYoc3RydWN0IGRvbWFpbiAqZCwNCj4gPiAgICAgICAgICBl
bHNlDQo+ID4gICAgICAgICAgew0KPiA+ICAgICAgICAgICAgICB3aGlsZSAoIGNvdW50LS0gKQ0K
PiA+IC0gICAgICAgICAgICAgICAgaWYgKCBhc3NpZ25fcGFnZXMoZCwgbWZuX3RvX3BhZ2UoX21m
bihtZm4rKykpLCAwLCAwKSApDQo+ID4gKyAgICAgICAgICAgICAgICBpZiAoIGFzc2lnbl9wYWdl
cyhkLCBtZm5fdG9fcGFnZShfbWZuKG1mbisrKSksIDEsIDApDQo+ID4gKyApDQo+IA0KPiBUaGlz
IGNoYW5nZSBhbG9uZSBkZW1vbnN0cmF0ZXMgdGhlIHByb2JsZW0gd2hlbiBpdCBjb21lcyB0byBi
YWNrcG9ydGluZw0KPiBmdXR1cmUgY2hhbmdlczogSWYgdGhlIG9yaWdpbmFsIHBhdGNoIGNvbnRh
aW5lZCBhIGNvZGUgYWRkaXRpb24gc2ltaWxhciB0byB3aGF0DQo+IHlvdSBjaGFuZ2UgdG8sIHdp
dGhvdXQgdGhlIHBlcnNvbiBkb2luZyB0aGUgYmFja3BvcnRpbmcgcGF5aW5nIGNsb3NlDQo+IGF0
dGVudGlvbiwgdGhlIHJlc3VsdCB3aWxsIGJlIGFuIG9yZGVyLTEgcmVxdWVzdCB3aGVuIGFuIG9y
ZGVyLTAgb25lIGlzIHdhbnRlZC4NCj4gSXQgd2FzIGV4cGxhaW5lZCB0byB5b3UgdGhhdCBpbiBv
cmRlciB0byBtYWtlIHBlb3BsZSBkb2luZyBiYWNrcG9ydHMgYXdhcmUgb2YNCj4gdGhpcyBzZW1h
bnRpYyBjaGFuZ2UsIHRoZSBvcmRlciBvZiBwYXJhbWV0ZXJzIG9mIHRoZSBmdW5jdGlvbiBvdWdo
dCB0byBiZQ0KPiBhbHRlcmVkLiBUaGF0IHdheSB0aGUgY29tcGlsZXIgd2lsbCBjb21wbGFpbiwg
YW5kIHRoZSBwZXJzb24gd2lsbCBrbm93IHRvIGxvb2sNCj4gY2xvc2VseSB3aGF0IGFkanVzdG1l
bnRzIGFyZSBuZWVkZWQuDQo+IA0KPiBJbiB0aGlzIGNvbnRleHQgSSBmaW5kIGl0IGZ1cnRoZXIg
cHV6emxpbmcgLi4uDQo+IA0KPiA+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9tbS5oDQo+ID4gKysr
IGIveGVuL2luY2x1ZGUveGVuL21tLmgNCj4gPiBAQCAtMTMzLDggKzEzMywxNCBAQCB2b2lkIGhl
YXBfaW5pdF9sYXRlKHZvaWQpOw0KPiA+DQo+ID4gIGludCBhc3NpZ25fcGFnZXMoDQo+ID4gICAg
ICBzdHJ1Y3QgZG9tYWluICpkLA0KPiA+ICsgICAgc3RydWN0IHBhZ2VfaW5mbyAqcGcsDQo+ID4g
KyAgICB1bnNpZ25lZCBsb25nIG5yLA0KPiA+ICsgICAgdW5zaWduZWQgaW50IG1lbWZsYWdzKTsN
Cj4gPiArDQo+ID4gK2ludCBhc3NpZ25fcGFnZSgNCj4gPiAgICAgIHN0cnVjdCBwYWdlX2luZm8g
KnBnLA0KPiA+ICAgICAgdW5zaWduZWQgaW50IG9yZGVyLA0KPiA+ICsgICAgc3RydWN0IGRvbWFp
biAqZCwNCj4gPiAgICAgIHVuc2lnbmVkIGludCBtZW1mbGFncyk7DQo+IA0KPiAuLi4gdGhhdCB5
b3UgYWxzbyBuZWdsZWN0ZWQgdGhlIHJlcXVlc3QgdG8gaGFybW9uaXplIHRoZSBhcmd1bWVudCBv
cmRlciBvZg0KPiBib3RoIGZ1bmN0aW9ucy4gV2hhdCB3ZSB3YW50IChhbmQgd2hhdCBJIHRob3Vn
aHQgaGFzIGxvbmcgYmVlbiBhZ3JlZWQNCj4gdXBvbikgaXMgZS5nLg0KPiANCj4gaW50IGFzc2ln
bl9wYWdlcygNCj4gICAgIHN0cnVjdCBwYWdlX2luZm8gKnBnLA0KPiAgICAgdW5zaWduZWQgbG9u
ZyBuciwNCj4gICAgIHN0cnVjdCBkb21haW4gKmQsDQo+ICAgICB1bnNpZ25lZCBpbnQgbWVtZmxh
Z3MpOw0KPiANCj4gaW50IGFzc2lnbl9wYWdlKA0KPiAgICAgc3RydWN0IHBhZ2VfaW5mbyAqcGcs
DQo+ICAgICB1bnNpZ25lZCBpbnQgb3JkZXIsDQo+ICAgICBzdHJ1Y3QgZG9tYWluICpkLA0KPiAg
ICAgdW5zaWduZWQgaW50IG1lbWZsYWdzKTsNCj4gDQoNClNvcnJ5LCBteSBmYXVsdCwgSSd2ZSB3
cm9uZ2x5IGludGVycHJldGF0ZSBqdWxpZW4ncyBoYXJtb25pemUgcmVxdWVzdCB0d2ljZS4NCiAN
Cj4gSmFuDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:40:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:40:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182870.330693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGXE-0001iE-C7; Thu, 09 Sep 2021 09:39:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182870.330693; Thu, 09 Sep 2021 09:39:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGXE-0001i7-81; Thu, 09 Sep 2021 09:39:56 +0000
Received: by outflank-mailman (input) for mailman id 182870;
 Thu, 09 Sep 2021 09:39:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGXC-0001i1-V4
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:39:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7eb37e46-7309-4579-8b72-710457a4f8b5;
 Thu, 09 Sep 2021 09:39:54 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-v7YX5zgnOUe-OtfuP7aC1w-1;
 Thu, 09 Sep 2021 11:39:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 09:39:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:39:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0121.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 09:39:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7eb37e46-7309-4579-8b72-710457a4f8b5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631180393;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h0lILzb8aXfHStZ4bHwZau/9qfFE+vcjL8U5FhxolXw=;
	b=Kjj/kFK5MwfRQMeA07ITgU1FZsQSn5aiUfhDcOy/2roNSbOwGdu1+joX1O4mmpNPUg96VM
	2Kxd7OQmokDU0BnMAgXvJyr4/r2FDi8fmhAAUHZ456QPklWRP/RRWW4n9wUyBDB0DJMryn
	sM/Di4b+fZLGnZIQsnU7iR0G4L0m2CM=
X-MC-Unique: v7YX5zgnOUe-OtfuP7aC1w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HxgAtZGG/kx0hAhZqOlM4HtTS+TT80M4MJcPDiJw9s3OxZyQJf3OroZttd+kFUE/D8S5T4Bu1myu6LCOiUeUELJYtcajDDw5q6C5gLBZ6+tG2jbfIa9y/8sZBBj/gBeCrAQBuEkp6hVOboRbu5GQQy5RgYEcJJCBI6xG8jJzxntGPGItifgLjdCLy0S2c73GXeNQjZuYLqt2SZeN8f+nSToBGLKiRLKITCHUn9Qb2SoC7rPR3cdJFE5NNJR4ltC0kuyt/GCF6PSllyVMpAQ7YxuiaEf28BvFuh+SbkLgwLHinLMx6Svcvs9iHIkaXg3W8qIAS3AD/UFDEnMI58fujQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=h0lILzb8aXfHStZ4bHwZau/9qfFE+vcjL8U5FhxolXw=;
 b=HNA40bmZRC4aYw7wYFA0XU29uS11YbWcaiOXouEhrMAcH+THz4UFZ1C3T7r/rXNQkf8/JNOMWPsfjjLoXp6ssXB1clG2S2vmVkOGlmJ3XsglLg7fTudeqKiRjaxBGh2VmID5OUeCxtV7mg5/toJNeBf36wtPjRm9j7q7JM/9zZiNw47LMpK1LEGS2tNiXdTv+is6XEZ6oUGouOseXv39oWjmFfBUKSC9SqSwFIKojULlExRk8haLPZvFwJ+IaXoMCfd97kkj6cA/qnOBpEOFgfU9jAkNNIj+wqC6xo/4kmthvTsytQ5Z1cukQN++I7vmkT/rdGYtkefBOL6I/cxsuQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
 <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
Date: Thu, 9 Sep 2021 11:39:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0121.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 51f14924-af8d-407b-b70d-08d97375c486
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB73601CFA41DA3A686A012D1AB3D59@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nqwSU+/iLwq5rV0+42Yh5dq6UnKfmtzQgV/b/UUMaf/VHkPK1x3wLoi8IGeA8Retg1kWbrWxsOe9OwNhxsVPXGSBkz7qBPuob+ocOOp4w8qFn0A7B36Jy1yQUwqxFLUlmOUtpo7RtuJYMAq9N2eG1rgCcC4kHDBu6xDIydDMotn/e6Fnd+cHA5pjb4Ni3UYZ7H5igIvocjeSdkOxI6XiVsF6YltrV5MMTjTzAkJLuvHzneHrzvygsXrzSSmmPZ5reRp+lFHVqSd1Z6vZkYTAR68pNYIbeWj/eyiFmbmh/PRdtpJ733SiLkrUjr9dXcWZ2DkcBiBqkNSL4jBaW7oexS3dNdB3DqZN3CaD5tSyf2N7exQVsCCQ6uiwPk/9Q8snwxNr0ZuIkg4bhPsDd9tiUKKsGtryNA4YLs6U6awxGkkguk9H+dN+TW3AxzWoCm6vn6XQSPuJ1CSeZUs1ZzucGGEQ9OXP3m3nE2ULE0G0jxBYOHrLksPw8pQQzW/v1xAmPbWqhPma+o4iqGEoNc7jzBunUxMQgfCcm5QZ37UAmOI2GuIlfCEO9xdMV0FfcatHovpOoytnd2uJJtOTNAMnvOxh3aSO+c3H+WSpSFn1EVK1hTht3htRH2HLo2DqrLOcS5oK+YaLadERgl0BphUcMUWzkK1i5BWgb73KK4leQ84oNBBoWCBMff3DkoSKGOULE/zJEFHIt/MNE0Pv3vg6UK812oSnNR7XXizbgbMRWrqHNRpn5RN9fkLNuHDZGHdV
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(396003)(136003)(346002)(366004)(53546011)(8936002)(110136005)(316002)(16576012)(956004)(54906003)(31696002)(83380400001)(5660300002)(2906002)(6486002)(31686004)(66556008)(2616005)(66946007)(36756003)(186003)(4326008)(26005)(8676002)(38100700002)(86362001)(478600001)(66476007)(7416002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R0F6cFB0WmJycG9LcFFXRlJYOVRkQzBOT3hsdkJPL28zNTRON1JnOS9PSXI5?=
 =?utf-8?B?eHhPaUlyU1ZIakVpU2NROW9TbHh4U0c3Y1Q2NmlhdkNvN3dIRXFOMDZvcWdL?=
 =?utf-8?B?RnhaZlVXU0w4ZGRNR1VOaVFHby9TMHVxbWZOMzUvVUZ4QVRJSTNDZm1oU2VG?=
 =?utf-8?B?dXhLS1BoZ29lb01vbmpuODZYNkQ0eVFOT3M0T1lqMnQ1ZDRCd3Jjb1VmM1RW?=
 =?utf-8?B?YmlzK2Q5MDZpM1BzTHdtei83cGJ3VWhmenJMNkNkbi9NNHJiSzRlM2tJOURW?=
 =?utf-8?B?RDRiazczNzJtVkw1MDBKM1oxMWtBdE53bTBFa3AvVkl3a0MxaTBsU1dBd3BZ?=
 =?utf-8?B?SVQxdFp3MUc0eU9DMnJBalBaYmtEQVBFZGpsZFBHb0NUMUdpdjV4dWJnekUy?=
 =?utf-8?B?Z1hPVEZSeTNTZ1FLejJNanJmcnNkb3BwNVcwbHRJRU9ra2FmM3ZCME51QW5s?=
 =?utf-8?B?bW1vS2toaUxzVUwyblNaQW1SaVRlTGxlTmpad2VUL0FrSWNESHp2dDlwUmRR?=
 =?utf-8?B?cTI0SlI5bG5IckpLYitmVk91UnROMFhVNkh5UXJ3REdnNDlWR1NoYkI3VHJM?=
 =?utf-8?B?NmlmbUpmajI4dnBwLzZIdDhOejJ2WTZmbWtBUk1vY21nZVBOL2FEMjNwTkVT?=
 =?utf-8?B?TEZ3dWs3SGI2R2wvdWRYMEN6aFRaQlJUQnRvN3NEemhxSXpSOUVHZTJoVFZ6?=
 =?utf-8?B?U1hScnVCeG5SR2NkREs1cEY4YXBQNmJ0dW9wS0J6OUdMUVJGRFJUQTVLc0JU?=
 =?utf-8?B?QS9RWVVlTVdWTzBuamtUajUzYlhUQXJEOHlIWHVpcDUzTG5ZZDA3cUV4dFh4?=
 =?utf-8?B?ZFlrVkI0bjEzalN4bUR1ZThPczRoRTRQb0FnekoxSVNlRnYzRk5Sc1l4TFls?=
 =?utf-8?B?MFZQZ0ovaTBKdWJ1WDk2UzNKYlk3Ung3V1gzN1J5QU1qYUg2Unk3UmMxUWgx?=
 =?utf-8?B?NGtBYTFCUmtEYkJjbUlDMVZjT0lPZ0sxMHk0U3N2WDdSTzhtVGtkZTZST1Uz?=
 =?utf-8?B?cGFmamY0ZGRzUHBEYy9OVHk5a3dWRHN2UCt0WlJvZTN3azlVYmU3c3hZM2lF?=
 =?utf-8?B?MEtSR0pyMVpKbHlGQnhyMkNVcUN4SlU0aUFYMS9MOXNnZ3Q0blo5cWlkelJM?=
 =?utf-8?B?MkhPVTQyN0M5eENRbzFaaHVRK1hoSGM1L3ZJVmtDN3VmQW14akVmMFZWTzlP?=
 =?utf-8?B?R3hkVktTZm1aTGtaVFlqWEorTnl4d1Z3cEc4SU1ZRThNT2M2dFhrMko4SzE4?=
 =?utf-8?B?UGpIc2hwVnRNUzVQT3BwRnMvTk1xajFEL3p0SWxXTnZ0QzFjczgza1hZdC9O?=
 =?utf-8?B?eXU4bi9zcU5nWDdlWmRhcUttaUdBa2U2dUFFSXhEQW1SNU9kYjJjNG9PUlRn?=
 =?utf-8?B?TWsxb2gxTWtPb1pjLzhrNlJXMkNyajhPYi83NWNXb1hWSlA1a09HVWt2amxQ?=
 =?utf-8?B?SW5idkxpd2cyYk14Rzh3Wm1PdDlzS2xlbVk2Z1pYVW9SMkFCSFo5TVBqdjJJ?=
 =?utf-8?B?STJnN3RzSGY4dFJmMVBGMEVaZ0ZkVE42S3ZxTnpkeU5KWmo1WXd1eVROcW9r?=
 =?utf-8?B?cmsxdGpSWHprT1hHRVIyR2huN1VhNE5jQUFLZlpTMEZFcThDY3dvdDN4WThj?=
 =?utf-8?B?dkZtTnk2YVZSQjlYZEI4dUovRjBUbDU4WjFhanJ5YVJ2aFpIYXd0b1NMSWp5?=
 =?utf-8?B?MkswQURCNGZJZnJLdUxNTFJVbE4weXNhSU1QNnV2NHFxcGpEM1lUUFpGcEV3?=
 =?utf-8?Q?vXf/GAXwVHSnJRVlSery4ZmQQds7Mx+bKp6ElXj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 51f14924-af8d-407b-b70d-08d97375c486
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:39:49.6296
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0Ww/na8YrPqu/jpD8M7znn+16jAK8mKqrtFaOsCGmDCCiPpp83ymXZfloLdl3nDX8RYc3Zx9Q8PY1lmsDepbTw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 09.09.2021 11:12, Oleksandr Andrushchenko wrote:
> Anyways, I am open to any decision on what would be the right approach here:
> 
> 1. Use range sets per BAR as in the patch
> 
> 2. Remove range sets completely and have a per-vCPU list with mapping
> 
> data as I described above
> 
> 3. Anything else?

A decision first requires a proposal. I think 3 is the way to investigate
first: Rather than starting from the code we currently have, start from
what you need for DomU-s to work. If there's enough overlap with how we
handle Dom0, code can be shared. If things are sufficiently different,
separate code paths are likely better. As said - to me a guest altering a
BAR is merely a very special form of a request to change its P2M. The M
parts remains unchanged (which is the major difference from Dom0), while
the P part changes. As long as you can assume no two BARs to share a page,
this would appear to suggest that it's simply a P2M operation plus book
keeping at the vPCI layer. Completely different from Dom0 handling.

All of this applies only with memory decoding enabled, I expect.
Disabling memory decoding on a device ought to be a simple "unmap all
BARs", while enabling is "map all BARs". Which again is, due to the
assumed lack of sharing of pages, much simpler than on Dom0: You only
need to subtract the MSI-X table range(s) (if any, and perhaps not
necessary when unmapping, as there's nothing wrong to unmap a P2M slot
which wasn't mapped); this may not even require any rangeset at all to
represent.

And in fact I wonder whether for DomU-s you want to support BAR changes
in the first place while memory decoding is enabled. Depends much on
how quirky the guest OSes are that ought to run on top.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:41:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:41:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182876.330704 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGYM-0003IY-NR; Thu, 09 Sep 2021 09:41:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182876.330704; Thu, 09 Sep 2021 09:41:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGYM-0003IR-Ir; Thu, 09 Sep 2021 09:41:06 +0000
Received: by outflank-mailman (input) for mailman id 182876;
 Thu, 09 Sep 2021 09:41:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGYL-0003BU-8F
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:41:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f7cdabcb-308f-4224-8e25-b303ff245fa9;
 Thu, 09 Sep 2021 09:41:04 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-8-ey27L9XlOky01RcoT1pE-Q-1; Thu, 09 Sep 2021 11:41:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 09:40:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:40:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0902CA0018.eurprd09.prod.outlook.com (2603:10a6:200:9b::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Thu, 9 Sep 2021 09:40:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7cdabcb-308f-4224-8e25-b303ff245fa9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631180463;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wlMga0mUYNhs7SvP6Ti5KthRY5hVV+M0UFQXOq92hf8=;
	b=d/lxTS9Hte5yDpXafaruUWGd9A6wMYLHZvwPT+A/0Vc/2PtS3kvyG14IwX6wpxdBtwkTsC
	coLH5Nm2hVI7yJ15ajCRiCinkJ1fDYYh9SvlBY1uu5/t+WEQtfryWyi0Mfnnhe2ZsIOXFy
	Ujk1IxQUC2Anx934NnRhziCwf/VXbz4=
X-MC-Unique: ey27L9XlOky01RcoT1pE-Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=INuQQql2Y59ieJUv3ntsMtni+/+HABtKMYu+GLZKXhNZOX9O2XmTnE0CThsOBu+FTc4S50P4HVoiyA8xRswRgc3KFaBMg5K5vfGOcMoe6Jv84uuTWvSSRr1Z5fdWjIpZN9zLt73/EvAWbMxO154bZqexW1QiRrmHO1liTOGfMFX2JbNW5pcZ+LGyvJPUl2F0qV94wwEp4Ng7+AdEX2UonFJgtFeUosgORugkodRDEc07iP1zvLjW62MT4zXt43PnhQhgSVhVU2k34tLv2JAbnPGJiGrWk6woUouoeNHETvqEcUSXhYTWggbeZuB3bT5q7jgd00Vw6+NTowLeyPwrYg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wlMga0mUYNhs7SvP6Ti5KthRY5hVV+M0UFQXOq92hf8=;
 b=f3pbn/1XUwxZecJSNIR4fL/9LJqlt2rJHF+QkI0F8JcpMGhxT+GtPQYN+eg+21raeo9AsdCqpjgK9eLLJC4UAB2K4K2XXA8GQasTNWynjQjAyeyT6UiZepyVMRg4S7FvgmYdd3yvHvw65xoonvvjs3hd3QUF8v7EmkSJVFPZlYd3co3+NdxFKErRFnS/rd6ZkxE3DTvUWclqJy6Qz3fcGX3jDdzIOYomx9qRo8vaXe8LapeWPPoWvxbeNbfUadt6H7CaOvovxRwL1rKTADdgX1ki+S8tW+qrvHMVV104EBOh98WE8vu80jsBNR2MjX5OXCYzzjwCdbWULNsFqp4YuA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
 <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
 <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
 <ee9a69f1-6b47-a944-7255-4ffaa3e1c601@suse.com>
 <b9fe88da-3bdb-0b37-f2f4-9a575806e6de@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2cffae86-1a2d-a9c1-e5f1-1083ed24480f@suse.com>
Date: Thu, 9 Sep 2021 11:40:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b9fe88da-3bdb-0b37-f2f4-9a575806e6de@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0902CA0018.eurprd09.prod.outlook.com
 (2603:10a6:200:9b::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bf5e3832-995f-43bf-0eb1-08d97375ee54
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71524A9A256C8CC4835FFCD8B3D59@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	te3/KmN7Fn8GrbUYGtIvUzBp4fdkc9IXk22IBejbbH7tND0ejvLydY65SPiV2H73m9HX6HTHttEuT8/s4WjJgqcOUBTD3ZseBo53dHtDbUUhpoOnMpr/eTRlYm1kvTeJSEyDCUn9CZLeI+ZFkWyBDyUjyOyIscYvZoZKCecXRpo9jB373nvJizWJaUEYPzQtz2M6yXRBcqMpOizPtTFt8plxAmNYdAwaVfr4aI331kdLYHIQs6E8OkZRE5jcfB80fMaCtveo6dPgQSLZOVWRu2cNMAUs2cPkBzs7O6DdNjyL8I0L54VdltbZf5zkp8HwuYM9AS8vIG8s8WZJDEdwkZTmFJZObHijmXjTYzZnCNK3puBbz63ExRio+G1UEOL3sVGcgDS/uMFeCDBjJ0z1VWA1x9aRLFBXcILDenWRZ7Y1U62WglR+a2hg8VPcFF1qMN095IaeINfJHVTLtw9Atlo6Vx70QGa7D8r2SQ9fp2S7/ZeGsGe3A5e1Q1HANoWNbSIHvnSC/LqHZrIFL+hf7x/tlCbXrvVDIeWL/X1eeXF+uvcDJdlXIJnvWIrKHp2zHvh5U/FpULVNPt/wcCQTZm7sfZccNoAXc8P85zfB25v1ndOR85UrIcLDfSnFy3ZOFjIB1COuzAeLBTCIAawbO8wGp0IYo4/rgdTAUhobJLfo4+kAcLrLMFIuDVMVi6PaVEnGO48EDGdiSo07LwCXNVbqKfvaUdVxnZ1ZW3JOXWk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(136003)(346002)(366004)(39860400002)(31686004)(956004)(38100700002)(2616005)(83380400001)(4326008)(66946007)(36756003)(6666004)(86362001)(186003)(2906002)(8936002)(478600001)(16576012)(8676002)(31696002)(110136005)(66556008)(66476007)(316002)(5660300002)(6486002)(7416002)(54906003)(53546011)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ckxCdUMzaXZ1Qk9VdWc1MDZXYU54Z3ljRVcyZVQrYk1uTy9DQS8wQUVUZ2tl?=
 =?utf-8?B?R3gzRUlReFlweHVnWXRmMWt4U3JkeXBhVFg3UEpNMDVEbUdHL1d2VlRwZDNa?=
 =?utf-8?B?Q3Z2d3l4NnJvWVJvUDZuY3pRenlWRjN1VUtYOU1OWTBCVVd0S1dQbFZZM1c2?=
 =?utf-8?B?NkVkZWZBZ0NJUGt3MXBzSW5wWGl4eTFPRVRjNVNlTWZqVldJTTF2Q2puZmtu?=
 =?utf-8?B?ajcwK2dabmJubVdDY3VNVWpIUkpSN0J4OVMwMDczM1dIQUQ5TzBhZnk1Nkl2?=
 =?utf-8?B?dEpBcnIwUkNCL0M1V3VTYTllRzZiVXBOQWpkNGo2dnlhQnJpbk41QTFPQm81?=
 =?utf-8?B?dGdTRlBNVnhMdTJNTUtpQ2tIdEhJZ25BNjdZc0ZmTW1IYXlKQnRSMWhpR2Jr?=
 =?utf-8?B?bzVXbFBqY3ZJOEQ4dUlhSmdhbGs4TDZMSjVTSWIzOU5EdStGVmNmdmpWUnRl?=
 =?utf-8?B?TTU1aTREd1ZQMEVpOXhGSnZiUXlqcWJmWDRSWjBwM1RaRkNhT0huUENHK093?=
 =?utf-8?B?eUJGTis4bXZyUEpXTVUvbVlTa3ROdU8yQWdxRnk2NGFFRFRDclhtaFR3SE50?=
 =?utf-8?B?N0Zsb0NzYkxoZlhtd1hDRXFEc2FLNi91aExRaS9yZURuS0owR3A4Rm9kT3BL?=
 =?utf-8?B?cFlPTngxVFQ1L1JCYk8vZW1pdWlHdi9xRGRBUmhwVk5CcnlzVWZBOHBkWGlR?=
 =?utf-8?B?NTJEQmErcndiRWJ6c05zZ09pTG5zUXlOSCs4WVVSelZUeDN3a2lGZnZib0x5?=
 =?utf-8?B?T0wwb2MyalBRcTJBbHNqSVBBSjNqcEV0ZlBJR3dqVkUybjRLT2tKKzcxSEVv?=
 =?utf-8?B?cVRPa1EvaDZqdXAydDhLSnovbTh1QW9pL0tZNXMrQUNlOGx1RnQ3SjR4c3JN?=
 =?utf-8?B?bDlCOFd1eHF1Nk1XSDRnblBsVU5XcStYYVFyUUZQNkJJTTJWRGFKV2FLT2dM?=
 =?utf-8?B?MllSUVB1KzQ2SnFVOUhyWFdGQ3RFUXUzcUFDR1Y4Qkw5TW5LOHNwUVYyNFVj?=
 =?utf-8?B?VmE4YW1mYklnbFMvaGxFNG9FYlp6Wnc5dmUxdUdzcjdja1FIUTREQks5Nm9x?=
 =?utf-8?B?KzhWdGlQeWVJV1hXVXdKcmgzWU44VEF2Q001VEc3eXNvWU5JRzN3YzkyL29E?=
 =?utf-8?B?azJnanJiMWlFaVRFWGcxdjhGbHRGZEtmOGl4QWx0ZzhYbzY2UjNxK01jZmNN?=
 =?utf-8?B?ZHFyZmtWb0E1UjR4UERIaHNRU1Fydi8rNHBSNnNwSTkyYmtiNlhOdGlYbDcy?=
 =?utf-8?B?R1NsTU95WWhMOUJpVE5EYW9NN2ZyQUY3elJYOEY0REVQaXUrN2NSWjR4M3h4?=
 =?utf-8?B?MFQ1U1BzWFJNd20wblgrU3ZsRTlldTBPN3EwVVo5REFDeDNIaG9PWG5UZi9F?=
 =?utf-8?B?enZmcXBpeExCalVadWluZGozQWx0LzFZb1Awd3VBUmljTkJvSGVMRkRjLytD?=
 =?utf-8?B?WEJmLzJpa3dXUXBydHRZRUtKTmZjQkRPZWlZZ2hqeVhiWVJSY3dIck1PcnFX?=
 =?utf-8?B?RWhJVzMrOEpTTlcvbDIzU0laVGNKQ1JFazdreVNlbFNuTWRyZzJZcnVRMytv?=
 =?utf-8?B?Z0xrRHhEaUllK3FZK1A3UXhnZ2M4bkFVSjFZd3ZDUkdnaVlkZFhRdVJwNlha?=
 =?utf-8?B?clJ5cjZUVjExOGR4VzF0QndEVSsxd0hJNUx5UE1kL3JBZjlXTnpSMXI4SUxq?=
 =?utf-8?B?VWZWdUhaSnNNbVpqdXl4QU4yWWNoUGlhOUlJZittbTUrc1NRejVVSGpOMTJD?=
 =?utf-8?Q?F4wfSHmxe+9bVtltV0pBv+a46uaJ2Ki3yMBDDyh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf5e3832-995f-43bf-0eb1-08d97375ee54
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:40:59.7609
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RPWuVR/4LhLGfYZdRJJhJGDFgUmXu3IX191SCUX0Nu9W8yqOstqgrv1U/EVJ5CkvDdu782bokeBUPv8Fp+J03g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

On 09.09.2021 11:16, Oleksandr Andrushchenko wrote:
> 
> On 09.09.21 11:26, Jan Beulich wrote:
>> On 09.09.2021 08:13, Oleksandr Andrushchenko wrote:
>>> On 06.09.21 17:51, Jan Beulich wrote:
>>>> On 03.09.2021 12:08, Oleksandr Andrushchenko wrote:
>>>>> @@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>>>>>                         unsigned long *c)
>>>>>    {
>>>>>        const struct map_data *map = data;
>>>>> +    gfn_t start_gfn;
>>>>>        int rc;
>>>>>    
>>>>>        for ( ; ; )
>>>>>        {
>>>>>            unsigned long size = e - s + 1;
>>>>>    
>>>>> +        /*
>>>>> +         * Any BAR may have holes in its memory we want to map, e.g.
>>>>> +         * we don't want to map MSI regions which may be a part of that BAR,
>>>>> +         * e.g. when a single BAR is used for both MMIO and MSI.
>>>>> +         * In this case MSI regions are subtracted from the mapping, but
>>>>> +         * map->start_gfn still points to the very beginning of the BAR.
>>>>> +         * So if there is a hole present then we need to adjust start_gfn
>>>>> +         * to reflect the fact of that substraction.
>>>>> +         */
>>>>> +        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
>>>> I may be missing something, but don't you need to adjust "size" then
>>>> as well?
>>> No, as range sets get consumed we have e and s updated accordingly,
>>> so each time size represents the right value.
>> It feels like something's wrong with the rangeset construction then:
>> Either it represents _all_ holes (including degenerate ones at the
>> start of end of a range), or none of them.
> 
> The resulting range set only has the MMIOs in it. While constructing the range set
> we cut off MSI-X out of it (make holes). But finally it only has the ranges that we
> need to map/unmap.

And then why is there a need to adjust start_gfn?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:43:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:43:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182883.330715 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGaf-0003xz-46; Thu, 09 Sep 2021 09:43:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182883.330715; Thu, 09 Sep 2021 09:43:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGaf-0003xs-0E; Thu, 09 Sep 2021 09:43:29 +0000
Received: by outflank-mailman (input) for mailman id 182883;
 Thu, 09 Sep 2021 09:43:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGad-0003xk-ML
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:43:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 60feb706-1152-11ec-b1a8-12813bfff9fa;
 Thu, 09 Sep 2021 09:43:26 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-h02rg7DRPUykDqzmikeFpg-1; Thu, 09 Sep 2021 11:43:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 09:43:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:43:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0252.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 09:43:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60feb706-1152-11ec-b1a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631180605;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gRoBOh5Wipkr5PeXQUpNw1uYM4KDM+r1pHZzMrWbHpM=;
	b=QiwTSptg4mxdGQEJCyLekJREYzeHhWCacKzOB7dKpV9kwp12d7S7Q15INNvN8sH2YSntQ8
	83lw7ued/YhgqKtaV3h9dBZZ21SkXt2sDYNy7G4d8Km886BEcBgwa4NxcWwE0KTD6Xnw0e
	/V6QW414BZfjiAW8vWINzqGcK1uE0bI=
X-MC-Unique: h02rg7DRPUykDqzmikeFpg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hMKAQ0BTtoKXz+aTlCfWxHI1n2snpWt0BJlFyF20tiNjVYHRVuH4tFtxkFOzDDn65RL/FrP14WRd1UBm5EgcnHCKfe74joaI/sX4g8gFbzLCELkMXF+K0E7JOwx+xOCoDPLy4HJc9SFM00HD+26dAlsJ3Y/kKfhArVLb6DjJ0qblAy0WVnb1lPjOoTtStumttq5wePt7l5Mf/YMqSgeIGeI2I9JfspeW095kgEcnW25QK7L+clp53lap7WKq65zItJc+gIIB6UE3umQiZDYLdoJmYYsLPUB/Qcxj/9cbWZChOxb5AO987wpigTOl8AGhwIcS8+g1sU/eZYZC77c73w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gRoBOh5Wipkr5PeXQUpNw1uYM4KDM+r1pHZzMrWbHpM=;
 b=EKX+8uDAjpMl129VxONrAzdxVjPikJXgPNTdzwj3SZPmFLm47u/e9JRH8uKF6CEJDvbS43vpdJ7opfxfK7jeRb1ZNtL3Pbwy312BqO514w41zVJA6C6m0DjEloR3outX72jlkf33RMUYgIucUFm+9nLRjyPTUoTkn+0cK3578JkqwkxYnwOb3mQH5WMc8JK8DvHz4/rMT4Lj30zRZ7vVSXnyV8lXtjJdCFgnHt8ZFHCHcG9iZzKkHJTQeoswj5OJ8axzpanoeEDz+K9HMo0StaRgwe7E/i/bDPEb2at2ftkNiDnXjGUA0PgzXf0u5cnA30fsigKqVXLnJ6TgW2fFbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/spec-ctrl: Print all AMD speculative hints/features
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908173352.3627-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <cf3c8ca0-7df1-0f30-ca6b-aba524bb80d4@suse.com>
Date: Thu, 9 Sep 2021 11:43:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210908173352.3627-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0252.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::24)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a976689a-0b59-4d33-a987-08d9737643d7
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68630BCAEB2E1D76CDC82E78B3D59@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:586;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WzSy3PO8YyWW0+g5Q87O7IbWiza6wUTk/Vln+R5Prol9Uc3grVlNziQ3WbU5ZRBPt+nJUWuxqkrFuoklk13+RXmB1TGzWajlCbU/GYpJVPiwxM+EVul4b1QQg6D0CEsOp6wQqtHSZ6DgEq/4nqpGma+9hW5BRaxX4PT6xpqORTRUDsIJc9jXAMRLzKzWtbwSzgNRpdlNAC8rWRT7tgLfYx8H8uPw9XElJnj9LH5rbWpEpN7gHgh9PPDFkTF/aE7/ZK9XrRbz4DAaasX/Ub6jXWERo6TP0Yk/g63qwzhMc8GpE0LzXjpQicMnnio2QrK8Lzw7lhTbU1Q6JSCMe9pksOffIPV5T938zw2xHiSbC/xCtka1o6cvdrZ4g4VWm7TW//a97Mt+PMW1UawszwUqbosZI3+SxfeTj48qHBrZm+dRIxmORYpTOGYSAJn7cCn0OREC8iAY2bsaEnRR7jxSeZVWg+zFCcMFdVrnkmtjUGcTdlNzL19RcXTSRHo33IF9C4aQRmOmjTNtbyFA0dmkT6baE6Lmhg69aKVXZi0iFWl1XTqGmrFHFMpZAFuXoPWF7GOvheFP8UXL/uGtjCwAItNFdtSlY2V5h0a3zrTrEwzpEkWewiHYLd130YOqWiXLT96bYET0LS3YXBdSRySUC30ltWAuqbynVXPmwgCDycEYMiqXwzkGvoSttAtjbaC9piGTntQjK2kC5S4x9i8IAOIY6y7S3gbsXDjgDNz8fH4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(136003)(396003)(376002)(6916009)(186003)(5660300002)(478600001)(316002)(8936002)(54906003)(8676002)(2906002)(66556008)(36756003)(16576012)(31686004)(38100700002)(2616005)(6486002)(53546011)(86362001)(66946007)(31696002)(26005)(558084003)(4326008)(66476007)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dDh3YVNpS2czamE1VG1qeVJnb2Y5a2JjTnp0aG54aEprV3k5NDNWMWFGUmha?=
 =?utf-8?B?Q2xPRU5pL0szYTUva1JWeFB3Qm1aNk1CZHRSMjVYeGxFdi9iTHFzTTBmTG1i?=
 =?utf-8?B?QitqZXplbkppMFNBZk1pSEdWQXh5Wm5YUW4yaVp1aThxSE1TWHd2VEtPVFBY?=
 =?utf-8?B?dHpYN2hodFlGc0hsbTljN1IzSERFK3plYjBtSWhNN3EyZ0ZPd2UydDFMRzdX?=
 =?utf-8?B?endsZFZzU0s5UkxqUWt5UTdqYkszNUp1aDNMbWNoVUdMaUVqZlgwWFdVZ0dz?=
 =?utf-8?B?NUo3NGJxcU5RbXpLTGFDU0w0cSs3MFJHRXJwWitob2FTcEJYY2p4MFhhTlRX?=
 =?utf-8?B?NWNmYU5KY0NDWWg2ZWpGdHFmaytPakdrMVdleE94NWFYTTlSRTZpOW9FM0xt?=
 =?utf-8?B?YkZDUmdTKzJJTGtxdFNsSnZZWjdoRnVWS0hOOXZzSVJ0TE1QMUtDeUpGZC9T?=
 =?utf-8?B?TUx2S3gySkpQekJ5OFUxS1FZQldGYTd6SFR5VmRnMUpzbXQvUjlKVEx3QUlF?=
 =?utf-8?B?YlNJRHhLSCtETnNPSHZWeXBiTks1aTlNdS81NWdNeGhheG1Tbjc4Y21PQzh6?=
 =?utf-8?B?MitJSnAwTGxjNDhkeDVuRitDUDlsNGtvOWhYdTN1eEFncVZXcXU5d3N1U0V6?=
 =?utf-8?B?aGc1MUsranBqSVc3NGdJaUwxZzRlOWVCUUJUVm5KaVZGZDR5aWd3R2wyMGR1?=
 =?utf-8?B?cU1rSXFrMllOZzVrZi9VNkZwYUJvTGYwMlBycEd1SmZxbjNVVmNSazlQUGpP?=
 =?utf-8?B?b1lqN1FkdmRteVMrT1llWVlTNE00OGxDZktGdHFmVTd5Zk9uTUh5by9TOUZ6?=
 =?utf-8?B?ZzMvR2RxeCtka3JXQTY1SVI1ODQwZFNRSnNhV1pHK3lsZEcwOTJBaU9LZk44?=
 =?utf-8?B?OXZCWUlUWTNrTkJUdEhTNklqeXY1dGZXQ2lyQTBSTFFVOEVNQWppRzhleFFG?=
 =?utf-8?B?MkVqR0RMOWhySFVjVEQzUTl1a3hyd2FGcThFbHBuSWtKWC9FN3pJUjlaVWg4?=
 =?utf-8?B?SkRZdXk0SHlMVnBpSTBtU2dUNW9Ja016TUZXSDB4aFNDeG1SWDJhbWJGenAv?=
 =?utf-8?B?aEYvOWNZQ3IzWXJxZmJoWjBiVS93MW9wRjh6dDJ4ZkZRSU5RMlY2ZVRCUkRT?=
 =?utf-8?B?OVNRNEJGc3JrcHR6cjhxSU9uOTFhK3hrNEJQUE1RM1FuT1YvdjEycFl2RFFa?=
 =?utf-8?B?NEEveEpSTHE2aE9UakFCWGUwSFIvR1NPdjE3cGQ4d3grRnZUaml4SWRNTUVF?=
 =?utf-8?B?SVVRMFl0dWdHQ29sZzRlcHc4N3JJaFlVbGpKM2lzYnV6dWpGY0p2QzdQVVU0?=
 =?utf-8?B?aCt6azVTYlgybmNSMXhMb3dpVHduWGlBRXdUN1JDRW83NmkyeGcxLzJFN0JM?=
 =?utf-8?B?NVFMZ1FHcStsb0xlZXFYRUhkOEhpQXdTVDJSRkphNklPcEFGSjhMR3FlVERi?=
 =?utf-8?B?cnZnTU1NV1cyUGJoVmkrazR3cUROdzJTKzFuWmJaSmdJMkprUGV3MUFxNm1n?=
 =?utf-8?B?OGM5eWk1WC80TDBRbU9HbTVqejhRZ3BBU3QzZ21yb3FjUHNTOEd1UFBoc2kr?=
 =?utf-8?B?TDFtV0tGOFZkcWQwU0tCMVJTNGF0SjJlaENBTnVhK0pyUkpXQVVFbVg5bGhT?=
 =?utf-8?B?VENBRERqeHlzaStaMzhncHR4M0Evd3pPenl2NFBpUjBmQ202WjhZWUhnOExy?=
 =?utf-8?B?a2ZrY1dkQjkzT1ZyQUg3dll2UnowQmh4aEJKNmZmR1ZqdDZxeDQybUxmV3I1?=
 =?utf-8?Q?KG60Yx9+/jZa4zz3Grl8zWM+RAO70I1YIvznauR?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a976689a-0b59-4d33-a987-08d9737643d7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:43:23.2567
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7NNmr0WUCBkbDVOtBiZCftfcZEM73NjTekiSvIOSbkU60zNOyE0nZ1R3ZydsWRGO9eOe579m4AKxEmes2RzpTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 08.09.2021 19:33, Andrew Cooper wrote:
> We already print Intel features that aren't yet implemented/used, so be
> consistent on AMD too.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:53:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:53:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182892.330725 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGje-0005kP-4x; Thu, 09 Sep 2021 09:52:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182892.330725; Thu, 09 Sep 2021 09:52:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGje-0005kI-20; Thu, 09 Sep 2021 09:52:46 +0000
Received: by outflank-mailman (input) for mailman id 182892;
 Thu, 09 Sep 2021 09:52:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOGjd-0005k8-Hl; Thu, 09 Sep 2021 09:52:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOGjd-0002c7-9Z; Thu, 09 Sep 2021 09:52:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOGjd-0005dh-1M; Thu, 09 Sep 2021 09:52:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOGjd-0004AY-0t; Thu, 09 Sep 2021 09:52:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Message-Id:Subject:To;
	bh=C4zkEz3RFyY9bxBQvuSkHMgx8nzPx/dihSlfDUtQMKU=; b=u+aex7BWkAcVkcF3S8p/RIB3Mq
	OXg0isxCFfoS982KKL7aOMToMivNqkWHF+Z8pEYiSTOBn21Uf1LMoTQL77uQz38q9o8a2yaQk8uUV
	EUKw6QSRVVpgmptM4XIakub/g+FlE+333uk0OA0I/zdCplhp+CKPmSJL8nLfKcM868BI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Subject: [xen-4.13-testing bisection] complete build-amd64
Message-Id: <E1mOGjd-0004AY-0t@osstest.test-lab.xenproject.org>
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 09:52:45 +0000

branch xen-4.13-testing
xenbranch xen-4.13-testing
job build-amd64
testid xen-build

Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf git://xenbits.xen.org/osstest/ovmf.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164910/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-4.13-testing/build-amd64.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/xen-4.13-testing/build-amd64.xen-build --summary-out=tmp/164910.bisection-summary --basis-template=163761 --blessings=real,real-bisect,real-retry xen-4.13-testing build-amd64 xen-build
Searching for failure / basis pass:
 164887 fail [host=himrod1] / 164644 ok.
Failure / basis pass flights: 164887 / 164644
(tree with no url: minios)
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
Basis pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
Generating revisions with ./adhoc-revtuple-generator  git://xenbits.xen.org/osstest/ovmf.git#a82bad9730178a1e3a67c9bfc83412b87a8ad734-a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0 git://xenbits.xen.org/qemu-xen-traditional.git#d0d8ad39ecb51cd7497cd524484fe09f50876798-d0d8ad39ecb51cd7497cd524484fe09f50876798 git://xenbits.xen.org/qemu-xen.git#7269466a5b0c0e89b36dc9a7db0554ae404aa230-7269466a5b0c0e89b36dc9a7db0554ae404aa230 git://xenbits.xen.org/osstest/seabios.git#54082c81d96028ba8c76fbe6784085cf1df76\
 b20-54082c81d96028ba8c76fbe6784085cf1df76b20 git://xenbits.xen.org/xen.git#dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b-9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
Loaded 10001 nodes in revision graph
Searching for test results:
 164555 pass irrelevant
 164560 [host=himrod2]
 164565 [host=himrod2]
 164568 pass irrelevant
 164572 pass irrelevant
 164579 [host=himrod2]
 164583 [host=himrod2]
 164587 [host=himrod2]
 164589 [host=himrod2]
 164594 [host=himrod2]
 164577 [host=himrod2]
 164638 [host=himrod2]
 164640 [host=himrod2]
 164642 [host=himrod2]
 164643 [host=himrod2]
 164645 pass irrelevant
 164648 pass irrelevant
 164651 [host=himrod2]
 164652 [host=himrod2]
 164656 [host=himrod2]
 164644 pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164887 fail a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
 164899 pass a82bad9730178a1e3a67c9bfc83412b87a8ad734 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164900 fail a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 9c4b19c110e1410e50a9f1dbd15d337b05e9cc9d
 164901 fail f0fe55bca4651734abf1752a1d7c69fb5bee00b9 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164902 pass dc995ce9069b683d55fdb8d51235cb5920db8041 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164903 pass cb0d24637dfdd869618b9635dfb8e3b0746393a6 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164904 pass 52e2dabc0f8d3af09c213072ce8ba734302f585d d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164905 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164906 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164907 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164908 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164909 pass 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
 164910 fail 9f3eda177a4b2d4a33ff1b0307cad42906396562 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
Searching for interesting versions
 Result found: flight 164644 (pass), for basis pass
 For basis failure, parent search stopping at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b, results HASH(0x5583659b4730) HASH(0x55836599d160) HASH(0x558364fd97c8) For basis failure, parent search stopping at 52e2dabc0f8d3af09c213072ce8ba734302f585d d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d\
 96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b, results HASH(0x5583659b62b8) For basis failure, parent search stopping at cb0d24637dfdd869618b9635dfb8e3b0746393a6 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b, results HASH(0x5583659b1380) For basis failure, parent search stopping at dc995ce9069b683d55fdb8d51235cb5920db8041 d0d8ad39ecb51cd7497cd524484fe09\
 f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b, results HASH(0x5583659b8ee8) For basis failure, parent search stopping at a82bad9730178a1e3a67c9bfc83412b87a8ad734 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b, results HASH(0x5583659a54a8) HASH(0x5583659b00f8) Result found: flight 164887 (fail), \
 for basis failure (at ancestor ~5005)
 Repro found: flight 164899 (pass), for basis pass
 Repro found: flight 164900 (fail), for basis failure
 0 revisions at 5b5f10d7465004e3e40ec1f50a3b490b4db595e7 d0d8ad39ecb51cd7497cd524484fe09f50876798 7269466a5b0c0e89b36dc9a7db0554ae404aa230 54082c81d96028ba8c76fbe6784085cf1df76b20 dd29f4f4961d5c99660874c7ff090bd3c2ef6e5b
No revisions left to test, checking graph state.
 Result found: flight 164905 (pass), for last pass
 Result found: flight 164906 (fail), for first failure
 Repro found: flight 164907 (pass), for last pass
 Repro found: flight 164908 (fail), for first failure
 Repro found: flight 164909 (pass), for last pass
 Repro found: flight 164910 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  ovmf git://xenbits.xen.org/osstest/ovmf.git
  Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
  Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164910/


  commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
  Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
  Date:   Tue Aug 31 09:29:48 2021 +0800
  
      OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
      
      Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
      the kernel, initrd, and kernel command line from QEMU's fw_cfg.
      
      Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
      Signed-off-by: Gary Lin <gary.lin@hpe.com>
      Acked-by: Anthony PERARD <anthony.perard@citrix.com>
      Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
      Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
      Tested-by: Jim Fehlig <jfehlig@suse.com>

Revision graph left in /home/logs/results/bisect/xen-4.13-testing/build-amd64.xen-build.{dot,ps,png,html,svg}.
----------------------------------------
164910: tolerable ALL FAIL

flight 164910 xen-4.13-testing real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/164910/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 build-amd64                   6 xen-build               fail baseline untested


jobs:
 build-amd64                                                  fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:53:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:53:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182900.330743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGkf-0006KE-IG; Thu, 09 Sep 2021 09:53:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182900.330743; Thu, 09 Sep 2021 09:53:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGkf-0006K7-F7; Thu, 09 Sep 2021 09:53:49 +0000
Received: by outflank-mailman (input) for mailman id 182900;
 Thu, 09 Sep 2021 09:53:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOGkd-0006Jx-Qv
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:53:48 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d25a51f3-1153-11ec-b1a9-12813bfff9fa;
 Thu, 09 Sep 2021 09:53:46 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 1899hO8Y004708;
 Thu, 9 Sep 2021 09:53:43 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2057.outbound.protection.outlook.com [104.47.4.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayfvur19d-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 09:53:42 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3469.eurprd03.prod.outlook.com (2603:10a6:803:18::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Thu, 9 Sep
 2021 09:53:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 09:53:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d25a51f3-1153-11ec-b1a9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aSiP+RDkk3DSKYoR4Vi8KHpglHz8yClcE76tr782TgQXzsvxvyz/QIebyeusHrfaxPr5sa24q90Rvq+Nh5b/c1DmK4W10eQ48N+BbTvCzK5o8ONCQmkKoAbfYoYI6KQK4MIhWXXTzpSiwa1ZnZlzloaP+iZk2fAeV7vPIPECUOk3444zBQeMhzEu7kxyZYjIVySrgdICcvCQ1ji0xv2js2WRdtcymaNMpHTSdaRvjfmBPiul0wWhueXEyqhCUzph4K1MlBmi4YicafyHX1LzauHP9Y6ezFToG2S9BtLGtd/vSS+StDIagUKCdVgeDCiIupy9LvCZwbrUplKI7RAwvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Aa779dZMNrHLiJ6/fJbdM9JNUroII9N4XsxRLIy+C3A=;
 b=iXMb9uaWF9sbazoSJF5Qy0fTwMh1p245pgEvD50+oIlZwrgDTn9wXGb1xmNAmzAc6R6F96fQet/wk/cp0/CvqVueUovHi/Q8dXp5i+2QU+5kYWoUfuy+OzCwiYKCflsRuxnoCEWx+Yl66qDh31wnsgB7QuG1dwK3ncmOPgIb5P7k12LE65VQEJfL97lJBi+MKDb8CI3OYxqpBqv+4DT63bacU400fKnp1lXd8tiJAaNU2AFA9MT5d1vktrOEBfR6RXGjkU2xECVRKVl8AQjvYW9kZl0pzXuXNalN9Ck7mguQUts8DXDJ/1QCc0jaKfbglAp6o4stidGmsgT2TJOj2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Aa779dZMNrHLiJ6/fJbdM9JNUroII9N4XsxRLIy+C3A=;
 b=RtdyHFYX3gPKLUxjxFGo1v0xZUn361mlcF2lOKbKQ7opaDeazD3lr5Sm8sMw293D4X2F3Vk7WKmkK4MgAWXeBdI6Gueer82z8GLwhKnpywzQXDO388Az0rUTEXilqMyMQnFeIh4ftSScxlPqmqEy+lfgDv137K1jMtvbQfWuzUKDBEs70CPCrP8l1pa5xGc798fonpquqEL18w0uG+qXg9rNK9j6wNKn1Th7S2hIceZ9oj+L0M8q++dn+TwMiGGFiNpD8e+78i8VHi9h7AIWHO3LSiAyOzw2KZwEri9vUuTPM2MD+jpCsctGKmyCG7cLzGC80ZzVfpMdBw7Tu7GToQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH 7/9] vpci/header: program p2m with guest BAR view
Thread-Index: 
 AQHXoKxowwaOW1OR4UKei+2TOBJan6uXHAEAgAQmJoCAACVpAIAADfgAgAAGsgCAAAOLgA==
Date: Thu, 9 Sep 2021 09:53:38 +0000
Message-ID: <08f50550-de12-d6c0-e873-9f293639b0d9@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-8-andr2000@gmail.com>
 <90813678-d873-d346-5f53-ab97aaedd3e6@suse.com>
 <f35a7bbb-8a9a-06c3-c7b3-60b49013d8e5@epam.com>
 <ee9a69f1-6b47-a944-7255-4ffaa3e1c601@suse.com>
 <b9fe88da-3bdb-0b37-f2f4-9a575806e6de@epam.com>
 <2cffae86-1a2d-a9c1-e5f1-1083ed24480f@suse.com>
In-Reply-To: <2cffae86-1a2d-a9c1-e5f1-1083ed24480f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c8210ded-80c6-498b-dd7f-08d97377b273
x-ms-traffictypediagnostic: VI1PR0302MB3469:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB34695B85E9FA4917D63DA966E7D59@VI1PR0302MB3469.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 B4DmryE+AZOFMlgqcLFvC7ojsO+Xl3n/GeS5DaAbaRPeNtIa/oX7IHQl6jsVOS3yw9FAIRxB6M75z+BtUDeqMdKqnmr17fanehQfINlAoz+lQ8mfbwCqjqSxMZhxKDU+2nDK4nhfJeSxKXKqs7RYpoq8gQSeMV2FFXf0VJXd21JDaHPWxqs7IoNBMh71f/CgrGHHiu6sDSSRpkZM9OPOw3JfAIYDtTg9gDqlMY4jJWv/BQtIVMfc4Utm1Dd0acW3YL6sqmWr7LNvJgCjBP9qdeOTw3ZP1N/Fm81qRIZ5OOtgduE69Nz+puFpvMIxtOfgNw+BGTBIf8qn4sFy59tNtXyxAzNc3aMlvKGUEcpjjvpQLtrrdsdOYmTM0qiH/oTkLRlH9GKX9g/xVszADC1K0GwyUtzloylLvg9dWQY/Rs3RHqOdgy9y3CTEU9lVuoHvpvpwrSgZgh4bJz5SKaJNQj+ASRcypZ0HL4ekdD2eUSnuFXYuYCY/H+altD+UOVmOsgN6a53n29SHopRMeMlnIqHzaqbEQoRhdJb3JnYTJsuL5nv8Qa0vstWF1zlyjdTThpF2myvhUDHVyC9Fkb+Vnq71sErFTpIkGVrVuKwv72yShIP++uOyIl4J+Z32QAtXG2tB5BvuukPP/4hwhW6QhBUyWikKzqFG0qLH9nBtA9X7+NdOvdWOe4IIqS3cYZGVo9TJPJP2TE1LuNXvNherdGVxQzwvxMi15byLIgnkhZ6kIzE/dZdwCKP5eaAdU6Dx
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(366004)(39860400002)(346002)(396003)(76116006)(316002)(2616005)(122000001)(2906002)(186003)(26005)(110136005)(38070700005)(53546011)(66446008)(91956017)(6506007)(36756003)(71200400001)(6512007)(31686004)(86362001)(31696002)(83380400001)(8676002)(66946007)(66476007)(54906003)(66556008)(64756008)(5660300002)(6486002)(4326008)(478600001)(8936002)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SVpTMFdGNENabHJ4NmlQb003bzhjUlFDZUU4LzNQcHpvS09XK1ZTZDRYK1p1?=
 =?utf-8?B?dVRmTWFxKzRTOFdKUVRwTzdramUxNlp5Vm9YS3lCNDdwcFdiRFF6NTVTeFln?=
 =?utf-8?B?ejJYeGNHUWxEd0JlaC9PWjNqd1JzSjdSYlFBN1NYNUN5Um9aTmNudzhsZ2FL?=
 =?utf-8?B?MVZPeU12eFhMNGxibjduQzdnT0M0dXEveDBvWTh1SDFZcTVDZE1sZGpqNm5r?=
 =?utf-8?B?QWtqakdsS1RvMVZMWEtoSkdROEVRdW1ZYzFWQVVKMXVNWmwzQ1RvbkI0NXE1?=
 =?utf-8?B?OVBOa3JKN0Fpc2p3dm03STBYU1pMbXduMVZndGpCZUVmYlRQb0N5N2t0ZnZ3?=
 =?utf-8?B?YjJYMlJjUDNXQ3NzcXBuUjlmbWFJb3lrSCtGV2N5R0pzRjhHUFZ0T3k4dFVV?=
 =?utf-8?B?NUpGcFZvanNmeGY3QUFzZmdkd1V6dkphcjRLWnF3ekRNcm5oTlhaZHliVzAw?=
 =?utf-8?B?eG9FUEVoM21vRkhlcmR5NlBGWklQNG1TcXN4WVNPWkRob3ZseVZwQWpOcWk1?=
 =?utf-8?B?ekNoMHRBUGQwQjdFKzN2R1JRSDd5akFCQ2FOOUlBUHFCOTJBNVB3Vmt6YlFy?=
 =?utf-8?B?VDF0Zi8yUlgyaUFTNS9wcFRDZVAvK3R5d0RCbW9aa0NBdndKZFhmM1ZXMllQ?=
 =?utf-8?B?Skp2OCszQ1pqRWNBb1RJb3Fib3RjL2RCVEp5dzNDakR2L3Y0MWc4NGNpa2hh?=
 =?utf-8?B?bEdFTXdJOGhpcnE1cEdzK1Bzd2x1Tm4yZ2V1RlhHa0VjQVp0ekh4OExGWlV3?=
 =?utf-8?B?YVlaTjBPbXJLclE0RndDQmNuRDhWTUJ4eE5pU2I5cS9vR3BwVDBmSFlQVFht?=
 =?utf-8?B?em5XcmxYanQyYStRYTlIU21CaEJTYlI5Z3FpNlpTK2oyNFU2eGlkYnJDWlVp?=
 =?utf-8?B?RExuSTQ3cWpSTTRJUFcxbW1pZDJYQ3ZoTmp4SUpxSzZvZytIdy9Gdk0rVGNF?=
 =?utf-8?B?a1NmZ1A1Y0tiaEdNbnowR2poSmVpcnRPdUpvbTVmQ0hCVXQ1eWZ4OXAxbEhL?=
 =?utf-8?B?UHNLY2xsc0V6YVFyUzJMWEphQVZnZlJLUWFvd1BmUUp0Z1RwbE1vNm10TXhC?=
 =?utf-8?B?QTNhUU9aLzI4WlQveUtNWTFuRit3Z3JhV2w4OEZjeURwN0N0bTJqYlBYQStp?=
 =?utf-8?B?OGdSYlI0K0ZPTDVCQk1OYnJCck8wbjluWWU1ZlRXVlJJcEVrZm5tR0taTThO?=
 =?utf-8?B?T0UzM2JwaytSTDJNaDNOQ0ZLelNYTTNmMkMvVm82SzJVRFpxb1JCYk9GS1JY?=
 =?utf-8?B?TGQwWjZWZEdsQUkwSUtUaFZTSGxJWUlpdDB4WUhIZkh6MlpYZXpybXFGc0pV?=
 =?utf-8?B?cW9ncmdCQjVPMWh3LzdIUlV1ODVoV0J2Z0ZENXF4NmNjMEc0OUhwQW1zVXh5?=
 =?utf-8?B?K0djL25NNjU5Qzk1MnJkL2FKaS94VFlUSWM5aEY5b2xoVGh2Mk9zeXliclgr?=
 =?utf-8?B?alphc2l3TzU4RHRDdXlpc0VpK3g2NXQ1eXM0ZGcwOUNKeHhsS1Z1V0xQeE5U?=
 =?utf-8?B?dU43S3hZOXpESEVTSGFvMFVJMnprQnZuNlNGd1VHVUcvNjZIUlFXMlhTQVVL?=
 =?utf-8?B?TW9JQlMrTDYzZzRuY1VUT2swRlpmaldwcnlUT3BXRGI5TG13enJaeEx6L1hz?=
 =?utf-8?B?WXBUdDdqTkU5a0Y1bDNXL1RPNS9RNk90RDg4RWN3dzAwZENiRVJWTlh5UFdp?=
 =?utf-8?B?a1NQTTFjenF4cnlDZTYzNEsrL1liNW15amQ5WkZqaGZMYjhPdjVCMHVHNm9C?=
 =?utf-8?Q?+ix6rQEwq663l2VJzV8twbDkBnbsRhRrxCcBSdw?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A05C204D3831EC44835A0C080D731A93@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c8210ded-80c6-498b-dd7f-08d97377b273
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 09:53:38.1069
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: hH9k8jiKv6J+1EwhZ071DpCbOTl+Fq6LSVat6Wnw9VBn4oIiOTmAfUSMnPbU0G9CFTlrijd9vNQqpoWDpTYwiwEXS1aF2rJ9D3haJOArhVwZ72SCfC3uR9wsZ7G3yYs8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3469
X-Proofpoint-ORIG-GUID: jmxWlJgogdM5J7pKS1Bm40samr2sbib9
X-Proofpoint-GUID: jmxWlJgogdM5J7pKS1Bm40samr2sbib9
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_03,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 bulkscore=0
 lowpriorityscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0
 impostorscore=0 priorityscore=1501 suspectscore=0 malwarescore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090060

DQpPbiAwOS4wOS4yMSAxMjo0MCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTE6MTYsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTE6
MjYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMDg6MTMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNi4wOS4yMSAxNzo1MSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwMy4wOS4yMDIxIDEyOjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBAQCAtMzcsMTIgKzQxLDI4IEBAIHN0YXRpYyBpbnQgbWFwX3Jh
bmdlKHVuc2lnbmVkIGxvbmcgcywgdW5zaWduZWQgbG9uZyBlLCB2b2lkICpkYXRhLA0KPj4+Pj4+
ICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nICpjKQ0KPj4+Pj4+ICAgICB7
DQo+Pj4+Pj4gICAgICAgICBjb25zdCBzdHJ1Y3QgbWFwX2RhdGEgKm1hcCA9IGRhdGE7DQo+Pj4+
Pj4gKyAgICBnZm5fdCBzdGFydF9nZm47DQo+Pj4+Pj4gICAgICAgICBpbnQgcmM7DQo+Pj4+Pj4g
ICAgIA0KPj4+Pj4+ICAgICAgICAgZm9yICggOyA7ICkNCj4+Pj4+PiAgICAgICAgIHsNCj4+Pj4+
PiAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIHNpemUgPSBlIC0gcyArIDE7DQo+Pj4+Pj4gICAg
IA0KPj4+Pj4+ICsgICAgICAgIC8qDQo+Pj4+Pj4gKyAgICAgICAgICogQW55IEJBUiBtYXkgaGF2
ZSBob2xlcyBpbiBpdHMgbWVtb3J5IHdlIHdhbnQgdG8gbWFwLCBlLmcuDQo+Pj4+Pj4gKyAgICAg
ICAgICogd2UgZG9uJ3Qgd2FudCB0byBtYXAgTVNJIHJlZ2lvbnMgd2hpY2ggbWF5IGJlIGEgcGFy
dCBvZiB0aGF0IEJBUiwNCj4+Pj4+PiArICAgICAgICAgKiBlLmcuIHdoZW4gYSBzaW5nbGUgQkFS
IGlzIHVzZWQgZm9yIGJvdGggTU1JTyBhbmQgTVNJLg0KPj4+Pj4+ICsgICAgICAgICAqIEluIHRo
aXMgY2FzZSBNU0kgcmVnaW9ucyBhcmUgc3VidHJhY3RlZCBmcm9tIHRoZSBtYXBwaW5nLCBidXQN
Cj4+Pj4+PiArICAgICAgICAgKiBtYXAtPnN0YXJ0X2dmbiBzdGlsbCBwb2ludHMgdG8gdGhlIHZl
cnkgYmVnaW5uaW5nIG9mIHRoZSBCQVIuDQo+Pj4+Pj4gKyAgICAgICAgICogU28gaWYgdGhlcmUg
aXMgYSBob2xlIHByZXNlbnQgdGhlbiB3ZSBuZWVkIHRvIGFkanVzdCBzdGFydF9nZm4NCj4+Pj4+
PiArICAgICAgICAgKiB0byByZWZsZWN0IHRoZSBmYWN0IG9mIHRoYXQgc3Vic3RyYWN0aW9uLg0K
Pj4+Pj4+ICsgICAgICAgICAqLw0KPj4+Pj4+ICsgICAgICAgIHN0YXJ0X2dmbiA9IGdmbl9hZGQo
bWFwLT5zdGFydF9nZm4sIHMgLSBtZm5feChtYXAtPnN0YXJ0X21mbikpOw0KPj4+Pj4gSSBtYXkg
YmUgbWlzc2luZyBzb21ldGhpbmcsIGJ1dCBkb24ndCB5b3UgbmVlZCB0byBhZGp1c3QgInNpemUi
IHRoZW4NCj4+Pj4+IGFzIHdlbGw/DQo+Pj4+IE5vLCBhcyByYW5nZSBzZXRzIGdldCBjb25zdW1l
ZCB3ZSBoYXZlIGUgYW5kIHMgdXBkYXRlZCBhY2NvcmRpbmdseSwNCj4+Pj4gc28gZWFjaCB0aW1l
IHNpemUgcmVwcmVzZW50cyB0aGUgcmlnaHQgdmFsdWUuDQo+Pj4gSXQgZmVlbHMgbGlrZSBzb21l
dGhpbmcncyB3cm9uZyB3aXRoIHRoZSByYW5nZXNldCBjb25zdHJ1Y3Rpb24gdGhlbjoNCj4+PiBF
aXRoZXIgaXQgcmVwcmVzZW50cyBfYWxsXyBob2xlcyAoaW5jbHVkaW5nIGRlZ2VuZXJhdGUgb25l
cyBhdCB0aGUNCj4+PiBzdGFydCBvZiBlbmQgb2YgYSByYW5nZSksIG9yIG5vbmUgb2YgdGhlbS4N
Cj4+IFRoZSByZXN1bHRpbmcgcmFuZ2Ugc2V0IG9ubHkgaGFzIHRoZSBNTUlPcyBpbiBpdC4gV2hp
bGUgY29uc3RydWN0aW5nIHRoZSByYW5nZSBzZXQNCj4+IHdlIGN1dCBvZmYgTVNJLVggb3V0IG9m
IGl0IChtYWtlIGhvbGVzKS4gQnV0IGZpbmFsbHkgaXQgb25seSBoYXMgdGhlIHJhbmdlcyB0aGF0
IHdlDQo+PiBuZWVkIHRvIG1hcC91bm1hcC4NCj4gQW5kIHRoZW4gd2h5IGlzIHRoZXJlIGEgbmVl
ZCB0byBhZGp1c3Qgc3RhcnRfZ2ZuPw0KDQpCZWNhdXNlIG9mIHRoZSBob2xlczogdGhlIHJhbmdl
IHNldCdzIHByaXZhdGUgZGF0YSBjYW4gb25seSBob2xkIEJBUnMgc3RhcnQgTUZODQoNCmFuZCBz
dGFydCBHRk4uIEl0IGRvZXNuJ3QgaGF2ZSBhIGxpc3Qgb2Ygc3RhcnRfe21mbnxnZm59IHdoaWNo
IGRlc2NyaWJlIGVhY2ggcmFuZ2UsDQoNCmJ1dCBvbmx5IHRoZSBzdGFydF97bWZufGdmbn0gb2Yg
dGhlIHdob2xlIHJhbmdlIHNldCwgZS5nLiB3aGVyZSB0aGUgQkFSIHN0YXJ0cw0KDQpTbywgYmVj
YXVzZSBvZiB0aGUgaG9sZXMgd2UgbmVlZCB0byBhZGp1c3QgdGhlIHN0YXJ0aW5nIGFkZHJlc3Nl
czoNCg0KMC4gTU1JTzAgPC0gd2UgcGFzcyBzdGFydF9tZm4gYW5kIHN0YXJ0X2dmbiBwb2ludGlu
ZyB0byB0aGUgQkFSIHN0YXJ0DQoNCjEuIE1TSS1YIDwtIGhvbGUNCg0KMi4gTU1JTzEgPC0gbmVl
ZCB0byBhZGp1c3Qgc3RhcnRfbWZuIGFuZCBzdGFydF9nZm4gd2l0aCByZXNwZWN0IHRvIHRoZSBo
b2xlIGFib3ZlDQoNCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:53:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:53:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182901.330754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGkn-0006ei-Qw; Thu, 09 Sep 2021 09:53:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182901.330754; Thu, 09 Sep 2021 09:53:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGkn-0006eY-NO; Thu, 09 Sep 2021 09:53:57 +0000
Received: by outflank-mailman (input) for mailman id 182901;
 Thu, 09 Sep 2021 09:53:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+XH1=N7=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mOGkm-0006dM-67
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:53:56 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id d82c19c7-1153-11ec-b1a9-12813bfff9fa;
 Thu, 09 Sep 2021 09:53:55 +0000 (UTC)
Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com
 [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-239-mnPkolytPYuKBlSgF8ashQ-1; Thu, 09 Sep 2021 05:53:52 -0400
Received: by mail-wr1-f72.google.com with SMTP id
 u2-20020adfdd42000000b001579f5d6779so333841wrm.8
 for <xen-devel@lists.xenproject.org>; Thu, 09 Sep 2021 02:53:52 -0700 (PDT)
Received: from [192.168.1.36] (21.red-83-52-55.dynamicip.rima-tde.net.
 [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id a133sm1238458wme.5.2021.09.09.02.53.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 Sep 2021 02:53:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d82c19c7-1153-11ec-b1a9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631181234;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vwr+3pvdxH1NRlgdXk14JF+JyDx/qDTwNZRMxhXGyjE=;
	b=Kyvdfd6JPFtmbyo9bNjcveIEg8umASDrTRRJy7yU8PiIdRAnA7McDFqfofJWFzyvESpSlV
	MCXykKbBUW6uIQMHKml7OBVXbKOOeY5Mf4fQnQJkITZuEOMKG3povZs64OrFU+LmGdW7sZ
	iuC9x5iLbjUK2uym4VFs7hUCyB7C98U=
X-MC-Unique: mnPkolytPYuKBlSgF8ashQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=vwr+3pvdxH1NRlgdXk14JF+JyDx/qDTwNZRMxhXGyjE=;
        b=h4j2yooyrGbmFOM6rDU83t0IyMaafF2zBCKX6kjxt6CcGcq0ka79t883dIDTkw98Bi
         buYHJeUSWzBgGlgqIw3CBCw8+g/Z0qA8HO66A5TJf0UQ6ex1sKESDl3pwji0wtBhaMV3
         FOVEo4pWzpwdghncR915s3/CQklxyvHOGvCIRr937q0t2h1j9e0pRGLIG+kvgw4Tt2HJ
         FyVyUCTNmG97AolzvbJXsJVkq72/2Zxj6Jcx5y/72S6vV3U2/pAslwTNt7En71BtluS3
         e7rYrnfbzC4G2SyXVdMWix4noZYAsjhKP/exA+ylEUHG2o6ELxT7Q+5HjHlggbd/jVmc
         hW+Q==
X-Gm-Message-State: AOAM532VIJJOytJBw/QIxXKMBmFZZa4p/p5of09+PQqNPDVr1xIpqFsx
	Ae+Kah0dloAnBHLpT9nHYbkNonRCsh9Wg2zc9fFUFF7RQucN3tar53O0wTE/T9QWuT0XQq3RzF5
	ZSI19MAD/VX+ybFe0IT1oNYyQWa/XwI460CUK8w65F58hjBFAUBsAEb+CLAiAQYC0uWgrffN83h
	lR
X-Received: by 2002:a7b:c441:: with SMTP id l1mr1908287wmi.69.1631181231085;
        Thu, 09 Sep 2021 02:53:51 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyIkZfHni7JT96UiXXXKpQ6vrbvW/tTxBRZU6eZ7/YkCglB7MvuULXjbpqr0tqBjlH8EEPhiw==
X-Received: by 2002:a7b:c441:: with SMTP id l1mr1908267wmi.69.1631181230881;
        Thu, 09 Sep 2021 02:53:50 -0700 (PDT)
Subject: Re: [RFC PATCH 03/10] block: Use qemu_security_policy_taint() API
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>,
 qemu-devel@nongnu.org
Cc: Thomas Huth <thuth@redhat.com>, Prasad J Pandit <pjp@fedoraproject.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Markus Armbruster
 <armbru@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= <berrange@redhat.com>,
 Eric Blake <eblake@redhat.com>,
 Richard Henderson <richard.henderson@linaro.org>, qemu-block@nongnu.org,
 Peter Maydell <peter.maydell@linaro.org>, xen-devel@lists.xenproject.org
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-4-philmd@redhat.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>
Message-ID: <eea8a6d7-72b8-9fd6-07cf-b14ab6099833@redhat.com>
Date: Thu, 9 Sep 2021 11:53:48 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-4-philmd@redhat.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/9/21 1:20 AM, Philippe Mathieu-Daudé wrote:
> Add the BlockDriver::bdrv_taints_security_policy() handler.
> Drivers implementing it might taint the global QEMU security
> policy.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/block/block_int.h | 6 +++++-
>  block.c                   | 6 ++++++
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index f1a54db0f8c..0ec0a5c06e9 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -169,7 +169,11 @@ struct BlockDriver {
>      int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
>                            Error **errp);
>      void (*bdrv_close)(BlockDriverState *bs);
> -
> +    /*
> +     * Return %true if the driver is withing QEMU security policy boundary,
> +     * %false otherwise. See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool (*bdrv_taints_security_policy)(BlockDriverState *bs);
>  
>      int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
>                                         Error **errp);
> diff --git a/block.c b/block.c
> index b2b66263f9a..696ba486001 100644
> --- a/block.c
> +++ b/block.c
> @@ -49,6 +49,7 @@
>  #include "qemu/timer.h"
>  #include "qemu/cutils.h"
>  #include "qemu/id.h"
> +#include "qemu-common.h"
>  #include "block/coroutines.h"
>  
>  #ifdef CONFIG_BSD
> @@ -1587,6 +1588,11 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv,
>          }
>      }
>  
> +    if (drv->bdrv_taints_security_policy) {
> +        qemu_security_policy_taint(drv->bdrv_taints_security_policy(bs),
> +                                   "Block protocol '%s'", drv->format_name);

Hmm I should check for phase_check(PHASE_MACHINE_READY)
and qemu_security_policy_is_strict() somehow, to refuse
adding unsafe drv at runtime instead of exiting...

> +    }
> +
>      return 0;
>  open_failed:
>      bs->drv = NULL;
> 



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 09:58:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 09:58:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182913.330764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGog-0007h3-GY; Thu, 09 Sep 2021 09:57:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182913.330764; Thu, 09 Sep 2021 09:57:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGog-0007gw-D7; Thu, 09 Sep 2021 09:57:58 +0000
Received: by outflank-mailman (input) for mailman id 182913;
 Thu, 09 Sep 2021 09:57:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGoe-0007gq-Nb
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 09:57:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6764c71e-1154-11ec-b1aa-12813bfff9fa;
 Thu, 09 Sep 2021 09:57:55 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-OeYDtL1-Pza57_bz_DDN-Q-1; Thu, 09 Sep 2021 11:57:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7375.eurprd04.prod.outlook.com (2603:10a6:800:1a8::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 09:57:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 09:57:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0141.eurprd04.prod.outlook.com (2603:10a6:208:55::46) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 09:57:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6764c71e-1154-11ec-b1aa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631181474;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=eEL9qNvSrPO3a2KYULh+dWzPR46hwF5kM/seQC0Iyus=;
	b=LTezPbAnViKKLsoQ/eA7Z4IxGvxrXoSEcTw3kYDDWWVj9ZZexnsrE/P1euzUD8GksV22ah
	nsbyFuIAwW26l6fJX9v6foH3yMbGklDsyJ/HEKM67INF/qgCA7IgxLR9oIEFevuJvGxvLb
	GzxdLOZAzmmsVjLZogFbiYxOGCFm9UY=
X-MC-Unique: OeYDtL1-Pza57_bz_DDN-Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f9vzmnOWx4dfdCVunZ2TH7Ts+ORsAs533phdQ8czWF/gVnnbCEIAP6JN1X83G9OwpSRu3ERQOfvyn5K0scYZbYQAHACyj8ybrOR1fo43d3Is98iGsQrm/IA0LpK4iOLX9MnKNa6+7pIWUagUcXGW0P9rxYKUnTxrOc46DMfq5PoexiS9vcJ+GpeO60eOs6/AdTtbRfo3fMobFYWfhPpe2REObkSB4QRwH5gt1J/isPK4TCRrywaSWjqli8PHGliJTcsR/QS6og0dvnwhzvYjwi8ZTcL4MUrcSuw75MRUs7rXEArHeZni8hkotAfFI5+l/L/u6Rg+hFILlSLl5RJFsg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=A86rnhXZOE1JDW4J3/2PB7wdjmEoc211vRmY04sxYjc=;
 b=NVV2na+bWcRJRmjkvgyi9jqTzbjfd73HII6tbJXF3zmrziHCqcV3ENVCAGJ31aXwKQIHqbOGngHRY5woyDAJrec6NKeHmh4xrkSoxmvI+/H1YwL7u5Usgr1s0FnfMaqxixOhJUdgdm/5znMkQk+6lwn3xB0ZIYfegwmzIQ4EzY6IgTZjlRNFYfxcfE9mV39kDCINZ8OAUXF07IOyUiuf1V4caW+ibxNg2QL8IQeGafzVtDyB99oJngTliWAClxxJpjW4nPb+asHLOMBuQCoUe2QRWpH37EruZFA/4VkDMR6aeIeLHr0DnvZm4zxfHXQzaQfq3C9bWv9mNzXBO1nSWg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/svm: Intercept and terminate RDPRU with #UD
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908161918.25911-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3f272ea6-02ab-8f70-19cf-28a92421531e@suse.com>
Date: Thu, 9 Sep 2021 11:57:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210908161918.25911-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR04CA0141.eurprd04.prod.outlook.com
 (2603:10a6:208:55::46) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 573a0c99-a700-4b48-9efc-08d973784a10
X-MS-TrafficTypeDiagnostic: VE1PR04MB7375:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB7375CE0155D888BEF07CF62EB3D59@VE1PR04MB7375.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	myzAP1Cxp0rTDCvzmEiB9+v0AwE2u9LuCXnIMbDyaJN4dI7Y3m/EUePjsH9igWro9HwOpxJqQHM3pac4GsLOPIyWbs0DEgJbQI6yE/8ihgNZj1mnHaUPT5MwSMMR22kUyi56iis+dj4Sn9OqUmsDYMWnD56pr2p9ka9XnhhBpgV8CNxIc0ALZkUqFUwp6QOmYtKvK2f+YS8ub5J1dfAmdQg7e5HD96CrfiXfGglRzA1pgO2GQKZVVAmk1OvbAPXcNnMU8jVNLv2Hgkm1mzz1o6IgGBRNUoBYgPZIFfnNMgmQnNBV3uBaRjY+C3yh9r8qYUQleEbzuYBl3jYHfqHqEOlx/QUXfpnmRNGmSB13XJkY9sXRj5PSbytrrNFPIWtg9Z+gWokTpjBDDv8CacEJ+snwZ4xddpFokZu+jv87GpUOHOppoxuPXl7sRJ7ErKdfhSvHv6YKkQsEmB2RETrjgLH9g644PvrtuQkO+O3/m5KGCI+8xHgPTDSz/I4TtvZV0w2CH4mZjcFyJO92FVPDTI83wKA9K7Kk2cToZZIIgvHAqw+82pf6jwGaB30SvZxOwrzIeaImfczcqxusbsVfyCsXxleCUdE+4ShhhG2pFayYUIn0BCixg5j+cvieT/i44AwQznZ+yM/QcFVtVN0RsOJ8MUQSbB6z5i3j0IDpdKPT5h+Ju75T6opSg5J4eOyobc0ZcRVwac8PMU5aYxZcsoo7k8ulNmroxgp05zT18ng=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(39860400002)(366004)(346002)(376002)(8936002)(6916009)(38100700002)(5660300002)(86362001)(26005)(83380400001)(36756003)(53546011)(2906002)(2616005)(66946007)(186003)(66556008)(4326008)(31686004)(16576012)(316002)(6486002)(8676002)(478600001)(31696002)(66476007)(956004)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?AtKoyOuafpcFb0CYp4jgN/gThEMf8evGjpKIMstW3/FUytEm933aGUWODVi7?=
 =?us-ascii?Q?KqbifIWetXrSm8dUP1xL2GOEE4iy39WZaTTU6XVKqQ9Dpr++wV/vSkeIaSQH?=
 =?us-ascii?Q?ZhIInZ6mwMprMGgGt8cQBrIVpOFqxe0+ZBlHx/8xYiSt6nEYX8VBlf2EOQPh?=
 =?us-ascii?Q?9Gxc1EUe1mTpW4qvTn+Q2h9N9o9w4ZhBSfBi1M/F2OVRqHARun1rLyiFAeE7?=
 =?us-ascii?Q?7YTx39uVUjWWA3p2QHMqYU9JdzeZu8bSftaq+J2MGDegR09TZ6beMJAMFtXo?=
 =?us-ascii?Q?c1d6blvWyvC1ZmtmkN0850MlT4oVxH1jtfL3V6U1hrJNZdA6/USIdU7BneQm?=
 =?us-ascii?Q?v9/Fn3ElW1vAGzzQcQiCUwW3QYd4azpdLhTmXkseuCpcwTiE9dG7I0qY/N5q?=
 =?us-ascii?Q?B3H0FlwTc6BE6jrbxw838J4mdntRvVovjvgB7iwcXMVeLOSpd0iuBQjVEWFc?=
 =?us-ascii?Q?Te7VBiAZ0eTtYB1BCAoGZNT+REKuHdvhT0lGtCnK7IVImyueFUx/M/t1KbsQ?=
 =?us-ascii?Q?4r1FDjF7CE6HTcfbnfdbp78TGVZTVX+CxSQAJue1sJFlamk1XKDE/uzRMSVw?=
 =?us-ascii?Q?ivcqJF1gNvI05ZG6sHkC8jt3tod4GVXqGd+EYraWwVVPpVsB9Rmu3JMs3iMG?=
 =?us-ascii?Q?aw5Aq+kLlbz/DgLNxzRgsenNQxxxvKPcmWl+yRR/SCbbXxdIBhAJMBwoWV1z?=
 =?us-ascii?Q?ALbPfNTaL3imKTgon5YQ1lkzGa0rwzy2oTkBXuX5S9G5ryka8ro8M6mZ3BP8?=
 =?us-ascii?Q?6E6lnCrgoZHMA1YZIODYM+5JIZm/X/eTYvsnAIY1fnpAahPUwTso1pD1lxxJ?=
 =?us-ascii?Q?aDYjsR25HJFRX7yPfx5UstxhOWPIouXdL/A2OUxR9fE2qpV7d03za3Trg1SN?=
 =?us-ascii?Q?EYt+TW8K/NHCsD428TNv/tCoOSNyAKbYosE3bG6WoKYTSCHlWtH/Oriv+MiU?=
 =?us-ascii?Q?xXOVKNIYptQkTnZQk7wC1HBvpN3ohbsbMp8mWu2XbJYw4J23yimLZScQd4OW?=
 =?us-ascii?Q?TalOS95Ij8BHioe83jENGoN2iH9VOomIXEEpFazFFsdQdf5ngmOg5iiQHYaZ?=
 =?us-ascii?Q?LMq1UIVvN+uJ5C7i9vxO0d4oUPXWkYa/rFNhBvMtKXSuL/iR6ERMjAtcnLHd?=
 =?us-ascii?Q?7Td8DrLFkF7+xq0MA5h9jcS6BlAd0fASqwAL3mX+6XDgD8vXojR6v1fXEsBQ?=
 =?us-ascii?Q?YsT0PkUSvX9pHZaos7XB3k02RgzG09Ikmp+yLpODkvGLuo2f5tWppoHy6rV8?=
 =?us-ascii?Q?mRNnCrG0UvrdYqeZ+LF8NgHPoec5RXloJRxMCEhB/Eankb4g5bYWsOfxRYVl?=
 =?us-ascii?Q?n0PD8dKUKI/X7SHhCwLTMcKX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 573a0c99-a700-4b48-9efc-08d973784a10
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 09:57:52.6658
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kfeemXsUcyEAWRbkWHRd9m8OqsVR/4eD3c1lzNulpyHCxqcfum2H5xADNt+TYfgy7tvruxXa+JWXoTf3xNDZZQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7375

On 08.09.2021 18:19, Andrew Cooper wrote:
> The RDPRU instruction isn't supported at all (and it is unclear how this =
can
> ever be offered safely to guests).

An implicit hint to me to consider "x86emul: support RDPRU" rejected? That'=
s
still in my queue waiting for ...

>  However, a guest which ignores CPUID and
> blindly executes RDPRU will find that it functions.
>=20
> Use the intercept and terminate with #UD.  While at it, fold SKINIT into =
the
> same "unconditionally disabled" path.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
>=20
> I could have sworn that I'd posted this before, but I can't locate any
> evidence of it.  I've got a separate patch adding the CPUID infrastructur=
e for
> rdpru, but that is better left until we've got more libx86 levelling logi=
c in
> place.

... this. Which - if exposure to guests makes no sense - would seem pretty
pointless then as well?

> --- a/xen/arch/x86/hvm/svm/vmcb.c
> +++ b/xen/arch/x86/hvm/svm/vmcb.c
> @@ -70,7 +70,8 @@ static int construct_vmcb(struct vcpu *v)
>          GENERAL2_INTERCEPT_STGI        | GENERAL2_INTERCEPT_CLGI        =
|
>          GENERAL2_INTERCEPT_SKINIT      | GENERAL2_INTERCEPT_MWAIT       =
|
>          GENERAL2_INTERCEPT_WBINVD      | GENERAL2_INTERCEPT_MONITOR     =
|
> -        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP;
> +        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP       =
|
> +        GENERAL2_INTERCEPT_RDPRU;

Some of the other intercepts here suggest it is okay to enable ones
in the absence of support in the underlying hardware, but I thought
I'd double check. I couldn't find any statement either way in the PM.
Assuming this is fine
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:01:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:01:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182920.330776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGs8-0000yN-0q; Thu, 09 Sep 2021 10:01:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182920.330776; Thu, 09 Sep 2021 10:01:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGs7-0000yG-Tu; Thu, 09 Sep 2021 10:01:31 +0000
Received: by outflank-mailman (input) for mailman id 182920;
 Thu, 09 Sep 2021 10:01:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOGs7-0000yA-F4
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:01:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e74bfcc2-1154-11ec-b1aa-12813bfff9fa;
 Thu, 09 Sep 2021 10:01:30 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-kRlQmk3wNVuaEPf7USvpug-1; Thu, 09 Sep 2021 12:01:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5327.eurprd04.prod.outlook.com (2603:10a6:803:5c::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Thu, 9 Sep
 2021 10:01:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 10:01:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0030.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Thu, 9 Sep 2021 10:01:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e74bfcc2-1154-11ec-b1aa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631181689;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lSAsmsVfmUcG0j3XC3o6a3VVs7ZK9vGA25F0VptQlvU=;
	b=M/GvZqUk7c638XcKEJXz5B5PWBUXYuUuOcSGvUVmuZmqTszlnFh8FDPsHBTAaRy3LkOb50
	m0nnY2gosPo0QCx6dDXpXUuy7kMXSVZL6pm8GHsLAiLdddwgjuo+r4r+rmV6sTBCd/a+oe
	/6K2V3J6Rb9vWuj/lAmU87L88hLf2W0=
X-MC-Unique: kRlQmk3wNVuaEPf7USvpug-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SsYKgVtf4ixVXA2JbqV5OZmGWPy5+i5h/rd5AMk94N/qq1tXaIcsZXRmIPEwoGmKy0qFlvWzMJNmqtU3CahEKuzO6pBi8zTS2HpHoCYJyBN3lrcQAlVhQpm+gTFe6t0JnpPEyvNFGRRixOltTXDUOIGFabhLVigLtKv5ruTPI2svkEEozEkJwaul0UPY6TbPxbVZYspOMo/O8C0d8Y9kFQ+6DGYbzeMLddsZFOcK7+ufnTEXLjZxTwvyAMIcEvCusD6/gm95kthSK0fjISF+Y3AbUmS+OS+A5wYDVJ4Wh4pFICAE4+MP8Qg+u2sQDLuTQMG98rmSKBMWyZUl7NPedg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lSAsmsVfmUcG0j3XC3o6a3VVs7ZK9vGA25F0VptQlvU=;
 b=OHf2GlMmy1PAvGoCWhn95DyRh4wB46jSLHGyfRHhZqJlDroVzBJs2l2aQIvL8uWs656F6dQ9xhc8pKZfvXbRE2OZhPMgnqnRgwjC81TDMMBfyM0URZ4TnnFCNZP/TyNPeZ2wY5Zs7rlB+LnCg/RN1+tJaFSzjRw4CaDMsly21smCLi6Itjf0QrAi0VbzSvmTffcE4CG5yFvEGwjIHlmM1fmjq0+PuY9AURToKCp8lewumdelRb8GUYNbP8RDJLeRKJBmEwZOBiahJozjcSU4j7xK3OpvPpgyR42XihTqKQOaMaF6+6LuTeCg0OWRNfZ11/MuHdl/VnpGKuRK3QaIjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xenproject.org; dkim=none (message not signed)
 header.d=none;xenproject.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [xen-4.13-testing bisection] complete build-amd64
To: Ian Jackson <iwj@xenproject.org>
References: <E1mOGjd-0004AY-0t@osstest.test-lab.xenproject.org>
Cc: xen-devel@lists.xenproject.org,
 osstest service owner <osstest-admin@xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c3664ec4-dcc3-f7ed-9701-f08d291efd37@suse.com>
Date: Thu, 9 Sep 2021 12:01:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <E1mOGjd-0004AY-0t@osstest.test-lab.xenproject.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0030.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a5632c3-c9a1-4bbd-e3e0-08d97378c9a7
X-MS-TrafficTypeDiagnostic: VI1PR04MB5327:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB532755103CF27F5EBA44FC88B3D59@VI1PR04MB5327.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:949;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Y1e25GdvlL/2NzyznSWDoD/FKNK3ElWC5FjaEy1MkC39eSmAQXNeVails8v88TCGxW+f8XJY1DLcWMZPo7I070qn+wu2D3rjOm6yKKVxKJC5KLRblRhdRvRmOeQArT7WbMQ0T64Surd9lbYeRhQ6LZQXOLys7PTrsWE2VW5UiZX5q0v4sPGomZBgBRPGTd4UBXcKd6rvmLxRrabEzkIPEWaoksce5UaoFO4C3Kh0Bx6yV7KDaLu4yqBa9EoF9yy/jQsm8QkUMr31mpl/yGjQ7CQ1f+K8eiRMLT8D0u9kj6ExL3Wv2rfI48ExeieFUQ+ROMwA743yR7+3I2W70FcEs5BdGyRZqJK66j1UsuUyMHI2AEPP2gTpsGaEkdKSpGqdMlQGapwxB+n1nQv2gwLMsCpjXOprWVx3PPfHJRiIIaHWiFWgZAtDtuUTaC5+yC1Pt9egd9ccxja7OaGKkCrxon0Zhcw7R1Jl4AHhitOZyW4OuJWwigZ4KUE57xcT7TKXTfWNMjQi+LOwuBooDidP0v/0tBsdmV7zLapFK9qpKfRZJhEL/EpehTZc50M8qMwCkPzVAjwrwDuXyCuNReShPQreOY3Pc8nrRTzwBQuPsW2LoImj6VWCFDcx+TDLHISOVp07j6JP5jc1DGuC6WvM/XaOjvuwI29cM0urr9e7xk4HTF4Yh6KBhyUHbsmMBEiezDeSNHxjaX8yj7x7WSD9T77FSS82Zn4ebtwR8iJmg9sDm3z4KEb/7srKjXsqsrF7YFmwAUKCWMGZTD8T/stBlO07p1JEJX3ImzE+pe0oAKR2oQzhHMvwPH/oBRmAggE6
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(136003)(366004)(396003)(36756003)(8936002)(2906002)(956004)(83380400001)(38100700002)(2616005)(31696002)(66556008)(66946007)(186003)(66476007)(478600001)(26005)(86362001)(4326008)(966005)(316002)(6916009)(53546011)(8676002)(6486002)(31686004)(5660300002)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?elF6RVgvajJmaVpsN0dTRGdPNm1sQnNoeG90cDhMeGxPd0hpMFhyVmtDVWFL?=
 =?utf-8?B?NVdHRlU5cTRqQmlCdnlrRGREbzBtVllHMU96d1pNN2QxQUJsQURFVitxcklF?=
 =?utf-8?B?Ny8xb1hxd0MzWHRUSllsN1lIWEV5anZyc1o5ajd4MEszaW9EMU1iWno3a3RP?=
 =?utf-8?B?RFdZMmgyZ3p5V1ZNd1U0RFNRaW44S2FWSkV6dVlVeDZTS1J2UWovWExYYk1y?=
 =?utf-8?B?cWxRUUNFTFhrV3l5Y25FdXJxczJjbytKdnJqdEk0OC8vZTZidzZZMUxpMnJN?=
 =?utf-8?B?UzhFZW1zdWFRTHlQU1Q2bVBabUVKNGFNT3JFUG5GdG9JcVpzRnVpVWRHdU0y?=
 =?utf-8?B?Z1VzT3JZSGVYQVgwUkxBZ1ovNm05cXpmbllHUUJuMlgwOXc2b0JIRXFQa1Rz?=
 =?utf-8?B?eDY2RmtxcXhmaFBLSlhKaE9ta004MkNKV3h1ZENnMnoralZ4cklTYUZOc2lJ?=
 =?utf-8?B?cHpNQWJwVHBKZjJrRzhZSXF5TlZ1dnV1Q05nVUNUZGR6SWY0WGFNZ2xSUDN3?=
 =?utf-8?B?TWZCa00vQ1ZtQ3FCdDQxZGEwbDZjMDZyc0h3eEtMRzNxLzNxcUtyR3dGZ2NH?=
 =?utf-8?B?MjU2ODluZ05seGhFQmwxUTZCUzllRGxpVTIwcUl3bHZHSlE2dFZTQXlsb1dh?=
 =?utf-8?B?bTRRSFJSQlptdGY4TWxGUitPeFJYN0FWNXpNdlE0ekREdkNzNEEzY0tMbEV0?=
 =?utf-8?B?R3JPSURMRUxXWWxZU3ZxZGlISkpiUlBTUUllS1R4cVRvc0k1QW9Rc09XZnVQ?=
 =?utf-8?B?NnVkVkM5WDRRQWxnMTJMYnptM3dJWnRwLzNXdXBKK2lhMEtuaHJRY1BiU2N5?=
 =?utf-8?B?a0VxMUNsWjh5TlRaM1I3bkg1cnJOSzY3cW8zVGpocFI4aG5zWWE3TlZZbWtr?=
 =?utf-8?B?K0VkSUo1WjNHZmgvVERpVUkvV3lnNk42dWcrSWFjT2JOanEwc1Z2TkRqVTN1?=
 =?utf-8?B?Ujk0SkNNK0c0amdGU29pMGx4dnNaVElBTDNIQUhXOW9OR1ZwNHlwaEZVb0RO?=
 =?utf-8?B?TnVzSWJvemJCUTdxQjhBbzl4WGVTaklZaWRIazVDVGpyQlNxeEJpNnlkdlB0?=
 =?utf-8?B?TDFYMUc5cXJ5ZkRkOFZEMWJtZWxXZ21JdWF6UWxhZERqQU85MllrS3NUSTMr?=
 =?utf-8?B?VGZ6Tkp2YjJKQnp0ZVd3bjBGcEZrK3k5M3NTbVlXQ09tdFlwelZtQXpaQzlP?=
 =?utf-8?B?bzRTZWtBWk5IQXRHVGsrbEM0dmhrS0xGTTg0M2NnL3Y4RUUzaldRSWJ1WWoy?=
 =?utf-8?B?a3FMS05NQ2ZQd2V0T25lMmNjMTlHck0wY2V3b1VqWmgzLzFDVmhpdHhWYXJH?=
 =?utf-8?B?NG1HV2NqZVlRUFJXRTJpYnB0NlFYN3UxdGEyNjNaTTdobndzb05EWVZmNStR?=
 =?utf-8?B?ZnBjMW9qTW9yamYzalhvcHJQMVR6aFF1aHVzcDJDYXc2ZXp4b05yTStwc3pO?=
 =?utf-8?B?dTk5emhML2FDWDdBUFNuN3RWOWF6SXFESE1qdmI3ZjhGNjVpcmM3MFNCVVRX?=
 =?utf-8?B?Q2xNckhRQlZPNkx5UVFkYmY3REhkRGtEcUpIYU5udVBqUTU0UCt2TzNyQTNM?=
 =?utf-8?B?cHkyZmI4Q0JIYWdpdHAvZFA0QnJJWVg0blBRVGo0NVpJL05aSmhJZVdiZWFM?=
 =?utf-8?B?Ym9YazJveU0yWkQ5T290cjByRC8rMHFaMGJvOFRyMXBZaExha1Z5R2plcFl3?=
 =?utf-8?B?THZMMHdJZmJxekFyWEEvSy80dnB6K05tTzJrcXVZVGN4dlVhY05DSFZ1V2ZO?=
 =?utf-8?Q?oFAMQkTULC8oOg8C+6SqeUXqheKSEfpDThXth2Y?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a5632c3-c9a1-4bbd-e3e0-08d97378c9a7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 10:01:26.8127
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4nXWb2FLqmGCu4bGur8qu1AroFGLKTiDpJvo1oeL3mL6J6JFngM222CKDvS8YhVqGVtR6j+ZmvwIphnXACAG6g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5327

Ian,

On 09.09.2021 11:52, osstest service owner wrote:
> branch xen-4.13-testing
> xenbranch xen-4.13-testing
> job build-amd64
> testid xen-build
> 
> Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
> Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> Tree: seabios git://xenbits.xen.org/osstest/seabios.git
> Tree: xen git://xenbits.xen.org/xen.git
> 
> *** Found and reproduced problem changeset ***
> 
>   Bug is in tree:  ovmf git://xenbits.xen.org/osstest/ovmf.git
>   Bug introduced:  9f3eda177a4b2d4a33ff1b0307cad42906396562
>   Bug not present: 5b5f10d7465004e3e40ec1f50a3b490b4db595e7
>   Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/164910/
> 
> 
>   commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
>   Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
>   Date:   Tue Aug 31 09:29:48 2021 +0800
>   
>       OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
>       
>       Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
>       the kernel, initrd, and kernel command line from QEMU's fw_cfg.
>       
>       Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
>       Signed-off-by: Gary Lin <gary.lin@hpe.com>
>       Acked-by: Anthony PERARD <anthony.perard@citrix.com>
>       Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
>       Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
>       Tested-by: Jim Fehlig <jfehlig@suse.com>

I'm really curious as to how this could have been identified as the
culprit, when the build issue was a plain hypervisor side one (which
Andrew did supply a fix for yesterday).

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:01:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:01:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182921.330787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGsG-0001Hb-9A; Thu, 09 Sep 2021 10:01:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182921.330787; Thu, 09 Sep 2021 10:01:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGsG-0001HU-5j; Thu, 09 Sep 2021 10:01:40 +0000
Received: by outflank-mailman (input) for mailman id 182921;
 Thu, 09 Sep 2021 10:01:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qxuu=N7=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mOGsE-0001Gi-Un
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:01:38 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ec5a16fe-1154-11ec-b1ab-12813bfff9fa;
 Thu, 09 Sep 2021 10:01:38 +0000 (UTC)
Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com
 [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-89--HBP5DumN-qv7_B9fryJzA-1; Thu, 09 Sep 2021 06:01:36 -0400
Received: by mail-ed1-f69.google.com with SMTP id
 z17-20020a05640240d100b003cac681f4f4so691544edb.21
 for <xen-devel@lists.xenproject.org>; Thu, 09 Sep 2021 03:01:36 -0700 (PDT)
Received: from ?IPv6:2001:b07:add:ec09:c399:bc87:7b6c:fb2a?
 ([2001:b07:add:ec09:c399:bc87:7b6c:fb2a])
 by smtp.gmail.com with ESMTPSA id l9sm808653edt.55.2021.09.09.03.01.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 Sep 2021 03:01:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec5a16fe-1154-11ec-b1ab-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631181698;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NeGWlrf5vev+gEOfCXbrlvWAI/u03MMiPQrjaUt1lXM=;
	b=b8z0Q5hPdxq4Dc/NS+9ZjK6JHkTOh6QOFqeBApJOS4I/3KXyyVXP9p9eJY8JJEx+LeoSdq
	H8A+q2crvkinRqzqk6B3Xdz6zq8LdUQHvSg1V75q88wvenQn3NLmQvIhZYnYD/tSwY+HOX
	+KQsKAXKIvspfKo1h/WcoKfWl63O3JU=
X-MC-Unique: -HBP5DumN-qv7_B9fryJzA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=NeGWlrf5vev+gEOfCXbrlvWAI/u03MMiPQrjaUt1lXM=;
        b=5Ey6+gwA0zUMhjY/YVJhPIO3a6zylN6zI8Ig4VADV6z+LNYfCpZ6aU0wZ6TcSVc0cB
         NZYdWD9rc0CpO5YMgGm6d4Kne6Wiokd4pqIffcAUpOddBv7za5X1HrlHVQcYNtpZ9vTb
         b44EOxKs2SVxdOf6aoktaRtv0yrknwxf4CTuFuZFPtw/pP7nH5rzpG8yxNa63T5lpiri
         O2YOUQA1sMPkJD3x9ZQP9ZVi3V1xRgpb9Z3YhkNU2CIXRIKm3F7BItOtxDPRMde4s92Y
         1bL1nvxRI3kfMx0IO5e5ohJLMmorD8YmK4xlhj/wV8zd2Rt4DOnnVnJ5CCGKsRkghg2x
         dHow==
X-Gm-Message-State: AOAM532jwetbmqGWCeQbfNd4vdmsaLQKomUCyACuQOGsTJ5ktnHqRZ9B
	zAEtIICg85PC4ITu5v1BB3Pc8wY9l+DiwvJH/L29ceHbXSW++VmSMdRFFHohT7jp7m1PkURD04X
	rJT1IgNlZioKlOlTHkpfON4D/ikI=
X-Received: by 2002:a17:906:b782:: with SMTP id dt2mr2490180ejb.310.1631181695717;
        Thu, 09 Sep 2021 03:01:35 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyaaVojQ78dzvuwBQG6dnOgiYYtED8hU6C1zxFqk1STHW4PQ44zaKff1f8hXuv91Gn62rzOaw==
X-Received: by 2002:a17:906:b782:: with SMTP id dt2mr2490145ejb.310.1631181695393;
        Thu, 09 Sep 2021 03:01:35 -0700 (PDT)
Subject: Re: [RFC PATCH 01/10] sysemu: Introduce qemu_security_policy_taint()
 API
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>,
 qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>, Thomas Huth <thuth@redhat.com>,
 =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= <berrange@redhat.com>,
 Prasad J Pandit <pjp@fedoraproject.org>, qemu-block@nongnu.org,
 "Michael S. Tsirkin" <mst@redhat.com>,
 Richard Henderson <richard.henderson@linaro.org>,
 Markus Armbruster <armbru@redhat.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 xen-devel@lists.xenproject.org, Eric Blake <eblake@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-2-philmd@redhat.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <b56b5716-a461-0dbe-f706-524c42589e33@redhat.com>
Date: Thu, 9 Sep 2021 12:01:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-2-philmd@redhat.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 09/09/21 01:20, Philippe Mathieu-Daudé wrote:
> +static QemuOptsList qemu_security_policy_opts = {
> +    .name = "security-policy",
> +    .implied_opt_name = "policy",
> +    .merge_lists = true,
> +    .head = QTAILQ_HEAD_INITIALIZER(qemu_security_policy_opts.head),
> +    .desc = {
> +        {
> +            .name = "policy",
> +            .type = QEMU_OPT_STRING,
> +        },
> +        { /* end of list */ }
> +    },
> +};

No new command line options please.  You could rename -compat-policy to 
just -policy, and make this a "security" suboption.

Paolo



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:03:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:03:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182933.330797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGtk-0002C5-LN; Thu, 09 Sep 2021 10:03:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182933.330797; Thu, 09 Sep 2021 10:03:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGtk-0002By-IX; Thu, 09 Sep 2021 10:03:12 +0000
Received: by outflank-mailman (input) for mailman id 182933;
 Thu, 09 Sep 2021 10:03:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o0l/=N7=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mOGtj-0002Bo-KF
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:03:11 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 230f70c2-1155-11ec-b1ab-12813bfff9fa;
 Thu, 09 Sep 2021 10:03:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 230f70c2-1155-11ec-b1ab-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631181790;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=IeINUvl4C5/dwCvB0P3GDK7VX2gf3en260aSHwb3l2s=;
  b=RQOkfmaGf2jS9OC3w/3NvDqOTl2vKW/hRYkdkq+hv4uKL0cJRqIHVQtk
   Tg2aYw6Bn4ELBDqBszlKiw7b2ZVYlEVYqb/5G/Jxk/Qn5J182lFc49Iw/
   PANkind510sybwER+lSU/ZCJc1POlP2JSLlvAgJxSGE82KNJO0xREYCpg
   A=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: Beew7uxmj13Bf6E7BNydK8vV6TYcJqe4lFR7RgGL6M1GtVcvhQO1/klaiR+xBrIiW/OaB699zQ
 4Tpbx1mFQx4axiKrz4ijccBj3bCBmH2UWj5+5ErYZbj4J+GC2OmQZMsCccvC+G+PF6xdrkOpw7
 7IVpTFS8J3deaF9bfg7vA1a2HeOISzqTuBfn7uBeaesg2UbUkE0rSWm/qMzeECPjPBmhY7x0qE
 tFWw3outJTEQzSF5GBKhGC0UJb8HmPWG0f0moEFqEQpxcNVNl7/wivYUHpIpjcY5p30qxMrFYc
 mFUw8KV5KShNbMUSvXGgjRZa
X-SBRS: 5.1
X-MesageID: 52326854
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:dlFoSaHOfsfhSQjwpLqE6seALOsnbusQ8zAXP0AYc31om+ij5q
 eTdZUgpHvJYVkqNE3I9eruBEDEewK7yXcX2/h1AV7BZniEhILAFugLhuGO/9SjIVydygc079
 YYT0EUMr3N5DZB4/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,279,1624334400"; 
   d="scan'208";a="52326854"
Date: Thu, 9 Sep 2021 11:03:04 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, "Tim
 Deegan" <tim@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v7 05/51] x86/mm: avoid building multiple .o from a
 single .c file
Message-ID: <YTnb2J/49D5KaLeQ@perard>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
 <20210824105038.1257926-6-anthony.perard@citrix.com>
 <e633af0e-a34c-3c11-49ba-1e2c6f544613@suse.com>
 <YTibMaayiWlJwcZ/@perard>
 <27874b1a-70cb-e647-d271-93ef12dc40dc@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <27874b1a-70cb-e647-d271-93ef12dc40dc@suse.com>

On Wed, Sep 08, 2021 at 02:01:43PM +0200, Jan Beulich wrote:
> On 08.09.2021 13:14, Anthony PERARD wrote:
> > What kind of issue is there with those tiny source files?
> 
> To me they're ugly and their presence is at least mildly confusing.
> And apparently I'm not the only one thinking that way, or else such
> tiny stubs would have been put there right when introducing these
> multiply built sources.

Well, at the time when this was introduced, tools/symbols didn't exist,
so avoiding the stubs might have been ok by then, but it meant to
duplicated %.o:%.c rules in the Makefile...

Now, we have tools/symbols, a hack put into the source file to add a
".file instruction", a heuristic in tools/symbols to pick up the ".file"
we want, and a workaround to change in behavior of binutils.

This is a lot of complexity to avoid introducing those extra source
files... complexity that were needed to by added _after_ the initial
introduction of multiply built sources, due to changes in the build
system.

My patches have attempted to remove all the complexity, and having ugly
small source files is the price to pay. And you want to add back some
complexity in the build system just to avoid those tiny files? I mean
hypervisor and build system are complex software to write, do we really
need to add complexity at every opportunity?

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:03:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:03:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182934.330809 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGts-0002Vv-36; Thu, 09 Sep 2021 10:03:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182934.330809; Thu, 09 Sep 2021 10:03:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOGtr-0002Vk-W0; Thu, 09 Sep 2021 10:03:19 +0000
Received: by outflank-mailman (input) for mailman id 182934;
 Thu, 09 Sep 2021 10:03:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOGtq-0002Ul-6S
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:03:18 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 273245e4-1155-11ec-b1ab-12813bfff9fa;
 Thu, 09 Sep 2021 10:03:17 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1899XCo8002401; 
 Thu, 9 Sep 2021 10:03:14 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayex9gdh6-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 10:03:13 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3469.eurprd03.prod.outlook.com (2603:10a6:803:18::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Thu, 9 Sep
 2021 10:03:10 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 10:03:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 273245e4-1155-11ec-b1ab-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mBykgXf5IhV7uSHlBLbcLr5QKlHyR9BZMBRrFqW68Ho7LpddzoRA+7iU4MxjeN0K8B/OVcUL0d9XtLvMAikfx/l1sEEnCAZIP44LHDD5TGVvtghI1evBCopU4kfceG9qaTDjFzje874Mlaj9/fSRDQRXRdmnhU7QNhlCkQgOtN6sC6L5d5j5tyktqgCn/NeWHjV6SKIyPX2tuY4CdQB+kWJ9yu8UMXa3sQPGIRQiqxRM9ls5hGjJnyqN4LjLqWK8OcYR9N3VFxUWfwVTyFn2Q5s8zfu9zOVA5Ww495qdJ9KHPfOglF/wyDPq/7HSi7LS5H5EX4/qOt+KRZBzVClKUw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4OVP7vzYhLS51vaJuyi4faSWKkZbprdRsiFbSk3zpaI=;
 b=CIO2QyxCT0waOT0J7ZC9lVZxQ9v3m7DsQoLQAEpcufz8ozwdqGycj1V4RPezTuGxhqF0ul8EN2dhU4LcM6Ju1Ob9tz4DH2DdwyOsrwtsQwkOzgq99vdHUme4WVTseW85Nulr4O34MdvhYc3GQiGMLg+saomx5HYcRzS4i5IxfWs/mx8P1bj3Cky2lI2COaUjFMOHS8KHNXYrqqOsIkc8bsIUjy2Dmo9gtXDlxEtgEqxLNneP2Hir5vMi2jdISwFNi09DhA2beV5nJVZMiYLNfbWDrrhjYcC6ka/TQJs3nN/UBHOO0kJuS2aOGY2ntEXrI9oQUXEJdH8CdKR5i7GR0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4OVP7vzYhLS51vaJuyi4faSWKkZbprdRsiFbSk3zpaI=;
 b=Pln2odSvfyZn9fh0HoX5/EOLfYJ6Rnb59B3+smGL65K5OUgiUDVp2ZGgeudofcDTcoVvfYBYuGF49MN72knGC1J+Y8f4sPhlqdGMaHHdFJ1g6FlixPf0MFHBWArNvMfnvoEeEG1aQ8/ElSf8osVpAmPUQyLo9BvMyRaIKH3FcIZTxtMOCn5e69ieT2fR65evMJ0jEB7pqdlsiKCGfsNiiyAt4W3Y9D4VkVQ+ET0jb1AlYkpbZVIW1pNTgTw57MEI9Nr7ojxDkI7xjek2pM9Fpu4Ato76Rt9XBuaZOnJHEjdjgBfaWB8N9KEhsP+cHRbOBg7W67ezA2NRZYxhBfpBTQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Topic: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Index: 
 AQHXoKxjXfke7B20QEeo6YjlfUMhlquXGvYAgAMgDgCAAAg4gIAA8L+AgAAy0wCAAA1uAIAAB6sAgAAGiIA=
Date: Thu, 9 Sep 2021 10:03:10 +0000
Message-ID: <e154e4f7-bd60-2645-1ce5-1c2b5404236f@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
 <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
 <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
In-Reply-To: <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 89e91b7e-e527-4a59-b94d-08d97379077b
x-ms-traffictypediagnostic: VI1PR0302MB3469:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB3469CA5069A21C5FA1582DC0E7D59@VI1PR0302MB3469.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 3FV/1jdeuAB/6Lg8yvgVXfNj/Qqia7jDPR6X+XCf/oHnWpef91raucOMmzpBXNnULlbn1yu9mx9ZYcDkYxaeeJ1Zd9F/spG0hiyAz7Gzp8ZoYOCd+xRllUEUGedjfPfTHe8USRdiIat8HNnjB/QdOjlE4PXgpRhOe2rPU+gZwu4tnxXI8UqH1o6N3Ycr3rktH08B9BYLOa+fgpgPANbwFZSTNqResOP9z6KK5u3AuPMl6hy5eWzE2oVx0s3pxotyRYeL2yenk+d3LTTUTN7UDJhgayZgIt/Uf/VXTz03FZ/MAlktb6U5+dsragPHP3Nr5NN07RMZQKO+btAPBdPkbvuRNSvOEuPBZKVz9rX/wfkyDnrt1jRxlK9N5jdaPA0+e4nrgKkUKtGw3XBdvswM7ZIxL4YjAHSfS/loQEAqkS900jBYKOmC1I54Mhuq/rerjp8XyyTGSG+Y6A+xIkwRewDcfJvjlpQhdIgfDkQ86jJaGrNbTitfTAXqFYk3QJe3qB4xq/Jdf+YeznxTRxGhVCyTGONZ598QVv0fUzws6dDpW7Q/8t/hAGIGKMJ1HA6x+kJibSHlxcwaDXM0KPb2ibtX2MsmlrWzPktfUuUf292RKaYxsU7+X6j77hTVBZAWhB8X9SCnh4SzCCf8NcJddGH67EkZ4zAosRkdvPYn+WGUQVx35bng8WXB3nLNnWLRo3vkcptVQjg/+SDXpQ7GKVYEjWKcka15HY2rY3GMfnXGtO8bN6Js7kWl5HsJK7gv
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(366004)(376002)(136003)(346002)(54906003)(66476007)(66556008)(64756008)(8676002)(66946007)(38100700002)(8936002)(6486002)(5660300002)(4326008)(478600001)(2906002)(122000001)(186003)(2616005)(76116006)(316002)(31686004)(86362001)(71200400001)(6512007)(36756003)(31696002)(83380400001)(26005)(110136005)(38070700005)(66446008)(91956017)(6506007)(53546011)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UXllUnMvdW9NcnB6TjRndEZvUHZHdHF3ZzlzSXNyais1UHRzQU0zRmZTUUlM?=
 =?utf-8?B?UGlqelh3NnpncU5uUlBBRXR1cmVvYnlOQzlMdlFSdjg5MkhFZDZndUxQOHNz?=
 =?utf-8?B?SHpWMTlGeUp3MDZJalhLZDFRWVRlYUFncTZLZ3JGcnpnRndvOXJoL1ZSQTFE?=
 =?utf-8?B?K3FwbUcvckM2Qlk3VTZoVU52YU1uRlVBMjJKcjFJZFVCWWMvd0VHY2pnRmJC?=
 =?utf-8?B?dHNQS3dUSDlBdVNPY2FzdGYvTUZ6VDFjQWUyMURBcVU3VFZ0SlJ5RU1ONWEx?=
 =?utf-8?B?K2xYbjlYOXZ3dERlTXZBcjB6dGhwN1Yyci90dXpIaURadGU4RHlOa0VvaU1G?=
 =?utf-8?B?UGgvWVdVWm4rMXI0enRDTEx1NlNmNE9iSDgzVUpDZEVFSEwzVnVoZUxsQkh0?=
 =?utf-8?B?b0ZrK2trcEtEMGZZQ1N5WUdCZFU3OVc1b0dxdEVWZzNBYUhMWTZMemNIaEcw?=
 =?utf-8?B?L2xLbS84WXVib3g4b2Y2UHExR0xaY0tiOUhYTXgxY0FCQ2RENERJTlcrZ1FR?=
 =?utf-8?B?YXlBVlNhckRjUU03Z2JIUnRiWkh3QnBrcy9RQmtMbVU3Y08yQXU2eWJ3Y253?=
 =?utf-8?B?c0MrckpWVnJtYytaYnc0elQ0anhmSjdhN0FwZU5PNVl1ZUNzd3RCVk9vT0Ux?=
 =?utf-8?B?ck5zSHpzZnVybFpTL2FSUU9ob0FJTkJBeGs0bkVTOWw0azZnV0xaTlVLSWNn?=
 =?utf-8?B?c3NxdXA0bjBEUXIzVGJzY3BBSCtqendTeWlkQzlnS2pnR3NWTTIyK245Y3Vh?=
 =?utf-8?B?eVNXdjJzNFNNUzFXRUtuVjNxUmtIZ0QzVmpOc1pJUkc4OE9JVnd4TVk1Z2l2?=
 =?utf-8?B?NWczSHQySXVUVnI2eGFoK1dzWEo1bHpxZmxsNFU5U2JZYWFrdHNWbFA4N095?=
 =?utf-8?B?WnM3dWFhWFF1MU5qYWRhdzVadi9JYVk4UUtFcjE0blVLc2djZU96T2FGcFc5?=
 =?utf-8?B?OUhXUllnWkxGRXY0ZXNPOEhXZXdLc2wrVjk2SWo1dmlvTGFHNE1RUjBlSjEz?=
 =?utf-8?B?N2hCcVB3Y0VTRlRrYml6dW9jV0oxQjk2N0dNMytwemJjUjRRMTQrY1ZCbHF4?=
 =?utf-8?B?dVRYb2RJdThsTlphSVB0ckVqeXNQMUlIWmhtZmk3YlJwQnE4aGxtK1lSZnp6?=
 =?utf-8?B?NGhLU2VNa0NxZkI5RzFycUtoVkZLUVEwVFhoZ3hzSWhidXlEMU14MmlGekov?=
 =?utf-8?B?SGk3NVF4U2czYTdJQndFZE00dy9RVTNTYjJrK0pWSHRVaWhUaGpxSU42ZkFE?=
 =?utf-8?B?SlNaWFlXT2ZQNEpBWXdUclRTZm9vNUE4MTBnQXhGVjFLc1BtbWIyWXB3U2dt?=
 =?utf-8?B?RWJtQ0NTNGlUNGJweGU0NjVMQ2RGc1V5K1pWUXZnU2UwTDkyeXhoSHZxK0pr?=
 =?utf-8?B?Z2hVVVpSd1VMTDlmUC9jVDNiUklJSFR0dlB6TDY2Y09KSWhGSnFzdlE1ak9Z?=
 =?utf-8?B?MGRnWWh0UFZzOUFuR2VTTDg0WVNqQ1pFR09UaEpmcEFoNWhqbFBwa29YNEIz?=
 =?utf-8?B?VGVFRStmQXBOQ05tZnBQTXpiTVpQaHFhem5SbDVnWG01S3hnTjJ3VnBaa3FL?=
 =?utf-8?B?WlA4alJJTWIzSldLVWV2VnB5MEtRVG9USkVWcHFHOWQzUDg1b3hGdnNmTGw5?=
 =?utf-8?B?c0tmaWc0WlloWU1CT0xnM2RRUFQ1S09LWjRvTnpVc2h3ZVZtQ05pV1VNd2ZI?=
 =?utf-8?B?K1BpOTNBc2xTTlREMEE1NWdZdkpZdzBZUEF6Rmd6Rlg3dGVjNm5md0QzWE5Y?=
 =?utf-8?Q?3s4jIQaPHUbNqR07QxVfZXdDquNAaAGC//xRIWX?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <2A3B563C10C5E142B2357C285BB566B1@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89e91b7e-e527-4a59-b94d-08d97379077b
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 10:03:10.2127
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 04n0sFCwWov3jdzn5J9d1GyDPtgo4FSIERLC3YsRHQOwts6sctqasBgeJQqW5ZXKU/fCpQrTfgEGV+2vMR87nbdc2aOVdwkV3YD4qXJ8p3pChTZ4C0Ar6XkXjZmZLxPF
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3469
X-Proofpoint-GUID: vHwv61ZpRXIMYvEbbIc9VZBDHGVg9wDu
X-Proofpoint-ORIG-GUID: vHwv61ZpRXIMYvEbbIc9VZBDHGVg9wDu
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_03,2021-09-07_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxscore=0
 phishscore=0 bulkscore=0 spamscore=0 priorityscore=1501 adultscore=0
 clxscore=1015 lowpriorityscore=0 malwarescore=0 mlxlogscore=889
 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090060

DQpPbiAwOS4wOS4yMSAxMjozOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTE6MTIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gQW55d2F5cywgSSBhbSBv
cGVuIHRvIGFueSBkZWNpc2lvbiBvbiB3aGF0IHdvdWxkIGJlIHRoZSByaWdodCBhcHByb2FjaCBo
ZXJlOg0KPj4NCj4+IDEuIFVzZSByYW5nZSBzZXRzIHBlciBCQVIgYXMgaW4gdGhlIHBhdGNoDQo+
Pg0KPj4gMi4gUmVtb3ZlIHJhbmdlIHNldHMgY29tcGxldGVseSBhbmQgaGF2ZSBhIHBlci12Q1BV
IGxpc3Qgd2l0aCBtYXBwaW5nDQo+Pg0KPj4gZGF0YSBhcyBJIGRlc2NyaWJlZCBhYm92ZQ0KPj4N
Cj4+IDMuIEFueXRoaW5nIGVsc2U/DQo+IEEgZGVjaXNpb24gZmlyc3QgcmVxdWlyZXMgYSBwcm9w
b3NhbC4NCg0KSSBhbHJlYWR5IGhhdmUgMjogb25lIGluIHRoZSBwYXRjaCB3aXRoIHRoZSByYW5n
ZSBzZXQgcGVyIEJBUiBhbmQgb25lIGRlc2NyaWJlZA0KDQplYXJsaWVyIGluIHRoZSB0aHJlYWQg
d2l0aCBhIHNpbmdsZSByYW5nZSBzZXQgYW5kIGEgbGlzdCBmb3IgR0ZOIDwtPiBNRk4uDQoNCklm
IHlvdSBjYW4gdGVsbCB5b3VyIG9waW5pb24gSSBhbSBhbGwgZWFycy4gQnV0LCBwbGVhc2UgYmUg
c3BlY2lmaWMgYXMgY29tbW9uIHdvcmRzDQoNCmRvbid0IGNoYW5nZSBhbnl0aGluZyB0byBtZS4N
Cg0KQXQgdGhlIHNhbWUgdGltZSBJIGRvIHVuZGVyc3RhbmQgdGhhdCB0aGUgY3VycmVudCBjb2Rl
IGlzIG5vdCBzZXQgaW4gc3RvbmUsDQoNCmJ1dCB3ZSBzaG91bGQgaGF2ZSBhIGdvb2QgcmVhc29u
IGZvciBtYWpvciBjaGFuZ2VzIHRvIGl0LCBJTU8uIEkgbWVhbiB0aGF0IGJlZm9yZQ0KDQpEb21V
J3Mgd2Ugd2VyZSBmaW5lIHdpdGggdGhlIHJhbmdlIHNldHMgZXRjLCBhbmQgbm93IHdlIGFyZSBu
b3Q6DQoNCnNvIHdoYXQgaGFzIGNoYW5nZWQgc28gbXVjaD8NCg0KPiAgIEkgdGhpbmsgMyBpcyB0
aGUgd2F5IHRvIGludmVzdGlnYXRlDQo+IGZpcnN0OiBSYXRoZXIgdGhhbiBzdGFydGluZyBmcm9t
IHRoZSBjb2RlIHdlIGN1cnJlbnRseSBoYXZlLCBzdGFydCBmcm9tDQo+IHdoYXQgeW91IG5lZWQg
Zm9yIERvbVUtcyB0byB3b3JrLiBJZiB0aGVyZSdzIGVub3VnaCBvdmVybGFwIHdpdGggaG93IHdl
DQo+IGhhbmRsZSBEb20wLCBjb2RlIGNhbiBiZSBzaGFyZWQuDQoNCllvdSBjYW4gc2VlIHRoYXQg
aW4gbXkgcGF0Y2ggdGhlIHNhbWUgY29kZSBpcyB1c2VkIGJ5IGJvdGggaHdkb20gYW5kDQoNCmd1
ZXN0LiBXaGF0IGVsc2UgbmVlZHMgdG8gYmUgcHJvdmVuPyBUaGUgcGF0Y2ggc2hvd3MgdGhhdCBh
bGwgdGhlIGNvZGUNCg0KYmVzaWRlcyBndWVzdCByZWdpc3RlciBoYW5kbGVycyAod2hpY2ggaXMg
ZXhwZWN0ZWQpIGlzIGFsbCBjb21tb24uDQoNCj4gICBJZiB0aGluZ3MgYXJlIHN1ZmZpY2llbnRs
eSBkaWZmZXJlbnQsDQo+IHNlcGFyYXRlIGNvZGUgcGF0aHMgYXJlIGxpa2VseSBiZXR0ZXIuIEFz
IHNhaWQgLSB0byBtZSBhIGd1ZXN0IGFsdGVyaW5nIGENCj4gQkFSIGlzIG1lcmVseSBhIHZlcnkg
c3BlY2lhbCBmb3JtIG9mIGEgcmVxdWVzdCB0byBjaGFuZ2UgaXRzIFAyTS4gVGhlIE0NCj4gcGFy
dHMgcmVtYWlucyB1bmNoYW5nZWQgKHdoaWNoIGlzIHRoZSBtYWpvciBkaWZmZXJlbmNlIGZyb20g
RG9tMCksIHdoaWxlDQo+IHRoZSBQIHBhcnQgY2hhbmdlcy4gQXMgbG9uZyBhcyB5b3UgY2FuIGFz
c3VtZSBubyB0d28gQkFScyB0byBzaGFyZSBhIHBhZ2UsDQo+IHRoaXMgd291bGQgYXBwZWFyIHRv
IHN1Z2dlc3QgdGhhdCBpdCdzIHNpbXBseSBhIFAyTSBvcGVyYXRpb24gcGx1cyBib29rDQo+IGtl
ZXBpbmcgYXQgdGhlIHZQQ0kgbGF5ZXIuIENvbXBsZXRlbHkgZGlmZmVyZW50IGZyb20gRG9tMCBo
YW5kbGluZy4NCg0KVW5kZXJuZWF0aCwgeWVzLCBwb3NzaWJseS4gQnV0IGF0IHRoZSBsZXZlbCB2
UENJIG9wZXJhdGVzIHRoZXJlIGlzIG5vDQoNCnN1Y2ggZGlmZmVyZW5jZSBJIGNhbiBjbGVhcmx5
IHNlZSBpbiB2UENJIGNvZGUgYW5kIHRoZSBwYXRjaCBpbiBxdWVzdGlvbi4NCg0KUGxlYXNlIHBv
aW50IG1lIHRvIHRoZSB2UENJIGNvZGUgSSBmYWlsIHRvIHNlZS4NCg0KPg0KPiBBbGwgb2YgdGhp
cyBhcHBsaWVzIG9ubHkgd2l0aCBtZW1vcnkgZGVjb2RpbmcgZW5hYmxlZCwgSSBleHBlY3QuDQo+
IERpc2FibGluZyBtZW1vcnkgZGVjb2Rpbmcgb24gYSBkZXZpY2Ugb3VnaHQgdG8gYmUgYSBzaW1w
bGUgInVubWFwIGFsbA0KPiBCQVJzIiwgd2hpbGUgZW5hYmxpbmcgaXMgIm1hcCBhbGwgQkFScyIu
IFdoaWNoIGFnYWluIGlzLCBkdWUgdG8gdGhlDQo+IGFzc3VtZWQgbGFjayBvZiBzaGFyaW5nIG9m
IHBhZ2VzLCBtdWNoIHNpbXBsZXIgdGhhbiBvbiBEb20wOiBZb3Ugb25seQ0KPiBuZWVkIHRvIHN1
YnRyYWN0IHRoZSBNU0ktWCB0YWJsZSByYW5nZShzKSAoaWYgYW55LCBhbmQgcGVyaGFwcyBub3QN
Cj4gbmVjZXNzYXJ5IHdoZW4gdW5tYXBwaW5nLCBhcyB0aGVyZSdzIG5vdGhpbmcgd3JvbmcgdG8g
dW5tYXAgYSBQMk0gc2xvdA0KPiB3aGljaCB3YXNuJ3QgbWFwcGVkKTsgdGhpcyBtYXkgbm90IGV2
ZW4gcmVxdWlyZSBhbnkgcmFuZ2VzZXQgYXQgYWxsIHRvDQo+IHJlcHJlc2VudC4NCj4NCj4gQW5k
IGluIGZhY3QgSSB3b25kZXIgd2hldGhlciBmb3IgRG9tVS1zIHlvdSB3YW50IHRvIHN1cHBvcnQg
QkFSIGNoYW5nZXMNCj4gaW4gdGhlIGZpcnN0IHBsYWNlIHdoaWxlIG1lbW9yeSBkZWNvZGluZyBp
cyBlbmFibGVkLg0KDQpObywgd2h5PyBJIHdhbnQgdG8ga2VlcCB0aGUgZXhpc3RpbmcgbG9naWMs
IGUuZy4gd2l0aCBtZW1vcnkgZGVjb2RpbmcNCg0KZGlzYWJsZWQgYXMgaXQgaXMgbm93Lg0KDQo+
ICAgRGVwZW5kcyBtdWNoIG9uDQo+IGhvdyBxdWlya3kgdGhlIGd1ZXN0IE9TZXMgYXJlIHRoYXQg
b3VnaHQgdG8gcnVuIG9uIHRvcC4NCj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5k
cg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:13:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:13:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182948.330820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOH3E-0004bi-2t; Thu, 09 Sep 2021 10:13:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182948.330820; Thu, 09 Sep 2021 10:13:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOH3D-0004bb-VG; Thu, 09 Sep 2021 10:12:59 +0000
Received: by outflank-mailman (input) for mailman id 182948;
 Thu, 09 Sep 2021 10:12:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOH3D-0004bV-EE
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:12:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOH3D-00035D-9Z
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:12:59 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOH3D-0003yU-8M
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:12:59 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOH39-00063F-Ku; Thu, 09 Sep 2021 11:12:55 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=BTdbwlBdvgKBjs9z5IbKaBeIwb8EDVvgQhr+rx+gmhI=; b=yuI11NUbofmgb+1HjaBMum0Tzg
	R3cB397K761kT+SKLBcVP2LBaYbvyb42KMTNWDhN3/v4JE7SRcwpSXkKo0qoa8ymdGbq6mnMMy9pS
	HnASf8XwmHXfRoQXwMDKgPpmZvZxDcCHkqMNynUQpPrPgJlB9s62G9BIUsa9OUy8npWY=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24889.56871.267994.502191@mariner.uk.xensource.com>
Date: Thu, 9 Sep 2021 11:12:55 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org,
Subject: Re: [xen-4.13-testing bisection] complete build-amd64
In-Reply-To: <c3664ec4-dcc3-f7ed-9701-f08d291efd37@suse.com>
References: <E1mOGjd-0004AY-0t@osstest.test-lab.xenproject.org>
	<c3664ec4-dcc3-f7ed-9701-f08d291efd37@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [xen-4.13-testing bisection] complete build-amd64"):
> On 09.09.2021 11:52, osstest service owner wrote:
> >   commit 9f3eda177a4b2d4a33ff1b0307cad42906396562
> >   Author: Lin, Gary (HPS OE-Linux) <gary.lin@hpe.com>
> >   Date:   Tue Aug 31 09:29:48 2021 +0800
> >   
> >       OvmfPkg/OvmfXen: add QemuKernelLoaderFsDxe
> >       
> >       Without QemuKernelLoaderFsDxe, QemuLoadKernelImage() couldn't download
> >       the kernel, initrd, and kernel command line from QEMU's fw_cfg.
> >       
> >       Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3574
> >       Signed-off-by: Gary Lin <gary.lin@hpe.com>
> >       Acked-by: Anthony PERARD <anthony.perard@citrix.com>
> >       Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
> >       Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
> >       Tested-by: Jim Fehlig <jfehlig@suse.com>
> 
> I'm really curious as to how this could have been identified as the
> culprit, when the build issue was a plain hypervisor side one (which
> Andrew did supply a fix for yesterday).

It's a bisector.  It doesn't look at error messages.  So it can't tell
whether a build failure is "the same" as another build failure.

What it starts out knowing is this:


  --good---------------------------------------------------------bad
   (long ago)                                                   [gfn_t]

It picks some random spot ~halfway[1]

  --good-------------------------bad-----------------------------bad
   (long ago)                   [ovmf]                          [gfn_t]

And then of course it's bisecting the ovmf build failure.

Or to put it another way, the build has been broken since the ovmf
change broke it.

[1] It has actually identified the change in the ovmf tree.  For the
purposes of bisection it bisects over a constructed graph whose nodes
are tuples of versions of all the relevant tree.

Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:28:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:28:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182954.330830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHIW-0006O4-E9; Thu, 09 Sep 2021 10:28:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182954.330830; Thu, 09 Sep 2021 10:28:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHIW-0006Nx-BI; Thu, 09 Sep 2021 10:28:48 +0000
Received: by outflank-mailman (input) for mailman id 182954;
 Thu, 09 Sep 2021 10:28:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sph1=N7=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mOHIV-0006Np-9W
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:28:47 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 1058e31a-77f1-4299-a739-163ad18ce41e;
 Thu, 09 Sep 2021 10:28:46 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-336-TWYTzTWWOT6YTQ70pewE1g-1; Thu, 09 Sep 2021 06:28:36 -0400
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
 [10.5.11.11])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 23F751006AAA;
 Thu,  9 Sep 2021 10:28:35 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.19])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 8F2815B826;
 Thu,  9 Sep 2021 10:28:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1058e31a-77f1-4299-a739-163ad18ce41e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631183326;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Cc/jxy7+//DCARbQzXLT+htA7O/y961unhjav93Y0yg=;
	b=g8aD+Ta1higNz1Hklrk6oTe9F9LIckfGONwOZlBpyP043mZZWtktHqBWjeARtsXoiPKEgA
	Ci5E1/o9M4AMxFIKBe187Vo74BMW4NKJTYmqrUN604K2rVa00t2RarstZOcvHib49SWkcS
	samzV80/KbyIgwjwLuj7S5PTt/LK+UQ=
X-MC-Unique: TWYTzTWWOT6YTQ70pewE1g-1
Date: Thu, 9 Sep 2021 11:28:27 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 00/10] security: Introduce
 qemu_security_policy_taint() API
Message-ID: <YTnhy/Wo5M1qVs1k@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:14AM +0200, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series is experimental! The goal is to better limit the
> boundary of what code is considerated security critical, and
> what is less critical (but still important!).
> 
> This approach was quickly discussed few months ago with Markus
> then Daniel. Instead of classifying the code on a file path
> basis (see [1]), we insert (runtime) hints into the code
> (which survive code movement). Offending unsafe code can
> taint the global security policy. By default this policy
> is 'none': the current behavior. It can be changed on the
> command line to 'warn' to display warnings, and to 'strict'
> to prohibit QEMU running with a tainted policy.

Ok, so I infer that you based this idea on the "--compat-policy"
arg used to control behaviour wrt to deprecations.

With the deprecation support, the QAPI introspection data can
report deprecations for machines / CPUs (and in theory devices
later).  Libvirt records this deprecation info and can report
it to the user before even starting a guest, so they can avoid
using a deprecated device in the first place.  We also use this
info to mark a guest as tainted + deprecation at the libvirt
level and let mgmt apps query this status.

The --compat-policy support has been integrated into libvirt
but it is not something we expect real world deployments to
use - rather it is targeted as a testing framework.

Essentially I see the security reporting as needing to operate
in a pretty similar manner.

IOW, the reporting via QAPI introspetion is much more important
for libvirt and mgmt apps, than any custom cli arg / printfs
at the QEMU level.


In terms of QAPI design we currently have

   'deprecated': 'bool'

field against MachineInfo and CpuDefinitionInfo types.

it feels like we need

   'secure': 'bool'

field against the relevant types here too, though I think
maybe we might need to make it an optional field  to let
us distinguish lack of information, since it will take a
long time to annotate all areas. eg

   '*secure': 'bool'

 - not set  => no info available on security characteristics
 - true => device is considered secure wrt malicious guest
 - false => device is not considered secure wrt malicious guest


> As examples I started implementing unsafe code taint from
> 3 different pieces of code:
> - accelerators (KVM and Xen in allow-list)
> - block drivers (vvfat and parcial null-co in deny-list)
> - qdev (hobbyist devices regularly hit by fuzzer)
> 
> I don't want the security researchers to not fuzz QEMU unsafe
> areas, but I'd like to make it clearer what the community
> priority is (currently 47 opened issues on [3]).

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:37:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:37:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182961.330842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHQu-000860-A6; Thu, 09 Sep 2021 10:37:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182961.330842; Thu, 09 Sep 2021 10:37:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHQu-00085t-6u; Thu, 09 Sep 2021 10:37:28 +0000
Received: by outflank-mailman (input) for mailman id 182961;
 Thu, 09 Sep 2021 10:37:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sph1=N7=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mOHQs-00085n-Qx
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:37:26 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 54efee11-f86f-42c1-a52a-0c8c1828ad38;
 Thu, 09 Sep 2021 10:37:25 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-32-nHnegXWZNDKFzlbTRBMcJQ-1; Thu, 09 Sep 2021 06:37:23 -0400
Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com
 [10.5.11.22])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 559E61B18BC5;
 Thu,  9 Sep 2021 10:37:22 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.19])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 7475C1002388;
 Thu,  9 Sep 2021 10:37:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54efee11-f86f-42c1-a52a-0c8c1828ad38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631183845;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mFpyUFg5iicRG4N67bToPFSscnmzl25Wk4DeTDhXjsA=;
	b=h1mPFuMPK1mzZjmM0OkBnuIsaGTTDHYFSbkIp1YHGVFkZESzeUJXUHGxcYv+ivs3OHONsl
	hZaBc7uiFpAI6Dx/8A5/ExDJ7ExYClHHNyowRyq5IVNaXCue7u2tQmifjsbY4wfUUJZJeS
	afZp1iknuBu55HhFH1JKwo0aO+W1OB4=
X-MC-Unique: nHnegXWZNDKFzlbTRBMcJQ-1
Date: Thu, 9 Sep 2021 11:37:12 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 02/10] accel: Use qemu_security_policy_taint(), mark
 KVM and Xen as safe
Message-ID: <YTnj2M+lygKzdsgO@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-3-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-3-philmd@redhat.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:16AM +0200, Philippe Mathieu-Daudé wrote:
> Add the AccelClass::secure_policy_supported field to classify
> safe (within security boundary) vs unsafe accelerators.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/qemu/accel.h | 5 +++++
>  accel/kvm/kvm-all.c  | 1 +
>  accel/xen/xen-all.c  | 1 +
>  softmmu/vl.c         | 3 +++
>  4 files changed, 10 insertions(+)
> 
> diff --git a/include/qemu/accel.h b/include/qemu/accel.h
> index 4f4c283f6fc..895e30be0de 100644
> --- a/include/qemu/accel.h
> +++ b/include/qemu/accel.h
> @@ -44,6 +44,11 @@ typedef struct AccelClass {
>                         hwaddr start_addr, hwaddr size);
>  #endif
>      bool *allowed;
> +    /*
> +     * Whether the accelerator is withing QEMU security policy boundary.
> +     * See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool secure_policy_supported;

The security handling policy is a high level concept that is
open to variation over time and also by downstream distro
vendors.

At a code level we should be dealing in a more fundamental
concept. At an accelerator level we should really jsut
declare whether or not the accelerator impl is considered
to be secure against malicious guest code.

eg

    /* Whether this accelerator is secure against execution
     * of malciious guest machine code */
    bool secure;


>      /*
>       * Array of global properties that would be applied when specific
>       * accelerator is chosen. It works like MachineClass.compat_props
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 0125c17edb8..eb6b9e44df2 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -3623,6 +3623,7 @@ static void kvm_accel_class_init(ObjectClass *oc, void *data)
>      ac->init_machine = kvm_init;
>      ac->has_memory = kvm_accel_has_memory;
>      ac->allowed = &kvm_allowed;
> +    ac->secure_policy_supported = true;
>  
>      object_class_property_add(oc, "kernel-irqchip", "on|off|split",
>          NULL, kvm_set_kernel_irqchip,
> diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
> index 69aa7d018b2..57867af5faf 100644
> --- a/accel/xen/xen-all.c
> +++ b/accel/xen/xen-all.c
> @@ -198,6 +198,7 @@ static void xen_accel_class_init(ObjectClass *oc, void *data)
>      ac->setup_post = xen_setup_post;
>      ac->allowed = &xen_allowed;
>      ac->compat_props = g_ptr_array_new();
> +    ac->secure_policy_supported = true;
>  
>      compat_props_add(ac->compat_props, compat, G_N_ELEMENTS(compat));
>  
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 92c05ac97ee..e4f94e159c3 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -2388,6 +2388,9 @@ static int do_configure_accelerator(void *opaque, QemuOpts *opts, Error **errp)
>          return 0;
>      }
>  
> +    qemu_security_policy_taint(!ac->secure_policy_supported,
> +                               "%s accelerator", acc);

We need this information to be introspectable, becuase stuff printed
to stderr is essentially opaque to libvirt and mgmt apps above.

We don't have a convenient "query-accel" command but I think this
could possibly fit into 'query-target'. ie the TargetInfo struct
gain a field:
 

  ##
  # @TargetInfo:
  #
  # Information describing the QEMU target.
  #
  # @arch: the target architecture
  # @secure: Whether the currently active accelerator for this target
  #          is secure against execution of malicous guest code
  #
  # Since: 1.2
  ##
  { 'struct': 'TargetInfo',
    'data': { 'arch': 'SysEmuTarget',
              'secure': 'bool'} }

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:40:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:40:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182968.330853 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHTh-00013K-Qy; Thu, 09 Sep 2021 10:40:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182968.330853; Thu, 09 Sep 2021 10:40:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHTh-00013D-Nr; Thu, 09 Sep 2021 10:40:21 +0000
Received: by outflank-mailman (input) for mailman id 182968;
 Thu, 09 Sep 2021 10:40:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sph1=N7=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mOHTg-000135-13
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:40:20 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 528a7fe3-115a-11ec-b1ac-12813bfff9fa;
 Thu, 09 Sep 2021 10:40:18 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-409-P32uXViMPg6xrcmBrVASwA-1; Thu, 09 Sep 2021 06:40:17 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
 [10.5.11.12])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4F11D802929;
 Thu,  9 Sep 2021 10:40:16 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.19])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 9C0DC61156;
 Thu,  9 Sep 2021 10:40:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 528a7fe3-115a-11ec-b1ac-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631184018;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oL33Aa6CtkK/7ZX9sUngGFwqhn1cvVQ/PE0/Nntta0Q=;
	b=CBBpD3RPJbz64EG4V2nqxguOM82RQYoUfSpmWVhJHkt7Ix7tFzcsd8zj9KUyEp2Lreuun5
	iERGQunj15NpwKQsUsoH6oL4YbFhccp/wK9zp7y1+QAZCa3PVbVcZnmp8FUZG7LzqEo1nC
	2A6AbWLly2/W9JPwAA0JF7IExBWCjpE=
X-MC-Unique: P32uXViMPg6xrcmBrVASwA-1
Date: Thu, 9 Sep 2021 11:40:07 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 03/10] block: Use qemu_security_policy_taint() API
Message-ID: <YTnkhxWbm3NvGo/T@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-4-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-4-philmd@redhat.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:17AM +0200, Philippe Mathieu-Daudé wrote:
> Add the BlockDriver::bdrv_taints_security_policy() handler.
> Drivers implementing it might taint the global QEMU security
> policy.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/block/block_int.h | 6 +++++-
>  block.c                   | 6 ++++++
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index f1a54db0f8c..0ec0a5c06e9 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -169,7 +169,11 @@ struct BlockDriver {
>      int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
>                            Error **errp);
>      void (*bdrv_close)(BlockDriverState *bs);
> -
> +    /*
> +     * Return %true if the driver is withing QEMU security policy boundary,
> +     * %false otherwise. See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool (*bdrv_taints_security_policy)(BlockDriverState *bs);
>  
>      int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
>                                         Error **errp);
> diff --git a/block.c b/block.c
> index b2b66263f9a..696ba486001 100644
> --- a/block.c
> +++ b/block.c
> @@ -49,6 +49,7 @@
>  #include "qemu/timer.h"
>  #include "qemu/cutils.h"
>  #include "qemu/id.h"
> +#include "qemu-common.h"
>  #include "block/coroutines.h"
>  
>  #ifdef CONFIG_BSD
> @@ -1587,6 +1588,11 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv,
>          }
>      }
>  
> +    if (drv->bdrv_taints_security_policy) {
> +        qemu_security_policy_taint(drv->bdrv_taints_security_policy(bs),
> +                                   "Block protocol '%s'", drv->format_name);
> +    }
> +
>      return 0;
>  open_failed:
>      bs->drv = NULL;

Again we need a way to report this via QAPI, but we don't have a natural
place is hang this off for introspection before starting a guest.

The best we can do is report the information after a block backend has
been instantiated. eg  Modify "BlockInfo" struct to gain

   '*secure': 'bool'

Note I made this an optional field, since unless we mark every single
block driver impl straight away, we'll need to cope with the absence
of information.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:46:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:46:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182975.330863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHZu-000211-I1; Thu, 09 Sep 2021 10:46:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182975.330863; Thu, 09 Sep 2021 10:46:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHZu-00020u-F6; Thu, 09 Sep 2021 10:46:46 +0000
Received: by outflank-mailman (input) for mailman id 182975;
 Thu, 09 Sep 2021 10:46:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOHZt-00020o-Fp
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:46:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 38b91a94-115b-11ec-b1ac-12813bfff9fa;
 Thu, 09 Sep 2021 10:46:44 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2053.outbound.protection.outlook.com [104.47.0.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-4NOy8kZ6O7uNGPQyT0jnrw-1; Thu, 09 Sep 2021 12:46:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 10:46:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 10:46:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0043.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Thu, 9 Sep 2021 10:46:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38b91a94-115b-11ec-b1ac-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631184403;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=k7Bg3vCyvgNq5aTvh4vWrEklP+4G2UUCw/LFvFruh60=;
	b=U2un77uHAgwZfD3uOr2ISI3T/fNJrkOtdPQdwrqshuaCD+3SBGaYYF6GrwpVkhHNB9aR7u
	NBQuWQmSERFZTlu0A9VAjThN/GfK96665EUJnJGXIo0jnilAypt4RBMZEty+w17omgv+SH
	ViGzw5CUM6xW96u1FKZRmkHutQtrkFY=
X-MC-Unique: 4NOy8kZ6O7uNGPQyT0jnrw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=htL6uxm7y4QqH0mttZRUTldqQVo28BjihUn9+r1QMZTfyHctBD7PJeVszH/F1q/RoTOoJX4U7Ed7WUBnxS8lVXauoGXq4Qi1gwES1UuSd7bOJSq9Ai6Nrj6c/RXxMQCa6lRrzRHK3ThjTT7E1D4aoBnVL8hM/yN4khij5oinGlAj6zgnl1wShpRpz6lMw/zM1ccLVtMMfLjZM5PK7OzIiu1pJdj7N2ix1EpUu8fd0M3WIMIwfdt9jSSXQ5xljor00Kg88fMjB/39hWqjGPBT8/ZK9LeWGh+YoKvVnB26Ylqf+/lkw0zRVXz3ut/IKIcotlfXQaKttLjrLEmEsmD0FQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=k7Bg3vCyvgNq5aTvh4vWrEklP+4G2UUCw/LFvFruh60=;
 b=NUwp+8SCNkoPUJTg1pZUJX4InU9LzHklzH116FQR7Zzg3hoBGMcH2ycvoVXm8cuasfLDKdsovdXSpKCnYvM2X7ggEq0gX7Nau7X9hsL80+fIScn+g5T3bTLWDjoP6aTrwUXdiU7wc1EcL3oyYmIzL9gmKH2QyxPcivyTtzAKXoT3g9ZXUluVePdLIVgSj9N8q6GRwv2/DRxad+BPHxSUEqkDE/Xti8LkfaziPie+xw/KRfMdX0MMQWU1JAP6euWJ+NHpCWGFRdR6L287TK53flbymuH73A6dSfyG2H6jdzADh7/lEYd3BMi/riAhGFSba+mQCK/C3IeGXjRaE8TTrg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
 <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
 <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
 <e154e4f7-bd60-2645-1ce5-1c2b5404236f@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3e7dc9f3-1536-0643-b732-4a203bee34d4@suse.com>
Date: Thu, 9 Sep 2021 12:46:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e154e4f7-bd60-2645-1ce5-1c2b5404236f@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0043.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1efae6b8-6a56-48c1-f482-08d9737f1a7a
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6176DBC542709042C7194A01B3D59@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	r0SHdz8f27u6menU2hsGB0VV0twuKdAh1g6wUqP4zePxw+/WNpKrt3RdzzfeTz/J3NMuhkNY7h+I9XDnAo/l6m6iYsdGJ/ILtxwtDGdKLZzsdXCrCNjnmQI5ZqHqhsw2LPDX2rAuXvVbRsp9jiBiApEwn7HBLSG41qzGaYEbrNAiF0Bjs6bAqTc66CsITRfJebgmjTngQHZLVUfZCBo62NG1/A42Ene5KuOCH2On8yLJ9Mmrbsv2uUIkMbTlAtlaPHjFRbbm4qZITYZqpGRFCJM3HgVNPQIr9ByL70pNX+BabbbMpSwM8fWBmAdV+oFu9aV9qyjb9jFgiqfKGcJOOOZIOHfz71EP3S9RvOR1vs3XM0N+21TIcrjNJKQMkS3sjSi6u89LjUDfSJZOkudPSXFWFd8yOrJjcV1hmtZupVOvUQ01LRi0ftF7K6Vh9cgQRX33XzATGOL8TX2VSazYbGUdjYE1BaJcLPgZcIvQfeV9xit+/78UbE67AJwxZkHRV1fzIbKg27+x4JkvVyxT/eJ69UWF+8/7vIWVrOFtc4r0rywdau3r/INj/fg5v+bQPA8s9QvCBMt4ltu/FnIdipZihPPkhh78CqODqHxLEotQQxEKbymgIUr4uoUYhk0dR7xbLxrEhwSqWicr1Dm3NSxqSZ1XMB46ZqtQopXcj0S7rEioHMk0aVKfUvTjb61b8LB8Wk57gg/dtrT5q3IpKdTojcJWvIenTj8JrX9uPfb+eTJiWC8/PuFVTZ8TKqOT
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(376002)(366004)(346002)(136003)(110136005)(54906003)(83380400001)(316002)(16576012)(36756003)(86362001)(31696002)(7416002)(31686004)(38100700002)(4326008)(53546011)(8936002)(186003)(26005)(5660300002)(8676002)(66476007)(66556008)(956004)(66946007)(2616005)(2906002)(6486002)(478600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Q3FWbGh6azBhWFpoWTRVVFJnOWZVK2NiTzhiM3VqTzVKZEVzZllueFZPaGFP?=
 =?utf-8?B?dGZHUzZ0Rk93MU9WZEV6SVNJeUJaQ2g4Z2c0NFNnV2VrQU9mUHE1elJIRGtF?=
 =?utf-8?B?LzE5MzRQMFpoa3Y4bVluSEQ2ZGV2bDhnRDFHQ1lSdmNrSGJ4dTlsMEpkREd3?=
 =?utf-8?B?V2F0N01yR3Z0SHl2OTZtVit2YUxzSTBxcWg2enpKajhzOHJRQkJSVjJDTnp1?=
 =?utf-8?B?UUwvWmhCZ1Y0THpwL3VUeE52djAxL0tsTnlIdXRRak5WMXQxbjBpYWF0Q2Nl?=
 =?utf-8?B?WDNaYXJMZlI1S3VkcXNDMnZtc0VNaEFjSHBLR29MM0FVazVrU1JDM1R0MU1m?=
 =?utf-8?B?NERhNXUxN3IzRXpYZnBLKy8yTDNmbFRFUE4rdjVkTkw2SzFDSVE5ZkVJNnJ6?=
 =?utf-8?B?aGhuUXlrUm1ENmtUcng1aS9iR25hSHdzLys1Q01IdDJ5K0lydUV5MGhsUkZ2?=
 =?utf-8?B?U0tZZENyN3JkL3ZacExXdnczdEd6Y0d2SjM4eGQydlNrTlp2YTkzd3NTYnBD?=
 =?utf-8?B?eDJLV1VqTXlINWVHbVd0STExbithelgzZ0pmMmtwRE1DZE1YV0NjdU9mQmpT?=
 =?utf-8?B?b1Vrb0tFVVV5RzZjcGVjYXhWdlp3YW1tdXIxOFhwUXhLUHRpSk03eGFVNms5?=
 =?utf-8?B?aG9iUldzN1VWdkFtODFYaWNHaWVmUURid3h3OS9uMkRuNmM4MFFobVordVp1?=
 =?utf-8?B?UWs2YUVVZWY4SGZRbk14K0hreHhwWjRpL0ZXYTBqRXdycnp5Y2tBYXdMUXZw?=
 =?utf-8?B?cENsSUF4Tzd5OXYxSUtKL1I2b0x1cXRPK1RZTG1vTWt0Z1ZsREJhRTZISmhF?=
 =?utf-8?B?NnM2Z0xCaUl2TE83TFdpVDRxWHdBZ3AzSGVkclFNTGRMUkJKb2xnU21NeC9l?=
 =?utf-8?B?TlFPTnY5eXQwdXNFVjNoL1pQaTlqM01ZSEd4NFQrU0lFZ3A3bjZsQWdDVUNG?=
 =?utf-8?B?UGkyUFNxcjNpNkdaVXZEOUVvOUZhSkF2OU5qVU4yaER6UnlMaTJUUWxQWktt?=
 =?utf-8?B?cWdtbDFyTFVETnpoWmtEWDVaOHcwMW1FSlUzSE4rcFZkRnRLS3dWSm5ReHhM?=
 =?utf-8?B?dm9VWm1BdDQ4R2p2eUlDMEtpVHhxbTBOOVZFOHphNGk2SkNja2RiNmZsZ2R6?=
 =?utf-8?B?SCt4aTNXZmlXQWE1aGtsdUhraG1wZWZCNjloZFdDYUQ0Um04QUJRL2FvY2Ix?=
 =?utf-8?B?dzE0N29XTlRQNWhvZVhZellqcU5KZjE2OWg2dERFaFBlV3pYT2pzMVBVNDFq?=
 =?utf-8?B?a1J0QlZmY1d5STFDck16WTVlVy9XVWJQYy9hM0ltRTE5K3kyTUc1eHN1ZWhj?=
 =?utf-8?B?K3hHSk5zSkZYampocGk3aWJ2cERzc2RZbFJKVDlCZ1JlSzBjeC9VN2VPVWx6?=
 =?utf-8?B?Vm5xMGw0bWk3NlhvTDJ6dk5BZFIxSjdVWjEvdlA5VEJOUXZQS2RMZGF6MkJT?=
 =?utf-8?B?L2h2enVGME0vUkJBMW5FMHByYzdjdkJVU0RTK2hHMG5SanZvMVoyaW5QZEd1?=
 =?utf-8?B?V1d6WEZMWmlDc2hRNWhkMlhYejF0OTJweXB1bG5rMlJpVCtudStoSFVCaEpy?=
 =?utf-8?B?TzNHaUlEWlByaENyTExjK1lPbTFsVlBkNTJ3SVR3ckQ2MG1qZXIwRWxrVVY3?=
 =?utf-8?B?YnJpWlJiYmtoblhsME9MTEFVSyt5cC9oOVdPVllTQ0tkUld4Rm94VnZ2dk9M?=
 =?utf-8?B?NHpBdHhFSE4wQmhXVDR5eEFLQ3Iyc3d4emlsbEx3eFVYN1BJcStFTW1TckQ5?=
 =?utf-8?Q?xlPdp9bbNTwVAg/K1d4QDhqOTuDdPd0zJXGlEY1?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1efae6b8-6a56-48c1-f482-08d9737f1a7a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 10:46:39.3201
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 25Dm1XtoXRBkCENiNIxoXqcJ7JwzrEA50gniVr9X19WT3/FqZF8UZ4Zbu/Ul7iMkEJIrPxjqscAI4QvYqqntyg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

On 09.09.2021 12:03, Oleksandr Andrushchenko wrote:
> On 09.09.21 12:39, Jan Beulich wrote:
>> On 09.09.2021 11:12, Oleksandr Andrushchenko wrote:
>>> Anyways, I am open to any decision on what would be the right approach here:
>>>
>>> 1. Use range sets per BAR as in the patch
>>>
>>> 2. Remove range sets completely and have a per-vCPU list with mapping
>>>
>>> data as I described above
>>>
>>> 3. Anything else?
>> A decision first requires a proposal.
> 
> I already have 2: one in the patch with the range set per BAR and one described
> earlier in the thread with a single range set and a list for GFN <-> MFN.
> If you can tell your opinion I am all ears. But, please be specific as common words
> don't change anything to me.
> At the same time I do understand that the current code is not set in stone,
> but we should have a good reason for major changes to it, IMO.

And I view your change, as proposed, as a major one. You turn the logic all
over imo.

> I mean that before
> DomU's we were fine with the range sets etc, and now we are not:
> so what has changed so much?

Nothing has changed. I'm not advocating for removal of the rangeset use in
handling Dom0's needs. I'm suggesting that their use might not be a good
fit for DomU.

>>   I think 3 is the way to investigate
>> first: Rather than starting from the code we currently have, start from
>> what you need for DomU-s to work. If there's enough overlap with how we
>> handle Dom0, code can be shared.
> 
> You can see that in my patch the same code is used by both hwdom and
> guest. What else needs to be proven? The patch shows that all the code
> besides guest register handlers (which is expected) is all common.

The complexity of dealing with Dom0 has increased. I've outlined the
process that I think should be followed: First determine what DomU needs.
Then see how much of this actually fits the existing code (handling Dom0).
Then decide whether altering Dom0 handling is actually worth it,
compared to handling DomU separately. In fact handling it separately
first may have its own benefits, like easing review and reducing the risk
of breaking Dom0 handling. If then there are enough similarities, in a
2nd step both may want folding.

>> All of this applies only with memory decoding enabled, I expect.
>> Disabling memory decoding on a device ought to be a simple "unmap all
>> BARs", while enabling is "map all BARs". Which again is, due to the
>> assumed lack of sharing of pages, much simpler than on Dom0: You only
>> need to subtract the MSI-X table range(s) (if any, and perhaps not
>> necessary when unmapping, as there's nothing wrong to unmap a P2M slot
>> which wasn't mapped); this may not even require any rangeset at all to
>> represent.
>>
>> And in fact I wonder whether for DomU-s you want to support BAR changes
>> in the first place while memory decoding is enabled.
> 
> No, why? I want to keep the existing logic, e.g. with memory decoding
> disabled as it is now.

Afaict existing code deals with both cases. What I was putting under
question is whether DomU handling code also needs to.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 10:56:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 10:56:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182982.330874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHje-0003ju-Hu; Thu, 09 Sep 2021 10:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182982.330874; Thu, 09 Sep 2021 10:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHje-0003jn-F1; Thu, 09 Sep 2021 10:56:50 +0000
Received: by outflank-mailman (input) for mailman id 182982;
 Thu, 09 Sep 2021 10:56:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sph1=N7=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mOHjd-0003jh-5q
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 10:56:49 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id a0fec65c-115c-11ec-b1ac-12813bfff9fa;
 Thu, 09 Sep 2021 10:56:47 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-473-4Wz7aK0gNlazTywP9efKtw-1; Thu, 09 Sep 2021 06:56:46 -0400
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
 [10.5.11.11])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 688E01093943;
 Thu,  9 Sep 2021 10:55:14 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.19])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id A963A6A255;
 Thu,  9 Sep 2021 10:55:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0fec65c-115c-11ec-b1ac-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631185007;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=m8VaXxkl6U6Rqdk0TTHun1PGjCpzFTVuNs2kej6qU7w=;
	b=aK9qH6QqgEz9bldU+5IIaOXsDYbKAUaNDm51mVLuX8BzvC1/6N7Kb8yDuLRRg22cuaElTm
	1innCy44sHLRJcuHGHv79g7z+vh9/Pt5xkUSDLT0pi4DwRnpbMO/rtGjwAT7VcJSXv31jh
	55xsYByu6FoI3a2IJulqY4f9IQUieAM=
X-MC-Unique: 4Wz7aK0gNlazTywP9efKtw-1
Date: Thu, 9 Sep 2021 11:55:09 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>,
	Peter Maydell <peter.maydell@linaro.org>,
	Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>, qemu-block@nongnu.org,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	Markus Armbruster <armbru@redhat.com>, qemu-devel@nongnu.org,
	xen-devel@lists.xenproject.org, Paolo Bonzini <pbonzini@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: [RFC PATCH 03/10] block: Use qemu_security_policy_taint() API
Message-ID: <YTnoDYpDaj055PnS@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-4-philmd@redhat.com>
 <YTnkhxWbm3NvGo/T@redhat.com>
MIME-Version: 1.0
In-Reply-To: <YTnkhxWbm3NvGo/T@redhat.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 11:40:07AM +0100, Daniel P. Berrangé wrote:
> On Thu, Sep 09, 2021 at 01:20:17AM +0200, Philippe Mathieu-Daudé wrote:
> > Add the BlockDriver::bdrv_taints_security_policy() handler.
> > Drivers implementing it might taint the global QEMU security
> > policy.
> > 
> > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> > ---
> >  include/block/block_int.h | 6 +++++-
> >  block.c                   | 6 ++++++
> >  2 files changed, 11 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/block/block_int.h b/include/block/block_int.h
> > index f1a54db0f8c..0ec0a5c06e9 100644
> > --- a/include/block/block_int.h
> > +++ b/include/block/block_int.h
> > @@ -169,7 +169,11 @@ struct BlockDriver {
> >      int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
> >                            Error **errp);
> >      void (*bdrv_close)(BlockDriverState *bs);
> > -
> > +    /*
> > +     * Return %true if the driver is withing QEMU security policy boundary,
> > +     * %false otherwise. See: https://www.qemu.org/contribute/security-process/
> > +     */
> > +    bool (*bdrv_taints_security_policy)(BlockDriverState *bs);

Also as with previous comments, I think we should not refer
to tainting or the security policy here, but instead simply
document whether we consider the bdrv to be secure or not.

Tainting is merely one action that is taken in accordance with
the security policy, as a result of the information presented.

> >      int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
> >                                         Error **errp);
> > diff --git a/block.c b/block.c
> > index b2b66263f9a..696ba486001 100644
> > --- a/block.c
> > +++ b/block.c
> > @@ -49,6 +49,7 @@
> >  #include "qemu/timer.h"
> >  #include "qemu/cutils.h"
> >  #include "qemu/id.h"
> > +#include "qemu-common.h"
> >  #include "block/coroutines.h"
> >  
> >  #ifdef CONFIG_BSD
> > @@ -1587,6 +1588,11 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv,
> >          }
> >      }
> >  
> > +    if (drv->bdrv_taints_security_policy) {
> > +        qemu_security_policy_taint(drv->bdrv_taints_security_policy(bs),
> > +                                   "Block protocol '%s'", drv->format_name);
> > +    }
> > +
> >      return 0;
> >  open_failed:
> >      bs->drv = NULL;
> 
> Again we need a way to report this via QAPI, but we don't have a natural
> place is hang this off for introspection before starting a guest.
> 
> The best we can do is report the information after a block backend has
> been instantiated. eg  Modify "BlockInfo" struct to gain
> 
>    '*secure': 'bool'
> 
> Note I made this an optional field, since unless we mark every single
> block driver impl straight away, we'll need to cope with the absence
> of information.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:04:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:04:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182988.330886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHqn-0005SS-Ax; Thu, 09 Sep 2021 11:04:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182988.330886; Thu, 09 Sep 2021 11:04:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHqn-0005SL-7C; Thu, 09 Sep 2021 11:04:13 +0000
Received: by outflank-mailman (input) for mailman id 182988;
 Thu, 09 Sep 2021 11:04:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sph1=N7=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mOHql-0005SF-H1
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:04:11 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 162f0b66-117a-437e-b55b-9e29ac5e89fc;
 Thu, 09 Sep 2021 11:04:10 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-433-4VNc8KnCO7GnYm0cbZ0gqg-1; Thu, 09 Sep 2021 07:04:08 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
 [10.5.11.12])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 643CD801B3D;
 Thu,  9 Sep 2021 11:04:07 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.19])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 20E2760C04;
 Thu,  9 Sep 2021 11:03:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 162f0b66-117a-437e-b55b-9e29ac5e89fc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631185450;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Faiv6H45qsLcLF+E3eChwe664ABfD2GKqV97YcVlGIQ=;
	b=gBDvCopABo2W0+OVxBF7hTHFg8Zx3U8hdGn9kLLfs0acAvrtV+YgEeft+hRQVvvz4mrvIl
	pYIvlfLmw8Xyx1llv7YTukSnc1R/hZDqqbeM+4/CqpoNMeoM5BKS+9AWVUQMktlbIYgeOi
	95jR5YqrSAcpWUOTYe1FBHsVbiR1DN4=
X-MC-Unique: 4VNc8KnCO7GnYm0cbZ0gqg-1
Date: Thu, 9 Sep 2021 12:03:48 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 06/10] qdev: Use qemu_security_policy_taint() API
Message-ID: <YTnqFHnolMLeY9c8@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-7-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-7-philmd@redhat.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:20AM +0200, Philippe Mathieu-Daudé wrote:
> Add DeviceClass::taints_security_policy field to allow an
> unsafe device to eventually taint the global security policy
> in DeviceRealize().
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/hw/qdev-core.h |  6 ++++++
>  hw/core/qdev.c         | 11 +++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> index bafc311bfa1..ff9ce6671be 100644
> --- a/include/hw/qdev-core.h
> +++ b/include/hw/qdev-core.h
> @@ -122,6 +122,12 @@ struct DeviceClass {
>       */
>      bool user_creatable;
>      bool hotpluggable;
> +    /*
> +     * %false if the device is within the QEMU security policy boundary,
> +     * %true if there is no guarantee this device can be used safely.
> +     * See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool taints_security_policy;
>  
>      /* callbacks */
>      /*

Although your use case is for devices, it probably makes more sense
to push this up into the Object base class.

I think it will need to be a tri-state value too, not a simple
bool. It isn't feasible to mark all devices with this property,
so initially we'll have no information about whether most
devices are secure or insecure. This patch gives the implication
that all devices are secure, except for the few that have been
marked otherwise, which is not a good default IMHO.

We want to be able to make it clear when introspecting, that we
have no information on security available for most devices ie

 - unset  => no information on security (the current default)
 - true => considered secure against malicious guest
 - false => considered insecure against malicious guest

Then we can also extend 'ObjectTypeInfo' to have a

   '*secure': 'bool'

to make 'qom-list-types' be able to introspect this upfront.



> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> index cefc5eaa0a9..a5a00f3564c 100644
> --- a/hw/core/qdev.c
> +++ b/hw/core/qdev.c
> @@ -31,6 +31,7 @@
>  #include "qapi/qmp/qerror.h"
>  #include "qapi/visitor.h"
>  #include "qemu/error-report.h"
> +#include "qemu-common.h"
>  #include "qemu/option.h"
>  #include "hw/hotplug.h"
>  #include "hw/irq.h"
> @@ -257,6 +258,13 @@ bool qdev_hotplug_allowed(DeviceState *dev, Error **errp)
>      MachineClass *mc;
>      Object *m_obj = qdev_get_machine();
>  
> +    if (qemu_security_policy_is_strict()
> +            && DEVICE_GET_CLASS(dev)->taints_security_policy) {
> +        error_setg(errp, "Device '%s' can not be hotplugged when"
> +                         " 'strict' security policy is in place",
> +                   object_get_typename(OBJECT(dev)));

Do you need a 'return' here to stop execution after reportig
the error ?

> +    }
> +
>      if (object_dynamic_cast(m_obj, TYPE_MACHINE)) {
>          machine = MACHINE(m_obj);
>          mc = MACHINE_GET_CLASS(machine);
> @@ -385,6 +393,9 @@ bool qdev_realize(DeviceState *dev, BusState *bus, Error **errp)
>      } else {
>          assert(!DEVICE_GET_CLASS(dev)->bus_type);
>      }
> +    qemu_security_policy_taint(DEVICE_GET_CLASS(dev)->taints_security_policy,
> +                               "device type %s",
> +                               object_get_typename(OBJECT(dev)));
>  
>      return object_property_set_bool(OBJECT(dev), "realized", true, errp);
>  }
> -- 
> 2.31.1
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:10:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:10:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.182996.330897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHws-0006uX-53; Thu, 09 Sep 2021 11:10:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 182996.330897; Thu, 09 Sep 2021 11:10:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOHws-0006uQ-1A; Thu, 09 Sep 2021 11:10:30 +0000
Received: by outflank-mailman (input) for mailman id 182996;
 Thu, 09 Sep 2021 11:10:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o0l/=N7=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mOHwq-0006uK-1p
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:10:28 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 31e1fb9a-2485-4bec-9601-8cff0be7f33b;
 Thu, 09 Sep 2021 11:10:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31e1fb9a-2485-4bec-9601-8cff0be7f33b
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631185826;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=BtkMclQ2vZA6mEV1ncKTUnBps1S9dGrsDmiV7418blI=;
  b=OX1cst4WHWbokfyOQBnOnxqeQ2loR8QDjViB25uhopsm+6LtoKndrpQQ
   99Keoc1yvPH390ixRTkABziHUbH+ZmtFGtDD4BOear1u1hwa5JqWlvroA
   uxfRS2uDZENDKYf1zRE9vUgwivl/cUl/0Ru6Qh0WM94cpjfQhIPXX82fU
   s=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: oeS8OCS6MLWm29N85pjyiZpxkOflXHTsDGYpLD1INgTv74mkHaRqwfwq2iXExnnPy2ePlITjqD
 QYTdkQ5InAMGqh5KgDpoEIlhNXcv+oMxMuW76g1k5IyfyONsElIg2hgcZrEszm9KqAeLK7loto
 dO1wm7CREQ1s4COC2/p/fwIqByjMwgmzI2uzUY6uM8R+45pvbDKzMv7IQgmK76yz0wtmJN7Ce2
 m8Td9cgtWS7MRoKj03ggkbuLfJA6OOiNgTd8GlCOtt8c1FyoUo04KpCpDpi9EWMf/nC9Qh4fk/
 EzoGfhZ2WFxmBZk4nmSQYN0g
X-SBRS: 5.1
X-MesageID: 52353231
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:7slNFKFVVSogsWeRpLqE6MeALOsnbusQ8zAXP0AYc3Jom+ij5q
 STdZUgpHrJYVkqNU3I9ertBEDEewK6yXcX2/hyAV7BZmnbUQKTRekIh7cKgQeQeBEWntQts5
 uIGJIeNDSfNzdHsfo=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208,217";a="52353231"
Date: Thu, 9 Sep 2021 12:10:22 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Daniel De Graaf
	<dgdegra@tycho.nsa.gov>, "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH 2/2] build: add --full to version.sh to guess
 $(XEN_FULLVERSION)
Message-ID: <YTnrnqz7Vt0pugFZ@perard>
References: <20210908095422.438324-1-anthony.perard@citrix.com>
 <20210908095422.438324-3-anthony.perard@citrix.com>
 <be89ae8c-99cf-5385-00a7-d9433e0fc50b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <be89ae8c-99cf-5385-00a7-d9433e0fc50b@suse.com>

On Wed, Sep 08, 2021 at 02:27:17PM +0200, Jan Beulich wrote:
> On 08.09.2021 11:54, Anthony PERARD wrote:
> > --- a/tools/flask/policy/Makefile.common
> > +++ b/tools/flask/policy/Makefile.common
> > @@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER)
> >  #
> >  ########################################
> >  
> > -POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory)
> > +POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)
> 
> Shell scripts better get invoked by "$(SHELL) <script>", to avoid
> depending on the script actually being marked as executable (which is
> impossible on some file systems).
> 
> > --- a/version.sh
> > +++ b/version.sh
> > @@ -1,5 +1,18 @@
> >  #!/bin/sh
> >  
> > +opt_full=false
> > +while [ $# -gt 1 ]; do
> > +    case "$1" in
> > +        --full) opt_full=true ;;
> > +        *) break ;;
> > +    esac
> > +    shift
> > +done
> > +
> >  MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "`
> >  MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "`
> > -printf "%d.%d" $MAJOR $MINOR
> > +
> > +if $opt_full; then
> > +    EXTRAVERSION=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
> > +fi
> > +printf "%d.%d%s" $MAJOR $MINOR $EXTRAVERSION
> 
> I guess you want to clear EXTRAVERSION either prior to the "if" or
> in an "else".

Actually, I'm thinking of writing this instead:
    if $opt_full; then
        extraversion=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
        : ${XEN_EXTRAVERSION:=${extraversion}${XEN_VENDORVERSION}}
    else
        unset XEN_EXTRAVERSION
    fi

This would takes care of cases where a packager add XEN_EXTRAVERSION or
XEN_VENDORVERSION to the environment, and I think would produce the same
result as `make xenversion` in that case.

But that leaves the case of running e.g. "make XEN_VENDORVERSION=-neat
xenversion" out, hopefully, no one does that. Otherwise, we might need
"export" or a new macro in the build system (and that would still leave
out someone having "xen/xen-version" file).

:-( I think I just found a package overriding XEN_VENDORVERSION with a
command line argument, the AUR package. So ./version.sh needs help from
the build system:

Maybe

    XEN_FULLVERSION=$(shell env XEN_EXTRAVERSION=$(XEN_EXTRAVERSION) XEN_VENDORVERSION=$(XEN_VENDORVERSION) $(SHELL) $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)

or probably just exporting those two var in Config.mk or tools/Rules.mk.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:30:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:30:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183003.330908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIGV-0001N4-Q3; Thu, 09 Sep 2021 11:30:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183003.330908; Thu, 09 Sep 2021 11:30:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIGV-0001Mx-N0; Thu, 09 Sep 2021 11:30:47 +0000
Received: by outflank-mailman (input) for mailman id 183003;
 Thu, 09 Sep 2021 11:30:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOIGU-0001Mr-MZ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:30:47 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 03e79965-54eb-4a48-8c17-57101e46a7bc;
 Thu, 09 Sep 2021 11:30:44 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189BRR2t005162; 
 Thu, 9 Sep 2021 11:30:41 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2057.outbound.protection.outlook.com [104.47.4.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayhdm80mq-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 11:30:40 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB2768.eurprd03.prod.outlook.com (2603:10a6:800:e3::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 11:30:37 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 11:30:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03e79965-54eb-4a48-8c17-57101e46a7bc
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nfL0utlFDbU+7GUU2JL5bfMOyLXwv3UPWJScXUiMjhW+yr/NL6zckl+fIvzhI7yXO1UvXdztVwD0eYQpHszUNgEXJSyVkVsbmDnSycnRNOuh7SVqp5C7Vw5qHMxEdr8anYkMFTyhW5DjfQ67RKsFt9JxZWOS+Zv9yA7okGmIUy6Fk8CqGj2Z7Q5X2rtloUimiHUqhBEX3peoUyUcaOQcYEl+M98IzTejzkRtYE3EM1+JVCU67Vuocpi8O38qUd1AevX5Vc4kpakU3zqcsQpFVZnzcfb6ulcZbfjWa29XJPcLsD+mi0hu/vIDX60RwosJ5CfaTKBY+MzYaVhHo2DNVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eAHbnAwPso8A7XidnU7Twg9ps/SmWQk1WSgQVmzPR0Y=;
 b=etpTtkfkXmiZcGtFgdf+87+qVX7ay+eiHjYQ+PMrNaGGo6R9KDf/yCZH2J9WzgOf7i2V2wvemFuEewrxD6Qq6rVZV5YFrWA52DNd6b5GnTrA3W76Xcqlgi77vBBCMNchuvxM2Rb4ooCqOVy0nWPYmHyITJfn9+cSZ+/i2oXGCT7FjPzaORD+QENUEE23kYVQ/h3tqgvj5m5c2GcWBtPZOTCRRMrQFmB8D21TrPAold38gehf+FJFTCtrHkF4oTnV8hkN4dVES+18472o3nW5LDW4nFozGOVOJ5azvdVTAXVk8DyF7g5NJd57bTdZqhZc8/wgdWgMtG9311jpedrcvA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eAHbnAwPso8A7XidnU7Twg9ps/SmWQk1WSgQVmzPR0Y=;
 b=D6xDldpcM6DccfCBgbKbUaUihtUKmbvw2pPROyGaeNpGKKwopEo0/odS6VvojNzZdEco6acKDCuYevrgPyY6VRde/XLeUj6cflYSsltYv3dPZkHu4f5n7qX4RvBqSRS8Cc2KgODU2OqRFPTal2nI0tWZhj3fm7oJM1cflYV9MlQAS2dcb8gpvlv6EYuVIQANpARckdamBTRKwS9DurjZ96nZ8U7Yct6DORq6XrNYq/d1WP8gj5VV3ApPWvLhLxOf8cS49GY2mh+MmPRR+JViTIuYyAV0FcvfhzYaA0e8MdHUvYI83L2HAfRWWdFy/Cpz/3rrpUMjJRX2LTQc4M/B8w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Topic: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
Thread-Index: 
 AQHXoKxjXfke7B20QEeo6YjlfUMhlquXGvYAgAMgDgCAAAg4gIAA8L+AgAAy0wCAAA1uAIAAB6sAgAAGiICAAAwkAIAADEsA
Date: Thu, 9 Sep 2021 11:30:37 +0000
Message-ID: <d81e9eb2-c16c-1f59-644e-09bf3466be76@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
 <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
 <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
 <e154e4f7-bd60-2645-1ce5-1c2b5404236f@epam.com>
 <3e7dc9f3-1536-0643-b732-4a203bee34d4@suse.com>
In-Reply-To: <3e7dc9f3-1536-0643-b732-4a203bee34d4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 298632fb-045e-4e8a-a863-08d973853ef4
x-ms-traffictypediagnostic: VI1PR0302MB2768:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB2768DA68C2CE8E88861B07D3E7D59@VI1PR0302MB2768.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7691;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 SFHRAmSlkOKN74bcJ8z5sRAUs/XU16wENqk2j0tI0z+VAyxsklV4PVunD+9FYsUTFwyFo4uIj8Caqy75gvPMGcp3wtIjwhAcP0Gy+oys4zkAg4ctHhdEbw8wHLKgdNU9wAOcFidkl9XS6Y3Kfj7SfgQxIoPGnu9lf7iYpL/gUXPOlTQau2yu66AbK3OrmZBpfG70uUrK744VF2bCAboug8bSK+k/SJgbL26avIBUhECAgmaxbR1hQBGTGfQ6JALnIWFYmwqXpEDrnCSjTG7BlvufgrSngRB2xb9z+/G561NbjgSvnjPME1Mbp3UGu5K5CR95LuHBjvuYfOpL2mCvkIBdHxduSZthlfTK9SfxFzfIo6+oWUimGmWpgUtxSIhm0wz4W4QkZEGT1mexZvIg2g25AsSgP4hPo0kOfLsjVmupLuroWReO2PS/CNagD5ungIdXzsnwPu6+elC26ZIFNwG4aOugLAXxukaxmIQupozEoejrpdgtETKdarxUp30dTdKp4JNVGPTq4v0v4GgW8C21SgA7ZEdO/xe0H7ItAjUx1mG/NC1ZPhZ5zKGyqjSRHortEoLV70JX/tAp5U8WvAoy1W9J76ufZBrM2WQG8JTPqNQ8hHp1woLvVCj9vpk4n6eWs445/vTafi5Y9Ahy/QOJdapfTn7Ce83thA5MJmEXsmLmHfvDz5z3x0mbtCXCGAXCsAYJkSzMjF6G90PtVKmS831+4wbLYUMDs8Rmgu34WIilzK8G781Cd/V1GgGF
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(366004)(8936002)(6512007)(38100700002)(71200400001)(122000001)(86362001)(5660300002)(110136005)(83380400001)(36756003)(26005)(53546011)(6506007)(2906002)(66946007)(186003)(91956017)(76116006)(66446008)(4326008)(66556008)(64756008)(316002)(31686004)(6486002)(478600001)(8676002)(31696002)(38070700005)(66476007)(54906003)(2616005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?b1QyZE5uRGlqcHRiaVZXOENFeSszVFdoS2J5Y2xrelVDZjkxN09Eem4vQzRw?=
 =?utf-8?B?YkkxM3ZpQU1HSG0ya1BIK0lHR2RIYUtsc3hKN0ZGaW5vcmJZeE5uNHMwVFNw?=
 =?utf-8?B?aHpObnZOR1VxRGtPR1lNZEVtZjVLaWRsVU8xQkdISnU5Z0RWcE0yMFFiUlYv?=
 =?utf-8?B?OWh5REtaeTY3bU15NG5jK2xPNjErQm9CbGVmQzBTSHFXd0NkVlEzZzVWa1VG?=
 =?utf-8?B?eURVQXFOT3FKME9ydFcwdjBYOEFMbHBvaFZiMmRMUk5qQW1TY0wwNno3ek55?=
 =?utf-8?B?dFpUb2tZT3NNb1gvSXVaUTRDZDJ6ZjY3OGFwcHJ2TFhQZ0pacGZNYnltYkQ2?=
 =?utf-8?B?aDZTYllkQStwZEgyVmJXc0dPdnY1ZzRpMGlwVkJhSmdUVFI0Vks4WSsvMU5y?=
 =?utf-8?B?czFMNnYyTDNPTjExYnVJTVNoSG1OTnNwSzRGYTk0RU1RbHVpaUxCaUVkYkZ2?=
 =?utf-8?B?WjlLMDZFL1FqczZHVkRKVzhDZk12dVc4WWpNY21xK2daZzVvaHV4QWR5OEZw?=
 =?utf-8?B?SUdaNllCSUNjSTJuUjNobDI2OVZvVVpZQTRNdW5hRDg1WjFMK2hxS1l5OCtC?=
 =?utf-8?B?TkZQc2NFTUNramswVFNvWWRmdnc2aDhjMlpJRkJsTnVneVJIbjhFd0FObEwy?=
 =?utf-8?B?c2oveTMyU000V3hpb3czWDBIMWZ3T1lGL3FCeGovUHlLZWN4MnJVR09ZcDh3?=
 =?utf-8?B?bFB3czVHOUtub1R3QVFGUldSb1o0NFRZcjhXVldoRkxNbzgzOUM0ZnJiY2hZ?=
 =?utf-8?B?a0NXSlR4Q2RCUnYxb3FkR3F4MjJUWk1SVTBIeTJ5STBSNGdoOFFwaUVmd3I4?=
 =?utf-8?B?NXFJOVQ0YVFNVXZTbGJ5d2czalNpOS9KaHVSK3lZdHZNRWg3a1V6TEFjTzRn?=
 =?utf-8?B?WGREOUZGN0VTOThNQ2VYNm5wWXZGSkhDS3NMeGtxM2oyK1VDZzBFS1dtOG1I?=
 =?utf-8?B?SnZJOUpLYVZEQmlVbWw0bmVNMnhhT1hKUXRObVpaV0R6WE1tSzRhRDZCd1pI?=
 =?utf-8?B?MnVSRHpLWXJZMlBSYmdOdmdmNnNvTCtqUS9oUHZJU2JoRmdIS1RWbnhSYkYr?=
 =?utf-8?B?MWlBVEdBNWk2M1pDNGk2THlQTk1TYWdCKzlheXBJcktWaEtEOWwzQi9VZlpj?=
 =?utf-8?B?WGNrbjQ1ZTZRenYxL3hXbGhoTmtCelkyYmFkazZTb3dOc2tGU2xacUNMSmdx?=
 =?utf-8?B?MGxwYmdVeVAzemVxNG8rY290TlozdmNsLzVYTGIyLzBrbzJTUHFRUUVzb2Fi?=
 =?utf-8?B?RGp4dXBNazE4RXA4SUluaEtYWkpwWmFGenpZUFZoU1Q2V1ZZQVB4Z0JQT3h4?=
 =?utf-8?B?TStTUlQrT2U1RkxVZTRyVzd2OFU4SEx4U3plNjhOb2E1alhqcWRjMDlHVFdB?=
 =?utf-8?B?dEpRR1hncy9KV2plQWttTm1jYXlDRHJjUy9OY2VWbjAwZExyb2NXZVZvRmRO?=
 =?utf-8?B?NmNSUllXcWtGWVBMOFl6dlQvM2FBeU5uN2kxWXVRZGtHT24vMXIyV2JpVWkz?=
 =?utf-8?B?cGozM2NSRnppK00vWGpEZm1zWXRoVmY0cVBqWk82ZkhPN2lFZGVmRkRNaWpl?=
 =?utf-8?B?dExoVGJyUnZaY2VuUVhqZUdFTFpGWXU1V0s3Y3p4RFAyUU9ySVNYallJeS8x?=
 =?utf-8?B?ODUxOU15Ulo3TFFQNHZpV25BQ2ZOL3ZvZnNuSFBPd1A4eWxGRmk3NlZBS2Rz?=
 =?utf-8?B?bEFVVnNXd2k1RDlaOEdJVkJ0MG5yS2RsRW15eHp0RmpKY2ZLR1FpeUZSc0ZJ?=
 =?utf-8?Q?cqwjsR/d2kdrBkRtQRrXnmMIZUQoXGGp9Z53y2k?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <0AF619CE15250B4D95FF8B1202F8EE0F@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 298632fb-045e-4e8a-a863-08d973853ef4
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 11:30:37.1760
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: YhacY5nhW/zOc4PzfFlLUYU/BigakPdjBKyOrk07ToA+Srlfq2GHn5DMyG0AVjT4pyFhnY2rBOuMpap47Y7XoFoy/r1MZf2OCMh9IrNHwGliKB+duEMjis7J1QprEYmi
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2768
X-Proofpoint-ORIG-GUID: pPEeKv-VkVlwsa2Lk1mjoRL4t4Cyoger
X-Proofpoint-GUID: pPEeKv-VkVlwsa2Lk1mjoRL4t4Cyoger
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 mlxlogscore=999 lowpriorityscore=0 mlxscore=0 bulkscore=0 spamscore=0
 priorityscore=1501 adultscore=0 malwarescore=0 phishscore=0 clxscore=1015
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090070

DQpPbiAwOS4wOS4yMSAxMzo0NiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTI6MDMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTI6
MzksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMTE6MTIsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBBbnl3YXlzLCBJIGFtIG9wZW4gdG8gYW55IGRl
Y2lzaW9uIG9uIHdoYXQgd291bGQgYmUgdGhlIHJpZ2h0IGFwcHJvYWNoIGhlcmU6DQo+Pj4+DQo+
Pj4+IDEuIFVzZSByYW5nZSBzZXRzIHBlciBCQVIgYXMgaW4gdGhlIHBhdGNoDQo+Pj4+DQo+Pj4+
IDIuIFJlbW92ZSByYW5nZSBzZXRzIGNvbXBsZXRlbHkgYW5kIGhhdmUgYSBwZXItdkNQVSBsaXN0
IHdpdGggbWFwcGluZw0KPj4+Pg0KPj4+PiBkYXRhIGFzIEkgZGVzY3JpYmVkIGFib3ZlDQo+Pj4+
DQo+Pj4+IDMuIEFueXRoaW5nIGVsc2U/DQo+Pj4gQSBkZWNpc2lvbiBmaXJzdCByZXF1aXJlcyBh
IHByb3Bvc2FsLg0KPj4gSSBhbHJlYWR5IGhhdmUgMjogb25lIGluIHRoZSBwYXRjaCB3aXRoIHRo
ZSByYW5nZSBzZXQgcGVyIEJBUiBhbmQgb25lIGRlc2NyaWJlZA0KPj4gZWFybGllciBpbiB0aGUg
dGhyZWFkIHdpdGggYSBzaW5nbGUgcmFuZ2Ugc2V0IGFuZCBhIGxpc3QgZm9yIEdGTiA8LT4gTUZO
Lg0KPj4gSWYgeW91IGNhbiB0ZWxsIHlvdXIgb3BpbmlvbiBJIGFtIGFsbCBlYXJzLiBCdXQsIHBs
ZWFzZSBiZSBzcGVjaWZpYyBhcyBjb21tb24gd29yZHMNCj4+IGRvbid0IGNoYW5nZSBhbnl0aGlu
ZyB0byBtZS4NCj4+IEF0IHRoZSBzYW1lIHRpbWUgSSBkbyB1bmRlcnN0YW5kIHRoYXQgdGhlIGN1
cnJlbnQgY29kZSBpcyBub3Qgc2V0IGluIHN0b25lLA0KPj4gYnV0IHdlIHNob3VsZCBoYXZlIGEg
Z29vZCByZWFzb24gZm9yIG1ham9yIGNoYW5nZXMgdG8gaXQsIElNTy4NCj4gQW5kIEkgdmlldyB5
b3VyIGNoYW5nZSwgYXMgcHJvcG9zZWQsIGFzIGEgbWFqb3Igb25lLiBZb3UgdHVybiB0aGUgbG9n
aWMgYWxsDQo+IG92ZXIgaW1vLg0KPg0KPj4gSSBtZWFuIHRoYXQgYmVmb3JlDQo+PiBEb21VJ3Mg
d2Ugd2VyZSBmaW5lIHdpdGggdGhlIHJhbmdlIHNldHMgZXRjLCBhbmQgbm93IHdlIGFyZSBub3Q6
DQo+PiBzbyB3aGF0IGhhcyBjaGFuZ2VkIHNvIG11Y2g/DQo+IE5vdGhpbmcgaGFzIGNoYW5nZWQu
IEknbSBub3QgYWR2b2NhdGluZyBmb3IgcmVtb3ZhbCBvZiB0aGUgcmFuZ2VzZXQgdXNlIGluDQo+
IGhhbmRsaW5nIERvbTAncyBuZWVkcy4gSSdtIHN1Z2dlc3RpbmcgdGhhdCB0aGVpciB1c2UgbWln
aHQgbm90IGJlIGEgZ29vZA0KPiBmaXQgZm9yIERvbVUuDQoNClRoZSBwcm9wb3NlZCBjaGFuZ2Ug
bWFrZXMgdGhlIHNhbWUgY29kZSB3b3JrIGZvciBib3RoIERvbTAgYW5kIERvbVUuDQoNClNvLCBp
bnN0ZWFkIG9mIGhhdmluZyB0aGUgY29tbW9uIGNvZGUgYXMgaXQgcHJvcG9zZWQgZG8geW91IHN1
Z2dlc3QgdG8gaW52ZW50DQoNCnNvbWV0aGluZyBzcGVjaWFsIGZvciBEb21VIChtYWtpbmcgdGhl
IHNhbWUgam9iIGFzIHdlIGFscmVhZHkgZG8gZm9yIERvbTApDQoNCmFuZCB0aGVuIHNlZSBpZiB3
ZSBjYW4gdGhlbiBjb21iaW5lIHRoZSBib3RoIHRvIGhhdmUgdGhlIGNvZGUgY29tbW9uDQoNCmFn
YWluPyBJIGFtIHNheWluZyB0aGF0IHRoZSBjb2RlIGlzIGFscmVhZHkgY29tbW9uIGV2ZW4gaWYg
eW91IHRoaW5rIHRoYXQNCg0KZm9yIERvbVUgaXQgY2FuIGJlIHNpbXBsZXIgKEkgY2FuJ3Qgc3Rp
bGwgc2VlIGluIHdoaWNoIHdheSBhcyBwMm0gYW5kIG90aGVyDQoNCnRoaW5ncyBhcmUgbm90IGRp
cmVjdGx5IHRvdWNoZWQgYnkgdGhlIHZQQ0kgY29kZSwgZS5nLiBib3RoIERvbTAgYW5kIERvbVUN
Cg0KdXNlIHttYXB8dW5tYXB9X21taW9fcmVnaW9ucyBhbmQgdGhlIG9ubHkgZGlmZmVyZW5jZSBp
cyB0aGF0IGZvciBEb20wDQoNCndlIGhhdmUgTUZOID09IEdGTiBhbmQgZm9yIERvbVUgaXQncyBu
b3QpLg0KDQpTbywgZXZlbiBpZiByYW5nZXMgc2V0cyBhcmUgbm90IGdvb2QgZm9yIERvbVVzIChJ
IGNhbid0IHNlZSB3aHkpLCBidXQgaWYgdGhleSBoZWxwDQoNCmhhdmUgdGhlIGNvZGUgY29tbW9u
IEkgdGhpbmsgaXQgaXMgd29ydGggaGF2aW5nIHRoZW0uDQoNCj4NCj4+PiAgICBJIHRoaW5rIDMg
aXMgdGhlIHdheSB0byBpbnZlc3RpZ2F0ZQ0KPj4+IGZpcnN0OiBSYXRoZXIgdGhhbiBzdGFydGlu
ZyBmcm9tIHRoZSBjb2RlIHdlIGN1cnJlbnRseSBoYXZlLCBzdGFydCBmcm9tDQo+Pj4gd2hhdCB5
b3UgbmVlZCBmb3IgRG9tVS1zIHRvIHdvcmsuIElmIHRoZXJlJ3MgZW5vdWdoIG92ZXJsYXAgd2l0
aCBob3cgd2UNCj4+PiBoYW5kbGUgRG9tMCwgY29kZSBjYW4gYmUgc2hhcmVkLg0KPj4gWW91IGNh
biBzZWUgdGhhdCBpbiBteSBwYXRjaCB0aGUgc2FtZSBjb2RlIGlzIHVzZWQgYnkgYm90aCBod2Rv
bSBhbmQNCj4+IGd1ZXN0LiBXaGF0IGVsc2UgbmVlZHMgdG8gYmUgcHJvdmVuPyBUaGUgcGF0Y2gg
c2hvd3MgdGhhdCBhbGwgdGhlIGNvZGUNCj4+IGJlc2lkZXMgZ3Vlc3QgcmVnaXN0ZXIgaGFuZGxl
cnMgKHdoaWNoIGlzIGV4cGVjdGVkKSBpcyBhbGwgY29tbW9uLg0KPiBUaGUgY29tcGxleGl0eSBv
ZiBkZWFsaW5nIHdpdGggRG9tMCBoYXMgaW5jcmVhc2VkLiBJJ3ZlIG91dGxpbmVkIHRoZQ0KPiBw
cm9jZXNzIHRoYXQgSSB0aGluayBzaG91bGQgYmUgZm9sbG93ZWQ6IEZpcnN0IGRldGVybWluZSB3
aGF0IERvbVUgbmVlZHMuDQpJdCBpcyBhbHJlYWR5IGtub3duLCBHRk4gPC0+IE1GTiBub24taWRl
bnRpdHkgbWFwcGluZ3MNCj4gVGhlbiBzZWUgaG93IG11Y2ggb2YgdGhpcyBhY3R1YWxseSBmaXRz
IHRoZSBleGlzdGluZyBjb2RlIChoYW5kbGluZyBEb20wKS4NCkl0IGlzIGFscmVhZHkgaW4gdGhl
IHBhdGNoOiB3ZSBoYXZlIGFsbCBjb2RlIGNvbW1vbiBmb3IgYm90aCBEb20wIGFuZCBEb21VDQo+
IFRoZW4gZGVjaWRlIHdoZXRoZXIgYWx0ZXJpbmcgRG9tMCBoYW5kbGluZyBpcyBhY3R1YWxseSB3
b3J0aCBpdCwNCj4gY29tcGFyZWQgdG8gaGFuZGxpbmcgRG9tVSBzZXBhcmF0ZWx5Lg0KSXQgbGVh
ZHMgdG8gdGhlIHNhbWUgZnVuY3Rpb25hbGl0eSBpbXBsZW1lbnRlZCB0d2ljZQ0KPiAgIEluIGZh
Y3QgaGFuZGxpbmcgaXQgc2VwYXJhdGVseQ0KPiBmaXJzdCBtYXkgaGF2ZSBpdHMgb3duIGJlbmVm
aXRzLCBsaWtlIGVhc2luZyByZXZpZXcgYW5kIHJlZHVjaW5nIHRoZSByaXNrDQo+IG9mIGJyZWFr
aW5nIERvbTAgaGFuZGxpbmcuIElmIHRoZW4gdGhlcmUgYXJlIGVub3VnaCBzaW1pbGFyaXRpZXMs
IGluIGENCj4gMm5kIHN0ZXAgYm90aCBtYXkgd2FudCBmb2xkaW5nLg0KDQpZb3UgY2FuIHNlZSBm
cm9tIHRoZSBwYXRjaCBpZiB3ZSBoYXZlICJpZiAoIGh3ZG9tICkiIHNwcmVhZCBvdmVyIHRoZQ0K
DQppbXBsZW1lbnRhdGlvbi4gSSBndWVzcyB5b3Ugd29uJ3QgZmluZCB0aGF0IChiZXNpZGVzIGd1
ZXN0IHJlZ2lzdGVyDQoNCmhhbmRsZXJzIHdoaWNoIGlzIGV4cGVjdGVkKS4NCg0KPg0KPj4+IEFs
bCBvZiB0aGlzIGFwcGxpZXMgb25seSB3aXRoIG1lbW9yeSBkZWNvZGluZyBlbmFibGVkLCBJIGV4
cGVjdC4NCj4+PiBEaXNhYmxpbmcgbWVtb3J5IGRlY29kaW5nIG9uIGEgZGV2aWNlIG91Z2h0IHRv
IGJlIGEgc2ltcGxlICJ1bm1hcCBhbGwNCj4+PiBCQVJzIiwgd2hpbGUgZW5hYmxpbmcgaXMgIm1h
cCBhbGwgQkFScyIuIFdoaWNoIGFnYWluIGlzLCBkdWUgdG8gdGhlDQo+Pj4gYXNzdW1lZCBsYWNr
IG9mIHNoYXJpbmcgb2YgcGFnZXMsIG11Y2ggc2ltcGxlciB0aGFuIG9uIERvbTA6IFlvdSBvbmx5
DQo+Pj4gbmVlZCB0byBzdWJ0cmFjdCB0aGUgTVNJLVggdGFibGUgcmFuZ2UocykgKGlmIGFueSwg
YW5kIHBlcmhhcHMgbm90DQo+Pj4gbmVjZXNzYXJ5IHdoZW4gdW5tYXBwaW5nLCBhcyB0aGVyZSdz
IG5vdGhpbmcgd3JvbmcgdG8gdW5tYXAgYSBQMk0gc2xvdA0KPj4+IHdoaWNoIHdhc24ndCBtYXBw
ZWQpOyB0aGlzIG1heSBub3QgZXZlbiByZXF1aXJlIGFueSByYW5nZXNldCBhdCBhbGwgdG8NCj4+
PiByZXByZXNlbnQuDQo+Pj4NCj4+PiBBbmQgaW4gZmFjdCBJIHdvbmRlciB3aGV0aGVyIGZvciBE
b21VLXMgeW91IHdhbnQgdG8gc3VwcG9ydCBCQVIgY2hhbmdlcw0KPj4+IGluIHRoZSBmaXJzdCBw
bGFjZSB3aGlsZSBtZW1vcnkgZGVjb2RpbmcgaXMgZW5hYmxlZC4NCj4+IE5vLCB3aHk/IEkgd2Fu
dCB0byBrZWVwIHRoZSBleGlzdGluZyBsb2dpYywgZS5nLiB3aXRoIG1lbW9yeSBkZWNvZGluZw0K
Pj4gZGlzYWJsZWQgYXMgaXQgaXMgbm93Lg0KPiBBZmFpY3QgZXhpc3RpbmcgY29kZSBkZWFscyB3
aXRoIGJvdGggY2FzZXMuDQoNCkhtLCBJIHRob3VnaHQgdGhhdCB3ZSBvbmx5IG1hcC91bm1hcCB3
aXRoIG1lbW9yeSBkZWNvZGluZyBkaXNhYmxlZC4NCg0KRm9yIG15IGVkdWNhdGlvbjogd2hhdCBo
YXBwZW5zIGlmIHlvdSB1bm1hcCB3aXRoIGRlY29kaW5nIGVuYWJsZWQgYW5kDQoNCmRvbWFpbiBh
Y2Nlc3NlcyB0aGUgTU1JT3M/DQoNCj4gICBXaGF0IEkgd2FzIHB1dHRpbmcgdW5kZXINCj4gcXVl
c3Rpb24gaXMgd2hldGhlciBEb21VIGhhbmRsaW5nIGNvZGUgYWxzbyBuZWVkcyB0by4NCj4NCj4g
SmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:32:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:32:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183009.330918 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIHv-0002Ge-64; Thu, 09 Sep 2021 11:32:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183009.330918; Thu, 09 Sep 2021 11:32:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIHv-0002GX-30; Thu, 09 Sep 2021 11:32:15 +0000
Received: by outflank-mailman (input) for mailman id 183009;
 Thu, 09 Sep 2021 11:32:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOIHt-0002GJ-Qd
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:32:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOIHs-0004Q5-M1; Thu, 09 Sep 2021 11:32:12 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOIHs-0000OZ-FH; Thu, 09 Sep 2021 11:32:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=XntlN2THUrnynui6bflOC+jkjIzEk7ke1N2vacz9ij0=; b=L7orFDCheURfX+/amdo2gDN6Z+
	4/2X0JnVJQBcgZ09ec1VU3HiQHEltSFKUsKvSgPViw1buhExu4g1lAGkvV+bPPhpH/cM3OYL7XyKj
	CpLGSYtTPb8soFNRKRobDX4vVSOGp2iMILewZgHvJ8X/NtXDbBVdP0D10H8U9GrPRvHY=;
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
To: Rahul Singh <rahul.singh@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <d7da2d5c-fdc0-d72c-789c-c6ba75412f3f@xen.org>
Date: Thu, 9 Sep 2021 12:32:10 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Rahul,

On 19/08/2021 13:02, Rahul Singh wrote:
> Add support for PCI ecam operations to access the PCI
> configuration space.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>   xen/arch/arm/pci/Makefile           |  1 +
>   xen/arch/arm/pci/ecam.c             | 63 +++++++++++++++++++++++++++++
>   xen/arch/arm/pci/pci-access.c       | 53 ++++++++++++++++++++++++
>   xen/arch/arm/pci/pci-host-common.c  | 13 +++++-
>   xen/arch/arm/pci/pci-host-generic.c |  8 +++-
>   xen/include/asm-arm/pci.h           | 32 +++++++++++++++
>   6 files changed, 167 insertions(+), 3 deletions(-)
>   create mode 100644 xen/arch/arm/pci/ecam.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index f3d97f859e..6f32fbbe67 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -2,3 +2,4 @@ obj-y += pci.o
>   obj-y += pci-access.o
>   obj-y += pci-host-generic.o
>   obj-y += pci-host-common.o
> +obj-y += ecam.o
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> new file mode 100644
> index 0000000000..91c691b41f
> --- /dev/null
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -0,0 +1,63 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/ecam.c
> + * Copyright 2016 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/sched.h>
> +
> +/*
> + * Function to implement the pci_ops ->map_bus method.
> + */
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                                      uint32_t sbdf, uint32_t where)
> +{
> +    const struct pci_config_window *cfg = bridge->sysdata;
> +    unsigned int devfn_shift = cfg->ops->bus_shift - 8;
> +    void __iomem *base;
> +
> +    pci_sbdf_t sbdf_t = (pci_sbdf_t) sbdf ;

AFAICT, pci_sbdf is an union between a 32-bit and a structure. So please 
don't use the cast and use the 32-bit field to assign the value.

Also, there is an extra space before ';'.

> +    unsigned int busn = sbdf_t.bus;
> +
> +    if ( busn < cfg->busn_start || busn > cfg->busn_end )
> +        return NULL;
> +
> +    busn -= cfg->busn_start;
> +    base = cfg->win + (busn << cfg->ops->bus_shift);
> +
> +    return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;

How about using PCI_DEVFN2(sbdf)? This would allow you to drop the use 
of sbdf_t completely (sbdf_t.bus could be replaced with PCI_BUS(sdbf)).

> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops pci_generic_ecam_ops = {
> +    .bus_shift  = 20,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index b938047c03..f39f6a3a38 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -15,6 +15,59 @@
>    */
>   
>   #include <xen/pci.h>
> +#include <asm/io.h>
> +
> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);

Please add a newline here.

> +    if (!addr) {

You seem to use a mix of Xen and Linux coding style. If the file 
contains mostly Xen code, then we should use the former.

> +        *value = ~0;
> +        return -ENODEV;
> +    }
> +
> +    switch (len)
> +    {
> +    case 1:
> +        *value = readb(addr);
> +        break;
> +    case 2:
> +        *value = readw(addr);
> +        break;
> +    case 4:
> +        *value = readl(addr);
> +        break;
> +    default:
> +        BUG();
> +    }
> +
> +    return 0;
> +}
> +
> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +    if (!addr)
> +        return -ENODEV;
> +
> +    switch (len)
> +    {
> +    case 1:
> +        writeb(value, addr);
> +        break;
> +    case 2:
> +        writew(value, addr);
> +        break;
> +    case 4:
> +        writel(value, addr);
> +        break;
> +    default:
> +        BUG();
> +    }
> +
> +    return 0;
> +}
>   
>   static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>                                   unsigned int len)
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 9dd9b02271..c582527e92 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -68,6 +68,7 @@ static void pci_ecam_free(struct pci_config_window *cfg)
>   }
>   
>   static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
> +                                              const struct pci_ecam_ops *ops,
>                                                 int ecam_reg_idx)
>   {
>       int err;
> @@ -96,6 +97,7 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
>   
>       cfg->phys_addr = addr;
>       cfg->size = size;
> +    cfg->ops = ops;
>   
>       /*
>        * On 64-bit systems, we do a single ioremap for the whole config space
> @@ -111,6 +113,13 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
>       printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr,
>               cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end);
>   
> +    if ( ops->init )
> +    {
> +        err = ops->init(cfg);
> +        if (err)
> +            goto err_exit;
> +    }
> +
>       return cfg;
>   
>   err_exit_remap:
> @@ -216,6 +225,7 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev)
>   }
>   
>   int pci_host_common_probe(struct dt_device_node *dev,
> +                          const struct pci_ecam_ops *ops,
>                             int ecam_reg_idx)
>   {
>       struct pci_host_bridge *bridge;
> @@ -227,7 +237,7 @@ int pci_host_common_probe(struct dt_device_node *dev,
>           return -ENOMEM;
>   
>       /* Parse and map our Configuration Space windows */
> -    cfg = gen_pci_init(dev, ecam_reg_idx);
> +    cfg = gen_pci_init(dev, ops, ecam_reg_idx);
>       if ( !cfg )
>       {
>           err = -ENOMEM;
> @@ -236,6 +246,7 @@ int pci_host_common_probe(struct dt_device_node *dev,
>   
>       bridge->dt_node = dev;
>       bridge->sysdata = cfg;
> +    bridge->ops = &ops->pci_ops;
>       bridge->bus_start = cfg->busn_start;
>       bridge->bus_end = cfg->busn_end;
>   
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> index 13d0f7f999..2d652e8910 100644
> --- a/xen/arch/arm/pci/pci-host-generic.c
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -23,20 +23,24 @@
>   #include <asm/pci.h>
>   
>   static const struct dt_device_match gen_pci_dt_match[] = {
> -    { .compatible = "pci-host-ecam-generic" },
> +    { .compatible = "pci-host-ecam-generic",
> +      .data =       &pci_generic_ecam_ops },
> +
>       { },
>   };
>   
>   static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
>   {
>       const struct dt_device_match *of_id;
> +    const struct pci_ecam_ops *ops;
>   
>       of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);
> +    ops = (struct pci_ecam_ops *) of_id->data;
>   
>       printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n",
>              dt_node_full_name(dev), of_id->compatible);
>   
> -    return pci_host_common_probe(dev, 0);
> +    return pci_host_common_probe(dev, ops, 0);
>   }
>   
>   DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 58a51e724e..22866244d2 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -37,6 +37,7 @@ struct pci_config_window {
>       uint8_t         busn_start;
>       uint8_t         busn_end;
>       void __iomem    *win;
> +    const struct    pci_ecam_ops *ops;
>   };
>   
>   /*
> @@ -50,10 +51,41 @@ struct pci_host_bridge {
>       u8 bus_start;                    /* Bus start of this bridge. */
>       u8 bus_end;                      /* Bus end of this bridge. */
>       void *sysdata;                   /* Pointer to the config space window*/
> +    const struct pci_ops *ops;
>   };
>   
> +struct pci_ops {
> +    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                             uint32_t offset);
> +    int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                uint32_t reg, uint32_t len, uint32_t *value);
> +    int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                 uint32_t reg, uint32_t len, uint32_t value);
> +};
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_ecam_ops {
> +    unsigned int            bus_shift;
> +    struct pci_ops          pci_ops;
> +    int (*init)(struct pci_config_window *);
> +};
> +
> +/* Default ECAM ops */
> +extern const struct pci_ecam_ops pci_generic_ecam_ops;
> +
>   int pci_host_common_probe(struct dt_device_node *dev,
> +                          const struct pci_ecam_ops *ops,
>                             int ecam_reg_idx);
> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value);
> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value);
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               uint32_t sbdf, uint32_t where);
> +
>   #else   /*!CONFIG_HAS_PCI*/
>   
>   struct arch_pci_dev { };
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:34:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:34:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183017.330929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIJs-0002zc-M4; Thu, 09 Sep 2021 11:34:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183017.330929; Thu, 09 Sep 2021 11:34:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIJs-0002zV-J8; Thu, 09 Sep 2021 11:34:16 +0000
Received: by outflank-mailman (input) for mailman id 183017;
 Thu, 09 Sep 2021 11:34:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mYjb=N7=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mOIJr-0002zK-5w
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:34:15 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id dbacd8de-1161-11ec-b1ae-12813bfff9fa;
 Thu, 09 Sep 2021 11:34:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbacd8de-1161-11ec-b1ae-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631187254;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=A31SC3K57yoskmMRtA4jxh9pkrDlnp5Vw08nURW03Oc=;
  b=hHxXOVA5KXJ2rAPkuDFUN7om41bA429XyBPmWLXQDXwyrQzp+6sOem64
   MU8bndkOswNslEXH+nlifA00r/9+AB2mRKoaW/C5afu2yhueK/Cgxnvqm
   r1kKhk5NaDac1ZwXhPi75zn7hwxW3pxS5wBJ7zLUkpeAwRQV3lx7QuMYt
   Y=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: lxtmIx3RJKTlGWXmlpCxSB6eULd/P5BwmRO9hYBjqfvz0P7qrLtW9xtl721yAKGp3SvfBrgdzE
 6jr2pKYneGEgH9Pg/sB56syjoGMHXLBBWbtkAH+oWuyTiLybNi8CNtJxWCrrJDYgxLf1YkoHY4
 fvl7gOqTOtKYhmYycb70WaXHy5ouY5jZHCSB3TnuLY6DQdYTzhax1prwpHWwBJFVIawhiVDQ0s
 EOZT9gaS771T2f2LXGMC1eBIhj0dtNFN6pU4gn2Vh5JlWA4++f1/+4VGwC8HjxtmP0xnZSMQR+
 rb5qJ7lMBT2KFl1MmAyqm3GN
X-SBRS: 5.1
X-MesageID: 51941907
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:Xo7vB6wtOfX9UJFoHCxlKrPxbuskLtp133Aq2lEZdPULSKzo77
 HcoB1L736E8UdgZJh/o77wXtjwfZq9z+8Q3WDeB8biYOCGghrpEGgG1/qY/9SOIVyyygcw79
 YrT0E6Mqy6MbCV5fyKvTVRPb4bsYO6GeOT9KHjJ04Ed3AwV0gC1XYxNu4veXcGAzWuZ6BJWK
 Z0vfA34wZIEE5/Bq/LZAhjLpez1ay+qH//W29idmsaAUu1/HuVAdbBYnulN3wlInxyKNkZgC
 b4e82Q3NTej9iLjjP76k+71eUU6YeRleerx/bntiCkQQ+c0jpAqL4MZ1WO1wpF5d1GGD0R4Z
 vxSt4bUqsDkQKtDx/F1mqa5yDQlAw243un416Vi3nurIjYQ3YVEMxcnOtiA0Lk13Y=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="51941907"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cpdw5A444n7/PMecXpNTIFQQRZn8jdW0huYC+tD8hoA0S/v3WxS72Y8/Vx9wcAfPuy4UcdcvHvmD/CRJBoNrqMJjqmfV5cPjFvx02VlbqcunGrv7oudkKvVjA/ypd251cQC98nLWWW1n3fFGmJodBTTSdUcmt39Mnn9Se9BBmqKsEvKOVQXkozc/vwuxKnuUecXz4OUsts7D5vj3l9s19eUfcsdNALPQal2y08n6Zxq/qlwaFMJBvMSqKU0PJ9aSTzbvU031haOhijJj+45Z7HCCuvAsuUnepSzwfSCxmzrZ6F63Eo+5wwtqUdPh3My43YD06cxm9vDVAVihVqqujg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6nn9WxfhhUGRAcX7rmQNrfi/mWoK2G/7C/2HhLQ7WOY=;
 b=FYiBYkeGx3OQQZJ89OuU7acc4J6D++U7fIBg34cPL5vdfYxMnjMsPlYKtTT2MymUBsXeVr2+t6hU8HKkRP0Q+cqBm1p+OB6LPCfSYK5Lwr/wTgfKCGPGRt3qXBOO57kH4+yYuiCTxaXLKqvWwIwazvwgr6qA+3TGp9S/0lGTWObYK3NpuHcnzEcNW3pp1vW056NweAKIefV1UmdcPnNlUoqZqFuVDiezuKvE1osAIyrpy4x1Rom9RSWouwcQmohNYHRIgnG/UWtRKGQcIE+/+pYYEBmmEVjFr2qOwS/rig2AKc3XiobA2UBu1ErqiitPNo1uZKVp7oziT8qSbZM1Wg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6nn9WxfhhUGRAcX7rmQNrfi/mWoK2G/7C/2HhLQ7WOY=;
 b=LOj0Ah76fdqa+rAKWbMQuFpo9dq349EZYYQVwRjzwZV6kdinPNk3qPbr1cVkMlmzKF6t+JrgSg/paUAG6OyxdaMk5nHyew6p3hIX0EElg8ova44g1X5QT+vgM+KP0YB4/nluV0KKlGxfmo/D6bQ99LvYiclxav7hcURcBy61e8k=
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908161918.25911-1-andrew.cooper3@citrix.com>
 <3f272ea6-02ab-8f70-19cf-28a92421531e@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] x86/svm: Intercept and terminate RDPRU with #UD
Message-ID: <9a4392f1-dc8b-76aa-f5c0-1db70dce3b42@citrix.com>
Date: Thu, 9 Sep 2021 12:34:00 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <3f272ea6-02ab-8f70-19cf-28a92421531e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P265CA0016.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2ad::10) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2cde61e0-22c5-4d9e-b299-08d97385bba3
X-MS-TrafficTypeDiagnostic: BYAPR03MB3861:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB386121C81A720F52D99C688CBAD59@BYAPR03MB3861.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: h4tHRmIAC9WtW3wVLBsWS3aGsGCsFyBJb97QSP8zEtiLSL0bUnhvBXAY9kOgg4B2g8TEC7yYyn7U3NC5xnvtvAyYxG39aw2VVqu/BWoYNKrfAzkWgAI+86vdwXLRPmcJsxXqHwrfDFdBECi8aIgylN2j5LqfPHGumczuBccAASLK7sLIHH3V5qMyDjPj6TvO3kpk3D2kPec0SaXg69dL/s69hTZzJJaKIdahXznWM/wAt0zcL2kWE547PkQhHWqoulXS13dV2xo45eeRdb+LwNgdee1hOk4R3u4dOQA8weru/Z5FVOUsuVwLxi70muBqbmqEtjH4fhZonH6na20SQpkctK/DCwkzm2HJkDKxISbhw8KckbvwoD/AraoAW18rnHI340ulZmaipU+pJnd5ImNncc7ly3gtgKdPR/HGQBo9EdntmYv988KRMbUTU4vBZWo/+A3Hl3cuMyyBQUYefGHUtIjHcNws0hiz7Hgx4+BLVYtg2uRIs58yUBScjiZdDFYont/JVO9GJeKTn1EjM+ebO39wanGZG284Y9dR+/ZiAjuzix9jgmez/G7yJ3SXMSLtTo2SbbfttZsECmjAvRvEbU0f01dCNkSb2z5hitkmJSyPc+AlKSfKrAoUt5tdsCklJFXKykW052rnC/9yEB5xxYRHTLEgaeb/tw1ws6iggNOXvrTRXSiyif3LBl4LYWOdadRKhpBFIYkZ/ZJcWpfStXSA4qvxvhDeEC35Hyk=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(396003)(346002)(366004)(136003)(26005)(6666004)(6916009)(36756003)(31686004)(5660300002)(186003)(31696002)(86362001)(4326008)(478600001)(53546011)(55236004)(83380400001)(16576012)(54906003)(6486002)(66946007)(2906002)(66476007)(66556008)(38100700002)(8676002)(956004)(8936002)(316002)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OTZLV1U3VDN3b0JRckJnQVZ5OGFCNVBYZGpDVHJsaE03bEhNbDQxci9Cemp3?=
 =?utf-8?B?ZnVwLzRnTldmWHdvMVVZcHVVc3hPT1p0NVR3NDNoWk9hb21ISkljSHZ4aDlm?=
 =?utf-8?B?d1Y1OGFFSTNnZTJQVDF3ZjNHOUxRTGoxbTJFUVJKRC9IWjIvc2xodXlOeVpW?=
 =?utf-8?B?RmtqVUJEVHNGVU1YMlBFOStaZ3lnL2RJMmpUVFlYYkU2cXNnUUpzalVmVWlP?=
 =?utf-8?B?ZXY0MnZVdm0xMWM3NUVaQXVFOUQ3ZVN1anR1SjM5ZXlXN0NPTFprYjVqbDJm?=
 =?utf-8?B?NUdyeEN2Wko1ditjOW92d1VFZzI3R3h3WXgzNk1ITU5NUGd0cHo0UHo0VFdI?=
 =?utf-8?B?UU5qUk10LzNXd2VyS0R6ZURVYytzdlZzaTJpNlh2VW5TdmRBcE1sK2c1ak5m?=
 =?utf-8?B?RGZrbXhDaUVPSjFFSTBVajc1M0hkbEJBNVJMaGkvWG5uaUNkZEY2UFI5R01X?=
 =?utf-8?B?RXNCU1RWbEVGMUJJdXlQS3ZwZ1MxT2s5TFN2Y0g2SWtLcks5SWtuZmZZbGhm?=
 =?utf-8?B?L1ZNTDNKVWkzaWNucFZRdXpoaUZQWmhFeG9BSDdjQWh4S1BYVDBKZlQzS1RU?=
 =?utf-8?B?SnYwdXlRdjc1Z2s3RVZEVGpyT2htcUtYdXVnWjlOb3JrQUVjUklXSEhySmRU?=
 =?utf-8?B?eThzaUNmM0FOZkFySm9qZk5BL0VJYzN5ZjNrL1pJaUthR01OZ1RueURRTHFL?=
 =?utf-8?B?U01XT1dCakdSL05hSjFjUmlHYUFZMEtlMm1ra2NMUVhOeW1uM213eVFDTFF1?=
 =?utf-8?B?enJFRXRtQTVlY0dYWFJjclB3N2Q4bnR2VGlrRkJPdnRZaVpmTzZkMTZzcHNo?=
 =?utf-8?B?MHloRjVXSFVJWEQ5Q3Q4cnZUcW9CUkNXSVpySjFqZkV5aHlnbWRsOWZFT1p6?=
 =?utf-8?B?a0tTaGFLSUV3dDF0bFRkVW9uMWFCQnFZZlBtU0lIcU1hRG1XL1dyeWlaQUxa?=
 =?utf-8?B?TW1UeGNkQ3Uwck5HZHRTU2cwMVFMU1lhU25vSnJkbENYWjgvY1JyZTQzcHpN?=
 =?utf-8?B?d0JmZ090TzBwZ3BJc0UxbnBiNDZiOXd1S1dGSFB6Wk5OQm9XWnZ5WmwrSGdx?=
 =?utf-8?B?U0VtNTc4RHFjK2REb2d6L21LS215TzNTVG5SSUVjQUw3Q29xRmpGUDA3SFFa?=
 =?utf-8?B?bmNtVmZtZWIwVDR5ckVzT3pJWkhzMEpCN3JIblRtTDhHWmJSQ3pWdlpFK3V6?=
 =?utf-8?B?WURSVXM4M1JUVldyMVFxYisxczkvNHNnSlZQdnNrSk1pRWVqR2VTT0lDN2pT?=
 =?utf-8?B?Q1BmdUkwV0prTXNXMGN0RlNORnZzQUVWVXcwN1oxYmhOUkJ5MW41RnZzOEhp?=
 =?utf-8?B?MSs5L21xZTQ1VHNRL3VsY3djS0hEKzFaam9Nc3hGYmhLaklOeVkzZWtZNEVH?=
 =?utf-8?B?cFR6STdtdWJlb3dwcHVGV2w1NVVpUC90cGNJUUluQzhlRFArSkIwaXE1UmJN?=
 =?utf-8?B?UXhFSHZyeGRTd3crZ292VUcxUGdlM0g3QWloRlJiN1k0K0lCVWwxYXVrY3hM?=
 =?utf-8?B?Rk1kWk9lZHBXa1RMUWJaZ1lYQVV2TzFwVUVOYnlEQmZ2SGJ2Sk1JblVPZmJN?=
 =?utf-8?B?dU8zT2g0YllSQllraDNkbVFqa2NadVl1a2I1TGdCRzk5dURReXdSUmwzVFhY?=
 =?utf-8?B?cDNmNFZjT2R2ZjRZaHhQUEplSzFWV0VRQzYyTldPc0tpZUMvNVdCR2UvL29B?=
 =?utf-8?B?VldiK3ExUW1LRjhPUXRScjJnWENjRnl6RmdoSXgyK0FiUEJIbU8zdHBVdjAx?=
 =?utf-8?Q?AcspQh8Foh2bbLvS9sWDFRomJTOSBSMC7uulPRx?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2cde61e0-22c5-4d9e-b299-08d97385bba3
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:34:06.8836
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BLRNn+xTOs4YGHqHZqv1MdRydPu/ZiWGe63OVU9OiCSSz55QO3SRhrz1u7mRFHgdawRtASGfV5nXCucsXMEK1nNrlS2Z0X98LTzlJOzHB78=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3861
X-OriginatorOrg: citrix.com

On 09/09/2021 10:57, Jan Beulich wrote:
> On 08.09.2021 18:19, Andrew Cooper wrote:
>> The RDPRU instruction isn't supported at all (and it is unclear how this=
 can
>> ever be offered safely to guests).
> An implicit hint to me to consider "x86emul: support RDPRU" rejected? Tha=
t's
> still in my queue waiting for ...
>
>>  However, a guest which ignores CPUID and
>> blindly executes RDPRU will find that it functions.
>>
>> Use the intercept and terminate with #UD.  While at it, fold SKINIT into=
 the
>> same "unconditionally disabled" path.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>> CC: Wei Liu <wl@xen.org>
>>
>> I could have sworn that I'd posted this before, but I can't locate any
>> evidence of it.  I've got a separate patch adding the CPUID infrastructu=
re for
>> rdpru, but that is better left until we've got more libx86 levelling log=
ic in
>> place.
> ... this. Which - if exposure to guests makes no sense - would seem prett=
y
> pointless then as well?

Well - given how many people want aperf/mperf, I suspect we might end up
having it as an opt-in only thing, so no - this isn't a rejection of the
x86emul work.

We can *almost* provide a safe way for VM's to use this infrastructure
if we had something like Linux's restartable sequences infrastructure.=C2=
=A0
In that case, only an SMI would mess things up, from the guest kernel's
perspective.

On the other hand, it's probably easier to lobby Intel and AMD for an
interface here which isn't fundamentally broken in the face of NMI/SMI/etc.

>
>> --- a/xen/arch/x86/hvm/svm/vmcb.c
>> +++ b/xen/arch/x86/hvm/svm/vmcb.c
>> @@ -70,7 +70,8 @@ static int construct_vmcb(struct vcpu *v)
>>          GENERAL2_INTERCEPT_STGI        | GENERAL2_INTERCEPT_CLGI       =
 |
>>          GENERAL2_INTERCEPT_SKINIT      | GENERAL2_INTERCEPT_MWAIT      =
 |
>>          GENERAL2_INTERCEPT_WBINVD      | GENERAL2_INTERCEPT_MONITOR    =
 |
>> -        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP;
>> +        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP      =
 |
>> +        GENERAL2_INTERCEPT_RDPRU;
> Some of the other intercepts here suggest it is okay to enable ones
> in the absence of support in the underlying hardware, but I thought
> I'd double check. I couldn't find any statement either way in the PM.
> Assuming this is fine

They're just bits in memory.=C2=A0 Older CPUs will ignore them, and indeed =
-
pre-RDPRU hardware is fine running with this intercept bit set.

Honestly, I've got half a mind to default the intercepts to ~0 rather
than 0.=C2=A0 For running older Xen on new hardware, it will lead to fewer
unexpected surprises.

> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:43:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:43:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183024.330941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIT5-0004jn-Jo; Thu, 09 Sep 2021 11:43:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183024.330941; Thu, 09 Sep 2021 11:43:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIT5-0004jg-Gb; Thu, 09 Sep 2021 11:43:47 +0000
Received: by outflank-mailman (input) for mailman id 183024;
 Thu, 09 Sep 2021 11:43:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOIT3-0004ja-GK
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:43:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2eeae68e-1163-11ec-b1ae-12813bfff9fa;
 Thu, 09 Sep 2021 11:43:43 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-LQE3Kx4ZOfiNcJnYv1Xr8Q-1; Thu, 09 Sep 2021 13:43:41 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 11:43:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 11:43:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0004.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 11:43:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2eeae68e-1163-11ec-b1ae-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631187822;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HF0S0KYJ4nB0qOMnXGPL8jt8y++5z9GqqRNKB6MARN0=;
	b=JK66C1XgO8TmLSTxUokS5LmFnoIwIxQJyGHDhZ6roy8q8RYUNAcV4BPr0LO7PEnGQQceB9
	UozVIvESeu6b9/4PECthERnpEcNgiGzcc9Uc4/dmV78mgkXYHTkBGU0RT7I6q/rzAY6Qry
	g5EXlQF+hM4KZadGxAR5yD8MX5zUR64=
X-MC-Unique: LQE3Kx4ZOfiNcJnYv1Xr8Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VIy+oDLGnRjpgsKvqpolAjYIr+V5PQOcRojrv8z/qmfm1viYMd3yJp0ImtJNh851qP86v455b3ekouopqidl2/JemaFr5WhjsHBFpgFgh7AuyQb7fZJe/VA3KfWezDDgbscCvtZxPNY2OKYfQ7gNxdC5PboSjGzuM1Ez2PWI2SPaVQWDAahFzPyIBAvCCpivl2k0VW8mHMhCJGFIzdzXqltqZmlJwJChrPfU5Rl1KZ5QLaiqSFAY+I+6qVv8A0izRgihD5Bxvlxhx/aL/NfxQINBw9Es3b1U3gpjsKpWdELMOnfqj+NMYA1hy46K+6ExIpICQ5U5qHgJCjh9tWt+9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HF0S0KYJ4nB0qOMnXGPL8jt8y++5z9GqqRNKB6MARN0=;
 b=m231F7oqGHiCrc8r1s6Eq3ap/3YW/n+jHc98Z4eyHI8UUfWvHUPzKnIejqGQR4z7xcGgZZ8z+4TsRCEw9vk1GYXiiV3BfllEKMhnaNsTNyy/jAnMVanLCwym91fI0oJhvE/PiA0yvDGQlWhAEeVRRgASVwy/E3Wnt95782nub00dn55RGARMGAc4KObzd51PyTwauDrW5fC0NoYmcv9IwwIZ6DRv7TTo+mmKS2a1OiwEAa9U4RJdGxnppPeM5c/sDeQBEcWRUd/Y9HRR7K2Mvyfhccmt5wxkRlIfm8bD6co+11LtryXs231wyIzLekfZgdAQ+gfGPQM96ILTTy+fsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH 2/2] build: add --full to version.sh to guess
 $(XEN_FULLVERSION)
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20210908095422.438324-1-anthony.perard@citrix.com>
 <20210908095422.438324-3-anthony.perard@citrix.com>
 <be89ae8c-99cf-5385-00a7-d9433e0fc50b@suse.com> <YTnrnqz7Vt0pugFZ@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d4b99e9d-7c33-2edb-c53c-548e57d8514e@suse.com>
Date: Thu, 9 Sep 2021 13:43:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YTnrnqz7Vt0pugFZ@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0004.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e4efccf0-9e14-4a9f-30cc-08d9738710bf
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7039CA57C611BC934FA9A6E3B3D59@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QtPOLpxksWt/RvQ6PuSN7rHHpxiO5AGJSy0vfnbh+fAeVv+Lf3Btl70sTYvSYwN9N9bpcQSwX2bhZEf37HaROC0nrb+E0Jx3ccIYGp3+96ApEfSHgrQp7i6lpHkuu0YRSN959a1JPhFzZWjXDze5G7YucokYKLoByoUl2xZw7+rKTP7z/5C5RtaCxP525IE03CEIywxCaCXlplILREX6BOUNOcunzXTrEWzzAlCpO4xSdsBs5akWRjWvALxXrHGeSb7Oru1kmRuhZr53FeIm1inuirAJDiICg5QB4MWY4ZCgK1OP/HUD4X1Ob9Ir3u3AYghRn8KAvMCeheVg6O7vGUPR4vpJXcypf6HAg60aIualBvQupQLKxs6qPX+ikvVWrDk9QxvCF/wSB05P1qaPJFKhBUleWeRiHS2nqWL2dRP80pwBvzATI9K9LEyVff+A43Vb2PyipFm58tsFGRe3LmHDArcw6cUheWz9PqiX8FzDmDl9YEVHL6O9U5yUVvflbeDmbVhrPxZr7kun1Txh98gFBLCZ/P7n+5wonSAivIoS3ZshgZ5RBxFtwcByHKpUffs3lxY8T8uOAoHtwjKI7CpI/8i/JNGcqTkJ59QX0NJsD/d0tDY7V4HezPVFYA66wR89PWu78D/GSiII5Fc6/U1VtqZdiZ4NetxEC+GjXcHPPcKBt4qLqOyi9OwjyvYk2Efuj2sSmx/JaCb/+fRQAC8vPi9WxjEiU8HRUtDDzChVoWkUpNSviB/xQuTggkb9
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(366004)(136003)(376002)(346002)(66946007)(6486002)(38100700002)(66476007)(26005)(66556008)(7416002)(478600001)(2906002)(31686004)(53546011)(16576012)(4744005)(54906003)(36756003)(5660300002)(316002)(8936002)(86362001)(6916009)(4326008)(8676002)(2616005)(186003)(31696002)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NGhISDN4WkNpUkkwQWdhbHllRFVIMU1oTEZmN3JlbE5WS1lvbkd4dU9lY28w?=
 =?utf-8?B?RWJBNTFVdE1wT1o1djBBMThodmg3Yy9weU1oMmJnOFUwU2YrNnY2RDV4NnJE?=
 =?utf-8?B?c3llZjcwdzMva3N4MkYvSGx0ZGZBdkEwbTBuZStuT0xOcHd4MlNDd3RacWEz?=
 =?utf-8?B?ZmtWWjM4V21Xb1lGcXMyZkcvd3pGb3VTUzB2UjhnSEUwT0RENDY5aGlSendM?=
 =?utf-8?B?UmF2YmtublJPcDhuU2l3dW9yUTFld0xxVkRJc1Y1VHFpL3h0UWpnSVB5ZU1Z?=
 =?utf-8?B?anAycWJyK1lpcVFWQ1U2eC9YNEFhSXRMTStrWG1vbFA1UHlCemtNTzVVbmFi?=
 =?utf-8?B?UjhtU1lKZlNydmNFRDVtQjlWWnZNWTBsbFFVajVRcnpLSzdMSDJvalg0TERj?=
 =?utf-8?B?akpzNjR0RXplaXpwcDRuc2Fzd3M0T2J3dGovdXIzZ0NEN2ZVaWRSR3hPQXpR?=
 =?utf-8?B?bnhzcVpTY0s5eGgvUUtyTmZJZHVucUNrSnhLakdXNzNzTTlWNjJyKzl4djBv?=
 =?utf-8?B?K1JSVU0zUytqSG5EWFQ4N0VMWE5KOFNjYTJGMHU2WHFUTnlnQjVzVm9GUXBa?=
 =?utf-8?B?QkVhajIzdXNxcTdFLzM0bGxhUVlwVEdLZWJkaTZlQ0RKL0JBK2MvRXZaNUI4?=
 =?utf-8?B?bkJzMGU3RFMxZmwzeUlKKzlEQkRQK3d1VytyVEgzb2ZrbFdvbjRMbk9yc3Ix?=
 =?utf-8?B?bmRmbGlQL3ZVMDcxWWJHbkJvZFJFMkU4VlFDUk1xY3Jyc2lBTG9VbENZTXpS?=
 =?utf-8?B?S2pUWC9NVmJCby9GUUtLWnFZcEdqU2lqWC9UQlpKelNYQU1DZmN0WnFjajRk?=
 =?utf-8?B?OThuZW1NR1BiQ2ptcWpsZ3JKajdnQXFsbGMvVVVMemo4aFZrRm9rU3FSSFUw?=
 =?utf-8?B?QmdFbTRHL3hjQmZ2TzNYRDJwelhsTUtrNGw0dzM2QUpjVEpBdlFxc1pZSW9J?=
 =?utf-8?B?S2RFazhyMmVvM0I5MkxFUXhEWUh4L1kyMWZSZVYzdHRiN3ZkdEtiUFJzaTVS?=
 =?utf-8?B?VzlBMFlBTHplYTlRQi9xVG5ERGZnTzhXQnhqdG4yL3RFWlF5SlVYQTVNQW1G?=
 =?utf-8?B?c3VDbGFWZTFmU28rcUNnbUxNaHgrMkVmd0FpakFhakE2UERqa0lZN2V5U0ta?=
 =?utf-8?B?ZG9aLzFtL0VEUmNLc0JjY0hzUEpPK1ZJWVVLNGN3dDVzckU3NC80RGxCcnlp?=
 =?utf-8?B?VDlob1cxY0o2Q1ZQSDd0ZTY5bWlXUEF1T2dGeG14TGZkRnFvK25PUElOZXVK?=
 =?utf-8?B?dldrbEdyNVpaak5jdm9OeWZnRXYxQ0FrdVcwd0MzMTBIcjZMUzV0R3BNT1ZB?=
 =?utf-8?B?ZTFhMTRheGd3RVJqMllkcDJQOXh6RCt6M2lkei9yM2xkS0JENHgwdExwSnhF?=
 =?utf-8?B?UXpPV2ZZWVdma2trSzY2T3FxeUFYLy9BWXZIUnN0K1ZXT2FpZkdkRjM1czZE?=
 =?utf-8?B?ZXpuZDJWM2s5UEx4cmRRYXZ0a2dWMnpmL1dDcldHaDIwbVdJY1V3blhFa3Ji?=
 =?utf-8?B?MVIreFdKNnd5R1d4U2JyQ0xuWlRzNTgrVlpoMS84b3EydnVnVm5OTnRkdjNj?=
 =?utf-8?B?MzA3akZqTU1rZ1IrNGtHR0VQQ2NObkNlN0ViN3RLczJET1FhUkVscVJVVnpQ?=
 =?utf-8?B?dW93eEo2M0RuQ1hsVnNmY1RsWmJ2RHM0S0JnTFdjRHVHWDUwZkZnOFVBb1JH?=
 =?utf-8?B?MjJRVXd1NmM2YzYrYjQvUUZqZG9iV0V4MDNPanhoNk5hcmowMFBrOEM4M3dx?=
 =?utf-8?Q?JBND1aSQH0wik7XPKcSJkkAlwXpXjhP3d/r50PK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e4efccf0-9e14-4a9f-30cc-08d9738710bf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:43:38.9659
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zgGaIkwqOqQEUn45xIbttyjoyWvuuj0ZWTUGCPObnSpAJamlVSN/PQ3T8Hzs/IW4NWydniTu3jaS+4WGoChcxg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 09.09.2021 13:10, Anthony PERARD wrote:
> :-( I think I just found a package overriding XEN_VENDORVERSION with a
> command line argument, the AUR package. So ./version.sh needs help from
> the build system:
> 
> Maybe
> 
>     XEN_FULLVERSION=$(shell env XEN_EXTRAVERSION=$(XEN_EXTRAVERSION) XEN_VENDORVERSION=$(XEN_VENDORVERSION) $(SHELL) $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)
> 
> or probably just exporting those two var in Config.mk or tools/Rules.mk.

Either sounds okay to me, fwiw.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183035.330960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIWQ-0005Tm-5T; Thu, 09 Sep 2021 11:47:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183035.330960; Thu, 09 Sep 2021 11:47:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIWQ-0005Tf-28; Thu, 09 Sep 2021 11:47:14 +0000
Received: by outflank-mailman (input) for mailman id 183035;
 Thu, 09 Sep 2021 11:47:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOIWP-0005TW-76
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:47:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7d0ba1e8-3f70-40cb-a175-bcc29db97450;
 Thu, 09 Sep 2021 11:47:12 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-FCqiz0XqMk6rcmwqthxy6A-1;
 Thu, 09 Sep 2021 13:47:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 11:47:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 11:47:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P251CA0017.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:21b::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Thu, 9 Sep 2021 11:47:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d0ba1e8-3f70-40cb-a175-bcc29db97450
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631188031;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sULp+l2kqAgCSokCE5w2q5oOsJM9PUFaGgsREdzx00E=;
	b=YYaYaeMlc+PmE5ouMJqd0qoMT4SZGJv3jlv0e3NR7Id1aQDkwGkNIBkUpeHToIjAx+tVGE
	vPZAFUOx/PMwgFepP0oXoNcXJqX8ZV1xdaEM5dGFuiJgqECKL3gYAgDRsBPMGO8dznx6Nn
	o6MFqEJBiE8hXSBH88i1Bmw3ze5xsus=
X-MC-Unique: FCqiz0XqMk6rcmwqthxy6A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KDBVUTZCF6obwKU3FSpBfoXBvrxSb4WTOk5+IJSglRSP2r0Zgu8/eXhWwlnKbTZidVSgyAFXzoYu125xhVy97ywsWDFayyIgUVlAj2RrwCGBANfUIp+mRzjFBd9EjtNPa1A3dwP2jZfxBwankccqvw/PQJYkHSkyAnSTgODupxGMOgsv/ELXAUeZHZyGaPzDq/qsm2Dgz4OGYfCwIw8ANXYxYpmlDkdx6qaFShe2PYZEdip36N73/LCYziPNTwGBx5Qikx2vWots21px+wuN/eJnBX86KVpvJcsGErEIrE5Ux5pIlS5CBLDmyNmzwr4z5Ur1BtN7PHb3U3YazJwdrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0JzcPlTJl3BbF5ISgULEF5qWV4rb7xfJfe1oGQ4LpKw=;
 b=OizRKQGZRz+Cs/ditIXgmd444vnnSqsvPQEXucIxrKH3fwqxJtwcp1OZ8bfs79yoA0CEOCPUiwESxKqxPSIb4S0CBxLYljGtu1OP/8gFJNgv4PK9Dsdk72/dAr/WCNDBxP7TvTiFaInksK5o0VZn9dehCKYfV+rhUZZbWqICK7TzQ3gw023wE7q6JEMKMtl1TM4fPn3fgAjdT2OSn1+ezcG+MPyDK7ldEe2kowrYFeefR2bwd54v3JrV9Nhdi08UEEmzqBZdb8kmSIkIVbPzVKNyTFWpzlvdaqYEcf77s5fTWxmUXhOzEAMkJ1eASlOISWJjSziGAiTCfRaVOVqfiA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/svm: Intercept and terminate RDPRU with #UD
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908161918.25911-1-andrew.cooper3@citrix.com>
 <3f272ea6-02ab-8f70-19cf-28a92421531e@suse.com>
 <9a4392f1-dc8b-76aa-f5c0-1db70dce3b42@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b4603360-c105-8f69-2c1a-081ea2605e05@suse.com>
Date: Thu, 9 Sep 2021 13:47:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a4392f1-dc8b-76aa-f5c0-1db70dce3b42@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM8P251CA0017.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:20b:21b::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40fa862e-8f0b-4c09-89a9-08d973878e64
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB703764830F24A5525DDB4A18B3D59@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	452srsurza27m6JWSXeRkVxBZo32AHWDdsPxlKAsPjOpi9UOOe+0sIXWkr6BocBambdP2mccLHBHAojjLRXR1gLYDPpy0qP6o1ibjhBqGf3sXt+zCJCzKf76TA5MZKVzH7tRu94y8fIEUAi8nyf3f+ImykQw3c1uO2d5Y3Vv0hq25FB3CfYEBoOkEZmsXf7jXPmE2UV1SMMWgMk8lj9Qy9OhrR8ktBF4wDBkr34KYNb/9LT9I8n3NLp+OPhdNGwKwSbfKJhfzqNkblOfRCy4ZJVd/inlXKVQPk7JLPPhS54IIdt1u9amt6XC/Qf1rk3U6h7vEJoAyoVKeETT8FG7Yig+3tSHZO6XicHAoSCYOP8mbFTxCLFKn4ZOa4Nx/iDjNVIaNbU2Xr9DupQKPgiS/961JtAP8b1cOcYgFv9qmupTOSBR9yHa3z2L9Q5B4AsXvXPwI8+LGSyNQbEx/tl9xHqK/7uu4Vmqap1Nc3jRXEm/IBQbvbb4JftfDGmEZZRWA5pny4r8hUPr/pUmEBPEdFBCCSuiJ/sQu3OSAKYO26knjidimTW+CGNPE9UiZzJ7itUEwMz+8zVjYmC8ywKwiHbhTh80+IRiJPMpFURB45MqJqXqBvU8xYCL1c+u/ZRlRu01QvWmyQ3HxCJBpDIRhp5frE23RDy1eKjAczzSNAYzJEJGrfAXXUeUSLgA7E+RCuzh9losQ7L9Noscr1vnyJd5rLd1q90gPT/pZCQBZtkdQMio8/VRpjm1vunmG5vD
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(136003)(346002)(39860400002)(2616005)(16576012)(956004)(26005)(186003)(36756003)(54906003)(2906002)(316002)(53546011)(478600001)(6486002)(5660300002)(4326008)(8676002)(66476007)(66946007)(66556008)(8936002)(31696002)(31686004)(86362001)(6916009)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?2J7XD/JbpZwEQR6rFQUpJ+m9nIZYJ04sVLS5SFTszjG5U92bnoMB4q8i5bgO?=
 =?us-ascii?Q?hN8YhuMwbLX6u0deGz+pGzHOf5ngW0cmOWgqF0DGNniW/rI6HIy4zxStTmat?=
 =?us-ascii?Q?GepRrOitD937X3tcsrXZcz1we3bKfJd3WRRkMM5J5YlyF//MhcwmVDWVwpZv?=
 =?us-ascii?Q?iiRcIUOsKtZSg7A+Rxhr4cpeZyp6YC3xqfSluq5oBfrRGKNPQZ34vGm1yzmm?=
 =?us-ascii?Q?kPLkewhI1FdV55TjSZPHYblusfR84EBiFDGpDtzT/VNKBTDom76ygOaUR/0M?=
 =?us-ascii?Q?EIWHjqHjwu4HZKASB+LKMWULc5aA0JeqvTtVMP10D4cVeImIt93h4MPVDftb?=
 =?us-ascii?Q?gmvG6pTAIwl19LtmxhDt5DOhfwRjb3M7Z738zXp3OI6DhN19BH/1bsyd0O6x?=
 =?us-ascii?Q?nH9i6HNQ6KIGixECdU1My37LnNplhdbF3mB2QpCNzfTDrvF8GqaFty3wbQpt?=
 =?us-ascii?Q?BQC+hposE6OZF9stHV+A4oP/zn/5BMxc385Qf34ba8jrPQ4dOz3nQ0ZYDLzX?=
 =?us-ascii?Q?pAyn8MYfZXiEw4xa7NHvWdhwuW8bntUu4ixI7sZCAuZpUFPeRVGhRN5Y9TsE?=
 =?us-ascii?Q?omCUVaboPqHGKE3ISODYNKoZj/vRUyBFHTR4moRbz6PD1pFuihgzyrHPeJ11?=
 =?us-ascii?Q?oF6qSeCWUFxGgAIyc/gndcu3HLo2mg8e35NZYfDBvCVb3wp2IWAH6EcStwWi?=
 =?us-ascii?Q?206KuiY87A/I4nf6dE/5X4r5ehvlr86GP6isdnrRbYAp4B8j1qPwGFSKkjIM?=
 =?us-ascii?Q?EOQ1spYyyqSbVbnMs9djvJr9JiyJOGsCxHMwaMuysnBUUrXhREdr0zBlT9UM?=
 =?us-ascii?Q?r8PFyQyoEw04nzaV96hYNXr75LRVdelPOUMKL+os/0lDOGbclh/BxeGiPNKO?=
 =?us-ascii?Q?uMxnPR36res2pCOUUy/lhzqLX7iaN+to/AefdylRGVf355ddVu0JVXgJaTRt?=
 =?us-ascii?Q?7rUDJyn/EPMgeZVAyYuL++921RgKzzmzuxeJP/EJGODSpSfZCyMlsQAuVuRD?=
 =?us-ascii?Q?a3YXnfYV7H/8kI9waORoU2RtAASwE1lwbLrDMFygGXx8A03dnefIlwEm/CJh?=
 =?us-ascii?Q?uNsSDEdNBnQ2Gg2PWKb4cde1i9a1cUZ+G3n++3fPcL8dV5hBuSvvrDbvjRtE?=
 =?us-ascii?Q?x9AO+aQuu0wsRwrNdEEkBogC+tgcnfY4KAPT01S9xPaQDxfBcotNmSn7ReWP?=
 =?us-ascii?Q?b0UcdeRLYHDQpobstzoJsa9X1rS2lTLWozSH/UZXsTmtbxS0LGpNKh+GOepT?=
 =?us-ascii?Q?/DcznSlHFD0CEENjK+sw70Nf+R/7QyYqCMbzkj/DZoW4ab066EzU+dtFkn27?=
 =?us-ascii?Q?AamMi+fy6xr7LCwQS633hpbS?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40fa862e-8f0b-4c09-89a9-08d973878e64
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:47:09.7457
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Uc4FE5tBdW46BlB8wN7OUK2lvntpjXD7K3Ocgd9J0pAKZCsoNTJk6PxZKdhBSfuOSe6fRchddeAbFoMjYnpjTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

On 09.09.2021 13:34, Andrew Cooper wrote:
> On 09/09/2021 10:57, Jan Beulich wrote:
>> On 08.09.2021 18:19, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/hvm/svm/vmcb.c
>>> +++ b/xen/arch/x86/hvm/svm/vmcb.c
>>> @@ -70,7 +70,8 @@ static int construct_vmcb(struct vcpu *v)
>>>          GENERAL2_INTERCEPT_STGI        | GENERAL2_INTERCEPT_CLGI      =
  |
>>>          GENERAL2_INTERCEPT_SKINIT      | GENERAL2_INTERCEPT_MWAIT     =
  |
>>>          GENERAL2_INTERCEPT_WBINVD      | GENERAL2_INTERCEPT_MONITOR   =
  |
>>> -        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP;
>>> +        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP     =
  |
>>> +        GENERAL2_INTERCEPT_RDPRU;
>> Some of the other intercepts here suggest it is okay to enable ones
>> in the absence of support in the underlying hardware, but I thought
>> I'd double check. I couldn't find any statement either way in the PM.
>> Assuming this is fine
>=20
> They're just bits in memory.=C2=A0 Older CPUs will ignore them, and indee=
d -
> pre-RDPRU hardware is fine running with this intercept bit set.
>=20
> Honestly, I've got half a mind to default the intercepts to ~0 rather
> than 0.=C2=A0 For running older Xen on new hardware, it will lead to fewe=
r
> unexpected surprises.

I, too, was considering this, but then we have

    default:
    unexpected_exit_type:
        gprintk(XENLOG_ERR, "Unexpected vmexit: reason %#"PRIx64", "
                "exitinfo1 %#"PRIx64", exitinfo2 %#"PRIx64"\n",
                exit_reason, vmcb->exitinfo1, vmcb->exitinfo2);
    crash_or_fault:
        svm_crash_or_fault(v);
        break;
    }

at the bottom of the switch() statement handling the exit codes.
Getting crashed (or crashing because of an unexpected fault) is
surely a surprise as well (to a guest).

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:48:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:48:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183042.330970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIXj-00068d-JP; Thu, 09 Sep 2021 11:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183042.330970; Thu, 09 Sep 2021 11:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIXj-00068W-GN; Thu, 09 Sep 2021 11:48:35 +0000
Received: by outflank-mailman (input) for mailman id 183042;
 Thu, 09 Sep 2021 11:48:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOIXi-00068N-34
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:48:34 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id db7f4de0-1163-11ec-b1ae-12813bfff9fa;
 Thu, 09 Sep 2021 11:48:32 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189BRQMo001867; 
 Thu, 9 Sep 2021 11:48:29 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayhdp03f2-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 11:48:29 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3585.eurprd03.prod.outlook.com (2603:10a6:208:47::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 9 Sep
 2021 11:48:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 11:48:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db7f4de0-1163-11ec-b1ae-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iqMXMPEkV/Rrsj++2aJXCuJYgnZiRKmUo0YIv5f5WB84SXcn1mSiX8Gkw3rw/zLDm5sIzvIClFOgdInzBLeNxcfsHtEj7ByQd2Rudor8fFdhoKGK4jOJXYDI8yKcu8PoHAEOnUJMfmMxwgPiD0iegORFZ/uSfVwL0+qHOFy851zHX7PUS1RqTyCvFylO8z7tS727wtQR5uhxwRHCv+vKe1Wxpyd+UXP1sRDyH518EFdzPWgkQDK6+VSHk/hHIk0zzAksGOoZq1EP0h9idW8SrfsnHWg9fQC8fAscrXnuBMerJnhFzhB5k2CupzekyU7SBRrf+hB6Jc0gzu778SFDdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1zGTQpWCzk8h/3mKJTKON4qYNgPUrpaZOdxoShTFzZI=;
 b=hD+4YImy3J7I9ViElKXip4liyjdHyPl2J4kA5B9r4GrLXxSlc6Hcd6+VJQ0HWkjt6bfpYRii99dezQyBLwaq4hfd3hyMgQhFOaNcb+8pAiIfviBsAE2NXb8w87tZFYY1tVNBmEN7dfU93rX0nRVajzrXC7RMSenDVavfvJGhCw0CbWixpX59Uf273iVt0mKh5uhy4JcynPqur/HQeUzbdJZd6GaQhxAKSLNPEj5Z+hSfDBMNZxnjGKNN4wREeMeno26M12DqzTWSpmc37Bk6qNmrosxlhyRivfOtBQcecUt1W2tdyJcZjg1NPXRdLc/Esc4TwUZAFSyhWXaWybbKXg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1zGTQpWCzk8h/3mKJTKON4qYNgPUrpaZOdxoShTFzZI=;
 b=rylZDow9hebjCpYn2TU6PfY/lWQQFgBTM/Fm4rRAYH/Vkn/UFgTIuAclreLnE4uGFpOwX9QjHeps6YzP7mWXtF8+gFk2TjPb60bxAd8RrydkyRmIUOD51qK6eGpxWqu1vREIkYlWe+D4biNYcXTjTWp6yauFiNBwgAva+VzeQR2+pCr3Ew2Fgkv1YnMooWe1NSYeKc7S+KAZL9o8SMekcmBa67Myay2YbUOVNq8FIV/ot43AE2GXk/NqGEBlYdASRYbvksb0lUw0kErQrOPkL+n6POICLU5VkgwzH480ppJFieysFnKlEEn9YZC/4HfdXSZ55OtOPsbJqXhyPW9hSg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AIAACKGAgAApFAA=
Date: Thu, 9 Sep 2021 11:48:26 +0000
Message-ID: <38fac37c-a4ec-a1dc-ea5e-37ad77cb115a@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
In-Reply-To: <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 730ad81d-9511-4b7f-dfb9-08d97387bc8b
x-ms-traffictypediagnostic: AM0PR03MB3585:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB35855D7EA7C258DAF4FE47ACE7D59@AM0PR03MB3585.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 naFP23+ALlS5uA283g/UwysXEX9E+rcbqvlKOFz22PIWArwD+pFPChDRvm1qrsM3kxLKC/c79cHjmYYGTSuvXQV6CywVaOS+9o2X5yc7j23uNRnH19AJIz08pvFMEIhxkomePlGGndryiqHxUpLmf8m1LbJdREzubAvohBWIwpbHIcga+6cebtJmbPcm+ELkbwrJ/WGFbbqhkUfCfJyHh92J1dFxNMxMgi1+kvz5aCIAziGtWQa5qV+lCD8OexqE1sCi4kS1MU3jQCt9MjHcVG0eauksyPRJoPvSq5pEwSTxqoYZi4/qYHScMdEPbj2ONGGylzxqjIyKZkAeeSFwHkYTB4Lrk09F6UnL6U2CHyC/ZOZWTe2hO5BRSZ+83v3PL5YxKYn8W9ho/unfYjHecaNiPKMC5/GHGg08XZ80SguDEYnqzHGsZ/pkn6ntX+51QaLq15HWb0Wf/HpWHchf1WhjVv4aIW8HKHFNZtwMaq7yvLTTYUvD1WwP/KYcY8F7Aw3WUPaZusuUAmAbF2XXHJ6LxPUPRutlQodvZDdChbMd1ShaFbzt1ftLKm4gji3wANIfyfCxm+I/Vz2ZC2C0qpeOjTKurM+EYicW7Prs9mK2f9HS6TZbSHNtYCB4sF04hudSUTj+rgl0tMQUJelsw/YENWjKdPU0w2thx285SmP2vIviu6GGAbbIhYsq6iBCeHPnht6iov/gUmHVvOqUwnUu36OMtVQhNazk0qtzgtk6dSvGnhkd/Nal1Y50Pr/J
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(366004)(136003)(376002)(346002)(396003)(8676002)(71200400001)(5660300002)(6512007)(76116006)(38070700005)(66446008)(478600001)(6486002)(53546011)(86362001)(2906002)(83380400001)(6506007)(38100700002)(122000001)(66946007)(2616005)(66476007)(4326008)(64756008)(66556008)(316002)(54906003)(8936002)(91956017)(26005)(31686004)(31696002)(186003)(110136005)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?WG1jL0kvTHNLSW9DNFVXenJNaTZnZEE0VWpueXZ2dmRabkZHWFlQSjdPYVBU?=
 =?utf-8?B?UVFSdWtPbUk3bTIzd1VHdUhyS1hGL2lkV3VnRE1NYjNPc3p0UHB6cjRCS2Fr?=
 =?utf-8?B?aGhqakRyQ0drMWVIS0NxVnRFNkhyc1ZxL2xmT2xFVXB4Z3d1UDlaVUFEMHA0?=
 =?utf-8?B?WWZmOCsxakJUUEc1VVgvQ2laUVNRcnZOVEliZWpuUDhleEIrWmFWTXNsek8w?=
 =?utf-8?B?bnhKakRXSGtUQW1UWFlackVPaDg5eUF5Wm96blJxZXlUZXpIdkxPbGtBTmI5?=
 =?utf-8?B?U2VhN09pcmFZSU9BQjdMbFFqazVpRFBVNkFtZnB3OEEzYkYxRnNodENRSERz?=
 =?utf-8?B?a01TOGhBakZJRE1JQjZuNWNxbkRKKzFreEYzR1BvSENyMy9TSFVjOSsrR242?=
 =?utf-8?B?eFNBZlhyK1VFUmlRdDNkMksyVHppaFpkcnNRZHNQS1lncXhySTN1MzI3VnJy?=
 =?utf-8?B?VmNQSC9qSDBWN3NRQ1pRdFdyakx0ajV1am5vaW9adEx6MjBmZEk0ekE1dVBt?=
 =?utf-8?B?NUlibjhiVjZjMGFzU2gwVERYV0dacjRsd2ZoR0VGUEUyRXpwWGE2SEVuWXZJ?=
 =?utf-8?B?dXRMSXZWUUs0SXZnUmdGU2VBL2RMbXhkZ3hxYmE1eG9ZMFZPQ2wzZ2R1dkFu?=
 =?utf-8?B?RmhGcHhLWkJidWpXZkFOTzBRTHFFcmhqYkZ5anE2eEVPN2xBTnZQS044Wmd3?=
 =?utf-8?B?K0ZPM05tQjFFVWtNdGRtS3dTMENFSXdQdWVxWDl3L0tNTkJyTGFmb0RlVEpY?=
 =?utf-8?B?R1lGOHpZNDRZcnVwYlk1RC8wTmdVL01ZOXVha3lYOXNnYUc5MW5vMkJieklX?=
 =?utf-8?B?aU5pWGNFdUQ3UDRjZGdjL3d5SVBtVjhvZmNGQUNIOFprQVl3aVA5Zk40RjN3?=
 =?utf-8?B?Q0haYUlHTFdiTmN5dGlVa3RkbXBUQUZza29ON1Bqc1FDb3R6c2MrN3lRSG5G?=
 =?utf-8?B?Lzc2T0oyMkN5QkQvMTRLY0d2VXJvRUlNOTloV3V6T2VBeno0VUxRd1RMdTJO?=
 =?utf-8?B?RDlsdWhQZElzdUxCTS9NdlFHcksyMEc4T0N4VEJ3WFpXRk9kVFQvQnFnWWVK?=
 =?utf-8?B?ajVET3RtdkJ1NWhrYURnUmo5SXlRbjlUcHJHOFFZSGhhT3c4VTVQYlNRQTRK?=
 =?utf-8?B?TWw4VmZYVVVrUUp0WU02cjhKMVRWdXZvcmdnMWVMUFhjenRmNlJIYmlpYmhH?=
 =?utf-8?B?S1gvRjB4M2VONXdPNVl2NHFiTjFPNUpVSlp4ZU9XR1g1ek4xeUd6S0drbC9t?=
 =?utf-8?B?WUsxVlUxRlAzSGJTKzdVNnlXZzhXbDNNRXpuYnhoQzVpamZZQ0JCME5HdXZS?=
 =?utf-8?B?MVQ4QXcyTXgwK2l1THFXbzBNZllQanJabXdENGV3VmhMbEdSc1daSWI5T0ZL?=
 =?utf-8?B?cStwZlU5ei9CMUFwa0VlZVk5MTR3MFljQ0ZSKy92aDJVVzZZYXVvQktVQ0FW?=
 =?utf-8?B?ZC9VL0tuWU9EYkJhOXQwdjBtSVY5enJMbUVQUHR0VGNjcXQxTlJ6QWdNdThW?=
 =?utf-8?B?RUxWN1BMZm4yb2tXeTRTV2xxSG5jdUliZVlBM0pvblptM1lDcTNQTjdGM29M?=
 =?utf-8?B?YVJ1U1hDci8vbXhQUzNTSkp3WW5kdjZwMGZFUnoyUzNqYVZZOERCa1VyQmxO?=
 =?utf-8?B?eVg3ZDZhYytDdmMyRjVqRTFsM1FFQkZ5VG8xRkpsKzJWand0dERUSW5SMkor?=
 =?utf-8?B?bk1pVnowYjB3QjhZT3Z5V1pRYjVKTzN6NWNYNXpLOExGd3h2NktyQ3BOSkJl?=
 =?utf-8?Q?DH3poHOfAogHQfu/iex85Oam8mj9c+iHcMKIdDu?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <14AEED8C150B9D4F89F3D240F7905DB4@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 730ad81d-9511-4b7f-dfb9-08d97387bc8b
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 11:48:26.9334
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: xBtjYI5hEfExW3QYcFlhvUYFMM302t4P/9UxFmNm3bKDjw55RoSTky8SX7XjiyY5A8eSMkfm0QRPN5GIP8p1jdKDulzyM1lPqeBOZ/jJF0niT+lvlEfV3EsN2eGtkA6x
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3585
X-Proofpoint-ORIG-GUID: AvavbyhBflWFDuIbN8i3fW-JILBYT4Pz
X-Proofpoint-GUID: AvavbyhBflWFDuIbN8i3fW-JILBYT4Pz
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015
 priorityscore=1501 mlxscore=0 mlxlogscore=999 malwarescore=0
 lowpriorityscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090072

DQpPbiAwOS4wOS4yMSAxMjoyMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTA6NTAsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTE6
NDMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMTA6MzksIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxMzowNiwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDExOjUyLCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTE6MDcsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gT24gMDcuMDkuMjEgMTE6NDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+
Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTA6MTgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4+Pj4+PiBTbywgaWYgd2UgaGF2ZSBhIGhpZGRlbiBQQ0kgZGV2aWNlIHdoaWNoIGNh
biBiZSBhc3NpZ25lZCB0byBhIGd1ZXN0IGFuZCBpdCBpcyBsaXRlcmFsbHkgdW50b3VjaGVkDQo+
Pj4+Pj4+Pj4+IChub3QgZW5hYmxlZCBpbiBEb20wKSB0aGVuIEkgdGhpbmsgdGhlcmUgd2lsbCBi
ZSBubyBzdWNoIHJlZmVyZW5jZSBhcyAiaG9zdCBhc3NpZ25lZCB2YWx1ZXMiIGFzDQo+Pj4+Pj4+
Pj4+IG1vc3QgcHJvYmFibHkgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgd2lsbCByZW1haW4gaW4gaXRz
IGFmdGVyIHJlc2V0IHN0YXRlLg0KPj4+Pj4+Pj4+IFdoYXQgbWVhbmluZyBvZiAiaGlkZGVuIiBk
byB5b3UgaW1wbHkgaGVyZT8gRGV2aWNlcyBwYXNzZWQgdG8NCj4+Pj4+Pj4+PiBwY2lfe2hpZGUs
cm99X2RldmljZSgpIG1heSBub3QgYmUgYXNzaWduZWQgdG8gZ3Vlc3RzIC4uLg0KPj4+Pj4+Pj4g
WW91IGFyZSBjb21wbGV0ZWx5IHJpZ2h0IGhlcmUuDQo+Pj4+Pj4+Pj4gRm9yIGFueSBvdGhlciBt
ZWFuaW5nIG9mICJoaWRkZW4iLCBldmVuIGlmIHRoZSBkZXZpY2UgaXMgY29tcGxldGVseQ0KPj4+
Pj4+Pj4+IGlnbm9yZWQgYnkgRG9tMCwNCj4+Pj4+Pj4+IERvbTBsZXNzIGlzIHN1Y2ggYSBjYXNl
IHdoZW4gYSBkZXZpY2UgaXMgYXNzaWduZWQgdG8gdGhlIGd1ZXN0DQo+Pj4+Pj4+PiB3aXRob3V0
IERvbTAgYXQgYWxsPw0KPj4+Pj4+PiBJbiB0aGlzIGNhc2UgaXQgaXMgZW50aXJlbHkgdW5jbGVh
ciB0byBtZSB3aGF0IGVudGl0eSBpdCBpcyB0byBoYXZlDQo+Pj4+Pj4+IGEgZ2xvYmFsIHZpZXcg
b24gdGhlIFBDSSBzdWJzeXN0ZW0uDQo+Pj4+Pj4+DQo+Pj4+Pj4+Pj4gICAgICAgY2VydGFpbiBv
ZiB0aGUgcHJvcGVydGllcyBzdGlsbCBjYW5ub3QgYmUgYWxsb3dlZA0KPj4+Pj4+Pj4+IHRvIGJl
IERvbVUtY29udHJvbGxlZC4NCj4+Pj4+Pj4+IFRoZSBsaXN0IGlzIG5vdCB0aGF0IGJpZywgY291
bGQgeW91IHBsZWFzZSBuYW1lIGEgZmV3IHlvdSB0aGluayBjYW5ub3QNCj4+Pj4+Pj4+IGJlIGNv
bnRyb2xsZWQgYnkgYSBndWVzdD8gSSBjYW4gdGhpbmsgb2YgUENJX0NPTU1BTkRfU1BFQ0lBTCg/
KSwNCj4+Pj4+Pj4+IFBDSV9DT01NQU5EX0lOVkFMSURBVEUoPyksIFBDSV9DT01NQU5EX1BBUklU
WSwgUENJX0NPTU1BTkRfV0FJVCwNCj4+Pj4+Pj4+IFBDSV9DT01NQU5EX1NFUlIsIFBDSV9DT01N
QU5EX0lOVFhfRElTQUJMRSB3aGljaCB3ZSBtYXkgd2FudCB0bw0KPj4+Pj4+Pj4gYmUgYWxpZ25l
ZCB3aXRoIHRoZSAiaG9zdCByZWZlcmVuY2UiIHZhbHVlcywgZS5nLiB3ZSBvbmx5IGFsbG93IHRo
b3NlIGJpdHMNCj4+Pj4+Pj4+IHRvIGJlIHNldCBhcyB0aGV5IGFyZSBpbiBEb20wLg0KPj4+Pj4+
PiBXZWxsLCB5b3UndmUgY29tcGlsZSBhIGxpc3QgYWxyZWFkeSwgYW5kIEkgZGlkIHNheSBzbyBi
ZWZvcmUgYXMgd2VsbDoNCj4+Pj4+Pj4gRXZlcnl0aGluZyBleGNlcHQgSS9PIGFuZCBtZW1vcnkg
ZGVjb2RpbmcgYXMgd2VsbCBhcyBidXMgbWFzdGVyaW5nDQo+Pj4+Pj4+IG5lZWRzIGF0IGxlYXN0
IGNsb3NlbHkgbG9va2luZyBhdC4gSU5UWF9ESVNBQkxFLCBmb3IgZXhhbXBsZSwgaXMNCj4+Pj4+
Pj4gc29tZXRoaW5nIEkgZG9uJ3QgdGhpbmsgYSBndWVzdCBzaG91bGQgYmUgYWJsZSB0byBkaXJl
Y3RseSBjb250cm9sLg0KPj4+Pj4+PiBJdCBtYXkgc3RpbGwgYmUgdGhlIGNhc2UgdGhhdCB0aGUg
aG9zdCBwZXJtaXRzIGl0IGNvbnRyb2wsIGJ1dCB0aGVuDQo+Pj4+Pj4+IG9ubHkgaW5kaXJlY3Rs
eSwgYWxsb3dpbmcgdGhlIGhvc3QgdG8gYXBwcm9wcmlhdGVseSBhZGp1c3QgaXRzDQo+Pj4+Pj4+
IGludGVybmFscy4NCj4+Pj4+Pj4NCj4+Pj4+Pj4gTm90ZSB0aGF0IGV2ZW4gZm9yIEkvTyBhbmQg
bWVtb3J5IGRlY29kaW5nIGFzIHdlbGwgYXMgYnVzIG1hc3RlcmluZw0KPj4+Pj4+PiBpdCBtYXkg
YmUgbmVjZXNzYXJ5IHRvIGxpbWl0IGd1ZXN0IGNvbnRyb2w6IEluIGNhc2UgdGhlIGhvc3Qgd2Fu
dHMNCj4+Pj4+Pj4gdG8gZGlzYWJsZSBhbnkgb2YgdGhlc2UgKHBlcmhhcHMgdHJhbnNpZW50bHkp
IGRlc3BpdGUgdGhlIGd1ZXN0DQo+Pj4+Pj4+IHdhbnRpbmcgdGhlbSBlbmFibGVkLg0KPj4+Pj4+
IE9rLCBzbyBpdCBpcyBub3cgY2xlYXIgdGhhdCB3ZSBuZWVkIGEgeWV0IGFub3RoZXIgcGF0Y2gg
dG8gYWRkIGEgcHJvcGVyDQo+Pj4+Pj4gY29tbWFuZCByZWdpc3RlciBlbXVsYXRpb24uIFdoYXQg
aXMgeW91ciBwcmVmZXJlbmNlOiBkcm9wIHRoZSBjdXJyZW50DQo+Pj4+Pj4gcGF0Y2gsIGltcGxl
bWVudCBjb21tYW5kIHJlZ2lzdGVyIGVtdWxhdGlvbiBhbmQgYWRkIGEgInJlc2V0IHBhdGNoIg0K
Pj4+Pj4+IGFmdGVyIHRoYXQgb3Igd2UgY2FuIGhhdmUgdGhlIHBhdGNoIGFzIGlzIG5vdywgYnV0
IEknbGwgb25seSByZXNldCBJTy9tZW0gYW5kIGJ1cw0KPj4+Pj4+IG1hc3RlciBiaXRzLCBlLmcu
IHJlYWQgdGhlIHJlYWwgdmFsdWUsIG1hc2sgdGhlIHdhbnRlZCBiaXRzIGFuZCB3cml0ZSBiYWNr
Pw0KPj4+Pj4gRWl0aGVyIG9yZGVyIGlzIGZpbmUgd2l0aCBtZSBhcyBsb25nIGFzIHRoZSByZXN1
bHQgd2lsbCBiZSBjbGFpbWVkIHRvDQo+Pj4+PiBiZSBjb21wbGV0ZSB1bnRpbCBwcm9wZXIgZW11
bGF0aW9uIGlzIGluIHBsYWNlLg0KPj4+PiBJIHRyaWVkIHRvIHNlZSB3aGF0IG90aGVycyBkbyBp
biBvcmRlciB0byBlbXVsYXRlIFBDSV9DT01NQU5EIHJlZ2lzdGVyDQo+Pj4+IGFuZCBpdCBzZWVt
cyB0aGF0IGF0IG1vc3QgdGhleSBjYXJlIGFib3V0IHRoZSBvbmx5IElOVFggYml0IChiZXNpZGVz
DQo+Pj4+IElPL21lbW9yeSBlbmFibGUgYW5kIGJ1cyBtdXN0ZXIgd2hpY2ggYXJlIHdyaXRlIHRo
cm91Z2gpLiBQbGVhc2Ugc2VlDQo+Pj4+IFsxXSBhbmQgWzJdLiBQcm9iYWJseSBJIG1pc3Mgc29t
ZXRoaW5nLCBidXQgaXQgY291bGQgYmUgYmVjYXVzZSBpbiBvcmRlcg0KPj4+PiB0byBwcm9wZXJs
eSBlbXVsYXRlIHRoZSBDT01NQU5EIHJlZ2lzdGVyIHdlIG5lZWQgdG8ga25vdyBhYm91dCB0aGUN
Cj4+Pj4gd2hvbGUgUENJIHRvcG9sb2d5LCBlLmcuIGlmIGFueSBzZXR0aW5nIGluIGRldmljZSdz
IGNvbW1hbmQgcmVnaXN0ZXINCj4+Pj4gaXMgYWxpZ25lZCB3aXRoIHRoZSB1cHN0cmVhbSBwb3J0
IGV0Yy4gVGhpcyBtYWtlcyBtZSB0aGluayB0aGF0IGJlY2F1c2UNCj4+Pj4gb2YgdGhpcyBjb21w
bGV4aXR5IG90aGVycyBqdXN0IGlnbm9yZSB0aGF0LiBOZWl0aGVyIEkgdGhpbmsgdGhpcyBjYW4g
YmUNCj4+Pj4gZWFzaWx5IGRvbmUgaW4gb3VyIGNhc2UuIFNvIEkgd291bGQgc3VnZ2VzdCB3ZSBq
dXN0IGFkZCB0aGUgZm9sbG93aW5nDQo+Pj4+IHNpbXBsZSBsb2dpYyB0byBvbmx5IGVtdWxhdGUg
UENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFOiBhbGxvdyBndWVzdCB0bw0KPj4+PiBkaXNhYmxlIHRo
ZSBpbnRlcnJ1cHRzLCBidXQgZG9uJ3QgYWxsb3cgdG8gZW5hYmxlIGlmIGhvc3QgaGFzIGRpc2Fi
bGVkDQo+Pj4+IHRoZW0uIFRoaXMgaXMgYWxzbyBjb3VsZCBiZSB0cmlja3kgYSBiaXQgZm9yIHRo
ZSBkZXZpY2VzIHdoaWNoIGFyZSBub3QNCj4+Pj4gZW5hYmxlZCBhbmQgdGh1cyBub3QgY29uZmln
dXJlZCBpbiBEb20wLCBlLmcuIHdlIGRvIG5vdCBrbm93IGZvciBzdXJlDQo+Pj4+IGlmIHRoZSB2
YWx1ZSBpbiB0aGUgUENJX0NPTU1BTkQgcmVnaXN0ZXIgKGluIHBhcnRpY3VsYXINCj4+Pj4gUENJ
X0NPTU1BTkRfSU5UWF9ESVNBQkxFIGJpdCkgY2FuIGJlIHVzZWQgYXMgdGhlIHJlZmVyZW5jZSBo
b3N0IHZhbHVlIG9yDQo+Pj4+IG5vdC4gSXQgY2FuIGJlIHRoYXQgdGhlIHZhbHVlIHRoZXJlIGlz
IGp1c3QgdGhlIG9uZSBhZnRlciByZXNldCBvciBzby4NCj4+Pj4gVGhlIHJlc3Qgb2YgdGhlIGNv
bW1hbmQgcmVnaXN0ZXIgYml0cyB3aWxsIGdvIGRpcmVjdGx5IHRvIHRoZSBjb21tYW5kDQo+Pj4+
IHJlZ2lzdGVyIHVudG91Y2hlZC4NCj4+Pj4gU28sIGF0IHRoZSBlbmQgb2YgdGhlIGRheSB0aGUg
cXVlc3Rpb24gaXMgaWYgUENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFDQo+Pj4+IGlzIGVub3VnaCBh
bmQgaG93IHRvIGdldCBpdHMgcmVmZXJlbmNlIGhvc3QgdmFsdWUuDQo+Pj4gV2VsbCwgaW4gb3Jk
ZXIgZm9yIHRoZSB3aG9sZSB0aGluZyB0byBiZSBzZWN1cml0eSBzdXBwb3J0ZWQgaXQgbmVlZHMg
dG8NCj4+PiBiZSBleHBsYWluZWQgZm9yIGV2ZXJ5IGJpdCB3aHkgaXQgaXMgc2FmZSB0byBhbGxv
dyB0aGUgZ3Vlc3QgdG8gZHJpdmUgaXQuDQo+PiBTbywgZG8gd2Ugd2FudCBhdCBsZWFzdCBQQ0lf
Q09NTUFORF9JTlRYX0RJU0FCTEUgYml0IGFsaWduZWQNCj4+IGJldHdlZW4gdGhlIGhvc3QgYW5k
IGd1ZXN0PyBJZiBzbywgd2hhdCBkbyB5b3UgeW91IHRoaW5rIGFib3V0DQo+PiB0aGUgcmVmZXJl
bmNlIHZhbHVlIGZvciBpdCAocGxlYXNlIHNlZSBhYm92ZSkuDQo+IFBsZWFzZSBtYXkgSSBhc2sg
dGhhdCB5b3UgY29tZSB1cCB3aXRoIGEgcHJvcG9zYWw/IEkgZG9uJ3QgdGhpbmsgSSd2ZQ0KPiBz
YWlkIHlvdSBuZWVkIHRvIGVtdWxhdGUgdGhpcyBvciBhbnkgb2YgdGhlIG90aGVyIGJpdHMuIEFs
bCBJJ3ZlIGFza2VkDQo+IGZvciBpcyB0aGF0IGZvciBldmVyeSBiaXQgeW91IGFsbG93IHRoZSBn
dWVzdCB0byBjb250cm9sIGRpcmVjdGx5LCB5b3UNCj4ganVzdGlmeSB3aHkgdGhhdCdzIHNhZmUg
YW5kIHNlY3VyZS4gSWYgbm8ganVzdGlmaWNhdGlvbiBjYW4gYmUgZ2l2ZW4sDQo+IGVtdWxhdGlv
biBpcyBnb2luZyB0byBiZSBuZWNlc3NhcnkuIEhvdyB0byBzb2x2ZSB0aGF0IGlzIGZpcnN0IGFu
ZA0KPiBmb3JlbW9zdCBwYXJ0IG9mIHlvdXIgdW5kZXJ0YWtpbmcuDQoNClRoZSB0aGluZyBoZXJl
IGlzIHRoYXQgd2UgY2FuJ3QgdHJ1bHkganVzdGlmeSBpZiB3ZSBjYW4gbGV0IHRoZSBndWVzdA0K
DQpjb250cm9sIHRob3NlIGJpdHMgb3Igbm90IGFzIGl0IGFsbCBtYXkgZGVwZW5kIG9uIHRoZSB0
b3BvbG9neSB0aGF0IHNvbWUNCg0Kc3BlY2lmaWMgc2V0dXAgbWlnaHQgaGF2ZS4gTm90IHRoYXQg
d2UgdGVjaG5pY2FsbHkgY2FuJ3QsIGJ1dCBub3QgaW4gYQ0KDQpwcmFjdGljYWwgYW5kIGVhc3kg
d2F5IElNTy4gVGFraW5nIHRoYXQgaW50byBhY2NvdW50IHdlIGNvbWUgdG8gYQ0KDQpjb25jbHVz
aW9uIHRoYXQgd2UgbmVlZCB0byBlbXVsYXRlIHRob3NlIHRoZW4uIEJ1dCwgYWdhaW4gd2UgdW5k
ZXJzdGFuZA0KDQp0aGF0IGZ1bGwgZW11bGF0aW9uLCBpZiBwcm9wZXJseSBpbXBsZW1lbnRlZCwg
aXMgZ29pbmcgdG/CoCBiZSBhIGJpZyBwaWVjZSBvZiBjb2RlDQoNCndoaWNoIG5lZWRzIHRvIHRh
a2UgaW50byBhY2NvdW50IHRoZSBwaHlzaWNhbCBQQ0kgdG9wb2xvZ3kgZXRjLg0KDQpTbywgdGhp
cyBpcyBteSB1bmRlcnN0YW5kaW5nIHdoeSBvdGhlcnMgZG8gbm90IGltcGxlbWVudCB0aGF0IChR
RU1VLCBBUkNOKQ0KDQphbmQgbGV0IHRoZSBndWVzdCBjb250cm9sIGFsbCBiaXRzLCBidXQgSU5U
eERJU0FCTEUgKERpc2NsYWltZXI6IGlmIEkgdW5kZXJzdG9vZCB0aGVpcg0KDQpjb2RlIGNvcnJl
Y3RseSkNCg0KU28sIG15IHByb3Bvc2FsIGhlcmUgaXMgdG8gb25seSBlbXVsYXRlIFBDSV9DT01N
QU5EX0lOVFhfRElTQUJMRSBhbmQgbGV0DQoNCnRoZSBvdGhlciBiaXRzIGJlIGNvbnRyb2xsZWQg
YnkgdGhlIGd1ZXN0IChwbGVhc2UgYWxzbyBzZWUgdGhlIG5vdGUgYmVsb3cpLg0KDQpJIGRvIHVu
ZGVyc3RhbmQgdGhpcyBpcyBub3QgY29ycmVjdCwgYnV0IEkgY2FuJ3QgdGVsbCBob3cgdG8gZGVh
bCB3aXRoIHRoaXMgb3RoZXIgd2F5Lg0KDQo+DQo+IEZvciB0aGUgYml0IGluIHF1ZXN0aW9uLCB3
aGVyZSB0aGUgZ29hbCBhcHBlYXJzIHRvIGJlIHRvIGhhdmUgaGFyZHdhcmUNCj4gaG9sZCB0aGUg
T1Igb2YgZ3Vlc3QgYW5kIGhvc3QgdmFsdWVzLCBhbiBhcHByb2FjaCBzaW1pbGFyIHRvIHRoYXQg
dXNlZA0KPiBmb3Igc29tZSBvZiB0aGUgTVNJIC8gTVNJLVggYml0cyBtaWdodCBiZSBjaG9zZW46
IE1haW50YWluIGd1ZXN0IGFuZA0KPiBob3N0IGJpdHMgaW4gc29mdHdhcmUsIGFuZCB1cGRhdGUg
aGFyZHdhcmUgKGF0IGxlYXN0KSB3aGVuIHRoZQ0KPiBlZmZlY3RpdmUgcmVzdWx0aW5nIHZhbHVl
IGNoYW5nZXMuIEEgY29tcGxpY2F0aW5nIGZhY3QgaGVyZSBpcywgdGhvdWdoLA0KPiB0aGF0IHVu
bGlrZSBmb3IgdGhlIE1TSSAvIE1TSS1YIGJpdHMgaGVyZSBEb20wIChwY2liYWNrIG9yIGl0cyBQ
Q0kNCj4gc3VzYnN0ZW0pIG1heSBhbHNvIGhhdmUgYSB2aWV3IG9uIHdoYXQgdGhlIHNldHRpbmcg
b3VnaHQgdG8gYmUuDQoNClRoZSBiaWdnZXIgcXVlc3Rpb24gaGVyZSBpcyB3aGF0IGNhbiB3ZSB0
YWtlIGFzIHRoZSByZWZlcmVuY2UgZm9yIElOVHgNCg0KYml0LCBlLmcuIGlmIERvbTAgZGlkbid0
IGVuYWJsZS9jb25maWd1cmVkIHRoZSBkZXZpY2UgYmVpbmcgcGFzc2VkIHRocm91Z2gNCg0KdGhh
biBpdHMgQ09NTUFORCByZWdpc3RlciBtYXkgc3RpbGwgYmUgaW4gYWZ0ZXIgcmVzZXQgc3RhdGUg
YW5kIElNTyB0aGVyZSBpcw0KDQpubyBndWFyYW50ZWUgaXQgaGFzIHRoZSB2YWx1ZXMgd2UgY2Fu
IHNheSBhcmUgImFzIGhvc3Qgd2FudHMgdGhlbSINCg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwN
Cg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:48:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:48:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183044.330982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIXw-0006VA-Tg; Thu, 09 Sep 2021 11:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183044.330982; Thu, 09 Sep 2021 11:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIXw-0006V1-Qc; Thu, 09 Sep 2021 11:48:48 +0000
Received: by outflank-mailman (input) for mailman id 183044;
 Thu, 09 Sep 2021 11:48:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOIXu-0006RJ-Sb
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:48:46 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c7a9dd28-144d-449d-8ad2-3b60ddc1d82e;
 Thu, 09 Sep 2021 11:48:41 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189BRSec001895; 
 Thu, 9 Sep 2021 11:48:39 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayhdp03g4-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 11:48:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4145.eurprd03.prod.outlook.com (2603:10a6:208:c9::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 11:48:04 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 11:48:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7a9dd28-144d-449d-8ad2-3b60ddc1d82e
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XT3RYiiBMIKK/pYekrea9Jobzz38A9vNUHTu6ANgj2xp1QT0phjJHFnaQCiQS5T5rBEnBPPpdskHeedVX+Iq1dCoE5jr0X1jbAmXol/ZQJo13bLBDuIZoU0dC8a5AXg0BiAR/nKfU2YG4bxRwhOkfUm12wdLNpPA9Uj1FRJiBeqhSow4JPvf6H00CG9022aBmgm8w49+Vzj6cfeHXjmHvcn+98Iu8BiJ7Iom6Ztpx0dJjyct7tqmmWQ0vqlSCFgMFk5sxg1AwY0QQA0TzQvWNJYfUst7IVNG3S01esKZ4k58WaubMsvuJI8va5Mq+6E4QLUdUGjnjSzOad1j5VDxiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1zGTQpWCzk8h/3mKJTKON4qYNgPUrpaZOdxoShTFzZI=;
 b=OWpfhmcs6IeuhEO5aaqj6VjTmMNu0hIaVR31dtFVbz4CobWHubiIVqTJP/GtkVli5RF+kApv9sc1MUdj9kXGjmAvNOegxQXtEwRHwNxhCSHussSaOtz4eG3c8RnwpRwkamTAFOiY5+7grr2N0zZ+QuliFwYiBW6RFGZdCI3U/Pciw2wyPpaD8/WHjqpMW1t8eAPEyVbSi1MNO4kae7FptuY84nBDypDXfvo3mJ2MpVLMPbGuWSzumiuAG+rTfXXtppfLyRJspdbRDEPtKBiSVN9GoVRyXRJJbbVrDgirn8N+ljAEZ14wSgRK9hLddyu/l+9XfKcn30ngP5j68F3RMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1zGTQpWCzk8h/3mKJTKON4qYNgPUrpaZOdxoShTFzZI=;
 b=ZhtU0tOfnQROSYX3u+Jzqx3Dpu0g9S3dd8fwg0/p7byEgM8RW1YnuXKUAfHWni23QDPUtqRCkcUCNdzOEfYysUDB2cE8FU3VP/Na8UFcNm0fYOCVCg//bYE44cqtcVSegIIywIAam8Mfqf7YF5Nh310WVvfKFSQ2c5dlIwnIsT5k6kbAzV/KEuQrL33D0rq9dgXlfhrYFm1yVAl6OT+DEuGihzcGx9WoAU9WmHxKngM8dZqwrNA932YlAxsqlBeyd5JPEppPCf/C+iMohQXnpmVSf/GToIXL1MJE4xYgENwFI3To9l7dehybvAnsNLA3IRb/UzjwITbcnu0v8Esuuw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AIAACKGAgAAo+QA=
Date: Thu, 9 Sep 2021 11:48:04 +0000
Message-ID: <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
In-Reply-To: <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 12eeae0c-882a-4a7d-12a3-08d97387af49
x-ms-traffictypediagnostic: AM0PR03MB4145:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB414535EEFC240F1A7E6EDB0AE7D59@AM0PR03MB4145.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 MfNWkaBPYWh9Ask6Woho7VzbGWBNjYkX5coDPsEIRr8nFE41egjxMajYk/pXDNWagTsLl2Hs503UVYfsT5+LRzxJriL2NhqEsiwL5tUKpWIc/TBLMW/DPNmHX0FxNkyrqif6RB3OOkwZfmv9r/tgbLklKTTtqUXknVQuxHOX7/E+pwFkYpPLnE1gleyxFjpopJGjfsJW5ruaKuB0ViGsizWiHSYxIEQEJ4BPAAkSyc22SaTUqAgT/mDERcUQJlyq6mWjXRgFGP6UJ1vvk46McOt0foVL8z9v5B6FOXXQzsgFDNuZGf0BTeuZUy7A0HoPMfXMHl4eugqtD+jYqe3Ho+FVzAeNbpq5LoEmyMR2PAVMZSS4aPlDi7Lt7lolKqhBv/SDWEHQZcnD7QEc0VaTP0lxQN4H3M2urrCXpqaDAgN+f6YVhbt4VBUMSAfDYTmUqh1+N7G3H7WgNfCw4wrs/p/QkoanKyfzJxNGwRHRS6pO8y02zHHW07IVE7mF2WzCvzHVEnTyAXWmJ6zxgn/OKPfn1sP3iYg1xFLG6Wf7RIcInMk83w0FiBw404h+DcMHs7NcP3qRq25aCh79yF5CM1XA2Tif67o8Ai/KJPv/8YKv9o2zKUUgAO5OrPvNFKBo9y1EoMynEa1Z+TT2aR/rR5OlQlwizKt9d2nTb6VBWgx7FVDgOxxpdVUpQTjQGHX7s3WHx04EEzkF8eoOkyA5Bt1Ay54FFk/QhN38z+NOkIjyKjboqiRXDB+7l/hl+8I/
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(39860400002)(376002)(346002)(396003)(53546011)(6506007)(8936002)(38070700005)(2616005)(66446008)(66946007)(6486002)(6512007)(91956017)(478600001)(4326008)(2906002)(76116006)(8676002)(186003)(71200400001)(26005)(5660300002)(66476007)(66556008)(64756008)(36756003)(316002)(83380400001)(110136005)(54906003)(31696002)(86362001)(38100700002)(122000001)(31686004)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Y09aV01DVTdmNjlVU0o5Qm0xZ1VlRTRGRVJ6YjJKejR4ci9hL2pFMkFqaU01?=
 =?utf-8?B?NXAweGswTkFoaHVOM0Z5NjFyeUxoVDVBYlVSYnpZZ2t5enovS1VZNlFFblM1?=
 =?utf-8?B?dUU4WEtNMXl2SG1pTThTK2ZwU1dpeFZyR3lQQ05tbFU5SmJ2c1RmTjYvRnJi?=
 =?utf-8?B?UTlPSjlWWmJhOTRWcmR6R3o0c054TWNJenVjVW5ENXRVbEN1cVRoWG9TN21s?=
 =?utf-8?B?Y1NORWR2YlNDK1kvdmtWcHlNREhIdExvNWVmeEVHY1VqRmtQZi9nSk84cnBP?=
 =?utf-8?B?QzFlQnBMK0tZMERpTmQvRzZ1SVlUd1pIZWUwYjhrWktEQnRWV0VZZ3phdjhG?=
 =?utf-8?B?VStWU2R1WFdhb2tsdEhia2wwVnZMS2RyT05Vd1R3MzJtUjlTLzQ4aHBDaVhO?=
 =?utf-8?B?SzB6SFYrZlJ2M21pa0ZudHAxRnBlNTB4LzZyaUY5MmhnbGVxbGEwY09rYXQ4?=
 =?utf-8?B?RXREbGprS3I5Z0RHVDc1LzhaN3d4UHA5NzdveFpDd3lhamdLUUlPemtPWFJj?=
 =?utf-8?B?d2x2LytrWXBmaHZFSFJCQVNaeFZmY1Z3REkwRG5nbGVGdTdrM2FjUFNYMkZh?=
 =?utf-8?B?Uyt4VUM0YitzWkx2T3cwTXBISUg1bnR4aTQ0QmJOeFRpME9zSHBSWFpkeXkr?=
 =?utf-8?B?YkhVY1lCcDRjejdqYmJYVHExZHU5K1hSQUJmcDZ1UG9BUjFoL21BSklUaEVn?=
 =?utf-8?B?cndYV1dnUFNwMTJTYkcrUGd1QmRZMVpEWGxkQnV5bE5UMmNlbzFqUUJBMTQ3?=
 =?utf-8?B?UHl4V0xrQXpaWVVKVUNrNEZJdFNPZWpuZ0pBVnYwQWJFS3pldFdGKzR1ZExU?=
 =?utf-8?B?KzFkY3RmVitUR25SNVVDWVBCaVpPbFdJYVFqdTNQTjFlVjZ5ZlRmMDcxRHhw?=
 =?utf-8?B?MDdrdGZmOWZWNjVzUVR2NzgyWDBLTWo4Z2RpUGFmbGovcE4yaUlwWDRJRjVh?=
 =?utf-8?B?V2tCNXJYYnNCQU4xMndZUGVRU05XblI4TnFNeUNTY093UjF0V3paTUgydFZ5?=
 =?utf-8?B?NHVNWHdXMzgxcmtxMVFtcUw0Vm9xYms1MWR1MU80LzF6VlhBWk5vUlQ0aENY?=
 =?utf-8?B?bTBkM0FoWjdLdmxRcmhiOHNTTHBoeFhzLzVwTUJKOGR2S0FHZHQ2cGxVcEdx?=
 =?utf-8?B?VXIxRjZHQU5sUFdYTDgyd3BSUmxuNkZBT01LYUp1bngyM3JhTXB5eVVUQWox?=
 =?utf-8?B?ajFrVDFZNTB6bktsSWU0b282Y2FyWGtFVmNtVG0rRTFsdVpXbkk0ODFHUExZ?=
 =?utf-8?B?QlNiSndTdW1ObkE4akczUCtpSXNyYng1eERpT3Q3ejNQQUJwVkNIYTBaN0x4?=
 =?utf-8?B?MTZlSTA3ZWtUL2JiR3FMV082QkJ1V3J6UTJrUGtNS3IvbUxiRDN3clptT0Ni?=
 =?utf-8?B?TG9ieVRJK2Z3QVVUdk9vNXN4TjByVTVvbitJV2N1MHZIbjlwVVdGMTFWK05z?=
 =?utf-8?B?TC9LU1hXNmZwNFg3NVpuTTk0TmRZTUE3MW1UT0t2UElqMkcrNTR3d1Jaando?=
 =?utf-8?B?K3R0c1I0Q3JnNmIxRWJpSTRsN0tDTUp3SDRaejRnN1QvZnk2SG51Z21ZczFl?=
 =?utf-8?B?ZUo5TzBvRXRKOVQ3MVlLUlU5QVFCWHpIVThyS3o3ckxXMkdCZUpZQXY3Mzgz?=
 =?utf-8?B?MmVIdFdua1EzVGp3eXlIdURUSzlPMzZ0clIzMlhRbGJud2h4MS9NQkI5bWs5?=
 =?utf-8?B?ZDIzVVIzbnJsNC96bUYxYWtibHhUUkZRcDJFRmtXZERhWklUdFVPalpUUHNm?=
 =?utf-8?Q?u+gHFT0mYb6aZRXDKbklMtvd4PX3pJLJvr3EyvI?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <9B89FA9901920849A13EF9AB5F7036D0@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 12eeae0c-882a-4a7d-12a3-08d97387af49
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 11:48:04.6255
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qiNnOIetqn9XbYSqcRn90Y02Hbyxzi7aL3Vwh9o32znfAc0FQcfj12wnhOWOvmZCK6oz8DHdOSm2FxF5YVdM728r78C1BuWb8Ec3a97MrJgY2IH6WmlbxlFXiWgH8Azo
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4145
X-Proofpoint-ORIG-GUID: elluFbn-6lVw3du8hnmLdExAPIT0IAVG
X-Proofpoint-GUID: elluFbn-6lVw3du8hnmLdExAPIT0IAVG
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015
 priorityscore=1501 mlxscore=0 mlxlogscore=999 malwarescore=0
 lowpriorityscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090072

DQpPbiAwOS4wOS4yMSAxMjoyMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTA6NTAsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTE6
NDMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMTA6MzksIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwNy4wOS4yMSAxMzowNiwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAwNy4wOS4yMDIxIDExOjUyLCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAwNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTE6MDcsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gT24gMDcuMDkuMjEgMTE6NDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+
Pj4+Pj4+IE9uIDA3LjA5LjIwMjEgMTA6MTgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4+Pj4+PiBTbywgaWYgd2UgaGF2ZSBhIGhpZGRlbiBQQ0kgZGV2aWNlIHdoaWNoIGNh
biBiZSBhc3NpZ25lZCB0byBhIGd1ZXN0IGFuZCBpdCBpcyBsaXRlcmFsbHkgdW50b3VjaGVkDQo+
Pj4+Pj4+Pj4+IChub3QgZW5hYmxlZCBpbiBEb20wKSB0aGVuIEkgdGhpbmsgdGhlcmUgd2lsbCBi
ZSBubyBzdWNoIHJlZmVyZW5jZSBhcyAiaG9zdCBhc3NpZ25lZCB2YWx1ZXMiIGFzDQo+Pj4+Pj4+
Pj4+IG1vc3QgcHJvYmFibHkgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgd2lsbCByZW1haW4gaW4gaXRz
IGFmdGVyIHJlc2V0IHN0YXRlLg0KPj4+Pj4+Pj4+IFdoYXQgbWVhbmluZyBvZiAiaGlkZGVuIiBk
byB5b3UgaW1wbHkgaGVyZT8gRGV2aWNlcyBwYXNzZWQgdG8NCj4+Pj4+Pj4+PiBwY2lfe2hpZGUs
cm99X2RldmljZSgpIG1heSBub3QgYmUgYXNzaWduZWQgdG8gZ3Vlc3RzIC4uLg0KPj4+Pj4+Pj4g
WW91IGFyZSBjb21wbGV0ZWx5IHJpZ2h0IGhlcmUuDQo+Pj4+Pj4+Pj4gRm9yIGFueSBvdGhlciBt
ZWFuaW5nIG9mICJoaWRkZW4iLCBldmVuIGlmIHRoZSBkZXZpY2UgaXMgY29tcGxldGVseQ0KPj4+
Pj4+Pj4+IGlnbm9yZWQgYnkgRG9tMCwNCj4+Pj4+Pj4+IERvbTBsZXNzIGlzIHN1Y2ggYSBjYXNl
IHdoZW4gYSBkZXZpY2UgaXMgYXNzaWduZWQgdG8gdGhlIGd1ZXN0DQo+Pj4+Pj4+PiB3aXRob3V0
IERvbTAgYXQgYWxsPw0KPj4+Pj4+PiBJbiB0aGlzIGNhc2UgaXQgaXMgZW50aXJlbHkgdW5jbGVh
ciB0byBtZSB3aGF0IGVudGl0eSBpdCBpcyB0byBoYXZlDQo+Pj4+Pj4+IGEgZ2xvYmFsIHZpZXcg
b24gdGhlIFBDSSBzdWJzeXN0ZW0uDQo+Pj4+Pj4+DQo+Pj4+Pj4+Pj4gICAgICAgY2VydGFpbiBv
ZiB0aGUgcHJvcGVydGllcyBzdGlsbCBjYW5ub3QgYmUgYWxsb3dlZA0KPj4+Pj4+Pj4+IHRvIGJl
IERvbVUtY29udHJvbGxlZC4NCj4+Pj4+Pj4+IFRoZSBsaXN0IGlzIG5vdCB0aGF0IGJpZywgY291
bGQgeW91IHBsZWFzZSBuYW1lIGEgZmV3IHlvdSB0aGluayBjYW5ub3QNCj4+Pj4+Pj4+IGJlIGNv
bnRyb2xsZWQgYnkgYSBndWVzdD8gSSBjYW4gdGhpbmsgb2YgUENJX0NPTU1BTkRfU1BFQ0lBTCg/
KSwNCj4+Pj4+Pj4+IFBDSV9DT01NQU5EX0lOVkFMSURBVEUoPyksIFBDSV9DT01NQU5EX1BBUklU
WSwgUENJX0NPTU1BTkRfV0FJVCwNCj4+Pj4+Pj4+IFBDSV9DT01NQU5EX1NFUlIsIFBDSV9DT01N
QU5EX0lOVFhfRElTQUJMRSB3aGljaCB3ZSBtYXkgd2FudCB0bw0KPj4+Pj4+Pj4gYmUgYWxpZ25l
ZCB3aXRoIHRoZSAiaG9zdCByZWZlcmVuY2UiIHZhbHVlcywgZS5nLiB3ZSBvbmx5IGFsbG93IHRo
b3NlIGJpdHMNCj4+Pj4+Pj4+IHRvIGJlIHNldCBhcyB0aGV5IGFyZSBpbiBEb20wLg0KPj4+Pj4+
PiBXZWxsLCB5b3UndmUgY29tcGlsZSBhIGxpc3QgYWxyZWFkeSwgYW5kIEkgZGlkIHNheSBzbyBi
ZWZvcmUgYXMgd2VsbDoNCj4+Pj4+Pj4gRXZlcnl0aGluZyBleGNlcHQgSS9PIGFuZCBtZW1vcnkg
ZGVjb2RpbmcgYXMgd2VsbCBhcyBidXMgbWFzdGVyaW5nDQo+Pj4+Pj4+IG5lZWRzIGF0IGxlYXN0
IGNsb3NlbHkgbG9va2luZyBhdC4gSU5UWF9ESVNBQkxFLCBmb3IgZXhhbXBsZSwgaXMNCj4+Pj4+
Pj4gc29tZXRoaW5nIEkgZG9uJ3QgdGhpbmsgYSBndWVzdCBzaG91bGQgYmUgYWJsZSB0byBkaXJl
Y3RseSBjb250cm9sLg0KPj4+Pj4+PiBJdCBtYXkgc3RpbGwgYmUgdGhlIGNhc2UgdGhhdCB0aGUg
aG9zdCBwZXJtaXRzIGl0IGNvbnRyb2wsIGJ1dCB0aGVuDQo+Pj4+Pj4+IG9ubHkgaW5kaXJlY3Rs
eSwgYWxsb3dpbmcgdGhlIGhvc3QgdG8gYXBwcm9wcmlhdGVseSBhZGp1c3QgaXRzDQo+Pj4+Pj4+
IGludGVybmFscy4NCj4+Pj4+Pj4NCj4+Pj4+Pj4gTm90ZSB0aGF0IGV2ZW4gZm9yIEkvTyBhbmQg
bWVtb3J5IGRlY29kaW5nIGFzIHdlbGwgYXMgYnVzIG1hc3RlcmluZw0KPj4+Pj4+PiBpdCBtYXkg
YmUgbmVjZXNzYXJ5IHRvIGxpbWl0IGd1ZXN0IGNvbnRyb2w6IEluIGNhc2UgdGhlIGhvc3Qgd2Fu
dHMNCj4+Pj4+Pj4gdG8gZGlzYWJsZSBhbnkgb2YgdGhlc2UgKHBlcmhhcHMgdHJhbnNpZW50bHkp
IGRlc3BpdGUgdGhlIGd1ZXN0DQo+Pj4+Pj4+IHdhbnRpbmcgdGhlbSBlbmFibGVkLg0KPj4+Pj4+
IE9rLCBzbyBpdCBpcyBub3cgY2xlYXIgdGhhdCB3ZSBuZWVkIGEgeWV0IGFub3RoZXIgcGF0Y2gg
dG8gYWRkIGEgcHJvcGVyDQo+Pj4+Pj4gY29tbWFuZCByZWdpc3RlciBlbXVsYXRpb24uIFdoYXQg
aXMgeW91ciBwcmVmZXJlbmNlOiBkcm9wIHRoZSBjdXJyZW50DQo+Pj4+Pj4gcGF0Y2gsIGltcGxl
bWVudCBjb21tYW5kIHJlZ2lzdGVyIGVtdWxhdGlvbiBhbmQgYWRkIGEgInJlc2V0IHBhdGNoIg0K
Pj4+Pj4+IGFmdGVyIHRoYXQgb3Igd2UgY2FuIGhhdmUgdGhlIHBhdGNoIGFzIGlzIG5vdywgYnV0
IEknbGwgb25seSByZXNldCBJTy9tZW0gYW5kIGJ1cw0KPj4+Pj4+IG1hc3RlciBiaXRzLCBlLmcu
IHJlYWQgdGhlIHJlYWwgdmFsdWUsIG1hc2sgdGhlIHdhbnRlZCBiaXRzIGFuZCB3cml0ZSBiYWNr
Pw0KPj4+Pj4gRWl0aGVyIG9yZGVyIGlzIGZpbmUgd2l0aCBtZSBhcyBsb25nIGFzIHRoZSByZXN1
bHQgd2lsbCBiZSBjbGFpbWVkIHRvDQo+Pj4+PiBiZSBjb21wbGV0ZSB1bnRpbCBwcm9wZXIgZW11
bGF0aW9uIGlzIGluIHBsYWNlLg0KPj4+PiBJIHRyaWVkIHRvIHNlZSB3aGF0IG90aGVycyBkbyBp
biBvcmRlciB0byBlbXVsYXRlIFBDSV9DT01NQU5EIHJlZ2lzdGVyDQo+Pj4+IGFuZCBpdCBzZWVt
cyB0aGF0IGF0IG1vc3QgdGhleSBjYXJlIGFib3V0IHRoZSBvbmx5IElOVFggYml0IChiZXNpZGVz
DQo+Pj4+IElPL21lbW9yeSBlbmFibGUgYW5kIGJ1cyBtdXN0ZXIgd2hpY2ggYXJlIHdyaXRlIHRo
cm91Z2gpLiBQbGVhc2Ugc2VlDQo+Pj4+IFsxXSBhbmQgWzJdLiBQcm9iYWJseSBJIG1pc3Mgc29t
ZXRoaW5nLCBidXQgaXQgY291bGQgYmUgYmVjYXVzZSBpbiBvcmRlcg0KPj4+PiB0byBwcm9wZXJs
eSBlbXVsYXRlIHRoZSBDT01NQU5EIHJlZ2lzdGVyIHdlIG5lZWQgdG8ga25vdyBhYm91dCB0aGUN
Cj4+Pj4gd2hvbGUgUENJIHRvcG9sb2d5LCBlLmcuIGlmIGFueSBzZXR0aW5nIGluIGRldmljZSdz
IGNvbW1hbmQgcmVnaXN0ZXINCj4+Pj4gaXMgYWxpZ25lZCB3aXRoIHRoZSB1cHN0cmVhbSBwb3J0
IGV0Yy4gVGhpcyBtYWtlcyBtZSB0aGluayB0aGF0IGJlY2F1c2UNCj4+Pj4gb2YgdGhpcyBjb21w
bGV4aXR5IG90aGVycyBqdXN0IGlnbm9yZSB0aGF0LiBOZWl0aGVyIEkgdGhpbmsgdGhpcyBjYW4g
YmUNCj4+Pj4gZWFzaWx5IGRvbmUgaW4gb3VyIGNhc2UuIFNvIEkgd291bGQgc3VnZ2VzdCB3ZSBq
dXN0IGFkZCB0aGUgZm9sbG93aW5nDQo+Pj4+IHNpbXBsZSBsb2dpYyB0byBvbmx5IGVtdWxhdGUg
UENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFOiBhbGxvdyBndWVzdCB0bw0KPj4+PiBkaXNhYmxlIHRo
ZSBpbnRlcnJ1cHRzLCBidXQgZG9uJ3QgYWxsb3cgdG8gZW5hYmxlIGlmIGhvc3QgaGFzIGRpc2Fi
bGVkDQo+Pj4+IHRoZW0uIFRoaXMgaXMgYWxzbyBjb3VsZCBiZSB0cmlja3kgYSBiaXQgZm9yIHRo
ZSBkZXZpY2VzIHdoaWNoIGFyZSBub3QNCj4+Pj4gZW5hYmxlZCBhbmQgdGh1cyBub3QgY29uZmln
dXJlZCBpbiBEb20wLCBlLmcuIHdlIGRvIG5vdCBrbm93IGZvciBzdXJlDQo+Pj4+IGlmIHRoZSB2
YWx1ZSBpbiB0aGUgUENJX0NPTU1BTkQgcmVnaXN0ZXIgKGluIHBhcnRpY3VsYXINCj4+Pj4gUENJ
X0NPTU1BTkRfSU5UWF9ESVNBQkxFIGJpdCkgY2FuIGJlIHVzZWQgYXMgdGhlIHJlZmVyZW5jZSBo
b3N0IHZhbHVlIG9yDQo+Pj4+IG5vdC4gSXQgY2FuIGJlIHRoYXQgdGhlIHZhbHVlIHRoZXJlIGlz
IGp1c3QgdGhlIG9uZSBhZnRlciByZXNldCBvciBzby4NCj4+Pj4gVGhlIHJlc3Qgb2YgdGhlIGNv
bW1hbmQgcmVnaXN0ZXIgYml0cyB3aWxsIGdvIGRpcmVjdGx5IHRvIHRoZSBjb21tYW5kDQo+Pj4+
IHJlZ2lzdGVyIHVudG91Y2hlZC4NCj4+Pj4gU28sIGF0IHRoZSBlbmQgb2YgdGhlIGRheSB0aGUg
cXVlc3Rpb24gaXMgaWYgUENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFDQo+Pj4+IGlzIGVub3VnaCBh
bmQgaG93IHRvIGdldCBpdHMgcmVmZXJlbmNlIGhvc3QgdmFsdWUuDQo+Pj4gV2VsbCwgaW4gb3Jk
ZXIgZm9yIHRoZSB3aG9sZSB0aGluZyB0byBiZSBzZWN1cml0eSBzdXBwb3J0ZWQgaXQgbmVlZHMg
dG8NCj4+PiBiZSBleHBsYWluZWQgZm9yIGV2ZXJ5IGJpdCB3aHkgaXQgaXMgc2FmZSB0byBhbGxv
dyB0aGUgZ3Vlc3QgdG8gZHJpdmUgaXQuDQo+PiBTbywgZG8gd2Ugd2FudCBhdCBsZWFzdCBQQ0lf
Q09NTUFORF9JTlRYX0RJU0FCTEUgYml0IGFsaWduZWQNCj4+IGJldHdlZW4gdGhlIGhvc3QgYW5k
IGd1ZXN0PyBJZiBzbywgd2hhdCBkbyB5b3UgeW91IHRoaW5rIGFib3V0DQo+PiB0aGUgcmVmZXJl
bmNlIHZhbHVlIGZvciBpdCAocGxlYXNlIHNlZSBhYm92ZSkuDQo+IFBsZWFzZSBtYXkgSSBhc2sg
dGhhdCB5b3UgY29tZSB1cCB3aXRoIGEgcHJvcG9zYWw/IEkgZG9uJ3QgdGhpbmsgSSd2ZQ0KPiBz
YWlkIHlvdSBuZWVkIHRvIGVtdWxhdGUgdGhpcyBvciBhbnkgb2YgdGhlIG90aGVyIGJpdHMuIEFs
bCBJJ3ZlIGFza2VkDQo+IGZvciBpcyB0aGF0IGZvciBldmVyeSBiaXQgeW91IGFsbG93IHRoZSBn
dWVzdCB0byBjb250cm9sIGRpcmVjdGx5LCB5b3UNCj4ganVzdGlmeSB3aHkgdGhhdCdzIHNhZmUg
YW5kIHNlY3VyZS4gSWYgbm8ganVzdGlmaWNhdGlvbiBjYW4gYmUgZ2l2ZW4sDQo+IGVtdWxhdGlv
biBpcyBnb2luZyB0byBiZSBuZWNlc3NhcnkuIEhvdyB0byBzb2x2ZSB0aGF0IGlzIGZpcnN0IGFu
ZA0KPiBmb3JlbW9zdCBwYXJ0IG9mIHlvdXIgdW5kZXJ0YWtpbmcuDQoNClRoZSB0aGluZyBoZXJl
IGlzIHRoYXQgd2UgY2FuJ3QgdHJ1bHkganVzdGlmeSBpZiB3ZSBjYW4gbGV0IHRoZSBndWVzdA0K
DQpjb250cm9sIHRob3NlIGJpdHMgb3Igbm90IGFzIGl0IGFsbCBtYXkgZGVwZW5kIG9uIHRoZSB0
b3BvbG9neSB0aGF0IHNvbWUNCg0Kc3BlY2lmaWMgc2V0dXAgbWlnaHQgaGF2ZS4gTm90IHRoYXQg
d2UgdGVjaG5pY2FsbHkgY2FuJ3QsIGJ1dCBub3QgaW4gYQ0KDQpwcmFjdGljYWwgYW5kIGVhc3kg
d2F5IElNTy4gVGFraW5nIHRoYXQgaW50byBhY2NvdW50IHdlIGNvbWUgdG8gYQ0KDQpjb25jbHVz
aW9uIHRoYXQgd2UgbmVlZCB0byBlbXVsYXRlIHRob3NlIHRoZW4uIEJ1dCwgYWdhaW4gd2UgdW5k
ZXJzdGFuZA0KDQp0aGF0IGZ1bGwgZW11bGF0aW9uLCBpZiBwcm9wZXJseSBpbXBsZW1lbnRlZCwg
aXMgZ29pbmcgdG/CoCBiZSBhIGJpZyBwaWVjZSBvZiBjb2RlDQoNCndoaWNoIG5lZWRzIHRvIHRh
a2UgaW50byBhY2NvdW50IHRoZSBwaHlzaWNhbCBQQ0kgdG9wb2xvZ3kgZXRjLg0KDQpTbywgdGhp
cyBpcyBteSB1bmRlcnN0YW5kaW5nIHdoeSBvdGhlcnMgZG8gbm90IGltcGxlbWVudCB0aGF0IChR
RU1VLCBBUkNOKQ0KDQphbmQgbGV0IHRoZSBndWVzdCBjb250cm9sIGFsbCBiaXRzLCBidXQgSU5U
eERJU0FCTEUgKERpc2NsYWltZXI6IGlmIEkgdW5kZXJzdG9vZCB0aGVpcg0KDQpjb2RlIGNvcnJl
Y3RseSkNCg0KU28sIG15IHByb3Bvc2FsIGhlcmUgaXMgdG8gb25seSBlbXVsYXRlIFBDSV9DT01N
QU5EX0lOVFhfRElTQUJMRSBhbmQgbGV0DQoNCnRoZSBvdGhlciBiaXRzIGJlIGNvbnRyb2xsZWQg
YnkgdGhlIGd1ZXN0IChwbGVhc2UgYWxzbyBzZWUgdGhlIG5vdGUgYmVsb3cpLg0KDQpJIGRvIHVu
ZGVyc3RhbmQgdGhpcyBpcyBub3QgY29ycmVjdCwgYnV0IEkgY2FuJ3QgdGVsbCBob3cgdG8gZGVh
bCB3aXRoIHRoaXMgb3RoZXIgd2F5Lg0KDQo+DQo+IEZvciB0aGUgYml0IGluIHF1ZXN0aW9uLCB3
aGVyZSB0aGUgZ29hbCBhcHBlYXJzIHRvIGJlIHRvIGhhdmUgaGFyZHdhcmUNCj4gaG9sZCB0aGUg
T1Igb2YgZ3Vlc3QgYW5kIGhvc3QgdmFsdWVzLCBhbiBhcHByb2FjaCBzaW1pbGFyIHRvIHRoYXQg
dXNlZA0KPiBmb3Igc29tZSBvZiB0aGUgTVNJIC8gTVNJLVggYml0cyBtaWdodCBiZSBjaG9zZW46
IE1haW50YWluIGd1ZXN0IGFuZA0KPiBob3N0IGJpdHMgaW4gc29mdHdhcmUsIGFuZCB1cGRhdGUg
aGFyZHdhcmUgKGF0IGxlYXN0KSB3aGVuIHRoZQ0KPiBlZmZlY3RpdmUgcmVzdWx0aW5nIHZhbHVl
IGNoYW5nZXMuIEEgY29tcGxpY2F0aW5nIGZhY3QgaGVyZSBpcywgdGhvdWdoLA0KPiB0aGF0IHVu
bGlrZSBmb3IgdGhlIE1TSSAvIE1TSS1YIGJpdHMgaGVyZSBEb20wIChwY2liYWNrIG9yIGl0cyBQ
Q0kNCj4gc3VzYnN0ZW0pIG1heSBhbHNvIGhhdmUgYSB2aWV3IG9uIHdoYXQgdGhlIHNldHRpbmcg
b3VnaHQgdG8gYmUuDQoNClRoZSBiaWdnZXIgcXVlc3Rpb24gaGVyZSBpcyB3aGF0IGNhbiB3ZSB0
YWtlIGFzIHRoZSByZWZlcmVuY2UgZm9yIElOVHgNCg0KYml0LCBlLmcuIGlmIERvbTAgZGlkbid0
IGVuYWJsZS9jb25maWd1cmVkIHRoZSBkZXZpY2UgYmVpbmcgcGFzc2VkIHRocm91Z2gNCg0KdGhh
biBpdHMgQ09NTUFORCByZWdpc3RlciBtYXkgc3RpbGwgYmUgaW4gYWZ0ZXIgcmVzZXQgc3RhdGUg
YW5kIElNTyB0aGVyZSBpcw0KDQpubyBndWFyYW50ZWUgaXQgaGFzIHRoZSB2YWx1ZXMgd2UgY2Fu
IHNheSBhcmUgImFzIGhvc3Qgd2FudHMgdGhlbSINCg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwN
Cg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:51:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:51:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183058.330993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIaq-0008RG-H4; Thu, 09 Sep 2021 11:51:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183058.330993; Thu, 09 Sep 2021 11:51:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIaq-0008R9-Dq; Thu, 09 Sep 2021 11:51:48 +0000
Received: by outflank-mailman (input) for mailman id 183058;
 Thu, 09 Sep 2021 11:51:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOIap-0008R3-Hl
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:51:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9bc9dadd-65f4-405d-83ec-8073e79dde32;
 Thu, 09 Sep 2021 11:51:46 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-pnuuh1hBPkeRCJYtQ35FaA-1; Thu, 09 Sep 2021 13:51:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4944.eurprd04.prod.outlook.com (2603:10a6:803:60::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 11:51:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 11:51:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0015.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 11:51:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9bc9dadd-65f4-405d-83ec-8073e79dde32
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631188305;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YPhBFtz05r6QxOq9a0dG0RzpyQqo+X417gOuB5y8k/M=;
	b=kjjmLZd/JIR+tf6GeQ+DxUg/WJXer8mnayyCYpAt7Bz2aKNp6rUv0LS0tB81iJiieh0i/2
	b7QdMxkZ6iqPtO29nCx1ws0mkv8Gh7tvkmyU5KPHEW69AhM4X9A8LqOs24zO5ZBwauNcr3
	MS/PkhUvomaNAQAOZOz71c60YsLkaz8=
X-MC-Unique: pnuuh1hBPkeRCJYtQ35FaA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EJEAIjzhPeDtTk0/nwARO0bp3z3HtqFxyNH2Zvwfm8iyBumpo17OsNDqJl0rYVp8FhbJ111eGePhkafbTyogKhmc8mLmJGX//lP90EwMq8od8KiUIZ6SiterMhpdn3By6cQ3Jzk7BRIHJQREkGezEmLFc1zUasA5FrJIG73PokTrdSW107xRL1BZUk/yN0WRUzhYkhFNIJBvPI4amL/Ea8HDXhUlOFfENEKWafer38tLGpTvHpHzeA+ekjgHOxZ2sj4HTL2L9wxVY3fUFkmj0hcZCUSJhlD/4fn1T1EuQ8yiJsKtrzJP6P/qITwNptdmDB/IZJla+svvLwZlxi9q3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YPhBFtz05r6QxOq9a0dG0RzpyQqo+X417gOuB5y8k/M=;
 b=iYNoIi8noba6aWXkMvsx8Mol7RKVQ14OxthGGHcqVmrwnRTVKA+5gqZtAP7IAaSMw19Yd3Um/T79lHboSlWX/Uy0cdrfTHPC+t5Rrp+1QXweBc6UkwcYimwyG/O2wH02cbgtcQp7+8ZZhouKX6bE1oq9UXKEyPx0VZBEA/s5ntnIwlf4RRAEgoRbwJBLCcKUTRgh8Pm+ZKc2Fy6Pcc1MW1yjkrlDZafJ/yCwXTt7+iQjs2JBV7cIUsEtIPQwvo03Qke2wQ7kBZFS3MwrgkK6NhUasfz/pUKM5iw/QyufQDuZ6PyK9moeHfYMqG3hiJm2M5aDdYpLjY8A4qorGlQzQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-7-andr2000@gmail.com>
 <1fb1223e-005d-dc78-cd34-dc97191b316e@suse.com>
 <ba644629-a251-4865-8a7a-075392b9dffe@epam.com>
 <7e0c9d9b-6087-83ef-f9e8-1e9e6eb2d1e9@suse.com>
 <206c7b6c-6c0f-ce5e-6e62-15b46035b036@epam.com>
 <422a6543-ec2e-0793-3db5-09456e04f65b@suse.com>
 <dfb66ff2-9c9e-645f-4789-2dc6c21ff751@epam.com>
 <27043082-bf9f-5bba-f5a0-bb0e10426aba@suse.com>
 <e154e4f7-bd60-2645-1ce5-1c2b5404236f@epam.com>
 <3e7dc9f3-1536-0643-b732-4a203bee34d4@suse.com>
 <d81e9eb2-c16c-1f59-644e-09bf3466be76@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8927d56b-7449-15cb-1dba-799d7f027e4e@suse.com>
Date: Thu, 9 Sep 2021 13:51:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d81e9eb2-c16c-1f59-644e-09bf3466be76@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0015.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7800d995-67f2-4cc0-215b-08d97388312d
X-MS-TrafficTypeDiagnostic: VI1PR04MB4944:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4944CDBB3BD4BA10694A2729B3D59@VI1PR04MB4944.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2N7SjugojQjxd9rcjhAYcRwkukwisalID9Mu2Y29YEj+mej0VnatqZrL/vjprYwckqVqwRTkEeNZsBDU+PV0/+S8tXMvaf45GfTc4wywZ5Ajl6Ve2CZ04ZNB/yHcegdYc3OyOgLoe/Eo13O/H1cniIKqNH9ei+F7euwM2/6JXJxEEUJX/E3SaEK2XRBUrdfBT1wp4XuAfCURELiXPzY1EP6pw3rpzPmVdFY6kvp4hNlyHhv7t0opfgag8uH6CVznQnNDXIXi8g+JK4UeuOnDD5uxilAMkFwClaLpCiL74K7XKN5Mng2APq1UL14j2R/63+qtRBFAvjUqvbqjoDm+VeEkasNX5CclLAljiuo1eDoiK6ia2wvrnVHxWGYSXrkLaSAwgB0VwVK/wiUi0knjS46tcArNuWKKdNDfM3RN8Q6Kx0J1vz4yqIhIDjvW9hrKs8qeiIv0fs2GyW/iu0Pv0Ya8F+tQBOPL2BB9OEhVm4+FRvFQ5K+smUDH6x0/jgBeAQyDnRMGUZSWUqeNamdCuTypO1SzH73x8uW0yzqrGOhltxqJ3LF8EhzFjpsbGSgKaScphRrHwhc06UMd1FA5tfjj4Nznsdvv7MebROANYhul9G6rCFAHErMmxWUE8trTwu2RfDAMIyqtoTIgNq5+3Czxgc0GvGs+0uhqOFyytE7Wma1Sgebk7mqdYpwEshMNZSEFad0G8E/olrHfkKE3f7trqmM32E8gbef9YJw4hWjfN3smmlWRh/NQrq2h57pH
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(366004)(346002)(39860400002)(376002)(16576012)(8676002)(31696002)(2906002)(7416002)(54906003)(6486002)(26005)(53546011)(110136005)(66476007)(316002)(5660300002)(66556008)(66946007)(83380400001)(4326008)(36756003)(956004)(31686004)(2616005)(38100700002)(478600001)(4744005)(86362001)(8936002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RWR1OUFhbkkxaDhmdUVxc0xXeWJlaDlhbzhKdUZVb01VSW44OHI3aFErNTEr?=
 =?utf-8?B?NDhSVUlrUHVVRXUvVFA3YWZYdHM3MStrWXExenpHSENUZno0azdGZjdUZnFr?=
 =?utf-8?B?V3l0VVI1N09pUjNTV1hxMCtwTkpxRmN0QzA0YkZTRWV4R0NpSzlIUWF5clZk?=
 =?utf-8?B?Mm1qV3dRU1c3SUpMV2dqbzVMTHV3YzhKWjVFY0dtU2k1czJsUjF2T0MyQTZI?=
 =?utf-8?B?RHNiTjluMFhvdHRTL3VpbWhjQXpyS3U4NTgxd3hlaVhxZytSNUFDek13Z2tM?=
 =?utf-8?B?bTJxZ2R3RzBUbHRYVWhTNkR1Ny9GdlhVSDJUUFNxWnNCOG93T2RBdmhOUGVM?=
 =?utf-8?B?OUJIbFJaaWJTdndCSmE4QVpGWS8zVHBIUlFER2prNURQV1NGY0ZDSUZ2ZTUx?=
 =?utf-8?B?ZEoxUHM4d3l1dXNCZ0dEV2VqWlo4OFNiWHU4SUdzM2ZUMmFnVnBWTmFpRXBz?=
 =?utf-8?B?UnRSWDJKRUo5MEtqZzFtSEw0TnVhTkdzampjWTV0K1pGR21OVkdBK3pCNUE1?=
 =?utf-8?B?dGF4QzE0ME9LeDU4WC8rZFZlSzRWeDBKNmZ3Z2M1c09zbGpTbHZqVm5Va002?=
 =?utf-8?B?WC8yZndzVm11ZUNtTHdMZi83RG05eGxVZ2dVaWlna2VSd1ZHVzNFSlV1Q2l5?=
 =?utf-8?B?VWJxaTcvb1lCVXFDZ3VZcWdUcjVHWVpmdmNkWTUydUJ1ZUsyNnZkQlFRdE95?=
 =?utf-8?B?bEtCak1aTTlyenFhRVlmYkdwcjJUUWRxam9VMUpBSkp5b3hvTnZrYUpVbjgr?=
 =?utf-8?B?cWZsN2RSbnBWMVo2b2Q0WnR1dGNmeDEreEMydkNuYndmRzNZd015NHFjdG1D?=
 =?utf-8?B?VzZyMDRnbFROc3pQbHFJNzl2bDdKZituQUszelU3QzFaWkNUZlg3a2pEcHAy?=
 =?utf-8?B?NzhyNkpFNU9Da001V1FzYVpuc1RUUWZKcUtIdktRWHpjVW1HRXR0N05BNEpE?=
 =?utf-8?B?bERYNzZrMW5NOG9YdjB4SHMwTVlsdi9LWldkOW1HM0VlaE1aWTFHVm43MUt3?=
 =?utf-8?B?ZDQzNHZMQ0I4Z1pnYlk4ZXZDYnd3bHRObXVXcGkvNmszV0lPU2o1QWRxQlB2?=
 =?utf-8?B?UDFKazVJZ0hWaVk0NnNWbTYxTjNqTlZJVytkZ1NPZlU3MTZqQVNDYlZyU0xY?=
 =?utf-8?B?QmdGeCtyS1Nic2FIY0FsNU0vQWExckpGcjR4WTZPREFOQy9HcGI1UWFOT3BU?=
 =?utf-8?B?bkU1em9QTWtJU2tQTDBwSXdTUy9lVXZSYXg2U2IzOTc4SjVjWlo1U0FWYmNX?=
 =?utf-8?B?VHZsVGdxYmUxUTFqL0xZcXBwWGZRb21LWlRkajk3VTlwM2VSaUdnMzBIT1Nz?=
 =?utf-8?B?K0k5Rno3NWFtRnc5aFoyRWhPV2UranFkbTgrRjk1L01lTDU3MHNnZUJXRzRw?=
 =?utf-8?B?Q2RCblduT0liOTNrT21GSkYrbDdGaGQvL0xvSFJIeUV6aEloTHp6TjdpRUJD?=
 =?utf-8?B?NWlWdG9YTWJMcWI2dTc4T2them1xQjd0UW1QVzh2aUFPcFF0UUNobHhyZnZL?=
 =?utf-8?B?L3ozSzEzYnVrZVJ5QzNDRVZNMTFUNGtkNDdtcG9EVDhBWkJwclRQbmIweVow?=
 =?utf-8?B?L3ByR1lJbE9GUTFSLy9XMFJsWDF2YzlnZ012aVllZlY5bnk1QjZtbUxBYzNU?=
 =?utf-8?B?MnllbUdIZmdheXRmdlhWWS9pekt2TXJ3QjU5eVZDZWVad2dLVHUyU2tsUy9l?=
 =?utf-8?B?OFdYNndFOWJPU1dKMElGVEh2Vzh6UHBwZUltemNWOHZSdmo4MHJDaDRPcmJw?=
 =?utf-8?Q?cV3Ysn8owLcOw+J/8qTkr4DLqI4iOWE2BynnwEO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7800d995-67f2-4cc0-215b-08d97388312d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:51:42.9102
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 43oHzWWZLsJ0vpFk1nU5TPu6hLxBCUYFW37pqKJuEi7hcIvcAdVFeS+RmOizuQjKdrfPOUaVkcxeqd3Wd6YAJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4944

On 09.09.2021 13:30, Oleksandr Andrushchenko wrote:
> On 09.09.21 13:46, Jan Beulich wrote:
>> On 09.09.2021 12:03, Oleksandr Andrushchenko wrote:
>>> On 09.09.21 12:39, Jan Beulich wrote:
>>>> And in fact I wonder whether for DomU-s you want to support BAR changes
>>>> in the first place while memory decoding is enabled.
>>> No, why? I want to keep the existing logic, e.g. with memory decoding
>>> disabled as it is now.
>> Afaict existing code deals with both cases.
> 
> Hm, I thought that we only map/unmap with memory decoding disabled.
> For my education: what happens if you unmap with decoding enabled and
> domain accesses the MMIOs?

That would depend on the precise timing; it's certainly not well defined.
But supporting this may be needed for quirky OSes, as said before, as
they may get away with that on real hardware if they avoid actual accesses
at the time of the BAR change.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 11:54:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 11:54:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183064.331004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOId6-0000eB-Uc; Thu, 09 Sep 2021 11:54:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183064.331004; Thu, 09 Sep 2021 11:54:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOId6-0000e4-RV; Thu, 09 Sep 2021 11:54:08 +0000
Received: by outflank-mailman (input) for mailman id 183064;
 Thu, 09 Sep 2021 11:54:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOId6-0000dy-4J
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 11:54:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0f586ebe-8d29-4fe3-a23b-a8e6613f48cf;
 Thu, 09 Sep 2021 11:54:07 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-1salnaTRMXi3yILUMtRNeA-1; Thu, 09 Sep 2021 13:54:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 11:53:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 11:53:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR07CA0081.eurprd07.prod.outlook.com (2603:10a6:207:6::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Thu, 9 Sep 2021 11:53:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f586ebe-8d29-4fe3-a23b-a8e6613f48cf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631188446;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TgsW9dSRnjtKa60RLy4P2pvYkjr9xFazVfaG+r2YUiM=;
	b=W7tF26TaKxSr+jUJebVplR2bXuYezhP0+roZajXsu7Lk4inE1Rn/M9bpaUwpHf8monOqNX
	ZtLTSIbkAvd/6c57ggLc2tIJNK0mtc+hdoFQM0lVH/1LbGJ8OMjOtnSiQ5MaZrrIsAxQqM
	HxQlOjRDl2/kzvllBP0uoa5lR0R48vk=
X-MC-Unique: 1salnaTRMXi3yILUMtRNeA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e9jtM8tAumPlVyMT06PX7MInqmCz0zjNbFoTUqvlUGz1fC+uMNXli1/jZXyyfRSXDd1p6DQ6Ba3kNNu0B8G1gxbtd244cLODFRhRN4gqRzxrb2upxyppnr7vSuGg7ZpU9Ja62VxhmcOs/aBhBXJ9kXgPLsX5Y0S6Rlrmmun/kO4QRnvPtO+og1cNky6TH9ZNaN1Zi0g02dL0YxW0b7fsAQ6nBmDXeZ2RkYo9+p0OaQ5Ch4zmft98fqmUob6jK6w5HBk9wP4mRFaibGgsylSeTTbCOQ5BU79X6YF1mUEBzRPIG5bxHUfeu7siEmHTFBRx20RY8qrPGpekczZvquNZfg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TgsW9dSRnjtKa60RLy4P2pvYkjr9xFazVfaG+r2YUiM=;
 b=emaD0BFYl1rP6DYAZ7i+ZLf74NFpoq4+x0c0Qwlv+N4/yJf7zf/+TSFDEmCuj9SMdswTrebh0ygWyfmolJqXGpkTyKYZhDRFx2ZXDwCCTght+Ec0onVxUCy9TNwtvbX+QJMlVuxA4ZBlGCyRY/BFsLzwlECfaf3IkHddFrmh3ng6X2UcuE2oj1gRc+X2lBMeLlAaDIbFTcE9nJ4872b+5XQ3krwvzH1ZAjM2S7a0asIV21CB7y+BU1cotxXJLSHCtN2HWtoV7OmpEgpGughEjZ+dDTUS6CMK5KaXgLQ7xHu2RiGnz7vajv3kBUg+nrfUFAHyYIMyNZDQhqMwfm2u1g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
 <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
Date: Thu, 9 Sep 2021 13:53:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR07CA0081.eurprd07.prod.outlook.com
 (2603:10a6:207:6::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 816cf052-18fa-48f5-c7e8-08d97388824e
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68630E2D53D6447A5FC26ED3B3D59@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kadVDskirG/uW2IJMQgpWcEo1tNIn4exKzgiyy60qJR22yLu3JL/hY3aMx50ct0oRKQDwdN/o8j7NHpme1ErcA5LGCY2+q3jAa5SFCnrDTlciUfcPrMAvxYliRp0czaecqDUE8zRjvhJu5nYJacqgfblcSk2r/+zdTJWQiwGB7bzotOjcxsoNwmeCaNH0LCi9m3+4WobSayMX31Ke6yTHCn9NCHDJwB1F8bxXtBZpLOkbt0kN8/eKHN1TANoOdnFX/iv/kOSLvavlo/q0U7I5p+K5XHxBtXv8ZkH1AfCfgKweVI/Y/rrtzeWjgm0ino6G/0M1vmoApUqiiA7KkK44QhOFl/bHtqc/9u3vPTrkb8iVcGU0yrbcmhgYp8vzz5hkgg4HbZma932KP6IInT+gDeRsheUroyFnQ1s+T6amAr8I2R6bKJZAsy5rBrIF4DBsI7AV6A5I41fPxTrNPzKUc5mpEutgRQVbpNjQwZ0cNkVTT2enaR/JQ4DIArvnvr9wFDgLhhiAGOj87exwO4m5WVHbt+2FD74eGE6QEkzVj/XtFSNemRJX15eepgzal2GWe+ukiqpLHyH0UOetHhs/aidNmwvNYccLevZ/gHRNpXMJOAqb8Wwe0sUWAfusseEekHw9hEA4M5eEP4myzunS4cUnpg1/LBFb4IeK9MUrp7iTtldCUhck1CmCSIjMBzqJxZn87iYaDyd+JW6uXoBK7QVOSq9rycyVUOhn+L49gCYF6xNYqbWaDo6SRoksidS
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(366004)(396003)(376002)(346002)(39860400002)(6486002)(86362001)(66946007)(53546011)(16576012)(31686004)(2616005)(38100700002)(66556008)(36756003)(66476007)(956004)(31696002)(83380400001)(26005)(186003)(5660300002)(7416002)(4326008)(478600001)(8676002)(2906002)(54906003)(316002)(8936002)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QnRMbWhJOWZvaUhRQUsweks1eEdWQmVGOFE3TmJMY0luRXhQZU1OY3JKYkk0?=
 =?utf-8?B?WmZwZ0FXdWVXL0pvRHpoa1YvTWVya1p3cjIxSVYranVhZUcxVDFwd244M1N1?=
 =?utf-8?B?RFZweUVrYjk2ZDhmTGN3YUNpMkpvRENEbEpMT2xVNE5iK3h2Y0ZPYm56ZnNi?=
 =?utf-8?B?Y2RRR2NaYk40ZGhRMk8ydDhWODZwamJjZ2thQUsycjZwSnJJazN1eGZRUENm?=
 =?utf-8?B?K3BTN2pzSWFjSlc5ZFpWYWlVQjg0V2NiUWRTNncyL3VhM09BUzJGQ2dyMXF4?=
 =?utf-8?B?cUdHSmptRGhsL1FlMkdJUnJlRE5SQ1Zla2ZvYUtFSFFYcmVMU3pHa3c5aVk5?=
 =?utf-8?B?M2xIL1BDOEdmNVFvbUVmd052UURzME1aTlppemN4ZGdJSC9BOFd6NlRBVStl?=
 =?utf-8?B?UTk4YXdHQmZaVHU4MldtdFJQQ215blBBQUIvMmd4bGxGMTZvOXZ3TU9UcEdh?=
 =?utf-8?B?RUxRNnNkWTdwNkM3SHBrY00rMitGUlpaai9DSElEYmpaeHdzSmMwS0tUZlBY?=
 =?utf-8?B?UWlabW14N1ZlVVVCRW96ak4zdHozVDRJVnFKVjZSMWFIN1RSTktkU2lHdUpp?=
 =?utf-8?B?TTc4S0N0TDBnZkczWGw5VFVpcWsrSVRCcG44ZGFsYjNERzVRdHBKeHNHL2hT?=
 =?utf-8?B?bzlBVzhrUHRMbFloR0VQNjRjQVA3SkRoYUVFM1BDTDZSOUNDVVVDMGhIN3U5?=
 =?utf-8?B?NHZHODQxMDZiL1djMFhuMy8zYkFpZ2dFRkxKaXZ4RzkwSFhkMGp2RnRHYzMr?=
 =?utf-8?B?WTV0dGRLL2RUUThqZHppcGxwZDFwNTRnVEt1Y201ZXkwV3RmSTFXazI4ODFp?=
 =?utf-8?B?RzFrUE5vaUt4d2Y0V0VkSHVuemRSRVp1bi9lUFZNQVBSa0FRRnNXMzFsOE1t?=
 =?utf-8?B?aUd4b284a3JFR3hRTk1ZMW1UbEpHTUlpaEJaeFJyYkVUY21CaG1FM1NvZVNa?=
 =?utf-8?B?TE8vKytPek1CakVyU1ZtbkpGeGJ1YWlpdHBPcEo0ZVprREFXWGhzZ0YyZ1k5?=
 =?utf-8?B?VDNQN1RIMmdqc2NyUjF4ajBLKzZhVVRsVTRLcTVPSWNzWHBpTGkzTERGWXox?=
 =?utf-8?B?QVhINVZadnlWUzUvMUZIbXB1SW96NDhkMm0wamU1RjdOTjdnUHE3SDc0ZXVt?=
 =?utf-8?B?VnlocVRtblRMNFUrU0V5Q3dqSXZ0N1FENk9mTGFzYUd2TW43bkFaazRRVWlP?=
 =?utf-8?B?ZlRmMXVuZWR4blVNakhKTkZpNFY5bTVNYUMwbUlBcWZIRzIvckdGckpaRWJY?=
 =?utf-8?B?cjlWMk1qSm93WTVSVlZaMGY2dDRvOW1WZVlyTXk2RmRjQzBTSEsrcmdYdG84?=
 =?utf-8?B?T05XdUVRSzhYK1VqbjdaRTFRTDlRc0RGak0zVlJ4REkxN3I2YkJhc0Z0WDRC?=
 =?utf-8?B?SG1LR2RpbjJleW9adkdTK0JVWEtLT0l6akJTd2ZORGMzZ0tCWmVmOTMwbWRU?=
 =?utf-8?B?UWY2UVlEckhnNGx2Ui9SWjY5UWpKbzV2RDB0Z3o5R0ZyaHNNR2M1RlZNVUs3?=
 =?utf-8?B?SDBMOUlsd1dMaXEwQVBEUmtsWjVLaTE1bFZxeG9xNStKNkhBMGdmWEVPdk1n?=
 =?utf-8?B?aWQrcnlwZjBzMXZUMW5Kd1cwNUJXVTBkQWQxeWRRTG9tSjFGVFlOMVF5MzlV?=
 =?utf-8?B?QXp2K3l6cEwvd2Y4d3I5R0pKTlVqcXdtVnRLUG94UVdndld2eWwzV0FoUkV3?=
 =?utf-8?B?Q3l1YkJsS001Y2NFOFc5OFkybmowZFAwSU5hbFVPQTcrTjNnTkVZOHMxbVMr?=
 =?utf-8?Q?wa2+Y3/xuFtN3dn+iBXjVdNLbaJIde7YjGIcj2X?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 816cf052-18fa-48f5-c7e8-08d97388824e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:53:58.9702
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cuuVGh0MV2sASbA9cITYHrOTlquO4rkV681HECJ65Ol9C75xRb48z7N6LWUP12g8yQUzecIVG+y7g2wLyeNUMA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 09.09.2021 13:48, Oleksandr Andrushchenko wrote:
> On 09.09.21 12:21, Jan Beulich wrote:
>> For the bit in question, where the goal appears to be to have hardware
>> hold the OR of guest and host values, an approach similar to that used
>> for some of the MSI / MSI-X bits might be chosen: Maintain guest and
>> host bits in software, and update hardware (at least) when the
>> effective resulting value changes. A complicating fact here is, though,
>> that unlike for the MSI / MSI-X bits here Dom0 (pciback or its PCI
>> susbstem) may also have a view on what the setting ought to be.
> 
> The bigger question here is what can we take as the reference for INTx
> bit, e.g. if Dom0 didn't enable/configured the device being passed through
> than its COMMAND register may still be in after reset state and IMO there is
> no guarantee it has the values we can say are "as host wants them"

In the absence of Dom0 controlling the device, I think we ought to take
Xen's view as the "host" one. Which will want the bit set at least as
long as either MSI or MSI-X is enabled for the device.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:16:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:16:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183083.331015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIyf-0003th-D8; Thu, 09 Sep 2021 12:16:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183083.331015; Thu, 09 Sep 2021 12:16:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOIyf-0003ta-9R; Thu, 09 Sep 2021 12:16:25 +0000
Received: by outflank-mailman (input) for mailman id 183083;
 Thu, 09 Sep 2021 12:04:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tzae=N7=bu.edu=alxndr@srs-us1.protection.inumbo.net>)
 id 1mOImj-0002l6-R9
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:04:05 +0000
Received: from relay64.bu.edu (unknown [128.197.228.104])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ddcee35a-216f-40cd-97da-bfe1c6a81e44;
 Thu, 09 Sep 2021 12:04:04 +0000 (UTC)
Received: from BU-AUTH (localhost.localdomain [127.0.0.1]) (authenticated
 bits=0)
 by relay64.bu.edu (8.14.3/8.14.3) with ESMTP id 189C32MN016396
 (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256 verify=NO);
 Thu, 9 Sep 2021 08:03:05 -0400
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddcee35a-216f-40cd-97da-bfe1c6a81e44
X-Envelope-From: alxndr@bu.edu
X-BU-AUTH: mozz.bu.edu [128.197.127.33]
Date: Thu, 9 Sep 2021 08:03:02 -0400
From: Alexander Bulekov <alxndr@bu.edu>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
        Thomas Huth <thuth@redhat.com>,
        Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>,
        Prasad J Pandit <pjp@fedoraproject.org>, qemu-block@nongnu.org,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Richard Henderson <richard.henderson@linaro.org>,
        Markus Armbruster <armbru@redhat.com>,
        Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
        xen-devel@lists.xenproject.org, Paolo Bonzini <pbonzini@redhat.com>,
        Eric Blake <eblake@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: [RFC PATCH 00/10] security: Introduce
 qemu_security_policy_taint() API
Message-ID: <20210909120302.nuq5churn6kjsfqs@mozz.bu.edu>
References: <20210908232024.2399215-1-philmd@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210908232024.2399215-1-philmd@redhat.com>

On 210909 0120, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series is experimental! The goal is to better limit the
> boundary of what code is considerated security critical, and
> what is less critical (but still important!).
> 
> This approach was quickly discussed few months ago with Markus
> then Daniel. Instead of classifying the code on a file path
> basis (see [1]), we insert (runtime) hints into the code
> (which survive code movement). Offending unsafe code can
> taint the global security policy. By default this policy
> is 'none': the current behavior. It can be changed on the
> command line to 'warn' to display warnings, and to 'strict'
> to prohibit QEMU running with a tainted policy.
> 
> As examples I started implementing unsafe code taint from
> 3 different pieces of code:
> - accelerators (KVM and Xen in allow-list)
> - block drivers (vvfat and parcial null-co in deny-list)
> - qdev (hobbyist devices regularly hit by fuzzer)

Just looking through the list of hci, storage, network and graphics
devices available on i386 to see which others are potential good
candidates for this tag. Obviously a lot of guesswork here:

USB devices:
name "ich9-usb-ehci1", bus PCI
name "ich9-usb-ehci2", bus PCI
name "ich9-usb-uhci1", bus PCI
name "ich9-usb-uhci2", bus PCI
name "ich9-usb-uhci3", bus PCI
name "ich9-usb-uhci4", bus PCI
name "ich9-usb-uhci5", bus PCI
name "ich9-usb-uhci6", bus PCI
name "nec-usb-xhci", bus PCI
name "pci-ohci", bus PCI, desc "Apple USB Controller"
name "piix3-usb-uhci", bus PCI
name "piix4-usb-uhci", bus PCI
name "qemu-xhci", bus PCI
name "usb-ehci", bus PCI

Not sure about these. Maybe ohci isn't sensitive?

Storage devices:
=== Sensitive ===
name "floppy", bus floppy-bus, desc "virtual floppy drive"
name "ide-cd", bus IDE, desc "virtual IDE CD-ROM"
name "ide-hd", bus IDE, desc "virtual IDE disk"
name "isa-fdc", bus ISA, desc "virtual floppy controller"
name "isa-ide", bus ISA
name "piix3-ide", bus PCI
name "piix3-ide-xen", bus PCI
name "piix4-ide", bus PCI
name "scsi-block", bus SCSI, desc "SCSI block device passthrough"
name "scsi-cd", bus SCSI, desc "virtual SCSI CD-ROM"
name "scsi-generic", bus SCSI, desc "pass through generic scsi device (/dev/sg*)"
name "scsi-hd", bus SCSI, desc "virtual SCSI disk"
name "vhost-scsi", bus virtio-bus
name "vhost-scsi-pci", bus PCI
name "vhost-scsi-pci-non-transitional", bus PCI
name "vhost-scsi-pci-transitional", bus PCI
name "vhost-user-blk", bus virtio-bus
name "vhost-user-blk-pci", bus PCI
name "vhost-user-blk-pci-non-transitional", bus PCI
name "vhost-user-blk-pci-transitional", bus PCI
name "vhost-user-fs-device", bus virtio-bus
name "vhost-user-fs-pci", bus PCI
name "vhost-user-scsi", bus virtio-bus
name "vhost-user-scsi-pci", bus PCI
name "vhost-user-scsi-pci-non-transitional", bus PCI
name "vhost-user-scsi-pci-transitional", bus PCI
name "virtio-9p-device", bus virtio-bus
name "virtio-9p-pci", bus PCI, alias "virtio-9p"
name "virtio-9p-pci-non-transitional", bus PCI
name "virtio-9p-pci-transitional", bus PCI
name "virtio-blk-device", bus virtio-bus
name "virtio-blk-pci", bus PCI, alias "virtio-blk"
name "virtio-blk-pci-non-transitional", bus PCI
name "virtio-blk-pci-transitional", bus PCI
name "virtio-pmem", bus virtio-bus
name "virtio-scsi-device", bus virtio-bus
name "virtio-scsi-pci", bus PCI, alias "virtio-scsi"
name "virtio-scsi-pci-non-transitional", bus PCI
name "virtio-scsi-pci-transitional", bus PCI

=== Tainting/Not Sensitive ===
name "am53c974", bus PCI, desc "AMD Am53c974 PCscsi-PCI SCSI adapter"
name "dc390", bus PCI, desc "Tekram DC-390 SCSI adapter"
name "ich9-ahci", bus PCI, alias "ahci"
name "lsi53c810", bus PCI
name "lsi53c895a", bus PCI, alias "lsi"
name "megasas", bus PCI, desc "LSI MegaRAID SAS 1078"
name "megasas-gen2", bus PCI, desc "LSI MegaRAID SAS 2108"
name "mptsas1068", bus PCI, desc "LSI SAS 1068"
name "nvdimm", desc "DIMM memory module"
name "nvme", bus PCI, desc "Non-Volatile Memory Express"
name "nvme-ns", bus nvme-bus, desc "Virtual NVMe namespace"
name "nvme-subsys", desc "Virtual NVMe subsystem"
name "pvscsi", bus PCI
name "sd-card", bus sd-bus
name "sdhci-pci", bus PCI
name "usb-bot", bus usb-bus
name "usb-mtp", bus usb-bus, desc "USB Media Transfer Protocol device"
name "usb-storage", bus usb-bus
name "usb-uas", bus usb-bus

Network devices:
=== Sensitive ===
name "e1000", bus PCI, alias "e1000-82540em", desc "Intel Gigabit Ethernet"
name "e1000e", bus PCI, desc "Intel 82574L GbE Controller"
name "virtio-net-device", bus virtio-bus
name "virtio-net-pci", bus PCI, alias "virtio-net"
name "virtio-net-pci-non-transitional", bus PCI
name "virtio-net-pci-transitional", bus PCI

=== Tainting/Not Sensitive ===
name "e1000-82544gc", bus PCI, desc "Intel Gigabit Ethernet"
name "e1000-82545em", bus PCI, desc "Intel Gigabit Ethernet"
name "i82550", bus PCI, desc "Intel i82550 Ethernet"
name "i82551", bus PCI, desc "Intel i82551 Ethernet"
name "i82557a", bus PCI, desc "Intel i82557A Ethernet"
name "i82557b", bus PCI, desc "Intel i82557B Ethernet"
name "i82557c", bus PCI, desc "Intel i82557C Ethernet"
name "i82558a", bus PCI, desc "Intel i82558A Ethernet"
name "i82558b", bus PCI, desc "Intel i82558B Ethernet"
name "i82559a", bus PCI, desc "Intel i82559A Ethernet"
name "i82559b", bus PCI, desc "Intel i82559B Ethernet"
name "i82559c", bus PCI, desc "Intel i82559C Ethernet"
name "i82559er", bus PCI, desc "Intel i82559ER Ethernet"
name "i82562", bus PCI, desc "Intel i82562 Ethernet"
name "i82801", bus PCI, desc "Intel i82801 Ethernet"
name "ne2k_isa", bus ISA
name "ne2k_pci", bus PCI
name "pcnet", bus PCI
name "rocker", bus PCI, desc "Rocker Switch"
name "rtl8139", bus PCI
name "tulip", bus PCI
name "usb-net", bus usb-bus
name "vmxnet3", bus PCI, desc "VMWare Paravirtualized Ethernet v3"

Display devices:
=== Sensitive ===
name "isa-vga", bus ISA
name "qxl", bus PCI, desc "Spice QXL GPU (secondary)"
name "qxl-vga", bus PCI, desc "Spice QXL GPU (primary, vga compatible)"
name "vhost-user-gpu", bus virtio-bus
name "vhost-user-gpu-pci", bus PCI
name "vhost-user-vga", bus PCI
name "virtio-gpu-device", bus virtio-bus
name "virtio-gpu-pci", bus PCI, alias "virtio-gpu"
name "virtio-vga", bus PCI
name "VGA", bus PCI

=== Tainting/Not Sensitive ===
name "ati-vga", bus PCI
name "bochs-display", bus PCI
name "cirrus-vga", bus PCI, desc "Cirrus CLGD 54xx VGA"
name "isa-cirrus-vga", bus ISA
name "ramfb", bus System, desc "ram framebuffer standalone device"
name "secondary-vga", bus PCI
name "sga", bus ISA, desc "Serial Graphics Adapter"
name "vmware-svga", bus PCI

Sound devices:
=== Sensitive ===
name "hda-duplex", bus HDA, desc "HDA Audio Codec, duplex (line-out, line-in)"
name "hda-micro", bus HDA, desc "HDA Audio Codec, duplex (speaker, microphone)"
name "hda-output", bus HDA, desc "HDA Audio Codec, output-only (line-out)"
name "ich9-intel-hda", bus PCI, desc "Intel HD Audio Controller (ich9)"

=== Tainting/Not Sensitive ===
name "AC97", bus PCI, alias "ac97", desc "Intel 82801AA AC97 Audio"
name "adlib", bus ISA, desc "Yamaha YM3812 (OPL2)"
name "cs4231a", bus ISA, desc "Crystal Semiconductor CS4231A"
name "ES1370", bus PCI, alias "es1370", desc "ENSONIQ AudioPCI ES1370"
name "gus", bus ISA, desc "Gravis Ultrasound GF1"
name "intel-hda", bus PCI, desc "Intel HD Audio Controller (ich6)"
name "sb16", bus ISA, desc "Creative Sound Blaster 16"
name "usb-audio", bus usb-bus


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:18:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:18:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183091.331025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJ0Q-0004Vl-OY; Thu, 09 Sep 2021 12:18:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183091.331025; Thu, 09 Sep 2021 12:18:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJ0Q-0004Ve-Le; Thu, 09 Sep 2021 12:18:14 +0000
Received: by outflank-mailman (input) for mailman id 183091;
 Thu, 09 Sep 2021 12:18:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mYjb=N7=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mOJ0P-0004VY-QT
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:18:13 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 238194ec-479c-43fd-bb6d-1362f3983df9;
 Thu, 09 Sep 2021 12:18:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 238194ec-479c-43fd-bb6d-1362f3983df9
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631189891;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=pimxR8QoeSvNAouB9tomCBO+RN+Z128zQ7MLxIyRVgk=;
  b=Z1pEZVR5oI7oxsASyALyg3/Hbz3jgrvdvIKf5iMv1joHfEetjM/4kVVk
   166ykYyhFupm+5wrsydzohV6yRjlwVSQmmgMUbp5gQ6CtuGRRLLFIPK2T
   lLW+OTBluJ/I8GPAUTaBaL/GFRnQbzr+p4fRiw3qHp8kYZz9/Ht3EuHGN
   g=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: bJwOn1BB4zuZ6WI59OkbbUH3a7CYukbRZaK7hSck0QADhJmJ/ayfh7EWOlfAgJ3A4034esyLtp
 SpH5RcdOBifyxIPa322DJ6ICJAiVNYZZUvaIWmFDe+MQckbI0DiPgBglmouwvrxBrPiVEg86Hl
 mVDfzgsCNZQ6N0m6oJXVVYN0bBkmD+rwdcQ53BTKTLCG81IJzz+GK2UJxzUruKbbaXhnLA0GoT
 WaEBhI8Oi/M81AKZOh3LPhTJ82Hi0uCUvyk7r3CgnKd1cWsmEjOWpS12IRo9Sd/rc3wkzyY5Ca
 jSayMxh8ZfOyAP5smWP4sexa
X-SBRS: 5.1
X-MesageID: 51944700
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:1bBKzKNhwgl1gsBcT1b155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/exoS5PwPk80kqQFnbX5XI3SITUO3VHHEGgM1/qb/9SNIVyZygcZ79
 YbT0EcMqyBMbEZt7eC3ODQKb9Jq7PmgcPY9ts2jU0dKj2CA5sQnjuRYTzrcHGeKjM2YKbRWK
 Dsnfau8FGbCAoqh4mAdzU4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kLEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 bxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72zeBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlJXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbZrmGuhHjXkV1RUsZiRtixZJGbAfqFCgL3V79FupgE686NCr/Zv2Evp9/oGOtF5Dq
 r/Q/1VfBwndL5gUUtHPpZ1fSKAMB2Fffv9ChPhHb3ZLtByB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="51944700"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U+COnPC3ouNh6HR1/Nah3xQcW/2HtI1re7DxDuhNnsJuirJy95N8Nj2EPTA8yDAa4v/jBWGNECCYMCgTYjGyS/35QzUh8s/kAKg/qBiANqsMRGOfin4pr39lbFrf5c5E3oB4ZAaK7Q0T0a6S98Y7EgBSU1M++gk5wU6weKcsiQzME4o6P42Aen1m2xvNhhzUDY7Q41axuViAxJt8mcvKXC0TqKAvGyLcp2wscXLw5CuT6Za0ZPbjbl4Iw5KnbjkZ2Az9NqEi23Kfi3T2fVsAx9yOCIkaowI6dBWE9XpjzOtna9i+HZ7GZ7J7KLu2+gE/Np4eKqraxTg+Vj8mklZ+WQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HrrZx/U9kE3EnNeaVe7vUWQx6h2cb3clHB489Nw06hY=;
 b=hNlcZgztMDKfU7dgICqhmaXG9HnKegmY1ECDo7xAwNN+sOqTLclNevA7amGKI2jPglhNVzA+9kpFgrF9wjVjp3IMAlWqG1yyCFCV0qUrJ23v1IgWk5DRH0Uzyr6sAfXGVJQwhGot63t/WoQebhd3uYI5CV2P47RV+GcGkmzjo3buJXcLxyCqO6Vj6pHGFrNWmkL3xnd8s8CPCY2K+jgeSbf9isZuTejPlNLL8wnUie6TlPHMrJ4I2CNOqXHrJLE0pV+okEQF5xjf1VQKix+KejmayT2PfKZgUM3V6eDwni8igaQx5A9t4Q00Bdz3eUNgeTVe5SBPoFjvKkSXiNsnJA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HrrZx/U9kE3EnNeaVe7vUWQx6h2cb3clHB489Nw06hY=;
 b=PNhcIRe8NEd1U95CadS/AqnMxKxkSgNRlvFkT3ylfKq7Bpp2XFPDb6VKtHLLwx/x9C/31Do8VvFyyBqx7KyxHpMUI2if09+s8mR2GQQxAe1AbSxiQzIxy4R7tEPIVJHXDiL1OjNYxQ4JcpAB/RKd+pdMVTGabscC1Wi7Enfz+XI=
Subject: Re: [PATCH] x86/svm: Intercept and terminate RDPRU with #UD
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210908161918.25911-1-andrew.cooper3@citrix.com>
 <3f272ea6-02ab-8f70-19cf-28a92421531e@suse.com>
 <9a4392f1-dc8b-76aa-f5c0-1db70dce3b42@citrix.com>
 <b4603360-c105-8f69-2c1a-081ea2605e05@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <038269c5-2b81-b6cb-892a-41bbe272a138@citrix.com>
Date: Thu, 9 Sep 2021 13:18:00 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b4603360-c105-8f69-2c1a-081ea2605e05@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0024.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:151::11) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 894d4c8c-2292-4e65-9f8c-08d9738be0dd
X-MS-TrafficTypeDiagnostic: BY5PR03MB4967:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB4967D95B537EDCE5C82988E8BAD59@BY5PR03MB4967.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: BfIcuS9pyDLqBdM6/rrYPQROMfeHCDaFgCposPMZw+F+cxVX9/79AtN0S6F/enkbnGIa1pQjAFcNjZvhh1kbVaUGiOc9TjSoCRzAQR9nH/eYtpO6A/F+NP1lrHEiV23TA4wGjF6XnclyRy3MGRivN/Hy3js1DGq91iwQlTkRYEJDKk/pKNWiGBC6L6ffcJwbtP17kYbCWu7+htbrNq1sw01bU+KwIPv6Kp52WaL7kMFhWhugqgYFk3W/wC2jL+QbIY+wsg4YHJQUqMB0cUHQlrVS372e6y+7PXhzCxvCuAmK83cq2o1n3RIt7ICI+9ICLpUwbyjPitdkKb5RkJPRe3wV5DPUOO00KSB0dlphhLw5AT2aGUwZoInvCxbj70tfTQGaUv9jhHLghQ/Gb2wFS0AClqDNrmfLS4/VhYTEMPnbsM/malKJ8+wwRv809GEOG5ii2StTQf6qdLVDjkPiDIja9U3uEbYl0Hmt5pJwAENpW77dbyD0JZ1u/P8zT9bG03ONdCqTiZbb08CsMjqE66H8I/XUON75VtE9a4XX0ZlXFxCkPvpx7XCWsdELn+zC6b7Fpdf229gW4u42ZaSu1SSc4qc42vNCJAQsMcTCBAGC4ShJYwU+m5uYa0kiOsotBPiW0TLdh/7wpHIB+GJBN7wt6+Nwoz4Fmjkh0sYC3aZ4vbyDk5TgPzaatW0SsSwr2v6+zJqS3elk5nrXXAITayNOh0YyfykZWKxauMuBIh0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(136003)(39860400002)(366004)(66556008)(6486002)(86362001)(66476007)(38100700002)(5660300002)(6916009)(83380400001)(6666004)(54906003)(31696002)(53546011)(55236004)(8676002)(478600001)(316002)(2616005)(186003)(66946007)(36756003)(956004)(8936002)(2906002)(16576012)(31686004)(4326008)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R2NpdStLQWpIeXBjZVp5cVc3TXZicnF6eUdQdjVKd1RpWEQwTjZoMW1JNHBs?=
 =?utf-8?B?cUdwWjcybnJWS0xGRU1hc05EV3loeThNelozVzk1WFRBSWxVQVVaSmNmSlBY?=
 =?utf-8?B?V0dkY2hHQXRZZTNUV0hzc1BlQ1RON3hLK2dqSnB6dTFkS0Y4QjZsa3E0bXA1?=
 =?utf-8?B?eWlHbkFCWi9JQzlMUkRWem1oNkRtRFFQa25nV3BaNi8wL0pjU2RHdGlNaEd1?=
 =?utf-8?B?OXQ0Wlk2enF3YmVjczE3aFBISFhZUWQ1aGhZalRzN0ZqaUY1a1cvZkdEelpE?=
 =?utf-8?B?cUNiRFlDSi92S0p5Nyt4ZFg1VjA5b2VSSDR4RjMrY0RSNTlPaGtBbkFZMTV2?=
 =?utf-8?B?VmgyT2RUeVNBWnc2TkQxb3VRbDNuM21xZXo5UUQ5MkVmcjEvczVwL3JUbUdY?=
 =?utf-8?B?Mmtralp3ckhMc2RmclJpV3FOSGxGL0hKdHptdlhQVjljWnFuL0tndDVicWJO?=
 =?utf-8?B?emFvTm5RajdqTzlsYjU3VFBNcmRONVdpMHlMMFBOKy9mVlNEdkU0bXRZd01H?=
 =?utf-8?B?UmFIbHQ0U2dJSFRRS0RZdXpiTzVYaHZTSjZTVTc5RENGdHEyWXBsSzdzS3NM?=
 =?utf-8?B?ZENsTkZwTnZJN3BZWlpjNzJXVEN4QWRtRUlDdHhBNm5CVGs3QmdvY0JOMUhR?=
 =?utf-8?B?OXp4a0dWTDlxR1YrUy9XeXlicU9OUXlsOSs5anBVKzJNUzNRMCsxUnlra2ZT?=
 =?utf-8?B?Z0ZzaFlWd0JVNVluSjRFd1B6cFJHMVR0K3hQRHlsQWlBRHhJejRRR2RIeXRo?=
 =?utf-8?B?eCtjdUNEUFlNWWpWeWJaRzNvTTBjSENsbmcyRGhpdGpWaVlkb1RERjl6MGlX?=
 =?utf-8?B?Rjc4U3J1b0pabGpqcG5RRitJNHMrUU9RKzcxYmJMUkhGYmVwUCtHVE9wZTV4?=
 =?utf-8?B?RTQxWXFRWkNNRFk4OTFmeUZZYThucWpQamxWZi81N0ZNRlhBbUJjLzkvT281?=
 =?utf-8?B?ZUo2SCtuK1ZMc21mVHFjTThOUy9xenF4bU1VT3R1ZDFKb0l5TWhVRFpZMFd4?=
 =?utf-8?B?Wk5CRXpaaC9TbU1Ub2c1djlSZTBCVWdMOXJzMTZUQW9MZnFHSU1NWmZwS3Np?=
 =?utf-8?B?YWZ2bUU2NVNGYkxtdklpNEVzWkVWNTI1RTVBUjhOeXJMa01Gdm4wTkZ3MUZv?=
 =?utf-8?B?TUpMc0kvRDdKT3I4NTR1WEk1NEVQWGtReEorbERmOEJSU0ozRjZxUkxFTm13?=
 =?utf-8?B?ZUNPeTJYWlNFNGU5UDV6NlorRkVDTlZPdGtTNktBTVo0UnVlN0w5aENObUsz?=
 =?utf-8?B?azZYMDdGQzV2ekRJZHVqRW5xZmI5UmswZGMxTzIraE5kTkR2dXFRVUxDdW9D?=
 =?utf-8?B?eGp4YzlLdXBRbllnY1ljMXd1NEpLRWpibkx2Z0FUbzFWZFM2aFo5eUdPaGw2?=
 =?utf-8?B?eXFDMFE0SE1qWFZVdEQ3VEowdEthSFNBV3FpYVNmMlNoM1dTWHlFL3VXZkZH?=
 =?utf-8?B?M2o3bTJTTkFCeHhONTg0WVNCTEQxMnQ2eHZaOWtqbEVkdTB5MFM5YXFTWGxZ?=
 =?utf-8?B?d0h2SEpmYkZ1SCtwTEphbXN5bldGZEFuTTRZM0lyMzVwb0MxZjltOTZCdDhi?=
 =?utf-8?B?ODRXelpTdXhKak5kYWxoMjg2OHlObGJ6bzZlTUowZGhSTVNBaVZGbVJWc0JJ?=
 =?utf-8?B?QkJDTVkxTVMrWm51VlF4dTdETGhJUDJsNUJCbmtBQ2JwU29abktOMWJwcmtu?=
 =?utf-8?B?c3BXYWpmdDFEUU9Ka3J3L3pHRHg3cFhIVU1zMmhLMzcvbnIwRHJEb3I5bDk1?=
 =?utf-8?Q?SBwzOtBFoFV7+WPkUCGwkVaAP+Icm06NoNSJcS/?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 894d4c8c-2292-4e65-9f8c-08d9738be0dd
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 12:18:06.1034
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XAbyv5pIMr6Li7qE7ywBHVIQSXanKe227xrZKG6/NcxhWerTPNicA08NlTyVhgHFz/kvXnDDF8cybpEtwjn2YdsJZ+i9WCD9d2qWQdbFHqQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4967
X-OriginatorOrg: citrix.com

On 09/09/2021 12:47, Jan Beulich wrote:
> On 09.09.2021 13:34, Andrew Cooper wrote:
>> On 09/09/2021 10:57, Jan Beulich wrote:
>>> On 08.09.2021 18:19, Andrew Cooper wrote:
>>>> --- a/xen/arch/x86/hvm/svm/vmcb.c
>>>> +++ b/xen/arch/x86/hvm/svm/vmcb.c
>>>> @@ -70,7 +70,8 @@ static int construct_vmcb(struct vcpu *v)
>>>>          GENERAL2_INTERCEPT_STGI        | GENERAL2_INTERCEPT_CLGI        |
>>>>          GENERAL2_INTERCEPT_SKINIT      | GENERAL2_INTERCEPT_MWAIT       |
>>>>          GENERAL2_INTERCEPT_WBINVD      | GENERAL2_INTERCEPT_MONITOR     |
>>>> -        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP;
>>>> +        GENERAL2_INTERCEPT_XSETBV      | GENERAL2_INTERCEPT_ICEBP       |
>>>> +        GENERAL2_INTERCEPT_RDPRU;
>>> Some of the other intercepts here suggest it is okay to enable ones
>>> in the absence of support in the underlying hardware, but I thought
>>> I'd double check. I couldn't find any statement either way in the PM.
>>> Assuming this is fine
>> They're just bits in memory.  Older CPUs will ignore them, and indeed -
>> pre-RDPRU hardware is fine running with this intercept bit set.
>>
>> Honestly, I've got half a mind to default the intercepts to ~0 rather
>> than 0.  For running older Xen on new hardware, it will lead to fewer
>> unexpected surprises.
> I, too, was considering this, but then we have
>
>     default:
>     unexpected_exit_type:
>         gprintk(XENLOG_ERR, "Unexpected vmexit: reason %#"PRIx64", "
>                 "exitinfo1 %#"PRIx64", exitinfo2 %#"PRIx64"\n",
>                 exit_reason, vmcb->exitinfo1, vmcb->exitinfo2);
>     crash_or_fault:
>         svm_crash_or_fault(v);
>         break;
>     }
>
> at the bottom of the switch() statement handling the exit codes.
> Getting crashed (or crashing because of an unexpected fault) is
> surely a surprise as well (to a guest).

It is less bad than the alternative, which was failing to intercept e.g.
XSETBV.

Something's going to crash or malfunction either way.  Intercepting
everything makes it far more obvious, and limits the damage to only the
offending guest.

And yes - this is once again why we really need a credible support
statement for CPUs.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:20:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:20:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183096.331036 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJ2k-0005qy-6v; Thu, 09 Sep 2021 12:20:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183096.331036; Thu, 09 Sep 2021 12:20:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJ2k-0005qr-3p; Thu, 09 Sep 2021 12:20:38 +0000
Received: by outflank-mailman (input) for mailman id 183096;
 Thu, 09 Sep 2021 12:20:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOJ2i-0005qh-BU; Thu, 09 Sep 2021 12:20:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOJ2i-0005Le-6I; Thu, 09 Sep 2021 12:20:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOJ2h-0004Gv-RO; Thu, 09 Sep 2021 12:20:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOJ2h-0006ln-Qe; Thu, 09 Sep 2021 12:20:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jhEK8/ZnCjzCKunVuOtqTNG2BYgp8qAcDVpIV0btWx8=; b=Fx4hz9XnnzqANvl4CyTY/25GHL
	EaDkeFK5yo3g+TOKnZX4rmaaT1I3eIfRQeUbnRYSjBiJlyA4i2jOp0p3gh7E9YieeuHTgJfnftYea
	gwBEJSDN5vjY9lOyOSsDcOGBnj2+ysnToMwZwPWUHAKwL2l+7NkGRm2LAuyhBfqxT2jY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164889-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.15-testing test] 164889: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.15-testing:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    xen-4.15-testing:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6f92f38419d6bd052da9076a7d89534b1f85ded9
X-Osstest-Versions-That:
    xen=dba774896f7dd74773c14d537643b7d7477fefcd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 12:20:35 +0000

flight 164889 xen-4.15-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164889/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 163759
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 163759

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 163759
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 163759
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 163759
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163759
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 163759
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 163759
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 163759
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 163759
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 163759
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 163759
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 163759
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  6f92f38419d6bd052da9076a7d89534b1f85ded9
baseline version:
 xen                  dba774896f7dd74773c14d537643b7d7477fefcd

Last test of basis   163759  2021-07-17 04:12:25 Z   54 days
Failing since        164262  2021-08-19 17:07:29 Z   20 days    9 attempts
Testing same since   164889  2021-09-08 14:08:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Dario Faggioli <dfaggioli@suse.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   dba774896f..6f92f38419  6f92f38419d6bd052da9076a7d89534b1f85ded9 -> stable-4.15


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:42:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:42:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183202.331234 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJO0-0002vp-DM; Thu, 09 Sep 2021 12:42:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183202.331234; Thu, 09 Sep 2021 12:42:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJO0-0002vi-AF; Thu, 09 Sep 2021 12:42:36 +0000
Received: by outflank-mailman (input) for mailman id 183202;
 Thu, 09 Sep 2021 12:42:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOJNy-0002vc-Lt
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:42:34 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 66efb642-116b-11ec-b1b2-12813bfff9fa;
 Thu, 09 Sep 2021 12:42:33 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189BRQbA001867; 
 Thu, 9 Sep 2021 12:42:30 GMT
Received: from eur02-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2057.outbound.protection.outlook.com [104.47.6.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayhdp0bpj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 12:42:30 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7298.eurprd03.prod.outlook.com (2603:10a6:20b:26c::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Thu, 9 Sep
 2021 12:42:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 12:42:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66efb642-116b-11ec-b1b2-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mG7Bs+fz4tSWEDsPPq/ePr8a4LrZaR+bgOF45LsaUaBlDzNL7A20rw1v9YZZVkDznooTg9AjMz3QlY8XOfenRG7qeQ7y4gfJoGRLNx/IKIdQ0a3CB7ulEdHdOvTDCWAfw9m7+pTDCUq4tqmpvONE3V6PCuWR1YuTMcyaPs5qNTQHx9B6e7WsAMTyKUsJe9FxMZSRUVjuPrpri1/MJnLkNwATs/T+aN/vOn2NDMynVbJgZiaUHCav1y6oaFB56mlJ8xfeOcqQ0e7/J2BJ7RshQi/mDpc97vadm94EYfWUrKncoBMekw0Up8AgOTNq2b6l9+o7P6Swod64DCE4vHXXDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dKHyWW00OiPqWRQnC1OyVPFE5xoD65yUS79ETpBcvdU=;
 b=YduDzhyaYx1IB+LPZBpRlK744qawuwpxhITlnWsbfRGftcK2tt/zXcV65ewoKPZN/80xGau8QTQmIZsZlIMzAz4kSZkFqcO+Ou38mPKAK6YzP0rIOYT/Eet3JcleFk1r1j7l9XdkHRFbu6IJuCdOXG4vrjnAFvNOfewsNzGVpZ93+GUAlwLEuE3MQfjk1o31fN/z4f5HOsOfaPGJyspYh19WJnaz6j0PJ/4Y76jU7yMkA/QaIv7WIvQ5w9eC+0yFQtB5XeKlogdOtyOKGyLpInz3WmfSoem1fcmKqBooeL8OU2z/gKMGqMMQBpu7oVxj73FyPVtZmUtpHakz8+KJIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dKHyWW00OiPqWRQnC1OyVPFE5xoD65yUS79ETpBcvdU=;
 b=J14nCagwaV0aAnuveTTwGXt76pOY2me4V9er5hKixDKWub9ATGsWNLQc1e+fuxoc0IROXkdtG0I6x/MFmD2HrwQrJAHBU2tLBvwmGLnIvCKaouO0Jzj5b6LGezMhJBi2itOrBFM4cRAyOaBtuLkDsqMFijHvxhSaV6klkCVNun5yd4jE1Zt2SdpIgDUAXMUQTpygIEzRW2TMwC67eg2ImqZyz6f9pR45JT/Pj010q17K1CpUhmbJLYHWtGBvrJ4sNME/lg4Ym+FieHcEf1njPo4icj09hKOdXQhjtPSqwMVg0KPLEwjjE9BOsUYvEoVl7tbuxRPNKiIZvATunPRBZw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AIAACKGAgAAo+QCAAAGkAIAADY6A
Date: Thu, 9 Sep 2021 12:42:27 +0000
Message-ID: <38669583-3e3a-36e2-8714-a4b6a4e10173@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
 <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
 <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
In-Reply-To: <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7f9d50f5-b2d5-4718-35fb-08d9738f4833
x-ms-traffictypediagnostic: AM9PR03MB7298:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB72989C974245E2BD6C240224E7D59@AM9PR03MB7298.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 C5XUyF2a737ooqdl8l+PbEx0T40GDf/srlu7w/LX2ZT2rv2qFXhAG8x6FxBZARUjZzNkToaCfVmGyAcw5dyh09YMi4h/UegZ57PPOVeJ+EBOpRtNLYUoGGyENCwX9/kMsN5NlJa4f2nVmtuK63nwurQ0Jvdy7creP3D34FLGyoqODX90JBnFkn2gZc+x7T2pHJg8nUbyxc4NuznRAxcLE3Zz58XyGboZ4abm5SrMb78BVxviFgJDFKhdx1OqdT5hv1jN5/bWER/VJa206tAiSllSArf2NdblhG2EBDLR2hCHdnIPu+IZJ/hsAsIzx3F6g5hpwq3XcCDmkVRT90zwel2AO7hicSJ2giszV38rKFPabWNWN6QpVe7srrke+L9+PZac2zrfeyVSwU59nZoBxeyMxi8JefJC86wXBQ+pychd/2gz3D6gFy1RsKEGG/NYzn8p40hSVm8MoCiaic+28hC9HPTf9yYWS/l3sVuSHHCkqxCUsIVeSQHWTl3hSV11eb6DXq4dCu/3VS7oB2uRS04smhVPZWBHebkXCn0+dgNeE7yj4WPDGB4E2ZU31WPY1mRZ2udlQsOKprnqPBEfka2Mwv36yOwA6Om9TrwfVC5KHiNIanSL7L+8UhmVkhxrqQlpiOHX6vcWHOoSgJpSjcry+2L9ZLhQUWbXxPIwAf/7Dhtl6Aw+gt5NxMjxs2RFFRAY+pGdEB+5bWtGHHo2R0v1Fn9F96lsS7ZwSEUXaVGOUIOo8bdsFP0rJA/IsvpF
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(346002)(39860400002)(376002)(396003)(2616005)(53546011)(36756003)(66476007)(66556008)(91956017)(2906002)(6506007)(66446008)(76116006)(86362001)(64756008)(478600001)(31686004)(66946007)(8936002)(71200400001)(6486002)(8676002)(5660300002)(6512007)(38100700002)(38070700005)(186003)(110136005)(54906003)(4326008)(316002)(26005)(122000001)(83380400001)(31696002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?akhYOVpIcUZzNlBRMUJlMDZlZkhFVGlyUTZ1N1dCV1VNSXVTSDNBRG8rYkUr?=
 =?utf-8?B?R1ArSG90WEhQaThWVzIwTWgveHBTR1pyWFp5NlBuVHZscjJRQUZTMXY3N1lM?=
 =?utf-8?B?K3NIenQ4SmJKb2FCRUQ0RkMxQ1B2bnpqMHJGc0hia2ZVMXFjaTM3ZEtSb1Fx?=
 =?utf-8?B?bjRNeDJRUWpEcmpYUTh1VUN4K0dxcjRLTURwd1pvbkJpU0hWUzBpbnJYR3pq?=
 =?utf-8?B?a251VW9ObEJBRnpQVGdTYzJna0paVHZWWTNuQUNwbmJwWldNQWN2QjNNYVJY?=
 =?utf-8?B?VnJnbXZYb1dTbGdTTzBuRC9BdXpURlRwTE9GSjhBMEdkUmRIMHlUN1RIQ2Ri?=
 =?utf-8?B?dXZmVy94NU9hK1c5S1NPd2p0a0Q4blovQWoxZzNxdktXNlFrK2w0aXU4THdS?=
 =?utf-8?B?MmtLbGp3RmtXL3l0RGUwVG95R3ZQNDBLMTVwWmk5NWoyRGN1dWVESitaZHhP?=
 =?utf-8?B?N0xYK3RHak9uNUg4Wkl3TnUrTjdvWjF0V2p6VFZkRFZMNGk3c2ZiSnp0MTdo?=
 =?utf-8?B?c2ExSW1zT1AyT0xqT2lPdUJ2Mm5ZaGdTaFFZSzlyVXBXSHZCK0ZsbitJNmVH?=
 =?utf-8?B?Z3RQVTZUNzhmc2w3ZWU2WFNVd3F6emZ1MWMveGJlVEdCLzdGVUFjdmdYckJw?=
 =?utf-8?B?RzcweC9tZk9mYVNqUnhMN3Y4M04xYzkybEhyVldZREJyTWJqMGdGWittaUFP?=
 =?utf-8?B?dzd2d00zTDY1TUR4b3R5amNGNmp5c3dLNnZjWFY2QXVpbDJuelFMVGpQL3FY?=
 =?utf-8?B?bDdiT3VxQnZaYzR6Umw5NGI5NU1ZeEMxdFRGeFhXQkc2bUFubU1HcE5lblFx?=
 =?utf-8?B?U2FraThDRnUwYlF0LzRNaDlUK0FjbytNaFJMTDlLSm5mbzY1WVFhOTI2TWw5?=
 =?utf-8?B?Nndnc2NzVUNYYXdPN0NlZ3gyRU5FQWdxSnpBajJhUDFkUlE1S1hDbUl2UUR3?=
 =?utf-8?B?TytiK3ZyQS9hYjdaQW16R3J4NGpIRE50bmpqZmxTMFVOUEVyNEg0elE1MjVa?=
 =?utf-8?B?VHV3YlM1Ulo5a0szaUdncUtFQ3B6RXdBZm9ra1FhanVlSDNncEUvbTUyNThN?=
 =?utf-8?B?WnRMOTdKc3VndXRoZFVZWkNYQndXRnlUdVkxV3FJUGt0NFJJcjVqRDFVcC9k?=
 =?utf-8?B?MW1UUG5Qck84b1dNRWhKSWZob1gyYy9oeGkyTEd1M2QzRVNYMFRQWG5xN3Q4?=
 =?utf-8?B?djlqOWRYTExWZTI4R1lrL21rd3VEWmJvM1dtSDlPUHVzdUxkNWpxWUQvWjBQ?=
 =?utf-8?B?MTZoNGk1Z1JIQmhNL2R4S2ZMcytqcnhkbXdyUFVaQ2RlNUVHRXpQdjRnQVBE?=
 =?utf-8?B?ekFZczF4d2UzQ3Z4UW0zVUpCSUJ0V2YzN0tOZTB4MXp3M2t0R1lYK2NIMlBZ?=
 =?utf-8?B?djh1cWIvMkptbUYzdnRvRUkzYUt2dzdqWlRHU1Z6UVJFbmdZRlpBaHFxYXBh?=
 =?utf-8?B?NEJQTkliKzdLSXhZdWhSK3I5bFRhR0Y5R1dZbXAxNW5ZdlRHTzZJZHM3azVr?=
 =?utf-8?B?bXVkTGZQdno0ZkU5NGNWNmN5a0FwdC8yK2VRUUdnZXlnWFdkL1MwSytIZXZG?=
 =?utf-8?B?WUlQdi9PMnFkeVVlUUt6NW5hcWp5RGxmL0lERkZNSVhtK2hiOTJnUHdTTHBx?=
 =?utf-8?B?Nk1hdWsvRHppcG5rZmhVakpzT29sbm1YbURXTmtqNmsxQmVwNTV5Um9Ubmg4?=
 =?utf-8?B?OUp3ZUN1SlU0VmlrelpjNVNJUG5Xb1hDbmcxWWlLL3RCSVk1bTBEY0Q4MkdN?=
 =?utf-8?Q?FOubP0LbN7dlhz2yddLk2JTUus3iLRw3EQcsrSq?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <AB2867363B03C847BA16045B515F6C7D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f9d50f5-b2d5-4718-35fb-08d9738f4833
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 12:42:27.7443
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: zGsVaIuUo0J6cE2ZyeSVqMi3+gg3/vzutz91rpbnxkqaZFdRYkKHLpevRtQnF1QQm5ZKnxWFISo5rBHOS7LXJ9PZFzsRGjhghBbuAbDpia0dL+IeCNSgu3P3i1SJ2mtg
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7298
X-Proofpoint-ORIG-GUID: 4L_o4yukpH9DqQ7v3yHKED5WGqQydk72
X-Proofpoint-GUID: 4L_o4yukpH9DqQ7v3yHKED5WGqQydk72
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015
 priorityscore=1501 mlxscore=0 mlxlogscore=999 malwarescore=0
 lowpriorityscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090077

DQpPbiAwOS4wOS4yMSAxNDo1MywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTM6NDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTI6
MjEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IEZvciB0aGUgYml0IGluIHF1ZXN0aW9uLCB3aGVy
ZSB0aGUgZ29hbCBhcHBlYXJzIHRvIGJlIHRvIGhhdmUgaGFyZHdhcmUNCj4+PiBob2xkIHRoZSBP
UiBvZiBndWVzdCBhbmQgaG9zdCB2YWx1ZXMsIGFuIGFwcHJvYWNoIHNpbWlsYXIgdG8gdGhhdCB1
c2VkDQo+Pj4gZm9yIHNvbWUgb2YgdGhlIE1TSSAvIE1TSS1YIGJpdHMgbWlnaHQgYmUgY2hvc2Vu
OiBNYWludGFpbiBndWVzdCBhbmQNCj4+PiBob3N0IGJpdHMgaW4gc29mdHdhcmUsIGFuZCB1cGRh
dGUgaGFyZHdhcmUgKGF0IGxlYXN0KSB3aGVuIHRoZQ0KPj4+IGVmZmVjdGl2ZSByZXN1bHRpbmcg
dmFsdWUgY2hhbmdlcy4gQSBjb21wbGljYXRpbmcgZmFjdCBoZXJlIGlzLCB0aG91Z2gsDQo+Pj4g
dGhhdCB1bmxpa2UgZm9yIHRoZSBNU0kgLyBNU0ktWCBiaXRzIGhlcmUgRG9tMCAocGNpYmFjayBv
ciBpdHMgUENJDQo+Pj4gc3VzYnN0ZW0pIG1heSBhbHNvIGhhdmUgYSB2aWV3IG9uIHdoYXQgdGhl
IHNldHRpbmcgb3VnaHQgdG8gYmUuDQo+PiBUaGUgYmlnZ2VyIHF1ZXN0aW9uIGhlcmUgaXMgd2hh
dCBjYW4gd2UgdGFrZSBhcyB0aGUgcmVmZXJlbmNlIGZvciBJTlR4DQo+PiBiaXQsIGUuZy4gaWYg
RG9tMCBkaWRuJ3QgZW5hYmxlL2NvbmZpZ3VyZWQgdGhlIGRldmljZSBiZWluZyBwYXNzZWQgdGhy
b3VnaA0KPj4gdGhhbiBpdHMgQ09NTUFORCByZWdpc3RlciBtYXkgc3RpbGwgYmUgaW4gYWZ0ZXIg
cmVzZXQgc3RhdGUgYW5kIElNTyB0aGVyZSBpcw0KPj4gbm8gZ3VhcmFudGVlIGl0IGhhcyB0aGUg
dmFsdWVzIHdlIGNhbiBzYXkgYXJlICJhcyBob3N0IHdhbnRzIHRoZW0iDQo+IEluIHRoZSBhYnNl
bmNlIG9mIERvbTAgY29udHJvbGxpbmcgdGhlIGRldmljZSwgSSB0aGluayB3ZSBvdWdodCB0byB0
YWtlDQo+IFhlbidzIHZpZXcgYXMgdGhlICJob3N0IiBvbmUuDQpBZ3JlZQ0KPiAgIFdoaWNoIHdp
bGwgd2FudCB0aGUgYml0IHNldCBhdCBsZWFzdCBhcw0KPiBsb25nIGFzIGVpdGhlciBNU0kgb3Ig
TVNJLVggaXMgZW5hYmxlZCBmb3IgdGhlIGRldmljZS4NCkJ1dCB3aGF0IGlzIHRoZSBJTlR4IHJl
bGF0aW9uIHRvIE1TSS9NU0ktWCBoZXJlPw0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0KT2xl
a3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:47:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:47:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183209.331246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJSV-0003ah-1A; Thu, 09 Sep 2021 12:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183209.331246; Thu, 09 Sep 2021 12:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJSU-0003aa-TH; Thu, 09 Sep 2021 12:47:14 +0000
Received: by outflank-mailman (input) for mailman id 183209;
 Thu, 09 Sep 2021 12:47:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOJST-0003aR-CE
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:47:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0d1cf57a-116c-11ec-b1b2-12813bfff9fa;
 Thu, 09 Sep 2021 12:47:12 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2059.outbound.protection.outlook.com [104.47.6.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-AuqBeqxtOOm188AUA2pyyA-2; Thu, 09 Sep 2021 14:47:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 12:47:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 12:47:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0022.eurprd03.prod.outlook.com (2603:10a6:208:14::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 12:47:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d1cf57a-116c-11ec-b1b2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631191631;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z66l6Ki7HWGDOqCU7aG0SHKqlUt01RaZXeqp5qD1xE0=;
	b=m6Tk5fxJvGBxJmc72r9s23/HhSBMRnMu/jOgeW0GQJrd7aOxrUeT67xbBopT+Y0hh6EWLa
	iGHFQt2NYDj6fRkC6Y6if+2q+E60G8BUE5m9X1FcNIAsQoSUZBJ86v9PsmFdakideu64k8
	mZmrfZBwSCE9UEyR3ZoOBdTuMm7BcAU=
X-MC-Unique: AuqBeqxtOOm188AUA2pyyA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xr/odA6fn4gJv8ArHHrLo+T7UhQ7FYG8MODcWShiU7zntA0W+q+UTR6YveRklPYMo+e8DZQ0yK3P3EjMdyu6k93KXCzWfZu28IHZE5Ln710lYgjtgO4tbmEY2vzPWNBilH60pLqqKk7zG3RsUovIGTA45gooLm2N9ZjWLRlPdgEWi8KttYBMgmHmYPGsvOOWKRSK2bC9wZNH9Vsmz+6ciecVN+js6A3OYzrK37yoX0pleY27aX4b34zC1lJzvseNYSch3aS28AWa/3nSIu4BnTvrRzq4oEQQRBOgch8jKbveJ/u8VCTBjy8ZFOqLfsOqSUbbiLY5gIhhXjxDzS8dlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Z66l6Ki7HWGDOqCU7aG0SHKqlUt01RaZXeqp5qD1xE0=;
 b=IetjLL26D+sWXnOmopqO8e4IoF8ZJW+21eKPCRlPStChJk4gFytLLrvIIuyAe+T+Hlk+Bk6mp3JhgpaRtJJ2snN2OlntXFBV07pP04rwNulSnLD/gVXbttpD4u0RfleUKd00ddXrX0ps0DRPEqY0v7LtKoBVwGaJaOG+XqzuF0zGZypx8S8T7YmueGEZnNe8Faj20FwwJyDtBXhkwVDJWl0m76QG4fn+ZrD0FNMbcIml74xSaxai5rtdQlCMtLuD4O/qAkK7P0ag5Ts3gHU88dq4vq8jFlFC4eTHEeuHG9xN53kD+I/901SLxJeYDcJ8V0ZSXn4l2PG8bG1Uou6ouA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
 <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
 <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
 <38669583-3e3a-36e2-8714-a4b6a4e10173@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d2ebd8dc-0dc1-80dc-aa15-45a02b01004f@suse.com>
Date: Thu, 9 Sep 2021 14:47:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <38669583-3e3a-36e2-8714-a4b6a4e10173@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0022.eurprd03.prod.outlook.com
 (2603:10a6:208:14::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9c8f2768-056a-4a74-e841-08d9738fefdd
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686113E46E52025E2670EF2BB3D59@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZNM7Eq2tB/35IHkSu3iM2VZNuEpI+JZyj01tHmfYJHtovTTyLvGdHyXzdj3BtgjOb1NR3IWs6BhDFXC5IEdCJLayKellx/3v3bBu0FIyQp2yY19CawyIIU9pT5b8nXilOEa7DUmYlFixOuxUjkaZMLqBG3qnoiwwZSJOF9WTrXOmK0wJjvsKQjWuhMxluLNTQ5SwJ74vNmKy9sWDTinRVRp+xvHH7Y5thorDR6zqLqIVlJgoe5jAJMoXq0Ob3bTCF/Bz5RrK21EvhEKAov6++agfMleaaYFvNKGfoeEhOKlUHgJ1qTGIy92phsZgcW3p5DWMwgvPvSH4c/dbO6DMzIVV0DmFm6W4UIJrGbY4v13j+nVEsPIQhxof+/wvgvr/NSDxZnnOh+12TkgqbhMnqegFbpexEHPJS844W60J89tMVEP6bfPRWNSrKjH46IiUdvqrWxuFqgAUtO5onYUQx01zPoT19rbG3jMOZXs56hpFgNDEwHI8upWjvCbnSYJUXf+iU8Xud/F1PxMcWNjvRgByCyj3xtMH7hd+Y9oG7db7KmygVnFKODO/S+nZysK/Qpj5fAcySLfHzxYMo3cp4lFJGOAjqRIVQ0Vo/IREv3BdXvB3eDhsEuHkerTt0hA6EcqpejMW/9Zj914MwrqkjPMWOS8h9FoaVFYls/N7+PhelDZpMg+0Nl71aW7ORwyhNdCuslT9J3BQZYh2exOP8PKcPd3X3VZH/Hz066dzfbU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(376002)(346002)(39860400002)(186003)(86362001)(31686004)(26005)(66946007)(83380400001)(66556008)(316002)(31696002)(4326008)(66476007)(7416002)(8676002)(54906003)(956004)(6486002)(16576012)(478600001)(110136005)(53546011)(36756003)(8936002)(2906002)(2616005)(5660300002)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bWNWejd5b2lJdUxmNDdyNVNzSEtHOW5UVEVYYVM1a09NcnZyTHlidThBeFpK?=
 =?utf-8?B?MkxVcjM2b2RZRTlMdFVUd3pZZFZ1YU16N1I1dnB3emx2VENCWlRwaHVmcXJ4?=
 =?utf-8?B?Zks3cE5EZ0x2RWhkSHpTdzZ4aEZxU00vN3ZZUFlWWFJzbGJpMVhkemI2RmNt?=
 =?utf-8?B?NUQydlB4anFKcWtXNFhIMkREU0JHUmNBZmdZNWQxYnNoai96MFRiT3ZraXQy?=
 =?utf-8?B?aFlGRmIxbENCUWJTbnp5N0p3VFlGZElkSVZta0pyVlJaYXlaS2lWWUFkMWhL?=
 =?utf-8?B?WjZoRlFKSUVTdjlxRjFFVHNsdnNpMHByR2FxbkhHajh6UFdMOWRiUE9uL2Uz?=
 =?utf-8?B?M09pOUJUZkhVbVJjaGxDY3pnVzRyTGJSdUNqNEJCbmhOV2diWkRhUlk3TGtF?=
 =?utf-8?B?b2hYT3l2K1dSdjhRTEhKdWRsV1BreE51S0IwM3RuVk91RVNQYmwzUEFDT01P?=
 =?utf-8?B?ZnkwY3VTcmZLTzFmeTZWUGtCUU9zMDFseEUwVWlzS1dwQjdZdEY1OG9RMCtD?=
 =?utf-8?B?Yk85WjFEdWF2QXFIb0FtSGJyYzduMlpmcXNHcVBkbjhhMzNFcXE3N3AveFdD?=
 =?utf-8?B?SFBXcEJ5TFJSMWRDRmtjMWdIeXBxYmdRUWVmRlZWNFRXemJaMk5NVkNiSkJs?=
 =?utf-8?B?dkVLQjRsUlk1eHdnalYwNUtqekdMd3lpQk1vOFZyYmtjaktPcXRrREh0emZn?=
 =?utf-8?B?Z2U1YzNYSy8xdFhpcXN2M3hEeFFvbVJjRktKZFM2RmxjRGZnUHJ5N1N3U29V?=
 =?utf-8?B?dEo3NHk5ay9SNFpqQ1E1UlBkT0U1Zm5UbjFJUUFpNldKTW1XdDdkRzlqVGZE?=
 =?utf-8?B?amM5Z2NiY0laY3dEK0p5VmRzS0dkMDRERS9yRzF6SnZQWFdOYWNnckJJSW5F?=
 =?utf-8?B?UEJFSmZxQ3BTTDFSVlhtNEVPRi9RUmtQcnMrRU55ZmIvNGxwYkxIcEtYUGlE?=
 =?utf-8?B?Qi9YSytQMThNTFJ0akRtS3A5d1FEdWFmc1BUZzZSeEVaTFFrd2JNTk9MWlo0?=
 =?utf-8?B?emdNc2VjRG1CbFZVTVZHcVd4ak5RMHdUeHNaQ0s3aVBkdGVhdVZ5blhvN2Nq?=
 =?utf-8?B?OE9laFlHUStXZnhoVEFrWVc1LzcyUWNmY2Y5ait5aVkxbXFRR05yYzRGVjZR?=
 =?utf-8?B?QUU3eXZjVHRrQzBNRitEb0NSa0hwMTdUbDdidm0rQ0xwUVRYZGJURnhxNmQv?=
 =?utf-8?B?THkvK2hpcVNXU0ptRTJaWDVGNzNhSVVVSE9qNUovS2tpR0dGbHpKcno3bGF5?=
 =?utf-8?B?RkliQWkybGRCZzdHM0pGdkNadlZvRjVZYjREc3M3TnFvRHpCRzU5WWtjbkUv?=
 =?utf-8?B?d0V4NkRPQnJ4OVJVTWRDb2hKZU1GMlFzSXlQNkU3Sm1NOGVaRGJHUzdZZzBk?=
 =?utf-8?B?NHNUaFNLbHNIRTh5NVBrd2sxQThIbmM0WnhCWU5VMHZLT1NVWFhibGFPV1dx?=
 =?utf-8?B?SmN2Yk5PSXFXc3A2NkZXTDBQSGZSbHFrMExMYzdBL0U0d0xtc1hLcUFxZy90?=
 =?utf-8?B?Y0V1L0VNRFB2SklrS2NTcDdIRFUrb3BML01lOE5XUVpjbGJrTTE0U2NLTjZn?=
 =?utf-8?B?NEQvaEZFV05BMnBYUXJaSFU2TzlnNFJzcmtOdy8rRUdySEsweHUreVVSdXFH?=
 =?utf-8?B?cW8zZzh2RFNGM2hxZkVac2V2dC83c3E0NE14d3FFTzVpVlFSYmZXQ2t2bE1z?=
 =?utf-8?B?bmozQlR5NUVtdVlBTGZrbVUyUU1HWkw2ZzdFZHp0eHljd1pvTEw0Mkt1M05x?=
 =?utf-8?Q?4xad63fNd72Ab9bmFtbn/BKAzWBIuqd2Abjp5PJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c8f2768-056a-4a74-e841-08d9738fefdd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 12:47:09.2550
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zWlwK8uQ9HuBMWBlwonu+NOEVBI3EevXcV4KcD5oA1u/MEe7eq9uDGPN/LAVRRL1cOvxOu5D2v7szynz4wZ17w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 09.09.2021 14:42, Oleksandr Andrushchenko wrote:
> On 09.09.21 14:53, Jan Beulich wrote:
>> On 09.09.2021 13:48, Oleksandr Andrushchenko wrote:
>>> On 09.09.21 12:21, Jan Beulich wrote:
>>>> For the bit in question, where the goal appears to be to have hardware
>>>> hold the OR of guest and host values, an approach similar to that used
>>>> for some of the MSI / MSI-X bits might be chosen: Maintain guest and
>>>> host bits in software, and update hardware (at least) when the
>>>> effective resulting value changes. A complicating fact here is, though,
>>>> that unlike for the MSI / MSI-X bits here Dom0 (pciback or its PCI
>>>> susbstem) may also have a view on what the setting ought to be.
>>> The bigger question here is what can we take as the reference for INTx
>>> bit, e.g. if Dom0 didn't enable/configured the device being passed through
>>> than its COMMAND register may still be in after reset state and IMO there is
>>> no guarantee it has the values we can say are "as host wants them"
>> In the absence of Dom0 controlling the device, I think we ought to take
>> Xen's view as the "host" one.
> Agree
>>   Which will want the bit set at least as
>> long as either MSI or MSI-X is enabled for the device.
> But what is the INTx relation to MSI/MSI-X here?

Devices are not supposed to signal interrupts two different ways at a
time. They may enable only one - pin based, MSI, or MSI-X.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:49:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:49:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183216.331257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUH-0004Bs-DV; Thu, 09 Sep 2021 12:49:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183216.331257; Thu, 09 Sep 2021 12:49:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUH-0004Bl-A8; Thu, 09 Sep 2021 12:49:05 +0000
Received: by outflank-mailman (input) for mailman id 183216;
 Thu, 09 Sep 2021 12:49:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOJUG-0004Bd-Dq
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:49:04 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 11701118-3fbd-49fc-a570-733283234214;
 Thu, 09 Sep 2021 12:49:03 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189COC8M023026; 
 Thu, 9 Sep 2021 12:49:00 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3ayj8a03aa-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 12:49:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4578.eurprd03.prod.outlook.com (2603:10a6:208:c9::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 12:48:55 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 12:48:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11701118-3fbd-49fc-a570-733283234214
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Na0VzoZVjnFr5MeRyQql0zRCU2hw8QFHru31P/px3fdmi3PtsSMvl2V84g7kW9Tocgf1mp44Tmj9ltvSgB2XOL4ulnUbmQlD1x/j4sIExbkLlZRddcUze2Yv69v0vnC65qFQNsWC3IjQoqqj932zDmnPJFrbC+JoP/enufJgkEWCVq5z7tyCkh4Hqsft7EzZNzKDaTFlEtvSFYZOEWAQiiAGblD76XaiZ8+8x3hgpjVgR4C8w2NydmhqOeWwXxsrWlyBTTgmN/LaQ2Svr4n5Q9C28Hh1Pnc+hwYeNIvfQ1jtmixYmn0trmJY8iZnb6LE26ljrEgzpNNDaUtivnh9uQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qCCP9O1dEv5PRL6E4RaWiKkWBtZFrCqDDTbgKwZbKIA=;
 b=lA39Qy8uDQDkNa83SIp2E1RIC2TCebvbhA41FBpm2ttDk3hJkfMLgLGUDAfOp/7kzE3BbWlQ4HF0wN9pKUrvkyXWzoG1yFs9kumc4jBcRyNGHflYeye+ydz+AdA30V31HF9oYCL0THpejJvHiuTLYz3/FtDUUroharSCQZH4E052ljAMF1DZQ75sWiqwG7opF6syoKRe89D6qpssAOdf0gi+pXao+KKQV0GPX0mv1H3Iw7npK8upCB6aYDNYscUqShNnGEI4KA0dyivqPm1wn2bfd0PBUOuuxJWA2vG1r6aNZ8DSMmYvqs8GeAg6mUaKMTN7EO8QyYPJUx9hX0EUoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qCCP9O1dEv5PRL6E4RaWiKkWBtZFrCqDDTbgKwZbKIA=;
 b=1n1NLCAWJ8aqqDSUb1ICWblFsggkvg12T3F+iwfiHxqfRrOy2pgcBn7EX/qRY0vxNxZOMb8jKZsuZ4DgOzo0i31ldzUMN82EIA04cGdzeCEZ4iJ9oA41iZPM5cA/NCFQF2yvZFrKDAAlL1aV9lTqBxrOkRIG/O2YMDAUDlPU2ytMKVx1/8dIwMd1/5gbVFjGwtNl+ApC7ozw3B/cSjrNCc+/DMEDvBAxAUISq59O03asNbL9yDHkCn1fNpbYHI6D01tQxG+/guBOmnXSc546PCyRXGbUzsu1Gs0n8z6TnMA53iITqFGI1IdD+Jkk0qI+1vTSEtCwlhh8b3589rjPlg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AIAACKGAgAAo+QCAAAGkAIAADY6AgAABTQCAAACBAA==
Date: Thu, 9 Sep 2021 12:48:54 +0000
Message-ID: <562585c1-d812-7edc-30e5-0c750eb98f40@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
 <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
 <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
 <38669583-3e3a-36e2-8714-a4b6a4e10173@epam.com>
 <d2ebd8dc-0dc1-80dc-aa15-45a02b01004f@suse.com>
In-Reply-To: <d2ebd8dc-0dc1-80dc-aa15-45a02b01004f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6263a93d-cdba-4e3f-74be-08d973902ef3
x-ms-traffictypediagnostic: AM0PR03MB4578:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4578DE1423EF0683E023DA29E7D59@AM0PR03MB4578.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 TOni8ak3OTN+0D8NvIw9m9SUim13vNQgk2inySnNaQgxeLkhBx4UwwU3jsGBWPjZW+vesG7T8EEsKPWiyTu7inuiMoh8Ajdd4wc3CtAKEKChlD14K3GWl7WWIxghJJOz0aUDuPjw9VW94U5UK7r3ABGPKeivhgPe5IKgF+Dnv1blz4e6YF1kLCbk6rn7RQuxdG/vpvUJrFEr47V8j/c2WObWJjnxp8ZNLeirLZJQgS4gt4/IML7fpS1dWatqx6znicU9Qm4tt5sSDOz9iyMXXzej4FgjpH9601aj1N7Iv2m+IC8wW0xLEpYKas1MTbRFOlgrVb0SrZIiRn1suYnBFRlKeIINaJdopFfG3ukmWpDIV4ijeyEkybA1RoigTR5tLpQpcDTbDizNALcFeqlHM/jyiWQAE9plBRToPXYn7lyIxoG1knbZpRB5C8XuwEDGt4I/SiEEeCbqNr0EZVsrBTPb4Oq1ybwKcsp7mftxH/CnE9Z5BpxZSyfNs5Ao2GA4v+0FqtkvVeO7cK1j8NxP+M2iubd3jXfvQhP3bGSTL4vXqrFIJrZUCCguF/Ya5riDTQ6OtFvHtWLS3tcwvkgM989NKx5n0L+SDFLSajv3rHgjE0xjEpgrQIWVYlR4FbZP5m3NKYZVmBgeBblCkP7mtIr8/9PmmdpSgx9SLJmW4tIa6D8Dk4CQa/Mwq5aZyws4pUxi+UOeosjY5FoKfxGbPO4IxlaY1zZ0KCME/25ByYrBDY9uK7wzg4r+zjGGi32u
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(76116006)(6512007)(36756003)(86362001)(83380400001)(2906002)(38100700002)(122000001)(8936002)(71200400001)(316002)(110136005)(54906003)(2616005)(31696002)(31686004)(38070700005)(53546011)(4326008)(66946007)(66556008)(8676002)(6506007)(6486002)(508600001)(66446008)(64756008)(66476007)(5660300002)(186003)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?emhrUFVVUWt2VFQ3ZUNuV1Fmc3N4dlMrVTJSMW1KZ1BkWDhBQ0Rxb2dFTFlw?=
 =?utf-8?B?cWVDVmFWRHVPQVlnQ01WdVV3YU94QnBGN0dMQlJhYUtpUi93b01RUVN0NGFY?=
 =?utf-8?B?UzRORFd4RFRPNjJMV1FpNjM5SlgxSWkxemRnaS9aWG1CWlUrdFM4cFdpeTZa?=
 =?utf-8?B?NmZJN1kwK1NiMmJ0QW1VTWNNZWwwM2FuTThlb2VoODh1dkh4cEluamszRzkw?=
 =?utf-8?B?SXZEVG5IWFdPNU1RS1ovMUc0SFhzSVE1bndGMlFrckJ1Nk45ZTNaYmd1K2Fu?=
 =?utf-8?B?UXZmdmt6Ui9JY0pVSTlMd2dPRzc4SmtISUl1aVpKN2xJWjVMNENNTVl0cC80?=
 =?utf-8?B?M2d0VzFGNmxtWGlMYnFnd1pFc1RybWVDdk1WeUhDa0I1SjBJQ2k5ZVFRVmFQ?=
 =?utf-8?B?c3hydXVSeWpmcXNoYVFwc0U3clk2Rk5wVzZ3R1h5TEt1WXpsWjEvczNrVStv?=
 =?utf-8?B?N2VFMjN5bm0vdHpQK3J0ZFhDbnZvZVFGOHpqcVBoTVIvNzUyc2V0eHM0WG9Y?=
 =?utf-8?B?RmM1YkJWT2RjVS9mb0orSzZndWNKbnV3S0VXYjFmcmVhV3d4ajlyV3ZKcFZ2?=
 =?utf-8?B?andjeVJzSmdaR3Q0eGo1YzNWeXFvZzVYTjVsaUJ5QldKR2ljSVFWSVR0MnQ5?=
 =?utf-8?B?dzZGUHJzNDhTeld5ek9aYnBtd04rTE1ncGJzZFlhZ3BaQ2Y2bnpodVNEV0Q2?=
 =?utf-8?B?RS9XOURwTlRmbHhZVXpEVTN6Z1FHM08xMS9ZdkVoUGlQL1hsWmErZ2tZaUww?=
 =?utf-8?B?QTM5QVFFQXNhdU8zdks2c0RKb0Vmb1lQNU10K3NSNEY3NVMzUkJTTEJwZ25q?=
 =?utf-8?B?aUFEeitKMTBKR1VWMnRZNDFPYWxvbXZaYlBQYVAyZEJ3ZmZxYXJhV0V4OVBC?=
 =?utf-8?B?WWZ0UGF2Q3g2VnhnSDNIZEQ3N0NXR0hVOUtWT0xRTSswUTgrYmRxaEpxeHda?=
 =?utf-8?B?OUFFRWZoYWl4NkZoVTJ6bVBZUXVZWEY1eDhtZUxPeDdoQTBGYk81bU9Ma0VL?=
 =?utf-8?B?Uk5KUTZrVkIxck11eVVZV2krRXUxd0NGY2x2QlRONHpVVU11Skd0NUtEdFRa?=
 =?utf-8?B?TU50eXJvVmt0Z3ZkMUZaWkdPWWtlZ29oU1lqM2VRMVBqZmorK2RuVWNkQ1Zy?=
 =?utf-8?B?YkZMTGFiQkNHekU1SHhmY2g3bisyUlBEVC9mTDVRbjhZT0ZvcDc2V0pJWURS?=
 =?utf-8?B?TDNvR25ZY3BOT2p0YzJBcW10Y0hONThLNFpJeWw5TC95WlJ3dmVWTGdXcFBY?=
 =?utf-8?B?RHMxeFMyanRFZWVKdjhzWFJiazNGOGk4dDEveFRnalVGSzVIa2Z1MkdiSFk3?=
 =?utf-8?B?aFBTNllNaXNQQ3J1Ukt0MEwzQU0rdXNzRFE1UXg5YjhGemFvd2plc3NOeHNj?=
 =?utf-8?B?VXpNR2xYcE1qVlEyc0luNFVkQjNDWjBwWUI1NFVTdzR1ODBvNG14Z3BWTmVH?=
 =?utf-8?B?Vk5IVmd3TkJpLzhSbWlRbG9kY0tNUkJwNjBKUUZYdFNQblpkUU1RUU1zb2Yr?=
 =?utf-8?B?QXhIdzBCTjBPYTlJSVJJdThBaXF4SlhJQmR2UkJvTG5zMU0zc3czT1ZuS0s5?=
 =?utf-8?B?S1p3bEVWYmVpcXNzU1F4dFpyTStGR0szN3B0am1KalRpeUNYdkR5UkxybTNM?=
 =?utf-8?B?YmlyTXRqSUVrRWtla29QclNKY0NBbDdNcVl1Tng2RTFoQWFZYlNGdDF2cklO?=
 =?utf-8?B?TFlUYjNSWXRPTmk4NU5RcTQrWjZnb0dsWFhqcTdGN1JleTlyMW55aENKYmdJ?=
 =?utf-8?Q?kS3Pm2MSNW5I5v6617G3IFUbg5KSH7bbfaZAHsw?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <78F7D9D31597144C88194DB554DA64E6@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6263a93d-cdba-4e3f-74be-08d973902ef3
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 12:48:54.8762
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qRssFzKtwImRJijVuP+Svo2VXKzUeCnmgQjLR5DuxJjBsLSSSbyngLOGNWeL9GuyaKo80GvdjGyuIz01Y8P/pvKoRKS7hmi2HahOe4THkzT9IMgnzSjUG7a9I3Pu9FWs
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4578
X-Proofpoint-GUID: WWQJHBLZ3VSzD1fvVplRu94znC9myg-f
X-Proofpoint-ORIG-GUID: WWQJHBLZ3VSzD1fvVplRu94znC9myg-f
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 spamscore=0 malwarescore=0 adultscore=0 suspectscore=0 clxscore=1015
 priorityscore=1501 mlxlogscore=999 impostorscore=0 bulkscore=0
 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090078

DQpPbiAwOS4wOS4yMSAxNTo0NywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTQ6NDIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTQ6
NTMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMTM6NDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwOS4wOS4yMSAxMjoyMSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBGb3IgdGhlIGJpdCBpbiBxdWVzdGlvbiwgd2hlcmUgdGhlIGdvYWwg
YXBwZWFycyB0byBiZSB0byBoYXZlIGhhcmR3YXJlDQo+Pj4+PiBob2xkIHRoZSBPUiBvZiBndWVz
dCBhbmQgaG9zdCB2YWx1ZXMsIGFuIGFwcHJvYWNoIHNpbWlsYXIgdG8gdGhhdCB1c2VkDQo+Pj4+
PiBmb3Igc29tZSBvZiB0aGUgTVNJIC8gTVNJLVggYml0cyBtaWdodCBiZSBjaG9zZW46IE1haW50
YWluIGd1ZXN0IGFuZA0KPj4+Pj4gaG9zdCBiaXRzIGluIHNvZnR3YXJlLCBhbmQgdXBkYXRlIGhh
cmR3YXJlIChhdCBsZWFzdCkgd2hlbiB0aGUNCj4+Pj4+IGVmZmVjdGl2ZSByZXN1bHRpbmcgdmFs
dWUgY2hhbmdlcy4gQSBjb21wbGljYXRpbmcgZmFjdCBoZXJlIGlzLCB0aG91Z2gsDQo+Pj4+PiB0
aGF0IHVubGlrZSBmb3IgdGhlIE1TSSAvIE1TSS1YIGJpdHMgaGVyZSBEb20wIChwY2liYWNrIG9y
IGl0cyBQQ0kNCj4+Pj4+IHN1c2JzdGVtKSBtYXkgYWxzbyBoYXZlIGEgdmlldyBvbiB3aGF0IHRo
ZSBzZXR0aW5nIG91Z2h0IHRvIGJlLg0KPj4+PiBUaGUgYmlnZ2VyIHF1ZXN0aW9uIGhlcmUgaXMg
d2hhdCBjYW4gd2UgdGFrZSBhcyB0aGUgcmVmZXJlbmNlIGZvciBJTlR4DQo+Pj4+IGJpdCwgZS5n
LiBpZiBEb20wIGRpZG4ndCBlbmFibGUvY29uZmlndXJlZCB0aGUgZGV2aWNlIGJlaW5nIHBhc3Nl
ZCB0aHJvdWdoDQo+Pj4+IHRoYW4gaXRzIENPTU1BTkQgcmVnaXN0ZXIgbWF5IHN0aWxsIGJlIGlu
IGFmdGVyIHJlc2V0IHN0YXRlIGFuZCBJTU8gdGhlcmUgaXMNCj4+Pj4gbm8gZ3VhcmFudGVlIGl0
IGhhcyB0aGUgdmFsdWVzIHdlIGNhbiBzYXkgYXJlICJhcyBob3N0IHdhbnRzIHRoZW0iDQo+Pj4g
SW4gdGhlIGFic2VuY2Ugb2YgRG9tMCBjb250cm9sbGluZyB0aGUgZGV2aWNlLCBJIHRoaW5rIHdl
IG91Z2h0IHRvIHRha2UNCj4+PiBYZW4ncyB2aWV3IGFzIHRoZSAiaG9zdCIgb25lLg0KPj4gQWdy
ZWUNCj4+PiAgICBXaGljaCB3aWxsIHdhbnQgdGhlIGJpdCBzZXQgYXQgbGVhc3QgYXMNCj4+PiBs
b25nIGFzIGVpdGhlciBNU0kgb3IgTVNJLVggaXMgZW5hYmxlZCBmb3IgdGhlIGRldmljZS4NCj4+
IEJ1dCB3aGF0IGlzIHRoZSBJTlR4IHJlbGF0aW9uIHRvIE1TSS9NU0ktWCBoZXJlPw0KPiBEZXZp
Y2VzIGFyZSBub3Qgc3VwcG9zZWQgdG8gc2lnbmFsIGludGVycnVwdHMgdHdvIGRpZmZlcmVudCB3
YXlzIGF0IGENCj4gdGltZS4gVGhleSBtYXkgZW5hYmxlIG9ubHkgb25lIC0gcGluIGJhc2VkLCBN
U0ksIG9yIE1TSS1YLg0KDQpBaCwgdGhhdCBzaW1wbGUgOykgWWVzLCBvZiBjb3Vyc2UNCg0KPg0K
PiBKYW4NCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:49:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:49:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183220.331268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUg-0004kR-RJ; Thu, 09 Sep 2021 12:49:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183220.331268; Thu, 09 Sep 2021 12:49:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUg-0004kK-Nb; Thu, 09 Sep 2021 12:49:30 +0000
Received: by outflank-mailman (input) for mailman id 183220;
 Thu, 09 Sep 2021 12:49:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOJUf-0004iJ-5N
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:49:29 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 600b41ca-0bd0-4bd4-b3f9-5f9e83374f20;
 Thu, 09 Sep 2021 12:49:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7D5EE222FA;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 242D513CC1;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id QB9TB9cCOmEyAgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 12:49:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 600b41ca-0bd0-4bd4-b3f9-5f9e83374f20
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631191767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=apPVVY/Or6068NSgAcAK1NyxZlsZ0S/Udfzjsh8c7Lc=;
	b=E1vtToLOsUatlm7chF20KBa/molARloaccxU0c3pKL4bDfD7/pgwUnIA6P4vc8qYJDbiRK
	kNY5hXJyJZLDjEvJZip43BDoF2f+8gbrdmjgl5jC23MCp133QAIJdXIi6bRxU2zuiTGebd
	WNsnqvVMjYS58fqH0YUQxEKT7qwJIzE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
Date: Thu,  9 Sep 2021 14:49:22 +0200
Message-Id: <20210909124924.1698-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210909124924.1698-1-jgross@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Today the build will fail if --disable-pv-grub as a parameter of
configure, as the main Makefile will unconditionally try to build a
32-bit pv-grub stubdom.

Fix that by introducing a pv-grub32 target in stubdom/Makefile taking
care of this situation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 Makefile         |  4 ++--
 stubdom/Makefile | 13 +++++++++++++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 96d32cfd50..5b5cef3e49 100644
--- a/Makefile
+++ b/Makefile
@@ -72,7 +72,7 @@ build-tools-oxenstored: build-tools-public-headers
 build-stubdom: mini-os-dir build-tools-public-headers
 	$(MAKE) -C stubdom build
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub32
 endif
 
 .PHONY: build-docs
@@ -143,7 +143,7 @@ install-tools: install-tools-public-headers
 install-stubdom: mini-os-dir install-tools
 	$(MAKE) -C stubdom install
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub32
 endif
 
 .PHONY: tools/firmware/seabios-dir-force-update
diff --git a/stubdom/Makefile b/stubdom/Makefile
index 06aa69d8bc..b339ae701c 100644
--- a/stubdom/Makefile
+++ b/stubdom/Makefile
@@ -531,6 +531,13 @@ vtpmmgr-stubdom: mini-os-$(XEN_TARGET_ARCH)-vtpmmgr vtpmmgr
 pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxenguest grub
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
 
+.PHONY: pv-grub32
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+pv-grub32: pv-grub
+else
+pv-grub32:
+endif
+
 .PHONY: xenstore-stubdom
 xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxenguest xenstore
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
@@ -560,6 +567,12 @@ install-grub: pv-grub
 	$(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)"
 	$(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz"
 
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+install-grub32: install-grub
+else
+install-grub32:
+endif
+
 install-c: c-stubdom
 
 install-caml: caml-stubdom
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:49:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:49:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183221.331279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUl-00055Y-56; Thu, 09 Sep 2021 12:49:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183221.331279; Thu, 09 Sep 2021 12:49:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUl-00055J-0w; Thu, 09 Sep 2021 12:49:35 +0000
Received: by outflank-mailman (input) for mailman id 183221;
 Thu, 09 Sep 2021 12:49:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOJUj-0004eL-DT
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:49:33 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5e0b0f30-116c-11ec-b1b2-12813bfff9fa;
 Thu, 09 Sep 2021 12:49:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1DCAB222ED;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AF60013C53;
 Thu,  9 Sep 2021 12:49:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id zvm/J9YCOmEyAgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 12:49:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e0b0f30-116c-11ec-b1b2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631191767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Toydu0D3i5KxQN6Ya4n3cSCDGf0+3fEILAITE1BDLrs=;
	b=ijCsBqW1VXyQN96muFlMPS3yHJjQsSe+cfU1movOzZjVvgyH2rUgrcS/QAXag1ltTECEh9
	KtSOHW+2kxtZYEsZbyStJRAwYe0j7ahHYj3oCkk5VeVwSkWOc5R89bHsslsTZx6h0aAagf
	5kCWaaEvUYdF1bf4kioYMivTZ4s4CHM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Community Manager <community.manager@xenproject.org>
Subject: [PATCH v2 0/3] disable building of pv-grub and qemu-trad per default
Date: Thu,  9 Sep 2021 14:49:21 +0200
Message-Id: <20210909124924.1698-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is a first step of deprecating pv-grub and qemu-trad including
ioemu-stubdom. Switch the default to not building it.

Changes in V2:
- new patch 3
- added CHANGELOG.md entry in patch 2

Juergen Gross (3):
  stubdom: fix build with disabled pv-grub
  stubdom: disable building pv-grub
  tools: disable building qemu-trad per default

 CHANGELOG.md         |  5 +++++
 Makefile             |  4 ++--
 stubdom/Makefile     | 13 +++++++++++++
 stubdom/configure    | 16 ++--------------
 stubdom/configure.ac | 10 ++--------
 tools/configure      | 17 ++---------------
 tools/configure.ac   | 13 +------------
 7 files changed, 27 insertions(+), 51 deletions(-)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:49:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:49:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183222.331284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUl-00059x-J5; Thu, 09 Sep 2021 12:49:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183222.331284; Thu, 09 Sep 2021 12:49:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUl-000591-BH; Thu, 09 Sep 2021 12:49:35 +0000
Received: by outflank-mailman (input) for mailman id 183222;
 Thu, 09 Sep 2021 12:49:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOJUk-0004iJ-4C
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:49:34 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bf1dbd9c-3e4d-415e-b987-b6d84d2fc777;
 Thu, 09 Sep 2021 12:49:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C0AA320041;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 851A513C53;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 4GAOH9cCOmEyAgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 12:49:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf1dbd9c-3e4d-415e-b987-b6d84d2fc777
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631191767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VDnmS7PEHavwjTjDgrskPsWB2O0plcriin7YJuNthnc=;
	b=fDk4sR4I11DrMQkGj9VBRnyVivX+WpBqLZ2Wt8VK2Q8AMahd+4X8MeZ8cHUD29gqP8k4yt
	Aco/LHI7UwU4qX3Ij87PACYNREoJOT1KSZWA6obPmLcUexrpLM6ZiYje4bwuywen7eJ07k
	isdjQUvxfsYEavqhg6UUH8b+OzeVe3c=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 2/3] stubdom: disable building pv-grub
Date: Thu,  9 Sep 2021 14:49:23 +0200
Message-Id: <20210909124924.1698-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210909124924.1698-1-jgross@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The stubdom based pv-grub is using a very outdated version of grub
(0.97) and should not be used any longer. Mainline grub has support for
PV guests for a long time now, so that should be used as a boot loader
of a PV domain.

So disable building pv-grub per default. In case someone really wants
to continue using it he/she can still use a pv-grub binary from an older
Xen version or manually enable building it via:

  configure --enable-pv-grub

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V2:
- add CHANGELOG.md entry (Jan Beulich)
---
 CHANGELOG.md         | 2 ++
 stubdom/configure    | 8 ++------
 stubdom/configure.ac | 2 +-
 3 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 22cfdb4298..e7107ac3de 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    appearing in 4.12.2 and 4.11.4). Prior (4.13...4.15-like) behavior can be arranged for
    either by enabling the IOMMU_QUARANTINE_SCRATCH_PAGE setting at build (configuration) time
    or by passing "iommu=quarantine=scratch-page" on the hypervisor command line.
+ - pv-grub stubdoms will no longer be built per default. In order to be able to use pv-grub
+   configure needs to be called with "--enable-pv-grub" as parameter.
 
 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.15.0) - TBD
 
diff --git a/stubdom/configure b/stubdom/configure
index aa48df986d..df31532abb 100755
--- a/stubdom/configure
+++ b/stubdom/configure
@@ -1342,7 +1342,7 @@ Optional Features:
   --enable-ioemu-stubdom  Build and install ioemu-stubdom
   --enable-c-stubdom      Build and install c-stubdom (default is DISABLED)
   --enable-caml-stubdom   Build and install caml-stubdom (default is DISABLED)
-  --disable-pv-grub       Build and install pv-grub (default is ENABLED)
+  --enable-pv-grub        Build and install pv-grub (default is DISABLED)
   --disable-xenstore-stubdom
                           Build and install xenstore-stubdom (default is
                           ENABLED)
@@ -2129,11 +2129,7 @@ fi
 else
 
 
-grub=y
-STUBDOM_TARGETS="$STUBDOM_TARGETS grub"
-STUBDOM_BUILD="$STUBDOM_BUILD pv-grub"
-STUBDOM_INSTALL="$STUBDOM_INSTALL install-grub"
-STUBDOM_UNINSTALL="$STUBDOM_UNINSTALL install-grub"
+grub=n
 
 
 fi
diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index bd6f765929..a07a1edae5 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -21,7 +21,7 @@ m4_include([../m4/fetcher.m4])
 AX_STUBDOM_CONDITIONAL([ioemu-stubdom], [ioemu])
 AX_STUBDOM_DEFAULT_DISABLE([c-stubdom], [c])
 AX_STUBDOM_DEFAULT_DISABLE([caml-stubdom], [caml])
-AX_STUBDOM_DEFAULT_ENABLE([pv-grub], [grub])
+AX_STUBDOM_DEFAULT_DISABLE([pv-grub], [grub])
 AX_STUBDOM_DEFAULT_ENABLE([xenstore-stubdom], [xenstore])
 AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
 AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 12:49:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 12:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183226.331301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUq-0005ld-OL; Thu, 09 Sep 2021 12:49:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183226.331301; Thu, 09 Sep 2021 12:49:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJUq-0005lK-Kc; Thu, 09 Sep 2021 12:49:40 +0000
Received: by outflank-mailman (input) for mailman id 183226;
 Thu, 09 Sep 2021 12:49:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOJUp-0004iJ-4X
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 12:49:39 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fe8db3fc-2966-4895-adef-34f3a31ca3dc;
 Thu, 09 Sep 2021 12:49:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 05F7E22300;
 Thu,  9 Sep 2021 12:49:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C518E13CC1;
 Thu,  9 Sep 2021 12:49:27 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id ANOoLtcCOmEyAgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 12:49:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe8db3fc-2966-4895-adef-34f3a31ca3dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631191768; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KD1KyoxklKM5WAWgMLa0ZRJlvuuDos+1D1Wk4QJhI1s=;
	b=pjAHshmH4I+RV8YoVguibbSPyGf4Khskbk9ziNGNUWHnlj0pZzEMgTLNsq4D5riR9hMeLN
	5cDW8wdiBzzJNOYqLkA4NSamEV17siLZw8CVw/m5EvN0GiCNVbbvR5heI837ugyrVm3LP3
	FKvjoOjVQfCPtAj7CYXQ14p105NVPuY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 3/3] tools: disable building qemu-trad per default
Date: Thu,  9 Sep 2021 14:49:24 +0200
Message-Id: <20210909124924.1698-4-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210909124924.1698-1-jgross@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Using qemu-traditional as device model is deprecated for some time now.

So change the default for building it to "disable". This will affect
ioemu-stubdom, too, as there is a direct dependency between the two.

Today it is possible to use a PVH/HVM Linux-based stubdom as device
model. Additionally using ioemu-stubdom isn't really helping for
security, as it requires to run a very old and potentially buggy qemu
version in a PV domain. This is adding probably more security problems
than it is removing by using a stubdom.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- new patch
---
 CHANGELOG.md         |  3 +++
 stubdom/configure    |  8 --------
 stubdom/configure.ac |  8 +-------
 tools/configure      | 17 ++---------------
 tools/configure.ac   | 13 +------------
 5 files changed, 7 insertions(+), 42 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index e7107ac3de..e5ab49e779 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,6 +18,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    or by passing "iommu=quarantine=scratch-page" on the hypervisor command line.
  - pv-grub stubdoms will no longer be built per default. In order to be able to use pv-grub
    configure needs to be called with "--enable-pv-grub" as parameter.
+ - qemu-traditional based device models (both, qemu-traditional and ioemu-stubdom) will
+   no longer be built per default. In order to be able to use those, configure needs to
+   be called with "--enable-qemu-traditional" as parameter.
 
 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.15.0) - TBD
 
diff --git a/stubdom/configure b/stubdom/configure
index df31532abb..07b709f998 100755
--- a/stubdom/configure
+++ b/stubdom/configure
@@ -2286,14 +2286,6 @@ fi
 # Check whether --enable-qemu-traditional was given.
 if test "${enable_qemu_traditional+set}" = set; then :
   enableval=$enable_qemu_traditional;
-else
-
-    case "$host_cpu" in
-        i[3456]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-
 fi
 
 if test "x$enable_qemu_traditional" = "xyes"; then :
diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index a07a1edae5..e20d99edac 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -27,13 +27,7 @@ AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
 AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
 AX_STUBDOM_CONDITIONAL([vtpmmgr-stubdom], [vtpmmgr])
 
-AC_ARG_ENABLE([qemu-traditional],,,[
-    case "$host_cpu" in
-        i[[3456]]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-])
+AC_ARG_ENABLE([qemu-traditional])
 AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
     qemu_traditional=y],[
     qemu_traditional=n
diff --git a/tools/configure b/tools/configure
index 33814b24b3..8bf8fe75b8 100755
--- a/tools/configure
+++ b/tools/configure
@@ -1502,8 +1502,8 @@ Optional Features:
   --disable-seabios       Disable SeaBIOS (default is ENABLED)
   --disable-golang        Disable Go tools (default is ENABLED)
   --enable-qemu-traditional
-                          Enable qemu traditional device model, (DEFAULT is on
-                          for Linux or NetBSD x86, otherwise off)
+                          Enable qemu traditional device model, (DEFAULT is
+                          off)
   --enable-rombios        Enable ROMBIOS, (DEFAULT is on if qemu-traditional
                           is enabled, otherwise off)
   --disable-ipxe          Enable in-tree IPXE, (DEFAULT is on if rombios is
@@ -4287,19 +4287,6 @@ LINUX_BACKEND_MODULES="`eval echo $LINUX_BACKEND_MODULES`"
 # Check whether --enable-qemu-traditional was given.
 if test "${enable_qemu_traditional+set}" = set; then :
   enableval=$enable_qemu_traditional;
-else
-
-    case "$host_cpu" in
-        i[3456]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-    case "$host_os" in
-        freebsd*)
-           enable_qemu_traditional="no";;
-    esac
-
-
 fi
 
 if test "x$enable_qemu_traditional" = "xyes"; then :
diff --git a/tools/configure.ac b/tools/configure.ac
index 6414fcbb44..a713fd34d6 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -120,18 +120,7 @@ AC_SUBST(LINUX_BACKEND_MODULES)
 
 AC_ARG_ENABLE([qemu-traditional],
     AS_HELP_STRING([--enable-qemu-traditional],
-                   [Enable qemu traditional device model, (DEFAULT is on for Linux or NetBSD x86, otherwise off)]),,[
-    case "$host_cpu" in
-        i[[3456]]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-    case "$host_os" in
-        freebsd*)
-           enable_qemu_traditional="no";;
-    esac
-
-])
+                   [Enable qemu traditional device model, (DEFAULT is off)]))
 AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
 AC_DEFINE([HAVE_QEMU_TRADITIONAL], [1], [Qemu traditional enabled])
     qemu_traditional=y],[
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:16:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:16:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183249.331311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJut-0002Ex-QU; Thu, 09 Sep 2021 13:16:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183249.331311; Thu, 09 Sep 2021 13:16:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJut-0002Eq-N1; Thu, 09 Sep 2021 13:16:35 +0000
Received: by outflank-mailman (input) for mailman id 183249;
 Thu, 09 Sep 2021 13:16:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOJus-0002Ek-3d
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:16:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOJur-0006T6-1P; Thu, 09 Sep 2021 13:16:33 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOJuq-0000nU-RC; Thu, 09 Sep 2021 13:16:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=6lp/cFCmBymmNHRqD/8gkxMPLb7mzL2pKISVqLjtZFM=; b=g7o5FyQUTsYzffKu6BnUA7ZDOW
	7OBlj1KmB6yRTFUT20rwJG/PCgSSotb3V5mhgcIxD89OB9eT8Ch+59NzQ5pmMkd8H7JR8Ny3PNSZ9
	427EJPjdwOp5+6g/9eH/JeDULAJ4+HkMG9F/XFJh7bNZQcqw140tSLLm8QMNzCpEXtCY=;
Subject: Re: [PATCH v1 02/14] xen/pci: solve compilation error on ARM with
 HAS_PCI enabled
To: Rahul Singh <rahul.singh@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <c6009c327eaed66206fa093732ab6672530acd27.1629366665.git.rahul.singh@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <5b3fb4aa-ec56-bbda-dee9-899fa466aacd@xen.org>
Date: Thu, 9 Sep 2021 14:16:31 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <c6009c327eaed66206fa093732ab6672530acd27.1629366665.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

On 19/08/2021 13:02, Rahul Singh wrote:
> Compilation error is observed when HAS_PCI is enabled for ARM
> architecture.
> 
> Add definition for arch_iommu_use_permitted() and
> arch_pci_clean_pirqs().Implement dummy functions for pci_conf_read*() to
> fix compilation error.
> 
> pci.c: In function ‘deassign_device’:
> pci.c:849:49: error: implicit declaration of function ‘pci_to_dev’;
> did you mean ‘dt_to_dev’? [-Werror=implicit-function-declaration]
>              pci_to_dev(pdev));
> 
> pci.c:18: undefined reference to `pci_conf_read16’
> pci.c:880: undefined reference to `arch_pci_clean_pirqs’
> pci.c:1392: undefined reference to `arch_iommu_use_permitted'
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>   xen/arch/arm/Makefile               |  1 +
>   xen/arch/arm/pci/Makefile           |  2 +
>   xen/arch/arm/pci/pci-access.c       | 61 +++++++++++++++++++++++++++++
>   xen/arch/arm/pci/pci.c              | 32 +++++++++++++++
>   xen/drivers/passthrough/arm/iommu.c |  5 +++
>   xen/include/asm-arm/pci.h           | 33 ++++++++++++++--
>   6 files changed, 131 insertions(+), 3 deletions(-)
>   create mode 100644 xen/arch/arm/pci/Makefile
>   create mode 100644 xen/arch/arm/pci/pci-access.c
>   create mode 100644 xen/arch/arm/pci/pci.c
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b4..0e14a5e5c8 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -6,6 +6,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
>   obj-y += platforms/
>   endif
>   obj-$(CONFIG_TEE) += tee/
> +obj-$(CONFIG_HAS_PCI) += pci/

This should go before platforms/ to keep the order alphabetically.


Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:17:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:17:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183255.331323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJvW-0002mi-4Y; Thu, 09 Sep 2021 13:17:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183255.331323; Thu, 09 Sep 2021 13:17:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOJvW-0002mb-0f; Thu, 09 Sep 2021 13:17:14 +0000
Received: by outflank-mailman (input) for mailman id 183255;
 Thu, 09 Sep 2021 13:17:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6so6=N7=epam.com=prvs=9886dc581d=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOJvU-0002mM-Ke
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:17:12 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3d94dc78-1170-11ec-b1b4-12813bfff9fa;
 Thu, 09 Sep 2021 13:17:11 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189CwTca010184; 
 Thu, 9 Sep 2021 13:17:08 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2059.outbound.protection.outlook.com [104.47.0.59])
 by mx0b-0039f301.pphosted.com with ESMTP id 3ayjph855y-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 13:17:07 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4884.eurprd03.prod.outlook.com (2603:10a6:208:fe::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 13:17:04 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 13:17:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d94dc78-1170-11ec-b1b4-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iicYcw7F1T4KQqYrXxWe1LIP9eWg9IUCdVgo9bZ6vzAZ2IdPHgJ/gSxEDgvBmlDFHR1t8Gn/e/kwTCBnosB4squxgDLX1gvsLRHdvdx9mGOhoZMVvDDGnjR1xsLe3mxEs4h3TEwbLlWdgyMijJ+PomOtif8V4AFZLu6e42ZXpVMgmIirSmLsqwAXBYjR+wsvGmsRAnN0kK9e0+mCP7kw/gzJyey/YJOx55wC9Urh/2KETdbN+E+aYA2nY1fhZjBiZ85yQF22orH7LMuY2JDqyJoxCmUDOLwn5HuA7yikazCz2HL+zt6mC30uCCtchi3/Jp2AYDWr+zSRhS1RA1mavg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zbY9fevbrbh0e0i2EJEENxq9Ol5M7fZa4MJBTVyCm9U=;
 b=GmuCMFza04Re25bHBD22F5j5LgEGiIfoEDkdCQWIFiKlm4g01k9sJRcqSs9v851vajqIo2M/3vh7AMberKDwq5fO/LgmY1Azo21EqeuKrhwUkJwqYLe47hdIHUc7W5SMiE1M7XFvaMikKwLKcTaanUc9HcaMTC6JOyCg4MhWK9kc7pLwG/IIeU1CTQ0T2HTQHrupgzDXHWCaBK+DyJAIoW2fri+EMbMmO3pV0oURPHxDuCT6suwAmNZd2eTJrHGGMLMXpnti/ECIF6ewoXbI4zqgEYXqHPGXEs6bdCvn3Hp6LA4wGPgApyCbHZDcKZFHehBuUvDwyU6pQsDsXPOSvA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zbY9fevbrbh0e0i2EJEENxq9Ol5M7fZa4MJBTVyCm9U=;
 b=RKj7XFKl4bfyAegIpMkDTmrl5RQ4OPbX25PoPRCzX4X1uG/Map3Hn3Hr+NISKCwTuCE79Op0sAzCAjn3jV4WuC5DH+H3uGKLCZxlSK6KUch3KcKKmZeFF5BW9fIohmccazu7cqxf/5zFcdCba1bTj/37omlwf2tyb4DPJRyuTQ0O9VgeEJWcadoE6ZVUJhPdvPUvIB4CSi+hVTmuPHy74FHchEAL+8JYUUVyK5hG6MXU6Zh3zh1G5i96nVSgfq/2rFvrGo9fPdQXeBpTNKJCiG4NpEckiQes+V/2QrQUdy0uVXzkFAVFzM5llPPxvVM7k2JJeYA++K6btEZJT3TTBQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Topic: [PATCH 8/9] vpci/header: Reset the command register when adding
 devices
Thread-Index: 
 AQHXoKxjFI8WSqpeJkWkpzZfft/zi6uXHR6AgAEZm4CAAASsgIAABSgAgAAIrgCAAAT5AIAAA1EAgAAJNYCAAAPZgIADDJuAgAAA+ICAAAH/AIAACKGAgAAo+QCAAAGkAIAADY6AgAABTQCAAAhegA==
Date: Thu, 9 Sep 2021 13:17:04 +0000
Message-ID: <47bc8652-54af-f6f5-42a9-88adbbf0eb57@epam.com>
References: <20210903100831.177748-1-andr2000@gmail.com>
 <20210903100831.177748-9-andr2000@gmail.com>
 <ffddb533-399d-3b34-69f2-33ba1e977231@suse.com>
 <5366265c-d169-041d-f812-1c49260f6499@epam.com>
 <8f1816db-798b-15d9-7343-2199eb8f39e0@suse.com>
 <0e3942a5-9105-c99e-f15e-dcf35aae142a@epam.com>
 <c6702cee-9c37-8f0f-77d7-20da718e3e94@suse.com>
 <5d0d345d-db16-f0c5-9a78-4ad5f4733886@epam.com>
 <5ffbd0a6-b34f-4de4-b316-2376211039f1@suse.com>
 <6db7c55c-93c6-7901-6097-687287463c78@epam.com>
 <d78f2211-ff79-6bbc-f8ca-95cf07e3d7ae@suse.com>
 <73697ef0-85de-a530-7601-d8b1490ec5b5@epam.com>
 <f4758911-0927-f31a-a617-9d8566edd2b1@suse.com>
 <6d8a4bae-cfed-07fb-d6e0-7587eb85069c@epam.com>
 <10bccd76-bbbb-1cb5-b9ad-4298014befd4@suse.com>
 <3ca10b3a-35c9-00b4-f736-293589d641d4@epam.com>
 <06560d04-6d71-bb94-20af-fc43cf686b0a@suse.com>
 <38669583-3e3a-36e2-8714-a4b6a4e10173@epam.com>
 <d2ebd8dc-0dc1-80dc-aa15-45a02b01004f@suse.com>
In-Reply-To: <d2ebd8dc-0dc1-80dc-aa15-45a02b01004f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7ee3861b-a540-45b1-6352-08d973941e0e
x-ms-traffictypediagnostic: AM0PR03MB4884:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB488453E1907DF738068A29F4E7D59@AM0PR03MB4884.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 TqP9ikD0nEyeZq/ZZE8RIIg2rp9fPMA/vQoRdozTXev2TZ3KDXRiUzIOaJKbEUuSXZ9JelZlmLkI/Z5rEhuDr9dT1lCWhJig6f1gFjN68APbz7RLO/8pgw9Ou7EInZbnIUXoa1PjcdHbQfu8rY7QVwbCO6bqIGHG34Oz9gZpdqHU17PJ0ABcDJ6ARwvczJpA+UtGfbRT4ZhFJg2SwthmzZrK6RWnKBwLRznRitz98jInz8vHvE1YMmH5MEdWdTWifVVtCOxvQPNYLHlhZJcBGvHElyLpnEVi8/+5aAks3F9TgkjF28+cqMdB7ot3we3Cteg1bPcs7FpZ0pLXv9N63QMt1sJbhbu1+Rl6dfqJM/+fsLQw5a7CwaPWT/QGDxdYJF35GZV1F/cunuBUtOEpX8gHZcZ2fSGWHrAzR7yovjLeOetJF8hFIpoj7cTi+qkVdTrZEoTVejXtDPixBHyMREm/Ml1JGlAW164zHfQ8qkJZMFPN/IH4Sf/HCmqUEgGB2lTdG7aWjJ4Yjix9ivzjqkKICUoi4uOOWH4980wigC0DE+rbuJtMj2keswF1SpAtimN8ZAplrN9P/0KNiu5wyiw6XIleeFqhHIYdLYhR0ZobTDXEpkJOZlSrcDCPG5ABaEntBo91BrDuGPngzVSrooZaMQbNFt5zCS/kKC2TBMzaemImCM7QR5bbqYCTqbXiA1xPA1Jua5BWF6xrJAf8rKGK9ZZ2oz8r7z3oQnTF0Ha6xkVNsNoX1CMpz5TvLFJN
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(366004)(376002)(346002)(396003)(122000001)(38100700002)(71200400001)(86362001)(186003)(6512007)(478600001)(6486002)(2616005)(31686004)(110136005)(5660300002)(66556008)(66476007)(66446008)(2906002)(91956017)(26005)(76116006)(54906003)(64756008)(83380400001)(66946007)(31696002)(38070700005)(53546011)(6506007)(4326008)(316002)(36756003)(8676002)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?cmRoa0gvVXcySjdhSlVOd2hyT2VFYmluNlVDVXhQNkkxMk5ham14M0I0T1pC?=
 =?utf-8?B?bzlrNS9rYXNMYjgvN3lhTzhBSHlvWkFxM05WWmdGRHNCcVdxUkNlMEhpT3c0?=
 =?utf-8?B?UUw4RlU5VDJxRWh3VmoyL1VRN2VtaFFyWXY3bkFFdkc5L0FEYXk2cERlV3JH?=
 =?utf-8?B?RGg5NFRNVndwbzk0VjFZM1Y0cUR1bEoxWlNFWDhMSjJkeTUrVTZPRVRoMmNH?=
 =?utf-8?B?ZkRmay9sbkU5cmt4YlNGaytOK2hiQW15TnNDRkdWNWRhcUNuS1R5cUxPRVZF?=
 =?utf-8?B?bGRrbzdxODFFMWVLNlVDMkhncXlPY015Y25ZSkVSNmlNMXRTdzhYdVhzT25S?=
 =?utf-8?B?UFd5bDRqUmtYdzJYRWxUbUVoczREV2hPNjlhMG9LRzZDRHAza0VCeUUvMmNK?=
 =?utf-8?B?ODBOK2tseUpDZGxXcnY3N2VlUmFqcHN2ZkJvTjZxT0J4QWowZThRMXBHeHVR?=
 =?utf-8?B?T09jVGlsUVlhRkI1YVdGU0UxczhsUkdxZmowMy9ZRnlsRXFtUGhPY1lNNXlk?=
 =?utf-8?B?VTAvWHJPK095VjRrT0kzWENIS1M5RUl6aE9UaTdrdHVxSjFheUd0Zm9OZ09I?=
 =?utf-8?B?WW5YZnh3SkRlOUV6MDZsNzBlUVlBbXlKRzlLK1U3YWtLeXQ1eVkxVjFUaEZU?=
 =?utf-8?B?b2grMys4a2NqRDZ3VWRkL0xseHVvVkdoY1Q5cVFvQXRsWW5TcUlheU9QZjJQ?=
 =?utf-8?B?d25KS3hIMzBySUtRNG9WNlpqRnBBb1pZenFaQWZ3anF3S0FQU2t2UjlhZzNv?=
 =?utf-8?B?bXU2eXRoV0R6OUlMZ3Z1MEJlU2xsRXlQaWIrZ1JGRWNZdWFMcHhmY1E1eDlu?=
 =?utf-8?B?cFlaTEc5LzI2VEdURzVSQnNWWXJoVUpjMGhZN2dZRFpyc3ZJTm5XaDFJcWNG?=
 =?utf-8?B?WDZkQkExWElSekpFQlUrcEttZDVTUnVHa3k3b0tJanFtREVwRjR3K0xJTVJL?=
 =?utf-8?B?R1UrOWVNSURRdzdSeEJQMXZ5eTNITFhHQ0k0cXhOZ1VYYnAycCsvdDFpSWVx?=
 =?utf-8?B?QzZTdVZVVlhNYmpVL0cvRlVNWnJkWFB5bHBmeTVJWU5HckV1ZlNtZDVlY01T?=
 =?utf-8?B?c0w1SU9TOU95R0FIVXF3ZlF6dFRtdVNvaDFYSkxTOEMxQTVrTGh2ajErWkpC?=
 =?utf-8?B?czZtK29ONnpna1FqU29yNHpRVmNkSnZiTGNRUXJlVDl0NG9wb1hNSC9ZUE1m?=
 =?utf-8?B?cGtZcEN4T3dqV0dZRkFndytweHVEd3FJQ3I3MDhRYUdkdE1iSmNKS2QzVWRl?=
 =?utf-8?B?MmFVQmpFMW9mZDg2bjNBb3dzY25aVkJrbHU4UUZXQ3ZRVVcvcEFrQ0d2UDll?=
 =?utf-8?B?WncwVmNFZzZlRW5LTXBaSmVXbGlqRFQ3dkhmaUV5dnBXblFla3dRdWI4dEVL?=
 =?utf-8?B?YXIySTl5RFlJdnBnOEpZdnJCSmZ6MVZ6alduMG1wU0FmblRVaEkzUG8rTmE3?=
 =?utf-8?B?azg0SVFYVG0xNE9NSEpOMFhYS0lXVDJiUkJHY3VMRmZSQWJxaFlYK1UwWDNs?=
 =?utf-8?B?M0JVbVNHdUJaMHFhdVIxT01jN3QwZWJCRmdtQWtoSlRGcnpuMTdMSnk4SndU?=
 =?utf-8?B?ZlRXRlcrYXN0cXBXbkluSUMvQjQwYzBMRHVkTGtFN1VuTjJ6YXh1Q0RqNlU4?=
 =?utf-8?B?cmQ4QVVUTjFiWTFHY0E2RGhpZ01LbHJyMmZSQnY5Um9oVmtaM3pOellQRG9h?=
 =?utf-8?B?ZDZ3N1RReE9sM3hINnk3VCtaTGwxeEtpd2wvVjNvWm5QOE1GVys0UUhjNFhG?=
 =?utf-8?Q?wDN8AJer9aFER2D0E/5vv/m1Ne2SNshm7mBr/o6?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E849FA8469BD7741B5A8693A822429AA@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ee3861b-a540-45b1-6352-08d973941e0e
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 13:17:04.4964
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: j/udZa7qljCsN/QsZA9HK/ubwU2aw5exiD4bMAJRLWE3Gykkz4+Vg97HwAyObjjecS3y7U17ArpPq1JkWOTjfuFnGntv73+SNmXKXAq0Ooa5dHAUGJDjzDd2Tis92Box
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4884
X-Proofpoint-GUID: PwO0NC4BRDLtFhrCqug9hdOv3NSGYd4j
X-Proofpoint-ORIG-GUID: PwO0NC4BRDLtFhrCqug9hdOv3NSGYd4j
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0
 clxscore=1015 adultscore=0 mlxscore=0 priorityscore=1501 spamscore=0
 lowpriorityscore=0 mlxlogscore=999 bulkscore=0 phishscore=0
 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090081

DQpPbiAwOS4wOS4yMSAxNTo0NywgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDA5LjA5LjIwMjEg
MTQ6NDIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMDkuMDkuMjEgMTQ6
NTMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDA5LjA5LjIwMjEgMTM6NDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAwOS4wOS4yMSAxMjoyMSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBGb3IgdGhlIGJpdCBpbiBxdWVzdGlvbiwgd2hlcmUgdGhlIGdvYWwg
YXBwZWFycyB0byBiZSB0byBoYXZlIGhhcmR3YXJlDQo+Pj4+PiBob2xkIHRoZSBPUiBvZiBndWVz
dCBhbmQgaG9zdCB2YWx1ZXMsIGFuIGFwcHJvYWNoIHNpbWlsYXIgdG8gdGhhdCB1c2VkDQo+Pj4+
PiBmb3Igc29tZSBvZiB0aGUgTVNJIC8gTVNJLVggYml0cyBtaWdodCBiZSBjaG9zZW46IE1haW50
YWluIGd1ZXN0IGFuZA0KPj4+Pj4gaG9zdCBiaXRzIGluIHNvZnR3YXJlLCBhbmQgdXBkYXRlIGhh
cmR3YXJlIChhdCBsZWFzdCkgd2hlbiB0aGUNCj4+Pj4+IGVmZmVjdGl2ZSByZXN1bHRpbmcgdmFs
dWUgY2hhbmdlcy4gQSBjb21wbGljYXRpbmcgZmFjdCBoZXJlIGlzLCB0aG91Z2gsDQo+Pj4+PiB0
aGF0IHVubGlrZSBmb3IgdGhlIE1TSSAvIE1TSS1YIGJpdHMgaGVyZSBEb20wIChwY2liYWNrIG9y
IGl0cyBQQ0kNCj4+Pj4+IHN1c2JzdGVtKSBtYXkgYWxzbyBoYXZlIGEgdmlldyBvbiB3aGF0IHRo
ZSBzZXR0aW5nIG91Z2h0IHRvIGJlLg0KPj4+PiBUaGUgYmlnZ2VyIHF1ZXN0aW9uIGhlcmUgaXMg
d2hhdCBjYW4gd2UgdGFrZSBhcyB0aGUgcmVmZXJlbmNlIGZvciBJTlR4DQo+Pj4+IGJpdCwgZS5n
LiBpZiBEb20wIGRpZG4ndCBlbmFibGUvY29uZmlndXJlZCB0aGUgZGV2aWNlIGJlaW5nIHBhc3Nl
ZCB0aHJvdWdoDQo+Pj4+IHRoYW4gaXRzIENPTU1BTkQgcmVnaXN0ZXIgbWF5IHN0aWxsIGJlIGlu
IGFmdGVyIHJlc2V0IHN0YXRlIGFuZCBJTU8gdGhlcmUgaXMNCj4+Pj4gbm8gZ3VhcmFudGVlIGl0
IGhhcyB0aGUgdmFsdWVzIHdlIGNhbiBzYXkgYXJlICJhcyBob3N0IHdhbnRzIHRoZW0iDQo+Pj4g
SW4gdGhlIGFic2VuY2Ugb2YgRG9tMCBjb250cm9sbGluZyB0aGUgZGV2aWNlLCBJIHRoaW5rIHdl
IG91Z2h0IHRvIHRha2UNCj4+PiBYZW4ncyB2aWV3IGFzIHRoZSAiaG9zdCIgb25lLg0KPj4gQWdy
ZWUNCj4+PiAgICBXaGljaCB3aWxsIHdhbnQgdGhlIGJpdCBzZXQgYXQgbGVhc3QgYXMNCj4+PiBs
b25nIGFzIGVpdGhlciBNU0kgb3IgTVNJLVggaXMgZW5hYmxlZCBmb3IgdGhlIGRldmljZS4NCj4+
IEJ1dCB3aGF0IGlzIHRoZSBJTlR4IHJlbGF0aW9uIHRvIE1TSS9NU0ktWCBoZXJlPw0KPiBEZXZp
Y2VzIGFyZSBub3Qgc3VwcG9zZWQgdG8gc2lnbmFsIGludGVycnVwdHMgdHdvIGRpZmZlcmVudCB3
YXlzIGF0IGENCj4gdGltZS4gVGhleSBtYXkgZW5hYmxlIG9ubHkgb25lIC0gcGluIGJhc2VkLCBN
U0ksIG9yIE1TSS1YLg0KDQpPaywgc28gSSBzZWUgdGhhdCB3ZSBjYW4gcGFydGlhbGx5IGVtdWxh
dGUgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgYXM6DQoNCnN0YXRpYyB2b2lkIGd1ZXN0X2NtZF93cml0
ZShjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldiwgdW5zaWduZWQgaW50IHJlZywNCiDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDMyX3Qg
Y21kLCB2b2lkICpkYXRhKQ0Kew0KIMKgwqDCoCAvKiBUT0RPOiBBZGQgcHJvcGVyIGVtdWxhdGlv
biBmb3IgYWxsIGJpdHMgb2YgdGhlIGNvbW1hbmQgcmVnaXN0ZXIuICovDQoNCiDCoMKgwqAgaWYg
KCAoY21kICYgUENJX0NPTU1BTkRfSU5UWF9ESVNBQkxFKSA9PSAwICkNCiDCoMKgwqAgew0KIMKg
wqDCoMKgwqDCoMKgIC8qDQogwqDCoMKgwqDCoMKgwqDCoCAqIEd1ZXN0IHdhbnRzIHRvIGVuYWJs
ZSBJTlR4LiBJdCBjYW4ndCBiZSBlbmFibGVkIGlmOg0KIMKgwqDCoMKgwqDCoMKgwqAgKsKgIC0g
aG9zdCBoYXMgSU5UeCBkaXNhYmxlZA0KIMKgwqDCoMKgwqDCoMKgwqAgKsKgIC0gTVNJL01TSS1Y
IGVuYWJsZWQNCiDCoMKgwqDCoMKgwqDCoMKgICovDQogwqDCoMKgwqDCoMKgwqAgaWYgKCBwZGV2
LT52cGNpLT5tc2ktPmVuYWJsZWQgKQ0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgY21kIHw9IFBD
SV9DT01NQU5EX0lOVFhfRElTQUJMRTsNCiDCoMKgwqDCoMKgwqDCoCBlbHNlDQogwqDCoMKgwqDC
oMKgwqAgew0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDE2X3QgY3VycmVudF9jbWQgPSBw
Y2lfY29uZl9yZWFkMTYocGRldi0+c2JkZiwgcmVnKTsNCg0KIMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgaWYgKCBjdXJyZW50X2NtZCAmIFBDSV9DT01NQU5EX0lOVFhfRElTQUJMRSApDQogwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNtZCB8PSBQQ0lfQ09NTUFORF9JTlRYX0RJU0FCTEU7
DQogwqDCoMKgwqDCoMKgwqAgfQ0KIMKgwqDCoCB9DQoNCiDCoMKgwqAgY21kX3dyaXRlKHBkZXYs
IHJlZywgY21kLCBkYXRhKTsNCn0NCg0KYW5kIG9mIGNvdXJzZSBoYXZlIGdyYW5kIFRPRE8gZm9y
IHRoZSByZXN0Lg0KDQo+DQo+IEphbg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:23:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:23:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183263.331334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOK1n-0004au-Vj; Thu, 09 Sep 2021 13:23:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183263.331334; Thu, 09 Sep 2021 13:23:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOK1n-0004an-SY; Thu, 09 Sep 2021 13:23:43 +0000
Received: by outflank-mailman (input) for mailman id 183263;
 Thu, 09 Sep 2021 13:23:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOK1m-0004ah-FF
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:23:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOK1m-0006Zw-Dk
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:23:42 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOK1m-0001No-CZ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:23:42 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOK1j-0006S7-30; Thu, 09 Sep 2021 14:23:39 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=6mn2J4fCktS3gK9y9aI9Ib+3/dFLwQ5S1oqkz03U5iY=; b=3EwW3eeIaUTdIdwcjcWaC1LYKA
	KsCvmlk3OIx+undqBYTiIDmLmF3nr1rXk8h7J3v3Bn3Rf6vvvs29n/vR/f/9bYpiaLKFMsunpXlqY
	YTI17ERgV0WVcWi+51LnIGZBbAHg+T1bVG9tEgRkVsifsjMPyjb22pU6GCh6XqlwlAOs=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24890.2778.780247.724651@mariner.uk.xensource.com>
Date: Thu, 9 Sep 2021 14:23:38 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Ian Jackson <iwj@xenproject.org>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
In-Reply-To: <20210909124924.1698-2-jgross@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
	<20210909124924.1698-2-jgross@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("[PATCH v2 1/3] stubdom: fix build with disabled pv-grub"):
> Today the build will fail if --disable-pv-grub as a parameter of
> configure, as the main Makefile will unconditionally try to build a
> 32-bit pv-grub stubdom.
> 
> Fix that by introducing a pv-grub32 target in stubdom/Makefile taking
> care of this situation.

I approve of this whole series, with one resrvation:

I think the name "pv-grub32" for this target is confusing.  It's not
really specifically 32-bit.  The difference between the targets
"pv-grub32" and "pv-grub" is that "pv-grub32" is unconditionally
built but might mean nothing; it has a conditional dependency on
"pv-grub" which is conditionally built but always implies the actual
build.

I don't think the suffix "32" really conveys this :-).

How about "pv-grub-maybe" ?  Or something.

You can put my ack on patches 2 and 3 right away.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:50:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:50:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183272.331345 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKRj-0008De-5Z; Thu, 09 Sep 2021 13:50:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183272.331345; Thu, 09 Sep 2021 13:50:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKRj-0008DX-27; Thu, 09 Sep 2021 13:50:31 +0000
Received: by outflank-mailman (input) for mailman id 183272;
 Thu, 09 Sep 2021 13:50:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOKRh-0008DR-Pp
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:50:29 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOKRe-00073v-J8; Thu, 09 Sep 2021 13:50:26 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOKRe-0003BM-CP; Thu, 09 Sep 2021 13:50:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=mYCChir0O1ZGDCIEkLmpWJAywMYlYctuPRMC/DLCKSU=; b=2l8a+ZZ+j4Ji2/6jjKqx3bbsWS
	uIW+H2FxwXaO+MVR5XTftbG7Ltbq9UOYUbIRkT1iPn/gJR1nkUic2idf62mmT1zAuj8KzEMocclvl
	6L6UK0cq8EBTzvk0gabwMQcLoUSWjoMAF1qQAzDMa/s42dqrWfKezwPFnUzMmCT8VYrc=;
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
To: Rahul Singh <rahul.singh@arm.com>, xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com, Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <a868e1e7-8400-45df-6eaa-69f1e2c99383@xen.org>
Date: Thu, 9 Sep 2021 14:50:24 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Rahul,

On 19/08/2021 13:02, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> access is added to the Xen to emulate the PCI devices config space.
> 
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>   xen/arch/arm/Makefile         |  1 +
>   xen/arch/arm/domain.c         |  4 ++
>   xen/arch/arm/vpci.c           | 96 +++++++++++++++++++++++++++++++++++
>   xen/arch/arm/vpci.h           | 37 ++++++++++++++
>   xen/drivers/passthrough/pci.c |  7 +++
>   xen/drivers/vpci/Makefile     |  3 +-
>   xen/drivers/vpci/header.c     |  2 +
>   xen/include/asm-arm/domain.h  |  5 +-
>   xen/include/asm-arm/pci.h     |  8 +++
>   xen/include/public/arch-arm.h |  4 ++
>   10 files changed, 165 insertions(+), 2 deletions(-)
>   create mode 100644 xen/arch/arm/vpci.c
>   create mode 100644 xen/arch/arm/vpci.h
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 0e14a5e5c8..7cdce684a4 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -7,6 +7,7 @@ obj-y += platforms/
>   endif
>   obj-$(CONFIG_TEE) += tee/
>   obj-$(CONFIG_HAS_PCI) += pci/
> +obj-$(CONFIG_HAS_VPCI) += vpci.o
>   
>   obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>   obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..d99c653626 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -40,6 +40,7 @@
>   #include <asm/vtimer.h>
>   
>   #include "vuart.h"
> +#include "vpci.h"

Please order the includes alphabetically. So this one should go before 
"vuart.h".

>   
>   DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
>   
> @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
>       if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
>           goto fail;
>   
> +    if ( (rc = domain_vpci_init(d)) != 0 )
> +        goto fail;
> +
>       return 0;
>   
>   fail:
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> new file mode 100644
> index 0000000000..da8b1ca13c
> --- /dev/null
> +++ b/xen/arch/arm/vpci.c
> @@ -0,0 +1,96 @@
> +/*
> + * xen/arch/arm/vpci.c
> + * Copyright (c) 2021 Arm Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +#include <xen/sched.h>

NIT: Please add a newline between generic and arch specific includes.

> +#include <asm/mmio.h>
> +
> +/* Do some sanity checks. */
> +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
> +{
> +    /* Check access size. */
> +    if ( len != 1 && len != 2 && len != 4 && len != 8 )
> +        return false;
You will allow all the possible value of len (this is coming from the 
HW). So I feels this is a bit too much to check for every I/O to the vPCI.

If you really want to keep the check then you can simply check that len 
is < 8 because the two callers compute it with (1 << S). So there is no 
way to set it 3, 5, 6 and 7.

> +
> +    /* Check that access is size aligned. */
> +    if ( (reg & (len - 1)) )
> +        return false;
> +
> +    return true;
> +}
> +
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                          register_t *r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    uint32_t data = 0;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12);
> +    reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3));

This logic is the same as below in vpci_mmio_write(). So I think you 
want to provide an helper because this is not trivial to read.

Also, for the first line, I think you can re-use MMCFG_BDF() from the 
x86 code. For the second line, I would define the value so it is clearer 
to understand that they mean (although & 3 is fine to me) .

> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 1;
So, you will a guest will read 0 if the access is unaligned. This seems 
an odd behavior given this is not an allowed access. AFAIU, the HW would 
likely trow a data abort because you can't do unalign access on 
uncachable memory. So I think we should return 0 here to let the MMIO 
handler inject a data abort.

> +
> +    data = vpci_read(sbdf, reg, size);

So in vpci_mmio_access_allowed(), you will allow a guest to read a 
64-bit value. But... vpci_read() will return a 32-bit value.

Looking at the x86 code, they have a second call to vpci_read() to 
handle the top 32-bit. Any reason why this was not implemented on Arm?

If we need to implement it then I think this should be implement in 
vpci_read() to avoid duplication between x86 and arm.

> +
> +    memcpy(r, &data, size);

 From my understanding, any unused bit should be 0. So this should be:

*r = data;

> +
> +    return 1;
> +}
> +
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                           register_t r, void *p)

My remarks on vpci_mmio_read() applies here too.

> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    uint32_t data = r;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12);
> +    reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3));
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 1;
> +
> +    vpci_write(sbdf, reg, size, data);
> +
> +    return 1;
> +}
> +
> +static const struct mmio_handler_ops vpci_mmio_handler = {
> +    .read  = vpci_mmio_read,
> +    .write = vpci_mmio_write,
> +};
> +
> +int domain_vpci_init(struct domain *d)
> +{
> +    if ( !has_vpci(d) )
> +        return 0;
> +
> +    register_mmio_handler(d, &vpci_mmio_handler,
> +                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> +
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> new file mode 100644
> index 0000000000..8a093bb705
> --- /dev/null
> +++ b/xen/arch/arm/vpci.h
> @@ -0,0 +1,37 @@
> +/*
> + * xen/arch/arm/vpci.h
> + * Copyright (c) 2021 Arm Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ARCH_ARM_VPCI_H__
> +#define __ARCH_ARM_VPCI_H__
> +
> +#ifdef CONFIG_HAS_VPCI
> +int domain_vpci_init(struct domain *d);
> +#else
> +static inline int domain_vpci_init(struct domain *d)
> +{
> +    return 0;
> +}
> +#endif
> +
> +#endif /* __ARCH_ARM_VPCI_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index c23c8cb06b..56e261e9bd 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -767,6 +767,13 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>       else
>           iommu_enable_device(pdev);
>   
> +#ifdef CONFIG_ARM
> +    ret = vpci_add_handlers(pdev);
> +    if ( ret ) {
> +        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
> +        goto out;
> +    }
> +#endif
>       pci_enable_acs(pdev);
>   
>   out:
> diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
> index 55d1bdfda0..1a1413b93e 100644
> --- a/xen/drivers/vpci/Makefile
> +++ b/xen/drivers/vpci/Makefile
> @@ -1 +1,2 @@
> -obj-y += vpci.o header.o msi.o msix.o
> +obj-y += vpci.o header.o
> +obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index ba9a036202..f8cd55e7c0 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
>        * FIXME: punching holes after the p2m has been set up might be racy for
>        * DomU usage, needs to be revisited.
>        */
> +#ifdef CONFIG_HAS_PCI_MSI
>       if ( map && !rom_only && vpci_make_msix_hole(pdev) )
>           return;
> +#endif
>   
>       for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
>       {
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..d742b94bd6 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -262,7 +262,10 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
>   
>   #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
>   
> -#define has_vpci(d)    ({ (void)(d); false; })
> +/* For X86 VPCI is enabled and tested for PVH DOM0 only but
> + * for ARM we enable support VPCI for guest domain also.
> + */
> +#define has_vpci(d) ({ (void)(d); IS_ENABLED(CONFIG_HAS_VPCI); })
>   
>   #endif /* __ASM_DOMAIN_H__ */
>   
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 756f8637ab..c58152de80 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -27,6 +27,14 @@ struct arch_pci_dev {
>       struct device dev;
>   };
>   
> +/* Arch-specific MSI data for vPCI. */
> +struct vpci_arch_msi {
> +};
> +
> +/* Arch-specific MSI-X entry data for vPCI. */
> +struct vpci_arch_msix_entry {
> +};
> +
>   /*
>    * struct to hold the mappings of a config space window. This
>    * is expected to be used as sysdata for PCI controllers that
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 64a2ca30da..0a9749e768 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -422,6 +422,10 @@ typedef uint64_t xen_callback_t;
>   #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
>   #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
>   
> +/* VPCI ECAM mappings */
> +#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)

All the values for the memory layout has been defined in ascending 
order. So please add the vCPI at the correct place. If I am not 
mistaken, dhis should be before the GUEST_ACPI_*.

> +#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)

Please document how to decide the size. This is important for the future 
if we need to change the size.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:59:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:59:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183278.331356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKaV-0000ot-3I; Thu, 09 Sep 2021 13:59:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183278.331356; Thu, 09 Sep 2021 13:59:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKaU-0000om-Vn; Thu, 09 Sep 2021 13:59:34 +0000
Received: by outflank-mailman (input) for mailman id 183278;
 Thu, 09 Sep 2021 13:59:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOKaT-0000og-OI
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:59:33 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOKaQ-0007C6-11; Thu, 09 Sep 2021 13:59:30 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOKaP-0003oh-QS; Thu, 09 Sep 2021 13:59:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=DISAgcwJ9G17JAtLAr0zj7opL0G1q7kbwPhmCq4fhPA=; b=1KcMhlmPt/UPDXlUEiA8EZocoa
	LICM6sBrUOhsww+zsA84pVWqD5H/7SWjcqBLPgBlHxyeKocJTCC3h9TuNMMr1XbA9bVG9MrupS52h
	JeNRdivHu9lX5bUsCyuam3oOP2BD06o5vgG4Z8wBK5bdCX3UeGOnuUa1/7fH6vFzfOT8=;
Subject: Re: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
To: Rahul Singh <Rahul.Singh@arm.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>, Ian Jackson
 <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com>
 <c0ea2f05-bf0a-a8b0-a811-d899516c26ba@xen.org>
 <664B0578-B440-4229-8D6B-7B98857E75BF@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <3e192581-4167-c9aa-2c32-ff0338e9b800@xen.org>
Date: Thu, 9 Sep 2021 14:59:27 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <664B0578-B440-4229-8D6B-7B98857E75BF@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 20/08/2021 17:03, Rahul Singh wrote:
> Hi Julien,

Hi Rahul,

>> On 19 Aug 2021, at 2:00 pm, Julien Grall <julien@xen.org> wrote:
>>
>> Hi Rahul,
>>
>> On 19/08/2021 13:02, Rahul Singh wrote:
>>> libxl will create an emulated PCI device tree node in the device tree to
>>> enable the guest OS to discover the virtual PCI during guest boot.
>>> Emulated PCI device tree node will only be created when there is any
>>> device assigned to guest.
>>> A new area has been reserved in the arm guest physical map at
>>> which the VPCI bus is declared in the device tree (reg and ranges
>>> parameters of the node).
>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>> ---
>>>   tools/libs/light/libxl_arm.c          | 109 ++++++++++++++++++++++++++
>>>   tools/libs/light/libxl_types.idl      |   1 +
>>>   tools/xl/xl_parse.c                   |   2 +
>>>   xen/include/public/arch-arm.h         |  11 +++
>>>   xen/include/public/device_tree_defs.h |   1 +
>>>   5 files changed, 124 insertions(+)
>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>>> index e3140a6e00..a091e97e76 100644
>>> --- a/tools/libs/light/libxl_arm.c
>>> +++ b/tools/libs/light/libxl_arm.c
>>> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
>>>       return fdt_property(fdt, "reg", regs, sizeof(regs));
>>>   }
>>>   +static int fdt_property_values(libxl__gc *gc, void *fdt,
>>> +        const char *name, unsigned num_cells, ...)
>>> +{
>>> +    uint32_t prop[num_cells];
>>> +    be32 *cells = &prop[0];
>>> +    int i;
>>> +    va_list ap;
>>> +    uint32_t arg;
>>> +
>>> +    va_start(ap, num_cells);
>>> +    for (i = 0 ; i < num_cells; i++) {
>>> +        arg = va_arg(ap, uint32_t);
>>> +        set_cell(&cells, 1, arg);
>>> +    }
>>> +    va_end(ap);
>>> +
>>> +    return fdt_property(fdt, name, prop, sizeof(prop));
>>> +}
>>> +
>>> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
>>> +                                    unsigned addr_cells,
>>> +                                    unsigned size_cells,
>>> +                                    unsigned num_regs, ...)
>>> +{
>>> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
>>> +    be32 *cells = &regs[0];
>>> +    int i;
>>> +    va_list ap;
>>> +    uint64_t arg;
>>> +
>>> +    va_start(ap, num_regs);
>>> +    for (i = 0 ; i < num_regs; i++) {
>>> +        /* Set the memory bit field */
>>> +        arg = va_arg(ap, uint64_t);
>>> +        set_cell(&cells, 1, arg);
>>> +
>>> +        /* Set the vpci bus address */
>>> +        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
>>> +        set_cell(&cells, addr_cells , arg);
>>> +
>>> +        /* Set the cpu bus address where vpci address is mapped */
>>> +        set_cell(&cells, addr_cells, arg);
>>> +
>>> +        /* Set the vpci size requested */
>>> +        arg = size_cells ? va_arg(ap, uint64_t) : 0;
>>> +        set_cell(&cells, size_cells,arg);
>>> +    }
>>> +    va_end(ap);
>>> +
>>> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
>>> +}
>>> +
>>>   static int make_root_properties(libxl__gc *gc,
>>>                                   const libxl_version_info *vers,
>>>                                   void *fdt)
>>> @@ -668,6 +720,57 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
>>>       return 0;
>>>   }
>>>   +static int make_vpci_node(libxl__gc *gc, void *fdt,
>>> +        const struct arch_info *ainfo,
>>> +        struct xc_dom_image *dom)
>>> +{
>>> +    int res;
>>> +    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
>>> +    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
>>> +    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
>>> +
>>> +    res = fdt_begin_node(fdt, name);
>>> +    if (res) return res;
>>> +
>>> +    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
>>> +    if (res) return res;
>>> +
>>> +    res = fdt_property_string(fdt, "device_type", "pci");
>>> +    if (res) return res;
>>> +
>>> +    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
>>> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
>>> +    if (res) return res;
>>> +
>>> +    res = fdt_property_values(gc, fdt, "bus-range", 2, 0,17);
>>
>> AFAICT, the "bus-range" is optional. Can you explain why we need it?
> 
> We need it to implement the function pci_ecam_map_bus().

Ok. Then why next question is what does the 17 mean? Is it tie to how we
implement the vPCI in Xen or the region we reserved?

[...]

>>
>>> +
>>>       if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
>>>           return;
>>>   diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>>> index 3f9fff653a..78b1ddf0b8 100644
>>> --- a/tools/libs/light/libxl_types.idl
>>> +++ b/tools/libs/light/libxl_types.idl
>>> @@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>>>         ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>>>                                  ("vuart", libxl_vuart_type),
>>> +                               ("vpci", libxl_defbool),
>>
>> Any new addition in the structure should be accompanied with a LIBXL_HAVE_* in the libxl.h header.
> 
> OK.
>>
>>>                                 ])),
>>>       ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>>>                                 ])),
>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>> index 17dddb4cd5..ffafbeffb4 100644
>>> --- a/tools/xl/xl_parse.c
>>> +++ b/tools/xl/xl_parse.c
>>> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>>>           }
>>>           if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
>>>               libxl_defbool_set(&b_info->u.pv.e820_host, true);
>>> +        if (d_config->num_pcidevs)
>>> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>>>       }
>>>         if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
>>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>>> index 0a9749e768..01d13e669e 100644
>>> --- a/xen/include/public/arch-arm.h
>>> +++ b/xen/include/public/arch-arm.h
>>> @@ -426,6 +426,17 @@ typedef uint64_t xen_callback_t;
>>>   #define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
>>>   #define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
>>>   +/* PCI-PCIe memory space types */
>>> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000)
>>> +#define GUEST_VPCI_ADDR_TYPE_MEM          xen_mk_ullong(0x02000000)
>>
>> What the size of those regions?
> 
> Non Prefetch Memory: Size 64 MB start at 512 MB
> Prefetch Memory: Size 128 GB start at 36 GB
>>
>>> +
>>> +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
>>> +#define GUEST_VPCI_PREFETCH_MEM_ADDR  xen_mk_ullong(0x900000000)
>>> +#define GUEST_VPCI_MEM_ADDR           xen_mk_ullong(0x20000000)
>> So far the memory layout defines the address in ascending order. So please add that after GUEST_RAM_BANK_BASES_*.
> 
> Ok.
>>
>> However, if I am not mistaken that base address you provide will clash with RAM bank 1. It also seem to be pretty high which means that this will not work for 32-bit domain or on CPUs that don't allow offer large IPA bits.
> 
> Yes I also checked that now that it is having clash with RAM bank 1.
> There is unused space is guest memory that we can use for Non Prefetch Memory as per below guest memory map.
> https://gitlab.com/xen-project/fusa/fusa-docs/-/blob/master/high-level/guest-memory-layout-arm.rst
> 
> Proposed value:
> Non Prefetch Memory: Size 64 MB start at 0x22001000
> Prefetch Memory: Size 4 GB start at 4 GB.

The base address looks fine to me. However, the sizes are much smaller 
to what you initially suggested. Would you be able to clarify why the 
smaller sizes are fine?

> 
>>
>> I think we need to start making the guest layout more dynamic. The VPCI memory space would have to go right after the end of the RAM allocated for a given guest.
>>
>>> +
>>> +#define GUEST_VPCI_PREFETCH_MEM_SIZE      xen_mk_ullong(0x2000000000)
>>> +#define GUEST_VPCI_MEM_SIZE               xen_mk_ullong(0x04000000)
>>
>> It would be better if the size for each region is defined right after each base.
> OK.
> 
>>
>> Also, how did you decide the size of each region?
> 
> I thought 64 MB will be sufficient. I think it should be based on number of devices we can assign to the guest.

We don't have to get the size right now. What I am more interested is to 
have a trace about how those values were decided (even if it just saying 
random). This will help to make any decision if in the future we need to 
resize (in particular downsize) the regions.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 13:59:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 13:59:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183279.331367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKac-00017N-BY; Thu, 09 Sep 2021 13:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183279.331367; Thu, 09 Sep 2021 13:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKac-00017G-7I; Thu, 09 Sep 2021 13:59:42 +0000
Received: by outflank-mailman (input) for mailman id 183279;
 Thu, 09 Sep 2021 13:59:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vCZp=N7=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOKaa-00016Y-Oy
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 13:59:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2c7854c8-1176-11ec-b1ba-12813bfff9fa;
 Thu, 09 Sep 2021 13:59:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id DC6A4222E3;
 Thu,  9 Sep 2021 13:59:38 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 873E313CB1;
 Thu,  9 Sep 2021 13:59:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id iKmDH0oTOmG4KAAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 09 Sep 2021 13:59:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c7854c8-1176-11ec-b1ba-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631195978; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=hCRggMsORaaG1sgrfpMDP9w+tbtLEOQD8pweB7qQrt8=;
	b=CBFbFlNw0Pn5SrBSSKqniu4tBQFFkPxhMwAkxto4eX9vt/ypofPDkWvm4yz0kzvVPMgkqu
	4iV1lRAGCIgaVQn7R4iHptcvSi30jOdPO5GjtIhZpFveqG996P3/ha/QC1XyVrwYk9q2pz
	xIjyo+G2Zx72cPq8TO/vFijPa2WjFag=
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
Date: Thu, 9 Sep 2021 15:59:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24890.2778.780247.724651@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="J2fl8HZjBfp1CsJyPXdH9VfMC0EvM5HUA"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--J2fl8HZjBfp1CsJyPXdH9VfMC0EvM5HUA
Content-Type: multipart/mixed; boundary="L2kFOaVTdsvVHEo9Y8ITk2SDNzTDVbEl2";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-ID: <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
In-Reply-To: <24890.2778.780247.724651@mariner.uk.xensource.com>

--L2kFOaVTdsvVHEo9Y8ITk2SDNzTDVbEl2
Content-Type: multipart/mixed;
 boundary="------------076DFAAB71190CDFBB4DE472"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------076DFAAB71190CDFBB4DE472
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 09.09.21 15:23, Ian Jackson wrote:
> Juergen Gross writes ("[PATCH v2 1/3] stubdom: fix build with disabled =
pv-grub"):
>> Today the build will fail if --disable-pv-grub as a parameter of
>> configure, as the main Makefile will unconditionally try to build a
>> 32-bit pv-grub stubdom.
>>
>> Fix that by introducing a pv-grub32 target in stubdom/Makefile taking
>> care of this situation.
>=20
> I approve of this whole series, with one resrvation:
>=20
> I think the name "pv-grub32" for this target is confusing.  It's not
> really specifically 32-bit.  The difference between the targets
> "pv-grub32" and "pv-grub" is that "pv-grub32" is unconditionally
> built but might mean nothing; it has a conditional dependency on
> "pv-grub" which is conditionally built but always implies the actual
> build.
>=20
> I don't think the suffix "32" really conveys this :-).
>=20
> How about "pv-grub-maybe" ?  Or something.

What about "pv-grub-if-enabled"?

And could that be done when committing, or should I send another round?

>=20
> You can put my ack on patches 2 and 3 right away.

Thanks,


Juergen

--------------076DFAAB71190CDFBB4DE472
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------076DFAAB71190CDFBB4DE472--

--L2kFOaVTdsvVHEo9Y8ITk2SDNzTDVbEl2--

--J2fl8HZjBfp1CsJyPXdH9VfMC0EvM5HUA
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE6E0kFAwAAAAAACgkQsN6d1ii/Ey+q
oQf/ZsqsngOPh3iSNQT0GhGU+fjWX6FdRh+zZLVMCg6Sb659TwjU2G3zwljYe9Lyra0As69CROyl
ttbVwxKs+S21n6rE3aDduFo8ili9dTSkdit+bgNtHaFhratl5D5NCgxNYEBgjoCjJXD67nHd+Xp2
Ikyp9RV8nvEVcBMIcdx71SgWyvxn/SCHuZ0JMSIJFa1o7a944PQPgUh5HC5SSIvwgbnuLZkuvrYH
sSkh38rX69EjsIbOd1KAQUZZRVQlCMJ6QBbRyv5PtU7vqGZUfj56GCgeDJwsOjELnStf0OAAe6x4
JwrGM8TXbNF6OHrWYvLNk4P4cRXEdSEPjCCv+ae+gw==
=4yYx
-----END PGP SIGNATURE-----

--J2fl8HZjBfp1CsJyPXdH9VfMC0EvM5HUA--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 14:26:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 14:26:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183292.331377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKzn-0005Ss-CL; Thu, 09 Sep 2021 14:25:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183292.331377; Thu, 09 Sep 2021 14:25:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOKzn-0005Sl-9S; Thu, 09 Sep 2021 14:25:43 +0000
Received: by outflank-mailman (input) for mailman id 183292;
 Thu, 09 Sep 2021 14:25:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s+U0=N7=epam.com=prvs=9886b5da0c=roman_skakun@srs-us1.protection.inumbo.net>)
 id 1mOKzm-0005Sf-CE
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 14:25:42 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cf24765e-1179-11ec-b1bc-12813bfff9fa;
 Thu, 09 Sep 2021 14:25:40 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 189E3jUb022402; 
 Thu, 9 Sep 2021 14:25:38 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 by mx0b-0039f301.pphosted.com with ESMTP id 3aykpp064p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 09 Sep 2021 14:25:38 +0000
Received: from AM7PR03MB6593.eurprd03.prod.outlook.com (2603:10a6:20b:1b4::10)
 by AM6PR03MB5959.eurprd03.prod.outlook.com (2603:10a6:20b:e2::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep
 2021 14:25:29 +0000
Received: from AM7PR03MB6593.eurprd03.prod.outlook.com
 ([fe80::7863:ae83:98b2:59d6]) by AM7PR03MB6593.eurprd03.prod.outlook.com
 ([fe80::7863:ae83:98b2:59d6%9]) with mapi id 15.20.4500.016; Thu, 9 Sep 2021
 14:25:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf24765e-1179-11ec-b1bc-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=m4GfDRmBZTUUX/ShchjpqnNtE/NY9eEmSUMhSLLmFVCXDW0KYtG6EtCLe4aytdgkNFJe22/GnDkfjuILn6/oBPrlJe+58PYyPiANXi9MLzEAux/cbkWCDOeni1tkHj/RW23qPCAp0wubbpwDy2mSSX8nbcnEieY94VP8OTPZrrON+DjOqOV4Jf0bnF1whdFCU1QQjbHKeIBBLpeVvY6LSTMs0j4CHDRUA6wboxCWKafqT6oeNsoB6LfS4fY60DfKAQ69UcvvQeYs4yM124TduZZCd8BRsZCKO1IYzjY5Hd+c3D768sQ27xcOUtqbg75628wD47hmixNY/nXIP46/2Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xM/+i8ftZh3y1yhrEYlxwAFDr8KH9cXmU7SVWDfOIHs=;
 b=ZjUPboLOhNzmEf1pSUHBS5L/+05zbHj98DfnPjdo/i4Sb+yF0weSB2x+82s0ik1bX1v4bsRF3UwDhD0tf3BicYP/6NCHnOnWnGwqmXBbSLJ0a2CBtnNmmsA7y0EhV0obECO12SM9kbTLxNKlALBYDe5EwBno0G1fQESzL4O4q+xUekB3Dw1V8weT6i367so1dXg7x0kandaPyJRdr2LMfD331KMfO8S465aPepWt2TRJMt6oIFTRMbN7fGfJB6/0xBy+ng20BV0ct7SqdwFJzrOhTbAIKgJH6WrLGYeVz4KF939GJpnl6Tg39XoZkDtIuwOTg2j87chmI8yDIDBFCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xM/+i8ftZh3y1yhrEYlxwAFDr8KH9cXmU7SVWDfOIHs=;
 b=Sdt66G1UeCaaE3m4iegWM6ArTz9pE1HJTxymg7gLnjcjsQxQHL7pmX5015bPSu/P9LsJ++FXX62UZD8UCQOpClQ0bcJC6eqXHW0+BuWjXzBqxL9/wW12pXqpMHAx1h68Fw8WKDpmvl3lyHzCRyWQPZ7s9js1pjQDq7iLJLBHL0f5Jn/cVm2bjlPZG04FX4kvJ/LrRZpCTciolviAo+RcMPWcZLIaCEmGxOzoKGlaHmL2cXn1tQZbJrxQjSZgQyE+9iD47O1pUKphcf+3lVqAzpbLqCHYQONq/esGifEFV4ZwxfFDjWvdMPf+0wtr35ApYXV3lyx1jPPLx9nWRdGh3Q==
From: Roman Skakun <Roman_Skakun@epam.com>
To: Julien Grall <julien@xen.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        Andrii Anisov
	<Andrii_Anisov@epam.com>,
        Roman Skakun <rm.skakun@gmail.com>,
        Oleksandr
 Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
Subject: Re: Disable IOMMU in Dom0
Thread-Topic: Disable IOMMU in Dom0
Thread-Index: AQHXnnd7IVjUu+L6S0mPq9naNxJ1FauOJ3YAgADDGouAAA7qgIAMwIiG
Date: Thu, 9 Sep 2021 14:25:29 +0000
Message-ID: 
 <AM7PR03MB6593E7F0C691F98E07B8141C85D59@AM7PR03MB6593.eurprd03.prod.outlook.com>
References: 
 <AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com>
 <alpine.DEB.2.21.2108311444310.18862@sstabellini-ThinkPad-T480s>
 <AM7PR03MB659358C0EBA9D71AEEF7A60885CD9@AM7PR03MB6593.eurprd03.prod.outlook.com>
 <179e4b7f-38d9-991d-3f99-64a74559986d@xen.org>
In-Reply-To: <179e4b7f-38d9-991d-3f99-64a74559986d@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
suggested_attachment_session_id: 6f9836ef-1bd8-a00e-77c2-5b5617176f83
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c48b36da-45f1-4dbf-0594-08d9739dacc1
x-ms-traffictypediagnostic: AM6PR03MB5959:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM6PR03MB59594CD885E01F3CED5A585B85D59@AM6PR03MB5959.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 MXXnr1cilkOPE8wLHkxfjjNVa4tRfq037O3wzDqCl9bS3oSHFZ7KjbZ0lk/rYy+m4/WOnVuaBuqBFnvzgDhiFlqFylRLh/DqBmurpSoayZUz8xodbV2GYCbJEYR1klyuqsRwcDb1hEVZMddMvav2YE+yDnOLG3feZTzqz31vItWY6Y47MXG8vKme7gOD4eAUtC3pKiu9jhRyrgY6QboEr3hNUtEA/Dbn0TPC5/pG4JVzgrdnN5KjsIht0NSRO0b3aABIyQsa0WowKVDZd4vOzaPT3gWzWHMWIxRcfyr/2CHoK0F+e1zA5Uq2wYnIUG5DUfIgLKxgJ3M1H2Pa/maO8ouTcuhyKqT0ALrK63ZsqMXuhOpFB+2YwYgk81vPiaADGLTKdCzX0Km8z8fWoTjGGgjWv9wxj8BheLHurfDzNz/fQMqondPC1V9QMROIYGWT8sWnepFxd2C4RvbN3iltUwcoO+u4i9hIsWurpc8MnV0vMHesdyJSl7YRwPDw91b+HdFI44P+ylrps2R9VU86op9SO1gCMAiQ/A/L1Of6Zqz9ElpsPx7mzl3xSxEqrF/tW49HHsNmOdJfuTaDADxSIj7Yu2DPc2utpTV6p/0gpzClKjOeGsityiy3BZro6zW1XjuTI3cKvloBoScPJUW1o1NLlcyBeVT/VS6Umu90XaN96Oq1Gdh2UKNvmDL1tVO8+esgPGqqJs/wsYBriLhscHXfegI/n2UreNAGZa3M80xv+FBn3YzRzSElWtnDBOCpq4JauImCWy0e6czBMBIIcbh76dJVexJ1qirCO1y5GldVycdPee2Jdv3wZfNIfu2oJIaANAn9hg1q+3WraZxmGQ==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR03MB6593.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(396003)(346002)(39850400004)(376002)(8676002)(2906002)(6916009)(76116006)(91956017)(26005)(19627405001)(6506007)(53546011)(54906003)(71200400001)(55236004)(9686003)(52536014)(66446008)(5660300002)(66556008)(66476007)(7696005)(316002)(64756008)(66946007)(83380400001)(4326008)(33656002)(55016002)(166002)(38100700002)(122000001)(38070700005)(18265965003)(478600001)(966005)(86362001)(8936002)(186003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?us-ascii?Q?svjwp2paua5f8TtHscZRxfjoXKW2XOQtJCdO+nODgwf2L+4thqZwh15he0vr?=
 =?us-ascii?Q?GFdDMi3ffuF8zP2IFDa/xIhWouz993AhgIjdExVLjN4+67klc8jbDYpDDWgJ?=
 =?us-ascii?Q?nQAI45xsgyMqqTR0vB6Yzq4wiZf9DRUVucaR8bajhOdujJ6gUn2Fq09WiYGN?=
 =?us-ascii?Q?nHdsGKLCOE3BiohCmSl8yLsREb2mEqwuCxmk4qKXOknk6F0IS24n5v8p780l?=
 =?us-ascii?Q?AAWyJCiK+pStbVhxrklae1b85J9kXoWJ9FNptaufPeYjgMkAOD9OLOl5B+EY?=
 =?us-ascii?Q?LqSE2ricOOPqu3ZYa+iiNoHVQKBmn5rVf5WDcRYBiYs6wijKc7VUePYOHoiM?=
 =?us-ascii?Q?Y5oupko1ef8sYxgilDYd9EBCyV7ea5SEDNcBRE73kDjMoa3vPevIZf6t8GgZ?=
 =?us-ascii?Q?wQMj6LB7xfTUNZSYOcNb/rQUcinZrIgzUZwEHHJCfoYlFecPXJhmZBC3yCAw?=
 =?us-ascii?Q?vmQETfBJ0Z1zooIWDGCWp7p/n8XVWlHX9AeQ8WGgVaNMjLODZH1TMuRzhVZ0?=
 =?us-ascii?Q?bHCB/Y9b9rqjO9mA0R0R5xdwi360hT7D7idal9yjlcd9G18IiFGuC6cBjNbw?=
 =?us-ascii?Q?L2B0nHsd6o7521GggCA2WdIuSNJGPhmHcvWgkoSapj7ZVGrvLXVrBuVBnK1J?=
 =?us-ascii?Q?+KB4r4rgrnygLzNq+d5H7t6Jf+Io65GPIxH/IEw9SlhqClv9oeqPpADVxuJQ?=
 =?us-ascii?Q?rwQVDkDHOBhSBMilZPoR3psDdVC2jqoxHOGoxxcIUFOqfgNeFsWEaa5sTKpN?=
 =?us-ascii?Q?WBtT+R8GvM2pys17veTIOASvXVeUnljU6c27MVCcpXO+67ZhC4mOanTuGihN?=
 =?us-ascii?Q?/swqVsFiIbPQqkCcailxf90jkQo8rrIaHxJtGZHpF+WTZiinBFf1oNdUlf3/?=
 =?us-ascii?Q?jdXJ1why4S5e922CIiJqiFyLxOB49XltNKOE8HoqzqFFzeRZM3kSz1zoJVyY?=
 =?us-ascii?Q?JsCIAY7UWMtVp1s6dCZb0hs+npDxC96cLu0zJZAILD07tmNXsU/O/Zlw4M7n?=
 =?us-ascii?Q?TczeKIajVFU1CJ6MBHiXv3mB+38WmgCGHsC2dwcBnOgjpMe+CnXRpBCq6h3G?=
 =?us-ascii?Q?hz+spYy4WXL7UhuTLy/fC5xM1a0ukJflRz9n6B2jtTwLr86f+L8uDP6plLR4?=
 =?us-ascii?Q?p73OfBa7ZCrfU95EfUqUXaLPd6+ukXYfxC8DCluhzhj1nFPrIbcO/gZJppBA?=
 =?us-ascii?Q?CjblRrSWeJItZKRAfJjePleMUjk8uzhxVP0UqIMrTqx/lMKAQDJSfPuJ8EHs?=
 =?us-ascii?Q?bFzvEdgj+W13zRM0exrrP+1yuhwcEjXjNAQhxbWGv9cCQi0QeOFACzt/d+nk?=
 =?us-ascii?Q?xps=3D?=
Content-Type: multipart/alternative;
	boundary="_000_AM7PR03MB6593E7F0C691F98E07B8141C85D59AM7PR03MB6593eurp_"
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6593.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c48b36da-45f1-4dbf-0594-08d9739dacc1
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2021 14:25:29.4239
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: RRmEW+UpTEDAnDaBbT/Wo8IOE+zUnJtVj4qITsqX4FqwM/9DMCs1ka/swWMC8g05XMGi2HdQq2gmuJyvCzqGvQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB5959
X-Proofpoint-ORIG-GUID: BJZqtw0DytT34faNTsqi_o4Ai6Un9IWa
X-Proofpoint-GUID: BJZqtw0DytT34faNTsqi_o4Ai6Un9IWa
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-09_05,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0
 priorityscore=1501 bulkscore=0 mlxscore=0 clxscore=1011 phishscore=0
 malwarescore=0 adultscore=0 mlxlogscore=569 suspectscore=0 impostorscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109090088

--_000_AM7PR03MB6593E7F0C691F98E07B8141C85D59AM7PR03MB6593eurp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi Julien,

Thanks for the clarification!

I aim towards to prepare implementation for upstream to disable SWIOTLB for=
 IOMMU-protected devices in Dom0.
To provide this functionality need to add additional binding for each prote=
cted device in device-tree.
After this step, I will also prepare the patch to make ensure that ballooni=
ng code prepares all allocations below 4GB.

We are going to prepare this functionality only for device-tree based syste=
m configurations.
We don't have resources to support ACPI configuration.

Would you be ok with upstreaming only device-tree configuration?

Cheers,
Roman
________________________________
From: Julien Grall <julien@xen.org>
Sent: Wednesday, September 1, 2021 1:22 PM
To: Roman Skakun <Roman_Skakun@epam.com>; Stefano Stabellini <sstabellini@k=
ernel.org>
Cc: xen-devel@lists.xenproject.org <xen-devel@lists.xenproject.org>; Bertra=
nd Marquis <bertrand.marquis@arm.com>; Andrii Anisov <Andrii_Anisov@epam.co=
m>; Roman Skakun <rm.skakun@gmail.com>; Oleksandr Tyshchenko <Oleksandr_Tys=
hchenko@epam.com>
Subject: Re: Disable IOMMU in Dom0

Hi Roman

On 01/09/2021 10:59, Roman Skakun wrote:
>> If you have a setup  where Dom0 is not 1:1 mapped (which is not currentl=
y
>> possible with upstream  Xen but it is possible with cache coloring) and
>> uses the IOMMU to make  device DMA work like regular DomUs, then passing
>> XENFEAT_not_direct_mapped  to Linux would make it work. Without
>> XENFEAT_not_direct_mapped,  Linux would try to use swiotlb-xen which has
>> code that relies on  Linux being 1:1 mapped to work properly.
>
> I'm using 1:1 Dom0.
> According to your patch series, xen-swiotlb fops will be applied for all
> devices
> because XENFEAT_direct_mapped is active, as shown here:
> https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source=
/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFk=
gRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]
> <https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/sourc=
e/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeF=
kgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]>
>
> I agreed, that xen-swiotlb should work correctly, but in my case, I
> retrieved MFN here:
> https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source=
/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkg=
RQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]
> <https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/sourc=
e/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFk=
gRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]>
> which is greater than 32bit and xen-swiotlb tries to use bounce buffer
> as expected.
> It can lead to decrease a performance because I have a long buffer ~4MB.
>
> I thought, that we can disable swiotlb fops for devices which are
> controlled by IOMMU.

Yes you can disable swiotlb for devices which are controlled by the
IOMMU. But this will not make your problem disappear, it simply hides
until it bites you in a more subttle way.

 From what you wrote, you have a 32-bit DMA capable. So you always need
to have an address below 4GB. For foreign mapping, there is no guarantee
the Guest Physical Address will actually be below 4GB.

Today, the ballooning code only ask Linux to steal *a* RAM page for
mapping the foreign page. This may or may not be below 4GB depending on
how you assigned the RAM to dom0 (IIRC you had some RAM above 4GB).

But that's the current behavior. One of your work colleague is looking
at avoid to steal RAM page to avoid exhausting the memory. So foreign
mapping may end up to be a lot higher in memory.

IOW, you will need to be able to bounce the DMA buffer for your device.
If you want to avoid bouncing, the proper way would be to rework the
ballonning code so pages are allocated below 4GB.

Cheers,

--
Julien Grall

--_000_AM7PR03MB6593E7F0C691F98E07B8141C85D59AM7PR03MB6593eurp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size=
: 12pt; color: rgb(0, 0, 0);">
</div>
<div style=3D"box-sizing:border-box;font-family:&quot;Segoe UI&quot;, syste=
m-ui, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, sans-serif=
;font-size:14px">
<span style=3D"font-size:12pt"></span>Hi Julien,
<div><br>
</div>
<div>Thanks for the clarification!</div>
<div><br>
</div>
<div>I aim towards to prepare implementation for upstream to disable SWIOTL=
B for IOMMU-protected devices in Dom0.</div>
<div>To provide this functionality need to add additional binding for each =
protected device in device-tree.</div>
<div>After this step, I will also prepare the patch to make ensure that bal=
looning code prepares all allocations below 4GB.</div>
<div><br>
We are going to prepare this functionality only for device-tree based syste=
m configurations.</div>
<div>We don't have resources to support ACPI configuration.</div>
<div><br>
</div>
<div>Would you be ok with upstreaming only device-tree configuration?</div>
<div><br>
</div>
<div>Cheers,</div>
Roman
<div style=3D"font-size:12pt"></div>
</div>
<div id=3D"appendonsend"></div>
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Julien Grall &lt;juli=
en@xen.org&gt;<br>
<b>Sent:</b> Wednesday, September 1, 2021 1:22 PM<br>
<b>To:</b> Roman Skakun &lt;Roman_Skakun@epam.com&gt;; Stefano Stabellini &=
lt;sstabellini@kernel.org&gt;<br>
<b>Cc:</b> xen-devel@lists.xenproject.org &lt;xen-devel@lists.xenproject.or=
g&gt;; Bertrand Marquis &lt;bertrand.marquis@arm.com&gt;; Andrii Anisov &lt=
;Andrii_Anisov@epam.com&gt;; Roman Skakun &lt;rm.skakun@gmail.com&gt;; Olek=
sandr Tyshchenko &lt;Oleksandr_Tyshchenko@epam.com&gt;<br>
<b>Subject:</b> Re: Disable IOMMU in Dom0</font>
<div>&nbsp;</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">Hi Roman<br>
<br>
On 01/09/2021 10:59, Roman Skakun wrote:<br>
&gt;&gt; If you have a setup&nbsp; where Dom0 is not 1:1 mapped (which is n=
ot currently<br>
&gt;&gt; possible with upstream&nbsp; Xen but it is possible with cache col=
oring) and<br>
&gt;&gt; uses the IOMMU to make&nbsp; device DMA work like regular DomUs, t=
hen passing<br>
&gt;&gt; XENFEAT_not_direct_mapped&nbsp; to Linux would make it work. Witho=
ut<br>
&gt;&gt; XENFEAT_not_direct_mapped,&nbsp; Linux would try to use swiotlb-xe=
n which has<br>
&gt;&gt; code that relies on&nbsp; Linux being 1:1 mapped to work properly.=
<br>
&gt; <br>
&gt; I'm using 1:1 Dom0.<br>
&gt; According to your patch series, xen-swiotlb fops will be applied for a=
ll <br>
&gt; devices<br>
&gt; because XENFEAT_direct_mapped is active, as shown here:<br>
&gt; <a href=3D"https://urldefense.com/v3/__https://elixir.bootlin.com/linu=
x/v5.14/source/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxC=
bP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$">
https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/a=
rch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgR=
QbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$</a> [elixir[.]bootlin[.]com]
<br>
&gt; &lt;<a href=3D""></a>https://urldefense.com/v3/__https://elixir.bootli=
n.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUB=
PA!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixi=
r[.]bootlin[.]com]&gt;<br>
&gt; <br>
&gt; I agreed, that xen-swiotlb should work correctly, but in my case, I <b=
r>
&gt; retrieved MFN here:<br>
&gt; <a href=3D"https://urldefense.com/v3/__https://elixir.bootlin.com/linu=
x/v5.14/source/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCb=
P4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$">
https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/d=
rivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgRQ=
bKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$</a> [elixir[.]bootlin[.]com]
<br>
&gt; &lt;<a href=3D""></a>https://urldefense.com/v3/__https://elixir.bootli=
n.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBP=
A!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir=
[.]bootlin[.]com]&gt;<br>
&gt; which is greater than 32bit and xen-swiotlb tries to use bounce buffer=
 <br>
&gt; as expected.<br>
&gt; It can lead to decrease a performance because I have a long buffer ~4M=
B.<br>
&gt; <br>
&gt; I thought, that we can disable swiotlb fops for devices which are <br>
&gt; controlled by IOMMU.<br>
<br>
Yes you can disable swiotlb for devices which are controlled by the <br>
IOMMU. But this will not make your problem disappear, it simply hides <br>
until it bites you in a more subttle way.<br>
<br>
&nbsp;From what you wrote, you have a 32-bit DMA capable. So you always nee=
d <br>
to have an address below 4GB. For foreign mapping, there is no guarantee <b=
r>
the Guest Physical Address will actually be below 4GB.<br>
<br>
Today, the ballooning code only ask Linux to steal *a* RAM page for <br>
mapping the foreign page. This may or may not be below 4GB depending on <br=
>
how you assigned the RAM to dom0 (IIRC you had some RAM above 4GB).<br>
<br>
But that's the current behavior. One of your work colleague is looking <br>
at avoid to steal RAM page to avoid exhausting the memory. So foreign <br>
mapping may end up to be a lot higher in memory.<br>
<br>
IOW, you will need to be able to bounce the DMA buffer for your device. <br=
>
If you want to avoid bouncing, the proper way would be to rework the <br>
ballonning code so pages are allocated below 4GB.<br>
<br>
Cheers,<br>
<br>
-- <br>
Julien Grall<br>
</div>
</span></font></div>
</body>
</html>

--_000_AM7PR03MB6593E7F0C691F98E07B8141C85D59AM7PR03MB6593eurp_--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 14:33:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 14:33:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183300.331389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOL73-0007A6-7J; Thu, 09 Sep 2021 14:33:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183300.331389; Thu, 09 Sep 2021 14:33:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOL73-00079z-2P; Thu, 09 Sep 2021 14:33:13 +0000
Received: by outflank-mailman (input) for mailman id 183300;
 Thu, 09 Sep 2021 14:33:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o0l/=N7=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mOL72-00079t-6n
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 14:33:12 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 322fa9be-74c5-4c54-bbee-62c6766e70ab;
 Thu, 09 Sep 2021 14:33:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 322fa9be-74c5-4c54-bbee-62c6766e70ab
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631197990;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=lIpqQNuHljxSkJiImjDa5X3NwvmCumV1RL/ABnPfCXI=;
  b=YbRJFIJN4PNDJTbftlEcOy4qXH/RE56C0oGTSBHTQNxhl1O8+R99FFiT
   UNSmxhVBolJ6SMOfxDvwSN4utID9s9XC7mRcO0wMPhBH49T0fXO/yroFO
   rQr9VB2Dy0dzF5hSNbHouPN1oVAqhxeWiHbcCrEU5ZSlkNb2+dWouWeiw
   4=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 4kB1AnWwSaEYHskY/fY0NPHswqM0uKetiB0ZbOf360/YTe1Yr7VUUnZ6L7M2wTxDC9uy67kaNg
 ntQtZUTqQC0RTgU3r8ZdHaH1KyIIrwVEsN9PUhDrEn0KLJwgvt4S5nod9bDUMsZ00RFZPndNKv
 tnESlfetfWHYBI4Izcy+XfgqcPP2Z5COE/XweVVqM79pSUvkJMZJ8Ach9JkpPeww0cEUtULxs0
 Lh2bDKodQHMgdJvN1/DAHDXoOctqfvWaiV/TNXBnELOz5Ow4RFcrsxP1gqm+Bup0BsRdaKYbMi
 N0M2swF8eihbF99Ns7tP3gzD
X-SBRS: 5.1
X-MesageID: 54127042
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:bvBu56hNQcBlz1PJdwDU51WJR3BQXiAji2hC6mlwRA09TyX5ra
 2TdTogtSMc6QxhPE3I/OrrBEDuexzhHPJOj7X5Xo3SOTUO2lHYT72KhLGKq1Hd8kXFndK1vp
 0QEZSWZueQMbB75/yKnTVREbwbsaW6GHbDv5ag859vJzsaFZ2J921Ce2Gm+tUdfng8OXI+fq
 DsgPZvln6bVlk8SN+0PXUBV/irnaywqHq3CSR2fiLO8WO1/EuV1II=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="54127042"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Anthony PERARD <anthony.perard@citrix.com>, "Ian
 Jackson" <iwj@xenproject.org>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>, "Daniel P.
 Smith" <dpsmith@apertussolutions.com>
Subject: [XEN PATCH v2] build: add --full to version.sh to guess $(XEN_FULLVERSION)
Date: Thu, 9 Sep 2021 15:33:06 +0100
Message-ID: <20210909143306.466836-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Running $(MAKE) like that in a $(shell ) while parsing the Makefile
doesn't work reliably. In some case, make will complain with
"jobserver unavailable: using -j1.  Add '+' to parent make rule.".
Also, it isn't possible to distinguish between the output produced by
the target "xenversion" and `make`'s own output.

Instead of running make, this patch "improve" `version.sh` to try to
guess the output of `make xenversion`.

In order to have version.sh works in more scenario, it will use
XEN_EXTRAVERSION and XEN_VENDORVERSION from the environment when
present. As for the cases were those two variables are overridden by a
make command line arguments, we export them when invoking version.sh
via a new $(XEN_FULLVERSION) macro.

That should hopefully get us to having ./version.sh returning the same
value that `make xenversion` would.

This fix GitLab CI's build job "debian-unstable-gcc-arm64".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
v2:
- use $(SHELL) to execute ./version.sh
- get XEN_EXTRAVERSION from the environment if exist
- use XEN_VENDORVERSION when exist in the environment
- introduce a new macro $(XEN_FULLVERSION) whose jobs is to export
  $(XEN_EXTRAVERSION) and $(XEN_VENDORVERSION) in cases where those
  are overridden by command line argument of make.
  (we can't just use make's "export" because it doesn't work when
  parsing the makefile which is when $(shell ) get's executed for
  POLICY_FILENAME, so it wouldn't work when running `make -C
  tools/flask/policy XEN_VENDORVERSION=-os`.)
---
 tools/Rules.mk                     |  5 +++++
 tools/flask/policy/Makefile.common |  2 +-
 version.sh                         | 18 +++++++++++++++++-
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/tools/Rules.mk b/tools/Rules.mk
index 2907ed2d3972..b022da3336c4 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -6,6 +6,11 @@ all:
 -include $(XEN_ROOT)/config/Tools.mk
 include $(XEN_ROOT)/Config.mk
 
+XEN_FULLVERSION=$(shell env \
+    XEN_EXTRAVERSION=$(XEN_EXTRAVERSION) \
+    XEN_VENDORVERSION=$(XEN_VENDORVERSION) \
+    $(SHELL) $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)
+
 export _INSTALL := $(INSTALL)
 INSTALL = $(XEN_ROOT)/tools/cross-install
 
diff --git a/tools/flask/policy/Makefile.common b/tools/flask/policy/Makefile.common
index bea5ba4b6a40..e5ed58200e75 100644
--- a/tools/flask/policy/Makefile.common
+++ b/tools/flask/policy/Makefile.common
@@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER)
 #
 ########################################
 
-POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory)
+POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
 POLICY_LOADPATH = /boot
 
 # List of policy versions supported by the hypervisor
diff --git a/version.sh b/version.sh
index e894ee7e0469..c6a5692c1982 100755
--- a/version.sh
+++ b/version.sh
@@ -1,5 +1,21 @@
 #!/bin/sh
 
+opt_full=false
+while [ $# -gt 1 ]; do
+    case "$1" in
+        --full) opt_full=true ;;
+        *) break ;;
+    esac
+    shift
+done
+
 MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "`
 MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "`
-printf "%d.%d" $MAJOR $MINOR
+
+if $opt_full; then
+    extraversion=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
+    : ${XEN_EXTRAVERSION:=${extraversion}${XEN_VENDORVERSION}}
+else
+    unset XEN_EXTRAVERSION
+fi
+printf "%d.%d%s" $MAJOR $MINOR $XEN_EXTRAVERSION
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 14:36:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 14:36:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183307.331400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOL9j-0007mQ-Iy; Thu, 09 Sep 2021 14:35:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183307.331400; Thu, 09 Sep 2021 14:35:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOL9j-0007mJ-Fz; Thu, 09 Sep 2021 14:35:59 +0000
Received: by outflank-mailman (input) for mailman id 183307;
 Thu, 09 Sep 2021 14:35:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOL9h-0007mB-PX
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 14:35:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3db65096-117b-11ec-b1bc-12813bfff9fa;
 Thu, 09 Sep 2021 14:35:56 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2051.outbound.protection.outlook.com [104.47.9.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-79raWJstPiumGt3O_JwzJQ-1; Thu, 09 Sep 2021 16:35:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5597.eurprd04.prod.outlook.com (2603:10a6:803:dd::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 14:35:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 14:35:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0018.eurprd09.prod.outlook.com (2603:10a6:101:16::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 14:35:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3db65096-117b-11ec-b1bc-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631198155;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=fOUx97FiXhN5eIKucfl+62XT9uRezxxwSDnxXO8LNd8=;
	b=R1ybchOqN7R1Dxclv+DQwC7VN7/8yPbitLvm7bYTK5prnIbhu8CKFBUHaPGMa4yLbiwSqH
	NaXLftVGNUstpz2bFXT6UTAtZbC4SaocmoKCrHn9pfmjIyjWnhQEACAQc+PNqq/JJ9QgwO
	DYkf4u1Fr6VaKaf+mYNJrZ5VPEpPKEc=
X-MC-Unique: 79raWJstPiumGt3O_JwzJQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CdZNvtgxFI4fIdsRS2/wX+cISGwbBogtShwRqR04VcDK6p8Qtk6ipcNnbVTu5YCA5CadvSUvaH8Qkjo3Mb/Ay0G8VsoRhRuhjklwks+LGlLaVtkTw+CMbMxlZWfczXcjWpbwUdcv2rrmPmEDAa1uP6t5XBPMIyNHIB148P0bpX8ZOGZWFVeGDaybz9Baq8ohFDbV6U6moyEi6VoG/83cIZOS4+e0y9jzI8GQpnjbyGcAmi8MoU+LFmSCAFo38BziN0qa+t7awO9RDfi7a6e3Z3TtpoIXknL0/ISFFeX40xgsypotRxn1ExuZ7gwXtuFNnDlVsGK0pNC4AH6ZopsasQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fOUx97FiXhN5eIKucfl+62XT9uRezxxwSDnxXO8LNd8=;
 b=JebL5qhYiStRF/sboN+KVW4pxu7mQxumZ9jg7za15T9gDsAVDM+PbdG8ayCZqUw0eR0Nr758uFWjl2AU+QIX1+zyLlrbyB3IJgdvZAI7L6Q4/LZU/DG5sLrTtWJ5DDVjRWpXAaK4DzqUU1hAp//nXZ1Ibp2WZ7n6xoUT6XiwGNY4TIIu1dcwpIICkwCXaBTBrRXClcKGEKAB0kk9XxiJur6QblKLcWucmRwviBrCLm845JcuV5/azGv/AutEq/Y1QozjygBlVyWyByI0xYzDQLLlxVOZVeAdH2qJnun7AwQHhV21QaiYqW/zUWCWPn7BQP+VlQ8TOL9JqJcsoazLVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86: conditionalize workaround for build issue with GNU ld
 2.37
Message-ID: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
Date: Thu, 9 Sep 2021 16:35:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0018.eurprd09.prod.outlook.com
 (2603:10a6:101:16::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b31ac3c4-5d69-4443-2abf-08d9739f1ff0
X-MS-TrafficTypeDiagnostic: VI1PR04MB5597:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5597805D5EFF38EB7F0A47C3B3D59@VI1PR04MB5597.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aBbfSbO7cQpCOGu4yzY5Nl+tuZ+2aNaUqTl+r9wDxGcjUPf4si3AMA+Nv/51cqWd7RpvmrboAVHU9eleULgkN8u/bWbYpEr7o04U6qgABT4e2+8UcZvNJy6nBr47/8pG1jUQw3ER4Wlpxmzh9egQKx8mqdkizCrG7ZMuASn8PTpaH7Obt6CuiNa6QTGnmPV5m+Usz+iBUIxKyLWPieBu7H09VnT/Xnl+fObMLxVS9pPWJs3GrOkxrrkqwXT2LC83Tv4ApoIhsIE7MKdpQCiyQplp56X25o/Zd3w4dzu5HIaMJUWu90efOdorukoTPdNgG/ic+vxATf6+VrX1n+DvnBcJXrpsUYPt9tih7IlWmVt2kzXtPGSKgrq0rtF7FP1HjaDu1rj/zOx9eibGcvbYMdop75T7ZvflHW86/xrT5i55ZKGNYI5D/DrF4/p5AmenLXiHjsapidBvUG9NdUw4wPLbqrjy5ceyVC538XdAbfvwNerlnvknH8ZWQOXa29sXqkcEVTvxYcqUc5zFQpLuuVb/08oWYoFm2hVtcAJMLz3pshUJVSHDsIPFUVhe8eElmXhvA30XcHxCOz/sV+eEekzrC8XA41mIkhWLMHg7Vo+28juv63zApmuzUwdJ34/IfuZEWJf84gvO4XfII/d8LpbNWIJPGTMZCh4pBsIr4xrEjKeekrQGYKQnFjzEeEmjxMUypTVdYtx7jDXUTyE9kJq6j8NZ6i2xjKAD40zSV4zfHY8EwlHzGkwS69K221nbBoMebWTModXcNdog9H+nyOpkqDqc8eIE1i82b/DVqHX7jcE8SWisSZJWSgWNs0IX
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(346002)(366004)(136003)(396003)(6486002)(66476007)(478600001)(8676002)(66946007)(66556008)(16576012)(186003)(26005)(5660300002)(2906002)(86362001)(966005)(6916009)(36756003)(956004)(2616005)(54906003)(31696002)(31686004)(4326008)(38100700002)(316002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZmVhaGU4aXNEenBwdXRueEFVbWdnbVpPYUtJdmhXU0Jpb0paeHhjYTE4bWlE?=
 =?utf-8?B?ZGF1VEdTbEJoT2Y2YWNUYUlhZjlUdUZJcXQyMUwwaEh1LzIyWElLemV2d1VT?=
 =?utf-8?B?RFk0MVcwSU8zM1JaSEMyMlRvaHJDeU9kWHNLVmtrM2lvUG9DWVNLVWUwN0or?=
 =?utf-8?B?OURKT1J3Z3prb2ZmaTRQc2xnSzJDNkp0eGxXMmViV2h1TlpueUZZcXlCUW9Q?=
 =?utf-8?B?L3c0L01DVTFpalI3WjB5RDhMZExOTnJUT1FqMXBBYnJ2UlpHL0ZvODF5VytU?=
 =?utf-8?B?aVA4Vm84YTRsRVpWVU5OMlozVGNOZlBrU2dFQ1BHZ0grWG5reCs3bGttSG9E?=
 =?utf-8?B?Z2V6OVBsbVkxdTZGUG5qdnFjNW9CTE9McEZ6L2EwVGU4clZyN3p2d3VDZmdO?=
 =?utf-8?B?VEFnUWVlWGs0c0NiRHFrVHRSdWFCRGgrUHltOVI4SWJlNXNNWUt6YjdLR0Qy?=
 =?utf-8?B?UjBlS05yTDc2a3RJSzJ1UlRoNzRpZXB1UmVQa05EL3BrdGdJM2Frc1ZkVThi?=
 =?utf-8?B?TGgvbnRBM2NmNmxQT1VsNHRVNzBjUjF0emVsenpNaXcxK3dBVkp0M2MzNUNv?=
 =?utf-8?B?YUl1NVlUZW9QS0JkSkZ3N2lqTkROSVNhaWZsWWhZbHd1cVlXY2NLdCs0WG40?=
 =?utf-8?B?b2E0L2Z2ams3SlZjVXZmbThON05vQXJpZy9WUm8yeEI3aHR3WUVCWWVzaGJr?=
 =?utf-8?B?MEU3eEdWOGFiZ1FsM3M5VTZWcDY5OG5VOGkzeDJRL1o5NFhwRi9rbk9pUDdo?=
 =?utf-8?B?enJDNUY2RmUrMHpsUk94UTkwYzRIWGt6SjRiaHhINUd4YnkwVkNuYkkySDJm?=
 =?utf-8?B?ZGQwbERUdnJuVit2c0pEblJKdGlORU9OQmYwYVk5d29MalV6dnQwV2ZJZzc4?=
 =?utf-8?B?c3RKRTRBVTNaTkVmZG00bWk2L0pWbTNLeXNFWEhaQjZnbU1DL3JQOWlvOEIw?=
 =?utf-8?B?ajg3OCtyVUNvR2FHd2JWdVFCY0VjQ1huZ3gxK2xnRlBPQlh5NWFidWg5QmtO?=
 =?utf-8?B?S0hneithcjNaUm1pUGRmbkhBTUxqV1kyUGE4YjliRE9EUHBSNVB1WDQ2OXdK?=
 =?utf-8?B?YU9TOXNoQ1lQTzJDN1JMMlBTd3dCOFdVbDJ5YjBUdlFaS040dWM5ZWRJYm9p?=
 =?utf-8?B?dkpsTzRLZW96V1dLNGMrODNLLzVpT1dmb1JtQzlqTXJmR3NHTFlwNUE2Vkxm?=
 =?utf-8?B?VUJvY3ROd1dsZGEvcDMwcGd0bzdVK2IxMWRZOTlMaVZxemxudXRHRm1tUDR3?=
 =?utf-8?B?amhaZGNkQVdLcDlEaDREd3RNM25tYUpKQUZtR0I4clRtVHRhQ1hra1JBaGow?=
 =?utf-8?B?Y1NXSEF1VndLZFBmZzM3ZUR2cDQzOWovRVBqSlBYL01PYmdTNWZ6MmExZ3Y0?=
 =?utf-8?B?eUNOcDlsUTZkL3gxamdBWE9UZlpVVFdpMkQrejFtazh3Sy9tMG85d1lReGJn?=
 =?utf-8?B?ZzlmeWlmTGpNQ25rZnVaRTA5bHJ5THpMbVNLNDBGZmN4amszNy82ZTh1RGdt?=
 =?utf-8?B?aEJER3lKYk5YbzB2dmJRRW9TeEE0SHRJNVYyV3BoMWlQcjkvZUo2UWJQZ2hy?=
 =?utf-8?B?Ykp4cEw3RnFZelRhUDJjNlpha3lqZldJaVpTaHJld2ZvanorYXhobEV5SDhw?=
 =?utf-8?B?QjMyM29mZzhFdHpoTXlaN0NmUHNON3BWTE8xNHYySEo1MzY3UzU0RVZJczdw?=
 =?utf-8?B?YjFlOTdZeG5UbTdCK0JrUXR3S29LVk1QcGtRd2FENVlPSWhlN1ZFZUhncG8v?=
 =?utf-8?Q?WbrPWZ6VoSNZt0UFqpF3I9bDJLvStg1muGaKhej?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b31ac3c4-5d69-4443-2abf-08d9739f1ff0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 14:35:52.3622
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M5A2sUbEu4OWc8q5mK+IGE5dWZMlnU+L5w10jE+XlE980RQtrUbo7r+f8Ya32zWZ/eGUrTx7g/mVK2Yv3RUD7A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5597

While LLVM's lld is supposed to be a drop-in replacement for GNU ld [1],
it appears to not understand quoted section names as operands to e.g.
ADDR(). Therefore the original workaround broke the build in
environments where ld is actually LLVM's, like on FreeBSD.

Fixes: 58ad654ebce7 ("x86: work around build issue with GNU ld 2.37")
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

[1] https://lld.llvm.org/
---
I haven't been able to find an environment where I could actually try
with lld (ld.lld); all testing was with GNU ld (ld.bfd).

--- a/.gitignore
+++ b/.gitignore
@@ -306,6 +306,7 @@
 xen/.config.old
 xen/.xen.elf32
 xen/System.map
+xen/arch/x86/.check.*
 xen/arch/x86/asm-macros.i
 xen/arch/x86/boot/mkelf32
 xen/arch/x86/boot/cmdline.S
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
 
 ifneq ($(build_id_linker),)
 notes_phdrs = --notes
+# Determine whether to engage a workaround for GNU ld 2.37.
+build-id-ld-good = $(shell echo 'void test(void) {}' \
+                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - 2>.check.err \
+                           && $(LD) -T check.lds -o .check.elf .check.o 2>>.check.err \
+                           && echo y)
 else
 ifeq ($(CONFIG_PVH_GUEST),y)
 notes_phdrs = --notes
 endif
+build-id-ld-good := y
 endif
 
 ifdef CONFIG_LIVEPATCH
@@ -291,6 +297,10 @@ $(BASEDIR)/include/asm-x86/asm-macros.h:
 	$(call move-if-changed,$@.new,$@)
 
 efi.lds: AFLAGS-y += -DEFI
+xen.lds: Makefile
+ifneq ($(build-id-ld-good),y)
+xen.lds: CFLAGS-y += -DQUOTE_SECTION_NAMES
+endif
 xen.lds efi.lds: xen.lds.S
 	$(CPP) -P $(call cpp_flags,$(a_flags)) -MQ $@ -o $@ $<
 
@@ -302,7 +312,7 @@ hweight.o: CFLAGS-y += $(foreach reg,cx
 
 .PHONY: clean
 clean::
-	rm -f *.lds *.new boot/*.o boot/*~ boot/core boot/mkelf32
+	rm -f ???.lds *.new .check.* boot/*.o boot/*~ boot/core boot/mkelf32
 	rm -f asm-macros.i $(BASEDIR)/include/asm-x86/asm-macros.*
 	rm -f $(BASEDIR)/.xen-syms.[0-9]* boot/.*.d $(BASEDIR)/.xen.elf32
 	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
--- /dev/null
+++ b/xen/arch/x86/check.lds
@@ -0,0 +1,11 @@
+/*
+ * Minimal linker script used to check whether the linker understands section
+ * names containing a dash (like in .note.gnu.build-id) inside ADDR() and alike
+ * without quoting them.
+ */
+OUTPUT_FORMAT("elf64-x86-64")
+OUTPUT_ARCH("i386:x86-64")
+SECTIONS
+{
+ .text-1 : AT(ADDR(.text-1)) { *(.text) }
+}
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -18,7 +18,12 @@ ENTRY(efi_start)
 #else /* !EFI */
 
 #define FORMAT "elf64-x86-64"
+
+#if defined(QUOTE_SECTION_NAMES) /* GNU ld 2.37 workaround */
 #define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
+#else
+#define DECL_SECTION(x) x : AT(ADDR(x) - __XEN_VIRT_START)
+#endif
 
 ENTRY(start_pa)
 



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 14:55:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 14:55:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183315.331411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOLSq-0002HF-Bd; Thu, 09 Sep 2021 14:55:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183315.331411; Thu, 09 Sep 2021 14:55:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOLSq-0002H8-8P; Thu, 09 Sep 2021 14:55:44 +0000
Received: by outflank-mailman (input) for mailman id 183315;
 Thu, 09 Sep 2021 14:55:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOLSo-0002Gy-UZ; Thu, 09 Sep 2021 14:55:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOLSo-0008Gt-O8; Thu, 09 Sep 2021 14:55:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOLSo-0001qz-CZ; Thu, 09 Sep 2021 14:55:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOLSo-0005Uh-C3; Thu, 09 Sep 2021 14:55:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sm208tfSgtLw/6MtAsjHzNpyRjsO9wmXIZtAlRdCzJs=; b=ohblCUoRai61dX3J0G5elXDFfF
	XTOd221djzhHlSezsICo+ueKBHvzBnYTr1QEr9cegWeCEAwm0iw1O9BKSTUwQk/lcyWXKgFpBPfCp
	BmciIVeOwmhXjLY0eL2cmtWlZMYhJrL9aTUgdzWl/5Sp6/CBF3iVM6LLbFMTKXhZ0+7c=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164890-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164890: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-pvshim:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ac08b1c68d1b1ed3cebb218fc3ea2c07484eb07d
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 14:55:42 +0000

flight 164890 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164890/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot         fail in 164879 pass in 164890
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10     fail pass in 164879
 test-amd64-amd64-xl-pvshim   22 guest-start/debian.repeat  fail pass in 164879

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                ac08b1c68d1b1ed3cebb218fc3ea2c07484eb07d
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  404 days
Failing since        152366  2020-08-01 20:49:34 Z  403 days  702 attempts
Testing same since   164879  2021-09-08 04:07:16 Z    1 days    2 attempts

------------------------------------------------------------
7251 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2234642 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 14:56:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 14:56:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183322.331424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOLTy-0002qr-O2; Thu, 09 Sep 2021 14:56:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183322.331424; Thu, 09 Sep 2021 14:56:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOLTy-0002qk-Kz; Thu, 09 Sep 2021 14:56:54 +0000
Received: by outflank-mailman (input) for mailman id 183322;
 Thu, 09 Sep 2021 14:56:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOLTx-0002qc-Ec
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 14:56:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d2222d6c-c232-4888-be03-d4703148f5af;
 Thu, 09 Sep 2021 14:56:52 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2057.outbound.protection.outlook.com [104.47.6.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18--BbNBVHTO7eMSYyU5GdQqg-1; Thu, 09 Sep 2021 16:56:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 14:56:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 14:56:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0213.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 14:56:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2222d6c-c232-4888-be03-d4703148f5af
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631199411;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=MQGjhvRmw+/DydvCB7DT+8K9D0V+T+CrHAAC2eFUce4=;
	b=gcRo7mTja9lIro+PZOBYEpOKcyj0BuqoK8janPP3TWgFBNj4NDiimYVf41GFhNhemQ6C5S
	5niYCtRnWxz4JH4E++mlQbTrRky+2nx3RNl93tqUqhOx/4abgpAkttbfHVDwkbwSM/ldbJ
	p1wKs2PFGgTMFnbuEJXhxMqisIm3aWM=
X-MC-Unique: -BbNBVHTO7eMSYyU5GdQqg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iVC/sKZRrV6eljHWC+2OsplFxjOmhizqj88M6VidgFCynh/t0CPQmMCJ+6JYnPqgboD3d7D1jHfEy+17Ss8/MJWt7IQ2xcCIDiX6hwMd/RiUBjLzR0rv/49XyT2JMSM0K+0WSVb87jB+22koKE2fo0iyiG61Sw1txZHpPtKrFDALU87DTCkIp8EhLFIQL1AsKayb+QcuV8CLcLEihgObh33CAJsJu9NjLKbbx07QgtTEixsGWoloRLy2Qx/Igkg4g1QxHvbpneZFQGGLDB1Krkqs3UKfdDVQPQTkDkDZOUM75wgwr0Qnp0zu6TZImig6TRXXXlWbpb6L7J1KEog4bA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MQGjhvRmw+/DydvCB7DT+8K9D0V+T+CrHAAC2eFUce4=;
 b=OUMr46FNoLsuwW0uEstPr9F3E3Aj9DZ8hntFl6ULQB2W+8jqfvO+LSgqjFYsy/mtFfvq4Kl8N2AHrNAcpLrkDEHZdi3F1eAuIzgPb12nia8n4HXbTKjqKLjfYGAR96QznUXcKDUKw03ZCRnh+ICtZWzrKNmc9lSPIhS6J7vLrQFMKvE/PLsheVpdOeL0A6IueK14DP+irL0vG8HEr+N//d34lXvaKPwrebnur8Ealkan4GU3p4BOoQHNqniPnhgiXseYk4HYuYQsRNEeuI+3dRL1JUsY2P/DuUT9Byilc18ND94lheiQYl5Wwvq/5/lXi8R4Csy3VVV+zt6FoqJkZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86: correct asm() constraints when dealing with immediate
 selector values
Message-ID: <87278164-2395-1fb4-7569-9151d0151e8a@suse.com>
Date: Thu, 9 Sep 2021 16:56:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0213.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e703931b-0c66-4282-9691-08d973a20d3c
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6861829793C22BAEE67FBBB9B3D59@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IVI+AuAVHz8676GPm3n/8RuMwdCMrx+4DB7ye7cNGJ+RAZnk3/aYTomV6yWYEtYbbtQZ+sE1aZl9RQrlJOSgKk5I78lHa4FLE8yBwaU4G/CNonwKZbMU2yAJXu2k2hOayjGEWWy1A8NeYqW/sNJ0lb9GSMPQNzJkoCo1eDuvFWtVM2hrhI+duTB8VOmyXv9Ut4OwqOChImlLs2aH0+++1K+iWWK0jTBW0kYNajS1LWTgRvpSh9o+CYeA6Jqv3xN2lsoxKZ9VFQumdmHuTI2twBFt+U254KLV2VET8AsLBBejA+1hkdq2PQm2DQ33NXXk9rb2o34pwPctVZtjhV0IdUCbBvXBkhotjAEc9KowkTKLUS/IVe26pu8DRB4f5tFALC6FK0M9IU2x3so/wpTSV50pDT/1FuhmJ3QUXmwpg7oQhGXDud05OXpSkdmAbWUdISyxhZVCrr/zNWBkvmEvvT5pZzrDv4u23TMZHgNiILG5faBc2+/b4GgCB8+yMLt00nFiNy8ZkXPtsfkxiQn5ea2utJzjHdLeBJfhfRYayCLCWTeH5OSjkG1IByka05WU59rdmZhpYVMrXzvXrEwXRRpZ+SyrDJ0+6lLe75k5udnzSsjhAIdqZQi62w8vPOWkU1ykwKnaJtXb0YIOcWlBG7j0o41GQvfEPK0DW1vvCLi1sTDd4l07ZgSQS4OLoAo1UdEhN/YK35e1KQd2wp80ZE+r9e0eGJh9hwcfejEzdTw5cqanLLl+rN8Wf3S2s/hban+nNIWW7Zl1nC7n6kb3uw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(136003)(366004)(346002)(39860400002)(376002)(186003)(86362001)(31686004)(26005)(6916009)(83380400001)(66946007)(66556008)(6666004)(316002)(4326008)(31696002)(66476007)(8676002)(54906003)(956004)(6486002)(16576012)(478600001)(2616005)(36756003)(8936002)(2906002)(38100700002)(5660300002)(14773001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MExJa2U5UllGa1dJdDI4MmJSbzV0ZlZydlNIdEl1NXdhTU5wRnhxbGp6MmJU?=
 =?utf-8?B?VXVrdGFJWnNFMnhtYlVXd1pHQkROVlpJdlR4cGNiZ2dZalo5R1pXUnpOT0F6?=
 =?utf-8?B?MUM4VTlDVnBQMXhGb3ltL2FOZEx0V0lRU3dwTXZxdUp2RXJqTytibUx3TXlk?=
 =?utf-8?B?N2hqSGJ1KzVHdEhOdjdSRk5VMnBXQVFIMkR3cjlPaE1aWFloRGVvZ0lKeHph?=
 =?utf-8?B?bFR2Y004Yk1RemRNWjBFZ1VsZDVyamkvbit0bDhjbnBlcklDZVVxc0Y2VENv?=
 =?utf-8?B?cC9ady9WaHNNc0NYVktXakxiUDQvbDZnb2F3T25IbWxpWTRITkU0MDUyTVZE?=
 =?utf-8?B?bVMrWXBFcUxnektMM25XeHc3a0tjdG1VOFgvUVVYbWJsU1pxdlhzc2U0TU9u?=
 =?utf-8?B?NGQ1Rm1EdHQwQ2dvZlM2ejVYemtKcmhTcStWeXRUWGlaRHNzMDNBS1diWGdI?=
 =?utf-8?B?VktHS3N0TjBWbDRpQktScmQ0M3NmRVhaYlZUR0dIdFJZaGZLbkYwcTFUYzVW?=
 =?utf-8?B?bFUza05BbmV1NWRIZDlHVkRNL0FpSUFpWFdFM1lHMndkdlpaVzFUK2RNdU15?=
 =?utf-8?B?dkdMZlI0N3NnN1B5bTJJZW9jR3YyUzZFZXMwNkdtdDBHRGpkSStSOXY4b3Rp?=
 =?utf-8?B?RkcwKzNJbmVxNG5UWWFyRjI0bE9rOGVGWXNOaVJDM0V5bm5QQ0g5RkxVdS9z?=
 =?utf-8?B?TVFIN1ZIdUFTMzFpZmtxRmQ0R0cvSzA0T0hodWlrV0RsOGsveCswZkc1WWFs?=
 =?utf-8?B?WVZPelF4Y2hMM2RUYmI5Ymt3ZzJCK1RHQWhVeW9Ic3NYQ29nZ1VMM1Jka0tp?=
 =?utf-8?B?UnJ3ZVh6Z2lDWHd6UUtRVHlKVzZCSUZsMzZvc0wrczBIMFVXM2lySFFVbExo?=
 =?utf-8?B?K08rT1lNUDRON1BCWm90SEZtSFB5TURqZ0RuK09EQjA1TFZKSFpaNHJHb0pv?=
 =?utf-8?B?UndnVlJ5U1dEUG51MlNlcTg4RENROStSSnVvb1RFSUV6UnVpVkF0UnpyMmdx?=
 =?utf-8?B?cXBoZzJ1Q25pTDVrY1c2aHBPR25DYU8yMGlYT2QrZ3Z0OThndDh3QnBjZGhZ?=
 =?utf-8?B?eUg5ZWFYR0E1NWxITTVuMERWazMvR2d5YlhOZ2pwb2R1U0JRMzRpNkJUWW1H?=
 =?utf-8?B?d2I3LzhjSHRuaEtKemtlbVFNQm9oN0xYSGlOUE5GNUVDa242ZFdzNlkyc29o?=
 =?utf-8?B?NTRLeG9NWkxLc0w5OWdvMnB2TnFjUnRTbDdSbHcrSXI3NnpadDBtVThLVU5h?=
 =?utf-8?B?WnlNVHB0clgwWWMzbE1TdFdhM3B2Rk4zTnArcUF0ZVVGU0x2YWJNVDlrSHNE?=
 =?utf-8?B?d0FUNUFkaEZWZklFd1ZRSm05NGpIQjhDeVRjVkY3akxyYWxCOFhVZE9TeTQ1?=
 =?utf-8?B?RDZ3SU1pYjBtUS9qME02WHpwR29SZ0dWeDBUWm5FQmZ3UEcvd3NMcDF1VUds?=
 =?utf-8?B?S3JiV1ROdXZpMElRb2xQQ0huOWNKM2cvZ0NQVG1sMFdnSmFSMTJyUmtIN1V0?=
 =?utf-8?B?empIR2dWbU51aFdweGJqbXd3czhkYWQvYUhjM3loVFdRYzBZK2FLTmgzWjVq?=
 =?utf-8?B?TUJ6WTlyVFZUb241dmIyU2RsU2N6V1JPbTdIc3dwRVZGVWtTN2krMjNsNThn?=
 =?utf-8?B?YjN0WENCMDlVSVdvblA2dDBDYUhManFxL0hHZm5COEYzTXZrdExqUkc0bUwx?=
 =?utf-8?B?TkxuVUJCNkRqNHR2NGQyR2NEYVdGcStNYnpibStwQVpNaEtsQm1pQzlsQW9Y?=
 =?utf-8?Q?WdMCXuaaQOZretgrwqIixsXTioWT1A6E94tv2gL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e703931b-0c66-4282-9691-08d973a20d3c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 14:56:49.5385
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 93HMGiRI/rspO8zkSs4plFO+/DxMNHDcxm+4a7AXXkBIWPuvIdD5pDTYfwRLzZqiZjlZA0YfCODyvrJxZkSPFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

asm() constraints need to fit both the intended insn(s) which the
respective operands are going to be used with as well as the actual kind
of value specified. "m" (alone) together with a constant, however, leads
to gcc saying

error: memory input <N> is not directly addressable

while clang complains

error: invalid lvalue in asm input for constraint 'm'

And rightly so - in order to access a memory operand, an address needs
to be specified to the insn. In some cases it might be possible for a
compiler to synthesize a memory operand holding the requested constant,
but I think any solution there would have sharp edges.

If "m" alone doesn't work with constants, it is at best pointless (and
perhaps misleading or even risky - the compiler might decide to actually
pick "m" and not try very hard to find a suitable register) to specify
it alongside "r".

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -736,7 +736,7 @@ void __init detect_zen2_null_seg_behavio
 	uint64_t base;
 
 	wrmsrl(MSR_FS_BASE, 1);
-	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
+	asm volatile ( "mov %0, %%fs" :: "r" (0) );
 	rdmsrl(MSR_FS_BASE, base);
 
 	if (base == 0)
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -248,7 +248,7 @@ void do_double_fault(struct cpu_user_reg
 
     console_force_unlock();
 
-    asm ( "lsll %1, %0" : "=r" (cpu) : "rm" (PER_CPU_SELECTOR) );
+    asm ( "lsll %1, %0" : "=r" (cpu) : "r" (PER_CPU_SELECTOR) );
 
     /* Find information saved during fault and dump it to the console. */
     printk("*** DOUBLE FAULT ***\n");



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 15:35:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 15:35:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183332.331439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOM5N-00083g-TH; Thu, 09 Sep 2021 15:35:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183332.331439; Thu, 09 Sep 2021 15:35:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOM5N-00083Z-Pi; Thu, 09 Sep 2021 15:35:33 +0000
Received: by outflank-mailman (input) for mailman id 183332;
 Thu, 09 Sep 2021 15:35:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOM5M-00083T-68
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 15:35:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7bc137ec-1b6a-458d-8af5-adba222f2778;
 Thu, 09 Sep 2021 15:35:30 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2051.outbound.protection.outlook.com [104.47.6.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-Vjc0mHLZPvumYYTrEfKQ5A-1; Thu, 09 Sep 2021 17:35:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6478.eurprd04.prod.outlook.com (2603:10a6:803:12a::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Thu, 9 Sep
 2021 15:35:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 15:35:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P251CA0001.EURP251.PROD.OUTLOOK.COM (2603:10a6:102:b5::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 15:35:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7bc137ec-1b6a-458d-8af5-adba222f2778
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631201729;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OaZlGOQVIu/SHCco4zvHNMIsRoe3fyRLX5UBV2NO/tU=;
	b=MsWf7tbEk/A64a3W4w0uI8xDq14OQuDQw4u7c7wWDWaNVzM86PMnVSkSAEoOFkIjBI93Al
	wPh9KjOZi/4ug0QWdYD00A+EUnMt/dAjeDvpXf0zvewIKp6Jbqc8addj2XrT2IDUwlFN+r
	ufOkyIrMe5usGZTMq8e4CnzC7icJHt4=
X-MC-Unique: Vjc0mHLZPvumYYTrEfKQ5A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mDFT9ujSQSI5/b/T2MG8G1lxxDyzeZ2U15vSH3q30Cf4sjpudrWxv0OPHO4B13zg/kwb/4Yt7CVk/ep7JHPCdEpYzAH6hCwVA7UbboY2fSJKLMDIZGoBtvos1suKFNZqh4zIwa2/DWPIn4mRxdY+iQKPZaxj7XZGeXyPd4hyb9gtMsJafv2TEtsetcaCWX7G7OAOupsckdCpnVfSIzo7sNp0BFxoauRoSXIzAv88ftXnWqwz1bVkrA9BBSCM8kXxppjbtmYshFBwfTFN7XgsMFGyLkMpM4U48anW3+p5pmTTeSC4cFyCGn3Cj/AgtC7k/V7Zx2lc+ZJjHoBrtto7cQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OaZlGOQVIu/SHCco4zvHNMIsRoe3fyRLX5UBV2NO/tU=;
 b=gYL3fQ2Th0HY6aq1e1p4le45VFsKUd4QNZKHqLDX92kUpzXbZB1eTHDyWp22sMLiG071UcuEeUF3r8VDbd1+kdqpYZgGI0c6wP5ygAqEMhzEZcYOzlgLzGXA8+Nhi2zgvBIUrIpM1OUxX8BrxF8MfqgRpfrUiMd0+zijoE13G2xXi/xV3TBpaj/Xuj7LPyrS4PwLPyq775y9v8qgrQLAF1ppWxWMNrisMA2CthAwkytfd6wZGri6kOG/cyXS9V4araRveyMKnYlDtTVBBVHlwDhBTUryK5fyNcFklS/7tlwzu7dDt1sdqZm6JSTvyD/2WprbSYg6+1a/UWFdijBGUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 08/11] xsm: drop generic event channel labeling
 exclusion
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-9-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e920645a-ffda-c8e5-ea81-c5a607e0bcbf@suse.com>
Date: Thu, 9 Sep 2021 17:35:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-9-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P251CA0001.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:102:b5::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5e36ea22-45b1-4b02-6254-08d973a7733e
X-MS-TrafficTypeDiagnostic: VE1PR04MB6478:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6478B86272229514FB01D90DB3D59@VE1PR04MB6478.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1IcJ10fr7Wpn+TmzRZa6kJ9b85CygtvB4QWRBxWmufgizrmztYHcmPFLJ7VsEdHI9TbE/+5zFVFdd4/lXvhgunG9HILkfIuXYN0UzBWBqeHoKMQONdyxT12PdsW75q3VTbJKlWaR5/Ea44VUwvKV563bXIQ3KrBgNTvyTkEcW8cxZMWkN3nDQx1zgtOj78XSfA0oiZKUfXKhbgsMoyU20h7Nj4iFIR3m4hzK0jZp9z2ugsQ+TWkyiOPyFt2FWVA/Hu2vTiOOO7JrGAD8CKI95e2JSNw8Tq6rdgfJX9C8zAzLCXCzvIO0LXQzLQ2lQ3C6NtfsIHvl35UmDk3xMlfQrze7vw+9Ypbj3Smu4WtOhqE8Dqn0nf2EUCsb03gn/hWSJl0SPJB7/soMP0KFcDI+2Tln0NoQzNuqy/yvthbzEHIKrFjfneMTvjIRelPK/VvYHbe2101IBqIh4GQ7HlGLNvu5D8ehUhczl9EKjDPPHVXYu+a+SEGCZu2hDVD85qBrynzaOfxTNyTWLtboQeutwn2cidSrR2VsmBxBIzqSyACr8M4ldBehZfstu32IcrQkzLs3NIw7CmnojNCUu0GMYt2y2jNQ8EN7bucDJC711Tv8OntG+3xo12aJ3jkUI2srXKGtc1VbBQ5rOiaVf6Toa2YZwYuIXB1a4OQN0YfH+ZzWAhmeWpH5QcFv8A0fVEqYg7MbWgc4cFO3bxGv3mmluaVfb8d1/H2dcFi5sD18NUcX5LiOOXN7C4TeFQQVJDTey4jB6e9oO0HR0Ey4tbjf9A==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(39860400002)(136003)(366004)(396003)(346002)(31696002)(2616005)(956004)(6916009)(31686004)(53546011)(8936002)(6486002)(8676002)(26005)(5660300002)(66556008)(83380400001)(36756003)(54906003)(66476007)(16576012)(66946007)(316002)(38100700002)(186003)(86362001)(478600001)(2906002)(4326008)(41533002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TVRHODN1bmo3YXYvaFFaYVZSVVEwbFBmNGg1MUlaQ1FxdUVwUXRCV20ybmNX?=
 =?utf-8?B?TXh2N1ZDRElEY0c0L2lSZXRKM1l1Wm1RdW1NQ2pvcEIvbkgrMkFGazlDSzln?=
 =?utf-8?B?dGtQTVFvQ0ZBeTByWlpCQ0NBOVdtSTdYTHd0ME45K1lXT1Q0ZHBFTFliZkhz?=
 =?utf-8?B?Yk10dSsvaTh0KzZtVnVNb2NVam9rUVI5REFUVk1pTUdnVk9oNlB6NnJyNDRZ?=
 =?utf-8?B?bmlwQml4amNHdlFWbzBRTnFoR3l1RDhJV1NyU0pZdHdEVk02WEhJVGtiZStW?=
 =?utf-8?B?c0ZaNStRendQRFpTTitPeUJlMVFuWmFMSFlpR1E4dTRpMGphclVYVnBSU0pC?=
 =?utf-8?B?aTVRY3k2SWFwN0wzVDJKeUZvZmRWUnI2bFA4YW1pZ1ZXY29aaFN1dzN6LzE3?=
 =?utf-8?B?M2FmMUc5RmxINEJxRS9ua1d3MWJ1dDZud1k2Z21aMTRsQ0M5L2JQa3BtZUUz?=
 =?utf-8?B?WVdXZWdKVDZYbWpyUDVZN3I1STY3N2pzQ3FUcWZlcWh3bDV6ZVNWYWE4cTkx?=
 =?utf-8?B?NTdRa0ptbm9PUHFpWS9sWTBmQTkvMmxnbkwyZUI3Ung0c0E3NjhubkVTYjBF?=
 =?utf-8?B?YjA1Wm5LZ1U5RDRieWVMdUo3NEp1RXNMcmRZb0NqRGcxcERROXovTTZWSXJD?=
 =?utf-8?B?a1UvZVk3cVVGRFpaV2MzM3ZuWFRNNm9aQS9pRWs5V0V3OUpyQWltS0sxckpX?=
 =?utf-8?B?WnJsdFhnSzZ3OExtUFhiRjhpOFM3ZWloMlVzQVQ2bTljVHZxcFJYckJScGsz?=
 =?utf-8?B?bUlaOUw3WjM1SUY2ZHFZaCt1QjV3SUFOT3Evb09XV0JWTjZ3TE1ZZU9LUWJT?=
 =?utf-8?B?VFp4cTBBTWIyS2dscFoxSXNRVU5YT1N5eGhKRGJPZ1hZRXFXdTlrMmdjK3pM?=
 =?utf-8?B?aVhFSlpzcDRxMUpNTTFRQ1hJZ01sN0hQRUx5U1NDNGRaRWlHQ0NYSHBDUEFV?=
 =?utf-8?B?VWdRcU1xMGdVMkVoZXRIb0Nmb0lYWS9uNUsvazRCNllNbkVUWnNWK0FUUGVF?=
 =?utf-8?B?Uk9YeGExMXBJLzVxeUhjcUtJbS9JWUlNbU9hU0hmcDJLQnBIcDRPSVdTYnhk?=
 =?utf-8?B?R1hVNElGT3hxVlRTckhOUFN6R0g4ellUU2ZHUjROMGRFSEJIUTR0MUVmL1Z3?=
 =?utf-8?B?Sys4cmwrUTFaanR0MmNhUDFNVWQvWmplWnhBcG5QZGdFd1MwT2F1cVFIM2FH?=
 =?utf-8?B?c1FsV3doQ3JLSGZSaFpCbkt5Y0F4R1cwL3h0WW9YVjFvOHJtYnA0Ynd5aFIx?=
 =?utf-8?B?QVk4NjVCcERRMWx0cEFFZGhQT2phQVVKSGl5SVl3OTNid0RhQlNXSzJtaHJJ?=
 =?utf-8?B?V09zRmpkRzFWakoxRHVFaUlBNUo2UGh6YlBaZnVNV0ZsMjU1RG8xM2ExMSt1?=
 =?utf-8?B?WWROVVVrZVUxUnZMZk1QUFVkc2lFOXhOWWFWWTI4OGxBQnNST1BBYlZ0Zksy?=
 =?utf-8?B?VklSN3RtaTNyZFg4cWF5Y0J3cGFicUJ4NWxPRU5oK1hXUkV6UEFCTWYzcnpE?=
 =?utf-8?B?VWpucWRva3d4dVVjVnhjZ1dFcjd3cWFBZWRoREtzdGVPVGszZDBpUGErNXE0?=
 =?utf-8?B?ZkR2eEhvMWJDZTJqZTc0dEEyeFhINGZNRXFJLy9zcGhCS09FeTUvdmpteGdk?=
 =?utf-8?B?NkpVeWFKeDAvb1JPZnN4QnI0aTIxNkFLR0Vhajl3enpIc0lINnNkZm1pYU10?=
 =?utf-8?B?NTZ5V3M3U05UYVFmeXZpRHdzeER4dHpKYVlFSUlaU3RpK0FDWENZNXBmbU5G?=
 =?utf-8?Q?vYlWwVxzv2fFmCCMoPJuT8dGzO0qIwuhGVbt1YT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5e36ea22-45b1-4b02-6254-08d973a7733e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 15:35:28.0486
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5WF0A+S3gqgaWgWp0Gm2wMyD+rvRHumucOCDetgfO12fxN0++X4MEQMYGhfpPhku4XJ1s2zBUEZds+IgxFHfOQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6478

On 03.09.2021 21:06, Daniel P. Smith wrote:
> The internal define flag is not used by any XSM module, removing the #ifdef
> leaving the generic event channel labeling as always present.

Already on v2 I did ask

"I'm not fully convinced of this removal: Does it get in the way of
 anything?"

I have no record of getting reply, so I'm still wondering.

> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -122,13 +122,11 @@ struct evtchn
>  
>  #ifdef CONFIG_XSM
>      union {
> -#ifdef XSM_NEED_GENERIC_EVTCHN_SSID
>          /*
>           * If an XSM module needs more space for its event channel context,
>           * this pointer stores the necessary data for the security server.
>           */
>          void *generic;
> -#endif

I don't consider this any better than what you had originally: Then
you've removed everything inside the #ifdef as well. Now you keep
it and drop just the #ifdef, exposing the field universally (as long
as XSM is set) despite Flask's 32-bit only field being enough for
all practical purposes.

What is there demonstrates what the original intentions were. It's
easy enough to put back if needed in the future, but I think it's
even easier to simply leave as is.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 15:45:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 15:45:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183343.331456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMF8-0001NB-0h; Thu, 09 Sep 2021 15:45:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183343.331456; Thu, 09 Sep 2021 15:45:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMF7-0001N4-T3; Thu, 09 Sep 2021 15:45:37 +0000
Received: by outflank-mailman (input) for mailman id 183343;
 Thu, 09 Sep 2021 15:45:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZYDC=N7=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOMF6-0001My-J9
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 15:45:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fdfed71c-6df8-43ab-9f65-e3057c4c4177;
 Thu, 09 Sep 2021 15:45:35 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-C1NDGUdgOMKX5VLV9O9Ugw-1; Thu, 09 Sep 2021 17:45:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep
 2021 15:45:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Thu, 9 Sep 2021
 15:45:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR07CA0030.eurprd07.prod.outlook.com (2603:10a6:205:1::43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Thu, 9 Sep 2021 15:45:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdfed71c-6df8-43ab-9f65-e3057c4c4177
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631202334;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Obk2aI5gcsFFFM6Z23Rp7Yx/We9v/EARn0Z5pydQ+N8=;
	b=g2QiTi82FB2yGbjhV56WqlHpNg4yB3picEsLvSVxSkITrRAUAXA1pakc7htFYFRZR8OHwW
	U7GEkOPxZbKure/Lw+COqjfJHPTCbmecfwXgZQrWHsGKNornXy39pDJRU5L/Mh9yDHlB7X
	/WxrarrbhLiK5wlemd2WSoe/8cY5v+8=
X-MC-Unique: C1NDGUdgOMKX5VLV9O9Ugw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ypja0fzlQ58crFjt2SzThOPS3RtPdBsOS2fZEPRlz8W5O6roSTuq8UGLka3v1oPxWT3BIq8rPYsK8GO6GArEV6FgGoGuxyspJTTBIVtZlpO+CE1vTfSwbUFzWeXPqLFLchIFZRozA+iilBqGyO8qlRcWlS7SCKDNaDcnglOSfNnkz3LBZOe6ShSxZgY+u00k6FR0pw3ojcomFIvH5GsnSRFjP34Jdais+vgtpChtexeA7AHAn7wS1ZRUSQQzgd0nWtEAQE4Pes7CWCsb97HYSuaWO7GZV6vX/UDd6/pycYrhfCNVerkC6uvRbd0sz94fu9zq0T0stpun7H2Y4TsywA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Obk2aI5gcsFFFM6Z23Rp7Yx/We9v/EARn0Z5pydQ+N8=;
 b=QWeO1xo9LqwTA1QUK52+BRPjAJjdPlmSkd0xG+2ZW2BbGxCKA/WW9uCwJdzSuJTVRaH/cyrii0Jjcwnup4yPg9rJRklJVTQ40aSeFyNbl2B4zwJwfpWRj7M5gS/f3MQmEMm1N3isTHd2QN9jRPjhSj3SBppfRFdEQqA9+IdyuheNP3vZfLzF5x83Rrd4c+JjT2DGY/On3KxDZG2nvDzHWWgF51ir3PPFva8Sgh3Rwk36DRF801RJq4Ep99O6SqS+zqZwZ79LZewZw5SKCbAo1qoqS/wwUGNcU2wo5Lp/UAQ2s66VcdgOtnz+/mpybTtHxcvrW9Egb9idTPP/qq0SJA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 09/11] silo: remove circular xsm hook call
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-10-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8d7824e1-3c7d-c0c5-f3ad-0644d4c49e07@suse.com>
Date: Thu, 9 Sep 2021 17:45:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210903190629.11917-10-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR07CA0030.eurprd07.prod.outlook.com
 (2603:10a6:205:1::43) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b5fef1f4-1ad4-4a47-f4ac-08d973a8db37
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB736039CC41ABD42977013403B3D59@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Pb3qceEy9KFLhoQsIriEuKkInIo49l2KBQwJGcYrjCJm3YEVYvdaWVhOKhcTk1JMjK2gP2j0EUT+hrj+N3qKuN0KWwYc1rHBxaUIBk8OKS1VFmYzlyQdzdE3ugXp9rcDm7pCibM6PQZaf1+Ll70icSJc1rTnJDe9nCgkPKmsRWaX2Luuh7s8n95Fd5mRH8wLez9abXYjn3Mb88oJ3x3hKX45pvS13+bB0vkx20k4TuuFq/2nGj1ag9bAjuYE4Q11hzEpI8aSn1TgxpbmoEDi8F9VViPeV7ebdFqPMIkgzCV/liex7NBH9QlbGwKh5GtERe48HI8YV2k+lhT79LvN4eapreGA4U+HmvG8hNA9idNdp9qN6CDW9J7MxPoOEU7faydMdwzw/WG/kgekLUmv9+u0fBvILIwSWbH/VowzQvDASk8kYhXfPSQVAlvdm+9crBi3aYZmwW9IFP9Uud6G4Wp9sICIvTDV3QUDC6Ln0p0t5sDLf1pH0GufyotQwUzKPZ2KGUaXCyPHTqjDtzzHX8qJ5C0mc0aOJvLXIleCALNrCFRDpMzhZX/IAgj6eqgLBTYooowC6XM0WJXuHGLdCkLbKm/JE88SSAdBsD4PhmPodRX01vB/odCJ3+KamLFJ3pcs9CX/tREAHuAYEPBoarsplj160T9wCa5lOTGyHbS4b0DYXv9lBc9wf8olgihYiyFkrYZpQ0ADVGMrCpLc7CSq5dRbw3znX/UTnGyuBfsin0obJJVMPFtmVeoEvHEu
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(136003)(396003)(366004)(346002)(8936002)(53546011)(6916009)(316002)(956004)(16576012)(31696002)(5660300002)(2906002)(6486002)(31686004)(66946007)(66556008)(2616005)(36756003)(4326008)(26005)(186003)(8676002)(86362001)(478600001)(66476007)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dWtSMmx5R1o3TmlKckxVbXZSMkIxVFBYc092dTNrQ2tsZWxkZkFrKzdRZXEx?=
 =?utf-8?B?YnQvazQ5VGYzMGMwWlo0S2hMMzNoU0NleU5vRS9hZGx4akplVEtPbEo2YUg5?=
 =?utf-8?B?MGJTTG9UejZTcndxaDF0R1l0Slk0WXVZYTZsdDEvc00xbEp6ekliWlN2VTdv?=
 =?utf-8?B?aXJxQUpLRVRPVHZWNkJ0MEdEM0UzMjV3T1RuQ25DMld6Tm4xSk03S2dlbDNi?=
 =?utf-8?B?S0RNa2pjb2xNL1BIbUpKeWxDYm8vdU15R1psaTZhMm5uY2xtQ3kzYlN0ODRk?=
 =?utf-8?B?YjZURFFmTk9BV0o4OGM4cy9PdzN0alBtWitlNHpzM3pIbFdzQzIwRi9XUG4z?=
 =?utf-8?B?VWtseENqV1FwV3UwY3ZlLzI2MUVLQ0FPWTR4NzZJWkEybnlldXNvV2tEOHc4?=
 =?utf-8?B?QWJOWHpJZWJjWm9VZC9qcDd6MWlKMmsyT2xBWHo2M3cxazBwTWRRYzJsalNv?=
 =?utf-8?B?YXVDZ1I3TVQwcnQrc2Qvc0lpOE5CelZMRkhnT2NkZUlSZkh1OE5yWkhOWUo2?=
 =?utf-8?B?UHhVMkFoY1Fvb1lsRlBiZUZJalpzcVlKM0p5NWNvaHV4VFgxc1NuVWhIZytH?=
 =?utf-8?B?STU4TndQQVJoUS9xQWV0UFphQktQaTlhcU13OWpTMCtDZ2pXWUdjZTA2TmpZ?=
 =?utf-8?B?R1Mvb2lnZ0JzTStuSGVrdnVQUmh1YjFzdHZDWG1KT0V4S1dnS3pIeElkRkZM?=
 =?utf-8?B?RzZYbEFVbmtwSm1vY1NkRzJkRmticS8yTW1OVmJld1NGRllwOWZaRVpBdHlC?=
 =?utf-8?B?aDVPMUVVbmpkbnJQcjhCdHB0NWtDZm9qVHJJWDBDWW83cUdWcUJYVXE4RVdm?=
 =?utf-8?B?bE5rc2EzYXJKYnJabUhGN2hTQlVabUhLcHUyVDNSNGVXN3B4Y1pIeUxNL2xr?=
 =?utf-8?B?eGFtU0FoV0RSWlIvVUVKWUhoQ0x1ckk2bE9rb2dkaVZINS9DMVg2OGc0OFJz?=
 =?utf-8?B?b29YZGZSS1NSM3FBYzRYUXNXd011bkwwWndlY3JKMktlTjdPMnBVMlVyNk1E?=
 =?utf-8?B?aFF6NzlsOFdJNjBNZnFmU3FUbndMd2l3aWFvOTd6NHZxOHhFa2RDUzMwbjZz?=
 =?utf-8?B?bm05UVlCM3kxcW4xSTNoOSs2L0lqUUYvMlkwMkhJSXR3S3FKakVrTFRlOGNi?=
 =?utf-8?B?enRJalJENXpWQWF1ejlZc1Y4bVhMU1RjYzBXNDNQMFdxZE80QkU4YkZSRTlL?=
 =?utf-8?B?ZytrbEYxZ1VXdFBDYzViQks2bGFmc3l5djQ4NEtPOVl1N1J1bnBZKzlDano0?=
 =?utf-8?B?dUZsRjc3ZzZLVFNVVzhFVWFlVHQ5TnRMWjNGbzNtdXFOY3JDQk13MDVxSkJZ?=
 =?utf-8?B?dTBpNkNVanNYcDFMNERGdGdsWFZKTGNlVzZTUmV3cGp4dnkxSVl2ZUdiVW5x?=
 =?utf-8?B?U2NNUWFScWFFc1lkTGs0NHo3WlpSaVdWQTRtbUhsbE16V0NDM3YyTVhEWEds?=
 =?utf-8?B?NGFET0FHMEppZm8rRnVqQ0g4T3JWMFRtNmI3cHNGcVJjYml1YURhZE1LZjhj?=
 =?utf-8?B?R3kwdE9GeDVDKzd5cWtydUhObUhEeGVNQzRoWEVuSUtEaVRicXhKUWZKQ3cy?=
 =?utf-8?B?c1VPQmU2R1hHZWp4RGRJZVpMajluTnU2YWNVYUdKc3ZRZ1dyZHN3RGdzbFNV?=
 =?utf-8?B?clVOaW9YdkRKR09udzVQcjFRYW1acVZ2L0I3dnRnVW5BUGNUZUVrMWQ2Vjd5?=
 =?utf-8?B?bExGK1R3T1RCR2UwL05JWVQ1bVM0ckF6Umg1Wm1vTmNZNCt3cmhYUGt3MFhl?=
 =?utf-8?Q?CCa818cG2SNHbvBh93hZMo5R/LryAbcAUaNTJUE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5fef1f4-1ad4-4a47-f4ac-08d973a8db37
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 15:45:32.0612
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Uq0xTgeSFVo8YgjYB1ruwNvxM3NlUHF4y3i1s2/IYEC6k3po74o85cMVg+aXE5Dr2+B0ckcoFZLjsXekDI0DdA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 03.09.2021 21:06, Daniel P. Smith wrote:
> SILO implements a few XSM hooks to extended the decision logic beyond
> what is defined in the dummy/default policy. For each of the hooks, it
> falls back to the dummy/default policy. The fall back is done a slight
> round-about way. This commit makes the direct call to the default policy's
> logic, xsm_default_action().

Again it's not clear to me what you're finding wrong here. The way it's
done is not as direct as it could be, but going through the extra layer
allows the functions to document things at the same time. You lose not
only that documentation, but also ...

> @@ -43,7 +44,7 @@ static int silo_evtchn_unbound(struct domain *d1, struct evtchn *chn,
>      else
>      {
>          if ( silo_mode_dom_check(d1, d2) )
> -            rc = xsm_evtchn_unbound(d1, chn, id2);
> +            rc = xsm_default_action(XSM_TARGET, current->domain, d1);

... will need to sync changes to the dummy xsm_evtchn_unbound(), no
matter how unlikely such may be, back to here. This would be quite
easy to forget.

But maybe I'm overlooking something where how things are really gets in
the way of something you mean to do in the remaining two patches (or
later)?

>  static int silo_grant_copy(struct domain *d1, struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_grant_copy(d1, d2);
> +        return xsm_default_action(XSM_HOOK, d1, d2);
>      return -EPERM;
>  }
>  
> @@ -86,14 +87,14 @@ static int silo_argo_register_single_source(const struct domain *d1,
>                                              const struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_argo_register_single_source(d1, d2);
> +        return 0;
>      return -EPERM;
>  }
>  
>  static int silo_argo_send(const struct domain *d1, const struct domain *d2)
>  {
>      if ( silo_mode_dom_check(d1, d2) )
> -        return xsm_argo_send(d1, d2);
> +        return 0;
>      return -EPERM;
>  }

This would then also avoid introducing the anomaly observed by Andrew here.
And in fact the Argo dummy functions may be a good example where a change
might happen down the road - them being all empty doesn't seem quite right
to me.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 15:50:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 15:50:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183349.331466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMJx-0002la-Jf; Thu, 09 Sep 2021 15:50:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183349.331466; Thu, 09 Sep 2021 15:50:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMJx-0002lT-Gl; Thu, 09 Sep 2021 15:50:37 +0000
Received: by outflank-mailman (input) for mailman id 183349;
 Thu, 09 Sep 2021 15:50:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOMJw-0002lI-09; Thu, 09 Sep 2021 15:50:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOMJv-0000oK-QO; Thu, 09 Sep 2021 15:50:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOMJv-0005Yx-If; Thu, 09 Sep 2021 15:50:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOMJv-0007ZK-ID; Thu, 09 Sep 2021 15:50:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+OsoCsLiONlUyLDiDkUyUqQ4j8FHCFp7g1gMFqooq5I=; b=nZLe4e64EcLbygKoZfBWVNDPxX
	v2DSCTKXsKO+X8N6MnNbwb/qjcey8xi3emTMrfwI+zZoZAiD69lE/+b4/D84WCb20V9EQ4ilMRan8
	NuA3yx4ybTZW6T9djyTo93V9TsHPX6hWn+riPYFUblsj09Cuzf6prvaW0ubFi/1ldkLc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164912-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164912: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2e2e22c7d50392fa5cced3e0334b217426f48b7a
X-Osstest-Versions-That:
    xen=56abcf1a58bdaef18579cf2ce8645c8c72a2b749
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 15:50:35 +0000

flight 164912 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164912/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2e2e22c7d50392fa5cced3e0334b217426f48b7a
baseline version:
 xen                  56abcf1a58bdaef18579cf2ce8645c8c72a2b749

Last test of basis   164891  2021-09-08 17:01:44 Z    0 days
Testing same since   164912  2021-09-09 12:01:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   56abcf1a58..2e2e22c7d5  2e2e22c7d50392fa5cced3e0334b217426f48b7a -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:09:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:09:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183360.331487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMbh-0005Rb-9a; Thu, 09 Sep 2021 16:08:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183360.331487; Thu, 09 Sep 2021 16:08:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOMbh-0005RU-6N; Thu, 09 Sep 2021 16:08:57 +0000
Received: by outflank-mailman (input) for mailman id 183360;
 Thu, 09 Sep 2021 16:08:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOMbf-0005RO-NT
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:08:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOMbf-0001ee-Kb
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:08:55 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOMbf-0006kJ-JK
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:08:55 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOMbc-0006oV-9z; Thu, 09 Sep 2021 17:08:52 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=E76sYHJv5LOBDBqfWN5W/QfSk1uHSHyU+LukfShWK4E=; b=g9MddJbm8Q5F6hfwL2V4rw0aDv
	E0Wz2HmNIiFJtknytnBO8DIBGGlfoX1VNj/m8ng5k/F/DV/tumugC78URUYo2WYB8LGEQbr7BR7+3
	Uz7eR2eiraaiYTQLYNZS7LBNV+9GJerDKw3bt+H6M1a47DT2HrVeaQrlbj6iSq+YeIoM=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24890.12692.44064.995392@mariner.uk.xensource.com>
Date: Thu, 9 Sep 2021 17:08:52 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper  <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
In-Reply-To: <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
	<20210909124924.1698-2-jgross@suse.com>
	<24890.2778.780247.724651@mariner.uk.xensource.com>
	<7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub"):
> On 09.09.21 15:23, Ian Jackson wrote:
> > How about "pv-grub-maybe" ?  Or something.
> 
> What about "pv-grub-if-enabled"?

Fine by me.

> And could that be done when committing, or should I send another round?

Please do send another round (sorry).

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:35:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:35:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183372.331505 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON0v-00013I-Uf; Thu, 09 Sep 2021 16:35:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183372.331505; Thu, 09 Sep 2021 16:35:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON0v-00012U-QK; Thu, 09 Sep 2021 16:35:01 +0000
Received: by outflank-mailman (input) for mailman id 183372;
 Thu, 09 Sep 2021 16:35:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5xa6=N7=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mON0u-0000zO-WF
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:35:01 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ba95b034-5d64-4cc1-a1cd-3fed2c23f413;
 Thu, 09 Sep 2021 16:35:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba95b034-5d64-4cc1-a1cd-3fed2c23f413
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631205299;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=XyAZ9tVbkw9oc4UHQy0JFhFEp2DcwLj5o9RUfsQCO+g=;
  b=VFk6BqzxThNnwtO4gmsA9vgW024qNm7ph8AeUvGJfz2IVPQYxkvG9UyP
   fdi3LEayfC4k6SPgBJmKOVSCcBlECmJp1hh9hpMUphZc6ZV5xt9DBpzdb
   pNAHHnG4+tUkGFAHMupxqVvhbvPY5oU9rroDxTJ4qG/KLpyhLAvERtGJE
   c=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: iE/xgJg82yj680ioyAa0b2lSBDNg2lrMOdcpF/khdp9jdLLxQOmQeCOzQtO7f1GJaOgtEH5PBi
 R5oof9iLR+Lxsi1f1hR1tlcwVfcNDowsKLHn7LjqJfyXeYbo8rqHhqukWRLUTLBe29vi1QfL88
 dbs0kweLe5SxIIrUDlgdsRDLixFc9gQBjUmPyKLWsWTIjHVZaE9/WseJoI0vfUDiEBUtKfMbC4
 Pd9lYF+CuxJqqZEnEqitm2CNOc5xUbeo1YVPH761ZYxZ/tdcp3tIcghAWw5XlAPuf/5701V8Ct
 ROtsKKj1iZuocqYWslQbcSHo
X-SBRS: 5.1
X-MesageID: 52390010
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:kN+zQaDblaqxrPvlHelo55DYdb4zR+YMi2TDt3oddfU1SL38qy
 nKpp4mPHDP5wr5NEtPpTniAtjjfZq/z/5ICOAqVN/PYOCPggCVxepZnOjfKlPbehEX9oRmpN
 1dm6oVMqyMMbCt5/yKnDVRELwbsaa6GLjDv5a785/0JzsaE52J6W1Ce2GmO3wzfiZqL7wjGq
 GR48JWzgDQAkj+PqyAdx84t/GonayzqK7b
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="52390010"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS
Date: Thu, 9 Sep 2021 17:34:39 +0100
Message-ID: <20210909163441.44418-2-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210909163441.44418-1-kevin.stefanov@citrix.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The spec requires 64-byte alignment, not 16.

Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>

Note: This does not fix the FACS alignment inside guests yet. See next
patch.
---
 tools/libacpi/build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libacpi/build.c b/tools/libacpi/build.c
index a61dd5583a..fe2db66a62 100644
--- a/tools/libacpi/build.c
+++ b/tools/libacpi/build.c
@@ -532,7 +532,7 @@ int acpi_build_tables(struct acpi_ctxt *ctxt, struct acpi_config *config)
      * Fill in high-memory data structures, starting at @buf.
      */
 
-    facs = ctxt->mem_ops.alloc(ctxt, sizeof(struct acpi_20_facs), 16);
+    facs = ctxt->mem_ops.alloc(ctxt, sizeof(struct acpi_20_facs), 64);
     if (!facs) goto oom;
     memcpy(facs, &Facs, sizeof(struct acpi_20_facs));
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:35:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:35:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183371.331501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON0v-0000zb-Kp; Thu, 09 Sep 2021 16:35:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183371.331501; Thu, 09 Sep 2021 16:35:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON0v-0000zU-H8; Thu, 09 Sep 2021 16:35:01 +0000
Received: by outflank-mailman (input) for mailman id 183371;
 Thu, 09 Sep 2021 16:35:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5xa6=N7=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mON0u-0000zH-H3
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:35:00 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id df86ced6-118b-11ec-b1cb-12813bfff9fa;
 Thu, 09 Sep 2021 16:34:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df86ced6-118b-11ec-b1cb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631205299;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=jxIDQkKsTXbVItmungXARfHQlLf5LNa97ZaMET1EAnQ=;
  b=ZCrBLm8z12plVurr4U8G2n7uOJ3cMUOnEBwcaAr+h68gZRA5eQqexpxj
   3nDkLnnlE4F8oEVf9/QLZGtKxxo+DjRCqXx7O/0bG2K7PWhQIwk8+AZS0
   r98J6K6qYb6m+qYXyewrYY9jKwfDiWa2UkVB+PVK9L/AjvOIUrcYNgbB3
   s=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: WRHW/DJUzgC1I3mxF5Mnk71MyY7xV/GezfWw7jSVLQfqpPVLk98vwMu/+9UeTR66QByRZxFURa
 mcnGydFCkFtmgv/FmiyzxYU0GW/vxBr9SNODtNz3eqQnUeWqHBYNpqg++VJyT6cgjc/zcwc9a8
 FA/4WVMQWztCNxmRaAxaD8A3jTEYNfQY1XVhV5emTuu4NAkJ3IHkssikGBIZCAVQ8xB5kQfcqo
 VNzreqtZ8oHjLjWOuQN7hhuADOivkJtdTsSTs/QtlD0cvBqXRt1hM3Pkh9URbD1NVwQwF0c3uf
 RnylTMDmFGvX4f76lKUzoMod
X-SBRS: 5.1
X-MesageID: 51975736
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:7yYLK6NXAmB0ScBcTjejsMiBIKoaSvp037BK7S1MoNJuEvBw9v
 re+sjzsCWftN9/Yh4dcLy7VpVoBEmsl6KdgrNhWotKPjOW21dARbsKheffKn/bakjDH4Zmvp
 uIGJIObOEYY2IasS77ijPIbOrJwrO8gd6VbTG19QYdceloAZsQnzuQEmygYzRLrJEtP+tFKH
 KbjPA33waISDAsQemQIGIKZOTHr82jruObXfZXbyRXkzVnlFmTmcTHLyQ=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="51975736"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>
Subject: [PATCH 0/3] Fix alignment of FACS in guests
Date: Thu, 9 Sep 2021 17:34:38 +0100
Message-ID: <20210909163441.44418-1-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

When booting Xen as a PVH guest, it currently complains:

  (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:b004,1:0], pm1x_evt[1:b000,1:0]
  (XEN) ACPI: FACS is not 64-byte aligned: 0xfc001010
  (XEN) ACPI: wakeup_vec[fc00101c], vec_size[20]
  (XEN) ACPI: Local APIC address 0xfee00000

This is caused by several bugs in the toolstack whilst writing ACPI
tables.

Kevin Stefanov (3):
  tools/libacpi: Use 64-byte alignment for FACS
  tools/libxl: Correctly aligned buffer for ACPI tables
  tools/libxl: Only allocate 64 bytes for RSDP

 tools/libacpi/build.c             | 2 +-
 tools/libs/light/libxl_x86_acpi.c | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:35:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:35:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183373.331523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON11-0001YY-7P; Thu, 09 Sep 2021 16:35:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183373.331523; Thu, 09 Sep 2021 16:35:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON11-0001YR-3G; Thu, 09 Sep 2021 16:35:07 +0000
Received: by outflank-mailman (input) for mailman id 183373;
 Thu, 09 Sep 2021 16:35:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5xa6=N7=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mON0z-0000zH-AQ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:35:05 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e0509478-118b-11ec-b1cb-12813bfff9fa;
 Thu, 09 Sep 2021 16:35:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0509478-118b-11ec-b1cb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631205300;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=05ujbDwCHYeB1etSWskRRxGCGEOzl35sXRnTpQwyYyQ=;
  b=cH07yqkzdQwzzNUANa+GYhsu01sNhjU0VydsHYHzi3h2lMFlqvDI3k2A
   nsuNzoVRf4fKZnqtzNDdDnxYWAx1t+6N5Mf2sKqTRRe+9N2f3z9j8UYL1
   K+IHV4cXSM0N7e0nYff82ZksjU7gI1l2ULYxcAumk/Kxi3Qpub9PRsat+
   8=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: XVrzfeg/RpgzLA8csCHC+Mc3NGz0xP5HCaQGB9gx1e3k6S9orcQfSFmA0+ddQ1jZv1hQpKl7Lt
 TAx3JpmSlonEk2KEpz5CfKpHdOs22WSLO4F3HgPjwjfKG8+jNgACYVqVHP0zksNEuZc4kNvkJ7
 c4rVsOFS01/8Y66KPMfohKkfFtx5CJHo2W2gn7WeNoG/4R+ERdHnRZfVY/+207j7R89HZ8W3Fj
 gHReyXDEGgvC7jQjXf6rtm3IPMMNE87lh3IIPJy3/QqJcyyskQ6/aC8C/dYAjMA8HGUDye4z8A
 tsQ4oQ/2Q3tv69Jp0U1HmxZS
X-SBRS: 5.1
X-MesageID: 51975738
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:GiRqhKzLr91ncqQjwy5MKrPw2r1zdoMgy1knxilNoHxuH/BwWf
 rPoB17726TtN91YhsdcL+7V5VoLUmzyXcx2/hyAV7AZniAhILLFvAA0WKK+VSJdxEWtNQtsJ
 uIG5IUNDSaNykfsS+V2miF+9ZL+qj5zEir792usUuEm2tRGtBdBwQSMHfqLqVvLjM2fKbQjP
 Cnl7d6TzzLQwVuUu2LQkMrcsLkvNPxmJfvcXc9dmIaAFnnt0LS1FbieSLopCsjbw==
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="51975738"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 2/3] tools/libxl: Correctly aligned buffer for ACPI tables
Date: Thu, 9 Sep 2021 17:34:40 +0100
Message-ID: <20210909163441.44418-3-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210909163441.44418-1-kevin.stefanov@citrix.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The pointer resulting from libxl__malloc() has no explicit alignment.
As an implementation detail, it has 16-byte alignment.

When this buffer is used by libacpi aligning ACPI tables to greater
than 16 does not work correctly.  This causes the FACS to not be
64-byte aligned when the ACPI tables are copied into guest memory.

Align the ACPI tables buffer to a page, to match the alignment
inside guest memory. The buffer is already one page too large,
presumably intended for this purpose originally.

Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libs/light/libxl_x86_acpi.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 3eca1c7a9f..0a82e7cacd 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -193,6 +193,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
      * Set up allocator memory.
      * Start next to acpi_info page to avoid fracturing e820.
      */
+    acpi_pages = (void *)ROUNDUP((unsigned long)acpi_pages, libxl_ctxt.page_shift);
     libxl_ctxt.alloc_base_paddr = ACPI_INFO_PHYSICAL_ADDRESS +
         libxl_ctxt.page_size;
     libxl_ctxt.alloc_base_vaddr = libxl_ctxt.alloc_currp =
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:35:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:35:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183381.331534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON1L-0002Ru-G2; Thu, 09 Sep 2021 16:35:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183381.331534; Thu, 09 Sep 2021 16:35:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mON1L-0002Rn-CO; Thu, 09 Sep 2021 16:35:27 +0000
Received: by outflank-mailman (input) for mailman id 183381;
 Thu, 09 Sep 2021 16:35:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5xa6=N7=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mON1J-0001rN-SV
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:35:25 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 24a98b95-d76e-4665-9e6e-14ec9f6de728;
 Thu, 09 Sep 2021 16:35:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24a98b95-d76e-4665-9e6e-14ec9f6de728
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631205319;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=35xQpM/thu7zR5kkv+/e98XVppVGc4tUolZlEnqQblo=;
  b=gSswPpum84z4Z/owIYBzPDHY6McJ5qQI2EwOAlQsnYk1jR4gpM2wqE1/
   YE9Vhkx9n960L7WettWQAKbsuVQdzCEzqQ2+DcKSHq3vUDxB8v+Xn8Qru
   tEBVbLqqMZSULsF1WuWJQaSxAQE8mbV0XHJNQbDCfmSu3kJKX3LccnvtE
   A=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: +3BrRwVXpvZbjpR1G8slxnNa4TcWQuzGqFyaK4jzKYmOgVzZAGW7BJak7M4ilt8IuygVnycUPl
 fdnIS4tL8LlFuebY7o+OLbA0v9e//IBQ9q9kqcvl+nT9Bj8btOFNfmJignX1TnFIYuoY3oRj2G
 vP9HSz2jG83xwFbpHkTtSD3IXnpT6oi0eQk1TXEQS4LPGIGQxaKK38dpiv5epUWk1S6BMATZyx
 NOfjkxBxXJRH7Hvela+Gvf2tgq2NlALRjl6zJqfd0K+57+a9Huu2cYNTARp6qfrgAUzQ8x0iRf
 MibL/Z7zcqxNeFZvLC4eRm1G
X-SBRS: 5.1
X-MesageID: 54142780
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:JrjuMKqUL6XpiV/Pgrs0ai8aV5oReYIsimQD101hICG8cqSj9v
 xG+85rrCMc6QxhI03I9urwW5VoLUmyyXcx2/h0AV7AZniBhILLFvAB0WKK+VSJcEeSmtK1l5
 0QFJSWYOeAdWSS5vyb3ODXKbgdKaG8gcWVuds=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="54142780"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 3/3] tools/libxl: Only allocate 64 bytes for RSDP
Date: Thu, 9 Sep 2021 17:34:41 +0100
Message-ID: <20210909163441.44418-4-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210909163441.44418-1-kevin.stefanov@citrix.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

RSDP's size is 64 bytes and later in the function, its buffer is
hardcoded to be 64 bytes long. Don't bother to allocate a whole page.

Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libs/light/libxl_x86_acpi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 0a82e7cacd..2aea1eca31 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -183,7 +183,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
         goto out;
     }
 
-    config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
+    config.rsdp = (unsigned long)libxl__malloc(gc, 64);
     config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
     /* Pages to hold ACPI tables */
     acpi_pages =  libxl__malloc(gc, (NUM_ACPI_PAGES + 1) *
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:45:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:45:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183398.331544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONAW-0004Wp-FU; Thu, 09 Sep 2021 16:44:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183398.331544; Thu, 09 Sep 2021 16:44:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONAW-0004Wi-C4; Thu, 09 Sep 2021 16:44:56 +0000
Received: by outflank-mailman (input) for mailman id 183398;
 Thu, 09 Sep 2021 16:44:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mONAV-0004Wc-5D
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:44:55 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16e29bf0-368f-4ebc-bef9-f64169265700;
 Thu, 09 Sep 2021 16:44:54 +0000 (UTC)
Received: from [10.10.1.24] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631205885476839.4236945617671;
 Thu, 9 Sep 2021 09:44:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16e29bf0-368f-4ebc-bef9-f64169265700
ARC-Seal: i=1; a=rsa-sha256; t=1631205891; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=SK4bv8DeioH6F7i7aFydxxpqV1FWfgs3Esu+32B3hrDQsKkjOiCmDbbKZifpdGgaJ3vJzsFNElKx4sRr8spE97PQjv6tepg1o3txzdoBs5VlXPv9bbtgkiUm6J09hu/Mhji8fwAhJ5y66RiQQDFaafKK+TpwziE9krIsEXJUVxw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631205891; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=fjPStVlaqAWOklvbdsZb4F02NclSWm/Ozl/EFyCNWKU=; 
	b=Aru+lYZ8MO0rs2p9iQFc4vjtwU2RXuN9NAw93f4753OhLyDQJAZsDL76/zqwT+j6UyIWzQPflQ0XYb/drsxTCo6L2Bv6mg2TQMn+Y6W/mbNndV2imE+uigaZJyHz9x0Mgdq98WhWUtQCg4pEh8FUH7JAGTESs1731WKkPrMki5g=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631205891;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=fjPStVlaqAWOklvbdsZb4F02NclSWm/Ozl/EFyCNWKU=;
	b=chsmgXgqVBUdic7ccu1+NuUAdqrr70517xxDfRj9TuzGttFoaZwUdwQqiCCQRyvi
	rCBEKQMKZCr8dbkwtHFRZ+NhV++0DgKCKG691E9QF8O0IsnCKUrxheSY43+FzBfbwe9
	OijEd5BJwAYdI6qgUmAu1Qj7R6PG+QgQYZIfvFks=
Subject: Re: [PATCH v4 08/11] xsm: drop generic event channel labeling
 exclusion
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-9-dpsmith@apertussolutions.com>
 <e920645a-ffda-c8e5-ea81-c5a607e0bcbf@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <ea93a0f6-9b36-10b6-4c10-333b736d2a6c@apertussolutions.com>
Date: Thu, 9 Sep 2021 12:44:43 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <e920645a-ffda-c8e5-ea81-c5a607e0bcbf@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External

On 9/9/21 11:35 AM, Jan Beulich wrote:
> On 03.09.2021 21:06, Daniel P. Smith wrote:
>> The internal define flag is not used by any XSM module, removing the #ifdef
>> leaving the generic event channel labeling as always present.
> 
> Already on v2 I did ask
> 
> "I'm not fully convinced of this removal: Does it get in the way of
>  anything?"
> 
> I have no record of getting reply, so I'm still wondering.

I can't find any email record of it but I know I made this change
because of a request to drop XSM_NEED_GENERIC_EVTCHN_SSID since it is
completely unused. Honestly I am not concerned on whether this
completely unused field is kept or not that is behind a flag that is
never set but it clearly is concerning for you, so I will just drop this
for now.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183406.331564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCl-0005Jp-FC; Thu, 09 Sep 2021 16:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183406.331564; Thu, 09 Sep 2021 16:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCl-0005FZ-BI; Thu, 09 Sep 2021 16:47:15 +0000
Received: by outflank-mailman (input) for mailman id 183406;
 Thu, 09 Sep 2021 16:47:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-00059k-7I
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-0002Mr-6V
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-0001RU-4n
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mONCi-0006u8-9f; Thu, 09 Sep 2021 17:47:12 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=x/Z6JtsWJCPSlEmR+qVnOar1lbZPIk02VCEiJE0/kp8=; b=fOTlz14qWOZosaesslU/xfdF+g
	jVBn/DhAxoHPnIFg/NgkuHwKAnWldO4IMWsX+YKzuCDfhfEsrSi3pbggGc/n74F1jLvDDzAiaoexu
	2FxZyJdrl+N/1BU1rHYHfxdjibWL7usSt5Q4X9FkBSjdPJBYnVTZX3AkuIYy+lO3zuEI=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 3/4] fmtarches: Use dom0arches, not hardcoded arch list
Date: Thu,  9 Sep 2021 17:47:02 +0100
Message-Id: <20210909164703.19426-3-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210909164703.19426-1-iwj@xenproject.org>
References: <20210909164703.19426-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This will make us reallocate fmt tests when the arch list changes.
It's not ideal because it means tests jumping about across arches and
might let regressions go through but it's better than just dropping
them, and doing a better approach is complex.

This changes some jobs and adds others, roughly:

  +               test-amd64-i386-libvirt-fraw
  -               test-amd64-i386-xl-raw
  +               test-amd64-i386-xl-vhd
  +               test-arm64-arm64-libvirt-raw
  +               test-arm64-arm64-xl-vhd
  +               test-armhf-armhf-libvirt-qcow2

(as reported by standalone-generate-dump-flight-runvars)

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 make-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index e0d11c80..ecbb195b 100755
--- a/make-flight
+++ b/make-flight
@@ -652,7 +652,7 @@ do_pv_debian_tests () {
   # Within each fmt we rotate through the list of arches
   # The starting list rotates once per ts, so that we try to
   # exercise each fmt on each arch family.
-  local fmtarches_outer="i386 armhf amd64 arm64"
+  local fmtarches_outer="$dom0arches"
   local endfmt="do_pv_debian_tests-missing-ts-fmt-for-dom0arch="
 
   for ts in xl libvirt ; do
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183404.331555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCk-0005A3-TS; Thu, 09 Sep 2021 16:47:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183404.331555; Thu, 09 Sep 2021 16:47:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCk-00059x-QX; Thu, 09 Sep 2021 16:47:14 +0000
Received: by outflank-mailman (input) for mailman id 183404;
 Thu, 09 Sep 2021 16:47:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-00059V-8J
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-0002Mj-5m
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-0001R1-4j
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mONCh-0006u8-6X; Thu, 09 Sep 2021 17:47:11 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	Message-Id:Date:Subject:Cc:To:From;
	bh=UMqWVNP+rUg/q7PFg9vaAE7KX7sL7PFJCgDLmgRZxek=; b=TH7E17IQMBWN8yFdiE0k0D+UWL
	20iBRjJS7okhuFzAAqJkILp7G5PQ+m1CZ0rxz45y17RqHqZcP6dXwAw9WvDZt5ULureYHKJolKxyy
	Wzn6icPaJjIMp8yvgH9ADplCW2+pPr6i2/6YOa3ha45EXq30QolnhvbzDenjyFnvXtxU=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 1/4] mfi-common: break out dom0arches variable (nfc)
Date: Thu,  9 Sep 2021 17:47:00 +0100
Message-Id: <20210909164703.19426-1-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This will allow test choices to depend on the actual rather than
entire dom0arches.

No change to output from standalone-generate-dump-flight-runvars.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mfi-common | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/mfi-common b/mfi-common
index 2834411f..70e17f7a 100644
--- a/mfi-common
+++ b/mfi-common
@@ -643,6 +643,7 @@ test_matrix_iterate () {
       *)          echo >&2 "kernbuild ?  $kern"; exit 1 ;;
       esac
 
+      dom0arches=''
       for dom0arch in i386 amd64 armhf arm64; do
 
         case ${xenarch}_${dom0arch} in
@@ -654,6 +655,11 @@ test_matrix_iterate () {
             *) continue ;;
         esac
 
+        dom0arches+=" $dom0arch"
+      done
+
+      for dom0arch in $dom0arches; do
+
         eval "
             arch_runvars=\"\$ARCH_RUNVARS_$dom0arch\"
         "
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183405.331561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCl-0005Da-7q; Thu, 09 Sep 2021 16:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183405.331561; Thu, 09 Sep 2021 16:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCl-0005Cm-2N; Thu, 09 Sep 2021 16:47:15 +0000
Received: by outflank-mailman (input) for mailman id 183405;
 Thu, 09 Sep 2021 16:47:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-00059b-I2
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-0002Mn-HE
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCj-0001RE-G5
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:13 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mONCh-0006u8-Kh; Thu, 09 Sep 2021 17:47:11 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=Pjp5XstcArsqNKuCQ700S1r/NZU8YkJaIMSB03Glj6s=; b=is/HxYZ5tdD/4XdLtxcfsSUD4V
	m/f5Z8WweG28os7M9w/987JUgAV+36MSfdEnX4m5vX1YJQ6QTOvXU4yknmn7cJBXKAXghjNmaqZ/V
	NXHItVLbxFO8H3sou0HmuCTsahhbB92HewKsxc3nOx7NDBVAS5Wvgj9nNRh/d3wi7CaI=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 2/4] mfi-common: Change dom0arch list order (nfc)
Date: Thu,  9 Sep 2021 17:47:01 +0100
Message-Id: <20210909164703.19426-2-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210909164703.19426-1-iwj@xenproject.org>
References: <20210909164703.19426-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Change the order to match fmtarches_outer in do_pv_debian_tests.

We are going to want to add an indirection here but not change
anything.

No change to output from standalone-generate-dump-flight-runvars.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mfi-common | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mfi-common b/mfi-common
index 70e17f7a..02278420 100644
--- a/mfi-common
+++ b/mfi-common
@@ -644,7 +644,7 @@ test_matrix_iterate () {
       esac
 
       dom0arches=''
-      for dom0arch in i386 amd64 armhf arm64; do
+      for dom0arch in i386 armhf amd64 arm64; do
 
         case ${xenarch}_${dom0arch} in
             amd64_amd64) ;;
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 16:47:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 16:47:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183407.331577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCm-0005XS-8P; Thu, 09 Sep 2021 16:47:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183407.331577; Thu, 09 Sep 2021 16:47:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONCm-0005W1-1O; Thu, 09 Sep 2021 16:47:16 +0000
Received: by outflank-mailman (input) for mailman id 183407;
 Thu, 09 Sep 2021 16:47:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-00059s-L0
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-0002Mv-KF
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mONCk-0001Rn-J1
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 16:47:14 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mONCi-0006u8-NO; Thu, 09 Sep 2021 17:47:12 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=2+4QD+hpkm6kgypvB1lp50M/jnKqf3BdU/Qs7r96jAo=; b=NiSJ7mnc1wqpzNMdNCM10IPLou
	0aZqHSSXr7OvBgXjN13JrcAsaMfwD6aX1eIh0l9p+Q/fJUDAbcx7g+msAixTscKeuTUWUsqfdYEoY
	dj0o7wU+7y/VOj9EYoG7qae+oC2YWJKcO1xiWiwbMXRJE5HhR+Rj/tzE+Cl5eF+17uVg=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [OSSTEST PATCH 4/4] mfi-common: Drop Linux dom0 i386 tests for newer Linux branches
Date: Thu,  9 Sep 2021 17:47:03 +0100
Message-Id: <20210909164703.19426-4-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210909164703.19426-1-iwj@xenproject.org>
References: <20210909164703.19426-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This makes radical changes to the test ste for the linux-linus and
linux-next branches.

Mostly, tests are dropped but some 64-bit dom0 tests are added to
replace them.

Requested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mfi-common | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/mfi-common b/mfi-common
index 02278420..771843f0 100644
--- a/mfi-common
+++ b/mfi-common
@@ -655,6 +655,11 @@ test_matrix_iterate () {
             *) continue ;;
         esac
 
+        case "${branch}_${dom0arch}" in
+          linux-5.4_i386 | linux-[2-4].*_i386) ;; # keep 32-bit for old linux
+          linux-*_i386) continue;;
+        esac
+
         dom0arches+=" $dom0arch"
       done
 
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 17:04:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 17:04:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183432.331606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONSv-0001SH-J2; Thu, 09 Sep 2021 17:03:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183432.331606; Thu, 09 Sep 2021 17:03:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONSv-0001SA-Fx; Thu, 09 Sep 2021 17:03:57 +0000
Received: by outflank-mailman (input) for mailman id 183432;
 Thu, 09 Sep 2021 17:03:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mYjb=N7=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mONSt-0001S4-GU
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 17:03:55 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 689a7643-5b64-4972-bb69-be7388a6cca5;
 Thu, 09 Sep 2021 17:03:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 689a7643-5b64-4972-bb69-be7388a6cca5
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631207034;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=3aARRyjjI0EjxYyqpL6Ioai4rZieo2Y1y2K/3KUz/Z8=;
  b=T6nU1qG/JHIibj+YBdW5V05Y2Y4kdvC5hcHGi/uTo55Ibi8eY5z5xh/Q
   5tQYKRtLgF/83vLyusmf7U/mu8acyXu3h7Sq7A7AeCqs1nZxPhJ7aDJ0R
   U4fppKk54LkPkamnIMMpmFX+QDRCyeN/jHe3559/HK/2ihsnL6e/p0BRu
   8=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: POJUicAwHf6ZMjosqRR4nOMTtTvVOnUmgQN2qLVIb0hoJzK4xNj4gurq4No581Vbi/cNoaBmNV
 CjVEnoiOCpSqHELjkUKAGHqyZxPANOhOpB0+5otUYbokj1hwNekOF4qnHw3fBzHbprRybzSTxz
 BF1vzxW4Fe7hexxeIvmY9K619kj2t5jOFJtAJoCweijD0BS6vLwnCa4EOjFCn93i7uPF3/FvQQ
 X3gyBfpyz1OaaKe3dS0zZxs0NsejOg86XF6VCn8jnVFABBCLxXnJpa11UZwcTFi6ER/vs0fZpA
 5Nh30JLqs5xBt5FIyQvODLVa
X-SBRS: 5.1
X-MesageID: 52784178
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:ewR4WK/kVWEjlO/IvkFuk+FPdb1zdoMgy1knxilNoENuHfBwxv
 rDoB1E73LJYVYqOU3Jmbi7Scy9qADnhOFICO4qTMuftWjdyRaVxeRZg7cKrAeQYxEWmtQtsp
 uINpIOcuEYbmIK/foSgjPIaurIqePvmMvD5Za8vgRQpENRGsVdBm9Ce3em+yZNNW977PQCZf
 ihD4Z81kGdkSN9VLXEOpBJZZmOm/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTsj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZuA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQPZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv2nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLbU5nghBgr/DWQZAV2Iv/fKXJy/fB9kgIm3UyR9nFohvD2xRw7hdQAo5ot3Z
 WNDk0nrsAWcie6BZgNc9vpevHHf1Aldyi8eV56EW6XZp3vBEi936IfwI9Frt1CK6Z4gafbpv
 z6ISVlXCgJChrTNfE=
X-IronPort-AV: E=Sophos;i="5.85,280,1624334400"; 
   d="scan'208";a="52784178"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g/OSEbZ5B4g8Yv4wmKDU+zXx/q/++1FpqTY2vmG3q6+W8hOSO3ocwyY3fLngqBeCds5nzW/jtRZ/+ckBF+blxjjBvx79RIulfUifdhiosqzhQ8izVQsIHXzJ8b5v4Wo0tsLvddK93aNRhjscT4P6Fim86hpd+dWMWlp1Rj49hb8icJiUMac9qcQkXDnwhouO8hIUQyrumD9uGAnFICoRlI35XhlOIMnbBRWm8iQru0ySp4X5jjEsaGDVKmaiXCkgfKCO/yJfaYWjmClMPTNhbEfRpk3FG6E/Mc03B+7Wo1q/B4R/3PELAuY5bnvWRLjcnZ0CBoymBNR3TT+KQ1dqlQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3aARRyjjI0EjxYyqpL6Ioai4rZieo2Y1y2K/3KUz/Z8=;
 b=nuxoUclUFf+RRNWW0e1sYFbYvgxEdw91WKJOeAU+ubw/QM0Li51MOxS+PtUsvW5iRibK9DpdCZ2EWw+XC6ohCl47rS7c/AtQVk7cUXwVRfZC7rpUGuzYZr6YoGb/WO+bvPmPNQJmj1QYm+Mj1bkVmjPi5JSA3wZMhjDqZHgW50jf/91L1B73Y7Adg7+ALWdfpqdEMi5FOk9H635M9+sgIVIWlN7puStPwgxYyN4mfJV9D3fXqsJlVPME6559shR8Kwa9cd8m3T+hpgG/dPaLCTqZqQvrcdPWxK23zUKPRT39OwCyLnDeJEWCSFB/BO45gOs410jLHrgbM18y9GxZkQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3aARRyjjI0EjxYyqpL6Ioai4rZieo2Y1y2K/3KUz/Z8=;
 b=Jo9huMSK3yoY9ZjhWl28YCOdeZEG7S4fjwPO34MDkiOors5sv5GedO3sVfQSCYAz9VpvDp4dp/dHjKj/F80mJ9ezV/xT4tleegiQkDziPvGTePbZ7C0mvy9MgDJlrlpwd+fJtgl5+srI48IDN2VMw+BJ2oAjSWMaCwiKo3Qjfok=
To: Kevin Stefanov <kevin.stefanov@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
 <20210909163441.44418-2-kevin.stefanov@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS
Message-ID: <bbde0f13-a1a5-53ed-82eb-704cd5d43178@citrix.com>
Date: Thu, 9 Sep 2021 18:03:39 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210909163441.44418-2-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0483.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a8::20) To BN7PR03MB3618.namprd03.prod.outlook.com
 (2603:10b6:406:c3::27)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 486b3e85-393e-42ed-b3fc-08d973b3c81b
X-MS-TrafficTypeDiagnostic: BN8PR03MB4947:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BN8PR03MB49475B9CDED52562E5EF72D0BAD59@BN8PR03MB4947.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: TwFA/1jy7r4+46bHOnvAIBUSUfm68O6C+4U9/IFYpDqjkzgiJAQWPm0FZrj6K37Q36O2jUIxDu8GCYGMh14pMxIOEkc3hwPw35pVZjrNE/AQkwFHIqv3gy2KJbSDFPd3vmERYczzxYigQSlZ0ekkpPQs/FbrLIvhm8CpJDiDRBKjhl8n6GeXne5fs2Lwnob3JWwNzBrVZELE+0Fmm6JvpaQWAOwPsr+gOS7pVLOVzBEL7ht7Y/pCyHW/Wlf4grlaiAsgmZxfUFIgqgNxtyb0UxrkSaBOy0RGPdb4qjUIK9vNtmtwHN7In79203XUX0+tPC9KKdD5z+w+S40T3sNO5o3Sp4eUzJ2YKGF8WlJbIbaiqahEPMoXW0PQKlhzX5cXNR5NrwZIfe53mlq3jspcB9UxtYR3T/nb9tqEAofLbmfDeqP4+WL1VLaDMTTgHCkRQsvhybZdaQSsXXtTScDkLqbqT0uVROUASMWxCY2MSR5xAdSxsFx1wNePJAsxfj795jHFCRwAcyPbMkGE+Layu2dyoAsj54c7bFHcwQhaf0H+OCFkEJ1b7mUxwi+x64coAzSsVTa//nNjsVnvJZ9TU2+K5j+BEQLEbINUArd9h0qo2SiT/NwQ2uIDGZPJOFtgDXSTCsP/dCKzN2zmqLyVSDhQJPmmuedPPA50mqLGTKJnNZtlkQyJzg9lVmBlPfB58EoOrqGxU62zUBQBUqKM/AlxrJt3C5PgVAXE6EvlNTI=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN7PR03MB3618.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39860400002)(396003)(366004)(376002)(136003)(66946007)(478600001)(66476007)(66556008)(8936002)(55236004)(31686004)(6666004)(8676002)(6486002)(31696002)(38100700002)(186003)(26005)(956004)(86362001)(2616005)(5660300002)(16576012)(110136005)(83380400001)(36756003)(316002)(53546011)(2906002)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WVRYSDJYTVR3UHprSmF0MThSYVpJajBwRk5LNXpSN3pkYXR5ekJCV1NoZmNK?=
 =?utf-8?B?Y1VFTjlOYU51dkowU0loV1UvZC9sWEpleWlnaW1XVlpneGEvOElDWUx3NjNl?=
 =?utf-8?B?UHdYL1h5aW1wR2ovdmhqUW5hTzhnVW5XL0NMYTFwM1VxV1YyV2dBdDhnWko5?=
 =?utf-8?B?Z0NXVDBKWUFKeFNSSzE1T1Z5TFNLU25NeEpjbzVLMTRzbEZKTzE5ZisybzF6?=
 =?utf-8?B?M1pPU0RCdDJWSlJwTGVzMnRYa3Ntd3MzWVN2cWYwcWcrUlNpVlFTMXhaVFRE?=
 =?utf-8?B?S2pESXU4NExpR2Jsc1V6amVONklmcnlkaUsyeTJ3UEJnYnFiMGdpYk5vRzRi?=
 =?utf-8?B?YTVXcm9xU0FORk5sSmhVM1VHOHY5UVdnSHRiZXczOWthbEJmQTRTM3JTNk5x?=
 =?utf-8?B?T25rYkRaZkovWDhYMmVvT0dJa2tBd0tJWVFGL1FNYmsrM0RGcWNQUDhnS1hw?=
 =?utf-8?B?eUNhd3NVR3BITHlKRFVPa3N6UTBIV2wyYjUxRkRQOFVwaGdPZkM3eldSK1lE?=
 =?utf-8?B?cU44Sy9oRW9ickZRM08zSFFSbzQzeUw2REZPZUhCRUpxd0thazI3cko3NjQ0?=
 =?utf-8?B?Y25tR3FuRFFVUFJJSmc4Z0MvMU5ucXlNUnVCOVB0VGp4ZmhBbFM1c21IWmZN?=
 =?utf-8?B?VzNQMUI2QytFN1pSc0lZeHR1WDJmTDhzRTR1eXErdzNJc0U2OUhSc3haUkdp?=
 =?utf-8?B?elNia250bnZrRVhwMFEwUHpxQmZiWHJSemZUL2ZmbWlmczM0dzk2VEl5T3ZW?=
 =?utf-8?B?aVVWZThTVnByTmRwRUZtQXRaQllyWVY1amR5cFRSa1VlZVloRVVIZExMUTVl?=
 =?utf-8?B?ZnR3OGo2OWtzd1Nadmk2NG9hYmhUZy9PTGVPcXZPQUVjTzdOOVFqREFGUTJy?=
 =?utf-8?B?YUZ2ZzV2bTR2NUV4NCt0MC9RamNoM1BrQlpLbzVXS1F6bDQ3ZkV1MzhYYkJy?=
 =?utf-8?B?cGZNZGhzTHdCWkxmK2ttdG1RTklMOVFOdHgyZ3Q0d3dGRHZUNGJCcGt2aHJP?=
 =?utf-8?B?UlE3ZEJYaWVVL3J4dDVCR1VFakNLcW8yUGU1RmxCR2pRSTc3Q2F1VVdlY0gx?=
 =?utf-8?B?OGI3SGFpN3J1bDZMcGN2MytLWHhVd01VdnJyYlBFTm51NW5DSVgzNGFpanI5?=
 =?utf-8?B?SngwSFBNZlFCTEJocGtKci9LK2IrNVZBL1VmaWMrYmRqWlRST3d4RlB4cC93?=
 =?utf-8?B?M3c2Y0h0SURzWTlsVmMvd040YlJ1L21uOWM4bXVHSUtQRGxERW5udXhLR2RJ?=
 =?utf-8?B?S1FnRFRGUjNXOFo4by9aZ252NCtUdDdad2x5RnpqN1ZxUG9GdEx4THNGVERj?=
 =?utf-8?B?dWVPamd2bzZ6enpSSEhCZ1lVL2hRS0FMZ1BQbzQ1bHpXdEpzQVFrNGVOb2lw?=
 =?utf-8?B?dC9RT25DR21RMDZmQlRHOFFxeFptWTk0MXlhSUo0ZGExM0RUNnVxZWdxQW9p?=
 =?utf-8?B?cVZaYUFsY2xEbVBCaWxqNjVWNUEvZXZJdWtQeHh4M2ZvaVhOQldVQktuSlRB?=
 =?utf-8?B?NklZNGtwTWJ3RHU4OG5hTDZ6Z25yT29GYWlGelpQM2hlajdBOTk1QWtxNjJ4?=
 =?utf-8?B?bWJCMzEwMUZGL3QrVE1Pa0FRUys0MG1yeUhpdGU3ak5DOSsycUprSzdHWk9s?=
 =?utf-8?B?RHQ3RG0zcXM3eEI0Yzc3MXBybXY4SmpMQ21mY3M3L1ZjMUg3Y1VZTWlDdDF6?=
 =?utf-8?B?MTR0WkdXQzllUTZHK1JIQkVoenRsUTNXaUl3anFmU1p6STVvVDZIaW54OHlp?=
 =?utf-8?Q?b/ZjtS8pZlfcH/TguXUzqocBZbreaG0pCk4COmz?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 486b3e85-393e-42ed-b3fc-08d973b3c81b
X-MS-Exchange-CrossTenant-AuthSource: BN7PR03MB3618.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 17:03:44.6343
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3l9BTpUq1uXp6z8pK2m/TbEq33amSw0NlNGC4qGqxqowDAGTuP5WTvNO7xHxSbrfB2ezgmVsoITOQ5+jG/D+Yy0fyBfZQOgOlRCqNoPHMJE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR03MB4947
X-OriginatorOrg: citrix.com

On 09/09/2021 17:34, Kevin Stefanov wrote:
> The spec requires 64-byte alignment, not 16.
>
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
>
> Note: This does not fix the FACS alignment inside guests yet. See next
> patch.

The history here is complex.

c/s 938cee9d41d3 in 2006 deleted a comment saying

// FACS: should be 64-bit alignment
// so it is put at the start of buffer
// as the buffer is 64 bit alignment

Clearly it means byte rather than bit, but the change also introduced
logic to the effect of buf +=3D ROUNDUP(sizeof(facs), 16).

This (incorrect) alignment survived several morphs of the logic and was
moved from hvmloader into libacpi by c/s 73b72736e6ca.

The current code is clearly wrong, but happens to work correctly in
hvmloader because FACS is the first table written and it starts on a
page boundary.=C2=A0 The logic does not work correctly when libxl passes a
buffer which doesn't start on a page boundary.

As a consequence, I'm not sure what to suggest as a Fixes tag here,
except "the logic has been wrong for 15 years - patch everything".

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 17:19:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 17:19:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183438.331617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONhe-0003FT-Uh; Thu, 09 Sep 2021 17:19:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183438.331617; Thu, 09 Sep 2021 17:19:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mONhe-0003FM-RG; Thu, 09 Sep 2021 17:19:10 +0000
Received: by outflank-mailman (input) for mailman id 183438;
 Thu, 09 Sep 2021 17:19:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mONhd-0003FG-OH
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 17:19:09 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mONhb-0002xR-2E; Thu, 09 Sep 2021 17:19:07 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mONha-0008Fx-Rx; Thu, 09 Sep 2021 17:19:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=DSXntFz1K6MMSBxiwjqUnifCkDzWAFsf49MNV7vK8WI=; b=KoQ9pphil1aCm0O0mSFqlVLMH2
	EBerKzE3ezYFlElJ8RvYLRDseiIgwwMSvv1g2pxvvu+7h40HaQllQm9PaU4Lw8eCraciGLrLV8cM7
	Wkps7de0vxejnGqCA0us+4Yx3lDa/6udY4eKJRmfz3ZKXKREafyIW+KM+Ts1zKY1Sqiw=;
Subject: Re: [PATCH 01/11] xen/arm: Add new device type for PCI
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-2-andr2000@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <5a32422f-0782-342d-911b-116af2c85859@xen.org>
Date: Thu, 9 Sep 2021 18:19:04 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210903083347.131786-2-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add new device type (DEV_PCI) to distinguish PCI devices from platform
> DT devices, so some drivers, like IOMMU, can handle PCI devices
> differently.

I think it would be better to fold this change in the next patch as this 
is where you add all the helpers for converting dev to PCI.

> 
> While at it fix dev_is_dt macro.

I would keep this change separate. It also needs an explanation of 
what's the problem and mention it is a latent bug because no-one use it 
(so we know this doesn't require a backport).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 17:41:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 17:41:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183444.331628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOO38-00076S-PD; Thu, 09 Sep 2021 17:41:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183444.331628; Thu, 09 Sep 2021 17:41:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOO38-00076L-Ls; Thu, 09 Sep 2021 17:41:22 +0000
Received: by outflank-mailman (input) for mailman id 183444;
 Thu, 09 Sep 2021 17:41:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOO37-00076B-Ko; Thu, 09 Sep 2021 17:41:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOO37-0003Lf-Av; Thu, 09 Sep 2021 17:41:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOO37-00033t-38; Thu, 09 Sep 2021 17:41:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOO37-0001km-2a; Thu, 09 Sep 2021 17:41:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Ii02lmlmWpaFV7SJLX5veJCJzfFYGFVZKdfbYRkhnN8=; b=NFzcfHWSxagSkFMFLqsgKOtNJ7
	FusDaoB+7Yd5TDpGNH/ZP/Pv8HuUhYV7kAIBBOicfV/azBohJyD5R/ujI0N2dkSZHCJbubNfw+Eap
	0R8/krXQM9YvOeUcqtljVfxI2F+6jC4CSEgwvyb4TrFSnIc60Ciht1he+ZL+fXm7MVgs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164895-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164895: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=fa8ce012a328701054f0cbd191f792589793ea13
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 17:41:21 +0000

flight 164895 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164895/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              fa8ce012a328701054f0cbd191f792589793ea13
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  426 days
Failing since        151818  2020-07-11 04:18:52 Z  425 days  416 attempts
Testing same since   164895  2021-09-09 04:18:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 72201 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 17:43:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 17:43:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183452.331642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOO4y-0007nQ-Ca; Thu, 09 Sep 2021 17:43:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183452.331642; Thu, 09 Sep 2021 17:43:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOO4y-0007nJ-9K; Thu, 09 Sep 2021 17:43:16 +0000
Received: by outflank-mailman (input) for mailman id 183452;
 Thu, 09 Sep 2021 17:43:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOO4x-0007nD-Ai
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 17:43:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOO4u-0003NE-Iy; Thu, 09 Sep 2021 17:43:12 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOO4u-0001tu-BO; Thu, 09 Sep 2021 17:43:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=HirYJ1RJd/TiLDXKH9u8ITgTJ18WeYSi95VgsTPECEA=; b=vq9wL135Sc++G+ZLEF+EIeBPZw
	P7o+G3Rhj8WMhwrC5OLUiMs+M8ZKDShDfm7l7ogDuFD6wZNHmRqgoUMbePr5bJsH1OgWbhHCWH8GG
	DGrIraohQqgKUEiDjYTVoxPK8qeqAD8O6LQf4wTWdeRaZjH1lEGX9A3d2Cw9m/RWLg1k=;
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
Date: Thu, 9 Sep 2021 18:43:09 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210903083347.131786-10-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> In order vPCI to work it needs all access to PCI configuration space
> (ECAM) to be synchronized among all entities, e.g. hardware domain and
> guests.

I am not entirely sure what you mean by "synchronized" here. Are you 
refer to the content of the configuration space?

> For that implement PCI host bridge specific callbacks to
> properly setup those ranges depending on particular host bridge
> implementation.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>   xen/arch/arm/pci/ecam.c            | 11 +++++++++++
>   xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
>   xen/arch/arm/vpci.c                | 13 +++++++++++++
>   xen/include/asm-arm/pci.h          |  8 ++++++++
>   4 files changed, 48 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> index 91c691b41fdf..92ecb2e0762b 100644
> --- a/xen/arch/arm/pci/ecam.c
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>       return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
>   }
>   
> +static int pci_ecam_register_mmio_handler(struct domain *d,
> +                                          struct pci_host_bridge *bridge,
> +                                          const struct mmio_handler_ops *ops)
> +{
> +    struct pci_config_window *cfg = bridge->sysdata;
> +
> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);

We have a fixed array for handling the MMIO handlers. So you need to 
make sure we have enough space in it to store one handler per handler.

This is quite similar to the problem we had with the re-distribuor on 
GICv3. Have a look there to see how we dealt with it.

> +    return 0;
> +}
> +
>   /* ECAM ops */
>   const struct pci_ecam_ops pci_generic_ecam_ops = {
>       .bus_shift  = 20,
> @@ -49,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>           .map_bus                = pci_ecam_map_bus,
>           .read                   = pci_generic_config_read,
>           .write                  = pci_generic_config_write,
> +        .register_mmio_handler  = pci_ecam_register_mmio_handler,
>       }
>   };
>   
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index d2fef5476b8e..a89112bfbb7c 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -318,6 +318,22 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
>       }
>       return bridge->dt_node;
>   }
> +
> +int pci_host_iterate_bridges(struct domain *d,
> +                             int (*clb)(struct domain *d,

NIT: We tend to name callback variable 'cb'.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 17:58:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 17:58:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183458.331653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOOJf-0001AL-M4; Thu, 09 Sep 2021 17:58:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183458.331653; Thu, 09 Sep 2021 17:58:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOOJf-0001AE-Iz; Thu, 09 Sep 2021 17:58:27 +0000
Received: by outflank-mailman (input) for mailman id 183458;
 Thu, 09 Sep 2021 17:58:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOOJd-0001A8-K0
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 17:58:25 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOOJa-0003bx-Gs; Thu, 09 Sep 2021 17:58:22 +0000
Received: from [54.239.6.185] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOOJa-00032I-A9; Thu, 09 Sep 2021 17:58:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Yy4yoOqlTNdTU72q+wjtOFjBk8vaQvOE39O2MrR64RA=; b=pNoIPZWn/oLTsa8jGEQX4iPG1z
	HwqMS3iY3PxPvyHA64R6PQN16PktejzE+VbqG1ExkRS6lac2DKZh+I8Sin3W4sy4tFMlOYkKSdHtW
	qu2TMBg5kb8+zA7PXB4V81HXle1iBAny1uu77t2dGPSWeblbEw1tcbDTp6vr76PFKHxo=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
Date: Thu, 9 Sep 2021 18:58:19 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210903083347.131786-11-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Host bridge controller's ECAM space is mapped into Domain-0's p2m,
> thus it is not possible to trap the same for vPCI via MMIO handlers.
> For this to work we need to not map those while constructing the domain.
> 
> Note, that during Domain-0 creation there is no pci_dev yet allocated for
> host bridges, thus we cannot match PCI host and its associated
> bridge by SBDF. Use dt_device_node field for checks instead.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>   xen/arch/arm/domain_build.c        |  3 +++
>   xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
>   xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
>   xen/include/asm-arm/pci.h          | 12 ++++++++++++
>   4 files changed, 54 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index da427f399711..76f5b513280c 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
>           }
>       }
>   
> +    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI) ) > +        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev, 
addr, len);

AFAICT, with device_get_class(dev), you know whether the hostbridge is 
used by Xen. Therefore, I would expect that we don't want to map all the 
regions of the hostbridges in dom0 (including the BARs).

Can you clarify it?

> + >       if ( need_mapping )
>       {
>           res = map_regions_p2mt(d,
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> index 92ecb2e0762b..d32efb7fcbd0 100644
> --- a/xen/arch/arm/pci/ecam.c
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -52,6 +52,22 @@ static int pci_ecam_register_mmio_handler(struct domain *d,
>       return 0;
>   }
>   
> +static int pci_ecam_need_p2m_mapping(struct domain *d,
> +                                     struct pci_host_bridge *bridge,
> +                                     uint64_t addr, uint64_t len)
> +{
> +    struct pci_config_window *cfg = bridge->sysdata;
> +
> +    if ( !is_hardware_domain(d) )
> +        return true;

I am a bit puzzled with this check. If the ECAM has been initialized by 
Xen, then I believe we cannot expose it to any domain at all.

> +
> +    /*
> +     * We do not want ECAM address space to be mapped in domain's p2m,
> +     * so we can trap access to it.
> +     */
> +    return cfg->phys_addr != addr;
> +}
> +
>   /* ECAM ops */
>   const struct pci_ecam_ops pci_generic_ecam_ops = {
>       .bus_shift  = 20,
> @@ -60,6 +76,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>           .read                   = pci_generic_config_read,
>           .write                  = pci_generic_config_write,
>           .register_mmio_handler  = pci_ecam_register_mmio_handler,
> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>       }
>   };
>   
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index a89112bfbb7c..c04be636452d 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -334,6 +334,28 @@ int pci_host_iterate_bridges(struct domain *d,
>       }
>       return 0;
>   }
> +
> +bool pci_host_bridge_need_p2m_mapping(struct domain *d,
> +                                      const struct dt_device_node *node,
> +                                      uint64_t addr, uint64_t len)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->dt_node != node )
> +            continue;
> +
> +        if ( !bridge->ops->need_p2m_mapping )
> +            return true;
> +
> +        return bridge->ops->need_p2m_mapping(d, bridge, addr, len);
> +    }
> +    printk(XENLOG_ERR "Unable to find PCI bridge for %s segment %d, addr %lx\n",
> +           node->full_name, bridge->segment, addr);
> +    return true;
> +}

If you really need to map the hostbridges, then I would suggest to defer 
the P2M mappings for all of them and then walk all the bridge in one go 
to do the mappings.

This would avoid going through the bridges every time during setup.

> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 2c7c7649e00f..9c28a4bdc4b7 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -82,6 +82,8 @@ struct pci_ops {
>       int (*register_mmio_handler)(struct domain *d,
>                                    struct pci_host_bridge *bridge,
>                                    const struct mmio_handler_ops *ops);
> +    int (*need_p2m_mapping)(struct domain *d, struct pci_host_bridge *bridge,
> +                            uint64_t addr, uint64_t len);
>   };
>   
>   /*
> @@ -115,9 +117,19 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
>   int pci_host_iterate_bridges(struct domain *d,
>                                int (*clb)(struct domain *d,
>                                           struct pci_host_bridge *bridge));
> +bool pci_host_bridge_need_p2m_mapping(struct domain *d,
> +                                      const struct dt_device_node *node,
> +                                      uint64_t addr, uint64_t len);
>   #else   /*!CONFIG_HAS_PCI*/
>   
>   struct arch_pci_dev { };
>   
> +static inline bool
> +pci_host_bridge_need_p2m_mapping(struct domain *d,
> +                                 const struct dt_device_node *node,
> +                                 uint64_t addr, uint64_t len)
> +{
> +    return true;
> +}
>   #endif  /*!CONFIG_HAS_PCI*/
>   #endif /* __ARM_PCI_H__ */
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 18:45:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 18:45:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183487.331687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOP3D-00007k-Kw; Thu, 09 Sep 2021 18:45:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183487.331687; Thu, 09 Sep 2021 18:45:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOP3D-00007d-HO; Thu, 09 Sep 2021 18:45:31 +0000
Received: by outflank-mailman (input) for mailman id 183487;
 Thu, 09 Sep 2021 18:45:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Llr=N7=redhat.com=eblake@srs-us1.protection.inumbo.net>)
 id 1mOP3B-00007D-MQ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 18:45:29 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 19b4977a-119e-11ec-b1e5-12813bfff9fa;
 Thu, 09 Sep 2021 18:45:27 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-319-BpmAA2n0MWGi-nGcYVOKCg-1; Thu, 09 Sep 2021 14:45:25 -0400
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
 [10.5.11.14])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C20C28B8D3F;
 Thu,  9 Sep 2021 18:45:06 +0000 (UTC)
Received: from redhat.com (ovpn-112-181.phx2.redhat.com [10.3.112.181])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 892105DA2D;
 Thu,  9 Sep 2021 18:45:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19b4977a-119e-11ec-b1e5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631213127;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KlL3i7aLjbTCNV8w6dHeoPvCpP3Y0rLMHrE4Kn6x5oI=;
	b=ZOraNPqIXQjRXxBuxULpzSORwtl8xMm64RwsRENy6CqaNzVv+4Wc6sFUNj/U/P96ukkWMQ
	so5OSsnpsQmnfOLDsVZbtqpQGngh+8pQQiCsj0jnX9qDDoCsQJWXLGnWf1jgl1lJ6v9HZ9
	oJ3YReYY3bukffTNL4mAx53zHr6dv64=
X-MC-Unique: BpmAA2n0MWGi-nGcYVOKCg-1
Date: Thu, 9 Sep 2021 13:45:03 -0500
From: Eric Blake <eblake@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 01/10] sysemu: Introduce qemu_security_policy_taint()
 API
Message-ID: <20210909184503.mjg37ckq3ga4fzag@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-2-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-2-philmd@redhat.com>
User-Agent: NeoMutt/20210205-739-420e15
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:15AM +0200, Philippe Mathieu-Daudé wrote:
> Introduce qemu_security_policy_taint() which allows unsafe (read
> "not very maintained") code to 'taint' QEMU security policy.
> 
> The "security policy" is the @SecurityPolicy QAPI enum, composed of:
> - "none"   (no policy, current behavior)
> - "warn"   (display a warning when the policy is tainted, keep going)
> - "strict" (once tainted, exit QEMU before starting the VM)
> 
> The qemu_security_policy_is_strict() helper is also provided, which
> will be proved useful once a VM is started (example we do not want

s/be proved/prove/

> to kill a running VM if an unsafe device is hot-added).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  qapi/run-state.json   | 16 +++++++++++
>  include/qemu-common.h | 19 ++++++++++++
>  softmmu/vl.c          | 67 +++++++++++++++++++++++++++++++++++++++++++
>  qemu-options.hx       | 17 +++++++++++
>  4 files changed, 119 insertions(+)
> 
> diff --git a/qapi/run-state.json b/qapi/run-state.json
> index 43d66d700fc..b15a107fa01 100644
> --- a/qapi/run-state.json
> +++ b/qapi/run-state.json
> @@ -638,3 +638,19 @@
>  { 'struct': 'MemoryFailureFlags',
>    'data': { 'action-required': 'bool',
>              'recursive': 'bool'} }
> +
> +##
> +# @SecurityPolicy:
> +#
> +# An enumeration of the actions taken when the security policy is tainted.
> +#
> +# @none: do nothing.
> +#
> +# @warn: display a warning.
> +#
> +# @strict: prohibit QEMU to start a VM.

s/to start/from starting/

> +#
> +# Since: 6.2
> +##
> +{ 'enum': 'SecurityPolicy',
> +  'data': [ 'none', 'warn', 'strict' ] }

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 18:46:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 18:46:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183493.331699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOP4T-0000iC-W8; Thu, 09 Sep 2021 18:46:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183493.331699; Thu, 09 Sep 2021 18:46:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOP4T-0000i5-Sl; Thu, 09 Sep 2021 18:46:49 +0000
Received: by outflank-mailman (input) for mailman id 183493;
 Thu, 09 Sep 2021 18:46:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Llr=N7=redhat.com=eblake@srs-us1.protection.inumbo.net>)
 id 1mOP4S-0000hz-NP
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 18:46:48 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id e82ba8eb-09fc-4eaf-9218-735e4a13c8a6;
 Thu, 09 Sep 2021 18:46:48 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-250-C_EXj8nOOXekSgqTKAXzYA-1; Thu, 09 Sep 2021 14:46:44 -0400
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
 [10.5.11.16])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C1F44A0CAC;
 Thu,  9 Sep 2021 18:46:43 +0000 (UTC)
Received: from redhat.com (ovpn-112-181.phx2.redhat.com [10.3.112.181])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id ADCE05C1C5;
 Thu,  9 Sep 2021 18:46:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e82ba8eb-09fc-4eaf-9218-735e4a13c8a6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631213207;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0pzbBYCWqbgRW+ufRHz+MM4Oqg466o5liD9bply3WTM=;
	b=ULBp7teNuu5Ig+Xt0R9o2nskE/7wmT4lubyvgY73I8RU4KjLKRmzoWLV3R1qjbCjjW+pAE
	1JMO7Hx4TExtbQO5SDlH/jzX+fFGpneCs9NGno2tFoZ3WQqL461Al0wsAuWLZSuBUIce5M
	mdqXd2Uzu8DYxkmwrXAnnrkeIT3n+dc=
X-MC-Unique: C_EXj8nOOXekSgqTKAXzYA-1
Date: Thu, 9 Sep 2021 13:46:37 -0500
From: Eric Blake <eblake@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 02/10] accel: Use qemu_security_policy_taint(), mark
 KVM and Xen as safe
Message-ID: <20210909184637.pmb6mo26ss6ldnoa@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-3-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-3-philmd@redhat.com>
User-Agent: NeoMutt/20210205-739-420e15
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:16AM +0200, Philippe Mathieu-Daudé wrote:
> Add the AccelClass::secure_policy_supported field to classify
> safe (within security boundary) vs unsafe accelerators.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/qemu/accel.h | 5 +++++
>  accel/kvm/kvm-all.c  | 1 +
>  accel/xen/xen-all.c  | 1 +
>  softmmu/vl.c         | 3 +++
>  4 files changed, 10 insertions(+)
> 
> diff --git a/include/qemu/accel.h b/include/qemu/accel.h
> index 4f4c283f6fc..895e30be0de 100644
> --- a/include/qemu/accel.h
> +++ b/include/qemu/accel.h
> @@ -44,6 +44,11 @@ typedef struct AccelClass {
>                         hwaddr start_addr, hwaddr size);
>  #endif
>      bool *allowed;
> +    /*
> +     * Whether the accelerator is withing QEMU security policy boundary.

within

> +     * See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool secure_policy_supported;
>      /*
>       * Array of global properties that would be applied when specific
>       * accelerator is chosen. It works like MachineClass.compat_props

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 19:05:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 19:05:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183504.331710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPMW-0003eA-Lb; Thu, 09 Sep 2021 19:05:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183504.331710; Thu, 09 Sep 2021 19:05:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPMW-0003e3-GI; Thu, 09 Sep 2021 19:05:28 +0000
Received: by outflank-mailman (input) for mailman id 183504;
 Thu, 09 Sep 2021 19:05:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1Llr=N7=redhat.com=eblake@srs-us1.protection.inumbo.net>)
 id 1mOPMU-0003dx-S0
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 19:05:26 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e414308c-11a0-11ec-b1ea-12813bfff9fa;
 Thu, 09 Sep 2021 19:05:26 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-185-LUX5BKbeOsCm86MKlCjUSg-1; Thu, 09 Sep 2021 15:05:23 -0400
Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com
 [10.5.11.22])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E9C581084680;
 Thu,  9 Sep 2021 19:05:21 +0000 (UTC)
Received: from redhat.com (ovpn-112-181.phx2.redhat.com [10.3.112.181])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id D696A100EBBF;
 Thu,  9 Sep 2021 19:05:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e414308c-11a0-11ec-b1ea-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1631214325;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=G51EAgedhr1FxW4ZOe/06ann1cdm4Vk3p7x/R0Gk14c=;
	b=hBjdeHJO7XxtaYty7OZkBsSibgjmiLNueQhMgWSqz8oLfnTa2XPrDMhCIMgHfQ4LRY9OmO
	k4O469p+piehf2kJTgD/MPZrNvn03pNNW/Ud+l5kImJ69F2MFCAAC/YT3bePsAF5q8SLUm
	xCG3Nv8BfA5XiR/MgH872wrshQGUPqA=
X-MC-Unique: LUX5BKbeOsCm86MKlCjUSg-1
Date: Thu, 9 Sep 2021 14:05:19 -0500
From: Eric Blake <eblake@redhat.com>
To: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Cc: qemu-devel@nongnu.org, Thomas Huth <thuth@redhat.com>,
	Prasad J Pandit <pjp@fedoraproject.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	qemu-block@nongnu.org, Peter Maydell <peter.maydell@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [RFC PATCH 03/10] block: Use qemu_security_policy_taint() API
Message-ID: <20210909190519.il2urwstbivtjktc@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <20210908232024.2399215-4-philmd@redhat.com>
MIME-Version: 1.0
In-Reply-To: <20210908232024.2399215-4-philmd@redhat.com>
User-Agent: NeoMutt/20210205-739-420e15
X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Thu, Sep 09, 2021 at 01:20:17AM +0200, Philippe Mathieu-Daudé wrote:
> Add the BlockDriver::bdrv_taints_security_policy() handler.
> Drivers implementing it might taint the global QEMU security
> policy.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/block/block_int.h | 6 +++++-
>  block.c                   | 6 ++++++
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index f1a54db0f8c..0ec0a5c06e9 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -169,7 +169,11 @@ struct BlockDriver {
>      int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
>                            Error **errp);
>      void (*bdrv_close)(BlockDriverState *bs);
> -
> +    /*
> +     * Return %true if the driver is withing QEMU security policy boundary,

within

> +     * %false otherwise. See: https://www.qemu.org/contribute/security-process/
> +     */
> +    bool (*bdrv_taints_security_policy)(BlockDriverState *bs);
>  
>      int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
>                                         Error **errp);

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 19:14:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 19:14:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183512.331727 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPVT-0005O8-Hb; Thu, 09 Sep 2021 19:14:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183512.331727; Thu, 09 Sep 2021 19:14:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPVT-0005O1-Ed; Thu, 09 Sep 2021 19:14:43 +0000
Received: by outflank-mailman (input) for mailman id 183512;
 Thu, 09 Sep 2021 19:14:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOPVS-0005Nu-Nj
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 19:14:42 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cb8edbae-2c50-41c1-b71b-f62ca653c12d;
 Thu, 09 Sep 2021 19:14:41 +0000 (UTC)
Received: from [10.10.1.24] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631214877401278.2502482860194;
 Thu, 9 Sep 2021 12:14:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb8edbae-2c50-41c1-b71b-f62ca653c12d
ARC-Seal: i=1; a=rsa-sha256; t=1631214880; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=j3L5o5aeeznB4XNecWft0aEDbtDi1ivmqKg28WFHnrepXkKlnTXP5pFhYlOpCahd/8tR32XkK/TDTHzFtqW4jBArczQvmhWXIpAA1Bv/EoOyA4cM+Lw7bOddkCJu5uyOKJ+FIa6YCEChn2cmNFjcWkNkRGovDghpkR0ZjbXfQnA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631214880; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=ODeFCicDvawAcTinn7/rak3JqgqM+ZCyMPqIlQYJDjY=; 
	b=hkKtfQXbJwjMKm6df8JMTr04vGm+nAKXnwRSXIIj/lfam5hkWoGgj8tMLjZXXaG7x9rvTLz9S86acNN/9/p9199jnCRADghNXxE+b1uzxq8w5YdPlDRdrZorHi8q8a3ugTpgBTlKeBWMOEiIncCcQWpJkrRhAIvdJcMTLAFLApo=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631214880;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=ODeFCicDvawAcTinn7/rak3JqgqM+ZCyMPqIlQYJDjY=;
	b=KxtBCGmru/LmPtO9sh7i3B3nU/6+DzinpkhmIQ6u4y82skSnPew5Q9uteofseSoT
	Kj/25swEQlI+4VzzqzqrPnZyYGq2vWlLX6rI1rUXyPgV4gzy7ee/c98tcYqpHBYkbV9
	XyYr9nLM2aBJCYoSUZEgQwFVZlE+O85cKQ0CwzAg=
Subject: Re: [PATCH v4 09/11] silo: remove circular xsm hook call
To: Jan Beulich <jbeulich@suse.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210903190629.11917-1-dpsmith@apertussolutions.com>
 <20210903190629.11917-10-dpsmith@apertussolutions.com>
 <8d7824e1-3c7d-c0c5-f3ad-0644d4c49e07@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <0ad0d8a9-5f9a-dfb0-51b9-c19e4f4dc5aa@apertussolutions.com>
Date: Thu, 9 Sep 2021 15:14:36 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <8d7824e1-3c7d-c0c5-f3ad-0644d4c49e07@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External

On 9/9/21 11:45 AM, Jan Beulich wrote:
> On 03.09.2021 21:06, Daniel P. Smith wrote:
>> SILO implements a few XSM hooks to extended the decision logic beyond
>> what is defined in the dummy/default policy. For each of the hooks, it
>> falls back to the dummy/default policy. The fall back is done a slight
>> round-about way. This commit makes the direct call to the default policy's
>> logic, xsm_default_action().
> 
> Again it's not clear to me what you're finding wrong here. The way it's
> done is not as direct as it could be, but going through the extra layer
> allows the functions to document things at the same time. You lose not
> only that documentation, but also ...

It is only for six calls, thus I figured the slight overhead would be
worth cutting out the indirection. If now one is worried about the extra
indirection, than I can adjust to call the default's handlers.


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 19:32:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 19:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183519.331738 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPmR-0008H7-5a; Thu, 09 Sep 2021 19:32:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183519.331738; Thu, 09 Sep 2021 19:32:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOPmR-0008H0-1T; Thu, 09 Sep 2021 19:32:15 +0000
Received: by outflank-mailman (input) for mailman id 183519;
 Thu, 09 Sep 2021 19:32:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mYjb=N7=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mOPmQ-0008Gu-2l
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 19:32:14 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9144ba9a-28b3-46ff-a8b2-e04c6ce47ad1;
 Thu, 09 Sep 2021 19:32:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9144ba9a-28b3-46ff-a8b2-e04c6ce47ad1
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631215932;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=nJpzC1u+7lEXyVswVkCY5sFmtKIznFVFfphKGPf9a+k=;
  b=YKeEaL2eMjl0CUCia9+Vd1wKi5BRF9/Dwq9VEAK97hyin/lyEuk6hadc
   /kq29JW/2i2Qru7z7X6nlaJHy26UY3XKfvUyjHLtwVpaaFu3gQm2pqacR
   5MrTp55Dg0PSV2eTOWokgGdbSXx0IDzYwFO5ClUmDlUnhnaGmVZsF4tgk
   Q=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: dw5OZMrsD+9zO8ImFGDaUpaAFDEzTI9boI3fr5inGP9zyGfrKUemh9ER7euQFr8iaJZW1Wb2bW
 YtRVDrc8Xw+txYOvkxSQl35ncscM4IOAW6siC8Uyg22CzrIoFZ2YfeUe96uSm85FZi2vtH2/eY
 EYKffUd8vilGKt+clWYJniuX/FyZDSMA38DSpOyUK83Kgv98SNdBTduC0VNGwATfjt4TPSsqXK
 PeU9X49TyI2DltAB8Pt3Blvy1UrRt+c6dsXGaxPGt+vxMaWIXFXmKihleh1bPJQ5iPmir7DpnP
 FNZ9WZ7arsXNrEc59wwBgR79
X-SBRS: 5.1
X-MesageID: 51992268
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-HdrOrdr: A9a23:2o7wsqBbs5RZhtzlHelW55DYdb4zR+YMi2TDt3oddfWaSKylfq
 GV7ZAmPHrP4gr5N0tOpTntAse9qBDnhPtICOsqTNSftWDd0QPFEGgL1+DfKlbbak/DH4BmtJ
 uJc8JFeaDN5VoRt7eH3OFveexQv+Vu88qT9JnjJ28Gd3AMV0n5hT0JcTpyFCdNNW97LKt8Lr
 WwzOxdqQGtfHwGB/7LfEXsD4D41qT2fIuNW29/OyIa
X-IronPort-AV: E=Sophos;i="5.85,281,1624334400"; 
   d="scan'208";a="51992268"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Fum0S/y0/m1tjEpBrZ5sth4We3h6/yBuMOFVJhHtKw7LB4NqDnfXwRrdus+CcClxXVGLUkpb1Kf51t3ajsUBqryO4IUHlfIpxJ7lWMcK+TALB+Jaf8cEWjDy3WnrKNcFTQ0wvSzJ50yZW05L9J7bIPUEEHqVbxT9dnPviIoRNAmbVJbO4W13z/L0EeA3lvXSkSd83Od/fFK3Sj+1MApyrM6auR5tdCLUw3fqBOjzBHzEq1rnW8kLgqh+vS/4RpjUZqJMUbuNWXi/hCHlF9qiQOQ/srkx0CykJZ7qN6WNhFp+rYllu6/HD+s7psQta0fQKcFzGvw5Elm+y4nBLnwiWQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xw5biZfDNnOryD4SC+n2AwZEmSv15Wy4mSqEMZivfxg=;
 b=g3hm5uDy8y7l849Ieo5c5kJqlHxO2uJRr3EibQKlwYF0hzKCzCVIwEZzNB/eVJ1lEpUg1wKovdW+Pv9Ev57kydN5+uymd0NLMhC7LQ4RpX10kgHha+PiSx8LT41oeSxLZhYSJy71FhBFov8DIkZuE399ZoWxNaZ8GQoa0J1e6Q1D+0Jk274RzW6nWO246QWSFb8PfXlJM7MAAkVzkHdljGJeyEO2wCFD0PGLeQB+Rtr1J/xEGb9PAMfpQgKRpNRw1tPKpv6fP334oL/OOHJL70F/yZYU2mEPsZpAo5hzbY333hUe9oeUG7sj/qR19E9MyUlMEWvOhwBYFlOoGWPlqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xw5biZfDNnOryD4SC+n2AwZEmSv15Wy4mSqEMZivfxg=;
 b=gVuft3rhPesPhh8oCskIVX1PcReuDkdBI4L8YJ5HBPMexBI6+e5rA0d+BKn7gKy5/MXRi5XIKQa1eCMMMnRgzDMDHsCRsSiGuwR/Zls7athzRVf0B80Pu7UuT9u3DzdQQzPmY+na15RvDVNbO7xjYIACgeO3mO2kJrVSK7UpBkg=
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>
References: <87278164-2395-1fb4-7569-9151d0151e8a@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] x86: correct asm() constraints when dealing with
 immediate selector values
Message-ID: <0800289d-7744-2c77-b369-707f4947595b@citrix.com>
Date: Thu, 9 Sep 2021 20:31:47 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <87278164-2395-1fb4-7569-9151d0151e8a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0501.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1ab::20) To BN7PR03MB3618.namprd03.prod.outlook.com
 (2603:10b6:406:c3::27)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 349e153f-e01b-467c-4bb5-08d973c8836f
X-MS-TrafficTypeDiagnostic: BN8PR03MB4706:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BN8PR03MB4706DACE87FFC9D4DE394547BAD59@BN8PR03MB4706.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: FFXucYwrA97Tq6rB0CkNsupt0DOllft+sgFoq3BeeaITcMp0gZINMNY6Q4CT31ZEliIc4MF5CAhiD1TBKY21+3cj6/CIZh5peo1WNvpT5c/ANkB5LyHS3Gd0cSamkFECDyKed2GhkYLT8WfuIwKj5rBEHg2kWOKbz6wo8QhLHsPumzUaRaUYm5pqmS/9IfswzcFAILV2zP6G0/SuTJg+SNBLa+NYSy2VDthd+O7dsha6cijeuBTiFsAme9VoGoh886OPTIL2w8J/eKt5igflLqj4taOa2fEabiiXwWZZlCzfeX/37LM2cvC5ler+9JA6gacqW1lE04ksy4Q+KeKZD4YR80xlEqQpeDD9r8AyDzTxcPHQ10x5ZrQvf+el0HKD5R+qPZT4Ces9QMfG9QrlAqTKrravuiy/4ey6BEXgDrrNx3qtc7XZxqnfqZ6oZ8Gp8h64Pp6ohM1LzHAETwVdDqeNsVwCixcbzF+ED+Bpf553c7KYsLd6vHQ5QNAZoH2cFoTGD3JxsKNHHyVo0SUGVmavOmlhA0BmbSAbE8pOi74wn5xJBCeuo3nS/HT5zgRP9f0MetDzhdGDfa8U1kWVFnG38axnQiiDUjC6Bp4u2KisImo5iBuOAzCP7OwFkxmM2wpyr22q8R4ODAeBK1BlX59TPZbdkWers0iOPt0ES1oIm4HntgSFP5BShWQ//XZftYRIFVKKNcmbT5tl3IX5qay1h/msIyqMDFFRXDvKoMVpT3l4HAqueg3zjbzYrixjMdCtWrmgMohk12vbI/jx6xTBJm//q8ZQ5ef1brb58EFRFk8JICMnKc1CujXoLvHeq70vUTwAiwlZLeR5nbaRIQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN7PR03MB3618.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(136003)(376002)(396003)(39860400002)(2906002)(26005)(38100700002)(53546011)(55236004)(8936002)(8676002)(66476007)(478600001)(31686004)(6486002)(66946007)(4326008)(186003)(5660300002)(66556008)(83380400001)(107886003)(54906003)(110136005)(316002)(16576012)(86362001)(31696002)(956004)(2616005)(36756003)(6666004)(966005)(14773001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YnU0MDR4dkRHM3lTZ05UU0J0ZlN4ZldwemRwMnJXVUducmNvQ0k0ZHBnZjRp?=
 =?utf-8?B?eUhhNlZrMUI4cHNVYTFaNjFjcEVWY1U0TlVKZVl1NEc4UHJ3cTgyTHp3dW84?=
 =?utf-8?B?WEZMeDRFb0xUTWhwSkYzS2Q4alA3eEQrdVdpN0FIb1hiU1dQUzBvQ0JoSklU?=
 =?utf-8?B?S2dPS3VMVmtUaFNOYjZKK0dVWjdhUzVIQStTNDF3VkF6Nk5iTEQxK01aMWE3?=
 =?utf-8?B?OEdlWjk2WnpBbk5iZDkvMWVQSmFCcXNVSWt5OXhBbGRYUFlZZGJFSUJ5TEl1?=
 =?utf-8?B?TjdpM1FqQmRrYmszaGMrYlppUjI3Q0hVQlNVbmd1Vy9NS0lVTXRtSHFtR0Ru?=
 =?utf-8?B?RGNQWThrdFRieVdEc0pwUmVpUnVIbU96K1RsZlRJbDBFcDA2ZzVhNk5UcERO?=
 =?utf-8?B?cXBWdk1FQ29BbzkxMUdpcmx6eS9MK2ZSam9OTmdvQXJld3Vqam1iS01PMTV0?=
 =?utf-8?B?SDR5dFphLys4QTR0a0tLd0RLQTVyajhldGc4OWVCRjJhc2RjdnhKR2IyZks3?=
 =?utf-8?B?bEF0VTQwNms5K29yV01pYmtCTVAyZU40NDF2V2VFdnZwME1YdkVqTStJRWZM?=
 =?utf-8?B?RkgzRzV6VFlHM2I1QWhNbkVZUlRleUVrd0N1MkYwVm92NENNWUd1OHhxUG5Q?=
 =?utf-8?B?NkdFRzRFUCt4R2xaZUZaSGJNdHJXMjNENXNWYWVqRFpGbkNrN052Qk1jRlJa?=
 =?utf-8?B?T3hpQUhxRUF0MTk4Myt2R2F6K3E5TC9QUi85UnhxUFY2MFd1Z2ljUjdwM1pU?=
 =?utf-8?B?cE9iQXFhUXVJKzZDUUFmMWZhejk1NUUvcHdCQS91Q1F2azgzMXNkNzJ4eWNp?=
 =?utf-8?B?T3dWVlV0WGxJdjVlbXk0VkNTeUlTRGRDd3YvbzNEREV3Qjc2dyt5OXlJMzdx?=
 =?utf-8?B?dk5GK2pENjBxNXJabFFOZzNna2Q2UVFMRkJ0WUZEQ3AzZmxiMXpqeDM3QS9K?=
 =?utf-8?B?bnB4U0dTUW5mbitNQkJ1VDFxakpPSTdLb21qUzc3WllFM1Y5U0hMZ0tkVFlh?=
 =?utf-8?B?MzRxZ2xKMFBBd0ZVelB5Q2xnT080eHhhaVFYYWN1ejNNM09OekI3TElOdTZi?=
 =?utf-8?B?Y3ZPOTdNazJaVFZ4QUsxVnBvQlJ5VmhMOG4yR2JnQzJRekUwcFEzL0xEOTBx?=
 =?utf-8?B?RFk5cFhZVHZZV3FiOXFHek90eFhBSGdTNEVqUlFmamZNeDE1V3ZERXpqd3hR?=
 =?utf-8?B?L3F2eC8ydUg1ZEFKZnRJUmxMMDdwTkFJaEczTERaVnFMRDZQdk9LOTFNWEg1?=
 =?utf-8?B?NmI1Mll2a2YzNjkwKzRwR2FoM2wzUTdPYTJOU0h4QVVHb1Z2UW9iV05HZEgw?=
 =?utf-8?B?eFRnTGJXZk9BZmhQS2ZMVm9uck5PR0RTQTBYRzAyb1NiMjg1TmdlbmpueGp1?=
 =?utf-8?B?ZllVZTZlVlQxQVFqNHhtMFRHaStWMzNPWG5JK2k1WFFMZklkNmF5OERhYnll?=
 =?utf-8?B?QVVuNEs3RGRwSlhiZjBvUDBuWWwxOGpzSVhIZlFIK29xYi9ZcUlaVzEyL2Vy?=
 =?utf-8?B?ZnZBVldmRjlzcER5MVlMN1JSZnN2a2tuSVQyRXB0Y0luWE9yZmdvNlBWbEtG?=
 =?utf-8?B?aG91TXhuMmdGaFZKbmNsSjdDLzNBbkxNald6eGJhN3BxcEFnQmxDTkdmZFB6?=
 =?utf-8?B?N0txMjdZSnJrQWNiOEsvWnc1eGtpc3NmQU9ua2dkZ1dMM0hpcVRjZmF0VUto?=
 =?utf-8?B?OVZJYTB3ZGloTERrTTlINUVQVUo5aGQ0UTN6aEpyaHh5ckVTa05iR1FXK0E5?=
 =?utf-8?Q?5dYQeQS/6Y8FjJ8bfI59wfVwU8Ggc1rwfJ/5ND1?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 349e153f-e01b-467c-4bb5-08d973c8836f
X-MS-Exchange-CrossTenant-AuthSource: BN7PR03MB3618.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 19:32:08.6787
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AYYZvwXba3lUWzi9aobGKzoEpbFQi+Yh3b4T7A3A7OOBIkzmZbOq2lU03qjk9Fu7dRl6+IUIWxatWmnaXI27+K7nwcSykPJ+HIKSZESft+E=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR03MB4706
X-OriginatorOrg: citrix.com

On 09/09/2021 15:56, Jan Beulich wrote:
> asm() constraints need to fit both the intended insn(s) which the
> respective operands are going to be used with as well as the actual kind
> of value specified. "m" (alone) together with a constant, however, leads
> to gcc saying
>
> error: memory input <N> is not directly addressable
>
> while clang complains
>
> error: invalid lvalue in asm input for constraint 'm'
>
> And rightly so - in order to access a memory operand, an address needs
> to be specified to the insn. In some cases it might be possible for a
> compiler to synthesize a memory operand holding the requested constant,
> but I think any solution there would have sharp edges.

It's precisely what happens in the other uses of constants which you
haven't adjusted below.=C2=A0 Constants are fine if being propagated into a
static inline which has properly typed parameters.

Or are you saying automatic spilling when a width isn't necessarily known?

> If "m" alone doesn't work with constants, it is at best pointless (and
> perhaps misleading or even risky - the compiler might decide to actually
> pick "m" and not try very hard to find a suitable register) to specify
> it alongside "r".
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

I'm slightly surprised you didn't spot and comment about what Clang does
with this.

https://godbolt.org/z/M4nrerrWM

For "rm" (0), clang really does spill the constant into a global and
generate a rip-relative mov to fs, which is especially funny considering
the rejection of "m" as a constraint.

Clang even spills "rm" (local) into a global, while "m" (local) does
come from the stack.


I think there is a reasonable argument to say "m" (const) doesn't have
enough type (width) information for a compiler to do anything sensible
with, and therefore it is fair to be dropped.

But for "rm" (var), where there is enough width information, I don't
think it is reasonable to drop the "m" and restrict the flexibility.

Furthermore, I'm going to argue that we shouldn't work around this
behaviour by removing "m" elsewhere.=C2=A0 This code generation
bug/misfeature affects every use of "rm", even when the referenced
operand is on the stack and can be used without unspilling first.

>
> --- a/xen/arch/x86/cpu/amd.c
> +++ b/xen/arch/x86/cpu/amd.c
> @@ -736,7 +736,7 @@ void __init detect_zen2_null_seg_behavio
>  	uint64_t base;
> =20
>  	wrmsrl(MSR_FS_BASE, 1);
> -	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
> +	asm volatile ( "mov %0, %%fs" :: "r" (0) );
>  	rdmsrl(MSR_FS_BASE, base);
> =20
>  	if (base =3D=3D 0)
> --- a/xen/arch/x86/x86_64/traps.c
> +++ b/xen/arch/x86/x86_64/traps.c
> @@ -248,7 +248,7 @@ void do_double_fault(struct cpu_user_reg
> =20
>      console_force_unlock();
> =20
> -    asm ( "lsll %1, %0" : "=3Dr" (cpu) : "rm" (PER_CPU_SELECTOR) );
> +    asm ( "lsll %1, %0" : "=3Dr" (cpu) : "r" (PER_CPU_SELECTOR) );

Any chance we can clean this up to: lsl %[lim], %[sel] seeing as the
line is being edited?

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 19:53:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 19:53:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183526.331748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOQ7H-0002in-VQ; Thu, 09 Sep 2021 19:53:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183526.331748; Thu, 09 Sep 2021 19:53:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOQ7H-0002ig-SS; Thu, 09 Sep 2021 19:53:47 +0000
Received: by outflank-mailman (input) for mailman id 183526;
 Thu, 09 Sep 2021 19:53:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOQ7G-0002ia-1C
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 19:53:46 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 958e9087-1d69-43f8-8d51-8d40c6756bae;
 Thu, 09 Sep 2021 19:53:44 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AC7EF610A3;
 Thu,  9 Sep 2021 19:53:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 958e9087-1d69-43f8-8d51-8d40c6756bae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631217224;
	bh=iiysE0Z9f0QioDhx3Nak9Gu1u/jJsXfut2cpl96zhto=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=QtGumlrpMDEsD6CmW948+MWJIbKUOVpd3l8CyaKYosA91NwdFjtYvelg1AXAmhLD2
	 lFrfATKiF1die7ZJmOyfLHQe/Rx6GyXo2xoHGhAgyNf5GEK7CtCo3HCgobQ5kGKQ+Y
	 FvaUfc6UMbjv+5MKn9CI5giHGg4M0IJMfTi3YN3Nog5vCHt5RPTJuDQvvVbn39d5Jy
	 ELVYmjxpcWP+51lQHasiiavC4P/i0pn1cwNE1S1ZN3rQgqPuoGvDKRooMobFwB9JNh
	 vm6HVQ2lF9TvEHjAs3yRwiAllTp6t9xrRt3NvsA0DCuJ2deoU2kGEGeKMRrVRwHvOj
	 ucTAQKSx3wVHg==
Date: Thu, 9 Sep 2021 12:53:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Roman Skakun <Roman_Skakun@epam.com>
cc: Julien Grall <julien@xen.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Andrii Anisov <Andrii_Anisov@epam.com>, Roman Skakun <rm.skakun@gmail.com>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: Disable IOMMU in Dom0
In-Reply-To: <AM7PR03MB6593E7F0C691F98E07B8141C85D59@AM7PR03MB6593.eurprd03.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109091232100.10523@sstabellini-ThinkPad-T480s>
References: <AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com> <alpine.DEB.2.21.2108311444310.18862@sstabellini-ThinkPad-T480s> <AM7PR03MB659358C0EBA9D71AEEF7A60885CD9@AM7PR03MB6593.eurprd03.prod.outlook.com>
 <179e4b7f-38d9-991d-3f99-64a74559986d@xen.org> <AM7PR03MB6593E7F0C691F98E07B8141C85D59@AM7PR03MB6593.eurprd03.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1048009336-1631216039=:10523"
Content-ID: <alpine.DEB.2.21.2109091234050.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1048009336-1631216039=:10523
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109091234051.10523@sstabellini-ThinkPad-T480s>

I am fine with adding this functionality only to device tree initially.


It is certainly true that if a DMA-capable device is behind an IOMMU,
then we can skip swiotlb-xen for foreign pages address transactions.
Those addresses will be translated just fine thanks to the IOMMU.
Skipping swiotlb-xen could provide a non-negligible performance
improvement, which is good.

However, you should be aware that your proposal should not be needed for
correctness or to make devices with a 4GB DMA mask work.

Thanks to commit 04085ec1a "xen/arm: fix gnttab_need_iommu_mapping"
foreign pages are added to the Dom0 p2m when mapped to Dom0. swiotlb-xen
translates foreign pages gfn to mfn then uses the mfn to program the
device DMA. The DMA transaction will be accepted by the IOMMU thanks to
the 1:1 GFN<->MFN entry added to Dom0 at the time of mapping.

If the mfn is >4GB and the device requires an address <4GB, then the
dma_capable check at the beginning of xen_swiotlb_map_page fails and the
DMA transaction gets bounced on a swiotlb-xen buffer lower than 4GB.

Am I missing anything?


On Thu, 9 Sep 2021, Roman Skakun wrote:
> Hi Julien,
> Thanks for the clarification!
> 
> I aim towards to prepare implementation for upstream to disable SWIOTLB for IOMMU-protected devices in Dom0.
> To provide this functionality need to add additional binding for each protected device in device-tree.
> After this step, I will also prepare the patch to make ensure that ballooning code prepares all allocations below 4GB.
> 
> We are going to prepare this functionality only for device-tree based system configurations.
> We don't have resources to support ACPI configuration.
> 
> Would you be ok with upstreaming only device-tree configuration?
> 
> Cheers,
> Roman
> 
> ___________________________________________________________________________________________________________________________________________
> From: Julien Grall <julien@xen.org>
> Sent: Wednesday, September 1, 2021 1:22 PM
> To: Roman Skakun <Roman_Skakun@epam.com>; Stefano Stabellini <sstabellini@kernel.org>
> Cc: xen-devel@lists.xenproject.org <xen-devel@lists.xenproject.org>; Bertrand Marquis <bertrand.marquis@arm.com>; Andrii Anisov
> <Andrii_Anisov@epam.com>; Roman Skakun <rm.skakun@gmail.com>; Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
> Subject: Re: Disable IOMMU in Dom0  
> Hi Roman
> 
> On 01/09/2021 10:59, Roman Skakun wrote:
> >> If you have a setup  where Dom0 is not 1:1 mapped (which is not currently
> >> possible with upstream  Xen but it is possible with cache coloring) and
> >> uses the IOMMU to make  device DMA work like regular DomUs, then passing
> >> XENFEAT_not_direct_mapped  to Linux would make it work. Without
> >> XENFEAT_not_direct_mapped,  Linux would try to use swiotlb-xen which has
> >> code that relies on  Linux being 1:1 mapped to work properly.
> >
> > I'm using 1:1 Dom0.
> > According to your patch series, xen-swiotlb fops will be applied for all
> > devices
> > because XENFEAT_direct_mapped is active, as shown here:
> >https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4i
> bLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]
> ><https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/arch/arm64/mm/dma-mapping.c*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4
> ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]>
> >
> > I agreed, that xen-swiotlb should work correctly, but in my case, I
> > retrieved MFN here:
> >https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ib
> LDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]
> ><https://urldefense.com/v3/__https://elixir.bootlin.com/linux/v5.14/source/drivers/xen/swiotlb-xen.c*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4i
> bLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]>
> > which is greater than 32bit and xen-swiotlb tries to use bounce buffer
> > as expected.
> > It can lead to decrease a performance because I have a long buffer ~4MB.
> >
> > I thought, that we can disable swiotlb fops for devices which are
> > controlled by IOMMU.
> 
> Yes you can disable swiotlb for devices which are controlled by the
> IOMMU. But this will not make your problem disappear, it simply hides
> until it bites you in a more subttle way.
> 
>  From what you wrote, you have a 32-bit DMA capable. So you always need
> to have an address below 4GB. For foreign mapping, there is no guarantee
> the Guest Physical Address will actually be below 4GB.
> 
> Today, the ballooning code only ask Linux to steal *a* RAM page for
> mapping the foreign page. This may or may not be below 4GB depending on
> how you assigned the RAM to dom0 (IIRC you had some RAM above 4GB).
> 
> But that's the current behavior. One of your work colleague is looking
> at avoid to steal RAM page to avoid exhausting the memory. So foreign
> mapping may end up to be a lot higher in memory.
> 
> IOW, you will need to be able to bounce the DMA buffer for your device.
> If you want to avoid bouncing, the proper way would be to rework the
> ballonning code so pages are allocated below 4GB.
> 
> Cheers,
> 
> --
> Julien Grall
> 
> 
--8323329-1048009336-1631216039=:10523--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 20:59:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 20:59:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183553.331802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOR8m-0002G0-C2; Thu, 09 Sep 2021 20:59:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183553.331802; Thu, 09 Sep 2021 20:59:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOR8m-0002Ft-92; Thu, 09 Sep 2021 20:59:24 +0000
Received: by outflank-mailman (input) for mailman id 183553;
 Thu, 09 Sep 2021 20:59:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOR8k-0002Fj-L6; Thu, 09 Sep 2021 20:59:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOR8k-0006u4-En; Thu, 09 Sep 2021 20:59:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOR8k-0003Cf-5E; Thu, 09 Sep 2021 20:59:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOR8k-0001aC-4l; Thu, 09 Sep 2021 20:59:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0jYPVyZALNDOdXVQ0NJIVvyuPZKdrkvBTAZgTbcmcd0=; b=vLixTtD7M/PU0h/Vq3BT6mbPk9
	jRNWxIv/Oo/GpqMiDjAt6Uf2nCtWwx6FX7nrwBVYCvhWuJ3xP3R3LJrACdnJ5W/mWoT87/rZe+8/u
	KSGN48oyQ0DySs671SpEEH52zVwKlrY9wRfRUI6T+Ilf6wHyFxYpjPkDdVtvpzzxTTfc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164892-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164892: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=56abcf1a58bdaef18579cf2ce8645c8c72a2b749
X-Osstest-Versions-That:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 09 Sep 2021 20:59:22 +0000

flight 164892 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164892/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164883

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164883
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164883
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164883
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164883
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164883
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164883
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164883
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164883
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164883
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164883
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164883
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  56abcf1a58bdaef18579cf2ce8645c8c72a2b749
baseline version:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1

Last test of basis   164883  2021-09-08 10:10:28 Z    1 days
Testing same since   164892  2021-09-08 21:09:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e70a9a043a..56abcf1a58  56abcf1a58bdaef18579cf2ce8645c8c72a2b749 -> master


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:12:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:12:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183571.331833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHZ-0003cX-0p; Thu, 09 Sep 2021 22:12:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183571.331833; Thu, 09 Sep 2021 22:12:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHY-0003cQ-Tw; Thu, 09 Sep 2021 22:12:32 +0000
Received: by outflank-mailman (input) for mailman id 183571;
 Thu, 09 Sep 2021 22:12:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSHX-0003c4-QP
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:12:31 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f3d50f6-4828-449e-8654-149f939dc103;
 Thu, 09 Sep 2021 22:12:30 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225544701162.02198658548343;
 Thu, 9 Sep 2021 15:12:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f3d50f6-4828-449e-8654-149f939dc103
ARC-Seal: i=1; a=rsa-sha256; t=1631225548; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=Tlnm4EJ05rZzsP9cknZFvs/tffNhjJyL2Ja6MkLClKvXZNffMYaZkALdOXFCu7vRamvzBGq9p3sklT7BXfgbYnoLLtJ+sv5752kLFzSXW4Ad91IgACwN/tZuZvCrPv2CMFBskkaRsb4JYOfe+EAiVjvxvwIlC9fr1kSVUuBkWko=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225548; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:MIME-Version:Message-ID:Subject:To; 
	bh=jfgI+YuhgZ+q3DewpTocAmdfPzKAzfGL5vsDIKUFnGE=; 
	b=g+gl71bTI1pHUlGZo3cpLKKKzIej8hq/+O0x7LSU06s91kDKN9nU0/oEV7vO5fyapmQyzDMvnO/UIS6av8gJJP4I3NrzmN+WoK1lpuRfB6QHjAG+FJuFoXt7h66qZohDNIuGNF6XkEP08jX0f7umgyP5n3dh/Wo97yjbJ1uVQfc=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225548;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=jfgI+YuhgZ+q3DewpTocAmdfPzKAzfGL5vsDIKUFnGE=;
	b=a9+u4zwMP9eMIUn9slvQuoii671WB7whRF4nomXfMXV9tTHPRjAhPECw9wBlLjbZ
	4VcvCbdzBP6uZwrg4kmbJHP9HRCbVPLIhj58uJ99QvrVkDv2Ayb7msjMJb1EYm6IanH
	fCtnRI0mMyvXolQAw5mALtwTTJzy22KORnaVT7To=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: xen-devel@lists.xenproject.org
Cc: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v5 00/10] xsm: refactoring xsm hooks
Date: Thu,  9 Sep 2021 22:12:25 -0400
Message-Id: <20210910021236.15082-1-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Based on feedback from 2021 Xen Developers Summit the xsm-roles RFC patch set
is being split into two separate patch sets. This is the first patch set and is
focused purely on the clean up and refactoring of the XSM hooks.

This patch set refactors the xsm_ops wrapper hooks to use the alternative_call
infrastructure. Then proceeds to move and realign the headers to simplify the
unnecessarily complicated header inclusions, remove the facad of being able to
enable/disable XSM, and simplify the XSM hooks down to a single interface and
implementation. The remainder of the changes are clean up and removing no
longer necessary abstractions.

v2:
 - restructured the patches, breaking them up as needed
 - incorporate Andrew Cooper's alternative call common code
 - change XSM module registration, removing register_xsm
 - incoporate KConfig recommendations
 - reworded commit messages
 - incorporate macro expansion recommendations
 - misc clean-up fallout from recommendations

v3:
 - renamed struct xsm_operations to struct xsm_ops
 - flask and silo ops structs made __initconst
 - fixed misplacement of __init on flask/silo_init
 - lots of coding style alignment
 - further clean up from FLASK_DISABLE removal
 - addressed commit message comments
 - removed missed guard around alternative-call include
- reworked approach to XSM hooks, merging the two interfaces instead of
   dropping one

v4:
- make __alt_call_maybe_initdata resolve to __read_mostly when
  CONFIG_ALTERNATIVE_CALL is not set
- removed the masking of void with xsm_op_t
- moved all the coding style conformity to an early commit
- fixed declaration on real and stub declarations in xsm.h
- corrected to __initconstrel
- made the xsm_ops global __read_mostly
- adjusted blank lines in xsm_ops struct to provide consistent grouping
- moved extern references to built in policy to where they are used
- Added back in the #ifdef CONFIG_XSM into struct evtchn
- split the patch removing the duplicate interface up further

v5
- changed 80 col wrapping style to preferred style
- a few additional code style cleanups caught in v4 review
- made xsm_ops_registered __initdata
- pulled duplicate if out of switch cases in xsm_core_init()
- dropped struct evtchn commit
- dropped silo hooks direct calls to xsm_default_action

Andrew Cooper (1):
  xen: Implement xen/alternative-call.h for use in common code

Daniel P. Smith (9):
  xsm: remove the ability to disable flask
  xsm: remove remnants of xsm_memtype hook
  xsm: drop dubious xsm_op_t type
  xsm: apply coding style
  xsm: refactor xsm_ops handling
  xsm: convert xsm_ops hook calls to alternative call
  xsm: decouple xsm header inclusion selection
  kconfig: update xsm config to reflect reality
  xsm: remove alternate xsm hook interface

 xen/arch/x86/Kconfig               |   1 +
 xen/common/Kconfig                 |  52 +-
 xen/include/public/xsm/flask_op.h  |   2 +-
 xen/include/xen/alternative-call.h |  63 +++
 xen/include/xen/hypercall.h        |   4 +-
 xen/include/xen/sched.h            |   2 +-
 xen/include/xsm/dummy.h            | 774 ---------------------------
 xen/include/xsm/xsm-core.h         | 266 ++++++++++
 xen/include/xsm/xsm.h              | 806 +++++++++++++----------------
 xen/xsm/Makefile                   |   4 +-
 xen/xsm/dummy.c                    |   7 +-
 xen/xsm/dummy.h                    | 659 +++++++++++++++++++++++
 xen/xsm/flask/flask_op.c           |  32 +-
 xen/xsm/flask/hooks.c              |  16 +-
 xen/xsm/silo.c                     |  11 +-
 xen/xsm/xsm_core.c                 |  93 ++--
 xen/xsm/xsm_policy.c               |   7 +-
 17 files changed, 1436 insertions(+), 1363 deletions(-)
 create mode 100644 xen/include/xen/alternative-call.h
 delete mode 100644 xen/include/xsm/dummy.h
 create mode 100644 xen/include/xsm/xsm-core.h
 create mode 100644 xen/xsm/dummy.h

-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:12:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:12:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183572.331844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHf-0003tT-97; Thu, 09 Sep 2021 22:12:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183572.331844; Thu, 09 Sep 2021 22:12:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHf-0003tM-5G; Thu, 09 Sep 2021 22:12:39 +0000
Received: by outflank-mailman (input) for mailman id 183572;
 Thu, 09 Sep 2021 22:12:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSHe-0003t6-QS
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:12:38 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 27bdd14b-110b-436e-a028-df8262e444b9;
 Thu, 09 Sep 2021 22:12:37 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163122554711490.57395943312747;
 Thu, 9 Sep 2021 15:12:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27bdd14b-110b-436e-a028-df8262e444b9
ARC-Seal: i=1; a=rsa-sha256; t=1631225552; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=JftKRVijsDv7+cQVOBsiTPJIEgxiWP2ToMQVQaWt9NvZo5x7jzbG6RIF/TNi8pY4qXvEffOVAucOLkGxmlfLOnfv7ulbetcnWVbQDieYSuqEQD7f3QaoodhvEgOBhY1yQVdrdDsBJSWuXEKuSLWgIQOwoqf0KAqdl7MOzw7Ft7c=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225552; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=XBdaaKkwEvH0I0vbH2y5Ky8zmphGkZHpWf/ybI9eSE4=; 
	b=Oggn84Lu5ldQYhM2v5ielJ1IHq/Ib5a+AWFXm9bhEOgssLlHSK/zw6XocxbuMdcQlnoc9uG93j4kQXSllxeZNnD6HcqW78ZGmxDfLV4PebHVLQ+pMq9nPw0/Gfb8koYd9OOuXCh0AcT0/kfqcbKyVKPA8zqNr7Rws6Vr0Y8F9p0=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225552;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=XBdaaKkwEvH0I0vbH2y5Ky8zmphGkZHpWf/ybI9eSE4=;
	b=RIjmXCTMIcrujVH5Gaudc6m++A+q0PN/Eeq1hvJcVC+mQSVlzz0MLT3US5IgZs08
	nrtd+49t7f7eDn0wn6pvrOMstTFFqI1Rd1JEkTDyNJ8n9fXv9p26amt5nsjCLxRcStG
	JW5uCPFqehQakVvK6AlmD+/ZnaEwyB0kK6I0zcwo=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Connor Davis <connojdavis@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v5 01/10] xen: Implement xen/alternative-call.h for use in common code
Date: Thu,  9 Sep 2021 22:12:26 -0400
Message-Id: <20210910021236.15082-2-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

From: Andrew Cooper <andrew.cooper3@citrix.com>

The alternative call infrastructure is x86-only for now, but the common iommu
code has a variant and more common code wants to use the infrastructure.

Introduce CONFIG_ALTERNATIVE_CALL and a conditional implemetnation so common
code can use the optimisation when available, without requiring all
architectures to implement no-op stubs.

Write some documentation, which was thus far entirely absent, covering the
requirements for an architecture to implement this optimsiation, and how to
use the infrastructure in general code.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bob Eshleman <bobbyeshleman@gmail.com>
CC: Alistair Francis <alistair.francis@wdc.com>
CC: Connor Davis <connojdavis@gmail.com>
CC: Daniel P. Smith <dpsmith@apertussolutions.com>

v3:
 * Drop __alt_call_maybe_initconst

This is a pre-requisite to "xsm: refactor xsm_ops handling" to avoid breaking
the ARM build.

Build test for the XSM code:

  diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
  index 5eab21e1b168..592074e8f41c 100644
  --- a/xen/xsm/xsm_core.c
  +++ b/xen/xsm/xsm_core.c
  @@ -195,6 +195,16 @@ bool __init has_xsm_magic(paddr_t start)
   }
    #endif

  +#include <xen/alternative-call.h>
  +struct foo {
  +    int (*bar)(void *);
  +} foo __alt_call_maybe_initdata;
  +
  +int test_alternative_call(void)
  +{
  +    return alternative_call(foo.bar, NULL);
  +}
  +
   int __init register_xsm(struct xsm_operations *ops)
    {
         if ( verify(ops) )
---
 xen/arch/x86/Kconfig               |  1 +
 xen/common/Kconfig                 |  3 ++
 xen/include/xen/alternative-call.h | 63 ++++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+)
 create mode 100644 xen/include/xen/alternative-call.h

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 9b164db641..1f83518ee0 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,6 +6,7 @@ config X86
 	def_bool y
 	select ACPI
 	select ACPI_LEGACY_TABLES_LOOKUP
+	select ALTERNATIVE_CALL
 	select ARCH_SUPPORTS_INT128
 	select CORE_PARKING
 	select HAS_ALTERNATIVE
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11a..ac5491b1cc 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -22,6 +22,9 @@ config GRANT_TABLE
 
 	  If unsure, say Y.
 
+config ALTERNATIVE_CALL
+	bool
+
 config HAS_ALTERNATIVE
 	bool
 
diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternative-call.h
new file mode 100644
index 0000000000..d10af90b1b
--- /dev/null
+++ b/xen/include/xen/alternative-call.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef XEN_ALTERNATIVE_CALL
+#define XEN_ALTERNATIVE_CALL
+
+/*
+ * Some subsystems in Xen may have multiple implementions, which can be
+ * resolved to a single implementation at boot time.  By default, this will
+ * result in the use of function pointers.
+ *
+ * Some architectures may have mechanisms for dynamically modifying .text.
+ * Using this mechnaism, function pointers can be converted to direct calls
+ * which are typically more efficient at runtime.
+ *
+ * For architectures to support:
+ *
+ * - Implement alternative_{,v}call() in asm/alternative.h.  Code generation
+ *   requirements are to emit a function pointer call at build time, and stash
+ *   enough metadata to simplify the call at boot once the implementation has
+ *   been resolved.
+ * - Select ALTERNATIVE_CALL in Kconfig.
+ *
+ * To use:
+ *
+ * Consider the following simplified example.
+ *
+ *  1) struct foo_ops __alt_call_maybe_initdata ops;
+ *
+ *  2) const struct foo_ops __initconst foo_a_ops = { ... };
+ *     const struct foo_ops __initconst foo_b_ops = { ... };
+ *
+ *     void foo_init(void)
+ *     {
+ *         ...
+ *         if ( use_impl_a )
+ *             ops = *foo_a_ops;
+ *         else if ( use_impl_b )
+ *             ops = *foo_b_ops;
+ *         ...
+ *     }
+ *
+ *  3) alternative_call(ops.bar, ...);
+ *
+ * There needs to a single ops object (1) which will eventually contain the
+ * function pointers.  This should be populated in foo's init() function (2)
+ * by one of the available implementations.  To call functions, use
+ * alternative_{,v}call() referencing the main ops object (3).
+ */
+
+#ifdef CONFIG_ALTERNATIVE_CALL
+
+#include <asm/alternative.h>
+
+#define __alt_call_maybe_initdata __initdata
+
+#else
+
+#define alternative_call(func, args...)  (func)(args)
+#define alternative_vcall(func, args...) (func)(args)
+
+#define __alt_call_maybe_initdata __read_mostly
+
+#endif /* !CONFIG_ALTERNATIVE_CALL */
+#endif /* XEN_ALTERNATIVE_CALL */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:12:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:12:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183575.331856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHt-0004QZ-JH; Thu, 09 Sep 2021 22:12:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183575.331856; Thu, 09 Sep 2021 22:12:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSHt-0004QK-Fo; Thu, 09 Sep 2021 22:12:53 +0000
Received: by outflank-mailman (input) for mailman id 183575;
 Thu, 09 Sep 2021 22:12:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSHs-0004NZ-PX
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:12:52 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e301738c-c3e4-4e65-bcfa-f8bf142ad535;
 Thu, 09 Sep 2021 22:12:51 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163122554989340.61571982161797;
 Thu, 9 Sep 2021 15:12:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e301738c-c3e4-4e65-bcfa-f8bf142ad535
ARC-Seal: i=1; a=rsa-sha256; t=1631225554; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=jXMW0Op2uwEaM98zj6uIqmHjFMfELqIGwcX2qz/CdVDaBdI+c75dBfYrkmslRnQNwIY04J1SLHz4URgX0cKLfkwyoAUzobKZ5KfRLXJGy4vFhMcB31HVk3KQyPtPxSNLmcXv9DuGbNDUf2WEl7++rqzVRbDslvUNlHAKYJ7mAX8=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225554; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=R9fWktmmlY/XpFnqpgzdSmnNqMw4SOU1eoy5oHGQ8QI=; 
	b=TdFHjMnwTn9eHjQV2a61+opzOIk/pD9jJaj49DH+FckBUMd3v8EnfUo711Y0nDJb3m+A7CHatrt9RoTseWURDretmaq3+FTHNG4iA3Ucyo4MN+lVIivhdKQwnw63W7gZJ7CQql0Kvrgp7N1vM/B/EFOwZ73bzg2CJsdu2IJ/gYc=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225554;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=R9fWktmmlY/XpFnqpgzdSmnNqMw4SOU1eoy5oHGQ8QI=;
	b=M9qw8vHTU7ksHQlCTkve+CV9LcOlQwiNn5F4K0qGwiPuZisMSdZs72gQ8J4vUj3G
	0cW35wDsdHjkDzzNHfKtpEudxFKZnhv2LYyOCr8jjbt3Ufkl9dkJTdTG8NAEugvo+0u
	iLWkACvMSZcyko47nCfUCvUsDpoj/fJ2lJNRDiX0=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 02/10] xsm: remove the ability to disable flask
Date: Thu,  9 Sep 2021 22:12:27 -0400
Message-Id: <20210910021236.15082-3-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On Linux when SELinux is put into permissive mode the descretionary access
controls are still in place. Whereas for Xen when the enforcing state of flask
is set to permissive, all operations for all domains would succeed, i.e. it
does not fall back to the default access controls. To provide a means to mimic
a similar but not equivalent behavior, a flask op is present to allow a
one-time switch back to the default access controls, aka the "dummy policy".

While this may be desirable for an OS, Xen is a hypervisor and should not allow
the switching of which security policy framework is being enforced after boot.
This patch removes the flask op to enforce the desired XSM usage model
requiring a reboot of Xen to change the XSM policy module in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/public/xsm/flask_op.h |  2 +-
 xen/xsm/flask/flask_op.c          | 30 ------------------------------
 2 files changed, 1 insertion(+), 31 deletions(-)

diff --git a/xen/include/public/xsm/flask_op.h b/xen/include/public/xsm/flask_op.h
index 16af7bc22f..b41dd6dac8 100644
--- a/xen/include/public/xsm/flask_op.h
+++ b/xen/include/public/xsm/flask_op.h
@@ -188,7 +188,7 @@ struct xen_flask_op {
 #define FLASK_SETBOOL           12
 #define FLASK_COMMITBOOLS       13
 #define FLASK_MLS               14
-#define FLASK_DISABLE           15
+#define FLASK_DISABLE           15 /* No longer implemented */
 #define FLASK_GETAVC_THRESHOLD  16
 #define FLASK_SETAVC_THRESHOLD  17
 #define FLASK_AVC_HASHSTATS     18
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 01e52138a1..f41c025391 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -223,32 +223,6 @@ static int flask_security_sid(struct xen_flask_sid_context *arg)
 
 #ifndef COMPAT
 
-static int flask_disable(void)
-{
-    static int flask_disabled = 0;
-
-    if ( ss_initialized )
-    {
-        /* Not permitted after initial policy load. */
-        return -EINVAL;
-    }
-
-    if ( flask_disabled )
-    {
-        /* Only do this once. */
-        return -EINVAL;
-    }
-
-    printk("Flask:  Disabled at runtime.\n");
-
-    flask_disabled = 1;
-
-    /* Reset xsm_ops to the original module. */
-    xsm_ops = &dummy_xsm_ops;
-
-    return 0;
-}
-
 static int flask_security_setavc_threshold(struct xen_flask_setavc_threshold *arg)
 {
     int rv = 0;
@@ -698,10 +672,6 @@ ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
         rv = flask_mls_enabled;
         break;    
 
-    case FLASK_DISABLE:
-        rv = flask_disable();
-        break;
-
     case FLASK_GETAVC_THRESHOLD:
         rv = avc_cache_threshold;
         break;
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183589.331867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIE-0005DX-Rz; Thu, 09 Sep 2021 22:13:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183589.331867; Thu, 09 Sep 2021 22:13:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIE-0005DQ-On; Thu, 09 Sep 2021 22:13:14 +0000
Received: by outflank-mailman (input) for mailman id 183589;
 Thu, 09 Sep 2021 22:13:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSID-0004ti-HH
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:13 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1ed835e6-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:12 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 16312255518171022.0019350432905;
 Thu, 9 Sep 2021 15:12:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ed835e6-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225555; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=gMQYlJZzopPp8jK7IKyN8Yf0Kemzm4l+W4gkn1I4gndKwOv+TuSpZ37pzJ+W/KHwqB9euhqST+Y38zvCjFmC1msBWj9x/XAwql2uvgDADsUmXiJFnyXnpK7jVtKsNFKiCTi/Xgc+TM9MBqdbKcMv8lCNU8/RbKjd3cbzZWv/gII=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225555; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=gyz+bf9QEJJHVoMwH4co/BQLSlmGwar3N5lL6fpMtbE=; 
	b=FyuQltnzuom3bSYXCvcEHnY19ve3kdfgLmx65Ngi5EfbDNbtd91fklafrekLyXtFYkACw4Qyr14SXtsEjV9NBvY7Q0/0TdqujPRVYIuci58hsx0uEaPjx3RdK1IU3QUzLHa6biZWzgPUuvZ1NwjeDsZGXLMHQYuQbkIeeQts5Sg=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225555;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=gyz+bf9QEJJHVoMwH4co/BQLSlmGwar3N5lL6fpMtbE=;
	b=HAsMQ1/vYzkGoGEJKY1MXVdMsTpghTWjjSGcLJnnvC+nBJNQ+Ar0Eb+GwiP+GCZk
	lP6ndHxuNA6vHAr1Gs31D9mXntRM62a22sQGq8VFkhgzJSceIi62u92LNqRF9F3dhWL
	Ayi+m+Q7wq04nziwGkkHT3Ac2GXMID7ZUxcX+uVw=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 03/10] xsm: remove remnants of xsm_memtype hook
Date: Thu,  9 Sep 2021 22:12:28 -0400
Message-Id: <20210910021236.15082-4-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
left behind. This commit cleans up those remnants.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/xsm.h | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index ad3cddbf7d..335ba1b022 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -161,7 +161,6 @@ struct xsm_operations {
     int (*shadow_control) (struct domain *d, uint32_t op);
     int (*mem_sharing_op) (struct domain *d, struct domain *cd, int op);
     int (*apic) (struct domain *d, int cmd);
-    int (*memtype) (uint32_t access);
     int (*machine_memory_map) (void);
     int (*domain_memory_map) (struct domain *d);
 #define XSM_MMU_UPDATE_READ      1
@@ -632,11 +631,6 @@ static inline int xsm_apic (xsm_default_t def, struct domain *d, int cmd)
     return xsm_ops->apic(d, cmd);
 }
 
-static inline int xsm_memtype (xsm_default_t def, uint32_t access)
-{
-    return xsm_ops->memtype(access);
-}
-
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
     return xsm_ops->machine_memory_map();
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183591.331878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIL-0005Y8-5g; Thu, 09 Sep 2021 22:13:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183591.331878; Thu, 09 Sep 2021 22:13:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIL-0005Xz-1F; Thu, 09 Sep 2021 22:13:21 +0000
Received: by outflank-mailman (input) for mailman id 183591;
 Thu, 09 Sep 2021 22:13:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSIJ-0005Wg-DQ
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:19 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 22891d54-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:18 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225553085843.3848670410742;
 Thu, 9 Sep 2021 15:12:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22891d54-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225556; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=a3Pf1tE7Nspz842GiwsO72BhZEnKPGUiMnl1d+w+3x8ELtCtkiYhNuPgXEd8SDM32mqt8Z8iVyrOkIJOZiWhYQ8iKLy700T8XwVCvYn72ES9ZY4dyRUERs9N1PFuljoA3LXk9jZBWzhTTezoYSnr1mhPo29o1Eaj8oPUTNEKgSk=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225556; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=Lq07B1AF5Rs92+IfuaXrGvK7UyuYlp7NAcCIhzsP44w=; 
	b=fIxBp4dcZeG8fpz7JwNAOZVBXEhYa/akKKD4d0CX6EykZxeyH8HyvrY01yNbjOyw2jZCRcI379Exr4IiSUT1g4mWjpOcXxJg4LdRiaDQ7Tihd14pM839rWrPExelhc83qGMjoOA4aBTYiLI1CB/aPFz3aHctzyGncC8rc4AyqvM=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225556;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=Lq07B1AF5Rs92+IfuaXrGvK7UyuYlp7NAcCIhzsP44w=;
	b=T0GaFxoHSV65L6RNu15yDQE69Y0HyGyLK42UERy1U6XbgkFRQ6bR1kvMy2MNmJ6d
	SErSbEnwXHk9iEzVS0008QBY8zdNCbB+6Ld7rQyZGHh/A42J+DgjujBHFrYtl1syzId
	+3gwOhmAxH7AjNF9jYrf0Qo8jwJeMSyaILL+1Rlc=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 04/10] xsm: drop dubious xsm_op_t type
Date: Thu,  9 Sep 2021 22:12:29 -0400
Message-Id: <20210910021236.15082-5-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

The type xsm_op_t masks the use of void pointers. This commit drops the
xsm_op_t type and replaces it and all its uses with an explicit void.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xen/hypercall.h |  4 ++--
 xen/include/xsm/dummy.h     |  4 ++--
 xen/include/xsm/xsm.h       | 11 ++++-------
 xen/xsm/flask/flask_op.c    |  2 +-
 xen/xsm/flask/hooks.c       |  4 ++--
 xen/xsm/xsm_core.c          |  4 ++--
 6 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 34b7f1fed6..3771487a30 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -127,7 +127,7 @@ do_kexec_op(
 
 extern long
 do_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_xsm_op);
+    XEN_GUEST_HANDLE_PARAM(void) u_xsm_op);
 
 #ifdef CONFIG_ARGO
 extern long do_argo_op(
@@ -198,7 +198,7 @@ compat_set_timer_op(
     s32 hi);
 
 extern int compat_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    XEN_GUEST_HANDLE_PARAM(void) op);
 
 extern int compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg);
 
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 363c6d7798..214b5408b1 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -442,13 +442,13 @@ static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
 
 #ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 335ba1b022..16b90be2c5 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,9 +18,6 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
-typedef void xsm_op_t;
-DEFINE_XEN_GUEST_HANDLE(xsm_op_t);
-
 /* policy magic number (defined by XSM_MAGIC) */
 typedef u32 xsm_magic_t;
 
@@ -129,9 +126,9 @@ struct xsm_operations {
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
     int (*hvm_param) (struct domain *d, unsigned long op);
@@ -542,13 +539,13 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index f41c025391..221ff00fd3 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -607,7 +607,7 @@ static int flask_relabel_domain(struct xen_flask_relabel *arg)
 
 #endif /* !COMPAT */
 
-ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
+ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op)
 {
     xen_flask_op_t op;
     int rv;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index f1a1217c98..1465db125a 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1742,8 +1742,8 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 
 #endif
 
-long do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
-int compat_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
+long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
 static struct xsm_operations flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 5eab21e1b1..ac553f9c0d 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -213,13 +213,13 @@ int __init register_xsm(struct xsm_operations *ops)
 
 #endif
 
-long do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-int compat_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+int compat_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_compat_op(op);
 }
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183600.331888 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIX-0006BF-L3; Thu, 09 Sep 2021 22:13:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183600.331888; Thu, 09 Sep 2021 22:13:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIX-0006B8-I2; Thu, 09 Sep 2021 22:13:33 +0000
Received: by outflank-mailman (input) for mailman id 183600;
 Thu, 09 Sep 2021 22:13:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSIW-00067V-O1
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:32 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 28d73df8-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:29 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225555014620.6087293213108;
 Thu, 9 Sep 2021 15:12:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28d73df8-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225558; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=UMUdlEL/TsGgJxdCS6nvFFZhgE3wAuQfJiKJ5FPjDpFZpkAPS1Eq9mB8Y5eCNo8R55E64Te/8lZTDu7+oyQ4pr+tcstQo/Ep8qkN8JVhqMsWJm3szGwUDMzyZA0yliCTLZv7T33S/TaXTwI60HgJkTcDCayo5vEbW7/ib5epk20=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225558; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=DHtmQ/KklNUGBW5ChJ6zA6VgOYa8ooPC62GymxumpGA=; 
	b=EgmSYbgtdfs60Vp/XqXSy6Pm1vH/jLb694Wq6salCtJwLaS/l1fGNsLlfzmPBw0fXLhoqFLTuf/h8KqyQkGnHrB0Nay68jWEp9ifWuxGwI/yUG069VJJ4pcK2ZTnvcmenRJIM5LaTsoSywS7+rbsGypx4/Xads+y2VfLQB3n1Rs=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225558;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=DHtmQ/KklNUGBW5ChJ6zA6VgOYa8ooPC62GymxumpGA=;
	b=YLnDk0r8yS8b2FSQktupv+HqNy4j1ZrbboV1s51sNhAuebfW7lEnvOzB287TaFvN
	1OP3VpkXrnu53OjnDzOdavqbbFEzu2z8UTIUfjrstEfT7Lg5wb3VAE0j2lEWn+6xFuJ
	lfcl5/sOBiLIJhIGopEctxdDECZpnSh4Pq1Lydec=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 05/10] xsm: apply coding style
Date: Thu,  9 Sep 2021 22:12:30 -0400
Message-Id: <20210910021236.15082-6-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Instead of intermixing coding style changes with code changes as they
are come upon in this patch set, moving all coding style changes
into a single commit. The focus of coding style changes here are,

 - move trailing comments to line above
 - ensuring line length does not exceed 80 chars
 - ensuring proper indentation for 80 char wrapping
 - covert u32 type statements to  uint32_t
 - remove space between closing and opening parens
 - drop extern on function declarations

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h | 196 ++++++++++------
 xen/include/xsm/xsm.h   | 499 ++++++++++++++++++++++------------------
 xen/xsm/xsm_core.c      |   8 +-
 xen/xsm/xsm_policy.c    |   7 +-
 4 files changed, 401 insertions(+), 309 deletions(-)

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 214b5408b1..bfcfee1d9e 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -19,7 +19,8 @@
 #include <xsm/xsm.h>
 #include <public/hvm/params.h>
 
-/* Cannot use BUILD_BUG_ON here because the expressions we check are not
+/*
+ * Cannot use BUILD_BUG_ON here because the expressions we check are not
  * considered constant at compile time. Instead, rely on constant propagation to
  * inline out the calls to this invalid function, which will cause linker errors
  * if references remain at link time.
@@ -44,7 +45,8 @@ void __xsm_action_mismatch_detected(void);
 
 #ifdef CONFIG_XSM
 
-/* In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
+/*
+ * In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
  * contains static (not inline) functions compiled to the dummy XSM module.
  * There is no xsm_default_t argument available, so the value from the assertion
  * is used to initialize the variable.
@@ -57,7 +59,8 @@ void __xsm_action_mismatch_detected(void);
 
 #else /* CONFIG_XSM */
 
-/* In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
+/*
+ * In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
  * contains inline functions for each XSM hook. These functions also perform
  * compile-time checks on the xsm_default_t argument to ensure that the behavior
  * of the dummy XSM module is the same as the behavior with XSM disabled.
@@ -70,7 +73,7 @@ void __xsm_action_mismatch_detected(void);
 #endif /* CONFIG_XSM */
 
 static always_inline int xsm_default_action(
-    xsm_default_t action, struct domain *src, struct domain *target)
+	xsm_default_t action, struct domain *src, struct domain *target)
 {
     switch ( action ) {
     case XSM_HOOK:
@@ -98,13 +101,14 @@ static always_inline int xsm_default_action(
     }
 }
 
-static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
-                                    struct xen_domctl_getdomaininfo *info)
+static XSM_INLINE void xsm_security_domaininfo(
+	struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
     return;
 }
 
-static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d, u32 ssidref)
+static XSM_INLINE int xsm_domain_create(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t ssidref)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -116,7 +120,8 @@ static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_domctl_scheduler_op(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int xsm_domctl_scheduler_op(
+	XSM_DEFAULT_ARG struct domain *d, int cmd)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -128,7 +133,8 @@ static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_set_target(XSM_DEFAULT_ARG struct domain *d, struct domain *e)
+static XSM_INLINE int xsm_set_target(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *e)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, NULL);
@@ -173,38 +179,43 @@ static XSM_INLINE void xsm_free_security_domain(struct domain *d)
     return;
 }
 
-static XSM_INLINE int xsm_grant_mapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static XSM_INLINE int xsm_grant_mapref(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2, uint32_t flags)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_unmapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_unmapref(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_setup(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_setup(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_transfer(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_transfer(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_copy(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_copy(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_query_size(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_query_size(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -216,14 +227,15 @@ static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_memory_adjust_reservation(XSM_DEFAULT_ARG struct domain *d1,
-                                                            struct domain *d2)
+static XSM_INLINE int xsm_memory_adjust_reservation(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_memory_stat_reservation(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -253,14 +265,16 @@ static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_schedop_shutdown(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_schedop_shutdown(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_pin_page(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-                                          struct page_info *page)
+static XSM_INLINE int xsm_memory_pin_page(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
+    struct page_info *page)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
@@ -272,15 +286,16 @@ static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_unbound(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn,
-                                         domid_t id2)
+static XSM_INLINE int xsm_evtchn_unbound(
+	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn, domid_t id2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_interdomain(XSM_DEFAULT_ARG struct domain *d1, struct evtchn
-                                *chan1, struct domain *d2, struct evtchn *chan2)
+static XSM_INLINE int xsm_evtchn_interdomain(
+	XSM_DEFAULT_ARG struct domain *d1, struct evtchn *chan1, struct domain *d2,
+    struct evtchn *chan2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
@@ -291,37 +306,41 @@ static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
     return;
 }
 
-static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_send(
+	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d, NULL);
 }
 
-static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_status(
+	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_evtchn_reset(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
 static XSM_INLINE int xsm_alloc_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+	struct evtchn chn[], unsigned int nr)
 {
     return 0;
 }
 
 static XSM_INLINE void xsm_free_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+	struct evtchn chn[], unsigned int nr)
 {
     return;
 }
 
-static XSM_INLINE char *xsm_show_security_evtchn(struct domain *d, const struct evtchn *chn)
+static XSM_INLINE char *xsm_show_security_evtchn(
+	struct domain *d, const struct evtchn *chn)
 {
     return NULL;
 }
@@ -357,13 +376,15 @@ static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_assign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_assign_device(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_deassign_device(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -372,15 +393,15 @@ static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d, uint
 #endif /* HAS_PASSTHROUGH && HAS_PCI */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static XSM_INLINE int xsm_assign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                          const char *dtpath)
+static XSM_INLINE int xsm_assign_dtdevice(
+	XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_deassign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                            const char *dtpath)
+static XSM_INLINE int xsm_deassign_dtdevice(
+	XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -400,19 +421,22 @@ static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_plug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_plug_pci(
+	XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_unplug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_unplug_pci(
+	XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_setup_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_setup_pci(
+	XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
@@ -465,8 +489,8 @@ static XSM_INLINE int xsm_map_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_map_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                         int irq, const void *data)
+static XSM_INLINE int xsm_map_domain_irq(
+	XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -478,76 +502,86 @@ static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_bind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_bind_pt_irq(
+	XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_unbind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_unbind_pt_irq(
+	XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_unmap_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                           int irq, const void *data)
+static XSM_INLINE int xsm_unmap_domain_irq(
+	XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_irq_permission(XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
+static XSM_INLINE int xsm_irq_permission(
+	XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_permission(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_permission(
+	XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_mapping(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_mapping(
+	XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf,
-                                        uint16_t start, uint16_t end,
-                                        uint8_t access)
+static XSM_INLINE int xsm_pci_config_permission(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, uint16_t start,
+    uint16_t end, uint8_t access)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_add_to_physmap(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_remove_from_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_remove_from_physmap(
+	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_map_gmfn_foreign(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_map_gmfn_foreign(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_hvm_param(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_param(
+	XSM_DEFAULT_ARG struct domain *d, unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_control(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_control(
+	XSM_DEFAULT_ARG struct domain *d, unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -559,7 +593,8 @@ static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
+static XSM_INLINE int xsm_hvm_altp2mhvm_op(
+	XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
 
@@ -578,7 +613,8 @@ static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uin
     }
 }
 
-static XSM_INLINE int xsm_vm_event_control(XSM_DEFAULT_ARG struct domain *d, int mode, int op)
+static XSM_INLINE int xsm_vm_event_control(
+	XSM_DEFAULT_ARG struct domain *d, int mode, int op)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -621,13 +657,15 @@ static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_shadow_control(XSM_DEFAULT_ARG struct domain *d, uint32_t op)
+static XSM_INLINE int xsm_shadow_control(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mem_sharing_op(XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
+static XSM_INLINE int xsm_mem_sharing_op(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, cd);
@@ -651,8 +689,9 @@ static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct domain *t,
-                                     struct domain *f, uint32_t flags)
+static XSM_INLINE int xsm_mmu_update(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *t, struct domain *f,
+    uint32_t flags)
 {
     int rc = 0;
     XSM_ASSERT_ACTION(XSM_TARGET);
@@ -663,38 +702,43 @@ static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct do
     return rc;
 }
 
-static XSM_INLINE int xsm_mmuext_op(XSM_DEFAULT_ARG struct domain *d, struct domain *f)
+static XSM_INLINE int xsm_mmuext_op(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *f)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_update_va_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *f, 
-                                                            l1_pgentry_t pte)
+static XSM_INLINE int xsm_update_va_mapping(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_priv_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_priv_mapping(
+	XSM_DEFAULT_ARG struct domain *d, struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_ioport_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_permission(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_ioport_mapping(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_mapping(
+	XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct domain *d, unsigned int op)
+static XSM_INLINE int xsm_pmu_op(
+	XSM_DEFAULT_ARG struct domain *d, unsigned int op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
@@ -723,8 +767,8 @@ static XSM_INLINE int xsm_argo_enable(const struct domain *d)
     return 0;
 }
 
-static XSM_INLINE int xsm_argo_register_single_source(const struct domain *d,
-                                                      const struct domain *t)
+static XSM_INLINE int xsm_argo_register_single_source(
+	const struct domain *d, const struct domain *t)
 {
     return 0;
 }
@@ -734,8 +778,8 @@ static XSM_INLINE int xsm_argo_register_any_source(const struct domain *d)
     return 0;
 }
 
-static XSM_INLINE int xsm_argo_send(const struct domain *d,
-                                    const struct domain *t)
+static XSM_INLINE int xsm_argo_send(
+	const struct domain *d, const struct domain *t)
 {
     return 0;
 }
@@ -743,7 +787,7 @@ static XSM_INLINE int xsm_argo_send(const struct domain *d,
 #endif /* CONFIG_ARGO */
 
 #include <public/version.h>
-static XSM_INLINE int xsm_xen_version (XSM_DEFAULT_ARG uint32_t op)
+static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 16b90be2c5..42b0c188a9 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -19,7 +19,7 @@
 #include <xen/multiboot.h>
 
 /* policy magic number (defined by XSM_MAGIC) */
-typedef u32 xsm_magic_t;
+typedef uint32_t xsm_magic_t;
 
 #ifdef CONFIG_XSM_FLASK
 #define XSM_MAGIC 0xf97cff8c
@@ -27,7 +27,8 @@ typedef u32 xsm_magic_t;
 #define XSM_MAGIC 0x0
 #endif
 
-/* These annotations are used by callers and in dummy.h to document the
+/*
+ * These annotations are used by callers and in dummy.h to document the
  * default actions of XSM hooks. They should be compiled out otherwise.
  */
 enum xsm_default {
@@ -41,147 +42,154 @@ enum xsm_default {
 typedef enum xsm_default xsm_default_t;
 
 struct xsm_operations {
-    void (*security_domaininfo) (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info);
-    int (*domain_create) (struct domain *d, u32 ssidref);
-    int (*getdomaininfo) (struct domain *d);
-    int (*domctl_scheduler_op) (struct domain *d, int op);
-    int (*sysctl_scheduler_op) (int op);
-    int (*set_target) (struct domain *d, struct domain *e);
-    int (*domctl) (struct domain *d, int cmd);
-    int (*sysctl) (int cmd);
-    int (*readconsole) (uint32_t clear);
-
-    int (*evtchn_unbound) (struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain) (struct domain *d1, struct evtchn *chn1,
-                                        struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post) (struct evtchn *chn);
-    int (*evtchn_send) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_status) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset) (struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref) (struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref) (struct domain *d1, struct domain *d2);
-    int (*grant_setup) (struct domain *d1, struct domain *d2);
-    int (*grant_transfer) (struct domain *d1, struct domain *d2);
-    int (*grant_copy) (struct domain *d1, struct domain *d2);
-    int (*grant_query_size) (struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain) (struct domain *d);
-    void (*free_security_domain) (struct domain *d);
-    int (*alloc_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn) (struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain) (struct domain *d);
-
-    int (*get_pod_target) (struct domain *d);
-    int (*set_pod_target) (struct domain *d);
-    int (*memory_exchange) (struct domain *d);
-    int (*memory_adjust_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_pin_page) (struct domain *d1, struct domain *d2, struct page_info *page);
-    int (*add_to_physmap) (struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap) (struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign) (struct domain *d, struct domain *t);
-    int (*claim_pages) (struct domain *d);
-
-    int (*console_io) (struct domain *d, int cmd);
-
-    int (*profile) (struct domain *d, int op);
-
-    int (*kexec) (void);
-    int (*schedop_shutdown) (struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid) (int irq);
-    int (*map_domain_pirq) (struct domain *d);
-    int (*map_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq) (struct domain *d);
-    int (*unmap_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission) (struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*iomem_mapping) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*pci_config_permission) (struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access);
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group) (uint32_t machine_bdf);
-    int (*assign_device) (struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device) (struct domain *d, uint32_t machine_bdf);
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
 #endif
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice) (struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice) (struct domain *d, const char *dtpath);
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
 #endif
 
-    int (*resource_plug_core) (void);
-    int (*resource_unplug_core) (void);
-    int (*resource_plug_pci) (uint32_t machine_bdf);
-    int (*resource_unplug_pci) (uint32_t machine_bdf);
-    int (*resource_setup_pci) (uint32_t machine_bdf);
-    int (*resource_setup_gsi) (int gsi);
-    int (*resource_setup_misc) (void);
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
 
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
-    int (*hvm_param) (struct domain *d, unsigned long op);
-    int (*hvm_control) (struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm) (struct domain *d);
-    int (*hvm_altp2mhvm_op) (struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo) (struct domain *d);
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
 
-    int (*vm_event_control) (struct domain *d, int mode, int op);
+    int (*vm_event_control)(struct domain *d, int mode, int op);
 
 #ifdef CONFIG_MEM_ACCESS
-    int (*mem_access) (struct domain *d);
+    int (*mem_access)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-    int (*mem_paging) (struct domain *d);
+    int (*mem_paging)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing) (struct domain *d);
+    int (*mem_sharing)(struct domain *d);
 #endif
 
-    int (*platform_op) (uint32_t cmd);
+    int (*platform_op)(uint32_t cmd);
 
 #ifdef CONFIG_X86
-    int (*do_mca) (void);
-    int (*shadow_control) (struct domain *d, uint32_t op);
-    int (*mem_sharing_op) (struct domain *d, struct domain *cd, int op);
-    int (*apic) (struct domain *d, int cmd);
-    int (*machine_memory_map) (void);
-    int (*domain_memory_map) (struct domain *d);
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
 #define XSM_MMU_UPDATE_READ      1
 #define XSM_MMU_UPDATE_WRITE     2
 #define XSM_MMU_NORMAL_UPDATE    4
 #define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update) (struct domain *d, struct domain *t,
-                       struct domain *f, uint32_t flags);
-    int (*mmuext_op) (struct domain *d, struct domain *f);
-    int (*update_va_mapping) (struct domain *d, struct domain *f, l1_pgentry_t pte);
-    int (*priv_mapping) (struct domain *d, struct domain *t);
-    int (*ioport_permission) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*pmu_op) (struct domain *d, unsigned int op);
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
 #endif
-    int (*dm_op) (struct domain *d);
-    int (*xen_version) (uint32_t cmd);
-    int (*domain_resource_map) (struct domain *d);
+    int (*dm_op)(struct domain *d);
+    int (*xen_version)(uint32_t cmd);
+    int (*domain_resource_map)(struct domain *d);
 #ifdef CONFIG_ARGO
-    int (*argo_enable) (const struct domain *d);
-    int (*argo_register_single_source) (const struct domain *d,
-                                        const struct domain *t);
-    int (*argo_register_any_source) (const struct domain *d);
-    int (*argo_send) (const struct domain *d, const struct domain *t);
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
 #endif
 };
 
@@ -191,191 +199,208 @@ extern struct xsm_operations *xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
-static inline void xsm_security_domaininfo (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info)
+static inline void xsm_security_domaininfo(
+    struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
     xsm_ops->security_domaininfo(d, info);
 }
 
-static inline int xsm_domain_create (xsm_default_t def, struct domain *d, u32 ssidref)
+static inline int xsm_domain_create(
+    xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
     return xsm_ops->domain_create(d, ssidref);
 }
 
-static inline int xsm_getdomaininfo (xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->getdomaininfo(d);
 }
 
-static inline int xsm_domctl_scheduler_op (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl_scheduler_op(
+    xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->domctl_scheduler_op(d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op (xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl_scheduler_op(cmd);
 }
 
-static inline int xsm_set_target (xsm_default_t def, struct domain *d, struct domain *e)
+static inline int xsm_set_target(
+    xsm_default_t def, struct domain *d, struct domain *e)
 {
     return xsm_ops->set_target(d, e);
 }
 
-static inline int xsm_domctl (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->domctl(d, cmd);
 }
 
-static inline int xsm_sysctl (xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl(cmd);
 }
 
-static inline int xsm_readconsole (xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
     return xsm_ops->readconsole(clear);
 }
 
-static inline int xsm_evtchn_unbound (xsm_default_t def, struct domain *d1, struct evtchn *chn,
-                                                                    domid_t id2)
+static inline int xsm_evtchn_unbound(
+    xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
     return xsm_ops->evtchn_unbound(d1, chn, id2);
 }
 
-static inline int xsm_evtchn_interdomain (xsm_default_t def, struct domain *d1,
-                struct evtchn *chan1, struct domain *d2, struct evtchn *chan2)
+static inline int xsm_evtchn_interdomain(
+    xsm_default_t def, struct domain *d1, struct evtchn *chan1,
+    struct domain *d2, struct evtchn *chan2)
 {
     return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
-static inline void xsm_evtchn_close_post (struct evtchn *chn)
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
     xsm_ops->evtchn_close_post(chn);
 }
 
-static inline int xsm_evtchn_send (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_send(
+    xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
     return xsm_ops->evtchn_send(d, chn);
 }
 
-static inline int xsm_evtchn_status (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_status(
+    xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
     return xsm_ops->evtchn_status(d, chn);
 }
 
-static inline int xsm_evtchn_reset (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_evtchn_reset(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->evtchn_reset(d1, d2);
 }
 
-static inline int xsm_grant_mapref (xsm_default_t def, struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static inline int xsm_grant_mapref(
+    xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
     return xsm_ops->grant_mapref(d1, d2, flags);
 }
 
-static inline int xsm_grant_unmapref (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_unmapref(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_unmapref(d1, d2);
 }
 
-static inline int xsm_grant_setup (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_setup(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_setup(d1, d2);
 }
 
-static inline int xsm_grant_transfer (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_transfer(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_transfer(d1, d2);
 }
 
-static inline int xsm_grant_copy (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_copy(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_copy(d1, d2);
 }
 
-static inline int xsm_grant_query_size (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_query_size(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_query_size(d1, d2);
 }
 
-static inline int xsm_alloc_security_domain (struct domain *d)
+static inline int xsm_alloc_security_domain(struct domain *d)
 {
     return xsm_ops->alloc_security_domain(d);
 }
 
-static inline void xsm_free_security_domain (struct domain *d)
+static inline void xsm_free_security_domain(struct domain *d)
 {
     xsm_ops->free_security_domain(d);
 }
 
 static inline int xsm_alloc_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+	struct evtchn chn[], unsigned int nr)
 {
     return xsm_ops->alloc_security_evtchns(chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(
-    struct evtchn chn[], unsigned int nr)
+	struct evtchn chn[], unsigned int nr)
 {
     xsm_ops->free_security_evtchns(chn, nr);
 }
 
-static inline char *xsm_show_security_evtchn (struct domain *d, const struct evtchn *chn)
+static inline char *xsm_show_security_evtchn(
+	struct domain *d, const struct evtchn *chn)
 {
     return xsm_ops->show_security_evtchn(d, chn);
 }
 
-static inline int xsm_init_hardware_domain (xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->init_hardware_domain(d);
 }
 
-static inline int xsm_get_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_pod_target(d);
 }
 
-static inline int xsm_set_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->set_pod_target(d);
 }
 
-static inline int xsm_memory_exchange (xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->memory_exchange(d);
 }
 
-static inline int xsm_memory_adjust_reservation (xsm_default_t def, struct domain *d1, struct
-                                                                    domain *d2)
+static inline int xsm_memory_adjust_reservation(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->memory_adjust_reservation(d1, d2);
 }
 
-static inline int xsm_memory_stat_reservation (xsm_default_t def, struct domain *d1,
-                                                            struct domain *d2)
+static inline int xsm_memory_stat_reservation(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->memory_stat_reservation(d1, d2);
 }
 
-static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1, struct domain *d2,
-                                      struct page_info *page)
+static inline int xsm_memory_pin_page(
+	xsm_default_t def, struct domain *d1, struct domain *d2,
+    struct page_info *page)
 {
     return xsm_ops->memory_pin_page(d1, d2, page);
 }
 
-static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_add_to_physmap(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->add_to_physmap(d1, d2);
 }
 
-static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_remove_from_physmap(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->remove_from_physmap(d1, d2);
 }
 
-static inline int xsm_map_gmfn_foreign (xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_map_gmfn_foreign(
+	xsm_default_t def, struct domain *d, struct domain *t)
 {
     return xsm_ops->map_gmfn_foreign(d, t);
 }
@@ -385,79 +410,87 @@ static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
     return xsm_ops->claim_pages(d);
 }
 
-static inline int xsm_console_io (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->console_io(d, cmd);
 }
 
-static inline int xsm_profile (xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
     return xsm_ops->profile(d, op);
 }
 
-static inline int xsm_kexec (xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t def)
 {
     return xsm_ops->kexec();
 }
 
-static inline int xsm_schedop_shutdown (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_schedop_shutdown(
+	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->schedop_shutdown(d1, d2);
 }
 
-static inline char *xsm_show_irq_sid (int irq)
+static inline char *xsm_show_irq_sid(int irq)
 {
     return xsm_ops->show_irq_sid(irq);
 }
 
-static inline int xsm_map_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->map_domain_pirq(d);
 }
 
-static inline int xsm_map_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_map_domain_irq(
+	xsm_default_t def, struct domain *d, int irq, void *data)
 {
     return xsm_ops->map_domain_irq(d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->unmap_domain_pirq(d);
 }
 
-static inline int xsm_unmap_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_unmap_domain_irq(
+	xsm_default_t def, struct domain *d, int irq, void *data)
 {
     return xsm_ops->unmap_domain_irq(d, irq, data);
 }
 
-static inline int xsm_bind_pt_irq(xsm_default_t def, struct domain *d,
-                                  struct xen_domctl_bind_pt_irq *bind)
+static inline int xsm_bind_pt_irq(
+	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     return xsm_ops->bind_pt_irq(d, bind);
 }
 
-static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
-                                    struct xen_domctl_bind_pt_irq *bind)
+static inline int xsm_unbind_pt_irq(
+	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     return xsm_ops->unbind_pt_irq(d, bind);
 }
 
-static inline int xsm_irq_permission (xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
+static inline int xsm_irq_permission(
+	xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
     return xsm_ops->irq_permission(d, pirq, allow);
 }
 
-static inline int xsm_iomem_permission (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_permission(
+	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_permission(d, s, e, allow);
 }
 
-static inline int xsm_iomem_mapping (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_mapping(
+	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pci_config_permission (xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access)
+static inline int xsm_pci_config_permission(
+	xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
+    uint16_t end, uint8_t access)
 {
     return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
 }
@@ -468,63 +501,67 @@ static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
     return xsm_ops->get_device_group(machine_bdf);
 }
 
-static inline int xsm_assign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_assign_device(
+	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
     return xsm_ops->assign_device(d, machine_bdf);
 }
 
-static inline int xsm_deassign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_deassign_device(
+	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
     return xsm_ops->deassign_device(d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static inline int xsm_assign_dtdevice(xsm_default_t def, struct domain *d,
-                                      const char *dtpath)
+static inline int xsm_assign_dtdevice(
+	xsm_default_t def, struct domain *d, const char *dtpath)
 {
     return xsm_ops->assign_dtdevice(d, dtpath);
 }
 
-static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
-                                        const char *dtpath)
+static inline int xsm_deassign_dtdevice(
+	xsm_default_t def, struct domain *d, const char *dtpath)
 {
     return xsm_ops->deassign_dtdevice(d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_plug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_unplug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_unplug_pci(
+	xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_unplug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_plug_core (xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t def)
 {
     return xsm_ops->resource_plug_core();
 }
 
-static inline int xsm_resource_unplug_core (xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
     return xsm_ops->resource_unplug_core();
 }
 
-static inline int xsm_resource_setup_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_setup_pci(
+	xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_setup_pci(machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi (xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
     return xsm_ops->resource_setup_gsi(gsi);
 }
 
-static inline int xsm_resource_setup_misc (xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
     return xsm_ops->resource_setup_misc();
 }
@@ -539,70 +576,74 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
 #endif
 
-static inline int xsm_hvm_param (xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_param(
+	xsm_default_t def, struct domain *d, unsigned long op)
 {
     return xsm_ops->hvm_param(d, op);
 }
 
-static inline int xsm_hvm_control(xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_control(
+	xsm_default_t def, struct domain *d, unsigned long op)
 {
     return xsm_ops->hvm_control(d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm (xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->hvm_param_altp2mhvm(d);
 }
 
-static inline int xsm_hvm_altp2mhvm_op (xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
+static inline int xsm_hvm_altp2mhvm_op(
+	xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
     return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo (xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_vnumainfo(d);
 }
 
-static inline int xsm_vm_event_control (xsm_default_t def, struct domain *d, int mode, int op)
+static inline int xsm_vm_event_control(
+	xsm_default_t def, struct domain *d, int mode, int op)
 {
     return xsm_ops->vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_sharing(d);
 }
 #endif
 
-static inline int xsm_platform_op (xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->platform_op(op);
 }
@@ -613,17 +654,19 @@ static inline int xsm_do_mca(xsm_default_t def)
     return xsm_ops->do_mca();
 }
 
-static inline int xsm_shadow_control (xsm_default_t def, struct domain *d, uint32_t op)
+static inline int xsm_shadow_control(
+	xsm_default_t def, struct domain *d, uint32_t op)
 {
     return xsm_ops->shadow_control(d, op);
 }
 
-static inline int xsm_mem_sharing_op (xsm_default_t def, struct domain *d, struct domain *cd, int op)
+static inline int xsm_mem_sharing_op(
+	xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
     return xsm_ops->mem_sharing_op(d, cd, op);
 }
 
-static inline int xsm_apic (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->apic(d, cmd);
 }
@@ -638,39 +681,45 @@ static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
     return xsm_ops->domain_memory_map(d);
 }
 
-static inline int xsm_mmu_update (xsm_default_t def, struct domain *d, struct domain *t,
-                                  struct domain *f, uint32_t flags)
+static inline int xsm_mmu_update(
+	xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
+    uint32_t flags)
 {
     return xsm_ops->mmu_update(d, t, f, flags);
 }
 
-static inline int xsm_mmuext_op (xsm_default_t def, struct domain *d, struct domain *f)
+static inline int xsm_mmuext_op(
+	xsm_default_t def, struct domain *d, struct domain *f)
 {
     return xsm_ops->mmuext_op(d, f);
 }
 
-static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d, struct domain *f,
-                                                            l1_pgentry_t pte)
+static inline int xsm_update_va_mapping(
+	xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
     return xsm_ops->update_va_mapping(d, f, pte);
 }
 
-static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_priv_mapping(
+	xsm_default_t def, struct domain *d, struct domain *t)
 {
     return xsm_ops->priv_mapping(d, t);
 }
 
-static inline int xsm_ioport_permission (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_permission(
+	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_permission(d, s, e, allow);
 }
 
-static inline int xsm_ioport_mapping (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_mapping(
+	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pmu_op (xsm_default_t def, struct domain *d, unsigned int op)
+static inline int xsm_pmu_op(
+	xsm_default_t def, struct domain *d, unsigned int op)
 {
     return xsm_ops->pmu_op(d, op);
 }
@@ -682,7 +731,7 @@ static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
     return xsm_ops->dm_op(d);
 }
 
-static inline int xsm_xen_version (xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->xen_version(op);
 }
@@ -698,8 +747,8 @@ static inline int xsm_argo_enable(const struct domain *d)
     return xsm_ops->argo_enable(d);
 }
 
-static inline int xsm_argo_register_single_source(const struct domain *d,
-                                                  const struct domain *t)
+static inline int xsm_argo_register_single_source(
+	const struct domain *d, const struct domain *t)
 {
     return xsm_ops->argo_register_single_source(d, t);
 }
@@ -719,12 +768,11 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 #endif /* XSM_NO_WRAPPERS */
 
 #ifdef CONFIG_MULTIBOOT
-extern int xsm_multiboot_init(unsigned long *module_map,
-                              const multiboot_info_t *mbi);
-extern int xsm_multiboot_policy_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi,
-                                     void **policy_buffer,
-                                     size_t *policy_size);
+int xsm_multiboot_init(
+	unsigned long *module_map, const multiboot_info_t *mbi);
+int xsm_multiboot_policy_init(
+	unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size);
 #endif
 
 #ifdef CONFIG_HAS_DEVICE_TREE
@@ -733,18 +781,19 @@ extern int xsm_multiboot_policy_init(unsigned long *module_map,
  *
  * On success, return 1 if using SILO mode else 0.
  */
-extern int xsm_dt_init(void);
-extern int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-extern bool has_xsm_magic(paddr_t);
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
 #endif
 
-extern int register_xsm(struct xsm_operations *ops);
+int register_xsm(struct xsm_operations *ops);
 
 extern struct xsm_operations dummy_xsm_ops;
-extern void xsm_fixup_ops(struct xsm_operations *ops);
+
+void xsm_fixup_ops(struct xsm_operations *ops);
 
 #ifdef CONFIG_XSM_FLASK
-extern void flask_init(const void *policy_buffer, size_t policy_size);
+void flask_init(const void *policy_buffer, size_t policy_size);
 #else
 static inline void flask_init(const void *policy_buffer, size_t policy_size)
 {
@@ -757,7 +806,7 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-extern void silo_init(void);
+void silo_init(void);
 #else
 static inline void silo_init(void) {}
 #endif
@@ -767,8 +816,8 @@ static inline void silo_init(void) {}
 #include <xsm/dummy.h>
 
 #ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init (unsigned long *module_map,
-                                      const multiboot_info_t *mbi)
+static inline int xsm_multiboot_init (
+	unsigned long *module_map, const multiboot_info_t *mbi)
 {
     return 0;
 }
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index ac553f9c0d..d313e8a6c6 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -117,8 +117,8 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
 }
 
 #ifdef CONFIG_MULTIBOOT
-int __init xsm_multiboot_init(unsigned long *module_map,
-                              const multiboot_info_t *mbi)
+int __init xsm_multiboot_init(
+	unsigned long *module_map, const multiboot_info_t *mbi)
 {
     int ret = 0;
     void *policy_buffer = NULL;
@@ -128,8 +128,8 @@ int __init xsm_multiboot_init(unsigned long *module_map,
 
     if ( XSM_MAGIC )
     {
-        ret = xsm_multiboot_policy_init(module_map, mbi,
-                                        &policy_buffer, &policy_size);
+        ret = xsm_multiboot_policy_init(module_map, mbi, &policy_buffer,
+                                        &policy_size);
         if ( ret )
         {
             bootstrap_map(NULL);
diff --git a/xen/xsm/xsm_policy.c b/xen/xsm/xsm_policy.c
index 33ab37717f..0ab1b3bb76 100644
--- a/xen/xsm/xsm_policy.c
+++ b/xen/xsm/xsm_policy.c
@@ -30,10 +30,9 @@
 #endif
 
 #ifdef CONFIG_MULTIBOOT
-int __init xsm_multiboot_policy_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi,
-                                     void **policy_buffer,
-                                     size_t *policy_size)
+int __init xsm_multiboot_policy_init(
+	unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size)
 {
     int i;
     module_t *mod = (module_t *)__va(mbi->mods_addr);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183606.331900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIh-0006hF-4D; Thu, 09 Sep 2021 22:13:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183606.331900; Thu, 09 Sep 2021 22:13:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIh-0006h4-0L; Thu, 09 Sep 2021 22:13:43 +0000
Received: by outflank-mailman (input) for mailman id 183606;
 Thu, 09 Sep 2021 22:13:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSIg-00067V-39
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:42 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2c8a5228-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:35 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163122555607978.87582860313216;
 Thu, 9 Sep 2021 15:12:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c8a5228-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225559; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=nO1bPAg+Mg89shWji9HtRLyv/wh7lAC4kluZBdV+sOF+nRNv/8D0Q5NB53rCCS7D5+CiSx9pBTM/WocrAD5VHNHr+y1ntBTkKNHjTtx3Axrf5x4EcxZSjB35yVHtR9tXUrvWdwJ93GGEttt7XymAQMKqy2He9wicZH5AakIJUPA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225559; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=3kFTpPpz7N6JilRkV79lBS4yIatdlwXsFoe6MNcLgi4=; 
	b=hmE5nN/uCY4XIRvrjX+jeixcfIXfAs05DvjovTBq5LrOHZZD+Vk0uO1pmQLxb8HR4HQyimM8b1jJceOO0HwM66nI1H65/5nqc1bBpCWs8ELmsDSscdDx0wwqyOfTqUwO4GJfyaZx9b4wiXMVwzIiElcrIv4oe5e+IOQfeaHbsWE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225559;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=3kFTpPpz7N6JilRkV79lBS4yIatdlwXsFoe6MNcLgi4=;
	b=Cmnl55u2GoOHlwxjFPl+DvMTlRZ+hIUjor0iLFNda9/8eES3LGdbrg+N6IB12XE+
	OgkBGic8Fo6KsCYIit9LmUlpR17tbP+AkcWWSrTGqs79oiIAyS2BnNQ+jef3x2NRH1e
	wtpSgwMEWHKFW0Qj0J31SA5/UuULOwP6WV6Qk5Lk=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 06/10] xsm: refactor xsm_ops handling
Date: Thu,  9 Sep 2021 22:12:31 -0400
Message-Id: <20210910021236.15082-7-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

This renames the `struct xsm_operations` to the shorter `struct xsm_ops` and
converts the global xsm_ops from being a pointer to an explicit instance. As
part of this conversion, it reworks the XSM modules init function to return
their xsm_ops struct which is copied in to the global xsm_ops instance.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xsm/xsm.h | 220 +++++++++++++++++++++---------------------
 xen/xsm/dummy.c       |   4 +-
 xen/xsm/flask/hooks.c |  12 +--
 xen/xsm/silo.c        |   7 +-
 xen/xsm/xsm_core.c    |  72 +++++++-------
 5 files changed, 161 insertions(+), 154 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 42b0c188a9..1d3a1be69d 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -41,7 +41,7 @@ enum xsm_default {
 };
 typedef enum xsm_default xsm_default_t;
 
-struct xsm_operations {
+struct xsm_ops {
     void (*security_domaininfo)(struct domain *d,
                                 struct xen_domctl_getdomaininfo *info);
     int (*domain_create)(struct domain *d, uint32_t ssidref);
@@ -195,322 +195,322 @@ struct xsm_operations {
 
 #ifdef CONFIG_XSM
 
-extern struct xsm_operations *xsm_ops;
+extern struct xsm_ops xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops->security_domaininfo(d, info);
+    xsm_ops.security_domaininfo(d, info);
 }
 
 static inline int xsm_domain_create(
     xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
-    return xsm_ops->domain_create(d, ssidref);
+    return xsm_ops.domain_create(d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->getdomaininfo(d);
+    return xsm_ops.getdomaininfo(d);
 }
 
 static inline int xsm_domctl_scheduler_op(
     xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->domctl_scheduler_op(d, cmd);
+    return xsm_ops.domctl_scheduler_op(d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl_scheduler_op(cmd);
+    return xsm_ops.sysctl_scheduler_op(cmd);
 }
 
 static inline int xsm_set_target(
     xsm_default_t def, struct domain *d, struct domain *e)
 {
-    return xsm_ops->set_target(d, e);
+    return xsm_ops.set_target(d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->domctl(d, cmd);
+    return xsm_ops.domctl(d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl(cmd);
+    return xsm_ops.sysctl(cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops->readconsole(clear);
+    return xsm_ops.readconsole(clear);
 }
 
 static inline int xsm_evtchn_unbound(
     xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops->evtchn_unbound(d1, chn, id2);
+    return xsm_ops.evtchn_unbound(d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
     xsm_default_t def, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
-    return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
+    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops->evtchn_close_post(chn);
+    xsm_ops.evtchn_close_post(chn);
 }
 
 static inline int xsm_evtchn_send(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops->evtchn_send(d, chn);
+    return xsm_ops.evtchn_send(d, chn);
 }
 
 static inline int xsm_evtchn_status(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops->evtchn_status(d, chn);
+    return xsm_ops.evtchn_status(d, chn);
 }
 
 static inline int xsm_evtchn_reset(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->evtchn_reset(d1, d2);
+    return xsm_ops.evtchn_reset(d1, d2);
 }
 
 static inline int xsm_grant_mapref(
     xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
-    return xsm_ops->grant_mapref(d1, d2, flags);
+    return xsm_ops.grant_mapref(d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_unmapref(d1, d2);
+    return xsm_ops.grant_unmapref(d1, d2);
 }
 
 static inline int xsm_grant_setup(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_setup(d1, d2);
+    return xsm_ops.grant_setup(d1, d2);
 }
 
 static inline int xsm_grant_transfer(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_transfer(d1, d2);
+    return xsm_ops.grant_transfer(d1, d2);
 }
 
 static inline int xsm_grant_copy(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_copy(d1, d2);
+    return xsm_ops.grant_copy(d1, d2);
 }
 
 static inline int xsm_grant_query_size(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_query_size(d1, d2);
+    return xsm_ops.grant_query_size(d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops->alloc_security_domain(d);
+    return xsm_ops.alloc_security_domain(d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops->free_security_domain(d);
+    xsm_ops.free_security_domain(d);
 }
 
 static inline int xsm_alloc_security_evtchns(
 	struct evtchn chn[], unsigned int nr)
 {
-    return xsm_ops->alloc_security_evtchns(chn, nr);
+    return xsm_ops.alloc_security_evtchns(chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(
 	struct evtchn chn[], unsigned int nr)
 {
-    xsm_ops->free_security_evtchns(chn, nr);
+    xsm_ops.free_security_evtchns(chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(
 	struct domain *d, const struct evtchn *chn)
 {
-    return xsm_ops->show_security_evtchn(d, chn);
+    return xsm_ops.show_security_evtchn(d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->init_hardware_domain(d);
+    return xsm_ops.init_hardware_domain(d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_pod_target(d);
+    return xsm_ops.get_pod_target(d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->set_pod_target(d);
+    return xsm_ops.set_pod_target(d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->memory_exchange(d);
+    return xsm_ops.memory_exchange(d);
 }
 
 static inline int xsm_memory_adjust_reservation(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->memory_adjust_reservation(d1, d2);
+    return xsm_ops.memory_adjust_reservation(d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->memory_stat_reservation(d1, d2);
+    return xsm_ops.memory_stat_reservation(d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
 	xsm_default_t def, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
-    return xsm_ops->memory_pin_page(d1, d2, page);
+    return xsm_ops.memory_pin_page(d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->add_to_physmap(d1, d2);
+    return xsm_ops.add_to_physmap(d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->remove_from_physmap(d1, d2);
+    return xsm_ops.remove_from_physmap(d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
 	xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops->map_gmfn_foreign(d, t);
+    return xsm_ops.map_gmfn_foreign(d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->claim_pages(d);
+    return xsm_ops.claim_pages(d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->console_io(d, cmd);
+    return xsm_ops.console_io(d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops->profile(d, op);
+    return xsm_ops.profile(d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops->kexec();
+    return xsm_ops.kexec();
 }
 
 static inline int xsm_schedop_shutdown(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->schedop_shutdown(d1, d2);
+    return xsm_ops.schedop_shutdown(d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops->show_irq_sid(irq);
+    return xsm_ops.show_irq_sid(irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->map_domain_pirq(d);
+    return xsm_ops.map_domain_pirq(d);
 }
 
 static inline int xsm_map_domain_irq(
 	xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops->map_domain_irq(d, irq, data);
+    return xsm_ops.map_domain_irq(d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->unmap_domain_pirq(d);
+    return xsm_ops.unmap_domain_pirq(d);
 }
 
 static inline int xsm_unmap_domain_irq(
 	xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops->unmap_domain_irq(d, irq, data);
+    return xsm_ops.unmap_domain_irq(d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
 	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->bind_pt_irq(d, bind);
+    return xsm_ops.bind_pt_irq(d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
 	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->unbind_pt_irq(d, bind);
+    return xsm_ops.unbind_pt_irq(d, bind);
 }
 
 static inline int xsm_irq_permission(
 	xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
-    return xsm_ops->irq_permission(d, pirq, allow);
+    return xsm_ops.irq_permission(d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
 	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_permission(d, s, e, allow);
+    return xsm_ops.iomem_permission(d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
 	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_mapping(d, s, e, allow);
+    return xsm_ops.iomem_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
     uint16_t end, uint8_t access)
 {
-    return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
+    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->get_device_group(machine_bdf);
+    return xsm_ops.get_device_group(machine_bdf);
 }
 
 static inline int xsm_assign_device(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops->assign_device(d, machine_bdf);
+    return xsm_ops.assign_device(d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops->deassign_device(d, machine_bdf);
+    return xsm_ops.deassign_device(d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -518,249 +518,249 @@ static inline int xsm_deassign_device(
 static inline int xsm_assign_dtdevice(
 	xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops->assign_dtdevice(d, dtpath);
+    return xsm_ops.assign_dtdevice(d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
 	xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops->deassign_dtdevice(d, dtpath);
+    return xsm_ops.deassign_dtdevice(d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_plug_pci(machine_bdf);
+    return xsm_ops.resource_plug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
 	xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_unplug_pci(machine_bdf);
+    return xsm_ops.resource_unplug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_plug_core();
+    return xsm_ops.resource_plug_core();
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_unplug_core();
+    return xsm_ops.resource_unplug_core();
 }
 
 static inline int xsm_resource_setup_pci(
 	xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_setup_pci(machine_bdf);
+    return xsm_ops.resource_setup_pci(machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops->resource_setup_gsi(gsi);
+    return xsm_ops.resource_setup_gsi(gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops->resource_setup_misc();
+    return xsm_ops.resource_setup_misc();
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops->page_offline(cmd);
+    return xsm_ops.page_offline(cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops->hypfs_op();
+    return xsm_ops.hypfs_op();
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_xsm_op(op);
+    return xsm_ops.do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
 static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_compat_op(op);
+    return xsm_ops.do_compat_op(op);
 }
 #endif
 
 static inline int xsm_hvm_param(
 	xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops->hvm_param(d, op);
+    return xsm_ops.hvm_param(d, op);
 }
 
 static inline int xsm_hvm_control(
 	xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops->hvm_control(d, op);
+    return xsm_ops.hvm_control(d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->hvm_param_altp2mhvm(d);
+    return xsm_ops.hvm_param_altp2mhvm(d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
 	xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
-    return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
+    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_vnumainfo(d);
+    return xsm_ops.get_vnumainfo(d);
 }
 
 static inline int xsm_vm_event_control(
 	xsm_default_t def, struct domain *d, int mode, int op)
 {
-    return xsm_ops->vm_event_control(d, mode, op);
+    return xsm_ops.vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_access(d);
+    return xsm_ops.mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_paging(d);
+    return xsm_ops.mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_sharing(d);
+    return xsm_ops.mem_sharing(d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->platform_op(op);
+    return xsm_ops.platform_op(op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops->do_mca();
+    return xsm_ops.do_mca();
 }
 
 static inline int xsm_shadow_control(
 	xsm_default_t def, struct domain *d, uint32_t op)
 {
-    return xsm_ops->shadow_control(d, op);
+    return xsm_ops.shadow_control(d, op);
 }
 
 static inline int xsm_mem_sharing_op(
 	xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
-    return xsm_ops->mem_sharing_op(d, cd, op);
+    return xsm_ops.mem_sharing_op(d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->apic(d, cmd);
+    return xsm_ops.apic(d, cmd);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops->machine_memory_map();
+    return xsm_ops.machine_memory_map();
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_memory_map(d);
+    return xsm_ops.domain_memory_map(d);
 }
 
 static inline int xsm_mmu_update(
 	xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
-    return xsm_ops->mmu_update(d, t, f, flags);
+    return xsm_ops.mmu_update(d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
 	xsm_default_t def, struct domain *d, struct domain *f)
 {
-    return xsm_ops->mmuext_op(d, f);
+    return xsm_ops.mmuext_op(d, f);
 }
 
 static inline int xsm_update_va_mapping(
 	xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
-    return xsm_ops->update_va_mapping(d, f, pte);
+    return xsm_ops.update_va_mapping(d, f, pte);
 }
 
 static inline int xsm_priv_mapping(
 	xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops->priv_mapping(d, t);
+    return xsm_ops.priv_mapping(d, t);
 }
 
 static inline int xsm_ioport_permission(
 	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_permission(d, s, e, allow);
+    return xsm_ops.ioport_permission(d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
 	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_mapping(d, s, e, allow);
+    return xsm_ops.ioport_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
 	xsm_default_t def, struct domain *d, unsigned int op)
 {
-    return xsm_ops->pmu_op(d, op);
+    return xsm_ops.pmu_op(d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->dm_op(d);
+    return xsm_ops.dm_op(d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->xen_version(op);
+    return xsm_ops.xen_version(op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_resource_map(d);
+    return xsm_ops.domain_resource_map(d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops->argo_enable(d);
+    return xsm_ops.argo_enable(d);
 }
 
 static inline int xsm_argo_register_single_source(
 	const struct domain *d, const struct domain *t)
 {
-    return xsm_ops->argo_register_single_source(d, t);
+    return xsm_ops.argo_register_single_source(d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops->argo_register_any_source(d);
+    return xsm_ops.argo_register_any_source(d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops->argo_send(d, t);
+    return xsm_ops.argo_send(d, t);
 }
 
 #endif /* CONFIG_ARGO */
@@ -786,17 +786,18 @@ int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
 bool has_xsm_magic(paddr_t);
 #endif
 
-int register_xsm(struct xsm_operations *ops);
+extern struct xsm_ops dummy_xsm_ops;
 
-extern struct xsm_operations dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_operations *ops);
+void xsm_fixup_ops(struct xsm_ops *ops);
 
 #ifdef CONFIG_XSM_FLASK
-void flask_init(const void *policy_buffer, size_t policy_size);
+extern const struct xsm_ops *flask_init(const void *policy_buffer,
+                                        size_t policy_size);
 #else
-static inline void flask_init(const void *policy_buffer, size_t policy_size)
+static const inline struct xsm_ops *flask_init(const void *policy_buffer,
+                                               size_t policy_size)
 {
+    return NULL;
 }
 #endif
 
@@ -806,9 +807,12 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-void silo_init(void);
+extern const struct xsm_ops *silo_init(void);
 #else
-static inline void silo_init(void) {}
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
 #endif
 
 #else /* CONFIG_XSM */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index de44b10130..d8c935328e 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -13,15 +13,13 @@
 #define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
-struct xsm_operations dummy_xsm_ops;
-
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
         if ( !ops->function )                                          \
             ops->function = xsm_##function;                            \
     } while (0)
 
-void __init xsm_fixup_ops (struct xsm_operations *ops)
+void __init xsm_fixup_ops (struct xsm_ops *ops)
 {
     set_to_dummy_if_null(ops, security_domaininfo);
     set_to_dummy_if_null(ops, domain_create);
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 1465db125a..25e87180b4 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1745,7 +1745,7 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
-static struct xsm_operations flask_ops = {
+static const struct xsm_ops __initconstrel flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
     .domain_create = flask_domain_create,
     .getdomaininfo = flask_getdomaininfo,
@@ -1883,7 +1883,8 @@ static struct xsm_operations flask_ops = {
 #endif
 };
 
-void __init flask_init(const void *policy_buffer, size_t policy_size)
+const struct xsm_ops *__init flask_init(const void *policy_buffer,
+                                        size_t policy_size)
 {
     int ret = -ENOENT;
 
@@ -1891,7 +1892,7 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
     {
     case FLASK_BOOTPARAM_DISABLED:
         printk(XENLOG_INFO "Flask: Disabled at boot.\n");
-        return;
+        return NULL;
 
     case FLASK_BOOTPARAM_PERMISSIVE:
         flask_enforcing = 0;
@@ -1908,9 +1909,6 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
 
     avc_init();
 
-    if ( register_xsm(&flask_ops) )
-        panic("Flask: Unable to register with XSM\n");
-
     if ( policy_size && flask_bootparam != FLASK_BOOTPARAM_LATELOAD )
         ret = security_load_policy(policy_buffer, policy_size);
 
@@ -1923,6 +1921,8 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
         printk(XENLOG_INFO "Flask:  Starting in enforcing mode.\n");
     else
         printk(XENLOG_INFO "Flask:  Starting in permissive mode.\n");
+
+    return &flask_ops;
 }
 
 /*
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index fc2ca5cd2d..3550dded7b 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -100,7 +100,7 @@ static int silo_argo_send(const struct domain *d1, const struct domain *d2)
 
 #endif
 
-static struct xsm_operations silo_xsm_ops = {
+static const struct xsm_ops __initconstrel silo_xsm_ops = {
     .evtchn_unbound = silo_evtchn_unbound,
     .evtchn_interdomain = silo_evtchn_interdomain,
     .grant_mapref = silo_grant_mapref,
@@ -112,12 +112,11 @@ static struct xsm_operations silo_xsm_ops = {
 #endif
 };
 
-void __init silo_init(void)
+const struct xsm_ops *__init silo_init(void)
 {
     printk("Initialising XSM SILO mode\n");
 
-    if ( register_xsm(&silo_xsm_ops) )
-        panic("SILO: Unable to register with XSM\n");
+    return &silo_xsm_ops;
 }
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index d313e8a6c6..a49fdec741 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -28,9 +28,17 @@
 #include <asm/setup.h>
 #endif
 
-#define XSM_FRAMEWORK_VERSION    "1.0.0"
+#define XSM_FRAMEWORK_VERSION    "1.0.1"
 
-struct xsm_operations *xsm_ops;
+struct xsm_ops __read_mostly xsm_ops;
+
+enum xsm_ops_state {
+    XSM_OPS_UNREGISTERED,
+    XSM_OPS_REG_FAILED,
+    XSM_OPS_REGISTERED,
+};
+
+static enum xsm_ops_state __initdata xsm_ops_registered = XSM_OPS_UNREGISTERED;
 
 enum xsm_bootparam {
     XSM_BOOTPARAM_DUMMY,
@@ -68,17 +76,10 @@ static int __init parse_xsm_param(const char *s)
 }
 custom_param("xsm", parse_xsm_param);
 
-static inline int verify(struct xsm_operations *ops)
-{
-    /* verify the security_operations structure exists */
-    if ( !ops )
-        return -EINVAL;
-    xsm_fixup_ops(ops);
-    return 0;
-}
-
 static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
 {
+    const struct xsm_ops *ops = NULL;
+
 #ifdef CONFIG_XSM_FLASK_POLICY
     if ( policy_size == 0 )
     {
@@ -87,25 +88,25 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
     }
 #endif
 
-    if ( verify(&dummy_xsm_ops) )
+    if ( xsm_ops_registered != XSM_OPS_UNREGISTERED )
     {
-        printk(XENLOG_ERR "Could not verify dummy_xsm_ops structure\n");
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register already attempted\n");
         return -EIO;
     }
 
-    xsm_ops = &dummy_xsm_ops;
-
     switch ( xsm_bootparam )
     {
     case XSM_BOOTPARAM_DUMMY:
+        xsm_ops_registered = XSM_OPS_REGISTERED;
         break;
 
     case XSM_BOOTPARAM_FLASK:
-        flask_init(policy_buffer, policy_size);
+        ops = flask_init(policy_buffer, policy_size);
         break;
 
     case XSM_BOOTPARAM_SILO:
-        silo_init();
+        ops = silo_init();
         break;
 
     default:
@@ -113,6 +114,27 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
         break;
     }
 
+    if ( ops )
+    {
+        xsm_ops_registered = XSM_OPS_REGISTERED;
+        xsm_ops = *ops;
+    }
+    /*
+     * This handles three cases,
+     *   - dummy policy module was selected
+     *   - a policy module does not provide all handlers
+     *   - a policy module failed to init
+     */
+    xsm_fixup_ops(&xsm_ops);
+
+    if ( xsm_ops_registered != XSM_OPS_REGISTERED )
+    {
+        xsm_ops_registered = XSM_OPS_REG_FAILED;
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register failed\n");
+        return -EFAULT;
+    }
+
     return 0;
 }
 
@@ -195,22 +217,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-int __init register_xsm(struct xsm_operations *ops)
-{
-    if ( verify(ops) )
-    {
-        printk(XENLOG_ERR "Could not verify xsm_operations structure\n");
-        return -EINVAL;
-    }
-
-    if ( xsm_ops != &dummy_xsm_ops )
-        return -EAGAIN;
-
-    xsm_ops = ops;
-
-    return 0;
-}
-
 #endif
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183610.331911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIl-0007Ac-Kr; Thu, 09 Sep 2021 22:13:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183610.331911; Thu, 09 Sep 2021 22:13:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIl-0007AV-HW; Thu, 09 Sep 2021 22:13:47 +0000
Received: by outflank-mailman (input) for mailman id 183610;
 Thu, 09 Sep 2021 22:13:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSIl-00067V-3F
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:47 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2fe0a832-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:40 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163122555715363.275802575096236;
 Thu, 9 Sep 2021 15:12:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fe0a832-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225560; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=c1c4KJu5PKQBH7VbnL8EdGjOUDqLmNzkabWco/GVYzFNO0Z8oGmw+Xj01hFn1XA5gKgMD4SMnclkkUqEoRMdcrPXZ6xV6E16PqV7zkziXvR8gXBB3aiq1PfNoh67RBP3vCJDWMNC7ibNJyYeb3vPos5cWxD5rrZ9mAO31XPEzak=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225560; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=CDXoQhlHqYlB6RcyCL6RNbfFCV/qaWUvnEQwBo6K6fw=; 
	b=cgdIGz7nc+Bq2GqIn3QHDRvQKqJmoUfz2Vf3qkN7853v7VNkB4QHvInd2EVe5WX61rdG84yeWzdmFsUaQMhGroUkAbaLHclasfakcJUFxO0kCvJV6aI/Klouxd3j1p25KcDNYQQpH5JPZzfwWRwdT+ICyPIdPbfScCHt/1pVkbs=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225560;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=CDXoQhlHqYlB6RcyCL6RNbfFCV/qaWUvnEQwBo6K6fw=;
	b=NgqPxYWrpJDTXSDSuV3D67uF+JdRpTvZNnCHZUSsoPAQJPVrYBjwWf7BQaMqGlTT
	YRXWrQiJ7y00pli4BwuC5+h1nZy5zyWlg1AQmxqhbpfoS5ZWOsN8LgpMtZ4F2YdM8c+
	Rxp0AtwAJYljU4InJAva3ylTfq8Dg4P04R3VFWbk=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 07/10] xsm: convert xsm_ops hook calls to alternative call
Date: Thu,  9 Sep 2021 22:12:32 -0400
Message-Id: <20210910021236.15082-8-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

To reduce retpolines convert all the pointer function calls of the
xsm_ops hooks over to the alternative_call infrastructure.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xsm/xsm.h | 191 +++++++++++++++++++++---------------------
 1 file changed, 96 insertions(+), 95 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 1d3a1be69d..80dc7f5373 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -15,6 +15,7 @@
 #ifndef __XSM_H__
 #define __XSM_H__
 
+#include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
@@ -202,315 +203,315 @@ extern struct xsm_ops xsm_ops;
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops.security_domaininfo(d, info);
+    alternative_vcall(xsm_ops.security_domaininfo, d, info);
 }
 
 static inline int xsm_domain_create(
     xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
-    return xsm_ops.domain_create(d, ssidref);
+    return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.getdomaininfo(d);
+    return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
 static inline int xsm_domctl_scheduler_op(
     xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.domctl_scheduler_op(d, cmd);
+    return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl_scheduler_op(cmd);
+    return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
 static inline int xsm_set_target(
     xsm_default_t def, struct domain *d, struct domain *e)
 {
-    return xsm_ops.set_target(d, e);
+    return alternative_call(xsm_ops.set_target, d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.domctl(d, cmd);
+    return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl(cmd);
+    return alternative_call(xsm_ops.sysctl, cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops.readconsole(clear);
+    return alternative_call(xsm_ops.readconsole, clear);
 }
 
 static inline int xsm_evtchn_unbound(
     xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops.evtchn_unbound(d1, chn, id2);
+    return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
     xsm_default_t def, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
-    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
+    return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops.evtchn_close_post(chn);
+    alternative_vcall(xsm_ops.evtchn_close_post, chn);
 }
 
 static inline int xsm_evtchn_send(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops.evtchn_send(d, chn);
+    return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
 static inline int xsm_evtchn_status(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops.evtchn_status(d, chn);
+    return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
 static inline int xsm_evtchn_reset(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.evtchn_reset(d1, d2);
+    return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
 static inline int xsm_grant_mapref(
     xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
-    return xsm_ops.grant_mapref(d1, d2, flags);
+    return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_unmapref(d1, d2);
+    return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
 static inline int xsm_grant_setup(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_setup(d1, d2);
+    return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
 static inline int xsm_grant_transfer(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_transfer(d1, d2);
+    return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
 static inline int xsm_grant_copy(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_copy(d1, d2);
+    return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
 static inline int xsm_grant_query_size(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_query_size(d1, d2);
+    return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops.alloc_security_domain(d);
+    return alternative_call(xsm_ops.alloc_security_domain, d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops.free_security_domain(d);
+    alternative_vcall(xsm_ops.free_security_domain, d);
 }
 
 static inline int xsm_alloc_security_evtchns(
 	struct evtchn chn[], unsigned int nr)
 {
-    return xsm_ops.alloc_security_evtchns(chn, nr);
+    return alternative_call(xsm_ops.alloc_security_evtchns, chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(
 	struct evtchn chn[], unsigned int nr)
 {
-    xsm_ops.free_security_evtchns(chn, nr);
+    alternative_vcall(xsm_ops.free_security_evtchns, chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(
 	struct domain *d, const struct evtchn *chn)
 {
-    return xsm_ops.show_security_evtchn(d, chn);
+    return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.init_hardware_domain(d);
+    return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_pod_target(d);
+    return alternative_call(xsm_ops.get_pod_target, d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.set_pod_target(d);
+    return alternative_call(xsm_ops.set_pod_target, d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.memory_exchange(d);
+    return alternative_call(xsm_ops.memory_exchange, d);
 }
 
 static inline int xsm_memory_adjust_reservation(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.memory_adjust_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.memory_stat_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
 	xsm_default_t def, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
-    return xsm_ops.memory_pin_page(d1, d2, page);
+    return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.add_to_physmap(d1, d2);
+    return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.remove_from_physmap(d1, d2);
+    return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
 	xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops.map_gmfn_foreign(d, t);
+    return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.claim_pages(d);
+    return alternative_call(xsm_ops.claim_pages, d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.console_io(d, cmd);
+    return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops.profile(d, op);
+    return alternative_call(xsm_ops.profile, d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops.kexec();
+    return alternative_call(xsm_ops.kexec);
 }
 
 static inline int xsm_schedop_shutdown(
 	xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.schedop_shutdown(d1, d2);
+    return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops.show_irq_sid(irq);
+    return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.map_domain_pirq(d);
+    return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
 static inline int xsm_map_domain_irq(
 	xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops.map_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.unmap_domain_pirq(d);
+    return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
 static inline int xsm_unmap_domain_irq(
 	xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops.unmap_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
 	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.bind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
 	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.unbind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
 static inline int xsm_irq_permission(
 	xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
-    return xsm_ops.irq_permission(d, pirq, allow);
+    return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
 	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
 	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
     uint16_t end, uint8_t access)
 {
-    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.get_device_group(machine_bdf);
+    return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
 static inline int xsm_assign_device(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops.assign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
 	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops.deassign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -518,62 +519,62 @@ static inline int xsm_deassign_device(
 static inline int xsm_assign_dtdevice(
 	xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops.assign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
 	xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops.deassign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_plug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
 	xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_unplug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_plug_core();
+    return alternative_call(xsm_ops.resource_plug_core);
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_unplug_core();
+    return alternative_call(xsm_ops.resource_unplug_core);
 }
 
 static inline int xsm_resource_setup_pci(
 	xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_setup_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops.resource_setup_gsi(gsi);
+    return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops.resource_setup_misc();
+    return alternative_call(xsm_ops.resource_setup_misc);
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops.page_offline(cmd);
+    return alternative_call(xsm_ops.page_offline, cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops.hypfs_op();
+    return alternative_call(xsm_ops.hypfs_op);
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
@@ -591,107 +592,107 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 static inline int xsm_hvm_param(
 	xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops.hvm_param(d, op);
+    return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
 static inline int xsm_hvm_control(
 	xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops.hvm_control(d, op);
+    return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.hvm_param_altp2mhvm(d);
+    return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
 	xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
-    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
+    return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_vnumainfo(d);
+    return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
 static inline int xsm_vm_event_control(
 	xsm_default_t def, struct domain *d, int mode, int op)
 {
-    return xsm_ops.vm_event_control(d, mode, op);
+    return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_access(d);
+    return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_paging(d);
+    return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_sharing(d);
+    return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.platform_op(op);
+    return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops.do_mca();
+    return alternative_call(xsm_ops.do_mca);
 }
 
 static inline int xsm_shadow_control(
 	xsm_default_t def, struct domain *d, uint32_t op)
 {
-    return xsm_ops.shadow_control(d, op);
+    return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
 static inline int xsm_mem_sharing_op(
 	xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
-    return xsm_ops.mem_sharing_op(d, cd, op);
+    return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.apic(d, cmd);
+    return alternative_call(xsm_ops.apic, d, cmd);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops.machine_memory_map();
+    return alternative_call(xsm_ops.machine_memory_map);
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_memory_map(d);
+    return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
 static inline int xsm_mmu_update(
 	xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
-    return xsm_ops.mmu_update(d, t, f, flags);
+    return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
 	xsm_default_t def, struct domain *d, struct domain *f)
 {
-    return xsm_ops.mmuext_op(d, f);
+    return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
 static inline int xsm_update_va_mapping(
@@ -703,64 +704,64 @@ static inline int xsm_update_va_mapping(
 static inline int xsm_priv_mapping(
 	xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops.priv_mapping(d, t);
+    return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
 static inline int xsm_ioport_permission(
 	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
 	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
 	xsm_default_t def, struct domain *d, unsigned int op)
 {
-    return xsm_ops.pmu_op(d, op);
+    return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.dm_op(d);
+    return alternative_call(xsm_ops.dm_op, d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.xen_version(op);
+    return alternative_call(xsm_ops.xen_version, op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_resource_map(d);
+    return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops.argo_enable(d);
+    return alternative_call(xsm_ops.argo_enable, d);
 }
 
 static inline int xsm_argo_register_single_source(
 	const struct domain *d, const struct domain *t)
 {
-    return xsm_ops.argo_register_single_source(d, t);
+    return alternative_call(xsm_ops.argo_register_single_source, d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops.argo_register_any_source(d);
+    return alternative_call(xsm_ops.argo_register_any_source, d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops.argo_send(d, t);
+    return alternative_call(xsm_ops.argo_send, d, t);
 }
 
 #endif /* CONFIG_ARGO */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:13:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:13:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183616.331922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIv-0007pn-UN; Thu, 09 Sep 2021 22:13:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183616.331922; Thu, 09 Sep 2021 22:13:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSIv-0007pf-QW; Thu, 09 Sep 2021 22:13:57 +0000
Received: by outflank-mailman (input) for mailman id 183616;
 Thu, 09 Sep 2021 22:13:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSIv-00067V-3f
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:13:57 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 328711f2-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:45 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225558199880.4487268993798;
 Thu, 9 Sep 2021 15:12:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 328711f2-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225561; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=XPkyH0VMWxZ14wKo3qQ4Fy/57RI5ByTzkwkZi+RYvzrRjdwZgN8PdRt0fC3aB2JoFezuETksWAD+WRi5P77N416AjT2rA3S1u/+IWHP0VmcH4CyXes15cZgghD/cqmMnEzuHUvBRKOAFNZVLWMDEWYsZHfQrnmHkMTRkT8kvPAw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225561; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=KBSDJGpTWXEy7/PFGTXPIuRb2Leatn16J4JfBz57Zik=; 
	b=QjC4atGRq+vT+SA24Bsqw6fsPSZOT1UdAzVLn7cZuMvgtb4FS4h/cnDE0ttxp6IWL1RN8YaIB63YKVjUWC9gVIk3xLT07JxOO7flAoP1fdPw86Q+gseWLBQ1rrrAPKMKAOJHBSrPJZp/MobMpCCES5vaQGupd0xet1PELG9kftg=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225561;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=KBSDJGpTWXEy7/PFGTXPIuRb2Leatn16J4JfBz57Zik=;
	b=T8EOl66e5ixzSvRYylm1dWnxMr6AAUQXUij0x+zysIi7lvvtiNWj1+VKrViIfxtX
	rzKpjQGN4zR5yd3JV34QMKVTmqIn6pTYL2yU2a0xi2q/q71Jy45WIiqrKYiQDIQbvJA
	nCQTuFK4XaqSXyvED2DKOFCCIzMISW3uGs/6UbVA=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 08/10] xsm: decouple xsm header inclusion selection
Date: Thu,  9 Sep 2021 22:12:33 -0400
Message-Id: <20210910021236.15082-9-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Multiple preprocessor defines were used as a mechanism to selective include
parts of the xsm.h header file. This makes it difficult to know which portion
is being included at any one time. This commit works to simplify this by
separating the core structures and functions of XSM into xsm-core.h away from
the wrapper functions which remain in xsm.h and dummy.h.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h    |   2 +-
 xen/include/xsm/xsm-core.h | 267 +++++++++++++++++++++++++++++++++++++
 xen/include/xsm/xsm.h      | 248 +---------------------------------
 xen/xsm/dummy.c            |   1 -
 xen/xsm/silo.c             |   2 +-
 xen/xsm/xsm_core.c         |   5 +
 6 files changed, 275 insertions(+), 250 deletions(-)
 create mode 100644 xen/include/xsm/xsm-core.h

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index bfcfee1d9e..641b07e8b1 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -16,7 +16,7 @@
  */
 
 #include <xen/sched.h>
-#include <xsm/xsm.h>
+#include <xsm/xsm-core.h>
 #include <public/hvm/params.h>
 
 /*
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
new file mode 100644
index 0000000000..514c198ee6
--- /dev/null
+++ b/xen/include/xsm/xsm-core.h
@@ -0,0 +1,267 @@
+/*
+ *  This file contains the XSM hook definitions for Xen.
+ *
+ *  This work is based on the LSM implementation in Linux 2.6.13.4.
+ *
+ *  Author:  George Coker, <gscoker@alpha.ncsc.mil>
+ *
+ *  Contributors: Michael LeMay, <mdlemay@epoch.ncsc.mil>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ */
+
+#ifndef __XSM_CORE_H__
+#define __XSM_CORE_H__
+
+#include <xen/sched.h>
+#include <xen/multiboot.h>
+
+/* policy magic number */
+typedef uint32_t xsm_magic_t;
+
+#ifdef CONFIG_XSM_FLASK
+#define XSM_MAGIC 0xf97cff8c
+#else
+#define XSM_MAGIC 0x0
+#endif
+
+/*
+ * These annotations are used by callers and in dummy.h to document the
+ * default actions of XSM hooks. They should be compiled out otherwise.
+ */
+enum xsm_default {
+    XSM_HOOK,     /* Guests can normally access the hypercall */
+    XSM_DM_PRIV,  /* Device model can perform on its target domain */
+    XSM_TARGET,   /* Can perform on self or your target domain */
+    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
+    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
+    XSM_OTHER     /* Something more complex */
+};
+typedef enum xsm_default xsm_default_t;
+
+struct xsm_ops {
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
+#endif
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
+#endif
+
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
+
+    int (*page_offline)(uint32_t cmd);
+    int (*hypfs_op)(void);
+
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#ifdef CONFIG_COMPAT
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#endif
+
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
+
+    int (*vm_event_control)(struct domain *d, int mode, int op);
+
+#ifdef CONFIG_MEM_ACCESS
+    int (*mem_access)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+    int (*mem_paging)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+    int (*mem_sharing)(struct domain *d);
+#endif
+
+    int (*platform_op)(uint32_t cmd);
+
+#ifdef CONFIG_X86
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
+#define XSM_MMU_UPDATE_READ      1
+#define XSM_MMU_UPDATE_WRITE     2
+#define XSM_MMU_NORMAL_UPDATE    4
+#define XSM_MMU_MACHPHYS_UPDATE  8
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
+#endif
+
+    int (*dm_op)(struct domain *d);
+
+    int (*xen_version)(uint32_t cmd);
+
+    int (*domain_resource_map)(struct domain *d);
+
+#ifdef CONFIG_ARGO
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
+#endif
+};
+
+void xsm_fixup_ops(struct xsm_ops *ops);
+
+#ifdef CONFIG_XSM
+
+#ifdef CONFIG_MULTIBOOT
+int xsm_multiboot_init(
+	unsigned long *module_map, const multiboot_info_t *mbi);
+int xsm_multiboot_policy_init(
+	unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size);
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+/*
+ * Initialize XSM
+ *
+ * On success, return 1 if using SILO mode else 0.
+ */
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
+#endif
+
+#ifdef CONFIG_XSM_FLASK
+const struct xsm_ops *flask_init(
+	const void *policy_buffer, size_t policy_size);
+#else
+static inline const struct xsm_ops *flask_init(
+	const void *policy_buffer, size_t policy_size)
+{
+    return NULL;
+}
+#endif
+
+#ifdef CONFIG_XSM_SILO
+const struct xsm_ops *silo_init(void);
+#else
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
+#endif
+
+#else /* CONFIG_XSM */
+
+#ifdef CONFIG_MULTIBOOT
+static inline int xsm_multiboot_init(
+	unsigned long *module_map, const multiboot_info_t *mbi)
+{
+    return 0;
+}
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+static inline int xsm_dt_init(void)
+{
+    return 0;
+}
+
+static inline bool has_xsm_magic(paddr_t start)
+{
+    return false;
+}
+#endif /* CONFIG_HAS_DEVICE_TREE */
+
+#endif /* CONFIG_XSM */
+
+#endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 80dc7f5373..47337ecbd3 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,188 +18,12 @@
 #include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
-
-/* policy magic number (defined by XSM_MAGIC) */
-typedef uint32_t xsm_magic_t;
-
-#ifdef CONFIG_XSM_FLASK
-#define XSM_MAGIC 0xf97cff8c
-#else
-#define XSM_MAGIC 0x0
-#endif
-
-/*
- * These annotations are used by callers and in dummy.h to document the
- * default actions of XSM hooks. They should be compiled out otherwise.
- */
-enum xsm_default {
-    XSM_HOOK,     /* Guests can normally access the hypercall */
-    XSM_DM_PRIV,  /* Device model can perform on its target domain */
-    XSM_TARGET,   /* Can perform on self or your target domain */
-    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
-    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
-    XSM_OTHER     /* Something more complex */
-};
-typedef enum xsm_default xsm_default_t;
-
-struct xsm_ops {
-    void (*security_domaininfo)(struct domain *d,
-                                struct xen_domctl_getdomaininfo *info);
-    int (*domain_create)(struct domain *d, uint32_t ssidref);
-    int (*getdomaininfo)(struct domain *d);
-    int (*domctl_scheduler_op)(struct domain *d, int op);
-    int (*sysctl_scheduler_op)(int op);
-    int (*set_target)(struct domain *d, struct domain *e);
-    int (*domctl)(struct domain *d, int cmd);
-    int (*sysctl)(int cmd);
-    int (*readconsole)(uint32_t clear);
-
-    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
-                              struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post)(struct evtchn *chn);
-    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
-    int (*grant_setup)(struct domain *d1, struct domain *d2);
-    int (*grant_transfer)(struct domain *d1, struct domain *d2);
-    int (*grant_copy)(struct domain *d1, struct domain *d2);
-    int (*grant_query_size)(struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain)(struct domain *d);
-    void (*free_security_domain)(struct domain *d);
-    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain)(struct domain *d);
-
-    int (*get_pod_target)(struct domain *d);
-    int (*set_pod_target)(struct domain *d);
-    int (*memory_exchange)(struct domain *d);
-    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
-                           struct page_info *page);
-    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
-    int (*claim_pages)(struct domain *d);
-
-    int (*console_io)(struct domain *d, int cmd);
-
-    int (*profile)(struct domain *d, int op);
-
-    int (*kexec)(void);
-    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid)(int irq);
-    int (*map_domain_pirq)(struct domain *d);
-    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq)(struct domain *d);
-    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
-                            uint8_t allow);
-    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
-                         uint8_t allow);
-    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
-                                 uint16_t start, uint16_t end, uint8_t access);
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group)(uint32_t machine_bdf);
-    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
-#endif
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
-#endif
-
-    int (*resource_plug_core)(void);
-    int (*resource_unplug_core)(void);
-    int (*resource_plug_pci)(uint32_t machine_bdf);
-    int (*resource_unplug_pci)(uint32_t machine_bdf);
-    int (*resource_setup_pci)(uint32_t machine_bdf);
-    int (*resource_setup_gsi)(int gsi);
-    int (*resource_setup_misc)(void);
-
-    int (*page_offline)(uint32_t cmd);
-    int (*hypfs_op)(void);
-
-    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#ifdef CONFIG_COMPAT
-    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#endif
-
-    int (*hvm_param)(struct domain *d, unsigned long op);
-    int (*hvm_control)(struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm)(struct domain *d);
-    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo)(struct domain *d);
-
-    int (*vm_event_control)(struct domain *d, int mode, int op);
-
-#ifdef CONFIG_MEM_ACCESS
-    int (*mem_access)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_PAGING
-    int (*mem_paging)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing)(struct domain *d);
-#endif
-
-    int (*platform_op)(uint32_t cmd);
-
-#ifdef CONFIG_X86
-    int (*do_mca)(void);
-    int (*shadow_control)(struct domain *d, uint32_t op);
-    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
-    int (*apic)(struct domain *d, int cmd);
-    int (*machine_memory_map)(void);
-    int (*domain_memory_map)(struct domain *d);
-#define XSM_MMU_UPDATE_READ      1
-#define XSM_MMU_UPDATE_WRITE     2
-#define XSM_MMU_NORMAL_UPDATE    4
-#define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update)(struct domain *d, struct domain *t,
-                      struct domain *f, uint32_t flags);
-    int (*mmuext_op)(struct domain *d, struct domain *f);
-    int (*update_va_mapping)(struct domain *d, struct domain *f,
-                             l1_pgentry_t pte);
-    int (*priv_mapping)(struct domain *d, struct domain *t);
-    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
-                             uint8_t allow);
-    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
-                          uint8_t allow);
-    int (*pmu_op)(struct domain *d, unsigned int op);
-#endif
-    int (*dm_op)(struct domain *d);
-    int (*xen_version)(uint32_t cmd);
-    int (*domain_resource_map)(struct domain *d);
-#ifdef CONFIG_ARGO
-    int (*argo_enable)(const struct domain *d);
-    int (*argo_register_single_source)(const struct domain *d,
-                                       const struct domain *t);
-    int (*argo_register_any_source)(const struct domain *d);
-    int (*argo_send)(const struct domain *d, const struct domain *t);
-#endif
-};
+#include <xsm/xsm-core.h>
 
 #ifdef CONFIG_XSM
 
 extern struct xsm_ops xsm_ops;
 
-#ifndef XSM_NO_WRAPPERS
-
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
@@ -766,80 +590,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#endif /* XSM_NO_WRAPPERS */
-
-#ifdef CONFIG_MULTIBOOT
-int xsm_multiboot_init(
-	unsigned long *module_map, const multiboot_info_t *mbi);
-int xsm_multiboot_policy_init(
-	unsigned long *module_map, const multiboot_info_t *mbi,
-    void **policy_buffer, size_t *policy_size);
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-/*
- * Initialize XSM
- *
- * On success, return 1 if using SILO mode else 0.
- */
-int xsm_dt_init(void);
-int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-bool has_xsm_magic(paddr_t);
-#endif
-
-extern struct xsm_ops dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_ops *ops);
-
-#ifdef CONFIG_XSM_FLASK
-extern const struct xsm_ops *flask_init(const void *policy_buffer,
-                                        size_t policy_size);
-#else
-static const inline struct xsm_ops *flask_init(const void *policy_buffer,
-                                               size_t policy_size)
-{
-    return NULL;
-}
-#endif
-
-#ifdef CONFIG_XSM_FLASK_POLICY
-extern const unsigned char xsm_flask_init_policy[];
-extern const unsigned int xsm_flask_init_policy_size;
-#endif
-
-#ifdef CONFIG_XSM_SILO
-extern const struct xsm_ops *silo_init(void);
-#else
-static const inline struct xsm_ops *silo_init(void)
-{
-    return NULL;
-}
-#endif
-
 #else /* CONFIG_XSM */
 
 #include <xsm/dummy.h>
 
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init (
-	unsigned long *module_map, const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
 #endif /* CONFIG_XSM */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index d8c935328e..b848580eaa 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,6 @@
  *  as published by the Free Software Foundation.
  */
 
-#define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
 #define set_to_dummy_if_null(ops, function)                            \
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 3550dded7b..66a2cb1505 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -17,7 +17,7 @@
  * You should have received a copy of the GNU General Public License along with
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
-#define XSM_NO_WRAPPERS
+#include <xsm/xsm-core.h>
 #include <xsm/dummy.h>
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index a49fdec741..ec05dac8d1 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -32,6 +32,11 @@
 
 struct xsm_ops __read_mostly xsm_ops;
 
+#ifdef CONFIG_XSM_FLASK_POLICY
+extern const unsigned char xsm_flask_init_policy[];
+extern const unsigned int xsm_flask_init_policy_size;
+#endif
+
 enum xsm_ops_state {
     XSM_OPS_UNREGISTERED,
     XSM_OPS_REG_FAILED,
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:14:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183624.331932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSJ6-00007u-Cl; Thu, 09 Sep 2021 22:14:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183624.331932; Thu, 09 Sep 2021 22:14:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSJ6-00007h-8l; Thu, 09 Sep 2021 22:14:08 +0000
Received: by outflank-mailman (input) for mailman id 183624;
 Thu, 09 Sep 2021 22:14:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSJ5-00067V-3q
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:14:07 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 364f8382-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:13:51 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225559555508.7917953763915;
 Thu, 9 Sep 2021 15:12:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 364f8382-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225564; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=Ozux4rflbbNJTBdB474VORr2195nhATMuQX3wfh8SeC3paUfyK3/lPT0VljJpRqhiW4eSws4O8dkpokkP14rhcxR77eKp7QpslXBpws4KqNiHJrMAvLoK4VsRnRJf/9bonxee0lxlbyXdlxTHKAEc42oXaxRPAr1XCmycLnbKfA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225564; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=b23IIVqB26y4hrFpoL1FhA9zQXWSHZxKb6JCEsWQ8eo=; 
	b=CMvOpDCUy49kKyVLTSN6CPjfRxzEmf/Uq2lvaf3Wmi66jAmNvp2iYVLdDrqGpZHbU7/7goMNyD9CTRxgxOvFcpvaB0gX15ssKg+3mI382W7P/5DJilGn/m0Er/V9hIm8RTZEUQXM4cfIVlbBI7J9LKOOSC5iPkKSmGPJKoCzYDE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225564;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=b23IIVqB26y4hrFpoL1FhA9zQXWSHZxKb6JCEsWQ8eo=;
	b=erpY8zvEHG6n5Ny+hZ43IUk6daEculoQjfJGXLtWOyBHOU6upeF8O+FyiaIMEuLp
	s1iOwmz2Wg5xWhrh/J41K8Y4fFwfoMBZH5IwmqPYAUWGmzBE7Yz5S8YOaPxFnWEV3o8
	uhLWIx2MZJPnWIX5cllXe4UO3mViaJnddvRQDzfE=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 09/10] kconfig: update xsm config to reflect reality
Date: Thu,  9 Sep 2021 22:12:34 -0400
Message-Id: <20210910021236.15082-10-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

It has been a very long time since XSM Flask was the only XSM module, yet the
concenpt of turning XSM on/off continues to be synonymous with enabling and
disabling XSM Flask. Even when XSM Flask was the only module, turning XSM
on/off did not disable or remove the XSM hooks but simply controlled whether
they were implemented as direct inline functions or dispatch calls.

This commit updates XSM kconfig to ensure that it is clear in the code as well
to the user, via the help messages, that the option is about configuring which
XSM policy module(s) are available and which is the default.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/common/Kconfig         | 49 ++++++++++++++++++--------------------
 xen/include/xen/sched.h    |  2 +-
 xen/include/xsm/dummy.h    | 23 +++++++++---------
 xen/include/xsm/xsm-core.h |  6 ++---
 xen/include/xsm/xsm.h      |  6 ++---
 xen/xsm/Makefile           |  4 ++--
 xen/xsm/xsm_core.c         |  4 ++--
 7 files changed, 46 insertions(+), 48 deletions(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index ac5491b1cc..2f85538920 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -200,23 +200,20 @@ config XENOPROF
 
 	  If unsure, say Y.
 
-config XSM
-	bool "Xen Security Modules support"
+config XSM_CONFIGURABLE
+	bool "Configure Xen Security Modules"
 	default ARM
-	---help---
-	  Enables the security framework known as Xen Security Modules which
-	  allows administrators fine-grained control over a Xen domain and
-	  its capabilities by defining permissible interactions between domains,
-	  the hypervisor itself, and related resources such as memory and
-	  devices.
+	help
+	  Allows for configuring the Xen Security Modules (XSM) policy or policies
+	  modules that will be availble and which will be the default.
 
 	  If unsure, say N.
 
 config XSM_FLASK
-	def_bool y
-	prompt "FLux Advanced Security Kernel support"
-	depends on XSM
-	---help---
+	bool "FLux Advanced Security Kernel support"
+	depends on XSM_CONFIGURABLE
+	select XSM_EVTCHN_LABELING
+	help
 	  Enables FLASK (FLux Advanced Security Kernel) as the access control
 	  mechanism used by the XSM framework.  This provides a mandatory access
 	  control framework by which security enforcement, isolation, and
@@ -226,10 +223,10 @@ config XSM_FLASK
 	  If unsure, say Y.
 
 config XSM_FLASK_AVC_STATS
-	def_bool y
-	prompt "Maintain statistics on the FLASK access vector cache" if EXPERT
+	bool "Maintain statistics on the FLASK access vector cache" if EXPERT
+	default y
 	depends on XSM_FLASK
-	---help---
+	help
 	  Maintain counters on the access vector cache that can be viewed using
 	  the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.  Disabling
 	  this will save a tiny amount of memory and time to update the stats.
@@ -240,7 +237,7 @@ config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
 	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
 	depends on XSM_FLASK
-	---help---
+	help
 	  This includes a default XSM policy in the hypervisor so that the
 	  bootloader does not need to load a policy to get sane behavior from an
 	  XSM-enabled hypervisor.  If this is disabled, a policy must be
@@ -253,10 +250,10 @@ config XSM_FLASK_POLICY
 	  If unsure, say Y.
 
 config XSM_SILO
-	def_bool y
-	prompt "SILO support"
-	depends on XSM
-	---help---
+	bool "SILO support"
+	default y if ARM
+	depends on XSM_CONFIGURABLE
+	help
 	  Enables SILO as the access control mechanism used by the XSM framework.
 	  This is not the default module, add boot parameter xsm=silo to choose
 	  it. This will deny any unmediated communication channels (grant tables
@@ -265,14 +262,14 @@ config XSM_SILO
 	  If unsure, say Y.
 
 choice
-	prompt "Default XSM implementation"
-	depends on XSM
+	prompt "Default XSM module"
+	depends on XSM_CONFIGURABLE
 	default XSM_SILO_DEFAULT if XSM_SILO && ARM
 	default XSM_FLASK_DEFAULT if XSM_FLASK
 	default XSM_SILO_DEFAULT if XSM_SILO
 	default XSM_DUMMY_DEFAULT
 	config XSM_DUMMY_DEFAULT
-		bool "Match non-XSM behavior"
+		bool "Classic Dom0 behavior"
 	config XSM_FLASK_DEFAULT
 		bool "FLux Advanced Security Kernel" if XSM_FLASK
 	config XSM_SILO_DEFAULT
@@ -282,15 +279,15 @@ endchoice
 config LATE_HWDOM
 	bool "Dedicated hardware domain"
 	default n
-	depends on XSM && X86
-	---help---
+	depends on XSM_FLASK && X86
+	help
 	  Allows the creation of a dedicated hardware domain distinct from
 	  domain 0 that manages devices without needing access to other
 	  privileged functionality such as the ability to manage domains.
 	  This requires that the actual domain 0 be a stub domain that
 	  constructs the actual hardware domain instead of initializing the
 	  hardware itself.  Because the hardware domain needs access to
-	  hypercalls not available to unprivileged guests, an XSM policy
+	  hypercalls not available to unprivileged guests, an XSM Flask policy
 	  is required to properly define the privilege of these domains.
 
 	  This feature does nothing if the "hardware_dom" boot parameter is
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404..7f4bfa855b 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -120,7 +120,7 @@ struct evtchn
     unsigned short notify_vcpu_id; /* VCPU for local delivery notification */
     uint32_t fifo_lastq;           /* Data for identifying last queue. */
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
     union {
 #ifdef XSM_NEED_GENERIC_EVTCHN_SSID
         /*
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 641b07e8b1..002e0bcc4f 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -43,13 +43,13 @@ static inline void __xsm_action_mismatch_detected(void)
 void __xsm_action_mismatch_detected(void);
 #endif
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 /*
- * In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
- * contains static (not inline) functions compiled to the dummy XSM module.
- * There is no xsm_default_t argument available, so the value from the assertion
- * is used to initialize the variable.
+ * In CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/dummy.c, and contains static (not inline) functions compiled to the
+ * dummy XSM module.  There is no xsm_default_t argument available, so the
+ * value from the assertion is used to initialize the variable.
  */
 #define XSM_INLINE __maybe_unused
 
@@ -57,20 +57,21 @@ void __xsm_action_mismatch_detected(void);
 #define XSM_DEFAULT_VOID void
 #define XSM_ASSERT_ACTION(def) xsm_default_t action = def; (void)action
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 /*
- * In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
- * contains inline functions for each XSM hook. These functions also perform
- * compile-time checks on the xsm_default_t argument to ensure that the behavior
- * of the dummy XSM module is the same as the behavior with XSM disabled.
+ * In !CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/xsm.h, and contains inline functions for each XSM hook. These functions
+ * also perform compile-time checks on the xsm_default_t argument to ensure
+ * that the behavior of the dummy XSM module is the same as the behavior with
+ * XSM disabled.
  */
 #define XSM_INLINE always_inline
 #define XSM_DEFAULT_ARG xsm_default_t action,
 #define XSM_DEFAULT_VOID xsm_default_t action
 #define XSM_ASSERT_ACTION(def) LINKER_BUG_ON(def != action)
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 static always_inline int xsm_default_action(
 	xsm_default_t action, struct domain *src, struct domain *target)
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index 514c198ee6..1bf41bfd0d 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -199,7 +199,7 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(
@@ -240,7 +240,7 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #ifdef CONFIG_MULTIBOOT
 static inline int xsm_multiboot_init(
@@ -262,6 +262,6 @@ static inline bool has_xsm_magic(paddr_t start)
 }
 #endif /* CONFIG_HAS_DEVICE_TREE */
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 47337ecbd3..1ae11eeaa1 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,7 +20,7 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 extern struct xsm_ops xsm_ops;
 
@@ -590,10 +590,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #include <xsm/dummy.h>
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index cf0a728f1c..09b9311b1d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
-obj-$(CONFIG_XSM) += xsm_policy.o
-obj-$(CONFIG_XSM) += dummy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index ec05dac8d1..446fff6be3 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,7 +18,7 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
@@ -222,7 +222,7 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:21:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:21:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183667.331944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSPt-0002OE-6F; Thu, 09 Sep 2021 22:21:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183667.331944; Thu, 09 Sep 2021 22:21:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSPt-0002O7-2b; Thu, 09 Sep 2021 22:21:09 +0000
Received: by outflank-mailman (input) for mailman id 183667;
 Thu, 09 Sep 2021 22:21:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mNcr=N7=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOSJF-00067V-4M
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:14:17 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3dd432a6-11bb-11ec-b1fd-12813bfff9fa;
 Thu, 09 Sep 2021 22:14:04 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631225561654476.0107130223921;
 Thu, 9 Sep 2021 15:12:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dd432a6-11bb-11ec-b1fd-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631225565; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=eF8L9xNUKxrmrq5u+QlxNi6jTGykDuQmZ/x8eHbUcHkph+1XuhsWcmekx3ZKmh8uksbkq/otrD3fMEgiylkTyKGGefq2wn/c4FertR3boO4VxF9CkhVblbfh2jHiiObVo+750py9EeKtA/v8N4K5uvSf3/GQP8vdCwOPyFImwJ8=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631225565; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=gxeXB5svWpcttsgrZvh81hoLL/bHHrd07F+Dm2+Gyo8=; 
	b=BB94jkwXK8mvsfNwt/K5ossWsS7SRLhd7iIcTtP5r7YIxtZsPAMCyW96S+GmEd0jlWI2uqG3ZRitm+mcR8jlGEKx9A1bJjWofHF3Xg1gZTiJmabytjupyZ554bwo0m+2U94efgGABfr5zm91PzYIJpVIXx+3iNMxMZGguARf8b0=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631225565;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=gxeXB5svWpcttsgrZvh81hoLL/bHHrd07F+Dm2+Gyo8=;
	b=unT1Q80oLMA33TgfThS+s4fmLuI0AEEhLrrUGniTGW4Dkn3VGBQXiV/Km2jMGcNS
	WQqtSHMINp2sIi5wNbmAAZ4d+1qG9oOoW1rsr0PlC9FnSLqTKljeOQFJ35legTJK09W
	Dz3KHh9ZB/E6Urc0SJxdPZLVoS4WZfqJfHPcvP5c=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v5 10/10] xsm: remove alternate xsm hook interface
Date: Thu,  9 Sep 2021 22:12:35 -0400
Message-Id: <20210910021236.15082-11-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910021236.15082-1-dpsmith@apertussolutions.com>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Hidden behind macro magic is an alternative xsm hook interface dedicated for
use when the dummy/default policy is the only one built. This alternative
interface increases code complexity and code size in the core security
framework of Xen.  This results in code requiring additional maintanence and
additional risk for securit-relevant bugs.

This patch removes this additional interface, making Xen's security framework
have a single, consistent interface that works in a single and consistent
manner regardless of which XSM policy is in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h    | 819 -------------------------------------
 xen/include/xsm/xsm-core.h |  51 ++-
 xen/include/xsm/xsm.h      | 275 ++++++++-----
 xen/xsm/Makefile           |   2 +-
 xen/xsm/dummy.c            |   2 +-
 xen/xsm/dummy.h            | 659 +++++++++++++++++++++++++++++
 xen/xsm/silo.c             |   2 +-
 xen/xsm/xsm_core.c         |   4 -
 8 files changed, 868 insertions(+), 946 deletions(-)
 delete mode 100644 xen/include/xsm/dummy.h
 create mode 100644 xen/xsm/dummy.h

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
deleted file mode 100644
index 002e0bcc4f..0000000000
--- a/xen/include/xsm/dummy.h
+++ /dev/null
@@ -1,819 +0,0 @@
-/*
- *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
- *
- *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License version 2,
- *  as published by the Free Software Foundation.
- *
- *
- *  Each XSM hook implementing an access check should have its first parameter
- *  preceded by XSM_DEFAULT_ARG (or use XSM_DEFAULT_VOID if it has no
- *  arguments). The first non-declaration statement shold be XSM_ASSERT_ACTION
- *  with the expected type of the hook, which will either define or check the
- *  value of action.
- */
-
-#include <xen/sched.h>
-#include <xsm/xsm-core.h>
-#include <public/hvm/params.h>
-
-/*
- * Cannot use BUILD_BUG_ON here because the expressions we check are not
- * considered constant at compile time. Instead, rely on constant propagation to
- * inline out the calls to this invalid function, which will cause linker errors
- * if references remain at link time.
- */
-#define LINKER_BUG_ON(x) do { if (x) __xsm_action_mismatch_detected(); } while (0)
-
-#if defined(CONFIG_COVERAGE) && defined(__clang__)
-/*
- * LLVM coverage support seems to disable some of the optimizations needed in
- * order for XSM to compile. Since coverage should not be used in production
- * provide an implementation of __xsm_action_mismatch_detected to satisfy the
- * linker.
- */
-static inline void __xsm_action_mismatch_detected(void)
-{
-    ASSERT_UNREACHABLE();
-}
-#else
-/* DO NOT implement this function; it is supposed to trigger link errors */
-void __xsm_action_mismatch_detected(void);
-#endif
-
-#ifdef CONFIG_XSM_CONFIGURABLE
-
-/*
- * In CONFIG_XSM_CONFIGURABLE builds, this header file is included from
- * xsm/dummy.c, and contains static (not inline) functions compiled to the
- * dummy XSM module.  There is no xsm_default_t argument available, so the
- * value from the assertion is used to initialize the variable.
- */
-#define XSM_INLINE __maybe_unused
-
-#define XSM_DEFAULT_ARG /* */
-#define XSM_DEFAULT_VOID void
-#define XSM_ASSERT_ACTION(def) xsm_default_t action = def; (void)action
-
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-/*
- * In !CONFIG_XSM_CONFIGURABLE builds, this header file is included from
- * xsm/xsm.h, and contains inline functions for each XSM hook. These functions
- * also perform compile-time checks on the xsm_default_t argument to ensure
- * that the behavior of the dummy XSM module is the same as the behavior with
- * XSM disabled.
- */
-#define XSM_INLINE always_inline
-#define XSM_DEFAULT_ARG xsm_default_t action,
-#define XSM_DEFAULT_VOID xsm_default_t action
-#define XSM_ASSERT_ACTION(def) LINKER_BUG_ON(def != action)
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
-static always_inline int xsm_default_action(
-	xsm_default_t action, struct domain *src, struct domain *target)
-{
-    switch ( action ) {
-    case XSM_HOOK:
-        return 0;
-    case XSM_TARGET:
-        if ( evaluate_nospec(src == target) )
-        {
-            return 0;
-    case XSM_XS_PRIV:
-            if ( evaluate_nospec(is_xenstore_domain(src)) )
-                return 0;
-        }
-        /* fall through */
-    case XSM_DM_PRIV:
-        if ( target && evaluate_nospec(src->target == target) )
-            return 0;
-        /* fall through */
-    case XSM_PRIV:
-        if ( is_control_domain(src) )
-            return 0;
-        return -EPERM;
-    default:
-        LINKER_BUG_ON(1);
-        return -EPERM;
-    }
-}
-
-static XSM_INLINE void xsm_security_domaininfo(
-	struct domain *d, struct xen_domctl_getdomaininfo *info)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_domain_create(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t ssidref)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_domctl_scheduler_op(
-	XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_set_target(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *e)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_domctl(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( cmd )
-    {
-    case XEN_DOMCTL_ioport_mapping:
-    case XEN_DOMCTL_memory_mapping:
-    case XEN_DOMCTL_bind_pt_irq:
-    case XEN_DOMCTL_unbind_pt_irq:
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    case XEN_DOMCTL_getdomaininfo:
-        return xsm_default_action(XSM_XS_PRIV, current->domain, d);
-    default:
-        return xsm_default_action(XSM_PRIV, current->domain, d);
-    }
-}
-
-static XSM_INLINE int xsm_sysctl(XSM_DEFAULT_ARG int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG uint32_t clear)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_alloc_security_domain(struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE void xsm_free_security_domain(struct domain *d)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_grant_mapref(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2, uint32_t flags)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_unmapref(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_setup(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_transfer(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_copy(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_grant_query_size(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_memory_adjust_reservation(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_stat_reservation(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_console_io(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    if ( d->is_console )
-        return xsm_default_action(XSM_HOOK, d, NULL);
-#ifdef CONFIG_VERBOSE_DEBUG
-    if ( cmd == CONSOLEIO_write )
-        return xsm_default_action(XSM_HOOK, d, NULL);
-#endif
-    return xsm_default_action(XSM_PRIV, d, NULL);
-}
-
-static XSM_INLINE int xsm_profile(XSM_DEFAULT_ARG struct domain *d, int op)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_schedop_shutdown(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_memory_pin_page(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-    struct page_info *page)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_unbound(
-	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn, domid_t id2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_interdomain(
-	XSM_DEFAULT_ARG struct domain *d1, struct evtchn *chan1, struct domain *d2,
-    struct evtchn *chan2)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
-{
-    return;
-}
-
-static XSM_INLINE int xsm_evtchn_send(
-	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_evtchn_status(
-	XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_evtchn_reset(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_alloc_security_evtchns(
-	struct evtchn chn[], unsigned int nr)
-{
-    return 0;
-}
-
-static XSM_INLINE void xsm_free_security_evtchns(
-	struct evtchn chn[], unsigned int nr)
-{
-    return;
-}
-
-static XSM_INLINE char *xsm_show_security_evtchn(
-	struct domain *d, const struct evtchn *chn)
-{
-    return NULL;
-}
-
-static XSM_INLINE int xsm_init_hardware_domain(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_get_pod_target(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_set_pod_target(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_get_vnumainfo(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_assign_device(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_deassign_device(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#endif /* HAS_PASSTHROUGH && HAS_PCI */
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static XSM_INLINE int xsm_assign_dtdevice(
-	XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_deassign_dtdevice(
-	XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
-
-static XSM_INLINE int xsm_resource_plug_core(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_plug_pci(
-	XSM_DEFAULT_ARG uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_unplug_pci(
-	XSM_DEFAULT_ARG uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_pci(
-	XSM_DEFAULT_ARG uint32_t machine_bdf)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_gsi(XSM_DEFAULT_ARG int gsi)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_resource_setup_misc(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_page_offline(XSM_DEFAULT_ARG uint32_t cmd)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
-{
-    return -ENOSYS;
-}
-
-#ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
-{
-    return -ENOSYS;
-}
-#endif
-
-static XSM_INLINE char *xsm_show_irq_sid(int irq)
-{
-    return NULL;
-}
-
-static XSM_INLINE int xsm_map_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_map_domain_irq(
-	XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_bind_pt_irq(
-	XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unbind_pt_irq(
-	XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_unmap_domain_irq(
-	XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_irq_permission(
-	XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_iomem_permission(
-	XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_iomem_mapping(
-	XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_pci_config_permission(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, uint16_t start,
-    uint16_t end, uint8_t access)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_add_to_physmap(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_remove_from_physmap(
-	XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d1, d2);
-}
-
-static XSM_INLINE int xsm_map_gmfn_foreign(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *t)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, t);
-}
-
-static XSM_INLINE int xsm_hvm_param(
-	XSM_DEFAULT_ARG struct domain *d, unsigned long op)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_control(
-	XSM_DEFAULT_ARG struct domain *d, unsigned long op)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(
-	XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-
-    switch ( mode )
-    {
-    case XEN_ALTP2M_mixed:
-        return xsm_default_action(XSM_TARGET, current->domain, d);
-    case XEN_ALTP2M_external:
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    case XEN_ALTP2M_limited:
-        if ( HVMOP_altp2m_vcpu_enable_notify == op )
-            return xsm_default_action(XSM_TARGET, current->domain, d);
-        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
-    default:
-        return -EPERM;
-    }
-}
-
-static XSM_INLINE int xsm_vm_event_control(
-	XSM_DEFAULT_ARG struct domain *d, int mode, int op)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#ifdef CONFIG_MEM_ACCESS
-static XSM_INLINE int xsm_mem_access(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-#ifdef CONFIG_MEM_PAGING
-static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-#ifdef CONFIG_MEM_SHARING
-static XSM_INLINE int xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-#endif
-
-static XSM_INLINE int xsm_platform_op(XSM_DEFAULT_ARG uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-#ifdef CONFIG_X86
-static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_shadow_control(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_mem_sharing_op(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, cd);
-}
-
-static XSM_INLINE int xsm_apic(XSM_DEFAULT_ARG struct domain *d, int cmd)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, d, NULL);
-}
-
-static XSM_INLINE int xsm_machine_memory_map(XSM_DEFAULT_VOID)
-{
-    XSM_ASSERT_ACTION(XSM_PRIV);
-    return xsm_default_action(action, current->domain, NULL);
-}
-
-static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_mmu_update(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *t, struct domain *f,
-    uint32_t flags)
-{
-    int rc = 0;
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    if ( f != dom_io )
-        rc = xsm_default_action(action, d, f);
-    if ( evaluate_nospec(t) && !rc )
-        rc = xsm_default_action(action, d, t);
-    return rc;
-}
-
-static XSM_INLINE int xsm_mmuext_op(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *f)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, f);
-}
-
-static XSM_INLINE int xsm_update_va_mapping(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *f, l1_pgentry_t pte)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, f);
-}
-
-static XSM_INLINE int xsm_priv_mapping(
-	XSM_DEFAULT_ARG struct domain *d, struct domain *t)
-{
-    XSM_ASSERT_ACTION(XSM_TARGET);
-    return xsm_default_action(action, d, t);
-}
-
-static XSM_INLINE int xsm_ioport_permission(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_ioport_mapping(
-	XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
-{
-    XSM_ASSERT_ACTION(XSM_HOOK);
-    return xsm_default_action(action, current->domain, d);
-}
-
-static XSM_INLINE int xsm_pmu_op(
-	XSM_DEFAULT_ARG struct domain *d, unsigned int op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( op )
-    {
-    case XENPMU_init:
-    case XENPMU_finish:
-    case XENPMU_lvtpc_set:
-    case XENPMU_flush:
-        return xsm_default_action(XSM_HOOK, d, current->domain);
-    default:
-        return xsm_default_action(XSM_PRIV, d, current->domain);
-    }
-}
-
-#endif /* CONFIG_X86 */
-
-static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
-
-#ifdef CONFIG_ARGO
-static XSM_INLINE int xsm_argo_enable(const struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_register_single_source(
-	const struct domain *d, const struct domain *t)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_register_any_source(const struct domain *d)
-{
-    return 0;
-}
-
-static XSM_INLINE int xsm_argo_send(
-	const struct domain *d, const struct domain *t)
-{
-    return 0;
-}
-
-#endif /* CONFIG_ARGO */
-
-#include <public/version.h>
-static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
-{
-    XSM_ASSERT_ACTION(XSM_OTHER);
-    switch ( op )
-    {
-    case XENVER_version:
-    case XENVER_platform_parameters:
-    case XENVER_get_features:
-        /* These sub-ops ignore the permission checks and return data. */
-        block_speculation();
-        return 0;
-    case XENVER_extraversion:
-    case XENVER_compile_info:
-    case XENVER_capabilities:
-    case XENVER_changeset:
-    case XENVER_pagesize:
-    case XENVER_guest_handle:
-        /* These MUST always be accessible to any guest by default. */
-        return xsm_default_action(XSM_HOOK, current->domain, NULL);
-    default:
-        return xsm_default_action(XSM_PRIV, current->domain, NULL);
-    }
-}
-
-static XSM_INLINE int xsm_domain_resource_map(XSM_DEFAULT_ARG struct domain *d)
-{
-    XSM_ASSERT_ACTION(XSM_DM_PRIV);
-    return xsm_default_action(action, current->domain, d);
-}
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index 1bf41bfd0d..45d92314fe 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -18,6 +18,31 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
+/* Cannot use BUILD_BUG_ON here because the expressions we check are not
+ * considered constant at compile time. Instead, rely on constant propagation to
+ * inline out the calls to this invalid function, which will cause linker errors
+ * if references remain at link time.
+ */
+#define XSM_LINKER_BUG_ON(x) do { if (x) __xsm_action_mismatch_detected(); } while (0)
+
+#if defined(CONFIG_COVERAGE) && defined(__clang__)
+/*
+ * LLVM coverage support seems to disable some of the optimizations needed in
+ * order for XSM to compile. Since coverage should not be used in production
+ * provide an implementation of __xsm_action_mismatch_detected to satisfy the
+ * linker.
+ */
+static inline void __xsm_action_mismatch_detected(void)
+{
+    ASSERT_UNREACHABLE();
+}
+#else
+/* DO NOT implement this function; it is supposed to trigger link errors */
+void __xsm_action_mismatch_detected(void);
+#endif
+
+#define XSM_ASSERT_ACTION(def) XSM_LINKER_BUG_ON(def != action)
+
 /* policy magic number */
 typedef uint32_t xsm_magic_t;
 
@@ -199,8 +224,6 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(
 	unsigned long *module_map, const multiboot_info_t *mbi);
@@ -240,28 +263,4 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init(
-	unsigned long *module_map, const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 1ae11eeaa1..c74e5866f4 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,8 +20,6 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 extern struct xsm_ops xsm_ops;
 
 static inline void xsm_security_domaininfo(
@@ -31,58 +29,68 @@ static inline void xsm_security_domaininfo(
 }
 
 static inline int xsm_domain_create(
-    xsm_default_t def, struct domain *d, uint32_t ssidref)
+    xsm_default_t action, struct domain *d, uint32_t ssidref)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
-static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
 static inline int xsm_domctl_scheduler_op(
-    xsm_default_t def, struct domain *d, int cmd)
+    xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
 static inline int xsm_set_target(
-    xsm_default_t def, struct domain *d, struct domain *e)
+    xsm_default_t action, struct domain *d, struct domain *e)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.set_target, d, e);
 }
 
-static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
-static inline int xsm_sysctl(xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.sysctl, cmd);
 }
 
-static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t action, uint32_t clear)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.readconsole, clear);
 }
 
 static inline int xsm_evtchn_unbound(
-    xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
+    xsm_default_t action, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
-    xsm_default_t def, struct domain *d1, struct evtchn *chan1,
+    xsm_default_t action, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
@@ -92,56 +100,65 @@ static inline void xsm_evtchn_close_post(struct evtchn *chn)
 }
 
 static inline int xsm_evtchn_send(
-    xsm_default_t def, struct domain *d, struct evtchn *chn)
+    xsm_default_t action, struct domain *d, struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
 static inline int xsm_evtchn_status(
-    xsm_default_t def, struct domain *d, struct evtchn *chn)
+    xsm_default_t action, struct domain *d, struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
 static inline int xsm_evtchn_reset(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
 static inline int xsm_grant_mapref(
-    xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
+    xsm_default_t action, struct domain *d1, struct domain *d2, uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
 static inline int xsm_grant_setup(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
 static inline int xsm_grant_transfer(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
 static inline int xsm_grant_copy(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
 static inline int xsm_grant_query_size(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
@@ -173,86 +190,103 @@ static inline char *xsm_show_security_evtchn(
     return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
-static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t action,
+                                           struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
-static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.get_pod_target, d);
 }
 
-static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.set_pod_target, d);
 }
 
-static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_exchange, d);
 }
 
 static inline int xsm_memory_adjust_reservation(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
-	xsm_default_t def, struct domain *d1, struct domain *d2,
+	xsm_default_t action, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
-	xsm_default_t def, struct domain *d, struct domain *t)
+	xsm_default_t action, struct domain *d, struct domain *t)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
-static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
+static inline int xsm_claim_pages(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.claim_pages, d);
 }
 
-static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t action, struct domain *d,
+                                 int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
-static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t action, struct domain *d, int op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.profile, d, op);
 }
 
-static inline int xsm_kexec(xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.kexec);
 }
 
 static inline int xsm_schedop_shutdown(
-	xsm_default_t def, struct domain *d1, struct domain *d2)
+	xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
@@ -261,143 +295,172 @@ static inline char *xsm_show_irq_sid(int irq)
     return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
-static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
 static inline int xsm_map_domain_irq(
-	xsm_default_t def, struct domain *d, int irq, void *data)
+	xsm_default_t action, struct domain *d, int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
 static inline int xsm_unmap_domain_irq(
-	xsm_default_t def, struct domain *d, int irq, void *data)
+	xsm_default_t action, struct domain *d, int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
-	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+	xsm_default_t action, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
-	xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+	xsm_default_t action, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
 static inline int xsm_irq_permission(
-	xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
+	xsm_default_t action, struct domain *d, int pirq, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
-	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+	xsm_default_t action, struct domain *d, uint64_t s, uint64_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
-	xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+	xsm_default_t action, struct domain *d, uint64_t s, uint64_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
-	xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
-    uint16_t end, uint8_t access)
+	xsm_default_t action, struct domain *d, uint32_t machine_bdf,
+    uint16_t start, uint16_t end, uint8_t access)
 {
-    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
+    XSM_ASSERT_ACTION(XSM_HOOK);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf,
+                            start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_get_device_group(xsm_default_t action,
+                                       uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
 static inline int xsm_assign_device(
-	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+	xsm_default_t action, struct domain *d, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
-	xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+	xsm_default_t action, struct domain *d, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
 static inline int xsm_assign_dtdevice(
-	xsm_default_t def, struct domain *d, const char *dtpath)
+	xsm_default_t action, struct domain *d, const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
-	xsm_default_t def, struct domain *d, const char *dtpath)
+	xsm_default_t action, struct domain *d, const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t action,
+                                        uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
-	xsm_default_t def, uint32_t machine_bdf)
+	xsm_default_t action, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
-static inline int xsm_resource_plug_core(xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_plug_core);
 }
 
-static inline int xsm_resource_unplug_core(xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_unplug_core);
 }
 
 static inline int xsm_resource_setup_pci(
-	xsm_default_t def, uint32_t machine_bdf)
+	xsm_default_t action, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t action, int gsi)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
-static inline int xsm_resource_setup_misc(xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_misc);
 }
 
-static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
+static inline int xsm_page_offline(xsm_default_t action, uint32_t cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.page_offline, cmd);
 }
 
-static inline int xsm_hypfs_op(xsm_default_t def)
+static inline int xsm_hypfs_op(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hypfs_op);
 }
 
@@ -414,155 +477,185 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 #endif
 
 static inline int xsm_hvm_param(
-	xsm_default_t def, struct domain *d, unsigned long op)
+	xsm_default_t action, struct domain *d, unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
 static inline int xsm_hvm_control(
-	xsm_default_t def, struct domain *d, unsigned long op)
+	xsm_default_t action, struct domain *d, unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
-	xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
+	xsm_default_t action, struct domain *d, uint64_t mode, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
 static inline int xsm_vm_event_control(
-	xsm_default_t def, struct domain *d, int mode, int op)
+	xsm_default_t action, struct domain *d, int mode, int op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
-static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
-static inline int xsm_do_mca(xsm_default_t def)
+static inline int xsm_do_mca(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.do_mca);
 }
 
 static inline int xsm_shadow_control(
-	xsm_default_t def, struct domain *d, uint32_t op)
+	xsm_default_t action, struct domain *d, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
 static inline int xsm_mem_sharing_op(
-	xsm_default_t def, struct domain *d, struct domain *cd, int op)
+	xsm_default_t action, struct domain *d, struct domain *cd, int op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
-static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.apic, d, cmd);
 }
 
-static inline int xsm_machine_memory_map(xsm_default_t def)
+static inline int xsm_machine_memory_map(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.machine_memory_map);
 }
 
-static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_memory_map(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
 static inline int xsm_mmu_update(
-	xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
+	xsm_default_t action, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
-	xsm_default_t def, struct domain *d, struct domain *f)
+	xsm_default_t action, struct domain *d, struct domain *f)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
 static inline int xsm_update_va_mapping(
-	xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
+	xsm_default_t action, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_ops.update_va_mapping(d, f, pte);
 }
 
 static inline int xsm_priv_mapping(
-	xsm_default_t def, struct domain *d, struct domain *t)
+	xsm_default_t action, struct domain *d, struct domain *t)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
 static inline int xsm_ioport_permission(
-	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+	xsm_default_t action, struct domain *d, uint32_t s, uint32_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
-	xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+	xsm_default_t action, struct domain *d, uint32_t s, uint32_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
-	xsm_default_t def, struct domain *d, unsigned int op)
+	xsm_default_t action, struct domain *d, unsigned int op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
-static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
+static inline int xsm_dm_op(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.dm_op, d);
 }
 
-static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.xen_version, op);
 }
 
-static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_resource_map(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
@@ -590,10 +683,4 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#include <xsm/dummy.h>
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index 09b9311b1d..121a0eda7d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
+obj-y += dummy.o
 obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
-obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index b848580eaa..e18afe0673 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,7 @@
  *  as published by the Free Software Foundation.
  */
 
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
diff --git a/xen/xsm/dummy.h b/xen/xsm/dummy.h
new file mode 100644
index 0000000000..b716bf7371
--- /dev/null
+++ b/xen/xsm/dummy.h
@@ -0,0 +1,659 @@
+/*
+ *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
+ *
+ *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ *
+ *
+ *  Dummy XSM hooks implementing the default access check. Each hook should
+ *  have as its first line XSM_DEFAULT_ACTION declaring the privilege level
+ *  required for this access.
+ */
+
+#ifndef __XSM_DUMMY_H__
+#define __XSM_DUMMY_H__
+
+#include <xen/sched.h>
+#include <xsm/xsm-core.h>
+#include <public/hvm/params.h>
+
+#define XSM_DEFAULT_ACTION(def) xsm_default_t action = def; (void)action
+
+static always_inline int xsm_default_action(
+    xsm_default_t action, struct domain *src, struct domain *target)
+{
+    switch ( action )
+    {
+    case XSM_HOOK:
+        return 0;
+    case XSM_TARGET:
+        if ( evaluate_nospec(src == target) )
+        {
+            return 0;
+    case XSM_XS_PRIV:
+            if ( evaluate_nospec(is_xenstore_domain(src)) )
+                return 0;
+        }
+        /* fall through */
+    case XSM_DM_PRIV:
+        if ( target && evaluate_nospec(src->target == target) )
+            return 0;
+        /* fall through */
+    case XSM_PRIV:
+        if ( is_control_domain(src) )
+            return 0;
+        return -EPERM;
+    default:
+        XSM_LINKER_BUG_ON(1);
+        return -EPERM;
+    }
+}
+
+static inline void xsm_security_domaininfo(
+	struct domain *d, struct xen_domctl_getdomaininfo *info)
+{
+    return;
+}
+
+static inline int xsm_domain_create(struct domain *d, uint32_t ssidref)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_getdomaininfo(struct domain *d)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_domctl_scheduler_op(struct domain *d, int cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_sysctl_scheduler_op(int cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_set_target(struct domain *d, struct domain *e)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_domctl(struct domain *d, int cmd)
+{
+    switch ( cmd )
+    {
+    case XEN_DOMCTL_ioport_mapping:
+    case XEN_DOMCTL_memory_mapping:
+    case XEN_DOMCTL_bind_pt_irq:
+    case XEN_DOMCTL_unbind_pt_irq:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_DOMCTL_getdomaininfo:
+        return xsm_default_action(XSM_XS_PRIV, current->domain, d);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, d);
+    }
+}
+
+static inline int xsm_sysctl(int cmd)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_readconsole(uint32_t clear)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_alloc_security_domain(struct domain *d)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_domain(struct domain *d)
+{
+    return;
+}
+
+static inline int xsm_grant_mapref(
+	struct domain *d1, struct domain *d2, uint32_t flags)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_unmapref(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_setup(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_grant_transfer(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_copy(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_query_size(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_memory_exchange(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_memory_adjust_reservation(
+	struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_memory_stat_reservation(
+	struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_console_io(struct domain *d, int cmd)
+{
+    if ( d->is_console )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#ifdef CONFIG_VERBOSE_DEBUG
+    if ( cmd == CONSOLEIO_write )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#endif
+    return xsm_default_action(XSM_PRIV, d, NULL);
+}
+
+static inline int xsm_profile(struct domain *d, int op)
+{
+    return xsm_default_action(XSM_HOOK, d, NULL);
+}
+
+static inline int xsm_kexec(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_schedop_shutdown(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_DM_PRIV, d1, d2);
+}
+
+static inline int xsm_memory_pin_page(
+	struct domain *d1, struct domain *d2, struct page_info *page)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_claim_pages(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_evtchn_unbound(
+	struct domain *d, struct evtchn *chn, domid_t id2)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_evtchn_interdomain(
+	struct domain *d1, struct evtchn *chan1, struct domain *d2,
+    struct evtchn *chan2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
+{
+    return;
+}
+
+static inline int xsm_evtchn_send(struct domain *d, struct evtchn *chn)
+{
+    return xsm_default_action(XSM_HOOK, d, NULL);
+}
+
+static inline int xsm_evtchn_status(struct domain *d, struct evtchn *chn)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_evtchn_reset(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_alloc_security_evtchns(
+	struct evtchn chn[], unsigned int nr)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_evtchns(
+	struct evtchn chn[], unsigned int nr)
+{
+    return;
+}
+
+static inline char *xsm_show_security_evtchn(
+	struct domain *d, const struct evtchn *chn)
+{
+    return NULL;
+}
+
+static inline int xsm_init_hardware_domain(struct domain *d)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_get_pod_target(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_set_pod_target(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_get_vnumainfo(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+static inline int xsm_get_device_group(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_assign_device(struct domain *d, uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_deassign_device(struct domain *d, uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_PCI */
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+static inline int xsm_assign_dtdevice(struct domain *d, const char *dtpath)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_deassign_dtdevice(struct domain *d, const char *dtpath)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
+
+static inline int xsm_resource_plug_core(void)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_core(void)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_resource_plug_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_gsi(int gsi)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_misc(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_page_offline(uint32_t cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_hypfs_op(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+
+#ifdef CONFIG_COMPAT
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+#endif
+
+static inline char *xsm_show_irq_sid(int irq)
+{
+    return NULL;
+}
+
+static inline int xsm_map_domain_pirq(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_map_domain_irq(
+	struct domain *d, int irq, const void *data)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_pirq(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_bind_pt_irq(
+	struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unbind_pt_irq(
+	struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_irq(
+	struct domain *d, int irq, const void *data)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_irq_permission(struct domain *d, int pirq, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_iomem_permission(
+	struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_iomem_mapping(
+	struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_pci_config_permission(
+	struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end,
+    uint8_t access)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_map_gmfn_foreign(struct domain *d, struct domain *t)
+{
+    return xsm_default_action(XSM_TARGET, d, t);
+}
+
+static inline int xsm_hvm_param(struct domain *d, unsigned long op)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_hvm_control(struct domain *d, unsigned long op)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_hvm_param_altp2mhvm(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_hvm_altp2mhvm_op(
+	struct domain *d, uint64_t mode, uint32_t op)
+{
+
+    switch ( mode )
+    {
+    case XEN_ALTP2M_mixed:
+        return xsm_default_action(XSM_TARGET, current->domain, d);
+    case XEN_ALTP2M_external:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_ALTP2M_limited:
+        if ( HVMOP_altp2m_vcpu_enable_notify == op )
+            return xsm_default_action(XSM_TARGET, current->domain, d);
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    default:
+        return -EPERM;
+    }
+}
+
+static inline int xsm_vm_event_control(struct domain *d, int mode, int op)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+#ifdef CONFIG_MEM_ACCESS
+static inline int xsm_mem_access(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+static inline int xsm_mem_paging(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+static inline int xsm_mem_sharing(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+static inline int xsm_platform_op(uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+#ifdef CONFIG_X86
+static inline int xsm_do_mca(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_shadow_control(struct domain *d, uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_mem_sharing_op(
+	struct domain *d, struct domain *cd, int op)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, cd);
+}
+
+static inline int xsm_apic(struct domain *d, int cmd)
+{
+    return xsm_default_action(XSM_PRIV, d, NULL);
+}
+
+static inline int xsm_machine_memory_map(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_domain_memory_map(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_mmu_update(
+	struct domain *d, struct domain *t, struct domain *f, uint32_t flags)
+{
+    int rc = 0;
+    if ( f != dom_io )
+        rc = xsm_default_action(XSM_TARGET, d, f);
+    if ( evaluate_nospec(t) && !rc )
+        rc = xsm_default_action(XSM_TARGET, d, t);
+    return rc;
+}
+
+static inline int xsm_mmuext_op(struct domain *d, struct domain *f)
+{
+    return xsm_default_action(XSM_TARGET, d, f);
+}
+
+static inline int xsm_update_va_mapping(
+	struct domain *d, struct domain *f, l1_pgentry_t pte)
+{
+    return xsm_default_action(XSM_TARGET, d, f);
+}
+
+static inline int xsm_priv_mapping(struct domain *d, struct domain *t)
+{
+    return xsm_default_action(XSM_TARGET, d, t);
+}
+
+static inline int xsm_ioport_permission(
+	struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_ioport_mapping(
+	struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_pmu_op(struct domain *d, unsigned int op)
+{
+    switch ( op )
+    {
+    case XENPMU_init:
+    case XENPMU_finish:
+    case XENPMU_lvtpc_set:
+    case XENPMU_flush:
+        return xsm_default_action(XSM_HOOK, d, current->domain);
+    default:
+        return xsm_default_action(XSM_PRIV, d, current->domain);
+    }
+}
+
+#endif /* CONFIG_X86 */
+
+static inline int xsm_dm_op(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+#ifdef CONFIG_ARGO
+static inline int xsm_argo_enable(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_single_source(
+	const struct domain *d, const struct domain *t)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_any_source(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
+{
+    return 0;
+}
+
+#endif /* CONFIG_ARGO */
+
+#include <public/version.h>
+static inline int xsm_xen_version(uint32_t op)
+{
+    switch ( op )
+    {
+    case XENVER_version:
+    case XENVER_platform_parameters:
+    case XENVER_get_features:
+        /* These sub-ops ignore the permission checks and return data. */
+        block_speculation();
+        return 0;
+    case XENVER_extraversion:
+    case XENVER_compile_info:
+    case XENVER_capabilities:
+    case XENVER_changeset:
+    case XENVER_pagesize:
+    case XENVER_guest_handle:
+        /* These MUST always be accessible to any guest by default. */
+        return xsm_default_action(XSM_HOOK, current->domain, NULL);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, NULL);
+    }
+}
+
+static inline int xsm_domain_resource_map(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+#endif /* __XSM_DUMMY_H__ */
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 66a2cb1505..350612971f 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -18,7 +18,7 @@
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
 #include <xsm/xsm-core.h>
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 /*
  * Check if inter-domain communication is allowed.
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 446fff6be3..72b9dd1dac 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,8 +18,6 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
 #endif
@@ -222,8 +220,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 22:54:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 22:54:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183683.331961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSwA-0006mJ-Vv; Thu, 09 Sep 2021 22:54:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183683.331961; Thu, 09 Sep 2021 22:54:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOSwA-0006mC-Si; Thu, 09 Sep 2021 22:54:30 +0000
Received: by outflank-mailman (input) for mailman id 183683;
 Thu, 09 Sep 2021 22:54:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOSw9-0006m6-K3
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 22:54:29 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 86979d71-c850-45bf-9d2d-ad83fc7b2055;
 Thu, 09 Sep 2021 22:54:28 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1ECFA61074;
 Thu,  9 Sep 2021 22:54:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86979d71-c850-45bf-9d2d-ad83fc7b2055
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631228067;
	bh=KqKonysZXzQ1LTEy+oVeLNDH9vOD4qd2Sqb5YMgnwxM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HxldIXbqsCnU2XPEmNb/LdvFZQezlI0abfIY0theLNKBTME/BBCxywshBf0h53lwt
	 X2w8UjrfLgsldzFn6lk5I4adn0VDjJUu/1TwOM+rY6Rl8OVf8puy43Ojna9u41fFEi
	 EJIpcRl6QLp+kKIyMWorAJR1DCc6zv5QkOKsZejplLSwPQd5oKxzD24isfuGSIVF2Z
	 r5AIVquX4RX87eVdP1Afv1RDsn1Ggz6gPWON5vGjCokocCvszHB0hls0Q5DNSZvtzU
	 oSDBNeTNty7TmAF4Nla7V+f5/SwU/qvbE3d0RQQTFHQFgVqhKcn9WrBbTg33wxsMJB
	 w61zfpiZGypgg==
Date: Thu, 9 Sep 2021 15:54:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091540470.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1112020077-1631227292=:10523"
Content-ID: <alpine.DEB.2.21.2109091541340.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1112020077-1631227292=:10523
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109091541341.10523@sstabellini-ThinkPad-T480s>

On Thu, 19 Aug 2021, Rahul Singh wrote:
> XEN during boot will read the PCI device tree node “reg” property
> and will map the PCI config space to the XEN memory.
> 
> As of now "pci-host-ecam-generic" compatible board is supported.
> 
> "linux,pci-domain" device tree property assigns a fixed PCI domain
> number to a host bridge, otherwise an unstable (across boots) unique
> number will be assigned by Linux.This property has to be in sync with
> XEN to access the PCI devices.
> 
> XEN will read the “linux,pci-domain” property from the device tree node
> and configure the host bridge segment number accordingly. If this
> property is not available XEN will allocate the unique segment number
> to the host bridge.
> 
> dt_get_pci_domain_nr(..) and dt_pci_bus_find_domain_nr(..) are directly
> imported from the Linux source tree.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/pci/Makefile           |   2 +
>  xen/arch/arm/pci/pci-host-common.c  | 261 ++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-generic.c |  55 ++++++
>  xen/include/asm-arm/pci.h           |  28 +++
>  4 files changed, 346 insertions(+)
>  create mode 100644 xen/arch/arm/pci/pci-host-common.c
>  create mode 100644 xen/arch/arm/pci/pci-host-generic.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index a9ee0b9b44..f3d97f859e 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -1,2 +1,4 @@
>  obj-y += pci.o
>  obj-y += pci-access.o
> +obj-y += pci-host-generic.o
> +obj-y += pci-host-common.o
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> new file mode 100644
> index 0000000000..9dd9b02271
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -0,0 +1,261 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/ecam.c
> + * Copyright 2016 Broadcom.
> + *
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
> + *
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +#include <xen/rwlock.h>
> +#include <xen/sched.h>
> +#include <xen/vmap.h>
> +
> +/*
> + * List for all the pci host bridges.
> + */
> +
> +static LIST_HEAD(pci_host_bridges);
> +
> +static atomic_t domain_nr = ATOMIC_INIT(-1);
> +
> +bool dt_pci_parse_bus_range(struct dt_device_node *dev,
> +                            struct pci_config_window *cfg)
> +{
> +    const __be32 *cells;
> +    uint32_t len;
> +
> +    cells = dt_get_property(dev, "bus-range", &len);
> +    /* bus-range should at least be 2 cells */
> +    if ( !cells || (len < (sizeof(*cells) * 2)) )
> +        return false;
> +
> +    cfg->busn_start = dt_next_cell(1, &cells);
> +    cfg->busn_end = dt_next_cell(1, &cells);
> +
> +    return true;
> +}
> +
> +static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
> +{
> +    return ioremap_nocache(start, len);
> +}
> +
> +static void pci_ecam_free(struct pci_config_window *cfg)
> +{
> +    if ( cfg->win )
> +        iounmap(cfg->win);
> +
> +    xfree(cfg);
> +}
> +
> +static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
> +                                              int ecam_reg_idx)

If it is only called at init time, then the function should be __init


> +{
> +    int err;
> +    struct pci_config_window *cfg;
> +    paddr_t addr, size;
> +
> +    cfg = xzalloc(struct pci_config_window);
> +    if ( !cfg )
> +        return NULL;
> +
> +    err = dt_pci_parse_bus_range(dev, cfg);
> +    if ( !err ) {
> +        cfg->busn_start = 0;
> +        cfg->busn_end = 0xff;
> +        printk(XENLOG_ERR "%s:No bus range found for pci controller\n",
> +               dt_node_full_name(dev));
> +    } else {
> +        if ( cfg->busn_end > cfg->busn_start + 0xff )
> +            cfg->busn_end = cfg->busn_start + 0xff;

Is this a hard limit in the specification? Or is it a limit in the Xen
implementation?


> +    }
> +
> +    /* Parse our PCI ecam register address*/
                                            ^ space

> +    err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size);
> +    if ( err )
> +        goto err_exit;
> +
> +    cfg->phys_addr = addr;
> +    cfg->size = size;
> +
> +    /*
> +     * On 64-bit systems, we do a single ioremap for the whole config space
> +     * since we have enough virtual address range available.  On 32-bit, we
> +     * ioremap the config space for each bus individually.
> +     *
> +     * As of now only 64-bit is supported 32-bit is not supported.
> +     */
> +    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
> +    if ( !cfg->win )
> +        goto err_exit_remap;
> +
> +    printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr,
> +            cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end);
> +
> +    return cfg;
> +
> +err_exit_remap:
> +    printk(XENLOG_ERR "ECAM ioremap failed\n");
> +err_exit:
> +    pci_ecam_free(cfg);
> +    return NULL;
> +}
> +
> +struct pci_host_bridge *pci_alloc_host_bridge(void)
> +{
> +    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
> +
> +    if ( !bridge )
> +        return NULL;
> +
> +    INIT_LIST_HEAD(&bridge->node);
> +    bridge->bus_start = ~0;
> +    bridge->bus_end = ~0;

Please use INVALID_PADDR instead of ~0


> +    return bridge;
> +}
> +
> +void pci_add_host_bridge(struct pci_host_bridge *bridge)
> +{
> +    list_add_tail(&bridge->node, &pci_host_bridges);
> +}
> +
> +static int pci_get_new_domain_nr(void)
> +{
> +    return atomic_inc_return(&domain_nr);
> +}
> +
> +/*
> + * This function will try to obtain the host bridge domain number by
> + * finding a property called "linux,pci-domain" of the given device node.
> + *
> + * @node: device tree node with the domain information
> + *
> + * Returns the associated domain number from DT in the range [0-0xffff], or
> + * a negative value if the required property is not found.
> + */
> +static int dt_get_pci_domain_nr(struct dt_device_node *node)
> +{
> +    u32 domain;
> +    int error;
> +
> +    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
> +    if ( !error )
> +        return -EINVAL;
> +
> +    return (u16)domain;

Let's check that domain <= UINT16_MAX


> +}
> +
> +static int pci_bus_find_domain_nr(struct dt_device_node *dev)
> +{
> +    static int use_dt_domains = -1;
> +    int domain;
> +
> +    domain = dt_get_pci_domain_nr(dev);
> +
> +    /*
> +     * Check DT domain and use_dt_domains values.
> +     *
> +     * If DT domain property is valid (domain >= 0) and
> +     * use_dt_domains != 0, the DT assignment is valid since this means
> +     * we have not previously allocated a domain number by using
> +     * pci_get_new_domain_nr(); we should also update use_dt_domains to
> +     * 1, to indicate that we have just assigned a domain number from
> +     * DT.
> +     *
> +     * If DT domain property value is not valid (ie domain < 0), and we
> +     * have not previously assigned a domain number from DT
> +     * (use_dt_domains != 1) we should assign a domain number by
> +     * using the:
> +     *
> +     * pci_get_new_domain_nr()
> +     *
> +     * API and update the use_dt_domains value to keep track of method we
> +     * are using to assign domain numbers (use_dt_domains = 0).
> +     *
> +     * All other combinations imply we have a platform that is trying
> +     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
> +     * which is a recipe for domain mishandling and it is prevented by
> +     * invalidating the domain value (domain = -1) and printing a
> +     * corresponding error.
> +     */
> +    if ( domain >= 0 && use_dt_domains )
> +    {
> +        use_dt_domains = 1;
> +    }
> +    else if ( domain < 0 && use_dt_domains != 1 )
> +    {
> +        use_dt_domains = 0;
> +        domain = pci_get_new_domain_nr();
> +    }
> +    else
> +    {
> +        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
> +        BUG();
> +    }
> +
> +    return domain;
> +}
> +
> +int pci_host_common_probe(struct dt_device_node *dev,
> +                          int ecam_reg_idx)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct pci_config_window *cfg;
> +    int err;
> +
> +    bridge = pci_alloc_host_bridge();
> +    if ( !bridge )
> +        return -ENOMEM;
> +
> +    /* Parse and map our Configuration Space windows */
> +    cfg = gen_pci_init(dev, ecam_reg_idx);
> +    if ( !cfg )
> +    {
> +        err = -ENOMEM;
> +        goto err_exit;
> +    }
> +
> +    bridge->dt_node = dev;
> +    bridge->sysdata = cfg;
> +    bridge->bus_start = cfg->busn_start;
> +    bridge->bus_end = cfg->busn_end;
> +
> +    bridge->segment = pci_bus_find_domain_nr(dev);
> +
> +    pci_add_host_bridge(bridge);
> +
> +    return 0;
> +
> +err_exit:
> +    xfree(bridge);
> +    return err;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> new file mode 100644
> index 0000000000..13d0f7f999
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -0,0 +1,55 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static const struct dt_device_match gen_pci_dt_match[] = {
> +    { .compatible = "pci-host-ecam-generic" },
> +    { },
> +};
> +
> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
> +{
> +    const struct dt_device_match *of_id;
> +
> +    of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);
> +
> +    printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n",
> +           dt_node_full_name(dev), of_id->compatible);
> +
> +    return pci_host_common_probe(dev, 0);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> +.dt_match = gen_pci_dt_match,
> +.init = gen_pci_dt_init,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 61e43da088..58a51e724e 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,34 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/*
> + * struct to hold the mappings of a config space window. This
> + * is expected to be used as sysdata for PCI controllers that
> + * use ECAM.
> + */
> +struct pci_config_window {
> +    paddr_t         phys_addr;
> +    paddr_t         size;
> +    uint8_t         busn_start;
> +    uint8_t         busn_end;
> +    void __iomem    *win;
> +};
> +
> +/*
> + * struct to hold pci host bridge information
> + * for a PCI controller.
> + */
> +struct pci_host_bridge {
> +    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
> +    struct list_head node;           /* Node in list of host bridges */
> +    uint16_t segment;                /* Segment number */
> +    u8 bus_start;                    /* Bus start of this bridge. */
> +    u8 bus_end;                      /* Bus end of this bridge. */
> +    void *sysdata;                   /* Pointer to the config space window*/
> +};
> +
> +int pci_host_common_probe(struct dt_device_node *dev,
> +                          int ecam_reg_idx);
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  struct arch_pci_dev { };
> -- 
> 2.17.1
> 
--8323329-1112020077-1631227292=:10523--


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:01:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:01:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183689.331972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOT3C-0008Ug-Ny; Thu, 09 Sep 2021 23:01:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183689.331972; Thu, 09 Sep 2021 23:01:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOT3C-0008UZ-K1; Thu, 09 Sep 2021 23:01:46 +0000
Received: by outflank-mailman (input) for mailman id 183689;
 Thu, 09 Sep 2021 23:01:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+l3J=N7=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mOT3B-0008UT-Fu
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:01:45 +0000
Received: from hera.aquilenet.fr (unknown [185.233.100.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01c31b27-78f3-4f5d-8e86-ff306f655648;
 Thu, 09 Sep 2021 23:01:43 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id 4BBBC4CA;
 Fri, 10 Sep 2021 01:01:42 +0200 (CEST)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id PB4PWQxBd3YX; Fri, 10 Sep 2021 01:01:38 +0200 (CEST)
Received: from begin (unknown [IPv6:2a01:cb19:956:1b00:de41:a9ff:fe47:ec49])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id 62B4C1BF;
 Fri, 10 Sep 2021 01:01:38 +0200 (CEST)
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mOT33-00Hagn-6V;
 Fri, 10 Sep 2021 01:01:37 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01c31b27-78f3-4f5d-8e86-ff306f655648
X-Virus-Scanned: Debian amavisd-new at aquilenet.fr
Date: Fri, 10 Sep 2021 01:01:37 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH v2 3/3] tools: disable building qemu-trad per default
Message-ID: <20210909230137.bl7rd64z42vfhtau@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
	Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Wei Liu <wl@xen.org>
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-4-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210909124924.1698-4-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)
X-Spamd-Bar: --
Authentication-Results: hera.aquilenet.fr
X-Rspamd-Server: hera
X-Rspamd-Queue-Id: 4BBBC4CA
X-Spamd-Result: default: False [-2.50 / 15.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 RCPT_COUNT_FIVE(0.00)[5];
	 HAS_ORG_HEADER(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 RCVD_NO_TLS_LAST(0.10)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MID_RHS_NOT_FQDN(0.50)[];
	 BAYES_HAM(-3.00)[100.00%]

Juergen Gross, le jeu. 09 sept. 2021 14:49:24 +0200, a ecrit:
> Using qemu-traditional as device model is deprecated for some time now.
> 
> So change the default for building it to "disable". This will affect
> ioemu-stubdom, too, as there is a direct dependency between the two.
> 
> Today it is possible to use a PVH/HVM Linux-based stubdom as device
> model. Additionally using ioemu-stubdom isn't really helping for
> security, as it requires to run a very old and potentially buggy qemu
> version in a PV domain. This is adding probably more security problems
> than it is removing by using a stubdom.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
> V2:
> - new patch
> ---
>  CHANGELOG.md         |  3 +++
>  stubdom/configure    |  8 --------
>  stubdom/configure.ac |  8 +-------
>  tools/configure      | 17 ++---------------
>  tools/configure.ac   | 13 +------------
>  5 files changed, 7 insertions(+), 42 deletions(-)
> 
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index e7107ac3de..e5ab49e779 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -18,6 +18,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
>     or by passing "iommu=quarantine=scratch-page" on the hypervisor command line.
>   - pv-grub stubdoms will no longer be built per default. In order to be able to use pv-grub
>     configure needs to be called with "--enable-pv-grub" as parameter.
> + - qemu-traditional based device models (both, qemu-traditional and ioemu-stubdom) will
> +   no longer be built per default. In order to be able to use those, configure needs to
> +   be called with "--enable-qemu-traditional" as parameter.
>  
>  ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.15.0) - TBD
>  
> diff --git a/stubdom/configure b/stubdom/configure
> index df31532abb..07b709f998 100755
> --- a/stubdom/configure
> +++ b/stubdom/configure
> @@ -2286,14 +2286,6 @@ fi
>  # Check whether --enable-qemu-traditional was given.
>  if test "${enable_qemu_traditional+set}" = set; then :
>    enableval=$enable_qemu_traditional;
> -else
> -
> -    case "$host_cpu" in
> -        i[3456]86|x86_64)
> -           enable_qemu_traditional="yes";;
> -        *) enable_qemu_traditional="no";;
> -    esac
> -
>  fi
>  
>  if test "x$enable_qemu_traditional" = "xyes"; then :
> diff --git a/stubdom/configure.ac b/stubdom/configure.ac
> index a07a1edae5..e20d99edac 100644
> --- a/stubdom/configure.ac
> +++ b/stubdom/configure.ac
> @@ -27,13 +27,7 @@ AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
>  AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
>  AX_STUBDOM_CONDITIONAL([vtpmmgr-stubdom], [vtpmmgr])
>  
> -AC_ARG_ENABLE([qemu-traditional],,,[
> -    case "$host_cpu" in
> -        i[[3456]]86|x86_64)
> -           enable_qemu_traditional="yes";;
> -        *) enable_qemu_traditional="no";;
> -    esac
> -])
> +AC_ARG_ENABLE([qemu-traditional])
>  AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
>      qemu_traditional=y],[
>      qemu_traditional=n
> diff --git a/tools/configure b/tools/configure
> index 33814b24b3..8bf8fe75b8 100755
> --- a/tools/configure
> +++ b/tools/configure
> @@ -1502,8 +1502,8 @@ Optional Features:
>    --disable-seabios       Disable SeaBIOS (default is ENABLED)
>    --disable-golang        Disable Go tools (default is ENABLED)
>    --enable-qemu-traditional
> -                          Enable qemu traditional device model, (DEFAULT is on
> -                          for Linux or NetBSD x86, otherwise off)
> +                          Enable qemu traditional device model, (DEFAULT is
> +                          off)
>    --enable-rombios        Enable ROMBIOS, (DEFAULT is on if qemu-traditional
>                            is enabled, otherwise off)
>    --disable-ipxe          Enable in-tree IPXE, (DEFAULT is on if rombios is
> @@ -4287,19 +4287,6 @@ LINUX_BACKEND_MODULES="`eval echo $LINUX_BACKEND_MODULES`"
>  # Check whether --enable-qemu-traditional was given.
>  if test "${enable_qemu_traditional+set}" = set; then :
>    enableval=$enable_qemu_traditional;
> -else
> -
> -    case "$host_cpu" in
> -        i[3456]86|x86_64)
> -           enable_qemu_traditional="yes";;
> -        *) enable_qemu_traditional="no";;
> -    esac
> -    case "$host_os" in
> -        freebsd*)
> -           enable_qemu_traditional="no";;
> -    esac
> -
> -
>  fi
>  
>  if test "x$enable_qemu_traditional" = "xyes"; then :
> diff --git a/tools/configure.ac b/tools/configure.ac
> index 6414fcbb44..a713fd34d6 100644
> --- a/tools/configure.ac
> +++ b/tools/configure.ac
> @@ -120,18 +120,7 @@ AC_SUBST(LINUX_BACKEND_MODULES)
>  
>  AC_ARG_ENABLE([qemu-traditional],
>      AS_HELP_STRING([--enable-qemu-traditional],
> -                   [Enable qemu traditional device model, (DEFAULT is on for Linux or NetBSD x86, otherwise off)]),,[
> -    case "$host_cpu" in
> -        i[[3456]]86|x86_64)
> -           enable_qemu_traditional="yes";;
> -        *) enable_qemu_traditional="no";;
> -    esac
> -    case "$host_os" in
> -        freebsd*)
> -           enable_qemu_traditional="no";;
> -    esac
> -
> -])
> +                   [Enable qemu traditional device model, (DEFAULT is off)]))
>  AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
>  AC_DEFINE([HAVE_QEMU_TRADITIONAL], [1], [Qemu traditional enabled])
>      qemu_traditional=y],[
> -- 
> 2.26.2
> 

-- 
Samuel
<A>  mr  -  remove the home of correct users who accidentally enter mr
<A>        instead of rm


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:06:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:06:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183695.331983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOT7r-0000j2-A1; Thu, 09 Sep 2021 23:06:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183695.331983; Thu, 09 Sep 2021 23:06:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOT7r-0000iv-6G; Thu, 09 Sep 2021 23:06:35 +0000
Received: by outflank-mailman (input) for mailman id 183695;
 Thu, 09 Sep 2021 23:06:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j+xv=N7=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOT7p-0000ip-8K
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:06:33 +0000
Received: from mail-lj1-x22f.google.com (unknown [2a00:1450:4864:20::22f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 820d7511-77e1-4a27-affe-70f3a827654a;
 Thu, 09 Sep 2021 23:06:30 +0000 (UTC)
Received: by mail-lj1-x22f.google.com with SMTP id i28so69789ljm.7
 for <xen-devel@lists.xenproject.org>; Thu, 09 Sep 2021 16:06:30 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id m7sm351737ljj.58.2021.09.09.16.06.27
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 09 Sep 2021 16:06:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 820d7511-77e1-4a27-affe-70f3a827654a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=bniGfgIdDUqgT663lKQJPFvOk8NcJzQ94iqstSMehBc=;
        b=NUlHVLUHs9X5EljxJUT3ySBsqIlW7NCuyH1PaugohXQ0LsWPDED0dUQsYbevgRWCrz
         uaFPSAXTRA4bNQS0n4LmjmlygpDXLwWDp25sUtyGj1EMfQOmXr7qigd48L2tnwZnjPjs
         XxWh30jB0R5JQvzJe/oFb6UHHkVOFzPL6pTLNbo+gTce2m/m39WZrQAVFcbAxW8jXrXG
         V/dZhWENzDXEKRC8bOfB4vo5Z46NS5O6hZeCLdr9W2w2Y2fnNHYxMKbZ8OgCZSG87xlm
         DPQoijAw2HVqUcZXCTQR1s59sCNyrjkKvgyHNFJ+9urLTmc4qFfGAlyC9WLG7xXZsoFC
         oSsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=bniGfgIdDUqgT663lKQJPFvOk8NcJzQ94iqstSMehBc=;
        b=NhWhYFABVqvgtFyDEdpmrlWe2ScjqNylIE6foNlr+ZbmtyPuySbzXTjNs8CF5XW1ym
         zz/K6Uu2frsA/+gNQEys1kOJMInvfIm9IrujNuFCHQuJVmXHripVNp9PwVDhfCMvEqsX
         gl9TE9fjZbAWvAyUL+yHiVffGQXWSXnnqOtyrtzPZISdeZEg3h75Dy7fhHq5J66MCyU1
         6aPbDSC8maV4CPvc3e/V78mDFv1EAUVolU0zzfZXnqjOlNVPa40yp32VQZuzMeGXASOP
         h57LUBeub0X2/CiNPvb17Hmqybm+pLRxRv6aGYg9NFCBFIEEx5KDRcPcjSlj4EmIfOl3
         7Uqg==
X-Gm-Message-State: AOAM530zRRvjOEC4y0Q898XsYpCqxSy14EsySeIvZng1SZwXmWmMUjc7
	yUuoRcVu9+9rpISFbgcLC8qPJhCL8uc=
X-Google-Smtp-Source: ABdhPJzKbnEJyz26TsA6xyg4GO75xdCAIa/9qv/TD1BueHbjPh0mM9Icb9/tcxO4OIwoky1Dpmlqbw==
X-Received: by 2002:a2e:b16a:: with SMTP id a10mr1742332ljm.18.1631228788967;
        Thu, 09 Sep 2021 16:06:28 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on Arm
Date: Fri, 10 Sep 2021 02:04:48 +0300
Message-Id: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Rework Arm implementation to store grant table frame GFN
in struct page_info directly instead of keeping it in
standalone status/shared arrays.

To cover 64-bit/40-bit IPA on Arm64/Arm32 we need the new
field to hold 52-bit/28-bit respectively. In order to not
grow the size of struct page_info reduce the type_info field
which current context won't suffer. Add new frame_gfn field
on top of the freed bits and introduce access macros. Update
existing macros to deal with GFN value according to new
location.

Also update P2M code on Arm to clean said GFN when putting
a reference on the grant table page in p2m_put_l3_page().

This patch is intended to fix the potential issue on Arm
which might happen when remapping grant-table frame.
A guest (or the toolstack) will unmap the grant-table frame
using XENMEM_remove_physmap. This is a generic hypercall,
so on x86, we are relying on the fact the M2P entry will
be cleared on removal. For architecture without the M2P,
the GFN would still be present in the grant frame/status
array. So on the next call to map the page, we will end up to
request the P2M to remove whatever mapping was the given GFN.
This could well be another mapping.

Besides that, this patch simplifies arch code on Arm by
removing arrays and corresponding management code and
as the result gnttab_init_arch/gnttab_destroy_arch helpers
and struct grant_table_arch become useless and can be dropped.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
You can find the related discussions at:
https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/

! Please note, there is still unresolved locking question here for which
I failed to find a suitable solution !

According to the internal conversation:
Now the GFN field in the struct page_info is accessed from
gnttab_set_frame_gfn() in the grant table code and from page_set_frame_gfn()
in the P2M code (the former uses the latter).

We need to prevent the concurrent access to this field. But, we cannot grab
the grant lock from the P2M code because we will introduce a lock inversion.
The page_set_frame_gfn() will be called from the P2M code with the p2m lock held
and then acquire the grant table lock. The gnttab_map_frame() will do the inverse.
---
 xen/arch/arm/p2m.c                | 18 ++++++++++---
 xen/common/grant_table.c          |  9 -------
 xen/common/page_alloc.c           |  1 +
 xen/include/asm-arm/grant_table.h | 53 +++++++++++++--------------------------
 xen/include/asm-arm/mm.h          | 27 ++++++++++++++++----
 xen/include/asm-x86/grant_table.h |  5 ----
 xen/include/asm-x86/mm.h          |  2 ++
 7 files changed, 56 insertions(+), 59 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index eff9a10..7bef210 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -718,8 +718,10 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
  * TODO: Handle superpages, for now we only take special references for leaf
  * pages (specifically foreign ones, which can't be super mapped today).
  */
-static void p2m_put_l3_page(const lpae_t pte)
+static void p2m_put_l3_page(struct p2m_domain *p2m, const lpae_t pte)
 {
+    mfn_t mfn = lpae_get_mfn(pte);
+
     ASSERT(p2m_is_valid(pte));
 
     /*
@@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
      */
     if ( p2m_is_foreign(pte.p2m.type) )
     {
-        mfn_t mfn = lpae_get_mfn(pte);
-
         ASSERT(mfn_valid(mfn));
         put_page(mfn_to_page(mfn));
     }
+
+#ifdef CONFIG_GRANT_TABLE
+    /*
+     * Check whether we deal with grant table page. As the grant table page
+     * is xen_heap page and its entry has known p2m type, detect it and mark
+     * the stored GFN as invalid.
+     */
+    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
+        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
+#endif
 }
 
 /* Free lpae sub-tree behind an entry */
@@ -768,7 +778,7 @@ static void p2m_free_entry(struct p2m_domain *p2m,
         p2m->stats.mappings[level]--;
         /* Nothing to do if the entry is a super-page. */
         if ( level == 3 )
-            p2m_put_l3_page(entry);
+            p2m_put_l3_page(p2m, entry);
         return;
     }
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index b1930e2..9cc3550 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -93,8 +93,6 @@ struct grant_table {
 
     /* Domain to which this struct grant_table belongs. */
     const struct domain *domain;
-
-    struct grant_table_arch arch;
 };
 
 unsigned int __read_mostly opt_max_grant_frames = 64;
@@ -1981,14 +1979,9 @@ int grant_table_init(struct domain *d, int max_grant_frames,
 
     grant_write_lock(gt);
 
-    ret = gnttab_init_arch(gt);
-    if ( ret )
-        goto unlock;
-
     /* gnttab_grow_table() allocates a min number of frames, so 0 is okay. */
     ret = gnttab_grow_table(d, 0);
 
- unlock:
     grant_write_unlock(gt);
 
  out:
@@ -3895,8 +3888,6 @@ grant_table_destroy(
     if ( t == NULL )
         return;
 
-    gnttab_destroy_arch(t);
-
     for ( i = 0; i < nr_grant_frames(t); i++ )
         free_xenheap_page(t->shared_raw[i]);
     xfree(t->shared_raw);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 958ba0c..f7428f9 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1021,6 +1021,7 @@ static struct page_info *alloc_heap_pages(
 
         /* Initialise fields which have other uses for free pages. */
         pg[i].u.inuse.type_info = 0;
+        page_arch_init(&pg[i]);
         page_set_owner(&pg[i], NULL);
 
     }
diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h
index 0ce77f9..77dfa8e 100644
--- a/xen/include/asm-arm/grant_table.h
+++ b/xen/include/asm-arm/grant_table.h
@@ -11,11 +11,6 @@
 #define INITIAL_NR_GRANT_FRAMES 1U
 #define GNTTAB_MAX_VERSION 1
 
-struct grant_table_arch {
-    gfn_t *shared_gfn;
-    gfn_t *status_gfn;
-};
-
 static inline void gnttab_clear_flags(struct domain *d,
                                       unsigned int mask, uint16_t *addr)
 {
@@ -46,35 +41,11 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
 #define gnttab_dom0_frames()                                             \
     min_t(unsigned int, opt_max_grant_frames, PFN_DOWN(_etext - _stext))
 
-#define gnttab_init_arch(gt)                                             \
-({                                                                       \
-    unsigned int ngf_ = (gt)->max_grant_frames;                          \
-    unsigned int nsf_ = grant_to_status_frames(ngf_);                    \
-                                                                         \
-    (gt)->arch.shared_gfn = xmalloc_array(gfn_t, ngf_);                  \
-    (gt)->arch.status_gfn = xmalloc_array(gfn_t, nsf_);                  \
-    if ( (gt)->arch.shared_gfn && (gt)->arch.status_gfn )                \
-    {                                                                    \
-        while ( ngf_-- )                                                 \
-            (gt)->arch.shared_gfn[ngf_] = INVALID_GFN;                   \
-        while ( nsf_-- )                                                 \
-            (gt)->arch.status_gfn[nsf_] = INVALID_GFN;                   \
-    }                                                                    \
-    else                                                                 \
-        gnttab_destroy_arch(gt);                                         \
-    (gt)->arch.shared_gfn ? 0 : -ENOMEM;                                 \
-})
-
-#define gnttab_destroy_arch(gt)                                          \
-    do {                                                                 \
-        XFREE((gt)->arch.shared_gfn);                                    \
-        XFREE((gt)->arch.status_gfn);                                    \
-    } while ( 0 )
-
 #define gnttab_set_frame_gfn(gt, st, idx, gfn)                           \
     do {                                                                 \
-        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =    \
-            (gfn);                                                       \
+        struct page_info *pg_ = (st) ? gnttab_status_page(gt, idx)       \
+                                     : gnttab_shared_page(gt, idx);      \
+        page_set_frame_gfn(pg_, gfn);                                    \
     } while ( 0 )
 
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
@@ -82,11 +53,21 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
         : gnttab_shared_gfn(NULL, gt, idx);                              \
 })
 
-#define gnttab_shared_gfn(d, t, i)                                       \
-    (((i) >= nr_grant_frames(t)) ? INVALID_GFN : (t)->arch.shared_gfn[i])
+#define gnttab_shared_page(t, i)    \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->shared_raw[i])))
+
+#define gnttab_status_page(t, i)    \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->status[i])))
 
-#define gnttab_status_gfn(d, t, i)                                       \
-    (((i) >= nr_status_frames(t)) ? INVALID_GFN : (t)->arch.status_gfn[i])
+#define gnttab_shared_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_shared_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
+
+#define gnttab_status_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_status_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
 
 #define gnttab_need_iommu_mapping(d)                    \
     (is_domain_direct_mapped(d) && is_iommu_enabled(d))
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index ded74d2..dea4c49 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -39,7 +39,15 @@ struct page_info
         /* Page is in use: ((count_info & PGC_count_mask) != 0). */
         struct {
             /* Type reference count and various PGT_xxx flags and fields. */
-            unsigned long type_info;
+            unsigned long type_info:4;
+
+            /*
+             * Stored GFN if page is used for grant table frame
+             * (bits 59(27)-0).
+             */
+#define PGT_FRAME_GFN_WIDTH      (BITS_PER_LONG - 4)
+#define PGT_INVALID_FRAME_GFN    _gfn((1UL << PGT_FRAME_GFN_WIDTH) - 1)
+            unsigned long frame_gfn:PGT_FRAME_GFN_WIDTH;
         } inuse;
         /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
         union {
@@ -94,12 +102,12 @@ struct page_info
 #define PG_shift(idx)   (BITS_PER_LONG - (idx))
 #define PG_mask(x, idx) (x ## UL << PG_shift(idx))
 
-#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
-#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
-#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+#define PGT_none          (0UL << 3)  /* no special uses of this page   */
+#define PGT_writable_page (1UL << 3)  /* has writable mappings?         */
+#define PGT_type_mask     (1UL << 3)
 
  /* Count of uses of this frame as its current type. */
-#define PGT_count_width   PG_shift(2)
+#define PGT_count_width   1
 #define PGT_count_mask    ((1UL<<PGT_count_width)-1)
 
  /* Cleared when the owning guest 'frees' this page. */
@@ -163,6 +171,15 @@ extern unsigned long xenheap_base_pdx;
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+#define page_get_frame_gfn(_p) ({                               \
+    gfn_t gfn_ = _gfn((_p)->u.inuse.frame_gfn);                 \
+    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
+})
+
+#define page_set_frame_gfn(_p, _gfn) ((_p)->u.inuse.frame_gfn = gfn_x(_gfn))
+
+#define page_arch_init(_p)   page_set_frame_gfn(_p, INVALID_GFN)
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 /* PDX of the first page in the frame table. */
 extern unsigned long frametable_base_pdx;
diff --git a/xen/include/asm-x86/grant_table.h b/xen/include/asm-x86/grant_table.h
index 84e3296..0eb018f 100644
--- a/xen/include/asm-x86/grant_table.h
+++ b/xen/include/asm-x86/grant_table.h
@@ -14,9 +14,6 @@
 
 #define INITIAL_NR_GRANT_FRAMES 1U
 
-struct grant_table_arch {
-};
-
 static inline int create_grant_host_mapping(uint64_t addr, mfn_t frame,
                                             unsigned int flags,
                                             unsigned int cache_flags)
@@ -35,8 +32,6 @@ static inline int replace_grant_host_mapping(uint64_t addr, mfn_t frame,
     return replace_grant_pv_mapping(addr, frame, new_addr, flags);
 }
 
-#define gnttab_init_arch(gt) 0
-#define gnttab_destroy_arch(gt) do {} while ( 0 )
 #define gnttab_set_frame_gfn(gt, st, idx, gfn) do {} while ( 0 )
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
     mfn_t mfn_ = (st) ? gnttab_status_mfn(gt, idx)                       \
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index cb90527..d2ea39b 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -327,6 +327,8 @@ struct page_info
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+#define page_arch_init(_p)    do {} while ( 0 )
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 extern unsigned long max_page;
 extern unsigned long total_pages;
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:20:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:20:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183703.331993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTLE-0003I1-Ka; Thu, 09 Sep 2021 23:20:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183703.331993; Thu, 09 Sep 2021 23:20:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTLE-0003Hu-He; Thu, 09 Sep 2021 23:20:24 +0000
Received: by outflank-mailman (input) for mailman id 183703;
 Thu, 09 Sep 2021 23:20:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j+xv=N7=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOTLE-0003Ho-0m
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:20:24 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ecdb650c-acbe-44d2-b7b7-3024e35dd7eb;
 Thu, 09 Sep 2021 23:20:22 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id k4so166976lfj.7
 for <xen-devel@lists.xenproject.org>; Thu, 09 Sep 2021 16:20:22 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id f21sm340577lfe.108.2021.09.09.16.20.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 09 Sep 2021 16:20:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecdb650c-acbe-44d2-b7b7-3024e35dd7eb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=tJhxMGD54i/lcUR8ooXibQA/tFAz+ffZMN72C3xDPZ8=;
        b=cRedJeJCVhg6CK2ip7s9irCSticVQ4lQtg0VEp1+nYmTLj77h9dxTzUl59dEU6qmfN
         FrsFZs7XUU/KfIkpFnAv3v9lW57ynII5Mj+PMdCMnadYuhDir+aw3jO7o5xDn8NBxu94
         Dij1rrY312ibGsvChlT3iq6+i6B3uCZw22ykTSXxXgC6kvlTiAntvEfYFR8BvKjU80W9
         kOkWGx+huaN6s8JzPZY9CHY6qr9kLQ2zvy7suWD4hqjHg45lcGOkHdqQoSCzYQz3f2HF
         ezkn68BnD/OnDH+Aopf7TH2WRye9IdXQHlJmP8maQon++IxEA5b3/+Z3cA3BqMDMpH4U
         QQOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=tJhxMGD54i/lcUR8ooXibQA/tFAz+ffZMN72C3xDPZ8=;
        b=U7SDbN3yp5fuW0+EzwjKsOVCJRALWbeF5p6dhfyN8uMMnjFIsHKAu6kvIOQd6Tpzsl
         8Yv2Edc+dtfu8r+MV7n3Tjva/4VH70mcmI6G92McIqQvvLAMADpwxCJGbJ2AP4d43Xzx
         KiVrN6bkqMRtGxGV5k41NjDgmczCYyHBJY0z3BW27P202aFXKhIvaQVQvNCWidc8kLsu
         yHRtBmUqR4NwLVa7ZkToCK7ocYD55FIvDoazSHWWQg6COC/jOItIUOozheOvnOuJKAOw
         r7cidwmQNF4o/6ebv35KJpDWgNL4lP05td/jqDIVKCjtYwClkv8C4N3d7kNLBbnje9tA
         qA9g==
X-Gm-Message-State: AOAM530mi30HXe/26XBsdJVGQWJijbwXOndAjrvfB2t3QPH0sAryzIQn
	bapi2QVHCAuHxaQcIz1xYe8=
X-Google-Smtp-Source: ABdhPJwSmkvui2HtTzArosTJLrwloPgqM/dcU7oZQfjufeFkbe5/QTTYOv7USoAF7bTpklUsB+5cXA==
X-Received: by 2002:ac2:5c46:: with SMTP id s6mr1631266lfp.78.1631229621519;
        Thu, 09 Sep 2021 16:20:21 -0700 (PDT)
Subject: Re: [RFC PATCH 0/3] Add handling of extended regions (safe ranges) on
 Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
From: Oleksandr <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
Message-ID: <6b2bea7a-63dc-bb42-9e0b-fb49b05e310a@gmail.com>
Date: Fri, 10 Sep 2021 02:20:20 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


Hello all

On 07.09.21 20:09, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> You can find an initial discussion at [1].
>
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
>
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree.
>
> The extended regions are calculated differently for direct mapped
> Dom0 (with and without IOMMU) and non-direct mapped DomUs.
>
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain.
> - The ACPI case is not covered.
>
> Also please note that we haven't figured out yet how to properly
> extend the Xen hypervisor device-tree bindings on Arm (either via new
> compatible or via new property). I decided to go with new property
> for now, but this can be changed. This uncertainty is the main reason
> why this series is marked as RFC.

Sorry, I messed up the device tree binding's purpose here.

New DT property "extended-region" (to inform guest about the presence of 
additional regions in "reg" property) is not really needed. Guest can 
simply infer that from the number of regions
in "reg" property (region 0 - reserved for grant table space, regions 
1...n - extended regions).
Instead, new compatible/property will be needed (but only after this 
patch [1] or alternative goes in) to indicate that "region 0 is safe to 
use". Until this patch is merged it is
not safe to use extended regions for the grant table space.

Thanks to Julien for clarifying these bits.

I am going to remove the advertisement of unneeded "extended-region" 
property in the code and send new version soon.

[1] 
https://lore.kernel.org/xen-devel/1631228688-30347-1-git-send-email-olekstysh@gmail.com/


>
> Patch series is also available at [2].
>
> The corresponding Linux changes is not in a good shape now (require
> some cleanup and refactoring), I will publish them once put them
> in order (I hope, it will be in a few days).
>
> [1] https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
> [2] https://github.com/otyshchenko1/xen/commits/map_opt_ml2
>
> Oleksandr Tyshchenko (3):
>    xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
>    xen/arm: Add handling of extended regions for Dom0
>    toolstack/arm: Add handling of extended regions for DomU
>
>   tools/include/libxl.h            |   7 ++
>   tools/libs/light/libxl.c         |   2 +
>   tools/libs/light/libxl_arm.c     |  92 +++++++++++++++-
>   tools/libs/light/libxl_types.idl |   2 +
>   xen/arch/arm/domain_build.c      | 233 ++++++++++++++++++++++++++++++++++++++-
>   xen/arch/arm/sysctl.c            |   2 +
>   xen/arch/x86/sysctl.c            |   2 +
>   xen/include/public/sysctl.h      |   3 +-
>   8 files changed, 338 insertions(+), 5 deletions(-)
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:22:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:22:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183710.332004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTMn-0004Av-0g; Thu, 09 Sep 2021 23:22:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183710.332004; Thu, 09 Sep 2021 23:22:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTMm-0004Ao-Ts; Thu, 09 Sep 2021 23:22:00 +0000
Received: by outflank-mailman (input) for mailman id 183710;
 Thu, 09 Sep 2021 23:21:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOTMl-0004Ag-Di
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:21:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ebf5888b-05dd-4e2f-bcf9-0430b6d9b844;
 Thu, 09 Sep 2021 23:21:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1ABEB6108B;
 Thu,  9 Sep 2021 23:21:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebf5888b-05dd-4e2f-bcf9-0430b6d9b844
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631229717;
	bh=URuwH9vcdH0dEifdnbUurmo8XzSDxgi09Z5gQ0WUa/w=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gimQBcQbuIa9989u6m1Pwp2SiyBYCkhNGkwDkVf6XKNDaQpV243HMoqmXxVGUXP9e
	 HEA7UsqZ0pT2TO56At9KLNBg6F+YRZV2IfUbyR7os+drnoW/9KGVBScqAmQUmTw2QR
	 21b1D/cQdEc5C94MVKzMV2ppAKTWCx8RdfgB3E/FU4GjYHOthhel+PhHNSi8dRtky4
	 tZFaqWNFJCB6BRHunKDzVDM20xseNlVipkGADr+u/VCqsnmKA81mVfv6daqczhlEZc
	 3/iyByYKgxC3LfOTvirmoOrQJH1svvQCIv67C6PBuALbWF+Xin+mXxRTsrFG4v+oP7
	 axXrW6Yp9Gp/w==
Date: Thu, 9 Sep 2021 16:21:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
In-Reply-To: <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> Add support for PCI ecam operations to access the PCI
> configuration space.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/pci/Makefile           |  1 +
>  xen/arch/arm/pci/ecam.c             | 63 +++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-access.c       | 53 ++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c  | 13 +++++-
>  xen/arch/arm/pci/pci-host-generic.c |  8 +++-
>  xen/include/asm-arm/pci.h           | 32 +++++++++++++++
>  6 files changed, 167 insertions(+), 3 deletions(-)
>  create mode 100644 xen/arch/arm/pci/ecam.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index f3d97f859e..6f32fbbe67 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -2,3 +2,4 @@ obj-y += pci.o
>  obj-y += pci-access.o
>  obj-y += pci-host-generic.o
>  obj-y += pci-host-common.o
> +obj-y += ecam.o
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> new file mode 100644
> index 0000000000..91c691b41f
> --- /dev/null
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -0,0 +1,63 @@
> +/*
> + * Copyright (C) 2021 Arm Ltd.
> + *
> + * Based on Linux drivers/pci/ecam.c
> + * Copyright 2016 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/sched.h>
> +
> +/*
> + * Function to implement the pci_ops ->map_bus method.
> + */
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                                      uint32_t sbdf, uint32_t where)

Code style: alignment


> +{
> +    const struct pci_config_window *cfg = bridge->sysdata;
> +    unsigned int devfn_shift = cfg->ops->bus_shift - 8;

Is it a guarantee that devfn_shift == bus_shift - 8, or is it just so
for ECAM?


> +    void __iomem *base;
> +
> +    pci_sbdf_t sbdf_t = (pci_sbdf_t) sbdf ;
> +    unsigned int busn = sbdf_t.bus;
> +
> +    if ( busn < cfg->busn_start || busn > cfg->busn_end )

Genuine question: should it be busn >= cfg->busn_end ?  I don't know if
the range includes busn_end or not.


> +        return NULL;
> +
> +    busn -= cfg->busn_start;
> +    base = cfg->win + (busn << cfg->ops->bus_shift);
> +
> +    return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
> +}

I understand that the arm32 part is not implemented and not part of this
series, that's fine. However if the plan is that arm32 will dynamically
map each bus individually, then I imagine this function will have an
ioremap in the arm32 version. Which means that we also need an
unmap_bus call in struct pci_ops. I understand that pci_ecam_unmap_bus
would be a NOP today for arm64, but I think it makes sense to have it if
we want the API to be generic.


> +/* ECAM ops */
> +const struct pci_ecam_ops pci_generic_ecam_ops = {
> +    .bus_shift  = 20,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index b938047c03..f39f6a3a38 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -15,6 +15,59 @@
>   */
>  
>  #include <xen/pci.h>
> +#include <asm/io.h>
> +
> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +    if (!addr) {
> +        *value = ~0;

Is this a standard error? If so, I think we should define it with a
macro (e.g. INVALID_PADDR).


> +        return -ENODEV;
> +    }
> +
> +    switch (len)
> +    {
> +    case 1:
> +        *value = readb(addr);
> +        break;
> +    case 2:
> +        *value = readw(addr);
> +        break;
> +    case 4:
> +        *value = readl(addr);
> +        break;
> +    default:
> +        BUG();

A BUG here is harsh because it could be potentially guest-triggered. An
ASSERT would be better.


> +    }
> +
> +    return 0;
> +}
> +
> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +    if (!addr)
> +        return -ENODEV;
> +
> +    switch (len)
> +    {
> +    case 1:
> +        writeb(value, addr);
> +        break;
> +    case 2:
> +        writew(value, addr);
> +        break;
> +    case 4:
> +        writel(value, addr);
> +        break;
> +    default:
> +        BUG();

Same here


> +    }
> +
> +    return 0;
> +}
>  
>  static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>                                  unsigned int len)
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 9dd9b02271..c582527e92 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -68,6 +68,7 @@ static void pci_ecam_free(struct pci_config_window *cfg)
>  }
>  
>  static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
> +                                              const struct pci_ecam_ops *ops,
>                                                int ecam_reg_idx)
>  {
>      int err;
> @@ -96,6 +97,7 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
>  
>      cfg->phys_addr = addr;
>      cfg->size = size;
> +    cfg->ops = ops;
>  
>      /*
>       * On 64-bit systems, we do a single ioremap for the whole config space
> @@ -111,6 +113,13 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
>      printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr,
>              cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end);
>  
> +    if ( ops->init )
> +    {
> +        err = ops->init(cfg);
> +        if (err)
> +            goto err_exit;
> +    }
> +
>      return cfg;
>  
>  err_exit_remap:
> @@ -216,6 +225,7 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev)
>  }
>  
>  int pci_host_common_probe(struct dt_device_node *dev,
> +                          const struct pci_ecam_ops *ops,
>                            int ecam_reg_idx)
>  {
>      struct pci_host_bridge *bridge;
> @@ -227,7 +237,7 @@ int pci_host_common_probe(struct dt_device_node *dev,
>          return -ENOMEM;
>  
>      /* Parse and map our Configuration Space windows */
> -    cfg = gen_pci_init(dev, ecam_reg_idx);
> +    cfg = gen_pci_init(dev, ops, ecam_reg_idx);
>      if ( !cfg )
>      {
>          err = -ENOMEM;
> @@ -236,6 +246,7 @@ int pci_host_common_probe(struct dt_device_node *dev,
>  
>      bridge->dt_node = dev;
>      bridge->sysdata = cfg;
> +    bridge->ops = &ops->pci_ops;
>      bridge->bus_start = cfg->busn_start;
>      bridge->bus_end = cfg->busn_end;
>  
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> index 13d0f7f999..2d652e8910 100644
> --- a/xen/arch/arm/pci/pci-host-generic.c
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -23,20 +23,24 @@
>  #include <asm/pci.h>
>  
>  static const struct dt_device_match gen_pci_dt_match[] = {
> -    { .compatible = "pci-host-ecam-generic" },
> +    { .compatible = "pci-host-ecam-generic",
> +      .data =       &pci_generic_ecam_ops },
> +
>      { },
>  };
>  
>  static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
>  {
>      const struct dt_device_match *of_id;
> +    const struct pci_ecam_ops *ops;
>  
>      of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);
> +    ops = (struct pci_ecam_ops *) of_id->data;
>  
>      printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n",
>             dt_node_full_name(dev), of_id->compatible);
>  
> -    return pci_host_common_probe(dev, 0);
> +    return pci_host_common_probe(dev, ops, 0);
>  }
>  
>  DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 58a51e724e..22866244d2 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -37,6 +37,7 @@ struct pci_config_window {
>      uint8_t         busn_start;
>      uint8_t         busn_end;
>      void __iomem    *win;
> +    const struct    pci_ecam_ops *ops;
>  };
>  
>  /*
> @@ -50,10 +51,41 @@ struct pci_host_bridge {
>      u8 bus_start;                    /* Bus start of this bridge. */
>      u8 bus_end;                      /* Bus end of this bridge. */
>      void *sysdata;                   /* Pointer to the config space window*/
> +    const struct pci_ops *ops;
>  };
>  
> +struct pci_ops {
> +    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                             uint32_t offset);
> +    int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                uint32_t reg, uint32_t len, uint32_t *value);
> +    int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                 uint32_t reg, uint32_t len, uint32_t value);
> +};
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_ecam_ops {
> +    unsigned int            bus_shift;
> +    struct pci_ops          pci_ops;
> +    int (*init)(struct pci_config_window *);

Is this last member of the struct needed/used?


> +};
> +
> +/* Default ECAM ops */
> +extern const struct pci_ecam_ops pci_generic_ecam_ops;
> +
>  int pci_host_common_probe(struct dt_device_node *dev,
> +                          const struct pci_ecam_ops *ops,
>                            int ecam_reg_idx);
> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value);
> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value);
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               uint32_t sbdf, uint32_t where);
> +
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  struct arch_pci_dev { };
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:34:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:34:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183718.332015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTYu-0005yD-1s; Thu, 09 Sep 2021 23:34:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183718.332015; Thu, 09 Sep 2021 23:34:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTYt-0005y6-VJ; Thu, 09 Sep 2021 23:34:31 +0000
Received: by outflank-mailman (input) for mailman id 183718;
 Thu, 09 Sep 2021 23:34:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOTYt-0005y0-7s
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:34:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7f8b6389-02cd-4948-9a59-3656849e494a;
 Thu, 09 Sep 2021 23:34:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 62366610C9;
 Thu,  9 Sep 2021 23:34:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f8b6389-02cd-4948-9a59-3656849e494a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631230469;
	bh=Ry3nsNInnIdfbYlD8M3MCLrSiNb9Qdq/qLmuQdF6Ogg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=H7IMKQ2L+G3oIZtrk34NQjFOnz4Jl1mL2eBaWdWWxM6dcPUbr7f+S5NmJ2Mwi7bqt
	 nrocuUix1dIRVDE7n0vu4X/r1IFMTFcZSafG6nyvIi5s+3uqvqujZThGMOI/91YbMD
	 DWb4mjeEeD4WAGHFJRl2G1QQIW6rXsMMCjK73S+FhDtKYMMbUHG8o3ziNvLGvl9lRn
	 4u+zm2A/RyRDhF9499uE+wm+KP41XT0cqc8BcmAuApmHosN7hsLHeCOywL0aFy//G2
	 x72wu1ITNTS6m1Chje4Ln3yW2G/76LuPV/jZ6hEd9FVzbYmrTr5Sq7o+LVnxwyxnvj
	 k6DIYkAMKvBtQ==
Date: Thu, 9 Sep 2021 16:34:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
In-Reply-To: <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
> space to the XEN memory.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>  xen/arch/arm/pci/Makefile          |  1 +
>  xen/arch/arm/pci/pci-host-zynqmp.c | 59 ++++++++++++++++++++++++++++++
>  2 files changed, 60 insertions(+)
>  create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index 6f32fbbe67..1d045ade01 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -3,3 +3,4 @@ obj-y += pci-access.o
>  obj-y += pci-host-generic.o
>  obj-y += pci-host-common.o
>  obj-y += ecam.o
> +obj-y += pci-host-zynqmp.o
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> new file mode 100644
> index 0000000000..fe103e3855
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -0,0 +1,59 @@
> +/*
> + * Copyright (C) 2020-2021 EPAM Systems
> + *
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>

Only one Copyright line per file is enough :-)

But actually all the Copyright lines with a name or a company name are
not really required or useful, as the copyright is noted in full details
in the commit messages (author and signed-off-by lines). I would remove
them all from the new files added by this series.


> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static const struct dt_device_match gen_pci_dt_match[] = {
> +    { .compatible = "xlnx,nwl-pcie-2.11",
> +      .data =       &pci_generic_ecam_ops },
> +    { },
> +};
> +
> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
> +{
> +    const struct dt_device_match *of_id;
> +    const struct pci_ecam_ops *ops;
> +
> +    of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);

This should be superfluous


> +    ops = (struct pci_ecam_ops *) of_id->data;
> +
> +    printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n",
> +            dt_node_full_name(dev), of_id->compatible);
> +
> +    return pci_host_common_probe(dev, ops, 2);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
> +.dt_match = gen_pci_dt_match,
> +.init = gen_pci_dt_init,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:41:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:41:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183724.332027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTg2-0007ey-Qp; Thu, 09 Sep 2021 23:41:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183724.332027; Thu, 09 Sep 2021 23:41:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTg2-0007er-NN; Thu, 09 Sep 2021 23:41:54 +0000
Received: by outflank-mailman (input) for mailman id 183724;
 Thu, 09 Sep 2021 23:41:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOTg1-0007el-3Y
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:41:53 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 66639481-6b48-4419-9e0b-64e51774c742;
 Thu, 09 Sep 2021 23:41:52 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4966760F13;
 Thu,  9 Sep 2021 23:41:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66639481-6b48-4419-9e0b-64e51774c742
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631230911;
	bh=Y5OqA2Lg6ctGni4k4QvswC173ZoiTboRcsdPJvJl+2c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=T8g5FENdsSTD46kbj8Ds79AfcTqjlT/RO3OoFsYphkf/8nALjVNLaAfnXBaO5lk+b
	 DZ/O1Ecx8ulUau+ljP/3WjP4adaP9I7NKn5bvuYKho54PPt7EuTbyym6dAbhX5K9+b
	 cndvkp77+hDX1stDeU9EsqmQHrfDz3PpNEQStVcTLr4JO7PqYV8ImS49WSWtICD6md
	 DJMnxTLG4jXjrtlPCEp90GusADpVFMt3PZ3+Ws/w5uhgiDwLzvMetFitL8/xp2iHeJ
	 VbNHs1Isa3fSSZZ0ZOXU3Ues7J4zI2/QoLauvxfU92boaoTpnaIYNYXq2r6ny6tSqq
	 1xnMBsFdl6zog==
Date: Thu, 9 Sep 2021 16:41:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 08/14] xen:arm: Implement pci access functions
In-Reply-To: <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> Implement generic pci access functions to read/write the configuration
> space.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/pci/pci-access.c      | 31 +++++++++++++++++++++++++++++-
>  xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++++++++++
>  xen/include/asm-arm/pci.h          |  2 ++
>  3 files changed, 51 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index f39f6a3a38..b94de3c3ac 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -72,12 +72,41 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>  static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>                                  unsigned int len)
>  {
> -    return ~0U;
> +    uint32_t val = GENMASK(0, len * 8);

This seems to be another default error value that it would be better to
define with its own macro


> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +    {
> +        printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n",
> +                sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn);

You are not actually printing sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn ?


> +        return val;
> +    }
> +
> +    if ( unlikely(!bridge->ops->read) )
> +        return val;
> +
> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);

Would it make sense to make the interface take a pci_sbdf_t directly
instead of casting to uint32_t and back?


> +    return val;
>  }
>  
>  static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
>                               unsigned int len, uint32_t val)
>  {
> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +    {
> +        printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n",
> +                sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn);

same here


> +        return;
> +    }
> +
> +    if ( unlikely(!bridge->ops->write) )
> +        return;
> +
> +    bridge->ops->write(bridge, (uint32_t) sbdf.sbdf, reg, len, val);

same here


>  }
>  
>  /*
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index c582527e92..62715b4676 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -261,6 +261,25 @@ err_exit:
>      return err;
>  }
>  
> +/*
> + * This function will lookup an hostbridge based on the segment and bus
> + * number.
> + */
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->segment != segment )
> +            continue;
> +        if ( (bus < bridge->bus_start) || (bus > bridge->bus_end) )
> +            continue;
> +        return bridge;
> +    }
> +
> +    return NULL;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 22866244d2..756f8637ab 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -20,6 +20,7 @@
>  #ifdef CONFIG_HAS_PCI
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> +#define PRI_pci "%04x:%02x:%02x.%u"
>  
>  /* Arch pci dev struct */
>  struct arch_pci_dev {
> @@ -86,6 +87,7 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>  void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                 uint32_t sbdf, uint32_t where);
>  
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  struct arch_pci_dev { };
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:46:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:46:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183732.332038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTk6-0008NS-HF; Thu, 09 Sep 2021 23:46:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183732.332038; Thu, 09 Sep 2021 23:46:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTk6-0008NL-Db; Thu, 09 Sep 2021 23:46:06 +0000
Received: by outflank-mailman (input) for mailman id 183732;
 Thu, 09 Sep 2021 23:46:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOTk5-0008NF-64
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:46:05 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 17de3832-11c8-11ec-b1ff-12813bfff9fa;
 Thu, 09 Sep 2021 23:46:03 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AA98F610C8;
 Thu,  9 Sep 2021 23:46:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17de3832-11c8-11ec-b1ff-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631231163;
	bh=qCHuprjemJfJf5P4BSEt414s3j0OwdMpC9rBbvIuo9c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bdjwFSME/PYNs67qR+Y/nzejh2okMKWu+IyHqyUjT5alRNfbs0WAglnHwYQwQDRq3
	 h+QV1Bv1kfWbWEN+oMFJtmr6ywgXvGkmmbXZ1k9FWYyQo/foSpsv6NW2PmHOo1wxrd
	 yMoTc5dhIJJpBbJUfJn0Mia2bN/pKEsCYV6vcwp6owyzij0bKOcylVnyYkUzjJp/Wc
	 UAPr5QZBs7PcXcLVksVuF0SIGssOqCubXQMnNZab5fYstoiQ8djGk9H1HHjaWMczcM
	 J+3l2bTHCPwv3dBIhJYUEEO2VXcppNh2KOEKTHZuRkvr+9OAb4FCyprQRD60odlJjp
	 J8edMbYR6B6Yg==
Date: Thu, 9 Sep 2021 16:46:01 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Julien Grall <julien@xen.org>, Rahul Singh <Rahul.Singh@arm.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 09/14] xen/arm: Add cmdline boot option "pci=on"
In-Reply-To: <f9c4837c-085d-d7a3-d79d-a77c6b5bf201@suse.com>
Message-ID: <alpine.DEB.2.21.2109091645270.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <e279636ea47b7d06056c2f70e76900b8d0b30ee9.1629366665.git.rahul.singh@arm.com> <731afb80-bb68-0c66-4f0f-341a46118770@xen.org> <DBFA6CEF-F8BC-4516-B609-99985F0C4698@arm.com> <b17f7970-ec4c-96e8-1a1c-0d3039d4d43e@xen.org>
 <f9c4837c-085d-d7a3-d79d-a77c6b5bf201@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 20 Aug 2021, Jan Beulich wrote:
> On 20.08.2021 16:34, Julien Grall wrote:
> > On 20/08/2021 13:19, Rahul Singh wrote:
> >>> On 19 Aug 2021, at 1:31 pm, Julien Grall <julien@xen.org> wrote:
> >>> On 19/08/2021 13:02, Rahul Singh wrote:
> >>>> Add cmdline boot option "pci=on" to enable/disable the PCI init during
> >>>> boot.
> >>>
> >>> I read this as "PCI" will be either disabled/enabled for the platform. Whereas, I think it will be used to decide whether Xen discover PCI and PCI passthrough is supported or not.
> >>
> >> Yes. I will modify the option to "pci-passthrough== <boolean>"
> >>>
> >>> Can you also clarify why a user would want to select "pci=off"?
> >>
> >> As pci-passthrough support emulate the PCI devices for DOM0 also, I thought if someone want to
> >> boot the DOM0 without emulating the PCI device in XEN and wants to have direct access to device.
> > 
> > Dom0 will always have direct access to the PCI device. The only 
> > difference is whether the access to the hostbridge and config space will 
> > be trapped by Xen. I expect the both to mainly happen during boot and 
> > therefore the overhead will be limited.
> > 
> >>
> >> I am ok to drop this patch if you feel adding the option is not required at all.
> > One of the reason I could see this option to be useful is to figure out 
> > if an issue occurs because of the hostbridge emulation. Yet, I am still 
> > not fully convinced adding an option is worth it.
> > 
> > Jan and others, any opinions?
> 
> Well, if there's a proper fallback, then why not allow using it in
> case of problems?

I think it would be good to have the option, if nothing else for
debugging.


From xen-devel-bounces@lists.xenproject.org Thu Sep 09 23:48:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 09 Sep 2021 23:48:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183738.332049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTmY-0000ar-RQ; Thu, 09 Sep 2021 23:48:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183738.332049; Thu, 09 Sep 2021 23:48:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOTmY-0000ak-OQ; Thu, 09 Sep 2021 23:48:38 +0000
Received: by outflank-mailman (input) for mailman id 183738;
 Thu, 09 Sep 2021 23:48:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ArGz=N7=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOTmX-0000ae-Fj
 for xen-devel@lists.xenproject.org; Thu, 09 Sep 2021 23:48:37 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9fc6ea4d-cb9c-4236-9527-01c4e7821c4d;
 Thu, 09 Sep 2021 23:48:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9ACED610E9;
 Thu,  9 Sep 2021 23:48:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9fc6ea4d-cb9c-4236-9527-01c4e7821c4d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631231316;
	bh=MjL8P5PlE4SMfGNTO/XyrbFDqheeCO80UsiIherSWho=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Wy5sH2NwjHmMGfDuQkVfv4Lec6vWtLvClUcXIrdgiDTpnFxe5ePS3Yyish+gRyCD1
	 YXbOZuQakwrdYvUacOzl2ycjCirpHhGoj9Ra6cM4Zvv1FHIRLoby99aiaaNsFB/wqS
	 +KSJz/pcNoLyto4KmQeXz0NA1WJqsw2+fm9Z24rXR+Wqs5m7n9xtquc3pcbo0pUC45
	 HVOgCfPxJWDtxtpbiBiH4g1GPiEwn0QSyxjn9mtsMnVhfCMvHuK0WbRUOImyoo7fsF
	 PvtEiCGynRuhT2bC0uI3sSzmz45rOollwOPw5vx08d9M3K0FdK6wfzeQjOxlCl5ztE
	 26apMdpaJLD6w==
Date: Thu, 9 Sep 2021 16:48:35 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 09/14] xen/arm: Add cmdline boot option "pci=on"
In-Reply-To: <e279636ea47b7d06056c2f70e76900b8d0b30ee9.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091646060.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <e279636ea47b7d06056c2f70e76900b8d0b30ee9.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> Add cmdline boot option "pci=on" to enable/disable the PCI init during
> boot.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/pci/pci.c | 30 ++++++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index d1c9cf997d..dc63bbc2a2 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -62,8 +62,38 @@ static void __init acpi_pci_init(void)
>  static inline void __init acpi_pci_init(void) { }
>  #endif
>  
> +static bool __initdata param_pci_enable;
> +
> +static int __init parse_pci_param(const char *arg)
> +{
> +    if ( !arg )
> +    {
> +        param_pci_enable = false;
> +        return 0;
> +    }
> +
> +    switch ( parse_bool(arg, NULL) )
> +    {
> +    case 0:
> +        param_pci_enable = false;
> +        return 0;
> +    case 1:
> +        param_pci_enable = true;
> +        return 0;
> +    }
> +
> +    return -EINVAL;
> +}
> +custom_param("pci", parse_pci_param);

Consider using boolean_param instead. It supports "on".


>  static int __init pci_init(void)
>  {
> +    /*
> +     * Enable PCI when has been enabled explicitly (pci=on)
> +     */
> +    if ( !param_pci_enable)
> +        return 0;
> +
>      if ( acpi_disabled )
>          dt_pci_init();
>      else
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:18:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:18:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183756.332066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUEx-0005FK-DU; Fri, 10 Sep 2021 00:17:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183756.332066; Fri, 10 Sep 2021 00:17:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUEx-0005FD-9n; Fri, 10 Sep 2021 00:17:59 +0000
Received: by outflank-mailman (input) for mailman id 183756;
 Fri, 10 Sep 2021 00:17:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUEw-0005F7-Lv
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:17:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8c60b5f0-11cc-11ec-b204-12813bfff9fa;
 Fri, 10 Sep 2021 00:17:57 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D4099611BF;
 Fri, 10 Sep 2021 00:17:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c60b5f0-11cc-11ec-b204-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233076;
	bh=L1NascCO+RqV2EXe7WgTVOgru+3jDwh0a4LVdwaZR/I=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=arFFHtckENB/uaxPUEVv7ZUYqMWFts/MhT7jppWwPMHuIbxB74zrMtB87W0Ghq9ft
	 ioBgf5/NtxpeX0JpCEJwCqr0B2oHqC8G+0qV1px9BWYrX9qjx+/uvO+5wA/1PKoEEt
	 Jfy9T0Ur7fqA+1Z3lDyM49Hwi5epZNQDckSf7SzW1ofg2qfqzHw/Ts2V1TwXhkrHcN
	 SkHx/4itK+7zPdE0VP2Qx/4Mu0tNhyef9H6LzL6+EfMiKrJe+VdtaHeUHbjxJdfi8f
	 v1vjyiRG8+mijL7rP7ClsUnnapzz+yItaqq6zqAdZu0iWK37SWqkiLPl45BTwMHg4L
	 ENRKxc5nK3AZQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.14 85/99] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:15:44 -0400
Message-Id: <20210910001558.173296-85-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910001558.173296-1-sashal@kernel.org>
References: <20210910001558.173296-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index c2ac319f11a4..96afadf9878e 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -64,7 +64,6 @@ static void cpu_bringup(void)
 	cr4_init();
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:20:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:20:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183763.332077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUH3-0006Ye-Pi; Fri, 10 Sep 2021 00:20:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183763.332077; Fri, 10 Sep 2021 00:20:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUH3-0006YX-Mg; Fri, 10 Sep 2021 00:20:09 +0000
Received: by outflank-mailman (input) for mailman id 183763;
 Fri, 10 Sep 2021 00:20:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUH3-0006YR-06
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:20:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 84d9b3e1-c335-46fb-a16b-40da901459d2;
 Fri, 10 Sep 2021 00:20:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1C9E96023D;
 Fri, 10 Sep 2021 00:20:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84d9b3e1-c335-46fb-a16b-40da901459d2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233207;
	bh=L1NascCO+RqV2EXe7WgTVOgru+3jDwh0a4LVdwaZR/I=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=l05YtZEfvVFThD13KAlYDT9vch5IiHqQgo5ekDYHAAyZTJpeN2mX3amqS3vQM2dYL
	 8/+dZf3v1Kppe5voLzSR5kjcnrlgfa5xJNUD7vhVHXlKs+bfZGInzFblBHKD7rMvXH
	 HAIQmtbLQF0vZD6q/9u9onfMwSgc8NU/GR1G5LiBy6Sgz1JzYKqpZqdyfDmKN+fbyx
	 luVvS0+vMQSLbK5yV9BOBQ15tNFu65LmkGAlrPTVlVDm4q1nlJdjQ9HK86Eum8Di0Y
	 IYHIW/OXj32oRnCrvLz4SsmwWyRD7VCsKZh2apHqUI2q8XaJrN6G97CHBkklmwqGtk
	 /d+f3cBDrjasw==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.13 76/88] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:18:08 -0400
Message-Id: <20210910001820.174272-76-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910001820.174272-1-sashal@kernel.org>
References: <20210910001820.174272-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index c2ac319f11a4..96afadf9878e 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -64,7 +64,6 @@ static void cpu_bringup(void)
 	cr4_init();
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:21:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183770.332088 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUIU-0007Pj-4c; Fri, 10 Sep 2021 00:21:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183770.332088; Fri, 10 Sep 2021 00:21:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUIU-0007Pc-1c; Fri, 10 Sep 2021 00:21:38 +0000
Received: by outflank-mailman (input) for mailman id 183770;
 Fri, 10 Sep 2021 00:21:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUIS-0007PU-L6
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:21:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3b1c97d-a4c6-4169-b5f8-282d0232db61;
 Fri, 10 Sep 2021 00:21:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C4B52604DC;
 Fri, 10 Sep 2021 00:21:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3b1c97d-a4c6-4169-b5f8-282d0232db61
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233295;
	bh=L1NascCO+RqV2EXe7WgTVOgru+3jDwh0a4LVdwaZR/I=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=le8JpYTYX7fEO1b8fyXgaZSfOYm6pv+I74B3y145J2LTwFcvJQboMg4FdHCHlf8Sz
	 JX9OL+DQspZpvs3RuwBiXaJmZZImio38QXGbzvmP2FiXGO+RiuhgKWiyrdqvZntCcp
	 atXYYbcl7myrM22wfHZyFTMXm9P5BRuaPdheLtSAWydprnGMz++gdJ8iXQK+E82a97
	 JRWspK+h4nPQz9ABH+QX5C86kkTwDqnCFtPF4YQHTqgeJb/utqMec3RUbAy3ruy38W
	 WgrfdVVxOLUjzKubIS+3fCBt7+nVRfP32LR6h8HDHJCCVvP5cx3idVqgp4FP/wTGKV
	 FKjN8jVTKd4Ig==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.10 49/53] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:20:24 -0400
Message-Id: <20210910002028.175174-49-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910002028.175174-1-sashal@kernel.org>
References: <20210910002028.175174-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index c2ac319f11a4..96afadf9878e 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -64,7 +64,6 @@ static void cpu_bringup(void)
 	cr4_init();
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:22:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:22:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183776.332098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUJL-00084H-Fe; Fri, 10 Sep 2021 00:22:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183776.332098; Fri, 10 Sep 2021 00:22:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUJL-00084A-Ca; Fri, 10 Sep 2021 00:22:31 +0000
Received: by outflank-mailman (input) for mailman id 183776;
 Fri, 10 Sep 2021 00:22:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUJJ-00083w-FN
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:22:29 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fc833408-2555-407d-bfd7-1e5e3ae8a562;
 Fri, 10 Sep 2021 00:22:28 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A0018610A3;
 Fri, 10 Sep 2021 00:22:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc833408-2555-407d-bfd7-1e5e3ae8a562
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233348;
	bh=B3+dapiH7ZqT0zNupi/gPeO0vO8/7rY0FDBnebcTA1c=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=ESZg825KvV5tGsZ0dn3orW51NieRVTYim0DUkFfMPslxm/8Q7zPksDkuSpsuaO2UN
	 WMAD3uthyDVN4+1/OrCYmGj7DBflKywPW8bRZD/FaTHYHeXbx3AqawJdQGmmY6uJX/
	 QDC3Bkb0sKH/on9vNNviNaeF75Xw0nKLe8+tXOHgoU24R18YWsEDS6/QIcfMxqB05l
	 hy0F1Jx0wzzKn/dQZ9WBiLsxdR4z7ZBLxPD5EgySjysfQFHb5NH/10nkp7rWkhova2
	 AJmS9/47AZNj8OFC7alwNAyo5otn1j7OGKo6bTP8Va9LqHlb0wuuwXpht5bPZ+w2Ig
	 bvDZhaAQhTyHQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.4 34/37] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:21:39 -0400
Message-Id: <20210910002143.175731-34-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910002143.175731-1-sashal@kernel.org>
References: <20210910002143.175731-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index 0cebe5db691d..b87c6403b5bd 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -61,7 +61,6 @@ static void cpu_bringup(void)
 	cr4_init();
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:23:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:23:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183782.332110 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUJu-0000Dn-PD; Fri, 10 Sep 2021 00:23:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183782.332110; Fri, 10 Sep 2021 00:23:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUJu-0000Dg-M9; Fri, 10 Sep 2021 00:23:06 +0000
Received: by outflank-mailman (input) for mailman id 183782;
 Fri, 10 Sep 2021 00:23:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUJs-0000DC-Sj
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:23:04 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 43510a62-11cd-11ec-b204-12813bfff9fa;
 Fri, 10 Sep 2021 00:23:04 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D2057611BD;
 Fri, 10 Sep 2021 00:23:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43510a62-11cd-11ec-b204-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233383;
	bh=ISzSvm4C6f8lHwgz+pYqhFqWZ8I8lIYQVNbzl6VCqi8=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=a5MczqkOv12YbXLfY+4UES3v7pDgRRD1dU8G94Qd7iHMhMSSSq8cmGwT+rXcnRt8m
	 /Mc5s268pWlkd4bq/VhK/PT4cHc6Dwo8ij7i3AjMTp9pA7ak+OKVDc6lo8YwQdVEGG
	 SXzdnBzR4qHIbj0mmYPkiBqRnjFTdqblDnis4MdC9yTyX+tkSDKHCeVBeekwsJKqcZ
	 c/OCy7kpOFXYakCa3kchfeiujZE+xS8+oa/y7aZfAPpxb/FCay3TBPYBBG8gDranT1
	 D9gGC0CIMdD1WlhldaPoPg9/h6oG9vWZgjPW7j2M8bKPBvhhDX2gYas3mVI8MIe7Or
	 biPeLnvF+4N8w==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 4.19 22/25] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:22:30 -0400
Message-Id: <20210910002234.176125-22-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910002234.176125-1-sashal@kernel.org>
References: <20210910002234.176125-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index 41fd4c123165..7b6e56703bb9 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -59,7 +59,6 @@ static void cpu_bringup(void)
 
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:23:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:23:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183788.332120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUKM-0000oI-1i; Fri, 10 Sep 2021 00:23:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183788.332120; Fri, 10 Sep 2021 00:23:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUKL-0000oB-Ui; Fri, 10 Sep 2021 00:23:33 +0000
Received: by outflank-mailman (input) for mailman id 183788;
 Fri, 10 Sep 2021 00:23:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1So/=OA=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mOUKL-0000ng-99
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:23:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6ef35839-5a5a-4b13-b38b-7a900243c4d6;
 Fri, 10 Sep 2021 00:23:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 60CB7611BF;
 Fri, 10 Sep 2021 00:23:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ef35839-5a5a-4b13-b38b-7a900243c4d6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233412;
	bh=2MjKKLz7JzePmBVcQDgPEuCsbqkT4iHdNk9yelbTGWk=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=CCsx7xpCr6g88miLZiMbIAMqW/KeFbGynaKDu/fWPrm9nB4OTJM5+18BO73W6XUAM
	 N6B3hGIBLx8tT39U6qO2SJwAEJUpwPmw2nJzZxUh5Tq4FabdXG9MV6yu5GJo7goXnh
	 3kwR9u9TFP88/S34OmmWGrs3eRnMyBjoHkakS0mN9JB+p+A1oB/0WnfAd6ZzKJw+mo
	 OR/WXihDPMlVjqNlN2VmLX4bELQeD8K4X9YoBAO6UXyFDESmiZsaV1g+xGL19VW6ig
	 dj+3VdHVzJPFgJlLXnmUiNI0JHN0YD2xNwAzA+ZPx2VuIPEt+JPzJHkG7jWUieO9vG
	 ZJAN/noPlfI7A==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 4.14 16/19] xen: remove stray preempt_disable() from PV AP startup code
Date: Thu,  9 Sep 2021 20:23:06 -0400
Message-Id: <20210910002309.176412-16-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210910002309.176412-1-sashal@kernel.org>
References: <20210910002309.176412-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 58e636039b512697554b579c2bb23774061877f5 ]

In cpu_bringup() there is a call of preempt_disable() without a paired
preempt_enable(). This is not needed as interrupts are off initially.
Additionally this will result in early boot messages like:

BUG: scheduling while atomic: swapper/1/0/0x00000002

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20210825113158.11716-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp_pv.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index f779d2a5b04c..44468d6fa188 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -59,7 +59,6 @@ static void cpu_bringup(void)
 
 	cpu_init();
 	touch_softlockup_watchdog();
-	preempt_disable();
 
 	/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */
 	if (!xen_feature(XENFEAT_supervisor_mode_kernel)) {
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:26:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:26:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183796.332132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUNO-0001a5-Jk; Fri, 10 Sep 2021 00:26:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183796.332132; Fri, 10 Sep 2021 00:26:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUNO-0001Zy-G8; Fri, 10 Sep 2021 00:26:42 +0000
Received: by outflank-mailman (input) for mailman id 183796;
 Fri, 10 Sep 2021 00:26:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOUNN-0001Zs-2b
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:26:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c3d07b28-11cd-11ec-b204-12813bfff9fa;
 Fri, 10 Sep 2021 00:26:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A85316103E;
 Fri, 10 Sep 2021 00:26:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3d07b28-11cd-11ec-b204-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631233599;
	bh=HWFJxwazt43LjU8kGOp2RBB87gZB63e4RK/M9WgRSIg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KKZnIynXfGQ0X85V/DtEnYaT6NGzYgqSc7U2FL3W/MUy35pUTss6sdpY0JAOjw9Sv
	 Dl1Qxdf2Uf3mFYrMr+EpNIz6R1dmmVN7QlxFClG9bNnLzHWBukQkmrg/JWA7mRH8jO
	 GFQaj21kHsH0ZucHsBplSefkIAA3bxvLwkdJjBK17Nu4IN+2mbbp+M5CIaFvcKXSmi
	 fTK5hlsDftdm5mlSPd7/NrsOzuJ+mw1Rrw2/Ca861+A1La2gUJTrl39F1khhnkWyw6
	 +P7aDF8J6PzM9wZkxss4N6ySBv2riDr+8rvOyziJBrwSy3qyNIgRmrP2mOdkXNr0xJ
	 yyKP1+j0rpLgQ==
Date: Thu, 9 Sep 2021 17:26:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Jan Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
In-Reply-To: <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-468448648-1631233261=:10523"
Content-ID: <alpine.DEB.2.21.2109091721050.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-468448648-1631233261=:10523
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109091721051.10523@sstabellini-ThinkPad-T480s>

On Thu, 19 Aug 2021, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> access is added to the Xen to emulate the PCI devices config space.

This is done just for device discovery, right?

Although it is currently not implemented (and I am not asking to
implement it now, I am only trying to understand the architecture), it
would be possible to discover all PCI devices just by walking down the
PCI hierarchy by ourselves in Xen (no Dom0 interactions) given that we
have an ECAM driver.

I take that would be the way to implement PCI support for Dom0less?

 
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.
>
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/Makefile         |  1 +
>  xen/arch/arm/domain.c         |  4 ++
>  xen/arch/arm/vpci.c           | 96 +++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vpci.h           | 37 ++++++++++++++
>  xen/drivers/passthrough/pci.c |  7 +++
>  xen/drivers/vpci/Makefile     |  3 +-
>  xen/drivers/vpci/header.c     |  2 +
>  xen/include/asm-arm/domain.h  |  5 +-
>  xen/include/asm-arm/pci.h     |  8 +++
>  xen/include/public/arch-arm.h |  4 ++
>  10 files changed, 165 insertions(+), 2 deletions(-)
>  create mode 100644 xen/arch/arm/vpci.c
>  create mode 100644 xen/arch/arm/vpci.h
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 0e14a5e5c8..7cdce684a4 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -7,6 +7,7 @@ obj-y += platforms/
>  endif
>  obj-$(CONFIG_TEE) += tee/
>  obj-$(CONFIG_HAS_PCI) += pci/
> +obj-$(CONFIG_HAS_VPCI) += vpci.o
>  
>  obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>  obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..d99c653626 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -40,6 +40,7 @@
>  #include <asm/vtimer.h>
>  
>  #include "vuart.h"
> +#include "vpci.h"
>  
>  DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
>  
> @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
>      if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
>          goto fail;
>  
> +    if ( (rc = domain_vpci_init(d)) != 0 )
> +        goto fail;
> +
>      return 0;
>  
>  fail:
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> new file mode 100644
> index 0000000000..da8b1ca13c
> --- /dev/null
> +++ b/xen/arch/arm/vpci.c
> @@ -0,0 +1,96 @@
> +/*
> + * xen/arch/arm/vpci.c
> + * Copyright (c) 2021 Arm Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +#include <xen/sched.h>
> +#include <asm/mmio.h>
> +
> +/* Do some sanity checks. */
> +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
> +{
> +    /* Check access size. */
> +    if ( len != 1 && len != 2 && len != 4 && len != 8 )
> +        return false;
> +
> +    /* Check that access is size aligned. */
> +    if ( (reg & (len - 1)) )
> +        return false;
> +
> +    return true;
> +}
> +
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                          register_t *r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    uint32_t data = 0;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12);
> +    reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3));
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 1;
> +
> +    data = vpci_read(sbdf, reg, size);
> +
> +    memcpy(r, &data, size);
> +
> +    return 1;
> +}
> +
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                           register_t r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    uint32_t data = r;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12);
> +    reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3));
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 1;
> +
> +    vpci_write(sbdf, reg, size, data);
> +
> +    return 1;
> +}
> +
> +static const struct mmio_handler_ops vpci_mmio_handler = {
> +    .read  = vpci_mmio_read,
> +    .write = vpci_mmio_write,
> +};
> +
> +int domain_vpci_init(struct domain *d)
> +{
> +    if ( !has_vpci(d) )
> +        return 0;
> +
> +    register_mmio_handler(d, &vpci_mmio_handler,
> +                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> +
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> new file mode 100644
> index 0000000000..8a093bb705
> --- /dev/null
> +++ b/xen/arch/arm/vpci.h
> @@ -0,0 +1,37 @@
> +/*
> + * xen/arch/arm/vpci.h
> + * Copyright (c) 2021 Arm Ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ARCH_ARM_VPCI_H__
> +#define __ARCH_ARM_VPCI_H__
> +
> +#ifdef CONFIG_HAS_VPCI
> +int domain_vpci_init(struct domain *d);
> +#else
> +static inline int domain_vpci_init(struct domain *d)
> +{
> +    return 0;
> +}
> +#endif
> +
> +#endif /* __ARCH_ARM_VPCI_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index c23c8cb06b..56e261e9bd 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -767,6 +767,13 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>      else
>          iommu_enable_device(pdev);
>  
> +#ifdef CONFIG_ARM
> +    ret = vpci_add_handlers(pdev);
> +    if ( ret ) {
> +        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
> +        goto out;
> +    }
> +#endif
>      pci_enable_acs(pdev);
>  
>  out:
> diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
> index 55d1bdfda0..1a1413b93e 100644
> --- a/xen/drivers/vpci/Makefile
> +++ b/xen/drivers/vpci/Makefile
> @@ -1 +1,2 @@
> -obj-y += vpci.o header.o msi.o msix.o
> +obj-y += vpci.o header.o
> +obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index ba9a036202..f8cd55e7c0 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
>       * FIXME: punching holes after the p2m has been set up might be racy for
>       * DomU usage, needs to be revisited.
>       */
> +#ifdef CONFIG_HAS_PCI_MSI
>      if ( map && !rom_only && vpci_make_msix_hole(pdev) )
>          return;
> +#endif
>  
>      for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
>      {
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..d742b94bd6 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -262,7 +262,10 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
>  
>  #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
>  
> -#define has_vpci(d)    ({ (void)(d); false; })
> +/* For X86 VPCI is enabled and tested for PVH DOM0 only but
> + * for ARM we enable support VPCI for guest domain also.
> + */
> +#define has_vpci(d) ({ (void)(d); IS_ENABLED(CONFIG_HAS_VPCI); })
>  
>  #endif /* __ASM_DOMAIN_H__ */
>  
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 756f8637ab..c58152de80 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -27,6 +27,14 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/* Arch-specific MSI data for vPCI. */
> +struct vpci_arch_msi {
> +};
> +
> +/* Arch-specific MSI-X entry data for vPCI. */
> +struct vpci_arch_msix_entry {
> +};
> +
>  /*
>   * struct to hold the mappings of a config space window. This
>   * is expected to be used as sysdata for PCI controllers that
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 64a2ca30da..0a9749e768 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -422,6 +422,10 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
>  #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
>  
> +/* VPCI ECAM mappings */
> +#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
> +#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)

Isn't 256MB a bit too small? Usually PCI comes with two ranges:

- a small range below 4GB like this one
- an additional large range above 4GB, around 1GB or larger

We are missing the second larger range?
--8323329-468448648-1631233261=:10523--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 00:52:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 00:52:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183804.332142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUlv-0005Z6-C7; Fri, 10 Sep 2021 00:52:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183804.332142; Fri, 10 Sep 2021 00:52:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUlv-0005Yz-8i; Fri, 10 Sep 2021 00:52:03 +0000
Received: by outflank-mailman (input) for mailman id 183804;
 Fri, 10 Sep 2021 00:52:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOUlu-0005Yt-2e
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 00:52:02 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4d43fae8-11d1-11ec-b205-12813bfff9fa;
 Fri, 10 Sep 2021 00:52:00 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0FDBD6105A;
 Fri, 10 Sep 2021 00:51:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d43fae8-11d1-11ec-b205-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631235119;
	bh=xs7DuKIsDRL4I4JKDAcKjHSGgLUEZ3zDFut0cG4XkZs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Jg4Hq3ebU2YqIekLRE76/Ax23ohR5LCnK7CaV8fXaGrMWMW7Ovm7e97ROHFOfYwUh
	 +967EdY4lI6gxx2MMsO1RHFFn9yXtGhKzVNKtBT1i8Bk+uFNrew20YgVXI/QYfBXvH
	 pQdA87gxBXp679l4zf/16/M7e7LcWP8kGPdzPEF1Tj7bFCiRT0H7fEsnDn9IVA2X++
	 QTAXncX8ZEKSrvdd7I4LGEhrLzD/4dgnu80G+mwP+9JllBEOSsG8Im8vNNgDibuyvT
	 fIWidvTMueZS1xYP5DbEo5aSKyoqh//a+EbACGxPxoK6FTyuS1str6fGZmSwLaFKBr
	 h0Iwm5OhFnHBw==
Date: Thu, 9 Sep 2021 17:51:58 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
In-Reply-To: <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091742210.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> libxl will create an emulated PCI device tree node in the device tree to
> enable the guest OS to discover the virtual PCI during guest boot.
> Emulated PCI device tree node will only be created when there is any
> device assigned to guest.
> 
> A new area has been reserved in the arm guest physical map at
> which the VPCI bus is declared in the device tree (reg and ranges
> parameters of the node).
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  tools/libs/light/libxl_arm.c          | 109 ++++++++++++++++++++++++++
>  tools/libs/light/libxl_types.idl      |   1 +
>  tools/xl/xl_parse.c                   |   2 +
>  xen/include/public/arch-arm.h         |  11 +++
>  xen/include/public/device_tree_defs.h |   1 +
>  5 files changed, 124 insertions(+)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e00..a091e97e76 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
>      return fdt_property(fdt, "reg", regs, sizeof(regs));
>  }
>  
> +static int fdt_property_values(libxl__gc *gc, void *fdt,
> +        const char *name, unsigned num_cells, ...)
> +{
> +    uint32_t prop[num_cells];
> +    be32 *cells = &prop[0];
> +    int i;
> +    va_list ap;
> +    uint32_t arg;
> +
> +    va_start(ap, num_cells);
> +    for (i = 0 ; i < num_cells; i++) {
> +        arg = va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, name, prop, sizeof(prop));
> +}
> +
> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
> +                                    unsigned addr_cells,
> +                                    unsigned size_cells,
> +                                    unsigned num_regs, ...)
> +{
> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
> +    be32 *cells = &regs[0];
> +    int i;
> +    va_list ap;
> +    uint64_t arg;
> +
> +    va_start(ap, num_regs);
> +    for (i = 0 ; i < num_regs; i++) {
> +        /* Set the memory bit field */
> +        arg = va_arg(ap, uint64_t);
> +        set_cell(&cells, 1, arg);

Shouldn't this be uint32_t given that it is 1 cell exactly?


> +        /* Set the vpci bus address */
> +        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, addr_cells , arg);
> +
> +        /* Set the cpu bus address where vpci address is mapped */
> +        set_cell(&cells, addr_cells, arg);
> +
> +        /* Set the vpci size requested */
> +        arg = size_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, size_cells,arg);
                                       ^ space


> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
> +}
> +
>  static int make_root_properties(libxl__gc *gc,
>                                  const libxl_version_info *vers,
>                                  void *fdt)
> @@ -668,6 +720,57 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
>      return 0;
>  }
>  
> +static int make_vpci_node(libxl__gc *gc, void *fdt,
> +        const struct arch_info *ainfo,
> +        struct xc_dom_image *dom)
> +{
> +    int res;
> +    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
> +    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
> +    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "device_type", "pci");
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
> +    if (res) return res;
> +
> +    res = fdt_property_values(gc, fdt, "bus-range", 2, 0,17);
                                                            ^ space


> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 3);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", 2);
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "status", "okay");
> +    if (res) return res;
> +
> +    res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +        GUEST_ROOT_SIZE_CELLS, 2,
> +        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE,
> +        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
> +        GUEST_VPCI_PREFETCH_MEM_SIZE);
> +    if (res) return res;
> +
> +    res = fdt_property_values(gc, fdt, "msi-map", 4, 0, GUEST_PHANDLE_ITS,
> +                              0, 0x10000);
> +    if (res) return res;

I agree with Julien that we shouldn't add it now if it is not working.

One question: what about legacy interrupts? If they are supported,
shouldn't we have interrupts and interrupt-parent properties?


> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
>  static const struct arch_info *get_arch_info(libxl__gc *gc,
>                                               const struct xc_dom_image *dom)
>  {
> @@ -971,6 +1074,9 @@ next_resize:
>          if (info->tee == LIBXL_TEE_TYPE_OPTEE)
>              FDT( make_optee_node(gc, fdt) );
>  
> +        if (libxl_defbool_val(info->arch_arm.vpci))
> +            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
> +
>          if (pfdt)
>              FDT( copy_partial_fdt(gc, fdt, pfdt) );
>  
> @@ -1189,6 +1295,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
>      /* ACPI is disabled by default */
>      libxl_defbool_setdefault(&b_info->acpi, false);
>  
> +    /* VPCI is disabled by default */
> +    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
> +
>      if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
>          return;
>  
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 3f9fff653a..78b1ddf0b8 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>  
>      ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                 ("vuart", libxl_vuart_type),
> +                               ("vpci", libxl_defbool),
>                                ])),
>      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                                ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd5..ffafbeffb4 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>          }
>          if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
>              libxl_defbool_set(&b_info->u.pv.e820_host, true);
> +        if (d_config->num_pcidevs)
> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>      }
>  
>      if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 0a9749e768..01d13e669e 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -426,6 +426,17 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
>  #define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
>  
> +/* PCI-PCIe memory space types */
> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000)
> +#define GUEST_VPCI_ADDR_TYPE_MEM          xen_mk_ullong(0x02000000)
> +
> +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
> +#define GUEST_VPCI_PREFETCH_MEM_ADDR  xen_mk_ullong(0x900000000)
> +#define GUEST_VPCI_MEM_ADDR           xen_mk_ullong(0x20000000)
> +
> +#define GUEST_VPCI_PREFETCH_MEM_SIZE      xen_mk_ullong(0x2000000000)
> +#define GUEST_VPCI_MEM_SIZE               xen_mk_ullong(0x04000000)

Now I get it: GUEST_VPCI_ECAM_BASE was just for the config space, not
the aperture. The apertures are starting at 0x20000000 and 0x900000000.

Speaking of which, isn't GUEST_VPCI_MEM_SIZE a bit too small? There
could be PCI devices with a <4GB requirement. Maybe 256MB?


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 01:00:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 01:00:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183811.332154 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUuF-0003SI-88; Fri, 10 Sep 2021 01:00:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183811.332154; Fri, 10 Sep 2021 01:00:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOUuF-0003RO-4I; Fri, 10 Sep 2021 01:00:39 +0000
Received: by outflank-mailman (input) for mailman id 183811;
 Fri, 10 Sep 2021 01:00:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOUuD-0002nF-IV
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 01:00:37 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81f0d8ce-11d2-11ec-b206-12813bfff9fa;
 Fri, 10 Sep 2021 01:00:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B8D5F610C9;
 Fri, 10 Sep 2021 01:00:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81f0d8ce-11d2-11ec-b206-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631235635;
	bh=ijbpxVEv2WqN6U7QSvEPX/gA8Hrre+cnWPsQKbMPbxE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=qjwp1h+uEAie4pfQh0jQiw2tOs+4vmQfugGcegniEN4Ybr1+EgSNlkrHgIeALz6Ba
	 3y4hqE8DhSgO084XehcFOuA1ICUCMSRoFnGcqoPDqbNPAU3pNZ5GKxk2Dl8uvG4fvL
	 J3u8ESfJuweXdhCjBye7RwsKT3CIsQiKmkhxKkH3KWGrJTKnP3uz8Ue9pioT6D64Zz
	 qYCmX757iyR0ThuB/IERkm28MNoWD+HwxuWio4YFvz+0hcViMeBwrayuHJjhy/dz7i
	 n+j2exdRA7MVRT2oHAnvuE6JmwQbIqGs5QVfKXadNoH4qaiwwlsMjBg/JXz1v1YjK/
	 qrUNTL3v98fNw==
Date: Thu, 9 Sep 2021 18:00:35 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 14/14] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
In-Reply-To: <0b979ccab6a4c9b2070748709f737f41fa108a42.1629366665.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109091758290.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <0b979ccab6a4c9b2070748709f737f41fa108a42.1629366665.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 19 Aug 2021, Rahul Singh wrote:
> If the property is not present in the device tree node for host bridge,
> XEN while creating the dtb for hwdom will create this property and
> assigns the already allocated segment to the host bridge
> so that XEN and linux will have the same segment for the host bridges.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
>  xen/arch/arm/domain_build.c        | 18 ++++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
>  xen/include/asm-arm/pci.h          |  3 +++
>  3 files changed, 42 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 6c86d52781..e0cf2ff19d 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -581,6 +581,24 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>              return res;
>      }
>  
> +#ifdef CONFIG_HAS_PCI
> +    if ( dt_device_type_is_equal(node, "pci") )
> +    {
> +        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
> +        {
> +            uint16_t segment;
> +
> +            res = pci_get_host_bridge_segment(node, &segment);
> +            if ( res < 0 )
> +                return res;
> +
> +            res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment);
> +            if ( res )
> +                return res;
> +        }
> +    }
> +#endif

If param_pci_enable is false it might be possible that Xen didn't
allocate a segment. In that case, we should just let Linux do whatever
it wants in terms of segment allocation. So I think the code here should
not return error if param_pci_enable is false.
returning an error instead.


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 01:48:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 01:48:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183820.332171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOVeD-0002Nz-Ve; Fri, 10 Sep 2021 01:48:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183820.332171; Fri, 10 Sep 2021 01:48:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOVeD-0002Nq-Pu; Fri, 10 Sep 2021 01:48:09 +0000
Received: by outflank-mailman (input) for mailman id 183820;
 Fri, 10 Sep 2021 01:48:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOVeC-0002Ng-Jm; Fri, 10 Sep 2021 01:48:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOVeC-0001yH-Cl; Fri, 10 Sep 2021 01:48:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOVeC-0002pL-3M; Fri, 10 Sep 2021 01:48:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOVeC-00010z-2f; Fri, 10 Sep 2021 01:48:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/vM6S0ai14d2RI40Z3IWIVHlKT8kizsRpE/4+eqonj4=; b=ArgGucsFaB5voduxA8nIaLotTq
	qPM6PSNCk3fAx2YoJK+1uoBf/cUiGYGHt54SwLQQPMaEMnbWIKoplI26qidckBrA2f2ySse29HM44
	i8D6UoVbsQ+8IA6QJrrrMETW+3svWHL70CiQ82OVwKEElhMns9XiqSyaPQecICQc01RQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164896-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.12-testing test] 164896: regressions - FAIL
X-Osstest-Failures:
    xen-4.12-testing:test-amd64-amd64-xl-qcow2:guest-saverestore.2:fail:regression
    xen-4.12-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=47193c78887734c2f95a50125684bf81419f1565
X-Osstest-Versions-That:
    xen=35ba323378d05509f2e0dc049520e140be183003
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 01:48:08 +0000

flight 164896 xen-4.12-testing real [real]
flight 164928 xen-4.12-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164896/
http://logs.test-lab.xenproject.org/osstest/logs/164928/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qcow2    18 guest-saverestore.2      fail REGR. vs. 164489

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164489
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164489
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164489
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164489
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164489
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164489
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164489
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 xen                  47193c78887734c2f95a50125684bf81419f1565
baseline version:
 xen                  35ba323378d05509f2e0dc049520e140be183003

Last test of basis   164489  2021-08-25 17:57:16 Z   15 days
Testing same since   164886  2021-09-08 13:06:02 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 47193c78887734c2f95a50125684bf81419f1565
Author: Jan Beulich <jbeulich@suse.com>
Date:   Wed Sep 8 15:03:11 2021 +0200

    gnttab: deal with status frame mapping race
    
    Once gnttab_map_frame() drops the grant table lock, the MFN it reports
    back to its caller is free to other manipulation. In particular
    gnttab_unpopulate_status_frames() might free it, by a racing request on
    another CPU, thus resulting in a reference to a deallocated page getting
    added to a domain's P2M.
    
    Obtain a page reference in gnttab_map_frame() to prevent freeing of the
    page until xenmem_add_to_physmap_one() has actually completed its acting
    on the page. Do so uniformly, even if only strictly required for v2
    status pages, to avoid extra conditionals (which then would all need to
    be kept in sync going forward).
    
    This is CVE-2021-28701 / XSA-384.
    
    Reported-by: Julien Grall <jgrall@amazon.com>
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>
    master commit: eb6bbf7b30da5bae87932514d54d0e3c68b23757
    master date: 2021-09-08 14:37:45 +0200
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:37:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:37:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183837.332197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWQ9-0000jQ-49; Fri, 10 Sep 2021 02:37:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183837.332197; Fri, 10 Sep 2021 02:37:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWQ9-0000jJ-1D; Fri, 10 Sep 2021 02:37:41 +0000
Received: by outflank-mailman (input) for mailman id 183837;
 Fri, 10 Sep 2021 02:37:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MJ4u=OA=wdc.com=prvs=8803762df=Damien.LeMoal@srs-us1.protection.inumbo.net>)
 id 1mOWQ7-0000jD-6S
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:37:39 +0000
Received: from esa2.hgst.iphmx.com (unknown [68.232.143.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0f1c2412-11e0-11ec-b209-12813bfff9fa;
 Fri, 10 Sep 2021 02:37:37 +0000 (UTC)
Received: from mail-dm6nam08lp2048.outbound.protection.outlook.com (HELO
 NAM04-DM6-obe.outbound.protection.outlook.com) ([104.47.73.48])
 by ob1.hgst.iphmx.com with ESMTP; 10 Sep 2021 10:37:35 +0800
Received: from DM6PR04MB7081.namprd04.prod.outlook.com (2603:10b6:5:244::21)
 by DM6PR04MB5372.namprd04.prod.outlook.com (2603:10b6:5:10f::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Fri, 10 Sep
 2021 02:37:34 +0000
Received: from DM6PR04MB7081.namprd04.prod.outlook.com
 ([fe80::7c0e:3e95:80d3:7a70]) by DM6PR04MB7081.namprd04.prod.outlook.com
 ([fe80::7c0e:3e95:80d3:7a70%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 02:37:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f1c2412-11e0-11ec-b209-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com;
  t=1631241457; x=1662777457;
  h=from:to:cc:subject:date:message-id:references:
   content-transfer-encoding:mime-version;
  bh=BWtePkt9PqofwLtfWCZGDbYW9YD8b8ZbKaVXyrz3aAg=;
  b=bKyladshmfNX1VPZPXPNFT7+wQsDrfLj8vZI2exqJDWdXbKQgdrJByXv
   lBpQ7PmGYDWJ6pLYjyNo0BRdDifMs6QPTarwkIzz6JZ1INdEDOWmZzDnU
   dc/LcYW6gww/5GBoSkaxp3nlngjElPjxeAsWYrMZIT39p9TWqcdMJEK7W
   gmXXmG9unrh74X/MnsaGTS69L1qUWiOpA2ByGVHcIqvqnazoQaX6806WR
   naj8WMc7GTrq2GBjs8po95E/THkQWNsXZfe7IudYckLJxRQaMUwXOyLpO
   9btKskcynjwbh+EFrxkYhI3E69DulJ0bs/r8zgpGZDaM0Tb3Qs/kTkYcN
   Q==;
X-IronPort-AV: E=Sophos;i="5.85,282,1624291200"; 
   d="scan'208";a="283438094"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dJm6Pj0ylW4qumBod0yYvP4JZqaH+MpFb796W0DF1friqlQZfwBS8fE26325SlgdSsIPPHJFzwtWhYIg4ERpHHexSOSS+1VydCHwFReK0TAJaeFYQLSVKwTj+HkP5ArRRMQafjM1E97AZS5aFYijBAs63g1+1Lv87avNy/N+ag9ToD2fPFRHIe0HYmYjsvuEkhJfTEchW4AT6MuvfqCLfasV6yyniD7naiN15gg4DDBwPX0sQQsklxG8n0luWx8Ish6vOBvBUXedo9rHJRiUFmNay5VIRpM8xSOTjeOcp+iVMVCObTOAQw4vPtt9/l9lddYFmttdqngKNrmlu/TTxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=a932QBPzLDfxarVXbsNqNaEU7H/Ny5RsOuK78mSbhUY=;
 b=WKrb47leA9cEri7Y+QJ3+SgAkFjIRWDSQOhQ4Ymh9RnErAGhQZ0DAaRh9MR4S3IVKGj5zk/5944RySanJaJ3QTdTOZ4HjxfrG+I43sDHjB0ITsIlyRh2ugE4w/NY3wTcgNwJIZGyLArzlmMR5Dq6mnBAC0buyr+P3pPRcojpNbz6T+k+x6u9BJ0qnzSHrrIqDS6fupNMf89qqxayzp78/J9yYXIzEUJkmml63gO3sNHyVdxsjVwKWPrsY9OWi6g+59nM3x/wIe8KodOj/Jcqfyy2ccTs9sPVQHWgcB+gsJtqsfTkoEYGJhTxof+rrW2JWOFVcyeBshfFPB5R+Mk3gQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass
 header.d=wdc.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=a932QBPzLDfxarVXbsNqNaEU7H/Ny5RsOuK78mSbhUY=;
 b=oONdsTbHnm1vOQLV2hkVhfNLt1Gt4vD6EF3F7Dva4KA7ZH/C5xyUBatzLrRDFaNwqRxULadaWdSMQ5Pr3kqthjILa1Aq4Q9zJwwKhRx6C5jLSjEvDXL73+/Denff3ndMoDdzu4AVEaLVsD964ksr/ZqH2kNkMVYWT/tikpsGi9I=
From: Damien Le Moal <Damien.LeMoal@wdc.com>
To: Xu Wang <vulab@iscas.ac.cn>, "konrad.wilk@oracle.com"
	<konrad.wilk@oracle.com>, "roger.pau@citrix.com" <roger.pau@citrix.com>,
	"axboe@kernel.dk" <axboe@kernel.dk>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] xen-blkback: Remove needless request_queue NULL pointer
 check
Thread-Topic: [PATCH] xen-blkback: Remove needless request_queue NULL pointer
 check
Thread-Index: AQHXpewIpomVCHmkEEmflQIfj6rk6w==
Date: Fri, 10 Sep 2021 02:37:33 +0000
Message-ID:
 <DM6PR04MB7081D2E353283692759CECFAE7D69@DM6PR04MB7081.namprd04.prod.outlook.com>
References: <20210910022457.22963-1-vulab@iscas.ac.cn>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: iscas.ac.cn; dkim=none (message not signed)
 header.d=none;iscas.ac.cn; dmarc=none action=none header.from=wdc.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f8351f02-c6b5-45d8-ab4b-08d97403f1cc
x-ms-traffictypediagnostic: DM6PR04MB5372:
x-microsoft-antispam-prvs:
 <DM6PR04MB5372D497BE8495BDA0973CD9E7D69@DM6PR04MB5372.namprd04.prod.outlook.com>
wdcipoutbound: EOP-TRUE
x-ms-oob-tlc-oobclassifiers: OLM:2089;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 Olnaf6l0wJxiCrkZd1P+rmP1Sk6v+hljmxOhTi+2//rk23Eowa5mj+V4lazZFjizZSW9udnjzyHiJMHMHt5QBv21IKvSc6iORRK57a7XUVSOl+XveJJoAub+tOsVCjy2wZ+1DMAbGxq1BwzBQaWrmi2o+dZR7eFmyKyUN7mmDP6cJRQu8pnMQfu3+MmedpRaK+B0JDDoyt7b0Hih1ZriSmo5/T0U8BqE/BOyUpV0vvKV28b58YAnZ+5MtduiI/3mJFdCVmprxSVf2Y93d7j9BDECDoK7jYMJHOQfddDglqgg/TEwjH9xHh/UxcplWAajJL2y2BQlJuZaTtk33v1BeGjn0zTAoE167qmIP8kJNsT8pjnJBbrvcEUSCtYnt96IWsH4JcgaQKbmFRhdW4MkHLMCqI4VN3TyKSY76UX45mRkNEwcZ6Xu6oS1tBvT3G+YjVFGMIw5oGfJVEz9u1/4QVIL2Y1AINAaZ06M7tdHVwK0DqukCFsyg6n/d62GTbBfRgourvkrOvWK9nhGI/02DVK92uPzXXDGdbhL+eM0NwwzjAPouKzvV7JqjgixlcK8nBhOw0k0+vp7onsW8aQ7i0YofqpqNKeDvrKyjWKNHQmtMBzrIJcuRjeijtTzHUKmUJJC4eTGAFHCZCo34dLaDtAoliafYAK9/g/ThStarkmnF7hQwrc707YWY5V1yMnQWz+9/EFQpZfRL++4nIsadw==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR04MB7081.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(396003)(376002)(136003)(39860400002)(5660300002)(55016002)(38100700002)(186003)(54906003)(110136005)(86362001)(52536014)(66476007)(8676002)(4744005)(38070700005)(9686003)(83380400001)(7696005)(478600001)(33656002)(71200400001)(6506007)(4326008)(316002)(64756008)(66446008)(66556008)(66946007)(53546011)(8936002)(91956017)(76116006)(2906002)(122000001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?xeAtoRMah3qLQB1367F3KDHQBBt2S9K6sQw0QoizCTVB3o35LiTp+JsbbEHQ?=
 =?us-ascii?Q?M3ks9w170AbvbfqfPNlvBxvR80v2Cy1UpHbOMRiYQ1CsLO1X7zuGsGX+cdtF?=
 =?us-ascii?Q?hsv3OQ9MP3wHCm6lXpnDCo37nCChqanA+WrWG4lbcfQkOWLnPwdeznB1eb5A?=
 =?us-ascii?Q?pqx9QlJ7CpZAt20oREfJXU/DfF6UI0BtpJYq9+pcAHfXCuTnwLKG3Zt8tmBz?=
 =?us-ascii?Q?39V8kurg21WL+KjdiYjx4sFxiR+fVIqqwr2MwD3VYlsxrLFx8eLz3gwFo8eq?=
 =?us-ascii?Q?oA8jz3H+tWMHSX/kzZC9q+n4WzhU7i+VTRagZBQUjAJkAJfAU5p6KL3fuhpp?=
 =?us-ascii?Q?f1k7M3aQffd04gZek8Kz9Gy8VZTtzXqGOUSWhcdexTeAlzLEDtOzqQHcD4zF?=
 =?us-ascii?Q?WhUA0ytC9RJooOKXHCvvyinf7vmZtGrVw/B/GBUh4Dk8ai4/N/KFqWLyoqPq?=
 =?us-ascii?Q?9LxXojKujNcJ0VtFCCJDYfg8y5QqBXLEe3PvCp0WzIta8Wa4U95heR+Kq+IO?=
 =?us-ascii?Q?vKZPXZFJImvYlJ7dSfdkjWjpHb9VGOLMFkQya2ejeB6sI0hHb5jZis5+NCy/?=
 =?us-ascii?Q?GX7jnBXmfvaNRRsH004koDrTV12M1uZmPYbcwSnN8/OcO+M7xFzGgCzdbNGv?=
 =?us-ascii?Q?9JEMqtqwoxWdENx0UCTYLfzE63v40iMpVcVKMR+eJc57VhK7fNPIOADnuPAR?=
 =?us-ascii?Q?gVhw6DuoUip3W25VXiCGOmBC5YyrjHA7nxmFC3zKF7z6cOZhSLipwYgy249T?=
 =?us-ascii?Q?D1x462NWmBXXp/nlMeIQ5J6BZ8CY7lJaHeYTUdu34rykeJqk5C9w1bWEG7uu?=
 =?us-ascii?Q?swHUjFvHo3HSYJMt6Gb9Dez7hiP8xAssBL6gOUVQhH43zWGZRWH93C/EJ2H/?=
 =?us-ascii?Q?E9/FnjcqMMDwEdTlRc1UevpW91YtdjrNcoeJQYfiG3WYkvpcneC2qWTC1KM8?=
 =?us-ascii?Q?A6GaJ4ohX0JldHtrOZAiZTrFYO5+kURkDtZX612NroHzMHVcVCc35msY7ARR?=
 =?us-ascii?Q?S9sqIbwkRdG8lj5emk3URedlrM3sVYYN2zSCs+su3Uxtp7g3nXl2IfTv0r9B?=
 =?us-ascii?Q?9HUB+FVa+0JO4yp6FG+2HB5Zx6xCkTg6C2n3fpYr+xqLygd92smV07Vj7qSZ?=
 =?us-ascii?Q?mQnGPvXWTrQqcnrF/KZPmZfUPLHFZnouF51k50TmYg2NgaRapKN7Ez+XGUfd?=
 =?us-ascii?Q?50rxGhsqxM6MHoHqmbmk+KFdCfIJ4o/aDbWAloKmY695BNQrW7r6muGOuUf8?=
 =?us-ascii?Q?mA2ztVaOB+eaYE8UpQUCJ01rf/LMlxfuPOeB3zxZLig2h81dYDMd5atatD4o?=
 =?us-ascii?Q?5BzBZnxRUuYWwSOEcKKgmE6DRxEI9oV6ShW1d33L20Fd5B/HCMZQvpqq8Uci?=
 =?us-ascii?Q?XXkLw604nlot2VHzgDgVUeDzpHMqOj/sM316VLgmRGONjCB8QQ=3D=3D?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: wdc.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR04MB7081.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f8351f02-c6b5-45d8-ab4b-08d97403f1cc
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 02:37:33.9036
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: pF7HjWnuwerALbxpOaiWw1qasGv50ELlt80EJfm8h3Fl+v5N4qCTSMDd66gD3Q5ZD5PVp/d5qAipYiCVMD+tPQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB5372

On 2021/09/10 11:32, Xu Wang wrote:=0A=
> The request_queue pointer returned from bdev_get_queue() shall=0A=
> never be NULL, so the null check is unnecessary, just remove it.=0A=
> =0A=
> Signed-off-by: Xu Wang <vulab@iscas.ac.cn>=0A=
> ---=0A=
>  drivers/block/xen-blkback/xenbus.c | 2 +-=0A=
>  1 file changed, 1 insertion(+), 1 deletion(-)=0A=
> =0A=
> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkba=
ck/xenbus.c=0A=
> index 33eba3df4dd9..f05132b9ddbf 100644=0A=
> --- a/drivers/block/xen-blkback/xenbus.c=0A=
> +++ b/drivers/block/xen-blkback/xenbus.c=0A=
> @@ -516,7 +516,7 @@ static int xen_vbd_create(struct xen_blkif *blkif, bl=
kif_vdev_t handle,=0A=
>  		vbd->type |=3D VDISK_REMOVABLE;=0A=
>  =0A=
>  	q =3D bdev_get_queue(bdev);=0A=
> -	if (q && test_bit(QUEUE_FLAG_WC, &q->queue_flags))=0A=
> +	if test_bit(QUEUE_FLAG_WC, &q->queue_flags)=0A=
=0A=
Missing parenthesis. Did you even compile this ?=0A=
=0A=
=0A=
>  		vbd->flush_support =3D true;=0A=
>  =0A=
>  	if (q && blk_queue_secure_erase(q))=0A=
=0A=
And why not change this one too ?=0A=
=0A=
> =0A=
=0A=
=0A=
-- =0A=
Damien Le Moal=0A=
Western Digital Research=0A=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:50:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183847.332214 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWcp-0003EJ-Bb; Fri, 10 Sep 2021 02:50:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183847.332214; Fri, 10 Sep 2021 02:50:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWcp-0003EC-8L; Fri, 10 Sep 2021 02:50:47 +0000
Received: by outflank-mailman (input) for mailman id 183847;
 Fri, 10 Sep 2021 02:50:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OOxz=OA=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mOWcn-0003E6-Tt
 for xen-devel@lists.xen.org; Fri, 10 Sep 2021 02:50:45 +0000
Received: from mail-pg1-x533.google.com (unknown [2607:f8b0:4864:20::533])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2d094094-e35f-4dc6-8f25-3bdbfeb5e8c2;
 Fri, 10 Sep 2021 02:50:43 +0000 (UTC)
Received: by mail-pg1-x533.google.com with SMTP id w7so500049pgk.13
 for <xen-devel@lists.xen.org>; Thu, 09 Sep 2021 19:50:43 -0700 (PDT)
Received: from laputa (p784a2304.tkyea130.ap.so-net.ne.jp. [120.74.35.4])
 by smtp.gmail.com with ESMTPSA id u24sm3732106pfm.81.2021.09.09.19.50.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 Sep 2021 19:50:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d094094-e35f-4dc6-8f25-3bdbfeb5e8c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=SyqNjTfXjvn5KMtNqk1SycXkvMuo8jFzXwCGc//gy2A=;
        b=dXMF46b+INFby5XLocLnw07Iql5h1tQuPSb1hmzlIrcKrr2T9prHxo1QdYlpak2HAN
         ADIqRCy23U90hwFnwPKyxNw/0CYp4vykPDR3mEV8ysBd/GxYfbNC+GOwr6qz71WkbSAR
         8vmnmpJWovBAC7w1VaMTZg7f+0YxUggAegqqY2Bqsx3TyvIanbNc8T+UW3qH14C7ijWg
         nER12MDDZeE2FlFoyMHfZsv3+uXyzeYODKN1M8oHUzPrqJPN1ktm+dk4M0dpepYEhIJT
         b5/kDNw6LIObZ5JD826O2byaCat3dCh29OURsojwIhth9ipP+KZObJCGpMyCn3c7ROhQ
         wKsw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=SyqNjTfXjvn5KMtNqk1SycXkvMuo8jFzXwCGc//gy2A=;
        b=RYGgisYr0EVBtx4C1VD9TLzxBBctcaaG+nNC01QoF3Cn0/KH+B+LyWJ3K7Qo3kPg3n
         IcdR1jp3WruLmto2CCUa4akNoCwwv2IjHLuK+ZGuexQozJDPT3me1N1PLqmjEHwaYvYI
         mLAxusuoeKm5is9OJOCp/TDpz59uOThmkpuTobpwsV2qoTTWbK8o9fMvVHH++zvEXXI1
         dyl8UmRmB15ZbMwADkUZGv6CBKezOgfsZuQlb2BIBSywrxa5GqGNj9eI3rvHwb3AaDNy
         UQCkB7ExYg3RTeln24/ZQJK4SDUH6dsmofNy0ocoEknGNOKY7efxdTAueQtLYK1t24rl
         zNzg==
X-Gm-Message-State: AOAM531I4ZiCIzVMID+IW9U0UD/LDO8bt64FAtZ8u/k2VJAFVo0YTfdB
	bCTwYMosdqd63KwBH3yVQUz32Q==
X-Google-Smtp-Source: ABdhPJwf8Mldf8MGcdJrItBBr12CMt3FqyFuCQZs/DtoJ/nw7hKXlNdjizY88FhDskrTlFGfNhIe8g==
X-Received: by 2002:a05:6a00:26cb:b0:408:f3b8:bdad with SMTP id p11-20020a056a0026cb00b00408f3b8bdadmr6132023pfw.48.1631242242691;
        Thu, 09 Sep 2021 19:50:42 -0700 (PDT)
Date: Fri, 10 Sep 2021 11:50:34 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: Alex Benn??e <alex.bennee@linaro.org>, Wei Chen <Wei.Chen@arm.com>,
	Paul Durrant <paul@xen.org>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	virtio-dev@lists.oasis-open.org,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Juergen Gross <jgross@suse.com>, Julien Grall <julien@xen.org>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Xen-devel <xen-devel@lists.xen.org>,
	Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Elena Afanasova <eafanasova@gmail.com>,
	James McKenzie <james@bromium.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Rich Persaud <persaur@gmail.com>,
	Daniel Smith <dpsmith@apertussolutions.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	eric chanudet <eric.chanudet@gmail.com>,
	Roger Pau Monn?? <roger.pau@citrix.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210910025034.GC42777@laputa>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain>
 <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa>
 <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
 <CACMJ4GaJyAnguzAEH87DSNN_+GhEa5jRbw11hVj-yWMAXx8V7w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CACMJ4GaJyAnguzAEH87DSNN_+GhEa5jRbw11hVj-yWMAXx8V7w@mail.gmail.com>

On Mon, Sep 06, 2021 at 07:41:48PM -0700, Christopher Clark wrote:
> On Sun, Sep 5, 2021 at 7:24 PM AKASHI Takahiro via Stratos-dev <
> stratos-dev@op-lists.linaro.org> wrote:
> 
> > Alex,
> >
> > On Fri, Sep 03, 2021 at 10:28:06AM +0100, Alex Benn??e wrote:
> > >
> > > AKASHI Takahiro <takahiro.akashi@linaro.org> writes:
> > >
> > > > Alex,
> > > >
> > > > On Wed, Sep 01, 2021 at 01:53:34PM +0100, Alex Benn??e wrote:
> > > >>
> > > >> Stefan Hajnoczi <stefanha@redhat.com> writes:
> > > >>
> > > >> > [[PGP Signed Part:Undecided]]
> > > >> > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote:
> > > >> >> > Could we consider the kernel internally converting IOREQ
> > messages from
> > > >> >> > the Xen hypervisor to eventfd events? Would this scale with
> > other kernel
> > > >> >> > hypercall interfaces?
> > > >> >> >
> > > >> >> > So any thoughts on what directions are worth experimenting with?
> > > >> >>
> > > >> >> One option we should consider is for each backend to connect to
> > Xen via
> > > >> >> the IOREQ interface. We could generalize the IOREQ interface and
> > make it
> > > >> >> hypervisor agnostic. The interface is really trivial and easy to
> > add.
> > > >> >> The only Xen-specific part is the notification mechanism, which is
> > an
> > > >> >> event channel. If we replaced the event channel with something
> > else the
> > > >> >> interface would be generic. See:
> > > >> >>
> > https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52
> > > >> >
> > > >> > There have been experiments with something kind of similar in KVM
> > > >> > recently (see struct ioregionfd_cmd):
> > > >> >
> > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/
> > > >>
> > > >> Reading the cover letter was very useful in showing how this provides
> > a
> > > >> separate channel for signalling IO events to userspace instead of
> > using
> > > >> the normal type-2 vmexit type event. I wonder how deeply tied the
> > > >> userspace facing side of this is to KVM? Could it provide a common FD
> > > >> type interface to IOREQ?
> > > >
> > > > Why do you stick to a "FD" type interface?
> > >
> > > I mean most user space interfaces on POSIX start with a file descriptor
> > > and the usual read/write semantics or a series of ioctls.
> >
> > Who do you assume is responsible for implementing this kind of
> > fd semantics, OSs on BE or hypervisor itself?
> >
> > I think such interfaces can only be easily implemented on type-2
> > hypervisors.
> >
> > # In this sense, I don't think rust-vmm, as it is, cannot be
> > # a general solution.
> >
> > > >> As I understand IOREQ this is currently a direct communication between
> > > >> userspace and the hypervisor using the existing Xen message bus. My
> > > >
> > > > With IOREQ server, IO event occurrences are notified to BE via Xen's
> > event
> > > > channel, while the actual contexts of IO events (see struct ioreq in
> > ioreq.h)
> > > > are put in a queue on a single shared memory page which is to be
> > assigned
> > > > beforehand with xenforeignmemory_map_resource hypervisor call.
> > >
> > > If we abstracted the IOREQ via the kernel interface you would probably
> > > just want to put the ioreq structure on a queue rather than expose the
> > > shared page to userspace.
> >
> > Where is that queue?
> >
> > > >> worry would be that by adding knowledge of what the underlying
> > > >> hypervisor is we'd end up with excess complexity in the kernel. For
> > one
> > > >> thing we certainly wouldn't want an API version dependency on the
> > kernel
> > > >> to understand which version of the Xen hypervisor it was running on.
> > > >
> > > > That's exactly what virtio-proxy in my proposal[1] does; All the
> > hypervisor-
> > > > specific details of IO event handlings are contained in virtio-proxy
> > > > and virtio BE will communicate with virtio-proxy through a virtqueue
> > > > (yes, virtio-proxy is seen as yet another virtio device on BE) and will
> > > > get IO event-related *RPC* callbacks, either MMIO read or write, from
> > > > virtio-proxy.
> > > >
> > > > See page 8 (protocol flow) and 10 (interfaces) in [1].
> > >
> > > There are two areas of concern with the proxy approach at the moment.
> > > The first is how the bootstrap of the virtio-proxy channel happens and
> >
> > As I said, from BE point of view, virtio-proxy would be seen
> > as yet another virtio device by which BE could talk to "virtio
> > proxy" vm or whatever else.
> >
> > This way we guarantee BE's hypervisor-agnosticism instead of having
> > "common" hypervisor interfaces. That is the base of my idea.
> >
> > > the second is how many context switches are involved in a transaction.
> > > Of course with all things there is a trade off. Things involving the
> > > very tightest latency would probably opt for a bare metal backend which
> > > I think would imply hypervisor knowledge in the backend binary.
> >
> > In configuration phase of virtio device, the latency won't be a big matter.
> > In device operations (i.e. read/write to block devices), if we can
> > resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue
> > is
> > how efficiently we can deliver notification to the opposite side. Right?
> > And this is a very common problem whatever approach we would take.
> >
> > Anyhow, if we do care the latency in my approach, most of virtio-proxy-
> > related code can be re-implemented just as a stub (or shim?) library
> > since the protocols are defined as RPCs.
> > In this case, however, we would lose the benefit of providing "single
> > binary"
> > BE.
> > (I know this is is an arguable requirement, though.)
> >
> > # Would we better discuss what "hypervisor-agnosticism" means?
> >
> Is there a call that you could recommend that we join to discuss this and
> the topics of this thread?

Stratos call?
Alex should have more to say.

-Takahiro Akashi


> There is definitely interest in pursuing a new interface for Argo that can
> be implemented in other hypervisors and enable guest binary portability
> between them, at least on the same hardware architecture, with VirtIO
> transport as a primary use case.
> 
> The notes from the Xen Summit Design Session on VirtIO Cross-Project BoF
> for Xen and Guest OS, which include context about the several separate
> approaches to VirtIO on Xen, have now been posted here:
> https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00472.html
> 
> Christopher
> 
> 
> 
> > -Takahiro Akashi
> >
> >
> >


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:52:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:52:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183853.332224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWek-00049o-OR; Fri, 10 Sep 2021 02:52:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183853.332224; Fri, 10 Sep 2021 02:52:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWek-00049h-LH; Fri, 10 Sep 2021 02:52:46 +0000
Received: by outflank-mailman (input) for mailman id 183853;
 Fri, 10 Sep 2021 02:52:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWej-00049b-PL
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:52:45 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.63]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2b8db406-11e2-11ec-b209-12813bfff9fa;
 Fri, 10 Sep 2021 02:52:43 +0000 (UTC)
Received: from DB8P191CA0014.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:130::24)
 by PR3PR08MB5593.eurprd08.prod.outlook.com (2603:10a6:102:84::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Fri, 10 Sep
 2021 02:52:42 +0000
Received: from DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:130:cafe::e) by DB8P191CA0014.outlook.office365.com
 (2603:10a6:10:130::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT047.mail.protection.outlook.com (10.152.21.232) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:52:41 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Fri, 10 Sep 2021 02:52:41 +0000
Received: from 860f5e1d0fde.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 829D0E2F-8CD4-4B8F-98FE-785383FBBCED.1; 
 Fri, 10 Sep 2021 02:52:35 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 860f5e1d0fde.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:35 +0000
Received: from AS9PR06CA0180.eurprd06.prod.outlook.com (2603:10a6:20b:45c::27)
 by AS8PR08MB6727.eurprd08.prod.outlook.com (2603:10a6:20b:393::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:34 +0000
Received: from AM5EUR03FT061.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::2c) by AS9PR06CA0180.outlook.office365.com
 (2603:10a6:20b:45c::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:34 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT061.mail.protection.outlook.com (10.152.16.247) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:34 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:39 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b8db406-11e2-11ec-b209-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TAwD+O7ALEnhbk9hWFjHxBehgLgxUIQfpBy8konpdig=;
 b=vcpkQ5OrrnCLq8S9OD7vKXVb1hyiiLAVfwcz/fuXQnU1UW24uGj6belg5iSm5Dc6Q+r2v91jIeAe0nLPy2psEJATcLp6BjbxqaqnfG7j/Bo7jD3t9fT0PnBg0FWp0LxPOpWOkMl27b8nzhWOnVmKw2puendDV+mhvxRb03TUef4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6939813991210d9f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=A3SoljXG8pYWWqWR7pgTW8kz3Ub+IMw71/XrpPT3fshdi/gNWHKgIdl9boO205Ulgs5kSwK/+Z8DMXVOXdCaWa4QvwVd+KbZd2iPLC82CnIepGQT/INjpdwtG/0AzEmg+8krL9j9z8AvNb8kL/HvyL6Tqcdf/UHCozMX8wI7PvuNoCZM0cQCFl5kMU1v28JnikD7n6iRj86QNZJ9d6i92LUR2AkGn3MG383huVjDGbnYarbaNkk++zwfPuH6XmlPTf2t68tyFSiLA6oKf1H8dBPGonmNLyP7JYXqF4E5/jc8j9R78GLAVyK/sszwGrFw3gm74aC1pQ+h144XIoTB9Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TAwD+O7ALEnhbk9hWFjHxBehgLgxUIQfpBy8konpdig=;
 b=BGbmjdQlkP88xgJXyBihQNh8OiFTJVC4Nz3G1gZ/zhsqOyLOIi3exlsQA9tPYdQFWPgdk0cEk3JxUkm+MZ0VJp4YyumTrMjPVh+/aschMa9ep4G6g3RHwBu2GyrwJUZ6pw3ubzY3RfC/nHR4sN5G4Y8JRZFBdFI+bIaiLcwOT1PvAhg2vwNupeT+OcOpKENpFwG8ceaIkYHm1oIa8wuiVOJ/TOnnF6eImS/VTdYUVbnyyrJlTq8P+oujhqZvFAg6qYjlm4IskWzXNHC1DVSgMdudiTPns4sMq0pwWKLA1zuhzLJ7kq5pqbnGdQ8hIb4FdukWuGaRL1KL8ttnPiO7FA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TAwD+O7ALEnhbk9hWFjHxBehgLgxUIQfpBy8konpdig=;
 b=vcpkQ5OrrnCLq8S9OD7vKXVb1hyiiLAVfwcz/fuXQnU1UW24uGj6belg5iSm5Dc6Q+r2v91jIeAe0nLPy2psEJATcLp6BjbxqaqnfG7j/Bo7jD3t9fT0PnBg0FWp0LxPOpWOkMl27b8nzhWOnVmKw2puendDV+mhvxRb03TUef4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 0/7] Domain on Static Allocation
Date: Fri, 10 Sep 2021 02:52:08 +0000
Message-ID: <20210910025215.1671073-1-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: af1dc25e-75e1-4c25-d778-08d974060f00
X-MS-TrafficTypeDiagnostic: AS8PR08MB6727:|PR3PR08MB5593:
X-Microsoft-Antispam-PRVS:
	<PR3PR08MB559392DE98419EB4BD3692EBF7D69@PR3PR08MB5593.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 dYqj5cf2hLFpHgRSKc7eVqRGJtwgTSBkj8c2mqwlJaUVJPLvhV9hcn1vjNnygWE2L0JA/do79rHlNJR1J2kWR2WEbJIhEb1CvbLpwH+xLsyPmIJ4IffSI8O7mIIwQxyLJJi0/E32ee7PusSBwBM+q7ilmnEz9CN5UANbYj3S1N9P8xnm46MKioAokgQm5yJ4qcY3gMgVPkGcAI57NPdkvkusAtZMwVvevdi8CJZytujNrG1vvN+j84MYxWwRsbR5Y8DSMY60/4Wiu8I9vpF+ezultxuO9T9iM69V2cB6lcMFWehWYNsxfH5UOdQASL4Wp9i7ba7V08QmobmHjENeHChUoSAd+Bm/Pz43GQHjJN2SWHhd7+hhPXmdPlItUyXPV1FYrF0QS+8i4t4O/BFCTflX+1f4k0dl7TFabPV5yD2AaYmSIx9K6Tj/Y+YgKv2CZCaFAVjZLvTuvrPdhAEfV9+29bPl10Z7rxo0y8hH+uG9VgZ+9lIcQ182mYbitPtEy6iCTspTF5rKqxRGeQn19/yPY/AHwVdkIiP2Tn9XZSwxr9l2t9q2Pi6KK2+WKc2n24Arc5+tgYdBYUNyZZ1y8dRqRe66wdsZXmMhM30RYeH6RSfmTuR/piOsi0QjYybFUm8T4uayN40oYvw9ZCe61OZZXipxyYCTsVBhzdqneokxaW5asGA3/8A4BLdZ/I2s1TIpbzKYNp4cH8o2Whsg8/LVwFDU6pEgm65cKWyHtTLNWzlyQpFQRwNRW8Am67PP/9NpLKJJsxcAIUAkYbeVzNrYbotExixNfvZUZRTwqODAfAd7xnlyBt1ZQWEofAa6
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39860400002)(376002)(346002)(136003)(396003)(46966006)(36840700001)(186003)(316002)(110136005)(5660300002)(36756003)(44832011)(54906003)(8936002)(82310400003)(336012)(1076003)(478600001)(26005)(2906002)(966005)(8676002)(36860700001)(6666004)(70586007)(70206006)(4326008)(82740400003)(47076005)(2616005)(7696005)(83380400001)(81166007)(426003)(86362001)(356005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6727
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e54dafaf-b35d-4c85-3cfe-08d974060a59
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LxqPFEzCW7XTHEvN0E3sQt6FVHn1g/89hWpmKR5oJxXPAzzHqMJCkOlLNswQLQOS0qguH1uObPMoO/htTuS3xf0c0bOl0fIvYQh/aiLPLALxm5ewyiZLDcQR79WPIXaX1rqvxfI0j90LAjbrypSB/MabiWXrWhYgCL7Mp6tfhxRj73WSZjiUMcguMXDAWzKLER/IIZ3cPFgBQnDlTeJuizHrBRirGrD0T9FyG8hhb/9d6IM5RJJDBB6OJieJN1RKF2MU+4BBDLjwRyfxBE233upR08jUJx+JYhtOE2S35s0nh0chTDgw02qjfsrUPPyTqQzr1ms9e3C0US7WpBpbCrmRppBo3CjIqucQRHljcK7KiijI7DNP/rhHBIu6QUdoVNNSPS9AZBSgUHKO9x9OkCAOGrAyNvCJrtnz+uFYDTN8PDJunHZ8KWELt8/JFWu9hnlQFmOrgqAnVYjh9bhLQfVbpIxMNT8EeSUBkwXaKYSkaOhz+mVtOZAr5WJHLWK4B7xF3P3MoQMlxcYTHGaLdsxSQxvYVCemWokTturYk2DG8DmPDm2z/y05xW8weh1nQpG+p6CtzpzTDV8mC4u3xoy/R+ZsbNhu1JsKwnchTFUibP7NcLNPkEcZuwYcFre8L9s7VZQTUJ9FJNYpknGS1EzyEnoKfwL9waEgyF9NeYakVav+LhquRVp+vAt5tMCPsdQFv+/JNGz2NfekX440g20l279gRYZMsCaEKYmLRKMA84x+oEGqSKiGCZ2JKpQtZLhNWjKXnxK53ULWoub9qHKofOoK3mHJAYi4XaSNC+4=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(39860400002)(136003)(376002)(346002)(36840700001)(46966006)(107886003)(316002)(7696005)(478600001)(6666004)(336012)(82310400003)(1076003)(426003)(83380400001)(186003)(44832011)(5660300002)(70206006)(966005)(70586007)(47076005)(36756003)(86362001)(36860700001)(8676002)(8936002)(82740400003)(4326008)(2906002)(81166007)(110136005)(54906003)(2616005)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:41.9895
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: af1dc25e-75e1-4c25-d778-08d974060f00
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5593

Static Allocation refers to system or sub-system(domains) for which memory
areas are pre-defined by configuration using physical address ranges.

Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
beginning, shall never go to heap allocator or boot allocator for any use.

Memory can be statically allocated to a domain using the property "xen,static-
mem" defined in the domain configuration. The number of cells for the address
and the size must be defined using respectively the properties
"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".

The property 'memory' is still needed and should match the amount of memory
given to the guest. Currently, it either comes from static memory or lets Xen
allocate from heap. *Mixing* is not supported.

The static memory will be mapped in the guest at the usual guest memory
addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

This Patch Serie only talks about Domain on Static Allocation.

Looking into related [design link](
https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg00882.html)
for more details.

The whole design is about Static Allocation and 1:1 direct-map, and this
Patch Serie only covers parts of it, which are Domain on Static Allocation.
Other features will be delievered through different patch series.
---
changes in v2:
 - consider reserved-memory binding and use phandle to refer(still in
discussion)
 - remove unused reserved field in struct page_info, unused helper
page_get_reserved_owner and page_set_reserved_owner
 - introduce CONFIG_STATIC_ALLOCATION to avoid bringing dead codes in
other archs.
 - extract common code from free_heap_pages and free_staticmem_pages
 - introduce new interface assign_pages_nr
 - change pfn-named to mfn-named
 - remove meaningless MEMF_no_owner case
 - leave zone concept out of DMA limitation check
 - take care of concurrency issue on static memory allocation
 - ensure the consistency when `xen,static-mem` and `memory` are both defined
 - fix the scalability issue in allocate_static_memory
 - allocate static memory when parse
---
changes in v3:
- use "xen,static-mem" property to be compatible with System Sevice Tree
in the future
- introduce new helper device_tree_get_meminfo
- extract common codes for dealing with reserved memory stored in
bootinfo
- rename from "free_page" to "mark_page_free"
- remove non-trivial page_to_mfn conversion in "mark_page_free" due to
pdx compression, and let the MFN passed in
- let all switch-cases shared in "mark_page_free"
- change CONFIG_STATIC_ALLOCATION to CONFIG_STATIC_MEMORY
- put init_staticmem_pages in setup_mm
- rename assign_pages_nr to assign_pages
- alter the order of assign_pages parameters to help backporting
- change name from alloc_staticmem_pages/alloc_domstatic_pages to
acquire_staticmem_pages and acquire_domstatic_pages.
- remove hunks' #ifdef-ary by introducing PGC_reserved = 0
- remove DMA restriction
- "memory" property shall be mandatory
- rename allocate_static_bank_memory to append_static_memory_to_bank
- infer the next GFN from the bank information in append_static_memory_to_bank
- simplify the code of double loop in allocate_static_memory
---
changes in v4:
- move the option CONFIG_STATIC_MEMORY to common code, and with Arm
"select"ing it
- replace round_pg{down,up}() with PFN_DOWN()/PFN_UP()
- in all cases where order-0 pages get passed, prefer using new assign_pages
to pass literal 1
- reconstruct the order of assign_pages parameters
- moving tlb/cache flush outside of the locked region, considering XSA-364
and reducing the amount of work happening with the heap_lock held
- remove MEMF_no_refcount case
- make acquire_staticmem_pages/acquire_domstatic_pages being __init
---
changes in v5:
- check the node using the Xen domain binding whether contains the property
"xen,static-mem", not the property itself
- add "rc = ..." to get the error propagated
- introduce new field "bool xen_domain", then static memory shall be also stored
as reserved memory(bootinfo.reserved_mem), but being bind to one
specific Xen domain node.
- make CONFIG_STATIC_MEMORY user selectable and gated by UNSUPPORTED.
- wrap all static-allocation-related codes with CONFIG_STATIC_MEMORY
even in arm-specific file.
- make bank_start/bank_end type of mfn_t, and rename bank_size to
bank_pages.
- Having both functions assign_pages/assign_page with similar parameter
arrangement
- bundle all the functions for static allocation in a single place
- return an error and revert the changes, when the page is not free
and reserved.
- check the MFN is valid for every page and also add a comment to warn
that this function needs to be reworked if used outside of boot.
- use less of mfn_to_page/page_to_mfn
- use ASSERT_UNREACHABLE() to also check that the two flags are clear
- pass the start MFN first and then the number of pages in both
acquire_staticmem_pages and acquire_domstatic_pages
- make acquire_domstatic_pages() to return an errno
- don't split comment over multi-line (even they are more than 80 characters)
- simply use dt_find_property(node, "xen,static-mem", NULL) to tell
whether using allocate_static_memory, and add error comment when
"xen,static-mem" is used but CONFIG_STATIC_MEMORY is not enabled.
- exporting p2m_insert_mapping() and introduce guest_physmap_add_pages
to cope with adding guest RAM p2m mapping with nr_pages.
- check both pbase and psize are page aligned
- simplify the code in the loops by moving append_static_memory_to_bank()
outside of the if/else.
---
change v6:
- move address_cells and size_cells check into device_tree_get_meminfo
- doc and comment refinement
- indentation, and the kconfig message fix
- let assign_page have a different order for the arguments, not assign_pages
- fix variable declaration and some format
- use PFN_DOWN(...)
- provide a stub allocate_static_memory implementation for
!CONFIG_STATIC_MEMORY
- add a statement to clarify the binding between the "memory" property
and "xen,static-mem"
---
change v7:
- harmonize the argument order of assign_pages and assign_page
---
TODO:
- reboot domain on static allocation
- Implement all memory-ops(hypercalls) regarding domain on static allocation
to balloon in/out memory
- asynchronously scrubbing PGC_reserved pages
Penny Zheng (7):
  xen/arm: introduce new helper device_tree_get_meminfo
  xen/arm: introduce domain on Static Allocation
  xen: introduce mark_page_free
  xen/arm: static memory initialization
  xen: re-define assign_pages and introduce a new function assign_page
  xen/arm: introduce acquire_staticmem_pages and acquire_domstatic_pages
  xen/arm: introduce allocate_static_memory

 docs/misc/arm/device-tree/booting.txt |  42 +++++
 xen/arch/arm/bootfdt.c                | 109 +++++++----
 xen/arch/arm/domain_build.c           | 161 ++++++++++++++++-
 xen/arch/arm/p2m.c                    |   7 +-
 xen/arch/arm/setup.c                  |  27 +++
 xen/arch/x86/pv/dom0_build.c          |   2 +-
 xen/common/Kconfig                    |  13 ++
 xen/common/grant_table.c              |   2 +-
 xen/common/memory.c                   |   6 +-
 xen/common/page_alloc.c               | 251 ++++++++++++++++++++------
 xen/include/asm-arm/mm.h              |   3 +
 xen/include/asm-arm/p2m.h             |  11 ++
 xen/include/asm-arm/setup.h           |   1 +
 xen/include/xen/mm.h                  |  14 ++
 14 files changed, 550 insertions(+), 99 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183854.332236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWer-0004Sz-6q; Fri, 10 Sep 2021 02:52:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183854.332236; Fri, 10 Sep 2021 02:52:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWer-0004Ss-1A; Fri, 10 Sep 2021 02:52:53 +0000
Received: by outflank-mailman (input) for mailman id 183854;
 Fri, 10 Sep 2021 02:52:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWep-0004S2-Ss
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:52:51 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.71]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 11f5d985-b6c4-409f-a054-1cb3dd7e8314;
 Fri, 10 Sep 2021 02:52:49 +0000 (UTC)
Received: from AM6PR08CA0008.eurprd08.prod.outlook.com (2603:10a6:20b:b2::20)
 by AM4PR08MB2883.eurprd08.prod.outlook.com (2603:10a6:205:9::30) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 02:52:45 +0000
Received: from AM5EUR03FT019.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:b2:cafe::c5) by AM6PR08CA0008.outlook.office365.com
 (2603:10a6:20b:b2::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:45 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT019.mail.protection.outlook.com (10.152.16.104) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:52:45 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Fri, 10 Sep 2021 02:52:45 +0000
Received: from aef7096e1d60.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 18FEB2E9-24A8-463E-B391-F094B9E62699.1; 
 Fri, 10 Sep 2021 02:52:38 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id aef7096e1d60.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:38 +0000
Received: from AS9PR06CA0175.eurprd06.prod.outlook.com (2603:10a6:20b:45c::25)
 by VE1PR08MB4653.eurprd08.prod.outlook.com (2603:10a6:802:b1::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:36 +0000
Received: from AM5EUR03FT061.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::f5) by AS9PR06CA0175.outlook.office365.com
 (2603:10a6:20b:45c::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:36 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT061.mail.protection.outlook.com (10.152.16.247) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:36 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:42 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Fri, 10
 Sep 2021 02:52:34 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11f5d985-b6c4-409f-a054-1cb3dd7e8314
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qWGJef8NpyPIjTGv5tGyQyanogwWdxG4O/jjsJWp4l8=;
 b=rdwRP2BhnFfHzdTn+7NGASRkWhm5/2ij8A894nlkuam8MHwmn6bRXbSjDtfPvZxhjd9CJ+pPjQ5sXjkwJrYEEPnLGCEVgnua+ghmnyKkU7wV3vh86rIfgogruZ59V/CXQwz3ld9YBNO0NG6eOwFVEIZE37tJPj1Hhk61/7cfOv8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f99e22bbac556734
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ADzIWDzld4KnCWL311lXNKsj/TuZveQVZ4KazgGdVeHwnRsdJ0VRg632zUCty45wCztYJjWjyF+Wpuefgd7Ce4LVUpPdj7XqY9WLgRoH7eeHs6UgV0LwqA+oO+EiPNAOBSQqu2MkELusZaBa0jIgq8wwfvML1B/oWebsj68axdk7Wvp8Ggm8NChPeGUWTFuCvBclRJHOR1I1pYJFgOQryDbaFxNqWbe2J+fNCOcx18mZqBmieOGeCLQ/4k4azahfRx6RwLq1jyolv4tpB8ajV5jLy6Y6NdPM1yktB72UNpSOUvc1QN0f3w0f5rYHCAplBh//iKULNd10eMkC4rJCUw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qWGJef8NpyPIjTGv5tGyQyanogwWdxG4O/jjsJWp4l8=;
 b=PN3wBOnBN1+6JIPSXfAXS5Hr0rdwDaPLtZ4Ae1w8HeI9pgGAGT5wd3WnBANlpsoSYMN0jMLtuTBfA+uHvTWPvcZlpnUYSqbiiUfTuUfwS8A8jH0BNL8J+Gpmr1Gi/AES/ZGLsiI0zhjI7KA3Ba5ZXNmsIr+cemhRh9j4RWXJKCbTi1TGvmfDBLZnEWD70YpmBxQ0y5q9QGD29cGdYElirodgqr0RyipRgL53HpMZzr7gpxZDUonijYe6rmt08VVjrpoz2aRPlkM1c8r5+b4vOyuJNePSkQ92y2kEM+fhNK/iKrBhO7pgk/eIOOlcnlhBSywdq1spM0hCpzPcfuKNhw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qWGJef8NpyPIjTGv5tGyQyanogwWdxG4O/jjsJWp4l8=;
 b=rdwRP2BhnFfHzdTn+7NGASRkWhm5/2ij8A894nlkuam8MHwmn6bRXbSjDtfPvZxhjd9CJ+pPjQ5sXjkwJrYEEPnLGCEVgnua+ghmnyKkU7wV3vh86rIfgogruZ59V/CXQwz3ld9YBNO0NG6eOwFVEIZE37tJPj1Hhk61/7cfOv8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 1/7] xen/arm: introduce new helper device_tree_get_meminfo
Date: Fri, 10 Sep 2021 02:52:09 +0000
Message-ID: <20210910025215.1671073-2-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d6514ac7-b076-4b74-7ebd-08d9740610f0
X-MS-TrafficTypeDiagnostic: VE1PR08MB4653:|AM4PR08MB2883:
X-Microsoft-Antispam-PRVS:
	<AM4PR08MB2883AA29A43B8FE2F467CB28F7D69@AM4PR08MB2883.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:41;OLM:41;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 j6TPtWGBobWDt2nNwtZp72ov6yuFpO7z45+5tXGNUAcDlGzxHVOdajsumaFTyM6JxFCvrdyF+qB3AqYFM3lESGT3XY32wbTpoa76sp9NxtB4ETT7RUNutIa931ujmrGuSW/zPdvaz+H7I5wZbNcW4qEjEJtoK01HV8YN8ra+r4ACLoEE7d1aSR3bR0/bBGUdqOEkx90q+oUD9kB6YNriSEpiVfNaS0Yf1BbcN01NWHZJlbgrnwimN8Bkfq0vjr/UessQ7uXjw2vThtvDoc4kMV5rd3LyaMfek2bJuz0KApxBp1LmrKhF4Rfz/T77HbHCbSiqwZNiU7XDWS2e1g2ASdcxZtKrIQqT2ehTNwQAmxFqxu8JwOUlY8TdFPqKuxlo+TBxKt7FmRNoeBuuPo82CtaZ7RU089yLGBBjkBaVzRvCWHUJuSTwQXJKAynSrlBoroXzUASvD1Kf9/b49ZyZe1aasHS6NApACVIOizOD6/xoRxp22DUFyS1hvmf+jZ00JmecMqPQZNZY6C7HyGshoOvez1TplKDN/8k2Tm2EtEBsU6i3cSxvaBLgPOGR2EaD1cCf7ACWKWsnrbJE3axOuIkptxGOE7lzvokUrretzEj3H/Naa3Wp2z5ryQdrd8zd2Zl+9AGxh37LWLuyYYw3wdr7mUWfKsOePWCn+ut/stf+Z9Spsx5EgkzsmWq74Bq8d7lb/MN0qsCZBoVJBMooc96O6kotRzGk+iGzEMV+GNM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(7696005)(8676002)(70206006)(1076003)(36860700001)(70586007)(83380400001)(44832011)(82310400003)(81166007)(2616005)(426003)(336012)(356005)(86362001)(2906002)(4326008)(316002)(36756003)(54906003)(110136005)(5660300002)(26005)(508600001)(6666004)(186003)(8936002)(47076005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4653
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d7277bf9-ea7f-4f1b-1d4c-08d974060b9c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Xf6oME0yBKglru7326W8HRM+P/Cd6K9bVlaxburBsdEBtwHVFcJyiCLeLnyM0vHjQVch8aLVQVPe19TTDnWA4vi8OkAQsEXF+wxPgMbvMmQt2upHA1XTX0Wv4PLaDj5xftvrWGcLkpebKMfuwcqVV8eqkWrsUDENEOiKol6PWQNDaIHDYXVN7Wez/VjB9BiQzpHT7XilNeaU3hgrJyuT3dKf0ysFTNG0ZyKB/4qx9UCVGYYULDd/2U9WWbyBCx75HZs2tVNbv0cwjxR7sl6n5poXpWCoje3sCmHXV6arh11eaGHVeEoLcJ87zoxnHE50Ousx4CIODKZDyu4/3i30TmLKYvl0x3kieVRkkpqcUdYNRsQuq2un0vIc9nAqLEVn6OeMogt6Ln7yn2b81/5NiBCHQTZMxVu7vOQbEGOQ+szjvUeMUKkUlvjJzVjD/TjkbFJVC6A5oQt8Aki8yQLYfdg/aEPQ2eAP1zUMOreN8DN2F7Ka7yobh4AP22AE0+VN06+Da3erTCvGNjjUT63m72W5q3jjGcbvBAhzdggkc+sz0pWosH3EkiYhA27xVKHC86XXtdmyTSLcomNTzK3Z8UogpFH06RRtprITZNX7uN+mhx08DqHuNEjqsLv6hHMPcTYIbvaitVMgMnmHzu5SCJS1sLyzmNstjOMFGHXYh0aUFiqNUcwiVR6NNPg+2FH6QfUpXKvPFM8qKPhKp9v5bA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(136003)(376002)(39860400002)(346002)(46966006)(36840700001)(1076003)(2906002)(110136005)(26005)(316002)(4326008)(6666004)(107886003)(478600001)(82310400003)(83380400001)(54906003)(81166007)(70206006)(5660300002)(47076005)(2616005)(336012)(44832011)(426003)(8676002)(82740400003)(8936002)(186003)(86362001)(36756003)(36860700001)(70586007)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:45.1908
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d6514ac7-b076-4b74-7ebd-08d9740610f0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR08MB2883

This commit creates a new helper device_tree_get_meminfo to iterate over a
device tree property to get memory info, like "reg".

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/bootfdt.c | 83 ++++++++++++++++++++++++------------------
 1 file changed, 47 insertions(+), 36 deletions(-)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 476e32e0f5..b01badda3e 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -63,6 +63,52 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
     *size = dt_next_cell(size_cells, cell);
 }
 
+static int __init device_tree_get_meminfo(const void *fdt, int node,
+                                          const char *prop_name,
+                                          u32 address_cells, u32 size_cells,
+                                          void *data)
+{
+    const struct fdt_property *prop;
+    unsigned int i, banks;
+    const __be32 *cell;
+    u32 reg_cells = address_cells + size_cells;
+    paddr_t start, size;
+    struct meminfo *mem = data;
+
+    if ( address_cells < 1 || size_cells < 1 )
+    {
+        printk("fdt: property `%s': invalid #address-cells or #size-cells",
+               prop_name);
+        return -EINVAL;
+    }
+
+    prop = fdt_get_property(fdt, node, prop_name, NULL);
+    if ( !prop )
+        return -ENOENT;
+
+    cell = (const __be32 *)prop->data;
+    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
+
+    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
+    {
+        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
+        /* Some DT may describe empty bank, ignore them */
+        if ( !size )
+            continue;
+        mem->bank[mem->nr_banks].start = start;
+        mem->bank[mem->nr_banks].size = size;
+        mem->nr_banks++;
+    }
+
+    if ( i < banks )
+    {
+        printk("Warning: Max number of supported memory regions reached.\n");
+        return -ENOSPC;
+    }
+
+    return 0;
+}
+
 u32 __init device_tree_get_u32(const void *fdt, int node,
                                const char *prop_name, u32 dflt)
 {
@@ -139,42 +185,7 @@ static int __init process_memory_node(const void *fdt, int node,
                                       u32 address_cells, u32 size_cells,
                                       void *data)
 {
-    const struct fdt_property *prop;
-    int i;
-    int banks;
-    const __be32 *cell;
-    paddr_t start, size;
-    u32 reg_cells = address_cells + size_cells;
-    struct meminfo *mem = data;
-
-    if ( address_cells < 1 || size_cells < 1 )
-    {
-        printk("fdt: node `%s': invalid #address-cells or #size-cells",
-               name);
-        return -EINVAL;
-    }
-
-    prop = fdt_get_property(fdt, node, "reg", NULL);
-    if ( !prop )
-        return -ENOENT;
-
-    cell = (const __be32 *)prop->data;
-    banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32));
-
-    for ( i = 0; i < banks && mem->nr_banks < NR_MEM_BANKS; i++ )
-    {
-        device_tree_get_reg(&cell, address_cells, size_cells, &start, &size);
-        /* Some DT may describe empty bank, ignore them */
-        if ( !size )
-            continue;
-        mem->bank[mem->nr_banks].start = start;
-        mem->bank[mem->nr_banks].size = size;
-        mem->nr_banks++;
-    }
-
-    if ( i < banks )
-        return -ENOSPC;
-    return 0;
+    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
 }
 
 static int __init process_reserved_memory_node(const void *fdt, int node,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183855.332247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWes-0004kK-ER; Fri, 10 Sep 2021 02:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183855.332247; Fri, 10 Sep 2021 02:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWes-0004k5-9w; Fri, 10 Sep 2021 02:52:54 +0000
Received: by outflank-mailman (input) for mailman id 183855;
 Fri, 10 Sep 2021 02:52:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWeq-0004Sh-UC
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:52:52 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 301de676-11e2-11ec-b209-12813bfff9fa;
 Fri, 10 Sep 2021 02:52:51 +0000 (UTC)
Received: from AS9PR06CA0069.eurprd06.prod.outlook.com (2603:10a6:20b:464::18)
 by VI1PR08MB3309.eurprd08.prod.outlook.com (2603:10a6:803:41::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:49 +0000
Received: from AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:464:cafe::fa) by AS9PR06CA0069.outlook.office365.com
 (2603:10a6:20b:464::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:49 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT014.mail.protection.outlook.com (10.152.16.130) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:49 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Fri, 10 Sep 2021 02:52:49 +0000
Received: from 84a38c9bd2fd.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 46A29243-FC7F-4B10-A916-47B71ACCF0B8.1; 
 Fri, 10 Sep 2021 02:52:42 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 84a38c9bd2fd.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:42 +0000
Received: from AS8PR04CA0078.eurprd04.prod.outlook.com (2603:10a6:20b:313::23)
 by PAXPR08MB7320.eurprd08.prod.outlook.com (2603:10a6:102:223::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 02:52:40 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::9a) by AS8PR04CA0078.outlook.office365.com
 (2603:10a6:20b:313::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:40 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:40 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:43 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 301de676-11e2-11ec-b209-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yzUXt/Gqs28bjIO6M0/X3qZ/21raERDi7GDI7xhSsGg=;
 b=nVkALiXt6FKKB37t6txu2AjGXor0TqQuiBYRAURs7Sbpm5WkOeNAEubgvT8FKRDzORNrdNeU3DLid9q3cwveEJixGkc9TmL8gC1GQHtrn+kWqQY90cWKQD/52nSrWrZ6fteJXdwYLjPEEjk7pRbY+BivnQ6ThtdjS0vi76/CB40=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a6c2d208d22835e4
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YjIhp61sTRlNRhx7xgM0IZICf9Ea48BxK1/OVL7mrWSKMtkYHZeDBuOV3fkEWmCUdz/lvPnR0ldMYNvhHwNq5LTMhZn3vaX4jrRTN5xUYx78aUIa3LMZ+UugFLCetOo/I+67D3I8OSRq96S9TR7vjP9hqCKqFPwuKUZuMolo+Rw7AGReWKsu/Ssf/yTxn7atOrNDWp0aOpE44sSrRZtjtOMJlwZqOL63pl4xr9tX0UOMrAR59yss0vGKnTIyNMfwRdSmcZ+DK3ruxEAtPlS80uzz2BgFTnd54vZgTLVEwfI9/26W46j965OjM9GcoXTdXvqtT1hC97f5Gj8mn5lpaA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=yzUXt/Gqs28bjIO6M0/X3qZ/21raERDi7GDI7xhSsGg=;
 b=hSq3v9e26erGgEwUb0NLPImQ8TqAPZc4J3nrWY1FHrxKpRpE1MAS213X5MQI6W/vR+aT04t4pWCaj8MSYEqkV4ZaDtLJeZfmabzuBYztemvyni5YduCfZ/SkxNyW++vSCZg4tICpeslVFn/qUZjeq1gqmUyJgNeUWQQFb5mkKdTEjgtKVpSXROMcKzXyzL7b1Cwny6dHS36qXkOj/Ar6EvAEXPBddAWaNQq3cW65LkUgsvFvCP9jTeuUbhIddT5eoUh4m+hiQ2+pnpxcdvePI+EXTeSnzXbcU1USdh1e8EUyR1KEpMWF5SBr2NUOYBhRsdAYkEL+TbjkTfSkiaSZTg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yzUXt/Gqs28bjIO6M0/X3qZ/21raERDi7GDI7xhSsGg=;
 b=nVkALiXt6FKKB37t6txu2AjGXor0TqQuiBYRAURs7Sbpm5WkOeNAEubgvT8FKRDzORNrdNeU3DLid9q3cwveEJixGkc9TmL8gC1GQHtrn+kWqQY90cWKQD/52nSrWrZ6fteJXdwYLjPEEjk7pRbY+BivnQ6ThtdjS0vi76/CB40=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 2/7] xen/arm: introduce domain on Static Allocation
Date: Fri, 10 Sep 2021 02:52:10 +0000
Message-ID: <20210910025215.1671073-3-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a09e70be-2866-440d-84bc-08d97406134f
X-MS-TrafficTypeDiagnostic: PAXPR08MB7320:|VI1PR08MB3309:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB33097ACE8878E7F194A71984F7D69@VI1PR08MB3309.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ZD+CgfeJobLy9gqTgZSbTVW2CSgGgur+Kz69v/oTKp1GwdNEqQyPx8xoRjtNHIQo/oJu99COlDtBRwkVUqzXqosBEEovDRtZ/NIn4Qm73Tq0qi2xZeLNKEa8b96LlfBRH2tYn9yuSD0EGMtURdSoftesjTHfI1m+4JN7HKicvsab5LgJu+10iNhclsk0Qv5atlMN2qfEEdcOjCjFeueHmdUBG4/ZBaB3BB3VzPBsj2yMbYxdMVvV0mEWBR0ZRWBQEl736Hhku6RKW7U3GY39yPEXbJCFbi6Z/L4PE7iQ1lPo2BftdL4VGt+nLoG0pNny41nDCMxb4Nd7ChsomiZmT1wuNY18RKfQ7BomSo4N0OhSTHtfK/8yYozliJKercXEyiW/XOEDHdHHar0sgjT0FOfieomM6VGm0JbyDM5ITOboXe0JZJQ2R2cpVCT0CKo/yf0nIFqzqB1gYUU+MLes8koAeefLsjbUdtb1yvMzMZIMMnBqstQrtDLDYev6D1czbQpvvlY3YcwM0MrgmOv2WcTkq4nOpl9mrKt4cNfzAHmVplTsioAVFhFe/6as9eCLIJ41xpAwPvrUzvt3PLtSuPrVioQ49L/rc3dgix07rIE51DZy5kqTK72ACepjI+oaAN+V9GDHdmHU31Luq6t6hBwE9mxjlzFXfbG1KnBNJsqzvx1A/MZK0Tjek8bAIXm4ZY701F8iitkJPRiH7nDwtHq9wAzKaN4FMHZ4WKoVrXU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(39860400002)(396003)(36840700001)(46966006)(36860700001)(36756003)(316002)(110136005)(83380400001)(186003)(47076005)(82310400003)(2616005)(4326008)(70586007)(426003)(70206006)(26005)(5660300002)(478600001)(6666004)(2906002)(86362001)(336012)(82740400003)(81166007)(356005)(54906003)(44832011)(8936002)(8676002)(7696005)(1076003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7320
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e2be9174-2c99-4ace-0a8d-08d974060e08
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	P2/wyYznGzsm4nHQEwBb6uXI8CJwGVzwnxBN1lsrOjusHE+6qOsMHZO4dE+i+8C40Mqa5B2NZkGWOsDPUDt5g/GZr8Mmv9tinZlW3BbGjQ251DXHMqOU+fOyHgURj6KTWcbf2VOsl1DWPH9VODiyKhHcTha0451tVX65QIDi/g6KDjQFmFtFYjpPjjhU8LmpTNY0LEmb5OvnOU9cnUzL83FW8J0od4j08hrDHiDLdHIEjcXphqjkVVppLsOj/5orKnaCYiNUEQl9eL5oYQJyGKBsWHuFHg7XluQoyzLWNcS9Vxpg01v7BHNaF3b1rRlG9F4rcLJ5Jc0gy6dvY51PJtMAzafePP6uIOAs+SAF2GqWpaC7Yl3W9ErfEbZpkcTsEgGfExZYSCGBN0ZmlUIkCx9LK/an+KheChYh2HBfqLg6IZlAlZrcn1d/i9LPcqD+pmPEAjx7FIPaKxJSi1EH5t/oMcF4+4XWlI0EyqGdgcQzTgqOSFmeOrRZjksjhuL/Zal9Rm6SXBuANqsrZ6kjYknTLVZg7wqqc4OHyzoqwb4vDbApxtmnQwUuWGTdUWzPCzABX0J3j+sjgBi2mJhsF4VMN2vag7+FX+9S6hpehE0t+1Huq/1bKSR4SPS2CDCWIiSDVWJRFsDbIrl32vgvi8V4zI66/kl0UXkuVVXY65H6uR3SYDaDvomUlmTLphEkgbmRyxvpuUE+29NAwnrkwA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(346002)(396003)(39860400002)(136003)(46966006)(36840700001)(2906002)(2616005)(316002)(83380400001)(478600001)(107886003)(82310400003)(110136005)(186003)(6666004)(44832011)(54906003)(70206006)(70586007)(5660300002)(8936002)(26005)(336012)(86362001)(426003)(4326008)(8676002)(82740400003)(36860700001)(36756003)(1076003)(81166007)(7696005)(47076005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:49.1710
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a09e70be-2866-440d-84bc-08d97406134f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3309

Static Allocation refers to system or sub-system(domains) for which memory
areas are pre-defined by configuration using physical address ranges.

Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
beginning, shall never go to heap allocator or boot allocator for any use.

Memory can be statically allocated to a domain using the property "xen,static-
mem" defined in the domain configuration. The number of cells for the address
and the size must be defined using respectively the properties
"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".

The property 'memory' is still needed and should match the amount of memory
given to the guest. Currently, it either comes from static memory or lets Xen
allocate from heap. *Mixing* is not supported.

The static memory will be mapped in the guest at the usual guest memory
addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

This patch introduces this new `xen,static-mem` feature, and also documents
and parses this new attribute at boot time.

This patch also introduces a new field "bool xen_domain" in "struct membank"
to tell whether the memory bank is reserved as the whole hardware resource,
or bind to a xen domain node, through "xen,static-mem"

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 docs/misc/arm/device-tree/booting.txt | 42 +++++++++++++++++++++++++++
 xen/arch/arm/bootfdt.c                | 30 +++++++++++++++++--
 xen/include/asm-arm/setup.h           |  1 +
 3 files changed, 71 insertions(+), 2 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 5243bc7fd3..44cd9e1a9a 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -268,3 +268,45 @@ The DTB fragment is loaded at 0xc000000 in the example above. It should
 follow the convention explained in docs/misc/arm/passthrough.txt. The
 DTB fragment will be added to the guest device tree, so that the guest
 kernel will be able to discover the device.
+
+
+Static Allocation
+=============
+
+Static Allocation refers to system or sub-system(domains) for which memory
+areas are pre-defined by configuration using physical address ranges.
+
+Memory can be statically allocated to a domain using the property "xen,static-
+mem" defined in the domain configuration. The number of cells for the address
+and the size must be defined using respectively the properties
+"#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
+
+The property 'memory' is still needed and should match the amount of memory
+given to the guest. Currently, it either comes from static memory or lets Xen
+allocate from heap. *Mixing* is not supported.
+
+The static memory will be mapped in the guest at the usual guest memory
+addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
+xen/include/public/arch-arm.h.
+
+Below is an example on how to specify the static memory region in the
+device-tree:
+
+    / {
+        chosen {
+            domU1 {
+                compatible = "xen,domain";
+                #address-cells = <0x2>;
+                #size-cells = <0x2>;
+                cpus = <2>;
+                memory = <0x0 0x80000>;
+                #xen,static-mem-address-cells = <0x1>;
+                #xen,static-mem-size-cells = <0x1>;
+                xen,static-mem = <0x30000000 0x20000000>;
+                ...
+            };
+        };
+    };
+
+This will reserve a 512MB region starting at the host physical address
+0x30000000 to be exclusively used by DomU1.
diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index b01badda3e..afaa0e249b 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -66,7 +66,7 @@ void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
 static int __init device_tree_get_meminfo(const void *fdt, int node,
                                           const char *prop_name,
                                           u32 address_cells, u32 size_cells,
-                                          void *data)
+                                          void *data, bool xen_domain)
 {
     const struct fdt_property *prop;
     unsigned int i, banks;
@@ -97,6 +97,7 @@ static int __init device_tree_get_meminfo(const void *fdt, int node,
             continue;
         mem->bank[mem->nr_banks].start = start;
         mem->bank[mem->nr_banks].size = size;
+        mem->bank[mem->nr_banks].xen_domain = xen_domain;
         mem->nr_banks++;
     }
 
@@ -185,7 +186,8 @@ static int __init process_memory_node(const void *fdt, int node,
                                       u32 address_cells, u32 size_cells,
                                       void *data)
 {
-    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, data);
+    return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells,
+                                   data, false);
 }
 
 static int __init process_reserved_memory_node(const void *fdt, int node,
@@ -339,6 +341,28 @@ static void __init process_chosen_node(const void *fdt, int node,
     add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
 }
 
+static int __init process_domain_node(const void *fdt, int node,
+                                      const char *name,
+                                      u32 address_cells, u32 size_cells)
+{
+    const struct fdt_property *prop;
+
+    printk("Checking for \"xen,static-mem\" in domain node\n");
+
+    prop = fdt_get_property(fdt, node, "xen,static-mem", NULL);
+    if ( !prop )
+        /* No "xen,static-mem" present. */
+        return 0;
+
+    address_cells = device_tree_get_u32(fdt, node,
+                                        "#xen,static-mem-address-cells", 0);
+    size_cells = device_tree_get_u32(fdt, node,
+                                     "#xen,static-mem-size-cells", 0);
+
+    return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells,
+                                   size_cells, &bootinfo.reserved_mem, true);
+}
+
 static int __init early_scan_node(const void *fdt,
                                   int node, const char *name, int depth,
                                   u32 address_cells, u32 size_cells,
@@ -357,6 +381,8 @@ static int __init early_scan_node(const void *fdt,
         process_multiboot_node(fdt, node, name, address_cells, size_cells);
     else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") )
         process_chosen_node(fdt, node, name, address_cells, size_cells);
+    else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") )
+        rc = process_domain_node(fdt, node, name, address_cells, size_cells);
 
     if ( rc < 0 )
         printk("fdt: node `%s': parsing failed\n", name);
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index c4b6af6029..95da0b7ab9 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -24,6 +24,7 @@ typedef enum {
 struct membank {
     paddr_t start;
     paddr_t size;
+    bool xen_domain; /* whether the memory bank is bound to a Xen domain. */
 };
 
 struct meminfo {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:53:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:53:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183857.332258 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf1-0005Hm-3K; Fri, 10 Sep 2021 02:53:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183857.332258; Fri, 10 Sep 2021 02:53:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf0-0005GR-UW; Fri, 10 Sep 2021 02:53:02 +0000
Received: by outflank-mailman (input) for mailman id 183857;
 Fri, 10 Sep 2021 02:53:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWez-0004S2-Rd
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:53:01 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::607])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 83267da6-c972-4f40-bdfc-62b4bd15b13e;
 Fri, 10 Sep 2021 02:52:53 +0000 (UTC)
Received: from AS8PR04CA0014.eurprd04.prod.outlook.com (2603:10a6:20b:310::19)
 by PAXPR08MB6605.eurprd08.prod.outlook.com (2603:10a6:102:153::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:51 +0000
Received: from AM5EUR03FT049.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:310:cafe::8f) by AS8PR04CA0014.outlook.office365.com
 (2603:10a6:20b:310::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT049.mail.protection.outlook.com (10.152.17.130) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:51 +0000
Received: ("Tessian outbound e27daf245730:v103");
 Fri, 10 Sep 2021 02:52:50 +0000
Received: from d1bc6d506f25.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 22C3B9A6-7B49-43B4-8982-4B95B4DD05A9.1; 
 Fri, 10 Sep 2021 02:52:44 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d1bc6d506f25.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:44 +0000
Received: from AS8PR04CA0070.eurprd04.prod.outlook.com (2603:10a6:20b:313::15)
 by AS8PR08MB6278.eurprd08.prod.outlook.com (2603:10a6:20b:29a::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 02:52:42 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::9b) by AS8PR04CA0070.outlook.office365.com
 (2603:10a6:20b:313::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:42 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:42 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:46 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83267da6-c972-4f40-bdfc-62b4bd15b13e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=YA7xJNX+7lrO/F6Bubd9wbDUzGgnyZApHYJThO4/ceyFmBcW/frlQHUSRiR7Me2T6lx/e+21dk5o0+uaHPsB1jHcMoRLOja8vlp7AWghXhlLcSiEcmBwxmZtB7660c8LV3s9aBhg35+qBHhNjZFtbCrKfon89xdlRp9hM9piD4Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 1c4e958c66743e38
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=a6APgFWMbDbJ7BIj3hOAHXH9gIRbcq5u9cX5EWRKoaIl1jDVEMOtOO4wHs1SG4RPV6tdk+E9coz58O4zC0Pvq6pX89H6IIAUVcP8mbze7TOG4pfLhT/7S4Y2iDw4u6UMSvDpbgxadxMwSD6MjnbS/4yU4iVIuotdPCvcFf+MG4+3Qc93yG4kfYYN9ugLbgzHl6BqGq/grUoTqNrGxj+jq0Enw5vo/WTRVeWO6CKIL9OGb8NEa0j+7UDXRpxoh164/D+/pIEdjSZAtDmlkGk6MWFN5mMEf95kgCesZWSm6ahFwum/Ye0rXafNF5ses14u2mYLXxETpJa+/v8RjOZlEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=Pv6hR7Hr8Rhx1i279//PJW8jzO8ortlchi7uNxAUf5uiUu2pV+7xARKX7RCgYujF+na5ay2Ct9tzoGCg7XnWATFqmx8Itmx3qYW8vDW65UTJLjZSGagYEM4LEBqMiUQVQ4GTuMxp5zDXhS8HXsDJU7p4YrGvZdte5/liar5C8gml9Q5suVxsPagTkr7I6gv9zPA4tXK2OVLOHUWN4dU2+KC0sC0G3j5vccC1mpr2vHnvbUw6FKa5Pm2Yy/RSi78uFI6r1mkrY9pQDsQuc9moEkJURcuxWb6XZfgwXWqVoXAfDPiVs2t6Zsak6bV6tAvkHT0KCVThgMO8kAsaPUyWvQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AMCW5xXNqkaV3p45gVg1gz98GeXvw0DWsgYQwmmZLho=;
 b=YA7xJNX+7lrO/F6Bubd9wbDUzGgnyZApHYJThO4/ceyFmBcW/frlQHUSRiR7Me2T6lx/e+21dk5o0+uaHPsB1jHcMoRLOja8vlp7AWghXhlLcSiEcmBwxmZtB7660c8LV3s9aBhg35+qBHhNjZFtbCrKfon89xdlRp9hM9piD4Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 3/7] xen: introduce mark_page_free
Date: Fri, 10 Sep 2021 02:52:11 +0000
Message-ID: <20210910025215.1671073-4-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4083b3d1-d504-44cd-96af-08d97406147d
X-MS-TrafficTypeDiagnostic: AS8PR08MB6278:|PAXPR08MB6605:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB66053B7A389454A78A63858EF7D69@PAXPR08MB6605.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1051;OLM:1051;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 OBmtivbTaE937o5rHRs0Vpi7zGRVV9j7gF4JAAkcwRM86hdVHySjqYcW813vmeULkKhGHK1Tik9tCdZS7XBL3J+38fINKsn1P7j6Qhz5JjeNSNBU+0KZ4N+n6c1GekxnzwqjSgXXpkwf7W7bU79C055/+OSLWfhMp4Pf8zu1c8Xne2nbOe2KnQurI9KV8YslvpPeYbG6Sr7VMBL6UwtvrAsihckAFZF+imMNXHNeCGLdQ9chKrMimtyB3I+BhQH3X4qQ1H53MSvcbtEywB+vsOKjl40phf64jj6aEqxDNB3JnsEW83YD0+u5qnA+eqzJkPGMhDU8Miw+BFuLpi8A9sVVO+k5iLr9xyVzt8IyWDhLJsCGBvzs16h3c87L07JTQGFpRJ16R0tC3QWxXKflhJv9nieuXUYEX603sIr9l8S241+Q/KB+qN+tFnh5YWah6NjZr9/ZmMDDc3j2jaq1t3Zj9T2PbQ00BGGg2cZRupG9NO+2UKK3dK0DBhfLV3e6XDoPIYAa50EfJ6fY3a7D2ys5vxbnU6RLvemoX9uLzZJz3IFKxbE0eiUgW0JxiyMVxJJ+WHZKDoRsEuuIqxPjL4bYz8vOnd/lgwn/udjkUxaeCM0OztvN/mas9k7coYXqp9QK7aiLQ1HhUvXL8Z1RyTUhTmQnF9weg6we1jR243qUpxPLJeLZmBaQum1A1Kucjuo7hYoSketfRCrG03jmBfc97ekRoY2baFHgiqL3Ib4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39860400002)(396003)(376002)(346002)(136003)(46966006)(36840700001)(47076005)(6666004)(70206006)(316002)(2906002)(110136005)(82740400003)(54906003)(86362001)(478600001)(70586007)(8676002)(426003)(186003)(8936002)(4326008)(26005)(1076003)(44832011)(36756003)(7696005)(81166007)(356005)(2616005)(36860700001)(336012)(83380400001)(82310400003)(5660300002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6278
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3d7e8852-6ece-4814-dc54-08d974060f4e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5sXt+8QZxLQZbD3C4mkLkKlvAftYZaO9HHLW2xU0PPi074nnC/cjRGQllYkXdjlu/obOUKKy05lEd+q33//VaGdS4kWZ2HXijIZ1yE//VJScCOx5xnpDVTr+kslHOqY4EMLYxQhkD+OL3Xw6b4QH+By9+ZOpPkOVTdvCwd7rKFG20pKsuH0MCe8MHmFPTJOUkhDFiAHNsHYED8/J8+wxWJUFiLslXYn/qjmxKUmBoTlG7vrLW4KvlyOS/AsOwSxpr4H9Vek8vyynItFUL5xyal9SMfGs8q9zd/eaDMFhtDtlHlrGcQkj8jVRbEeRfjBJj7nCq+g6ZDxBHXhrYr0sLh2PdxeBkK35D2fk9JwaF7Jy1HfNwvjlG2i/+TSEq7jAtHO2DE5xsJx8G7997XTNvzOSYDLtLbNR327AyIYX89X4gF7bUHg4ULkoupNnNsX+q16Rh8TJgVqHuOUHTcVh/CEffGBg1F/h6ZLZjPmYpNpdua3h+xI7oEG/Xncdx44cZswdGrm4qKg9ga4QjZT1nYHSa5OjgP0OV6GHIxE9iI0AbrZ9BMCrLKC2wh7e3tDCjWAclDZyrRipq5fuWpCdnTpFOc/bi+d4YJX3HLYPr5M/Gp+FUvXoXmKBxp61zR+cPw61I/NBzE7LFaXcNCmz2G90CpzlUNybPOa3mqCj3oFWX6G57vag5VWihaaqdJjw9IZzB7DnC8n143Amof7Yxg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(346002)(39860400002)(396003)(136003)(46966006)(36840700001)(316002)(107886003)(36756003)(2616005)(4326008)(8676002)(70206006)(478600001)(5660300002)(70586007)(110136005)(336012)(81166007)(186003)(54906003)(426003)(82740400003)(36860700001)(82310400003)(1076003)(26005)(2906002)(83380400001)(86362001)(44832011)(47076005)(7696005)(8936002)(6666004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:51.1416
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4083b3d1-d504-44cd-96af-08d97406147d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6605

This commit defines a new helper mark_page_free to extract common code,
like following the same cache/TLB coherency policy, between free_heap_pages
and the new function free_staticmem_pages, which will be introduced later.

The PDX compression makes that conversion between the MFN and the page can
be potentially non-trivial. As the function is internal, pass the MFN and
the page. They are both expected to match.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 xen/common/page_alloc.c | 89 ++++++++++++++++++++++-------------------
 1 file changed, 48 insertions(+), 41 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 958ba0cd92..a3ee5eca9e 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1376,6 +1376,53 @@ bool scrub_free_pages(void)
     return node_to_scrub(false) != NUMA_NO_NODE;
 }
 
+static void mark_page_free(struct page_info *pg, mfn_t mfn)
+{
+    ASSERT(mfn_x(mfn) == mfn_x(page_to_mfn(pg)));
+
+    /*
+     * Cannot assume that count_info == 0, as there are some corner cases
+     * where it isn't the case and yet it isn't a bug:
+     *  1. page_get_owner() is NULL
+     *  2. page_get_owner() is a domain that was never accessible by
+     *     its domid (e.g., failed to fully construct the domain).
+     *  3. page was never addressable by the guest (e.g., it's an
+     *     auto-translate-physmap guest and the page was never included
+     *     in its pseudophysical address space).
+     * In all the above cases there can be no guest mappings of this page.
+     */
+    switch ( pg->count_info & PGC_state )
+    {
+    case PGC_state_inuse:
+        BUG_ON(pg->count_info & PGC_broken);
+        pg->count_info = PGC_state_free;
+        break;
+
+    case PGC_state_offlining:
+        pg->count_info = (pg->count_info & PGC_broken) |
+                         PGC_state_offlined;
+        tainted = 1;
+        break;
+
+    default:
+        printk(XENLOG_ERR
+               "pg MFN %"PRI_mfn" c=%#lx o=%u v=%#lx t=%#x\n",
+               mfn_x(mfn),
+               pg->count_info, pg->v.free.order,
+               pg->u.free.val, pg->tlbflush_timestamp);
+        BUG();
+    }
+
+    /* If a page has no owner it will need no safety TLB flush. */
+    pg->u.free.need_tlbflush = (page_get_owner(pg) != NULL);
+    if ( pg->u.free.need_tlbflush )
+        page_set_tlbflush_timestamp(pg);
+
+    /* This page is not a guest frame any more. */
+    page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
+    set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
+}
+
 /* Free 2^@order set of pages. */
 static void free_heap_pages(
     struct page_info *pg, unsigned int order, bool need_scrub)
@@ -1392,47 +1439,7 @@ static void free_heap_pages(
 
     for ( i = 0; i < (1 << order); i++ )
     {
-        /*
-         * Cannot assume that count_info == 0, as there are some corner cases
-         * where it isn't the case and yet it isn't a bug:
-         *  1. page_get_owner() is NULL
-         *  2. page_get_owner() is a domain that was never accessible by
-         *     its domid (e.g., failed to fully construct the domain).
-         *  3. page was never addressable by the guest (e.g., it's an
-         *     auto-translate-physmap guest and the page was never included
-         *     in its pseudophysical address space).
-         * In all the above cases there can be no guest mappings of this page.
-         */
-        switch ( pg[i].count_info & PGC_state )
-        {
-        case PGC_state_inuse:
-            BUG_ON(pg[i].count_info & PGC_broken);
-            pg[i].count_info = PGC_state_free;
-            break;
-
-        case PGC_state_offlining:
-            pg[i].count_info = (pg[i].count_info & PGC_broken) |
-                               PGC_state_offlined;
-            tainted = 1;
-            break;
-
-        default:
-            printk(XENLOG_ERR
-                   "pg[%u] MFN %"PRI_mfn" c=%#lx o=%u v=%#lx t=%#x\n",
-                   i, mfn_x(mfn) + i,
-                   pg[i].count_info, pg[i].v.free.order,
-                   pg[i].u.free.val, pg[i].tlbflush_timestamp);
-            BUG();
-        }
-
-        /* If a page has no owner it will need no safety TLB flush. */
-        pg[i].u.free.need_tlbflush = (page_get_owner(&pg[i]) != NULL);
-        if ( pg[i].u.free.need_tlbflush )
-            page_set_tlbflush_timestamp(&pg[i]);
-
-        /* This page is not a guest frame any more. */
-        page_set_owner(&pg[i], NULL); /* set_gpfn_from_mfn snoops pg owner */
-        set_gpfn_from_mfn(mfn_x(mfn) + i, INVALID_M2P_ENTRY);
+        mark_page_free(&pg[i], mfn_add(mfn, i));
 
         if ( need_scrub )
         {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:53:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:53:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183858.332269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf3-0005dl-EI; Fri, 10 Sep 2021 02:53:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183858.332269; Fri, 10 Sep 2021 02:53:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf3-0005dY-AR; Fri, 10 Sep 2021 02:53:05 +0000
Received: by outflank-mailman (input) for mailman id 183858;
 Fri, 10 Sep 2021 02:53:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWf1-0005Hy-36
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:53:03 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 364fa0b6-11e2-11ec-b209-12813bfff9fa;
 Fri, 10 Sep 2021 02:53:01 +0000 (UTC)
Received: from AM5PR04CA0011.eurprd04.prod.outlook.com (2603:10a6:206:1::24)
 by DBAPR08MB5768.eurprd08.prod.outlook.com (2603:10a6:10:1b1::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:59 +0000
Received: from VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:1:cafe::d2) by AM5PR04CA0011.outlook.office365.com
 (2603:10a6:206:1::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT053.mail.protection.outlook.com (10.152.19.198) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:52:58 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Fri, 10 Sep 2021 02:52:58 +0000
Received: from b326055b7f74.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8D64FAC2-F47D-4F5A-80BE-9FD6D2FC3941.1; 
 Fri, 10 Sep 2021 02:52:52 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b326055b7f74.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:52 +0000
Received: from AS8PR04CA0079.eurprd04.prod.outlook.com (2603:10a6:20b:313::24)
 by HE1PR0802MB2585.eurprd08.prod.outlook.com (2603:10a6:3:d4::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Fri, 10 Sep
 2021 02:52:50 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::39) by AS8PR04CA0079.outlook.office365.com
 (2603:10a6:20b:313::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:50 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:49 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:53 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 364fa0b6-11e2-11ec-b209-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hGwFKYY5V6TMXu7tjZlS3TFGVb7eOMQrJfMUNLSc/DY=;
 b=4CEwSqaztcXIYoi435uzhaDzyqi/pZBUgaj+QKh0G5xGKehylQTnknaPQwuzOpKCGC5lfmum3zjrZyUdCvUiwh43JN2IW3SI/tf5/mcDo6yElRwZprWai26VjHg9C3IkMODeVR46HVuscIOfGKKgMM9VJsXFP2GceH3/ui8INXo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 703c4f62234778a0
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Sp/1nHV5tgMLNZF/mVW5m/BvqZoL3/KjHFRz38FlkeBk88s12C6ryq92ZIPCSR9/VaaMIgjaZo/jymiCPXioT0gfKt+xnhGfnyO3Vxw1l7wyuCuKZ4ROA/fhTux+S+r3/1i9lMxFcInh2ocEJVECTmcwZI1zg75mC9j6FCEUWdk4w7Y/hYW0mjCm25JwBTf/+T//1D3ohSG86ZgRyHGifWZqAmuV48pqT8Ll3uufHQbMMsacR79niGizcCGM70G5A1Lu4b/qrlrd1KtfhQWxvrEQmEq0RwbaZwni8NNoHDuMHEVxIenGUgc6jc4nebzwTADH6SOtmZ6IUTKASs2ssg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hGwFKYY5V6TMXu7tjZlS3TFGVb7eOMQrJfMUNLSc/DY=;
 b=il0AROm0oKYVcDlQ924c9bQCWKQ87gZwqTOzycq/CQnJH3EfZDOASke9ewntmFURMaC3biFmMqdM/kg+sQ9cAuKdsR3vDeB+vbg33VOeeCuQ63xc4Q6JjeFOky7gzu8iGQMPPcxR6jTGMwB58QKc5sOGE6uazoWyCaIyrc8Onler7JrIUA3ZpTohA/2zIlxpPyrg271k6SA9ae0IiWQZCurTdII8hZ2aal/SWh82Y+y+8sEHuOdSdMOse4feH9pIFzTm6MdJkqORvj/WXgFebdtFNKXXbL26VcnGLFVVUDigIv8/AEHxH/b6F+A/i7zQn48z9NnBOxLrtQHknOCEzA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hGwFKYY5V6TMXu7tjZlS3TFGVb7eOMQrJfMUNLSc/DY=;
 b=4CEwSqaztcXIYoi435uzhaDzyqi/pZBUgaj+QKh0G5xGKehylQTnknaPQwuzOpKCGC5lfmum3zjrZyUdCvUiwh43JN2IW3SI/tf5/mcDo6yElRwZprWai26VjHg9C3IkMODeVR46HVuscIOfGKKgMM9VJsXFP2GceH3/ui8INXo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 6/7] xen/arm: introduce acquire_staticmem_pages and acquire_domstatic_pages
Date: Fri, 10 Sep 2021 02:52:14 +0000
Message-ID: <20210910025215.1671073-7-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 789e4663-c80f-4918-ea94-08d974061902
X-MS-TrafficTypeDiagnostic: HE1PR0802MB2585:|DBAPR08MB5768:
X-Microsoft-Antispam-PRVS:
	<DBAPR08MB576884CA06D2ABA9DD61724FF7D69@DBAPR08MB5768.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 B69ufQAkoo4x7JS2n/eCI1CIMVScJzIvR2cTvZ5tfp9rJbr0OnWVmRY7ucjGHV1BfyZKNjHHpWUrnwR6bMD3crXPiAN9g9cu1309g/qpp87z8yuWvKb+DTxHWPaknUWk7aqV3Z6NpPomXQNfoo4h/mXBPcEmISuXSsb9zVW/hXPnx3isH8C/jLLCcIlxpMlQ4XMCHcJsOozJRtGAKNujmwPj1/lP5jKd9HDEccODXzpw7K8SeYtkQM/1CMYtH53On6hAo10MkpEXrVgyHUCT+LzHpFznSxjziVmhbs6tq4sv1IluPBnFVa99WYq8wF2/CTLXzdp/FWs6XuEg7UJL/axI6hw87+FDGsg/2R3aut6X4JL0BK9yEUuybWbB+duXfatNwCTATVwZhmii6TRGKDHVtdeTIygVcgXAsfltKQNd9YB0FYXlXSnbe/PdgaKpkzSzYeo1rfDaKQkBKS8/Rh5T8k1ta/7TGf3IQ4Nm/KIDg+IFho+ZAr7CLYCc6UTV1UN4H4l2JyAuq3QJjRJYBYSUauad/8Oxpm7G1JBuUURa0492WN/nS4UxNx+YTnUKT2k9op4ZxvThyc+NnduYke2UEFfnVc8FqDkDxWSr4w50EgYQDDgvGqqJrLeQTOg7nN4nunpXokvN1LBLTkaRgGHwOrwrKvlDc6PO7UtSVPth2W1tUzSqmngWwnfr6LzOHLmQOifDPuxtGgW26Hdk3/oQbPxOr+v5r7hzLU40Krc=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(39860400002)(346002)(36840700001)(46966006)(36756003)(356005)(8936002)(4326008)(82740400003)(2906002)(70586007)(316002)(83380400001)(5660300002)(478600001)(70206006)(82310400003)(81166007)(47076005)(7696005)(336012)(186003)(1076003)(2616005)(44832011)(426003)(6666004)(26005)(54906003)(86362001)(110136005)(8676002)(36860700001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2585
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f77242cb-d175-4fb6-d253-08d9740613c4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0J7Mrz8oQmoemMFwEMjm/LvD/WDiu+YTTLEkrBKzKzrvBE4mLJt9dBH8E7WWdcZUphEkN9JF2xCe92xYNq5DPN4TT7nz+Bzz8h4ncyY5ALP97vbySJ1qr/fczoAVU9NVyRHOt4ne11pwVoYmdaXO85l76XIgZMMxCkts9hxbOWBKcPmUIgo3HOziyLDW4ycytROzf9tfmeC695ACbIWaoPjKumh6qyzPDPyY82MTW+64rX/Fhv0S6SJxiP+BGtTCh5n5je/tttMGoe0p/PmH5aa+Ym/J7udGpuTlVUSMsInl0WXFtYRt11jvMitoAVz+ImMJ8/8K3RYW9zgoRq25kBM7qDZ2RhdbCG8AA9K1+7kEHCkCPNnbw7xdQnH+sBAvHt1Kp4d1+jE2pUbn7wvf9/1Tru5ujHsIAsE3YWYV8+6pFi8oU6JkUxvVLQUfuxw5JbVxGIi62iiOjLl2KMlrAJ5JTFdNN0QOHZ6TeVWBYPbPR9QCPz/Ap9kVXMc4Zucsu0cPfxfE7E+EfpJpSI+V+PkWawePuI7lqkIOTqJrxKvyjdFzYB+/+BRYuZTerC5uRwZcYnOCjxjrOERzOa/C8NVp+D9zWbl06vrLKh4792dXN4cvZ+SsxP8T+3LrlxkE8MJFVXvBrqyGpSztISUlWKH3MreTKcAVghIG46Rg11gJh5JGFus+C6fFoFxtAHKZvX9hgtuapOAKt1wWSIuBMQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(2616005)(8936002)(508600001)(81166007)(316002)(2906002)(110136005)(4326008)(54906003)(336012)(47076005)(83380400001)(6666004)(107886003)(426003)(86362001)(1076003)(5660300002)(82310400003)(186003)(26005)(36860700001)(44832011)(8676002)(70586007)(7696005)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:58.6819
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 789e4663-c80f-4918-ea94-08d974061902
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5768

New function acquire_staticmem_pages aims to acquire nr_mfns contiguous pages
of static memory, starting at #smfn. And it is the equivalent of
alloc_heap_pages for static memory.

For each page, it shall check if the page is reserved(PGC_reserved)
and free. It shall also do a set of necessary initialization, which are
mostly the same ones in alloc_heap_pages, like, following the same
cache-coherency policy and turning page status into PGC_state_inuse, etc.

New function acquire_domstatic_pages is the equivalent of alloc_domheap_pages
for static memory, and it is to acquire nr_mfns contiguous pages of
static memory and assign them to one specific domain.

It uses acquire_staticmem_pages to acquire nr_mfns pages of static memory.
Then on success, it will use assign_pages to assign those pages to one
specific domain.

In order to differentiate pages of static memory from those allocated from
heap, this patch introduces a new page flag PGC_reserved, then mark pages of
static memory PGC_reserved when initializing them.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/common/page_alloc.c  | 118 ++++++++++++++++++++++++++++++++++++++-
 xen/include/asm-arm/mm.h |   3 +
 xen/include/xen/mm.h     |   2 +
 3 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 2aa8edac8c..5eb87b51c8 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -151,6 +151,10 @@
 #define p2m_pod_offline_or_broken_replace(pg) BUG_ON(pg != NULL)
 #endif
 
+#ifndef PGC_reserved
+#define PGC_reserved 0
+#endif
+
 /*
  * Comma-separated list of hexadecimal page numbers containing bad bytes.
  * e.g. 'badpage=0x3f45,0x8a321'.
@@ -2283,7 +2287,7 @@ int assign_pages(
 
         for ( i = 0; i < nr; i++ )
         {
-            ASSERT(!(pg[i].count_info & ~PGC_extra));
+            ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
             if ( pg[i].count_info & PGC_extra )
                 extra_pages++;
         }
@@ -2322,7 +2326,8 @@ int assign_pages(
         page_set_owner(&pg[i], d);
         smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
         pg[i].count_info =
-            (pg[i].count_info & PGC_extra) | PGC_allocated | 1;
+            (pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated | 1;
+
         page_list_add_tail(&pg[i], page_to_list(d, &pg[i]));
     }
 
@@ -2626,8 +2631,117 @@ void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
             /* TODO: asynchronous scrubbing for pages of static memory. */
             scrub_one_page(pg);
         }
+
+        /* In case initializing page of static memory, mark it PGC_reserved. */
+        pg[i].count_info |= PGC_reserved;
     }
 }
+
+/*
+ * Acquire nr_mfns contiguous reserved pages, starting at #smfn, of
+ * static memory.
+ * This function needs to be reworked if used outside of boot.
+ */
+static struct page_info * __init acquire_staticmem_pages(mfn_t smfn,
+                                                         unsigned long nr_mfns,
+                                                         unsigned int memflags)
+{
+    bool need_tlbflush = false;
+    uint32_t tlbflush_timestamp = 0;
+    unsigned long i;
+    struct page_info *pg;
+
+    ASSERT(nr_mfns);
+    for ( i = 0; i < nr_mfns; i++ )
+        if ( !mfn_valid(mfn_add(smfn, i)) )
+            return NULL;
+
+    pg = mfn_to_page(smfn);
+
+    spin_lock(&heap_lock);
+
+    for ( i = 0; i < nr_mfns; i++ )
+    {
+        /* The page should be reserved and not yet allocated. */
+        if ( pg[i].count_info != (PGC_state_free | PGC_reserved) )
+        {
+            printk(XENLOG_ERR
+                   "pg[%lu] Static MFN %"PRI_mfn" c=%#lx t=%#x\n",
+                   i, mfn_x(smfn) + i,
+                   pg[i].count_info, pg[i].tlbflush_timestamp);
+            goto out_err;
+        }
+
+        if ( !(memflags & MEMF_no_tlbflush) )
+            accumulate_tlbflush(&need_tlbflush, &pg[i],
+                                &tlbflush_timestamp);
+
+        /*
+         * Preserve flag PGC_reserved and change page state
+         * to PGC_state_inuse.
+         */
+        pg[i].count_info = PGC_reserved | PGC_state_inuse;
+        /* Initialise fields which have other uses for free pages. */
+        pg[i].u.inuse.type_info = 0;
+        page_set_owner(&pg[i], NULL);
+    }
+
+    spin_unlock(&heap_lock);
+
+    if ( need_tlbflush )
+        filtered_flush_tlb_mask(tlbflush_timestamp);
+
+    /*
+     * Ensure cache and RAM are consistent for platforms where the guest
+     * can control its own visibility of/through the cache.
+     */
+    for ( i = 0; i < nr_mfns; i++ )
+        flush_page_to_ram(mfn_x(smfn) + i, !(memflags & MEMF_no_icache_flush));
+
+    return pg;
+
+ out_err:
+    while ( i-- )
+        pg[i].count_info = PGC_reserved | PGC_state_free;
+
+    spin_unlock(&heap_lock);
+
+    return NULL;
+}
+
+/*
+ * Acquire nr_mfns contiguous pages, starting at #smfn, of static memory,
+ * then assign them to one specific domain #d.
+ */
+int __init acquire_domstatic_pages(struct domain *d, mfn_t smfn,
+                                   unsigned long nr_mfns, unsigned int memflags)
+{
+    struct page_info *pg;
+
+    ASSERT(!in_irq());
+
+    pg = acquire_staticmem_pages(smfn, nr_mfns, memflags);
+    if ( !pg )
+        return -ENOENT;
+
+    if ( !d || (memflags & (MEMF_no_owner | MEMF_no_refcount)) )
+    {
+        /*
+         * Respective handling omitted here because right now
+         * acquired static memory is only for guest RAM.
+         */
+        ASSERT_UNREACHABLE();
+        return -EINVAL;
+    }
+
+    if ( assign_pages(pg, nr_mfns, d, memflags) )
+    {
+        free_staticmem_pages(pg, nr_mfns, memflags & MEMF_no_scrub);
+        return -EINVAL;
+    }
+
+    return 0;
+}
 #endif
 
 /*
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index ded74d29da..7b5e7b7f69 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -108,6 +108,9 @@ struct page_info
   /* Page is Xen heap? */
 #define _PGC_xen_heap     PG_shift(2)
 #define PGC_xen_heap      PG_mask(1, 2)
+  /* Page is reserved */
+#define _PGC_reserved     PG_shift(3)
+#define PGC_reserved      PG_mask(1, 3)
 /* ... */
 /* Page is broken? */
 #define _PGC_broken       PG_shift(7)
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 9d6a45174e..f8500f04d4 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -89,6 +89,8 @@ bool scrub_free_pages(void);
 /* These functions are for static memory */
 void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
                           bool need_scrub);
+int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned long nr_mfns,
+                            unsigned int memflags);
 #endif
 
 /* Map machine page range in Xen virtual address space. */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:53:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:53:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183860.332280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf6-00064w-02; Fri, 10 Sep 2021 02:53:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183860.332280; Fri, 10 Sep 2021 02:53:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWf5-00064R-Sb; Fri, 10 Sep 2021 02:53:07 +0000
Received: by outflank-mailman (input) for mailman id 183860;
 Fri, 10 Sep 2021 02:53:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWf4-0004S2-Rj
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:53:06 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.45]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 18b20c5c-a14e-4c69-a699-0e31d94047d8;
 Fri, 10 Sep 2021 02:52:57 +0000 (UTC)
Received: from AM0PR10CA0043.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::23)
 by VI1PR08MB5358.eurprd08.prod.outlook.com (2603:10a6:803:13c::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 02:52:55 +0000
Received: from VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:150:cafe::a9) by AM0PR10CA0043.outlook.office365.com
 (2603:10a6:20b:150::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:55 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT044.mail.protection.outlook.com (10.152.19.106) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:52:54 +0000
Received: ("Tessian outbound 8e26f7114b75:v103");
 Fri, 10 Sep 2021 02:52:54 +0000
Received: from 467429ba86a6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 DD9D45A3-6A48-45DF-8B9B-50F3A0CE39CE.1; 
 Fri, 10 Sep 2021 02:52:47 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 467429ba86a6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:47 +0000
Received: from AS8PR04CA0070.eurprd04.prod.outlook.com (2603:10a6:20b:313::15)
 by AM0PR08MB5060.eurprd08.prod.outlook.com (2603:10a6:208:15e::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 02:52:46 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::f9) by AS8PR04CA0070.outlook.office365.com
 (2603:10a6:20b:313::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:45 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:44 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:48 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18b20c5c-a14e-4c69-a699-0e31d94047d8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YidLo1ofxwAYNecKLtQzrrxTjqkOLuglQ7KQpw39T7U=;
 b=rvDi12Z9MGGj2E5HXuHnfM+VUtvlasDQioIhYxX58L+3aUr7eZ/gueqLFbLLjGPj2IACBOGDISnQ80ciw4E8cmJF9OrNITnKgg4CSNtNxne3obR3Kfmq/3m+wOPbdIJ8426XQs5wkZNXJHSrVzwucfZgrUuuaIwgMT+FRtMR6og=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a20cb15cde3001a7
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hvLWhXzjfi4T5GeKjASTGUQgMXOMfbgIpDR7BUBmBxnaL4PAaDo8SU5rhqvcBlSrc582nRkpRNHpfdxGHPOjBjNOSfgnRcLFimgGzzke4z5merUBaHkRS0iXpNQNSGmiWacvVzKLcJGUbUqW7rtVZxd7CnWYlA06L3E/OHcfMINWj9IC9mHuGTGuILenk/tzyvUJC0j8JtYG0t6+Ozltge9tq16Q52IIn5RrNwOwjwncGZapsZjI2UStJysd2DriwSDjH8j4WwMBUmKEW7qa4J9OqQuroicQsXCU7cRQCTshWzBmlTQ3+EgYNRNiKQcZOrzUj5pohWcbx+ZHCths5Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YidLo1ofxwAYNecKLtQzrrxTjqkOLuglQ7KQpw39T7U=;
 b=dBJfV9Uzw20+GbqZPr5l4heFWrNdmQxrZGt+jUPbt3BzHkRXBTOzBnb4kFbyMIXRWx6oDV+gPJs/akqfVJ9QBRjkGFZ1MSQI7kt2mbP8DEetXvBCDbON9TKMaen4QMk02OKwiwBer+HPmuAEjcdL7hpZmwdArsRrBNjd4lZvVI6NeZsFo5GYetkOeYM68hdzPNiGV9mfSdy0951EM0IpJE5hAxgQR9PUmbkGuzYFLly064so8gTia1tsE+j4q3pzt5rXyhl1B5FdjXHJhUNO0idBoNPcwzjrBaS83J1m0bC1bdaK2/pnvSVc320NoxfUMfqQbxziNifLfuTYQTBnxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YidLo1ofxwAYNecKLtQzrrxTjqkOLuglQ7KQpw39T7U=;
 b=rvDi12Z9MGGj2E5HXuHnfM+VUtvlasDQioIhYxX58L+3aUr7eZ/gueqLFbLLjGPj2IACBOGDISnQ80ciw4E8cmJF9OrNITnKgg4CSNtNxne3obR3Kfmq/3m+wOPbdIJ8426XQs5wkZNXJHSrVzwucfZgrUuuaIwgMT+FRtMR6og=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 4/7] xen/arm: static memory initialization
Date: Fri, 10 Sep 2021 02:52:12 +0000
Message-ID: <20210910025215.1671073-5-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ee647ff8-e2cd-4225-f2b6-08d9740616b5
X-MS-TrafficTypeDiagnostic: AM0PR08MB5060:|VI1PR08MB5358:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB5358872452B2CA37800519BAF7D69@VI1PR08MB5358.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:207;OLM:207;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 SMAmA19PCuYGuHyg9GzYGzyS/auodm99njZ/jgVio1hA/CqYLFMhJOCdSusERoF4Pnuf/0n/brQjw3kY1y5MQ1Pnj5+kisfA51cYUFJ8RN+GySXUnVZHHDc9tKs9FVZIGmKEZIcU3Fw6DDePXZ8f7GVUATgXsTO1K37dYYg3e4X0u/cmCt1hnnHogXe8jvCetNPvt6nn/1wVLsX0qB5hk+Tn4nWms/Wz1hblh1KYpS0ppIJynm56rMNxgcg+ZROmWtaSZlUT+k1bJouhC9FwogW/vq6vncubiqBwxcSfea7LVRBJaayDlpNsTNudlqNAIzl4p+V9+7D6yCeWhaJAJSxMgllispR2Gtcvp6WJiIJUERLuxQr2fJYVVzJROiSUS3yY5iTj0dlbR2LcMU/TLYC4omt+ofdZblHlygKLM6ENThi+6g6zmXKAS0at3zfpq6FH5r5+lwLsz6ROX3o3m0NPF5oHVacl8524w04Nvsqq8SeVfj6OhMgTAvr96QGoUSHXSaSkg+8wQZDfZKDnEuEQM0iEMuBpCvv3STQlroeZGUHgfor6PGPiVR50XYWTuGXmsL4tD53i3z9YIkL7KHQURxFk0I6PKa02WQCNcMkgpTvE65gBss/H4Ijdf4Rsr9wiFJA+kIy4XMyQ33AUdPjZggD7b4UaH9MBaDjajwOCeK98Z1tp8TaIYQIoNk6ncC36KTdtCevo9+qpi/3aXCuBLX+z6vY74lQg/UUcVTc=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(136003)(396003)(46966006)(36840700001)(7696005)(316002)(5660300002)(426003)(82740400003)(6666004)(36756003)(81166007)(82310400003)(356005)(110136005)(54906003)(478600001)(4326008)(70586007)(1076003)(2616005)(8936002)(186003)(336012)(26005)(70206006)(83380400001)(8676002)(2906002)(86362001)(47076005)(44832011)(36860700001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5060
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ee39b5d5-4f95-42c8-b97b-08d9740610aa
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dRgEMkURts6a72aqC7g80Vi9W0NfFmyXWDkIRfMhH2xbpZ2ERnFEMI5pGfvoJQW2Bq4UsXW8pH85YgfnLPihFV4mBT4Smv6rhSJz9JsmKhVwl7AdJcOQFubdnUm63IFN6Q7XYUCV4y3w1wPsyjw3uV5PnifzmQZUdHyOu2cUnsj0McOfyT2aXC6uHbjykdqOk+mIzVyCSPh1WA2mcG64utXAG3MkEZ7XBRw1KAFPopKcg+NhwdkasPK0xf516PbOZAVnYkVOnxX45p8tzFOMBceqABnvXjQ7f8054QwlSOMBWpGLW0W4ZcXe1PVXEafticCUhN83EZrs8TW6LT0e2y51wwbZZen/YqQTCmYWzXd8H5Cd1hxoF2TgXn0PhecjHQC5UE8NnE3iB08/oIAGm7rFMIcfl6VkOIJt6OeKfDnf+JP2Q0wRRMBBWlwoHKb6vZitVP32Jl/FdzoOQr3llFDgKcj15JByPGEMzMMkUKxJZpZdgrtNuAwnSqIN1ILcn13f7BjxSVTMmhXJ/w8+2/dTMwCEKy2z+XZs9ml49PS0tskxHt9W7v2Svt4g1928kxragHjB2JgScn0D0g7vR4z2NJ1gG/Eg2McPi5j/+BTtm7qFzeYeW03nwEdm8Xc/1e4K6a4e2eF2uC/AulXEyQmld8oEk13egCICi9AaIlVaMSvmcIjVUffzdPBTJclbTUhz6SVX6zYgJ9TvO0HCxw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(316002)(107886003)(54906003)(47076005)(336012)(2906002)(508600001)(426003)(81166007)(4326008)(36756003)(83380400001)(86362001)(8936002)(7696005)(110136005)(5660300002)(8676002)(186003)(1076003)(82310400003)(36860700001)(44832011)(70206006)(70586007)(6666004)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:54.8145
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ee647ff8-e2cd-4225-f2b6-08d9740616b5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5358

This patch introduces static memory initialization, during system boot-up.

The new function init_staticmem_pages is responsible for static memory
initialization.

Helper free_staticmem_pages is the equivalent of free_heap_pages, to free
nr_mfns pages of static memory.

This commit also introduces a new CONFIG_STATIC_MEMORY option to wrap all
static-allocation-related code.

Put asynchronously scrubbing pages of static memory in TODO list.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/setup.c    | 27 +++++++++++++++++++++++++++
 xen/common/Kconfig      | 13 +++++++++++++
 xen/common/page_alloc.c | 21 +++++++++++++++++++++
 xen/include/xen/mm.h    |  6 ++++++
 4 files changed, 67 insertions(+)

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 63a908e325..5be7f2b0c2 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -609,6 +609,29 @@ static void __init init_pdx(void)
     }
 }
 
+/* Static memory initialization */
+static void __init init_staticmem_pages(void)
+{
+#ifdef CONFIG_STATIC_MEMORY
+    unsigned int bank;
+
+    for ( bank = 0 ; bank < bootinfo.reserved_mem.nr_banks; bank++ )
+    {
+        if ( bootinfo.reserved_mem.bank[bank].xen_domain )
+        {
+            mfn_t bank_start = _mfn(PFN_UP(bootinfo.reserved_mem.bank[bank].start));
+            unsigned long bank_pages = PFN_DOWN(bootinfo.reserved_mem.bank[bank].size);
+            mfn_t bank_end = mfn_add(bank_start, bank_pages);
+
+            if ( mfn_x(bank_end) <= mfn_x(bank_start) )
+                return;
+
+            free_staticmem_pages(mfn_to_page(bank_start), bank_pages, false);
+        }
+    }
+#endif
+}
+
 #ifdef CONFIG_ARM_32
 static void __init setup_mm(void)
 {
@@ -736,6 +759,8 @@ static void __init setup_mm(void)
     /* Add xenheap memory that was not already added to the boot allocator. */
     init_xenheap_pages(mfn_to_maddr(xenheap_mfn_start),
                        mfn_to_maddr(xenheap_mfn_end));
+
+    init_staticmem_pages();
 }
 #else /* CONFIG_ARM_64 */
 static void __init setup_mm(void)
@@ -789,6 +814,8 @@ static void __init setup_mm(void)
 
     setup_frametable_mappings(ram_start, ram_end);
     max_page = PFN_DOWN(ram_end);
+
+    init_staticmem_pages();
 }
 #endif
 
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11a..3558be0dbc 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -67,6 +67,19 @@ config MEM_ACCESS
 config NEEDS_LIBELF
 	bool
 
+config STATIC_MEMORY
+	bool "Static Allocation Support (UNSUPPORTED)" if UNSUPPORTED
+	depends on ARM
+	help
+	  Static Allocation refers to system or sub-system(domains) for
+	  which memory areas are pre-defined by configuration using physical
+	  address ranges.
+
+	  When enabled, memory can be statically allocated to a domain using
+	  the property "xen,static-mem" defined in the domain configuration.
+
+	  If unsure, say N.
+
 menu "Speculative hardening"
 
 config SPECULATIVE_HARDEN_ARRAY
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index a3ee5eca9e..ba7adc80db 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2604,6 +2604,27 @@ struct domain *get_pg_owner(domid_t domid)
     return pg_owner;
 }
 
+#ifdef CONFIG_STATIC_MEMORY
+/* Equivalent of free_heap_pages to free nr_mfns pages of static memory. */
+void __init free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
+                                 bool need_scrub)
+{
+    mfn_t mfn = page_to_mfn(pg);
+    unsigned long i;
+
+    for ( i = 0; i < nr_mfns; i++ )
+    {
+        mark_page_free(&pg[i], mfn_add(mfn, i));
+
+        if ( need_scrub )
+        {
+            /* TODO: asynchronous scrubbing for pages of static memory. */
+            scrub_one_page(pg);
+        }
+    }
+}
+#endif
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 667f9dac83..8e8fb5a615 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -85,6 +85,12 @@ bool scrub_free_pages(void);
 } while ( false )
 #define FREE_XENHEAP_PAGE(p) FREE_XENHEAP_PAGES(p, 0)
 
+#ifdef CONFIG_STATIC_MEMORY
+/* These functions are for static memory */
+void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
+                          bool need_scrub);
+#endif
+
 /* Map machine page range in Xen virtual address space. */
 int map_pages_to_xen(
     unsigned long virt,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:53:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:53:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183878.332291 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWfQ-0007aA-C3; Fri, 10 Sep 2021 02:53:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183878.332291; Fri, 10 Sep 2021 02:53:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWfQ-0007a0-8f; Fri, 10 Sep 2021 02:53:28 +0000
Received: by outflank-mailman (input) for mailman id 183878;
 Fri, 10 Sep 2021 02:53:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWfO-0004S2-Sb
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:53:26 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.67]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed32ae24-4cbf-446e-a3c5-fa9207010020;
 Fri, 10 Sep 2021 02:53:01 +0000 (UTC)
Received: from AM0PR10CA0067.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:15::20)
 by DU2PR08MB7325.eurprd08.prod.outlook.com (2603:10a6:10:2e4::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:59 +0000
Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:15:cafe::ce) by AM0PR10CA0067.outlook.office365.com
 (2603:10a6:208:15::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:52:59 +0000
Received: ("Tessian outbound e27daf245730:v103");
 Fri, 10 Sep 2021 02:52:58 +0000
Received: from 34fcc7f94187.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F7C93D37-C100-4FDC-8380-E3355918ACA6.1; 
 Fri, 10 Sep 2021 02:52:52 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 34fcc7f94187.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:52 +0000
Received: from AS8PR04CA0074.eurprd04.prod.outlook.com (2603:10a6:20b:313::19)
 by VE1PR08MB5838.eurprd08.prod.outlook.com (2603:10a6:800:1a7::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 02:52:48 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::2) by AS8PR04CA0074.outlook.office365.com
 (2603:10a6:20b:313::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:48 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:48 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:51 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed32ae24-4cbf-446e-a3c5-fa9207010020
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qka5FohwRchLmsQOFo9yjoRS80zQ82TnqHEM4A+5Y08=;
 b=6jMJB6WLgu5Jruu1Bq3Evw3lOvPlUOPorN4UB2DbybVwGSxHzrRaZ2bF96yYrXy7KW5HmC3bYI5zpQSno54u5voJ1p1myAenEBDlOvRnBIBDW3WKoAjjb2R7x5lkQaG0ogkXJU1l1gOynWWPGb274jz3PrFvSvKzmudB8ln6Tr4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: bee34eeaa078c05d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RuPjdp1LkdQs3N94bhUpqCKd2cRAGtBnjd6as9qImAd4WiIoNB1/+7HWhXanxE/BAYB/jXxfawmzr7N5m3QQGCS+2WoY424jFEHBr8adnfeIljxXzEjXblUtOg2mahGziCBI3EwqrJ4dTlw8wqaKkvcQXegpaPqmqqkDl5qw/RavHIgd94PBghN3PxUDlxEMt523PjsPM+mBh3NTyWCWsTS2EUyEw7k2TNGDn43gaZtck7Lp7gq3zJoJHQnVr2vUtveUEMwwIociMXwpBr2wdaYH9anECEojLaSxft3itBeYWajc2WSsm8sEHj8BesZ1vMyRZaRjXs5RKUrVljmjBg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Qka5FohwRchLmsQOFo9yjoRS80zQ82TnqHEM4A+5Y08=;
 b=G0lqfNRDsaVnFOwwYgPLG+rDDLY5rBJpfk+CKmnHwDGazr16oFC1zurRNbUgaAC4MqGY6yraqE5VdsoepR6P+9k9EWoQ0Dw2p+5nChzdVfOXzGJ+hdkgkddftd1M5YoRR/hrbtin50VvLuWYa0aOkANrS6LFwrFNtvOVLzwqc/QK62GJOcGZDaKUIKQKWEMBZVrO6IKfhk7fg7NI0DI2eCbaXBJq4mge293xGO+J3WEx8ivt1VojuGjbpsWinbvf2cUfxXqJa2Q4sqrvCc1VEW93ubvx0gKlko+1mlWs+wSdOFCiW9IG3/YiEg9uzmFtYKDlB36hJ3/CAvAhvDFk2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qka5FohwRchLmsQOFo9yjoRS80zQ82TnqHEM4A+5Y08=;
 b=6jMJB6WLgu5Jruu1Bq3Evw3lOvPlUOPorN4UB2DbybVwGSxHzrRaZ2bF96yYrXy7KW5HmC3bYI5zpQSno54u5voJ1p1myAenEBDlOvRnBIBDW3WKoAjjb2R7x5lkQaG0ogkXJU1l1gOynWWPGb274jz3PrFvSvKzmudB8ln6Tr4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new function assign_page
Date: Fri, 10 Sep 2021 02:52:13 +0000
Message-ID: <20210910025215.1671073-6-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d82fe461-5d74-49e7-c1b5-08d974061950
X-MS-TrafficTypeDiagnostic: VE1PR08MB5838:|DU2PR08MB7325:
X-Microsoft-Antispam-PRVS:
	<DU2PR08MB73250D6C88353CE7B3D38964F7D69@DU2PR08MB7325.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Z+FRo+RfN5ZisZvPsYpoe2Cu+G9Qf/vHP9BH5spmL0ZDah5VzmXZZ1DicNSiDm6VbyRyiMAE/431/EDRisP9Jr75JFMGedJNDH3MIfyGBaQjCPJNrXolRqOsDvwNVlsBHvdubhA89b9tmxGP2KpWffSaFaTPpHuobUtQr+fwndIvjr09H1SgJVWssdGDWSCPLuA5hwTpz7j1S9V3FTcXnPc6k2fo5iTKAfhXagBkOu5Yc8C7gmCDDgZ79Xif2R2e2+Qwl7gdQgTEWYb3oy+T7JiYi29ojqABkO4y2vqYhTK5lbfkUIxAxO2FDXLY7ukBzE+G0kF7YxIhxp1oumSnZomyebX2RLwkDKbZYxd13Uori4hrtqFlFj3ZjS+BBJmwIr5W8OT7ikyU/j8C3HfLSTtZ5Y/AHZ2lQkqdj+zBYGs+FJlUG+mIbwhcgHkWcR/nSBd8bOAXUIOCk+38RMy1fX5m4pUSzbcvuoieSPNrJRV4OmFhHawDJC7VZg7Ehl5gSABTdp/nC6zp50DrhTSMcnGd7Xoa9cS3TM44xCZCvPk1QJd2fHVSnwserbb29aJYDqRblqsNaI3ImPbUHCJ47O3j/BbhyTMiybwDgHXmmc+SXhoWYiHkULweiuI98f/U1fVh2Gkb1x53qPOphZteWk1hcKtdFpfxyncIQPGV5F/GE9fPtCJw8GHzWP7Ft15swz2ZkO9HN8SHv5geIPeNhaO432jdqu3sSkjUrbOcgLU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(39860400002)(136003)(36840700001)(46966006)(1076003)(83380400001)(426003)(8676002)(36756003)(478600001)(8936002)(2906002)(316002)(44832011)(82310400003)(47076005)(186003)(54906003)(70206006)(336012)(2616005)(86362001)(110136005)(36860700001)(70586007)(6666004)(26005)(356005)(4326008)(5660300002)(7696005)(81166007)(82740400003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5838
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	17b54f2f-5173-45f4-ba8f-08d9740612c8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	p9d9q0VWXuc7PauQJZRdcIZWexwBjmWhcPMqVQY2LkjQ9qFi6qwrRt1d03VI35biHsyd7FpfEKADBOnEbnZMJ4qPnV4KBpVOXpxzkhgRSVxX4XSfkdE8u34Oi1Iy1pqvBqgt5oIdmli/K5qPxOTtp7zPtei48RUM0tMITxDDNgm/O0fgiMBOF4X2Nr5zXgUiznzS3syQWSa3U/bMDUkxIPLvk6x5gH/AfQcpA44YIQSM68bcV0W/G7TMfjpI/Sk2eiGQR+3/eI+er+QUfDiT4U+iAdDVBYdEy+8XaijqiWgBlhI66t60BfybNumYct8xDAEpIfevWxE7yHlGu7PrcWq7Uwh5preNr37zUMhnEfLiberZQGSqPPxB0ndBg7GYTJsH0eikjPqhdIN3rLcoAjfD2gkDJM+hlI3qhBqDAZ2pVN/lLC+upwhG3FxdYou7KRsJurAGZ14cULZpO8LPJwF5BJTWZVej9g74e9Nxh1YK/QB3ZQ5kj6rd9AiD5jpD28ax0B9xNhizXFLdn80wKYqsFAwy7Fxupo6+pGAtjlcPqT0IjxyW9WAh4jfG8jG395gMiYngEkQxNTinK6xgn4kRvJoHuRSmNXgfaFVgA70AXdXRYYzgRNagUKBmyIMpnu0cDF24zxd13xVhGKz9WRVFt+FoRBS8DJd/LCOX6JjrS/WENABM1XCZFAQqaKvb
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(346002)(376002)(39860400002)(136003)(46966006)(36840700001)(7696005)(82310400003)(54906003)(186003)(70206006)(1076003)(70586007)(336012)(8676002)(82740400003)(47076005)(81166007)(4326008)(86362001)(36756003)(110136005)(316002)(2906002)(44832011)(107886003)(5660300002)(2616005)(6666004)(8936002)(426003)(36860700001)(478600001)(26005)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:52:59.1775
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d82fe461-5d74-49e7-c1b5-08d974061950
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB7325

In order to deal with the trouble of count-to-order conversion when page number
is not in a power-of-two, this commit re-define assign_pages for nr pages and
assign_page for original page with a single order.

Backporting confusion could be helped by altering the order of assign_pages
parameters, such that the compiler would point out that adjustments at call
sites are needed.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/x86/pv/dom0_build.c |  2 +-
 xen/common/grant_table.c     |  2 +-
 xen/common/memory.c          |  6 +++---
 xen/common/page_alloc.c      | 23 ++++++++++++++---------
 xen/include/xen/mm.h         |  6 ++++++
 5 files changed, 25 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index d7f9e04b28..77efd3918c 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -568,7 +568,7 @@ int __init dom0_construct_pv(struct domain *d,
         else
         {
             while ( count-- )
-                if ( assign_pages(d, mfn_to_page(_mfn(mfn++)), 0, 0) )
+                if ( assign_pages(mfn_to_page(_mfn(mfn++)), 1, d, 0) )
                     BUG();
         }
         initrd->mod_end = 0;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index e80f8d044d..fe1fc11b22 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -2358,7 +2358,7 @@ gnttab_transfer(
          * is respected and speculative execution is blocked accordingly
          */
         if ( unlikely(!evaluate_nospec(okay)) ||
-            unlikely(assign_pages(e, page, 0, MEMF_no_refcount)) )
+            unlikely(assign_pages(page, 1, e, MEMF_no_refcount)) )
         {
             bool drop_dom_ref;
 
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 74babb0bd7..63642278fd 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -728,8 +728,8 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
         /* Assign each output page to the domain. */
         for ( j = 0; (page = page_list_remove_head(&out_chunk_list)); ++j )
         {
-            if ( assign_pages(d, page, exch.out.extent_order,
-                              MEMF_no_refcount) )
+            if ( assign_page(page, exch.out.extent_order, d,
+                             MEMF_no_refcount) )
             {
                 unsigned long dec_count;
                 bool_t drop_dom_ref;
@@ -797,7 +797,7 @@ static long memory_exchange(XEN_GUEST_HANDLE_PARAM(xen_memory_exchange_t) arg)
      * cleared PGC_allocated.
      */
     while ( (page = page_list_remove_head(&in_chunk_list)) )
-        if ( assign_pages(d, page, 0, MEMF_no_refcount) )
+        if ( assign_pages(page, 1, d, MEMF_no_refcount) )
         {
             BUG_ON(!d->is_dying);
             free_domheap_page(page);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index ba7adc80db..2aa8edac8c 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
 
 
 int assign_pages(
-    struct domain *d,
     struct page_info *pg,
-    unsigned int order,
+    unsigned long nr,
+    struct domain *d,
     unsigned int memflags)
 {
     int rc = 0;
@@ -2281,7 +2281,7 @@ int assign_pages(
     {
         unsigned int extra_pages = 0;
 
-        for ( i = 0; i < (1ul << order); i++ )
+        for ( i = 0; i < nr; i++ )
         {
             ASSERT(!(pg[i].count_info & ~PGC_extra));
             if ( pg[i].count_info & PGC_extra )
@@ -2290,18 +2290,18 @@ int assign_pages(
 
         ASSERT(!extra_pages ||
                ((memflags & MEMF_no_refcount) &&
-                extra_pages == 1u << order));
+                extra_pages == nr));
     }
 #endif
 
     if ( pg[0].count_info & PGC_extra )
     {
-        d->extra_pages += 1u << order;
+        d->extra_pages += nr;
         memflags &= ~MEMF_no_refcount;
     }
     else if ( !(memflags & MEMF_no_refcount) )
     {
-        unsigned int tot_pages = domain_tot_pages(d) + (1 << order);
+        unsigned int tot_pages = domain_tot_pages(d) + nr;
 
         if ( unlikely(tot_pages > d->max_pages) )
         {
@@ -2313,10 +2313,10 @@ int assign_pages(
     }
 
     if ( !(memflags & MEMF_no_refcount) &&
-         unlikely(domain_adjust_tot_pages(d, 1 << order) == (1 << order)) )
+         unlikely(domain_adjust_tot_pages(d, nr) == nr) )
         get_knownalive_domain(d);
 
-    for ( i = 0; i < (1 << order); i++ )
+    for ( i = 0; i < nr; i++ )
     {
         ASSERT(page_get_owner(&pg[i]) == NULL);
         page_set_owner(&pg[i], d);
@@ -2331,6 +2331,11 @@ int assign_pages(
     return rc;
 }
 
+int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
+                unsigned int memflags)
+{
+    return assign_pages(pg, 1UL << order, d, memflags);
+}
 
 struct page_info *alloc_domheap_pages(
     struct domain *d, unsigned int order, unsigned int memflags)
@@ -2373,7 +2378,7 @@ struct page_info *alloc_domheap_pages(
                 pg[i].count_info = PGC_extra;
             }
         }
-        if ( assign_pages(d, pg, order, memflags) )
+        if ( assign_page(pg, order, d, memflags) )
         {
             free_heap_pages(pg, order, memflags & MEMF_no_scrub);
             return NULL;
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 8e8fb5a615..9d6a45174e 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -132,9 +132,15 @@ int query_page_offline(mfn_t mfn, uint32_t *status);
 void heap_init_late(void);
 
 int assign_pages(
+    struct page_info *pg,
+    unsigned long nr,
     struct domain *d,
+    unsigned int memflags);
+
+int assign_page(
     struct page_info *pg,
     unsigned int order,
+    struct domain *d,
     unsigned int memflags);
 
 /* Dump info to serial console */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 02:53:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 02:53:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183887.332302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWfZ-00082Z-Su; Fri, 10 Sep 2021 02:53:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183887.332302; Fri, 10 Sep 2021 02:53:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWfZ-00082M-NY; Fri, 10 Sep 2021 02:53:37 +0000
Received: by outflank-mailman (input) for mailman id 183887;
 Fri, 10 Sep 2021 02:53:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I6Hc=OA=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mOWfY-0004S2-TE
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:53:36 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.75]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c35f1c2e-438f-4a00-a602-2221e6a25293;
 Fri, 10 Sep 2021 02:53:05 +0000 (UTC)
Received: from AM6PR08CA0015.eurprd08.prod.outlook.com (2603:10a6:20b:b2::27)
 by AM5PR0802MB2547.eurprd08.prod.outlook.com (2603:10a6:203:a0::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 02:53:03 +0000
Received: from VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:b2:cafe::b7) by AM6PR08CA0015.outlook.office365.com
 (2603:10a6:20b:b2::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:53:02 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT051.mail.protection.outlook.com (10.152.19.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 02:53:02 +0000
Received: ("Tessian outbound a8bfe25d7364:v103");
 Fri, 10 Sep 2021 02:53:02 +0000
Received: from 7bf717e4cf3e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EEC0EDA1-6B1F-42C1-846C-A46BDE7F37EB.1; 
 Fri, 10 Sep 2021 02:52:55 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7bf717e4cf3e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 02:52:55 +0000
Received: from AS8PR04CA0069.eurprd04.prod.outlook.com (2603:10a6:20b:313::14)
 by VE1PR08MB5053.eurprd08.prod.outlook.com (2603:10a6:803:10e::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 02:52:53 +0000
Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::9c) by AS8PR04CA0069.outlook.office365.com
 (2603:10a6:20b:313::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:53 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4478.19 via Frontend Transport; Fri, 10 Sep 2021 02:52:52 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Fri, 10 Sep
 2021 02:52:56 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Fri, 10
 Sep 2021 02:52:49 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 10 Sep 2021 02:52:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c35f1c2e-438f-4a00-a602-2221e6a25293
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/NnnHPaDcIoXHeQJGuPrSxCo6z4bgyFzyrpk+3oSabk=;
 b=tKxlwijYVMSRjesKxhZ4iIWhpzMJoLYnKrxktAAu/KtDeccIRinN09OvWEa0K5oBFXVj8FmWgvXBhGOL8WNMfJrWDqV8R6ih3NYd7bCVIKSEcGnkjDh3k/NBRdqOdmTJVrzMf1K7ct+ZZBTBpdmbTdY1/GgYIA/jAVZm9B6Wlmk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cbfe770c9afe4a32
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nH8lg8Rv0Nv93ZNYv3aIo8GI4J2k4MxNZyMunM3JWn9zUditGYIFI6mTaqb4zdSyoM3fVVkApOIyfqTklyaYl+ydJjIqcLegXJZDmi8Xqv1JkVS5oA0JCKg1Zd4FXp7OzAPOb9TZG2PMmx8cYRBY+ciG4HHwVOiPP6GOEQ4qMmPZhGojvNvjEmz48fG9HWXr/0k4GmHMSvsZdNAS72kukZmp1S5lntKDJox6w/TfTSQ8Tw8ONURf0qmXnqEs7Yth1FfBT4NUMcBrGZw8kJCWvTmzjsQ0M/EmTNUXdHIcXA7XPGGftl3adOgHfFJbfLgOBK4RXkgMeNa18pSfD0TQMg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/NnnHPaDcIoXHeQJGuPrSxCo6z4bgyFzyrpk+3oSabk=;
 b=lxjI83gk2TO8VwD63j/5FHnh5x171Jo53s4cz1xSenI+BdNv5f6F+HoWhl4dhBDDc2fpJG8Xqfsm4MGpF+nlmFMItP6SDcbxT0eFTM7w+lQTvVOXeIxPZnYTLscnZ4O9x/8pCmJogksqD2zJNPzCmMuoecchfHBQhd/MI7Q4mtLJcFKF+ggSPSOQ8bcmLmbkHKU8Mea2NEAlYdajgcFq+OBj0AUhlRxdodSD6a8WoZdyQ7BCp0NzgSL7QKJFFPKb4XDZog9b4RyQKDgPrV7lAXMxigKhvU3bAjA6Z0sX3oZ+UAO1sR/gJDpu2/ThCfWESkR5kwAjHK9WPqR3x9F8pg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/NnnHPaDcIoXHeQJGuPrSxCo6z4bgyFzyrpk+3oSabk=;
 b=tKxlwijYVMSRjesKxhZ4iIWhpzMJoLYnKrxktAAu/KtDeccIRinN09OvWEa0K5oBFXVj8FmWgvXBhGOL8WNMfJrWDqV8R6ih3NYd7bCVIKSEcGnkjDh3k/NBRdqOdmTJVrzMf1K7ct+ZZBTBpdmbTdY1/GgYIA/jAVZm9B6Wlmk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <jbeulich@suse.com>
Subject: [PATCH v7 7/7] xen/arm: introduce allocate_static_memory
Date: Fri, 10 Sep 2021 02:52:15 +0000
Message-ID: <20210910025215.1671073-8-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210910025215.1671073-1-penny.zheng@arm.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8fd965e8-6750-4471-159b-08d974061b62
X-MS-TrafficTypeDiagnostic: VE1PR08MB5053:|AM5PR0802MB2547:
X-Microsoft-Antispam-PRVS:
	<AM5PR0802MB25470D2845DCF47354A6CAF4F7D69@AM5PR0802MB2547.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:644;OLM:644;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 kDxhc3askg4m2P4IgpQNEJz8f+qGLfhKQJ4gVw/9odCd+hg84KH8RtQvQZ9T0lixbwhf1+RYnbyl4YHsHLq6Qd0TqXTVKrkutvmN6dEPYgNxFFQdHzhG+6AbgdHBwrlc0Z6L7QRqcEyy4ir1WZFDGj6cnSWwpreuKl3IvGxKQxXTd3ZyupUPBOlLl0oizpGzb3mF5Qyu+XQSUeVrym2AjGfaNE95KSebQaHGOfuMpJ+Poc+HzIKUJrOyBWftbj71JfUuCe4mHneLIFIHdFudaJwI2rataXYCuq6wmJyL+KOlC94LWLKP418BbWbpjwfNRvWnJd3A7ZsfCNA4GsR1lnZRuWHapbEJiGvCz/LxdqM4MvQ/3Pqjl0GckcEX758R4F2jlvKhhg4QVDcKAzs4uIk5YxCfw7+eLPUaslN0iL8ZCMBIgEr2BVhz7KZl2jKEUASiSmB1Vx1zvLqZ1QS6IoLWpPRbx7qw7hI9fCxdn7icmknODWl/lmuXi+HXokGdhvIWKFBBSEaXmO3Ozsepq3a+fYGVpoF5GHFHGoHZbYbUPzWMLtpIzvZhMX61skxMBtWeM9sIfsircSOhXaxAUqUIHZU2lwGjbYRySByCG9br1hcBZisgb9S9E8yMh+8rH5oV16ybCOa94GjTXT1uJgwK9sEh5YjNY3OmRs/iyYgnlv8Num9gyrrf+muQsvQbxqMLl1v3ol12LjlMfzIp0N6HR4DALXFLSoqx+1LK8Wk=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(136003)(396003)(346002)(376002)(39860400002)(46966006)(36840700001)(8936002)(82310400003)(82740400003)(70586007)(5660300002)(110136005)(81166007)(83380400001)(26005)(6666004)(86362001)(36756003)(1076003)(47076005)(2906002)(70206006)(426003)(44832011)(2616005)(356005)(7696005)(4326008)(186003)(36860700001)(336012)(316002)(8676002)(478600001)(54906003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5053
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	14892662-ccf3-43a0-2344-08d974061579
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YRoBAxkRqgnCeOrBFbjLIRhw7eyklvZUFuEiO0rmu++CPegIJKGBS34nwu4R15aY2gAIetz6QGtiMUIn5FKj3uGBbJmpIj710tAKGo8Cyo4WdMD96zPY5peKatZHlJJlYunFrRGPB6Vy1TRWZ1fvpmpH3xiDNAv0jpuqrP706aSctKMzjespoSTTm3jcefahEs0Jn9AJJkCPsAp7VWzwoOaWAO4s7xCUZr5Kn+u8q44+iJHAsT93twTLvUJbpsSb8xjDy//zWWOqL4yuazjpdYK1pwmxucgnXxHk6lhjFzQ0w7BO06YLRHJgGZU+jO0gHw8cW98uWzZKzoppRavSXr+/MonsTlcTxAJr4Y0dPfuyYZSuEYr6n/wEf0VYiT2hXWyLIe577uolyW6+KLdhj98C5ovN1f/PwrqvCtAXaUtoBOXYwHKS4oo8LhcH9B8xtpmLCN0+2+0QDkYxDAjkVRTPtzRRpXNFkxRXsa/N+KdrmMzRW3Glh3z0xcHwPqtklDKAMfZvER5DZLiQqXFndjwUxVb2nWsAHZMrQSeYesRflNNHp3gIdk9lz96AD6uZZdroouXNVXlF38Ghge1MuM7aHtxfwF8xACJnfC3Xn/9czMSmkB1NoOWoD575TDxU+SleNmTKPJXw27e5uR/pyyBviJ94pV41rmt+OYKF0Z4rzpARsWV7N18BLEpo5BQUyOp/sR6KeW7YUFBK2I0Spw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8936002)(81166007)(36756003)(26005)(7696005)(2616005)(107886003)(82310400003)(186003)(47076005)(36860700001)(83380400001)(508600001)(110136005)(54906003)(336012)(5660300002)(70586007)(70206006)(316002)(86362001)(44832011)(8676002)(426003)(6666004)(2906002)(4326008)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 02:53:02.6266
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fd965e8-6750-4471-159b-08d974061b62
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2547

This commit introduces a new function allocate_static_memory to allocate
static memory as guest RAM for domains on Static Allocation.

It uses acquire_domstatic_pages to acquire pre-configured static memory
for the domain, and uses guest_physmap_add_pages to set up the P2M table.
These pre-defined static memory banks shall be mapped to the usual guest
memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
xen/include/public/arch-arm.h.

In order to deal with the trouble of count-to-order conversion when page number
is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
a new function guest_physmap_add_pages to cope with adding guest RAM p2m
mapping with nr_pages.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/arch/arm/domain_build.c | 161 +++++++++++++++++++++++++++++++++++-
 xen/arch/arm/p2m.c          |   7 +-
 xen/include/asm-arm/p2m.h   |  11 +++
 3 files changed, 173 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d1c0..62ab7d0ead 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -480,6 +480,162 @@ fail:
           (unsigned long)kinfo->unassigned_mem >> 10);
 }
 
+#ifdef CONFIG_STATIC_MEMORY
+static bool __init append_static_memory_to_bank(struct domain *d,
+                                                struct membank *bank,
+                                                mfn_t smfn,
+                                                paddr_t size)
+{
+    int res;
+    unsigned int nr_pages = PFN_DOWN(size);
+    /* Infer next GFN. */
+    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
+
+    res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res);
+        return false;
+    }
+
+    bank->size = bank->size + size;
+
+    return true;
+}
+
+/* Allocate memory from static memory as RAM for one specific domain d. */
+static void __init allocate_static_memory(struct domain *d,
+                                          struct kernel_info *kinfo,
+                                          const struct dt_device_node *node)
+{
+    const struct dt_property *prop;
+    u32 addr_cells, size_cells, reg_cells;
+    unsigned int nr_banks, gbank, bank = 0;
+    const uint64_t rambase[] = GUEST_RAM_BANK_BASES;
+    const uint64_t ramsize[] = GUEST_RAM_BANK_SIZES;
+    const __be32 *cell;
+    u64 tot_size = 0;
+    paddr_t pbase, psize, gsize;
+    mfn_t smfn;
+    int res;
+
+    prop = dt_find_property(node, "xen,static-mem", NULL);
+    if ( !dt_property_read_u32(node, "#xen,static-mem-address-cells",
+                               &addr_cells) )
+    {
+        printk(XENLOG_ERR
+               "%pd: failed to read \"#xen,static-mem-address-cells\".\n", d);
+        goto fail;
+    }
+
+    if ( !dt_property_read_u32(node, "#xen,static-mem-size-cells",
+                               &size_cells) )
+    {
+        printk(XENLOG_ERR
+               "%pd: failed to read \"#xen,static-mem-size-cells\".\n", d);
+        goto fail;
+    }
+    reg_cells = addr_cells + size_cells;
+
+    /*
+     * The static memory will be mapped in the guest at the usual guest memory
+     * addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
+     * xen/include/public/arch-arm.h.
+     */
+    gbank = 0;
+    gsize = ramsize[gbank];
+    kinfo->mem.bank[gbank].start = rambase[gbank];
+
+    cell = (const __be32 *)prop->value;
+    nr_banks = (prop->length) / (reg_cells * sizeof (u32));
+
+    for ( ; bank < nr_banks; bank++ )
+    {
+        device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
+        ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
+
+        smfn = maddr_to_mfn(pbase);
+        res = acquire_domstatic_pages(d, smfn, PFN_DOWN(psize), 0);
+        if ( res )
+        {
+            printk(XENLOG_ERR
+                   "%pd: failed to acquire static memory: %d.\n", d, res);
+            goto fail;
+        }
+
+        printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
+               d, bank, pbase, pbase + psize);
+
+        while ( 1 )
+        {
+            /* Map as much as possible the static range to the guest bank */
+            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
+                                               min(psize, gsize)) )
+                goto fail;
+
+            /*
+             * The current physical bank is fully mapped.
+             * Handle the next physical bank.
+             */
+            if ( gsize >= psize )
+            {
+                gsize = gsize - psize;
+                break;
+            }
+            /*
+             * When current guest bank is not enough to map, exhaust
+             * the current one and seek to the next.
+             * Before seeking to the next, check if we still have available
+             * guest bank.
+             */
+            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
+            {
+                printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
+                goto fail;
+            }
+            else
+            {
+                psize = psize - gsize;
+                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
+                /* Update to the next guest bank. */
+                gbank++;
+                gsize = ramsize[gbank];
+                kinfo->mem.bank[gbank].start = rambase[gbank];
+            }
+        }
+
+        tot_size += psize;
+    }
+
+    kinfo->mem.nr_banks = ++gbank;
+
+    kinfo->unassigned_mem -= tot_size;
+    /*
+     * The property 'memory' should match the amount of memory given to the
+     * guest.
+     * Currently, it is only possible to either acquire static memory or let
+     * Xen allocate. *Mixing* is not supported.
+     */
+    if ( kinfo->unassigned_mem )
+    {
+        printk(XENLOG_ERR
+               "Size of \"memory\" property doesn't match up with the sum-up of \"xen,static-mem\". Unsupported configuration.\n");
+        goto fail;
+    }
+
+    return;
+
+ fail:
+    panic("Failed to allocate requested static memory for domain %pd.", d);
+}
+#else
+static void __init allocate_static_memory(struct domain *d,
+                                          struct kernel_info *kinfo,
+                                          const struct dt_device_node *node)
+{
+}
+#endif
+
 static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                                    const struct dt_device_node *node)
 {
@@ -2453,7 +2609,10 @@ static int __init construct_domU(struct domain *d,
     /* type must be set before allocate memory */
     d->arch.type = kinfo.type;
 #endif
-    allocate_memory(d, &kinfo);
+    if ( !dt_find_property(node, "xen,static-mem", NULL) )
+        allocate_memory(d, &kinfo);
+    else
+        allocate_static_memory(d, &kinfo, node);
 
     rc = prepare_dtb_domU(d, &kinfo);
     if ( rc < 0 )
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index eff9a105e7..6e01e83967 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1293,11 +1293,8 @@ out:
     return resolved;
 }
 
-static inline int p2m_insert_mapping(struct domain *d,
-                                     gfn_t start_gfn,
-                                     unsigned long nr,
-                                     mfn_t mfn,
-                                     p2m_type_t t)
+int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
+                       mfn_t mfn, p2m_type_t t)
 {
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     int rc;
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 6a2108398f..f885cc522b 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -300,6 +300,9 @@ int map_dev_mmio_region(struct domain *d,
                         unsigned long nr,
                         mfn_t mfn);
 
+int p2m_insert_mapping(struct domain *d, gfn_t start_gfn, unsigned long nr,
+                       mfn_t mfn, p2m_type_t t);
+
 int guest_physmap_add_entry(struct domain *d,
                             gfn_t gfn,
                             mfn_t mfn,
@@ -315,6 +318,14 @@ static inline int guest_physmap_add_page(struct domain *d,
     return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
 }
 
+static inline int guest_physmap_add_pages(struct domain *d,
+                                          gfn_t gfn,
+                                          mfn_t mfn,
+                                          unsigned int nr_pages)
+{
+    return p2m_insert_mapping(d, gfn, nr_pages, mfn, p2m_ram_rw);
+}
+
 mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn);
 
 /* Look up a GFN and take a reference count on the backing page. */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 03:12:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 03:12:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183920.332319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWxf-0003E5-GB; Fri, 10 Sep 2021 03:12:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183920.332319; Fri, 10 Sep 2021 03:12:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOWxf-0003Dy-Cd; Fri, 10 Sep 2021 03:12:19 +0000
Received: by outflank-mailman (input) for mailman id 183920;
 Fri, 10 Sep 2021 03:12:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OOxz=OA=linaro.org=takahiro.akashi@srs-us1.protection.inumbo.net>)
 id 1mOWxe-0003Dk-3l
 for xen-devel@lists.xen.org; Fri, 10 Sep 2021 03:12:18 +0000
Received: from mail-pg1-x530.google.com (unknown [2607:f8b0:4864:20::530])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4d57cb65-bec4-41f7-b946-12b3bc5dc6c5;
 Fri, 10 Sep 2021 03:12:15 +0000 (UTC)
Received: by mail-pg1-x530.google.com with SMTP id e7so572353pgk.2
 for <xen-devel@lists.xen.org>; Thu, 09 Sep 2021 20:12:15 -0700 (PDT)
Received: from laputa (p784a2304.tkyea130.ap.so-net.ne.jp. [120.74.35.4])
 by smtp.gmail.com with ESMTPSA id q20sm4099969pgu.31.2021.09.09.20.12.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 09 Sep 2021 20:12:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d57cb65-bec4-41f7-b946-12b3bc5dc6c5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=qYylmtiStGmWtWp49euWoU9YAeVO7de/oFQa51IHGW4=;
        b=KkVLB5TEqB3KRMw/FCvWsmuSOxEmwy8d+KtK0LOjbGHlOHAEU7OVzEQAhM6pfc1QEH
         KvIFV2XJNmeQbOSao9dlRcfvnF32to+MF3uwDH5IwlbwqLkEVqam97gVkVSTARHtbqvA
         foUk9Wvoi/DKISX9iB1vyRZ/TS/HqO/h/6WSEUmPQoQ0A5amlPkOMpXzKjESaB7zAszW
         mVOHprBKMX4JXDeD2MY0AfEPwyZsb4MUQ5ulK2aiQcwnvi7lsjsaDQbCYdNp/BcLfqk6
         M+VFSzbGDkMx/5Pk4ELJg0NVMET+MMZG2wHenO23PoSDt+/rCcAFZCZ8vIw6vOjj77Zq
         ZKCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=qYylmtiStGmWtWp49euWoU9YAeVO7de/oFQa51IHGW4=;
        b=ULQFPTohneujaO6W9hZIB3kZpBNFUGIkSxJRxLxL8WAB2PpIEgj+1YXvEQ2xE24iqw
         aaAlqva9XwpMdo3+cCfsbpf2oGYIdNOlLEGI/+hXhanKuZZcSlmvlf2q8Zcl/TjXeigG
         xMOJU5FHEbKvWpJKw2gk7KSgrwU8Tf0C9U0j6yTQK0XtMHYIE2KJmSMZBa6urMsG0fX0
         DrsOAVxnvECXX3wZXWq+cY79n+bavss7+5/R4kbGR8++IU+bWhmgI/OE5NbedMQqTzr7
         nFx+W7oTw+cHF41GV0GTJqmhFG3ybMJJaFIgysD8lTBHSLUoq5PFxOYEtTlk/jPhT7DT
         lJEQ==
X-Gm-Message-State: AOAM533YOm3phZdL5nZnE5ovI96XIHCDjLwAGbrCPbXm2okVWB8XLp2w
	WY9EQ7VQemmccDfqePkgG8KwqQ==
X-Google-Smtp-Source: ABdhPJxUFqx7oqVUFicGZ08ZdiIvX4OasIz44I5Efn0I5pK7zNiObpai0LAG5TzC7jangEodRtvZ4g==
X-Received: by 2002:aa7:8e81:0:b0:3fe:f212:f9dd with SMTP id a1-20020aa78e81000000b003fef212f9ddmr6217231pfr.46.1631243534646;
        Thu, 09 Sep 2021 20:12:14 -0700 (PDT)
Date: Fri, 10 Sep 2021 12:12:05 +0900
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: Wei Chen <Wei.Chen@arm.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alex Benn??e <alex.bennee@linaro.org>, Kaly Xin <Kaly.Xin@arm.com>,
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	"virtio-dev@lists.oasis-open.org" <virtio-dev@lists.oasis-open.org>,
	Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Stefano Stabellini <stefano.stabellini@xilinx.com>,
	"stefanha@redhat.com" <stefanha@redhat.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Carl van Schaik <cvanscha@qti.qualcomm.com>,
	"pratikp@quicinc.com" <pratikp@quicinc.com>,
	Srivatsa Vaddagiri <vatsa@codeaurora.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
	Paul Durrant <paul@xen.org>, Xen Devel <xen-devel@lists.xen.org>,
	Rich Persaud <persaur@gmail.com>,
	Daniel Smith <dpsmith@apertussolutions.com>,
	James McKenzie <james@bromium.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Message-ID: <20210910031205.GD42777@laputa>
References: <DB9PR08MB6857D1BE810B1D1DAF7B12AE9EFF9@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <20210820064150.GC13452@laputa>
 <20210826094047.GA55218@laputa>
 <DB9PR08MB68578198FF352EDC473D619E9EC79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CACMJ4GbmNgbB5ponYt3NGEk3j6YCksot+kDy2qs8HMdFXWnQbw@mail.gmail.com>
 <CACMJ4GbqPc29n+CAknY4kCjrQnkCSR=W+rymuY7Xa1EZb0MW5w@mail.gmail.com>
 <20210902071902.GC71098@laputa>
 <CACMJ4GYY0wkm0vOzJfq8M9dCxPq1ZFwWT8Q73rMjqtrP7if-Zw@mail.gmail.com>
 <20210907115501.GC49004@laputa>
 <CACMJ4GYvndK6Cat-cZxM3rJ+6Ys2-hT=0_QivQ4md1MHAGbN3Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CACMJ4GYvndK6Cat-cZxM3rJ+6Ys2-hT=0_QivQ4md1MHAGbN3Q@mail.gmail.com>

Hi Christopher,

On Tue, Sep 07, 2021 at 11:09:34AM -0700, Christopher Clark wrote:
> On Tue, Sep 7, 2021 at 4:55 AM AKASHI Takahiro <takahiro.akashi@linaro.org>
> wrote:
> 
> > Hi,
> >
> > I have not covered all your comments below yet.
> > So just one comment:
> >
> > On Mon, Sep 06, 2021 at 05:57:43PM -0700, Christopher Clark wrote:
> > > On Thu, Sep 2, 2021 at 12:19 AM AKASHI Takahiro <
> > takahiro.akashi@linaro.org>
> > > wrote:
> >
> > (snip)
> >
> > > >    It appears that, on FE side, at least three hypervisor calls (and
> > data
> > > >    copying) need to be invoked at every request, right?
> > > >
> > >
> > > For a write, counting FE sendv ops:
> > > 1: the write data payload is sent via the "Argo ring for writes"
> > > 2: the descriptor is sent via a sync of the available/descriptor ring
> > >   -- is there a third one that I am missing?
> >
> > In the picture, I can see
> > a) Data transmitted by Argo sendv
> > b) Descriptor written after data sendv
> > c) VirtIO ring sync'd to back-end via separate sendv
> >
> > Oops, (b) is not a hypervisor call, is it?
> >
> 
> That's correct, it is not - the blue arrows in the diagram are not
> hypercalls, they are intended to show data movement or action in the flow
> of performing the operation, and (b) is a data write within the guest's
> address space into the descriptor ring.
> 
> 
> 
> > (But I guess that you will have to have yet another call for notification
> > since there is no config register of QueueNotify?)
> >
> 
> Reasoning about hypercalls necessary for data movement:
> 
> VirtIO transport drivers are responsible for instantiating virtqueues
> (setup_vq) and are able to populate the notify function pointer in the
> virtqueue that they supply. The virtio-argo transport driver can provide a
> suitable notify function implementation that will issue the Argo hypercall
> sendv hypercall(s) for sending data from the guest frontend to the backend.
> By issuing the sendv at the time of the queuenotify, rather than as each
> buffer is added to the virtqueue, the cost of the sendv hypercall can be
> amortized over multiple buffer additions to the virtqueue.
> 
> I also understand that there has been some recent work in the Linaro
> Project Stratos on "Fat Virtqueues", where the data to be transmitted is
> included within an expanded virtqueue, which could further reduce the
> number of hypercalls required, since the data can be transmitted inline
> with the descriptors.
> Reference here:
> https://linaro.atlassian.net/wiki/spaces/STR/pages/25626313982/2021-01-21+Project+Stratos+Sync+Meeting+notes
> https://linaro.atlassian.net/browse/STR-25

Ah, yes. Obviously, "fatvirtqueue" has pros and cons.
One of cons is that it won't be suitable for bigger payload
with limited space in descriptors.

> As a result of the above, I think that a single hypercall could be
> sufficient for communicating data for multiple requests, and that a
> two-hypercall-per-request (worst case) upper bound could also be
> established.

When it comes to the payload or data plane, "fatvirtqueue" as well as
Argo utilizes copying. You dub it "DMA operations".
A similar approach can be also seen in virtio-over-ivshmem, where
a limited amount of memory are shared and FE will allocate some space
in this buffer and copy the payload into there. Those allocation will
be done via dma_ops of virtio_ivshmem driver. BE, on the other hand,
fetches the data from the shared memory by using the "offset" described
in a descriptor.
Shared memory is divided into a couple of different groups;
one for read/write for all, others for one writer with many readers.
(I hope I'm right here :)

Looks close to Argo? What is different is who is responsible for
copying data, the kernel or the hypervisor.
(Yeah, I know that Argo has more crucial aspects like access controls.)

In this sense, ivshmem can also be a candidate for hypervisor-agnostic
framework. Jailhouse doesn't say so explicitly AFAIK.
Jan may have some more to say.

Thanks,
-Takahiro Akashi


> Christopher
> 
> 
> 
> >
> > Thanks,
> > -Takahiro Akashi
> >
> >
> > > Christopher
> > >
> > >
> > > >
> > > > Thanks,
> > > > -Takahiro Akashi
> > > >
> > > >
> > > > > * Here are the design documents for building VirtIO-over-Argo, to
> > > > support a
> > > > >   hypervisor-agnostic frontend VirtIO transport driver using Argo.
> > > > >
> > > > > The Development Plan to build VirtIO virtual device support over Argo
> > > > > transport:
> > > > >
> > > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> > > > >
> > > > > A design for using VirtIO over Argo, describing how VirtIO data
> > > > structures
> > > > > and communication is handled over the Argo transport:
> > > > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo
> > > > >
> > > > > Diagram (from the above document) showing how VirtIO rings are
> > > > synchronized
> > > > > between domains without using shared memory:
> > > > >
> > > >
> > https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-blob-url=true&id=01f7d0e1-7686-4f0b-88e1-457c1d30df40&collection=contentId-1348763698&contextId=1348763698&mimeType=image%2Fpng&name=device-buffer-access-virtio-argo.png&size=243175&width=1106&height=1241
> > > > >
> > > > > Please note that the above design documents show that the existing
> > VirtIO
> > > > > device drivers, and both vring and virtqueue data structures can be
> > > > > preserved
> > > > > while interdomain communication can be performed with no shared
> > memory
> > > > > required
> > > > > for most drivers; (the exceptions where further design is required
> > are
> > > > those
> > > > > such as virtual framebuffer devices where shared memory regions are
> > > > > intentionally
> > > > > added to the communication structure beyond the vrings and
> > virtqueues).
> > > > >
> > > > > An analysis of VirtIO and Argo, informing the design:
> > > > >
> > > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO
> > > > >
> > > > > * Argo can be used for a communication path for configuration
> > between the
> > > > > backend
> > > > >   and the toolstack, avoiding the need for a dependency on XenStore,
> > > > which
> > > > > is an
> > > > >   advantage for any hypervisor-agnostic design. It is also amenable
> > to a
> > > > > notification
> > > > >   mechanism that is not based on Xen event channels.
> > > > >
> > > > > * Argo does not use or require shared memory between VMs and
> > provides an
> > > > > alternative
> > > > >   to the use of foreign shared memory mappings. It avoids some of the
> > > > > complexities
> > > > >   involved with using grants (eg. XSA-300).
> > > > >
> > > > > * Argo supports Mandatory Access Control by the hypervisor,
> > satisfying a
> > > > > common
> > > > >   certification requirement.
> > > > >
> > > > > * The Argo headers are BSD-licensed and the Xen hypervisor
> > implementation
> > > > > is GPLv2 but
> > > > >   accessible via the hypercall interface. The licensing should not
> > > > present
> > > > > an obstacle
> > > > >   to adoption of Argo in guest software or implementation by other
> > > > > hypervisors.
> > > > >
> > > > > * Since the interface that Argo presents to a guest VM is similar to
> > > > DMA, a
> > > > > VirtIO-Argo
> > > > >   frontend transport driver should be able to operate with a physical
> > > > > VirtIO-enabled
> > > > >   smart-NIC if the toolstack and an Argo-aware backend provide
> > support.
> > > > >
> > > > > The next Xen Community Call is next week and I would be happy to
> > answer
> > > > > questions
> > > > > about Argo and on this topic. I will also be following this thread.
> > > > >
> > > > > Christopher
> > > > > (Argo maintainer, Xen Community)
> > > > >
> > > > >
> > > >
> > --------------------------------------------------------------------------------
> > > > > [1]
> > > > > An introduction to Argo:
> > > > >
> > > >
> > https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> > > > > https://www.youtube.com/watch?v=cnC0Tg3jqJQ
> > > > > Xen Wiki page for Argo:
> > > > >
> > > >
> > https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen
> > > > >
> > > > > [2]
> > > > > OpenXT Linux Argo driver and userspace library:
> > > > > https://github.com/openxt/linux-xen-argo
> > > > >
> > > > > Windows V4V at OpenXT wiki:
> > > > > https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> > > > > Windows v4v driver source:
> > > > > https://github.com/OpenXT/xc-windows/tree/master/xenv4v
> > > > >
> > > > > HP/Bromium uXen V4V driver:
> > > > > https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
> > > > >
> > > > > [3]
> > > > > v2 of the Argo test unikernel for XTF:
> > > > >
> > > >
> > https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.html
> > > > >
> > > > > [4]
> > > > > Argo HMX Transport for VirtIO meeting minutes:
> > > > >
> > > >
> > https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html
> > > > >
> > > > > VirtIO-Argo Development wiki page:
> > > > >
> > > >
> > https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
> > > > >
> > > >
> > > >
> >


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 04:44:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 04:44:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183932.332336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYP1-00066Z-Fp; Fri, 10 Sep 2021 04:44:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183932.332336; Fri, 10 Sep 2021 04:44:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYP1-00066S-Bp; Fri, 10 Sep 2021 04:44:39 +0000
Received: by outflank-mailman (input) for mailman id 183932;
 Fri, 10 Sep 2021 04:44:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOYOz-00066M-Ul
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 04:44:37 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c494b8d3-6bb9-46d3-82e0-3b4fa6b2a295;
 Fri, 10 Sep 2021 04:44:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 08AED1FE3D;
 Fri, 10 Sep 2021 04:44:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D815013BB2;
 Fri, 10 Sep 2021 04:44:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id le9DM7PiOmE4eAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 04:44:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c494b8d3-6bb9-46d3-82e0-3b4fa6b2a295
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631249076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=gDHnr6rSrqz9WZGMFBuEgXHRVpjqKevg2d5p7wUIXic=;
	b=YrOoe8H1VnhcP7MWfmWDicjO2rcuT2iLS+i15YpPCJIxWpuJbw6fl5L8eetbd/Imsoe4eR
	FMa0v1YAT5hpReF1roDHDG9mGtO6ddjdYFodxZt3R8/UffWbNazdrxMI76mEUK8Hk80VDP
	9U2mUwsOu1UZgLgO8dOEJZLOuDOej9w=
Subject: Re: [OSSTEST PATCH 4/4] mfi-common: Drop Linux dom0 i386 tests for
 newer Linux branches
To: Ian Jackson <iwj@xenproject.org>, xen-devel@lists.xenproject.org
References: <20210909164703.19426-1-iwj@xenproject.org>
 <20210909164703.19426-4-iwj@xenproject.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <32619ebe-5967-4c1e-78ea-cc3e517d14ae@suse.com>
Date: Fri, 10 Sep 2021 06:44:35 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210909164703.19426-4-iwj@xenproject.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="NzGroIA1Re0ObjdRqVTG2qoqMbIDhaR84"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--NzGroIA1Re0ObjdRqVTG2qoqMbIDhaR84
Content-Type: multipart/mixed; boundary="Fs0diPdXcDv82H41xA5aU7wjavJVjJ47O";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>, xen-devel@lists.xenproject.org
Message-ID: <32619ebe-5967-4c1e-78ea-cc3e517d14ae@suse.com>
Subject: Re: [OSSTEST PATCH 4/4] mfi-common: Drop Linux dom0 i386 tests for
 newer Linux branches
References: <20210909164703.19426-1-iwj@xenproject.org>
 <20210909164703.19426-4-iwj@xenproject.org>
In-Reply-To: <20210909164703.19426-4-iwj@xenproject.org>

--Fs0diPdXcDv82H41xA5aU7wjavJVjJ47O
Content-Type: multipart/mixed;
 boundary="------------687478118A328316FDA25142"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------687478118A328316FDA25142
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 09.09.21 18:47, Ian Jackson wrote:
> This makes radical changes to the test ste for the linux-linus and
> linux-next branches.
>=20
> Mostly, tests are dropped but some 64-bit dom0 tests are added to
> replace them.
>=20
> Requested-by: Juergen Gross <jgross@suse.com>
> Signed-off-by: Ian Jackson <iwj@xenproject.org>

Thanks!

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------687478118A328316FDA25142
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------687478118A328316FDA25142--

--Fs0diPdXcDv82H41xA5aU7wjavJVjJ47O--

--NzGroIA1Re0ObjdRqVTG2qoqMbIDhaR84
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE64rMFAwAAAAAACgkQsN6d1ii/Ey9x
6gf+OuvV7ognuFHN0e2iMVFRk6D/oJKn8eYBdeX/1WQLMC2TX1ttpqnKwJHUhMVqGPadTPWfo9e9
7hZQWchIqDCDUSwGRVS064pUCXly+IG1pJ5ZtSvExVN8tXua+3QYWaTt7xqupsyl3JcGlfSriF9R
bkasW0kjRZvbewbasNxX9+U2ocRR8SsgMuprBKHcz6v+dOzmZ6n7HHbTVbAogIv8SAdy4A3eBBZk
G7jleVLvjduDyGfzf9GlUTmaRTW43sTd1nCFL8NQIXGMKgDq4vZAKjhXyx6PyCYkSqgI+SuuSinY
yQF5y581NgeLQNjGOfzM3HLnul89ppHCpcsmLQts7Q==
=DaCC
-----END PGP SIGNATURE-----

--NzGroIA1Re0ObjdRqVTG2qoqMbIDhaR84--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 04:48:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 04:48:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183928.332354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYSx-0006oV-Bd; Fri, 10 Sep 2021 04:48:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183928.332354; Fri, 10 Sep 2021 04:48:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYSx-0006o8-3K; Fri, 10 Sep 2021 04:48:43 +0000
Received: by outflank-mailman (input) for mailman id 183928;
 Fri, 10 Sep 2021 03:59:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=T0+8=OA=iscas.ac.cn=vulab@srs-us1.protection.inumbo.net>)
 id 1mOXhL-00006t-Cu
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 03:59:31 +0000
Received: from cstnet.cn (unknown [159.226.251.21])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 816f1de5-a5e7-4008-9a2d-5a6e2fe09e7d;
 Fri, 10 Sep 2021 03:59:26 +0000 (UTC)
Received: from localhost.localdomain (unknown [124.16.141.243])
 by APP-01 (Coremail) with SMTP id qwCowABX2dgZ2Dpho9EOAA--.30793S2;
 Fri, 10 Sep 2021 11:59:22 +0800 (CST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 816f1de5-a5e7-4008-9a2d-5a6e2fe09e7d
From: Xu Wang <vulab@iscas.ac.cn>
To: Damien.LeMoal@wdc.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	axboe@kernel.dk,
	xen-devel@lists.xenproject.org
Cc: linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: [PATCH v2] xen-blkback: Remove needless request_queue NULL pointer check
Date: Fri, 10 Sep 2021 03:59:18 +0000
Message-Id: <20210910035918.24907-1-vulab@iscas.ac.cn>
X-Mailer: git-send-email 2.17.1
X-CM-TRANSID:qwCowABX2dgZ2Dpho9EOAA--.30793S2
X-Coremail-Antispam: 1UD129KBjvdXoWrZw43Kry8uF4xXF47Wr1xXwb_yoWDKrc_Wr
	18ZrZaqr1v9rs09w1DCFnay340k3WkZFy09an2qFWaq347AFyaq3y7Zrn5Gw47uayUKrn5
	CFyUG3W2yr1xAjkaLaAFLSUrUUUUUb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT
	9fnUUIcSsGvfJTRUUUbwAYjsxI4VWkCwAYFVCjjxCrM7AC8VAFwI0_Gr0_Xr1l1xkIjI8I
	6I8E6xAIw20EY4v20xvaj40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l8cAvFVAK0II2c7xJM2
	8CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0
	cI8IcVCY1x0267AKxVW8JVWxJwA2z4x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I
	8E87Iv6xkF7I0E14v26F4UJVW0owAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC
	0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Gr1j6F
	4UJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l
	4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67
	AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8I
	cVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r1j6r4UMIIF0xvE42xK8VAvwI
	8IcIk0rVWrZr1j6s0DMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAF
	wI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07j1_M3UUUUU=
X-Originating-IP: [124.16.141.243]
X-CM-SenderInfo: pyxotu46lvutnvoduhdfq/1tbiCQoAA102ag5bSAAAsc

The request_queue pointer returned from bdev_get_queue() shall
never be NULL, so the null check is unnecessary, just remove it.

Signed-off-by: Xu Wang <vulab@iscas.ac.cn>
---
Changes since v2:
- Add missing parenthesis.
- Remove another null pointer check.
---
 drivers/block/xen-blkback/xenbus.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index 33eba3df4dd9..aac08b4369ac 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -516,10 +516,10 @@ static int xen_vbd_create(struct xen_blkif *blkif, blkif_vdev_t handle,
 		vbd->type |= VDISK_REMOVABLE;
 
 	q = bdev_get_queue(bdev);
-	if (q && test_bit(QUEUE_FLAG_WC, &q->queue_flags))
+	if (test_bit(QUEUE_FLAG_WC, &q->queue_flags))
 		vbd->flush_support = true;
 
-	if (q && blk_queue_secure_erase(q))
+	if (blk_queue_secure_erase(q))
 		vbd->discard_secure = true;
 
 	vbd->feature_gnt_persistent = feature_persistent;
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 04:48:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 04:48:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183834.332347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYSw-0006l0-Uf; Fri, 10 Sep 2021 04:48:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183834.332347; Fri, 10 Sep 2021 04:48:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYSw-0006kt-RK; Fri, 10 Sep 2021 04:48:42 +0000
Received: by outflank-mailman (input) for mailman id 183834;
 Fri, 10 Sep 2021 02:25:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=T0+8=OA=iscas.ac.cn=vulab@srs-us1.protection.inumbo.net>)
 id 1mOWE0-0007yA-65
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 02:25:08 +0000
Received: from cstnet.cn (unknown [159.226.251.23])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 4eaadc92-11de-11ec-b208-12813bfff9fa;
 Fri, 10 Sep 2021 02:25:05 +0000 (UTC)
Received: from localhost.localdomain (unknown [124.16.141.243])
 by APP-03 (Coremail) with SMTP id rQCowADHz3v7wTphEPUJAA--.10149S2;
 Fri, 10 Sep 2021 10:25:00 +0800 (CST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4eaadc92-11de-11ec-b208-12813bfff9fa
From: Xu Wang <vulab@iscas.ac.cn>
To: konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	axboe@kernel.dk,
	xen-devel@lists.xenproject.org
Cc: linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: [PATCH] xen-blkback: Remove needless request_queue NULL pointer check
Date: Fri, 10 Sep 2021 02:24:57 +0000
Message-Id: <20210910022457.22963-1-vulab@iscas.ac.cn>
X-Mailer: git-send-email 2.17.1
X-CM-TRANSID:rQCowADHz3v7wTphEPUJAA--.10149S2
X-Coremail-Antispam: 1UD129KBjvdXoWrZw43Kry8uw1rGFy7WryrtFb_yoWfJwb_Wr
	1UZrsrXr1kurn0934DCFsay340ya48ZF1SkFn2q3ySq34UAF1av39Fvrn5Gw47X3yUKr90
	kFyUGF17tr48AjkaLaAFLSUrUUUUUb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT
	9fnUUIcSsGvfJTRUUUbc8YjsxI4VWDJwAYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I
	6I8E6xAIw20EY4v20xvaj40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l8cAvFVAK0II2c7xJM2
	8CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0
	cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwV
	C2z280aVCY1x0267AKxVW8Jr0_Cr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVAC
	Y4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r106r15McIj6I8E87Iv67AKxVW8Jr
	0_Cr1UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCY02Avz4vE14v_Gr1l
	42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJV
	WUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAK
	I48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r1j6r
	4UMIIF0xvE42xK8VAvwI8IcIk0rVW3JVWrJr1lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAI
	cVC2z280aVCY1x0267AKxVWUJVW8JbIYCTnIWIevJa73UjIFyTuYvjxU7M5lUUUUU
X-Originating-IP: [124.16.141.243]
X-CM-SenderInfo: pyxotu46lvutnvoduhdfq/1tbiCwgAA1z4kZv+wwAAsT

The request_queue pointer returned from bdev_get_queue() shall
never be NULL, so the null check is unnecessary, just remove it.

Signed-off-by: Xu Wang <vulab@iscas.ac.cn>
---
 drivers/block/xen-blkback/xenbus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index 33eba3df4dd9..f05132b9ddbf 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -516,7 +516,7 @@ static int xen_vbd_create(struct xen_blkif *blkif, blkif_vdev_t handle,
 		vbd->type |= VDISK_REMOVABLE;
 
 	q = bdev_get_queue(bdev);
-	if (q && test_bit(QUEUE_FLAG_WC, &q->queue_flags))
+	if test_bit(QUEUE_FLAG_WC, &q->queue_flags)
 		vbd->flush_support = true;
 
 	if (q && blk_queue_secure_erase(q))
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:08:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:08:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.183951.332368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYm4-00027h-TP; Fri, 10 Sep 2021 05:08:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 183951.332368; Fri, 10 Sep 2021 05:08:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOYm4-00027a-QN; Fri, 10 Sep 2021 05:08:28 +0000
Received: by outflank-mailman (input) for mailman id 183951;
 Fri, 10 Sep 2021 05:08:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOYm4-00027Q-9t; Fri, 10 Sep 2021 05:08:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOYm4-0006KL-2y; Fri, 10 Sep 2021 05:08:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOYm3-0005vw-Lv; Fri, 10 Sep 2021 05:08:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOYm3-0002G6-LP; Fri, 10 Sep 2021 05:08:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fVnP/2WNxDpqanpNR/NEuV5BGgZHlbduVYU4jZIlEYk=; b=rZfh31DoRfhUB8yXJBGpDU19wG
	AS4tlzvh3pgvYsqv7N6P1UjxhVnQVvhDS95vjw5zEjTcZUx+5gIKntNmB6UINPWwtOo7pLyNn3nFR
	yMpo1HEGZ4h4nUpg8Fuwm6HbOuvQ4GkG0vy8+iGixa1TGx/uqUruif2AIUUlOf33IGpc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164898-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.13-testing test] 164898: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=065fff7af08f7eaf300c9bef86ae3cec8150d3aa
X-Osstest-Versions-That:
    xen=32d580902b959000d79d51dff03a3560653c4fcb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 05:08:27 +0000

flight 164898 xen-4.13-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164898/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 163761
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 163761
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 163761
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 163761
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163761
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 163761
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 163761
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 163761
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 163761
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 163761
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 163761
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  065fff7af08f7eaf300c9bef86ae3cec8150d3aa
baseline version:
 xen                  32d580902b959000d79d51dff03a3560653c4fcb

Last test of basis   163761  2021-07-17 07:57:22 Z   54 days
Failing since        164260  2021-08-19 17:07:29 Z   21 days   10 attempts
Testing same since   164898  2021-09-09 05:02:45 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Dario Faggioli <dfaggioli@suse.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   32d580902b..065fff7af0  065fff7af08f7eaf300c9bef86ae3cec8150d3aa -> stable-4.13


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:26:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:26:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184017.332511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZ37-0006bD-86; Fri, 10 Sep 2021 05:26:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184017.332511; Fri, 10 Sep 2021 05:26:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZ37-0006b6-3N; Fri, 10 Sep 2021 05:26:05 +0000
Received: by outflank-mailman (input) for mailman id 184017;
 Fri, 10 Sep 2021 05:26:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOZ34-0006aw-TY; Fri, 10 Sep 2021 05:26:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOZ34-0006jI-HB; Fri, 10 Sep 2021 05:26:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOZ34-0006rx-3I; Fri, 10 Sep 2021 05:26:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOZ34-0006PD-2q; Fri, 10 Sep 2021 05:26:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DNtD9QnBpbTHuWk8p6RmDAF2foR0BtxKJq368sAbQYQ=; b=URqvsxfwJsxZQbsGvki2taf9Cm
	r2m4Hi+zLA8+S8MRSZoY3L509+jYBTKU+mjcaZbSSEAIPYa4rSO04vCijR8LK9kNw+SqUtC2Di/u0
	OmZJngyie/8LwE3XoaS2/eBYZ/eyEtfSK2r+NpchsfS8xkduVUjw1r5WWcgKMFsvTjzg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164913-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164913: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=d248516b3a190c5cb5d51164b5721ead9d24469a
X-Osstest-Versions-That:
    ovmf=a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 05:26:02 +0000

flight 164913 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164913/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 d248516b3a190c5cb5d51164b5721ead9d24469a
baseline version:
 ovmf                 a7cf2c5664b9605162b20ab6b51c7bdcab3e14f0

Last test of basis   164869  2021-09-07 04:11:11 Z    3 days
Testing same since   164913  2021-09-09 13:11:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Sravanthi <k.kavyax.sravanthi@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   a7cf2c5664..d248516b3a  d248516b3a190c5cb5d51164b5721ead9d24469a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:49:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:49:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184025.332528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZPM-00016K-2O; Fri, 10 Sep 2021 05:49:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184025.332528; Fri, 10 Sep 2021 05:49:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZPL-00016D-VH; Fri, 10 Sep 2021 05:49:03 +0000
Received: by outflank-mailman (input) for mailman id 184025;
 Fri, 10 Sep 2021 05:49:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOZPL-000167-1q
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 05:49:03 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc7721be-11fa-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 05:49:01 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id CB2F51FE45;
 Fri, 10 Sep 2021 05:49:00 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6CB7E13D02;
 Fri, 10 Sep 2021 05:49:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id kliZGMzxOmHiDQAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 05:49:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc7721be-11fa-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631252940; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=CVQYFneEpxXGr/1pOngzMGqStazfKW04mzES2WC26/g=;
	b=qZGYPkm+KdwdYxfh7k8r6JiEiPL6I10f1YhLHsgSQSuF94loXNI8dKlQVtYoxnO1Z+69r/
	rIhKtW/M/CN5mgYStglCkyTz7FXt9IWMT4P+JFJNgLh3IlgaebnG3Owd70sF285AyLz0PL
	JVnLlSMvewDsHuoxtLd38p334IqlTvc=
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
 <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
 <24890.12692.44064.995392@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
Date: Fri, 10 Sep 2021 07:48:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24890.12692.44064.995392@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="FB87yRzkXPOes7Xk4mfczWU7MXM6c7um0"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--FB87yRzkXPOes7Xk4mfczWU7MXM6c7um0
Content-Type: multipart/mixed; boundary="G7yk8I5DY8SeMAqrXhzXB6QqbxRtsBGuY";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-ID: <462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
 <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
 <24890.12692.44064.995392@mariner.uk.xensource.com>
In-Reply-To: <24890.12692.44064.995392@mariner.uk.xensource.com>

--G7yk8I5DY8SeMAqrXhzXB6QqbxRtsBGuY
Content-Type: multipart/mixed;
 boundary="------------01FED0D97D741FAEECD4A949"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------01FED0D97D741FAEECD4A949
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 09.09.21 18:08, Ian Jackson wrote:
> Juergen Gross writes ("Re: [PATCH v2 1/3] stubdom: fix build with disab=
led pv-grub"):
>> On 09.09.21 15:23, Ian Jackson wrote:
>>> How about "pv-grub-maybe" ?  Or something.
>>
>> What about "pv-grub-if-enabled"?
>=20
> Fine by me.
>=20
>> And could that be done when committing, or should I send another round=
?
>=20
> Please do send another round (sorry).

NP.

BTW, you probably want to modify OSStest to use configure with:

--enable-pv-grub --enable-qemu-traditional

in order to not let the tests fail (applies to x86 only, of course).


Juergen

--------------01FED0D97D741FAEECD4A949
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------01FED0D97D741FAEECD4A949--

--G7yk8I5DY8SeMAqrXhzXB6QqbxRtsBGuY--

--FB87yRzkXPOes7Xk4mfczWU7MXM6c7um0
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE68csFAwAAAAAACgkQsN6d1ii/Ey+F
vwgAmplXoANW+mhw9bZlcGOr3WTuwXiK7siIGPCl6FuHYtW2o7KmdZW8QDdd4MLDxSOEmydIJbBP
h+Wlwy8Ji19E0KIS6hs6bPJ04QPLhTODtC5vtAkdmpCLcJJYb+tjWtg23xg2WvahcqFtm4wlg79l
6NllGGKco3w+i4H0VNL6K05pr9Z8WPr/MsiTrMHU0yHo8CAebdCHA79qk++iFV0n6OgGpnOQJaDp
9bxH4vX00CIyIh3c8CUE52oK+V0WOadmz5ep/JPW7UpS9dzmRMgPCTlKBcIQsIRPg89oyFQRhlTR
NwcQIcmEZI/BYA+HQQjeK/AhPJANUM8gp/T+vqU+BQ==
=hr5g
-----END PGP SIGNATURE-----

--FB87yRzkXPOes7Xk4mfczWU7MXM6c7um0--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:55:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:55:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184032.332538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVV-0002mz-Oe; Fri, 10 Sep 2021 05:55:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184032.332538; Fri, 10 Sep 2021 05:55:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVV-0002ms-LZ; Fri, 10 Sep 2021 05:55:25 +0000
Received: by outflank-mailman (input) for mailman id 184032;
 Fri, 10 Sep 2021 05:55:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOZVU-0002mm-ET
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 05:55:24 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id afb5fd60-11fb-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 05:55:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2BC1A223F2;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CC9BB13D0C;
 Fri, 10 Sep 2021 05:55:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id cJiAMEnzOmEDEAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 05:55:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afb5fd60-11fb-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631253322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xV8rdwiG4q9toCr/6b+aaGACeRQj1/7eJo0Ol2OKeRo=;
	b=YqZjTZsf0xfHXnshigWUy37ia2m5fkNKqan+OsRvi2PlL3ZNh03lWky1a6Rou/oVVVijsv
	EZ9GAkSwOOHDZIaAZctSYWUJNv3WmIzjkv0Lcv48lTO2ZqFuL2yoMn0U/sD+6sSQFJpeiL
	a9BR0OCfw5byHfSLdAWpwizGi9yjYPA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: [PATCH v3 1/3] stubdom: fix build with disabled pv-grub
Date: Fri, 10 Sep 2021 07:55:16 +0200
Message-Id: <20210910055518.562-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210910055518.562-1-jgross@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Today the build will fail if --disable-pv-grub as a parameter of
configure, as the main Makefile will unconditionally try to build a
32-bit pv-grub stubdom.

Fix that by introducing a pv-grub-if-enabled target in
stubdom/Makefile taking care of this situation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
--
V3:
- rename pv-grub32 target (Ian Jackson)
---
 Makefile         |  4 ++--
 stubdom/Makefile | 13 +++++++++++++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 96d32cfd50..346d73a0dc 100644
--- a/Makefile
+++ b/Makefile
@@ -72,7 +72,7 @@ build-tools-oxenstored: build-tools-public-headers
 build-stubdom: mini-os-dir build-tools-public-headers
 	$(MAKE) -C stubdom build
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub-if-enabled
 endif
 
 .PHONY: build-docs
@@ -143,7 +143,7 @@ install-tools: install-tools-public-headers
 install-stubdom: mini-os-dir install-tools
 	$(MAKE) -C stubdom install
 ifeq (x86_64,$(XEN_TARGET_ARCH))
-	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
+	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub-if-enabled
 endif
 
 .PHONY: tools/firmware/seabios-dir-force-update
diff --git a/stubdom/Makefile b/stubdom/Makefile
index 06aa69d8bc..ccfcf5b75f 100644
--- a/stubdom/Makefile
+++ b/stubdom/Makefile
@@ -531,6 +531,13 @@ vtpmmgr-stubdom: mini-os-$(XEN_TARGET_ARCH)-vtpmmgr vtpmmgr
 pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxenguest grub
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
 
+.PHONY: pv-grub-if-enabled
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+pv-grub-if-enabled: pv-grub
+else
+pv-grub-if-enabled:
+endif
+
 .PHONY: xenstore-stubdom
 xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxenguest xenstore
 	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
@@ -560,6 +567,12 @@ install-grub: pv-grub
 	$(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)"
 	$(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz"
 
+ifneq ($(filter grub,$(STUBDOM_TARGETS)),)
+install-grub-if-enabled: install-grub
+else
+install-grub-if-enabled:
+endif
+
 install-c: c-stubdom
 
 install-caml: caml-stubdom
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:55:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:55:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184033.332550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVb-00035k-3a; Fri, 10 Sep 2021 05:55:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184033.332550; Fri, 10 Sep 2021 05:55:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVa-00035b-Uj; Fri, 10 Sep 2021 05:55:30 +0000
Received: by outflank-mailman (input) for mailman id 184033;
 Fri, 10 Sep 2021 05:55:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOZVZ-0002mm-C9
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 05:55:29 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id af7f13f4-11fb-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C6B74223C1;
 Fri, 10 Sep 2021 05:55:21 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6713113D02;
 Fri, 10 Sep 2021 05:55:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id M3SlF0nzOmEDEAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 05:55:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af7f13f4-11fb-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631253321; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=m9R6JMYMRyyJJMwi14rsvjDv5ZbRSUEL1JTkP/zl71I=;
	b=AqR0Gj8KqJV2nWhzwhz9cjIVThvMswcrr6OaewBuqcbcHEpN5owh/OfSmx98kppJv7Ruti
	2JodyrqHo8v81cw4voAhWxt6X0FjT91RG5EcimLdUYSWmYgf7iprNiA7S0TEdDJQNvdKR5
	c86qmwD18M4nV+Rz+j40fVKvv7zKW4U=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Community Manager <community.manager@xenproject.org>
Subject: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per default
Date: Fri, 10 Sep 2021 07:55:15 +0200
Message-Id: <20210910055518.562-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is a first step of deprecating pv-grub and qemu-trad including
ioemu-stubdom. Switch the default to not building it.

Changes in V3:
- rename pv-grub32 make target

Changes in V2:
- new patch 3
- added CHANGELOG.md entry in patch 2

Juergen Gross (3):
  stubdom: fix build with disabled pv-grub
  stubdom: disable building pv-grub
  tools: disable building qemu-trad per default

 CHANGELOG.md         |  5 +++++
 Makefile             |  4 ++--
 stubdom/Makefile     | 13 +++++++++++++
 stubdom/configure    | 16 ++--------------
 stubdom/configure.ac | 10 ++--------
 tools/configure      | 17 ++---------------
 tools/configure.ac   | 13 +------------
 7 files changed, 27 insertions(+), 51 deletions(-)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:55:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:55:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184035.332561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVg-0003Rg-AQ; Fri, 10 Sep 2021 05:55:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184035.332561; Fri, 10 Sep 2021 05:55:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVg-0003RV-78; Fri, 10 Sep 2021 05:55:36 +0000
Received: by outflank-mailman (input) for mailman id 184035;
 Fri, 10 Sep 2021 05:55:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOZVe-0002mm-CH
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 05:55:34 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id afd42b3c-11fb-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 05:55:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6B22B1FE45;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3015D13D02;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id kAP+CUrzOmEDEAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 05:55:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afd42b3c-11fb-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631253322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YgngfMEyR4Vr8reqmxkBKjSYP0wtCAuyapjE7LwlNwk=;
	b=TaC/0oyaSn5bktgqEl+S15Rrq+YIlbrsvlGxljMRhwIbFci4Si1HyHHdTXcafK0UTKVicb
	SHNhbgfvGN0hMff6oCzagAb+eIzPk3dyw0Gxtd2OSmKnmcczRutVbK8ZB/9LAP/PgSH3BJ
	lCT/cxzHmgenA7zauRAF4UrzudIQfe0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 2/3] stubdom: disable building pv-grub
Date: Fri, 10 Sep 2021 07:55:17 +0200
Message-Id: <20210910055518.562-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210910055518.562-1-jgross@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The stubdom based pv-grub is using a very outdated version of grub
(0.97) and should not be used any longer. Mainline grub has support for
PV guests for a long time now, so that should be used as a boot loader
of a PV domain.

So disable building pv-grub per default. In case someone really wants
to continue using it he/she can still use a pv-grub binary from an older
Xen version or manually enable building it via:

  configure --enable-pv-grub

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Acked-by: Ian Jackson <iwj@xenproject.org>
---
V2:
- add CHANGELOG.md entry (Jan Beulich)
---
 CHANGELOG.md         | 2 ++
 stubdom/configure    | 8 ++------
 stubdom/configure.ac | 2 +-
 3 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 22cfdb4298..e7107ac3de 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    appearing in 4.12.2 and 4.11.4). Prior (4.13...4.15-like) behavior can be arranged for
    either by enabling the IOMMU_QUARANTINE_SCRATCH_PAGE setting at build (configuration) time
    or by passing "iommu=quarantine=scratch-page" on the hypervisor command line.
+ - pv-grub stubdoms will no longer be built per default. In order to be able to use pv-grub
+   configure needs to be called with "--enable-pv-grub" as parameter.
 
 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.15.0) - TBD
 
diff --git a/stubdom/configure b/stubdom/configure
index aa48df986d..df31532abb 100755
--- a/stubdom/configure
+++ b/stubdom/configure
@@ -1342,7 +1342,7 @@ Optional Features:
   --enable-ioemu-stubdom  Build and install ioemu-stubdom
   --enable-c-stubdom      Build and install c-stubdom (default is DISABLED)
   --enable-caml-stubdom   Build and install caml-stubdom (default is DISABLED)
-  --disable-pv-grub       Build and install pv-grub (default is ENABLED)
+  --enable-pv-grub        Build and install pv-grub (default is DISABLED)
   --disable-xenstore-stubdom
                           Build and install xenstore-stubdom (default is
                           ENABLED)
@@ -2129,11 +2129,7 @@ fi
 else
 
 
-grub=y
-STUBDOM_TARGETS="$STUBDOM_TARGETS grub"
-STUBDOM_BUILD="$STUBDOM_BUILD pv-grub"
-STUBDOM_INSTALL="$STUBDOM_INSTALL install-grub"
-STUBDOM_UNINSTALL="$STUBDOM_UNINSTALL install-grub"
+grub=n
 
 
 fi
diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index bd6f765929..a07a1edae5 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -21,7 +21,7 @@ m4_include([../m4/fetcher.m4])
 AX_STUBDOM_CONDITIONAL([ioemu-stubdom], [ioemu])
 AX_STUBDOM_DEFAULT_DISABLE([c-stubdom], [c])
 AX_STUBDOM_DEFAULT_DISABLE([caml-stubdom], [caml])
-AX_STUBDOM_DEFAULT_ENABLE([pv-grub], [grub])
+AX_STUBDOM_DEFAULT_DISABLE([pv-grub], [grub])
 AX_STUBDOM_DEFAULT_ENABLE([xenstore-stubdom], [xenstore])
 AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
 AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 05:55:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 05:55:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184037.332571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVk-0003ti-QR; Fri, 10 Sep 2021 05:55:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184037.332571; Fri, 10 Sep 2021 05:55:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZVk-0003tZ-N8; Fri, 10 Sep 2021 05:55:40 +0000
Received: by outflank-mailman (input) for mailman id 184037;
 Fri, 10 Sep 2021 05:55:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOZVj-0002mm-CV
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 05:55:39 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id af7f13f5-11fb-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 05:55:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id AB71E201FF;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7069F13D0C;
 Fri, 10 Sep 2021 05:55:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id cOz8GUrzOmEDEAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 05:55:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af7f13f5-11fb-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631253322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aRM6wu2g4hDT9jEA3EflR4Cj2R9f8xFsffqAuZHwTXA=;
	b=AQ5PV++cXFuCWJJXpXhn+xk7e4+cMySIj4n6enI7lyYU0f50v4P2XSFX7McEG1OeENHNGF
	TZszTzWX0AonwDPN5Lo+cHdC/fQggZODSu87xhHCTa5+aDgky2Ys1t0uL6Qp3GBp3J6N1B
	Hw1gdVc+4U8WbliZgqbr4q441rTejHs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Community Manager <community.manager@xenproject.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 3/3] tools: disable building qemu-trad per default
Date: Fri, 10 Sep 2021 07:55:18 +0200
Message-Id: <20210910055518.562-4-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210910055518.562-1-jgross@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Using qemu-traditional as device model is deprecated for some time now.

So change the default for building it to "disable". This will affect
ioemu-stubdom, too, as there is a direct dependency between the two.

Today it is possible to use a PVH/HVM Linux-based stubdom as device
model. Additionally using ioemu-stubdom isn't really helping for
security, as it requires to run a very old and potentially buggy qemu
version in a PV domain. This is adding probably more security problems
than it is removing by using a stubdom.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Acked-by: Ian Jackson <iwj@xenproject.org>
---
V2:
- new patch
---
 CHANGELOG.md         |  3 +++
 stubdom/configure    |  8 --------
 stubdom/configure.ac |  8 +-------
 tools/configure      | 17 ++---------------
 tools/configure.ac   | 13 +------------
 5 files changed, 7 insertions(+), 42 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index e7107ac3de..e5ab49e779 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,6 +18,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    or by passing "iommu=quarantine=scratch-page" on the hypervisor command line.
  - pv-grub stubdoms will no longer be built per default. In order to be able to use pv-grub
    configure needs to be called with "--enable-pv-grub" as parameter.
+ - qemu-traditional based device models (both, qemu-traditional and ioemu-stubdom) will
+   no longer be built per default. In order to be able to use those, configure needs to
+   be called with "--enable-qemu-traditional" as parameter.
 
 ## [4.15.0 UNRELEASED](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.15.0) - TBD
 
diff --git a/stubdom/configure b/stubdom/configure
index df31532abb..07b709f998 100755
--- a/stubdom/configure
+++ b/stubdom/configure
@@ -2286,14 +2286,6 @@ fi
 # Check whether --enable-qemu-traditional was given.
 if test "${enable_qemu_traditional+set}" = set; then :
   enableval=$enable_qemu_traditional;
-else
-
-    case "$host_cpu" in
-        i[3456]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-
 fi
 
 if test "x$enable_qemu_traditional" = "xyes"; then :
diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index a07a1edae5..e20d99edac 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -27,13 +27,7 @@ AX_STUBDOM_DEFAULT_ENABLE([xenstorepvh-stubdom], [xenstorepvh])
 AX_STUBDOM_CONDITIONAL([vtpm-stubdom], [vtpm])
 AX_STUBDOM_CONDITIONAL([vtpmmgr-stubdom], [vtpmmgr])
 
-AC_ARG_ENABLE([qemu-traditional],,,[
-    case "$host_cpu" in
-        i[[3456]]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-])
+AC_ARG_ENABLE([qemu-traditional])
 AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
     qemu_traditional=y],[
     qemu_traditional=n
diff --git a/tools/configure b/tools/configure
index 33814b24b3..8bf8fe75b8 100755
--- a/tools/configure
+++ b/tools/configure
@@ -1502,8 +1502,8 @@ Optional Features:
   --disable-seabios       Disable SeaBIOS (default is ENABLED)
   --disable-golang        Disable Go tools (default is ENABLED)
   --enable-qemu-traditional
-                          Enable qemu traditional device model, (DEFAULT is on
-                          for Linux or NetBSD x86, otherwise off)
+                          Enable qemu traditional device model, (DEFAULT is
+                          off)
   --enable-rombios        Enable ROMBIOS, (DEFAULT is on if qemu-traditional
                           is enabled, otherwise off)
   --disable-ipxe          Enable in-tree IPXE, (DEFAULT is on if rombios is
@@ -4287,19 +4287,6 @@ LINUX_BACKEND_MODULES="`eval echo $LINUX_BACKEND_MODULES`"
 # Check whether --enable-qemu-traditional was given.
 if test "${enable_qemu_traditional+set}" = set; then :
   enableval=$enable_qemu_traditional;
-else
-
-    case "$host_cpu" in
-        i[3456]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-    case "$host_os" in
-        freebsd*)
-           enable_qemu_traditional="no";;
-    esac
-
-
 fi
 
 if test "x$enable_qemu_traditional" = "xyes"; then :
diff --git a/tools/configure.ac b/tools/configure.ac
index 6414fcbb44..a713fd34d6 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -120,18 +120,7 @@ AC_SUBST(LINUX_BACKEND_MODULES)
 
 AC_ARG_ENABLE([qemu-traditional],
     AS_HELP_STRING([--enable-qemu-traditional],
-                   [Enable qemu traditional device model, (DEFAULT is on for Linux or NetBSD x86, otherwise off)]),,[
-    case "$host_cpu" in
-        i[[3456]]86|x86_64)
-           enable_qemu_traditional="yes";;
-        *) enable_qemu_traditional="no";;
-    esac
-    case "$host_os" in
-        freebsd*)
-           enable_qemu_traditional="no";;
-    esac
-
-])
+                   [Enable qemu traditional device model, (DEFAULT is off)]))
 AS_IF([test "x$enable_qemu_traditional" = "xyes"], [
 AC_DEFINE([HAVE_QEMU_TRADITIONAL], [1], [Qemu traditional enabled])
     qemu_traditional=y],[
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 06:10:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 06:10:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184059.332583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZk8-0007GQ-4Q; Fri, 10 Sep 2021 06:10:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184059.332583; Fri, 10 Sep 2021 06:10:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZk8-0007GJ-1X; Fri, 10 Sep 2021 06:10:32 +0000
Received: by outflank-mailman (input) for mailman id 184059;
 Fri, 10 Sep 2021 06:10:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOZk7-0007GD-7k
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:10:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 20faa24c-9c0a-41e6-bf83-4b6531a90ab3;
 Fri, 10 Sep 2021 06:10:30 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-7-7CqhNc6IPtKDaK_k_fppfQ-1; Fri, 10 Sep 2021 08:10:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5744.eurprd04.prod.outlook.com (2603:10a6:803:e4::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 06:10:25 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 06:10:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0028.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.8 via Frontend Transport; Fri, 10 Sep 2021 06:10:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20faa24c-9c0a-41e6-bf83-4b6531a90ab3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631254229;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=m9uOvY/Pl51wD3i0eQQ8FBhnALb8LVICAwNjRHuvdLI=;
	b=kHRSW3nwzPr2KMQ0mHNigN16SAowFwAYtACEiZuKLFdFzhzTKqhGc/EZkuR9iRgJVy/PkU
	EjfGTkTUxaw3m3bR/bmGv1Wu5VqVOI3x+oV0oVHqfcLKC5gm3wnvNt/7MzO8HcSL2Zpt9L
	Oey1HmQnlhfCoxx3LAN2UXqDQdQ39rU=
X-MC-Unique: 7CqhNc6IPtKDaK_k_fppfQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kz4J99pX0C3HzDzKrGUH1osSo3NPvylBDpnmc7Edfpa212vfQX4o/YSaYIsuMecibdnELtjuAwkpT3/AdKFtvhTXSTdojUWsyS8JVPjwi16hUUHrcLh+v2QtTwd5IWaAmVfKzh9nBWhUA4BrScHI6nUH4p3JpDYtgWakNui+F12m5841GM/WkObg14UUDL/O9l0XWvujSqZjwCY4lrpaUMDd9PkDNMjlB/xgyYjpB1wecwvXAMg7XqU8TAsoozE8zhcPtT64yBkWRCNyvzvFoqAMfc5HEvCeFgNHMDq8uutZBAOdioXhY5hdTMYKHjdVrrg6XTdmjpcDEyRfGq+f+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=m9uOvY/Pl51wD3i0eQQ8FBhnALb8LVICAwNjRHuvdLI=;
 b=TovbCvxvVF9JDCBFIild87SDMxpEmdepkF5bKmGRIYQs3X1JV8O1ofwGzAUVr/K83m2EjIxbSu3blmUe+YoPbLKIvmV9MAMcUjBjMsqWk0UIKaVoX+HEm1JwCjPRj/aPrXBVEZjjQSbez+ZNii9vT8zDSmz4Ic3ami/rSzcLFzsc6AahnpG8UfC8Wh4CMSu4SUekdN8rjtrG2vEFmQMKSwFWXytPtN+xXIDvgKlWplP8PuuBZs74jRi1k/G6LKLSPZSBeExoVqB7LAfRIR7xYqbk0HFy/c2uYFCW7Pr+RtoMXPvKfAjuYg5O35Rm7jmAOYWS+LGCuFDT43UnjOqB2Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v5 01/10] xen: Implement xen/alternative-call.h for use in
 common code
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org, Bob Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-2-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ac8670c3-a7d2-cc33-5043-22474567ed11@suse.com>
Date: Fri, 10 Sep 2021 08:10:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910021236.15082-2-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0028.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e58d47ff-9176-4ebc-0c9b-08d97421add2
X-MS-TrafficTypeDiagnostic: VI1PR04MB5744:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5744B87294C68C19BC3CFDF4B3D69@VI1PR04MB5744.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hsSjKzS9as5XGdyWIsWOsNzxpyfIiUaCdMyH1K6fUaTtyY3aLbzQO0duO7vPRCSFHWc21sNwjbhCa0lCJ5Hb4so/B0H5RFFEF83WC4MO4EGkTLVMfmEKIEc3J6eYYbNuF1A0YQOH1prMJBUieyPv5hRgdDyhMMrWued5UGmC6j6eomPYVsPHDVg6aSGFabsJmPQYX92XxRWy9PkOBSYsvhmtMxtiHGr4hi6TA/RtccieNPGQMC+TYZKHiNSrIfUy7es5qxiR+hJWjnoU36+qhSbBujDPiQz0fkFf0OQtUtlYta/6olgUDN0RbdkfDFRV/AC52NEOA2Rht2EsxsZ8FD8sKNVC9kj4klpIVF0lZE+NGb5+QytdI42AmpWaZRqfPS6st888FNn0ewlN5yQKTKnNd36OaZI+KiN1z7+9OfDFu+rKVNo1bauvK+xkxPbxJEKxabZNf1ptDj16gXajMz66AZP5tDLO2bryeuhGRbKTu44VIhweM28CTsmRLKplqIRX8GejOOzEGL0H7s0sSlo9X9X3ctB7VWlrsMXuRVm1mYUQz0okslwOwA/LPrXbgyoLn+y5EZhvZXKX7EXL2VbKm9Kr5Q169E3VecNnhA8H0kOXUUaMMuLOQUZWoDZw6OURwknLxUxvZTVF8m/BjMzcdgmYirDdZd6wISbjHLTv1cweyXHir1tfc5+2oJ538zBo4hm1c4ArHExNgESd6pDlX+unxtAENc4GCp3J4kQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39850400004)(346002)(366004)(396003)(376002)(136003)(956004)(31686004)(6916009)(7416002)(186003)(8676002)(86362001)(4326008)(478600001)(2616005)(16576012)(66476007)(66556008)(2906002)(66946007)(38100700002)(36756003)(26005)(316002)(31696002)(54906003)(53546011)(6486002)(5660300002)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NUNXV01XSXdGMkE1RW9pTXZGSExmVC9YZ3h0cHFCVVF2NHN2ck9jMGlZNFlB?=
 =?utf-8?B?Y3gxSXc3MllZY1p0aFJ6UXl6MTRsaWZJT0lNajd6dGtqcE5MWk85ek1QbkVV?=
 =?utf-8?B?aWlWS3hSaHJvWFBKdFdKSzhacytHSjJqRXRQTkZFbzl0Zlh5TFc5aXcwV1BB?=
 =?utf-8?B?MmJsMXhCQjFTM2VQL1Z5U2o5V3FST28xSFlGNkV5M012ZHdKVlVUQTJ6T2FI?=
 =?utf-8?B?dlFPMVlPNGZGSUVFZStOd0FyNExFdWxRN3F6a2l2Y0lnMDh4OVYwcHd2eG9i?=
 =?utf-8?B?QkFiMlpZcTVqMVVmejQva3JQRXA1UUpLVUFZK0ZMYURKQ0dhQ001OUVkN0t6?=
 =?utf-8?B?V2NOdW1MK0FJMm9zUEtiYUNnRjlIUlFNRlUxaFhZR0JsN29wV3crRGVqZDBv?=
 =?utf-8?B?SmczWUpLTWJJR2RFWTVKTzVZMnp0dzg5U1hkRThobTZiVG1FQnR4WUEvY1o5?=
 =?utf-8?B?T2xZWVIvRkpZem5Lc3BMQiswaDM1L3MzMXRDZDhEWmIxZnk3N080TUIxZ1Ra?=
 =?utf-8?B?MStqOU1ua2JkYkhlb3NnYUdTejJpZGNEbktaOUpiSXRDSzh3MElxVFNWTEVN?=
 =?utf-8?B?cVVKWldsOHZZb1JVQ3A5REdDZldwNDVlRnJGZERmZkR4SGFJSVFuckZXSU5v?=
 =?utf-8?B?UExrQzJYcGJ3ODhHZXp6SktUeTZzcXlrMEh5UDZUZkZDY3RWcEYyVjZhUWlQ?=
 =?utf-8?B?ckNveDRNQjBJaUpYZGhQa2txNDhPR2hJRllBVm5aRkRkRllIUXhlNFFrUkFm?=
 =?utf-8?B?QnRjdE5TSkFMdGJ3NmZxSXN2Z3VnRkFTaERJVzlCZjNaeEYrMHZsdjdwUEw4?=
 =?utf-8?B?QVdYRXprNVk4bkhEODRrUTRQRVp5VitSYTAwRUZ1ZmNydlhXVXBQUWRVZEZQ?=
 =?utf-8?B?ZStwaTVXaUNJdTRQZ2pRRVhia1cwWk9HS3VrYmdvaG1DOFpIRTAyWWZ2VjV1?=
 =?utf-8?B?UlEyc3ZPMGRsVStqK0s5SWZDTzlhRHZlVlFCV2hQK2RvV1hjcVU4NTQ5bHpK?=
 =?utf-8?B?NzZsTXd2aCt2UzdqMmN4UkYxbU40ZkwvZjVWcmJYeHhwZ3NtTEtoWXZsUk40?=
 =?utf-8?B?THVSaDZ3ZFVyL2p6bXAwRXg4M2RnTld6aGh3UmZ3bkZiOXBqV21rZGVEdzJD?=
 =?utf-8?B?a2xKOWJFK2I5QmJyRFNIU2JZYnBYL2JRdForN245dGMxZ0x3ODJQeVZFQmEz?=
 =?utf-8?B?SE1CNXZ1Nnl5REpvQStrQkdOdTRyeWFVMVBMOTc4cGV5cGJxRUpURkwrTzJs?=
 =?utf-8?B?TXdISTROMittNjhra1lZZnNMcVBQT3lJKzUwQXJPRjlDS2hSZ1VlRXFnRG5p?=
 =?utf-8?B?QlVuMUJMd2lqWExyL3NmYjFqbW51WWtpWXJKR1llQ1hHdktKR2lzOGVxTHQy?=
 =?utf-8?B?Ymtsdk9vTytXK3ExOTl0UEZRWloyZS9TZVZJNmN3SGViRmJSQndkNUdGUzhH?=
 =?utf-8?B?d0ZMVlcrQkNTYnlIelVTUFQza3hmOW81QlQ4UzNOVU9YR0p4dGtJUkV1SjBX?=
 =?utf-8?B?ZjlrWHgzemV6UWNEa1hNOW5tczBYMFJwZFRpTGlCK25OM082NXRBQUhsN1BG?=
 =?utf-8?B?cWs3RnNaVS9qODFOZkVuNkZhWjI1YUdMOVB2MHVBcWtwT2lrN1BPQjBoSUdu?=
 =?utf-8?B?Q0ZUV0ZIazE3bDMreE4wSjIzQ1FnUGh6MGRXbjVzV01KZU0vdTB2M3RGdEZK?=
 =?utf-8?B?cW1ycGt1TG1YQnNUbVJ4YmY2Y0g4MWpCYjlkSkdmTEU5djIrcHd3bUJPQTgr?=
 =?utf-8?Q?eHQygju8BLZ7muFXolx8Rv9LNx+nSjN0+6rFTD9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e58d47ff-9176-4ebc-0c9b-08d97421add2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 06:10:25.0091
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e+pZC32I+a9UqB552BqoFWR7v/OG9y8XOuBfxAr5D5Qz5i4oyWtDxkvlDmoVBNrqMaKdUHkU4zDHvnksTwK1iQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5744

On 10.09.2021 04:12, Daniel P. Smith wrote:
> From: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> The alternative call infrastructure is x86-only for now, but the common iommu
> code has a variant and more common code wants to use the infrastructure.
> 
> Introduce CONFIG_ALTERNATIVE_CALL and a conditional implemetnation so common
> code can use the optimisation when available, without requiring all
> architectures to implement no-op stubs.
> 
> Write some documentation, which was thus far entirely absent, covering the
> requirements for an architecture to implement this optimsiation, and how to
> use the infrastructure in general code.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

I did offer an A-b on v4, dependent upon two small adjustments. Was it
deliberate to neither make these adjustments nor add the ack? (At that
point I did offer making the adjustments while committing, but I think
it goes without saying that upon re-submission the changes should be
made. Unless of course there's disagreement about making them in the
first place. Yet such disagreement would be worth making explicit then,
imo.)

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 06:12:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 06:12:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184066.332594 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZly-0008CL-Hc; Fri, 10 Sep 2021 06:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184066.332594; Fri, 10 Sep 2021 06:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZly-0008CE-EP; Fri, 10 Sep 2021 06:12:26 +0000
Received: by outflank-mailman (input) for mailman id 184066;
 Fri, 10 Sep 2021 06:12:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOZlw-0008C3-Gm
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:12:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 04723aa6-d4be-4eb9-82ee-0b1da2f406e9;
 Fri, 10 Sep 2021 06:12:23 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-pZt-J5z_MeK8YH6e6R-K3g-1; Fri, 10 Sep 2021 08:12:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 06:12:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 06:12:19 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM9P250CA0011.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:21c::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Fri, 10 Sep 2021 06:12:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04723aa6-d4be-4eb9-82ee-0b1da2f406e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631254342;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mSYVLDhbTGIq2aNJOYktFKxjwoOudL5Ur6cAUfGtOXA=;
	b=BANUZWP68SNE4BaWKShWKDAJYMPeCEptgjlUnpjskhc+NEsaTgeeshOJ18Iyc+E+haXHZ/
	wrHBBC+gQh/9L7SPFBIHzP5DYmbkWVBieZ3Nc96bz6ZJYQ2cOl2uji/9vEnwsBK0gffInK
	3gtNCQnxLaeZsbnA+IBhSP6cb3byozc=
X-MC-Unique: pZt-J5z_MeK8YH6e6R-K3g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KK1XLAthrU903TuE2Yiy4hpLWWLjnzSicitxVpKo2hlL9c+q2Ke2GoZsKwyrSlyMqf5gMVjpiflsisNm6EHr/4DEB9mt7YE0dUQXO01Let3sQaOm9wJWIF96Ej5mdl6tSIC0wzlg83fpyXNPDVlRjJgZa2gQWk34OamHW+5iri196qxOSbWJ9er5cnvQjBZKiBv3/fVOXWQAtXUwhhWxmFc7ci7RhKcSFm1G2BHvU2qt4hCxTniGAP/WWH033YGpupbme15+r7fxCKR+DPK4dS8rGnUtSqc7JzlgEA+9Velq/7phf8MGSL6iVPG/FdSwmgSP/UHrB3g/dMA5HzGyXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mSYVLDhbTGIq2aNJOYktFKxjwoOudL5Ur6cAUfGtOXA=;
 b=aEm3qJBNlBNXUAHFFu+hhoHVyXA1DRCCOE1tUQZQ4PpJsEc8KBsQDdnVoWy/B+hlCN9cTtp3AB7Ajye/rCHzRuAeucwddt3CVyR85JjTGPM2PNdPDpoOp7ayD3aV4zN/bOIyd6emOKuyaGWeuhTS6H8ZNLhOihJXvT+q8Jzmq092gSNs6xKYksAemY4/q4WPmw3pyUf9Wb3D/0yVSTbtns4MvhRod2wN6QoEEm85zXvFLrMbkIrE/qj3zZbCuklpvoEMuOYV5TQCZkNLkUM6OajghH6mKUzSp112603Jhk2mgfNXWkjm6I8QJYdX/YOA/EAb15+xiUhXannHu2kTLw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v5 03/10] xsm: remove remnants of xsm_memtype hook
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-4-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b69b98ca-afcc-3943-d305-706913b8c16a@suse.com>
Date: Fri, 10 Sep 2021 08:12:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910021236.15082-4-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM9P250CA0011.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:20b:21c::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fa2dfcb3-4db4-4e17-0770-08d97421f244
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70384648001324A022817DC3B3D69@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:173;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lbUODLHmWdjPa9XCml0bhaBuKPqxXtP5XDyySuALifegv70iM7wbrfVfOcQAmxbHeg0OuyNsqeDq7xukvnYfWcZbVWxXkNBZ2rteu9glUDJgRQTFU6j420eexxBy1WgdpJakOXOUwKT1tWVZZIy9LzNMdgAN4Ja8RnLpT1Vfm+GvXVnnO2Q83kRVtfDBvn4A6kuvpquMeRGg6FUN7AgMXx2ZJCu9FBbNzqIIZCuBqm/3b0OaoKb0fldhrP8UOfRCSK5TjhSDSHx2kZ39WRgjfXbVwOeBCGKpgICG2+5fcAAl67MhqojrtHZQTHyJdyAnPeoSlkCpKUXgRFM2kOR6u+fmEhJUwy06YLNT9ct0XofScqbxyWLbw8rzbgRl2MNaIZ2jD18CWjJq93VWKutnlJVYS6VqCEzDcEEV1VHCabJk/Bk7Os5NF6bln2r2yEIBg1RBz0u7qCi+L6uvdDYUIStQo5tG3KrZkzqcXWN6k+zsGbOXycLFwUMgrrr/NjAgJuegJaErcFfynF4ILfLpJ+OXU1eLimZBH40ux8JrNQJsAaw4XZHIk5tY1nPiwxI+Eizx/j0db8Yrp4wsvSKvAoGr76pUdKznYvV41ZCV6leTTRXuQI9MKDbdwErp4izkTTQrjP0ZimPzj/ihiB3T8NFl3WvDbii17NKXJgZGEUtJQQBo65zOxRICIZG/1EsNkxXa53/Qw22uqIVYeDoyswP75Hna8/v5/sEmLkr9OQ3sQ4H61/8hyOctodUsgbGM
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(39850400004)(136003)(346002)(376002)(6486002)(8676002)(186003)(2616005)(38100700002)(66476007)(66556008)(26005)(31686004)(8936002)(2906002)(53546011)(16576012)(31696002)(5660300002)(956004)(86362001)(316002)(478600001)(66946007)(4326008)(6916009)(36756003)(558084003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?STRDbklDcGhZTlpMWkQvK202bGdZckhhQ2JVOThTSkszK3pXaUlYVkhpaytL?=
 =?utf-8?B?LzlvYnNTaFBCVVhpR0MvZ05YYnRndjRFZ3hJR0dtTE5JRkxLRTNIZHNwL3B0?=
 =?utf-8?B?cHluQ3o3MWpsQWNjSWN5bTlweHhCMVhBRHFZZ3VidUkvbm9rOW9VaUgxaFF5?=
 =?utf-8?B?b1NwakgxeW5JOHkrY1RSTkJSZzhtQ3BLSDBLeDAzMEtoVkNONE0wU1p2VVEw?=
 =?utf-8?B?MkxRK0tBYWhqZ3B5UTlOeVVPUkZjZk1Gcnh3Z1BFdGdGbUVUV0RDU0VHMXd1?=
 =?utf-8?B?QXYxZ1FhUFY3OVhSelhZTU1remZCNS94NmRQakF2Tm9HRVY4UzhnTkNiY2FW?=
 =?utf-8?B?cEJIbWdqdkhHMmhjUWs5a0lGZXBTUHNoR1U3blBTVVJhRkw1bXNTMzAxb3Jl?=
 =?utf-8?B?ZHgvck1GZDE4c3NKQ3hNeE9OUGMzc1RidEMwQ3RPcmM3Z1J0ZU1LVGxYQnBF?=
 =?utf-8?B?Q3l6bnhmM2IvWk1JeTZuaWJleXpWNVNhUjgzYnVCV3krQ3psbnhHOUNETm5n?=
 =?utf-8?B?TWpod2Y3eFExV0Y5a1d3UTgxeldUWVdUL3A2TGxnU3hBWFZjbGE4cnpweEpE?=
 =?utf-8?B?U2V0dVVKQjdMM2d0ZlF4NFNqNjR0aVZUOVVvck1VRUNMSnh1MHU3a3dMb0tt?=
 =?utf-8?B?NFhKMEhzZjZLSlFvdnlCY2lEMlltQkRwRnFVTm5hcVp4UWMydDlkRlhpVWRL?=
 =?utf-8?B?T3B1MDBmTGVtR1YxMTdxMzVjQ2RCRjMxMnBjV05SV2hpMzdqNzNScmlZTjlG?=
 =?utf-8?B?b0FqSEhJWnFrWDRZZnE5ZXF4WDhrbTFyYUdjdng4TVVnaUxoQnJKN25yeWpR?=
 =?utf-8?B?TDJsVEtJdlljcU5Yb2dnL2FqY244TTFzcjVTd25JSTBSVktTSU9Bcit1a0p4?=
 =?utf-8?B?aldLWmZCdnN3L242WFovYVhSeXMwV29TMVNGVFYyUEQzK0pBeUgwZU5PTHJm?=
 =?utf-8?B?bWpzZldqa2M4Sm9SaElJZFZYdjdDWUtrcDZJL1NqWVkzTzdldTVic2NRRm1X?=
 =?utf-8?B?c01oTlAzWE83dytoZUN5WTBKa0l0ejFPUlFyVUlPMXlmL1pQWmdxU3oyTVE0?=
 =?utf-8?B?YWFUbjRMckFpQjY3a21DRXk1NEJGcEprQmM3ZWdsMGV3Z0VjcUZrRE5qK3Ay?=
 =?utf-8?B?S3VMdzlwZFRwLzlxdDhtendEZ0JwNkE0bWZQZmVPVXN4OFNaR3VIRFFpTzNt?=
 =?utf-8?B?a3ZqaFNFclhsYVExMGM5MHVlYTJpOUdabE1FQ2FvQjhPd3Rnc1pDQUhGUEdj?=
 =?utf-8?B?VlVweGk3WmZNUHdNdGx3YzA2RFpjRGR5ZkdqeVc5TzhPQ1lJcTIzcGhKT1Ny?=
 =?utf-8?B?SlJvelhjOHRTOUhnMVZpMmpRS3gvMnkvYmRzQTl6YnVaeU9tMEpMeHZCV1lY?=
 =?utf-8?B?VXJMZi9oNXJVMVBLOWd6TEZYL1M2Rm9GWFg4VWhhNnJmclRWNVpYUEY3TGVk?=
 =?utf-8?B?cS9VYVp2ekhjNGh5T1pxemw3VG5vWHlNYlpOS3RIZXNENC9zV3ZQaHNPdjQ2?=
 =?utf-8?B?ZFhxeVlIT3hvUER2eW9Gc0ZqQnRMN3dobyt1b0R1bCtpVDIyYXNXOW05ZzQz?=
 =?utf-8?B?SXNtWUJheUsyT3dONmIwbS8ybm5hQjFHUkttRHB6MktRelFqYzh4YmVnZjFC?=
 =?utf-8?B?bk1zQ0RGVTkzemEwc3RDS1dEeWYwNFBIb1h0bmo1TVIzaThhTE1ZT1lBZi9U?=
 =?utf-8?B?MTFSdnd2TmpORTM2MElOZ21xc01ibW5Sa2g1WWtxWTV0dXZ5b1ppYjl2MHNr?=
 =?utf-8?Q?xAuHJU/lFs/64isONRZKYV+HI8PNDSjXe+33zuT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fa2dfcb3-4db4-4e17-0770-08d97421f244
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 06:12:19.8482
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3mYwecBvnRpqj4wA0q4ZWlclAGmvOCQ26hxBTZmg46BZ3xOv+g4srbHYFKg2IHq2qgJi6VJIyVQQxU8M4Qkggw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

On 10.09.2021 04:12, Daniel P. Smith wrote:
> In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
> left behind. This commit cleans up those remnants.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 06:15:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 06:15:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184071.332605 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZoj-0000Og-1U; Fri, 10 Sep 2021 06:15:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184071.332605; Fri, 10 Sep 2021 06:15:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOZoi-0000OZ-U6; Fri, 10 Sep 2021 06:15:16 +0000
Received: by outflank-mailman (input) for mailman id 184071;
 Fri, 10 Sep 2021 06:15:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOZoh-0000OR-9J
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:15:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 75c9cca0-11fe-11ec-b210-12813bfff9fa;
 Fri, 10 Sep 2021 06:15:14 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-7-6s-UVfiWOA-GJXKrP2ThHw-1; Fri, 10 Sep 2021 08:15:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 06:15:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 06:15:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0035.eurprd03.prod.outlook.com (2603:10a6:208:14::48) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 06:15:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75c9cca0-11fe-11ec-b210-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631254513;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oa6kPufnjAtEzVpGtQRxfkESNGWmP/ZewBL4bZsgzo8=;
	b=iOMdzYspu1DvCCI2et5B7zYyhAvLAOE9+S9xEmnagz86a1XjwVLRLi6IzY6bjv9fftSzaH
	EtoEHPqHhjAQjJqqXduIU3DAjcKzsgvTrgLTCqJ3EFUk6c85dIrxPvU93mrY2dAAsW0htL
	W2mAb7gFIlcjByBovEuzJaD/jrNmrE4=
X-MC-Unique: 6s-UVfiWOA-GJXKrP2ThHw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P/XWkjstGGo7eXhH5EthMNse6B8Am9T8n4jSNytI/5Eoh2r9jxEYK7mVcq2rxx27nEL5TY31z9jprxHmZspMR/Hy1W5AV2rOGQDJ/EduxaQa4qRQfyRH1Ld87ZWf3SKUqi952zE1j7EtIa2Yrddf47DHMyPhLu0DLCTwkw5kX1hXeIVBttCzYQC3cTqXPB+44PVwFwCFyhvuNrdEVqnEeJisRboQJqx4YD55Tu/ZdHXkO6gdGx3aevZYx1csvvMr2L9gfHGqsiJzMTYGg7N6Y9yL88gerEe2p9AGEfeWjRVeH1XYM+VdVMb7GJfgeeiP8EZO+82BjHv42ZqG3DVZXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oa6kPufnjAtEzVpGtQRxfkESNGWmP/ZewBL4bZsgzo8=;
 b=DSZDP5LHsnNwx9IR6r4R+mRTDThajz9d9q44YFc2JWlZ+6napwrBAKRI5hxQBX9z1GoVkTUElm+KcvWfRfBPwI8VBUp/oYeQ2GWGXb6+C85UwTSLCup5zpIngt6dcOPjVjZGyHlRwp7G9o4fqJ/lqYP19rY7T2uNi91S+XvLam45LjGvc8qnIPcigq5JWedpv+U1HW32b07knSbZJN2esFMalifBFYEheKEqeJdCDt+FmcrawmsYQVFOw1cJzkZAHepmf3FaV5BFO4HU4pLwTRkAvuchl/jKLs8bzOssRb5GGZD7+W/wywIlkGVwaARqAsbKWnVGvyoL55WibgqkxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v5 05/10] xsm: apply coding style
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-6-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ea0a2e43-6c8c-bfcd-34aa-ec181600ea2d@suse.com>
Date: Fri, 10 Sep 2021 08:15:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910021236.15082-6-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0035.eurprd03.prod.outlook.com
 (2603:10a6:208:14::48) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1175bc13-a1b9-4afd-3ec5-08d9742258ed
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB703828FFFA4D9EF9F7988F7AB3D69@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+WzLWLfgkZS6WzC3PxUR/jA4Q6OjKN5LqdXB2QelYN5yP1tjfVOT4pnpJPBLIPU/JEMk39KLeNLQV6oylpgCHNArt4KWdw5kfWU2cNftKF/rcYw1cbHekB9oxBrSeLyCPWLcCW9SZ2dWYILSRTSWQZlbh19WZisfJlsQFBDWxRAagQX3R5STRVYQL5vHCuDh/DYp4tIV9NM4QJ76bTrxnlSSIB10kuYuAuxGW4lUlUqGSO0RQFh7SsTKIvbyzA/oLjI6lsS0O+uptQteX0k6LUyznYZrCS/Uif+pv2Too68ZWgaQx1fA/XMDyJds3Yx9ZyuD8TN3jaz9MDC+/SY2/Fed/Vr5FVrbfhJA8gPP1W76SRaXbkUX0H5c33GB+hzxWsBsEuFp1TfNWx3971cpPCGbTTuYkPTR3ICFJjRHrys5SdRFiE3OQuET4BnjXnugatFSE7K3CJ7nb+r3cVX3JjzTa7JXKNotw1XZ44Ahy/Msqj4oxmTFiuCZN+MG/jlG3KiN9+W3C9P7hB2FNLpaC83Kg4Qis8o7NVtNW9KEAAbpeOztMSk9vbFPfClAwZRsCjMotZVN+6oUsaGMWCvs7EeQq+DOwtovvzC4ohNn2lhX1hwsNLZE8Xal3uV/SGfNSoswXsQKOUCGT5+6yLSg+HFaMg5G6NYCqPMdbYzUcHTUWck3BEmX3PyqGw0onOwxaU9f3+xyqtFwIhcBtiEIbEJlh5U6Y0G7AEQ2GhcdsFyCbkQeNI+qHmvO+fyJDY9o
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(39850400004)(136003)(346002)(376002)(6486002)(83380400001)(8676002)(186003)(2616005)(38100700002)(66476007)(66556008)(4744005)(26005)(31686004)(8936002)(2906002)(53546011)(16576012)(31696002)(5660300002)(956004)(86362001)(316002)(478600001)(66946007)(4326008)(6916009)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MlFkVnRvZlFnUE9QYW5naTBXS1ptenloRVFtVXF1V3ZacFZNRkEwV2U2MWd0?=
 =?utf-8?B?VU1na0VFNTMzSHVZYW9kdjlwYVNVMzhIUjhGTG9qcTU0dEZGMGJwYjN5VGly?=
 =?utf-8?B?OGMwaHVqV0JqamtMTDJwYTF3UUNyQzdsVVo0U3R4Rm5peVJTbi9rNVVXakdC?=
 =?utf-8?B?WElNclU2N1NnSGM4VmMyU29lUm4xMWpVVlJJQ0NHS3hwcjYvK0gvWmxvUTcw?=
 =?utf-8?B?OHJjY08vaklDSHpjYldMcWx2aGtkbnA3WFdkeVZzc1F3YjkveEhMc3UrSzZJ?=
 =?utf-8?B?Sm1wUW9OMWtwTFZEZHRzK1doN25UclJ6RFB4bGxLKzlaWWxua21JS1pSakdq?=
 =?utf-8?B?M21jSXRuYWE0Tm0yWE9PVjRBaFluWnFCZkpqMThIQ2VvT2hsZHZCYlZUMWU1?=
 =?utf-8?B?TU5BeWZQTi83Y25YaXQ4Z1ViNUU1QmFzUkRvN1QyUkNPR0NkRjZibmxNa2FE?=
 =?utf-8?B?ZjRoWStDQnB3WmRlY29pdlliZkZUUW1WakFFV05yWnFSYy9TVHRZa09GVVNn?=
 =?utf-8?B?VEhOS2ZxWU8yb25WWjJwajFxRmJqZEdjeFRTanN1ampLWjBnQlRFdVNuTUx3?=
 =?utf-8?B?SWl6dVBTVGRHM3pVOUlVMS8wVUNHMzVWVURvZVRGbjg5TThOaXNETHJCTHN4?=
 =?utf-8?B?RmN0ZVV6U2JseVpBT2FkSkZ6ZU1oYlFDUWpFUHVSVE5WdlhYOFFBdkY2SjVW?=
 =?utf-8?B?OEZmTW5lY21DU2hGUXNrdmpoTTBINlgvd0dUeUFqOTJYQU9IUVNtMVVpTEVo?=
 =?utf-8?B?bDMxRUJINlVtL3NTNHhKUEZGd3hGekFGVTFVT09NNTNkeFJoUmVpTnFPcnhx?=
 =?utf-8?B?MTRabkd1Y0pTKytJbUo1cXZZdmFuU3JqaUM5U0ZRUXJyRXRzYUF3Y3FRRmE3?=
 =?utf-8?B?MVhCMG5XVHRkT1AvQ3VtanFqQlFibGZ6L2ZGOFZIbzRmM21hbm4yRy9JYmJj?=
 =?utf-8?B?L3VxdStOa0djbnVhUFlFZHBuN00xR1ZWamZWQU03ODM4WUhxR0JYdHJqZnc1?=
 =?utf-8?B?T2lsZmt3MjZnMlJaenFxNW1aZmNpdU92SmRibVRaSFV5THdGeTRJZFVhVDZn?=
 =?utf-8?B?ZXM3SFdyYVNKNGN1bnRxKzNvWDFsdVpvZlpUYStiM0RXV2gvTE1CSFVUN01E?=
 =?utf-8?B?QXVFWDlHeXRyTldjSFg5OWZLWTkwMHU4ZkxwVUpqdHBrZ2x6Qkh5SFBoTXg0?=
 =?utf-8?B?UHQ4WGNndWFjaWFUMnQ5TmNCR2paeTNBWlpxdDNnMFNEWlZsM2w0K2w2dlNy?=
 =?utf-8?B?V3ZPWTJEcGoydFk0NDBGVDhZTThjQlJVZFRvOTVTSGRYcWdQdGtVbTRSQ29h?=
 =?utf-8?B?T3FFT0V4VlNZTTVXdTNGWVR2emxSZUdlY2dTUnhSZzZzSXZHU2ljaVkweUZG?=
 =?utf-8?B?UkJreTZHT0c2b2JId1IyYlNFNzVRUVBZZE03MjNWL1pVN0hGOElqLzd3MElK?=
 =?utf-8?B?Y2hlMWhzZHl3NzE3UmJxWVhDZGllNHdTdHlMaWFqMjF0WnBwakZ6dFpGbG1m?=
 =?utf-8?B?cEFZdjBFcEFuNTAxUWU1VmxYKyt5aWRSWUFWWDl5bEhDYzZrWWoyMWhnd0U3?=
 =?utf-8?B?L2xvK3k0ZlljM0lOY0w5enZuMU95UUxtQkJjYXpGRFpjSHdwcjhvd1AzeUps?=
 =?utf-8?B?TXFyRnl1d0YzWmFhU243L0Uzd1U2dUMyaGdmS1cxOEhLNWJOM204NzVoc3BX?=
 =?utf-8?B?R3piMk9WTll6RmNvZjlXU3c3QkJjRWdEYnlvL3pDN0NtUUY5N3RGOGlOM29L?=
 =?utf-8?Q?G1qMLbHGqc7BsMME1dhtSRdf2+hwK3MbdXcbTbo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1175bc13-a1b9-4afd-3ec5-08d9742258ed
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 06:15:12.0768
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: X82tJxGKQky9uldzStufKCNkeuP1WaPGj8cp+f+v00bRjBQ43H2ViNLy70zdckgGg2AemcAmlgGbHNLUNpR0Dw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

On 10.09.2021 04:12, Daniel P. Smith wrote:
> @@ -70,7 +73,7 @@ void __xsm_action_mismatch_detected(void);
>  #endif /* CONFIG_XSM */
>  
>  static always_inline int xsm_default_action(
> -    xsm_default_t action, struct domain *src, struct domain *target)
> +	xsm_default_t action, struct domain *src, struct domain *target)

Here and below you're now introducing hard tabs. In Xen style
we don't ever use hard tabs for indentation; this is limited to
files inherited from elsewhere.

It's also not clear why you've touched this instance at all:
The 4 chars indentation was correct already, as previously
pointed out by Andrew (on perhaps a different example).

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 06:40:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 06:40:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184080.332615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaDT-0004Ah-77; Fri, 10 Sep 2021 06:40:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184080.332615; Fri, 10 Sep 2021 06:40:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaDT-0004Aa-4D; Fri, 10 Sep 2021 06:40:51 +0000
Received: by outflank-mailman (input) for mailman id 184080;
 Fri, 10 Sep 2021 06:40:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOaDR-0004AU-HY
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:40:49 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 07450c1e-1202-11ec-b211-12813bfff9fa;
 Fri, 10 Sep 2021 06:40:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id EB2921FD52;
 Fri, 10 Sep 2021 06:40:45 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BF20813D12;
 Fri, 10 Sep 2021 06:40:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id IUb8LO39OmGIIAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 06:40:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07450c1e-1202-11ec-b211-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631256045; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=i6QZbTx5XiHB9TsuBBNdDhq37UbkhMYg1y+2T87ru54=;
	b=VGjhwRu2GxmPQY5YcDcKf4+pHq9zLYlyFZwFyAZFpkwKDJUQnfybEtJ5zCcXM0IDBbp650
	ICipyAR1+kxWKu6eyuYnjbaKlZ3k/6kl8a2OiMhfhqWubfIHnXt0TunipdTqo9ZOiRcaVG
	v/LeKssGWFa/GNhDfPBejN5dbWGmLII=
Subject: Re: [PATCH] mini-os: xenbus: support large messages
To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <722703e5-efa5-9868-5926-79a449e90285@suse.com>
Date: Fri, 10 Sep 2021 08:40:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210818152610.6114-1-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2
Content-Type: multipart/mixed; boundary="W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org, wl@xen.org
Message-ID: <722703e5-efa5-9868-5926-79a449e90285@suse.com>
Subject: Re: [PATCH] mini-os: xenbus: support large messages
References: <20210818152610.6114-1-jgross@suse.com>
In-Reply-To: <20210818152610.6114-1-jgross@suse.com>

--W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI
Content-Type: multipart/mixed;
 boundary="------------B3256F1D978D2E4AA868E07A"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------B3256F1D978D2E4AA868E07A
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 18.08.21 17:26, Juergen Gross wrote:
> Today the implementation of the xenbus protocol in Mini-OS will only
> allow to transfer the complete message to or from the ring page buffer.=

> This is limiting the maximum message size to lower values as the xenbus=

> protocol normally would allow.
>=20
> Change that by allowing to transfer the xenbus message in chunks as
> soon as they are available.
>=20
> Avoid crashing Mini-OS in case of illegal data read from the ring
> buffer.
>=20
> Signed-off-by: Juergen Gross <jgross@suse.com>

Ping?


Juergen

> ---
>   xenbus/xenbus.c | 212 ++++++++++++++++++++++++++++-------------------=
-
>   1 file changed, 124 insertions(+), 88 deletions(-)
>=20
> diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
> index 23de61e..3fbb122 100644
> --- a/xenbus/xenbus.c
> +++ b/xenbus/xenbus.c
> @@ -29,6 +29,7 @@
>   #include <xen/hvm/params.h>
>   #include <mini-os/spinlock.h>
>   #include <mini-os/xmalloc.h>
> +#include <mini-os/semaphore.h>
>  =20
>   #define min(x,y) ({                       \
>           typeof(x) tmpx =3D (x);                 \
> @@ -46,6 +47,7 @@
>   static struct xenstore_domain_interface *xenstore_buf;
>   static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
>   DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
> +static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
>  =20
>   xenbus_event_queue xenbus_events;
>   static struct watch {
> @@ -231,75 +233,105 @@ char *xenbus_wait_for_state_change(const char* p=
ath, XenbusState *state, xenbus_
>   }
>  =20
>  =20
> +static void xenbus_read_data(char *buf, unsigned int len)
> +{
> +    unsigned int off =3D 0;
> +    unsigned int prod;
> +    unsigned int size;
> +    int notify;
> +
> +    while (off !=3D len)
> +    {
> +        if (xenstore_buf->rsp_prod =3D=3D xenstore_buf->rsp_cons)
> +            wait_event(xb_waitq,
> +                       xenstore_buf->rsp_prod !=3D xenstore_buf->rsp_c=
ons);
> +
> +        prod =3D xenstore_buf->rsp_prod;
> +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, p=
rod);
> +        size =3D min(len - off, prod - xenstore_buf->rsp_cons);
> +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
> +                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), si=
ze);
> +        off +=3D size;
> +        notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D=3D=

> +                  xenstore_buf->rsp_prod);
> +        rmb();
> +        xenstore_buf->rsp_cons +=3D size;
> +        wmb();
> +        if (notify)
> +            notify_remote_via_evtchn(xenbus_evtchn);
> +    }
> +}
> +
>   static void xenbus_thread_func(void *ign)
>   {
>       struct xsd_sockmsg msg;
> -    unsigned prod =3D xenstore_buf->rsp_prod;
> +    char *data;
>  =20
>       for (;;) {
> -        wait_event(xb_waitq, prod !=3D xenstore_buf->rsp_prod);
> -        while (1) {
> -            prod =3D xenstore_buf->rsp_prod;
> -            DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_con=
s,
> -                  xenstore_buf->rsp_prod);
> -            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < size=
of(msg))
> -                break;
> -            rmb();
> -            memcpy_from_ring(xenstore_buf->rsp, &msg,
> -                             MASK_XENSTORE_IDX(xenstore_buf->rsp_cons)=
,
> -                             sizeof(msg));
> -            DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(m=
sg),
> -                  xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg=
=2Ereq_id);
> -
> -            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons <
> -                sizeof(msg) + msg.len)
> -                break;
> -
> -            DEBUG("Message is good.\n");
> -
> -            if (msg.type =3D=3D XS_WATCH_EVENT) {
> -                struct xenbus_event *event =3D malloc(sizeof(*event) +=
 msg.len);
> -                xenbus_event_queue *events =3D NULL;
> -                char *data =3D (char*)event + sizeof(*event);
> -                struct watch *watch;
> -
> -                memcpy_from_ring(xenstore_buf->rsp, data,
> -                    MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(=
msg)),
> -                    msg.len);
> -
> -                event->path =3D data;
> -                event->token =3D event->path + strlen(event->path) + 1=
;
> -
> -                mb();
> -                xenstore_buf->rsp_cons +=3D msg.len + sizeof(msg);
> -
> -                for (watch =3D watches; watch; watch =3D watch->next)
> -                    if (!strcmp(watch->token, event->token)) {
> -                        events =3D watch->events;
> -                        break;
> -                    }
> -
> -                if (events) {
> -                    event->next =3D *events;
> -                    *events =3D event;
> -                    wake_up(&xenbus_watch_queue);
> -                } else {
> -                    printk("unexpected watch token %s\n", event->token=
);
> -                    free(event);
> +        xenbus_read_data((char *)&msg, sizeof(msg));
> +        DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),=

> +              xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req=
_id);
> +
> +        if (msg.len > XENSTORE_PAYLOAD_MAX) {
> +            printk("Xenstore violates protocol, message longer than al=
lowed.\n");
> +            return;
> +        }
> +
> +        if (msg.type =3D=3D XS_WATCH_EVENT) {
> +            struct xenbus_event *event =3D malloc(sizeof(*event) + msg=
=2Elen);
> +            xenbus_event_queue *events =3D NULL;
> +            struct watch *watch;
> +            char *c;
> +            int zeroes =3D 0;
> +
> +            data =3D (char*)event + sizeof(*event);
> +            xenbus_read_data(data, msg.len);
> +
> +            for (c =3D data; c < data + msg.len; c++)
> +                if (!*c)
> +                    zeroes++;
> +            if (zeroes !=3D 2) {
> +                printk("Xenstore: illegal watch event data\n");
> +                free(event);
> +                continue;
> +            }
> +
> +            event->path =3D data;
> +            event->token =3D event->path + strlen(event->path) + 1;
> +
> +            for (watch =3D watches; watch; watch =3D watch->next)
> +                if (!strcmp(watch->token, event->token)) {
> +                    events =3D watch->events;
> +                    break;
>                   }
> +
> +            if (events) {
> +                event->next =3D *events;
> +                *events =3D event;
> +                wake_up(&xenbus_watch_queue);
>               } else {
> -                req_info[msg.req_id].reply =3D malloc(sizeof(msg) + ms=
g.len);
> -                memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_i=
d].reply,
> -                                 MASK_XENSTORE_IDX(xenstore_buf->rsp_c=
ons),
> -                                 msg.len + sizeof(msg));
> -                mb();
> -                xenstore_buf->rsp_cons +=3D msg.len + sizeof(msg);
> -                wake_up(&req_info[msg.req_id].waitq);
> +                printk("Xenstore: unexpected watch token %s\n", event-=
>token);
> +                free(event);
>               }
>  =20
> -            wmb();
> -            notify_remote_via_evtchn(xenbus_evtchn);
> +            continue;
>           }
> +
> +        data =3D malloc(sizeof(msg) + msg.len);
> +        memcpy(data, &msg, sizeof(msg));
> +        xenbus_read_data(data + sizeof(msg), msg.len);
> +
> +        if (msg.req_id >=3D NR_REQS || !req_info[msg.req_id].in_use) {=

> +            printk("Xenstore: illegal request id %d\n", msg.req_id);
> +            free(data);
> +            continue;
> +        }
> +
> +        DEBUG("Message is good.\n");
> +
> +        req_info[msg.req_id].reply =3D data;
> +
> +        wake_up(&req_info[msg.req_id].waitq);
>       }
>   }
>  =20
> @@ -451,36 +483,40 @@ static void xb_write(int type, int req_id, xenbus=
_transaction_t trans_id,
>  =20
>       cur_req =3D &header_req;
>  =20
> -    BUG_ON(len > XENSTORE_RING_SIZE);
> -    /* Wait for the ring to drain to the point where we can send the
> -       message. */
> -    prod =3D xenstore_buf->req_prod;
> -    if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE)
> -    {
> -        /* Wait for there to be space on the ring */
> -        DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n",
> -                prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE)=
;
> -        wait_event(xb_waitq,
> -                xenstore_buf->req_prod + len - xenstore_buf->req_cons =
<=3D
> -                XENSTORE_RING_SIZE);
> -        DEBUG("Back from wait.\n");
> -        prod =3D xenstore_buf->req_prod;
> -    }
> +    BUG_ON(len > XENSTORE_PAYLOAD_MAX);
>  =20
> -    /* We're now guaranteed to be able to send the message without
> -       overflowing the ring.  Do so. */
> +    /* Make sure we are the only thread trying to write. */
> +    down(&xb_write_sem);
> +
> +    /* Send the message in chunks using free ring space when available=
=2E */
>       total_off =3D 0;
>       req_off =3D 0;
> -    while (total_off < len)
> +    while (total_off < len)
>       {
> +        prod =3D xenstore_buf->req_prod;
> +        if (prod - xenstore_buf->req_cons >=3D XENSTORE_RING_SIZE)
> +        {
> +            /* Send evtchn to notify remote */
> +            notify_remote_via_evtchn(xenbus_evtchn);
> +
> +            /* Wait for there to be space on the ring */
> +            DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", pro=
d,
> +                  len - total_off, xenstore_buf->req_cons, XENSTORE_RI=
NG_SIZE);
> +            wait_event(xb_waitq,
> +                       prod - xenstore_buf->req_cons < XENSTORE_RING_S=
IZE);
> +            DEBUG("Back from wait.\n");
> +        }
> +
>           this_chunk =3D min(cur_req->len - req_off,
> -                XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod));
> +                         XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod))=
;
> +        this_chunk =3D min(this_chunk,
> +                         xenstore_buf->req_cons + XENSTORE_RING_SIZE -=
 prod);
>           memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod),
> -                (char *)cur_req->data + req_off, this_chunk);
> +               (char *)cur_req->data + req_off, this_chunk);
>           prod +=3D this_chunk;
>           req_off +=3D this_chunk;
>           total_off +=3D this_chunk;
> -        if (req_off =3D=3D cur_req->len)
> +        if (req_off =3D=3D cur_req->len)
>           {
>               req_off =3D 0;
>               if (cur_req =3D=3D &header_req)
> @@ -488,20 +524,20 @@ static void xb_write(int type, int req_id, xenbus=
_transaction_t trans_id,
>               else
>                   cur_req++;
>           }
> +
> +        /* Remote must see entire message before updating indexes */
> +        wmb();
> +        xenstore_buf->req_prod =3D prod;
>       }
>  =20
> +    /* Send evtchn to notify remote */
> +    notify_remote_via_evtchn(xenbus_evtchn);
> +
>       DEBUG("Complete main loop of xb_write.\n");
>       BUG_ON(req_off !=3D 0);
>       BUG_ON(total_off !=3D len);
> -    BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE);
> -
> -    /* Remote must see entire message before updating indexes */
> -    wmb();
> -
> -    xenstore_buf->req_prod +=3D len;
>  =20
> -    /* Send evtchn to notify remote */
> -    notify_remote_via_evtchn(xenbus_evtchn);
> +    up(&xb_write_sem);
>   }
>  =20
>   /* Send a mesasge to xenbus, in the same fashion as xb_write, and
>=20


--------------B3256F1D978D2E4AA868E07A
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------B3256F1D978D2E4AA868E07A--

--W7fjh8u4KjmjIgkNXOz4RSFwVIIAMGvrI--

--X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE6/e0FAwAAAAAACgkQsN6d1ii/Ey8d
DAf/QEDma3LEitNzsAM/u77x/bLeUxlNBaSDL76FNzjcOZvBMHtQ3ekqoYdjfjfojPJIQ3UVbhH6
rU5yGZZ3sF7oUzeFmK9YRBRAt59XMp+mxdwpMzR1kqObkPeFUwKQSiCtcJsQZTJqlBMy/84Yy+Z5
lZbd1mENpTplK6PGUz/xsf+XBse6QlKxBBRcQBkzq30N92cQI27obgTfx3SkoSZmDBRw5ji2Dsww
hvAiG3YAQenbbJMjxLfVXyL7R2F1AAwPSoHgysaH54XRlnr+U7lIy68CUxcVKj/nPiW/JQeaNqLf
mlRMNLuVceM/bgxN3A1VrZJ6yGKqbqi4dFHA/+NIVQ==
=alK0
-----END PGP SIGNATURE-----

--X9mDkyBHbWuCbILhXetosgGLPk2Phjqj2--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 06:45:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 06:45:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184087.332631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaI5-0005Dc-S2; Fri, 10 Sep 2021 06:45:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184087.332631; Fri, 10 Sep 2021 06:45:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaI5-0005DC-Of; Fri, 10 Sep 2021 06:45:37 +0000
Received: by outflank-mailman (input) for mailman id 184087;
 Fri, 10 Sep 2021 06:45:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOaI4-0005D6-C8
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:45:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eed68b8a-06e5-429c-a58f-177ade3d5eee;
 Fri, 10 Sep 2021 06:45:35 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-gXv_cU46O_mSBStKmmIJbg-1;
 Fri, 10 Sep 2021 08:45:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5903.eurprd04.prod.outlook.com (2603:10a6:803:e0::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 06:45:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 06:45:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0037.eurprd03.prod.outlook.com (2603:10a6:208::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 06:45:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eed68b8a-06e5-429c-a58f-177ade3d5eee
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631256334;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=iYQthzBh+79AvcHHM7e5E3Mis4sMFNPv/bvg2n+qhfQ=;
	b=Cp9F6Pib7gmdb2QfxsvkFp6YgTiSKHj/LT8SIrXzDThbsIQU80YeyQn0Bss4nUG3XqVrN/
	8Hlu41bmk+oQdbRDqcCvzs3s4XhGOjLQ90ja6T77bOGZoIt2NihC9JktWoxQmYAavlnmck
	Uh12fkKvGn9ApHZMxY0uoilKuUCSrwA=
X-MC-Unique: gXv_cU46O_mSBStKmmIJbg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MB6PpJV2wbLG1fqHMj/A5akLZwfn+uIgKxTKmGb5dDHUWeCH01+kAGbN0UslDyr/1lKag/TNrTq4nx9ZYPqX03Itp2fMshqysU10rwpKDbBnJm7DhxUNpZ3gaEdm9sF88TuUx01kjVs56fWc43MGSYgy8uItjXYuJL4kbtCeeWKXapTpdlCWY1qtEXzVLcAzSM9jf/soZTSP4CnXD5B1hE2GLYl7rC+R8ernn9S60VIYeRIstilrSQxaeiWT2IoM55qZby3fboFTE89LZ8DiZGuLu0zO/5fi54KWHRH9yS+1pN/YF/SyyT2B1M0ARwta3y26+6Uv8rv178+5l4sGOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZFtGgDZNEYcRTHzgveHza0cShINv9CjEiQ+hgj/KJdA=;
 b=MuAiYOK5a/bYFvKgNq4Hxb8W2Qgmgt2JlxSPy4dzTCg3DeZZfsEuX7lY7xKf2oqh8ibtwTarCYXNG5I9Stwz/mX5C4Tpyez/+c1mPPfSIFfWqdaEGVdJ6BrUocLL3IIwiejT0eTQ1n4hZAVmXpqY0+mYNC2/32bAVHzhJvyB+GePIfrOyitLEBfiVlU9JaUkpWPkNFJF8XUMU2fR8X2xPP79mH/3q7cJZx/bgoQckHDE8KIdh3o44ROetLjUKoZtyWbpz+K2rEgTejDJt+MtaGM8RC/eNsWCkT/m3My4m4zPS39u5kX17cqcksgRofOSrxFMymcl6+tsjB6ybeJ1Gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86: correct asm() constraints when dealing with
 immediate selector values
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <87278164-2395-1fb4-7569-9151d0151e8a@suse.com>
 <0800289d-7744-2c77-b369-707f4947595b@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3b93ae2d-cce0-d36c-e3ac-d5c619ba6218@suse.com>
Date: Fri, 10 Sep 2021 08:45:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0800289d-7744-2c77-b369-707f4947595b@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR03CA0037.eurprd03.prod.outlook.com (2603:10a6:208::14)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c39f02fd-b04f-4e18-82f0-08d974269552
X-MS-TrafficTypeDiagnostic: VI1PR04MB5903:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB590320439634CBBAF52CAD7FB3D69@VI1PR04MB5903.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	z48FlflMn/rGpND3M5Jpww21p9lzQbBWk5hCY+LdnBNJNVUYGzZqrTYb5rFV4K4FkNGqWUJnDcnUfJ2Zlhms9S/WJT2T3OOnNFu+QkQ+aXVfEJf9cZ495pUBmK/WghwXKIL1AHMQGfz06JTZ3kMbjjsvRmN5u/ayEDv+b/SJa4OyxMehtmRzin+fhWbHIBFqDXGY0PPXYyI9aULgmkRiEfUgT3i/SKtbgzHSsCNW4Eol9e5RCwsOMynVsaHY4WmHDE07lZaP2jXa1zrRcRwhNS/bYDTdSBShcIzbIWZA+LBch/AjGE+7ZF7PyDsrtfinHXYzQ2Fo8a6GFip2HOsgB/Qr6UJMNhTLBVBQOFymxZCCsg6/gAN1MJ9FOlgiZqW7/pspsVFKaQ3YM+g+19bZwydpGFzPIjKB3UKxQphdSeKy/9g0L48cXi/Ux7mS7FxKMwabz0X3GOmJkhvI8+WVs6KtG906fmIu45+gnB/qsNT8gXzHeFGBxioogQc3qBDGM+uwx+cpm5Ktj6QqRsBbO839CuNMpWjhJhHE4GFXo9PwvXlfrOHFaQ3eZB8Yo8DUDY4ZQdSekt/eVCpu9HNPjkmJgV8OdcXz0bSS6QRBWmwesIsnqA0ZHNoGhK+CLx1Ds7lLgp1M4Qghp+1NALOAW0ZooFxM8jQcCicBN9FLCtTklbPV5ulrQ5Csfg8I1QfBQ0S3ZKNv1kchc+MhymGuBguCPt1B4gFUxC4J7OBUBrOdErCZoyr2KAf2cm8EoH03rzsHVs5SPPsjxGQiRtCxDGmsHTHNoKphXeBnMOl2eYkU5yQl+1gBjB4uVR+5sOrWfdXK+T9AM0OpBACbpPIxMdwh1oXUBn4x4X7kDavgAiI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(396003)(136003)(186003)(26005)(53546011)(5660300002)(38100700002)(4326008)(31696002)(86362001)(956004)(2616005)(83380400001)(478600001)(54906003)(8936002)(31686004)(66556008)(36756003)(6486002)(6916009)(8676002)(66946007)(16576012)(2906002)(966005)(66476007)(316002)(14773001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?E4BmW1cB3sWR/XFN9mzuy6kNUN+s+ndHkoM5H1LL9/sWfZ4qkpm1ftbv5dO1?=
 =?us-ascii?Q?1uRrTXU8hC/gVs9GnpD0OUak87ZYMKzqbhKuLx8GUkQ5Z8wDffks8+aKizGr?=
 =?us-ascii?Q?6LMpBkDtX9I3sTIh1X7Le6RrGMQ/Y20JD+5q1eL/nVcywUkp2aP/h5eFssaB?=
 =?us-ascii?Q?RrHU/vjh3j5zWLa4arP6nIJTO8rjIV3E8B5DvZdKUEUTMMx5wFA4Nr6mO6Jt?=
 =?us-ascii?Q?6lzmv+2JraK1Q/THVM4Ffotvnx6iL4NhBk5lPjf1aBOrpAxxBME5QlfCsGHf?=
 =?us-ascii?Q?TuRxIguN9Lu35CM3JGlqKtkGCFP0KqUwYo8TqMn3jXLLvp8J9aSgcYGHrCaF?=
 =?us-ascii?Q?4MQKjbsAdPJn25ZW7Rty0xjBvU3l2HH649ZcPzJr3pxr/1TbRgMiIR2SdS2s?=
 =?us-ascii?Q?XU49t/WfWLTu0efQvV9181i3p0JRWbfB1a8Sf+FtAEVcrUYEzmSDXrQm+T9a?=
 =?us-ascii?Q?3DHP7AKFa39Uv2b7BY1fWa/kLg2QKgYxs0YcljsyN0LswLtM2nT9JrFYLhDH?=
 =?us-ascii?Q?3sS9w/CXBg6nec1jbIxzkukZAm/sVcH9qv5xQmzdYWshCiR6xF0/SPTJe1CC?=
 =?us-ascii?Q?CCeH9Yd1GvQxstibd4/7gffAorevFMLWR4SYfUcU/yATA7lFV+ZN6eun8cqG?=
 =?us-ascii?Q?qbPBxkLmzQKG/bhRcg2FI5JEL5F+bI2w5Z8w0s1y/SCrTz6keVXv1H+sE0Nb?=
 =?us-ascii?Q?b6ctH5oe7rd/BiVmLTJ9yO9uXzZ465EUMGtAoVAxP8KHMs4avuPRQ4Kah3Xt?=
 =?us-ascii?Q?jkXgMqePBJIsqBud20QK0btVa1uqIldM9Y2ys4q7wDFO5jNIWh3mCO4JuK+9?=
 =?us-ascii?Q?hJ+YNsw53NgphDyCRD3IU0G+aBaJjfpBsTQzI42DLazAKwMh3xy0qI/jA4DX?=
 =?us-ascii?Q?Z2q1D3f8+M3n+y6V7vuQ/3fac9J0TTiUN08r+BgzV2oCD5y2V3Y+qLn/D0cp?=
 =?us-ascii?Q?ThwlgZEAzWniWM50ON7HQLJ61rExDDFP1Lhghv3WY/D6TSPLPPm27Cig+bJA?=
 =?us-ascii?Q?hf48X2oSXb1eQr8PDgIm4wIGMFmOF/IzPAoPBKhs8rMG27khcEEu4YsmJnQD?=
 =?us-ascii?Q?IDXVHHt/j81TwTi8WApflqReEJYddUdfcHBq2WhrCbgTqfPxKhTwrBnKdH6s?=
 =?us-ascii?Q?oxrsXQgxJbwewLZ9lmS4BHtnthw6Sh+i3Yw/nGdAsorTo5ndhHsCmwhWbUkU?=
 =?us-ascii?Q?UlXMYzp3cbKBwEbeyDqbe03+uNfuHCi1KPCb6t1xFp9KKyEV60i5o1rmIe7y?=
 =?us-ascii?Q?rFkk4JQG1ZjTDCGcb3wVDm2e25ZSnfWtpi8CnAJp/SCrBeqZ78O6y1JhUkrP?=
 =?us-ascii?Q?IIhphAC7r/utLSBeb79jfCWz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c39f02fd-b04f-4e18-82f0-08d974269552
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 06:45:31.3397
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +9/LaxFQFNuHpv2GT+km3LjW4tAiQ84yfAMLQrxroV3KUFYBXWt2njhAMSL96O/GR568ExwEYHMfZ7UzMzPLZw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5903

On 09.09.2021 21:31, Andrew Cooper wrote:
> On 09/09/2021 15:56, Jan Beulich wrote:
>> asm() constraints need to fit both the intended insn(s) which the
>> respective operands are going to be used with as well as the actual kind
>> of value specified. "m" (alone) together with a constant, however, leads
>> to gcc saying
>>
>> error: memory input <N> is not directly addressable
>>
>> while clang complains
>>
>> error: invalid lvalue in asm input for constraint 'm'
>>
>> And rightly so - in order to access a memory operand, an address needs
>> to be specified to the insn. In some cases it might be possible for a
>> compiler to synthesize a memory operand holding the requested constant,
>> but I think any solution there would have sharp edges.
>=20
> It's precisely what happens in the other uses of constants which you
> haven't adjusted below.=C2=A0 Constants are fine if being propagated into=
 a
> static inline which has properly typed parameters.
>=20
> Or are you saying automatic spilling when a width isn't necessarily known=
?

The lack of width information is a secondary aspect, yes. But the primary
aspects with inline functions (as opposed to open-coded uses) is that the
inline function's parameter can be taken the address of, and hence is
both addressable (gcc) and an lvalue (clang).

>> If "m" alone doesn't work with constants, it is at best pointless (and
>> perhaps misleading or even risky - the compiler might decide to actually
>> pick "m" and not try very hard to find a suitable register) to specify
>> it alongside "r".
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> I'm slightly surprised you didn't spot and comment about what Clang does
> with this.
>=20
> https://godbolt.org/z/M4nrerrWM
>=20
> For "rm" (0), clang really does spill the constant into a global and
> generate a rip-relative mov to fs, which is especially funny considering
> the rejection of "m" as a constraint.

I had no reason to suspect such imo entirely inconsistent behavior, so
I didn't look at the generated code.

> Clang even spills "rm" (local) into a global, while "m" (local) does
> come from the stack.

"rm" (local)? That would be racy (and hence imo a compiler bug). DYM
"rm" (constant)? Or DYM spilling onto the stack (which is what I
observe with clang5, oddly enough independent of optimization level)?

> I think there is a reasonable argument to say "m" (const) doesn't have
> enough type (width) information for a compiler to do anything sensible
> with, and therefore it is fair to be dropped.
>=20
> But for "rm" (var), where there is enough width information, I don't
> think it is reasonable to drop the "m" and restrict the flexibility.

Correct, and I don't do this. What I alter are instances of "rm" (const).

> Furthermore, I'm going to argue that we shouldn't work around this
> behaviour by removing "m" elsewhere.=C2=A0 This code generation
> bug/misfeature affects every use of "rm", even when the referenced
> operand is on the stack and can be used without unspilling first.

As long as the generated code is correct, I agree. See above for a case
where it might actually not be.

>> --- a/xen/arch/x86/x86_64/traps.c
>> +++ b/xen/arch/x86/x86_64/traps.c
>> @@ -248,7 +248,7 @@ void do_double_fault(struct cpu_user_reg
>> =20
>>      console_force_unlock();
>> =20
>> -    asm ( "lsll %1, %0" : "=3Dr" (cpu) : "rm" (PER_CPU_SELECTOR) );
>> +    asm ( "lsll %1, %0" : "=3Dr" (cpu) : "r" (PER_CPU_SELECTOR) );
>=20
> Any chance we can clean this up to: lsl %[lim], %[sel] seeing as the
> line is being edited?

Sure, no problem at all.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 07:00:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 07:00:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184096.332641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaVr-00075e-9E; Fri, 10 Sep 2021 06:59:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184096.332641; Fri, 10 Sep 2021 06:59:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOaVr-00075X-6H; Fri, 10 Sep 2021 06:59:51 +0000
Received: by outflank-mailman (input) for mailman id 184096;
 Fri, 10 Sep 2021 06:59:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOaVq-00075R-HJ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 06:59:50 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea9d2f41-6f98-4981-9f2a-7945fb07a15a;
 Fri, 10 Sep 2021 06:59:49 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2052.outbound.protection.outlook.com [104.47.1.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-scd2QKGiOOuiYiN-6B-nsg-1; Fri, 10 Sep 2021 08:59:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 06:59:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 06:59:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0105.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 06:59:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea9d2f41-6f98-4981-9f2a-7945fb07a15a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631257188;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FCf55x9SnfeNjoh0lPU2nefNa0hUu7q46T6aVLEjcOM=;
	b=X/8ti74CPB8BiLheasanw7RIMK59KD3DbbPAn73KAJKqLNdB0dEtlvoOFWncqTGESxIVoS
	0OQq+ZRHeNMMM4UBNtDRprOAX0PcPh+5r3SOsvpJxv8oSxn//d0N5/kPeXUCShNOWIxudl
	pqJC8gyNKv0H2t4vaiuPkgZ8vJmxeXk=
X-MC-Unique: scd2QKGiOOuiYiN-6B-nsg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ho4Z02Gs+I05vsATkC0xAtOD9bLqS2UOH31G5zNi4vh/XVO6mlRUxjLDNSR+mIeTSKHmBIItux8oHFMciKnPZYeiaHTk/FmJQxZn14Wz/ZpCUz+2InvJcKDv2uLA/giZ6J6dBinMVR4Pk4P9qrFVdQSnSa0vh6NvCD5itgupb7pBCr8bL0U1v+YOKKykDJoH4H7a4gTC1/Ln3DuAk0wKpeAfKAwVpkrYrzF7kZjY+XN/+HqR5q53yV+vZbO5RnXF59fMx5x11rFmG8MVfGF04t344HtSlrgh+uhRJXIbxzjt7Y2tEtJrUW/ogaeH1W5ZW9ezxOkoBYWAW9lHHB8DNQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=y94f08PmpbGymxwcg2LlCnG9x+YXIiqERiRQjkQp8sQ=;
 b=T1KZ7AzUeK7gN0rhkqg5tlZGMqpja1hqj6MFNwqXWdOohZZjDCQ84GVtGTIU9g3hy3p3Kw0c9q2zVSNOI4W8F5PKgyP4wXgYq9ePWsDZjsCfG0ATwjl8FVaztaMbVlHA6Y01JMleXCitpFkSje/YGX2cW39vSglZfGrxxXIsaWPM3B8tF+5ujz5McDZN0hfPvPlLZ6cqRUnqLOD3Q2etvsnBE2rt3ufeJpATRrlaYHnVY15a15TMkZ5GxGqNVtcLQRIvUxKm6b5SMRH1TwQnAK3s+EHaqAj/ImtVW5cNA2rVLqB54YPxoRQhOlHT1+7NatRfRkxowzmx86rdCDKa5Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86: correct asm() constraints when dealing with
 immediate selector values
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <87278164-2395-1fb4-7569-9151d0151e8a@suse.com>
 <0800289d-7744-2c77-b369-707f4947595b@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <adca3cf8-d9fd-f6b5-9754-754df4c06787@suse.com>
Date: Fri, 10 Sep 2021 08:59:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0800289d-7744-2c77-b369-707f4947595b@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0105.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae56a255-a609-4314-ab47-08d974289243
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260764436A80707E5260A065B3D69@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gWibJ++nPUS+Vo2FsxJZfaSrdAByAL5dmyT5yKggzraBrExykc6PYw+uJgf15wJp46nzN06CQk9pbrLbhRGC9hA2gXdmVaK3TCcgL1ovR4Fsptqr3MTNySBca0e8TWD7dqyM5aPnRwJxOy4gA/mo4nsTjNmssCnGO09iEbMR7HaZbn8VwXwKF6sXnXQCQibi0L/y9Jozxl8GvRorzkmhooxV6oipd/oztywA0rxBnkQeNplu9OfNj2M+2wVY0OqMPbqMb8sGpV7v709YlhNOHFHekI5SJWDsoxrUr9x+tLrBAWFJXMBmweOV/Mfhov3LZ0J72WIXrPrVwOI7QWbpGbwPCC8ZTrbcTZZNPWp21LV5V+rMMhnQNFSPnqgClD+48bID8/8lc0P5eO9L+zQOUAUgC11IGy3ECqWZxVv/mNbOxY+9GLgkE8eQ1WTOxiBANNhuGJsUU+C9fOMZoHzCNIVsWmE7cIvpEGhm03qfF2hnAIj4fcfgYbeKYTKv+o2IBAURykIU9UdBn5Oc2YZF8cYFDv9kDLOd8Ex3X8u7gnuP0Vgnll3KL9qJLhWFQRbMh+9/hy+GX85vHfjqrB+9LOXL8NBKS7Q0xKNgM3QWwOrPlXt8fwqaElb47vA25ytQ8teHzr1SaxdyK08l1E9wgJfRyMLWQLGzF45C/asQfvL3WfOnS6iopOQhJtaBci06/AWE3Wax2gy1i6hZA/eZHeyl7nIM7eGWdHmAS0Bot5Tit18KupVy1kyoTBmDwOrE5Gjfm+XyfwbTzfURrcUSUNZlVRRzrD2dkHG54ZHhufJ+w0tL7EHv2cgUKzrNCI8RUkzA4xFGd0lb/E+7ku7GWzP/Cg850pRjH4WMaTp5IS4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39850400004)(366004)(136003)(376002)(346002)(396003)(66946007)(66476007)(8936002)(8676002)(26005)(5660300002)(31696002)(38100700002)(86362001)(6486002)(6916009)(2616005)(956004)(66556008)(478600001)(4326008)(316002)(2906002)(966005)(186003)(83380400001)(36756003)(31686004)(16576012)(54906003)(53546011)(14773001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?piCv0TaJxxYWwJF2Z9tf2Q/kjRLQ7qF2jFosMsmRavHUgJXKyPT4YEcAo+3c?=
 =?us-ascii?Q?aEeAjf3NqcZ4XruIt2r79oNz1qE8wCOeVzSDcCa56AZuEGlx8cDeqVK/ZGrU?=
 =?us-ascii?Q?h+e5RFDKtFTNBmcpOsBJHW0WDTKik0E1TEO5XMvNxE7TqmvoxpETbIVdjoOX?=
 =?us-ascii?Q?NApUGr6hDQwv6/z6acC/lmmiJjO8KTM7xFEgwnDDyjBo4JDSFXDoSJLEDcCy?=
 =?us-ascii?Q?AFlMkrF9muM5XrPC8gUON+o3hly2n24ESqnilyaAXdqwW7ZB2TNMUNoMoyRm?=
 =?us-ascii?Q?KpJ1f6nK0AfbLTO43ARQeIs8d6gMsaMNWKmAfPklm6ES3tCbGZfVXFwVv3ra?=
 =?us-ascii?Q?COQyV2fU0El/VE9nSPdYZOduhN8BrPM4D8HHANRyFims7P7FH9xv8Px4oK6V?=
 =?us-ascii?Q?90SBj+lfxji77cvVk38Y/BjEUOBYmgb5d9x3df8QXMIGSNGxHsoU3utU+jjj?=
 =?us-ascii?Q?eFX4+k5/fijkzaV9bWFdnpDciFTVfFN6alrMhQw6CQcDvl1zihdzwYZImSXs?=
 =?us-ascii?Q?Mv+H4BqWeYCuGcpAxGobJeM+D5zLidImu5s488c29wC98YwxoFKiHe2qDeuL?=
 =?us-ascii?Q?5ImdD8bSLTzrZ+SsquPMUL1yJY2iVvq5RfqsU30ZBbHLshRP0Ogawqy9qpuV?=
 =?us-ascii?Q?GmeTTCksuj12Y9Ve9n/7NSlaNYmtITqLi2p+V36MLhfSD9jx+eNmycELOxOk?=
 =?us-ascii?Q?Zi9856r2UxqWV47+Ai3QrpGgwYJ019UahCxt8YAEGYR555bGXH2JJB/F3BqH?=
 =?us-ascii?Q?mAb0sXmjFcrccQV6QNzDGQkOrFdZMWf4gvwff9kJCFlmr5mG4S2oEJWPo5Nv?=
 =?us-ascii?Q?LU78gRyrBicaTjhHM+wTKpv/ZYoG+GV792l/T0FrZa2AcmQW8Jz2Di2r5D+6?=
 =?us-ascii?Q?u5qOizXUeiY9BJ1vJToPKUEQ4aLSCwgD4vaYh+DQir6pyvhBuPKuuge9DFWR?=
 =?us-ascii?Q?bsQqTJnBFJ95Ms+AmEb9G4Usa/okRDGLFczZKkDwqG4uGPfCtZN4eyCrhGAm?=
 =?us-ascii?Q?c4ktQmtJNQvU+KSotPgnj72PHRBEZGRXTUf4V0hGshcTFWBXJFwbO+xFR6n6?=
 =?us-ascii?Q?dLHQC3WUvMWv9uW0qwuevrz/RsXsch3e4ofwQHDbumf3zXf7cBMFX58dlez3?=
 =?us-ascii?Q?Vk3DIXxoASzOLAIZSVqAEA4PokTgVpqJi7KrXTW2YSDKlido7NIG9X5uAR/y?=
 =?us-ascii?Q?+ib/tkvnAYbd03u21qKSj1MBU+becJg/O9HF16OmQOKU8KV2FtJYCfiZRv8G?=
 =?us-ascii?Q?lb/255fluAjwgWKHDHaiifHrml6TvVZG6ndkWLijTx6I3XKR2YcP8vKmrhb5?=
 =?us-ascii?Q?Ni+WGwvdeb/AqH5Jo0N6Josa?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae56a255-a609-4314-ab47-08d974289243
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 06:59:45.1727
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: g62zdnffQxUE9TldUu0xFDd3l2JJig66f0PjjB4iusi8FEx0LL0VykkTO6XnjB8gizghA6V02915zJ9P8CDpxw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

On 09.09.2021 21:31, Andrew Cooper wrote:
> On 09/09/2021 15:56, Jan Beulich wrote:
>> asm() constraints need to fit both the intended insn(s) which the
>> respective operands are going to be used with as well as the actual kind
>> of value specified. "m" (alone) together with a constant, however, leads
>> to gcc saying
>>
>> error: memory input <N> is not directly addressable
>>
>> while clang complains
>>
>> error: invalid lvalue in asm input for constraint 'm'
>>
>> And rightly so - in order to access a memory operand, an address needs
>> to be specified to the insn. In some cases it might be possible for a
>> compiler to synthesize a memory operand holding the requested constant,
>> but I think any solution there would have sharp edges.
>=20
> It's precisely what happens in the other uses of constants which you
> haven't adjusted below.=C2=A0 Constants are fine if being propagated into=
 a
> static inline which has properly typed parameters.
>=20
> Or are you saying automatic spilling when a width isn't necessarily known=
?
>=20
>> If "m" alone doesn't work with constants, it is at best pointless (and
>> perhaps misleading or even risky - the compiler might decide to actually
>> pick "m" and not try very hard to find a suitable register) to specify
>> it alongside "r".
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> I'm slightly surprised you didn't spot and comment about what Clang does
> with this.
>=20
> https://godbolt.org/z/M4nrerrWM
>=20
> For "rm" (0), clang really does spill the constant into a global and
> generate a rip-relative mov to fs, which is especially funny considering
> the rejection of "m" as a constraint.
>=20
> Clang even spills "rm" (local) into a global, while "m" (local) does
> come from the stack.

Which leads to a lack of diagnosing bad code, in e.g.:

void movq(unsigned val) {
//	asm volatile ( "movq %0, %%xmm0" :: "r" (0) );
//	asm volatile ( "movq %0, %%xmm0" :: "m" (0) );
	asm volatile ( "movq %0, %%xmm0" :: "rm" (0) );
}

The "r" variant gets diagnosed (at the assembly stage, as a 32-bit GPR
is invalid as an operand to MOVQ). The "rm" variant gets compiled to a
64-bit memory access of a 32-bit memory location.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 07:28:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 07:28:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184107.332661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOax9-0002uu-LH; Fri, 10 Sep 2021 07:28:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184107.332661; Fri, 10 Sep 2021 07:28:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOax9-0002un-GT; Fri, 10 Sep 2021 07:28:03 +0000
Received: by outflank-mailman (input) for mailman id 184107;
 Fri, 10 Sep 2021 07:28:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOax8-0002uh-AT
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 07:28:02 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 096f62d7-7f65-4deb-9e7a-7f7e5c45b792;
 Fri, 10 Sep 2021 07:28:01 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 59FF12229F;
 Fri, 10 Sep 2021 07:28:00 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id F318213D1D;
 Fri, 10 Sep 2021 07:27:59 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id PCE8Of8IO2GBMwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 07:27:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 096f62d7-7f65-4deb-9e7a-7f7e5c45b792
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631258880; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=7tCaeYZC58tBDrJqNk0nn5PtnnlNlX6dAF2RgPaH35k=;
	b=OCx6QAPj+yLpBTeSc+Q6mflKHsCfICjBqBV8asbgkLw/zOU7OQRMnN2v0wT82dQJfwfZ+T
	CT5QKygIuWcBllQbtfsWmQ/uvXyOS5fyLkoiF1NhwO0Y6WTR+qTJ+yct3Yc13vnfWtaKiF
	gqTLgO6zZD3tz5vFt9waWB2xZLILkg0=
Subject: Re: [PATCH 2/2] xen: reset legacy rtc flag for PV domU
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-3-jgross@suse.com>
 <b2cd2936-7ec4-e0b2-458e-51c12a3f56aa@oracle.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <8385702d-aeb2-5898-a4ea-00991dd7bca9@suse.com>
Date: Fri, 10 Sep 2021 09:27:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <b2cd2936-7ec4-e0b2-458e-51c12a3f56aa@oracle.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="e1YgqMDepsa4po08haHcuF4n97TgpYtNd"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--e1YgqMDepsa4po08haHcuF4n97TgpYtNd
Content-Type: multipart/mixed; boundary="YOevWxiq2bQARiqYFjXPhUWKFLWRRLAL4";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org
Message-ID: <8385702d-aeb2-5898-a4ea-00991dd7bca9@suse.com>
Subject: Re: [PATCH 2/2] xen: reset legacy rtc flag for PV domU
References: <20210903084937.19392-1-jgross@suse.com>
 <20210903084937.19392-3-jgross@suse.com>
 <b2cd2936-7ec4-e0b2-458e-51c12a3f56aa@oracle.com>
In-Reply-To: <b2cd2936-7ec4-e0b2-458e-51c12a3f56aa@oracle.com>

--YOevWxiq2bQARiqYFjXPhUWKFLWRRLAL4
Content-Type: multipart/mixed;
 boundary="------------87B7477B304C177D84C82456"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------87B7477B304C177D84C82456
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 08.09.21 16:54, Boris Ostrovsky wrote:
>=20
> On 9/3/21 4:49 AM, Juergen Gross wrote:
>> A Xen PV guest doesn't have a legacy RTC device, so reset the legacy
>> RTC flag. Otherwise the following WARN splat will occur at boot:
>>
>> [    1.333404] WARNING: CPU: 1 PID: 1 at /home/gross/linux/head/driver=
s/rtc/rtc-mc146818-lib.c:25 mc146818_get_time+0x1be/0x210
>> [    1.333404] Modules linked in:
>> [    1.333404] CPU: 1 PID: 1 Comm: swapper/0 Tainted: G        W      =
   5.14.0-rc7-default+ #282
>> [    1.333404] RIP: e030:mc146818_get_time+0x1be/0x210
>> [    1.333404] Code: c0 64 01 c5 83 fd 45 89 6b 14 7f 06 83 c5 64 89 6=
b 14 41 83 ec 01 b8 02 00 00 00 44 89 63 10 5b 5d 41 5c 41 5d 41 5e 41 5f=
 c3 <0f> 0b 48 c7 c7 30 0e ef 82 4c 89 e6 e8 71 2a 24 00 48 c7 c0 ff ff
>> [    1.333404] RSP: e02b:ffffc90040093df8 EFLAGS: 00010002
>> [    1.333404] RAX: 00000000000000ff RBX: ffffc90040093e34 RCX: 000000=
0000000000
>> [    1.333404] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 000000=
000000000d
>> [    1.333404] RBP: ffffffff82ef0e30 R08: ffff888005013e60 R09: 000000=
0000000000
>> [    1.333404] R10: ffffffff82373e9b R11: 0000000000033080 R12: 000000=
0000000200
>> [    1.333404] R13: 0000000000000000 R14: 0000000000000002 R15: ffffff=
ff82cdc6d4
>> [    1.333404] FS:  0000000000000000(0000) GS:ffff88807d440000(0000) k=
nlGS:0000000000000000
>> [    1.333404] CS:  10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [    1.333404] CR2: 0000000000000000 CR3: 000000000260a000 CR4: 000000=
0000050660
>> [    1.333404] Call Trace:
>> [    1.333404]  ? wakeup_sources_sysfs_init+0x30/0x30
>> [    1.333404]  ? rdinit_setup+0x2b/0x2b
>> [    1.333404]  early_resume_init+0x23/0xa4
>> [    1.333404]  ? cn_proc_init+0x36/0x36
>> [    1.333404]  do_one_initcall+0x3e/0x200
>> [    1.333404]  kernel_init_freeable+0x232/0x28e
>> [    1.333404]  ? rest_init+0xd0/0xd0
>> [    1.333404]  kernel_init+0x16/0x120
>> [    1.333404]  ret_from_fork+0x1f/0x30
>>
>> Cc: <stable@vger.kernel.org>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>=20
>=20
> Fixes: 211e5db19d15 ("rtc: mc146818: Detect and handle broken RTCs") ?

I don't think so. Its rather:

Fixes: 8d152e7a5c7537 ("x86/rtc: Replace paravirt rtc check with=20
platform legacy quirk")

as this was the commit leading to domUs having the rtc capability being
set.

> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

Thanks,

Juergen


--------------87B7477B304C177D84C82456
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------87B7477B304C177D84C82456--

--YOevWxiq2bQARiqYFjXPhUWKFLWRRLAL4--

--e1YgqMDepsa4po08haHcuF4n97TgpYtNd
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE7CP8FAwAAAAAACgkQsN6d1ii/Ey8R
+gf/aas2jxgmhjzaKLPpJMPjSpGxMOUtXZOXgTTYIX6vH+eLWVZAJQ8wJXesZGeJSQtgwT6K4wOD
tUSP0nGwBCinQQSLGt0Wzk3myQk/EvbKQ9u7Ch+V7Vg7uZkbkqZN1yz9bzqN3J/fbB4D4qj+75dF
ugkVQ7srpbdCmlu1hxJLurZ/R1JDfYBZwP53XoqeP4f61zLD0euDm3P7K3TCRmFs6GzlzcruPR+d
K9zw7qV+tSQW95k4qHprEAry1a8PdGAd66cj39V4DbuRotIIuiQCkyNA3oNUgtRz35ckX5jYyGOl
pVxQOC0FgeLpPtt3uQO1r9iF25H7fL5CkYTqqzxHVg==
=oXCQ
-----END PGP SIGNATURE-----

--e1YgqMDepsa4po08haHcuF4n97TgpYtNd--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 07:40:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 07:40:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184113.332672 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOb9H-0005RF-Po; Fri, 10 Sep 2021 07:40:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184113.332672; Fri, 10 Sep 2021 07:40:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOb9H-0005R8-Mg; Fri, 10 Sep 2021 07:40:35 +0000
Received: by outflank-mailman (input) for mailman id 184113;
 Fri, 10 Sep 2021 07:40:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOb9F-0005R2-Tx
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 07:40:34 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21ee7c53-5c56-4cf9-863e-e542214ebefc;
 Fri, 10 Sep 2021 07:40:33 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18A7VOF2029743;
 Fri, 10 Sep 2021 07:40:30 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b02mt0471-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 07:40:29 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4146.eurprd03.prod.outlook.com (2603:10a6:208:ca::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Fri, 10 Sep
 2021 07:40:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 07:40:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21ee7c53-5c56-4cf9-863e-e542214ebefc
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g/lVCsqv6YgaZw0wXPEwzH0mLy+4B1WE85GXraAJhF076UKI/fnGiU3FAFHs2JMjJQpi6BgRULVxF4GYJaBV69ixYGphcUTb7kHGvd5xzN2tc5ZSWXioCNAwC9DJDCCE7RMpY4o8EFVd9U1xgMu07VK0NrJqwdoqIBesPK1jRUjO9tutacdj5JAaadT8xpPqZU8hmE+7gUtHhZBHHu3+HbBfk88gJtWn7VVFFAPr9+2NHsB3jX0KoFSGyfXNaqDrNpCsqyo1ox+GWp5rkcy3ryJxCkupK+mc28kXWFyFaVSQeHV2eAJizOuoDtsyfZEuFnYvHfQcTFR0cjfKHkqS0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ti+No49gbaz412CJo1DZu2r1siXmL2rdhSlZTtMKmyM=;
 b=fPDF2SqPqdvAPqGdSmEKtegC1f+z76rlXfkZW00cuR/XMSR8Og8rO+JWMMSnMlrMy2eB4tuGlg1UHr4HkCadF30o/jX7ElSJFcrQbeqcXkVDsb8Q/8aS0EOVim0A0nJxCWLHoWsuZ1e0LecbribiToWvesVvwbu6bpihAjlottpp+66ZGIk0LZKBPm8mqvSajddk/zqtCT4+fz5xHKzQshpfIzY9gn6Ux/5NQPyzMyGDaIp8S9sLAXjlVRoJQRuNkBC7AiHvdWM7F4hdLjOwJ4NDR8Jzpiut7rCOxWRiSFVKl7uZWDxhk7L3lwHzfPe62wT1MRHJ1hoiIlnOt0+HtQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ti+No49gbaz412CJo1DZu2r1siXmL2rdhSlZTtMKmyM=;
 b=O/Gv4oXHYWg4WAtPyWdBVPZ1AQWr2D3Go0DM762prqrVqMgBJN+qitSczMGH5XUp7Ajwdc7ARRsMF7bFyYcro0S6kTJGUnzAouiw6eN+USltA75UvG2BlkhfoxwTXIiNNZDvNtRXZNYvXc7iyYNkU6ig/VtTVWLlT3+L45oCpN/v9/gEZdJgdyPTtViVtqn8OqnkdvG/aA8E2tIPRHG8HjhabNeYs+532jcr9u+OqnTG3KjclSkYQB9Eo+69mkck6JJpCJACgYSeyNoln/L5Ezk/ruRa4UChujznPUUCmVj1D+NO+Jn+653r9IpPEeUgk8UIjWjhUtBFfTcY9IBixQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 01/11] xen/arm: Add new device type for PCI
Thread-Topic: [PATCH 01/11] xen/arm: Add new device type for PCI
Thread-Index: AQHXoJ5vVHmVQfsS80C08bAadrkmc6ub/FYAgADwqgA=
Date: Fri, 10 Sep 2021 07:40:26 +0000
Message-ID: <4c43fdd4-c10c-c1da-440b-34eef43d0716@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-2-andr2000@gmail.com>
 <5a32422f-0782-342d-911b-116af2c85859@xen.org>
In-Reply-To: <5a32422f-0782-342d-911b-116af2c85859@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 82b43258-3dec-4124-714e-08d9742e41c5
x-ms-traffictypediagnostic: AM0PR03MB4146:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB41467E4BB86435B979F27002E7D69@AM0PR03MB4146.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 NN3HaKlWuSmKgiC9KUdPMWNLygf6cLgQS5o0KdFI2UgZJCWkN2NvwX3RZ8VWQYXVYetrpMTwiVV5MjMUNgc1wvOa1MHyWzHrOullvTIX01x2rYhXp71wrAc1H+zoejr3yMidl7175JcipN42hn4cHxXdlvIlRv1Gpr2qk+kw1yr83NpqnRr8+sazFkVIIQ0wBfD7iVDNrjyQBpeGBmnH08lD3OIUFE9dIbOPQ2x1qXMfL1XhvO9EMo99xZ46cHmPkEtiE4PjxHBnERx0z2TXXc+J9R6eO2NWWL4fkJeholhneEUeStKGMyu3YIsZRLDBttvz4LyJrKkfv8neaqG1KvtMR8bRs35+4i3PvZ/FS+xk+kkv5BXt1IaH+O6AUZtDV6XNaOI5SOMW7j6sbIlyhEgTk2gGHka7CuTgQFrXqsx66Y1wx2Ao4n9RWCWc744FqmrVtuYIVXwq0CM9IIRlR0K3CA/m3pr8BO7A1XHy1P4Zb1IOLlxxkjMG/Jyyi6jJ/TpBemn3Z0ISeXtiw4Ct5eB+GSZuwpRVZAIP3Vu04zfsB4xRs3yTBK76kGh5gzcDs57au3WpCdzGo6GjEQrffti2zkxm9aiGY0RzRSHcmCLKu4Ozpgmhszkx/IcjsvClI9YDmFR2u6cVfYNkT5QpGz5Cm1/uzxTP8+0G3Q7chi196Z2D/p3bSYX9Sj60mBNdsR/ipnQc8x5sLb8WZ/tk3iFXfEDoO5wwEqjZQ5B5mpY=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(366004)(396003)(376002)(346002)(136003)(66946007)(38100700002)(76116006)(5660300002)(31686004)(4326008)(91956017)(86362001)(31696002)(53546011)(36756003)(6506007)(4744005)(66446008)(66556008)(55236004)(66476007)(8676002)(64756008)(110136005)(6512007)(2616005)(6486002)(316002)(54906003)(8936002)(71200400001)(122000001)(478600001)(2906002)(26005)(186003)(38070700005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Z2x0K2J1TXJydmk5WkVJNmlFOUtpVnkvSlpZRXYrM2hncVBiQ28waG5vcStp?=
 =?utf-8?B?RXVKanpBdGhyTDFYQzJkRnVPbllpNkQ3R2NoMHVxWnpvMytQZVZzTmJXdHJq?=
 =?utf-8?B?dzZMZTlIR0pvQ1lXNVN0YWliMGZjRkJlaWlBL3MxcklGZ1pPdGpaak10UCti?=
 =?utf-8?B?ZUxNdVRlb2F0MThqanpiSkpiNTdEUnRaWjR2UVBVRTQxcHF2cHdlb3FWN0tN?=
 =?utf-8?B?R0M4Zk1BR0o2NUtCNkZBc2hTUFB3cm8xNU9FWmlyRDV0TlpJbWpyNEs3TFdO?=
 =?utf-8?B?ekJyWUtzTnZvVzJEaHlKSk8xRmdGb3h4OCtSY3E0TFZuSFhuL1hhWVlhbE9r?=
 =?utf-8?B?ZTd5NVFlV0FRSXE4TXFYT29FRHhoRmI1djRTSE9TSU9UWDNEbEt0d3hrU2lK?=
 =?utf-8?B?VVE4VjVDSTZROGF0OW9SVG5vU2VMNVVONkhoWmhtOU1uNmJhL3o1bllJa1E0?=
 =?utf-8?B?VGgyRGZRTnM5d3FwQUQrOXZTTEtnd0QvL1dxUjFIYWpCd2pHQmNZQ3ZZYy9O?=
 =?utf-8?B?WmQyMjZHeC9YK2k3aGFua3YveDRsN2RUTSt1L0g5VGFWVFA1dkIyeVlpdW9C?=
 =?utf-8?B?SDdqRXd2S1d5VnJSU0JRQjJlYjZuNDc0MlZBWi80YmJCM3B1MFJVb01mM3hH?=
 =?utf-8?B?NGpRUWpnUTJxNXBPaWN2blhPM05nY0lyOG1wNXIrNndtOENNQ2RxR2MzTDd6?=
 =?utf-8?B?MkNnWTlPcFVKa3lWRHJ4OGZEM3dpSnNDcnZTcll1QUc3UlN4eVcyK29jVk9T?=
 =?utf-8?B?ZTVGYjBZM3JOQml2dklBQnZZZE54RzA0YnA2N3Q2QnJibExMeUpSMXNpaUlh?=
 =?utf-8?B?WmxHVTlSK3dFaS9EZVBUcXZuOWk0ZkwvZldXTzZlZFZYVkxkWUt5T3F4c3B0?=
 =?utf-8?B?M1FIcW9DM2FUYUZGMmM0YkY0bXZENEhOa2hzU2VtMmJrRGRLZDVVTVJNUGIy?=
 =?utf-8?B?c3I3TEVub0g1aU5yVXl6ZnYxM243Mm56TVg1cS9qbHpHUTVjcDltc1F2L3Ey?=
 =?utf-8?B?S3ZQUWFNSENYditTbm1ueVpGdW04WW9QRW5VTE1ZRGpURENoUlZNd29ySWVp?=
 =?utf-8?B?ckFYTFY3b3czSDBKclk4aVg4elg1amQya3Z6SGgyOGZhUGlJS3NJcStjQzZy?=
 =?utf-8?B?d2RRMi9EZXc4YVhGVG9sZXlmR0tiMVhiQ21QUjdubnM1R2VTbklJa2NjSjR6?=
 =?utf-8?B?NmcveExUUFltQVRmSUt1eWd2cUlhb0c4WnNwTC8zZmpQbC9RalQxRTNQUkpu?=
 =?utf-8?B?dmFaVUdYcDExRUpGbnpnM0NNcHB4N1BhbGpNT1VNS0NUK2s2UVFBc0pRYVc3?=
 =?utf-8?B?UUJ2bDA1RjVUU0Y1L1FJcVpIbE1rZWJDTWxXbWNaUy9wbTNvUTE0NW5DcTNl?=
 =?utf-8?B?ZStkdzZHcE1WSG8rOVFCbzYzN2VVWEpiaUdWR3ZGSmNHc3dBb21FaDJ6bk1r?=
 =?utf-8?B?ZmhRVld6VE9GNGFzS2NLT1AzeVNtZjRZcmRKdE9PaWlMYkpmRDZBQkJRM2V6?=
 =?utf-8?B?VTZNYXNzQUFwY0tJeWJibHljb0pVZ0J3VlBKUCsvbnJFUnZSNWJzWTE3WUJG?=
 =?utf-8?B?Vmx4R2Z2dFp1VzV3VDZ4UDlpZE9IaXV3T2YxdTY5bVpRd290NWozZjFMTnp5?=
 =?utf-8?B?c3dtR29IL1p6dE4zNDRyRzBtQm1ZbXdQdHJYSlV0ZTVJL2ltN0l3MlhCbHdX?=
 =?utf-8?B?cWlJSzJXM0VWVFhMaTJDc2lObm9NYTN6akk1ODVzcGdocGNZVTVOR2pBcFY3?=
 =?utf-8?B?UFZTQWJYQW4yNlZtSzZnUUc5K1laR05lbSs1TkN1V09uYWFxRVdwTzRZU3d1?=
 =?utf-8?B?b1JPTEpUQnQ2Qkx4R0ZWUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <3752BAE6A6743444B8BDCBBA1376BD1A@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 82b43258-3dec-4124-714e-08d9742e41c5
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 07:40:27.0033
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: u8naP46wYM1b+uZEUNs13EqmSAHP8ICOs1Xc8lNWG5GfwuAqLDF3B1tAnITZ8NddxDgufoeHJXQWAOvkNhSCrzT9of1TXcm4rdW/XT5BrXxH09U1H11wW5Mg96Sel5ST
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4146
X-Proofpoint-GUID: _i_Ym09Ibqe7YmBXMbArAcIVzi_mpocP
X-Proofpoint-ORIG-GUID: _i_Ym09Ibqe7YmBXMbArAcIVzi_mpocP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_02,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0
 impostorscore=0 priorityscore=1501 mlxscore=0 suspectscore=0
 lowpriorityscore=0 malwarescore=0 phishscore=0 clxscore=1015
 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109100046

SGksIEp1bGllbiENCg0KT24gMDkuMDkuMjEgMjA6MTksIEp1bGllbiBHcmFsbCB3cm90ZToNCj4g
SGkgT2xla3NhbmRyLA0KPg0KPiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiBBZGQgbmV3IGRldmljZSB0eXBlIChE
RVZfUENJKSB0byBkaXN0aW5ndWlzaCBQQ0kgZGV2aWNlcyBmcm9tIHBsYXRmb3JtDQo+PiBEVCBk
ZXZpY2VzLCBzbyBzb21lIGRyaXZlcnMsIGxpa2UgSU9NTVUsIGNhbiBoYW5kbGUgUENJIGRldmlj
ZXMNCj4+IGRpZmZlcmVudGx5Lg0KPg0KPiBJIHRoaW5rIGl0IHdvdWxkIGJlIGJldHRlciB0byBm
b2xkIHRoaXMgY2hhbmdlIGluIHRoZSBuZXh0IHBhdGNoIGFzIHRoaXMgaXMgd2hlcmUgeW91IGFk
ZCBhbGwgdGhlIGhlbHBlcnMgZm9yIGNvbnZlcnRpbmcgZGV2IHRvIFBDSS4NCk9rLCBJIHdpbGwN
Cj4NCj4+DQo+PiBXaGlsZSBhdCBpdCBmaXggZGV2X2lzX2R0IG1hY3JvLg0KPg0KPiBJIHdvdWxk
IGtlZXAgdGhpcyBjaGFuZ2Ugc2VwYXJhdGUuIEl0IGFsc28gbmVlZHMgYW4gZXhwbGFuYXRpb24g
b2Ygd2hhdCdzIHRoZSBwcm9ibGVtIGFuZCBtZW50aW9uIGl0IGlzIGEgbGF0ZW50IGJ1ZyBiZWNh
dXNlIG5vLW9uZSB1c2UgaXQgKHNvIHdlIGtub3cgdGhpcyBkb2Vzbid0IHJlcXVpcmUgYSBiYWNr
cG9ydCkuDQpTdXJlLCB3aWxsIGNyZWF0ZSBhIGRlZGljYXRlZCBwYXRjaCBmb3IgdGhhdA0KPg0K
PiBDaGVlcnMsDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 07:52:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 07:52:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184123.332683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObL3-0007be-4e; Fri, 10 Sep 2021 07:52:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184123.332683; Fri, 10 Sep 2021 07:52:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObL3-0007bX-01; Fri, 10 Sep 2021 07:52:45 +0000
Received: by outflank-mailman (input) for mailman id 184123;
 Fri, 10 Sep 2021 07:52:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mObL1-0007bR-Jq
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 07:52:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 75db1c2a-ea9e-4f8e-a82d-73e4786aa680;
 Fri, 10 Sep 2021 07:52:42 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2058.outbound.protection.outlook.com [104.47.5.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-0fZQqXS2O0OERv7fL145BQ-1; Fri, 10 Sep 2021 09:52:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 07:52:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 07:52:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0018.eurprd01.prod.exchangelabs.com (2603:10a6:102::31) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 07:52:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75db1c2a-ea9e-4f8e-a82d-73e4786aa680
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631260361;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6vICiYVtqGfp9eW6V/8l6IFgWwdeWllZwzXOLg3Q9tQ=;
	b=YpsUkS6Zpk1X26k4+30nz/44fpQHg9QhlXCkMMGm4tI3gDV6dKiTIrVlFJT4My1m8TF3AS
	0eVLdpbY70hK9pFOF+FR9tJDvs7Pu1bkVZLjnO6SU8qtAIKkqDVeLq5cS9nx7rnSOfNHXY
	PjkK5/5isuugQfKyPsyJ8zMRYfjMSJY=
X-MC-Unique: 0fZQqXS2O0OERv7fL145BQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PrA6SFZlse8tmXgSRdT4hivXmMTtr/PnM+5Y3jQ32mVLiUzh2DrNoejd/NxBmieFNDExcEVP7BORVUboIg1zxskJo0melJcVg2UgGqt23beHNx72wNStTRVAVQgfu+O5r7Kt16fsZyav+UdrZYYM1nkgR6UDftw+0qffuW1kJhZk86LddZoMfWkvbYDxrzIeIi8YLsbgnFmfvVpIBpYFSPytAIxWgvdqyJEAlGQu+ugElSG1ZJSrRaMpfj8ofJWs5QfaBKj2HmGeNVLBYq6Z+jJe0BwJ189zIzzfhj3xd5WForHBP6D8DRH7GFaUopG3JQj3hlmlqQrfdLIai7F6Fw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6vICiYVtqGfp9eW6V/8l6IFgWwdeWllZwzXOLg3Q9tQ=;
 b=i3tuCm5uQMYMcayBRQlCmzdMEo9ON0vmA9A9NuDZEUWw6qd25+o6wMXEOQGQ+GsWNTRxRNh4kuAbtuhtfnOtw+RcVF66IfpRjMroFmPh/LuQRaLnyXDcvFVgYQ1LsIeJf4ssZs2wFQ+ztKqzcbrZY7ZT1U5FRL5yRqgeRVLFQPnoiXoqrv4TcjiwM+mV6t5NyP9U77Dt2sMnpLka9biUJLWvC0uw2/+AcvInoWcArDqSgX7zhvobAJAsQFB8tF/uCfwEA6W55/LppYDg5DiFeqzPSntt5rvOZhtPkMXd0v9xT1UXwUtKjaGHhiGG21oNttCSOKm8sdmUYAERdklzxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
Date: Fri, 10 Sep 2021 09:52:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0018.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9b324d15-d55d-4d4b-3841-08d9742ff51d
X-MS-TrafficTypeDiagnostic: VI1PR04MB5600:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB560096F20E1E58A9CA1A48B7B3D69@VI1PR04MB5600.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ytbq+jatCFD51Pu9vfRwqfwWVThJUsUMuFQf6cTD235sDSs5OduPNa/i8+cSRfKR/dL4Wos6dC/te6WPH6pkVAiRVjmCK8wA+ZtDmOTy+1OzbssQLv0Zr2JWbm2XzR25BJmjNKMLvW5VkTuVyynqNKFlLRJtcsg/drLHJ1POQbIkfik09N9s4JFoPuA8bY6hbzQ1TUj3tkLaaJ8ITTgxQHr9SsEo8DamNlm5tOtvqhtAxtedvk5wmSJ/mdbO21a3z4unyfUpUt/CDA2Ooi8lYdrccgBd3PsFwG6hfez+tZHK181440FoTSR7GQy9saExCp21bShIxXHuZYqBtBZouE1pUjFOGrL3tBkXe7mYyVFMcpMc2zGeXg3IKDjd9vdXSqaWy+uC+EeQzvlwew4kYcBtg8zZzDAl5rznOrAVNE23eARsyuYF/SCMEmeUrz+NRcMmP81Si615MFjQ39NL2Tr9lZmF8AVXWDajAy+NGuUcpooas5GwLUN0Nmzc1bIKxRzuIZprRFlGOHben+RjHhOhOoousz9GQZ5ufasQ5jzLIrR0S1X3PcirzNkDoY5W0rvEa2fFR1INeMOwDHxjuPp0zWijBLVKDOC97SJBFO3VBrWqBQnRJCQw/WwU698sd5vMDx0pIC5UKb4znEsynwUVy0NFRMp1Hq567wLnnZ11vhA385ky645+yf1Hp9IMrKtCpicQE5iunkRmHmhF5Pt61GID8C2Vsumi4DTPGjk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(16576012)(31696002)(66476007)(956004)(4326008)(6486002)(38100700002)(316002)(8936002)(186003)(2906002)(54906003)(8676002)(26005)(6916009)(31686004)(53546011)(7416002)(5660300002)(508600001)(66946007)(83380400001)(86362001)(66556008)(2616005)(21314003)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VVRjTFhheU9DdmoxSzRaSi9LeWxBR296NmRkQ290eU5HUkVseEJzVGlNbFR4?=
 =?utf-8?B?Y1FiQzNOeGFFb3djSmpodWUxVzl2TCtUaVZyREh4Y3dTMTdQN0E0SUF5bytw?=
 =?utf-8?B?UXJWdnBJTDdQQ0Y4Yno5V1BNV0tTRXdFVUt2SnFWQy9WN0YzVHVOL20zTk01?=
 =?utf-8?B?dCtUaEZZNzVlY0RsNExMVlFMMWFHdW0yRjYzYVUzWGg1eVJNTzYya1ZpeVFp?=
 =?utf-8?B?WjVnenJ5SVhvU09JclV0RGpKa0VRKzBsRWVNRGFWUjJJT0cyL25yS1N5dnh4?=
 =?utf-8?B?bUVzS2haSHhSSUVwVzVDSDRwM04yVHljVndRTFJ3VlJXUEFMWExra2ZIVU4z?=
 =?utf-8?B?VkQ2QnZhbEgyc3RzdUxiWi95NEh6S1V2VnRXTnJVWU5BWnFOUzN5Qzc4UENC?=
 =?utf-8?B?d0lPZ2ZteTAxR281YXY1NTd0WVQ1T2c1blFDOTE2VlVMVEdleVpySEtNSm1u?=
 =?utf-8?B?dmxTUEdvWkRaVjJwZnR0VkpEQzVqWTlEdnNqdk9XMEhUMFpYUVI4L3oxM0RK?=
 =?utf-8?B?TVFKc2E1d2gwSUh6N0Y0azZBd0RXZUJQNDE4UlZSazJkMHRIaE9Ha2pyTWxV?=
 =?utf-8?B?VXBiVWdGaFVOYVRrS05NMGFpcDgyWkdsUzFCMnFLZmJhOUlvT2FIdWN6TjBu?=
 =?utf-8?B?aU5tZnhaVStLQkNxSTdUZExGUWNtUmNLRmVDTWFWYzJrdVM1djUzKzh3Ti9R?=
 =?utf-8?B?UlViTlhjRS96cGpvSWVSNjlvN3o4UXE1MGdCMVVoK3pvU2drdzFDaXhQOWRG?=
 =?utf-8?B?TklxLytSZXkwQStMeHFtMy92enZ2NkloaUo4SU9jM0JnOHR1eHdOV1M2cDF0?=
 =?utf-8?B?QWNCUnByT1ZWa2hLTDBiMk13UlhFZTZEQ0ozU3djZUs1OXlJdWo5SXdhNC90?=
 =?utf-8?B?RzBSUmtsVGI2ZGVMazBNcWxkNEZQUjd4Rmw1NVdNWDhDWmtsa3RXekhZckxI?=
 =?utf-8?B?Y1RFSHJST09STFVwd1k3TkZQb3g3SUFBV3pud0hwdWVtNE9Xcll6RHExYktN?=
 =?utf-8?B?VE9Pdlh4ZGJKVjR3WjV6aGVzS2RoVXYvN1d3ZXhmUTBRNWdob3V5NUpKZmw5?=
 =?utf-8?B?NkVadFpPMW00MVcyVDZaV0twYkRjWUpnbXo4SGU1cDFSMWh5Z25waXpzQXJV?=
 =?utf-8?B?SXlLUWhtTmtUNm5QaXJCb3RMLzExUWExcmJ0NU1mbnA5eENSb25BdXlSUVFn?=
 =?utf-8?B?OVRDUHoxT3dkNlcyRC9taC91YUNJVk81SmN0Nlp5UG56MVA2TElFbm5KWXlL?=
 =?utf-8?B?azc3NEd3TFJLSmtBbkJsRWRYaE55ekFNd3FGSmZLcGpyTGhINFFwYlo0QXh5?=
 =?utf-8?B?RFJsRWNXUFJyL04yQjMvSFhkb2xUdlBvbDY1eTgxdDQrNUNkMnNkbkRDdEt0?=
 =?utf-8?B?d1FPaDBIS3k0V2dyeDdzbVcrbmNlZ3pCNlFmRXp4a1JhSlNNOVNiRUFrdzdR?=
 =?utf-8?B?STZaVEpaSlBNZFZHRG9ycXAwdk11THdiRklsZkNJc0ovZ0toZ3MzWUhvK05s?=
 =?utf-8?B?SHZibjFXeDR3di93RXdEU1duMVZMbWRYejFsdTBaQW9UUnF0Qm5iV0NWUktm?=
 =?utf-8?B?Z3dYdTF3RUppUGJzV0tMYmRIQ0pOelFkK3dyeGVEQVdCRFFjVVpGZlI4SGEw?=
 =?utf-8?B?VHBzakVvZFpkZk1nMkE5ZTcySEY0MWRPMnFmVEZ0U28vVmMyTTdiNHJQV3RS?=
 =?utf-8?B?UE9QMkdBTzBacFA4UXN2U2JpbW5ad3NlRmhPMXZNNVk5MlJWaUVzVEpUSFpX?=
 =?utf-8?Q?rTBLkn0kHf0acDnCIfcJCE4hCdqL+DLmguO+V+m?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9b324d15-d55d-4d4b-3841-08d9742ff51d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 07:52:37.5293
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VSpjaOeCR7NPBEIObpkWLQyI+PxjO1sClASS2iDLIkff76MhSZazXfRC3dq5JvuI1YP9UBYCOtnP2C2nXGiz1w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5600

On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>       */
>      if ( p2m_is_foreign(pte.p2m.type) )
>      {
> -        mfn_t mfn = lpae_get_mfn(pte);
> -
>          ASSERT(mfn_valid(mfn));
>          put_page(mfn_to_page(mfn));
>      }
> +
> +#ifdef CONFIG_GRANT_TABLE
> +    /*
> +     * Check whether we deal with grant table page. As the grant table page
> +     * is xen_heap page and its entry has known p2m type, detect it and mark
> +     * the stored GFN as invalid.
> +     */
> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
> +#endif

I take it the write done is benign for other Xen heap pages? I suppose
this would want making very explicit, as such an assumption is easy to
go stale by entirely unrelated changes.

I also wonder whether you really mean to include p2m_ram_ro pages here
as well.

> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1021,6 +1021,7 @@ static struct page_info *alloc_heap_pages(
>  
>          /* Initialise fields which have other uses for free pages. */
>          pg[i].u.inuse.type_info = 0;
> +        page_arch_init(&pg[i]);

u.type_info's count portion also gets checked upon freeing a page.
Don't you want to have an arch-custom check for your new use of the
field as well? Or do you consider it not a problem (bug) if a page
was freed which still has a GFN set on it?

> @@ -82,11 +53,21 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
>          : gnttab_shared_gfn(NULL, gt, idx);                              \
>  })
>  
> -#define gnttab_shared_gfn(d, t, i)                                       \
> -    (((i) >= nr_grant_frames(t)) ? INVALID_GFN : (t)->arch.shared_gfn[i])
> +#define gnttab_shared_page(t, i)    \
> +    mfn_to_page(_mfn(__virt_to_mfn((t)->shared_raw[i])))
> +
> +#define gnttab_status_page(t, i)    \
> +    mfn_to_page(_mfn(__virt_to_mfn((t)->status[i])))

I wonder whether you wouldn't want to at least ASSERT() here that
the virtual address you start from is actually non-NULL.

> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -39,7 +39,15 @@ struct page_info
>          /* Page is in use: ((count_info & PGC_count_mask) != 0). */
>          struct {
>              /* Type reference count and various PGT_xxx flags and fields. */
> -            unsigned long type_info;
> +            unsigned long type_info:4;
> +
> +            /*
> +             * Stored GFN if page is used for grant table frame
> +             * (bits 59(27)-0).

Are these bit numbers correct? I thought Arm, like x86, counted bits
from low to high (unlike PPC, for example)?

> +             */
> +#define PGT_FRAME_GFN_WIDTH      (BITS_PER_LONG - 4)
> +#define PGT_INVALID_FRAME_GFN    _gfn((1UL << PGT_FRAME_GFN_WIDTH) - 1)
> +            unsigned long frame_gfn:PGT_FRAME_GFN_WIDTH;
>          } inuse;
>          /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
>          union {
> @@ -94,12 +102,12 @@ struct page_info
>  #define PG_shift(idx)   (BITS_PER_LONG - (idx))
>  #define PG_mask(x, idx) (x ## UL << PG_shift(idx))
>  
> -#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
> -#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
> -#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
> +#define PGT_none          (0UL << 3)  /* no special uses of this page   */
> +#define PGT_writable_page (1UL << 3)  /* has writable mappings?         */
> +#define PGT_type_mask     (1UL << 3)
>  
>   /* Count of uses of this frame as its current type. */
> -#define PGT_count_width   PG_shift(2)
> +#define PGT_count_width   1
>  #define PGT_count_mask    ((1UL<<PGT_count_width)-1)

Leaving just a single bit seems pretty risky to me, and I can't see
why you do so. You need 52 bits on Arm64. Which means you have 12
bits left. Why not use them in full? Even on Arm32 you have 3 bits
available for the count afaict.

Also there's a disconnect here: If anyone wanted to change the number
of bits used for the various fields, each such change should require
an adjustment in as few independent places as possible. That is, there
shouldn't be multiple uses of literal 4 further up, and there also
shouldn't be a hidden connection between that 4 and the PGT_* values
here. I think you'd better express the GFN as such a PGT_* construct
as well. And you better would keep using PG_mask() and PG_shift().

> @@ -163,6 +171,15 @@ extern unsigned long xenheap_base_pdx;
>  
>  #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
>  
> +#define page_get_frame_gfn(_p) ({                               \
> +    gfn_t gfn_ = _gfn((_p)->u.inuse.frame_gfn);                 \
> +    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
> +})
> +
> +#define page_set_frame_gfn(_p, _gfn) ((_p)->u.inuse.frame_gfn = gfn_x(_gfn))
> +
> +#define page_arch_init(_p)   page_set_frame_gfn(_p, INVALID_GFN)

What's the purpose of the leading underscore in the macro parameter
names? They're not in line with the C standard's designation of sub-
namespaces for identifiers. (At least for the x86 counterpart please
read this as a request to drop the underscore there.)

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 07:56:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 07:56:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184129.332694 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObOB-0008Dx-Iq; Fri, 10 Sep 2021 07:55:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184129.332694; Fri, 10 Sep 2021 07:55:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObOB-0008Dq-Fn; Fri, 10 Sep 2021 07:55:59 +0000
Received: by outflank-mailman (input) for mailman id 184129;
 Fri, 10 Sep 2021 07:55:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mObOA-0008Dk-N9
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 07:55:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 87dfd106-120c-11ec-b217-12813bfff9fa;
 Fri, 10 Sep 2021 07:55:57 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-wO7ZVUGlNEy4R0Rmg1XYKA-1; Fri, 10 Sep 2021 09:55:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 07:55:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 07:55:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0007.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Fri, 10 Sep 2021 07:55:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87dfd106-120c-11ec-b217-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631260556;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w0AUSXHWrpYau9cutFqeoraUwY7Jd8T4zu3u+14PR4M=;
	b=INXHYrxDsJCyManrKCYBEL0u2e2CoZoBHlGZeTHhtXb0DNN59OVwKPLYazRnDC2C0tbmR3
	ek0lpiLR3viU7wO87VEOJ4iYXP/4AZxnXxSLHmkmLjkSFjzRNytumQXQSfXZHVrw8s/CxD
	W+c1ZkcZn3UvtmzQIwtF97+i0DwaMV4=
X-MC-Unique: wO7ZVUGlNEy4R0Rmg1XYKA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d4oD2ATM11z4tadBIciX76Sw9vYXDIIn+P5kuhq/EH62whl6eSHvq8wiquXgCHdw57Ph7q99mD/ObjlVNbQXvazrUNXWKVWLh9lCGUAB+nqIGhKpL0vTdPBkI8GZrciOQVlZOP2yJMsuVYHltSgQOJGkfclQS91lM4HNlG/1zvBidvU668O9zZIBjdgW4Id1g1iv4f0Zp7UQWJmSKekfSEaAvguRu7exbfNK2oCSmYT3zTKXMgDZS0mqhh1obW1vOouAJ6fDN7mxS+bW68tsxyOHHNgf2tWFsIBfGwvUKpKOSIM9i3K9tA4n0rMyai8j+NAvT9y+UJq9goEKF3YABg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=w0AUSXHWrpYau9cutFqeoraUwY7Jd8T4zu3u+14PR4M=;
 b=mx+1rmc6FusyNCekOcKSlst451S6DRCcehtWXWGQUfSJQ5aQG6FFdkSsatLcoHYGVdTDYaEDsBJfQRAPAstc1Yvw+n50Z/JTNoPWmv/RgRo8q4xex+0m8CGphYzi/B/c6xoZesrPE29GGM0bH6eVgdgfxREEi4f/6+syi/6DLJ6Sqdu3YPggFYMT4JyhKP0DJ9//4b856tc6W7WqVafQcnldnbu/anvtLGBPOuUSlm5zTybjMy9T5oh6aV/+/XQI7Igxjfw+nXJd38xn60H+lwhDwi9LnXou4KmZTbED38NZ0nJJKRq/Cq0mFl1Zs7iE35F6usacgXmxFhz6YDppnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS
To: Kevin Stefanov <kevin.stefanov@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
 <20210909163441.44418-2-kevin.stefanov@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a948d9d7-bc87-b6bc-f3b7-8fac9915f3b3@suse.com>
Date: Fri, 10 Sep 2021 09:55:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210909163441.44418-2-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0007.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: aa41d1d0-b63d-4765-06e1-08d9743069ee
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2447CD407B8D54B103B3415AB3D69@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:813;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	b+yfBo7rQUT/ObOR2PYlW0u0TWbD5YFtSwRPyFKH4y1A9lO3h51OTV5pi5q+THxfdt1ewo3auW+Y5zlK8TXMfuOfCsOVL2RUJu7BqdFNqaNJZ8qW7m2Ke5J6jQ2aOZa09zYEAE2Sfa5KbRHZ6xbEbjIOKO1QxpWlekOH1hRlRM7Z9chYwIz5/+pQXeE+NhXCIx2fi+Uj/V/w2mmPHdrYHpjtO5pqP2p/sKoY3EcddIdr6EuOBqvOuC9GRO1A/T0/MxUQJrpWtMVpM443pVrV5jh56U/pAjMjTiH+DWLTzFbjIfN9aCRQMRumTrybW0hrf3ckj/WDupKOIUakwu7wUnP0ZTOr+kla0otEB7Vgq+ZUbt8BFJmiVYbk+uMjRUR3f1SCwoPN6+fjZ+GpfKm1k8tCxKTgVAEcxWkjHbsZ/vV8PXHvP2fnKaF7QGLXjG8Tv4yM9p674JQuIz1ZLmyusWflDV+0BsuY9M7pWvheSEpNjkkjv73aU0PDCPqVNPcjmF5xEgLU6IHtAcBRZjGte9NxeTL6LhtBc/JEd60YYsj1H/NquX7/2yFzfiR18it8Tberh9+szX8G6k09JDR1+hcWqECYA/qUJUVQhoJe6t+IDXhxyeWkNKw6+3tlY/n6SNaPGgg8c5/bBhFG1MLyLRwdZ2aTgP67yzMjDKdAhveTS5jkK73xJWuIdem1Rc9NfozRiOu8Q5ilPqeEG0lVtXDDO1E18m4oMYkFnhS2/r7ZLLAGeSHPSluuUlvaZSHx
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39850400004)(366004)(346002)(396003)(136003)(376002)(36756003)(31696002)(2616005)(956004)(5660300002)(54906003)(38100700002)(2906002)(86362001)(16576012)(316002)(8676002)(478600001)(53546011)(26005)(6916009)(66556008)(6486002)(66946007)(31686004)(66476007)(186003)(8936002)(4326008)(558084003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TndrS3dNaWllNzl6ZzJsMXYrdE5QOFJBb1ZMRDlLdUh4KzIzVmFhNW1na0Rr?=
 =?utf-8?B?Wnk4SDJIZVg2VW5Ha0xKaWdVUm9pdWFRRE1RTndycEJ4Z2VVN1llWnFnektl?=
 =?utf-8?B?czNKMW9MN1gwYXZLWFdlUGk3UStQajVFSGo1WEpKRUM1SkhXM2hLU3dUcTFn?=
 =?utf-8?B?N1RLdUdmUGJ4cUdsSnNvOUFXbDJZaDNHVFY4aENETVY3VEpNTGY0a0lHcHdF?=
 =?utf-8?B?UHpoN050ZlpFTVI3SWtQU0VMcndSMmNMYUk5WGNtdU5iVUtiekFFb251eXZz?=
 =?utf-8?B?enVtbjVaWU9xWGs5SENnbXpwZEFUc1VaUlAxc1c2aGZJdUFQNm1SdUF0cm0x?=
 =?utf-8?B?MWJ5S1hqRiszRDVxV2RBR2lvYlF6TUJRSnNrWWZCT1pSbmppSjRSNm1XNXJu?=
 =?utf-8?B?RktKRmRVSVJ0S1V1U0RzSHBmMURwb2l1ZWJ3KzI3OUpURkFwdndmSTYzL0ZL?=
 =?utf-8?B?RzRMQXZscFhHVjZsWDM2OHBSeGJRN0g1K1l6Y3NLSUJwUjA3bzRBbktsRVhZ?=
 =?utf-8?B?Ym9GWHR1OVZZQzBTeS9JcnI3U2Foa2hzb09rNDlpRGNvRzNNc0xuWVplK1Vv?=
 =?utf-8?B?cmxINW1IYzNMNmZ6ZTZoVk96QzdQL2xlTmNzMTQ2cVk4QlNjVE93MXhYYzQw?=
 =?utf-8?B?VHNuMjdFaGlZaUFodW9pOXN0MjdVM0VUZUVMZlpiUy9GU3VHRXg5VUdJc3FQ?=
 =?utf-8?B?dUxuaWV2YnRaMHlGRFl0anZ3cldZQWtSZFVTb2hidXNjcStDRklNd0tVM3M2?=
 =?utf-8?B?UUwwWm1yVVdrdkRiaC9KTGV0WGhkUVlqUHQvcENDamF4ckxFaEF0UHFObFh2?=
 =?utf-8?B?UVRZNWY4UFkzd1k5Y05vdW1EV3BULzI0UTJ6aDlKRUVuaUgvLzRHYXBUUm1n?=
 =?utf-8?B?NVU0d1FDSkNpUnpPZTZjYStmZ08xeHY3bS8zL0lmSUFySzdYSXdPZUxhUm5u?=
 =?utf-8?B?VU90eFp0c2hYZ3JEQXV0RGEwcG0rZml5Um14Y0RtWUpxQ0hXTURnQ25wcFdz?=
 =?utf-8?B?M0hycndKbm9SVnNnMVlQSFNZNUs3L1Y5Y3FNT0pINW5MTU8vUEJJRytqWno2?=
 =?utf-8?B?NXlXazJnMVd5MHZFT1QwWlNGWWJHdlpPcTVqSnRnZncyUmhjTFpHT1paTFhT?=
 =?utf-8?B?aDJEZ2IyOUJuWHB6bzA2UjBsODZ2SE04LzZNb25OYXhocGx2R2ZYeGFkazdm?=
 =?utf-8?B?SE1BYU4xRWEzRmlpcmtZSVBlZEJvUGVaSVhQWk52amU1TkFLaExnbjRuQ1JK?=
 =?utf-8?B?SnhLdmsySFdaQjRFMTgxTktkUWEwS3Q5VjM1MnhaZ1V3Sk9mT243R0E3N291?=
 =?utf-8?B?SmN1alhLZHFPSHZ4S3Z2MGV0RVd3L01mcGkrcTJWTFhQR1QvUzNuUDk1OVpB?=
 =?utf-8?B?OXFFTWZnOUEyWWhJampLWVdrWGtRTmsrZ1NsRmo5WllQdzVVZy9YY0N3OXhh?=
 =?utf-8?B?cmx1UXV5Qjk5aUxZQzg2a2lnNVg1MVFKUnFZa2dOWVlIWjUvN0hsbkQxWE9O?=
 =?utf-8?B?bC9aeTBScEpiZzRGeSt6NnlJSjdKQngrbEhubW00K0d2MXVxOWVNS3JXR0pX?=
 =?utf-8?B?YnFRN2gvS3ZqZjlhdi9ySXNYNzhWSXl5ZEF6aTYyZGN4Ky80a2dBNjdiWExl?=
 =?utf-8?B?ZWNwOXVBRlk2bDRwTkJZTFRmOUpHeVF6c1RpaGFFdHZFM2tydnVnTDU2WTI4?=
 =?utf-8?B?d2hsanZ0Ym9qTUtqOEVvcEpOMFBKUkF5VUN4bFJUK3Z2cUhySTBKcjczcCtF?=
 =?utf-8?Q?DXAqgZJFiiEVpOlAO10JiIXwzCyDyd6rDL+IhuM?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aa41d1d0-b63d-4765-06e1-08d9743069ee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 07:55:53.5304
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Sz9tgDQTztLnrmtjc9dMn4vtELbQeU4AwJXg/ggZV9UV6q5czoDoFIPMOEospWaXecxfOb7ku7kyJOl8ZG91FA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

On 09.09.2021 18:34, Kevin Stefanov wrote:
> The spec requires 64-byte alignment, not 16.
> 
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 08:18:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 08:18:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184144.332705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObjf-0003JQ-Mw; Fri, 10 Sep 2021 08:18:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184144.332705; Fri, 10 Sep 2021 08:18:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mObjf-0003JJ-Jg; Fri, 10 Sep 2021 08:18:11 +0000
Received: by outflank-mailman (input) for mailman id 184144;
 Fri, 10 Sep 2021 08:18:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mObje-0003J9-He; Fri, 10 Sep 2021 08:18:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mObje-0001uY-9k; Fri, 10 Sep 2021 08:18:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mObjd-0004TC-UC; Fri, 10 Sep 2021 08:18:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mObjd-0005pi-Tg; Fri, 10 Sep 2021 08:18:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jwwQGzuDynC9JCBjVPUcfec9rSofD7oOeIexG7xlu/A=; b=K8TmNMJNYGQ46rrXkh2RM4TAnH
	esIw18xtV6Gjt7xLDfFvw18GBpcjLDcY4SyjtPuPgdzN3Yx+8A7oXSQ1fUmBbhuaMshjdxoa5hGlo
	ou+UiUg/7b497K1V5XF+h25CQ2VvqwuaKL38NmGF9ypisfKTHEyDRq3kcbWLmqXKUzNA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164914-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164914: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a3fa7a101dcff93791d1b1bdb3affcad1410c8c1
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 08:18:09 +0000

flight 164914 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164914/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                a3fa7a101dcff93791d1b1bdb3affcad1410c8c1
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  405 days
Failing since        152366  2020-08-01 20:49:34 Z  404 days  703 attempts
Testing same since   164914  2021-09-09 14:59:48 Z    0 days    1 attempts

------------------------------------------------------------
7272 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2241760 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 08:20:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 08:20:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184152.332719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOblX-0004cr-Aw; Fri, 10 Sep 2021 08:20:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184152.332719; Fri, 10 Sep 2021 08:20:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOblX-0004ck-5f; Fri, 10 Sep 2021 08:20:07 +0000
Received: by outflank-mailman (input) for mailman id 184152;
 Fri, 10 Sep 2021 08:20:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOblV-0004Yg-Sk
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 08:20:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b12492b0-d69d-45f8-8cb9-d8391c012ffd;
 Fri, 10 Sep 2021 08:20:05 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-j6w8s61OO0e764bxutmWmQ-1;
 Fri, 10 Sep 2021 10:20:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3389.eurprd04.prod.outlook.com (2603:10a6:803:b::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 08:20:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 08:20:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0014.eurprd09.prod.outlook.com (2603:10a6:102:b7::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 08:20:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b12492b0-d69d-45f8-8cb9-d8391c012ffd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631262004;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=H2X3bXUxHHTgI4mmQ1aDzuY4bYr7VKDx8vAHGZJEbFQ=;
	b=TAM2HsFZQCunS0mRjGPUgkMypCcAuZZu1oTwsWvVwrvU/tdh0yp3FJ1uTVG7Kuam+R3VBo
	LU1/jeNeRC85IBOIIqUlRQbf0TfJLQLpIjWV9Q4+5WPQLzxearwuvF4fzQOsbqCo4xqKGc
	32pXVfGrmmsOkmvpvLPB53UvXqTxacE=
X-MC-Unique: j6w8s61OO0e764bxutmWmQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dnwBOsXM2KRsYVKEt6xjgNk/krZltEBkQo8J4CkrLcttkjN6CWR/ROQkjDZBi7kMi+8snFUqELuB+lYt+7yWtIHRO2EeVIy+DdocaWA86c2/m3lO0TrY6cPGrUQKpIli8o6gTX6MlUhDySRpzUcBbRMGfzFGj2DVN5LVpLPqHHnKXxAcAyFd2IvzH/LZ3PT8KbO1RU3bCr90cTTM0orsJCnS89XtEMB8/da52vauaItE21MjpLKXWi37kALZo0r3JN9UXvW2V/VjOCrn4Is8wF5BdO08GRACs9Hoogg1dBCVF2oFdA/W9056Ps71DWTknI7oO5WhkebIZJJCh6I+eg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=H2X3bXUxHHTgI4mmQ1aDzuY4bYr7VKDx8vAHGZJEbFQ=;
 b=AC5BghWwKQolaLgRHV0O3cVsCtTPlL/E3/tBFIaXV8o6h+Kh7qfSK9nZ1cazAuXfCECG9U+kIhDsz9gb5eTrtYCNKGV/m293aJVoOXuCDYv2NiYjB/j70rtjH9QZS/B2KCth2aSQqF+O2Iayetz8mKM2Ket3at7oXR9mMWMqZFIfHS2LcDzAkvoPbjoi83t3Vty4U12INZibzZHFudsRmd7NhtoFd+A9SRFi6G2ZMtWQFu+Upv7obu5JEOFPTKBkB1tSBTVIuy9RWgd3zN7KmZdToWeNdTGN3S/1NjAyxqy4c2TBLDznHA68trcOhPNRA/RrRe9cs/yk9ZdztaLpoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/3] tools/libxl: Correctly aligned buffer for ACPI tables
To: Kevin Stefanov <kevin.stefanov@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
 <20210909163441.44418-3-kevin.stefanov@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <affca979-4082-e825-dede-b27b5574631b@suse.com>
Date: Fri, 10 Sep 2021 10:19:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210909163441.44418-3-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0014.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 589d8c83-f580-4f7d-dc51-08d97433c8ab
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3389:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB338971B0A20662CE0CC5EA83B3D69@VI1PR0402MB3389.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NaH+Hs+gsQDqu5aQ1bsVu3cPOgY+CHOca+q9whMi65NmGZMs4F0HT6Jq+bL+M3GfXRtu6DWnuBzUMx1rWeWvRTCb+HTqvGI0QNzb2oefHUcHbthJ6Aj5M05o0+S1c48DI8blyZMoPwmKaa0frxBTWmvxNM76t76he+IgrVsLzhQ9xoTsGhjQJOuRZVNBsSPstRqNGuBlSaZtpDSqxURXLkicLN81F4+AZPHEu4TRmPNmnaKAYqJ95kBpGy3ZW4RBcevn9FzwVig4Lzp2JBHzNBlE1b5NEGE5mw3eqhezU+FVGN5WsGmsCel9a00EytVCxWP5MpYhA7SXlm58A2j/n2YOZ7fiuW9O5erWwp5a8J7Zi0JF7APyzUjhIe/6A1nkGgqlIWdvclrhj8/xoI8meCQ2Q557iTnWK8G0K+FvCOH3SlMK438J7W1+9eFnssMLMGCA+J5s3u6g07icclnxWcYK75Bwn8iuIJ0EG3E977n41hdFeQ4nNldO+oFLs0hmFM5o4UF6LDDD2a8372hZh6mWAU4TkXifxrBXjeHIS6Kia+gzSHV0+SZiPoNwcxIsys2rMqa7baXOHlA1BAh1srVjmLXmabS+xbCOcrFAgeOqSzbc4o/uSu/aBK7jfQSLlGxTKgI9uF8xVJPAI+3qg9ZejmSGp1wAP2bIwiFtN3gPRU9wJTxVyB5tCopqFOlHklw+uDGrqhT0pt3YhnDRXzU/nx0xKAPVMKH6JKmszQsWACW9+1MaRU4omqjTKHip
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(186003)(6916009)(26005)(2616005)(956004)(5660300002)(4326008)(38100700002)(31696002)(508600001)(8936002)(31686004)(54906003)(53546011)(6486002)(36756003)(16576012)(8676002)(2906002)(83380400001)(316002)(66476007)(66556008)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SG4zRTRobjZoTEJ3Rk4yWklqdXQwUWM0clZneUMyUmQ5UlBOVkJHRkhSbWUz?=
 =?utf-8?B?YmhnNzR3bXBONjhua0ZJOGVxYXk4N1JIR2lDSE1aMFFPR2hjZUNPUDE1YUF5?=
 =?utf-8?B?OFFPQU9SSlY3Si9aRWd0Z2xJdlNGRmplZTJuc3A5cFZ1ZVowZWFsL2o1b3Ni?=
 =?utf-8?B?WlYwM211ZnQwNVFYa2hVZXVOdU0rczViUmtLRmhkS3o5RlNjZytmdHhOOHkr?=
 =?utf-8?B?QTk4MVZwY0lFVFcyV0NwSHdvZjd3MVZsOExTOVFtSXo5aFcyZlRzTDdGUHZN?=
 =?utf-8?B?M3lseEM3M0ZKWWYzZ3U4YTd4bUZlRWxLb3VEbFZsempsU1FjMFBjUVBFQit0?=
 =?utf-8?B?ZithZk56OCtQYVdZU1Bjbm9BSXMxUzNNcWhRMFB0WGV0WlZPWDhOZk1MUUZt?=
 =?utf-8?B?em5oTWVvSEtJZml3UmdqT0tveGxkQk1vK2hjU3JzQk1mSlRUMUhjMUh4TlhV?=
 =?utf-8?B?aHVIVk9HM2xBc2pyellLWUtueWowSXZYRXVkcVFBa2t3MDcyRlk3QS8vVmZX?=
 =?utf-8?B?WEtJZ2lUZXB1ZmFYMklBTDV2allFek9rTE13blBXK3h2bFlCOE43WUVjUG9k?=
 =?utf-8?B?ZzlqakVRQTZqNU10ZUF4VkszcC9tSEY1WFE5MXZiMWZVWGpQQytvdUZLL2NQ?=
 =?utf-8?B?VzZsL2VRaVkxcEpNZHg3Y3IxaWxpWmF2TlFJb0oxUHJZMnk2VWxucXhpTGNK?=
 =?utf-8?B?eStpc3U5R2VINVJQYmR1SjN3MzVnblh3WGZYZThVVlRlVFdQVjBFaDljUjBV?=
 =?utf-8?B?OXJHZUMwSU5tZXJzbTRIRDl0MXdQTVBwc0dMNU8vWGZRbnZONVNvTWp2VHYw?=
 =?utf-8?B?MXhDK0pkUktFOGY4cCtMU2FMVjRDZkMxWGpQc1VUaDlrMjdBMS8rc2dERThE?=
 =?utf-8?B?Q1A3clk3OE1QSU5EazNHQ1ZHYUNncmpsQlFscEh5dFNLOEZOL2QwOGRyek1T?=
 =?utf-8?B?UHQ5Ti8vbW5XRDBzVGRIeURpT21USm93NVpBTkNiQzU0SU9xL2pQVTB4ZHRH?=
 =?utf-8?B?aCtnY212R25PNUhIS0ZkQ1l3enpzYXByQ24zbW1JdU51clpZZEZvMVpKRGtC?=
 =?utf-8?B?YUFkbWFieHpPZHhFYm5rUXdvdEJSUWxQZUhzQThwZHhzRUw0VU9HdXlRckQ4?=
 =?utf-8?B?T1JoU3Uxbkh6U0FDempiUEFGeXRycFdNRkExOUZvY2tWZXNCa05uU0pFbmRE?=
 =?utf-8?B?ZUx0ak0zVk5BK3ZDTjFIblpOSmJZdHVIT0lFK1ZtNDhKeXF2d3IzVUdNVEdI?=
 =?utf-8?B?T0h2aFhOZ2xHZDlLNkFuUWh5OXhaSGJtVkVFdWoyV20xU0FvNXZkdzc5TURU?=
 =?utf-8?B?aVh0OERQbjA1RC9qL1BMRzZ4NTBZQjhrcHd6Ujl4WnI1WHBoZkJDbmd0ZHMr?=
 =?utf-8?B?YlluSS9DaWdISSt6c1JGbHhvMjlYdlFydHBKZjlVUzZvbEFhejJPbWFlaFRm?=
 =?utf-8?B?TTg4QTd6aXpiS3lSZFM0K0JXclgwSlVDeVdYWjFvNE9sVUc3UHFEdHNTYnZT?=
 =?utf-8?B?S0pZSHBtSko3Z05yZjlXQm55VGNQRkJJOHJtZzVzTFVseE1TSWVYZXVDaS9v?=
 =?utf-8?B?d1Q2eFVSaHN1MmN0Z0g0TmJyN1cyWEtwZ1ZEREUya0YvS1JHMVFBN2tObzFm?=
 =?utf-8?B?MFlwcms1RllsRzNRbWRzYWlaUnZJTk54QzZiTmtBNG85bGdPK1Q5VHpDaGZH?=
 =?utf-8?B?cEU1ZG1IM3U5a3ltNUk3akN4Y0pwU2xsWUprZFc5OTE3NHU1WTY5NkI5NXlk?=
 =?utf-8?Q?bJRi9t+cQX+Hqvrd4k8jHVyBnWw05G3S+6SbxJR?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 589d8c83-f580-4f7d-dc51-08d97433c8ab
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 08:20:00.9235
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iy/E9wapMELC4uYn71iozfiPOZBKhEPu88cewvaGR+HSfY+hlgz6do1m42QkN5RrS6PUaWVIkZFMNTnKbVql6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3389

On 09.09.2021 18:34, Kevin Stefanov wrote:
> The pointer resulting from libxl__malloc() has no explicit alignment.
> As an implementation detail, it has 16-byte alignment.

But the buffers obtained via libxl__malloc() have no association with
guest address space layout (or at least they aren't supposed to have).
That's solely determined by mem_alloc(). I think it is a bug of
mem_alloc() to determine padding from alloc_currp; it should
rather/also maintain a current address in guest address space (e.g.
by having separate alloc_currp and alloc_currv). Not doing so leaves
us prone to encountering the same issue again down the road. For
example if higher than page alignment was requested from somewhere in
libacpi.

> --- a/tools/libs/light/libxl_x86_acpi.c
> +++ b/tools/libs/light/libxl_x86_acpi.c
> @@ -193,6 +193,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
>       * Set up allocator memory.
>       * Start next to acpi_info page to avoid fracturing e820.
>       */
> +    acpi_pages = (void *)ROUNDUP((unsigned long)acpi_pages, libxl_ctxt.page_shift);
>      libxl_ctxt.alloc_base_paddr = ACPI_INFO_PHYSICAL_ADDRESS +
>          libxl_ctxt.page_size;
>      libxl_ctxt.alloc_base_vaddr = libxl_ctxt.alloc_currp =

Below from here there's also a suspicious use of 4096. From surrounding
code I would conclude libxl_ctxt.page_size is meant instead. Could you
consider taking care of this as well at this occasion (possibly in yet
another tiny patch), while playing in this area?

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 08:36:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 08:36:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184183.332748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOc1B-0007tn-2G; Fri, 10 Sep 2021 08:36:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184183.332748; Fri, 10 Sep 2021 08:36:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOc1A-0007tg-Um; Fri, 10 Sep 2021 08:36:16 +0000
Received: by outflank-mailman (input) for mailman id 184183;
 Fri, 10 Sep 2021 08:36:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOc19-0007ta-0y
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 08:36:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4dc45235-298e-48c7-a8b1-570f8dc50fea;
 Fri, 10 Sep 2021 08:36:14 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2051.outbound.protection.outlook.com [104.47.2.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-MAJFzGm6MaayGsie9RA_lg-1;
 Fri, 10 Sep 2021 10:36:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 08:36:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 08:36:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0180.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 08:36:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4dc45235-298e-48c7-a8b1-570f8dc50fea
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631262973;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TZZUXn54cWZ3j2p/0IrNe7n8D3YP75eKJVHxyNlkK+s=;
	b=FteVXUC7xLQMA4iXwlgN/Zj+ix9hOYbtywHve3oUWQdr5524i16dAW9UTX9NjOAXPWS3KE
	rLU7nIzShn2bFxM+2qvNbBsbLeTUU7xwk58OUGM5H7GB7ONhgTLsdcBKzRvVQyt3beEWTS
	PghKibrbJ+ICvGElnPPrT7u+PzSbO+U=
X-MC-Unique: MAJFzGm6MaayGsie9RA_lg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TX2TvLRuNQcvTw47cIP+CpeY6PWaownceDaROxluRLJkFs1iFdQE+G+BbJ6vnKVSGHJRo+9yfvyb4vLGNB9+rSu0iQVPrJrPD/gEpBRXWkd/VzEYyPWYkSIlf19Tg03W4adpqc8+W8XLDRsJk53wHGuYWzdQlo19TmINvZS9TIGVxq/UA46azYoPckZHDuG5BcvqoWUOTnLs5fZ+NqIIsyFG28fwfm0fXI9geTJLw9wwiSX6RgE6hnJLTqiUykyYHYw0MsneFIEpD/uPc2adU25CbFCrNhfHbPJlRjTReaIHOUEq52pP0XjbPoaBcyCimjWBGhHFjqklNoHcqo7GDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TZZUXn54cWZ3j2p/0IrNe7n8D3YP75eKJVHxyNlkK+s=;
 b=no4z6rQyjfZdy6h7ubZhD8/YANvBpdBh40P1J9dC6ZAGyA3f9n2QmvlCxuzG2KOi0BydIElnqyZ/eDnKFQr9wyr5wratX8S24S1w51YScBwK3yuiWlGJyQ0mYtdqLu8Cfio2B5vrcqQIzLt8CU1lQJohHHu2UOxehoTVBz473hiOHVuMTrjYgGu718bRi0JJ4kC4wh0IehB4AJReyPge0AAubO6cU+1HzTb8ZMidAKBIdHrpBLNVFBxFI/RRqRFHupIivBlvQ19V35XZbqkqXYrBwT3vwkRZI3DBCLQkn0Eop32HSdTdbpWBOQLHN+gF58N+DgdqLZB7RI9+DJEBQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 3/3] tools/libxl: Only allocate 64 bytes for RSDP
To: Kevin Stefanov <kevin.stefanov@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
 <20210909163441.44418-4-kevin.stefanov@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <da7fd9b6-6bd7-3bdb-1f44-8ceab73ff809@suse.com>
Date: Fri, 10 Sep 2021 10:36:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210909163441.44418-4-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0180.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1c::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b961846e-8ad0-4962-3873-08d974360a4d
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB702481AFB5D9A3EBBAD2C5D2B3D69@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CIN/bCP+Aebax8f5DHg+pOwWwL/ugj3YBdUxKaRYbOEgve/1N9YUeg2f47YNFd+502UFQWu3wPOtk7qCMOUgp8NOKsf12twKpgykm7f3yOn2mptbngzEZOrCyyHpK7ukQu7G/gok6qdS+epAqpwhjZpw135yONPCz3Kj5wisi8hU1mcSKXN3ZyQYsAti6MCRoYp+HUiBtEvw9a6zI8JUeqJdH+O0Zs+FelyJbB50T9Kp3PZUhrPNahvWeM5lOpCvpk4mtMLs8mKqMHigKy/naIzAtORZSJ7JH+OP0OnPmp0ajxUQse/QkLpq2DWYCkltsj3hwPYVyP5kS/JkDbIDfQgUm0VKzW1HONbrGE3W2Z9npWciIFcKiLpLciWXrm42ZsVWScI0k+FWxriDGH7lPnQGJgX2pLpxneVT76so0nUvz1iQzTg/3K+2kkmfRuMUVvGBry5gKTKE/SnXNcCMnoU0ywjmhhlTmfn871wDJ5GkiXUpo5q4T5gz+p4YtfZ36YxG9r4C99W+WVQqmXulfNRCh6ZIS2Za9ztCB4fvyjo+Yzmux6cu536s4uZlcOjF5RGa5H2Vm+qb7zphusYJ9d+MiJqGdOnzvU/BrtEONpggnJNrCHt/ZEKL8iCpnlq6jADh7StTnWBUYAAN78w2hikdzrNkNcZ/+aPQRvsYQcjRFogU4n4BNLXrd8fc/ORWVSnYFnRTXPXnx4b0Z0hH3jLuPud+VxAuzL5JFBE4w1kBr+fFeA44vIu5fj36ALb1
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(366004)(346002)(39860400002)(396003)(376002)(53546011)(38100700002)(8936002)(31686004)(6916009)(4326008)(956004)(478600001)(2906002)(66946007)(6486002)(8676002)(26005)(186003)(5660300002)(66476007)(66556008)(2616005)(36756003)(54906003)(316002)(83380400001)(31696002)(86362001)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TGcyQkwwZFA1cW1KaDZrMUFlRm9xaVhkUmkwV3JMOXhTMjJDMkllTzM4dDYw?=
 =?utf-8?B?dFJianlzeS9LdzZYUWtQazdyMmtPMmtmSnM0RDR3TVZBcWRGY05QMFBtOTZ6?=
 =?utf-8?B?RXRUTFhPVTRzSVlza3VFOU82b0lTaFE3cWRKL1JQenRTU2hmL0xjNCsyQU5i?=
 =?utf-8?B?VEt2RDZsMFNvcFlnNS9aelhzYUdtNmN6ZjhUbGlrTnhKNEhNSndLMlVZeU5n?=
 =?utf-8?B?ZHJmeW84VnhaemluR1FYdEtwc0Y0OFJOQWtXblNPWEpqaVNtT0ZpZFM3cTNO?=
 =?utf-8?B?cFlwWmo3dGNOdDkyWVQ2NXdQcWMrVk5HKzI3akVFTlpEUTBHUTlZNDd6Yk9D?=
 =?utf-8?B?c3N5ZDd4dFFOdlJwOFI4QjF2T1RQRVBTMkhWVkFBaTRCVElZUW1TcWtQNG9C?=
 =?utf-8?B?UVBOTk1ZYlMvVnJVd2ZIMFI0WE8xSjdmRWhORVlQSDdVRTVhc0oraVlmemFw?=
 =?utf-8?B?d3A2Kzc5THFRZ2pVTmJTSkRPNkdNek1iUGduV29qeVhRWXdlclZKNVZpQ0Vu?=
 =?utf-8?B?NTZkNnZPY2hZc1NyOW5ZVUd4S0wvWHBOMGJoUEdYbnFsRzhHNlQxUHdEQ1Zo?=
 =?utf-8?B?K0ExeVVvK0pQajRVR1NpcWE3YnBCZFRkdzhFZUFLQnRTZzZvcGFma3ZXUndB?=
 =?utf-8?B?TklHNXIrZjFjV2l1YjRxKzdoVDNwVzYzbmZVSUhtcUwrc09PTmlIN1NNOGhi?=
 =?utf-8?B?Z1hVZEhUUXRuYlRycmVJMUJodzNmWDRoQkc4d1FLeXVSUnNHR3dUcGVDeG9y?=
 =?utf-8?B?WTdoYmZLT3I1RExieVpYc25BSkZNR0I2Z0NTY25haVZ0NU5rWXJXQXNZYlFm?=
 =?utf-8?B?NHpZSURBT2RvYlZURnRsQlgvMWVpNHNackFMSXZwK3JMa0wyTDU2bGprbXBJ?=
 =?utf-8?B?WkVGVjV6Y1h4MENiZCtpK2pGUE05TFNUOFo1SFZKK2szQkl4ekt4S21BVzFI?=
 =?utf-8?B?THFHcjdISEl6cWV1RXozVjdXbDB2ajdvUnh2aE92ZGRtdG80a29oeDVPM0Rx?=
 =?utf-8?B?SHlrNVlOSXBiUXNUbldweXhsQXJlRDA1WWVITHNhUnA3dWo0TDJQMG1ra3dV?=
 =?utf-8?B?YWZ0OXhiaXArSDlOT2dPc3JJYkF0Nk9mYWVNY3VYMnV0SGFFd3J4SjdFQmpz?=
 =?utf-8?B?cU9wdjQ5dk9CcDBiRzNORExrNU1KWVZGRlMxQkJ2c2VBcHJXNnNZRlZrNWJX?=
 =?utf-8?B?WXJzN2JRNG0vT2ZKOC9KNTZWWkpUUDB1OExwQ01EeEcwTlRuRnZRNVgzUjg2?=
 =?utf-8?B?NlNBOE5DNWRnTUF6Nld6ZC93ZWpEV0ZCZ2N3RWMrYkFzUGFOYWdvb2J5NzF0?=
 =?utf-8?B?SU9TOWRHK0Yxb3ZPU0VlSTBIcVYvSFBQNE9GSHd2WDcyc3Nkam1ocHZZZktv?=
 =?utf-8?B?WEM5R2MvY1d0bjBBLzU5RDM0amFSb2dDZE53cWJtZ3doSklFZUFxRHJlUzRl?=
 =?utf-8?B?TFZDVEhIMTRWQUo4S0JockJKcGtsZDNhU1hDWDY1Wm9nK1hpc2ovYlVVQjVi?=
 =?utf-8?B?ZUI4NWdCbjh5eG9RTGR3QTZValVFRndYYjRuck9WdlBraDJCNjQrY2pRZTRr?=
 =?utf-8?B?Y0NBVFB6dFhXYzFrdkVRVkRHV05GLzVGMENYblJraURJNlFJKzNNWHlHUVhn?=
 =?utf-8?B?SUhBdHdkekJ1M2ExY1FMb01vSytNbmNKN0l6c1lQOTRpMWpHK2U2NEpVV3Rh?=
 =?utf-8?B?anIrRm1KdlY5d25aSzNEcllsdThaYXhITzlUVHRaWHBpN2RsTXR4am8yTDNT?=
 =?utf-8?Q?0vQcov99c/VcEgnEJJLHfHKNRAYCaRAbYibdRGc?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b961846e-8ad0-4962-3873-08d974360a4d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 08:36:10.0374
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KEET+VEqUPFOghXmA0uUr3hXB3JH/LniST4eCsGSsKce2M3PQ2hhql5maijbWy5I/nJkWtkIlJJR4DSG2dxx3g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 09.09.2021 18:34, Kevin Stefanov wrote:
> RSDP's size is 64 bytes and later in the function, its buffer is
> hardcoded to be 64 bytes long. Don't bother to allocate a whole page.
> 
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>

Purely technically and within the constraints of the present code:
Reviewed-by: Jan Beulich <jbeulich@suse.com>
However, ...

> --- a/tools/libs/light/libxl_x86_acpi.c
> +++ b/tools/libs/light/libxl_x86_acpi.c
> @@ -183,7 +183,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
>          goto out;
>      }
>  
> -    config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
> +    config.rsdp = (unsigned long)libxl__malloc(gc, 64);

... this is the 4th literal 64 in the function (including one in a
comment), all of which are meant to represent the same abstract thing.
And none of which look to be really correct:
sizeof(struct acpi_20_rsdp) == 36 according to my counting. While I
don't mind using 64 (for the time being), I think this should then be
via a #define, which would be accompanied by a respective comment. Or
else via sizeof(struct acpi_20_rsdp).

But of course hard-coding the size isn't really forward compatible
anyway. It should rather be libacpi to specify what size the blob is.
And then it might be safer to stick to allocating a full page here, as
the actual size won't be known up front. Or the allocated size would
need to become an input to acpi_build_tables(), so that it wouldn't
risk overrunning the space.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 09:45:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 09:45:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184198.332758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOd5z-0008AF-6W; Fri, 10 Sep 2021 09:45:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184198.332758; Fri, 10 Sep 2021 09:45:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOd5z-0008A8-3K; Fri, 10 Sep 2021 09:45:19 +0000
Received: by outflank-mailman (input) for mailman id 184198;
 Fri, 10 Sep 2021 09:45:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f6wo=OA=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mOd5x-0008A2-SH
 for xen-devel@lists.xen.org; Fri, 10 Sep 2021 09:45:17 +0000
Received: from mail-wm1-x335.google.com (unknown [2a00:1450:4864:20::335])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b72956e5-a275-42c7-a525-f0065cfe5a4e;
 Fri, 10 Sep 2021 09:45:16 +0000 (UTC)
Received: by mail-wm1-x335.google.com with SMTP id
 n14-20020a05600c3b8e00b002f8bd2f8ab6so978485wms.5
 for <xen-devel@lists.xen.org>; Fri, 10 Sep 2021 02:45:16 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id o2sm4476233wrh.13.2021.09.10.02.45.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 10 Sep 2021 02:45:14 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 88D361FF96;
 Fri, 10 Sep 2021 10:45:13 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b72956e5-a275-42c7-a525-f0065cfe5a4e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=tdlZIRZ4LUXk5hXsKmMnfmEHeHorw3simqIdNCqbaxI=;
        b=jR6p+yz2FoXpA8HUJmOWuxRpfnVCKoHfuGCdBrsN0SyVmQJzoGqIOroWJI5il33Hg1
         QJFKBvetQfZS5D2nfd5x+M+MYUNd+/im3OALF+77BFenByFCrH8SB21h6b/ewcfZZRxp
         DJH5aRimkU+/Oh39oULZAsiAR1zqZvioU73pm76d8Znve9sVc9MXdd/n1D7fdf5MlKG5
         RRmgw5oPM5k0sJ6RpNXQQeg9OmE9ay2sYVv61teqWLBbB6yuBljxbtHZP3837x2BlWuQ
         r74BU/1AEdqXcwrCty2iCX7tcW9kOABMLEEEJgGByIqdSttI2nlYIr+bIwmtKKgYu9+o
         PtZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=tdlZIRZ4LUXk5hXsKmMnfmEHeHorw3simqIdNCqbaxI=;
        b=WTqiRRivuvARSa6yDrwrc15Ls6t6u84AkHVcynmvDPY7mKgOVXrnGOkL+kFQL/lD4z
         xh3NGQaTNP8PImGMqc0RWl29EJ1x6dzPUCuv2WPowuS2f2Drl75rlNBq1tXWSYIDfPjm
         I5J//7o2pZvbFA7Ax1MWfe0aioNsb1Xs0mSHWMxLCkindqiCyYQ9AObJl4Ds5ACdDSNf
         s/XlLLBc96yQUUZDXKcelX4UiUpe6LpkVw9fJ6FrJwMU3cgX1RDYcqjFwcxZbyVe+SEY
         YNo4kROXsKzaFg2HtolSSYHMOO0VVWsQe3uoQlPiubHICv0jVnZgP/oSWGn8Hq/S+5aj
         yQDw==
X-Gm-Message-State: AOAM533J63bqGaFa7pQ8outB+f+z7Dg1KSQravsIXJ1KxZoUMKoeXbP9
	5sMfOoG2dhmQAleKMIv1vj1ntw==
X-Google-Smtp-Source: ABdhPJwjpuFNSXiF9a/KjwI4ZM9kqVvk3d/Ut+zQ7ShetXeWPEEts2okaL73vNcnh0ZVbxhLKuAC0Q==
X-Received: by 2002:a05:600c:19cb:: with SMTP id u11mr7060905wmq.16.1631267115491;
        Fri, 10 Sep 2021 02:45:15 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
 <CACMJ4GaJyAnguzAEH87DSNN_+GhEa5jRbw11hVj-yWMAXx8V7w@mail.gmail.com>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>, Wei Chen
 <Wei.Chen@arm.com>, Paul Durrant <paul@xen.org>, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org,
 Stefano Stabellini <sstabellini@kernel.org>, Jan Kiszka
 <jan.kiszka@siemens.com>, Arnd Bergmann <arnd.bergmann@linaro.org>,
 Juergen Gross <jgross@suse.com>, Julien Grall <julien@xen.org>, Carl van
 Schaik <cvanscha@qti.qualcomm.com>, Bertrand Marquis
 <Bertrand.Marquis@arm.com>, Stefan Hajnoczi <stefanha@redhat.com>, Artem
 Mygaiev <Artem_Mygaiev@epam.com>, Xen-devel <xen-devel@lists.xen.org>,
 Oleksandr Tyshchenko <olekstysh@gmail.com>, Oleksandr Tyshchenko
 <Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>,
 James McKenzie <james@bromium.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, Rich Persaud <persaur@gmail.com>, Daniel
 Smith <dpsmith@apertussolutions.com>, Jason Andryuk <jandryuk@gmail.com>,
 eric chanudet <eric.chanudet@gmail.com>, Roger Pau =?utf-8?Q?Monn=C3=A9?=
 <roger.pau@citrix.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
Date: Fri, 10 Sep 2021 10:35:52 +0100
In-reply-to: <CACMJ4GaJyAnguzAEH87DSNN_+GhEa5jRbw11hVj-yWMAXx8V7w@mail.gmail.com>
Message-ID: <87o890wyqe.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Christopher Clark <christopher.w.clark@gmail.com> writes:

> On Sun, Sep 5, 2021 at 7:24 PM AKASHI Takahiro via Stratos-dev <stratos-d=
ev@op-lists.linaro.org> wrote:
>
>  Alex,
>
>  On Fri, Sep 03, 2021 at 10:28:06AM +0100, Alex Benn??e wrote:
<snip>
>
>  In configuration phase of virtio device, the latency won't be a big matt=
er.
>  In device operations (i.e. read/write to block devices), if we can
>  resolve 'mmap' issue, as Oleksandr is proposing right now, the only issu=
e is
>  how efficiently we can deliver notification to the opposite side. Right?
>  And this is a very common problem whatever approach we would take.
>
>  Anyhow, if we do care the latency in my approach, most of virtio-proxy-
>  related code can be re-implemented just as a stub (or shim?) library
>  since the protocols are defined as RPCs.
>  In this case, however, we would lose the benefit of providing "single bi=
nary"
>  BE.
>  (I know this is is an arguable requirement, though.)

The proposal for a single binary would always require something to shim
between hypervisors. This is still an area of discussion though. Having
a compile time selectable approach is practically unavoidable for "bare
metal" backends though because there are no other processes/layers that
communication with the hypervisor can be delegated to.

>
>  # Would we better discuss what "hypervisor-agnosticism" means?
>
> Is there a call that you could recommend that we join to discuss this and=
 the topics of this thread?
> There is definitely interest in pursuing a new interface for Argo that ca=
n be implemented in other hypervisors and enable guest binary
> portability between them, at least on the same hardware architecture,
> with VirtIO transport as a primary use case.

There is indeed ;-)

We have a regular open call every two week for the Stratos project which
you are welcome to attend. You can find the details on the project
overview page:

  https://linaro.atlassian.net/wiki/spaces/STR/overview

we regularly have teams from outside the project present their work as well.

> The notes from the Xen Summit Design Session on VirtIO Cross-Project BoF =
for Xen and Guest OS, which include context about the
> several separate approaches to VirtIO on Xen, have now been posted here:
> https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00472.html

Thanks for the link - looks like a very detailed summary.

>
> Christopher
>
>=20=20
>  -Takahiro Akashi


--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:06:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:06:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184208.332769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdQV-0002iO-Vy; Fri, 10 Sep 2021 10:06:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184208.332769; Fri, 10 Sep 2021 10:06:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdQV-0002iH-T5; Fri, 10 Sep 2021 10:06:31 +0000
Received: by outflank-mailman (input) for mailman id 184208;
 Fri, 10 Sep 2021 10:06:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOdQU-0002i8-7Z
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:06:30 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c3ce0270-121e-11ec-b221-12813bfff9fa;
 Fri, 10 Sep 2021 10:06:29 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631268383894215.78289560400754;
 Fri, 10 Sep 2021 03:06:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3ce0270-121e-11ec-b221-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631268387; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=BGmHo5Akbbsj/xfbtRFimf0JZCjDHbYhYHKHPW0NOKtl/fzRdxmefXas4bV1XI8GCuh+jE8Jx7pQwRqAe8qIOjc2QpCYFGXRP9//5attJzHVt3eP2SbBC8SQzo45u+u7QcoHHaOwkM1wm5bu2OmWTui7lIe6i0AoNnLHrttjDRI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631268387; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=QEjZX0zuy3A6qMLNdFILZu10UrGIeNde2vBbeM+kN9o=; 
	b=QApNXB/zxGbP30KURDmIMh0dnCnqYG3EJVv87Ka8utwZ02vfTqgSkh5+BIpRfQIEC03eud4W/WAd99DrUcjkwUzeUgYaqrxGYcT78yki9JRBbgWLAqX+mcSxkTcmx0zZvaziZpwHzH+JYCNe1SbsAOY3kUSFy35mzPf6ugp73KA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631268387;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=QEjZX0zuy3A6qMLNdFILZu10UrGIeNde2vBbeM+kN9o=;
	b=aaDOgMpkKC1cnwT7Goy6XkFManh+q5T+kf91QZ4Z1oDm65DGs3wYwAbCyWeOAT88
	QnWfTej3vN/NqoIcn593E/qr3synyGj673E7i5OHN7ecsH2PwX/JUHW65ueRkgMSrL/
	TPVtNVsfVstCJKJROhZCTK/kaeT9mwpIpQc3rX34=
Subject: Re: [PATCH v5 05/10] xsm: apply coding style
To: Jan Beulich <jbeulich@suse.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-6-dpsmith@apertussolutions.com>
 <ea0a2e43-6c8c-bfcd-34aa-ec181600ea2d@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <dd15228e-529e-7de6-856a-964d469ef658@apertussolutions.com>
Date: Fri, 10 Sep 2021 06:04:39 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <ea0a2e43-6c8c-bfcd-34aa-ec181600ea2d@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External

On 9/10/21 2:15 AM, Jan Beulich wrote:
> On 10.09.2021 04:12, Daniel P. Smith wrote:
>> @@ -70,7 +73,7 @@ void __xsm_action_mismatch_detected(void);
>>   #endif /* CONFIG_XSM */
>>   
>>   static always_inline int xsm_default_action(
>> -    xsm_default_t action, struct domain *src, struct domain *target)
>> +	xsm_default_t action, struct domain *src, struct domain *target)
> 
> Here and below you're now introducing hard tabs. In Xen style
> we don't ever use hard tabs for indentation; this is limited to
> files inherited from elsewhere.
> 
> It's also not clear why you've touched this instance at all:
> The 4 chars indentation was correct already, as previously
> pointed out by Andrew (on perhaps a different example).
> 
> Jan
> 

I honestly don't know what is going on. Since right now my primary focus 
is on Xen, I have my vimrc set to 4 space tabs with expand tabs and I am 
not manually changing. I will retab the file in the next submission.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:14:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184217.332780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdXm-0004Wd-TC; Fri, 10 Sep 2021 10:14:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184217.332780; Fri, 10 Sep 2021 10:14:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdXm-0004WW-QK; Fri, 10 Sep 2021 10:14:02 +0000
Received: by outflank-mailman (input) for mailman id 184217;
 Fri, 10 Sep 2021 10:14:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOdXl-0004WQ-6k
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:14:01 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d0afe5e8-121f-11ec-b222-12813bfff9fa;
 Fri, 10 Sep 2021 10:14:00 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631268831562445.8523312716843;
 Fri, 10 Sep 2021 03:13:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0afe5e8-121f-11ec-b222-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631268835; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=NO1dgVFE0PlIzP6RH4i10RLvbMjb8BWwy4SNWUXDmA+/7IWrM4IPg9bWk7LGcFHo0kET5I0Y1mZL9jfS2iC1KbWpqXP13Rn6xsR49v8GdqlK+jxeMB/llC2EX9noacs+bYg9ynM+8SdaqR/Y/npMZZGshDt6NlnoKUt+7QKbS0I=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631268835; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=c6pHtHfkmhfXA1VtjahOIFe4JoDeuAHfh41/EsW2tqk=; 
	b=DyTjr4x3Z8waaymBMMx1DfOJBX5hnIE0y24Jl0B8hNj/iodkFYHgQIDjiTwAHAjWxiEzaBwP5xFkY139Y61hFjUa2p05dQJ7SSowHf2sGOucr+VgTe89gg33HUEhw+9Cj2netG1Oadf+eOgt5+EFYuuIFw68bTIUW+h5xLRIFoA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631268835;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=c6pHtHfkmhfXA1VtjahOIFe4JoDeuAHfh41/EsW2tqk=;
	b=JPO4yAVm8GxI5EKFUOwutSqAGr4Wy2byAMwt9sJlwMRToDfHaP7n/Yc3pdNNTRq7
	thUkosKASCXADv7exf8drYymOCnf6aDdQrQKpaSqxDexRcU4caTpgBUoEmqeecE9ccw
	/NAGZ9Xp8vjPM4VsNIpJZhrBZHrHUh4Uqe4dN3jw=
Subject: Re: [PATCH v5 01/10] xen: Implement xen/alternative-call.h for use in
 common code
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org, Bob Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-2-dpsmith@apertussolutions.com>
 <ac8670c3-a7d2-cc33-5043-22474567ed11@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Message-ID: <61cb35c6-f657-fe14-1258-799184c76145@apertussolutions.com>
Date: Fri, 10 Sep 2021 06:12:06 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <ac8670c3-a7d2-cc33-5043-22474567ed11@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External



On 9/10/21 2:10 AM, Jan Beulich wrote:
> On 10.09.2021 04:12, Daniel P. Smith wrote:
>> From: Andrew Cooper <andrew.cooper3@citrix.com>
>>
>> The alternative call infrastructure is x86-only for now, but the common iommu
>> code has a variant and more common code wants to use the infrastructure.
>>
>> Introduce CONFIG_ALTERNATIVE_CALL and a conditional implemetnation so common
>> code can use the optimisation when available, without requiring all
>> architectures to implement no-op stubs.
>>
>> Write some documentation, which was thus far entirely absent, covering the
>> requirements for an architecture to implement this optimsiation, and how to
>> use the infrastructure in general code.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> 
> I did offer an A-b on v4, dependent upon two small adjustments. Was it
> deliberate to neither make these adjustments nor add the ack? (At that
> point I did offer making the adjustments while committing, but I think
> it goes without saying that upon re-submission the changes should be
> made. Unless of course there's disagreement about making them in the
> first place. Yet such disagreement would be worth making explicit then,
> imo.)
> 
> Jan
> 

Yes you did, perhaps I did not follow the thread correctly. Re-reading 
it you mentioned they could be addressed during committing. Since I am 
going to have to respin this because the hard-tab foul up, I can pick up 
those to adjustments and add your Ack-by.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:14:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:14:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184222.332792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdYS-00054s-7B; Fri, 10 Sep 2021 10:14:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184222.332792; Fri, 10 Sep 2021 10:14:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdYS-00054l-4D; Fri, 10 Sep 2021 10:14:44 +0000
Received: by outflank-mailman (input) for mailman id 184222;
 Fri, 10 Sep 2021 10:14:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/vwH=OA=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mOdYP-00054W-SR
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:14:42 +0000
Received: from out5-smtp.messagingengine.com (unknown [66.111.4.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e948a432-121f-11ec-b222-12813bfff9fa;
 Fri, 10 Sep 2021 10:14:40 +0000 (UTC)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailout.nyi.internal (Postfix) with ESMTP id B458D5C0134;
 Fri, 10 Sep 2021 06:14:40 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute4.internal (MEProxy); Fri, 10 Sep 2021 06:14:40 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 10 Sep 2021 06:14:38 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e948a432-121f-11ec-b222-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=/wNS6+
	or464rDz3sY0LcWdGJtIegU66I3vn7RxITel4=; b=ixc+is5ewLeMt2ZWDhbj5L
	LfY9wtMjuhJ5SjEko9OAPNqOeOlC3UirV1YLhye04C6w3G7P07EPSbWk6sargoaR
	ektsl/n6lNBCK7OGAxG28qpTWDU8aPfbfMcKX/xQJOlgn4uy+Q2aT0vh+qHHJld6
	1oUnvpCbtEK2NL3/Q64R+XwssHRECXjXIcaqw3zg9oJy0Dlv6pnl64IQhLbWp+dk
	c36aze02e4H4q9caIlTiFRBp89dI3+wN9KBp9ZMflYJUAF+cVhkA0p5FDBjSZByA
	IU18tkfmEMnEH4GQ9GBb/ELPnomqy7iQsmuGbeke18aB2eahJ0AaDuCb9D3etSHw
	==
X-ME-Sender: <xms:EDA7YUwM28sR5NGGJjXqMSnz_hn3JYUHBtFMjbC94lJl7Nj_8k2eTA>
    <xme:EDA7YYSgVh-JJK3SK8VidWG6aBBC1pw7HLLmmPt9LESE1CNgGC7NLnIPFHiUgj6ZU
    4Wf8LZHHnoE1g>
X-ME-Received: <xmr:EDA7YWVIKZU4XlPPDoryRgVvTVOPSeVyUFJM1w9pDfs52i5IVsr-zl9N5o5Sm9eWAlpoI8T5U6gTCaRpgwVM8hdpr4V3SJJO>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeguddgvdehucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeetveff
    iefghfekhffggeeffffhgeevieektedthfehveeiheeiiedtudegfeetffenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:EDA7YSj9nTHQKF1bDS1CtxKM6t9TLU6eSM29aS1M9dg8424dbutpPw>
    <xmx:EDA7YWB-2l5VUhCyPkEZHHBamW5_0jRPiUmo8iMH5lRmF-wiOeLu5A>
    <xmx:EDA7YTKr-PE2ixBo5wmEQ7vv8tBMV7JiaquyjmJYnsR99TEBRjDLvw>
    <xmx:EDA7YcBTedB7etV_vVbWJQW8gzxfjc2co8Md5IrRLq998BojXTywnA>
Date: Fri, 10 Sep 2021 12:14:35 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jens Axboe <axboe@kernel.dk>, Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v3 2/3] xen/blkfront: don't take local copy of a request
 from the ring page
Message-ID: <YTswC2T2cvsEw2dP@mail-itl>
References: <20210730103854.12681-1-jgross@suse.com>
 <20210730103854.12681-3-jgross@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="bm4wVgWanBEqw53D"
Content-Disposition: inline
In-Reply-To: <20210730103854.12681-3-jgross@suse.com>


--bm4wVgWanBEqw53D
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Fri, 10 Sep 2021 12:14:35 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jens Axboe <axboe@kernel.dk>, Jan Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v3 2/3] xen/blkfront: don't take local copy of a request
 from the ring page

On Fri, Jul 30, 2021 at 12:38:53PM +0200, Juergen Gross wrote:
> In order to avoid a malicious backend being able to influence the local
> copy of a request build the request locally first and then copy it to
> the ring page instead of doing it the other way round as today.
>=20
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> V2:
> - init variable to avoid potential compiler warning (Jan Beulich)
> ---
>  drivers/block/xen-blkfront.c | 25 +++++++++++++++----------
>  1 file changed, 15 insertions(+), 10 deletions(-)
>=20
> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> index 15e840287734..b7301006fb28 100644

(...)

> @@ -827,10 +832,10 @@ static int blkif_queue_rw_req(struct request *req, =
struct blkfront_ring_info *ri
>  	if (setup.segments)
>  		kunmap_atomic(setup.segments);
> =20
> -	/* Keep a private copy so we can reissue requests when recovering. */
> -	rinfo->shadow[id].req =3D *ring_req;
> +	/* Copy request(s) to the ring page. */
> +	*final_ring_req =3D *ring_req;

Is this guaranteed to not be optimized by the compiler in an unsafe way
(like, do the operation the other way around)?
My version of the patch had "wmb()" just before, maybe a good idea to
add it here too?

>  	if (unlikely(require_extra_req))
> -		rinfo->shadow[extra_id].req =3D *extra_ring_req;
> +		*final_extra_ring_req =3D *extra_ring_req;
> =20
>  	if (new_persistent_gnts)
>  		gnttab_free_grant_references(setup.gref_head);
> --=20
> 2.26.2
>=20
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--bm4wVgWanBEqw53D
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmE7MAwACgkQ24/THMrX
1ywDTgf/cDDueolNeVBsdSwc5Iux/HGLE7puVzmumjIhi8y29ZXfN2PQ2sb2B/dj
/cQLAGsch4KXU0IzHIAzzG8RHMLAToSTCisFHArlrpIYkxy1MlmbpboffrxqTohl
GEbW+f3lh+NT2/z5bA/iBVNnbWk4fLX/IdxuohxpsDKC/W1LKihh5HDO1lJAWmCj
uw+Ijvw7ljCAHlQaJVoGjjZv21OOu0bnT5+SCJjceL32UFQhOmBdL5bcGfvpcJRL
lLF27Bw4jMLYwF7uhZVxPthldKLqCz2x4vqCNMQWc17bSAJzXTdSFxE9KeyaYy1l
spoh1nVkPtrC+hrflObZihp/IQPX6g==
=mjV6
-----END PGP SIGNATURE-----

--bm4wVgWanBEqw53D--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:20:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:20:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184228.332803 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOddb-00065F-Rq; Fri, 10 Sep 2021 10:20:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184228.332803; Fri, 10 Sep 2021 10:20:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOddb-00064g-Ni; Fri, 10 Sep 2021 10:20:03 +0000
Received: by outflank-mailman (input) for mailman id 184228;
 Fri, 10 Sep 2021 10:20:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/vwH=OA=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mOdda-0005ty-Ji
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:20:02 +0000
Received: from out5-smtp.messagingengine.com (unknown [66.111.4.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a844ba56-1220-11ec-b223-12813bfff9fa;
 Fri, 10 Sep 2021 10:20:01 +0000 (UTC)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailout.nyi.internal (Postfix) with ESMTP id 183FC5C0151;
 Fri, 10 Sep 2021 06:20:01 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Fri, 10 Sep 2021 06:20:01 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 10 Sep 2021 06:19:59 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a844ba56-1220-11ec-b223-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=NrqFJE
	A7K6DO9LA/zODEzDVYxlIRo+E5W5azI/AS25s=; b=gnCMg5Sa3W3YRGREGMaFBI
	kjYQ38F7khqobc9HZSyAeTpG0otGcxXMaEDKFfOntPPw8u9uq5TUK6f/PVe5uFU/
	vhUgu0VRVgVC0hqcNvU4sAgJMZvx1xwfIA9toHcVz7QFcmxTVqapFu+/uowYJSLl
	7+qhj6GzQ4Ic5197tOIRPKcqUxDlwUEEA0JOA+SzHsKoKD67pV1F88SG4/4RMTa+
	rZE/jGJGBztORg9Vp3s7DDyL+SNIWRmt4P9bG/+CfjZpLMGBMaZdpgeu7TlQtjRF
	w+72l5LepDSEkt7Jx4sjlJjfNpMx1ib2DMLuIdA63I/tbiQZEjoP/g3Vr3FgvSqA
	==
X-ME-Sender: <xms:UDE7YQ_psUErcvCdfEma6qtxyiDa_x-tzPPBMWdIhCeHGMjQKmX7Fg>
    <xme:UDE7YYvR7fOyf3bAkoYQq2G6EPSTJimrGxTtBEX7OQJzdwpax9enI_v84rWiy7lZz
    0Ebdqr5e69jBg>
X-ME-Received: <xmr:UDE7YWByCTTyEz0wEH7cUKTgig4vEtRj5NOvrFUcZ6p0sUbHH-zIJBrsWtX5pJ-Rx1vM9ptFnyItnunA-bEa0SKADITQyp8a>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeguddgvdehucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeetveff
    iefghfekhffggeeffffhgeevieektedthfehveeiheeiiedtudegfeetffenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:UDE7YQdfK5rsmuH5zxY9z7O41G-OHzAzlpYQZ7DnGydc_3gRj0kmxg>
    <xmx:UDE7YVOEj4S9hxKU22a3vyunluZgbFQ8r3RfEk0uMADMyw9ol52-pA>
    <xmx:UDE7YamFaNmf9uXKUyOyI8eFCeabYqYfT52kru9E7hukuh-VS6GNfg>
    <xmx:UTE7YRCwN_hsfXUEm5_f4Q1IlQiJ6FpfZY86I-8R_RiG6HK9DjZA4w>
Date: Fri, 10 Sep 2021 12:19:56 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>, xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/4] xen: harden netfront against malicious backends
Message-ID: <YTsxTKNlL1KgMhB3@mail-itl>
References: <20210824102809.26370-1-jgross@suse.com>
 <1f98d97c-1610-6a66-e269-29b2a9e41004@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="a+WymmwsT1dAoNOc"
Content-Disposition: inline
In-Reply-To: <1f98d97c-1610-6a66-e269-29b2a9e41004@suse.com>


--a+WymmwsT1dAoNOc
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Fri, 10 Sep 2021 12:19:56 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>, xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/4] xen: harden netfront against malicious backends

On Tue, Aug 24, 2021 at 05:33:10PM +0200, Jan Beulich wrote:
> On 24.08.2021 12:28, Juergen Gross wrote:
> > It should be mentioned that a similar series has been posted some years
> > ago by Marek Marczykowski-G=C3=B3recki, but this series has not been ap=
plied
> > due to a Xen header not having been available in the Xen git repo at
> > that time. Additionally my series is fixing some more DoS cases.
>=20
> With this, wouldn't it have made sense to Cc Marek, in case he wants to
> check against his own version (which over time may have evolved and
> hence not match the earlier submission anymore)?

I have compared this, and the blkfront series against my patches and
they seem to cover exactly the same set of issues. Besides one comment I
made separately, I think nothing is missing. Thanks!

BTW, shouldn't those those patches land in stable branches too? In some
threat models, I'd qualify them as security fixes.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--a+WymmwsT1dAoNOc
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmE7MU0ACgkQ24/THMrX
1yyPgwf/UhMrTFQDi4aAO939ZYoIqkqGRgzJvXk3uKeBm1e3Cja+W310tHHN9YcV
jArR36lzQkV9/DzUSIE/OTvtxJ4kw/5/iGOaDNTAZhpr7AVU0JTjOhJSD0FINqeh
V2/97cuouBTK69DRBoB7XPhGgeM6qhhkJIKns9alnvsXy3Xjdar7SHg4GjWBWIEV
aSPQwiac1JCoGQC4y0U2Ke1WT81ozbztAZCSoYORTzmwzRNwufuf0n1kgfdt1LK/
2s01e4WJebsYowlqTDYZNPKL0L0qyHj9XMeIHjuBZG2Xa86k/Q5RwEN3rAMtsH6O
L2eIsY98ExjJmqH29Og0/EqbOeoCSg==
=HpA0
-----END PGP SIGNATURE-----

--a+WymmwsT1dAoNOc--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:22:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:22:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184238.332814 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdgP-0007SL-9o; Fri, 10 Sep 2021 10:22:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184238.332814; Fri, 10 Sep 2021 10:22:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdgP-0007SE-6Z; Fri, 10 Sep 2021 10:22:57 +0000
Received: by outflank-mailman (input) for mailman id 184238;
 Fri, 10 Sep 2021 10:22:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOdgN-0007S8-Cj
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:22:55 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.79]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3ce8180-b45a-49c0-8710-afcc2e1f8bd0;
 Fri, 10 Sep 2021 10:22:52 +0000 (UTC)
Received: from AS8PR04CA0085.eurprd04.prod.outlook.com (2603:10a6:20b:313::30)
 by AM9PR08MB6787.eurprd08.prod.outlook.com (2603:10a6:20b:2fe::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 10:22:49 +0000
Received: from AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::e8) by AS8PR04CA0085.outlook.office365.com
 (2603:10a6:20b:313::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend
 Transport; Fri, 10 Sep 2021 10:22:49 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT029.mail.protection.outlook.com (10.152.16.150) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 10:22:48 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Fri, 10 Sep 2021 10:22:48 +0000
Received: from 5f710def0c6b.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 FF75DACF-A543-4192-8E30-A52C2B37F692.1; 
 Fri, 10 Sep 2021 10:22:38 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5f710def0c6b.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 10:22:38 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB5142.eurprd08.prod.outlook.com (2603:10a6:20b:d4::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 10:22:37 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 10:22:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3ce8180-b45a-49c0-8710-afcc2e1f8bd0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0IH2HzNoNfuCeV/9Zst8NrF5laXRyY0kXLZMgIrX9IQ=;
 b=L0zomk+/V965H9I60qIVlPItBkbpR4lZGQGq0+iGu+xAytQGJRrog2HgiRGc/968UxKQ71R0DN+MkP93uE9gDtAgDtdC40NklGXdizlxhppeoC3QDxprRHUG+KbWW7FjNQOpJFG8xE7N5LQ0pp1NNuVyKFCJb7HyWX04F+/60Ks=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fc5448fffb42a874
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lp292GZZnetE+LimbD9HvWHr/eAIEx5moK+JG4tT+mmChNGLf0TFD7P/YrSaTpF6Ajg3qB3EiJZxbCdjda7N3Z5w073uT40juOcfNeofXHQwH5aA6Tojzh2lZB+ckMLojPXZEZ7j3RtNcub1Dylyf+7etlGvJVMS+s7YI86YzQd6PJrkzX3mxbb0pYjg9v0uZz9JWlurbgXsHnAYL9hfxtT+ZppTwu2zOD5OIpH0q9Xa86ekcDymQG56miQo+Iu+WSeKFVk4tvIle7u5pPzMxZDy/DtZSZOe26girHUM8R4WwjBas/kWfsbHFh618dlIC98AuZgmYiETy88kyFv9mA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0IH2HzNoNfuCeV/9Zst8NrF5laXRyY0kXLZMgIrX9IQ=;
 b=PseFCo/tp8n89tNyGC1fIR/19NKoywAkMYUmsE3S2MiO0MgyzOIYJIFl1N8s4YBsv/y8LfYr0t7tnUWRk53bMlB1V92yGPTRiy50A9T5rYUw4AX9/7DdbFwsB8f4Bv5653Pkj7AVsy2L3l908I/plrQQO+6auV4JvRf27bAmT7nubMufgjQOfctyD3BSKogzH4fw07MUngj1hY03vqSQb4BqU13bGbmbDkfPU52K7lETyiAnVhZ8Con6HBX2R7aFJRe/EfgZZWC9K7ShI/B9TbbgCOzDCAM+EQ2xzXhN6PbuIUI/9epfBO5s1NFEk1HyF5Uy+3xJxh7AGJJdeykq/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0IH2HzNoNfuCeV/9Zst8NrF5laXRyY0kXLZMgIrX9IQ=;
 b=L0zomk+/V965H9I60qIVlPItBkbpR4lZGQGq0+iGu+xAytQGJRrog2HgiRGc/968UxKQ71R0DN+MkP93uE9gDtAgDtdC40NklGXdizlxhppeoC3QDxprRHUG+KbWW7FjNQOpJFG8xE7N5LQ0pp1NNuVyKFCJb7HyWX04F+/60Ks=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Paul Durrant <paul@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v1 03/14] xen/pci: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Topic: [PATCH v1 03/14] xen/pci: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Index: AQHXlPKNppsGoVMjlkS0HQqYv0RYK6t7+ecAgABMtoCAAAOrgIAeuwQAgAIsWAA=
Date: Fri, 10 Sep 2021 10:22:36 +0000
Message-ID: <4CECF64B-EFAC-43B6-9D47-4C3040C9A286@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <97d39d3ee398d6018bdcaf745f00d039df6a92ef.1629366665.git.rahul.singh@arm.com>
 <5fad114f-06c4-120e-0a48-da7045fb85c5@suse.com>
 <76520AA8-D02C-4207-A722-33D935E76776@arm.com>
 <a8982dae-0928-3230-f10d-ada277bc8397@suse.com>
 <alpine.DEB.2.21.2109081802190.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109081802190.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 5fb3432c-2425-42b0-2864-08d97444f03e
x-ms-traffictypediagnostic: AM6PR08MB5142:|AM9PR08MB6787:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB678757E47643AEBD5AC5F031FCD69@AM9PR08MB6787.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 xW3ULkzwZYk5HIevKRj6ymHp+lB/sDqJ4CJ0mymkSAkbLpCbpQ52Tno9UOKfhPxc8YW1Y5GpFQc+Qi4qnTKdUByCNLWd6KRwDqUuow8pOMGxyAhemkyxoNCB2WfIjiCfiIxienT0e3eau6WBUz/CF1ngLumOpr1p138d/QS8Z61GLLaPYbxgV8h7+MQOcGknNee5Ixo231RFflbSgvV08Upc5tPIXS5M4OohvyGzTuENM2SgHyWk1t2eVfASyJ1mhk2BzTgArZt4MNfEVyz97p+uCqeO/K4OEx3IzUrJY+2DNWHrwn9rusZEwHtXO8fDAdQ1KtQOZQDMRyyT3aU8qgjfkIPNRF8EVG4K2iX+msiWvZ5kYLhwG05cdk6aWVf5VZptnPYKMJ8h/rCk6Rcuwm8qNl+TZDlJRPz2DwqQcuP4IuK5t+n2EjVtutwEMG+0Olnhmdc5ASMmMNkEZDFR+HMYO/cpFAdI+ltKPcTjumfsE/8U0OymiHesBHo7UmLFvW1P4SpykiPgZPTcQvA0XouPql793mYXx4cfNR//qum+i2Ew1MkZ4TeN59FTICJPn6NHxq2mRbunB7pzTChC2NDZnM0Uia6SgYLePZ7XG912+IqKu43Yrc/Py0Lqop0tDERee/Y2+dAVw/W6g4XqShu3MDJDtlxSa8oCEVdjzkMvyqAp2pt0Bk0aNu8WbfKBsE+52GXK69cXatBthD6RjvZQxR6g8EMmcc1NZLZiyw/wox8MA70y4F0cGwqTRP9k
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(136003)(366004)(39850400004)(396003)(5660300002)(38070700005)(316002)(6506007)(53546011)(71200400001)(36756003)(6916009)(54906003)(478600001)(66946007)(33656002)(4326008)(8936002)(76116006)(26005)(2616005)(66556008)(66476007)(6486002)(122000001)(6512007)(38100700002)(83380400001)(186003)(8676002)(66446008)(64756008)(2906002)(86362001)(91956017)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <7D461B4BFA540E4EABFC5669A51CB09F@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5142
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3bf1e85f-d6c0-4d9f-80dc-08d97444e8de
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HQEla8JVcJ/dfdiKImrQaxvXD9jIwNs/VTTs+/1ig3n6mXYxOicL5QyObDJMB1MrYvouEdPmluD0Kp8ssJIcET2Yk5TVXGSWM2SDh77CoxwA5y6UrduPhrQc6lTS6AuKmHptZHa+LbB3BJxEo2Q94k0Z3QTeSOftcbG0Lzbc+oD5BaMxZ+ik+JWm9OXMdpO6vUPQ73mHuog2q7xsVQbxqOTSBcUWz+mheBCGLzAsmpj5bfMeflgo0BjLQ1X0KfZxHffL1isj8r9KgUDO76n+Fgtfk/Hi6kFDSO5m5pRpcmF18NApnHeBp6Op3arzOpLKAA68Q/XZpyay+WvodHJZSxsoMwy0XzSxbOWoJKAJV1iMBHzf9RUpFPzJ18Rj2F7rVhQDBK11A3OCgAsuCgvUYKeqDzaEeZPhfsu6G7RayWnoF4tCgrG4tx8/SchJ2vbZt5yyeVmDtqpji7QZ7cYI4YA0rPRornIueMl5QhMwldVnnszBpLyZ66KU8FpX6qWFhAajr4B0GydxyNruxeyBwxPqrqD5TfhknZtHQnQKPp7icGrgMxVe1t1Z4GpSvogvuaPxlOnegUGCjCMD6wLcxjHLyLyv+O5y5BfQOiPDYSCz64ozcbkpRJOuYczAMM3PUDBGKN+k+kgyrGmhPQ/KGD2Rl+CcDA8O+7rM3IZ636lp6fdE+Y3L4c0+ZhEO0o0h0L043Aj7GV232muJeGubeA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(396003)(39850400004)(376002)(346002)(36840700001)(46966006)(36756003)(8676002)(82310400003)(316002)(6506007)(6862004)(82740400003)(53546011)(70206006)(83380400001)(356005)(4326008)(2616005)(26005)(8936002)(33656002)(70586007)(6512007)(54906003)(86362001)(336012)(186003)(2906002)(478600001)(5660300002)(36860700001)(6486002)(47076005)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 10:22:48.6029
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5fb3432c-2425-42b0-2864-08d97444f03e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6787

SGkgU3RlZmFubywNCg0KPiBPbiA5IFNlcCAyMDIxLCBhdCAyOjExIGFtLCBTdGVmYW5vIFN0YWJl
bGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+IHdyb3RlOg0KPiANCj4gT24gRnJpLCAyMCBB
dWcgMjAyMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+PiBPbiAyMC4wOC4yMDIxIDEzOjQxLCBSYWh1
bCBTaW5naCB3cm90ZToNCj4+PiBIaSBKYW4NCj4+PiANCj4+Pj4gT24gMjAgQXVnIDIwMjEsIGF0
IDg6MDYgYW0sIEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4gd3JvdGU6DQo+Pj4+IA0K
Pj4+PiBPbiAxOS4wOC4yMDIxIDE0OjAyLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+Pj4+IENvbXBp
bGF0aW9uIGVycm9yIGlzIG9ic2VydmVkIHdoZW4gQUNQSSBhbmQgSEFTX1BDSSBpcyBlbmFibGVk
IGZvciBBUk0NCj4+Pj4+IGFyY2hpdGVjdHVyZS4gTW92ZSB0aGUgY29kZSB1bmRlciBDT05GSUdf
WDg2IGZsYWcgdG8gZ2F0ZSB0aGUgY29kZSBmb3INCj4+Pj4+IEFSTS4NCj4+Pj4gDQo+Pj4+IFBs
ZWFzZSBnaXZlIGF0IGxlYXN0IG9uZSBleGFtcGxlIG9mIHdoYXQgaXQgaXMgdGhhdCBmYWlscyB0
byBjb21waWxlLg0KPj4+PiBXaGF0IGFuIGFwcHJvcHJpYXRlIGFjdGlvbiBpcyBkZXBlbmRzIG9u
IHRoZSBuYXR1cmUgb2YgdGhlIGVycm9yKHMpLA0KPj4+PiBhbmQgZnJvbSBsb29raW5nIGF0IHRo
ZSBlbmNsb3NlZCBjb2RlIEkgY2Fubm90IGVhc2lseSBzZWUgd2hhdCBpdA0KPj4+PiBtaWdodCBi
ZSB0aGF0IGJyZWFrcyBmb3IgQXJtLg0KPj4+IA0KPj4+IEkgYW0gb2JzZXJ2aW5nIGJlbG93IGVy
cm9yIHdoZW4gZW5hYmxlZCBBQ1BJICYmIEhBU19QQ0kgZm9yIEFSTS4NCj4+PiANCj4+PiBwcmVs
aW5rLm86IEluIGZ1bmN0aW9uIGBwY2llX2Flcl9nZXRfZmlybXdhcmVfZmlyc3TigJk6DQo+Pj4g
L3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jOjEyNTE6IHVuZGVmaW5lZCByZWZlcmVuY2Ug
dG8gYGFwZWlfaGVzdF9wYXJzZScNCj4+PiBhYXJjaDY0LWxpbnV4LWdudS1sZDogL2hvbWUvcmFo
c2luMDEvd29yay94ZW4vcGNpLXBhc3N0aHJvdWdoLXVwc3RyZWFtL3hlbi94ZW4vLnhlbi1zeW1z
LjA6IGhpZGRlbiBzeW1ib2wgYGFwZWlfaGVzdF9wYXJzZScgaXNuJ3QgZGVmaW5lZA0KPj4+IA0K
Pj4+IEkgZm91bmQgdGhhdCBhcGVpLyBpcyBvbmx5IGVuYWJsZWQgZm9yIHg4NiBhbmQgcGNpZV9h
ZXJfZ2V0X2Zpcm13YXJlX2ZpcnN0KCkgaXMgb25seSBjYWxsZWQgZnJvbSB4ODYgY29kZS4NCj4+
PiBvYmotJChDT05GSUdfWDg2KSArPSBhcGVpLw0KPj4+IA0KPj4+IEkgYW0gbm90IHN1cmUgd2hl
dGhlciB3ZSBuZWVkIHRoaXMgY29kZSBmb3IgQVJNIGFyY2hpdGVjdHVyZSANCj4+PiB0aGF0IGlz
IHdoeSBJIGdhdGUgdGhlIGNvZGUgZm9yIEFSTSB2aWEgQ09ORklHX1g4Ng0KPj4gDQo+PiBTbyB5
b3UgQXJtIGZvbGtzIHdpbGwgcHJvYmFibHkgd2FudCB0byBzZXR0bGUgb24gdGhhdCBhc3BlY3Qg
Zmlyc3QuIFdoYXQNCj4+IGlzIHdhbnRlZCB0byBrZWVwIHRoaW5ncyBidWlsZGluZyBkZXBlbmRz
IG9uIHRoYXQuDQo+IA0KPiBSZWFkaW5nIHRoZSBBUEVJIGRlc2NyaXB0aW9uLCBpdCBsb29rcyBs
aWtlIHRoZXJlIG1pZ2h0IGJlIHNvbWUgdXNlIGZvcg0KPiBpdCBvbiBBUk0gYnV0IGl0IHdvdWxk
IHdvcmsgYSBiaXQgZGlmZmVyZW50bHkgZnJvbSB4ODYgYXMgdGhlcmUgYXJlIG5vDQo+IE5NSXMg
b24gQVJNLiBTbyBlbmFibGluZyBBUEVJIG9uIEFSTSBpcyBub3QganVzdCBhIG1hdHRlciBvZiBl
bmFibGluZw0KPiB0aGUgYnVpbGQgb2YgYXBlaS8sIGl0IGlzIG5vdCBnb2luZyB0byBiZSBzdHJh
aWdodGZvcndhcmQuDQo+IA0KPiBGb3IgdGhlIHNjb3BlIG9mIHRoaXMgc2VyaWVzICh3aGljaCBp
cyBhY3R1YWxseSBhYm91dCBQQ0kpLCBJIHdvdWxkDQo+IGxlYXZlIGl0IGFsb25lLCBhbmQga2Vl
cCBhcGVpLyB4ODYgb25seSwgd2hpY2ggbWVhbnMgI2lmZGVmaW5nDQo+IHBjaWVfYWVyX2dldF9m
aXJtd2FyZV9maXJzdC4NCj4gDQo+IEkgd291bGQganVzdCBhZGQgYW4gaW4tY29kZSBjb21tZW50
IHNheWluZyAiQVBFSSBub3Qgc3VwcG9ydGVkIG9uIEFSTQ0KPiB5ZXTigJ0uDQogDQpPay4gSSB3
aWxsIGFkZCB0aGUgY29tbWVudCBpbiBuZXh0IHZlcnNpb24uDQoNCj4gQW5vdGhlciBvcHRpb24g
d291bGQgYmUgdG8gaW50cm9kdWNlIGEgc3ltYm9sIGxpa2UgSEFTX0FDUElfQVBFSQ0KPiBidXQg
aXQgaXMgYSBiaXQgb3ZlcmtpbGwgZm9yIHRoaXMuDQoNClJlZ2FyZHMsDQpSYWh1bA==


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:33:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:33:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184249.332825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdqF-0000qE-F8; Fri, 10 Sep 2021 10:33:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184249.332825; Fri, 10 Sep 2021 10:33:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdqF-0000q7-B0; Fri, 10 Sep 2021 10:33:07 +0000
Received: by outflank-mailman (input) for mailman id 184249;
 Fri, 10 Sep 2021 10:33:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOdqD-0000q1-Ie
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:33:05 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0a::622])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eacb578a-e1aa-4dc3-a57f-378e6cd2c552;
 Fri, 10 Sep 2021 10:33:01 +0000 (UTC)
Received: from AM0PR08CA0028.eurprd08.prod.outlook.com (2603:10a6:208:d2::41)
 by AS8PR08MB7281.eurprd08.prod.outlook.com (2603:10a6:20b:420::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 10:32:59 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:d2:cafe::76) by AM0PR08CA0028.outlook.office365.com
 (2603:10a6:208:d2::41) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 10:32:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 10:32:59 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Fri, 10 Sep 2021 10:32:59 +0000
Received: from 184721cf214e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BF4399AF-1EF1-4E0F-9CC5-18DFF5057E65.1; 
 Fri, 10 Sep 2021 10:32:38 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 184721cf214e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 10:32:38 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB2998.eurprd08.prod.outlook.com (2603:10a6:209:4e::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Fri, 10 Sep
 2021 10:32:27 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 10:32:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eacb578a-e1aa-4dc3-a57f-378e6cd2c552
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sBm663vIp1pIJh8pt/Vtbd5Q3WrEzvITnnu51uJ0R2Y=;
 b=bTolQo2BASwWRot2vZQvuWUf85AErznDi71zVoaVLpCpBav4BVlhDFtxd/jX9FCGAslD83OJ7n6CcmAN2BrdXTEEvrIG/+OWjk86IoFDCgHPA7BEI7xRXDo/5AP2SmL9/HVNXMVsGVpK4e0bNVei1IkeKMx26lhMd/WLQOLbqD0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5de358e8ed07d05a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FzcVefv7LhQLoi4XY+qc4QQBig07KD6LldaleyvF3RcNyAXRENsGXpV+Oe0PoqP7t1rGKY8k3Rpt1Ay+SS/bL9JCYBtDvZblOnD+yYvkhExd68Atj27sT844xh56t3pgKWUbbGv1GjEhYL1ol6S1FrG8OJDg1AEXKS7GuFu+lRvp0jSYAqTUB2t05RAveoOiMjddg2FG4aa92r8eDCrZxvkj29ZbQWLNZ8a0Xl8sWNrli8nQyl59N96gCxrDL5/PMQCj5347Hitw+OfBrpqwA7a3HjDDE/0XntO7FhUMLR/jTyArgUXwXwNinbjik80wJkEjNmm+OJ5IKVOf1XgLZw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sBm663vIp1pIJh8pt/Vtbd5Q3WrEzvITnnu51uJ0R2Y=;
 b=FPzN66bWqXeR8080MtDtlRb+eAah0zxMUYDbk0OOHrAlzxXMa31MpHpxuTY/w3UB8N0CY75pZwm+G/rgteA01CFPuu+SLoflTIfZPUPtE87+Of5W2xMwBqHNkWkCwlta9H2UxujQhBVWaHwhq/DP529FznlWS9c3+kSlnqYD1kAb3ASWdmp9mW0bRK5E2MtJf3tpfWrNUpqU2vXj7qnpf3ybpYs80k9ojUy9BnCfPnXvwshA273ZQv0H6XQGyjElTcmYmsMaCGQ+6k44GuuNB/nX2O8XKs5fNaFrWVx5oAqgl26qBRMVuD8WtaOA+EJWGVikeCKljwvSIdeHk7B+iA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sBm663vIp1pIJh8pt/Vtbd5Q3WrEzvITnnu51uJ0R2Y=;
 b=bTolQo2BASwWRot2vZQvuWUf85AErznDi71zVoaVLpCpBav4BVlhDFtxd/jX9FCGAslD83OJ7n6CcmAN2BrdXTEEvrIG/+OWjk86IoFDCgHPA7BEI7xRXDo/5AP2SmL9/HVNXMVsGVpK4e0bNVei1IkeKMx26lhMd/WLQOLbqD0=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Topic: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Index: AQHXlPKX2eY2zjLj706Keqt9aql1MKubBqqAgAItu4A=
Date: Fri, 10 Sep 2021 10:32:26 +0000
Message-ID: <0805876C-EBC6-4B84-8D62-EC93EBA98F9D@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109081814120.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109081814120.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 65d62329-0f46-413e-39c7-08d974465c54
x-ms-traffictypediagnostic: AM6PR08MB2998:|AS8PR08MB7281:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB72812496001308B6DD43BFFDFCD69@AS8PR08MB7281.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Pv7y3bN3Y8ts5olFp2ILAQgaWOFj1JwOke2gtnxaAESVhYaaOmLk2S7nTDRzpf0aD39LAd3D6vZNjgNAgPlJ6yd1TWNJKl0H/KlZrHIHYiV193LFAlsJ9DeWOLA7FdtA+4VCwC9cMtdbfCUtNxWbCFdFurgcWbRqJuncM+NfwkRX/XtgxFlxmbbMECUbzvxSPpKPShlnzznO7w1fe8Jtav0ThuL34M1H3M7V2AGUK1KSUc9oT/bm3tPyulRnpuSDirhhTnQC05XVrBG0sbhq83vCusbr0Ps/FvAjoLfPwnUSOorbmnAxC3dxPdUVz3xlZZ62DWRHyOcrIhXpqqUONMX+o+NrjXq+l8V92/pqtsaiR66RRDx1HNXohC10siu0441GhPu1DiceLMbkt5o0VrE4TGHp9lfvOfTADrvPyJgUmIRYNeb1by3BUS5hy6mKymsrb5UZkIVSJ2pnl+E5tV7QO6jOYQzXH4cCGhCrgbcmqOZZjtDeSwqOBG66sRiKxFusm/eFGfou9s1Tdx5rVLMAQuqxavi76cz3ugLJkvG/MtIR9+dL4+o4OTm9t01zQ6HgjbwbQKu3OGk8DTlr6HqBsn/WGYF13imagTiviv/s48OfBOBLfPWKe6S9RiXYSjkw6squ8rvW1WCLO7d1+Yr4MrxMXxqYTsnflzhD6IgbrlggCNJW6lj5nAXdmWemp970bdm8e+8VWvIKpUGzKruCD1y4/V5y+ltiQcJ3HWghnigm4YfyT+QBUqImmzFZCRCC5p0p4VVRxantVV8GLiintxKC37jUpFDV8rfcGcY53iPbOsZKrGMZKNr4OZ81
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39850400004)(366004)(396003)(136003)(376002)(8936002)(26005)(6512007)(316002)(4326008)(478600001)(6486002)(6916009)(2616005)(2906002)(36756003)(8676002)(64756008)(66446008)(122000001)(66556008)(66946007)(53546011)(38100700002)(66476007)(54906003)(5660300002)(38070700005)(33656002)(91956017)(186003)(86362001)(71200400001)(6506007)(76116006)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <C382CDAB7B69AC4EAF33469541DB9A3A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB2998
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7bbf11d3-e1be-4fdd-21e1-08d97446491f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mMnRTV7YhnxzjzIzihaOx5HWfsZqCJUOBToki5CoYAxoaUOMDAzfW9iqFOpSYbJqJw6/LnJwR8lj+zASO4usVd9rbeAiPHMjjB1nWJLgv6amNVBamKP+UawMFmJIaIShVhlkGDxjphFy99W5GV5o6HKDg1Y+hX67BW+u9EpBt2Q5Q6AteL74rD55Hs7zQn0sZK5aXfMYR/dnfAnO2sYOPIHk5ptAOb8MGuS0BnU1PbgOYbhTXrR+trfuDSbTY/P0qKQKJCY7ws9UQsew4hBQMqrlHHtxBmUZKOnfTFm3rWXxePQjmeQMNzjDAehoXFNgN9TegdzN/8vJ1Me0N2sqMGjUpAzH06b3I+snb4iuWU/+4nWwB9AiytqDNEMTf1KNsNs+1m1g/Nj4LCFyHBG94pGWQm/IzY+IMA/LzW9tzUR+LwVcucdOBFdOc4lwWnK76LWvn79AY0mnC7Bi4W1lSTodsxlkQsaf+fRoI3apD1rW1nbGK1I/qj81nbKtoOhp9nVv9yQSNH/RrreeNZ7eeuxYBCsH6UpdIMXMztm/Kgc49rdJP4V+SoOV1ks/EbTe63vuGLa5fjAxabnYWEn9vK+e69J53CUBF+dDnbhUIbBQP5R/0uJtlMHZ/dPeubzQXV5UPznFt14A2umbEqGdVNfWWdR+4XuJGNGJrQYgvyovLVVdUlN4JCgXgkXdLUjhBkDrpPaHfHqw/eagvzWMijim6Ig0xt+enn05J8U0iGAc8sv/B25PRzaKVDF662Jl+1luR2IKyEEoD4ZYHWzBHslyaiM9sZdAXmMfT6b+pLQ=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(36840700001)(46966006)(2616005)(4326008)(356005)(26005)(6862004)(81166007)(6506007)(82740400003)(54906003)(53546011)(2906002)(8936002)(33656002)(8676002)(336012)(186003)(70586007)(86362001)(36860700001)(36756003)(478600001)(5660300002)(47076005)(107886003)(6486002)(82310400003)(70206006)(316002)(6512007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 10:32:59.4613
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 65d62329-0f46-413e-39c7-08d974465c54
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7281

Hi Stefano,

> On 9 Sep 2021, at 2:16 am, Stefano Stabellini <sstabellini@kernel.org> wr=
ote:
>=20
> On Thu, 19 Aug 2021, Rahul Singh wrote:
>> pci_init(..) will be called during xen startup to initialize and probe
>> the PCI host-bridge driver.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/device.h |  1 +
>> 2 files changed, 55 insertions(+)
>>=20
>> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
>> index dc55d23778..d1c9cf997d 100644
>> --- a/xen/arch/arm/pci/pci.c
>> +++ b/xen/arch/arm/pci/pci.c
>> @@ -14,13 +14,67 @@
>>  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>  */
>>=20
>> +#include <xen/acpi.h>
>> +#include <xen/device_tree.h>
>> +#include <xen/errno.h>
>> +#include <xen/init.h>
>> #include <xen/pci.h>
>> +#include <xen/param.h>
>>=20
>> int arch_pci_clean_pirqs(struct domain *d)
>> {
>>     return 0;
>> }
>>=20
>> +static int __init dt_pci_init(void)
>> +{
>> +    struct dt_device_node *np;
>> +    int rc;
>> +
>> +    dt_for_each_device_node(dt_host, np)
>> +    {
>> +        rc =3D device_init(np, DEVICE_PCI, NULL);
>> +        if( !rc )
>> +            continue;
>> +        /*
>> +         * Ignore the following error codes:
>> +         *   - EBADF: Indicate the current is not an pci
>                                                     ^ a
>=20
>> +         *   - ENODEV: The pci device is not present or cannot be used =
by
>> +         *     Xen.
>> +         */
>> +        else if ( rc !=3D -EBADF && rc !=3D -ENODEV )
>> +        {
>> +            printk(XENLOG_ERR "No driver found in XEN or driver init er=
ror.\n");
>> +            return rc;
>> +        }
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +#ifdef CONFIG_ACPI
>> +static void __init acpi_pci_init(void)
>=20
> If the DT init function returns int, then it would make sense for the
> ACPI init function to return int as well?

Ok. I will modify acpi_pci_init(..)  function to return int.

Regards,
Rahul


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:36:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:36:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184255.332835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdtr-0001Su-U0; Fri, 10 Sep 2021 10:36:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184255.332835; Fri, 10 Sep 2021 10:36:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOdtr-0001Sn-Qx; Fri, 10 Sep 2021 10:36:51 +0000
Received: by outflank-mailman (input) for mailman id 184255;
 Fri, 10 Sep 2021 10:36:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOdtp-0001Sh-NP
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:36:49 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 00142df0-1223-11ec-b224-12813bfff9fa;
 Fri, 10 Sep 2021 10:36:48 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 57C2D1FE65;
 Fri, 10 Sep 2021 10:36:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EAC4813D29;
 Fri, 10 Sep 2021 10:36:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id qSszNz41O2FOFwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 10:36:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00142df0-1223-11ec-b224-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631270207; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=H2VnXoHa2A4K95NUzL7qITQw88USxIzmreaA433cCYQ=;
	b=HR548Zw8DIrBJ/igCxh2VQX2odWSM3fMFY/SXo9k5d2wwRft6NEROE0M2SyIgTThv/c/uc
	Kafb6F8o2BYzI1gnT6G9xbSF4sFE2rnnOcVK7hUw+wgnQDRJ9lTkw6WSJm5kKY5UoVBaLI
	I+BHBnDRL6eAh9GcNYYk7x41wJdP2tw=
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, linux-block@vger.kernel.org,
 linux-kernel@vger.kernel.org, Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Jens Axboe <axboe@kernel.dk>,
 Jan Beulich <jbeulich@suse.com>
References: <20210730103854.12681-1-jgross@suse.com>
 <20210730103854.12681-3-jgross@suse.com> <YTswC2T2cvsEw2dP@mail-itl>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 2/3] xen/blkfront: don't take local copy of a request
 from the ring page
Message-ID: <ca6e6e23-6e24-38cb-1d06-b25c83767e6d@suse.com>
Date: Fri, 10 Sep 2021 12:36:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YTswC2T2cvsEw2dP@mail-itl>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="YDiXBiNt21aKFiTvqXl3q9RqqBf0BQSSN"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--YDiXBiNt21aKFiTvqXl3q9RqqBf0BQSSN
Content-Type: multipart/mixed; boundary="JUVTtvF3oNKhDqHwfJSukXexFi2WlmUoF";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>
Cc: xen-devel@lists.xenproject.org, linux-block@vger.kernel.org,
 linux-kernel@vger.kernel.org, Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Jens Axboe <axboe@kernel.dk>,
 Jan Beulich <jbeulich@suse.com>
Message-ID: <ca6e6e23-6e24-38cb-1d06-b25c83767e6d@suse.com>
Subject: Re: [PATCH v3 2/3] xen/blkfront: don't take local copy of a request
 from the ring page
References: <20210730103854.12681-1-jgross@suse.com>
 <20210730103854.12681-3-jgross@suse.com> <YTswC2T2cvsEw2dP@mail-itl>
In-Reply-To: <YTswC2T2cvsEw2dP@mail-itl>

--JUVTtvF3oNKhDqHwfJSukXexFi2WlmUoF
Content-Type: multipart/mixed;
 boundary="------------011F05A4CB9DE931D3CC7582"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------011F05A4CB9DE931D3CC7582
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 12:14, Marek Marczykowski-G=C3=B3recki wrote:
> On Fri, Jul 30, 2021 at 12:38:53PM +0200, Juergen Gross wrote:
>> In order to avoid a malicious backend being able to influence the loca=
l
>> copy of a request build the request locally first and then copy it to
>> the ring page instead of doing it the other way round as today.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>> Acked-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>> ---
>> V2:
>> - init variable to avoid potential compiler warning (Jan Beulich)
>> ---
>>   drivers/block/xen-blkfront.c | 25 +++++++++++++++----------
>>   1 file changed, 15 insertions(+), 10 deletions(-)
>>
>> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront=
=2Ec
>> index 15e840287734..b7301006fb28 100644
>=20
> (...)
>=20
>> @@ -827,10 +832,10 @@ static int blkif_queue_rw_req(struct request *re=
q, struct blkfront_ring_info *ri
>>   	if (setup.segments)
>>   		kunmap_atomic(setup.segments);
>>  =20
>> -	/* Keep a private copy so we can reissue requests when recovering. *=
/
>> -	rinfo->shadow[id].req =3D *ring_req;
>> +	/* Copy request(s) to the ring page. */
>> +	*final_ring_req =3D *ring_req;
>=20
> Is this guaranteed to not be optimized by the compiler in an unsafe way=

> (like, do the operation the other way around)?

I don't think the C standard allows that. AFAIK reordering writes is
allowed only between sequence points. And each external function call is
a sequence point, making such an optimization in our case illegal.


Juergen

--------------011F05A4CB9DE931D3CC7582
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------011F05A4CB9DE931D3CC7582--

--JUVTtvF3oNKhDqHwfJSukXexFi2WlmUoF--

--YDiXBiNt21aKFiTvqXl3q9RqqBf0BQSSN
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE7NT4FAwAAAAAACgkQsN6d1ii/Ey8t
2gf+N+WivElKoy9i6Whxd4rrrfEgFb+GzHmhiPCJfNEpstG02jsb0k43eda74wAcdEVpHl7EIG7i
8gBbJwSxH5ZAm295X5wMia00fFtlyppP5uinCToZsE/sbGx2nOCako4HrVNfCzP16R6tXclaY9ho
0dvWYDCF/owSYAzT5OgDkSxVRRqLbZz1CnjFtVRDgp/UXCPJnehCMZ1nxBh6HNBBLPgrE6YldLqN
HzkEqs4n3CBFGOD7BzZ25UbLS7PH+JGDWaEWk1ZLFZFm1unwxM5qn42FQf0f0IvrAGAS3WNiq7Nh
Al23xfAoUu3Nulw+cTq3M2avOvoC5C5/y4XeC8aGSg==
=Dr55
-----END PGP SIGNATURE-----

--YDiXBiNt21aKFiTvqXl3q9RqqBf0BQSSN--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:43:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184263.332847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOe0D-0003D0-L2; Fri, 10 Sep 2021 10:43:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184263.332847; Fri, 10 Sep 2021 10:43:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOe0D-0003Ct-Hm; Fri, 10 Sep 2021 10:43:25 +0000
Received: by outflank-mailman (input) for mailman id 184263;
 Fri, 10 Sep 2021 10:43:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOe0C-0003Cn-9i
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:43:24 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eb1eff97-1223-11ec-b226-12813bfff9fa;
 Fri, 10 Sep 2021 10:43:23 +0000 (UTC)
Received: from [10.10.1.146] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631270597614243.9414157261756;
 Fri, 10 Sep 2021 03:43:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb1eff97-1223-11ec-b226-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631270601; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=gW9dAaSibMbt9EnnYMXY6pFS0ij/1RlWloEiNteT1C1LQ20ei36c/nCuN1U8xEXR/oRUZrFP0PAgCjBDtrox4PcPBwHIuvanM6QbPJw2oIfSnLG9U0EW90phKyeJZ8Q80BHahsCq2RjxtgafEflgCW78EBLCSyG+UNbblOo88PA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631270601; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=/4QfoESrYxGzqyr1l71wmYw4NxxbWHrPuKAOvJt+O38=; 
	b=Gvk9wCtgbIo6SNt1L23V0qG7u3AXfXyAXDSbL3AjM9amYOobrzEG4TxjrksQI2ph4+wdot5WMjMseO/E3Aebm48eptVYI06UtUQGRyw0QA8qf9UihkutO1tReFmRQiNN3vBt0Q8ZFszLCuIaVO3PafK8u9UNepFImgKTdh/5jvE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631270601;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Subject:From:To:Cc:References:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=/4QfoESrYxGzqyr1l71wmYw4NxxbWHrPuKAOvJt+O38=;
	b=RbToVONKK7G+ay5iiRUFIiwsH2iNtWHxxYgyFU8K5PGnaF9ZXbUpape6vLuonIIY
	ElYS8PlVNEZPHWmY7YSRklYPAZHrAK1Vk7FVUuORe54qLckXG9t3v6FAnzcdJZyLtQz
	B7+bVK0b+zU4OWUMjkhTchaR6cTfTtjjgpRUEVFY=
Subject: Re: [PATCH v5 05/10] xsm: apply coding style
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910021236.15082-1-dpsmith@apertussolutions.com>
 <20210910021236.15082-6-dpsmith@apertussolutions.com>
 <ea0a2e43-6c8c-bfcd-34aa-ec181600ea2d@suse.com>
 <dd15228e-529e-7de6-856a-964d469ef658@apertussolutions.com>
Message-ID: <e6246989-87d8-af25-2430-783f0bd19f90@apertussolutions.com>
Date: Fri, 10 Sep 2021 06:41:33 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <dd15228e-529e-7de6-856a-964d469ef658@apertussolutions.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External



On 9/10/21 6:04 AM, Daniel P. Smith wrote:
> On 9/10/21 2:15 AM, Jan Beulich wrote:
>> On 10.09.2021 04:12, Daniel P. Smith wrote:
>>> @@ -70,7 +73,7 @@ void __xsm_action_mismatch_detected(void);
>>>   #endif /* CONFIG_XSM */
>>>   static always_inline int xsm_default_action(
>>> -    xsm_default_t action, struct domain *src, struct domain *target)
>>> +    xsm_default_t action, struct domain *src, struct domain *target)
>>
>> Here and below you're now introducing hard tabs. In Xen style
>> we don't ever use hard tabs for indentation; this is limited to
>> files inherited from elsewhere.
>>
>> It's also not clear why you've touched this instance at all:
>> The 4 chars indentation was correct already, as previously
>> pointed out by Andrew (on perhaps a different example).
>>
>> Jan
>>
> 
> I honestly don't know what is going on. Since right now my primary focus 
> is on Xen, I have my vimrc set to 4 space tabs with expand tabs and I am 
> not manually changing. I will retab the file in the next submission.

URG, going back to correct it and I now know how it happened. My regex 
substitution to assist in the mass reformating injected a tab that I 
expected expandtab to have expanded for me. I should have known better 
and double checked. My sincere apologies for causing unnecessary noise.

v/r,
dps


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 10:47:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 10:47:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184269.332858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOe4C-0003rz-5p; Fri, 10 Sep 2021 10:47:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184269.332858; Fri, 10 Sep 2021 10:47:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOe4C-0003rs-2m; Fri, 10 Sep 2021 10:47:32 +0000
Received: by outflank-mailman (input) for mailman id 184269;
 Fri, 10 Sep 2021 10:47:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOe4A-0003rm-KX
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 10:47:30 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.85]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7b9f855e-89d1-4d69-8f28-02eeb98c7998;
 Fri, 10 Sep 2021 10:47:28 +0000 (UTC)
Received: from AM7PR02CA0028.eurprd02.prod.outlook.com (2603:10a6:20b:100::38)
 by VE1PR08MB5709.eurprd08.prod.outlook.com (2603:10a6:800:1a7::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 10:47:17 +0000
Received: from AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:100:cafe::6d) by AM7PR02CA0028.outlook.office365.com
 (2603:10a6:20b:100::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 10:47:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT039.mail.protection.outlook.com (10.152.17.185) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 10:47:16 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Fri, 10 Sep 2021 10:47:16 +0000
Received: from 354c5663a417.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8D2C8A72-DFA3-4892-8622-3A8C51F9DC1A.1; 
 Fri, 10 Sep 2021 10:47:09 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 354c5663a417.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 10:47:09 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6390.eurprd08.prod.outlook.com (2603:10a6:20b:31b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 10:47:07 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 10:47:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b9f855e-89d1-4d69-8f28-02eeb98c7998
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=q3T5QrqH9Wr/xr6+l3AdimKefw6fTFiBu4kjW1CQhac=;
 b=3v++o1eSxQ8b2jdIJQZfgT+RTXsruf4FfXp8i/K0w8ZFbMZOHvMbXem8tDGULiMNw3cEV0sGaGIXjpS6/6BDBo1FA95jFtSyvTO1S6k+3wV71XUril2OngyWS75OQIkA0n2zMh6Ux0cJiEUp2ivS3zcKCx3VpmwRm7E+HFC9L20=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cdf5c3c594e18989
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IJAjMGGrD6WCvoK6/BPb8b9aiaupusvuqRbHqyll2ZR1YqoZ5EdrvmO/0rjQQeRDOXpOkmGF7cV/TpYr6tH7nlSgUTR3dOCcVo5y60YbqG/sEifZdszkv2NQ834BmoGD7FAYAV0qL2IDF+cWTxnuX5OfqxIIJr7W/AfrsZXiMQiH2C/0CbthvezvX2mY7JeLE0a9oAUXveyk1zMAwuz2iY9MJchcyV6ZMQTD3hPTgX6ltzDSP1Itpb38h5d/C9eOIYQRop6mX9fORoF9Eear2JZrLgu5lXJ8wtBT1wSeTbtNdXLGEUrJvcB8fOQ2GO97qsoVvnoKmFAziVuWXJpFmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=q3T5QrqH9Wr/xr6+l3AdimKefw6fTFiBu4kjW1CQhac=;
 b=cHHnqmISqrVadKZ03sCJqejBra6kzCbeZVCebWZBj8vsM1I5D7Ecro49rzDOZScWR2VodYZZ0bS3RMCHIpSim8pXil45/ib+T1VpWhby4c5Pvf1xVpNUD4dr+JQ5eR/Peemj9WqIpm1AOViGXrKkSCoaQTSjddlRB3RnXktY6zi4UyB6rbUoMl0MJZfGD3GNGQTrgJMhEyYsg9Mu909BRNqJ8Z9jw2/NThcRiSOnzyk1qDsgDFd9GIDsWfqWzMNBe3Z/zeE7qf+kGKFpktAUuJszJUAhbfSsfrqoJ3v9LxnmHavLfkwPIEgpgm8Y5uIE7L/HAANYWu62P3KBCcPbIA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=q3T5QrqH9Wr/xr6+l3AdimKefw6fTFiBu4kjW1CQhac=;
 b=3v++o1eSxQ8b2jdIJQZfgT+RTXsruf4FfXp8i/K0w8ZFbMZOHvMbXem8tDGULiMNw3cEV0sGaGIXjpS6/6BDBo1FA95jFtSyvTO1S6k+3wV71XUril2OngyWS75OQIkA0n2zMh6Ux0cJiEUp2ivS3zcKCx3VpmwRm7E+HFC9L20=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Topic: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Index: AQHXlPKX2eY2zjLj706Keqt9aql1MKuYWIqAgATf9QA=
Date: Fri, 10 Sep 2021 10:47:06 +0000
Message-ID: <09B3A73D-9ABA-4145-94C6-9F3F621C98A8@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com>
 <9bbf553c-80ff-7a1a-a87e-e2162085caca@xen.org>
In-Reply-To: <9bbf553c-80ff-7a1a-a87e-e2162085caca@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 0372bed3-2d86-40a9-97d8-08d974485b4e
x-ms-traffictypediagnostic: AS8PR08MB6390:|VE1PR08MB5709:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB5709B3D5189D940980387449FCD69@VE1PR08MB5709.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:6430;OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 XT+NYJpCqmd17FXLHCYWUtcUNEpthTgyIERLUzx8ldmuexnpFgd3JRo0/DWXm7NHyXWhecTbRfZUGFIhKTEfVa3hrP4V+SlEitGMKY9jnUPrKyMtLKywAn3dGC4kdp30PIsE7gt7OHSmRk4CdonUM+z+O0NApsjfV7Y0hOGzxGAJIHK+EfbHVOYFRoDGE1JQui3et4oogK7rMfJPmGKXLEKHUPLt+n3CbGUHthoZux6vEF/7xMwOK27MrXf1ZyRlZtRjJoiYUnXjmYY05woB+PNa6B2zS7qYQMKLJvvqIrvfPvIimBFRa+rtgvy+4jTEYhHOUyrD11967HjAqohsA6EX60UFdKPO8eSt4ZIfK08F57aDZ3pvrNOp+MsI9oFNpsOlBYaAbRt/jAsgUPk3F9nP+2nKBOb+HK3PZq1nP2uxT8OGEVWtFDzL2BsScIrsfaXahAOk+cfxmCBH8jph0yJQdOUhYasGdM7U4b0sM89xNPaNDv648QQzbCrTETrljT69IcNSesX6Ecyy1nxn1C2goqW0l9LPQEmFNSMvY5MWdDVyqD0Mj/QdayF0UxG34n4vKvYIjR7o6T34cjosjgHlowiD+kCSZHhmcijs/9H6TO6blgLD59rGxLGYb+vdwwtIhWZOp3rC+Gaaaz6MCC6wLJV0GxCNjTP+1h3b7SVwaAViApKFI9bEVt8gXT9BTQ1oQy1KOGUm6yIlDFcDYXeOgtbOE+IVKKVzA9BcNErbHEa/P/+urswal/uKvHAYiVvKgrJ2yi7IsO+ClNTeFsNdT/mt3F80v5lNV8RD+sqEiQJPxNW3c9Zmf52t2o3V
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(376002)(396003)(39850400004)(136003)(36756003)(5660300002)(64756008)(6486002)(478600001)(6512007)(54906003)(33656002)(122000001)(2616005)(38100700002)(86362001)(6916009)(186003)(4326008)(8676002)(38070700005)(316002)(66946007)(76116006)(91956017)(71200400001)(6506007)(8936002)(53546011)(66446008)(83380400001)(26005)(66556008)(2906002)(66476007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <AC18C9912B01C0448F98527967C2EF38@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6390
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d3c7bb96-5051-4bf3-add5-08d974485594
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QcNF6HM7Kxw4WHesrx4KEoO3A5xgca8eZabpEor5UTu1kd20gXKBLPOReW+9rynvgkjKUpKsbwdV/LYXZ0i50cHjAlmU1+frLdtEeHnCm9A8JktmY+BCc7scwZEbW2pdmXDi9dZJ94B7/mrtxerbBdsX8MK7TCz1sj34GjKlYV/6c41jdYgjBuTk8s7LZToFfNpy88duDwzoZOanep708axW+ehLMXFe77GtRmdEQ9Y02VUKSS15BnHgKFRFNw0Bky9SAeDifDqqF1aNuc8Dt7qVmDJ/KGupeR8ePoPQWe77ZLP0diJ7U/gzaxr7nnFtaMTrTgsqg5jfXrb/1ZnmXg34rCViFCe1vhLm8tQY0gnW2SSuSTofUaXpFpCkSNJA4tj44bMSwD60c2fMVsbwQjjmyQO42xsCdDZBlqYFJEnZTRGBD6EMc8YDxbtx5rbooFgxhu253mOjhf5zfTPU7gaHtc2VHCdMmoanGC1j2lmBZpFOK3pAW/5p6Hd1WiYi5ntYilY7XwSXliJenKV1yZPzMS/+CS6D4BAtvBkXBAl+TTfmPQeM4ivRvcv9wfJrhP9qXeRBcab0KVUX9/LxYj4f+cLnMFMl9l5TA+X4S0WbRqkzSDrUxjnvGXGCUJQ06Ph/1Vadvt0zilpsKZxKU3xM29+fQYveUQyD/gqsSL68bD6B4OBIYkvURF2wS9vvYolgncVSsCgWxFxkKJeOxTZd16qXQyIlhCs+wzFILw4yv0HB9xIN6cz75NVZrZbzEWKpx1yBaRZEMX1nTIYXf92Qw8c7ywmzKJ7kh5ssPXI=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(376002)(136003)(396003)(39860400002)(46966006)(36840700001)(36860700001)(5660300002)(70206006)(36756003)(8676002)(6506007)(53546011)(186003)(316002)(47076005)(83380400001)(33656002)(2616005)(70586007)(26005)(4326008)(82310400003)(336012)(2906002)(86362001)(478600001)(6486002)(82740400003)(81166007)(6512007)(107886003)(8936002)(356005)(6862004)(54906003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 10:47:16.7170
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0372bed3-2d86-40a9-97d8-08d974485b4e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5709

Hi Julien,

> On 7 Sep 2021, at 9:20 am, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Rahul,
>=20
> On 19/08/2021 13:02, Rahul Singh wrote:
>> pci_init(..) will be called during xen startup to initialize and probe
>> the PCI host-bridge driver.
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>>  xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>>  xen/include/asm-arm/device.h |  1 +
>>  2 files changed, 55 insertions(+)
>> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
>> index dc55d23778..d1c9cf997d 100644
>> --- a/xen/arch/arm/pci/pci.c
>> +++ b/xen/arch/arm/pci/pci.c
>> @@ -14,13 +14,67 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>=
.
>>   */
>>  +#include <xen/acpi.h>
>> +#include <xen/device_tree.h>
>> +#include <xen/errno.h>
>> +#include <xen/init.h>
>>  #include <xen/pci.h>
>> +#include <xen/param.h>
>=20
> This include doesn't look to be necessary (yet?).
=20
Yes you are right this is not necessary .I will remove "param.h"
>=20
>>    int arch_pci_clean_pirqs(struct domain *d)
>>  {
>>      return 0;
>>  }
>>  +static int __init dt_pci_init(void)
>> +{
>> +    struct dt_device_node *np;
>> +    int rc;
>> +
>> +    dt_for_each_device_node(dt_host, np)
>> +    {
>> +        rc =3D device_init(np, DEVICE_PCI, NULL);
>> +        if( !rc )
>> +            continue;
>> +        /*
>> +         * Ignore the following error codes:
>> +         *   - EBADF: Indicate the current is not an pci
>> +         *   - ENODEV: The pci device is not present or cannot be used =
by
>> +         *     Xen.
>> +         */
>> +        else if ( rc !=3D -EBADF && rc !=3D -ENODEV )
>> +        {
>> +            printk(XENLOG_ERR "No driver found in XEN or driver init er=
ror.\n");
>> +            return rc;
>> +        }
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +#ifdef CONFIG_ACPI
>> +static void __init acpi_pci_init(void)
>> +{
>> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
>> +    return;
>> +}
>> +#else
>> +static inline void __init acpi_pci_init(void) { }
>> +#endif
>> +
>> +static int __init pci_init(void)
>> +{
>> +    if ( acpi_disabled )
>> +        dt_pci_init();
>> +    else
>> +        acpi_pci_init();
>> +
>> +    pci_segments_init();
>=20
> Shouldn't this happen before the PCI initialization?

Calling pci_segment_init(..) before or after PCI initialization will not ma=
ke any=20
difference as this is independent call. Anyway I will move the pci_segment_=
init(..) =20
before PCI initialization.

Regards,
Rahul=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:08:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:08:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184278.332869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeOQ-0006sP-1Y; Fri, 10 Sep 2021 11:08:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184278.332869; Fri, 10 Sep 2021 11:08:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeOP-0006sI-Sp; Fri, 10 Sep 2021 11:08:25 +0000
Received: by outflank-mailman (input) for mailman id 184278;
 Fri, 10 Sep 2021 11:08:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOeOO-0006sC-Cp
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:08:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e22acd82-c857-4bed-9850-3d65dd72e900;
 Fri, 10 Sep 2021 11:08:23 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-WcJUBCybPsWRGO0-TsH9WQ-1; Fri, 10 Sep 2021 13:08:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4381.eurprd04.prod.outlook.com (2603:10a6:803:6d::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Fri, 10 Sep
 2021 11:08:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 11:08:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0176.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 11:08:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e22acd82-c857-4bed-9850-3d65dd72e900
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631272102;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=U6PaK6ivapCMzL9OJchzgrN1fvV7qgqf9f/WPfBEGH0=;
	b=AM5aj5Nyvh/GP9AfbpXfz2VK983abdbzlN0NNia0yPttvZmFWVokOnnhhfKkezb/FOIo8b
	+NU8ShaLzXnU/4C9ifqR+D0D3Pj1pe/Q4V5+dXQVDZ7KvckS64AAGekISxowj9UvniDuAm
	YqI6aBGQv2k6fL7XJ44e8d5YZlqzDjk=
X-MC-Unique: WcJUBCybPsWRGO0-TsH9WQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CsBb6ve44klN8xjX5VmDG/WLTZSiQutOlvwYkjdHLMV2Bdggg+5k4Zb7MHnh0sj5sec7VijkXNyUp5lp/pl+L4R15nO717p0xsg1HCEm90IUr2mi7KuuCHxWnDso7nowHUA3oLw7/+OjPT6McjTBukP3MgQ9Yo0jKDmYd8Z75yyK9EYZESYD2gyeAykpLKO9/rLgYdbwtQJpKanixUQK6Etr7xmH2MiizDEv4Rxku/VjDwJlwx3Eb1WvqxYRftNpNbZe8jdWy2QV9SjSp4U8b8gorA2h1hu5H6+jflHUwqfxT0NKaSDURVswBKVD4U51g4kKwuluWTQwpz5Q2O+7kg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=U6PaK6ivapCMzL9OJchzgrN1fvV7qgqf9f/WPfBEGH0=;
 b=GZocYwOQqKX75sI8g9iC0i2RgsxxbY0SnvlK3WQUYndhXWpuBrN5RwNU27cN3FUjEEkjm1yToAkK0vInf7WN3BwP2RaU0v9bc5VReDWz3VpUKyb3yd+fQGkzAV3D91tZnNUqw1bePjsTdRJO/hsCcr2GFiGcuRd8G7xpDwx6CklLadZlqLrWNSijadyneR7bp9q8aw2BJz23LtlL9amq2sFVxhA+2dJSSjsemjrnVBPrGGMm64CDvsQdZTdsNiOMbcb58q58CmDNt5HTpOVX6JF4mfoK0p0MN5IQRpjZW2Djn2CelgNM4lgyxebDY7ec2WZ+7CHToO2qzymWlDZqGw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
To: Penny Zheng <penny.zheng@arm.com>
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com,
 xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org
References: <20210910025215.1671073-1-penny.zheng@arm.com>
 <20210910025215.1671073-6-penny.zheng@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com>
Date: Fri, 10 Sep 2021 13:08:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910025215.1671073-6-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0176.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1c::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c66232eb-8728-4e6c-ed4c-08d9744b4b82
X-MS-TrafficTypeDiagnostic: VI1PR04MB4381:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43812FBC327A9DE03FCF2D97B3D69@VI1PR04MB4381.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	R1BVq2hMwURoCE2gsHIifSYd4LVVO+lcdlnFTWBtLezkAwUyUwa8/klqpBJ/VdTPxrtncVug+p7GYWc/oGungdjWgeyZH/fVt/Ab2JN/pJEiKftN47ALX24y3fhsD7U+/2zcYlPYxUZAhUIftd5U7z07Ihin7gwVBkxui3wMbG9WkPY3u31qkrdl7Arv6D1MVFrKaA10X+f6L82fjATlmn+SYpAomF4UXqjncRdXjqr4P7ds9WdRE6tUdyi2oD52s1t7NWATRdTDHf+BMX2KpYVDgedXOkYY2qUgkJB337fpvzQV4Q+wpsUxOCNqiPvIn/esWjQgRLvd1GqV8PnBNqevj+buGm1HAUGfv5nJYvdziVc0c2oD6k8BLEuBAey4n2cUumeqhDZlUZ7Qw1CmFII8/Gk3Z4ykhYqI335fYG6UejWG3Yb52DyhhRnVOqtor+XXymZpmCZiNODJB1oygrMovzTQWrahDUP7kc2jla1Vvp5pzf2b0nifhkrqkm9K5nflfREyfYkrA8O/PFUUWW7mYGA2ak/N9pXiEHppjAv4pAej/0kYgxjPhflbbsVXIXA5I+StFhq39dDjZFZU78/5Zebn+rQRQtPxWC/W0kkSuaW511GEUZ+AExe1c4gmWfis/fIfmH4rny4/Lk0NUk2Xkm4RvUC9uzJYOF/80PKs51Efs0CjMFX+uo9sWD8RrAAhrBe7VyFOi7kBWga/pa9MRGnV2N1tGHyJO6PD2S4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(376002)(39860400002)(396003)(136003)(366004)(53546011)(31696002)(31686004)(316002)(2906002)(38100700002)(86362001)(956004)(478600001)(16576012)(2616005)(66946007)(36756003)(83380400001)(6486002)(26005)(66556008)(66476007)(6916009)(4326008)(5660300002)(186003)(8936002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WndMRWFHVFNrTHFNZWQ5MWtpc3pUVk1zV0NwUWV4K1hEeWxGOEpqZW5BNkZq?=
 =?utf-8?B?eWRIbE5WYzNYK3dwTmZXNG4ycy9hTjRQVVlVZFI0UWZ1R2k4cU5Xd1dUaXZP?=
 =?utf-8?B?eWRHRGhHYmhmaktMQmoyYXNPUTdlRG5veitNbGZ1b09iQzZVZUVRWVZoK0gw?=
 =?utf-8?B?ZkIyRUtOcFArQlJHeXB0S0FYRjZONVhaa2JpckxCUHdETStFaHpHWEVHcHFx?=
 =?utf-8?B?NTdlNXpvVnJEZVBkOXpiUDJQSTZNUkF6Zy9vMzU5Tjhqa3NhQXNrS2Jhc1Ft?=
 =?utf-8?B?UnRiY251UjFsSTBnOHB2SWxGL1paZkpVVUt2MXA3dXAwSHhWckNldDI5VUlU?=
 =?utf-8?B?UDVNMUpuNUkyWk9IazI2eTNxVWpkMU02anRFS0x6cWJ4VW5CbVh3S08yd0l6?=
 =?utf-8?B?OEVZQktqYVRKMExLT0FkUURoNTJnTXlqbkZyN2tXUDVHR0dEOXFjREduaVpP?=
 =?utf-8?B?VnVqMEdIZEt2Wm82Q2hYckhuZHFZZ3AxUjEzcndEOUl5bzZKN2kzc0Rwd2FO?=
 =?utf-8?B?NTFWeEZCcXVsREJwUEJUVC9oalVKNGRHcW5ZeVBZTXQ2UjlONExGVllxR3BV?=
 =?utf-8?B?MEdBR25YQXpBTytDSDh3SmtNd2VMdzdwMGJabWFKbXdacTVqV2RRQTdBekFC?=
 =?utf-8?B?QmtyRjJVS0JMYnU4UlF5NWNsUkY2dkZEM3BCRDI0RG53ZFFmTkdxOWlOSXpS?=
 =?utf-8?B?dUdnZHpVN21oVUJab1hwaGxtVEdBYTZjekJWU0tCTnEwc2VSTUVlYkhvSWZx?=
 =?utf-8?B?Q2d1Z082VS9NYlJ2RXRNRE1NdmhWSHM0d05xZVlFUnhLZlBFdWNDeFVZVHpq?=
 =?utf-8?B?UzNNellGaUViajRkRC9MSVliSGs2VmM4NDc3M0daQytLQ2xhNjJQYnR3M3la?=
 =?utf-8?B?QjdLbFB0Y2drT1lwaWZBaUwwM1M3M0hSdGI0cU1SZGZad3A5QUhCa2hEczZE?=
 =?utf-8?B?V3pKMU4rNzUvMGEwT0dqSFBmdkxndW13ZHBNTjY4MTRFMW1md1l0ZlBKRHRq?=
 =?utf-8?B?ZWVSMCthd3FYSUlHcE93MzF4UTVwRGN1aXVGYndEYlZwVVVJM2o4blBTeVFF?=
 =?utf-8?B?TjgzZk9pMGJsNzYveXNCVERyNWtNRktHblR5eFRjYWFISDZ3ZS80N3pyNXlt?=
 =?utf-8?B?L3cxTmRra21BSC9ZcFZVN3A4bDArWVZOVk5DSWRKYXJLYUowN2haSThPcU5O?=
 =?utf-8?B?Q05NeC9mVGlKaU1HeUtOUzF3TW5YcC9lUTZqZGl2Q3RzWmthUkNicHJMWlpq?=
 =?utf-8?B?RmZMNm91V091Ty9yQU9JcDVrT2RyWm5QbkZ4Mzlsai9VdGd6alBad3ZWZGw4?=
 =?utf-8?B?RytKTEZhLzRXbkpoZ05JWElOOCtEMG5KSXB5bkdseXFObExuRHR4VGlEU0FE?=
 =?utf-8?B?QWdtVjEwMjlWL0wxUUZZRFA1SFZzZDFqMGRiR3c4RnB6eGJNdE5vNCsxdUN0?=
 =?utf-8?B?bEtlM1c4RFFLL2NHNzdCZHIra2wwOVFCdHUvSVNsWVhmakJiYU1ENjVNdmpV?=
 =?utf-8?B?T0YyVXhhY2lCVVVOdTYzdE4xRFlub09qc3kxNlR0MlNFUXh6eHhqZmdLQnAr?=
 =?utf-8?B?OXlldHFIM1FlZ09ydGY4VDBnUGZpN205MUE5QUxnWFc4S2FrVjhNWW0xbmFP?=
 =?utf-8?B?dGxOOHlHNnZhRFBQQmtOdE9zdDEweTEydVpySlJFZFJFYTRCM1dERU1hWUFs?=
 =?utf-8?B?WnFhNEc3dWtacVFNVWVRUUJnT1FFdDZ4OTllYjZOd0RwekRBcWJtS2E2NEgr?=
 =?utf-8?Q?/cZmxQXmgIyGh/9j/YEWfiukJ+llzltHd4kBHh8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c66232eb-8728-4e6c-ed4c-08d9744b4b82
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 11:08:18.9095
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hMXPDzckw4CcM+ipQWghN2nPnZKBhvE+lk0hBVnVH4Mq8xMeUT/XR/fuPzNZ5TYojASrexqcWJg8valkk/y3ZQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4381

On 10.09.2021 04:52, Penny Zheng wrote:
> In order to deal with the trouble of count-to-order conversion when page number
> is not in a power-of-two, this commit re-define assign_pages for nr pages and
> assign_page for original page with a single order.
> 
> Backporting confusion could be helped by altering the order of assign_pages
> parameters, such that the compiler would point out that adjustments at call
> sites are needed.

Thanks, this now takes care of my primary concern. However, I (now?) have
another (I thought I would have mentioned this before):

> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
>  
>  
>  int assign_pages(
> -    struct domain *d,
>      struct page_info *pg,
> -    unsigned int order,
> +    unsigned long nr,

If this really is to be "unsigned long" (and not "unsigned int"), then...

> @@ -2281,7 +2281,7 @@ int assign_pages(
>      {
>          unsigned int extra_pages = 0;
>  
> -        for ( i = 0; i < (1ul << order); i++ )
> +        for ( i = 0; i < nr; i++ )

... you will want to (a) show that there is a need for this in the
remaining patches of this series and (b) prove that despite the
remaining patches potentially using this, albeit at boot/init time
only, there isn't any problem from ending up with 4 billion (or
more) iteration loops that would then result. On x86 at least I'm
sure this would be an issue.

Otherwise I would request that in the subsequent patches requests
be suitably broken up, with process_pending_softirqs() invoked
in between. Which would get me back to my feeling that the original
assign_pages() is quite good enough, as your new code would need to
split requests now anyway (and to avoid the need for that splitting
was the primary argument in favor of the change here).

> @@ -2290,18 +2290,18 @@ int assign_pages(
>  
>          ASSERT(!extra_pages ||
>                 ((memflags & MEMF_no_refcount) &&
> -                extra_pages == 1u << order));
> +                extra_pages == nr));
>      }
>  #endif
>  
>      if ( pg[0].count_info & PGC_extra )
>      {
> -        d->extra_pages += 1u << order;
> +        d->extra_pages += nr;
>          memflags &= ~MEMF_no_refcount;
>      }
>      else if ( !(memflags & MEMF_no_refcount) )
>      {
> -        unsigned int tot_pages = domain_tot_pages(d) + (1 << order);
> +        unsigned int tot_pages = domain_tot_pages(d) + nr;
>  
>          if ( unlikely(tot_pages > d->max_pages) )
>          {
> @@ -2313,10 +2313,10 @@ int assign_pages(
>      }
>  
>      if ( !(memflags & MEMF_no_refcount) &&
> -         unlikely(domain_adjust_tot_pages(d, 1 << order) == (1 << order)) )
> +         unlikely(domain_adjust_tot_pages(d, nr) == nr) )
>          get_knownalive_domain(d);
>  
> -    for ( i = 0; i < (1 << order); i++ )
> +    for ( i = 0; i < nr; i++ )
>      {
>          ASSERT(page_get_owner(&pg[i]) == NULL);
>          page_set_owner(&pg[i], d);
> @@ -2331,6 +2331,11 @@ int assign_pages(
>      return rc;
>  }
>  
> +int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
> +                unsigned int memflags)
> +{
> +    return assign_pages(pg, 1UL << order, d, memflags);
> +}
>  
>  struct page_info *alloc_domheap_pages(
>      struct domain *d, unsigned int order, unsigned int memflags)
> @@ -2373,7 +2378,7 @@ struct page_info *alloc_domheap_pages(
>                  pg[i].count_info = PGC_extra;
>              }
>          }
> -        if ( assign_pages(d, pg, order, memflags) )
> +        if ( assign_page(pg, order, d, memflags) )

Note that here, for example, order is necessarily no larger than
MAX_ORDER.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:10:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:10:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184284.332879 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeQI-0008By-Bu; Fri, 10 Sep 2021 11:10:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184284.332879; Fri, 10 Sep 2021 11:10:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeQI-0008Br-8u; Fri, 10 Sep 2021 11:10:22 +0000
Received: by outflank-mailman (input) for mailman id 184284;
 Fri, 10 Sep 2021 11:10:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOeQG-0008Bi-Ds
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:10:20 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 535a20e0-7a4c-48da-a871-e0d683b0176d;
 Fri, 10 Sep 2021 11:10:19 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C0DDC22417;
 Fri, 10 Sep 2021 11:10:18 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 70C8413D34;
 Fri, 10 Sep 2021 11:10:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id X63eGBo9O2F9KAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 11:10:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 535a20e0-7a4c-48da-a871-e0d683b0176d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631272218; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=B85pWNIawjf+ROs1E6I9WKVnZ0AMScExck/UPNJ8eAw=;
	b=vC8AXFAeTRGGst2lp9oJ3Pu6Xvn9CuSnhGjrc03/Skt+/cJvW/w0Mry3ZRPI7tVaIR2jTS
	5UbDgi6lqtnQri4kpDUqFSifj6vl3OChQmCaIw2BGD3vIkzBc/+/hXik9rsxQ/sL8oo2SY
	keZ1cXG8nmZ/88CcLju9QDETzT7B0Vs=
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 "David S. Miller" <davem@davemloft.net>, Jakub Kicinski <kuba@kernel.org>,
 xen-devel@lists.xenproject.org, netdev@vger.kernel.org,
 linux-kernel@vger.kernel.org, Jan Beulich <jbeulich@suse.com>
References: <20210824102809.26370-1-jgross@suse.com>
 <1f98d97c-1610-6a66-e269-29b2a9e41004@suse.com> <YTsxTKNlL1KgMhB3@mail-itl>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 0/4] xen: harden netfront against malicious backends
Message-ID: <f7c2d087-fae9-00b4-c616-0a4183f2fb33@suse.com>
Date: Fri, 10 Sep 2021 13:10:17 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YTsxTKNlL1KgMhB3@mail-itl>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="FjaKuwnjx13J0odcEUplWJ7LScrlhUZmY"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--FjaKuwnjx13J0odcEUplWJ7LScrlhUZmY
Content-Type: multipart/mixed; boundary="o8OMKQYBoVm3U8xmgXWnzMvYO6ZDy3Ay3";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 "David S. Miller" <davem@davemloft.net>, Jakub Kicinski <kuba@kernel.org>,
 xen-devel@lists.xenproject.org, netdev@vger.kernel.org,
 linux-kernel@vger.kernel.org, Jan Beulich <jbeulich@suse.com>
Message-ID: <f7c2d087-fae9-00b4-c616-0a4183f2fb33@suse.com>
Subject: Re: [PATCH v2 0/4] xen: harden netfront against malicious backends
References: <20210824102809.26370-1-jgross@suse.com>
 <1f98d97c-1610-6a66-e269-29b2a9e41004@suse.com> <YTsxTKNlL1KgMhB3@mail-itl>
In-Reply-To: <YTsxTKNlL1KgMhB3@mail-itl>

--o8OMKQYBoVm3U8xmgXWnzMvYO6ZDy3Ay3
Content-Type: multipart/mixed;
 boundary="------------55CE04332F4BD8EF3EA80965"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------55CE04332F4BD8EF3EA80965
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 12:19, Marek Marczykowski-G=C3=B3recki wrote:
> On Tue, Aug 24, 2021 at 05:33:10PM +0200, Jan Beulich wrote:
>> On 24.08.2021 12:28, Juergen Gross wrote:
>>> It should be mentioned that a similar series has been posted some yea=
rs
>>> ago by Marek Marczykowski-G=C3=B3recki, but this series has not been =
applied
>>> due to a Xen header not having been available in the Xen git repo at
>>> that time. Additionally my series is fixing some more DoS cases.
>>
>> With this, wouldn't it have made sense to Cc Marek, in case he wants t=
o
>> check against his own version (which over time may have evolved and
>> hence not match the earlier submission anymore)?
>=20
> I have compared this, and the blkfront series against my patches and
> they seem to cover exactly the same set of issues. Besides one comment =
I
> made separately, I think nothing is missing. Thanks!
>=20
> BTW, shouldn't those those patches land in stable branches too? In some=

> threat models, I'd qualify them as security fixes.

Its on my todo list.

Most stable branches will need backports, so this might require some
more time to get it finished.


Juergen

--------------55CE04332F4BD8EF3EA80965
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------55CE04332F4BD8EF3EA80965--

--o8OMKQYBoVm3U8xmgXWnzMvYO6ZDy3Ay3--

--FjaKuwnjx13J0odcEUplWJ7LScrlhUZmY
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE7PRkFAwAAAAAACgkQsN6d1ii/Ey+0
zAf8Dtct4+rMmXDhRPEQ40F+nWxxZ6QH2JUCITz9c+jobgeamihvzRbI32HcRFtbiNqCtzaikmaG
hd/BS+MexBuqupWl6AAs7ThD1vbvyQK7gLFgnRnHoVbyGZKGi0MtifdY4yOU4L/TJ8Dv/tGzJFCQ
Q51A+CSKLz434A16Jq09qRgDn8Hr2f7QKi+Av39ZxcS3qc2hjvssYjwvnwDBZYRnPoSOcK1KD1Nt
TDRmlCY7HcksW+hfj1hPnc3k+jE9dlOZd05+ZSmpAEfbOT5wv6nApAhqY02NKmqBCWfkmBZxPe00
ZGAbTMQJUrwUgshMn2ImAe4/ZfyjXvYQJHgG5xdGdQ==
=a9tK
-----END PGP SIGNATURE-----

--FjaKuwnjx13J0odcEUplWJ7LScrlhUZmY--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:23:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:23:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184296.332891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeca-0001wY-H7; Fri, 10 Sep 2021 11:23:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184296.332891; Fri, 10 Sep 2021 11:23:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOeca-0001wR-DV; Fri, 10 Sep 2021 11:23:04 +0000
Received: by outflank-mailman (input) for mailman id 184296;
 Fri, 10 Sep 2021 11:23:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOecZ-0001wL-Le
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:23:03 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.40]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 74a47908-1229-11ec-b229-12813bfff9fa;
 Fri, 10 Sep 2021 11:23:00 +0000 (UTC)
Received: from DB6PR0802CA0033.eurprd08.prod.outlook.com (2603:10a6:4:a3::19)
 by AM0PR08MB3027.eurprd08.prod.outlook.com (2603:10a6:208:61::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 11:22:58 +0000
Received: from DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:a3:cafe::1c) by DB6PR0802CA0033.outlook.office365.com
 (2603:10a6:4:a3::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 11:22:58 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT046.mail.protection.outlook.com (10.152.21.230) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 11:22:58 +0000
Received: ("Tessian outbound 8e26f7114b75:v103");
 Fri, 10 Sep 2021 11:22:58 +0000
Received: from af43751d8d24.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 14B8B67F-6D05-42C2-9180-8783C0434E85.1; 
 Fri, 10 Sep 2021 11:22:52 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id af43751d8d24.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 11:22:52 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB3173.eurprd08.prod.outlook.com (2603:10a6:209:4c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 11:22:50 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 11:22:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74a47908-1229-11ec-b229-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v6s7nFJt3PKNFDvDKJ1Wf29p8PEe4jfSAplNlg+iA9w=;
 b=xOQJqy+vbyXzE8pLPZZRSJT6F5PukXTcAYJUOi8DLm8olbvoIKn7PU2T6eZBh0C8KQb7g5+QzMX7qcm2c/Ax1LdHJzEibnkpenCOJ7IuJTf5vn+MiSsGOcasC5rnOZw0YOXC37Xd69nhIrLsGMAKGSHxyuePj5AOb1adFDEX/R4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 308c182b90821283
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=It0f85d2oFl3/SwvNkRRgHwZNafojyO2PR7/2IlaMZsIrMU+1Q5MwhPHW1l+GKZR0Rziopgtv7sZk2Cuk8TPAw9dkqAJq7L/21L+nalPvX92KvsNTCNIGoCIXXOKzjV8yCHhyhQd45Xek/SgEJ8wwjYWhe4THb2wPu9r+jTojFPN33O+T/4Z7Rt7FDpVFwuG4LbADopughQ2lyftrJwYTUp9VoExlJpiguUK2nRlKXL95sLEVgnoMWwlw8IP7pDdjFhfXLlO8dUS+ZCiQIVGe3wHtDFsL8E611937flmCthluf4TR4NJm5ic8U+uUY3lPjBjrpeLkhLYSCRVoSJaBg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=v6s7nFJt3PKNFDvDKJ1Wf29p8PEe4jfSAplNlg+iA9w=;
 b=Y7+Y6sksMPrRhztm0eJ7vTLnd+GvkpKFd3h0rtL4vQd38gt+SEMhPeBTML5m4fXt9aVMakYFZ9YyvIJ3gse28+Fh0Qrx1ZkMaoKnE2i7RjtRQ1pnIfDpiTUyILMV0Vas3xY2trShJshwc8P6dcstLodyYh5bP9s1K0k/mv3hEY02BLQxp0yG8/GyavImjJiyKf7V/b6kvyqH47MJLL4qv1Y3Nt5oESDnqxdghgCkYLDdTFwDwDoJXwuP+rRRSNQBIsrmQonkE3XXs2iBv40c3d137MJltnXMulkPITXByiApsjDe17hxLLha5D6DwI0NGkF9mUd1qeIluP2Y0k/tig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v6s7nFJt3PKNFDvDKJ1Wf29p8PEe4jfSAplNlg+iA9w=;
 b=xOQJqy+vbyXzE8pLPZZRSJT6F5PukXTcAYJUOi8DLm8olbvoIKn7PU2T6eZBh0C8KQb7g5+QzMX7qcm2c/Ax1LdHJzEibnkpenCOJ7IuJTf5vn+MiSsGOcasC5rnOZw0YOXC37Xd69nhIrLsGMAKGSHxyuePj5AOb1adFDEX/R4=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXlPKqrI+7ld7PxEaIC1sXiQa2TauYZQyAgATdboA=
Date: Fri, 10 Sep 2021 11:22:50 +0000
Message-ID: <332992DF-56A9-42CF-B266-B46E4622A03E@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
 <1f4a0284-302e-47e9-6958-47dca449d580@xen.org>
In-Reply-To: <1f4a0284-302e-47e9-6958-47dca449d580@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 7964ce45-361b-461c-1b60-08d9744d57a8
x-ms-traffictypediagnostic: AM6PR08MB3173:|AM0PR08MB3027:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB302783284C11769BE853F46FFCD69@AM0PR08MB3027.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NbM5nKCH019RgJx8sTylLY3nUXQX4s02TH5m2qtE02H3xFq+hAvlve7mZB8ewLqtdNwTS6Kfo1O2aEFLzsXBXpKbcVSC7B8/jABSZFiDrop9cadwiHKywodVlvBFX0QuiAfhHWB92CHPg5HbifQ6qWUdFN/bMYlgPpVj2sHG9+LXDJjjmAQ+nv0NN0JQ2WRy5nHRdMNYkgqOHNC2cO0qOGyQAPclj2khGk18PdeSD1ur62YH4KTS62Xst4YmxEyCVHYpdUovn8k2cTnDl9f9/lVEHhU3zdmjEuGMbWY5TsxPUsyMeTUSEC8RB1L0xDAVXLoxxnMTBDuSVm8dK4R4IqMgApHnqWbAbw9psGOs0Bltma+IXx6auZQkGgF+wo0os7dYHOoHcYT5SSHe1QzFN+kRQtBeTlNrETSpsy0Sxez0cdPZGjUdG/ein1pKZhhB221QHnUVohRHC/FkIjcArylDkXJXl0BclGuOuMJyTGqz7tPGPW0NucUlP8314lpZmpiPDgjl4V54rPn0T6+PkSx9wzBzSNEIwkYblaXpQVRtDRgI6UeE5DaqNTwN8t6i0eYLsQcMLNYuCqCxZ61BpHFiqupaLA9WQA519RMNVWbd5q7S2trKuEkclWt9ClVmuYQoXoQ5rrFsuq3tcCq9SaO0S7wvBzxAZ36hUSI2LrtJRYrPiT3t9s7mXGm9v/ieCa2QQo6/52VQikQV03TgAY0sa0s2fI0JPOnUhvQzLz2oHTLHy5RjK55pOwk822oFLUMJYCjvZgF26yXlh5b9Gl07W2FaxBjQXpgv8J3ljyfUs1knj3QSnJA53uZ53yxq5rqS4ZALBH82/dGHTAgbEK3FNkCs+TmtG5pRY5w1kM7FTQvL8Mr8GfWk0myYL4O8
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39850400004)(122000001)(38100700002)(36756003)(2616005)(33656002)(4326008)(83380400001)(71200400001)(86362001)(6512007)(66946007)(966005)(38070700005)(478600001)(66476007)(186003)(91956017)(2906002)(8676002)(76116006)(30864003)(5660300002)(66446008)(66556008)(64756008)(316002)(6486002)(8936002)(6506007)(53546011)(26005)(6916009)(54906003)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <096CC4472966414C83EE4639CCEB89AB@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3173
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	72b5a20d-4266-4fd2-8370-08d9744d534a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iQEnzjQLwoke3YGI3gkhbXcTPPc/CK5zzGPiPdwpEZnCd2ebq58XDCpsdPOw1tzcWJBeeVj8Bmjh8RHy53bwZaq8/RZwNCN4PVmpiGdKLwWYQPe9d8cfWnyj6Ido4qVAq2uHfI3yKzsBpMnIaOJUTxt4Usje/WvXZv6v5g3ZNCsofSI8re24/Sv0OOcwVmnEHjECpKFHGv2oaJj0dtQvneRkE3aaq6eUbqt9aocKyzF5rjTANcOJ6uAccu1nH+ZyBiNtNrNaQP7hJ5rWH0jNjWJXP+iFNujJOFSJu5b30+Fm2J0vsOqWQKNmnqLxVLXDMp8ElznWZFG5nRHP5AXX+EtkJ732kk3h6qqklIgOpOzwwJwaiF2vqWVzjMEoyoSFBjshhpGFT7z2Rn4lm59UGJJtprxNM5XrTva3iPEdWQ0HeCSgnBecGuO3X9fZPU/0wPbaFTLumYgISfGPALLdgmF9eX+KB59DEWguCccFDfeMzJtvkAVO/GObFWGSFu0o9gkf6FU/m+Ef6mX9jf3g/CbxXIrpnKwO0VfeTrMgrRGWf/5wXr1+ttU6jN9uTosGsgrKSCoNMAr3hRVIy6jHeamqmtu+3pCuGUkGVYXPaA05hgJo0rd75+IlcqXm6B8HWIrnBo0v7Zov5qRWPoOHZJxvfVPP+XgxR8J80fDQIEQVNlivylYmPJ5dN0329agqVCe2PTtc0zUgIzKTfojUXTucAgWKZe1imANDVkvbAq89NbhzFIDub5lCHV0cxvCVdWrn54te7TUC3hiJ14sGKwcCI74evnNspcysbaepu+3geXaaQNb2As/7jS8ICTsS+OXSMNqpiMWgvQBrPVpMZQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39850400004)(396003)(376002)(136003)(36840700001)(46966006)(83380400001)(6862004)(107886003)(70206006)(336012)(8676002)(54906003)(36756003)(30864003)(5660300002)(86362001)(70586007)(4326008)(316002)(36860700001)(2616005)(26005)(6486002)(33656002)(478600001)(81166007)(47076005)(53546011)(8936002)(966005)(6506007)(2906002)(6512007)(356005)(82740400003)(186003)(82310400003)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 11:22:58.1494
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7964ce45-361b-461c-1b60-08d9744d57a8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3027

SGkgSnVsaWVuDQoNCj4gT24gNyBTZXAgMjAyMSwgYXQgMTA6MDUgYW0sIEp1bGllbiBHcmFsbCA8
anVsaWVuQHhlbi5vcmc+IHdyb3RlOg0KPiANCj4gSGkgUmFodWwsDQo+IA0KPiBPbiAxOS8wOC8y
MDIxIDEzOjAyLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+IFhFTiBkdXJpbmcgYm9vdCB3aWxsIHJl
YWQgdGhlIFBDSSBkZXZpY2UgdHJlZSBub2RlIOKAnHJlZ+KAnSBwcm9wZXJ0eQ0KPj4gYW5kIHdp
bGwgbWFwIHRoZSBQQ0kgY29uZmlnIHNwYWNlIHRvIHRoZSBYRU4gbWVtb3J5Lg0KPj4gQXMgb2Yg
bm93ICJwY2ktaG9zdC1lY2FtLWdlbmVyaWMiIGNvbXBhdGlibGUgYm9hcmQgaXMgc3VwcG9ydGVk
Lg0KPiANCj4gSSB0aGluayB0aGUgd29yZCAib25seSIgaXMgbWlzc2luZy4NCk9rLiANCj4gDQo+
PiAibGludXgscGNpLWRvbWFpbiIgZGV2aWNlIHRyZWUgcHJvcGVydHkgYXNzaWducyBhIGZpeGVk
IFBDSSBkb21haW4NCj4+IG51bWJlciB0byBhIGhvc3QgYnJpZGdlLCBvdGhlcndpc2UgYW4gdW5z
dGFibGUgKGFjcm9zcyBib290cykgdW5pcXVlDQo+PiBudW1iZXIgd2lsbCBiZSBhc3NpZ25lZCBi
eSBMaW51eC5UaGlzIHByb3BlcnR5IGhhcyB0byBiZSBpbiBzeW5jIHdpdGgNCj4gDQo+IFR5cG86
IG1pc3Npbmcgc3BhY2UgYWZ0ZXIgdGhlIOKAmC7igJkNCkFjay4NCj4+IFhFTiB0byBhY2Nlc3Mg
dGhlIFBDSSBkZXZpY2VzLg0KPiANCj4gSSB3b3VsZCBleHBhbmQgYSBsaXR0bGUgYml0IHRoZSBs
YXN0IHNlbnRlbmNlIHRvIGV4cGxhaW4gd2h5IHRoZSBuZWVkIHRvIGJlIHN5bmMtZWQuDQoNCkkg
d2lsbCBleHBsYWluIHdoeSBzZWdtZW50IGFuZCBkb21haW4gbmVlZCB0byBiZSBpbiBzeW5jIGlu
IG5leHQgdmVyc2lvbi4NCj4gDQo+PiA+IFhFTiB3aWxsIHJlYWQgdGhlIOKAnGxpbnV4LHBjaS1k
b21haW7igJ0gcHJvcGVydHkgZnJvbSB0aGUgZGV2aWNlIHRyZWUgbm9kZQ0KPj4gYW5kIGNvbmZp
Z3VyZSB0aGUgaG9zdCBicmlkZ2Ugc2VnbWVudCBudW1iZXIgYWNjb3JkaW5nbHkuIElmIHRoaXMN
Cj4+IHByb3BlcnR5IGlzIG5vdCBhdmFpbGFibGUgWEVOIHdpbGwgYWxsb2NhdGUgdGhlIHVuaXF1
ZSBzZWdtZW50IG51bWJlcg0KPj4gdG8gdGhlIGhvc3QgYnJpZGdlLg0KPj4gZHRfZ2V0X3BjaV9k
b21haW5fbnIoLi4pIGFuZCBkdF9wY2lfYnVzX2ZpbmRfZG9tYWluX25yKC4uKSBhcmUgZGlyZWN0
bHkNCj4+IGltcG9ydGVkIGZyb20gdGhlIExpbnV4IHNvdXJjZSB0cmVlLg0KPiANCj4gV2hhdCB3
YXMgdGhlIExpbnV4IGNvbW1pdCB1c2VkPyBJIGFsc28gcmVhZCAiZGlyZWN0bHkgaW1wb3J0ZWQi
IGFzIGEgdmVyYmFydGltIGNvcHkgYnV0IEFGQUlDVCB0aGUgaW1wbGVtZW50YXRpb24gaGFzIGJl
ZW4gc2xpZ2h0bHkgcmV3b3JrZWQuDQoNCkkgd2lsbCBhZGQgdGhlIExpbnV4IGNvbW1pdCB1c2Vk
IGluIGNvbW1pdCBtc2cgYW5kIGFsc28gd2lsbCBhZGQgdGhhdCBjb2RlIGlzIHNsaWdodGx5IG1v
ZGlmaWVkLg0KPiANCj4+IFNpZ25lZC1vZmYtYnk6IFJhaHVsIFNpbmdoIDxyYWh1bC5zaW5naEBh
cm0uY29tPg0KPj4gLS0tDQo+PiAgeGVuL2FyY2gvYXJtL3BjaS9NYWtlZmlsZSAgICAgICAgICAg
fCAgIDIgKw0KPj4gIHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMgIHwgMjYxICsr
KysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+ICB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0
LWdlbmVyaWMuYyB8ICA1NSArKysrKysNCj4+ICB4ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oICAg
ICAgICAgICB8ICAyOCArKysNCj4+ICA0IGZpbGVzIGNoYW5nZWQsIDM0NiBpbnNlcnRpb25zKCsp
DQo+PiAgY3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMNCj4+ICBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5l
cmljLmMNCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlIGIveGVuL2Fy
Y2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4gaW5kZXggYTllZTBiOWI0NC4uZjNkOTdmODU5ZSAxMDA2
NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+ICsrKyBiL3hlbi9hcmNo
L2FybS9wY2kvTWFrZWZpbGUNCj4+IEBAIC0xLDIgKzEsNCBAQA0KPj4gIG9iai15ICs9IHBjaS5v
DQo+PiAgb2JqLXkgKz0gcGNpLWFjY2Vzcy5vDQo+PiArb2JqLXkgKz0gcGNpLWhvc3QtZ2VuZXJp
Yy5vDQo+PiArb2JqLXkgKz0gcGNpLWhvc3QtY29tbW9uLm8NCj4+IGRpZmYgLS1naXQgYS94ZW4v
YXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9z
dC1jb21tb24uYw0KPj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4IDAwMDAwMDAwMDAu
LjlkZDliMDIyNzENCj4+IC0tLSAvZGV2L251bGwNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kv
cGNpLWhvc3QtY29tbW9uLmMNCj4+IEBAIC0wLDAgKzEsMjYxIEBADQo+PiArLyoNCj4+ICsgKiBD
b3B5cmlnaHQgKEMpIDIwMjEgQXJtIEx0ZC4NCj4+ICsgKg0KPj4gKyAqIEJhc2VkIG9uIExpbnV4
IGRyaXZlcnMvcGNpL2VjYW0uYw0KPj4gKyAqIENvcHlyaWdodCAyMDE2IEJyb2FkY29tLg0KPj4g
KyAqDQo+PiArICogQmFzZWQgb24gTGludXggZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktaG9z
dC1jb21tb24uYw0KPj4gKyAqIEJhc2VkIG9uIExpbnV4IGRyaXZlcnMvcGNpL2NvbnRyb2xsZXIv
cGNpLWhvc3QtZ2VuZXJpYy5jDQo+PiArICogQ29weXJpZ2h0IChDKSAyMDE0IEFSTSBMaW1pdGVk
IFdpbGwgRGVhY29uIDx3aWxsLmRlYWNvbkBhcm0uY29tPg0KPj4gKyAqDQo+PiArICoNCj4+ICsg
KiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgaXQg
YW5kL29yIG1vZGlmeQ0KPj4gKyAqIGl0IHVuZGVyIHRoZSB0ZXJtcyBvZiB0aGUgR05VIEdlbmVy
YWwgUHVibGljIExpY2Vuc2UgdmVyc2lvbiAyIGFzDQo+PiArICogcHVibGlzaGVkIGJ5IHRoZSBG
cmVlIFNvZnR3YXJlIEZvdW5kYXRpb24uDQo+PiArICoNCj4+ICsgKiBUaGlzIHByb2dyYW0gaXMg
ZGlzdHJpYnV0ZWQgaW4gdGhlIGhvcGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1bCwNCj4+ICsgKiBi
dXQgV0lUSE9VVCBBTlkgV0FSUkFOVFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGllZCB3YXJyYW50
eSBvZg0KPj4gKyAqIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIg
UFVSUE9TRS4gIFNlZSB0aGUNCj4+ICsgKiBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBmb3Ig
bW9yZSBkZXRhaWxzLg0KPj4gKyAqDQo+PiArICogWW91IHNob3VsZCBoYXZlIHJlY2VpdmVkIGEg
Y29weSBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UNCj4+ICsgKiBhbG9uZyB3aXRo
IHRoaXMgcHJvZ3JhbS4gIElmIG5vdCwgc2VlIDxodHRwOi8vd3d3LmdudS5vcmcvbGljZW5zZXMv
Pi4NCj4+ICsgKi8NCj4+ICsNCj4+ICsjaW5jbHVkZSA8eGVuL2luaXQuaD4NCj4+ICsjaW5jbHVk
ZSA8eGVuL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDxhc20vcGNpLmg+DQo+IA0KPiBBRkFJQ1QsIDx4
ZW4vcGNpLmg+IGFscmVhZHkgaW5jbHVkZXMgPGFzbS9wY2kuaD4uIFNvIHRoaXMgbG9va3MgdW5u
ZWNjZXNzYXJ5Lg0KQWNrLg0KPiANCj4+ICsjaW5jbHVkZSA8eGVuL3J3bG9jay5oPg0KPj4gKyNp
bmNsdWRlIDx4ZW4vc2NoZWQuaD4NCj4+ICsjaW5jbHVkZSA8eGVuL3ZtYXAuaD4NCj4+ICsNCj4+
ICsvKg0KPj4gKyAqIExpc3QgZm9yIGFsbCB0aGUgcGNpIGhvc3QgYnJpZGdlcy4NCj4+ICsgKi8N
Cj4+ICsNCj4+ICtzdGF0aWMgTElTVF9IRUFEKHBjaV9ob3N0X2JyaWRnZXMpOw0KPj4gKw0KPj4g
K3N0YXRpYyBhdG9taWNfdCBkb21haW5fbnIgPSBBVE9NSUNfSU5JVCgtMSk7DQo+PiArDQo+PiAr
Ym9vbCBkdF9wY2lfcGFyc2VfYnVzX3JhbmdlKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0K
Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cg
KmNmZykNCj4gDQo+IEFzaWRlLCAicGNpX2NvbmZpZ193aW5kb3ciLCB0aGUgZnVuY3Rpb24gaXMg
bm90IEFybSBzcGVjaWZpYy4gV291bGQgaXQgYmUgcG9zc2libGUgdG8gY29uc2lkZXIgdG8gaW50
cm9kdWNlICJzdHJ1Y3QgcmVzb3VyY2UiIGluIFhlbiBzbyB0aGlzIGZ1bmN0aW9uIGNhbiBiZSBt
b3ZlZCBpbiBjb21tb24vZGV2aWNlX3RyZWUuYz8NCg0KSSBjYW4gaW50cm9kdWNlIHRoZSAic3Ry
dWN0IHJlc291cmNl4oCdIGJ1dCBJIGFtIG5vdCBzdXJlIHdoZXRoZXIgInN0cnVjdCByZXNvdXJj
ZeKAnSB3aWxsIGJlDQp1c2VmdWwgbGF0ZXIgcG9pbnQgaW4gdGltZS4gV2hhdCBJIHByZWZlciBh
cyBvZiBub3csIHdlIGNhbiBoYXZlIHRoaXMgZnVuY3Rpb24gYW5kIHdlIGNhbg0KbW92ZSB0aGlz
IHRvIGNvbW1vbi9kZXZpY2VfdHJlZS5jIG9uY2Ugd2UgaGF2ZSB0aGUgcmVxdWlyZW1lbnQgZm9y
ICJzdHJ1Y3QgcmVzb3VyY2XigJ0uDQo+IA0KPj4gK3sNCj4+ICsgICAgY29uc3QgX19iZTMyICpj
ZWxsczsNCj4+ICsgICAgdWludDMyX3QgbGVuOw0KPj4gKw0KPj4gKyAgICBjZWxscyA9IGR0X2dl
dF9wcm9wZXJ0eShkZXYsICJidXMtcmFuZ2UiLCAmbGVuKTsNCj4+ICsgICAgLyogYnVzLXJhbmdl
IHNob3VsZCBhdCBsZWFzdCBiZSAyIGNlbGxzICovDQo+PiArICAgIGlmICggIWNlbGxzIHx8IChs
ZW4gPCAoc2l6ZW9mKCpjZWxscykgKiAyKSkgKQ0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0K
PiANCj4gSG93IGFib3V0IGludHJvZHVjaW5nIGR0X3Byb3BlcnR5X3JlYWRfdTMyX2FycmF5KCk/
DQoNCk9rLiBJIHdpbGwgaW50cm9kdWNlIGR0X3Byb3BlcnR5X3JlYWRfdTMyX2FycmF5KCkuDQoN
Cj4gDQo+PiArDQo+PiArICAgIGNmZy0+YnVzbl9zdGFydCA9IGR0X25leHRfY2VsbCgxLCAmY2Vs
bHMpOw0KPj4gKyAgICBjZmctPmJ1c25fZW5kID0gZHRfbmV4dF9jZWxsKDEsICZjZWxscyk7DQo+
PiArDQo+PiArICAgIHJldHVybiB0cnVlOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW5saW5l
IHZvaWQgX19pb21lbSAqcGNpX3JlbWFwX2NmZ3NwYWNlKHBhZGRyX3Qgc3RhcnQsIHNpemVfdCBs
ZW4pDQo+PiArew0KPj4gKyAgICByZXR1cm4gaW9yZW1hcF9ub2NhY2hlKHN0YXJ0LCBsZW4pOw0K
Pj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgdm9pZCBwY2lfZWNhbV9mcmVlKHN0cnVjdCBwY2lfY29u
ZmlnX3dpbmRvdyAqY2ZnKQ0KPj4gK3sNCj4+ICsgICAgaWYgKCBjZmctPndpbiApDQo+PiArICAg
ICAgICBpb3VubWFwKGNmZy0+d2luKTsNCj4+ICsNCj4+ICsgICAgeGZyZWUoY2ZnKTsNCj4+ICt9
DQo+PiArDQo+PiArc3RhdGljIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqZ2VuX3BjaV9pbml0
KHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBpbnQgZWNhbV9yZWdfaWR4KQ0KPj4gK3sNCj4+ICsgICAg
aW50IGVycjsNCj4+ICsgICAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmc7DQo+PiArICAg
IHBhZGRyX3QgYWRkciwgc2l6ZTsNCj4+ICsNCj4+ICsgICAgY2ZnID0geHphbGxvYyhzdHJ1Y3Qg
cGNpX2NvbmZpZ193aW5kb3cpOw0KPj4gKyAgICBpZiAoICFjZmcgKQ0KPj4gKyAgICAgICAgcmV0
dXJuIE5VTEw7DQo+PiArDQo+PiArICAgIGVyciA9IGR0X3BjaV9wYXJzZV9idXNfcmFuZ2UoZGV2
LCBjZmcpOw0KPj4gKyAgICBpZiAoICFlcnIgKSB7DQo+PiArICAgICAgICBjZmctPmJ1c25fc3Rh
cnQgPSAwOw0KPj4gKyAgICAgICAgY2ZnLT5idXNuX2VuZCA9IDB4ZmY7DQo+PiArICAgICAgICBw
cmludGsoWEVOTE9HX0VSUiAiJXM6Tm8gYnVzIHJhbmdlIGZvdW5kIGZvciBwY2kgY29udHJvbGxl
clxuIiwNCj4gDQo+IFR5cG86IE1pc3Npbmcgc3BhY2UgYWZ0ZXIgJzrigJkuDQpBY2suDQo+IA0K
Pj4gKyAgICAgICAgICAgICAgIGR0X25vZGVfZnVsbF9uYW1lKGRldikpOw0KPj4gKyAgICB9IGVs
c2Ugew0KPj4gKyAgICAgICAgaWYgKCBjZmctPmJ1c25fZW5kID4gY2ZnLT5idXNuX3N0YXJ0ICsg
MHhmZiApDQo+PiArICAgICAgICAgICAgY2ZnLT5idXNuX2VuZCA9IGNmZy0+YnVzbl9zdGFydCAr
IDB4ZmY7DQo+PiArICAgIH0NCj4+ICsNCj4+ICsgICAgLyogUGFyc2Ugb3VyIFBDSSBlY2FtIHJl
Z2lzdGVyIGFkZHJlc3MqLw0KPj4gKyAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJlc3MoZGV2
LCBlY2FtX3JlZ19pZHgsICZhZGRyLCAmc2l6ZSk7DQo+PiArICAgIGlmICggZXJyICkNCj4+ICsg
ICAgICAgIGdvdG8gZXJyX2V4aXQ7DQo+PiArDQo+PiArICAgIGNmZy0+cGh5c19hZGRyID0gYWRk
cjsNCj4+ICsgICAgY2ZnLT5zaXplID0gc2l6ZTsNCj4+ICsNCj4+ICsgICAgLyoNCj4+ICsgICAg
ICogT24gNjQtYml0IHN5c3RlbXMsIHdlIGRvIGEgc2luZ2xlIGlvcmVtYXAgZm9yIHRoZSB3aG9s
ZSBjb25maWcgc3BhY2UNCj4+ICsgICAgICogc2luY2Ugd2UgaGF2ZSBlbm91Z2ggdmlydHVhbCBh
ZGRyZXNzIHJhbmdlIGF2YWlsYWJsZS4gIE9uIDMyLWJpdCwgd2UNCj4gDQo+IEluIFhlbiBvbiBB
cm02NCwgdGhlIFZNQVAgaXMgYWN0dWFsbHkgb25seSAxR0IuIFNvIGl0IGlzIG5vdCB0aGF0IGJp
ZyBhbmQgdGhpcyB3aWxsIGNvbXBldGUgd2l0aCBvdGhlciBtYXBwaW5nIGxpa2UgSVRTLCBnbG9i
YWwgZG9tYWluIG1hcHBpbmcuLi4NCj4gDQo+IFNvIEkgdGhpbmsgdGhlIHZNQVAgYXJlYSB3aWxs
IG5lZWQgdG8gaW5jcmVhc2UgdG8gY2F0ZXIgdG8gaW5jcmVhc2UgdXNhZ2UuDQoNCkxldCBtZSBj
aGVjayBvbiB0aGlzIGFuZCBjb21lIGJhY2sgdG8geW91Lg0KDQo+IA0KPj4gKyAgICAgKiBpb3Jl
bWFwIHRoZSBjb25maWcgc3BhY2UgZm9yIGVhY2ggYnVzIGluZGl2aWR1YWxseS4NCj4+ICsgICAg
ICoNCj4+ICsgICAgICogQXMgb2Ygbm93IG9ubHkgNjQtYml0IGlzIHN1cHBvcnRlZCAzMi1iaXQg
aXMgbm90IHN1cHBvcnRlZC4NCj4+ICsgICAgICovDQo+PiArICAgIGNmZy0+d2luID0gcGNpX3Jl
bWFwX2NmZ3NwYWNlKGNmZy0+cGh5c19hZGRyLCBjZmctPnNpemUpOw0KPj4gKyAgICBpZiAoICFj
ZmctPndpbiApDQo+PiArICAgICAgICBnb3RvIGVycl9leGl0X3JlbWFwOw0KPj4gKw0KPj4gKyAg
ICBwcmludGsoIkVDQU0gYXQgW21lbSAlbHgtJWx4XSBmb3IgW2J1cyAleC0leF0gXG4iLGNmZy0+
cGh5c19hZGRyLA0KPiANCj4gVGhlIHBoeXNpY2FsIGFkZHJlc3MgaXMgYSBwYWRkcl90LiBTbyB0
aGlzIG5lZWRzIHRvIHVzZSBQUklwYWRkci4gQWxzbywgcGxlYXNlIHVzZSBwcmVwcmVudCBoZXhh
ZGVjaW1hbCB3aXRoIDB4LiBUaGlzIG1ha2VzIGEgbG90IGVhc2llciB0byBkaWZmZXJlbnRpYXRl
IGhleGEgdnMgZGVjaW1hbCBpbiB0aGUgbG9nLg0KQWNrLg0KPiANCj4+ICsgICAgICAgICAgICBj
ZmctPnBoeXNfYWRkciArIGNmZy0+c2l6ZSAtIDEsIGNmZy0+YnVzbl9zdGFydCwgY2ZnLT5idXNu
X2VuZCk7DQo+PiArDQo+PiArICAgIHJldHVybiBjZmc7DQo+PiArDQo+PiArZXJyX2V4aXRfcmVt
YXA6DQo+PiArICAgIHByaW50ayhYRU5MT0dfRVJSICJFQ0FNIGlvcmVtYXAgZmFpbGVkXG4iKTsN
Cj4+ICtlcnJfZXhpdDoNCj4+ICsgICAgcGNpX2VjYW1fZnJlZShjZmcpOw0KPiANCj4gQ29kaW5n
IHN0eWxlOiBQbGVhc2UgYWRkIGEgbmV3IGxpbmUgYmVmb3JlIHJldHVybi4NCkFjay4NCj4gDQo+
PiArICAgIHJldHVybiBOVUxMOw0KPj4gK30NCj4+ICsNCj4+ICtzdHJ1Y3QgcGNpX2hvc3RfYnJp
ZGdlICpwY2lfYWxsb2NfaG9zdF9icmlkZ2Uodm9pZCkNCj4+ICt7DQo+PiArICAgIHN0cnVjdCBw
Y2lfaG9zdF9icmlkZ2UgKmJyaWRnZSA9IHh6YWxsb2Moc3RydWN0IHBjaV9ob3N0X2JyaWRnZSk7
DQo+PiArDQo+PiArICAgIGlmICggIWJyaWRnZSApDQo+PiArICAgICAgICByZXR1cm4gTlVMTDsN
Cj4+ICsNCj4+ICsgICAgSU5JVF9MSVNUX0hFQUQoJmJyaWRnZS0+bm9kZSk7DQo+PiArICAgIGJy
aWRnZS0+YnVzX3N0YXJ0ID0gfjA7DQo+PiArICAgIGJyaWRnZS0+YnVzX2VuZCA9IH4wOw0KPiAN
Cj4gQ29kaW5nIHN0eWxlOiBQbGVhc2UgYWRkIGEgbmV3IGxpbmUgYmVmb3JlIHJldHVybi4NCkFj
ay4NCj4gDQo+IA0KPj4gKyAgICByZXR1cm4gYnJpZGdlOw0KPj4gK30NCj4+ICsNCj4+ICt2b2lk
IHBjaV9hZGRfaG9zdF9icmlkZ2Uoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlKQ0KPj4g
K3sNCj4+ICsgICAgbGlzdF9hZGRfdGFpbCgmYnJpZGdlLT5ub2RlLCAmcGNpX2hvc3RfYnJpZGdl
cyk7DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRpYyBpbnQgcGNpX2dldF9uZXdfZG9tYWluX25yKHZv
aWQpDQo+PiArew0KPj4gKyAgICByZXR1cm4gYXRvbWljX2luY19yZXR1cm4oJmRvbWFpbl9ucik7
DQo+PiArfQ0KPj4gKw0KPj4gKy8qDQo+PiArICogVGhpcyBmdW5jdGlvbiB3aWxsIHRyeSB0byBv
YnRhaW4gdGhlIGhvc3QgYnJpZGdlIGRvbWFpbiBudW1iZXIgYnkNCj4+ICsgKiBmaW5kaW5nIGEg
cHJvcGVydHkgY2FsbGVkICJsaW51eCxwY2ktZG9tYWluIiBvZiB0aGUgZ2l2ZW4gZGV2aWNlIG5v
ZGUuDQo+PiArICoNCj4+ICsgKiBAbm9kZTogZGV2aWNlIHRyZWUgbm9kZSB3aXRoIHRoZSBkb21h
aW4gaW5mb3JtYXRpb24NCj4+ICsgKg0KPj4gKyAqIFJldHVybnMgdGhlIGFzc29jaWF0ZWQgZG9t
YWluIG51bWJlciBmcm9tIERUIGluIHRoZSByYW5nZSBbMC0weGZmZmZdLCBvcg0KPj4gKyAqIGEg
bmVnYXRpdmUgdmFsdWUgaWYgdGhlIHJlcXVpcmVkIHByb3BlcnR5IGlzIG5vdCBmb3VuZC4NCj4+
ICsgKi8NCj4+ICtzdGF0aWMgaW50IGR0X2dldF9wY2lfZG9tYWluX25yKHN0cnVjdCBkdF9kZXZp
Y2Vfbm9kZSAqbm9kZSkNCj4gDQo+IE5vdGhpbmcgbG9va3MgQXJtIHNwZWNpZmljIGZvciB0aGlz
IGZ1bmN0aW9uLiBDYW4geW91IG1vdmUgaXQgaW4gZGV2aWNlX3RyZWUuYz8NCkFjay4NCj4gDQo+
PiArew0KPj4gKyAgICB1MzIgZG9tYWluOw0KPj4gKyAgICBpbnQgZXJyb3I7DQo+PiArDQo+PiAr
ICAgIGVycm9yID0gZHRfcHJvcGVydHlfcmVhZF91MzIobm9kZSwgImxpbnV4LHBjaS1kb21haW4i
LCAmZG9tYWluKTsNCj4+ICsgICAgaWYgKCAhZXJyb3IgKQ0KPj4gKyAgICAgICAgcmV0dXJuIC1F
SU5WQUw7DQo+PiArDQo+PiArICAgIHJldHVybiAodTE2KWRvbWFpbjsNCj4+ICt9DQo+PiArDQo+
PiArc3RhdGljIGludCBwY2lfYnVzX2ZpbmRfZG9tYWluX25yKHN0cnVjdCBkdF9kZXZpY2Vfbm9k
ZSAqZGV2KQ0KPj4gK3sNCj4+ICsgICAgc3RhdGljIGludCB1c2VfZHRfZG9tYWlucyA9IC0xOw0K
Pj4gKyAgICBpbnQgZG9tYWluOw0KPj4gKw0KPj4gKyAgICBkb21haW4gPSBkdF9nZXRfcGNpX2Rv
bWFpbl9ucihkZXYpOw0KPj4gKw0KPj4gKyAgICAvKg0KPj4gKyAgICAgKiBDaGVjayBEVCBkb21h
aW4gYW5kIHVzZV9kdF9kb21haW5zIHZhbHVlcy4NCj4+ICsgICAgICoNCj4+ICsgICAgICogSWYg
RFQgZG9tYWluIHByb3BlcnR5IGlzIHZhbGlkIChkb21haW4gPj0gMCkgYW5kDQo+PiArICAgICAq
IHVzZV9kdF9kb21haW5zICE9IDAsIHRoZSBEVCBhc3NpZ25tZW50IGlzIHZhbGlkIHNpbmNlIHRo
aXMgbWVhbnMNCj4+ICsgICAgICogd2UgaGF2ZSBub3QgcHJldmlvdXNseSBhbGxvY2F0ZWQgYSBk
b21haW4gbnVtYmVyIGJ5IHVzaW5nDQo+PiArICAgICAqIHBjaV9nZXRfbmV3X2RvbWFpbl9ucigp
OyB3ZSBzaG91bGQgYWxzbyB1cGRhdGUgdXNlX2R0X2RvbWFpbnMgdG8NCj4+ICsgICAgICogMSwg
dG8gaW5kaWNhdGUgdGhhdCB3ZSBoYXZlIGp1c3QgYXNzaWduZWQgYSBkb21haW4gbnVtYmVyIGZy
b20NCj4+ICsgICAgICogRFQuDQo+PiArICAgICAqDQo+PiArICAgICAqIElmIERUIGRvbWFpbiBw
cm9wZXJ0eSB2YWx1ZSBpcyBub3QgdmFsaWQgKGllIGRvbWFpbiA8IDApLCBhbmQgd2UNCj4+ICsg
ICAgICogaGF2ZSBub3QgcHJldmlvdXNseSBhc3NpZ25lZCBhIGRvbWFpbiBudW1iZXIgZnJvbSBE
VA0KPj4gKyAgICAgKiAodXNlX2R0X2RvbWFpbnMgIT0gMSkgd2Ugc2hvdWxkIGFzc2lnbiBhIGRv
bWFpbiBudW1iZXIgYnkNCj4+ICsgICAgICogdXNpbmcgdGhlOg0KPj4gKyAgICAgKg0KPj4gKyAg
ICAgKiBwY2lfZ2V0X25ld19kb21haW5fbnIoKQ0KPj4gKyAgICAgKg0KPj4gKyAgICAgKiBBUEkg
YW5kIHVwZGF0ZSB0aGUgdXNlX2R0X2RvbWFpbnMgdmFsdWUgdG8ga2VlcCB0cmFjayBvZiBtZXRo
b2Qgd2UNCj4+ICsgICAgICogYXJlIHVzaW5nIHRvIGFzc2lnbiBkb21haW4gbnVtYmVycyAodXNl
X2R0X2RvbWFpbnMgPSAwKS4NCj4+ICsgICAgICoNCj4+ICsgICAgICogQWxsIG90aGVyIGNvbWJp
bmF0aW9ucyBpbXBseSB3ZSBoYXZlIGEgcGxhdGZvcm0gdGhhdCBpcyB0cnlpbmcNCj4+ICsgICAg
ICogdG8gbWl4IGRvbWFpbiBudW1iZXJzIG9idGFpbmVkIGZyb20gRFQgYW5kIHBjaV9nZXRfbmV3
X2RvbWFpbl9ucigpLA0KPj4gKyAgICAgKiB3aGljaCBpcyBhIHJlY2lwZSBmb3IgZG9tYWluIG1p
c2hhbmRsaW5nIGFuZCBpdCBpcyBwcmV2ZW50ZWQgYnkNCj4+ICsgICAgICogaW52YWxpZGF0aW5n
IHRoZSBkb21haW4gdmFsdWUgKGRvbWFpbiA9IC0xKSBhbmQgcHJpbnRpbmcgYQ0KPj4gKyAgICAg
KiBjb3JyZXNwb25kaW5nIGVycm9yLg0KPj4gKyAgICAgKi8NCj4+ICsgICAgaWYgKCBkb21haW4g
Pj0gMCAmJiB1c2VfZHRfZG9tYWlucyApDQo+PiArICAgIHsNCj4+ICsgICAgICAgIHVzZV9kdF9k
b21haW5zID0gMTsNCj4+ICsgICAgfQ0KPj4gKyAgICBlbHNlIGlmICggZG9tYWluIDwgMCAmJiB1
c2VfZHRfZG9tYWlucyAhPSAxICkNCj4+ICsgICAgew0KPj4gKyAgICAgICAgdXNlX2R0X2RvbWFp
bnMgPSAwOw0KPj4gKyAgICAgICAgZG9tYWluID0gcGNpX2dldF9uZXdfZG9tYWluX25yKCk7DQo+
PiArICAgIH0NCj4+ICsgICAgZWxzZQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBwcmludGsoWEVO
TE9HX0VSUiAiSW5jb25zaXN0ZW50IFwibGludXgscGNpLWRvbWFpblwiIHByb3BlcnR5IGluIERU
XG4iKTsNCj4+ICsgICAgICAgIEJVRygpOw0KPiANCj4gSSBkb24ndCB0aGluayBwY2lfYnVzX2Zp
bmRfZG9tYWluX25yKCkgc2hvdWxkIGJlIHRoZSBmdW5jdGlvbiB0aGF0IGNyYXNoZXMgWGVuLiBJ
bnN0ZWFkLCB0aGlzIHNob3VsZCBiZSBwcm9wYWdhdGVkIHRvIHRoZSBoaWdoZXN0IHBvc3NpYmxl
IGNhbGxlcnMgYW5kIGxldCBpdCBkZWNpZGUgd2hhdCB0byBkby4NCg0KQWNrLiANCj4gDQo+PiAr
ICAgIH0NCj4+ICsNCj4+ICsgICAgcmV0dXJuIGRvbWFpbjsNCj4+ICt9DQo+PiArDQo+PiAraW50
IHBjaV9ob3N0X2NvbW1vbl9wcm9iZShzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+ICsg
ICAgICAgICAgICAgICAgICAgICAgICAgIGludCBlY2FtX3JlZ19pZHgpDQo+PiArew0KPj4gKyAg
ICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2U7DQo+PiArICAgIHN0cnVjdCBwY2lfY29u
ZmlnX3dpbmRvdyAqY2ZnOw0KPj4gKyAgICBpbnQgZXJyOw0KPj4gKw0KPj4gKyAgICBicmlkZ2Ug
PSBwY2lfYWxsb2NfaG9zdF9icmlkZ2UoKTsNCj4+ICsgICAgaWYgKCAhYnJpZGdlICkNCj4+ICsg
ICAgICAgIHJldHVybiAtRU5PTUVNOw0KPj4gKw0KPj4gKyAgICAvKiBQYXJzZSBhbmQgbWFwIG91
ciBDb25maWd1cmF0aW9uIFNwYWNlIHdpbmRvd3MgKi8NCj4+ICsgICAgY2ZnID0gZ2VuX3BjaV9p
bml0KGRldiwgZWNhbV9yZWdfaWR4KTsNCj4+ICsgICAgaWYgKCAhY2ZnICkNCj4+ICsgICAgew0K
Pj4gKyAgICAgICAgZXJyID0gLUVOT01FTTsNCj4+ICsgICAgICAgIGdvdG8gZXJyX2V4aXQ7DQo+
PiArICAgIH0NCj4+ICsNCj4+ICsgICAgYnJpZGdlLT5kdF9ub2RlID0gZGV2Ow0KPj4gKyAgICBi
cmlkZ2UtPnN5c2RhdGEgPSBjZmc7DQo+PiArICAgIGJyaWRnZS0+YnVzX3N0YXJ0ID0gY2ZnLT5i
dXNuX3N0YXJ0Ow0KPj4gKyAgICBicmlkZ2UtPmJ1c19lbmQgPSBjZmctPmJ1c25fZW5kOw0KPj4g
Kw0KPj4gKyAgICBicmlkZ2UtPnNlZ21lbnQgPSBwY2lfYnVzX2ZpbmRfZG9tYWluX25yKGRldik7
DQo+PiArDQo+PiArICAgIHBjaV9hZGRfaG9zdF9icmlkZ2UoYnJpZGdlKTsNCj4+ICsNCj4+ICsg
ICAgcmV0dXJuIDA7DQo+PiArDQo+PiArZXJyX2V4aXQ6DQo+PiArICAgIHhmcmVlKGJyaWRnZSk7
DQo+PiArICAgIHJldHVybiBlcnI7DQo+PiArfQ0KPj4gKw0KPj4gKy8qDQo+PiArICogTG9jYWwg
dmFyaWFibGVzOg0KPj4gKyAqIG1vZGU6IEMNCj4+ICsgKiBjLWZpbGUtc3R5bGU6ICJCU0QiDQo+
PiArICogYy1iYXNpYy1vZmZzZXQ6IDQNCj4+ICsgKiB0YWItd2lkdGg6IDQNCj4+ICsgKiBpbmRl
bnQtdGFicy1tb2RlOiBuaWwNCj4+ICsgKiBFbmQ6DQo+PiArICovDQo+PiBkaWZmIC0tZ2l0IGEv
eGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5lcmljLmMgYi94ZW4vYXJjaC9hcm0vcGNpL3Bj
aS1ob3N0LWdlbmVyaWMuYw0KPj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4IDAwMDAw
MDAwMDAuLjEzZDBmN2Y5OTkNCj4+IC0tLSAvZGV2L251bGwNCj4+ICsrKyBiL3hlbi9hcmNoL2Fy
bS9wY2kvcGNpLWhvc3QtZ2VuZXJpYy5jDQo+PiBAQCAtMCwwICsxLDU1IEBADQo+PiArLyoNCj4+
ICsgKiBDb3B5cmlnaHQgKEMpIDIwMjEgQXJtIEx0ZC4NCj4+ICsgKg0KPj4gKyAqIEJhc2VkIG9u
IExpbnV4IGRyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpLWhvc3QtY29tbW9uLmMNCj4+ICsgKiBC
YXNlZCBvbiBMaW51eCBkcml2ZXJzL3BjaS9jb250cm9sbGVyL3BjaS1ob3N0LWdlbmVyaWMuYw0K
Pj4gKyAqIENvcHlyaWdodCAoQykgMjAxNCBBUk0gTGltaXRlZCBXaWxsIERlYWNvbiA8d2lsbC5k
ZWFjb25AYXJtLmNvbT4NCj4+ICsgKg0KPj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3
YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9kaWZ5DQo+PiArICogaXQgdW5k
ZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSB2ZXJzaW9uIDIg
YXMNCj4+ICsgKiBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbi4NCj4+
ICsgKg0KPj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0aGUgaG9wZSB0aGF0
IGl0IHdpbGwgYmUgdXNlZnVsLA0KPj4gKyAqIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0
aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9mDQo+PiArICogTUVSQ0hBTlRBQklMSVRZ
IG9yIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRoZQ0KPj4gKyAqIEdO
VSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuDQo+PiArICoNCj4+ICsg
KiBZb3Ugc2hvdWxkIGhhdmUgcmVjZWl2ZWQgYSBjb3B5IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJs
aWMgTGljZW5zZQ0KPj4gKyAqIGFsb25nIHdpdGggdGhpcyBwcm9ncmFtLiAgSWYgbm90LCBzZWUg
PGh0dHA6Ly93d3cuZ251Lm9yZy9saWNlbnNlcy8+Lg0KPj4gKyAqLw0KPj4gKw0KPj4gKyNpbmNs
dWRlIDxhc20vZGV2aWNlLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5jbHVk
ZSA8YXNtL3BjaS5oPg0KPj4gKw0KPj4gK3N0YXRpYyBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21h
dGNoIGdlbl9wY2lfZHRfbWF0Y2hbXSA9IHsNCj4+ICsgICAgeyAuY29tcGF0aWJsZSA9ICJwY2kt
aG9zdC1lY2FtLWdlbmVyaWMiIH0sDQo+PiArICAgIHsgfSwNCj4+ICt9Ow0KPj4gKw0KPj4gK3N0
YXRpYyBpbnQgZ2VuX3BjaV9kdF9pbml0KHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LCBjb25z
dCB2b2lkICpkYXRhKQ0KPj4gK3sNCj4+ICsgICAgY29uc3Qgc3RydWN0IGR0X2RldmljZV9tYXRj
aCAqb2ZfaWQ7DQo+PiArDQo+PiArICAgIG9mX2lkID0gZHRfbWF0Y2hfbm9kZShnZW5fcGNpX2R0
X21hdGNoLCBkZXYtPmRldi5vZl9ub2RlKTsNCj4gDQo+IFRoaXMgc2VlbXMgdG8gYmUgYSBiaXQg
cG9pbnRsZXNzIHRvIG1lIGFzIHlvdSBhbHJlYWR5IGtub3cgdGhlIGNvbXBhdGlibGUgKHRoZXJl
IGlzIG9ubHkgb25lIHBvc3NpYmxlLi4uKS4NCiBBcyBvZiBub3cgd2UgYXJlIG9ubHkgaW1wbGVt
ZW50aW5nIHRoZSAicGNpLWhvc3QtZWNhbS1nZW5lcmlj4oCdIGNvbXBhdGlibGUgUENJLCBidXQg
aW4gZnV0dXJlIHdlIG1pZ2h0IA0KbmVlZCB0byBpbXBsZW1lbnQgdGhlIG90aGVyIGNvbXBhdGli
bGUgbGlrZSBMaW51eCBzZWUgYXMgYmVsb3cuDQoNCmh0dHBzOi8vZWxpeGlyLmJvb3RsaW4uY29t
L2xpbnV4L2xhdGVzdC9zb3VyY2UvZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktaG9zdC1nZW5l
cmljLmMjTDU5DQoNCj4gDQo+PiArDQo+PiArICAgIHByaW50ayhYRU5MT0dfSU5GTyAiRm91bmQg
UENJIGhvc3QgYnJpZGdlICVzIGNvbXBhdGlibGU6JXMgXG4iLA0KPj4gKyAgICAgICAgICAgZHRf
bm9kZV9mdWxsX25hbWUoZGV2KSwgb2ZfaWQtPmNvbXBhdGlibGUpOw0KPj4gKw0KPj4gKyAgICBy
ZXR1cm4gcGNpX2hvc3RfY29tbW9uX3Byb2JlKGRldiwgMCk7DQo+PiArfQ0KPj4gKw0KPj4gK0RU
X0RFVklDRV9TVEFSVChwY2lfZ2VuLCAiUENJIEhPU1QgR0VORVJJQyIsIERFVklDRV9QQ0kpDQo+
PiArLmR0X21hdGNoID0gZ2VuX3BjaV9kdF9tYXRjaCwNCj4+ICsuaW5pdCA9IGdlbl9wY2lfZHRf
aW5pdCwNCj4+ICtEVF9ERVZJQ0VfRU5EDQo+PiArDQo+PiArLyoNCj4+ICsgKiBMb2NhbCB2YXJp
YWJsZXM6DQo+PiArICogbW9kZTogQw0KPj4gKyAqIGMtZmlsZS1zdHlsZTogIkJTRCINCj4+ICsg
KiBjLWJhc2ljLW9mZnNldDogNA0KPj4gKyAqIHRhYi13aWR0aDogNA0KPj4gKyAqIGluZGVudC10
YWJzLW1vZGU6IG5pbA0KPj4gKyAqIEVuZDoNCj4+ICsgKi8NCj4+IGRpZmYgLS1naXQgYS94ZW4v
aW5jbHVkZS9hc20tYXJtL3BjaS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KPj4gaW5k
ZXggNjFlNDNkYTA4OC4uNThhNTFlNzI0ZSAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2Fz
bS1hcm0vcGNpLmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+IEBAIC0y
Niw2ICsyNiwzNCBAQCBzdHJ1Y3QgYXJjaF9wY2lfZGV2IHsNCj4+ICAgICAgc3RydWN0IGRldmlj
ZSBkZXY7DQo+PiAgfTsNCj4+ICArLyoNCj4+ICsgKiBzdHJ1Y3QgdG8gaG9sZCB0aGUgbWFwcGlu
Z3Mgb2YgYSBjb25maWcgc3BhY2Ugd2luZG93LiBUaGlzDQo+PiArICogaXMgZXhwZWN0ZWQgdG8g
YmUgdXNlZCBhcyBzeXNkYXRhIGZvciBQQ0kgY29udHJvbGxlcnMgdGhhdA0KPj4gKyAqIHVzZSBF
Q0FNLg0KPj4gKyAqLw0KPj4gK3N0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyB7DQo+PiArICAgIHBh
ZGRyX3QgICAgICAgICBwaHlzX2FkZHI7DQo+PiArICAgIHBhZGRyX3QgICAgICAgICBzaXplOw0K
Pj4gKyAgICB1aW50OF90ICAgICAgICAgYnVzbl9zdGFydDsNCj4+ICsgICAgdWludDhfdCAgICAg
ICAgIGJ1c25fZW5kOw0KPj4gKyAgICB2b2lkIF9faW9tZW0gICAgKndpbjsNCj4+ICt9Ow0KPj4g
Kw0KPj4gKy8qDQo+PiArICogc3RydWN0IHRvIGhvbGQgcGNpIGhvc3QgYnJpZGdlIGluZm9ybWF0
aW9uDQo+PiArICogZm9yIGEgUENJIGNvbnRyb2xsZXIuDQo+PiArICovDQo+PiArc3RydWN0IHBj
aV9ob3N0X2JyaWRnZSB7DQo+PiArICAgIHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZHRfbm9kZTsg
IC8qIFBvaW50ZXIgdG8gdGhlIGFzc29jaWF0ZWQgRFQgbm9kZSAqLw0KPj4gKyAgICBzdHJ1Y3Qg
bGlzdF9oZWFkIG5vZGU7ICAgICAgICAgICAvKiBOb2RlIGluIGxpc3Qgb2YgaG9zdCBicmlkZ2Vz
ICovDQo+PiArICAgIHVpbnQxNl90IHNlZ21lbnQ7ICAgICAgICAgICAgICAgIC8qIFNlZ21lbnQg
bnVtYmVyICovDQo+PiArICAgIHU4IGJ1c19zdGFydDsgICAgICAgICAgICAgICAgICAgIC8qIEJ1
cyBzdGFydCBvZiB0aGlzIGJyaWRnZS4gKi8NCj4+ICsgICAgdTggYnVzX2VuZDsgICAgICAgICAg
ICAgICAgICAgICAgLyogQnVzIGVuZCBvZiB0aGlzIGJyaWRnZS4gKi8NCj4gDQo+IFBsZWFzZSB1
c2UgdWludDhfdCByYXRoZXIgdGhhbiB1OC4NCkFjay4gDQoNClJlZ2FyZHMsDQpSYWh1bA0KDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:43:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:43:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184308.332902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOewf-0004yS-9u; Fri, 10 Sep 2021 11:43:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184308.332902; Fri, 10 Sep 2021 11:43:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOewf-0004yL-6y; Fri, 10 Sep 2021 11:43:49 +0000
Received: by outflank-mailman (input) for mailman id 184308;
 Fri, 10 Sep 2021 11:43:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOewd-0004yF-Ha
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:43:47 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5b362b62-122c-11ec-b229-12813bfff9fa;
 Fri, 10 Sep 2021 11:43:46 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18ABeUUg019229;
 Fri, 10 Sep 2021 11:43:41 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b06pjr0bu-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 11:43:40 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6243.eurprd03.prod.outlook.com (2603:10a6:20b:144::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Fri, 10 Sep
 2021 11:43:37 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 11:43:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b362b62-122c-11ec-b229-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e7LpKeCkkjAoXD151bYGd4dasO0GC2hFpu664c6E0p2wtgxfZ4s/5bRVZTYbBN1Ez5uBrQsLWkNs2R2vtRc566jN+duPagSXzviqnx9VwQpZZWKKsmZmvQjACcphu+KHr+xHvngUFW7av4+BH6aXDmZv6mrN22wLvNhkQftkB5EShsdYPxCY8sFrBg3ZGxuSyIT7o/02FTsoEWcApiaEi5ZXebM3+LaDkNrcKJBJym1O+qkoaASsbgXUIRhYUEGxzkVYPpsH2+3+lmPIQCaAEaFPbwRSOxCNAv3E32Lsfx0e/i5k7xHieIvwEVmk44O2I2GMmPuRP3677+gptDFYzQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fifWRo+JYpV13ABChztcP0esT8DdC3/q1irASM9cEDQ=;
 b=drLxdoMmoTxzYy5Ml5sHeMtadxGVI3nfMsWY1IIQh7IasR20c7SR375aL2MVz91LU93wDXdp2BEbBlDzqt9rxY6OLzvtreuWSNoarSyu1g50K1QVwYEgcxpk8yTenNnMX1ms9CGmDpWyGWCHSLwTUALw2acDmrX6xDnaxWC9TZ1TYzkLtd185YK5gMYP21tnwMLhTq3ZHOXyCQxwvw0iIu6//VXfc/7XLEMYQUMg2m3EKMOcHgVpq89I0tA50F3X4FwHK3UriXruCu8Z2dOqFvqGHOZhE2jYNElDcrGAbzycKHujfYZqpbiuowNZSzNnXpv5foDN2H/mNA5T3SwIdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fifWRo+JYpV13ABChztcP0esT8DdC3/q1irASM9cEDQ=;
 b=CDjGCFxCwJ5/yMm1+WxXzJ2Gf7UDUSg8qRd5Z5RK9aFhTh9RO6JTCvnOz3tQlZomAzx6Es4OBmtThKiGm9N2hFVNrCNxh/zqJq1jZ8K2jkDYvqtDEb+mt4GHulTJBejGUORSXuZhjdzOUkPhVewQ8XP9cTfZsXI4OXJnlJ7iYiagXjreaPcPHjfmuVJisX477LsXECEsyMgryzrzxm/niWY4KsFqVRP9xLPg0j4aMk8QN7doKTIEhJQ1Tc8Wdz2qVG5R+nXfeJdc/Tpgb3fBw0oVfORGiGJptfpc0A7UTqjCTRHJFrZlTCcTc7LKolenh6Ww8fIbXc3lhpdaLU35/w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4AA=
Date: Fri, 10 Sep 2021 11:43:36 +0000
Message-ID: <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
In-Reply-To: <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9594fe9d-b503-493b-a26a-08d974503a27
x-ms-traffictypediagnostic: AM0PR03MB6243:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB624364292FA38F4EED29A13DE7D69@AM0PR03MB6243.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 GpD66M+wr5H+GQbGY00Jg2ZB/87O3nmi6w7uXIyx5loc7+96CokMXBvfGY/x4S6CrqrtMt4PLQuJKfLjuYckQ00knnEcW7g4Mn2lsiNMbEauQCSKVNZRbfH1HMQb8algEFsFubQhh6NaPZ+su0BeosDuGbx3mFZ3MBQ76lA2V7KiiYuToZuGDLRHvEOmZSy3W0lahzioy/ZPTZHMCITYIWQUw9H/XOTcvLdMn9+0+qB6VFU7waVFfHHN/urdfDGEjsGXt7djJdDD4AxYvFQTqVTRtELZ1klrYBTeV9NpEY0tPnxv0l2oBxf2+3MN3L2cZlpaeFVXg5ufiBBEb8jhmrVIA9q9z9s/ZVMSGlqaTMq0uKkMuOFQygwhwXVOq56TntOGI27o9tDJopKovxmkWGNsLi9zT63o2M8cfTQjyC+gs75/nm4ckiO6TzEKFzLB7wDjhTQYR2LBmrm8yRhKy6jmD2KrzN2jITAIQip64x5pIgJL33+WrxWKKsAm6/lo2xP8xJ1lZD2uNdoT4c5M9a3MLOp1eRaN33u8oGymDLznggzBlYzccKBI76l3x+C8SAbQzlUUBupBs3Hb6FuxfncCSodMdX9iGMBonEWN86LuajZ6nSopu2Eeq1uEBaXk7adwYJfWSdOL5RVJi0rrLZhAzB/+2Mkspq1YzMcs75+HvJGms630wR0+iRPjAUw1tyRAL3Yo282uyy8bYDoIvLD/N9ZLJnZfn9/QRLFHwiA=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(346002)(376002)(396003)(366004)(4326008)(316002)(36756003)(54906003)(76116006)(66556008)(64756008)(66476007)(38070700005)(53546011)(71200400001)(110136005)(6506007)(26005)(6486002)(2906002)(186003)(8936002)(8676002)(55236004)(478600001)(31686004)(91956017)(2616005)(6512007)(38100700002)(83380400001)(31696002)(122000001)(66446008)(86362001)(5660300002)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ZS9sWlhxd2Z0dmpWRDUyU0UvZktBYngvOUkzZjdGZDRmYit3THNZc2lpdGd2?=
 =?utf-8?B?dTNSM2lsTEFsNWVIRnZieEYyemVDeTNpUTN5c3hsZmk1VEpzS0JhekErRnN6?=
 =?utf-8?B?MGpsV1puUjdKTUJkOUVDb21Bb0RnSmVyVitSNG1VWUtHRnpITjhqWnV1TEFz?=
 =?utf-8?B?Z0xWUWNJbjVjZ0pzTkpsbitWR1BtL1YzNVpLaFFIOVdDM2FURlJrTUVOTkdh?=
 =?utf-8?B?RXV1TVYyUlJ2SkU3Mit1L0x4b0JZNEpqV0lXYklaODByVE9YdFdZUkJhelFy?=
 =?utf-8?B?ZExCNHk4Y1lWV3B1RnQwUE9WRktoTjAwZ085VzQxRUZMeUdCU2ZkbEdvd0pB?=
 =?utf-8?B?blcrM2lnZDNic0VKNldyWWxTK1M1eFI1MEZhazJRVWtmakVMaENDTWVsVFND?=
 =?utf-8?B?MVh6TnFlREdyZ0pIRlJlaUVJaEZYRTJSOStPbjV6WW9LbkFhWjhMR3dOS2x3?=
 =?utf-8?B?bThib1ZZbVNsc1cyZUpkQmVIcGhxeHRFc3d5anBHaUMvY1NrRkUwS25lRVlM?=
 =?utf-8?B?YUpWVldNOTRiVUNxYlczUWJtb0l0ZWlSVXcrVFM0UjFlbmpOZ0hsaVdqUjhX?=
 =?utf-8?B?SmIyVERRSnhZNy9IeGQzZzhuU1hXbnAwMHF4N1FvNk50TmhlS2RDdDhzbzdS?=
 =?utf-8?B?MSs1clpFU20ydVN0MXZzNzVsZzBtY1piQUUwTmdUOUUzL280bXFnMjh1aDVk?=
 =?utf-8?B?cTNpSi9PQkVPTkJzQTRTSFk0elRvd1F6QzJ3YjMvcldNcVk3UUIwdTF6MXFC?=
 =?utf-8?B?a256L1dkR3FsZHZKbW1XbTZHT2dZUTNIS2lIZnE3T2F3RXgwclJzaHR6VzJq?=
 =?utf-8?B?RDdJSFl4d2NPWDJBcFR3ZFp6Mm9mTURla2dCcFozY0p1Y3k3bUwzcktPRmpt?=
 =?utf-8?B?c3pwcnZkcWRjNHExcjZEamhFd3RlcGtlQytTM3NkMFlNOXNMbk1QbENFVktJ?=
 =?utf-8?B?NzRleWJoS2NvdkpMMFE3RWVXQTRleTEzSnJIaGxuc1ZlT3NtSFh0MTVBUGtQ?=
 =?utf-8?B?bVR4RUZobjIraHNDd3BQMHhFMUZmSmZCcGlVS20vQ0tHSlNCd2J2OWNKSHhF?=
 =?utf-8?B?em9HTlJtQzAzWTZLM0pPbFR3VWhIMDdaWE5VcHlqSXRYQUNyMDRXQ0pJSERI?=
 =?utf-8?B?Wm5QKytyL3kzTitMR24wYzlMUWhRbys4bU9kV1A4YUZIdFRxVndRSHlPdTh5?=
 =?utf-8?B?d2VYb05TaStrQ1dzZWRQRTVaN3pOc20xZDRNaWE3d3Z4WHN3bFduV0dtMU5N?=
 =?utf-8?B?ZE5EdmxJTkdXcUU2Vk4xUW9wcVZQRGxhMWtsK0NmTDY0cEJhWlRabVQwZjZE?=
 =?utf-8?B?ZnM2QzZ4NjF3TkZTTFJkWEQ3MmdzRVJrbFJ5MGNEYkVMRU85cEtLMEJGaC9Y?=
 =?utf-8?B?RkR2b2xKT1JtdjdRa3dEYy9JZkJlRFkyTW1FbW9IVWRxdWpFcjhncURzNDYw?=
 =?utf-8?B?dVBYbVJkd3A3QlJtaFFhakVYRzJVS256Zi83eW96YlVMZTA4T0ZySEV3RUhy?=
 =?utf-8?B?ZmsxYVZSU0Y3dkdDRm5QNkhBNHdtNGRJVlVtbVRGa0xON2JGMzdOL3pPd0Nn?=
 =?utf-8?B?eUtLVDlxK2w0Yzd0QkFMemdFdWZ0MTBZYnZLNzk5ckxFQWl2YlQzRE8xVHNN?=
 =?utf-8?B?aE5uSXpJUklzd0hEV2t2ODMvWHVDLzlmbEYrbk96SWZVMytIYXJmd1p3MWQ5?=
 =?utf-8?B?WmE3N1U5aTVRbUEwWFhsTEpOZUxYcE1HSE93WUJzRkNNWWN0UkkxM253cmtS?=
 =?utf-8?B?QXZiM1BmWmNnbjV4NmxFcWJha2t4NTF1UGJqVlFncVpJQm9OdGFHN29MQ0h6?=
 =?utf-8?B?NUN3YXdhNHY4K3J1VzYrQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <47BE6A28DB284E4284C1732E00F077FC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9594fe9d-b503-493b-a26a-08d974503a27
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 11:43:37.0272
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: LvUVwNRQKBWmf2ll2dqEs1u3ymaooAFKuAl3nSB7i/Ncq6VY1WU146rZijymY3G5j3aZfWN706fshP4wD45DBnNYmkJ4fzuN4oNG8qFs3OVnjwZzLsxionY0E+tM7NX4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6243
X-Proofpoint-GUID: xUOhw-FnYBRUsCOeRlkPR_ZoHoKEfNIC
X-Proofpoint-ORIG-GUID: xUOhw-FnYBRUsCOeRlkPR_ZoHoKEfNIC
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0
 lowpriorityscore=0 suspectscore=0 impostorscore=0 adultscore=0
 phishscore=0 bulkscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100071

SGksIEp1bGllbiENCg0KT24gMDkuMDkuMjEgMjA6NDMsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4g
SGkgT2xla3NhbmRyLA0KPg0KPiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiBJbiBvcmRlciB2UENJIHRvIHdvcmsg
aXQgbmVlZHMgYWxsIGFjY2VzcyB0byBQQ0kgY29uZmlndXJhdGlvbiBzcGFjZQ0KPj4gKEVDQU0p
IHRvIGJlIHN5bmNocm9uaXplZCBhbW9uZyBhbGwgZW50aXRpZXMsIGUuZy4gaGFyZHdhcmUgZG9t
YWluIGFuZA0KPj4gZ3Vlc3RzLg0KPg0KPiBJIGFtIG5vdCBlbnRpcmVseSBzdXJlIHdoYXQgeW91
IG1lYW4gYnkgInN5bmNocm9uaXplZCIgaGVyZS4gQXJlIHlvdSByZWZlciB0byB0aGUgY29udGVu
dCBvZiB0aGUgY29uZmlndXJhdGlvbiBzcGFjZT8NCg0KV2UgbWFpbnRhaW4gaHdkb20ncyBhbmQg
Z3Vlc3QncyB2aWV3IG9mIHRoZSByZWdpc3RlcnMgd2UgYXJlIGludGVyZXN0ZWQgaW4NCg0KU28s
IHRvIGhhdmUgaHdkb20ncyB2aWV3IHdlIGFsc28gbmVlZCB0byB0cmFwIGl0cyBhY2Nlc3MgdG8g
dGhlIGNvbmZpZ3VyYXRpb24gc3BhY2UuDQoNClByb2JhYmx5ICJzeW5jaHJvbml6ZWQiIGlzIG5v
dCB0aGUgcmlnaHQgd29yZGluZyBoZXJlLg0KDQo+DQo+PiBGb3IgdGhhdCBpbXBsZW1lbnQgUENJ
IGhvc3QgYnJpZGdlIHNwZWNpZmljIGNhbGxiYWNrcyB0bw0KPj4gcHJvcGVybHkgc2V0dXAgdGhv
c2UgcmFuZ2VzIGRlcGVuZGluZyBvbiBwYXJ0aWN1bGFyIGhvc3QgYnJpZGdlDQo+PiBpbXBsZW1l
bnRhdGlvbi4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8
b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+PiAtLS0NCj4+IMKgIHhlbi9hcmNo
L2FybS9wY2kvZWNhbS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDExICsrKysrKysrKysrDQo+
PiDCoCB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIHwgMTYgKysrKysrKysrKysr
KysrKw0KPj4gwqAgeGVuL2FyY2gvYXJtL3ZwY2kuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCB8IDEzICsrKysrKysrKysrKysNCj4+IMKgIHhlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmjC
oMKgwqDCoMKgwqDCoMKgwqAgfMKgIDggKysrKysrKysNCj4+IMKgIDQgZmlsZXMgY2hhbmdlZCwg
NDggaW5zZXJ0aW9ucygrKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL2Vj
YW0uYyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+PiBpbmRleCA5MWM2OTFiNDFmZGYuLjky
ZWNiMmUwNzYyYiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+PiAr
KysgYi94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0KPj4gQEAgLTQyLDYgKzQyLDE2IEBAIHZvaWQg
X19pb21lbSAqcGNpX2VjYW1fbWFwX2J1cyhzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2Us
DQo+PiDCoMKgwqDCoMKgIHJldHVybiBiYXNlICsgKFBDSV9ERVZGTihzYmRmX3QuZGV2LCBzYmRm
X3QuZm4pIDw8IGRldmZuX3NoaWZ0KSArIHdoZXJlOw0KPj4gwqAgfQ0KPj4gwqAgK3N0YXRpYyBp
bnQgcGNpX2VjYW1fcmVnaXN0ZXJfbW1pb19oYW5kbGVyKHN0cnVjdCBkb21haW4gKmQsDQo+PiAr
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2Us
DQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgbW1pb19oYW5kbGVy
X29wcyAqb3BzKQ0KPj4gK3sNCj4+ICvCoMKgwqAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpj
ZmcgPSBicmlkZ2UtPnN5c2RhdGE7DQo+PiArDQo+PiArwqDCoMKgIHJlZ2lzdGVyX21taW9faGFu
ZGxlcihkLCBvcHMsIGNmZy0+cGh5c19hZGRyLCBjZmctPnNpemUsIE5VTEwpOw0KPg0KPiBXZSBo
YXZlIGEgZml4ZWQgYXJyYXkgZm9yIGhhbmRsaW5nIHRoZSBNTUlPIGhhbmRsZXJzLg0KDQpEaWRu
J3Qga25vdyB0aGF0Og0KDQp4ZW4vaW5jbHVkZS9hc20tYXJtL21taW8uaDoyNzojZGVmaW5lIE1B
WF9JT19IQU5ETEVSwqAgMTYNCg0KPiBTbyB5b3UgbmVlZCB0byBtYWtlIHN1cmUgd2UgaGF2ZSBl
bm91Z2ggc3BhY2UgaW4gaXQgdG8gc3RvcmUgb25lIGhhbmRsZXIgcGVyIGhhbmRsZXIuDQo+DQo+
IFRoaXMgaXMgcXVpdGUgc2ltaWxhciB0byB0aGUgcHJvYmxlbSB3ZSBoYWQgd2l0aCB0aGUgcmUt
ZGlzdHJpYnVvciBvbiBHSUN2My4gSGF2ZSBhIGxvb2sgdGhlcmUgdG8gc2VlIGhvdyB3ZSBkZWFs
dCB3aXRoIGl0Lg0KDQpDb3VsZCB5b3UgcGxlYXNlIHBvaW50IG1lIHRvIHRoYXQgc29sdXRpb24/
IEkgY2FuIG9ubHkgc2VlDQoNCiDCoMKgwqAgLyogUmVnaXN0ZXIgbW1pbyBoYW5kbGUgZm9yIHRo
ZSBEaXN0cmlidXRvciAqLw0KIMKgwqDCoCByZWdpc3Rlcl9tbWlvX2hhbmRsZXIoZCwgJnZnaWNf
ZGlzdHJfbW1pb19oYW5kbGVyLCBkLT5hcmNoLnZnaWMuZGJhc2UsDQogwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgU1pfNjRLLCBOVUxMKTsNCg0KIMKg
wqDCoCAvKg0KIMKgwqDCoMKgICogUmVnaXN0ZXIgbW1pbyBoYW5kbGVyIHBlciBjb250aWd1b3Vz
IHJlZ2lvbiBvY2N1cGllZCBieSB0aGUNCiDCoMKgwqDCoCAqIHJlZGlzdHJpYnV0b3JzLiBUaGUg
aGFuZGxlciB3aWxsIHRha2UgY2FyZSB0byBjaG9vc2Ugd2hpY2gNCiDCoMKgwqDCoCAqIHJlZGlz
dHJpYnV0b3IgaXMgdGFyZ2V0ZWQuDQogwqDCoMKgwqAgKi8NCiDCoMKgwqAgZm9yICggaSA9IDA7
IGkgPCBkLT5hcmNoLnZnaWMubnJfcmVnaW9uczsgaSsrICkNCiDCoMKgwqAgew0KIMKgwqDCoMKg
wqDCoMKgIHN0cnVjdCB2Z2ljX3JkaXN0X3JlZ2lvbiAqcmVnaW9uID0gJmQtPmFyY2gudmdpYy5y
ZGlzdF9yZWdpb25zW2ldOw0KDQogwqDCoMKgwqDCoMKgwqAgcmVnaXN0ZXJfbW1pb19oYW5kbGVy
KGQsICZ2Z2ljX3JkaXN0cl9tbWlvX2hhbmRsZXIsDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZWdpb24tPmJhc2UsIHJlZ2lvbi0+
c2l6ZSwgcmVnaW9uKTsNCiDCoMKgwqAgfQ0Kd2hpY2ggSU1PIGRvZXNuJ3QgY2FyZSBhYm91dCB0
aGUgbnVtYmVyIG9mIE1NSU9zIHdlIGNhbiBoYW5kbGUNCg0KPg0KPj4gK8KgwqDCoCByZXR1cm4g
MDsNCj4+ICt9DQo+PiArDQo+PiDCoCAvKiBFQ0FNIG9wcyAqLw0KPj4gwqAgY29uc3Qgc3RydWN0
IHBjaV9lY2FtX29wcyBwY2lfZ2VuZXJpY19lY2FtX29wcyA9IHsNCj4+IMKgwqDCoMKgwqAgLmJ1
c19zaGlmdMKgID0gMjAsDQo+PiBAQCAtNDksNiArNTksNyBAQCBjb25zdCBzdHJ1Y3QgcGNpX2Vj
YW1fb3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0KPj4gwqDCoMKgwqDCoMKgwqDCoMKgIC5t
YXBfYnVzwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gcGNpX2VjYW1fbWFwX2J1cywN
Cj4+IMKgwqDCoMKgwqDCoMKgwqDCoCAucmVhZMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCA9IHBjaV9nZW5lcmljX2NvbmZpZ19yZWFkLA0KPj4gwqDCoMKgwqDCoMKgwqDCoMKg
IC53cml0ZcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgPSBwY2lfZ2VuZXJpY19j
b25maWdfd3JpdGUsDQo+PiArwqDCoMKgwqDCoMKgwqAgLnJlZ2lzdGVyX21taW9faGFuZGxlcsKg
ID0gcGNpX2VjYW1fcmVnaXN0ZXJfbW1pb19oYW5kbGVyLA0KPj4gwqDCoMKgwqDCoCB9DQo+PiDC
oCB9Ow0KPj4gwqAgZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMgYi94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+PiBpbmRleCBkMmZlZjU0
NzZiOGUuLmE4OTExMmJmYmI3YyAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wY2kvcGNp
LWhvc3QtY29tbW9uLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMNCj4+IEBAIC0zMTgsNiArMzE4LDIyIEBAIHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqcGNpX2Zp
bmRfaG9zdF9icmlkZ2Vfbm9kZShzdHJ1Y3QgZGV2aWNlICpkZXYpDQo+PiDCoMKgwqDCoMKgIH0N
Cj4+IMKgwqDCoMKgwqAgcmV0dXJuIGJyaWRnZS0+ZHRfbm9kZTsNCj4+IMKgIH0NCj4+ICsNCj4+
ICtpbnQgcGNpX2hvc3RfaXRlcmF0ZV9icmlkZ2VzKHN0cnVjdCBkb21haW4gKmQsDQo+PiArwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW50
ICgqY2xiKShzdHJ1Y3QgZG9tYWluICpkLA0KPg0KPiBOSVQ6IFdlIHRlbmQgdG8gbmFtZSBjYWxs
YmFjayB2YXJpYWJsZSAnY2InLg0KU3VyZQ0KPg0KPiBDaGVlcnMsDQo+DQpUaGFuayB5b3UsDQoN
Ck9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:53:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:53:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184314.332913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOf6C-0006hU-9X; Fri, 10 Sep 2021 11:53:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184314.332913; Fri, 10 Sep 2021 11:53:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOf6C-0006hN-6O; Fri, 10 Sep 2021 11:53:40 +0000
Received: by outflank-mailman (input) for mailman id 184314;
 Fri, 10 Sep 2021 11:53:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOf6A-0006hH-O6
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:53:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOf69-0005sP-NM; Fri, 10 Sep 2021 11:53:37 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOf69-0003jA-H0; Fri, 10 Sep 2021 11:53:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=E432K6PZQY8zmduY1UeenjxjP++XpHejF65LrVD4sdA=; b=n6trwaFYHdu7HF3uX76GrPatXS
	NdMw6MuM9QzAZPoeyEKvFUK/8NkKyins23FfGeHE0EzHG6uwjv4h2an9wdS904oYpho2620r9Qsec
	QaRDc/tWS89xq0YtF6MDibDslBkDKcejrSSoSxKNpLxhlh59HIp5IIAY+/jRsLrQRDhI=;
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
To: Rahul Singh <Rahul.Singh@arm.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
 <1f4a0284-302e-47e9-6958-47dca449d580@xen.org>
 <332992DF-56A9-42CF-B266-B46E4622A03E@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <39a8e71a-f73b-e704-622d-d38cdad70066@xen.org>
Date: Fri, 10 Sep 2021 12:53:35 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <332992DF-56A9-42CF-B266-B46E4622A03E@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 12:22, Rahul Singh wrote:
> Hi Julien

Hi Rahul,
>> On 7 Sep 2021, at 10:05 am, Julien Grall <julien@xen.org> wrote:
>> On 19/08/2021 13:02, Rahul Singh wrote:
>>> +#include <xen/rwlock.h>
>>> +#include <xen/sched.h>
>>> +#include <xen/vmap.h>
>>> +
>>> +/*
>>> + * List for all the pci host bridges.
>>> + */
>>> +
>>> +static LIST_HEAD(pci_host_bridges);
>>> +
>>> +static atomic_t domain_nr = ATOMIC_INIT(-1);
>>> +
>>> +bool dt_pci_parse_bus_range(struct dt_device_node *dev,
>>> +                            struct pci_config_window *cfg)
>>
>> Aside, "pci_config_window", the function is not Arm specific. Would it be possible to consider to introduce "struct resource" in Xen so this function can be moved in common/device_tree.c?
> 
> I can introduce the "struct resource” but I am not sure whether "struct resource” will be
> useful later point in time. What I prefer as of now, we can have this function and we can
> move this to common/device_tree.c once we have the requirement for "struct resource”.

TBH, I wasn't asking about using "struct resource". This was simply a 
suggestion how to make the code common.

What I am more interested is moving this function in common code so it 
can be re-used by RISC-v in the future. I don't particularly mind 
whether this is using "struct resource" or a different structure.

>>> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
>>> new file mode 100644
>>> index 0000000000..13d0f7f999
>>> --- /dev/null
>>> +++ b/xen/arch/arm/pci/pci-host-generic.c
>>> @@ -0,0 +1,55 @@
>>> +/*
>>> + * Copyright (C) 2021 Arm Ltd.
>>> + *
>>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>>> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License version 2 as
>>> + * published by the Free Software Foundation.
>>> + *
>>> + * This program is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>> + */
>>> +
>>> +#include <asm/device.h>
>>> +#include <xen/pci.h>
>>> +#include <asm/pci.h>
>>> +
>>> +static const struct dt_device_match gen_pci_dt_match[] = {
>>> +    { .compatible = "pci-host-ecam-generic" },
>>> +    { },
>>> +};
>>> +
>>> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
>>> +{
>>> +    const struct dt_device_match *of_id;
>>> +
>>> +    of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);
>>
>> This seems to be a bit pointless to me as you already know the compatible (there is only one possible...).
>   As of now we are only implementing the "pci-host-ecam-generic” compatible PCI, but in future we might
> need to implement the other compatible like Linux see as below.
> 
> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/pci-host-generic.c#L59

Right... You know the node matched and you don't need any data. So this 
is a bit pointless to have to walk again the structure for just printing 
the compatible that matched.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 11:53:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 11:53:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184315.332923 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOf6I-0006zV-Hf; Fri, 10 Sep 2021 11:53:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184315.332923; Fri, 10 Sep 2021 11:53:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOf6I-0006zM-Ed; Fri, 10 Sep 2021 11:53:46 +0000
Received: by outflank-mailman (input) for mailman id 184315;
 Fri, 10 Sep 2021 11:53:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOf6G-0006yc-QC
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 11:53:44 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.5.83]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bdfebdb3-122d-11ec-b229-12813bfff9fa;
 Fri, 10 Sep 2021 11:53:42 +0000 (UTC)
Received: from DU2PR04CA0291.eurprd04.prod.outlook.com (2603:10a6:10:28c::26)
 by AM0PR08MB3107.eurprd08.prod.outlook.com (2603:10a6:208:60::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 11:53:40 +0000
Received: from DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:28c:cafe::54) by DU2PR04CA0291.outlook.office365.com
 (2603:10a6:10:28c::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 11:53:40 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT017.mail.protection.outlook.com (10.152.20.114) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 11:53:40 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Fri, 10 Sep 2021 11:53:40 +0000
Received: from b2d6a23dd4de.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0B5FA5CC-FE3C-4297-AC41-7DB5572AF8C1.1; 
 Fri, 10 Sep 2021 11:53:33 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b2d6a23dd4de.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 11:53:33 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6791.eurprd08.prod.outlook.com (2603:10a6:20b:392::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 11:53:32 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 11:53:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdfebdb3-122d-11ec-b229-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3e1nTa5OseKNzWf9vNS7Gg8KVYr7EC9nTFiIJzIoA2w=;
 b=8/8UiGWG5DiEN7ov71/NyTgRFI8FazOfftPB5yuw/2EdUkVzitZrkZzgkuW14QB5RG2x34T1fkE10oF/ZejUDYtO0EhtkO37V82Kz1fSN++Alu2+UHHewp8Hqyh5EbwvAr2cF7a6QadhIhnbGqUUq2w1EQWEZ9S0+3cP0eT5Wwo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 36e402446b5e4f4b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lIfr4QctgNe2KKZ07SEJkMcXUM4SaM17r4suTayOVOq71Zfa4EsYmXvFEDsTpvaYllby1bG2HAEt3yCm970LVNYVghUeOBFnXAx5PM2ptoIC3OWfxHAQBlKVGY8vOYJuisk6qKVA5UBkQ/o1O863hXdYePp1zstBtHKJ/Js/jT3nQiL68zGJ7FKZhaRKTNopq/pc3H45d7WB9mgYAcRpt2Y4JBFT42S+3nuNo7s0e75jJcUQ5i/rSxk1kVJ+d/NmK7NGZNXZN+hIJk+IA7ors7EavWsVpKNneXdL1KW8MBs+JZ/YNyld2N/oA/ekh/A2KYIj4SqA7CEUWx3Qu4GKKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3e1nTa5OseKNzWf9vNS7Gg8KVYr7EC9nTFiIJzIoA2w=;
 b=GF8iUNSPqQus80DLB7a/Ya7h6ssvWptA2ROSF2rp40QgEWHOtBW88KmDRew7HcsxFIYq3xMIIcM++jVqje0R8BT4AWnDQyQ3S3H/Qm8GUbJY4ehhuDoJmdbLKq6eSDJ38tLoqI1hCwbvo7QX38uamUO6sk+HNTYrmgxAfVpAWl3zGLgFBI9G9tlN615ph04sC5BriAYeJ8akyr4Ffr+AL7LPAlwahME8FD1eZ8mKMQuRW7y5jSiBXMvqyY/OmHnCzQRpCcjMQdwoOa0oLaMgs9y6Ja09wzE4rwQczN5iFOF/gxdTYhDV1lj+d0uSxT/LEgxeqWDaVNPx12PN/xS9OQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3e1nTa5OseKNzWf9vNS7Gg8KVYr7EC9nTFiIJzIoA2w=;
 b=8/8UiGWG5DiEN7ov71/NyTgRFI8FazOfftPB5yuw/2EdUkVzitZrkZzgkuW14QB5RG2x34T1fkE10oF/ZejUDYtO0EhtkO37V82Kz1fSN++Alu2+UHHewp8Hqyh5EbwvAr2cF7a6QadhIhnbGqUUq2w1EQWEZ9S0+3cP0eT5Wwo=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXlPKqrI+7ld7PxEaIC1sXiQa2TauccWEAgADZrIA=
Date: Fri, 10 Sep 2021 11:53:32 +0000
Message-ID: <D716F747-6055-4520-B4DB-189A8AF3945C@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091540470.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091540470.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c15bb444-4908-4304-d8c2-08d97451a1c7
x-ms-traffictypediagnostic: AS8PR08MB6791:|AM0PR08MB3107:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB310709FE26EAC12F3F62092CFCD69@AM0PR08MB3107.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 16oRcyfGi88BarJAEnwUyr9jn/+FUQzHJ/haO+hqQAxIIoE5MFCIhfbvxLRHTPYUopqfbE8IdMWFUGGp1KvqdzgND2N9/opnKxXrFDdCQNT6JnLh1TZijpT50vssWtttPRgZQeSuTx1a2eayRbZdPqrglrJrQwQ4EWnKrSm9f8yzgiMBl5NvpeqOU+1XM+pTh5eqsWdDA6drEDqfL/i+bju8VNNO0Y9PEW8MsvdIUZVT7FzQIwdfpl5cJJCZTopd7tX4x77U5T628edEJDn6LjpoKfls7eoAypaWObMJ9MXunFPI3JCh1sP3qGMNxLDFlxJbyjKloMvkgy4VCjfys8Emv9RmY94/Y376NtgUlaD6PRLmn5VITE1rYSjCd/5+9hcYTSVrRI+f4g4sht118Toc41lbKwk1oPl4LzXYgpHQTT9IiEreCiE5ClBBgBL/5XxPd1rxy2L7St68o0SZIHALVQbME0hN7UoP/xUrO5rMI6tk/4bSKXuoVi/F0HUgjgiwxkvsMFWhzokfJlgbKLoZ6fHE3MTfAJ0XsGNCAaftCJuwxwlFqmrYXZ3C/2hYdpurTTLTKX22ei2oroLIDTYexo5zYGp97jGt4gZq0utiusxGcmoJ9xeZhEWn3q8FcoCt/Twb3rHK0MaxXQCi1iSX3Z87it9FOLuGc+Wd8nl4PHrgnDUFM5c3kz+ly+bvcrVVSep0Lap6+5CC2ltHEh6GNWyqZ8d1f016nnsiQURz9jBWDoR6SFrdi/F9S3+JBbSkh1yFPEkKJX5WzaiS/fxhG41I0Tz8j6HLtpG0idZbqmrPivzw9A2MQfG7o0XIxHntaynl1jZZ+L1q9gJV5BQOWah/B8evT/AAiE+HqorQ4UPIhuBapCeZYslKuIJR
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(346002)(366004)(39860400002)(396003)(36756003)(2616005)(186003)(66946007)(2906002)(71200400001)(316002)(478600001)(66446008)(64756008)(66556008)(66476007)(76116006)(91956017)(966005)(8936002)(54906003)(53546011)(33656002)(5660300002)(38100700002)(4326008)(122000001)(26005)(6506007)(38070700005)(6916009)(8676002)(6512007)(86362001)(83380400001)(6486002)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <C88DF19F4C4871418FC50B4B7164C757@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6791
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9076e8a3-3318-49f5-adbe-08d974519d0f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MrTKbAKlBj4ycimaeIw0b1j4bs3oCZyZYidIFLCHd5R6/i92lCUPmWCJg8hFmz8KrTsLq0DmRmvfbhyoetf1L7ct2o4xm2sQh1+Gln4DSLTt7wqofLPwH0LoW78h2U8zOHDnA1LdCwHz/KHR4ceeOU+22Q6U5bKNcyYSjY++21skDvUgZDT9EZmr1r1zhVmKLLr+LrFS7CwJntbjRSC1tQCK2mPGpclwrA0a5XqBBrQAs1Fjs1jdtLNJ6KjCYaIaLtOAM769GVFAiH1HXBmtkOIJeDeE2SNVROJwM2Q0iIv8XblAql24p4VmTdZoktlv2n1lhZuN5NYlhyEfjfsW4MIvT0p1JVEm5etFLZxUBYSjfAhudz10YtYf4EJRbXWZS5+ruh16hs0JRMPJN1y/E4iHbqhLEGcrb9tUSEWE+2kadoGNMB5RkfnPBSzZ2kxRAtiJW62HitWw+B6a9ZsNKGEhPjQtRGl/2wGHc/0NzC6zhghWwlhjfRPhJOxzh6wCsXY0STrIdyw4uJ7BlmGDG1QhLvrFRyAEaQPG0j9hYHaD6JeM+yZqSsz5vRv3d2865gxwWtf4Hi21OtaWp8jOvz0gLX0PSe1Z1nB9fevV1OpVU3RBnUcrKnrhq515Vk3uW7so+2+7qo+2m9+Tk3+HnSI4Geqmd8MA/ABxEXjac6+xY0fdFpDZAgL/2E7NZsGwe7W3/QkcwRjfk263WSfDtBbXGDb5MIqiq1Ua8Mq6mX5oGEGptHgTHbBVfkKC5RCvh6xaQwRcGrh/7OXMumUpHcSiIHPtLIEtSARYKuhwMqbs0e6KZZbpJovvGCN5xNRn2g5Z7pVjUUzoxQCHWlcBnQo6Q9DlUo5D4rp7hK19FN8=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(346002)(376002)(36840700001)(46966006)(26005)(107886003)(86362001)(36756003)(4326008)(82740400003)(966005)(82310400003)(336012)(54906003)(36860700001)(70206006)(356005)(53546011)(2616005)(6486002)(33656002)(47076005)(6862004)(6506007)(6512007)(8936002)(83380400001)(2906002)(478600001)(8676002)(5660300002)(316002)(70586007)(81166007)(186003)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 11:53:40.4947
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c15bb444-4908-4304-d8c2-08d97451a1c7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3107

SGkgU3RlZmFubywNCg0KPiBPbiA5IFNlcCAyMDIxLCBhdCAxMTo1NCBwbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFRodSwgMTkg
QXVnIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4gWEVOIGR1cmluZyBib290IHdpbGwgcmVh
ZCB0aGUgUENJIGRldmljZSB0cmVlIG5vZGUg4oCccmVn4oCdIHByb3BlcnR5DQo+PiBhbmQgd2ls
bCBtYXAgdGhlIFBDSSBjb25maWcgc3BhY2UgdG8gdGhlIFhFTiBtZW1vcnkuDQo+PiANCj4+IEFz
IG9mIG5vdyAicGNpLWhvc3QtZWNhbS1nZW5lcmljIiBjb21wYXRpYmxlIGJvYXJkIGlzIHN1cHBv
cnRlZC4NCj4+IA0KPj4gImxpbnV4LHBjaS1kb21haW4iIGRldmljZSB0cmVlIHByb3BlcnR5IGFz
c2lnbnMgYSBmaXhlZCBQQ0kgZG9tYWluDQo+PiBudW1iZXIgdG8gYSBob3N0IGJyaWRnZSwgb3Ro
ZXJ3aXNlIGFuIHVuc3RhYmxlIChhY3Jvc3MgYm9vdHMpIHVuaXF1ZQ0KPj4gbnVtYmVyIHdpbGwg
YmUgYXNzaWduZWQgYnkgTGludXguVGhpcyBwcm9wZXJ0eSBoYXMgdG8gYmUgaW4gc3luYyB3aXRo
DQo+PiBYRU4gdG8gYWNjZXNzIHRoZSBQQ0kgZGV2aWNlcy4NCj4+IA0KPj4gWEVOIHdpbGwgcmVh
ZCB0aGUg4oCcbGludXgscGNpLWRvbWFpbuKAnSBwcm9wZXJ0eSBmcm9tIHRoZSBkZXZpY2UgdHJl
ZSBub2RlDQo+PiBhbmQgY29uZmlndXJlIHRoZSBob3N0IGJyaWRnZSBzZWdtZW50IG51bWJlciBh
Y2NvcmRpbmdseS4gSWYgdGhpcw0KPj4gcHJvcGVydHkgaXMgbm90IGF2YWlsYWJsZSBYRU4gd2ls
bCBhbGxvY2F0ZSB0aGUgdW5pcXVlIHNlZ21lbnQgbnVtYmVyDQo+PiB0byB0aGUgaG9zdCBicmlk
Z2UuDQo+PiANCj4+IGR0X2dldF9wY2lfZG9tYWluX25yKC4uKSBhbmQgZHRfcGNpX2J1c19maW5k
X2RvbWFpbl9uciguLikgYXJlIGRpcmVjdGx5DQo+PiBpbXBvcnRlZCBmcm9tIHRoZSBMaW51eCBz
b3VyY2UgdHJlZS4NCj4+IA0KPj4gU2lnbmVkLW9mZi1ieTogUmFodWwgU2luZ2ggPHJhaHVsLnNp
bmdoQGFybS5jb20+DQo+PiAtLS0NCj4+IHhlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUgICAgICAg
ICAgIHwgICAyICsNCj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMgIHwgMjYx
ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtZ2VuZXJpYy5jIHwgIDU1ICsrKysrKw0KPj4geGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCAg
ICAgICAgICAgfCAgMjggKysrDQo+PiA0IGZpbGVzIGNoYW5nZWQsIDM0NiBpbnNlcnRpb25zKCsp
DQo+PiBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24u
Yw0KPj4gY3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtZ2VuZXJp
Yy5jDQo+PiANCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlIGIveGVu
L2FyY2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4gaW5kZXggYTllZTBiOWI0NC4uZjNkOTdmODU5ZSAx
MDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+ICsrKyBiL3hlbi9h
cmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+IEBAIC0xLDIgKzEsNCBAQA0KPj4gb2JqLXkgKz0gcGNp
Lm8NCj4+IG9iai15ICs9IHBjaS1hY2Nlc3Mubw0KPj4gK29iai15ICs9IHBjaS1ob3N0LWdlbmVy
aWMubw0KPj4gK29iai15ICs9IHBjaS1ob3N0LWNvbW1vbi5vDQo+PiBkaWZmIC0tZ2l0IGEveGVu
L2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtY29tbW9uLmMNCj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+PiBpbmRleCAwMDAwMDAwMDAw
Li45ZGQ5YjAyMjcxDQo+PiAtLS0gL2Rldi9udWxsDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vcGNp
L3BjaS1ob3N0LWNvbW1vbi5jDQo+PiBAQCAtMCwwICsxLDI2MSBAQA0KPj4gKy8qDQo+PiArICog
Q29weXJpZ2h0IChDKSAyMDIxIEFybSBMdGQuDQo+PiArICoNCj4+ICsgKiBCYXNlZCBvbiBMaW51
eCBkcml2ZXJzL3BjaS9lY2FtLmMNCj4+ICsgKiBDb3B5cmlnaHQgMjAxNiBCcm9hZGNvbS4NCj4+
ICsgKg0KPj4gKyAqIEJhc2VkIG9uIExpbnV4IGRyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpLWhv
c3QtY29tbW9uLmMNCj4+ICsgKiBCYXNlZCBvbiBMaW51eCBkcml2ZXJzL3BjaS9jb250cm9sbGVy
L3BjaS1ob3N0LWdlbmVyaWMuYw0KPj4gKyAqIENvcHlyaWdodCAoQykgMjAxNCBBUk0gTGltaXRl
ZCBXaWxsIERlYWNvbiA8d2lsbC5kZWFjb25AYXJtLmNvbT4NCj4+ICsgKg0KPj4gKyAqDQo+PiAr
ICogVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0
IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5l
cmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24gMiBhcw0KPj4gKyAqIHB1Ymxpc2hlZCBieSB0aGUg
RnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLg0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlz
IGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICog
YnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFu
dHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFCSUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFS
IFBVUlBPU0UuICBTZWUgdGhlDQo+PiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9y
IG1vcmUgZGV0YWlscy4NCj4+ICsgKg0KPj4gKyAqIFlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBh
IGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlDQo+PiArICogYWxvbmcgd2l0
aCB0aGlzIHByb2dyYW0uICBJZiBub3QsIHNlZSA8aHR0cDovL3d3dy5nbnUub3JnL2xpY2Vuc2Vz
Lz4uDQo+PiArICovDQo+PiArDQo+PiArI2luY2x1ZGUgPHhlbi9pbml0Lmg+DQo+PiArI2luY2x1
ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5jbHVkZSA8YXNtL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDx4
ZW4vcndsb2NrLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9zY2hlZC5oPg0KPj4gKyNpbmNsdWRlIDx4
ZW4vdm1hcC5oPg0KPj4gKw0KPj4gKy8qDQo+PiArICogTGlzdCBmb3IgYWxsIHRoZSBwY2kgaG9z
dCBicmlkZ2VzLg0KPj4gKyAqLw0KPj4gKw0KPj4gK3N0YXRpYyBMSVNUX0hFQUQocGNpX2hvc3Rf
YnJpZGdlcyk7DQo+PiArDQo+PiArc3RhdGljIGF0b21pY190IGRvbWFpbl9uciA9IEFUT01JQ19J
TklUKC0xKTsNCj4+ICsNCj4+ICtib29sIGR0X3BjaV9wYXJzZV9idXNfcmFuZ2Uoc3RydWN0IGR0
X2RldmljZV9ub2RlICpkZXYsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj
dCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnKQ0KPj4gK3sNCj4+ICsgICAgY29uc3QgX19iZTMyICpj
ZWxsczsNCj4+ICsgICAgdWludDMyX3QgbGVuOw0KPj4gKw0KPj4gKyAgICBjZWxscyA9IGR0X2dl
dF9wcm9wZXJ0eShkZXYsICJidXMtcmFuZ2UiLCAmbGVuKTsNCj4+ICsgICAgLyogYnVzLXJhbmdl
IHNob3VsZCBhdCBsZWFzdCBiZSAyIGNlbGxzICovDQo+PiArICAgIGlmICggIWNlbGxzIHx8IChs
ZW4gPCAoc2l6ZW9mKCpjZWxscykgKiAyKSkgKQ0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0K
Pj4gKw0KPj4gKyAgICBjZmctPmJ1c25fc3RhcnQgPSBkdF9uZXh0X2NlbGwoMSwgJmNlbGxzKTsN
Cj4+ICsgICAgY2ZnLT5idXNuX2VuZCA9IGR0X25leHRfY2VsbCgxLCAmY2VsbHMpOw0KPj4gKw0K
Pj4gKyAgICByZXR1cm4gdHJ1ZTsNCj4+ICt9DQo+PiArDQo+PiArc3RhdGljIGlubGluZSB2b2lk
IF9faW9tZW0gKnBjaV9yZW1hcF9jZmdzcGFjZShwYWRkcl90IHN0YXJ0LCBzaXplX3QgbGVuKQ0K
Pj4gK3sNCj4+ICsgICAgcmV0dXJuIGlvcmVtYXBfbm9jYWNoZShzdGFydCwgbGVuKTsNCj4+ICt9
DQo+PiArDQo+PiArc3RhdGljIHZvaWQgcGNpX2VjYW1fZnJlZShzdHJ1Y3QgcGNpX2NvbmZpZ193
aW5kb3cgKmNmZykNCj4+ICt7DQo+PiArICAgIGlmICggY2ZnLT53aW4gKQ0KPj4gKyAgICAgICAg
aW91bm1hcChjZmctPndpbik7DQo+PiArDQo+PiArICAgIHhmcmVlKGNmZyk7DQo+PiArfQ0KPj4g
Kw0KPj4gK3N0YXRpYyBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmdlbl9wY2lfaW5pdChzdHJ1
Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgaW50IGVjYW1fcmVnX2lkeCkNCj4gDQo+IElmIGl0IGlzIG9ubHkg
Y2FsbGVkIGF0IGluaXQgdGltZSwgdGhlbiB0aGUgZnVuY3Rpb24gc2hvdWxkIGJlIF9faW5pdA0K
DQpBY2suDQoNCj4gDQo+IA0KPj4gK3sNCj4+ICsgICAgaW50IGVycjsNCj4+ICsgICAgc3RydWN0
IHBjaV9jb25maWdfd2luZG93ICpjZmc7DQo+PiArICAgIHBhZGRyX3QgYWRkciwgc2l6ZTsNCj4+
ICsNCj4+ICsgICAgY2ZnID0geHphbGxvYyhzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cpOw0KPj4g
KyAgICBpZiAoICFjZmcgKQ0KPj4gKyAgICAgICAgcmV0dXJuIE5VTEw7DQo+PiArDQo+PiArICAg
IGVyciA9IGR0X3BjaV9wYXJzZV9idXNfcmFuZ2UoZGV2LCBjZmcpOw0KPj4gKyAgICBpZiAoICFl
cnIgKSB7DQo+PiArICAgICAgICBjZmctPmJ1c25fc3RhcnQgPSAwOw0KPj4gKyAgICAgICAgY2Zn
LT5idXNuX2VuZCA9IDB4ZmY7DQo+PiArICAgICAgICBwcmludGsoWEVOTE9HX0VSUiAiJXM6Tm8g
YnVzIHJhbmdlIGZvdW5kIGZvciBwY2kgY29udHJvbGxlclxuIiwNCj4+ICsgICAgICAgICAgICAg
ICBkdF9ub2RlX2Z1bGxfbmFtZShkZXYpKTsNCj4+ICsgICAgfSBlbHNlIHsNCj4+ICsgICAgICAg
IGlmICggY2ZnLT5idXNuX2VuZCA+IGNmZy0+YnVzbl9zdGFydCArIDB4ZmYgKQ0KPj4gKyAgICAg
ICAgICAgIGNmZy0+YnVzbl9lbmQgPSBjZmctPmJ1c25fc3RhcnQgKyAweGZmOw0KPiANCj4gSXMg
dGhpcyBhIGhhcmQgbGltaXQgaW4gdGhlIHNwZWNpZmljYXRpb24/IE9yIGlzIGl0IGEgbGltaXQg
aW4gdGhlIFhlbg0KPiBpbXBsZW1lbnRhdGlvbj8NCg0KSSBqdXN0IHRha2UgdGhlIHJlZmVyZW5j
ZSBmcm9tIExpbnV4IGNvZGUuDQpodHRwczovL2VsaXhpci5ib290bGluLmNvbS9saW51eC9sYXRl
c3Qvc291cmNlL2RyaXZlcnMvcGNpL29mLmMjTDMwNg0KDQo+IA0KPiANCj4+ICsgICAgfQ0KPj4g
Kw0KPj4gKyAgICAvKiBQYXJzZSBvdXIgUENJIGVjYW0gcmVnaXN0ZXIgYWRkcmVzcyovDQo+ICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeIHNwYWNlDQo+IA0KQWNr
Lg0KPj4gKyAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJlc3MoZGV2LCBlY2FtX3JlZ19pZHgs
ICZhZGRyLCAmc2l6ZSk7DQo+PiArICAgIGlmICggZXJyICkNCj4+ICsgICAgICAgIGdvdG8gZXJy
X2V4aXQ7DQo+PiArDQo+PiArICAgIGNmZy0+cGh5c19hZGRyID0gYWRkcjsNCj4+ICsgICAgY2Zn
LT5zaXplID0gc2l6ZTsNCj4+ICsNCj4+ICsgICAgLyoNCj4+ICsgICAgICogT24gNjQtYml0IHN5
c3RlbXMsIHdlIGRvIGEgc2luZ2xlIGlvcmVtYXAgZm9yIHRoZSB3aG9sZSBjb25maWcgc3BhY2UN
Cj4+ICsgICAgICogc2luY2Ugd2UgaGF2ZSBlbm91Z2ggdmlydHVhbCBhZGRyZXNzIHJhbmdlIGF2
YWlsYWJsZS4gIE9uIDMyLWJpdCwgd2UNCj4+ICsgICAgICogaW9yZW1hcCB0aGUgY29uZmlnIHNw
YWNlIGZvciBlYWNoIGJ1cyBpbmRpdmlkdWFsbHkuDQo+PiArICAgICAqDQo+PiArICAgICAqIEFz
IG9mIG5vdyBvbmx5IDY0LWJpdCBpcyBzdXBwb3J0ZWQgMzItYml0IGlzIG5vdCBzdXBwb3J0ZWQu
DQo+PiArICAgICAqLw0KPj4gKyAgICBjZmctPndpbiA9IHBjaV9yZW1hcF9jZmdzcGFjZShjZmct
PnBoeXNfYWRkciwgY2ZnLT5zaXplKTsNCj4+ICsgICAgaWYgKCAhY2ZnLT53aW4gKQ0KPj4gKyAg
ICAgICAgZ290byBlcnJfZXhpdF9yZW1hcDsNCj4+ICsNCj4+ICsgICAgcHJpbnRrKCJFQ0FNIGF0
IFttZW0gJWx4LSVseF0gZm9yIFtidXMgJXgtJXhdIFxuIixjZmctPnBoeXNfYWRkciwNCj4+ICsg
ICAgICAgICAgICBjZmctPnBoeXNfYWRkciArIGNmZy0+c2l6ZSAtIDEsIGNmZy0+YnVzbl9zdGFy
dCwgY2ZnLT5idXNuX2VuZCk7DQo+PiArDQo+PiArICAgIHJldHVybiBjZmc7DQo+PiArDQo+PiAr
ZXJyX2V4aXRfcmVtYXA6DQo+PiArICAgIHByaW50ayhYRU5MT0dfRVJSICJFQ0FNIGlvcmVtYXAg
ZmFpbGVkXG4iKTsNCj4+ICtlcnJfZXhpdDoNCj4+ICsgICAgcGNpX2VjYW1fZnJlZShjZmcpOw0K
Pj4gKyAgICByZXR1cm4gTlVMTDsNCj4+ICt9DQo+PiArDQo+PiArc3RydWN0IHBjaV9ob3N0X2Jy
aWRnZSAqcGNpX2FsbG9jX2hvc3RfYnJpZGdlKHZvaWQpDQo+PiArew0KPj4gKyAgICBzdHJ1Y3Qg
cGNpX2hvc3RfYnJpZGdlICpicmlkZ2UgPSB4emFsbG9jKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2Up
Ow0KPj4gKw0KPj4gKyAgICBpZiAoICFicmlkZ2UgKQ0KPj4gKyAgICAgICAgcmV0dXJuIE5VTEw7
DQo+PiArDQo+PiArICAgIElOSVRfTElTVF9IRUFEKCZicmlkZ2UtPm5vZGUpOw0KPj4gKyAgICBi
cmlkZ2UtPmJ1c19zdGFydCA9IH4wOw0KPj4gKyAgICBicmlkZ2UtPmJ1c19lbmQgPSB+MDsNCj4g
DQo+IFBsZWFzZSB1c2UgSU5WQUxJRF9QQUREUiBpbnN0ZWFkIG9mIH4wDQoNCmJyaWRnZS0+YnVz
X3N0YXJ0IGlzIGlmIHR5cGUgdWludDhfdCBpZiBJICB1c2UgSU5WQUxJRF9QQUREUiBJIHdpbGwg
Z2V0IG92ZXJmbG93IGVycm9yLg0KSSB3aWxsIHVzZSBVOF9NQVggaW5zdGVhZCBvZiBJTlZBTElE
X1BBRERSLg0KDQo+IA0KPiANCj4+ICsgICAgcmV0dXJuIGJyaWRnZTsNCj4+ICt9DQo+PiArDQo+
PiArdm9pZCBwY2lfYWRkX2hvc3RfYnJpZGdlKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRn
ZSkNCj4+ICt7DQo+PiArICAgIGxpc3RfYWRkX3RhaWwoJmJyaWRnZS0+bm9kZSwgJnBjaV9ob3N0
X2JyaWRnZXMpOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW50IHBjaV9nZXRfbmV3X2RvbWFp
bl9ucih2b2lkKQ0KPj4gK3sNCj4+ICsgICAgcmV0dXJuIGF0b21pY19pbmNfcmV0dXJuKCZkb21h
aW5fbnIpOw0KPj4gK30NCj4+ICsNCj4+ICsvKg0KPj4gKyAqIFRoaXMgZnVuY3Rpb24gd2lsbCB0
cnkgdG8gb2J0YWluIHRoZSBob3N0IGJyaWRnZSBkb21haW4gbnVtYmVyIGJ5DQo+PiArICogZmlu
ZGluZyBhIHByb3BlcnR5IGNhbGxlZCAibGludXgscGNpLWRvbWFpbiIgb2YgdGhlIGdpdmVuIGRl
dmljZSBub2RlLg0KPj4gKyAqDQo+PiArICogQG5vZGU6IGRldmljZSB0cmVlIG5vZGUgd2l0aCB0
aGUgZG9tYWluIGluZm9ybWF0aW9uDQo+PiArICoNCj4+ICsgKiBSZXR1cm5zIHRoZSBhc3NvY2lh
dGVkIGRvbWFpbiBudW1iZXIgZnJvbSBEVCBpbiB0aGUgcmFuZ2UgWzAtMHhmZmZmXSwgb3INCj4+
ICsgKiBhIG5lZ2F0aXZlIHZhbHVlIGlmIHRoZSByZXF1aXJlZCBwcm9wZXJ0eSBpcyBub3QgZm91
bmQuDQo+PiArICovDQo+PiArc3RhdGljIGludCBkdF9nZXRfcGNpX2RvbWFpbl9ucihzdHJ1Y3Qg
ZHRfZGV2aWNlX25vZGUgKm5vZGUpDQo+PiArew0KPj4gKyAgICB1MzIgZG9tYWluOw0KPj4gKyAg
ICBpbnQgZXJyb3I7DQo+PiArDQo+PiArICAgIGVycm9yID0gZHRfcHJvcGVydHlfcmVhZF91MzIo
bm9kZSwgImxpbnV4LHBjaS1kb21haW4iLCAmZG9tYWluKTsNCj4+ICsgICAgaWYgKCAhZXJyb3Ig
KQ0KPj4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+PiArDQo+PiArICAgIHJldHVybiAodTE2
KWRvbWFpbjsNCj4gDQo+IExldCdzIGNoZWNrIHRoYXQgZG9tYWluIDw9IFVJTlQxNl9NQVgNCj4g
DQpBY2sNCiANClJlZ2FyZHMsDQpSYWh1bA==


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 12:01:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 12:01:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184330.332935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOfDs-0000jw-Q4; Fri, 10 Sep 2021 12:01:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184330.332935; Fri, 10 Sep 2021 12:01:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOfDs-0000jp-Mz; Fri, 10 Sep 2021 12:01:36 +0000
Received: by outflank-mailman (input) for mailman id 184330;
 Fri, 10 Sep 2021 12:01:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wql=OA=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mOfDr-0000jQ-A2
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 12:01:35 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.87]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a57629ad-f595-4edc-83b0-232d0aa08d54;
 Fri, 10 Sep 2021 12:01:30 +0000 (UTC)
Received: from AS8PR05CA0024.eurprd05.prod.outlook.com (2603:10a6:20b:311::29)
 by DB6PR0802MB2583.eurprd08.prod.outlook.com (2603:10a6:4:99::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Fri, 10 Sep
 2021 12:01:28 +0000
Received: from VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:311:cafe::1d) by AS8PR05CA0024.outlook.office365.com
 (2603:10a6:20b:311::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 12:01:28 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT036.mail.protection.outlook.com (10.152.19.204) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 12:01:28 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Fri, 10 Sep 2021 12:01:27 +0000
Received: from 919b81e88633.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6BCC445D-E2B8-4F57-A990-944E7845D385.1; 
 Fri, 10 Sep 2021 12:01:19 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 919b81e88633.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 10 Sep 2021 12:01:19 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4293.eurprd08.prod.outlook.com (2603:10a6:20b:be::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 12:01:18 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 12:01:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a57629ad-f595-4edc-83b0-232d0aa08d54
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=u0gq4qr2jI+1FpXsAkUAnAZEh9Ot1CYUtuKyLj+j4u8=;
 b=MCL8WBjil32UQ3VJ+j1Uwe0R2MF7okoeWBYQapgyz99Q7hkTJjGRBUamNW5+IRzsq72ZSD0ePSdkRsYvRhEo7uPOw2oaIjD6GBAOkC0ntTQsa+1j5IZwkC9h1LKkO2CAYl+gKCHfUooj4SFT10q4a988KPcfYYARBeUNxq+UEek=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ec71279265c49a80
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MgxgOiQ+YDPy6Cf+sGmOkF7brdtvKnHa1OPW1h0PJzSWROWLm9E1GMYA7hijyxI4n+AjDQPRG5wPm4frB0/HUh9L06LOy7Z+M+DgkvAUTSPutWB7vpaYGcDktfdKz0KYX0hJ/0VrVciK8TuYU1fGsZVlHWXpz9H64KM8mkuye1ga7AbjjbeWII3V/ZNLRaJRz5GA6gSNgTNpqe5mNIh/oOKjbv4uQb+gtBvdm4lobdiQQUHwYHnh3YsF3cKMTDb8pmdmY6YaV5qwzSovitbzzGGR09YSmS0iED0/ylj9H7kokGMpg1ey8yHPzGJrKSx+mp53oFXirQjhhvhqQMJiEQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=u0gq4qr2jI+1FpXsAkUAnAZEh9Ot1CYUtuKyLj+j4u8=;
 b=bJF5M3KFAIGRVJqN4kZ4WHFmddwNKE3+5DkywR1my7xpAiKFgMmfOdexAnR/Sk1RwcPxMCdYGnRG8Ukojzv0wQ7DIPKfWuGQ73xrASDp+Z6OG6y7qII5B89mXZp4Iad+TdRPhI1wBGaYJjBjLzYXVQ3BEpJ8EgWtU4OLYZrtmsCoZZ0Ury/4cVTg29uiSAlj8xzibWB/m8qPVZtOPp16wcqR8Jxh9hMwI5b/upJVwQIu8nDQu+JQ84gDmaa2dbtb0m3PQMkMPt2SLu2EoDnPiEt5Bv1AhyFDzE8T0hww7IOwHuBspnvuD8R6BvK7TmmtPX3FYUYvQwBTsRuFDuv4Yw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=u0gq4qr2jI+1FpXsAkUAnAZEh9Ot1CYUtuKyLj+j4u8=;
 b=MCL8WBjil32UQ3VJ+j1Uwe0R2MF7okoeWBYQapgyz99Q7hkTJjGRBUamNW5+IRzsq72ZSD0ePSdkRsYvRhEo7uPOw2oaIjD6GBAOkC0ntTQsa+1j5IZwkC9h1LKkO2CAYl+gKCHfUooj4SFT10q4a988KPcfYYARBeUNxq+UEek=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Oleksandr Andrushchenko
	<oleksandr_andrushchenko@epam.com>, Julien Grall <julien@xen.org>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXlPLBYGJaUEBt502oB439VJRFraucfJAAgADQqYA=
Date: Fri, 10 Sep 2021 12:01:18 +0000
Message-ID: <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 38b0a54c-37a2-4a84-19a6-08d97452b8b3
x-ms-traffictypediagnostic: AM6PR08MB4293:|DB6PR0802MB2583:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB2583534F019010A970126565FCD69@DB6PR0802MB2583.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 KFIR6CsBZXL+psT0Bfn3M+Efa6dCbuOznZi1PcQsFXbDjGF/7ifiL4Ayr0meZhc7qI19aTX+qeRodMcrG3zs+DnfRhtdoVu9JVZB1A6fMT56lhnj+qm3LwLQbJAtSEC2/SDAq0GWoUdmtD845UXJTy/T2Dsvn50GeTsmbuLh1E4/mw60q05PPh2EQx35YZ8U9nQJjR1IKU6lU6wIEiILmwo6waIh58QmXBzmX2M+TKlVN44N+1hHWhwfXK9LtcrohOtWHMtFQ7fFknfYLCQmV3b+flLwS4YFbAjosuKQb1q3A7P+5cf5oVF45sGlkg60l/b0Adh8s1tw6JgIJHfvL85YHUOjWv7R8tNhqw8KVi2TttUHVOixzzwcUnSLxItZ6c2sKO75vBqF7Aq3ZexMD8wGBHYAhZNbgVwkqLgobk40fR6pIya5QneUK0KW+DvWNWCcD6ruugy/3y71KRT791+HzCnjf9uQ/JjfD8c4759mKM25CarN5zePLLdO1K5APrZBRKmlMGRe8I+p1B2SbFJLHNgM6YcI/JZf1xy+OttR/fd/DEcXs1/oTpLfBLrDiw8N7CRjBlDLOR1D+4b3zSAG/DccJr61eVVRN5x/5KYFeWzYMYWAGS4UHLsCVjQaF8U+dujhS2odhXfrOTdlVxRakrdajy0SPg9Dq03NHu+mcfjfXQbr/R/sAcO94pZNiKgnAD1ZG+uoIjbSIokkVYB/hcq+XsrBvhYsu1KGkYX/wIlH/PAQO6kqJO1Rb9r3cEwghmkbn0WVuu6vxmLzDTAQNFjcYR9/WItzas0ZPsIOvHmaGNP8ATdn6N+itrLnWTBTWkBZelIUoq5w9R9fEg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(366004)(39850400004)(396003)(136003)(122000001)(38100700002)(2616005)(83380400001)(4326008)(33656002)(71200400001)(478600001)(86362001)(36756003)(6512007)(66946007)(38070700005)(2906002)(91956017)(6916009)(8676002)(76116006)(64756008)(5660300002)(66556008)(66476007)(66446008)(6486002)(186003)(8936002)(316002)(26005)(53546011)(6506007)(54906003)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <ED938E709D3A0248B235C02DE48A7EBD@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4293
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a6168449-58b8-4766-3164-08d97452b2a2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Rr7qdu9/44md0NpPq7qpjqj5SRlyStvSNzGYnvg9dyF9NJznp6fIjqNHgxMYB+ZNLCjxrL406MTjpn0kIcJa6UN4hwbIejn2wtzTc+gJu32HTicSxxAFdRu5WbzDYQLZdV0pXuH3EfQcZzOpkgl8zSS++V82bqYtkPoXwwsoY0mty/DpW9NxSJGMD/VRK2tgvaU8Ver7ju6bsvIr7ofC6a7GX0n19GtvK7LNXN46AfWiZu8H1vl2MlBQ7qQM6lLfqM8plhZ/BsWXF9w3g0N+P4o4NxltqkPzeD9SN3yD5mFiH/e6JnhuMGDEk0ywZ6uvqj4wIyC0RDgQp7/gQd+em4Vul+xZNgkpwbQyLuxd2s/Q2y86a9ex/51QjyV+CZlyiDBWvsP0A0d9l/Z04hhx1xUXMyxXBL+a4VWmhTDzuEJy4bAP/Zz5a+12/ALp0MMfqJrL696whNHzw0ukDNABhn4OQvb6Ke3CzDIUd51MQ6BJe+8lQAp2F3UQ3lbxFEHERVMP8wyaYZ+FWjmcIqAgUAcjG8WGBv2QHupFWyeRbdfZrJYI58ziHjYLrk2ibk1q8suAp5vaLOrQAaKFJwBsevuwFHEvGxVNQgLR0Ed22jUw2zjJhNiHQ2mZ4tKhQqGFNYwkTWIcGPu3udkjfkfGwDibWnev+3IT8Bl9xk8hV6IktMWEX7d2XhObEKSQ3BfT0uq8U5C8QnWb2T/yFQsYkbgbwi0HV30gTa43rV0877r/Yxb6MzS6BlfQqsan1YDtT7U35oSpfjEsw62L1VnyT0Bi0NLans2IeDi+ryOPeAuoigAm2mMk7I+GZZHIN1AB
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6486002)(6512007)(2616005)(107886003)(8676002)(54906003)(316002)(33656002)(81166007)(5660300002)(26005)(70586007)(53546011)(6506007)(83380400001)(70206006)(508600001)(36756003)(2906002)(336012)(186003)(4326008)(82310400003)(36860700001)(47076005)(86362001)(6862004)(356005)(8936002)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 12:01:28.3431
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 38b0a54c-37a2-4a84-19a6-08d97452b8b3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2583

Hi Stefano,

> On 10 Sep 2021, at 12:34 am, Stefano Stabellini <sstabellini@kernel.org> =
wrote:
>=20
> On Thu, 19 Aug 2021, Rahul Singh wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>=20
>> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
>> space to the XEN memory.
>>=20
>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com=
>
>> ---
>> xen/arch/arm/pci/Makefile          |  1 +
>> xen/arch/arm/pci/pci-host-zynqmp.c | 59 ++++++++++++++++++++++++++++++
>> 2 files changed, 60 insertions(+)
>> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>>=20
>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
>> index 6f32fbbe67..1d045ade01 100644
>> --- a/xen/arch/arm/pci/Makefile
>> +++ b/xen/arch/arm/pci/Makefile
>> @@ -3,3 +3,4 @@ obj-y +=3D pci-access.o
>> obj-y +=3D pci-host-generic.o
>> obj-y +=3D pci-host-common.o
>> obj-y +=3D ecam.o
>> +obj-y +=3D pci-host-zynqmp.o
>> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-h=
ost-zynqmp.c
>> new file mode 100644
>> index 0000000000..fe103e3855
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
>> @@ -0,0 +1,59 @@
>> +/*
>> + * Copyright (C) 2020-2021 EPAM Systems
>> + *
>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>> + * Based on xen/arch/arm/pci/pci-host-generic.c
>> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
>=20
> Only one Copyright line per file is enough :-)
>=20
> But actually all the Copyright lines with a name or a company name are
> not really required or useful, as the copyright is noted in full details
> in the commit messages (author and signed-off-by lines). I would remove
> them all from the new files added by this series.

Ok. Let me remove in next version.
>=20
>=20
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>=
.
>> + */
>> +
>> +#include <asm/device.h>
>> +#include <xen/pci.h>
>> +#include <asm/pci.h>
>> +
>> +static const struct dt_device_match gen_pci_dt_match[] =3D {
>> +    { .compatible =3D "xlnx,nwl-pcie-2.11",
>> +      .data =3D       &pci_generic_ecam_ops },
>> +    { },
>> +};
>> +
>> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data=
)
>> +{
>> +    const struct dt_device_match *of_id;
>> +    const struct pci_ecam_ops *ops;
>> +
>> +    of_id =3D dt_match_node(gen_pci_dt_match, dev->dev.of_node);
>=20
> This should be superfluous

Ack. I will remove the dt_match_node(..) in next version.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 12:38:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 12:38:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184346.332954 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOfmz-00057g-Nh; Fri, 10 Sep 2021 12:37:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184346.332954; Fri, 10 Sep 2021 12:37:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOfmz-00057Z-Kd; Fri, 10 Sep 2021 12:37:53 +0000
Received: by outflank-mailman (input) for mailman id 184346;
 Fri, 10 Sep 2021 12:37:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOfmy-00057T-7a
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 12:37:52 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01c1eb1f-6ccc-4d40-aa80-7a70c438bc03;
 Fri, 10 Sep 2021 12:37:50 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ABTHwB025704; 
 Fri, 10 Sep 2021 12:37:47 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b04tx0swx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 12:37:47 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3374.eurprd03.prod.outlook.com (2603:10a6:803:18::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 12:37:44 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 12:37:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01c1eb1f-6ccc-4d40-aa80-7a70c438bc03
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VG4qmk8IRMawKYHO4H2DolR11+n7ZYfQU2QKWZidnArZKv8irpE6zE/tpdJMdKZYTwhZbBrBdOC70LgTKCnKa+RoA/KvcOiZXhjCS3rmLkaUmrdDOnlfizPOBIShL1//0kcaY/QKheKgPjUd3tsb4Xi4SWjLMYLyTjLFPbcgcxYx7o4LZRM1fH+Dc8AhecKD66U2YZFVR2alfYvXE2t+UcqVcuyqku8fqd3sHLD2T6WiU1Y3pknWfEqojckN++8QgOPcFsRHjj1gxwrO6V2jg/UfBMWIq63Xedf/prBjs4EnM7n4FGey8WA9/RWtKMkE+GPqJI4dupoStdnqecdpng==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fOY/k+Cl916ZN8DknowvG6mZDifaIyk5CAKNXWJuwOg=;
 b=ZMckj97MYUnaPkecitEZ9uvTWEzKY5fgFh2eKvq/CTZA+jjjwrVjAfdxE5qXjqwYG/M8FPkUsRWG8ISbvewV82YGwA8BEHqmw2C/Se0IVmgDMh49BmMxQneSIEM3WMGktt7bRkAPABiE5UyR+U6NqlEupBc+i2NkV2GN9dCm1HqipW44irN4bO57t0v6bdVQiKMUklNGLOfLXXZlBekd5zmQWsBsndOr+ELHBoIyHlilztfHRdn18L4DWHjkRNQNvcPFJYw9rZWAxlCs4c0JGvsH8z5enWd4tlplMn/5hg7c0XtnZyeBOrU3VEb3f1I6x6H7BQh7b/BwjV+9s9n0qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fOY/k+Cl916ZN8DknowvG6mZDifaIyk5CAKNXWJuwOg=;
 b=KRtvBXJBwVNDZX/aTTTpwt2ppyRjdwORRhNcqFZea1xjTBq82C3mprrs0XSFgP+NEruwi+pGplgipzVx+Xl2tXdGK4rpgSfUkUPWUKA8yOW37g2oYDZJNuUebgoESydHKzKze9ob63JTMtnCZDYs/M/cFUuJBZYc65qkfWnl6TkDI7AqBUbaCk2YS0CXhmuC5WiRdvT/sOKraIQj9aal6PoC5uslt0qhzAGthtHJcFaR0ewfX0XRG2FsxImAWCvde4KbhzjLiuWlTxTd+in2ugJo8NkEhTYEbA2tmFYTH7YYVFdPgf3VQ8MojB6K08Vs+aC+WaVMNgnf+8HcnSvUvA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYA=
Date: Fri, 10 Sep 2021 12:37:44 +0000
Message-ID: <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
In-Reply-To: <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1b0858b1-c2b7-4a63-af01-08d97457c9d9
x-ms-traffictypediagnostic: VI1PR0302MB3374:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB337404E0C4B3A2E153B89C6CE7D69@VI1PR0302MB3374.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 +XM95JvvPpWJczxA+8BU0i8Hx5DPIo2/twRaw4eQyZiH8zS5wRZbPmE4cm5fqmtuGXDVq3fLrijVH8W97boIDKW0+AOxMJWDDM9hQCE+x+58G2GY5uY9emd/ut7M5mb9xe8uxBJVXwy1bEfJJjbOnBM+a+O1gevy5FOLbJizL+3rYjzKyj91CgfiqvdZzt+e5g7sl4LbJAVGM50XgBdJfbQIsWRdbWW1POSkH8NhRdO5NKLvbmG+lriRhAboK9HBLNb5NbR5fXKa5UT46F4+3/epjR4G/oWg6H5D/XxsEsMqtY5yPVT5dO16JZRS++4Xm4I6MgjJOOVsdGJf7XL0wiqGqnQLyVEYBufmboU/Q0pDhXUQv8tIYxgdIDt+RK1xh8a9pBQbG09BePJ98vR79awQ76SwX3uCtTgK1ROswCWOXzrcES87UrkXRtH+J+vqX2IrfOxVj9sC5veJ0OWeJ0l/XpXZ3zwtz4sbHJIyv0RLkOBULHIkQoOlhqOAUbaoKrXWfec6i3E6p0jfsynamUU6j/iIGR2dCPscBV963AF2nq14mgoE7i5XllPENv1wabwB7XQLzcOl7Eii7peGQewzZ2ASC6iMrK6vqAoXlpW959N6sOqmI6I9rKT3BLD9kZvJeHxmm7YcgwIcLrdO3tDLoaQX2b23up4KpN0ODZ/IJOsCVgz4e048hcqRVxR6pKufwjeXwmbAO8CApOTiQGC9BG5ZqEQKS6dOJvPSZc36rZGYf80wsAw5bcFz64n0
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(76116006)(83380400001)(86362001)(5660300002)(66556008)(91956017)(54906003)(36756003)(66476007)(38070700005)(64756008)(110136005)(6486002)(66946007)(316002)(66446008)(508600001)(71200400001)(2616005)(55236004)(53546011)(6506007)(38100700002)(31696002)(122000001)(2906002)(6512007)(31686004)(8936002)(186003)(26005)(8676002)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?TWpKelVZRmRqS0FlNTE3MEl0SHpIVGVQdXF4L0FCYk1WUHgvRzF0Y0lYOTkw?=
 =?utf-8?B?cE9mMFZaLytVa1BmZHNEdjhqTkFWZzBDbTRObVZxTUo0b3ZPR1lZdlJWZUtL?=
 =?utf-8?B?REJBaWJvcllTemZBUlB6bnR0SDRMdUZoelZNTXQyYnhCTnRTMXZ6ZSticUhh?=
 =?utf-8?B?L0pZRTAxQzdCY25UamM4Yk9CZlVpWVhxaERsVzhPZndXdEVKbXV5dklSdENO?=
 =?utf-8?B?enRxM1IvaXBzNG45QmVoR3c1aklWcGdWU0NOTWtaN0ZXSk5pa1JOYnhIWm81?=
 =?utf-8?B?NnlrSVVYbENQN2x0Mk85d3FwZ20wVTBFN1A1WHdOQWkzdjBtYm9VbWkwaEN0?=
 =?utf-8?B?ZGlHaUkwR25KbzdYNzBIM2xmRUlITm9HaENIaGt3Qm5sWGkvRTdxcWh2OFI1?=
 =?utf-8?B?YTh3VmhrRXF1bHNtSXdmSVdUZHJ4dUZ4K3hvcDhYejNyNU5LL1NlK2Y4TGtC?=
 =?utf-8?B?QnhnT2VvalZRdzk3Q3JWSGtIekxSVzhhWENwdHlzY3owK2pSckdTVEk4M3VF?=
 =?utf-8?B?Qy9oOGlIUkxuVE5waWcvQVpRbUpIV0RUOUhvVDdRUWEwektCS0MzamhILzFw?=
 =?utf-8?B?bzZFSWtDaXRiZGtneEs2ODdYU1BOMHgrTG9OZnF4UTZYbGgyVllFS0JwOVY5?=
 =?utf-8?B?c0k2UjZ3TittcFB0NEwvREtaR0RIaEllRm1vdGtaaGl5Mk1FanVnZWVOR21F?=
 =?utf-8?B?c1NFd0pCWjQyNStjaGV0OCtRYUZoWmcyQk9qeUQ1OVY0U2UwQ25pTUlqY3JU?=
 =?utf-8?B?Tk5uRHB6T3ZmQVRpdGVRZFlJTFFWWmYyUStSS0w3V2t1Y1ZRQWZPVEZwMjNV?=
 =?utf-8?B?OHZCMUVrVmtkZmtuNldzYUc1MEFNOVJ4bXNydnVLbkJVbXpCOGJqbmFjNnI3?=
 =?utf-8?B?RXkrWXJOMXVwTUphTWx0bXhwa28vRGh4OERqUHNKUlU0YzJrUTh3K254dG1U?=
 =?utf-8?B?YXQyQlFXeUlpOXF6a1BsQ2NxNUgyY1JUZ3hNQnZnbkV6cjRPdTlHZ3JTc1Bs?=
 =?utf-8?B?eW1Tb0c2TXBXS2hnSE9tZ25PbitQcUZBeHMyV3d0M005MU9XMWlWejZDVVV5?=
 =?utf-8?B?V01yNDlSR3JPMk42SEtSM0N2Q1ZvSVY3dGw2bThsa0xGOUtnYXhiQ3B6Tkw0?=
 =?utf-8?B?RHNGb0tmVmZRYnUwVXhYNDNqQTBXd21pWXRML0UxWjRtOWdGRGMyWUZMMzVk?=
 =?utf-8?B?azBJdU10Q1RrWGhxNURleVhBK0lhaklLeC9pZEpoK3h3bzVkMDIvMFN0c0ZE?=
 =?utf-8?B?c3VyM1pMQ21mQ0MzUkFydkNUWnRESmlwcmxpWHJielo2S3BxSEdsRUVkRk5U?=
 =?utf-8?B?eGlCQVZMb2YwR3JPYWlEdkNqc1l0RkIxV0RrTi9aUVA4ZTJGUnVCNTBaWDFM?=
 =?utf-8?B?WkhBTkhGeXpPRDNFdnUyMkZLWm5Kd29ORWF1eHpMdnN1QXM3N055TzZSTEJ1?=
 =?utf-8?B?c3pTaG5raGlUN3NCN3VPQ2tCQ0wyY0NDWlg0WEJQcnJGMVRtU2d4SG1lU1hN?=
 =?utf-8?B?RktyN29HeElERFJSbnhyNDNKYVlUTUxhZDRPRXE0OUVLVWNPc2ZicGpzVFkw?=
 =?utf-8?B?VkVET1Z5bVlOSTdjeVRpakxlaGNBMGh4L0QwbFlrU1hnSGlqRGlZeU1FTjJr?=
 =?utf-8?B?cUc2WStZR290cE5WR1dlYVg3MzFsVnpQSHhnZGdMTkYvcCtBZmlTdERuWEZq?=
 =?utf-8?B?eGk1VGJrcCtKSmZRa2dqd2NRcDkxdm5oKzhyNFI5eEx2QW1IMFdGUzhHRUs1?=
 =?utf-8?B?eGFSZGY0c1N0aVVVRFBCbSsxV0pBbTZ3ajZmUFAyWUpGZDVWR0doN2dBKy9r?=
 =?utf-8?B?S0t4K1lvclRmRmszTE1SQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <023085D875ED244C86E92F8E8F6A6D79@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1b0858b1-c2b7-4a63-af01-08d97457c9d9
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 12:37:44.6222
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: xCqO2nK5Nw47iJOfXuiH+/bG7+nj72V4k4QQbVb91elF+A1gE1VBTDJjcS2mtmcb26cFwChcSb9Uw8i+uhMdNW6SVTU4MxdDzGulygemY6jSq8GqGk1pkOTXMFJgWRlJ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3374
X-Proofpoint-ORIG-GUID: pNAUkeGQULJSEn8vPvgbJfxqpJghB-Od
X-Proofpoint-GUID: pNAUkeGQULJSEn8vPvgbJfxqpJghB-Od
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0
 priorityscore=1501 phishscore=0 bulkscore=0 lowpriorityscore=0
 adultscore=0 malwarescore=0 mlxlogscore=999 impostorscore=0 clxscore=1015
 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100074

SGksIEp1bGllbiENCg0KT24gMDkuMDkuMjEgMjA6NTgsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4g
SGkgT2xla3NhbmRyLA0KPg0KPiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiBIb3N0IGJyaWRnZSBjb250cm9sbGVy
J3MgRUNBTSBzcGFjZSBpcyBtYXBwZWQgaW50byBEb21haW4tMCdzIHAybSwNCj4+IHRodXMgaXQg
aXMgbm90IHBvc3NpYmxlIHRvIHRyYXAgdGhlIHNhbWUgZm9yIHZQQ0kgdmlhIE1NSU8gaGFuZGxl
cnMuDQo+PiBGb3IgdGhpcyB0byB3b3JrIHdlIG5lZWQgdG8gbm90IG1hcCB0aG9zZSB3aGlsZSBj
b25zdHJ1Y3RpbmcgdGhlIGRvbWFpbi4NCj4+DQo+PiBOb3RlLCB0aGF0IGR1cmluZyBEb21haW4t
MCBjcmVhdGlvbiB0aGVyZSBpcyBubyBwY2lfZGV2IHlldCBhbGxvY2F0ZWQgZm9yDQo+PiBob3N0
IGJyaWRnZXMsIHRodXMgd2UgY2Fubm90IG1hdGNoIFBDSSBob3N0IGFuZCBpdHMgYXNzb2NpYXRl
ZA0KPj4gYnJpZGdlIGJ5IFNCREYuIFVzZSBkdF9kZXZpY2Vfbm9kZSBmaWVsZCBmb3IgY2hlY2tz
IGluc3RlYWQuDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28g
PG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4gLS0tDQo+PiDCoCB4ZW4vYXJj
aC9hcm0vZG9tYWluX2J1aWxkLmPCoMKgwqDCoMKgwqDCoCB8wqAgMyArKysNCj4+IMKgIHhlbi9h
cmNoL2FybS9wY2kvZWNhbS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDE3ICsrKysrKysrKysr
KysrKysrDQo+PiDCoCB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIHwgMjIgKysr
KysrKysrKysrKysrKysrKysrKw0KPj4gwqAgeGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaMKgwqDC
oMKgwqDCoMKgwqDCoCB8IDEyICsrKysrKysrKysrKw0KPj4gwqAgNCBmaWxlcyBjaGFuZ2VkLCA1
NCBpbnNlcnRpb25zKCspDQo+Pg0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9kb21haW5f
YnVpbGQuYyBiL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KPj4gaW5kZXggZGE0MjdmMzk5
NzExLi43NmY1YjUxMzI4MGMgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vZG9tYWluX2J1
aWxkLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KPj4gQEAgLTEyNTcs
NiArMTI1Nyw5IEBAIHN0YXRpYyBpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qg
c3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgfQ0KPj4g
wqDCoMKgwqDCoCB9DQo+PiDCoCArwqDCoMKgIGlmICggbmVlZF9tYXBwaW5nICYmIChkZXZpY2Vf
Z2V0X2NsYXNzKGRldikgPT0gREVWSUNFX1BDSSkgKSA+ICvCoMKgwqDCoMKgwqDCoCBuZWVkX21h
cHBpbmcgPSBwY2lfaG9zdF9icmlkZ2VfbmVlZF9wMm1fbWFwcGluZyhkLCBkZXYsIA0KPiBhZGRy
LCBsZW4pOw0KPg0KPiBBRkFJQ1QsIHdpdGggZGV2aWNlX2dldF9jbGFzcyhkZXYpLCB5b3Uga25v
dyB3aGV0aGVyIHRoZSBob3N0YnJpZGdlIGlzIHVzZWQgYnkgWGVuLiBUaGVyZWZvcmUsIEkgd291
bGQgZXhwZWN0IHRoYXQgd2UgZG9uJ3Qgd2FudCB0byBtYXAgYWxsIHRoZSByZWdpb25zIG9mIHRo
ZSBob3N0YnJpZGdlcyBpbiBkb20wIChpbmNsdWRpbmcgdGhlIEJBUnMpLg0KPg0KPiBDYW4geW91
IGNsYXJpZnkgaXQ/DQpXZSBvbmx5IHdhbnQgdG8gdHJhcCBFQ0FNLCBub3QgTU1JT3MgYW5kIGFu
eSBvdGhlciBtZW1vcnkgcmVnaW9ucyBhcyB0aGUgYnJpZGdlIGlzDQoNCmluaXRpYWxpemVkIGFu
ZCB1c2VkIGJ5IERvbWFpbi0wIGNvbXBsZXRlbHkuIEJ1dCBhdCB0aGUgc2FtZSB0aW1lIHdlIHdh
bnQgdG8gdHJhcCBhbGwNCg0KY29uZmlndXJhdGlvbiBzcGFjZSBhY2Nlc3MsIHNvIHdlIGNhbiBz
ZWUgd2hhdCBpcyBoYXBwZW5pbmcgdG8gaXQuDQoNCj4NCj4+ICsgPsKgwqDCoMKgwqDCoCBpZiAo
IG5lZWRfbWFwcGluZyApDQo+PiDCoMKgwqDCoMKgIHsNCj4+IMKgwqDCoMKgwqDCoMKgwqDCoCBy
ZXMgPSBtYXBfcmVnaW9uc19wMm10KGQsDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3Bj
aS9lY2FtLmMgYi94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0KPj4gaW5kZXggOTJlY2IyZTA3NjJi
Li5kMzJlZmI3ZmNiZDAgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0K
Pj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMNCj4+IEBAIC01Miw2ICs1MiwyMiBAQCBz
dGF0aWMgaW50IHBjaV9lY2FtX3JlZ2lzdGVyX21taW9faGFuZGxlcihzdHJ1Y3QgZG9tYWluICpk
LA0KPj4gwqDCoMKgwqDCoCByZXR1cm4gMDsNCj4+IMKgIH0NCj4+IMKgICtzdGF0aWMgaW50IHBj
aV9lY2FtX25lZWRfcDJtX21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICvCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4gK8KgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCB1aW50NjRfdCBhZGRyLCB1aW50NjRfdCBsZW4pDQo+PiArew0KPj4gK8KgwqDCoCBzdHJ1Y3Qg
cGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0YTsNCj4+ICsNCj4+ICvCoMKg
wqAgaWYgKCAhaXNfaGFyZHdhcmVfZG9tYWluKGQpICkNCj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1
cm4gdHJ1ZTsNCj4NCj4gSSBhbSBhIGJpdCBwdXp6bGVkIHdpdGggdGhpcyBjaGVjay4gSWYgdGhl
IEVDQU0gaGFzIGJlZW4gaW5pdGlhbGl6ZWQgYnkgWGVuLCB0aGVuIEkgYmVsaWV2ZSB3ZSBjYW5u
b3QgZXhwb3NlIGl0IHRvIGFueSBkb21haW4gYXQgYWxsLg0KWW91IGFyZSByaWdodCwgdGhpcyBj
aGVjayBuZWVkcyB0byBiZSByZW1vdmVkIGFzIHRoaXMgZnVuY3Rpb24gaXMgb25seSBjYWxsZWQg
Zm9yIERvbWFpbi0wDQoNCj4NCj4+ICsNCj4+ICvCoMKgwqAgLyoNCj4+ICvCoMKgwqDCoCAqIFdl
IGRvIG5vdCB3YW50IEVDQU0gYWRkcmVzcyBzcGFjZSB0byBiZSBtYXBwZWQgaW4gZG9tYWluJ3Mg
cDJtLA0KPj4gK8KgwqDCoMKgICogc28gd2UgY2FuIHRyYXAgYWNjZXNzIHRvIGl0Lg0KPj4gK8Kg
wqDCoMKgICovDQo+PiArwqDCoMKgIHJldHVybiBjZmctPnBoeXNfYWRkciAhPSBhZGRyOw0KPj4g
K30NCj4+ICsNCj4+IMKgIC8qIEVDQU0gb3BzICovDQo+PiDCoCBjb25zdCBzdHJ1Y3QgcGNpX2Vj
YW1fb3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0KPj4gwqDCoMKgwqDCoCAuYnVzX3NoaWZ0
wqAgPSAyMCwNCj4+IEBAIC02MCw2ICs3Niw3IEBAIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMg
cGNpX2dlbmVyaWNfZWNhbV9vcHMgPSB7DQo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgLnJlYWTCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgPSBwY2lfZ2VuZXJpY19jb25maWdfcmVh
ZCwNCj4+IMKgwqDCoMKgwqDCoMKgwqDCoCAud3JpdGXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgID0gcGNpX2dlbmVyaWNfY29uZmlnX3dyaXRlLA0KPj4gwqDCoMKgwqDCoMKgwqDC
oMKgIC5yZWdpc3Rlcl9tbWlvX2hhbmRsZXLCoCA9IHBjaV9lY2FtX3JlZ2lzdGVyX21taW9faGFu
ZGxlciwNCj4+ICvCoMKgwqDCoMKgwqDCoCAubmVlZF9wMm1fbWFwcGluZ8KgwqDCoMKgwqDCoCA9
IHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmcsDQo+PiDCoMKgwqDCoMKgIH0NCj4+IMKgIH07DQo+
PiDCoCBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYyBiL3hl
bi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMNCj4+IGluZGV4IGE4OTExMmJmYmI3Yy4u
YzA0YmU2MzY0NTJkIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1j
b21tb24uYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KPj4g
QEAgLTMzNCw2ICszMzQsMjggQEAgaW50IHBjaV9ob3N0X2l0ZXJhdGVfYnJpZGdlcyhzdHJ1Y3Qg
ZG9tYWluICpkLA0KPj4gwqDCoMKgwqDCoCB9DQo+PiDCoMKgwqDCoMKgIHJldHVybiAwOw0KPj4g
wqAgfQ0KPj4gKw0KPj4gK2Jvb2wgcGNpX2hvc3RfYnJpZGdlX25lZWRfcDJtX21hcHBpbmcoc3Ry
dWN0IGRvbWFpbiAqZCwNCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2
aWNlX25vZGUgKm5vZGUsDQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWlu
dDY0X3QgbGVuKQ0KPj4gK3sNCj4+ICvCoMKgwqAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJp
ZGdlOw0KPj4gKw0KPj4gK8KgwqDCoCBsaXN0X2Zvcl9lYWNoX2VudHJ5KCBicmlkZ2UsICZwY2lf
aG9zdF9icmlkZ2VzLCBub2RlICkNCj4+ICvCoMKgwqAgew0KPj4gK8KgwqDCoMKgwqDCoMKgIGlm
ICggYnJpZGdlLT5kdF9ub2RlICE9IG5vZGUgKQ0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
Y29udGludWU7DQo+PiArDQo+PiArwqDCoMKgwqDCoMKgwqAgaWYgKCAhYnJpZGdlLT5vcHMtPm5l
ZWRfcDJtX21hcHBpbmcgKQ0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIHRydWU7
DQo+PiArDQo+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIGJyaWRnZS0+b3BzLT5uZWVkX3AybV9t
YXBwaW5nKGQsIGJyaWRnZSwgYWRkciwgbGVuKTsNCj4+ICvCoMKgwqAgfQ0KPj4gK8KgwqDCoCBw
cmludGsoWEVOTE9HX0VSUiAiVW5hYmxlIHRvIGZpbmQgUENJIGJyaWRnZSBmb3IgJXMgc2VnbWVu
dCAlZCwgYWRkciAlbHhcbiIsDQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqAgbm9kZS0+ZnVsbF9u
YW1lLCBicmlkZ2UtPnNlZ21lbnQsIGFkZHIpOw0KPj4gK8KgwqDCoCByZXR1cm4gdHJ1ZTsNCj4+
ICt9DQo+DQo+IElmIHlvdSByZWFsbHkgbmVlZCB0byBtYXAgdGhlIGhvc3RicmlkZ2VzLCB0aGVu
IEkgd291bGQgc3VnZ2VzdCB0byBkZWZlciB0aGUgUDJNIG1hcHBpbmdzIGZvciBhbGwgb2YgdGhl
bSBhbmQgdGhlbiB3YWxrIGFsbCB0aGUgYnJpZGdlIGluIG9uZSBnbyB0byBkbyB0aGUgbWFwcGlu
Z3MuDQo+DQo+IFRoaXMgd291bGQgYXZvaWQgZ29pbmcgdGhyb3VnaCB0aGUgYnJpZGdlcyBldmVy
eSB0aW1lIGR1cmluZyBzZXR1cC4NCg0KV2VsbCwgdGhpcyBjYW4gYmUgZG9uZSwgYnV0OiBteSBp
bXBsZW1lbnRhdGlvbiBwcmV2ZW50cyBtYXBwaW5ncyBhbmQgd2hhdA0KDQp5b3Ugc3VnZ2VzdCB3
aWxsIHJlcXVpcmUgdW5tYXBwaW5nLiBTbywgdGhlIGNvc3QgaW4gbXkgc29sdXRpb24gaXMgd2Ug
bmVlZA0KDQp0byBnbyBvdmVyIGFsbCB0aGUgYnJpZGdlcyAodW50aWwgd2UgZmluZCB0aGUgb25l
IHdlIG5lZWQpIGFuZCBpbiB3aGF0IHlvdQ0KDQpzdWdnZXN0IHdlJ2xsIG5lZWQgdG8gdW5tYXAg
d2hhdCB3ZSBoYXZlIGp1c3QgbWFwcGVkLg0KDQpJIHRoaW5rIHByZXZlbnRpbmcgdW5uZWVkZWQg
bWFwcGluZ3MgaXMgY2hlYXBlciB0aGFuIHVubWFwcGluZyBhZnRlcndhcmRzLg0KDQo+DQo+PiAr
DQo+PiDCoCAvKg0KPj4gwqDCoCAqIExvY2FsIHZhcmlhYmxlczoNCj4+IMKgwqAgKiBtb2RlOiBD
DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCBiL3hlbi9pbmNsdWRl
L2FzbS1hcm0vcGNpLmgNCj4+IGluZGV4IDJjN2M3NjQ5ZTAwZi4uOWMyOGE0YmRjNGI3IDEwMDY0
NA0KPj4gLS0tIGEveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KPj4gKysrIGIveGVuL2luY2x1
ZGUvYXNtLWFybS9wY2kuaA0KPj4gQEAgLTgyLDYgKzgyLDggQEAgc3RydWN0IHBjaV9vcHMgew0K
Pj4gwqDCoMKgwqDCoCBpbnQgKCpyZWdpc3Rlcl9tbWlvX2hhbmRsZXIpKHN0cnVjdCBkb21haW4g
KmQsDQo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+PiDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgbW1pb19oYW5kbGVyX29wcyAqb3BzKTsNCj4+ICvCoMKg
wqAgaW50ICgqbmVlZF9wMm1fbWFwcGluZykoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHBjaV9o
b3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1aW50NjRfdCBhZGRyLCB1aW50NjRfdCBsZW4pOw0KPj4g
wqAgfTsNCj4+IMKgIMKgIC8qDQo+PiBAQCAtMTE1LDkgKzExNywxOSBAQCBzdHJ1Y3QgZHRfZGV2
aWNlX25vZGUgKnBjaV9maW5kX2hvc3RfYnJpZGdlX25vZGUoc3RydWN0IGRldmljZSAqZGV2KTsN
Cj4+IMKgIGludCBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoc3RydWN0IGRvbWFpbiAqZCwNCj4+
IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCBpbnQgKCpjbGIpKHN0cnVjdCBkb21haW4gKmQsDQo+PiDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSkpOw0KPj4gK2Jvb2wgcGNpX2hv
c3RfYnJpZGdlX25lZWRfcDJtX21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICvCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKm5vZGUsDQo+PiArwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWludDY0X3QgbGVuKTsNCj4+IMKgICNlbHNlwqDC
oCAvKiFDT05GSUdfSEFTX1BDSSovDQo+PiDCoCDCoCBzdHJ1Y3QgYXJjaF9wY2lfZGV2IHsgfTsN
Cj4+IMKgICtzdGF0aWMgaW5saW5lIGJvb2wNCj4+ICtwY2lfaG9zdF9icmlkZ2VfbmVlZF9wMm1f
bWFwcGluZyhzdHJ1Y3QgZG9tYWluICpkLA0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgY29uc3Qgc3RydWN0IGR0X2Rl
dmljZV9ub2RlICpub2RlLA0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWludDY0X3QgbGVu
KQ0KPj4gK3sNCj4+ICvCoMKgwqAgcmV0dXJuIHRydWU7DQo+PiArfQ0KPj4gwqAgI2VuZGlmwqAg
LyohQ09ORklHX0hBU19QQ0kqLw0KPj4gwqAgI2VuZGlmIC8qIF9fQVJNX1BDSV9IX18gKi8NCj4+
DQo+DQo+IENoZWVycywNCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:04:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:04:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184358.332965 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgCz-0000nF-3S; Fri, 10 Sep 2021 13:04:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184358.332965; Fri, 10 Sep 2021 13:04:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgCy-0000n8-VR; Fri, 10 Sep 2021 13:04:44 +0000
Received: by outflank-mailman (input) for mailman id 184358;
 Fri, 10 Sep 2021 13:04:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOgCy-0000n2-FL
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:04:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgCw-00077k-Qn; Fri, 10 Sep 2021 13:04:42 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgCw-0002dr-Jl; Fri, 10 Sep 2021 13:04:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=B77i2dOU48JTHDsxL1CJUcUGR+hn26MJ3GotrzId29k=; b=CRt2loXjaw1HLV55jUAwm0d/zD
	BncbRdqs3I6cOr8Zeg6GLyEg1cYg5QntR3/DufkaGsX3C9COAn3OouALVr7Ijm6gnsIoYLr6EHsCT
	jrw/OWue0fCZhDxZL1KSl+9dm2bfVAgyOiYT4KgAvhNIwEGG+xClW2FHkuKGju0A8QHA=;
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
Date: Fri, 10 Sep 2021 14:04:40 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 12:43, Oleksandr Andrushchenko wrote:
> Hi, Julien!

Hi Oleksandr,

> On 09.09.21 20:43, Julien Grall wrote:
>> Hi Oleksandr,
>>
>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> In order vPCI to work it needs all access to PCI configuration space
>>> (ECAM) to be synchronized among all entities, e.g. hardware domain and
>>> guests.
>>
>> I am not entirely sure what you mean by "synchronized" here. Are you refer to the content of the configuration space?
> 
> We maintain hwdom's and guest's view of the registers we are interested in
> 
> So, to have hwdom's view we also need to trap its access to the configuration space.
> 
> Probably "synchronized" is not the right wording here.
I would simply say that we want to expose an emulated hostbridge to the 
dom0 so we need to unmap the configuration space.

> 
>>
>>> For that implement PCI host bridge specific callbacks to
>>> properly setup those ranges depending on particular host bridge
>>> implementation.
>>>
>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>> ---
>>>    xen/arch/arm/pci/ecam.c            | 11 +++++++++++
>>>    xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
>>>    xen/arch/arm/vpci.c                | 13 +++++++++++++
>>>    xen/include/asm-arm/pci.h          |  8 ++++++++
>>>    4 files changed, 48 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
>>> index 91c691b41fdf..92ecb2e0762b 100644
>>> --- a/xen/arch/arm/pci/ecam.c
>>> +++ b/xen/arch/arm/pci/ecam.c
>>> @@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>>>        return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
>>>    }
>>>    +static int pci_ecam_register_mmio_handler(struct domain *d,
>>> +                                          struct pci_host_bridge *bridge,
>>> +                                          const struct mmio_handler_ops *ops)
>>> +{
>>> +    struct pci_config_window *cfg = bridge->sysdata;
>>> +
>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
>>
>> We have a fixed array for handling the MMIO handlers.
> 
> Didn't know that:
> 
> xen/include/asm-arm/mmio.h:27:#define MAX_IO_HANDLER  16
> 
>> So you need to make sure we have enough space in it to store one handler per handler.
>>
>> This is quite similar to the problem we had with the re-distribuor on GICv3. Have a look there to see how we dealt with it.
> 
> Could you please point me to that solution? I can only see
> 
>       /* Register mmio handle for the Distributor */
>       register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
>                             SZ_64K, NULL);
> 
>       /*
>        * Register mmio handler per contiguous region occupied by the
>        * redistributors. The handler will take care to choose which
>        * redistributor is targeted.
>        */
>       for ( i = 0; i < d->arch.vgic.nr_regions; i++ )
>       {
>           struct vgic_rdist_region *region = &d->arch.vgic.rdist_regions[i];
> 
>           register_mmio_handler(d, &vgic_rdistr_mmio_handler,
>                                 region->base, region->size, region);
>       }
> which IMO doesn't care about the number of MMIOs we can handle

Please see vgic_v3_init(). We update mmio_count that is then used as a 
the second argument for domain_io_init().

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:15:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:15:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184368.332976 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgNW-0002Xh-3z; Fri, 10 Sep 2021 13:15:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184368.332976; Fri, 10 Sep 2021 13:15:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgNW-0002Xa-0i; Fri, 10 Sep 2021 13:15:38 +0000
Received: by outflank-mailman (input) for mailman id 184368;
 Fri, 10 Sep 2021 13:15:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOgNU-0002XU-8P
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:15:36 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 50c155ae-bbb2-4ea8-9424-023f5ed4e9d0;
 Fri, 10 Sep 2021 13:15:34 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ACp7RG005427; 
 Fri, 10 Sep 2021 13:15:32 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b0731g8fv-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 13:15:32 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3428.eurprd03.prod.outlook.com (2603:10a6:208:c::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 13:15:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:15:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50c155ae-bbb2-4ea8-9424-023f5ed4e9d0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EpHogWpHWdDEqW5aAatpHV/hy9bQVOxVhgaxTPIh2DGKfEoQQr+8MvY1+9wZ1eRHFVJMaIm2b0RZ9nCW15t1zpV6zXGgMZ/w204X3EUAh0DFbrCqrNgQlpTWzLBjhlYpOuL4C9imifJ9kn7uEvRgHm/0i0QinyfNBjMN+dU9O81p//mc/3luZhjJ3OSU9XxE2Ik+76oHyTjjCbf/DDk0qBopjoXdyOtYQnPG8j48gyVbKJ82bMIbZWwb1NExxcOVZ+Xww4RcCsJ90CavUELCxXbZAFJ/AOq8lJaE1EOABIDwJmUtyfsHxReIHVAb1y1NVcuOg6AUR1rZYdQKPrtcYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qnXFK9J5r336VnHGmOYVR3MgZv7b7Vu+tI5JU+V9tGk=;
 b=iqrVOGOV6CgMCsIcTwS1CLkkjGdT0Qgw+INNnFc10BW0mW76Jt1ptL/xEiVaw7rJRFcL2BUlMUOSrKQ4dmtdA/25H6ypeE5HmBANSM132OFEFcYndG8eDVMsfmzkD98H91LnRXrbj8ui5hlJ6w49I0JGGZzipMy5T8Og9Kr+v0P2cG/QaBk84gSfO2PpmqRPtHqxzI9dB3D3Z0DPYW6MOPQg58D3+5M7Dm1UeePKPgdJ9I9p1oDZgtwYpgn4Dm0uzDBaedyL3zghSEE/c4U1yen59HdV113x5F3ckvGDmlFzj/8/cVub052coStaDUg/D4gn1RoBZpygjWNqOlKKCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qnXFK9J5r336VnHGmOYVR3MgZv7b7Vu+tI5JU+V9tGk=;
 b=LG1PUcSxJt2VOh5scTVoWcLxY6NPtkeiQlbspN9ha+GfIzL1eyuGlMFrIfH2LOlLG1e3hShQmhrIx1BjFBvaRp2zW5QQmt6Z+rao9HolxOPjKgXSiFbdukcKdsdvlF1t5T+lF1ryY0KxH03C0u9GseGCqLSJ/SRF6/K6kzN2QLjvNyU7iRiHw2RTYP4TkZdCCv6V5J3aVlxsMZYNRT8p0XKPCyz+tqkcX2LJHEeVO6jD5u6kF5iIcX7W+xXm6KmX8dqQvSs9uY+S/xJLrepwowKAj0DpeG+B17VFLL+4xf/8mFFCx3KkN/H580ePA59TAnr7zUbu8hYzptiFnaXBvg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4ACAABanAIAAAwIA
Date: Fri, 10 Sep 2021 13:15:26 +0000
Message-ID: <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
In-Reply-To: <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b85f6c8d-e9bc-40f8-85d8-08d9745d0e51
x-ms-traffictypediagnostic: AM0PR0302MB3428:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR0302MB3428BF462675C7261A879AEBE7D69@AM0PR0302MB3428.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 QrGgf4ivqeWU0BVZ1gW9aeWZ1RbbnmFJmWX0adv61yUinr5s2yacVgf7mOArBAsHpVCjKIyVMi5IYx99eKyOSiIFai/8T3/OxwXxLUdMCK/i6G80VRCkjh/dEUJsz94dC9KznU7A9MWu8ZkGPNfbbUMjctmQ+6MA4uTnpNi0R0AyurKp6IObVCDDo+aYT5hFBHk7Ek65m/vQOIcxft/vIlHRYDgZh30N964ATrUiy2/8LvBdZ0i2gjpUoL2GdCFSJ+vvohFW6YaLI99X0h4SpQyJOyiuBeZFfXoCUfPCA4OASDATKkzyYPVDQFomSs/NOvlrrYzQS2gNDN41iq54tubsAiULNg7uM7k5ZKXtRo3YHrQVaIQ3Hb93Ai/fYHi86GOCGcGEWrqeIK0Gq0yjb9m6TcuBDub9kkPQL6eF6FMsMTHOi0yNs7JMfP6fVxc3QzXE766f9LiC1dBnpvXQplTuVWCqOXIzgAQw9m5oq3v1zTSbgmDwPoOTY1uh3rJleks9dM++1iWmVPGj1HZtYEkSIhdyka2bhlKfzZhE+2489zVSeonbT8Ae9l8cngtc3TH/k7pPxmTPuFyHBkxAQ4qkUIoph0XunP+BtAknsBLgi9kZsBf8OnWEe2b+Znh0XUaP7VhuUDwQrmf8qvsHKpw22dFfDOegwObfL0nbwK18cUUJO6qkU5EuU3Qk5TelM1tK/YirPAXCbokAueUFbf1JR271judHM8xMLafPeZs=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(366004)(396003)(376002)(346002)(8936002)(6506007)(91956017)(38100700002)(53546011)(31696002)(66946007)(76116006)(55236004)(86362001)(66476007)(66556008)(64756008)(66446008)(26005)(36756003)(316002)(2616005)(54906003)(110136005)(2906002)(4326008)(186003)(8676002)(6486002)(122000001)(71200400001)(31686004)(478600001)(38070700005)(83380400001)(6512007)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dkw0UmtyTHE1aHNQNlo2ZzE4LytxUUhXWGpFQXA3cm9GN01nc041VENIeDhy?=
 =?utf-8?B?bU4zYTZOeGFUWGROQzEweHgrbjVSQlhVTmpSL1NsR2pJN3g5UkFnZE92T0Jw?=
 =?utf-8?B?dW1ndUlJak5mMXhFWElPZHZLeEJEd1RBK1VTTUJyT01hUTh3KzJiOWpVTDhH?=
 =?utf-8?B?RTgxQVorKysyRjcwWUZzR3ZHdjZuemJpZExUN2lZZ2pqQlNsNHFoQmFJY3ZG?=
 =?utf-8?B?bmI0bnNoYit6RXZXNHlFbW9iSzNVdDJWWXdJWXZNZjF6WlVrZ1dNVkhxNjM2?=
 =?utf-8?B?TGlOeER0enBKM1ZHaGNLRmhmUjNWMDBLWHYwYXROeHpoUGF2bW55MnVLRkdT?=
 =?utf-8?B?NzVLT09UcDNGdk5QaGpOSFFLSys0VzhuZWFiT05XeVlqeW0wZ3YvRTFBSjF3?=
 =?utf-8?B?TWtKMGVKeTBETEViYUtiZEtUYVozYThoTFM1d09sQlBtaC9jSThIdXJtdmEx?=
 =?utf-8?B?MmdZNDlSYXhYYzlveWNONGs2S2pXMkhLUThibWpLZGRXVlRtU3plM2tvcUw5?=
 =?utf-8?B?UlBqZm1OWGtlc2VqbnEvRit6TVViWHA1NlBFWU5tK3NscTZxbStMQUZFWGZQ?=
 =?utf-8?B?YXBLUm9zWGpPeXpZVUZwVnhOc3NNbTZLNko2WStGbjNMM1pxNjhhcmRscGN0?=
 =?utf-8?B?eHIvUmhSRDVteDlrbERDNU5nY0lTQzE0NHVDb0JvT21NZEZORXlZRWpVUzRN?=
 =?utf-8?B?clhrTE1penRGWEs4NERiS2F6aDl0aXdpNUZoaHB2ZitXSGxZZ3NqM3RkR2sx?=
 =?utf-8?B?YXVOMmUxdDFTZWRQUGxjMlpubHhwMElOSVRoYjhyTFVJbzJpbHRNelk1WXox?=
 =?utf-8?B?dmxFaFY4R0d1UERmWG1MMVBhOGwwSXlkd0NiMXhVdmJiZkM2bUhRdVN3MVBS?=
 =?utf-8?B?a3FSN253RVBSVlJBOWpiekdPY0VoQS9hRFM4VUQ4d2QwRTc0QXMyUk5jRVFI?=
 =?utf-8?B?WmJOR1YvUjgvTFR6bkI1TTZOUFJBbGtBWTlFdzJRUDl3cTZ2Y2IzRUpZMEx4?=
 =?utf-8?B?Z0tMM201bHdrOWNHTkE5dFZmSHM0L1k1SmVWWTZIVS91ZnNlMFZOeDZJQ2dB?=
 =?utf-8?B?UmRCL0tILzVwNG5oT1VKMzBYMjZlMkFwY0EvSlRuVCtHYitkcC9mbkRXMDhl?=
 =?utf-8?B?UnJ5dmwwMU9LSDRQR21ZYUVBWmJNcGxEU2FDOFlSSFByeWJZQlJINXlIU1gz?=
 =?utf-8?B?V3U0SXFMT2JVSUFCU1pxZlZkZ3pDaXF3YXgxTzRHODQ0VDNiUVFEd1dCdExG?=
 =?utf-8?B?emFiM2dwem5rN0kvNmk5dy84TjdJcXRRMC9DRzBGQXhEMm10d09kTTE2OW14?=
 =?utf-8?B?U0hPRFFhWFZOeHZ0QnB5Mkxna29kOEdnUjYrU0ppSXZ4Q1k2SkU3Z21xZWQ4?=
 =?utf-8?B?LzdzcnlxZ3dNNTNvSkp0MG1PbXYrWjViV2x2WGF4eEErL25YL1JTSklqNHZM?=
 =?utf-8?B?eWJ3ZXpMcHZoRWFiQWRkRWxrZnZUTjY3aGhETnJEOWlQcW5ySElqbCtKNjNC?=
 =?utf-8?B?eGlYYlFuRlNqZE5wRFFoeUZSamt1bXFYZkhiRkw1TlBLMTBBOGZJV0k5VGtn?=
 =?utf-8?B?ZUc0T0htdklHOG9aM01DQmRlSURCTW4rZ3FybjJQYlE0SVRITGpGQmY0ZVhz?=
 =?utf-8?B?TVk3bXBMaGQ0b081MVVCckNTSTN2TFJjR3NrampvZTBZSVJ4c2Q0THllZmx3?=
 =?utf-8?B?V0JpWUF1WXFVWC9aNGtyejE2eGI4VXljNmJtQXpmZDVCcE9aL1YwdXB3SVZB?=
 =?utf-8?B?NVU0ZGJYaFY5bk5NYnVaeTJRM1FnUCs0MFdaMkxpalA5ZkY2M3hrRW1nTyt1?=
 =?utf-8?B?a3QxeEZUZThhVi9qN3lYQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A96FCB11BC4EAA408DEB06BC5EFC19D5@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b85f6c8d-e9bc-40f8-85d8-08d9745d0e51
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 13:15:26.9793
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: PyYqw8e3GMhqiemR1tOwDtP17Z+JpxDs87rb6hVVBNDU114rKM8iayPFdLgXV7wUcN7AifVUvEQT8GjrJ9AySZCkrkw0bI4CO5xbLd1GjRGLu+YN0QTYfBU0KrhESgmW
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3428
X-Proofpoint-GUID: Xt37kSfilYmwZvCTPDxB4hmXu91Mv1MB
X-Proofpoint-ORIG-GUID: Xt37kSfilYmwZvCTPDxB4hmXu91Mv1MB
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0
 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 mlxlogscore=999
 impostorscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100078

SGksIEp1bGllbiENCg0KT24gMTAuMDkuMjEgMTY6MDQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4N
Cj4NCj4gT24gMTAvMDkvMjAyMSAxMjo0MywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6
DQo+PiBIaSwgSnVsaWVuIQ0KPg0KPiBIaSBPbGVrc2FuZHIsDQo+DQo+PiBPbiAwOS4wOS4yMSAy
MDo0MywgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPj4+IEhpIE9sZWtzYW5kciwNCj4+Pg0KPj4+IE9u
IDAzLzA5LzIwMjEgMDk6MzMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBG
cm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBh
bS5jb20+DQo+Pj4+DQo+Pj4+IEluIG9yZGVyIHZQQ0kgdG8gd29yayBpdCBuZWVkcyBhbGwgYWNj
ZXNzIHRvIFBDSSBjb25maWd1cmF0aW9uIHNwYWNlDQo+Pj4+IChFQ0FNKSB0byBiZSBzeW5jaHJv
bml6ZWQgYW1vbmcgYWxsIGVudGl0aWVzLCBlLmcuIGhhcmR3YXJlIGRvbWFpbiBhbmQNCj4+Pj4g
Z3Vlc3RzLg0KPj4+DQo+Pj4gSSBhbSBub3QgZW50aXJlbHkgc3VyZSB3aGF0IHlvdSBtZWFuIGJ5
ICJzeW5jaHJvbml6ZWQiIGhlcmUuIEFyZSB5b3UgcmVmZXIgdG8gdGhlIGNvbnRlbnQgb2YgdGhl
IGNvbmZpZ3VyYXRpb24gc3BhY2U/DQo+Pg0KPj4gV2UgbWFpbnRhaW4gaHdkb20ncyBhbmQgZ3Vl
c3QncyB2aWV3IG9mIHRoZSByZWdpc3RlcnMgd2UgYXJlIGludGVyZXN0ZWQgaW4NCj4+DQo+PiBT
bywgdG8gaGF2ZSBod2RvbSdzIHZpZXcgd2UgYWxzbyBuZWVkIHRvIHRyYXAgaXRzIGFjY2VzcyB0
byB0aGUgY29uZmlndXJhdGlvbiBzcGFjZS4NCj4+DQo+PiBQcm9iYWJseSAic3luY2hyb25pemVk
IiBpcyBub3QgdGhlIHJpZ2h0IHdvcmRpbmcgaGVyZS4NCj4gSSB3b3VsZCBzaW1wbHkgc2F5IHRo
YXQgd2Ugd2FudCB0byBleHBvc2UgYW4gZW11bGF0ZWQgaG9zdGJyaWRnZSB0byB0aGUgZG9tMCBz
byB3ZSBuZWVkIHRvIHVubWFwIHRoZSBjb25maWd1cmF0aW9uIHNwYWNlLg0KU291bmRzIGdvb2QN
Cj4NCj4+DQo+Pj4NCj4+Pj4gRm9yIHRoYXQgaW1wbGVtZW50IFBDSSBob3N0IGJyaWRnZSBzcGVj
aWZpYyBjYWxsYmFja3MgdG8NCj4+Pj4gcHJvcGVybHkgc2V0dXAgdGhvc2UgcmFuZ2VzIGRlcGVu
ZGluZyBvbiBwYXJ0aWN1bGFyIGhvc3QgYnJpZGdlDQo+Pj4+IGltcGxlbWVudGF0aW9uLg0KPj4+
Pg0KPj4+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRy
X2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+IC0tLQ0KPj4+PiDCoMKgIHhlbi9hcmNoL2Fy
bS9wY2kvZWNhbS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDExICsrKysrKysrKysrDQo+Pj4+
IMKgwqAgeGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYyB8IDE2ICsrKysrKysrKysr
KysrKysNCj4+Pj4gwqDCoCB4ZW4vYXJjaC9hcm0vdnBjaS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIHwgMTMgKysrKysrKysrKysrKw0KPj4+PiDCoMKgIHhlbi9pbmNsdWRlL2FzbS1h
cm0vcGNpLmjCoMKgwqDCoMKgwqDCoMKgwqAgfMKgIDggKysrKysrKysNCj4+Pj4gwqDCoCA0IGZp
bGVzIGNoYW5nZWQsIDQ4IGluc2VydGlvbnMoKykNCj4+Pj4NCj4+Pj4gZGlmZiAtLWdpdCBhL3hl
bi9hcmNoL2FybS9wY2kvZWNhbS5jIGIveGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMNCj4+Pj4gaW5k
ZXggOTFjNjkxYjQxZmRmLi45MmVjYjJlMDc2MmIgMTAwNjQ0DQo+Pj4+IC0tLSBhL3hlbi9hcmNo
L2FybS9wY2kvZWNhbS5jDQo+Pj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+Pj4+
IEBAIC00Miw2ICs0MiwxNiBAQCB2b2lkIF9faW9tZW0gKnBjaV9lY2FtX21hcF9idXMoc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4+PiDCoMKgwqDCoMKgwqAgcmV0dXJuIGJhc2Ug
KyAoUENJX0RFVkZOKHNiZGZfdC5kZXYsIHNiZGZfdC5mbikgPDwgZGV2Zm5fc2hpZnQpICsgd2hl
cmU7DQo+Pj4+IMKgwqAgfQ0KPj4+PiDCoMKgICtzdGF0aWMgaW50IHBjaV9lY2FtX3JlZ2lzdGVy
X21taW9faGFuZGxlcihzdHJ1Y3QgZG9tYWluICpkLA0KPj4+PiArwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+Pj4+ICvCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIGNvbnN0IHN0cnVjdCBtbWlvX2hhbmRsZXJfb3BzICpvcHMpDQo+Pj4+
ICt7DQo+Pj4+ICvCoMKgwqAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmcgPSBicmlkZ2Ut
PnN5c2RhdGE7DQo+Pj4+ICsNCj4+Pj4gK8KgwqDCoCByZWdpc3Rlcl9tbWlvX2hhbmRsZXIoZCwg
b3BzLCBjZmctPnBoeXNfYWRkciwgY2ZnLT5zaXplLCBOVUxMKTsNCj4+Pg0KPj4+IFdlIGhhdmUg
YSBmaXhlZCBhcnJheSBmb3IgaGFuZGxpbmcgdGhlIE1NSU8gaGFuZGxlcnMuDQo+Pg0KPj4gRGlk
bid0IGtub3cgdGhhdDoNCj4+DQo+PiB4ZW4vaW5jbHVkZS9hc20tYXJtL21taW8uaDoyNzojZGVm
aW5lIE1BWF9JT19IQU5ETEVSwqAgMTYNCj4+DQo+Pj4gU28geW91IG5lZWQgdG8gbWFrZSBzdXJl
IHdlIGhhdmUgZW5vdWdoIHNwYWNlIGluIGl0IHRvIHN0b3JlIG9uZSBoYW5kbGVyIHBlciBoYW5k
bGVyLg0KPj4+DQo+Pj4gVGhpcyBpcyBxdWl0ZSBzaW1pbGFyIHRvIHRoZSBwcm9ibGVtIHdlIGhh
ZCB3aXRoIHRoZSByZS1kaXN0cmlidW9yIG9uIEdJQ3YzLiBIYXZlIGEgbG9vayB0aGVyZSB0byBz
ZWUgaG93IHdlIGRlYWx0IHdpdGggaXQuDQo+Pg0KPj4gQ291bGQgeW91IHBsZWFzZSBwb2ludCBt
ZSB0byB0aGF0IHNvbHV0aW9uPyBJIGNhbiBvbmx5IHNlZQ0KPj4NCj4+IMKgIMKgwqDCoCAvKiBS
ZWdpc3RlciBtbWlvIGhhbmRsZSBmb3IgdGhlIERpc3RyaWJ1dG9yICovDQo+PiDCoCDCoMKgwqAg
cmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsICZ2Z2ljX2Rpc3RyX21taW9faGFuZGxlciwgZC0+YXJj
aC52Z2ljLmRiYXNlLA0KPj4gwqAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgU1pfNjRLLCBOVUxMKTsNCj4+DQo+PiDCoCDCoMKgwqAgLyoNCj4+IMKg
IMKgwqDCoMKgICogUmVnaXN0ZXIgbW1pbyBoYW5kbGVyIHBlciBjb250aWd1b3VzIHJlZ2lvbiBv
Y2N1cGllZCBieSB0aGUNCj4+IMKgIMKgwqDCoMKgICogcmVkaXN0cmlidXRvcnMuIFRoZSBoYW5k
bGVyIHdpbGwgdGFrZSBjYXJlIHRvIGNob29zZSB3aGljaA0KPj4gwqAgwqDCoMKgwqAgKiByZWRp
c3RyaWJ1dG9yIGlzIHRhcmdldGVkLg0KPj4gwqAgwqDCoMKgwqAgKi8NCj4+IMKgIMKgwqDCoCBm
b3IgKCBpID0gMDsgaSA8IGQtPmFyY2gudmdpYy5ucl9yZWdpb25zOyBpKysgKQ0KPj4gwqAgwqDC
oMKgIHsNCj4+IMKgIMKgwqDCoMKgwqDCoMKgIHN0cnVjdCB2Z2ljX3JkaXN0X3JlZ2lvbiAqcmVn
aW9uID0gJmQtPmFyY2gudmdpYy5yZGlzdF9yZWdpb25zW2ldOw0KPj4NCj4+IMKgIMKgwqDCoMKg
wqDCoMKgIHJlZ2lzdGVyX21taW9faGFuZGxlcihkLCAmdmdpY19yZGlzdHJfbW1pb19oYW5kbGVy
LA0KPj4gwqAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCByZWdpb24tPmJhc2UsIHJlZ2lvbi0+c2l6ZSwgcmVnaW9uKTsNCj4+IMKgIMKg
wqDCoCB9DQo+PiB3aGljaCBJTU8gZG9lc24ndCBjYXJlIGFib3V0IHRoZSBudW1iZXIgb2YgTU1J
T3Mgd2UgY2FuIGhhbmRsZQ0KPg0KPiBQbGVhc2Ugc2VlIHZnaWNfdjNfaW5pdCgpLiBXZSB1cGRh
dGUgbW1pb19jb3VudCB0aGF0IGlzIHRoZW4gdXNlZCBhcyBhIHRoZSBzZWNvbmQgYXJndW1lbnQg
Zm9yIGRvbWFpbl9pb19pbml0KCkuDQoNCkFoLCBzbw0KDQoxKSBUaGlzIG5lZWRzIHRvIGJlIGRv
bmUgYmVmb3JlIHRoZSBhcnJheSBmb3IgdGhlIGhhbmRsZXJzIGlzIGFsbG9jYXRlZA0KDQoyKSBI
b3cgZG8gSSBrbm93IGF0IHRoZSB0aW1lIG9mIDEpIGhvdyBtYW55IGJyaWRnZXMgd2UgaGF2ZT8N
Cg0KPg0KPiBDaGVlcnMsDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:18:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:18:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184375.332987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgQ7-0003CO-Io; Fri, 10 Sep 2021 13:18:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184375.332987; Fri, 10 Sep 2021 13:18:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgQ7-0003CG-Fo; Fri, 10 Sep 2021 13:18:19 +0000
Received: by outflank-mailman (input) for mailman id 184375;
 Fri, 10 Sep 2021 13:18:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOgQ5-0003CA-TX
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:18:17 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgQ3-0007QG-TB; Fri, 10 Sep 2021 13:18:15 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgQ3-0003ue-Mj; Fri, 10 Sep 2021 13:18:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=ReZTkjPctmI/lwgTipBy5OFGmkW2i3Wiy8e3pxjxuBw=; b=kZXiRJ/XZK6EJDUWZi2VYzT95P
	mhwsFhxYAbWDPUTypgo7oMvY3D2biIiw19oqw8i7zSz574pXJiKEBtTEoAoc5CwWD20/zAGBGPJVY
	PpSvkydROmGbtZeA5F1M7IWVex28lUjvKhvy2pnm2iymlNYI6FZNjZ3ZvLfmM1fQ4XpU=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
Date: Fri, 10 Sep 2021 14:18:13 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:
> Hi, Julien!

Hi Oleksandr,

> On 09.09.21 20:58, Julien Grall wrote:
>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> Host bridge controller's ECAM space is mapped into Domain-0's p2m,
>>> thus it is not possible to trap the same for vPCI via MMIO handlers.
>>> For this to work we need to not map those while constructing the domain.
>>>
>>> Note, that during Domain-0 creation there is no pci_dev yet allocated for
>>> host bridges, thus we cannot match PCI host and its associated
>>> bridge by SBDF. Use dt_device_node field for checks instead.
>>>
>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>> ---
>>>    xen/arch/arm/domain_build.c        |  3 +++
>>>    xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
>>>    xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
>>>    xen/include/asm-arm/pci.h          | 12 ++++++++++++
>>>    4 files changed, 54 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index da427f399711..76f5b513280c 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
>>>            }
>>>        }
>>>    +    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI) ) > +        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev,
>> addr, len);
>>
>> AFAICT, with device_get_class(dev), you know whether the hostbridge is used by Xen. Therefore, I would expect that we don't want to map all the regions of the hostbridges in dom0 (including the BARs).
>>
>> Can you clarify it?
> We only want to trap ECAM, not MMIOs and any other memory regions as the bridge is
> 
> initialized and used by Domain-0 completely. 

What do you mean by "used by Domain-0 completely"? The hostbridge is 
owned by Xen so I don't think we can let dom0 access any MMIO regions by
default.

In particular, we may want to hide a device from dom0 for security 
reasons. This is not going to be possible if you map by default 
everything to dom0.

Instead, the BARs should be mapped on demand when dom0 when we trap 
access to the configuration space.

For other regions, could you provide an example of what you are 
referring too?

>>> +
>>> +    /*
>>> +     * We do not want ECAM address space to be mapped in domain's p2m,
>>> +     * so we can trap access to it.
>>> +     */
>>> +    return cfg->phys_addr != addr;
>>> +}
>>> +
>>>    /* ECAM ops */
>>>    const struct pci_ecam_ops pci_generic_ecam_ops = {
>>>        .bus_shift  = 20,
>>> @@ -60,6 +76,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>>>            .read                   = pci_generic_config_read,
>>>            .write                  = pci_generic_config_write,
>>>            .register_mmio_handler  = pci_ecam_register_mmio_handler,
>>> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>>>        }
>>>    };
>>>    diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
>>> index a89112bfbb7c..c04be636452d 100644
>>> --- a/xen/arch/arm/pci/pci-host-common.c
>>> +++ b/xen/arch/arm/pci/pci-host-common.c
>>> @@ -334,6 +334,28 @@ int pci_host_iterate_bridges(struct domain *d,
>>>        }
>>>        return 0;
>>>    }
>>> +
>>> +bool pci_host_bridge_need_p2m_mapping(struct domain *d,
>>> +                                      const struct dt_device_node *node,
>>> +                                      uint64_t addr, uint64_t len)
>>> +{
>>> +    struct pci_host_bridge *bridge;
>>> +
>>> +    list_for_each_entry( bridge, &pci_host_bridges, node )
>>> +    {
>>> +        if ( bridge->dt_node != node )
>>> +            continue;
>>> +
>>> +        if ( !bridge->ops->need_p2m_mapping )
>>> +            return true;
>>> +
>>> +        return bridge->ops->need_p2m_mapping(d, bridge, addr, len);
>>> +    }
>>> +    printk(XENLOG_ERR "Unable to find PCI bridge for %s segment %d, addr %lx\n",
>>> +           node->full_name, bridge->segment, addr);
>>> +    return true;
>>> +}
>>
>> If you really need to map the hostbridges, then I would suggest to defer the P2M mappings for all of them and then walk all the bridge in one go to do the mappings.
>>
>> This would avoid going through the bridges every time during setup.
> 
> Well, this can be done, but: my implementation prevents mappings and what
> 
> you suggest will require unmapping. So, the cost in my solution is we need
> 
> to go over all the bridges (until we find the one we need) and in what you
> 
> suggest we'll need to unmap what we have just mapped.
> 
> I think preventing unneeded mappings is cheaper than unmapping afterwards.

I think you misundertood what I am suggesting. What I said is you could 
defer the mappings (IOW not do the mapping) until later for the 
hostbridges. And then you can walk all the hostbridges to decide how to 
map them.

The regions will only mapped once and never be unmapped.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:21:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:21:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184382.332998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgSe-0004bg-3r; Fri, 10 Sep 2021 13:20:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184382.332998; Fri, 10 Sep 2021 13:20:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgSe-0004bZ-0d; Fri, 10 Sep 2021 13:20:56 +0000
Received: by outflank-mailman (input) for mailman id 184382;
 Fri, 10 Sep 2021 13:20:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOgSc-0004bQ-Gs
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:20:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgSa-0007Tt-7A; Fri, 10 Sep 2021 13:20:52 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgSZ-00049a-T4; Fri, 10 Sep 2021 13:20:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=/PeZxYBYI8hoEO4UWcgM3yIVrA953BkRdFDn3iqBSj8=; b=wLOGaeDQOd6IqbLaoGmisr6wUB
	Oxgnl91L2Q0m61hkE0FD5T4imnkp8tNNVfGBrf4hXcGnL7XbGdFq1w42EMpz1wjUQgRPz+0nO/jJg
	CXAc7meqTRa/lKiwfEPDTwLTAkOCSwefQ6HeFt314iyFHiBD2Y+qWUgTeS8xddS6CrAQ=;
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
Date: Fri, 10 Sep 2021 14:20:49 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 14:15, Oleksandr Andrushchenko wrote:
> Hi, Julien!

Hi,

> On 10.09.21 16:04, Julien Grall wrote:
>>
>>
>> On 10/09/2021 12:43, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>
>> Hi Oleksandr,
>>
>>> On 09.09.21 20:43, Julien Grall wrote:
>>>> Hi Oleksandr,
>>>>
>>>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> In order vPCI to work it needs all access to PCI configuration space
>>>>> (ECAM) to be synchronized among all entities, e.g. hardware domain and
>>>>> guests.
>>>>
>>>> I am not entirely sure what you mean by "synchronized" here. Are you refer to the content of the configuration space?
>>>
>>> We maintain hwdom's and guest's view of the registers we are interested in
>>>
>>> So, to have hwdom's view we also need to trap its access to the configuration space.
>>>
>>> Probably "synchronized" is not the right wording here.
>> I would simply say that we want to expose an emulated hostbridge to the dom0 so we need to unmap the configuration space.
> Sounds good
>>
>>>
>>>>
>>>>> For that implement PCI host bridge specific callbacks to
>>>>> properly setup those ranges depending on particular host bridge
>>>>> implementation.
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>> ---
>>>>>     xen/arch/arm/pci/ecam.c            | 11 +++++++++++
>>>>>     xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
>>>>>     xen/arch/arm/vpci.c                | 13 +++++++++++++
>>>>>     xen/include/asm-arm/pci.h          |  8 ++++++++
>>>>>     4 files changed, 48 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
>>>>> index 91c691b41fdf..92ecb2e0762b 100644
>>>>> --- a/xen/arch/arm/pci/ecam.c
>>>>> +++ b/xen/arch/arm/pci/ecam.c
>>>>> @@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>>>>>         return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
>>>>>     }
>>>>>     +static int pci_ecam_register_mmio_handler(struct domain *d,
>>>>> +                                          struct pci_host_bridge *bridge,
>>>>> +                                          const struct mmio_handler_ops *ops)
>>>>> +{
>>>>> +    struct pci_config_window *cfg = bridge->sysdata;
>>>>> +
>>>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
>>>>
>>>> We have a fixed array for handling the MMIO handlers.
>>>
>>> Didn't know that:
>>>
>>> xen/include/asm-arm/mmio.h:27:#define MAX_IO_HANDLER  16
>>>
>>>> So you need to make sure we have enough space in it to store one handler per handler.
>>>>
>>>> This is quite similar to the problem we had with the re-distribuor on GICv3. Have a look there to see how we dealt with it.
>>>
>>> Could you please point me to that solution? I can only see
>>>
>>>        /* Register mmio handle for the Distributor */
>>>        register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
>>>                              SZ_64K, NULL);
>>>
>>>        /*
>>>         * Register mmio handler per contiguous region occupied by the
>>>         * redistributors. The handler will take care to choose which
>>>         * redistributor is targeted.
>>>         */
>>>        for ( i = 0; i < d->arch.vgic.nr_regions; i++ )
>>>        {
>>>            struct vgic_rdist_region *region = &d->arch.vgic.rdist_regions[i];
>>>
>>>            register_mmio_handler(d, &vgic_rdistr_mmio_handler,
>>>                                  region->base, region->size, region);
>>>        }
>>> which IMO doesn't care about the number of MMIOs we can handle
>>
>> Please see vgic_v3_init(). We update mmio_count that is then used as a the second argument for domain_io_init().
> 
> Ah, so
> 
> 1) This needs to be done before the array for the handlers is allocated
> 
> 2) How do I know at the time of 1) how many bridges we have?

By counting the number of bridge you want to expose to dom0? I am not 
entirely sure what else you expect me to say.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:22:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:22:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184391.333008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgUa-0005Xy-Fd; Fri, 10 Sep 2021 13:22:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184391.333008; Fri, 10 Sep 2021 13:22:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgUa-0005Xr-Co; Fri, 10 Sep 2021 13:22:56 +0000
Received: by outflank-mailman (input) for mailman id 184391;
 Fri, 10 Sep 2021 13:22:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOgUZ-0005Xl-2x
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:22:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1b25ffb4-c141-4374-ac28-b233c2df4476;
 Fri, 10 Sep 2021 13:22:54 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2059.outbound.protection.outlook.com [104.47.4.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-TCh-We65NYSj-URFog50YQ-1; Fri, 10 Sep 2021 15:22:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4606.eurprd04.prod.outlook.com (2603:10a6:803:70::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.23; Fri, 10 Sep
 2021 13:22:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:22:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0158.eurprd01.prod.exchangelabs.com (2603:10a6:208:aa::27) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Fri, 10 Sep 2021 13:22:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b25ffb4-c141-4374-ac28-b233c2df4476
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631280173;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oiFVKQIxRWxLlX3K6tsHaH0EGKjhQ4+KcCSrfELvS60=;
	b=RRxFzjd58ZjYgdNcnIJnvt8rNRaMueDap/U5NPCty2V2vEP66L3bDDMKwS9RUqVizxoN/q
	uuPtdEdIi9QMDUqeq5CKg1hZMWmDD5EWnMHXT4OZ+aqzMmN1+K3snb2JvtqJzUlBkjBdr2
	HGHpZRCgf/gQjDzRjZ+EZBrgUjiUUdA=
X-MC-Unique: TCh-We65NYSj-URFog50YQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XTAoj5ne5WynUrGqiKgxtqoJoatjjYrbM080Fw4wwIqkSuK6ycNI5q4lZp6k0f4u7jfbqInMr/DlcbACEiMWSTyMKz1q2g7Uo+jk9l2BUz6gnDiEsbh6628kGsg89Pv+9hMOWiU8UQKBbdoz3yepv/MRGsmoJ8vSFFtKt4VQKmlGR24nHiB3i4WvdKZTRLJVbaqF8j0JOpl0iNlsNXA4bW/3rh6Xw9vEvxdXtU0VbEJwTSA9VBZzuz//nYLoAuVlYfPFyL/0o8GPcyvv2I+rNnAZCICfJ6xs507NZTI/mSwyeuaCuwJDddxIRwLexQggAYABvbIeODpAapBurIHfCw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oiFVKQIxRWxLlX3K6tsHaH0EGKjhQ4+KcCSrfELvS60=;
 b=RbKFTyxBHbBVqB3MImGuaoCYl0OIUC0gPy8yGOTIMST6ASG58lYi6vYCduWeOeCr0d/kLnodmC0oKJ1yBzy8huZ1c/BZiLl4IIP4vu+cu1DDx1kQ3mzsd2n3ihHMJDNlaZHUwoAYdOm7qO49wuR0wyWpQYyjaAhzFt4zXXp9EP0ky8Zv17wQQSBHnKpaobwAdilS75EyEw0UNxW3dhs1r5DJfeRp0XolEl6Bb7/Ha10nKVflUCYGPCfU0zRYEOAgbDJtrNdPRmjOo8J6evy0ogTdYUAoFgfCA49w0r1RVR/d4vVY3HN2c73wfnBGLfz01ZqLuxNlQaoAdL1lpoz28w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
Date: Fri, 10 Sep 2021 15:22:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0158.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:aa::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c8278f10-15a3-4d6e-9f8a-08d9745e1590
X-MS-TrafficTypeDiagnostic: VI1PR04MB4606:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB46060558A3FB7A84DDC67DBFB3D69@VI1PR04MB4606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0IZYz2zhqzaq2V4Y4j6gSB16i/DP0ZwzE2x3QQX9dr/9s3w7nqD0xjfDVsa0nQe1o9yN6YSXB7Z3GsWvwGt81KoQjzCqtNKObHF41k4nh+jxU1HUU4JArnN3wTkkByqIdsLHUaKHqRw8NpJN6an+YN7qVjGXRD0sYjf6DnPp2jGhaRvHubj+KsvpiqF0D2RXgpkzgvFdT9/MwM+a3Tv6YSnVRnlfYIcfRyoLDLvg5aZ09w62slcsmmnaXyg4RhCaKKgSWDakGTy+ux3ochTEUercAQ5N5XoCI4TCAZchsOOiEqfcgR10q2zKTDbxI5O4mxGQePrHgiQrdnRSurXx+pjeTg1S7Xe+cgKGRRsEpZL/9sPNVeH1t7APP26iAUpU1nUzjwvHi+ToUV9rEqKIN5orlGAIYqGcLRM9zhV4+r4jQq94xZDdf0/TnfVbXzKY5/UQRckX6xQuUaSvC4ZhwSwmuKFERwW6LeJUxuuDih+5316Eib/RAO5YAYx1WfU+H1GRs/yQ82XWcamxaHQqqvlUAbAJYUi4iU7T4JmK71ebCxbGgUsoMDFkm5/1pMRCe7e/p8VMTpN94Z/PTCyTMuLT2GOYPzihkCzBwq2LOuk4pY0wcs/E9RjMoCL2ScAWpABt/LXP3M+fWmqZ85zvZXKpVUqwFzcts5RwO3fN/d2EjKcsaT1SYOdDvrEtDiKfut8QB+PpJsOX0r0FdMcZvQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(346002)(39860400002)(376002)(396003)(366004)(31696002)(6862004)(54906003)(36756003)(5660300002)(186003)(2616005)(38100700002)(26005)(86362001)(66946007)(8936002)(6486002)(53546011)(16576012)(2906002)(8676002)(316002)(31686004)(6636002)(83380400001)(37006003)(478600001)(66556008)(66476007)(4326008)(956004)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VmE0QXo3OFVSSkRCWGFTMHJpbmhhOFlhRm1aZWY4WGNVOXp2Rk5pS01wV3dt?=
 =?utf-8?B?d2l0Tlo1Z0NaMkF3alRZWEZGNWFUKzFvZEVrempnaEJpeFRUSmErcHdWUEh4?=
 =?utf-8?B?V0Z6dnNtNk9UVnpWb0dmTFREaitoUU1nOUNKK3lPUVY1R3JxYTVEanBTdEhz?=
 =?utf-8?B?a2lCbjF5aHUzZDZIb2dLdnNpcHJaeTJ1NHAySjV6ZGJtM3c0bHNzbkdUUHFD?=
 =?utf-8?B?SWV2VVZBQnY2aXNUSS9QSUJQbU1sUGh6dWp4Y01iWjdCOWwrazlYOEZtWUk2?=
 =?utf-8?B?TDl3N0l4VENrOHJpTy92ZXhESXdvQTNiS3hYZVF1NUliTkRQekN0bjl0YVBx?=
 =?utf-8?B?YzVTRmhvK1JIc29Tb3ZadE0vZTNkajg5T2wxd2pQa09EM2xBS2JmaDVZQmZZ?=
 =?utf-8?B?c25uRVY1VVVmajBaeXBsTDFYM0IzR1d4eHFpYTJ4VDBLWVg5K0lxRmJ6bGYv?=
 =?utf-8?B?cDhYbjE3Y3NmN04zM01Fa2dlZXBzSUlDb0RabUFtUVJBdkpOUEJiRll6NWJn?=
 =?utf-8?B?TXk1anowTm5UWVBWWE9ZekZodFdFYll1ZENCaHpoc0FEbTNFSmpHSFdqSnhw?=
 =?utf-8?B?ZXhaK2J4MkNnaHVJSHFGY2E2YWxtdVpWVldJQmlSbVh5Sm14SGU2UTZiM01X?=
 =?utf-8?B?NXRYSlljVHBKTGlpczY2TEdaT0ttYXdVUzBVYUhzaXBKSEgzY2pZZE1OVkU2?=
 =?utf-8?B?b3NsSXNIN3FOSEZHaEd1ZmRZK0JhbWlVbGdUaW85Qk9LeWF6cUJGWEs1aEo2?=
 =?utf-8?B?aHI2NHRncG1pUE1BUjVKSHp4TmNtSE5XNE5Ib3VFZEFQTHVvbzNGM3gxcFY0?=
 =?utf-8?B?OHd6a0VRekU4RUtmd25BellsNTEyUk9QSUl5OTNBR3RQNmlGc1ZmNXc1U00y?=
 =?utf-8?B?eEZQeitLaUNmbGVHay92emVFbGxubzE5aDRjQVNtSWFSQ21ZczNzL2hGODhj?=
 =?utf-8?B?Vm8zNkd3d0s3VC9EMFFZMXl0a0xXV2cxRWdhcDZ1Wm9zVHlZQi9SWjR2bFNp?=
 =?utf-8?B?OGE1dU1VeStLWlhIVHB2NDFYQ2ZGVlhEbG9yOC9QUm9QRktKRHZNVlVvT1Ft?=
 =?utf-8?B?RzhtMEV1VnJsTzludjJWWitleVNvYWNaL0ExaGVDWndRMVNqRmpHSWgxajA1?=
 =?utf-8?B?ckxQd2krU1BSbUdCbDNsYzlkaG1aZ2ptTlppTkdPV1QyS2tkaU1GV0VxRXBw?=
 =?utf-8?B?Vjh1bXFlNVNQZzNTK1FvVWF3a0Fwa1dmT0NFL1pLN3BzelZxNmVubXAzam9x?=
 =?utf-8?B?WGxkT3JlOVdabVU1YklIV25zbytEOEhQVlc5VU0xSUlwdldpaDQwT0NweVRZ?=
 =?utf-8?B?am52OC96Z01KVTVNNlJVd0xBRk40K2NpVU85TzB5MFRLRTRVYVZrOTdmL3FV?=
 =?utf-8?B?S3NwYlo0cm8yVWwrVnc4RmFKbzVBMXp0MWJ5Q3pFMFI5N3FWU3ExRVNtTW9E?=
 =?utf-8?B?WGdvbWZtaUtnSTVlT25WL0hJL2FEaWc4dVhFL1VMNmNKMGdjbVZrWGh1VUE5?=
 =?utf-8?B?anR4YjAySDZES1JabVo5MWNPSENZV3ViVGhCTGRCTHVjWG1aZTdXRHVtVVhK?=
 =?utf-8?B?RTdVQlB1TEtINEhGditGTlF4MzcyMlBqeFBQK21TbDhqWGRJc2tFMVozNnZH?=
 =?utf-8?B?d204NGxJd2pZamVVTHdIbXBIVmVTZnh4K0trcjIyNC9lRmtlSmFEWC9IY2g1?=
 =?utf-8?B?a3ZScWlBcURuVHo5WlNnU1lIb29zcWNPU2ZRRWY2WWRxdGVMREZkRTl2dEJB?=
 =?utf-8?Q?csW3FS2QZKFK9/pzCISQXKzC0lRMGd1dQrluquw?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c8278f10-15a3-4d6e-9f8a-08d9745e1590
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 13:22:48.8362
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Y3xxOn6GdvsOQDRs7Zwf8vefC3HgAu74XhPaMuw0ddS+t0B3tN7w2xnB4v11yAlciehyL1EuWpD+1m4sJKVeEA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4606

On 09.09.2021 08:27, Juergen Gross wrote:
> Today Xenstore is using one unstable interface to retrieve the state
> of a domain (existing, dying).
> 
> In order to decouple Xenstore further from the hypervisor, a stable
> interface should be established. This new interface should cover the
> following functionality:
> 
> - get the needed state of a domain (existing, dying)
> - get a unique identifier per domain telling Xenstore that a domid
>    has been reused (today Xenstore could in theory miss this)
> - provide a way to avoid having to get the state of all domains when
>    a state change of a domain has been signaled (there is only an event
>    "some domain has changed state" today: VIRQ_DOM_EXC)
> 
> I'm suggesting the following hypervisor modifications:
> 
> - Addition of a global 64 bit domain sequence counter. This counter
>    will be incremented each time a new domain is created.
> - Addition of a sequence count to struct domain. It will be set to
>    the value of the new global domain sequence counter when the domain
>    is being created.

This looks like an "extended domain ID", which I think we want to
avoid unless exposure is extremely limited. Since we can't easily
predict what future usage this may have, I would suggest to seed the
global with a 15-bit (or wider) random value. I'd like to further
suggest considering to not have this be maintained by simply
incrementing, to make very clear to consumers that the actual value
has no other meaning. This could be done by multiplying by a random
odd number larger than 1 (determined at build or boot time). All we
want to guarantee is that numbers don't repeat (earlier than a 64-bit
value would wrap when linearly incremented).

> - Addition of a global bitmap with one bit for each possible domid
>    (resulting in a 4 kB page). Each time the state of a domain is changed
>    (domain creation, shutdown, deletion) the related bit is set.
>    When the VIRQ_DOM_EXC event is being registered the bitmap is reset to
>    have the bits set for all existing domains.

This page, as I understand from the subsequent item, is not intended
to be exposed to the domain running xenstored (or any other one)?

> - Addition of a new stable hypercall ("get domain state") returning the
>    following information:
>    + domid of a domain having the bit set in above bitmap
>    + state of that domain (existing, dying)
>    + sequence count of that domain
>    The related bit is reset in the bitmap as a side effect of the call.

What I'd like us to consider up front is whether xenstored is going
to remain only entity interested in this kind of information. The
entire design looks to leverage that there's only a single consumer
in the system.

I understand the caller would iterate over this hypercall. Is there
concern about this iteration never finishing, if e.g. a guest gets
rebooted quickly enough?

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:25:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:25:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184397.333020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgWn-0006Av-Rr; Fri, 10 Sep 2021 13:25:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184397.333020; Fri, 10 Sep 2021 13:25:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgWn-0006Ao-OY; Fri, 10 Sep 2021 13:25:13 +0000
Received: by outflank-mailman (input) for mailman id 184397;
 Fri, 10 Sep 2021 13:25:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOgWm-0006Ai-Su
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:25:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d89e5bfe-c4ed-46ae-be38-8b57830a60a0;
 Fri, 10 Sep 2021 13:25:12 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2055.outbound.protection.outlook.com [104.47.8.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-WzngEDSTOZGyBwy1UiR7Mg-1; Fri, 10 Sep 2021 15:25:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 13:25:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:25:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0207.eurprd02.prod.outlook.com (2603:10a6:20b:28f::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16 via Frontend
 Transport; Fri, 10 Sep 2021 13:25:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d89e5bfe-c4ed-46ae-be38-8b57830a60a0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631280311;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=n6ORjZOM7dm567gcGxcVx+DSkWtwBTE0bA/15enp1lY=;
	b=bWT4qarH7W82D/t3DSzunnVzFmuDoOMT16D6meA6/IKkDwaeY4hY3P3LcsLop8onnrEuqo
	Fv/Hvt8jkuC1ejULmaucjFdBWaaeV4SRBwBjgnadEXZbB9lIH1E2lp3lrNIMPnJRCWnW6L
	XxdNrfosjQXWfM9T+PIsE/WWEaLZc/w=
X-MC-Unique: WzngEDSTOZGyBwy1UiR7Mg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=INURRoBI1y1KTqU8NrYRycMFyUCGBOdJ8mzRxEw5kYzqYJfimjfH+zXeRqCtqVFCyrKi53VYOOIzcSC4bCOJht9YFO3z0pAhuWctK4q86wBJoaXCgeEeVEIBYFrqhn/5JWS8w/izFk2Co33I5pdjlmbuU05aoaF7qJ0/URzBSy/rDaobwK+eEzcWQOEDk2OV4yL+Ud505WLIUoiaEZu24pGuHqy02koR2+jY0k9/l7uVY/WN69MQ9iXBtIinGnbxBWqFTgKAUY9yOxsGvx33Cfu6k2rrsBX5SXwjWvtxDAa6yzC6/Mp/hkVMY70UnIVlwuSXPaBrnsRpRxpkAjmFaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=n6ORjZOM7dm567gcGxcVx+DSkWtwBTE0bA/15enp1lY=;
 b=NNWNJRbwjbWa84e5eM3/jFbFMO2nUxJHx7uNHEtISUiuI+Hl5z+cemDf42wEHGwDAH6rBEgYUQoFD5cy2kE64kv62GEUDTcYuH5wTuDdS17O2zGEWVmzNvQ9RktTnzEjROPJjomJBpLi7wiGA1sORWxyMV0W713vCfgEEosxft0N1V/mLPve2KOp3ZYcz+unChmH74nJ6KHDX6AmywCtXJZJUJMQDnE+jzTnLq92I2zFTYnuW/EC+E8dyAyCsDquub1scoG6oO87XXB14hEXc2eg9yj0X/GemJMqAWEYhtHHXkFCSS9m2Y2u5/2dHqLZmrGMK1OxK6wkiI0uUDH2qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: Xen 4.13.4 released
To: xen-announce@lists.xenproject.org
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <d076b231-1bc5-e479-72c8-277f0bcfd84d@suse.com>
Date: Fri, 10 Sep 2021 15:25:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0207.eurprd02.prod.outlook.com
 (2603:10a6:20b:28f::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6c6a538e-6367-4d39-f2c2-08d9745e690e
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2606D88D95A193039E65457FB3D69@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9dQJO1h3xpeD1psYimH0DK0hGAskBDui8TVF4rbrCGFIazCS5BMyTU6GBdt7GxxJrp1nEtMKizAO1EXkjWTUMFSXekViHUriENFcds2+y9CVd2HOQiXF6nYb0sAbGAYBhd2aVV9IYj7Stkjc5TJHPN9Oh5TKV5q2ZX/ob2SRq5NhIvvqmtMpEds3tzh8pC+M5VWWGe8sD/uy2xH4IJgJLSbtn3/LDLeta0PTRKrFKue/795bkvOn4Mj0USVEU4CJRVXA6jG9h3wraFnZVrsCLC5jG83yBBAsJmaOeRmPo1S7XtfbTgP67B3UhDmLprpV2Jl5OIjdJM0nL39X09VSXhzvGQbnO2h7OSP2T0YQNr/0p+QoduiT+H9g8dnmB5rwEkM5nClVOPs7lmkqZ1ED2kPEHMms7zbdY0Bc/Hy39OUlBBjGUPKTy5Szg1thQD8c/2DnXXs/uj8MpvimyiM9PN04c3sAOxXHfSwoLdMGJ4v9eDVaegdjW3dtUV+z1UqO5GnlxwCDskP1RevMwu/FTj/1xHz2UcBjsG8Ehcgt8z3My5LFvK6nflCs64MLMgAaVl7xOzl7oDyQCOg6xFhosatj7HiLTdduSArP/4Z6s276lFZr62k/WzQsBcBvM38y2PcjOyUDIgmtlGPaS2EjQ6jSfeS8Rzry2ijPIlIMVUjvgWitbLiWSSjzQvYO8BGVFtqvKKwDhGyBX/ibZ+7yKQblKzzaerx/7EGmGuRctuSTIf6/3hSWRzyD0N/Qz9JvqlHmpjxTlhMWu7/xQc4AGX2qdaYn/qm9LqyF9frF8RApG8DbSLcJj1n9oml2bu4J
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(136003)(346002)(366004)(39860400002)(6486002)(966005)(6916009)(478600001)(83380400001)(4326008)(4744005)(450100002)(31686004)(5660300002)(66476007)(66556008)(66946007)(2616005)(316002)(36756003)(956004)(31696002)(7116003)(38100700002)(2906002)(86362001)(8936002)(16576012)(186003)(26005)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RU9YZFowLzZrempOdzh3dzBvZk1nTGNEY05XUGFiWW8vTmJxbGhLVE04aUNR?=
 =?utf-8?B?bW1ONzZxK1lCeEJzRzB6SytFeDUzbEtQMStkREVvcWJiTjRVNWx5Qmd2a3lt?=
 =?utf-8?B?YkFUQndIVk9RY3o1L25ack1YaUZxajA4OVRXeGhpNUw4Z21rbUFpdk5Rbyt1?=
 =?utf-8?B?NTB3b3ZidGFPaXU2VG5VK3N6dUwrVFEwa1laUlBzdXJ6M2J1dEZ0Vk94MEd2?=
 =?utf-8?B?bmxoQko1M0xWUU5BK1MrdVFGejhzUHEyN1VkYkRkV3hsWTRFNGZFS0FBTDNS?=
 =?utf-8?B?bmZIOHFrd2tKNmsza29xY1pySGlUZkUramxLWU5iUktSUXpMNWRkbXVUVC9H?=
 =?utf-8?B?a2kvMEszUStYa29BVWgxa3lVbzFTUS9FcnMyekNwVno3SzcrYVBjemJQZ0gx?=
 =?utf-8?B?Rm1aL1BoTElFcENqamxIZzJZeHVMWis1TWYrTnZiRXE2Qm1UL0h4ZGJrL2FB?=
 =?utf-8?B?WXNVMXc3MDk2azBERUc4SEVnVHZZV2kreWtTZHB0eWoxanhWUWk4NXhMMDZy?=
 =?utf-8?B?OC81WEdsYUpoZkVUaVpTWTc0cXMxV0hNVkk0R05mSC9CVVdHdTBrKzlGZndD?=
 =?utf-8?B?T0pwUDIxTVlZUkxwS0FRQ0pFaWlJMWdDc1dXMk1iTTZFUjI1TEhrSktKb3N3?=
 =?utf-8?B?TlYvZGhhcUFCbkQ4ZHJLZitXamk5M2FKZEh1N2krQjBESDV1cEhoYnh5aUly?=
 =?utf-8?B?NHh0dVVjcTZibE1zbzdybk81dlBFcHp3SkZ5Uzc1cmpvNExybTVNRmxkNm96?=
 =?utf-8?B?YnNuNU1OMGVwaTBUL0IzV3djKzM0MUdEQWRKZ1EyV2p3Mk1rNEdFdW8vbExQ?=
 =?utf-8?B?MWFxN000TThJTkhtSEYxQlhrQU94dllyRjFVR2pQZ214aCs4S3pKeGlibWNP?=
 =?utf-8?B?RVU0UVhrbDNIeCtFSFRBbG5aNmgzSEVzVGlPdFE3NDRjamlnUDVRY2xXS01t?=
 =?utf-8?B?cmRXS3g3TTBBMGN3R0E2M052Nk1aQnFuZ0xNb1pOb2YxQ0tKcjFFeGZTN01p?=
 =?utf-8?B?NzBkT1EzSnA4YURkQVhlTHI1ajFXa0dYRHllZ3JnVmpjY2FLYkV5VTA4VEdN?=
 =?utf-8?B?cVY0OVVvVngvOXlzZXY2d1drNmJpZDUzbEh4SjBsUDZPY2RrSmVnL0paRXU5?=
 =?utf-8?B?T0Y4MnZKa2VMV2dhL2JCTFhLS05WYzdTY3hxTEp0bVBHZGhmRWUxMWN1d21E?=
 =?utf-8?B?MVZSN3JVRmhlQ2dRR01BSVdnWmpFNTgvN3AwbHk1TkZ6UHoxV2dwTXN1emdr?=
 =?utf-8?B?ZTUyU2dVZG1iUTg3N2hlSU13RXg4cVlwZSt4L0t1Um5TM0krWDVSMGJFUjZK?=
 =?utf-8?B?aG5lK3E4ODNLSlB3blptRlhVcGhWdkJ0Vk9FVnJkVGhlN1FtdS9VTS85cFh5?=
 =?utf-8?B?SjVsNDZMcWFUYW15djhLSWRZaDVqaWRhRzhQRkhQWHg5elRSL2ZhM2NaTEJ4?=
 =?utf-8?B?K2RaM3NpNTBZc3RKeTk3RnBCTEZ5OFVPd2tJNFZiTDg0cE84aUxOa1RUU2FH?=
 =?utf-8?B?R0ZlOHJjY3A4OEl3SG1XZ3p5YkJ2K1FTK3hLSDNXZG5kaW5VMGRWNXg3Vy9I?=
 =?utf-8?B?YlJuU0Rwd3ZGRTRrUVBmTDJLUm10VndqUnkvL2V4Uzl4M3VGWlRYR2VJMXFl?=
 =?utf-8?B?azl5THNDRjNLb0dLbHo1TzZ4SCtFY1ZvVzhBWHZjeTAxRS84Q1NaZ2o4UXpT?=
 =?utf-8?B?dktOUWNpQVhJcTVRLzFsdjlKQWRKUlhLWkp0NVEwbnQ2Nmx0YklOWlNsd1dl?=
 =?utf-8?Q?nqjlroirM3u/Bv0onJXtjcDWEGan8gpckqx3k8f?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6c6a538e-6367-4d39-f2c2-08d9745e690e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 13:25:08.8960
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xT7QRCW5SspvjjOfShORGL1TGRckiJbzp+Q2IHymvdW5uqxPhcenGoBKD4wdiuX+aBjaHPsC2A8S+JZaTeVTtQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

All,

I am pleased to announce the release of Xen 4.13.4. This is available
immediately from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.13
(tag RELEASE-4.13.4) or from the XenProject download page
https://xenproject.org/downloads/xen-project-archives/xen-project-4-13-series/xen-project-4-13-4/
(where a list of changes can also be found).

We recommend all users of the 4.13 stable series to update to this
last point release made by the Xen Project team from this branch.

Regards, Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:26:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:26:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184406.333031 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgXz-0006mH-9Y; Fri, 10 Sep 2021 13:26:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184406.333031; Fri, 10 Sep 2021 13:26:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgXz-0006mA-56; Fri, 10 Sep 2021 13:26:27 +0000
Received: by outflank-mailman (input) for mailman id 184406;
 Fri, 10 Sep 2021 13:26:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FfYJ=OA=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mOgXx-0006lz-EQ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:26:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 88cd4798-fc8f-409d-b760-8ecc172aae59;
 Fri, 10 Sep 2021 13:26:24 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2051.outbound.protection.outlook.com [104.47.9.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-9hVV-9vhPlyoQ1d04o-D1g-1; Fri, 10 Sep 2021 15:26:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5743.eurprd04.prod.outlook.com (2603:10a6:803:e0::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 13:26:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:26:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0013.eurprd09.prod.outlook.com (2603:10a6:102:b7::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 13:26:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88cd4798-fc8f-409d-b760-8ecc172aae59
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631280383;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=4iegrVGJOJkI5DVYkifWSk3s++UjHOET9in6aAVVUBk=;
	b=b5qFTc/A4JwDWaULLdIjZixiVZtUsfwR7dMqQAXdK+V0riqBP334GiNkNLEyJoG0T02K9N
	Knm0rOhNHw8DPJ85MzeHyz5tIBvPmnAYIpG5OAFd4tyccQG/bBy5L/P98vvY0AtHcHeeQJ
	6a0QPMr0ZpY9r9zI/YBOsky9AdO5rxc=
X-MC-Unique: 9hVV-9vhPlyoQ1d04o-D1g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k3d9fgVQJupoY6XYVq9iFLWW/cey2KkZ2S4QcoOawFJedDhTFRF1CA4eYU7HwiD7Fg+uh2FKUPLg2iWzfhNWMfPoWn0lkU3zhSi/rnuHWQTwV9JIYFesjMatFv5gVRoznEIn0OmF/Yd0mC7Te5tgErpsxZQ2pLHzx52iOUqqVej4e1aXHPUIRmVl6+andB3pg6RJkNjpbklceBGTDdkdRrOydOiPAoWeNLad/FZEY6XMsC5+43rLCE/z5As/O652B3iKaOK2q92ynIeMMP9SgsZ8HSyTO3uZknxWkF46NbvJCLZolPOQ5JAHrWYB3EMacBjjsNUyegxzWt04P98wLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4iegrVGJOJkI5DVYkifWSk3s++UjHOET9in6aAVVUBk=;
 b=IeTzxQ6MonMVKSsHC2Idds2NwslQIn8pzte1qPJGyNORAlm8YaNNtJP4iXnNTZ+5qpAhY+/acREGTtQJwqWtWxMWfR46AqRYPk1khk8brEH2CdAOxmGwGnuq1uM+7nW/wWGgXXDbrVYBtac8UXrmLzrwA2Xomkv8Qxch5wCPDbtOQl+0QhFtlSWDzxn5d8+/fj0DlIPMVBr0ATWmFPn3uh8Nu/ENckJE0QjcVWHZHGUTmcfECXoGemQ6n5728ZpYOMpYAMADiwQZzQHC8yi77RD0dqOp1uXGygpovO+megwhXPZs4FzLdQx+Z+rD0g00fkEmbxvcaErqVJGMR5DsRg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: Xen 4.15.1 released
To: xen-announce@lists.xenproject.org
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <35ca059d-77c7-49a1-81e2-e87687732e23@suse.com>
Date: Fri, 10 Sep 2021 15:26:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0013.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ea47de04-83e2-4ad5-9a73-08d9745e9404
X-MS-TrafficTypeDiagnostic: VI1PR04MB5743:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB574383FE0F4525D225B6A1D4B3D69@VI1PR04MB5743.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4Rhxcxb2yCIFE0JviAyTRGSxGaaFGMpwi10/SIDvnF83ZK68X2naI88cRUcVpCOsABEEQwz6jgEW7BY0KVmfkNqyB1aMZHV77OhIBfQN6C6pDExwNyS4DkkrKNjexNVf4GBDfPzNUc3DTmIEJIq/cHeQhNcmBuYPjBHnGIZwPlRlfn59T8cA2QtmZ5sNbc+FCZILIcaGe8UlLcSt/qOTxw8R62oA25+5VidAjIXymnmvMuY7/c47CCPp0ApsUnbkNHpj5k0zdJ0a08ViV+5+BSV6LAjkY+ZIaknkbkcT7XvkeDRvOtslSTSiPeALPiExdgIIUSRQhBHsWhxnyi+4Z3i6eJda0qVBZb53ckOia2n9g5JkyDRZV+5I85jPQMacOIpUArRk8Gn172NNQzuSXYCVEf77D6PcygV4uYk/WwZl4q6IRupxSIAjt3pJfPXR7FHXQSz3+YLz+yvJ0tngtFSEGxNAlmD3FAJWa447gU08vdLiESEH1KcHnaxp+m2ho5n4VqTUstLdSNTerxDqXlXmt4vgYA2yYSLWf5i2OEIsa1KIl5C9VffbgXM7zNFBW4Fsm9fNvu5vIEYeQmc97ZK+U7GvEghzOdDsjIi8hJcqK/P40tpLJ2Ust2+cmJoLzm4jxVyri6fQvUv0kp8xo4C5GTFeMv/T4Cyla5egBEaU+tSB94O/n8dgMMKukqr/NTFgh3kRGjBo26Yn3+VKaVgmLDn6/BIzJiNqV5vMsdYrqq4CObPA+vGjZmRw5NmaGjvuftl1Y15QKzX0eI1iJSH+CICmYsVKSRbPgvSRssimSHH9k9Sg5e2UAIKPW+LW
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(396003)(39860400002)(366004)(346002)(4744005)(186003)(31696002)(6916009)(4326008)(956004)(2616005)(450100002)(38100700002)(5660300002)(966005)(83380400001)(478600001)(66946007)(6666004)(31686004)(2906002)(16576012)(66476007)(316002)(8936002)(6486002)(86362001)(26005)(8676002)(7116003)(66556008)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZE5LU09McFQrNlZqcUJqTVNzSCtjOTFxYklDVUJtWXZZRWkzNnYrdU13YlJV?=
 =?utf-8?B?Tk50bnBXdVF1RTgzU2dqZEZaN1RIQWlIaVdqVEkrcktBT3RGYjQrdmVRTmJp?=
 =?utf-8?B?VjNVcVliaDFwYWw5VGlZeTRwSXlJQ0Z6WThIT2M1NVhOTjRDeDMyd1R0K05G?=
 =?utf-8?B?VW92UUJmSEVwN05jWGs3SjdUQmZFbG4ralpqY2tyZmJjK085T2ovRjk5dVNq?=
 =?utf-8?B?SU1PY01DQUxrazBYVW5YZlgrcThJRzdCRW5DN0xsMW5tR2dadFJpTG12emJE?=
 =?utf-8?B?Q2ttRExRSFJWdkZHUTRHVGd0TCtvV3ZOY0I5L21rKzEvM0VudFY1S21wbGpD?=
 =?utf-8?B?SXdpY2dnUGlYWE81R1MwbkdMQnZaNFd3c0MxSEt1YlVTYWVtb0lEN1Fwenhs?=
 =?utf-8?B?dnJZMzJRQXB6VDVQQzVpcExFeHhzUW5Scm9pY2M2ell5NjNxSlhhV2VUU0h6?=
 =?utf-8?B?ZVkvajhXdVlhSytpajlmMmRTUnAvYlR2ZGhTaWovT3dsMzBTNFJEZXpXZjRa?=
 =?utf-8?B?cG83MWtlNDI4V3AyUW9FakxBcmN0aCtHVmFXQ2ZxdHhSZ1JkUGRzN3NIc1Iy?=
 =?utf-8?B?QjRQKzFqZVkyU0dtUHh1YXFPeFUrSVJaT0VnMW83OGREQy9QaUJYT1lpOWM2?=
 =?utf-8?B?VTg2UkhKY2NoQm9Ld3VHaVFlNEVRL0V2YnlPWHBWM0hhRGxKTUs0NjlsOW9z?=
 =?utf-8?B?VEdFWjc4MlhDV1lNTEpMd0NIaUZrTDRlc0hCdnRXV000Z1lMWmpKOE93NU9w?=
 =?utf-8?B?WjZTaVdjYWJEbzkyTnBHaXFCRTZGNWdhbEtzSC9ra01zTENFbXZOYXE5WDBx?=
 =?utf-8?B?WjJGbWdaYVRVSThYT2FoR3JhU3pIZXR1bHU5cTJ6V0d0V2RsQk5mcHJEZzRJ?=
 =?utf-8?B?T3JMcXFsUUF3QjRuWDV3NUFtSnJYMW0rdFFsSWJ5cXJoMWRFSm5hWkdVZmQr?=
 =?utf-8?B?WnBkZXZvYjRkNWhiKzB2VGxObDJtSjdabUp1UVV2ZDBzVkQ5ODRGc3dMQ2U0?=
 =?utf-8?B?MTVqUnpETENUTUkycG0xdWJsOUNaNDhhTGZ2Ukk4ZWNJMWdOREcwYW5oMUFr?=
 =?utf-8?B?UnlkYS96K3hQTUVtMWthT1N6QTJxMDB3ZUZrU1c5Y0p5bnhuM1hROGZNNnBp?=
 =?utf-8?B?cWdwV21WdThLRy9DTmxEN1JrNnpQYkVUbURKRVZ3WUFrdVEyN1pUTmVSUUVX?=
 =?utf-8?B?akNOZitPaUgyTndHYlZzQXZTVXlDWTdpVmx6eGoyUGR3a3ZTRnE5ZjlnNlpX?=
 =?utf-8?B?RjI2bXVCenVheVVDM3A2SHVPSVZZbW9mbnUzRDlzbWFCbGczMDAyOHJ6U2RY?=
 =?utf-8?B?RTFvK0wzRUJmRHVQOTZRVC8vRlAyeStpTDZVS0FXblg0OHpYeWlIYldFZk5O?=
 =?utf-8?B?SkozMTl1SkFWRVpFM3A2TmJZYndYTVdhNzZZWE9OYlNzL0ZEVm9LWXNFbysz?=
 =?utf-8?B?azNIeGlka3NPUnR6M0xOVW5WaWo2OHVSSUI2N3c4THlxdWcvOWNhdGYvOWNm?=
 =?utf-8?B?R2w0ZHpFMDJSemEreEhtSHNMTmxpblpOcjEzWWw3YnlaTUttMlBIVTVNbVJX?=
 =?utf-8?B?eHlDcmFRNFBUeHZiclRKcXJSWlMwTFBtc2VCNmtUZ052TlI2Vm1SdzdpMklk?=
 =?utf-8?B?ZWJTYjBEdVAraUErSTgzZkdrOG0wSE5CSlZyYjljakJiTGJWNkZYazFCakcv?=
 =?utf-8?B?cU5FV1BBL3VJNFBqcnZhK21tRUpSSTN1alJSWjJ1bFUxYVg5eElDdUQwYW1l?=
 =?utf-8?Q?HIW0USM6tOLeQr7v8gYHnCSGWxdHRyKA6I38QpN?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ea47de04-83e2-4ad5-9a73-08d9745e9404
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 13:26:20.9552
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Af0OWs+oNM8fWh4QP6AGbPhn/mZ6gJJSeNVXIgMbPyLWOMfDHs2dkbTQWr+pgh9nYdU+3LrrgLB/qsK5BE1IUA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5743

All,

we're pleased to announce the release of Xen 4.15.1. This is available
immediately from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.15
(tag RELEASE-4.15.1) or from the XenProject download page
https://xenproject.org/downloads/xen-project-archives/xen-project-4-15-series/xen-project-4-15-1/
(where a list of changes can also be found).

We recommend all users of the 4.15 stable series to update to this
first point release.

Regards, Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:27:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:27:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184415.333042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgYx-0007Sn-OH; Fri, 10 Sep 2021 13:27:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184415.333042; Fri, 10 Sep 2021 13:27:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgYx-0007Sg-LJ; Fri, 10 Sep 2021 13:27:27 +0000
Received: by outflank-mailman (input) for mailman id 184415;
 Fri, 10 Sep 2021 13:27:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOgYv-0007SM-Nj
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:27:25 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dca78294-4a82-4387-8f62-33d6b88a9b72;
 Fri, 10 Sep 2021 13:27:24 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ACsh3u006954; 
 Fri, 10 Sep 2021 13:27:22 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b07s684fm-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 13:27:21 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5555.eurprd03.prod.outlook.com (2603:10a6:208:16d::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 13:27:19 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:27:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dca78294-4a82-4387-8f62-33d6b88a9b72
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oN4TAwZw4Q7xktTu6jTsA+UKf+/3/x7ox1nX/0qZ3I0eVGjTBqaQ9GpqClhqkrGXGIFrlTnkGyNs6agyZ9MmqL6ZljIx0PONY0qmMMNVcZXUco69zlffNen7SoLfblDFANwjGkl5OGwLx/+OmHPE+BiyzDxitMvrjQhAQHYN5KB9ZlTHF0WGLrYjgkJY+Qjz6tGU3QC0ebaDqmeg4K7mQ0FUtaBFAB5ZF78QEIdpi25VYz/giBKpgn4s8sOeiT0AkdLN/+4Mk/haZeWw/rTe/nWRcTKaq+cBQU4X01bmS/IHLm00+R0e/pFjPajjkXB2KmKL2fhaNY4gbqHl3h4KuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=SqzZ+5bXf5LG+jvRqpxwrpZAT1fbiOEnvBA0HJlkHOg=;
 b=QvKS8YW13/O9y7z0Zwa95ObQunjvyZictDxv3sZ/uct4//iEwcM3NIOD/b84/PcpUHSo+IHzzbINVaiMD9H9GaqJ7G2PdE0azKlRzJ49o/S3NfgkcdHcJjhBzNZoCWvmOIjoG/vmlPpVL3M6uF8OE4HBD9whWXbkaHLvlUOBq62gFYH9nNvP+x81Uu3jpNhFD4QbaTLBRYyzangg04T9JAR4Y8gdLY6nf0Wg3+GeoACkq6PRpOYAXvbpd2/Euy2vDJgtaVRpjuRdRV7AydMhmgIu1/yHNZU7xbNjLEoUXEiHDLs4TA+4H2tTWzkaEM2Hc7tbgB+rN27J6jnHCpza1g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SqzZ+5bXf5LG+jvRqpxwrpZAT1fbiOEnvBA0HJlkHOg=;
 b=Du59NqAsbmq4amQMAtNdgELaL1/3O9HF9ttT+I5kEJlc8o00vPrfBQObWLFWDtneeSkFSNDrBIkC5iA/DF+AjTKYphBdJMglDIAT79I3iy166IaBmceZ0kjcRTrFI86FXj91fE6I3mUwPEVfDRxJesAjFuNe6kIuU5ZLbvvLkuwnqLqMCl+aeqgd/GvksaebKB/DbzZGx3+23lwuYs1fW8NghnYA1vbb82j7cBoaxkQGNZHBIW/ocM16sKwxa7CcmaGFL4JtZuHXGP7guIZEg1bbqNiND2Zdl1jnDm5dazUJyr5NVGaZY0NkAtWKDr+/ouD1da/RJvQPGL6ZUh+C0Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: 
 AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4ACAABanAIAAAwIAgAABgYCAAAHPAA==
Date: Fri, 10 Sep 2021 13:27:18 +0000
Message-ID: <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
 <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
In-Reply-To: <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 17ffaa83-e15d-4568-987c-08d9745eb6ae
x-ms-traffictypediagnostic: AM0PR03MB5555:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5555694AEF55942738DF9CC7E7D69@AM0PR03MB5555.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 0FS06TcNkTeJT7J0zpJWOcU49LNpcKzxthopuCDEb3G3Iy4eKyE7VwI/LP9br/4K4uYx9Bzs2hNADiiDrxeC5SNwgoXlCxJ6+Nha35lbe4ORNkrnBR8pvghdAXmNedduOHbyZwLir6nJdG1RrJ7jAhmGjCYfj3gluEjxUdpi3lJ/48FLbjbDVTKG9qc4qKM4panfH/sLS6AckeaPt4ANpEuxNpE6Hi7h0/+ZCvApHVWoufh6HDK280HVVHEELJAU1kr7miMnBZm7TViCs3jwwOOCeTzkUons5FW3cHUM5wclOPvblsQPqHHuDAA6U8d0iYONn5jnywKtrcXIrVM7NyG6YISpe+N4YvlzBTehx2L/v3LowXq3uZQySwksajuvvImYF82nfQ01s8INPftPCNAQos+MbmoLuijofv2Y/50E2iPQJ9pNTSXn7y7esJe6Q0F3U+7wUm5ZE1fWlYCdQpq2eSuSNTFMNHiP7jwO9dc/4DZIFvsetXyjWEw+yBtm0HXNs3bzN6FhLLv77sMxMatfme6KANmQdFTP6c9Q/84d+qV9FMt6rye4oXvKrYbbhkNs0D30RpWYxFZXSRnYb8wBiNWj+1vNFsQoBHh29aP1JOLrfheZ4Dc0i9JNiKIJ62q2ohiIxYLLW+NqPtNKQ1dyyuihqaansR29CyWtiWZ3Tmn6ax9TdltC1sSocvNWsUzOglqv6+TeyPQvWdMcN41f9d0kg3NtK8vvJO6elOI=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(396003)(376002)(39860400002)(346002)(366004)(31696002)(38100700002)(478600001)(31686004)(71200400001)(76116006)(66446008)(2906002)(66946007)(66476007)(122000001)(54906003)(86362001)(4326008)(8936002)(5660300002)(55236004)(64756008)(66556008)(2616005)(8676002)(6486002)(53546011)(83380400001)(6506007)(91956017)(36756003)(316002)(26005)(38070700005)(6512007)(110136005)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?V2g1UlZCbXp6aXduMG93UFE2clhxMnRYNFJ6TGdoNW0vSEhBVHVxZk0rbTZK?=
 =?utf-8?B?eE9WNUwwWU15cmd0dUNFRWQ1WWNwUFl3RXJNbUViY1dSWW5DQXFKQm1saWpU?=
 =?utf-8?B?Y2JVMERHd0cycGFCOWVBRXhvZHlpbUJJVWo1WHZlOHRXM3Z4UlZNeUtUb0JQ?=
 =?utf-8?B?a0M4V2NzZHB2dUY2YnFwZkg4RFMySDkzeC83RDV3K1I5eHpxanlIQ28rTXdY?=
 =?utf-8?B?cStHTk9UaVh2ZHVrTk53Skc0R3ZLaWNyWVZ1ZExXVFdTSTJxK0pHcTdvdm9M?=
 =?utf-8?B?NHdnNWV5QW4vVUhReEtVT3poeWoxRDVvS2U4ajZTNWN6UUdEaWJIa3lRWXgv?=
 =?utf-8?B?YklXa281RkxiOWp6WUZuemhucy9yL1lFN0FUVzZVU0VyZ1RjUWlZMTdaWE5N?=
 =?utf-8?B?NE9rclpHMkdTVkxsUkNEMnllRVBEakJFaDFIUzYyamNJNlVEUlpZWnVaSTRv?=
 =?utf-8?B?MHpKSFpyUFRUa3MyYWlCRmw5L016c1o1a0E0UWl5VFZrVDJGR1ZvMUJ2SHpC?=
 =?utf-8?B?OUNyT2lpaVNZOTROaEd0UFU4V3lrb2hpUEJmSm90WkdxNGc5TVNzVDljcS81?=
 =?utf-8?B?TGRTeHd1UklhQ1hpUEJKWnJzWkRZS3ZOSDhUNGlsMTlzQi9VUFE4UGFHY0dl?=
 =?utf-8?B?SUlvWkdTNmF2bHhpSitlZGx1enBDRUg1cUpSSnd3RC9FZ2o4TjZ0d2ZVQUVI?=
 =?utf-8?B?TzdWZGZRME1Od2V4MDdEQU1ZNVk5MkxBOUhvSVE3WDBKN25tTm0zYkdRRnUx?=
 =?utf-8?B?czdqVlJsajJKR2JYMHN2MENNT1VxekJseGw4cnFHYUxreUY4c1JlRktLUDVk?=
 =?utf-8?B?QVhPYytNSkxjU2JHTmlvU0dxbG40K3lRSXUxRHF6UzloVk9GZVZOSnl4b3Zq?=
 =?utf-8?B?R2NpVWZKODBWa09Gb1l5aFIrU0xhak1hSnRRS05OaFdvb0VSTTNjUTRMbTg3?=
 =?utf-8?B?VTQxVVNJKzRCaHRBOUt6NmJSRDRESlVNKzVWYVNoZmRRN1hFQk55ZTdIcEcy?=
 =?utf-8?B?RGJvWDR4bitubUxEVHdPNE4yNERNMHpQdmt3UVlUWjJYSk1yeDh1WW8xWFJY?=
 =?utf-8?B?RUNGWmRPR3JFSDRVU2NrblI1K0NDY0lmRHhadlpVeVdOSFRPeDloamtvczdr?=
 =?utf-8?B?cVk1dDBSTTBRTVI4cFJ5dWhjODJIUWNPTWJlQU13Slk3VzU3Uit1WEZBOWg4?=
 =?utf-8?B?d3h2N0RrWTVIdWpQVTZLTFFUQ1B2SVJCREFzS2cxUU1UdHo1b1ozOGp3TlAz?=
 =?utf-8?B?c2pQQ1Jjcno0T1dCYVdrdGJRbkVYUnlkUEJjOGM0Y2d3ZklLWTdtWlplNmt6?=
 =?utf-8?B?OTNiMUVNMHArSmg4eGdBRHV3UUxEYmtEZVVkNXBTQTMvRUJtU2ozMnlDeVVC?=
 =?utf-8?B?ak83d3kvVk14RnVJWjV0WGhhc1g0cGtSRndZZ0M4cDJld3ZjZGFNVzFWQjJ4?=
 =?utf-8?B?WTd6Q3h5eWtqeGluTnR5eXpHQ0oyWnlyTmRUNDRVK0tJelNoYXJvWmJBSXVq?=
 =?utf-8?B?a3owNzJnNEtQVk16bkgzd2xwOXRya0NPSlQrSFhqRnpIRjFEWkZyVGcvdzZB?=
 =?utf-8?B?MktuSndwMWxVTEJ2WVVDRHJiTTBNNTNPSFZvQmxRdUMwYXlYdGw2aHpTMGo0?=
 =?utf-8?B?RWN6VFIycGNLbGFLZ2Z4RkYwSUpNVFFVdHhIL29NdVlGaXFJbGpzSnN2cWoy?=
 =?utf-8?B?S1luOHlqM2c5T2VRaHVSZGM0OUp3K3JqMUxkZjdEUFZXeWtJS1VxOW5ZMml0?=
 =?utf-8?B?WXF3S1pFK2NLTkpENXFsM3g0djRuUlFjUWtjUFd6S2hudGt5NlBhQmdTL01I?=
 =?utf-8?B?L0pGVkIyUWNieE9NS0xyZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <9919D1F7A015CB42AFC9E9556F476784@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 17ffaa83-e15d-4568-987c-08d9745eb6ae
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 13:27:18.9701
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Y0ta+j2O1ToQXE/aan9n16D6cye3bU9bR1+4LJ8kSQVi0XnfCik833XQpOQTdqvyq5uL6EEUqd0N9uM+15I+2cHyUzKr+BjL1wBzcqA1CGUpzHTwHTl2qyP8LKmUYN2h
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5555
X-Proofpoint-GUID: TfTll4-HUvYDpVXpLWza4i-rcYlbsLOK
X-Proofpoint-ORIG-GUID: TfTll4-HUvYDpVXpLWza4i-rcYlbsLOK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 impostorscore=0 bulkscore=0 malwarescore=0 clxscore=1015
 priorityscore=1501 mlxscore=0 phishscore=0 mlxlogscore=999 suspectscore=0
 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100079

SGksDQoNCk9uIDEwLjA5LjIxIDE2OjIwLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+DQo+DQo+IE9u
IDEwLzA5LzIwMjEgMTQ6MTUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gSGks
IEp1bGllbiENCj4NCj4gSGksDQo+DQo+PiBPbiAxMC4wOS4yMSAxNjowNCwgSnVsaWVuIEdyYWxs
IHdyb3RlOg0KPj4+DQo+Pj4NCj4+PiBPbiAxMC8wOS8yMDIxIDEyOjQzLCBPbGVrc2FuZHIgQW5k
cnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4gSGksIEp1bGllbiENCj4+Pg0KPj4+IEhpIE9sZWtzYW5k
ciwNCj4+Pg0KPj4+PiBPbiAwOS4wOS4yMSAyMDo0MywgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPj4+
Pj4gSGkgT2xla3NhbmRyLA0KPj4+Pj4NCj4+Pj4+IE9uIDAzLzA5LzIwMjEgMDk6MzMsIE9sZWtz
YW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+Pj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNo
Y2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+Pg0KPj4+Pj4+
IEluIG9yZGVyIHZQQ0kgdG8gd29yayBpdCBuZWVkcyBhbGwgYWNjZXNzIHRvIFBDSSBjb25maWd1
cmF0aW9uIHNwYWNlDQo+Pj4+Pj4gKEVDQU0pIHRvIGJlIHN5bmNocm9uaXplZCBhbW9uZyBhbGwg
ZW50aXRpZXMsIGUuZy4gaGFyZHdhcmUgZG9tYWluIGFuZA0KPj4+Pj4+IGd1ZXN0cy4NCj4+Pj4+
DQo+Pj4+PiBJIGFtIG5vdCBlbnRpcmVseSBzdXJlIHdoYXQgeW91IG1lYW4gYnkgInN5bmNocm9u
aXplZCIgaGVyZS4gQXJlIHlvdSByZWZlciB0byB0aGUgY29udGVudCBvZiB0aGUgY29uZmlndXJh
dGlvbiBzcGFjZT8NCj4+Pj4NCj4+Pj4gV2UgbWFpbnRhaW4gaHdkb20ncyBhbmQgZ3Vlc3QncyB2
aWV3IG9mIHRoZSByZWdpc3RlcnMgd2UgYXJlIGludGVyZXN0ZWQgaW4NCj4+Pj4NCj4+Pj4gU28s
IHRvIGhhdmUgaHdkb20ncyB2aWV3IHdlIGFsc28gbmVlZCB0byB0cmFwIGl0cyBhY2Nlc3MgdG8g
dGhlIGNvbmZpZ3VyYXRpb24gc3BhY2UuDQo+Pj4+DQo+Pj4+IFByb2JhYmx5ICJzeW5jaHJvbml6
ZWQiIGlzIG5vdCB0aGUgcmlnaHQgd29yZGluZyBoZXJlLg0KPj4+IEkgd291bGQgc2ltcGx5IHNh
eSB0aGF0IHdlIHdhbnQgdG8gZXhwb3NlIGFuIGVtdWxhdGVkIGhvc3RicmlkZ2UgdG8gdGhlIGRv
bTAgc28gd2UgbmVlZCB0byB1bm1hcCB0aGUgY29uZmlndXJhdGlvbiBzcGFjZS4NCj4+IFNvdW5k
cyBnb29kDQo+Pj4NCj4+Pj4NCj4+Pj4+DQo+Pj4+Pj4gRm9yIHRoYXQgaW1wbGVtZW50IFBDSSBo
b3N0IGJyaWRnZSBzcGVjaWZpYyBjYWxsYmFja3MgdG8NCj4+Pj4+PiBwcm9wZXJseSBzZXR1cCB0
aG9zZSByYW5nZXMgZGVwZW5kaW5nIG9uIHBhcnRpY3VsYXIgaG9zdCBicmlkZ2UNCj4+Pj4+PiBp
bXBsZW1lbnRhdGlvbi4NCj4+Pj4+Pg0KPj4+Pj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBB
bmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+PiAt
LS0NCj4+Pj4+PiDCoMKgwqAgeGVuL2FyY2gvYXJtL3BjaS9lY2FtLmPCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgIHwgMTEgKysrKysrKysrKysNCj4+Pj4+PiDCoMKgwqAgeGVuL2FyY2gvYXJtL3BjaS9w
Y2ktaG9zdC1jb21tb24uYyB8IDE2ICsrKysrKysrKysrKysrKysNCj4+Pj4+PiDCoMKgwqAgeGVu
L2FyY2gvYXJtL3ZwY2kuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDEzICsrKysr
KysrKysrKysNCj4+Pj4+PiDCoMKgwqAgeGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaMKgwqDCoMKg
wqDCoMKgwqDCoCB8wqAgOCArKysrKysrKw0KPj4+Pj4+IMKgwqDCoCA0IGZpbGVzIGNoYW5nZWQs
IDQ4IGluc2VydGlvbnMoKykNCj4+Pj4+Pg0KPj4+Pj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9h
cm0vcGNpL2VjYW0uYyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+Pj4+Pj4gaW5kZXggOTFj
NjkxYjQxZmRmLi45MmVjYjJlMDc2MmIgMTAwNjQ0DQo+Pj4+Pj4gLS0tIGEveGVuL2FyY2gvYXJt
L3BjaS9lY2FtLmMNCj4+Pj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0KPj4+Pj4+
IEBAIC00Miw2ICs0MiwxNiBAQCB2b2lkIF9faW9tZW0gKnBjaV9lY2FtX21hcF9idXMoc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4+Pj4+IMKgwqDCoMKgwqDCoMKgIHJldHVybiBi
YXNlICsgKFBDSV9ERVZGTihzYmRmX3QuZGV2LCBzYmRmX3QuZm4pIDw8IGRldmZuX3NoaWZ0KSAr
IHdoZXJlOw0KPj4+Pj4+IMKgwqDCoCB9DQo+Pj4+Pj4gwqDCoMKgICtzdGF0aWMgaW50IHBjaV9l
Y2FtX3JlZ2lzdGVyX21taW9faGFuZGxlcihzdHJ1Y3QgZG9tYWluICpkLA0KPj4+Pj4+ICvCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+
Pj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgbW1pb19oYW5kbGVy
X29wcyAqb3BzKQ0KPj4+Pj4+ICt7DQo+Pj4+Pj4gK8KgwqDCoCBzdHJ1Y3QgcGNpX2NvbmZpZ193
aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0YTsNCj4+Pj4+PiArDQo+Pj4+Pj4gK8KgwqDCoCBy
ZWdpc3Rlcl9tbWlvX2hhbmRsZXIoZCwgb3BzLCBjZmctPnBoeXNfYWRkciwgY2ZnLT5zaXplLCBO
VUxMKTsNCj4+Pj4+DQo+Pj4+PiBXZSBoYXZlIGEgZml4ZWQgYXJyYXkgZm9yIGhhbmRsaW5nIHRo
ZSBNTUlPIGhhbmRsZXJzLg0KPj4+Pg0KPj4+PiBEaWRuJ3Qga25vdyB0aGF0Og0KPj4+Pg0KPj4+
PiB4ZW4vaW5jbHVkZS9hc20tYXJtL21taW8uaDoyNzojZGVmaW5lIE1BWF9JT19IQU5ETEVSwqAg
MTYNCj4+Pj4NCj4+Pj4+IFNvIHlvdSBuZWVkIHRvIG1ha2Ugc3VyZSB3ZSBoYXZlIGVub3VnaCBz
cGFjZSBpbiBpdCB0byBzdG9yZSBvbmUgaGFuZGxlciBwZXIgaGFuZGxlci4NCj4+Pj4+DQo+Pj4+
PiBUaGlzIGlzIHF1aXRlIHNpbWlsYXIgdG8gdGhlIHByb2JsZW0gd2UgaGFkIHdpdGggdGhlIHJl
LWRpc3RyaWJ1b3Igb24gR0lDdjMuIEhhdmUgYSBsb29rIHRoZXJlIHRvIHNlZSBob3cgd2UgZGVh
bHQgd2l0aCBpdC4NCj4+Pj4NCj4+Pj4gQ291bGQgeW91IHBsZWFzZSBwb2ludCBtZSB0byB0aGF0
IHNvbHV0aW9uPyBJIGNhbiBvbmx5IHNlZQ0KPj4+Pg0KPj4+PiDCoMKgIMKgwqDCoCAvKiBSZWdp
c3RlciBtbWlvIGhhbmRsZSBmb3IgdGhlIERpc3RyaWJ1dG9yICovDQo+Pj4+IMKgwqAgwqDCoMKg
IHJlZ2lzdGVyX21taW9faGFuZGxlcihkLCAmdmdpY19kaXN0cl9tbWlvX2hhbmRsZXIsIGQtPmFy
Y2gudmdpYy5kYmFzZSwNCj4+Pj4gwqDCoCDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBTWl82NEssIE5VTEwpOw0KPj4+Pg0KPj4+PiDCoMKgIMKgwqDC
oCAvKg0KPj4+PiDCoMKgIMKgwqDCoMKgICogUmVnaXN0ZXIgbW1pbyBoYW5kbGVyIHBlciBjb250
aWd1b3VzIHJlZ2lvbiBvY2N1cGllZCBieSB0aGUNCj4+Pj4gwqDCoCDCoMKgwqDCoCAqIHJlZGlz
dHJpYnV0b3JzLiBUaGUgaGFuZGxlciB3aWxsIHRha2UgY2FyZSB0byBjaG9vc2Ugd2hpY2gNCj4+
Pj4gwqDCoCDCoMKgwqDCoCAqIHJlZGlzdHJpYnV0b3IgaXMgdGFyZ2V0ZWQuDQo+Pj4+IMKgwqAg
wqDCoMKgwqAgKi8NCj4+Pj4gwqDCoCDCoMKgwqAgZm9yICggaSA9IDA7IGkgPCBkLT5hcmNoLnZn
aWMubnJfcmVnaW9uczsgaSsrICkNCj4+Pj4gwqDCoCDCoMKgwqAgew0KPj4+PiDCoMKgIMKgwqDC
oMKgwqDCoMKgIHN0cnVjdCB2Z2ljX3JkaXN0X3JlZ2lvbiAqcmVnaW9uID0gJmQtPmFyY2gudmdp
Yy5yZGlzdF9yZWdpb25zW2ldOw0KPj4+Pg0KPj4+PiDCoMKgIMKgwqDCoMKgwqDCoMKgIHJlZ2lz
dGVyX21taW9faGFuZGxlcihkLCAmdmdpY19yZGlzdHJfbW1pb19oYW5kbGVyLA0KPj4+PiDCoMKg
IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgcmVnaW9uLT5iYXNlLCByZWdpb24tPnNpemUsIHJlZ2lvbik7DQo+Pj4+IMKgwqAgwqDCoMKg
IH0NCj4+Pj4gd2hpY2ggSU1PIGRvZXNuJ3QgY2FyZSBhYm91dCB0aGUgbnVtYmVyIG9mIE1NSU9z
IHdlIGNhbiBoYW5kbGUNCj4+Pg0KPj4+IFBsZWFzZSBzZWUgdmdpY192M19pbml0KCkuIFdlIHVw
ZGF0ZSBtbWlvX2NvdW50IHRoYXQgaXMgdGhlbiB1c2VkIGFzIGEgdGhlIHNlY29uZCBhcmd1bWVu
dCBmb3IgZG9tYWluX2lvX2luaXQoKS4NCj4+DQo+PiBBaCwgc28NCj4+DQo+PiAxKSBUaGlzIG5l
ZWRzIHRvIGJlIGRvbmUgYmVmb3JlIHRoZSBhcnJheSBmb3IgdGhlIGhhbmRsZXJzIGlzIGFsbG9j
YXRlZA0KPj4NCj4+IDIpIEhvdyBkbyBJIGtub3cgYXQgdGhlIHRpbWUgb2YgMSkgaG93IG1hbnkg
YnJpZGdlcyB3ZSBoYXZlPw0KPg0KPiBCeSBjb3VudGluZyB0aGUgbnVtYmVyIG9mIGJyaWRnZSB5
b3Ugd2FudCB0byBleHBvc2UgdG8gZG9tMD8gSSBhbSBub3QgZW50aXJlbHkgc3VyZSB3aGF0IGVs
c2UgeW91IGV4cGVjdCBtZSB0byBzYXkuDQoNCk9rLCBzbyBJJ2xsIGdvIG92ZXIgdGhlIGRldmlj
ZSB0cmVlIGFuZCBmaW5kIG91dCBhbGwgdGhlIGJyaWRnZXMsIGUuZy4gZGV2aWNlcyB3aXRoIERF
VklDRV9QQ0kgdHlwZS4NCg0KVGhlbiBJJ2xsIGFsc28gbmVlZCB0byBleGNsdWRlIHRob3NlIGJl
aW5nIHBhc3NlZCB0aHJvdWdoICh4ZW4scGFzc3Rocm91Z2gpIGFuZCB0aGUgcmVzdCBhcmUgdGhl
IGJyaWRnZXMgZm9yIERvbWFpbi0wPw0KDQpJcyB0aGlzIHdoYXQgeW91IG1lYW4/DQoNCj4NCj4g
Q2hlZXJzLA0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:33:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:33:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184424.333052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgf4-0000lJ-FN; Fri, 10 Sep 2021 13:33:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184424.333052; Fri, 10 Sep 2021 13:33:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgf4-0000lC-CO; Fri, 10 Sep 2021 13:33:46 +0000
Received: by outflank-mailman (input) for mailman id 184424;
 Fri, 10 Sep 2021 13:33:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOgf3-0000l6-1w
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:33:45 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgf0-0007hB-TX; Fri, 10 Sep 2021 13:33:42 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOgf0-0005Xt-MX; Fri, 10 Sep 2021 13:33:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=fvjkvypgt+DjOOq2Wo+wfkw3UU8Rd4VVDj86LlKjXew=; b=bMXgPo4K3EIrgAFPatTRbLKC1+
	rolJvqQYCsItP1g4O3vJ7rKxOzPCL6sn4hOlV4VREisRXqDMnF+XvBEnltOdwV/XbnMOZgewDICGb
	VrE27cTR6c0aWI6wgOtAL1s+/1DpnkBcX7W6U1/Jildjfu0loff2EdgjFMK1xv/Dd4Vk=;
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
 <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
 <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org>
Date: Fri, 10 Sep 2021 14:33:40 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 14:27, Oleksandr Andrushchenko wrote:
> Hi,
> 
> On 10.09.21 16:20, Julien Grall wrote:
>>
>>
>> On 10/09/2021 14:15, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>
>> Hi,
>>
>>> On 10.09.21 16:04, Julien Grall wrote:
>>>>
>>>>
>>>> On 10/09/2021 12:43, Oleksandr Andrushchenko wrote:
>>>>> Hi, Julien!
>>>>
>>>> Hi Oleksandr,
>>>>
>>>>> On 09.09.21 20:43, Julien Grall wrote:
>>>>>> Hi Oleksandr,
>>>>>>
>>>>>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>>>
>>>>>>> In order vPCI to work it needs all access to PCI configuration space
>>>>>>> (ECAM) to be synchronized among all entities, e.g. hardware domain and
>>>>>>> guests.
>>>>>>
>>>>>> I am not entirely sure what you mean by "synchronized" here. Are you refer to the content of the configuration space?
>>>>>
>>>>> We maintain hwdom's and guest's view of the registers we are interested in
>>>>>
>>>>> So, to have hwdom's view we also need to trap its access to the configuration space.
>>>>>
>>>>> Probably "synchronized" is not the right wording here.
>>>> I would simply say that we want to expose an emulated hostbridge to the dom0 so we need to unmap the configuration space.
>>> Sounds good
>>>>
>>>>>
>>>>>>
>>>>>>> For that implement PCI host bridge specific callbacks to
>>>>>>> properly setup those ranges depending on particular host bridge
>>>>>>> implementation.
>>>>>>>
>>>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>>> ---
>>>>>>>      xen/arch/arm/pci/ecam.c            | 11 +++++++++++
>>>>>>>      xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
>>>>>>>      xen/arch/arm/vpci.c                | 13 +++++++++++++
>>>>>>>      xen/include/asm-arm/pci.h          |  8 ++++++++
>>>>>>>      4 files changed, 48 insertions(+)
>>>>>>>
>>>>>>> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
>>>>>>> index 91c691b41fdf..92ecb2e0762b 100644
>>>>>>> --- a/xen/arch/arm/pci/ecam.c
>>>>>>> +++ b/xen/arch/arm/pci/ecam.c
>>>>>>> @@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>>>>>>>          return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
>>>>>>>      }
>>>>>>>      +static int pci_ecam_register_mmio_handler(struct domain *d,
>>>>>>> +                                          struct pci_host_bridge *bridge,
>>>>>>> +                                          const struct mmio_handler_ops *ops)
>>>>>>> +{
>>>>>>> +    struct pci_config_window *cfg = bridge->sysdata;
>>>>>>> +
>>>>>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
>>>>>>
>>>>>> We have a fixed array for handling the MMIO handlers.
>>>>>
>>>>> Didn't know that:
>>>>>
>>>>> xen/include/asm-arm/mmio.h:27:#define MAX_IO_HANDLER  16
>>>>>
>>>>>> So you need to make sure we have enough space in it to store one handler per handler.
>>>>>>
>>>>>> This is quite similar to the problem we had with the re-distribuor on GICv3. Have a look there to see how we dealt with it.
>>>>>
>>>>> Could you please point me to that solution? I can only see
>>>>>
>>>>>         /* Register mmio handle for the Distributor */
>>>>>         register_mmio_handler(d, &vgic_distr_mmio_handler, d->arch.vgic.dbase,
>>>>>                               SZ_64K, NULL);
>>>>>
>>>>>         /*
>>>>>          * Register mmio handler per contiguous region occupied by the
>>>>>          * redistributors. The handler will take care to choose which
>>>>>          * redistributor is targeted.
>>>>>          */
>>>>>         for ( i = 0; i < d->arch.vgic.nr_regions; i++ )
>>>>>         {
>>>>>             struct vgic_rdist_region *region = &d->arch.vgic.rdist_regions[i];
>>>>>
>>>>>             register_mmio_handler(d, &vgic_rdistr_mmio_handler,
>>>>>                                   region->base, region->size, region);
>>>>>         }
>>>>> which IMO doesn't care about the number of MMIOs we can handle
>>>>
>>>> Please see vgic_v3_init(). We update mmio_count that is then used as a the second argument for domain_io_init().
>>>
>>> Ah, so
>>>
>>> 1) This needs to be done before the array for the handlers is allocated
>>>
>>> 2) How do I know at the time of 1) how many bridges we have?
>>
>> By counting the number of bridge you want to expose to dom0? I am not entirely sure what else you expect me to say.
> 
> Ok, so I'll go over the device tree and find out all the bridges, e.g. devices with DEVICE_PCI type.
> 
> Then I'll also need to exclude those being passed through (xen,passthrough) and the rest are the bridges for Domain-0?

What you want to know if how many times register_mmio_handler() will be 
called from domain_vpci_init().

You introduced a function pci_host_iterate_bridges() that will walk over 
the bridges and then call the callback vpci_setup_mmio_handler(). So you 
could introduce a new callback that will return 1 if 
bridge->ops->register_mmio_handler is not NULL or 0.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:40:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:40:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184431.333064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOglM-000290-54; Fri, 10 Sep 2021 13:40:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184431.333064; Fri, 10 Sep 2021 13:40:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOglM-00028t-25; Fri, 10 Sep 2021 13:40:16 +0000
Received: by outflank-mailman (input) for mailman id 184431;
 Fri, 10 Sep 2021 13:40:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOglL-00028n-6V
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:40:15 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9ff4861d-123c-11ec-b23b-12813bfff9fa;
 Fri, 10 Sep 2021 13:40:14 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ACsPxE006763; 
 Fri, 10 Sep 2021 13:40:11 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b07s685yh-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 13:40:11 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7347.eurprd03.prod.outlook.com (2603:10a6:20b:268::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 10 Sep
 2021 13:40:08 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 13:40:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ff4861d-123c-11ec-b23b-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mkrVU1ej+anzw8ZTLdpW3ZOMgs920Aqdiz+JNpj2/kUmZRpdCwG82VlrXu6t8MUuZTzj168aXa/Zajyk3Eqy0TrVr6VFUbOPYvs5EMYQRij+TC7ptihU3Y67+TF1JLZyEuP48EGIdDcPX7BL2WJOWrJ1ON9Oqd7A0O/c2XQVfMy95H9PURUm4ToetR56Hw8ZVUjisjBaoOLya0N6hPZHIgIg0q5CQFAax0p8X9rQuwKIaPCKH2oau/ydcwh7mKR5DMAbb8XIEmbLCWs7fLoBpPdG5VmwpqvZLSZ0YQK+XRkVM2Z8fFPyBwsqx3iUKJsny1/iLFOuHtesNZC+AddFtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OlePrKkrCb9/MDgXM37LTRZB+WEk5eclca8hSeLEvXo=;
 b=h2x+sQWH+GBN2bFnT4ue21GIsW31Z0xq6rUorfkMmZDZWED0+U37GyKnQehp85Ngxr92rHumvOPVSHzbcK65QX2Br2yKUOuEkxmscTqzhoOgRNCGFTcCQ8gvBIFA922P30xTubZamP08uY6vZ8V7L+QJDsc3PgYe/lIM6qaABmRyVQ/xEPI4VLSv4MLNVkSF0j83tkHpdPT2dJ/a28CnAQA40gt3vArSGktmCjuxgrgtCGyUEKCcCdWYCllWj9oRkCXJkHaQWAlWkIZlQ4hpqBwnzTZd/9AW+7MqTmhles/tv6yu7+N182IzdFIlWHG21BCoP+aOs1aEauv+H5piSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OlePrKkrCb9/MDgXM37LTRZB+WEk5eclca8hSeLEvXo=;
 b=eJoZNZoqK413hKRnVI2WhkxzhwWE5f+onaG1dzRTek8oIgISVsvVaQYmWB+BQ+fLm6lZTBIXKLrBJ73cgY1/0p1XhZFxvnZEacrUFnCFz7e0kf7yPsurFlPZTLK0Xquadn7wfViLeHMiYFwP2r0LOG7uqi18Qrtec+S+KefBApTI/povm8rHmDThqcRWXpIcpvXV4x2tktC/JHwM+KAzOXIY4iCaRTcwWJYDsyl6SuJhGQCt5q754soQGIUw2ZRQgatuzWDIQiofSZJV8aVHTqHv511vqoaR2HQqb/mc00GVKjfjyB99V/rP8l/U9Zj9LOkw93GezQQ3Lk69twf2CA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: 
 AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4ACAABanAIAAAwIAgAABgYCAAAHPAIAAAcgAgAABzYA=
Date: Fri, 10 Sep 2021 13:40:08 +0000
Message-ID: <52da7955-ead7-f404-6e51-7d1ee7bc3142@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
 <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
 <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
 <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org>
In-Reply-To: <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3e3b3be9-6e64-4ab1-8e2d-08d974608191
x-ms-traffictypediagnostic: AM9PR03MB7347:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB7347BD8F266EB4A82CDB13D8E7D69@AM9PR03MB7347.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 9vjJ5k/nPt0t9hWw+0nNz+m1nQdXjv/Pjq82qHppjkXHldgUyFuOKy7xB1xB0M55v/4vdoei8Jh9YLp1+MbP6BV3IOCBDQ1vAZMheTLdlT14Fl6ZeiDoOKcuh1hLgOGqMVSAL0NCpeQSgFJLxgiiLEVO0k4yAEi8ACsD8fQbUsVKXnoQnUdIZtMa7IeHAVP3OjqybXmUObjgdVsC48e56y4xadB8FPJjgGiC0B7AN4DxmWVpg4SfpbtBxG1GIABjID6f5d15meViJPqmKU6BSCQpltfUdIdA4iUm7/G7UnNbqNk5ODUjH1qSwcMxmNweZySTXT1pE5I3POJ1++31kxwz1j3IaHDmTKxsO6Qjk23SH96Z6uAippULDFYZbPYxC5ZNVxOarPD+Gf9Nu/bbCG9nFO00r2IknVaGPFdMv3s2diEWl51PqzbpJKrF8/v0QCSR0VP5teX+JjaH+Edqf86AlUNj+JRDyv2T5kgnbWpWhdJrzCqZ3nr1eS+bQtm6z9CVJ/lrlqjrkufCtD9g63mmJE0tglrnoxFJqHhModIlrZok/1G02oScpTTlTh7N0qePXYyxs7khvoTgxIAaWRiwpdfNZdLCtSSubczTH/eduYqE7wz6eITgKnCDIi6aZlUBooY1yiL409BLa7p00TRKeASh5C3UZAbNMODj6WGsJ9dlJ4d75txqH6zz1uJ3wIDWATMRDxqcPB46GpHuZzoTaHsY6FXkz/Qym+lFPT4=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(346002)(136003)(366004)(376002)(66946007)(66476007)(38100700002)(66556008)(64756008)(316002)(66446008)(6512007)(76116006)(2616005)(91956017)(31696002)(83380400001)(31686004)(478600001)(186003)(5660300002)(86362001)(26005)(71200400001)(4326008)(53546011)(122000001)(55236004)(38070700005)(54906003)(8676002)(6506007)(110136005)(6486002)(2906002)(8936002)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RWlXSDMrWnE3M0tLWm5nOTRtZmh3aVpzdHFEVHM4NkZDRTVES2p6dHMvYUVZ?=
 =?utf-8?B?Z0VKRE9LVFBWd0FpM3dLREpBSHk2T2t0QjFvNURFT0RzU2htVStLckFhN2Jv?=
 =?utf-8?B?WDJrT1BYYjhIcnRCRFUyUDVRdHNyWnA5MVBxbDVXS3BFYlVXUHZNR09SbnN4?=
 =?utf-8?B?NnN4MG5oeEFueGd0ek5RVEpyMGtoNlcrRm9kSFFlanJBMmdzcjlmVG10NW5I?=
 =?utf-8?B?TDdJeVRHaFFiQnJBSXprNE1NREJWTUlBblMreWluQ0I1TzRZdW1vRTgxV2Iw?=
 =?utf-8?B?VDBxemF6bmZLRDFKOHJDT3IwVGZZMzVDVEQ1MFlLYnBZcjdzTk1CYlZRL1ln?=
 =?utf-8?B?Y0cvOUVuYVl6RkVpaEw5bndkMmJjZHhScDhGU3h1UWlmbW1GSFhZZ2NWcEFK?=
 =?utf-8?B?SU1mTXZmaGxjTWZFMG9hQ0pPYjZRdkNLSlpwWXR1dEZKKzBKMmVGNGlvWEpX?=
 =?utf-8?B?UmNBbmxiaFQxdFRSSjNCdHE0akF3YUYxeU9zTkJsaFRhSEhwNnBXbzJ6OHQx?=
 =?utf-8?B?amZVLzdVMTE3OUlhRjc3bFlBYWRGTXhPcGdUVThzdjg1K0U0WWZEVlFDQzFi?=
 =?utf-8?B?ZjBpRWdSZ0s5TEhWZk45YlpHTHRDSVdtN2tob3dheWlLUy9GdEtORHBiUUVK?=
 =?utf-8?B?QUxWQVFtUE1ocHBiejJuWHJxZk43SGlzcnYwV3UxZlN1OUMyNVoyUlkyakNr?=
 =?utf-8?B?WkdPZDhiMC9MMmVwaXoxRTdXUkhGbWQ4WkI2RktDOHY2aHdQUzZSTjlGdkJq?=
 =?utf-8?B?c2RTSStPVHVOSS9FbkNGNzZ6ckYvbHlGWnRFQ1JJaWRoYlRyQWZnRUdBZFlw?=
 =?utf-8?B?U3pMcDlVU3dSdytuR1RoY3FWQU1keTc0VFJxQUxDNXBGL3BMQTFNUWZHVzda?=
 =?utf-8?B?V21SdGVYY2g0b0haS2JpSjJiUFZneTFud3pFbk1CRWwwU0VQb2NHclJ0em5V?=
 =?utf-8?B?UkV6RmxYTysvNzk0Q3pYYVE1dFVHdkN5L3lxSUhZT01lVHI2ZVdHU3F0VGtS?=
 =?utf-8?B?SjF4Uk9iYVVHeGFNK2N6QTRJVklMcmVuaklEK2t1ZzhJN0dCUFVoVEJKd0Zt?=
 =?utf-8?B?MGhBTjdGQ1o1NzgyNDBvT0hTeTJWbHlNOUtPdXdJbEx2SW1kVFRMY3R4elVH?=
 =?utf-8?B?YkMzU2lkTkFZVGJXTk5ORnE5MGVpYU8wbDBjVkRJWlhsVUpmM3NuRS82ZVdr?=
 =?utf-8?B?KzR6N2ZEQVNZYkZaYXdSczkwa0dQWFJ3NHpMaWxzbFp3c2dkK2JDK0xIT2JE?=
 =?utf-8?B?cldKd3lUeVRnZUhLT25LVHVGUlZNZ2hFUUlpdXNoZ2VKR2F1eGNsTlpGd2No?=
 =?utf-8?B?Nm5odmt3MVdETzY5N1QrenMvWnhRNmRKQ0RVYjNkejM4NUpVTkROSm5hUWln?=
 =?utf-8?B?cE94b2dxRkZmUEJ1WUZBZTVFNkROVmQrVXhlSkZPMTBPQjhSbnk1MnprLzdY?=
 =?utf-8?B?Z1ZGZnVaOHk1WUZ1bjNNeEVQcDdIbEVjbVJGd1JUNmwvVjdRMC9RbEdBdmtO?=
 =?utf-8?B?VCtocy85VHorWEl6dUZYSDFucnBzWU1uNzVINkxXUklGSjA0ZEFmL3dQN2tv?=
 =?utf-8?B?RnVsUUV0Wk9wSC9KUDMyM25vYkU0Rm15RU5Gc3FjN2R2VDBlN1FIR1pqVGdu?=
 =?utf-8?B?ZUdyOG56MnE5Y2ZVS0c4VVM2dE1QY1AyZ3RQNGY3bHFiMXJIKzlPSndNMXZs?=
 =?utf-8?B?V0VqREZNelBCY1NIYUFkM0NQejFOVGRIbVU3dVdyQUNmcmdSRGFxaGpCY3dz?=
 =?utf-8?B?YzRQTzBPRlozZk84RXJsTUVvaEQwRFo4WTFkNVdqeUFlNmNkRFFhTjBXR1JH?=
 =?utf-8?B?NEJoc2VzVVdLUnJyZkdKZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <EFCEA36F29000A469A3776AC3C440A97@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3e3b3be9-6e64-4ab1-8e2d-08d974608191
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 13:40:08.7067
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ZBU7zRyUNexKrsc72jcd6Rv7YdgkSncG1zaIPlSRjPHrDyApb+vL1slDx/ZWp9vzv+l2Y0kfiVJnj9LCw13tGQUBgQhEfFQHfcrpvvBOC/R9/PqebI3fmwKMvbYN2q+I
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7347
X-Proofpoint-GUID: gXIo9RXHwuG4YUyny2NU1_C23pMyFUJD
X-Proofpoint-ORIG-GUID: gXIo9RXHwuG4YUyny2NU1_C23pMyFUJD
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 impostorscore=0 bulkscore=0 malwarescore=0 clxscore=1015
 priorityscore=1501 mlxscore=0 phishscore=0 mlxlogscore=999 suspectscore=0
 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100079

DQpPbiAxMC4wOS4yMSAxNjozMywgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPg0KPg0KPiBPbiAxMC8w
OS8yMDIxIDE0OjI3LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEhpLA0KPj4N
Cj4+IE9uIDEwLjA5LjIxIDE2OjIwLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+Pj4NCj4+Pg0KPj4+
IE9uIDEwLzA5LzIwMjEgMTQ6MTUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+
PiBIaSwgSnVsaWVuIQ0KPj4+DQo+Pj4gSGksDQo+Pj4NCj4+Pj4gT24gMTAuMDkuMjEgMTY6MDQs
IEp1bGllbiBHcmFsbCB3cm90ZToNCj4+Pj4+DQo+Pj4+Pg0KPj4+Pj4gT24gMTAvMDkvMjAyMSAx
Mjo0MywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+Pj4gSGksIEp1bGllbiEN
Cj4+Pj4+DQo+Pj4+PiBIaSBPbGVrc2FuZHIsDQo+Pj4+Pg0KPj4+Pj4+IE9uIDA5LjA5LjIxIDIw
OjQzLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+Pj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+Pj4N
Cj4+Pj4+Pj4gT24gMDMvMDkvMjAyMSAwOTozMywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3Jv
dGU6DQo+Pj4+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2Fu
ZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pj4+Pg0KPj4+Pj4+Pj4gSW4gb3JkZXIgdlBDSSB0
byB3b3JrIGl0IG5lZWRzIGFsbCBhY2Nlc3MgdG8gUENJIGNvbmZpZ3VyYXRpb24gc3BhY2UNCj4+
Pj4+Pj4+IChFQ0FNKSB0byBiZSBzeW5jaHJvbml6ZWQgYW1vbmcgYWxsIGVudGl0aWVzLCBlLmcu
IGhhcmR3YXJlIGRvbWFpbiBhbmQNCj4+Pj4+Pj4+IGd1ZXN0cy4NCj4+Pj4+Pj4NCj4+Pj4+Pj4g
SSBhbSBub3QgZW50aXJlbHkgc3VyZSB3aGF0IHlvdSBtZWFuIGJ5ICJzeW5jaHJvbml6ZWQiIGhl
cmUuIEFyZSB5b3UgcmVmZXIgdG8gdGhlIGNvbnRlbnQgb2YgdGhlIGNvbmZpZ3VyYXRpb24gc3Bh
Y2U/DQo+Pj4+Pj4NCj4+Pj4+PiBXZSBtYWludGFpbiBod2RvbSdzIGFuZCBndWVzdCdzIHZpZXcg
b2YgdGhlIHJlZ2lzdGVycyB3ZSBhcmUgaW50ZXJlc3RlZCBpbg0KPj4+Pj4+DQo+Pj4+Pj4gU28s
IHRvIGhhdmUgaHdkb20ncyB2aWV3IHdlIGFsc28gbmVlZCB0byB0cmFwIGl0cyBhY2Nlc3MgdG8g
dGhlIGNvbmZpZ3VyYXRpb24gc3BhY2UuDQo+Pj4+Pj4NCj4+Pj4+PiBQcm9iYWJseSAic3luY2hy
b25pemVkIiBpcyBub3QgdGhlIHJpZ2h0IHdvcmRpbmcgaGVyZS4NCj4+Pj4+IEkgd291bGQgc2lt
cGx5IHNheSB0aGF0IHdlIHdhbnQgdG8gZXhwb3NlIGFuIGVtdWxhdGVkIGhvc3RicmlkZ2UgdG8g
dGhlIGRvbTAgc28gd2UgbmVlZCB0byB1bm1hcCB0aGUgY29uZmlndXJhdGlvbiBzcGFjZS4NCj4+
Pj4gU291bmRzIGdvb2QNCj4+Pj4+DQo+Pj4+Pj4NCj4+Pj4+Pj4NCj4+Pj4+Pj4+IEZvciB0aGF0
IGltcGxlbWVudCBQQ0kgaG9zdCBicmlkZ2Ugc3BlY2lmaWMgY2FsbGJhY2tzIHRvDQo+Pj4+Pj4+
PiBwcm9wZXJseSBzZXR1cCB0aG9zZSByYW5nZXMgZGVwZW5kaW5nIG9uIHBhcnRpY3VsYXIgaG9z
dCBicmlkZ2UNCj4+Pj4+Pj4+IGltcGxlbWVudGF0aW9uLg0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IFNp
Z25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNo
ZW5rb0BlcGFtLmNvbT4NCj4+Pj4+Pj4+IC0tLQ0KPj4+Pj4+Pj4gwqDCoMKgwqAgeGVuL2FyY2gv
YXJtL3BjaS9lY2FtLmPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgMTEgKysrKysrKysrKysNCj4+
Pj4+Pj4+IMKgwqDCoMKgIHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMgfCAxNiAr
KysrKysrKysrKysrKysrDQo+Pj4+Pj4+PiDCoMKgwqDCoCB4ZW4vYXJjaC9hcm0vdnBjaS5jwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgMTMgKysrKysrKysrKysrKw0KPj4+Pj4+Pj4g
wqDCoMKgwqAgeGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaMKgwqDCoMKgwqDCoMKgwqDCoCB8wqAg
OCArKysrKysrKw0KPj4+Pj4+Pj4gwqDCoMKgwqAgNCBmaWxlcyBjaGFuZ2VkLCA0OCBpbnNlcnRp
b25zKCspDQo+Pj4+Pj4+Pg0KPj4+Pj4+Pj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kv
ZWNhbS5jIGIveGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMNCj4+Pj4+Pj4+IGluZGV4IDkxYzY5MWI0
MWZkZi4uOTJlY2IyZTA3NjJiIDEwMDY0NA0KPj4+Pj4+Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3Bj
aS9lY2FtLmMNCj4+Pj4+Pj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+Pj4+Pj4+
PiBAQCAtNDIsNiArNDIsMTYgQEAgdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVj
dCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+Pj4+Pj4+IMKgwqDCoMKgwqDCoMKgwqAgcmV0
dXJuIGJhc2UgKyAoUENJX0RFVkZOKHNiZGZfdC5kZXYsIHNiZGZfdC5mbikgPDwgZGV2Zm5fc2hp
ZnQpICsgd2hlcmU7DQo+Pj4+Pj4+PiDCoMKgwqDCoCB9DQo+Pj4+Pj4+PiDCoMKgwqDCoCArc3Rh
dGljIGludCBwY2lfZWNhbV9yZWdpc3Rlcl9tbWlvX2hhbmRsZXIoc3RydWN0IGRvbWFpbiAqZCwN
Cj4+Pj4+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0cnVjdCBwY2lfaG9zdF9icmlk
Z2UgKmJyaWRnZSwNCj4+Pj4+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNvbnN0IHN0
cnVjdCBtbWlvX2hhbmRsZXJfb3BzICpvcHMpDQo+Pj4+Pj4+PiArew0KPj4+Pj4+Pj4gK8KgwqDC
oCBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0YTsNCj4+Pj4+
Pj4+ICsNCj4+Pj4+Pj4+ICvCoMKgwqAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsIG9wcywgY2Zn
LT5waHlzX2FkZHIsIGNmZy0+c2l6ZSwgTlVMTCk7DQo+Pj4+Pj4+DQo+Pj4+Pj4+IFdlIGhhdmUg
YSBmaXhlZCBhcnJheSBmb3IgaGFuZGxpbmcgdGhlIE1NSU8gaGFuZGxlcnMuDQo+Pj4+Pj4NCj4+
Pj4+PiBEaWRuJ3Qga25vdyB0aGF0Og0KPj4+Pj4+DQo+Pj4+Pj4geGVuL2luY2x1ZGUvYXNtLWFy
bS9tbWlvLmg6Mjc6I2RlZmluZSBNQVhfSU9fSEFORExFUsKgIDE2DQo+Pj4+Pj4NCj4+Pj4+Pj4g
U28geW91IG5lZWQgdG8gbWFrZSBzdXJlIHdlIGhhdmUgZW5vdWdoIHNwYWNlIGluIGl0IHRvIHN0
b3JlIG9uZSBoYW5kbGVyIHBlciBoYW5kbGVyLg0KPj4+Pj4+Pg0KPj4+Pj4+PiBUaGlzIGlzIHF1
aXRlIHNpbWlsYXIgdG8gdGhlIHByb2JsZW0gd2UgaGFkIHdpdGggdGhlIHJlLWRpc3RyaWJ1b3Ig
b24gR0lDdjMuIEhhdmUgYSBsb29rIHRoZXJlIHRvIHNlZSBob3cgd2UgZGVhbHQgd2l0aCBpdC4N
Cj4+Pj4+Pg0KPj4+Pj4+IENvdWxkIHlvdSBwbGVhc2UgcG9pbnQgbWUgdG8gdGhhdCBzb2x1dGlv
bj8gSSBjYW4gb25seSBzZWUNCj4+Pj4+Pg0KPj4+Pj4+IMKgwqDCoCDCoMKgwqAgLyogUmVnaXN0
ZXIgbW1pbyBoYW5kbGUgZm9yIHRoZSBEaXN0cmlidXRvciAqLw0KPj4+Pj4+IMKgwqDCoCDCoMKg
wqAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsICZ2Z2ljX2Rpc3RyX21taW9faGFuZGxlciwgZC0+
YXJjaC52Z2ljLmRiYXNlLA0KPj4+Pj4+IMKgwqDCoCDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBTWl82NEssIE5VTEwpOw0KPj4+Pj4+DQo+Pj4+Pj4g
wqDCoMKgIMKgwqDCoCAvKg0KPj4+Pj4+IMKgwqDCoCDCoMKgwqDCoCAqIFJlZ2lzdGVyIG1taW8g
aGFuZGxlciBwZXIgY29udGlndW91cyByZWdpb24gb2NjdXBpZWQgYnkgdGhlDQo+Pj4+Pj4gwqDC
oMKgIMKgwqDCoMKgICogcmVkaXN0cmlidXRvcnMuIFRoZSBoYW5kbGVyIHdpbGwgdGFrZSBjYXJl
IHRvIGNob29zZSB3aGljaA0KPj4+Pj4+IMKgwqDCoCDCoMKgwqDCoCAqIHJlZGlzdHJpYnV0b3Ig
aXMgdGFyZ2V0ZWQuDQo+Pj4+Pj4gwqDCoMKgIMKgwqDCoMKgICovDQo+Pj4+Pj4gwqDCoMKgIMKg
wqDCoCBmb3IgKCBpID0gMDsgaSA8IGQtPmFyY2gudmdpYy5ucl9yZWdpb25zOyBpKysgKQ0KPj4+
Pj4+IMKgwqDCoCDCoMKgwqAgew0KPj4+Pj4+IMKgwqDCoCDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3Qg
dmdpY19yZGlzdF9yZWdpb24gKnJlZ2lvbiA9ICZkLT5hcmNoLnZnaWMucmRpc3RfcmVnaW9uc1tp
XTsNCj4+Pj4+Pg0KPj4+Pj4+IMKgwqDCoCDCoMKgwqDCoMKgwqDCoCByZWdpc3Rlcl9tbWlvX2hh
bmRsZXIoZCwgJnZnaWNfcmRpc3RyX21taW9faGFuZGxlciwNCj4+Pj4+PiDCoMKgwqAgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZWdp
b24tPmJhc2UsIHJlZ2lvbi0+c2l6ZSwgcmVnaW9uKTsNCj4+Pj4+PiDCoMKgwqAgwqDCoMKgIH0N
Cj4+Pj4+PiB3aGljaCBJTU8gZG9lc24ndCBjYXJlIGFib3V0IHRoZSBudW1iZXIgb2YgTU1JT3Mg
d2UgY2FuIGhhbmRsZQ0KPj4+Pj4NCj4+Pj4+IFBsZWFzZSBzZWUgdmdpY192M19pbml0KCkuIFdl
IHVwZGF0ZSBtbWlvX2NvdW50IHRoYXQgaXMgdGhlbiB1c2VkIGFzIGEgdGhlIHNlY29uZCBhcmd1
bWVudCBmb3IgZG9tYWluX2lvX2luaXQoKS4NCj4+Pj4NCj4+Pj4gQWgsIHNvDQo+Pj4+DQo+Pj4+
IDEpIFRoaXMgbmVlZHMgdG8gYmUgZG9uZSBiZWZvcmUgdGhlIGFycmF5IGZvciB0aGUgaGFuZGxl
cnMgaXMgYWxsb2NhdGVkDQo+Pj4+DQo+Pj4+IDIpIEhvdyBkbyBJIGtub3cgYXQgdGhlIHRpbWUg
b2YgMSkgaG93IG1hbnkgYnJpZGdlcyB3ZSBoYXZlPw0KPj4+DQo+Pj4gQnkgY291bnRpbmcgdGhl
IG51bWJlciBvZiBicmlkZ2UgeW91IHdhbnQgdG8gZXhwb3NlIHRvIGRvbTA/IEkgYW0gbm90IGVu
dGlyZWx5IHN1cmUgd2hhdCBlbHNlIHlvdSBleHBlY3QgbWUgdG8gc2F5Lg0KPj4NCj4+IE9rLCBz
byBJJ2xsIGdvIG92ZXIgdGhlIGRldmljZSB0cmVlIGFuZCBmaW5kIG91dCBhbGwgdGhlIGJyaWRn
ZXMsIGUuZy4gZGV2aWNlcyB3aXRoIERFVklDRV9QQ0kgdHlwZS4NCj4+DQo+PiBUaGVuIEknbGwg
YWxzbyBuZWVkIHRvIGV4Y2x1ZGUgdGhvc2UgYmVpbmcgcGFzc2VkIHRocm91Z2ggKHhlbixwYXNz
dGhyb3VnaCkgYW5kIHRoZSByZXN0IGFyZSB0aGUgYnJpZGdlcyBmb3IgRG9tYWluLTA/DQo+DQo+
IFdoYXQgeW91IHdhbnQgdG8ga25vdyBpZiBob3cgbWFueSB0aW1lcyByZWdpc3Rlcl9tbWlvX2hh
bmRsZXIoKSB3aWxsIGJlIGNhbGxlZCBmcm9tIGRvbWFpbl92cGNpX2luaXQoKS4NCj4NCj4gWW91
IGludHJvZHVjZWQgYSBmdW5jdGlvbiBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoKSB0aGF0IHdp
bGwgd2FsayBvdmVyIHRoZSBicmlkZ2VzIGFuZCB0aGVuIGNhbGwgdGhlIGNhbGxiYWNrIHZwY2lf
c2V0dXBfbW1pb19oYW5kbGVyKCkuIFNvIHlvdSBjb3VsZCBpbnRyb2R1Y2UgYSBuZXcgY2FsbGJh
Y2sgdGhhdCB3aWxsIHJldHVybiAxIGlmIGJyaWRnZS0+b3BzLT5yZWdpc3Rlcl9tbWlvX2hhbmRs
ZXIgaXMgbm90IE5VTEwgb3IgMC4NCg0KT2ssIGNsZWFyLiBTb21ldGhpbmcgbGlrZToNCg0KIMKg
wqDCoCBpZiAoIChyYyA9IGRvbWFpbl92Z2ljX3JlZ2lzdGVyKGQsICZjb3VudCkpICE9IDAgKQ0K
IMKgwqDCoMKgwqDCoMKgIGdvdG8gZmFpbDsNCg0KIMKgwqDCoCAqZmluZCBvdXQgaG93IG1hbnkg
YnJpZGdlcyBhbmQgdXBkYXRlIGNvdW50Kg0KDQoNCiDCoMKgwqAgaWYgKCAocmMgPSBkb21haW5f
aW9faW5pdChkLCBjb3VudCArIE1BWF9JT19IQU5ETEVSKSkgIT0gMCApDQogwqDCoMKgwqDCoMKg
wqAgZ290byBmYWlsOw0KDQo+DQo+IENoZWVycywNCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRy
DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:46:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:46:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184440.333074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgrb-0003Cb-RN; Fri, 10 Sep 2021 13:46:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184440.333074; Fri, 10 Sep 2021 13:46:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgrb-0003CU-OO; Fri, 10 Sep 2021 13:46:43 +0000
Received: by outflank-mailman (input) for mailman id 184440;
 Fri, 10 Sep 2021 13:46:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOgra-0003C7-7H
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:46:42 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id acce3cbb-a07a-45c3-91db-962c45475aad;
 Fri, 10 Sep 2021 13:46:41 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 54AED22408;
 Fri, 10 Sep 2021 13:46:40 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 0F7DD132CD;
 Fri, 10 Sep 2021 13:46:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id 44uxAcBhO2GtfQAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 13:46:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acce3cbb-a07a-45c3-91db-962c45475aad
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631281600; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=0/oO1bqbBnt7Xpm28H334yoDk09pjQKN6+Y0ISC1RHI=;
	b=KxL6xASRvKYNXEe0YrnYokzNxCEBy06Xb72otUJmSARWOp8VI98mci1uA1hAYKB0ReKEy3
	CMvKijxW9ql1OWb7RhnJCMb6Koj2B8Qbcu74yi7VUd25JMTAkP88PFcx2Y/bbBApilITdJ
	QslkEUd93SeT84sy14bVj52Nh6qVtCk=
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
Message-ID: <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
Date: Fri, 10 Sep 2021 15:46:39 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="zwU9lWMpLzWFFwpgNuI06qBL2dNPydqjG"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--zwU9lWMpLzWFFwpgNuI06qBL2dNPydqjG
Content-Type: multipart/mixed; boundary="KpBU4i2MbvQ60vBekd6K6rjXd5XYtZb4H";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
In-Reply-To: <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>

--KpBU4i2MbvQ60vBekd6K6rjXd5XYtZb4H
Content-Type: multipart/mixed;
 boundary="------------8579A1FA190F7A2B21FF6EFB"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8579A1FA190F7A2B21FF6EFB
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 15:22, Jan Beulich wrote:
> On 09.09.2021 08:27, Juergen Gross wrote:
>> Today Xenstore is using one unstable interface to retrieve the state
>> of a domain (existing, dying).
>>
>> In order to decouple Xenstore further from the hypervisor, a stable
>> interface should be established. This new interface should cover the
>> following functionality:
>>
>> - get the needed state of a domain (existing, dying)
>> - get a unique identifier per domain telling Xenstore that a domid
>>     has been reused (today Xenstore could in theory miss this)
>> - provide a way to avoid having to get the state of all domains when
>>     a state change of a domain has been signaled (there is only an eve=
nt
>>     "some domain has changed state" today: VIRQ_DOM_EXC)
>>
>> I'm suggesting the following hypervisor modifications:
>>
>> - Addition of a global 64 bit domain sequence counter. This counter
>>     will be incremented each time a new domain is created.
>> - Addition of a sequence count to struct domain. It will be set to
>>     the value of the new global domain sequence counter when the domai=
n
>>     is being created.
>=20
> This looks like an "extended domain ID", which I think we want to
> avoid unless exposure is extremely limited. Since we can't easily
> predict what future usage this may have, I would suggest to seed the
> global with a 15-bit (or wider) random value. I'd like to further
> suggest considering to not have this be maintained by simply
> incrementing, to make very clear to consumers that the actual value
> has no other meaning. This could be done by multiplying by a random
> odd number larger than 1 (determined at build or boot time). All we
> want to guarantee is that numbers don't repeat (earlier than a 64-bit
> value would wrap when linearly incremented).

Fine with me.

>=20
>> - Addition of a global bitmap with one bit for each possible domid
>>     (resulting in a 4 kB page). Each time the state of a domain is cha=
nged
>>     (domain creation, shutdown, deletion) the related bit is set.
>>     When the VIRQ_DOM_EXC event is being registered the bitmap is rese=
t to
>>     have the bits set for all existing domains.
>=20
> This page, as I understand from the subsequent item, is not intended
> to be exposed to the domain running xenstored (or any other one)?

Correct.

>=20
>> - Addition of a new stable hypercall ("get domain state") returning th=
e
>>     following information:
>>     + domid of a domain having the bit set in above bitmap
>>     + state of that domain (existing, dying)
>>     + sequence count of that domain
>>     The related bit is reset in the bitmap as a side effect of the cal=
l.
>=20
> What I'd like us to consider up front is whether xenstored is going
> to remain only entity interested in this kind of information. The
> entire design looks to leverage that there's only a single consumer
> in the system.

Right. I'm just writing some RFC patches, and I have coded this
interface to be usable only for the domain having VIRQ_DOM_EXC
registered.

The alternative (IMO) would have been to expose the domain-state
bitmap to Xenstore (and/or other interested parties).

> I understand the caller would iterate over this hypercall. Is there
> concern about this iteration never finishing, if e.g. a guest gets
> rebooted quickly enough?

No. As a reboot will always include Xenstore activity, there is no
chance for that to happen.


Juergen

--------------8579A1FA190F7A2B21FF6EFB
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8579A1FA190F7A2B21FF6EFB--

--KpBU4i2MbvQ60vBekd6K6rjXd5XYtZb4H--

--zwU9lWMpLzWFFwpgNuI06qBL2dNPydqjG
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE7Yb8FAwAAAAAACgkQsN6d1ii/Ey+K
IAf+L4LVjB3uw/sIQTRyvNfKTZk/vgN6ItT1iVZD118YL15mPomM9jbOAO9DfnLtoRZOZoL1x0gB
nxnu9p1qP4QaXpn8mAhCBBtUJ3H5P1zeJmy7PTSCZi2K4EW0lAAL7t2fpkxncF1YyCbJKNleEbQN
4KOmVxxb04nhVDpqd7kiioKAisJOdpulRsUA9ktmndeYGpb451kfZA6VIZfTuqfQZPhOiHAfT5fr
Um29Vgb6Ruru9snP5phAe8jV7SX6s3Exf2d2m9VCTdFRd7a+MMc1j9yUVTQNzIh3ob6jKTe9ySn/
OaywrrtCl98lHSl2xYwvpzMeO5Uwf9PBWHuFPzWz4A==
=1gxN
-----END PGP SIGNATURE-----

--zwU9lWMpLzWFFwpgNuI06qBL2dNPydqjG--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:49:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:49:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184446.333086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOguh-0003r2-Ah; Fri, 10 Sep 2021 13:49:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184446.333086; Fri, 10 Sep 2021 13:49:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOguh-0003qv-7d; Fri, 10 Sep 2021 13:49:55 +0000
Received: by outflank-mailman (input) for mailman id 184446;
 Fri, 10 Sep 2021 13:49:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOgug-0003ql-2c
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:49:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOguf-0007yX-9G; Fri, 10 Sep 2021 13:49:53 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOguf-0007Gn-2t; Fri, 10 Sep 2021 13:49:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=qQZIHAjGUtLWXgYntRdFPbYKImH0yNbGeAm0RCbB1Lw=; b=x+5SbjARVQFJ1oZypDSVUACgFa
	4oORLUlGQ779sX7EjfTrLbQB7KumraUDijBWSl1tSg4OE2jab/W0rQw5jF9H80CEAqqH75oVa9alT
	DHL/v7CNd3Q6vMsUM5egzFCPEW00ShDdM2vsORVPzLOuwWx/7oZvKjQqlx1hfWPu/qMQ=;
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
To: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>
Date: Fri, 10 Sep 2021 14:49:51 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 14:46, Juergen Gross wrote:
> On 10.09.21 15:22, Jan Beulich wrote:
>> On 09.09.2021 08:27, Juergen Gross wrote:
>>> - Addition of a new stable hypercall ("get domain state") returning the
>>>     following information:
>>>     + domid of a domain having the bit set in above bitmap
>>>     + state of that domain (existing, dying)
>>>     + sequence count of that domain
>>>     The related bit is reset in the bitmap as a side effect of the call.
>>
>> What I'd like us to consider up front is whether xenstored is going
>> to remain only entity interested in this kind of information. The
>> entire design looks to leverage that there's only a single consumer
>> in the system.
> 
> Right. I'm just writing some RFC patches, and I have coded this
> interface to be usable only for the domain having VIRQ_DOM_EXC
> registered.
> 
> The alternative (IMO) would have been to expose the domain-state
> bitmap to Xenstore (and/or other interested parties).

If we do that, let's do in such way that guest_*_bit() helpers are not 
necessary because they are a massive hammer on Arm. This means the 
bitmap would have to be read-only for the domain.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 13:51:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 13:51:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184455.333097 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgwO-0005TZ-Le; Fri, 10 Sep 2021 13:51:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184455.333097; Fri, 10 Sep 2021 13:51:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOgwO-0005TS-In; Fri, 10 Sep 2021 13:51:40 +0000
Received: by outflank-mailman (input) for mailman id 184455;
 Fri, 10 Sep 2021 13:51:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dvCC=OA=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mOgwM-0005TK-U8
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:51:38 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 147edc5c-e521-4a01-9dd6-aed65e8687cb;
 Fri, 10 Sep 2021 13:51:38 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6BA5022411;
 Fri, 10 Sep 2021 13:51:37 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 383811365E;
 Fri, 10 Sep 2021 13:51:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id DvJDDOliO2HBfgAAGKfGzw
 (envelope-from <jgross@suse.com>); Fri, 10 Sep 2021 13:51:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 147edc5c-e521-4a01-9dd6-aed65e8687cb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631281897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=IL7RrHLyO1ms5CLZ2fiGdAXRGauatY8lCYmybpVEy6E=;
	b=MVkezbcD9xozlvJMYO26w2ZuQzGEKuNkTZZKSGy25jk+uEvEUCpe6BgO0kx1JTplneWk0b
	yMq2XjRNmdrTeUWN2lG5T6+W7qCeRttnSR+rBpiQ7u4J8CsKhWU4IqHH0ovRCQm9uBUjOC
	nJdtnu/TiHtaPA65aSX1XFmmU6PEh1Q=
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
 <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <a96ea57e-ab7b-c9ce-02c1-2061a43ee72b@suse.com>
Date: Fri, 10 Sep 2021 15:51:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="UEX5uIssnXQZuzl1oKR8iyVrTo4aNVydB"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--UEX5uIssnXQZuzl1oKR8iyVrTo4aNVydB
Content-Type: multipart/mixed; boundary="OC1KpLb1c0HpBKLsF0z28viaoh8UhY3JT";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <a96ea57e-ab7b-c9ce-02c1-2061a43ee72b@suse.com>
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
 <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>
In-Reply-To: <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>

--OC1KpLb1c0HpBKLsF0z28viaoh8UhY3JT
Content-Type: multipart/mixed;
 boundary="------------AFE3E502EE27EA7685A5FC01"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------AFE3E502EE27EA7685A5FC01
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 15:49, Julien Grall wrote:
>=20
>=20
> On 10/09/2021 14:46, Juergen Gross wrote:
>> On 10.09.21 15:22, Jan Beulich wrote:
>>> On 09.09.2021 08:27, Juergen Gross wrote:
>>>> - Addition of a new stable hypercall ("get domain state") returning =
the
>>>> =C2=A0=C2=A0=C2=A0 following information:
>>>> =C2=A0=C2=A0=C2=A0 + domid of a domain having the bit set in above b=
itmap
>>>> =C2=A0=C2=A0=C2=A0 + state of that domain (existing, dying)
>>>> =C2=A0=C2=A0=C2=A0 + sequence count of that domain
>>>> =C2=A0=C2=A0=C2=A0 The related bit is reset in the bitmap as a side =
effect of the=20
>>>> call.
>>>
>>> What I'd like us to consider up front is whether xenstored is going
>>> to remain only entity interested in this kind of information. The
>>> entire design looks to leverage that there's only a single consumer
>>> in the system.
>>
>> Right. I'm just writing some RFC patches, and I have coded this
>> interface to be usable only for the domain having VIRQ_DOM_EXC
>> registered.
>>
>> The alternative (IMO) would have been to expose the domain-state
>> bitmap to Xenstore (and/or other interested parties).
>=20
> If we do that, let's do in such way that guest_*_bit() helpers are not =

> necessary because they are a massive hammer on Arm. This means the=20
> bitmap would have to be read-only for the domain.

I strongly prefer my current approach not needing a shared memory page.


Juergen

--------------AFE3E502EE27EA7685A5FC01
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------AFE3E502EE27EA7685A5FC01--

--OC1KpLb1c0HpBKLsF0z28viaoh8UhY3JT--

--UEX5uIssnXQZuzl1oKR8iyVrTo4aNVydB
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE7YugFAwAAAAAACgkQsN6d1ii/Ey/b
xAgAlGAzep9SGniObF0t2+TXMcvxoMDfMD4nFabPse+18FvmguR6q/2jEPGG3/2zYKOLFDa9KPUi
FX+dP5+x4hfQkJEEccHcuOyQ7qYiA3zAZxCv/67/iDJ16YS413pAXmB23mTMg8XPyCZ7/9ruaUAm
7yx8eqRji3Bx8G+2W37u6NqIXSBJGQQYSLYBZ9QrYbWEw1XKHYCui5FlEInkJF+VCjtxmbmFrj1t
Qw2Gbvh+XbwiMsYAwMdpNdH4gB7N1qXBZzKtopNugmRErzVQ10yktDwu1UwsLadjyhl0EyYPK6+8
xOjhqPnBLvJ8f1dP+nNT5K1pVcssY1ltMYtYdqPIsw==
=Rc+g
-----END PGP SIGNATURE-----

--UEX5uIssnXQZuzl1oKR8iyVrTo4aNVydB--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:00:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:00:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184461.333108 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh4S-0006Bv-HL; Fri, 10 Sep 2021 14:00:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184461.333108; Fri, 10 Sep 2021 14:00:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh4S-0006Bo-E0; Fri, 10 Sep 2021 14:00:00 +0000
Received: by outflank-mailman (input) for mailman id 184461;
 Fri, 10 Sep 2021 13:59:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOh4R-0006Bi-Gq
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 13:59:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOh4R-0008AB-9J; Fri, 10 Sep 2021 13:59:59 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOh4R-00088J-2q; Fri, 10 Sep 2021 13:59:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:Cc:References:To:Subject;
	bh=bpXAA5w1/r3kp0w3tt0UH+H9P70TI+xto0sCug+OWW4=; b=nIjnIBrp8Wgxvi+08Jr3STiWwn
	EcxQf9j9GOzM52pJR5d0Y+czG+HB3mvZM6hbZyQaCCg0LW8c0mHkxHiTT1iauGSdSqc3Y+6JgU8t+
	DPpp1EgdwrAZJQJwe3LbAZpDzYKvoiQsvlFYeD/+qizD33jpaQ0PtbszBb9zvKfFjrDs=;
Subject: Re: Xen Booting Problem on ARM Machine
To: Sai Kiran <ysaikiran1997@gmail.com>
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Jan Beulich
 <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Julien Grall <julien@xen.org>
Message-ID: <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org>
Date: Fri, 10 Sep 2021 14:59:57 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 08/09/2021 11:43, Sai Kiran wrote:
> Hello,

Hi,

Thank you for the report. Moving the discussion back to xen-devel 
because this looks like a potential issue in the UEFI stub in Xen.

> I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it.

Would you be able to give more details on the Arm machine you are using? 
Also, are you using ACPI or DT to boot?

>  I 
> am able to do “make world” and “make install”, after “./configure”, as 
> specified in README file. When I reboot the system, I get the following 
> message:
> 
> Warning: All 128 bootinfo mem banks exhausted.
> 
> Warning: All 128 bootinfo mem banks exhausted.

Hmmm... This means that you have more than 128 memory regions described 
in the EFI memory map. That's quite a lot.

Although, this should be harmless as it means Xen will not use the extra 
memory banks.

> 
> Cannot exit boot services: ErrCode: 0x8000000000000002

This means EFI_INVALID_PARAMETER. We have code to retry because AFAICT 
ExitBootServices() may sometime fails (I have CCed Jan may have more 
idea what's happening).

Would you be able to provide more details on the UEFI firmware you are 
using? Is it EDK2 or U-boot?

Also, do you know if Linux is boot on the same system?

However, AFAICT, the error message would not prevent Xen to continue 
booting. So you may get stuck later in the boot process.

My suggestion would be to enable earlyprintk for your platform. You can 
setup it up from the menuconfig in "Debugging Options".

> 
> Here is the snapshot of the error.

Please avoid posting image on the ML. Instead, if you need to do it, you 
can upload on a web server and provide the URL.

Best regards,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:02:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:02:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184470.333119 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh6X-00084W-2D; Fri, 10 Sep 2021 14:02:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184470.333119; Fri, 10 Sep 2021 14:02:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh6W-00084P-VS; Fri, 10 Sep 2021 14:02:08 +0000
Received: by outflank-mailman (input) for mailman id 184470;
 Fri, 10 Sep 2021 14:02:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOh6V-00083Y-EB
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:02:07 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ada91ffe-123f-11ec-b242-12813bfff9fa;
 Fri, 10 Sep 2021 14:02:05 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ADi9kI004046; 
 Fri, 10 Sep 2021 14:02:02 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b0731ge0j-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 14:02:01 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4305.eurprd03.prod.outlook.com (2603:10a6:208:c0::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 14:01:58 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 14:01:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ada91ffe-123f-11ec-b242-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hhH7L5D1JjdVHdTtdKB4kBq6ZDCel+b08VU67z4RdDQoBSbzqOsjyI642TyWLSmKryhzQGAgCEGomEXgVhUoc5alZ78x0jyaBKQFbSRGKYb9UQlY3ao/17IElWlkNd5pWNu1W/Qg3q//5V2MnR0EmU6r/R9qie+o2ygZ328jd+y0FviXmoV1RMCD6+LTsekvyr9ncbzLgf7CUATALF+ZccFQDFXbmqtMH4GRLMBg+Zcjkba3G474JBQuP4Kp1nplKLiqO2C8b/WU1qRHBMeMH74IYqbRC9ESBZMTMlMoIzGMxbIXmt9fCxJ2gcqbvx2H41nl7j94kXJSGMeGKY5jiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CU+lIt2uva7UKXKnBs1OiCP1uunsHpfUorn1ZfAfK5Q=;
 b=b0ox3sFF0BB4pzCbiVHKZkoN68JU9RT/s87oCAjUWDFrQuM2dNQ6genWy3TRJn62RJx+GwWVYKs7DsIyUPwSnFxJFrcsTx+jSKDYX26FwFondm1qpMSEpQXcALZr0QDup8Q6H5Ak1VsiD3/5KnLkS8aIyqnDpkLHk9V/XGYWZjWVeoGjjsg70he6cTOzerW+T8grB7tQ5o1vJVsEQdzjvmvKq3VgbmBDnV2gvyvblkOvMs5BtqnLJI5jpWKpBo/sYZmaQ1nRhXYQpkBIece6DUDIDAbqRB1Q85LFqRMExM+LTaayIIzSKN6/bp23wyozRI98FKzmpD8OOh6gh438Eg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CU+lIt2uva7UKXKnBs1OiCP1uunsHpfUorn1ZfAfK5Q=;
 b=qHTNMaKBGaFsLiKgLtFeHQg8zjGUgj0ICsLzoScl7ij+5PoifyATXYZx9emTBfRQbrxPL064C96HRA1HXM1yDAyon2nIwcBm0LLJ5BdmrMaHFkH7DSUwiNqzjYp8KKnAp5WInMdOV3z8JYkaRNAq17V15z6pwlef834Kz3qYfRMIwwCPtzH+7oDPFhADdNXG5ciC1dt5V9PKxO4xBisqs4TidIZpQrH4LASEjTqLqyRd73eApEzWHyGLMkhtZPGBOUHWWiH5Du95cWT8wgRyYaBy5LBfkDWGB5mxP5+93T8VhD6PdpymFpsoEFV1TCepDlqEI5AqxPYCPyOocSm7dg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiA
Date: Fri, 10 Sep 2021 14:01:57 +0000
Message-ID: <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
In-Reply-To: <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: cf8378c9-8315-400c-0312-08d974638dd9
x-ms-traffictypediagnostic: AM0PR03MB4305:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB430535927169433910E3D5E5E7D69@AM0PR03MB4305.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 EIyH6XzrvAHgxgtYiLkJq8STRukS7ay8DOP2bvKnc35q0I8Rb2mlxvrJO6L1LsWX760sPD56ftxu+NVliRn0MbPPHLX6bkBS//mm4C4V4o2GfDjWjz1Tli52kVXiCZoEcVytmQdp6ofp3qlsDsOs2y/fhqkvnHrOjfVg2jqsipZ9DMyAzmM2SUHkbzNlayqAcNI2XNC7lcbPWIdSlXNVP1Ie7545zGgBmNkFYHvZcB4NS1Q4m06Pohp10stoHo664s5Z0weJLgkvtDTtC+1o3R46cAaNr4UHSjn8xM5gXPV/UIXDbvM3YnWm9iO3p1Ybtv+/qwHs/6uyBZ261X9T5R/nfJhC2PdzjyA6RtRJqeDSWaMaW1G7sxuBvDNumoZje5EhulauwSJZ2TCLD+9qycybaMFSPxK1kCoXS80LGeID4XxjnSTrKP6Rh4EuDvrtzu+Qglhv1UvhEEsHeCgE6Bwjnqu2OBaPFt17fDIh/pB7SQ4tisgs3N59T+Sqf6CdraUwUCiWZ3aKSLdUCf/dkJW9fJZXwWWjbPniUo1FaW+yN5r7Z3moyQPDatDAlyoh963CasvFkbrOao6vyEblGs0ev0w+sgSsAVCJir8P9wrYaDu5uW2qlu3fGy6ho78vy43O2flyHnZHjVy+hLh2XE1RlHWlDzu/f8LQDAEG1kI0flKC3lbZ7gyj7adJ6PMNcbR9Do6bznTJTcSqQAs5YJlAoMdrbBlgiVlNc+h5YicBWcKHThRxGPgFW+n60Szk
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(39860400002)(376002)(346002)(366004)(6486002)(66946007)(2616005)(66476007)(55236004)(186003)(5660300002)(36756003)(316002)(26005)(86362001)(8936002)(6512007)(110136005)(64756008)(71200400001)(31686004)(76116006)(66446008)(54906003)(8676002)(53546011)(6506007)(66556008)(91956017)(478600001)(2906002)(38100700002)(38070700005)(4326008)(31696002)(83380400001)(122000001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?VFRvRG5UY2ZSM1JrL3B2VHU3L0hlajR5SFdBdW5mNXJpT1d1NUpVL0Zkd0N0?=
 =?utf-8?B?ZEh6M3BhL1ltVUhuQmJEMU1TQmZQbGF2bXFTZmZNN3ZmMnRWeWVDOExBdFRY?=
 =?utf-8?B?WHVyWkpGaGVLdVQxZ2hjZ3ZhbWxMUi90ZXY2cmY3bFdXL2pHWDdTdURRVlo5?=
 =?utf-8?B?OEkwUkZBQ2hSNmtYT0NzaUhPK2xiQ1FIc2RGV2NQWmVOVG02NFM0K00rMkZD?=
 =?utf-8?B?aVpFZm9tUit2ZEE0UElwTzRzM1psc3dIWlArN0didk8vakU0RjhJVXk5MjJo?=
 =?utf-8?B?L0NRU000ckF4bkRxN0xrVVRxdDA2MWgvWURWR2krbWxTUEo0WXk2eit1bi9U?=
 =?utf-8?B?ZVlqZ3FQbCtJOGZIR0FMTXl4UGdhQ0k1UFk2b0UyTWRlWlFBOVdpYng1Wk1K?=
 =?utf-8?B?UHZPcmt6Z2ZXRTBBeHF1c0xJM2p1RUQ2TngyMGczRFVXbGEwTnExRVNHSVlR?=
 =?utf-8?B?SmMxNnh2QklGa0Mydmx2NGNVUFcxd2wvemEraWsxS1g4SnB5d0s1WGRUNjJa?=
 =?utf-8?B?K0hEQ2FkamFxZkttcXhLMGRIZml1bTl1dFRpWU16VVB3c1lJQVE1OXhkZE5M?=
 =?utf-8?B?Ui9yRzIvd3lHTDRRa0lPd3QxZEh1TnVSZjVHTWhqcUdoRGFHZmJvUjUxcjZ2?=
 =?utf-8?B?MGpKczdNeGtoV1Vqa1Y0a2RRSkpXY09GVmZ6ZVdjYm1JT0hFVHVXWEx1Nk54?=
 =?utf-8?B?TEdzTG0xNTc0OHRGN0J4Vm1weG0wdUdTeW9HVnJQL1VCSTZqZUp6bkM5R2ND?=
 =?utf-8?B?WDdmWWhoV0xNME5zeUU5UGFJSS9vVGVNOFdzYm9mV1dzd2VueENGTFNYaTkz?=
 =?utf-8?B?QzF3ck1VbHI4NTFhQjdheEIvcWZaU2k0NmRxOFlBOWVMWTByejZZZnNQM3cx?=
 =?utf-8?B?VHhQeVVuZlA0eVdudldySDdRMlFva0lzeVhXQzc2RlBXd3NmRTlXVmN0emxE?=
 =?utf-8?B?U1Z0L1ZDVWNkMkRhK1ZGbStHODdac2VoZFpQT09RdlIwdFhqQWVvdVhsU1pB?=
 =?utf-8?B?S2dkN3RjNHpTdjhkd2tVbDZYSnY1enAwcWNxN2drZ2gydzQxWGdKT3QrZnlO?=
 =?utf-8?B?QStVaVVJMjJhNjM3Z3lIQUdQYnkyRFRBUjRjeXFLMUxFblF1d0FWam9TQWFy?=
 =?utf-8?B?anVlK1ZveVluMXFHeWhTTXMwVkpxRmR1eGpXUkVibHBkNitDTkE5WEJCaDdD?=
 =?utf-8?B?MjRDMElhRzR5VlRiemYvM0prTktiR3ZiN0k3Y29VWFJ1YmNFaTlpSkVsMWcv?=
 =?utf-8?B?ME9JL2dDVCtEdFcxdTlZdUVDYWxuK2lNbWR6Q2xSaEZSVDNRMEhTb2xkMzg0?=
 =?utf-8?B?b0VCNmc3ZVJ2d3lYenFjMFBtTlFKR21aNTNha2VwNUhtbDJnYVBFNzRxOFBU?=
 =?utf-8?B?QVNaSkZWRzlGcHdzeEtBQ2FBci9RNlV0aVkrYXFPa3B0MFFqNnpkVGVJM3FL?=
 =?utf-8?B?QVRSUmJIdFU5dEpxcTBKMFFna1o5T1V1dmIyRHZPbXFGanlxdUUvVFRCZ2ZG?=
 =?utf-8?B?eDU4WndoZWVhK3lTelFPbTl2ZG1oTGlYeUN2OFNDOGhsR2lsejh1ejNZYUh5?=
 =?utf-8?B?eS9zOFlHRHE1TlBVeHcydG1mQUZ0TFg3dGkwNmhXTVVoYmdKUmNpRjllR1E0?=
 =?utf-8?B?MCtFak1sSFdZSllFYlNDZDdxMFRpVS9QVnRvdXluTXB0enhuMUw4MDZaek5k?=
 =?utf-8?B?RTJPa2RPOVFQMzdKT1JqbDVhSVBRQlZqUFRmSkYzTVNGRXJZTnhhVi93UnQv?=
 =?utf-8?B?Uyt0UnBRNWRnekYvT1dYUmlNWlFvRFdYVktsMDJlUFo2TWtwclJtUERodGIw?=
 =?utf-8?B?b2tDbDdWRTRkdU0rZDAyUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <3E7C776A2AD0EA45AA99AF0A3C2E69EC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cf8378c9-8315-400c-0312-08d974638dd9
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 14:01:57.8327
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FDe3abXQ8IyzNuukeDbeWUxSNxT/hglfZashJtw9cN27aYM0a830HJPNiaDjNqBb/TvVyHGC6xYgMxpLWE+OpMbajI01hht4vTMPvTX07vpMEW6aIvudjemM9mwEr/sW
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4305
X-Proofpoint-GUID: 4u3ouQh2CPlNMZ1LK501c-TPDK21liSf
X-Proofpoint-ORIG-GUID: 4u3ouQh2CPlNMZ1LK501c-TPDK21liSf
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_04,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0
 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 mlxlogscore=999
 impostorscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109100080

DQpPbiAxMC4wOS4yMSAxNjoxOCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPg0KPg0KPiBPbiAxMC8w
OS8yMDIxIDEzOjM3LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEhpLCBKdWxp
ZW4hDQo+DQo+IEhpIE9sZWtzYW5kciwNCj4NCj4+IE9uIDA5LjA5LjIxIDIwOjU4LCBKdWxpZW4g
R3JhbGwgd3JvdGU6DQo+Pj4gT24gMDMvMDkvMjAyMSAwOTozMywgT2xla3NhbmRyIEFuZHJ1c2hj
aGVua28gd3JvdGU6DQo+Pj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4NCj4+Pj4gSG9zdCBicmlkZ2UgY29udHJv
bGxlcidzIEVDQU0gc3BhY2UgaXMgbWFwcGVkIGludG8gRG9tYWluLTAncyBwMm0sDQo+Pj4+IHRo
dXMgaXQgaXMgbm90IHBvc3NpYmxlIHRvIHRyYXAgdGhlIHNhbWUgZm9yIHZQQ0kgdmlhIE1NSU8g
aGFuZGxlcnMuDQo+Pj4+IEZvciB0aGlzIHRvIHdvcmsgd2UgbmVlZCB0byBub3QgbWFwIHRob3Nl
IHdoaWxlIGNvbnN0cnVjdGluZyB0aGUgZG9tYWluLg0KPj4+Pg0KPj4+PiBOb3RlLCB0aGF0IGR1
cmluZyBEb21haW4tMCBjcmVhdGlvbiB0aGVyZSBpcyBubyBwY2lfZGV2IHlldCBhbGxvY2F0ZWQg
Zm9yDQo+Pj4+IGhvc3QgYnJpZGdlcywgdGh1cyB3ZSBjYW5ub3QgbWF0Y2ggUENJIGhvc3QgYW5k
IGl0cyBhc3NvY2lhdGVkDQo+Pj4+IGJyaWRnZSBieSBTQkRGLiBVc2UgZHRfZGV2aWNlX25vZGUg
ZmllbGQgZm9yIGNoZWNrcyBpbnN0ZWFkLg0KPj4+Pg0KPj4+PiBTaWduZWQtb2ZmLWJ5OiBPbGVr
c2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+
Pj4+IC0tLQ0KPj4+PiDCoMKgIHhlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuY8KgwqDCoMKgwqDC
oMKgIHzCoCAzICsrKw0KPj4+PiDCoMKgIHhlbi9hcmNoL2FybS9wY2kvZWNhbS5jwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCB8IDE3ICsrKysrKysrKysrKysrKysrDQo+Pj4+IMKgwqAgeGVuL2FyY2gv
YXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYyB8IDIyICsrKysrKysrKysrKysrKysrKysrKysNCj4+
Pj4gwqDCoCB4ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5owqDCoMKgwqDCoMKgwqDCoMKgIHwgMTIg
KysrKysrKysrKysrDQo+Pj4+IMKgwqAgNCBmaWxlcyBjaGFuZ2VkLCA1NCBpbnNlcnRpb25zKCsp
DQo+Pj4+DQo+Pj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMgYi94
ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMNCj4+Pj4gaW5kZXggZGE0MjdmMzk5NzExLi43NmY1
YjUxMzI4MGMgMTAwNjQ0DQo+Pj4+IC0tLSBhL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0K
Pj4+PiArKysgYi94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMNCj4+Pj4gQEAgLTEyNTcsNiAr
MTI1Nyw5IEBAIHN0YXRpYyBpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3Ry
dWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0NCj4+
Pj4gwqDCoMKgwqDCoMKgIH0NCj4+Pj4gwqDCoCArwqDCoMKgIGlmICggbmVlZF9tYXBwaW5nICYm
IChkZXZpY2VfZ2V0X2NsYXNzKGRldikgPT0gREVWSUNFX1BDSSkgKSA+ICvCoMKgwqDCoMKgwqDC
oCBuZWVkX21hcHBpbmcgPSBwY2lfaG9zdF9icmlkZ2VfbmVlZF9wMm1fbWFwcGluZyhkLCBkZXYs
DQo+Pj4gYWRkciwgbGVuKTsNCj4+Pg0KPj4+IEFGQUlDVCwgd2l0aCBkZXZpY2VfZ2V0X2NsYXNz
KGRldiksIHlvdSBrbm93IHdoZXRoZXIgdGhlIGhvc3RicmlkZ2UgaXMgdXNlZCBieSBYZW4uIFRo
ZXJlZm9yZSwgSSB3b3VsZCBleHBlY3QgdGhhdCB3ZSBkb24ndCB3YW50IHRvIG1hcCBhbGwgdGhl
IHJlZ2lvbnMgb2YgdGhlIGhvc3RicmlkZ2VzIGluIGRvbTAgKGluY2x1ZGluZyB0aGUgQkFScyku
DQo+Pj4NCj4+PiBDYW4geW91IGNsYXJpZnkgaXQ/DQo+PiBXZSBvbmx5IHdhbnQgdG8gdHJhcCBF
Q0FNLCBub3QgTU1JT3MgYW5kIGFueSBvdGhlciBtZW1vcnkgcmVnaW9ucyBhcyB0aGUgYnJpZGdl
IGlzDQo+Pg0KPj4gaW5pdGlhbGl6ZWQgYW5kIHVzZWQgYnkgRG9tYWluLTAgY29tcGxldGVseS4g
DQo+DQo+IFdoYXQgZG8geW91IG1lYW4gYnkgInVzZWQgYnkgRG9tYWluLTAgY29tcGxldGVseSI/
IFRoZSBob3N0YnJpZGdlIGlzIG93bmVkIGJ5IFhlbiBzbyBJIGRvbid0IHRoaW5rIHdlIGNhbiBs
ZXQgZG9tMCBhY2Nlc3MgYW55IE1NSU8gcmVnaW9ucyBieQ0KPiBkZWZhdWx0Lg0KDQpOb3cgaXQn
cyBteSB0aW1lIHRvIGFzayB3aHkgZG8geW91IHRoaW5rIFhlbiBvd25zIHRoZSBicmlkZ2U/DQoN
CkFsbCB0aGUgYnJpZGdlcyBhcmUgcGFzc2VkIHRvIERvbWFpbi0wLCBhcmUgZnVsbHkgdmlzaWJs
ZSB0byBEb21haW4tMCwgaW5pdGlhbGl6ZWQgaW4gRG9tYWluLTAuDQoNCkVudW1lcmF0aW9uIGV0
Yy4gaXMgZG9uZSBpbiBEb21haW4tMC4gU28gaG93IGNvbWVzIHRoYXQgWGVuIG93bnMgdGhlIGJy
aWRnZT8gSW4gd2hhdCB3YXkgaXQgZG9lcz8NCg0KWGVuIGp1c3QgYWNjZXNzZXMgdGhlIEVDQU0g
d2hlbiBpdCBuZWVkcyBpdCwgYnV0IHRoYXQncyBpdC4gWGVuIHRyYXBzIEVDQU0gYWNjZXNzIC0g
dGhhdCBpcyB0cnVlLg0KDQpCdXQgaXQgaW4gbm8gd2F5IHVzZXMgTU1JT3MgZXRjLiBvZiB0aGUg
YnJpZGdlIC0gdGhvc2UgdW5kZXIgZGlyZWN0IGNvbnRyb2wgb2YgRG9tYWluLTANCg0KPg0KPiBJ
biBwYXJ0aWN1bGFyLCB3ZSBtYXkgd2FudCB0byBoaWRlIGEgZGV2aWNlIGZyb20gZG9tMCBmb3Ig
c2VjdXJpdHkgcmVhc29ucy4gVGhpcyBpcyBub3QgZ29pbmcgdG8gYmUgcG9zc2libGUgaWYgeW91
IG1hcCBieSBkZWZhdWx0IGV2ZXJ5dGhpbmcgdG8gZG9tMC4NCg0KVGhlbiB0aGUgYnJpZGdlIG1v
c3QgcHJvYmFibHkgd2lsbCBiZWNvbWUgdW51c2FibGUgYXMgd2UgZG8gbm90IGhhdmUgcmVsZXZh
bnQgZHJpdmVycyBpbiBYZW4uDQoNCkF0IGJlc3Qgd2UgbWF5IHJlbHkgb24gdGhlIGZpcm13YXJl
IGRvaW5nIHRoZSBpbml0aWFsaXphdGlvbiBmb3IgdXMsIHRoZW4geWVzLCB3ZSBjYW4gY29udHJv
bCBhbiBFQ0FNIGJyaWRnZSBpbiBYZW4uDQoNCkJ1dCB0aGlzIGlzIG5vdCB0aGUgY2FzZSBub3c6
IHdlIHJlbHkgb24gRG9tYWluLTAgdG8gaW5pdGlhbGl6ZSBhbmQgc2V0dXAgdGhlIGJyaWRnZQ0K
DQo+DQo+IEluc3RlYWQsIHRoZSBCQVJzIHNob3VsZCBiZSBtYXBwZWQgb24gZGVtYW5kIHdoZW4g
ZG9tMCB3aGVuIHdlIHRyYXAgYWNjZXNzIHRvIHRoZSBjb25maWd1cmF0aW9uIHNwYWNlLg0KPg0K
PiBGb3Igb3RoZXIgcmVnaW9ucywgY291bGQgeW91IHByb3ZpZGUgYW4gZXhhbXBsZSBvZiB3aGF0
IHlvdSBhcmUgcmVmZXJyaW5nIHRvbz8NClJlZ2lzdGVycyBvZiB0aGUgYnJpZGdlIGZvciBleGFt
cGxlLCBhbGwgd2hhdCBpcyBsaXN0ZWQgaW4gInJlZyIgcHJvcGVydHkNCj4NCj4+Pj4gKw0KPj4+
PiArwqDCoMKgIC8qDQo+Pj4+ICvCoMKgwqDCoCAqIFdlIGRvIG5vdCB3YW50IEVDQU0gYWRkcmVz
cyBzcGFjZSB0byBiZSBtYXBwZWQgaW4gZG9tYWluJ3MgcDJtLA0KPj4+PiArwqDCoMKgwqAgKiBz
byB3ZSBjYW4gdHJhcCBhY2Nlc3MgdG8gaXQuDQo+Pj4+ICvCoMKgwqDCoCAqLw0KPj4+PiArwqDC
oMKgIHJldHVybiBjZmctPnBoeXNfYWRkciAhPSBhZGRyOw0KPj4+PiArfQ0KPj4+PiArDQo+Pj4+
IMKgwqAgLyogRUNBTSBvcHMgKi8NCj4+Pj4gwqDCoCBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1fb3Bz
IHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0KPj4+PiDCoMKgwqDCoMKgwqAgLmJ1c19zaGlmdMKg
ID0gMjAsDQo+Pj4+IEBAIC02MCw2ICs3Niw3IEBAIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMg
cGNpX2dlbmVyaWNfZWNhbV9vcHMgPSB7DQo+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIC5yZWFk
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gcGNpX2dlbmVyaWNfY29uZmln
X3JlYWQsDQo+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIC53cml0ZcKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgPSBwY2lfZ2VuZXJpY19jb25maWdfd3JpdGUsDQo+Pj4+IMKgwqDC
oMKgwqDCoMKgwqDCoMKgIC5yZWdpc3Rlcl9tbWlvX2hhbmRsZXLCoCA9IHBjaV9lY2FtX3JlZ2lz
dGVyX21taW9faGFuZGxlciwNCj4+Pj4gK8KgwqDCoMKgwqDCoMKgIC5uZWVkX3AybV9tYXBwaW5n
wqDCoMKgwqDCoMKgID0gcGNpX2VjYW1fbmVlZF9wMm1fbWFwcGluZywNCj4+Pj4gwqDCoMKgwqDC
oMKgIH0NCj4+Pj4gwqDCoCB9Ow0KPj4+PiDCoMKgIGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0v
cGNpL3BjaS1ob3N0LWNvbW1vbi5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24u
Yw0KPj4+PiBpbmRleCBhODkxMTJiZmJiN2MuLmMwNGJlNjM2NDUyZCAxMDA2NDQNCj4+Pj4gLS0t
IGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KPj4+PiArKysgYi94ZW4vYXJj
aC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+Pj4+IEBAIC0zMzQsNiArMzM0LDI4IEBAIGlu
dCBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoc3RydWN0IGRvbWFpbiAqZCwNCj4+Pj4gwqDCoMKg
wqDCoMKgIH0NCj4+Pj4gwqDCoMKgwqDCoMKgIHJldHVybiAwOw0KPj4+PiDCoMKgIH0NCj4+Pj4g
Kw0KPj4+PiArYm9vbCBwY2lfaG9zdF9icmlkZ2VfbmVlZF9wMm1fbWFwcGluZyhzdHJ1Y3QgZG9t
YWluICpkLA0KPj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgY29uc3Qgc3RydWN0IGR0X2RldmljZV9u
b2RlICpub2RlLA0KPj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWludDY0
X3QgbGVuKQ0KPj4+PiArew0KPj4+PiArwqDCoMKgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJy
aWRnZTsNCj4+Pj4gKw0KPj4+PiArwqDCoMKgIGxpc3RfZm9yX2VhY2hfZW50cnkoIGJyaWRnZSwg
JnBjaV9ob3N0X2JyaWRnZXMsIG5vZGUgKQ0KPj4+PiArwqDCoMKgIHsNCj4+Pj4gK8KgwqDCoMKg
wqDCoMKgIGlmICggYnJpZGdlLT5kdF9ub2RlICE9IG5vZGUgKQ0KPj4+PiArwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBjb250aW51ZTsNCj4+Pj4gKw0KPj4+PiArwqDCoMKgwqDCoMKgwqAgaWYgKCAh
YnJpZGdlLT5vcHMtPm5lZWRfcDJtX21hcHBpbmcgKQ0KPj4+PiArwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCByZXR1cm4gdHJ1ZTsNCj4+Pj4gKw0KPj4+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIGJy
aWRnZS0+b3BzLT5uZWVkX3AybV9tYXBwaW5nKGQsIGJyaWRnZSwgYWRkciwgbGVuKTsNCj4+Pj4g
K8KgwqDCoCB9DQo+Pj4+ICvCoMKgwqAgcHJpbnRrKFhFTkxPR19FUlIgIlVuYWJsZSB0byBmaW5k
IFBDSSBicmlkZ2UgZm9yICVzIHNlZ21lbnQgJWQsIGFkZHIgJWx4XG4iLA0KPj4+PiArwqDCoMKg
wqDCoMKgwqDCoMKgwqAgbm9kZS0+ZnVsbF9uYW1lLCBicmlkZ2UtPnNlZ21lbnQsIGFkZHIpOw0K
Pj4+PiArwqDCoMKgIHJldHVybiB0cnVlOw0KPj4+PiArfQ0KPj4+DQo+Pj4gSWYgeW91IHJlYWxs
eSBuZWVkIHRvIG1hcCB0aGUgaG9zdGJyaWRnZXMsIHRoZW4gSSB3b3VsZCBzdWdnZXN0IHRvIGRl
ZmVyIHRoZSBQMk0gbWFwcGluZ3MgZm9yIGFsbCBvZiB0aGVtIGFuZCB0aGVuIHdhbGsgYWxsIHRo
ZSBicmlkZ2UgaW4gb25lIGdvIHRvIGRvIHRoZSBtYXBwaW5ncy4NCj4+Pg0KPj4+IFRoaXMgd291
bGQgYXZvaWQgZ29pbmcgdGhyb3VnaCB0aGUgYnJpZGdlcyBldmVyeSB0aW1lIGR1cmluZyBzZXR1
cC4NCj4+DQo+PiBXZWxsLCB0aGlzIGNhbiBiZSBkb25lLCBidXQ6IG15IGltcGxlbWVudGF0aW9u
IHByZXZlbnRzIG1hcHBpbmdzIGFuZCB3aGF0DQo+Pg0KPj4geW91IHN1Z2dlc3Qgd2lsbCByZXF1
aXJlIHVubWFwcGluZy4gU28sIHRoZSBjb3N0IGluIG15IHNvbHV0aW9uIGlzIHdlIG5lZWQNCj4+
DQo+PiB0byBnbyBvdmVyIGFsbCB0aGUgYnJpZGdlcyAodW50aWwgd2UgZmluZCB0aGUgb25lIHdl
IG5lZWQpIGFuZCBpbiB3aGF0IHlvdQ0KPj4NCj4+IHN1Z2dlc3Qgd2UnbGwgbmVlZCB0byB1bm1h
cCB3aGF0IHdlIGhhdmUganVzdCBtYXBwZWQuDQo+Pg0KPj4gSSB0aGluayBwcmV2ZW50aW5nIHVu
bmVlZGVkIG1hcHBpbmdzIGlzIGNoZWFwZXIgdGhhbiB1bm1hcHBpbmcgYWZ0ZXJ3YXJkcy4NCj4N
Cj4gSSB0aGluayB5b3UgbWlzdW5kZXJ0b29kIHdoYXQgSSBhbSBzdWdnZXN0aW5nLiBXaGF0IEkg
c2FpZCBpcyB5b3UgY291bGQgZGVmZXIgdGhlIG1hcHBpbmdzIChJT1cgbm90IGRvIHRoZSBtYXBw
aW5nKSB1bnRpbCBsYXRlciBmb3IgdGhlIGhvc3RicmlkZ2VzLg0KDQpGb3IgZWFjaCBkZXZpY2Ug
dHJlZSBkZXZpY2Ugd2UgY2FsbA0KDQpzdGF0aWMgaW50IF9faW5pdCBtYXBfcmFuZ2VfdG9fZG9t
YWluKGNvbnN0IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KIMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IHU2NCBhZGRyLCB1NjQgbGVuLA0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHZvaWQgKmRhdGEpDQp3aGlj
aCB3aWxsIGNhbGwNCg0KIMKgwqDCoMKgwqDCoMKgIHJlcyA9IG1hcF9yZWdpb25zX3AybXQoZCwN
ClNvLCBFQ0FNIHdpbGwgYmUgbWFwcGVkIGFuZCB0aGVuIHdlJ2xsIG5lZWQgdG8gdW5tYXAgaXQN
Cg0KPiBBbmQgdGhlbiB5b3UgY2FuIHdhbGsgYWxsIHRoZSBob3N0YnJpZGdlcyB0byBkZWNpZGUg
aG93IHRvIG1hcCB0aGVtLg0KV2UgZG9uJ3Qgd2FudCB0byBtYXAgRUNBTSwgd2Ugd2FudCB0byB0
cmFwIGl0DQo+DQo+IFRoZSByZWdpb25zIHdpbGwgb25seSBtYXBwZWQgb25jZSBhbmQgbmV2ZXIg
YmUgdW5tYXBwZWQuDQo+DQo+IENoZWVycywNCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:03:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:03:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184477.333129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh7z-0000Fk-Db; Fri, 10 Sep 2021 14:03:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184477.333129; Fri, 10 Sep 2021 14:03:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOh7z-0000Fc-AZ; Fri, 10 Sep 2021 14:03:39 +0000
Received: by outflank-mailman (input) for mailman id 184477;
 Fri, 10 Sep 2021 14:03:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOh7x-0000FW-TQ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:03:37 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOh7w-0008LG-Us; Fri, 10 Sep 2021 14:03:36 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOh7w-0000HJ-Oz; Fri, 10 Sep 2021 14:03:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=CQkUy1jyoCfZBRkpSONeHxayYSAoRhIwcbH7AUT4PdU=; b=tSTo0TZufbsoR6rOHN00vjOIcu
	dL/EDE7S9VUkcc7ibU6l6Y8fcZ5A5sZ2av6yTzks2/dCoC3Cz+65qPTSqoX5eq0zKSlBMxoU4z36t
	wBFAK+nftuw6KVIeHxf0TYoXY+VD60SYRPivuDOmFCGSmjM5mrT/KYAQJysVHxupSYdc=;
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
To: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
 <f8c0e296-a942-dd95-6dfc-dc8569386bd9@xen.org>
 <a96ea57e-ab7b-c9ce-02c1-2061a43ee72b@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <25e17c13-4774-8523-cc41-1fd9117e5349@xen.org>
Date: Fri, 10 Sep 2021 15:03:34 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <a96ea57e-ab7b-c9ce-02c1-2061a43ee72b@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 10/09/2021 14:51, Juergen Gross wrote:
> On 10.09.21 15:49, Julien Grall wrote:
>>
>>
>> On 10/09/2021 14:46, Juergen Gross wrote:
>>> On 10.09.21 15:22, Jan Beulich wrote:
>>>> On 09.09.2021 08:27, Juergen Gross wrote:
>>>>> - Addition of a new stable hypercall ("get domain state") returning 
>>>>> the
>>>>>     following information:
>>>>>     + domid of a domain having the bit set in above bitmap
>>>>>     + state of that domain (existing, dying)
>>>>>     + sequence count of that domain
>>>>>     The related bit is reset in the bitmap as a side effect of the 
>>>>> call.
>>>>
>>>> What I'd like us to consider up front is whether xenstored is going
>>>> to remain only entity interested in this kind of information. The
>>>> entire design looks to leverage that there's only a single consumer
>>>> in the system.
>>>
>>> Right. I'm just writing some RFC patches, and I have coded this
>>> interface to be usable only for the domain having VIRQ_DOM_EXC
>>> registered.
>>>
>>> The alternative (IMO) would have been to expose the domain-state
>>> bitmap to Xenstore (and/or other interested parties).
>>
>> If we do that, let's do in such way that guest_*_bit() helpers are not 
>> necessary because they are a massive hammer on Arm. This means the 
>> bitmap would have to be read-only for the domain.
> 
> I strongly prefer my current approach not needing a shared memory page.

That wasn't a way to say I prefer the shared bitmap. I wanted to point 
out that if this is the chosen approach then we ought to avoid using 
guest_*_bit().

I will be away for the next 4 weeks, so I preferred to mention it now to 
avoid the surprise of having yet another interface using guest_*_bit() 
when I am back :).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:18:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:18:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184485.333141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhMC-00024Y-Oe; Fri, 10 Sep 2021 14:18:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184485.333141; Fri, 10 Sep 2021 14:18:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhMC-00024R-KT; Fri, 10 Sep 2021 14:18:20 +0000
Received: by outflank-mailman (input) for mailman id 184485;
 Fri, 10 Sep 2021 14:18:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOhMB-00024L-9H
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:18:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOhM8-0000BI-Pc; Fri, 10 Sep 2021 14:18:16 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOhM8-0001fv-JQ; Fri, 10 Sep 2021 14:18:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=PhhESR2Ndyq1w/a2Acswdzb3pAtJx1nPY4XNO6wxuyA=; b=JdG7jyLC1z+GGSPkdnUlbZpqfM
	L69Wov15MHXbksI8v9yvJBAsqwqJLYHDHVKp3uflwkRO2ptm5wC+cF142fe17Alq6z2nGvLa2dl3V
	Q7JKanSiQka8nySX7tAqcCPi3NHseoBxkVD2hI/NOQeZKGViXudZbSPW47gTH1aE0Bpk=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
Date: Fri, 10 Sep 2021 15:18:14 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 15:01, Oleksandr Andrushchenko wrote:
> 
> On 10.09.21 16:18, Julien Grall wrote:
>>
>>
>> On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>
>> Hi Oleksandr,
>>
>>> On 09.09.21 20:58, Julien Grall wrote:
>>>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> Host bridge controller's ECAM space is mapped into Domain-0's p2m,
>>>>> thus it is not possible to trap the same for vPCI via MMIO handlers.
>>>>> For this to work we need to not map those while constructing the domain.
>>>>>
>>>>> Note, that during Domain-0 creation there is no pci_dev yet allocated for
>>>>> host bridges, thus we cannot match PCI host and its associated
>>>>> bridge by SBDF. Use dt_device_node field for checks instead.
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>> ---
>>>>>     xen/arch/arm/domain_build.c        |  3 +++
>>>>>     xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
>>>>>     xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
>>>>>     xen/include/asm-arm/pci.h          | 12 ++++++++++++
>>>>>     4 files changed, 54 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>>> index da427f399711..76f5b513280c 100644
>>>>> --- a/xen/arch/arm/domain_build.c
>>>>> +++ b/xen/arch/arm/domain_build.c
>>>>> @@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
>>>>>             }
>>>>>         }
>>>>>     +    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI) ) > +        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev,
>>>> addr, len);
>>>>
>>>> AFAICT, with device_get_class(dev), you know whether the hostbridge is used by Xen. Therefore, I would expect that we don't want to map all the regions of the hostbridges in dom0 (including the BARs).
>>>>
>>>> Can you clarify it?
>>> We only want to trap ECAM, not MMIOs and any other memory regions as the bridge is
>>>
>>> initialized and used by Domain-0 completely.
>>
>> What do you mean by "used by Domain-0 completely"? The hostbridge is owned by Xen so I don't think we can let dom0 access any MMIO regions by
>> default.
> 
> Now it's my time to ask why do you think Xen owns the bridge?

Because nothing in this series indicates either way. So I assumed this 
should be owned by Xen because it will drive it.

 From what you wrote below, it sounds like this is not the case. I think 
you want to have a design document sent along the series so we can 
easily know what's the expected design and validate that the code match 
the agreement.

There was already a design document sent a few months ago. So you may 
want to refresh it (if needed) and send it again with this series.

> 
> All the bridges are passed to Domain-0, are fully visible to Domain-0, initialized in Domain-0.
> 
> Enumeration etc. is done in Domain-0. So how comes that Xen owns the bridge? In what way it does?
> 
> Xen just accesses the ECAM when it needs it, but that's it. Xen traps ECAM access - that is true.
> 
> But it in no way uses MMIOs etc. of the bridge - those under direct control of Domain-0
> 
>>
>> In particular, we may want to hide a device from dom0 for security reasons. This is not going to be possible if you map by default everything to dom0.
> 
> Then the bridge most probably will become unusable as we do not have relevant drivers in Xen.
> 
> At best we may rely on the firmware doing the initialization for us, then yes, we can control an ECAM bridge in Xen.
> 
> But this is not the case now: we rely on Domain-0 to initialize and setup the bridge
> 
>>
>> Instead, the BARs should be mapped on demand when dom0 when we trap access to the configuration space.
>>
>> For other regions, could you provide an example of what you are referring too?
> Registers of the bridge for example, all what is listed in "reg" property
>>
>>>>> +
>>>>> +    /*
>>>>> +     * We do not want ECAM address space to be mapped in domain's p2m,
>>>>> +     * so we can trap access to it.
>>>>> +     */
>>>>> +    return cfg->phys_addr != addr;
>>>>> +}
>>>>> +
>>>>>     /* ECAM ops */
>>>>>     const struct pci_ecam_ops pci_generic_ecam_ops = {
>>>>>         .bus_shift  = 20,
>>>>> @@ -60,6 +76,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>>>>>             .read                   = pci_generic_config_read,
>>>>>             .write                  = pci_generic_config_write,
>>>>>             .register_mmio_handler  = pci_ecam_register_mmio_handler,
>>>>> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>>>>>         }
>>>>>     };
>>>>>     diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
>>>>> index a89112bfbb7c..c04be636452d 100644
>>>>> --- a/xen/arch/arm/pci/pci-host-common.c
>>>>> +++ b/xen/arch/arm/pci/pci-host-common.c
>>>>> @@ -334,6 +334,28 @@ int pci_host_iterate_bridges(struct domain *d,
>>>>>         }
>>>>>         return 0;
>>>>>     }
>>>>> +
>>>>> +bool pci_host_bridge_need_p2m_mapping(struct domain *d,
>>>>> +                                      const struct dt_device_node *node,
>>>>> +                                      uint64_t addr, uint64_t len)
>>>>> +{
>>>>> +    struct pci_host_bridge *bridge;
>>>>> +
>>>>> +    list_for_each_entry( bridge, &pci_host_bridges, node )
>>>>> +    {
>>>>> +        if ( bridge->dt_node != node )
>>>>> +            continue;
>>>>> +
>>>>> +        if ( !bridge->ops->need_p2m_mapping )
>>>>> +            return true;
>>>>> +
>>>>> +        return bridge->ops->need_p2m_mapping(d, bridge, addr, len);
>>>>> +    }
>>>>> +    printk(XENLOG_ERR "Unable to find PCI bridge for %s segment %d, addr %lx\n",
>>>>> +           node->full_name, bridge->segment, addr);
>>>>> +    return true;
>>>>> +}
>>>>
>>>> If you really need to map the hostbridges, then I would suggest to defer the P2M mappings for all of them and then walk all the bridge in one go to do the mappings.
>>>>
>>>> This would avoid going through the bridges every time during setup.
>>>
>>> Well, this can be done, but: my implementation prevents mappings and what
>>>
>>> you suggest will require unmapping. So, the cost in my solution is we need
>>>
>>> to go over all the bridges (until we find the one we need) and in what you
>>>
>>> suggest we'll need to unmap what we have just mapped.
>>>
>>> I think preventing unneeded mappings is cheaper than unmapping afterwards.
>>
>> I think you misundertood what I am suggesting. What I said is you could defer the mappings (IOW not do the mapping) until later for the hostbridges.
> 
> For each device tree device we call
> 
> static int __init map_range_to_domain(const struct dt_device_node *dev,
>                                         u64 addr, u64 len,
>                                         void *data)
> which will call
> 
>           res = map_regions_p2mt(d,
> So, ECAM will be mapped and then we'll need to unmap it

Well yes in the current code. But I specifically wrote "defer" as in 
adding a check "if hostbridge then return 0".

> 
>> And then you can walk all the hostbridges to decide how to map them.
> We don't want to map ECAM, we want to trap it

I know that... But as you wrote above, you want to map other regions of 
the hostbridge. So if you defer *all* the mappings for the hostbridge, 
then you can walk the hostbridges and check decide which regions should 
be mapped.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:38:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:38:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184495.333155 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhfm-00052H-FU; Fri, 10 Sep 2021 14:38:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184495.333155; Fri, 10 Sep 2021 14:38:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhfm-00052A-Bi; Fri, 10 Sep 2021 14:38:34 +0000
Received: by outflank-mailman (input) for mailman id 184495;
 Fri, 10 Sep 2021 14:38:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOhfl-000524-B3
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:38:33 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f6a11cf-0bc3-430f-8d38-68df7e31faf2;
 Fri, 10 Sep 2021 14:38:32 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ADhr7r007242; 
 Fri, 10 Sep 2021 14:38:29 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b08ebr6sy-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 14:38:29 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6882.eurprd03.prod.outlook.com (2603:10a6:20b:283::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 10 Sep
 2021 14:38:18 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 14:38:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f6a11cf-0bc3-430f-8d38-68df7e31faf2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gGdxlPU4yzrr6ZF3rl2eQEvoK+tdVCD/st6CBuEYRSyF/348w5w/+mFEH7TE/s4cHevCoD8eUDPsYu5APeHHc5D4xgu/N6ra6o6idhnx6s6e6ihf22TCLh43p98/xpwUT8/1nwoE3V4/L6mLHIxULCzbcalPAL4eDxlv4mSri9N0f/D8wpQWmAUjx0q1bEi7FH11Z3uKKBZsjOx3JRb51nQIk0KvB9EahD5q4T9BqAkvlJ+bCRtp9tqa1WromnG/h5q+FGAKMKfVKRKJ6U2hI/M7DKMltzDUDq1V5GOvCcHAWL1PErp4pbfdFwjZS0Mw9j03/f7wiu+Q//S4VwibNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2s9lAzxIwaxbTgLX1SzEKlDX5hXwILM8bu2ZRILhYJA=;
 b=OMiy6lP1acndZqDnX88C/QS0heeNFpWvGWmgSAJCn18j3FNIQMV8fl63gAm3ZOfo8LfCn1ys6MdY+sZ/uen54ElwvaqkuodM4VBjmkNgpavwDRkMKlvXxGxufhAvO0KOcsSwrKZQbPLnRoAjuCWfFFIc0l+iez/DZDpg9HyQ8KFN6jDEKKlQIj/nmawYE4Xod3pFRU1O/agYa4ysw3FWsK3kywZwayHKZfkCNjUlovCjfPMWxdXq975yprkdZCrMy58vZEvpjrxr46SLyNHQT7Zd7QmWHbl0ODNieCkEbsQE+h9w2emDwY0ZBie9SM+1EWCzDO007yUNM5NOEFMKOA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2s9lAzxIwaxbTgLX1SzEKlDX5hXwILM8bu2ZRILhYJA=;
 b=vv4FlgVTL4/YBvr05G9n9o01v/0Im+ne1T8FrDnO7PxR3uNWqR6NM9CpZeV18wSFAaLqZcIQbzsXEHS9msQoDbYfH/HK8EnWnvN5lq7PrkwDSHkAhPmRzvZirlY6CU6CsvO8nCtvsbMj1S/UHdyCOih8GpYayMoefk4nvlDQDoHRhLSQG29tX8qVel8MBt1oENrynTay7CJ/K/4rQEUd+jXzRL1wSBOOpbSmTmhjIny0uItGgOVvI7eW0JWKJo/X/Yfa0F44Y/xh3C+Etceo7EflHujKi89PGy9NNszKG9+XQUruulfDnSkmmPhSQx/s24rM+MyE+fG0LCHwvZ//5w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAAEjQCAAAWagA==
Date: Fri, 10 Sep 2021 14:38:17 +0000
Message-ID: <3c3c253c-4af7-ea25-5f73-a0f7319dab50@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
In-Reply-To: <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c1ef3677-68dc-4bb5-443d-08d97468a136
x-ms-traffictypediagnostic: AM9PR03MB6882:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB688254913D77F04C47959246E7D69@AM9PR03MB6882.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 eHnjFcW45+iLf11ahlQIp1Er00sDRDUrOazCFX8ntX93Hi28v7NIQN/vZegYOpkx28JTxJDZCrj3isWg8vbUAoSXvwqlepg7Wefq4F6VLE4uGyCtB/O+bVjDimjX/SOSuxTOog6kQj0dFFuarwGQPs1F/nLvJHovA6PldfBdGSDYG/7q6JWvD89exxJ+9SZ2tGAqJ63vFie6T7hkr6fg0pCe/mTFK6VhpEIFKlVKZugl8tnjIWwHQBtJQ9hBrf2Qgd0DdMbNOvRUsiip0LzrcrHJV0tGlI7h+EdgI1afBYsTuUnvTny2fimm9+itaxDY9AJ7AP5+s4iJCNTyxIJ/Jo9C6ZPj6Zd5yccepHm+4rcw74khzLW5zLpBT1sWMWyjslqk67jIOL3MmlNJkhyI4stUg69lD1LMJQotRuLhpjHfXxfT5wBKzWHtRMzaYmGqd7szKsKtz/qFeKclp92QdP8Fv8Y3r7kuWMcGN79+Yisw+wnIGZ/DXaUQofqL3lwTzhpCVY/gS9g43lEyPwqykAD2PN/1puJrPmlPFH3a+Jr+TeyJKoPgV810kiCOUQbirRqDESWhZzkjFbFNgwvAWo2IWx6zVPSJkIJPRw/ew3J7GdJP6UmohNzsw4ixjFW0gAEXuMRorwXE0sz0yL0iL3akEXqACJW5EAQ4e04aKZMFHAVe5uOdR5cq0I2y0cSxTXUahSKAiKz/HqvX/NXtOrgzQUKm5Z6thHvG1QqiE6ziRHt5bOjWdO7gxa7xAQVC2d4aq40ZND/WSGhAFaL2NGgG3PpkKKRQSkEmzr40gXbQM3xaj7NNZnfzCUw196Ge
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(136003)(396003)(39860400002)(346002)(376002)(8676002)(76116006)(186003)(6512007)(66446008)(64756008)(2616005)(38100700002)(71200400001)(6486002)(122000001)(966005)(66946007)(2906002)(316002)(54906003)(4326008)(38070700005)(66476007)(31686004)(83380400001)(6506007)(8936002)(478600001)(86362001)(110136005)(26005)(55236004)(66556008)(31696002)(36756003)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?cVNoL2hva0M2YzVQV0NQeEV6dFlnWEorR05CdGlaQWJ6bjRUZ1VrQ0IxYSsr?=
 =?utf-8?B?M3o3UDJzQk9RbmdvYmRkbmtaZWoya1RPbnc0bzRCNDJnQ1dTMWF3WjJ5VGZs?=
 =?utf-8?B?UkxPbTRmck9ybWJJTDE5Sm11dHAzSGhSOFFIblROd0tIOWV3VzU5WkRnSGFG?=
 =?utf-8?B?L09zdzdKZUhzL0dpSnFwWGt1bUtVYzVnb3VnKzgxRTVQS1RtNXVUd2NUazdD?=
 =?utf-8?B?dEM4akd6M3Yrc1l5L05wOGpOUjlYQXlERnB6YWRieWQ5VmdNSnBUV2tpSTVI?=
 =?utf-8?B?MEg3c0hxQ05kR0syRFA0VHZja2VRUkxZNFJHL2xEcWV5KzI4TEtHTWcxUHFE?=
 =?utf-8?B?MmJyT2p1VGRzTVowclQxdEZJai9icW9vTk9JTjhvc0ZCRVpVTFRVY0IwOCtG?=
 =?utf-8?B?ZWtxTzJiZlBJdnczb3d6NU1SQWtsaHFLNFNQRWd4R0RsRXc0YWpiWnNVb2N0?=
 =?utf-8?B?c3hOL0NjeXRyUnVqekgyMVoxMXFFMU1yZjU0MlhJOGgzemk1dnlOSzB2UENF?=
 =?utf-8?B?cnpTNEQrS2xpMllDY2pZODVvYnpremNib3hqRUxhSVhTT0VoTmU1blRjdUpn?=
 =?utf-8?B?WkhhV21qSHB4MEh0QkYxZXltbEJZTnZxNWN2OXA5MVhzdVpOb1gya2YvTEwx?=
 =?utf-8?B?Z1EvUGo0R0ZDWmp5NVp2UnRQeFgrTTZXenJOa09STkhjK1AweGRKOXhtVHlV?=
 =?utf-8?B?bFdhOTZnQi9CUld5U3NjQThUMjN6U3psYnZLTy9hYk91bnlIb2lRd3JrVG11?=
 =?utf-8?B?ZDNFWVhuWE5aRGc2aGoxYk8xY3E3ZjR1aENxRk5sU082Zkk4eGgva3p1aTdB?=
 =?utf-8?B?ZUw5S0ZISXlvc2hIMTdVN01hSDN2OWJEc2NHUFBzVUxEamhyVk43MExyM1pC?=
 =?utf-8?B?aTlFWW1CcytDcnBWMXlmejZuZTUwMFdvdVBpM0UramRaRFIvMkMxejg1UlNm?=
 =?utf-8?B?UTYvV244YlVLZmpTVlNJaXJpWFVFUUtJT2lsczNhR1hLWHZzcUdhVEVGQVZj?=
 =?utf-8?B?TkxvVS95eHdnNmlvd3FDNG9EV2trSkc3UGljNDM5YjJ3d0h5VEU0VW01emg5?=
 =?utf-8?B?VFU2UkY0RmViK0I3Q01LcXcxWU5RVzBGa1JMelNFU3RVODdObmF4N2tSSkI5?=
 =?utf-8?B?cDYzbEFscnJ2R0hlNnA5cUV3VG1uVStZWkp1VEZnaGlqNDZiRnVyME1EbGs4?=
 =?utf-8?B?a2w0VEFWclIwTms1OW9MTURJYkE2TUdGN1JkQzdpWXpDNXMrRXFHcHV5Qkhw?=
 =?utf-8?B?bDQvcGlBaktoM2NxZVpzVStGaFY5dW1HSDFnVHpUeXJDM1dWSDFOKzlYR20x?=
 =?utf-8?B?Rlo0U1pkT201U1p3dDhDelpDU3h6YkxCQ2ZkTi9NV3lGcVE1dFgvV1EreG9G?=
 =?utf-8?B?UjFMdWxCT2tKa2tQeWIvUExHTHloRjljMFBwUUNqdzdtWWt3N0VZR2RkcVFk?=
 =?utf-8?B?ZVBXbHlYTGVSYWdrNlF2OTRTczhmN2szb0JYSmNVdnJZdHNhMG1YaGhVeHlI?=
 =?utf-8?B?UVJYQzU1Zzl5MHF0ZnhzRWJkbzA5RldRQ0pCeWxHNTFUQjlUWEVkVGpZK3pQ?=
 =?utf-8?B?TGE1WWpSc1hMdktSL09QaFpwTzk2emdwOGxranRBSnh6cVY1QnRoajhVakpy?=
 =?utf-8?B?eERkaUJ3NWhKM1V6UnM5S2xocFZDS1o5SUhvb3FXTVRNRXBzcW1QQSt2M1lw?=
 =?utf-8?B?M2ZHaTlLZ1lmYStHbUh2cE1yZTQ2MENxMlRBQ0tEWEN4ZnZ0UGpJR0pKaEJF?=
 =?utf-8?B?ZTlra3YvOHpLT0R5cFlnYlJkN1VGK1RWUHljV1RXbzhLeWF3OHJnMXpUZUUz?=
 =?utf-8?B?NnVpN3NkaVJBMmpCMG1sdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D0C5D925BC4BE246B78973C164E79930@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c1ef3677-68dc-4bb5-443d-08d97468a136
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 14:38:17.8350
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: KWMktcjhkn/9UDtVS9wJTpO8PJdnGfe9c7OYJOUMQ2x0lPOFajRTUPTy5DkKKu5v19BKumEDGtW0ohd13bGB7HExJlR6sxpWXPrQoVKhUjPaA5xS6ksUL/2rzftnLQv7
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6882
X-Proofpoint-GUID: DzURJVdPqcZ0qNS5nYH0ird1oyV4xUWc
X-Proofpoint-ORIG-GUID: DzURJVdPqcZ0qNS5nYH0ird1oyV4xUWc
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_06,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 malwarescore=0 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0
 mlxlogscore=999 priorityscore=1501 phishscore=0 clxscore=1015
 lowpriorityscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109100083

W3NuaXBdDQoNCg0KPj4+IFdoYXQgZG8geW91IG1lYW4gYnkgInVzZWQgYnkgRG9tYWluLTAgY29t
cGxldGVseSI/IFRoZSBob3N0YnJpZGdlIGlzIG93bmVkIGJ5IFhlbiBzbyBJIGRvbid0IHRoaW5r
IHdlIGNhbiBsZXQgZG9tMCBhY2Nlc3MgYW55IE1NSU8gcmVnaW9ucyBieQ0KPj4+IGRlZmF1bHQu
DQo+Pg0KPj4gTm93IGl0J3MgbXkgdGltZSB0byBhc2sgd2h5IGRvIHlvdSB0aGluayBYZW4gb3du
cyB0aGUgYnJpZGdlPw0KPg0KPiBCZWNhdXNlIG5vdGhpbmcgaW4gdGhpcyBzZXJpZXMgaW5kaWNh
dGVzIGVpdGhlciB3YXkuIFNvIEkgYXNzdW1lZCB0aGlzIHNob3VsZCBiZSBvd25lZCBieSBYZW4g
YmVjYXVzZSBpdCB3aWxsIGRyaXZlIGl0Lg0KPg0KPiBGcm9tIHdoYXQgeW91IHdyb3RlIGJlbG93
LCBpdCBzb3VuZHMgbGlrZSB0aGlzIGlzIG5vdCB0aGUgY2FzZS4gSSB0aGluayB5b3Ugd2FudCB0
byBoYXZlIGEgZGVzaWduIGRvY3VtZW50IHNlbnQgYWxvbmcgdGhlIHNlcmllcyBzbyB3ZSBjYW4g
ZWFzaWx5IGtub3cgd2hhdCdzIHRoZSBleHBlY3RlZCBkZXNpZ24gYW5kIHZhbGlkYXRlIHRoYXQg
dGhlIGNvZGUgbWF0Y2ggdGhlIGFncmVlbWVudC4NCj4NCj4gVGhlcmUgd2FzIGFscmVhZHkgYSBk
ZXNpZ24gZG9jdW1lbnQgc2VudCBhIGZldyBtb250aHMgYWdvLiBTbyB5b3UgbWF5IHdhbnQgdG8g
cmVmcmVzaCBpdCAoaWYgbmVlZGVkKSBhbmQgc2VuZCBpdCBhZ2FpbiB3aXRoIHRoaXMgc2VyaWVz
Lg0KPg0KUGxlYXNlIHNlZSBbMV0gd2hpY2ggaXMgdGhlIGRlc2lnbiBkb2N1bWVudCB3ZSB1c2Ug
dG8gaW1wbGVtZW50IFBDSSBwYXNzdGhyb3VnaCBvbiBBcm0uDQoNCkZvciB5b3VyIGNvbnZlbmll
bmNlOg0KDQoiDQoNCiMgUHJvYmxlbSBzdGF0ZW1lbnQ6DQpbc25pcF0NCg0KT25seSBEb20wIGFu
ZCBYZW4gd2lsbCBoYXZlIGFjY2VzcyB0byB0aGUgcmVhbCBQQ0kgYnVzLOKAiyBndWVzdCB3aWxs
IGhhdmUgYQ0KZGlyZWN0IGFjY2VzcyB0byB0aGUgYXNzaWduZWQgZGV2aWNlIGl0c2VsZuKAiy4g
SU9NRU0gbWVtb3J5IHdpbGwgYmUgbWFwcGVkIHRvDQp0aGUgZ3Vlc3Qg4oCLYW5kIGludGVycnVw
dCB3aWxsIGJlIHJlZGlyZWN0ZWQgdG8gdGhlIGd1ZXN0LiBTTU1VIGhhcyB0byBiZQ0KY29uZmln
dXJlZCBjb3JyZWN0bHkgdG8gaGF2ZSBETUEgdHJhbnNhY3Rpb24uIg0KDQoiDQoNCiMgRGlzY292
ZXJpbmcgUENJIEhvc3QgQnJpZGdlIGluIFhFTjoNCg0KSW4gb3JkZXIgdG8gc3VwcG9ydCB0aGUg
UENJIHBhc3N0aHJvdWdoIFhFTiBzaG91bGQgYmUgYXdhcmUgb2YgYWxsIHRoZSBQQ0kgaG9zdA0K
YnJpZGdlcyBhdmFpbGFibGUgb24gdGhlIHN5c3RlbSBhbmQgc2hvdWxkIGJlIGFibGUgdG8gYWNj
ZXNzIHRoZSBQQ0kNCmNvbmZpZ3VyYXRpb24gc3BhY2UuIEVDQU0gY29uZmlndXJhdGlvbiBhY2Nl
c3MgaXMgc3VwcG9ydGVkIGFzIG9mIG5vdy4gWEVODQpkdXJpbmcgYm9vdCB3aWxsIHJlYWQgdGhl
IFBDSSBkZXZpY2UgdHJlZSBub2RlIOKAnHJlZ+KAnSBwcm9wZXJ0eSBhbmQgd2lsbCBtYXAgdGhl
DQpFQ0FNIHNwYWNlIHRvIHRoZSBYRU4gbWVtb3J5IHVzaW5nIHRoZSDigJxpb3JlbWFwX25vY2Fj
aGUgKCnigJ0gZnVuY3Rpb24uDQoNCltzbmlwXQ0KDQpXaGVuIERvbTAgdHJpZXMgdG8gYWNjZXNz
IHRoZSBQQ0kgY29uZmlnIHNwYWNlIG9mIHRoZSBkZXZpY2UsIFhFTiB3aWxsIGZpbmQgdGhlDQpj
b3JyZXNwb25kaW5nIGhvc3QgYnJpZGdlIGJhc2VkIG9uIHNlZ21lbnQgbnVtYmVyIGFuZCBhY2Nl
c3MgdGhlIGNvcnJlc3BvbmRpbmcNCmNvbmZpZyBzcGFjZSBhc3NpZ25lZCB0byB0aGF0IGJyaWRn
ZS4NCg0KTGltaXRhdGlvbjoNCiogT25seSBQQ0kgRUNBTSBjb25maWd1cmF0aW9uIHNwYWNlIGFj
Y2VzcyBpcyBzdXBwb3J0ZWQuDQoqIERldmljZSB0cmVlIGJpbmRpbmcgaXMgc3VwcG9ydGVkIGFz
IG9mIG5vdywgQUNQSSBpcyBub3Qgc3VwcG9ydGVkLg0KKiBOZWVkIHRvIHBvcnQgdGhlIFBDSSBo
b3N0IGJyaWRnZSBhY2Nlc3MgY29kZSB0byBYRU4gdG8gYWNjZXNzIHRoZQ0KY29uZmlndXJhdGlv
biBzcGFjZSAoZ2VuZXJpYyBvbmUgd29ya3MgYnV0IGxvdHMgb2YgcGxhdGZvcm1zIHdpbGwgcmVx
dWlyZWQNCnNvbWUgc3BlY2lmaWMgY29kZSBvciBxdWlya3MpLg0KDQoiDQoNClVuZm9ydHVuYXRl
bHkgdGhlIGRvY3VtZW50IGhhZCBub3QgYmVlbiB1cGRhdGVkIHNpbmNlIHRoZW4sIGJ1dCB0aGUg
cXVlc3Rpb24NCg0KYmVpbmcgZGlzY3Vzc2VkIGhhcyBub3QgY2hhbmdlZCBpbiB0aGUgZGVzaWdu
OiBEb21haW4tMCBoYXMgZnVsbCBhY2Nlc3MgdG8gdGhlIGJyaWRnZSwNCg0KWGVuIHRyYXBzIEVD
QU0uIChJIGFtIHRha2luZyBkb20wbGVzcyBhc2lkZSBhcyB0aGF0IHdhcyBub3QgdGhlIHRhcmdl
dCBmb3IgdGhlIGZpcnN0IHBoYXNlKQ0KDQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0KDQpbMV0g
aHR0cHM6Ly9saXN0cy54ZW5wcm9qZWN0Lm9yZy9hcmNoaXZlcy9odG1sL3hlbi1kZXZlbC8yMDIw
LTA3L21zZzAwNzc3Lmh0bWwNCg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:51:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:51:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184502.333165 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhrp-0007Uu-KR; Fri, 10 Sep 2021 14:51:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184502.333165; Fri, 10 Sep 2021 14:51:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOhrp-0007Un-HO; Fri, 10 Sep 2021 14:51:01 +0000
Received: by outflank-mailman (input) for mailman id 184502;
 Fri, 10 Sep 2021 14:51:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOhro-0007Uh-BI
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:51:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOhro-0000jn-8p
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:51:00 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOhro-0004tS-7t
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:51:00 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOhrh-0002pD-U5; Fri, 10 Sep 2021 15:50:53 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=pq0J7Mq0GeXOrUXc8cqB0PEiSKid71V0BuwGQP9rSKw=; b=sAGmRavtW8F/wUyC7ptFzAzhse
	YAhJg9TXT8lVOWcWaTMh8qjrB929pDyrWzTFP7m3/Hmpzr+mz8KMHEOYnBuapk5ILyvHV5FA06jzw
	SAcOwNPRzx0egTaauGTjfqtlCk/GJWOJNcylqVXSe+V9o1o355YR3rKgxjhD8fznufPM=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24891.28877.607703.662095@mariner.uk.xensource.com>
Date: Fri, 10 Sep 2021 15:50:53 +0100
To: Jan Beulich <jbeulich@suse.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Kevin Stefanov <kevin.stefanov@citrix.com>,
    Xen-devel <xen-devel@lists.xenproject.org>,
    Wei Liu <wl@xen.org>,
    Anthony PERARD <anthony.perard@citrix.com>
Subject: Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS [and 1 more messages]
Newsgroups: chiark.mail.xen.devel
In-Reply-To: <bbde0f13-a1a5-53ed-82eb-704cd5d43178@citrix.com>,
	<affca979-4082-e825-dede-b27b5574631b@suse.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
	<20210909163441.44418-3-kevin.stefanov@citrix.com>
	<affca979-4082-e825-dede-b27b5574631b@suse.com>
	<20210909163441.44418-2-kevin.stefanov@citrix.com>
	<bbde0f13-a1a5-53ed-82eb-704cd5d43178@citrix.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Andrew Cooper writes ("Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS"):
> The current code is clearly wrong, but happens to work correctly in
> hvmloader because FACS is the first table written and it starts on a
> page boundary. The logic does not work correctly when libxl passes a
> buffer which doesn't start on a page boundary.
> 
> As a consequence, I'm not sure what to suggest as a Fixes tag here,
> except "the logic has been wrong for 15 years - patch everything".

Jan Beulich writes ("Re: [PATCH 2/3] tools/libxl: Correctly aligned buffer for ACPI tables"):
> But the buffers obtained via libxl__malloc() have no association with
> guest address space layout (or at least they aren't supposed to have).
> That's solely determined by mem_alloc(). I think it is a bug of
> mem_alloc() to determine padding from alloc_currp; it should
> rather/also maintain a current address in guest address space (e.g.
> by having separate alloc_currp and alloc_currv). Not doing so leaves
> us prone to encountering the same issue again down the road. For
> example if higher than page alignment was requested from somewhere in
> libacpi.

I think the two of you are saying roughly the same thing here ?

There was a question about how (and if) this should be backported.
I'm afraid I don't quite follow all the implications, but I doubt that
a a substantial overhaul as described would be a good idea for a
backport.  What is the impact for backports, and can we have a more
targeted fix there ?  Are, in fact, Kevin's original patches, suitable
to fix the issue for stable trees ?

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 14:52:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 14:52:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184511.333177 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOht2-0008Pk-Ur; Fri, 10 Sep 2021 14:52:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184511.333177; Fri, 10 Sep 2021 14:52:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOht2-0008Pd-RZ; Fri, 10 Sep 2021 14:52:16 +0000
Received: by outflank-mailman (input) for mailman id 184511;
 Fri, 10 Sep 2021 14:52:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOht1-0008PX-RP
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 14:52:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOhsz-0000lM-O8; Fri, 10 Sep 2021 14:52:13 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOhsz-0004xE-I2; Fri, 10 Sep 2021 14:52:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=mPuB3hdtYyFKmGS5TtNtW6v1boLy/B6UMFdek9kZDBk=; b=ffVIPntJ/u8W8hHaRnPr8ReTld
	LqXE6+dY02MewXqcgJGk0HDRdJt+2SU+s+g/1IueTEywduha+XF6X/6zflh8+7H4NU8PU5PFBOMrL
	p99SIQ2YHW5zV/KdAk+FYD5jFhfrKoyRUzTK2qmjA2FvlMW9lnQUDOcP1+PtUo2NhxQY=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
 <3c3c253c-4af7-ea25-5f73-a0f7319dab50@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <34abe11e-c2f9-50ce-fb79-836bfcfb11e0@xen.org>
Date: Fri, 10 Sep 2021 15:52:11 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <3c3c253c-4af7-ea25-5f73-a0f7319dab50@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit



On 10/09/2021 15:38, Oleksandr Andrushchenko wrote:
> [snip]
> 
> 
>>>> What do you mean by "used by Domain-0 completely"? The hostbridge is owned by Xen so I don't think we can let dom0 access any MMIO regions by
>>>> default.
>>>
>>> Now it's my time to ask why do you think Xen owns the bridge?
>>
>> Because nothing in this series indicates either way. So I assumed this should be owned by Xen because it will drive it.
>>
>>  From what you wrote below, it sounds like this is not the case. I think you want to have a design document sent along the series so we can easily know what's the expected design and validate that the code match the agreement.
>>
>> There was already a design document sent a few months ago. So you may want to refresh it (if needed) and send it again with this series.
>>
> Please see [1] which is the design document we use to implement PCI passthrough on Arm.

Thank. Can you make sure to include at least in a link in your cover 
letter next time?

> 
> For your convenience:
> 
> "
> 
> # Problem statement:
> [snip]
> 
> Only Dom0 and Xen will have access to the real PCI bus,​ guest will have a
> direct access to the assigned device itself​. IOMEM memory will be mapped to
> the guest ​and interrupt will be redirected to the guest. SMMU has to be
> configured correctly to have DMA transaction."
> 
> "
> 
> # Discovering PCI Host Bridge in XEN:
> 
> In order to support the PCI passthrough XEN should be aware of all the PCI host
> bridges available on the system and should be able to access the PCI
> configuration space. ECAM configuration access is supported as of now. XEN
> during boot will read the PCI device tree node “reg” property and will map the
> ECAM space to the XEN memory using the “ioremap_nocache ()” function.
> 
> [snip]
> 
> When Dom0 tries to access the PCI config space of the device, XEN will find the
> corresponding host bridge based on segment number and access the corresponding
> config space assigned to that bridge.
> 
> Limitation:
> * Only PCI ECAM configuration space access is supported.
> * Device tree binding is supported as of now, ACPI is not supported.
> * Need to port the PCI host bridge access code to XEN to access the
> configuration space (generic one works but lots of platforms will required
> some specific code or quirks).
> 
> "
> 
> Unfortunately the document had not been updated since then, but the question
> 
> being discussed has not changed in the design: Domain-0 has full access to the bridge,
> 
> Xen traps ECAM. (I am taking dom0less aside as that was not the target for the first phase)

Having an update design document is quite important. This will allow 
reviewer to comment easily on overall approach and speed up the review 
as we can match to the agreed approach.

So can this please be updated and sent along the work?

In addition to that, it feels to me that the commit message should 
contain a summary of what you just pasted as this helps understanding 
the goal and approach of this patch.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:01:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:01:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184521.333188 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi21-0001kc-SQ; Fri, 10 Sep 2021 15:01:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184521.333188; Fri, 10 Sep 2021 15:01:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi21-0001kV-PJ; Fri, 10 Sep 2021 15:01:33 +0000
Received: by outflank-mailman (input) for mailman id 184521;
 Fri, 10 Sep 2021 15:01:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p6NG=OA=epam.com=prvs=9887a95815=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mOi20-0001kP-7R
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:01:32 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fb660664-1247-11ec-b250-12813bfff9fa;
 Fri, 10 Sep 2021 15:01:31 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18AExGmg010270; 
 Fri, 10 Sep 2021 15:01:25 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b09kjg09b-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 10 Sep 2021 15:01:24 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3472.eurprd03.prod.outlook.com (2603:10a6:803:25::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.26; Fri, 10 Sep
 2021 15:01:20 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021
 15:01:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb660664-1247-11ec-b250-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kbb7fdK0k5JOMPjhSLtIRUdhGOAsT33+roZ2I0NqlE5CjX/Z8hrN9bJOzk12vGdgmUYZ0f0KdACUBXm+w4b1LxKchNaiDQVy0iD4Pusa8mJE1GAfzHziwK0NUYS+liu5Aq9FZHJFuzstE81hnIDdhDeYDgorEnzhCuhVRyknSsQBqXehLYkM1jECGXjtYlF/9W0VHt0USc14z77PFxdVs4EKN0smeHke36uCabuolywubWXC8XTIg6ekuDsKGyZplsqDjiNwdGMnbfa2LHRjgwiLnikNTGg/ZlOpBdcC8zd2nY+7BC5OystSueIZFvpf8PwK550juBX7JvAp/DGLsA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UsBQJVE9lvsH/VOk2QeEH2QTh48KUIp1w73HLPSBwT8=;
 b=HTN4oLr2RpTOQDHCPo+0B4RvaqmS/y7PfuAUiKI72Zo1whgm9U3Q4fLWwfK14T99rsD+gFRwYr2DqG+ITk1RwTbmxwP+g+DLTMdTLdmH9x1Y3TKzGj5OO8fCnHN09xwBeU4lPwIEHtwz4/6oGQ2IRPjvdYNQ/M2/VtV2cnB+D9TrklGkt1XDZnHpn/exzVhaCTsRmDYcccRzamaqJoKFmXhP9e7QLTfj92k/fnt3QMRQC3VbJhMhl2IqUj+4u/aDLItBqHUbufnamyXyn6j+Fb7E1Zqsuj274tf4CtCGPg/dMYfNg0ZOunFqqPmJE9IRlqKUrOgYGiwpgkwj4DSCrw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UsBQJVE9lvsH/VOk2QeEH2QTh48KUIp1w73HLPSBwT8=;
 b=H/EvRLThRnuW8fKJgure95rIsLmomy0pG5KXKbvQ7j7D1SGIZQsxJezAP/a9sQicsFEUeIMT6heNp32RudRWL7hVh1qwjyEMJsgDVKIsMszk0Drg6T11FGQ5+yJ9mF1/PdU0a5T9NGT9zuFT4jK0ZZYf7g5Xp/jcbdJNwqmovMa9WhSf53i59WNtwZKq+5wYhb2aeXtrkBdZi3WFtNjyYpgm41bxwGWKnyclbEncOqA/KeYUsTWPolWrS30PkWFl+q3HJ7lltRVMYx6xOTyr06lgwr16sHRRmjCpaiyR60DLDLCTX8t4Q1ED82rxxyADCEM1nxR+1emIhbTOlA9x1g==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "sstabellini@kernel.org" <sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAAEjQCAAAWagIAAA+KAgAACjwA=
Date: Fri, 10 Sep 2021 15:01:20 +0000
Message-ID: <f1ff6186-c483-3de1-cbdf-ac96a6009d15@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
 <3c3c253c-4af7-ea25-5f73-a0f7319dab50@epam.com>
 <34abe11e-c2f9-50ce-fb79-836bfcfb11e0@xen.org>
In-Reply-To: <34abe11e-c2f9-50ce-fb79-836bfcfb11e0@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ddda9c2e-73eb-4558-febe-08d9746bd96b
x-ms-traffictypediagnostic: VI1PR0302MB3472:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB347236CFF6F4AF4EEC359678E7D69@VI1PR0302MB3472.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 2/uy8CDI0ZBwLctUS4oVT0cgSyQiz79JZvE+NfMJstvPjfIKtItxFnrHh9VyV0hyBZb1vOSiqHgYqPAd+xZjZuQOsmD+zn/K1p+o88097rNCup7tJ3PxI1Zw4/HsZyY2WFabBZdwp4UBeiL8AnDEboEU9X30Gl2U0Zy315maeI6bLt7xopLSsWbxxLxNYSXFtqtejAOZZmN7vC66/Hbx7Yky9BRy8AAo6WtmWF9UG9apOcbiRORI3+IeS/ayZfu0C3Ffyrc3OVjcqcVVEyUnxt3hUHcZsbsgDxjhsdzx6XwJ01fLY5lRq0kK7/hIBS8lGunnb//hvoVhTi0HE43uzsDQmehb6fn9rlJlcz4HhTEkTijbCinEbVGTZKWx+431Mzbft7kGxHHDEeN8i3iJtuWl+u5gO9jtZoNeKlpbH68bmARTcZ52RWy4S+7ISMGJTfadYIdTlYq9uTAf68kEfJZcUgWpFRNTukS2myVevuZSxmgCO/ncCTVkMASPfJ3gENEnl8e8OTgEaVIDUwU6CvThVUPGoOSQzA/W+Pd5hNM4fNUSBi4o4luncOiSIKBeqtprg9xmAk53lV/yIcmBSQ23VR2A8g+WLRRUqgzz3eBgCg3PI5zrUjkpQ4bNjEnolyDxF2J/PU3zrBHcY6Cs0Y2WHV1M5pIrNGquUTKMN5Hi9HO+GjdMPjIRDX4SaHxiS9jFyVD/X4fPDHG6siSnOnYEZWcIL0SLLow9sHl0kaiCV2CVvvaf+3aDwNcO1gbq
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(346002)(136003)(366004)(71200400001)(8676002)(6512007)(5660300002)(31686004)(66946007)(478600001)(86362001)(76116006)(26005)(36756003)(316002)(122000001)(186003)(54906003)(6486002)(38100700002)(64756008)(66476007)(83380400001)(55236004)(6506007)(66556008)(8936002)(53546011)(110136005)(2616005)(38070700005)(66446008)(31696002)(91956017)(2906002)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Uy9ST24xcHZvMXJWd3NxZndOb3BpODJNYVZRSHRZcW5GSGpMZWRrVEExQmNR?=
 =?utf-8?B?YzJDMVpCdUo1S3p3alNtVGMzVXAvdUd0eTgwNm9LTFJVTFJPZDdjWTlSMzBm?=
 =?utf-8?B?RGxQZWpUYndTVGNIZlNmb3R0VCs5c2REWlprZUJEaTg4d25ZU2xaT3ZjWXMz?=
 =?utf-8?B?T3BacDA4OXpic3lVRjBUNHg2YnBjSzd6UWh0MExSSEZ3Q2F0d1JHcVMxa1Zh?=
 =?utf-8?B?S0dMT0tTYUhGSDVsZFRieDlPMmZ5eEZ3dCs1bmpMZjBBWWxFVmc4eUcrTVBt?=
 =?utf-8?B?WGJCWWpuYzh0UlRSdVplcFdqSDdYcjJIWHJGRVdSVHlPVDdYOUdtUktOSUdk?=
 =?utf-8?B?UjZoNGNKMmNJelIvQnpJWUhNRW9PQWo0TFRqR001QnV3bkhjM0oxWFpGdWgy?=
 =?utf-8?B?MWM4b2hnNUVmNW0rcGpWMm8vUlJTTk9udE5xREp1RmRYNVRYWFhPdGVzY29O?=
 =?utf-8?B?amRySVNXbEZmbkhwU0NWajhiQ1poYm1zUHNFc0V0ZVhYSjhqTkJ1ak56bUVu?=
 =?utf-8?B?eXBFZURrV2hxcGZ5NDlPbmJ3b210bGJFVXYwYXJwTkdSZjEvbktFWmdHQmtL?=
 =?utf-8?B?ZHhqZVlwMG5JVjVNTzQ0S0tKWmVmc3NjaFBGeVNJUlRMV2NZMHI4RU1nVVEz?=
 =?utf-8?B?Z3ArUWs1Y0pyR2t0M0xQQnhPdVNTd0ZrcDJQSzkvNm51aEkrWlNSK0N3dHNZ?=
 =?utf-8?B?MnM3L0J1c0RYU1Q4a0NWVDd1VmhHNjdCdzhwSzJEcm9WU1BwSDE2Skl1bUpx?=
 =?utf-8?B?REFqdmN1MFVpVGgrTWI5NXVyQ1JUakFMdVpWZkhVcmRmU29jazc2T0I2M2lG?=
 =?utf-8?B?VjB6a1VMeDRiNEwrbE03YVZqRTZvbFprMTU4bWtYMFNSN1NxQ01PdGZiK20x?=
 =?utf-8?B?TlhXYXE4VTF1TEdqM2E4bERlVlprblpkdWxUQ21INXNUZmNJSnJpMjJ5cEtN?=
 =?utf-8?B?RjhGMUt3SU1JWGVSMm9lV0RtVTNMNUpTQ0Jma2p6S1hsbUREQ2x5UWtaUmRF?=
 =?utf-8?B?MkFvTEZRNkZWdlBVZmNKaWIrLzlqbklYTWY2WUoyeDdEWEE5TFk3amFWNUM4?=
 =?utf-8?B?VzJUS0Iwd1hDZTZ0SGE5STJLcjJRZHpKSlVUczljTkZLOVcwYWR1QkV6YTkw?=
 =?utf-8?B?VFFERFZwWlFjZk44M3JpV3J3YlE4Z3V1Y0NCTHJPMmsxbjFReWdydVdPY1Ro?=
 =?utf-8?B?SHFTYXNLSVIvZVdCSmxBS1JmQThRM0VsdCtJZ295REJrK3VzM1MvOFVSQVd1?=
 =?utf-8?B?R3lEVjZlNWppRkJWY2UxVEZlRmpaSFFlTHBHQ3dVUVN1L0JVS2dGWXRFbjdB?=
 =?utf-8?B?Z3piMkI5OUc5WTF5Y0RVZkpPNW56WVc5Z0tiMFhRbFNmY2E1aWFqbE9yM050?=
 =?utf-8?B?QXRjNFVHSlEvQ1hyNVVFYnFLeWZNV2hBTGl0M0djKzk1RTdoelJYRmgxbEJB?=
 =?utf-8?B?NWlhSit3bVZCT3hYRVp6WnJaVmFOaHEvaUJxUVhLcEtnbTdaT3dTQlBtaUpB?=
 =?utf-8?B?ZVlsVmpxYmtmL0hqVnMvSUwzR3dZa1dWUmwvWVpObGQzME9aY0pHMUc0OWU5?=
 =?utf-8?B?MzZPc0xFb0Z3WUVhZUpzeEZ0bHJrcEdtRlA3YXpJMmtuOVNHaHdJQkJKUjU1?=
 =?utf-8?B?TE43MC9ITWIxdmZmdGpHMytoNG1iTy9CYXFpMnp2WDFzVHFrQllvREs4UjM2?=
 =?utf-8?B?dStTS1djdXhRU0NFVkc1YVFHdzcycWtBaWNrOEV0Qi84eWgvbmc3dFhEblJ5?=
 =?utf-8?B?N01CNlhCdDJ1SXdlK2pvVnJCRXQxTXV6NkdhRFAySDAzaURyRXdMRzBmVWU4?=
 =?utf-8?B?UXVvcjFtU3hVa1Y5MFplUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <5B0565DB6AB1224DA65E13D67C440489@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ddda9c2e-73eb-4558-febe-08d9746bd96b
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2021 15:01:20.6437
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kZsL1nHeEHeeOd63aRz3suyVTZezMB79/2AWds/mnuT67LP/JG/UVxO736MnoLIYsGIn7kOq2E2Gx4DjS0Zj0CWRd0wHsLjpteN2CXeYjiMVBByCccA6Jq63coPnvbe1
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3472
X-Proofpoint-GUID: P2JD_tC0atxjC4MXtOUrouHomTu-lHnN
X-Proofpoint-ORIG-GUID: P2JD_tC0atxjC4MXtOUrouHomTu-lHnN
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-10_06,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 suspectscore=0 bulkscore=0 adultscore=0 phishscore=0 mlxlogscore=999
 lowpriorityscore=0 priorityscore=1501 mlxscore=0 spamscore=0
 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109100086

DQpPbiAxMC4wOS4yMSAxNzo1MiwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPg0KPg0KPiBPbiAxMC8w
OS8yMDIxIDE1OjM4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IFtzbmlwXQ0K
Pj4NCj4+DQo+Pj4+PiBXaGF0IGRvIHlvdSBtZWFuIGJ5ICJ1c2VkIGJ5IERvbWFpbi0wIGNvbXBs
ZXRlbHkiPyBUaGUgaG9zdGJyaWRnZSBpcyBvd25lZCBieSBYZW4gc28gSSBkb24ndCB0aGluayB3
ZSBjYW4gbGV0IGRvbTAgYWNjZXNzIGFueSBNTUlPIHJlZ2lvbnMgYnkNCj4+Pj4+IGRlZmF1bHQu
DQo+Pj4+DQo+Pj4+IE5vdyBpdCdzIG15IHRpbWUgdG8gYXNrIHdoeSBkbyB5b3UgdGhpbmsgWGVu
IG93bnMgdGhlIGJyaWRnZT8NCj4+Pg0KPj4+IEJlY2F1c2Ugbm90aGluZyBpbiB0aGlzIHNlcmll
cyBpbmRpY2F0ZXMgZWl0aGVyIHdheS4gU28gSSBhc3N1bWVkIHRoaXMgc2hvdWxkIGJlIG93bmVk
IGJ5IFhlbiBiZWNhdXNlIGl0IHdpbGwgZHJpdmUgaXQuDQo+Pj4NCj4+PiDCoEZyb20gd2hhdCB5
b3Ugd3JvdGUgYmVsb3csIGl0IHNvdW5kcyBsaWtlIHRoaXMgaXMgbm90IHRoZSBjYXNlLiBJIHRo
aW5rIHlvdSB3YW50IHRvIGhhdmUgYSBkZXNpZ24gZG9jdW1lbnQgc2VudCBhbG9uZyB0aGUgc2Vy
aWVzIHNvIHdlIGNhbiBlYXNpbHkga25vdyB3aGF0J3MgdGhlIGV4cGVjdGVkIGRlc2lnbiBhbmQg
dmFsaWRhdGUgdGhhdCB0aGUgY29kZSBtYXRjaCB0aGUgYWdyZWVtZW50Lg0KPj4+DQo+Pj4gVGhl
cmUgd2FzIGFscmVhZHkgYSBkZXNpZ24gZG9jdW1lbnQgc2VudCBhIGZldyBtb250aHMgYWdvLiBT
byB5b3UgbWF5IHdhbnQgdG8gcmVmcmVzaCBpdCAoaWYgbmVlZGVkKSBhbmQgc2VuZCBpdCBhZ2Fp
biB3aXRoIHRoaXMgc2VyaWVzLg0KPj4+DQo+PiBQbGVhc2Ugc2VlIFsxXSB3aGljaCBpcyB0aGUg
ZGVzaWduIGRvY3VtZW50IHdlIHVzZSB0byBpbXBsZW1lbnQgUENJIHBhc3N0aHJvdWdoIG9uIEFy
bS4NCj4NCj4gVGhhbmsuIENhbiB5b3UgbWFrZSBzdXJlIHRvIGluY2x1ZGUgYXQgbGVhc3QgaW4g
YSBsaW5rIGluIHlvdXIgY292ZXIgbGV0dGVyIG5leHQgdGltZT8NCkkgd2lsbCB1cGRhdGUgdGhl
IGNvbW1pdCBtZXNzYWdlIHRvIGhhdmUgbW9yZSBkZXNjcmlwdGlvbiBvbiB0aGUgZGVzaWduIGFz
cGVjdHMNCj4NCj4+DQo+PiBGb3IgeW91ciBjb252ZW5pZW5jZToNCj4+DQo+PiAiDQo+Pg0KPj4g
IyBQcm9ibGVtIHN0YXRlbWVudDoNCj4+IFtzbmlwXQ0KPj4NCj4+IE9ubHkgRG9tMCBhbmQgWGVu
IHdpbGwgaGF2ZSBhY2Nlc3MgdG8gdGhlIHJlYWwgUENJIGJ1cyzigIsgZ3Vlc3Qgd2lsbCBoYXZl
IGENCj4+IGRpcmVjdCBhY2Nlc3MgdG8gdGhlIGFzc2lnbmVkIGRldmljZSBpdHNlbGbigIsuIElP
TUVNIG1lbW9yeSB3aWxsIGJlIG1hcHBlZCB0bw0KPj4gdGhlIGd1ZXN0IOKAi2FuZCBpbnRlcnJ1
cHQgd2lsbCBiZSByZWRpcmVjdGVkIHRvIHRoZSBndWVzdC4gU01NVSBoYXMgdG8gYmUNCj4+IGNv
bmZpZ3VyZWQgY29ycmVjdGx5IHRvIGhhdmUgRE1BIHRyYW5zYWN0aW9uLiINCj4+DQo+PiAiDQo+
Pg0KPj4gIyBEaXNjb3ZlcmluZyBQQ0kgSG9zdCBCcmlkZ2UgaW4gWEVOOg0KPj4NCj4+IEluIG9y
ZGVyIHRvIHN1cHBvcnQgdGhlIFBDSSBwYXNzdGhyb3VnaCBYRU4gc2hvdWxkIGJlIGF3YXJlIG9m
IGFsbCB0aGUgUENJIGhvc3QNCj4+IGJyaWRnZXMgYXZhaWxhYmxlIG9uIHRoZSBzeXN0ZW0gYW5k
IHNob3VsZCBiZSBhYmxlIHRvIGFjY2VzcyB0aGUgUENJDQo+PiBjb25maWd1cmF0aW9uIHNwYWNl
LiBFQ0FNIGNvbmZpZ3VyYXRpb24gYWNjZXNzIGlzIHN1cHBvcnRlZCBhcyBvZiBub3cuIFhFTg0K
Pj4gZHVyaW5nIGJvb3Qgd2lsbCByZWFkIHRoZSBQQ0kgZGV2aWNlIHRyZWUgbm9kZSDigJxyZWfi
gJ0gcHJvcGVydHkgYW5kIHdpbGwgbWFwIHRoZQ0KPj4gRUNBTSBzcGFjZSB0byB0aGUgWEVOIG1l
bW9yeSB1c2luZyB0aGUg4oCcaW9yZW1hcF9ub2NhY2hlICgp4oCdIGZ1bmN0aW9uLg0KPj4NCj4+
IFtzbmlwXQ0KPj4NCj4+IFdoZW4gRG9tMCB0cmllcyB0byBhY2Nlc3MgdGhlIFBDSSBjb25maWcg
c3BhY2Ugb2YgdGhlIGRldmljZSwgWEVOIHdpbGwgZmluZCB0aGUNCj4+IGNvcnJlc3BvbmRpbmcg
aG9zdCBicmlkZ2UgYmFzZWQgb24gc2VnbWVudCBudW1iZXIgYW5kIGFjY2VzcyB0aGUgY29ycmVz
cG9uZGluZw0KPj4gY29uZmlnIHNwYWNlIGFzc2lnbmVkIHRvIHRoYXQgYnJpZGdlLg0KPj4NCj4+
IExpbWl0YXRpb246DQo+PiAqIE9ubHkgUENJIEVDQU0gY29uZmlndXJhdGlvbiBzcGFjZSBhY2Nl
c3MgaXMgc3VwcG9ydGVkLg0KPj4gKiBEZXZpY2UgdHJlZSBiaW5kaW5nIGlzIHN1cHBvcnRlZCBh
cyBvZiBub3csIEFDUEkgaXMgbm90IHN1cHBvcnRlZC4NCj4+ICogTmVlZCB0byBwb3J0IHRoZSBQ
Q0kgaG9zdCBicmlkZ2UgYWNjZXNzIGNvZGUgdG8gWEVOIHRvIGFjY2VzcyB0aGUNCj4+IGNvbmZp
Z3VyYXRpb24gc3BhY2UgKGdlbmVyaWMgb25lIHdvcmtzIGJ1dCBsb3RzIG9mIHBsYXRmb3JtcyB3
aWxsIHJlcXVpcmVkDQo+PiBzb21lIHNwZWNpZmljIGNvZGUgb3IgcXVpcmtzKS4NCj4+DQo+PiAi
DQo+Pg0KPj4gVW5mb3J0dW5hdGVseSB0aGUgZG9jdW1lbnQgaGFkIG5vdCBiZWVuIHVwZGF0ZWQg
c2luY2UgdGhlbiwgYnV0IHRoZSBxdWVzdGlvbg0KPj4NCj4+IGJlaW5nIGRpc2N1c3NlZCBoYXMg
bm90IGNoYW5nZWQgaW4gdGhlIGRlc2lnbjogRG9tYWluLTAgaGFzIGZ1bGwgYWNjZXNzIHRvIHRo
ZSBicmlkZ2UsDQo+Pg0KPj4gWGVuIHRyYXBzIEVDQU0uIChJIGFtIHRha2luZyBkb20wbGVzcyBh
c2lkZSBhcyB0aGF0IHdhcyBub3QgdGhlIHRhcmdldCBmb3IgdGhlIGZpcnN0IHBoYXNlKQ0KPg0K
PiBIYXZpbmcgYW4gdXBkYXRlIGRlc2lnbiBkb2N1bWVudCBpcyBxdWl0ZSBpbXBvcnRhbnQuIFRo
aXMgd2lsbCBhbGxvdyByZXZpZXdlciB0byBjb21tZW50IGVhc2lseSBvbiBvdmVyYWxsIGFwcHJv
YWNoIGFuZCBzcGVlZCB1cCB0aGUgcmV2aWV3IGFzIHdlIGNhbiBtYXRjaCB0byB0aGUgYWdyZWVk
IGFwcHJvYWNoLg0KPg0KPiBTbyBjYW4gdGhpcyBwbGVhc2UgYmUgdXBkYXRlZCBhbmQgc2VudCBh
bG9uZyB0aGUgd29yaz8NCj4NCj4gSW4gYWRkaXRpb24gdG8gdGhhdCwgaXQgZmVlbHMgdG8gbWUg
dGhhdCB0aGUgY29tbWl0IG1lc3NhZ2Ugc2hvdWxkIGNvbnRhaW4gYSBzdW1tYXJ5IG9mIHdoYXQg
eW91IGp1c3QgcGFzdGVkIGFzIHRoaXMgaGVscHMgdW5kZXJzdGFuZGluZyB0aGUgZ29hbCBhbmQg
YXBwcm9hY2ggb2YgdGhpcyBwYXRjaC4NCj4NCklmIHdlIGFyZSBvbiB0aGUgc2FtZSBwYWdlIG5v
dyB3aWxsIHlvdSBiZSBhYmxlIHRvIHJldmlldyB0aGUgcGF0Y2gNCg0Kd2l0aCByZXNwZWN0IHRv
IHRoZSBkZXNpZ24gZnJvbSBSRkM/DQoNCj4gQ2hlZXJzLA0KPg0KVGhhbmsgeW91LA0KDQpPbGVr
c2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:04:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:04:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184529.333199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi4i-0002Sa-FD; Fri, 10 Sep 2021 15:04:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184529.333199; Fri, 10 Sep 2021 15:04:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi4i-0002ST-Bd; Fri, 10 Sep 2021 15:04:20 +0000
Received: by outflank-mailman (input) for mailman id 184529;
 Fri, 10 Sep 2021 15:04:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOi4h-0002SN-6w
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:04:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOi4f-0000yy-7P; Fri, 10 Sep 2021 15:04:17 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOi4f-0006Pz-0x; Fri, 10 Sep 2021 15:04:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=ecZMSGUDyMRhp1lToSbMHgQUvf+rFQlnEq+VhZl4kQs=; b=KhoKcQXCUHsLJCiM12mCaoHVWb
	CbBEvWF5VvzrENfOZkQ+D7F+QpKWqRaWm0Lio91N3yegnWcgA36+r8PvmVuq6LdkaRLmuNGZ0xA9J
	bzWmNX+mYLWoOppp7Hw59OSOAvA8k7Ij6pUFX4M8AALZx1zIVOmjg155GdK5YaMaV8SM=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
Date: Fri, 10 Sep 2021 16:04:14 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit


Hi Oleksandr,

On 10/09/2021 15:01, Oleksandr Andrushchenko wrote:
> 
> On 10.09.21 16:18, Julien Grall wrote:
>>
>>
>> On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>
>> Hi Oleksandr,
>>
>>> On 09.09.21 20:58, Julien Grall wrote:
>>>> On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> Host bridge controller's ECAM space is mapped into Domain-0's p2m,
>>>>> thus it is not possible to trap the same for vPCI via MMIO handlers.
>>>>> For this to work we need to not map those while constructing the domain.
>>>>>
>>>>> Note, that during Domain-0 creation there is no pci_dev yet allocated for
>>>>> host bridges, thus we cannot match PCI host and its associated
>>>>> bridge by SBDF. Use dt_device_node field for checks instead.
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>> ---
>>>>>     xen/arch/arm/domain_build.c        |  3 +++
>>>>>     xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
>>>>>     xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
>>>>>     xen/include/asm-arm/pci.h          | 12 ++++++++++++
>>>>>     4 files changed, 54 insertions(+)
>>>>>
>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>>> index da427f399711..76f5b513280c 100644
>>>>> --- a/xen/arch/arm/domain_build.c
>>>>> +++ b/xen/arch/arm/domain_build.c
>>>>> @@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
>>>>>             }
>>>>>         }
>>>>>     +    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI) ) > +        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev,
>>>> addr, len);
>>>>
>>>> AFAICT, with device_get_class(dev), you know whether the hostbridge is used by Xen. Therefore, I would expect that we don't want to map all the regions of the hostbridges in dom0 (including the BARs).
>>>>
>>>> Can you clarify it?
>>> We only want to trap ECAM, not MMIOs and any other memory regions as the bridge is
>>>
>>> initialized and used by Domain-0 completely.
>>
>> What do you mean by "used by Domain-0 completely"? The hostbridge is owned by Xen so I don't think we can let dom0 access any MMIO regions by
>> default.
> 
> Now it's my time to ask why do you think Xen owns the bridge?
> 
> All the bridges are passed to Domain-0, are fully visible to Domain-0, initialized in Domain-0.
> 
> Enumeration etc. is done in Domain-0. So how comes that Xen owns the bridge? In what way it does?
> 
> Xen just accesses the ECAM when it needs it, but that's it. Xen traps ECAM access - that is true.
> 
> But it in no way uses MMIOs etc. of the bridge - those under direct control of Domain-0

So I looked on the snipped of the design document you posted. I think 
you are instead referring to a different part:

" PCI-PCIe enumeration is a process of detecting devices connected to 
its host.
It is the responsibility of the hardware domain or boot firmware to do 
the PCI enumeration and configure the BAR, PCI capabilities, and 
MSI/MSI-X configuration."

But I still don't see why it means we have to map the MMIOs right now. 
Dom0 should not need to access the MMIOs (aside the hostbridge 
registers) until the BARs are configured.

You can do the mapping when the BAR is accessed. In fact, AFAICT, there 
are code to do the identity mapping in the vPCI. So to me, the mappings 
for the MMIOs are rather pointless if not confusing.

>>
>> In particular, we may want to hide a device from dom0 for security reasons. This is not going to be possible if you map by default everything to dom0.
> 
> Then the bridge most probably will become unusable as we do not have relevant drivers in Xen.
> 
> At best we may rely on the firmware doing the initialization for us, then yes, we can control an ECAM bridge in Xen.
> 
> But this is not the case now: we rely on Domain-0 to initialize and setup the bridge
> 
>>
>> Instead, the BARs should be mapped on demand when dom0 when we trap access to the configuration space.
>>
>> For other regions, could you provide an example of what you are referring too?
> Registers of the bridge for example, all what is listed in "reg" property

I was more after a real life example.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:05:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:05:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184536.333211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi65-00034W-Sd; Fri, 10 Sep 2021 15:05:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184536.333211; Fri, 10 Sep 2021 15:05:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOi65-00034P-NP; Fri, 10 Sep 2021 15:05:45 +0000
Received: by outflank-mailman (input) for mailman id 184536;
 Fri, 10 Sep 2021 15:05:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mOi64-00034D-Qn
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:05:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOi62-00011w-Vb; Fri, 10 Sep 2021 15:05:42 +0000
Received: from [54.239.6.184] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mOi62-0006Z7-PY; Fri, 10 Sep 2021 15:05:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=gFJ+sQJ/dsLn0g1GcAS4YzhE7toEGNPQlDe3fadGH8c=; b=CWQgc/Jv6rpwjuXVC+1PZOHGRN
	DChmFfv1p+NCNHTs5zOgvKOvJshSbFdI4ZmOPlOZOB1/yyLxPSWSi8jQk3lsZdxVNtZ0v7U7apu2t
	GfQ5HeiK0x5MSuw8wRKpGpDR7KUkQxsELCgNZ0HoIz2hcjc3/Plax8tFvFbMlcKClJUY=;
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <733cd423-14f5-c028-8fdd-39aed34cd352@xen.org>
 <3c3c253c-4af7-ea25-5f73-a0f7319dab50@epam.com>
 <34abe11e-c2f9-50ce-fb79-836bfcfb11e0@xen.org>
 <f1ff6186-c483-3de1-cbdf-ac96a6009d15@epam.com>
From: Julien Grall <julien@xen.org>
Message-ID: <9d563708-16c6-5400-7598-4502dfbb6c5d@xen.org>
Date: Fri, 10 Sep 2021 16:05:40 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <f1ff6186-c483-3de1-cbdf-ac96a6009d15@epam.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Oleksandr,

On 10/09/2021 16:01, Oleksandr Andrushchenko wrote:
> 
> On 10.09.21 17:52, Julien Grall wrote:
>>
>>
>> On 10/09/2021 15:38, Oleksandr Andrushchenko wrote:
>>> [snip]
>>>
>>>
>>>>>> What do you mean by "used by Domain-0 completely"? The hostbridge is owned by Xen so I don't think we can let dom0 access any MMIO regions by
>>>>>> default.
>>>>>
>>>>> Now it's my time to ask why do you think Xen owns the bridge?
>>>>
>>>> Because nothing in this series indicates either way. So I assumed this should be owned by Xen because it will drive it.
>>>>
>>>>   From what you wrote below, it sounds like this is not the case. I think you want to have a design document sent along the series so we can easily know what's the expected design and validate that the code match the agreement.
>>>>
>>>> There was already a design document sent a few months ago. So you may want to refresh it (if needed) and send it again with this series.
>>>>
>>> Please see [1] which is the design document we use to implement PCI passthrough on Arm.
>>
>> Thank. Can you make sure to include at least in a link in your cover letter next time?
> I will update the commit message to have more description on the design aspects
>>
>>>
>>> For your convenience:
>>>
>>> "
>>>
>>> # Problem statement:
>>> [snip]
>>>
>>> Only Dom0 and Xen will have access to the real PCI bus,​ guest will have a
>>> direct access to the assigned device itself​. IOMEM memory will be mapped to
>>> the guest ​and interrupt will be redirected to the guest. SMMU has to be
>>> configured correctly to have DMA transaction."
>>>
>>> "
>>>
>>> # Discovering PCI Host Bridge in XEN:
>>>
>>> In order to support the PCI passthrough XEN should be aware of all the PCI host
>>> bridges available on the system and should be able to access the PCI
>>> configuration space. ECAM configuration access is supported as of now. XEN
>>> during boot will read the PCI device tree node “reg” property and will map the
>>> ECAM space to the XEN memory using the “ioremap_nocache ()” function.
>>>
>>> [snip]
>>>
>>> When Dom0 tries to access the PCI config space of the device, XEN will find the
>>> corresponding host bridge based on segment number and access the corresponding
>>> config space assigned to that bridge.
>>>
>>> Limitation:
>>> * Only PCI ECAM configuration space access is supported.
>>> * Device tree binding is supported as of now, ACPI is not supported.
>>> * Need to port the PCI host bridge access code to XEN to access the
>>> configuration space (generic one works but lots of platforms will required
>>> some specific code or quirks).
>>>
>>> "
>>>
>>> Unfortunately the document had not been updated since then, but the question
>>>
>>> being discussed has not changed in the design: Domain-0 has full access to the bridge,
>>>
>>> Xen traps ECAM. (I am taking dom0less aside as that was not the target for the first phase)
>>
>> Having an update design document is quite important. This will allow reviewer to comment easily on overall approach and speed up the review as we can match to the agreed approach.
>>
>> So can this please be updated and sent along the work?
>>
>> In addition to that, it feels to me that the commit message should contain a summary of what you just pasted as this helps understanding the goal and approach of this patch.
>>
> If we are on the same page now will you be able to review the patch
> 
> with respect to the design from RFC?

I believe this was already done as I covered both side in my review.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:33:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:33:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184544.333221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiWP-00074k-1i; Fri, 10 Sep 2021 15:32:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184544.333221; Fri, 10 Sep 2021 15:32:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiWO-00074d-V1; Fri, 10 Sep 2021 15:32:56 +0000
Received: by outflank-mailman (input) for mailman id 184544;
 Fri, 10 Sep 2021 15:32:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWN-00074X-54
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:32:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWN-0001So-11
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:32:55 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWM-0000xp-WC
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:32:55 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOiWJ-0002ur-Gt; Fri, 10 Sep 2021 16:32:51 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=uTOf+LaTavirUm6aGDFAvXM5Dug4UP5qAacs5WI7GxU=; b=BZNYkYB1yhNxLi1TVODm+QI4eO
	8/NWHUzQZ944E+B34vUz+vPFiwNKDgopa1bHEaxdnIJrHlZ2lJ0y4nAMLZyn/lMnR+dVuJO2Urtdn
	23bK5OJLr+ZiFqYOsVhOjp1A4kGpfq1Dnrbppcwe51izLeP00uqR9GYyof5W2ORJOu6A=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24891.31395.187330.546856@mariner.uk.xensource.com>
Date: Fri, 10 Sep 2021 16:32:51 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper  <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
In-Reply-To: <462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
References: <20210909124924.1698-1-jgross@suse.com>
	<20210909124924.1698-2-jgross@suse.com>
	<24890.2778.780247.724651@mariner.uk.xensource.com>
	<7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
	<24890.12692.44064.995392@mariner.uk.xensource.com>
	<462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub"):
> BTW, you probably want to modify OSStest to use configure with:
> 
> --enable-pv-grub --enable-qemu-traditional
> 
> in order to not let the tests fail (applies to x86 only, of course).

I think it might be better to disable those tests.  What do people
think ?

Ian.


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:33:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:33:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184546.333232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiWh-0007Sz-AZ; Fri, 10 Sep 2021 15:33:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184546.333232; Fri, 10 Sep 2021 15:33:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiWh-0007Ss-76; Fri, 10 Sep 2021 15:33:15 +0000
Received: by outflank-mailman (input) for mailman id 184546;
 Fri, 10 Sep 2021 15:33:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWf-0007RG-Kl
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:33:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWf-0001Tx-Jx
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:33:13 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiWf-0001Ea-Ix
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:33:13 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOiWc-0002vd-Cw; Fri, 10 Sep 2021 16:33:10 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=YFsFWZbKf9G0xnGvaiu0K9QyyR72cMC4F81CTsIMKMY=; b=jBJkG+Y5SoUlfA1rry5OvoNPg4
	TmRn0CFtMk7RZu5dND0F/T37/BVLITw2GDMBaqdVF2ZbtoQmFLOKQAAkIMv6jeIv5EJ/BI8Er0+b2
	LJAS65jw0MNUx4BBK/DkB2qcJUsSQUHpyBiXq23PU8Hmu4hxVfDtsm3qM85quWz93tYo=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24891.31414.191381.348639@mariner.uk.xensource.com>
Date: Fri, 10 Sep 2021 16:33:10 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: Re: [PATCH v3 1/3] stubdom: fix build with disabled pv-grub
In-Reply-To: <20210910055518.562-2-jgross@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
	<20210910055518.562-2-jgross@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("[PATCH v3 1/3] stubdom: fix build with disabled pv-grub"):
> Today the build will fail if --disable-pv-grub as a parameter of
> configure, as the main Makefile will unconditionally try to build a
> 32-bit pv-grub stubdom.
> 
> Fix that by introducing a pv-grub-if-enabled target in
> stubdom/Makefile taking care of this situation.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Reviewed-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:34:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:34:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184556.333243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiXP-0008Ax-LY; Fri, 10 Sep 2021 15:33:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184556.333243; Fri, 10 Sep 2021 15:33:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiXP-0008Aq-Ic; Fri, 10 Sep 2021 15:33:59 +0000
Received: by outflank-mailman (input) for mailman id 184556;
 Fri, 10 Sep 2021 15:33:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOiXO-0008AU-7q; Fri, 10 Sep 2021 15:33:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOiXO-0001UW-1n; Fri, 10 Sep 2021 15:33:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOiXN-0000mn-Pl; Fri, 10 Sep 2021 15:33:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOiXN-0005Ej-PG; Fri, 10 Sep 2021 15:33:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Fxb6Y52oCDgQNBJnraYCFDeLhQK8vHPNHM5v+mwfIbY=; b=GsPl+pSxZpWlxqlDjL5rqcS1w3
	RMMvZ0R/oKMjzLOpwnYW02r0enpfVfdFFFexs6srVUQkq425JDanelodb87tlzOYPl2Z5R3RkXo0g
	9A0KBiVLx1udGEAaXauTSfoallsfTLabOWGiYTAYsT9pchAx35iERCvMjn0lNMGHFeLQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164935-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164935: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=05e71df9f3642252744e63cc0d1b0d976109140c
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 15:33:57 +0000

flight 164935 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164935/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              05e71df9f3642252744e63cc0d1b0d976109140c
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  427 days
Failing since        151818  2020-07-11 04:18:52 Z  426 days  417 attempts
Testing same since   164935  2021-09-10 04:21:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73147 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 15:34:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 15:34:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184561.333257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiXl-0000K7-3K; Fri, 10 Sep 2021 15:34:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184561.333257; Fri, 10 Sep 2021 15:34:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOiXl-0000Jy-0J; Fri, 10 Sep 2021 15:34:21 +0000
Received: by outflank-mailman (input) for mailman id 184561;
 Fri, 10 Sep 2021 15:34:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiXj-0000I5-J3
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:34:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiXj-0001VG-II
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:34:19 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mOiXj-0001Ly-HT
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 15:34:19 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mOiXg-0002wW-CN; Fri, 10 Sep 2021 16:34:16 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=PY+koOgOtrOY8a+2WMPC2QRjJ2w9TYtusU9QWR9owkg=; b=QFOnpAi3iiazL6XQDIpwh3RKfe
	VTq4Sg7L5FqaXHC1r5oYJCV1mkv61v0h05TPJO2ZtPFZn+hMzV0Vpu/I0oBsO574PdxobeHZWHwLE
	YjIf8x04bbw1uJtdX1LwChz7Frsxw5vKa2Q98Hq/Z20/70uB2aik+1NJJYfPj7VUcz/g=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24891.31480.165445.521062@mariner.uk.xensource.com>
Date: Fri, 10 Sep 2021 16:34:16 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>,
    Community Manager <community.manager@xenproject.org>
Subject: Re: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per default
In-Reply-To: <20210910055518.562-1-jgross@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("[PATCH v3 0/3] disable building of pv-grub and qemu-trad per default"):
> This is a first step of deprecating pv-grub and qemu-trad including
> ioemu-stubdom. Switch the default to not building it.

This is now fully acked.  But can we wait with committing it until we
have a decision about whether to (a) have osstest explicitly enable
the pv-grub and qemu-trad builds (b) have osstest stop testing these
configurations ?

Ian.


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:13:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:13:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184575.333268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj99-0006N1-4H; Fri, 10 Sep 2021 16:12:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184575.333268; Fri, 10 Sep 2021 16:12:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj99-0006Mu-18; Fri, 10 Sep 2021 16:12:59 +0000
Received: by outflank-mailman (input) for mailman id 184575;
 Fri, 10 Sep 2021 16:12:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOj97-0006Mo-Cx
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:12:57 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b40aa41c-c8c4-45f7-922e-27ac559ccc0a;
 Fri, 10 Sep 2021 16:12:56 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290370242444.1939855081141;
 Fri, 10 Sep 2021 09:12:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b40aa41c-c8c4-45f7-922e-27ac559ccc0a
ARC-Seal: i=1; a=rsa-sha256; t=1631290373; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=S7zH1Yr+e5vBmhRWJYSz9bqQdfjzzlKQLDc1erq4JGAZCaaudXm9QJxvYErSAlXqsUmXBpFckBfnXxg2nTrgM8LCojU8xmV1XN/drrsLWRzZMCZ2qQhx87Wdf+kFTF/QT7YfNL/kGsFQw5Fc8X7my5qcZXi50H+pD4ucMU2VzTk=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290373; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:MIME-Version:Message-ID:Subject:To; 
	bh=2Q9d75Ru4N44075sHiEBdOazR6OiC0qVA4Rjf8vgGlM=; 
	b=CThCCe2uhc/ol9EpIeA17vAYVkjYxy5D5eoNehKmIXsgX11f8/AfaTkjgH0e3NqZGk8RSsT5j1i++NUqRJzLp7Jn2eoPIxh+YxzxGfCsUGujMYQFSu9k1zyx0hCof4AwcAgERHD9PiOhw4V4PCu/0fshESX1/WPtLNW3qcb2UKc=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290373;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=2Q9d75Ru4N44075sHiEBdOazR6OiC0qVA4Rjf8vgGlM=;
	b=YniGmm+PdOuDK0xEwillFqEVU9D6Y5uU7DTZsHpDrIsA102bV5IhySvcRZQCswNg
	jTiPHccYTvwuHthnQCIJO5giGhFvDUszUiHoiZe2TJlmpgukAaulW5Ghj+SgZ4U+YZS
	Ms1JoA3cPPabOZH57ziLqXxO/5p0JjJP2PUt0Sas=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: xen-devel@lists.xenproject.org
Cc: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v6 00/10] xsm: refactoring xsm hooks
Date: Fri, 10 Sep 2021 16:12:55 -0400
Message-Id: <20210910201305.32526-1-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Based on feedback from 2021 Xen Developers Summit the xsm-roles RFC patch set
is being split into two separate patch sets. This is the first patch set and is
focused purely on the clean up and refactoring of the XSM hooks.

This patch set refactors the xsm_ops wrapper hooks to use the alternative_call
infrastructure. Then proceeds to move and realign the headers to simplify the
unnecessarily complicated header inclusions, remove the facad of being able to
enable/disable XSM, and simplify the XSM hooks down to a single interface and
implementation. The remainder of the changes are clean up and removing no
longer necessary abstractions.

v2:
 - restructured the patches, breaking them up as needed
 - incorporate Andrew Cooper's alternative call common code
 - change XSM module registration, removing register_xsm
 - incoporate KConfig recommendations
 - reworded commit messages
 - incorporate macro expansion recommendations
 - misc clean-up fallout from recommendations

v3:
 - renamed struct xsm_operations to struct xsm_ops
 - flask and silo ops structs made __initconst
 - fixed misplacement of __init on flask/silo_init
 - lots of coding style alignment
 - further clean up from FLASK_DISABLE removal
 - addressed commit message comments
 - removed missed guard around alternative-call include
- reworked approach to XSM hooks, merging the two interfaces instead of
   dropping one

v4:
- make __alt_call_maybe_initdata resolve to __read_mostly when
  CONFIG_ALTERNATIVE_CALL is not set
- removed the masking of void with xsm_op_t
- moved all the coding style conformity to an early commit
- fixed declaration on real and stub declarations in xsm.h
- corrected to __initconstrel
- made the xsm_ops global __read_mostly
- adjusted blank lines in xsm_ops struct to provide consistent grouping
- moved extern references to built in policy to where they are used
- Added back in the #ifdef CONFIG_XSM into struct evtchn
- split the patch removing the duplicate interface up further

v5
- changed 80 col wrapping style to preferred style
- a few additional code style cleanups caught in v4 review
- made xsm_ops_registered __initdata
- pulled duplicate if out of switch cases in xsm_core_init()
- dropped struct evtchn commit
- dropped silo hooks direct calls to xsm_default_action

v6
- adjusted alt call documentation comment
- fixed the leak of hard tabs into the commits

Andrew Cooper (1):
  xen: Implement xen/alternative-call.h for use in common code

Daniel P. Smith (9):
  xsm: remove the ability to disable flask
  xsm: remove remnants of xsm_memtype hook
  xsm: drop dubious xsm_op_t type
  xsm: apply coding style
  xsm: refactor xsm_ops handling
  xsm: convert xsm_ops hook calls to alternative call
  xsm: decouple xsm header inclusion selection
  kconfig: update xsm config to reflect reality
  xsm: remove alternate xsm hook interface

 xen/arch/x86/Kconfig               |   1 +
 xen/common/Kconfig                 |  52 +-
 xen/include/public/xsm/flask_op.h  |   2 +-
 xen/include/xen/alternative-call.h |  63 +++
 xen/include/xen/hypercall.h        |   4 +-
 xen/include/xen/sched.h            |   2 +-
 xen/include/xsm/dummy.h            | 215 +++++---
 xen/include/xsm/xsm-core.h         | 266 ++++++++++
 xen/include/xsm/xsm.h              | 802 +++++++++++++----------------
 xen/xsm/Makefile                   |   4 +-
 xen/xsm/dummy.c                    |   7 +-
 xen/xsm/dummy.h                    | 659 ++++++++++++++++++++++++
 xen/xsm/flask/flask_op.c           |  32 +-
 xen/xsm/flask/hooks.c              |  16 +-
 xen/xsm/silo.c                     |  11 +-
 xen/xsm/xsm_core.c                 |  93 ++--
 xen/xsm/xsm_policy.c               |   7 +-
 17 files changed, 1564 insertions(+), 672 deletions(-)
 create mode 100644 xen/include/xen/alternative-call.h
 create mode 100644 xen/include/xsm/xsm-core.h
 create mode 100644 xen/xsm/dummy.h

-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:13:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:13:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184576.333279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9D-0006de-Cb; Fri, 10 Sep 2021 16:13:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184576.333279; Fri, 10 Sep 2021 16:13:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9D-0006dV-94; Fri, 10 Sep 2021 16:13:03 +0000
Received: by outflank-mailman (input) for mailman id 184576;
 Fri, 10 Sep 2021 16:13:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOj9C-0006Mo-8Z
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:13:02 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e5969009-43a6-454c-8afc-04b5103295e9;
 Fri, 10 Sep 2021 16:13:01 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290372344276.49709791440694;
 Fri, 10 Sep 2021 09:12:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5969009-43a6-454c-8afc-04b5103295e9
ARC-Seal: i=1; a=rsa-sha256; t=1631290375; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=NbcvrcfBQp9+lo4wOZaNruDGRNjgGTsmc/WlQ0wfS3uPwT8RmDhZqESzWGo5D8zqqP6aADEy78h8KOt99J+Ka60zOvjFJQy+6hcLBJG9M5ENwsQ0m4Ww18GyrqGGYo56pChthlBVaspsl0bnrFZdqs8mAVIh+mwxzd+glODztho=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290375; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=tLQ4wb+FrQv7YprfemUlYGSVQiae5X1F/rSp63hNSzo=; 
	b=BqW/PhIxreNOFzDfzGghyPUJXNjckJ4PuCCBTpm9KQWEl7zWI0iVnqcwrwxg8KKU6oEj6d8i7ZT7l4SHA6XczoG2iuVOPxAR0fdsw6vXVSHj0/4t8HaPIsiCOpfL2OrWAC7YAwvnl8z9L0hALGGb2NTjXfXqpZmDNeAvH3fAdDk=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290375;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=tLQ4wb+FrQv7YprfemUlYGSVQiae5X1F/rSp63hNSzo=;
	b=Y1NDr+IeQvHJnFHw3K9gOCe0QP4uipoZhPVw0XLjaLmwta1fUBNx9pZou4khacaU
	nzOJ61ipmq7gTYJePV7KKECJDW/1f1HP3mX4RpwHct0HbuKd61rxryMqfO9NirsWKEI
	L64L5Bgian5hVSzR504RcgwP/CwisP6v65skoajo=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Jan Beulich <jbeulich@suse.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Connor Davis <connojdavis@gmail.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v6 01/10] xen: Implement xen/alternative-call.h for use in common code
Date: Fri, 10 Sep 2021 16:12:56 -0400
Message-Id: <20210910201305.32526-2-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

From: Andrew Cooper <andrew.cooper3@citrix.com>

The alternative call infrastructure is x86-only for now, but the common iommu
code has a variant and more common code wants to use the infrastructure.

Introduce CONFIG_ALTERNATIVE_CALL and a conditional implemetnation so common
code can use the optimisation when available, without requiring all
architectures to implement no-op stubs.

Write some documentation, which was thus far entirely absent, covering the
requirements for an architecture to implement this optimsiation, and how to
use the infrastructure in general code.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Jan Beulich <jbeulich@suse.com>

---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bob Eshleman <bobbyeshleman@gmail.com>
CC: Alistair Francis <alistair.francis@wdc.com>
CC: Connor Davis <connojdavis@gmail.com>
CC: Daniel P. Smith <dpsmith@apertussolutions.com>

v3:
 * Drop __alt_call_maybe_initconst

This is a pre-requisite to "xsm: refactor xsm_ops handling" to avoid breaking
the ARM build.

Build test for the XSM code:

  diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
  index 5eab21e1b168..592074e8f41c 100644
  --- a/xen/xsm/xsm_core.c
  +++ b/xen/xsm/xsm_core.c
  @@ -195,6 +195,16 @@ bool __init has_xsm_magic(paddr_t start)
   }
    #endif

  +#include <xen/alternative-call.h>
  +struct foo {
  +    int (*bar)(void *);
  +} foo __alt_call_maybe_initdata;
  +
  +int test_alternative_call(void)
  +{
  +    return alternative_call(foo.bar, NULL);
  +}
  +
   int __init register_xsm(struct xsm_operations *ops)
    {
         if ( verify(ops) )
---
 xen/arch/x86/Kconfig               |  1 +
 xen/common/Kconfig                 |  3 ++
 xen/include/xen/alternative-call.h | 63 ++++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+)
 create mode 100644 xen/include/xen/alternative-call.h

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 9b164db641..1f83518ee0 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,6 +6,7 @@ config X86
 	def_bool y
 	select ACPI
 	select ACPI_LEGACY_TABLES_LOOKUP
+	select ALTERNATIVE_CALL
 	select ARCH_SUPPORTS_INT128
 	select CORE_PARKING
 	select HAS_ALTERNATIVE
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 0ddd18e11a..ac5491b1cc 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -22,6 +22,9 @@ config GRANT_TABLE
 
 	  If unsure, say Y.
 
+config ALTERNATIVE_CALL
+	bool
+
 config HAS_ALTERNATIVE
 	bool
 
diff --git a/xen/include/xen/alternative-call.h b/xen/include/xen/alternative-call.h
new file mode 100644
index 0000000000..c0be270e28
--- /dev/null
+++ b/xen/include/xen/alternative-call.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef XEN_ALTERNATIVE_CALL
+#define XEN_ALTERNATIVE_CALL
+
+/*
+ * Some subsystems in Xen may have multiple implementions, which can be
+ * resolved to a single implementation at boot time.  By default, this will
+ * result in the use of function pointers.
+ *
+ * Some architectures may have mechanisms for dynamically modifying .text.
+ * Using this mechnaism, function pointers can be converted to direct calls
+ * which are typically more efficient at runtime.
+ *
+ * For architectures to support:
+ *
+ * - Implement alternative_{,v}call() in asm/alternative.h.  Code generation
+ *   requirements are to emit a function pointer call at build time, and stash
+ *   enough metadata to simplify the call at boot once the implementation has
+ *   been resolved.
+ * - Select ALTERNATIVE_CALL in Kconfig.
+ *
+ * To use:
+ *
+ * Consider the following simplified example.
+ *
+ *  1) struct foo_ops __alt_call_maybe_initdata ops;
+ *
+ *  2) const struct foo_ops __initconstrel foo_a_ops = { ... };
+ *     const struct foo_ops __initconstrel foo_b_ops = { ... };
+ *
+ *     void __init foo_init(void)
+ *     {
+ *         ...
+ *         if ( use_impl_a )
+ *             ops = *foo_a_ops;
+ *         else if ( use_impl_b )
+ *             ops = *foo_b_ops;
+ *         ...
+ *     }
+ *
+ *  3) alternative_call(ops.bar, ...);
+ *
+ * There needs to a single ops object (1) which will eventually contain the
+ * function pointers.  This should be populated in foo's init() function (2)
+ * by one of the available implementations.  To call functions, use
+ * alternative_{,v}call() referencing the main ops object (3).
+ */
+
+#ifdef CONFIG_ALTERNATIVE_CALL
+
+#include <asm/alternative.h>
+
+#define __alt_call_maybe_initdata __initdata
+
+#else
+
+#define alternative_call(func, args...)  (func)(args)
+#define alternative_vcall(func, args...) (func)(args)
+
+#define __alt_call_maybe_initdata __read_mostly
+
+#endif /* !CONFIG_ALTERNATIVE_CALL */
+#endif /* XEN_ALTERNATIVE_CALL */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:13:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:13:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184580.333290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9T-00077e-L3; Fri, 10 Sep 2021 16:13:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184580.333290; Fri, 10 Sep 2021 16:13:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9T-00077X-HH; Fri, 10 Sep 2021 16:13:19 +0000
Received: by outflank-mailman (input) for mailman id 184580;
 Fri, 10 Sep 2021 16:13:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOj9S-00075v-GJ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:13:18 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 87d67a80-b29d-4c26-8405-e69cb21b773d;
 Fri, 10 Sep 2021 16:13:17 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290374880385.695511741147;
 Fri, 10 Sep 2021 09:12:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87d67a80-b29d-4c26-8405-e69cb21b773d
ARC-Seal: i=1; a=rsa-sha256; t=1631290378; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=eftcgl6WBbiBfHcWeuvjVd4/qZ8X9u42X6fasPZ2qPXlTG5r79DCz37iXh9nVOyHnh2aIeLDypzdcder0sgkQDVF74Y4EBY3u1uBiMw1pxUsM62nD3CWHluaOLahdf8NobOM/cYLzallBx1Z795TJ4F3YKKziz5jmO966aQRxJ0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290378; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=R9fWktmmlY/XpFnqpgzdSmnNqMw4SOU1eoy5oHGQ8QI=; 
	b=Ujk8UKtyRxBcHSK/03ujSNKhs/hpyY21D+xMAp3nYgi88uHY0Mjh23oMSt9/FCeW01AE8cJT5bfRoDd/i+Y0nhZ6Yor9I7HsEVHCk1b52LRBcKuyWuFDCqdWNTTOBNIeYFgv8q5ebopnqfUnDacmMCMlZIBh1+qC9IEt7LFQT64=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290378;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=R9fWktmmlY/XpFnqpgzdSmnNqMw4SOU1eoy5oHGQ8QI=;
	b=ZhTDnfQS85Ean6yaJtku37HJH9WA+GP+D5wzSnLVtH/8J+WDzntUuwhurhi46EtE
	4oXYXm6XpoVuCltFirV+sDBQDvBORsxhaAueSG46Pj3rkwdgs3AY541ovPbAgQp08zC
	gUbXLZTU0FO9sJguuuYTz5TCQNxk7Q1OVHN8hQUM=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 02/10] xsm: remove the ability to disable flask
Date: Fri, 10 Sep 2021 16:12:57 -0400
Message-Id: <20210910201305.32526-3-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On Linux when SELinux is put into permissive mode the descretionary access
controls are still in place. Whereas for Xen when the enforcing state of flask
is set to permissive, all operations for all domains would succeed, i.e. it
does not fall back to the default access controls. To provide a means to mimic
a similar but not equivalent behavior, a flask op is present to allow a
one-time switch back to the default access controls, aka the "dummy policy".

While this may be desirable for an OS, Xen is a hypervisor and should not allow
the switching of which security policy framework is being enforced after boot.
This patch removes the flask op to enforce the desired XSM usage model
requiring a reboot of Xen to change the XSM policy module in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/public/xsm/flask_op.h |  2 +-
 xen/xsm/flask/flask_op.c          | 30 ------------------------------
 2 files changed, 1 insertion(+), 31 deletions(-)

diff --git a/xen/include/public/xsm/flask_op.h b/xen/include/public/xsm/flask_op.h
index 16af7bc22f..b41dd6dac8 100644
--- a/xen/include/public/xsm/flask_op.h
+++ b/xen/include/public/xsm/flask_op.h
@@ -188,7 +188,7 @@ struct xen_flask_op {
 #define FLASK_SETBOOL           12
 #define FLASK_COMMITBOOLS       13
 #define FLASK_MLS               14
-#define FLASK_DISABLE           15
+#define FLASK_DISABLE           15 /* No longer implemented */
 #define FLASK_GETAVC_THRESHOLD  16
 #define FLASK_SETAVC_THRESHOLD  17
 #define FLASK_AVC_HASHSTATS     18
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 01e52138a1..f41c025391 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -223,32 +223,6 @@ static int flask_security_sid(struct xen_flask_sid_context *arg)
 
 #ifndef COMPAT
 
-static int flask_disable(void)
-{
-    static int flask_disabled = 0;
-
-    if ( ss_initialized )
-    {
-        /* Not permitted after initial policy load. */
-        return -EINVAL;
-    }
-
-    if ( flask_disabled )
-    {
-        /* Only do this once. */
-        return -EINVAL;
-    }
-
-    printk("Flask:  Disabled at runtime.\n");
-
-    flask_disabled = 1;
-
-    /* Reset xsm_ops to the original module. */
-    xsm_ops = &dummy_xsm_ops;
-
-    return 0;
-}
-
 static int flask_security_setavc_threshold(struct xen_flask_setavc_threshold *arg)
 {
     int rv = 0;
@@ -698,10 +672,6 @@ ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
         rv = flask_mls_enabled;
         break;    
 
-    case FLASK_DISABLE:
-        rv = flask_disable();
-        break;
-
     case FLASK_GETAVC_THRESHOLD:
         rv = avc_cache_threshold;
         break;
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:13:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:13:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184593.333301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9m-0007pm-Td; Fri, 10 Sep 2021 16:13:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184593.333301; Fri, 10 Sep 2021 16:13:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9m-0007pb-QH; Fri, 10 Sep 2021 16:13:38 +0000
Received: by outflank-mailman (input) for mailman id 184593;
 Fri, 10 Sep 2021 16:13:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOj9l-00075v-8G
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:13:37 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ca8ce114-82cb-4ca1-8ddf-7eba1622de29;
 Fri, 10 Sep 2021 16:13:31 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163129037659656.3504099923116;
 Fri, 10 Sep 2021 09:12:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca8ce114-82cb-4ca1-8ddf-7eba1622de29
ARC-Seal: i=1; a=rsa-sha256; t=1631290379; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=DWzvGC/MvXBC2nLTXp+5lSjhdtveh+ZwiCJY05cuX5NJVEG6wf2jJvYQEbQ7xxhkxe0C6gpIDlH0MTswDvd16IZySqGFziow+rit+LT2sa2AxHhgLcR8hoEOKCgx+8vkLCOi3jwBO0wwdQ5GrAiQjpuveZqZpwnxeRsXDJuQba0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290379; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=2IE2exj9FVV6j0OTxj+ijceEC3Bs7d1+JzBzHRRpcYI=; 
	b=Ubkb2ItdvZldo5RTvxFoM10O3W79QUSDKc8hB41cgJux2Hu21qyDNuUUT1O//ZgNcBFgXLwJRIhK24QkEAjW8L9WZXjl3Z/gFyLnZLAfxvY+Qp3qVYqTwILLSQ2z98oTIyh0qoHy2SdIVvyNwpPme5S97f1Gv+Cpbk1O4KipJkk=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290379;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=2IE2exj9FVV6j0OTxj+ijceEC3Bs7d1+JzBzHRRpcYI=;
	b=L9EdEVmVhiaJX0ta79XPP7F17m1vHwUT9/YLGTQQvDDy3rFnpCYZOsHgQxq56nxL
	DZ+jyfyFkk5GDDp5xhEZrgTq+qXzI1f3YDQyvTbbuNPds0jZn0ljewG6q0Lnr2p0K/b
	gARtpsR1r5iNK0BlarD6ZtPeTlCpxStDShf8NUB0=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 03/10] xsm: remove remnants of xsm_memtype hook
Date: Fri, 10 Sep 2021 16:12:58 -0400
Message-Id: <20210910201305.32526-4-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
left behind. This commit cleans up those remnants.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/include/xsm/xsm.h | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index ad3cddbf7d..335ba1b022 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -161,7 +161,6 @@ struct xsm_operations {
     int (*shadow_control) (struct domain *d, uint32_t op);
     int (*mem_sharing_op) (struct domain *d, struct domain *cd, int op);
     int (*apic) (struct domain *d, int cmd);
-    int (*memtype) (uint32_t access);
     int (*machine_memory_map) (void);
     int (*domain_memory_map) (struct domain *d);
 #define XSM_MMU_UPDATE_READ      1
@@ -632,11 +631,6 @@ static inline int xsm_apic (xsm_default_t def, struct domain *d, int cmd)
     return xsm_ops->apic(d, cmd);
 }
 
-static inline int xsm_memtype (xsm_default_t def, uint32_t access)
-{
-    return xsm_ops->memtype(access);
-}
-
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
     return xsm_ops->machine_memory_map();
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:13:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:13:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184596.333312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9t-0008EI-BD; Fri, 10 Sep 2021 16:13:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184596.333312; Fri, 10 Sep 2021 16:13:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOj9t-0008EB-7E; Fri, 10 Sep 2021 16:13:45 +0000
Received: by outflank-mailman (input) for mailman id 184596;
 Fri, 10 Sep 2021 16:13:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOj9r-0008BY-SZ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:13:43 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d8ef56ff-e209-46b2-b010-340fcfd9b559;
 Fri, 10 Sep 2021 16:13:42 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290377952988.6809551754033;
 Fri, 10 Sep 2021 09:12:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8ef56ff-e209-46b2-b010-340fcfd9b559
ARC-Seal: i=1; a=rsa-sha256; t=1631290382; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=DbVRloX7A6lGIS8oE42ZEP4KKTTc7HlacfVMy2uv2QrGrCRoiGRr7CAIOBhAYgVlodQhVr1hv/DqrretUTuO3xehn8oPDG2DXhadJMr3PCDY575FaNxuW28Q7u3NwbDGsvMhXm3BqK5K644SziVIyl41oxhRbJzOHoUNyyQxYP4=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290382; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=Lq07B1AF5Rs92+IfuaXrGvK7UyuYlp7NAcCIhzsP44w=; 
	b=UTgU0f2lY3Vnx/jw13HUIepIBNBSyGgaXEEmmqRFPjlYwLY5/MYxj/b06xXm72t/p5mAwN4cVUuXia6BekdYWd0hiTllJugqPpWGVH1yK9sneC++K6RlTvT/1rWhYgMvRoOJNw5Zr/YdyRqsSzMi3mDolox8xBAyUYEXulzIIYI=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290382;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=Lq07B1AF5Rs92+IfuaXrGvK7UyuYlp7NAcCIhzsP44w=;
	b=cKLCvkgDCgR/ef1rWKLrref4oC1TDlrwcIt0rJh/jkoNlVsEobNn7OFWilx2nWIh
	BFE/7N0U+6HuLq0rmLFyEiMo0E/K0SZgTcw21eT1mdc1WrnSKxdJUON4T8XIoikQPBC
	fefq1TXxFnAjlj395CnU8SaF5aztRftz4UUjLAqQ=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 04/10] xsm: drop dubious xsm_op_t type
Date: Fri, 10 Sep 2021 16:12:59 -0400
Message-Id: <20210910201305.32526-5-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

The type xsm_op_t masks the use of void pointers. This commit drops the
xsm_op_t type and replaces it and all its uses with an explicit void.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xen/hypercall.h |  4 ++--
 xen/include/xsm/dummy.h     |  4 ++--
 xen/include/xsm/xsm.h       | 11 ++++-------
 xen/xsm/flask/flask_op.c    |  2 +-
 xen/xsm/flask/hooks.c       |  4 ++--
 xen/xsm/xsm_core.c          |  4 ++--
 6 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 34b7f1fed6..3771487a30 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -127,7 +127,7 @@ do_kexec_op(
 
 extern long
 do_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_xsm_op);
+    XEN_GUEST_HANDLE_PARAM(void) u_xsm_op);
 
 #ifdef CONFIG_ARGO
 extern long do_argo_op(
@@ -198,7 +198,7 @@ compat_set_timer_op(
     s32 hi);
 
 extern int compat_xsm_op(
-    XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    XEN_GUEST_HANDLE_PARAM(void) op);
 
 extern int compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg);
 
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 363c6d7798..214b5408b1 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -442,13 +442,13 @@ static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
 
 #ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return -ENOSYS;
 }
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 335ba1b022..16b90be2c5 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,9 +18,6 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
-typedef void xsm_op_t;
-DEFINE_XEN_GUEST_HANDLE(xsm_op_t);
-
 /* policy magic number (defined by XSM_MAGIC) */
 typedef u32 xsm_magic_t;
 
@@ -129,9 +126,9 @@ struct xsm_operations {
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op);
+    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
     int (*hvm_param) (struct domain *d, unsigned long op);
@@ -542,13 +539,13 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index f41c025391..221ff00fd3 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -607,7 +607,7 @@ static int flask_relabel_domain(struct xen_flask_relabel *arg)
 
 #endif /* !COMPAT */
 
-ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op)
+ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op)
 {
     xen_flask_op_t op;
     int rv;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index f1a1217c98..1465db125a 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1742,8 +1742,8 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 
 #endif
 
-long do_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
-int compat_flask_op(XEN_GUEST_HANDLE_PARAM(xsm_op_t) u_flask_op);
+long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
 static struct xsm_operations flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 5eab21e1b1..ac553f9c0d 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -213,13 +213,13 @@ int __init register_xsm(struct xsm_operations *ops)
 
 #endif
 
-long do_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-int compat_xsm_op (XEN_GUEST_HANDLE_PARAM(xsm_op_t) op)
+int compat_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_compat_op(op);
 }
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:14:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:14:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184604.333322 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAB-0000YX-L1; Fri, 10 Sep 2021 16:14:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184604.333322; Fri, 10 Sep 2021 16:14:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAB-0000XU-Hf; Fri, 10 Sep 2021 16:14:03 +0000
Received: by outflank-mailman (input) for mailman id 184604;
 Fri, 10 Sep 2021 16:14:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjAA-0000Vk-IU
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:14:02 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a1a144a5-645d-440a-8495-135b51333765;
 Fri, 10 Sep 2021 16:13:58 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290379745178.98198710030078;
 Fri, 10 Sep 2021 09:12:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1a144a5-645d-440a-8495-135b51333765
ARC-Seal: i=1; a=rsa-sha256; t=1631290383; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=SN11h5uFKpvyekn+Li0+8REwK6MV5UXaW2uDEdjPDCY+wmRkSMTbFMun+rzCivad3R5udeHv1X+nUmFYvAf9I6sWIAKsPzXcD0ia57PBM2w3q/8AOMcya+URTnhU8uJP4BpAaHZKiPZSzrzkjiGa7daVci+Kdv9hUltT29hZ0zQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290383; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=SpoxjhcWcUqmXLou3DVlP7swX206Dz0qNxPdTVQSBNw=; 
	b=LtwY4YEwwp15jjVbUcc+XaAp+LxhE0P/07tbIRWjNeniy0WneUHXaeBSwRSkajj0/4/VL9WhDmJJhWJbCP/L0xk3Mymh4k2tnCsGzjXfRmLkr6uRDl872BYLyn+i9kF+K39sXdK9FEjit6AuqWWS59xobzmu/lyEC1KV+/N3jxQ=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290383;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=SpoxjhcWcUqmXLou3DVlP7swX206Dz0qNxPdTVQSBNw=;
	b=h78uGQttDHiCCKGiH0BPm/c/TYijwKK680IXfZsCFQ0V93RNZ3DRpLGThecTYWuD
	OzoO8ktKHO7yWxYHH94a8ydq8BD5L7H+7VUbrYTzDElqYULQ5BPZhnHV3/CVRffgdQR
	EFY7+dYxV4WihMJvbO/49pSn64xPll0OyELpXGQQ=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 05/10] xsm: apply coding style
Date: Fri, 10 Sep 2021 16:13:00 -0400
Message-Id: <20210910201305.32526-6-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Instead of intermixing coding style changes with code changes as they
are come upon in this patch set, moving all coding style changes
into a single commit. The focus of coding style changes here are,

 - move trailing comments to line above
 - ensuring line length does not exceed 80 chars
 - ensuring proper indentation for 80 char wrapping
 - covert u32 type statements to  uint32_t
 - remove space between closing and opening parens
 - drop extern on function declarations

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h | 190 +++++++++------
 xen/include/xsm/xsm.h   | 495 ++++++++++++++++++++++------------------
 xen/xsm/xsm_core.c      |   8 +-
 xen/xsm/xsm_policy.c    |   7 +-
 4 files changed, 396 insertions(+), 304 deletions(-)

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 214b5408b1..3b1b378b58 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -19,7 +19,8 @@
 #include <xsm/xsm.h>
 #include <public/hvm/params.h>
 
-/* Cannot use BUILD_BUG_ON here because the expressions we check are not
+/*
+ * Cannot use BUILD_BUG_ON here because the expressions we check are not
  * considered constant at compile time. Instead, rely on constant propagation to
  * inline out the calls to this invalid function, which will cause linker errors
  * if references remain at link time.
@@ -44,7 +45,8 @@ void __xsm_action_mismatch_detected(void);
 
 #ifdef CONFIG_XSM
 
-/* In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
+/*
+ * In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
  * contains static (not inline) functions compiled to the dummy XSM module.
  * There is no xsm_default_t argument available, so the value from the assertion
  * is used to initialize the variable.
@@ -57,7 +59,8 @@ void __xsm_action_mismatch_detected(void);
 
 #else /* CONFIG_XSM */
 
-/* In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
+/*
+ * In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
  * contains inline functions for each XSM hook. These functions also perform
  * compile-time checks on the xsm_default_t argument to ensure that the behavior
  * of the dummy XSM module is the same as the behavior with XSM disabled.
@@ -98,13 +101,14 @@ static always_inline int xsm_default_action(
     }
 }
 
-static XSM_INLINE void xsm_security_domaininfo(struct domain *d,
-                                    struct xen_domctl_getdomaininfo *info)
+static XSM_INLINE void xsm_security_domaininfo(
+    struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
     return;
 }
 
-static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d, u32 ssidref)
+static XSM_INLINE int xsm_domain_create(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t ssidref)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -116,7 +120,8 @@ static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_domctl_scheduler_op(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int xsm_domctl_scheduler_op(
+    XSM_DEFAULT_ARG struct domain *d, int cmd)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -128,7 +133,8 @@ static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_set_target(XSM_DEFAULT_ARG struct domain *d, struct domain *e)
+static XSM_INLINE int xsm_set_target(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *e)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, NULL);
@@ -173,38 +179,43 @@ static XSM_INLINE void xsm_free_security_domain(struct domain *d)
     return;
 }
 
-static XSM_INLINE int xsm_grant_mapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static XSM_INLINE int xsm_grant_mapref(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2, uint32_t flags)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_unmapref(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_unmapref(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_setup(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_setup(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_transfer(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_transfer(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_copy(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_copy(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_grant_query_size(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_grant_query_size(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -216,14 +227,15 @@ static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_memory_adjust_reservation(XSM_DEFAULT_ARG struct domain *d1,
-                                                            struct domain *d2)
+static XSM_INLINE int xsm_memory_adjust_reservation(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_memory_stat_reservation(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -253,14 +265,16 @@ static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_schedop_shutdown(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_schedop_shutdown(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_memory_pin_page(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
-                                          struct page_info *page)
+static XSM_INLINE int xsm_memory_pin_page(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
+    struct page_info *page)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
@@ -272,15 +286,16 @@ static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_unbound(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn,
-                                         domid_t id2)
+static XSM_INLINE int xsm_evtchn_unbound(
+    XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn, domid_t id2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_interdomain(XSM_DEFAULT_ARG struct domain *d1, struct evtchn
-                                *chan1, struct domain *d2, struct evtchn *chan2)
+static XSM_INLINE int xsm_evtchn_interdomain(
+    XSM_DEFAULT_ARG struct domain *d1, struct evtchn *chan1, struct domain *d2,
+    struct evtchn *chan2)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d1, d2);
@@ -291,19 +306,22 @@ static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
     return;
 }
 
-static XSM_INLINE int xsm_evtchn_send(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_send(
+    XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, d, NULL);
 }
 
-static XSM_INLINE int xsm_evtchn_status(XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
+static XSM_INLINE int xsm_evtchn_status(
+    XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_evtchn_reset(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_evtchn_reset(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
@@ -321,7 +339,8 @@ static XSM_INLINE void xsm_free_security_evtchns(
     return;
 }
 
-static XSM_INLINE char *xsm_show_security_evtchn(struct domain *d, const struct evtchn *chn)
+static XSM_INLINE char *xsm_show_security_evtchn(
+    struct domain *d, const struct evtchn *chn)
 {
     return NULL;
 }
@@ -357,13 +376,15 @@ static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_assign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_assign_device(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
+static XSM_INLINE int xsm_deassign_device(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -372,15 +393,15 @@ static XSM_INLINE int xsm_deassign_device(XSM_DEFAULT_ARG struct domain *d, uint
 #endif /* HAS_PASSTHROUGH && HAS_PCI */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static XSM_INLINE int xsm_assign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                          const char *dtpath)
+static XSM_INLINE int xsm_assign_dtdevice(
+    XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_deassign_dtdevice(XSM_DEFAULT_ARG struct domain *d,
-                                            const char *dtpath)
+static XSM_INLINE int xsm_deassign_dtdevice(
+    XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -400,19 +421,22 @@ static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_plug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_plug_pci(
+    XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_unplug_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_unplug_pci(
+    XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_resource_setup_pci(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int xsm_resource_setup_pci(
+    XSM_DEFAULT_ARG uint32_t machine_bdf)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, NULL);
@@ -465,8 +489,8 @@ static XSM_INLINE int xsm_map_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_map_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                         int irq, const void *data)
+static XSM_INLINE int xsm_map_domain_irq(
+    XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
@@ -478,76 +502,86 @@ static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_bind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_bind_pt_irq(
+    XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_unbind_pt_irq(XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static XSM_INLINE int xsm_unbind_pt_irq(
+    XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_unmap_domain_irq(XSM_DEFAULT_ARG struct domain *d,
-                                           int irq, const void *data)
+static XSM_INLINE int xsm_unmap_domain_irq(
+    XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_irq_permission(XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
+static XSM_INLINE int xsm_irq_permission(
+    XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_permission(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_permission(
+    XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_iomem_mapping(XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static XSM_INLINE int xsm_iomem_mapping(
+    XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pci_config_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf,
-                                        uint16_t start, uint16_t end,
-                                        uint8_t access)
+static XSM_INLINE int xsm_pci_config_permission(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, uint16_t start,
+    uint16_t end, uint8_t access)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_add_to_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_add_to_physmap(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_remove_from_physmap(XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
+static XSM_INLINE int xsm_remove_from_physmap(
+    XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d1, d2);
 }
 
-static XSM_INLINE int xsm_map_gmfn_foreign(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_map_gmfn_foreign(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_hvm_param(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_param(
+    XSM_DEFAULT_ARG struct domain *d, unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_control(XSM_DEFAULT_ARG struct domain *d, unsigned long op)
+static XSM_INLINE int xsm_hvm_control(
+    XSM_DEFAULT_ARG struct domain *d, unsigned long op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -559,7 +593,8 @@ static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
+static XSM_INLINE int xsm_hvm_altp2mhvm_op(
+    XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
 
@@ -578,7 +613,8 @@ static XSM_INLINE int xsm_hvm_altp2mhvm_op(XSM_DEFAULT_ARG struct domain *d, uin
     }
 }
 
-static XSM_INLINE int xsm_vm_event_control(XSM_DEFAULT_ARG struct domain *d, int mode, int op)
+static XSM_INLINE int xsm_vm_event_control(
+    XSM_DEFAULT_ARG struct domain *d, int mode, int op)
 {
     XSM_ASSERT_ACTION(XSM_PRIV);
     return xsm_default_action(action, current->domain, d);
@@ -621,13 +657,15 @@ static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
     return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_shadow_control(XSM_DEFAULT_ARG struct domain *d, uint32_t op)
+static XSM_INLINE int xsm_shadow_control(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mem_sharing_op(XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
+static XSM_INLINE int xsm_mem_sharing_op(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
 {
     XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return xsm_default_action(action, current->domain, cd);
@@ -651,8 +689,9 @@ static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct domain *t,
-                                     struct domain *f, uint32_t flags)
+static XSM_INLINE int xsm_mmu_update(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *t, struct domain *f,
+    uint32_t flags)
 {
     int rc = 0;
     XSM_ASSERT_ACTION(XSM_TARGET);
@@ -663,38 +702,43 @@ static XSM_INLINE int xsm_mmu_update(XSM_DEFAULT_ARG struct domain *d, struct do
     return rc;
 }
 
-static XSM_INLINE int xsm_mmuext_op(XSM_DEFAULT_ARG struct domain *d, struct domain *f)
+static XSM_INLINE int xsm_mmuext_op(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *f)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_update_va_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *f, 
-                                                            l1_pgentry_t pte)
+static XSM_INLINE int xsm_update_va_mapping(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, f);
 }
 
-static XSM_INLINE int xsm_priv_mapping(XSM_DEFAULT_ARG struct domain *d, struct domain *t)
+static XSM_INLINE int xsm_priv_mapping(
+    XSM_DEFAULT_ARG struct domain *d, struct domain *t)
 {
     XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_default_action(action, d, t);
 }
 
-static XSM_INLINE int xsm_ioport_permission(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_permission(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_ioport_mapping(XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static XSM_INLINE int xsm_ioport_mapping(
+    XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
     return xsm_default_action(action, current->domain, d);
 }
 
-static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct domain *d, unsigned int op)
+static XSM_INLINE int xsm_pmu_op(
+    XSM_DEFAULT_ARG struct domain *d, unsigned int op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
@@ -723,8 +767,8 @@ static XSM_INLINE int xsm_argo_enable(const struct domain *d)
     return 0;
 }
 
-static XSM_INLINE int xsm_argo_register_single_source(const struct domain *d,
-                                                      const struct domain *t)
+static XSM_INLINE int xsm_argo_register_single_source(
+    const struct domain *d, const struct domain *t)
 {
     return 0;
 }
@@ -734,8 +778,8 @@ static XSM_INLINE int xsm_argo_register_any_source(const struct domain *d)
     return 0;
 }
 
-static XSM_INLINE int xsm_argo_send(const struct domain *d,
-                                    const struct domain *t)
+static XSM_INLINE int xsm_argo_send(
+    const struct domain *d, const struct domain *t)
 {
     return 0;
 }
@@ -743,7 +787,7 @@ static XSM_INLINE int xsm_argo_send(const struct domain *d,
 #endif /* CONFIG_ARGO */
 
 #include <public/version.h>
-static XSM_INLINE int xsm_xen_version (XSM_DEFAULT_ARG uint32_t op)
+static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
     switch ( op )
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 16b90be2c5..7673d605a6 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -19,7 +19,7 @@
 #include <xen/multiboot.h>
 
 /* policy magic number (defined by XSM_MAGIC) */
-typedef u32 xsm_magic_t;
+typedef uint32_t xsm_magic_t;
 
 #ifdef CONFIG_XSM_FLASK
 #define XSM_MAGIC 0xf97cff8c
@@ -27,7 +27,8 @@ typedef u32 xsm_magic_t;
 #define XSM_MAGIC 0x0
 #endif
 
-/* These annotations are used by callers and in dummy.h to document the
+/*
+ * These annotations are used by callers and in dummy.h to document the
  * default actions of XSM hooks. They should be compiled out otherwise.
  */
 enum xsm_default {
@@ -41,147 +42,154 @@ enum xsm_default {
 typedef enum xsm_default xsm_default_t;
 
 struct xsm_operations {
-    void (*security_domaininfo) (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info);
-    int (*domain_create) (struct domain *d, u32 ssidref);
-    int (*getdomaininfo) (struct domain *d);
-    int (*domctl_scheduler_op) (struct domain *d, int op);
-    int (*sysctl_scheduler_op) (int op);
-    int (*set_target) (struct domain *d, struct domain *e);
-    int (*domctl) (struct domain *d, int cmd);
-    int (*sysctl) (int cmd);
-    int (*readconsole) (uint32_t clear);
-
-    int (*evtchn_unbound) (struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain) (struct domain *d1, struct evtchn *chn1,
-                                        struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post) (struct evtchn *chn);
-    int (*evtchn_send) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_status) (struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset) (struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref) (struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref) (struct domain *d1, struct domain *d2);
-    int (*grant_setup) (struct domain *d1, struct domain *d2);
-    int (*grant_transfer) (struct domain *d1, struct domain *d2);
-    int (*grant_copy) (struct domain *d1, struct domain *d2);
-    int (*grant_query_size) (struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain) (struct domain *d);
-    void (*free_security_domain) (struct domain *d);
-    int (*alloc_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns) (struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn) (struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain) (struct domain *d);
-
-    int (*get_pod_target) (struct domain *d);
-    int (*set_pod_target) (struct domain *d);
-    int (*memory_exchange) (struct domain *d);
-    int (*memory_adjust_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation) (struct domain *d1, struct domain *d2);
-    int (*memory_pin_page) (struct domain *d1, struct domain *d2, struct page_info *page);
-    int (*add_to_physmap) (struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap) (struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign) (struct domain *d, struct domain *t);
-    int (*claim_pages) (struct domain *d);
-
-    int (*console_io) (struct domain *d, int cmd);
-
-    int (*profile) (struct domain *d, int op);
-
-    int (*kexec) (void);
-    int (*schedop_shutdown) (struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid) (int irq);
-    int (*map_domain_pirq) (struct domain *d);
-    int (*map_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq) (struct domain *d);
-    int (*unmap_domain_irq) (struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission) (struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*iomem_mapping) (struct domain *d, uint64_t s, uint64_t e, uint8_t allow);
-    int (*pci_config_permission) (struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access);
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group) (uint32_t machine_bdf);
-    int (*assign_device) (struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device) (struct domain *d, uint32_t machine_bdf);
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
 #endif
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice) (struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice) (struct domain *d, const char *dtpath);
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
 #endif
 
-    int (*resource_plug_core) (void);
-    int (*resource_unplug_core) (void);
-    int (*resource_plug_pci) (uint32_t machine_bdf);
-    int (*resource_unplug_pci) (uint32_t machine_bdf);
-    int (*resource_setup_pci) (uint32_t machine_bdf);
-    int (*resource_setup_gsi) (int gsi);
-    int (*resource_setup_misc) (void);
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
 
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
 
-    long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
-    int (*do_compat_op) (XEN_GUEST_HANDLE_PARAM(void) op);
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
 #endif
 
-    int (*hvm_param) (struct domain *d, unsigned long op);
-    int (*hvm_control) (struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm) (struct domain *d);
-    int (*hvm_altp2mhvm_op) (struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo) (struct domain *d);
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
 
-    int (*vm_event_control) (struct domain *d, int mode, int op);
+    int (*vm_event_control)(struct domain *d, int mode, int op);
 
 #ifdef CONFIG_MEM_ACCESS
-    int (*mem_access) (struct domain *d);
+    int (*mem_access)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-    int (*mem_paging) (struct domain *d);
+    int (*mem_paging)(struct domain *d);
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing) (struct domain *d);
+    int (*mem_sharing)(struct domain *d);
 #endif
 
-    int (*platform_op) (uint32_t cmd);
+    int (*platform_op)(uint32_t cmd);
 
 #ifdef CONFIG_X86
-    int (*do_mca) (void);
-    int (*shadow_control) (struct domain *d, uint32_t op);
-    int (*mem_sharing_op) (struct domain *d, struct domain *cd, int op);
-    int (*apic) (struct domain *d, int cmd);
-    int (*machine_memory_map) (void);
-    int (*domain_memory_map) (struct domain *d);
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
 #define XSM_MMU_UPDATE_READ      1
 #define XSM_MMU_UPDATE_WRITE     2
 #define XSM_MMU_NORMAL_UPDATE    4
 #define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update) (struct domain *d, struct domain *t,
-                       struct domain *f, uint32_t flags);
-    int (*mmuext_op) (struct domain *d, struct domain *f);
-    int (*update_va_mapping) (struct domain *d, struct domain *f, l1_pgentry_t pte);
-    int (*priv_mapping) (struct domain *d, struct domain *t);
-    int (*ioport_permission) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
-    int (*pmu_op) (struct domain *d, unsigned int op);
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
 #endif
-    int (*dm_op) (struct domain *d);
-    int (*xen_version) (uint32_t cmd);
-    int (*domain_resource_map) (struct domain *d);
+    int (*dm_op)(struct domain *d);
+    int (*xen_version)(uint32_t cmd);
+    int (*domain_resource_map)(struct domain *d);
 #ifdef CONFIG_ARGO
-    int (*argo_enable) (const struct domain *d);
-    int (*argo_register_single_source) (const struct domain *d,
-                                        const struct domain *t);
-    int (*argo_register_any_source) (const struct domain *d);
-    int (*argo_send) (const struct domain *d, const struct domain *t);
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
 #endif
 };
 
@@ -191,121 +199,133 @@ extern struct xsm_operations *xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
-static inline void xsm_security_domaininfo (struct domain *d,
-                                        struct xen_domctl_getdomaininfo *info)
+static inline void xsm_security_domaininfo(
+    struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
     xsm_ops->security_domaininfo(d, info);
 }
 
-static inline int xsm_domain_create (xsm_default_t def, struct domain *d, u32 ssidref)
+static inline int xsm_domain_create(
+    xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
     return xsm_ops->domain_create(d, ssidref);
 }
 
-static inline int xsm_getdomaininfo (xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->getdomaininfo(d);
 }
 
-static inline int xsm_domctl_scheduler_op (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl_scheduler_op(
+    xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->domctl_scheduler_op(d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op (xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl_scheduler_op(cmd);
 }
 
-static inline int xsm_set_target (xsm_default_t def, struct domain *d, struct domain *e)
+static inline int xsm_set_target(
+    xsm_default_t def, struct domain *d, struct domain *e)
 {
     return xsm_ops->set_target(d, e);
 }
 
-static inline int xsm_domctl (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->domctl(d, cmd);
 }
 
-static inline int xsm_sysctl (xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
     return xsm_ops->sysctl(cmd);
 }
 
-static inline int xsm_readconsole (xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
     return xsm_ops->readconsole(clear);
 }
 
-static inline int xsm_evtchn_unbound (xsm_default_t def, struct domain *d1, struct evtchn *chn,
-                                                                    domid_t id2)
+static inline int xsm_evtchn_unbound(
+    xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
     return xsm_ops->evtchn_unbound(d1, chn, id2);
 }
 
-static inline int xsm_evtchn_interdomain (xsm_default_t def, struct domain *d1,
-                struct evtchn *chan1, struct domain *d2, struct evtchn *chan2)
+static inline int xsm_evtchn_interdomain(
+    xsm_default_t def, struct domain *d1, struct evtchn *chan1,
+    struct domain *d2, struct evtchn *chan2)
 {
     return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
-static inline void xsm_evtchn_close_post (struct evtchn *chn)
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
     xsm_ops->evtchn_close_post(chn);
 }
 
-static inline int xsm_evtchn_send (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_send(
+    xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
     return xsm_ops->evtchn_send(d, chn);
 }
 
-static inline int xsm_evtchn_status (xsm_default_t def, struct domain *d, struct evtchn *chn)
+static inline int xsm_evtchn_status(
+    xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
     return xsm_ops->evtchn_status(d, chn);
 }
 
-static inline int xsm_evtchn_reset (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_evtchn_reset(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->evtchn_reset(d1, d2);
 }
 
-static inline int xsm_grant_mapref (xsm_default_t def, struct domain *d1, struct domain *d2,
-                                                                uint32_t flags)
+static inline int xsm_grant_mapref(
+    xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
     return xsm_ops->grant_mapref(d1, d2, flags);
 }
 
-static inline int xsm_grant_unmapref (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_unmapref(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_unmapref(d1, d2);
 }
 
-static inline int xsm_grant_setup (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_setup(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_setup(d1, d2);
 }
 
-static inline int xsm_grant_transfer (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_transfer(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_transfer(d1, d2);
 }
 
-static inline int xsm_grant_copy (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_copy(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_copy(d1, d2);
 }
 
-static inline int xsm_grant_query_size (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_grant_query_size(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->grant_query_size(d1, d2);
 }
 
-static inline int xsm_alloc_security_domain (struct domain *d)
+static inline int xsm_alloc_security_domain(struct domain *d)
 {
     return xsm_ops->alloc_security_domain(d);
 }
 
-static inline void xsm_free_security_domain (struct domain *d)
+static inline void xsm_free_security_domain(struct domain *d)
 {
     xsm_ops->free_security_domain(d);
 }
@@ -322,60 +342,65 @@ static inline void xsm_free_security_evtchns(
     xsm_ops->free_security_evtchns(chn, nr);
 }
 
-static inline char *xsm_show_security_evtchn (struct domain *d, const struct evtchn *chn)
+static inline char *xsm_show_security_evtchn(
+    struct domain *d, const struct evtchn *chn)
 {
     return xsm_ops->show_security_evtchn(d, chn);
 }
 
-static inline int xsm_init_hardware_domain (xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->init_hardware_domain(d);
 }
 
-static inline int xsm_get_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_pod_target(d);
 }
 
-static inline int xsm_set_pod_target (xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->set_pod_target(d);
 }
 
-static inline int xsm_memory_exchange (xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->memory_exchange(d);
 }
 
-static inline int xsm_memory_adjust_reservation (xsm_default_t def, struct domain *d1, struct
-                                                                    domain *d2)
+static inline int xsm_memory_adjust_reservation(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->memory_adjust_reservation(d1, d2);
 }
 
-static inline int xsm_memory_stat_reservation (xsm_default_t def, struct domain *d1,
-                                                            struct domain *d2)
+static inline int xsm_memory_stat_reservation(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->memory_stat_reservation(d1, d2);
 }
 
-static inline int xsm_memory_pin_page(xsm_default_t def, struct domain *d1, struct domain *d2,
-                                      struct page_info *page)
+static inline int xsm_memory_pin_page(
+    xsm_default_t def, struct domain *d1, struct domain *d2,
+    struct page_info *page)
 {
     return xsm_ops->memory_pin_page(d1, d2, page);
 }
 
-static inline int xsm_add_to_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_add_to_physmap(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->add_to_physmap(d1, d2);
 }
 
-static inline int xsm_remove_from_physmap(xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_remove_from_physmap(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->remove_from_physmap(d1, d2);
 }
 
-static inline int xsm_map_gmfn_foreign (xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_map_gmfn_foreign(
+    xsm_default_t def, struct domain *d, struct domain *t)
 {
     return xsm_ops->map_gmfn_foreign(d, t);
 }
@@ -385,79 +410,87 @@ static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
     return xsm_ops->claim_pages(d);
 }
 
-static inline int xsm_console_io (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->console_io(d, cmd);
 }
 
-static inline int xsm_profile (xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
     return xsm_ops->profile(d, op);
 }
 
-static inline int xsm_kexec (xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t def)
 {
     return xsm_ops->kexec();
 }
 
-static inline int xsm_schedop_shutdown (xsm_default_t def, struct domain *d1, struct domain *d2)
+static inline int xsm_schedop_shutdown(
+    xsm_default_t def, struct domain *d1, struct domain *d2)
 {
     return xsm_ops->schedop_shutdown(d1, d2);
 }
 
-static inline char *xsm_show_irq_sid (int irq)
+static inline char *xsm_show_irq_sid(int irq)
 {
     return xsm_ops->show_irq_sid(irq);
 }
 
-static inline int xsm_map_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->map_domain_pirq(d);
 }
 
-static inline int xsm_map_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_map_domain_irq(
+    xsm_default_t def, struct domain *d, int irq, void *data)
 {
     return xsm_ops->map_domain_irq(d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq (xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->unmap_domain_pirq(d);
 }
 
-static inline int xsm_unmap_domain_irq (xsm_default_t def, struct domain *d, int irq, void *data)
+static inline int xsm_unmap_domain_irq(
+    xsm_default_t def, struct domain *d, int irq, void *data)
 {
     return xsm_ops->unmap_domain_irq(d, irq, data);
 }
 
-static inline int xsm_bind_pt_irq(xsm_default_t def, struct domain *d,
-                                  struct xen_domctl_bind_pt_irq *bind)
+static inline int xsm_bind_pt_irq(
+    xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     return xsm_ops->bind_pt_irq(d, bind);
 }
 
-static inline int xsm_unbind_pt_irq(xsm_default_t def, struct domain *d,
-                                    struct xen_domctl_bind_pt_irq *bind)
+static inline int xsm_unbind_pt_irq(
+    xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
     return xsm_ops->unbind_pt_irq(d, bind);
 }
 
-static inline int xsm_irq_permission (xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
+static inline int xsm_irq_permission(
+    xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
     return xsm_ops->irq_permission(d, pirq, allow);
 }
 
-static inline int xsm_iomem_permission (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_permission(
+    xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_permission(d, s, e, allow);
 }
 
-static inline int xsm_iomem_mapping (xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+static inline int xsm_iomem_mapping(
+    xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
     return xsm_ops->iomem_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pci_config_permission (xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access)
+static inline int xsm_pci_config_permission(
+    xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
+    uint16_t end, uint8_t access)
 {
     return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
 }
@@ -468,63 +501,67 @@ static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
     return xsm_ops->get_device_group(machine_bdf);
 }
 
-static inline int xsm_assign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_assign_device(
+    xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
     return xsm_ops->assign_device(d, machine_bdf);
 }
 
-static inline int xsm_deassign_device(xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+static inline int xsm_deassign_device(
+    xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
     return xsm_ops->deassign_device(d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static inline int xsm_assign_dtdevice(xsm_default_t def, struct domain *d,
-                                      const char *dtpath)
+static inline int xsm_assign_dtdevice(
+    xsm_default_t def, struct domain *d, const char *dtpath)
 {
     return xsm_ops->assign_dtdevice(d, dtpath);
 }
 
-static inline int xsm_deassign_dtdevice(xsm_default_t def, struct domain *d,
-                                        const char *dtpath)
+static inline int xsm_deassign_dtdevice(
+    xsm_default_t def, struct domain *d, const char *dtpath)
 {
     return xsm_ops->deassign_dtdevice(d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_plug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_unplug_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_unplug_pci(
+    xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_unplug_pci(machine_bdf);
 }
 
-static inline int xsm_resource_plug_core (xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t def)
 {
     return xsm_ops->resource_plug_core();
 }
 
-static inline int xsm_resource_unplug_core (xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
     return xsm_ops->resource_unplug_core();
 }
 
-static inline int xsm_resource_setup_pci (xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_setup_pci(
+    xsm_default_t def, uint32_t machine_bdf)
 {
     return xsm_ops->resource_setup_pci(machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi (xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
     return xsm_ops->resource_setup_gsi(gsi);
 }
 
-static inline int xsm_resource_setup_misc (xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
     return xsm_ops->resource_setup_misc();
 }
@@ -539,70 +576,74 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
-static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
-static inline int xsm_do_compat_op (XEN_GUEST_HANDLE_PARAM(void) op)
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_compat_op(op);
 }
 #endif
 
-static inline int xsm_hvm_param (xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_param(
+    xsm_default_t def, struct domain *d, unsigned long op)
 {
     return xsm_ops->hvm_param(d, op);
 }
 
-static inline int xsm_hvm_control(xsm_default_t def, struct domain *d, unsigned long op)
+static inline int xsm_hvm_control(
+    xsm_default_t def, struct domain *d, unsigned long op)
 {
     return xsm_ops->hvm_control(d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm (xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->hvm_param_altp2mhvm(d);
 }
 
-static inline int xsm_hvm_altp2mhvm_op (xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
+static inline int xsm_hvm_altp2mhvm_op(
+    xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
     return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo (xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->get_vnumainfo(d);
 }
 
-static inline int xsm_vm_event_control (xsm_default_t def, struct domain *d, int mode, int op)
+static inline int xsm_vm_event_control(
+    xsm_default_t def, struct domain *d, int mode, int op)
 {
     return xsm_ops->vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing (xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
     return xsm_ops->mem_sharing(d);
 }
 #endif
 
-static inline int xsm_platform_op (xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->platform_op(op);
 }
@@ -613,17 +654,19 @@ static inline int xsm_do_mca(xsm_default_t def)
     return xsm_ops->do_mca();
 }
 
-static inline int xsm_shadow_control (xsm_default_t def, struct domain *d, uint32_t op)
+static inline int xsm_shadow_control(
+    xsm_default_t def, struct domain *d, uint32_t op)
 {
     return xsm_ops->shadow_control(d, op);
 }
 
-static inline int xsm_mem_sharing_op (xsm_default_t def, struct domain *d, struct domain *cd, int op)
+static inline int xsm_mem_sharing_op(
+    xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
     return xsm_ops->mem_sharing_op(d, cd, op);
 }
 
-static inline int xsm_apic (xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
     return xsm_ops->apic(d, cmd);
 }
@@ -638,39 +681,45 @@ static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
     return xsm_ops->domain_memory_map(d);
 }
 
-static inline int xsm_mmu_update (xsm_default_t def, struct domain *d, struct domain *t,
-                                  struct domain *f, uint32_t flags)
+static inline int xsm_mmu_update(
+    xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
+    uint32_t flags)
 {
     return xsm_ops->mmu_update(d, t, f, flags);
 }
 
-static inline int xsm_mmuext_op (xsm_default_t def, struct domain *d, struct domain *f)
+static inline int xsm_mmuext_op(
+    xsm_default_t def, struct domain *d, struct domain *f)
 {
     return xsm_ops->mmuext_op(d, f);
 }
 
-static inline int xsm_update_va_mapping(xsm_default_t def, struct domain *d, struct domain *f,
-                                                            l1_pgentry_t pte)
+static inline int xsm_update_va_mapping(
+    xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
     return xsm_ops->update_va_mapping(d, f, pte);
 }
 
-static inline int xsm_priv_mapping(xsm_default_t def, struct domain *d, struct domain *t)
+static inline int xsm_priv_mapping(
+    xsm_default_t def, struct domain *d, struct domain *t)
 {
     return xsm_ops->priv_mapping(d, t);
 }
 
-static inline int xsm_ioport_permission (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_permission(
+    xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_permission(d, s, e, allow);
 }
 
-static inline int xsm_ioport_mapping (xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+static inline int xsm_ioport_mapping(
+    xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
     return xsm_ops->ioport_mapping(d, s, e, allow);
 }
 
-static inline int xsm_pmu_op (xsm_default_t def, struct domain *d, unsigned int op)
+static inline int xsm_pmu_op(
+    xsm_default_t def, struct domain *d, unsigned int op)
 {
     return xsm_ops->pmu_op(d, op);
 }
@@ -682,7 +731,7 @@ static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
     return xsm_ops->dm_op(d);
 }
 
-static inline int xsm_xen_version (xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
     return xsm_ops->xen_version(op);
 }
@@ -698,8 +747,8 @@ static inline int xsm_argo_enable(const struct domain *d)
     return xsm_ops->argo_enable(d);
 }
 
-static inline int xsm_argo_register_single_source(const struct domain *d,
-                                                  const struct domain *t)
+static inline int xsm_argo_register_single_source(
+    const struct domain *d, const struct domain *t)
 {
     return xsm_ops->argo_register_single_source(d, t);
 }
@@ -719,12 +768,11 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 #endif /* XSM_NO_WRAPPERS */
 
 #ifdef CONFIG_MULTIBOOT
-extern int xsm_multiboot_init(unsigned long *module_map,
-                              const multiboot_info_t *mbi);
-extern int xsm_multiboot_policy_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi,
-                                     void **policy_buffer,
-                                     size_t *policy_size);
+int xsm_multiboot_init(
+    unsigned long *module_map, const multiboot_info_t *mbi);
+int xsm_multiboot_policy_init(
+    unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size);
 #endif
 
 #ifdef CONFIG_HAS_DEVICE_TREE
@@ -733,18 +781,19 @@ extern int xsm_multiboot_policy_init(unsigned long *module_map,
  *
  * On success, return 1 if using SILO mode else 0.
  */
-extern int xsm_dt_init(void);
-extern int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-extern bool has_xsm_magic(paddr_t);
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
 #endif
 
-extern int register_xsm(struct xsm_operations *ops);
+int register_xsm(struct xsm_operations *ops);
 
 extern struct xsm_operations dummy_xsm_ops;
-extern void xsm_fixup_ops(struct xsm_operations *ops);
+
+void xsm_fixup_ops(struct xsm_operations *ops);
 
 #ifdef CONFIG_XSM_FLASK
-extern void flask_init(const void *policy_buffer, size_t policy_size);
+void flask_init(const void *policy_buffer, size_t policy_size);
 #else
 static inline void flask_init(const void *policy_buffer, size_t policy_size)
 {
@@ -757,7 +806,7 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-extern void silo_init(void);
+void silo_init(void);
 #else
 static inline void silo_init(void) {}
 #endif
@@ -767,8 +816,8 @@ static inline void silo_init(void) {}
 #include <xsm/dummy.h>
 
 #ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init (unsigned long *module_map,
-                                      const multiboot_info_t *mbi)
+static inline int xsm_multiboot_init (
+    unsigned long *module_map, const multiboot_info_t *mbi)
 {
     return 0;
 }
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index ac553f9c0d..ac8560aade 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -117,8 +117,8 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
 }
 
 #ifdef CONFIG_MULTIBOOT
-int __init xsm_multiboot_init(unsigned long *module_map,
-                              const multiboot_info_t *mbi)
+int __init xsm_multiboot_init(
+    unsigned long *module_map, const multiboot_info_t *mbi)
 {
     int ret = 0;
     void *policy_buffer = NULL;
@@ -128,8 +128,8 @@ int __init xsm_multiboot_init(unsigned long *module_map,
 
     if ( XSM_MAGIC )
     {
-        ret = xsm_multiboot_policy_init(module_map, mbi,
-                                        &policy_buffer, &policy_size);
+        ret = xsm_multiboot_policy_init(module_map, mbi, &policy_buffer,
+                                        &policy_size);
         if ( ret )
         {
             bootstrap_map(NULL);
diff --git a/xen/xsm/xsm_policy.c b/xen/xsm/xsm_policy.c
index 33ab37717f..8dafbc9381 100644
--- a/xen/xsm/xsm_policy.c
+++ b/xen/xsm/xsm_policy.c
@@ -30,10 +30,9 @@
 #endif
 
 #ifdef CONFIG_MULTIBOOT
-int __init xsm_multiboot_policy_init(unsigned long *module_map,
-                                     const multiboot_info_t *mbi,
-                                     void **policy_buffer,
-                                     size_t *policy_size)
+int __init xsm_multiboot_policy_init(
+    unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size)
 {
     int i;
     module_t *mod = (module_t *)__va(mbi->mods_addr);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:14:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:14:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184615.333334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAS-0001C5-6v; Fri, 10 Sep 2021 16:14:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184615.333334; Fri, 10 Sep 2021 16:14:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAS-0001Bv-0p; Fri, 10 Sep 2021 16:14:20 +0000
Received: by outflank-mailman (input) for mailman id 184615;
 Fri, 10 Sep 2021 16:14:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjAQ-0001BB-KZ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:14:18 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 24e84592-1252-11ec-b254-12813bfff9fa;
 Fri, 10 Sep 2021 16:14:16 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290380841623.9985037915408;
 Fri, 10 Sep 2021 09:13:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24e84592-1252-11ec-b254-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631290384; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=YUjP6pZpyZzKSDkQxMcFN8AM+Fpfob0GWW5OYLm5aFDe/bw0s54SGqCotW53qSKMp/z7HnLTRQYeVFh5OjOLesmuD9ZVGOW5gw/lUhTC3oREh9rjtf5U1NLVBc2QznI+U+APOqb8QRweNlY12TMQZfW4ai3piCRQXw9LaFEIQrw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290384; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=62lCiJwjowOdYP+QWrqMZ3ZE9UsyiwSgMGIsRjZIrF0=; 
	b=crNdhpjaOldbFu6s12AxAmXk3KALxH2oxz0KVoRhld3sgC8cjZM9O7Siqd0P51uuokzYWCcQmGGix0O+H4qvrByXx/KMTW7lGT92pmdHc1OE6jLZyu92cmaKL+uRQ2ytc2ar17SLrNtG9qduYzpiwDpRutEsW9JL6TRdm0L/5nM=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290384;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=62lCiJwjowOdYP+QWrqMZ3ZE9UsyiwSgMGIsRjZIrF0=;
	b=KuzWJLYEID8pEzQq2X+tg32FUxoeVFk3dccPXFRKQAdOQ3iE4Y1WSdu/JwPS+C4n
	o/ikkb2iCMNXw6Xe+4iyXnP71TGvTimgqtuyO70n3pmpJUzUq/5rh9GNiM+Kwi4rJhz
	3KRmc+rTCObhRguD+IeejtE2GBufnoZwDd3xmApM=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 06/10] xsm: refactor xsm_ops handling
Date: Fri, 10 Sep 2021 16:13:01 -0400
Message-Id: <20210910201305.32526-7-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

This renames the `struct xsm_operations` to the shorter `struct xsm_ops` and
converts the global xsm_ops from being a pointer to an explicit instance. As
part of this conversion, it reworks the XSM modules init function to return
their xsm_ops struct which is copied in to the global xsm_ops instance.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xsm/xsm.h | 220 +++++++++++++++++++++---------------------
 xen/xsm/dummy.c       |   4 +-
 xen/xsm/flask/hooks.c |  12 +--
 xen/xsm/silo.c        |   7 +-
 xen/xsm/xsm_core.c    |  72 +++++++-------
 5 files changed, 161 insertions(+), 154 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 7673d605a6..556903aa21 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -41,7 +41,7 @@ enum xsm_default {
 };
 typedef enum xsm_default xsm_default_t;
 
-struct xsm_operations {
+struct xsm_ops {
     void (*security_domaininfo)(struct domain *d,
                                 struct xen_domctl_getdomaininfo *info);
     int (*domain_create)(struct domain *d, uint32_t ssidref);
@@ -195,322 +195,322 @@ struct xsm_operations {
 
 #ifdef CONFIG_XSM
 
-extern struct xsm_operations *xsm_ops;
+extern struct xsm_ops xsm_ops;
 
 #ifndef XSM_NO_WRAPPERS
 
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops->security_domaininfo(d, info);
+    xsm_ops.security_domaininfo(d, info);
 }
 
 static inline int xsm_domain_create(
     xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
-    return xsm_ops->domain_create(d, ssidref);
+    return xsm_ops.domain_create(d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->getdomaininfo(d);
+    return xsm_ops.getdomaininfo(d);
 }
 
 static inline int xsm_domctl_scheduler_op(
     xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->domctl_scheduler_op(d, cmd);
+    return xsm_ops.domctl_scheduler_op(d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl_scheduler_op(cmd);
+    return xsm_ops.sysctl_scheduler_op(cmd);
 }
 
 static inline int xsm_set_target(
     xsm_default_t def, struct domain *d, struct domain *e)
 {
-    return xsm_ops->set_target(d, e);
+    return xsm_ops.set_target(d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->domctl(d, cmd);
+    return xsm_ops.domctl(d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops->sysctl(cmd);
+    return xsm_ops.sysctl(cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops->readconsole(clear);
+    return xsm_ops.readconsole(clear);
 }
 
 static inline int xsm_evtchn_unbound(
     xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops->evtchn_unbound(d1, chn, id2);
+    return xsm_ops.evtchn_unbound(d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
     xsm_default_t def, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
-    return xsm_ops->evtchn_interdomain(d1, chan1, d2, chan2);
+    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops->evtchn_close_post(chn);
+    xsm_ops.evtchn_close_post(chn);
 }
 
 static inline int xsm_evtchn_send(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops->evtchn_send(d, chn);
+    return xsm_ops.evtchn_send(d, chn);
 }
 
 static inline int xsm_evtchn_status(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops->evtchn_status(d, chn);
+    return xsm_ops.evtchn_status(d, chn);
 }
 
 static inline int xsm_evtchn_reset(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->evtchn_reset(d1, d2);
+    return xsm_ops.evtchn_reset(d1, d2);
 }
 
 static inline int xsm_grant_mapref(
     xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
-    return xsm_ops->grant_mapref(d1, d2, flags);
+    return xsm_ops.grant_mapref(d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_unmapref(d1, d2);
+    return xsm_ops.grant_unmapref(d1, d2);
 }
 
 static inline int xsm_grant_setup(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_setup(d1, d2);
+    return xsm_ops.grant_setup(d1, d2);
 }
 
 static inline int xsm_grant_transfer(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_transfer(d1, d2);
+    return xsm_ops.grant_transfer(d1, d2);
 }
 
 static inline int xsm_grant_copy(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_copy(d1, d2);
+    return xsm_ops.grant_copy(d1, d2);
 }
 
 static inline int xsm_grant_query_size(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->grant_query_size(d1, d2);
+    return xsm_ops.grant_query_size(d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops->alloc_security_domain(d);
+    return xsm_ops.alloc_security_domain(d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops->free_security_domain(d);
+    xsm_ops.free_security_domain(d);
 }
 
 static inline int xsm_alloc_security_evtchns(
     struct evtchn chn[], unsigned int nr)
 {
-    return xsm_ops->alloc_security_evtchns(chn, nr);
+    return xsm_ops.alloc_security_evtchns(chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(
     struct evtchn chn[], unsigned int nr)
 {
-    xsm_ops->free_security_evtchns(chn, nr);
+    xsm_ops.free_security_evtchns(chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(
     struct domain *d, const struct evtchn *chn)
 {
-    return xsm_ops->show_security_evtchn(d, chn);
+    return xsm_ops.show_security_evtchn(d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->init_hardware_domain(d);
+    return xsm_ops.init_hardware_domain(d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_pod_target(d);
+    return xsm_ops.get_pod_target(d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->set_pod_target(d);
+    return xsm_ops.set_pod_target(d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->memory_exchange(d);
+    return xsm_ops.memory_exchange(d);
 }
 
 static inline int xsm_memory_adjust_reservation(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->memory_adjust_reservation(d1, d2);
+    return xsm_ops.memory_adjust_reservation(d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->memory_stat_reservation(d1, d2);
+    return xsm_ops.memory_stat_reservation(d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
     xsm_default_t def, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
-    return xsm_ops->memory_pin_page(d1, d2, page);
+    return xsm_ops.memory_pin_page(d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->add_to_physmap(d1, d2);
+    return xsm_ops.add_to_physmap(d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->remove_from_physmap(d1, d2);
+    return xsm_ops.remove_from_physmap(d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
     xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops->map_gmfn_foreign(d, t);
+    return xsm_ops.map_gmfn_foreign(d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->claim_pages(d);
+    return xsm_ops.claim_pages(d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->console_io(d, cmd);
+    return xsm_ops.console_io(d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops->profile(d, op);
+    return xsm_ops.profile(d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops->kexec();
+    return xsm_ops.kexec();
 }
 
 static inline int xsm_schedop_shutdown(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops->schedop_shutdown(d1, d2);
+    return xsm_ops.schedop_shutdown(d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops->show_irq_sid(irq);
+    return xsm_ops.show_irq_sid(irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->map_domain_pirq(d);
+    return xsm_ops.map_domain_pirq(d);
 }
 
 static inline int xsm_map_domain_irq(
     xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops->map_domain_irq(d, irq, data);
+    return xsm_ops.map_domain_irq(d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->unmap_domain_pirq(d);
+    return xsm_ops.unmap_domain_pirq(d);
 }
 
 static inline int xsm_unmap_domain_irq(
     xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops->unmap_domain_irq(d, irq, data);
+    return xsm_ops.unmap_domain_irq(d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
     xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->bind_pt_irq(d, bind);
+    return xsm_ops.bind_pt_irq(d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
     xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops->unbind_pt_irq(d, bind);
+    return xsm_ops.unbind_pt_irq(d, bind);
 }
 
 static inline int xsm_irq_permission(
     xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
-    return xsm_ops->irq_permission(d, pirq, allow);
+    return xsm_ops.irq_permission(d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
     xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_permission(d, s, e, allow);
+    return xsm_ops.iomem_permission(d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
     xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops->iomem_mapping(d, s, e, allow);
+    return xsm_ops.iomem_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
     uint16_t end, uint8_t access)
 {
-    return xsm_ops->pci_config_permission(d, machine_bdf, start, end, access);
+    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->get_device_group(machine_bdf);
+    return xsm_ops.get_device_group(machine_bdf);
 }
 
 static inline int xsm_assign_device(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops->assign_device(d, machine_bdf);
+    return xsm_ops.assign_device(d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops->deassign_device(d, machine_bdf);
+    return xsm_ops.deassign_device(d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -518,249 +518,249 @@ static inline int xsm_deassign_device(
 static inline int xsm_assign_dtdevice(
     xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops->assign_dtdevice(d, dtpath);
+    return xsm_ops.assign_dtdevice(d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
     xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops->deassign_dtdevice(d, dtpath);
+    return xsm_ops.deassign_dtdevice(d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_plug_pci(machine_bdf);
+    return xsm_ops.resource_plug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
     xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_unplug_pci(machine_bdf);
+    return xsm_ops.resource_unplug_pci(machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_plug_core();
+    return xsm_ops.resource_plug_core();
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops->resource_unplug_core();
+    return xsm_ops.resource_unplug_core();
 }
 
 static inline int xsm_resource_setup_pci(
     xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops->resource_setup_pci(machine_bdf);
+    return xsm_ops.resource_setup_pci(machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops->resource_setup_gsi(gsi);
+    return xsm_ops.resource_setup_gsi(gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops->resource_setup_misc();
+    return xsm_ops.resource_setup_misc();
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops->page_offline(cmd);
+    return xsm_ops.page_offline(cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops->hypfs_op();
+    return xsm_ops.hypfs_op();
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_xsm_op(op);
+    return xsm_ops.do_xsm_op(op);
 }
 
 #ifdef CONFIG_COMPAT
 static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-    return xsm_ops->do_compat_op(op);
+    return xsm_ops.do_compat_op(op);
 }
 #endif
 
 static inline int xsm_hvm_param(
     xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops->hvm_param(d, op);
+    return xsm_ops.hvm_param(d, op);
 }
 
 static inline int xsm_hvm_control(
     xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops->hvm_control(d, op);
+    return xsm_ops.hvm_control(d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->hvm_param_altp2mhvm(d);
+    return xsm_ops.hvm_param_altp2mhvm(d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
     xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
-    return xsm_ops->hvm_altp2mhvm_op(d, mode, op);
+    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->get_vnumainfo(d);
+    return xsm_ops.get_vnumainfo(d);
 }
 
 static inline int xsm_vm_event_control(
     xsm_default_t def, struct domain *d, int mode, int op)
 {
-    return xsm_ops->vm_event_control(d, mode, op);
+    return xsm_ops.vm_event_control(d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_access(d);
+    return xsm_ops.mem_access(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_paging(d);
+    return xsm_ops.mem_paging(d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->mem_sharing(d);
+    return xsm_ops.mem_sharing(d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->platform_op(op);
+    return xsm_ops.platform_op(op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops->do_mca();
+    return xsm_ops.do_mca();
 }
 
 static inline int xsm_shadow_control(
     xsm_default_t def, struct domain *d, uint32_t op)
 {
-    return xsm_ops->shadow_control(d, op);
+    return xsm_ops.shadow_control(d, op);
 }
 
 static inline int xsm_mem_sharing_op(
     xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
-    return xsm_ops->mem_sharing_op(d, cd, op);
+    return xsm_ops.mem_sharing_op(d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops->apic(d, cmd);
+    return xsm_ops.apic(d, cmd);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops->machine_memory_map();
+    return xsm_ops.machine_memory_map();
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_memory_map(d);
+    return xsm_ops.domain_memory_map(d);
 }
 
 static inline int xsm_mmu_update(
     xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
-    return xsm_ops->mmu_update(d, t, f, flags);
+    return xsm_ops.mmu_update(d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
     xsm_default_t def, struct domain *d, struct domain *f)
 {
-    return xsm_ops->mmuext_op(d, f);
+    return xsm_ops.mmuext_op(d, f);
 }
 
 static inline int xsm_update_va_mapping(
     xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
-    return xsm_ops->update_va_mapping(d, f, pte);
+    return xsm_ops.update_va_mapping(d, f, pte);
 }
 
 static inline int xsm_priv_mapping(
     xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops->priv_mapping(d, t);
+    return xsm_ops.priv_mapping(d, t);
 }
 
 static inline int xsm_ioport_permission(
     xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_permission(d, s, e, allow);
+    return xsm_ops.ioport_permission(d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
     xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops->ioport_mapping(d, s, e, allow);
+    return xsm_ops.ioport_mapping(d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
     xsm_default_t def, struct domain *d, unsigned int op)
 {
-    return xsm_ops->pmu_op(d, op);
+    return xsm_ops.pmu_op(d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->dm_op(d);
+    return xsm_ops.dm_op(d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops->xen_version(op);
+    return xsm_ops.xen_version(op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops->domain_resource_map(d);
+    return xsm_ops.domain_resource_map(d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops->argo_enable(d);
+    return xsm_ops.argo_enable(d);
 }
 
 static inline int xsm_argo_register_single_source(
     const struct domain *d, const struct domain *t)
 {
-    return xsm_ops->argo_register_single_source(d, t);
+    return xsm_ops.argo_register_single_source(d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops->argo_register_any_source(d);
+    return xsm_ops.argo_register_any_source(d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops->argo_send(d, t);
+    return xsm_ops.argo_send(d, t);
 }
 
 #endif /* CONFIG_ARGO */
@@ -786,17 +786,18 @@ int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
 bool has_xsm_magic(paddr_t);
 #endif
 
-int register_xsm(struct xsm_operations *ops);
+extern struct xsm_ops dummy_xsm_ops;
 
-extern struct xsm_operations dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_operations *ops);
+void xsm_fixup_ops(struct xsm_ops *ops);
 
 #ifdef CONFIG_XSM_FLASK
-void flask_init(const void *policy_buffer, size_t policy_size);
+extern const struct xsm_ops *flask_init(const void *policy_buffer,
+                                        size_t policy_size);
 #else
-static inline void flask_init(const void *policy_buffer, size_t policy_size)
+static const inline struct xsm_ops *flask_init(const void *policy_buffer,
+                                               size_t policy_size)
 {
+    return NULL;
 }
 #endif
 
@@ -806,9 +807,12 @@ extern const unsigned int xsm_flask_init_policy_size;
 #endif
 
 #ifdef CONFIG_XSM_SILO
-void silo_init(void);
+extern const struct xsm_ops *silo_init(void);
 #else
-static inline void silo_init(void) {}
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
 #endif
 
 #else /* CONFIG_XSM */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index de44b10130..d8c935328e 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -13,15 +13,13 @@
 #define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
-struct xsm_operations dummy_xsm_ops;
-
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
         if ( !ops->function )                                          \
             ops->function = xsm_##function;                            \
     } while (0)
 
-void __init xsm_fixup_ops (struct xsm_operations *ops)
+void __init xsm_fixup_ops (struct xsm_ops *ops)
 {
     set_to_dummy_if_null(ops, security_domaininfo);
     set_to_dummy_if_null(ops, domain_create);
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 1465db125a..25e87180b4 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1745,7 +1745,7 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
 long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
 
-static struct xsm_operations flask_ops = {
+static const struct xsm_ops __initconstrel flask_ops = {
     .security_domaininfo = flask_security_domaininfo,
     .domain_create = flask_domain_create,
     .getdomaininfo = flask_getdomaininfo,
@@ -1883,7 +1883,8 @@ static struct xsm_operations flask_ops = {
 #endif
 };
 
-void __init flask_init(const void *policy_buffer, size_t policy_size)
+const struct xsm_ops *__init flask_init(const void *policy_buffer,
+                                        size_t policy_size)
 {
     int ret = -ENOENT;
 
@@ -1891,7 +1892,7 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
     {
     case FLASK_BOOTPARAM_DISABLED:
         printk(XENLOG_INFO "Flask: Disabled at boot.\n");
-        return;
+        return NULL;
 
     case FLASK_BOOTPARAM_PERMISSIVE:
         flask_enforcing = 0;
@@ -1908,9 +1909,6 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
 
     avc_init();
 
-    if ( register_xsm(&flask_ops) )
-        panic("Flask: Unable to register with XSM\n");
-
     if ( policy_size && flask_bootparam != FLASK_BOOTPARAM_LATELOAD )
         ret = security_load_policy(policy_buffer, policy_size);
 
@@ -1923,6 +1921,8 @@ void __init flask_init(const void *policy_buffer, size_t policy_size)
         printk(XENLOG_INFO "Flask:  Starting in enforcing mode.\n");
     else
         printk(XENLOG_INFO "Flask:  Starting in permissive mode.\n");
+
+    return &flask_ops;
 }
 
 /*
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index fc2ca5cd2d..3550dded7b 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -100,7 +100,7 @@ static int silo_argo_send(const struct domain *d1, const struct domain *d2)
 
 #endif
 
-static struct xsm_operations silo_xsm_ops = {
+static const struct xsm_ops __initconstrel silo_xsm_ops = {
     .evtchn_unbound = silo_evtchn_unbound,
     .evtchn_interdomain = silo_evtchn_interdomain,
     .grant_mapref = silo_grant_mapref,
@@ -112,12 +112,11 @@ static struct xsm_operations silo_xsm_ops = {
 #endif
 };
 
-void __init silo_init(void)
+const struct xsm_ops *__init silo_init(void)
 {
     printk("Initialising XSM SILO mode\n");
 
-    if ( register_xsm(&silo_xsm_ops) )
-        panic("SILO: Unable to register with XSM\n");
+    return &silo_xsm_ops;
 }
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index ac8560aade..21fffbcb41 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -28,9 +28,17 @@
 #include <asm/setup.h>
 #endif
 
-#define XSM_FRAMEWORK_VERSION    "1.0.0"
+#define XSM_FRAMEWORK_VERSION    "1.0.1"
 
-struct xsm_operations *xsm_ops;
+struct xsm_ops __read_mostly xsm_ops;
+
+enum xsm_ops_state {
+    XSM_OPS_UNREGISTERED,
+    XSM_OPS_REG_FAILED,
+    XSM_OPS_REGISTERED,
+};
+
+static enum xsm_ops_state __initdata xsm_ops_registered = XSM_OPS_UNREGISTERED;
 
 enum xsm_bootparam {
     XSM_BOOTPARAM_DUMMY,
@@ -68,17 +76,10 @@ static int __init parse_xsm_param(const char *s)
 }
 custom_param("xsm", parse_xsm_param);
 
-static inline int verify(struct xsm_operations *ops)
-{
-    /* verify the security_operations structure exists */
-    if ( !ops )
-        return -EINVAL;
-    xsm_fixup_ops(ops);
-    return 0;
-}
-
 static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
 {
+    const struct xsm_ops *ops = NULL;
+
 #ifdef CONFIG_XSM_FLASK_POLICY
     if ( policy_size == 0 )
     {
@@ -87,25 +88,25 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
     }
 #endif
 
-    if ( verify(&dummy_xsm_ops) )
+    if ( xsm_ops_registered != XSM_OPS_UNREGISTERED )
     {
-        printk(XENLOG_ERR "Could not verify dummy_xsm_ops structure\n");
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register already attempted\n");
         return -EIO;
     }
 
-    xsm_ops = &dummy_xsm_ops;
-
     switch ( xsm_bootparam )
     {
     case XSM_BOOTPARAM_DUMMY:
+        xsm_ops_registered = XSM_OPS_REGISTERED;
         break;
 
     case XSM_BOOTPARAM_FLASK:
-        flask_init(policy_buffer, policy_size);
+        ops = flask_init(policy_buffer, policy_size);
         break;
 
     case XSM_BOOTPARAM_SILO:
-        silo_init();
+        ops = silo_init();
         break;
 
     default:
@@ -113,6 +114,27 @@ static int __init xsm_core_init(const void *policy_buffer, size_t policy_size)
         break;
     }
 
+    if ( ops )
+    {
+        xsm_ops_registered = XSM_OPS_REGISTERED;
+        xsm_ops = *ops;
+    }
+    /*
+     * This handles three cases,
+     *   - dummy policy module was selected
+     *   - a policy module does not provide all handlers
+     *   - a policy module failed to init
+     */
+    xsm_fixup_ops(&xsm_ops);
+
+    if ( xsm_ops_registered != XSM_OPS_REGISTERED )
+    {
+        xsm_ops_registered = XSM_OPS_REG_FAILED;
+        printk(XENLOG_ERR
+               "Could not init XSM, xsm_ops register failed\n");
+        return -EFAULT;
+    }
+
     return 0;
 }
 
@@ -195,22 +217,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-int __init register_xsm(struct xsm_operations *ops)
-{
-    if ( verify(ops) )
-    {
-        printk(XENLOG_ERR "Could not verify xsm_operations structure\n");
-        return -EINVAL;
-    }
-
-    if ( xsm_ops != &dummy_xsm_ops )
-        return -EAGAIN;
-
-    xsm_ops = ops;
-
-    return 0;
-}
-
 #endif
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:14:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:14:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184618.333344 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAY-0001cj-Iq; Fri, 10 Sep 2021 16:14:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184618.333344; Fri, 10 Sep 2021 16:14:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAY-0001cH-FH; Fri, 10 Sep 2021 16:14:26 +0000
Received: by outflank-mailman (input) for mailman id 184618;
 Fri, 10 Sep 2021 16:14:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjAX-0001bc-QQ
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:14:25 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2923eaf9-1252-11ec-b254-12813bfff9fa;
 Fri, 10 Sep 2021 16:14:24 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 163129038197236.20358463523735;
 Fri, 10 Sep 2021 09:13:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2923eaf9-1252-11ec-b254-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631290385; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=fqgyhdyjgF8V/9UEth2NtEpYfmyh/zBDvgDqe6vPUYZdMs25Q0j4i0+/NiJE/Y1QKzGH9xbYCDvbuplZrOgi48uVrCLK5AbIsjPH47+r1x+rKcAA8nzuBCIwtrXB2KMiuDYbl2etOQdtyeYjzAJfRqgTD39gy5up1nZ1C3tXeFo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290385; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=/B6/3FHiNB3IVMfbChqH0I39/uMG5o4Sw9RJ3RiDpic=; 
	b=TZCEN+9c43tDj+4tKPJz0BWXpI3VMBNCD3qQpqNhljbX+PlmFGrPTQQa79VUfeMjZ1nyX9Zrm7nlSZtMhqo/mgKJOwv2+L/bLGdm5sN0b4kvq+TKBM1kUb99mauCAlhaP0SOa5GJp+b/sIc6qnaMssIKbkXVtgdap1xJrRLL3lE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290385;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=/B6/3FHiNB3IVMfbChqH0I39/uMG5o4Sw9RJ3RiDpic=;
	b=SohyymauANCXM83TXc/1JCgAUM8EkyBtdVK4c2KWVFs3yhPHZFy8VAJiqzHwEB/n
	SsaRy+aj451lNc4UzzgkWMRiuAyypCaNoUbVhtsvQ1EfvL/S9duj+LUAM9Lm9gdg2Fq
	Un/7wgXLJ0JEOigp2b8wPq10dJwd5ZGmHDFzifkw=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 07/10] xsm: convert xsm_ops hook calls to alternative call
Date: Fri, 10 Sep 2021 16:13:02 -0400
Message-Id: <20210910201305.32526-8-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

To reduce retpolines convert all the pointer function calls of the
xsm_ops hooks over to the alternative_call infrastructure.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/include/xsm/xsm.h | 191 +++++++++++++++++++++---------------------
 1 file changed, 96 insertions(+), 95 deletions(-)

diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 556903aa21..c101e653f6 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -15,6 +15,7 @@
 #ifndef __XSM_H__
 #define __XSM_H__
 
+#include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
@@ -202,315 +203,315 @@ extern struct xsm_ops xsm_ops;
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
-    xsm_ops.security_domaininfo(d, info);
+    alternative_vcall(xsm_ops.security_domaininfo, d, info);
 }
 
 static inline int xsm_domain_create(
     xsm_default_t def, struct domain *d, uint32_t ssidref)
 {
-    return xsm_ops.domain_create(d, ssidref);
+    return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
 static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.getdomaininfo(d);
+    return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
 static inline int xsm_domctl_scheduler_op(
     xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.domctl_scheduler_op(d, cmd);
+    return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
 static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl_scheduler_op(cmd);
+    return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
 static inline int xsm_set_target(
     xsm_default_t def, struct domain *d, struct domain *e)
 {
-    return xsm_ops.set_target(d, e);
+    return alternative_call(xsm_ops.set_target, d, e);
 }
 
 static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.domctl(d, cmd);
+    return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
 static inline int xsm_sysctl(xsm_default_t def, int cmd)
 {
-    return xsm_ops.sysctl(cmd);
+    return alternative_call(xsm_ops.sysctl, cmd);
 }
 
 static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
 {
-    return xsm_ops.readconsole(clear);
+    return alternative_call(xsm_ops.readconsole, clear);
 }
 
 static inline int xsm_evtchn_unbound(
     xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
-    return xsm_ops.evtchn_unbound(d1, chn, id2);
+    return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
     xsm_default_t def, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
-    return xsm_ops.evtchn_interdomain(d1, chan1, d2, chan2);
+    return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
 static inline void xsm_evtchn_close_post(struct evtchn *chn)
 {
-    xsm_ops.evtchn_close_post(chn);
+    alternative_vcall(xsm_ops.evtchn_close_post, chn);
 }
 
 static inline int xsm_evtchn_send(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops.evtchn_send(d, chn);
+    return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
 static inline int xsm_evtchn_status(
     xsm_default_t def, struct domain *d, struct evtchn *chn)
 {
-    return xsm_ops.evtchn_status(d, chn);
+    return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
 static inline int xsm_evtchn_reset(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.evtchn_reset(d1, d2);
+    return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
 static inline int xsm_grant_mapref(
     xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
 {
-    return xsm_ops.grant_mapref(d1, d2, flags);
+    return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_unmapref(d1, d2);
+    return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
 static inline int xsm_grant_setup(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_setup(d1, d2);
+    return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
 static inline int xsm_grant_transfer(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_transfer(d1, d2);
+    return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
 static inline int xsm_grant_copy(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_copy(d1, d2);
+    return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
 static inline int xsm_grant_query_size(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.grant_query_size(d1, d2);
+    return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
 static inline int xsm_alloc_security_domain(struct domain *d)
 {
-    return xsm_ops.alloc_security_domain(d);
+    return alternative_call(xsm_ops.alloc_security_domain, d);
 }
 
 static inline void xsm_free_security_domain(struct domain *d)
 {
-    xsm_ops.free_security_domain(d);
+    alternative_vcall(xsm_ops.free_security_domain, d);
 }
 
 static inline int xsm_alloc_security_evtchns(
     struct evtchn chn[], unsigned int nr)
 {
-    return xsm_ops.alloc_security_evtchns(chn, nr);
+    return alternative_call(xsm_ops.alloc_security_evtchns, chn, nr);
 }
 
 static inline void xsm_free_security_evtchns(
     struct evtchn chn[], unsigned int nr)
 {
-    xsm_ops.free_security_evtchns(chn, nr);
+    alternative_vcall(xsm_ops.free_security_evtchns, chn, nr);
 }
 
 static inline char *xsm_show_security_evtchn(
     struct domain *d, const struct evtchn *chn)
 {
-    return xsm_ops.show_security_evtchn(d, chn);
+    return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
 static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.init_hardware_domain(d);
+    return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
 static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_pod_target(d);
+    return alternative_call(xsm_ops.get_pod_target, d);
 }
 
 static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.set_pod_target(d);
+    return alternative_call(xsm_ops.set_pod_target, d);
 }
 
 static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.memory_exchange(d);
+    return alternative_call(xsm_ops.memory_exchange, d);
 }
 
 static inline int xsm_memory_adjust_reservation(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.memory_adjust_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.memory_stat_reservation(d1, d2);
+    return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
     xsm_default_t def, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
-    return xsm_ops.memory_pin_page(d1, d2, page);
+    return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.add_to_physmap(d1, d2);
+    return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.remove_from_physmap(d1, d2);
+    return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
     xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops.map_gmfn_foreign(d, t);
+    return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
 static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.claim_pages(d);
+    return alternative_call(xsm_ops.claim_pages, d);
 }
 
 static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.console_io(d, cmd);
+    return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
 static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
 {
-    return xsm_ops.profile(d, op);
+    return alternative_call(xsm_ops.profile, d, op);
 }
 
 static inline int xsm_kexec(xsm_default_t def)
 {
-    return xsm_ops.kexec();
+    return alternative_call(xsm_ops.kexec);
 }
 
 static inline int xsm_schedop_shutdown(
     xsm_default_t def, struct domain *d1, struct domain *d2)
 {
-    return xsm_ops.schedop_shutdown(d1, d2);
+    return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
 static inline char *xsm_show_irq_sid(int irq)
 {
-    return xsm_ops.show_irq_sid(irq);
+    return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
 static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.map_domain_pirq(d);
+    return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
 static inline int xsm_map_domain_irq(
     xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops.map_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
 static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.unmap_domain_pirq(d);
+    return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
 static inline int xsm_unmap_domain_irq(
     xsm_default_t def, struct domain *d, int irq, void *data)
 {
-    return xsm_ops.unmap_domain_irq(d, irq, data);
+    return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
     xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.bind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
     xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
-    return xsm_ops.unbind_pt_irq(d, bind);
+    return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
 static inline int xsm_irq_permission(
     xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
 {
-    return xsm_ops.irq_permission(d, pirq, allow);
+    return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
     xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
     xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
 {
-    return xsm_ops.iomem_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
     uint16_t end, uint8_t access)
 {
-    return xsm_ops.pci_config_permission(d, machine_bdf, start, end, access);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
 static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.get_device_group(machine_bdf);
+    return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
 static inline int xsm_assign_device(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops.assign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
     xsm_default_t def, struct domain *d, uint32_t machine_bdf)
 {
-    return xsm_ops.deassign_device(d, machine_bdf);
+    return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
@@ -518,62 +519,62 @@ static inline int xsm_deassign_device(
 static inline int xsm_assign_dtdevice(
     xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops.assign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
     xsm_default_t def, struct domain *d, const char *dtpath)
 {
-    return xsm_ops.deassign_dtdevice(d, dtpath);
+    return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
 static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_plug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
     xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_unplug_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_plug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_plug_core();
+    return alternative_call(xsm_ops.resource_plug_core);
 }
 
 static inline int xsm_resource_unplug_core(xsm_default_t def)
 {
-    return xsm_ops.resource_unplug_core();
+    return alternative_call(xsm_ops.resource_unplug_core);
 }
 
 static inline int xsm_resource_setup_pci(
     xsm_default_t def, uint32_t machine_bdf)
 {
-    return xsm_ops.resource_setup_pci(machine_bdf);
+    return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
 static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
 {
-    return xsm_ops.resource_setup_gsi(gsi);
+    return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
 static inline int xsm_resource_setup_misc(xsm_default_t def)
 {
-    return xsm_ops.resource_setup_misc();
+    return alternative_call(xsm_ops.resource_setup_misc);
 }
 
 static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
 {
-    return xsm_ops.page_offline(cmd);
+    return alternative_call(xsm_ops.page_offline, cmd);
 }
 
 static inline int xsm_hypfs_op(xsm_default_t def)
 {
-    return xsm_ops.hypfs_op();
+    return alternative_call(xsm_ops.hypfs_op);
 }
 
 static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
@@ -591,107 +592,107 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 static inline int xsm_hvm_param(
     xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops.hvm_param(d, op);
+    return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
 static inline int xsm_hvm_control(
     xsm_default_t def, struct domain *d, unsigned long op)
 {
-    return xsm_ops.hvm_control(d, op);
+    return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
 static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.hvm_param_altp2mhvm(d);
+    return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
     xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
 {
-    return xsm_ops.hvm_altp2mhvm_op(d, mode, op);
+    return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
 static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.get_vnumainfo(d);
+    return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
 static inline int xsm_vm_event_control(
     xsm_default_t def, struct domain *d, int mode, int op)
 {
-    return xsm_ops.vm_event_control(d, mode, op);
+    return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
 static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_access(d);
+    return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
 static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_paging(d);
+    return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
 static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.mem_sharing(d);
+    return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
 static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.platform_op(op);
+    return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
 static inline int xsm_do_mca(xsm_default_t def)
 {
-    return xsm_ops.do_mca();
+    return alternative_call(xsm_ops.do_mca);
 }
 
 static inline int xsm_shadow_control(
     xsm_default_t def, struct domain *d, uint32_t op)
 {
-    return xsm_ops.shadow_control(d, op);
+    return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
 static inline int xsm_mem_sharing_op(
     xsm_default_t def, struct domain *d, struct domain *cd, int op)
 {
-    return xsm_ops.mem_sharing_op(d, cd, op);
+    return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
 static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
 {
-    return xsm_ops.apic(d, cmd);
+    return alternative_call(xsm_ops.apic, d, cmd);
 }
 
 static inline int xsm_machine_memory_map(xsm_default_t def)
 {
-    return xsm_ops.machine_memory_map();
+    return alternative_call(xsm_ops.machine_memory_map);
 }
 
 static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_memory_map(d);
+    return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
 static inline int xsm_mmu_update(
     xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
-    return xsm_ops.mmu_update(d, t, f, flags);
+    return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
     xsm_default_t def, struct domain *d, struct domain *f)
 {
-    return xsm_ops.mmuext_op(d, f);
+    return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
 static inline int xsm_update_va_mapping(
@@ -703,64 +704,64 @@ static inline int xsm_update_va_mapping(
 static inline int xsm_priv_mapping(
     xsm_default_t def, struct domain *d, struct domain *t)
 {
-    return xsm_ops.priv_mapping(d, t);
+    return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
 static inline int xsm_ioport_permission(
     xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_permission(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
     xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
 {
-    return xsm_ops.ioport_mapping(d, s, e, allow);
+    return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
     xsm_default_t def, struct domain *d, unsigned int op)
 {
-    return xsm_ops.pmu_op(d, op);
+    return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
 static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.dm_op(d);
+    return alternative_call(xsm_ops.dm_op, d);
 }
 
 static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
 {
-    return xsm_ops.xen_version(op);
+    return alternative_call(xsm_ops.xen_version, op);
 }
 
 static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
 {
-    return xsm_ops.domain_resource_map(d);
+    return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
 #ifdef CONFIG_ARGO
 static inline int xsm_argo_enable(const struct domain *d)
 {
-    return xsm_ops.argo_enable(d);
+    return alternative_call(xsm_ops.argo_enable, d);
 }
 
 static inline int xsm_argo_register_single_source(
     const struct domain *d, const struct domain *t)
 {
-    return xsm_ops.argo_register_single_source(d, t);
+    return alternative_call(xsm_ops.argo_register_single_source, d, t);
 }
 
 static inline int xsm_argo_register_any_source(const struct domain *d)
 {
-    return xsm_ops.argo_register_any_source(d);
+    return alternative_call(xsm_ops.argo_register_any_source, d);
 }
 
 static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 {
-    return xsm_ops.argo_send(d, t);
+    return alternative_call(xsm_ops.argo_send, d, t);
 }
 
 #endif /* CONFIG_ARGO */
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:14:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:14:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184630.333356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAn-0002R8-Tx; Fri, 10 Sep 2021 16:14:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184630.333356; Fri, 10 Sep 2021 16:14:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjAn-0002Qz-QM; Fri, 10 Sep 2021 16:14:41 +0000
Received: by outflank-mailman (input) for mailman id 184630;
 Fri, 10 Sep 2021 16:14:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjAm-0002BT-9D
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:14:40 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 313b4880-1252-11ec-b254-12813bfff9fa;
 Fri, 10 Sep 2021 16:14:37 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290383073126.11756631031028;
 Fri, 10 Sep 2021 09:13:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 313b4880-1252-11ec-b254-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631290387; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=kmr7QhAEn2CC6arXUVSB1zGITc8hi+KffyslYNzn1BmOT6DsIFKc+yBImn0Lu33uTcowuP1xaCevrZfNaFRQLPuXiDyMS75IUgOW1s0LSXDczBH2YDXQdygaZbC+e+ElBkGxaPCGwPmnK8iupbhuCtNOQ1/F2hrkGGfRvIrfDpQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290387; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=U4vkiI6jA5JfL4SpY5vIqb7cvn2myZYK53ZNu0HWpvw=; 
	b=DaRw+ncpJUt5UzcyyCcdUiVm5xmcd3T2Gx29P0QntrtkpUpxQjvVEJGxtVXMyf5noTn6vcD40GAzE+Qe6BBbeMkqtCHU2RC660Rj4gqnaTmSGIxfP5wFSjyfV+BJI29TEpQhpiYB/DIaxr17wqC9EKr8ELDqKsbzumbIzFaI22U=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290387;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=U4vkiI6jA5JfL4SpY5vIqb7cvn2myZYK53ZNu0HWpvw=;
	b=FAicfY6jCy3hHmBNgnQIC5bvg6zpCpfSNFZp2cxcRcHkeUvwjIORdzYS1WErMsow
	DRgxU62pNRf35tVQ5MX+ghtlC9ZZOUsBNEcQIKZlSWsPVA0wpfO7MMhIVCm4CLlL/xD
	01rn4Qz3ciHd94apLhcRrfGx1+feqWsdYzyLC8ac=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 08/10] xsm: decouple xsm header inclusion selection
Date: Fri, 10 Sep 2021 16:13:03 -0400
Message-Id: <20210910201305.32526-9-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Multiple preprocessor defines were used as a mechanism to selective include
parts of the xsm.h header file. This makes it difficult to know which portion
is being included at any one time. This commit works to simplify this by
separating the core structures and functions of XSM into xsm-core.h away from
the wrapper functions which remain in xsm.h and dummy.h.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/dummy.h    |   2 +-
 xen/include/xsm/xsm-core.h | 267 +++++++++++++++++++++++++++++++++++++
 xen/include/xsm/xsm.h      | 248 +---------------------------------
 xen/xsm/dummy.c            |   1 -
 xen/xsm/silo.c             |   2 +-
 xen/xsm/xsm_core.c         |   5 +
 6 files changed, 275 insertions(+), 250 deletions(-)
 create mode 100644 xen/include/xsm/xsm-core.h

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 3b1b378b58..be8877a195 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -16,7 +16,7 @@
  */
 
 #include <xen/sched.h>
-#include <xsm/xsm.h>
+#include <xsm/xsm-core.h>
 #include <public/hvm/params.h>
 
 /*
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
new file mode 100644
index 0000000000..3c4a25d7fc
--- /dev/null
+++ b/xen/include/xsm/xsm-core.h
@@ -0,0 +1,267 @@
+/*
+ *  This file contains the XSM hook definitions for Xen.
+ *
+ *  This work is based on the LSM implementation in Linux 2.6.13.4.
+ *
+ *  Author:  George Coker, <gscoker@alpha.ncsc.mil>
+ *
+ *  Contributors: Michael LeMay, <mdlemay@epoch.ncsc.mil>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ */
+
+#ifndef __XSM_CORE_H__
+#define __XSM_CORE_H__
+
+#include <xen/sched.h>
+#include <xen/multiboot.h>
+
+/* policy magic number */
+typedef uint32_t xsm_magic_t;
+
+#ifdef CONFIG_XSM_FLASK
+#define XSM_MAGIC 0xf97cff8c
+#else
+#define XSM_MAGIC 0x0
+#endif
+
+/*
+ * These annotations are used by callers and in dummy.h to document the
+ * default actions of XSM hooks. They should be compiled out otherwise.
+ */
+enum xsm_default {
+    XSM_HOOK,     /* Guests can normally access the hypercall */
+    XSM_DM_PRIV,  /* Device model can perform on its target domain */
+    XSM_TARGET,   /* Can perform on self or your target domain */
+    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
+    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
+    XSM_OTHER     /* Something more complex */
+};
+typedef enum xsm_default xsm_default_t;
+
+struct xsm_ops {
+    void (*security_domaininfo)(struct domain *d,
+                                struct xen_domctl_getdomaininfo *info);
+    int (*domain_create)(struct domain *d, uint32_t ssidref);
+    int (*getdomaininfo)(struct domain *d);
+    int (*domctl_scheduler_op)(struct domain *d, int op);
+    int (*sysctl_scheduler_op)(int op);
+    int (*set_target)(struct domain *d, struct domain *e);
+    int (*domctl)(struct domain *d, int cmd);
+    int (*sysctl)(int cmd);
+    int (*readconsole)(uint32_t clear);
+
+    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
+    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
+                              struct domain *d2, struct evtchn *chn2);
+    void (*evtchn_close_post)(struct evtchn *chn);
+    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
+    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
+
+    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
+    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
+    int (*grant_setup)(struct domain *d1, struct domain *d2);
+    int (*grant_transfer)(struct domain *d1, struct domain *d2);
+    int (*grant_copy)(struct domain *d1, struct domain *d2);
+    int (*grant_query_size)(struct domain *d1, struct domain *d2);
+
+    int (*alloc_security_domain)(struct domain *d);
+    void (*free_security_domain)(struct domain *d);
+    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
+    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
+    int (*init_hardware_domain)(struct domain *d);
+
+    int (*get_pod_target)(struct domain *d);
+    int (*set_pod_target)(struct domain *d);
+    int (*memory_exchange)(struct domain *d);
+    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
+    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
+                           struct page_info *page);
+    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
+    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
+    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
+    int (*claim_pages)(struct domain *d);
+
+    int (*console_io)(struct domain *d, int cmd);
+
+    int (*profile)(struct domain *d, int op);
+
+    int (*kexec)(void);
+    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
+
+    char *(*show_irq_sid)(int irq);
+    int (*map_domain_pirq)(struct domain *d);
+    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*unmap_domain_pirq)(struct domain *d);
+    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
+    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
+    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
+    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
+                            uint8_t allow);
+    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
+                         uint8_t allow);
+    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
+                                 uint16_t start, uint16_t end, uint8_t access);
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+    int (*get_device_group)(uint32_t machine_bdf);
+    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
+    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
+#endif
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
+    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
+#endif
+
+    int (*resource_plug_core)(void);
+    int (*resource_unplug_core)(void);
+    int (*resource_plug_pci)(uint32_t machine_bdf);
+    int (*resource_unplug_pci)(uint32_t machine_bdf);
+    int (*resource_setup_pci)(uint32_t machine_bdf);
+    int (*resource_setup_gsi)(int gsi);
+    int (*resource_setup_misc)(void);
+
+    int (*page_offline)(uint32_t cmd);
+    int (*hypfs_op)(void);
+
+    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#ifdef CONFIG_COMPAT
+    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
+#endif
+
+    int (*hvm_param)(struct domain *d, unsigned long op);
+    int (*hvm_control)(struct domain *d, unsigned long op);
+    int (*hvm_param_altp2mhvm)(struct domain *d);
+    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
+    int (*get_vnumainfo)(struct domain *d);
+
+    int (*vm_event_control)(struct domain *d, int mode, int op);
+
+#ifdef CONFIG_MEM_ACCESS
+    int (*mem_access)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+    int (*mem_paging)(struct domain *d);
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+    int (*mem_sharing)(struct domain *d);
+#endif
+
+    int (*platform_op)(uint32_t cmd);
+
+#ifdef CONFIG_X86
+    int (*do_mca)(void);
+    int (*shadow_control)(struct domain *d, uint32_t op);
+    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
+    int (*apic)(struct domain *d, int cmd);
+    int (*machine_memory_map)(void);
+    int (*domain_memory_map)(struct domain *d);
+#define XSM_MMU_UPDATE_READ      1
+#define XSM_MMU_UPDATE_WRITE     2
+#define XSM_MMU_NORMAL_UPDATE    4
+#define XSM_MMU_MACHPHYS_UPDATE  8
+    int (*mmu_update)(struct domain *d, struct domain *t,
+                      struct domain *f, uint32_t flags);
+    int (*mmuext_op)(struct domain *d, struct domain *f);
+    int (*update_va_mapping)(struct domain *d, struct domain *f,
+                             l1_pgentry_t pte);
+    int (*priv_mapping)(struct domain *d, struct domain *t);
+    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
+                             uint8_t allow);
+    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
+                          uint8_t allow);
+    int (*pmu_op)(struct domain *d, unsigned int op);
+#endif
+
+    int (*dm_op)(struct domain *d);
+
+    int (*xen_version)(uint32_t cmd);
+
+    int (*domain_resource_map)(struct domain *d);
+
+#ifdef CONFIG_ARGO
+    int (*argo_enable)(const struct domain *d);
+    int (*argo_register_single_source)(const struct domain *d,
+                                       const struct domain *t);
+    int (*argo_register_any_source)(const struct domain *d);
+    int (*argo_send)(const struct domain *d, const struct domain *t);
+#endif
+};
+
+void xsm_fixup_ops(struct xsm_ops *ops);
+
+#ifdef CONFIG_XSM
+
+#ifdef CONFIG_MULTIBOOT
+int xsm_multiboot_init(
+    unsigned long *module_map, const multiboot_info_t *mbi);
+int xsm_multiboot_policy_init(
+    unsigned long *module_map, const multiboot_info_t *mbi,
+    void **policy_buffer, size_t *policy_size);
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+/*
+ * Initialize XSM
+ *
+ * On success, return 1 if using SILO mode else 0.
+ */
+int xsm_dt_init(void);
+int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
+bool has_xsm_magic(paddr_t);
+#endif
+
+#ifdef CONFIG_XSM_FLASK
+const struct xsm_ops *flask_init(
+    const void *policy_buffer, size_t policy_size);
+#else
+static inline const struct xsm_ops *flask_init(
+    const void *policy_buffer, size_t policy_size)
+{
+    return NULL;
+}
+#endif
+
+#ifdef CONFIG_XSM_SILO
+const struct xsm_ops *silo_init(void);
+#else
+static const inline struct xsm_ops *silo_init(void)
+{
+    return NULL;
+}
+#endif
+
+#else /* CONFIG_XSM */
+
+#ifdef CONFIG_MULTIBOOT
+static inline int xsm_multiboot_init(
+    unsigned long *module_map, const multiboot_info_t *mbi)
+{
+    return 0;
+}
+#endif
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+static inline int xsm_dt_init(void)
+{
+    return 0;
+}
+
+static inline bool has_xsm_magic(paddr_t start)
+{
+    return false;
+}
+#endif /* CONFIG_HAS_DEVICE_TREE */
+
+#endif /* CONFIG_XSM */
+
+#endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index c101e653f6..7b1611d10b 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -18,188 +18,12 @@
 #include <xen/alternative-call.h>
 #include <xen/sched.h>
 #include <xen/multiboot.h>
-
-/* policy magic number (defined by XSM_MAGIC) */
-typedef uint32_t xsm_magic_t;
-
-#ifdef CONFIG_XSM_FLASK
-#define XSM_MAGIC 0xf97cff8c
-#else
-#define XSM_MAGIC 0x0
-#endif
-
-/*
- * These annotations are used by callers and in dummy.h to document the
- * default actions of XSM hooks. They should be compiled out otherwise.
- */
-enum xsm_default {
-    XSM_HOOK,     /* Guests can normally access the hypercall */
-    XSM_DM_PRIV,  /* Device model can perform on its target domain */
-    XSM_TARGET,   /* Can perform on self or your target domain */
-    XSM_PRIV,     /* Privileged - normally restricted to dom0 */
-    XSM_XS_PRIV,  /* Xenstore domain - can do some privileged operations */
-    XSM_OTHER     /* Something more complex */
-};
-typedef enum xsm_default xsm_default_t;
-
-struct xsm_ops {
-    void (*security_domaininfo)(struct domain *d,
-                                struct xen_domctl_getdomaininfo *info);
-    int (*domain_create)(struct domain *d, uint32_t ssidref);
-    int (*getdomaininfo)(struct domain *d);
-    int (*domctl_scheduler_op)(struct domain *d, int op);
-    int (*sysctl_scheduler_op)(int op);
-    int (*set_target)(struct domain *d, struct domain *e);
-    int (*domctl)(struct domain *d, int cmd);
-    int (*sysctl)(int cmd);
-    int (*readconsole)(uint32_t clear);
-
-    int (*evtchn_unbound)(struct domain *d, struct evtchn *chn, domid_t id2);
-    int (*evtchn_interdomain)(struct domain *d1, struct evtchn *chn1,
-                              struct domain *d2, struct evtchn *chn2);
-    void (*evtchn_close_post)(struct evtchn *chn);
-    int (*evtchn_send)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_status)(struct domain *d, struct evtchn *chn);
-    int (*evtchn_reset)(struct domain *d1, struct domain *d2);
-
-    int (*grant_mapref)(struct domain *d1, struct domain *d2, uint32_t flags);
-    int (*grant_unmapref)(struct domain *d1, struct domain *d2);
-    int (*grant_setup)(struct domain *d1, struct domain *d2);
-    int (*grant_transfer)(struct domain *d1, struct domain *d2);
-    int (*grant_copy)(struct domain *d1, struct domain *d2);
-    int (*grant_query_size)(struct domain *d1, struct domain *d2);
-
-    int (*alloc_security_domain)(struct domain *d);
-    void (*free_security_domain)(struct domain *d);
-    int (*alloc_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    void (*free_security_evtchns)(struct evtchn chn[], unsigned int nr);
-    char *(*show_security_evtchn)(struct domain *d, const struct evtchn *chn);
-    int (*init_hardware_domain)(struct domain *d);
-
-    int (*get_pod_target)(struct domain *d);
-    int (*set_pod_target)(struct domain *d);
-    int (*memory_exchange)(struct domain *d);
-    int (*memory_adjust_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_stat_reservation)(struct domain *d1, struct domain *d2);
-    int (*memory_pin_page)(struct domain *d1, struct domain *d2,
-                           struct page_info *page);
-    int (*add_to_physmap)(struct domain *d1, struct domain *d2);
-    int (*remove_from_physmap)(struct domain *d1, struct domain *d2);
-    int (*map_gmfn_foreign)(struct domain *d, struct domain *t);
-    int (*claim_pages)(struct domain *d);
-
-    int (*console_io)(struct domain *d, int cmd);
-
-    int (*profile)(struct domain *d, int op);
-
-    int (*kexec)(void);
-    int (*schedop_shutdown)(struct domain *d1, struct domain *d2);
-
-    char *(*show_irq_sid)(int irq);
-    int (*map_domain_pirq)(struct domain *d);
-    int (*map_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*unmap_domain_pirq)(struct domain *d);
-    int (*unmap_domain_irq)(struct domain *d, int irq, const void *data);
-    int (*bind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*unbind_pt_irq)(struct domain *d, struct xen_domctl_bind_pt_irq *bind);
-    int (*irq_permission)(struct domain *d, int pirq, uint8_t allow);
-    int (*iomem_permission)(struct domain *d, uint64_t s, uint64_t e,
-                            uint8_t allow);
-    int (*iomem_mapping)(struct domain *d, uint64_t s, uint64_t e,
-                         uint8_t allow);
-    int (*pci_config_permission)(struct domain *d, uint32_t machine_bdf,
-                                 uint16_t start, uint16_t end, uint8_t access);
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-    int (*get_device_group)(uint32_t machine_bdf);
-    int (*assign_device)(struct domain *d, uint32_t machine_bdf);
-    int (*deassign_device)(struct domain *d, uint32_t machine_bdf);
-#endif
-
-#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-    int (*assign_dtdevice)(struct domain *d, const char *dtpath);
-    int (*deassign_dtdevice)(struct domain *d, const char *dtpath);
-#endif
-
-    int (*resource_plug_core)(void);
-    int (*resource_unplug_core)(void);
-    int (*resource_plug_pci)(uint32_t machine_bdf);
-    int (*resource_unplug_pci)(uint32_t machine_bdf);
-    int (*resource_setup_pci)(uint32_t machine_bdf);
-    int (*resource_setup_gsi)(int gsi);
-    int (*resource_setup_misc)(void);
-
-    int (*page_offline)(uint32_t cmd);
-    int (*hypfs_op)(void);
-
-    long (*do_xsm_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#ifdef CONFIG_COMPAT
-    int (*do_compat_op)(XEN_GUEST_HANDLE_PARAM(void) op);
-#endif
-
-    int (*hvm_param)(struct domain *d, unsigned long op);
-    int (*hvm_control)(struct domain *d, unsigned long op);
-    int (*hvm_param_altp2mhvm)(struct domain *d);
-    int (*hvm_altp2mhvm_op)(struct domain *d, uint64_t mode, uint32_t op);
-    int (*get_vnumainfo)(struct domain *d);
-
-    int (*vm_event_control)(struct domain *d, int mode, int op);
-
-#ifdef CONFIG_MEM_ACCESS
-    int (*mem_access)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_PAGING
-    int (*mem_paging)(struct domain *d);
-#endif
-
-#ifdef CONFIG_MEM_SHARING
-    int (*mem_sharing)(struct domain *d);
-#endif
-
-    int (*platform_op)(uint32_t cmd);
-
-#ifdef CONFIG_X86
-    int (*do_mca)(void);
-    int (*shadow_control)(struct domain *d, uint32_t op);
-    int (*mem_sharing_op)(struct domain *d, struct domain *cd, int op);
-    int (*apic)(struct domain *d, int cmd);
-    int (*machine_memory_map)(void);
-    int (*domain_memory_map)(struct domain *d);
-#define XSM_MMU_UPDATE_READ      1
-#define XSM_MMU_UPDATE_WRITE     2
-#define XSM_MMU_NORMAL_UPDATE    4
-#define XSM_MMU_MACHPHYS_UPDATE  8
-    int (*mmu_update)(struct domain *d, struct domain *t,
-                      struct domain *f, uint32_t flags);
-    int (*mmuext_op)(struct domain *d, struct domain *f);
-    int (*update_va_mapping)(struct domain *d, struct domain *f,
-                             l1_pgentry_t pte);
-    int (*priv_mapping)(struct domain *d, struct domain *t);
-    int (*ioport_permission)(struct domain *d, uint32_t s, uint32_t e,
-                             uint8_t allow);
-    int (*ioport_mapping)(struct domain *d, uint32_t s, uint32_t e,
-                          uint8_t allow);
-    int (*pmu_op)(struct domain *d, unsigned int op);
-#endif
-    int (*dm_op)(struct domain *d);
-    int (*xen_version)(uint32_t cmd);
-    int (*domain_resource_map)(struct domain *d);
-#ifdef CONFIG_ARGO
-    int (*argo_enable)(const struct domain *d);
-    int (*argo_register_single_source)(const struct domain *d,
-                                       const struct domain *t);
-    int (*argo_register_any_source)(const struct domain *d);
-    int (*argo_send)(const struct domain *d, const struct domain *t);
-#endif
-};
+#include <xsm/xsm-core.h>
 
 #ifdef CONFIG_XSM
 
 extern struct xsm_ops xsm_ops;
 
-#ifndef XSM_NO_WRAPPERS
-
 static inline void xsm_security_domaininfo(
     struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
@@ -766,80 +590,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#endif /* XSM_NO_WRAPPERS */
-
-#ifdef CONFIG_MULTIBOOT
-int xsm_multiboot_init(
-    unsigned long *module_map, const multiboot_info_t *mbi);
-int xsm_multiboot_policy_init(
-    unsigned long *module_map, const multiboot_info_t *mbi,
-    void **policy_buffer, size_t *policy_size);
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-/*
- * Initialize XSM
- *
- * On success, return 1 if using SILO mode else 0.
- */
-int xsm_dt_init(void);
-int xsm_dt_policy_init(void **policy_buffer, size_t *policy_size);
-bool has_xsm_magic(paddr_t);
-#endif
-
-extern struct xsm_ops dummy_xsm_ops;
-
-void xsm_fixup_ops(struct xsm_ops *ops);
-
-#ifdef CONFIG_XSM_FLASK
-extern const struct xsm_ops *flask_init(const void *policy_buffer,
-                                        size_t policy_size);
-#else
-static const inline struct xsm_ops *flask_init(const void *policy_buffer,
-                                               size_t policy_size)
-{
-    return NULL;
-}
-#endif
-
-#ifdef CONFIG_XSM_FLASK_POLICY
-extern const unsigned char xsm_flask_init_policy[];
-extern const unsigned int xsm_flask_init_policy_size;
-#endif
-
-#ifdef CONFIG_XSM_SILO
-extern const struct xsm_ops *silo_init(void);
-#else
-static const inline struct xsm_ops *silo_init(void)
-{
-    return NULL;
-}
-#endif
-
 #else /* CONFIG_XSM */
 
 #include <xsm/dummy.h>
 
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init (
-    unsigned long *module_map, const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
 #endif /* CONFIG_XSM */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index d8c935328e..b848580eaa 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,6 @@
  *  as published by the Free Software Foundation.
  */
 
-#define XSM_NO_WRAPPERS
 #include <xsm/dummy.h>
 
 #define set_to_dummy_if_null(ops, function)                            \
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 3550dded7b..66a2cb1505 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -17,7 +17,7 @@
  * You should have received a copy of the GNU General Public License along with
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
-#define XSM_NO_WRAPPERS
+#include <xsm/xsm-core.h>
 #include <xsm/dummy.h>
 
 /*
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 21fffbcb41..e1fe4e66a7 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -32,6 +32,11 @@
 
 struct xsm_ops __read_mostly xsm_ops;
 
+#ifdef CONFIG_XSM_FLASK_POLICY
+extern const unsigned char xsm_flask_init_policy[];
+extern const unsigned int xsm_flask_init_policy_size;
+#endif
+
 enum xsm_ops_state {
     XSM_OPS_UNREGISTERED,
     XSM_OPS_REG_FAILED,
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:14:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:14:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184634.333367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjB0-00034R-C7; Fri, 10 Sep 2021 16:14:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184634.333367; Fri, 10 Sep 2021 16:14:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjB0-00034I-7C; Fri, 10 Sep 2021 16:14:54 +0000
Received: by outflank-mailman (input) for mailman id 184634;
 Fri, 10 Sep 2021 16:14:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjAy-0002zi-Hb
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:14:52 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3991169a-1252-11ec-b254-12813bfff9fa;
 Fri, 10 Sep 2021 16:14:51 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290384333494.1230487143405;
 Fri, 10 Sep 2021 09:13:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3991169a-1252-11ec-b254-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631290388; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=Znt32WbdrCZ9UZmMCZr9wX8IyotivqfqP4pebYdnoqZfWYHythBprnt1aDrZivLThdU3hRwPdPchI0GswlfI4KWgBPw7w4EgYNJH8m+osU6KIAfAew/b0wDv3UpWbJr4ORb8N+hXTm4WCoHJF4WfHMtnLmrNez4layvLFhpwP14=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290388; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=jZDCRllTZCiN1iiOgOzXnECMuRaKvU/NWYVDT7sgqzA=; 
	b=ZFLBcMRy+oAUYrxf9X32UsbhaPyH2cIfRpwyrv7bHXGV0hblJhrJMLwIrel40FfktgqCzxsYL2jLjPAa0bkRg+KFBpG67LbO41itMCi0kRU0qISKInW7HsVl5gMTPYYCfyb569cbudgX9X6KFW8G4ZtuNTQcAaLL33PB8UAhVaE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290388;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=jZDCRllTZCiN1iiOgOzXnECMuRaKvU/NWYVDT7sgqzA=;
	b=gVtmlZl48kPW3qfkm1ZSZ/TBsAIj1kiyfvtWnvY9AfPRTBsmslhLXsUYnVt4aDpt
	siekwF9onwidKXBf0jp+in49e5uPjYxF3Oeq14iwaqEijLZ+lp5nOxGDHsn4wSv91dY
	NbgfBwM+vafJon8OdyMIx+mSmuoTPjjRTI67dBmo=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 09/10] kconfig: update xsm config to reflect reality
Date: Fri, 10 Sep 2021 16:13:04 -0400
Message-Id: <20210910201305.32526-10-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

It has been a very long time since XSM Flask was the only XSM module, yet the
concenpt of turning XSM on/off continues to be synonymous with enabling and
disabling XSM Flask. Even when XSM Flask was the only module, turning XSM
on/off did not disable or remove the XSM hooks but simply controlled whether
they were implemented as direct inline functions or dispatch calls.

This commit updates XSM kconfig to ensure that it is clear in the code as well
to the user, via the help messages, that the option is about configuring which
XSM policy module(s) are available and which is the default.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/common/Kconfig         | 49 ++++++++++++++++++--------------------
 xen/include/xen/sched.h    |  2 +-
 xen/include/xsm/dummy.h    | 23 +++++++++---------
 xen/include/xsm/xsm-core.h |  6 ++---
 xen/include/xsm/xsm.h      |  6 ++---
 xen/xsm/Makefile           |  4 ++--
 xen/xsm/xsm_core.c         |  4 ++--
 7 files changed, 46 insertions(+), 48 deletions(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index ac5491b1cc..2f85538920 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -200,23 +200,20 @@ config XENOPROF
 
 	  If unsure, say Y.
 
-config XSM
-	bool "Xen Security Modules support"
+config XSM_CONFIGURABLE
+	bool "Configure Xen Security Modules"
 	default ARM
-	---help---
-	  Enables the security framework known as Xen Security Modules which
-	  allows administrators fine-grained control over a Xen domain and
-	  its capabilities by defining permissible interactions between domains,
-	  the hypervisor itself, and related resources such as memory and
-	  devices.
+	help
+	  Allows for configuring the Xen Security Modules (XSM) policy or policies
+	  modules that will be availble and which will be the default.
 
 	  If unsure, say N.
 
 config XSM_FLASK
-	def_bool y
-	prompt "FLux Advanced Security Kernel support"
-	depends on XSM
-	---help---
+	bool "FLux Advanced Security Kernel support"
+	depends on XSM_CONFIGURABLE
+	select XSM_EVTCHN_LABELING
+	help
 	  Enables FLASK (FLux Advanced Security Kernel) as the access control
 	  mechanism used by the XSM framework.  This provides a mandatory access
 	  control framework by which security enforcement, isolation, and
@@ -226,10 +223,10 @@ config XSM_FLASK
 	  If unsure, say Y.
 
 config XSM_FLASK_AVC_STATS
-	def_bool y
-	prompt "Maintain statistics on the FLASK access vector cache" if EXPERT
+	bool "Maintain statistics on the FLASK access vector cache" if EXPERT
+	default y
 	depends on XSM_FLASK
-	---help---
+	help
 	  Maintain counters on the access vector cache that can be viewed using
 	  the FLASK_AVC_CACHESTATS sub-op of the xsm_op hypercall.  Disabling
 	  this will save a tiny amount of memory and time to update the stats.
@@ -240,7 +237,7 @@ config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
 	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
 	depends on XSM_FLASK
-	---help---
+	help
 	  This includes a default XSM policy in the hypervisor so that the
 	  bootloader does not need to load a policy to get sane behavior from an
 	  XSM-enabled hypervisor.  If this is disabled, a policy must be
@@ -253,10 +250,10 @@ config XSM_FLASK_POLICY
 	  If unsure, say Y.
 
 config XSM_SILO
-	def_bool y
-	prompt "SILO support"
-	depends on XSM
-	---help---
+	bool "SILO support"
+	default y if ARM
+	depends on XSM_CONFIGURABLE
+	help
 	  Enables SILO as the access control mechanism used by the XSM framework.
 	  This is not the default module, add boot parameter xsm=silo to choose
 	  it. This will deny any unmediated communication channels (grant tables
@@ -265,14 +262,14 @@ config XSM_SILO
 	  If unsure, say Y.
 
 choice
-	prompt "Default XSM implementation"
-	depends on XSM
+	prompt "Default XSM module"
+	depends on XSM_CONFIGURABLE
 	default XSM_SILO_DEFAULT if XSM_SILO && ARM
 	default XSM_FLASK_DEFAULT if XSM_FLASK
 	default XSM_SILO_DEFAULT if XSM_SILO
 	default XSM_DUMMY_DEFAULT
 	config XSM_DUMMY_DEFAULT
-		bool "Match non-XSM behavior"
+		bool "Classic Dom0 behavior"
 	config XSM_FLASK_DEFAULT
 		bool "FLux Advanced Security Kernel" if XSM_FLASK
 	config XSM_SILO_DEFAULT
@@ -282,15 +279,15 @@ endchoice
 config LATE_HWDOM
 	bool "Dedicated hardware domain"
 	default n
-	depends on XSM && X86
-	---help---
+	depends on XSM_FLASK && X86
+	help
 	  Allows the creation of a dedicated hardware domain distinct from
 	  domain 0 that manages devices without needing access to other
 	  privileged functionality such as the ability to manage domains.
 	  This requires that the actual domain 0 be a stub domain that
 	  constructs the actual hardware domain instead of initializing the
 	  hardware itself.  Because the hardware domain needs access to
-	  hypercalls not available to unprivileged guests, an XSM policy
+	  hypercalls not available to unprivileged guests, an XSM Flask policy
 	  is required to properly define the privilege of these domains.
 
 	  This feature does nothing if the "hardware_dom" boot parameter is
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404..7f4bfa855b 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -120,7 +120,7 @@ struct evtchn
     unsigned short notify_vcpu_id; /* VCPU for local delivery notification */
     uint32_t fifo_lastq;           /* Data for identifying last queue. */
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
     union {
 #ifdef XSM_NEED_GENERIC_EVTCHN_SSID
         /*
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index be8877a195..a0fe257b8e 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -43,13 +43,13 @@ static inline void __xsm_action_mismatch_detected(void)
 void __xsm_action_mismatch_detected(void);
 #endif
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 /*
- * In CONFIG_XSM builds, this header file is included from xsm/dummy.c, and
- * contains static (not inline) functions compiled to the dummy XSM module.
- * There is no xsm_default_t argument available, so the value from the assertion
- * is used to initialize the variable.
+ * In CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/dummy.c, and contains static (not inline) functions compiled to the
+ * dummy XSM module.  There is no xsm_default_t argument available, so the
+ * value from the assertion is used to initialize the variable.
  */
 #define XSM_INLINE __maybe_unused
 
@@ -57,20 +57,21 @@ void __xsm_action_mismatch_detected(void);
 #define XSM_DEFAULT_VOID void
 #define XSM_ASSERT_ACTION(def) xsm_default_t action = def; (void)action
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 /*
- * In !CONFIG_XSM builds, this header file is included from xsm/xsm.h, and
- * contains inline functions for each XSM hook. These functions also perform
- * compile-time checks on the xsm_default_t argument to ensure that the behavior
- * of the dummy XSM module is the same as the behavior with XSM disabled.
+ * In !CONFIG_XSM_CONFIGURABLE builds, this header file is included from
+ * xsm/xsm.h, and contains inline functions for each XSM hook. These functions
+ * also perform compile-time checks on the xsm_default_t argument to ensure
+ * that the behavior of the dummy XSM module is the same as the behavior with
+ * XSM disabled.
  */
 #define XSM_INLINE always_inline
 #define XSM_DEFAULT_ARG xsm_default_t action,
 #define XSM_DEFAULT_VOID xsm_default_t action
 #define XSM_ASSERT_ACTION(def) LINKER_BUG_ON(def != action)
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 static always_inline int xsm_default_action(
     xsm_default_t action, struct domain *src, struct domain *target)
diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index 3c4a25d7fc..59355937ef 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -199,7 +199,7 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(
@@ -240,7 +240,7 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #ifdef CONFIG_MULTIBOOT
 static inline int xsm_multiboot_init(
@@ -262,6 +262,6 @@ static inline bool has_xsm_magic(paddr_t start)
 }
 #endif /* CONFIG_HAS_DEVICE_TREE */
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 7b1611d10b..14f039673e 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,7 +20,7 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 extern struct xsm_ops xsm_ops;
 
@@ -590,10 +590,10 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM */
+#else /* CONFIG_XSM_CONFIGURABLE */
 
 #include <xsm/dummy.h>
 
-#endif /* CONFIG_XSM */
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index cf0a728f1c..09b9311b1d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
-obj-$(CONFIG_XSM) += xsm_policy.o
-obj-$(CONFIG_XSM) += dummy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
+obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index e1fe4e66a7..8fd2f6e180 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,7 +18,7 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM
+#ifdef CONFIG_XSM_CONFIGURABLE
 
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
@@ -222,7 +222,7 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif
+#endif /* CONFIG_XSM_CONFIGURABLE */
 
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:15:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:15:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184651.333378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjBQ-0003xT-M8; Fri, 10 Sep 2021 16:15:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184651.333378; Fri, 10 Sep 2021 16:15:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjBQ-0003xL-Hm; Fri, 10 Sep 2021 16:15:20 +0000
Received: by outflank-mailman (input) for mailman id 184651;
 Fri, 10 Sep 2021 16:15:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OyUy=OA=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mOjBO-0003wo-Kv
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:15:18 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 54df16f4-79c9-4d68-834a-5d0a1fb313df;
 Fri, 10 Sep 2021 16:15:14 +0000 (UTC)
Received: from sisyou.hme. (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631290386362893.7000982855852;
 Fri, 10 Sep 2021 09:13:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54df16f4-79c9-4d68-834a-5d0a1fb313df
ARC-Seal: i=1; a=rsa-sha256; t=1631290389; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=fqcb1to0vCor3i4hwdrT5/dzOOVc0VwsVGKYuZKXY4ohmVhZ4iPdWjsi1b65UOZJG9RJU89T5i+1hc2jNI3IPqg+eqcJrEdQriV2clQwJGK291AIs0UDFYi8HuriFHzlAXaXipvs4Rv3E3p1Yap5jmJRGV552UpchOr7J6PZS1E=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631290389; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=DSfNAwyaArRS4T2ujyt4gshuXZrkX6KzNK3M7dH8EmI=; 
	b=B72qb1lr3+X5WFutespgImogO1y/rikMx+jNLmYPkLCQK+jD54QOkKVwN+ydDKHCNHIBeBFO7WM4Rk+ozmIsQgCbXy8E8OYmEFEZMeAcRfSMvryybZFpAjYLtH76GmSKVW9WwinGrV1IulKcfIcV34t4ZvnKJTLKvS2IG1u6ZDE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631290389;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding;
	bh=DSfNAwyaArRS4T2ujyt4gshuXZrkX6KzNK3M7dH8EmI=;
	b=ujhyX22+sRzXJEILCJJp8nDhdf2tQSsVvd8E4dT+7GTGRqJOi+NVevTiZLAa0wi9
	LBg21TZopqM7iRht3srgyrc+87kG7OevHjmBMqFVD08ZmTqOPob9OOcP1/BuGU5ElYK
	BWAuYmHistAx/aj7NXbPcOnK2zYwfNT2KdUKCTNU=
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	xen-devel@lists.xenproject.org
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: [PATCH v6 10/10] xsm: remove alternate xsm hook interface
Date: Fri, 10 Sep 2021 16:13:05 -0400
Message-Id: <20210910201305.32526-11-dpsmith@apertussolutions.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210910201305.32526-1-dpsmith@apertussolutions.com>
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

Hidden behind macro magic is an alternative xsm hook interface dedicated for
use when the dummy/default policy is the only one built. This alternative
interface increases code complexity and code size in the core security
framework of Xen.  This results in code requiring additional maintanence and
additional risk for securit-relevant bugs.

This patch removes this additional interface, making Xen's security framework
have a single, consistent interface that works in a single and consistent
manner regardless of which XSM policy is in use.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
 xen/include/xsm/xsm-core.h |  51 ++-
 xen/include/xsm/xsm.h      | 275 ++++++++++------
 xen/xsm/Makefile           |   2 +-
 xen/xsm/dummy.c            |   2 +-
 xen/xsm/dummy.h            | 659 +++++++++++++++++++++++++++++++++++++
 xen/xsm/silo.c             |   2 +-
 xen/xsm/xsm_core.c         |   4 -
 7 files changed, 868 insertions(+), 127 deletions(-)
 create mode 100644 xen/xsm/dummy.h

diff --git a/xen/include/xsm/xsm-core.h b/xen/include/xsm/xsm-core.h
index 59355937ef..4b01564410 100644
--- a/xen/include/xsm/xsm-core.h
+++ b/xen/include/xsm/xsm-core.h
@@ -18,6 +18,31 @@
 #include <xen/sched.h>
 #include <xen/multiboot.h>
 
+/* Cannot use BUILD_BUG_ON here because the expressions we check are not
+ * considered constant at compile time. Instead, rely on constant propagation to
+ * inline out the calls to this invalid function, which will cause linker errors
+ * if references remain at link time.
+ */
+#define XSM_LINKER_BUG_ON(x) do { if (x) __xsm_action_mismatch_detected(); } while (0)
+
+#if defined(CONFIG_COVERAGE) && defined(__clang__)
+/*
+ * LLVM coverage support seems to disable some of the optimizations needed in
+ * order for XSM to compile. Since coverage should not be used in production
+ * provide an implementation of __xsm_action_mismatch_detected to satisfy the
+ * linker.
+ */
+static inline void __xsm_action_mismatch_detected(void)
+{
+    ASSERT_UNREACHABLE();
+}
+#else
+/* DO NOT implement this function; it is supposed to trigger link errors */
+void __xsm_action_mismatch_detected(void);
+#endif
+
+#define XSM_ASSERT_ACTION(def) XSM_LINKER_BUG_ON(def != action)
+
 /* policy magic number */
 typedef uint32_t xsm_magic_t;
 
@@ -199,8 +224,6 @@ struct xsm_ops {
 
 void xsm_fixup_ops(struct xsm_ops *ops);
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 int xsm_multiboot_init(
     unsigned long *module_map, const multiboot_info_t *mbi);
@@ -240,28 +263,4 @@ static const inline struct xsm_ops *silo_init(void)
 }
 #endif
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#ifdef CONFIG_MULTIBOOT
-static inline int xsm_multiboot_init(
-    unsigned long *module_map, const multiboot_info_t *mbi)
-{
-    return 0;
-}
-#endif
-
-#ifdef CONFIG_HAS_DEVICE_TREE
-static inline int xsm_dt_init(void)
-{
-    return 0;
-}
-
-static inline bool has_xsm_magic(paddr_t start)
-{
-    return false;
-}
-#endif /* CONFIG_HAS_DEVICE_TREE */
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_CORE_H */
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 14f039673e..e58dd96974 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -20,8 +20,6 @@
 #include <xen/multiboot.h>
 #include <xsm/xsm-core.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 extern struct xsm_ops xsm_ops;
 
 static inline void xsm_security_domaininfo(
@@ -31,58 +29,68 @@ static inline void xsm_security_domaininfo(
 }
 
 static inline int xsm_domain_create(
-    xsm_default_t def, struct domain *d, uint32_t ssidref)
+    xsm_default_t action, struct domain *d, uint32_t ssidref)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domain_create, d, ssidref);
 }
 
-static inline int xsm_getdomaininfo(xsm_default_t def, struct domain *d)
+static inline int xsm_getdomaininfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.getdomaininfo, d);
 }
 
 static inline int xsm_domctl_scheduler_op(
-    xsm_default_t def, struct domain *d, int cmd)
+    xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.domctl_scheduler_op, d, cmd);
 }
 
-static inline int xsm_sysctl_scheduler_op(xsm_default_t def, int cmd)
+static inline int xsm_sysctl_scheduler_op(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.sysctl_scheduler_op, cmd);
 }
 
 static inline int xsm_set_target(
-    xsm_default_t def, struct domain *d, struct domain *e)
+    xsm_default_t action, struct domain *d, struct domain *e)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.set_target, d, e);
 }
 
-static inline int xsm_domctl(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_domctl(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.domctl, d, cmd);
 }
 
-static inline int xsm_sysctl(xsm_default_t def, int cmd)
+static inline int xsm_sysctl(xsm_default_t action, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.sysctl, cmd);
 }
 
-static inline int xsm_readconsole(xsm_default_t def, uint32_t clear)
+static inline int xsm_readconsole(xsm_default_t action, uint32_t clear)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.readconsole, clear);
 }
 
 static inline int xsm_evtchn_unbound(
-    xsm_default_t def, struct domain *d1, struct evtchn *chn, domid_t id2)
+    xsm_default_t action, struct domain *d1, struct evtchn *chn, domid_t id2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_unbound, d1, chn, id2);
 }
 
 static inline int xsm_evtchn_interdomain(
-    xsm_default_t def, struct domain *d1, struct evtchn *chan1,
+    xsm_default_t action, struct domain *d1, struct evtchn *chan1,
     struct domain *d2, struct evtchn *chan2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_interdomain, d1, chan1, d2, chan2);
 }
 
@@ -92,56 +100,65 @@ static inline void xsm_evtchn_close_post(struct evtchn *chn)
 }
 
 static inline int xsm_evtchn_send(
-    xsm_default_t def, struct domain *d, struct evtchn *chn)
+    xsm_default_t action, struct domain *d, struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.evtchn_send, d, chn);
 }
 
 static inline int xsm_evtchn_status(
-    xsm_default_t def, struct domain *d, struct evtchn *chn)
+    xsm_default_t action, struct domain *d, struct evtchn *chn)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_status, d, chn);
 }
 
 static inline int xsm_evtchn_reset(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.evtchn_reset, d1, d2);
 }
 
 static inline int xsm_grant_mapref(
-    xsm_default_t def, struct domain *d1, struct domain *d2, uint32_t flags)
+    xsm_default_t action, struct domain *d1, struct domain *d2, uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_mapref, d1, d2, flags);
 }
 
 static inline int xsm_grant_unmapref(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_unmapref, d1, d2);
 }
 
 static inline int xsm_grant_setup(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_setup, d1, d2);
 }
 
 static inline int xsm_grant_transfer(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_transfer, d1, d2);
 }
 
 static inline int xsm_grant_copy(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.grant_copy, d1, d2);
 }
 
 static inline int xsm_grant_query_size(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.grant_query_size, d1, d2);
 }
 
@@ -173,86 +190,103 @@ static inline char *xsm_show_security_evtchn(
     return alternative_call(xsm_ops.show_security_evtchn, d, chn);
 }
 
-static inline int xsm_init_hardware_domain(xsm_default_t def, struct domain *d)
+static inline int xsm_init_hardware_domain(xsm_default_t action,
+                                           struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.init_hardware_domain, d);
 }
 
-static inline int xsm_get_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_get_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.get_pod_target, d);
 }
 
-static inline int xsm_set_pod_target(xsm_default_t def, struct domain *d)
+static inline int xsm_set_pod_target(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.set_pod_target, d);
 }
 
-static inline int xsm_memory_exchange(xsm_default_t def, struct domain *d)
+static inline int xsm_memory_exchange(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_exchange, d);
 }
 
 static inline int xsm_memory_adjust_reservation(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_adjust_reservation, d1, d2);
 }
 
 static inline int xsm_memory_stat_reservation(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.memory_stat_reservation, d1, d2);
 }
 
 static inline int xsm_memory_pin_page(
-    xsm_default_t def, struct domain *d1, struct domain *d2,
+    xsm_default_t action, struct domain *d1, struct domain *d2,
     struct page_info *page)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.memory_pin_page, d1, d2, page);
 }
 
 static inline int xsm_add_to_physmap(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.add_to_physmap, d1, d2);
 }
 
 static inline int xsm_remove_from_physmap(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.remove_from_physmap, d1, d2);
 }
 
 static inline int xsm_map_gmfn_foreign(
-    xsm_default_t def, struct domain *d, struct domain *t)
+    xsm_default_t action, struct domain *d, struct domain *t)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.map_gmfn_foreign, d, t);
 }
 
-static inline int xsm_claim_pages(xsm_default_t def, struct domain *d)
+static inline int xsm_claim_pages(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.claim_pages, d);
 }
 
-static inline int xsm_console_io(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_console_io(xsm_default_t action, struct domain *d,
+                                 int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.console_io, d, cmd);
 }
 
-static inline int xsm_profile(xsm_default_t def, struct domain *d, int op)
+static inline int xsm_profile(xsm_default_t action, struct domain *d, int op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.profile, d, op);
 }
 
-static inline int xsm_kexec(xsm_default_t def)
+static inline int xsm_kexec(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.kexec);
 }
 
 static inline int xsm_schedop_shutdown(
-    xsm_default_t def, struct domain *d1, struct domain *d2)
+    xsm_default_t action, struct domain *d1, struct domain *d2)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.schedop_shutdown, d1, d2);
 }
 
@@ -261,143 +295,172 @@ static inline char *xsm_show_irq_sid(int irq)
     return alternative_call(xsm_ops.show_irq_sid, irq);
 }
 
-static inline int xsm_map_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_map_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.map_domain_pirq, d);
 }
 
 static inline int xsm_map_domain_irq(
-    xsm_default_t def, struct domain *d, int irq, void *data)
+    xsm_default_t action, struct domain *d, int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.map_domain_irq, d, irq, data);
 }
 
-static inline int xsm_unmap_domain_pirq(xsm_default_t def, struct domain *d)
+static inline int xsm_unmap_domain_pirq(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.unmap_domain_pirq, d);
 }
 
 static inline int xsm_unmap_domain_irq(
-    xsm_default_t def, struct domain *d, int irq, void *data)
+    xsm_default_t action, struct domain *d, int irq, void *data)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unmap_domain_irq, d, irq, data);
 }
 
 static inline int xsm_bind_pt_irq(
-    xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+    xsm_default_t action, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.bind_pt_irq, d, bind);
 }
 
 static inline int xsm_unbind_pt_irq(
-    xsm_default_t def, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+    xsm_default_t action, struct domain *d, struct xen_domctl_bind_pt_irq *bind)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.unbind_pt_irq, d, bind);
 }
 
 static inline int xsm_irq_permission(
-    xsm_default_t def, struct domain *d, int pirq, uint8_t allow)
+    xsm_default_t action, struct domain *d, int pirq, uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.irq_permission, d, pirq, allow);
 }
 
 static inline int xsm_iomem_permission(
-    xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+    xsm_default_t action, struct domain *d, uint64_t s, uint64_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_permission, d, s, e, allow);
 }
 
 static inline int xsm_iomem_mapping(
-    xsm_default_t def, struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+    xsm_default_t action, struct domain *d, uint64_t s, uint64_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.iomem_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pci_config_permission(
-    xsm_default_t def, struct domain *d, uint32_t machine_bdf, uint16_t start,
-    uint16_t end, uint8_t access)
+    xsm_default_t action, struct domain *d, uint32_t machine_bdf,
+    uint16_t start, uint16_t end, uint8_t access)
 {
-    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf, start, end, access);
+    XSM_ASSERT_ACTION(XSM_HOOK);
+    return alternative_call(xsm_ops.pci_config_permission, d, machine_bdf,
+                            start, end, access);
 }
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static inline int xsm_get_device_group(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_get_device_group(xsm_default_t action,
+                                       uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.get_device_group, machine_bdf);
 }
 
 static inline int xsm_assign_device(
-    xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+    xsm_default_t action, struct domain *d, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_device, d, machine_bdf);
 }
 
 static inline int xsm_deassign_device(
-    xsm_default_t def, struct domain *d, uint32_t machine_bdf)
+    xsm_default_t action, struct domain *d, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_device, d, machine_bdf);
 }
 #endif /* HAS_PASSTHROUGH && HAS_PCI) */
 
 #if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
 static inline int xsm_assign_dtdevice(
-    xsm_default_t def, struct domain *d, const char *dtpath)
+    xsm_default_t action, struct domain *d, const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.assign_dtdevice, d, dtpath);
 }
 
 static inline int xsm_deassign_dtdevice(
-    xsm_default_t def, struct domain *d, const char *dtpath)
+    xsm_default_t action, struct domain *d, const char *dtpath)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.deassign_dtdevice, d, dtpath);
 }
 
 #endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
 
-static inline int xsm_resource_plug_pci(xsm_default_t def, uint32_t machine_bdf)
+static inline int xsm_resource_plug_pci(xsm_default_t action,
+                                        uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_plug_pci, machine_bdf);
 }
 
 static inline int xsm_resource_unplug_pci(
-    xsm_default_t def, uint32_t machine_bdf)
+    xsm_default_t action, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_unplug_pci, machine_bdf);
 }
 
-static inline int xsm_resource_plug_core(xsm_default_t def)
+static inline int xsm_resource_plug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_plug_core);
 }
 
-static inline int xsm_resource_unplug_core(xsm_default_t def)
+static inline int xsm_resource_unplug_core(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.resource_unplug_core);
 }
 
 static inline int xsm_resource_setup_pci(
-    xsm_default_t def, uint32_t machine_bdf)
+    xsm_default_t action, uint32_t machine_bdf)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_pci, machine_bdf);
 }
 
-static inline int xsm_resource_setup_gsi(xsm_default_t def, int gsi)
+static inline int xsm_resource_setup_gsi(xsm_default_t action, int gsi)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_gsi, gsi);
 }
 
-static inline int xsm_resource_setup_misc(xsm_default_t def)
+static inline int xsm_resource_setup_misc(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.resource_setup_misc);
 }
 
-static inline int xsm_page_offline(xsm_default_t def, uint32_t cmd)
+static inline int xsm_page_offline(xsm_default_t action, uint32_t cmd)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.page_offline, cmd);
 }
 
-static inline int xsm_hypfs_op(xsm_default_t def)
+static inline int xsm_hypfs_op(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hypfs_op);
 }
 
@@ -414,155 +477,185 @@ static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
 #endif
 
 static inline int xsm_hvm_param(
-    xsm_default_t def, struct domain *d, unsigned long op)
+    xsm_default_t action, struct domain *d, unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.hvm_param, d, op);
 }
 
 static inline int xsm_hvm_control(
-    xsm_default_t def, struct domain *d, unsigned long op)
+    xsm_default_t action, struct domain *d, unsigned long op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.hvm_control, d, op);
 }
 
-static inline int xsm_hvm_param_altp2mhvm(xsm_default_t def, struct domain *d)
+static inline int xsm_hvm_param_altp2mhvm(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.hvm_param_altp2mhvm, d);
 }
 
 static inline int xsm_hvm_altp2mhvm_op(
-    xsm_default_t def, struct domain *d, uint64_t mode, uint32_t op)
+    xsm_default_t action, struct domain *d, uint64_t mode, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.hvm_altp2mhvm_op, d, mode, op);
 }
 
-static inline int xsm_get_vnumainfo(xsm_default_t def, struct domain *d)
+static inline int xsm_get_vnumainfo(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.get_vnumainfo, d);
 }
 
 static inline int xsm_vm_event_control(
-    xsm_default_t def, struct domain *d, int mode, int op)
+    xsm_default_t action, struct domain *d, int mode, int op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.vm_event_control, d, mode, op);
 }
 
 #ifdef CONFIG_MEM_ACCESS
-static inline int xsm_mem_access(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_access(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_access, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_PAGING
-static inline int xsm_mem_paging(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_paging(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_paging, d);
 }
 #endif
 
 #ifdef CONFIG_MEM_SHARING
-static inline int xsm_mem_sharing(xsm_default_t def, struct domain *d)
+static inline int xsm_mem_sharing(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing, d);
 }
 #endif
 
-static inline int xsm_platform_op(xsm_default_t def, uint32_t op)
+static inline int xsm_platform_op(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.platform_op, op);
 }
 
 #ifdef CONFIG_X86
-static inline int xsm_do_mca(xsm_default_t def)
+static inline int xsm_do_mca(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.do_mca);
 }
 
 static inline int xsm_shadow_control(
-    xsm_default_t def, struct domain *d, uint32_t op)
+    xsm_default_t action, struct domain *d, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.shadow_control, d, op);
 }
 
 static inline int xsm_mem_sharing_op(
-    xsm_default_t def, struct domain *d, struct domain *cd, int op)
+    xsm_default_t action, struct domain *d, struct domain *cd, int op)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.mem_sharing_op, d, cd, op);
 }
 
-static inline int xsm_apic(xsm_default_t def, struct domain *d, int cmd)
+static inline int xsm_apic(xsm_default_t action, struct domain *d, int cmd)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.apic, d, cmd);
 }
 
-static inline int xsm_machine_memory_map(xsm_default_t def)
+static inline int xsm_machine_memory_map(xsm_default_t action)
 {
+    XSM_ASSERT_ACTION(XSM_PRIV);
     return alternative_call(xsm_ops.machine_memory_map);
 }
 
-static inline int xsm_domain_memory_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_memory_map(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.domain_memory_map, d);
 }
 
 static inline int xsm_mmu_update(
-    xsm_default_t def, struct domain *d, struct domain *t, struct domain *f,
+    xsm_default_t action, struct domain *d, struct domain *t, struct domain *f,
     uint32_t flags)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmu_update, d, t, f, flags);
 }
 
 static inline int xsm_mmuext_op(
-    xsm_default_t def, struct domain *d, struct domain *f)
+    xsm_default_t action, struct domain *d, struct domain *f)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.mmuext_op, d, f);
 }
 
 static inline int xsm_update_va_mapping(
-    xsm_default_t def, struct domain *d, struct domain *f, l1_pgentry_t pte)
+    xsm_default_t action, struct domain *d, struct domain *f, l1_pgentry_t pte)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return xsm_ops.update_va_mapping(d, f, pte);
 }
 
 static inline int xsm_priv_mapping(
-    xsm_default_t def, struct domain *d, struct domain *t)
+    xsm_default_t action, struct domain *d, struct domain *t)
 {
+    XSM_ASSERT_ACTION(XSM_TARGET);
     return alternative_call(xsm_ops.priv_mapping, d, t);
 }
 
 static inline int xsm_ioport_permission(
-    xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+    xsm_default_t action, struct domain *d, uint32_t s, uint32_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_permission, d, s, e, allow);
 }
 
 static inline int xsm_ioport_mapping(
-    xsm_default_t def, struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+    xsm_default_t action, struct domain *d, uint32_t s, uint32_t e,
+    uint8_t allow)
 {
+    XSM_ASSERT_ACTION(XSM_HOOK);
     return alternative_call(xsm_ops.ioport_mapping, d, s, e, allow);
 }
 
 static inline int xsm_pmu_op(
-    xsm_default_t def, struct domain *d, unsigned int op)
+    xsm_default_t action, struct domain *d, unsigned int op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.pmu_op, d, op);
 }
 
 #endif /* CONFIG_X86 */
 
-static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
+static inline int xsm_dm_op(xsm_default_t action, struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.dm_op, d);
 }
 
-static inline int xsm_xen_version(xsm_default_t def, uint32_t op)
+static inline int xsm_xen_version(xsm_default_t action, uint32_t op)
 {
+    XSM_ASSERT_ACTION(XSM_OTHER);
     return alternative_call(xsm_ops.xen_version, op);
 }
 
-static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)
+static inline int xsm_domain_resource_map(xsm_default_t action,
+                                          struct domain *d)
 {
+    XSM_ASSERT_ACTION(XSM_DM_PRIV);
     return alternative_call(xsm_ops.domain_resource_map, d);
 }
 
@@ -590,10 +683,4 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
 
 #endif /* CONFIG_ARGO */
 
-#else /* CONFIG_XSM_CONFIGURABLE */
-
-#include <xsm/dummy.h>
-
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 #endif /* __XSM_H */
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index 09b9311b1d..121a0eda7d 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -1,6 +1,6 @@
 obj-y += xsm_core.o
+obj-y += dummy.o
 obj-$(CONFIG_XSM_CONFIGURABLE) += xsm_policy.o
-obj-$(CONFIG_XSM_CONFIGURABLE) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
 obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index b848580eaa..e18afe0673 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -10,7 +10,7 @@
  *  as published by the Free Software Foundation.
  */
 
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 #define set_to_dummy_if_null(ops, function)                            \
     do {                                                               \
diff --git a/xen/xsm/dummy.h b/xen/xsm/dummy.h
new file mode 100644
index 0000000000..206e168815
--- /dev/null
+++ b/xen/xsm/dummy.h
@@ -0,0 +1,659 @@
+/*
+ *  Default XSM hooks - IS_PRIV and IS_PRIV_FOR checks
+ *
+ *  Author: Daniel De Graaf <dgdegra@tyhco.nsa.gov>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2,
+ *  as published by the Free Software Foundation.
+ *
+ *
+ *  Dummy XSM hooks implementing the default access check. Each hook should
+ *  have as its first line XSM_DEFAULT_ACTION declaring the privilege level
+ *  required for this access.
+ */
+
+#ifndef __XSM_DUMMY_H__
+#define __XSM_DUMMY_H__
+
+#include <xen/sched.h>
+#include <xsm/xsm-core.h>
+#include <public/hvm/params.h>
+
+#define XSM_DEFAULT_ACTION(def) xsm_default_t action = def; (void)action
+
+static always_inline int xsm_default_action(
+    xsm_default_t action, struct domain *src, struct domain *target)
+{
+    switch ( action )
+    {
+    case XSM_HOOK:
+        return 0;
+    case XSM_TARGET:
+        if ( evaluate_nospec(src == target) )
+        {
+            return 0;
+    case XSM_XS_PRIV:
+            if ( evaluate_nospec(is_xenstore_domain(src)) )
+                return 0;
+        }
+        /* fall through */
+    case XSM_DM_PRIV:
+        if ( target && evaluate_nospec(src->target == target) )
+            return 0;
+        /* fall through */
+    case XSM_PRIV:
+        if ( is_control_domain(src) )
+            return 0;
+        return -EPERM;
+    default:
+        XSM_LINKER_BUG_ON(1);
+        return -EPERM;
+    }
+}
+
+static inline void xsm_security_domaininfo(
+    struct domain *d, struct xen_domctl_getdomaininfo *info)
+{
+    return;
+}
+
+static inline int xsm_domain_create(struct domain *d, uint32_t ssidref)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_getdomaininfo(struct domain *d)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_domctl_scheduler_op(struct domain *d, int cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_sysctl_scheduler_op(int cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_set_target(struct domain *d, struct domain *e)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_domctl(struct domain *d, int cmd)
+{
+    switch ( cmd )
+    {
+    case XEN_DOMCTL_ioport_mapping:
+    case XEN_DOMCTL_memory_mapping:
+    case XEN_DOMCTL_bind_pt_irq:
+    case XEN_DOMCTL_unbind_pt_irq:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_DOMCTL_getdomaininfo:
+        return xsm_default_action(XSM_XS_PRIV, current->domain, d);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, d);
+    }
+}
+
+static inline int xsm_sysctl(int cmd)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_readconsole(uint32_t clear)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_alloc_security_domain(struct domain *d)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_domain(struct domain *d)
+{
+    return;
+}
+
+static inline int xsm_grant_mapref(
+    struct domain *d1, struct domain *d2, uint32_t flags)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_unmapref(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_setup(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_grant_transfer(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_copy(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_grant_query_size(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_memory_exchange(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_memory_adjust_reservation(
+    struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_memory_stat_reservation(
+    struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_console_io(struct domain *d, int cmd)
+{
+    if ( d->is_console )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#ifdef CONFIG_VERBOSE_DEBUG
+    if ( cmd == CONSOLEIO_write )
+        return xsm_default_action(XSM_HOOK, d, NULL);
+#endif
+    return xsm_default_action(XSM_PRIV, d, NULL);
+}
+
+static inline int xsm_profile(struct domain *d, int op)
+{
+    return xsm_default_action(XSM_HOOK, d, NULL);
+}
+
+static inline int xsm_kexec(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_schedop_shutdown(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_DM_PRIV, d1, d2);
+}
+
+static inline int xsm_memory_pin_page(
+    struct domain *d1, struct domain *d2, struct page_info *page)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline int xsm_claim_pages(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_evtchn_unbound(
+    struct domain *d, struct evtchn *chn, domid_t id2)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_evtchn_interdomain(
+    struct domain *d1, struct evtchn *chan1, struct domain *d2,
+    struct evtchn *chan2)
+{
+    return xsm_default_action(XSM_HOOK, d1, d2);
+}
+
+static inline void xsm_evtchn_close_post(struct evtchn *chn)
+{
+    return;
+}
+
+static inline int xsm_evtchn_send(struct domain *d, struct evtchn *chn)
+{
+    return xsm_default_action(XSM_HOOK, d, NULL);
+}
+
+static inline int xsm_evtchn_status(struct domain *d, struct evtchn *chn)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_evtchn_reset(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_alloc_security_evtchns(
+    struct evtchn chn[], unsigned int nr)
+{
+    return 0;
+}
+
+static inline void xsm_free_security_evtchns(
+    struct evtchn chn[], unsigned int nr)
+{
+    return;
+}
+
+static inline char *xsm_show_security_evtchn(
+    struct domain *d, const struct evtchn *chn)
+{
+    return NULL;
+}
+
+static inline int xsm_init_hardware_domain(struct domain *d)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_get_pod_target(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_set_pod_target(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_get_vnumainfo(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
+static inline int xsm_get_device_group(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_assign_device(struct domain *d, uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_deassign_device(struct domain *d, uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_PCI */
+
+#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
+static inline int xsm_assign_dtdevice(struct domain *d, const char *dtpath)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_deassign_dtdevice(struct domain *d, const char *dtpath)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
+
+static inline int xsm_resource_plug_core(void)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_core(void)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_resource_plug_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_unplug_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_pci(uint32_t machine_bdf)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_gsi(int gsi)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_resource_setup_misc(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_page_offline(uint32_t cmd)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, NULL);
+}
+
+static inline int xsm_hypfs_op(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+
+#ifdef CONFIG_COMPAT
+static inline int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
+{
+    return -ENOSYS;
+}
+#endif
+
+static inline char *xsm_show_irq_sid(int irq)
+{
+    return NULL;
+}
+
+static inline int xsm_map_domain_pirq(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_map_domain_irq(
+    struct domain *d, int irq, const void *data)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_pirq(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_bind_pt_irq(
+    struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unbind_pt_irq(
+    struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_unmap_domain_irq(
+    struct domain *d, int irq, const void *data)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_irq_permission(struct domain *d, int pirq, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_iomem_permission(
+    struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_iomem_mapping(
+    struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_pci_config_permission(
+    struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end,
+    uint8_t access)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2)
+{
+    return xsm_default_action(XSM_TARGET, d1, d2);
+}
+
+static inline int xsm_map_gmfn_foreign(struct domain *d, struct domain *t)
+{
+    return xsm_default_action(XSM_TARGET, d, t);
+}
+
+static inline int xsm_hvm_param(struct domain *d, unsigned long op)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_hvm_control(struct domain *d, unsigned long op)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+static inline int xsm_hvm_param_altp2mhvm(struct domain *d)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+static inline int xsm_hvm_altp2mhvm_op(
+    struct domain *d, uint64_t mode, uint32_t op)
+{
+
+    switch ( mode )
+    {
+    case XEN_ALTP2M_mixed:
+        return xsm_default_action(XSM_TARGET, current->domain, d);
+    case XEN_ALTP2M_external:
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    case XEN_ALTP2M_limited:
+        if ( HVMOP_altp2m_vcpu_enable_notify == op )
+            return xsm_default_action(XSM_TARGET, current->domain, d);
+        return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+    default:
+        return -EPERM;
+    }
+}
+
+static inline int xsm_vm_event_control(struct domain *d, int mode, int op)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, d);
+}
+
+#ifdef CONFIG_MEM_ACCESS
+static inline int xsm_mem_access(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_PAGING
+static inline int xsm_mem_paging(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+#ifdef CONFIG_MEM_SHARING
+static inline int xsm_mem_sharing(struct domain *d)
+{
+    XSM_DEFAULT_ACTION(XSM_DM_PRIV);
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+#endif
+
+static inline int xsm_platform_op(uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+#ifdef CONFIG_X86
+static inline int xsm_do_mca(void)
+{
+    XSM_DEFAULT_ACTION(XSM_PRIV);
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_shadow_control(struct domain *d, uint32_t op)
+{
+    XSM_DEFAULT_ACTION(XSM_HOOK);
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_mem_sharing_op(
+    struct domain *d, struct domain *cd, int op)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, cd);
+}
+
+static inline int xsm_apic(struct domain *d, int cmd)
+{
+    return xsm_default_action(XSM_PRIV, d, NULL);
+}
+
+static inline int xsm_machine_memory_map(void)
+{
+    return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+
+static inline int xsm_domain_memory_map(struct domain *d)
+{
+    return xsm_default_action(XSM_TARGET, current->domain, d);
+}
+
+static inline int xsm_mmu_update(
+    struct domain *d, struct domain *t, struct domain *f, uint32_t flags)
+{
+    int rc = 0;
+    if ( f != dom_io )
+        rc = xsm_default_action(XSM_TARGET, d, f);
+    if ( evaluate_nospec(t) && !rc )
+        rc = xsm_default_action(XSM_TARGET, d, t);
+    return rc;
+}
+
+static inline int xsm_mmuext_op(struct domain *d, struct domain *f)
+{
+    return xsm_default_action(XSM_TARGET, d, f);
+}
+
+static inline int xsm_update_va_mapping(
+    struct domain *d, struct domain *f, l1_pgentry_t pte)
+{
+    return xsm_default_action(XSM_TARGET, d, f);
+}
+
+static inline int xsm_priv_mapping(struct domain *d, struct domain *t)
+{
+    return xsm_default_action(XSM_TARGET, d, t);
+}
+
+static inline int xsm_ioport_permission(
+    struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_ioport_mapping(
+    struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
+{
+    return xsm_default_action(XSM_HOOK, current->domain, d);
+}
+
+static inline int xsm_pmu_op(struct domain *d, unsigned int op)
+{
+    switch ( op )
+    {
+    case XENPMU_init:
+    case XENPMU_finish:
+    case XENPMU_lvtpc_set:
+    case XENPMU_flush:
+        return xsm_default_action(XSM_HOOK, d, current->domain);
+    default:
+        return xsm_default_action(XSM_PRIV, d, current->domain);
+    }
+}
+
+#endif /* CONFIG_X86 */
+
+static inline int xsm_dm_op(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+#ifdef CONFIG_ARGO
+static inline int xsm_argo_enable(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_single_source(
+    const struct domain *d, const struct domain *t)
+{
+    return 0;
+}
+
+static inline int xsm_argo_register_any_source(const struct domain *d)
+{
+    return 0;
+}
+
+static inline int xsm_argo_send(const struct domain *d, const struct domain *t)
+{
+    return 0;
+}
+
+#endif /* CONFIG_ARGO */
+
+#include <public/version.h>
+static inline int xsm_xen_version(uint32_t op)
+{
+    switch ( op )
+    {
+    case XENVER_version:
+    case XENVER_platform_parameters:
+    case XENVER_get_features:
+        /* These sub-ops ignore the permission checks and return data. */
+        block_speculation();
+        return 0;
+    case XENVER_extraversion:
+    case XENVER_compile_info:
+    case XENVER_capabilities:
+    case XENVER_changeset:
+    case XENVER_pagesize:
+    case XENVER_guest_handle:
+        /* These MUST always be accessible to any guest by default. */
+        return xsm_default_action(XSM_HOOK, current->domain, NULL);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, NULL);
+    }
+}
+
+static inline int xsm_domain_resource_map(struct domain *d)
+{
+    return xsm_default_action(XSM_DM_PRIV, current->domain, d);
+}
+
+#endif /* __XSM_DUMMY_H__ */
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 66a2cb1505..350612971f 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -18,7 +18,7 @@
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
 #include <xsm/xsm-core.h>
-#include <xsm/dummy.h>
+#include "dummy.h"
 
 /*
  * Check if inter-domain communication is allowed.
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 8fd2f6e180..a1cd2eb6b9 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -18,8 +18,6 @@
 #include <xen/hypercall.h>
 #include <xsm/xsm.h>
 
-#ifdef CONFIG_XSM_CONFIGURABLE
-
 #ifdef CONFIG_MULTIBOOT
 #include <asm/setup.h>
 #endif
@@ -222,8 +220,6 @@ bool __init has_xsm_magic(paddr_t start)
 }
 #endif
 
-#endif /* CONFIG_XSM_CONFIGURABLE */
-
 long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_do_xsm_op(op);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 16:24:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 16:24:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184686.333389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjJp-00065w-Lv; Fri, 10 Sep 2021 16:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184686.333389; Fri, 10 Sep 2021 16:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjJp-00065p-Hd; Fri, 10 Sep 2021 16:24:01 +0000
Received: by outflank-mailman (input) for mailman id 184686;
 Fri, 10 Sep 2021 16:24:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOjJo-00065j-CS
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 16:24:00 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 80b45f86-1253-11ec-b255-12813bfff9fa;
 Fri, 10 Sep 2021 16:23:59 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AC63E611AD;
 Fri, 10 Sep 2021 16:23:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80b45f86-1253-11ec-b255-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631291038;
	bh=FAMH8UOR1BFdb9OeNpXyvfariFRCVSw0ymZGcKGhxVE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=b2/WvYxLIN4sUuxFTtH1CcP96c/oT4wvZKa0N5cnPg5Zl72UYbeTTTNb1iZf53yuK
	 8t0P+ZvBySSf1Xw1I27jK+8lnDyKbHJ483iaOgLv+73GFuFUPUJagSL3pIOt+3CZZ3
	 Hb9XAtmAzJTb8EXFtEsloeSvsUuqIO+HuhEQ4XM1oYcm/wV0mKsfQuFdWJw2HqSpYZ
	 DkNP6PjXtqDKdQRG/oPrqBMkHRpOZvy5AQC3/f0VqOcb9SUjMp4VXVX2MyHvqQzppT
	 +txQNRwZu/wMNlREPpooz36Wl5w7BkmkQHXsb0G055me35hbLZyvDD6QN6PcjSVOHX
	 kMCymvxrreBzA==
Date: Fri, 10 Sep 2021 09:23:58 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Penny Zheng <penny.zheng@arm.com>, Bertrand.Marquis@arm.com, 
    Wei.Chen@arm.com, xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    julien@xen.org
Subject: Re: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
In-Reply-To: <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com>
Message-ID: <alpine.DEB.2.21.2109100916270.10523@sstabellini-ThinkPad-T480s>
References: <20210910025215.1671073-1-penny.zheng@arm.com> <20210910025215.1671073-6-penny.zheng@arm.com> <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 10 Sep 2021, Jan Beulich wrote:
> On 10.09.2021 04:52, Penny Zheng wrote:
> > In order to deal with the trouble of count-to-order conversion when page number
> > is not in a power-of-two, this commit re-define assign_pages for nr pages and
> > assign_page for original page with a single order.
> > 
> > Backporting confusion could be helped by altering the order of assign_pages
> > parameters, such that the compiler would point out that adjustments at call
> > sites are needed.
> 
> Thanks, this now takes care of my primary concern. However, I (now?) have
> another (I thought I would have mentioned this before):
> 
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
> >  
> >  
> >  int assign_pages(
> > -    struct domain *d,
> >      struct page_info *pg,
> > -    unsigned int order,
> > +    unsigned long nr,
> 
> If this really is to be "unsigned long" (and not "unsigned int"), then...

Hi Jan,

Thanks for spotting this. I think it makes sense to use "unsigned int
nr" here.


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 17:05:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 17:05:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184693.333400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjxJ-0002qA-TG; Fri, 10 Sep 2021 17:04:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184693.333400; Fri, 10 Sep 2021 17:04:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOjxJ-0002q3-Q4; Fri, 10 Sep 2021 17:04:49 +0000
Received: by outflank-mailman (input) for mailman id 184693;
 Fri, 10 Sep 2021 17:04:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOjxI-0002pt-Rv; Fri, 10 Sep 2021 17:04:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOjxI-0003f5-IH; Fri, 10 Sep 2021 17:04:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOjxI-0005WD-7x; Fri, 10 Sep 2021 17:04:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOjxI-0001ML-7Q; Fri, 10 Sep 2021 17:04:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PWbekrL5RyhgAPoTmRUi9m2l57zRwEHjbhmMgZX1Dew=; b=KDY1bWleA2GDl8vvYW95jDugkG
	RjALmu+kjUWL4MPcg2ipmvcc/kTD1smz0yEw16B0V6aEPgG+t2PXKto+gF38P4xNTKfkukhZFuYnf
	FSva6JdD4fh/luKQYzVXdAbrVD3kx7y32Eq0DMhK51d1wWcBvMS4tPhrHgGuBuZk5Vas=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164943-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164943: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
X-Osstest-Versions-That:
    xen=2e2e22c7d50392fa5cced3e0334b217426f48b7a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 17:04:48 +0000

flight 164943 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164943/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782
baseline version:
 xen                  2e2e22c7d50392fa5cced3e0334b217426f48b7a

Last test of basis   164912  2021-09-09 12:01:41 Z    1 days
Testing same since   164943  2021-09-10 13:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Kevin Stefanov <kevin.stefanov@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2e2e22c7d5..c76cfada1c  c76cfada1cfad05aaf64ce3ad305c5467650e782 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:19:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:19:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184708.333414 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7Y-0003Ag-A2; Fri, 10 Sep 2021 18:19:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184708.333414; Fri, 10 Sep 2021 18:19:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7Y-0003AZ-5e; Fri, 10 Sep 2021 18:19:28 +0000
Received: by outflank-mailman (input) for mailman id 184708;
 Fri, 10 Sep 2021 18:19:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ConJ=OA=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOl7W-0003AT-4x
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:26 +0000
Received: from mail-lj1-x22b.google.com (unknown [2a00:1450:4864:20::22b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a0a8c750-8b39-48f1-ae36-b4a0802e83f7;
 Fri, 10 Sep 2021 18:19:24 +0000 (UTC)
Received: by mail-lj1-x22b.google.com with SMTP id h1so4570466ljl.9
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 11:19:24 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.22
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 10 Sep 2021 11:19:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0a8c750-8b39-48f1-ae36-b4a0802e83f7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=2wDtrZhJLGcQTMVD7Lr0thv21N4TDU7fV2GjxD/iV08=;
        b=luTtFfp9KMWgKC5T1lBIVoOK7Go4qfHeEWHubo7zAqMPEmbaUhDt5zTKryom0S7/JM
         UUxCSO8QTrJTJBO/MVIsaZaktWA9wNZMHS6HZQCpE7TmCGkhVYhfJZjLUvRNWB9xVa/s
         9UdyNjo+/T4lf8c6h1sKs2ZxsgAbcUz627HuJuyfyJVUAVsoIi/Y2V0IgGEQwNJelS5Y
         dfDPo4Un8LntQqSidk/OArIlckEazIyMqNAoLuFkku+5JdHk3LHC58EweJ3Soz+1z4zi
         YnRnNfJKXihAwQGMlwVT73O1tN597utXANRio6U9xL9mHwPQHNpsWmWDp0PziKmUKNTm
         5amg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=2wDtrZhJLGcQTMVD7Lr0thv21N4TDU7fV2GjxD/iV08=;
        b=p5I3MYyCL2K7sLJ2Ekvrsx++maPW0ngisloB2yrBYzw1cqdzO9zJzWwfc9zWSZFTrD
         8qvBr5ZGJArLwvTwDgl719eFetwxUVtCwN9lETxrxT934evItWHY4pLtURoIqdyFSXoL
         O3HtjxNeU1rEPYV3ToiaB5G4+7wNDFvAk6kVhugpS28QqpHjAC0HxdKv7LaBpKK6mySY
         h8YD1Zz7bOTIoJEP5bJmWo/o4Ita9trdSEjhz7AW5jdqEkBCSA70ogIFWXbJ3bPC0e/K
         sdbbKH7QiqCMfSC2bR792tuKpzbIlFly8ZzQrRTnE6QEyLhp38KPp9cr81fF5SxRzlIs
         tfbA==
X-Gm-Message-State: AOAM533Uy4EnNt5C6kh/73U2MkrZXK77A1sITW65XAK2WtMym75T0u9Z
	3KSrM8psM5k7kIZwfxNsQ6IOrCdcSXI=
X-Google-Smtp-Source: ABdhPJz9QlSP5NndOvc9v2JpCsObXt/dZ91lpDHfb2Iqv1jj7tluDjLjMLEDqLbRR/Vsx1y+eaEC2Q==
X-Received: by 2002:a2e:814e:: with SMTP id t14mr5280480ljg.473.1631297963344;
        Fri, 10 Sep 2021 11:19:23 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>
Subject: [PATCH V2 0/3] Add handling of extended regions (safe ranges) on Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
Date: Fri, 10 Sep 2021 21:18:41 +0300
Message-Id: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

You can find an initial discussion at [1] and [2].

The extended region (safe range) is a region of guest physical address space
which is unused and could be safely used to create grant/foreign mappings instead
of wasting real RAM pages from the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and advertised
to it via "reg" property under hypervisor node in the guest device-tree
(the indexes for extended regions are 1...N).
No device tree bindings update is needed, guest infers the presense of extended
regions from the number of regions in "reg" property.
New compatible/property will be needed (but only after this patch [3] or alternative
goes in) to indicate that "region 0 is safe to use". Until this patch is merged it is
not safe to use extended regions for the grant table space.

The extended regions are calculated differently for direct mapped
Dom0 (with and without IOMMU) and non-direct mapped DomUs.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

Xen patch series is also available at [4]. The corresponding Linux patch series is at [5] for now
(last 4 patches).

Tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with updated virtio-disk backend [6]
running in Dom0 (256MB RAM) and DomD (2GB RAM). In both cases the backend pre-maps DomU memory
which is 3GB. All foreign memory gets mapped into extended regions (so the amount of RAM in
the backend domain is not reduced). No issues were observed.

[1] https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
[2] https://lore.kernel.org/xen-devel/1631034578-12598-1-git-send-email-olekstysh@gmail.com/
[3] https://lore.kernel.org/xen-devel/1631228688-30347-1-git-send-email-olekstysh@gmail.com/
[4] https://github.com/otyshchenko1/xen/commits/map_opt_ml3
[5] https://github.com/otyshchenko1/linux/commits/map_opt_ml2
[6] https://github.com/otyshchenko1/virtio-disk/commits/map_opt_next

Oleksandr Tyshchenko (3):
  xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
  xen/arm: Add handling of extended regions for Dom0
  libxl/arm: Add handling of extended regions for DomU

 tools/include/libxl.h            |   7 ++
 tools/libs/light/libxl.c         |   2 +
 tools/libs/light/libxl_arm.c     |  89 ++++++++++++++-
 tools/libs/light/libxl_types.idl |   2 +
 xen/arch/arm/domain_build.c      | 226 ++++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/sysctl.c            |   2 +
 xen/arch/x86/sysctl.c            |   2 +
 xen/include/public/sysctl.h      |   3 +-
 8 files changed, 328 insertions(+), 5 deletions(-)

-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:19:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:19:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184710.333436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7g-0003kn-Qz; Fri, 10 Sep 2021 18:19:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184710.333436; Fri, 10 Sep 2021 18:19:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7g-0003ke-Mv; Fri, 10 Sep 2021 18:19:36 +0000
Received: by outflank-mailman (input) for mailman id 184710;
 Fri, 10 Sep 2021 18:19:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ConJ=OA=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOl7f-0003AT-WB
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:36 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d8cdc84b-f567-4eae-bb76-6688abd738e9;
 Fri, 10 Sep 2021 18:19:26 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id k4so5789292lfj.7
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 11:19:26 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.24
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 10 Sep 2021 11:19:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8cdc84b-f567-4eae-bb76-6688abd738e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=6OqwJJkVPWfknF1ipcJOm+d5im8+QUIzbjtyLrkFb/M=;
        b=W1rFwzZH8zTl0VZSHEaQMpOhOo3ip+nzB56r7+WBk0X0NEQ+1VqIuPamRv5m7WGHKq
         ZTE8m7ngwpqPWIOhMbglmfpRyUmI4WO/A3qpexBiLh9klKvPdVEA08/c4PmUPjaMwd59
         ML/SD3wxAnA0XutCWMsq7tUea45boYezdoYcE5DCBrlLuDqbuf/OZB5FJOseCB3BMtEp
         qJX+2YXswxPMAauDRH0x7rBr+GpI0RfzCi/tGRsjfJ/Y4rjlWNlq0ZTROXytKQhcoB2p
         DTyuLryBdFnzUYOGnGhd0S0uDLyahDxQQSW/e67iculbiHvaDzOkZPRwQafgoc+RZSaC
         n/dQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=6OqwJJkVPWfknF1ipcJOm+d5im8+QUIzbjtyLrkFb/M=;
        b=YH3AHKjmXwQRp/i1wzAlvFh6c/s3hCN4VMQCCTYetwyw5rKVOwY5gLC9vrd4Kvct4o
         gKY5uYTY5cgxM2iK4EPXvZ99RAUe/Hku6iqva6wTF2Qgvzybenvmc07Nn9fj63bB5ELt
         XGBkHirk6A/JMo66QDR0s7oZwCEFefRJ4Nm2ND0NvTcNbjKNZsiOa9p9lP5E2P0PJVwW
         3H//DBIb67bYlFL2AL1wqlFsysclgTsAPS/v9ixEpS4CYchtscHsLYQMgKxrDhnOQi5p
         0laPPSH/DLBjOw1SJrUgJxobIAxooD+FPrZexVdP+jQIiOS2lKUcBMEmTobxJCYzTw5C
         UwhA==
X-Gm-Message-State: AOAM531j7+eXRygBUl5c1Bq3zmNkO9MkH8HkV9o7pxwL/9gkVn5mebfc
	sO/t+bGNV4Jdc6/fMER99n23STuFGU0=
X-Google-Smtp-Source: ABdhPJwOkAqyiJqstacgeYt9Ih7dmN6D82FgzbEmHKzP6tBr/Ie+o6OyzcVjMlFpoBHbXhUA1N4gcQ==
X-Received: by 2002:a05:6512:3b9d:: with SMTP id g29mr4866022lfv.77.1631297965309;
        Fri, 10 Sep 2021 11:19:25 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>
Subject: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
Date: Fri, 10 Sep 2021 21:18:43 +0300
Message-Id: <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

***

As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
the algorithm to choose extended regions for it is different
in comparison with the algorithm for non-direct mapped DomU.
What is more, that extended regions should be chosen differently
whether IOMMU is enabled or not.

Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
holes found in host device-tree if otherwise. Make sure that
extended regions are 2MB-aligned and located within maximum possible
addressable physical memory range. The maximum number of extended
regions is 128.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Changes since RFC:
   - update patch description
   - drop uneeded "extended-region" DT property
---

 xen/arch/arm/domain_build.c | 226 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 224 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d..070ec27 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domain *d,
     return res;
 }
 
+static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
+{
+    struct meminfo *ext_regions = data;
+    paddr_t start, size;
+
+    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
+        return 0;
+
+    /* Both start and size of the extended region should be 2MB aligned */
+    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
+    if ( start > e )
+        return 0;
+
+    size = (e - start + 1) & ~(SZ_2M - 1);
+    if ( !size )
+        return 0;
+
+    ext_regions->bank[ext_regions->nr_banks].start = start;
+    ext_regions->bank[ext_regions->nr_banks].size = size;
+    ext_regions->nr_banks ++;
+
+    return 0;
+}
+
+/*
+ * The extended regions will be prevalidated by the memory hotplug path
+ * in Linux which requires for any added address range to be within maximum
+ * possible addressable physical memory range for which the linear mapping
+ * could be created.
+ * For 48-bit VA space size the maximum addressable range are:
+ * 0x40000000 - 0x80003fffffff
+ */
+#define EXT_REGION_START   0x40000000ULL
+#define EXT_REGION_END     0x80003fffffffULL
+
+static int __init find_unallocated_memory(const struct kernel_info *kinfo,
+                                          struct meminfo *ext_regions)
+{
+    const struct meminfo *assign_mem = &kinfo->mem;
+    struct rangeset *unalloc_mem;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find unallocated memory for extended regions\n");
+
+    unalloc_mem = rangeset_new(NULL, NULL, 0);
+    if ( !unalloc_mem )
+        return -ENOMEM;
+
+    /* Start with all available RAM */
+    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
+    {
+        start = bootinfo.mem.bank[i].start;
+        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
+        res = rangeset_add_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove RAM assigned to Dom0 */
+    for ( i = 0; i < assign_mem->nr_banks; i++ )
+    {
+        start = assign_mem->bank[i].start;
+        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
+        res = rangeset_remove_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove reserved-memory regions */
+    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
+    {
+        start = bootinfo.reserved_mem.bank[i].start;
+        end = bootinfo.reserved_mem.bank[i].start +
+            bootinfo.reserved_mem.bank[i].size - 1;
+        res = rangeset_remove_range(unalloc_mem, start, end);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove grant table region */
+    start = kinfo->gnttab_start;
+    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
+    res = rangeset_remove_range(unalloc_mem, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_report_ranges(unalloc_mem, start, end,
+                                 add_ext_regions, ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(unalloc_mem);
+
+    return res;
+}
+
+static int __init find_memory_holes(const struct kernel_info *kinfo,
+                                    struct meminfo *ext_regions)
+{
+    struct dt_device_node *np;
+    struct rangeset *mem_holes;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find memory holes for extended regions\n");
+
+    mem_holes = rangeset_new(NULL, NULL, 0);
+    if ( !mem_holes )
+        return -ENOMEM;
+
+    /* Start with maximum possible addressable physical memory range */
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_add_range(mem_holes, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
+    dt_for_each_device_node( dt_host, np )
+    {
+        unsigned int naddr;
+        u64 addr, size;
+
+        naddr = dt_number_of_address(np);
+
+        for ( i = 0; i < naddr; i++ )
+        {
+            res = dt_device_get_address(np, i, &addr, &size);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                       i, dt_node_full_name(np));
+                goto out;
+            }
+
+            start = addr & PAGE_MASK;
+            end = PAGE_ALIGN(addr + size) - 1;
+            res = rangeset_remove_range(mem_holes, start, end);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                       start, end);
+                goto out;
+            }
+        }
+    }
+
+    start = EXT_REGION_START;
+    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
+    res = rangeset_report_ranges(mem_holes, start, end,
+                                 add_ext_regions,  ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(mem_holes);
+
+    return res;
+}
+
 static int __init make_hypervisor_node(struct domain *d,
                                        const struct kernel_info *kinfo,
                                        int addrcells, int sizecells)
@@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain *d,
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
         "xen,xen";
-    __be32 reg[4];
+    __be32 reg[(NR_MEM_BANKS + 1) * 4];
     gic_interrupt_t intr;
     __be32 *cells;
     int res;
     void *fdt = kinfo->fdt;
+    struct meminfo *ext_regions;
+    unsigned int i;
 
     dt_dprintk("Create hypervisor node\n");
 
@@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct domain *d,
     if ( res )
         return res;
 
+    ext_regions = xzalloc(struct meminfo);
+    if ( !ext_regions )
+        return -ENOMEM;
+
+    if ( is_32bit_domain(d) )
+        printk(XENLOG_WARNING "The extended region is only supported for 64-bit guest\n");
+    else
+    {
+        if ( !is_iommu_enabled(d) )
+            res = find_unallocated_memory(kinfo, ext_regions);
+        else
+            res = find_memory_holes(kinfo, ext_regions);
+
+        if ( res )
+            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
+    }
+
     /* reg 0 is grant table space */
     cells = &reg[0];
     dt_child_set_range(&cells, addrcells, sizecells,
                        kinfo->gnttab_start, kinfo->gnttab_size);
+    /* reg 1...N are extended regions */
+    for ( i = 0; i < ext_regions->nr_banks; i++ )
+    {
+        u64 start = ext_regions->bank[i].start;
+        u64 size = ext_regions->bank[i].size;
+
+        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
+                   i, start, start + size);
+
+        dt_child_set_range(&cells, addrcells, sizecells, start, size);
+    }
+    xfree(ext_regions);
+
     res = fdt_property(fdt, "reg", reg,
-                       dt_cells_to_size(addrcells + sizecells));
+                       dt_cells_to_size(addrcells + sizecells) * (i + 1));
     if ( res )
         return res;
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:19:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:19:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184709.333424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7b-0003Qo-Ga; Fri, 10 Sep 2021 18:19:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184709.333424; Fri, 10 Sep 2021 18:19:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7b-0003Qh-Di; Fri, 10 Sep 2021 18:19:31 +0000
Received: by outflank-mailman (input) for mailman id 184709;
 Fri, 10 Sep 2021 18:19:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ConJ=OA=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOl7a-0003AT-Vv
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:31 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a408877b-c702-4904-a009-8007a6c3139b;
 Fri, 10 Sep 2021 18:19:25 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id t19so5738077lfe.13
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 11:19:25 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.23
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 10 Sep 2021 11:19:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a408877b-c702-4904-a009-8007a6c3139b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=Pi6yuMUH/Hje2P5MmRyjScNmoovfYd7eFi3wUXINX8Y=;
        b=JyXhznYvxdb0bvxR3UsrKgnKsipi87KCT1G6tVcKV02z0YOthdqcv3HpPjSOuUnEJ7
         h861M2f6pupojWC4neHnTfdHkXJK3Afm/m4NHlW25yD7bg1X6m5gl6mJfzS9Z5LdPdpz
         G3gF06YS+2QzNmvH37P6ZUxt2GDpXN3KrATRRkGhI4rUUbWLkrlDAVdCb4oW1EBZTbvo
         YOBCePVzygHtTfzrn6USJJ5ZogaLeG+nOQdvzotaSwMm/Nd9THm6PrVxRcQ407EJNiBO
         RaWVGHqLUV8AOJha3ij2fChrDqAtZpC1PLV5bT5ewLYepus1Ku1NbFBYAWUg5YVTI6wE
         jzUw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=Pi6yuMUH/Hje2P5MmRyjScNmoovfYd7eFi3wUXINX8Y=;
        b=XG+n3NJ//tePK0nQ9Vm0gkZD50quHIjYOjGLh/P3BlOUQSYrO6FNcNKOtazO6pG36x
         sHr8YcYcw3z7CMcxMoq6KueHMbRaG8BKGC8SwMzx1rgETa5gd/aSlQsGSELWnT4D/+ls
         H4d493RPVIMEmMeldfILT2++xvOSxwRp9kps47+47mcERF0g+maJJwRMnbTLJJdPR4Wk
         aRMoCJlrrXwqU02ErZyXDjW4dZKU6o2I+cJ6rc8Jbpi02HfyM2huyqjucMStv6LhQvO8
         3EQtKoeoA8Cm0VbRAmYPgw5rVl5msZ848oDzUXXmJL0O/SmG2HKJPa/1g/Wk0yDpXMKJ
         i9+A==
X-Gm-Message-State: AOAM533Ncr4//lIW/sCJYsj2Fp4fNstH/4tWgIrxDJLa0KEE9AEqwFLj
	s67jGAO95ywcFJ/EdfIVhHRseyRnzYg=
X-Google-Smtp-Source: ABdhPJzNQCIIRu+uqLkS0O2ZDBsx9oexcKKbmKXJ78NwnvHvyco3jTOxEc+D9ChSiOnM07fIgLxwwQ==
X-Received: by 2002:a19:c753:: with SMTP id x80mr1170968lff.267.1631297964434;
        Fri, 10 Sep 2021 11:19:24 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>
Subject: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
Date: Fri, 10 Sep 2021 21:18:42 +0300
Message-Id: <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

We need to pass info about maximum supported guest address
space size to the toolstack on Arm in order to properly
calculate the base and size of the extended region (safe range)
for the guest. The extended region is unused address space which
could be safely used by domain for foreign/grant mappings on Arm.
The extended region itself will be handled by the subsequents
patch.

Use p2m_ipa_bits variable on Arm, the x86 equivalent is
hap_paddr_bits.

As we change the size of structure bump the interface version.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, that recent review comments [1] haven't been addressed yet.
It is not forgotten, some clarification is needed. It will be addressed for the next version.

[1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/

Changes since RFC:
   - update patch subject/description
   - replace arch-specific sub-struct with common gpaddr_bits
     field and update code to reflect that
---
 tools/include/libxl.h            | 7 +++++++
 tools/libs/light/libxl.c         | 2 ++
 tools/libs/light/libxl_types.idl | 2 ++
 xen/arch/arm/sysctl.c            | 2 ++
 xen/arch/x86/sysctl.c            | 2 ++
 xen/include/public/sysctl.h      | 3 ++-
 6 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d..da44944 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -855,6 +855,13 @@ typedef struct libxl__ctx libxl_ctx;
  */
 #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
 
+ /*
+  * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
+  *
+  * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
+  */
+ #define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
+
 /*
  * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1
  *
diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c
index 204eb0b..c86624f 100644
--- a/tools/libs/light/libxl.c
+++ b/tools/libs/light/libxl.c
@@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
     physinfo->cap_vmtrace =
         !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace);
 
+    physinfo->gpaddr_bits = xcphysinfo.gpaddr_bits;
+
     GC_FREE;
     return 0;
 }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff6..f7437e4 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -1061,6 +1061,8 @@ libxl_physinfo = Struct("physinfo", [
     ("cap_shadow", bool),
     ("cap_iommu_hap_pt_share", bool),
     ("cap_vmtrace", bool),
+
+    ("gpaddr_bits", uint32),
     ], dir=DIR_OUT)
 
 libxl_connectorinfo = Struct("connectorinfo", [
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index f87944e..91dca4f 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -15,6 +15,8 @@
 void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
 {
     pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
+
+    pi->gpaddr_bits = p2m_ipa_bits;
 }
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a1..7b14865 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
     if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow;
+
+    pi->gpaddr_bits = hap_paddr_bits;
 }
 
 long arch_do_sysctl(
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 039ccf8..f53b42e 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -35,7 +35,7 @@
 #include "domctl.h"
 #include "physdev.h"
 
-#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014
 
 /*
  * Read console content from Xen buffer ring.
@@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
     uint64_aligned_t outstanding_pages;
     uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
     uint32_t hw_cap[8];
+    uint32_t gpaddr_bits;
 };
 
 /*
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:19:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:19:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184711.333447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7m-00049L-9R; Fri, 10 Sep 2021 18:19:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184711.333447; Fri, 10 Sep 2021 18:19:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOl7m-00049A-4z; Fri, 10 Sep 2021 18:19:42 +0000
Received: by outflank-mailman (input) for mailman id 184711;
 Fri, 10 Sep 2021 18:19:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ConJ=OA=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOl7k-0003AT-WD
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:19:41 +0000
Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6dc5f12d-6d91-4378-a5aa-df471a9af16a;
 Fri, 10 Sep 2021 18:19:27 +0000 (UTC)
Received: by mail-lf1-x136.google.com with SMTP id f18so5751061lfk.12
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 11:19:27 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a7sm625045lfi.15.2021.09.10.11.19.25
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 10 Sep 2021 11:19:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6dc5f12d-6d91-4378-a5aa-df471a9af16a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=Eoy+Pchg4s0QithVevz+IoMG+lYh7VneYRIDbLe6C1s=;
        b=UBYQf8hSPuukUyAYT2w/QXM9jAPllIyX8yIQsyPXp3C0DClgZqz8w6lhVCJYdGYnsg
         wXWf7hb9DsR5gTB2nuJUaciQJW83S1ORICEsJeiI3s2gWGy5sHJcfM760Pfu5BxXZWwY
         zVhW2B/xUXLj1mz3cK9S/Oz+bFkBeI6Nxk+477wBEjNFL6ddowBMUTJX8LIE2syddjON
         bAWPyXbL7Q3ErzNrgElyqWjxOrx3z2U/yH4DkNRSn8blmVr7HP4IJuL6OzRUTCaoNoWl
         QX3v+SnDSIKzIWsoApkuzxXWq460TtXoC6QB9ns/00b6eUr9AhW/DFpAGexKypebWoDs
         bEbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=Eoy+Pchg4s0QithVevz+IoMG+lYh7VneYRIDbLe6C1s=;
        b=NNsRaqZ57i/xucuXn7Nib4Azc+aKg80qLuAlJQ0N4jm5RxRbT6Z7JD3bGx8fT8FGbq
         q6EvfEWdbH1adwVGDsMQf9zjeCBOtoVZXVzUieK+IK0jetjfxBKa9MG4bN5r3ue4LWKL
         Yq5W3YHSwVZXewTFTeX0gA0Tq5xye2JzvvuLJI2WwcR+wFhgFgmoqiS+/Un406MBYS9i
         F/ITYv9pnyP5gpST1DV2FTL+4MSty4AEouE8flPbiNB7nHMK+wU1hS3dfQRwrLCGyMA3
         d33fmovu/Ld/Ulu8elT0aIM2jIBHvjVUvytiyoFiqx91DtH99rilNNSz2oyCjJQvpHEf
         kySA==
X-Gm-Message-State: AOAM530/XDZyjhjqzRa2Vigy0MVY8nC0ME1avj2rcf/w7PyqJFo61gGb
	CxKNKHa3rlqIjE7vzBxecfJsiwXng9M=
X-Google-Smtp-Source: ABdhPJwE934eD6zKOi3fkeOuf9eSpR3/Gy5RT+rR/hMXdjN1ibo7klQLCDOl4lFOQ2K23gVgWM1y7g==
X-Received: by 2002:a19:480c:: with SMTP id v12mr4779180lfa.60.1631297966164;
        Fri, 10 Sep 2021 11:19:26 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH V2 3/3] libxl/arm: Add handling of extended regions for DomU
Date: Fri, 10 Sep 2021 21:18:44 +0300
Message-Id: <1631297924-8658-4-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain.
- The ACPI case is not covered.

***

The algorithm to choose extended regions for non-direct mapped
DomU is simpler in comparison with the algorithm for direct mapped
Dom0. As we have a lot of unused space above 4GB, provide single
1GB-aligned region from the second RAM bank taking into the account
the maximum supported guest address space size and the amount of
memory assigned to the guest. The maximum size of the region is 128GB.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Changes since RFC:
   - update patch description
   - drop uneeded "extended-region" DT property
   - clear reg array in finalise_ext_region() and add a TODO
---
 tools/libs/light/libxl_arm.c | 89 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 87 insertions(+), 2 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6..8c1d9d7 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -615,9 +615,12 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
                               "xen,xen");
     if (res) return res;
 
-    /* reg 0 is grant table space */
+    /*
+     * reg 0 is a placeholder for grant table space, reg 1 is a placeholder
+     * for the extended region.
+     */
     res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+                            2, 0, 0, 0, 0);
     if (res) return res;
 
     /*
@@ -1069,6 +1072,86 @@ static void finalise_one_node(libxl__gc *gc, void *fdt, const char *uname,
     }
 }
 
+#define ALIGN_UP_TO_GB(x)   (((x) + GB(1) - 1) & (~(GB(1) - 1)))
+
+#define EXT_REGION_SIZE   GB(128)
+
+static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image *dom)
+{
+    void *fdt = dom->devicetree_blob;
+    uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
+    be32 *cells = &regs[0];
+    uint64_t region_size = 0, region_base, bank1end_align, bank1end_max;
+    uint32_t gpaddr_bits;
+    libxl_physinfo info;
+    int offset, rc;
+
+    offset = fdt_path_offset(fdt, "/hypervisor");
+    assert(offset > 0);
+
+    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
+        LOG(WARN, "The extended region is only supported for 64-bit guest");
+        goto out;
+    }
+
+    rc = libxl_get_physinfo(CTX, &info);
+    assert(!rc);
+
+    gpaddr_bits = info.gpaddr_bits;
+    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
+
+    /*
+     * Try to allocate single 1GB-aligned extended region from the second RAM
+     * bank (above 4GB) taking into the account the maximum supported guest
+     * address space size and the amount of memory assigned to the guest.
+     * The maximum size of the region is 128GB.
+     */
+    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
+    bank1end_align = GUEST_RAM1_BASE +
+        ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << XC_PAGE_SHIFT);
+
+    if (bank1end_max <= bank1end_align) {
+        LOG(WARN, "The extended region cannot be allocated, not enough space");
+        goto out;
+    }
+
+    if (bank1end_max - bank1end_align > EXT_REGION_SIZE) {
+        region_base = bank1end_max - EXT_REGION_SIZE;
+        region_size = EXT_REGION_SIZE;
+    } else {
+        region_base = bank1end_align;
+        region_size = bank1end_max - bank1end_align;
+    }
+
+out:
+    /*
+     * The first region for grant table space must be always present.
+     * If we managed to allocate the extended region then insert it as
+     * a second region.
+     * TODO If we failed to allocate the region, we end up inserting
+     * zero-sized region. This is because we don't know in advance when
+     * creating hypervisor node whether it would be possible to allocate
+     * a region, but we have to create a placeholder anyway. The Linux driver
+     * is able to deal with by checking the region size. We cannot choose
+     * a region when creating hypervisor node because the guest memory layout
+     * is not know at that moment (and dom->rambank_size[1] is empty).
+     * We need to find a way not to expose invalid regions.
+     */
+    memset(regs, 0, sizeof(regs));
+    set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+              GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+    if (region_size > 0) {
+        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
+            region_base, region_base + region_size);
+
+        set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                  region_base, region_size);
+    }
+
+    rc = fdt_setprop_inplace(fdt, offset, "reg", regs, sizeof(regs));
+    assert(!rc);
+}
+
 int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
                                                uint32_t domid,
                                                libxl_domain_config *d_config,
@@ -1109,6 +1192,8 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
 
     }
 
+    finalise_ext_region(gc, dom);
+
     for (i = 0; i < GUEST_RAM_BANKS; i++) {
         const uint64_t size = (uint64_t)dom->rambank_size[i] << XC_PAGE_SHIFT;
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:42:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:42:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184738.333458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlU0-0000Zd-4A; Fri, 10 Sep 2021 18:42:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184738.333458; Fri, 10 Sep 2021 18:42:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlU0-0000ZW-0y; Fri, 10 Sep 2021 18:42:40 +0000
Received: by outflank-mailman (input) for mailman id 184738;
 Fri, 10 Sep 2021 18:42:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ConJ=OA=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mOlTz-0000ZQ-Dj
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:42:39 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b9318662-e837-45c7-8ad4-8ab76cf1901b;
 Fri, 10 Sep 2021 18:42:38 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id n2so6006981lfk.0
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 11:42:38 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id n25sm664708ljj.42.2021.09.10.11.42.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 10 Sep 2021 11:42:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9318662-e837-45c7-8ad4-8ab76cf1901b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=QPygq3zscXFjX5u+NyrwQkEwCIrdjjmY8mMP7XiNmpY=;
        b=kHXJh8mfEcsLIcEtHnsLDcwFbrCZqle/rOf6MJGepSmxnTOCzlajt7QSKHXc8JRznW
         0EHNv5gHKZpB2Y87OCs2yLgYPNxV3052YAC2mLoiQzxfM3HUl+0imYpQQCdRg0siaq2W
         hG0ZojrWSrgmmmPtivXNaV2bp5eCAZt2OiaHlQhyZqFRVoRx6u8qgrdbohn1PJ6lyNrv
         FuXI9WFLLmQKj9jp+19IPBGtS4FL5uXf5UpEMZ56FLyBZVOfgkX6DGzzdrxNvLIW1fFS
         L9Tr2KG49kfpVU503VL/Fx54y6XzRt5jG4oWzMPjkVOfp1K/tfKGBW+W0UmYqXK9nzEg
         8rew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=QPygq3zscXFjX5u+NyrwQkEwCIrdjjmY8mMP7XiNmpY=;
        b=jm7Kn2qppWE/D+iVyONuVfh+HvuJFbo5hHFsebe6IlzFceqYw40tXPoR0u4bspRwzJ
         2LOFKBknoLi2kNGQLTf8xp3W/LmBDmo6DJpbm0yFhInoEaWCs7TLDZ1wLxn5fdDHjDTc
         mUy4+PCZZzUArXMGwlQPfQvu0ysD1aWsYY8BkuJDIJfr99/hQqcOt1XDmSMgISIRkguU
         jLixrzy7GQ1Q0I73XDgVr5FwOX+LEpR1Q4Sn2x0ADkpaamflGAcQ2p+zWEv2VwTbJMMw
         pXJWZJt7gGEIx/YtytmNJHAdLUPnL2Gl52XF3J+9Zs+wHV/XY+mEC66ZZU3241k6ygi+
         Y0hg==
X-Gm-Message-State: AOAM533zsouZ8OAJxhZothSUuJ1iKKyU49RxRSJ2BtpNmXbNdGt+aVee
	spELBV92w8JAzaPQpYU7BfM=
X-Google-Smtp-Source: ABdhPJz0c3VoBzA6YMfDF3kRSx9Np/OicfnbcQbSHkKpf+somS1PAKHXJ8ORqrIs8jU6fV2m8fklSA==
X-Received: by 2002:a05:6512:c3:: with SMTP id c3mr5107029lfp.364.1631299357388;
        Fri, 10 Sep 2021 11:42:37 -0700 (PDT)
Subject: Re: [RFC PATCH 0/3] Add handling of extended regions (safe ranges) on
 Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
From: Oleksandr <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
 <6b2bea7a-63dc-bb42-9e0b-fb49b05e310a@gmail.com>
Message-ID: <530cc82a-9727-71ea-536a-4d4824de83fd@gmail.com>
Date: Fri, 10 Sep 2021 21:42:35 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <6b2bea7a-63dc-bb42-9e0b-fb49b05e310a@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


Hello all


On 10.09.21 02:20, Oleksandr wrote:
>
> Hello all
>
> On 07.09.21 20:09, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> You can find an initial discussion at [1].
>>
>> The extended region (safe range) is a region of guest physical
>> address space which is unused and could be safely used to create
>> grant/foreign mappings instead of wasting real RAM pages from
>> the domain memory for establishing these mappings.
>>
>> The extended regions are chosen at the domain creation time and
>> advertised to it via "reg" property under hypervisor node in
>> the guest device-tree.
>>
>> The extended regions are calculated differently for direct mapped
>> Dom0 (with and without IOMMU) and non-direct mapped DomUs.
>>
>> Please note the following limitations:
>> - The extended region feature is only supported for 64-bit domain.
>> - The ACPI case is not covered.
>>
>> Also please note that we haven't figured out yet how to properly
>> extend the Xen hypervisor device-tree bindings on Arm (either via new
>> compatible or via new property). I decided to go with new property
>> for now, but this can be changed. This uncertainty is the main reason
>> why this series is marked as RFC.
>
> Sorry, I messed up the device tree binding's purpose here.
>
> New DT property "extended-region" (to inform guest about the presence 
> of additional regions in "reg" property) is not really needed. Guest 
> can simply infer that from the number of regions
> in "reg" property (region 0 - reserved for grant table space, regions 
> 1...n - extended regions).
> Instead, new compatible/property will be needed (but only after this 
> patch [1] or alternative goes in) to indicate that "region 0 is safe 
> to use". Until this patch is merged it is
> not safe to use extended regions for the grant table space.
>
> Thanks to Julien for clarifying these bits.
>
> I am going to remove the advertisement of unneeded "extended-region" 
> property in the code and send new version soon.

I have just pushed new version [1]. Please ignore this one. Sorry for 
the inconvenience.

[1] 
https://lore.kernel.org/xen-devel/1631297924-8658-1-git-send-email-olekstysh@gmail.com/


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Fri Sep 10 18:45:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 18:45:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184745.333468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlWY-0001Bu-I5; Fri, 10 Sep 2021 18:45:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184745.333468; Fri, 10 Sep 2021 18:45:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlWY-0001Bn-Eq; Fri, 10 Sep 2021 18:45:18 +0000
Received: by outflank-mailman (input) for mailman id 184745;
 Fri, 10 Sep 2021 18:45:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOlWX-0001Bh-SN
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 18:45:17 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3da3eacc-1267-11ec-b276-12813bfff9fa;
 Fri, 10 Sep 2021 18:45:17 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 008C961209;
 Fri, 10 Sep 2021 18:45:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3da3eacc-1267-11ec-b276-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631299516;
	bh=80PZEAVpmtJQ9Y24Fvr58NoHO7NPdZB30fqvyK+khYM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=aymawEC+KOtspruSMgp1DP9jTIdM16Q1TKzf6QjsDymJwkIWP/8pazP8jVw5llhJD
	 rlBdNyOV2wMF5DnHDP4sPd9QxYbaCwlyMr0aBG9Ak/v0gh/2GwJ/2iVI2gYnTz4CjM
	 llnArayXVO6VQYs35CO4j7HcUoTmQgb/HJn4xGOS5iXYegVIPR2H3N/0gsh8j0pmg1
	 hfQxbk1Cpb0VzX02Xr8ymkxu4yGzzYYoInIzPDKoZvPqMZVajwUi9i2YyqJ3P+pjtM
	 2KniCysHm6y7kvXlPJuGGX6hiciyS6S1J80uTNMtfYi/M+5luCywfHVNvaH9lzP4/N
	 FeysNyquzXMfw==
Date: Fri, 10 Sep 2021 11:45:14 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com
Subject: Re: [PATCH 06/11] xen/domain: Call pci_release_devices() when
 releasing domain resources
In-Reply-To: <20210903083347.131786-7-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109101143550.10523@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-7-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> This is the very same what we do for DT devices. What is more
> that x86 already calls pci_release_devices().

Rewording suggestion:

This is the very same that we already do for DT devices. Moreover, x86
already calls pci_release_devices().



> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
>  xen/arch/arm/domain.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index d99c653626e4..4e40c4098280 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -985,7 +985,8 @@ static int relinquish_memory(struct domain *d, struct page_list_head *list)
>   * function which may return -ERESTART.
>   */
>  enum {
> -    PROG_tee = 1,
> +    PROG_pci = 1,
> +    PROG_tee,
>      PROG_xen,
>      PROG_page,
>      PROG_mapping,
> @@ -1022,6 +1023,12 @@ int domain_relinquish_resources(struct domain *d)
>  #ifdef CONFIG_IOREQ_SERVER
>          ioreq_server_destroy_all(d);
>  #endif
> +#ifdef CONFIG_HAS_PCI
> +    PROGRESS(pci):
> +        ret = pci_release_devices(d);
> +        if ( ret )
> +            return ret;
> +#endif
>  
>      PROGRESS(tee):
>          ret = tee_relinquish_resources(d);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 19:07:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 19:07:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184753.333480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlrU-0004AG-CG; Fri, 10 Sep 2021 19:06:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184753.333480; Fri, 10 Sep 2021 19:06:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOlrU-0004A9-8g; Fri, 10 Sep 2021 19:06:56 +0000
Received: by outflank-mailman (input) for mailman id 184753;
 Fri, 10 Sep 2021 19:06:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOlrS-0004A3-Ji
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 19:06:54 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 423c1c50-126a-11ec-b278-12813bfff9fa;
 Fri, 10 Sep 2021 19:06:53 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2F2F36103D;
 Fri, 10 Sep 2021 19:06:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 423c1c50-126a-11ec-b278-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631300812;
	bh=BZGtJxCFbzrAbWpwiJa1S/SwJiopZbvSxX6mT696BQo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Q2mY/68TDirhGXyHdmzIYUdd39IPawgXbKu9nu1dZ2DKoOQ52RFJZ2vpOGeG1zn28
	 QUaiChP553x9Ssf5iRhFHI+jR1s5NB0f279W0QYVWhD7v+G+s8KrS9QN60psucPCGb
	 kZsudF3PiUv9FfZ5IMDN0R0I8DP15CMyYTsK/MmoNEZuaMe5GQUuYsc9nojBlG3lMt
	 KeS1IEwWOxpn2AJDusp56Ns2nTwYJoUXUfq2mCLSJskLQrnd20uxZCpmZC7qo10yCy
	 HbF0KwTDnvUI5Lwv5sSdE7gD8M6rTid39Ogd7TmQ+g/bqHhXi5LPrfdb8XkqIw8CtL
	 C4Uvf1LBDcPgA==
Date: Fri, 10 Sep 2021 12:06:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
In-Reply-To: <20210903083347.131786-9-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109101159100.10523@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-9-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Arm's PCI passthrough implementation doesn't support legacy interrupts,
> but MSI/MSI-X. This can be the case for other platforms too.
> For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
> it to the CFLAGS and compile the relevant code in the toolstack only if
> applicable.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Ian Jackson <iwj@xenproject.org>
> Cc: Juergen Gross <jgross@suse.com>
> ---
>  tools/libs/light/Makefile    |  4 ++++
>  tools/libs/light/libxl_pci.c | 13 +++++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
> index 7d8c51d49242..bd3f6be2a183 100644
> --- a/tools/libs/light/Makefile
> +++ b/tools/libs/light/Makefile
> @@ -46,6 +46,10 @@ CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
>  	-Wno-declaration-after-statement -Wformat-nonliteral
>  CFLAGS += -I.
>  
> +ifeq ($(CONFIG_X86),y)
> +CFLAGS += -DCONFIG_PCI_SUPP_LEGACY_IRQ
> +endif
> +
>  SRCS-$(CONFIG_X86) += libxl_cpuid.c
>  SRCS-$(CONFIG_X86) += libxl_x86.c
>  SRCS-$(CONFIG_X86) += libxl_psr.c
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> index 59f3686fc85e..cd4fea46c3f7 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1434,6 +1434,7 @@ static void pci_add_dm_done(libxl__egc *egc,
>          }
>      }
>      fclose(f);
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ

As Juergen pointed out the logic is inverted.

I also think we need to come up with a better way to handle this #ifdef
logic in this file.

For instance, could we let this function try to open sysfs_path? I
imagine it would fail, right? If so, we could just have an #ifdef inside
the failure check.

Alternatively, could we have a small #ifdef right here doing:

#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
goto out_no_irq;
#endif

?


Even better, would be to introduce a static inline as follows:

static inline bool pci_supp_legacy_irq(void)
{
#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
    return false;
#else
    return true;
#endif
}

Then in libxl_pci.c you can avoid all ifdefs:

if (!pci_supp_legacy_irq()))
    goto out_no_irq;


>      sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                  pci->bus, pci->dev, pci->func);
>      f = fopen(sysfs_path, "r");
> @@ -1460,6 +1461,7 @@ static void pci_add_dm_done(libxl__egc *egc,
>          }
>      }
>      fclose(f);
> +#endif
>  
>      /* Don't restrict writes to the PCI config space from this VM */
>      if (pci->permissive) {
> @@ -1471,7 +1473,9 @@ static void pci_add_dm_done(libxl__egc *egc,
>          }
>      }
>  
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
>  out_no_irq:
> +#endif
>      if (!isstubdom) {
>          if (pci->rdm_policy == LIBXL_RDM_RESERVE_POLICY_STRICT) {
>              flag &= ~XEN_DOMCTL_DEV_RDM_RELAXED;
> @@ -1951,7 +1955,9 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
>                                       pci->bus, pci->dev, pci->func);
>          FILE *f = fopen(sysfs_path, "r");
>          unsigned int start = 0, end = 0, flags = 0, size = 0;
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
>          int irq = 0;
> +#endif

I'd let this compile if possible.


>          int i;
>          if (f == NULL) {
> @@ -1983,6 +1989,7 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
>          }
>          fclose(f);
>  skip1:
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ

Here we could do instead:

if (!pci_supp_legacy_irq()))
    goto skip_irq;


>          sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                 pci->bus, pci->dev, pci->func);
>          f = fopen(sysfs_path, "r");
> @@ -2001,8 +2008,14 @@ skip1:
>              }
>          }
>          fclose(f);
> +#else
> +        /* Silence error: label at end of compound statement */
> +        ;
> +#endif
>      }
> +#ifndef CONFIG_PCI_SUPP_LEGACY_IRQ
>  skip_irq:
> +#endif
>      rc = 0;
>  out_fail:
>      pci_remove_detached(egc, prs, rc); /* must be last */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 20:12:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 20:12:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184773.333507 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOmt5-0004aS-F3; Fri, 10 Sep 2021 20:12:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184773.333507; Fri, 10 Sep 2021 20:12:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOmt5-0004aL-C2; Fri, 10 Sep 2021 20:12:39 +0000
Received: by outflank-mailman (input) for mailman id 184773;
 Fri, 10 Sep 2021 20:12:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOmt4-0004aF-32
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 20:12:38 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 70cd74a2-1273-11ec-b284-12813bfff9fa;
 Fri, 10 Sep 2021 20:12:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id CDD4360F5D;
 Fri, 10 Sep 2021 20:12:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70cd74a2-1273-11ec-b284-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631304756;
	bh=tIrimlsj88EoS8APLqnSckCT6jioW5gAK0ZIE3Kgbqw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ch0BCizF3TFysMGqZDLCVbgWDvB1JGhNhpBj7z+PtHgcRHXa6cW7t9A6tITq5VL1X
	 YzL0MU2SW1kkpOEoWDuWBcbgE8pEwE+mhdGmJ+s7ro9UGNajiUpmFTp7ide3akkatr
	 eTOdSEDjcjaA15AJP1UXKubnTdk9DhSRqxtNJqLkDSBqCWxJjIJJcMKpeE3F+vcSJI
	 MbhO50A3JkepoTPBriNLZqMKBeugogPAj9NupZQJIjJ4RUMZII9kFKEPCQNn+fDG8u
	 r32dANhKEdaWRo+hHmW45WCGobzY1mUWUfsZGxs+D+5YNQoofOEFm0HUrVnnWbt5QN
	 GqwC+ompL4CTA==
Date: Fri, 10 Sep 2021 13:12:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for hardware
 domain
In-Reply-To: <20210903083347.131786-10-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-10-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> In order vPCI to work it needs all access to PCI configuration space
> (ECAM) to be synchronized among all entities, e.g. hardware domain and
> guests. For that implement PCI host bridge specific callbacks to
> properly setup those ranges depending on particular host bridge
> implementation.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>  xen/arch/arm/pci/ecam.c            | 11 +++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
>  xen/arch/arm/vpci.c                | 13 +++++++++++++
>  xen/include/asm-arm/pci.h          |  8 ++++++++
>  4 files changed, 48 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> index 91c691b41fdf..92ecb2e0762b 100644
> --- a/xen/arch/arm/pci/ecam.c
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -42,6 +42,16 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>      return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
>  }
>  
> +static int pci_ecam_register_mmio_handler(struct domain *d,
> +                                          struct pci_host_bridge *bridge,
> +                                          const struct mmio_handler_ops *ops)
> +{
> +    struct pci_config_window *cfg = bridge->sysdata;
> +
> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
> +    return 0;
> +}

Given that struct pci_config_window is generic (it is not specific to
one bridge), I wonder if we even need the .register_mmio_handler
callback here.

In fact, pci_host_bridge->sysdata doesn't even need to be a void*, it
could be a struct pci_config_window*, right?

We could simply call:

register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);

for each bridge directly from domain_vpci_init ?




>  /* ECAM ops */
>  const struct pci_ecam_ops pci_generic_ecam_ops = {
>      .bus_shift  = 20,
> @@ -49,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>          .map_bus                = pci_ecam_map_bus,
>          .read                   = pci_generic_config_read,
>          .write                  = pci_generic_config_write,
> +        .register_mmio_handler  = pci_ecam_register_mmio_handler,
>      }
>  };
>  
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index d2fef5476b8e..a89112bfbb7c 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -318,6 +318,22 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
>      }
>      return bridge->dt_node;
>  }
> +
> +int pci_host_iterate_bridges(struct domain *d,
> +                             int (*clb)(struct domain *d,
> +                                        struct pci_host_bridge *bridge))
> +{
> +    struct pci_host_bridge *bridge;
> +    int err;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        err = clb(d, bridge);
> +        if ( err )
> +            return err;
> +    }
> +    return 0;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index da8b1ca13c07..258134292458 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -74,11 +74,24 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
>      .write = vpci_mmio_write,
>  };
>  
> +static int vpci_setup_mmio_handler(struct domain *d,
> +                                   struct pci_host_bridge *bridge)
> +{
> +    if ( bridge->ops->register_mmio_handler )
> +        return bridge->ops->register_mmio_handler(d, bridge,
> +                                                  &vpci_mmio_handler);
> +    return 0;
> +}
> +
>  int domain_vpci_init(struct domain *d)
>  {
>      if ( !has_vpci(d) )
>          return 0;
>  
> +    if ( is_hardware_domain(d) )
> +        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
> +
> +    /* Guest domains use what is programmed in their device tree. */
>      register_mmio_handler(d, &vpci_mmio_handler,
>                            GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
>  
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 7dc4c8dc9026..2c7c7649e00f 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -17,6 +17,8 @@
>  #ifndef __ARM_PCI_H__
>  #define __ARM_PCI_H__
>  
> +#include <asm/mmio.h>
> +
>  #ifdef CONFIG_HAS_PCI
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> @@ -77,6 +79,9 @@ struct pci_ops {
>                  uint32_t reg, uint32_t len, uint32_t *value);
>      int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
>                   uint32_t reg, uint32_t len, uint32_t value);
> +    int (*register_mmio_handler)(struct domain *d,
> +                                 struct pci_host_bridge *bridge,
> +                                 const struct mmio_handler_ops *ops);
>  };
>  
>  /*
> @@ -107,6 +112,9 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
>                                  uint16_t *segment);
>  struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
>  
> +int pci_host_iterate_bridges(struct domain *d,
> +                             int (*clb)(struct domain *d,
> +                                        struct pci_host_bridge *bridge));
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  struct arch_pci_dev { };
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 20:30:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 20:30:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184780.333518 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnA0-0007Al-VE; Fri, 10 Sep 2021 20:30:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184780.333518; Fri, 10 Sep 2021 20:30:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnA0-0007Ae-Ri; Fri, 10 Sep 2021 20:30:08 +0000
Received: by outflank-mailman (input) for mailman id 184780;
 Fri, 10 Sep 2021 20:30:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOnA0-0007AY-E4
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 20:30:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7dd30426-6353-49b5-965c-8376f547e563;
 Fri, 10 Sep 2021 20:30:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2476061074;
 Fri, 10 Sep 2021 20:30:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7dd30426-6353-49b5-965c-8376f547e563
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631305806;
	bh=rfdBbdaI5Rc/WfF8f5SA31qpkpvfKGbPHuwbBJfsqbU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Ou1fBqQNYMA6UXSMIoYfQzVm5NCH0sb071L0ykcTN9SxF/NL/+OV/mmMc87oGthLw
	 fMdFKSXPt5GbzTlcvHnMWcE1dp0P/BuxJtrGqXsJ9xyamzTpY2LULLcOfskf3QO/jd
	 H8q9E/bZP9Je1lCqgBQIXPm8RgRgccboIejM//YyINxHwY7GYNyRutt6M0exJvAJEa
	 zLGSAGVyM7KhrY+5IXDzAvF6D6uLqII23gSKJg4CktO9xjRMzvIfmk2eo73gtS8Xa4
	 haxtxc5joZs3swoBwKKhH7PzsZwyyAve2ghx/PO13HmJTPNaAJ2J1A9ai0fU+le10v
	 i08JHGPYELd0A==
Date: Fri, 10 Sep 2021 13:30:04 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "sstabellini@kernel.org" <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
In-Reply-To: <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
Message-ID: <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com> <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com> <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com> <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1425355773-1631305806=:10523"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1425355773-1631305806=:10523
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 10 Sep 2021, Julien Grall wrote:
> On 10/09/2021 15:01, Oleksandr Andrushchenko wrote:
> > On 10.09.21 16:18, Julien Grall wrote:
> > > On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:
> > > > Hi, Julien!
> > > 
> > > Hi Oleksandr,
> > > 
> > > > On 09.09.21 20:58, Julien Grall wrote:
> > > > > On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> > > > > > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > > > > 
> > > > > > Host bridge controller's ECAM space is mapped into Domain-0's p2m,
> > > > > > thus it is not possible to trap the same for vPCI via MMIO handlers.
> > > > > > For this to work we need to not map those while constructing the
> > > > > > domain.
> > > > > > 
> > > > > > Note, that during Domain-0 creation there is no pci_dev yet
> > > > > > allocated for
> > > > > > host bridges, thus we cannot match PCI host and its associated
> > > > > > bridge by SBDF. Use dt_device_node field for checks instead.
> > > > > > 
> > > > > > Signed-off-by: Oleksandr Andrushchenko
> > > > > > <oleksandr_andrushchenko@epam.com>
> > > > > > ---
> > > > > >     xen/arch/arm/domain_build.c        |  3 +++
> > > > > >     xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
> > > > > >     xen/arch/arm/pci/pci-host-common.c | 22 ++++++++++++++++++++++
> > > > > >     xen/include/asm-arm/pci.h          | 12 ++++++++++++
> > > > > >     4 files changed, 54 insertions(+)
> > > > > > 
> > > > > > diff --git a/xen/arch/arm/domain_build.c
> > > > > > b/xen/arch/arm/domain_build.c
> > > > > > index da427f399711..76f5b513280c 100644
> > > > > > --- a/xen/arch/arm/domain_build.c
> > > > > > +++ b/xen/arch/arm/domain_build.c
> > > > > > @@ -1257,6 +1257,9 @@ static int __init map_range_to_domain(const
> > > > > > struct dt_device_node *dev,
> > > > > >             }
> > > > > >         }
> > > > > >     +    if ( need_mapping && (device_get_class(dev) == DEVICE_PCI)
> > > > > > ) > +        need_mapping = pci_host_bridge_need_p2m_mapping(d, dev,
> > > > > addr, len);
> > > > > 
> > > > > AFAICT, with device_get_class(dev), you know whether the hostbridge is
> > > > > used by Xen. Therefore, I would expect that we don't want to map all
> > > > > the regions of the hostbridges in dom0 (including the BARs).
> > > > > 
> > > > > Can you clarify it?
> > > > We only want to trap ECAM, not MMIOs and any other memory regions as the
> > > > bridge is
> > > > 
> > > > initialized and used by Domain-0 completely.
> > > 
> > > What do you mean by "used by Domain-0 completely"? The hostbridge is owned
> > > by Xen so I don't think we can let dom0 access any MMIO regions by
> > > default.
> > 
> > Now it's my time to ask why do you think Xen owns the bridge?
> > 
> > All the bridges are passed to Domain-0, are fully visible to Domain-0,
> > initialized in Domain-0.
> > 
> > Enumeration etc. is done in Domain-0. So how comes that Xen owns the bridge?
> > In what way it does?
> > 
> > Xen just accesses the ECAM when it needs it, but that's it. Xen traps ECAM
> > access - that is true.
> > 
> > But it in no way uses MMIOs etc. of the bridge - those under direct control
> > of Domain-0
> 
> So I looked on the snipped of the design document you posted. I think you are
> instead referring to a different part:
> 
> " PCI-PCIe enumeration is a process of detecting devices connected to its
> host.
> It is the responsibility of the hardware domain or boot firmware to do the PCI
> enumeration and configure the BAR, PCI capabilities, and MSI/MSI-X
> configuration."
> 
> But I still don't see why it means we have to map the MMIOs right now. Dom0
> should not need to access the MMIOs (aside the hostbridge registers) until the
> BARs are configured.

This is true especially when we are going to assign a specific PCIe
device to a DomU. In that case, the related MMIO regions are going to be
mapped to the DomU and it would be a bad idea to also keep them mapped
in Dom0. Once we do PCIe assignment, the MMIO region of the PCIe device
being assigned should only be mapped in the DomU, right?

If so, it would be better if the MMIO region in question was never
mapped into Dom0 at all rather having to unmap it.

With the current approach, given that the entire PCIe aperture is mapped
to Dom0 since boot, we would have to identify the relevant subset region
and unmap it from Dom0 when doing assignment.
--8323329-1425355773-1631305806=:10523--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 21:14:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 21:14:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184788.333528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnqt-0004J3-EM; Fri, 10 Sep 2021 21:14:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184788.333528; Fri, 10 Sep 2021 21:14:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnqt-0004Iw-BT; Fri, 10 Sep 2021 21:14:27 +0000
Received: by outflank-mailman (input) for mailman id 184788;
 Fri, 10 Sep 2021 21:14:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOnqs-0004Iq-C9
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 21:14:26 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 130529e2-127c-11ec-b289-12813bfff9fa;
 Fri, 10 Sep 2021 21:14:25 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id DD81D600D4;
 Fri, 10 Sep 2021 21:14:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 130529e2-127c-11ec-b289-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631308464;
	bh=vfbat+5CVfuz+zm5IcGyNDCj1urgE5qTYsCRF2aePKs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ZbLQdcb9QTi8Nc4l7ON++RpOo/2dXgQGNdWr8oDqMFhWQG3GDCK/0Ec/x7EcsPnLl
	 21QpoaI3CobP2jotyikHbsAwvuNJyLM51qzKdsfAeOXvFXc++uaHWCpLscQchTUTU0
	 JkdqlAEyAl+AYKnAO7uI+U82aPqUd1sy5tW+XicksKpv722LPXBWnh2TOjAUojvimz
	 5spgiqjGTEd2fepQF6P5XN2II+FTtfP2uRYDtcu0Bea8x1yEjq5bISsJXQiHRiMS6v
	 4X5W5OTHU5YZZQnAjBYNj1j8WuY651K3qp8p1D4aC6LVa/0VSmtsAMx5n09siCqu2R
	 YBEGbr+bPMc/w==
Date: Fri, 10 Sep 2021 14:14:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, 
    bertrand.marquis@arm.com, rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH 4/9] vpci/header: Add and remove register handlers
 dynamically
In-Reply-To: <20210903100831.177748-5-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109101412111.10523@sstabellini-ThinkPad-T480s>
References: <20210903100831.177748-1-andr2000@gmail.com> <20210903100831.177748-5-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 3 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add relevant vpci register handlers when assigning PCI device to a domain
> and remove those when de-assigning. This allows having different
> handlers for different domains, e.g. hwdom and other guests.
> 
> Use stubs for guest domains for now.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>  xen/drivers/vpci/header.c | 78 +++++++++++++++++++++++++++++++++++----
>  xen/drivers/vpci/vpci.c   |  4 +-
>  xen/include/xen/vpci.h    |  4 ++
>  3 files changed, 76 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index 31bca7a12942..5218b1af247e 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -397,6 +397,17 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
>      pci_conf_write32(pdev->sbdf, reg, val);
>  }
>  
> +static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
> +                            uint32_t val, void *data)
> +{
> +}
> +
> +static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
> +                               void *data)
> +{
> +    return 0xffffffff;
> +}
>  static void rom_write(const struct pci_dev *pdev, unsigned int reg,
>                        uint32_t val, void *data)
>  {
> @@ -445,14 +456,25 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
>          rom->addr = val & PCI_ROM_ADDRESS_MASK;
>  }
>  
> -static int add_bar_handlers(struct pci_dev *pdev)
> +static void guest_rom_write(const struct pci_dev *pdev, unsigned int reg,
> +                            uint32_t val, void *data)
> +{
> +}
> +
> +static uint32_t guest_rom_read(const struct pci_dev *pdev, unsigned int reg,
> +                               void *data)
> +{
> +    return 0xffffffff;
> +}
> +
> +static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom)
>  {
>      unsigned int i;
>      struct vpci_header *header = &pdev->vpci->header;
>      struct vpci_bar *bars = header->bars;
>      int rc;
>  
> -    /* Setup a handler for the command register. */
> +    /* Setup a handler for the command register: same for hwdom and guests. */
>      rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
>                             2, header);
>      if ( rc )
> @@ -475,8 +497,13 @@ static int add_bar_handlers(struct pci_dev *pdev)
>                  rom_reg = PCI_ROM_ADDRESS;
>              else
>                  rom_reg = PCI_ROM_ADDRESS1;
> -            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
> -                                   rom_reg, 4, &bars[i]);
> +            if ( is_hwdom )
> +                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
> +                                       rom_reg, 4, &bars[i]);
> +            else
> +                rc = vpci_add_register(pdev->vpci,
> +                                       guest_rom_read, guest_rom_write,
> +                                       rom_reg, 4, &bars[i]);
>              if ( rc )
>                  return rc;
>          }
> @@ -485,8 +512,13 @@ static int add_bar_handlers(struct pci_dev *pdev)
>              uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
>  
>              /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
> -            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
> -                                   4, &bars[i]);
> +            if ( is_hwdom )
> +                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write,
> +                                       reg, 4, &bars[i]);
> +            else
> +                rc = vpci_add_register(pdev->vpci,
> +                                       guest_bar_read, guest_bar_write,
> +                                       reg, 4, &bars[i]);
>              if ( rc )
>                  return rc;
>          }
> @@ -520,7 +552,7 @@ static int init_bars(struct pci_dev *pdev)
>      }
>  
>      if ( pdev->ignore_bars )
> -        return add_bar_handlers(pdev);
> +        return add_bar_handlers(pdev, true);
>  
>      /* Disable memory decoding before sizing. */
>      cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
> @@ -582,7 +614,7 @@ static int init_bars(struct pci_dev *pdev)
>                                PCI_ROM_ADDRESS_ENABLE;
>      }
>  
> -    rc = add_bar_handlers(pdev);
> +    rc = add_bar_handlers(pdev, true);
>      if ( rc )
>      {
>          pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
> @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev)
>  }
>  REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
>  
> +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev)
> +{
> +    int rc;
> +
> +    /* Remove previously added registers. */
> +    vpci_remove_device_registers(pdev);
> +
> +    /* It only makes sense to add registers for hwdom or guest domain. */
> +    if ( d->domain_id >= DOMID_FIRST_RESERVED )
> +        return 0;

This check is redundant, isn't it? Because it is already checked by the
caller?


> +    if ( is_hardware_domain(d) )
> +        rc = add_bar_handlers(pdev, true);
> +    else
> +        rc = add_bar_handlers(pdev, false);

NIT:

  rc = add_bar_handlers(pdev, is_hardware_domain(d));


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 21:20:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 21:20:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184795.333539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnwj-0005gk-4J; Fri, 10 Sep 2021 21:20:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184795.333539; Fri, 10 Sep 2021 21:20:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOnwj-0005gd-1U; Fri, 10 Sep 2021 21:20:29 +0000
Received: by outflank-mailman (input) for mailman id 184795;
 Fri, 10 Sep 2021 21:20:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOnwh-0005gT-OT; Fri, 10 Sep 2021 21:20:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOnwh-0008IT-EH; Fri, 10 Sep 2021 21:20:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOnwh-0002Cj-2e; Fri, 10 Sep 2021 21:20:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOnwh-0002wE-0I; Fri, 10 Sep 2021 21:20:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=McOgotGpV9qKx1/a8GRAQ7+p0dg4L7OTrfelG/DmSI8=; b=N3jjd385csnDnZGJ8I+o9UIob4
	zo20sZ8h3q65ifOdkqZY6tpVSYrVnoqkjolDIHciDxQIK9FjBxtxrJawoLb0PCwdyJRWXoWl8cnYK
	c/I7dMsf4hO1Qda1qk+YCrydCSt+cpaXt8lpB6xZJrvjlf4qjYU6YgRejebsiNzwf1js=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164922-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164922: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2e2e22c7d50392fa5cced3e0334b217426f48b7a
X-Osstest-Versions-That:
    xen=56abcf1a58bdaef18579cf2ce8645c8c72a2b749
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 10 Sep 2021 21:20:27 +0000

flight 164922 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164922/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164892
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164892
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164892
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164892
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164892
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164892
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164892
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164892
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164892
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164892
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164892
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164892
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  2e2e22c7d50392fa5cced3e0334b217426f48b7a
baseline version:
 xen                  56abcf1a58bdaef18579cf2ce8645c8c72a2b749

Last test of basis   164892  2021-09-08 21:09:17 Z    2 days
Testing same since   164922  2021-09-09 21:01:25 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   56abcf1a58..2e2e22c7d5  2e2e22c7d50392fa5cced3e0334b217426f48b7a -> master


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 21:42:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 21:42:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184808.333562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOoHd-0000bW-64; Fri, 10 Sep 2021 21:42:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184808.333562; Fri, 10 Sep 2021 21:42:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOoHd-0000bP-2l; Fri, 10 Sep 2021 21:42:05 +0000
Received: by outflank-mailman (input) for mailman id 184808;
 Fri, 10 Sep 2021 21:42:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VIAe=OA=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mOoHc-0000bJ-6G
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 21:42:04 +0000
Received: from mail-ed1-x529.google.com (unknown [2a00:1450:4864:20::529])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2ad851b2-c48d-47c6-b7e3-1883313dcaba;
 Fri, 10 Sep 2021 21:42:02 +0000 (UTC)
Received: by mail-ed1-x529.google.com with SMTP id i6so4465210edu.1
 for <xen-devel@lists.xenproject.org>; Fri, 10 Sep 2021 14:42:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ad851b2-c48d-47c6-b7e3-1883313dcaba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=uxfgzMlBFAsHdhhTIbAJ202DoS4PD95dqeSuumClICc=;
        b=CDlVhSmKEkEpemhEEIUguzMtHEWu7Hew4ok5miKSF51mlWb545f9B2GCwSXDee/fHs
         Xa8idfAqsIAceYz/GnAekbIeS0G1rNycwqEFbYeDXhncp9sTKuH0SzW/HFG92Ruxs3QL
         4auIXFJcTIiE6wTd96MlzgdUFlZy7/Vn1pKoxN49RsDllefS8cK+7XuJcvyXRu97sxbh
         i764o9joy/hhbSfl80wLD01zeUBpyHIFTVSazl4QZb66QA0QDQJZXsQ4Ocd9dqsaxJyu
         /Gh/C2eOf1X1MXDjWGBGdH/J/HBGRXVxwJlPgHyt9BOCQNNlx6NhR++iie56v/In95HX
         fSEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=uxfgzMlBFAsHdhhTIbAJ202DoS4PD95dqeSuumClICc=;
        b=XO4+KtZQ2mUmZdJzm/Z405US40dBpSz/veHpeFbEIcBLcNZi7102L7mjEHS5WlAtW0
         nmO2bVEy9ZcC9w/1VWLUEgM3Z+ZojP6PBaN84tb9LsS3pQP+BpVxBMelxRRy43sgH7W0
         ozoevnnxaE/n0MRbrpa50dRwQc8NqEOAk2dzilnRqELtRiYNepBYtns5skcslKLJM3xk
         +YycA3wVF8FE6ISAppEWx2FrnP7cAvRhO3i0ilk6Z/gTkK6UsthUwoYF7/qHnzq4ciZC
         j6RHTlgn2vl69EZri4i/4MvTbMf22ZqOH5uZmHeOmNEHTZtg4nmlvshA014/QV44xdwW
         xaEA==
X-Gm-Message-State: AOAM531QtzJPiMEdz0ZWZvUFxcHruN5HlVvuLeMnQgUN6YsrnWF/j4vl
	mqG7URgqcIzTm1ahZM2Ue8kPbuJwghzppOGCvZo=
X-Google-Smtp-Source: ABdhPJyCdqe8GSTIHTaW1fwa31u0Mcm70AZKWMewBDoxVL/QCqLfLagQP6Q0tnGo5YEHm6msc3A7oivOfl/YHGTtYtE=
X-Received: by 2002:aa7:df92:: with SMTP id b18mr4863250edy.47.1631310121526;
 Fri, 10 Sep 2021 14:42:01 -0700 (PDT)
MIME-Version: 1.0
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org> <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org> <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Fri, 10 Sep 2021 22:41:50 +0100
Message-ID: <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's p2m
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>, 
	Oleksandr Andrushchenko <andr2000@gmail.com>, xen-devel <xen-devel@lists.xenproject.org>, 
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Rahul Singh <rahul.singh@arm.com>
Content-Type: multipart/alternative; boundary="0000000000004fe6de05cbaafc09"

--0000000000004fe6de05cbaafc09
Content-Type: text/plain; charset="UTF-8"

On Fri, 10 Sep 2021, 21:30 Stefano Stabellini, <sstabellini@kernel.org>
wrote:

> On Fri, 10 Sep 2021, Julien Grall wrote:
> > On 10/09/2021 15:01, Oleksandr Andrushchenko wrote:
> > > On 10.09.21 16:18, Julien Grall wrote:
> > > > On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:
> > > > > Hi, Julien!
> > > >
> > > > Hi Oleksandr,
> > > >
> > > > > On 09.09.21 20:58, Julien Grall wrote:
> > > > > > On 03/09/2021 09:33, Oleksandr Andrushchenko wrote:
> > > > > > > From: Oleksandr Andrushchenko <
> oleksandr_andrushchenko@epam.com>
> > > > > > >
> > > > > > > Host bridge controller's ECAM space is mapped into Domain-0's
> p2m,
> > > > > > > thus it is not possible to trap the same for vPCI via MMIO
> handlers.
> > > > > > > For this to work we need to not map those while constructing
> the
> > > > > > > domain.
> > > > > > >
> > > > > > > Note, that during Domain-0 creation there is no pci_dev yet
> > > > > > > allocated for
> > > > > > > host bridges, thus we cannot match PCI host and its associated
> > > > > > > bridge by SBDF. Use dt_device_node field for checks instead.
> > > > > > >
> > > > > > > Signed-off-by: Oleksandr Andrushchenko
> > > > > > > <oleksandr_andrushchenko@epam.com>
> > > > > > > ---
> > > > > > >     xen/arch/arm/domain_build.c        |  3 +++
> > > > > > >     xen/arch/arm/pci/ecam.c            | 17 +++++++++++++++++
> > > > > > >     xen/arch/arm/pci/pci-host-common.c | 22
> ++++++++++++++++++++++
> > > > > > >     xen/include/asm-arm/pci.h          | 12 ++++++++++++
> > > > > > >     4 files changed, 54 insertions(+)
> > > > > > >
> > > > > > > diff --git a/xen/arch/arm/domain_build.c
> > > > > > > b/xen/arch/arm/domain_build.c
> > > > > > > index da427f399711..76f5b513280c 100644
> > > > > > > --- a/xen/arch/arm/domain_build.c
> > > > > > > +++ b/xen/arch/arm/domain_build.c
> > > > > > > @@ -1257,6 +1257,9 @@ static int __init
> map_range_to_domain(const
> > > > > > > struct dt_device_node *dev,
> > > > > > >             }
> > > > > > >         }
> > > > > > >     +    if ( need_mapping && (device_get_class(dev) ==
> DEVICE_PCI)
> > > > > > > ) > +        need_mapping =
> pci_host_bridge_need_p2m_mapping(d, dev,
> > > > > > addr, len);
> > > > > >
> > > > > > AFAICT, with device_get_class(dev), you know whether the
> hostbridge is
> > > > > > used by Xen. Therefore, I would expect that we don't want to map
> all
> > > > > > the regions of the hostbridges in dom0 (including the BARs).
> > > > > >
> > > > > > Can you clarify it?
> > > > > We only want to trap ECAM, not MMIOs and any other memory regions
> as the
> > > > > bridge is
> > > > >
> > > > > initialized and used by Domain-0 completely.
> > > >
> > > > What do you mean by "used by Domain-0 completely"? The hostbridge is
> owned
> > > > by Xen so I don't think we can let dom0 access any MMIO regions by
> > > > default.
> > >
> > > Now it's my time to ask why do you think Xen owns the bridge?
> > >
> > > All the bridges are passed to Domain-0, are fully visible to Domain-0,
> > > initialized in Domain-0.
> > >
> > > Enumeration etc. is done in Domain-0. So how comes that Xen owns the
> bridge?
> > > In what way it does?
> > >
> > > Xen just accesses the ECAM when it needs it, but that's it. Xen traps
> ECAM
> > > access - that is true.
> > >
> > > But it in no way uses MMIOs etc. of the bridge - those under direct
> control
> > > of Domain-0
> >
> > So I looked on the snipped of the design document you posted. I think
> you are
> > instead referring to a different part:
> >
> > " PCI-PCIe enumeration is a process of detecting devices connected to its
> > host.
> > It is the responsibility of the hardware domain or boot firmware to do
> the PCI
> > enumeration and configure the BAR, PCI capabilities, and MSI/MSI-X
> > configuration."
> >
> > But I still don't see why it means we have to map the MMIOs right now.
> Dom0
> > should not need to access the MMIOs (aside the hostbridge registers)
> until the
> > BARs are configured.
>
> This is true especially when we are going to assign a specific PCIe
> device to a DomU. In that case, the related MMIO regions are going to be
> mapped to the DomU and it would be a bad idea to also keep them mapped
> in Dom0. Once we do PCIe assignment, the MMIO region of the PCIe device
> being assigned should only be mapped in the DomU, right?
>

That's actually a good point. This is a recipe for disaster because dom0
and the domU may map the BARs with conflicting caching attributes.

So we ought to unmap the BAR from dom0. When the device is assigned to the
domU


> If so, it would be better if the MMIO region in question was never
> mapped into Dom0 at all rather having to unmap it.
>
> With the current approach, given that the entire PCIe aperture is mapped
> to Dom0 since boot, we would have to identify the relevant subset region
> and unmap it from Dom0 when doing assignment.

--0000000000004fe6de05cbaafc09
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Fri, 10 Sep 2021, 21:30 Stefano Stabellini, &lt;<a =
href=3D"mailto:sstabellini@kernel.org">sstabellini@kernel.org</a>&gt; wrote=
:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor=
der-left:1px #ccc solid;padding-left:1ex">On Fri, 10 Sep 2021, Julien Grall=
 wrote:<br>
&gt; On 10/09/2021 15:01, Oleksandr Andrushchenko wrote:<br>
&gt; &gt; On 10.09.21 16:18, Julien Grall wrote:<br>
&gt; &gt; &gt; On 10/09/2021 13:37, Oleksandr Andrushchenko wrote:<br>
&gt; &gt; &gt; &gt; Hi, Julien!<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; Hi Oleksandr,<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; On 09.09.21 20:58, Julien Grall wrote:<br>
&gt; &gt; &gt; &gt; &gt; On 03/09/2021 09:33, Oleksandr Andrushchenko wrote=
:<br>
&gt; &gt; &gt; &gt; &gt; &gt; From: Oleksandr Andrushchenko &lt;<a href=3D"=
mailto:oleksandr_andrushchenko@epam.com" target=3D"_blank" rel=3D"noreferre=
r">oleksandr_andrushchenko@epam.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; &gt; Host bridge controller&#39;s ECAM space is ma=
pped into Domain-0&#39;s p2m,<br>
&gt; &gt; &gt; &gt; &gt; &gt; thus it is not possible to trap the same for =
vPCI via MMIO handlers.<br>
&gt; &gt; &gt; &gt; &gt; &gt; For this to work we need to not map those whi=
le constructing the<br>
&gt; &gt; &gt; &gt; &gt; &gt; domain.<br>
&gt; &gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; &gt; Note, that during Domain-0 creation there is =
no pci_dev yet<br>
&gt; &gt; &gt; &gt; &gt; &gt; allocated for<br>
&gt; &gt; &gt; &gt; &gt; &gt; host bridges, thus we cannot match PCI host a=
nd its associated<br>
&gt; &gt; &gt; &gt; &gt; &gt; bridge by SBDF. Use dt_device_node field for =
checks instead.<br>
&gt; &gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; &gt; Signed-off-by: Oleksandr Andrushchenko<br>
&gt; &gt; &gt; &gt; &gt; &gt; &lt;<a href=3D"mailto:oleksandr_andrushchenko=
@epam.com" target=3D"_blank" rel=3D"noreferrer">oleksandr_andrushchenko@epa=
m.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; ---<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 xen/arch/arm/domain_build.=
c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 3 +++<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 xen/arch/arm/pci/ecam.c=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 17 ++++++=
+++++++++++<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 xen/arch/arm/pci/pci-host-=
common.c | 22 ++++++++++++++++++++++<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 xen/include/asm-arm/pci.h=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 12 ++++++++++++<br=
>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 4 files changed, 54 insert=
ions(+)<br>
&gt; &gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; &gt; diff --git a/xen/arch/arm/domain_build.c<br>
&gt; &gt; &gt; &gt; &gt; &gt; b/xen/arch/arm/domain_build.c<br>
&gt; &gt; &gt; &gt; &gt; &gt; index da427f399711..76f5b513280c 100644<br>
&gt; &gt; &gt; &gt; &gt; &gt; --- a/xen/arch/arm/domain_build.c<br>
&gt; &gt; &gt; &gt; &gt; &gt; +++ b/xen/arch/arm/domain_build.c<br>
&gt; &gt; &gt; &gt; &gt; &gt; @@ -1257,6 +1257,9 @@ static int __init map_r=
ange_to_domain(const<br>
&gt; &gt; &gt; &gt; &gt; &gt; struct dt_device_node *dev,<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 }<br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }<=
br>
&gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 =C2=A0=C2=A0 +=C2=A0=C2=A0=C2=A0 if ( n=
eed_mapping &amp;&amp; (device_get_class(dev) =3D=3D DEVICE_PCI)<br>
&gt; &gt; &gt; &gt; &gt; &gt; ) &gt; +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 need_mapping =3D pci_host_bridge_need_p2m_mapping(d, dev,<br>
&gt; &gt; &gt; &gt; &gt; addr, len);<br>
&gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; AFAICT, with device_get_class(dev), you know wheth=
er the hostbridge is<br>
&gt; &gt; &gt; &gt; &gt; used by Xen. Therefore, I would expect that we don=
&#39;t want to map all<br>
&gt; &gt; &gt; &gt; &gt; the regions of the hostbridges in dom0 (including =
the BARs).<br>
&gt; &gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; &gt; Can you clarify it?<br>
&gt; &gt; &gt; &gt; We only want to trap ECAM, not MMIOs and any other memo=
ry regions as the<br>
&gt; &gt; &gt; &gt; bridge is<br>
&gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; initialized and used by Domain-0 completely.<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; What do you mean by &quot;used by Domain-0 completely&quot;?=
 The hostbridge is owned<br>
&gt; &gt; &gt; by Xen so I don&#39;t think we can let dom0 access any MMIO =
regions by<br>
&gt; &gt; &gt; default.<br>
&gt; &gt; <br>
&gt; &gt; Now it&#39;s my time to ask why do you think Xen owns the bridge?=
<br>
&gt; &gt; <br>
&gt; &gt; All the bridges are passed to Domain-0, are fully visible to Doma=
in-0,<br>
&gt; &gt; initialized in Domain-0.<br>
&gt; &gt; <br>
&gt; &gt; Enumeration etc. is done in Domain-0. So how comes that Xen owns =
the bridge?<br>
&gt; &gt; In what way it does?<br>
&gt; &gt; <br>
&gt; &gt; Xen just accesses the ECAM when it needs it, but that&#39;s it. X=
en traps ECAM<br>
&gt; &gt; access - that is true.<br>
&gt; &gt; <br>
&gt; &gt; But it in no way uses MMIOs etc. of the bridge - those under dire=
ct control<br>
&gt; &gt; of Domain-0<br>
&gt; <br>
&gt; So I looked on the snipped of the design document you posted. I think =
you are<br>
&gt; instead referring to a different part:<br>
&gt; <br>
&gt; &quot; PCI-PCIe enumeration is a process of detecting devices connecte=
d to its<br>
&gt; host.<br>
&gt; It is the responsibility of the hardware domain or boot firmware to do=
 the PCI<br>
&gt; enumeration and configure the BAR, PCI capabilities, and MSI/MSI-X<br>
&gt; configuration.&quot;<br>
&gt; <br>
&gt; But I still don&#39;t see why it means we have to map the MMIOs right =
now. Dom0<br>
&gt; should not need to access the MMIOs (aside the hostbridge registers) u=
ntil the<br>
&gt; BARs are configured.<br>
<br>
This is true especially when we are going to assign a specific PCIe<br>
device to a DomU. In that case, the related MMIO regions are going to be<br=
>
mapped to the DomU and it would be a bad idea to also keep them mapped<br>
in Dom0. Once we do PCIe assignment, the MMIO region of the PCIe device<br>
being assigned should only be mapped in the DomU, right?<br></blockquote></=
div></div><div dir=3D"auto"><br></div><div dir=3D"auto">That&#39;s actually=
 a good point. This is a recipe for disaster because dom0 and the domU may =
map the BARs with conflicting caching attributes.</div><div dir=3D"auto"><b=
r></div><div dir=3D"auto">So we ought to unmap the BAR from dom0. When the =
device is assigned to the domU=C2=A0</div><div dir=3D"auto"><br></div><div =
dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
If so, it would be better if the MMIO region in question was never<br>
mapped into Dom0 at all rather having to unmap it.<br>
<br>
With the current approach, given that the entire PCIe aperture is mapped<br=
>
to Dom0 since boot, we would have to identify the relevant subset region<br=
>
and unmap it from Dom0 when doing assignment.</blockquote></div></div></div=
>

--0000000000004fe6de05cbaafc09--


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 22:55:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 22:55:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184816.333573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpQL-0000pi-BY; Fri, 10 Sep 2021 22:55:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184816.333573; Fri, 10 Sep 2021 22:55:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpQL-0000pb-6m; Fri, 10 Sep 2021 22:55:09 +0000
Received: by outflank-mailman (input) for mailman id 184816;
 Fri, 10 Sep 2021 22:55:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpQK-0000pV-NO
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 22:55:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 24c38af8-128a-11ec-b291-12813bfff9fa;
 Fri, 10 Sep 2021 22:55:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BEE1C611AD;
 Fri, 10 Sep 2021 22:55:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24c38af8-128a-11ec-b291-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631314507;
	bh=VJ5ipnbbr7/hh4BSPRAZdwAh7k+W87MfA9moLBnzp0s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WkQpk6kZbm/ZHT+FsrAzCBeXuYhUISywAxkP9plMEH2n2yyeze9UKk7QZTjhgDLXN
	 51ugguyIEa/+V8LllhC/PNyCzFILxyuwld5PTQB2gF4M9FfA5EJt7JAcq3fUkXc44z
	 RjLXp9WELfVVxbNKwIxQ+1Fn2mq3NPHvUxZC/cf6KQkGWTHozf5VM/7/PVvDpAxYv0
	 zQx4Vfyar5oZnFDNHBNymesr4CjJu1q1T9D1vPA995FUeYqCST0KdgOrc2bd+G/sP+
	 xlo2BLhCiOi7UGnuyGemVt7DQkI2HbKUIPvBywaJNaw14X2oT5UAyJCJ2CdShrDZ2Z
	 ZUniSmG/6PxxA==
Date: Fri, 10 Sep 2021 15:55:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 01/12] swiotlb-xen: avoid double free
In-Reply-To: <ce9c2adb-8a52-6293-982a-0d6ece943ac6@suse.com>
Message-ID: <alpine.DEB.2.21.2109101554370.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <ce9c2adb-8a52-6293-982a-0d6ece943ac6@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> Of the two paths leading to the "error" label in xen_swiotlb_init() one
> didn't allocate anything, while the other did already free what was
> allocated.
> 
> Fixes: b82776005369 ("xen/swiotlb: Use the swiotlb_late_init_with_tbl to init Xen-SWIOTLB late when PV PCI is used")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Cc: stable@vger.kernel.org

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -216,7 +216,6 @@ error:
>  		goto retry;
>  	}
>  	pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
> -	free_pages((unsigned long)start, order);
>  	return rc;
>  }
>  
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:07:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:07:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184823.333584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpbj-0002bj-Ed; Fri, 10 Sep 2021 23:06:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184823.333584; Fri, 10 Sep 2021 23:06:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpbj-0002bc-8u; Fri, 10 Sep 2021 23:06:55 +0000
Received: by outflank-mailman (input) for mailman id 184823;
 Fri, 10 Sep 2021 23:06:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5Qoh=OA=seco.com=sean.anderson@srs-us1.protection.inumbo.net>)
 id 1mOpbi-0002bW-Gq
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:06:54 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.59]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c8a55100-128b-11ec-b291-12813bfff9fa;
 Fri, 10 Sep 2021 23:06:52 +0000 (UTC)
Received: from DB7PR03MB4523.eurprd03.prod.outlook.com (2603:10a6:10:19::27)
 by DB8PR03MB5579.eurprd03.prod.outlook.com (2603:10a6:10:102::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep
 2021 23:06:50 +0000
Received: from DB7PR03MB4523.eurprd03.prod.outlook.com
 ([fe80::a9aa:f363:66e:fadf]) by DB7PR03MB4523.eurprd03.prod.outlook.com
 ([fe80::a9aa:f363:66e:fadf%6]) with mapi id 15.20.4500.016; Fri, 10 Sep 2021
 23:06:50 +0000
Received: from [172.27.1.65] (50.195.82.171) by
 BL1P221CA0011.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 23:06:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8a55100-128b-11ec-b291-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FHwOpcNvosBLdN0AYR9CYKEyAMVHK7hqufs5J/FuH6XYf9IlbKlbia3SAQTT2fO3y2mcXBBDkfvusq4/ZsVwt86JQlIZFs7EJt67syk3p78d9TDMlQUOGpvSZDykeqoQjfoLHtQvbCFtG3JmJ9CeextAwwy+peGnxvtfjuO+0HKZOABIrcTCpBFgV+bKPfaOpRueWNSKJW21tTMY469tN0Dx/8/mN1gUxQYn0q3nSggRS8oh8ugRLOhEGJ5gBnIPz3Gwg4gwgJEzyFGRH6jOm/IBN8ib3I9uQVfCOhijPRVXH7IFBvqI+l3z2zeqEMdI+0bcGWP+dFW18ZmA1+gEOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=USMOcNe+e7mKJi1OwIpgvdAMRk45vJxObEWyuY6zBIs=;
 b=gKxqEuqm3ddV0B5hw5xo+Sd3P2L3Ee9Ah38xZI0d+agdmAQVpdTlJxMvaprnad0r/b5HmkvXTkjrei6tF4wMw0dNkFALtMG4gtnA4wQvTMPJZBzatDUgkaX48mmngUJSbYwPytKx0Wv+9vv9O46Y6TgFkkLWUPlD2/P/1OY0HFjrsac93xKd7jHaoa1RC+n0isGgerdEqLZHWLUMi9h5iviRjAc9cHy3YCB9T/lcH443FU/PHjTBRqP2CDhuxyKqwL9qYJgU0UDrSUO6iHRq+y6SgxjQhXwlVRJ5SZBtQtHap9NJJealQ5YHnB7Rs9j13haK5WfQ5hySz9pFH8KTZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=seco.com; dmarc=pass action=none header.from=seco.com;
 dkim=pass header.d=seco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=secospa.onmicrosoft.com; s=selector2-secospa-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=USMOcNe+e7mKJi1OwIpgvdAMRk45vJxObEWyuY6zBIs=;
 b=tYrkJyQL817ICVQq2jKpX8BQdb6X+tsDyzrjCTINvwlPiTbS9jk0tRf6p1T++SI8Hls5Snw80M9lLK6hs/L1v61SVQluUCL5zT7TcnlSGvL2XX0EuCzei7SfOANx2mk4c/19mBgLwwB/0qYD7jWA0dnJ6LASR0PLDJhCTeKjY8Q=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=seco.com;
From: Sean Anderson <sean.anderson@seco.com>
Subject: Re: [PATCH v2 4/5] mm/page_alloc: place pages to tail in
 __free_pages_core()
To: David Hildenbrand <david@redhat.com>
Cc: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com,
 dave.hansen@intel.com, haiyangz@microsoft.com, kys@microsoft.com,
 linux-acpi@vger.kernel.org, linux-hyperv@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mm@kvack.org,
 mgorman@techsingularity.net, mhocko@kernel.org, mhocko@suse.com,
 osalvador@suse.de, pankaj.gupta.linux@gmail.com,
 richard.weiyang@linux.alibaba.com, rppt@kernel.org, sthemmin@microsoft.com,
 vbabka@suse.cz, wei.liu@kernel.org, willy@infradead.org,
 xen-devel@lists.xenproject.org
References: <20201005121534.15649-5-david@redhat.com>
 <a52dacbe-5649-7245-866f-ceaba44975b5@seco.com>
 <528e8d9c-b148-30ec-d8cc-3dd072eaa7f2@redhat.com>
Message-ID: <c9fc3cc7-2c6e-dc6f-b5f6-0b48fb49aedb@seco.com>
Date: Fri, 10 Sep 2021 19:06:45 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
In-Reply-To: <528e8d9c-b148-30ec-d8cc-3dd072eaa7f2@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: BL1P221CA0011.NAMP221.PROD.OUTLOOK.COM
 (2603:10b6:208:2c5::23) To DB7PR03MB4523.eurprd03.prod.outlook.com
 (2603:10a6:10:19::27)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 38f72cd7-d7e4-41de-512c-08d974afabe4
X-MS-TrafficTypeDiagnostic: DB8PR03MB5579:
X-Microsoft-Antispam-PRVS:
	<DB8PR03MB5579D5B5CFAC5C3B7751606B96D69@DB8PR03MB5579.eurprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dgPMxATIUBWKkjtnSJcBLRqSTjR8V9XA20NFl0HYfunMNVK0Rg/33OR8bITBQPRl49glEWx4GHGwJt8e+NJk5xv7YsXE9zXQhw1AdHrMA0AX1MlP7tRamOnzVxwXqFsLAvIUfzJZIGWiTvodJ4Pqo1a54t79alOYBRtNGYD/bI43j8YJQHL5cr3wtrgVR4mInpLrTq4S9ROQhy7KZOKL9ANgghaOYXrotJnwMCv0HrBILYKQrlsxltVk7DPk6AV+QA0aL++fMune03ZleEMQJIEQJYgRpHckM1ltJgIr0cJ/fTpk35EvotfGbUv6Hsb7xJCj+gasfQmMcYb2FrT8zj06xDjKmE+PxAXlUT1aDvnsruVsbLzj4SYth9pSVeR2LU2/bPaPM9KOHVZHLR/hEXNteqJVEtiCJJ0lIr3iVgwuoAb/8dHJml9hHRFa0P+q9LrI4CX1KpG3mXm32fs6a38YrTLodZv8IP4MqbfEHl/gqu1bZ1N2qpEaxzZLlYo5U3kBzCnUflLcMcdih7kNIetZAPQPBAPTOL4JyAvi+LwsXJJxjPrG8SPgCmYzbUc0Ay1YPb92lRu+ac1xXNNL9h9lbJA4T2IZSy+DYWEs9qMwoWbfnaa12ISWCwwCqx0CQz/bwvUqV73GfqYxLPeGSER7q+klKvKHPaGmbULLu0Lq6MzbHjOFZUJesYDQEtYPh9IAQ66Rlj/wICDX8MKrvy/I0GGEA5Z5Jd81pR6KmyBWgY1l1nO7zj1Q/OphklWJY8itcJ1jy14hi2EX/IcRMw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB7PR03MB4523.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(39850400004)(366004)(396003)(346002)(136003)(376002)(16576012)(36756003)(83380400001)(6666004)(316002)(44832011)(5660300002)(86362001)(6486002)(38100700002)(38350700002)(186003)(7416002)(2906002)(2616005)(31686004)(66476007)(52116002)(66556008)(956004)(4326008)(6916009)(66946007)(8936002)(31696002)(26005)(53546011)(8676002)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dWRHMDRESVNSSytiT21hNFpFYXk0UE5jZmdtNlplNzdwVjJoOVZyVC8yMXZ5?=
 =?utf-8?B?UXpKeFhVckRTZFlzU2Zaa0lQUmdTbU0za093a2l3c1BKVnpVTDdVVVFPY3F6?=
 =?utf-8?B?dHJVeERUclcyRVI0cFZ2bmIwWitCN0RiendMaU85L3pJMHVVK0xPZjUvdkxq?=
 =?utf-8?B?TEV1UXEvUDg0djZUdFdDdGxiblN4ejNUMURVRkNHREpsMytqOVo3bkJpaFl2?=
 =?utf-8?B?S3NBcFRTZjV0eDNGMWhaK2MzMktUT1RsS3NBa1RYenJ6V3UrSEVZMlFXZXNI?=
 =?utf-8?B?UlZTWWlTVkVxenBoOTYzR3BHTzBLRnJaK3hqSGE3Q2ZjTThzTVMvb2kzanRP?=
 =?utf-8?B?VHE5WHV0UXpRRk9aRXFyR28ybHFvSTNEeFNZbG5vMFN5VytYSGRyaUd4MG5k?=
 =?utf-8?B?bFNPTHpxc0FlVVZHMDYyMGhDU3YrMU1XeTJzNFNSeUtjOHI0NWpHandNV1l4?=
 =?utf-8?B?RGpVVzh6L3ZsMzdsR0RDeGMwcmJuQ0tLYU9hc2t6UG5ZNmNGc1p2ZUx3LzQ0?=
 =?utf-8?B?MGNBN0hxRWx6RTdFMTVTWHpVSDlpR3c1RXYyR0lJRjg5R3hKSDQ1dDZqZ0Iv?=
 =?utf-8?B?Ums2NTluTXViWXMvMnU0MGNaZDJEU2JXM0FjK3AwZW5qelREcVIyV2xkUVlG?=
 =?utf-8?B?RDVWYnlKQ0x5OWFybDdITTdCalRodmJHREtnbzFibW5EWHJpQ1pKWTZSeEJp?=
 =?utf-8?B?L3Rtb0VEUHlOeC82VzFCSWJ6aC83MWoxVGxmMGFTSVJKZng3L1pnWTVRTEw5?=
 =?utf-8?B?bVNTSlZweEFxTUloRitaWkVsNXllaTRBSUZUbFFNaEZNb21kQ0k5SmM3NU5B?=
 =?utf-8?B?eFBWenBQLzh1TGt3cHBQdEdyTE5BTDI4a1lMZXo3VWpEWlExcWl1WnRDcExC?=
 =?utf-8?B?VWI3djcwMFVFY1ByS3VRNGFXZUUyNXRaL1dUNDRjTmNONmJlS2JlQUwwOElu?=
 =?utf-8?B?MTRObStuOHZ2dTNlcWp2OElab2Q4WUpiUFFSQUpRemVsYXVzT0lXZmJxelJF?=
 =?utf-8?B?SnFpZGs5em1LYUl0bGpGeDdqMFJvOGlwUlI4VnRBV1docjhVR25mMmhZeVNu?=
 =?utf-8?B?bHg2aFdlUmcyc0xEU0JvVmtqb3BmQjNzRTNPbFAxUDlYREh0NVhaZTRZQTNT?=
 =?utf-8?B?eXI0TnprWHhYdzlzRHJSMitMYm9LdXpiMXhZMnRwUk1DdUFnOE5FaWFKNDkz?=
 =?utf-8?B?Rm1yYyt5VzZKYzNZc3kyTHUwaFhhNEFIUWxvem0xemlYZUV2eHAreTR1dXVQ?=
 =?utf-8?B?cSt5WHRObG5vMmhBOGIxQ2RBRHRaczMwQlVNNk9TNFl5QmVDRlRsTGpLY1Y1?=
 =?utf-8?B?VHIyakR2S2d0RFA5WDZFNitWZEtlZ1VTOWJuOXRoQTB2T1NjWlRUcFFwRGs5?=
 =?utf-8?B?NkUyTjZ1eUpVWXE3OGRFVVkrWURhQ3FjSUtGTW4xS29xOENsdzhRdmFxZlhx?=
 =?utf-8?B?b2Ira0NqaXhCUENzQTFGaHpBamxhWVdsN3Q0cEtlcDhzS2UxNSt1a0hFeDJp?=
 =?utf-8?B?Z1BPb2R6aVl6QllwVDZQaUpKaUpXcUZ3eCtkWHBmeDFXUnk0RVhPczViRVZR?=
 =?utf-8?B?QURLN21ERm9rNGNUWnhDQTY4QkVqVXFUTWdpSmRra0syWmxpRWR4WTN6N0Mv?=
 =?utf-8?B?amtNdzQ2ZERHNGNYbnZ1Q2xzdzhDbTBmckQ4YmIzMFB1YWptVEE1U1lDcHY3?=
 =?utf-8?B?WmFBeENnSC9DNDZvUis5Rk1Tc2RRcUpSQ0o2SlQwbEpxU1RmSTF5SlM5REo4?=
 =?utf-8?Q?WwqSazMKMnjTRQ1Im2uSH8rhGtT1qj0FrPlDTTG?=
X-OriginatorOrg: seco.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 38f72cd7-d7e4-41de-512c-08d974afabe4
X-MS-Exchange-CrossTenant-AuthSource: DB7PR03MB4523.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 23:06:50.3430
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: bebe97c3-6438-442e-ade3-ff17aa50e733
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yAEtbieFv3pp+KYRe9L7bgGRa+/tF4lDZiYdXyXHPuNccYtL4Tz4ZAMytYTIkOp9Q2FX+0I6eEvCAoa8Ta/Bew==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR03MB5579



On 9/8/21 2:21 AM, David Hildenbrand wrote:
> On 08.09.21 00:40, Sean Anderson wrote:
>> Hi David,
>>
>> This patch breaks booting on my custom Xilinx ZynqMP board. Booting
>> fails just after/during GIC initialization:
>>
>> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
>> [    0.000000] Linux version 5.14.0 (sean@plantagenet) (aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #251 SMP Tue Sep 7 18:11:50 EDT 2021
>> [    0.000000] Machine model: xlnx,zynqmp
>> [    0.000000] earlycon: cdns0 at MMIO 0x00000000ff010000 (options '115200n8')
>> [    0.000000] printk: bootconsole [cdns0] enabled
>> [    0.000000] efi: UEFI not found.
>> [    0.000000] Zone ranges:
>> [    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
>> [    0.000000]   Normal   [mem 0x0000000100000000-0x000000087fffffff]
>> [    0.000000] Movable zone start for each node
>> [    0.000000] Early memory node ranges
>> [    0.000000]   node   0: [mem 0x0000000000000000-0x000000007fefffff]
>> [    0.000000]   node   0: [mem 0x0000000800000000-0x000000087fffffff]
>> [    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000087fffffff]
>> [    0.000000] On node 0, zone Normal: 256 pages in unavailable ranges
>> [    0.000000] cma: Reserved 1000 MiB at 0x0000000041400000
>> [    0.000000] psci: probing for conduit method from DT.
>> [    0.000000] psci: PSCIv1.1 detected in firmware.
>> [    0.000000] psci: Using standard PSCI v0.2 function IDs
>> [    0.000000] psci: MIGRATE_INFO_TYPE not supported.
>> [    0.000000] psci: SMC Calling Convention v1.1
>> [    0.000000] percpu: Embedded 19 pages/cpu s46752 r0 d31072 u77824
>> [    0.000000] Detected VIPT I-cache on CPU0
>> [    0.000000] CPU features: detected: ARM erratum 845719
>> [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1033987
>> [    0.000000] Kernel command line: earlycon clk_ignore_unused root=/dev/mmcblk0p2 rootwait rw cma=1000M
>> [    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
>> [    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
>> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
>> [    0.000000] software IO TLB: mapped [mem 0x000000003d400000-0x0000000041400000] (64MB)
>> [    0.000000] Memory: 3023384K/4193280K available (4288K kernel code, 514K rwdata, 1200K rodata, 896K init, 187K bss, 145896K reserved, 1024000K cma-reserved)
>> [    0.000000] rcu: Hierarchical RCU implementation.
>> [    0.000000] rcu:     RCU event tracing is enabled.
>> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
>> [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
>> [    0.000000] GIC: Adjusting CPU interface base to 0x00000000f902f000
>> [    0.000000] Root IRQ handler: gic_handle_irq
>> [    0.000000] GIC: Using split EOI/Deactivate mode
>>
>> and I bisected it to this patch. Applying the following patch (for 5.14)
>> fixes booting again:
>
> Hi Sean,
>
> unfortunately that patch most likely (with 99.9999% confidence) revealed another latent BUG in your setup.

I suspected as much; however even after inspecting this patch I was
unsure what I should investigate further.

>
> Some memory that shouldn't be handed to the buddy as free memory is getting now allocated earlier than later, resulting in that issue.
>
>
> I had all different kinds of reports, but they were mostly
>
> a) Firmware bugs that result in uncached memory getting exposed to the buddy, resulting in severe performance degradation such that the system will no longer boot. [3]
>
> I wrote kstream [1] to be run under the old kernel, to identify these.
>
> b) BUGs that result in unsuitable memory getting exposed to either the buddy or devices, resulting in errors during device initialization. [6]
>
> c) Use after free BUGs.
>
> Exposing memory, such as used for ACPI tables, to the buddy as free memory although it's still in use. [4]
>
> d) Hypervisor BUGs
>
> The last report (heavy performance degradation) was due to a BUG in dpdk. [2]
>
>
> What the exact symptoms you're experiencing? Really slow boot/stall? Then it could be a) and kstream might help.

Well, as it turns out DDR chips of half the correct size were installed.
This caused the upper half of memory to alias to the lower half. As it
happened, due to some lucky circumstances this didn't initially cause
problems. Sorry for the noise.

--Sean


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:10:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:10:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184831.333595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpfX-00043j-3T; Fri, 10 Sep 2021 23:10:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184831.333595; Fri, 10 Sep 2021 23:10:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpfW-00043c-VY; Fri, 10 Sep 2021 23:10:50 +0000
Received: by outflank-mailman (input) for mailman id 184831;
 Fri, 10 Sep 2021 23:10:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpfV-00043W-FY
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:10:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 54f67aa9-128c-11ec-b291-12813bfff9fa;
 Fri, 10 Sep 2021 23:10:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id ABDA0611F0;
 Fri, 10 Sep 2021 23:10:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54f67aa9-128c-11ec-b291-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631315447;
	bh=v1M29iLRAqXSSDeBk/51fpGq9pHOSqcyUCgaAF8VAks=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=CBNorB/E8eA4w72YHFfKWHdfNeYfLRgu481B0NG49YWlcZNiRl21G0kW8qFKFcUyl
	 9+1kpAGY7w8oqTJXqHar70mSo6y7vvVMwf7rC2Mk1N09IZ1bVCCRvRD/kUC346aqAa
	 ezIHSfK8Z+DPikcCY9NF05wUXu7vYkK5kRCziaAISfQ0iSlzJtuRXpQUUfM0Z1amIt
	 mQqUgGgDJu42/YOhnHZECQ49308v1jTikLOlIli5MrOL4oP17VizTYxYK07OKeKiHl
	 sGzGSELnh5pTLmCHqp3KosfbvXmqJGn8XTWiQPV354xR4Tj/kC3O0e7N/hHRruUe0I
	 PYgQ3ysxxAHMQ==
Date: Fri, 10 Sep 2021 16:10:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 03/12] swiotlb-xen: maintain slab count properly
In-Reply-To: <dc054cb0-bec4-4db0-fc06-c9fc957b6e66@suse.com>
Message-ID: <alpine.DEB.2.21.2109101610380.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <dc054cb0-bec4-4db0-fc06-c9fc957b6e66@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> Generic swiotlb code makes sure to keep the slab count a multiple of the
> number of slabs per segment. Yet even without checking whether any such
> assumption is made elsewhere, it is easy to see that xen_swiotlb_fixup()
> might alter unrelated memory when calling xen_create_contiguous_region()
> for the last segment, when that's not a full one - the function acts on
> full order-N regions, not individual pages.
> 
> Align the slab count suitably when halving it for a retry. Add a build
> time check and a runtime one. Replace the no longer useful local
> variable "slabs" by an "order" one calculated just once, outside of the
> loop. Re-use "order" for calculating "dma_bits", and change the type of
> the latter as well as the one of "i" while touching this anyway.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -106,27 +106,26 @@ static int is_xen_swiotlb_buffer(struct
>  
>  static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
>  {
> -	int i, rc;
> -	int dma_bits;
> +	int rc;
> +	unsigned int order = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT);
> +	unsigned int i, dma_bits = order + PAGE_SHIFT;
>  	dma_addr_t dma_handle;
>  	phys_addr_t p = virt_to_phys(buf);
>  
> -	dma_bits = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT) + PAGE_SHIFT;
> +	BUILD_BUG_ON(IO_TLB_SEGSIZE & (IO_TLB_SEGSIZE - 1));
> +	BUG_ON(nslabs % IO_TLB_SEGSIZE);
>  
>  	i = 0;
>  	do {
> -		int slabs = min(nslabs - i, (unsigned long)IO_TLB_SEGSIZE);
> -
>  		do {
>  			rc = xen_create_contiguous_region(
> -				p + (i << IO_TLB_SHIFT),
> -				get_order(slabs << IO_TLB_SHIFT),
> +				p + (i << IO_TLB_SHIFT), order,
>  				dma_bits, &dma_handle);
>  		} while (rc && dma_bits++ < MAX_DMA_BITS);
>  		if (rc)
>  			return rc;
>  
> -		i += slabs;
> +		i += IO_TLB_SEGSIZE;
>  	} while (i < nslabs);
>  	return 0;
>  }
> @@ -210,7 +209,7 @@ retry:
>  error:
>  	if (repeat--) {
>  		/* Min is 2MB */
> -		nslabs = max(1024UL, (nslabs >> 1));
> +		nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
>  		bytes = nslabs << IO_TLB_SHIFT;
>  		pr_info("Lowering to %luMB\n", bytes >> 20);
>  		goto retry;
> @@ -245,7 +244,7 @@ retry:
>  		memblock_free(__pa(start), PAGE_ALIGN(bytes));
>  		if (repeat--) {
>  			/* Min is 2MB */
> -			nslabs = max(1024UL, (nslabs >> 1));
> +			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
>  			bytes = nslabs << IO_TLB_SHIFT;
>  			pr_info("Lowering to %luMB\n", bytes >> 20);
>  			goto retry;
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:15:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:15:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184837.333606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpjb-00050y-Jl; Fri, 10 Sep 2021 23:15:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184837.333606; Fri, 10 Sep 2021 23:15:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpjb-00050r-Go; Fri, 10 Sep 2021 23:15:03 +0000
Received: by outflank-mailman (input) for mailman id 184837;
 Fri, 10 Sep 2021 23:15:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpjZ-00050l-T1
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:15:01 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ebf57008-128c-11ec-b291-12813bfff9fa;
 Fri, 10 Sep 2021 23:15:00 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F03D7611F0;
 Fri, 10 Sep 2021 23:14:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebf57008-128c-11ec-b291-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631315700;
	bh=vJNj/V5gr36eIctaJUoPC89yjzCT0LnaEG2pV27VBPs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OI1DCVnn0xilrmUuSWEetAwLMWahluaHTMFEFvR8u5WMW2j7EIqzKwJ1N7d0t7d9s
	 lYU8f6PgKS0MbktKeEwQcp2NeQbCMka68x6vdCQJAxBgj5AB7oop0aZViU2IW7L0mU
	 kjXLlPVNGTZUzphhrE7aW0f5RxAbNfdRlIU9s3TPG9EyPE/lphthLrGBoTcKxVf5qJ
	 3qSS6+CG2hoIvDciSe5YChDwDXz8taSyLT+YuVrM48k0MqfpcyRDr0oNmqYLnqP32/
	 bdsjTZMimIT1AGxLbKLfK+ZwPPntd/e60jOiTtXUOJs8McLFOwdZ5ePp1lk0G0l2XK
	 7YoiB+BS5ab5g==
Date: Fri, 10 Sep 2021 16:14:59 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
In-Reply-To: <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
Message-ID: <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> While the hypervisor hasn't been enforcing this, we would still better
> avoid issuing requests with GFNs not aligned to the requested order.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I wonder how useful it is to include the alignment in the panic()
> message.

Not very useful given that it is static. I don't mind either way but you
can go ahead and remove it if you prefer (and it would make the line
shorter.)


> I further wonder how useful it is to wrap "bytes" in
> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
> least was supposed to be, prior to "swiotlb-xen: maintain slab count
> properly").

This one I would keep, to make sure to print out the same amount passed
to memblock_alloc.


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -231,10 +231,10 @@ retry:
>  	/*
>  	 * Get IO TLB memory from any location.
>  	 */
> -	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
> +	start = memblock_alloc(PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
>  	if (!start)
> -		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
> -		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
> +		panic("%s: Failed to allocate %lu bytes align=%#x\n",
> +		      __func__, PAGE_ALIGN(bytes), IO_TLB_SEGSIZE << IO_TLB_SHIFT);
>  
>  	/*
>  	 * And replace that memory with pages under 4GB.
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:18:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:18:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184844.333616 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpmp-0005f9-2r; Fri, 10 Sep 2021 23:18:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184844.333616; Fri, 10 Sep 2021 23:18:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpmp-0005f2-02; Fri, 10 Sep 2021 23:18:23 +0000
Received: by outflank-mailman (input) for mailman id 184844;
 Fri, 10 Sep 2021 23:18:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpmo-0005ew-3v
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:18:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6378dffc-128d-11ec-b292-12813bfff9fa;
 Fri, 10 Sep 2021 23:18:21 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8C5D261167;
 Fri, 10 Sep 2021 23:18:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6378dffc-128d-11ec-b292-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631315900;
	bh=HNuFXlUzzdE3O5BLAoHNZ+FVASkIymP1ujqpRkY0Rrw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=vMjSAGX3eIpKKZ/2aboK5CtMRUyJ851TY0/yJLTuynzkG56DVaLrA0/4BQnNPuSAA
	 s9E3eJNRFsxeQ1g6+5CAU+W/vsRqEdaL5k+H1UFLsA09myTLjSUK5ilOvW/TKw1CaY
	 zmFJLiQWOD3Ox3UVyhthYpzR2hEEjvXGe8SsUbUqMqg5ZSon32sGzkG3ZHySvwsqtP
	 2SXkUNorUoCBNZVNblByQWetiBDvRdGwUTZl1egiCDnbrb1L2qJABEIA35Znsj0/ma
	 4unw9gFtcb8SI3Rk7IXEivt+urmWWJLFGSJgvbTE4vDk4yY5V8+waHBNk/EKlycYl3
	 3q4waikb+Y5Rg==
Date: Fri, 10 Sep 2021 16:18:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 05/12] swiotlb-xen: suppress certain init retries
In-Reply-To: <56477481-87da-4962-9661-5e1b277efde0@suse.com>
Message-ID: <alpine.DEB.2.21.2109101617290.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <56477481-87da-4962-9661-5e1b277efde0@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> Only on the 2nd of the paths leading to xen_swiotlb_init()'s "error"
> label it is useful to retry the allocation; the first one did already
> iterate through all possible order values.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> I'm not convinced of the (lack of) indentation of the label, but I made
> the new one matzch the existing one.
> 
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -184,7 +184,7 @@ retry:
>  		order--;
>  	}
>  	if (!start)
> -		goto error;
> +		goto exit;
>  	if (order != get_order(bytes)) {
>  		pr_warn("Warning: only able to allocate %ld MB for software IO TLB\n",
>  			(PAGE_SIZE << order) >> 20);
> @@ -214,6 +214,7 @@ error:
>  		pr_info("Lowering to %luMB\n", bytes >> 20);
>  		goto retry;
>  	}
> +exit:
>  	pr_err("%s (rc:%d)\n", xen_swiotlb_error(m_ret), rc);
>  	return rc;
>  }
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:23:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:23:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184850.333628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOprf-0007Kl-NK; Fri, 10 Sep 2021 23:23:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184850.333628; Fri, 10 Sep 2021 23:23:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOprf-0007Ke-Jw; Fri, 10 Sep 2021 23:23:23 +0000
Received: by outflank-mailman (input) for mailman id 184850;
 Fri, 10 Sep 2021 23:23:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpre-0007KY-On
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:23:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d208a4f4-f201-42f6-9efc-e5b3668f1202;
 Fri, 10 Sep 2021 23:23:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2949961206;
 Fri, 10 Sep 2021 23:23:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d208a4f4-f201-42f6-9efc-e5b3668f1202
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631316201;
	bh=N7LKktutD+QwQybp5f/rKrBsVZiJeC/R7umGYSMMqXk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TYKH7ZNKWMXywDAwVeL41AEHBwT+2Vzt7M/6twb7POXVBhQz+P2DkIFADYuFtOdGx
	 LF5gd7EKeDKaPExyAi4PbCfFtOpf4PZKeC0CAoYYc7ez0923ROqEZUr7zW85B4ktpa
	 lDlIki6s0Z4rv0McrrYS37v9U87OlK9g27aPge3XQfDC23vaKfSY+DESsMrbkfCJf6
	 VosSrN7sZEIiDmiae/9ba5j5NXfePWjNrK56ye2MER1IFu1M4Imlq6wZv8foOSJtF8
	 8QDQKcxjTlAh5c1MqN8Q6IUytTWPH7eBVXeIsK3kgdDK5RHZuOPPn4d6ft5vBuA0me
	 boG9DR4Cs9CZA==
Date: Fri, 10 Sep 2021 16:23:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 06/12] swiotlb-xen: limit init retries
In-Reply-To: <984fa426-2b7b-4b77-5ce8-766619575b7f@suse.com>
Message-ID: <alpine.DEB.2.21.2109101621430.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <984fa426-2b7b-4b77-5ce8-766619575b7f@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> Due to the use of max(1024, ...) there's no point retrying (and issuing
> bogus log messages) when the number of slabs is already no larger than
> this minimum value.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -207,7 +207,7 @@ retry:
>  	swiotlb_set_max_segment(PAGE_SIZE);
>  	return 0;
>  error:
> -	if (repeat--) {
> +	if (nslabs > 1024 && repeat--) {
>  		/* Min is 2MB */
>  		nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
>  		bytes = nslabs << IO_TLB_SHIFT;
> @@ -243,7 +243,7 @@ retry:
>  	rc = xen_swiotlb_fixup(start, nslabs);
>  	if (rc) {
>  		memblock_free(__pa(start), PAGE_ALIGN(bytes));
> -		if (repeat--) {
> +		if (nslabs > 1024 && repeat--) {
>  			/* Min is 2MB */
>  			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
>  			bytes = nslabs << IO_TLB_SHIFT;
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:24:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:24:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184857.333638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpt8-0007vf-25; Fri, 10 Sep 2021 23:24:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184857.333638; Fri, 10 Sep 2021 23:24:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpt7-0007vY-VO; Fri, 10 Sep 2021 23:24:53 +0000
Received: by outflank-mailman (input) for mailman id 184857;
 Fri, 10 Sep 2021 23:24:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpt6-0007vS-Am
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:24:52 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8a4bbb8e-2ce2-43b6-bc67-2d47e65c4c97;
 Fri, 10 Sep 2021 23:24:51 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id CFE1D61167;
 Fri, 10 Sep 2021 23:24:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a4bbb8e-2ce2-43b6-bc67-2d47e65c4c97
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631316291;
	bh=EH4b4wOqgmR5fMb5cwT0NEac+L5t0raAiWP84suO8JA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=smsy9tlu9CPSDn4fKjl46u3jjfq2R3njFi5e6jxWnldXvJHvGzQuMXnKBS5huhkpu
	 UH0VszBrYKYtR7yu7SoXbE03cDE9F2V8w96d535j2SuQBcWf+r5Jy3JubwD53wYzh7
	 YZyBepcZw2vZH+dRSiTXJDf7UKR4l3IpTt8R9MOwWQlAPVkfb+tSEe9+v+bceD06qx
	 M63ZxeL+/06x2IfWAJXY5tYhUtpERojIx+a5wncFAAkDjMFUZY5XrzJtbrtecUwPrH
	 0LbIow+aLPOKBEZ+gHjKyAbQGzjor6Pmm7diWflLvgc6aDn6EoCCtPAGtOzVMBoZ51
	 68yjfbrauu0/w==
Date: Fri, 10 Sep 2021 16:24:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 07/12] swiotlb-xen: drop leftover __ref
In-Reply-To: <7cd163e1-fe13-270b-384c-2708e8273d34@suse.com>
Message-ID: <alpine.DEB.2.21.2109101624431.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <7cd163e1-fe13-270b-384c-2708e8273d34@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> Commit a98f565462f0 ("xen-swiotlb: split xen_swiotlb_init") should not
> only have added __init to the split off function, but also should have
> dropped __ref from the one left.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -154,7 +154,7 @@ static const char *xen_swiotlb_error(enu
>  
>  #define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
>  
> -int __ref xen_swiotlb_init(void)
> +int xen_swiotlb_init(void)
>  {
>  	enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
>  	unsigned long bytes = swiotlb_size_or_default();
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:26:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:26:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184863.333650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpun-00006G-Dz; Fri, 10 Sep 2021 23:26:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184863.333650; Fri, 10 Sep 2021 23:26:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpun-000069-B0; Fri, 10 Sep 2021 23:26:37 +0000
Received: by outflank-mailman (input) for mailman id 184863;
 Fri, 10 Sep 2021 23:26:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpum-000063-1w
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:26:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 89efd266-128e-11ec-b293-12813bfff9fa;
 Fri, 10 Sep 2021 23:26:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 922D4611B0;
 Fri, 10 Sep 2021 23:26:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89efd266-128e-11ec-b293-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631316394;
	bh=nWCVWm37Cn4aTHD9fKKBK0iiTuNj5cGbnQ7UNy5fjiA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=p13Fcm19TN9LmTK4LlFXJYisdlx8j91SHYqgG2d+C217OScazn4CK+Goc+whmNQJa
	 ccx3b/fMa+GAFLShtrkw2wCd2vAY0tVJ+/AqBkqitqoL9L5oLqKutEGWAUNAK+R3Y/
	 sJqYpnAww5AVQqbFTNJObMcBwqD6gchOiTNTjN98IwZcwwgsltdB9KUR7ykVRaKEAN
	 o9ueydR3zF1hTRSg1zy6p3RmdqXl4lQpCX4HY/6cZ/OXW5AaL4X2m1jBP05g7PMx33
	 +7J7HWLwwv6rZO1z4o22ZhLOTgTt2k1dFajut6FdFj+OYvmqqJyJqr4D0oscTPpFLh
	 iOJfITNuWNWmw==
Date: Fri, 10 Sep 2021 16:26:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 08/12] swiotlb-xen: arrange to have buffer info logged
In-Reply-To: <2e3c8e68-36b2-4ae9-b829-bf7f75d39d47@suse.com>
Message-ID: <alpine.DEB.2.21.2109101626250.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <2e3c8e68-36b2-4ae9-b829-bf7f75d39d47@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> I consider it unhelpful that address and size of the buffer aren't put
> in the log file; it makes diagnosing issues needlessly harder. The
> majority of callers of swiotlb_init() also passes 1 for the "verbose"
> parameter. 
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -253,7 +253,7 @@ retry:
>  		panic("%s (rc:%d)", xen_swiotlb_error(XEN_SWIOTLB_EFIXUP), rc);
>  	}
>  
> -	if (swiotlb_init_with_tbl(start, nslabs, false))
> +	if (swiotlb_init_with_tbl(start, nslabs, true))
>  		panic("Cannot allocate SWIOTLB buffer");
>  	swiotlb_set_max_segment(PAGE_SIZE);
>  }
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:27:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:27:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184870.333661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpva-0000iE-OC; Fri, 10 Sep 2021 23:27:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184870.333661; Fri, 10 Sep 2021 23:27:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOpva-0000i7-LC; Fri, 10 Sep 2021 23:27:26 +0000
Received: by outflank-mailman (input) for mailman id 184870;
 Fri, 10 Sep 2021 23:27:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOpvZ-0000hz-Fc
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:27:25 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ce488288-3aa8-4297-82cb-7c0932692689;
 Fri, 10 Sep 2021 23:27:24 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 07A4E61211;
 Fri, 10 Sep 2021 23:27:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce488288-3aa8-4297-82cb-7c0932692689
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631316444;
	bh=MTpyGMzRwzcZckX1Knm3eAP7QqVukVmqh2sPwyO9Yu0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DyyDq8G4s5uAzUcahf9t9rQB8ahLVLLBpkqerShVx5NmalJXs5fAu1lbV6IZ+WhzX
	 XAcn58BFCMUD2sKtlYkoaWIFkckUj1/UG44RorEAw795xzQkBz7zwR6wSEi05NuOOs
	 N16VJ43Qr8cCd19Vqqc3v2WDMYtOFsUcEZ1Ea2rRi8kfMBUKc/QPPhBlKfvfLiWffH
	 2FtDXArYULt2YKZB6w9frMgxvgQzdr+CxoTPUThOoEMb5/xCBMOVbPt/RYXZb8oCc4
	 qhMZQ+fhiFMz1tqxpuv4aIaj00F7cRdYcTEnPOVI0O+73zBaN39eFLqp6qoZuep4Yw
	 XKCUYsZ6HSUEA==
Date: Fri, 10 Sep 2021 16:27:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 09/12] swiotlb-xen: drop DEFAULT_NSLABS
In-Reply-To: <15259326-209a-1d11-338c-5018dc38abe8@suse.com>
Message-ID: <alpine.DEB.2.21.2109101627170.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <15259326-209a-1d11-338c-5018dc38abe8@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> It was introduced by 4035b43da6da ("xen-swiotlb: remove xen_set_nslabs")
> and then not removed by 2d29960af0be ("swiotlb: dynamically allocate
> io_tlb_default_mem").
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -152,8 +152,6 @@ static const char *xen_swiotlb_error(enu
>  	return "";
>  }
>  
> -#define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
> -
>  int xen_swiotlb_init(void)
>  {
>  	enum xen_swiotlb_err m_ret = XEN_SWIOTLB_UNKNOWN;
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:34:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:34:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184877.333672 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOq20-0002Rl-Fk; Fri, 10 Sep 2021 23:34:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184877.333672; Fri, 10 Sep 2021 23:34:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOq20-0002Re-CV; Fri, 10 Sep 2021 23:34:04 +0000
Received: by outflank-mailman (input) for mailman id 184877;
 Fri, 10 Sep 2021 23:34:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOq1z-0002RY-0H
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:34:03 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a97889bd-7406-45c9-91dd-ea3ba2cf26b3;
 Fri, 10 Sep 2021 23:34:02 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6ED4261212;
 Fri, 10 Sep 2021 23:34:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a97889bd-7406-45c9-91dd-ea3ba2cf26b3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631316841;
	bh=AlGT+1ZkR5OARaW5Qf3AVAgmSVngDxhk8G1V2NgY8A0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=LSWH98E9Gp5vV79OPvBxPiatelKbDD5+9HxdN1YHHO4924YI1NgzILiFxLXotbXLx
	 86ptja5pEcL1FEipFRXC4IZDhGH6DztwwC712yTJl81CWT0fv1orEbu+8etn7MsPIu
	 D4Dp4eDhht2mKDBdHn0rnTwHksyO88yl7deMkKrnUM6iNpVU4a+xuR5KpSP+HY5tLV
	 C8Aw47BSI6aNoLfmxe4a6C/RzFVCgbEVhEesi7NVe47nmICW1Ax6PyHfMqbL9tJCR+
	 YsTPbrgdaRv+xCOGhKrnosM4p5ZGVqlv+CONxR11zR07FaWusdOhfaVhBmrWwvti7p
	 OsNjVv664L+nw==
Date: Fri, 10 Sep 2021 16:34:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Bjorn Helgaas <bhelgaas@google.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH 10/12] xen-pcifront: this module is PV-only
In-Reply-To: <bbfb4191-9e34-53da-f179-4549b10dcfb3@suse.com>
Message-ID: <alpine.DEB.2.21.2109101633530.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <bbfb4191-9e34-53da-f179-4549b10dcfb3@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 7 Sep 2021, Jan Beulich wrote:
> It's module init function does a xen_pv_domain() check first thing.
> Hence there's no point building it in non-PV configurations.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -110,7 +110,7 @@ config PCI_PF_STUB
>  
>  config XEN_PCIDEV_FRONTEND
>  	tristate "Xen PCI Frontend"
> -	depends on X86 && XEN
> +	depends on XEN_PV
>  	select PCI_XEN
>  	select XEN_XENBUS_FRONTEND
>  	default y
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 10 23:48:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 10 Sep 2021 23:48:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184885.333683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqGF-0004My-Tx; Fri, 10 Sep 2021 23:48:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184885.333683; Fri, 10 Sep 2021 23:48:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqGF-0004Mr-Qq; Fri, 10 Sep 2021 23:48:47 +0000
Received: by outflank-mailman (input) for mailman id 184885;
 Fri, 10 Sep 2021 23:48:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGUI=OA=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mOqGE-0004Ml-FP
 for xen-devel@lists.xenproject.org; Fri, 10 Sep 2021 23:48:46 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bfeadf06-7ce7-49ae-a051-d9b0c6445ca8;
 Fri, 10 Sep 2021 23:48:45 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9C6A561209;
 Fri, 10 Sep 2021 23:48:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfeadf06-7ce7-49ae-a051-d9b0c6445ca8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631317725;
	bh=CT+zN2hvsAGlq3FXd8oqicFmjUPdT7vc0738cJVqDhc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NBh9bj15FRDBR+BDeOLotaclfeKGuxxEoc4JgAJKKHSox00tTCMgZLOGby7uUyLYQ
	 pUtT8PJgxMce1q0BfXQGk+qIw445gAvZ6NLhM4y8jA3YmI46zR+zNzroZXVu/pZbaM
	 pTbn7Eto3WbEs5hwCEQxgUa7pk5OieVIazdXrixmWnVA0ZXVHOPpeLJa2vQT1Ei7aF
	 /ufGH9sxsebTeWF+phKh0lNfjV8oBhSvhX22Iap+CYZELu0dBswL/q/Q1LCFKtPJe6
	 t2CcUFqSt5aknM/ek0YF2QSjZKjYczxCuqCMGiGxDka+iZ6w5jVDpn2rd6xDDUxgLZ
	 AD+jGYwhQZKlw==
Date: Fri, 10 Sep 2021 16:48:44 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Christoph Hellwig <hch@infradead.org>
cc: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    the arch/x86 maintainers <x86@kernel.org>, 
    Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, 
    Borislav Petkov <bp@alien8.de>
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
In-Reply-To: <YThiyxG0d2tmCtb+@infradead.org>
Message-ID: <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com> <YThiyxG0d2tmCtb+@infradead.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 8 Sep 2021, Christoph Hellwig wrote:
> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
> > The code is unreachable for HVM or PVH, and it also makes little sense
> > in auto-translated environments. On Arm, with
> > xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
> > time seeing what good the Xen specific variant does - the generic one
> > ought to be fine for all purposes there. Still Arm code explicitly
> > references symbols here, so the code will continue to be included there.
> 
> Can the Xen/arm folks look into that?  Getting ARM out of using
> swiotlb-xen would be a huge step forward cleaning up some DMA APIs.

On ARM swiotlb-xen is used for a different purpose compared to x86.

Many ARM SoCs still don't have an IOMMU covering all DMA-mastering
devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest
physical == physical address).

Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need
swiotlb-xen. But when we start using PV drivers to share the network or
disk between Dom0 and DomU we are going to get DomU pages mapped in
Dom0, we call them "foreign pages".  They are not mapped 1:1. It can
happen that one of these foreign pages are used for DMA operations
(e.g. related to the NIC). swiotlb-xen is used to detect these
situations and translate the guest physical address to physical address
of foreign pages appropriately.

If an IOMMU is available and the DMA-mastering device is behind it, then
swiotlb-xen is not necessary. FYI there is community interest in
selectively disabling swiotlb-xen for devices that are behind an IOMMU.


> > Instead of making PCI_XEN's "select" conditional, simply drop it -
> > SWIOTLB_XEN will be available unconditionally in the PV case anyway, and
> > is - as explained above - dead code in non-PV environments.
> > 
> > This in turn allows dropping the stubs for
> > xen_{create,destroy}_contiguous_region(), the former of which was broken
> > anyway - it failed to set the DMA handle output.
> 
> Looks good:
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 00:06:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 00:06:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184892.333693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqWs-0007uv-1x; Sat, 11 Sep 2021 00:05:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184892.333693; Sat, 11 Sep 2021 00:05:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqWr-0007uo-VG; Sat, 11 Sep 2021 00:05:57 +0000
Received: by outflank-mailman (input) for mailman id 184892;
 Sat, 11 Sep 2021 00:05:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqWr-0007ue-5M; Sat, 11 Sep 2021 00:05:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqWq-0003HC-V1; Sat, 11 Sep 2021 00:05:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqWq-000360-E3; Sat, 11 Sep 2021 00:05:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqWq-0003HV-DZ; Sat, 11 Sep 2021 00:05:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Avu5ZY8stUgTM6nsPhnyiZ26MysLwdXfv86hgZ2iDX4=; b=J72osOirRyZD+hlo17I6OXcnps
	40Vs1PWAwMcOmfdBl+q/GLtWSsrbVivQgIKsLyFcNd+1Aik1/Mh8ZgfqTGdMU6au9oXPQXG/o8x0p
	XrvbyoGdn0kp6Siq+F29oCbzEZFBZ7NhlGfOcjIiIOUj0c+JyyDhVchgEL9Su/n490+A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164937-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164937: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=cf7c65059202e7858fbc00ce1f163ee243947e08
X-Osstest-Versions-That:
    ovmf=d248516b3a190c5cb5d51164b5721ead9d24469a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 00:05:56 +0000

flight 164937 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164937/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 cf7c65059202e7858fbc00ce1f163ee243947e08
baseline version:
 ovmf                 d248516b3a190c5cb5d51164b5721ead9d24469a

Last test of basis   164913  2021-09-09 13:11:28 Z    1 days
Testing same since   164937  2021-09-10 05:29:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ashraf Ali S <ashraf.ali.s@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   d248516b3a..cf7c650592  cf7c65059202e7858fbc00ce1f163ee243947e08 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 00:06:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 00:06:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184899.333707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqXO-0008R9-Bi; Sat, 11 Sep 2021 00:06:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184899.333707; Sat, 11 Sep 2021 00:06:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOqXO-0008R2-8b; Sat, 11 Sep 2021 00:06:30 +0000
Received: by outflank-mailman (input) for mailman id 184899;
 Sat, 11 Sep 2021 00:06:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqXM-0008Qo-UC; Sat, 11 Sep 2021 00:06:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqXM-0003Hq-Qt; Sat, 11 Sep 2021 00:06:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqXM-00037N-IV; Sat, 11 Sep 2021 00:06:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOqXM-00044c-Hy; Sat, 11 Sep 2021 00:06:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yT4CJQ/zXoCuqafNuM0yAxydOTu2gQ+/Q5DyGSJuFXI=; b=7E4L87rKBfMOUOv6cwj7T8gsPV
	/TkDInKqgQEVU+Wmq/k46PyaGL5spEl7s2M/rqDn6GOe47yFM/pLZQrUJQbAE8/eJ8yX5QKqquj+s
	2LoFAGDahSVnTrGyBgAgqa5LbwFALhixULZiW4qH+K/fr0T/vmiG4LdCFEFpsnxog+bY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164944-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164944: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 00:06:28 +0000

flight 164944 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164944/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164943  2021-09-10 13:00:28 Z    0 days
Testing same since   164944  2021-09-10 20:02:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c76cfada1c..6d45368a0a  6d45368a0a89e01a3a01d156af61fea565db96cc -> smoke


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 00:39:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 00:39:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184909.333722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOr3S-0004B2-4C; Sat, 11 Sep 2021 00:39:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184909.333722; Sat, 11 Sep 2021 00:39:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOr3S-0004Av-0u; Sat, 11 Sep 2021 00:39:38 +0000
Received: by outflank-mailman (input) for mailman id 184909;
 Sat, 11 Sep 2021 00:39:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOr3Q-0004Al-No; Sat, 11 Sep 2021 00:39:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOr3Q-0003no-Bp; Sat, 11 Sep 2021 00:39:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOr3P-0005TQ-Tl; Sat, 11 Sep 2021 00:39:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOr3P-00020S-T1; Sat, 11 Sep 2021 00:39:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=18R5DW6lAjxxE1+kRBaOw4+AT8vHxcYgRJwikgZq6AQ=; b=6AdcLbBm8gnaPdTi8IJoC3JRkS
	oYBE0uslvnscr2A5+Foc1cKImJSjlnBZZ5rGytBBCYUhFb206msYtD2n3ONswqzDqW7uUT7WmCO25
	YT9GWgZ9MAw+G65V8cdGJ5D7e5Jjx0JRtsMODHlOtZpsh1VEe0zJZC0D5f5AfKygXYCY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164930-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.12-testing test] 164930: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.12-testing:test-amd64-amd64-xl-qcow2:guest-saverestore.2:fail:heisenbug
    xen-4.12-testing:test-armhf-armhf-xl-rtds:xen-boot:fail:heisenbug
    xen-4.12-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qcow2:guest-localmigrate/x10:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.12-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=47193c78887734c2f95a50125684bf81419f1565
X-Osstest-Versions-That:
    xen=35ba323378d05509f2e0dc049520e140be183003
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 00:39:35 +0000

flight 164930 xen-4.12-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164930/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qcow2 18 guest-saverestore.2 fail in 164896 pass in 164930
 test-armhf-armhf-xl-rtds      8 xen-boot                   fail pass in 164896

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 164896 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 164896 never pass
 test-amd64-amd64-xl-qcow2    19 guest-localmigrate/x10       fail  like 164489
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164489
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164489
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164489
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164489
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164489
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164489
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164489
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164489
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass

version targeted for testing:
 xen                  47193c78887734c2f95a50125684bf81419f1565
baseline version:
 xen                  35ba323378d05509f2e0dc049520e140be183003

Last test of basis   164489  2021-08-25 17:57:16 Z   16 days
Testing same since   164886  2021-09-08 13:06:02 Z    2 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   35ba323378..47193c7888  47193c78887734c2f95a50125684bf81419f1565 -> stable-4.12


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 03:38:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 03:38:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184929.333740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOtq7-0007JV-D5; Sat, 11 Sep 2021 03:38:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184929.333740; Sat, 11 Sep 2021 03:38:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOtq7-0007J3-54; Sat, 11 Sep 2021 03:38:03 +0000
Received: by outflank-mailman (input) for mailman id 184929;
 Sat, 11 Sep 2021 03:38:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOtq6-0007It-0d; Sat, 11 Sep 2021 03:38:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOtq5-000575-Oz; Sat, 11 Sep 2021 03:38:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOtq5-00062y-BG; Sat, 11 Sep 2021 03:38:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOtq5-0002l4-AI; Sat, 11 Sep 2021 03:38:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fNjGMWVZpzOIQ19QEmSZSDQQWt9RUSs6KxNSWL5nCsk=; b=u6dFT5eEq/KGOM8t7NcAP3Me7j
	N+klLaGKWTyOEYvbzVFP52p8ShQMuKKb/oHYSQgzHWaNj0iAB8Incmwad7HIC2+aqw9Re+/wdZtk8
	0hrZMWJmdpJD1s7GGTCm6WkA22c//HeWWPc/kmmLlWkeMXaDcACQJzNjwrg3EI7FnGKs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164938-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.15-testing test] 164938: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.15-testing:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=84fa99099b920f7bcde8899e1a9b756078719d7d
X-Osstest-Versions-That:
    xen=6f92f38419d6bd052da9076a7d89534b1f85ded9
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 03:38:01 +0000

flight 164938 xen-4.15-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164938/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164889
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164889
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164889
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164889
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164889
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164889
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164889
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164889
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164889
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164889
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164889
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164889
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164889
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  84fa99099b920f7bcde8899e1a9b756078719d7d
baseline version:
 xen                  6f92f38419d6bd052da9076a7d89534b1f85ded9

Last test of basis   164889  2021-09-08 14:08:45 Z    2 days
Testing same since   164938  2021-09-10 07:07:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6f92f38419..84fa99099b  84fa99099b920f7bcde8899e1a9b756078719d7d -> stable-4.15


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 08:25:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 08:25:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.184972.333762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOyKA-0000PG-AY; Sat, 11 Sep 2021 08:25:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 184972.333762; Sat, 11 Sep 2021 08:25:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mOyKA-0000P9-7Q; Sat, 11 Sep 2021 08:25:22 +0000
Received: by outflank-mailman (input) for mailman id 184972;
 Sat, 11 Sep 2021 08:25:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOyK8-0000Oz-FU; Sat, 11 Sep 2021 08:25:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOyK8-0002WC-9I; Sat, 11 Sep 2021 08:25:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mOyK7-0004lp-UI; Sat, 11 Sep 2021 08:25:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mOyK7-0003OJ-Ts; Sat, 11 Sep 2021 08:25:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=H3gGcSevGnSJouFDmJxmNwo71P4COwSRfyhu/tM9VZU=; b=zPz8o9gWTZDUH27/yrBjcOfpGs
	LhOmmAe7eOHbr36W+mg5GNuOX6oZ1ynkdt6/CfJnKMnQaQFR9FKMQM335LM0ChVy3QaKeNsr8XcRi
	FYPp+0jQyvorzRpYoPqjvnSVK4r7k9v9vfhUu7vh+J/gQN0kN7PgspGETyTpXbKePWXI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164939-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.13-testing test] 164939: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.13-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6e2fc128eb1a7d8ff8c36123a0a03e4e60a4a44c
X-Osstest-Versions-That:
    xen=065fff7af08f7eaf300c9bef86ae3cec8150d3aa
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 08:25:19 +0000

flight 164939 xen-4.13-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164939/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164898
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164898
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164898
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164898
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164898
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164898
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164898
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164898
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164898
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164898
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164898
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6e2fc128eb1a7d8ff8c36123a0a03e4e60a4a44c
baseline version:
 xen                  065fff7af08f7eaf300c9bef86ae3cec8150d3aa

Last test of basis   164898  2021-09-09 05:02:45 Z    2 days
Testing same since   164939  2021-09-10 07:37:18 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   065fff7af0..6e2fc128eb  6e2fc128eb1a7d8ff8c36123a0a03e4e60a4a44c -> stable-4.13


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 10:33:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 10:33:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185007.333776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP0Jb-0007Rx-TT; Sat, 11 Sep 2021 10:32:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185007.333776; Sat, 11 Sep 2021 10:32:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP0Jb-0007Rq-P0; Sat, 11 Sep 2021 10:32:55 +0000
Received: by outflank-mailman (input) for mailman id 185007;
 Sat, 11 Sep 2021 10:32:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP0Ja-0007Rg-N9; Sat, 11 Sep 2021 10:32:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP0Ja-0004gY-GL; Sat, 11 Sep 2021 10:32:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP0Ja-0000se-3p; Sat, 11 Sep 2021 10:32:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mP0Ja-0005kB-3P; Sat, 11 Sep 2021 10:32:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=bZ22fPcMPvfOSJ6e1cwJcBkcPbGyq0vseXgZH77SESQ=; b=0wBzQNLkrDy8JprV8jPfmP9M0Z
	/jXgnyuhK7KrJf12cT4GcIs1DcB3X2LGXa46mevenTAfqDwJv5DyDCg1v3F/3/m9H1yhN3ZX+sMnN
	M1T+uSvk1H4rnjE4GD+2HSvCnCRcgOo2uVVSPrpNdwOzFQsO4NYKQSvk4pw6/G0HX00Y=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164940-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164940: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-i386-xl-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-intel:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-examine:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-debianhvm-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemuu-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-ws16-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-qemut-rhel6hvm-amd:xen-install:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-i386-xl-raw:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-pvshim:xen-install:fail:regression
    linux-linus:test-amd64-i386-freebsd10-i386:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-shadow:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-ovmf-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemuu-win7-amd64:xen-install:fail:regression
    linux-linus:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:xen-install:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/src_host:fail:regression
    linux-linus:test-amd64-i386-libvirt-pair:xen-install/dst_host:fail:regression
    linux-linus:test-amd64-coresched-i386-xl:xen-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=bf9f243f23e6623f310ba03fbb14e10ec3a61290
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 10:32:54 +0000

flight 164940 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164940/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install      fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine       6 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt       7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-xl            7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-install        fail REGR. vs. 152332
 test-amd64-i386-pair         10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-pair         11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-i386-xl-raw        7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install           fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-install            fail REGR. vs. 152332
 test-amd64-i386-xl-shadow     7 xen-install              fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install       fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host     fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host     fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install              fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                bf9f243f23e6623f310ba03fbb14e10ec3a61290
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  406 days
Failing since        152366  2020-08-01 20:49:34 Z  405 days  704 attempts
Testing same since   164940  2021-09-10 08:21:28 Z    1 days    1 attempts

------------------------------------------------------------
7281 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           fail    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     fail    
 test-amd64-i386-freebsd10-amd64                              fail    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         fail    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 fail    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       fail    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2246908 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 18:54:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 18:54:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185083.333790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP88U-0004le-Ve; Sat, 11 Sep 2021 18:53:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185083.333790; Sat, 11 Sep 2021 18:53:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP88U-0004lX-Rs; Sat, 11 Sep 2021 18:53:58 +0000
Received: by outflank-mailman (input) for mailman id 185083;
 Sat, 11 Sep 2021 18:53:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP88T-0004lN-Co; Sat, 11 Sep 2021 18:53:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP88T-0005E1-2Q; Sat, 11 Sep 2021 18:53:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP88S-00010E-Md; Sat, 11 Sep 2021 18:53:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mP88S-0002wR-MF; Sat, 11 Sep 2021 18:53:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/OXE9VJoM91E/ekSPnkAHpZjramMNTO9PZIk5I057Xg=; b=5Ts/f/nxrejtLptm3cWiCtx39D
	8Fsyn4nB34RfMjDJP8kpjbTpbUCnahwn/yiHdQunxFIeDt70oE25TmyfsRQxFnB5T2e7SSr4j64t+
	Tkygml6rfN8wZztsLuLqiO9Lhi7wMHH/PthBIwtiw+vFO9skF3lC/J/bO4ZSIGTYoODU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164941-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164941: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=500f1f3e81ad112e28b7c979136847e32dad83b6
X-Osstest-Versions-That:
    qemuu=bd662023e683850c085e98c8ff8297142c2dd9f2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 18:53:56 +0000

flight 164941 qemu-mainline real [real]
flight 164949 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164941/
http://logs.test-lab.xenproject.org/osstest/logs/164949/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164949-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164884
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164884
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164884
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164884
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164884
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164884
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164884
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164884
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                500f1f3e81ad112e28b7c979136847e32dad83b6
baseline version:
 qemuu                bd662023e683850c085e98c8ff8297142c2dd9f2

Last test of basis   164884  2021-09-08 12:38:05 Z    3 days
Testing same since   164941  2021-09-10 10:38:05 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   bd662023e6..500f1f3e81  500f1f3e81ad112e28b7c979136847e32dad83b6 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 20:17:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 20:17:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185091.333804 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP9RA-0005jN-RX; Sat, 11 Sep 2021 20:17:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185091.333804; Sat, 11 Sep 2021 20:17:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mP9RA-0005jG-OR; Sat, 11 Sep 2021 20:17:20 +0000
Received: by outflank-mailman (input) for mailman id 185091;
 Sat, 11 Sep 2021 20:17:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP9R9-0005j6-Uj; Sat, 11 Sep 2021 20:17:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP9R9-0006i1-KS; Sat, 11 Sep 2021 20:17:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mP9R9-00058F-BB; Sat, 11 Sep 2021 20:17:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mP9R9-0001UU-AO; Sat, 11 Sep 2021 20:17:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YUdLhY0gP3W8EBTOQg4cXKS4QSmG44MAJp5YtPR0/r8=; b=g7cTmTGDy9qp8q5oLnYbX2EAzl
	KjxcT0tfGySC23f5QK4elEwnX4Li10zaT1Fxx8k4y96+CbeZq4WoqkAB4K6ReLlRkP4GqYfaINeV9
	ka/BNM5Pt7BdeL23PahDUGZA4o575szYlpmDCvtm/01QqFPcg+rKf917n4hpTMQM4Les=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164942-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.14-testing test] 164942: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.14-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.14-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9f2b6c5ec2ded4c1caf149743e862c5f15d6d083
X-Osstest-Versions-That:
    xen=ef6455a3707cf1c7c61ad8af12558811eeee4ba8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 20:17:19 +0000

flight 164942 xen-4.14-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164942/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164888
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164888
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164888
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164888
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164888
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164888
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164888
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164888
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164888
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164888
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164888
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164888
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  9f2b6c5ec2ded4c1caf149743e862c5f15d6d083
baseline version:
 xen                  ef6455a3707cf1c7c61ad8af12558811eeee4ba8

Last test of basis   164888  2021-09-08 14:08:44 Z    3 days
Testing same since   164942  2021-09-10 12:37:25 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-xl-raw                                       pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ef6455a370..9f2b6c5ec2  9f2b6c5ec2ded4c1caf149743e862c5f15d6d083 -> stable-4.14


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 22:31:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 22:31:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185107.333822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPBX9-0004g3-Tl; Sat, 11 Sep 2021 22:31:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185107.333822; Sat, 11 Sep 2021 22:31:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPBX9-0004fw-Qp; Sat, 11 Sep 2021 22:31:39 +0000
Received: by outflank-mailman (input) for mailman id 185107;
 Sat, 11 Sep 2021 22:31:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQMP=OB=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mPBX8-0004fq-2J
 for xen-devel@lists.xenproject.org; Sat, 11 Sep 2021 22:31:38 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ccba321c-999d-4321-a992-831baeb6b42f;
 Sat, 11 Sep 2021 22:31:36 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id q26so8273756wrc.7
 for <xen-devel@lists.xenproject.org>; Sat, 11 Sep 2021 15:31:36 -0700 (PDT)
Received: from [192.168.1.36] (21.red-83-52-55.dynamicip.rima-tde.net.
 [83.52.55.21])
 by smtp.gmail.com with ESMTPSA id f1sm2689415wri.43.2021.09.11.15.31.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sat, 11 Sep 2021 15:31:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: ccba321c-999d-4321-a992-831baeb6b42f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=XT2R7gKOwJquO7Q7cTHp+dmuaUe/Pu9W6ODttpllYz4=;
        b=b2bJ2Px663Jyjjkb4rwi1aTGvhuL68k7L2E3bYKODogP2gfA7vnE94/Auh+BX5f7F7
         IxAnhUJZds4iYuJFkhOWmjPofJSvqsbY6WrkuCnbGGERgS7cEZpOTBt8aGtsMphs3VdL
         OX7vLhAYsbzT1EtpdVSRJ+DsVngeojmqXv8cuod8qKxMs0MSJEIaY+EzM+woIw6H4XEE
         lBGZfspthJ6BKFB8kK12vVadMcMIRo8lSQ57dn2P6T17nDCFuA+vhJwv2Mz6IUmkb0Tm
         QO8w2SRNdXbcd+NxpGfvtUBJsh+DJxXGgeo6OTlxBvI3gYTGhzuF6QLiOvSlqJK2+8mh
         HKEg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:subject:from:to:cc:references:message-id
         :date:user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=XT2R7gKOwJquO7Q7cTHp+dmuaUe/Pu9W6ODttpllYz4=;
        b=UVGXPI8mEPIo1zEDPl38XPGdB6oeTqoeGaCZODfNynvW3g8XSn+as2azN+ID5HTo4P
         H2tN4y00QqX9MEJtL2K/x2JIw2b9g9K3ZvnTd3qQLFV5WyNU7Hcq/PTWUHTrMVzeNsoh
         SoTEEMRN38omuiKN3zemEPAe5vInhvim3ir1Aq0s+k/P+FXcAockNxMmXz1w3XUZpdpi
         G4CnodyZAJ2LLDW1CnA/iU1GS7X3ywruyl5VxoDi4M4fRe0G97B0IRnnl9zaslZQETBj
         TR24wCy0KW9H2zhiHlHaRpPEBDq8NLSa/8GS8f/auhl3b9IOQtQNSz6j3lfzAiprjbNL
         j/8w==
X-Gm-Message-State: AOAM53326XZK3vRNCvL1xegR/N4pVtKkKz0VQal49uNDguWQa/929yGl
	49yCa4mFksHr0QQtvAYvbQk=
X-Google-Smtp-Source: ABdhPJwCU/eok/l7PQWSv0uFfyt3OLV/3TdzjV/d32F7wAyFIbNaXVYENwr0FBRMDcjW3s4ZBOaI7w==
X-Received: by 2002:adf:916f:: with SMTP id j102mr4796451wrj.422.1631399495458;
        Sat, 11 Sep 2021 15:31:35 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>
To: Richard Henderson <richard.henderson@linaro.org>,
 David Gibson <david@gibson.dropbear.id.au>, =?UTF-8?Q?C=c3=a9dric_Le_Goater?=
 <clg@kaod.org>, Greg Kurz <groug@kaod.org>
Cc: Peter Maydell <peter.maydell@linaro.org>, Chris Wulff
 <crwulff@gmail.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, qemu-devel@nongnu.org,
 Jiaxun Yang <jiaxun.yang@flygoat.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, haxm-team@intel.com,
 Colin Xu <colin.xu@intel.com>, Sunil Muthuswamy <sunilmut@microsoft.com>,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, qemu-s390x@nongnu.org,
 qemu-arm@nongnu.org, Michael Rolnik <mrolnik@gmail.com>,
 Paolo Bonzini <pbonzini@redhat.com>, qemu-ppc@nongnu.org,
 Stafford Horne <shorne@gmail.com>, qemu-riscv@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Cornelia Huck <cohuck@redhat.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 Laurent Vivier <laurent@vivier.eu>, Palmer Dabbelt <palmer@dabbelt.com>,
 Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org> <YTFxZb1Vg5pWVW9p@yekko>
 <fd383a02-fb9f-8641-937f-ebe1d8bb065f@linaro.org>
 <fc98e293-f2ba-8ca0-99c8-f07758b79d73@amsat.org>
Message-ID: <a49e0100-74d1-2974-990f-a05f9f796cc5@amsat.org>
Date: Sun, 12 Sep 2021 00:31:31 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <fc98e293-f2ba-8ca0-99c8-f07758b79d73@amsat.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/3/21 11:11 PM, Philippe Mathieu-Daudé wrote:
> On 9/3/21 10:42 PM, Richard Henderson wrote:
>> On 9/3/21 2:50 AM, David Gibson wrote:
>>> On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daudé wrote:
>>>> Each POWER cpu has its own has_work() implementation. Instead of
>>>> overloading CPUClass on each PowerPCCPUClass init, register the
>>>> generic ppc_cpu_has_work() handler, and have it call the POWER
>>>> specific has_work().
>>>
>>> I don't quite see the rationale for introducing a second layer of
>>> indirection here.  What's wrong with switching the base has_work for
>>> each cpu variant?
>>
>> We're moving the hook from CPUState to TCGCPUOps.
>> Phil was trying to avoid creating N versions of
>>
>> static const struct TCGCPUOps ppc_tcg_ops = {
>>     ...
>> };
> 
> Ah yes this is the reason! Too many context switching so
> I forgot about it.
> 
>> A plausible alternative is to remove the const from this struct and
>> modify it, just as we do for CPUState, on the assumption that we cannot
>> mix and match ppc cpu types in any one machine.
> 
> I thought about this case and remembered how it works on the ARM arch,
> i.e. ZynqMP machine uses both Cortex-R5F and Cortex-A53. Even if no
> similar PPC machine exists, IMHO we should try to generally allow to
> possibility to experiment machine with different CPUs. Restricting it
> on PPC goes the other way around. Thoughts?

I'm running out of ideas to do avoid the indirection and multiple
copies of TCGCPUOps. I'm not giving up, I suppose I'm simply not
seeing it... David, any suggestions?


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 22:40:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 22:40:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185113.333833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPBfb-00066N-QF; Sat, 11 Sep 2021 22:40:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185113.333833; Sat, 11 Sep 2021 22:40:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPBfb-00066G-Ml; Sat, 11 Sep 2021 22:40:23 +0000
Received: by outflank-mailman (input) for mailman id 185113;
 Sat, 11 Sep 2021 22:40:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPBfa-000666-3W; Sat, 11 Sep 2021 22:40:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPBfZ-0000ge-Vd; Sat, 11 Sep 2021 22:40:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPBfZ-0004KY-Nd; Sat, 11 Sep 2021 22:40:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPBfZ-0007Vn-NB; Sat, 11 Sep 2021 22:40:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zLpC6DZmRIlC0SDi4GZUrjIPUsbb7+KHaZA4DNMJsYQ=; b=ri3VyzsM16IXmvSgUbP5ilWSyN
	dyt0q+UhS2fGYY08RWUGQM++KlJCelGLlfp5g7+4ZNblqKV+MRtz4kPKr1Ukxwq7Bb6Ozhk+3kw4l
	qSwRzZBk+JEL9wxfat28DZtiX2aoR2iFW7URZSent62s5DMExF6yvEfVhaQJCRTXLMUI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164947-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164947: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 22:40:21 +0000

flight 164947 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164947/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  428 days
Failing since        151818  2020-07-11 04:18:52 Z  427 days  418 attempts
Testing same since   164947  2021-09-11 04:21:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73284 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 11 23:11:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 11 Sep 2021 23:11:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185120.333848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPC9B-0001Yu-71; Sat, 11 Sep 2021 23:10:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185120.333848; Sat, 11 Sep 2021 23:10:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPC9B-0001Yn-1P; Sat, 11 Sep 2021 23:10:57 +0000
Received: by outflank-mailman (input) for mailman id 185120;
 Sat, 11 Sep 2021 23:10:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPC99-0001Ya-8b; Sat, 11 Sep 2021 23:10:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPC98-0001DK-SE; Sat, 11 Sep 2021 23:10:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPC98-0006xa-FL; Sat, 11 Sep 2021 23:10:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPC98-00041F-En; Sat, 11 Sep 2021 23:10:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4CIOr3UD4lCq3jqLET8ODbPRp8QCXYPY9QDTcNKZVmA=; b=HbQLx/gqxfIFeWhwtAJqz9tzDF
	H+bluNY8KL412qCTQD9Q7lpjMX5v38TrRGEOqyGsCgip+sUv6pI13g8OaTNwY2KUamP3guE2xlp08
	tGzzfGmj5ao1fI69+sCZGjfh+3zdwZ+OtNtBI1ox+Pdp8NvPKyS+nHQWMEv0I7Y6L01s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164946-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164946: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=dcd3d63f4f77da5540bfe35d8ccb040d70644568
X-Osstest-Versions-That:
    ovmf=cf7c65059202e7858fbc00ce1f163ee243947e08
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 11 Sep 2021 23:10:54 +0000

flight 164946 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164946/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 dcd3d63f4f77da5540bfe35d8ccb040d70644568
baseline version:
 ovmf                 cf7c65059202e7858fbc00ce1f163ee243947e08

Last test of basis   164937  2021-09-10 05:29:00 Z    1 days
Testing same since   164946  2021-09-11 00:12:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dun Tan <dun.tan@intel.com>
  duntan <dun.tan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   cf7c650592..dcd3d63f4f  dcd3d63f4f77da5540bfe35d8ccb040d70644568 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 00:11:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 00:11:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185131.333865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPD5X-00015O-KS; Sun, 12 Sep 2021 00:11:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185131.333865; Sun, 12 Sep 2021 00:11:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPD5X-00015F-HB; Sun, 12 Sep 2021 00:11:15 +0000
Received: by outflank-mailman (input) for mailman id 185131;
 Sun, 12 Sep 2021 00:11:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPD5V-00014x-HS; Sun, 12 Sep 2021 00:11:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPD5V-0002rC-CQ; Sun, 12 Sep 2021 00:11:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPD5V-0001fY-1h; Sun, 12 Sep 2021 00:11:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPD5V-0002Jm-1B; Sun, 12 Sep 2021 00:11:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JqVSV4zeBQnEeG7rhNJBwyIVlg3mApYzibPRQhGgXzk=; b=lgulw+mg0kXjRzuw0wQivbfY/A
	1ziZUyH5c6iFbuQsWNWmaSv8rDYBAYL6vKGS88EwzRkr3X3yVIixSZpjhxcu+RquzrGytX0c0A6YV
	/htFkrXL1Nhixdkp+UyKfk13jflnEP2hHt6aIFhAupRNn3vvzR3cEITKJEoi7KlRr41w=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164945-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164945: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
X-Osstest-Versions-That:
    xen=2e2e22c7d50392fa5cced3e0334b217426f48b7a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 00:11:13 +0000

flight 164945 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164945/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 164922

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164922
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164922
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164922
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164922
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164922
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164922
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164922
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164922
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164922
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164922
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164922
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164922
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782
baseline version:
 xen                  2e2e22c7d50392fa5cced3e0334b217426f48b7a

Last test of basis   164922  2021-09-09 21:01:25 Z    2 days
Testing same since   164945  2021-09-10 21:23:48 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Kevin Stefanov <kevin.stefanov@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2e2e22c7d5..c76cfada1c  c76cfada1cfad05aaf64ce3ad305c5467650e782 -> master


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 02:10:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 02:10:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185142.333879 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPEwv-0004fG-NU; Sun, 12 Sep 2021 02:10:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185142.333879; Sun, 12 Sep 2021 02:10:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPEwv-0004f8-Ge; Sun, 12 Sep 2021 02:10:29 +0000
Received: by outflank-mailman (input) for mailman id 185142;
 Sun, 12 Sep 2021 02:10:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPEwt-0004ey-NQ; Sun, 12 Sep 2021 02:10:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPEwt-0003BJ-Hq; Sun, 12 Sep 2021 02:10:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPEwt-0005td-4b; Sun, 12 Sep 2021 02:10:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPEwt-0002Cu-3b; Sun, 12 Sep 2021 02:10:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YusWAYJnluW0iqOzQKETRq493UVIGiU9VjJoTJOKf9I=; b=0HSrjfMprLQgtlWZ3K5Zd8TGA2
	CcHDptqULPiP9jW8x5DN9Ca6B2Y+Es5+WalxftmaMgpC5ayaivpLTrHvmEARSc72SuI9tm0TKr9F3
	Qi5nK8kDotzonW8wPlJQa5TwJMFISYBiOCAqqV0O18nbFkp737JTou4kz4QNloxlQd3s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164948-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164948: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit2:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:guest-start.2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=926de8c4326c14fcf35f1de142019043597a4fac
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 02:10:27 +0000

flight 164948 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164948/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-libvirt-qcow2 20 guest-start.2         fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-freebsd11-amd64 21 guest-start/freebsd.repeat fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                926de8c4326c14fcf35f1de142019043597a4fac
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  407 days
Failing since        152366  2020-08-01 20:49:34 Z  406 days  705 attempts
Testing same since   164948  2021-09-11 10:37:01 Z    0 days    1 attempts

------------------------------------------------------------
7283 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  fail    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               fail    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2249566 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 07:02:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 07:02:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185158.333893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPJVL-0003ie-F9; Sun, 12 Sep 2021 07:02:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185158.333893; Sun, 12 Sep 2021 07:02:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPJVL-0003iX-C1; Sun, 12 Sep 2021 07:02:19 +0000
Received: by outflank-mailman (input) for mailman id 185158;
 Sun, 12 Sep 2021 07:02:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPJVK-0003iK-6q; Sun, 12 Sep 2021 07:02:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPJVK-000099-2i; Sun, 12 Sep 2021 07:02:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPJVJ-0003RD-Mt; Sun, 12 Sep 2021 07:02:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPJVJ-00070W-MR; Sun, 12 Sep 2021 07:02:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tv+il5PYnNjQKeI/njEw4Z+iqWXvL+r3zSyTzkCoi0I=; b=SV7MGzzYFAs1Fw3dbfLq5KwzRX
	otZkOO1jkm/3T6rNVDk8d2D7Pp2DeZP5emBPcXig/vXz9JhOHHlytqOM+j7IN4FWGU4td0chQnGKK
	Jg4J2VxpNXaNqn3n6HAUBi0zSB5LqsoXwfnelo1zO9Byf7bBVWFOpe6I43qjtRyGYGOA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164950-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164950: tolerable FAIL - PUSHED
X-Osstest-Failures:
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
X-Osstest-Versions-That:
    qemuu=500f1f3e81ad112e28b7c979136847e32dad83b6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 07:02:17 +0000

flight 164950 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164950/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164941
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164941
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164941
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164941
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164941
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164941
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164941
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164941
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081
baseline version:
 qemuu                500f1f3e81ad112e28b7c979136847e32dad83b6

Last test of basis   164941  2021-09-10 10:38:05 Z    1 days
Testing same since   164950  2021-09-11 18:57:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Kabaev <kan@FreeBSD.ORG>
  Colin Percival <cperciva@tarsnap.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Justin Hibbits <chmeeedalf@gmail.com>
  Kyle Evans <kevans@freebsd.org>
  Laurent Vivier <laurent@vivier.eu>
  Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
  Michal Meloun <mmel@FreeBSD.org>
  Mikaël Urankar <mikael.urankar@gmail.com>
  Peter Maydell <peter.maydell@linaro.org>
  Richard Henderson <richard.henderson@linaro.org>
  Sean Bruno <sbruno@FreeBSD.org>
  Stacey Son <sson@FreeBSD.org>
  Volker Rümelin <vr_qemu@t-online.de>
  Warner Losh <imp@bsdimp.com>
  Warner Losh <imp@FreeBSD.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/qemu-xen.git
   500f1f3e81..99c44988d5  99c44988d5ba1866a411450c877ed818b1b70081 -> upstream-tested


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 10:06:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 10:06:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185199.333907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPMN0-0007W7-Nc; Sun, 12 Sep 2021 10:05:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185199.333907; Sun, 12 Sep 2021 10:05:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPMN0-0007W0-Jy; Sun, 12 Sep 2021 10:05:54 +0000
Received: by outflank-mailman (input) for mailman id 185199;
 Sun, 12 Sep 2021 10:05:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMMz-0007Vq-Cy; Sun, 12 Sep 2021 10:05:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMMz-0003nX-47; Sun, 12 Sep 2021 10:05:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMMy-0006B7-PZ; Sun, 12 Sep 2021 10:05:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMMy-0000jh-Mm; Sun, 12 Sep 2021 10:05:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qTPdib4ty4MZMBFL7jAdZLBPhavThskz0PQ3EoJR26A=; b=utKh5LcSdsIRSFQ5B/zsRvvPwg
	qLPjLHzP080pegtCZFOzr9L+da1XTza6fK+iJmFsxcVyCCLdnylBw0eey+qVbBdZU+tfAOqU6tI6Y
	H4tKSftlKQoR3Pq/X5Ah5JMVtH9XpkqKQNfewcjDv1pNHGVTeKrla0ozB2b95KeKj/uM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164953-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164953: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 10:05:52 +0000

flight 164953 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164953/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  429 days
Failing since        151818  2020-07-11 04:18:52 Z  428 days  419 attempts
Testing same since   164947  2021-09-11 04:21:39 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73284 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 10:07:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 10:07:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185206.333921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPMOD-00086h-41; Sun, 12 Sep 2021 10:07:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185206.333921; Sun, 12 Sep 2021 10:07:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPMOD-00086a-07; Sun, 12 Sep 2021 10:07:09 +0000
Received: by outflank-mailman (input) for mailman id 185206;
 Sun, 12 Sep 2021 10:07:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMOB-00085d-NS; Sun, 12 Sep 2021 10:07:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMOB-0003oU-K5; Sun, 12 Sep 2021 10:07:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMOB-0006DC-Cw; Sun, 12 Sep 2021 10:07:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPMOB-00032H-CO; Sun, 12 Sep 2021 10:07:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SUrnfNPVfBwK4fF6sHXE4Q+bdeizReDoCgxcM3wCkEY=; b=Sv7y2doiosMvGM0/avgWNWc7+4
	jJpM9/TNEzp8wonebdI7CYOtPfY7Un5TBplyOiZzzVXt+fudzlzYh4ji26+5hKeaPtrOQAqQEbpbD
	V/GF2jc9m2vojKtP/+gPHp1kH5NJ+otsikNO8FC9jdT/s/f0K9sXTSEz6el8UGDdndN0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164955-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 164955: all pass - PUSHED
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=e70a9a043a5ce6d4025420f729bc473f711bf5d1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 10:07:07 +0000

flight 164955 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164955/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  e70a9a043a5ce6d4025420f729bc473f711bf5d1

Last test of basis   164882  2021-09-08 09:18:32 Z    4 days
Testing same since   164955  2021-09-12 09:19:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>
  Jane Malalane <jane.malalane@citrix.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e70a9a043a..6d45368a0a  6d45368a0a89e01a3a01d156af61fea565db96cc -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 12:32:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 12:32:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185238.333935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPOe5-0007tJ-SO; Sun, 12 Sep 2021 12:31:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185238.333935; Sun, 12 Sep 2021 12:31:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPOe5-0007tC-PE; Sun, 12 Sep 2021 12:31:41 +0000
Received: by outflank-mailman (input) for mailman id 185238;
 Sun, 12 Sep 2021 12:31:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AfbI=OC=linaro.org=richard.henderson@srs-us1.protection.inumbo.net>)
 id 1mPOe4-0007t6-VE
 for xen-devel@lists.xenproject.org; Sun, 12 Sep 2021 12:31:41 +0000
Received: from mail-pf1-x432.google.com (unknown [2607:f8b0:4864:20::432])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f738e494-b194-4bae-9350-8ecbb53cc5eb;
 Sun, 12 Sep 2021 12:31:39 +0000 (UTC)
Received: by mail-pf1-x432.google.com with SMTP id 18so6239852pfh.9
 for <xen-devel@lists.xenproject.org>; Sun, 12 Sep 2021 05:31:39 -0700 (PDT)
Received: from [192.168.1.11] ([71.212.134.125])
 by smtp.gmail.com with ESMTPSA id z9sm3970376pfn.22.2021.09.12.05.31.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 12 Sep 2021 05:31:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f738e494-b194-4bae-9350-8ecbb53cc5eb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=l2bSjNJqTwhQZphEwMAU513EeIhPI7Thg1lJpA7i8fs=;
        b=R66inDcAWt9GQlblcZebPLMwPlHlAnt0L3kyGlH8r3X6FsDI/N/vYwneGzy6f53yoC
         aJSbeJ2HVLZ5IETneJLglbqjOoAzPmPkvYhRyqU7w9yu4eGlZVUBvR4cBjWxsVNMWyQs
         qiLol3oXEuJFHpxM5XDG2HifDgtIlXc6Y+gNn2EVYS2UFBYwOF9tNaPzUqjb6c9xEMKD
         EpbKtmy5jQyQ9JwjSHSZdDyKcuXW75ONjIPM4WmszEr/HJrFRo+qRBybdHwe5FUddKSp
         VhyKUscbCxv1rziWUD7Bm32+mC15T8UjtL/0R3OkiO3/Lb77FRexMdrsOVYoCJhihCLc
         vdAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=l2bSjNJqTwhQZphEwMAU513EeIhPI7Thg1lJpA7i8fs=;
        b=sG97wLIrtQNifhcFYEA4qJkAH9NdWe/nzb0Z/raXmv3yMagpV/8TV8fjNCte668fAz
         NLr+CwiP7hvApaJb3K0inkqLtvZ3VESG5H+3+XtClZJJkqampfrved0H+aoAmTKQ2Kyl
         FlOYpW7CFQRDPlUl36OgU25rjWKu3DJc+Px+VTyKPAMJarV4/MumY2VC5CXQglfqSpZi
         zW4JuIvFg+9k2kO4OR4eR9PKHDOYgIwnoHmdYO0lXYgbLNaEd99Wz0cGfPw+Io96ZQ0E
         c5f70f9krxNRRkmYg4OY//xdX3JDWaLZeksWYFqg3TlnzxJ2iZjI7HJeRRUekleXCb8K
         bDww==
X-Gm-Message-State: AOAM531idabQ9nk8t5jveiwy2UJ3GRKMFKr88Lodn5tIH33+JFxeU+wf
	REdJIDnN67X/E1TyH0kDIaMfzQ==
X-Google-Smtp-Source: ABdhPJxYx4l/5Z0anqBV+G7Bghe5EKN+RrvdQIUDU4YrpZdFpP+VKFzDEwt6BtZ6HBag6jIPDdKEog==
X-Received: by 2002:a63:5413:: with SMTP id i19mr6469698pgb.297.1631449898733;
        Sun, 12 Sep 2021 05:31:38 -0700 (PDT)
Subject: Re: [PATCH v3 21/30] target/ppc: Introduce
 PowerPCCPUClass::has_work()
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <f4bug@amsat.org>,
 David Gibson <david@gibson.dropbear.id.au>, =?UTF-8?Q?C=c3=a9dric_Le_Goater?=
 <clg@kaod.org>, Greg Kurz <groug@kaod.org>
Cc: Peter Maydell <peter.maydell@linaro.org>, Chris Wulff
 <crwulff@gmail.com>, kvm@vger.kernel.org,
 David Hildenbrand <david@redhat.com>, Bin Meng <bin.meng@windriver.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, qemu-devel@nongnu.org,
 Jiaxun Yang <jiaxun.yang@flygoat.com>, Max Filippov <jcmvbkbc@gmail.com>,
 Taylor Simpson <tsimpson@quicinc.com>, haxm-team@intel.com,
 Colin Xu <colin.xu@intel.com>, Sunil Muthuswamy <sunilmut@microsoft.com>,
 Marek Vasut <marex@denx.de>, Stefano Stabellini <sstabellini@kernel.org>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Paul Durrant <paul@xen.org>,
 Kamil Rytarowski <kamil@netbsd.org>, Reinoud Zandijk <reinoud@netbsd.org>,
 Claudio Fontana <cfontana@suse.de>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Anthony Perard <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org,
 Artyom Tarasenko <atar4qemu@gmail.com>, Laurent Vivier <lvivier@redhat.com>,
 Thomas Huth <thuth@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Cameron Esfahani <dirty@apple.com>, qemu-s390x@nongnu.org,
 qemu-arm@nongnu.org, Michael Rolnik <mrolnik@gmail.com>,
 Paolo Bonzini <pbonzini@redhat.com>, qemu-ppc@nongnu.org,
 Stafford Horne <shorne@gmail.com>, qemu-riscv@nongnu.org,
 Bastian Koppelmann <kbastian@mail.uni-paderborn.de>,
 Cornelia Huck <cohuck@redhat.com>, Roman Bolshakov <r.bolshakov@yadro.com>,
 Laurent Vivier <laurent@vivier.eu>, Palmer Dabbelt <palmer@dabbelt.com>,
 Wenchao Wang <wenchao.wang@intel.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>
References: <20210902161543.417092-1-f4bug@amsat.org>
 <20210902161543.417092-22-f4bug@amsat.org> <YTFxZb1Vg5pWVW9p@yekko>
 <fd383a02-fb9f-8641-937f-ebe1d8bb065f@linaro.org>
 <fc98e293-f2ba-8ca0-99c8-f07758b79d73@amsat.org>
 <a49e0100-74d1-2974-990f-a05f9f796cc5@amsat.org>
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <82c9e627-016a-e834-6ed0-4c5d49b554e6@linaro.org>
Date: Sun, 12 Sep 2021 05:31:33 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <a49e0100-74d1-2974-990f-a05f9f796cc5@amsat.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/11/21 3:31 PM, Philippe Mathieu-Daudé wrote:
> On 9/3/21 11:11 PM, Philippe Mathieu-Daudé wrote:
>> On 9/3/21 10:42 PM, Richard Henderson wrote:
>>> On 9/3/21 2:50 AM, David Gibson wrote:
>>>> On Thu, Sep 02, 2021 at 06:15:34PM +0200, Philippe Mathieu-Daudé wrote:
>>>>> Each POWER cpu has its own has_work() implementation. Instead of
>>>>> overloading CPUClass on each PowerPCCPUClass init, register the
>>>>> generic ppc_cpu_has_work() handler, and have it call the POWER
>>>>> specific has_work().
>>>>
>>>> I don't quite see the rationale for introducing a second layer of
>>>> indirection here.  What's wrong with switching the base has_work for
>>>> each cpu variant?
>>>
>>> We're moving the hook from CPUState to TCGCPUOps.
>>> Phil was trying to avoid creating N versions of
>>>
>>> static const struct TCGCPUOps ppc_tcg_ops = {
>>>      ...
>>> };
>>
>> Ah yes this is the reason! Too many context switching so
>> I forgot about it.
>>
>>> A plausible alternative is to remove the const from this struct and
>>> modify it, just as we do for CPUState, on the assumption that we cannot
>>> mix and match ppc cpu types in any one machine.
>>
>> I thought about this case and remembered how it works on the ARM arch,
>> i.e. ZynqMP machine uses both Cortex-R5F and Cortex-A53. Even if no
>> similar PPC machine exists, IMHO we should try to generally allow to
>> possibility to experiment machine with different CPUs. Restricting it
>> on PPC goes the other way around. Thoughts?
> 
> I'm running out of ideas to do avoid the indirection and multiple
> copies of TCGCPUOps. I'm not giving up, I suppose I'm simply not
> seeing it... David, any suggestions?

I think multiple copies of TCGCPUOps is the solution.  Macro-ized, perhaps, so that the 
amount of typing is minimal across the versions.


r~


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 13:00:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 13:00:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185250.333946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPP6I-00035W-3T; Sun, 12 Sep 2021 13:00:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185250.333946; Sun, 12 Sep 2021 13:00:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPP6H-00035P-Vy; Sun, 12 Sep 2021 13:00:49 +0000
Received: by outflank-mailman (input) for mailman id 185250;
 Sun, 12 Sep 2021 13:00:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPP6G-00035F-RN; Sun, 12 Sep 2021 13:00:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPP6G-0006lG-HX; Sun, 12 Sep 2021 13:00:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPP6G-0005Xo-5w; Sun, 12 Sep 2021 13:00:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPP6G-0000oZ-5P; Sun, 12 Sep 2021 13:00:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3M4SnCOO0J8s1NKSywydNW5JkbqP2v3y1KLWXyGu2GM=; b=RDIJCQh23clexc+voPrOjFQ6ue
	mdbH6vXDlXJbsaAb3SymIRxlyZOvUInOzB6JXcbUG3gojpY+3h0BJPi6mJjSuDDfoyE2HpleTjzFa
	pZextw8h1hiJBPacAuaNIZ++/c0D6SQCCfwvZrq9mELxJbb8glgQi2uE/dkrtgCXfpbw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164952-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164952: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-xsm:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=78e709522d2c012cb0daad2e668506637bffb7c2
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 13:00:48 +0000

flight 164952 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164952/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-xsm      14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                78e709522d2c012cb0daad2e668506637bffb7c2
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  407 days
Failing since        152366  2020-08-01 20:49:34 Z  406 days  706 attempts
Testing same since   164952  2021-09-12 02:13:34 Z    0 days    1 attempts

------------------------------------------------------------
7297 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2253572 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 13:27:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 13:27:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185264.333959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPPVv-000685-As; Sun, 12 Sep 2021 13:27:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185264.333959; Sun, 12 Sep 2021 13:27:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPPVv-00067y-7s; Sun, 12 Sep 2021 13:27:19 +0000
Received: by outflank-mailman (input) for mailman id 185264;
 Sun, 12 Sep 2021 13:27:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPPVt-00067g-Ou; Sun, 12 Sep 2021 13:27:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPPVt-0007AM-HM; Sun, 12 Sep 2021 13:27:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPPVt-0006Zy-6O; Sun, 12 Sep 2021 13:27:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPPVt-00052a-5s; Sun, 12 Sep 2021 13:27:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=l5ZSjzwrqDcSmCqL1hT3GTpuvVnly9gx8MiJfi/rV9o=; b=6awA0A/Sq98NY71W0PJy1/+YTX
	NrGZ5+z8UaoNo50HJN5uHL22W/Lvf9P2rPayX1WK069CgcBcV6sr5vbxF8vSW1LS1+h1QVCgb1JLv
	NwayBDSBuyfacUL1u5kvoqpGW5Ki1qTXG+UoHIUnPn3BIilXlmutqC9pZZ20wohFqpZM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164951-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164951: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate:fail:allowable
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 13:27:17 +0000

flight 164951 xen-unstable real [real]
flight 164956 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164951/
http://logs.test-lab.xenproject.org/osstest/logs/164956/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     18 guest-localmigrate       fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    1 days
Testing same since   164951  2021-09-12 00:14:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 18:07:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 18:07:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185325.333980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPTt6-0002Wg-5G; Sun, 12 Sep 2021 18:07:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185325.333980; Sun, 12 Sep 2021 18:07:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPTt6-0002WZ-2F; Sun, 12 Sep 2021 18:07:32 +0000
Received: by outflank-mailman (input) for mailman id 185325;
 Sun, 12 Sep 2021 18:07:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPTt4-0002WP-7b; Sun, 12 Sep 2021 18:07:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPTt4-00043s-1c; Sun, 12 Sep 2021 18:07:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPTt3-0003w3-LU; Sun, 12 Sep 2021 18:07:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPTt3-0001Vl-L2; Sun, 12 Sep 2021 18:07:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sIHnZhS7O54TYA5+wPB+fKqloN3aXjiRUDv1BrJbqzk=; b=H5hU5jFN2gwCTXQY2XGIIzImdC
	mEl+vef47QtAs1yZGko8EsmhM9HpjotbdLbG8GDVsU5mpTCP8EmqvJWsWVB8L35lr8Xz9cv1BzkLt
	vXzSWXN4OrEf0CgVfSbChtaDaTbtMwWpO17976HBOrbnODh8+Iv4h7YBqVsPFBpZI1DQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164954-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 164954: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    linux-5.4:build-arm64-xsm:<job status>:broken:regression
    linux-5.4:build-arm64-xsm:host-install(4):broken:regression
    linux-5.4:test-amd64-amd64-xl-qemuu-ovmf-amd64:guest-start/debianhvm.repeat:fail:regression
    linux-5.4:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a0f68fb55ebc85e5ed3c6582ffc70379340c4a72
X-Osstest-Versions-That:
    linux=c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 18:07:29 +0000

flight 164954 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164954/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm                 <job status>                 broken
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164793
 test-amd64-amd64-xl-qemuu-ovmf-amd64 20 guest-start/debianhvm.repeat fail REGR. vs. 164793

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164793
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164793
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164793
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164793
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164793
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164793
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164793
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164793
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164793
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164793
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164793
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a0f68fb55ebc85e5ed3c6582ffc70379340c4a72
baseline version:
 linux                c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924

Last test of basis   164793  2021-09-03 08:40:28 Z    9 days
Testing same since   164954  2021-09-12 07:13:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Shishkin <alexander.shishkin@linux.intel.com>
  Alexander Tsoy <alexander@tsoy.me>
  Andrew Morton <akpm@linux-foundation.org>
  Ariel Elior <aelior@marvell.com>
  Ben Dooks <ben-linux@fluff.org>
  Ben Dooks <ben.dooks@codethink.co.uk>
  Bjorn Helgaas <bhelgaas@google.com>
  Christoph Hellwig <hch@lst.de>
  Chunfeng Yun <chunfeng.yun@mediatek.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Eric Biggers <ebiggers@google.com>
  Esben Haabendal <esben@geanix.com>
  Fangrui Song <maskray@google.com>
  Florian Fainelli <f.fainelli@gmail.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Harini Katakam <harini.katakam@xilinx.com>
  Hayes Wang <hayeswang@realtek.com>
  Hulk Robot <hulkrobot@huawei.com>
  Ingo Molnar <mingo@kernel.org>
  Jakub Kicinski <kuba@kernel.org>
  Jens Axboe <axboe@kernel.dk>
  Johan Hovold <johan@kernel.org>
  Kim Phillips <kim.phillips@amd.com>
  Krzysztof Halasa <khalasa@piap.pl>
  Krzysztof Hałasa <khalasa@piap.pl>
  Lee Jones <lee.jones@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Liu Jian <liujian56@huawei.com>
  Marek Behún <kabel@kernel.org>
  Mathias Nyman <mathias.nyman@linux.intel.com>
  Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
  Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
  Max Filippov <jcmvbkbc@gmail.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Simek <michal.simek@xilinx.com>
  Muchun Song <songmuchun@bytedance.com>
  Patrick Schaaf <bof@bof.de>
  Paul Gortmaker <paul.gortmaker@windriver.com>
  Pavel Skripkin <paskripkin@gmail.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Philipp Zabel <p.zabel@pengutronix.de>
  Prabhakar Kushwaha <pkushwaha@marvell.com>
  Qu Wenruo <wqu@suse.com>
  Randy Dunlap <rdunlap@infradead.org>
  Russell King <rmk+kernel@armlinux.org.uk>
  Sai Krishna Potthuri <lakshmi.sai.krishna.potthuri@xilinx.com>
  Sasha Levin <sashal@kernel.org>
  Shai Malin <smalin@marvell.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
  Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
  Takashi Iwai <tiwai@suse.de>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Gleixner <tglx@linutronix.de>
  Tom Rix <trix@redhat.com>
  Vineet Gupta <vgupta@synopsys.com>
  Vlastimil Babka <vbabka@suse.cz>
  Xiaoyao Li <xiaoyao.li@intel.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Zubin Mithra <zsm@chromium.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         fail    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 1015 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 12 20:18:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 12 Sep 2021 20:18:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185343.333994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPVvA-00083X-Th; Sun, 12 Sep 2021 20:17:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185343.333994; Sun, 12 Sep 2021 20:17:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPVvA-00083Q-Q4; Sun, 12 Sep 2021 20:17:48 +0000
Received: by outflank-mailman (input) for mailman id 185343;
 Sun, 12 Sep 2021 20:17:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPVv9-00083G-2x; Sun, 12 Sep 2021 20:17:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPVv8-0006Es-PJ; Sun, 12 Sep 2021 20:17:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPVv8-0002ut-Dp; Sun, 12 Sep 2021 20:17:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPVv8-0004MP-DK; Sun, 12 Sep 2021 20:17:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2nx6Ce8C4gt5QIrb3DUK9KJndOh3s0gVqfge05D+QFA=; b=HBtEvcnlF9Wrxd4J/9797IO7nG
	KZUqeE8lDZrbB6AzJOu6oYg09RLNpNzY5laAWwVsknTszkcHOalgtJX8d0Jj5M2REtTq3AJvL5Jcs
	BSNVEEh10+St20IF6AimFC70jtAPWub49RfiD2v2dyBg41aySKz4T6mi0fVd8keEeN98=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164957-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164957: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-start:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:debian-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:guest-start:fail:heisenbug
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=78e709522d2c012cb0daad2e668506637bffb7c2
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 12 Sep 2021 20:17:46 +0000

flight 164957 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164957/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-xsm      14 guest-start      fail in 164952 pass in 164957
 test-arm64-arm64-xl-xsm       8 xen-boot         fail in 164952 pass in 164957
 test-armhf-armhf-xl-rtds     12 debian-install             fail pass in 164952
 test-armhf-armhf-xl-vhd      13 guest-start                fail pass in 164952

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 164952 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 164952 never pass
 test-armhf-armhf-xl-vhd     14 migrate-support-check fail in 164952 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 164952 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                78e709522d2c012cb0daad2e668506637bffb7c2
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  408 days
Failing since        152366  2020-08-01 20:49:34 Z  406 days  707 attempts
Testing same since   164952  2021-09-12 02:13:34 Z    0 days    2 attempts

------------------------------------------------------------
7297 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2253572 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 02:49:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 02:49:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185385.334008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPc29-0005Zc-35; Mon, 13 Sep 2021 02:49:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185385.334008; Mon, 13 Sep 2021 02:49:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPc28-0005ZV-W4; Mon, 13 Sep 2021 02:49:24 +0000
Received: by outflank-mailman (input) for mailman id 185385;
 Mon, 13 Sep 2021 02:49:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPc28-0005ZL-4P; Mon, 13 Sep 2021 02:49:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPc27-0003ER-S2; Mon, 13 Sep 2021 02:49:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPc27-0006Aq-Co; Mon, 13 Sep 2021 02:49:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPc27-0006TP-C8; Mon, 13 Sep 2021 02:49:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=F51t118kZE9hLhWNi553t6Q2Jp5KqCHjaFZL65RWIjw=; b=68SMhv2CyGaRERQ1SVD+FeNBf0
	ub82KpZBbQqSQMvlc9qpKSLSFasczJn+BrWf2aNBLz1iqvMt9rfp99CZRkJpnPG5RC+Nuc9lfyF/+
	O8DlX5TOhfNL8HQ39BTASyp7LzTsDubECa2/Rc1lUXLPpHfKdi52MqFFsMNeA1JhHgKo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164958-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164958: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate:fail:heisenbug
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 02:49:23 +0000

flight 164958 xen-unstable real [real]
flight 164962 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164958/
http://logs.test-lab.xenproject.org/osstest/logs/164962/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail in 164951 REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-rtds   18 guest-localmigrate fail in 164951 pass in 164958
 test-arm64-arm64-libvirt-raw 13 guest-start                fail pass in 164951
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 164951

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw 14 migrate-support-check fail in 164951 never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check fail in 164951 never pass
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164945
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    2 days
Testing same since   164951  2021-09-12 00:14:36 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 04:42:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 04:42:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185395.334022 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPdnK-0001wz-Dt; Mon, 13 Sep 2021 04:42:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185395.334022; Mon, 13 Sep 2021 04:42:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPdnK-0001ws-Ak; Mon, 13 Sep 2021 04:42:14 +0000
Received: by outflank-mailman (input) for mailman id 185395;
 Mon, 13 Sep 2021 04:42:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPdnI-0001wf-TX; Mon, 13 Sep 2021 04:42:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPdnI-0005pn-HB; Mon, 13 Sep 2021 04:42:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPdnI-00045P-6s; Mon, 13 Sep 2021 04:42:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPdnI-0002Cp-6J; Mon, 13 Sep 2021 04:42:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3Jxnc/Ekw2vBvb+3MEA/k40O5B6D/xeAi8LpuBVC/CI=; b=hE8ULuLWTcuumZRZhlTz1iH3lQ
	jbilf6XK35LB+dUvla496ffTJQYKVVllMQJeDOdNSlUzEs0nHYGPPdrFdIEQsmXgS2MvPwwzZ5xuC
	ZGHNcC57D9v1RY/Z1xBlzjOvkWfJJGRf9amCKHAwGFd8d6xi7RmJSVhsGFvEJFfR2vxc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164960-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 164960: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a0f68fb55ebc85e5ed3c6582ffc70379340c4a72
X-Osstest-Versions-That:
    linux=c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 04:42:12 +0000

flight 164960 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164960/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164793
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164793
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164793
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164793
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164793
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164793
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164793
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164793
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164793
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164793
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164793
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a0f68fb55ebc85e5ed3c6582ffc70379340c4a72
baseline version:
 linux                c6bf0ed9d1a727fcd98f6a52bbdd2afbdfb44924

Last test of basis   164793  2021-09-03 08:40:28 Z    9 days
Testing same since   164954  2021-09-12 07:13:36 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Shishkin <alexander.shishkin@linux.intel.com>
  Alexander Tsoy <alexander@tsoy.me>
  Andrew Morton <akpm@linux-foundation.org>
  Ariel Elior <aelior@marvell.com>
  Ben Dooks <ben-linux@fluff.org>
  Ben Dooks <ben.dooks@codethink.co.uk>
  Bjorn Helgaas <bhelgaas@google.com>
  Christoph Hellwig <hch@lst.de>
  Chunfeng Yun <chunfeng.yun@mediatek.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Eric Biggers <ebiggers@google.com>
  Esben Haabendal <esben@geanix.com>
  Fangrui Song <maskray@google.com>
  Florian Fainelli <f.fainelli@gmail.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Harini Katakam <harini.katakam@xilinx.com>
  Hayes Wang <hayeswang@realtek.com>
  Hulk Robot <hulkrobot@huawei.com>
  Ingo Molnar <mingo@kernel.org>
  Jakub Kicinski <kuba@kernel.org>
  Jens Axboe <axboe@kernel.dk>
  Johan Hovold <johan@kernel.org>
  Kim Phillips <kim.phillips@amd.com>
  Krzysztof Halasa <khalasa@piap.pl>
  Krzysztof Hałasa <khalasa@piap.pl>
  Lee Jones <lee.jones@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Liu Jian <liujian56@huawei.com>
  Marek Behún <kabel@kernel.org>
  Mathias Nyman <mathias.nyman@linux.intel.com>
  Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
  Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
  Max Filippov <jcmvbkbc@gmail.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Simek <michal.simek@xilinx.com>
  Muchun Song <songmuchun@bytedance.com>
  Patrick Schaaf <bof@bof.de>
  Paul Gortmaker <paul.gortmaker@windriver.com>
  Pavel Skripkin <paskripkin@gmail.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Philipp Zabel <p.zabel@pengutronix.de>
  Prabhakar Kushwaha <pkushwaha@marvell.com>
  Qu Wenruo <wqu@suse.com>
  Randy Dunlap <rdunlap@infradead.org>
  Russell King <rmk+kernel@armlinux.org.uk>
  Sai Krishna Potthuri <lakshmi.sai.krishna.potthuri@xilinx.com>
  Sasha Levin <sashal@kernel.org>
  Shai Malin <smalin@marvell.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
  Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
  Takashi Iwai <tiwai@suse.de>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Gleixner <tglx@linutronix.de>
  Tom Rix <trix@redhat.com>
  Vineet Gupta <vgupta@synopsys.com>
  Vlastimil Babka <vbabka@suse.cz>
  Xiaoyao Li <xiaoyao.li@intel.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Zubin Mithra <zsm@chromium.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   c6bf0ed9d1a7..a0f68fb55ebc  a0f68fb55ebc85e5ed3c6582ffc70379340c4a72 -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 04:51:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 04:51:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185403.334036 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPdwK-0003Zz-Gf; Mon, 13 Sep 2021 04:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185403.334036; Mon, 13 Sep 2021 04:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPdwK-0003Zs-C5; Mon, 13 Sep 2021 04:51:32 +0000
Received: by outflank-mailman (input) for mailman id 185403;
 Mon, 13 Sep 2021 04:51:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=huvW=OD=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mPdwJ-0003Zm-6M
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 04:51:31 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dd9e87d0-3035-448f-8535-9acbb1cb5259;
 Mon, 13 Sep 2021 04:51:30 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 331791FF93;
 Mon, 13 Sep 2021 04:51:29 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id DAB6D132AB;
 Mon, 13 Sep 2021 04:51:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id Pt5QM9DYPmG2YgAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 13 Sep 2021 04:51:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd9e87d0-3035-448f-8535-9acbb1cb5259
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631508689; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=Pf70st+6PbyNeCrtQ1Sd32b6nRwU+ke6s+y3EZ9lB6k=;
	b=JJw/h4AFQc2dhVm7Z+QQXdhLeUWQgr+nYtgWIVDFLKIke7+3c1u3RRMKXs5eK+R2hkoUo2
	Q4Ls5siVP/IubFNgAc4ST+0b0zFnFrfsNJww+KgR2UVH2+f/s99jsWwtRdV60ssgm0FY6r
	ZvvCk4h0m9e4FRZhrQ74sXz/8UIxYLg=
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
 <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
 <24890.12692.44064.995392@mariner.uk.xensource.com>
 <462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
 <24891.31395.187330.546856@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
Message-ID: <08e63f7e-4d94-cf82-2563-384be00982b8@suse.com>
Date: Mon, 13 Sep 2021 06:51:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24891.31395.187330.546856@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="LHaRBHbZFDn1qXXhbKrxB3hH2AZPGpBaJ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--LHaRBHbZFDn1qXXhbKrxB3hH2AZPGpBaJ
Content-Type: multipart/mixed; boundary="EIrGniSJFH1HQI8aO9SEzBO6Rj3G77mKi";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-ID: <08e63f7e-4d94-cf82-2563-384be00982b8@suse.com>
Subject: Re: [PATCH v2 1/3] stubdom: fix build with disabled pv-grub
References: <20210909124924.1698-1-jgross@suse.com>
 <20210909124924.1698-2-jgross@suse.com>
 <24890.2778.780247.724651@mariner.uk.xensource.com>
 <7e635caa-0956-332c-bd17-0bec61e58ef0@suse.com>
 <24890.12692.44064.995392@mariner.uk.xensource.com>
 <462de39a-2edc-ee3e-8016-85ce29e7ca59@suse.com>
 <24891.31395.187330.546856@mariner.uk.xensource.com>
In-Reply-To: <24891.31395.187330.546856@mariner.uk.xensource.com>

--EIrGniSJFH1HQI8aO9SEzBO6Rj3G77mKi
Content-Type: multipart/mixed;
 boundary="------------BB5FF2595ABF07EBB24A5C87"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------BB5FF2595ABF07EBB24A5C87
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 17:32, Ian Jackson wrote:
> Juergen Gross writes ("Re: [PATCH v2 1/3] stubdom: fix build with disab=
led pv-grub"):
>> BTW, you probably want to modify OSStest to use configure with:
>>
>> --enable-pv-grub --enable-qemu-traditional
>>
>> in order to not let the tests fail (applies to x86 only, of course).
>=20
> I think it might be better to disable those tests.  What do people
> think ?

Disabling pv-grub tests is okay (I think it happened already, right?).

Disabling qemu-trad tests with qemu running in dom0 is fine, too, IMO.

Disabling the stubdom tests is much more concerning, as there is quite
some code in the tools which would be untested then.


Juergen

--------------BB5FF2595ABF07EBB24A5C87
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------BB5FF2595ABF07EBB24A5C87--

--EIrGniSJFH1HQI8aO9SEzBO6Rj3G77mKi--

--LHaRBHbZFDn1qXXhbKrxB3hH2AZPGpBaJ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE+2NAFAwAAAAAACgkQsN6d1ii/Ey+W
nwf/XdJe9ngprbPE4Jf8HN3e9bZmrsM4JUhy+nW/QwqX8OmBG9xhc7WfsuuCJuzj0vA/m7YRGaIt
jhscMcBMNSzVmAZshDb/5u3aeGP0c2lyMy0jfuuZKlJ/1rt858VtbZpc75Kq+F5JHZwh4QG4k2T7
8d25hhf8KfEuG8u/IokkqAymlQgYRvQIPivFoztKefTM+SN+jlENvmfKsjexi6ezg949IN1wOsfF
F4dREz8UN0/8Oy7pRmwkVqylXvhcEP5UsDwBzYGN/nR8L0d6HRtDt9yMW4Rp6aDDz/9n6RHUQTGX
7GXhCgzhMquBCPMPDI7jlQTpgw+tnzJMr+Nbtggxjw==
=CNUa
-----END PGP SIGNATURE-----

--LHaRBHbZFDn1qXXhbKrxB3hH2AZPGpBaJ--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:18:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:18:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185411.334047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfHm-0004Bs-L3; Mon, 13 Sep 2021 06:17:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185411.334047; Mon, 13 Sep 2021 06:17:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfHm-0004Bl-Hy; Mon, 13 Sep 2021 06:17:46 +0000
Received: by outflank-mailman (input) for mailman id 185411;
 Mon, 13 Sep 2021 06:17:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPfHl-0004Bf-SC
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:17:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4e630182-145a-11ec-b39c-12813bfff9fa;
 Mon, 13 Sep 2021 06:17:44 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-rXMWRt6jOlWcEjCfcLK_Yw-1; Mon, 13 Sep 2021 08:17:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2336.eurprd04.prod.outlook.com (2603:10a6:800:27::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 06:17:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:17:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0013.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 06:17:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e630182-145a-11ec-b39c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631513863;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VKAl0iYcj7NCke8e9iNAbFLKipAgg52emyZBF3rkWps=;
	b=Mp4ulQzuHZtDrmjwQLkL45TT1UzsmnkLoVgwZICrYXSQd6xdAp7Pvi1vffwIRM6Oisomeh
	1owCntmmbxrp56ofMZ/8mzOsNUp4Hy8qeowtPT4wNdkZXh9XijZjo42lXBtCarDt+oDY7i
	ggUcWne3U71cTtLEC5R/mhtg1qrGRm0=
X-MC-Unique: rXMWRt6jOlWcEjCfcLK_Yw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hzFBOeusD01//5LjL0aU8j6uXsH44rsKL6NpzUsI+GhCZKEY68nkSpzsy2XonUx62aZvfab1f4oR4KLenM/MTPF4AuMtdG/4IXhclNpfTMr5gGPCpq6Pr/eKIbWENXPNcgIHBw9Q4xXvP2SLTNXQsLa/x0HMq4Qoa7x0SU0oo2jGjQS++/n+UFWJi02MWi53bM5n3G2V72l7iJB3LiPS2T2Suwm1APsftGRRjdwC17flkAWxTO/l14ys0nBhG9L4cXOISNNJ5GtQ8W3nyVZbfjYS6hxFTiYgKbA7FZAKx7s/Y2tBBNjCInJePx04ISYBq3AukDUd5NZ0FIku227biA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VKAl0iYcj7NCke8e9iNAbFLKipAgg52emyZBF3rkWps=;
 b=faMTdXKpAvDHXO/rI5UdTNY6J7jOWvavTxgpQjUp1dXKIKk4A+oQWKtDWLB+nwo9JjcoBSSZlQMWilMvJa1Kb9VhVNgJyxuVjU2xi0IwXbMG4sRQyL/jd6mU/P3dloaLM7aNC1TLWmlVibuO43GbLrJI4P1IanOfRhL/kHfal3Gk6VLxxg7CfZvRVITnN2Qc/uOVxyJwNX/864VWKosq7171O+zNK9+nHwb2hXkdc8sa+M5Fq/+RIE7w8WfF4CZEVPX+Jc1flcO+BTL11jQUUj/rRPqarH15y1k4C3k36nAvRLkan4iUND0fICEukLqwviRvvb+1iT4q/1sQuOnwOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
From: Jan Beulich <jbeulich@suse.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
Message-ID: <c006550f-4fe7-29c5-f370-dc67bfac2b95@suse.com>
Date: Mon, 13 Sep 2021 08:17:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0013.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bf121ce6-f195-4c0b-1a84-08d9767e301f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2336:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23361D66D5AB4E8B656AD6F6B3D99@VI1PR0401MB2336.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8n5dx5NkBk8tsl9hyu1OW/1cb623pRWOKCuAryQjRtDLML8EGiBPAdKc0u24hmF7Rh2RM4sxCwh0EYKhOkcX6id8fiOevWBjgtqCVOD9Gaj0f1pj6JQJSj2N6r0qS5U5MSdZgu/f2F/91kBHjWrwYiqI9vuyvkBy2RvodjMrzYhBc1mV8m+5MWbzjh9G42xI7mQjnZxJk4W/nz3S9YtjuFaziuF5bFYtSjwJ1enwmDyfemR5yU7sY1JThxvrKICQvm4d7LpZ1lwmThNQJjdYEgx9NqjkF1iFVxohxXb/CULIuBGfCUxCDCJplE6oVT5dx+j6zkEoegRfTj3DWfHgnaN1s5pDXrZNj9mPAakjsZHqfyR/PvPsxR6tNFvZxRmzHzJKbJcTJmxEzwMi2x0GsabwOnZN9n3g63e/dwWAXtDfWqSQ5OYk7QSRR0J6Iah2BwtWIwFgNagHiuIs9GtfJHYPM43U9cNhvV3KP+mN1SDmExdPUYJCjSgvjwP9DuZ2qIu8EFLR/XZiOQWocrCdATcNw8WqtPaN7ZxcppLcNp00E5XkA47RyVZJwlWZkbCgDreb0xUAfBUNyESk/bkpHWc0zvqj/rUYleuBXh2HqmAmPgR8qfydOkA4AHzcIYYCcph8T0VQle7P2Yu4a6wh6EoowHfaly/bpfr5GPNOeTaTIt/SucDS88FPeuvDuKfhjbpet5LuAqTXXzeqYPuzGJxsO7K3iOFZk5J1UzXXQG4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(376002)(366004)(346002)(396003)(16576012)(8936002)(26005)(186003)(66946007)(53546011)(316002)(6916009)(8676002)(36756003)(38100700002)(54906003)(66556008)(66476007)(6486002)(4326008)(31686004)(2906002)(5660300002)(83380400001)(31696002)(86362001)(7416002)(478600001)(2616005)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YTAweTFkdHdHTFZzcTlZdVp4eDYrTEM1YW9MQVo4ZHFTNGE4eXpKTDFOT0Vl?=
 =?utf-8?B?Q24zVDJydFJHMU0yeUt4MWJhaXpCUTZvYXNGdUdXODdtV3QxN2wvZE52Nklo?=
 =?utf-8?B?elc3NmN3WDlrM3lvSThMaDJQb016VkpGZkxFenRTZngvWS9jSmQ4VnNMUXNs?=
 =?utf-8?B?WGVPMndDU2d4UFNCelZJWGdvMC9NMGJhS2RnMitWVjIwNVlqdVkzK1k0QTgw?=
 =?utf-8?B?UGlKY0cxc3NjbGFNNGc4SitKdWJSNmlpemhzV1dMRXpUczlXbUVkOEpWblJX?=
 =?utf-8?B?aWk1UnpybjBkZThuQUVvZVE0U2NZOUQzOTJTQnlrMWdkeHowVWFxWnEwY0dG?=
 =?utf-8?B?Z3ZmUDZhdGpRdzMxM244Tis3Q09waXNBamkveERHWXE2bGdSMmtGcGo4M08x?=
 =?utf-8?B?Vytndy9HQUFNdkNubmJoN1VCQUpTSGhHMVZzWDNJb0FUcUxKR096elVZUlFz?=
 =?utf-8?B?TXB1M043SkwzNVBKRStGNS95OExVenlIZFFkQ0VPYklmMmhlcW5MWnhKY1Av?=
 =?utf-8?B?Qys4MlB1QkVhRGVyK1RweEpqckZFL2Vhc1pBSlcxbGtLZlh3eHdqaCszdWF2?=
 =?utf-8?B?czZJbzJYZmUyZ0VPSkhCZ1V5UWVLZ3lqelpXOEg3WkxTUUloUGNKVjRwNVgx?=
 =?utf-8?B?ZVZRWVcySHhlaVlrR3FSSVVuSDduTFpiYWV4TWtwQWJRK2JoQm5PWXlnYSt4?=
 =?utf-8?B?SnRJaXBrcVdVUHNXT29MN1RoMmlwUjBNcE5mOU02aWl4ZXJuRGVRVENUN3c2?=
 =?utf-8?B?S3kwZkVSbzFIL3RrTThaSlpPVjZpNStpZmFQTHR0aEhIK2lrWjB0UmFVMEJp?=
 =?utf-8?B?Q1RRTmxXLzRjOHRTa2kxcFVmNytKcThpRTVqY2ZsMGIyeXlhSTQ1SVIzUWR0?=
 =?utf-8?B?bU43VzZvSDdmR3hjRVYrNkthM3NibjZNOG1wZGNSbnI3dzVpMXh4cWxzaDBU?=
 =?utf-8?B?c3FOVXBWaXA4ZWtLczdHOHpKOUhEYXNaZE1zRW9DVk5TdGRITnJERmQrNWF3?=
 =?utf-8?B?QU9kaFlpb0ZZZzZENDFSK3NyaVl0QnpKSGo0YXNka2ZxMHNZNlFjU0EwMzZS?=
 =?utf-8?B?aVJSTjNubEMyRzNKUENWUWdOSDZia3ZpMHZlNGtvd2tJcmF5WVphL251Q1kw?=
 =?utf-8?B?MGIwVWtxUEhJOVQ4czhKU05JQ0FiMGhDc3k0elBUMmhCSUVHT2Y4WjVRUDRk?=
 =?utf-8?B?cnNkTk9tc2drMkFYTnh2anhpd0UrLzJueFNwQ3FFU2k5ZVhWVzNGZTJtaG5Q?=
 =?utf-8?B?Yks1WU02MVRTWmhOanpzTHFxRUdkMG1mTHl3Qk1hbUw4dzhOY3VyMFc0REU5?=
 =?utf-8?B?TTBITkIrZlRhV1JibXJhaEVEK2kyZXpTd2lTTWZMWWNDUHdHOGY4d25uaHd6?=
 =?utf-8?B?ZEpYQUF2RGU2bjBJMDdiVlFwaXdtMnJ6Uy9XSndVclJUM0FsRE1uRjhRRzJr?=
 =?utf-8?B?UWVVT3U3V1p2N1AyS2FlN1VvaDAxNS9mdmxQcEdLSy9VcGx2R1BRT2VGSUZh?=
 =?utf-8?B?VVF0V00veVhDeWVVeTd3U05QWGxUd21kMXhucjhiMUltdWphNzF6WHZ6M3ZE?=
 =?utf-8?B?TWVRK1RyUXVjUUJuN2t3RVd4WENkekx6MndQZkRmM2RuY3dyTFhhSzIwc1NZ?=
 =?utf-8?B?ZmpJMjc1K0I3aUVEaytyeTRBV1VZbWsrb0N2eXZWb0xZOGJ6UEQ2ZURnSm9h?=
 =?utf-8?B?R2R6dHdWSW83NG9OeFJRa2ozdndRaXI0UU1KZHV0VHZYVDkwMXNVb3h4ejNo?=
 =?utf-8?Q?N2s5QDcesIlEdVkHh5rnkROPPQ2zCtlOI27eZ2h?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf121ce6-f195-4c0b-1a84-08d9767e301f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:17:39.6076
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TYeIODdfiji/BEHCqU/BnUYqTQlDz5L+6Ce04X+NifI0Qlps+UWzsU3J4a/VcTee95CRwyXnb6sQRPxoRM+I8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2336

On 10.09.2021 09:52, Jan Beulich wrote:
> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>       */
>>      if ( p2m_is_foreign(pte.p2m.type) )
>>      {
>> -        mfn_t mfn = lpae_get_mfn(pte);
>> -
>>          ASSERT(mfn_valid(mfn));
>>          put_page(mfn_to_page(mfn));
>>      }
>> +
>> +#ifdef CONFIG_GRANT_TABLE
>> +    /*
>> +     * Check whether we deal with grant table page. As the grant table page
>> +     * is xen_heap page and its entry has known p2m type, detect it and mark
>> +     * the stored GFN as invalid.
>> +     */
>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>> +#endif
> 
> I take it the write done is benign for other Xen heap pages? I suppose
> this would want making very explicit, as such an assumption is easy to
> go stale by entirely unrelated changes.
> 
> I also wonder whether you really mean to include p2m_ram_ro pages here
> as well.

Actually I've meanwhile realized I should put my question here quite
differently: Aren't you effectively introducing an M2P here for Arm,
except that you artificially limit it to the Xen heap pages needed for
the grant table?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:26:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:26:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185419.334058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfQC-0005m9-If; Mon, 13 Sep 2021 06:26:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185419.334058; Mon, 13 Sep 2021 06:26:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfQC-0005m2-ES; Mon, 13 Sep 2021 06:26:28 +0000
Received: by outflank-mailman (input) for mailman id 185419;
 Mon, 13 Sep 2021 06:26:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPfQA-0005lv-To
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:26:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 85350ec0-145b-11ec-b39d-12813bfff9fa;
 Mon, 13 Sep 2021 06:26:25 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-oZOy0JpJNhOW8YRKhy-RoA-1;
 Mon, 13 Sep 2021 08:26:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5165.eurprd04.prod.outlook.com (2603:10a6:803:54::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 06:26:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:26:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0051.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Mon, 13 Sep 2021 06:26:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85350ec0-145b-11ec-b39d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631514384;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=b56+p+h/+f3Lf2jWmYx6VT7RCUyvcBike6T9rfJgClA=;
	b=k0mySYbdTxdWrzY62ThhD/xGeEzjn8wyJjP4Eb8WwGxDenahRsl+oOtUC3c3s3VKgvnb/i
	C+bvnzq3cxciwKJqiu1gq7JyqxTNStNSGy5VsYAG2oUPb4os/Q5vT8LGmfmGxhZoyGivlL
	UtqmEY1laDeaNaTUSvUG/wRDY+/4c4Q=
X-MC-Unique: oZOy0JpJNhOW8YRKhy-RoA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UEYCO+yD7YhSXLCJ/zNBkiNfk7VflGHeAUIp3/20d50loDckfHm57V6xmeEfK+UvltIwE9tsQaZb1dALdJf1PkiB+2YeIdk3RrwnbbSbmjXO2YQmA3CAKQTopog4JHEcIehLJPYdgqGWBIAOlZaefInRRXJArVQXLkeREQgxnV4Q5wE7CIwYVmFzPNSERsnm2waY8i1+y8mqU/zskO+cW4OZJwWoT42KtmSV9awK/ikAmla5xLFwysGiGeqfy6m/lMlGaIdmw+2yaOS0W/lbHTU8qKEh/YiXxzTPtJbxN/ivsIUIJCS4nHfrTKduf7KybkEf1Bq9UzYn2tX7rvssGQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=b56+p+h/+f3Lf2jWmYx6VT7RCUyvcBike6T9rfJgClA=;
 b=gV7ZxZx2OX3WkiLVjZdN/jgueFx5CwYny/V2HcXIkoWwq0+P9ev/JpZqz2mDRh/PTfTvW+02Fb12cQ0LGxJhLPysus44O8emyK8kZiYvAIPfxaTMw0LD6zfA5oYMX3kF/mjl/twUjw7njcCnxOZVFpIU/93Dd+uRdHKAiEpLyV1yGkr1c+BxDKXxDlaWcwvdLlPdBplEpvGGkW5W+eB2djS4yb5MP6DmQPBJGo3GiA9M5mfoFu6D6BYrPEfjZBCAE/fyiOOvnK4r25KRgkreZyY+4JaJ/37ITs6kyn4gTYxXBCN98LzfnqlfoT47ZnBk4VZtYSIoBOtUeDhzdK5c+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2] x86: correct asm() constraints when dealing with immediate
 selector values
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <2c04b2aa-d41f-0a2c-6045-2d37a6fac53e@suse.com>
Date: Mon, 13 Sep 2021 08:26:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0051.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5048bcd1-86d4-4b7b-c8fe-08d9767f6777
X-MS-TrafficTypeDiagnostic: VI1PR04MB5165:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5165E7556D3092F1F4CAA512B3D99@VI1PR04MB5165.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CKwtM066mlHuvjXjh8NxKVX5NjYRQECs2dl3kfCqk1TH8TIKVNvXk+7jGpHba3sCl0JV5ekrJKqxChU0IxhZZ5PnCMN37q/FbKcifWBWYeEFFr29pNowOKP8cEuPaZ9ho3IqnrREZO6YM3ghkX65o9lyfQXwDC186Hu2GvLkCkEHuk18tXOTctu67E1PxmgEPUwR9ufTICQO7tQHLSNMxRNvW1HF1HOmPIYkcGXQAupPnywRCJxGA80UmuteVdUhRjLiKlGjV89W7y/FOEgtdhmcOAy02dEWPvwwhZdhck8mpwh0OKVLcrjvwMVXq77v9yThDdqx4MLJycgnlhfbcEx+bjRGQeRSCjxAhim8QlAB0WJbBZL/QeWmHMJoKd2CH0IiiB+RLzn9Kx16DSCYrIkIjmc7cJRSI1u1JBZSqiiMrC9Tqu/RHEW59GvNTeVZJdGYreGO6/YIJ9pSeaIdeR2GmHU5pKXrXOE6IP4dStP0c3DSKZR9g3MdyiIBoqoKZ/kI/rfjmbzan5V+cGBDi8mSUAXJtGKE2oqq2JRLMgsRF1Palf8e+1V6EBWQE9j9Liasi7GC6Q/Ppd01mJPeEFMm0I90NNaex9c4fyVMDkqRwO4/C1ci1RufvXLPI1f9+nPydCIKDBFwdpPmmS3uFCxcgY1vqp2fMn28KPD7nkvxepRo/mmB/KPHKU+zZuNa13V6P18YBtMFoWGwg9QID9A8zteJ3KhP8qpAzVvxCnBmoF58V+bT6OkwaYjbB/P5uk4/dSy2D/md5AFRSdmLuw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(136003)(39860400002)(396003)(376002)(2616005)(4326008)(66946007)(956004)(83380400001)(5660300002)(6916009)(36756003)(186003)(8936002)(8676002)(38100700002)(86362001)(31686004)(26005)(6486002)(31696002)(2906002)(54906003)(66556008)(66476007)(316002)(478600001)(16576012)(14773001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?azhQV2ZxVEZXbzlHNHlEVXpydDllYStrVy9xRHBBb3J5bkRTSlpjaVlOR3Bw?=
 =?utf-8?B?ZDk3clFRaFZ3RXdVVCtqbDBpL2NTNU5nYUl4SldBR0MwdTBGbHU5Mi9nT0h5?=
 =?utf-8?B?WEVsV3lwS0tjbU1RRUtwbXJzV3RCZDlHVEhQYjNQRG96VUdTVS9tcnBNdHJH?=
 =?utf-8?B?eEN4blNjN1VYUkRId0g1eXJ5M0RMbWVjSEpHUWNra0pJOTBuTUxxTFNkZlNR?=
 =?utf-8?B?K3NZanIzNmN5QWVoL3VyZFdHcU9RMDUxQnVnZ2piMnV5NnZtd1htTkdocVhF?=
 =?utf-8?B?S0oyYjg4Z3hpRUY0RHp5aldIWkVLMzRNM005dkRKYm5EL1JFY3NCMDN4WWpT?=
 =?utf-8?B?M0tHM09WQWJVVDJ3ZkVGSHU3RzlTVVQ2alo0dDBpSW1PWnRjMm84NkFWTytQ?=
 =?utf-8?B?K3RicU81eEQrVjlRWDZvNEhzNjVOVmk5cGZNeW5oRm9nLzVreHYwVDZGNWtS?=
 =?utf-8?B?ZVVZN090TGVEN08rZnFVTjV3SVlyUDc5T1YrOXpKSjcvVzBuSVBFY3p6U0hs?=
 =?utf-8?B?ZDBwT21BOFpHZ3Q1K21IanlkYm04REtJbHdMNTBHNlJmWlZNN3pUaFVLOUxT?=
 =?utf-8?B?bkg1Q3N4VmhXS2NUOGJ3ditIZWxlQUFUbXRuQmJPRDB5YjBBdEc5SzFDRlBE?=
 =?utf-8?B?cDAxZEc3cGd3ZG9vcnJMTWxIZ3ZVU2ZDdExDbXN2M0xvMmxRaExjcjhUeWdK?=
 =?utf-8?B?U2hNMVBwQWVLQittZTcrR1ZmZTRhZlJQTXZWNjc0MzNCUmRlTjNsb0dTR0t3?=
 =?utf-8?B?Nkcrem40UTBiY2FTVDgrVFdObm9tYmt3cVRjSHVBLzhwUFE0Slh0NWFnREs5?=
 =?utf-8?B?eDEvNkQwaTRBdHNPc3NtcjRXL3RvNHJ0U3ZCSTJzcnRTd0xPZUNvaG0wZjVC?=
 =?utf-8?B?YVMwYitKcWJVU3lkYllTaGhxT0RyMU42ZjZSQkVZVUpSaHFJdEs0aXkybUJO?=
 =?utf-8?B?TWRVMW1neXZ1dDBEdkdVbzBOUThoWlJkZm5jNWVnMFBGUllQQ1p4cW5sbGNl?=
 =?utf-8?B?bzFhZHpiSW1QdkJybmpkdFlSbHBoOGdWQkU0TmZxQWM0VDdkQ25ML0YrN09z?=
 =?utf-8?B?eHdqME53b0pZNUQ2djdWQWQxUmVYTzNPd2xEUHg5YlhoT2tkQXFrMW1UY254?=
 =?utf-8?B?MFFwdHQ0dmg2ZHNaQmp4NVRMZHpxRHhQcTVxQjJ3elVrNFF4UUtLajY0K3dF?=
 =?utf-8?B?ZTNQSUV4MzNvVDdEdDhCQnFsa3RSc25BYVE4bmhFbHVwNHNuL1lRSS94amxE?=
 =?utf-8?B?c0JKbDBMZXhCT2FmQTVDbE1hK1h2TzJxM2pwS1JBblh3Ymh5YWR4S01LWlZj?=
 =?utf-8?B?QnpyUDhwTHpPREJmS2lOdWlLNUk4bTcrNUlUNHZuN1puNCtneU54TFNjd052?=
 =?utf-8?B?a0hqdmZKZHZiMjJaOWh2TFZBVjhTaXN5UDVlUTVtREkyMG1HendJL2NZd1VW?=
 =?utf-8?B?YXp5OU1OTVI2dS9QS2xUdW1HSGUrRE50aVIzVm9oYXNBTWk4cHJoYUdzamlh?=
 =?utf-8?B?b3V5TUsvTWgyNExmbCtMTno0SW8zR21TenpjYk5GRWFMVnl1Q3ppRUcwRjRl?=
 =?utf-8?B?ejBQK3JseUxsSFdNN2tVQllFVzJVNTU2SWU4MndrTElURU1kL1AzY0ZIYldl?=
 =?utf-8?B?TFE5bkxHUlhDQ3FPcVZUYzBmZlAzc21qdWIwQnczd3Q2Y2pESmRuR0Zkd0dm?=
 =?utf-8?B?cVRzdFlsVVpmYWdUOW9XSWVQZEVxazFIbVJlSU4rSzgvTXkrOHd3cTVieXJi?=
 =?utf-8?Q?nt6MKcadC8izx2KKQQujkvK0VNZ1yH3D/Uqy7dB?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5048bcd1-86d4-4b7b-c8fe-08d9767f6777
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:26:21.9143
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XRqV+S5+kzQp6YY0Ucq3hq4cncMNhVAEPfNuEn91TES46jzN/HF4Fo4RBuHWqICHP+LAg9cs5VWamYZIcctxgQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5165

asm() constraints need to fit both the intended insn(s) which the
respective operands are going to be used with as well as the actual kind
of value specified. "m" (alone) together with a constant, however, leads
to gcc saying

error: memory input <N> is not directly addressable

while clang complains

error: invalid lvalue in asm input for constraint 'm'

And rightly so - in order to access a memory operand, an address needs
to be specified to the insn. In some cases it might be possible for a
compiler to synthesize a memory operand holding the requested constant,
but I think any solution there would have sharp edges.

If "m" alone doesn't work with constants, it is at best pointless (and
perhaps misleading or even risky - the compiler might decide to actually
pick "m" and not try very hard to find a suitable register) to specify
it alongside "r". And indeed clang does, oddly enough despite its
objection to "m" alone. Which means there the change also improves the
generated code.

While there also switch the two operand case to using named operands.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Use named operands in do_double_fault().

--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -736,7 +736,7 @@ void __init detect_zen2_null_seg_behavio
 	uint64_t base;
 
 	wrmsrl(MSR_FS_BASE, 1);
-	asm volatile ( "mov %0, %%fs" :: "rm" (0) );
+	asm volatile ( "mov %0, %%fs" :: "r" (0) );
 	rdmsrl(MSR_FS_BASE, base);
 
 	if (base == 0)
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -248,7 +248,8 @@ void do_double_fault(struct cpu_user_reg
 
     console_force_unlock();
 
-    asm ( "lsll %1, %0" : "=r" (cpu) : "rm" (PER_CPU_SELECTOR) );
+    asm ( "lsll %[sel], %[limit]" : [limit] "=r" (cpu)
+                                  : [sel] "r" (PER_CPU_SELECTOR) );
 
     /* Find information saved during fault and dump it to the console. */
     printk("*** DOUBLE FAULT ***\n");



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:27:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:27:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185426.334069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfR9-0006RV-0Q; Mon, 13 Sep 2021 06:27:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185426.334069; Mon, 13 Sep 2021 06:27:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfR8-0006RO-TV; Mon, 13 Sep 2021 06:27:26 +0000
Received: by outflank-mailman (input) for mailman id 185426;
 Mon, 13 Sep 2021 06:27:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xuNF=OD=epam.com=prvs=989052e8b7=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mPfR7-0006Qn-8c
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:27:25 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2f81c615-310f-49c2-8464-276381968571;
 Mon, 13 Sep 2021 06:27:24 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18D5YZKf022515; 
 Mon, 13 Sep 2021 06:27:21 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2057.outbound.protection.outlook.com [104.47.13.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b1urvrfes-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 Sep 2021 06:27:20 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6210.eurprd03.prod.outlook.com (2603:10a6:20b:15f::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 06:27:18 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:27:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f81c615-310f-49c2-8464-276381968571
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LRGHGXdl7NFOTWcEqeXEpZ4NG/HO+OZbRUXXV9hy6BzNYGt9/xkn6ELTvkOc5QafRjqLe6x3Icg7FqnL+klItZyZFRSbxP634eF4SGovI98BSI1FMaFMpybvfsfW5ZiMrFTVlKb+cnCOhsNswl8TRitsXQysZ4Mg4xpLIE0bHIIyhT0GCu3tPsI1qt3ulB6y1nLpWo63ljORtcuePG93L9cUUzhYOFiqrXwgP15r3oT4xH7fO30cSZokoAH4RpjyGPM1lDNTfK2+OaqOKp7G+VAdx92I6Qdllun+pClWxN8seviPgng0dmX6C6sd/vs/OCCyTRJFC5Vfe102mjpUvg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sIvjLVGHt/eEiAna3F5S2ufDXZC8QJ9KZpHECwaw8W8=;
 b=VfhVJ8dQGsAa9gB9XeI+t9u8ln+OWc6iPJl4WRkVCdDzOffq8HKMg9vCbxIGA3CVXcUHjCgJ5lIOC3t2ekoOYWnqAMyaGsz4GUx0E77dE5NCTNEQwl9abTfRIZ+KoRpZEBtMWJQAQ5f6SPjtdAy1vR9jdfli1BsGg/B/QDlOg+ijM2NWUU84UuE+BmBj1isuvVr3gzsggDlAOFFeEnjOHS1pt+dLuY55jgfk3A2WgTTOeD1Cg/pB4DpRCDHpnoFRtxWTmGVplbjfdsvNQTljoaYkz75IyTIoWxs8PFNcM9QPeGwsuY84dJprwDZVpV5NZ1KyYyPHwXOhxFovz4sxEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sIvjLVGHt/eEiAna3F5S2ufDXZC8QJ9KZpHECwaw8W8=;
 b=Xyut4IBlHm6WgO77CCS4zMnrpvLEpHpRfGqFrsQnA3BoJp6Pt+Ml0MiLASYOjTysZQsJnpzKHUc/q39rW3Wfi6xv7bmvX27W07kbrUZOjdKbBXzgOIwVOxypqD/0AfRDmmIe7UjJSrFI/lG+mL1iWHpnwgoryAdGOIyDd7svuf0kv/waY/6jR7KEX01+8vF2BCtgPdp/2tTIS+r3L6cfG5vi1CQhwbYdE+39s5i2xNtKlf0bAcHnsZtoplBR9Kn4AGD7HEJMA+TqSrwwmGO0x7ifx81LpHlN2PuErljhro87Ew/U2xDwgv2VJE5CfMJCm/4Q8R46OUzEnrGlakpJpg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien.grall.oss@gmail.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: Oleksandr Andrushchenko <andr2000@gmail.com>,
        xen-devel
	<xen-devel@lists.xenproject.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAARZwCAAFsJAIAAFA4AgAO3eIA=
Date: Mon, 13 Sep 2021 06:27:18 +0000
Message-ID: <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
 <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
In-Reply-To: 
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a108129b-5282-457d-f3e3-08d9767f8925
x-ms-traffictypediagnostic: AM0PR03MB6210:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB62106F5B34E713E8060C6339E7D99@AM0PR03MB6210.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 02ZkYNpvTNrj5iQhbhthQr9PqoxV+2M0MKdTKiMLXu5320nPDbwh0aeRDWzXrzwiAMPTT6Eab0Sc60IRg8sIbap5/Dd49XZia8EjtQvoAU5gMBD8/npS5BXFhuOIocubbCGEI4YWf2BGj0nQPB8heGaua+8armXNjR6XN6apk6ToRopzh7rJuQypwGBZmRN3QKKvsjQH3q9dJJhJzkedPo4gZJHSokcAtfrBYXzWiKQDdv2dZiZqeigNj6DjEDdjPe7Wz8WgKYYbVfnD/pwqwre7SYFoMcVRnTXAQQa7+N5REPxOJqEcZB4hid6cST8QAqLY8FTVrlu/aZfrFE4qbZsgIBplmzEow91QuDkn1C50GDAQod1f/oN/nNgAxWJrb9EkeazvkLs8tQXsT7O8ImX0znBwrL9c97YHVc04rheCM0/7Lj8EE1FRHHEe8u+oh4GYCFGsjKHy2afuAfL/b7bHljoSTmbRuP5AuvxXz8CmfhLkRYYZMxsT3yKFqYNkoVZEnwHD/qUoVhgTqO0oTy5r+T33cETNX7ZEzzBofPU5GNV0zPaIkIkYHWEd/5EHrYc9STtUZJRqNp7bdUGD++fUqffsWxM/1Dyg9Fl6o+Qw53ot7hTKqnouJ5jlv0tUIDd0b2KT0nQgX0Cc9dr63i/cqzSDLp124aLaVDO1QH3KwM1FC6CDOmNW2udGdhhQjaR4o69xGRTd9cF5GYMTtyyz4C3dc7g9NtC2b6OwVVwHJEbzx1O+xQLNP83OIc+k
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(366004)(39860400002)(376002)(396003)(478600001)(26005)(6486002)(64756008)(91956017)(122000001)(6512007)(38100700002)(110136005)(66446008)(31686004)(6506007)(186003)(66556008)(66476007)(8676002)(86362001)(76116006)(2616005)(5660300002)(83380400001)(36756003)(54906003)(31696002)(8936002)(38070700005)(66946007)(53546011)(55236004)(316002)(4326008)(2906002)(71200400001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ejVzcjMyTC9NKzB3akVNT1VHdmczVWRwdmZCS2xoR3BNODRGR1J5R01Mdy8z?=
 =?utf-8?B?TzdRQmFQVVVwWDJaeTR0dnFJTDVDZHljYzZ2bldsTWs2Zy9NWDh3MVE5TnhY?=
 =?utf-8?B?M2QrNGJaUDNCTEUrTmltL1FseG5mRFIyeS9JRU1LR0ZabkxnRk45UjZrb3p2?=
 =?utf-8?B?ZmE0NVo4ZjlpYjdjTFNBTmRmWXhyK3VjN002MWRxN1hrNXFyOEtnZHpUbzVY?=
 =?utf-8?B?RHdTRHJGTnowaXczcEdUa0YzR1A0YmZ5d1ZKOGFNZXpYZGQ1dGFrdHV0dm01?=
 =?utf-8?B?bUxYRzZITnkyM3lmYTN6R2VNMVhlTVROVXhaNmZVcitXSUlvUklGZm1IYUdV?=
 =?utf-8?B?bmVzZFV4aXFnRDNFZkVSRWYvZWpGUXBtZDRNQzZuMDFjNlNjV2hTS05WZis5?=
 =?utf-8?B?YktPaTZWVHNRc2dGaWhUbngrL01hRkgzUzlZdDNuM1VuRGp1bitBWUlNbnh0?=
 =?utf-8?B?ZitIWStLVXpGbjVNRVZZaFl5RUNQZ3N1Z3Z6RERnT3ZjR1RiTnRXZUVUeUJl?=
 =?utf-8?B?d3p1SmYxbGx1bUxBbTFHa3o5RUZaaG1DejhYQ29ZT05Weno2TGw4U003bGln?=
 =?utf-8?B?eTE2SlZiOU8wYURyZ2dndTltcUJ2UXIxcVNMYTN6ajVrWXgrSkxMQTQySzRV?=
 =?utf-8?B?YjRSVWJBQmUzeTNYa2pUNUpLa1VxOHJkb3ZEZUQ2YnVSQWN6UkRORjQwNWt3?=
 =?utf-8?B?ZzdudXFYZXJkRzRDOE5jMW1Gb3hGU1B0NXZzS3BqQVZiSGRCWUZuaU5XcTBD?=
 =?utf-8?B?d2ZGUS9LVmRibDdUL0NEQWJDbnh0ODdzeFQ3V2JPUDh5RXVGaG9NRllSSlYv?=
 =?utf-8?B?MWhVVGZNR3lOc0NLeXgxWXdyaW1PUitTNTh4ZHVCZVNNMUExazUxbEF1ZHpK?=
 =?utf-8?B?cWFCeGxHVDJrOWhlOUY0ZGxMZks1QS9FUEVnY1NIQm90WWVEcnI4U0U4c25t?=
 =?utf-8?B?OUk4UkpaV25pR2hwUlQxV1BkRFA0eTltazJyQ3VLV09yQVBUdC9vdmhkRmxj?=
 =?utf-8?B?Lzl6TlY3N0I3ZUlTS3R0cWlSbW9jVjZZdTU4dkNtVHZCZFZNS1c2LzJZaXpO?=
 =?utf-8?B?VUVLakt0TWdrN3RNYWZQditmQ2N2QmxUVTZhdkp4cGJQM2djVDFNaEJTYnUw?=
 =?utf-8?B?SmgzUFdKQkVXWkhaWGhLbkExZVd5VmpCSitheTBERkc0ZWpzMzF2SFFQQis5?=
 =?utf-8?B?amxIQXprNmJ2T3NRWk1EYm5DS0paMVZnVGlBVUVKb2pDMVU1TXhxQzVXZGNn?=
 =?utf-8?B?TjhVZWVHMHZEM0hzdExhL1JjcnlmUGJDWVJxMFZlK1ErQWtNNGRvNzNBRjNa?=
 =?utf-8?B?alpqZHJ3SWVsN3JScHVxS2JGZ3Zlb0ZVNXMrMnQveDVGeWx1c1lpMDJaUzBq?=
 =?utf-8?B?SldNSElvajQ0Y0JPK21wOFI0QXFEa2tmcDh3ME0zcy9QWXlJS3ZDOUFVMWZR?=
 =?utf-8?B?c3FhWVJaa3BtZXJ0TEV2RTE0Tis1a3duK05UVGxSTlM2MjFUMTJqdHo5V1lM?=
 =?utf-8?B?OVV2NktQeTF2TExITmdSdk1vVkpRRFNIQ2grL1VuUTFIRUlwTkJzUlBsM05l?=
 =?utf-8?B?aGg5bWd6bEhZV3J6VzBRUktoRlZZYldtMFZZY3ZqQTVVdDlHYThNS29LZTBv?=
 =?utf-8?B?SFEzdXhmdWY5ZTVsa1BTaTRBR3NNcGI5VTNZdURocy9rd1IxQTlQeFJvNVBm?=
 =?utf-8?B?OXd4a0pBYXJvc0k0ZjVyYjlUemtBaFZWSFVsZUxJU3VkbFJWRmltM0ppL1Zo?=
 =?utf-8?B?elhEa1dZbVZudVNlankvcmxiWXg5MEFwZTJYeVJVTllTek8yUVNqZEJnRTU5?=
 =?utf-8?B?VTEzOUhRTGMvQ3JzOXMzdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <917E9CBF014B534A80B8BB9984FE9002@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a108129b-5282-457d-f3e3-08d9767f8925
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Sep 2021 06:27:18.1713
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: AFGl8ohwxeshEYlf/Bsls9nm9oKTiIgfcFcBv1XsoN4+j3NvrkH+RTxoWgjZjYaE6pjdiEf5uRCMzMHUuooUiDQUvFmUHclhPV5zTNjSbWRCJRQfiwVqnln6M2QBAOwQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6210
X-Proofpoint-GUID: TctLKG0ppApGRgnf-RLptmJwnhzriVt9
X-Proofpoint-ORIG-GUID: TctLKG0ppApGRgnf-RLptmJwnhzriVt9
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_03,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 impostorscore=0 priorityscore=1501 mlxscore=0 clxscore=1011 spamscore=0
 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109130041

DQpPbiAxMS4wOS4yMSAwMDo0MSwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPg0KPg0KPiBPbiBGcmks
IDEwIFNlcCAyMDIxLCAyMTozMCBTdGVmYW5vIFN0YWJlbGxpbmksIDxzc3RhYmVsbGluaUBrZXJu
ZWwub3JnIDxtYWlsdG86c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4+IHdyb3RlOg0KPg0KPiAgICAg
T24gRnJpLCAxMCBTZXAgMjAyMSwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPiAgICAgPiBPbiAxMC8w
OS8yMDIxIDE1OjAxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4gICAgID4gPiBP
biAxMC4wOS4yMSAxNjoxOCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPiAgICAgPiA+ID4gT24gMTAv
MDkvMjAyMSAxMzozNywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+ICAgICA+ID4g
PiA+IEhpLCBKdWxpZW4hDQo+ICAgICA+ID4gPg0KPiAgICAgPiA+ID4gSGkgT2xla3NhbmRyLA0K
PiAgICAgPiA+ID4NCj4gICAgID4gPiA+ID4gT24gMDkuMDkuMjEgMjA6NTgsIEp1bGllbiBHcmFs
bCB3cm90ZToNCj4gICAgID4gPiA+ID4gPiBPbiAwMy8wOS8yMDIxIDA5OjMzLCBPbGVrc2FuZHIg
QW5kcnVzaGNoZW5rbyB3cm90ZToNCj4gICAgID4gPiA+ID4gPiA+IEZyb206IE9sZWtzYW5kciBB
bmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbSA8bWFpbHRvOm9s
ZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPj4NCj4gICAgID4gPiA+ID4gPiA+DQo+ICAg
ICA+ID4gPiA+ID4gPiBIb3N0IGJyaWRnZSBjb250cm9sbGVyJ3MgRUNBTSBzcGFjZSBpcyBtYXBw
ZWQgaW50byBEb21haW4tMCdzIHAybSwNCj4gICAgID4gPiA+ID4gPiA+IHRodXMgaXQgaXMgbm90
IHBvc3NpYmxlIHRvIHRyYXAgdGhlIHNhbWUgZm9yIHZQQ0kgdmlhIE1NSU8gaGFuZGxlcnMuDQo+
ICAgICA+ID4gPiA+ID4gPiBGb3IgdGhpcyB0byB3b3JrIHdlIG5lZWQgdG8gbm90IG1hcCB0aG9z
ZSB3aGlsZSBjb25zdHJ1Y3RpbmcgdGhlDQo+ICAgICA+ID4gPiA+ID4gPiBkb21haW4uDQo+ICAg
ICA+ID4gPiA+ID4gPg0KPiAgICAgPiA+ID4gPiA+ID4gTm90ZSwgdGhhdCBkdXJpbmcgRG9tYWlu
LTAgY3JlYXRpb24gdGhlcmUgaXMgbm8gcGNpX2RldiB5ZXQNCj4gICAgID4gPiA+ID4gPiA+IGFs
bG9jYXRlZCBmb3INCj4gICAgID4gPiA+ID4gPiA+IGhvc3QgYnJpZGdlcywgdGh1cyB3ZSBjYW5u
b3QgbWF0Y2ggUENJIGhvc3QgYW5kIGl0cyBhc3NvY2lhdGVkDQo+ICAgICA+ID4gPiA+ID4gPiBi
cmlkZ2UgYnkgU0JERi4gVXNlIGR0X2RldmljZV9ub2RlIGZpZWxkIGZvciBjaGVja3MgaW5zdGVh
ZC4NCj4gICAgID4gPiA+ID4gPiA+DQo+ICAgICA+ID4gPiA+ID4gPiBTaWduZWQtb2ZmLWJ5OiBP
bGVrc2FuZHIgQW5kcnVzaGNoZW5rbw0KPiAgICAgPiA+ID4gPiA+ID4gPG9sZWtzYW5kcl9hbmRy
dXNoY2hlbmtvQGVwYW0uY29tIDxtYWlsdG86b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5j
b20+Pg0KPiAgICAgPiA+ID4gPiA+ID4gLS0tDQo+ICAgICA+ID4gPiA+ID4gPiB4ZW4vYXJjaC9h
cm0vZG9tYWluX2J1aWxkLmPCoMKgwqDCoMKgwqDCoCB8wqAgMyArKysNCj4gICAgID4gPiA+ID4g
PiA+IHhlbi9hcmNoL2FybS9wY2kvZWNhbS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDE3ICsr
KysrKysrKysrKysrKysrDQo+ICAgICA+ID4gPiA+ID4gPiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1o
b3N0LWNvbW1vbi5jIHwgMjIgKysrKysrKysrKysrKysrKysrKysrKw0KPiAgICAgPiA+ID4gPiA+
ID4geGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaMKgwqDCoMKgwqDCoMKgwqDCoCB8IDEyICsrKysr
KysrKysrKw0KPiAgICAgPiA+ID4gPiA+ID7CoCDCoMKgIDQgZmlsZXMgY2hhbmdlZCwgNTQgaW5z
ZXJ0aW9ucygrKQ0KPiAgICAgPiA+ID4gPiA+ID4NCj4gICAgID4gPiA+ID4gPiA+IGRpZmYgLS1n
aXQgYS94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMNCj4gICAgID4gPiA+ID4gPiA+IGIveGVu
L2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jDQo+ICAgICA+ID4gPiA+ID4gPiBpbmRleCBkYTQyN2Yz
OTk3MTEuLjc2ZjViNTEzMjgwYyAxMDA2NDQNCj4gICAgID4gPiA+ID4gPiA+IC0tLSBhL3hlbi9h
cmNoL2FybS9kb21haW5fYnVpbGQuYw0KPiAgICAgPiA+ID4gPiA+ID4gKysrIGIveGVuL2FyY2gv
YXJtL2RvbWFpbl9idWlsZC5jDQo+ICAgICA+ID4gPiA+ID4gPiBAQCAtMTI1Nyw2ICsxMjU3LDkg
QEAgc3RhdGljIGludCBfX2luaXQgbWFwX3JhbmdlX3RvX2RvbWFpbihjb25zdA0KPiAgICAgPiA+
ID4gPiA+ID4gc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+ICAgICA+ID4gPiA+ID4gPsKg
IMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0NCj4gICAgID4gPiA+ID4gPiA+wqAgwqDCoMKgwqDCoMKg
IH0NCj4gICAgID4gPiA+ID4gPiA+wqAgwqDCoCArwqDCoMKgIGlmICggbmVlZF9tYXBwaW5nICYm
IChkZXZpY2VfZ2V0X2NsYXNzKGRldikgPT0gREVWSUNFX1BDSSkNCj4gICAgID4gPiA+ID4gPiA+
ICkgPiArwqDCoMKgwqDCoMKgwqAgbmVlZF9tYXBwaW5nID0gcGNpX2hvc3RfYnJpZGdlX25lZWRf
cDJtX21hcHBpbmcoZCwgZGV2LA0KPiAgICAgPiA+ID4gPiA+IGFkZHIsIGxlbik7DQo+ICAgICA+
ID4gPiA+ID4NCj4gICAgID4gPiA+ID4gPiBBRkFJQ1QsIHdpdGggZGV2aWNlX2dldF9jbGFzcyhk
ZXYpLCB5b3Uga25vdyB3aGV0aGVyIHRoZSBob3N0YnJpZGdlIGlzDQo+ICAgICA+ID4gPiA+ID4g
dXNlZCBieSBYZW4uIFRoZXJlZm9yZSwgSSB3b3VsZCBleHBlY3QgdGhhdCB3ZSBkb24ndCB3YW50
IHRvIG1hcCBhbGwNCj4gICAgID4gPiA+ID4gPiB0aGUgcmVnaW9ucyBvZiB0aGUgaG9zdGJyaWRn
ZXMgaW4gZG9tMCAoaW5jbHVkaW5nIHRoZSBCQVJzKS4NCj4gICAgID4gPiA+ID4gPg0KPiAgICAg
PiA+ID4gPiA+IENhbiB5b3UgY2xhcmlmeSBpdD8NCj4gICAgID4gPiA+ID4gV2Ugb25seSB3YW50
IHRvIHRyYXAgRUNBTSwgbm90IE1NSU9zIGFuZCBhbnkgb3RoZXIgbWVtb3J5IHJlZ2lvbnMgYXMg
dGhlDQo+ICAgICA+ID4gPiA+IGJyaWRnZSBpcw0KPiAgICAgPiA+ID4gPg0KPiAgICAgPiA+ID4g
PiBpbml0aWFsaXplZCBhbmQgdXNlZCBieSBEb21haW4tMCBjb21wbGV0ZWx5Lg0KPiAgICAgPiA+
ID4NCj4gICAgID4gPiA+IFdoYXQgZG8geW91IG1lYW4gYnkgInVzZWQgYnkgRG9tYWluLTAgY29t
cGxldGVseSI/IFRoZSBob3N0YnJpZGdlIGlzIG93bmVkDQo+ICAgICA+ID4gPiBieSBYZW4gc28g
SSBkb24ndCB0aGluayB3ZSBjYW4gbGV0IGRvbTAgYWNjZXNzIGFueSBNTUlPIHJlZ2lvbnMgYnkN
Cj4gICAgID4gPiA+IGRlZmF1bHQuDQo+ICAgICA+ID4NCj4gICAgID4gPiBOb3cgaXQncyBteSB0
aW1lIHRvIGFzayB3aHkgZG8geW91IHRoaW5rIFhlbiBvd25zIHRoZSBicmlkZ2U/DQo+ICAgICA+
ID4NCj4gICAgID4gPiBBbGwgdGhlIGJyaWRnZXMgYXJlIHBhc3NlZCB0byBEb21haW4tMCwgYXJl
IGZ1bGx5IHZpc2libGUgdG8gRG9tYWluLTAsDQo+ICAgICA+ID4gaW5pdGlhbGl6ZWQgaW4gRG9t
YWluLTAuDQo+ICAgICA+ID4NCj4gICAgID4gPiBFbnVtZXJhdGlvbiBldGMuIGlzIGRvbmUgaW4g
RG9tYWluLTAuIFNvIGhvdyBjb21lcyB0aGF0IFhlbiBvd25zIHRoZSBicmlkZ2U/DQo+ICAgICA+
ID4gSW4gd2hhdCB3YXkgaXQgZG9lcz8NCj4gICAgID4gPg0KPiAgICAgPiA+IFhlbiBqdXN0IGFj
Y2Vzc2VzIHRoZSBFQ0FNIHdoZW4gaXQgbmVlZHMgaXQsIGJ1dCB0aGF0J3MgaXQuIFhlbiB0cmFw
cyBFQ0FNDQo+ICAgICA+ID4gYWNjZXNzIC0gdGhhdCBpcyB0cnVlLg0KPiAgICAgPiA+DQo+ICAg
ICA+ID4gQnV0IGl0IGluIG5vIHdheSB1c2VzIE1NSU9zIGV0Yy4gb2YgdGhlIGJyaWRnZSAtIHRo
b3NlIHVuZGVyIGRpcmVjdCBjb250cm9sDQo+ICAgICA+ID4gb2YgRG9tYWluLTANCj4gICAgID4N
Cj4gICAgID4gU28gSSBsb29rZWQgb24gdGhlIHNuaXBwZWQgb2YgdGhlIGRlc2lnbiBkb2N1bWVu
dCB5b3UgcG9zdGVkLiBJIHRoaW5rIHlvdSBhcmUNCj4gICAgID4gaW5zdGVhZCByZWZlcnJpbmcg
dG8gYSBkaWZmZXJlbnQgcGFydDoNCj4gICAgID4NCj4gICAgID4gIiBQQ0ktUENJZSBlbnVtZXJh
dGlvbiBpcyBhIHByb2Nlc3Mgb2YgZGV0ZWN0aW5nIGRldmljZXMgY29ubmVjdGVkIHRvIGl0cw0K
PiAgICAgPiBob3N0Lg0KPiAgICAgPiBJdCBpcyB0aGUgcmVzcG9uc2liaWxpdHkgb2YgdGhlIGhh
cmR3YXJlIGRvbWFpbiBvciBib290IGZpcm13YXJlIHRvIGRvIHRoZSBQQ0kNCj4gICAgID4gZW51
bWVyYXRpb24gYW5kIGNvbmZpZ3VyZSB0aGUgQkFSLCBQQ0kgY2FwYWJpbGl0aWVzLCBhbmQgTVNJ
L01TSS1YDQo+ICAgICA+IGNvbmZpZ3VyYXRpb24uIg0KPiAgICAgPg0KPiAgICAgPiBCdXQgSSBz
dGlsbCBkb24ndCBzZWUgd2h5IGl0IG1lYW5zIHdlIGhhdmUgdG8gbWFwIHRoZSBNTUlPcyByaWdo
dCBub3cuIERvbTANCj4gICAgID4gc2hvdWxkIG5vdCBuZWVkIHRvIGFjY2VzcyB0aGUgTU1JT3Mg
KGFzaWRlIHRoZSBob3N0YnJpZGdlIHJlZ2lzdGVycykgdW50aWwgdGhlDQo+ICAgICA+IEJBUnMg
YXJlIGNvbmZpZ3VyZWQuDQo+DQo+ICAgICBUaGlzIGlzIHRydWUgZXNwZWNpYWxseSB3aGVuIHdl
IGFyZSBnb2luZyB0byBhc3NpZ24gYSBzcGVjaWZpYyBQQ0llDQo+ICAgICBkZXZpY2UgdG8gYSBE
b21VLiBJbiB0aGF0IGNhc2UsIHRoZSByZWxhdGVkIE1NSU8gcmVnaW9ucyBhcmUgZ29pbmcgdG8g
YmUNCj4gICAgIG1hcHBlZCB0byB0aGUgRG9tVSBhbmQgaXQgd291bGQgYmUgYSBiYWQgaWRlYSB0
byBhbHNvIGtlZXAgdGhlbSBtYXBwZWQNCj4gICAgIGluIERvbTAuIE9uY2Ugd2UgZG8gUENJZSBh
c3NpZ25tZW50LCB0aGUgTU1JTyByZWdpb24gb2YgdGhlIFBDSWUgZGV2aWNlDQo+ICAgICBiZWlu
ZyBhc3NpZ25lZCBzaG91bGQgb25seSBiZSBtYXBwZWQgaW4gdGhlIERvbVUsIHJpZ2h0Pw0KPg0K
Pg0KPiBUaGF0J3MgYWN0dWFsbHkgYSBnb29kIHBvaW50LiBUaGlzIGlzIGEgcmVjaXBlIGZvciBk
aXNhc3RlciBiZWNhdXNlIGRvbTAgYW5kIHRoZSBkb21VIG1heSBtYXAgdGhlIEJBUnMgd2l0aCBj
b25mbGljdGluZyBjYWNoaW5nIGF0dHJpYnV0ZXMuDQo+DQo+IFNvIHdlIG91Z2h0IHRvIHVubWFw
IHRoZSBCQVIgZnJvbSBkb20wLiBXaGVuIHRoZSBkZXZpY2UgaXMgYXNzaWduZWQgdG8gdGhlIGRv
bVUNCjEuIFllcywgY3VycmVudGx5IHdlIG1hcCBNTUlPcyB0byBEb20wIGZyb20gdGhlIGJlZ2lu
bmluZyAodGhlIHdob2xlIGFwZXJ0dXJlIGFjdHVhbGx5KQ0KDQoyLiBXaGVuIGEgUENJZSBkZXZp
Y2UgYXNzaWduZWQgdG8gYSBEb21VIHdlIGRvIHVubWFwIHRoZSByZWxldmFudCBNTUlPcw0KDQpm
cm9tIERvbTAgYW5kIG1hcCB0aGVtIHRvIERvbVUNCg0KDQo+DQo+ICAgICBJZiBzbywgaXQgd291
bGQgYmUgYmV0dGVyIGlmIHRoZSBNTUlPIHJlZ2lvbiBpbiBxdWVzdGlvbiB3YXMgbmV2ZXINCj4g
ICAgIG1hcHBlZCBpbnRvIERvbTAgYXQgYWxsIHJhdGhlciBoYXZpbmcgdG8gdW5tYXAgaXQuDQo+
DQpPaywgSSdsbCBkbyB0aGF0DQo+DQo+DQo+ICAgICBXaXRoIHRoZSBjdXJyZW50IGFwcHJvYWNo
LCBnaXZlbiB0aGF0IHRoZSBlbnRpcmUgUENJZSBhcGVydHVyZSBpcyBtYXBwZWQNCj4gICAgIHRv
IERvbTAgc2luY2UgYm9vdCwgd2Ugd291bGQgaGF2ZSB0byBpZGVudGlmeSB0aGUgcmVsZXZhbnQg
c3Vic2V0IHJlZ2lvbg0KPiAgICAgYW5kIHVubWFwIGl0IGZyb20gRG9tMCB3aGVuIGRvaW5nIGFz
c2lnbm1lbnQuDQo+DQpJdCBpcyBhbHJlYWR5IGluIHZQQ0kgY29kZSAod2l0aCBub24taWRlbnRp
dHkgbWFwcGluZ3MgaW4gdGhlIFBDSSBkZXZpY2VzIHBhc3N0aHJvdWdoIG9uIEFybSwgcGFydCAz
KQ==


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:39:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:39:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185433.334080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfcm-000883-59; Mon, 13 Sep 2021 06:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185433.334080; Mon, 13 Sep 2021 06:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfcm-00087w-0q; Mon, 13 Sep 2021 06:39:28 +0000
Received: by outflank-mailman (input) for mailman id 185433;
 Mon, 13 Sep 2021 06:39:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPfck-00087n-G7
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:39:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 07cef170-282f-458d-9049-8c3df5539f5d;
 Mon, 13 Sep 2021 06:39:25 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-KKygg5OBP16PqWmfh_x4bw-1; Mon, 13 Sep 2021 08:39:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Mon, 13 Sep
 2021 06:39:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:39:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.8 via Frontend Transport; Mon, 13 Sep 2021 06:39:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07cef170-282f-458d-9049-8c3df5539f5d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631515164;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=hZV80eLlnUbQz40OrBm/kaIbk08Pq01QuLeBX7Og0zQ=;
	b=JQt/T+zqOM++aIrCgGPXlD26O4puli3l7o12k+dplGGlpV6rWiRca350YRyPNawmxsiwvt
	4hdm+5PLJjLdDijLPMKb+IE9f74xg+PqYpylwMwheWbzWip1cQ3kIqTSUiKzBxbLoigpbs
	VFiSFtPWD+zdlVhkuIPwXAtcNneRUZI=
X-MC-Unique: KKygg5OBP16PqWmfh_x4bw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TPNt8yHXVBq4HbSLRFk1oeJ9j4UmkOPB3uWxeUcXdhdb2wPG3KH+hgCXNP11hItbnOf1FncMEZN3zS3rJ80iQ4aR+YnGXYQ+odBfdbCZfdEEg9hndlGXdoyYTL8qmkjLGC5jut/QWAN91FtuxuzAgsqg5rtCbXMlU/ryevOZJaaKHqDFL53LAEWJWKXguuP075BCM39cN2KrhvB7HpSCoCNwARo7OxdkzKs3m8MThEtPhGLmsZL6naS/pbWNzgTm+hXzTd9TEYNczuHjwn18q2kiIjaAhgbwiGALqF2amu997Na+hjBoQM29GwfCz97BUYXsTQn4wEqEQAm/c99k9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hZV80eLlnUbQz40OrBm/kaIbk08Pq01QuLeBX7Og0zQ=;
 b=BxpfFzibcKn/wpVzzBdNF4JJiPQjMn1oORWoBW5GfIGJeSS+qjK81eqxac6+M6arBTT1mFDbc72sPfug+0oDuxXRv7yzaGId0PLnsWaRI0fLOT607FNlqpPL76KbkskwrKPxDga+OiTIQjT2bpqrUpujipVmubEfeDpfx2rp83XTaYIN0BamoU/b1HZgqHCqa1z+GPBUqGWCLm14v522FhK5nzz+Joz2Tood5xTxekjl8ddCvg2TLPXvxE3azNZ7dXivLtVZyYrtWH+ocEapCLDkfaKXnKUDGXttkjTfLvyYYXxg6kO9DBuA+eRURjebpnouLJaSXCOcLRu+rGdfFA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/2] grant table and add-to-physmap adjustments on top of XSAs
 379 and 384
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Date: Mon, 13 Sep 2021 08:39:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0678c548-5aa7-4b58-5b8b-08d9768137db
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71520E576A677BE8D56F0F6CB3D99@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EehZm2qFwAwNF5qZYKjJmGFID65/sona8eM6CTeMwXnPXabbfgCjj5hM8GV6VzaZFUhxH7G2g+S+lXEqE7MwxqY7QHOyeQCh6GiJ13UgIDAh48GME+8AxX/bMPL4jQNtd5aqPey/c2ZoHRUWTieb/PoPamYwB3va9sPQCn4Z/6XF+3lDFSFys+7YPONiD5FV4xkVu8MIxoWmWUGRJa6rtdOSLlcIMPSuNyckvPhevj0zJ8QiopqsdwdVaNrL1qAKclkrktTZ7Ivjdd6DlabOvtgN86D5AqKGxLrOP2XCRnH7yTZ6OKxoIGjaq939p9Q4AoKh4C6Yd1MdYHaXd0Km9N4Er0yc8N5xnENwm5qjs8D3Ku3XCUZ+Ton5S27pX4chMNPdLQBPNk7ynCWxxeZ+oS3kPNLNtMOs7gUIIzaQqysy+QOjWxTvSLW5CX+Ch4++rF/peHMdNyOmX0dZKKHYW4fmU1/SQbNuvNslg5kLxna5rR+mpVJhN9Ty0OorEZXBrQw4fhLZXlHnEi/7dMWGb217aJPT1ZqbF6yM9FPI9gmaooUm4Utv2jU8zN+LaS32Rm1l6URQTZgOR2FlSeA+/b/hVQ9mP1A97Yd5nGnsbmMss0U3nAb2DlVsJcqTciOUR2wuSO1w8lSkkt31vNMsb9nb2WULv/oMm2LXfpq3ieRlIyIJ6fc0yQgwyZbIMbdxSogvCP6vBA2S7LSJThdw4K5xGbmukMJnjS683b3LK8z3BP0q7id77FCG3V/w4zaI
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(366004)(39860400002)(136003)(346002)(2906002)(66556008)(36756003)(66476007)(38100700002)(8936002)(31686004)(478600001)(86362001)(66946007)(316002)(6486002)(26005)(5660300002)(16576012)(4326008)(186003)(31696002)(956004)(54906003)(8676002)(2616005)(4744005)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SEc2WFpkazV5ZVI1NTkrWVJ6bnI1cmJWZlpyY2M0anBJUjBXbC9vSlM4OEd4?=
 =?utf-8?B?RFRkSVBCdS9BSTJndTlMV1h1NzlnTW13QlU4eU1iVGptQVVjY3hXUDFTbXdZ?=
 =?utf-8?B?cXhiZ2kvRDd4Z1RxMUFadERaenRzNHRham93RXJieGRRRHBPRlltQkVDb2th?=
 =?utf-8?B?cEl0Ykc1TS9abnM4NUUwSjBSMHBhNU5KR2N3dXcybmdxVGVwSUZpNDJyMHk4?=
 =?utf-8?B?UFVoQVlRVENtNUpOeXdKMGtvN1BmdnFCdnAxZGlkcHZLcmRmMTRiOXo4NFZH?=
 =?utf-8?B?WkNRRFpiaG14Q2tUbHVBS0pNeUdPbHNjRXJ1a2VTbXlydmhGMmNnNzFNVmhi?=
 =?utf-8?B?RFdndnhCYkJHM0FCbkRUL3ByNDZ6UERpenZtd2NndmIzdVhuRWJNT0hsRGth?=
 =?utf-8?B?YjlKYXVJNXRJaXRqZVlzd0pkVlhCdmxlR1dDRHJYc0VydktiRit1UHRCME1I?=
 =?utf-8?B?ZCtWYm01T1R3WHV3WE9RMjlZTmJESlZ0UDB3dC9HaFl0dVZxeklYZzBNVGg5?=
 =?utf-8?B?YVk0QlBWRWRkaTdveUhNc01iei9wL1pIWWV5dzlaTmR3YVRDTjVYQjFqSVhT?=
 =?utf-8?B?dy9Xcy81ZkdWdXNKSURvNHl6aERGM3NZZGJWTC9lTmIwUDQ3REI4OUw0Yy9U?=
 =?utf-8?B?LzRXbms2YWtJU2NWSExMdk9FMmkzVzNoMXIxSkpDdksxKy9FUTN1NmQxNDdE?=
 =?utf-8?B?K2ZqbGQ1YkxIeFVUbEh0cVR5L0hJaTFYQUo1Tk9wdmUwNk1VVm1kK0tLS01z?=
 =?utf-8?B?dmFiK3VxQUhzRDI3NmdWZjZzZS91azFWUnBTUU8vTFZ4UFo3T002a1RNYTRQ?=
 =?utf-8?B?a3BDSHE5Um1uTnJGUEZMai9yQTNtNjUyQVVYbU9sdElKOS9mSmpZS3VURm9H?=
 =?utf-8?B?aml4RHRQYTBjZEFIaXMrc29lK1UybHRoQ2pwZGF0eW81VGpyNFdpYjJXZU5D?=
 =?utf-8?B?VmgrRzR0WnBwdUIzY2VndnlBZlVKVmQzbkprckl6aXMvNkI0RTlhaWlGa0VN?=
 =?utf-8?B?SW5ZVUlzNE04dVkzWEp0NXhWSFdiMVdMU21KeUJ1bE9xTjljZmZwZ1IwZ1Va?=
 =?utf-8?B?endVWENSc2JPclpwSFF0cjVWRlVWUVhSQXl5YTRZVW03Y0lmRVJKUlZSUng5?=
 =?utf-8?B?VTJnK3lNN1lPMVBLeXR3WHVzVHVkN1VmV3IwWUJIalBONG9Ta2hyOHF2bW9I?=
 =?utf-8?B?enN2QnNIL1JyK1Mrb1VyVDNWWnRFMnArcmJTdDh5YjBpaWxhQzRLNWRHVFUr?=
 =?utf-8?B?c3dwa1ZnSDFjYTlMbWxIM3J6bjNqNlkyOWsrQm9PdnNBdHNZc0NyVDhQSk5t?=
 =?utf-8?B?cHRQaG9QUTFGUmNYSDkva0JYKzdKZ0d0cHNuSTRJWVVlUGRoV3JBTlN3U2k0?=
 =?utf-8?B?Rmw3VEVVSS9NbUN6dUxueEwyamVKSTNtaXlUeGRoL2dhV0hqenkvMEpDSzRD?=
 =?utf-8?B?ZzRMb05UVGhUdlZFTzNYWFU3RXYxODB5REUvcjlwZzA3dmN3SitwNXYvcDlt?=
 =?utf-8?B?TEVaZm1DeUpXNVEvYjJ6Mys2dXdNNDRWUGd1T05aSXdQWnR1UVpCMDNzSWtl?=
 =?utf-8?B?d0c3N2NBa0NQUHEzeVltMjhWQklzWmRqZWtIaENMb0NxbGF5blllTjUxeTNy?=
 =?utf-8?B?OVNXbzRPdWhPRkEwNW1Na0J5c1h3R0xINkhKYUlLSS9KbmlEN3d0ZkFzVWl0?=
 =?utf-8?B?RG1YMEQ4dWxmSWRzY1k2MkJHZzkrNVJWS3RpbzVrUW05eFpoZXErMTFaL2g4?=
 =?utf-8?Q?7QbiIgtoP3Cx3o2ncueeH3EmyulWB0fVSRyHzZa?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0678c548-5aa7-4b58-5b8b-08d9768137db
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:39:21.0776
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gEum/1StdDkmAicup/fXUlWXVOHCDuDHUB27ohlEhkFJD0sDkBAyU5GZDUkmsbghRpE4aQvMBlPSFXZ0u+DIQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

I'm prepared for the "how" aspect of the 1st patch here to end up
controversial. Since the observed quirk will imo want dealing with,
I'd appreciate any objection to the proposed change to be accompanied
by an alternative suggestion. An intention of mine was to not further
increase the number of arch hooks needed. I further realize that this
change conflicts with Oleksandr's "xen/gnttab: Store frame GFN in
struct page_info on Arm", at the very least contextually.

1: gnttab: remove guest_physmap_remove_page() call from gnttab_map_frame()
2: memory: XENMEM_add_to_physmap (almost) wrapping checks

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:41:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:41:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185440.334091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPff9-0001Bx-Ia; Mon, 13 Sep 2021 06:41:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185440.334091; Mon, 13 Sep 2021 06:41:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPff9-0001Bq-Ez; Mon, 13 Sep 2021 06:41:55 +0000
Received: by outflank-mailman (input) for mailman id 185440;
 Mon, 13 Sep 2021 06:41:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPff8-0001Bj-DQ
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:41:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ada51a09-145d-11ec-b39f-12813bfff9fa;
 Mon, 13 Sep 2021 06:41:52 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-6pMKTPYuMvO4DI8i3wO2Og-1; Mon, 13 Sep 2021 08:41:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 06:41:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:41:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0110.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 06:41:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ada51a09-145d-11ec-b39f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631515311;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hZ6Xe7LP9akiDNQLhl9Vn80ka02MrCwQlU4v/4KZpV4=;
	b=lVAj+miKOw6YsxP+Az5CUOFJivZjXeZb44a9h5CVJlZECFybfUqX6rqW26ScZ0aOkHxPvA
	WguGiWYyKWDt6S2hI4xERYjY6efTTBTwBLW20KfqjZDVCX5TcCFjzAS3nNpj4ZmMZ5VJfD
	LPR/OrTfi6/Aqgw35Zm4uq6GVjyZWf0=
X-MC-Unique: 6pMKTPYuMvO4DI8i3wO2Og-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TQduBnlLGrdJAQ7qJxOv5vgtTekongwViOJGMTHr/meOoGlSBX7R5wcnBYs5QJFt8egD87VM6ZmmZos+gbv7G4azq2VnJE2oQ4u30SaKrrmYIxqzu62QG5j0DchZ7DPdqe0HfFdPRlx8RgjcYY/51+8CBYSg8/NKZz9kixOYDeDz2+vXs0H7CSx2ooYHvBXKIYOT6URGs5pDzjmiIdobR1xE24IxGQ9omR/mgwMYOngtA0gIvh+miQ/dNREMheCyfcHch1+bAPou00PLw6yEPg5ZYDJ+CrMLRe4x0XIKRVjfFHhjC4Fo5le9FqYErfnhoZrYjchLUIVL2B/Bckt1QA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hZ6Xe7LP9akiDNQLhl9Vn80ka02MrCwQlU4v/4KZpV4=;
 b=i/ApnSjRabaBm28mIPMgsQcAiNsi6qT/g+0IcW1o/Iw410wIi/EmV0MqeAzq5Ir8xXL34gvgM5s18bbWwWS8umsQg02usXGQottQGTpY2jRvp11C4hfxcrQRHjeGuqofgIXDaGuntIsxMVE/DWP63ymXMMvailou5qyn13mTzBisLWrrKIsHeqhCWtQ4o2AvokJtGzLtZZzPb2bpYPm7P3wc2xtDOBIarzBMSj304ve/BsihciMVe6qDFEaFOMlxXBDK+qa4Xx2gPq6YDvK4+ee9QNBjFXUiYlkRqlo403JJEZnLQA5X/FYIlkQlyJP8ppJ4CTjh9eGur4K/sDVfNg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Message-ID: <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
Date: Mon, 13 Sep 2021 08:41:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0110.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fcc9d8af-6676-4208-c431-08d97681901e
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2447C7ED7562494EE01B58DEB3D99@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LnLJnzedASeS4C/cq640GpOj8nqHir8EJRPZTrg6Tq721xOqe+ngyEAN5Rmn6r0Soo40JztGTmFjQrLXfw3a6AwXvmP6nrzVigwHZ/I4QfI8WRvG9HgTWOcZEEXy6wPheqApvXO677GVgQLuFIberKMu+kKEyy5JmkYs54ca/jOfWB89J0uxNnEOs1bd/RWZslaNfSVxFF4hazV5cw0euF4whBnStqJlPNKuq4v3P+6Kx8BJcNbX7yXsuX8Piju1N6nUB6Zi0ZJXY9CGE7p4cW3ap9pOPD+oEHyqN05bjBa2gn6jxtRQQMkukbumJchhAt18HihkLPE8uj6AqNIgTE4bGwy7fvFfGnNvWz2bMu6OJYzhLX0C3heUsS9lVsGmm4/cGlvBX+QgUAgAg/0/b2zLUgVOt1LxmyPxZTnglfL2jU+4/bZeGbhkh6HCXj4uanEAHcdJ0PulmPJU0R+8hPxfZU4sJom01rtPjbb455/RD1LN8Lus9QkOqwuJON/hiZhASkG2PJxz+wA6s2TjQBiG8LMXLcsgVWia+lil061g6l++WQao08ARs7ZJd9B4guUi4yh1pElUUp+V3d3c/EBZNjoyYSWNZypGbjqzeljEXMgTqj24FoNHugNPu1UFqw5HqhFbLOTJVv62aYioIrprLUVRRjwkyx+ur40ZrViwB/X/9yjC3z4NYdnUbXtncOdbLiXJXD7DgwjZpIyiF3ScBtyLcXCf5YJV6nayI806GXWsMh5ZXu+FY84CZQ3IXEfJ/BX7MFn/HEwfEGXT3Q==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(136003)(366004)(39860400002)(346002)(6486002)(66946007)(31686004)(66556008)(2616005)(956004)(66476007)(43170500006)(5660300002)(4326008)(54906003)(38100700002)(8936002)(186003)(26005)(6916009)(31696002)(83380400001)(36756003)(2906002)(478600001)(86362001)(16576012)(316002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bHdkUDdsZmVBMUNWVE5OSUZaZEwxcUdFZmVLMndWMHR2ajR3L0piTWRZeFc0?=
 =?utf-8?B?QkhzM3FTd1gzYTBOMUY1cnBmNWhrcVN4Tk5wK2h3MktPRjdaVlNoVmhVaGw1?=
 =?utf-8?B?ek0rNDJPRm9LNzliL01RRE4yMTRUUTNFVTJab1NaZ3RUcG4vOHdVay93TU14?=
 =?utf-8?B?d2FoSW5VM1dxZ3BPU3V6MFlqdjlUVW5yZGVZbnlpNGFqdVNqSnh2Mmtmb0VC?=
 =?utf-8?B?OVNiTC8xWjhjdWFlMTdqTTFUbWwxcisrTTdxVXd0bXNDRjZCNUViQnArc2dQ?=
 =?utf-8?B?VVkwRG5vS2hPNHVvQ0xyT3RNSmwyTldoQTVkRjJnaDNwT0dSZ3l5aXphZWdu?=
 =?utf-8?B?WVF4TFAzd0U0NE9OdmRFV0ZZeU1FZENVNy80T1BmdlJJY0ZGQ3A4YXN0b2pq?=
 =?utf-8?B?a25yM0VrdEVpUnNoOVRpd0VwSkl4U1FuME9iaTZzQmhMZUlQeWZwSHJLMkNU?=
 =?utf-8?B?VmpiRHQxOUZ5REg3RnljVkZkT1dwa3doRHBLaVY5dmhueGtGNXhVZ05qS09W?=
 =?utf-8?B?V2swS3MyU04rOFJwNlBZYytmaVNkYXdxZDlUUXd6YjJCZGNKdDl0YndiQUNN?=
 =?utf-8?B?RFBYMHdPNkF5bmRUT2NvSE9LMUY5UUNLRmQxOVZ0Z3Y3Z2g4KzliOFhTOWxw?=
 =?utf-8?B?aVV5eDRTdmVwWGNwT0I3OWRSSDl5MnFyNWp0MDdZblV6RTVlaDR5NDNwZzVp?=
 =?utf-8?B?b2x1dE1NRDZZb001UStQTGpGSmNrTndWYXViRmFHbzJmNzJrRlpMV2E2YmtL?=
 =?utf-8?B?WlVscXBkRTBhTGtpZ2JmR1l4bkxUYkphNWVSU2JQK2RXcmwvd09Rajk1UWo4?=
 =?utf-8?B?V2Z6UDQvTzQ3d3dTci82TzlmTEJJWDcwYi9iQTVQMWlmNUVxcnd6RDFEVEdP?=
 =?utf-8?B?QzNiTjQ4YkJQL1VtUmNSUTd5bjBQamJwR2Q1eXdlNWxuelhPV0NHK0k2ZnBX?=
 =?utf-8?B?L1BWR3QzamhpNi9aL0Vmb0xiMWF4S2FrVVJ0MmNvUnhQclpnRGVIai80ZmNp?=
 =?utf-8?B?ODVFajZPUnpqcSt2dGxVSTBMNk1LeFBnMVBHbmFkZlN1OGVTRFYyMVYxdGp5?=
 =?utf-8?B?RU5aOTZkdkdHRWU5M1R4YWtLS2JmNGZrOWVrdkZsVEZSTUtaeldub0V2ekI0?=
 =?utf-8?B?bU5aZHpDNDZMZU9pRG1iRWh5bko1ZTBIZEpKc2tPQ0FWeVkwMk8yZWo0WEVu?=
 =?utf-8?B?ZG5CbkJVd3FlSG1VZEVuRkZzSDIyYTIvNzZOMlBlMkcwaWk4cUFmSmlpRFJp?=
 =?utf-8?B?cWhKYjFXcjBhNXNzTThKR0QxWkI5dDZWdmMyZHNTNGJ0VWlMR0JHM0dXYVlv?=
 =?utf-8?B?TzIwZldWcHBPeWFZM2NGV2kvalgwVGZqMzN1NnJuSG1kU25uditSOWNjdGVK?=
 =?utf-8?B?MXE1a0V1S0NMYWFjWmxsUFZmR0xicVNUck5lUFgycHQ4Q0p1TE00YWYrYmY0?=
 =?utf-8?B?L1ZDVmVDYVlFY1UydmUzM2hKYjkwY1V1SGFVb01aVFRZT053dkFVZEFKNlBs?=
 =?utf-8?B?dlR0cFJ3YW1JaU5DbTFDbmpsS2k0anBNakFscytvWHhROUlDcVVtTzh6RmVx?=
 =?utf-8?B?eGFaTWF3R3NRWFg0TEF0MUxqN21YbGNBc2NRZjZnN0dDNURxd1ltek8vU0JV?=
 =?utf-8?B?V293b2lqOHZmUHVTeVR6M2lNSEJjaVRPYi9ubjg4MlY4d0xEQTk0TnVyS1Vn?=
 =?utf-8?B?TUliSE5vMTcyT2hUSngxYnY4VytJMFFuRHE3ZGNHOVRhWXFrTjdTcUwreG4v?=
 =?utf-8?Q?9GyXbSXbrPKgfFAFcXcAX4nJVn/dNz4jg0zm6GI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fcc9d8af-6676-4208-c431-08d97681901e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:41:49.1149
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JBH1A/l9p1p3++jdcLr5TO7eJXRM9dEpslvlf7RW3zfY7EOWSbYw/eSYRx2mtTmVC18NpLAXQdymaMtPOJy03A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

Without holding appropriate locks, attempting to remove a prior mapping
of the underlying page is pointless, as the same (or another) mapping
could be re-established by a parallel request on another vCPU. Move the
code to Arm's gnttab_set_frame_gfn(). Of course this new placement
doesn't improve things in any way as far as the security of grant status
frame mappings goes (see XSA-379). Proper locking would be needed here
to allow status frames to be mapped securely.

In turn this then requires replacing the other use in
gnttab_unpopulate_status_frames(), which yet in turn requires adjusting
x86's gnttab_set_frame_gfn(). Note that with proper locking inside
guest_physmap_remove_page() combined with checking the GFN's mapping
there against the passed in MFN, there then is no issue with the
involved multiple gnttab_set_frame_gfn()-s potentially returning varying
values (due to a racing XENMAPSPACE_grant_table request).

This, as a side effect, does away with gnttab_map_frame() having a local
variable "gfn" which shadows a function parameter of the same name.

Together with XSA-379 this points out that XSA-255's addition to
gnttab_map_frame() was really useless.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -92,7 +92,7 @@ struct grant_table {
     struct radix_tree_root maptrack_tree;
 
     /* Domain to which this struct grant_table belongs. */
-    const struct domain *domain;
+    struct domain *domain;
 
     struct grant_table_arch arch;
 };
@@ -1795,8 +1795,8 @@ gnttab_unpopulate_status_frames(struct d
         {
             int rc = gfn_eq(gfn, INVALID_GFN)
                      ? 0
-                     : guest_physmap_remove_page(d, gfn,
-                                                 page_to_mfn(pg), 0);
+                     : gnttab_set_frame_gfn(gt, true, i, INVALID_GFN,
+                                            page_to_mfn(pg));
 
             if ( rc )
             {
@@ -1806,7 +1806,6 @@ gnttab_unpopulate_status_frames(struct d
                 domain_crash(d);
                 return rc;
             }
-            gnttab_set_frame_gfn(gt, true, i, INVALID_GFN);
         }
 
         BUG_ON(page_get_owner(pg) != d);
@@ -4132,6 +4131,9 @@ int gnttab_map_frame(struct domain *d, u
     struct grant_table *gt = d->grant_table;
     bool status = false;
 
+    if ( gfn_eq(gfn, INVALID_GFN) )
+        return -EINVAL;
+
     grant_write_lock(gt);
 
     if ( evaluate_nospec(gt->gt_version == 2) && (idx & XENMAPIDX_grant_table_status) )
@@ -4144,24 +4146,18 @@ int gnttab_map_frame(struct domain *d, u
     else
         rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
 
-    if ( !rc && paging_mode_translate(d) )
-    {
-        gfn_t gfn = gnttab_get_frame_gfn(gt, status, idx);
-
-        if ( !gfn_eq(gfn, INVALID_GFN) )
-            rc = guest_physmap_remove_page(d, gfn, *mfn, 0);
-    }
-
     if ( !rc )
     {
+        struct page_info *pg = mfn_to_page(*mfn);
+
         /*
          * Make sure gnttab_unpopulate_status_frames() won't (successfully)
          * free the page until our caller has completed its operation.
          */
-        if ( get_page(mfn_to_page(*mfn), d) )
-            gnttab_set_frame_gfn(gt, status, idx, gfn);
-        else
+        if ( !get_page(pg, d) )
             rc = -EBUSY;
+        else if ( (rc = gnttab_set_frame_gfn(gt, status, idx, gfn, *mfn)) )
+            put_page(pg);
     }
 
     grant_write_unlock(gt);
--- a/xen/include/asm-arm/grant_table.h
+++ b/xen/include/asm-arm/grant_table.h
@@ -71,11 +71,17 @@ int replace_grant_host_mapping(unsigned
         XFREE((gt)->arch.status_gfn);                                    \
     } while ( 0 )
 
-#define gnttab_set_frame_gfn(gt, st, idx, gfn)                           \
-    do {                                                                 \
-        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =    \
-            (gfn);                                                       \
-    } while ( 0 )
+#define gnttab_set_frame_gfn(gt, st, idx, gfn, mfn)                      \
+    ({                                                                   \
+        int rc_ = 0;                                                     \
+        gfn_t ogfn = gnttab_get_frame_gfn(gt, st, idx);                  \
+        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
+             (rc_ = guest_physmap_remove_page((gt)->domain, ogfn, mfn,   \
+                                              0)) == 0 )                 \
+            ((st) ? (gt)->arch.status_gfn                                \
+                  : (gt)->arch.shared_gfn)[idx] = (gfn);                 \
+        rc_;                                                             \
+    })
 
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
    (st) ? gnttab_status_gfn(NULL, gt, idx)                               \
--- a/xen/include/asm-x86/grant_table.h
+++ b/xen/include/asm-x86/grant_table.h
@@ -37,7 +37,12 @@ static inline int replace_grant_host_map
 
 #define gnttab_init_arch(gt) 0
 #define gnttab_destroy_arch(gt) do {} while ( 0 )
-#define gnttab_set_frame_gfn(gt, st, idx, gfn) do {} while ( 0 )
+#define gnttab_set_frame_gfn(gt, st, idx, gfn, mfn)                      \
+    (gfn_eq(gfn, INVALID_GFN)                                            \
+     ? guest_physmap_remove_page((gt)->domain,                           \
+                                 gnttab_get_frame_gfn(gt, st, idx),      \
+                                 mfn, 0)                                 \
+     : 0 /* Handled in add_to_physmap_one(). */)
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
     mfn_t mfn_ = (st) ? gnttab_status_mfn(gt, idx)                       \
                       : gnttab_shared_mfn(gt, idx);                      \



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:42:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:42:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185445.334102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPffZ-0001pf-12; Mon, 13 Sep 2021 06:42:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185445.334102; Mon, 13 Sep 2021 06:42:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPffY-0001pY-S7; Mon, 13 Sep 2021 06:42:20 +0000
Received: by outflank-mailman (input) for mailman id 185445;
 Mon, 13 Sep 2021 06:42:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPffX-0001ny-Qp
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:42:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bd354e6e-145d-11ec-b39f-12813bfff9fa;
 Mon, 13 Sep 2021 06:42:18 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-3-IrOzGsbqOBylWK6V0Rmh3Q-1; Mon, 13 Sep 2021 08:42:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6382.eurprd04.prod.outlook.com (2603:10a6:803:122::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 06:42:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:42:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0105.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 06:42:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd354e6e-145d-11ec-b39f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631515337;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Jcfr1R3ZTk9xcTyx2ZmFzXxoyWuNXyzsEsD8IfnCYTQ=;
	b=VnIB3r42XQd/nrA+WDjH2ILzlQAFc9h0EabwfzYd4RZW1+lqZgZHmqR/SSEpAAqY2FJVUL
	TLrpQQql2BKP7FuU4rWzQbv0sR1m9MU98LRQ87BKq1mXZpZdNIS3Vk597gUEBNF+gD/OZU
	4Ftf4fEeTzR/Y/RUF3lJQcAOiM74OoE=
X-MC-Unique: IrOzGsbqOBylWK6V0Rmh3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ey9Q/M4xw1SsoZFU7rqGpIu74hkiHzYncwtIGV6lcJO413zg1IqqHe34Gq2bCBJl7YjxLBTIocgIa4xZlJ85HI+uMoBiyxkELKhz6eqvsSHUVKme0kSTJO4qrE0O1R26GaT33+x50QPzdQCvDSCiIPJk7kxeRjs/qkMGb6ix+Cz2j9usS4eUqeBfk0lvF2fKQNfwZSyOLE9rOfSNYSHhOJPHroRZZeGfKVA8ESyAZH7pGt0DBcUzbmDRVo7HYrCE7ZIIoGmH6MiK6kxsS1QU45vs17DPDxtdCvj0Wa33DVKSToVWlBxh0vVEKwBm3F679tBw3iRpcYAsqQyCWr1dNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Jcfr1R3ZTk9xcTyx2ZmFzXxoyWuNXyzsEsD8IfnCYTQ=;
 b=Cr8XYmplnDsGWeXAFCLE01qsRyzY4bppSSTmvBqgKo66tw3FpILNPtLYNTlJinL4DMdrlP7RCdHI7o9hCANFn+pqH0+Z/vjDeD8WxMPUtE5EvxbaUoORHz43pTtAeFMVvgw57GXi/Sf8eTdmbUHlm7Z45SIXvd3UpsEF7pUtK3uiolKJLCexl56bSjuCvGPqrdIcVo4Ykxys75MufBrOsYQxY+vTEVQNzW625qf63yrC0V4vp5yeC59Br2nuc7J8+MRhCm7pIjy99TnD/sQ1UuRt2hQF6RdfiM85F1Incvz9VfGJ4cJp6lc4susWiIRHDsufprXYl6wB56XHC5TEvw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/2] memory: XENMEM_add_to_physmap (almost) wrapping checks
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Message-ID: <f78f8a30-39b7-cec2-2af0-27ebab28cedd@suse.com>
Date: Mon, 13 Sep 2021 08:42:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0105.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3a225497-28c2-4a34-0534-08d976819ffe
X-MS-TrafficTypeDiagnostic: VE1PR04MB6382:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB63826240144E30F50C905B19B3D99@VE1PR04MB6382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1122;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JAy0p4iTe3qylm7NbvDj3KLDkqfMeAim9mH9Ww4GGc56ngkyGGyLFhlVc5OMniLxhS62Im6//UGnFLblHmfERAuBXSDbt49zgPY+EpMrZJ12UFts/xx48rxkREFR60EsluVeEzByqiov7YjmKMqTmvyXvEylOqms65UK6ruxqVebcv9EBV+NopnFqYx+DnB7abGZMm+Do45qpPuiiwsuM1PKyIzYQmnZRHKr51mdLaAYG4191w4ts5KkPVtbMzX90bq01GFqa6smdeTTl7tdfnMLQlVrWNhLm16enoLFGn0gkpTw58/TJcHjoojLLuCsgeHpsdQXXnKflB+0Wy2+vv+ZFhf21gKblLWBpWJP+Y+JGPC9z/2PqmuRuKb/XzHrvthCL1VgBo7ZoTw6yhq/JuuDBjSO4rLjbgplcbRNVYBmfaUU4ZNtmsuUzFs5KENqjTNuGzNj1ZwwCZG3ksIQkeLAk6SogMa1NcHesGBpnlmA25+Zbq4KH1i4UMBNiWJ/m+JjispyilDghgLIpnomkqnrLW2LYiezy0dOHJjTmBpOrvWPKsxKVKD2mK4a2XZXNG1MhSrNEJ5Sgk3RqD8CndIudotKLjZ5EB1FTr5k/WNVF93QUFwIdpWevKJp9fS+s5Qomk8fygTim17Yy4Li/Y/2RXPmA8xknzMzzawBU3RSnnWR2FZrYqN9XMdV5eY+htKsFMYXlhu52eFO46HBLvhsJJJNKfixVbhZit9DcpY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(376002)(366004)(136003)(39860400002)(83380400001)(86362001)(26005)(8676002)(2616005)(956004)(316002)(2906002)(16576012)(5660300002)(6486002)(186003)(4326008)(66476007)(478600001)(6916009)(54906003)(66946007)(31696002)(36756003)(8936002)(38100700002)(66556008)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K2VTajZvSkpvWngxWmlwTTM4azY3dUVUZVNCVm9xcGxtcDh2eDBVR291OWh3?=
 =?utf-8?B?QzAyVERESkIyTDJNd1MyWW9CeTN0M1ZjR3RxV0srdk9DcGs0M3BOQlFRWjBk?=
 =?utf-8?B?UVVXYTF3QmVESW5ma2ZSbi9XVDNCQ0drVmJBT3JRRGRrWVFNdnhiR2FJdy8v?=
 =?utf-8?B?bW9icEh3THRYVGhSMWtZR2c2SU1pdzdYYXZ3aTkvRmtkbmcrK1hoekQ3Njdx?=
 =?utf-8?B?OXZtZGNkaHlyUWZ5RWYvT2dKbktWWUUvVWxVTTNwcDV2eWllWGFjcU5YcXFT?=
 =?utf-8?B?Rzc4YkFGYWtMeHcxWW85MHNhY0o5ZkNqT1E2RHczbUxUMFdZeC9ISnh0QnRN?=
 =?utf-8?B?WUN2NGc0cjAwTmp5YU80dmJnNVRLZjg3TUlETEQ1Nlg3MnJERGVOS2hrYTF3?=
 =?utf-8?B?THM2N1JDNUJBZ0ZPUjF2aGcrY1RYVWVZdTVZVWtPeHdTU1RJVWd1RXFCTDNa?=
 =?utf-8?B?U084cE5MdmxTb2szcy9CSlVOMkVJODZXaHBId1ArWm5aK20zeXZhR3VXYWZv?=
 =?utf-8?B?RjcwVEViYWU1TnBXbTM4NHJVc3c2MjhYQVAwdlNiT3hqVGxXUjBmSFkwV0Mv?=
 =?utf-8?B?MUJ1WGNabllESGpscW9mUU4wV2ZCWktuQTE5ZzZJdGtIanZ6a0szQUhzOGUy?=
 =?utf-8?B?cW5Yc2x2TU5OMXFmVE8zb29oYzQ0dHgxU2xTbEdHUDNRbFFCL09Ga3hwVU5L?=
 =?utf-8?B?eVU0WThIOUZWeThZMDBzSHpSK200Nzg0dE9BTERaR2QwTDJKQW1hUDQ2b0pr?=
 =?utf-8?B?VFFPQ0VTVVltczdBbjdvQlFRelg4T00yVmFJL3g1bGlDeitUYWg5OXJvTEhM?=
 =?utf-8?B?ZFVFNHhCamp3SEpPS1YrOVhuK3UzVFRacVEyR1grVnZCWHU5SzNkQk5EV0F4?=
 =?utf-8?B?aldGWm5XTEtScTArUUI2cjlYdmRaS2V4Y3hTNmJ0SG1TN2ZYdURMUm1ZSXk3?=
 =?utf-8?B?ekhuU3NvNCtkdzlpcXpuaitnZVFkcWpsejVabUdtQ1pHa1UzUERpRHQrZE5m?=
 =?utf-8?B?ZjdqQUZWT3YxL2FhQk1qNHQyZHliUzBveThCWWczS3BzRWpQeURjRjg4eUdy?=
 =?utf-8?B?dTNMZjJ2VmR3dmR4bWVYYjRoNlZIL2dOa0FvV2hEcE40N3VSQy9IM1JaZ2Yw?=
 =?utf-8?B?NkM2d251UzRyS0FzLzNUYXZVZkE3Sm13OTlDUngvOVpwUFVuYUpYUjJFdEYz?=
 =?utf-8?B?TkVibmxqMUY4UzlQK1drQ3N5c3NtY2FXbTR0WVFUNmlJV2tkbnlCSEVJdE5O?=
 =?utf-8?B?MnlkMWdJdy9hcnBuQ3NLbmV6cFR2eGMzRmsxYlhoNFFDYVk1RlFVNnVvL0hp?=
 =?utf-8?B?VFBTRzNCOTN0aE5QMmd0THRxdWk2dFdqZWsrOFRreHQxN01pM0FDelJNNWIx?=
 =?utf-8?B?ZGtEdTR6TE9QSml1Sk1NaFVmNnpWTmtrTng2Nk9iVitJbHRlUlUvWWhCcm9W?=
 =?utf-8?B?SnVpUlp4ZlRCMU5UcEd5bDREei84eERLOUliWkxNMWF0ZWZhS1d4azZ2cERY?=
 =?utf-8?B?MEdtOGhrMGdiaDJZcitPTGhydlQrMTIwS0xIWFJUTmRwcEhFeG5PeHNzYXNN?=
 =?utf-8?B?NVJXdnBtbXg5MnBITmYzVHh6RnVoM0FhSDM4bWFHWkU1ejhRYTlGdmFrU0x5?=
 =?utf-8?B?RHZZNEtwdEhxclN4bi8xNGg0Z2lkaElZTWM5a0x0clpLMEpzYjB3U0pvRzIx?=
 =?utf-8?B?Yk4rTytPVFc2MnJaenkrcEtJdlYvbFFRYnpxNG82M0tpU3RYa0d2USt2WVN3?=
 =?utf-8?Q?TL4S5RyeEGHAAP4tONF1wRzzZVNKhDsNAtoRsie?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3a225497-28c2-4a34-0534-08d976819ffe
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:42:15.7429
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: S3RYP+X/IZDY7ViNOig3luQVghlEVX8C45yvn/rjr55bDFBH9l4yyiVuFVrs3QL7/3z7qOPshP3Lnvfsf3ccmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6382

Determining that behavior is correct (i.e. results in failure) for a
passed in GFN equaling INVALID_GFN is non-trivial. Make this quite a
bit more obvious by checking input in generic code - both for singular
requests to not match the value and for range ones to not pass / wrap
through it.

For Arm similarly make more obvious that no wrapping of MFNs passed
for XENMAPSPACE_dev_mmio and thus to map_dev_mmio_region() can occur:
Drop the "nr" parameter of the function to avoid future callers
appearing which might not themselves check for wrapping. Otherwise
the respective ASSERT() in rangeset_contains_range() could trigger.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I find it odd that map_dev_mmio_region() returns success upon
iomem_access_permitted() indicating failure - is this really intended?
As per commit 102984bb1987 introducing it this also was added for ACPI
only - any reason XENMAPSPACE_dev_mmio isn't restricted to CONFIG_ACPI
builds?

I'd be happy to take suggestions towards avoiding the need to #define
_gfn() around the BUILD_BUG_ON() being added. I couldn't think of
anything prettier.

--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1479,7 +1479,7 @@ int xenmem_add_to_physmap_one(
         break;
     }
     case XENMAPSPACE_dev_mmio:
-        rc = map_dev_mmio_region(d, gfn, 1, _mfn(idx));
+        rc = map_dev_mmio_region(d, gfn, _mfn(idx));
         return rc;
 
     default:
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1360,19 +1360,18 @@ int unmap_mmio_regions(struct domain *d,
 
 int map_dev_mmio_region(struct domain *d,
                         gfn_t gfn,
-                        unsigned long nr,
                         mfn_t mfn)
 {
     int res;
 
-    if ( !(nr && iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn) + nr - 1)) )
+    if ( !iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn)) )
         return 0;
 
-    res = p2m_insert_mapping(d, gfn, nr, mfn, p2m_mmio_direct_c);
+    res = p2m_insert_mapping(d, gfn, 1, mfn, p2m_mmio_direct_c);
     if ( res < 0 )
     {
-        printk(XENLOG_G_ERR "Unable to map MFNs [%#"PRI_mfn" - %#"PRI_mfn" in Dom%d\n",
-               mfn_x(mfn), mfn_x(mfn) + nr - 1, d->domain_id);
+        printk(XENLOG_G_ERR "Unable to map MFN %#"PRI_mfn" in %pd\n",
+               mfn_x(mfn), d);
         return res;
     }
 
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -4132,7 +4132,10 @@ int gnttab_map_frame(struct domain *d, u
     bool status = false;
 
     if ( gfn_eq(gfn, INVALID_GFN) )
+    {
+        ASSERT_UNREACHABLE();
         return -EINVAL;
+    }
 
     grant_write_lock(gt);
 
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -831,6 +831,9 @@ int xenmem_add_to_physmap(struct domain
         return -EACCES;
     }
 
+    if ( gfn_eq(_gfn(xatp->gpfn), INVALID_GFN) )
+        return -EINVAL;
+
     if ( xatp->space == XENMAPSPACE_gmfn_foreign )
         extra.foreign_domid = DOMID_INVALID;
 
@@ -841,6 +844,15 @@ int xenmem_add_to_physmap(struct domain
     if ( xatp->size < start )
         return -EILSEQ;
 
+    if ( xatp->gpfn + xatp->size < xatp->gpfn ||
+         xatp->idx + xatp->size < xatp->idx )
+    {
+#define _gfn(x) (x)
+        BUILD_BUG_ON(INVALID_GFN + 1);
+#undef _gfn
+        return -EOVERFLOW;
+    }
+
     xatp->idx += start;
     xatp->gpfn += start;
     xatp->size -= start;
@@ -961,6 +973,9 @@ static int xenmem_add_to_physmap_batch(s
                                                extent, 1)) )
             return -EFAULT;
 
+        if ( gfn_eq(_gfn(gpfn), INVALID_GFN) )
+            return -EINVAL;
+
         rc = xenmem_add_to_physmap_one(d, xatpb->space, extra,
                                        idx, _gfn(gpfn));
 
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -297,7 +297,6 @@ int unmap_regions_p2mt(struct domain *d,
 
 int map_dev_mmio_region(struct domain *d,
                         gfn_t gfn,
-                        unsigned long nr,
                         mfn_t mfn);
 
 int guest_physmap_add_entry(struct domain *d,



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 06:52:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 06:52:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185453.334113 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfor-0003TH-TV; Mon, 13 Sep 2021 06:51:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185453.334113; Mon, 13 Sep 2021 06:51:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPfor-0003TA-QY; Mon, 13 Sep 2021 06:51:57 +0000
Received: by outflank-mailman (input) for mailman id 185453;
 Mon, 13 Sep 2021 06:51:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPfoq-0003T4-Nf
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 06:51:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 64abff88-4f6c-42fb-b1ee-57cc7e05ede0;
 Mon, 13 Sep 2021 06:51:55 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-ya_eWygEOBq1YmCrR1e6mQ-1; Mon, 13 Sep 2021 08:51:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3935.eurprd04.prod.outlook.com (2603:10a6:803:1f::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep
 2021 06:51:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 06:51:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0007.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Mon, 13 Sep 2021 06:51:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64abff88-4f6c-42fb-b1ee-57cc7e05ede0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631515914;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nx1X7nsgt9I6wHT41R9KWBdn2/EzJAwvqED5gqQadZc=;
	b=ULXedUjfExE0IHZIxCyji9T74dMHv2B7SwVqdmqHzsgCGVnqRUUpUR3eto05SZkhKayau7
	NtNBKLwh4P49wVKcRGiykw2pfMRUZ6UUxk+sTG76lVP3HANJQjTe1z99/r9YOfMCPzmaoN
	+BM8xbboWNmbquL7hFBmN6DhK4UrNCI=
X-MC-Unique: ya_eWygEOBq1YmCrR1e6mQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=D9KrF2Guv/MPVykmCSmq9LMLuic+47TqIu/OgxVe3lB8UoNaFTwfIUi+Xh3GLP6iNn6o7wG06qf+5/qBbOgVSOwelb3IxidL6VMW6MYXko6Muh+t4U3ZQ0U1h6vjEX6CYdEddMHPAzlTbJOb1TLjThJX7HBu6v8WhuAGInN200fi37OFpbDVpqpEMCbs14p9sxr5G40quC6EnYnvZQTZBsbflmCDtDyhj9G7oiTzk6hK9xdj78LGS5NowV/QpaMRJPftBU0uYx18+CEypk/Fxm81tLojSJUDs6O0I8iaKUJ86BbNWTxOfn5Co0bqV+Kxytf1B3aQpTPjV2QmgpxBmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nx1X7nsgt9I6wHT41R9KWBdn2/EzJAwvqED5gqQadZc=;
 b=B/8kj1YL5oQ+mz7s13tplbF+YA9HN4lCYEAt/w/MbmlQz13FnpZSw2C8yKXo44wVDA6VSUDvrxlyRD6ZSAi9NLJ84lUulmxzHgv7C6KoCx0SsGA8ZY7b0Uqld4Z2K2ycOgn9UuGHE5gR7ADh+4a36bYQvdoo4SruLi/n0t2F5bY5h6AlR8FNqMhxMx3HCwkfbTKhUHpNT5Cy9J6sZaZ8kzBSaJE+kZ/pO05udLQF6rR3M86akgc4RCouldZZ/WWvc0Z9uiInFZ2M+XEqI371x0euzenIxRTAzbw6ZcKJGdEYXWMxNPGt38abtDEKNBYO7ROwrj0hez8DbFqqEY0p/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v6 05/10] xsm: apply coding style
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
 <20210910201305.32526-6-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a5279128-c088-a1ab-b634-e20ff7dcdfd6@suse.com>
Date: Mon, 13 Sep 2021 08:51:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910201305.32526-6-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0007.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 511c15b5-26f9-4ebd-b4bd-08d97682f7fc
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3935:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3935218B4276B699B29F7B30B3D99@VI1PR0402MB3935.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:489;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zTOFObemz0prJ05U0g1SeY59R6z1j4DW4vwW1G0mYGn6UO9SXFs8gB8GsUUaSVs0Fj1VLUQg6F9cE6wzlbxWaUmvte5+BtYmUqUqt/WmK3mUKhUb8aBBdBUJYkiTY+x+vm7Hl+HnkQIbslZIJpMxJgYkTO/f6EJygI316B1gqkpBWtQSWzoiOGJ7MVXuK5sc1m2nHHm4x6R/VNKBqtyGkDcduxNKB1lmko824ZwY6x8d1rw1UUFbFTAGNCQkRz7g6SjTbRypaIE5rKCmmUAhpn2mD94lO8M+4zQKzmtlDc0vFVCZCIeIzme7NJxBoXcdYpPApQ7qiwKUTHsxXU0SI36gotV0UPY28p8JAnN6LmkncsUZQJMO+88qZEPBqQZGwAlBgv8f65HGrZ4CwWDf3sdSN0RpUdJAElhMW0hDau/Yb+MuKGYyIZ8vrcSEHJ3n6nl2/BZ6o3MvPZf/xDtMlzDgO+zXUwVQPYMg6EjMD3whkiwA49w98IqqldRyIrEgA55CxOVOCU2AV9UPH4PX9FDmdLSyhpeUXn6I0vYWOAmnM53p5Q6lgUsca772l6wRI/wkT7O283JjiP2Oyzq3qZN1WpxjM0KYLDzzXrjMJFkr/BG6VUwW5swLhdr5JdcL2YZtTlBtuRy5m/cdTxMNJpzu7bJLi9/W+Y7ZINHD/iQly1hxnyhSdqsx4fkHKJ+3lycDborTs+ynHmlbv5uu7a7lCQwIcXr0S1H/w5eKlNo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(396003)(346002)(376002)(8936002)(53546011)(4744005)(6486002)(478600001)(86362001)(8676002)(26005)(2906002)(186003)(36756003)(31696002)(316002)(31686004)(4326008)(66556008)(16576012)(83380400001)(956004)(38100700002)(66476007)(6916009)(5660300002)(66946007)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UzRWeGNLS01sSTJRdEZ4YkFBNlNqQ3Fqc0duWG1uc2xUc0tWQzU2U2VTNlNi?=
 =?utf-8?B?Q0hhMXEwV3prNHVDSjFnYW1RcCs2eWtKT2txdmR4eDdFSkpBeU5zcE1ObmQ2?=
 =?utf-8?B?M3EzYncwZ1l5MjJZMFdBT2pjWFdiMkMrdEJRRmN0VmxWeFdoemRUa01NM09s?=
 =?utf-8?B?a3IrUktnQngyeGxJOXo0S2RNVlA5clVSZGpqc2ppNlphVzdTd1lJOGk5Ukl4?=
 =?utf-8?B?eDdwN2lMMW4vTW42Ly9pUERVVHQ1UDNoR1RtSjFqN0dvcjk2M253aDVaZzkx?=
 =?utf-8?B?cTRtbWZyOUhQazliUXF3STdvRjZmZUhlNXE2MFRlNC9oaTlRODBYbnVqbEE1?=
 =?utf-8?B?b2IxZW1sblJyWXErSmZnZUtsci9VNFBDcmxVRGFpMHJoZ3hNdFF6aEs5MVFE?=
 =?utf-8?B?UzVBdkYrdWtRZVFaSDJsbnVURWd0SUIzVi9McEw3Z01GVXdzMDUvR21IMmFB?=
 =?utf-8?B?ZFZFZ1d1VmJQcVBZRmYwVnkzK3hSWElsb1hwaXZZTTg5NXNIM012NmQ3eFZ0?=
 =?utf-8?B?eXg3Ykh0UWdZOVk5dC9aR2VpN2dIVjQyR0trVmZYNkw2eE1HWGEyUmtPQzQv?=
 =?utf-8?B?bmNXV3RkVlhQcFdFTGpKRXhxaEpydmpUajZXUnFLZDJyT01jSUozZkJYUTgz?=
 =?utf-8?B?Q3VtU2NWNXExdGYxRm5KeitYTU5lV3BnbjhOQWhCVWU2SHVGb2lydDNLRUUz?=
 =?utf-8?B?WG5UczYwcGdaSnp0aEx6MGxvL0c5QzRkKy9naDBoWStvNzlJOTNzVml3MzFp?=
 =?utf-8?B?RS9UWmRObEJ1QTIvS216V3hjYytJR1NHa05laEtwYnByWDNEbVlNNXh1T1lr?=
 =?utf-8?B?aW44NUp5UlZ5OWVsUjdzbkJteklGWlJ4RTFRR2dIWHcrTFRkSmVyZHZmMVFD?=
 =?utf-8?B?K29oc21UeVhyNXZzMllrc21vZHYvVnpEWUpod2s2UFpBS1BzcTE5Zmc4VHhG?=
 =?utf-8?B?RmR4aXoramVPc3ZoMHgyZHdla2lFeEJqRHl6aWRwNUhOT2R6Tk1ZTXpFd3lG?=
 =?utf-8?B?Q1R2aDZaVGdVZFN6bTF1c2VCOG55NFNjbEszT0xIZUUySXRpQlpycytvR1Bj?=
 =?utf-8?B?d0MwdENmTnU5aU5QaGkvMDNNTlJKQ2pZV3J1QnFDZFVIUmdyZzZ1RllUOWlI?=
 =?utf-8?B?RytmejFFbS9xWE1PMUtzZW9XZi9JdlJ4ZDI4UDM1MGhQWlBKVWt1RzViWm0z?=
 =?utf-8?B?Q2ExaGR2YnBhOG5WbTVNa3NxSlFhZjRQVW5pVzVsWStBa0w3UGhSNFAvTit3?=
 =?utf-8?B?UG1kb2xyWjFQa3pOTEJyZEdDenowQWFmY2VmNWI5LzZ2L09xdVpHMm9FTjZa?=
 =?utf-8?B?TjNES2xrUkVmRUlreFZxazN3VGQzbTlXT2UrWUhoZklsZzdzSnhmSHJUdmVt?=
 =?utf-8?B?ZzQxN3pJb0lxMmFiSm1iK2dvWHlwQnZma0s2L2J0TzVsMDEwbmE0OVFhcUxu?=
 =?utf-8?B?bmVjOW1Tajl6OERBVDVXRWt0M1YyQWd1SkJ3NDd3VFRPc3o0NDF4YWdQRFpD?=
 =?utf-8?B?cTh0V0NvcWhScUdRVFc4Ry9VaWh4akJOYm9KeklSWHppTlVlMEdJWmVGTTlF?=
 =?utf-8?B?aXJhMGJmV1dBTjJscTlEYjJucnJOcU1FeERxTWNBVHR3QkZpUzRPSHd2dDNt?=
 =?utf-8?B?Q1FIeHE3TzZyanpVN1o1dG94a1pXbnE4V2xNZ1laMm9HZ2J6R3I5dys2L1gz?=
 =?utf-8?B?ZzFSemRwM2cyZEl1VUFFU0Q0VnBYeTd3dyt6VS96Vk9xWllxMjlJRjVDaE9K?=
 =?utf-8?Q?mNce9IaedtPtuGB0zVK4GYQu9fQ6SCvGOzUrlwQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 511c15b5-26f9-4ebd-b4bd-08d97682f7fc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 06:51:52.9324
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7dBWdf/tT4MLkWd0KnBR4jY1NqTyljp0nGdGqX82pog5W7q58tA+V3NJ4l345ZGZf5Xoh3y0xn/ZOCzVck2fsg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3935

On 10.09.2021 22:13, Daniel P. Smith wrote:
> Instead of intermixing coding style changes with code changes as they
> are come upon in this patch set, moving all coding style changes
> into a single commit. The focus of coding style changes here are,
> 
>  - move trailing comments to line above
>  - ensuring line length does not exceed 80 chars
>  - ensuring proper indentation for 80 char wrapping
>  - covert u32 type statements to  uint32_t
>  - remove space between closing and opening parens
>  - drop extern on function declarations
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:04:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:04:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185460.334124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPg1K-0005C7-4S; Mon, 13 Sep 2021 07:04:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185460.334124; Mon, 13 Sep 2021 07:04:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPg1K-0005C0-0A; Mon, 13 Sep 2021 07:04:50 +0000
Received: by outflank-mailman (input) for mailman id 185460;
 Mon, 13 Sep 2021 07:04:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPg1I-0005Bq-7P; Mon, 13 Sep 2021 07:04:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPg1H-0000Dm-UD; Mon, 13 Sep 2021 07:04:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPg1H-0001c0-Jw; Mon, 13 Sep 2021 07:04:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPg1H-0002PB-JP; Mon, 13 Sep 2021 07:04:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=34hWDtPdSFZfQu6y1hUcRpW8ofcNPlJQpFYoAEs8Zqk=; b=i5Sa0RJPbh5VwIPxE+g6x1zVRR
	kUZu48+S6snED6XXoKRK3rBqRBYW/75ch7YN7L7wRgyM+cuPQ78mk2F6DxoMqZCVl6zGqUXystLfT
	P9/sBt/5DstxVlZS7sMzAMPtRNuFrnSWDOXoUNd89uoETdHvpO86DSm8kiKcG3ZTMlI4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164961-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164961: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f306b90c69ce3994bb8046b54374a90a27f66be6
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 07:04:47 +0000

flight 164961 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164961/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                f306b90c69ce3994bb8046b54374a90a27f66be6
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  408 days
Failing since        152366  2020-08-01 20:49:34 Z  407 days  708 attempts
Testing same since   164961  2021-09-12 20:40:52 Z    0 days    1 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2254282 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:08:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:08:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185469.334138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPg4W-0005vK-RL; Mon, 13 Sep 2021 07:08:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185469.334138; Mon, 13 Sep 2021 07:08:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPg4W-0005vD-Nh; Mon, 13 Sep 2021 07:08:08 +0000
Received: by outflank-mailman (input) for mailman id 185469;
 Mon, 13 Sep 2021 07:08:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPg4V-0005v5-9Z
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:08:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 57915838-1461-11ec-b3a1-12813bfff9fa;
 Mon, 13 Sep 2021 07:08:06 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-7Dq20pdkNeeJJtpG_-3E4Q-1; Mon, 13 Sep 2021 09:08:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5597.eurprd04.prod.outlook.com (2603:10a6:803:dd::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 07:08:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:08:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0120.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 07:08:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57915838-1461-11ec-b3a1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631516885;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=32TqImMi+91i8BvvjeHans1WrkYR10xKOADJXdQtMOc=;
	b=UZMTVUZIzO5a1xPozicp3hOoYPm7aM5kGNWT8b81dARGxJsL2DkKi8MwI0nrWo5zS/viN/
	qOrsghrjQkZuCNVP+x1pdzcxm9ENIOO8tQFNxyK/90HVv9wxXusxf4HCkA5UuT4wDVBkoO
	ZjQ1YXZEI35v4oCp71I2o7DNztsC4Fs=
X-MC-Unique: 7Dq20pdkNeeJJtpG_-3E4Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OzpSyz+DCXr5KwWgJHenk1WbxRrw1loOlPBtpdKzF2Ar01yqtisAN8P0OX4DodTxDBDTySouNRhgoTGkTLJkBLnQcaF0GEbiQqQ5hSH0589gEd7aDCRuVUo34y5/tx8ddffiPYaB2WEcX/6eVIMBNmsaxHe/BgxdEVOT+ywkOQkR8xmAg8plK2DmQzV0f/ltc+C8c6+VPwqRDSSJ61EVNrL/rUkCcplvS+Zm9KTS8NHyqdS3TRD9ZUh2fqXH9toWeYnQCUOOmgJnEQTxF2ZWlRaOZLDVSoNpkFX46dbCLtLn5eakCCp8cpnQZffiFGudvVDQoeX0+58GO2Nki4itWA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2knjgkLZn1y6wm3AKSFjUsFCFY2s2it+KcMcLQZnVQE=;
 b=GWcHH+SVtMZ2V8YAxkcTxrsPpOpfRQIQ7LZ49//78rghTP6lskpVMIUih4bKqWTu+ML2h72u1LmnXQtKKrGEM4DXB2rd+Uq7FDW1M9oCNCjWAx1MRC1Aj6Dt3Rdh8Zj7dmB+daYTPSH+pyERWFJ3s2SHAdAqnJTVZ5S/EdEQY6DcJEx5J1PY5qsTXjJWnQynRSeoO71H9SNcakXVFtpWM61ht/RoE1JTDhgJO9OxMfo6i17CNzUh9SreWCXnORu4F2R1btP6pc/HBgBMvnclXqv9XyLPqCbJiYxF1x3rCLLfD+8ghZ0myoa3L7FiFKUA32dlcrJTR1rdiJ+GNPZScQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignment for FACS [and 1
 more messages]
To: Ian Jackson <iwj@xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <20210909163441.44418-1-kevin.stefanov@citrix.com>
 <20210909163441.44418-3-kevin.stefanov@citrix.com>
 <affca979-4082-e825-dede-b27b5574631b@suse.com>
 <20210909163441.44418-2-kevin.stefanov@citrix.com>
 <bbde0f13-a1a5-53ed-82eb-704cd5d43178@citrix.com>
 <24891.28877.607703.662095@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9bc27296-528e-41eb-b27c-23e4113e9d5b@suse.com>
Date: Mon, 13 Sep 2021 09:08:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24891.28877.607703.662095@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0120.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 83106f15-4cc0-4ebb-bb23-08d9768539ca
X-MS-TrafficTypeDiagnostic: VI1PR04MB5597:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5597BF5E2F0F6B027CCBF73FB3D99@VI1PR04MB5597.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SdnERmSht0mHawXEiwQj3AGqORyEqj+eEdKoNzq4OCPiGVjCKYM0gpaLxwv1Wop7/0HC8gNQWxgGuGL8Xd98AzZegW6Z71vAAZZ1QPnfWHr46jM5Xk4rG/J8byq0g0uA/zrth1ExuCFIAjNMW/EtlG3ceZ/RxffzZlEWEcDXuzAalmL4aVV6YOWrcTmcTNs7E7uJYJtRhWocBsJgxKkEVDQX28MSRNx9gCu97V46CfeHEgmE/w63Q9ARh+B6XcpqS37DCFNmv564NVhy3AmlyDvVtMoYSze8gDezPsz8GTU+DeVUCPNU60DuQNwOgSqhTZzipyz963jZ3PsBSkdh8+bn8qO/ZAW2q0aGyUQD/tTQHeBd1SBxXtGUZwaJt/vPhXDk/dVTpRU/Gq+Ssgqfl3TZfStJREV7N/56tOsHX8zvutG4EEhsoGvL2V4vhouJtnEsphLOcGBgOfjV8LndxKUZR1n6NDeq91+uR5NfWm13jxR56ZIqmWX6HDmFsLEn2L/b/jvj1oRjq9bMS7gkcMiodu7/P/UmmcP7utk/3CY2VOdKQZFz2QZPu9eYtknCmT3C/eediVpbE4KQBcVqOemmtgndQ/FOQEWn1USjfH+lHW7blz5/7ne5zMZUAfa+XElpJ1O/AinHPUepxfYYLycfy5nuJtNZ6DcT1gXtuOQ99rA+OeupSX77W9rDf75uUSG5u22q3EBshN3MpFjuTBXqsXddykpd5UQUxTIvTikAqipNiiKSpWsXplbmi0u+
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(396003)(346002)(366004)(376002)(8676002)(16576012)(186003)(66476007)(316002)(53546011)(54906003)(2906002)(4326008)(66556008)(66946007)(26005)(38100700002)(83380400001)(5660300002)(36756003)(31686004)(6916009)(6486002)(8936002)(956004)(31696002)(478600001)(2616005)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?dRqrYbXMkTpLWuKVgQu/2oHQxt+F5nl+o7mzyalFrG3nmDC4FNrTanKf8RbF?=
 =?us-ascii?Q?AIToBffkM9ASSHfzyKLjeJoaRenfz8aMMhoFF/8AK62ew8PKnRofu92GD217?=
 =?us-ascii?Q?8wTwy0/HUBTjjyAzKkOIK8nHmuaNlxZPd9LUX/ItWVN2jmqjpvYukGtQLgwA?=
 =?us-ascii?Q?1MY3iyDLEFTVWI3QRW7cvCXDYaazGpFLSNJEvNT85lb/M2ojPaot8UPp/BAm?=
 =?us-ascii?Q?icsaPrzMNCtPX6GVulqtq9SQGbMOgHUhyzIwmcPD+2rpab5cHpVltHBB28by?=
 =?us-ascii?Q?KcsQi3cSxK5rAtyFDHLOVEf62x2ig/bLk/NyVPGc0pWSg1SfwHl+9W0kfblX?=
 =?us-ascii?Q?0iy/TzNtKRDyA51BcKDnTm5l71e8W+p0pR2VkI3MzrPzfNPNd9TjljkoKK/9?=
 =?us-ascii?Q?mcgRekw/KUp9aMYlgkXkATOG4GnuYBAAgYx27mqOCdmuksL3cxSEtx4bFtWL?=
 =?us-ascii?Q?s6+3PRCA0cuehQkpvKGfAq9cvyyrM30Pbbq3TH29LX7VnAJUASEfrGYmY1F2?=
 =?us-ascii?Q?xJ+mXvFPfkts7WHqbpP7Osxbo9HgkizAnSWYczkSBiIp0D4tRulk3JQZ0jps?=
 =?us-ascii?Q?+UibE0HRIAKaE5tc0B9uAhYKqi5gdtmF+TESuXQp/OsyIe0YscfPHD+lFeif?=
 =?us-ascii?Q?dGM8Sx78B4qbYEbVHnPnj0JlG0HMUO3vTIgue9edoVK3+CoyQauWQC3mdU7D?=
 =?us-ascii?Q?9Rm/wE7r1iJof73dqhm3v2Z5zXynZQE8feW19kDzmQK1EGThp9dVof43pz9e?=
 =?us-ascii?Q?dxpHOgk5uZoDtMa1bbE7kSC/TU31RQJ/ZJtkYbqhUSsM+gFRJSUyr+rugB9z?=
 =?us-ascii?Q?KvWeZipCza98SppTdtFMrE+SkoPyLE3fTLbsMUk0st6nmLe2fKb6E3h75s2y?=
 =?us-ascii?Q?oeRjaHH8XRc72Nxm9MJjIVcSV66celuW2nb4N1iSRKkTO5DKVcyFZNCND1s+?=
 =?us-ascii?Q?f+5A4eS1S/49x0tU1VbeNwPBQR6fNOjsBhNZpWY11768hxeCWXf6uAac2W7x?=
 =?us-ascii?Q?2+p2HkibAKULkEriIoBUQp8bUEEX176OGVeaEHahnedfQmGXRkdyv/CsEvw7?=
 =?us-ascii?Q?xsexzL6zPqKAflrh2mzqRkK6MfcszmzTaYG1BMobo8wDnRonlGMuH9xC60cJ?=
 =?us-ascii?Q?nU0HLmY28q+6Bgy95fNjI7XMZTTNgjuPW+xDUoO+vy3t3OLZMQTrJxBDiWyk?=
 =?us-ascii?Q?SKRzlZIV3amZyt+7fiEZOfKhgcPQu2kXbLkFm0KOK44V0EOps/YEyY2VJtHZ?=
 =?us-ascii?Q?gUZ3yX0r3d1gIsSTStNefSgMkLFEIW5LdF8JnZPYn5BNZGEp3L3exZCZLle6?=
 =?us-ascii?Q?CqrxGjikzH1ImW1LgjcshwaZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 83106f15-4cc0-4ebb-bb23-08d9768539ca
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:08:02.2670
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qOa19K1ihyU7xLhTD0R2rvnNuLEIc+sFNcXL++Vhd4lEYqL7gmQzVQdrpGhlGXqPUUMqNmHPM+PX4lU7VBtS0A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5597

On 10.09.2021 16:50, Ian Jackson wrote:
> Andrew Cooper writes ("Re: [PATCH 1/3] tools/libacpi: Use 64-byte alignme=
nt for FACS"):
>> The current code is clearly wrong, but happens to work correctly in
>> hvmloader because FACS is the first table written and it starts on a
>> page boundary.=C2=A0 The logic does not work correctly when libxl passes=
 a
>> buffer which doesn't start on a page boundary.
>>
>> As a consequence, I'm not sure what to suggest as a Fixes tag here,
>> except "the logic has been wrong for 15 years - patch everything".
>=20
> Jan Beulich writes ("Re: [PATCH 2/3] tools/libxl: Correctly aligned buffe=
r for ACPI tables"):
>> But the buffers obtained via libxl__malloc() have no association with
>> guest address space layout (or at least they aren't supposed to have).
>> That's solely determined by mem_alloc(). I think it is a bug of
>> mem_alloc() to determine padding from alloc_currp; it should
>> rather/also maintain a current address in guest address space (e.g.
>> by having separate alloc_currp and alloc_currv). Not doing so leaves
>> us prone to encountering the same issue again down the road. For
>> example if higher than page alignment was requested from somewhere in
>> libacpi.
>=20
> I think the two of you are saying roughly the same thing here ?

That's my view of it, indeed.

> There was a question about how (and if) this should be backported.
> I'm afraid I don't quite follow all the implications, but I doubt that
> a a substantial overhaul as described would be a good idea for a
> backport.  What is the impact for backports, and can we have a more
> targeted fix there ?  Are, in fact, Kevin's original patches, suitable
> to fix the issue for stable trees ?

I think they are; the risk with not (also) backporting the more complete
fix to do away with the bad assumptions is that eventual subsequent
backports may then run into the same issue again. Like Andrew suggested,
I think we want to first see how intrusive a "proper" fix is. If it's
really "bad", we can still decide to backport only the original change
(which I'd rather view as a workaround) while trying to make sure future
backports won't end up hitting the underlying problem.

Of course there's also the option to simply declare "future backports
here are unlikely"; I wouldn't really like such, though.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:18:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:18:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185476.334149 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgE4-0007XE-Pj; Mon, 13 Sep 2021 07:18:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185476.334149; Mon, 13 Sep 2021 07:18:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgE4-0007X7-MR; Mon, 13 Sep 2021 07:18:00 +0000
Received: by outflank-mailman (input) for mailman id 185476;
 Mon, 13 Sep 2021 07:17:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPgE3-0007X1-EL
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:17:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 81082965-7031-4a74-a768-32b6370b588d;
 Mon, 13 Sep 2021 07:17:58 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-efV-9HgvNrKBM50lsS371A-1; Mon, 13 Sep 2021 09:17:56 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 07:17:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:17:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0072.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 07:17:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81082965-7031-4a74-a768-32b6370b588d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631517477;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JkupI+z+J+GPRZUv9g1ZG2/sSk1W40uKlMym8BknD0E=;
	b=eK4RMndQEJVtZb2+U8Fiup8TXLIJLKFdguw+uUFYTKkmYOAdOJ8VJAWn5/3HI/7M0uluhR
	iRX0uJPSmlNCOj2XXVO5frwBGZm/IHo80M+cmf9hNLIB48sj/zlLJ2hmVyWtVG53hP+3RD
	09bab/iAYIc3vwbTRju5XUjmD0roAAc=
X-MC-Unique: efV-9HgvNrKBM50lsS371A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OeRBxzYzUzjMK4NSiP34Uk/PcCgaLlmWWALtsohSs97gu8xJZfoIimujkQ83aG7Bl9U9uvLV0Py1kG1SmhkhENRuac72SfZCM43d4cZ5QCEctT8ZroBh6B7P6FXSuRC6HAp6eo5DruganR/FzlCGPXOTClkYUpV+9ubBHT71Pj2+jPs/5BxqQ13fDSgO7pXIKtoiYaUHvd0v2O00SP2/1grexcnmraLckL+OKjTG6isbkBCFuKNpBJffDpBCqhen1oFmXGMttkgQlEWz7Qn3g5nnO3g6/zKyDc0T8MK0EzLepoB+0XHePje0uIbd3/Vgl5aFqtxNJt2DFlDq1tiP+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JkupI+z+J+GPRZUv9g1ZG2/sSk1W40uKlMym8BknD0E=;
 b=lE3/l2qHJ5578JodgGjuzZbrAnE3nNapJG3/AG7YJkAzB4d9/GhPmnpwYGQ7NNi3Ww0kBlCsgzv8iCoqv2DgXYIeiv7o8DesFIZS1mP6HWd6yF3Ll2zmL3Vb1KqXelESjJv9wPP3bMCrROYSzaHXbAZ+EEK3xPRFbZR65KXHYputs+dgrXcdGVF3jRZgFXRxAYaNk4W+eLQkHOClctDDyGzgXJ88r95gwjd5eZi5CR/h7p2rYWQHg76+SjMsg1TIUD5pz0MnNj8Gwg4lDgL0Pgd91qjigzduOwCAwLL+Ipl9MeVsAL2/TspKGM63AyNM6jLJm/eQGX7BfKs0uzG7tQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
 <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com>
Date: Mon, 13 Sep 2021 09:17:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0072.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1d::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ab0a9fb6-a0ff-4534-4bfc-08d976869a60
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4608CB5E39527D57AB5B5DBBB3D99@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ELwgU/qWRg6aa6TUhVNRthVnYd5M3QCagD7iAImUeheRSkFE6UnpKdmivaY3efQpntV66iQBlV08LSO9t8kEPjKapIt98dwDa39WEz5Dy93061ectWzAqzTSJSIJe8xSvHqUIbhOhijODrU9/aFlHt8tTDmVzhT5pK46GiN0rfwSxO9tdL8dUsTh0APV0p9My1oW22jfE2tiDTpnQBTCgipx6H07avu4DAcMCQFJf1FRPENstRg8/XEDTwiTFRE9tZOj1wJs5XfO8CdZsj6f/hlCc9pRGT9H5suqwa93KjyBqdsDqNcP4eFWKEY1EAs4iIC1DIwJ9h2ACVrXN9ZQ6MeFcWv1WSDRyp+1fLEIIH3k/cF2UmQ80mgFyVSb6ksenxxdCRgPeZHIOg2ohYVF8BYj+wOwAl5dsLHDE4tLUfQY+dSJ6ojQg2RZwXLkLi4xAt0IoBt8G0DbknDV49fLkt3p7aIlq3kZdG24HdYcukzUXMcsYWtuoDj8LP7mrXTpkgm3QUUi9KpCugwtsVz2+rvt3UtRUwhQ4kLOtElnaww0WdXsF40JXgy7px8/lAiMi/+Okt7g8raP56GRZkL/FqTjpwd7zOha8Ul2B+637+VAzWtAj7C9LdD6RxoDJXpHKt59NAb0P/mY8e/QTvhsQ1kNrqP5AzTkM9VsG53KPMGxhJrptVDATe5x0R9yhqtleY8rKVPivEP7RvXoJxaT30kcBG4JQK548S648jhH04g=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(346002)(376002)(39860400002)(4744005)(6916009)(316002)(38100700002)(36756003)(86362001)(16576012)(6486002)(5660300002)(31686004)(53546011)(31696002)(83380400001)(478600001)(54906003)(956004)(8676002)(66946007)(66476007)(66556008)(26005)(8936002)(186003)(4326008)(2906002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WmJzYTBiM2l2YjlJUjJHNS9wdlRpekJJK3JScUUrN2hHZ3dzK2NGcW9FY0hw?=
 =?utf-8?B?VHVUb1ZuSU9PblVaaVREL0pDUGFLdHVLNzA1emlhdzdzd0tSVHlXQUxvZnRN?=
 =?utf-8?B?QkNrdnpFSEVsVWNxSmRmU08yVjRJMVFndW9nZVM2ZUNXd3J2U204TERHS0dO?=
 =?utf-8?B?dlZhQ2dVbUdXVjRiMTdKVlV0ZVQrM1Q2b01kUW9oQlFoejQydFNpclBBdUVP?=
 =?utf-8?B?ckwydEM1NEZoT2FxMnpwOEowZFhJSVhoTVppUlJDTDFGbDlFY0hSQVFiMWNt?=
 =?utf-8?B?KzlhTWJYMGF3aE9PcGs5emFVWXVJWjQ3ekFWSXV6ejFQZGs3TjdXc1NyNkVR?=
 =?utf-8?B?bHl6TlROajQxZVV2NmhodDg1bmFYL1k2YzFRKy9KWXdDaVZJTHZLZ1Q4S3F5?=
 =?utf-8?B?QnJwTDZLcm0wcExobnkzSDZZWFhyRUpSSUdBcmVEd3B4ZTJCYmxpeEl2Z3FZ?=
 =?utf-8?B?c2ZOS1lkVXd4Z3k4dldNUDZTdS9udWlpVHB3WmwxVGp0QlIxcVQzWmdRM2Rn?=
 =?utf-8?B?UGgyejNSdUluRC9sN3o3bk1WR3JnSGY1UGgrWWkvL0JPUnFPRUxreGNJa3Rz?=
 =?utf-8?B?U0pFQ3g0NldXa0QyZDlQdGxyTWF6bjJFZCszQ1gzekxMSDB3aGx0WFBWYkFp?=
 =?utf-8?B?YjNXYU45OUlpRVpsWjB0eDFiUUtWUFBmT0lKMis1NTNpU2ZjaENWUTAyUVB5?=
 =?utf-8?B?bTZGbGJHU2VTMHU0QU9wNmwycU1qRmlwa3B0cFhFZklSSDVBbSt0dFlOYThS?=
 =?utf-8?B?Y3F6MjNFTUU3U1hYZWRyV1NONTA3ei83TnVMSjBRelB0NFdMeGRIeU8zdTRT?=
 =?utf-8?B?MEluajRHbHhnUm9vbnhjR3dBTnZVRElJcmw5eDhHdno0ZFkrUFpQQ0pRNGNm?=
 =?utf-8?B?UklhaVJzQkVWYVRsalJWMS9GZk5tVlU4V1lFUUdocVBPL21JREUvR1FaMzl6?=
 =?utf-8?B?cmkyUFFFTGtoV0hqR0k2MkZLN0plWWxBVVFMYXhlY2ZHUEYrUm10SWM5MVNy?=
 =?utf-8?B?Ri83N1NrMUtPUjJMK3JHK0FHejhITU1Hdjh4OHNoVU9CTEExcnhTWU1vak02?=
 =?utf-8?B?S2kxcDlXdXRXSjJtcDRXbGhHNVR0RUNGdk5qRUhrWUFZWFN0d0FqT2NES1kr?=
 =?utf-8?B?UkVoRzk0clpJZGtnWllCVEQ0SFNKTS9RejZJT29zc0g2ck1PY3lDN3B2eUda?=
 =?utf-8?B?OVBwV3d0Vndhb2N4NzM3Vmw1ZWRaRjBOUnZ5ZEVnNXpnekdyR3BIbytlZVNH?=
 =?utf-8?B?WlBqQVlNSkRvL0F2bldUYWJVYnExSXR0UGdxc2prWmszQmR3S2dYVkJKTnd0?=
 =?utf-8?B?TXp4MlE2SVlveDdtWHJWWGhPVWVTQVFjTDZCUGw2blY3YXM5cG55RTAvM21K?=
 =?utf-8?B?MDJYSm9Oa2ZYbDRKNzEwRU9MSE9MVVZINmQzWGhNdTQ2dE1oMlRKWVQ4MEZD?=
 =?utf-8?B?VDBXcGQwb01WdDF1TE95Wk56S1FCNDJDTUovbzNpQkZUL1IyWUVDVGRMYURI?=
 =?utf-8?B?aGp1dmpKYkh5cjhkYXlYYTZQZ2RLeThueVd6RDdEbFVHdllJVjN4OFU4MmNm?=
 =?utf-8?B?Uyt3MFkxTDg0aTY1c1dlNkQrNlQwdHhaS1h5Qm1JazMvb0VrWk1FdG1RaFpI?=
 =?utf-8?B?d2NIN3NjWk5IQlh5cTdoSXNDR1ZwTXp1cDF0enBialYrdXZOK1hYQlpRTkJz?=
 =?utf-8?B?c3FWaVVnVXhvOXhiM1BGRmg1b0NhTW5jcjJJRDV2b0JxamdraGN0ZFFrNldq?=
 =?utf-8?Q?VhKgtOyr5o2Adz7Td6PkvcSoO8XdT/kRWU0jF2L?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ab0a9fb6-a0ff-4534-4bfc-08d976869a60
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:17:53.8262
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MhtZjWEP8Nv4s+v0fhfa2DlDNTNJmu2PStzSgUkCcO8WLmAmDVfjNcM1BZNVNUtReYOQdHiJwg4rdOC8CR89wA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

On 11.09.2021 01:14, Stefano Stabellini wrote:
> On Tue, 7 Sep 2021, Jan Beulich wrote:
>> While the hypervisor hasn't been enforcing this, we would still better
>> avoid issuing requests with GFNs not aligned to the requested order.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I wonder how useful it is to include the alignment in the panic()
>> message.
> 
> Not very useful given that it is static. I don't mind either way but you
> can go ahead and remove it if you prefer (and it would make the line
> shorter.)
> 
> 
>> I further wonder how useful it is to wrap "bytes" in
>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
>> properly").
> 
> This one I would keep, to make sure to print out the same amount passed
> to memblock_alloc.

Oh - if I was to drop it from the printk(), I would have been meaning to
also drop it there. If it's useless, then it's useless everywhere.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:28:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:28:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185482.334159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgOU-0000io-Pr; Mon, 13 Sep 2021 07:28:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185482.334159; Mon, 13 Sep 2021 07:28:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgOU-0000ih-Mx; Mon, 13 Sep 2021 07:28:46 +0000
Received: by outflank-mailman (input) for mailman id 185482;
 Mon, 13 Sep 2021 07:28:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPgOT-0000ib-Lg
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:28:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 96ddebae-61a9-480d-a630-699c6f515f38;
 Mon, 13 Sep 2021 07:28:44 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-EFU8yxbEMSi1f0NA_H9_Wg-1; Mon, 13 Sep 2021 09:28:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5599.eurprd04.prod.outlook.com (2603:10a6:803:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Mon, 13 Sep
 2021 07:28:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:28:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0102.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::43) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Mon, 13 Sep 2021 07:28:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96ddebae-61a9-480d-a630-699c6f515f38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631518123;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hwD0t7QtOjKdwm4YKCqr7HJsRnUZs/jS/v3G2iEWAa8=;
	b=UOSwvnFtCQpvKiQf+q4PRphDdsxizTls0/lrDLUpLqgKjadi/o+E0SQFXnBYwQt9CcoiZr
	/FsnNhNPZ8WnXq4bLhN2ENfqCucL7gQ2Zpp7zeC2A5BjifsyoiMfJTuMwp5poxU8j/u0F0
	MReNolaf+6GdH//QlfgTD4UMRJsHiNo=
X-MC-Unique: EFU8yxbEMSi1f0NA_H9_Wg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HetmQA6G6KQc0Ars8sFJcEgeKzVb+mUw4CCId6XhJzXN4XWw3LyXFEGzISJoIPM3yG7Fz8mPRmA5RAziqWKKtYWMK6z3czxj+5PNJ0bxax+3yDorLrhNETR9oh/ITNgolQuXxgemKrfabs7jeC99f4VJjwATHSHYY1KBXpcfmgYD0tqrXAGD+rwGioNnIgy3QgTU4ir87LDovPLVsG4d1wc3tBWSkR5FDyCOWzmZaU2DY/lHDyWRGCX1lRGIP/RC+AJZkc0PGKxPb7bU9k/QZUI3ERFOi/phZt5LBjTG7DOBc0TaHqovNxuV8dPLBhCveRMnPRY9gV/7M868B6Hr+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hwD0t7QtOjKdwm4YKCqr7HJsRnUZs/jS/v3G2iEWAa8=;
 b=OtQ9DRXvc5RNt1cf3iwJUrakR6XcLKtgHOR5bDSDkmid+QylMRZ+CE/lhH0YR+uMAYQKUqo2fGYEfZKsVzyVxCVrpwFoka9X8iMWowSIPtU4rcG94R9+kNFMkyRukOwDxIjX2gwijmxsI5OzK3ihnxPHcAw0IzivkXHwySs6D0y46dwkNIryxeDuDNjSdF4zvZDYnEFbivDcdxB10X5JoI7vW4VTKg5oVyEhqXecqhdfUGItKXlbUDhHxvLnohCH9hAPl1oBJu0V/7i4IPr5AfNgrjl1G4fXDQi0HhJ52LSOzoRf4FBoWJNsUGyXku08AaMKW3oFV6IxsEhiWh4hjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: infradead.org; dkim=none (message not signed)
 header.d=none;infradead.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 the arch/x86 maintainers <x86@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Christoph Hellwig <hch@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com>
 <YThiyxG0d2tmCtb+@infradead.org>
 <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <543b446b-0143-ad32-99ed-d7a6f79381e8@suse.com>
Date: Mon, 13 Sep 2021 09:28:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0102.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:10e::43) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6a77e551-9d0a-4196-c346-08d976881c14
X-MS-TrafficTypeDiagnostic: VI1PR04MB5599:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB55994C37912100451071B49DB3D99@VI1PR04MB5599.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	06ci2HYkNPNKQake3kequ1jcFFCnILJfHXoHuBTf+qVOu6eE46Iaz4Szw9KZwOZM12VQbsAPAFKdwTNymu4pn7q8S9el7HdpMKwJjABSv3sku9iCALROdg1r6ryI4ocBDbqN2UhdIAUyEJfgow3DtisPvrVyFlNos7RpGscsqn+aPzzvfHSlZQwc8QRbmduQJoWKWXnd5DVEb5wnNXya8yLdHOUkxQSBMajhjZwe3/pshjDnN6WFv0zcvHbeaIRe1GXUM9BFAfA0w6b1/tNLi4Qe7AIZk29fZzOfPmvgAe5tagb99qg5UYs2/no7vWCLVTlvhcQUIvACPUkGRkAa0c7aWdzpXrEnr2+QgKHgxI1SewdN+tq23UPuaJ1WZv1ZdSUqlz5M4MWnuCTlBB2+bNDaKH7UMsYMbasgC3TipqmQDDJlMQ4R2RAJiPqGRHgXK6j5jguYcIdgNP2PWGSnIJ8FMZ8KJYXbPf4pbPeLqDPwIVGg8ZLzy4igIUR5zpT1LT03ZTbmN0YISaem+xZ0m/a/cK0LG90dyYgYCx4QJxUx3kMAQ7pzupi1sPsZwiJh6RfMwsJN4qE/yvNJgd0VD35f67M3vXZMzstag7YhL6/y0AjANDkXj8RJ+x+v37M8B0HAPsbGDnxJRc8Hrla6dcQR0OnRmXEeUdGLwmFbDrZBQWXlgDFY4DKylviIpVICVvfygdk48uYxEXxPwe5jnQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(346002)(376002)(366004)(396003)(66946007)(66476007)(66556008)(5660300002)(86362001)(38100700002)(2906002)(36756003)(31696002)(8676002)(4326008)(83380400001)(2616005)(53546011)(8936002)(16576012)(478600001)(6486002)(956004)(54906003)(31686004)(316002)(186003)(6916009)(26005)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dVhDY1JTaVowYnBCM2I3QVMxTnRGZFhrbzByMFBPQUkrWFZ1U0d3bUhTR2VO?=
 =?utf-8?B?bU5XRWg5NHRDam1aWlkyek9uOGtQTUdLK2hhSnVIQWlTVkJOb09GM0dDZXRx?=
 =?utf-8?B?RkxEd3RZVTY2ME43UFpleGQyRzBBSUNhSEdZMkFIZzlqbFZUVDhvTUoxdWd1?=
 =?utf-8?B?SmRkTmdIR0xmS1dQQnlNazJSTU5aVFA5VzdNRUMrc2s2dXRzbGV5V2ZmWXZy?=
 =?utf-8?B?RmU5VHpHWmdvYW1PUVdQem0xSFNRTTdLQk1tT0tTemhpa29aYmZZbGNIbi9n?=
 =?utf-8?B?Y2xWOVVHeXUwTWFsMDA5eTUrSERzKzltQVJRNnJrT2J3TEFjQTlHcmNUQm1F?=
 =?utf-8?B?ZWtZc2lHR0U5b3F6WitNdWVTeVR2NFlOMFhhNFdGN2VRTlAzanlmZzNWZ3o4?=
 =?utf-8?B?RklnMk4weVU3VWhBY3BMck5KdkliNXFKVERBUjRXUW5ZZ3JpMkxlT2kzWnJz?=
 =?utf-8?B?U21wejNOenNYdFRGV0NJVkNYdjNyeGh0YmQ1VUtBQkxaK29sSVpQdlN5eDBk?=
 =?utf-8?B?M2JVbjlHb243SmdtVmY4OHVzM042T2tyZjRuOGNvemlrNXl2a3R1T3lUSDhE?=
 =?utf-8?B?L29SMmZSVHozQ3dyemlQNzlscm1tYlpLWGdMWEM2bnJtbWV1S2NpT1NGYUdK?=
 =?utf-8?B?Y0JRanloOG9DTTJYTys2OWFmeGtxWFBBeGFlRk1kL2h2SWJoOU1NU2MxZi8r?=
 =?utf-8?B?ejd0RGh2SUtaTjJoRnhST1dCWEJDdzMwanVjTGhlenh4S01QcnFKSzZDTk1C?=
 =?utf-8?B?ZERRZ2tITnpiWDFETG9qVFgrQ1dSTy9WYjJ2cWRJWGd3eksrclRmOVNYRlo0?=
 =?utf-8?B?aHluVzJOeHNQQ2h4cSs3YXAyL2xQUDBncGNueGZ0NnByZ1lNNkNySmR3aXpn?=
 =?utf-8?B?SnNTTThMS1pjSmNqK0Y4dmRMZUhSU2xQV1ZMSUg4WEtxQ2FscWZpV2J3Ykow?=
 =?utf-8?B?Mi82c0ZER2QzdmY0c0p0VlZVS1pqaGkwSDBpdHZtOVNWMTRUVHI4TVA0VkUx?=
 =?utf-8?B?eWVLcW43eDRDS2x6RUhEQjM5YVZHM2MvcTFnWDVJVXMwaXNraXlYYUUwekYx?=
 =?utf-8?B?T0VWeW1HWTJUTXJ2Q3ljTTV6RkJLNXlOaEVGK1daYTJodkZyeVY2QUtIYkV6?=
 =?utf-8?B?d0E5UkdLR3l5QVRHZnRpN1ZCM0JGaDQ3elJPTy9hNVo1MUxzT3UvU0Z5Ullq?=
 =?utf-8?B?a0NWU25sRHF1NGYxZkN4cldoUzdGTHFBeDR5YTFQZERIRVBldHNLcEdFWHlX?=
 =?utf-8?B?VE1uS3dJQUJjU1AxT0RxUjMwdFRVMHdZeXdMRnpKTktyQVJ2NXJZekJCMS9T?=
 =?utf-8?B?RTZwKyt2dm9MSEJLRkJxdHhtYjN1a0JnTEFaZFMxMkIySVBOZGJnYWJzdjMy?=
 =?utf-8?B?OUY5SERhR1RKVjIwelAvdFJyc1owN01yYUttNm85S2NlbGJ4blY0b0FEMFhZ?=
 =?utf-8?B?NnNzTFF2WkE3SUNHa1RWNFV1TC95N0pYOGdrOHVBTnVWWHZESkFhMFVjdjBQ?=
 =?utf-8?B?ajZxdWZCWTE0ekZtUDVUV2pTY1NEa2tGL09ja2QzK29OS0V6YWpqem0vWlVL?=
 =?utf-8?B?dWIrZU1VK2owZ0ZLSmVSMnhFRjhPNkV2NmVzU2dvU05xZEhTTHdUUzRQWW84?=
 =?utf-8?B?SDB0bnVBZUJoN3I2cHBLVHRCSmVFUnl2RnBjdy9JcEZmaUtYU0k4VXlGZ2dT?=
 =?utf-8?B?aitGUFFpci9sbGtSLzArNmVQZ3Q4eE8rVFRoVGxkZWJ0WUNMSzFDNXNTTnda?=
 =?utf-8?Q?g1NDQ/KngzTxu7Qd0+JQk6MpH2NCKMRhohC37RI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a77e551-9d0a-4196-c346-08d976881c14
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:28:40.9527
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qNLKJHn3NU9hMMJ4NQW5/dX7cJzl339+gCX5Y3XuF/IUJSmWkfmGVXJwh9rkBsHQaEvVayZq2cv4uMOz+6Cosg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5599

On 11.09.2021 01:48, Stefano Stabellini wrote:
> On Wed, 8 Sep 2021, Christoph Hellwig wrote:
>> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
>>> The code is unreachable for HVM or PVH, and it also makes little sense
>>> in auto-translated environments. On Arm, with
>>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
>>> time seeing what good the Xen specific variant does - the generic one
>>> ought to be fine for all purposes there. Still Arm code explicitly
>>> references symbols here, so the code will continue to be included there.
>>
>> Can the Xen/arm folks look into that?  Getting ARM out of using
>> swiotlb-xen would be a huge step forward cleaning up some DMA APIs.
> 
> On ARM swiotlb-xen is used for a different purpose compared to x86.
> 
> Many ARM SoCs still don't have an IOMMU covering all DMA-mastering
> devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest
> physical == physical address).
> 
> Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need
> swiotlb-xen. But when we start using PV drivers to share the network or
> disk between Dom0 and DomU we are going to get DomU pages mapped in
> Dom0, we call them "foreign pages".  They are not mapped 1:1. It can
> happen that one of these foreign pages are used for DMA operations
> (e.g. related to the NIC). swiotlb-xen is used to detect these
> situations and translate the guest physical address to physical address
> of foreign pages appropriately.

Hmm, you say "translate", which isn't my understanding of swiotlb's
purpose. As per my understanding swiotlb instead double buffers data
such that is becomes accessible, or suitably arranges underlying
machine addresses. The latter part is clearly a PV-only thing, unused
by Arm as can be seen by there not being any use of XENMEM_exchange.
So it must be the former part that you're talking about, but that's
also the purpose of the non-Xen swiotlb code. If only for my own
education and understanding, could you point me at the difference
between swiotlb-xen and generic swiotlb which addresses this specific
aspect of Arm behavior?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:40:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:40:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185489.334171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgZH-0002QX-WA; Mon, 13 Sep 2021 07:39:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185489.334171; Mon, 13 Sep 2021 07:39:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgZH-0002QQ-S2; Mon, 13 Sep 2021 07:39:55 +0000
Received: by outflank-mailman (input) for mailman id 185489;
 Mon, 13 Sep 2021 07:39:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPgZG-0002QK-Qp
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:39:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dd821b7a-1ef9-4da4-8a97-da4f494b461f;
 Mon, 13 Sep 2021 07:39:53 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-XPGbhdddNJ6x0kzj5owdZQ-2; Mon, 13 Sep 2021 09:39:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5326.eurprd04.prod.outlook.com (2603:10a6:803:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 07:39:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:39:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR07CA0068.eurprd07.prod.outlook.com (2603:10a6:207:4::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Mon, 13 Sep 2021 07:39:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd821b7a-1ef9-4da4-8a97-da4f494b461f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631518792;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hX/RP9X88cA+nkyIzwgkGoiOgzdc/4idb4/dybz09Wk=;
	b=GB+z4jRTCuHmZE50LP/R+jiNQ8x0AwS9b0fuXWJM3w1jPWF06LWrmGagSYEOsUsumeYeDd
	TrcDXtfoZCTE+KDRvrXG5HNHVw1oNY4STz327rzC0qyAr0wNrYLdikreCN9JhZOYlVlabh
	T/lS1yf09SubyOF7ux4YXh/aj0lzqMc=
X-MC-Unique: XPGbhdddNJ6x0kzj5owdZQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NEVnXl4dangeuCCTWbhI5/sAdrpXi+bo4ei0g4k7Zyy6BZuxliLRGZd3S0oR3kzX2xfEmmGDYykb/+8FxrlWZnqoU02X905R4df7rKN/zrj5rhTMm0noAk5+8BkCoP2VcBOytaGKF+VHDfD2ZcpYraroXrXv2IsfOlRMA0L1R8Y7VKeWkYYvoDz4IMmkI8loJ0OXGxV4RrFYxpUoFfx4pUFHY6D6GxKNAJAe82wxE4fbIRK+zf2toyRl+eqqSGVFFOAMuS4WVdDLnfpoYSvJuMwJ/hGfyVJLFkuhcxvHRe24EZFeX40Eu0ou/QT6zB2w+W7IZ4Dicc4aFL9Kc39LhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hX/RP9X88cA+nkyIzwgkGoiOgzdc/4idb4/dybz09Wk=;
 b=JVP0CfIm51EQ8uscUZ5F2Loyy6ruubpNTw7KoDPp2FsxlxgY975dOBYEkNJ5s1Z0FwUdb/Bd71ZMasRRSlOI4d6faF07M/gVd21tQ7bsOzBe56TWQizI7sonu9ytolkOiJSmylI9ND2ykLAMA16MYtm5R2QkM80qnuMEh9xG9nOb+CFyG39PeaYSnDMnAzRnG6erMtxUAn+cFmOIgAwfmrKNqfR1476KvmpjXRGwXSMATMuAI4Z+N6YnPakhVTEqMoCSU+0r/NnxOiANoFFDekU16+Oh9A5hNpsR/IsjPiVWzi1mGQlpL7qnn19V7JsXFaQi9MyrKn+bcJc7DpTFyw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <cf072c09-1a71-5b95-76e6-d10c9c9d1cdf@suse.com>
Date: Mon, 13 Sep 2021 09:39:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR07CA0068.eurprd07.prod.outlook.com
 (2603:10a6:207:4::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2c4caecf-fda8-4369-9409-08d97689a9e3
X-MS-TrafficTypeDiagnostic: VI1PR04MB5326:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB53265B236EAF729DEF51C155B3D99@VI1PR04MB5326.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5P8b5timFjT3R0QAfDbUIR6IVdk3XlUVRuert0HG8nSbUMnz/Fy9xAFwL2XTi9KJgxR89mRogWDFff9pnVquehwRWB5gfw/nJ7uFoitP+CGzAzZrJj1/lOE1GyBC59Q7SeB9dzS9PyiZNEkZLbBYLjdQw5uaPTmsvyovqfvKHZ9xCwuvpE3up51hE1ZXXDTLS0PAI86XkGQrs1SrAYFjrdH+xiSzAjtw/R5xhLx/+QPJjN37VxlFEVR02QLzu6985MqvzryEZedULOqNrck06py5aRC4kmn40WO7RbqOOLUyIBM3UPnVTRnijWlbB6eTRlXPiF2E2jRTLPN408QFsWJHaU7c2eC+6xHg1M9dLDGP4jvUS1o29QF/w/8UuA7oVckTn0lztdL+Vyd9v8JVQfGhDnguufWka7iIPb5SrkvS5y2QzAwdXpMPru5n5w6ugzQOmvpnKUwjAaTJgJu7mjDitJwuUlHn42b37z/ds7MDk5v+zk/3aGH/lorHDLqwcaOG3mu9YjN1ZDihCglmT9u79bF2Bn8TsGlGHxH1KzUCkxffoV0rxoSCDq+19HUwLd7pJG30yVjDgqW2lCzMqf9gGzgzb83jGj6ep0gidM2XEcqNZowNVLPYsbDvl+dzKicxfjU3W83bgJy0Kks+6RxXbYivkKB16TWHTsffhvH0DcA2lr23zUl1hNZcT/lUXtyXytItdkEPnxvjXujMjMeOuHIMXHP6i5Vrzqbbu2kuN/rBQEt1BrjVZ1vAwDTu
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(366004)(376002)(136003)(396003)(86362001)(37006003)(31696002)(6636002)(478600001)(956004)(2616005)(83380400001)(31686004)(53546011)(26005)(186003)(5660300002)(66556008)(66476007)(66946007)(54906003)(8936002)(6486002)(8676002)(316002)(16576012)(36756003)(38100700002)(4326008)(2906002)(6862004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VXpvTmZOTUhzanllbXJLUDlvWUlXeE0wckNUOVpDZ2o1cHhlSk12WkRsYnI5?=
 =?utf-8?B?N0crT3dBWWphZEZpTDRyUlFNcWY0dERhVm1JUThabDZ4MXJqZUtobFlJV1o3?=
 =?utf-8?B?SmVhaTd0Z3FUd0ZRMlJHY2p1STEweEVlOVhaSys1MlBuMUluNXh5Zzl0bmg4?=
 =?utf-8?B?Q2xzM0phaVpCaklGbTRJZnFWalRwMDNnS21BUk53TUdTd3FEUlJ3Y3RjTDl1?=
 =?utf-8?B?UUdjZkM0R2hlOXQ2SWc2MjVCUVQyd1FFTnU5c2Zza0lSOVk0U2JHL3hwK0NC?=
 =?utf-8?B?Y29CbWJ3SXo4MHk4dnlhNUNjZllEZ3B4eG9YUnZHVjJWdSs5dENHaURDeFow?=
 =?utf-8?B?L0NRcU05VW43b2tSVlJTZUJrWGd3NkN5eElQcDR1bkp1SGYxbWk5YzBUQXBa?=
 =?utf-8?B?eW5RSDZES1hvWmhONCthcG92U2g0eGo2MlgvcSt0bEp0OHA0Ni84T1hka2Jn?=
 =?utf-8?B?WGdRSGsvYTRMQnlFbElKaldJVmczZkRrNldrRWYxODBNSE9VMDcyTTNFa2Qw?=
 =?utf-8?B?eVdqM01ySnlKY3hHazF3V1EyYkhyc1FnbUtJeVArSlY2bmpFTDFrZHE4ZkRl?=
 =?utf-8?B?T2VSaHBzdDJobmxtOTduenhEcmF1K25TZEFqVmFnNmx0UVc1bThQaVRQTVli?=
 =?utf-8?B?Kzd1U1E1OUNHUExmS0dXNWc2YTVZRzFoNkdTaWg5TDZ0ZlVFYzRvVjVWeFdm?=
 =?utf-8?B?a1ZzVmsxblgwMWZmc2RrTERUSi9LdlFhSzY2dEZRWUhVN2NZSnlRTENNYlJr?=
 =?utf-8?B?MkpVNVR6ZlN0dUtiTWlueUZyOGFDakRyM0JBbE92czQrQ3ZqYWQ3MEo5K1RI?=
 =?utf-8?B?ZElXYjJZOGZLeWdNN040OXl2ZUlsRlkxM3FNWHMyMEZZRnJ0SGxOcG9WeE1q?=
 =?utf-8?B?c0pVdmQ3OE1RT3VVei9pTzhRK3hPVHcyTE56eVBoYmFSK1ROczNUR0xXaEk2?=
 =?utf-8?B?UldNdWFEYk51ZCtKbWdqaTltOEJJS2JXYW12bHJEbjhHdmhKRnd5TU5sNE9q?=
 =?utf-8?B?L2xqZlcyNDhYYUk1RGRVM3FHVGV3V1BsU0hidDhQeFQwbGhnK1B0SXFZTm4x?=
 =?utf-8?B?V0hsa3Z3QjRJV0w3UHNTK2RUWkZtdkg3RjBvYUFuZE1rb0J3MjNJNDhCR0hO?=
 =?utf-8?B?YnYwYzZDVlNIL25NTHVqeGJPMUtSUkVXbXpEYlBnNW8rZGI3UDNWbGVSYWhw?=
 =?utf-8?B?Qm1zUFdRK2ZjRnhSRXdmTThJTUQrRlRVMWFyRW4xZWFmNVVDb2QvWFVtSlhz?=
 =?utf-8?B?NmpmbGc5NmRFb2pPMjk2akdaUFRWU0VqcFdOR3BjRUJpWkR1TTM2WjRFaDdG?=
 =?utf-8?B?eHNST0M4Tm80emE5NW5VUlZiWUNlUXIvL1psd1Z6THdnSjEzR0dpeGZuUU5W?=
 =?utf-8?B?VDltSDlrazVidnRuVXBOTHhTTU9pSHFZK3VQeFI4T2E1QmJ5UlpzMzU2dHFu?=
 =?utf-8?B?L0p4elIrTi9hRkR0YzlJaG1nR2NXTmhPaDdMcE4xVjdZNlRnem01dzVrTXc3?=
 =?utf-8?B?MFdsV1JWM3Y3cm8rNjFQeVN5UGQvamllcnBDcDRTWUIzdVZPT2N6TkFteThQ?=
 =?utf-8?B?azFTUkxWOHg4dWJjWVQ2Nk5QTnpTV3p2TU00NlRZSU1zY1RZdlNtODZZVzBs?=
 =?utf-8?B?SDZycVpTZ3g0RlFGMHlsSEMrOHlnWUZzR05XT2RXSEY1TVBadC9OQ0s3TnFS?=
 =?utf-8?B?ajNZS0wzVVlQMndveW1ZQW14Zkp5NmNXRnlKa0NTSnBsVXFNaUdQL2JXTkRw?=
 =?utf-8?Q?LEtYSXKxskqh5fbSwX0m5+2LxwhKA5Asar/4izT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c4caecf-fda8-4369-9409-08d97689a9e3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:39:48.3736
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 47cuR7nxBeBH3zKimenFpJwZyCr/OSSgI7pKCdhYK33QNZqilGPsVT78GgJUTKZJnBM/yj8mp8GxECEQRWIl6Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5326

On 10.09.2021 15:46, Juergen Gross wrote:
> On 10.09.21 15:22, Jan Beulich wrote:
>> On 09.09.2021 08:27, Juergen Gross wrote:
>>> Today Xenstore is using one unstable interface to retrieve the state
>>> of a domain (existing, dying).
>>>
>>> In order to decouple Xenstore further from the hypervisor, a stable
>>> interface should be established. This new interface should cover the
>>> following functionality:
>>>
>>> - get the needed state of a domain (existing, dying)
>>> - get a unique identifier per domain telling Xenstore that a domid
>>>     has been reused (today Xenstore could in theory miss this)
>>> - provide a way to avoid having to get the state of all domains when
>>>     a state change of a domain has been signaled (there is only an event
>>>     "some domain has changed state" today: VIRQ_DOM_EXC)
>>>
>>> I'm suggesting the following hypervisor modifications:
>>>
>>> - Addition of a global 64 bit domain sequence counter. This counter
>>>     will be incremented each time a new domain is created.
>>> - Addition of a sequence count to struct domain. It will be set to
>>>     the value of the new global domain sequence counter when the domain
>>>     is being created.
>>
>> This looks like an "extended domain ID", which I think we want to
>> avoid unless exposure is extremely limited. Since we can't easily
>> predict what future usage this may have, I would suggest to seed the
>> global with a 15-bit (or wider) random value. I'd like to further
>> suggest considering to not have this be maintained by simply
>> incrementing, to make very clear to consumers that the actual value
>> has no other meaning. This could be done by multiplying by a random
>> odd number larger than 1 (determined at build or boot time). All we
>> want to guarantee is that numbers don't repeat (earlier than a 64-bit
>> value would wrap when linearly incremented).
> 
> Fine with me.
> 
>>
>>> - Addition of a global bitmap with one bit for each possible domid
>>>     (resulting in a 4 kB page). Each time the state of a domain is changed
>>>     (domain creation, shutdown, deletion) the related bit is set.
>>>     When the VIRQ_DOM_EXC event is being registered the bitmap is reset to
>>>     have the bits set for all existing domains.
>>
>> This page, as I understand from the subsequent item, is not intended
>> to be exposed to the domain running xenstored (or any other one)?
> 
> Correct.
> 
>>
>>> - Addition of a new stable hypercall ("get domain state") returning the
>>>     following information:
>>>     + domid of a domain having the bit set in above bitmap
>>>     + state of that domain (existing, dying)
>>>     + sequence count of that domain
>>>     The related bit is reset in the bitmap as a side effect of the call.
>>
>> What I'd like us to consider up front is whether xenstored is going
>> to remain only entity interested in this kind of information. The
>> entire design looks to leverage that there's only a single consumer
>> in the system.
> 
> Right. I'm just writing some RFC patches, and I have coded this
> interface to be usable only for the domain having VIRQ_DOM_EXC
> registered.
> 
> The alternative (IMO) would have been to expose the domain-state
> bitmap to Xenstore (and/or other interested parties).

Right, but then a simple bitmap wouldn't suffice, would it? What
would be the criteria to clear a bit there? All interested parties
would have to signal that they've observed the respective state
change.

Further, while - like you - I'd prefer to avoid sharing the bitmap,
the question remains whether such further interested parties are
conceivable.

>> I understand the caller would iterate over this hypercall. Is there
>> concern about this iteration never finishing, if e.g. a guest gets
>> rebooted quickly enough?
> 
> No. As a reboot will always include Xenstore activity, there is no
> chance for that to happen.

Is this really the case? I thought that was an implementation
aspect of the tool stack(s). Take an XTF test: For it to be run
(no drivers, no qemu) is it really necessary to fiddle with
Xenstore? If from an abstract pov it isn't, then the hypervisor
should not become dependent upon such. IMO, that is.

Apart from this - how would Xenstore activity prevent this loop
from becoming unbounded? Is this because you expect other
operations to be serialized with running this loop? If so, how
do you prevent starvation from this loop taking long?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:51:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:51:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185495.334182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgk8-0004ta-WB; Mon, 13 Sep 2021 07:51:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185495.334182; Mon, 13 Sep 2021 07:51:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgk8-0004tT-Sz; Mon, 13 Sep 2021 07:51:08 +0000
Received: by outflank-mailman (input) for mailman id 185495;
 Mon, 13 Sep 2021 07:51:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPgk8-0004tN-7X
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:51:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 59fbe4fc-1467-11ec-b3a4-12813bfff9fa;
 Mon, 13 Sep 2021 07:51:07 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-tUFgrveFND-f5Uh5aIyxrg-2; Mon, 13 Sep 2021 09:51:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Mon, 13 Sep
 2021 07:51:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:51:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0071.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Mon, 13 Sep 2021 07:51:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59fbe4fc-1467-11ec-b3a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631519466;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pr9NsquezdntMofNXZDBaoS0vQXjzZVpzsSAdnW8nHY=;
	b=GPS4mIjkpNXl2cG8KJDawjNSxuffZVT7yIp9I5CqV/6onw/+CuTxV0uI/MIh0MJl8Epq+M
	67AKkeCTesiQ3xUyMN1URHtM6e6uM7GB+qEW83GqUXRlnxCaSNx5jzrKZRcmnqnr0g1TaR
	4k5QPNLV5hR+lJ6sMalLVYp7LktJpyo=
X-MC-Unique: tUFgrveFND-f5Uh5aIyxrg-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k1bb3SPEaSiGZXT90apSKba+yGZHwzQ3H6XtkXaaRYppbCwlOxn+TFQCHjYTgtBNsDYdqEBDluIXP3PjRzNFYhw4gbURqaJi+MKfVD1b5m1pyFrcL04hZxB6dRk9m2r33zQxRbLGQr8W7HFeD0xlV6qO2fjux82If7Urybzm18k0CCIkn2eFYaeApPDY+qv2OLifinUrW/tpxYvuvU8EAFYss7zBTvy6lUmNieoBBgDTC3ifu7gmrBBAgz4BFyqP3Xn0FXXwKW2Wyibmb+DrdqzR1xgK/l2X1adfii6+Rz6f2eboa4Ckc05Gi/whzOUzwXAlfhMuM6UKffawLDSDZw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pr9NsquezdntMofNXZDBaoS0vQXjzZVpzsSAdnW8nHY=;
 b=VyzSpuFavPXiofaPaQTAm2I0AYATwVHcfIk024mlV44lOH3iOiz5gOAUE2KZCRYF/tljyI7aF7vXmWdzdq+v5z83Bzw+4B75wVKNNaq9bo8dCmQ6swez/UGJeklZFMsorKHHHcIVh6hI1eve1fXhS1b7QScK9XM4p6lWOHT/E4ZiCV70kQQVpp+oAKYb01V6tkwwBmoxdzgs8U4+xzzGVFjtbogiSFL7juDyGUFrJQFG7Qrx+1e2s0tNvwjcD8JUjPnYBgQlCVEqA5AfzRuAEzd3GKzC6GaD0S4uAKFfDnSZ6vJvmQlK70sl2ypIC9pJivu3t0lwpaflIMAy1BuDlA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: infradead.org; dkim=none (message not signed)
 header.d=none;infradead.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 the arch/x86 maintainers <x86@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Christoph Hellwig <hch@infradead.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com>
 <YThiyxG0d2tmCtb+@infradead.org>
 <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ea90d3d7-22c7-604f-d0c8-f83c2a58554d@suse.com>
Date: Mon, 13 Sep 2021 09:51:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0071.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 762ff284-8c96-4fae-4f5b-08d9768b3be9
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB715048CF70F9DBDFEEAD9649B3D99@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	b+2yabJw7c8ZF+8rLpJfPrxAoSQW7hCpXduq8D3xkfHTG0kS0Vg+W4DxWdjKx8PlAX0CrsQAvk/GJSPwFlik8jJxDsQkfdyEOXZfJuEKEpF2t3ZQyqO05K3T07mxCHDYvUbcvwPjnnUUAOwRXr75mx0tJcw3DLxowPCtAsUJPPFaowoDvv4ORzybh/6jMXCpUcPA9A9pGtANJednsfKm1twYJA4teXDlj0Iaa9XLacmE41e5TUj9b0cy/jiqFUqurZwro71NMBUluRVSgY9weUsgD09YjWQKOAs/T57ioDuQ2zVGnQUGtxDGixWrHn7vSx9mMjN9YtGXDIJXiLLrbF12GJhbgSv3TbIZCnsiMiO1v/RjZm5veFcgT4a4gD4+xAl8zpLMa3MT6n9i3kxzGKyWQBM5jqTT+hhwzA50korrfeRYVABPGplqWUvp0OBLzpOfIAUS2mtzLI8jfpjmhULnPrxB0SKe5C0WZyhO+lRk6Jesc4aPXff5bmKk53fFLr1euJ8i0ooSc1xu0LMkJuBvOT8q+M3jVzTNYyluI4NG10OCuTnO74WjfILuAFCrjO3ZYks1u/Ud5nMXbqXIHIU1kRT90qPSnREWI57/Ty59+xUElq8PB6f6XDA3WKlskce0GvoYpS23II+Ud+YbLGy6RI4FnNWBKASaCaIM3KDmwKA9OXWXE3DgmzEVc/sLmfE/g6XO/eiYusRSUQtePEaoZUw9XIfU619+RB2V1ouNauPm534S5w45rxVoRstE
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(346002)(39860400002)(376002)(396003)(8676002)(6916009)(66476007)(31696002)(86362001)(66556008)(186003)(38100700002)(478600001)(83380400001)(16576012)(36756003)(2616005)(316002)(26005)(53546011)(31686004)(8936002)(4326008)(54906003)(5660300002)(2906002)(66946007)(956004)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Rno4YldhTnFxd0RpV2E3TDNqSnlDMWFEMitlb1dYQUdyVHp2elRhVzRSeEZM?=
 =?utf-8?B?ejNMa2E5d085UGNwVDVIVDdrN1VzdStndWlzZ0lBOVc0enRBWlZFcndaQTNJ?=
 =?utf-8?B?NFd4VzFNSHFocmd4YVZlYjg1NWZJbjNRVnZYVE9DbUVIVVBFK0hvZlBQYytr?=
 =?utf-8?B?Vjh0UDV0MDNnNCtHeXBkVFZURUtBVkFSMXROK0lJb2dpTlVhclFMeUxBREw5?=
 =?utf-8?B?QUlVekJ4bnYxMzNJNHZka00vUjVjYzM5QVAvVlJSOFNOUWh0dGFmMWw1ZzZo?=
 =?utf-8?B?ZURiemhleVJFZ24vWDhpTTVmbzNuejhDK3BFdVdaTzkyNDQySzNBallQdTMr?=
 =?utf-8?B?N0dxTFVjS205bnBxYTFSeW1VV3owV01JY1NOTE8xV2dBMkhKeGtEeWdOSUNF?=
 =?utf-8?B?Y3grYVRScXVNcFZYbHhwSFhrOS81MERJODA1alRsVHhVOStoNGhRcGlmdVh3?=
 =?utf-8?B?NDRqWlZLanplS1B4aGRZb2ZIZ0tSS3NzZEp3R2h6aDNDam52OUNlUDlUM0tp?=
 =?utf-8?B?WXAyNTh1Ulpwa0s1RjBHb0ozOUJiLzZVcE1KOXFqc3hSN2dQR3UreU00QWNY?=
 =?utf-8?B?bGpxeGJ4UWU1MzlsK0dIWE45Y3JFc3RROGpLbTI4UGVmb1h1SEFvdUZxd3k4?=
 =?utf-8?B?TEJscTFNNzB4Q05YdjNZNE81OXo4UituWUtON0kzVVd5MHpYSUZJSVUxcXdo?=
 =?utf-8?B?MnQ0anpWSGY4dnYwYk91YlJ2VVNlZmdCQm9nMFJPSThROVF0NmpMalFNejB4?=
 =?utf-8?B?SE50SmhacE9WRENYSFVQMFBPUjVHV1dVQ3NXNkVDNWdRamxWVlRyNVIvVHBX?=
 =?utf-8?B?T0E5TzJLaGZ1NjA3NWg0SElMRHA0Yy9NdWE3TWJqSkpudm8vOUgvbnd5dGEr?=
 =?utf-8?B?emYzb1FHYWc4M3VBZUNkL0JacGtOVm9va1B5QWo2emg2WkgveE1ndTFsQ2l1?=
 =?utf-8?B?M0FSTGJROVd5Y0xydFJQQlQzUFVVQ1hRQll2dGoyckFzYk5NVXkwTnJxakFI?=
 =?utf-8?B?VFJ1and0eWhlK1owNUZ2UEwva1QyMGRnVWRjbzFWdzdBcjlXWHFLWktWaGdV?=
 =?utf-8?B?MTZBcG45S1ZTU0p5SEMza3R6QVhFclBQOXV5NnVpOWl4QXRYV2VxTiswY1lq?=
 =?utf-8?B?RlNRdURHSzdWSjZIYittTTFRUGFpSVV0MWxYOXJQeFZKTnBzQURaRlh1NDNj?=
 =?utf-8?B?MThGeXFCemtBOEMrUjEyVlhNdjR5T0ZiUTBzWnJ1bCszNVFzV0FuT0Zrb2p2?=
 =?utf-8?B?b2kvSjZaS2Q1aFFoRUo1SnRxbi9WTW5ablowWVFzZ3JzeG1JR1pJZFNybUFT?=
 =?utf-8?B?RUdqZnBkM2Nrb3kxTEJTYURETFFUbzQzaGU5bzJoMVQxeDNjWjh6TmkxL3Ur?=
 =?utf-8?B?OG5Yd0JZTTNCM1BpbHNkdm5QZkRtemRCOXE1TytCUllsM3ozMU43S1R1dVJ3?=
 =?utf-8?B?dDJpQ0MzdlFSeFNpNzRVK1hrK1dTTEJFdVYybFc5RjFVRUVOWEpDNjFrbmla?=
 =?utf-8?B?R3RSRzF2MXdhSnBvMFRRUVVTS1BZR0tULzZxSjlvSGp1c1E0L1FVRGZJcUNz?=
 =?utf-8?B?TjJZTkRvcS83ak9xVmo4dmMyTGx4d1hDYXdubjNpdGZCb0orZWFZcEdlZ0VF?=
 =?utf-8?B?RmErYkF6ZGR6V3plS1MwSzNmWCtsWjFrZHIzUXlkZWtQbGFPSE9YUXBTRFB5?=
 =?utf-8?B?MXVhb3V5OEowNXFqaFJkMWtvaXJMMlkwYnpWZE05K0grd1d2dURrUlNxQkQ1?=
 =?utf-8?Q?3IAHzROaXVsaqw1M9hZ7yKSOzskEBd30wJc6e8E?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 762ff284-8c96-4fae-4f5b-08d9768b3be9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:51:02.8362
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M95ReYbmDRFLD9uHvKcfF6HUVD4FKx1m0TyUAS1zPv5row9eF3rfID8qrUFRe7yEoNJtUInFFbi64SL76l7Gig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

On 11.09.2021 01:48, Stefano Stabellini wrote:
> On Wed, 8 Sep 2021, Christoph Hellwig wrote:
>> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
>>> The code is unreachable for HVM or PVH, and it also makes little sense
>>> in auto-translated environments. On Arm, with
>>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
>>> time seeing what good the Xen specific variant does - the generic one
>>> ought to be fine for all purposes there. Still Arm code explicitly
>>> references symbols here, so the code will continue to be included there.
>>
>> Can the Xen/arm folks look into that?  Getting ARM out of using
>> swiotlb-xen would be a huge step forward cleaning up some DMA APIs.
> 
> On ARM swiotlb-xen is used for a different purpose compared to x86.
> 
> Many ARM SoCs still don't have an IOMMU covering all DMA-mastering
> devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest
> physical == physical address).
> 
> Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need
> swiotlb-xen. But when we start using PV drivers to share the network or
> disk between Dom0 and DomU we are going to get DomU pages mapped in
> Dom0, we call them "foreign pages".  They are not mapped 1:1. It can
> happen that one of these foreign pages are used for DMA operations
> (e.g. related to the NIC). swiotlb-xen is used to detect these
> situations and translate the guest physical address to physical address
> of foreign pages appropriately.

Thinking about this some more - if Dom0 is 1:1 mapped, why don't you
map foreign pages 1:1 as well then?

>>> Instead of making PCI_XEN's "select" conditional, simply drop it -
>>> SWIOTLB_XEN will be available unconditionally in the PV case anyway, and
>>> is - as explained above - dead code in non-PV environments.
>>>
>>> This in turn allows dropping the stubs for
>>> xen_{create,destroy}_contiguous_region(), the former of which was broken
>>> anyway - it failed to set the DMA handle output.
>>
>> Looks good:
>>
>> Reviewed-by: Christoph Hellwig <hch@lst.de>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Thanks for this and the other reviews.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 07:54:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 07:54:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185501.334193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgn2-0005XW-EM; Mon, 13 Sep 2021 07:54:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185501.334193; Mon, 13 Sep 2021 07:54:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgn2-0005XP-BA; Mon, 13 Sep 2021 07:54:08 +0000
Received: by outflank-mailman (input) for mailman id 185501;
 Mon, 13 Sep 2021 07:54:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPgn1-0005XJ-4W
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 07:54:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id aacd3fa0-6a5d-4fc6-9af7-85e8a96afcda;
 Mon, 13 Sep 2021 07:54:06 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2058.outbound.protection.outlook.com [104.47.4.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-OVERijrXP2qmc_dFFo24kg-1; Mon, 13 Sep 2021 09:54:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2448.eurprd04.prod.outlook.com (2603:10a6:800:52::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 07:54:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 07:54:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0045.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 07:54:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aacd3fa0-6a5d-4fc6-9af7-85e8a96afcda
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631519645;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=oyitoaSUmmKQXuQt8LX+L4zVmOF8/FORFu7eBvzwIt8=;
	b=Da+SpHajWkqxifr29gUkBl7WNCFj3PJcduV3Z5iNqjF824jW9AuUG0oCD0vByJLLFa7mHL
	9iEtSWZ3kPPlS3nLIYKlRrsLYleszxNIE5GRF5XS+JHrlbVvhIBKtLkoEGsR7gYGnnAcSm
	85/uAO+5r09vv3Q2L26BNUqJbgsStXM=
X-MC-Unique: OVERijrXP2qmc_dFFo24kg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=C/Gu7hZ7xfvVe7ZKlC05a+eFu8XlPNaMq3IWKDfe2gC1Vscm+d/bd/yII0rIWtjxv+vZw4PMWF8ugEPNpr6j0wuNo4CTuqdiZi2oH7+rRBV5/TYpK4XlL3U1d2IkczjSWjY8eq2DrLZG/a9VF6kA5rHhQMcB422wm1beNaGWdQdhDYSx1I9uZOppmN+LyPtbAFDV/cVTrzS3FKPcw8ew/e2xHCWMMLOCA2rBZGcHkRIWb+AG/+bk8xiwQZsifn7CZqhbf/lpbj5t7oHnJOaccsnvlG5vjp/NnTLryiX0Z3oylwNaiSIwA7k9He1x7dIWqxeQ9NWjTERZnEjn2kqkYg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oyitoaSUmmKQXuQt8LX+L4zVmOF8/FORFu7eBvzwIt8=;
 b=COfuoosgniN+lUHyUfRjMP0Uip2jRJSinMogZ4RKuSqfznsch3RdcnERGDrAKUAy2OO52lQLSkAnAw1sM9WUZ3GN2H+p8J0r0PbLd8gspKidUsmQxKOfur3nOjB78eaptctT454f+wGcvgjf3Rqr7aiXXr1OwNouSWJDdOeaJNTOm3Uxqme6Ii2XHYQE1cEdDQBeaAkCMMWc/9y2Lq2VMhqfjdQEt5tFQ8kUn0QHFo6w/vC3ZYZZaQOTFp0t9gNbsv4H9sy2l3UJqfl4A6twr7DPIV6eKC9Yb+J2f3n2Qx8D6PeUm0jA7jwvuzgAwJ4OEY/yIlMx6BDr5hH9G2N4pw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/ACPI: ignore processors which cannot be brought online
Message-ID: <370f1ebf-b8d6-70f2-b9c5-d642f710c550@suse.com>
Date: Mon, 13 Sep 2021 09:54:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0045.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8e07178e-2b1e-44b4-9dac-08d9768ba760
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2448:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2448B215A03DDE7E5AED8AC2B3D99@VI1PR0401MB2448.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:949;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4CvPon3jrgMUD1y+aiTFPXYMuNtKsP8dUFa5/pKgfpcVYygqCFf/UtqfXpOSDwQfROfNKUy3lZDNAqTED2hrJDuvAhjAG5nqwDISDee6cC9s4I7X72HXT4XO3ZCqdohXO74gNPgF1ZLnzaD+sQMjWYg1L7xxrP3ikm4SEkJb/sVEXxhlsFrL9xVLFQFbocZqD6WK18gqrHUr5g2mpRY0yuVLvFRfbeuDe8NpmUHIwgRwsMuDoTvZ9TrLtl2OQ2D3efPV0eGLtuc6rQf8H5mFDrZFYyjx0SO72qiai8AD/0BM5qop1jZDZlyIVcikpbH4tgF4Rqels6xNFr7JZ8xnk5B68+LxwIargeO8bZnE+rKRreluAW7FnCVgLt173jKt35CbJSBiWo3ZZd+mwAe0C9RFj7dpsZzMvzQsFywfrjcn3UguSL/1DwmrQxMveXWq6XXicbrE26AdtR7KXSL9qW9eomNIf28QUbGLiau9G2YiWrVNFxnHr4WLthcotO/lem97YjkW+TQkQHNWg9VF1sU9vaKloolutayS/j5cMwntcZEaN3K+k59d6hrnSIfPjuEdwFLnCqU9vG5RtKgWInpzRWH5/Uyl3SihOvcOFI864OZKZvfs/xeHQzzgh6Ybq1T1Dpu3t9N3WSuux7GtUspDEBzgs2Bh0wQZstM3r9irjGyHrpchCVUtfcxLNamH1/yY3N5tA441fPDQtDWcudz+Euf+JB4kLmXP/YypElscMdJjZFig4CcRU/Vv8lex
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(396003)(346002)(366004)(5660300002)(956004)(2616005)(186003)(36756003)(83380400001)(8936002)(54906003)(38100700002)(66556008)(66476007)(66946007)(478600001)(31686004)(316002)(16576012)(31696002)(4326008)(26005)(2906002)(8676002)(6486002)(86362001)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R3NIMlRjNTErL2YvLzdjNnVmWjZubmxlcFduRDlEbHh6Qk9Ra25RTzdiSjV4?=
 =?utf-8?B?RkQ0RDdIL25YREdjUjk1MS9Hdkl3VHZwQ0tWRlRTQ1VKWkRidmt5NXFEQW82?=
 =?utf-8?B?NzlFY1Baa0swbjBocHMzcXBEVFc4M3RpQk5LT3Vlc256bVJENEhaZ3VEdDMw?=
 =?utf-8?B?cXYwSy9aWXdYYitZdWpOaWEvajRKU05WSncvUjIrTTkveW1pMm53V0JJMFZC?=
 =?utf-8?B?bXMrNXl1TzRiSGxlQkNGbm51eXhFZkdQSGFMYVQ3eXUxdE9DemdObWduOFlF?=
 =?utf-8?B?U0ZGZGNaSEdnZml5MVdySEE1cUJoRi92V0JGUU0rd1NITzROdDQvbkl2VFRt?=
 =?utf-8?B?SXRINFV3RjF1R1pLTE9xNnJOOThyc1VlcXA1OVhCU0RMZkVOWkwxWlNEQmJE?=
 =?utf-8?B?ZUtCZDRjK2dncFZNNVNlSVc4cFdHUkc2WDhvRUFsYkZTeDFYWDR0cStuYjJy?=
 =?utf-8?B?WUVJL1E3aEtGSUc2UUZjZnNkQWM0TWhjYjQzbXMrSHFRVnd3MkdMWG1tdDZn?=
 =?utf-8?B?MU9zcnZHVGhEODBLQTZocUtOU04vWTl4YzdwMTNsVE81VDE2UFpvbHl3Z1ls?=
 =?utf-8?B?cldyVnVoaHZLaklKY1FZSWk3clBzQUllNm8rdUJKS082QVcrRmxBV3NWbFVX?=
 =?utf-8?B?ZUNpdkZKa2FKdzVQSnpFRWYzbTU4UEw3VzNvVE5JTEFmYkhlNnVpeXhhekxm?=
 =?utf-8?B?cmtvSVVwTlNSS05COEx6UmxFZ2g1aloveXFKRHRhR2w1Sy90VkFXcjhBdi83?=
 =?utf-8?B?aVc0S3d5K0hsT0J5UkpYc0FIYW45bVowZHVqMHZhUmdZM3k3Rmd6cTJZT2l3?=
 =?utf-8?B?UTlNSnB5ZFpZVHFEOXRoRndsclRjT0xIZzhZU240UFhxYlRwWFk4dFRKZDJ5?=
 =?utf-8?B?TmRYQ3I5b3k5b1c1bmk4d2hydWMreWhiQW15Z21TL1o2NC9EVU5EMEtMbitv?=
 =?utf-8?B?YXhWRUdOUVhiZUFWUVJCbExaSU1leWNXbWprSGZOZ2E4VXpnVUhaUVp3eDQ2?=
 =?utf-8?B?WjgxcVE5NkNnU3ZxVGN3cy9QZEFsY0lTdm0rMzZUMnhjTFF2bnV1UmFkdjFw?=
 =?utf-8?B?UjFkRzUxNUlIVFZvd2pnc041RFUwbFZtc3JST2R4WnhiZlVoT1dqZk02cHUx?=
 =?utf-8?B?Ny9aaHBBRWVJL3ZLd2h0S0JEK0lnTWdnLzVNWVN6dmxOL0ZBQ0x5QUYzNEE5?=
 =?utf-8?B?RzdBN3lIbVpSOHhZKzhwOTlEVjRMV1ZjQ2hIWHVNbGt6N1VscEhmUTg3d0Ux?=
 =?utf-8?B?MitycHF3SW1GdGpIeldRUUhRQXRZWFBLNTFOY0ZUYSs4KzFUL0JBaG9CQm85?=
 =?utf-8?B?bFZTTEFRNzNiT245cW9HcWkwSmxBRkNWck9ESlY3ak5OQmIrQ2FWQUxtY21B?=
 =?utf-8?B?UVdDdmR6bC9vUy9MdHdCaHdOdWZnVjduamh2WmpEREI5bVp1aDVTNklnNDhB?=
 =?utf-8?B?YlRaQW5VOTRWZHVkOE5kS1EwRVV6R2F0a3hEa2szMWVGSnBPaVpneFZUNmpX?=
 =?utf-8?B?S08vNmFSd2lXZDZ0WWJYOWVBU2dUVFBEREZHOXphYkdDK3QyVkZvK3Flb1lK?=
 =?utf-8?B?K1J2aTRRRFVXc3AxSXpEcWNDeEJlOWwrWlQ5M3JsY2x5dlZXbmpoamlZZDRz?=
 =?utf-8?B?R3VFUDJZZDlreXl5TnpMK05qazBrNytzeVBGek1wM3JRSGRyQ05FaVN5bE9J?=
 =?utf-8?B?SjNRVS9OVTI1WHJjTW5SbzE1RjE3elU5bEwrdE92NERTK0wvaE5mdkRvMDlK?=
 =?utf-8?Q?3JYvw4hEc98RwaLy78O0BgMQ5oGRAcU9plDzrZE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8e07178e-2b1e-44b4-9dac-08d9768ba760
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 07:54:03.1293
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e3N++RcdBRXHbGFh/EE1uE5M8Rkg2gEJdOx/yVpN3tztpMHZk8jxWzH2DIBn6A8PWeMQPrNj3RHQuemYlPYOrQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2448

ACPI 6.3 introduced a flag allowing to tell MADT entries describing
hotpluggable processors from ones which are simply placeholders (often
used by firmware writers to simplify handling there).

Inspired by a Linux patch by Mario Limonciello <mario.limonciello@amd.com>.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/acpi/boot.c
+++ b/xen/arch/x86/acpi/boot.c
@@ -53,6 +53,8 @@ bool __initdata acpi_ioapic;
 static bool __initdata acpi_skip_timer_override;
 boolean_param("acpi_skip_timer_override", acpi_skip_timer_override);
 
+static uint8_t __initdata madt_revision;
+
 static u64 acpi_lapic_addr __initdata = APIC_DEFAULT_PHYS_BASE;
 
 /* --------------------------------------------------------------------------
@@ -64,6 +66,8 @@ static int __init acpi_parse_madt(struct
 	struct acpi_table_madt *madt =
 		container_of(table, struct acpi_table_madt, header);
 
+	madt_revision = madt->header.revision;
+
 	if (madt->address) {
 		acpi_lapic_addr = (u64) madt->address;
 
@@ -86,6 +90,12 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	if (BAD_MADT_ENTRY(processor, end))
 		return -EINVAL;
 
+	/* Don't register processors that cannot be onlined. */
+	if (madt_revision >= 5 &&
+	    !(processor->lapic_flags & ACPI_MADT_ENABLED) &&
+	    !(processor->lapic_flags & ACPI_MADT_ONLINE_CAPABLE))
+		return 0;
+
 	if ((processor->lapic_flags & ACPI_MADT_ENABLED) ||
 	    processor->local_apic_id != 0xffffffff || opt_cpu_info) {
 		acpi_table_print_madt_entry(header);
@@ -136,6 +146,12 @@ acpi_parse_lapic(struct acpi_subtable_he
 	if (BAD_MADT_ENTRY(processor, end))
 		return -EINVAL;
 
+	/* Don't register processors that cannot be onlined. */
+	if (madt_revision >= 5 &&
+	    !(processor->lapic_flags & ACPI_MADT_ENABLED) &&
+	    !(processor->lapic_flags & ACPI_MADT_ONLINE_CAPABLE))
+		return 0;
+
 	if ((processor->lapic_flags & ACPI_MADT_ENABLED) ||
 	    processor->id != 0xff || opt_cpu_info)
 		acpi_table_print_madt_entry(header);
--- a/xen/include/acpi/actbl1.h
+++ b/xen/include/acpi/actbl1.h
@@ -858,7 +858,8 @@ struct acpi_madt_generic_translator {
 
 /* MADT Local APIC flags */
 
-#define ACPI_MADT_ENABLED           (1)	/* 00: Processor is usable if set */
+#define ACPI_MADT_ENABLED           (1 << 0)	/* 00: Processor is usable if set */
+#define ACPI_MADT_ONLINE_CAPABLE    (1 << 1)	/* 01: Processor can be onlined */
 
 /* MADT MPS INTI flags (inti_flags) */
 



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:05:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:05:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185518.334204 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgyD-0007wT-Sw; Mon, 13 Sep 2021 08:05:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185518.334204; Mon, 13 Sep 2021 08:05:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPgyD-0007wM-PC; Mon, 13 Sep 2021 08:05:41 +0000
Received: by outflank-mailman (input) for mailman id 185518;
 Mon, 13 Sep 2021 08:05:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=huvW=OD=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mPgyC-0007wG-Pv
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:05:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 61e458aa-1469-11ec-b3a4-12813bfff9fa;
 Mon, 13 Sep 2021 08:05:39 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5A8E421FA1;
 Mon, 13 Sep 2021 08:05:38 +0000 (UTC)
Received: from imap1.suse-dmz.suse.de (imap1.suse-dmz.suse.de [192.168.254.73])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap1.suse-dmz.suse.de (Postfix) with ESMTPS id 203B213659;
 Mon, 13 Sep 2021 08:05:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap1.suse-dmz.suse.de with ESMTPSA id nwFRBlIGP2HXCgAAGKfGzw
 (envelope-from <jgross@suse.com>); Mon, 13 Sep 2021 08:05:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61e458aa-1469-11ec-b3a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631520338; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=2dfjrvmvc3Y2ulpeZSZLUXAHX6/eqhBEVG8YAU18fkA=;
	b=Pz2KtEvL5sQy/5SD/nrYKaEA2lQNaqJmIUzGT2gcGRhawIEp6hnALaGfFNuRnU9f282Y46
	dA9qJTSZuYkyRFdS52wJPRbXHUO+hvvTz3BuMHQlNwE/9BAo81GgGBRBQ3rEb5tqZYrgIM
	QtbgB0FzV7YvY0Rni7Ggo0Jpm/JO9MA=
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
 <cf072c09-1a71-5b95-76e6-d10c9c9d1cdf@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
Message-ID: <284bc539-6234-70f0-be93-92f40655dced@suse.com>
Date: Mon, 13 Sep 2021 10:05:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <cf072c09-1a71-5b95-76e6-d10c9c9d1cdf@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ToNKcA92ly3P5i9R9jeMiy0cehp5snm5g"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ToNKcA92ly3P5i9R9jeMiy0cehp5snm5g
Content-Type: multipart/mixed; boundary="TJUbi4qzMZBHDy5k73W36h7zze81OrKt3";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, George Dunlap <George.Dunlap@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <284bc539-6234-70f0-be93-92f40655dced@suse.com>
Subject: Re: Introduction of stable interface between Xenstore and hypervisor
References: <f7c54683-75d1-ded1-54d6-823dd76db030@suse.com>
 <a192a226-bdfe-6db2-c501-5e5e17c74de0@suse.com>
 <fbb173f8-f8a9-7eef-0e7d-f18dcbb422f3@suse.com>
 <cf072c09-1a71-5b95-76e6-d10c9c9d1cdf@suse.com>
In-Reply-To: <cf072c09-1a71-5b95-76e6-d10c9c9d1cdf@suse.com>

--TJUbi4qzMZBHDy5k73W36h7zze81OrKt3
Content-Type: multipart/mixed;
 boundary="------------D5DF2817921FCAE9EE809A44"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------D5DF2817921FCAE9EE809A44
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 13.09.21 09:39, Jan Beulich wrote:
> On 10.09.2021 15:46, Juergen Gross wrote:
>> On 10.09.21 15:22, Jan Beulich wrote:
>>> On 09.09.2021 08:27, Juergen Gross wrote:
>>>> Today Xenstore is using one unstable interface to retrieve the state=

>>>> of a domain (existing, dying).
>>>>
>>>> In order to decouple Xenstore further from the hypervisor, a stable
>>>> interface should be established. This new interface should cover the=

>>>> following functionality:
>>>>
>>>> - get the needed state of a domain (existing, dying)
>>>> - get a unique identifier per domain telling Xenstore that a domid
>>>>      has been reused (today Xenstore could in theory miss this)
>>>> - provide a way to avoid having to get the state of all domains when=

>>>>      a state change of a domain has been signaled (there is only an =
event
>>>>      "some domain has changed state" today: VIRQ_DOM_EXC)
>>>>
>>>> I'm suggesting the following hypervisor modifications:
>>>>
>>>> - Addition of a global 64 bit domain sequence counter. This counter
>>>>      will be incremented each time a new domain is created.
>>>> - Addition of a sequence count to struct domain. It will be set to
>>>>      the value of the new global domain sequence counter when the do=
main
>>>>      is being created.
>>>
>>> This looks like an "extended domain ID", which I think we want to
>>> avoid unless exposure is extremely limited. Since we can't easily
>>> predict what future usage this may have, I would suggest to seed the
>>> global with a 15-bit (or wider) random value. I'd like to further
>>> suggest considering to not have this be maintained by simply
>>> incrementing, to make very clear to consumers that the actual value
>>> has no other meaning. This could be done by multiplying by a random
>>> odd number larger than 1 (determined at build or boot time). All we
>>> want to guarantee is that numbers don't repeat (earlier than a 64-bit=

>>> value would wrap when linearly incremented).
>>
>> Fine with me.
>>
>>>
>>>> - Addition of a global bitmap with one bit for each possible domid
>>>>      (resulting in a 4 kB page). Each time the state of a domain is =
changed
>>>>      (domain creation, shutdown, deletion) the related bit is set.
>>>>      When the VIRQ_DOM_EXC event is being registered the bitmap is r=
eset to
>>>>      have the bits set for all existing domains.
>>>
>>> This page, as I understand from the subsequent item, is not intended
>>> to be exposed to the domain running xenstored (or any other one)?
>>
>> Correct.
>>
>>>
>>>> - Addition of a new stable hypercall ("get domain state") returning =
the
>>>>      following information:
>>>>      + domid of a domain having the bit set in above bitmap
>>>>      + state of that domain (existing, dying)
>>>>      + sequence count of that domain
>>>>      The related bit is reset in the bitmap as a side effect of the =
call.
>>>
>>> What I'd like us to consider up front is whether xenstored is going
>>> to remain only entity interested in this kind of information. The
>>> entire design looks to leverage that there's only a single consumer
>>> in the system.
>>
>> Right. I'm just writing some RFC patches, and I have coded this
>> interface to be usable only for the domain having VIRQ_DOM_EXC
>> registered.
>>
>> The alternative (IMO) would have been to expose the domain-state
>> bitmap to Xenstore (and/or other interested parties).
>=20
> Right, but then a simple bitmap wouldn't suffice, would it? What
> would be the criteria to clear a bit there? All interested parties
> would have to signal that they've observed the respective state
> change.

For this kind of interface to work with multiple consumers the state
information of each domain would need to be contained in this memory
area, so this would probably require at least one byte for each domain.
Each consumer would need to keep a shadow copy of the last read
information in order to be able to detect any state modifications.

But this data would again introduce today's problem: a fast shutdown
of a domain and creation of another domain with the same domid could
get lost. So the memory area would even need to include the unique
domain identifier, increasing the size of the data per domain to at
least 8 bytes.

> Further, while - like you - I'd prefer to avoid sharing the bitmap,
> the question remains whether such further interested parties are
> conceivable.

A good question. Today's design doesn't support multiple interested
parties at the hypervisor interface level. There can only be one
consumer of VIRQ_DOM_EXC.

>=20
>>> I understand the caller would iterate over this hypercall. Is there
>>> concern about this iteration never finishing, if e.g. a guest gets
>>> rebooted quickly enough?
>>
>> No. As a reboot will always include Xenstore activity, there is no
>> chance for that to happen.
>=20
> Is this really the case? I thought that was an implementation
> aspect of the tool stack(s). Take an XTF test: For it to be run
> (no drivers, no qemu) is it really necessary to fiddle with
> Xenstore? If from an abstract pov it isn't, then the hypervisor
> should not become dependent upon such. IMO, that is.

In case you want to run without Xenstore then don't start it. In case
Xenstore is running it will receive the VIRQ_DOM_EXC events and it
will react accordingly.

The unbounded loop can happen only if you are running Xenstore, but you
are creating new guests by not using the normal Xen tools (those will
interact with Xenstore for all guests, even those without any external
references). And even in this case all you would get is quite a busy
Xenstore.

> Apart from this - how would Xenstore activity prevent this loop
> from becoming unbounded? Is this because you expect other
> operations to be serialized with running this loop? If so, how
> do you prevent starvation from this loop taking long?

Xenstore is single threaded and it is processing any external events
from a main loop. So either it is in this hypercall loop, or it is
reacting to requests, e.g. from the tools needed for domain creation.
The activity in this loop isn't very time consuming (with the new
interface it is even much less time consuming than today), so the
starvation case would more look like a hickup.


Juergen

--------------D5DF2817921FCAE9EE809A44
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------D5DF2817921FCAE9EE809A44--

--TJUbi4qzMZBHDy5k73W36h7zze81OrKt3--

--ToNKcA92ly3P5i9R9jeMiy0cehp5snm5g
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmE/BlEFAwAAAAAACgkQsN6d1ii/Ey+h
vQf+IAr7Llpb0m9E2fyL4/LBGMW8kg8Z2b/zhwe5DHBul7LrFs2U+DcfYHM00QnotELWdkITvy13
aaPqkHygVBoCrRz7N7ny184KmxgeiBWbvUR4L97+IOHuArNKBT6ZLCpD70DutgtuO8BnXNiS0Xtr
LGbv++VGOey57KUv51ywzHo1McwlAHwIbUlr/umJuCrJeMWacJQMq4Jp3+7Ntfjv1sIofgEOKoRf
RdY9/EV30ZtDbRRR5NCCM4ni8zACSCC9J698KQw0VTI/Pg7FQBOk/n96zZjOxXQITBJFhPxJg68J
yPtAD1e4/kKmd55c/X1pmQs0Mdq4pT4x80coV+Pcag==
=d3ef
-----END PGP SIGNATURE-----

--ToNKcA92ly3P5i9R9jeMiy0cehp5snm5g--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:22:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:22:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185525.334215 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhEB-0002Qe-6T; Mon, 13 Sep 2021 08:22:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185525.334215; Mon, 13 Sep 2021 08:22:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhEB-0002QW-1I; Mon, 13 Sep 2021 08:22:11 +0000
Received: by outflank-mailman (input) for mailman id 185525;
 Mon, 13 Sep 2021 08:22:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xuNF=OD=epam.com=prvs=989052e8b7=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mPhE8-0002QQ-RL
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:22:09 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id aee17370-146b-11ec-b3a4-12813bfff9fa;
 Mon, 13 Sep 2021 08:22:07 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18D7hNx2029185; 
 Mon, 13 Sep 2021 08:22:04 GMT
Received: from eur03-db5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2059.outbound.protection.outlook.com [104.47.10.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b22ce05p1-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 Sep 2021 08:22:04 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3469.eurprd03.prod.outlook.com (2603:10a6:803:18::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.25; Mon, 13 Sep
 2021 08:22:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 08:22:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aee17370-146b-11ec-b3a4-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AhT6MPZedpjhwYBvFwYJQq5W+Bgkjsxs/3JG5++wDLNMs5Qdy6Hd603JPQg2xaNWSh0uCMbx/eqKM/UfBM8HyZaoI6F156AkX/5dopgMY/l5l7/K8CfIzR2WbzY1CSo1oAZGaIs5ZUhXQjyM67LNBkCu04Pri1SOOaB6B/zWKk8RnEtofnn3hCMxHT50cPo5pQaPhgGLXtYe9dgiG8eZFuMbLr9UhqPYN9uiBnnYFNabzOtLFPTND/rQZuVUtmlNMHUkEctQVOb1tYFLl4bzzFkUo3Q0vLG8cX43GS6Nus2Ekk/bp4VdNq67LT0oifcAD61MbdsKj61dItQmr1ZbKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hLCrlE47htuFoiiNNJ7KZVlAtSUvVGF9KRKk4rTBB3g=;
 b=Ov9h2kSWr0MEAIG5xZxohoeuJbhaob/gQ+cH+q9aOrBlq5S/OKlJ7SG+kBXbt8T9xit10xSAVHlzCGTij0MdIFj5xQc1mK6Fubwpnye0vlr+UGcnECxAeoxCCAdN9wEs5wEH2qYBwHmLw0WdnValflN2q3hC1NmRlKVqMdWSsDI/fvhDp48EA/nIYidQxu+9v7bcwH0ys/7tihO+27lSfkMh6c/oCYDJLVicF2bVcPWMO2eoCUhIqOV60Ngit+uv5PV57zoKbALUO4Jib5yBnq27PkEFLekEBhFPeLioe4fboLJP9k10ndB5WuZGzWLBlt9ZWR4PvNQB7/1E8H2eYQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hLCrlE47htuFoiiNNJ7KZVlAtSUvVGF9KRKk4rTBB3g=;
 b=IwS/AT8qG/+L+8R8RDC5hvRRVZ77CAVbhZcAMEl2HODsoja11oYda79NOgJDmCOHtHUme6gDZLivtSxZfe5FEYevF9F1kYuNN0y2PrpzQDcWYLqQXdoSAe0Eb+93EviCFBQFrXk/f02ZOI689kUSXN4aSyqVJfGWwpPlTWkEBM2BuQ+strGl9wNqEjBk8+/aXioogtVN3jwovMJXSVvYJF7mCAv1A2cAYymvrS6650a7Ao/OrRldCde1HQanvKmp5hf4lO2Zezlo9V4YqD+OkLrio/1oYNgZkFuYXFJrXUaZDQoFdtMULm3ekSNj1Jbo4fdMue+oOl32oU36dOZCWQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>, Ian Jackson
	<iwj@xenproject.org>,
        Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Topic: [PATCH 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Index: AQHXoJ50aOdU36kT/0mQrZPJ6sbBAaudrMcAgAQC04A=
Date: Mon, 13 Sep 2021 08:22:00 +0000
Message-ID: <b08e50a0-539e-3405-cc1a-8cab8232e037@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-9-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101159100.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109101159100.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2f179484-dcca-4397-5662-08d9768f8f21
x-ms-traffictypediagnostic: VI1PR0302MB3469:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB3469DDE5880F42D040CC2523E7D99@VI1PR0302MB3469.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 3z3xZAP2g9T63YSjmEqTCMR3QBR7HOlgAlybX/xhQczmuOFhfPqH+NuvODTbV+MIm/2SHmBW1p4JHRKyQhZh7OZ1xwE2NsAQb4bMWhtdZ4sh9SbP2VgzBCnt6xPIQkAIlc1xNasNFiAr5K4HjDTKQUpdRlO0M1Pr8H/+LdOpXZTkwWDwKOOW583Mlg7kziAVf5VYrsz2C1ZviW5lwL223+nVHfSCUb+2hcMRhg6bLe2PYFgSE3l4kclbCuCLGzogwS4Dvbs1p/LhzSfwK0n5vMe2LMpZYcqtXd6q+rpUawjjoUb3A/nL0ONpfaAD6nFsEHjdCVoahjS9JkbnFyNCC/VSXvXF+8Cs13IIkDeg0R8eRf6pwXimgOyHtAu0oeFJhS/TxLjAlgfmDwom6V6Q0mX69bBXOI+d/pm+Mg0ihrIIHrdN2L9AjLA+2GbMCcsNdtOr+6IhnwgK6F/PzFwmu5KGh8/iy1m3wmtUDLXzaacNhdt9wxAkPUO8EbU4GIr/2RoGVzV7UpFvnH1nL/yvaQ5yJgJl+OCl2hhvswUjFE1usaaMj7VhMoiyVgmMxzE3DPDyJAdHKtJgNdFENSm7r9PtDc9BetHu5ezpWkj4n/Uoc3hXjVyOvZ2hTCi11RMCrxzkdbxNx+37xcexFkc6oQQrW/H55G+yVevE0NcnJJH4cvv+3+/oO7zpuM8+XoPyx3uGU8tm7f4zJLs95MvCsNZuRzP8rVf70weiMiiZnELe06Rp96UmaVPNFoggH9EV
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(376002)(366004)(346002)(396003)(76116006)(186003)(316002)(122000001)(55236004)(478600001)(5660300002)(2616005)(66946007)(38070700005)(53546011)(66446008)(8936002)(31696002)(71200400001)(36756003)(6512007)(86362001)(31686004)(8676002)(91956017)(54906003)(66556008)(66476007)(64756008)(6486002)(4326008)(83380400001)(110136005)(26005)(6506007)(2906002)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UElpZTQyRmdPUGtIR1FkUjJjeGhjMGd5UW5aTENZRXNLbm5Zc1BmSm8vbnJj?=
 =?utf-8?B?enJOQlRCcW9ZVlhJMVp4VlJkalhZVEgramU5cHdyLytTRnZvc21IUW1XYlln?=
 =?utf-8?B?MGRyb2JJSXN3d2RMaTVMZnJPbHRLVDk0NXB1Tk9PSVZqWW81NUxKdXgrTGM4?=
 =?utf-8?B?RytWUjBTWTN4Ny9yWHNIUmN5THo4d3ZUZE1FVVY3VTBVNFk4cyszUTlpdVo5?=
 =?utf-8?B?dWpUZHJKSEJzNkRleVpTWjNJRzU3U1FtTjZMTVhUcGJuS1h4K1dpeWpGbVJ6?=
 =?utf-8?B?T3VNODlhS09zUzh1K3ErNXYxTDd5ZGlEdTFCME10VTdnZmljRVR4Q0lXeWcr?=
 =?utf-8?B?ME11Mm5pc09QcjBKbnA4aWtSTTRXNi9CYlZIWkY0eWV0S2VGejY0N2dWMXpy?=
 =?utf-8?B?UE9uOHBmMVVQcUE1WlBqeGF5QzhRYXU0RUt1VUVOZnBTMjlqdUxZbDhxUXpr?=
 =?utf-8?B?SGZYcHc5d0xaSENDZldTeWEyaTVTVDJsUGVUK3lwMFBhY3QybDZ1Q0wzcUNX?=
 =?utf-8?B?K1ZGOW9VamhNRzZEMVo2dTd2SlZLUEpjajhVcG5oSzVTTE5MdzF3d0o4RHdm?=
 =?utf-8?B?UW45NitsTnJ6dXhCOG9KQ2FHSWVzMkJlV0hoWG0yUlNqWUpYd1Y4OEpDcGtI?=
 =?utf-8?B?MFl2YU9jMGFiM0ZQVHpyU1RPMU9UQkZ6QjczMDhJN01QM1QwRkluRkpGT2gy?=
 =?utf-8?B?NHcyREN2dFdGSFl0Uk4wNUNGVEE0VDEzaXgyN01qMVdrcWRmSTVBRytQT1pB?=
 =?utf-8?B?SE1QM3hmSkFsL2N5cGczQk5acFdYc2hvekJtM0tzMm9oTW1xVk5VMW9pYk9O?=
 =?utf-8?B?L2Q2b3pVL1M2T2dGWUVCa3FiVmQ2S2t3bGg3SFdWaDlxTjNrQU1ZVFFZU2RL?=
 =?utf-8?B?ZEpYM3gySVZIZVlPQUdOWFRJU1VzSmVlVTZUb3RaZkM2UVJ3YzJKRzFSVFZW?=
 =?utf-8?B?RVpNb3VQa0Q5KzlvRUFqRXNrOS9XWVM2ZEJwSkNoYlZoSkFqTTNId04wZ1BP?=
 =?utf-8?B?bW1vRzBXcEZKeFFoa2M2WDY3TG9VWFUxU0gvQTFlSkxPUHdKWXc5YkJSUWx3?=
 =?utf-8?B?WGhaSmxCamtFMmxRaTAwcDVmek1QUFF3Q21yTUcyZERXd21UNjR2WFVyYUpw?=
 =?utf-8?B?RFJiVFA1WXhSNEpzTm9pRncrRk1oR0IzSDFLRVBwdStXSGl6S3FrL21zRzBq?=
 =?utf-8?B?dHRWbk4ybGptK2dnckhyODRTMStxc1lDelV3TDNxUjMzMFFaVEdPS29idkZW?=
 =?utf-8?B?TEFyRUR4aEVVMHdPdk5FamNZWEdaeWorSFFnaE5xL2FNKzF5cXlQV2JxYXhh?=
 =?utf-8?B?Y2lDTUJzdmtaWjNYVXU0QzF3MlE2Y1FpWjVBT3o5azMvaWlKKzdjSjY1L2ZT?=
 =?utf-8?B?NURRNHhZY3ZMeXJPb2xXSjRHc005Y1kzZ0VobytFVW1LMk5zV2pDV3FaVVhZ?=
 =?utf-8?B?VWJsNnRzZmNqNWV4OU1vYWxaZnRtcHBmOU0wTTNBeS9XNFdXdG1Vbk9jWEpy?=
 =?utf-8?B?VUZtUEgzNzRaYnJNT29pUzJSeC81bG9PRUNmb2xZVzBiZ2ZrSXBERlY3UXNy?=
 =?utf-8?B?YTFnOXZlOU1kM2JQeDZIdFBFWlN2TktSNmQ3R2VseGtmOXlBeUFJUkJpSG8y?=
 =?utf-8?B?dzN6ZnNySG1QamV5Y2E1M3A0TGhzYWJwMEtyazUzWnQ5U29aV3NkSVg3eWFF?=
 =?utf-8?B?TlZqQ2dkTWs2RTVtbm9nSEdGZXRKM1VJczlueXUvOTBOYXoyQlFXQ2VURTYv?=
 =?utf-8?B?RDgrNWwzQnZnd0cwRFQ3YnE1Ukh0b3pmdlVtOG1ociszR3Nad2VCbzAvRklv?=
 =?utf-8?B?VkE0eXNXS20xdjRCU0dLZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <DCC219145F5BF4458629104A001B0B63@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f179484-dcca-4397-5662-08d9768f8f21
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Sep 2021 08:22:00.2188
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ugwZpzJQIr5SO1TdMDZJ4oXpFc5jXRmVs+iTQzPvvtHMUQMTaZXcAolNJHYLJn/v9Y80tnhpjxqM1Ost7odVh+4VH1PR5/QbaTVKkGWW/d9YxhmwQ9qHOlx7v69HGGNw
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3469
X-Proofpoint-GUID: Re8M0SN2t1EEjNV5LImhxSo57cjOybio
X-Proofpoint-ORIG-GUID: Re8M0SN2t1EEjNV5LImhxSo57cjOybio
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_03,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999
 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 lowpriorityscore=0
 priorityscore=1501 phishscore=0 clxscore=1015 impostorscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109130055

SGksIFN0ZWZhbm8hDQoNCk9uIDEwLjA5LjIxIDIyOjA2LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3Jv
dGU6DQo+IE9uIEZyaSwgMyBTZXAgMjAyMSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6
DQo+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVu
a29AZXBhbS5jb20+DQo+Pg0KPj4gQXJtJ3MgUENJIHBhc3N0aHJvdWdoIGltcGxlbWVudGF0aW9u
IGRvZXNuJ3Qgc3VwcG9ydCBsZWdhY3kgaW50ZXJydXB0cywNCj4+IGJ1dCBNU0kvTVNJLVguIFRo
aXMgY2FuIGJlIHRoZSBjYXNlIGZvciBvdGhlciBwbGF0Zm9ybXMgdG9vLg0KPj4gRm9yIHRoYXQg
cmVhc29uIGludHJvZHVjZSBhIG5ldyBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUSBhbmQgYWRk
DQo+PiBpdCB0byB0aGUgQ0ZMQUdTIGFuZCBjb21waWxlIHRoZSByZWxldmFudCBjb2RlIGluIHRo
ZSB0b29sc3RhY2sgb25seSBpZg0KPj4gYXBwbGljYWJsZS4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5
OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5j
b20+DQo+PiBDYzogSWFuIEphY2tzb24gPGl3akB4ZW5wcm9qZWN0Lm9yZz4NCj4+IENjOiBKdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+DQo+PiAtLS0NCj4+ICAgdG9vbHMvbGlicy9saWdo
dC9NYWtlZmlsZSAgICB8ICA0ICsrKysNCj4+ICAgdG9vbHMvbGlicy9saWdodC9saWJ4bF9wY2ku
YyB8IDEzICsrKysrKysrKysrKysNCj4+ICAgMiBmaWxlcyBjaGFuZ2VkLCAxNyBpbnNlcnRpb25z
KCspDQo+Pg0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvTWFrZWZpbGUgYi90b29s
cy9saWJzL2xpZ2h0L01ha2VmaWxlDQo+PiBpbmRleCA3ZDhjNTFkNDkyNDIuLmJkM2Y2YmUyYTE4
MyAxMDA2NDQNCj4+IC0tLSBhL3Rvb2xzL2xpYnMvbGlnaHQvTWFrZWZpbGUNCj4+ICsrKyBiL3Rv
b2xzL2xpYnMvbGlnaHQvTWFrZWZpbGUNCj4+IEBAIC00Niw2ICs0NiwxMCBAQCBDRkxBR1MgKz0g
LVduby1mb3JtYXQtemVyby1sZW5ndGggLVdtaXNzaW5nLWRlY2xhcmF0aW9ucyBcDQo+PiAgIAkt
V25vLWRlY2xhcmF0aW9uLWFmdGVyLXN0YXRlbWVudCAtV2Zvcm1hdC1ub25saXRlcmFsDQo+PiAg
IENGTEFHUyArPSAtSS4NCj4+ICAgDQo+PiAraWZlcSAoJChDT05GSUdfWDg2KSx5KQ0KPj4gK0NG
TEFHUyArPSAtRENPTkZJR19QQ0lfU1VQUF9MRUdBQ1lfSVJRDQo+PiArZW5kaWYNCj4+ICsNCj4+
ICAgU1JDUy0kKENPTkZJR19YODYpICs9IGxpYnhsX2NwdWlkLmMNCj4+ICAgU1JDUy0kKENPTkZJ
R19YODYpICs9IGxpYnhsX3g4Ni5jDQo+PiAgIFNSQ1MtJChDT05GSUdfWDg2KSArPSBsaWJ4bF9w
c3IuYw0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMgYi90b29s
cy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jDQo+PiBpbmRleCA1OWYzNjg2ZmM4NWUuLmNkNGZlYTQ2
YzNmNyAxMDA2NDQNCj4+IC0tLSBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+ICsr
KyBiL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+IEBAIC0xNDM0LDYgKzE0MzQsNyBA
QCBzdGF0aWMgdm9pZCBwY2lfYWRkX2RtX2RvbmUobGlieGxfX2VnYyAqZWdjLA0KPj4gICAgICAg
ICAgIH0NCj4+ICAgICAgIH0NCj4+ICAgICAgIGZjbG9zZShmKTsNCj4+ICsjaWZuZGVmIENPTkZJ
R19QQ0lfU1VQUF9MRUdBQ1lfSVJRDQo+IEFzIEp1ZXJnZW4gcG9pbnRlZCBvdXQgdGhlIGxvZ2lj
IGlzIGludmVydGVkLg0KPg0KPiBJIGFsc28gdGhpbmsgd2UgbmVlZCB0byBjb21lIHVwIHdpdGgg
YSBiZXR0ZXIgd2F5IHRvIGhhbmRsZSB0aGlzICNpZmRlZg0KPiBsb2dpYyBpbiB0aGlzIGZpbGUu
DQo+DQo+IEZvciBpbnN0YW5jZSwgY291bGQgd2UgbGV0IHRoaXMgZnVuY3Rpb24gdHJ5IHRvIG9w
ZW4gc3lzZnNfcGF0aD8gSQ0KPiBpbWFnaW5lIGl0IHdvdWxkIGZhaWwsIHJpZ2h0PyBJZiBzbywg
d2UgY291bGQganVzdCBoYXZlIGFuICNpZmRlZiBpbnNpZGUNCj4gdGhlIGZhaWx1cmUgY2hlY2su
DQo+DQo+IEFsdGVybmF0aXZlbHksIGNvdWxkIHdlIGhhdmUgYSBzbWFsbCAjaWZkZWYgcmlnaHQg
aGVyZSBkb2luZzoNCj4NCj4gI2lmbmRlZiBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0KPiBn
b3RvIG91dF9ub19pcnE7DQo+ICNlbmRpZg0KPg0KPiA/DQo+DQo+DQo+IEV2ZW4gYmV0dGVyLCB3
b3VsZCBiZSB0byBpbnRyb2R1Y2UgYSBzdGF0aWMgaW5saW5lIGFzIGZvbGxvd3M6DQo+DQo+IHN0
YXRpYyBpbmxpbmUgYm9vbCBwY2lfc3VwcF9sZWdhY3lfaXJxKHZvaWQpDQo+IHsNCj4gI2lmbmRl
ZiBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0KPiAgICAgIHJldHVybiBmYWxzZTsNCj4gI2Vs
c2UNCj4gICAgICByZXR1cm4gdHJ1ZTsNCj4gI2VuZGlmDQo+IH0NCj4NCj4gVGhlbiBpbiBsaWJ4
bF9wY2kuYyB5b3UgY2FuIGF2b2lkIGFsbCBpZmRlZnM6DQo+DQo+IGlmICghcGNpX3N1cHBfbGVn
YWN5X2lycSgpKSkNCj4gICAgICBnb3RvIG91dF9ub19pcnE7DQoNClRoaXMgb25lIHNlZW1zIHRv
IGJlIHRoZSBiZXN0IHdpdGggbGVzcyBpZmRlZidlcnkNCg0KSSdsbCByZS13b3JrIHRoZSBjb2Rl
IGFzIGFib3ZlDQoNCj4NCj4NCj4+ICAgICAgIHN5c2ZzX3BhdGggPSBHQ1NQUklOVEYoU1lTRlNf
UENJX0RFViIvIlBDSV9CREYiL2lycSIsIHBjaS0+ZG9tYWluLA0KPj4gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4+ICAg
ICAgIGYgPSBmb3BlbihzeXNmc19wYXRoLCAiciIpOw0KPj4gQEAgLTE0NjAsNiArMTQ2MSw3IEBA
IHN0YXRpYyB2b2lkIHBjaV9hZGRfZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+PiAgICAgICAg
ICAgfQ0KPj4gICAgICAgfQ0KPj4gICAgICAgZmNsb3NlKGYpOw0KPj4gKyNlbmRpZg0KPj4gICAN
Cj4+ICAgICAgIC8qIERvbid0IHJlc3RyaWN0IHdyaXRlcyB0byB0aGUgUENJIGNvbmZpZyBzcGFj
ZSBmcm9tIHRoaXMgVk0gKi8NCj4+ICAgICAgIGlmIChwY2ktPnBlcm1pc3NpdmUpIHsNCj4+IEBA
IC0xNDcxLDcgKzE0NzMsOSBAQCBzdGF0aWMgdm9pZCBwY2lfYWRkX2RtX2RvbmUobGlieGxfX2Vn
YyAqZWdjLA0KPj4gICAgICAgICAgIH0NCj4+ICAgICAgIH0NCj4+ICAgDQo+PiArI2lmbmRlZiBD
T05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0KPj4gICBvdXRfbm9faXJxOg0KPj4gKyNlbmRpZg0K
Pj4gICAgICAgaWYgKCFpc3N0dWJkb20pIHsNCj4+ICAgICAgICAgICBpZiAocGNpLT5yZG1fcG9s
aWN5ID09IExJQlhMX1JETV9SRVNFUlZFX1BPTElDWV9TVFJJQ1QpIHsNCj4+ICAgICAgICAgICAg
ICAgZmxhZyAmPSB+WEVOX0RPTUNUTF9ERVZfUkRNX1JFTEFYRUQ7DQo+PiBAQCAtMTk1MSw3ICsx
OTU1LDkgQEAgc3RhdGljIHZvaWQgZG9fcGNpX3JlbW92ZShsaWJ4bF9fZWdjICplZ2MsIHBjaV9y
ZW1vdmVfc3RhdGUgKnBycykNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4+ICAgICAgICAgICBGSUxFICpm
ID0gZm9wZW4oc3lzZnNfcGF0aCwgInIiKTsNCj4+ICAgICAgICAgICB1bnNpZ25lZCBpbnQgc3Rh
cnQgPSAwLCBlbmQgPSAwLCBmbGFncyA9IDAsIHNpemUgPSAwOw0KPj4gKyNpZm5kZWYgQ09ORklH
X1BDSV9TVVBQX0xFR0FDWV9JUlENCj4+ICAgICAgICAgICBpbnQgaXJxID0gMDsNCj4+ICsjZW5k
aWYNCj4gSSdkIGxldCB0aGlzIGNvbXBpbGUgaWYgcG9zc2libGUuDQo+DQo+DQo+PiAgICAgICAg
ICAgaW50IGk7DQo+PiAgICAgICAgICAgaWYgKGYgPT0gTlVMTCkgew0KPj4gQEAgLTE5ODMsNiAr
MTk4OSw3IEBAIHN0YXRpYyB2b2lkIGRvX3BjaV9yZW1vdmUobGlieGxfX2VnYyAqZWdjLCBwY2lf
cmVtb3ZlX3N0YXRlICpwcnMpDQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgICAgIGZjbG9zZShm
KTsNCj4+ICAgc2tpcDE6DQo+PiArI2lmbmRlZiBDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0K
PiBIZXJlIHdlIGNvdWxkIGRvIGluc3RlYWQ6DQo+DQo+IGlmICghcGNpX3N1cHBfbGVnYWN5X2ly
cSgpKSkNCj4gICAgICBnb3RvIHNraXBfaXJxOw0KPg0KPg0KPj4gICAgICAgICAgIHN5c2ZzX3Bh
dGggPSBHQ1NQUklOVEYoU1lTRlNfUENJX0RFViIvIlBDSV9CREYiL2lycSIsIHBjaS0+ZG9tYWlu
LA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGNpLT5idXMsIHBjaS0+ZGV2
LCBwY2ktPmZ1bmMpOw0KPj4gICAgICAgICAgIGYgPSBmb3BlbihzeXNmc19wYXRoLCAiciIpOw0K
Pj4gQEAgLTIwMDEsOCArMjAwOCwxNCBAQCBza2lwMToNCj4+ICAgICAgICAgICAgICAgfQ0KPj4g
ICAgICAgICAgIH0NCj4+ICAgICAgICAgICBmY2xvc2UoZik7DQo+PiArI2Vsc2UNCj4+ICsgICAg
ICAgIC8qIFNpbGVuY2UgZXJyb3I6IGxhYmVsIGF0IGVuZCBvZiBjb21wb3VuZCBzdGF0ZW1lbnQg
Ki8NCj4+ICsgICAgICAgIDsNCj4+ICsjZW5kaWYNCj4+ICAgICAgIH0NCj4+ICsjaWZuZGVmIENP
TkZJR19QQ0lfU1VQUF9MRUdBQ1lfSVJRDQo+PiAgIHNraXBfaXJxOg0KPj4gKyNlbmRpZg0KPj4g
ICAgICAgcmMgPSAwOw0KPj4gICBvdXRfZmFpbDoNCj4+ICAgICAgIHBjaV9yZW1vdmVfZGV0YWNo
ZWQoZWdjLCBwcnMsIHJjKTsgLyogbXVzdCBiZSBsYXN0ICovDQo+PiAtLSANCj4+IDIuMjUuMQ0K
Pj4=


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:53:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:53:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185540.334226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhii-00065L-MK; Mon, 13 Sep 2021 08:53:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185540.334226; Mon, 13 Sep 2021 08:53:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhii-00065E-Hp; Mon, 13 Sep 2021 08:53:44 +0000
Received: by outflank-mailman (input) for mailman id 185540;
 Mon, 13 Sep 2021 08:53:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPhih-000658-Lm
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:53:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 180af408-1470-11ec-b3a5-12813bfff9fa;
 Mon, 13 Sep 2021 08:53:42 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2055.outbound.protection.outlook.com [104.47.5.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-mbO470eZMXeromEphKVvYw-1; Mon, 13 Sep 2021 10:53:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 08:53:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 08:53:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0010.eurprd02.prod.outlook.com (2603:10a6:208:3e::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 08:53:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 180af408-1470-11ec-b3a5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631523221;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=sYOUrvaKtGeeItpUNzW1+k88uDSJIbM/Tw/CJJL8xag=;
	b=d5y+dZNmJZjTXDxEwD7Hcrh2kBo+HQXfpMiQCNe8riCAfJvzvyQwEUJAUnEgzNBW7xqEHj
	enNDEzTlc6zFptAY8unJyXgtca6jlkKfNHjKNQv0kEPOgsxfN7QEwvkNLtLbpasRoVMk3o
	u7qXGU3GRq32z/dmckcpCSgZVIFx4Gw=
X-MC-Unique: mbO470eZMXeromEphKVvYw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Niw2RSE6fBd/Nsltk64sTexOokLjCJX4y7cvUwWgncjXhXzyDhA5iLOPBiqkvtgKIyCcHrSjlxHN4OBvjD+8GObLrB2kdZnKkpiPResvPZhQjJ1+UOAB3trj6lG5kUlWFatcAVafze3jkaK+ttEbvFhhJpI96ae5r8jOFNXOyO3JhP4db6lX9LOGVpUkMima88r9XYfesg7UH/j2abwD5pXbCl7C9nrjNKdtkpcwtIfmBFUELuiLPoLxyzlH3mi6HsJ6saPa+0Q/O1hD1X4gC/teyye0N+dfQxzCkEwwTsHpHC66RNdrYMAN2OH6TvD3iRRFIWe6hLMQuhwmI62LCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sYOUrvaKtGeeItpUNzW1+k88uDSJIbM/Tw/CJJL8xag=;
 b=bW9MzRmDwza7a3KY++L3+VhAx7Q7+PjT1Mvneoc4IO6OtqHeBTJnIHCQ+dIA5+CO+XNv1/GPfvf/fV2pxY24audF5ajDO1L8TRJFX9cq7IKMM77WBNrXB01uFedIN1TgCZXCaV+J5l09nTiQcZGf0mVNVbIPAduXOozjEflNkLpW/rsLaEtKm3JlVaDSMUV5rT8sD8Z+w0pk8HtgI2C+HcxQRR5e7IyYxFiT81nhmNIjiiRI7ySlWZ05zoqyS6bYgWbEKXLpf5p3DCzTE+6EHyiDBrkFC69mX+I94Fw+EdHdVc4M8bK4R8aNFDfocJRHi7rwt9wSOGKyI7YxU0NJZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/2] x86/mm: XENMEM_{get,set}_pod_target handling adjustments
Message-ID: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
Date: Mon, 13 Sep 2021 10:53:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0010.eurprd02.prod.outlook.com
 (2603:10a6:208:3e::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d223862b-47b7-4a91-a9e3-08d97693fa8d
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038263569A8BF4834A941F7B3D99@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LzEqn5jnSWEWuS1Q8jVjRk6k4CFpDGGDpEYNHJQ66bXZ6Mq7KNusPHXR/vN0zBWHbrCrSaWXi8sPslvgptdG6xMpMrHEMxgFhlsxK/94RjNKnkJ3jOPXTH4ei41k7dLIFp+U6+7EG01vGL0DSHdt/B372xqdVt+ZShi9FxTetUueOy1z6pV3e6zXJGxj4LduH3UmDjC2WYjqkfjEq8ug34USmKmnzcrgC/6X+PboIzKLPM/+V+BzmHmrelV6na6DneyX2uLQNqCYanR8cvMrqGDoohMqmQMzKjTc/iG+Q93RP40+mHRYmIZoxnHM+UGLZ+yj8fkhTyKzLojhyJsX4IJRHEt6Bkv4MhMwUf4Cw6uw35nTpXqDFPobRzUp1w1Qs5U5KLQYPZYw0xIevSmr2WHrj2wRWduCEsh8ybKU0SOLSg79X189hvaQcBrf+e44/BCOaJHs7Qfx+wDg9hpDo5gzrTWIE+SYvmKLgMB0tH/OQyjsqPE51VB0i3OT+J2OzdjbBL3QrsfAUV/LS/8KshYqmTXgwm4dzUjXuhjYQb3xO2glflpQUDK/LWUVnUgxn6pg1Hp68qEm39T72ZS/vsmCVzKube8t9hXysSp1/vbwQaSniGiYB/DHhdr6LdJHtuK8KU4zJqerqK8ylhFFiPD7WDBxCWuJvibdGcSR6MgmssrcDRyGIflxzEcBZ99yehtwqBcBgopsiqtRp+nJZO/ubtwRYR0w7B362+rkA/Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(39860400002)(396003)(136003)(376002)(86362001)(956004)(31696002)(316002)(5660300002)(31686004)(186003)(558084003)(2906002)(54906003)(6666004)(4326008)(478600001)(2616005)(26005)(66556008)(66476007)(8676002)(6486002)(38100700002)(36756003)(66946007)(8936002)(6916009)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZXo2Z3lxd1B1KzN6b3BtVGE5RFlmTzBmL3FRaktEem5ya2s2ZWliOThwTkJX?=
 =?utf-8?B?TGIyN1FWY1ptR2pnRWJDUmhTMlQyY25CSHhFeUhkeHlFcVBWQUlNZm1rSW1X?=
 =?utf-8?B?UlJVcWhzcmtNU04zUFh1cEJCQ1RERXJCaHFSdSttSW9sZnYxbDI4Zklsd3hi?=
 =?utf-8?B?T1lWaDZCT1gxcFRsL1JUZlYwSGhjS29KcVl6WGM3ZkNxR1dKYnA1M1hvSWc3?=
 =?utf-8?B?V0JBYjU2SXlZNHFiLzhYNnNXT3lzd2M3QjBYdEExQXMxWVp2cllhVmVabGp4?=
 =?utf-8?B?eWlZM1gzZmlZcHZhSGY0cHkzdDZSNmxaZTRTSmhYUzR4NDBwMHFXRWhFYjlj?=
 =?utf-8?B?Z3dVdGZpKzFHdURMSWxuc0ZMOHRscXdURWlnVkF2VUV6dWdRTnpwMzNlRS83?=
 =?utf-8?B?aFc2UTdkQkR2YnhhYW9ocU9YaFdPbjFSZ29SaDRGNnlYMExrZi9wRndseWp1?=
 =?utf-8?B?RWNralFiVjYvUFVlOWdmeWYrMkNCUGxlYjBXQWZnZ2daTFd2NklVSWRSZmdP?=
 =?utf-8?B?eENhNjAzb1FJc1ozWm5xdUMzWjFSZ01ZbUUzcWRkNFQ3M09HdnMzMzBMM3ZO?=
 =?utf-8?B?a2U4NlNiVTVPRllzK0ZwRTZ3ajlhMEpkaTdlUUNuK2FreHROWG5RMjl1M2Fw?=
 =?utf-8?B?SGROZmhFTHAyK0F6WW5VcnVZaXUzeVBYT1ZpbEVsVnhUZWNiZFlYRVBaOGo1?=
 =?utf-8?B?YzZzM2VRT0hFOGpaVXpPTysybUdWQVB0aDdiMHUrWlp0TGxzRXRSbmU5U2lm?=
 =?utf-8?B?TU5adFpMdTVTY2QrK1FHc085VXNuaW5XSWFhcmlpVDBsSWpTamg5UkNsZFlk?=
 =?utf-8?B?SjFsWlhML3g4ZnpaV3JXWDNLNW9mVzBSNWs5RElETGlOaGZvR1JUSW1BYUUw?=
 =?utf-8?B?Y2UweXFQd0ZIZW12VG5YR0ZQaHg1QnZ6Y1pFbDVZeEVBMlk5cXhwNEJ1ejNL?=
 =?utf-8?B?SG9ieHlXaXppSXVXdE9UVzVjMFhuazhZS2MzalZxRzhPenllSEMwZ1kvZGdT?=
 =?utf-8?B?RGNSRWRJN09Eck1FTWRwaG4vbmhHaWhlTUphM2wxbTMwV25HTHExTllXTkVI?=
 =?utf-8?B?NGhEOVBKaWlYV1JRQitQekdydXFuajdkMWpMZXZKUDJ6bTJGUExLY2tFb3pM?=
 =?utf-8?B?RlA0bm9Yc3FaSTRlV01VMWU4SDdoNlltRnZnOGVlamFqaVJYVzNOZmVuYVor?=
 =?utf-8?B?SitadTJpQlVuRzFKcE90SVRwZUtDQkpuTjFqZ3lYaHEwb1h3NFU5ZHFLSGNj?=
 =?utf-8?B?WXhLUC9vZzNKd1o2TWZ5cDhPais5UkZJUTNyQ3Z6OS9VQVo5WG1XVFY3bVNr?=
 =?utf-8?B?bDZTZFpadDYzQmppYm9YWEVJQjBSendVUlU5a0xGdStqVHlFczFyWHNna200?=
 =?utf-8?B?Wmh6MzdZMUt4QkpEdGRhYTdQQWQwQUhoQy9vUyttWU1ENVVhaUtsQm0vdTRk?=
 =?utf-8?B?eTBqQXRqeTZyVUZUOEEyQ2dmSkhCdk5YRldiaFdlZUNPTWhZM0JFZ2RrcC83?=
 =?utf-8?B?ckdmMnNkbDZsb0JjM1JPYm83Z2l4cUswcDc3TmpZSUxUUW5naldSd3NFMnhP?=
 =?utf-8?B?WU5Za29NQUNtQVlTMUFQc09GL1MrbGZER3V4bEhHTXpxVktwZzlrL01ZZTdY?=
 =?utf-8?B?TGxaWlZDUm5YQ3l0MjNaUmJPQzc0ZVNGSzE0NlJRYU5pQVVNbEUxc3A0a0ht?=
 =?utf-8?B?TElobzlEdTllZUt2M3dYbGg4RmplTkptVkJTMTJqYnNCMHRPRTlqeVBtQTho?=
 =?utf-8?Q?dkgmbfcuql9YIC4u4G6xGA63x+Cs3gwL9f+Njtj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d223862b-47b7-4a91-a9e3-08d97693fa8d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 08:53:38.6413
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: okKeK1fJK4qhSWntu26FzDpG1j082XOkHIu0Hfcyg7ZzRrjmcU/KmsEPAlnss1oau5jBm+ESxufS7xErr73IGw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

In the context of PCI pass-through I came to look at this code again,
while considering whether it might be usable for an issue there (which
I'll send separate mail about).

1: XENMEM_{get,set}_pod_target are restricted to HVM domains
2: tidy XENMEM_{get,set}_pod_target handling

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:54:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:54:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185547.334237 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhjm-0006gN-Vk; Mon, 13 Sep 2021 08:54:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185547.334237; Mon, 13 Sep 2021 08:54:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhjm-0006gG-SQ; Mon, 13 Sep 2021 08:54:50 +0000
Received: by outflank-mailman (input) for mailman id 185547;
 Mon, 13 Sep 2021 08:54:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPhjl-0006fy-4z
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:54:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b18ff7ad-259f-4181-b762-41b59b90148b;
 Mon, 13 Sep 2021 08:54:48 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-MMJXTS9qNGGWN5_2ObqwnQ-1; Mon, 13 Sep 2021 10:54:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 08:54:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 08:54:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0118.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 08:54:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b18ff7ad-259f-4181-b762-41b59b90148b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631523287;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Jo8bPyvtaH5xAEf64N9SfjM0M/xdcPBEVhdBoKC14/8=;
	b=FIQ347EG7S7gYFtXeS/TOhwKgUINmysVsbr1IBHL6SeLdEbjEcdFYhHtEqDnWDZq1wlzKJ
	W/b27Qe5nIMJXMbsHFPzyEZlE1rBhEqBU517hyYoRKIxI1AYmkpgdVONh3WwVubtfqOyaf
	X2ZH8jlKTRAjBuaq0rpr+g3zNvpbY5Q=
X-MC-Unique: MMJXTS9qNGGWN5_2ObqwnQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KSSAl6ffjBSou5yA57i1XjPffkOEI7u22DmOif7SKyYez3DlwqJ0vtbdO3wi3xcQGfORK1h7AUMvHo6ADAORfHSiASxviK67TiqItw/L58kmiWV7jsCJ1+1Tcnzbym86JIqdyGvxNPEPUofhtIpzbgb/BNlCP+Q1uF996sKZzrTYZnYsWJ4JN6bnjSXYZ7nsjQOLR8zgNgaJxKZdYWMTOHf1MC4Ec16+BUU+dlAwi5acN1uL8Vy0oFpX8HJFUF/xQ2rZnYMCn5huaBzP7X2t1//FQeQPWzDQ2AXQAkgOGgrZTl4NdeO8QQWBHL0Sg6S6osGY1gbskdx07g7ciKeBQw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Jo8bPyvtaH5xAEf64N9SfjM0M/xdcPBEVhdBoKC14/8=;
 b=JZ3SNtuADJq5bN5mFseYKVb7hDcDHDCNTqQ+Y8VFqOyTCbSmhIb/nYRGndZJZvXDYNtk3KUKvlE/+53X+pnUjCHr9+WGsLIEB4nP+b3OORi32TV3lx19Md7Zgcb71dya/owQD4Lhp/B5mdAxIeNrAA5OxdiliXy74Pynb5myzAfUTMS72N5XLR36LbqQFT84kI8FfGwG22FQbkSGe7ar2HgqNAs1y1g+ix8UsFV5G/zXFIX5Ozvzt8/r+eDN5eHrr2ucrauhmyuFBO7e5H640U32iLQFDttYhFC3tvbkCvj7hhI2HVUNoKJkV4I0M3owvbjFohN16gfL9E35Koedjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/2] x86/mm: XENMEM_{get,set}_pod_target are restricted to HVM
 domains
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
Message-ID: <70da93bf-b337-f275-bb7d-ed4ab86fda08@suse.com>
Date: Mon, 13 Sep 2021 10:54:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0118.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2dbe64f6-b9cb-4094-5e4e-08d976942155
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB63831A11799EE674402E0DD8B3D99@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uJWyzHGMEw2ynrZGI7bi0ZgJ+lOIUJBW/qnsBq9goSva2somCyMKHEwtkZyk2uX5EepzAfyvYIWW3ctoKnrfWxAoO5b8mZ3hACPm05v6Vb+v4ZtMtLDtr7gO0cBKWw1y5+twrn7GqltHib2ILzKzGZJ0+PHFEl1pKdgYrHPClY8nzki4j179Qn0RUX7DgNXGf1wwiF7zsTvmV/ej5c9b1VvbakelQLRmfVqCdGQdg/m/5/TwqgD09SvIbVaK4SeQan/vjpL3d3diAZWdVA7vU6tNqKn82Xojt8DiNAIJXQ7glRRTi7MoC9bTzMnpvbZDiZXlrNfrE80+VeSfmZSRioSuN6TlyAq6jGBibFCKY2sBWtiOEuNOpng6cdfTuXHOd3e47F3uj6wGofvM2m5J+IUapkUGEsSnt/Meqit+Zu6lWP9w3PtDG3gngUSTCUXcE0DDDQZaWNLG4xkK1gw2PeOP5CH5j0trmtKulJuyGyD/dOEc453J9yVHlACwVpGXKmTbEUg+FY4cv/fd03fMr23xpmXhsjCz0dlJ9UCjERPs+9PNsQXzD6t4ajZ/ZFLMXp2AnG3zGCm4TX4FuDJTmzBIwWajwsu0S5rELOhWTNPWN3uBshECSVIdwAuWPpQLOMaI3nk255GQssidf8xJZlkZZ4d12OXQ4d2hhe2mocVo/OPkBU/TpqafVJNTsF7jGvNlITaZQLJg1HCpk7/+kJ0F+W9T/wJ4v5aYtgbP1lODUvc9iYnbpIZPY6gzOSB5+6jbQWlh9KtC9dWDrWzoivobI9VLmVgU0qZYfsEwlWhmHK0uZvdMmvHvkFAZOxHmMQI/Xc8eiUrFYyj5Sgskh6bxnoLzTrtM0Ffo82G7RIk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(136003)(376002)(366004)(39860400002)(8676002)(5660300002)(8936002)(26005)(83380400001)(186003)(86362001)(478600001)(966005)(31696002)(16576012)(316002)(4326008)(6916009)(31686004)(54906003)(38100700002)(956004)(2616005)(2906002)(66946007)(66476007)(36756003)(6486002)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QnAzOGEzY21EUDcydmtTQ25ZcVp2djJxSkU5NWxZMGwvMHNuMXFaRXgrdUxR?=
 =?utf-8?B?ZXloYnRiZ1kyNHk2eEsvbVZmS3dTWlIwcXR2Y3BJLzhUajRlMTVrWStDelRD?=
 =?utf-8?B?MHZlUEdlejhmU1NtQXA5SE9zRzdUK0p1dUx2Q3lTTmphS0MvVU03V3BFUFBE?=
 =?utf-8?B?Q3F0dHRua1d6Zk5iZktUTFVPclhCREMvN01DSHVqaTkzZldWczdBMkVmelly?=
 =?utf-8?B?a0FLSkN2N1BVQlIxV2FvRGNqbHpBNTVLdUtVaVNjTXY4RG0xOGtMd3ZFWnl6?=
 =?utf-8?B?ZEl6MlJDREdqMStSZVhRd21FWENxcmc0RXp2NGFaQXdHenpaN2FzV2pCOUdW?=
 =?utf-8?B?TVNSTDdUeTkwbkJUUzhWQ2psckYrVzdRTHhqQytiQWJsamZqWjQwU3ExaW9J?=
 =?utf-8?B?d2VXL2VSNnI2dnFDNHFPcEtFQ0dreGo2djN1QzgxTmRHcnB3bTV3MG9GN2Rv?=
 =?utf-8?B?UC9LVnRXTklGWnJ5YzF5SnVYbjRwUWtkdDJMVEtubmszZnVPWnZPZWs5MHFw?=
 =?utf-8?B?SFErN1NXeml4M2loN3VZWHJHK1JsTUNkRGxxU2xXUm9XOVg5WndVWGxHb3dk?=
 =?utf-8?B?clJ1WW9nS0MrMVBhQXlkUDZFS1BSSVFBbEZpQXQ5VmZOOGcxSkxBSUpMVW5n?=
 =?utf-8?B?aU1Uc2Y2WEpza2dhWWxNRjhObHVzTmRzUXhlVFVFTnhPR1FIY21VZFM0MGtU?=
 =?utf-8?B?WkZYZDF4eGx4NE1qZitLRTlXcnpBT053RFZHQkdUaUpQN2dyWHJySnpPVDJo?=
 =?utf-8?B?R2hWZWxZdnRZano0NDZGV1pod0FmVm1LZjAxdjFBYUIybGMvS1V3bG9BOEZS?=
 =?utf-8?B?ZVQwWW9hcUs1S3Y5bkpMdVdxV1JBOHB4a0RSZjA5UTdWbm9LeDExemN4Um0y?=
 =?utf-8?B?MFREb3JYSUdVdmtyc3E0b2prUHNuclhsOTFOaWFzWmYzRkxNT3dOYnMycXIz?=
 =?utf-8?B?bDd2N3E3N1hPdEwxenJzemRRQ0NmOGtua2dZaXF0UU9WMDJYbzBDM04wZVlj?=
 =?utf-8?B?bnJsY3NmREw5aCtiWlZCN2ZOSnpRSFBQcWFHaWZTR3ErU1U1ZVNMdHVvc3V3?=
 =?utf-8?B?cXYwbzUweHNJbTcva2hmbG0reG5KS1doUEVvbE5KN2FoQWlGSFpoS2ZOdVR6?=
 =?utf-8?B?YUhMSzhFWUQzRmxaUHBrakcxR0lJVDMxbTRrWkl0T0g1dnFjQkxhNUZYTndi?=
 =?utf-8?B?eEdtV202cSsxdm4yZnd3NVMyak5pZGo0aTcvUHRBZUVVT3RML1FaUXlNUnZa?=
 =?utf-8?B?d3drL1lSNTlCV1p1SmovbGZpTkZFZ3NsenM4aStwcytkRytaOThFbEh5NlNm?=
 =?utf-8?B?RGQ5ZU9DbGM2U216eE9xR0d0RTZJUjdzUVJRQXBSRjc4eDZFYkkrY25kYy9u?=
 =?utf-8?B?UFFocTJGbGthQkhRWWpSZGhMZG9Jd3JPR1hNK1lsd0J2cVZMMVFqMW9aNFBr?=
 =?utf-8?B?K1phTnRqLzJvbXNJOVZ1eC9iWU9GS2diZ3FBMTU0VEFVcVo1bFluc1dYRUtn?=
 =?utf-8?B?SGFNelNLSGhWZW12ekd2eFpPSWpKb2RhMHpmaEV1Znh6c1g5NEJvUnFhRlIv?=
 =?utf-8?B?ZnRsSUdXUlkvSFZJYXdBNFpsTUlxSGVXS0hFb2x3MHNBQWNaYVRORUs5eXht?=
 =?utf-8?B?TE1LYXRyY1MxMmpXYW5nbU4xT05DTFpDUUlIcG5HdU1VR2wxU1BKOEdYZ2pS?=
 =?utf-8?B?TzRRUHNSYTd4Z01xQ0JiTmtvQ2VLdElUYUlEbkovci9iZHd3UVJFS2h6WGU3?=
 =?utf-8?Q?gbTwyuX1wv3gpfcz8ZTX8hLJtJvqV3j0Skl5ZVi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2dbe64f6-b9cb-4094-5e4e-08d976942155
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 08:54:43.7065
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rM6u3W/8W0SfDrV7MYtnktZoyZx49x6DsGeRwLH5kg6aFvvcMhVlDgmgYiH26Ca0LbY6fzimBqju5ETHu4cFSQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

While for now PV domains have a host P2M associated with them and hence
using XENMEM_get_pod_target on such may not be a real problem, calling
p2m_pod_set_mem_target() for a PV domain is surely wrong, even if benign
at present.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
While the output of the two subfunctions is stale by the time the caller
accesses it, I wonder in how far it is helpful to provide potentially
inconsistent values, due to the lack of any respective locking. It just
so happens that "get" has no in-tree caller at all (years ago I did
propose it to be exposed to guests themselves, with no real decision
taken either way), while of the two "set" callers neither cares about
the output.

https://lists.xenproject.org/archives/html/xen-devel/2014-01/msg01524.html
https://lists.xenproject.org/archives/html/xen-devel/2014-02/msg02154.html

Furthermore for "set" I also find it concerning that page-alloc-lock
protected data is being accessed without holding that lock. Both
d->max_pages and the result of domain_tot_pages() can change in
parallel. Then again I wonder whether "set" shouldn't be restricted to
before the guest gets actually started.

--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4786,7 +4786,9 @@ long arch_memory_op(unsigned long cmd, X
         if ( d == NULL )
             return -ESRCH;
 
-        if ( cmd == XENMEM_set_pod_target )
+        if ( !is_hvm_domain(d) )
+            rc = -EINVAL;
+        else if ( cmd == XENMEM_set_pod_target )
             rc = xsm_set_pod_target(XSM_PRIV, d);
         else
             rc = xsm_get_pod_target(XSM_PRIV, d);



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:55:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:55:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185549.334248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhk4-00076S-9i; Mon, 13 Sep 2021 08:55:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185549.334248; Mon, 13 Sep 2021 08:55:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhk4-00076H-5p; Mon, 13 Sep 2021 08:55:08 +0000
Received: by outflank-mailman (input) for mailman id 185549;
 Mon, 13 Sep 2021 08:55:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPhk2-00075p-Vo
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:55:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4a235426-1470-11ec-b3a5-12813bfff9fa;
 Mon, 13 Sep 2021 08:55:06 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2056.outbound.protection.outlook.com [104.47.2.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-b7UMIKwYMS6mcn0437Yr1Q-1; Mon, 13 Sep 2021 10:55:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5328.eurprd04.prod.outlook.com (2603:10a6:803:59::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 08:55:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 08:55:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0120.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 08:55:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a235426-1470-11ec-b3a5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631523305;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mAW0Q6nzFqwulvGL5+AZVzIb3y8lJv2Qg+rTIb8bmww=;
	b=bp/pQcDwd/snhriSba8gydy89PwhgQAJQuXKCzLeWrqpDGtvP6IozPPnPIyCEVJoEWzNCm
	u4uw/lQ667QB8in+/BSfU9maUiU10eGoeCCcAL17n4TdHwXFzUpOMIKJGsG9QaGOy5oY2F
	BcjhDydX+0HVmQhfF/czd0mU4yUEX68=
X-MC-Unique: b7UMIKwYMS6mcn0437Yr1Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j5vFjLf9VENQaEl5bObryAsq9mabmwyvgU68rDdIuQr5mdfBnaIKKE63kqYF0rshhVlriU+OsktgxxAnfV6TEpQo4CVeittuctSvMvE0LcqBN3gM2vnRp0usnr4OwCoXEyK17CECr4cV5F3K3e02TxV5cKMfz9jW7kLY4/vbpn2oEQ8HAaBRK75RUg/dSOnz/39drX/OrhmTcLZAQxt4fyg7kJgU5ywXG3bqPzf98/tOiXfIZKhmZPTO9uMYfSeztRIqzb7a9UwMtB0enTbosBq9Ym57SahAeWQdly1/NbT8VdtGRT9yEV5avyGO4Zj6kffLSrlAMqEm3tgPyBoD1g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mAW0Q6nzFqwulvGL5+AZVzIb3y8lJv2Qg+rTIb8bmww=;
 b=FTaYeiVAAqCGf6NlyM7KEOPn+BJAyOZCm9YY3v0Ug8EoaZL2W69C7YIDOadm5osjBbrQg3Pc+yWD/iqOb3AtbJwx0Ef+PpBGEuvrefn9W5OrHnf/f2lOyZciYyvIDskiA1b7jVFip4a7JihFvq1nhhkOqa8u7w92gbm9838r/iU6GhKX6tatKdd4sG1JZtRkdhyBJmqQ/Y/dO8b9nNhZqZbWeGM4eNjXcUkDPq7R1Uo+b5j7w8b9/8xOpXbLtv9MvJtVF/YpK2amUbM8P9HpeuULNHhFL20uOUDKu8HYRazDw+WKnG7cVBTxeeWEFNOFRP0fKX/nHDdQAPJkqgD5Vw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/2] x86/mm: tidy XENMEM_{get,set}_pod_target handling
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
Message-ID: <34f6a6b7-1ffb-5820-afcb-bc1ded0f0005@suse.com>
Date: Mon, 13 Sep 2021 10:55:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0120.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: beef266e-f1a3-49a4-1c27-08d976942ccc
X-MS-TrafficTypeDiagnostic: VI1PR04MB5328:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB53285CF5C6B56B470FB1BD82B3D99@VI1PR04MB5328.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:497;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nw18n9qxx2l8WRPohFgunh+3gyO7cXwomClp1ZyMHOlikh8V+XU9IbPiD4kbZqaQKDiDRieZnJnlSyNUKU9+/v3eV2Gq5h5FgidpCigUBoM/nipZ/QaQiJmbQNn4tp+v4x/PjpyYHMKZ9Au/BvLtrfJYLRf0DNhJlNifL5AS7sR0rqtH4fB7+0Mza72Zo5w8Sd2DHALjNmibpjrq+PqvQ1aBijih8fxvCbv/7DDQMjV3vfDSVWd0+6nDZSsbNQpXG8cM5nJw1Eh+lvAC9igMSAYcOzxN1EOuX/r57TQmrxLAVPRH8/9pNPYBTn0QoObHY3Z6IYxclan2kcZP/IaKwW+RkBYra8iPDM8zqxAOzXaQE9eVi1vaQuM/8SYupqxZt6/Dqf78JSz5IxIREylvcdYNA6f4kLWuqJIibAHoAFL49KasMd+xcGeKZlE8RLQU1o2uyMVzgxTDyCu193oAa3+ccBVWClYk/nuW0dCQVv7CFWhnVbHMRiriQOwIWdjADShG+ujwJIPVGI+y2Nt3pTI1B2/GXPISmhWml/5E3eguwjwFxhsz0yDqAoknImkiFjNnAkI7nrGD9gcxQW4hzFLq2MCABGB1f3bDGZtXOlSiarukJWtKtfPSNs654SJRSSOWE2KZ48jbcyH+RzP/ok83Ap1JmCTDdUXiE7JUIluyzsv0RDrVRzL4kA1X26AKrPXAsgRW57c4eZr09xrFlWU037kZLGSQziAvKqZ5078=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(376002)(366004)(396003)(346002)(8936002)(86362001)(4326008)(316002)(31686004)(31696002)(16576012)(38100700002)(2616005)(26005)(956004)(2906002)(36756003)(6916009)(5660300002)(83380400001)(8676002)(54906003)(478600001)(6486002)(186003)(66476007)(66946007)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bUx2anM1aFdCOG1RaFhHVFhVcnA4QmNTTEdoZ21IUHJVWkVHWTBoVjkxVU9L?=
 =?utf-8?B?VE5sSTVEL2FpOXIvRU82RGhHM3hSbk9NcWlIVUxmdTVuSTRObDc4NENjSlpZ?=
 =?utf-8?B?S1lYdWtaVVpsa3g4Y09pakRXbTM3dWZNbEM4VTlTc3hWZnNWODhaeGVrNjUy?=
 =?utf-8?B?bVo4NXJnZmJuSTJKVzd1dTZjN1F6a29UNzRYNGZ6Q05IKzQwSmZXcW9WQ1pH?=
 =?utf-8?B?QWEwV0QzWktwSFJ2cmJ2cnRKMlhOemZqVXN5Nnk1YmZ0NjNBbU5oa29uZWNn?=
 =?utf-8?B?bkozRkZBTDRpNVI5SFFRZUdYczJFeE1UK1ltU0svVDV0czdYbzZQOGpia3FT?=
 =?utf-8?B?b0Q0cDNRZEN3RjR1VkNyNk1MRmJmRDl1MXljWTJydTA2MWdvSE5lYXU2VFBH?=
 =?utf-8?B?ekVDRDAwV2p4cVJVS3RPVWdIdjNLSnhoclRJRUgvcGFZd2lESkh6SE0zcEE5?=
 =?utf-8?B?L0RkZ3A3bXMrdjd4dlN3aHhFdVZUZ2M4dWk1bzZDeGlrdEdGVFo5Rll1N0k4?=
 =?utf-8?B?dUtMVzQ2dHpRQ2tBbXpCWTJnMmwvb2poNlhkTndVZ2lTV1BkZ0ZrTDQxVnRY?=
 =?utf-8?B?ZENYZkZONVNFUHZNd2ZDUEplOW1qRWN5SnRxVkNza0hsK3h6ak9TbE84YllJ?=
 =?utf-8?B?NFR0aHJMT3paVGY3MzRzeHo4UWpRNVJtYlk4QkdTbmluUFBPcnpqdTM2dUtj?=
 =?utf-8?B?SStnQ0tuUUIrUjE5OVBSa1dQbXZlSVNqTVA4cTFVTWQwY3FsS1Y4d2xhcHc4?=
 =?utf-8?B?aGJsU1Yxa0lIamp4encrTWtsbmxEZkw5RW1aTzgwcTFEMXRiWExkLytVUGNU?=
 =?utf-8?B?NklvbnA2M1RobHRCU0pqL3J0K2NwT3l2QVlnYlNVT09seG5uR2NOdURvRzlk?=
 =?utf-8?B?SUNJRzVMeGI0R3d0UHZVeWFWbzFyOFh3U2JtVm03UVV0ZFZvTUxZdUVRSjhi?=
 =?utf-8?B?b2NtN0swQkdFUDZ0UFFmUmY4VzRHb1dCQ2tIMWlDQWtrSExpSjA2dkM4L3pr?=
 =?utf-8?B?K1FxR3FZME5GUHNmTkVsVjI2TjRWSURpeW9TejRGSm93NXZSMndrSlZ5azVw?=
 =?utf-8?B?WHpkNDh6T3pZQjhEaUd1UlhwYjNzczJmRGxSVEJIZDYrcXQ2aDZmMU9sbnBy?=
 =?utf-8?B?bGo2L2llM0ZqQUdOVTA0WGFoMnBMWDE0dU8zN1d1S2UxWEJSa2tqN3JROEVX?=
 =?utf-8?B?VnByaFNnb0VHL2ZpeENVem1VanI4SVlpeGw2NS8xQVNET1IydDEwdkxNMVRD?=
 =?utf-8?B?UERaRTdzNjhzMTMzY3ZidDNVZk9teGxWWW5Xd1pWNlF0TzJjdnJ2Y0pkZ3J1?=
 =?utf-8?B?dGFPMXJGUzZ5Z0gzN1ZRMld1L0xPMUZJSjBLcG9mOUtQNjVBTlRPVnY1cTBp?=
 =?utf-8?B?bFRSV1R3WTNQcktEMCtLYWlLQjFIOHMwMXdiZzBVa2pIYXh0czNHS0NzVDNt?=
 =?utf-8?B?UW1TdWJwdHRMTG9ZNndVVFk1LzZaVFhybjQ2cVVaakxZV1kvOEpDTExDZGxj?=
 =?utf-8?B?M2dabllmR002WG4yVUcwazRaZGRtT09ERHhNSjBNUlVnQzhNVXMwWVU2WUtL?=
 =?utf-8?B?dUNoUnJwcjBGRlZzN0M2S2RqcStLZ3JtNnJHVEFYOUUyTGQ1RDUzSkNxcmRt?=
 =?utf-8?B?Y0lyOFdRWnJZZVZhVFp3L1JDdnJoT1JmUmhYNjdma01ZSnVGcDhVVDlIVmdR?=
 =?utf-8?B?bVk1OVIxOU1Db0ZZZkVGYXdUK256TnZhQ2d0UExsZngvM1B4RnlGTU1iTmll?=
 =?utf-8?Q?wZLtZtYr5Dh9Q4YQt+ItCeao+Zp6d0e1XvfP4qi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: beef266e-f1a3-49a4-1c27-08d976942ccc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 08:55:02.9086
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mJTQyc/e+fsxVPRs1UDiZVhOrNf131cW931qdOrcMspCFwHQ9GxZEALd/JZYzl3nVYWei5aFW/ibPHesGrbYFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5328

Do away with the "pod_target_out_unlock" label. In particular by folding
if()-s, the logic can be expressed with less code (and no goto-s) this
way.

Limit scope of "p2m", constifying it at the same time.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4777,7 +4777,6 @@ long arch_memory_op(unsigned long cmd, X
     {
         xen_pod_target_t target;
         struct domain *d;
-        struct p2m_domain *p2m;
 
         if ( copy_from_guest(&target, arg, 1) )
             return -EFAULT;
@@ -4789,23 +4788,17 @@ long arch_memory_op(unsigned long cmd, X
         if ( !is_hvm_domain(d) )
             rc = -EINVAL;
         else if ( cmd == XENMEM_set_pod_target )
-            rc = xsm_set_pod_target(XSM_PRIV, d);
-        else
-            rc = xsm_get_pod_target(XSM_PRIV, d);
-
-        if ( rc != 0 )
-            goto pod_target_out_unlock;
-
-        if ( cmd == XENMEM_set_pod_target )
         {
-            if ( target.target_pages > d->max_pages )
-            {
+            rc = xsm_set_pod_target(XSM_PRIV, d);
+            if ( rc )
+                ASSERT(rc < 0);
+            else if ( target.target_pages > d->max_pages )
                 rc = -EINVAL;
-                goto pod_target_out_unlock;
-            }
-
-            rc = p2m_pod_set_mem_target(d, target.target_pages);
+            else
+                rc = p2m_pod_set_mem_target(d, target.target_pages);
         }
+        else
+            rc = xsm_get_pod_target(XSM_PRIV, d);
 
         if ( rc == -ERESTART )
         {
@@ -4814,19 +4807,16 @@ long arch_memory_op(unsigned long cmd, X
         }
         else if ( rc >= 0 )
         {
-            p2m = p2m_get_hostp2m(d);
+            const struct p2m_domain *p2m = p2m_get_hostp2m(d);
+
             target.tot_pages       = domain_tot_pages(d);
             target.pod_cache_pages = p2m->pod.count;
             target.pod_entries     = p2m->pod.entry_count;
 
             if ( __copy_to_guest(arg, &target, 1) )
-            {
-                rc= -EFAULT;
-                goto pod_target_out_unlock;
-            }
+                rc = -EFAULT;
         }
 
-    pod_target_out_unlock:
         rcu_unlock_domain(d);
         return rc;
     }



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 08:56:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 08:56:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185561.334259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhlb-0007xr-On; Mon, 13 Sep 2021 08:56:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185561.334259; Mon, 13 Sep 2021 08:56:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhlb-0007xk-Le; Mon, 13 Sep 2021 08:56:43 +0000
Received: by outflank-mailman (input) for mailman id 185561;
 Mon, 13 Sep 2021 08:56:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPhla-0007xe-Ul
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 08:56:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a77c48e6-3e3c-4092-bf75-aa1f37b17549;
 Mon, 13 Sep 2021 08:56:42 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2050.outbound.protection.outlook.com [104.47.2.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-ZqIHGITUMlGU0FZzTE3kvg-1; Mon, 13 Sep 2021 10:56:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6670.eurprd04.prod.outlook.com (2603:10a6:803:120::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 08:56:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 08:56:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0069.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 08:56:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a77c48e6-3e3c-4092-bf75-aa1f37b17549
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631523401;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JALDp/VUGx0Jw78BgXO6MzlUdZlfU4RREqGisKbNKd4=;
	b=BuUdeDHQyhIg39Bw7ohl87kveHpaSxuOeLl+4mi1lKO9Us2KSFtbNt5OjUdjADVWBRaNuo
	ByNoOMyj5TiT/JgrJ+DwIbXM9Gf//5OrtJr7NDVJnN3Tiadwgn5K3KdtTvmkk7eCRkideo
	I1bwvlBhr2LRRJJq51DfQ3lZY2s6X7Q=
X-MC-Unique: ZqIHGITUMlGU0FZzTE3kvg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ExUOR10L7Jw54nTUh4OUr4O7H3ggtjlFpB5hJQkbStFdRoj1armjEu/yxO2UzGpM7IQvvwVdthZ73Te6j5QmeKnKLbeCVF9T1e92dHd+Ybezilczmyd83LLQjEcTgj5A2KfNgZJCIInJmrkt6GF2nCqOxlNIP/F6ZKnTBcZWQ5OFz4UQ8wlkyslS7EQxlQ0iv6t8v7qStccmfzXMZJr7zq4ifXs9EWmR76+GG7RHZQYmhJ2kRgjOacTBBHIk4eG/uQFG3DwBcxtUYLSALI5V8RZPQe/PXpxE4MIM31dctoCrLB2fhXIphtCL/njNCeHQowAc3nEYDwUQEsm8m98jnQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JALDp/VUGx0Jw78BgXO6MzlUdZlfU4RREqGisKbNKd4=;
 b=AjqPcBsCTG0hpTIC5Nkc+Noymvh/yWJ+FE/lA+HkSMq1uMMQ5Z+gf8DMBnX78uL6R7lHq3VyvfmwI4+8ORdriO1xMG6HO/1xK5AZCAiejaiwlSNEir8b78N2/zjlhBp2y0bwBxP9xJFevbHxPmgcJkYZMtG6KsGl2xt1eZQJ8O9sSoj3UN96RMcdIOEltGEt0MDMGHgFKllz+F5h3PwcDrr/VxdzctJhRtH2pMT0toaa07KFAfOY8LwHBYsb62QVVCrDbatZaiXlccgNtg78DR7BpxRU4luMRyFUWDfX4umnKoiJOWFxhzgko4GIaXaHZlO7eNdjTLHlKZmsffLfdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] x86/mm: tidy XENMEM_{get,set}_pod_target handling
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <ff324215-ccad-bb05-e36e-d711ff7c960f@suse.com>
 <34f6a6b7-1ffb-5820-afcb-bc1ded0f0005@suse.com>
Message-ID: <9bb167bf-1542-32d7-9d24-2d865820e08c@suse.com>
Date: Mon, 13 Sep 2021 10:56:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <34f6a6b7-1ffb-5820-afcb-bc1ded0f0005@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0069.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1d::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7eec97f5-7a9a-49f9-9e94-08d97694658e
X-MS-TrafficTypeDiagnostic: VE1PR04MB6670:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB667032B12DF74AD4E544667BB3D99@VE1PR04MB6670.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1850;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	t2bqGAFp73XVDWiXtQVIzBcFBKwGKgDIr6JUKKrJaYnPK+2Mlu6M8msABYcdDzglq40J8JeLIczxLq2Mo7ac2kFJAYlF8kATlQ3be6dewuTgKJLES3dVjbHKEUewGhp+ZsAECgF8IviQetOAfPrlcyJFO86fQFvo+3HC6UWCuOC0+ud8HL5JWRy18/z5Zn5EKIZk0dvH9gfWkqBWvkTtcp+IsKFH+6O6XA/Hb2EVG7FVNqKh3LqoS8JehizOq4pxGWTUfPhFhahyZW2PdtbvTO5NJqZumSh+kG/yWUgzXVzo81fvbnVsOVIgZJQ5JJp3QxcyrUszqZar9GvrOaZE3IUbnJs4i+bHmf3FS7uTQzdtGm3QYHVviocBa3l+8oU+BI/xYxoCUqj1AmpGQa8jSAXVixpO6GQyI7ffQzwD6vA7qeQI7fsWfAK09nIRU0xmkHNOasiZ9fm/0x1Fc75qjATcBTeH/HG9a8bxpaCYMYuZkztjRjhqVslgS8FjYsHW97+8eZKocXlHlpwskOxGEFjcLfjZ7blr6hdnNAFT1ENvl2M4DJzeW4yunPrFSPfyIaHP/6xMSGdyNi1346SXO4DWGmS1+B0cIqQOCW4+yU4FWlKAg31pe3+3yHcIbO2qojfC6blHYsBMSUuUnckJiAoSayfzH1jkRMHt2Hhu9JeethJ6J1+JUyzYzLnTu3+hUZW84Bdpz9J3I8gKVm2tRRPqpa2TwWJvitZ9KE2+MlccQYyWb2p2e+RKlmw74l6G
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(39860400002)(366004)(136003)(396003)(8676002)(31696002)(956004)(316002)(31686004)(36756003)(53546011)(478600001)(66556008)(66476007)(5660300002)(16576012)(54906003)(2906002)(66946007)(6486002)(6916009)(38100700002)(4326008)(86362001)(186003)(2616005)(26005)(83380400001)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RlArZTFpbnpFcURkNlFzcjIrTVJvZUdWNjV5SXEvZ00wYTNGVE9idEI2YStx?=
 =?utf-8?B?c3JVeFNIUEJyaW1XbUx1UjlnZHZHczdsTm96VUNGaHVFSG4vbnVBa2VMamZT?=
 =?utf-8?B?di9RL096Q3VPcGVYbWZ1MVVkY3RIeEZXVGVvVnFQTERJQjZKZFZFb0k1NFRy?=
 =?utf-8?B?bE1lNzNZL1VjdFJjTUJ3K3ZXZHE1Y0ZRYmFJK01Jci9KZWJHajJUVGt4QUI0?=
 =?utf-8?B?U3BCK0xBTklhWktDUnl6b2h5S1ZMVE92TUltNHVpSi8vRFMySldycitpdFZm?=
 =?utf-8?B?SXkwakJoYjhNYXJYeWg1ZWY4cEFJejlsRzVjcG9RVVFIYm5tK0hxL2lsV0h3?=
 =?utf-8?B?NmZBMWJJTDJSK3liUnAwOG5IVkg5K1pQYk9JTTRabGVhNjBnVzJGWmdoQzdz?=
 =?utf-8?B?V2VoazRaVlJhbk9tSFNzVlk0ZjJRbnlmd1BDRWpHOUhGUC9BOHhJcGhlTWtY?=
 =?utf-8?B?US9rdllTQXpXWmMyTWFlcEMwVDhSVmFYaEdqc2c3YXJ3akhtZDdrN0lNYXYy?=
 =?utf-8?B?dFQyUG00cy81a3hZWVF4MTM1c2ZiSFNQUXRsSVAwQSsrTlRUOHNEZFc0bDNl?=
 =?utf-8?B?aC8yVzcySExCb3ZSM1Z3di9QUlAyOCt3WDBXUkcvaG42Q3A0OXdvN21lbTAx?=
 =?utf-8?B?TmlwWFpnSG54aU1OSkhBK2tBUlhXaGF0NWpiVkRnQU16OXR3NmhTQ3dSczhh?=
 =?utf-8?B?L2VKTWNKek1jUzdneUJnaVVObVozRFQ3V0lRRXVYSHlBR25jemF4bG9HZnh3?=
 =?utf-8?B?Z3ZVelBNanlabmxzcXZWNW9rdkd0dWtIS21samxGajZ0OVowd0h4c0hNWXZ5?=
 =?utf-8?B?OTlrQStzWjgzaGpHN2VpK3J4YVpDVzZzNnhEbUhKQS9xcmZwZ3Jlay83Sk5i?=
 =?utf-8?B?Vjg4cmx0TzJvWlR2Y2ZxcHk2aktCMDRlYTlSUUlyQzgwL3QyTFRQcTJNc0Va?=
 =?utf-8?B?VHZQbGNQR2ROcENkSmNqVmRjMzlscjN0Z1VBZkxSQ053cmRFVStFWmVYazVi?=
 =?utf-8?B?b1FIWUZVM1BsZjBBOW4rZTBMY1hrV0FFeVowTEpnd2dXVWd4VXNGL3hTMUcx?=
 =?utf-8?B?cEt5RXN5U1huNkNzVHJaYUtlbmJ1am44UHNuT3BDVDVjNllIMjNnMDEvNzNo?=
 =?utf-8?B?cXBGK0dpMVZxbUZCVUdNdVBCeFViYkZUN0xmWU41RlFBQnJleWdlZThBSFkx?=
 =?utf-8?B?c3NKeWREZDBFSjdkTFVtelJWdTU3cnNES09MRDlsdFVscjRjd3BqWE41dTl3?=
 =?utf-8?B?RjNLT3hWZDNTSXRuN2lRQWhsTFU0VHlQQXB5c2dLSHpuR2ZVTWVoRzMxOWhG?=
 =?utf-8?B?MjBvVWFPZHBBSit0ZTZCZ2NvSmdqbjhFRXZtS0xpcWgzZHR3TWFSUHp3V0NL?=
 =?utf-8?B?dCtlNjZrZy9Ocml0T2RVVjhwWjVEZ2JDZTB0UVFvVmxyUlpzTGFEL3JEM1BB?=
 =?utf-8?B?UmIzUXZYNlNQV2VsaGYvdVZNRXVkYWZlVVFBRDVzMUF2YjUyN1czZCs5Y1dF?=
 =?utf-8?B?OVdYc2NvUmYvSVJJdlV2RkkvREN3aTZZcXpJS1JIUWpYNWxuTXpwSXVJWTMw?=
 =?utf-8?B?RzJrWlA5L25ianVtcjEvcEFjeWRHb1I1RW9OQWE0VHFDQ2gwYTJGT29CaXN6?=
 =?utf-8?B?aEFVUVlSODZDL3RhQ25vRUR5S0owbG1WcjVGUzlINUVEdDF6ZlJUbVFCS2NT?=
 =?utf-8?B?SlpjeDBDa2ZwbDVpUEZ4MnZ2ajdsZXovbkpKR3g0ZHRacW82cnF0eVRrZDdr?=
 =?utf-8?Q?G/CweCHr2thPHjZv8iGo61+Izr49ytoMMLYmQOF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7eec97f5-7a9a-49f9-9e94-08d97694658e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 08:56:38.1438
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: d6VLKneqRCkcg+Hwb3KPgo16immyk/QgPPp0dZQ82jYkdvkbyu8qO/sRI7DEoNHVqvVUEfvt695vu8NNdgj7ew==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6670

On 13.09.2021 10:55, Jan Beulich wrote:
> Do away with the "pod_target_out_unlock" label. In particular by folding
> if()-s, the logic can be expressed with less code (and no goto-s) this
> way.
> 
> Limit scope of "p2m", constifying it at the same time.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

And this is of course patch 2/2. I'm sorry for the screw-up.

Jan

> --- a/xen/arch/x86/mm.c
> +++ b/xen/arch/x86/mm.c
> @@ -4777,7 +4777,6 @@ long arch_memory_op(unsigned long cmd, X
>      {
>          xen_pod_target_t target;
>          struct domain *d;
> -        struct p2m_domain *p2m;
>  
>          if ( copy_from_guest(&target, arg, 1) )
>              return -EFAULT;
> @@ -4789,23 +4788,17 @@ long arch_memory_op(unsigned long cmd, X
>          if ( !is_hvm_domain(d) )
>              rc = -EINVAL;
>          else if ( cmd == XENMEM_set_pod_target )
> -            rc = xsm_set_pod_target(XSM_PRIV, d);
> -        else
> -            rc = xsm_get_pod_target(XSM_PRIV, d);
> -
> -        if ( rc != 0 )
> -            goto pod_target_out_unlock;
> -
> -        if ( cmd == XENMEM_set_pod_target )
>          {
> -            if ( target.target_pages > d->max_pages )
> -            {
> +            rc = xsm_set_pod_target(XSM_PRIV, d);
> +            if ( rc )
> +                ASSERT(rc < 0);
> +            else if ( target.target_pages > d->max_pages )
>                  rc = -EINVAL;
> -                goto pod_target_out_unlock;
> -            }
> -
> -            rc = p2m_pod_set_mem_target(d, target.target_pages);
> +            else
> +                rc = p2m_pod_set_mem_target(d, target.target_pages);
>          }
> +        else
> +            rc = xsm_get_pod_target(XSM_PRIV, d);
>  
>          if ( rc == -ERESTART )
>          {
> @@ -4814,19 +4807,16 @@ long arch_memory_op(unsigned long cmd, X
>          }
>          else if ( rc >= 0 )
>          {
> -            p2m = p2m_get_hostp2m(d);
> +            const struct p2m_domain *p2m = p2m_get_hostp2m(d);
> +
>              target.tot_pages       = domain_tot_pages(d);
>              target.pod_cache_pages = p2m->pod.count;
>              target.pod_entries     = p2m->pod.entry_count;
>  
>              if ( __copy_to_guest(arg, &target, 1) )
> -            {
> -                rc= -EFAULT;
> -                goto pod_target_out_unlock;
> -            }
> +                rc = -EFAULT;
>          }
>  
> -    pod_target_out_unlock:
>          rcu_unlock_domain(d);
>          return rc;
>      }
> 
> 



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 09:02:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 09:02:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185569.334270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhrE-0001AE-E8; Mon, 13 Sep 2021 09:02:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185569.334270; Mon, 13 Sep 2021 09:02:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPhrE-0001A7-B9; Mon, 13 Sep 2021 09:02:32 +0000
Received: by outflank-mailman (input) for mailman id 185569;
 Mon, 13 Sep 2021 09:02:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPhrC-0001A1-Mp
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 09:02:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 528647bc-1471-11ec-b3a5-12813bfff9fa;
 Mon, 13 Sep 2021 09:02:29 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-t66PcaNqMCuHbM1Nt2dvmQ-1; Mon, 13 Sep 2021 11:02:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5471.eurprd04.prod.outlook.com (2603:10a6:803:d0::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 09:02:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 09:02:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0056.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.17 via Frontend Transport; Mon, 13 Sep 2021 09:02:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 528647bc-1471-11ec-b3a5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631523748;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=sTR8Hjj3qMeZkeQi25B8ABEAW7smRJgU/E3oAa0eI4k=;
	b=ngjkxpuLO6imWO6w5kDkmoXNV0aMf93AomsYzMNIXmfustxUWrxU+0EfzkrOAaxPW54Z30
	CTfb3rZI+HaHArZScIDmR8Sc2rBzf9gHCXNQm2iwz8M3OXRmLXVFgNFn3cHM6EKccoKm8H
	ynNBWeOmLGdVB9ARp6BxCdiqqXHG2pg=
X-MC-Unique: t66PcaNqMCuHbM1Nt2dvmQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IntaMR7OXOUXuyoqkmznVO0K4+VZ0fqUEq+0oz6gmxN5ExFe95NLM9UEPk0aR07muBu3PqulLOcLCC1rWBYC4sFYRW4zS6+8CMIK/1UFuD5nI+xWCHrqk1cQFRAJvTkInVO3GyUYD7228X7zYENphW84DFfuvB6NffDF/0a3FjXfaNQ6B2OzS4kcHvrrqJtIURgQ9qX/JtWyHdkr0RfgF32pk2o7ZBt0nZm0CW4bJ80y/mYOe4EUaGIzjxl3npZcQ8eGYCkyaktFlRgPoIFG5V+9Nms4B+5GcWZc7wXq8l+bDATTAljErni3wxCS2ac4zwadmXlfJFt8snj8t0IOug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sTR8Hjj3qMeZkeQi25B8ABEAW7smRJgU/E3oAa0eI4k=;
 b=PR1tyLS+GUvoeLreyhefD4FzLw9yyicn+7JLXt6yWCbaUb8jdW5VyVPCCpXo6byGn96BuFJMee2S9co12MepQIBkH7fI/U0vxSxuGFSS7/hcruzBJP+/yVW7pi1yqxEkTLLeqKRkH1cyW03PFTbRWz3flAyk97gvxOLt29cOGDru9M6MvxZ1GI+F+kydDQBVYn7Mfd+7aw2GDK1xebbHMw0JK4M9PDpqmSJQIOHKEyaosWo8NsM/9XTgpB6w8MywxNGnmTiiOHRPQGn58iuXddkH2KpoX8WD/LIBQp2bSZICjbwviKiE5Mq/Y3YlRwX/W6ay0GkiwyDyQvaipINaQg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Anthony Perard <anthony.perard@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Paul Durrant <paul@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: PCI pass-through vs PoD
Message-ID: <b6d598cd-c244-2b7f-c731-fe839df3382e@suse.com>
Date: Mon, 13 Sep 2021 11:02:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0056.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e7cf882-e3b4-4f43-7724-08d9769534d8
X-MS-TrafficTypeDiagnostic: VI1PR04MB5471:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5471E9E2A43B7C7C4DC50DA7B3D99@VI1PR04MB5471.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QNdgprphU2AN7dCNNXLFBjMMUZfG1CIOXjs+9wcch9K69XYi/+WDBXaXcxYKSbnY9gUPewTkfmybI0A8gotnZEhudeAXH1MsD6GS60krV/Yqt75XXmzmsAPN7J/2zW3ME9ppeo5XYQxKIb4R2ZDdj3XP30IVHcyFglt5lfatM4wie0PuONDtyYAjrqeQOL98sMFot+mVo5yG/PgMafNy0YliCFaGSgj1D7j6/Bxa59d2L0MdS2MvgmLDz57Tvx9I75gcIW8/eHtGjEJfiF7Nrw1bnUbqad5x88XtwM3Yjv1pCApGMVCrA5f3MGi9txKtSlS2Wim4uJt8cY7G6X5AgtwaRorCs9eEr604Jt4yzZmu1uRZdzebJDzlPPPruPNBWKklJ2eREVEwibrTa80EYpjEhb5peSeIDo8Z/0pCtY8N1FSQxGNJFtVU+y55Xs5ifM6TrXraM3tRZAKx73wFhgpBeI9H+2BzrI1iN1nWMlwmhUNS3dn8nA4nsTCcLGlwnb8V8w+ImsIrUvrfkNc5VX9OJUzPLUYALMTgC7mCSyZZVbVCaQ43DcxcjzzX6NE0QK38LIp8Mno35yXh44qan2T3/iNhdCk/cy5G4nucs+gDj+lZ8nEe3fzlVyb2EG5KHgTuntQX8o4SE8YdAltGhsD6BjfD2r2vQRzViJJ0FRuntgaWBP54KQKvSubhpRJlfl/OuN8+37nAOuVFemabqYGX1vdVCbGU9m1zaa1ik80=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(136003)(376002)(39860400002)(366004)(396003)(66946007)(8936002)(186003)(66476007)(66556008)(36756003)(26005)(6486002)(83380400001)(54906003)(4744005)(4326008)(5660300002)(478600001)(2616005)(38100700002)(3480700007)(16576012)(2906002)(31696002)(31686004)(956004)(6916009)(8676002)(86362001)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z3NDb1ora2pQYWNpUjNVU0ZYMDFaZ0RhTFZBNEdxbnUxUG9QeVBaL25MTCtO?=
 =?utf-8?B?K3YxV0wvczl0UkEvYjVRak5VV3J1OHFoS01oWWdxSko4WENHYzROTVUzMmpN?=
 =?utf-8?B?MUdZa2g5bklqMm02VWxKdUFTRFBDWkNPMFZNV2M3NlRKOUdnaExHWVlLVmth?=
 =?utf-8?B?cEFpUFdNSHMycy9McDJmWUxSWWhJU3lSWkxkQXpaTVVzNkJHbTZmWG5tekdj?=
 =?utf-8?B?L2I0TERxV3NBb2J3dkJHc3gxVmRnK3pZSTNzalFvK2Z2bUpPOFBXN29UcTlh?=
 =?utf-8?B?K2RSNnB2a2FhNDNMSW5Zc05kbEZCU2tIZTdHNjhMVTF4K0RSeHIxRWtGQS8r?=
 =?utf-8?B?U2R2QXRVY05BclBrVG1uMUpkZXgzazJwRHR3NmxrVW0vNERTbnZKMFp4Skh1?=
 =?utf-8?B?d0wyUm44QXJmd3MyWHpRdFB6M1JESFVZZ1ljcVk4eDF3ajZ5Q3orTHErVkVz?=
 =?utf-8?B?UVhmckZkRzE2Y1hJSCt4NG8xdmprbkpZUklONENBcjRwWVlBdGlJaTZ1Vzl5?=
 =?utf-8?B?VkxiVDMwUjc1clZ0MXNjdVIzT01PMnFycWczVTBmTXNNT2c2aWxxdWxiTjFF?=
 =?utf-8?B?UnRYSjBLSU1rV3ZKNmhML0hzdk9oR2dzVkovaVIrSlUxRlFBeGlvdjlUSUho?=
 =?utf-8?B?YUJCb2JyVmlMZmQzNFVTZGZ5VTFRUlNuRHJzSGFHcE1VeDBSaDFSK1BqMGFJ?=
 =?utf-8?B?aStiY2M1eEEzYlZZKzA4dUN3SDlkNmlkME8wR1V5QjJrdlVpUUxud1JYRmdz?=
 =?utf-8?B?RjhDNU1aMjdZLzVDMFZWNnNueUVhWlZZMXNhdTZDRm1VZkZUZWtvSFFqQVE2?=
 =?utf-8?B?eW5OeXZLT2thRmpIK3UzUENUQ0N3NXo5U3VlU3p1eUxaay9CeCt2eklSQTNo?=
 =?utf-8?B?K3lTek9zN20vRWhibUhsdlVtdUVHdFEzTjJrOXF3TmxpSGkremFqUHdsMHlX?=
 =?utf-8?B?dE1aS1J1bW1iUDVOMGU0RmpNdzZKOEFrb1ZVVk84NGdxMzRidkw3eE5iREgr?=
 =?utf-8?B?cjVWQ21vRHBWTER3YVlQamRwVnhIOGxFRTcyYjFxd1pEc2hKYTVGZFRjLytz?=
 =?utf-8?B?WmZYTTEvMHJLZFdQcHhON2NsYjVUTFpCK0ZuWGpYUDlkUFFmUFIrVlhzcThT?=
 =?utf-8?B?ZGc3dURTWEFrVU0zUGt1dXY2ZjhuaTJWbEpPcmNxTXBwTWFoeWQ0cXludEk1?=
 =?utf-8?B?OFhLNi9QdTd1cmxVa0NncW1FVVdkSU5POWI0c25mcnFNaEtxY3lmYkYvVzY5?=
 =?utf-8?B?RkZ2NXJFOHAwTzZGRVYxbk80TjdMMkFpK3o1R2dXd3VvcTVqYXc2LzhyRTU3?=
 =?utf-8?B?ZTRQamJOOWZaN0UyOWNJQ05yOGp3bHAzeVpYVDVGczdLWXQxR2tBOHlTQUln?=
 =?utf-8?B?U3lrWHhMUmNvOGNzMitGTGU5UkhaYnE4TWd5aHlRaisvcFErRHROaWRRR1c0?=
 =?utf-8?B?RjNnN2JMK0JCMmVPVFUwejF4TWthdFlicHdvVHZUV2w5ZGhSb0lBTlYvdkUx?=
 =?utf-8?B?UXJSVnZzcjRZekV1YXFEZjljUUhLck5rYjk0LzYrU3Q3NnhjTWk1Q3hWb3du?=
 =?utf-8?B?MmUwUWlRMUpIQUowdHcrQStKMENUbmdrdThOTW1RaTljOHhBazNVS09TVy9q?=
 =?utf-8?B?aUd2UU5kcVVrMU1qVW11cjBJL1ZtR3JoeU1EeXJ3dEhZSnc0eVJsZ2NodW16?=
 =?utf-8?B?aGh0cjFWNnZSbFVZQUlrWk5NOTVGci9xSjRUSUlSU2tIUmFtaTF5V2c3TU8y?=
 =?utf-8?Q?DA5XfiVEpmJhejSmrfSyaqkHd5hF2j/tOuSrbxn?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e7cf882-e3b4-4f43-7724-08d9769534d8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 09:02:25.9161
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ddaUJnlfiaOIBWPSr1+MoHDGCufgrUZiswRC2vHCeM3bQDAA2Da2qNuU27qHmp3MGzT1XJzlPak93/0skZIybw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5471

Hello,

libxl__domain_config_setdefault() checks whether PoD is going to be
enabled and fails domain creation if at the same time devices would get
assigned. Nevertheless setting up of IOMMU page tables is allowed.
However, when later assigning a device to a domain which has IOMMU page
tables, libxl__device_pci_add() does not appear to be concerned of PoD:
- xc_test_assign_device() / XEN_DOMCTL_test_assign_device only check the
  device for being available to assign,
- libxl__device_pci_setdefault() is only concerned about the RDM policy,
- other functions called look to not be related to such checking at all.

IMO assignment should fail if pod.count != pod.entry_count, and all PoD
entries should be resolved otherwise (whether explicitly by the
hypervisor or through some suitable existing hypercalls - didn't check
yet whether there are any reasonable candidates - by the tool stack is
secondary).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 09:26:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 09:26:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185580.334281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPiES-0003qh-EG; Mon, 13 Sep 2021 09:26:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185580.334281; Mon, 13 Sep 2021 09:26:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPiES-0003qa-BL; Mon, 13 Sep 2021 09:26:32 +0000
Received: by outflank-mailman (input) for mailman id 185580;
 Mon, 13 Sep 2021 09:26:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPiEQ-0003qQ-N0; Mon, 13 Sep 2021 09:26:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPiEQ-0001cC-DE; Mon, 13 Sep 2021 09:26:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPiEQ-0000t9-3O; Mon, 13 Sep 2021 09:26:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPiEQ-0004gT-2t; Mon, 13 Sep 2021 09:26:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=n/MHW7Wb8hd6T/OiZM/F3uuIJffvjwwNjP0vcwY2uww=; b=PjSGFK37z0RcoyjYykWPqCYpzG
	8JmRqxn32hAEUhwYO7N+SCB0Pv8EF91YdGI78o9/M+k8janCzZ7byF86IGjYHVxQeJvpU7Xf4xLd7
	FZdxnOrD10aPMdRe5rcJlMJuj5ySxrekD4wr65bt3U4Y9S+l5iQfEBITXgwEG76XfQ08=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164964-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164964: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 09:26:30 +0000

flight 164964 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164964/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              3e4f4c2eecd1eb4db20c4bb05286b89b730ddcd3
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  430 days
Failing since        151818  2020-07-11 04:18:52 Z  429 days  420 attempts
Testing same since   164947  2021-09-11 04:21:39 Z    2 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73284 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 09:36:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 09:36:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185591.334295 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPiOT-0005Vw-Im; Mon, 13 Sep 2021 09:36:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185591.334295; Mon, 13 Sep 2021 09:36:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPiOT-0005Vp-Fm; Mon, 13 Sep 2021 09:36:53 +0000
Received: by outflank-mailman (input) for mailman id 185591;
 Mon, 13 Sep 2021 09:36:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z+8d=OD=gmail.com=ysaikiran1997@srs-us1.protection.inumbo.net>)
 id 1mPiOT-0005Vj-49
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 09:36:53 +0000
Received: from mail-yb1-xb33.google.com (unknown [2607:f8b0:4864:20::b33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1e6a9516-1900-4625-849a-936c2f2f2b3b;
 Mon, 13 Sep 2021 09:36:52 +0000 (UTC)
Received: by mail-yb1-xb33.google.com with SMTP id z18so19183286ybg.8
 for <xen-devel@lists.xenproject.org>; Mon, 13 Sep 2021 02:36:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e6a9516-1900-4625-849a-936c2f2f2b3b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=M2BgUTmkSFrfNfDlgIjUV6hq1Z8OY6s5PO4dFMa0tY8=;
        b=R1pnkXEJcfYc+28FcQjiodepvoqAm2kgxDVaheyqwebTf3FjDDSIxcAyd0ADvE+KPX
         22VncIy/w2c/lOoZy7niOrzzSx9qkn7ZEKooQ9Y3e/+vGntWTTdXRGfgaUBMbfnYVVIc
         cZCj0SYuspUxnpvuDNPZ4vtnpvhAPviCL7uNVZ2S4H5NdzQBCJJCti1xk/gszGhwuc5Q
         fKLTRe3ToQgPHRL6BKCWVMUsjsrBZc93YAKEjd8Hj4zVfp3Fnn5RVturBii9El/GBCrC
         b7mwVRBq1ap7eH8tcYJdJry2c8UB/+oFfsAFrNylIEcc+t1EJeMH2yMoEl6tbPO0zCnC
         L+FA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=M2BgUTmkSFrfNfDlgIjUV6hq1Z8OY6s5PO4dFMa0tY8=;
        b=U/CJsdOCacWYd698of2ZtUYslSJpIBIeDkZMTPzQe881D9hycIIVgBmYN/mxtxMfLa
         /iFjcy8iRb5FDmF0QCOHIqO56K5kDsSeVt8vt0nf04vWtWnYj6Slcf33RpnEuumKVT7B
         D2vn7jG9GUGjCvtVbB2pE0Ag1Qzur/bprYN7aD6qhsbEl8UKd9a19d+NC8DdYWyM1W08
         +EPr0jJYTObZvyf8JvkaorRr3WjASdzIeanpMhHLn2tvmFBXtXRTN078NEMpfJZwgUgf
         B5vd4f5bL5JFcJZmMmRgGW4qvHc6lqGjlQP3dSdO6WoennVufQjnjvy1ocH5wa+yVOfI
         bAVQ==
X-Gm-Message-State: AOAM531FGGy0hCLiJSzGkJ4u/ezzfBLKgiECdOa3UW7v6GCDZvgf4GeX
	LM8UwnjhigiVpBLGjoATSdAk/3sqopd23Cxo+X0=
X-Google-Smtp-Source: ABdhPJy+nnQNBVJSZ2S1YKoRMLa7tn+xyYINMcU/0hMn54oN/NAK4IKQ4LxpufB3XjzInI8evLXofg83llKcxb87sTs=
X-Received: by 2002:a25:af07:: with SMTP id a7mr13941247ybh.329.1631525811540;
 Mon, 13 Sep 2021 02:36:51 -0700 (PDT)
MIME-Version: 1.0
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol> <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org>
In-Reply-To: <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org>
From: Sai Kiran Kumar Reddy Y <ysaikiran1997@gmail.com>
Date: Mon, 13 Sep 2021 15:06:40 +0530
Message-ID: <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com>
Subject: Re: Xen Booting Problem on ARM Machine
To: Julien Grall <julien@xen.org>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, kannan@cimware.in
Content-Type: multipart/alternative; boundary="0000000000007057c005cbdd34db"

--0000000000007057c005cbdd34db
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Sep 10, 2021 at 7:30 PM Julien Grall <julien@xen.org> wrote:

>
>
> On 08/09/2021 11:43, Sai Kiran wrote:
> > Hello,
>
> Hi,
>
> Thank you for the report. Moving the discussion back to xen-devel
> because this looks like a potential issue in the UEFI stub in Xen.
>
> > I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it.
>
> Would you be able to give more details on the Arm machine you are using?
> Also, are you using ACPI or DT to boot?
>

   Sai >> DT . ACPI configuration is disabled in Boot settings

>
> >  I
> > am able to do =E2=80=9Cmake world=E2=80=9D and =E2=80=9Cmake install=E2=
=80=9D, after =E2=80=9C./configure=E2=80=9D, as
> > specified in README file. When I reboot the system, I get the following
> > message:
> >
> > Warning: All 128 bootinfo mem banks exhausted.
> >
> > Warning: All 128 bootinfo mem banks exhausted.
>
> Hmmm... This means that you have more than 128 memory regions described
> in the EFI memory map. That's quite a lot.
>
> Although, this should be harmless as it means Xen will not use the extra
> memory banks.
>
> >
> > Cannot exit boot services: ErrCode: 0x8000000000000002
>
> This means EFI_INVALID_PARAMETER. We have code to retry because AFAICT
> ExitBootServices() may sometime fails (I have CCed Jan may have more
> idea what's happening).
>
> Would you be able to provide more details on the UEFI firmware you are
> using? Is it EDK2 or U-boot?
>
> Sai >>  EDK2

> Also, do you know if Linux is boot on the same system?
>
> Sai >> Yes

> However, AFAICT, the error message would not prevent Xen to continue
> booting. So you may get stuck later in the boot process.
>
> My suggestion would be to enable earlyprintk for your platform. You can
> setup it up from the menuconfig in "Debugging Options".
>
Sai >> Yes, I have enabled earlyprintk.
I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to 256,
from 128. The error message is no longer seen, but the device is stuck in
the boot process.

> >
> > Here is the snapshot of the error.
>
> Please avoid posting image on the ML. Instead, if you need to do it, you
> can upload on a web server and provide the URL.
>
> Best regards,
>
> --
> Julien Grall
>

--0000000000007057c005cbdd34db
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Fri, Sep 10, 2021 at 7:30 PM Julie=
n Grall &lt;<a href=3D"mailto:julien@xen.org">julien@xen.org</a>&gt; wrote:=
<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8=
ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
<br>
On 08/09/2021 11:43, Sai Kiran wrote:<br>
&gt; Hello,<br>
<br>
Hi,<br>
<br>
Thank you for the report. Moving the discussion back to xen-devel <br>
because this looks like a potential issue in the UEFI stub in Xen.<br>
<br>
&gt; I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it.<b=
r>
<br>
Would you be able to give more details on the Arm machine you are using? <b=
r>
Also, are you using ACPI or DT to boot?<br></blockquote><div><br></div><div=
>=C2=A0 =C2=A0Sai &gt;&gt; DT . ACPI configuration is disabled in Boot sett=
ings=C2=A0 =C2=A0=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">
<br>
&gt;=C2=A0 I <br>
&gt; am able to do =E2=80=9Cmake world=E2=80=9D and =E2=80=9Cmake install=
=E2=80=9D, after =E2=80=9C./configure=E2=80=9D, as <br>
&gt; specified in README file. When I reboot the system, I get the followin=
g <br>
&gt; message:<br>
&gt; <br>
&gt; Warning: All 128 bootinfo mem banks exhausted.<br>
&gt; <br>
&gt; Warning: All 128 bootinfo mem banks exhausted.<br>
<br>
Hmmm... This means that you have more than 128 memory regions described <br=
>
in the EFI memory map. That&#39;s quite a lot.<br>
<br>
Although, this should be harmless as it means Xen will not use the extra <b=
r>
memory banks.<br>
<br>
&gt; <br>
&gt; Cannot exit boot services: ErrCode: 0x8000000000000002<br>
<br>
This means EFI_INVALID_PARAMETER. We have code to retry because AFAICT <br>
ExitBootServices() may sometime fails (I have CCed Jan may have more <br>
idea what&#39;s happening).<br>
<br>
Would you be able to provide more details on the UEFI firmware you are <br>
using? Is it EDK2 or U-boot?<br>
<br></blockquote><div>Sai &gt;&gt;=C2=A0 EDK2=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">
Also, do you know if Linux is boot on the same system?<br>
<br></blockquote><div>Sai &gt;&gt; Yes=C2=A0</div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20=
4,204);padding-left:1ex">
However, AFAICT, the error message would not prevent Xen to continue <br>
booting. So you may get stuck later in the boot process.<br>
<br>
My suggestion would be to enable earlyprintk for your platform. You can <br=
>
setup it up from the menuconfig in &quot;Debugging Options&quot;.<br></bloc=
kquote><div>Sai &gt;&gt; Yes, I have enabled earlyprintk.</div><div>I tried=
 changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to 256, from 1=
28. The error message is no longer seen, but the device is stuck in the boo=
t process.</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
&gt; <br>
&gt; Here is the snapshot of the error.<br>
<br>
Please avoid posting image on the ML. Instead, if you need to do it, you <b=
r>
can upload on a web server and provide the URL.<br>
<br>
Best regards,<br>
<br>
-- <br>
Julien Grall<br>
</blockquote></div></div>

--0000000000007057c005cbdd34db--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 11:02:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 11:02:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185622.334306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPjjK-0006cA-MD; Mon, 13 Sep 2021 11:02:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185622.334306; Mon, 13 Sep 2021 11:02:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPjjK-0006c3-JC; Mon, 13 Sep 2021 11:02:30 +0000
Received: by outflank-mailman (input) for mailman id 185622;
 Mon, 13 Sep 2021 11:02:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPjjJ-0006bZ-73
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 11:02:29 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1203d6e4-1482-11ec-b3a8-12813bfff9fa;
 Mon, 13 Sep 2021 11:02:23 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-8dxmdYFKNO6XUMh9q41vyQ-1; Mon, 13 Sep 2021 13:02:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5902.eurprd04.prod.outlook.com (2603:10a6:803:ed::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 11:02:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 11:02:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0013.eurprd09.prod.outlook.com (2603:10a6:102:b7::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 11:02:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1203d6e4-1482-11ec-b3a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631530942;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=bUI1q/jRgHPdFA8Csjovr+XAr2YYQ4AnKtiho9gb5Pw=;
	b=UPiGs7t4IBnl6yxsel+XPi5Cj/GlaX2HRyMrtMPB6VC5fNffuM6kn8RrANpKktGteKF7MB
	xXmCSgr9/1dRuMv1wKtUrDLnoFC2Y8fwtNPnQrLfEF0RJ8P371CGBmAauDDTfoTs8WMMek
	XF4QJm4aAYy/fBnsXbmYqwCoSYj2C2M=
X-MC-Unique: 8dxmdYFKNO6XUMh9q41vyQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Qoh9AprcOE0yF+7kyYDcrghKnyRpAZ4ukKTraIk41zvlPcolo8lm0sAP+d4EWOkWYliSlCdd6HbK1YlHVFCwLIClAT7xOQvvRXavVnwXyzYNlC0OwlhzkuLsuFFGgUcM6HEXlUTz9DCUZLyibtsErmXG2SNYvXOebBIg91wal1CnyqQt4CW4N36FTdfvx690rIxj2MZ3Mf8PiB9o9+qsk2lRT+Y1EkLljShppnG+A/nbLhvrBtpggqpUMuAiGxL/GaDKsMsIwicOdl+ns6HUgtnxGPZY1poo7mrcpvPX/J/4O1wUyZYe0RYDC1b8672uL5DLgeLQd2SYd0PSFsbRmA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bUI1q/jRgHPdFA8Csjovr+XAr2YYQ4AnKtiho9gb5Pw=;
 b=LmI40m4iiMfXqep+l3GtDV+95vQCutkCbbqKVjh00QT7KLSFHpzTQM+FJEhNAqwBG67oshttFRd6g9ztglDeg8EutZFmIx3SiRBwUAmq3zCTZMBIqu4FSIh+U4f5A5onuWRgAj5h3bJi6cwUbgT1Zqoq/ZKTQuQaHb6Yms6nA8UYeJvypA0K5P1n1TOJU/r/Vez2sjw8GKgu+7PlDQbVSJheMqzvQL+dP6zb4EUMWdD6/6dFZvQmHSLO7JsKVLEIe2J5/9GwDczfTFm9t4eT9eecB1LQD3bnkYRDaw7m7YUvYlW/W9E92oLGU30BCDpZwjioUgnmmDc4ryzwUA4t2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: Xen 4.14.3 released
To: xen-announce@lists.xenproject.org
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <d0a3c066-f3be-ab7f-3194-5320ad01ea09@suse.com>
Date: Mon, 13 Sep 2021 13:02:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0013.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: defda201-1dd2-4c9f-7bc1-08d976a5f3f1
X-MS-TrafficTypeDiagnostic: VI1PR04MB5902:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB590268F83083755E6E327134B3D99@VI1PR04MB5902.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F2BXtLcSffSDYSvhd9UhVA/SxA9VPbfCD4pg8DeRr3TLi26HB5U/BjAFLPqHKdzrAsU/vcEMRZR3VIWjG87gue3+aH7bq+n5DV4SntpRoA+rSx8SvGlr7GqyE/n4FCK4XIYrdT/ivsZOnO6M+SAPrF8RxCYjnVfHarCUwuc6q2j5TlJzBYjHhCGs8cg0mDOjmnsMiL9rUqBHTZBY+Ak+vxpbuYZPhwae0yVJhh2MyDh+KRjjTEsgSNAlbsDGX7NM4LSfWUTlPjrPeNaDm+M9cxxxf8Ot4R8y5s/9k2H2f3gSSne0JumMbOu3XOX8UWwu8/zzcQMquww/NbK+OwQndJiAs68+9D4KGj9qA3Y9o5asXB67kWwHJGu5ilwmAqGwOlC6LBlXIadNH+I136IdgEk6k/VwN9pNZDLrn2HoVD262s2zJkPYt8H8JIKgwJxrMsLtYbTT79Z9i9llgXplNHHwIjwF7I0zeH3lvN+GcfTk14siIjO/cvhYRN3QA8mg4W8A2dH092eX/pdaVdR1YNacF6LfmMoGnHlw1dBv3tfgrHx0CiuCdqibJGP9d7K8nZ67jtabIfuWl3/VnWkdaecwbPJmk7wPCiN/pVt0lORvwXVLZKUdkMItzogDfnO5LZGDf1rKDimS6NJSwzZQxnwbndZJb4p6J0/NCzcRx+Mp3ZnU9J2wlzHcNGcm3ZOuoQ/iI63vgf8HFayA+q3I8yE7YVlcv6FqSF3zPJ275LEinTzODCJOWNv5oBSmUWgGwtP4vS64PdkRdMFyDTqoQg8Wi3SJTU/6mH9Lq30ajg/EVPAEN2pxt2MbHaYaqMER
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(346002)(366004)(136003)(376002)(31686004)(38100700002)(6916009)(316002)(66946007)(4744005)(956004)(2616005)(7116003)(5660300002)(66556008)(36756003)(66476007)(6486002)(2906002)(16576012)(26005)(966005)(450100002)(86362001)(186003)(4326008)(8676002)(31696002)(83380400001)(478600001)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cWhrREl2WlJEczNyTWp3UWdKV1oxYlM1MjV1ZnNselp4dU12TXZCK2pIZGs1?=
 =?utf-8?B?QTlwRUJOOTA2OTBNeHZmZUg1MzUyaTBHa2dycGdPNEZYOFZlVW9CdG5DQUFN?=
 =?utf-8?B?UWh4dW8xbmJTR0NuSW8wREgwMDFMS2RmVCt6cHZNR25ONWZDNnROaTVKZmVJ?=
 =?utf-8?B?SEVZN01ydS80NXpXK1NheVFOWGQxaWJaR1NtQnltNVFYZE41UHNscXQyRHZP?=
 =?utf-8?B?Yjdyd2pJbnZ4VFZqZHM3TXE1cnBUVHAxeEVCOTBpb2owRStWZjBRdWRVbkFo?=
 =?utf-8?B?NFlsZE1WZDM2QjFsTDZ0N2FnUnJGVjZIdFowTnhaSUh5TUxIQ1h5VUhGenhU?=
 =?utf-8?B?aFhiZDQ0MEppa2Q2QnQ5MzZxeVNSUUIycDlCWmtrdXBaemFrbzdDa3lHNVM1?=
 =?utf-8?B?L2dJLzUrLzFsTXc4TlNzd2J5ZGw4ZlBuakFtY2wzQmFSVEtkUXRleHFrcUw5?=
 =?utf-8?B?MVhTK3dCYWV0NndvU2hiTzVDSS9zYm91ODlzRURvQVVXbGJWVzFSSmlVemND?=
 =?utf-8?B?bDRXZWM1SlBQZm1zNUNhSVJlWHlNbmNmOFpJbC9wUzlNNGZRbzFwdStjWEFa?=
 =?utf-8?B?OStuMElINXBiNkU2dmo3VUJiWnAva0RxWTBFY1A0VjI1MlRFank4aXl6ZWJY?=
 =?utf-8?B?ZDZMbjV0aC9YODYwUGpUMy85SzMrTnAwQ21ra1Q5VVlZS2ZGcnpIUHBLOG1N?=
 =?utf-8?B?Rm1TMHNRYTE3NkNPMk1CSFZUREpJNDBIWWJjTkR3Y2gzU2M4ZUhmUlBVbWND?=
 =?utf-8?B?WHFESURIRnhpbS9yYlVWWWN2QmNrVXM0am42TTBKS0xsQTRoOTNYZHF2Ulo1?=
 =?utf-8?B?eVp2S2F1WnE0Mm9DcnAwMzVKRjMzVHZqYk1tRDFRS0lYVVlPcm8yTkN5QW1i?=
 =?utf-8?B?MTA2SldIcGZDeGxDYkFtcnpnL1N6U0trNVpQOCtvTm5oTW1rUXYvUXk2aU9r?=
 =?utf-8?B?c1FSeGFCQ291U2htZFQ4dTJDS2pYaDlPbitkQ3pKTnpFdjN6QnB3TVdnRUdM?=
 =?utf-8?B?Zlk0QkFQL3JDQ2RyUG5yY2pQT2JxWlFFS2poM2JtYjk4aVBjdzJjanhPVEhR?=
 =?utf-8?B?L3d5NXhKYXNqUnRFNks0VmtMQkNxVmw0cmZYN3NXZ1pTb3BzbmRkVDBUeUg4?=
 =?utf-8?B?Ly84SG0wMHJMYVFYSDFNVTRzNWx4YlRrZXlEQlNBaExHV3hLcjZway9sQngz?=
 =?utf-8?B?Tk5OeFVpQ0hxSHRpLzNwdjlLL2ZqN0ZHZXZOTEt6c05acXRYaTRGN0FQWnQ3?=
 =?utf-8?B?T0NNQ3p1SGNPV2dCOFNJaXRSMDBYTUVDaFJJK3ZWWjlMK1JCZm5lRWhUZ2l1?=
 =?utf-8?B?bERKMjVGeDArS1JvQytmRFZjS1FyN0N5RWNsZGlvR3hxWlJPR29jaEVlaGts?=
 =?utf-8?B?ZHRkMzR6WDVvVEZJdUVobU1sYmJodkh1UmJpSkFiUU1UZWpCNlpLVGdMVTcz?=
 =?utf-8?B?NFo0WnEwRjRvWnhyM3drcVQvKzJDSldlT01uK1RMS3dpYzl2bHhWNWo1MWs1?=
 =?utf-8?B?U0RnOXZ6YzNMNm4zVUV1UVNZT09aOFJaWmI1WElmM3BlWklQS2hoQkhlWFBN?=
 =?utf-8?B?UjU4a1JaT0RrR3E2TGk2Tzdjd1lqUXJPMTA5Wk53OGlhamtTK29zeDlwYkM3?=
 =?utf-8?B?Sktya0tobitOOHh3SVN1WWhMT3k0TUFsVW5NYkU4cE1OY3RIK1ozSWhRZDJu?=
 =?utf-8?B?dUNwd2tvTzhQMXFHRUxDMElDbGVqRVBxOGY3M3Fja1RoSW5tM3o3R3RQdEZV?=
 =?utf-8?Q?q6OiSMeXtmBBNaPVtySZ605TM0ZSwDOk3BgqHJV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: defda201-1dd2-4c9f-7bc1-08d976a5f3f1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 11:02:18.5036
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TQ6HIlie9qqT51ZF4rWgUoV0G0MXm61Xi2C7TlNVgB1UZPaEJMUDfC7EkkToJINQB2VsFDmiZKJJWc+gupEXJw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5902

All,

we're pleased to announce the release of Xen 4.14.3. This is available
immediately from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.14
(tag RELEASE-4.14.3) or from the XenProject download page
https://xenproject.org/downloads/xen-project-archives/xen-project-4-14-series/xen-project-4-14-3/
(where a list of changes can also be found).

We recommend all users of the 4.14 stable series to update to this
latest point release.

Regards, Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 11:20:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 11:20:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185630.334317 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPk12-0000k3-5l; Mon, 13 Sep 2021 11:20:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185630.334317; Mon, 13 Sep 2021 11:20:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPk12-0000jw-2j; Mon, 13 Sep 2021 11:20:48 +0000
Received: by outflank-mailman (input) for mailman id 185630;
 Mon, 13 Sep 2021 11:20:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hZsy=OD=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mPk10-0000jq-G9
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 11:20:46 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a7074c2-759b-44aa-8356-86c3356b1671;
 Mon, 13 Sep 2021 11:20:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a7074c2-759b-44aa-8356-86c3356b1671
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631532044;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=VqYPEljlgoLTiGyunFHVzOz/Hs8XXd0NXcxD7WPjkEc=;
  b=Y5u9QvRGQgnUVb553iw0s04zJktXBeUD38rOJmzsDmWPUAAkryFoAqpf
   geLf1dfoY8sByHeM0yGeomR481U6ewnEF9ltU0MWju+zujvKFN7wnUzRe
   kI6dLAFRbapcpnhZPcvWSQcZu6Dj/enwql7q+URC2IylHgz/6SqLRaH3+
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 2ku4Lm4i1i6PnQOBIkDlCUco2ia6Xpgza/K8AUK4TV66R+qSlRfu2YRmF6wE2klcxOEXf/1VSr
 o4FGxViDAYKUKBg6cfn5DyPcHzgOXXE3kYZZ3j3wUpO1E7W1X3Mxw0oBjvmngGreh+m/YFqQyo
 8W+pA+odna8hqYbaHCo70+/MHsc7SUCQ1DQqx+Oya6wJRCg1cnUCAys3wX6v3xumZZ5ipDL09Y
 ONHi+khlJ7XmLVJ2dXvTNRR74LIcbD3ZhC++W6l16jOMvEdGLuBtq08NvYH6FZa/dosXiV39dg
 dyr4UKL7qJ3oAND1W9svq0me
X-SBRS: 5.1
X-MesageID: 52977045
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:BbRc76u9m+7NyU58R1Guae66befnVK5ZMUV32f8akzHdYApBsoF/q
 tZmKWjQPq6DZGv1L9FxYY/jp0pTv8PcztRgQVNrry41QnwS+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y52IPhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NplsbaCVRZ3DIfwlOEPXzQEMmJ3I/BK0eqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25ofQqmGO
 5dxhTxHMhfpMzZSK34tDJ8spL2ql3vHTmF0pwfAzUYwyzeKl1EguFT3C/LPfceAX8JSnUCwr
 2PA8WPiDxpcP9uaoRKf9WmmnOjIniXTV4cbFbn+/flv6HWXz2oPF0FKfVS+qPi9zEW5Xrp3K
 lMW0jojq7Ao806mRcW7WAe3yFabujYMVtwWFPc1gCmdx6yR7wuHC2wsSj9adMdgpMIwXSYt1
 FKCg5XuHzMHjVGOYSvDrPHO92r0YHVLaz9ZDcMZcecby93Jkp0i0CDlctpmAqKY0vy2EC+tz
 xnf+UDSmI4vYd43O7STpA6c2mj3+8mVEWbZ9S2MATn0tVoRiJqNItXysAmFt64owJOxEwHZ1
 EXojfRy+wzn4XulryWLXOxFN7Wg/f/t3Nb00AM3QsVJG9hA/ReekWFsDNNWfxwB3iUsI2aBj
 KrvVeV5vsI70JyCN/MfXm5JI552pZUM7Py8PhwuUjarXnSXXFXalM2JTRXLt10BbWB2yf1vU
 XtlWZ/0UR729piLPBLpHrxAgNfHNwgVxH/JRICT8vhU+ePFPxaopUM+GALWNIgRtfrcyC2Mq
 oo3H5bamn13DbylCgGKoNF7ELz/BSVibXwAg5cMLbDrz8sPMDxJNsI9Npt4K9I8wPUEyb6Sl
 px/M2cBoGfCabT8AVziQlhoaa/1XIY5qnQ+PCc2Ok2v1WRlaoGqhJrzvbNuItHLLcRvkqx5S
 ecrYcKFDqgdQzjL4W1FP5L8sJZjZFKgggfXZ3ipZz02fphBQQ3V+4C7IluzpXdWVifn59Ejp
 7CA1x/ARcZRTQpVE8uLOumkyEm8vCZBlbsqDVfIONRaZG7l7JNud37qlvYyLsxVcUfDyzKW2
 hy4GxAdoeWR8YY5/MOQ3fKPrpuzEvs4FU1fRjGJ4bGzPCjc32yi3Y4fD7rYIWGDDDv5ofzwa
 /9UwvfwNOw8sGxL64csQax2ya8e5sf0o+MIxApTA3iWPU+gDaltIyfa0JAX5LFN3LJQpSC/R
 lmLpotBIbyMNc7oTAwRKQ4iYrjR3P0YgGCPv/E8IUG87y5r5ruXF05VOkDU2iBaKbJ0NqIjw
 Psg55FKu1Du1EJyP4bUlD1Q+kSNMmcEAvcuuZwtCYP2jhYmlwNZapvGBy6quJyCZr2g6KXxz
 uN4UEYau4lh+w==
IronPort-HdrOrdr: A9a23:h1sWU6rJBvFyEgXxw+4k76caV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NCZLXTbUQqTXfpfBO7ZrQEIdBeOlNK1uZ
 0QFpSWTeeAcWSS7vyKkTVQcexQueVvmZrA7Yy1rwYPPHFXguNbnn9E426gYzNLrWJ9dPwE/f
 Snl656T23KQwVpUi33PAhJY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX232oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iBnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDA4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWArqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocWTbqjVQGagoBT+q3oYpxqdS32BnTq+/blnQS+pUoJjHfxn6ck7zA9HJFUcegM2w
 2LCNUvqFniJvVmGZ6VP91xM/dfPFa9Ny4kAFjiUmgPK5t3Tk4li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,288,1624334400"; 
   d="scan'208";a="52977045"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CuoRI8TAZ/X8EBbOOydpEC3B4h7wVoeVeZOinPFkAHODstjk0Q7ttPxTd87v9nn7xEpW/+UBSTqRkbuRXfBbD8d92z5PaYMiyw2GNZ7XhLyQyHCymuvRtlirKBlCP86gWXcdFUGUXHwzwm+UrKQFcUsT/sh4Yetx0tw3fLc8zgStmnEUkRflyGE1mjZXWrtFjiRM3KJhbGCoiJsdq7ODCixQ20D4Satv0vTvwHQZ7YRdcGqJlZSmqAc2xHnJz0Zr/MBTg3pPCYFZfiqu/ZXTSuE9YkWXtXHdXyN12bVcD9apaaw2JsOdWb92xWk12K22JK3pDtkpxY52QqfAZTOJFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5oyrCDiMSrMpA7iodNQnls1mmEHmZJdzQbV25sFyCgs=;
 b=QAZzIy9emoi6BfQJ6lN0TZCUsKmyIrK0gz50gvV3G9DM5gjFpdDAP58UBTgAuN19beq33Y1/ABLYN+UbNVwZmG8SaWE98enq2eWFmB3Ks+4Pp7bWXJm9HkkKZfo46ULrs93RUrGwojCc4E5kmkazhmsgsAbZidQLr2U3XN07kwuGkh8gE44upziTmejjfgrdq+1bXv17Ntd+lq5hRnM8Qu/9/aDm5SAIZg+BtsIxyY8MhTV8XUCmjAIYB2+4eSQQqtCpfDf/fZTR4cARMM4RRmnKo6HdehjN5gu9oIlJmhsDLxW/sGb8PeeA88nb4QB+n2gLXcGLYDK4fmSdiKaRcQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5oyrCDiMSrMpA7iodNQnls1mmEHmZJdzQbV25sFyCgs=;
 b=oNqpGRCpteDJL9uRTKxrRUR5Y9ecEGG6Ruq8VTLaVITJ7ncyI8PbX2R/12yH4h5goE2RDxn2YtprVMS4vnXEogTQTuKhlt+PdZWlGmoPr9X+UZBG1PQ8BGcVoBkHWx9zk3RsGSGbjH9tU1fRWs7uivBhYtM40eNEh9SvVhUWM3M=
Date: Mon, 13 Sep 2021 13:20:20 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
CC: <xen-devel@lists.xenproject.org>, George Dunlap
	<george.dunlap@citrix.com>, Nick Rosbrook <rosbrookn@ainfosec.com>, Nick
 Rosbrook <rosbrookn@gmail.com>, Ting-Wei Lan <lantw44@gmail.com>
Subject: Re: [PATCH] tools/go: honor append build flags
Message-ID: <YT8z9DUaxOyx9gcz@MacBook-Air-de-Roger.local>
References: <20210707071531.69001-1-roger.pau@citrix.com>
 <YOWvPLcQywmEIPdI@FED-nrosbr-BE.crux.rad.ainfosec.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <YOWvPLcQywmEIPdI@FED-nrosbr-BE.crux.rad.ainfosec.com>
X-ClientProxiedBy: LO4P123CA0321.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 92956759-8197-4818-f33d-08d976a87c17
X-MS-TrafficTypeDiagnostic: DM6PR03MB5356:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB535629C479E314D481DA821E8FD99@DM6PR03MB5356.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 4rklQWGXnErEWzyLJeYuJo/7+/RznxY4H5rsAjq7GSzb4RheT7vSgOx3YiFBMI+6nzwRPfpKM7mPf6QmEHX7mqj8/Ez/ixdiZ+BaRSWAqPeNK9byoUqFnQPkJoBe18WeU2VKI9///YIJaICjqLeuq1fCu46ZHeUXiB+uRFu5eiRH2/AZJkX7xdfHPb/A7zeBAWqxCwjoU1RpQAjy8PJP5zX2Bc/6E3ufHTkBSS3GpzckxzeWSuCZk5hLaL1Sms8N37w9fcI/NJVS8reh9C+DNmsWLJUZAau079HUAnST7SlyyHkCP3xmDkQo1LDf5xHdCXxFEyK0tgwxUR1fsgQ5S5e2UOh0sQHamGRH/YawIpFEmXYmsxytXK3Wc18Ktox9X8LIhR4HAyHqvouQ94PdQISDcaHoCL2WwfYnT32HR3dvbB9xbp2pOtkqO+ADG9EKDmyPSyuLuOFQLbbEU7TEUzJeZhjbvYxv4S1oHB+OzXWnIzt2D8OtdYEFrcMhtDkVLaX4PqjtDRoy6lvgWuPCRuF5j/yI42t4SYJ86jfwIvxU+wy6dJkZviu/qg0l1M8dgQ5/jOIplE02wrTiZW718ALHib7tuzGdzO9D04ILt64r0f0cdXhquAxhIrbB7j4VCO5zad7Y1lG2rc0/zl1fwQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(346002)(366004)(376002)(5660300002)(316002)(66556008)(110136005)(54906003)(66946007)(4326008)(26005)(186003)(9686003)(6666004)(6486002)(956004)(478600001)(2906002)(6496006)(83380400001)(8676002)(38100700002)(86362001)(8936002)(66476007)(85182001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZzIzRmFxYW1Db09ZL3hqd0JNQlhoOWlIWEg2UXgzOWwxc0dVekVYdkFzVUhn?=
 =?utf-8?B?Wmw2RUpZaTV2YTZUdm1tSDZNb0lWYkV2MmtJNTZnRzUrVU1vT1BIdDNoaWVp?=
 =?utf-8?B?bThPMDI4Q0dheGtzaGlDd2N6Z0FzN2w5cm9WZ3JXQ3VJOE45R2tWVVlXVk5L?=
 =?utf-8?B?UUhBQ2V0SjdFcHhlQ2gzVWNMK2NvaXlja1NRL3ZuR0VPaDltU1VKdnN6TTJn?=
 =?utf-8?B?djN4V2JUejR5d1hxT2NkeWRuTDRBNXJvWjY0VERxcFdKSUtTSUhQK1NvT09C?=
 =?utf-8?B?WDJuMTdlalhWZ003T1VGVWtyazdXM3pDQm01UWhUNHJZc2h4UUhlU1dFekZ2?=
 =?utf-8?B?alo3em1sT0hGOGJwL3BzVDVyQVlFTWJoQUR2b2hFd3dURlZWN3kycytoRCtm?=
 =?utf-8?B?THJVTGY3YnpsWlNwWTVXaG8zWlhFRjJLK3I2WWo5Qm5xNmVRZjNxclVXNXJi?=
 =?utf-8?B?Q0p4NHFJb2Z6MkFEcDdCcVRXNmt4VjJpRXZqV0JYcXkwZ0ptcjNPekpKSEpZ?=
 =?utf-8?B?YkFyRUYzYm5HNnFiNkJwMzRmamdSbVNhT0ZlMWlocXRNeTdqVCswSG0rbS96?=
 =?utf-8?B?VlQ4VEFmdjA2cnhWYlgxc0tsa3BjTzRlOG0wcDkvQ2kvNFZ3eFJ5VE5zTGpY?=
 =?utf-8?B?TFNYdDl2VmZqcGxybXMwYkNSZTlLUFNFeU01M1B6dFJZaGordUZtT0JzOCtp?=
 =?utf-8?B?cHhzeHFPVlBJZU9sRXVkOUxqY2phSEtESVoyZTEweEorQ1kxcldFRkhJYzhF?=
 =?utf-8?B?amNvSnBweDBGSTVZUDRjTU56WDVrcCs1a29XdGxUZ25MTjlQMCszeExRK3Q2?=
 =?utf-8?B?WGxXRzVWeWlMMEhCMW4zZmJJcWpCaGtoMjRmWjBoaXQ5L1pYcnlYa2ZFUTU1?=
 =?utf-8?B?b1hBUUFEUVAyL0RRS1ZNNXIrUmoyWm42cG9Ia2tSeU0wNFcyQlJpRkgyVlRp?=
 =?utf-8?B?MFI5dmNyN0tUYTZNb0d0ZzdVdXZ5SWd4NjNPWnlqbi9BcjFPS0NxWjlvWVpD?=
 =?utf-8?B?REw3bzNwbXBMUUhmcjllWDJHWG5FYjh1eDJFTDVFZjV3ZlR4VHczQmt6UVNj?=
 =?utf-8?B?b2NQZXhDZ3JidEpkU1ZKeUIwOHNqS2lOdWR1OUJmaklGeU54aGtUWUlySG12?=
 =?utf-8?B?WkUxVEJiUk41Yk91RnRzYnBHdEY4OTFQMVgrY3B6N1JPSnJJdWxCbExkYk42?=
 =?utf-8?B?akFjRXNnaHk5T0dVRm1nTGY1a1RBbmVhT3lPalBvQmhtajhsdUY4Y0dVekNF?=
 =?utf-8?B?S3VyR2xCcGFQczYwMTcrM0wvMm1vL3JFTmtnQ25KVTYyZGVEYS9VTm5GNFAw?=
 =?utf-8?B?Smt6QUpUSWRFOW5UVjREWG10MmFvMjNCZXJicThlTC9zVVg5MXVnZEppNGFW?=
 =?utf-8?B?UW03c1BQUzdMU01Lcmh1MEFuK1YrZ0xncUljTGd1aGtuSGJtZVBmeDcwZG5r?=
 =?utf-8?B?OWtBdGdpV2hwVFQzaWhMdzdnZyswbUh4dVAvdVlRQU1NWXNNOE1OTGw5TXpW?=
 =?utf-8?B?alpQT3EvSkx3ZEl2emxuaXhZdkVKY1JvWHBxRUN3ZS9TQTJqSjB0V3o5NkpL?=
 =?utf-8?B?MU9FT3VGWmc0WXVQa1dTZEZzcEV0RGZrVXIrdkZoelF5QnVDYitQbjE4ZE5a?=
 =?utf-8?B?cHplRFRzUndPcFlGTGdUMFJlUXdIZGxkc0RFTjZXTTY3dEVWdmZ0ZXl5SlJ6?=
 =?utf-8?B?L21sRFAzRnhPZ0Y5V3dwM0pqVWFkNnowVHVGSjBxMkx6K1hkRlRYRWtNeTRk?=
 =?utf-8?Q?rFxF7sDTsygD4qFcuoZ+xDzJvD/I4norVn5GUiX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 92956759-8197-4818-f33d-08d976a87c17
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 11:20:26.0458
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M7ntZ/2OOimWMs5k903EWE13qqvjsTOU6FUlS3adSk8LlPn9uADra27uVIYUYi1KhockP9CijesbMd5qdrHF8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5356
X-OriginatorOrg: citrix.com

On Wed, Jul 07, 2021 at 09:42:20AM -0400, Nick Rosbrook wrote:
> On Wed, Jul 07, 2021 at 09:15:31AM +0200, Roger Pau Monne wrote:
> > Make the go build use APPEND_{C/LD}FLAGS when necessary, just like
> > other parts of the build.
> > 
> > Reported-by: Ting-Wei Lan <lantw44@gmail.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> > Note sure if it's the best way to add the appended flags, I'm not
> > familiar with the go build system. In any case this fixes the build
> > when required libraries (ie: yajl) are not part of the system search
> > path and instead passed in APPEND_{C/LD}FLAGS.
> 
> This is right. The CGO_{C,LD}FLAGS environment variables basically work
> just like regular {C,LD}FLAGS.
> 
> > ---
> >  tools/golang/xenlight/Makefile | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/golang/xenlight/Makefile b/tools/golang/xenlight/Makefile
> > index e394ef9b2b..64671f246c 100644
> > --- a/tools/golang/xenlight/Makefile
> > +++ b/tools/golang/xenlight/Makefile
> > @@ -27,7 +27,7 @@ GOXL_GEN_FILES = types.gen.go helpers.gen.go
> >  # so that it can find the actual library.
> >  .PHONY: build
> >  build: xenlight.go $(GOXL_GEN_FILES)
> > -	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog)" CGO_LDFLAGS="$(LDLIBS_libxenlight) $(LDLIBS_libxentoollog) -L$(XEN_libxenlight) -L$(XEN_libxentoollog)" $(GO) build -x
> > +	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog) $(APPEND_CFLAGS)" CGO_LDFLAGS="$(LDLIBS_libxenlight) $(LDLIBS_libxentoollog) -L$(XEN_libxenlight) -L$(XEN_libxentoollog) $(APPEND_LDFLAGS)" $(GO) build -x
> >  
> >  .PHONY: install
> >  install: build
> > -- 
> > 2.31.1
> > 
> > 
> Acked-by: Nick Rosbrook <rosbrookn@ainfosec.com>

Ping?

Unless someone else has opinions I think this is ready to be
committed. Can a committer please queue it up?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 12:39:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 12:39:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185654.334327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPlF7-00009t-T3; Mon, 13 Sep 2021 12:39:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185654.334327; Mon, 13 Sep 2021 12:39:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPlF7-00009l-QH; Mon, 13 Sep 2021 12:39:25 +0000
Received: by outflank-mailman (input) for mailman id 185654;
 Mon, 13 Sep 2021 12:39:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPlF6-00009d-SK
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 12:39:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPlF6-0004u5-Qa
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 12:39:24 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPlF6-0008Vw-PN
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 12:39:24 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mPlF2-0002gl-00; Mon, 13 Sep 2021 13:39:20 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=p93Z4e4zyxvtw9WGM+b9HiuMVHLePdBTKDUcFtY6zxI=; b=OktoPjF6SZbYtqrURq9HnWrwtd
	kozZSXKFZP3YkfdLjriHB7HCpMA9haGWnS85D4cW9XyzrP9a/hNskK4InjI61JHt/iKHgMBgE71lk
	rIaPLGufuUYoNIOSv1lBS/pUBVH6t/erXed1i9uo7ni8BjUmwcyar5LZm5QPVBXQltew=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24895.18039.739149.734153@mariner.uk.xensource.com>
Date: Mon, 13 Sep 2021 13:39:19 +0100
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Wei Liu <wl@xen.org>,
    <xen-devel@lists.xenproject.org>,
    George Dunlap <george.dunlap@citrix.com>,
    Nick Rosbrook <rosbrookn@ainfosec.com>,
    Nick  Rosbrook <rosbrookn@gmail.com>,
    Ting-Wei Lan <lantw44@gmail.com>
Subject: Re: [PATCH] tools/go: honor append build flags
In-Reply-To: <YT8z9DUaxOyx9gcz@MacBook-Air-de-Roger.local>
References: <20210707071531.69001-1-roger.pau@citrix.com>
	<YOWvPLcQywmEIPdI@FED-nrosbr-BE.crux.rad.ainfosec.com>
	<YT8z9DUaxOyx9gcz@MacBook-Air-de-Roger.local>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monn writes ("Re: [PATCH] tools/go: honor append build flags"):
> Unless someone else has opinions I think this is ready to be
> committed. Can a committer please queue it up?

Thanks for the ping.  Committed, with my ack.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 12:50:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 12:50:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185662.334342 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPlPW-0002WL-Tz; Mon, 13 Sep 2021 12:50:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185662.334342; Mon, 13 Sep 2021 12:50:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPlPW-0002WE-R6; Mon, 13 Sep 2021 12:50:10 +0000
Received: by outflank-mailman (input) for mailman id 185662;
 Mon, 13 Sep 2021 12:50:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6NeP=OD=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mPlPW-0002W7-0A
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 12:50:10 +0000
Received: from wout2-smtp.messagingengine.com (unknown [64.147.123.25])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 203fb624-1491-11ec-b3af-12813bfff9fa;
 Mon, 13 Sep 2021 12:50:09 +0000 (UTC)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42])
 by mailout.west.internal (Postfix) with ESMTP id 53C05320089C;
 Mon, 13 Sep 2021 08:50:07 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Mon, 13 Sep 2021 08:50:07 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 13 Sep 2021 08:50:04 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 203fb624-1491-11ec-b3af-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:message-id
	:mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender
	:x-me-sender:x-sasl-enc; s=fm3; bh=+ThLjayzZvSuiYey9xMndGiL316IC
	WP/XAzfhhK/XQ4=; b=QST6alfCdjmOCJeMGPiFrbgPY5XjwFUFGqt7uplIpJaKM
	eOo5VPdGZNeqchyNdUioX5D59OJCHQqjuJCpn2yZfUjxMNzZqYGD7oXSBKnRlcjE
	+/gBKlFfB7clEc1X4X896LpSW/ZoR4FZo6+ZIEP9esVsj2Qq+gb+T8IoxBfdT04P
	X2jATdlr+XyVZ8O8jnV7F7Uc7jsdp2zzPvBBMDH10d1WtW/EkERxHxOB0wKokST/
	9ONDWkrRpMjK6EHU8xe6DkOtmwcq6+WkrTki4K8XG8VLVZR6wlzcA7pA6vhnFAL7
	gDgFNGUrdNlU1rCoW3NxnbG8NlC5AxEv/yKdGmB9g==
X-ME-Sender: <xms:_Ug_YVjbwJME2flgxQg8EG093FJAH83V1xzhn76vfmBOLlFvuaxKmA>
    <xme:_Ug_YaCQRPtXrpy7VzQWGhl_XYYb_YlsubjGJ1vSMh2vP8xTq-Sbre4U-GEjRxBEv
    wVz2QKMwT043A>
X-ME-Received: <xmr:_Ug_YVGsAaE2InIUPwj3dgND8IpwphXc8FmKrBTuYzv_3DfKuvzMB8Pxefk6zQndbuDuZmQsp9s_wWWOogPMwsLD7lvsSISN>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudegjedgheegucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfggtggusehgtderredttdejnecuhfhrohhmpeforghrvghkucfo
    rghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvhhish
    hisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeklefhveej
    teeghfeiteevtddtueffieehgfelgfevffejveeuffdvgeffgeevkeenucffohhmrghinh
    epghhithhhuhgsuhhsvghrtghonhhtvghnthdrtghomhdpkhgvrhhnvghlrdhorhhgnecu
    vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrg
    hrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:_Ug_YaSQFmzELwJQ2aSVEWDx7qZEoBEyqr_jL6D1ADJBgxCRc7i0eA>
    <xmx:_Ug_YSyyAhfr6QgEpqJxG_ocJvrppdZHVEYD8tMGUqArflpMLv12tw>
    <xmx:_Ug_YQ4BsCHyuxb-pgCDS-0OxvAT3jIcCHPKuiVIxyMb8_SUuo5KAQ>
    <xmx:_kg_Ycs6Dm-J7IrZ6PdbMG6iQzxNcuEsMbjYTyjRniYXwvkgwF4JyQ>
Date: Mon, 13 Sep 2021 14:50:00 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Mike Rapoport <rppt@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
	Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
	linux-kernel@vger.kernel.org,
	xen-devel <xen-devel@lists.xenproject.org>
Subject: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading related?)
Message-ID: <YT9I+Xs9wOPVCIVd@mail-itl>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="5CWEI5nLGyVosQus"
Content-Disposition: inline


--5CWEI5nLGyVosQus
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Mon, 13 Sep 2021 14:50:00 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Mike Rapoport <rppt@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
	Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
	linux-kernel@vger.kernel.org,
	xen-devel <xen-devel@lists.xenproject.org>
Subject: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading related?)

Hi,

Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the
kernel version.
Test environment:
 - Xen 4.14.2
 - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
 - Linux 5.13.13, confirmed also on 5.14

The crash happens only if the initramfs has earlycpio with microcode.
I don't have a serial console, but I've got a photo with crash message
(from Xen, Linux doesn't managed to print anything):
https://user-images.githubusercontent.com/726704/133084966-5038f37e-001b-46=
88-9f90-83d09be3dc2d.jpg

Transcription of some of it:

    mapping kernel into physical memory
    about to get started
    (XEN) Pagetable walk from ffffffff82810888:
    (XEN)  L4[0x1ff] =3D 0000000332815067 0000000000002815
    (XEN)  L3[0x1fe] =3D 0000000332816067 0000000000002816
    (XEN)  L2[0x014] =3D 0000000334018067 0000000000004018
    (XEN)  L1[0x010] =3D 0000000332810067 0000000000002810
    (XEN) domain_crash_sync called from entry.S: fault at ffff82d04033e790 =
x86_64/entry.S#domain_crash_page_fault
    (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
    (XEN) ----[ Xen-4.14.2  x86_64  debug=3Dn  Not tainted ]----
    (XEN) CPU:    0
    (XEN) RIP:    e033:[<0000000000000000>]

I've bisected it down to the commit a799c2bd29d19c565f37fa038b31a0a1d44d0e4d

    x86/setup: Consolidate early memory reservations

    The early reservations of memory areas used by the firmware, bootloader,
    kernel text and data are spread over setup_arch(). Moreover, some of th=
em
    happen *after* memblock allocations, e.g trim_platform_memory_ranges() =
and
    trim_low_memory_range() are called after reserve_real_mode() that alloc=
ates
    memory.

    There was no corruption of these memory regions because memblock always
    allocates memory either from the end of memory (in top-down mode) or ab=
ove
    the kernel image (in bottom-up mode). However, the bottom up mode is go=
ing
    to be updated to span the entire memory [1] to avoid limitations caused=
 by
    KASLR.

    Consolidate early memory reservations in a dedicated function to improve
    robustness against future changes. Having the early reservations in one
    place also makes it clearer what memory must be reserved before memblock
    allocations are allowed.

    Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
    Signed-off-by: Borislav Petkov <bp@suse.de>
    Reviewed-by: Baoquan He <bhe@redhat.com>
    Acked-by: Borislav Petkov <bp@suse.de>
    Acked-by: David Hildenbrand <david@redhat.com>
    Link: [1] https://lore.kernel.org/lkml/20201217201214.3414100-2-guro@fb=
=2Ecom
    Link: https://lkml.kernel.org/r/20210302100406.22059-2-rppt@kernel.org

Since this seems to affect Xen boot only, I'm copying xen-devel too.

Any ideas?

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--5CWEI5nLGyVosQus
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmE/SPgACgkQ24/THMrX
1yyKTAf/RFLXnOX6sQx74ErcNKHowyeAVTrcIbactKB5dIcODunG+9eHXFR/HSk/
7ywnpCcFHZCjha5xx+ShTaskQ89yDEiE9jLIFwR7sQKxucRXoCcAAkJufYYxRbbQ
qYHUrOFFXeNFtEc73mZrbfqguc3y8Xm6BYBdzcHcgVk+bZCD5BY8A+A0FdKbm0rM
1E+AfbfIdN5avDDxBgy3xa+DVF1ksmZwZ4tbKDizZ2azkK+WMgPL6h+Hk718OA86
9TAXsPlotdMBsYFcFDvXBkKDQYN42+y/A6DOFmasUTsGw0xFroU/YpfarKnWq9OZ
w7lWKDBmRMV8wH25LAIXltai+6G1tA==
=GguL
-----END PGP SIGNATURE-----

--5CWEI5nLGyVosQus--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 13:37:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 13:37:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185680.334356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPm96-0007HL-Bs; Mon, 13 Sep 2021 13:37:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185680.334356; Mon, 13 Sep 2021 13:37:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPm96-0007HE-8z; Mon, 13 Sep 2021 13:37:16 +0000
Received: by outflank-mailman (input) for mailman id 185680;
 Mon, 13 Sep 2021 13:37:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hZsy=OD=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mPm94-0007H7-TH
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:37:15 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f5ab43a1-0c96-4bf6-a39a-28d172b00132;
 Mon, 13 Sep 2021 13:37:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5ab43a1-0c96-4bf6-a39a-28d172b00132
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631540233;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=A+FreUrlW+pyuDniqI775xFPQ9xesUP7rjMDdEVgdEs=;
  b=I82pZCwuRtkxpilMqbLYy/V6BRaUPNvFVU4NjMXiVug7CGjVhj0A4qoC
   etZjrRzqSp3lgfN4YMa1CLPH6rLs08XJ6yIdRZqymc0+RU6bHAI/G/rvQ
   FD32VeHAQWBGDlg9EEpYQBHLaGw3DxlBS+4HIkTxqpbp2ehaPscjRUbnI
   U=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: KcVVgRw+8sBPkarTPr/I0O2TS92Kyj4JrknxGUSma8H0kUdWMz/eLg5Icnm/a6puOwFUkhoD/8
 DXwS5DFlFhPkAJbcZ6zwUUxa3S8G7k92uOcGWhV66VB/6SJTHzdN4eVKzLLDbAv4b23u7oBC99
 gJJikBt9hQmFqxiwnI+NLWj0RycoCqjgcKDdlv+ViR0Aa5w2+79xk95zOOhs1ofYExjXLuQfLL
 s3P0gwG8bA8+VqcJacB+AV3O8d2j1j/qxl0JX/EfkaI7eP13DOKmRky0ZA63XuXWJdREA0oZbX
 q+qiyair41kUf/1nhDtlRrh+
X-SBRS: 5.1
X-MesageID: 52599778
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Rx5M3qocUl//bMAVBQCs1HAJX5BeBmIGYhIvgKrLsJaIsI4StFCzt
 garIBmFOP7fM2v0KNh3b4yx8EkAvcKAytFqQANqryA3En5D85uZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0IHiW1nlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnZj3RFoEYpLQpP1HCRUGSHw5FJ0W9KCSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFJkYtXx6iynQEN4tQIzZQrWM7thdtNs1rp0UQKePO
 JVJAdZpRDfjbzlBGlY8M5gvmOGMvljBVzZfrF3A8MLb5ECMlVcsgdABKuH9ZdiiVchT2EGCq
 Qru72n/Rx0XKtGb4T6E6W63wP/CmzvhX4AfH6H+8eRl6HWRzGEODBwdVXOgvOK0zEW5Xrpix
 1c8o3R06/JorQryE4e7D0bQTGO4UgA0X51dTsBm1x2x0/CO71jCWy8tfm5Nd4lz3CMpfgDGx
 mNljvuwW2c26ubIGC7CnluHhWjtYnlOdAfucQdBFFFcsoe5+OnfmzqSFo4LLUKjsjHi9dgcK
 RixpS4ijv04iccR3s1XFniW3mrx+vAlouMzjzg7v15JDCsiP+ZJhKTysDA3CMqsy67DEjG8U
 IAswZT20Qz3JcjleNaxrAAxIV1Uz6zdbG20baFT82kJqG32pi/LkXF4yzBiPkZ5Wvs5lcvSS
 BaL42t5vcYLVFPzNPMfS9/hWqwCkPm7ffy4B6+8Uza7SsUoHONx1Ho1PhD4MqGEuBVErJzTz
 r/CIZj1UidLVv09pNd0Ls9EuYIWKukF7Tq7bbjwzgi90KrYY3iQSLwfN0CJYPx/56SByDg5O
 f4FXydT4xkAAuD4fAfN9osfcQIDIXQhXMikoM1LbO+TZAFhHTh5WfPWxLogfa1jnrhUybiUr
 i3sBBcAxQqtn2DDJCWLdmtnNOHlU6FgoC9pJicrJ1uphSQuON798KcFepIrVrA77+g/n+VsR
 vwIdpzYUPRCQzjK4RoHapz5oNAwfRinn1vWbSGkfCI+b9hrQAmQoo3oeQ7m9S8vCCurtJRh/
 +38h12DGZdaHlZsFsfbbv6r3midh3lFlbIgRVbML/lSZF7orNpgJRvug6JlOMoLMxjCmGeXj
 l7EHRcCqODRiIYp692V17ucpoKkHuYiTEpXG27XseS/OSXApzfxxIZBVKCDfCzHVXOy86KnP
 L0Hw/b5OfwBvVBLr4sjTOo7kfNgv4Pi9+1A0wBpPHTXdFD6WLpvL06P0dRLqqAQlKRSvhG7W
 x7X99RXUVlT1BgJzLLFyNIZU9m+
IronPort-HdrOrdr: A9a23:gkVGz63bTZplzaRg9HY27gqjBIgkLtp133Aq2lEZdPUzSL3/qy
 nOpoV96faQsl0ssR4b9exoVJPufZq+z/5ICOsqU4tKNTOO0AHEEGgI1+rfKlPbakjD398Y+a
 B8c7VvTP3cZGIK6foSOTPIcOrIFuP3kpyVuQ==
X-IronPort-AV: E=Sophos;i="5.85,290,1624334400"; 
   d="scan'208";a="52599778"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VF5oiJ6R5sijX4YDGm28IYTBz83U/nNUL8BVN8FBVUtQtBokBuDUfzMigWK+6/Pm4YrCB2VSfrWuGc1zGBrJyGkOeVqVO7orj8ZcqQjj4g1lsB1YJAh8UuIaCxpUC34P3PYPFu+25bhs+68M+Ouu2IZ08f/URtRjLWwq2uZhcxDTb74JBiQC3L3vQbUrZT+llWw0sgBr9AXS7k4NN+aIcl3dLgiuEzx4XxrfEcXfBaPKFnaTzw3rKQLdjf8c/eZFf6PdEr59SN/XPgZ56SzH7EU8aH5tfvJ7PMaLrI9gJNNfrLQu0w58r9UrsMY4mKP06hxT95wvTfgNlxUR9JD/uQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LZ2FW8Iaupoo8E88XIF+kizmAFZvLvu+996YnALcNfw=;
 b=GTgYQJsRmxd2LQRglQM6DSvDDTLAD06CJaCFMdRIJnXHK5sDNErtbtfPSwxPocGlG4AzfmhtQODKpe9V68OBlr0A4sjYKJgrhdEBdBTXTP9m6bn4DzaqBmlekGv2EWn5jBLhgDUi8kmd7sNd3QifvPoSbDlAq4+s3ke3PTYWDl9bfgSpWYgAijfu8co07Gv5KKE+sX7kU/bv4bD36p3HbzyIC6K9teCPAKXk/4TzBZLn/kaplOrbNDM3Zm96NDa8x2cDF3yKGfo3r5OFOqp6VKmeDpHzOcSduLyqokJmw/cS9txCzXQI9XfQbvIuulClxoqdzkfiBS3V1XPANh2nnw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LZ2FW8Iaupoo8E88XIF+kizmAFZvLvu+996YnALcNfw=;
 b=sCJZJJohIyM0DT1Q7cZ1iPoKeXSEBtvBl3F2O9lCPjPrurnlmZp4tnE3CG2c+U+eSn01zEAE1xO2bDJtkY7V0pQlxUdH4qap9GfPbrh+4p25mvAPBZJYMUZS3S0xZyZnEPyEuHg/Pt1IZL97THAqtyR6jycJdNdasA9NeH0TLGQ=
Date: Mon, 13 Sep 2021 15:37:04 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
Message-ID: <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
X-ClientProxiedBy: LO4P123CA0343.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18d::6) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9998ee2c-384e-4e45-d7a8-08d976bb9640
X-MS-TrafficTypeDiagnostic: DM6PR03MB3579:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB3579E93BEB6A18A84443AEAD8FD99@DM6PR03MB3579.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: YRKl6L/JK+nDIdeEwC4kwTMZdS+O7nuCdOShdGqeoErF2CzoeRDJt4RvBC8g+Txv4EoYTqBxb6wWEH3IEhDQo904XH4oReXUSKZmqTjZRVbM7GSUo3+wcHiJ8ud6zc9hVAa9H8X2U6MKRvu6gedUMncqPLlfArjgwxCSsx3ZW2mjEOjKvdO9Az0HlLwoquJ1Of7XtX3vM+Xp+LRIEtwZtw0sJmaBqAZWwEfCFHNpgrkw03H5htTg5sHvzY1DvfYT4u7UpXBhthUgNifNh0vjNwa/9vKs3YNmaRXtlawU9x3xlbeZcl/aFMpnUoslCp2YdnaqbAQIqLsdofsSpHbSZvMWhqqprtMnfC6zQAqineU40O1D3Mq8NTY16XcYVmuZlhDV/qbBrh4pP5PbBijSDwF2kejfxh5lVgO6ccUwkpYxig2ZBqmstYhvkNrFShY+kPWnQocVUpBKxoXDzsvkJVmDK3lRkHGL1FiQ0+tPuZ37a+rIr2AdEKBVrUs97ady7p6X5X9rhmTkWJIyt3rlpP8Jh+CxS6pMdmANRDa6cXZWDeUDW5dCdY2VKR30LyXYYFQ7LhuWgf/e9ZxA+1EHlwEp3lWpgKQa0g0eXK0hATJmrvHD0e+68tcKHNufAz6NZROuCawVirMvRFYTSJHMscNqsUYyWiNjXaOh+wDKfpkGhqO7R8qY1LgdkU+c0xCIv7n1F5DBugP8WMv2+htWa0BuMrg5RRZO3K54TtOp7ek=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(366004)(136003)(346002)(396003)(376002)(2906002)(38100700002)(4326008)(66946007)(6916009)(54906003)(66556008)(956004)(66476007)(186003)(8936002)(8676002)(966005)(6496006)(478600001)(6486002)(6666004)(45080400002)(5660300002)(85182001)(9686003)(86362001)(316002)(26005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UC9vdTBoYWJKbm9iSmVQQlB1T3I4aDZpSGt3SGRzRzdSQ2RmcEM2TThtU3or?=
 =?utf-8?B?SU5kSXZpalNQazFqOERPYzArSS9UaG1TN01XbG4wSUY4Q051Q3ZXT2FBRDNa?=
 =?utf-8?B?VnovOEZuWjBxaFVKdWJDVjZEUytOU3BaLy9YbHh0ZjVVU0RackNyeFhCNEcr?=
 =?utf-8?B?Y3UzdVpZSmNzS0J4Q21lVUdXNUEzWENKNlNQdmd3NjdybHNRTW5NL0cyckxj?=
 =?utf-8?B?Mlk4ZEZER0R1eHExS3hXRjlEajl0S3pTRTlLRU5GM2JlOVFUQWlLbVF1M09K?=
 =?utf-8?B?UG15Z3hHQTU1b3U0OERVQjNsemwzYjdTZWZaU0xkd203dmUrSUJyWWMxa04y?=
 =?utf-8?B?RFdBc2paaVJONk13UkU2djFMempRcFA1RE5nVkVVUGhUdE4wZ3VONkcxOHlO?=
 =?utf-8?B?YkNUYllrY3pMbFFRMmlvSDBBbGpQV3RLd3NJclV1ZjJiOVN0djdhaUFWc1JB?=
 =?utf-8?B?eThLTnpvajJweXA2dTFLNHNFZmZhb2JTSVNYZWtjY2g0Y2lraFBKM1FzS1Vu?=
 =?utf-8?B?MytJdzhWMitUbW02ZE1peXhlVENjclVvTnN1ejdzdEh3NlNDLzJPd3N5VGVK?=
 =?utf-8?B?clZ4eEwyS1NMR3Juc1JYOTBjUUdLczZhS1FCcmo0N2xlQ3puM2JPYXZQR0Y4?=
 =?utf-8?B?a2JRVlpKbVpWVGQxT0FJcldXSTBTYkdjQ1UrMUVoaXJ3MkRJdkl5ODBqbzVa?=
 =?utf-8?B?ZC8xZlQyOW9wZytxYllZUDl5M2EvWG5rb3ZKRDJobjFVOG5GdGJRWlBCbmZZ?=
 =?utf-8?B?SDFuZE84cGN3dTJ5QmFHM25XSmpVaU0zeDRBQ2FsVHh2cDVGdTZLZ25sQSs3?=
 =?utf-8?B?MHV2V2JaMkZXbm10ZG8zTXp2RjFKVG5IMHFhd2lMWjZBK2pGUFNJNzY4dVUr?=
 =?utf-8?B?NVBmN29aT3VqOGlQOU9XT05RUzl5MTVNUmNlZ20vcjNWbFlnSDZ5Z2t2QXgz?=
 =?utf-8?B?UXJxRGwwcW9nOGVsSU5wYXNNNXZQT2ZCZjhCejhXeFcvRGUwZFovTWtBL0tD?=
 =?utf-8?B?ODJrdk0rTE1tTVhjNE5GN2tIa0NHcUIzNFRITFJReUhyN0ZQWDdsWGFOMFNt?=
 =?utf-8?B?NC9YQXBKbWRmUlFWREpvVVlpcjAxMSt1TXJVYW4vaWhnMlIxYkpYQ3MrTFVC?=
 =?utf-8?B?N0FEWXFuZWt4LzlvcHdRNWNpaU1xaUNBY2NmQnVhMGxYOEpYZWlIWWJRK092?=
 =?utf-8?B?WmtyWTRNd1dpWmVuaTNIS2x1aGdaZFdua296WkZEKytuTXpyczdaSVRNTEI2?=
 =?utf-8?B?L0k3NE0rdXo1UTVTZ0FUQ3c1a2dMUS96N1NLdmRVb3diS2Vpa0JuZ3gzQkNr?=
 =?utf-8?B?TXRHWEV2NWZDTkdXczl6Qkk4U1Jodko5dmdsNzlWYWlaRnNVbnZ5NlJ1RVZv?=
 =?utf-8?B?Rm1LZHpma29uYk05bWhSSkcrSmNLRXBpWDd5alYzcjNzdlhoNmZUWnpFZHZy?=
 =?utf-8?B?aWpkOWRxeHhPcVVSWHg5Vkx1MFRNUklxRTdpMHdhUzNPSE9WbmM5VGVDODJK?=
 =?utf-8?B?b3ovVk5VdEtpTlRIL1hEQU9FZDJXK1JuUEJzVm1OQURxVVFaeEhMejhVc0ZH?=
 =?utf-8?B?aWx1ZzhnR3Q0TEJkTm1ZelRoNHNTYkNCamMxbHlGZ29GL3orZURLOXBydlVO?=
 =?utf-8?B?bkJtTGgwSk51NUVoaDJVSVF1L0Q2NzU4b0tpbEFkMXRWcjVuUGEveUN1OWov?=
 =?utf-8?B?Qythbkg0Y0tSOHcrWWNlTlBHVHlHVzlRUkZOMXd0MUs1dUtmSTl4NmdkTzF6?=
 =?utf-8?Q?UHh7aakqcjndjAcyiwRTwc4BLvzkaEcdX/1ALv0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9998ee2c-384e-4e45-d7a8-08d976bb9640
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:37:10.2654
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: r1Un0Fz0+10Kkc8jHb3X5RrLZ6K6E4iw9spRFbmKtDUpPb1iyzVjwlOiQiIMkvjZTk15VnX3eSUb2Whe+0Z1rw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB3579
X-OriginatorOrg: citrix.com

On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
> While LLVM's lld is supposed to be a drop-in replacement for GNU ld [1],
> it appears to not understand quoted section names as operands to e.g.
> ADDR(). Therefore the original workaround broke the build in
> environments where ld is actually LLVM's, like on FreeBSD.
> 
> Fixes: 58ad654ebce7 ("x86: work around build issue with GNU ld 2.37")
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flld.llvm.org%2F&amp;data=04%7C01%7Croger.pau%40citrix.com%7C07abc3240fc14a3f095408d9739f3eb3%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637667950073151096%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa0BBWP2cWOhWsB3b4FQIZ1J3KvfWP%2BnINiqyXdChD0%3D&amp;reserved=0
> ---
> I haven't been able to find an environment where I could actually try
> with lld (ld.lld); all testing was with GNU ld (ld.bfd).

Thanks for fixing this. I've been able to test with LLVM ld and the
workaround is fine.

> --- a/.gitignore
> +++ b/.gitignore
> @@ -306,6 +306,7 @@
>  xen/.config.old
>  xen/.xen.elf32
>  xen/System.map
> +xen/arch/x86/.check.*
>  xen/arch/x86/asm-macros.i
>  xen/arch/x86/boot/mkelf32
>  xen/arch/x86/boot/cmdline.S
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
>  
>  ifneq ($(build_id_linker),)
>  notes_phdrs = --notes
> +# Determine whether to engage a workaround for GNU ld 2.37.
> +build-id-ld-good = $(shell echo 'void test(void) {}' \
> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - 2>.check.err \
> +                           && $(LD) -T check.lds -o .check.elf .check.o 2>>.check.err \
> +                           && echo y)

Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
then use is here?

We already have compiler and assembler checks in x86/Kconfig, so it
would seem more natural to place it there.

>  else
>  ifeq ($(CONFIG_PVH_GUEST),y)
>  notes_phdrs = --notes
>  endif
> +build-id-ld-good := y
>  endif

I also wonder whether we need to make the quoting tied to the usage of
build-id. I guess we don't add sections with dashes and instead
use underscores, but it might be prudent to always quote to be on the
safe side if dashes are not supported.

>  
>  ifdef CONFIG_LIVEPATCH
> @@ -291,6 +297,10 @@ $(BASEDIR)/include/asm-x86/asm-macros.h:
>  	$(call move-if-changed,$@.new,$@)
>  
>  efi.lds: AFLAGS-y += -DEFI
> +xen.lds: Makefile
> +ifneq ($(build-id-ld-good),y)
> +xen.lds: CFLAGS-y += -DQUOTE_SECTION_NAMES
> +endif
>  xen.lds efi.lds: xen.lds.S
>  	$(CPP) -P $(call cpp_flags,$(a_flags)) -MQ $@ -o $@ $<
>  
> @@ -302,7 +312,7 @@ hweight.o: CFLAGS-y += $(foreach reg,cx
>  
>  .PHONY: clean
>  clean::
> -	rm -f *.lds *.new boot/*.o boot/*~ boot/core boot/mkelf32
> +	rm -f ???.lds *.new .check.* boot/*.o boot/*~ boot/core boot/mkelf32
>  	rm -f asm-macros.i $(BASEDIR)/include/asm-x86/asm-macros.*
>  	rm -f $(BASEDIR)/.xen-syms.[0-9]* boot/.*.d $(BASEDIR)/.xen.elf32
>  	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
> --- /dev/null
> +++ b/xen/arch/x86/check.lds

I would maybe name this check-dash.lds, in case we need to add more ld
build tests.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 13:53:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 13:53:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185686.334368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmOp-0001Cu-QV; Mon, 13 Sep 2021 13:53:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185686.334368; Mon, 13 Sep 2021 13:53:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmOp-0001Cn-MK; Mon, 13 Sep 2021 13:53:31 +0000
Received: by outflank-mailman (input) for mailman id 185686;
 Mon, 13 Sep 2021 13:53:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=01x7=OD=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mPmOo-0001Ch-7B
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:53:30 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e1b934a3-f2f0-406a-87f8-6427bc7c4801;
 Mon, 13 Sep 2021 13:53:29 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id q26so14836354wrc.7
 for <xen-devel@lists.xenproject.org>; Mon, 13 Sep 2021 06:53:29 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id q7sm1838350wru.56.2021.09.13.06.53.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 13 Sep 2021 06:53:27 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 9BC831FF96;
 Mon, 13 Sep 2021 14:53:26 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1b934a3-f2f0-406a-87f8-6427bc7c4801
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=user-agent:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=FrxeFVXB0iHRu6ysUDY2Cx68qEn3yyoUjhHHUgs2xrc=;
        b=QWAMEFwudHmTzPrqwwC9aSNmIov2MWk2aff+MlCmsbNpUVWWkulcxXtU051WdUC+aN
         x8FxvFnPHX4wZcpaYlX3UYDNck8UwVAEs1wD3MyD7yQOLNSrB+qQ9Fh67EDr2hvWZ00V
         9u5wW9vowdmb9HIKRSiVKEitDjf1samltdI0PdtQbN32AOR8bet4eJE/6Ww7sOY2VdDq
         lxfUO+Wx3t+qITLGQQE21CuTRxWVNu0le9KXoBF0IYWEQZlnr8talGomecr2TkK6ddGE
         BQ1xlIqH17JcZ84CGPR28Eagj/Mna4851yiwM1OlxMTC5adWc8yqDbnSO0qsgDHXViQn
         FbOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:user-agent:from:to:cc:subject:date:message-id
         :mime-version:content-transfer-encoding;
        bh=FrxeFVXB0iHRu6ysUDY2Cx68qEn3yyoUjhHHUgs2xrc=;
        b=4Kyo1sapC1r1G8uRe+YK7xC/Er8A9b0M4UOmURHnfd9414EGuPxjv/3zVK7h6mYOXX
         0MHlkrmVZyYJHRPiEh/3GySkcFLMR5X9KKf9pgZDvoCSD7ECNKCNXuDL1+SgYNfH1W1Q
         GIbaiqpgA4wAE1hPKqgTrX/oyAKmbsNE2GAV+ic1FhLnQ4p2UhoqJ/1AyPj7GJrcTIJs
         O4X+WsbfWl1kWsLJx5lgeiCU6yJEGr64Ww0YHyW4v+Cm50POshw4i36x51HmAEfw2Q/9
         Gvh1t0XkzjQXc4yNq4MQ75AbP9JVOJ38qwdn0mu94EBMdl7fyB6SKqhiE8+PkEhKMIea
         QhlA==
X-Gm-Message-State: AOAM532foLvFMAa/lI+SLOwYOsZ1RJPAPlbFT0a+ZvnOwd9CZBmGbSVc
	S71VfPNw83HNo92Qx9acFfc2zw==
X-Google-Smtp-Source: ABdhPJyY9fKnZ104ODYyH3fJsphgPUR61GnQVgVDFVl0PD346R5DURujcNuLktUjJWRtn5MDYG6JnQ==
X-Received: by 2002:a5d:5408:: with SMTP id g8mr12877435wrv.34.1631541208154;
        Mon, 13 Sep 2021 06:53:28 -0700 (PDT)
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Mathieu Poirier <mathieu.poirier@linaro.org>, Viresh Kumar
 <viresh.kumar@linaro.org>, Wei Liu <wl@xen.org>
Cc: "rust-vmm@lists.opendev.org" <rust-vmm@lists.opendev.org>,
 xen-devel@lists.xenproject.org, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
 <olekstysh@gmail.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Is it time to start implementing Xen bindings for rust-vmm?
Date: Mon, 13 Sep 2021 13:44:55 +0100
Message-ID: <87lf40vay1.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi,

As we consider the next cycle for Project Stratos I would like to make
some more progress on hypervisor agnosticism for our virtio backends.
While we have implemented a number of virtio vhost-user backends using C
we've rapidly switched to using rust-vmm based ones for virtio-i2c,
virtio-rng and virtio-gpio. Given the interest in Rust for implementing
backends does it make sense to do some enabling work in rust-vmm to
support Xen?

There are two chunks of work I can think of:

  1. Enough of libxl/hypervisor interface to implement an IOREQ end point.

  This would require supporting enough of the hypervisor interface to
  support the implementation of an IOREQ server. We would also need to
  think about how we would map the IOREQ view of the world into the
  existing vhost-user interface so we can re-use the current vhost-user
  backends code base. The two approaches I can think of are:

    a) implement a vhost-user master that speaks IOREQ to the hypervisor
    and vhost-user to the vhost-user slave. In this case the bridge
    would be standing in for something like QEMU.

    b) implement some variants of the vhost-user slave traits that can
    talk directly to the hypervisor to get/send the equivalent
    kick/notify events. I don't know if this might be too complex as the
    impedance matching between the two interfaces might be too great.

  This assumes most of the setup is done by the existing toolstack, so
  the existing libxl tools are used to create, connect and configure the
  domains before the backend is launched.

which leads to:

  2. The rest of the libxl/hypervisor interface.

  This would be the rest of the interface to allow rust-vmm tools to be
  written that could create, configure and manage Xen domains with pure
  rust tools. My main concern about this is how rust-vmm's current model
  (which is very much KVM influenced) will be able to handle the
  differences for a type-1 hypervisor. Wei's pointed me to the Linux
  support that was added to expose a Hyper-V control interface via the
  Linux kernel. While I can see support has been merged on other rust
  based projects I think the rust-vmm crate is still outstanding:

    https://github.com/rust-vmm/community/issues/50

  and I guess this would need revisiting for Xen to see if the proposed
  abstraction would scale across other hypervisors.

Finally there is the question of how/if any of this would relate to the
concept of bare-metal rust backends? We've talked about bare metal
backends before but I wonder if the programming model for them is going
to be outside the scope of rust-vmm? Would be program just be hardwired
to IRQs and be presented a doorbell port to kick or would we want to
have at least some of the higher level rust-vmm abstractions for dealing
with navigating the virtqueues and responding and filling in data?

Thoughts?

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 13:57:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 13:57:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185694.334378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmT1-00023X-FP; Mon, 13 Sep 2021 13:57:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185694.334378; Mon, 13 Sep 2021 13:57:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmT1-00023Q-CW; Mon, 13 Sep 2021 13:57:51 +0000
Received: by outflank-mailman (input) for mailman id 185694;
 Mon, 13 Sep 2021 13:57:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmT0-00023K-LZ
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:57:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmT0-0006Cc-EY
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:57:50 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmT0-000841-Cm
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:57:50 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mPmSw-0002q9-Rv; Mon, 13 Sep 2021 14:57:46 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=B+L32RxaMM3skGwJccCYbHax/dU0aN22xFj4U3H5no0=; b=OJFglhNQ/zc4TC81xbBGB0MgYS
	0CP0iLiuRCIIoYFvG9G6JXT0RxYRJxfnKQQn5gkTDv7YoALrJ0n0fwIxMotzx2o6OPA06Dsc02VvQ
	DlU6FaHZ3XiXBOmb4Xm40UwoI9JIpqhP5YdGoeuowLSLfSGTyrWOL/eUq4unO9bqbv0c=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24895.22746.514043.814422@mariner.uk.xensource.com>
Date: Mon, 13 Sep 2021 14:57:46 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    julien@xen.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 1/2] tools/xenstore: set oom score for xenstore daemon
 on Linux
In-Reply-To: <1f079107-4685-aa68-bed1-867b23e1fd32@suse.com>
References: <20210730122643.2043-1-jgross@suse.com>
	<20210730122643.2043-2-jgross@suse.com>
	<24835.65010.870849.479582@mariner.uk.xensource.com>
	<1f079107-4685-aa68-bed1-867b23e1fd32@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v3 1/2] tools/xenstore: set oom score for xenstore daemon on Linux"):
> Correct. Otherwise the user would need to know about the oom_score_adj
> ABI.

I think they might know about it.  In particular, sysadmins might well
be used to configuring this directly (for other daemons).

> On 30.07.21 15:26, Ian Jackson wrote:
> > But maybe we could add something to the doc comment ?
> > 
> > Eg
> >    # (The specified value is multiplied by -10 and echoed into
> >    # /proc/PID/oom_score_adj.)
> > 
> > ?
> 
> Why? This is an internal implementation detail. I don't see why the
> user needs to know how this is accomplished. What is unclear with the
> XENSTORED_OOM_MEM_THRESHOLD semantics as described?

The underlying interface is both also-publicly-exposed, and has a
nonobvious mapping.  Speaking as a sometime sysadmin, I would often
appreciate something like this.

> There is no other parameter with an explanation how it's semantics are
> being accomplished.

I don't think the other parameters are as strange as this.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 13:59:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 13:59:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185700.334390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmUP-0002fx-Qp; Mon, 13 Sep 2021 13:59:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185700.334390; Mon, 13 Sep 2021 13:59:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmUP-0002fq-NR; Mon, 13 Sep 2021 13:59:17 +0000
Received: by outflank-mailman (input) for mailman id 185700;
 Mon, 13 Sep 2021 13:59:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmUO-0002fi-M3
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:59:16 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmUO-0006Fk-LB
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:59:16 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mPmUO-0008Bg-KE
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 13:59:16 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mPmUG-0002qd-RU; Mon, 13 Sep 2021 14:59:08 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=K/4rukjrq3TiHD+P6XMnltI0voVS9nKVYs0qzADhDJU=; b=DjdReweRMLEE2zgaRULMcxzrF1
	QWBQbG42GEiNSrYTKrD0MHH3sXi4QB20BBYW4Lb6/SJ3dqDQcxWc9sGGqHiYb8oI3jq+JrPDxDdC+
	QZQUylNWvMIZG/zrANMUP6s8iZuLsNnZbTYvXbjHbRuOmap0mtxpEcPZTtcLJOvxv2g8=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24895.22828.592096.976681@mariner.uk.xensource.com>
Date: Mon, 13 Sep 2021 14:59:08 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Ian Jackson <iwj@xenproject.org>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Julien Grall <julien@xen.org>,
    xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
In-Reply-To: <ea834f80-45ce-821a-bb2b-4a56dddff156@suse.com>
References: <20210730122643.2043-1-jgross@suse.com>
	<20210730122643.2043-3-jgross@suse.com>
	<24836.28.655841.510063@mariner.uk.xensource.com>
	<81a39ab5-5588-65ad-f1a1-7bfe0379b394@xen.org>
	<6ced9858-d425-887c-0045-1aad8521b826@suse.com>
	<c0a6bb5a-3bd8-b1cf-9973-7a95b347befc@xen.org>
	<9352f82a-1b43-4bf6-8b0a-5916627b7537@citrix.com>
	<24878.15168.981558.748531@mariner.uk.xensource.com>
	<ea834f80-45ce-821a-bb2b-4a56dddff156@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v3 2/2] tools/xenstore: set open file descriptor limit for xenstored"):
> On 31.08.21 16:22, Ian Jackson wrote:
> > ISTM that the following scheme is in the intersection of everyone's
> > requirements:
> > 
> >   * The limit will be adjusted/imposed in the startup script.
> >   * An /etc/{default,sysconfig} parameter will be provided to
> >     adjust the setting.
> >   * The default should be `unlimtied` since we cannot calculate
> >     a safe upper bound for all configurations.
> >   * Systems like Citrix Hypervisor (XenServer) which can calculate
> >     a safe upper bound can do so, and adjust the default, enabling
> >     them to spot fd leaks.
> 
> Makes sense for me.
> 
> So this would mean:
> - the sysconfig parameter will no longer be "number of domains", but the
>    fd limit of the Xenstore daemon
> - default should be "unlimited"
> - the comment should mention the current number of fds needed per domain
>    (5 for HVM, 2 for PV/PVH) plus some headroom in dom0 (without any
>    guest running on my test system 21 fds are active, so I guess a value
>    of 50 seems appropriate)
> 
> Is this okay?

Sounds good to me, thanks.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:05:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:05:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185707.334400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmaK-00049g-F9; Mon, 13 Sep 2021 14:05:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185707.334400; Mon, 13 Sep 2021 14:05:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmaK-00049Z-Bv; Mon, 13 Sep 2021 14:05:24 +0000
Received: by outflank-mailman (input) for mailman id 185707;
 Mon, 13 Sep 2021 14:05:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPmaI-00049T-M0
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:05:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6e0bf23b-5308-4252-8ba7-2126b1970795;
 Mon, 13 Sep 2021 14:05:21 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-OhnX9NYZPhu7dVWADy-BaA-1; Mon, 13 Sep 2021 16:05:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2702.eurprd04.prod.outlook.com (2603:10a6:800:b4::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 14:05:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 14:05:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0018.eurprd09.prod.outlook.com (2603:10a6:102:b7::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.15 via Frontend Transport; Mon, 13 Sep 2021 14:05:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e0bf23b-5308-4252-8ba7-2126b1970795
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631541920;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CQviCSMPM0iLMcGOFi/gWQRrDdQP8OM6s1oLKQzotR0=;
	b=Xkbdflcf7hiEYMhwxC17cmnFTEWs8jRFX5TFjeHv4y3WlZVipNaaz2K7sBejd3m0noeuis
	KRGWO/k5iznaHLdCKiKn0IQcJ5dyT5bG/baLdYfmo1nTF7vGHkc/ay/6RUA0WFRr3YRnB7
	KJVsNXm6W+csjdu95LlO8VsP2E4efPE=
X-MC-Unique: OhnX9NYZPhu7dVWADy-BaA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TugzH6qATNlpLlNU4WLuiHgTup/rxjpFpdxlO857h+z/L/eLn7JIzrCWWI/FfJndNoeim90X1mHF5iHvmRmVd6Mw+axlW0aQ6aQDH3LPTQDFlSAEFAwiuBn7EsFi5TSAHs9tTPdSC2kOQ8IV5Mx3AMpuoyFQXRy8hfLw/wwzDmone1sMVehZrk8GzkbtHQSiQHX30qVHUz/TqJl1t+BKFERp9GJR6M1U5HSpWEZYH6j8nsEIuXceSqnbnfCN2xVbTgd+o54vZH5PT5V32O9kQC01eByQoXE6g9FtmKIteQT0bVB0x5nWqitkqW3NyTPPkBTnnK68c2jE61i+pOEECw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NqcZ/JCvhiTM7eTgRBPnzioJbrgL+70KR0DQ93eXPLc=;
 b=nTQvQkDCT6BLQ1QZLsWu01fdqg1ycmI9vaCKIWvkS/WhRAGqCLJCb+5FubZi9sy45oGJ1a20+ORFEB8meFvYrGxmqsvO54HyLRZMchJIoYUlTVXbo5oXny8lDd4Wg+VgZJMU2Ax/52lqX0xxrsJHc6WpLExTMSmJcPZHE6DhGZYKl0pXIQUL7idC5WFnHVzWDJY+lRYpUD0HoicvPaD9MPEz5mk06OvOjGxuA3du5/FWC3xDsd4YUIDMVgE/EDZ67l010bTO8E4f+KBMFmNw/fPeSHK4BsHjkYaa2uDORcCzrWZqXHt7revLUPb7rETn/1v5UHEoOb/WNQxmZiII8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7c97de87-9687-a088-406b-33faf037244a@suse.com>
Date: Mon, 13 Sep 2021 16:05:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3PR09CA0018.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 522ff334-8dcc-44c3-1571-08d976bf8392
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2702:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB270287F7CA9A182CA3CDF9B9B3D99@VI1PR0402MB2702.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	re/Lss0HkttCEje1vrnO29VicXeae0IilSLD4ayIm4oYyXzcKAqZbB5vsc8CxyNnfQ/fUbBspZNlppWI7/Qy4DFn4Hs0m5haA75qGTyCbB9NGZ3NmCDVycLPiDZLxLmMFnLP9OcffBdxx/jblUgkYQDM7WKrnuPkYEGvcvklMoBC7YRh8VUkICeqQZzIiTW6CJdLbkBjC5pAHQLlHsIBu62a7tmkK+64m3H3mw9qo5Uhbt/R3fH+jko5oE2Cc/4Gc3VFqSS1h0pO1JsQvalkktrD6NbPrd0HoiyVziX4+8BiCAtEITVvuy+JL9+TVHk/QE6xf5fQn2tavvIgqnfYmUVrzzvVRqSqix589CCYzN/q/A5py/56ExV1HfbeCwZJZ9dRmxNlOytJ8PfVQ3uQ5S1WEaqkphQxGupyfZbBcSeoyIuso8WF/Lo82WzD/nUtq1FXY553E+CDsgDpxAk5B4IeUpqY949pYLA25iICMhM0ADp/XGiassRTJ9Io8YKXHuorgtPa2+7o/NoSPMmEDjPtW1RuI7wvXomltZ3+K4FXOgNLSDHaokx9xUVGCXl3XYHkpDAdVlFNlY+L7M62cKxhUgi//vXdi9qTyIqk5EUUhKxmygjesGDgDL7u3i5VnkwNap86E4XGLp7ir2y8G4Ag/2vh6WtT9Fq4blMRdhWUIhk7mZf0thZd0c9Woa0Qyl511YPUmFopc24hxIEk7cGnniu7/6ph/95hUhiEjJI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(136003)(396003)(39860400002)(376002)(53546011)(8936002)(66476007)(31686004)(4326008)(66946007)(36756003)(186003)(2906002)(8676002)(478600001)(2616005)(956004)(66556008)(54906003)(6486002)(86362001)(83380400001)(26005)(5660300002)(6916009)(38100700002)(316002)(31696002)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Bj4cm0VHXETEUb16+tgJvibTWvKdW8b9Uq9T0OuPE2HqjEn8j8klGHLtb00h?=
 =?us-ascii?Q?vRKxJ+7D9Y2cbGMaEOhPJtY9NoQpnbBSe+g7B+idvE/zD3biUiteerqxG2F8?=
 =?us-ascii?Q?jzTiogt+Igk1dVrzfXUqUMrdowmcQUoXZS8fXlGsh3W0r2C/SLleTs0jSHbJ?=
 =?us-ascii?Q?7OkGgIxiqm24he9LzCa7+HoOIlIWSqOzGcqMkwMKW5vkIpFk/g21/gwkkmWV?=
 =?us-ascii?Q?qDDNDbg4hh2y/sWbQBzobwe9V9z9BEeSw6QHqXukK4oTn5OQfB+X1ssuMBAu?=
 =?us-ascii?Q?xGb7G7m6SHGL1Q6x7ljRU6qXaupA5CFq4wKYUAVHVTViZpNU3vJBExfDHHXR?=
 =?us-ascii?Q?Y4zG4M/GZVw1QEJyWU2cqXO8DZ6CL6g85At6diu5OFlADVxryhEiaRN3bPwa?=
 =?us-ascii?Q?Ag7J2ZlSg13yBdI74+lsyAW8lFpJXUxzEf5DSkCYRPUgsrAZdcL99lOyIw+m?=
 =?us-ascii?Q?ruo3M9ZC+5eawxnQ35SVXe1+kjQFac7kumCYyhqeEYEx/nbPXRKvTFrAkucN?=
 =?us-ascii?Q?aBOMfgC7KxZf5X4CevW5eV/VWLQYdVEdJsnuLf0iVzDu6cfM7vr1qaYn4pXb?=
 =?us-ascii?Q?b/ANT8kx28mjg9EIV7/wJDo8Ot9B0Eh0C8cazwafAH6tygSw41C1gcxJ7fWh?=
 =?us-ascii?Q?Z39yvcgpTVIAEsyioy7rJcjU8Yl5/ncbH+F/DJBZ145bG1imcLb3XQAXNnaW?=
 =?us-ascii?Q?XQXQlwKjJWbkjsDjIwsBDOpYf/gczDPdQ/1dJillov68JWXYSt5xQSiMnp5P?=
 =?us-ascii?Q?JKrowWoH7sBOX3V1u0dasPjIGLEK2Qk8byOsax5bnc6/jFPedZFC5LVdlJ/d?=
 =?us-ascii?Q?L/yFEizhpFIPb4tAo5ZrRP5iC60xpRnz3PYDzYS5AeGY+boKug7/OywxJMhO?=
 =?us-ascii?Q?u5+x3xbvp533PEnz04F0As3fJfXRqZtXVV0irvpJ/s31DKi/vTuRD4HTKzqa?=
 =?us-ascii?Q?DsE+y6z4x7NmGlCfOsnOQWu3zDUlz9XTxqQgVRTQVl8zeKmpwTawXMRuGpFt?=
 =?us-ascii?Q?1zAv60320KLNEiGl9GiOgXuPUUqZDFipaXPB/Sh7Eh2MTfThq1VkMzmn8AOn?=
 =?us-ascii?Q?2rXO6LBNG6WvTDOEzadex2PdShUfX+siG5ZbdQe3t6GNL4eG64KmGoJIQJyE?=
 =?us-ascii?Q?cIhWItXW57U2Q8h7Njzl0w1rEA2z7bEBPcRKV7mhicR3uUJ+2PILhIwjQ3r9?=
 =?us-ascii?Q?KzyjhkzXisVn2pk/kqC8cynwIg0HnscVKew25o5kP9KUfRHKw1Ep8ok+Z27J?=
 =?us-ascii?Q?J/lHQCO0LEHwe1s+d05kQlEDVeXDJFCxjNmzIMaUmXq1zMY4Vk/Ada53Kp8c?=
 =?us-ascii?Q?4zxfU6Kt2uV/1INr0aFWWjQW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 522ff334-8dcc-44c3-1571-08d976bf8392
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 14:05:16.9664
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8fo1tgVLscpmzJjTrV0MWikAhlpkICD0xiq2ojWhlCJHxYbAsB9u3sRq3Dqn9ggycLuc+A3MtcBRE7YFRV8hHA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2702

On 13.09.2021 15:37, Roger Pau Monn=C3=A9 wrote:
> On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
>> I haven't been able to find an environment where I could actually try
>> with lld (ld.lld); all testing was with GNU ld (ld.bfd).
>=20
> Thanks for fixing this. I've been able to test with LLVM ld and the
> workaround is fine.

Oh, good, thanks for trying this out.

>> --- a/xen/arch/x86/Makefile
>> +++ b/xen/arch/x86/Makefile
>> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=3D
>> =20
>>  ifneq ($(build_id_linker),)
>>  notes_phdrs =3D --notes
>> +# Determine whether to engage a workaround for GNU ld 2.37.
>> +build-id-ld-good =3D $(shell echo 'void test(void) {}' \
>> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - =
2>.check.err \
>> +                           && $(LD) -T check.lds -o .check.elf .check.o=
 2>>.check.err \
>> +                           && echo y)
>=20
> Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
> then use is here?
>=20
> We already have compiler and assembler checks in x86/Kconfig, so it
> would seem more natural to place it there.

The question of whether to record tool chain capabilities in .config
is still pending. I'm not convinced this is a good idea, Andrew keeps
shouting me out for that, and an actual discussion doesn't really
happen. Yet unlike back at the time when I first raised my concern,
Anthony meanwhile supports me in at least the question (to Andrew) of
when such a discussion would have happened: Neither of us is aware,
yet Andrew claims it did happen, but so far didn't point out where
one could read about what was discussed and decided there.

For the few uses we've accumulated I gave (if at all) an ack for
things happening under some sort of pressure, with the request that
aformentioned discussion would happen subsequently (and, depending on
outcome, these would be converted to another approach if need be). I
have meanwhile realized that it was a mistake to allow such things in
on this basis - the more of them we gain, the more I'm hearing "we've
already got some".

>>  else
>>  ifeq ($(CONFIG_PVH_GUEST),y)
>>  notes_phdrs =3D --notes
>>  endif
>> +build-id-ld-good :=3D y
>>  endif
>=20
> I also wonder whether we need to make the quoting tied to the usage of
> build-id. I guess we don't add sections with dashes and instead
> use underscores, but it might be prudent to always quote to be on the
> safe side if dashes are not supported.

If quoting was uniformly supported, I might have considered that. But
it not being uniformly supported is the reason for this change in the
first place. Hence I'd prefer to generalize this only if really needed.

>> --- /dev/null
>> +++ b/xen/arch/x86/check.lds
>=20
> I would maybe name this check-dash.lds, in case we need to add more ld
> build tests.

I sincerely hope it was a one-off that a binutils release got cut with
this sort of a supposedly prominent bug. Considering that the dash is
merely what we're after in this specific case, but breakage was wider
(presumably about any printable char that's not alnum or underscore),
I'd consider check-dash too specific a name. You only say "maybe"; if
you were sufficiently convinced, this is an adjustment I'd be willing
to make. Yet even better would be if I / we could just be done with
this.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:14:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:14:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185715.334411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmiy-0005iK-An; Mon, 13 Sep 2021 14:14:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185715.334411; Mon, 13 Sep 2021 14:14:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmiy-0005iD-7i; Mon, 13 Sep 2021 14:14:20 +0000
Received: by outflank-mailman (input) for mailman id 185715;
 Mon, 13 Sep 2021 14:14:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPmiw-0005i3-Jm; Mon, 13 Sep 2021 14:14:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPmiw-0006cC-8W; Mon, 13 Sep 2021 14:14:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPmiv-0005O1-Q5; Mon, 13 Sep 2021 14:14:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPmiv-0005Du-Pe; Mon, 13 Sep 2021 14:14:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SsH/7JDw+e09lAhLT9EMT6SEEH/qVsbc+rK7+r7oLbg=; b=tOdJEB/hIF1yJvNjwFdjUt1Xd9
	epyg0LDJKATUofoIIMWUm2TGSuBCLLqyeYeK6Q8l0cb59KS6G/Ag86rNFHv7kwYgM6V+05gwckmJ9
	oyzpKm76+kjE+V9LLIXPMRp8cmpE2+KR7/Q2KgafDmngpHNTXWrkxfI4YaIteJDKRteE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164963-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164963: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 14:14:17 +0000

flight 164963 xen-unstable real [real]
flight 164968 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164963/
http://logs.test-lab.xenproject.org/osstest/logs/164968/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    2 days
Testing same since   164951  2021-09-12 00:14:36 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:20:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:20:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185723.334426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmpD-0007Ia-5B; Mon, 13 Sep 2021 14:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185723.334426; Mon, 13 Sep 2021 14:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPmpD-0007IT-27; Mon, 13 Sep 2021 14:20:47 +0000
Received: by outflank-mailman (input) for mailman id 185723;
 Mon, 13 Sep 2021 14:20:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hZsy=OD=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mPmpC-0007IN-6e
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:20:46 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e7ad5cc8-230c-408e-bc76-037f3d72c5c9;
 Mon, 13 Sep 2021 14:20:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7ad5cc8-230c-408e-bc76-037f3d72c5c9
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631542844;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=iAt0k1o/FLU52gRnoWXiAAiUeQEwBFyFuwHu5mpSN8M=;
  b=UTiu4Ud+Kr0NoWGY3l6HjykG1Cuono/HVYj0k2dJ0ULlmTbPw2MjQV6K
   0CjxxFaXgNI0x0uKRDUWiFCbQUtiCBYftd0z+oKctZ++uut/Y7gDjkJnn
   2T75YXrb0sDg842xLuZNbbjjcCUpbvu81kg0K1CHqoULMIJIXwfcgiG4n
   Y=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: GqhEAQ7TUiZ2jysIj/Ot6avN18egzAbYm5GCAHQmwItlVHFcUK1nJwkfQFInHumvKPegnw7f/t
 0Dd86tNDDQZ1qUe0Z50enWsej2+TkW28aK9ynS1avpLUX0SxqKeQYRWaxbmw3E6aByWGSf4v5p
 82pz0m79neQzqhsqbI1WdXL4KJ1Imm4Z6gyp2QQRAzQiqpQCSWPEbeYlXTuEk7gbyg1JqG1s+T
 xuRjVCnnFN97yoYb/mJ6/aq2+lRmoDXiAYBvTovM45yqhwNrjwZlGeAKWKB9zHhggFD4SqUzdc
 5/tcW2f7b+8lOTzM6L7B7Qem
X-SBRS: 5.1
X-MesageID: 54357994
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:0BYKLaz8MQ6jgOZJ3L16t+cpwSrEfRIJ4+MujC+fZmUNrF6WrkVUm
 2pMWzyDOvfZZjGgKd92ao+w9x4PsJfRx4RmTVY4+yAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrdi2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt9Myx
 NljpYbgcg0GMIj9s+AabkNiMhgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVort4kI8TxepsWp1lrzC3DDOZgSpfGK0nPzYIEh2pv15ESdRrYT
 8ApURo/bRD6XzxsO1RPDsJjuLqnhXaqJlW0r3rK/PFqsgA/1jdZz7zFINfTPNuQSq19jkue4
 27L4Wn9KhUbL8CEjyqI9Gq2ge3Clj+9X5gdfJW6/PN3hFyYxkQIFQYbE1C8pJGEZlWWAowFb
 RZOo2x38PZ0pBfDosTBswOQoz2IsjAMQYprGcI0zTCmzq3puTy4Pz1RJtJeU+DKpPPaVBRzi
 ATQxYi0VGAw2FGGYSnCrebP9FteLQBQdDVbNHFeFWPp9vG++Nlbs/7Zcjp0/EdZZPXOEDfsy
 nihqCEkjt3/ZuZaiv3moTgrb9+qz6UlrzLZBC2MBQpJDSsjPeZJgrBEDnCBsZ59wH6xFAXpg
 ZT9s5H2ABoy4XSxqcBwaL9VQOHBCwm53M303gc0QshJG8WF0H+/Z4FAiAxDyLNSGp9cI1fBO
 RaL0SsIvcM7FCb6PMdfPtPqY+x3nPeIKDgQfq2NBja4SsMqL1HvEeAHTRP44l0BZ2B1wf9mZ
 sbKLp/2ZZvYYIw+pAeLqy4m+eZD7gg1xH/JRIC9yBKi0LGEY2WSR6tDO1yLBt3VJovfyOkM2
 9oAZcaM1TtFV+jyPnve/YIJdAhYJnknH5Hm7cdQc7fbcAZhHWggDd7XwK8gJNM5z/gEyL+Q8
 yHvQFJcxXr+mWbDdVeAZEd8Ze69Rp14t386Y3AhZA76x3g5bI+zx64DbJ9rL6I//eluwKcsH
 fkIcsmNGNpVTTHD92hPZJXxttU6Jh+qmRiPL2yuZz1mJ8xsQAnA+9nFeArz9XZRUnrr5JVm+
 7D5j1HVW5sOQQhmHf36UvP3wgPjp2UZlcJzQ1DMfotZdnLz/dU4MCf2lPI2fZ0BcE2R2juA2
 g+KKh4Evu2R8ZQt+dzEiK3Y/YekF+xyQhhTE2XBtOvkMCDb+iyowJNaUfbOdjfYDTum9KKnb
 ORT7vf9LPxYwwoa79siS+5mnfAk+t/ih75G1QA1TnzEYmOiBq5kPnTbj9JEsbdAx+MBtAa7M
 q5VFgK25Vld1BvZLWMs
IronPort-HdrOrdr: A9a23:mPoSq6CcPeLI/8vlHeg3sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHP9OkPIs1NKZMjUO11HYTr2KgbGSpgEIXheOi9K1tp
 0QDZSWaueAdGSS5PySiGLTc6dCsai6GeKT9J/jJh9WPH5XgspbnmFE42igYylLrF4sP+tEKH
 PQ3LsMmxOQPVAsKuirDHgMWObO4/XNiZLdeBYDQzoq8hOHgz+E4KPzV0Hw5GZTbxp/hZMZtU
 TVmQ3w4auu99m91x/nzmfWq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczOgBkF5MWUrHo6mt
 jFpBkte+x19nPqZ2mw5SDg3gHxuQxeokPK+Bu9uz/OsMb5TDU1B45qnoRCaCbU7EImoZVVzL
 9L93jxjesXMTrw2ADGo/TYXRBjkUS55VA4l/QIsnBZWYwCLJdMsI0k+l9PGptoJlO11GkeKp
 gsMCjg3ocVTbvDBEqp+1WHgebcEUjbJy32AnTr4aeuonxrdHMQ9Tpv+CVQpAZGyHqRI6M0kt
 gsCZ4Y542mePVmGp6VNN1xMfdfNVa9My4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,290,1624334400"; 
   d="scan'208";a="54357994"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gz1VffEif7i2BvhAwMr0btZfVCTwnep0byajhkJFfX/pSo26sOQPt0dGw8xPhQq965aquJ4HMxBk0yCVzRYVgybdXhX4h8cpITcnGxnKdNNcqL1hPV7bAx7QdvB1I53JcM8Rkao6NxlGOKqqs1LQtrrxXrJ4I0QY3n5YWyGOrcwCK276HGukuSiF9+MHYUh5VRUVVC7i5IBpjw2bvJL08Np7emvS++5rBex7BIZADOPm5IwLqjbezgvyKcBdwVJOF5jwAtdWyBjakwOmDOY1ZnskGChGAFSdyIeG6G9UwNfatGuormObVeizUcQVW7Ow9wLBpQkuT1ZjGk0b+AwRUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lFIpKuxY1KDo93eqklJaIn6eKn2dt3rHoncR3xazink=;
 b=nHu5GyXbgpQEuQQeuNlOfwffUhsHKKAIhMmv3rVc9Rv5ZPRmIoIsywCETKfm5/6KjRERreOcKL7WEXWmUwtYt9PaNihMtLbyBhOBWllv/MRWfOmq9fQP/6GKb0rsR3eMbXREa9yY+8VPnL53j3a4+bRA7HrrYoz64hMmYbNFQzJshKCSzY5SnI5GoQlccQAMZHNsgCvSPbTWJYjHyjgsiihkKptpp/g4nE4WNA4dxn1eBlAW5/zpZGrPMvYVVlHq8hlMmwT5/aGEb7brWnOe0mZE2OSGA81k7JEm8FEwX/S9oQj9ZgLl3IFyJvaaYwUgSK51OpmBOXssZNnwiV/Zvw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lFIpKuxY1KDo93eqklJaIn6eKn2dt3rHoncR3xazink=;
 b=N2+Lua01SaXfv48pKump9FSx1O9L4AS0tNwP+/sPv3/E9MMMz5+VslPCu1bQOJzqeqdrayoixSxkGNLDrYQWmr7ntkkbf26j371PSmIWmHo/Pzr6wRO3pAxAMmCgfwWh2J5Vr7Wl9UhkTXmqGXnEjvZWx8++7dTMGZx5c9Rsp7w=
Date: Mon, 13 Sep 2021 16:20:31 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
Message-ID: <YT9eL7b7A8CirzOa@MacBook-Air-de-Roger.local>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
X-ClientProxiedBy: LO2P265CA0265.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a1::13) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6b85a7b8-a59f-410b-a536-08d976c1a78c
X-MS-TrafficTypeDiagnostic: DM6PR03MB5065:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB50655E42717F8A5A4D7FA2428FD99@DM6PR03MB5065.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 8AmVr4Oc+g5z/Vdl7txidD23RFXNs8x9KyYBbdIODyIi+heAwbk+S0KYQ4WIYAPdQrB2S6ZcKNClMadn0Ge4QhHBJ+nWVDSVA27F7SNwDdiClNiVWe+3eo2FYe4nsQ+e9RUEUgHWlM0FP+u6lqT+T+1X5NKUz6A23l9YiYYA15Fwn/QuLU2NpLChaOvDxeqe6dIOApNcfe6M4tlEOmcz2CR02yphMPANKDRMm+HpGR5uzpSVw8QNrQFM1CFaGo+cQTC+DxGqnV8/QvZD5l7SUt0B9A7muy6FkA6cnplt70li+yzo1LczRYObmw4wvXdBJYt+qdJpg8rbDaoAY6EpKLzyFwWZ9Vqeco8ZNpPwVhRwBJkB88AUj9OLRtcRjaN9AAyQ9oWNKeOURHdL70M2DmSP1Y6Ttss9Tcq+kMSTa+CkemjOq4Ij2y6W2g4hTX+pSdSg1Yzrh81N/N15do3FpcFbGL3hYxlCeAj3/ES8dATRWXr6nmKt8xnLV6PE6FXODeS4mipt1+Hpx33lWiNovvliLQSDaPdt4jT7QupoKU3FoM/RbV5//MgjK+/hd+qP28hWQesXBS+U0Q8COo5uLJHSKYwvjOcafOflghGS2xZ3BB/LlYfwc/yS1+mH3NmO+ZyN++GDAW4kQRfKC20nOA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(376002)(396003)(346002)(366004)(478600001)(66946007)(8936002)(5660300002)(66556008)(85182001)(6666004)(186003)(66476007)(6496006)(2906002)(54906003)(4326008)(38100700002)(956004)(8676002)(4744005)(316002)(6486002)(6916009)(86362001)(26005)(9686003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aHQ2OEl2eXo4cmI5NlN2K25SK21LQXRLZ2JuNzk1bWhTVFF4eS9WWlpmRllw?=
 =?utf-8?B?RTlmL2w0aDVvOG1GR0c0K0pDNHNpbkkxWTNaOTY4bk9aMDBmV3VPQ05WWGp4?=
 =?utf-8?B?TFJ6WEJnaXRUS3BkRUk2RURkN2dLdDR4bUx1ZzZRSmhoMkJJNU1xNDFPWDlW?=
 =?utf-8?B?Nm9Pa1VxV3hibVJJQllqdWpaWFU1SjU3dTduVTBnbEpaUTR0LzFNMXg0bVUx?=
 =?utf-8?B?ZUs4ckZ6N1lYR3hyL3R4anNnckhCWFJER0FuV0V2cUFaa0dLeDlUVXhZVTJp?=
 =?utf-8?B?MVQrVVRMUGcrQk8rSXV3N3ZzRWh6M3FKYkw4QnQzdlBwTkFLd21INzlCWVNz?=
 =?utf-8?B?VGd5SzRQelhIekdsZ0phQjV0WDdGTG9KZW9lc1dQaHZidWRCdWJWdU03dkVV?=
 =?utf-8?B?akJuUEd5SkxpYjNmUncxckR0ZUwzRlR1TjNaTjVyRXBiZVB3cnZRSTdTVnp6?=
 =?utf-8?B?K2gvNlpibllOenY5UWJIM2R0OC9UUW5YWDlHa0M0V0JHem9VQ0lNaEpRWDUr?=
 =?utf-8?B?K1lVWS9WNzhpV280ZUFKTFM3K1JWSFZudE9UalhaYk9QMW1yYjB2NUd2VmVQ?=
 =?utf-8?B?eUpKeGxLVGlvNythVHhMRHB0UGowVWxCSXhQb0dVOFFLdkhrNURWU256dFc5?=
 =?utf-8?B?SGJDSnN4RFVRYkhWMUdIUVFiOUlISzhkTFFOVnJuazRKMGhIT280M2g0cmRy?=
 =?utf-8?B?OVF6aWxWOGFUaFRLZTgycHFlS0hOTExXWWdwZk85MGFSMmlOZk54aVA2ZGFX?=
 =?utf-8?B?dGlDM1NTYUpCQ1ZLeEtiNndPL0szNUdJTlJQQ0pmbTZnczJqNHNHSG9QdEhS?=
 =?utf-8?B?MTg5b2lBN1ZxS0xhcmZ5Qi9McWtKQnFkV0ZNejBIMjJiNDRwRlo3QmVjTkJU?=
 =?utf-8?B?ZWFqRElYMU94NVIybGpMSXJ0ZktjTGp4UDV2QWQ3ZlhUdCt4dHZRNFA3OGNr?=
 =?utf-8?B?UVRTdmF1RGJDa1cvd3RaT2JPSlFFS01IQ3dMUTZkTEYyeWVvWnp4cEVRdDJM?=
 =?utf-8?B?aGd1eERCV2U3OGlpenM0MitFQmEvMlRlNm5XSTNBN3ZQL29sWXhWZTQrWVJG?=
 =?utf-8?B?YW5YL3RrdS9uUFppcHIwS05lNVZUV2xZWFZiQldUZ1MyM2pXK0NiTlBUdzVr?=
 =?utf-8?B?TXZuY3huaUp5N0lpVW9iNUxIb3B4b0l5cTVRUmZFaFFxcS9aeTRsN2tNMkM2?=
 =?utf-8?B?Sk1xbGZhWTZGUHBqTXE4ZUtSeEJ0M0taVDdsbTR4MnpzTjllNGZSNHRteTVP?=
 =?utf-8?B?bUluWEY4VTJsZWp6MWlwWFYwUHdSRXoxcEIrZnpQdnd0Z3BTR3hjQjNNS0Nh?=
 =?utf-8?B?aytZcXhKME94bnl1QS9KVHh1Tll5QzFlTm9PYUZtalZENjhnbzFLcE1JcUIy?=
 =?utf-8?B?dm5QTlVsajk4U0pTckFESWZrWXNYUk93dXY4TFZQelJPdDdrSjgwaEVYc09M?=
 =?utf-8?B?RDlKNW5lNEVPU1hDWlBuMEkrVkRhQWp1dlZRRHh3Wm95aGVkeDBFS1AzdlRJ?=
 =?utf-8?B?UFNPSjZnNzJxdjVnM2IrNzV2YXNDSHFXbnlWemtkVU5sZ05KaTdsbFRKWWxv?=
 =?utf-8?B?SUlZOFh0aUtKaGxmQkorWnFCMnlybEIrRm5zZktibmVneldQS2VqNWhQWXgx?=
 =?utf-8?B?b05ZVEZTVFU0RHJ6MWZuNWdWQVBKcjFJNnhFMDdrWVdBN3BzVVRYYzBnb2VR?=
 =?utf-8?B?L3ExbFNMMjJJS3ZoeHY5TzhTWnZYWUhvYmFKdzZRaFdJcVY5Z04rWSt6VmZL?=
 =?utf-8?Q?4kxh4+lUEYU/skO4pFava6fKaOq++aQWzpX//FL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6b85a7b8-a59f-410b-a536-08d976c1a78c
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 14:20:36.2130
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: n1fKqUc2anOV7swFgNW9+jjF1fN2zbh9Oj5dodV9XMhNRju6clIttmJT1SBsH/ZCrkslkkr+lj/pdw1CIn0qUg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5065
X-OriginatorOrg: citrix.com

On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
>  .PHONY: clean
>  clean::
> -	rm -f *.lds *.new boot/*.o boot/*~ boot/core boot/mkelf32
> +	rm -f ???.lds *.new .check.* boot/*.o boot/*~ boot/core boot/mkelf32

Forgot to mention in my previous reply, but what's the point of using
??? instead of just typing xen.lds?

AFAICT there are no other .lds artifacts, so doing a 3-letter .lds
wildcard seems weird.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:33:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:33:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185730.334436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPn1H-0000T7-9v; Mon, 13 Sep 2021 14:33:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185730.334436; Mon, 13 Sep 2021 14:33:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPn1H-0000T0-6u; Mon, 13 Sep 2021 14:33:15 +0000
Received: by outflank-mailman (input) for mailman id 185730;
 Mon, 13 Sep 2021 14:33:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hZsy=OD=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mPn1F-0000Su-IV
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:33:13 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 857bbae8-149f-11ec-b3b8-12813bfff9fa;
 Mon, 13 Sep 2021 14:33:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 857bbae8-149f-11ec-b3b8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631543591;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=dbEFosl56BvADuL2whBTXBmQMxHWcrJp/eHiu/xDp/8=;
  b=fZ50Mapmvh5x4FRO5/3Z0twfE8jHJjgo1T9eRkjhmCFE9xzN8dWCG87D
   qMw2bHC8fDSLRPV/40VkrwO3/28Mp6GBJOpcJ0ryvQcFDNdWZDXXRedTa
   d2WaCOcb8R8ois1mdpZD8o3Rl1tNk/PToyus2ITc4XcYEuf+JU1I5A40K
   Y=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: wV9EQ54t5JNj1L3zVWB1W4IH6mGj/xkjhDLcts7qCwcJZ+GSP2vBVgyrWnOUwnCJin6L3D/YIS
 6VNn0X04yi4tkUqQtG0bG7UtmTiWE4+jYHlx7cXbnrnVEanKYWyhr0cnUpXnOoF0Ac/ebjmFNq
 UW/2pRINmwSLDo1u2RMameRMh70bHsMBWH9urJdjtPuUUa2/bnEcriYTlCwdXVGsQyTihJLZ6b
 KTJsgvd/moQOfQNrIz8jloZmjC/XCr3i0xwPeKPE7Dwd9BO74sVs9SKXrNPQ33fKVHgYaaHHWZ
 jFTvmATyEwzvdXzgkky7GpGM
X-SBRS: 5.1
X-MesageID: 52606106
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:5JKQAqs1gd3tMzVn384k2kR+O+fnVE9ZMUV32f8akzHdYApBsoF/q
 tZmKWCBM6mNNjOge9AiPIS09EIHucLVn4UyGgJs/i4zRC9G+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y52IPhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npl7rfoSToDMPD3iukWdxdfHgheBL0aweqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25oQRaiHP
 5RxhTxHTw7LRhpzJUwrFI9hpe2QmEnHQRdAkQfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krW8mK8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECqF+Zy3EXCRYWfUCmuvT/gUm7M++zM
 GRNpHBo9/JrshX2EJ+tBHVUvUJooDZHcIJQLsc0tTqfx6Hw7QGUL0g9XmJ4PYlOWNANedA66
 rOYt4q3XmU16+DEFS31GqS89mzpaHNMRYMWTWpdF1JUvYO7yG0mpk+XFr5e/LiJYsoZ8N0a6
 wuDqjQ3z54XhNQCv0lQ1QGa22/wznQlowhc2+k2Yo5HxlgiDGJGT9bxgbQ+0RqnBNzFJmRtR
 FBex6CjABkmVPlhbhBhpdkw8EyBva7ZYFUwfmKD77F+rm/wqhZPjKh74S1kJVcBD/vojQTBO
 ReJ0SsIvcc7FCLzMcdfPtLgY+x3nPOIPYm0CZjpgi9mP8EZmPmvp3o1OyZ9HgnFzSARrE3IE
 c7LL5/zXCdFUP4PIfjfb7517ILHDxsWnAv7bZv60w6mwfyZYnuUQq0CK1yAcqYy66bsnekf2
 4wHXydT4xkAAuD4fAfN9osfcQIDIXQhXMikoM1LbO+TZAFhHTh5WfPWxLogfa1jnrhUybiUr
 i3sBBcAxQqtn2DDJCWLdmtnNOHlU6FgoC9pJicrJ1uphSQuON798KcFepIrVrA77+g/n+VsR
 vwIdpzYUPRCQzjK4RoHapz5oNAwfRinn1vWbSGkfCI+b9hrQAmQoo3oeQ7m9S8vCCurtJRh/
 +38h12DGZdaHlZsFsfbbv6r3midh3lFlbIgRVbML/lSZF7orNpgJRvug6JlOMoLMxjCmGeXj
 l7EHRcCqODRiIYp692V17ucpoKkHuYiTEpXG27XseS/OSXApzfxxIZBVKCDfCzHVXOy86KnP
 L0Hw/b5OfwBvVBLr4sjTOo7kfNgv4Pi9+1A0wBpPHTXdFD6WLpvL06P0dRLqqAQlKRSvhG7W
 x7X99RXUVlT1BgJzLLFyNIZU9m+
IronPort-HdrOrdr: A9a23:hk6j4aM3I61s28BcT1D155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/uxoHJPwO080kqQFnLX5XI3SJzUO3VHHEGgM1/qB/9SNIVyaygcZ79
 YdT0EcMqyAMbEZt7eC3ODQKb9Jq7PmgcPY9ds2jU0dNT2CA5sQkTuRYTzrdHGeKjM2YabQQ/
 Gnl7V6TnebCDkqR/X+IkNAc/nIptXNmp6jSRkaByQ/4A3LqT+z8rb1HzWRwx9bClp0sPgf2F
 mAtza8yrSosvm9xBOZ/2jP765OkN+k7tdYHsSDhuUcNz2poAe1Y4ZKXaGEoVkO0a2SwWdvtO
 OJjwYrPsx15X+UVmapoSH10w2l6zoq42+K8y7RvVLT5ejCAB4qActIgoxUNjHD7VA7gd162K
 VXm0qEqpt+F3r77WTAzumNcysvulu/oHIkn+JWpWdYS5EiZLhYqpFa1F9JEa0HADnx5OkcYa
 hT5fnnlbRrmG6hHjXkVjEF+q3pYp1zJGbJfqE6gL3X79AM90oJiHfxx6Qk7z49HdwGOt95D0
 mtCNUdqFh0dL5lUUtKPpZ2fSKGMB2/ffvyChPmHb3GLtBNB5ufke+83F0KjNvaD6DgiqFCwa
 j8bA==
X-IronPort-AV: E=Sophos;i="5.85,290,1624334400"; 
   d="scan'208";a="52606106"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XN0qeSVCaMfclGnqeSPe62B/U0a3hnHDHDEw9qXPGYhRi3ZSfCrrr3GvchntzOeDZM0WEj/S9QgtyL0z4dbExJCpKcTc87NJfXzal1QhVsZbYgUfpEW+JjUAmQ4Za20JbdNlOjvPApO0zXoqP/fvEfS8Yv2EWPurk6C1enyiolJBusgYK8sVvWpoNeMsZuKIK8rjHxOAyoYIHinUUwcg/cBXZ5hLtU63+B0VWSf0YViMMvIOrZAlqZIFUmdG8SG1s8MvPUYEY08pSa8DkLxiSbcP7D3ifX8s/m3xPRpmvrs/tnpQw9yASC3ItipuaX+MoPLrbdMuVSN6iO7Su6d6gg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ctat5Gg+/Z8V4ZI07tGQbtU60i88SPRxZBQcf/3yWwc=;
 b=ePSBfGjS2v186w9FzqBXvMn00GojuI3+UQAVB+HxLJMwquwT7xl1bXa49U7JScyWxmsw0Cbkas/WdjKUPe7NBPRYcKBzVQ9j9EwW4SlEuAo42v6AET1KuHSeMpP4NKWbzE/vbkcM9Dwo7HCYz4keboPwc4zyhfZxqlnvlUaveeHTonQhsHiduWuWbPf+rU19LdCN5WTLsFlczTnXYohx0xrszndNInq68r/WoDwcpxHbi8s3vAV/aih856lm5cO0PIT1JNTcWokqA7zUTnfHTMZ7gj1ql3ehuqK6SXXeb7d3NaJ5/vbkT5Puov+x7+hxzwTCfk5x3Y9H1ozjIjf4AA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ctat5Gg+/Z8V4ZI07tGQbtU60i88SPRxZBQcf/3yWwc=;
 b=fMYaS7a7DjXpPrwAJ+8RDhj66cnh7TI8R7rET8aYAZEChXIpu6R/gW2QACQxvw/oblQObHPedxOPf5OPT0Ggk4D938JrDyDHrMXI9L7gU2xarMAGuKP3e0o5vJzcsdqfHIEaorRYwC2wS9sqjqfznx4WrwCOhaV/z8T/fkMdx2Q=
Date: Mon, 13 Sep 2021 16:33:00 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
Message-ID: <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
 <7c97de87-9687-a088-406b-33faf037244a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <7c97de87-9687-a088-406b-33faf037244a@suse.com>
X-ClientProxiedBy: LO2P265CA0012.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:62::24) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cb361a21-7792-4f49-7075-08d976c365cb
X-MS-TrafficTypeDiagnostic: DM6PR03MB4603:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB460380EEF52CF047BD5743F38FD99@DM6PR03MB4603.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +CP18AFlbRsNLySkaao+/l318D8oLnHetKyscl8+BbUye97VKvLX0GHO3CFkcK6XuFDf3EQiVLFZq+hZkdsbcEhpdzUw/l/G8l4xY0B8QuuIvWpY+jaDW2YExbd++QyNkHK1asd2I4SsKdisu14OUHoGyBVVA1zc9x3vKQQteXvILjuo/xsHCLSxq85vCPci8+8evi3+J9mR1lqe8wFxMkJCQoet4bEdtsqQ2jhlzx4dWamXI5PsNRs5WS2KimRV58Mb/qI+K7p14+BWhsObnq32H89Aoe9Pnnp32aFwS9FXqoffrYJyrjW7QT5X1AMraqEMf2au08iFIHPQiBvW19d+tmaN9RxvuDoXBjI9mmWdi5XWwzcgVCauAiK7OyUotAIxsfSkAAHdS1fhtPlPhNmIts1x9NKTMQMysaHGWfcYswPzSh4ZIqlyfVZLhfO3JQww2qEe8A+u9zwVNfOVwtZglMq/jQRzFFsIMLoAm2p30ph+h/f7N3YFUqF3yBS0znbvHPI9YFtViMsKWM/rfFt6GSxoPY2o4sXGvnVexb9+uJE1fiQlapbXgdD8Ycc7boH3778VDdVCUwDylyBNaamU1lhGFMcWX3QCf8SvtWCkX0f9D8ayxUrfNhrDNc6CxtgezTQC5Gp6GnKQKUJ+EA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(6496006)(66946007)(86362001)(9686003)(38100700002)(5660300002)(2906002)(26005)(4326008)(8676002)(6666004)(83380400001)(956004)(54906003)(53546011)(8936002)(6486002)(508600001)(316002)(186003)(85182001)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S2dsWFRhOTV6dFlGWGk5anhKVzRjVHRBVW9rbXV4OGhZekNtVzNvenNRNGhP?=
 =?utf-8?B?OC9tM3hZa2QvV0h5aFJuaHQzSnAyK044Z2ZWbWh1aERXUFd2aTlMYkZvYm5w?=
 =?utf-8?B?QVh4dGZVVjVnMkZhRHRUZFpJWVluME02ZVJSTzljMVBqdEN3OUR3TTF3TktT?=
 =?utf-8?B?eksySVFaelVZVWtpNGluTUdFanU1Ui9pckN5K0VyZ0dFelFKcTdOd2VMdlpW?=
 =?utf-8?B?ZTQ3R3dhOW9TSVdDWmd2d2IrNjhRV3pWZEk1MHVMc1R3Y3BYUGt5UnY1T3VR?=
 =?utf-8?B?S2Zray8xOEZxOFdDek1jVk9EUm5GeE1tVWFvYnhOYjBUSTlWdXo4cnc3U0xs?=
 =?utf-8?B?aUU4VkFrQ1dYMDNtZG9OcEs2VTB4YXQzY1dMUkdwaHZwdmpEeGpsUzh3clJ1?=
 =?utf-8?B?SzFOOWlyWEFZaVFhOHBEMDFuSHltVUpRczRxK25mdFk4R0VTUjV5VEM5SmpL?=
 =?utf-8?B?RnExZDlpdUpGU1Q0TDJOd3JQQ2tUQlNmK0RnbCtxNHl1V2tIdFQ2b3Z6dGMy?=
 =?utf-8?B?WllJYTZDVnJodEZNVEpPcmdsMTZPcjZTZjlKd3JmN3NVc0t2Mks2cFFac2No?=
 =?utf-8?B?emFZNmZsYUVUZWt3dkFFMzZrV0tuKzYyamVkVzJ1MHpmWU1WZ2RPY0NYVXJO?=
 =?utf-8?B?eEFMMklWVGR3N214NzRIRVltSlZEVm9vNnpBQ1lSQUFsMEFYdjFEUlA1Y1Vl?=
 =?utf-8?B?YmgrNXVodFNwVTcxTmFidWhOYXFpUFRYV3krS2hhK01COFJOU2c4NE9iNUc4?=
 =?utf-8?B?OTFLZEtpVHovKzljSFdtYlBwOGpLWWpQS1M0M3dUQ2UxN3JoV0JyMVdSYzNj?=
 =?utf-8?B?c0RGNGJzL05Zd2liRjhjUEFjQlpFajIxZjMxOHl4WGNwL3pmdFVVTXRFa2gx?=
 =?utf-8?B?TENheExEa1UyMVd1MDVsZzVubTBkN0g1YlVudVZSc1dML21WajdvNnhxSXRv?=
 =?utf-8?B?dEswbEs2SUVqR0FSa3dDNEloMEl1cjcxa0hWSWZtU3FCb3pEUDJraWFHRFVM?=
 =?utf-8?B?QzEvQ20wbEFkU3JMWlRtbGtXUk91Y2YzYzRZSFlJTWFhSitkK3hna29BMzdK?=
 =?utf-8?B?bDlFcitKTjlvUkU0cmRuRzFlNzUvN2EzZXE5bTFHNzU5b1N6QWt0bUhqMkJy?=
 =?utf-8?B?d0JQRW5BUnQxdm15K3ZwT2Y1L2ZDZUxGeFVkQ2MrdGxnV1MybC9FL0t4SlF5?=
 =?utf-8?B?d1VJcHk3bGYvaDRRdFN5RlNwRUszOWpwS3hDdy9OaXc5c094WDQxaGs4QW5i?=
 =?utf-8?B?K3lnVTRaZWFVemt3enphalo2RERSU1cxMldjUUNMN1pUUjN6cjN1ajlNWXBW?=
 =?utf-8?B?UHRtTnFvZ0hUazdSZ2FsVmEyZXE1b2NmeDc5QlV6T1ErQ1dVMGhtbnJmOUFp?=
 =?utf-8?B?U1NUYkJXT3pyQlBvaG03aUVHWFYvYnVoK28xVXo4THhlZFhEcFBvekxhZlVs?=
 =?utf-8?B?QzVrRG5KdHI3Rm9aVGJIc1cwL1BRdDRJSEJaVjl5YmZ3YmczODJ6YkNhM29M?=
 =?utf-8?B?cnR2cWFvc3BDUEJTOEtSZ21zam9GY2FIK0NyVWpLbDFtUm8xZ2tlZFVpcEZt?=
 =?utf-8?B?Z3JXaTdnZEN4MGlXaGhyUzZsZVZITTBMN3kwZHpvMU5xYk1BR09PK09Oalkz?=
 =?utf-8?B?M2pJdHBHYTRYaG96LysxTmprZUZtSStubmtuemhoMTBqODJ1ZEtPRnZrcGJC?=
 =?utf-8?B?SUg3U0dSMDkxdW5FNFBNOGdSSDRvNjZkbFNzY3ZKWjNZWkN0ZWp6R0tZd2tn?=
 =?utf-8?Q?U8kQ5AZLaqoGG3Fi2nZ+cYSnBMRAUnIVyKcl06H?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cb361a21-7792-4f49-7075-08d976c365cb
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 14:33:04.9168
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UvtP0UkMPoEJyA9Aw2QwGRQUZXTR/O7sMX4dT/YKU0ylghPAVBd8fGgol/xt875TkDGejpnJXdRjaiEKbUASNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4603
X-OriginatorOrg: citrix.com

On Mon, Sep 13, 2021 at 04:05:15PM +0200, Jan Beulich wrote:
> On 13.09.2021 15:37, Roger Pau Monné wrote:
> > On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
> >> I haven't been able to find an environment where I could actually try
> >> with lld (ld.lld); all testing was with GNU ld (ld.bfd).
> > 
> > Thanks for fixing this. I've been able to test with LLVM ld and the
> > workaround is fine.
> 
> Oh, good, thanks for trying this out.
> 
> >> --- a/xen/arch/x86/Makefile
> >> +++ b/xen/arch/x86/Makefile
> >> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
> >>  
> >>  ifneq ($(build_id_linker),)
> >>  notes_phdrs = --notes
> >> +# Determine whether to engage a workaround for GNU ld 2.37.
> >> +build-id-ld-good = $(shell echo 'void test(void) {}' \
> >> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - 2>.check.err \
> >> +                           && $(LD) -T check.lds -o .check.elf .check.o 2>>.check.err \
> >> +                           && echo y)
> > 
> > Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
> > then use is here?
> > 
> > We already have compiler and assembler checks in x86/Kconfig, so it
> > would seem more natural to place it there.
> 
> The question of whether to record tool chain capabilities in .config
> is still pending. I'm not convinced this is a good idea, Andrew keeps
> shouting me out for that, and an actual discussion doesn't really
> happen. Yet unlike back at the time when I first raised my concern,
> Anthony meanwhile supports me in at least the question (to Andrew) of
> when such a discussion would have happened: Neither of us is aware,
> yet Andrew claims it did happen, but so far didn't point out where
> one could read about what was discussed and decided there.
> 
> For the few uses we've accumulated I gave (if at all) an ack for
> things happening under some sort of pressure, with the request that
> aformentioned discussion would happen subsequently (and, depending on
> outcome, these would be converted to another approach if need be). I
> have meanwhile realized that it was a mistake to allow such things in
> on this basis - the more of them we gain, the more I'm hearing "we've
> already got some".

I see, that's not an ideal situation from a review PoV, as we don't
have a clear placement for those and that will just cause confusion
(and more work since there are potentially two places to check).

What's the benefit of placing the checks in Kconfig instead of the
Makefiles?

As I understand Kconfig is run only once, while the Makefile could run
the check multiple times.

The downfall of having them in .config is that .config could suddenly
change as tools are updated or as it's moved around different systems
(yet .config already contains specific compiler version information).

> >>  else
> >>  ifeq ($(CONFIG_PVH_GUEST),y)
> >>  notes_phdrs = --notes
> >>  endif
> >> +build-id-ld-good := y
> >>  endif
> > 
> > I also wonder whether we need to make the quoting tied to the usage of
> > build-id. I guess we don't add sections with dashes and instead
> > use underscores, but it might be prudent to always quote to be on the
> > safe side if dashes are not supported.
> 
> If quoting was uniformly supported, I might have considered that. But
> it not being uniformly supported is the reason for this change in the
> first place. Hence I'd prefer to generalize this only if really needed.

OK, FE. As said I don't we would deliberately add sections with
slashes.

> >> --- /dev/null
> >> +++ b/xen/arch/x86/check.lds
> > 
> > I would maybe name this check-dash.lds, in case we need to add more ld
> > build tests.
> 
> I sincerely hope it was a one-off that a binutils release got cut with
> this sort of a supposedly prominent bug. Considering that the dash is
> merely what we're after in this specific case, but breakage was wider
> (presumably about any printable char that's not alnum or underscore),
> I'd consider check-dash too specific a name. You only say "maybe"; if
> you were sufficiently convinced, this is an adjustment I'd be willing
> to make. Yet even better would be if I / we could just be done with
> this.

Ack, we can always rename at a later point if there's a need to,
hopefully we are not being too optimistic.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:46:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:46:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185739.334448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnE1-00025B-H5; Mon, 13 Sep 2021 14:46:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185739.334448; Mon, 13 Sep 2021 14:46:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnE1-000254-DR; Mon, 13 Sep 2021 14:46:25 +0000
Received: by outflank-mailman (input) for mailman id 185739;
 Mon, 13 Sep 2021 14:46:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xuNF=OD=epam.com=prvs=989052e8b7=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mPnE0-00024y-2K
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:46:24 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5cf11b7a-14a1-11ec-b3b9-12813bfff9fa;
 Mon, 13 Sep 2021 14:46:22 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DE0unP011291; 
 Mon, 13 Sep 2021 14:46:21 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b273drfp2-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 Sep 2021 14:46:20 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3715.eurprd03.prod.outlook.com (2603:10a6:208:43::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 14:46:16 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 14:46:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cf11b7a-14a1-11ec-b3b9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gm2fDld+3YyqFYlnG6d/pZ9zVP08axbBU6+u6rmByVCyh5vWc2nt8UruYPRKpTiK7yZGms00bM9+uezrOuTGgJ8ls/Nh5SJnw6q8gDyhAwtNjICf6YNf5MQezo2RW2A7v4eU2yVvgRpy25HCLoh513lAq4iuQYbPXC6tGfqKWKCI2nQSoU9/Qx3gD5cNky4uAPJzQOVhGHJjsHxIxSuxN0lnPd84yAZOVNa6fAJbI8rf77EOUYvmSBB0r8ubZKWCEoF54WV8co9nBM9DdEwyH8ylTIR2zbg4lSJ2IGqKfoF1/yQllhkb5BuioamTtbul65WiktxBSoO7YNyzlFOXEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bCDrP9hgMnLG8V3dh06tN85ppNhw5CwR/2mQUbPas7E=;
 b=kxpOW9NJzCoNIQZSiAIyZLs0NJQnMQoHihl34mGCwqRWmeIkzg9qlzEF4UryLyodkOzcBG0n5XqdASJwZneBPQXU+PaRkXgUESx5k431AoaOPCebpUgtRTSqsr+N2Xj0sQIHEwKAYG/vrsi6pnoi3kKvMw0gYQ62OtZKyGgDHnyWnNAYbGmQ86WwcIOjyPhtwt6dkYDBH0htNGaQLZ94Qipr8tU9Z/FDYxx/WlnSjp7Enxi/miPsEDJpQrCkCJ5v2m/ndz79yhSmG4mE+suDBO1P3XtNOKpM9mAQ1u62fjipLWMtM/rP1fcZWpgvCcyXZzQFUcYwgf66OLFC6oOong==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bCDrP9hgMnLG8V3dh06tN85ppNhw5CwR/2mQUbPas7E=;
 b=jS1zGFfwBYvEbxRelNS2XyYHGGrefpphTt0OwMXvzVTtq1dcN9uKs5wp3pN4C0iJ8smNKBLVtGFTLorQf9luFEXqQa3VUV2f7ghWnIpE9p7BFL25zq60/lVkGxJriB02eGk/4ySBdd9RwQ8tcG36lsaWnNt56aLHElb4s5K3ehDxld1pNcfQpjkRnF/usmkuOIP9buPd384D/TGexaVdEtNffpGkRRfix8y/m8G9RtIxCFhZsD4LNfk4JiJYZxG0bymuXCZQoD4OShvrxZcXGeD0mWOOJ0G19sPfkELGcz6EQInFG/Yu763OVw0ZXg2R/oqjRMKmLWRmj1+k8ow97w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>,
        Julien Grall <julien@xen.org>,
        Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXlPK8ZsFWc1idT0uR0zIe97lMvKucfJAAgADQqgCABOUWAA==
Date: Mon, 13 Sep 2021 14:46:16 +0000
Message-ID: <aa251354-3e5f-e1ae-2647-3a112ad5d12e@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
 <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
In-Reply-To: <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e06da1be-a0f8-4508-e1c4-08d976c53de7
x-ms-traffictypediagnostic: AM0PR03MB3715:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB37156709146B0DD5B01BEDE6E7D99@AM0PR03MB3715.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 XZZvzTAbJhvAaJAfDcux8mpASosFNsv73D5siU8p0WiavO9kptoB8FAcok4T/OH5LwiGPiJRaHXpvP+m0AkMV7WeWoP8j7l5zQiknc7wVRhOOhRtyNJW/sA4sCH8Z2QuYG+v6qdeyyey6bgLLgHb+srzg4VUx9Bqa/iUVJYMz8DcaapyfPEWpKxmei8R3BHlWsLsbD9iHKSjNWqdGXw21HC6+vlm7vaAFB7UcU1kWWOyNahDYG4Dwzs6aiK21yFF0uK3YLiJ1YYjsrAJxv4XdiT/BmdfSg46G71MzhXB2L50B9icBA8JJAxaJbyg9lbM0ZTePgs12kOqeVHJsdvTFlUj60oGb9G7jxIcUYeieVvah/yIET2GRgdVzGaWKySsw2oEWzDx3pjSYU/unWrf0YUwB8m+jU62wUgpEF3/QuuOhoQ9Y8bLrtJpzXjQ8BdRvziJ9ENo+N82//16hXpoVxLcW6Nk/v+88RaXO5DPkepmiXQ01k2ySDaQGi14JlngnC59aCXJUTax75+tq0Dx9Lnv8B4jLi1xULlKmTfdvVhdXDX4AYX2yxDRYxPdA9s0CFl6fgLhWPtKOlvpzjSNzSD8bZNXGCOvA69CqDGLhoHpHB+/SiOWecC5UogbJoF5b5fJaHW5IU45qP37eieLvk0fEXl66pw8IIxL36q2YmV+Zs2WCEiMLTni5RnVcaJNJjMWp8X44qAxsmXazRsBRJBpcmYFzY2Jsuq8ettKxKFtHQRbs2/FCpN0ooI6wnXtgAsLsWXJtqiNE3Ek2+J3N5tvWZHVZKgPJErEUHkNp2xlNBQqT8/dJlqyzWX30v3zVymcoF5z+b+1/aD36zbXAw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(376002)(366004)(346002)(38100700002)(76116006)(26005)(91956017)(54906003)(5660300002)(66476007)(66446008)(86362001)(71200400001)(2906002)(107886003)(316002)(31686004)(6486002)(53546011)(6506007)(8936002)(83380400001)(8676002)(31696002)(6512007)(55236004)(122000001)(2616005)(4326008)(38070700005)(66556008)(66946007)(64756008)(478600001)(110136005)(186003)(36756003)(2004002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?MGFTNHVnRzBRS05ISGhjY2VEL3pLaUk4ZE1UUStEM2hmeFp3dEUzRVFBUlE5?=
 =?utf-8?B?RWxaSDVnMVprV2dVc094QlNYTXk4dVIveHUvN29CcE1wVkRXRUpFNlpmdi9I?=
 =?utf-8?B?RUlaMEJqSGszTUFPSG8vQkhDWncyTWFick4xeUtVQUU4LzkxZlh3bVlaUDlt?=
 =?utf-8?B?d2tSWDdRcmJDbnFwcEVSMDVCdGNlNktBR0hyK0tWNnJtTFh4RUthZDloWEd1?=
 =?utf-8?B?cjlka0lsRW5HdXlyNGZSTzdQKzJuS2lSbDFHMlVqTTRabFZhdHpIbzJua3kw?=
 =?utf-8?B?VUxScFQ0YkZNaW1JNzFWYzllU1lDbytnb1J6RnZHeG40a0FkV1p6ell6bWJZ?=
 =?utf-8?B?TE5lQWN5MzJTSDN1WEQ2UlVXV2xVcjVlbjBiMWtwSUtzV2ZzWWlaaUVJRVgz?=
 =?utf-8?B?bjZsOUZLK21ZRTNFUGdxQkxWS0J0NUpaWEl0KzMySklFMTFWbWxVOTdiTzdw?=
 =?utf-8?B?QU54NDRyYjQ2bDhHMTJMSUNSdXhjaUFGMGQrV0hyVWt5RWZjM3NWcFl5a3Rl?=
 =?utf-8?B?U29GVUFFcmhJOVBpOFo5MTQyY283MHJEbStXRXVYNkZhRG5NYUw4MDg4cVpk?=
 =?utf-8?B?WmxFcndZZVA3ZDNKNjBxZElaWWtJL204dUVOS2NObjhiSHY1TXJVUTVNSG1C?=
 =?utf-8?B?bGZjQmRVNmY4RExVVG04K29ROGQrNU9uY2FvWEE3TThTR2pYMTZSejdmTnM5?=
 =?utf-8?B?U3IwTVdmMzRQd0QrdVBYOHZHV2Z1ZkZWZnN2VG96dmdISjJHVmdrdDQ3aDRs?=
 =?utf-8?B?N3VoWHkwTm82RnluekR2Skdad1RUblNVNWY2d0pCTjJZQ2ZWdUJMOUpCam1S?=
 =?utf-8?B?RmdKcWJEL3lXQ0Z5Smlla0pmRlVLRlNHY3BuM3BTb1Jab1VtYVhiMHhpZkFP?=
 =?utf-8?B?STRaKzZ5UHlaTms1K1NkektjRFBibFMwTUdEcGJVclpmNU95WmtudDRVYnF4?=
 =?utf-8?B?M25aamlkZmVZdWFrWnA1aWhpeVNsSGRiRVFGeVF6R0JBV3NwblR3Q0J4Ylhl?=
 =?utf-8?B?N25DQTkzUm44cHBUTC80Q3dNcG9Jc2V6Mk5uK01MTWE5czRkZEYvTGN4VnN0?=
 =?utf-8?B?S0NXSGtsa2E0eE42czFGWGxweUMwd0VFdHg2R3lOaW1ldHQzNTNKbmpMQmVB?=
 =?utf-8?B?ZS9ORnFpS3pTNXpjRU85SmdpY3JxRmdFWVh4SWlablAyWEVSdHhNek9WYmZV?=
 =?utf-8?B?QWhJeHRmVUJoZmlDUnNUUkdGQVV2VDc3dE5tU09TTkhoVjYvRjVQNFBxVWdX?=
 =?utf-8?B?K1c5SjZHNUxudmRpWlNWcktKdWtiTjRKSmdZcmp1T0tEdi9SUDNVQWg0U0Zp?=
 =?utf-8?B?bzY2WElPZTZRNFErSjNlczFVTHdTdTBXV0ovNGdWeHptSU1pSWN4cDNWS2Rk?=
 =?utf-8?B?RE9kcWNZdDYwczF2SFN6NUdQVFRvZzVmam81SDBrM1h3RkxkdlhUVHA2SVNT?=
 =?utf-8?B?MTNzcFRxN1ViNzNBSW9JV1lMYXBxQzc4NUQzeTV2dmM1TDJiV1BUak9WeXRz?=
 =?utf-8?B?M3cxMVpjcnAxT0RMd01oQnIwc04xY2d4RE52eTJjOU9mNGxuZ0lUYThYYUF4?=
 =?utf-8?B?ZFl1bDcwMWlVaTRSenpNM3BNcWI4VGY2VG9wNURWZ2dpODNUQXdvclhVYVBx?=
 =?utf-8?B?VHc0dmI2RWNLVEpldk44anJZUzE2ekNkWkkzdkE3RGFhajUvNSthWkxyV0ov?=
 =?utf-8?B?alRYOVgrWWloNW1HM3FGTEcvMFIwemhxdllxdHJPenZZRUFVTnpRWTZTUVJR?=
 =?utf-8?B?bU9VaWNlVWtiQ1RMZjVYR2FvMUZkSUxWbzR5ZFYyN1ZCSG11MmtKV2F4MnJK?=
 =?utf-8?B?RklnUCtMU1pwU2tDMnhCdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E91386C78129024C8804B7A0ADB233C1@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e06da1be-a0f8-4508-e1c4-08d976c53de7
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Sep 2021 14:46:16.8090
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: w2vBonkwRt2yVX0Z+ChtETxIPCOkxAFJGFNI7oewfDnoAvrk8draPNm3N3hUBK/6Fj76mAoDnvOTOLhiTFo4Ik4k/HwUJAE45qrCezA7EGAUlF4F0jNHA7HnJyN+30XU
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3715
X-Proofpoint-GUID: GbaD7ypfiHtdr5s7ogZXPd9cQdkEJTa4
X-Proofpoint-ORIG-GUID: GbaD7ypfiHtdr5s7ogZXPd9cQdkEJTa4
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015
 mlxlogscore=999 adultscore=0 priorityscore=1501 impostorscore=0
 malwarescore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 mlxscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109130098

DQpPbiAxMC4wOS4yMSAxNTowMSwgUmFodWwgU2luZ2ggd3JvdGU6DQo+IEhpIFN0ZWZhbm8sDQo+
DQo+PiBPbiAxMCBTZXAgMjAyMSwgYXQgMTI6MzQgYW0sIFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0
YWJlbGxpbmlAa2VybmVsLm9yZz4gd3JvdGU6DQo+Pg0KPj4gT24gVGh1LCAxOSBBdWcgMjAyMSwg
UmFodWwgU2luZ2ggd3JvdGU6DQo+Pj4gRnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9s
ZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4+DQo+Pj4gQWRkIHN1cHBvcnQgZm9y
IFhpbGlueCBaeW5xTVAgUENJIGhvc3QgY29udHJvbGxlciB0byBtYXAgdGhlIFBDSSBjb25maWcN
Cj4+PiBzcGFjZSB0byB0aGUgWEVOIG1lbW9yeS4NCj4+Pg0KPj4+IFNpZ25lZC1vZmYtYnk6IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+PiAtLS0NCj4+PiB4ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlICAgICAgICAgIHwgIDEgKw0K
Pj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtenlucW1wLmMgfCA1OSArKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysNCj4+PiAyIGZpbGVzIGNoYW5nZWQsIDYwIGluc2VydGlvbnMoKykN
Cj4+PiBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC16eW5xbXAu
Yw0KPj4+DQo+Pj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUgYi94ZW4v
YXJjaC9hcm0vcGNpL01ha2VmaWxlDQo+Pj4gaW5kZXggNmYzMmZiYmU2Ny4uMWQwNDVhZGUwMSAx
MDA2NDQNCj4+PiAtLS0gYS94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlDQo+Pj4gKysrIGIveGVu
L2FyY2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4+IEBAIC0zLDMgKzMsNCBAQCBvYmoteSArPSBwY2kt
YWNjZXNzLm8NCj4+PiBvYmoteSArPSBwY2ktaG9zdC1nZW5lcmljLm8NCj4+PiBvYmoteSArPSBw
Y2ktaG9zdC1jb21tb24ubw0KPj4+IG9iai15ICs9IGVjYW0ubw0KPj4+ICtvYmoteSArPSBwY2kt
aG9zdC16eW5xbXAubw0KPj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0
LXp5bnFtcC5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC16eW5xbXAuYw0KPj4+IG5ldyBm
aWxlIG1vZGUgMTAwNjQ0DQo+Pj4gaW5kZXggMDAwMDAwMDAwMC4uZmUxMDNlMzg1NQ0KPj4+IC0t
LSAvZGV2L251bGwNCj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5j
DQo+Pj4gQEAgLTAsMCArMSw1OSBAQA0KPj4+ICsvKg0KPj4+ICsgKiBDb3B5cmlnaHQgKEMpIDIw
MjAtMjAyMSBFUEFNIFN5c3RlbXMNCj4+PiArICoNCj4+PiArICogQmFzZWQgb24gTGludXggZHJp
dmVycy9wY2kvY29udHJvbGxlci9wY2ktaG9zdC1jb21tb24uYw0KPj4+ICsgKiBCYXNlZCBvbiBM
aW51eCBkcml2ZXJzL3BjaS9jb250cm9sbGVyL3BjaS1ob3N0LWdlbmVyaWMuYw0KPj4+ICsgKiBC
YXNlZCBvbiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMuYw0KPj4+ICsgKiBDb3B5
cmlnaHQgKEMpIDIwMTQgQVJNIExpbWl0ZWQgV2lsbCBEZWFjb24gPHdpbGwuZGVhY29uQGFybS5j
b20+DQo+PiBPbmx5IG9uZSBDb3B5cmlnaHQgbGluZSBwZXIgZmlsZSBpcyBlbm91Z2ggOi0pDQo+
Pg0KPj4gQnV0IGFjdHVhbGx5IGFsbCB0aGUgQ29weXJpZ2h0IGxpbmVzIHdpdGggYSBuYW1lIG9y
IGEgY29tcGFueSBuYW1lIGFyZQ0KPj4gbm90IHJlYWxseSByZXF1aXJlZCBvciB1c2VmdWwsIGFz
IHRoZSBjb3B5cmlnaHQgaXMgbm90ZWQgaW4gZnVsbCBkZXRhaWxzDQo+PiBpbiB0aGUgY29tbWl0
IG1lc3NhZ2VzIChhdXRob3IgYW5kIHNpZ25lZC1vZmYtYnkgbGluZXMpLiBJIHdvdWxkIHJlbW92
ZQ0KPj4gdGhlbSBhbGwgZnJvbSB0aGUgbmV3IGZpbGVzIGFkZGVkIGJ5IHRoaXMgc2VyaWVzLg0K
PiBPay4gTGV0IG1lIHJlbW92ZSBpbiBuZXh0IHZlcnNpb24uDQo+Pg0KPj4+ICsgKiBUaGlzIHBy
b2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgaXQgYW5kL29yIG1v
ZGlmeQ0KPj4+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1Ymxp
YyBMaWNlbnNlIHZlcnNpb24gMiBhcw0KPj4+ICsgKiBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29m
dHdhcmUgRm91bmRhdGlvbi4NCj4+PiArICoNCj4+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3Ry
aWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+Pj4gKyAqIGJ1dCBX
SVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9m
DQo+Pj4gKyAqIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIgUFVS
UE9TRS4gIFNlZSB0aGUNCj4+PiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1v
cmUgZGV0YWlscy4NCj4+PiArICoNCj4+PiArICogWW91IHNob3VsZCBoYXZlIHJlY2VpdmVkIGEg
Y29weSBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UNCj4+PiArICogYWxvbmcgd2l0
aCB0aGlzIHByb2dyYW0uICBJZiBub3QsIHNlZSA8aHR0cDovL3d3dy5nbnUub3JnL2xpY2Vuc2Vz
Lz4uDQo+Pj4gKyAqLw0KPj4+ICsNCj4+PiArI2luY2x1ZGUgPGFzbS9kZXZpY2UuaD4NCj4+PiAr
I2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+PiArI2luY2x1ZGUgPGFzbS9wY2kuaD4NCj4+PiArDQo+
Pj4gK3N0YXRpYyBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21hdGNoIGdlbl9wY2lfZHRfbWF0Y2hb
XSA9IHsNCj4+PiArICAgIHsgLmNvbXBhdGlibGUgPSAieGxueCxud2wtcGNpZS0yLjExIiwNCj4+
PiArICAgICAgLmRhdGEgPSAgICAgICAmcGNpX2dlbmVyaWNfZWNhbV9vcHMgfSwNCj4+PiArICAg
IHsgfSwNCj4+PiArfTsNCj4+PiArDQo+Pj4gK3N0YXRpYyBpbnQgZ2VuX3BjaV9kdF9pbml0KHN0
cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LCBjb25zdCB2b2lkICpkYXRhKQ0KPj4+ICt7DQo+Pj4g
KyAgICBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21hdGNoICpvZl9pZDsNCj4+PiArICAgIGNvbnN0
IHN0cnVjdCBwY2lfZWNhbV9vcHMgKm9wczsNCj4+PiArDQo+Pj4gKyAgICBvZl9pZCA9IGR0X21h
dGNoX25vZGUoZ2VuX3BjaV9kdF9tYXRjaCwgZGV2LT5kZXYub2Zfbm9kZSk7DQo+PiBUaGlzIHNo
b3VsZCBiZSBzdXBlcmZsdW91cw0KPiBBY2suIEkgd2lsbCByZW1vdmUgdGhlIGR0X21hdGNoX25v
ZGUoLi4pIGluIG5leHQgdmVyc2lvbi4NCg0KSSBhbSBub3QgZW50aXJlbHkgc3VyZSB3ZSBuZWVk
IHRoaXMgcGF0Y2ggYXQgYWxsIGFzIHRoZSBtYWluIHJlYXNvbiBmb3IgaXRzIGV4aXN0ZW5jZQ0K
DQp3YXMgdGhhdCB3ZSBjYW4gcnVuIFhpbGlueCBRRU1VIGZvciBaQ1UxMDIuIEJ1dCwgdGhlIGZp
bmFsIHNldHVwIGlzIG5vdCBnb2luZw0KDQp0byBiZSBmdW5jdGlvbmFsIGFzIGxlZ2FjeSBJUlFz
IGFyZSBub3Qgc3VwcG9ydGVkIGFuZCBJVFMgaXMgbm90IGEgcGFydCBvZiBaeW5xTVAuDQoNClNv
LCBRRU1VIGFsbG93cyB0byBkbyBhIGxvdCB3aXRoIFBDSSBwYXNzdGhyb3VnaCwgYnV0IGF0IHRo
ZSBlbmQgb2YgdGhlIGRheSBvbmUNCg0Kd29uJ3QgaGF2ZSBpdCB3b3JraW5nLi4uDQoNClBsZWFz
ZSBjb25zaWRlcg0KDQpJZiB3ZSBkZWNpZGUgdG8gcmVtb3ZlIGl0IHRoZW4NCg0KaW50IHBjaV9o
b3N0X2NvbW1vbl9wcm9iZShzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCiDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgcGNp
X2VjYW1fb3BzICpvcHMsDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgaW50IGVjYW1fcmVnX2lkeCkNCg0KZG9lc24ndCBuZWVkIHRoZSBsYXN0IHBh
cmFtZXRlci4NCg0KPg0KPiBSZWdhcmRzLA0KPiBSYWh1bA0KPg0KPg0KVGhhbmtzLA0KDQpPbGVr
c2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:52:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:52:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185746.334458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnJq-0003eo-An; Mon, 13 Sep 2021 14:52:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185746.334458; Mon, 13 Sep 2021 14:52:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnJq-0003eh-7l; Mon, 13 Sep 2021 14:52:26 +0000
Received: by outflank-mailman (input) for mailman id 185746;
 Mon, 13 Sep 2021 14:52:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xuNF=OD=epam.com=prvs=989052e8b7=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mPnJp-0003ea-6c
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:52:25 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 34a3464c-14a2-11ec-b3b9-12813bfff9fa;
 Mon, 13 Sep 2021 14:52:24 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DDxnx3027407; 
 Mon, 13 Sep 2021 14:52:22 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2057.outbound.protection.outlook.com [104.47.9.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b279tge92-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 13 Sep 2021 14:52:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3588.eurprd03.prod.outlook.com (2603:10a6:208:50::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep
 2021 14:52:19 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 14:52:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34a3464c-14a2-11ec-b3b9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ky9tQ3KiIZgtYcW4j0kyLF/cXcrsKAuV0Hc7M0COPMaN1aUo/pBiZtx9u9V207WQzUtYhcGTtDo+KTYtUp9d3cTLtyCtOO9Cs4DvZDaP6HVK3N0gBMaocClBv1o+n9JkomGV4RrpY8cazjw7xgVgdXhh2MGRM1jfHcsZXiwzFofDIMKP/6VR1kZMJ09AV6GI0TmqXcuUsznd5cHV83XMKVn5GJ274EuqnU3/X0Nn+vkw9VAw4D39Pe60y37wc8/TovKZ7c3B2r8NF2npF6UfKJA6LYCzDba2OQaO3zAs/zPj6EQhk9RqdVoGATyQ3/yGTHRfB/VOMuK/brSV2kyiDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0ldnG/GMbT4VccrgdRTQVXDGlqcsYzq98mP0a7Rai3I=;
 b=EMUAk3SHkmAa1gTVnFHvR5VNxFEE+nQTzXwAnNRQ3bEF9ndOXrM52kdAw8WpeWonMrisk2xQhT2QmDw4m+PqOTRFSFrdjR7YP1PGMZEagWrWYstarZueAjw9QW/+DmFEKpLuK5Lle7xbh0M7VxlfwbG8Uu6FbNUtadPu+PsdayQuB0Xq+b1TEwXE1o5MFULILJWC00w9SLgMTDfwqQBMel1rabqCMzTJuewc2RWbCW0f+V08+8NK43Tr5xDXNf+v+4fIiv2+FXGAPlH0KM8nQ6n0vqyO7QmFXO4iCfzpY2zUDx2trN1J7H3Fc2S15RKGJgabEWYI+PpqIkN8iN28mg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0ldnG/GMbT4VccrgdRTQVXDGlqcsYzq98mP0a7Rai3I=;
 b=P3nG/qiP+eps97pJzR5c2YOkjBe0TshBSma0roSN6x+fXZeq4GUHvmiGEAApbzyFIjaMYEVKIMevnzSXrLrJ/rGv6ApwEEtaggnspOU+pr27ophRevFPM9Hc5rr7bSsP0JvaH+OWU7V+6CO5FHOOVXqCgUpMunQBnPcwuvCSO5U916KOfH50F4M/Ud+m7IrK8Ta6LTdg4UM3JjO+2MD1mZ0wE6Kv+G2pMxUxTsDhTjY7cJ6MIgnaGMhsoN3EYfOKzJrL0RXbzzjI+x5R0gigADSH5AlsDE8Ttlb8/AQzAYYgh7JNaBEXji2cF8Fd4fonyD7D+wjjCstKxai0W1YIBQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXqK7zTKPAnBLuBkixTqpTrxO8Dg==
Date: Mon, 13 Sep 2021 14:52:18 +0000
Message-ID: <c9484b8e-cad5-d4e4-a25f-749035fe5859@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
In-Reply-To: 
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 647da388-103a-400a-ad2e-08d976c615ca
x-ms-traffictypediagnostic: AM0PR03MB3588:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB358859A020CA3123B27C0F38E7D99@AM0PR03MB3588.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 otq8j1HdBH52E8OIOkynUqkKQleEvke4jG7rcrRMq/bGgARAuNzVSxBm4OXte9Bq714V7b8i/6hTH0MyO2ObJOTYf7aDKOPo/lgcGyIjZzuuiCB7UD00WO3l++wiSXvxRPRpUm3jleVGIM3gPZ369JUNbx7iG0QYrHHVzl0OQ0SeLJ7bAInfmrMA21aF9E7AAK/e4UFeOudtJLj8Y0na1jCMI2Mvca2mdINjoqGVAUfiit4jcs4RzgIdefI+iyX2tVpfEKMgSnUsuXD6zlTutNMFqAZLVAhgPoI4YM7XEqxttsQ/e8Gy6G5C4pF7CPWENfs+d0n6/Ce+nuMPMOA08y7eayRdFW/0Hx5t9CybDKGoRtXIXabeyrrPW0xBHA4OtAyxtdJkH10RXf8yOKGcZ8+HuiDn4/UZtCc3M0FjZ4InRRexz0+qwMop+q3RMoBfxfrvL5UtyxZqPGMVZGqZDCK5u/RNgJLhx8NllZkyas/aS0/DHhA5cN3r0U1dHg8MHFwD71pnUjXNfMnxyItao6AVG65Pxf1Swc4LMSmrrBz3q0PkDdEnmqqkd/4MP+aYc+86Btd9s5+FHc2HDYy4QRItnewJ8GqrrRMGMiB1MCFR5O8w3cTiiL8lTESTZdaX8cEs9N0BHXw4J1k5tintSTEqmauvIeAKS9OqCMGcDoFnj72Ljhm9EvIGoZlgP3fswAnrqg0k2wLxuxKiKA2jb1ncQNPpdMh6BmAcj11aKj23t4HP5RXEQEKF1v4fUY13
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(8676002)(6486002)(55236004)(8936002)(31686004)(2616005)(478600001)(186003)(26005)(53546011)(107886003)(6506007)(38070700005)(2906002)(6512007)(91956017)(76116006)(86362001)(5660300002)(38100700002)(110136005)(54906003)(122000001)(71200400001)(66946007)(66556008)(64756008)(66446008)(316002)(4326008)(66476007)(31696002)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?U0g0cER3MnZWTHg4ZTNFZ3ZDWXh2RDlaWjZJaGJ4T1pOSHA1Zlk3UEpXOFpL?=
 =?utf-8?B?MXRiMkg0UkRKK3crTUJqVE5vbHQyN094Vk1nOVk1d3RtOENUK3lyY2RjWWx1?=
 =?utf-8?B?VVVsSVJkQkJoRk5iQktXaW1NaERtYjFlZ3I0OTlrQU1xTHdCeGFhOEx3RWQ4?=
 =?utf-8?B?d2NRcTFZeGpFOC8xTTIvT0tNM3pnWVBEeXNoazQyNXpUMEp3RjQwT1laTG4x?=
 =?utf-8?B?MlJDVzlFL2J2cUVFUmw3a01DZi8yczVYdXIyWU1Uc2RnemVOSk95dHo0bUt1?=
 =?utf-8?B?STh0VURNd1JKek0yTE5keUZMUFVJM21xeDM3MkN3VGhQRnhyVEhMSHl6cFNC?=
 =?utf-8?B?bXBFcWZBdDJzZzdidFpZWXY1SDlSejJyUHlFM0pTOGFSZENoa1FvOUlpV05n?=
 =?utf-8?B?ODgxR1huN1A1WkJBUGhTWjhRNEZtdkdUTmFKeE93QzJwQUJzbWltQStyRTdq?=
 =?utf-8?B?NWZxNTNJTVhCeWZUbXZVc3l5UitaWDhrUWdpblg1OWFSelFKcHNacmhJK2FR?=
 =?utf-8?B?NUlXTGtoeDRiN0ozNG5Tajlud0JLWUl5ZW1yYndERUFxRUZrbkFSdStaSDh0?=
 =?utf-8?B?QTF3ckQ3MHBpUUEwZGRzaEtnb2tGRmlDV0VCZWN0SWQwVVVuVEVkSEhRSkly?=
 =?utf-8?B?SDFweE5VMmFESUFhbDBhbjhCR3ljVTFZQWVHNnpORk1yOVZtYXo4cEFuUDhF?=
 =?utf-8?B?K0FReDNBK3RKd3BNM1gvNDFmZmpVWDJ4UnFrM3FjTG5lMVBMdFdaT2MvUHEy?=
 =?utf-8?B?ZVN5QXFLa1ZjZ3dKNUZST3gvT1dvRVQ5MGVJS2pUY3QyZ0tDNmVSMHhNNUlw?=
 =?utf-8?B?L3lKZjR5d25qekRzVW4vNm8xbU1PMExXSUlNZHdpdjRNaWNuaW9FZmd0bGxm?=
 =?utf-8?B?UDhoTVAvUW95Ri9CVFlTcnNzeTFmVTNiM3lFQnJOQ3Q1OEhiYVVlUEIyMzNo?=
 =?utf-8?B?ZHVYbEIyS00vWGlYUFZMbFUrWk5LZ0NGZzhGQlNaU0tNZ2VDWDMwV0NRdjdV?=
 =?utf-8?B?WlplYzJ4QS9DbVpEMjRGb0VJRjQvQmpWSXVVM3RtNWVidi9LaXgyOFhFcWVk?=
 =?utf-8?B?ZkxISEJFaDUyQnVTUDRaRWFia1YrYlBNU1JpRXhtR0ZhRDg0MFFrcVZSbUJ1?=
 =?utf-8?B?aTI4bi9vRm50eXVHR1BuNFgxWUZBNUlKdnhCVEhhNm52UTY2cGtCK3N2b1My?=
 =?utf-8?B?bWF5TUErdHpIZE84N1lPenRacnlnV3ZvR2UyMWkxU3JxWmFtV1pFVjhqaTky?=
 =?utf-8?B?VHRKTlJPZ0NkaHUwMDFQSnBRUWFwZ2crQXlpRjgzWU1JOUJyTTljMjdPdE5l?=
 =?utf-8?B?YXhDYkVrN3JJeldFQlFVVUNHSXFVMk9QbmNwdy82S3h0WUZoVzZ0OWp4cElL?=
 =?utf-8?B?N1huL2U4bDZQTDExVWtLOWlRaEJqUnp1SVdmLzVrNFo1Q0laWEUvVHBya2xY?=
 =?utf-8?B?U3l4aGFXMkMxMGM2aXhZQ0lqbUN5Q1RpZ0dOL3podWJVYmw1NG1aRzhsSk1p?=
 =?utf-8?B?d3dDM3B5YUxKWC9EaXBNblljL2tnMXUvY3N4NC9zNEFvL0gyMUpYY25LamNl?=
 =?utf-8?B?TnRHVjhMWDRpbElINDgyTHRjV1VWVFVPOGhkVG5hNUdvWGcrRGdkajNGS2RV?=
 =?utf-8?B?NVNNRkVmdnNSaEQ5OGVLSEE1bjcxV2o5bU5OT29Gb1gxdTdKclp2TmdIbUp1?=
 =?utf-8?B?UmxYaFFWN3BkMWlvclBLNVhGbHVkOERkb1NhMmR3a2p3cmFCYXMvdUg5MXM4?=
 =?utf-8?B?dThpMkFscjllZlplYlNyY0lON2RCRkR4Nm5xdVlQbnZtbUI0QUZyeFYwWGha?=
 =?utf-8?B?UWpLUjFtZCtVVjNKRXg3QT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <067FF25794C5E24989DEA2554A0D9548@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 647da388-103a-400a-ad2e-08d976c615ca
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Sep 2021 14:52:19.0254
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ImIyrux7fW54OZcQZxvha7GPNItiWiEUQQZHvSat+NiNJAuaFbzJRxMqVnWo+fACmHzjIDRKGv4ROncnWbMOWfvsvV4h81fD82jitxAPTRmT4KRqjG+g7eWssCkxTg94
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3588
X-Proofpoint-GUID: gXodqNg7eyifdCn9Nm136iFSUQ8wSwj_
X-Proofpoint-ORIG-GUID: gXodqNg7eyifdCn9Nm136iFSUQ8wSwj_
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 mlxscore=0
 adultscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 malwarescore=0
 phishscore=0 suspectscore=0 priorityscore=1501 bulkscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109130098

SGksIFJhaHVsIQ0KDQpPbiAxOS4wOC4yMSAxNTowMiwgUmFodWwgU2luZ2ggd3JvdGU6DQo+IFhF
TiBkdXJpbmcgYm9vdCB3aWxsIHJlYWQgdGhlIFBDSSBkZXZpY2UgdHJlZSBub2RlIOKAnHJlZ+KA
nSBwcm9wZXJ0eQ0KPiBhbmQgd2lsbCBtYXAgdGhlIFBDSSBjb25maWcgc3BhY2UgdG8gdGhlIFhF
TiBtZW1vcnkuDQpbc25pcF0NCj4gK3N0YXRpYyBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmdl
bl9wY2lfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4gKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnQgZWNhbV9yZWdfaWR4KQ0KPiArew0K
PiArICAgIGludCBlcnI7DQo+ICsgICAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmc7DQo+
ICsgICAgcGFkZHJfdCBhZGRyLCBzaXplOw0KPiArDQo+ICsgICAgY2ZnID0geHphbGxvYyhzdHJ1
Y3QgcGNpX2NvbmZpZ193aW5kb3cpOw0KPiArICAgIGlmICggIWNmZyApDQo+ICsgICAgICAgIHJl
dHVybiBOVUxMOw0KPiArDQo+ICsgICAgZXJyID0gZHRfcGNpX3BhcnNlX2J1c19yYW5nZShkZXYs
IGNmZyk7DQo+ICsgICAgaWYgKCAhZXJyICkgew0KPiArICAgICAgICBjZmctPmJ1c25fc3RhcnQg
PSAwOw0KPiArICAgICAgICBjZmctPmJ1c25fZW5kID0gMHhmZjsNCj4gKyAgICAgICAgcHJpbnRr
KFhFTkxPR19FUlIgIiVzOk5vIGJ1cyByYW5nZSBmb3VuZCBmb3IgcGNpIGNvbnRyb2xsZXJcbiIs
DQo+ICsgICAgICAgICAgICAgICBkdF9ub2RlX2Z1bGxfbmFtZShkZXYpKTsNCj4gKyAgICB9IGVs
c2Ugew0KPiArICAgICAgICBpZiAoIGNmZy0+YnVzbl9lbmQgPiBjZmctPmJ1c25fc3RhcnQgKyAw
eGZmICkNCj4gKyAgICAgICAgICAgIGNmZy0+YnVzbl9lbmQgPSBjZmctPmJ1c25fc3RhcnQgKyAw
eGZmOw0KPiArICAgIH0NCj4gKw0KPiArICAgIC8qIFBhcnNlIG91ciBQQ0kgZWNhbSByZWdpc3Rl
ciBhZGRyZXNzKi8NCj4gKyAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJlc3MoZGV2LCBlY2Ft
X3JlZ19pZHgsICZhZGRyLCAmc2l6ZSk7DQoNCkkgYW0gYSBiaXQgd29ycmllZCBoZXJlIHRoYXQg
d2UgZG9uJ3QgZ2V0IHRoZSByZWcgaW5kZXggZnJvbSB0aGUgZGV2aWNlIHRyZWUsDQoNCmJ1dCBm
b3IgZ2VuZXJpYyBFQ0FNIHdlIHVzZSByZWdbMF0gYW5kIGZvciBYaWxpbnggd2UgdXNlIHJlZ1sy
XS4NCg0KRm9yIGV4YW1wbGUsIGZvciBYaWxpbnggd2UgaGF2ZQ0KDQpyZWcgPSA8MHgwMCAweGZk
MGUwMDAwIDB4MDAgMHgxMDAwIDB4MDAgMHhmZDQ4MDAwMCAweDAwIDB4MTAwMCAweDgwIDB4MDAg
MHgwMCAweDEwMDAwMDA+Ow0KcmVnLW5hbWVzID0gImJyZWdcMHBjaXJlZ1wwY2ZnIjsNCg0Kc28s
IHdlIGNhbiBwYXJzZSB0aGUgcmVnLW5hbWVzIGFuZCB1bmRlcnN0YW5kIHRoYXQgdGhlIGNvbmZp
Z3VyYXRpb24gc3BhY2UgaXMgdGhlIGxhc3QgaW4gdGhlIHJlZyBwcm9wZXJ0eS4NCg0KVGhlIHNh
bWUgSSB0aGluayBjYW4gYmUgZG9uZSBmb3Igb3RoZXIgZGV2aWNlIHRyZWVzIHByb2JhYmx5Lg0K
DQpSYWh1bCwgZG8geW91IGtub3cgaWYgcmVnLW5hbWVzICJjZmciIGlzIHZlbmRvciBzcGVjaWZp
YyBvZiB1c2VkIHdpZGVseT8NCg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 14:53:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 14:53:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185752.334470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnL9-0004Go-MQ; Mon, 13 Sep 2021 14:53:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185752.334470; Mon, 13 Sep 2021 14:53:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnL9-0004Gh-JE; Mon, 13 Sep 2021 14:53:47 +0000
Received: by outflank-mailman (input) for mailman id 185752;
 Mon, 13 Sep 2021 14:53:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPnL8-0004GY-SA
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 14:53:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 64aca1a8-14a2-11ec-b3b9-12813bfff9fa;
 Mon, 13 Sep 2021 14:53:45 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2055.outbound.protection.outlook.com [104.47.9.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-phxAIHv6OKGQtJSZsWWlvQ-1; Mon, 13 Sep 2021 16:53:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4846.eurprd04.prod.outlook.com (2603:10a6:803:5b::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep
 2021 14:53:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 14:53:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0035.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.8 via Frontend Transport; Mon, 13 Sep 2021 14:53:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64aca1a8-14a2-11ec-b3b9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631544824;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0BxbF0wOkqVuizTCkYCnWGEuxFb+oTJxIc/0InreWG4=;
	b=RYocRuYNmeZfG+a/7uh7b1ABlQqEvwswZFJAc0YZeTGbPECEdM5sTfuqx8AoDjvdYisSJo
	lXfKp06cDQf7X3hlvcAUgjWBEOlC3S+Gf/dZy9Peeo1wjSWMrulfTINIvkX3kABatWmk6k
	m3VJj0D53S/2CWm4XAqz93cY+Hl0mAg=
X-MC-Unique: phxAIHv6OKGQtJSZsWWlvQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jme+vsoZ0xXfVZHNJ0wvIG735bSu2yXWQOjWrLO+irL2OQuX2gIGtlX8J/H7m7/vLC1XZDTKZevj4iK5MafwzUWhjqWziRwH4uK2SA0GF2LDlixuEkvYWh7zYQp87t7em6/cPDxbi3LDsFzBQE9QoUO+IyiHG56+VXx2BZQMkNK0JRgZ92qZhA9lskkQywHl/wECcnK4+3MBaJvVrFR7zYNgRC2T3p11QoR8w8MkCXDxt/QyAvUZdeer6rQXnuPSSAxv0IzbusqstoVMn5bjFv5PjjiTHnbm6RGK+xnVuSKxnAer4fwsuoERr6rQh6vDtvaMuQQMHKiJsDOo1TQ18w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+CL2492WpoF54KuVMrvjinzjgufY4hDJluNqOO7t6fc=;
 b=Xb6ZRqRJMYcys3TxEKFYk77qGQcED5wre9AkGalm0O8a4ukGf3vAP/JDSLwdhQS5HCtrym9PeTS+efvYYANDiRvJOhdc2K7Aw0+t3LUrh0nIS80HYKk2ws8IPoaHqzty45UJbOlWQC8+4OzeNJWP6czg1RH/0Pd0qR4Ndczz6k+EAmNsz6YICPg+b60hTr8Aj5Kivw2ML+ZbUjkioqAP583JwytURPcRZ5QLZRx6QplBSbO5lz+byUvki8rdm3srr96nLFnAsjgOB6mLWm877gGW1R8gejTwgYD1DxCHd+nTRVPWr2UOqn8QXdz1tBbeiCUtRl4SRWau3A0ztONvOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9eL7b7A8CirzOa@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <25560e35-ad35-a775-7b38-284251b8a3b1@suse.com>
Date: Mon, 13 Sep 2021 16:53:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YT9eL7b7A8CirzOa@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0035.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c115ac7a-ae0f-4c70-e996-08d976c646f5
X-MS-TrafficTypeDiagnostic: VI1PR04MB4846:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB484691D1F002150AE4415563B3D99@VI1PR04MB4846.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aCAklCfx5k7IPF83hx5qCkKsO70Y/k/MvVe8V08lzKW2LnR1n+oGrMa7eGMVUdS/gZFcV1qed9zV3SsdnCruQvbWBNOPrwi3tOgO5mGikjCxvnySt0Zr6dmxCdZVVoEPyp2CQEW1HH4pqLZz63NdX5OY0Q096qhn4AUIzbw7sXRZpMy4UhaU9RNQgzXdEuY+n57eAjAuUX7QcaUNFuQ9MJ8FGiw/IEayd3Bv1qBWY/o1UPjCO/uaB2q0UB8d9BSV+pOvwAXJO3ke/qOHmCWXMCZO426H14W0xJFhklNdydGuqM9CFymeSBkl4D+jo+xRhrmtEHDSWCd8kFMCrrZm65Qyb8jk0TFcvzsO53/MFE8EibUWCiXOK4Wc3qbUZl8xu9y3RopQygK8aVnpbOmFE2zWZArMxxUil4K6/D9ty7SN8kOHVAc9W9lXAOpfhlkp+wGqbhaW4IaU48gbdFv5jFbg5Jc4OnSUshf5U/G1eExeYISc+arJQOuOHhzb44/eRl6nZjOGxIE88VtCXQMRoErNHjj7Eysu3GOi7/EfK6rxouxRTHgW+eiBUBqnd4lPlIPe0hcJrEzgwha8GKaamZlSLKeCgfPedeDHjueFDeiwvpjWQVaWjKW1sgdY3w2oYaPO3IASwM7rCUkH8am7NDsEFYTqPx3tuhwXDP4n03QWYRhu36VMEwBZUe4zgyDV4E2+NODA7O7g4O60N3hG8A1Z2bykX66e+hj9RvDrtRKhiqkXOTFT7tP4WCkj3TFf
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(136003)(376002)(39860400002)(396003)(956004)(2616005)(66946007)(4326008)(6916009)(5660300002)(8936002)(36756003)(186003)(6486002)(316002)(66556008)(31686004)(86362001)(53546011)(4744005)(31696002)(2906002)(26005)(54906003)(478600001)(66476007)(38100700002)(16576012)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?pxETmIzGf/qrDanq0byO45QuroIsqcrv8R2Fva/kjup8VhsqPAXuqLlHUWo0?=
 =?us-ascii?Q?Q9IKq1cq1MHP702cjwgCv9F+BJJLnJpC7ylxpk9mbSCmNatVwELA3otuTUB0?=
 =?us-ascii?Q?VfrnEXjmnT61kbYvgHOhX7bO+dsh2goPbtHZrTLIDWZPENNLugb2WZIGvCGf?=
 =?us-ascii?Q?WM8zRs1DZXFL8pYhVUNuazDfhbEOsecnkKdDtlMAXZjqiRx+uIv+lQJ/jh8s?=
 =?us-ascii?Q?onhWElDAEW690vGN7VnnfZZ7JZrRr0N9FfXzkhFFN0a1bMXdjht7T83Sd2Bb?=
 =?us-ascii?Q?Gca9y9FIaXp158NhLHdqoFckjYqvuaw1xpdWgzMtfwYipVXlqfSZiB+D77w4?=
 =?us-ascii?Q?KyNOAkxnjKWp/eWm+LcTeQ8SihrEBRJCnsznc3alJ2+PC7gMdaYx+gIrWRoi?=
 =?us-ascii?Q?7lq7jhmTuzODfMckLsBLH1G5g8dX7mkS/xx/h/Yp3I8LybhfeuBW4+nTCrYb?=
 =?us-ascii?Q?4ZPQeZOXogAt93pcM5GaX6/I5eMTejkXWwW0kulVRpBeUVgHh3yoxKqEqXBE?=
 =?us-ascii?Q?bch/MqCk45jtW8MkIXd7t9KMuJlNxU8h1oXGNbCiLc5Y0LgjnjKjQYfn6UfP?=
 =?us-ascii?Q?AT1rpZ++r2OY4hoe32YYJCcvEC3K3qusXLjlP3b1jv41arKfz+4hdiaSrMve?=
 =?us-ascii?Q?Ynhu8un2laJDeOLSujmL51MdHdiwEcU+TS7Fyz/up1CTkIg8tknJumg/pzVP?=
 =?us-ascii?Q?qY8Avxz8qpzKZEgi6pdG7W5B2aueiNb9TST87fta15JK5siE/qDbBgYKvavM?=
 =?us-ascii?Q?t8mrqbQzYuM/r4eWhIYtwIJuoa7USbBc8WKRzqWJtaKOhBtc84TdhYamEv3P?=
 =?us-ascii?Q?zfDuMsshHx8SivA2E0NSnrPRrni0BW7vVATwwYR0bBlIhR5uM+Jz7Gf8428w?=
 =?us-ascii?Q?z7CTYcdl/gagfhS+RyjmRu9xc8NzDkwxEhWbPwg4heEDDIx5NN7tGTEkuxYx?=
 =?us-ascii?Q?I1IaACyVScnkZSkD7cI0m4tB2dFG/8Dx+GYWTrlNbKRwszc0DLUPdXtfMDeQ?=
 =?us-ascii?Q?sO0gpzrV9YX7N0lFtR+GoILUCsqh1rNEXVkCFXKSfhBUbQ7jD5rS4yVEL4f6?=
 =?us-ascii?Q?hN5xdSQeF8qsY+K9CI2uwF/JoW8gF5Of/9lM9QPjpru7Fh2scaYmOyVflxMB?=
 =?us-ascii?Q?oZjrLwhcqq96E03QdSLdam+bmEo+ZCsUtXgKqOEyl+ShskTCrCOKL0QxdXUE?=
 =?us-ascii?Q?wGwCM34rTFrCO5GBgkmUT9+t7Ygx4Ph8SgzgnGTuHPoLhpcxTRbi87Hh7DTb?=
 =?us-ascii?Q?RHJXXFubZBynWxc0eGHvNb/znHIk4neStzwhks4A1++M2Xe2cbWFDjOSwKzz?=
 =?us-ascii?Q?btso2r18m03u7+Di9x7IS9PP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c115ac7a-ae0f-4c70-e996-08d976c646f5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 14:53:41.6868
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lgtKuuVCp0Suoyl9vGyUp8DvbhhzfMo28Hji61qxfV8TrCCpIJzOQRcHfvZ3Ut6sTEMt7AVT46EKzy9AN0O/5A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4846

On 13.09.2021 16:20, Roger Pau Monn=C3=A9 wrote:
> On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
>>  .PHONY: clean
>>  clean::
>> -	rm -f *.lds *.new boot/*.o boot/*~ boot/core boot/mkelf32
>> +	rm -f ???.lds *.new .check.* boot/*.o boot/*~ boot/core boot/mkelf32
>=20
> Forgot to mention in my previous reply, but what's the point of using
> ??? instead of just typing xen.lds?
>=20
> AFAICT there are no other .lds artifacts, so doing a 3-letter .lds
> wildcard seems weird.

efi.lds is the 2nd one to match.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 15:08:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 15:08:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185761.334481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnYv-00063C-W1; Mon, 13 Sep 2021 15:08:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185761.334481; Mon, 13 Sep 2021 15:08:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnYv-000635-Rc; Mon, 13 Sep 2021 15:08:01 +0000
Received: by outflank-mailman (input) for mailman id 185761;
 Mon, 13 Sep 2021 15:08:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPnYu-00062z-Hd
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 15:08:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 61acc80a-14a4-11ec-b3b9-12813bfff9fa;
 Mon, 13 Sep 2021 15:07:59 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-4MxS4ACeOqKJmFRT-yI_rg-1; Mon, 13 Sep 2021 17:07:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2446.eurprd04.prod.outlook.com (2603:10a6:800:4e::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep
 2021 15:07:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 15:07:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0023.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:17c::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Mon, 13 Sep 2021 15:07:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61acc80a-14a4-11ec-b3b9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631545678;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PvBaG6x2aZGACj1zGyUtNmCWIdmB40LsCt+NsRj9sVw=;
	b=iq59B2fBGX+VHrb5g1RabLxrQVs3Pm4raJh3NXJRX/UIymmoO7L89XoRHXhtp3nXbcNH6+
	Ua2LRhqmuPuhlbDVubUT3iP6TXaQwC1GIpZPB2z1aYK+Y+2XGlG0wHltOT5ZwiRqUPqXSr
	OKDkN2Y4IJ6ag6I66KtnHP2+EjxXwPQ=
X-MC-Unique: 4MxS4ACeOqKJmFRT-yI_rg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HwqD6sgUMvO8BMjIuSiaYlECZAM1S+it9IzMCfVfLQPkYo8pr13ZI1UVT6g2Mnb2BcdgBbiFkSM4WI9oRxcJgr4kqGIH923WlSFNJs2RG9w0SgGaIUi+OGfTCqFpdYVk2HsJQ4RVTcRqYcCO/AyDl8tYxB/YeY82DObx0+bFFsoEo/dZid8OqT+bn25mQJB2oOPdxF7yGTtWawBXoFAbEEl2aKOVxwUt1eX9FIwNlRWI1u5vUVq4vdL8HmOUwXyYXUvgBm8dNZFchQ2cbMmFyeQjg9WDNojMyx1Zr/QYRG4us9RBcQnHVOiifoWDOIiGvv/lhWVai3NAHpB6Jwo8uA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tDe0mJCFga/oCmqWClD56ZdtWxHgiXlSThWmhIwVMYQ=;
 b=BOGGsbuorCQq5WJKWUOu9h03qFGhft+vpMIyV4j1KU6sHi8Mmm5+wl9KO3DMCliLPmjLRfcZDdCi51ztDtQJ8Zz2+zQJpfTJtEfjN/ZVzlsvyF8cwNRp4xd/WTkdfulUbYa4ugWYZspqslm4bUSEjbvbiKoq0Xfe/6SSMFQqKpza6hTg7KwajAxqpJgGhCwVpfOryGHDHT/WaaFOfWbJKHsmfwEd//fJgxjPLOmdLx07wl7O7lqOE1iyTCAnJBujy8BaEJuEp/79OQyokXFzRFXGAKmU6JdTVZwcas2/iR3N+C94n1OHQeA0eaXQK0x16WL1Xu1DhkW71EsUKA5F1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
 <7c97de87-9687-a088-406b-33faf037244a@suse.com>
 <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4c8ddad3-e01f-06b0-1544-cffb477641b5@suse.com>
Date: Mon, 13 Sep 2021 17:07:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR10CA0023.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:17c::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fa81cb86-25b9-457d-a77c-08d976c843fd
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2446:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24460D319A6EBEDE40EFDE89B3D99@VI1PR0401MB2446.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ez5bliPYkNYLPwjPBsTBjDWrjkQ+HNMM3kigdHXsKObWHGbV9iS6iO8pHMNjiamXFXVhDEGtXcPfjTDYtB6L3T1qzFxLKYkIlsRdqpqFeTzOxdMZQ4/GEn+b7DwTLLu01YyGZ7PBj0iGHqN1UUy3/usu3HSEBRdKqeszuMqprNfDOzDFVent7W4EAH3NajZGse6wb/zt/c+cQwTdQcMGWxB3NTm4qIyUxPvBXMNDy7/bb9fbkdpQE9fILL0Xx59LU9EFJ1/H7DYbxQjiCNmX2nFxXk+XGkrz8kyRFEbiGdql9VZdsxoaxWyRJOj7PhurBurCOy1VdzmDPkI3YEW3Ng43iel9g+dXmJBSjO4hWghRKKAhvYurHpi7+LufUj8xw/C0l7uRTQr7QiNnpolFW3jPFjvHLPtF6leathzA2l1ggR/rmHqTtR/bI5PDQbhSG+grUJ2k3ppxhSimJb9SE/I/tHqNGLsYCeTqhcSHHAEV1I62NPHC7X6bUB22Tq8ZYski9nez7fOKcvkZFuMBaq16uqwyRy7pZXXWrG+rrZ+cxsT8BXY/DXu/pLjTQayltFs16R24gLNinBLp8e5IipeE6aLVHLR3hFO2iVJhJ27ZtFh7e2HJ4G7wKfQxa2J5Al0chcJcARKlcPwGh4qRyPQDugMo9SC/aRv1ZWpXHlP2GL/AzpQ1TcsXVC+YDuB4pXP8DPXUAzdb/ZL0OtexU+3Xhp93wpvGWQ9hZOHLLLaFjsjl2Z+PkrWC0TzspjhG
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(376002)(39860400002)(136003)(396003)(16576012)(8936002)(26005)(186003)(66946007)(53546011)(316002)(6916009)(38100700002)(36756003)(54906003)(66556008)(66476007)(2616005)(8676002)(31686004)(2906002)(5660300002)(83380400001)(31696002)(86362001)(6486002)(4326008)(478600001)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?JDWOMlMPFQZPwGF7fWJ7yiKnXUnZe/8ZoRfpr2DLm81Tf3yh5SRLJjlifZ1T?=
 =?us-ascii?Q?2zvlIwEMNNBGMY/5p1donJJFuIr2bXdMddcc7FbFTPQEcH8wZ7KQ7+nxaYKD?=
 =?us-ascii?Q?8R5NvxmjeLq1yKbpi3mlR3DzsR/OgBWMbJYrX1Ucxn9v9vlcRcVU2meF5UXc?=
 =?us-ascii?Q?0edFG3bEZPhoQGvN1vR1y8V9fDtVxs5RKcWicAX4ntIndzbxtDikPMrUPy1m?=
 =?us-ascii?Q?czBysaTk8rQXnZm5OiC6BGxwLzFsRSnTF0Fzx921ukEVlGF34zTKFF+IvIz2?=
 =?us-ascii?Q?9XSirnMAa18VZr23MlKODIWNvwcOvi0H3m+yybjMbqkK8mTO/wCMTy/HMuEw?=
 =?us-ascii?Q?TQ4+DPEfpc230M5SI9YV+F5Xnymix1MyRrrOf1PtJH/mKKt4SpvfTYVggGg8?=
 =?us-ascii?Q?2fJ5kwugJ15lcKXzK9f5Xwdz/d7dqnpLWcB5p7lPJaBL8z7/HMvGbwqbdfzU?=
 =?us-ascii?Q?VMcfzqTGyw7S2qrHy2Inrvy91Gut5+RDMnBUL482/Vuh/a1JtOaCSSFZc5jL?=
 =?us-ascii?Q?96zDjMQmjU4eCoSMFZVLHprMhGTyG8G3tTuVl6JVJsgKzRcboqkEpNONsPJ6?=
 =?us-ascii?Q?sH/HvFfNWRqDK1z7avbINWq4lNsNBHNOP9DRjoxHybodgNr0Pt/t/V3biVaw?=
 =?us-ascii?Q?PwrughehXjiGCCnRlA8sD0WrPfaw4nkBn+ZeoBvVbSX8mPHY8X1YMSf9omYR?=
 =?us-ascii?Q?kOHr2fdOYD1XERBJRLCa2V7fRp8FcNkwMoQPPyFn4U5aoQSC7UljMoWw/9bs?=
 =?us-ascii?Q?c1RAGN225oxe1LkKyh/EykLEfuGOWwNM6MmtR9peHJzGM2X6CnjlXodsUSo5?=
 =?us-ascii?Q?lhQAi4/HvPkYZBft6HLeMdtL4Uizrtg/ou5+rFv/HvIqGEI0KcgUm519+UYd?=
 =?us-ascii?Q?FFmjBN3NL4uP9aECSPhMpH5mkeEC4avME+67saRFHTJudDjAjDNM69f2GdmB?=
 =?us-ascii?Q?88q19KJk8X096K30FiHKj+aXLN6ZVLvQNYsUWtWRlgUi0BLxs1Bv4BEglRzu?=
 =?us-ascii?Q?U3HNCxGN3hXfYTVISnhPgfMTGyHPhyCgBGwVETqZIf0DnT+5/9e2yIKsGneK?=
 =?us-ascii?Q?3NeM15//RsQjNdQMgwxm3ooqZsPa+mnj4oIkPo7b8gP67Q+JOQGPo4cdFEYM?=
 =?us-ascii?Q?vysgCDmRjx16UFH+ZVlo8FeFqGMggiK/h3aD61fNK4T4Cu2DZCziya2kCPQe?=
 =?us-ascii?Q?8zla8HVY3hTpi64PM8gb/BGO3R70Jl1Q3osSeu+ZYW9flGzz0OZGD9P6gNLh?=
 =?us-ascii?Q?SinvjCPeMUA0LTqa43ZmSZR85YpDk5JMgPiGZXSFCyBWi7JUFeDupquFSlEU?=
 =?us-ascii?Q?pp8fNaBj9p1cvNBuONiQnnVC?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fa81cb86-25b9-457d-a77c-08d976c843fd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:07:55.7088
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZgyDwAexdoKk1YqHAFAdrjqkzuIhbmGZKzyN16tMTXhGK5rnytZwH8L3KwOAyfvrWci9C3jOssQIlBxnjJqtRA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2446

On 13.09.2021 16:33, Roger Pau Monn=C3=A9 wrote:
> On Mon, Sep 13, 2021 at 04:05:15PM +0200, Jan Beulich wrote:
>> On 13.09.2021 15:37, Roger Pau Monn=C3=A9 wrote:
>>> On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
>>>> I haven't been able to find an environment where I could actually try
>>>> with lld (ld.lld); all testing was with GNU ld (ld.bfd).
>>>
>>> Thanks for fixing this. I've been able to test with LLVM ld and the
>>> workaround is fine.
>>
>> Oh, good, thanks for trying this out.
>>
>>>> --- a/xen/arch/x86/Makefile
>>>> +++ b/xen/arch/x86/Makefile
>>>> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=3D
>>>> =20
>>>>  ifneq ($(build_id_linker),)
>>>>  notes_phdrs =3D --notes
>>>> +# Determine whether to engage a workaround for GNU ld 2.37.
>>>> +build-id-ld-good =3D $(shell echo 'void test(void) {}' \
>>>> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c =
- 2>.check.err \
>>>> +                           && $(LD) -T check.lds -o .check.elf .check=
.o 2>>.check.err \
>>>> +                           && echo y)
>>>
>>> Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
>>> then use is here?
>>>
>>> We already have compiler and assembler checks in x86/Kconfig, so it
>>> would seem more natural to place it there.
>>
>> The question of whether to record tool chain capabilities in .config
>> is still pending. I'm not convinced this is a good idea, Andrew keeps
>> shouting me out for that, and an actual discussion doesn't really
>> happen. Yet unlike back at the time when I first raised my concern,
>> Anthony meanwhile supports me in at least the question (to Andrew) of
>> when such a discussion would have happened: Neither of us is aware,
>> yet Andrew claims it did happen, but so far didn't point out where
>> one could read about what was discussed and decided there.
>>
>> For the few uses we've accumulated I gave (if at all) an ack for
>> things happening under some sort of pressure, with the request that
>> aformentioned discussion would happen subsequently (and, depending on
>> outcome, these would be converted to another approach if need be). I
>> have meanwhile realized that it was a mistake to allow such things in
>> on this basis - the more of them we gain, the more I'm hearing "we've
>> already got some".
>=20
> I see, that's not an ideal situation from a review PoV, as we don't
> have a clear placement for those and that will just cause confusion
> (and more work since there are potentially two places to check).
>=20
> What's the benefit of placing the checks in Kconfig instead of the
> Makefiles?
>=20
> As I understand Kconfig is run only once, while the Makefile could run
> the check multiple times.

Right - as many times as a directory would be entered for building,
times the number of evaluations of a respective variable.

> The downfall of having them in .config is that .config could suddenly
> change as tools are updated or as it's moved around different systems
> (yet .config already contains specific compiler version information).

Correct: Tool chain specific information may get updated, but then
further options may get silently turned off. Plus to update tool
chain specific information there needs to be a trigger to invoke
kconfig in the first place. Merely relying on make dependencies is
not enough there. Iirc we don't have any means in place yet to
actually enforce this even when there's no other reason to run
kconfig in the course of re-building a previously built tree.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 15:11:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 15:11:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185768.334492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnbq-0007Sr-Gn; Mon, 13 Sep 2021 15:11:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185768.334492; Mon, 13 Sep 2021 15:11:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnbq-0007Sk-Ce; Mon, 13 Sep 2021 15:11:02 +0000
Received: by outflank-mailman (input) for mailman id 185768;
 Mon, 13 Sep 2021 15:11:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPnbo-0007Se-L0
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 15:11:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 011c690e-9a43-44a4-a732-f5aab73ed942;
 Mon, 13 Sep 2021 15:10:59 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-DJ4C9hkdPYy5QWpvKEyCmg-1;
 Mon, 13 Sep 2021 17:10:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4942.eurprd04.prod.outlook.com (2603:10a6:803:59::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep
 2021 15:10:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 15:10:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0230.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:10:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 011c690e-9a43-44a4-a732-f5aab73ed942
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631545858;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=x0Nix/BQty3d4hMsYGdJoWyAd7PiBki7IBpOlUEVDe0=;
	b=ckEyi77AUO4wS9DalKaGZGEHWb01V5TKucVgMofcfyxRWKZCVEOKGe8AvStrmCSOKrHbXm
	NKsKzuPMJ2b4w1tG2uhBTM3z8MfgIDR69S0lAu2EH/aeAAFFrVRow/nnxcnEVcwbG4cKIA
	h6C7NORh1zJEPN6jDjVR61d9ImnOIvs=
X-MC-Unique: DJ4C9hkdPYy5QWpvKEyCmg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JkOhk5Dn296LSPUGuiRH4SHmk9LRIgkQLY2WkXSesWJu9wTRPtbAzblCqw8sDccD30xvhbqejeUn9c0avgc4lfS0syIN6aTsire0nftMeT0Vuyam/UTXvBwckINlGcxyA+F3oPP0w5d69IUdsh1du6BP5lZOoMWjosGOtHWd7MVtLiSEybbad1bO+OqQgxNRD65XlWwFq5VUrwZp5i1T15qUB8149WY1u/5KltM9/RaSflGMfxc4xJMxysiwUdO/JGixPgQY+SCX0LY4XgvUP4UOZY8RUX36Mr91uPBx+KVb6g/PEYzg2uk8XXPK/PO0lxJsOJUcFabMS8bbp5kbZw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rMX6OAGq30Sj46LFCENaQfupOkDxkPlF1839vkmC/+Q=;
 b=ZzYGWi6bCmzhtVMWCWWyTUAkUXcT+5keo/npTvKT0iTuyxyCQrXbe7MQiXe2JyQGCwYVmiLLxIUxqCnP3Kr6hesazQgaP9HU6luOOe6CVQnWxVYGX3P5v5mfY/lT/Kaq6z3o9XKCkRCnRABgDtK308Aj/8Fuk9m5VCFl6/KYyA/9JhISgkKr7mOKt7YJsQRBbXfmMMW32nU6WSSvQQOx/GrsVuK/glfWiMv+MRvd0mWeLNcVDabHtBKpxUH0afS5Vx7/zj+7kAFFcGPR+MlRHNxndmWyW6SSBacsVQEJBe6bmVsuHrAjQMnfmgriphPdv6iwk5bEdpWFizstKiO5gQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
 <7c97de87-9687-a088-406b-33faf037244a@suse.com>
 <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2959d91e-1048-a4dd-7f79-ec1d14b81d02@suse.com>
Date: Mon, 13 Sep 2021 17:10:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0230.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8173aff4-b3e2-49de-192a-08d976c8af4a
X-MS-TrafficTypeDiagnostic: VI1PR04MB4942:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB49427EE39D329711316AF379B3D99@VI1PR04MB4942.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KxjmM49fmlBtFL5IDm89PVcvP5rvgsBa7VmxDzxXT+12sTKjyheFb+peM6ASbG016S8LjmW6gsOx2wjhRLJpSNKockGDrYWjG+PirXLMHf0cHM0y5XnNJ5AE3uvkyS8StUJNPcGiMC0f742KATuvlc/yYaRPOTkI+1mJfyO7UEJUrnbxQC1cPI+yp4PYchodelwEkQ5UHYfPDl2byHjHVlRrgkmD6GNvDcuNgH1zS+9P/5XTNxP20U1SWuT/xW09CuOfR0IIgvLYAaRHVd+UeYoRm41n6wp9JLqw2Ya9gV7x4HxJP7JOfz7uS6u7d4pwYYxDJ7+l4/6Gl/wEtQAWTV9M/AWf9sQgs/Tqgc1qTd8HW8i8dYrR0jJOYiSVG3c5DFPAuYSiDk/m35+ldUrv4VFT9L0TegizzRVofzbtosJpFghgRrYJuwTzzDmdIh2QNX5W30q8JnZVGKsGKEpasVIyxq8AwlZOFtqQ6hDApe6y5Qz8UEeHcY2EH/+mK11aqMooAypJJBgfZge0HFqdcEOvlKTQd3V3BYqAE1WD9UySzdWLs9cdBQdbJMY99ULHa7yf+3jJx6WNsTnxn+IjZxq9WOVH3nI7BhzaZcDQuRQ8H6ACifQzQpY2x0dZQACZRY+DzN4g4bBYGeQUo+URn+FVCcQI2gRtrkmWzoNhQazGNjivh4/UH9Vc2m7jzHsO/4G3JTPXqyw6Arl9TsDy/Yhypz4AbFpp1aycgmQOkwVEBptKkPn9yJS0MoPXjBFj
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(396003)(376002)(366004)(346002)(186003)(478600001)(38100700002)(2616005)(956004)(36756003)(86362001)(8676002)(4326008)(54906003)(66556008)(66476007)(16576012)(316002)(53546011)(5660300002)(8936002)(31686004)(2906002)(26005)(6486002)(66946007)(6916009)(83380400001)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?EiMO4y6vQ+WHZMNeKHkWQdww+QVWyRx6HHVtYiOYnnwNvIhlZhSZRGotKYHY?=
 =?us-ascii?Q?6EM6pUuRnMtww2QjnOw3z/HtjjVc3d4XzfTJk5cUEb2zRbdPQWOCbwgFzPMJ?=
 =?us-ascii?Q?U3ArO46SnEB8s7goUgFVDDwfFiicfo/dcOONHRnk3Bhkj7hWes6yZ4e50lhi?=
 =?us-ascii?Q?RQLv2H0D9CFbjlZTO4a/4YpNitZzhgM/rXEquBIo6OL2NDXDaIxSmslnPzdB?=
 =?us-ascii?Q?hXC4lSr8qRpWdZvHiTIH7UXvIXp0c2XsIrBr3ZOax5oPQVqVCJTz7sUb2/RM?=
 =?us-ascii?Q?YJTd0qdKpJ7AyIj/NpDRPpdLUaAux42kGNp10Fr0BJ22nYudApHGlcBeqW7y?=
 =?us-ascii?Q?pTcQm+NIjz6Cxb6IEi9TDTaOOzF1EVSzlmmbg2GnpdYW/TFBukGXLXtAehmP?=
 =?us-ascii?Q?SccZum494dPYrK9b0c2G0FikbC1p7VmA8at1gmyjdR1tdIJZ2ZqfpYulG4eK?=
 =?us-ascii?Q?0fNF18IcHPWsJD+DEBr3OmVi4bxLUKS35Z9OwALDXwInWdWS3KfqYupCUKvD?=
 =?us-ascii?Q?4eH8IWGEUN80QUDr1+esCebhh9e/dKe8/WIxj5leYbCW9kSXr3gSRouoxRoB?=
 =?us-ascii?Q?r0anireWUMUy6FoxX7p4q+jYrlKIRQnM8RDLZ/bQr4lp9/+Bj3EjtQMMdPeg?=
 =?us-ascii?Q?TDZ1SSVxhC+NTPgQwovxreXihMysSegXqXShYoGDEwF0BhrIsHdOGXCWZkwa?=
 =?us-ascii?Q?mWaVq8ttQFP6xILqw9nboX8VzcCaaVYur7u3F0xSrzqDt+S5RqxlfZ3yJrLK?=
 =?us-ascii?Q?jwQzLda8hU5PyDqyIoJCqvHN2OBxjDA1TkmtC3KidML5d0qi1Fhj6mSmyyov?=
 =?us-ascii?Q?JU6VJSdfDPtaPnl9sA4cM9efKL7Kj2UtdvURDfbrCSCJqzLKNdkojrLbrUas?=
 =?us-ascii?Q?AD/PRBP9FHgMu+RviseNo4o8HDzMLKU9n2Wc5i3L4/Znq2s9j/pk2nnopp/h?=
 =?us-ascii?Q?51HhK/K2YgNnSK+Ryq4tHWtQp8JmW5NlyRiPg/7QYRAKDdnu6nu/Qw+fV4H4?=
 =?us-ascii?Q?fGfDrvYxVPTY5PTCMGWMY6rerQjDg+2XkHTEKnJEqiTGiy5X1MuBPgpURTxH?=
 =?us-ascii?Q?6qGvcRwxxhpQfJi2aYDZ0jLUN6iVmmGVzHPnnXME2aHvBqDDqstL7YkeVbpy?=
 =?us-ascii?Q?t3CX4FGOTRFPulwKF726q11/XEh8EZkT9NeGufPMofZOaQjNiDr3UD7sCxNN?=
 =?us-ascii?Q?Pl+egIrl7PTTjUfnJKymsdK1sBevr0q0UKV3owm0K99KiyjKyrzvOvYrKJMc?=
 =?us-ascii?Q?ow4xZa1ssxPRyGB1MN7SxkAhck1QjNqjUvYg5N1FPoO8JoX0M7BsmxKtPBCO?=
 =?us-ascii?Q?y2uhk9EVIBKWSlViwf29jVdF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8173aff4-b3e2-49de-192a-08d976c8af4a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:10:55.7100
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PpQmkxqBW7Yry2TFjD33wsBvv66CvEXeK7uqJoTfb8AqzFdDkZLWI7+WK9R7w83zaq9dsyJuIf3ptN7ciq/n0A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4942

On 13.09.2021 16:33, Roger Pau Monn=C3=A9 wrote:
> On Mon, Sep 13, 2021 at 04:05:15PM +0200, Jan Beulich wrote:
>> On 13.09.2021 15:37, Roger Pau Monn=C3=A9 wrote:
>>>> --- a/xen/arch/x86/Makefile
>>>> +++ b/xen/arch/x86/Makefile
>>>> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=3D
>>>> =20
>>>>  ifneq ($(build_id_linker),)
>>>>  notes_phdrs =3D --notes
>>>> +# Determine whether to engage a workaround for GNU ld 2.37.
>>>> +build-id-ld-good =3D $(shell echo 'void test(void) {}' \
>>>> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c =
- 2>.check.err \
>>>> +                           && $(LD) -T check.lds -o .check.elf .check=
.o 2>>.check.err \
>>>> +                           && echo y)
>>>
>>> Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
>>> then use is here?
>>>
>>> We already have compiler and assembler checks in x86/Kconfig, so it
>>> would seem more natural to place it there.
>>
>> The question of whether to record tool chain capabilities in .config
>> is still pending. I'm not convinced this is a good idea, Andrew keeps
>> shouting me out for that, and an actual discussion doesn't really
>> happen. Yet unlike back at the time when I first raised my concern,
>> Anthony meanwhile supports me in at least the question (to Andrew) of
>> when such a discussion would have happened: Neither of us is aware,
>> yet Andrew claims it did happen, but so far didn't point out where
>> one could read about what was discussed and decided there.
>>
>> For the few uses we've accumulated I gave (if at all) an ack for
>> things happening under some sort of pressure, with the request that
>> aformentioned discussion would happen subsequently (and, depending on
>> outcome, these would be converted to another approach if need be). I
>> have meanwhile realized that it was a mistake to allow such things in
>> on this basis - the more of them we gain, the more I'm hearing "we've
>> already got some".
>=20
> I see, that's not an ideal situation from a review PoV, as we don't
> have a clear placement for those and that will just cause confusion
> (and more work since there are potentially two places to check).
>=20
> What's the benefit of placing the checks in Kconfig instead of the
> Makefiles?
>=20
> As I understand Kconfig is run only once, while the Makefile could run
> the check multiple times.
>=20
> The downfall of having them in .config is that .config could suddenly
> change as tools are updated or as it's moved around different systems
> (yet .config already contains specific compiler version information).

I should have added in the earlier reply: Besides the pros and cons
there is, to me at least, also the abstract question of whether such
information belongs in .config in the first place. To me this file
has always been a record of build-meister decisions only. Quite
different from the output of userland ./configure, where checking
various aspects of the environment is the primary goal.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 15:33:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 15:33:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185774.334503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnx6-0001b2-8G; Mon, 13 Sep 2021 15:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185774.334503; Mon, 13 Sep 2021 15:33:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPnx6-0001av-53; Mon, 13 Sep 2021 15:33:00 +0000
Received: by outflank-mailman (input) for mailman id 185774;
 Mon, 13 Sep 2021 15:32:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=srN1=OD=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mPnx5-0001ap-6C
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 15:32:59 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id df0a8208-14a7-11ec-b3ba-12813bfff9fa;
 Mon, 13 Sep 2021 15:32:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df0a8208-14a7-11ec-b3ba-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631547178;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=pPeROkWkD+UwU1sWZrf/MXZ+XSftST3GSOomuRhw/hk=;
  b=eSv5DzEB9gVbWE+Ywos1OXRlx26Oq4NS4pxzCnQQwkjb3A9zW0HcM2Lx
   sxaIlKUl/CISPXuYtIflx+Su9PZNgLuEXW96FVggRaTFiGpQ84m1/9qkU
   TXDpqsGg7zhR+n15wVGUQnH78IoS/qRAmloxrjOtFGhgoy2ZAE7GLdS+f
   8=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: QNLwAyyZzbgymAIzuct5xh6d6+1WEsT+mGzfLzuZVZjoxgi6DPkA0HMv6Lmuzba0+cdhwufqFT
 kiK7XSkJnGGDE9FejV3KGY/QNS3hMDoYAQCSXW8uWjr8S6sq+/A4J5m8cdMapFOfFPZi7BZiGk
 hwV9DE/93Q4PUI3TWAjefhxvbcL90ipuMxt+tT2cKE0m8OLjUF6XLxoPQ7++izr1gt+GQ9YYtR
 XgfFECaQJZ+BoUD/u4PJH3PWpFh7/HkdOZjm+zV4YZ+RJIk4KB38RxkjiOQri4bkHfCk1xdxXz
 XhdxTusULy6ukwrA3442AKc3
X-SBRS: 5.1
X-MesageID: 54365829
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QJ1BnqKeh6mu+k6wFE+RyJMlxSXFcZb7ZxGr2PjKsXjdYENS0zIFz
 WsaWDyOPPuOamX2etFxa4y38U4FucKDyd4wTwZlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6wLZj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2F2Mkh9
 NFPuaWSdgs3Ip/HyectWRhHRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gps3Z4TTayPD
 yYfQSJ0TRfkQxJdAGseFYs6gtWqtiP8cjIN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wv
 H/d4yHnBxQyMN2E1SHD9WqhgOPCg2X8Qo16PK218LtmjUOewkQXCQYKTh2rrP+hkEm8VtlDb
 UsO9UIGpKw/5AqhQ9/7UhCQpH+CtwQbHd1KHIUS7QiRyqvZ/kCBAWkeTzNbQNgnssYsQnotz
 FDht83oHzF0sPuWVHeU7J+QrDW7Iy9TKnUNDQcPRBEJ5NTLq4gpghXCCN1kFcadgtDwGBnxx
 S6Ltyk0g7gPjc8N2L6/9FqBiDWpzrDMRxQw7x/aXUqk6B14f4+vY4G06Vnd4u1EJYzfRV6E1
 FAfh9KX8OcVSJ2AigSKWKAJHaq1/LCBMTvVm1kpGIMunxyq/3OgZpxB+zFWK0JgM8JCcjjsC
 GfIpQ5f7ZlUemC2ZKV6Z4awDcUC3ankFNL1EPvTa7JzjoNZLVHduns0PAjJgj6rwBNEfbwD1
 YmzfYG1XUoeC6he7gW6Wfsv2u4o9io/2jaGLXzk9ChLwYZydVbMF+xcagDTP7xmhE+XiF6Kq
 IcEbqNm3z0aCbenM3eNqeb/OHhXdSBTOHzglyBAmgdvyCJdEWc9Arf6xbo7cuSJdIwEy7+Vo
 hlRtqJeoWcTZEEryy3RMRiPi5u1BP6TSE7X2gR2YT6VN4ALO9rH0UvmX8JfkUMbGAlf8BKJZ
 6NdJ5Xo7gtzpsTvpG1GMMiVQH1KXxW3nwOeVxeYjMwEV8c4HWTho4a8FiO2rXVmJnfn5KMW/
 uz7viuGEMVreuiXJJuPAB5Z5wjq5iZ1dSMbdxagH+S/j221oNYzdH2g0aZtSyzOQD2arganO
 8+tKU5wjcHGopMv8cmPgqaBroyzFPB5EFYcFG7ehYta/wGAloZ66YMfAuuOYx7HU2b4pPera
 elPlqmuO/wbhlda9YF7Fu8zn6454tLuoZ5czxhlQyqXPwj6VOs4LynUx9RLu41M2qRd5Vm8V
 HWQ94QIIr6OIs7kTgIcfVJ3cuSZ2PgIsTDO9vBpcl7i7Sp68ePfA0VfNhWBkgJHK75xPN93y
 OstopdOuQe+lgArIpCNiSUNrzaAKXkJUqMGsJAGAdC01lp3mw8aOZGFU334+pCCbdlII3IGG
 D7MifqQnalYy2rDb2E3SSrH091CiMlcoxtN1lIDeQiEw4KXmv8t0RRN2j0rVQAJnA5f2ud+N
 2U3ZU14IaKCo2VhiMRZBj3+HghAAFuS+1DryktPn2rcFhH6WmvIJWw7GOCM4EFGrD4MImkFp
 OmVmDT/TDLnXMDtxS9jC0dqpsvqQcF16gCfytusGN6IHsVibDfo6kN0ibHkd/cz7RsNuXD6
IronPort-HdrOrdr: A9a23:lXqL9aFMnEKK2PiVpLqFeJHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HkBEDyewKiyXcT2/hsAV7CZniahILMFu9fBOTZskXd8kHFh4lgPO
 JbAtJD4b7LfChHZKTBkXCF+r8bqbHtmsDY5pas854ud3APV0gJ1XYJNu/xKDwReOApP+taKH
 PR3Ls9m9L2Ek5nEPhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2mdfVmGJ6VMt1xN/dfOla9My4kD1jiVWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,290,1624334400"; 
   d="scan'208";a="54365829"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CqMeOQB+cqMavrUM5tpTckMMB6QUjZeNOnExttexzZBL78se/cKUAFRDp3syH+ueEmJh2PXhhw5gCbUGoUEn/7pYACTltXxMcRklYKH6uO5nXV9l0aGKi4/9GeqUGQSyahgfxzUFowJ3WmL/yrjFcpmOrAW33OFPi95smNSGdtkx2TbUlIQz4LhMDpEe2MsD2X4BsTUeGEWBdzdoSW9EUs7KT36/s8JsYwmJW487oECNFXaAiAZV7VxK1p21vNohJHrq9RjqfEKF21lN8sdwPt/PQ+0bY+m2z5Mp3Jfml9CHcBONoZRh0i67KOdrIOad4JAT3kzsI/GarsFNN8I3vw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=N5dRS7s+ZDTghHB0p4T10LOuo63+vSRBRWP1ItYIYfc=;
 b=OmjaZO1+RnuxnY5Sjkmzvtm5og+aYyoCgLAPfvM7KFR7h2sJLg0V0UokQh0EyFJA95mZPs/Ln6BxFuHwlDP2gZIdJOYHTdHsOx5uTYdENhFZdQeJc9MwXEuB/74Lee0UdlEfWs/Qs/7YqjfjQrHBZLfGhXLF3UwbC3Juq+cAeaa4yYJtQ+8NAu8cIyzZ9IUyVPrzzfh/ed16oMciIAm5yIfq+uZr52ux3aAnV+OhSjIJxygDho4su8T7WHHN0zoPem2nrOZK/qc2GJ35hTlVG/mspTjw8jx+YnmTv5k4XeUIR/W4ViUsRry4ySQ9rkZxDJi+mcAiXFsl2Ey++lwu3g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=N5dRS7s+ZDTghHB0p4T10LOuo63+vSRBRWP1ItYIYfc=;
 b=sKSDnqm1MNv25JIruakIijszWp82uk3NqXoQEsgfrtgzHuXG21JDx3b0X4ciFIO799fPBImuGH0wcqteMATnqdGHQ7kZO6qkqhDIrVIdVJg3mEtZKkNyyRh2sko22/nQd7cprWFxnJN0waF+ulxC00pLcgfVDAVvd/KVb9lZz00=
To: =?UTF-8?Q?Alex_Benn=c3=a9e?= <alex.bennee@linaro.org>, Mathieu Poirier
	<mathieu.poirier@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>, Wei Liu
	<wl@xen.org>
CC: "rust-vmm@lists.opendev.org" <rust-vmm@lists.opendev.org>,
	<xen-devel@lists.xenproject.org>, Stratos Mailing List
	<stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
	<olekstysh@gmail.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>,
	Doug Goldstein <cardoe@cardoe.com>, Demi Marie Obenour
	<demi@invisiblethingslab.com>, =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
	<marmarek@invisiblethingslab.com>
References: <87lf40vay1.fsf@linaro.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
Message-ID: <abfa4f44-8c56-af3f-485e-41b58e790d92@citrix.com>
Date: Mon, 13 Sep 2021 16:32:46 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <87lf40vay1.fsf@linaro.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0397.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:189::6) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c223f48a-63b7-4f27-be8f-08d976cbc16b
X-MS-TrafficTypeDiagnostic: BY5PR03MB5032:
X-Microsoft-Antispam-PRVS: <BY5PR03MB50327E83ABF13F4F06E078FABAD99@BY5PR03MB5032.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: A4WDFE6nyM/kz5QiKjXxfIr+MKFwvJ3gtXBIEl59bEaUiUYz72IZQ2hsszSPWKxGtR0kihD5pmbgI2Vj4ggcfQ0P1JB96gqLE7uuptZkdor4seyaAWn+DetniHPin/00b0qrNF0sVfGwVC0pB0G5ymKtJAskhruLQhkRFXP/1Zwx5RFeLMpEqYv7R0inL9El8dpGokE3O4G9mK1ohgD7OnKSdKaGcMH+H6wJx19Ww8zbZJYWYYSqw0TSlYHI5l4Aqyt9/54iiWybI8Si5C327+TK5BOHWfGX0oUP6Pwn6xIKVgDCbdv6mgojPIR0j2NNCsEe4Q9QN6lpza0NfNyK8GsZKvo9gWMZr1i6pqdCmY65Hr+AWeH39ORvJGRkCLjEKqhrdgI8FXFBCQ9fd50AlTX0osV9MWqnnc4ayFI6GCbNE7rMISQ5qCcboEbBc9R+i5aVwiJXBl1AfO4LZ0I6CbwGW8b2a8eWsdm9jokfYcSPQPcUL4FVHcX6Kf+SXBhSwE3ft+3kHkLAZrf+RzsIf8jaUqLpnt4oPkL7rt7QA529c7uyP4MFPeToSnL1LX48Mvews13Q2UUgqLunbq6Sa/80jQ8YKjCQEUOKe61b4g3DB+NHrptssGClF0SGOesqLyr56eichmj1DZLdm/41yYWEKXdZbKZYQAMAMdodwzZiMhtL30KajIuo3ImnTQ+4L5XydAr0VZN3be64pYkt4hGZGUqm4DTqdSwJJDQRLr+d47NoVGQ/BFEIv/lxEUJHpnNgPakkwusHusBDy9zZyOHxNKNFZnnGJMSCkXA3r+aBb5Vt8BOLdLkAYMDeJ62D
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(136003)(346002)(366004)(6666004)(186003)(316002)(66476007)(16576012)(8676002)(53546011)(66946007)(110136005)(26005)(4326008)(66556008)(2906002)(55236004)(54906003)(478600001)(2616005)(66574015)(36756003)(5660300002)(6486002)(966005)(31686004)(956004)(31696002)(8936002)(86362001)(38100700002)(7416002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?emlOUEc0WVdhaG9UcWNIL3ZBei9sU0RtdnNaSENlZERvaHJjTzJNbUlkZTc1?=
 =?utf-8?B?ZVNJQWU4RnpJVER4Ykt6eWI5YXlMdWdCbGVzUU5iVWkvUzl3bWJURjk2UGxy?=
 =?utf-8?B?TFhHSWx6QkJ2My91R2k4dk5pcUNoZ3lZWUhhOEZua0pWVWE2WVVXd2NTVHNJ?=
 =?utf-8?B?Uk1UUmtadU9sUUNWUHcxK2hnR2RJU0xYekVwVjFxSDZSNGVJY1BpLzdQUEd3?=
 =?utf-8?B?cTlwQ2t3Vms1bUljaU1sNE84bXU5SjNoU0ZSUHZzSUtMR3lxZmRGSHdaVGZr?=
 =?utf-8?B?UTBKdzVPOWlWVkxqWXQ1U0ZyK1NneDB5aVZsNks1YnlIYXdVTkRvUGQ5My92?=
 =?utf-8?B?eXhNVklxRElXeHhUb3ZVQ2NTY0xOUWF4RTNsUkJpYWhJdW5Da3U1RXl6ZUJN?=
 =?utf-8?B?bnh2MUYwbWp6TFpIQkE2MVc2SVA3b2txN09YMkwwMXdreUN5T1lNOWcrS0ZK?=
 =?utf-8?B?Y3E2TU5jdzZ4czNza3Vhb05OTVFxVVdKdERyOUc2ZzJhM0lpeG1XRi9YaWh1?=
 =?utf-8?B?dTU2VFhjSi9xT0NWMUthYk5rMTlPeHZtQkR4VzVCZmpSVlBqblJLZGN3MUpC?=
 =?utf-8?B?YjV4RjVqamwzdnY2cCtwQVFxS2c0L2k3eTJBSnczTFJjSnNjWFh0YnFYaFNT?=
 =?utf-8?B?d3lHSmY3ditwWDFQcXdXNHV2S2R5NDRVMklrMjBncjRHazBhVXJuVWVGVGNX?=
 =?utf-8?B?QjNmbGR1allZVEFFUUFvWlB6akgyOWVmOFRud2lJQk9VeWZWbE0wVUMxTnYw?=
 =?utf-8?B?OHEwQXBEdzB1Snh6QnRuWG5Rc3VwaTF1Q3ZTc09SM2pUbS9MdFJYU3o2cWVT?=
 =?utf-8?B?QkpYb0w2Zmc1WjR2S2VyY0FEckhYZFU5cVVSNXhMNEEwMWJkcDlQWUh3bHBx?=
 =?utf-8?B?UUwvVGZ5cVN3cnpiajFUUTQ3SmRWOFV3YWdVRDRna2FoQzN4blZMN0xFK2Ni?=
 =?utf-8?B?ZkpyM0lnWjFaS29TREd1WWV3VVV1bmtRdWR0SkhkUzlHSnB4RG01b0xHbzlB?=
 =?utf-8?B?cDViMzQvOFdyN3JINGlScm55dkhwVkoyOWt3UUtnRGVQTXB4NW8vdEFQOVg1?=
 =?utf-8?B?bTFIZUp4OGtuY01TOTRFcHZzL0VaZ3g2Y0h4Snc3eUhJODNXa2xjN0Q3dW1z?=
 =?utf-8?B?dVpaZGtybjJXQlZ6RXQ2VHh2UERLSU84SE9TL1JSRk0rTzhFSnNXbGMxdGFU?=
 =?utf-8?B?MUd1TTZ4WVRCb0VYTjdzK3VRQnllelcwam02TUpLYlovV3dlNVJsd2xHeUto?=
 =?utf-8?B?N3hRZXZjTzZZa2NDV25Na3o0Ukljb1duTDRwUFlkTDBPN0VxRVBES0o0WDZk?=
 =?utf-8?B?dFc3RzRRR2k4cWcrUndMa2FuZ0NCanp2N3hvQmowR04xRWh0ZzdTTDhqT2lC?=
 =?utf-8?B?NmtkMDRXRmFueXFWbU1scHdjTVJBd2dRbG9KTHpiNnhFc0UzTndPdE1SUWt6?=
 =?utf-8?B?WnN3ZExGYUNqc2s3ZUxxK2FGZ1RDb0x0ejVTZG1vT0NqRk9ZQnNkUm02TUFx?=
 =?utf-8?B?TGNJdG9rSU9odld1OGpDeTNETFVCdTJhVUg2VkNYbFFJTWY1WWEzbDdRTG9m?=
 =?utf-8?B?L09MeDZTcDBvc203WU83djVzVkE5ekZndmtyZ1FTZUF4NGNpYWh6eVFHM1dm?=
 =?utf-8?B?YjVrc3hvMk1zWitOdW42Z3oxU1FEeUlzd3N0NXZ5K0Q5K3FnRWh6Mmt0bmp0?=
 =?utf-8?B?am1NbFRNUTRzYjB3S3ZFaENxRE1HekM4alIzSFVGTVZaR2g4VHl6c0JIaFAv?=
 =?utf-8?Q?yTzZ1txD0NL3a1Ok+sbVfTKi7rQYZFhsIhXLsan?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c223f48a-63b7-4f27-be8f-08d976cbc16b
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:32:54.8726
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aJakhdMhqCWRxujejKlKx/Mbb53G0saHrh08YSXqh+dgLCXk+wW/YQtX0Uyu9zUqi8hLK6SFEzNHg6cEQaU2X+Z6ZnN7qFpukbCWi72DRAs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5032
X-OriginatorOrg: citrix.com

On 13/09/2021 13:44, Alex Benn=C3=A9e wrote:
> Hi,
>
> As we consider the next cycle for Project Stratos I would like to make
> some more progress on hypervisor agnosticism for our virtio backends.
> While we have implemented a number of virtio vhost-user backends using C
> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
> backends does it make sense to do some enabling work in rust-vmm to
> support Xen?
>
> There are two chunks of work I can think of:
>
>   1. Enough of libxl/hypervisor interface to implement an IOREQ end point=
.

No libxl here at all.

As of Xen 4.15, there are enough stable interfaces to implement simple
IOERQ servers.

https://github.com/xapi-project/varstored/commit/fde707c59f7a189e1d4e97c1a4=
ee1a2d0c378ad1
was the commit where I removed the final unstable interface from
varstored (terrible name) which is a dom0 backend for UEFI secure
variable handling.=C2=A0 As such, it also serves as a (not totally simple)
reference of an IOERQ server.


There are a few bits and pieces of rust going on within Xen, and a whole
load of plans.=C2=A0 Also, there is a lot of interest from other downstream=
s
in being able to write Rust backends.

We've got a placeholder xen and xen-sys crates, and placeholder work for
supporting cross-compile as x86 PV and PVH stubdomains.

The want to have a simple IOREQ server compiled either as a dom0
backend, or as a PV or PVH stubdomains influences some of the design
decisions early on, but they're all no-brainers for the longevity of the
work.

I started work on trying to reimplement varstored entirely in Rust as a
hackathon project, although ran out of time trying to make hypercall
buffers work (there is a bug with Box and non-global allocators causing
rustc to hit an assert().=C2=A0 In the short term, we'll have to implement
hypercall buffers in a slightly more irritating way).

Furthermore, stick to the stable hypercalls only.=C2=A0 Xen's C libraries a=
re
disaster for cross-version compatibility, and you absolutely do not want
to recompile your rust program just to run it against a different
version of the hypervisor.=C2=A0 The plan is to start with simple IOREQ
servers, which are on fully stable interfaces, then stabilise further
hypercalls as necessary to expand functionality.

It's high time the Xen Rust working group (which has been talked about
for several years now) actually forms...

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 15:46:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 15:46:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185781.334514 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPo9s-0003Di-Es; Mon, 13 Sep 2021 15:46:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185781.334514; Mon, 13 Sep 2021 15:46:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPo9s-0003Db-Bt; Mon, 13 Sep 2021 15:46:12 +0000
Received: by outflank-mailman (input) for mailman id 185781;
 Mon, 13 Sep 2021 15:46:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPo9q-0003DR-Uo; Mon, 13 Sep 2021 15:46:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPo9q-0008C8-K8; Mon, 13 Sep 2021 15:46:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPo9q-0000Of-7C; Mon, 13 Sep 2021 15:46:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPo9q-0005JB-6d; Mon, 13 Sep 2021 15:46:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DDBgCLUwSR/zZZYZVxfcLdawUk1Uxi8j5DmzWWbYYjs=; b=5b32nxsLeeChtEJiDgbGdZiB3N
	hSPUpJRbKP9EByfti/XBEERYqYAvu5rrCKBTnl8JFUZXpu3OGrbflwZ6eh2OByWFyKcy7+9pbNbj5
	enrIwox8HZWLdrlHZKntOsEl1FnDfS3eempEE7FB/Lbnrff5WZfS2m449zkVOMPqWSzU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164965-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164965: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start:fail:regression
    linux-linus:test-armhf-armhf-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 15:46:10 +0000

flight 164965 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164965/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 14 guest-start              fail REGR. vs. 152332
 test-armhf-armhf-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  408 days
Failing since        152366  2020-08-01 20:49:34 Z  407 days  709 attempts
Testing same since   164965  2021-09-13 07:09:47 Z    0 days    1 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255326 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:03:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185792.334528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPoQC-0006JO-2t; Mon, 13 Sep 2021 16:03:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185792.334528; Mon, 13 Sep 2021 16:03:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPoQB-0006JH-VR; Mon, 13 Sep 2021 16:03:03 +0000
Received: by outflank-mailman (input) for mailman id 185792;
 Mon, 13 Sep 2021 16:03:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NzTd=OD=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mPoQA-0006JB-IK
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:03:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3459a061-50bf-46ab-bd10-e2eef857fd88;
 Mon, 13 Sep 2021 16:03:01 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-O-cC-OpeMyuDGSfaymYevA-1; Mon, 13 Sep 2021 18:02:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4845.eurprd04.prod.outlook.com (2603:10a6:803:51::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep
 2021 16:02:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.018; Mon, 13 Sep 2021
 16:02:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0017.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Mon, 13 Sep 2021 16:02:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3459a061-50bf-46ab-bd10-e2eef857fd88
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631548980;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=6lVGNSeDPKamwhrf6Flt+scb/eycOt6W9k3kD4zIHAo=;
	b=EkBFuf/p/eG7glXqg9WW6jBqFMFDGsIfTSVqh4uz3sefHosoH3hhmwV/rJpatQOvE6RPsO
	11VtTjVc7NlMDTG7um8+gw7JS0x/eaPTntaDFbWRcJTIrHZPWCbEcFig6H7Mjg6R5YHCaT
	e+fU8aykdDld7CwWTh+Z58t8dlKZ2tE=
X-MC-Unique: O-cC-OpeMyuDGSfaymYevA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HgbAo8RKb+C8vIRoXni4CjMCHnRpalhbcb2gQD0IWwBfhLK680+PdMwi3pWx9Tn3wukla5uIy8LilAIZ0IZEhQFCK2DidFSKqIPtb/Ux7DvX9XVLXAH5tqikNj6U1BBvgY7wtxZNyRJCiD8IDkZ1CtWyi48b5VFuVS5omnzDSbIfFMNitLffbtTjK/lBczISiWKf8J3yhQtBLQr6PafXrnOR0qs7wyy2WNnKrUhlsSAzwGEO0vpEyZunkdCjuAA3rjUy6/PofwrL+s1C1mzzU7s5oSRH7/G7DahX+76Eajn/n0MOkX/g3hI+YpXNMDqY7/90HZ30SHmGpmZV2DkAtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6lVGNSeDPKamwhrf6Flt+scb/eycOt6W9k3kD4zIHAo=;
 b=C7igSjMt4Lu9g43c9I5jC5SjLLD9ZYs5zCn/AisJD9FL7DiElSWTUCAQWhjjSqZuhC67QbQw6DnryTjLuz+khHfHWXvTHsXaAz7bekddUqgEkqmsq5oh32V/BQUt4mys41cYtgbMbkC/uOCF1mEFsHsKsAHzNZ7wYuxzX57ory04v0OTRqDx7I337HwXahBW01Qpdsy2+8jmAIMVxy3ieZFxFL7p0clsOIvqgqCqU8FHEEc5SblaIur78BwyHtVV49k8+IF3HoX3bzzs3D1W4Xv4zx/kIr1TBzaCHd9YsQgGpi85/odjEbfoH9Ay+y4evvClS3HvQDzpyhIWSNQwqQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/boot: properly "ignore" early evaluated "no-real-mode"
Message-ID: <6b7e3281-4893-a5fb-cd1e-5b1918a1ee05@suse.com>
Date: Mon, 13 Sep 2021 18:02:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0017.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fe030721-0855-482d-a5b5-08d976cff4c4
X-MS-TrafficTypeDiagnostic: VI1PR04MB4845:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB48452B97AA316AE7E111E75DB3D99@VI1PR04MB4845.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:972;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tpEuRLjoTrY0sHB4865CyqI6na+HWViWAiAWEavnWuUUNRl8Kfrnh6NJhxEzfqKPH9grAlDJEl4IV/GR/0f9X34HYYPpHR2gfB34pQ1CkG6E3LWINhKOPeFE274kLDsXPallBtUdzeXOwA5svbzKVI/ABdq/BlKVipmNDgFeT0PBNQxJ+VW1tRVpSmzaVkXnbGn41SVgjnnVb7jsyBD90tKP1VuF/RdqqWVHv5J5ukv2sniBO0YW2PCrL/+LwVywtBtg990Ly6SODlqYRp74R0e5dRxzbjKdACmXMaR6T1B74HnHkWZx/HMVSTnpVPGKIJK6dh7OzupjFLMFNQZUlg7CZ+l9oV8j93ycKCNesZSnj1UVxtA9ErxievtuSJgmaP8STEEuVHZSUmWYA1p4v26BheK3G4tyTZIBbdeJKqJSzI+69Aj+DQfI5a0SIKhZ8XtOuoB6ts6QSTkTLClfJkggui2p1KSvfJbXeyz55Zje2uahyc5nfvuHMpLXCP0kZis8f+dyhU8payO/DTsRZxsWz0rEbDpKc/TJc8flvKjy857U21KwOSSUi3zae+syOHnm1tKzHJwISYodyTbk6Rxw3gHiO6Z8tq1wvugQa6Cg0bz39g0N96C8TpbTW1rKsP4cXGV23fgG2Z4CljbO4FgNCebLUTXfbDRvEuWBrV7KIB/RfIo+mPEEmvYb3RdOvmgCyN0gf3wk8K/IlUqch0Oe/wNmvVrX1s9Hm7VTjD5q6uaADlnE1XL/onhuMlcO
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(956004)(36756003)(2616005)(508600001)(5660300002)(54906003)(6486002)(86362001)(8676002)(66946007)(16576012)(316002)(186003)(38100700002)(66556008)(4326008)(6916009)(26005)(66476007)(4744005)(31696002)(8936002)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RHkzQXU4RGZqcjJ3eldKc1I2U3lwc1p4ajZRc1Y3M05HV2JDU1dHSGNUdUs0?=
 =?utf-8?B?c0pKN2VPZ2ovMm1pTXF2TDdVdEU3bzBMckV0QXI0NUdRa2Z1d2l6cTkyVkR4?=
 =?utf-8?B?cmN5ekg3dG4venNLY3BPSFFlZmZYRTY5dnhZeDNXZElpcVdxbjdudUx6b1Jy?=
 =?utf-8?B?UUFVMHBVSGFwUHhOM1BQNnRodWluYkJzWWtoNTJmVENxOHFsbldOKzFoYzMy?=
 =?utf-8?B?MDNwZmFpaHl4S0hhTUVsR09qZ3VIY1ZVY0tQTUVNbXQzMGU0OGNRd0NVTCtn?=
 =?utf-8?B?MmVLRDJKRG8wK254R1NQeVFpVktZV1QwVTRmaWEwZndRaWpnVi9SRGw0N252?=
 =?utf-8?B?RnQ2ZXlta29LKzBlb0EzWGVKZ0VRTUFhM0NkcUh6UHI4Y1MrTmRkZVhpbXh0?=
 =?utf-8?B?UUtWUWtidjdJQlNLWkIwb1dvTmNZQkRPZXNUMWpPZEdTUjhOSFZnTmJvRHBV?=
 =?utf-8?B?RnF1L1V0Wml6MDM3MGdma3kwWlVWQ1hMQ0hNeSsxRU5hVkY0WnlFaVp5VkJi?=
 =?utf-8?B?U3dZaFdQK3FYV2J3WFpQd3hxandOYXowYjZuSUxjVlQxS2RZUjBTNDhsRHdu?=
 =?utf-8?B?Zi94RllKbmh1TWdRaXJYNE92eGZ6andnN2QxMFlKL09wYzVubStHSkdreVc2?=
 =?utf-8?B?MkZZZzdZdWVyMFhOVElHdms1a3hHRVJkRjVkVURMNERGNmRXdklzZFVwVlE3?=
 =?utf-8?B?SitWdzY3a0Mvc3JmeDlFdXV4dUdVcEJDUSs5emhLUTZuMnd6YmRTRHlRWWJP?=
 =?utf-8?B?a3lRajREV3BHTHNrVTREQUV5MXRZSXRZbThaeStuVWpYeldpeVN3SHdjSE1m?=
 =?utf-8?B?Z3pNbHovU0RBTGFVaDc5bWdJeFNndmVXMWc4b2NvdnRJMFRMZWtsWnQ4Z01o?=
 =?utf-8?B?SHpVYi9QR1pPU3JFTEwvYmRDWXV5K3VpYzVYQzY3cFBOZW5lVW40SlhyY3or?=
 =?utf-8?B?YmtZTW5VOEZZaVdTN1IvbGg3VGxSSFJieTlnZWVYOXFmMExnSlBtSTBRdU1V?=
 =?utf-8?B?MElSOTRuNE5jRU1wT242RHBsMm5zVWpDV1RXUDRVWVdXYUpQbXl2Y3BjNFpI?=
 =?utf-8?B?RlJtOTdIWk1nZ1ArOEJKNlowZmxKVG56ZlZMbkRaWkVXZFNicHJ1K0NUWEQ4?=
 =?utf-8?B?OThIYXZaM1NTRTd0MCtqQnpUTTZyU3JJZlpvTGFnUXdRZmcwRjhEOURLUUdy?=
 =?utf-8?B?TG0zMGV1MFNvMzdGN2p4MzBySVJ5SDNvN1AvMGRkMHVNcXMzOFBtMTBwZ05R?=
 =?utf-8?B?S1J5UnViaGdmRXRVRUFGYW5SUklhK0RMOXI3NHNrNHdMNU9vbHNEcU9LeEdj?=
 =?utf-8?B?c1RIVElzOTNwNld6M1lwY1JDcEN2R25HeG5kazZGQllnSFNEOGpMMlpEMHNq?=
 =?utf-8?B?NElweG5XeDVJYTBmbGZGV29aenFQVGVjWkRlL0tkaitqM0xJZlY5NjNiUmJz?=
 =?utf-8?B?SHIvWHBIYXV3QlZwOXZhTVljdXhKSi83VmtkZ3E1Tktibk03MkV4VVkwci9O?=
 =?utf-8?B?cTFvdWhCcUtZamkybEkrMGRndG01QWltQTI2K2lUSkl1eWI5N1Zwd3FGTnAz?=
 =?utf-8?B?clk2dmZIand1ZVN1OHVkV2VqK3BQY3hVME9SUkpVeCtwVEl1eTdpamJIRmVE?=
 =?utf-8?B?blErN21TLzRCeTRSQStKTVpUeGdWVVZ4akRZYnIvY0ZuOERPNXlxeTlPWkhK?=
 =?utf-8?B?dXhGdE1qNERlVlJhV2NES3cvOUtKYnhEUi9yQWdJYTdxTFhqS25VUGNYWjd0?=
 =?utf-8?Q?CFOJex0UTrRPMrhu0b7T6o4HcEt4swbTzGBg2XU?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fe030721-0855-482d-a5b5-08d976cff4c4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 16:02:58.7871
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: N0KhxZcS0pEmD4TLVtHSiA/3jNATMl1d/+Ng67sLfLelf8vEw2XAVT9zyHWZOzYMUbCByZXRzK+woNR9kHONfA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4845

The option parser takes off "no-" prefixes before matching, so they also
shouldn't be specified to match against.

Fixes: e44d98608476 ("x86/setup: Ignore early boot parameters like no-real-mode")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -692,7 +692,7 @@ static void __init noreturn reinit_bsp_s
  * has options that are only used during the very initial boot process,
  * so they can be ignored now.
  */
-ignore_param("no-real-mode");
+ignore_param("real-mode");
 ignore_param("edd");
 ignore_param("edid");
 



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:31:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185806.334538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001OO-8E; Mon, 13 Sep 2021 16:31:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185806.334538; Mon, 13 Sep 2021 16:31:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001OH-5H; Mon, 13 Sep 2021 16:31:31 +0000
Received: by outflank-mailman (input) for mailman id 185806;
 Mon, 13 Sep 2021 16:31:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0001Nn-G3
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-00013O-DU
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0006tn-CP
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mPorf-00038U-AR; Mon, 13 Sep 2021 17:31:27 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	Message-Id:Date:Subject:Cc:To:From;
	bh=LGI6qRXBN2wzxnmRjeSiqUX0lkTiIFWU8AV8ecawCvA=; b=xy0ilulaO5pk4T+s6kM8DZ5uw3
	dptotykisPW14ibh2cswk6ieDROFFMRBliIr/ZwCGs7EJuyFnX7c/VEhlPYt0ZXfRmRgWeLmZBGfY
	ClZ+zcMrlK+yLl+D1pcqZ8DX3+LvIVfuwXZhTPDBjIGdJw69Iow+d4LOHuH+vMFNULlk=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 1/5] copyright notices: Add many missing notices
Date: Mon, 13 Sep 2021 17:31:11 +0100
Message-Id: <20210913163115.12749-1-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mg-adjust-flight-makexrefs          | 16 ++++++++++++++++
 mg-blockage                         | 16 ++++++++++++++++
 mg-branch-setup                     | 16 ++++++++++++++++
 mg-crontab-install                  | 16 ++++++++++++++++
 mg-debug-fail                       | 16 ++++++++++++++++
 mg-force-push                       | 16 ++++++++++++++++
 mg-hostalloc-starvation-demo        | 16 ++++++++++++++++
 mg-list-all-branches                | 16 ++++++++++++++++
 mg-report-host-usage-tuples         | 16 ++++++++++++++++
 mg-schema-test-database             | 18 +++++++++++++++++-
 ms-flights-summary                  | 16 ++++++++++++++++
 schema/drop-old-resource-log.sql    | 16 ++++++++++++++++
 schema/flights-started-index.sql    | 16 ++++++++++++++++
 schema/host-lifecycle.sql           | 16 ++++++++++++++++
 schema/initial.sql                  | 16 ++++++++++++++++
 schema/runvars-built-index.sql      | 16 ++++++++++++++++
 schema/runvars-host-index.sql       | 16 ++++++++++++++++
 schema/runvars-revision-index.sql   | 16 ++++++++++++++++
 schema/schema-updates.sql           | 16 ++++++++++++++++
 schema/steps-broken-index.sql       | 16 ++++++++++++++++
 schema/steps-job-index.sql          | 16 ++++++++++++++++
 schema/steps-logfile-add.sql        | 16 ++++++++++++++++
 schema/steps-logfile-constraint.sql | 16 ++++++++++++++++
 schema/testid-constraint.sql        | 16 ++++++++++++++++
 standalone                          | 16 ++++++++++++++++
 ts-repeat-test                      | 16 ++++++++++++++++
 ts-xen-build-rump                   | 16 ++++++++++++++++
 27 files changed, 433 insertions(+), 1 deletion(-)

diff --git a/mg-adjust-flight-makexrefs b/mg-adjust-flight-makexrefs
index e3716c4c..73f3ed4e 100755
--- a/mg-adjust-flight-makexrefs
+++ b/mg-adjust-flight-makexrefs
@@ -23,6 +23,22 @@
 #             passing --debug in REF-CONDS too)
 #   -n      Dry run: do not actually run cs-adjust-flight
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 set -e -o posix
 
 exec 3>/dev/null
diff --git a/mg-blockage b/mg-blockage
index dbec6df5..a61cba38 100755
--- a/mg-blockage
+++ b/mg-blockage
@@ -2,6 +2,22 @@
 #
 # usage: ./mg-blockage <date-from> <date-to> <hostflag> [<extra-xinfo>]
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict qw(vars refs);
 use DBI;
 use Data::Dumper;
diff --git a/mg-branch-setup b/mg-branch-setup
index de61cc5d..ec48aee1 100755
--- a/mg-branch-setup
+++ b/mg-branch-setup
@@ -15,6 +15,22 @@
 #  - create any input/output tree on xenbits
 #  - run any test flights (obviously)
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 set -e -o posix
 
 fail () { echo >&2 "$*"; exit 1; }
diff --git a/mg-crontab-install b/mg-crontab-install
index 4df8e6e8..49184dfb 100755
--- a/mg-crontab-install
+++ b/mg-crontab-install
@@ -4,6 +4,22 @@
 # where CRONTAB is a file containing a line
 #  #@@ USER@HOSTFQDN
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 set -e -o posix
 
 case "$#.$1" in
diff --git a/mg-debug-fail b/mg-debug-fail
index 47ad68ab..89e98b30 100755
--- a/mg-debug-fail
+++ b/mg-debug-fail
@@ -4,6 +4,22 @@
 # wanted.  It prints its arguments, and its stdin, to its stderr, and
 # then exits nonzero.
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 if tty >/dev/null 2>&1; then
 	exec </dev/null
 fi
diff --git a/mg-force-push b/mg-force-push
index 9c3cc786..286c1442 100755
--- a/mg-force-push
+++ b/mg-force-push
@@ -7,6 +7,22 @@
 #  valid for ap-fetch and ap-print-url
 #  the branch of flight FLIGHT
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict;
 use warnings;
 
diff --git a/mg-hostalloc-starvation-demo b/mg-hostalloc-starvation-demo
index 7944cc24..f9441822 100755
--- a/mg-hostalloc-starvation-demo
+++ b/mg-hostalloc-starvation-demo
@@ -1,5 +1,21 @@
 #!/usr/bin/perl -w
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict;
 BEGIN { unshift @INC, qw(.); }
 
diff --git a/mg-list-all-branches b/mg-list-all-branches
index 989b5ae7..1a04f617 100755
--- a/mg-list-all-branches
+++ b/mg-list-all-branches
@@ -2,6 +2,22 @@
 # prints on stdout a list of all the "branches"
 # mentioned in cr-daily-branch or crontab
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict;
 use Sort::Versions;
 
diff --git a/mg-report-host-usage-tuples b/mg-report-host-usage-tuples
index 375da888..0837cdab 100755
--- a/mg-report-host-usage-tuples
+++ b/mg-report-host-usage-tuples
@@ -1,5 +1,21 @@
 #!/usr/bin/python3
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 import re as regexp
 import sys
 import json
diff --git a/mg-schema-test-database b/mg-schema-test-database
index 357646bb..eae5d97e 100755
--- a/mg-schema-test-database
+++ b/mg-schema-test-database
@@ -1,5 +1,21 @@
 #!/bin/bash
-#
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 # usages:
 #
 #
diff --git a/ms-flights-summary b/ms-flights-summary
index 8293d4f6..a2507249 100755
--- a/ms-flights-summary
+++ b/ms-flights-summary
@@ -1,5 +1,21 @@
 #!/usr/bin/perl -w
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict qw(vars refs);
 
 BEGIN { unshift @INC, qw(.); }
diff --git a/schema/drop-old-resource-log.sql b/schema/drop-old-resource-log.sql
index 9494e1b9..f3dd9ce6 100644
--- a/schema/drop-old-resource-log.sql
+++ b/schema/drop-old-resource-log.sql
@@ -4,5 +4,21 @@
 -- aborted attempt at resource logging.  Nothing actually reads or
 -- writes it.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 DROP TABLE resource_log;
 DROP SEQUENCE resource_log_evid_seq;
diff --git a/schema/flights-started-index.sql b/schema/flights-started-index.sql
index c230d9d8..b26fc641 100644
--- a/schema/flights-started-index.sql
+++ b/schema/flights-started-index.sql
@@ -3,5 +3,21 @@
 -- This index helps ts-hosts-allocate-Executive find recent instances
 -- of the same job.  It may be useful for other things too.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX flights_blessing_started_idx
     ON flights (blessing, started);
diff --git a/schema/host-lifecycle.sql b/schema/host-lifecycle.sql
index 7e4fc2aa..6a7b2546 100644
--- a/schema/host-lifecycle.sql
+++ b/schema/host-lifecycle.sql
@@ -5,6 +5,22 @@
 -- reinitialised.  The information is persisted by being copied
 -- into a runvar for each job.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE SEQUENCE host_lifecycle_lcseq_seq
     NO CYCLE;
 
diff --git a/schema/initial.sql b/schema/initial.sql
index 4feeeb20..f9d120ae 100644
--- a/schema/initial.sql
+++ b/schema/initial.sql
@@ -2,6 +2,22 @@
 -- PostgreSQL database dump
 --
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 SET statement_timeout = 0;
 SET client_encoding = 'UTF8';
 SET standard_conforming_strings = off;
diff --git a/schema/runvars-built-index.sql b/schema/runvars-built-index.sql
index 128e69e9..8d1eadc9 100644
--- a/schema/runvars-built-index.sql
+++ b/schema/runvars-built-index.sql
@@ -2,6 +2,22 @@
 --
 -- This index helps sg-report-flight find relevant flights.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX runvars_built_revision_idx
     ON runvars (val)
  WHERE name LIKE 'built\_revision\_%';
diff --git a/schema/runvars-host-index.sql b/schema/runvars-host-index.sql
index 6a3ef377..0760b219 100644
--- a/schema/runvars-host-index.sql
+++ b/schema/runvars-host-index.sql
@@ -2,6 +2,22 @@
 --
 -- This index helps sg-report-host-history find relevant flights.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX runvars_host_idx
     ON runvars (val, flight)
  WHERE name ='host'
diff --git a/schema/runvars-revision-index.sql b/schema/runvars-revision-index.sql
index 25306354..4224b644 100644
--- a/schema/runvars-revision-index.sql
+++ b/schema/runvars-revision-index.sql
@@ -2,6 +2,22 @@
 --
 -- This index helps Executive::report__find_test find relevant flights.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX runvars_revision_idx
     ON runvars (val)
  WHERE name LIKE 'revision\_%';
diff --git a/schema/schema-updates.sql b/schema/schema-updates.sql
index cd8dc0ca..9b932a81 100644
--- a/schema/schema-updates.sql
+++ b/schema/schema-updates.sql
@@ -1,5 +1,21 @@
 -- ##OSSTEST## 001 Harmless
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE TABLE schema_updates (
     updatename TEXT PRIMARY KEY,
     applytime integer NOT NULL
diff --git a/schema/steps-broken-index.sql b/schema/steps-broken-index.sql
index 770747cc..d2e18d73 100644
--- a/schema/steps-broken-index.sql
+++ b/schema/steps-broken-index.sql
@@ -2,6 +2,22 @@
 --
 -- This index helps cs-bisection-flight check if flighss are broken.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX steps_broken_idx
     ON steps (flight)
  WHERE status='broken';
diff --git a/schema/steps-job-index.sql b/schema/steps-job-index.sql
index 2c33af72..21a6cf20 100644
--- a/schema/steps-job-index.sql
+++ b/schema/steps-job-index.sql
@@ -2,6 +2,22 @@
 --
 -- This index helps sg-report-flight find if a test ever passed.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 CREATE INDEX steps_job_testid_status_idx
     ON steps (job, testid, status);
 
diff --git a/schema/steps-logfile-add.sql b/schema/steps-logfile-add.sql
index abc89b9e..8be5469e 100644
--- a/schema/steps-logfile-add.sql
+++ b/schema/steps-logfile-add.sql
@@ -5,4 +5,20 @@
 --
 -- This change is compatible with old code.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 ALTER TABLE steps ADD COLUMN logfile TEXT;
diff --git a/schema/steps-logfile-constraint.sql b/schema/steps-logfile-constraint.sql
index f6f8a744..3d8b556d 100644
--- a/schema/steps-logfile-constraint.sql
+++ b/schema/steps-logfile-constraint.sql
@@ -3,6 +3,22 @@
 -- Previously, the logfile name was inferred from "stepno" and
 -- "testid".  Newer code will need it to be set explicitly.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 UPDATE steps
    SET logfile = stepno || '.' || step || '.log'
  WHERE logfile IS NULL;
diff --git a/schema/testid-constraint.sql b/schema/testid-constraint.sql
index 541f26e6..d1c3421e 100644
--- a/schema/testid-constraint.sql
+++ b/schema/testid-constraint.sql
@@ -5,6 +5,22 @@
 --
 -- The time_t value is Tue Jul  3 17:09:12 BST 2012, 5 years ago.
 
+-- This is part of "osstest", an automated testing framework for Xen.
+-- Copyright (C) 2009-2020 Citrix Inc.
+-- 
+-- This program is free software: you can redistribute it and/or modify
+-- it under the terms of the GNU Affero General Public License as published by
+-- the Free Software Foundation, either version 3 of the License, or
+-- (at your option) any later version.
+-- 
+-- This program is distributed in the hope that it will be useful,
+-- but WITHOUT ANY WARRANTY; without even the implied warranty of
+-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+-- GNU Affero General Public License for more details.
+-- 
+-- You should have received a copy of the GNU Affero General Public License
+-- along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 WITH cutoff AS
    (SELECT 1341331754)
 UPDATE steps
diff --git a/standalone b/standalone
index 4d1f3513..bc4e8ba8 100755
--- a/standalone
+++ b/standalone
@@ -1,5 +1,21 @@
 #!/bin/bash
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 set -e -o posix
 
 usage() {
diff --git a/ts-repeat-test b/ts-repeat-test
index 5e17c335..619c0359 100755
--- a/ts-repeat-test
+++ b/ts-repeat-test
@@ -10,6 +10,22 @@
 # -          before script name means to ignore errors
 # \          at the start of any ARGSPEC is removed (after the checks above)
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 use strict;
 BEGIN { unshift @INC, qw(.); }
 use Osstest::TestSupport;
diff --git a/ts-xen-build-rump b/ts-xen-build-rump
index 9ea595b9..8b31a44b 100755
--- a/ts-xen-build-rump
+++ b/ts-xen-build-rump
@@ -1,6 +1,22 @@
 #!/bin/sh
 set -ex
 
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2020 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 exec ./ts-xen-build "$@"						\
 	--disable-ovmf --disable-seabios --disable-blktap2		\
 	--disable-rombios --disable-qemu-traditional			\
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:31:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185807.334544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001Qm-Ie; Mon, 13 Sep 2021 16:31:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185807.334544; Mon, 13 Sep 2021 16:31:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001QH-D4; Mon, 13 Sep 2021 16:31:31 +0000
Received: by outflank-mailman (input) for mailman id 185807;
 Mon, 13 Sep 2021 16:31:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0001Np-H0
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-00013R-GG
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0006tr-FC
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mPorf-00038U-Ms; Mon, 13 Sep 2021 17:31:27 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=dLanMiw+A0RhDEAiKWmtytFsoVaO6ItybDZSeWiVSFQ=; b=MC7ROdb7+aFy8PSTZOAjvx2YIA
	1mA8fM/ol3/RPhPaQgEA8KvbEDoHR0Y23qTIhCU3R3Ksk7PFkZCrfbBR5ug7eKJxnsHi2iCU46xDr
	Yq3f7QkLdP3FhMG20W7tOupAx54XCeIks7zNf2XEHdXwOxMWxY4uNc99JE5N1cfbiwgY=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 2/5] mg-schema-update: improve an error message
Date: Mon, 13 Sep 2021 17:31:12 +0100
Message-Id: <20210913163115.12749-2-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210913163115.12749-1-iwj@xenproject.org>
References: <20210913163115.12749-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mg-schema-update | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-schema-update b/mg-schema-update
index f22ed63a..fb754bed 100755
--- a/mg-schema-update
+++ b/mg-schema-update
@@ -268,5 +268,5 @@ die "need operation\n" unless @ARGV;
 my $subcmd= shift @ARGV;
 $subcmd =~ s/-/_/g;
 my $subcmdproc = ${*::}{"cmd_$subcmd"};
-die "unknown subcommand" unless $subcmdproc;
+die "unknown subcommand $subcmd" unless $subcmdproc;
 $subcmdproc->();
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:31:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185808.334550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001Zz-SP; Mon, 13 Sep 2021 16:31:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185808.334550; Mon, 13 Sep 2021 16:31:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPorj-0001YY-O6; Mon, 13 Sep 2021 16:31:31 +0000
Received: by outflank-mailman (input) for mailman id 185808;
 Mon, 13 Sep 2021 16:31:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0001Nz-VN
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-00013W-UZ
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPorh-0006uF-TX
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:29 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mPorf-00038U-W4; Mon, 13 Sep 2021 17:31:28 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=2KrAnSQzk3ed/sh0BW4PwuT+8cM48+hczIyiE6a28KY=; b=trh+PbrGGxpGlZ5YhZtpwpbati
	UUkcfTinh8yQlSLWKJl9d/Z7R17YMwD9jPKhdCXXvhP+VLh8pL1WTsz7lCNid41iSnojn6HpWgzC3
	arz6T6lfu337P567eb5cHuxbbxQ0XeGd8H/GM3XefqAQ7EV9WCmbutZECZBlPC8kIJKE=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 3/5] mg-schema-test-database: Fix an error message
Date: Mon, 13 Sep 2021 17:31:13 +0100
Message-Id: <20210913163115.12749-3-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210913163115.12749-1-iwj@xenproject.org>
References: <20210913163115.12749-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mg-schema-test-database | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mg-schema-test-database b/mg-schema-test-database
index eae5d97e..052a3577 100755
--- a/mg-schema-test-database
+++ b/mg-schema-test-database
@@ -283,6 +283,7 @@ create)
 	done
 
 	if [ "x$tasks" = x ]; then
+		arg='<based on username>'
 		moretasks error					\
 			"WHERE type = 'static'
 			   AND refkey = :'refkey'"		\
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:31:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185809.334561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPork-0001lK-F6; Mon, 13 Sep 2021 16:31:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185809.334561; Mon, 13 Sep 2021 16:31:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPork-0001iN-63; Mon, 13 Sep 2021 16:31:32 +0000
Received: by outflank-mailman (input) for mailman id 185809;
 Mon, 13 Sep 2021 16:31:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-0001O5-9W
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-00013a-8f
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-0006uT-7i
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mPorg-00038U-FS; Mon, 13 Sep 2021 17:31:28 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=sMavJ2/0+irHymF2bsrF77nW/ZVjI3FTaIZlspuZ0SU=; b=2NgfaoTIp28ifDBzrUtVKoCwmU
	qIsL6swcpRYoOdN89o5O6oR0SgywU0lZ2X3H9ckPvtEYxInOv12vHcyVptY1hgwOGuAbGklToG9YF
	QvQHVuqX17+ijTTXffppRFshkkgwp4yTBS0Lr+uFPqaEHsglRzVse28AifmA0LiP0DGs=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 4/5] mg-schema-test-database: Add a missing backslash
Date: Mon, 13 Sep 2021 17:31:14 +0100
Message-Id: <20210913163115.12749-4-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210913163115.12749-1-iwj@xenproject.org>
References: <20210913163115.12749-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 mg-schema-test-database | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-schema-test-database b/mg-schema-test-database
index 052a3577..b185e8b9 100755
--- a/mg-schema-test-database
+++ b/mg-schema-test-database
@@ -285,7 +285,7 @@ create)
 	if [ "x$tasks" = x ]; then
 		arg='<based on username>'
 		moretasks error					\
-			"WHERE type = 'static'
+			"WHERE type = 'static'			\
 			   AND refkey = :'refkey'"		\
 			-v refkey="${username}@$(uname -n)"
 	fi
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:31:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185810.334568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPork-0001uo-Rl; Mon, 13 Sep 2021 16:31:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185810.334568; Mon, 13 Sep 2021 16:31:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPork-0001r3-JH; Mon, 13 Sep 2021 16:31:32 +0000
Received: by outflank-mailman (input) for mailman id 185810;
 Mon, 13 Sep 2021 16:31:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-0001OB-Hi
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-00013e-Gr
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <ijackson@chiark.greenend.org.uk>) id 1mPori-0006ud-Fq
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:31:30 +0000
Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk)
 by mariner.uk.xensource.com with esmtp (Exim 4.89)
 (envelope-from <ijackson@chiark.greenend.org.uk>)
 id 1mPorg-00038U-Oa; Mon, 13 Sep 2021 17:31:28 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:
	References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;
	bh=/+x+VTxPOx8gBN+12QXDZqpJKD9whSaNezCRFKCFhP4=; b=ESEKoHcEFy+r0di5R3FhO92RKh
	wUacJiG+3jnXvWu0YSD2OOXBXe1ueom9Br/NbhFcE+AubAgzqIDs3oe0pOQRl0hpFU3O2+EhNUubN
	gGMiqlID+yNH3Q9GmMyOM2IteZhNPccuhhJ8FLwkBGzE6eJTJLUMFoW1U39kJgTD/A8Y=;
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>
Subject: [OSSTEST PATCH 5/5] db: Replace ExecutiveDbOwningRoleRegexp with ExecutiveDbOwningRoleMap
Date: Mon, 13 Sep 2021 17:31:15 +0100
Message-Id: <20210913163115.12749-5-iwj@xenproject.org>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210913163115.12749-1-iwj@xenproject.org>
References: <20210913163115.12749-1-iwj@xenproject.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

And use SET ROLE.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 Osstest/Executive.pm    |  2 +-
 README                  | 17 ++++++++---
 mg-schema-test-database |  2 +-
 mg-schema-update        | 68 ++++++++++++++++++++++++++++++++---------
 4 files changed, 68 insertions(+), 21 deletions(-)

diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm
index d95d848d..ec592ed5 100644
--- a/Osstest/Executive.pm
+++ b/Osstest/Executive.pm
@@ -115,7 +115,7 @@ augmentconfigdefaults(
     QueuePlanUpdateInterval => 300, # seconds
     Repos => "$ENV{'HOME'}/repos",
     BisectionRevisonGraphSize => '600x300',
-    ExecutiveDbOwningRoleRegexp => 'osstest',
+    ExecutiveDbOwningRoleMap => '=',
     MaxUmask => '007',
 );
 
diff --git a/README b/README
index 20d9802a..aa611f23 100644
--- a/README
+++ b/README
@@ -770,10 +770,19 @@ ExecutiveDbname_<DB>
    PostgreSQL dbname string for the database <DB>.  Default is to use
    ExecutiveDbnamePat.
 
-ExecutiveDbOwningRoleRegexp
-   Regexp which is supposed to match the database user used for schema
-   changes - because, that role will end up owning the database objects.
-   Defaults to `osstest'.
+ExecutiveDbOwningRoleMap
+   Adjust roles used for schema changes.  osstest uses the following
+   nominal roles
+      osstest        Main db runner, owns most tables, etc.
+      webflights     More privileged, can see webflights tokens
+   The syntaxes for ExecutiveDbOwningRoleMap are
+      =              use the built-in nominal role, literally (default)
+      -              always use the default role for the user
+      <NEW-ROLE>     always use NEW-ROLE for everything
+   or a list like this
+      <NOM-ROLE>=<NEW-ROLE>,...
+   where <NEW-ROLE> can be `-' (default user role); unspecified
+   roles are not mapped.
 
 Flights for by-hand testing
 ===========================
diff --git a/mg-schema-test-database b/mg-schema-test-database
index b185e8b9..33b644f5 100755
--- a/mg-schema-test-database
+++ b/mg-schema-test-database
@@ -361,7 +361,7 @@ OwnerDaemonHost $ctrlhost
 QueueDaemonHost $ctrlhost
 OwnerDaemonPort ${ctrlports%,*}
 QueueDaemonPort ${ctrlports#*,}
-ExecutiveDbOwningRoleRegexp .*
+ExecutiveDbOwningRoleMap -
 QueueDaemonHoldoff 3
 QueueDaemonRetry 5
 Logs $PWD/logs
diff --git a/mg-schema-update b/mg-schema-update
index fb754bed..5ec7220e 100755
--- a/mg-schema-update
+++ b/mg-schema-update
@@ -22,7 +22,6 @@
 #  ./mg-schema-update [<options>] apply [<updatename>...]
 #  ./mg-schema-update [<options>] show
 #  ./mg-schema-update [<options>] apply-all
-#  ./mg-schema-update [<options>] check-user
 #
 # Usual rune for applying updates:
 #
@@ -53,6 +52,7 @@
 
 use strict qw(vars);
 use DBI;
+use Carp;
 BEGIN { unshift @INC, qw(.); }
 use Osstest;
 use Osstest::Executive;
@@ -121,13 +121,6 @@ sub getstate () {
     @state = sort { $a->{Sortkey} <=> $b->{Sortkey} } values %state;
 }
 
-sub check_user () {
-    my $user = $dbh_tests->{pg_user};
-    my $re = $c{ExecutiveDbOwningRoleRegexp};
-    return if $user =~ m/^$re$/o;
-    die "running as wrong user \`$user', expected to match \`$re'\n";
-}
-
 sub cmd_list_applied () {
     die if @ARGV;
     getstate();
@@ -174,6 +167,56 @@ sub want_apply ($) {
     $v->{Todo} >= 2-$force;
 }
 
+sub check_user () {
+    my $user = $dbh_tests->{pg_user};
+    my ($map, $default) = roles_map();
+    my $n = $map->{'osstest'} // $default;
+    my $wanted =
+      ($n eq '-' ? return :
+       $n eq '=' ? 'osstest' :
+                   $n);
+    return if $user eq $wanted;
+    die "running as wrong user \`$user', expected \`$wanted'\n";
+}
+
+sub cmd_check_user () {
+    die "too many arguments\n" if @ARGV>0;
+    check_user();
+}
+
+sub roles_map () {
+    my $default;
+    my %map;
+    if ($c{ExecutiveDbOwningRoleMap} !~ m/,|.=/) {
+	$default = $c{ExecutiveDbOwningRoleMap};
+    } else {
+	foreach my $kv (split m/,/, $c{ExecutiveDbOwningRoleMap}) {
+	    my @kv = split m/=/, $kv;  @kv == 2 or die;
+	    my ($k, $v) = @kv;
+	    grep { $_ eq $k } qw(osstest webflights)
+	        or confess "bad NOM-ROLE $k";
+	    $map{$k} = $v;
+	}
+	$default = '=';
+    }
+    return (\%map, $default);
+}
+
+sub map_roles ($) {
+    local ($_) = @_;
+    my ($map, $default) = roles_map();
+
+    s{^\@\s*SET ROLE\s+'?(\S+?)'?\s*\;\s*$}{
+        my $n = $map->{$1} // $default;
+	($n eq '-' ? "SET ROLE NONE;" :
+	 $n eq '=' ? "SET ROLE '$1';" :
+	             "SET ROLE '$n';")
+    }mge;
+
+    #print STDERR ">$_<\n";
+    return $_;
+}    
+
 sub applyone ($) {
     my ($v) = @_;
     die "Will not apply $v->{Name}.sql: $v->{Msg}\n"
@@ -195,7 +238,8 @@ sub applyone ($) {
             SET client_min_messages = warning;
 END
 
-	$dbh_tests->do($sql);
+	$dbh_tests->do(map_roles('@SET ROLE osstest;'));
+	$dbh_tests->do(map_roles($sql));
 
 	$dbh_tests->do(<<END, {}, $v->{Name}, time);
 	    INSERT INTO schema_updates
@@ -253,12 +297,6 @@ sub cmd_apply_all () {
     print "Appropriate updates applied.\n" unless $quiet;
 }
 
-sub cmd_check_user () {
-    die "too many arguments\n" if @ARGV>1;
-    $c{ExecutiveDbOwningRoleRegexp} = shift @ARGV if @ARGV;
-    check_user();
-}
-
 GetOptions('f|force+' => \$force,
 	   'q+' => \$quiet,
 	   'o|oldest=s' => \$there);
-- 
2.20.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 16:39:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 16:39:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185842.334594 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPoyy-0004hp-L3; Mon, 13 Sep 2021 16:39:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185842.334594; Mon, 13 Sep 2021 16:39:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPoyy-0004hi-Ha; Mon, 13 Sep 2021 16:39:00 +0000
Received: by outflank-mailman (input) for mailman id 185842;
 Mon, 13 Sep 2021 16:38:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=srN1=OD=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mPoyx-0004ha-D9
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 16:38:59 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1748f60b-14b1-11ec-b3bb-12813bfff9fa;
 Mon, 13 Sep 2021 16:38:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1748f60b-14b1-11ec-b3bb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631551137;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=Dx2x2XqTpyd1p3QOah+oE8zdEEA/yEljPhBsZRZpkXo=;
  b=auogYl2f7gzGMHQDFsI9SjzzRw+MNqqgMXTbqzkICggbt6n9d1HPNHyd
   yuaOBHPOIJDmckmDSVTx8HTsE8cIiNNQhU6WlCXpVRT1BkZnzIlwC6tkc
   CMSs0GOCCjVF3fx6BcE7AymilGcNZRwh5iOZ8gEUnGOnMtG/k9bp8mf7q
   k=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: tf5oZ7PDK0TmhJPzoy6gm6CsjGC8Aw/21TQdtr//XMhnVptYLEAc8QyRsxdXmh1w2sgA2f0y8Y
 EnPECNN0vu1bCWHsQVE0mVX1rJc+I1sz/khkj5hxCPfobYLZ+Q98Bjh5uJ/1Nel0Skk+YLNOHY
 CVR0WaQm7qI7SYw8v/yXL06iOgjQb9sECyINu1F2QKbcqTIH81Zh3KcBU7Jjm4mjetfHl0n+DO
 +Toii920AbzKTKqEtpf5teZGiEfwNO++bKxq5dWRPlMcWY5PhGclYVDeTL8tpcqj/Vs3rxDoKt
 7/N7nk+rjc0Dykg/+T0yC7JZ
X-SBRS: 5.1
X-MesageID: 52618433
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:yB6eI62PDIq5C2sxAfbD5fZ2kn2cJEfYwER7XKvMYLTBsI5bp2ECz
 GMWXDyPP/iOZTP0e98iPYW3o0JXvMWEm4QyGVA/pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrVh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhuIwt0
 OxxlK2KRCAqA4HyteQQVwl4DHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t3J4XTaqFP
 KL1bxIxbk7JXyFiB2wUN55hhb+viUjha2VH/Qf9Sa0fvDGIkV0ZPKLWGNjfd8GORM5Vtl2Fv
 W+A9GP8ajkKOdraxTeb/3aEgu7UgTi9SI8UDKe/9PNhnBuU3GN7IAISfUu2p7++kEHWZj5EA
 xVKoGx09/F0rRH1CImmN/GlnJKaljlAB8JpL8wL0gyqyZf43SuZC2wUbiEUPbTKq/QKqSwWO
 k6hxo2yXGY07ezJFhpx5Z/P8mjjYnF9wXsqIHZeFFpbuYGLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3m7ESnQkSGcICuTrsoCNtVgRiG2NPdXABb3nARFodtrxc7V5l
 CJY8/VyFchXZX13qMBofNjh4Znzv6rVWNEjvbKfN8Z4rGn8k5JSVatR/Ct/NC9UDyrwQhewO
 BW7kVoIvPd7ZSL2BYcqM9PZI5l7lsDISIW6Ps04m/ITO/Cdgifcp3owDaNRtkiw+HURfVYXY
 MfFKJn9XClDU8yKDlOeHo8g7FPi/QhnrUv7TpHn1RW3l72YYX+eU7AeN1WSKOs+6cu5TM/9q
 b6z7uOGlEdSVvPQeC7S/dJBJFwGNyFjV5v3t9ZWZqiIJQ8/QDMtDPrYwLUAfY15nvsKyreUr
 y/lAkIImkDigXDnKBmRbiwxYr3YQpsi/2kwOjYhPAj01iF7M5qv9qoWa7A+YaIjqL541fdxQ
 vRcI5eAD/1DRy7p4TMYaZWh/oVueA7y3VCFPja/YSh5dJllHlSb9tjhdwrp1S8PEivo6pdu/
 +z+jlvWGMNRSR5jAcDabOOU42mw5XVNyvhvW0boI8VIfBm++oZdNCGs3OQ8JNsBKEufy2LCh
 RqWGxoRucLEv5QxrIvSnamBooqkT7l+E05dEzWJ5Lq6L3CHrG+qwIsGW+eUZzHNEmjz/fz6N
 +lSyvj9NtwBnUpL7NUgQ+o6k/pm6ou9vaJewyRlAG7PPgaiBb5XK3Wb2dVC6/9WzbhDtArqA
 k+C97G241lS1B8JxLLJGDcYUw==
IronPort-HdrOrdr: A9a23:kIoRhaPoUvrC0MBcT1b155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/exoS5PwPk80kqQFnbX5XI3SITUO3VHHEGgM1/qb/9SNIVyZygcZ79
 YbT0EcMqyBMbEZt7eC3ODQKb9Jq7PmgcPY9ts2jU0dKj2CA5sQnjuRYTzrcHGeKjM2YKbRWK
 Dsnfau8FGbCAoqh4mAdzU4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kLEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 bxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72zeBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlJXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbZrmGuhHjXkV1RUsZiRtixZJGbAfqFCgL3V79FupgE686NCr/Zv2Evp9/oGOtF5Dq
 r/Q/1VfBwndL5gUUtHPpZ1fSKAMB2Fffv9ChPhHb3ZLtByB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,290,1624334400"; 
   d="scan'208";a="52618433"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WF9pudjKpxlMX+mvBxvDqVuDgFISSfoIWWAJNd5ggnh3nRe8jzReNtihCluKtT5hI8LzllXIELEOY9Oqb1woJfSNXeD2misCrf+Swn85t3A7qSSyAG8W8+oefnD0pSTqA+Q05x7NSk0u4xDMRuDnsFhMfz+Nqjcm2lKfbzrJxwhdrIoBbudj9Z0WnFAqjko4pJYly2ZcFCMx7CqoDlGXpvc9ovtBKf5GuDL5OaA+6pDvJVtV5mfEtPe/2ZbjHWYOZDshRhBTHDw6PUIc0lfx6OQzqkO2xqM3SjdIKo1wbgcLKSdPButu7Re1yZY7FNdAwOzN++sGTZtHB8aj5ToNvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fDiQdlmJ5pKsz8BO73i8Oa/hYI1/pgvsgP1x8ianxO0=;
 b=oJYVLbabQyBQo2JbWP5x836/iA7n4DVSKpR9olaqAMvZXpTde7gFH7Av/+0iSpzuXRJDoPT/88TFFQNaMb3pwmNzzab1/zfO9x0DgFQ1pSpfPydxX+zVf97aAXoat3SNuVf5LwSEBz4cIJUGRLx3Xr6RHcgieGdinOwPJSPSelTZOj4BtgvaeHdM9HKc/F+MoxrDhy88JOqFrNZQOaYNtNNJPd97wzL3A+S3LmjOH5XTk7bSUM0ZS1bs/OlWDE5eXP9Vut+OWM2p1EvOr9fymYdK4D7lG5nuL+mnhT8e8v9rHQru1dwUTVdB2CnsFcGhbE38MNzDnj7K8ZJ93iDCFA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fDiQdlmJ5pKsz8BO73i8Oa/hYI1/pgvsgP1x8ianxO0=;
 b=XOH1nB0jNqPhRRIMHW4q5SU8+tN+ipmP2WoA/rGt09wh2x1fRBnVpG/MO1vZzMnRfSNu+v/QGZ6dlbzulMEIATZ6nZ/E5TvJy0eKPSFc/dm3Kg902ed7df2w93ZSuq9631XTvGX73GwRlzs6a8h2XPhLLA4h5AiIXxHbc8x35Bk=
Subject: Re: [PATCH] x86/boot: properly "ignore" early evaluated
 "no-real-mode"
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>
References: <6b7e3281-4893-a5fb-cd1e-5b1918a1ee05@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <cb65f6ad-7a43-6a66-e31b-69c123103c26@citrix.com>
Date: Mon, 13 Sep 2021 17:38:48 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b7e3281-4893-a5fb-cd1e-5b1918a1ee05@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0365.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18e::10) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a6827bb7-c37a-4a48-8ec2-08d976d4f9d6
X-MS-TrafficTypeDiagnostic: BYAPR03MB3495:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB34952D57727DB9C62B3B04ACBAD99@BYAPR03MB3495.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:972;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: qcARUnWDk3Pmo+FLFjvWuZMBt275G62myftknR61c7hbBaAXVKbKXh9alPanIk8SRGMr38lf5wkxRLVGlzHyuylCAvSrW0XeaJJ7pI3sDLfYAUpSQN8MRPfM7aU+BbV+TJZHbKce+T5Y1BxjZZok+5GqI26MYrgJ4WbnBSLJDU528r5IQQwaAeYm8cS/TXshr/oDS4W4CjtSN2ijKy2J+REKTc9YVt6/msECqMsdECWhvko2nXCQpNkOCVdUj/3Tq8g2UNsxxsm2Hdow8RBHSoncKfw5/eUPAkXsc3mP6RNiF00Ny7ElMfxjgtmejoSEqQf4omsfjEf1ZBJv7Mo1EYPvDi0zcfD+Fg5aCVgaEA9Oh31dqSUqY15MmoNcVQHSPWmbecOzLLmGYZug+AkHipAK0aZGhY+mASgu3GZuO94hvCm15jw9x0u2SNeh8/ZbJfwHUsh6gfOFkmX7H98qD/PcKR/SyzeR8XL1i5/RStTqs1JB/jloCIeL4KcMupVS+Mz1YWEwkVZsD2DPe3AHmx9xVtu7aa/1H2FU3FxbPxQEmKbz4OlvEqC0RVpMEy6fTMLcy65fEufxEegx7/A7kAcb0WAkjq6lHOVRYKRg4yCq67xw82JqfMsZQOulQ059Fn4v8VbfioMCdmJLw5m7A7XbPsPTGAmlI2RlojfwYOiGMfupO2TZfBmI+yLXW0VniTP4yIFovl3H7+s0xQdTg4NEtK3gjDw8olx6UGCszO0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(31696002)(54906003)(66946007)(508600001)(107886003)(956004)(2616005)(31686004)(66556008)(66476007)(55236004)(5660300002)(26005)(186003)(38100700002)(4326008)(316002)(6486002)(8676002)(36756003)(16576012)(4744005)(8936002)(6666004)(2906002)(53546011)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bTZTSGJoVUxscExOL1o2Z2kvajdZSGRkNUJnUWVpaVdQYTd4dlMreXFGakFn?=
 =?utf-8?B?c0dyTjNhRGZnc0xKSUkyckVrVjUrZ1hqVngwN2NJbmZTSzRvVnpPaGd1UW9C?=
 =?utf-8?B?NnFlYVVlRlNyeURCMU5oUkgwSmRJWHV5TjZhM1hnVjZ6THRKM1hOUW43UlA5?=
 =?utf-8?B?WmQzeDhPaExhdDYyenE1dTRHb29KSWVXaER0UXphVnl5aFA3V3RLeHlrc1c5?=
 =?utf-8?B?cFBka3JBZjBJd3NLN2pQbFFPbTBqdmhZd2syR3ZDc0FRM1M5TExwSHJRc0pw?=
 =?utf-8?B?Zm1KOTdobHdUMGI5RU80cXBBTUcxM1R2Qzk4ZitURElNZ0t4TzU5TlZqNUR6?=
 =?utf-8?B?Q21ZNnhtMGVCS0V0ZHcxQkFLYmNQZU1HNThFcHZCZzJIcmp6YTJTOXFJVURn?=
 =?utf-8?B?a0JlM3BtTTJodS9Xb1NaWFp6c25Va0g1d2JpT3NTeDhvQ0RDNXlzaE8zWXJB?=
 =?utf-8?B?d3h0U3lBem8zaVlJbVNGVE5vRGw2RWNnTUMxS1VQNTNMb3BpaTdFK1pGZkc1?=
 =?utf-8?B?cUpKenFxYmowNjRXTlIrUFFVbkpneWIwY3IrVnhWTlpiREZVbGRNRmxwajlj?=
 =?utf-8?B?RmxEb0NKVVhJWFFhT0FjaE5HUDFJdkwxUHdRV2xpRWJYRGRJNE5zbWFJcUEy?=
 =?utf-8?B?cFQyeG1vWWJ4SHBHUzh5ODl5a0wyZ0JTZDJKNDhZNDZPYmZWSTdFcFNGYSt2?=
 =?utf-8?B?K1l0M1FLdXJRTGozc0dleStNT1lVL1dSWmlxOWxFK2lxNWFibDVzaGxtdG01?=
 =?utf-8?B?TDhnNzRKbiticE4yMXdMV1FJZ1g1L21MWjUyV005clNrVTUvUDU4c0w0WFpQ?=
 =?utf-8?B?WlR6RW5HTG9sQnhCRXZwaXhtWExnWngzZ2g2RXNaaHdYN2JYY2lCYzc4alFw?=
 =?utf-8?B?Z3ZSbndKL1RLY2xlQllPNmFTQWZWcVFmQzNHODByU2RuVUFNWEFZSUVSMDFT?=
 =?utf-8?B?R0VMY0hXd0ZMSXVwMjU5b0hzdTdOM2hlY245L044eTBQZXhPbThwK3IzTk9t?=
 =?utf-8?B?Z0Y4REFrbUVrTzFyYzFhWXZ4RmViYmkvNXUvV0xDNlA1c3htWmdyWlkvTi9E?=
 =?utf-8?B?cmNBc0p1TkZLZElhbzdtblptWWNuclFsU082VlVKaTFhTXR3cTE0d0d3K0xS?=
 =?utf-8?B?WkMzZ29jSGlnNFR2cjJWcWp5RVpBalNkc1hvT3hsL1pEdzRTTFYrZlVwQjRI?=
 =?utf-8?B?bnVJSzMvRkxWZGVRKzNnYytiVFRmbFBDSkRTYUJuLzBXaXVDdmtsZW1HQkFh?=
 =?utf-8?B?L1VXSld6dU9TQTFHU0Vzek1iTEZzZkZnVEIrcUZlNi9uZ3BYUFJISk5Mem9a?=
 =?utf-8?B?NFdPOVNpL0xIemlMa3F6aDBIU3Y0clBoWEtsUFh1WmQrcks4aHFDc0RHMFBo?=
 =?utf-8?B?WE1vbE5QZ0NOMjhyQ0hWOEYySEU1WnNuMndWbE5sbkFZazM3eW1oZE5HWFpX?=
 =?utf-8?B?VjdiWGtjUTVIaXdHOUd4a0pLLzdmYWlzYWp2bEpTMElJcStYOGJaL3Q4dXFq?=
 =?utf-8?B?L25RbGdxUGMrMnlBRmhyN01ITjFSYi95bFRLMC93L3p2VnVQQ3J6NTBkWmll?=
 =?utf-8?B?MEJxa2s3NWs3UERjL0t6bm1BYnNlQUY3bE4waUQycVFNSEtEeDhnTy9wcjlB?=
 =?utf-8?B?SVpDQzAwdGZNNUVLUzdnQnhBR2RsdkdMWG5Weno3Y2JWSURLUlVUVGdNaGl0?=
 =?utf-8?B?V3hUTFNaZDhhQkFjcWJVcjJmaU9MbFV5TDFOTXBCSW5teGlJM1FjdkdtYzhZ?=
 =?utf-8?Q?ba0bg/cnUysojdLclk4hlfLoJlNqKjY+hsh7l+L?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a6827bb7-c37a-4a48-8ec2-08d976d4f9d6
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 16:38:54.7347
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: knZ+LIp2j3M9zbCb3OJPEeKs+IckGT4bfNrlbQcVfzfVXaK6yiRb9Fs7RzZcuXSdKRrFm8lZsnF6oC0NIhER5Y5nvfxY4hGEsZmR7RsuHnM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3495
X-OriginatorOrg: citrix.com

On 13/09/2021 17:02, Jan Beulich wrote:
> The option parser takes off "no-" prefixes before matching, so they also
> shouldn't be specified to match against.
>
> Fixes: e44d98608476 ("x86/setup: Ignore early boot parameters like no-real-mode")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Oops.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

>
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -692,7 +692,7 @@ static void __init noreturn reinit_bsp_s
>   * has options that are only used during the very initial boot process,
>   * so they can be ignored now.
>   */
> -ignore_param("no-real-mode");
> +ignore_param("real-mode");
>  ignore_param("edd");
>  ignore_param("edid");
>  
>



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 17:09:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 17:09:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185855.334605 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPpS8-0008H4-Np; Mon, 13 Sep 2021 17:09:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185855.334605; Mon, 13 Sep 2021 17:09:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPpS8-0008Gx-Km; Mon, 13 Sep 2021 17:09:08 +0000
Received: by outflank-mailman (input) for mailman id 185855;
 Mon, 13 Sep 2021 17:09:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPpS7-0008Gn-Nr; Mon, 13 Sep 2021 17:09:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPpS7-0001lS-Gv; Mon, 13 Sep 2021 17:09:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPpS7-0003Le-77; Mon, 13 Sep 2021 17:09:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPpS7-00077p-6d; Mon, 13 Sep 2021 17:09:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NT6HLEg4URBWZH5PRqP7pruSSJ0UZooSQofZDIs81AM=; b=yB7JHgVXw4zG5FxShAi2RV63P/
	e2z8snlgkbpjcXVO7LKoEp3gsUWsNVVKXHWK2VJk7HMzqugAi79aDfbWxzKQxoHCsJ4vvctElLSiq
	O4ths5DLBGsbWIAsDdiZET8EytDlBS1GnGmITBtyTjLf94t4cBogdBD5Wmskqe80i3uo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164966-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164966: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a89bcd9737757e4d671783588a6041a84a5e1754
X-Osstest-Versions-That:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 17:09:07 +0000

flight 164966 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164966/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a89bcd9737757e4d671783588a6041a84a5e1754
baseline version:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc

Last test of basis   164944  2021-09-10 20:02:52 Z    2 days
Testing same since   164966  2021-09-13 13:02:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ian Jackson <iwj@xenproject.org>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6d45368a0a..a89bcd9737  a89bcd9737757e4d671783588a6041a84a5e1754 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 17:09:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 17:09:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185859.334619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPpST-0000Iw-3w; Mon, 13 Sep 2021 17:09:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185859.334619; Mon, 13 Sep 2021 17:09:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPpST-0000Ip-10; Mon, 13 Sep 2021 17:09:29 +0000
Received: by outflank-mailman (input) for mailman id 185859;
 Mon, 13 Sep 2021 17:09:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cPWG=OD=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mPpSR-0000F2-M6
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 17:09:27 +0000
Received: from mail-ej1-x630.google.com (unknown [2a00:1450:4864:20::630])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e689f226-86f1-4e66-aff0-eae49c29e68a;
 Mon, 13 Sep 2021 17:09:25 +0000 (UTC)
Received: by mail-ej1-x630.google.com with SMTP id qq21so16685881ejb.10
 for <xen-devel@lists.xenproject.org>; Mon, 13 Sep 2021 10:09:25 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id l11sm1185043edv.67.2021.09.13.10.09.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 13 Sep 2021 10:09:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e689f226-86f1-4e66-aff0-eae49c29e68a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=d/3P8gEjVO3pe7G/EMj7p3kp3t/iMclNRIETUDSO4og=;
        b=mn1T8nhsy4IFGX+lMfiNFugnU4UGkcZ4s5HmpeedcnDxhxL7qtm9+PC19MK88hXo8c
         WbnKilKUaaAxtilsBZmncT/emwvR/zhwPuQ77/eKFFoW3LPjaZpe3JbxvxOm52d7iOc7
         cC0zvIi/n79IdAPM8/ev372YaESl39J3QrQelzqu/Ykn5aU7bkhb6hTP3VLHPY2PgywI
         uVETW6QHUEIuDPGEDO7vlGdApprcETtsYgqAZSMn03VZOKFfvsdQSuo2i08T2oKldRqb
         h5VCbhe5mFqkAIj8xv3A6Q0AmNOpJvFX/3nxrhpwWxl5pCo+I3XoRWCuuaX2yHJFx6RB
         ++zA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=d/3P8gEjVO3pe7G/EMj7p3kp3t/iMclNRIETUDSO4og=;
        b=Nh8RNFi448m0g3A9tLeIuy1CYQ7ICf25tFpAI4+XRkHFJDsSPDsRyMt1OG2BRw1YMq
         NFm/QxKnCvpBbIHLn/1jFfdBF4mGnVzwLwVLeSav9g2HT8akW+b/ELWRDRZ+xFvj0N09
         rwzEoWNfE/Pck6mP395wHRZOdYQ01jeu4fSs3cYbp9/1Vd3//SPD90N0f5ZRsRC9XASb
         WD+VpRRI3/DrVAW8IdkPZGr/UySQ2I6MAYA6sQyJeoRCWnvCs31GpvTgsDVVt2rE+mTj
         ltcB+QLX3SB6XSKNBjI/6pCY+YmAA/Gdi5JX5XIBD/UlIVDxsYU/vvxbidXlk36A9DG1
         p/Ng==
X-Gm-Message-State: AOAM533T5p8OGaGb6v1FFdvZ4EndMI7j2jocAvYX4axELpXnHttt8AlU
	lVi59qBodA2vifsfle/BUfBxKnZF4SA=
X-Google-Smtp-Source: ABdhPJzxcBD2PkQkiN+zJ/ToPvj7UwMOeoENIJikRbgYqWs0A/R00fl8WEcZ96WRVeoYpheQ/8aaFw==
X-Received: by 2002:a17:906:c205:: with SMTP id d5mr13875465ejz.74.1631552964818;
        Mon, 13 Sep 2021 10:09:24 -0700 (PDT)
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <c286febc-b7b4-5aa4-5331-f0c2f6a17000@gmail.com>
Date: Mon, 13 Sep 2021 20:09:23 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 10.09.21 10:52, Jan Beulich wrote:


Hi Jan

> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>        */
>>       if ( p2m_is_foreign(pte.p2m.type) )
>>       {
>> -        mfn_t mfn = lpae_get_mfn(pte);
>> -
>>           ASSERT(mfn_valid(mfn));
>>           put_page(mfn_to_page(mfn));
>>       }
>> +
>> +#ifdef CONFIG_GRANT_TABLE
>> +    /*
>> +     * Check whether we deal with grant table page. As the grant table page
>> +     * is xen_heap page and its entry has known p2m type, detect it and mark
>> +     * the stored GFN as invalid.
>> +     */
>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>> +#endif
> I take it the write done is benign for other Xen heap pages? I suppose
> this would want making very explicit, as such an assumption is easy to
> go stale by entirely unrelated changes.

Yes, I don't see what bad could happen if we would clear this field for 
non grant table pages. Except grant table pages I could detect only one 
page passed this check here which is, I assume, shared_info page. All 
pages have this field initialized with INVALID_GFN. But *only* grant 
table pages have this field in use. So, I think, no one will suffer. I 
will add a comment. Or you meant something more than just a comment?


>
> I also wonder whether you really mean to include p2m_ram_ro pages here
> as well.

You are right, only p2m_ram_rw is our target.


>
>> --- a/xen/common/page_alloc.c
>> +++ b/xen/common/page_alloc.c
>> @@ -1021,6 +1021,7 @@ static struct page_info *alloc_heap_pages(
>>   
>>           /* Initialise fields which have other uses for free pages. */
>>           pg[i].u.inuse.type_info = 0;
>> +        page_arch_init(&pg[i]);
> u.type_info's count portion also gets checked upon freeing a page.
> Don't you want to have an arch-custom check for your new use of the
> field as well? Or do you consider it not a problem (bug) if a page
> was freed which still has a GFN set on it?

Hmm, I didn't think about it. Good point. I can't say for sure, but I 
don't think it would be a normal behavior.
The valid GFN field for the page to be freed would mean that something 
doesn't work as expected (the corresponding page table entry wasn't 
freed, or it was, but
we couldn't detect the grant table page, etc). I will probably add 
corresponding page_arch_free() which triggers BUG if GFN is still valid.

But, there is a moment with that. The u.type_info's count portion is 
checked in free_domheap_pages(). So, I think we need to have arch-custom 
check in code path for xenheap_page.

Something like that:

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 958ba0c..bb359ca 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2204,7 +2204,10 @@ void *alloc_xenheap_pages(unsigned int order, 
unsigned int memflags)
          return NULL;

      for ( i = 0; i < (1u << order); i++ )
+    {
          pg[i].count_info |= PGC_xen_heap;
+        arch_init_xenheap_page(&pg[i]);
+    }

      return page_to_virt(pg);
  }
@@ -2222,7 +2225,10 @@ void free_xenheap_pages(void *v, unsigned int order)
      pg = virt_to_page(v);

      for ( i = 0; i < (1u << order); i++ )
+    {
          pg[i].count_info &= ~PGC_xen_heap;
+        arch_free_xenheap_page(&pg[i]);
+    }

      free_heap_pages(pg, order, true);
  }


Where on Arm, these are:

+#define arch_init_xenheap_page(_p)   page_set_frame_gfn(_p, INVALID_GFN)
+#define arch_free_xenheap_page(_p) \
+    BUG_ON(!gfn_eq(page_get_frame_gfn(_p), INVALID_GFN))

And just stubs on x86.

Please let me know if you think otherwise.


>
>> @@ -82,11 +53,21 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
>>           : gnttab_shared_gfn(NULL, gt, idx);                              \
>>   })
>>   
>> -#define gnttab_shared_gfn(d, t, i)                                       \
>> -    (((i) >= nr_grant_frames(t)) ? INVALID_GFN : (t)->arch.shared_gfn[i])
>> +#define gnttab_shared_page(t, i)    \
>> +    mfn_to_page(_mfn(__virt_to_mfn((t)->shared_raw[i])))
>> +
>> +#define gnttab_status_page(t, i)    \
>> +    mfn_to_page(_mfn(__virt_to_mfn((t)->status[i])))
> I wonder whether you wouldn't want to at least ASSERT() here that
> the virtual address you start from is actually non-NULL.

Agree, will add.


>
>> --- a/xen/include/asm-arm/mm.h
>> +++ b/xen/include/asm-arm/mm.h
>> @@ -39,7 +39,15 @@ struct page_info
>>           /* Page is in use: ((count_info & PGC_count_mask) != 0). */
>>           struct {
>>               /* Type reference count and various PGT_xxx flags and fields. */
>> -            unsigned long type_info;
>> +            unsigned long type_info:4;
>> +
>> +            /*
>> +             * Stored GFN if page is used for grant table frame
>> +             * (bits 59(27)-0).
> Are these bit numbers correct? I thought Arm, like x86, counted bits
> from low to high (unlike PPC, for example)?

I think, these are correct.  Yes, Little Endian is used.


>
>> +             */
>> +#define PGT_FRAME_GFN_WIDTH      (BITS_PER_LONG - 4)
>> +#define PGT_INVALID_FRAME_GFN    _gfn((1UL << PGT_FRAME_GFN_WIDTH) - 1)
>> +            unsigned long frame_gfn:PGT_FRAME_GFN_WIDTH;
>>           } inuse;
>>           /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
>>           union {
>> @@ -94,12 +102,12 @@ struct page_info
>>   #define PG_shift(idx)   (BITS_PER_LONG - (idx))
>>   #define PG_mask(x, idx) (x ## UL << PG_shift(idx))
>>   
>> -#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
>> -#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>> -#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>> +#define PGT_none          (0UL << 3)  /* no special uses of this page   */
>> +#define PGT_writable_page (1UL << 3)  /* has writable mappings?         */
>> +#define PGT_type_mask     (1UL << 3)
>>   
>>    /* Count of uses of this frame as its current type. */
>> -#define PGT_count_width   PG_shift(2)
>> +#define PGT_count_width   1
>>   #define PGT_count_mask    ((1UL<<PGT_count_width)-1)
> Leaving just a single bit seems pretty risky to me, and I can't see
> why you do so. You need 52 bits on Arm64. Which means you have 12
> bits left. Why not use them in full? Even on Arm32 you have 3 bits
> available for the count afaict.

Only 1 bit in the type_info field is really used on Arm, but anyway ...


>
> Also there's a disconnect here: If anyone wanted to change the number
> of bits used for the various fields, each such change should require
> an adjustment in as few independent places as possible. That is, there
> shouldn't be multiple uses of literal 4 further up, and there also
> shouldn't be a hidden connection between that 4 and the PGT_* values
> here. I think you'd better express the GFN as such a PGT_* construct
> as well. And you better would keep using PG_mask() and PG_shift().

... I think, this indeed makes sense. If got your request correctly, we 
can avoid disconnect here
and reserve 3-bit field for count for both Arm32 and Arm64. The struct 
page_info's type_info field remains untouched.


diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index ded74d2..8b73e1c 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -99,8 +99,14 @@ struct page_info
  #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 
63.                 */

   /* Count of uses of this frame as its current type. */
-#define PGT_count_width   PG_shift(2)
-#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
+#define PGT_count_base    PG_shift(4)
+#define PGT_count_mask    PG_mask(7, 4)
+
+/* Stored in bits [27:0] or [59:0] GFN if page is used for grant table 
frame */
+#define PGT_gfn_width     PG_shift(4)
+#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
+
+#define PGT_INVALID_FRAME_GFN   _gfn(PGT_gfn_mask)

   /* Cleared when the owning guest 'frees' this page. */
  #define _PGC_allocated    PG_shift(1)
@@ -163,6 +169,22 @@ extern unsigned long xenheap_base_pdx;

  #define maddr_get_owner(ma) (page_get_owner(maddr_to_page((ma))))

+#define page_get_frame_gfn(_p) ({                               \
+    gfn_t gfn_ = _gfn((_p)->u.inuse.type_info & PGT_gfn_mask);  \
+    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
+})
+
+#define page_set_frame_gfn(_p, _gfn) ({                         \
+    gfn_t gfn_ = gfn_eq(_gfn, INVALID_GFN) ?                    \
+                 PGT_INVALID_FRAME_GFN : _gfn;                  \
+    (_p)->u.inuse.type_info &= ~PGT_gfn_mask;                   \
+    (_p)->u.inuse.type_info |= gfn_x(gfn_);                     \
+})

[snip]

Is it close to what you keep in mind?


The single use of type_info on Arm needs updating:

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index 0e07335..f306a93 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1376,14 +1376,18 @@ unsigned long domain_get_maximum_gpfn(struct 
domain *d)
  void share_xen_page_with_guest(struct page_info *page, struct domain *d,
                                 enum XENSHARE_flags flags)
  {
+    unsigned long type_info;
+
      if ( page_get_owner(page) == d )
          return;

      spin_lock(&d->page_alloc_lock);

      /* The incremented type count pins as writable or read-only. */
-    page->u.inuse.type_info =
-        (flags == SHARE_ro ? PGT_none : PGT_writable_page) | 1;
+    type_info = page->u.inuse.type_info & ~(PGT_type_mask | 
PGT_count_mask);
+    page->u.inuse.type_info = type_info |
+        (flags == SHARE_ro ? PGT_none : PGT_writable_page) |
+        (1UL << PGT_count_base);

      page_set_owner(page, d);
      smp_wmb(); /* install valid domain ptr before updating refcnt. */


>
>> @@ -163,6 +171,15 @@ extern unsigned long xenheap_base_pdx;
>>   
>>   #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
>>   
>> +#define page_get_frame_gfn(_p) ({                               \
>> +    gfn_t gfn_ = _gfn((_p)->u.inuse.frame_gfn);                 \
>> +    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
>> +})
>> +
>> +#define page_set_frame_gfn(_p, _gfn) ((_p)->u.inuse.frame_gfn = gfn_x(_gfn))
>> +
>> +#define page_arch_init(_p)   page_set_frame_gfn(_p, INVALID_GFN)
> What's the purpose of the leading underscore in the macro parameter
> names? They're not in line with the C standard's designation of sub-
> namespaces for identifiers. (At least for the x86 counterpart please
> read this as a request to drop the underscore there.)

ok


Thank you.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 18:26:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 18:26:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185873.334630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPqfA-0000Pb-M8; Mon, 13 Sep 2021 18:26:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185873.334630; Mon, 13 Sep 2021 18:26:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPqfA-0000PU-J5; Mon, 13 Sep 2021 18:26:40 +0000
Received: by outflank-mailman (input) for mailman id 185873;
 Mon, 13 Sep 2021 18:26:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5Zzy=OD=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mPqf8-0000PO-9S
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 18:26:39 +0000
Received: from forward108o.mail.yandex.net (unknown [37.140.190.206])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 202d31fb-14c0-11ec-b3c2-12813bfff9fa;
 Mon, 13 Sep 2021 18:26:36 +0000 (UTC)
Received: from sas1-13a2ee69ea33.qloud-c.yandex.net
 (sas1-13a2ee69ea33.qloud-c.yandex.net
 [IPv6:2a02:6b8:c08:b520:0:640:13a2:ee69])
 by forward108o.mail.yandex.net (Yandex) with ESMTP id 1C60C5DD49CD;
 Mon, 13 Sep 2021 21:26:34 +0300 (MSK)
Received: from sas1-f4dc5f2fc86f.qloud-c.yandex.net
 (sas1-f4dc5f2fc86f.qloud-c.yandex.net [2a02:6b8:c08:cb28:0:640:f4dc:5f2f])
 by sas1-13a2ee69ea33.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 RnCTFsLKqN-QXEOQGOn; Mon, 13 Sep 2021 21:26:34 +0300
Received: by sas1-f4dc5f2fc86f.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id rGWIcXSjtA-QVAmAHL1; Mon, 13 Sep 2021 21:26:32 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 202d31fb-14c0-11ec-b3c2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631557594;
	bh=l4Xy+6AjeOD9IsajbIycNQtuqAoIo+6fbHJfaRuKTg0=;
	h=In-Reply-To:References:Date:Subject:To:From:Message-Id:Cc;
	b=DAAoAQqL93KaYxImWzNLkdtDvSZ6BjFc1aKLWk4RTK2rV9PimrTFQauTJP/Dz8QEe
	 5DwSDZYbQgiapiwLqvIyotFBEZ0Aj6SO2AMiZmkj05Bd68KqohK9rc47HM75EVodwv
	 fzI9yGdqg+KZ54T8sUyT+FqrWiPz/wP/ELBWaZpU=
Authentication-Results: sas1-13a2ee69ea33.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
To: xen-devel@lists.xenproject.org
Cc: Dmitry Isaikin <isaikin-dmitry@yandex.ru>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 1/1] tools/xl: Fix regular expression for autobaloon mode detection based on xen command line.
Date: Mon, 13 Sep 2021 21:26:21 +0300
Message-Id: <9dc667d2017e6e69877a2b9859041c1f7d965899.1631555561.git.isaikin-dmitry@yandex.ru>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <cover.1631555561.git.isaikin-dmitry@yandex.ru>
References: <cover.1631555561.git.isaikin-dmitry@yandex.ru>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The case of specifying a negative size was handled incorrectly.

>From misc/xen-command-line documentation:

dom0_mem (x86)
= List of ( min:<sz> | max:<sz> | <sz> )

If a size is positive, it represents an absolute value.
If a size is negative, it is subtracted from the total available memory.
---
 tools/xl/xl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..a9f7e769fd 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -81,7 +81,7 @@ static int auto_autoballoon(void)
         return 1; /* default to on */
 
     ret = regcomp(&regex,
-                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
+                  "(^| )dom0_mem=((|min:|max:)-?[0-9]+[bBkKmMgG]?,?)+($| )",
                   REG_NOSUB | REG_EXTENDED);
     if (ret)
         return 1;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 19:57:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 19:57:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185894.334640 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPs4i-0001dV-IM; Mon, 13 Sep 2021 19:57:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185894.334640; Mon, 13 Sep 2021 19:57:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPs4i-0001dO-FY; Mon, 13 Sep 2021 19:57:08 +0000
Received: by outflank-mailman (input) for mailman id 185894;
 Mon, 13 Sep 2021 19:57:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cPWG=OD=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mPs4g-0001dI-Sr
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 19:57:06 +0000
Received: from mail-ej1-x62f.google.com (unknown [2a00:1450:4864:20::62f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae909225-fe3a-4fed-bd4e-d1fb5c7621fa;
 Mon, 13 Sep 2021 19:57:05 +0000 (UTC)
Received: by mail-ej1-x62f.google.com with SMTP id qq21so17671391ejb.10
 for <xen-devel@lists.xenproject.org>; Mon, 13 Sep 2021 12:57:05 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id y8sm1700016ejm.104.2021.09.13.12.57.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 13 Sep 2021 12:57:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae909225-fe3a-4fed-bd4e-d1fb5c7621fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=nrq7NFlO2EfcXfM/mHT2LWBk1VgcUh3w7vl0T8rQAKY=;
        b=HmM69VptWPMRUpvskAeXYRy5difWYuf6Y8dpFISsSFY2otng3YqkQmqrwfvdzgUgYr
         bmw32cO5V5RdNKXo6VFzawqXx3lCKnupvsvZQtApjLA6INZRoOq1nNdlB3e6HynSfa22
         FDcZ5HzbRZcnUhz/Iuzpo24MdZX2yhsCMX/ocmWsaX7BVhDTSO2nTMRoVHCoe/p2ATiM
         PHA6MU4EaTuvN5MFzH4Iy+U96XqnHqj8dp9wg3l59U0G8l4vV5/NM0v7MI+OprXgLGr/
         RengbaeacvCtIyCuQYI2oCZkcPyaq8vy/xgoPcunJoUBAwpgzxEoSQR4X1wvJ4QyGNga
         1fGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=nrq7NFlO2EfcXfM/mHT2LWBk1VgcUh3w7vl0T8rQAKY=;
        b=Bwqxk7jUa2tpJapzfKpZJCeButYK1sWoY3MXrzH3SEKzg0Q09NWgSpHXmBoaLtsj2c
         EpNFle8M7hhz6u0KAWeHJ3JFu38x4Ww55pUjieAjSWWOJu1bxuulVa4mMz4D6gPrW8Q/
         7SPE4uLJmajpE0zgOa/B6G0CKq4wOyrawgaXfvb9CtqVhIldOF22l1NCRcaE2Xv/u6bT
         KU6Wq4BvvewpiADR8dtFDmZYxMIkHt5M6ROYemxpfBnHCmqjvrI4gECp0pdpwtc6wlXI
         BHdDdIUXYfqdgFi+5ZlqRBreVvs/sfGC+gLWsNz2uSigJqS48V8ybtICcWNzUY3jEs1K
         0Q2w==
X-Gm-Message-State: AOAM531ZmRpdGS2Oe8UfMxHgxb0sjpMb5GazuiMCbmUqa0wHTxb60s31
	oZxwt1dalIku9eldpOUMqfINchj0NyA=
X-Google-Smtp-Source: ABdhPJzKpfqz6MeVA8O1m1q131Cx+FM5i5zD32WSVEK9xg2YSMPhaMsiuEghPzMk5meCU0ccZy2+RQ==
X-Received: by 2002:a17:906:85d8:: with SMTP id i24mr4404518ejy.451.1631563024565;
        Mon, 13 Sep 2021 12:57:04 -0700 (PDT)
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
 <c006550f-4fe7-29c5-f370-dc67bfac2b95@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <f9e95fd9-d63e-675b-0236-653afee617b1@gmail.com>
Date: Mon, 13 Sep 2021 22:57:02 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <c006550f-4fe7-29c5-f370-dc67bfac2b95@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 13.09.21 09:17, Jan Beulich wrote:

Hi Jan

> On 10.09.2021 09:52, Jan Beulich wrote:
>> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>>        */
>>>       if ( p2m_is_foreign(pte.p2m.type) )
>>>       {
>>> -        mfn_t mfn = lpae_get_mfn(pte);
>>> -
>>>           ASSERT(mfn_valid(mfn));
>>>           put_page(mfn_to_page(mfn));
>>>       }
>>> +
>>> +#ifdef CONFIG_GRANT_TABLE
>>> +    /*
>>> +     * Check whether we deal with grant table page. As the grant table page
>>> +     * is xen_heap page and its entry has known p2m type, detect it and mark
>>> +     * the stored GFN as invalid.
>>> +     */
>>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>>> +#endif
>> I take it the write done is benign for other Xen heap pages? I suppose
>> this would want making very explicit, as such an assumption is easy to
>> go stale by entirely unrelated changes.
>>
>> I also wonder whether you really mean to include p2m_ram_ro pages here
>> as well.
> Actually I've meanwhile realized I should put my question here quite
> differently: Aren't you effectively introducing an M2P here for Arm,
> except that you artificially limit it to the Xen heap pages needed for
> the grant table?

Difficult to say, it might indeed look a bit close to M2P, so the answer 
to your question is more yes than no. But, I didn't have a plan to 
introduce M2P on Arm. It turned out that stashing GFN into page_info (as 
was suggested) avoided huge lookups, so we have got MFN->GFN in the end. 
The purpose of this patch was just to fix a potential issue with 
remapping grant-table frame on architecture without the M2P (Arm).


>
> Jan
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 13 20:24:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 20:24:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185904.334656 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsUg-00057d-Qt; Mon, 13 Sep 2021 20:23:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185904.334656; Mon, 13 Sep 2021 20:23:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsUg-00057W-Nq; Mon, 13 Sep 2021 20:23:58 +0000
Received: by outflank-mailman (input) for mailman id 185904;
 Mon, 13 Sep 2021 20:23:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPsUf-00057Q-3j
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:23:57 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 84e91ea0-14d0-11ec-b3c9-12813bfff9fa;
 Mon, 13 Sep 2021 20:23:56 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 301526108B;
 Mon, 13 Sep 2021 20:23:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84e91ea0-14d0-11ec-b3c9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631564635;
	bh=/YgwrBiEzvCww0bnA51UetOY6a+RunqT1PHF+2GBSw0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=BjBrqeaJ7Y26vB/hKo0WgbnPFuYcaqr9H/0jHVzmqY4q2RZlAs2CS3ZzGMkG0Zlv1
	 +0moAul+iRBsTCCPpx2Bakto1OwcFXCvwMGHXHnlFRDmmgYcf4to1rk8QFLGNZ737X
	 4ILP1CBhEHdZOhnK+n8sd2OJ3qUj77uUzQwJ6h3byzgQfPPBHBRFmYANpfyDHXP0rz
	 l6GvriwZStoXC3JFCxA3yIwPul7GHJcrryKt8dxpmA7sReVUd1e72zYoY4UO3pxOGS
	 Nq6EfPvkh6X4DTz33x4k02WwQyEd12cWs/+HM+tifeOz8j4jZUEr95AsyRsOddeNqO
	 ylr5cGmkkxunw==
Date: Mon, 13 Sep 2021 13:23:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Rahul Singh <rahul.singh@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <c9484b8e-cad5-d4e4-a25f-749035fe5859@epam.com>
Message-ID: <alpine.DEB.2.21.2109131322190.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com> <c9484b8e-cad5-d4e4-a25f-749035fe5859@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1927430885-1631564635=:10523"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1927430885-1631564635=:10523
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 13 Sep 2021, Oleksandr Andrushchenko wrote:
> Hi, Rahul!
> 
> On 19.08.21 15:02, Rahul Singh wrote:
> > XEN during boot will read the PCI device tree node “reg” property
> > and will map the PCI config space to the XEN memory.
> [snip]
> > +static struct pci_config_window *gen_pci_init(struct dt_device_node *dev,
> > +                                              int ecam_reg_idx)
> > +{
> > +    int err;
> > +    struct pci_config_window *cfg;
> > +    paddr_t addr, size;
> > +
> > +    cfg = xzalloc(struct pci_config_window);
> > +    if ( !cfg )
> > +        return NULL;
> > +
> > +    err = dt_pci_parse_bus_range(dev, cfg);
> > +    if ( !err ) {
> > +        cfg->busn_start = 0;
> > +        cfg->busn_end = 0xff;
> > +        printk(XENLOG_ERR "%s:No bus range found for pci controller\n",
> > +               dt_node_full_name(dev));
> > +    } else {
> > +        if ( cfg->busn_end > cfg->busn_start + 0xff )
> > +            cfg->busn_end = cfg->busn_start + 0xff;
> > +    }
> > +
> > +    /* Parse our PCI ecam register address*/
> > +    err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size);
> 
> I am a bit worried here that we don't get the reg index from the device tree,
> 
> but for generic ECAM we use reg[0] and for Xilinx we use reg[2].
> 
> For example, for Xilinx we have
> 
> reg = <0x00 0xfd0e0000 0x00 0x1000 0x00 0xfd480000 0x00 0x1000 0x80 0x00 0x00 0x1000000>;
> reg-names = "breg\0pcireg\0cfg";
> 
> so, we can parse the reg-names and understand that the configuration space is the last in the reg property.
> 
> The same I think can be done for other device trees probably.

Well spotted!


> Rahul, do you know if reg-names "cfg" is vendor specific of used widely?

Unfortunately it seems to be vendor specific :-(
Which means that "dt_device_get_address" or similar should be moved to a
vendor specific function.
--8323329-1927430885-1631564635=:10523--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 20:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 20:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185915.334667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsbe-0006ia-N3; Mon, 13 Sep 2021 20:31:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185915.334667; Mon, 13 Sep 2021 20:31:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsbe-0006iT-JK; Mon, 13 Sep 2021 20:31:10 +0000
Received: by outflank-mailman (input) for mailman id 185915;
 Mon, 13 Sep 2021 20:31:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPsbd-0006iN-5E
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:31:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 869cc9da-14d1-11ec-b3cb-12813bfff9fa;
 Mon, 13 Sep 2021 20:31:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9048C6108B;
 Mon, 13 Sep 2021 20:31:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 869cc9da-14d1-11ec-b3cb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631565067;
	bh=I1luLBrQT2QS6nSSKLDDdX9lKRGVXUWe967myVRKHq8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NkMwMKqzji1BQ9efEE20R8HGJLZmk2XracuUWD04u26bu5/zQYoT51EuIpnj8a98Y
	 XBMpos47u9CxnpLg4uCQ1HZ1GdVhmR/KIqMQQhR6m/MhF5B6JxbD5eKcm9DRBW7Iht
	 jxPJOcjcvS4UU3Saa6uy6ggSVWWJxZxuUplqTAyMol9C0a1Zkx5dlBJwdYYs09wp/x
	 Ax6gcedHW/T+qr+IViyy00YeVYY1+za2CK6cRf6KI+9CHS+FoPkhMM58+fDGeNGNPm
	 Pui7DgrkZnxXiaK6qWwnHK4FtowGTMImhm8nMMhb2AfgMe5+6ZtgegMOrY2Rvx8xuZ
	 9n+jW82xbiLwA==
Date: Mon, 13 Sep 2021 13:31:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
In-Reply-To: <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com>
Message-ID: <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <397bf325-f81e-e104-6142-e8c9c4955475@suse.com> <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s> <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 13 Sep 2021, Jan Beulich wrote:
> On 11.09.2021 01:14, Stefano Stabellini wrote:
> > On Tue, 7 Sep 2021, Jan Beulich wrote:
> >> While the hypervisor hasn't been enforcing this, we would still better
> >> avoid issuing requests with GFNs not aligned to the requested order.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> I wonder how useful it is to include the alignment in the panic()
> >> message.
> > 
> > Not very useful given that it is static. I don't mind either way but you
> > can go ahead and remove it if you prefer (and it would make the line
> > shorter.)
> > 
> > 
> >> I further wonder how useful it is to wrap "bytes" in
> >> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
> >> least was supposed to be, prior to "swiotlb-xen: maintain slab count
> >> properly").
> > 
> > This one I would keep, to make sure to print out the same amount passed
> > to memblock_alloc.
> 
> Oh - if I was to drop it from the printk(), I would have been meaning to
> also drop it there. If it's useless, then it's useless everywhere.

That's fine too


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 20:49:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 20:49:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185924.334677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPst2-00005B-5z; Mon, 13 Sep 2021 20:49:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185924.334677; Mon, 13 Sep 2021 20:49:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPst2-000050-2r; Mon, 13 Sep 2021 20:49:08 +0000
Received: by outflank-mailman (input) for mailman id 185924;
 Mon, 13 Sep 2021 20:49:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPst1-00004u-Ix
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:49:07 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 092b0662-14d4-11ec-b3cc-12813bfff9fa;
 Mon, 13 Sep 2021 20:49:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6463A610FE;
 Mon, 13 Sep 2021 20:49:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 092b0662-14d4-11ec-b3cc-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631566145;
	bh=hUiqmRGtEXepWEODLRwwvcp5j41ZXDQIlBCemxiVKNw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DHwsMX7ikvWdd/BI8zDeHuw+7Tapq9FDRv/8U/Wc5GJhsdDEr0PZBkADcNC3Cw4/l
	 XVIYVS3fpf4IB/pEf/TBDYctfkgqE5TknKGwK5w28r1T0PfvZgGxfh5u4emxheYskE
	 tfJGnP6f/saAr4DBa9u2FuM//73eD3+3Ma38STWt9N64jckrDfpLuZsst7TasKtIXx
	 vFlplwL562zn3HK9rwxFYC1y4bQe7btGfdnMb5FDi/TilVePV2YX60eoSVjbN+C9Oj
	 eGh0ylJCU9jw2iP9ocX1Qp0BMnuv15otYWsJrZYReV/tsLjmDecY0Nc2ev8c2d0QaN
	 wqK6HMy1fcSRg==
Date: Mon, 13 Sep 2021 13:49:04 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    the arch/x86 maintainers <x86@kernel.org>, 
    Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, 
    Borislav Petkov <bp@alien8.de>, Christoph Hellwig <hch@infradead.org>
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
In-Reply-To: <ea90d3d7-22c7-604f-d0c8-f83c2a58554d@suse.com>
Message-ID: <alpine.DEB.2.21.2109131331570.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com> <YThiyxG0d2tmCtb+@infradead.org> <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s> <ea90d3d7-22c7-604f-d0c8-f83c2a58554d@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 13 Sep 2021, Jan Beulich wrote:
> On 11.09.2021 01:48, Stefano Stabellini wrote:
> > On Wed, 8 Sep 2021, Christoph Hellwig wrote:
> >> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
> >>> The code is unreachable for HVM or PVH, and it also makes little sense
> >>> in auto-translated environments. On Arm, with
> >>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
> >>> time seeing what good the Xen specific variant does - the generic one
> >>> ought to be fine for all purposes there. Still Arm code explicitly
> >>> references symbols here, so the code will continue to be included there.
> >>
> >> Can the Xen/arm folks look into that?  Getting ARM out of using
> >> swiotlb-xen would be a huge step forward cleaning up some DMA APIs.
> > 
> > On ARM swiotlb-xen is used for a different purpose compared to x86.
> > 
> > Many ARM SoCs still don't have an IOMMU covering all DMA-mastering
> > devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest
> > physical == physical address).
> > 
> > Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need
> > swiotlb-xen. But when we start using PV drivers to share the network or
> > disk between Dom0 and DomU we are going to get DomU pages mapped in
> > Dom0, we call them "foreign pages".  They are not mapped 1:1. It can
> > happen that one of these foreign pages are used for DMA operations
> > (e.g. related to the NIC). swiotlb-xen is used to detect these
> > situations and translate the guest physical address to physical address
> > of foreign pages appropriately.
> 
> Thinking about this some more - if Dom0 is 1:1 mapped, why don't you
> map foreign pages 1:1 as well then?

That's because the foreign page, from Linux POV, would appear out of
thin air. It would just show up in a region not considered memory just
few moments before, so there would be no memblock, no struct page,
nothing. At least in the past that caused serious issues to the kernel.

This is the reason why the kernel is using ballooned-out pages to map
foreign pages even on x86:

drivers/block/xen-blkback/blkback.c:xen_blkbk_map -> gnttab_page_cache_get
drivers/xen/grant-table.c:gnttab_page_cache_get
drivers/xen/grant-table.c:gnttab_alloc_pages
drivers/xen/unpopulated-alloc.c:xen_alloc_unpopulated_pages
drivers/xen/balloon.c:alloc_xenballooned_pages


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 20:54:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 20:54:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185933.334689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsyN-0001Up-Qy; Mon, 13 Sep 2021 20:54:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185933.334689; Mon, 13 Sep 2021 20:54:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPsyN-0001Ui-NM; Mon, 13 Sep 2021 20:54:39 +0000
Received: by outflank-mailman (input) for mailman id 185933;
 Mon, 13 Sep 2021 20:54:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPsyM-0001Uc-8D
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 20:54:38 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b78795dd-aeee-4732-87d8-3aca28a901fd;
 Mon, 13 Sep 2021 20:54:37 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5A9E060F38;
 Mon, 13 Sep 2021 20:54:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b78795dd-aeee-4732-87d8-3aca28a901fd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631566476;
	bh=7L/OWAO6BBlfagMy79zsuNYBLLG3xgrbcgKW6Hv1xI8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rTmhVuZEryO4b6nAFbM2j0Y+fpPjuBAnsuBWQSLTJy2HvZCp5g0gtsbc6rr9nZGZY
	 NB5yCW14K1MhARAk9GBo2g28JnVcWLUbcWNNzN8GU8uGW9xpbTx7zlZSjIq+BTF4K4
	 Swl147x3L5jeFXmm36e/MdXHlz1bjDPTABEXeCGbua/uldJAXcT6yWYjimvafyZBVJ
	 KclzL6D66SpyEmbj21obt9chK504MofCESbbiBVGZMuzEX6fdBI6glfBxi5y5j6FG+
	 GQ1hbdJuB7SiH3q7EndmHdZjacQ+Psv5SNM7EA6uNqm4JhxxoOhqXHSNYzdUEorQ6h
	 LOK4KNDJ/74kA==
Date: Mon, 13 Sep 2021 13:54:35 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    the arch/x86 maintainers <x86@kernel.org>, 
    Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, 
    Borislav Petkov <bp@alien8.de>, Christoph Hellwig <hch@infradead.org>
Subject: Re: [PATCH 12/12] swiotlb-xen: this is PV-only on x86
In-Reply-To: <543b446b-0143-ad32-99ed-d7a6f79381e8@suse.com>
Message-ID: <alpine.DEB.2.21.2109131350150.10523@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <004feaef-f3bb-e4bb-fb10-f205a9f69f28@suse.com> <YThiyxG0d2tmCtb+@infradead.org> <alpine.DEB.2.21.2109101636470.10523@sstabellini-ThinkPad-T480s> <543b446b-0143-ad32-99ed-d7a6f79381e8@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 13 Sep 2021, Jan Beulich wrote:
> On 11.09.2021 01:48, Stefano Stabellini wrote:
> > On Wed, 8 Sep 2021, Christoph Hellwig wrote:
> >> On Tue, Sep 07, 2021 at 02:13:21PM +0200, Jan Beulich wrote:
> >>> The code is unreachable for HVM or PVH, and it also makes little sense
> >>> in auto-translated environments. On Arm, with
> >>> xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
> >>> time seeing what good the Xen specific variant does - the generic one
> >>> ought to be fine for all purposes there. Still Arm code explicitly
> >>> references symbols here, so the code will continue to be included there.
> >>
> >> Can the Xen/arm folks look into that?  Getting ARM out of using
> >> swiotlb-xen would be a huge step forward cleaning up some DMA APIs.
> > 
> > On ARM swiotlb-xen is used for a different purpose compared to x86.
> > 
> > Many ARM SoCs still don't have an IOMMU covering all DMA-mastering
> > devices (e.g. Raspberry Pi 4). As a consequence we map Dom0 1:1 (guest
> > physical == physical address).
> > 
> > Now if it was just for Dom0, thanks to the 1:1 mapping, we wouldn't need
> > swiotlb-xen. But when we start using PV drivers to share the network or
> > disk between Dom0 and DomU we are going to get DomU pages mapped in
> > Dom0, we call them "foreign pages".  They are not mapped 1:1. It can
> > happen that one of these foreign pages are used for DMA operations
> > (e.g. related to the NIC). swiotlb-xen is used to detect these
> > situations and translate the guest physical address to physical address
> > of foreign pages appropriately.
> 
> Hmm, you say "translate", which isn't my understanding of swiotlb's
> purpose. As per my understanding swiotlb instead double buffers data
> such that is becomes accessible, or suitably arranges underlying
> machine addresses. The latter part is clearly a PV-only thing, unused
> by Arm as can be seen by there not being any use of XENMEM_exchange.
> So it must be the former part that you're talking about, but that's
> also the purpose of the non-Xen swiotlb code. If only for my own
> education and understanding, could you point me at the difference
> between swiotlb-xen and generic swiotlb which addresses this specific
> aspect of Arm behavior?

If you look at xen_swiotlb_map_page, you'll see the call to
xen_phys_to_dma which eventually calls arch/arm/xen/p2m.c:__pfn_to_mfn.
If everything goes well and we only need to do translation we'll "goto
done". Otherwise, we'll fall back on a swiotlb buffer with
swiotlb_tbl_map_single, the result of which also needs to be translated,
see the second call to xen_phys_to_dma.


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 21:02:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 21:02:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185943.334699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPt5Y-00034o-IB; Mon, 13 Sep 2021 21:02:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185943.334699; Mon, 13 Sep 2021 21:02:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPt5Y-00034h-FJ; Mon, 13 Sep 2021 21:02:04 +0000
Received: by outflank-mailman (input) for mailman id 185943;
 Mon, 13 Sep 2021 21:02:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPt5W-00034L-VU
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 21:02:03 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d1506a6a-28ec-4e83-844a-ea869dd11cd4;
 Mon, 13 Sep 2021 21:02:01 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9A2BA60698;
 Mon, 13 Sep 2021 21:02:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1506a6a-28ec-4e83-844a-ea869dd11cd4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631566920;
	bh=10DfJjt/WDE3/J2yJDqYWXAs7Kfw/Peuw1zK53nTjlc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dhBtRFtPoRvZXpSU7C4Wkn1PXv2JgJBSZRZPaZL947uG3mngZ0wQPFQWsCD/NTdrL
	 nK0gAyZfnN2t7mfiE+EreS+PPeTHd4W0/fIqujNpiDFl52VsiAAAfYP2z3CPd9d0+j
	 XzyOk5KxqNiIElWaYO9cBZn1BzxbTIDE/IrHfTnGDQH5/8O2jzV5MYm8lsJI8pXmd4
	 6IM9DzBlRw0QnUood2YpKcIV5LB1WzLt3VbmfScLyjlDugFyEls28HgdCBJncrgHNO
	 DAcFtOGe0bJCOvilhapMiAbWRV7rO11cMCFmh1jZCsGnR/3ojNIeP6l3ktFB0g7ar4
	 7xiH37NLJ3gtg==
Date: Mon, 13 Sep 2021 14:02:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Rahul Singh <rahul.singh@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
In-Reply-To: <aa251354-3e5f-e1ae-2647-3a112ad5d12e@epam.com>
Message-ID: <alpine.DEB.2.21.2109131356090.10523@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s> <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
 <aa251354-3e5f-e1ae-2647-3a112ad5d12e@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-351725318-1631566589=:10523"
Content-ID: <alpine.DEB.2.21.2109131356330.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-351725318-1631566589=:10523
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109131356331.10523@sstabellini-ThinkPad-T480s>

On Mon, 13 Sep 2021, Oleksandr Andrushchenko wrote:
> On 10.09.21 15:01, Rahul Singh wrote:
> > Hi Stefano,
> >
> >> On 10 Sep 2021, at 12:34 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>
> >> On Thu, 19 Aug 2021, Rahul Singh wrote:
> >>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>>
> >>> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
> >>> space to the XEN memory.
> >>>
> >>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>> ---
> >>> xen/arch/arm/pci/Makefile          |  1 +
> >>> xen/arch/arm/pci/pci-host-zynqmp.c | 59 ++++++++++++++++++++++++++++++
> >>> 2 files changed, 60 insertions(+)
> >>> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
> >>>
> >>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> >>> index 6f32fbbe67..1d045ade01 100644
> >>> --- a/xen/arch/arm/pci/Makefile
> >>> +++ b/xen/arch/arm/pci/Makefile
> >>> @@ -3,3 +3,4 @@ obj-y += pci-access.o
> >>> obj-y += pci-host-generic.o
> >>> obj-y += pci-host-common.o
> >>> obj-y += ecam.o
> >>> +obj-y += pci-host-zynqmp.o
> >>> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> >>> new file mode 100644
> >>> index 0000000000..fe103e3855
> >>> --- /dev/null
> >>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> >>> @@ -0,0 +1,59 @@
> >>> +/*
> >>> + * Copyright (C) 2020-2021 EPAM Systems
> >>> + *
> >>> + * Based on Linux drivers/pci/controller/pci-host-common.c
> >>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> >>> + * Based on xen/arch/arm/pci/pci-host-generic.c
> >>> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
> >> Only one Copyright line per file is enough :-)
> >>
> >> But actually all the Copyright lines with a name or a company name are
> >> not really required or useful, as the copyright is noted in full details
> >> in the commit messages (author and signed-off-by lines). I would remove
> >> them all from the new files added by this series.
> > Ok. Let me remove in next version.
> >>
> >>> + * This program is free software; you can redistribute it and/or modify
> >>> + * it under the terms of the GNU General Public License version 2 as
> >>> + * published by the Free Software Foundation.
> >>> + *
> >>> + * This program is distributed in the hope that it will be useful,
> >>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> >>> + * GNU General Public License for more details.
> >>> + *
> >>> + * You should have received a copy of the GNU General Public License
> >>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> >>> + */
> >>> +
> >>> +#include <asm/device.h>
> >>> +#include <xen/pci.h>
> >>> +#include <asm/pci.h>
> >>> +
> >>> +static const struct dt_device_match gen_pci_dt_match[] = {
> >>> +    { .compatible = "xlnx,nwl-pcie-2.11",
> >>> +      .data =       &pci_generic_ecam_ops },
> >>> +    { },
> >>> +};
> >>> +
> >>> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data)
> >>> +{
> >>> +    const struct dt_device_match *of_id;
> >>> +    const struct pci_ecam_ops *ops;
> >>> +
> >>> +    of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node);
> >> This should be superfluous
> > Ack. I will remove the dt_match_node(..) in next version.
> 
> I am not entirely sure we need this patch at all as the main reason for its existence
> 
> was that we can run Xilinx QEMU for ZCU102. But, the final setup is not going
> 
> to be functional as legacy IRQs are not supported and ITS is not a part of ZynqMP.
> 
> So, QEMU allows to do a lot with PCI passthrough, but at the end of the day one
> 
> won't have it working...
> 
> Please consider
> 
> If we decide to remove it then
> 
> int pci_host_common_probe(struct dt_device_node *dev,
>                            const struct pci_ecam_ops *ops,
>                            int ecam_reg_idx)
> 
> doesn't need the last parameter.

With my open source maintainer hat on, I don't see this patch as very
important; from that point of view I'd be happy for it to be dropped.
However, it would be good to have at least one non-default host bridge
(doesn't have to be the Xilinx bridge), otherwise it becomes difficult
to understand how the generic infrastructure introduced by this series
could be useful.

Moreover, your recent comment [1] made it even more evident that it
would be good to have at least two different drivers to spot
compatibility issues between them more easily.

[1] https://marc.info/?l=xen-devel&m=163154474008598 
--8323329-351725318-1631566589=:10523--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 21:38:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 21:38:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185956.334710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPteZ-0006oy-Ev; Mon, 13 Sep 2021 21:38:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185956.334710; Mon, 13 Sep 2021 21:38:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPteZ-0006or-C3; Mon, 13 Sep 2021 21:38:15 +0000
Received: by outflank-mailman (input) for mailman id 185956;
 Mon, 13 Sep 2021 21:38:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPteY-0006ol-64
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 21:38:14 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e58adb22-14da-11ec-b3d0-12813bfff9fa;
 Mon, 13 Sep 2021 21:38:13 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 41FF1610CC;
 Mon, 13 Sep 2021 21:38:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e58adb22-14da-11ec-b3d0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631569092;
	bh=Rdv0jKDD++IGTZnpLfELey+D0L8ESYDKerKIBVDBBvw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=aNGistF5nR8gdC4CY6xsJSDI2+bIF3shn7XBhg/PGpLBojUbwcauauwH3IhnOUb2q
	 CVmfjK+xBzjOhR1iB33X6nEDY9L/DD6x0BsG26qMeAqbKQtgygyg7pv4hSy4gBiOT3
	 lrUtsVl4lLtA+6ZHl8jletGKvllaXTOv6gtTqQRcTH7KTuaVJ7de0djrobmsFQHa3L
	 2+g038VR05s9Eficuh9Kdn3DCdZU08Q4ZokeR6P40H3Q7zqX6iimfO5N43Hn3R6m+n
	 2c99Jj4lub37SiT7L2ZzbXxKZIWVqn612ekjsnRExORLywKQLdzY4X4NpDjVaNDgLW
	 h3TzhWNqRX70Q==
Date: Mon, 13 Sep 2021 14:38:11 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Sai Kiran Kumar Reddy Y <ysaikiran1997@gmail.com>
cc: Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    kannan@cimware.in
Subject: Re: Xen Booting Problem on ARM Machine
In-Reply-To: <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.2109131437030.10523@sstabellini-ThinkPad-T480s>
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol> <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org> <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1012743033-1631569092=:10523"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1012743033-1631569092=:10523
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 13 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
> On Fri, Sep 10, 2021 at 7:30 PM Julien Grall <julien@xen.org> wrote:
> 
> 
>       On 08/09/2021 11:43, Sai Kiran wrote:
>       > Hello,
> 
>       Hi,
> 
>       Thank you for the report. Moving the discussion back to xen-devel
>       because this looks like a potential issue in the UEFI stub in Xen.
> 
>       > I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it.
> 
>       Would you be able to give more details on the Arm machine you are using?
>       Also, are you using ACPI or DT to boot?
> 
> 
>    Sai >> DT . ACPI configuration is disabled in Boot settings    
> 
>       >  I
>       > am able to do “make world” and “make install”, after “./configure”, as
>       > specified in README file. When I reboot the system, I get the following
>       > message:
>       >
>       > Warning: All 128 bootinfo mem banks exhausted.
>       >
>       > Warning: All 128 bootinfo mem banks exhausted.
> 
>       Hmmm... This means that you have more than 128 memory regions described
>       in the EFI memory map. That's quite a lot.
> 
>       Although, this should be harmless as it means Xen will not use the extra
>       memory banks.
> 
>       >
>       > Cannot exit boot services: ErrCode: 0x8000000000000002
> 
>       This means EFI_INVALID_PARAMETER. We have code to retry because AFAICT
>       ExitBootServices() may sometime fails (I have CCed Jan may have more
>       idea what's happening).
> 
>       Would you be able to provide more details on the UEFI firmware you are
>       using? Is it EDK2 or U-boot?
> 
> Sai >>  EDK2 
>       Also, do you know if Linux is boot on the same system?
> 
> Sai >> Yes 
>       However, AFAICT, the error message would not prevent Xen to continue
>       booting. So you may get stuck later in the boot process.
> 
>       My suggestion would be to enable earlyprintk for your platform. You can
>       setup it up from the menuconfig in "Debugging Options".
> 
> Sai >> Yes, I have enabled earlyprintk.
> I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to 256, from 128. The error message is no longer seen, but the device
> is stuck in the boot process.

Could you please post the boot logs now that you enabled earlyprintk?
Ideally not a camera picture but a textual copy/paste from the target
serial?

Earlyprintk is pretty verbose, we should be able to figure out where it
gets stuck.
--8323329-1012743033-1631569092=:10523--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 22:42:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 22:42:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185979.334722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPueG-0005dn-9V; Mon, 13 Sep 2021 22:42:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185979.334722; Mon, 13 Sep 2021 22:42:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPueG-0005dg-5O; Mon, 13 Sep 2021 22:42:00 +0000
Received: by outflank-mailman (input) for mailman id 185979;
 Mon, 13 Sep 2021 22:41:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+zYr=OD=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPueF-0005da-JS
 for xen-devel@lists.xenproject.org; Mon, 13 Sep 2021 22:41:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a117f107-84ac-475c-b895-4fad6ab2bda8;
 Mon, 13 Sep 2021 22:41:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6BC8560EE5;
 Mon, 13 Sep 2021 22:41:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a117f107-84ac-475c-b895-4fad6ab2bda8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631572917;
	bh=i4LeucUZLkdN7pXKmEjyTssQ6r/JJQqxLWYz7r6pC3k=;
	h=Date:From:To:cc:Subject:From;
	b=JsNe0fliB/whyFslve9Un8WbgWFs0Wm2ZlE/vynYnDzdvoEm6Mtx82r65EFEtGs94
	 k6f/Hd79mdwZDDALsTtoj0SHwPlYXM6/9QWc2w6EuMeoQmm4ZE1I7kJRVhwcRPRxdw
	 7HkVH5ffMuu8MVL+oyoK9FFkoZAZ2PD3+26ljnGqQYxpNnbOcgh8edcsNdNjqZ6JXi
	 KGnclL7NLMRtz8M9qMOm1ArQudtRghefiD3lcroIJ65DiduS1RmyoveOkUeCPM4LZC
	 c06gE+EvtrcpHFGRpetEoFGFa+uwqN0Qyna5q1yG+AsRm1oL/LWBpXpsrP74VWZ0oA
	 8e3d+1NsgOtfA==
Date: Mon, 13 Sep 2021 15:41:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, Bertrand.Marquis@arm.com, 
    Alec.Kwapis@dornerworks.com, christopher.w.clark@gmail.com
Subject: Dom0less + Argo enablement
Message-ID: <alpine.DEB.2.21.2109131441520.10523@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-587971660-1631570062=:10523"
Content-ID: <alpine.DEB.2.21.2109131454340.10523@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-587971660-1631570062=:10523
Content-Type: text/plain; CHARSET=US-ASCII
Content-ID: <alpine.DEB.2.21.2109131454341.10523@sstabellini-ThinkPad-T480s>

Hi all,

This email is for anybody interested in using Argo with Dom0less setups
for domain-to-domain communications.

Argo is a secure VM-to-VM communication mechanism based on hypercalls
[1]. It is a good fit for Dom0less setups because Argo is lightweight
and doesn't make use of PV frontends, backends, and xenstore. Thus, it
is easier to enable in simple Dom0less setups where VMs are booting in
parallel, backends cannot be assumed to be already up and running, and
the goal is just to get two VMs to talk to each others.

Argo makes use of event channels for notifications. Dom0less VMs don't
have event channels support yet, although it is work-in-progress. The
attached patch for Linux (not ready for upstreaming) enables the
necessary event channels initialization.

In addition, you also need to add the following to the DomU device tree
(see xen/arch/arm/domain_build.c:prepare_dtb_domU):

event-channel {
        compatible = "xen,xen-4.15", "xen,xen";
        interrupts = <0x01 0x0f 0xf08>;
        interrupt-parent = <0xfde8>;
};


The final pieces to enable Argo are:
- CONFIG_ARGO in the Xen build
- the Argo Linux kernel module
  https://github.com/OpenXT/linux-xen-argo
- the Yocto recipe was based on
  https://github.com/dozylynx/meta-argo-linux
- pass dom0less_domU to the domU kernel command line


Then you are good to go for using Argo in your Dom0less VMs, e.g.:

>From receiver domain: streamTest -domid 5 -port 8022 -file test.txt -receive 
>From sender domain: streamTest -domid 4 -port 8022 -file test.txt -send -connect

Many thanks to Alec Kwapis from DornerWorks for all the information and
the patch!

Cheers,

Stefano


[1] https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen
--8323329-587971660-1631570062=:10523
Content-Type: text/x-diff; name=0001-Perform-Xen-Initialization-in-Dom0less.patch
Content-Transfer-Encoding: BASE64
Content-ID: <alpine.DEB.2.21.2109131541560.10523@sstabellini-ThinkPad-T480s>
Content-Description: 
Content-Disposition: attachment; filename=0001-Perform-Xen-Initialization-in-Dom0less.patch

RnJvbSA2NDg2YTBkNzNkM2Q0MjJmNTRmOGI1NzlhMzU1MDE2MDA5OGZjYzkz
IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogQWxlYyBLd2FwaXMg
PGFsZWMua3dhcGlzQGRvcm5lcndvcmtzLmNvbT4NCkRhdGU6IFdlZCwgOCBT
ZXAgMjAyMSAwOTowMTo0NiAtMDQwMA0KU3ViamVjdDogW1BBVENIXSBQZXJm
b3JtIFhlbiBJbml0aWFsaXphdGlvbiBpbiBEb20wbGVzcw0KDQpUaGlzIHBh
dGNoIGFsbG93cyBzb21lIFhlbiBpbml0aWFsaXphdGlvbiB0byBvY2N1ciBk
dXJpbmcgdGhlIGJvb3QgcHJvY2Vzcw0Kb2YgdGhlIExpbnV4IGtlcm5lbC4g
VGhpcyBhbHJlYWR5IG9jY3VycyBmb3IgZ3Vlc3RzIGluIGEgRG9tMC1tYW5h
Z2VkDQpjb25maWd1cmF0aW9uLCBob3dldmVyLCB0aGUgeGVuX2RvbWFpbigp
IEFQSSB3aWxsIGV2YWx1YXRlIHRvIGZhbHNlDQpmb3IgZ3Vlc3RzIGluIGEg
RG9tMGxlc3MgY29uZmlndXJhdGlvbi4NCg0KVGhlcmVmb3JlLCBhIExpbnV4
IGtlcm5lbCBjb21tYW5kIGxpbmUgYXJndW1lbnQgd2FzIGFkZGVkICgiZG9t
MGxlc3NfZG9tVSIpDQp3aGljaCBzaWduaWZpZXMgdGhhdCBMaW51eCBpcyBi
ZWluZyBib290ZWQgYXMgYSBndWVzdCBpbiBEb20wbGVzcy4gVGhpcw0KYWxs
b3dzIGNlcnRhaW4gaW5pdGlhbGl6YXRpb24gdG8gb2NjdXIsIHN1Y2ggYXMg
ZXZlbnQgY2hhbm5lbHMgYW5kIHRoZSBzaGFyZWQNCmluZm8gcGFnZS4gVGhp
cyBkb2VzIG5vdCBpbmNsdWRlIGdyYW50IHRhYmxlIGluaXRpYWxpemF0aW9u
Lg0KDQpUaGlzIHBhdGNoIGlzIG5lY2Vzc2FyeSBpbiBnZXR0aW5nIEFyZ28g
dG8gd29yayBpbiBEb20wbGVzcywgd2hpY2ggcmVxdWlyZXMNCmJvdGggWGVu
IGV2ZW50IGNoYW5uZWxzIGZvciB0aGUgQXJnbyB2aXJ0dWFsIGludGVycnVw
dCwgYW5kIHRoZSBzaGFyZWQgaW5mbyBwYWdlLg0KDQpTaWduZWQtb2ZmLWJ5
OiBBbGVjIEt3YXBpcyA8YWxlYy5rd2FwaXNAZG9ybmVyd29ya3MuY29tPg0K
LS0tDQogYXJjaC9hcm0veGVuL2VubGlnaHRlbi5jIHwgMjUgKysrKysrKysr
KysrKysrLS0tLS0tLS0tLQ0KIGluaXQvbWFpbi5jICAgICAgICAgICAgICB8
IDE4ICsrKysrKysrKysrKysrKysrKw0KIDIgZmlsZXMgY2hhbmdlZCwgMzMg
aW5zZXJ0aW9ucygrKSwgMTAgZGVsZXRpb25zKC0pDQoNCmRpZmYgLS1naXQg
YS9hcmNoL2FybS94ZW4vZW5saWdodGVuLmMgYi9hcmNoL2FybS94ZW4vZW5s
aWdodGVuLmMNCmluZGV4IGRkNjgwNGE2NGYxYS4uMGMyMTZiMWZmMjc5IDEw
MDY0NA0KLS0tIGEvYXJjaC9hcm0veGVuL2VubGlnaHRlbi5jDQorKysgYi9h
cmNoL2FybS94ZW4vZW5saWdodGVuLmMNCkBAIC0zNiw2ICszNiw4IEBADQog
DQogI2luY2x1ZGUgPGxpbnV4L21tLmg+DQogDQorZXh0ZXJuIGJvb2wgZG9t
MGxlc3NfZG9tVTsNCisNCiBzdHJ1Y3Qgc3RhcnRfaW5mbyBfeGVuX3N0YXJ0
X2luZm87DQogc3RydWN0IHN0YXJ0X2luZm8gKnhlbl9zdGFydF9pbmZvID0g
Jl94ZW5fc3RhcnRfaW5mbzsNCiBFWFBPUlRfU1lNQk9MKHhlbl9zdGFydF9p
bmZvKTsNCkBAIC0zMTEsNyArMzEzLDcgQEAgc3RhdGljIGludCBfX2luaXQg
eGVuX2d1ZXN0X2luaXQodm9pZCkNCiAJc3RydWN0IHNoYXJlZF9pbmZvICpz
aGFyZWRfaW5mb19wYWdlID0gTlVMTDsNCiAJaW50IGNwdTsNCiANCi0JaWYg
KCF4ZW5fZG9tYWluKCkpDQorCWlmICgheGVuX2RvbWFpbigpICYmICFkb20w
bGVzc19kb21VKQ0KIAkJcmV0dXJuIDA7DQogDQogCWlmICghYWNwaV9kaXNh
YmxlZCkNCkBAIC0zNjIsMTYgKzM2NCwxOSBAQCBzdGF0aWMgaW50IF9faW5p
dCB4ZW5fZ3Vlc3RfaW5pdCh2b2lkKQ0KIAlmb3JfZWFjaF9wb3NzaWJsZV9j
cHUoY3B1KQ0KIAkJcGVyX2NwdSh4ZW5fdmNwdV9pZCwgY3B1KSA9IGNwdTsN
CiANCi0JeGVuX2F1dG9feGxhdF9ncmFudF9mcmFtZXMuY291bnQgPSBnbnR0
YWJfbWF4X2dyYW50X2ZyYW1lcygpOw0KLQlpZiAoeGVuX3hsYXRlX21hcF9i
YWxsb29uZWRfcGFnZXMoJnhlbl9hdXRvX3hsYXRfZ3JhbnRfZnJhbWVzLnBm
biwNCi0JCQkJCSAgJnhlbl9hdXRvX3hsYXRfZ3JhbnRfZnJhbWVzLnZhZGRy
LA0KLQkJCQkJICB4ZW5fYXV0b194bGF0X2dyYW50X2ZyYW1lcy5jb3VudCkp
IHsNCi0JCWZyZWVfcGVyY3B1KHhlbl92Y3B1X2luZm8pOw0KLQkJcmV0dXJu
IC1FTk9NRU07DQorCWlmICh4ZW5fZG9tYWluKCkgJiYgIWRvbTBsZXNzX2Rv
bVUpDQorCXsNCisJCXhlbl9hdXRvX3hsYXRfZ3JhbnRfZnJhbWVzLmNvdW50
ID0gZ250dGFiX21heF9ncmFudF9mcmFtZXMoKTsNCisJCWlmICh4ZW5feGxh
dGVfbWFwX2JhbGxvb25lZF9wYWdlcygmeGVuX2F1dG9feGxhdF9ncmFudF9m
cmFtZXMucGZuLA0KKwkJCQkJCSAgJnhlbl9hdXRvX3hsYXRfZ3JhbnRfZnJh
bWVzLnZhZGRyLA0KKwkJCQkJCSAgeGVuX2F1dG9feGxhdF9ncmFudF9mcmFt
ZXMuY291bnQpKSB7DQorCQkJZnJlZV9wZXJjcHUoeGVuX3ZjcHVfaW5mbyk7
DQorCQkJcmV0dXJuIC1FTk9NRU07DQorCQl9DQorCQlnbnR0YWJfaW5pdCgp
Ow0KKwkJaWYgKCF4ZW5faW5pdGlhbF9kb21haW4oKSkNCisJCQl4ZW5idXNf
cHJvYmUoTlVMTCk7DQogCX0NCi0JZ250dGFiX2luaXQoKTsNCi0JaWYgKCF4
ZW5faW5pdGlhbF9kb21haW4oKSkNCi0JCXhlbmJ1c19wcm9iZShOVUxMKTsN
CiANCiAJLyoNCiAJICogTWFraW5nIHN1cmUgYm9hcmQgc3BlY2lmaWMgY29k
ZSB3aWxsIG5vdCBzZXQgdXAgb3BzIGZvcg0KZGlmZiAtLWdpdCBhL2luaXQv
bWFpbi5jIGIvaW5pdC9tYWluLmMNCmluZGV4IDkxZjZlYmIzMGVmMC4uOTE3
Y2Q2MGE4OTk3IDEwMDY0NA0KLS0tIGEvaW5pdC9tYWluLmMNCisrKyBiL2lu
aXQvbWFpbi5jDQpAQCAtMTQ5LDYgKzE0OSwyNCBAQCBzdGF0aWMgY2hhciAq
cmFtZGlza19leGVjdXRlX2NvbW1hbmQ7DQogYm9vbCBzdGF0aWNfa2V5X2lu
aXRpYWxpemVkIF9fcmVhZF9tb3N0bHk7DQogRVhQT1JUX1NZTUJPTF9HUEwo
c3RhdGljX2tleV9pbml0aWFsaXplZCk7DQogDQorLyoNCisgKiBJZiBzZXQs
IHRoaXMgaXMgYW4gaW5kaWNhdGlvbiB0aGF0IExpbnV4IHdpbGwgYmUgYm9v
dGVkIGluIGEgWGVuIGRvbTBsZXNzDQorICogY29uZmlndXJhdGlvbi4gVGhp
cyBpcyBuZWNlc3NhcnkgYmVjYXVzZSB0aGUgY3VycmVudCB4ZW5fZG9tYWlu
KCkgQVBJIGV2YWx1YXRlcw0KKyAqIHRvIGZhbHNlIGZvciBkb20wbGVzcyBk
b21haW5zIG9uIEFSTSwgYnV0IHRoZSBMaW51eCBrZXJuZWwgc3RpbGwgbmVl
ZHMgdG8ga25vdw0KKyAqIHRoYXQgaXQgaXMgYmVpbmcgYm9vdGVkIGluIFhl
biB0byBzZXR1cCBYZW4gZmVhdHVyZXMgc3VjaCBhcyBldmVudCBjaGFubmVs
cyBhbmQNCisgKiB0aGUgc2hhcmVkIGluZm8gcGFnZS4NCisgKi8NCitib29s
IGRvbTBsZXNzX2RvbVUgPSBmYWxzZTsNCitFWFBPUlRfU1lNQk9MKGRvbTBs
ZXNzX2RvbVUpOw0KKw0KK3N0YXRpYyBpbnQgX19pbml0IHNldF9kb20wbGVz
c19kb21VKGNoYXIgKnN0cikNCit7DQorCWRvbTBsZXNzX2RvbVUgPSB0cnVl
Ow0KKwlyZXR1cm4gMTsNCit9DQorDQorX19zZXR1cCgiZG9tMGxlc3NfZG9t
VSIsIHNldF9kb20wbGVzc19kb21VKTsNCisNCiAvKg0KICAqIElmIHNldCwg
dGhpcyBpcyBhbiBpbmRpY2F0aW9uIHRvIHRoZSBkcml2ZXJzIHRoYXQgcmVz
ZXQgdGhlIHVuZGVybHlpbmcNCiAgKiBkZXZpY2UgYmVmb3JlIGdvaW5nIGFo
ZWFkIHdpdGggdGhlIGluaXRpYWxpemF0aW9uIG90aGVyd2lzZSBkcml2ZXIg
bWlnaHQNCi0tIA0KMi4yNS4xDQoNCg==

--8323329-587971660-1631570062=:10523--


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 23:02:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 23:02:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.185990.334733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPuy3-0008Hq-3d; Mon, 13 Sep 2021 23:02:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 185990.334733; Mon, 13 Sep 2021 23:02:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPuy3-0008Hj-0b; Mon, 13 Sep 2021 23:02:27 +0000
Received: by outflank-mailman (input) for mailman id 185990;
 Mon, 13 Sep 2021 23:02:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPuy1-0008HZ-Uf; Mon, 13 Sep 2021 23:02:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPuy1-0007zC-Lz; Mon, 13 Sep 2021 23:02:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPuy1-0007SG-A5; Mon, 13 Sep 2021 23:02:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPuy1-0005tZ-7R; Mon, 13 Sep 2021 23:02:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JT8D8DlmnUJJW64QJ/Y5rq6Qf4fq8586+d90Rek7AQg=; b=3dUz4BbVeTD54XZm8m4Jt+HH3V
	2H+MS/xzLXoium0fMiyPi86TFXS2fV3rJ0aUIvah6IWeuckoDU5X+Fb4ru/u8buIhy4V7zEuC4c73
	OKof1/JXZs2ZgaLfltubJeFfoU8+dXzGb4ZUwO+hb0Y1LyuRwcB4/P8gqNHAjWQcxPvE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164967-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164967: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=eae587e8e3694b1aceab23239493fb4c7e1a80f5
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 13 Sep 2021 23:02:25 +0000

flight 164967 qemu-mainline real [real]
flight 164972 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164967/
http://logs.test-lab.xenproject.org/osstest/logs/164972/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                eae587e8e3694b1aceab23239493fb4c7e1a80f5
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    2 days
Testing same since   164967  2021-09-13 13:06:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Markus Armbruster <armbru@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit eae587e8e3694b1aceab23239493fb4c7e1a80f5
Merge: 99c44988d5 62f27589f8
Author: Peter Maydell <peter.maydell@linaro.org>
Date:   Mon Sep 13 11:00:30 2021 +0100

    Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-09-13' into staging
    
    QAPI patches patches for 2021-09-13
    
    # gpg: Signature made Mon 13 Sep 2021 08:53:42 BST
    # gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
    # gpg:                issuer "armbru@redhat.com"
    # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
    # gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
    # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653
    
    * remotes/armbru/tags/pull-qapi-2021-09-13:
      qapi: Fix bogus error for 'if': { 'not': '' }
      tests/qapi-schema: Cover 'not' condition with empty argument
      qapi: Bury some unused code in class Indentation
      qapi: Drop Indentation.__bool__()
      qapi: Fix a botched type annotation
    
    Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

commit 62f27589f8549d6cf1f7fe21ed55ce6f2f705450
Author: Markus Armbruster <armbru@redhat.com>
Date:   Wed Sep 8 06:54:28 2021 +0200

    qapi: Fix bogus error for 'if': { 'not': '' }
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20210908045428.2689093-6-armbru@redhat.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
    [check_infix()'s type hint fixed]

commit 71f03ef9f66b020d58acad5227e886102db41127
Author: Markus Armbruster <armbru@redhat.com>
Date:   Wed Sep 8 06:54:27 2021 +0200

    tests/qapi-schema: Cover 'not' condition with empty argument
    
    We flag this, but the error message is bogus:
    
        bad-if-not.json:2: 'if' condition [] of struct is useless
    
    The next commit will fix it.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20210908045428.2689093-5-armbru@redhat.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

commit e2ff14a5740c2fe3714a56221792b6d74bc64c08
Author: Markus Armbruster <armbru@redhat.com>
Date:   Wed Sep 8 06:54:26 2021 +0200

    qapi: Bury some unused code in class Indentation
    
    .__int__() has never been used.  Drop it.
    
    .decrease() raises ArithmeticError when asked to decrease indentation
    level below zero.  Nothing catches it.  It's a programming error.
    Dumb down to assert.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20210908045428.2689093-4-armbru@redhat.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

commit 916fca17c7445c17d4be37610c80c1f68784ef28
Author: Markus Armbruster <armbru@redhat.com>
Date:   Wed Sep 8 06:54:25 2021 +0200

    qapi: Drop Indentation.__bool__()
    
    Intentation.__bool__() is not worth its keep: it has just one user,
    which can just as well check .__str__() instead.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20210908045428.2689093-3-armbru@redhat.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

commit 7b275cdd69d5e6af0b95f4e6440d4664fe1c3674
Author: Markus Armbruster <armbru@redhat.com>
Date:   Wed Sep 8 06:54:24 2021 +0200

    qapi: Fix a botched type annotation
    
    Mypy is unhappy:
    
        $ mypy --config-file=scripts/qapi/mypy.ini `git-ls-files scripts/qapi/\*py`
        scripts/qapi/common.py:208: error: Function is missing a return type annotation
        scripts/qapi/common.py:227: error: Returning Any from function declared to return "str"
    
    Messed up in commit ccea6a8637 "qapi: Factor common recursion out of
    cgen_ifcond(), docgen_ifcond()".  Tidy up.
    
    Signed-off-by: Markus Armbruster <armbru@redhat.com>
    Message-Id: <20210908045428.2689093-2-armbru@redhat.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>


From xen-devel-bounces@lists.xenproject.org Mon Sep 13 23:51:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 13 Sep 2021 23:51:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186008.334750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPvjY-0005IU-SN; Mon, 13 Sep 2021 23:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186008.334750; Mon, 13 Sep 2021 23:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPvjY-0005IN-PC; Mon, 13 Sep 2021 23:51:32 +0000
Received: by outflank-mailman (input) for mailman id 186008;
 Mon, 13 Sep 2021 23:51:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uZue=OD=xilinx.com=stefanos@srs-us1.protection.inumbo.net>)
 id 1mPvjW-0005IH-Uw
 for xen-devel@lists.xen.org; Mon, 13 Sep 2021 23:51:31 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (unknown
 [40.107.96.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea8a4d6c-7a16-4808-a39b-9053d2451f38;
 Mon, 13 Sep 2021 23:51:29 +0000 (UTC)
Received: from BN0PR02CA0021.namprd02.prod.outlook.com (2603:10b6:408:e4::26)
 by CH2PR02MB6776.namprd02.prod.outlook.com (2603:10b6:610:7d::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Mon, 13 Sep
 2021 23:51:26 +0000
Received: from BN1NAM02FT037.eop-nam02.prod.protection.outlook.com
 (2603:10b6:408:e4:cafe::3b) by BN0PR02CA0021.outlook.office365.com
 (2603:10b6:408:e4::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend
 Transport; Mon, 13 Sep 2021 23:51:26 +0000
Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by
 BN1NAM02FT037.mail.protection.outlook.com (10.13.2.148) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 23:51:25 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Mon, 13 Sep 2021 16:51:16 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Mon, 13 Sep 2021 16:51:16 -0700
Received: from [10.23.120.123] (port=50347 helo=localhost)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <stefano.stabellini@xilinx.com>)
 id 1mPvjI-000DaH-15; Mon, 13 Sep 2021 16:51:16 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea8a4d6c-7a16-4808-a39b-9053d2451f38
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TlE+sqcZ84m+zoXfjG7NnbbcBUUOtVLZ5xgW1xl/wLrMC/QjY0IwU+y3oqprY7GTRepMNyW/wjgvFn7BaxgaQfTg3fJy9YKLdXuaUCnBfKiNYDK0Zs5I91li+7SDCVHB5VVdXfW0BQMCh96oLrhJAuZ3cKJEUnbXqmHN9UZhcU+8tBlg3EUTJ1zTOyFquBvnDfdddsVAuGT3++81tNRzfwy4di3LLg/3DBzTOxTN02w12fpqA+rNGxIFXI5RhN6mSMARuGKSpzaI9zZ1gRJux7hKj1XiyWnYL8VOwMf5MfKcQB86t5mJKiiozNpKwY2bg0mhYjtLSnGT71Yz/eBFjA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uvhrbYiih/BgnbeDkHjMOlRTobKBT60UTUhY7unmeDU=;
 b=oS0LHbyFvFOdQQReOoBnfCD9pREoWzup+EL4RpBAnb7ofUDuOccu/ClojHnILP5CAa96ev08xDX7agXSN/rzR5gBjQ3hA20rLUTTVMZCl/aImJ/bjublzBY7mYJ5eeRAheNdkPFqkDI1K9uRw6vZRXsYWWF4Az4AoWVAwSINRkR6RpM7oCYSe4Jn+DN24XexUKhnwkwjZrcaZfKamsjY7ySMJzGdO87nyLq9PbHBDY/JwERjtOmRZvMdX5DPDklEx4MVzXpsfenhZBpB5HJjsS90ZMztS919JkE9w7gRjyLp3ynZXmsC2Et/whOtD/wkLY22nQDDB+CMaJOuRvbciQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=linaro.org smtp.mailfrom=xilinx.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uvhrbYiih/BgnbeDkHjMOlRTobKBT60UTUhY7unmeDU=;
 b=gjgvOyFPPQXcYE7mZEDlBgG8vfgKaBX4IuIYiBTII/Fkw/1BvoICEN2Z5kljmmlBntJsSjGWlnpjXCtNgt4L5dwapruVqnnJBpb1fmdkeBYgSZanELQX23Z4pb7sHwjtsQbl2VODmxMRyGzNnP1ZjZktimm0U9FEs0xa8aZPjSY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; linaro.org; dkim=none (message not signed)
 header.d=none;linaro.org; dmarc=pass action=none header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com;
Date: Mon, 13 Sep 2021 16:51:15 -0700
From: Stefano Stabellini <stefano.stabellini@xilinx.com>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: AKASHI Takahiro <takahiro.akashi@linaro.org>
CC: Alex Benn??e <alex.bennee@linaro.org>, Stefan Hajnoczi
	<stefanha@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>, Stratos
 Mailing List <stratos-dev@op-lists.linaro.org>,
	<virtio-dev@lists.oasis-open.org>, Arnd Bergmann <arnd.bergmann@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>, Stefano Stabellini
	<stefano.stabellini@xilinx.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl
 van Schaik <cvanscha@qti.qualcomm.com>, <pratikp@quicinc.com>, Srivatsa
 Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker
	<jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
	<Wei.Chen@arm.com>, <olekstysh@gmail.com>, <Oleksandr_Tyshchenko@epam.com>,
	<Bertrand.Marquis@arm.com>, <Artem_Mygaiev@epam.com>, <julien@xen.org>,
	<jgross@suse.com>, <paul@xen.org>, <xen-devel@lists.xen.org>, Elena Afanasova
	<eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
In-Reply-To: <20210906022356.GD40187@laputa>
Message-ID: <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
References: <87v94ldrqq.fsf@linaro.org> <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s> <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org> <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9ac308c8-504f-4cdc-d5d2-08d977116620
X-MS-TrafficTypeDiagnostic: CH2PR02MB6776:
X-Microsoft-Antispam-PRVS:
	<CH2PR02MB67769995CDC9C29D0F19C6DCA0D99@CH2PR02MB6776.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tyDxsviuIQ8x/+fA5OsHGPoHwHXbIIzOLtbDSR9SwOzsT+KMWVDWNH2687hkHWwz4g3aNIyDHdW6HfqYA1MJs7nMgs4L+0DO1qo4Q4vsUhYNCSaBtu2WV9iigzKVm8KcZl97qFniNxSs1txBleoF0asXBIWuQC3vXwx6+YzKdYFi7rTUoQ/gGZMEO7ar6LgeS5Kc7s4TbJNqGtk7NxXr8KJWoUHKsyhICIGS600Xe7/5Sz2mKytnKW6nHQcQ9lcqp0xpmTvlDqZ+SZbzxaSb9ATBwt1Ayzk8lkAJfpGvWWQFeGTlLqVp4mbpDlYrFhpKXg8T3boamgCHMVu7CvgDLq0392/n9qL/M35X9hcu8fjZcSLvnDzFFV3Gi7Wd3g8Sz3fwxNo+QzDB0HGrnXNs+ytNL9+udWGmdoFawPMVqT8lJcClZVuOOr9bB0M1sMyvkYAtcLZC3gsmQ1fE7ei93UUTwrbk+n4WUFs/V8Yv69OT3z+532x3VqRiolbQD53Zdy3ntv1YuDfVfCB55K3HMZGiP0g70EiILQDXsdR3RwmHdLvbTUaWaJVSIha80rH9HURf3sePC3lQQRgwRkftzRmqDvrKpzq0wsscePby42l3taz3E5blS0NWVs49W116zhlRM1WuDBGc1xar8s69XbTRAVHV4KMwhxKO+hw02Q7BKcR9iTrj2fgw6N/Tfg4ii2IDYV73X6KYEEJFYF5qY2Hs+XeR6JS0W/BtUg1t8Spcdz/l28IDmsKpmRos6qB/iCtEFbBnCYyMNv3VZHL71BGfUwLjQ9bmNAyfGk8aqKk=
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch01.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(4636009)(7916004)(396003)(346002)(39860400002)(136003)(376002)(46966006)(36840700001)(316002)(426003)(36906005)(7636003)(70586007)(9786002)(70206006)(82310400003)(2906002)(44832011)(336012)(36860700001)(9686003)(47076005)(82740400003)(26005)(5660300002)(966005)(4326008)(8936002)(6916009)(478600001)(8676002)(7416002)(356005)(54906003)(186003)(83380400001)(33716001);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 23:51:25.9733
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ac308c8-504f-4cdc-d5d2-08d977116620
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch01.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1NAM02FT037.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR02MB6776

On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
> > the second is how many context switches are involved in a transaction.
> > Of course with all things there is a trade off. Things involving the
> > very tightest latency would probably opt for a bare metal backend which
> > I think would imply hypervisor knowledge in the backend binary.
> 
> In configuration phase of virtio device, the latency won't be a big matter.
> In device operations (i.e. read/write to block devices), if we can
> resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue is
> how efficiently we can deliver notification to the opposite side. Right?
> And this is a very common problem whatever approach we would take.
> 
> Anyhow, if we do care the latency in my approach, most of virtio-proxy-
> related code can be re-implemented just as a stub (or shim?) library
> since the protocols are defined as RPCs.
> In this case, however, we would lose the benefit of providing "single binary"
> BE.
> (I know this is is an arguable requirement, though.)

In my experience, latency, performance, and security are far more
important than providing a single binary.

In my opinion, we should optimize for the best performance and security,
then be practical on the topic of hypervisor agnosticism. For instance,
a shared source with a small hypervisor-specific component, with one
implementation of the small component for each hypervisor, would provide
a good enough hypervisor abstraction. It is good to be hypervisor
agnostic, but I wouldn't go extra lengths to have a single binary. I
cannot picture a case where a BE binary needs to be moved between
different hypervisors and a recompilation is impossible (BE, not FE).
Instead, I can definitely imagine detailed requirements on IRQ latency
having to be lower than 10us or bandwidth higher than 500 MB/sec.

Instead of virtio-proxy, my suggestion is to work together on a common
project and common source with others interested in the same problem.

I would pick something like kvmtool as a basis. It doesn't have to be
kvmtools, and kvmtools specifically is GPL-licensed, which is
unfortunate because it would help if the license was BSD-style for ease
of integration with Zephyr and other RTOSes.

As long as the project is open to working together on multiple
hypervisors and deployment models then it is fine. For instance, the
shared source could be based on OpenAMP kvmtool [1] (the original
kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
kvmtool was created to add support for hypervisor-less virtio but they
are very open to hypervisors too. It could be a good place to add a Xen
implementation, a KVM fatqueue implementation, a Jailhouse
implementation, etc. -- work together toward the common goal of a single
BE source (not binary) supporting multiple different deployment models.


[1] https://github.com/OpenAMP/kvmtool


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 00:55:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 00:55:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186028.334761 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPwjI-0003wK-60; Tue, 14 Sep 2021 00:55:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186028.334761; Tue, 14 Sep 2021 00:55:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPwjI-0003wD-1v; Tue, 14 Sep 2021 00:55:20 +0000
Received: by outflank-mailman (input) for mailman id 186028;
 Tue, 14 Sep 2021 00:55:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L61H=OE=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mPwjG-0003w7-5e
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 00:55:18 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 18c84e56-eef3-4c24-a539-9163e884bb12;
 Tue, 14 Sep 2021 00:55:16 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B600360EFE;
 Tue, 14 Sep 2021 00:55:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18c84e56-eef3-4c24-a539-9163e884bb12
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631580916;
	bh=Kmoiwlto49Hh3XxoOAEJ6PYDt1AQ/ZhjKUNnlPJv1Ow=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RNDsha5jiydInhr/8c6uPTYLqBDFlN46E0+PDZOkfOKJjEWUSa1fmHVUL+M7Ziv7S
	 VcKiWt7HC4iKs0jj7O1blP6kTi3Py30xhCKdFMMaeVF+Z/1hq/x4S+NFC9Oxrig5cb
	 75KH0ylSfPvBkm7+kOizef5mqRBeH78dcKMqNyThyBthG1BlKcitQUJEyh4rEpgJP1
	 GK+lo18y10K7bWgDirCqkXyUsdhG6b0g8Czw/edOW5AbEzv0xZEZ0un1nsExky/6Mt
	 PVxGzRUyP8cmqLBl3zSI8wovWkn6wOrJKzc5aVTmJXYciAV5DPY0sjU/1L+f0nOTYL
	 HJom4s4qCsakg==
Date: Mon, 13 Sep 2021 17:55:14 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain.
> - The ACPI case is not covered.
> 
> ***
> 
> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> the algorithm to choose extended regions for it is different
> in comparison with the algorithm for non-direct mapped DomU.
> What is more, that extended regions should be chosen differently
> whether IOMMU is enabled or not.
> 
> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> holes found in host device-tree if otherwise. Make sure that
> extended regions are 2MB-aligned and located within maximum possible
> addressable physical memory range. The maximum number of extended
> regions is 128.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> Changes since RFC:
>    - update patch description
>    - drop uneeded "extended-region" DT property
> ---
> 
>  xen/arch/arm/domain_build.c | 226 +++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 224 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 206038d..070ec27 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domain *d,
>      return res;
>  }
>  
> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
> +{
> +    struct meminfo *ext_regions = data;
> +    paddr_t start, size;
> +
> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> +        return 0;
> +
> +    /* Both start and size of the extended region should be 2MB aligned */
> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> +    if ( start > e )
> +        return 0;
> +
> +    size = (e - start + 1) & ~(SZ_2M - 1);
> +    if ( !size )
> +        return 0;

Can't you align size as well?

  size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);


> +    ext_regions->bank[ext_regions->nr_banks].start = start;
> +    ext_regions->bank[ext_regions->nr_banks].size = size;
> +    ext_regions->nr_banks ++;
                            ^ no space

> +    return 0;
> +}
> +
> +/*
> + * The extended regions will be prevalidated by the memory hotplug path
> + * in Linux which requires for any added address range to be within maximum
> + * possible addressable physical memory range for which the linear mapping
> + * could be created.
> + * For 48-bit VA space size the maximum addressable range are:
> + * 0x40000000 - 0x80003fffffff

Please don't make Linux-specific comments in Xen code for interfaces
that are supposed to be OS-agnostic.


> + */
> +#define EXT_REGION_START   0x40000000ULL
> +#define EXT_REGION_END     0x80003fffffffULL
> +
> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
> +                                          struct meminfo *ext_regions)
> +{
> +    const struct meminfo *assign_mem = &kinfo->mem;
> +    struct rangeset *unalloc_mem;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find unallocated memory for extended regions\n");
> +
> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> +    if ( !unalloc_mem )
> +        return -ENOMEM;
> +
> +    /* Start with all available RAM */
> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.mem.bank[i].start;
> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;

Is the -1 needed? Isn't it going to screw up the size calculation later?


> +        res = rangeset_add_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove RAM assigned to Dom0 */
> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> +    {
> +        start = assign_mem->bank[i].start;
> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
> +        res = rangeset_remove_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove reserved-memory regions */
> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.reserved_mem.bank[i].start;
> +        end = bootinfo.reserved_mem.bank[i].start +
> +            bootinfo.reserved_mem.bank[i].size - 1;
> +        res = rangeset_remove_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove grant table region */
> +    start = kinfo->gnttab_start;
> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> +    res = rangeset_remove_range(unalloc_mem, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_report_ranges(unalloc_mem, start, end,
> +                                 add_ext_regions, ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(unalloc_mem);
> +
> +    return res;
> +}
> +
> +static int __init find_memory_holes(const struct kernel_info *kinfo,
> +                                    struct meminfo *ext_regions)
> +{
> +    struct dt_device_node *np;
> +    struct rangeset *mem_holes;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find memory holes for extended regions\n");
> +
> +    mem_holes = rangeset_new(NULL, NULL, 0);
> +    if ( !mem_holes )
> +        return -ENOMEM;
> +
> +    /* Start with maximum possible addressable physical memory range */
> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_add_range(mem_holes, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
> +    dt_for_each_device_node( dt_host, np )

Don't you need something like device_tree_for_each_node ?
dt_for_each_device_node won't go down any deeper in the tree?

Alternatively, maybe we could simply record the highest possible address
of any memory/device/anything as we scan the device tree with
handle_node. Then we can use that as the starting point here. So that we
don't need to scan the device tree twice, and also we don't need my
suggestion below to remove 1GB-aligned 1GB-multiple regions.


> +    {
> +        unsigned int naddr;
> +        u64 addr, size;
> +
> +        naddr = dt_number_of_address(np);
> +
> +        for ( i = 0; i < naddr; i++ )
> +        {
> +            res = dt_device_get_address(np, i, &addr, &size);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> +                       i, dt_node_full_name(np));

It might be possible for a device not to have a range if it doesn't have
any MMIO regions, right? For instance, certain ARM timer nodes. I would
not print any errors and continue.


> +                goto out;
> +            }
> +
> +            start = addr & PAGE_MASK;
> +            end = PAGE_ALIGN(addr + size) - 1;
> +            res = rangeset_remove_range(mem_holes, start, end);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                       start, end);
> +                goto out;
> +            }
> +        }
> +    }

As is, it will result in a myriad of small ranges which is unuseful and
slow to parse. I suggest to simplify it by removing a larger region than
strictly necessary. For instance, you could remove a 1GB-aligned and
1GB-multiple region for each range. That way, you are going to get fewer
large free ranges instance of many small ones which we don't need.


> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_report_ranges(mem_holes, start, end,
> +                                 add_ext_regions,  ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(mem_holes);
> +
> +    return res;
> +}
> +
>  static int __init make_hypervisor_node(struct domain *d,
>                                         const struct kernel_info *kinfo,
>                                         int addrcells, int sizecells)
> @@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain *d,
>      const char compat[] =
>          "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>          "xen,xen";
> -    __be32 reg[4];
> +    __be32 reg[(NR_MEM_BANKS + 1) * 4];

If you are xzalloc'ing struct meminfo then you might as well xzalloc reg
too. Or keep both on the stack with a lower NR_MEM_BANKS.


>      gic_interrupt_t intr;
>      __be32 *cells;
>      int res;
>      void *fdt = kinfo->fdt;
> +    struct meminfo *ext_regions;
> +    unsigned int i;
>  
>      dt_dprintk("Create hypervisor node\n");
>  
> @@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct domain *d,
>      if ( res )
>          return res;
>  
> +    ext_regions = xzalloc(struct meminfo);
> +    if ( !ext_regions )
> +        return -ENOMEM;
> +
> +    if ( is_32bit_domain(d) )
> +        printk(XENLOG_WARNING "The extended region is only supported for 64-bit guest\n");

This is going to add an unconditional warning to all 32bit boots. I
would skip it entirely or only keep it as XENLOG_DEBUG.


> +    else
> +    {
> +        if ( !is_iommu_enabled(d) )
> +            res = find_unallocated_memory(kinfo, ext_regions);
> +        else
> +            res = find_memory_holes(kinfo, ext_regions);
> +
> +        if ( res )
> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> +    }
> +
>      /* reg 0 is grant table space */
>      cells = &reg[0];
>      dt_child_set_range(&cells, addrcells, sizecells,
>                         kinfo->gnttab_start, kinfo->gnttab_size);
> +    /* reg 1...N are extended regions */
> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
> +    {
> +        u64 start = ext_regions->bank[i].start;
> +        u64 size = ext_regions->bank[i].size;
> +
> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
> +                   i, start, start + size);
> +
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> +    }
> +    xfree(ext_regions);
> +
>      res = fdt_property(fdt, "reg", reg,
> -                       dt_cells_to_size(addrcells + sizecells));
> +                       dt_cells_to_size(addrcells + sizecells) * (i + 1));
>      if ( res )
>          return res;
>  
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 01:06:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 01:06:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186038.334771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPwu8-0003lg-5k; Tue, 14 Sep 2021 01:06:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186038.334771; Tue, 14 Sep 2021 01:06:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPwu8-0003lZ-2d; Tue, 14 Sep 2021 01:06:32 +0000
Received: by outflank-mailman (input) for mailman id 186038;
 Tue, 14 Sep 2021 01:06:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPwu7-0003lP-9h; Tue, 14 Sep 2021 01:06:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPwu6-0008TN-UM; Tue, 14 Sep 2021 01:06:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPwu6-0004tB-Fu; Tue, 14 Sep 2021 01:06:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPwu6-0002kR-FQ; Tue, 14 Sep 2021 01:06:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cZQZGFkTYP82s6r6IPSTpJQeyc4PuU9+eA3IO058KXI=; b=oSHPjkWW26ZG1QU0rlfA8UDHWY
	vYd8F8uhOhp/8+A89Kr698pScTil5XynSum7quv4BdZ7bkDKzE4voPxkC7YPGqoA9XszAjpktEVsI
	Qb71QECf8LMNwNVBCrxGdMwImPgxh+2XxH8XHlq8oPyPIUi8cjCT9lM0LdCHG/gKATJU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164969-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164969: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 01:06:30 +0000

flight 164969 xen-unstable real [real]
flight 164975 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164969/
http://logs.test-lab.xenproject.org/osstest/logs/164975/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    3 days
Testing same since   164951  2021-09-12 00:14:36 Z    2 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 01:31:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 01:31:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186050.334786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPxI9-0007Fu-DR; Tue, 14 Sep 2021 01:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186050.334786; Tue, 14 Sep 2021 01:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPxI9-0007Fn-7o; Tue, 14 Sep 2021 01:31:21 +0000
Received: by outflank-mailman (input) for mailman id 186050;
 Tue, 14 Sep 2021 01:31:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPxI7-0007Fd-SZ; Tue, 14 Sep 2021 01:31:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPxI7-0000S2-KC; Tue, 14 Sep 2021 01:31:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPxI7-0005vh-3h; Tue, 14 Sep 2021 01:31:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPxI7-0003EC-3F; Tue, 14 Sep 2021 01:31:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6RO5jrtwKxmwuB5jr9agTBOu6xKa7jPsywfzyfHYY/M=; b=F9Kp84EhsoWir2Iobc0skxcl7E
	hrN8c99XDlG5uH3Zc8suYzyOfgLjl10GX96XvKqn8hHqB5+eQG0ipfDEbVYC+6xg8JLGnGYGuPRjU
	YkE6xVEzpS+EbPIKEFyxt+7jzjjbb0+lwDKKdIjWmPIJU3JZcbVIdItewIFBPSa6QEik=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164971-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 164971: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3b69fcf5f849021aa3bc810f8100ea71c03019e5
X-Osstest-Versions-That:
    ovmf=dcd3d63f4f77da5540bfe35d8ccb040d70644568
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 01:31:19 +0000

flight 164971 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164971/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3b69fcf5f849021aa3bc810f8100ea71c03019e5
baseline version:
 ovmf                 dcd3d63f4f77da5540bfe35d8ccb040d70644568

Last test of basis   164946  2021-09-11 00:12:25 Z    3 days
Testing same since   164971  2021-09-13 17:12:37 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   dcd3d63f4f..3b69fcf5f8  3b69fcf5f849021aa3bc810f8100ea71c03019e5 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 03:16:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 03:16:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186080.334808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPyvr-0001GV-6X; Tue, 14 Sep 2021 03:16:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186080.334808; Tue, 14 Sep 2021 03:16:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mPyvr-0001GO-2R; Tue, 14 Sep 2021 03:16:27 +0000
Received: by outflank-mailman (input) for mailman id 186080;
 Tue, 14 Sep 2021 03:16:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPyvp-0001GE-1V; Tue, 14 Sep 2021 03:16:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPyvo-0002df-Sv; Tue, 14 Sep 2021 03:16:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mPyvo-00027u-Ke; Tue, 14 Sep 2021 03:16:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mPyvo-0003V4-K8; Tue, 14 Sep 2021 03:16:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=uVm4knESZuElQ3ogdHb4TVA7WjyzMd/3h1DT8BoRNy4=; b=j7ZhJlUozv2HZ2NHf8r7smn579
	4541E51udGVlHdNnYepeGbf51pS49rZpkfXNWeTkyAJfDlKB4yJJviT8xd6NGeicGhbLko+aTLy0B
	M2g9iPm4jiuZGrHFGcxQzvNpgu8ztei6eaJAL0rMtSzVW1V0n3lq0sPaQEyVoONyvJ24=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164970-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164970: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:heisenbug
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:heisenbug
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 03:16:24 +0000

flight 164970 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164970/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl          14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 14 guest-start    fail in 164965 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl          13 debian-fixup     fail in 164965 pass in 164970
 test-armhf-armhf-examine      8 reboot           fail in 164965 pass in 164970
 test-arm64-arm64-libvirt-xsm 13 debian-fixup               fail pass in 164965

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 164965 like 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  409 days
Failing since        152366  2020-08-01 20:49:34 Z  408 days  710 attempts
Testing same since   164965  2021-09-13 07:09:47 Z    0 days    2 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255326 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 04:31:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 04:31:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186092.334822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ06H-0001Gj-Om; Tue, 14 Sep 2021 04:31:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186092.334822; Tue, 14 Sep 2021 04:31:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ06H-0001Gc-Ld; Tue, 14 Sep 2021 04:31:17 +0000
Received: by outflank-mailman (input) for mailman id 186092;
 Tue, 14 Sep 2021 04:31:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ06G-0001GV-0L
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 04:31:16 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 100419b6-4536-427a-b06e-046a7628f688;
 Tue, 14 Sep 2021 04:31:14 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18E3wVCK019878; 
 Tue, 14 Sep 2021 04:31:13 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b2m9wr2cw-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 04:31:13 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6066.eurprd03.prod.outlook.com (2603:10a6:208:165::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.19; Tue, 14 Sep
 2021 04:31:09 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 04:31:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 100419b6-4536-427a-b06e-046a7628f688
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FxZDqoB2sGkKvRIj+b9AVjjch8aRiVEbOGOB/OQIU+DZhZFIcC7E1JvzNsn+5/in874B3wNxlqHpAZdpOCeFbSRMDzpeI5Q+tQuChHaphrXY2MkDSMfF/677gsa+eHw+ilSDOyskiDCJaEp9MXTObVxjLM7PlmTP+qjFao2Bt/UN2LUR+86WtnMtlXCA+J6y7oNX9QC6zSlcP1h4um8yfK+/bSGV5fZRXPIDXggejbu2I36uLQK4tqJe9dh3PbyHeAMa9m3jSRZlLhRxS9Geo+Yx7rI6Hm5IesCHHhmUfohJ+2CgbX7k3CoP9TBbYzYoN1tW4y2S0/arEgdKq7XXKA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zCE771L6j59le/sIGUKACdoWh4xC8lPieOqr9RM7Ng4=;
 b=VarXq4xnCEtCGsb/Dwa9My8eYHKfHqg/yz2ngwF4ZwOU3oNWVxaY8oObrofKfY0TZCYZZrtkmLRTxbm2Twhx/Zf5+i5f2OZ4+YQAH9K+AcZF2RDQHw0ajOJpcLCkgu1C7AjE7Jcv7geckKcllMZGNB+XNSovWDA9KrjdMpTMQgnk86W53lcbRl9GAjOy5FkObSE6BeVtE1pogqig76kAzzAof8FpTU7Kda3aeqF+7ujb+zxzihDqHbELDGe80NqWnMencWHYHmvtCp1JOI555J4/ppJxP8iLDUamyQs//IZ1xSg50tiEUIwWAYCpLskk/ssuSEnh3U5+UkFGWpnD/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zCE771L6j59le/sIGUKACdoWh4xC8lPieOqr9RM7Ng4=;
 b=zV/A4KeMYV99bHPqAds1FLVEmZLjT1vaAZLEdpQNL4FW+Hw3HSmRdyuEp26RwCmqxL1kjXKIZes2+3jkb+1sRqcDHC2lVhTTU2D5QmU2dicLarDsIT8wNv7KKpinw+OPTCcEj/aiPlBTRBW1iHZ3NE2OH/E48xsPe3Rv9mN2CyjTknao6iYQTPc/P92SkvI0StlVJ3mbMXxnsol33puM6kSZBTa9TSVD/bPI0MFYWlYbnB0GddmaRI6twSJ3R08lr18NpQ7H30nAm22YWkoxaDVTmV149YRnMtDpPAPx+JXcT/b+A4xKTrW1Nu8f/W94GoDZqrPESNZA+v0CGqClwg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Rahul Singh <rahul.singh@arm.com>,
        xen-devel
	<xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXlPK8ZsFWc1idT0uR0zIe97lMvKucfJAAgADQqgCABOUWAIAAaPoAgAB9fIA=
Date: Tue, 14 Sep 2021 04:31:09 +0000
Message-ID: <f580b6fa-24b4-f50f-7745-a7d4c1420ac0@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
 <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
 <aa251354-3e5f-e1ae-2647-3a112ad5d12e@epam.com>
 <alpine.DEB.2.21.2109131356090.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109131356090.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6fa59f49-e77d-40f9-9bb3-08d9773879e5
x-ms-traffictypediagnostic: AM0PR03MB6066:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB6066369EC3F910578F52D243E7DA9@AM0PR03MB6066.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ls7+mnxLm9Vfw35e+R5SbfjnhToe0/bRrIlAnlQLuk71FkhM+4fhWa8k9zPp2JxVj5Yh4DEtV+c6btAivO6JAiQgPCLPfjiyvWJW6TuKlsUCjBsarQMyvNmEB5Ma6cZY/4wRq2zKpJ5/lVzCYocZs5HV/XFmMff1GtABGOv8FIyDJ51Tni0sbrZXgDaTJa0ko/7cdGYy96NSUl0oQ2mQcGIN4VKtc4DmTdxXAAD554Z8nUAfGwREV9VSeiBdtWLbsRxcq5xu/DlaWAFyT6n9Ywn9VAlGlBnO/93BQlGHwCL5Ml9/bkmksh2kAjGBY/WQdd2/na4cAX2UYLx2XfGsSsOX1jBFcEa8EkfpYxKtCFF7bNWLAkip3U8PlOF/7eHcKL+4BA4sj20PHk6ZQ4qoBWGJ45yGF+oraAxo1hZzX1OYP8MVptYHmx5GQijTMxSe+k32gp+qBn69QpAZDOghoUypcM174bhvvhUm3BtefhkjZjpBRLNVQST4FxN9ua1+XnLf/bixkuhj2VD1/s0ogYAImXN+VL+nn5IuXMzoqGByK/jWYgRBhb4SKU1H78EsbQYLJhJDGqlzbS926hQV9vIBbJG1HVE/JTlBLSi4bhEx6G1Oi2UOaRrtiC0fJpWZuId2BGDpgeb1zWGRpW8l2LPlL9kx1OXdjpLHyb5VUmdVKVem9+lfLxpfy+XZDyo48BEtTdYVUxJLb60PFI63OpG3VbLIShm3m86R4VyMbHk6W6wJEU1ijq22ikbwOjMbvrfEu2Tj5lgwkxnaqJjlg9oQVvRTIHShfSBiH167uc2UbqDLNwXZ/Np78MIq+Gzxqfc3FEmC8L67lmBqP1V55Q==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(39860400002)(366004)(396003)(346002)(136003)(316002)(55236004)(8676002)(122000001)(38100700002)(186003)(6486002)(54906003)(4326008)(8936002)(31686004)(2906002)(31696002)(26005)(86362001)(6916009)(66476007)(83380400001)(5660300002)(478600001)(53546011)(6512007)(966005)(66946007)(66446008)(64756008)(91956017)(76116006)(107886003)(66556008)(36756003)(6506007)(2616005)(38070700005)(71200400001)(2004002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?MkxYR2Y4TVdETFBESFMwOWNGeGlYa3ZGRHJsbEZCdFhiYW42M3V0SmNIdW9S?=
 =?utf-8?B?eFFsY1FEVExNNlY0MkFiV1RFN0pzWDNaOGxNL2xTdW55VHJnZWNmYkl2QkI5?=
 =?utf-8?B?WGdtRlppNjhoNTkyV1dxaW4wRGsvUXA0ZUpaMloveThsczkyWlRBam9EajFm?=
 =?utf-8?B?Z25NdHdPWGdTQk1sN1dGWW9IMm9WNi9FZGF6ZXFMbkloRGQxTllDRmJNQUtv?=
 =?utf-8?B?MDYyb0Q5b09ldnN6Y1pyRW13SzRCR0JjWGR0bk54UzZ0VW5ueXRja3FZQlh5?=
 =?utf-8?B?S2FiKzBMa1JUSnB2UldyNXJuVjU0YXlRV0krK1ZPWmxxc1VMQzQyUUtGZCtl?=
 =?utf-8?B?SjJ2Rit2TUpaVi80N3Mwc1FTampSczc1Q2xQNkNTZ3hlNDZ1cjQzeFRSVlc1?=
 =?utf-8?B?OXA5dHFXTjBIcVUwcGlGR1c1ZUtqTGFTTFdqaHFIYTF6M1ZxdEozY3F2aTJN?=
 =?utf-8?B?bU1nVUJseWo0clgwcXhENHc1aENOODN1WVA0dHlOcko0TWJRajkxK094QXRT?=
 =?utf-8?B?NHVDamFVNGlCUkFlaXIxeFl3QTg3azFJeHl1M2ZIZ21qNVd0QmRadDEyTVJ5?=
 =?utf-8?B?Q3loeGt4VHNZT1YyTEdjWkEwVlJ1c3ZmK0xXYW5NMERCaTd6T0xQWUUwL3ow?=
 =?utf-8?B?aE5FbDJUNUpTK0RHWE1CUnVWU0FMeS9sSUZUemNmdWdFT1QzT3Zvb0pKakZy?=
 =?utf-8?B?VXluMEdQKzZpSk15Sm41eTZ1SnB3ZWk2R3k3SUJyVm12WjNOa085MHFrNEow?=
 =?utf-8?B?YUhpWFFYTnBmRW5POXdneFp4SS8yZGJXVUsvcXJmUTRKNW9aSERxN2tJYlJD?=
 =?utf-8?B?RWR3Wmh4RXFrL2NmK0xGb3gyTjhDd091MTFPV3F0ZEtNSDQ4SXQ4cWZrMGtM?=
 =?utf-8?B?cVhtMUg5dzJ5Q3BMVyt1M3pYK0JvTENnUW16ZVdsRjlkQmJSQ25ORStOUEpi?=
 =?utf-8?B?NEYvd3U1aUVOaW9iekk4TndSTUtaT1RyN1JoOUpWN3ErQnNkUmg3OWJQcURu?=
 =?utf-8?B?ZGxINXA3TjRPZEFLeG1hOEhlMTRLT3lib3hBQ09wYWppdEFCMjQ3aXVYV0tk?=
 =?utf-8?B?M1E3ekFiY21OL3lIS25mRVI4b3FvYUJVOW9IZXpLaTl5ZUZuZFlodFBobnFY?=
 =?utf-8?B?WENsSEUxaDZ6NkZFS3RXOXc2WDFMMXpUbStCUTVJVnVkYUdhQlpuYkg0eUNx?=
 =?utf-8?B?ODZ6WWJEdzJxYlF6YitVM2lrRjBtSGlQMVJ2cnJ6ZlduVEtTWkg5MXdzbFFS?=
 =?utf-8?B?azZxRVEyZGtCVXZjbHNZRHJhL1dEQ0J0and1MHJkZlJFQjNkbzhBVUJ2aVQz?=
 =?utf-8?B?T2liOEF3OWhJbXd0SlRHSzY0bDAxU3psbUtxUC9DYmhnbm00elp3WllyQWlW?=
 =?utf-8?B?YjNiK0c2ek5adnRhcjZWcTd2dmlZcEhzdTlQK2I1UDNxZ3dGTDRpcm9zV1du?=
 =?utf-8?B?ZWE0d3E0blBtT0lENTlFakt3Zlk3KzRhOWtlTlA1YktkQm14cXdZbGZpZGNi?=
 =?utf-8?B?UWdyTi9jUjJEeXRnUjhmSjdkbnpKSXhTZ0JsdGUyQy8yWnoyYlFRbE8vQTl2?=
 =?utf-8?B?Vlg1SEwrbk94WnZ3YzV5QnpwNFg2SU9BRUR0bDhrVGJ5NlprL3JoU29lNlgr?=
 =?utf-8?B?VTIvRyt0amZHelowbXV4MGR0eWk1dUxTdld1aU0zWkdjcUpQVlByRk1uS1Iy?=
 =?utf-8?B?a2xFTkJtRFNCWDZJNFVMNnBSOVlYeXNoR0dnazEwSEFnVFp0Y0VudjJaT0R4?=
 =?utf-8?B?NU5oVzV2Z0NjakRzbm51T0tDeVdOYTl6RTU2djBtbnF4SFpleUVxZGFhT3Fs?=
 =?utf-8?B?ZExhK1VkamlZNTNNWWJOdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <70C657702E2DC7469BFC903D6CAC1007@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6fa59f49-e77d-40f9-9bb3-08d9773879e5
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 04:31:09.5744
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sF/Vg3mdXS12LTzDFM4CcwfRcaOcv297HQpthwGf13vFevOxzxY/mtpwKkYpTD4s6ncosAWTC0KIyjEa7U/5lBL+CGoAoubmq4YYwvOH00BK7390O1A2rjApSrRvF27w
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6066
X-Proofpoint-ORIG-GUID: vGQYWfwtRFtvnO_LPbHUiXro1fk8dtSV
X-Proofpoint-GUID: vGQYWfwtRFtvnO_LPbHUiXro1fk8dtSV
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_09,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 impostorscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 spamscore=0
 priorityscore=1501 clxscore=1015 adultscore=0 malwarescore=0 phishscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140025

DQpPbiAxNC4wOS4yMSAwMDowMiwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBNb24s
IDEzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IE9uIDEwLjA5
LjIxIDE1OjAxLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+PiBIaSBTdGVmYW5vLA0KPj4+DQo+Pj4+
IE9uIDEwIFNlcCAyMDIxLCBhdCAxMjozNCBhbSwgU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVs
bGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4+Pj4NCj4+Pj4gT24gVGh1LCAxOSBBdWcgMjAyMSwg
UmFodWwgU2luZ2ggd3JvdGU6DQo+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8
b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pg0KPj4+Pj4gQWRkIHN1cHBv
cnQgZm9yIFhpbGlueCBaeW5xTVAgUENJIGhvc3QgY29udHJvbGxlciB0byBtYXAgdGhlIFBDSSBj
b25maWcNCj4+Pj4+IHNwYWNlIHRvIHRoZSBYRU4gbWVtb3J5Lg0KPj4+Pj4NCj4+Pj4+IFNpZ25l
ZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5r
b0BlcGFtLmNvbT4NCj4+Pj4+IC0tLQ0KPj4+Pj4geGVuL2FyY2gvYXJtL3BjaS9NYWtlZmlsZSAg
ICAgICAgICB8ICAxICsNCj4+Pj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtenlucW1wLmMg
fCA1OSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+Pj4+IDIgZmlsZXMgY2hhbmdl
ZCwgNjAgaW5zZXJ0aW9ucygrKQ0KPj4+Pj4gY3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL2Fy
bS9wY2kvcGNpLWhvc3QtenlucW1wLmMNCj4+Pj4+DQo+Pj4+PiBkaWZmIC0tZ2l0IGEveGVuL2Fy
Y2gvYXJtL3BjaS9NYWtlZmlsZSBiL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+Pj4+IGlu
ZGV4IDZmMzJmYmJlNjcuLjFkMDQ1YWRlMDEgMTAwNjQ0DQo+Pj4+PiAtLS0gYS94ZW4vYXJjaC9h
cm0vcGNpL01ha2VmaWxlDQo+Pj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlDQo+
Pj4+PiBAQCAtMywzICszLDQgQEAgb2JqLXkgKz0gcGNpLWFjY2Vzcy5vDQo+Pj4+PiBvYmoteSAr
PSBwY2ktaG9zdC1nZW5lcmljLm8NCj4+Pj4+IG9iai15ICs9IHBjaS1ob3N0LWNvbW1vbi5vDQo+
Pj4+PiBvYmoteSArPSBlY2FtLm8NCj4+Pj4+ICtvYmoteSArPSBwY2ktaG9zdC16eW5xbXAubw0K
Pj4+Pj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtenlucW1wLmMgYi94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jDQo+Pj4+PiBuZXcgZmlsZSBtb2RlIDEw
MDY0NA0KPj4+Pj4gaW5kZXggMDAwMDAwMDAwMC4uZmUxMDNlMzg1NQ0KPj4+Pj4gLS0tIC9kZXYv
bnVsbA0KPj4+Pj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC16eW5xbXAuYw0KPj4+
Pj4gQEAgLTAsMCArMSw1OSBAQA0KPj4+Pj4gKy8qDQo+Pj4+PiArICogQ29weXJpZ2h0IChDKSAy
MDIwLTIwMjEgRVBBTSBTeXN0ZW1zDQo+Pj4+PiArICoNCj4+Pj4+ICsgKiBCYXNlZCBvbiBMaW51
eCBkcml2ZXJzL3BjaS9jb250cm9sbGVyL3BjaS1ob3N0LWNvbW1vbi5jDQo+Pj4+PiArICogQmFz
ZWQgb24gTGludXggZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktaG9zdC1nZW5lcmljLmMNCj4+
Pj4+ICsgKiBCYXNlZCBvbiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMuYw0KPj4+
Pj4gKyAqIENvcHlyaWdodCAoQykgMjAxNCBBUk0gTGltaXRlZCBXaWxsIERlYWNvbiA8d2lsbC5k
ZWFjb25AYXJtLmNvbT4NCj4+Pj4gT25seSBvbmUgQ29weXJpZ2h0IGxpbmUgcGVyIGZpbGUgaXMg
ZW5vdWdoIDotKQ0KPj4+Pg0KPj4+PiBCdXQgYWN0dWFsbHkgYWxsIHRoZSBDb3B5cmlnaHQgbGlu
ZXMgd2l0aCBhIG5hbWUgb3IgYSBjb21wYW55IG5hbWUgYXJlDQo+Pj4+IG5vdCByZWFsbHkgcmVx
dWlyZWQgb3IgdXNlZnVsLCBhcyB0aGUgY29weXJpZ2h0IGlzIG5vdGVkIGluIGZ1bGwgZGV0YWls
cw0KPj4+PiBpbiB0aGUgY29tbWl0IG1lc3NhZ2VzIChhdXRob3IgYW5kIHNpZ25lZC1vZmYtYnkg
bGluZXMpLiBJIHdvdWxkIHJlbW92ZQ0KPj4+PiB0aGVtIGFsbCBmcm9tIHRoZSBuZXcgZmlsZXMg
YWRkZWQgYnkgdGhpcyBzZXJpZXMuDQo+Pj4gT2suIExldCBtZSByZW1vdmUgaW4gbmV4dCB2ZXJz
aW9uLg0KPj4+Pj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJl
ZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9kaWZ5DQo+Pj4+PiArICogaXQgdW5kZXIgdGhlIHRlcm1z
IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSB2ZXJzaW9uIDIgYXMNCj4+Pj4+ICsg
KiBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbi4NCj4+Pj4+ICsgKg0K
Pj4+Pj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0aGUgaG9wZSB0aGF0IGl0
IHdpbGwgYmUgdXNlZnVsLA0KPj4+Pj4gKyAqIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0
aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9mDQo+Pj4+PiArICogTUVSQ0hBTlRBQklM
SVRZIG9yIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRoZQ0KPj4+Pj4g
KyAqIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuDQo+Pj4+PiAr
ICoNCj4+Pj4+ICsgKiBZb3Ugc2hvdWxkIGhhdmUgcmVjZWl2ZWQgYSBjb3B5IG9mIHRoZSBHTlUg
R2VuZXJhbCBQdWJsaWMgTGljZW5zZQ0KPj4+Pj4gKyAqIGFsb25nIHdpdGggdGhpcyBwcm9ncmFt
LiAgSWYgbm90LCBzZWUgPGh0dHBzOi8vdXJsZGVmZW5zZS5jb20vdjMvX19odHRwOi8vd3d3Lmdu
dS5vcmcvbGljZW5zZXMvX187ISFHRl8yOWRiY1FJVUJQQSFsQWRMX0N2c3VNdXVYOWFpNmN3em0z
TllpVDF2d0lJbHhHVTduZXpTcXFfbnFKazQwWnota1Q0NExPc2VtY2doSl8zajJDZmZsUSQgW2du
dVsuXW9yZ10+Lg0KPj4+Pj4gKyAqLw0KPj4+Pj4gKw0KPj4+Pj4gKyNpbmNsdWRlIDxhc20vZGV2
aWNlLmg+DQo+Pj4+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+Pj4+ICsjaW5jbHVkZSA8YXNt
L3BjaS5oPg0KPj4+Pj4gKw0KPj4+Pj4gK3N0YXRpYyBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21h
dGNoIGdlbl9wY2lfZHRfbWF0Y2hbXSA9IHsNCj4+Pj4+ICsgICAgeyAuY29tcGF0aWJsZSA9ICJ4
bG54LG53bC1wY2llLTIuMTEiLA0KPj4+Pj4gKyAgICAgIC5kYXRhID0gICAgICAgJnBjaV9nZW5l
cmljX2VjYW1fb3BzIH0sDQo+Pj4+PiArICAgIHsgfSwNCj4+Pj4+ICt9Ow0KPj4+Pj4gKw0KPj4+
Pj4gK3N0YXRpYyBpbnQgZ2VuX3BjaV9kdF9pbml0KHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2
LCBjb25zdCB2b2lkICpkYXRhKQ0KPj4+Pj4gK3sNCj4+Pj4+ICsgICAgY29uc3Qgc3RydWN0IGR0
X2RldmljZV9tYXRjaCAqb2ZfaWQ7DQo+Pj4+PiArICAgIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9v
cHMgKm9wczsNCj4+Pj4+ICsNCj4+Pj4+ICsgICAgb2ZfaWQgPSBkdF9tYXRjaF9ub2RlKGdlbl9w
Y2lfZHRfbWF0Y2gsIGRldi0+ZGV2Lm9mX25vZGUpOw0KPj4+PiBUaGlzIHNob3VsZCBiZSBzdXBl
cmZsdW91cw0KPj4+IEFjay4gSSB3aWxsIHJlbW92ZSB0aGUgZHRfbWF0Y2hfbm9kZSguLikgaW4g
bmV4dCB2ZXJzaW9uLg0KPj4gSSBhbSBub3QgZW50aXJlbHkgc3VyZSB3ZSBuZWVkIHRoaXMgcGF0
Y2ggYXQgYWxsIGFzIHRoZSBtYWluIHJlYXNvbiBmb3IgaXRzIGV4aXN0ZW5jZQ0KPj4NCj4+IHdh
cyB0aGF0IHdlIGNhbiBydW4gWGlsaW54IFFFTVUgZm9yIFpDVTEwMi4gQnV0LCB0aGUgZmluYWwg
c2V0dXAgaXMgbm90IGdvaW5nDQo+Pg0KPj4gdG8gYmUgZnVuY3Rpb25hbCBhcyBsZWdhY3kgSVJR
cyBhcmUgbm90IHN1cHBvcnRlZCBhbmQgSVRTIGlzIG5vdCBhIHBhcnQgb2YgWnlucU1QLg0KPj4N
Cj4+IFNvLCBRRU1VIGFsbG93cyB0byBkbyBhIGxvdCB3aXRoIFBDSSBwYXNzdGhyb3VnaCwgYnV0
IGF0IHRoZSBlbmQgb2YgdGhlIGRheSBvbmUNCj4+DQo+PiB3b24ndCBoYXZlIGl0IHdvcmtpbmcu
Li4NCj4+DQo+PiBQbGVhc2UgY29uc2lkZXINCj4+DQo+PiBJZiB3ZSBkZWNpZGUgdG8gcmVtb3Zl
IGl0IHRoZW4NCj4+DQo+PiBpbnQgcGNpX2hvc3RfY29tbW9uX3Byb2JlKHN0cnVjdCBkdF9kZXZp
Y2Vfbm9kZSAqZGV2LA0KPj4gICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1fb3BzICpvcHMsDQo+PiAgIMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGludCBlY2FtX3Jl
Z19pZHgpDQo+Pg0KPj4gZG9lc24ndCBuZWVkIHRoZSBsYXN0IHBhcmFtZXRlci4NCj4gV2l0aCBt
eSBvcGVuIHNvdXJjZSBtYWludGFpbmVyIGhhdCBvbiwgSSBkb24ndCBzZWUgdGhpcyBwYXRjaCBh
cyB2ZXJ5DQo+IGltcG9ydGFudDsgZnJvbSB0aGF0IHBvaW50IG9mIHZpZXcgSSdkIGJlIGhhcHB5
IGZvciBpdCB0byBiZSBkcm9wcGVkLg0KPiBIb3dldmVyLCBpdCB3b3VsZCBiZSBnb29kIHRvIGhh
dmUgYXQgbGVhc3Qgb25lIG5vbi1kZWZhdWx0IGhvc3QgYnJpZGdlDQo+IChkb2Vzbid0IGhhdmUg
dG8gYmUgdGhlIFhpbGlueCBicmlkZ2UpLCBvdGhlcndpc2UgaXQgYmVjb21lcyBkaWZmaWN1bHQN
Cj4gdG8gdW5kZXJzdGFuZCBob3cgdGhlIGdlbmVyaWMgaW5mcmFzdHJ1Y3R1cmUgaW50cm9kdWNl
ZCBieSB0aGlzIHNlcmllcw0KPiBjb3VsZCBiZSB1c2VmdWwuDQo+DQo+IE1vcmVvdmVyLCB5b3Vy
IHJlY2VudCBjb21tZW50IFsxXSBtYWRlIGl0IGV2ZW4gbW9yZSBldmlkZW50IHRoYXQgaXQNCj4g
d291bGQgYmUgZ29vZCB0byBoYXZlIGF0IGxlYXN0IHR3byBkaWZmZXJlbnQgZHJpdmVycyB0byBz
cG90DQo+IGNvbXBhdGliaWxpdHkgaXNzdWVzIGJldHdlZW4gdGhlbSBtb3JlIGVhc2lseS4NCg0K
RG9uJ3QgdGFrZSBtZSB3cm9uZyBoZXJlIDspIEkgc3RpbGwgZG8gdXNlIFhpbGlueCBRRU1VIGZv
ciBtb3N0IG9mDQoNCnRoZSB0ZXN0cywgc28gaXQgaXMgZ29vZCBmb3IgbWUgdG8gaGF2ZSB0aGlz
IHBhdGNoIGluIHRoZSB0cmVlLiBCdXQsDQoNCnRvIGJlIGZhaXIsIFhpbGlueCBRRU1VIHdvbid0
IGdpdmUgeW91IGEgcG9zc2liaWxpdHkgdG8gc2VlIHRoZSBmdWxseQ0KDQpmdW5jdGlvbmFsIHN5
c3RlbS4gVGhpcyBpcyB3aHkgSSBzYXkgdGhlIHBhdGNoIGNhbiBiZSBkcm9wcGVkLg0KDQpJZiB3
ZSBhZGQgc29tZSB3b3JkcyBpbiB0aGUgY29tbWl0IG1lc3NhZ2UgYWJvdXQgdGhpcyBhbmQgaGF2
ZSB0aGUNCg0KcGF0Y2ggaW4gdGhlIHRyZWUgSSdsbCBiZSBtb3JlIHRoYW4gaGFwcHkNCg0KPg0K
PiBbMV0gaHR0cHM6Ly91cmxkZWZlbnNlLmNvbS92My9fX2h0dHBzOi8vbWFyYy5pbmZvLz9sPXhl
bi1kZXZlbCZtPTE2MzE1NDQ3NDAwODU5OF9fOyEhR0ZfMjlkYmNRSVVCUEEhbEFkTF9DdnN1TXV1
WDlhaTZjd3ptM05ZaVQxdndJSWx4R1U3bmV6U3FxX25xSms0MFp6LWtUNDRMT3NlbWNnaEpfMGJL
czZ6cEEkIFttYXJjWy5daW5mb10=


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 04:35:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 04:35:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186102.334833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ0Ae-0001zL-Fn; Tue, 14 Sep 2021 04:35:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186102.334833; Tue, 14 Sep 2021 04:35:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ0Ae-0001zE-B3; Tue, 14 Sep 2021 04:35:48 +0000
Received: by outflank-mailman (input) for mailman id 186102;
 Tue, 14 Sep 2021 04:35:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ0Ad-0001z8-5s
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 04:35:47 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3a0ce553-1515-11ec-b3f4-12813bfff9fa;
 Tue, 14 Sep 2021 04:35:46 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18E3wdqR005744; 
 Tue, 14 Sep 2021 04:35:45 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b2m9x02m9-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 04:35:45 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6769.eurprd03.prod.outlook.com (2603:10a6:20b:284::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Tue, 14 Sep
 2021 04:35:41 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 04:35:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a0ce553-1515-11ec-b3f4-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BoOtIWZh2/DPELcvQtLjPhHukonkO3OhkgCvqhSFdHMsA7TZQHhvLdb/ZywRlKhUl9Q2z+vLugtvR93c7ms6RfCYTgCWlOpR5p++t95P23U2wGrHZjqXFTH1beyS4RbTFW6KDb+mes1h/Q2xnBLiQ52nlmfhTdXQcgKQ15rggzwXfCjXG86Suwj9mgQkCF1Z0ezaci83zyWQ7xGUZvlNKhF4d15pJj+/LBNIbs6lAUKDPJJsb1vsBPqulJE8HRGV9t5jD6J+60CxEZwJKuZZ3jSMeBaUV2OuoA/59hE7zeJnMvJ9FjXMsCVhO2qTesdTNa3D2gOg0ytC10H4I0QgrA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TJOA6MUaPes8jOkE4ItpZOYt9qq325SK+uhICFJLD0k=;
 b=Z3sx2QVMOoqlM82epxinJCXBhciFKZlyKchPYDdKmQJchdjNxcpRp+xQ3v7VK7kcVZThF7uOGDNPgwFVGQPaqvdhske6OzXPMvgKzvjKgWr7GNHo/K4T12el8WTaxQ8zhZzBZ4ZC9o8JwGInvKpS7o0WOlYVLndR6IkHua7EIiB3bHO/QOZrnQd2OdItmottxMBQG8Qi+DHXt07zIx9gumCw+SPqKeg/H6alDzYNEM1Bvf40U5UWQN0rHD4MHoeH8VXlLVI/FlpHyTnjFomrUIMs/HywKeFaOESKo7tq/wlIUbejHKgCFn5sf+kXvJo7eSEf1VwiAg7nHMOOXCWMvg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TJOA6MUaPes8jOkE4ItpZOYt9qq325SK+uhICFJLD0k=;
 b=j4nlQ0ovz1kYKXlWRgUtLdsu4jM/qPTvt0InysPNfaaw8GwKmZ6rTVE52LaoslsS/IrJoQnRsLt1EcHtgPDYS7tNrbYgtw+qWZXwu9P8UJ6H4XZtbDoF0/fSdgGdK8Wpo846ovzgXYBLI7B3bHaWx083do9rjBT+NUI5l7EDDNuIg0s5LcZCPZChhsSy6SNkIHMyD4gPUhI6mi9nE1I3NqGLxJ+B1aA6klFuqQMND4pJqeJ2zLtz1A+unz41+RtY8MI6aT5OHN3980VKsa3JysisMRUpboD1rdLyhAPyD5RlZkfYdKQdtElAV/CUo0SVLAAG0Z6yJ7yo4LEvJ9CAEA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXqK7zTKPAnBLuBkixTqpTrxO8DquiaS0AgACJYwA=
Date: Tue, 14 Sep 2021 04:35:41 +0000
Message-ID: <b0fc12be-7228-b1c8-f98c-44adf92bfa16@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
 <c9484b8e-cad5-d4e4-a25f-749035fe5859@epam.com>
 <alpine.DEB.2.21.2109131322190.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109131322190.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4335ae96-3b18-43c4-e1d9-08d977391bcc
x-ms-traffictypediagnostic: AM9PR03MB6769:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB6769FCCDA3D078E6EFDE39E3E7DA9@AM9PR03MB6769.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 6Fif3tt/UDhJgS/rvEvH+UittFcWxWWegC5/ON5pKtZzN+6YK156282PtykDhHIy84itm/YZiOhA7GXNONlEQE7bgk8qivWRvNDgzmfFgrx73aD0jwZaW1ovQsZ16NZhveGyoU+x/X2z94Q5pRq1olVrqrawXWqUEsnaSv1zQHKvQW6Apygnyx/Yhb8UTsE19+zq/XKnxfXiV9GWE4/jw4G1sLgInO44lnpA9mVU3VErPKzHP8gjWFtjvf6Zf1JhB47R6KfT6aB1fS7ya6OXQd6bzsbiHDe35GKhCuk97SH+nO89bwfCfsZ529+7F8FXJ50otpkIV9oJMniLjJwYO5YR3GPkqqBbPNyscwurOO7Wtd9wb/YEQBnPRqTEOQjKcM1mufhmrElyoEjjCQRQjbeCwhPgfDeBQ1ZJyFPAl63QZWE17IJUNGAvRjTrs36NzFWuzMgq8dmnuqpG0ExWSsstHoKBEqo02QmEvVdVorvti4M9gJy4ak2wpGsCNfbC+oQJMR8kVK1L7iUOjpx63Ul6exjpd8cPomhJPJ96iFGscJStuKMwy/oN6DbHiUwzkOoFPjTZKt/bL71ZTFaZWULoVdRnBVHlLVnXU6ZPlB3gOx9dSlN/D0mFDRAv6Y1eCSyIVKREkQ9sIgV6Dc1k9yJjwUeMT/B6qTUEuS5FTqO6Fzi/01kxr8Qjj48NXiUnTlSn78mhCJdPtcTvO4bdvf6aq0+aLzo8igA+Ho2NjAmpqJbTPLKsTJVUyS33QDbM
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(376002)(396003)(346002)(39860400002)(366004)(54906003)(6916009)(5660300002)(2906002)(6512007)(8936002)(71200400001)(6486002)(86362001)(107886003)(66556008)(36756003)(76116006)(38070700005)(53546011)(316002)(6506007)(91956017)(66476007)(66946007)(2616005)(38100700002)(122000001)(478600001)(64756008)(31686004)(31696002)(55236004)(66446008)(4326008)(26005)(8676002)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ZzNBemxqWUkxcElHWlA5QWI5MGhkQnRPTjg2ejdPSlAwQm8rOXlpaWx2VFBS?=
 =?utf-8?B?cTNCNjRNRjQ4Wmh0aGVzY1REUHA2NzZ4NEJJblF2RnVtNXE1MVd1YUdkaWJh?=
 =?utf-8?B?Uk5wTi9qWTBadlltRXhPa3VJVnY4VHByWXlXQXZsc2ZaaTVqYUJoVlJQV3Rj?=
 =?utf-8?B?ZGtKbjBMQkg2SU4rNVg0eDNsRFB2VC92aWp5TjJEYjRodGppZi80VzJySU52?=
 =?utf-8?B?eTRYUEJMc3ZacTU1VmFsK2E1TVIvVXNxaFhOWjM1UHBBNnhFV002aW5nNjU2?=
 =?utf-8?B?QmZBZ1dFOU9sZ09VZGlyUkowZG9GL3VONWgxUkJIZmJkaEMvZ3dacHgxVURM?=
 =?utf-8?B?Y0lCSVJSYkZkSTYwZnlTQjluQzQvNXE3d09UdnF0TjRmT3NnZTZ4Z002dzd4?=
 =?utf-8?B?VFMrb0cyK1J6c2liS2FHdTFMZlFkai9SSkxrYjdTeUIyZllnRWY1VlZWN3dI?=
 =?utf-8?B?ZlNTSGF5aUtQbzV2ZW9RWWVlSGcyNEg0RjVGcEo1UFY4cUFzOW1zZm04aEcv?=
 =?utf-8?B?RDljRmN0OXZuenZxdFdvME1iUXVuYllWcURnOXlhNFoxMmFoMC9oRTBBemtm?=
 =?utf-8?B?cHFQWmlqcVB3bmt4cDNCWUxZa1lJVFdwbWp1eDl5RUQ4RTVleWdReGg4RU9i?=
 =?utf-8?B?cjNadE5pQjNSRGJqM1hIS1c1MnFzOWdLS3I5S1p3ZEdweElTeGdNTUh2Z0lF?=
 =?utf-8?B?WkVKYy9WRVZxODlLdU5Xb2JpMzJXTWtHT0VDcmFPbUdEYU0vWEhaVTZuSUF6?=
 =?utf-8?B?UnZuVzl4YitOY2JmVWlicUIyN3hPclYvS3ZwbzVXSm1UVjNXNnhDc1ZjZFN4?=
 =?utf-8?B?cmU0UXRDWDcxcW1pdHp6TUZNdDB0K0s0eDlaMjI5c2NQNTVsMnBEZ014UGJZ?=
 =?utf-8?B?clpwYU43dnMzaXVSUHlqQmhJaldCNDkwNjAvY1cvVlA3TWZRNnNuWHdHZEF4?=
 =?utf-8?B?d1ZYdFdKK3pnY2J2M3VCUUlQRmZ4cnFXVTQwcERBSUM4QzlrL2FDakEwdEJp?=
 =?utf-8?B?MTVjOFhyVUdLTVR3Z2I0NjZGd0E2aFhWVlNZS1ZObktmd2ZZdHM5TTA2MktN?=
 =?utf-8?B?UVVseWhCMmxQc21vUmZFMUNRZTNRdm1yUkwzdlNzQWg2eFZZUHplM0lWQ1ZX?=
 =?utf-8?B?SmhGamZLdGRkcWhBTmVVNUFTYWdxdGV2U1Ribm40Rld4cExVb3JNdDV3Ukd3?=
 =?utf-8?B?T1pNSURIVFVCWlNNbnhwakRHbi94cGQ3eFBDYXJuemx0YWtmRmZOcWFWcTF1?=
 =?utf-8?B?MGVWbVhQSkFKQzArR2cvayt1eEpLQ0h2VlovSGYxdFNIa3gwSHhNZFFnTWI4?=
 =?utf-8?B?c2dGbHI4TEJVSVdCQnpvdVhqckE0bE1Ga1dkaWJUKzI3MVhIOWdWWmQ3b2Iy?=
 =?utf-8?B?YkNPODVCRy9FZUFCRzlySCs4NC9uUVJlT1FHanNSOFBFdzg0RlJnaXhrMmho?=
 =?utf-8?B?OEZHNVkyK2pKYVBqUE1iYzB3cmwvQkUwWDVzendwUlIrZVV6aXNwdktpRkVk?=
 =?utf-8?B?b2ppc3c1YWwzZjBwVW1rZnpVL0txQXVqSitsUkJhbHdaa3ZDa3dtUGFzaHNW?=
 =?utf-8?B?YmR1bVN4TU5rTHNsc0dRUXRFdVJxRkF0dEJrTER3RVVKREdWU1BiRzZxck1n?=
 =?utf-8?B?NEgxczZSbHpIUElOeUU5YXdwWHowM0swRk10U1hDSGZmRWovYnFMODJPVmdt?=
 =?utf-8?B?eGJBMlBaQmpFc3B4eXhqQXIxUmJuN2dZZ0tObWRQUGpiTXB5UzNQTEZRTWM1?=
 =?utf-8?B?S3BMS0dlQ3JUZ05CTDFodXdPK0VkdFdVRHN5SnowY3pPQVkvbEk4ZGlaYjl1?=
 =?utf-8?B?c0tFa3JqUWF3ZUUwUzNvZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <EA25D76A5157C945858200BC27EF23EA@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4335ae96-3b18-43c4-e1d9-08d977391bcc
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 04:35:41.2042
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Z+OXIF4QtPmYb54aUsLNEe4sWAxxOn4iJrhpPxiDbPUzbz2zrcusPsEefaF4UE75D7pCzQD1eOaAeCohQrEZmlZc67U812slVVVD+3NkT55b77noaZdB9thl2VdLlpFo
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6769
X-Proofpoint-ORIG-GUID: y1Yp4mXo_w7xpOWdNBKwfG4-1KYWfQqh
X-Proofpoint-GUID: y1Yp4mXo_w7xpOWdNBKwfG4-1KYWfQqh
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-13_09,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 malwarescore=0 phishscore=0 suspectscore=0 adultscore=0 mlxlogscore=999
 clxscore=1015 bulkscore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140026

DQpPbiAxMy4wOS4yMSAyMzoyMywgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBNb24s
IDEzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEhpLCBSYWh1
bCENCj4+DQo+PiBPbiAxOS4wOC4yMSAxNTowMiwgUmFodWwgU2luZ2ggd3JvdGU6DQo+Pj4gWEVO
IGR1cmluZyBib290IHdpbGwgcmVhZCB0aGUgUENJIGRldmljZSB0cmVlIG5vZGUg4oCccmVn4oCd
IHByb3BlcnR5DQo+Pj4gYW5kIHdpbGwgbWFwIHRoZSBQQ0kgY29uZmlnIHNwYWNlIHRvIHRoZSBY
RU4gbWVtb3J5Lg0KPj4gW3NuaXBdDQo+Pj4gK3N0YXRpYyBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5k
b3cgKmdlbl9wY2lfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+PiArICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGludCBlY2FtX3JlZ19pZHgp
DQo+Pj4gK3sNCj4+PiArICAgIGludCBlcnI7DQo+Pj4gKyAgICBzdHJ1Y3QgcGNpX2NvbmZpZ193
aW5kb3cgKmNmZzsNCj4+PiArICAgIHBhZGRyX3QgYWRkciwgc2l6ZTsNCj4+PiArDQo+Pj4gKyAg
ICBjZmcgPSB4emFsbG9jKHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyk7DQo+Pj4gKyAgICBpZiAo
ICFjZmcgKQ0KPj4+ICsgICAgICAgIHJldHVybiBOVUxMOw0KPj4+ICsNCj4+PiArICAgIGVyciA9
IGR0X3BjaV9wYXJzZV9idXNfcmFuZ2UoZGV2LCBjZmcpOw0KPj4+ICsgICAgaWYgKCAhZXJyICkg
ew0KPj4+ICsgICAgICAgIGNmZy0+YnVzbl9zdGFydCA9IDA7DQo+Pj4gKyAgICAgICAgY2ZnLT5i
dXNuX2VuZCA9IDB4ZmY7DQo+Pj4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIiVzOk5vIGJ1
cyByYW5nZSBmb3VuZCBmb3IgcGNpIGNvbnRyb2xsZXJcbiIsDQo+Pj4gKyAgICAgICAgICAgICAg
IGR0X25vZGVfZnVsbF9uYW1lKGRldikpOw0KPj4+ICsgICAgfSBlbHNlIHsNCj4+PiArICAgICAg
ICBpZiAoIGNmZy0+YnVzbl9lbmQgPiBjZmctPmJ1c25fc3RhcnQgKyAweGZmICkNCj4+PiArICAg
ICAgICAgICAgY2ZnLT5idXNuX2VuZCA9IGNmZy0+YnVzbl9zdGFydCArIDB4ZmY7DQo+Pj4gKyAg
ICB9DQo+Pj4gKw0KPj4+ICsgICAgLyogUGFyc2Ugb3VyIFBDSSBlY2FtIHJlZ2lzdGVyIGFkZHJl
c3MqLw0KPj4+ICsgICAgZXJyID0gZHRfZGV2aWNlX2dldF9hZGRyZXNzKGRldiwgZWNhbV9yZWdf
aWR4LCAmYWRkciwgJnNpemUpOw0KPj4gSSBhbSBhIGJpdCB3b3JyaWVkIGhlcmUgdGhhdCB3ZSBk
b24ndCBnZXQgdGhlIHJlZyBpbmRleCBmcm9tIHRoZSBkZXZpY2UgdHJlZSwNCj4+DQo+PiBidXQg
Zm9yIGdlbmVyaWMgRUNBTSB3ZSB1c2UgcmVnWzBdIGFuZCBmb3IgWGlsaW54IHdlIHVzZSByZWdb
Ml0uDQo+Pg0KPj4gRm9yIGV4YW1wbGUsIGZvciBYaWxpbnggd2UgaGF2ZQ0KPj4NCj4+IHJlZyA9
IDwweDAwIDB4ZmQwZTAwMDAgMHgwMCAweDEwMDAgMHgwMCAweGZkNDgwMDAwIDB4MDAgMHgxMDAw
IDB4ODAgMHgwMCAweDAwIDB4MTAwMDAwMD47DQo+PiByZWctbmFtZXMgPSAiYnJlZ1wwcGNpcmVn
XDBjZmciOw0KPj4NCj4+IHNvLCB3ZSBjYW4gcGFyc2UgdGhlIHJlZy1uYW1lcyBhbmQgdW5kZXJz
dGFuZCB0aGF0IHRoZSBjb25maWd1cmF0aW9uIHNwYWNlIGlzIHRoZSBsYXN0IGluIHRoZSByZWcg
cHJvcGVydHkuDQo+Pg0KPj4gVGhlIHNhbWUgSSB0aGluayBjYW4gYmUgZG9uZSBmb3Igb3RoZXIg
ZGV2aWNlIHRyZWVzIHByb2JhYmx5Lg0KPiBXZWxsIHNwb3R0ZWQhDQo+DQo+DQo+PiBSYWh1bCwg
ZG8geW91IGtub3cgaWYgcmVnLW5hbWVzICJjZmciIGlzIHZlbmRvciBzcGVjaWZpYyBvZiB1c2Vk
IHdpZGVseT8NCj4gVW5mb3J0dW5hdGVseSBpdCBzZWVtcyB0byBiZSB2ZW5kb3Igc3BlY2lmaWMg
Oi0oDQo+IFdoaWNoIG1lYW5zIHRoYXQgImR0X2RldmljZV9nZXRfYWRkcmVzcyIgb3Igc2ltaWxh
ciBzaG91bGQgYmUgbW92ZWQgdG8gYQ0KPiB2ZW5kb3Igc3BlY2lmaWMgZnVuY3Rpb24uDQoNCkkg
anVzdCBzdGVwcGVkIG9uIHRoaXMgd2hpbGUgd29ya2luZyBvbiB0aGUgcGF0Y2ggZm9yIG5vdCBt
YXBwaW5nDQoNCk1NSU9zIGFuZCBjb25maWcgdG8gRG9tMCBhbmQgbmVlZCB0byBmaW5kIGEgd2F5
IHRvIHRlbGwgdGhlbSBmcm9tDQoNCmRldmljZSByZWdpc3RlcnMuIFNvLCB5ZXMsIGl0IHNlZW1z
IHdlIHdpbGwgbmVlZCBzb21ldGhpbmcgbW9yZSBnZW5lcmljDQoNCnRoYW4ganVzdCBhbiBpbmRl
eC4gV2UgY2FuIHN0aWxsIGhhdmUgdGhlIGNhbGxiYWNrIGZvciB0aGlzIG9wdGlvbmFsLA0KDQpz
byB0aGUgYnJpZGdlcyB3aXRoIHRoZSBzYW1lIHJlZyBuYW1lcyBjYW4gdXNlIHRoZSBkZWZhdWx0
IGNhbGxiYWNrDQoNCmFuZCBkb24ndCBuZWVkIHRvIGltcGxlbWVudCB0aGUgc2FtZQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 05:48:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 05:48:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186117.334844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1J2-0001J1-EF; Tue, 14 Sep 2021 05:48:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186117.334844; Tue, 14 Sep 2021 05:48:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1J2-0001Iu-BG; Tue, 14 Sep 2021 05:48:32 +0000
Received: by outflank-mailman (input) for mailman id 186117;
 Tue, 14 Sep 2021 05:48:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rbOp=OE=linux.ibm.com=rppt@srs-us1.protection.inumbo.net>)
 id 1mQ1J1-0001I3-2P
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 05:48:31 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.158.5])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e41cb6b8-8109-4ec0-bc3f-57996218d330;
 Tue, 14 Sep 2021 05:48:30 +0000 (UTC)
Received: from pps.filterd (m0098413.ppops.net [127.0.0.1])
 by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18E4Z3cw022268;
 Tue, 14 Sep 2021 01:48:15 -0400
Received: from pps.reinject (localhost [127.0.0.1])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3b2mub92ab-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 01:48:14 -0400
Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1])
 by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18E5ScOh028181;
 Tue, 14 Sep 2021 01:48:14 -0400
Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com
 [149.81.74.107])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3b2mub929x-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 01:48:14 -0400
Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1])
 by ppma03fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18E5m7mT029668;
 Tue, 14 Sep 2021 05:48:12 GMT
Received: from b06cxnps4075.portsmouth.uk.ibm.com
 (d06relay12.portsmouth.uk.ibm.com [9.149.109.197])
 by ppma03fra.de.ibm.com with ESMTP id 3b0m396afd-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 05:48:12 +0000
Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com
 [9.149.105.62])
 by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18E5mAxD42402234
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Tue, 14 Sep 2021 05:48:10 GMT
Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id DE347AE045;
 Tue, 14 Sep 2021 05:48:09 +0000 (GMT)
Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 2EBD8AE04D;
 Tue, 14 Sep 2021 05:48:09 +0000 (GMT)
Received: from linux.ibm.com (unknown [9.145.48.104])
 by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTPS;
 Tue, 14 Sep 2021 05:48:09 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e41cb6b8-8109-4ec0-bc3f-57996218d330
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc :
 subject : message-id : references : content-type : in-reply-to :
 content-transfer-encoding : mime-version; s=pp1;
 bh=/VpZtdNzYrqZuVxW4hiLpEQZyzc1BnA7ReJVDhGvUQM=;
 b=fRRa085lMIf10cd1d2LKhroHGVoD82F+WhYU2LacffXPmPzaesAqO6iF8/rhhdeY4lq/
 7nw5RQ2POFuEn/oCHDAF5JGnc8USWKF/+WcFMQl6qi0yea3VBC8otUUeD8R8885tyWeN
 Udqaya71hvz/fAP9WdQMrAhglHO5RDjHtI+XCEdRhpsrNedtYN2nGAkkR19HFUWEcYNx
 CkJntheHv9iZAztWpdSxVYalhP2c4TA3vrVLHCcswfsL7+MPWqXqO82vucM7fPfWk8Xf
 hb/K6QGORJIG25nCoaMOW/ldznO7R8HcPzt96K5ZpAtoTjAQCE8NG9LkiwF0Yzn2/2/j fA== 
Date: Tue, 14 Sep 2021 08:48:07 +0300
From: Mike Rapoport <rppt@linux.ibm.com>
To: Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>
Cc: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
        Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
        linux-kernel@vger.kernel.org,
        xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
Message-ID: <YUA3l0gnTh0gDJ4d@linux.ibm.com>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <YT9I+Xs9wOPVCIVd@mail-itl>
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: -uT1WsMIWRkcvek1E0xEUKDb7S2Mk9gy
X-Proofpoint-GUID: Lm4V19AmkqSPfjviWQ2kZm2WQyq4nFi0
Content-Transfer-Encoding: 8bit
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475
 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1011
 malwarescore=0 mlxlogscore=999 suspectscore=0 spamscore=0
 lowpriorityscore=0 mlxscore=0 adultscore=0 impostorscore=0 phishscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140018

Hi Marek,

On Mon, Sep 13, 2021 at 02:50:00PM +0200, Marek Marczykowski-Grecki wrote:
> Hi,
> 
> Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the
> kernel version.
> Test environment:
>  - Xen 4.14.2
>  - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
>  - Linux 5.13.13, confirmed also on 5.14
> 
> The crash happens only if the initramfs has earlycpio with microcode.

Does the crash happen if you boot the same kernel and initrd directly
without Xen?

> I don't have a serial console, but I've got a photo with crash message
> (from Xen, Linux doesn't managed to print anything):
> https://user-images.githubusercontent.com/726704/133084966-5038f37e-001b-4688-9f90-83d09be3dc2d.jpg
> 
> Transcription of some of it:
> 
>     mapping kernel into physical memory
>     about to get started
>     (XEN) Pagetable walk from ffffffff82810888:
>     (XEN)  L4[0x1ff] = 0000000332815067 0000000000002815
>     (XEN)  L3[0x1fe] = 0000000332816067 0000000000002816
>     (XEN)  L2[0x014] = 0000000334018067 0000000000004018
>     (XEN)  L1[0x010] = 0000000332810067 0000000000002810
>     (XEN) domain_crash_sync called from entry.S: fault at ffff82d04033e790 x86_64/entry.S#domain_crash_page_fault
>     (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
>     (XEN) ----[ Xen-4.14.2  x86_64  debug=n  Not tainted ]----
>     (XEN) CPU:    0
>     (XEN) RIP:    e033:[<0000000000000000>]

Is it possible to get the actual RIP of the instruction that faulted? 
Feeding that to scripts/faddr2line would be just lovely.
 
> I've bisected it down to the commit a799c2bd29d19c565f37fa038b31a0a1d44d0e4d
> 
>     x86/setup: Consolidate early memory reservations
> 
> Since this seems to affect Xen boot only, I'm copying xen-devel too.
> 
> Any ideas?

The only thing I can suggest for now is to move the reservations from
early_reserve_memory() back to where they were before this commit one by
one to see which move caused the crash.

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 06:09:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 06:09:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186133.334855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1d1-0003w4-Od; Tue, 14 Sep 2021 06:09:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186133.334855; Tue, 14 Sep 2021 06:09:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1d1-0003vx-LY; Tue, 14 Sep 2021 06:09:11 +0000
Received: by outflank-mailman (input) for mailman id 186133;
 Tue, 14 Sep 2021 06:09:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Vvn+=OE=linaro.org=francois.ozog@srs-us1.protection.inumbo.net>)
 id 1mQ1d0-0003vr-Pz
 for xen-devel@lists.xen.org; Tue, 14 Sep 2021 06:09:10 +0000
Received: from mail-ej1-x629.google.com (unknown [2a00:1450:4864:20::629])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42ca1a73-2553-4368-8875-c80cc8b593bf;
 Tue, 14 Sep 2021 06:09:08 +0000 (UTC)
Received: by mail-ej1-x629.google.com with SMTP id kt8so26287558ejb.13
 for <xen-devel@lists.xen.org>; Mon, 13 Sep 2021 23:09:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42ca1a73-2553-4368-8875-c80cc8b593bf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=x5x/V1DMBpywPx69cHTaTCF0A6jodnWq7plIsPoJoZk=;
        b=Z8u45XjMILFKL6nckgNBk1879wdxaTAqWCYIJBe+BOBzAzi8McV0zqm+CMG5JpN8XA
         gx6DSYrD1tXNYthKeuToJ//pa192J1PRfArkmnZR3MzwqNAqXmVNskFosVq+MbztsenH
         a0QVMmbmnRbSyGQzGTut94BiUiscpVm+5sljTvIK2eueg5qMAkIOrjTuIepwdfZ5MmNe
         2kWHAUG05Ztekfl+dzGHTbxik9VD1ClIckycGpjtVXWcUjRD+jlIxIGb2nOhn3OySec0
         J/K+60GmqMevwrPkQrDT0a+S5BwFepBU2auwPrerMnwXtFlp+f/sT6LXBsbbAdy/omrz
         f9JA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=x5x/V1DMBpywPx69cHTaTCF0A6jodnWq7plIsPoJoZk=;
        b=3CjSyAeApxsrtqWiBMyhCFK6QOzOHjNqzucv5ZuHeF3fiHJMQ2FUNAcBWkv759LPn7
         jRDA3zgBb0BzJvEPQ8VFyEIAcwevwDtMR0EfySTeoRumiH09P3onF5uxMf7+lQcHPZNx
         jv5uriQnXBI5pU5gd3b4wl8e+qoSdeXFbNpuqMEP5So3lQbCGn9Rz1jCdCixw+syJwUc
         Ue4ZtLQX8VmLSLf/PJH8Irl0hDjk5K1w403Mi10yyjdeTzBiyLe3TH5S/mjqq6eIJtYQ
         BCtGZNaChozW/Ums/MQK7MqDlPBqRF8AhGlpIpu7WXUcnKxOCVN+mUdTh/f/LSkOE6rB
         BJ4A==
X-Gm-Message-State: AOAM533iPOkBhfl7MFBHieE7NfhTuUkOJ1G3lmHe2Two8DXLlbGigM5u
	cXhZXLc9yboIJYMjLqb+9OmnrDHB51trGuB/P6Lcfw==
X-Google-Smtp-Source: ABdhPJwHefPyM/dt/KdBSgsrxLB2rVvE3l71w2Kr+TINTLh4FPMK7X9eRXOFh7owrwLasT/fiWrzFP7dbnrFty3nuBA=
X-Received: by 2002:a17:906:2c0b:: with SMTP id e11mr17193602ejh.284.1631599747573;
 Mon, 13 Sep 2021 23:09:07 -0700 (PDT)
MIME-Version: 1.0
References: <87v94ldrqq.fsf@linaro.org> <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa> <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
From: =?UTF-8?Q?Fran=C3=A7ois_Ozog?= <francois.ozog@linaro.org>
Date: Tue, 14 Sep 2021 08:08:56 +0200
Message-ID: <CAHFG_=WNfq7etkA1CG_xuPzaTJ7wtmBuWN5ES-uwwErLVCpEcw@mail.gmail.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
To: Stefano Stabellini <stefano.stabellini@xilinx.com>
Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	Artem_Mygaiev@epam.com, Bertrand.Marquis@arm.com, 
	Carl van Schaik <cvanscha@qti.qualcomm.com>, Elena Afanasova <eafanasova@gmail.com>, 
	Jan Kiszka <jan.kiszka@siemens.com>, Oleksandr_Tyshchenko@epam.com, 
	Stefan Hajnoczi <stefanha@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>, jgross@suse.com, julien@xen.org, 
	olekstysh@gmail.com, paul@xen.org, virtio-dev@lists.oasis-open.org, 
	xen-devel@lists.xen.org
Content-Type: multipart/alternative; boundary="0000000000005ec22105cbee6b8c"

--0000000000005ec22105cbee6b8c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi

Le mar. 14 sept. 2021 =C3=A0 01:51, Stefano Stabellini via Stratos-dev <
stratos-dev@op-lists.linaro.org> a =C3=A9crit :

> On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
> > > the second is how many context switches are involved in a transaction=
.
> > > Of course with all things there is a trade off. Things involving the
> > > very tightest latency would probably opt for a bare metal backend whi=
ch
> > > I think would imply hypervisor knowledge in the backend binary.
> >
> > In configuration phase of virtio device, the latency won't be a big
> matter.
> > In device operations (i.e. read/write to block devices), if we can
> > resolve 'mmap' issue, as Oleksandr is proposing right now, the only
> issue is
> > how efficiently we can deliver notification to the opposite side. Right=
?
> > And this is a very common problem whatever approach we would take.
> >
> > Anyhow, if we do care the latency in my approach, most of virtio-proxy-
> > related code can be re-implemented just as a stub (or shim?) library
> > since the protocols are defined as RPCs.
> > In this case, however, we would lose the benefit of providing "single
> binary"
> > BE.
> > (I know this is is an arguable requirement, though.)
>
> In my experience, latency, performance, and security are far more
> important than providing a single binary.
>
> In my opinion, we should optimize for the best performance and security,
> then be practical on the topic of hypervisor agnosticism. For instance,
> a shared source with a small hypervisor-specific component, with one
> implementation of the small component for each hypervisor, would provide
> a good enough hypervisor abstraction. It is good to be hypervisor
> agnostic, but I wouldn't go extra lengths to have a single binary. I
> cannot picture a case where a BE binary needs to be moved between
> different hypervisors and a recompilation is impossible (BE, not FE).
> Instead, I can definitely imagine detailed requirements on IRQ latency
> having to be lower than 10us or bandwidth higher than 500 MB/sec.
>
> Instead of virtio-proxy, my suggestion is to work together on a common
> project and common source with others interested in the same problem.
>
> I would pick something like kvmtool as a basis. It doesn't have to be
> kvmtools, and kvmtools specifically is GPL-licensed, which is
> unfortunate because it would help if the license was BSD-style for ease
> of integration with Zephyr and other RTOSes.
>
> As long as the project is open to working together on multiple
> hypervisors and deployment models then it is fine. For instance, the
> shared source could be based on OpenAMP kvmtool [1] (the original
> kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
> kvmtool was created to add support for hypervisor-less virtio but they
> are very open to hypervisors too. It could be a good place to add a Xen
> implementation, a KVM fatqueue implementation, a Jailhouse
> implementation, etc. -- work together toward the common goal of a single
> BE source (not binary) supporting multiple different deployment models.
>
i like the hypervisor-less approach described in the link below. it can
also be used to drfine abstract HAL between normal world and TrustZone to
implement confidential workloads in the TZ. Virtio-sock is of particular
interest.
In addition, this can define a HAL that can be re-used in many contexts :
could we use this to implement something similar to Android Generic Kernel
Image stuff ?

>
>
> [1] https://github.com/OpenAMP/kvmtool
> --
> Stratos-dev mailing list
> Stratos-dev@op-lists.linaro.org
> https://op-lists.linaro.org/mailman/listinfo/stratos-dev
>
--=20
Fran=C3=A7ois-Fr=C3=A9d=C3=A9ric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

--0000000000005ec22105cbee6b8c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Hi</div><div><br><div class=3D"gmail_quote"><div dir=3D"l=
tr" class=3D"gmail_attr">Le=C2=A0mar. 14 sept. 2021 =C3=A0 01:51, Stefano S=
tabellini via Stratos-dev &lt;<a href=3D"mailto:stratos-dev@op-lists.linaro=
.org">stratos-dev@op-lists.linaro.org</a>&gt; a =C3=A9crit=C2=A0:<br></div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">On Mon, 6 Sep 2021, AKASHI Takahiro wrote:<b=
r>
&gt; &gt; the second is how many context switches are involved in a transac=
tion.<br>
&gt; &gt; Of course with all things there is a trade off. Things involving =
the<br>
&gt; &gt; very tightest latency would probably opt for a bare metal backend=
 which<br>
&gt; &gt; I think would imply hypervisor knowledge in the backend binary.<b=
r>
&gt; <br>
&gt; In configuration phase of virtio device, the latency won&#39;t be a bi=
g matter.<br>
&gt; In device operations (i.e. read/write to block devices), if we can<br>
&gt; resolve &#39;mmap&#39; issue, as Oleksandr is proposing right now, the=
 only issue is<br>
&gt; how efficiently we can deliver notification to the opposite side. Righ=
t?<br>
&gt; And this is a very common problem whatever approach we would take.<br>
&gt; <br>
&gt; Anyhow, if we do care the latency in my approach, most of virtio-proxy=
-<br>
&gt; related code can be re-implemented just as a stub (or shim?) library<b=
r>
&gt; since the protocols are defined as RPCs.<br>
&gt; In this case, however, we would lose the benefit of providing &quot;si=
ngle binary&quot;<br>
&gt; BE.<br>
&gt; (I know this is is an arguable requirement, though.)<br>
<br>
In my experience, latency, performance, and security are far more<br>
important than providing a single binary.<br>
<br>
In my opinion, we should optimize for the best performance and security,<br=
>
then be practical on the topic of hypervisor agnosticism. For instance,<br>
a shared source with a small hypervisor-specific component, with one<br>
implementation of the small component for each hypervisor, would provide<br=
>
a good enough hypervisor abstraction. It is good to be hypervisor<br>
agnostic, but I wouldn&#39;t go extra lengths to have a single binary. I<br=
>
cannot picture a case where a BE binary needs to be moved between<br>
different hypervisors and a recompilation is impossible (BE, not FE).<br>
Instead, I can definitely imagine detailed requirements on IRQ latency<br>
having to be lower than 10us or bandwidth higher than 500 MB/sec.<br>
<br>
Instead of virtio-proxy, my suggestion is to work together on a common<br>
project and common source with others interested in the same problem.<br>
<br>
I would pick something like kvmtool as a basis. It doesn&#39;t have to be<b=
r>
kvmtools, and kvmtools specifically is GPL-licensed, which is<br>
unfortunate because it would help if the license was BSD-style for ease<br>
of integration with Zephyr and other RTOSes.<br>
<br>
As long as the project is open to working together on multiple<br>
hypervisors and deployment models then it is fine. For instance, the<br>
shared source could be based on OpenAMP kvmtool [1] (the original<br>
kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP<br=
>
kvmtool was created to add support for hypervisor-less virtio but they<br>
are very open to hypervisors too. It could be a good place to add a Xen<br>
implementation, a KVM fatqueue implementation, a Jailhouse<br>
implementation, etc. -- work together toward the common goal of a single<br=
>
BE source (not binary) supporting multiple different deployment models.<br>
</blockquote><div dir=3D"auto">i like the hypervisor-less approach describe=
d in the link below. it can also be used to drfine abstract HAL between nor=
mal world and TrustZone to implement confidential workloads in the TZ. Virt=
io-sock is of particular interest.=C2=A0</div><div dir=3D"auto">In addition=
, this can define a HAL that can be re-used in many contexts : could we use=
 this to implement something similar to Android Generic Kernel Image stuff =
?</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-=
left:1px #ccc solid;padding-left:1ex"><br>
<br>
[1] <a href=3D"https://github.com/OpenAMP/kvmtool" rel=3D"noreferrer" targe=
t=3D"_blank">https://github.com/OpenAMP/kvmtool</a><br>
-- <br>
Stratos-dev mailing list<br>
<a href=3D"mailto:Stratos-dev@op-lists.linaro.org" target=3D"_blank">Strato=
s-dev@op-lists.linaro.org</a><br>
<a href=3D"https://op-lists.linaro.org/mailman/listinfo/stratos-dev" rel=3D=
"noreferrer" target=3D"_blank">https://op-lists.linaro.org/mailman/listinfo=
/stratos-dev</a><br>
</blockquote></div></div>-- <br><div dir=3D"ltr" class=3D"gmail_signature" =
data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><=
div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div =
dir=3D"ltr"><div><div dir=3D"ltr"><div><div><div><div dir=3D"ltr"><div dir=
=3D"ltr"><div dir=3D"ltr"><table style=3D"font-size:small" border=3D"0" cel=
lpadding=3D"0" cellspacing=3D"0"><tbody><tr><td style=3D"padding-right:10px=
" valign=3D"top"><img src=3D"https://drive.google.com/a/linaro.org/uc?id=3D=
0BxTAygkus3RgQVhuNHMwUi1mYWc&amp;export=3Ddownload" width=3D"96" height=3D"=
53"></td><td valign=3D"top"><table border=3D"0" cellpadding=3D"0" cellspaci=
ng=3D"0"><tbody><tr><td style=3D"font-family:Arial,Helvetica,&quot;Sans Ser=
if&quot;;white-space:nowrap;font-size:9pt;padding-top:0px;color:rgb(87,87,8=
7)" valign=3D"top"><span style=3D"font-weight:bold">Fran=C3=A7ois-Fr=C3=A9d=
=C3=A9ric Ozog</span>=C2=A0<span style=3D"color:rgb(161,161,161)">|</span>=
=C2=A0<i>Director Business Development</i></td></tr><tr><td style=3D"font-f=
amily:Arial,Helvetica,&quot;Sans Serif&quot;;white-space:nowrap;font-size:9=
pt;padding-top:2px;color:rgb(87,87,87)" valign=3D"top">T:=C2=A0<a value=3D"=
+393384075993" style=3D"color:rgb(17,85,204)">+33.67221.6485</a><br><a href=
=3D"mailto:francois.ozog@linaro.org" style=3D"color:rgb(87,87,87);text-deco=
ration:none" target=3D"_blank">francois.ozog@linaro.org</a>=C2=A0<span styl=
e=3D"color:rgb(161,161,161)">|</span>=C2=A0Skype:=C2=A0ffozog</td></tr></tb=
ody></table></td></tr></tbody></table></div></div></div></div></div><div><d=
iv><br style=3D"font-size:small"></div></div></div></div></div></div></div>=
</div></div></div></div></div></div></div></div></div></div>

--0000000000005ec22105cbee6b8c--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 06:23:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 06:23:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186131.334866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1r1-0006K3-F4; Tue, 14 Sep 2021 06:23:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186131.334866; Tue, 14 Sep 2021 06:23:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ1r1-0006Jw-5s; Tue, 14 Sep 2021 06:23:39 +0000
Received: by outflank-mailman (input) for mailman id 186131;
 Tue, 14 Sep 2021 05:57:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RpFI=OE=gmail.com=ysaikiran1997@srs-us1.protection.inumbo.net>)
 id 1mQ1Rn-0002qZ-Fp
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 05:57:35 +0000
Received: from mail-yb1-xb2a.google.com (unknown [2607:f8b0:4864:20::b2a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f181a3d-a975-4519-891d-06086914d4e7;
 Tue, 14 Sep 2021 05:57:33 +0000 (UTC)
Received: by mail-yb1-xb2a.google.com with SMTP id z5so25749794ybj.2
 for <xen-devel@lists.xenproject.org>; Mon, 13 Sep 2021 22:57:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f181a3d-a975-4519-891d-06086914d4e7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=xUjKkAvkcYsTqJY1uScD8p/o97laM6WXDCum/41Odv8=;
        b=SLlpJZih2wBR/uj1xLXJt4kgC4aF81RJdIvefI5XfANZD4x09SXLILOnnxsjskiibg
         9vnpk18NsEwS2bKicnUPQS0yCF5UATAfKzBT+22m9Y6brhTTAbVKDASnEM5INcYkhSAq
         wOUeByLWfX/sM1ONxiooPKqlYP7k1UnGP40BSqD2BFNvcFq4Qa6MH3Pf0uZZ3aGNdzHF
         G1/QGyf6atwmn4HcaxENDt3va2MGYOpJMW43tS9P5E3cWvpXonbT6CoqY7p0xnVlM40k
         yYK+UYvw1fPj9HMmXiJBZEkR8SjBr5skREDJ+edR0A6QcdTmJ9OldWFxIGgTBDyVOPCl
         lzSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=xUjKkAvkcYsTqJY1uScD8p/o97laM6WXDCum/41Odv8=;
        b=KUr/hDLl3S0cDxkhJeqpuJYJxMwNsOs49EIluUkbNAgZmUtwDAnaSwTMOYhhw7ladn
         Vf909WlbzKGfYeDyQubBN8vjg31fwO0QLhkMVb360VqRVeHxPbvVcnH9fAqbM7pKClHy
         3RyuqEVcOnsD3/7gphXLaU07GJNvKZe9PQGkY0l1SYeEdgHZbJx6Hc4QFeI3aSudfySI
         P1xpfr5TF6EqpocOAxOO1Vv/j4Db/4nE8B0fc9zqPVuW2lBUSco5wohBMmtUXr+kdTLj
         oC0wKdjAuUOpfy/Tib8zrzE44QTSNrI2NtkQcTSckohbfjN5ERsPSMdfT/yRAXDZNcJU
         zfug==
X-Gm-Message-State: AOAM5315acduMwtCypwfyXPfL48EtR6cuOUJY2RxlEOadZEzdCYHxANI
	0MWuYn7snoChlS+8eKrBCK376VfiO28DnRHFMmY=
X-Google-Smtp-Source: ABdhPJzyGYSiZ/Tg3WkdTicrgsJ0UjW5hWrwvNsdcjIGEwV75cuPpGvCS4HSozDq9MHkVyiKnarpVzMk9kc+sYharK4=
X-Received: by 2002:a25:b94:: with SMTP id 142mr19786345ybl.508.1631599052895;
 Mon, 13 Sep 2021 22:57:32 -0700 (PDT)
MIME-Version: 1.0
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol>
 <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org> <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com>
 <alpine.DEB.2.21.2109131437030.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109131437030.10523@sstabellini-ThinkPad-T480s>
From: Sai Kiran Kumar Reddy Y <ysaikiran1997@gmail.com>
Date: Tue, 14 Sep 2021 11:27:21 +0530
Message-ID: <CAEsO4uwpiDO2QoqLRTOxpsYM9YSPAsbX0P=gagdEy21pk1VsPQ@mail.gmail.com>
Subject: Re: Xen Booting Problem on ARM Machine
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
	Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, kannan@cimware.in
Content-Type: multipart/mixed; boundary="000000000000f7287605cbee41cf"

--000000000000f7287605cbee41cf
Content-Type: multipart/alternative; boundary="000000000000f7287205cbee41cd"

--000000000000f7287205cbee41cd
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

In the folder "/var/log", there's a file called "xen", which is empty. As
far as the boot logs are concerned, I don't see any debug messages related
to xen. I am attaching the log files, "kern.txt" and "boot.txt"

On Tue, Sep 14, 2021 at 3:08 AM Stefano Stabellini <sstabellini@kernel.org>
wrote:

> On Mon, 13 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
> > On Fri, Sep 10, 2021 at 7:30 PM Julien Grall <julien@xen.org> wrote:
> >
> >
> >       On 08/09/2021 11:43, Sai Kiran wrote:
> >       > Hello,
> >
> >       Hi,
> >
> >       Thank you for the report. Moving the discussion back to xen-devel
> >       because this looks like a potential issue in the UEFI stub in Xen=
.
> >
> >       > I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed o=
n
> it.
> >
> >       Would you be able to give more details on the Arm machine you are
> using?
> >       Also, are you using ACPI or DT to boot?
> >
> >
> >    Sai >> DT . ACPI configuration is disabled in Boot settings
> >
> >       >  I
> >       > am able to do =E2=80=9Cmake world=E2=80=9D and =E2=80=9Cmake in=
stall=E2=80=9D, after
> =E2=80=9C./configure=E2=80=9D, as
> >       > specified in README file. When I reboot the system, I get the
> following
> >       > message:
> >       >
> >       > Warning: All 128 bootinfo mem banks exhausted.
> >       >
> >       > Warning: All 128 bootinfo mem banks exhausted.
> >
> >       Hmmm... This means that you have more than 128 memory regions
> described
> >       in the EFI memory map. That's quite a lot.
> >
> >       Although, this should be harmless as it means Xen will not use th=
e
> extra
> >       memory banks.
> >
> >       >
> >       > Cannot exit boot services: ErrCode: 0x8000000000000002
> >
> >       This means EFI_INVALID_PARAMETER. We have code to retry because
> AFAICT
> >       ExitBootServices() may sometime fails (I have CCed Jan may have
> more
> >       idea what's happening).
> >
> >       Would you be able to provide more details on the UEFI firmware yo=
u
> are
> >       using? Is it EDK2 or U-boot?
> >
> > Sai >>  EDK2
> >       Also, do you know if Linux is boot on the same system?
> >
> > Sai >> Yes
> >       However, AFAICT, the error message would not prevent Xen to
> continue
> >       booting. So you may get stuck later in the boot process.
> >
> >       My suggestion would be to enable earlyprintk for your platform.
> You can
> >       setup it up from the menuconfig in "Debugging Options".
> >
> > Sai >> Yes, I have enabled earlyprintk.
> > I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to
> 256, from 128. The error message is no longer seen, but the device
> > is stuck in the boot process.
>
> Could you please post the boot logs now that you enabled earlyprintk?
> Ideally not a camera picture but a textual copy/paste from the target
> serial?
>
> Earlyprintk is pretty verbose, we should be able to figure out where it
> gets stuck.

--000000000000f7287205cbee41cd
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">In the folder &quot;/var/log&quot;, there&#39;s a file cal=
led &quot;xen&quot;, which is empty. As far as the boot logs are=C2=A0conce=
rned, I don&#39;t see any debug messages related to xen. I am attaching the=
 log files, &quot;kern.txt&quot; and &quot;boot.txt&quot;<br></div><br><div=
 class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Sep 14=
, 2021 at 3:08 AM Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kern=
el.org" target=3D"_blank">sstabellini@kernel.org</a>&gt; wrote:<br></div><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft:1px solid rgb(204,204,204);padding-left:1ex">On Mon, 13 Sep 2021, Sai Ki=
ran Kumar Reddy Y wrote:<br>
&gt; On Fri, Sep 10, 2021 at 7:30 PM Julien Grall &lt;<a href=3D"mailto:jul=
ien@xen.org" target=3D"_blank">julien@xen.org</a>&gt; wrote:<br>
&gt; <br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0On 08/09/2021 11:43, Sai Kiran wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Hello,<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Hi,<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Thank you for the report. Moving the discuss=
ion back to xen-devel<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0because this looks like a potential issue in=
 the UEFI stub in Xen.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; I have Xen-4.15.0 on an ARM Machine, wi=
th Debian 11 installed on it.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Would you be able to give more details on th=
e Arm machine you are using?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Also, are you using ACPI or DT to boot?<br>
&gt; <br>
&gt; <br>
&gt; =C2=A0 =C2=A0Sai &gt;&gt; DT . ACPI configuration is disabled in Boot =
settings=C2=A0 =C2=A0=C2=A0<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 I<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; am able to do =E2=80=9Cmake world=E2=80=
=9D and =E2=80=9Cmake install=E2=80=9D, after =E2=80=9C./configure=E2=80=9D=
, as<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; specified in README file. When I reboot=
 the system, I get the following<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; message:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Warning: All 128 bootinfo mem banks exh=
austed.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Warning: All 128 bootinfo mem banks exh=
austed.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Hmmm... This means that you have more than 1=
28 memory regions described<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0in the EFI memory map. That&#39;s quite a lo=
t.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Although, this should be harmless as it mean=
s Xen will not use the extra<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0memory banks.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Cannot exit boot services: ErrCode: 0x8=
000000000000002<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0This means EFI_INVALID_PARAMETER. We have co=
de to retry because AFAICT<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0ExitBootServices() may sometime fails (I hav=
e CCed Jan may have more<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0idea what&#39;s happening).<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Would you be able to provide more details on=
 the UEFI firmware you are<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0using? Is it EDK2 or U-boot?<br>
&gt; <br>
&gt; Sai &gt;&gt;=C2=A0 EDK2=C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Also, do you know if Linux is boot on the sa=
me system?<br>
&gt; <br>
&gt; Sai &gt;&gt; Yes=C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0However, AFAICT, the error message would not=
 prevent Xen to continue<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0booting. So you may get stuck later in the b=
oot process.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0My suggestion would be to enable earlyprintk=
 for your platform. You can<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0setup it up from the menuconfig in &quot;Deb=
ugging Options&quot;.<br>
&gt; <br>
&gt; Sai &gt;&gt; Yes, I have enabled earlyprintk.<br>
&gt; I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to=
 256, from 128. The error message is no longer seen, but the device<br>
&gt; is stuck in the boot process.<br>
<br>
Could you please post the boot logs now that you enabled earlyprintk?<br>
Ideally not a camera picture but a textual copy/paste from the target<br>
serial?<br>
<br>
Earlyprintk is pretty verbose, we should be able to figure out where it<br>
gets stuck.</blockquote></div>

--000000000000f7287205cbee41cd--
--000000000000f7287605cbee41cf
Content-Type: text/plain; charset="windows-1252"; name="boot.txt"
Content-Disposition: attachment; filename="boot.txt"
Content-Transfer-Encoding: base64
Content-ID: <f_ktjnxjni0>
X-Attachment-Id: f_ktjnxjni0

LS0tLS0tLS0tLS0tIE1vbiBTZXAgMTMgMTI6MTM6MDAgSVNUIDIwMjEgLS0tLS0tLS0tLS0tCi9k
ZXYvbnZtZTNuMXAyOiBjbGVhbiwgMTAxMjQ1LzE1MjU5MjAgZmlsZXMsIDg3NzQ4NS82MTAzNTUy
IGJsb2Nrcw0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1DcmVhdGUgVm9s
YXRpbGUgRmlsZXMgYW5kIERpcmVjdG9yaWVzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsx
OzM5bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRpb24bWzBtLi4uDQogICAgICAgICBTdGFydGlu
ZyAbWzA7MTszOW1VcGRhdGUgVVRNUCBhYm91dCBTeXN0ZW0gQm9vdC9TaHV0ZG93bhtbMG0uLi4N
ClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltVXBkYXRlIFVUTVAgYWJvdXQg
U3lzdGVtIEJvb3QvU2h1dGRvd24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtb
MDsxOzM5bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTZXQbWzBtLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTeW5jaHJv
bml6ZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Mb2FkIEFw
cEFybW9yIHByb2ZpbGVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQg
G1swOzE7MzltU3lzdGVtIEluaXRpYWxpemF0aW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0g
U3RhcnRlZCAbWzA7MTszOW1UcmlnZ2VyIGFuYWNyb24gZXZlcnkgaG91chtbMG0uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IGRvd25sb2FkIGFjdGl2aXRp
ZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IGFwdCB1
cGdyYWRlIGFuZCBjbGVhbiBhY3Rpdml0aWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3Rh
cnRlZCAbWzA7MTszOW1QZXJpb2RpYyBleHQ0IE9ubGnigKZhdGEgQ2hlY2sgZm9yIEFsbCBGaWxl
c3lzdGVtcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGlzY2Fy
ZCB1bnVzZWQgYmxvY2tzIG9uY2UgYSB3ZWVrG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3Rh
cnRlZCAbWzA7MTszOW1EYWlseSByb3RhdGlvbiBvZiBsb2cgZmlsZXMbWzBtLg0KWxtbMDszMm0g
IE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IG1hbi1kYiByZWdlbmVyYXRpb24bWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IENsZWFudXAgb2Yg
VGVtcG9yYXJ5IERpcmVjdG9yaWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0
YXJnZXQgG1swOzE7MzltVGltZXJzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5n
IG9uIBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrIEFjdGl2YXRpb24gU29ja2V0G1sw
bS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUQtQnVzIFN5c3Rl
bSBNZXNzYWdlIEJ1cyBTb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRh
cmdldCAbWzA7MTszOW1Tb2NrZXRzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0
YXJnZXQgG1swOzE7MzltQmFzaWMgU3lzdGVtG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3Rh
cnRlZCAbWzA7MTszOW1SdW4gYW5hY3JvbiBqb2JzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtb
MDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBTdGFydGVkIBtbMDsxOzM5bVJlZ3VsYXIgYmFja2dyb3VuZCBwcm9ncmFtIHByb2Nlc3Npbmcg
ZGFlbW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1ELUJ1cyBT
eXN0ZW0gTWVzc2FnZSBCdXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTmV0d29y
ayBNYW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltUmVtb3ZlIFN0YWxl
IE9ubGnigKZ0NCBNZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLi4uDQogICAgICAgICBTdGFy
dGluZyAbWzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0u
Li4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNlcxtb
MG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdlchtb
MG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVN5c3RlbSBMb2dnaW5nIFNlcnZpY2Ub
WzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Vc2VyIExvZ2luIE1hbmFnZW1lbnQb
WzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1EaXNrIE1hbmFnZXIbWzBtLi4uDQog
ICAgICAgICBTdGFydGluZyAbWzA7MTszOW1XUEEgc3VwcGxpY2FudBtbMG0uLi4NClsbWzA7MzJt
ICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltSW5pdGlhbGl6ZSBoYXJkd2FyZSBtb25pdG9y
aW5nIHNlbnNvcnMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVN5
c3RlbSBMb2dnaW5nIFNlcnZpY2UbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAb
WzA7MTszOW1SZW1vdmUgU3RhbGUgT25saeKApmV4dDQgTWV0YWRhdGEgQ2hlY2sgU25hcHNob3Rz
G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1BdmFoaSBtRE5TL0RO
Uy1TRCBTdGFjaxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJh
aXNlIG5ldHdvcmsgaW50ZXJmYWNlcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltV1BBIHN1cHBsaWNhbnQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVk
IBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdlchtbMG0uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1Nb2RlbSBNYW5hZ2VyG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVk
IBtbMDsxOzM5bVVzZXIgTG9naW4gTWFuYWdlbWVudBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IFN0YXJ0ZWQgG1swOzE7MzltTmV0d29yayBNYW5hZ2VyG1swbS4NClsbWzA7MzJtICBPSyAgG1sw
bV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltTmV0d29yaxtbMG0uDQogICAgICAgICBTdGFydGlu
ZyAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIgV2FpdCBPbmxpbmUbWzBtLi4uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1UaGUgQXBhY2hlIEhUVFAgU2VydmVyG1swbS4uLg0KICAgICAgICAg
U3RhcnRpbmcgG1swOzE7MzltT3BlbkJTRCBTZWN1cmUgU2hlbGwgc2VydmVyG1swbS4uLg0KICAg
ICAgICAgU3RhcnRpbmcgG1swOzE7MzltUGVybWl0IFVzZXIgU2Vzc2lvbnMbWzBtLi4uDQogICAg
ICAgICBTdGFydGluZyAbWzA7MTszOW1Ib3N0bmFtZSBTZXJ2aWNlG1swbS4uLg0KWxtbMDszMm0g
IE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uDQog
ICAgICAgICBTdGFydGluZyAbWzA7MTszOW1MaWdodCBEaXNwbGF5IE1hbmFnZXIbWzBtLi4uDQog
ICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Ib2xkIHVudGlsIGJvb3QgcHJvY2VzcyBmaW5pc2hl
cyB1cBtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1Nb2RlbSBN
YW5hZ2VyG1swbS4NCi0tLS0tLS0tLS0tLSBNb24gU2VwIDEzIDEyOjM4OjQ1IElTVCAyMDIxIC0t
LS0tLS0tLS0tLQovZGV2L252bWUzbjFwMjogY2xlYW4sIDIxMDgwNi8xNTI1OTIwIGZpbGVzLCAx
NTQ3NjExLzYxMDM1NTIgYmxvY2tzDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsx
OzM5bUZsdXNoIEpvdXJuYWwgdG8gUGVyc2lzdGVudCBTdG9yYWdlG1swbS4NCiAgICAgICAgIFN0
YXJ0aW5nIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGlsZSBGaWxlcyBhbmQgRGlyZWN0b3JpZXMbWzBt
Li4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGls
ZSBGaWxlcyBhbmQgRGlyZWN0b3JpZXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7Mzlt
TmV0d29yayBUaW1lIFN5bmNocm9uaXphdGlvbhtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtb
MDsxOzM5bVVwZGF0ZSBVVE1QIGFib3V0IFN5c3RlbSBCb290L1NodXRkb3duG1swbS4uLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1VcGRhdGUgVVRNUCBhYm91dCBTeXN0
ZW0gQm9vdC9TaHV0ZG93bhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsx
OzM5bUxvYWQgQXBwQXJtb3IgcHJvZmlsZXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltUmFpc2UgbmV0d29yayBpbnRlcmZhY2VzG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBG
aW5pc2hlZCAbWzA7MTszOW1SYWlzZSBuZXR3b3JrIGludGVyZmFjZXMbWzBtLg0KWxtbMDszMm0g
IE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRpb24b
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0g
SW5pdGlhbGl6YXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5
bURhaWx5IENsZWFudXAgb2YgVGVtcG9yYXJ5IERpcmVjdG9yaWVzG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltU3lzdGVtIFRpbWUgU2V0G1swbS4NClsb
WzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltU3lzdGVtIFRpbWUgU3lu
Y2hyb25pemVkG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1Ucmln
Z2VyIGFuYWNyb24gZXZlcnkgaG91chtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltRGFpbHkgYXB0IGRvd25sb2FkIGFjdGl2aXRpZXMbWzBtLg0KWxtbMDszMm0gIE9L
ICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IGFwdCB1cGdyYWRlIGFuZCBjbGVhbiBhY3Rp
dml0aWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1QZXJpb2Rp
YyBleHQ0IE9ubGnigKZhdGEgQ2hlY2sgZm9yIEFsbCBGaWxlc3lzdGVtcxtbMG0uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGlzY2FyZCB1bnVzZWQgYmxvY2tzIG9uY2Ug
YSB3ZWVrG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSBy
b3RhdGlvbiBvZiBsb2cgZmlsZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtb
MDsxOzM5bURhaWx5IG1hbi1kYiByZWdlbmVyYXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1UaW1lcnMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBMaXN0ZW5pbmcgb24gG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sgQWN0aXZhdGlv
biBTb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7Mzlt
RC1CdXMgU3lzdGVtIE1lc3NhZ2UgQnVzIFNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVNvY2tldHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1CYXNpYyBTeXN0ZW0bWzBtLg0KWxtbMDszMm0gIE9L
ICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVJ1biBhbmFjcm9uIGpvYnMbWzBtLg0KICAgICAgICAg
U3RhcnRpbmcgG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sbWzBtLi4uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltUmVndWxhciBiYWNrZ3JvdW5kIHByb2dyYW0g
cHJvY2Vzc2luZyBkYWVtb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bUQtQnVzIFN5c3RlbSBNZXNzYWdlIEJ1cxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7
MTszOW1OZXR3b3JrIE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1S
ZW1vdmUgU3RhbGUgT25saeKApnQ0IE1ldGFkYXRhIENoZWNrIFNuYXBzaG90cxtbMG0uLi4NCiAg
ICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUluaXRpYWxpemUgaGFyZHdhcmUgbW9uaXRvcmluZyBz
ZW5zb3JzG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltQXV0aG9yaXphdGlvbiBN
YW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcg
U2VydmljZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVVzZXIgTG9naW4gTWFu
YWdlbWVudBtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bURpc2sgTWFuYWdlchtb
MG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVdQQSBzdXBwbGljYW50G1swbS4uLg0K
WxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJl
IG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1sw
OzE7MzltU3lzdGVtIExvZ2dpbmcgU2VydmljZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZp
bmlzaGVkIBtbMDsxOzM5bVJlbW92ZSBTdGFsZSBPbmxp4oCmZXh0NCBNZXRhZGF0YSBDaGVjayBT
bmFwc2hvdHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUF2YWhp
IG1ETlMvRE5TLVNEIFN0YWNrG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7
MTszOW1XUEEgc3VwcGxpY2FudBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1sw
OzE7MzltQXV0aG9yaXphdGlvbiBNYW5hZ2VyG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsx
OzM5bU1vZGVtIE1hbmFnZXIbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1sw
OzE7MzltVXNlciBMb2dpbiBNYW5hZ2VtZW50G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3Rh
cnRlZCAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBS
ZWFjaGVkIHRhcmdldCAbWzA7MTszOW1OZXR3b3JrG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtb
MDsxOzM5bU5ldHdvcmsgTWFuYWdlciBXYWl0IE9ubGluZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0
aW5nIBtbMDsxOzM5bVRoZSBBcGFjaGUgSFRUUCBTZXJ2ZXIbWzBtLi4uDQogICAgICAgICBTdGFy
dGluZyAbWzA7MTszOW1PcGVuQlNEIFNlY3VyZSBTaGVsbCBzZXJ2ZXIbWzBtLi4uDQogICAgICAg
ICBTdGFydGluZyAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uLi4NCiAgICAgICAg
IFN0YXJ0aW5nIBtbMDsxOzM5bUhvc3RuYW1lIFNlcnZpY2UbWzBtLi4uDQpbG1swOzMybSAgT0sg
IBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVBlcm1pdCBVc2VyIFNlc3Npb25zG1swbS4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUxpZ2h0IERpc3BsYXkgTWFuYWdlchtbMG0uLi4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhvbGQgdW50aWwgYm9vdCBwcm9jZXNzIGZpbmlzaGVzIHVw
G1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU1vZGVtIE1hbmFn
ZXIbWzBtLg0KLS0tLS0tLS0tLS0tIE1vbiBTZXAgMTMgMTI6NTE6MDcgSVNUIDIwMjEgLS0tLS0t
LS0tLS0tCi9kZXYvbnZtZTNuMXAyOiBjbGVhbiwgMjEwODEyLzE1MjU5MjAgZmlsZXMsIDE1NDgz
NjYvNjEwMzU1MiBibG9ja3MNClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7Mzlt
VGVsbCBQbHltb3V0aCBUbyBXcml0ZSBPdXQgUnVudGltZSBEYXRhG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gRm91bmQgZGV2aWNlIBtbMDsxOzM5bS9kZXYvdHR5QU1BMBtbMG0uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bUZsdXNoIEpvdXJuYWwgdG8gUGVyc2lzdGVu
dCBTdG9yYWdlG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGls
ZSBGaWxlcyBhbmQgRGlyZWN0b3JpZXMbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlz
aGVkIBtbMDsxOzM5bUxvYWQvU2F2ZSBSYW5kb20gU2VlZBtbMG0uDQpbG1swOzMybSAgT0sgIBtb
MG1dIEZpbmlzaGVkIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGlsZSBGaWxlcyBhbmQgRGlyZWN0b3Jp
ZXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTmV0d29yayBUaW1lIFN5bmNocm9u
aXphdGlvbhtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVVwZGF0ZSBVVE1QIGFi
b3V0IFN5c3RlbSBCb290L1NodXRkb3duG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5p
c2hlZCAbWzA7MTszOW1VcGRhdGUgVVRNUCBhYm91dCBTeXN0ZW0gQm9vdC9TaHV0ZG93bhtbMG0u
DQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bUxvYWQgQXBwQXJtb3IgcHJv
ZmlsZXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltUmFpc2UgbmV0d29yayBpbnRl
cmZhY2VzG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1SYWlz
ZSBuZXR3b3JrIGludGVyZmFjZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtb
MDsxOzM5bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gSW5pdGlhbGl6YXRpb24bWzBtLg0K
WxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IENsZWFudXAgb2YgVGVt
cG9yYXJ5IERpcmVjdG9yaWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJn
ZXQgG1swOzE7MzltU3lzdGVtIFRpbWUgU2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVh
Y2hlZCB0YXJnZXQgG1swOzE7MzltU3lzdGVtIFRpbWUgU3luY2hyb25pemVkG1swbS4NClsbWzA7
MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1UcmlnZ2VyIGFuYWNyb24gZXZlcnkgaG91
chtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IGRv
d25sb2FkIGFjdGl2aXRpZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bURhaWx5IGFwdCB1cGdyYWRlIGFuZCBjbGVhbiBhY3Rpdml0aWVzG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1QZXJpb2RpYyBleHQ0IE9ubGnigKZhdGEgQ2hl
Y2sgZm9yIEFsbCBGaWxlc3lzdGVtcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltRGlzY2FyZCB1bnVzZWQgYmxvY2tzIG9uY2UgYSB3ZWVrG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSByb3RhdGlvbiBvZiBsb2cgZmlsZXMb
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IG1hbi1kYiBy
ZWdlbmVyYXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7
MTszOW1UaW1lcnMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7
MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sgQWN0aXZhdGlvbiBTb2NrZXQbWzBtLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7MzltRC1CdXMgU3lzdGVtIE1lc3NhZ2Ug
QnVzIFNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsx
OzM5bVNvY2tldHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7
MTszOW1CYXNpYyBTeXN0ZW0bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bVJ1biBhbmFjcm9uIGpvYnMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltQXZh
aGkgbUROUy9ETlMtU0QgU3RhY2sbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltUmVndWxhciBiYWNrZ3JvdW5kIHByb2dyYW0gcHJvY2Vzc2luZyBkYWVtb24bWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUQtQnVzIFN5c3RlbSBNZXNz
YWdlIEJ1cxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIb
WzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1SZW1vdmUgU3RhbGUgT25saeKApnQ0
IE1ldGFkYXRhIENoZWNrIFNuYXBzaG90cxtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsx
OzM5bUluaXRpYWxpemUgaGFyZHdhcmUgbW9uaXRvcmluZyBzZW5zb3JzG1swbS4uLg0KICAgICAg
ICAgU3RhcnRpbmcgG1swOzE7MzltQXV0aG9yaXphdGlvbiBNYW5hZ2VyG1swbS4uLg0KICAgICAg
ICAgU3RhcnRpbmcgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcgU2VydmljZRtbMG0uLi4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVVzZXIgTG9naW4gTWFuYWdlbWVudBtbMG0uLi4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bURpc2sgTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0
aW5nIBtbMDsxOzM5bVdQQSBzdXBwbGljYW50G1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBG
aW5pc2hlZCAbWzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtb
MG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcg
U2VydmljZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJlbW92
ZSBTdGFsZSBPbmxp4oCmZXh0NCBNZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1sw
bS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1XUEEgc3VwcGxpY2FudBtb
MG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltQXV0aG9yaXphdGlvbiBN
YW5hZ2VyG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU1vZGVtIE1hbmFnZXIbWzBt
Li4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltVXNlciBMb2dpbiBNYW5h
Z2VtZW50G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1OZXR3b3Jr
IE1hbmFnZXIbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTsz
OW1OZXR3b3JrG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU5ldHdvcmsgTWFuYWdl
ciBXYWl0IE9ubGluZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVRoZSBBcGFj
aGUgSFRUUCBTZXJ2ZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1PcGVuQlNE
IFNlY3VyZSBTaGVsbCBzZXJ2ZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Q
ZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhv
c3RuYW1lIFNlcnZpY2UbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsx
OzM5bVBlcm1pdCBVc2VyIFNlc3Npb25zG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bUxpZ2h0IERpc3BsYXkgTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bUhvbGQgdW50aWwgYm9vdCBwcm9jZXNzIGZpbmlzaGVzIHVwG1swbS4uLg0KWxtbMDszMm0gIE9L
ICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU1vZGVtIE1hbmFnZXIbWzBtLg0KLS0tLS0tLS0tLS0t
IE1vbiBTZXAgMTMgMTM6MDg6MTggSVNUIDIwMjEgLS0tLS0tLS0tLS0tCi9kZXYvbnZtZTNuMXAy
OiBjbGVhbiwgMjEwODE2LzE1MjU5MjAgZmlsZXMsIDE1NDgzNzAvNjEwMzU1MiBibG9ja3MNClsb
WzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltVGVsbCBQbHltb3V0aCBUbyBXcml0
ZSBPdXQgUnVudGltZSBEYXRhG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRm91bmQgZGV2aWNl
IBtbMDsxOzM5bS9kZXYvdHR5QU1BMBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVk
IBtbMDsxOzM5bUZsdXNoIEpvdXJuYWwgdG8gUGVyc2lzdGVudCBTdG9yYWdlG1swbS4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGlsZSBGaWxlcyBhbmQgRGlyZWN0b3Jp
ZXMbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bUNyZWF0ZSBW
b2xhdGlsZSBGaWxlcyBhbmQgRGlyZWN0b3JpZXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1sw
OzE7MzltTmV0d29yayBUaW1lIFN5bmNocm9uaXphdGlvbhtbMG0uLi4NCiAgICAgICAgIFN0YXJ0
aW5nIBtbMDsxOzM5bVVwZGF0ZSBVVE1QIGFib3V0IFN5c3RlbSBCb290L1NodXRkb3duG1swbS4u
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Mb2FkL1NhdmUgUmFuZG9t
IFNlZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1VcGRhdGUg
VVRNUCBhYm91dCBTeXN0ZW0gQm9vdC9TaHV0ZG93bhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IEZpbmlzaGVkIBtbMDsxOzM5bUxvYWQgQXBwQXJtb3IgcHJvZmlsZXMbWzBtLg0KICAgICAgICAg
U3RhcnRpbmcgG1swOzE7MzltUmFpc2UgbmV0d29yayBpbnRlcmZhY2VzG1swbS4uLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1SYWlzZSBuZXR3b3JrIGludGVyZmFjZXMb
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU5ldHdvcmsgVGltZSBT
eW5jaHJvbml6YXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAb
WzA7MTszOW1TeXN0ZW0gSW5pdGlhbGl6YXRpb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBT
dGFydGVkIBtbMDsxOzM5bURhaWx5IENsZWFudXAgb2YgVGVtcG9yYXJ5IERpcmVjdG9yaWVzG1sw
bS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltU3lzdGVtIFRp
bWUgU2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7Mzlt
U3lzdGVtIFRpbWUgU3luY2hyb25pemVkG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRl
ZCAbWzA7MTszOW1UcmlnZ2VyIGFuYWNyb24gZXZlcnkgaG91chtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IGRvd25sb2FkIGFjdGl2aXRpZXMbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IGFwdCB1cGdyYWRl
IGFuZCBjbGVhbiBhY3Rpdml0aWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAb
WzA7MTszOW1QZXJpb2RpYyBleHQ0IE9ubGnigKZhdGEgQ2hlY2sgZm9yIEFsbCBGaWxlc3lzdGVt
cxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGlzY2FyZCB1bnVz
ZWQgYmxvY2tzIG9uY2UgYSB3ZWVrG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAb
WzA7MTszOW1EYWlseSByb3RhdGlvbiBvZiBsb2cgZmlsZXMbWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IG1hbi1kYiByZWdlbmVyYXRpb24bWzBtLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1UaW1lcnMbWzBtLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0Qg
U3RhY2sgQWN0aXZhdGlvbiBTb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5p
bmcgb24gG1swOzE7MzltRC1CdXMgU3lzdGVtIE1lc3NhZ2UgQnVzIFNvY2tldBtbMG0uDQpbG1sw
OzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVNvY2tldHMbWzBtLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1CYXNpYyBTeXN0ZW0bWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVJ1biBhbmFjcm9uIGpvYnMb
WzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sb
WzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltUmVndWxhciBiYWNr
Z3JvdW5kIHByb2dyYW0gcHJvY2Vzc2luZyBkYWVtb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBTdGFydGVkIBtbMDsxOzM5bUQtQnVzIFN5c3RlbSBNZXNzYWdlIEJ1cxtbMG0uDQogICAgICAg
ICBTdGFydGluZyAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFy
dGluZyAbWzA7MTszOW1SZW1vdmUgU3RhbGUgT25saeKApnQ0IE1ldGFkYXRhIENoZWNrIFNuYXBz
aG90cxtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUluaXRpYWxpemUgaGFyZHdh
cmUgbW9uaXRvcmluZyBzZW5zb3JzG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7Mzlt
QXV0aG9yaXphdGlvbiBNYW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7Mzlt
U3lzdGVtIExvZ2dpbmcgU2VydmljZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bVVzZXIgTG9naW4gTWFuYWdlbWVudBtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bURpc2sgTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVdQQSBzdXBw
bGljYW50G1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Jbml0
aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uDQpbG1swOzMybSAgT0sgIBtb
MG1dIFN0YXJ0ZWQgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcgU2VydmljZRtbMG0uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJlbW92ZSBTdGFsZSBPbmxp4oCmZXh0NCBN
ZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVk
IBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4NClsbWzA7MzJtICBPSyAgG1sw
bV0gU3RhcnRlZCAbWzA7MTszOW1XUEEgc3VwcGxpY2FudBtbMG0uDQpbG1swOzMybSAgT0sgIBtb
MG1dIFN0YXJ0ZWQgG1swOzE7MzltQXV0aG9yaXphdGlvbiBNYW5hZ2VyG1swbS4NCiAgICAgICAg
IFN0YXJ0aW5nIBtbMDsxOzM5bU1vZGVtIE1hbmFnZXIbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtb
MG1dIFN0YXJ0ZWQgG1swOzE7MzltVXNlciBMb2dpbiBNYW5hZ2VtZW50G1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIbWzBtLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1OZXR3b3JrG1swbS4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU5ldHdvcmsgTWFuYWdlciBXYWl0IE9ubGluZRtbMG0uLi4N
CiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVRoZSBBcGFjaGUgSFRUUCBTZXJ2ZXIbWzBtLi4u
DQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1PcGVuQlNEIFNlY3VyZSBTaGVsbCBzZXJ2ZXIb
WzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtb
MG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhvc3RuYW1lIFNlcnZpY2UbWzBtLi4u
DQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVBlcm1pdCBVc2VyIFNlc3Np
b25zG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUxpZ2h0IERpc3BsYXkgTWFuYWdl
chtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhvbGQgdW50aWwgYm9vdCBwcm9j
ZXNzIGZpbmlzaGVzIHVwG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bU1vZGVtIE1hbmFnZXIbWzBtLg0KLS0tLS0tLS0tLS0tIE1vbiBTZXAgMTMgMTQ6MDI6MjAg
SVNUIDIwMjEgLS0tLS0tLS0tLS0tCi9kZXYvbnZtZTNuMXAyOiBjbGVhbiwgMjEwODE2LzE1MjU5
MjAgZmlsZXMsIDE1NDgzNzMvNjEwMzU1MiBibG9ja3MNClsbWzA7MzJtICBPSyAgG1swbV0gRmlu
aXNoZWQgG1swOzE7MzltVGVsbCBQbHltb3V0aCBUbyBXcml0ZSBPdXQgUnVudGltZSBEYXRhG1sw
bS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltRmx1c2ggSm91cm5hbCB0
byBQZXJzaXN0ZW50IFN0b3JhZ2UbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltQ3Jl
YXRlIFZvbGF0aWxlIEZpbGVzIGFuZCBEaXJlY3RvcmllcxtbMG0uLi4NClsbWzA7MzJtICBPSyAg
G1swbV0gRmluaXNoZWQgG1swOzE7MzltQ3JlYXRlIFZvbGF0aWxlIEZpbGVzIGFuZCBEaXJlY3Rv
cmllcxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1OZXR3b3JrIFRpbWUgU3luY2hy
b25pemF0aW9uG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltVXBkYXRlIFVUTVAg
YWJvdXQgU3lzdGVtIEJvb3QvU2h1dGRvd24bWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZp
bmlzaGVkIBtbMDsxOzM5bUxvYWQvU2F2ZSBSYW5kb20gU2VlZBtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVVwZGF0ZSBVVE1QIGFib3V0IFN5c3RlbSBCb290L1No
dXRkb3duG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltTG9hZCBB
cHBBcm1vciBwcm9maWxlcxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1SYWlzZSBu
ZXR3b3JrIGludGVyZmFjZXMbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtb
MDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNlcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IFN0YXJ0ZWQgG1swOzE7MzltTmV0d29yayBUaW1lIFN5bmNocm9uaXphdGlvbhtbMG0uDQpbG1sw
OzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVN5c3RlbSBJbml0aWFsaXph
dGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgQ2xl
YW51cCBvZiBUZW1wb3JhcnkgRGlyZWN0b3JpZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBS
ZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTZXQbWzBtLg0KWxtbMDszMm0gIE9L
ICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTeW5jaHJvbml6ZWQb
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVRyaWdnZXIgYW5hY3Jv
biBldmVyeSBob3VyG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1E
YWlseSBhcHQgZG93bmxvYWQgYWN0aXZpdGllcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0
YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IHVwZ3JhZGUgYW5kIGNsZWFuIGFjdGl2aXRpZXMbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVBlcmlvZGljIGV4dDQgT25s
aeKApmF0YSBDaGVjayBmb3IgQWxsIEZpbGVzeXN0ZW1zG1swbS4NClsbWzA7MzJtICBPSyAgG1sw
bV0gU3RhcnRlZCAbWzA7MTszOW1EaXNjYXJkIHVudXNlZCBibG9ja3Mgb25jZSBhIHdlZWsbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IHJvdGF0aW9uIG9m
IGxvZyBmaWxlcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFp
bHkgbWFuLWRiIHJlZ2VuZXJhdGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQg
dGFyZ2V0IBtbMDsxOzM5bVRpbWVycxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3Rlbmlu
ZyBvbiAbWzA7MTszOW1BdmFoaSBtRE5TL0ROUy1TRCBTdGFjayBBY3RpdmF0aW9uIFNvY2tldBtb
MG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1ELUJ1cyBTeXN0
ZW0gTWVzc2FnZSBCdXMgU29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0
YXJnZXQgG1swOzE7MzltU29ja2V0cxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQg
dGFyZ2V0IBtbMDsxOzM5bUJhc2ljIFN5c3RlbRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0
YXJ0ZWQgG1swOzE7MzltUnVuIGFuYWNyb24gam9icxtbMG0uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1BdmFoaSBtRE5TL0ROUy1TRCBTdGFjaxtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1sw
bV0gU3RhcnRlZCAbWzA7MTszOW1SZWd1bGFyIGJhY2tncm91bmQgcHJvZ3JhbSBwcm9jZXNzaW5n
IGRhZW1vbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRC1CdXMg
U3lzdGVtIE1lc3NhZ2UgQnVzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU5ldHdv
cmsgTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVJlbW92ZSBTdGFs
ZSBPbmxp4oCmdDQgTWV0YWRhdGEgQ2hlY2sgU25hcHNob3RzG1swbS4uLg0KICAgICAgICAgU3Rh
cnRpbmcgG1swOzE7MzltSW5pdGlhbGl6ZSBoYXJkd2FyZSBtb25pdG9yaW5nIHNlbnNvcnMbWzBt
Li4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1BdXRob3JpemF0aW9uIE1hbmFnZXIbWzBt
Li4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1TeXN0ZW0gTG9nZ2luZyBTZXJ2aWNlG1sw
bS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltVXNlciBMb2dpbiBNYW5hZ2VtZW50G1sw
bS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltRGlzayBNYW5hZ2VyG1swbS4uLg0KICAg
ICAgICAgU3RhcnRpbmcgG1swOzE7MzltV1BBIHN1cHBsaWNhbnQbWzBtLi4uDQpbG1swOzMybSAg
T0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bUluaXRpYWxpemUgaGFyZHdhcmUgbW9uaXRvcmlu
ZyBzZW5zb3JzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1TeXN0
ZW0gTG9nZ2luZyBTZXJ2aWNlG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1sw
OzE7MzltUmVtb3ZlIFN0YWxlIE9ubGnigKZleHQ0IE1ldGFkYXRhIENoZWNrIFNuYXBzaG90cxtb
MG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltQXZhaGkgbUROUy9ETlMt
U0QgU3RhY2sbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVdQQSBz
dXBwbGljYW50G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1BdXRo
b3JpemF0aW9uIE1hbmFnZXIbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTW9kZW0g
TWFuYWdlchtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1OZXR3
b3JrIE1hbmFnZXIbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7
MTszOW1OZXR3b3JrG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU5ldHdvcmsgTWFu
YWdlciBXYWl0IE9ubGluZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVRoZSBB
cGFjaGUgSFRUUCBTZXJ2ZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1PcGVu
QlNEIFNlY3VyZSBTaGVsbCBzZXJ2ZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTsz
OW1QZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bUhvc3RuYW1lIFNlcnZpY2UbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1sw
OzE7MzltVXNlciBMb2dpbiBNYW5hZ2VtZW50G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmlu
aXNoZWQgG1swOzE7MzltUGVybWl0IFVzZXIgU2Vzc2lvbnMbWzBtLg0KICAgICAgICAgU3RhcnRp
bmcgG1swOzE7MzltTGlnaHQgRGlzcGxheSBNYW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRp
bmcgG1swOzE7MzltSG9sZCB1bnRpbCBib290IHByb2Nlc3MgZmluaXNoZXMgdXAbWzBtLi4uDQpb
G1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltTW9kZW0gTWFuYWdlchtbMG0uDQot
LS0tLS0tLS0tLS0gTW9uIFNlcCAxMyAxNDozMTo1NCBJU1QgMjAyMSAtLS0tLS0tLS0tLS0KL2Rl
di9udm1lM24xcDI6IGNsZWFuLCAyMTA4MTYvMTUyNTkyMCBmaWxlcywgMTU1MDExMy82MTAzNTUy
IGJsb2Nrcw0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1UZWxsIFBseW1v
dXRoIFRvIFdyaXRlIE91dCBSdW50aW1lIERhdGEbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBG
aW5pc2hlZCAbWzA7MTszOW1DcmVhdGUgVm9sYXRpbGUgRmlsZXMgYW5kIERpcmVjdG9yaWVzG1sw
bS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRp
b24bWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1VcGRhdGUgVVRNUCBhYm91dCBT
eXN0ZW0gQm9vdC9TaHV0ZG93bhtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQg
G1swOzE7MzltVXBkYXRlIFVUTVAgYWJvdXQgU3lzdGVtIEJvb3QvU2h1dGRvd24bWzBtLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Mb2FkIEFwcEFybW9yIHByb2ZpbGVz
G1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNl
cxtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltUmFpc2UgbmV0
d29yayBpbnRlcmZhY2VzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTsz
OW1OZXR3b3JrIFRpbWUgU3luY2hyb25pemF0aW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0g
UmVhY2hlZCB0YXJnZXQgG1swOzE7MzltU3lzdGVtIEluaXRpYWxpemF0aW9uG1swbS4NClsbWzA7
MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSBDbGVhbnVwIG9mIFRlbXBvcmFy
eSBEaXJlY3RvcmllcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtb
MDsxOzM5bVN5c3RlbSBUaW1lIFNldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQg
dGFyZ2V0IBtbMDsxOzM5bVN5c3RlbSBUaW1lIFN5bmNocm9uaXplZBtbMG0uDQpbG1swOzMybSAg
T0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltVHJpZ2dlciBhbmFjcm9uIGV2ZXJ5IGhvdXIbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IGFwdCBkb3dubG9h
ZCBhY3Rpdml0aWVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1E
YWlseSBhcHQgdXBncmFkZSBhbmQgY2xlYW4gYWN0aXZpdGllcxtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltUGVyaW9kaWMgZXh0NCBPbmxp4oCmYXRhIENoZWNrIGZv
ciBBbGwgRmlsZXN5c3RlbXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bURpc2NhcmQgdW51c2VkIGJsb2NrcyBvbmNlIGEgd2VlaxtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgcm90YXRpb24gb2YgbG9nIGZpbGVzG1swbS4N
ClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSBtYW4tZGIgcmVnZW5l
cmF0aW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7Mzlt
VGltZXJzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUF2
YWhpIG1ETlMvRE5TLVNEIFN0YWNrIEFjdGl2YXRpb24gU29ja2V0G1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUQtQnVzIFN5c3RlbSBNZXNzYWdlIEJ1cyBT
b2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1T
b2NrZXRzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7Mzlt
QmFzaWMgU3lzdGVtG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1S
dW4gYW5hY3JvbiBqb2JzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUF2YWhpIG1E
TlMvRE5TLVNEIFN0YWNrG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bVJlZ3VsYXIgYmFja2dyb3VuZCBwcm9ncmFtIHByb2Nlc3NpbmcgZGFlbW9uG1swbS4NClsb
WzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1ELUJ1cyBTeXN0ZW0gTWVzc2FnZSBC
dXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTmV0d29yayBNYW5hZ2VyG1swbS4u
Lg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltUmVtb3ZlIFN0YWxlIE9ubGnigKZ0NCBNZXRh
ZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1J
bml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uLi4NCiAgICAgICAgIFN0
YXJ0aW5nIBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0
YXJ0aW5nIBtbMDsxOzM5bVN5c3RlbSBMb2dnaW5nIFNlcnZpY2UbWzBtLi4uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1Vc2VyIExvZ2luIE1hbmFnZW1lbnQbWzBtLi4uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1EaXNrIE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1XUEEgc3VwcGxpY2FudBtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNo
ZWQgG1swOzE7MzltSW5pdGlhbGl6ZSBoYXJkd2FyZSBtb25pdG9yaW5nIHNlbnNvcnMbWzBtLg0K
WxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVN5c3RlbSBMb2dnaW5nIFNlcnZp
Y2UbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1SZW1vdmUgU3Rh
bGUgT25saeKApmV4dDQgTWV0YWRhdGEgQ2hlY2sgU25hcHNob3RzG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1BdmFoaSBtRE5TL0ROUy1TRCBTdGFjaxtbMG0uDQpb
G1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltV1BBIHN1cHBsaWNhbnQbWzBtLg0K
WxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdl
chtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Nb2RlbSBNYW5hZ2VyG1swbS4uLg0K
WxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU5ldHdvcmsgTWFuYWdlchtbMG0u
DQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bU5ldHdvcmsbWzBt
Lg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTmV0d29yayBNYW5hZ2VyIFdhaXQgT25saW5l
G1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltVGhlIEFwYWNoZSBIVFRQIFNlcnZl
chtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bU9wZW5CU0QgU2VjdXJlIFNoZWxs
IHNlcnZlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVBlcm1pdCBVc2VyIFNl
c3Npb25zG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVVzZXIg
TG9naW4gTWFuYWdlbWVudBtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Ib3N0bmFt
ZSBTZXJ2aWNlG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1Q
ZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1MaWdo
dCBEaXNwbGF5IE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Ib2xk
IHVudGlsIGJvb3QgcHJvY2VzcyBmaW5pc2hlcyB1cBtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1sw
bV0gU3RhcnRlZCAbWzA7MTszOW1Nb2RlbSBNYW5hZ2VyG1swbS4NCi0tLS0tLS0tLS0tLSBNb24g
U2VwIDEzIDE1OjQwOjQwIElTVCAyMDIxIC0tLS0tLS0tLS0tLQovZGV2L252bWUzbjFwMjogY2xl
YW4sIDIxNDY3NS8xNTI1OTIwIGZpbGVzLCAxNjUzMTM0LzYxMDM1NTIgYmxvY2tzDQpbG1swOzMy
bSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVRlbGwgUGx5bW91dGggVG8gV3JpdGUgT3V0
IFJ1bnRpbWUgRGF0YRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5
bUZsdXNoIEpvdXJuYWwgdG8gUGVyc2lzdGVudCBTdG9yYWdlG1swbS4NCiAgICAgICAgIFN0YXJ0
aW5nIBtbMDsxOzM5bUNyZWF0ZSBWb2xhdGlsZSBGaWxlcyBhbmQgRGlyZWN0b3JpZXMbWzBtLi4u
DQpbG1swOzMybSAgT0sgIBtbMG1dIEZvdW5kIGRldmljZSAbWzA7MTszOW0vZGV2L3R0eUFNQTAb
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1DcmVhdGUgVm9sYXRp
bGUgRmlsZXMgYW5kIERpcmVjdG9yaWVzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bU5ldHdvcmsgVGltZSBTeW5jaHJvbml6YXRpb24bWzBtLi4uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1VcGRhdGUgVVRNUCBhYm91dCBTeXN0ZW0gQm9vdC9TaHV0ZG93bhtbMG0uLi4NClsb
WzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltVXBkYXRlIFVUTVAgYWJvdXQgU3lz
dGVtIEJvb3QvU2h1dGRvd24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7
MTszOW1Mb2FkL1NhdmUgUmFuZG9tIFNlZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5p
c2hlZCAbWzA7MTszOW1Mb2FkIEFwcEFybW9yIHByb2ZpbGVzG1swbS4NCiAgICAgICAgIFN0YXJ0
aW5nIBtbMDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNlcxtbMG0uLi4NClsbWzA7MzJtICBP
SyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltUmFpc2UgbmV0d29yayBpbnRlcmZhY2VzG1swbS4N
ClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1OZXR3b3JrIFRpbWUgU3luY2hy
b25pemF0aW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7
MzltU3lzdGVtIEluaXRpYWxpemF0aW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRl
ZCAbWzA7MTszOW1EYWlseSBDbGVhbnVwIG9mIFRlbXBvcmFyeSBEaXJlY3RvcmllcxtbMG0uDQpb
G1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVN5c3RlbSBUaW1lIFNl
dBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVN5c3Rl
bSBUaW1lIFN5bmNocm9uaXplZBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1sw
OzE7MzltVHJpZ2dlciBhbmFjcm9uIGV2ZXJ5IGhvdXIbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IGFwdCBkb3dubG9hZCBhY3Rpdml0aWVzG1swbS4NClsb
WzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSBhcHQgdXBncmFkZSBhbmQg
Y2xlYW4gYWN0aXZpdGllcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7
MzltUGVyaW9kaWMgZXh0NCBPbmxp4oCmYXRhIENoZWNrIGZvciBBbGwgRmlsZXN5c3RlbXMbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURpc2NhcmQgdW51c2VkIGJs
b2NrcyBvbmNlIGEgd2VlaxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7
MzltRGFpbHkgcm90YXRpb24gb2YgbG9nIGZpbGVzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0g
U3RhcnRlZCAbWzA7MTszOW1EYWlseSBtYW4tZGIgcmVnZW5lcmF0aW9uG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltVGltZXJzG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNr
IEFjdGl2YXRpb24gU29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9u
IBtbMDsxOzM5bUQtQnVzIFN5c3RlbSBNZXNzYWdlIEJ1cyBTb2NrZXQbWzBtLg0KWxtbMDszMm0g
IE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7MzltTGlidmlydCBsb2NhbCBzb2NrZXQbWzBt
Lg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7MzltTGlidmlydCBhZG1p
biBzb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7Mzlt
TGlidmlydCBsb2NhbCByZWFkLW9ubHkgc29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0g
TGlzdGVuaW5nIG9uIBtbMDsxOzM5bVZpcnR1YWwgbWFjaGluZSBsb2NrIG1hbmFnZXIgc29ja2V0
G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bVZpcnR1YWwg
bWFjaGluZSBsb2NrIG1hbmFnZXIgYWRtaW4gc29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1sw
bV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bVZpcnR1YWwgbWFjaGluZSBsb2cgbWFuYWdlciBzb2Nr
ZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBMaXN0ZW5pbmcgb24gG1swOzE7MzltVmlydHVh
bCBtYWNoaW5lIGxvZyBtYW5hZ2VyIHNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJl
YWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVNvY2tldHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBS
ZWFjaGVkIHRhcmdldCAbWzA7MTszOW1CYXNpYyBTeXN0ZW0bWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBTdGFydGVkIBtbMDsxOzM5bVJ1biBhbmFjcm9uIGpvYnMbWzBtLg0KICAgICAgICAgU3Rh
cnRpbmcgG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sbWzBtLi4uDQpbG1swOzMybSAg
T0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltUmVndWxhciBiYWNrZ3JvdW5kIHByb2dyYW0gcHJv
Y2Vzc2luZyBkYWVtb24bWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5
bUQtQnVzIFN5c3RlbSBNZXNzYWdlIEJ1cxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTsz
OW1OZXR3b3JrIE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1SZW1v
dmUgU3RhbGUgT25saeKApnQ0IE1ldGFkYXRhIENoZWNrIFNuYXBzaG90cxtbMG0uLi4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUluaXRpYWxpemUgaGFyZHdhcmUgbW9uaXRvcmluZyBzZW5z
b3JzG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltQXV0aG9yaXphdGlvbiBNYW5h
Z2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcgU2Vy
dmljZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVVzZXIgTG9naW4gTWFuYWdl
bWVudBtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVZpcnR1YWwgTWFjaGluZSBh
4oCmdGFpbmVyIFJlZ2lzdHJhdGlvbiBTZXJ2aWNlG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcg
G1swOzE7MzltRGlzayBNYW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7Mzlt
V1BBIHN1cHBsaWNhbnQbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7
MzltU3lzdGVtIExvZ2dpbmcgU2VydmljZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlz
aGVkIBtbMDsxOzM5bUluaXRpYWxpemUgaGFyZHdhcmUgbW9uaXRvcmluZyBzZW5zb3JzG1swbS4N
ClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltUmVtb3ZlIFN0YWxlIE9ubGni
gKZleHQ0IE1ldGFkYXRhIENoZWNrIFNuYXBzaG90cxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IFN0YXJ0ZWQgG1swOzE7MzltQXZhaGkgbUROUy9ETlMtU0QgU3RhY2sbWzBtLg0KWxtbMDszMm0g
IE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVZpcnR1YWwgTWFjaGluZSBhbuKApm9udGFpbmVy
IFJlZ2lzdHJhdGlvbiBTZXJ2aWNlG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAb
WzA7MTszOW1XUEEgc3VwcGxpY2FudBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltQXV0aG9yaXphdGlvbiBNYW5hZ2VyG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtb
MDsxOzM5bU1vZGVtIE1hbmFnZXIbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQg
G1swOzE7MzltVXNlciBMb2dpbiBNYW5hZ2VtZW50G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0g
U3RhcnRlZCAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBt
XSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1OZXR3b3JrG1swbS4NCiAgICAgICAgIFN0YXJ0aW5n
IBtbMDsxOzM5bU5ldHdvcmsgTWFuYWdlciBXYWl0IE9ubGluZRtbMG0uLi4NCiAgICAgICAgIFN0
YXJ0aW5nIBtbMDsxOzM5bVRoZSBBcGFjaGUgSFRUUCBTZXJ2ZXIbWzBtLi4uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1WaXJ0dWFsaXphdGlvbiBkYWVtb24bWzBtLi4uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1PcGVuQlNEIFNlY3VyZSBTaGVsbCBzZXJ2ZXIbWzBtLi4uDQogICAg
ICAgICBTdGFydGluZyAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uLi4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhvc3RuYW1lIFNlcnZpY2UbWzBtLi4uDQpbG1swOzMybSAg
T0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVBlcm1pdCBVc2VyIFNlc3Npb25zG1swbS4NCiAg
ICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUxpZ2h0IERpc3BsYXkgTWFuYWdlchtbMG0uLi4NCiAg
ICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUhvbGQgdW50aWwgYm9vdCBwcm9jZXNzIGZpbmlzaGVz
IHVwG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bU1vZGVtIE1h
bmFnZXIbWzBtLg0KLS0tLS0tLS0tLS0tIE1vbiBTZXAgMTMgMTU6NTA6MzIgSVNUIDIwMjEgLS0t
LS0tLS0tLS0tCi9kZXYvbnZtZTNuMXAyOiBjbGVhbiwgMjE0Njc1LzE1MjU5MjAgZmlsZXMsIDE2
NTMxMzQvNjEwMzU1MiBibG9ja3MNClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7
MzltVGVsbCBQbHltb3V0aCBUbyBXcml0ZSBPdXQgUnVudGltZSBEYXRhG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gQWN0aXZhdGVkIHN3YXAgG1swOzE7MzltL2Rldi9kaXNrL2LigKYxLTJmYmEt
NGRjYy1hNjQzLTE1MGIyMzAwOGI0MxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQg
dGFyZ2V0IBtbMDsxOzM5bVN3YXAbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAb
WzA7MTszOW1Mb2FkL1NhdmUgUmFuZG9tIFNlZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBG
aW5pc2hlZCAbWzA7MTszOW1GbHVzaCBKb3VybmFsIHRvIFBlcnNpc3RlbnQgU3RvcmFnZRtbMG0u
DQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1DcmVhdGUgVm9sYXRpbGUgRmlsZXMgYW5kIERp
cmVjdG9yaWVzG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGb3VuZCBkZXZpY2UgG1swOzE7
MzltL2Rldi90dHlBTUEwG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7
MzltQ3JlYXRlIFZvbGF0aWxlIEZpbGVzIGFuZCBEaXJlY3RvcmllcxtbMG0uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1OZXR3b3JrIFRpbWUgU3luY2hyb25pemF0aW9uG1swbS4uLg0KICAg
ICAgICAgU3RhcnRpbmcgG1swOzE7MzltVXBkYXRlIFVUTVAgYWJvdXQgU3lzdGVtIEJvb3QvU2h1
dGRvd24bWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVVwZGF0
ZSBVVE1QIGFib3V0IFN5c3RlbSBCb290L1NodXRkb3duG1swbS4NClsbWzA7MzJtICBPSyAgG1sw
bV0gRmluaXNoZWQgG1swOzE7MzltTG9hZCBBcHBBcm1vciBwcm9maWxlcxtbMG0uDQogICAgICAg
ICBTdGFydGluZyAbWzA7MTszOW1SYWlzZSBuZXR3b3JrIGludGVyZmFjZXMbWzBtLi4uDQpbG1sw
OzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNl
cxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltTmV0d29yayBUaW1l
IFN5bmNocm9uaXphdGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0
IBtbMDsxOzM5bVN5c3RlbSBJbml0aWFsaXphdGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1d
IFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgQ2xlYW51cCBvZiBUZW1wb3JhcnkgRGlyZWN0b3JpZXMb
WzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0g
VGltZSBTZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTsz
OW1TeXN0ZW0gVGltZSBTeW5jaHJvbml6ZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFy
dGVkIBtbMDsxOzM5bVRyaWdnZXIgYW5hY3JvbiBldmVyeSBob3VyG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EYWlseSBhcHQgZG93bmxvYWQgYWN0aXZpdGllcxtb
MG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IHVwZ3Jh
ZGUgYW5kIGNsZWFuIGFjdGl2aXRpZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVk
IBtbMDsxOzM5bVBlcmlvZGljIGV4dDQgT25saeKApmF0YSBDaGVjayBmb3IgQWxsIEZpbGVzeXN0
ZW1zG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EaXNjYXJkIHVu
dXNlZCBibG9ja3Mgb25jZSBhIHdlZWsbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVk
IBtbMDsxOzM5bURhaWx5IHJvdGF0aW9uIG9mIGxvZyBmaWxlcxtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgbWFuLWRiIHJlZ2VuZXJhdGlvbhtbMG0uDQpb
G1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVRpbWVycxtbMG0uDQpb
G1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1BdmFoaSBtRE5TL0ROUy1T
RCBTdGFjayBBY3RpdmF0aW9uIFNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3Rl
bmluZyBvbiAbWzA7MTszOW1ELUJ1cyBTeXN0ZW0gTWVzc2FnZSBCdXMgU29ja2V0G1swbS4NClsb
WzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUxpYnZpcnQgbG9jYWwgc29j
a2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUxpYnZp
cnQgYWRtaW4gc29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtb
MDsxOzM5bUxpYnZpcnQgbG9jYWwgcmVhZC1vbmx5IHNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1WaXJ0dWFsIG1hY2hpbmUgbG9jayBtYW5hZ2Vy
IHNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1W
aXJ0dWFsIG1hY2hpbmUgbG9jayBtYW5hZ2VyIGFkbWluIHNvY2tldBtbMG0uDQpbG1swOzMybSAg
T0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1WaXJ0dWFsIG1hY2hpbmUgbG9nIG1hbmFn
ZXIgc29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5
bVZpcnR1YWwgbWFjaGluZSBsb2cgbWFuYWdlciBzb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1Tb2NrZXRzG1swbS4NClsbWzA7MzJtICBPSyAg
G1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltQmFzaWMgU3lzdGVtG1swbS4NClsbWzA7MzJt
ICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1SdW4gYW5hY3JvbiBqb2JzG1swbS4NCiAgICAg
ICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4uLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVJlZ3VsYXIgYmFja2dyb3VuZCBwcm9n
cmFtIHByb2Nlc3NpbmcgZGFlbW9uG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAb
WzA7MTszOW1ELUJ1cyBTeXN0ZW0gTWVzc2FnZSBCdXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcg
G1swOzE7MzltTmV0d29yayBNYW5hZ2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltUmVtb3ZlIFN0YWxlIE9ubGnigKZ0NCBNZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLi4u
DQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jp
bmcgc2Vuc29ycxtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUF1dGhvcml6YXRp
b24gTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVN5c3RlbSBMb2dn
aW5nIFNlcnZpY2UbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Vc2VyIExvZ2lu
IE1hbmFnZW1lbnQbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1WaXJ0dWFsIE1h
Y2hpbmUgYeKApnRhaW5lciBSZWdpc3RyYXRpb24gU2VydmljZRtbMG0uLi4NCiAgICAgICAgIFN0
YXJ0aW5nIBtbMDsxOzM5bURpc2sgTWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtb
MDsxOzM5bVdQQSBzdXBwbGljYW50G1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hl
ZCAbWzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uDQpb
G1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltU3lzdGVtIExvZ2dpbmcgU2Vydmlj
ZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJlbW92ZSBTdGFs
ZSBPbmxp4oCmZXh0NCBNZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLg0KWxtbMDszMm0gIE9L
ICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4NClsb
WzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1WaXJ0dWFsIE1hY2hpbmUgYW7igKZv
bnRhaW5lciBSZWdpc3RyYXRpb24gU2VydmljZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0
YXJ0ZWQgG1swOzE7MzltV1BBIHN1cHBsaWNhbnQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBT
dGFydGVkIBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdlchtbMG0uDQogICAgICAgICBTdGFy
dGluZyAbWzA7MTszOW1Nb2RlbSBNYW5hZ2VyG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBT
dGFydGVkIBtbMDsxOzM5bVVzZXIgTG9naW4gTWFuYWdlbWVudBtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltTmV0d29yayBNYW5hZ2VyG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gUmVhY2hlZCB0YXJnZXQgG1swOzE7MzltTmV0d29yaxtbMG0uDQogICAgICAgICBT
dGFydGluZyAbWzA7MTszOW1OZXR3b3JrIE1hbmFnZXIgV2FpdCBPbmxpbmUbWzBtLi4uDQogICAg
ICAgICBTdGFydGluZyAbWzA7MTszOW1UaGUgQXBhY2hlIEhUVFAgU2VydmVyG1swbS4uLg0KICAg
ICAgICAgU3RhcnRpbmcgG1swOzE7MzltVmlydHVhbGl6YXRpb24gZGFlbW9uG1swbS4uLg0KICAg
ICAgICAgU3RhcnRpbmcgG1swOzE7MzltT3BlbkJTRCBTZWN1cmUgU2hlbGwgc2VydmVyG1swbS4u
Lg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltUGVybWl0IFVzZXIgU2Vzc2lvbnMbWzBtLi4u
DQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Ib3N0bmFtZSBTZXJ2aWNlG1swbS4uLg0KWxtb
MDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtb
MG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1MaWdodCBEaXNwbGF5IE1hbmFnZXIbWzBt
Li4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Ib2xkIHVudGlsIGJvb3QgcHJvY2VzcyBm
aW5pc2hlcyB1cBtbMG0uLi4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1N
b2RlbSBNYW5hZ2VyG1swbS4NCi0tLS0tLS0tLS0tLSBUdWUgU2VwIDE0IDEwOjMwOjExIElTVCAy
MDIxIC0tLS0tLS0tLS0tLQovZGV2L252bWUzbjFwMjogY2xlYW4sIDIxNDcxOC8xNTI1OTIwIGZp
bGVzLCAxNjUzMzk2LzYxMDM1NTIgYmxvY2tzDQpbG1swOzMybSAgT0sgIBtbMG1dIEZvdW5kIGRl
dmljZSAbWzA7MTszOW0vZGV2L3R0eUFNQTAbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltQ3JlYXRlIFZvbGF0aWxlIEZpbGVzIGFuZCBEaXJlY3RvcmllcxtbMG0uLi4NClsbWzA7MzJt
ICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltTG9hZC9TYXZlIFJhbmRvbSBTZWVkG1swbS4N
ClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7MzltQ3JlYXRlIFZvbGF0aWxlIEZp
bGVzIGFuZCBEaXJlY3RvcmllcxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1OZXR3
b3JrIFRpbWUgU3luY2hyb25pemF0aW9uG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltVXBkYXRlIFVUTVAgYWJvdXQgU3lzdGVtIEJvb3QvU2h1dGRvd24bWzBtLi4uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVVwZGF0ZSBVVE1QIGFib3V0IFN5c3RlbSBC
b290L1NodXRkb3duG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gRmluaXNoZWQgG1swOzE7Mzlt
TG9hZCBBcHBBcm1vciBwcm9maWxlcxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1S
YWlzZSBuZXR3b3JrIGludGVyZmFjZXMbWzBtLi4uDQpbG1swOzMybSAgT0sgIBtbMG1dIEZpbmlz
aGVkIBtbMDsxOzM5bVJhaXNlIG5ldHdvcmsgaW50ZXJmYWNlcxtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltTmV0d29yayBUaW1lIFN5bmNocm9uaXphdGlvbhtbMG0u
DQpbG1swOzMybSAgT0sgIBtbMG1dIFJlYWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVN5c3RlbSBJbml0
aWFsaXphdGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFp
bHkgQ2xlYW51cCBvZiBUZW1wb3JhcnkgRGlyZWN0b3JpZXMbWzBtLg0KWxtbMDszMm0gIE9LICAb
WzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTZXQbWzBtLg0KWxtbMDsz
Mm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAbWzA7MTszOW1TeXN0ZW0gVGltZSBTeW5jaHJv
bml6ZWQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVRyaWdnZXIg
YW5hY3JvbiBldmVyeSBob3VyG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7
MTszOW1EYWlseSBhcHQgZG93bmxvYWQgYWN0aXZpdGllcxtbMG0uDQpbG1swOzMybSAgT0sgIBtb
MG1dIFN0YXJ0ZWQgG1swOzE7MzltRGFpbHkgYXB0IHVwZ3JhZGUgYW5kIGNsZWFuIGFjdGl2aXRp
ZXMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVBlcmlvZGljIGV4
dDQgT25saeKApmF0YSBDaGVjayBmb3IgQWxsIEZpbGVzeXN0ZW1zG1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1EaXNjYXJkIHVudXNlZCBibG9ja3Mgb25jZSBhIHdl
ZWsbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bURhaWx5IHJvdGF0
aW9uIG9mIGxvZyBmaWxlcxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7
MzltRGFpbHkgbWFuLWRiIHJlZ2VuZXJhdGlvbhtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFJl
YWNoZWQgdGFyZ2V0IBtbMDsxOzM5bVRpbWVycxtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExp
c3RlbmluZyBvbiAbWzA7MTszOW1BdmFoaSBtRE5TL0ROUy1TRCBTdGFjayBBY3RpdmF0aW9uIFNv
Y2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1ELUJ1
cyBTeXN0ZW0gTWVzc2FnZSBCdXMgU29ja2V0G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gTGlz
dGVuaW5nIG9uIBtbMDsxOzM5bUxpYnZpcnQgbG9jYWwgc29ja2V0G1swbS4NClsbWzA7MzJtICBP
SyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUxpYnZpcnQgYWRtaW4gc29ja2V0G1swbS4N
ClsbWzA7MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bUxpYnZpcnQgbG9jYWwg
cmVhZC1vbmx5IHNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAb
WzA7MTszOW1WaXJ0dWFsIG1hY2hpbmUgbG9jayBtYW5hZ2VyIHNvY2tldBtbMG0uDQpbG1swOzMy
bSAgT0sgIBtbMG1dIExpc3RlbmluZyBvbiAbWzA7MTszOW1WaXJ0dWFsIG1hY2hpbmUgbG9jayBt
YW5hZ2VyIGFkbWluIHNvY2tldBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIExpc3RlbmluZyBv
biAbWzA7MTszOW1WaXJ0dWFsIG1hY2hpbmUgbG9nIG1hbmFnZXIgc29ja2V0G1swbS4NClsbWzA7
MzJtICBPSyAgG1swbV0gTGlzdGVuaW5nIG9uIBtbMDsxOzM5bVZpcnR1YWwgbWFjaGluZSBsb2cg
bWFuYWdlciBzb2NrZXQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBSZWFjaGVkIHRhcmdldCAb
WzA7MTszOW1Tb2NrZXRzG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJnZXQg
G1swOzE7MzltQmFzaWMgU3lzdGVtG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAb
WzA7MTszOW1SdW4gYW5hY3JvbiBqb2JzG1swbS4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5
bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFy
dGVkIBtbMDsxOzM5bVJlZ3VsYXIgYmFja2dyb3VuZCBwcm9ncmFtIHByb2Nlc3NpbmcgZGFlbW9u
G1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1ELUJ1cyBTeXN0ZW0g
TWVzc2FnZSBCdXMbWzBtLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltTmV0d29yayBNYW5h
Z2VyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7MzltUmVtb3ZlIFN0YWxlIE9ubGni
gKZ0NCBNZXRhZGF0YSBDaGVjayBTbmFwc2hvdHMbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1Jbml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uLi4NCiAg
ICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bUF1dGhvcml6YXRpb24gTWFuYWdlchtbMG0uLi4NCiAg
ICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVN5c3RlbSBMb2dnaW5nIFNlcnZpY2UbWzBtLi4uDQog
ICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Vc2VyIExvZ2luIE1hbmFnZW1lbnQbWzBtLi4uDQog
ICAgICAgICBTdGFydGluZyAbWzA7MTszOW1WaXJ0dWFsIE1hY2hpbmUgYeKApnRhaW5lciBSZWdp
c3RyYXRpb24gU2VydmljZRtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bURpc2sg
TWFuYWdlchtbMG0uLi4NCiAgICAgICAgIFN0YXJ0aW5nIBtbMDsxOzM5bVdQQSBzdXBwbGljYW50
G1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVN5c3RlbSBMb2dn
aW5nIFNlcnZpY2UbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5pc2hlZCAbWzA7MTszOW1J
bml0aWFsaXplIGhhcmR3YXJlIG1vbml0b3Jpbmcgc2Vuc29ycxtbMG0uDQpbG1swOzMybSAgT0sg
IBtbMG1dIEZpbmlzaGVkIBtbMDsxOzM5bVJlbW92ZSBTdGFsZSBPbmxp4oCmZXh0NCBNZXRhZGF0
YSBDaGVjayBTbmFwc2hvdHMbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsx
OzM5bUF2YWhpIG1ETlMvRE5TLVNEIFN0YWNrG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gU3Rh
cnRlZCAbWzA7MTszOW1WaXJ0dWFsIE1hY2hpbmUgYW7igKZvbnRhaW5lciBSZWdpc3RyYXRpb24g
U2VydmljZRtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7MzltV1BBIHN1
cHBsaWNhbnQbWzBtLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bUF1dGhv
cml6YXRpb24gTWFuYWdlchtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1Nb2RlbSBN
YW5hZ2VyG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBTdGFydGVkIBtbMDsxOzM5bVVzZXIg
TG9naW4gTWFuYWdlbWVudBtbMG0uDQpbG1swOzMybSAgT0sgIBtbMG1dIFN0YXJ0ZWQgG1swOzE7
MzltTmV0d29yayBNYW5hZ2VyG1swbS4NClsbWzA7MzJtICBPSyAgG1swbV0gUmVhY2hlZCB0YXJn
ZXQgG1swOzE7MzltTmV0d29yaxtbMG0uDQogICAgICAgICBTdGFydGluZyAbWzA7MTszOW1OZXR3
b3JrIE1hbmFnZXIgV2FpdCBPbmxpbmUbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAbWzA7MTsz
OW1UaGUgQXBhY2hlIEhUVFAgU2VydmVyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltVmlydHVhbGl6YXRpb24gZGFlbW9uG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcgG1swOzE7
MzltT3BlbkJTRCBTZWN1cmUgU2hlbGwgc2VydmVyG1swbS4uLg0KICAgICAgICAgU3RhcnRpbmcg
G1swOzE7MzltUGVybWl0IFVzZXIgU2Vzc2lvbnMbWzBtLi4uDQogICAgICAgICBTdGFydGluZyAb
WzA7MTszOW1Ib3N0bmFtZSBTZXJ2aWNlG1swbS4uLg0KWxtbMDszMm0gIE9LICAbWzBtXSBGaW5p
c2hlZCAbWzA7MTszOW1QZXJtaXQgVXNlciBTZXNzaW9ucxtbMG0uDQogICAgICAgICBTdGFydGlu
ZyAbWzA7MTszOW1MaWdodCBEaXNwbGF5IE1hbmFnZXIbWzBtLi4uDQogICAgICAgICBTdGFydGlu
ZyAbWzA7MTszOW1Ib2xkIHVudGlsIGJvb3QgcHJvY2VzcyBmaW5pc2hlcyB1cBtbMG0uLi4NClsb
WzA7MzJtICBPSyAgG1swbV0gU3RhcnRlZCAbWzA7MTszOW1Nb2RlbSBNYW5hZ2VyG1swbS4NCg==
--000000000000f7287605cbee41cf
Content-Type: text/plain; charset="US-ASCII"; name="kern.txt"
Content-Disposition: attachment; filename="kern.txt"
Content-Transfer-Encoding: base64
Content-ID: <f_ktjnxjno1>
X-Attachment-Id: f_ktjnxjno1

U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEJvb3RpbmcgTGlu
dXggb24gcGh5c2ljYWwgQ1BVIDB4MDAwMDEyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIExpbnV4IHZlcnNpb24gNS4xMC4wLTgt
YXJtNjQgKGRlYmlhbi1rZXJuZWxAbGlzdHMuZGViaWFuLm9yZykgKGdjYy0xMCAoRGViaWFuIDEw
LjIuMS02KSAxMC4yLjEgMjAyMTAxMTAsIEdOVSBsZCAoR05VIEJpbnV0aWxzIGZvciBEZWJpYW4p
IDIuMzUuMikgIzEgU01QIERlYmlhbiA1LjEwLjQ2LTQgKDIwMjEtMDgtMDMpClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEVGSSB2Mi43MCBieSBBbWVy
aWNhbiBNZWdhdHJlbmRzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBlZmk6IEFDUEkgMi4wPTB4ZWU5MDAwMDAgU01CSU9TIDMuMD0weGYxYjNmZjk4IEVTUlQ9
MHhlOTRmOGY5OCBNT0t2YXI9MHhlOTRmMzAwMCBSTkc9MHhmNzY2ZTgxOCBNRU1SRVNFUlZFPTB4
ZTk2MWY4MTggClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBl
Zmk6IHNlZWRpbmcgZW50cm9weSBwb29sClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBzZWN1cmVib290OiBTZWN1cmUgYm9vdCBkaXNhYmxlZApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZXNydDogUmVzZXJ2aW5nIEVTUlQg
c3BhY2UgZnJvbSAweDAwMDAwMDAwZTk0ZjhmOTggdG8gMHgwMDAwMDAwMGU5NGY4ZmQwLgpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRWFybHkgdGFi
bGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2FibGVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAwMDBFRTkwMDAwMCAwMDAw
MjQgKHYwMiBBbXBlcmUpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDBFRThGMDAwMCAwMDAwQTQgKHYwMSBBbXBlcmUgQWx0
cmEgICAgMDAwMDAwMDAgQU1JICAwMTAwMDAxMykKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZBQ1AgMHgwMDAwMDAwMEVFOEQwMDAwIDAwMDExNCAo
djA2IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRFNEVCAweDAwMDAwMDAwRUU4
NjAwMDAgMDE5QTk1ICh2MDIgQW1wZXJlIEphZGUgICAgIDAwMDAwMDAxIElOVEwgMjAyMDA3MTcp
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBEQkcy
IDB4MDAwMDAwMDBFRThFMDAwMCAwMDAwNUMgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAg
SU5UTCAyMDE5MDUwOSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IEdURFQgMHgwMDAwMDAwMEVFOEMwMDAwIDAwMDExMCAodjAzIEFtcGVyZSBBbHRy
YSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1NEVCAweDAwMDAwMDAwRUU4QjAwMDAgMDAwMDJEICh2
MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIElOVEwgMjAxOTA1MDkpClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGSURUIDB4MDAwMDAwMDBFRTg1
MDAwMCAwMDAwOUMgKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQQ1Ig
MHgwMDAwMDAwMEVFODQwMDAwIDAwMDA1MCAodjAyIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBB
TUkgIDAwMDUwMDBGKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogQkdSVCAweDAwMDAwMDAwRUU4MzAwMDAgMDAwMDM4ICh2MDEgQUxBU0tBIEEgTSBJ
ICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBNQ0ZHIDB4MDAwMDAwMDBFRTgyMDAwMCAwMDAwQUMgKHYw
MSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IElPUlQgMHgwMDAwMDAwMEVFODEw
MDAwIDAwMDYxMCAodjAwIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUFBUVCAw
eDAwMDAwMDAwRUU3RjAwMDAgMDA2RTYwICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFN
UC4gMDEwMDAwMTMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTTElUIDB4MDAwMDAwMDBFRTdFMDAwMCAwMDAwMkQgKHYwMSBBbXBlcmUgQWx0cmEg
ICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQgMHgwMDAwMDAwMEVFN0QwMDAwIDAwMDZEMCAodjAz
IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQVBJQyAweDAwMDAwMDAwRUU4MDAw
MDAgMDAxOUY0ICh2MDUgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAzIEFNSSAgMDEwMDAwMTMpClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBQQ0NUIDB4
MDAwMDAwMDBFRTdDMDAwMCAwMDA1NzYgKHYwMiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDMgQU1Q
LiAwMTAwMDAxMykKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IFdTTVQgMHgwMDAwMDAwMEVFN0IwMDAwIDAwMDAyOCAodjAxIEFMQVNLQSBBIE0gSSAg
ICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogRlBEVCAweDAwMDAwMDAwRUU3QTAwMDAgMDAwMDQ0ICh2MDEg
QUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUENSOiBjb25zb2xlOiBwbDAxMSxt
bWlvMzIsMHgxMDAwMDI2MDAwMDAsMTE1MjAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDg4MzAwMDAw
LTB4ODgzZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDkwMDAwMDAwLTB4ZmZmZmZmZmZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBO
b2RlIDAgUFhNIDAgW21lbSAweDgwMDAwMDAwMDAwLTB4ODAwN2ZmZmZmZmZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhN
IDAgW21lbSAweDgwMTAwMDAwMDAwLTB4ODBmZmZmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBOVU1BOiBOT0RFX0RBVEEgW21lbSAweDgwZmY4MDg0
YjAwLTB4ODBmZjgwODZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBab25lIHJhbmdlczoKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdICAgRE1BICAgICAgW21lbSAweDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMGZm
ZmZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBE
TUEzMiAgICBlbXB0eQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gICBOb3JtYWwgICBbbWVtIDB4MDAwMDAwMDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNb3ZhYmxlIHpv
bmUgc3RhcnQgZm9yIGVhY2ggbm9kZQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gRWFybHkgbWVtb3J5IG5vZGUgcmFuZ2VzClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA4ODMw
MDAwMC0weDAwMDAwMDAwODgzZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MDAwMDAwMC0weDAwMDAw
MDAwOTFmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MjAwMDAwMC0weDAwMDAwMDAwOTNmZmZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDA5NDAwMDAwMC0weDAwMDAwMDAwZWUyZTZmZmZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDBlZTJlNzAwMC0weDAwMDAwMDAwZWU3NDBmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0MTAw
MC0weDAwMDAwMDAwZWU3NDRmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0NTAwMC0weDAwMDAwMDAw
ZWU3NGNmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZDAwMC0weDAwMDAwMDAwZWU3NGRmZmZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBlZTc0ZTAwMC0weDAwMDAwMDAwZWU3NGZmZmZdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBlZTc1MDAwMC0weDAwMDAwMDAwZWU5NmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk3MDAwMC0w
eDAwMDAwMDAwZWU5N2ZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk4MDAwMC0weDAwMDAwMDAwZjA2
NGZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmMDY1MDAwMC0weDAwMDAwMDAwZjc2NmZmZmZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDAwMDBmNzY3MDAwMC0weDAwMDAwMDAwZjc4NGZmZmZdClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBm
Nzg1MDAwMC0weDAwMDAwMDAwZjdmZGZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmN2ZlMDAwMC0weDAw
MDAwMDAwZmZjOGVmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM4ZjAwMC0weDAwMDAwMDAwZmZjOGZm
ZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM5MDAwMC0weDAwMDAwMDAwZmZmZmZmZmZdClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDA4MDAwMDAwMDAwMC0weDAwMDAwODAwN2ZmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDEwMDAw
MDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBJbml0bWVtIHNldHVwIG5vZGUgMCBbbWVtIDB4MDAwMDAwMDA4ODMwMDAw
MC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBPbiBub2RlIDAgdG90YWxwYWdlczogMTY3MTE5MzYKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDcxNzIgcGFnZXMgdXNl
ZCBmb3IgbWVtbWFwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIERNQSB6b25lOiAwIHBhZ2VzIHJlc2VydmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiA0NTkwMDggcGFnZXMsIExJRk8gYmF0Y2g6
NjMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFs
IHpvbmU6IDI1Mzk1MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDE2MjUyOTI4IHBhZ2VzLCBM
SUZPIGJhdGNoOjYzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBjbWE6IFJlc2VydmVkIDY0IE1pQiBhdCAweDAwMDAwMDAwZmJjMDAwMDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IHByb2JpbmcgZm9yIGNvbmR1
aXQgbWV0aG9kIGZyb20gQUNQSS4KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIHBzY2k6IFBTQ0l2MS4xIGRldGVjdGVkIGluIGZpcm13YXJlLgpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogVXNpbmcgc3RhbmRhcmQg
UFNDSSB2MC4yIGZ1bmN0aW9uIElEcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gcHNjaTogTUlHUkFURV9JTkZPX1RZUEUgbm90IHN1cHBvcnRlZC4KU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFNNQyBDYWxsaW5n
IENvbnZlbnRpb24gdjEuMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgwIC0+IE5vZGUgMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgzMDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHg2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NjAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHg3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ODAwMDAgLT4g
Tm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDgwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHg5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHhhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4YjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGIwMTAwIC0+IE5vZGUg
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4YzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGQwMDAwIC0+
IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4ZTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUw
MTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDAwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDEwMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDExMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIw
MDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMjAxMDAgLT4gTm9kZSAwClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MTMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAwMDAgLT4g
Tm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE0MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MTUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
NTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxNzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE3MDEwMCAt
PiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMDAwIC0+IE5vZGUgMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxODAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDE5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTkwMTAwIC0+IE5vZGUg
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDFhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYjAxMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDFjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MWMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAwMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MWUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZTAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDAwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MWYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgyMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDEwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgyMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIy
MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjIwMTAwIC0+IE5vZGUgMApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDIzMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMDAwIC0+
IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNDAxMDAgLT4gTm9kZSAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDI1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MjUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAwMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI2MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MjcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNzAxMDAg
LT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
ZXJjcHU6IEVtYmVkZGVkIDMzIHBhZ2VzL2NwdSBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjgK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6
IHM5NTE5MiByODE5MiBkMzE3ODQgdTEzNTE2OCBhbGxvYz0zMyo0MDk2ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMDAgWzBdIDAx
IFswXSAwMiBbMF0gMDMgWzBdIDA0IFswXSAwNSBbMF0gMDYgWzBdIDA3IApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDA4IFswXSAw
OSBbMF0gMTAgWzBdIDExIFswXSAxMiBbMF0gMTMgWzBdIDE0IFswXSAxNSAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSAxNiBbMF0g
MTcgWzBdIDE4IFswXSAxOSBbMF0gMjAgWzBdIDIxIFswXSAyMiBbMF0gMjMgClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMjQgWzBd
IDI1IFswXSAyNiBbMF0gMjcgWzBdIDI4IFswXSAyOSBbMF0gMzAgWzBdIDMxIApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDMyIFsw
XSAzMyBbMF0gMzQgWzBdIDM1IFswXSAzNiBbMF0gMzcgWzBdIDM4IFswXSAzOSAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA0MCBb
MF0gNDEgWzBdIDQyIFswXSA0MyBbMF0gNDQgWzBdIDQ1IFswXSA0NiBbMF0gNDcgClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNDgg
WzBdIDQ5IFswXSA1MCBbMF0gNTEgWzBdIDUyIFswXSA1MyBbMF0gNTQgWzBdIDU1IApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDU2
IFswXSA1NyBbMF0gNTggWzBdIDU5IFswXSA2MCBbMF0gNjEgWzBdIDYyIFswXSA2MyAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA2
NCBbMF0gNjUgWzBdIDY2IFswXSA2NyBbMF0gNjggWzBdIDY5IFswXSA3MCBbMF0gNzEgClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0g
NzIgWzBdIDczIFswXSA3NCBbMF0gNzUgWzBdIDc2IFswXSA3NyBbMF0gNzggWzBdIDc5IApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVy
ZmFjZQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogVmlydHVhbGl6YXRpb24gSG9zdCBFeHRlbnNpb25zClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGtlcm5l
bCBwYWdlIHRhYmxlIGlzb2xhdGlvbiBmb3JjZWQgT04gYnkgS0FTTFIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEtl
cm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiAoS1BUSSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEhhcmR3YXJlIGRp
cnR5IGJpdCBtYW5hZ2VtZW50ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBTcGVjdHJlLXY0ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBhbHRlcm5hdGl2ZXM6IHBhdGNoaW5nIGtlcm5lbCBjb2RlClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBCdWlsdCAxIHpvbmVsaXN0cywgbW9iaWxp
dHkgZ3JvdXBpbmcgb24uICBUb3RhbCBwYWdlczogMTY0NTA4MTIKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFBvbGljeSB6b25lOiBOb3JtYWwKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEtlcm5lbCBjb21tYW5kIGxpbmU6
IEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1hcm02NCByb290PVVVSUQ9NmMwODMy
N2MtNGJiZC00MzQxLWE0MjctYTcyYmMxNTMzODk1IHJvIHF1aWV0ClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuIGluZGl2aWR1
YWwgbWF4IGNwdSBjb250cmlidXRpb246IDQwOTYgYnl0ZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4gdG90YWwgY3B1X2V4
dHJhIGNvbnRyaWJ1dGlvbnM6IDMyMzU4NCBieXRlcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbiBtaW4gc2l6ZTogMTMxMDcy
IGJ5dGVzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmlu
dGs6IGxvZ19idWZfbGVuOiA1MjQyODggYnl0ZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogZWFybHkgbG9nIGJ1ZiBmcmVlOiAxMTkwMTYoOTAl
KQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGVudHJ5IGNh
Y2hlIGhhc2ggdGFibGUgZW50cmllczogODM4ODYwOCAob3JkZXI6IDE0LCA2NzEwODg2NCBieXRl
cywgbGluZWFyKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA0MTk0MzA0IChvcmRlcjogMTMsIDMzNTU0
NDMyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBtZW0gYXV0by1pbml0OiBzdGFjazpvZmYsIGhlYXAgYWxsb2M6b24sIGhlYXAgZnJl
ZTpvZmYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNvZnR3
YXJlIElPIFRMQjogbWFwcGVkIFttZW0gMHgwMDAwMDAwMGVhMmU3MDAwLTB4MDAwMDAwMDBlZTJl
NzAwMF0gKDY0TUIpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBNZW1vcnk6IDI1NTE2MzJLLzY2ODQ3NzQ0SyBhdmFpbGFibGUgKDExNzc2SyBrZXJuZWwgY29k
ZSwgMjQzNksgcndkYXRhLCA3MDA4SyByb2RhdGEsIDU0NDBLIGluaXQsIDU5OEsgYnNzLCAxNTg1
NTQ4SyByZXNlcnZlZCwgNjU1MzZLIGNtYS1yZXNlcnZlZCkKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJhbmRvbTogZ2V0X3JhbmRvbV91NjQgY2FsbGVkIGZy
b20gX19rbWVtX2NhY2hlX2NyZWF0ZSsweDM4LzB4NTYwIHdpdGggY3JuZ19pbml0PTAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNMVUI6IEhXYWxpZ249NjQs
IE9yZGVyPTAtMywgTWluT2JqZWN0cz0wLCBDUFVzPTgwLCBOb2Rlcz0xClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRpbmcgMzg1MzUg
ZW50cmllcyBpbiAxNTEgcGFnZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIGZ0cmFjZTogYWxsb2NhdGVkIDE1MSBwYWdlcyB3aXRoIDUgZ3JvdXBzClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IEhpZXJhcmNoaWNh
bCBSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSByY3U6IAlSQ1UgcmVzdHJpY3RpbmcgQ1BVcyBmcm9tIE5SX0NQVVM9MjU2IHRv
IG5yX2NwdV9pZHM9ODAuClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAJUnVkZSB2YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gCVRyYWNpbmcgdmFyaWFudCBvZiBUYXNrcyBS
Q1UgZW5hYmxlZC4KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IHJjdTogUkNVIGNhbGN1bGF0ZWQgdmFsdWUgb2Ygc2NoZWR1bGVyLWVubGlzdG1lbnQgZGVsYXkg
aXMgMjUgamlmZmllcy4KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHJjdTogQWRqdXN0aW5nIGdlb21ldHJ5IGZvciByY3VfZmFub3V0X2xlYWY9MTYsIG5yX2Nw
dV9pZHM9ODAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE5S
X0lSUVM6IDY0LCBucl9pcnFzOiA2NCwgcHJlYWxsb2NhdGVkIGlycXM6IDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBHSUM6IFVzaW5nIHNwbGl0
IEVPSS9EZWFjdGl2YXRlIG1vZGUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEdJQ3YzOiA2NzIgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDAgRXh0ZW5kZWQgU1BJcyBpbXBsZW1l
bnRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6
IERpc3RyaWJ1dG9yIGhhcyBubyBSYW5nZSBTZWxlY3RvciBzdXBwb3J0ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMTYgUFBJcyBpbXBsZW1lbnRl
ZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IENQ
VTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVjMDAw
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhN
IDAgLT4gSVRTIDAgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMSAtPiBOb2RlIDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAyIC0+IE5v
ZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDog
UFhNIDAgLT4gSVRTIDMgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNCAtPiBOb2RlIDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA1IC0+
IE5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JB
VDogUFhNIDAgLT4gSVRTIDYgLT4gTm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNyAtPiBOb2RlIDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDQw
MDAwLTB4MTAwMTAwMDVmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBA
ODAwMDAyYzAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQwMDAw
OiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDJkMDAwMCAoZmxh
dCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwNjAwMDAtMHgxMDAxMDAwN2ZmZmZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAw
MTAwMDYwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJmMDAwMCAoaW5kaXJlY3Qs
IGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRl
cnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzAwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hy
IDEpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21l
bSAweDEwMDEwMDA4MDAwMC0weDEwMDEwMDA5ZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCA4
MTkyIERldmljZXMgQDgwMDAwMzIwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAx
KQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDA4MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAw
MDAzMzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGEwMDAwLTB4MTAw
MTAwMGJmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTQDB4MDAwMDEwMDEwMDBhMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzNTAw
MDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxvY2F0
ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM2MDAwMCAoZmxhdCwgZXN6IDIs
IHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwYzAwMDAtMHgxMDAxMDAwZGZmZmZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAw
OiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDM4MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBw
c3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIElUU0AweDAwMDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29s
bGVjdGlvbnMgQDgwMDAwMzkwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEw
MDBlMDAwMC0weDEwMDEwMDBmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmlj
ZXMgQDgwMDAwM2IwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBl
MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzYzAwMDAg
KGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTAwMDAwLTB4MTAwMTAwMTFmZmZm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDEwMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzZTAwMDAgKGluZGly
ZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBhbGxvY2F0ZWQgMzI3Njgg
SW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDNmMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEss
IHNociAxKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRT
IFttZW0gMHgxMDAxMDAxMjAwMDAtMHgxMDAxMDAxM2ZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0
ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDgxMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBz
aHIgMSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0Aw
eDAwMDAxMDAxMDAxMjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMg
QDgwMDAwODIwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogdXNpbmcgTFBJIHByb3BlcnR5
IHRhYmxlIEAweDAwMDAwODAwMDA4MzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBDUFUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDg0MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSSBHVERUOiBmb3VuZCAxIG1lbW9yeS1tYXBwZWQgdGltZXIgYmxvY2socyku
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhcmNoX3RpbWVy
OiBjcDE1IGFuZCBtbWlvIHRpbWVyKHMpIHJ1bm5pbmcgYXQgMjUuMDBNSHogKHBoeXMvcGh5cyku
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbG9ja3NvdXJj
ZTogYXJjaF9zeXNfY291bnRlcjogbWFzazogMHhmZmZmZmZmZmZmZmZmZiBtYXhfY3ljbGVzOiAw
eDVjNDA5MzliNSwgbWF4X2lkbGVfbnM6IDQ0MDc5NTIwMjY0NiBucwpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMV0gc2NoZWRfY2xvY2s6IDU2IGJpdHMgYXQgMjVN
SHosIHJlc29sdXRpb24gNDBucywgd3JhcHMgZXZlcnkgNDM5ODA0NjUxMTEwMG5zClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDU0XSBDb25zb2xlOiBjb2xvdXIgZHVt
bXkgZGV2aWNlIDgweDI1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDc1XSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIwMjAwOTI1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMjY5XSBDYWxpYnJhdGluZyBkZWxheSBsb29wIChza2lwcGVkKSwg
dmFsdWUgY2FsY3VsYXRlZCB1c2luZyB0aW1lciBmcmVxdWVuY3kuLiA1MC4wMCBCb2dvTUlQUyAo
bHBqPTEwMDAwMCkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAyNzFd
IHBpZF9tYXg6IGRlZmF1bHQ6IDgxOTIwIG1pbmltdW06IDY0MApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDMwNF0gTFNNOiBTZWN1cml0eSBGcmFtZXdvcmsgaW5pdGlh
bGl6aW5nClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzEwXSBZYW1h
OiBkaXNhYmxlZCBieSBkZWZhdWx0OyBlbmFibGUgd2l0aCBzeXNjdGwga2VybmVsLnlhbWEuKgpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMyN10gQXBwQXJtb3I6IEFw
cEFybW9yIGluaXRpYWxpemVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMzI4XSBUT01PWU8gTGludXggaW5pdGlhbGl6ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAzNjldIE1vdW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMx
MDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDM5M10gTW91bnRwb2ludC1jYWNoZSBoYXNoIHRhYmxlIGVu
dHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEyNTNdIHJjdTogSGllcmFyY2hpY2FsIFNS
Q1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxMzkzXSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5NF0gUGxhdGZvcm0gTVNJ
OiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDEzOTZdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
Mzk3XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5OF0gUGxhdGZvcm0gTVNJOiBJ
VFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDEzOTldIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRv
bWFpbiBjcmVhdGVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDAx
XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwMl0gUGxhdGZvcm0gTVNJOiBJVFNA
MHgxMDAxMDAxMjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDE0MDZdIFBDSS9NU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3Jl
YXRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwN10gUENJL01T
STogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAxNDA4XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwODAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTBd
IFBDSS9NU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxM10gUENJL01TSTogSVRTQDB4MTAwMTAwMGMw
MDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxNDE1XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTZdIFBDSS9NU0k6IElUU0AweDEw
MDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMTQxN10gUENJL01TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVk
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDIwXSBmc2wtbWMgTVNJ
OiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDE0MjFdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA2MDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQy
M10gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI0XSBmc2wtbWMgTVNJOiBJVFNAMHgx
MDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDE0MjVdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3Jl
YXRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyNl0gZnNsLW1j
IE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI3XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMDAw
MDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDE0MjldIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzNF0gUmVtYXBwaW5nIGFuZCBl
bmFibGluZyBFRkkgc2VydmljZXMuClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAzOTA4XSBzbXA6IEJyaW5naW5nIHVwIHNlY29uZGFyeSBDUFVzIC4uLgpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDQ1M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ0NzFdIEdJ
Q3YzOiBDUFUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFhMDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA3YzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ0ODldIEdJ
Q3YzOiBDUFUxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MDg1MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDUyMV0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDUzMF0gQ1BVMTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDgxN10gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ4MzNd
IEdJQ3YzOiBDUFUyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA2NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ4NTFd
IEdJQ3YzOiBDUFUyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMDg2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDg4N10g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDg5N10gQ1BVMjog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTQwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTMzNV0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUz
NTFdIEdJQ3YzOiBDUFUzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA4NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUz
NjldIEdJQ3YzOiBDUFUzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDg3MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTQw
Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTQxMl0gQ1BV
MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWMwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTYyOF0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU2NDJdIEdJQ3YzOiBDUFU0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA1NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU2NjFdIEdJQ3YzOiBDUFU0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMDg4MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NTY5Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTcwNl0g
Q1BVNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTk5OF0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDYwMTRdIEdJQ3YzOiBDUFU1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE4MDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA3NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDYwMzJdIEdJQ3YzOiBDUFU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDg5MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNjA2N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjA3
N10gQ1BVNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTgwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjQwM10gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDY0MTldIEdJQ3YzOiBDUFU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA2YzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDY0MzhdIEdJQ3YzOiBDUFU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDhhMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNjQ3M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NjQ4M10gQ1BVNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjc5OF0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDY4MTZdIEdJQ3YzOiBDUFU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFlMDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA4YzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDY4MzVdIEdJQ3YzOiBDUFU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDhiMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNjg3MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNjg4MF0gQ1BVNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzIwMF0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDcyMTNdIEdJQ3YzOiBDUFU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNjMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNzIzMV0gR0lDdjM6IENQVTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGMwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA3MjY4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA3Mjc5XSBDUFU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYTAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3ODA2
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVOQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNzgyNV0gR0lDdjM6IENQVTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDljMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNzg0M10gR0lDdjM6IENQVTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGQwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA3ODczXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA3ODgzXSBDUFU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4
MTk1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgyMDhdIEdJQ3YzOiBDUFUxMDogZm91bmQgcmVkaXN0cmlidXRv
ciBjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgyMjhdIEdJQ3YzOiBDUFUxMDogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ZTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgyNjRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDgyNzVdIENQVTEwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAwYzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA4NTk5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTEKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2MThdIEdJQ3YzOiBDUFUxMTogZm91bmQgcmVkaXN0
cmlidXRvciAyNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTQwMDAwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjM4XSBHSUN2MzogQ1BVMTE6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGYwMDAwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjcxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA4NjgyXSBDUFUxMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMjQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwOTAwMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEyClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDEzXSBHSUN2MzogQ1BVMTI6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzQwMDAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDMyXSBHSUN2MzogQ1BVMTI6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTAwMDAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDY4XSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDc5XSBDUFUxMjogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMDgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwOTQwNV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEzClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NDI0XSBHSUN2MzogQ1BVMTM6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk0MDAwMApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ0M10gR0lDdjM6IENQVTEz
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkxMDAwMApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ3NV0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ4N10gQ1BVMTM6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk3OThdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFUxNApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTgxMl0gR0lDdjM6
IENQVTE0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRj
MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTgzNF0gR0lDdjM6
IENQVTE0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDky
MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg3MV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg4M10gQ1BVMTQ6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyMDJdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFUxNQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDIyMl0g
R0lDdjM6IENQVTE1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDBhYzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyNDJd
IEdJQ3YzOiBDUFUxNTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5MzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyNzZd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyODddIENQVTE1
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwNTk5XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMTYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA2MDldIEdJQ3YzOiBDUFUxNjogZm91bmQgcmVkaXN0cmlidXRvciAyMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDAxYzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA2MzBdIEdJQ3YzOiBDUFUxNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDA5NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA2NjhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA2ODBd
IENQVTE2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExMDAxXSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVMTcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTEwMTNdIEdJQ3YzOiBDUFUxNzogZm91bmQgcmVkaXN0cmlidXRvciA0MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAyNDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTEwMzRdIEdJQ3YzOiBDUFUxNzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA5NTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTEwNzJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTEwODRdIENQVTE3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAwMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNDAxXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTE0MTFdIEdJQ3YzOiBDUFUxODogZm91bmQgcmVkaXN0cmlidXRvciAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDE0MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTQzM10gR0lDdjM6IENQVTE4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDk2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTQ3MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMTQ4NF0gQ1BVMTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAwMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTE4MDJd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxOQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTgxNF0gR0lDdjM6IENQVTE5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDYw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJjMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTgzNl0gR0lDdjM6IENQVTE5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk3MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTg3Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTg4NV0gQ1BVMTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA2
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTIyMDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMjIxN10gR0lDdjM6IENQVTIwOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDEzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2MDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTIyMzhdIEdJQ3YzOiBDUFUyMDogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ODAwMDAKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTIyNzRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTIyODZdIENQVTIwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEyNTk4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjEKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI2MTVdIEdJQ3YzOiBDUFUyMTogZm91bmQgcmVk
aXN0cmlidXRvciAxYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODAwMDAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNjM1XSBHSUN2MzogQ1BVMjE6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTkwMDAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNjY4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNjgwXSBDUFUyMTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMWIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMjk5NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIyClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMDEwXSBHSUN2MzogQ1BVMjI6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMTUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY4MDAwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzAzMF0gR0lDdjM6IENQVTIyOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlhMDAwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzA2Nl0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzA3OF0gQ1BVMjI6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDE1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTMzOTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUy
MwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzQxMF0gR0lDdjM6IENQ
VTIzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4ODAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM0MzFdIEdJQ3YzOiBD
UFUyMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5YjAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM0NjRdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM0NzZdIENQVTIzOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzODEwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVMjQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM4MjRdIEdJ
Q3YzOiBDUFUyNDogZm91bmQgcmVkaXN0cmlidXRvciAxMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwNTgwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzODQ1XSBH
SUN2MzogQ1BVMjQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwOWMwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzODgwXSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzODkyXSBDUFUyNDog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDIwNV0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTI1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0
MjIxXSBHSUN2MzogQ1BVMjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkwMDAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDc4MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NDI0M10gR0lDdjM6IENQVTI1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMDlkMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NDI3Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDI4OV0g
Q1BVMjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5MDAwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ2MDFdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFUyNgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNDYxN10gR0lDdjM6IENQVTI2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE3MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA3MDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQ2MzldIEdJQ3YzOiBDUFUyNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA5ZTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQ2NzRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTQ2ODddIENQVTI2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNzAwMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MDAzXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTUwMjJdIEdJQ3YzOiBDUFUyNzogZm91bmQgcmVkaXN0cmlidXRvciAxZjAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1MDQzXSBHSUN2MzogQ1BVMjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1MDc4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1MDkxXSBDUFUyNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWYw
MDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NTM5OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI4ClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NDExXSBHSUN2MzogQ1BVMjg6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NDMzXSBHSUN2MzogQ1BVMjg6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NDY5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1NDgyXSBDUFUyODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMGIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNTgwN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI5ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1ODI1XSBHSUN2MzogQ1BVMjk6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEwMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTg0N10gR0lDdjM6IENQVTI5OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGExMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTg3OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNTg5MV0gQ1BVMjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDIzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTYxOTldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjIxMl0gR0lDdjM6IENQVTMwOiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIGQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ4MDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjIzM10gR0lDdjM6IENQVTMwOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEyMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjI2OF0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjI4Ml0gQ1BVMzA6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDBkMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTY2MDRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjYyM10gR0lDdjM6IENQVTMx
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhODAwMDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY2NDVdIEdJQ3YzOiBDUFUz
MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMzAwMDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY2NzhdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY2OTJdIENQVTMxOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3MDAxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMzIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcwMTRdIEdJQ3Yz
OiBDUFUzMjogZm91bmQgcmVkaXN0cmlidXRvciA5MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAz
ODAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcwMzddIEdJQ3Yz
OiBDUFUzMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBh
NDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcwNzJdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcwODZdIENQVTMyOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwOTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NDExXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMzMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc0MzBd
IEdJQ3YzOiBDUFUzMzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwOTgwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NDUz
XSBHSUN2MzogQ1BVMzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYTUwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NDg3
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NTAwXSBDUFUz
MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzgwNl0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTM0ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3ODIxXSBHSUN2MzogQ1BVMzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZjAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwNTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3ODQ1XSBHSUN2MzogQ1BVMzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYTYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3ODgyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3ODk2
XSBDUFUzNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGYwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODIwNV0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE4MjI1XSBHSUN2MzogQ1BVMzU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjcwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMGIwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODI0OV0gR0lDdjM6IENQVTM1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGE3MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODI4NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODI5OF0gQ1BVMzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI3MDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg2MDRd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODYxNV0gR0lDdjM6IENQVTM2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDMw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODYzOV0gR0lDdjM6IENQVTM2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE4MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODY3N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODY5Ml0gQ1BVMzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAz
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTkwMDVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNwpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTAxNl0gR0lDdjM6IENQVTM3OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI4MDAwMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTA0MF0gR0lDdjM6IENQVTM3OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE5MDAwMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTA3N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTA5Ml0gQ1BVMzc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDA1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTk0MDRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTQxNF0gR0lDdjM6IENQVTM4OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE4MDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTQ0MF0gR0lDdjM6IENQVTM4OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFhMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTQ3OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTQ5M10gQ1BVMzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDAxMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTk4MDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTgxNV0gR0lDdjM6IENQVTM5OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDMwMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTg0MF0gR0lDdjM6IENQVTM5OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFiMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTg3Nl0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTg5Ml0gQ1BVMzk6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDA3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjAyMDFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDIxNV0gR0lDdjM6IENQVTQw
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1ZTAwMDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAyMzddIEdJQ3YzOiBDUFU0
MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYzAwMDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAyNzBdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAyODZdIENQVTQwOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjA0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA2MjBdIEdJQ3Yz
OiBDUFU0MTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
N2UwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjQ0XSBHSUN2
MzogQ1BVNDE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YWQwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjc2XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjkxXSBDUFU0MTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTAwMl0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTQyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMDE4
XSBHSUN2MzogQ1BVNDI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDY2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTA0
Ml0gR0lDdjM6IENQVTQyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGFlMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTA3
Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTA5MV0gQ1BV
NDI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE0MDJdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU0MwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMTQxOV0gR0lDdjM6IENQVTQzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA4NjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjE0NDNdIEdJQ3YzOiBDUFU0MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBhZjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjE0NzZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE0
OTBdIENQVTQzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYzAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxODAyXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjE4MTZdIEdJQ3YzOiBDUFU0NDogZm91bmQgcmVkaXN0cmlidXRvciAxMDAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwNTYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIxODQxXSBHSUN2MzogQ1BVNDQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIxODc2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIxODkyXSBDUFU0NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjE5
OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIyMjE1XSBHSUN2MzogQ1BVNDU6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MTgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMjIzOV0gR0lDdjM6IENQVTQ1OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIxMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMjI3MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMjI4Nl0gQ1BVNDU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDE4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjI2MDJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NgpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjYxOF0gR0lDdjM6IENQVTQ2OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDE2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2ZTAwMDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI2NDNdIEdJQ3YzOiBDUFU0NjogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMjAwMDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI2NzldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjI2OTVdIENQVTQ2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAxNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIzMDA1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDcKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMwMjNdIEdJQ3YzOiBDUFU0NzogZm91bmQg
cmVkaXN0cmlidXRvciAxZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGUwMDAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMDQ3XSBHSUN2MzogQ1BVNDc6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjMwMDAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMDgyXSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMDk3XSBDUFU0NzogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMWUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMzQwMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ4ClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNDEzXSBHSUN2MzogQ1BVNDg6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2UwMDAwClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNDM4XSBHSUN2MzogQ1BVNDg6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjQwMDAwClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNDc1XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNDkxXSBDUFU0ODogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzgwN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTQ5ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzODI1XSBHSUN2Mzog
Q1BVNDk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDll
MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzg1MV0gR0lDdjM6
IENQVTQ5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI1
MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzg4NV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzkwMV0gQ1BVNDk6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQyMDJdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU1MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDIxNV0g
R0lDdjM6IENQVTUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDQ2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDIzOV0g
R0lDdjM6IENQVTUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGI2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDI3NV0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDI5MV0gQ1BVNTA6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ2MTFdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU1MQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NDYzMF0gR0lDdjM6IENQVTUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0MDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDBhNjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ2NTVdIEdJQ3YzOiBDUFU1MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBiNzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ2ODhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ3MDRd
IENQVTUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNDAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1MDA3XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNTIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUwMTldIEdJQ3YzOiBDUFU1MjogZm91bmQgcmVkaXN0cmlidXRvciA4MDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAzNjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUwNDZdIEdJQ3YzOiBDUFU1MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBiODAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUwODNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjUxMDBdIENQVTUyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwODAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NDAyXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjU0MjBdIEdJQ3YzOiBDUFU1MzogZm91bmQgcmVkaXN0cmlidXRvciAyMDAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1NDQ2XSBHSUN2MzogQ1BVNTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjkwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1NDgwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI1NDk3XSBDUFU1MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjAw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NTgwMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU0ClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1ODE0XSBHSUN2MzogQ1BVNTQ6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGUwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1ODQzXSBHSUN2MzogQ1BVNTQ6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYmEwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1ODgwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1ODk3XSBDUFU1NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMGUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyNjIwOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU1ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MjI4XSBHSUN2MzogQ1BVNTU6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFlMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjI1NF0gR0lDdjM6IENQVTU1OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJiMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjI4OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjMwNV0gQ1BVNTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDI2MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjY2MDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NgpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjYxOF0gR0lDdjM6IENQVTU2OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFlMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjY0NV0gR0lDdjM6IENQVTU2OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJjMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjY4M10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjcwMF0gQ1BVNTY6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDAyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjcwMDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NwpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzAxOF0gR0lDdjM6IENQVTU3
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI2MDAwMApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzA0NV0gR0lDdjM6IENQVTU3
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJkMDAwMApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzA4M10gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzEwMV0gQ1BVNTc6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc0MDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFU1OApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzQxOV0gR0lDdjM6
IENQVTU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxNjAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc0NDZdIEdJQ3YzOiBD
UFU1ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZTAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc0ODRdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc1MDJdIENQVTU4OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3ODEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNTkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc4MjRdIEdJ
Q3YzOiBDUFU1OTogZm91bmQgcmVkaXN0cmlidXRvciA2MDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAyZTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc4NTNdIEdJ
Q3YzOiBDUFU1OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBiZjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc4ODldIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc5MDhdIENQVTU5OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MTk4XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNjAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgy
MTJdIEdJQ3YzOiBDUFU2MDogZm91bmQgcmVkaXN0cmlidXRvciAxMzAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNjIwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
MjM5XSBHSUN2MzogQ1BVNjA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYzAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
Mjc2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MjkzXSBD
UFU2MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODYwMV0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTYxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4NjE4XSBHSUN2MzogQ1BVNjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDgyMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyODY0NV0gR0lDdjM6IENQVTYxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGMxMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyODY3OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
ODY5Nl0gQ1BVNjE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkwMDNdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyOTAxOV0gR0lDdjM6IENQVTYyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA2YTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjkwNDddIEdJQ3YzOiBDUFU2MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjkwODJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjkxMDBdIENQVTYyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5
NDA1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjMKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjk0MjNdIEdJQ3YzOiBDUFU2MzogZm91bmQgcmVkaXN0cmlidXRv
ciAxZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGEwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI5NDQ5XSBHSUN2MzogQ1BVNjM6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzMwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI5NDgyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI5NTAwXSBDUFU2MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMWQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyOTgwOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY0ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5ODIzXSBHSUN2MzogQ1BVNjQ6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVhMDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTg1MF0gR0lDdjM6IENQVTY0OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM0MDAwMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTg4NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyOTkwMV0gQ1BVNjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDExMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzAyMDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2NQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDIyN10gR0lDdjM6IENQVTY1OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDE5MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YTAwMDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAyNTNdIEdJQ3YzOiBDUFU2NTogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNTAwMDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAyODhdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAzMDNdIENQVTY1OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAxOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMwNTk5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjYK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA2MTVdIEdJQ3YzOiBDUFU2
NjogZm91bmQgcmVkaXN0cmlidXRvciAxNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzIwMDAw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNjQyXSBHSUN2MzogQ1BV
NjY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzYwMDAw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNjc5XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNjk2XSBDUFU2NjogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTAwMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTY3ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMDIxXSBHSUN2
MzogQ1BVNjc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDkyMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTA0OV0gR0lD
djM6IENQVTY3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGM3MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTA4NF0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTEwMV0gQ1BVNjc6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE0MDNdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU2OApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTQx
Nl0gR0lDdjM6IENQVTY4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGIwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDQyMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTQ0
NF0gR0lDdjM6IENQVTY4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGM4MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTQ4
MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTQ5OF0gQ1BV
Njg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBiMDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE3OTZdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU2OQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMTgxNV0gR0lDdjM6IENQVTY5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDBhMjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzE4NDNdIEdJQ3YzOiBDUFU2OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBjOTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzE4NzZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE4
OTJdIENQVTY5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyMTk2XSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzIyMTBdIEdJQ3YzOiBDUFU3MDogZm91bmQgcmVkaXN0cmlidXRvciBkMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA0YTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzIyMzhdIEdJQ3YzOiBDUFU3MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBjYTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzIyNzNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzIyOTFdIENQVTcwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNjAz
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzI2MjNdIEdJQ3YzOiBDUFU3MTogZm91bmQgcmVkaXN0cmlidXRvciAy
NTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYWEwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyNjUxXSBHSUN2MzogQ1BVNzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2IwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyNjg0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMyNzAyXSBDUFU3MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MjUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMzAwMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcyClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMDE1XSBHSUN2MzogQ1BVNzI6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2EwMDAwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMDQzXSBHSUN2MzogQ1BVNzI6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2MwMDAwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMDgxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMzMDk4XSBDUFU3MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMDkwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzMzQwN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTczClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNDI2XSBHSUN2MzogQ1BVNzM6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDlhMDAwMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzQ1Nl0gR0lDdjM6IENQVTczOiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNkMDAwMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzQ4OV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzUwN10gQ1BVNzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDIxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzM3OTRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzgwOF0gR0lDdjM6IENQVTc0OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIGYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUyMDAwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzgzOF0gR0lDdjM6IENQVTc0OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNlMDAwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzg3NV0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzg5M10gQ1BVNzQ6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDBmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzQyMDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3
NQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDIyM10gR0lDdjM6IENQ
VTc1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBiMjAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQyNTJdIEdJQ3YzOiBD
UFU3NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjZjAw
MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQyODZdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQzMDNdIENQVTc1OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0NjA1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNzYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ2MTZdIEdJ
Q3YzOiBDUFU3NjogZm91bmQgcmVkaXN0cmlidXRvciAzMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAyMjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ2NDZdIEdJ
Q3YzOiBDUFU3NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBkMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ2ODRdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ3MDJdIENQVTc2OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MDA2XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNzcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUw
MTddIEdJQ3YzOiBDUFU3NzogZm91bmQgcmVkaXN0cmlidXRvciA1MDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDAyYTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUw
NDddIEdJQ3YzOiBDUFU3NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBkMTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUw
ODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUxMDVdIENQ
VTc3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNTAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NDEwXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVNzgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU0MjBdIEdJQ3YzOiBDUFU3ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDAxYTAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU0NTFdIEdJQ3YzOiBDUFU3ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBkMjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU0ODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1
MDhdIENQVTc4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1ODAxXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzU4MTRdIEdJQ3YzOiBDUFU3OTogZm91bmQgcmVkaXN0cmlidXRvciA3MDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDAzMjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzU4NDRdIEdJQ3YzOiBDUFU3OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBkMzAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzU4ODJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU5MDBdIENQVTc5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1OTYz
XSBzbXA6IEJyb3VnaHQgdXAgMSBub2RlLCA4MCBDUFVzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM1OTgzXSBTTVA6IFRvdGFsIG9mIDgwIHByb2Nlc3NvcnMgYWN0aXZh
dGVkLgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTk4NF0gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogUHJpdmlsZWdlZCBBY2Nlc3MgTmV2ZXIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5ODVdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IExT
RSBhdG9taWMgaW5zdHJ1Y3Rpb25zClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM1OTg2XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBVc2VyIEFjY2VzcyBPdmVycmlkZQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTk4Nl0gQ1BVIGZlYXR1cmVz
OiBkZXRlY3RlZDogMzItYml0IEVMMCBTdXBwb3J0ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM1OTg3XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDb21tb24gbm90IFBy
aXZhdGUgdHJhbnNsYXRpb25zClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM1OTg4XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIFBvaW50
IG9mIFBlcnNpc3RlbmNlClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1
OTg5XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBSQVMgRXh0ZW5zaW9uIFN1cHBvcnQKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5ODldIENQVSBmZWF0dXJlczogZGV0
ZWN0ZWQ6IERhdGEgY2FjaGUgY2xlYW4gdG8gdGhlIFBvVSBub3QgcmVxdWlyZWQgZm9yIEkvRCBj
b2hlcmVuY2UKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5OTBdIENQ
VSBmZWF0dXJlczogZGV0ZWN0ZWQ6IENSQzMyIGluc3RydWN0aW9ucwpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTk5MF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3Bl
Y3VsYXRpdmUgU3RvcmUgQnlwYXNzaW5nIFNhZmUgKFNTQlMpClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDU4NDMzXSBDUFU6IEFsbCBDUFUocykgc3RhcnRlZCBhdCBFTDIK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDY0MDNdIG5vZGUgMCBkZWZl
cnJlZCBwYWdlcyBpbml0aWFsaXNlZCBpbiA0ODRtcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU0NjU3Ml0gZGV2dG1wZnM6IGluaXRpYWxpemVkClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3MTgzXSBSZWdpc3RlcmVkIGNwMTVfYmFycmllciBl
bXVsYXRpb24gaGFuZGxlcgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0
NzIwNV0gUmVnaXN0ZXJlZCBzZXRlbmQgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDcyMDddIEtBU0xSIGVuYWJsZWQKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDcyNjVdIGNsb2Nrc291cmNlOiBqaWZmaWVzOiBt
YXNrOiAweGZmZmZmZmZmIG1heF9jeWNsZXM6IDB4ZmZmZmZmZmYsIG1heF9pZGxlX25zOiA3NjQ1
MDQxNzg1MTAwMDAwIG5zClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3
MzE4XSBmdXRleCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOSwgMjA5NzE1MiBi
eXRlcywgbGluZWFyKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0Nzc1
MF0gcGluY3RybCBjb3JlOiBpbml0aWFsaXplZCBwaW5jdHJsIHN1YnN5c3RlbQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzkwOF0gU01CSU9TIDMuMy4wIHByZXNlbnQu
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3OTEyXSBETUk6IEdJR0FC
WVRFIFIxNTItUDMwLTAwL01QMzItQVIxLTAwLCBCSU9TIEYxMyAoU0NQOiAxLjUuMjAyMTA0MjYp
IDA1LzIxLzIwMjEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDgxNDFd
IE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTYKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC41NDg3MjldIERNQTogcHJlYWxsb2NhdGVkIDQwOTYgS2lCIEdGUF9L
RVJORUwgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTQ4ODU1XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VS
TkVMfEdGUF9ETUEgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ4OTc5XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBH
RlBfS0VSTkVMfEdGUF9ETUEzMiBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlvbnMKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDg5ODZdIGF1ZGl0OiBpbml0aWFsaXppbmcg
bmV0bGluayBzdWJzeXMgKGRpc2FibGVkKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjU0OTAyN10gYXVkaXQ6IHR5cGU9MjAwMCBhdWRpdCgwLjU0ODoxKTogc3RhdGU9aW5p
dGlhbGl6ZWQgYXVkaXRfZW5hYmxlZD0wIHJlcz0xClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTQ5MTE1XSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVy
bm9yICdmYWlyX3NoYXJlJwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0
OTExNl0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnYmFuZ19iYW5n
JwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0OTExN10gdGhlcm1hbF9z
eXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnc3RlcF93aXNlJwpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0OTExN10gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQg
dGhlcm1hbCBnb3Zlcm5vciAndXNlcl9zcGFjZScKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDkxMThdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJu
b3IgJ3Bvd2VyX2FsbG9jYXRvcicKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC41NDkxNzFdIGNwdWlkbGU6IHVzaW5nIGdvdmVybm9yIGxhZGRlcgpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU0OTE3OV0gY3B1aWRsZTogdXNpbmcgZ292ZXJub3IgbWVu
dQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0OTI3Ml0gRGV0ZWN0ZWQg
MTUgUENDIFN1YnNwYWNlcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0
OTI4OF0gUmVnaXN0ZXJpbmcgUENDIGRyaXZlciBhcyBNYWlsYm94IGNvbnRyb2xsZXIKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDkzMDddIGh3LWJyZWFrcG9pbnQ6IGZv
dW5kIDYgYnJlYWtwb2ludCBhbmQgNCB3YXRjaHBvaW50IHJlZ2lzdGVycy4KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDk3MDJdIEFTSUQgYWxsb2NhdG9yIGluaXRpYWxp
c2VkIHdpdGggMzI3NjggZW50cmllcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjU0OTcyNV0gQUNQSTogYnVzIHR5cGUgUENJIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDk3MjZdIGFjcGlwaHA6IEFDUEkgSG90IFBsdWcgUENJ
IENvbnRyb2xsZXIgRHJpdmVyIHZlcnNpb246IDAuNQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU0OTc2M10gU2VyaWFsOiBBTUJBIFBMMDExIFVBUlQgZHJpdmVyClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUyMTExXSBIdWdlVExCIHJlZ2lzdGVy
ZWQgMS4wMCBHaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTIxMTJdIEh1Z2VUTEIgcmVnaXN0ZXJlZCAzMi4w
IE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjU1MjExM10gSHVnZVRMQiByZWdpc3RlcmVkIDIuMDAgTWlCIHBh
Z2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTUyMTE0XSBIdWdlVExCIHJlZ2lzdGVyZWQgNjQuMCBLaUIgcGFnZSBzaXpl
LCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42Nzk1MTNdIEFDUEk6IEFkZGVkIF9PU0koTW9kdWxlIERldmljZSkKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzk1MTRdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vz
c29yIERldmljZSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzk1MTVd
IEFDUEk6IEFkZGVkIF9PU0koMy4wIF9TQ1AgRXh0ZW5zaW9ucykKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42Nzk1MTZdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIEFn
Z3JlZ2F0b3IgRGV2aWNlKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY3
OTUxN10gQUNQSTogQWRkZWQgX09TSShMaW51eC1EZWxsLVZpZGVvKQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY3OTUxOF0gQUNQSTogQWRkZWQgX09TSShMaW51eC1MZW5v
dm8tTlYtSERNSS1BdWRpbykKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42
Nzk1MThdIEFDUEk6IEFkZGVkIF9PU0koTGludXgtSFBJLUh5YnJpZC1HcmFwaGljcykKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42ODY1MDJdIEFDUEk6IDIgQUNQSSBBTUwg
dGFibGVzIHN1Y2Nlc3NmdWxseSBhY3F1aXJlZCBhbmQgbG9hZGVkClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjg3ODU5XSBBQ1BJOiBJbnRlcnByZXRlciBlbmFibGVkClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg3ODYwXSBBQ1BJOiBVc2luZyBH
SUMgZm9yIGludGVycnVwdCByb3V0aW5nClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNjg3ODY3XSBBQ1BJOiBNQ0ZHIHRhYmxlIGRldGVjdGVkLCA4IGVudHJpZXMKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTY2ODhdIEFSTUgwMDExOjAwOiB0dHlB
TUEwIGF0IE1NSU8gMHgxMDAwMDI2MDAwMDAgKGlycSA9IDM2LCBiYXNlX2JhdWQgPSAwKSBpcyBh
IFNCU0EKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTY3MjldIHByaW50
azogY29uc29sZSBbdHR5QU1BMF0gZW5hYmxlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5Nzg0Ml0gQVJNSDAwMTE6MDE6IHR0eUFNQTEgYXQgTU1JTyAweDEwMDAwMjYy
MDAwMCAoaXJxID0gMzcsIGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODkwN10gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kw
XSAoZG9tYWluIDAwMGMgW2J1cyAwMC1mZl0pClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk4OTEzXSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRl
bmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg5ODFdIGFjcGkgUE5QMEEwODowMDogX09T
QzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1F
IExUUl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkwNDFdIGFjcGkg
UE5QMEEwODowMDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MDQ1XSBhY3BpIFBOUDBBMDg6
MDA6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZm
Zl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmZDI0OWJmODdlYTU4ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5Njc1XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXJl
YSBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAy
OjAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NjgyXSBhY3BpIFBO
UDBBMDg6MDA6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9y
IFtidXMgMDAtZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NzQ4
XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMGM6MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTk3NTBdIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTc1MV0gcGNpX2J1cyAwMDBjOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3NTFdIHBjaV9idXMgMDAwYzowMDogcm9vdCBi
dXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTk3NzFdIHBjaSAwMDBjOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3Mg
MHgwNjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk4MTldIHBj
aSAwMDBjOjAwOjAxLjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk4MjZdIHBjaSAwMDBjOjAwOjAxLjA6
IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTk4NDJdIHBjaSAwMDBjOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5ODQzXSBwY2kgMDAwYzowMDowMS4wOiBQTUUj
IHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk5OTIwXSBhY3BpcGhwOiBTbG90IFsxXSByZWdpc3RlcmVkClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTMzXSBwY2kgMDAwYzowMTowMC4wOiBbMTAw
MDowMDk3XSB0eXBlIDAwIGNsYXNzIDB4MDEwNzAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjk5OTQyXSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxMDogW2lvICAweDRm
ZmY4MDAwLTB4NGZmZjgwZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk5OTQ5XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxNDogW21lbSAweDQwMTQwMDAwLTB4NDAx
NGZmZmYgNjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTU3
XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxYzogW21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYg
NjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTY2XSBwY2kg
MDAwYzowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NzBdIHBjaSAwMDBjOjAx
OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDAwNjRdIHBjaSAwMDBjOjAxOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTM1XSBwY2lfYnVzIDAwMGM6MDA6
IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
MTM5XSBwY2kgMDAwYzowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAw
MGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24g
MTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTQzXSBwY2kg
MDAwYzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE0NF0gcGNpIDAwMGM6
MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
MTQ1XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE0Nl0gcGNpIDAw
MGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTQ3XSBwY2kgMDAwYzowMDow
MS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE0OF0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzAwMTUwXSBwY2kgMDAwYzowMTowMC4wOiBCQVIgNjogYXNzaWdu
ZWQgW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDAxNTJdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAzOiBhc3NpZ25l
ZCBbbWVtIDB4NDAxMDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDAxNTddIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAxOiBhc3NpZ25l
ZCBbbWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDAxNjJdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MDEwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDAxNjNdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDAxMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
MTY0XSBwY2kgMDAwYzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTY2XSBwY2kgMDAwYzowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTY3XSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDMwMDAwMDAwMDAwMC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE2OF0gcGNpX2J1cyAwMDBjOjAw
OiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2VzIGFyZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGluZyB3
aXRoIHBjaT1yZWFsbG9jClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
MTcwXSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNlIDQgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZm
ZmYgd2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE3MV0g
cGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA1IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2ZmZGZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE3
MV0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAxIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE3Ml0gcGNpX2J1cyAw
MDBjOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAyODZdIEFD
UEk6IFBDSSBSb290IEJyaWRnZSBbUENJMV0gKGRvbWFpbiAwMDBkIFtidXMgMDAtZmZdKQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDI5MV0gYWNwaSBQTlAwQTA4OjAx
OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRz
IE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
MzU2XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BD
SWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAwNDE1XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IE9TIG5vdyBjb250cm9s
cyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMDQxN10gYWNwaSBQTlAwQTA4OjAxOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgz
N2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmQy
NDliZjg3ZWE1OApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTA0OF0g
YWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGFyZWEgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZm
ZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMTA1NV0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAw
MDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMTExNV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBkOjAw
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMTE2XSBwY2lfYnVzIDAw
MGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRv
d10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDExMTddIHBjaV9idXMg
MDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMTE4
XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMTM1XSBwY2kgMDAwZDowMDowMC4wOiBb
MWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAxMTc1XSBwY2kgMDAwZDowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBl
IDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAxMTgyXSBwY2kgMDAwZDowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMTk5XSBwY2kgMDAwZDowMDowMS4wOiBz
dXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTE5
OV0gcGNpIDAwMGQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTI0OV0gcGNpIDAwMGQ6MDA6MDIu
MDogWzFkZWY6ZTEwMl0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMTI3MF0gcGNpIDAwMGQ6MDA6MDIuMDogc3VwcG9ydHMgRDEg
RDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDEyNzFdIHBjaSAwMDBk
OjAwOjAyLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDEzMThdIHBjaSAwMDBkOjAwOjAzLjA6IFsxZGVmOmUx
MDNdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDEzMzhdIHBjaSAwMDBkOjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMzM5XSBwY2kgMDAwZDowMDowMy4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAxMzg2XSBwY2kgMDAwZDowMDowNC4wOiBbMWRlZjplMTA0XSB0eXBlIDAx
IGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAx
NDA2XSBwY2kgMDAwZDowMDowNC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMTQwNl0gcGNpIDAwMGQ6MDA6MDQuMDogUE1FIyBzdXBwb3J0
ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwMTQ3NF0gYWNwaXBocDogU2xvdCBbMS0yXSByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxNDkwXSBwY2kgMDAwZDowMTowMC4wOiBbODA4NjowYTU0
XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAxNTAyXSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMzEwMDAw
LTB4NTAzMTNmZmYgNjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAxNTI2XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMzAwMDAwLTB4NTAz
MGZmZmYgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE1MzBd
IHBjaSAwMDBkOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE2ODRdIGFjcGlwaHA6IFNsb3QgWzJdIHJlZ2lzdGVy
ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE2OTldIHBjaSAwMDBk
OjAyOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE3MDldIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAw
eDEwOiBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDE3MzFdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDMwOiBb
bWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMTczNF0gcGNpIDAwMGQ6MDI6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQg
VGFncwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTg3Ml0gYWNwaXBo
cDogU2xvdCBbM10gcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMTg4N10gcGNpIDAwMGQ6MDM6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAw
eDAxMDgwMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTg5OF0gcGNp
IDAwMGQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHg1MDExMDAwMC0weDUwMTEzZmZmIDY0Yml0
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTkyMF0gcGNpIDAwMGQ6
MDM6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDEwMDAwMC0weDUwMTBmZmZmIHByZWZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxOTI0XSBwY2kgMDAwZDowMzowMC4w
OiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAyMDY2XSBhY3BpcGhwOiBTbG90IFs0XSByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMDgwXSBwY2kgMDAwZDowNDowMC4wOiBbODA4Njow
YTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAyMDkxXSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMDEw
MDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAyMTEyXSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMDAwMDAwLTB4
NTAwMGZmZmYgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIx
MTZdIHBjaSAwMDBkOjA0OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyMzhdIHBjaV9idXMgMDAwZDowMDogb24gTlVN
QSBub2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNDBdIHBj
aSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVz
IDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAyMjQyXSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMjQzXSBw
Y2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZm
XSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI0NF0gcGNpIDAwMGQ6MDA6MDIuMDogYnJp
ZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNDVdIHBjaSAwMDBkOjAw
OjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJl
Zl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNDZdIHBjaSAwMDBkOjAwOjAyLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9z
aXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAyMjQ3XSBwY2kgMDAwZDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgx
MDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMjI0OV0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRf
c2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMjI1MF0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGln
biAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNTFdIHBj
aSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVz
IDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAyMjUyXSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMjU0XSBw
Y2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZm
XSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI1N10gcGNpIDAwMGQ6MDA6MDEuMDogQkFS
IDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNThdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxNTog
YXNzaWduZWQgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI1OV0gcGNpIDAwMGQ6MDA6
MDIuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNjFdIHBjaSAwMDBkOjAwOjAyLjA6
IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI2Ml0gcGNp
IDAwMGQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA0MDAwMDAtMHg1MDVmZmZm
Zl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNjNdIHBjaSAwMDBk
OjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MjI2NF0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAt
MHg1MDdmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNjVd
IHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDYwMDAwMC0w
eDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMjI2Nl0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIy
NjddIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI2OF0gcGNp
IDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNjhdIHBjaSAwMDBkOjAwOjAy
LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI2OV0gcGNpIDAwMGQ6MDA6MDMuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDIyNzBdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMjI3MV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDIyNzJdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI3NF0g
cGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNzVdIHBjaSAwMDBkOjAw
OjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI3Nl0gcGNpIDAwMGQ6MDA6MDMuMDog
QkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDIyNzddIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMjI3N10gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDIyNzhdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI3
OV0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIyODBdIHBjaSAwMDBk
OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjI4MV0gcGNpIDAwMGQ6MDE6MDAu
MDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMjgyXSBwY2kgMDAwZDowMTowMC4w
OiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMjg3XSBwY2kgMDAwZDowMDowMS4w
OiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAyMjg5XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUw
MDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAyMjkwXSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDAw
MDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMjI5Ml0gcGNpIDAwMGQ6MDI6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFtt
ZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAyMjkzXSBwY2kgMDAwZDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21l
bSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAyMjk3XSBwY2kgMDAwZDowMDowMi4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMjk5XSBwY2kgMDAw
ZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzAwXSBwY2kgMDAwZDowMDow
Mi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjMwMl0g
cGNpIDAwMGQ6MDM6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDQwMDAwMC0weDUwNDBm
ZmZmIHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzAzXSBw
Y2kgMDAwZDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNDEwMDAwLTB4NTA0MTNm
ZmYgNjRiaXRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzA3XSBw
Y2kgMDAwZDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzA5XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAyMzEwXSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjMxMV0gcGNpIDAwMGQ6MDQ6MDAuMDogQkFS
IDY6IGFzc2lnbmVkIFttZW0gMHg1MDYwMDAwMC0weDUwNjBmZmZmIHByZWZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzEyXSBwY2kgMDAwZDowNDowMC4wOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDUwNjEwMDAwLTB4NTA2MTNmZmYgNjRiaXRdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzE3XSBwY2kgMDAwZDowMDowNC4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDRdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAyMzE4XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNjAwMDAw
LTB4NTA3ZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzE5
XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDYwMDAwMC0w
eDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMjMyMV0gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA0IFttZW0gMHg1MDAwMDAw
MC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDIzMjJdIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzQwMDAwMDAwMDAw
LTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDIzMjNdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NTAwMDAwMDAt
MHg1MDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIzMjNd
IHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAyMzI0XSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDEgW21lbSAweDUwMjAwMDAwLTB4NTAz
ZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyMzI1XSBwY2lf
YnVzIDAwMGQ6MDI6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjMy
Nl0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAxIFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjMyN10gcGNpX2J1cyAw
MDBkOjAzOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIzMjddIHBj
aV9idXMgMDAwZDowNDogcmVzb3VyY2UgMSBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDIzMjhdIHBjaV9idXMgMDAwZDow
NDogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNDM1XSBBQ1BJOiBQ
Q0kgUm9vdCBCcmlkZ2UgW1BDSTJdIChkb21haW4gMDAwMSBbYnVzIDAwLWZmXSkKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDI0NDBdIGFjcGkgUE5QMEEwODowMjogX09T
QzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kg
SFBYLVR5cGUzXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjUwNF0g
YWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90
cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMjU2M10gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FF
UiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDI1NjZdIGFjcGkgUE5QMEEwODowMjogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2JmZmYw
MDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZkMjQ5YmY4
N2VhNTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMxOTVdIGFjcGkg
UE5QMEEwODowMjogRUNBTSBhcmVhIFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZd
IHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDMyMDFdIGFjcGkgUE5QMEEwODowMjogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAw
LTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDMyNjFdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMTowMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzI2Ml0gcGNpX2J1cyAwMDAxOjAw
OiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjYzXSBwY2lfYnVzIDAwMDE6
MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZmZmYg
d2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzI2NF0gcGNp
X2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzI4MV0gcGNpIDAwMDE6MDA6MDAuMDogWzFkZWY6
ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMzMyM10gcGNpIDAwMDE6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBj
bGFzcyAweDA2MDQwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzMz
Nl0gcGNpIDAwMDE6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzM2M10gcGNpIDAwMDE6MDA6MDEuMDogc3VwcG9y
dHMgRDEgRDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMzNjNdIHBj
aSAwMDAxOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM0NTFdIGFjcGlwaHA6IFNsb3QgWzEtM10g
cmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTU0MV0g
cGNpX2J1cyAwMDAxOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwNTU0M10gcGNpIDAwMDE6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8g
IDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDU1NDRdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3
aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDU1NDVdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1NTQ3
XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg2MDAwMDAwMC0weDYw
MWZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTU0OF0gcGNp
IDAwMDE6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4Mzgw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzA1NTQ5XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTU1MF0g
cGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1NTUxXSBwY2kgMDAw
MTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTU1Ml0gcGNpIDAwMDE6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1NTUzXSBwY2kgMDAwMTowMDowMS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1
NTU1XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDYwMDAwMDAwLTB4
NjAxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1NTU2XSBw
Y2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM4MDAwMDAwMDAwMC0weDM4
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNTU1OF0gcGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA0IFttZW0gMHg2MDAwMDAwMC0w
eDZmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDU1NTldIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4
M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDU1NjBdIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMSBbbWVtIDB4NjAwMDAwMDAtMHg2
MDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDU1NjFdIHBj
aV9idXMgMDAwMTowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1
NjY3XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTNdIChkb21haW4gMDAwMCBbYnVzIDAwLWZm
XSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDU2NzJdIGFjcGkgUE5Q
MEEwODowMzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBT
ZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNTczNV0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBw
b3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNTc5NF0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBub3cg
Y29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDU3OTZdIGFjcGkgUE5QMEEwODowMzogTUNGRyBxdWlyazogRUNBTSBhdCBb
bWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAw
eGZmZmZkMjQ5YmY4N2VhNTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDY0MjRdIGFjcGkgUE5QMEEwODowMzogRUNBTSBhcmVhIFttZW0gMHgzZmZmZjAwMDAwMDAtMHgz
ZmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDY0MzFdIGFjcGkgUE5QMEEwODowMzogRUNBTSBhdCBbbWVtIDB4
M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY0OTFdIFBDSSBob3N0IGJyaWRnZSB0byBidXMg
MDAwMDowMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjQ5M10gcGNp
X2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZm
ZiB3aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2NDkzXSBw
Y2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzYzAwMDAwMDAwMDAtMHgz
ZmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwNjQ5NF0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjUxMV0gcGNpIDAwMDA6MDA6
MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjU1M10gcGNpIDAwMDA6MDA6MDEuMDogWzFkZWY6ZTEw
MV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNjU2Nl0gcGNpIDAwMDA6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjU5M10gcGNpIDAwMDA6MDA6
MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDY1OTNdIHBjaSAwMDAwOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNo
b3QKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY2ODBdIGFjcGlwaHA6
IFNsb3QgWzEtNF0gcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwODc3MF0gcGNpX2J1cyAwMDAwOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwODc3Ml0gcGNpIDAwMDA6MDA6MDEuMDogYnJpZGdl
IHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg3NzNdIHBjaSAwMDAwOjAwOjAx
LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0g
dG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg3NzRdIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXpl
IDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA4Nzc2XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg3
MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwODc3N10gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4M2MwMDAw
MDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA4Nzc4XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwODc3OV0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4
NzgwXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwODc4MV0gcGNpIDAw
MDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4NzgyXSBwY2kgMDAwMDowMDow
MS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA4Nzg0XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzA4Nzg1XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDNjMDAw
MDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwODc4OF0gcGNpX2J1cyAwMDAwOjAwOiByZXNvdXJjZSA0IFttZW0g
MHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDg3ODldIHBjaV9idXMgMDAwMDowMDogcmVzb3VyY2UgNSBbbWVtIDB4M2Mw
MDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDg3ODldIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMSBbbWVtIDB4
NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDg3OTBdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMiBbbWVtIDB4M2MwMDAwMDAwMDAw
LTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA4ODk0XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTRdIChkb21haW4gMDAw
MiBbYnVzIDAwLWZmXSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg4
OThdIGFjcGkgUE5QMEEwODowNDogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFT
UE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwODk2Ml0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBwbGF0Zm9ybSBk
b2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwOTAyMF0gYWNwaSBQTlAwQTA4OjA0OiBf
T1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDkwMjJdIGFjcGkgUE5QMEEwODowNDogTUNGRyBxdWly
azogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAw
MC1mZl0gd2l0aCAweGZmZmZkMjQ5YmY4N2VhNTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDk2NDVdIGFjcGkgUE5QMEEwODowNDogRUNBTSBhcmVhIFttZW0gMHgyM2Zm
ZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk2NTBdIGFjcGkgUE5QMEEwODowNDogRUNB
TSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk3MDldIFBDSSBob3N0IGJy
aWRnZSB0byBidXMgMDAwMjowMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwOTcxMV0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MDA4MDAw
MDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzA5NzEyXSBwY2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAw
MDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwOTcxMl0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBb
YnVzIDAwLWZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwOTcyOV0g
cGNpIDAwMDI6MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwOTc3MV0gcGNpIDAwMDI6MDA6MDEu
MDogWzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwOTgwNl0gcGNpIDAwMDI6MDA6MDEuMDogc3VwcG9ydHMgRDEg
RDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk4MDddIHBjaSAwMDAy
OjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk4NjldIHBjaSAwMDAyOjAwOjA1LjA6IFsxZGVmOmUx
MTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDk5MDRdIHBjaSAwMDAyOjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA5OTA1XSBwY2kgMDAwMjowMDowNS4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA5OTg4XSBhY3BpcGhwOiBTbG90IFsxLTVdIHJlZ2lzdGVyZWQKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIxMDBdIGFjcGlwaHA6IFNsb3QgWzMt
Ml0gcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDE5
Ml0gcGNpX2J1cyAwMDAyOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNDE5M10gcGNpIDAwMDI6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBb
aW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQxOTVdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAw
MV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTQxOTZdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBh
ZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0
MTk3XSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0g
dG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNDE5OF0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAx
MDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
NDE5OV0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgw
MDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMDFdIHBjaSAwMDAyOjAwOjAx
LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjAyXSBwY2kgMDAwMjowMDowMS4wOiBC
QVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMDNdIHBjaSAw
MDAyOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjA0XSBwY2kgMDAwMjow
MDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAzZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQy
MDVdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjA2XSBwY2kgMDAw
MjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMDddIHBjaSAwMDAyOjAwOjA1
LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjA4XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTQyMDldIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE0MjEwXSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTQyMTFdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjEyXSBwY2kg
MDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMTNdIHBjaSAwMDAyOjAw
OjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTQyMTVdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTQyMTZdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAw
MDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE0MjE4XSBwY2kgMDAwMjowMDowNS4wOiBQQ0kgYnJpZGdlIHRv
IFtidXMgMDJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjIwXSBw
Y2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZm
ZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjIxXSBwY2kgMDAw
MjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
NDIyM10gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJjZSA0IFttZW0gMHgwMDgwMDAwMC0weDBmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMjRd
IHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRm
ZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQy
MjVdIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMSBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZm
Zl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMjZdIHBjaV9idXMg
MDAwMjowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjI3XSBw
Y2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDEgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MjI4XSBwY2lfYnVzIDAwMDI6
MDI6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDMzNV0gQUNQSTog
UENJIFJvb3QgQnJpZGdlIFtQQ0k1XSAoZG9tYWluIDAwMDMgW2J1cyAwMC1mZl0pClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MzQwXSBhY3BpIFBOUDBBMDg6MDU6IF9P
U0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJ
IEhQWC1UeXBlM10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ0MDNd
IGFjcGkgUE5QMEEwODowNTogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhv
dHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTQ0NjJdIGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtB
RVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE0NDY0XSBhY3BpIFBOUDBBMDg6MDU6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDI3ZmZm
MDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmZDI0OWJm
ODdlYTU4ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1MDk1XSBhY3Bp
IFBOUDBBMDg6MDU6IEVDQU0gYXJlYSBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZm
XSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE1MTAwXSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAw
MC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE1MTUzXSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDM6MDAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTUxNTRdIHBjaV9idXMgMDAwMzow
MDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDEwMDAwMDAwLTB4MWZmZmZmZmYgd2luZG93XQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTE1NV0gcGNpX2J1cyAwMDAz
OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjdmZmRmZmZmZmZm
IHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTUxNTZdIHBj
aV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTUxNzNdIHBjaSAwMDAzOjAwOjAwLjA6IFsxZGVm
OmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTUyMTJdIHBjaSAwMDAzOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEg
Y2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTUy
NDhdIHBjaSAwMDAzOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE1MjQ5XSBwY2kgMDAwMzowMDowMS4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE1MzA3XSBwY2kgMDAwMzowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYw
NDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1MzMzXSBwY2kgMDAw
MzowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNTMzM10gcGNpIDAwMDM6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBE
MSBEM2hvdApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTQwMF0gYWNw
aXBocDogU2xvdCBbMS02XSByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE3NTIyXSBwY2kgMDAwMzowMjowMC4wOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNs
YXNzIDB4MDIwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NTM4
XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDEwMDIwMDAwLTB4MTAwM2ZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NTU1XSBwY2kgMDAwMzow
MjowMC4wOiByZWcgMHgxODogW2lvICAweDFmZmY4MDIwLTB4MWZmZjgwM2ZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NTYzXSBwY2kgMDAwMzowMjowMC4wOiByZWcg
MHgxYzogW21lbSAweDEwMDQ0MDAwLTB4MTAwNDdmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE3Njc0XSBwY2kgMDAwMzowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzcxMl0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgw
MDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzcxM10gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAw
MDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzczNF0gcGNpIDAwMDM6MDI6MDAu
MDogcmVnIDB4MTkwOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzczNV0gcGNpIDAwMDM6MDI6MDAu
MDogVkYobikgQkFSMyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJl
Zl0gKGNvbnRhaW5zIEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNzgzMV0gcGNpIDAwMDM6MDI6MDAuMDogOC4wMDAgR2IvcyBhdmFpbGFibGUg
UENJZSBiYW5kd2lkdGgsIGxpbWl0ZWQgYnkgNS4wIEdUL3MgUENJZSB4MiBsaW5rIGF0IDAwMDM6
MDA6MDUuMCAoY2FwYWJsZSBvZiAxNi4wMDAgR2IvcyB3aXRoIDUuMCBHVC9zIFBDSWUgeDQgbGlu
aykKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc4NzhdIHBjaSAwMDAz
OjAyOjAwLjE6IFs4MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAwMDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc4OTRdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAw
eDEwOiBbbWVtIDB4MTAwMDAwMDAtMHgxMDAxZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTc5MTBdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDE4OiBbaW8gIDB4
MWZmZjgwMDAtMHgxZmZmODAxZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTc5MTldIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDFjOiBbbWVtIDB4MTAwNDAwMDAtMHgx
MDA0M2ZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgwMjZdIHBj
aSAwMDAzOjAyOjAwLjE6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MDU5XSBwY2kgMDAwMzowMjowMC4x
OiByZWcgMHgxODQ6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MDYwXSBwY2kgMDAwMzowMjowMC4x
OiBWRihuKSBCQVIwIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVm
XSAoY29udGFpbnMgQkFSMCBmb3IgOCBWRnMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE4MDgxXSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxOTA6IFttZW0gMHgwMDAw
MDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE4MDgyXSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIzIHNwYWNlOiBbbWVt
IDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMyBmb3IgOCBW
RnMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjA1XSBwY2lfYnVz
IDAwMDM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4MjA3XSBwY2lfYnVzIDAwMDM6MDA6IG1heCBidXMgZGVwdGg6IDEgcGNpX3RyeV9u
dW06IDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyMTBdIHBjaSAw
MDAzOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjExXSBwY2kgMDAwMzow
MDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgy
MTJdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4
MTAzZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjEzXSBw
Y2kgMDAwMzowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgy
NDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTgyMTRdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjE1
XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyMTZdIHBjaSAw
MDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjE3XSBwY2kgMDAwMzowMDowNS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyMThdIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBi
cmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTgyMjBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAt
MHgxMDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyMjFd
IHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4
MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4MjI0XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEw
MjAwMDAwLTB4MTAyMWZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE4MjI3XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjIwMDAw
LTB4MTAyM2ZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjMw
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQwMDAwLTB4MTAy
NDNmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjMzXSBwY2kg
MDAwMzowMjowMC4wOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAw
MDIxZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxODIzOF0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAw
MjIwMDAwLTB4MjQwMDAwMjNmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE4MjQyXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMzogYXNzaWduZWQg
W21lbSAweDEwMjQ0MDAwLTB4MTAyNDdmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE4MjQ1XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgNzogYXNzaWduZWQgW21lbSAw
eDI0MDAwMDI0MDAwMC0weDI0MDAwMDI1ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODI1MF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDEwOiBh
c3NpZ25lZCBbbWVtIDB4MjQwMDAwMjYwMDAwLTB4MjQwMDAwMjdmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjU0XSBwY2kgMDAwMzowMjow
MC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjU1XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxODI1Nl0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxODI1Nl0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgy
NThdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyNTldIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTgyNjBdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjYyXSBwY2lfYnVzIDAwMDM6MDA6
IE5vLiAyIHRyeSB0byBhc3NpZ24gdW5hc3NpZ25lZCByZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTgyNjRdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4MjY1XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTgyNjZdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjY2XSBwY2kg
MDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyNjddIHBjaSAwMDAzOjAw
OjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTgyNjldIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTgyNzBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQw
MDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE4MjcyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4MjczXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
ODI3NF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgw
MDIwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODI3NV0gcGNpIDAw
MDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyNzZdIHBjaSAwMDAzOjAwOjA1
LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTgyNzddIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTgyNzldIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAw
MjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE4MjgwXSBwY2lfYnVzIDAwMDM6MDA6IEF1dG9tYXRpY2FsbHkgZW5h
YmxlZCBwY2kgcmVhbGxvYywgaWYgeW91IGhhdmUgcHJvYmxlbSwgdHJ5IGJvb3Rpbmcgd2l0aCBw
Y2k9cmVhbGxvYz1vZmYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgy
ODFdIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2UgNCBbbWVtIDB4MTAwMDAwMDAtMHgxZmZmZmZm
ZiB3aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MjgyXSBw
Y2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDUgW21lbSAweDI0MDAwMDAwMDAwMC0weDI3ZmZkZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4Mjgz
XSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDEgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4Mjg0XSBwY2lfYnVzIDAw
MDM6MDE6IHJlc291cmNlIDIgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODI4NV0gcGNp
X2J1cyAwMDAzOjAyOiByZXNvdXJjZSAxIFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODI4Nl0gcGNpX2J1cyAwMDAzOjAy
OiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgzNTNdIEFDUEk6IFBD
SSBSb290IEJyaWRnZSBbUENJNl0gKGRvbWFpbiAwMDA0IFtidXMgMDAtZmZdKQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODM1N10gYWNwaSBQTlAwQTA4OjA2OiBfT1ND
OiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBI
UFgtVHlwZTNdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4NDIzXSBh
Y3BpIFBOUDBBMDg6MDY6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3Rw
bHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4NDgyXSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVS
IFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
ODQ4NF0gYWNwaSBQTlAwQTA4OjA2OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAw
MDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmQyNDliZjg3
ZWE1OApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTExMF0gYWNwaSBQ
TlAwQTA4OjA2OiBFQ0FNIGFyZWEgW21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0g
cmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxOTExNV0gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAt
MHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxOTE2Ml0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA0OjAwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MTYzXSBwY2lfYnVzIDAwMDQ6MDA6
IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkxNjRdIHBjaV9idXMgMDAwNDow
MDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3
aW5kb3ddClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MTY1XSBwY2lf
YnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MTgyXSBwY2kgMDAwNDowMDowMC4wOiBbMWRlZjpl
MTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE5MjE5XSBwY2kgMDAwNDowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNs
YXNzIDB4MDYwNDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjQ2
XSBwY2kgMDAwNDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxOTI0Nl0gcGNpIDAwMDQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQg
ZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
OTI4OV0gcGNpIDAwMDQ6MDA6MDMuMDogWzFkZWY6ZTExM10gdHlwZSAwMSBjbGFzcyAweDA2MDQw
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTMxNV0gcGNpIDAwMDQ6
MDA6MDMuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTkzMTZdIHBjaSAwMDA0OjAwOjAzLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEg
RDNob3QKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzNjFdIHBjaSAw
MDA0OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzOTddIHBjaSAwMDA0OjAwOjA1LjA6IHN1
cHBvcnRzIEQxIEQyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5Mzk4
XSBwY2kgMDAwNDowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDcwXSBwY2kgMDAwNDowMTowMC4w
OiBbMWEwMzoxMTUwXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE5NTE3XSBwY2kgMDAwNDowMTowMC4wOiBlbmFibGluZyBFeHRl
bmRlZCBUYWdzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NTc0XSBw
Y2kgMDAwNDowMTowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxOTU3NF0gcGNpIDAwMDQ6MDE6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJv
bSBEMCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTk2NjldIHBjaV9idXMgMDAwNDowMjogZXh0ZW5kZWQgY29uZmlnIHNwYWNlIG5vdCBh
Y2Nlc3NpYmxlClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5Njg3XSBw
Y2kgMDAwNDowMjowMC4wOiBbMWEwMzoyMDAwXSB0eXBlIDAwIGNsYXNzIDB4MDMwMDAwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NzA1XSBwY2kgMDAwNDowMjowMC4w
OiByZWcgMHgxMDogW21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NzE1XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxNDog
W21lbSAweDIyMDAwMDAwLTB4MjIwMWZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE5NzI1XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxODogW2lvICAweDJmZmY4
MDAwLTB4MmZmZjgwN2ZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5
NzcwXSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgdG8gZWZpZmIKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk4MDVdIHBjaSAwMDA0OjAyOjAwLjA6IHN1
cHBvcnRzIEQxIEQyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5ODA2
XSBwY2kgMDAwNDowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90IEQz
Y29sZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTkxNl0gcGNpIDAw
MDQ6MDM6MDAuMDogWzE5MTI6MDAxNF0gdHlwZSAwMCBjbGFzcyAweDBjMDMzMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTkzN10gcGNpIDAwMDQ6MDM6MDAuMDogcmVn
IDB4MTA6IFttZW0gMHgyMjIwMDAwMC0weDIyMjAxZmZmIDY0Yml0XQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDA0NV0gcGNpIDAwMDQ6MDM6MDAuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjExNzldIHBjaV9idXMgMDAwNDowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjExODFdIHBjaSAwMDA0OjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1
cyAwMS0wMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjExODNdIHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRnZSB3
aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTg0XSBwY2kgMDAwNDowMDowMy4w
OiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRv
IFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTg1XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ug
d2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAx
MDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMTE4Nl0gcGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0w
eDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjExODddIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjExODhdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAw
MDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAw
MDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTkyXSBwY2kgMDAw
NDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE5M10gcGNpIDAwMDQ6MDA6
MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTk0
XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIz
MWZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE5NV0gcGNp
IDAwMDQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4Mjgw
MDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIxMTk2XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzIw
MDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
MTE5N10gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwNDAw
MDAwLTB4MjgwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzIxMTk4XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMTE5OV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjAw
XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwMF0gcGNpIDAwMDQ6
MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjAxXSBwY2kgMDAwNDowMDowNS4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwMl0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzIxMjA0XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMTIwNV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIx
MjA2XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwN10gcGNpIDAw
MDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjA4XSBwY2kgMDAwNDowMDow
My4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwOF0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzIxMjEwXSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTQ6IGFzc2ln
bmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyMTIxMF0gcGNpIDAwMDQ6MDE6MDAuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjEyMTJdIHBjaSAwMDA0OjAxOjAwLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIx
M10gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIx
ZmZmZmZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIxNl0gcGNp
IDAwMDQ6MDI6MDAuMDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIyMF0gcGNpIDAwMDQ6
MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDgwXQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIyMV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFS
IDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDA4MF0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjEyMjJdIHBjaSAwMDA0OjAxOjAwLjA6IFBDSSBicmlkZ2Ug
dG8gW2J1cyAwMl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyMjZd
IHBjaSAwMDA0OjAxOjAwLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZm
ZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyMzNdIHBjaSAw
MDA0OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMS0wMl0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjEyMzRdIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjEyMzVdIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjM3XSBwY2kgMDAwNDowMzowMC4wOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDIzMDAwMDAwLTB4MjMwMDFmZmYgNjRiaXRdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjQ3XSBwY2kgMDAwNDowMDowMy4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzIxMjQ4XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIzMDAwMDAw
LTB4MjMxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjQ5
XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDIwMDAwMC0w
eDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTI1MV0gcGNpIDAwMDQ6MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI1M10gcGNpIDAwMDQ6MDA6
MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI1NF0gcGNpIDAwMDQ6MDA6MDUuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyNTZdIHBjaV9i
dXMgMDAwNDowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25lZCwgdHJ5
IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTI1N10gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA0IFttZW0gMHgyMDAwMDAw
MC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjEyNThdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjgwMDAwMDAwMDAw
LTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjEyNTldIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMSBbbWVtIDB4MjAwMDAwMDAt
MHgyMmZmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyNjBd
IHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzIxMjYxXSBwY2lfYnVzIDAwMDQ6MDI6IHJlc291cmNlIDEgW21lbSAweDIwMDAwMDAwLTB4MjJm
ZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjYyXSBwY2lf
YnVzIDAwMDQ6MDM6IHJlc291cmNlIDEgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjYyXSBwY2lfYnVzIDAwMDQ6MDM6
IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI2M10gcGNpX2J1cyAw
MDA0OjA0OiByZXNvdXJjZSAxIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI2NF0gcGNpX2J1cyAwMDA0OjA0OiByZXNv
dXJjZSAyIFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyOTJdIEFDUEk6IFBDSSBSb290
IEJyaWRnZSBbUENJN10gKGRvbWFpbiAwMDA1IFtidXMgMDAtZmZdKQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI5Nl0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBz
dXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlw
ZTNdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMzYwXSBhY3BpIFBO
UDBBMDg6MDc6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNI
UENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzIxNDE5XSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVD
YXBhYmlsaXR5XQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTQyMV0g
YWNwaSBQTlAwQTA4OjA3OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAt
MHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmQyNDliZjg3ZWE1OApT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjA1Ml0gYWNwaSBQTlAwQTA4
OjA3OiBFQ0FNIGFyZWEgW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gcmVzZXJ2
ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
MjA1OF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMjExNV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA1OjAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMTE2XSBwY2lfYnVzIDAwMDU6MDA6IHJvb3Qg
YnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMC0weDNmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjIxMTddIHBjaV9idXMgMDAwNTowMDogcm9v
dCBidXMgcmVzb3VyY2UgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMTE4XSBwY2lfYnVzIDAw
MDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzIyMTM2XSBwY2kgMDAwNTowMDowMC4wOiBbMWRlZjplMTEwXSB0
eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIyMTc2XSBwY2kgMDAwNTowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMjEyXSBwY2kg
MDAwNTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMjIxM10gcGNpIDAwMDU6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjI3M10g
cGNpIDAwMDU6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjMwOV0gcGNpIDAwMDU6MDA6MDUu
MDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjIzMTBdIHBjaSAwMDA1OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjIzNjJdIHBjaSAwMDA1OjAw
OjA3LjA6IFsxZGVmOmUxMTddIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjIzOThdIHBjaSAwMDA1OjAwOjA3LjA6IHN1cHBvcnRz
IEQxIEQyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMzk4XSBwY2kg
MDAwNTowMDowNy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyNDc0XSBhY3BpcGhwOiBTbG90IFsxLTddIHJl
Z2lzdGVyZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY2ODJdIHBj
aV9idXMgMDAwNTowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjY2ODNdIHBjaSAwMDA1OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAw
eDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2Njg1XSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI2Njg2XSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0g
MHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2Fs
aWduIDEwMDAwMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjY4N10g
cGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFti
dXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjY2ODhdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAw
LTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY2ODld
IHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZm
ZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2Njg5XSBwY2kgMDAwNTowMDowNy4wOiBi
cmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAw
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjY5MF0gcGNpIDAwMDU6
MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBw
cmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjY5MV0gcGNpIDAwMDU6MDA6MDcuMDog
YnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwM10gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjY2OTVdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21l
bSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI2Njk2XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgy
YzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjY2OTddIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxNDogYXNz
aWduZWQgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI2Njk4XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVk
IFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY2OTldIHBjaSAwMDA1OjAwOjA3LjA6IEJB
UiAxNDogYXNzaWduZWQgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzAwXSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTU6
IGFzc2lnbmVkIFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0K
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MDFdIHBjaSAwMDA1OjAw
OjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzAyXSBwY2kgMDAwNTowMDowMS4wOiBCQVIg
MTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MDNdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI2NzAzXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjY3MDRdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzA1XSBw
Y2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MDddIHBjaSAwMDA1
OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzA4XSBwY2kgMDAwNTowMDowNy4wOiBC
QVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MDldIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI2NzEwXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjY3MTFdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzEx
XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MTJdIHBjaSAw
MDA1OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjY3MTRdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjY3MTVdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzE3XSBwY2kgMDAwNTowMDowNS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2
NzE5XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMjAwMDAwLTB4
MzAzZmZmZmZdClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzIwXSBw
Y2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDIwMDAwMC0weDJj
MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNjcyMl0gcGNpIDAwMDU6MDA6MDcuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjcyNF0gcGNpIDAwMDU6MDA6MDcu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjcyNV0gcGNpIDAwMDU6MDA6MDcuMDogICBi
cmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MjhdIHBjaV9idXMg
MDAwNTowMDogcmVzb3VyY2UgNCBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3aW5kb3ddClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzI5XSBwY2lfYnVzIDAwMDU6
MDA6IHJlc291cmNlIDUgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzI5XSBwY2lfYnVzIDAw
MDU6MDE6IHJlc291cmNlIDEgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NzMwXSBwY2lfYnVzIDAwMDU6MDE6IHJlc291
cmNlIDIgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjczMV0gcGNpX2J1cyAwMDA1OjAy
OiByZXNvdXJjZSAxIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjczMl0gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAy
IFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3MzNdIHBjaV9idXMgMDAwNTowMzogcmVz
b3VyY2UgMSBbbWVtIDB4MzA0MDAwMDAtMHgzMDVmZmZmZl0KU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjY3MzNdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMiBbbWVt
IDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5MjI1XSBpb21tdTogRGVmYXVsdCBkb21haW4gdHlw
ZTogVHJhbnNsYXRlZCAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjky
NThdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogVkdBIGRldmljZSBhZGRlZDogZGVjb2Rlcz1p
byttZW0sb3ducz1ub25lLGxvY2tzPW5vbmUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjkyNjBdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogYnJpZGdlIGNvbnRyb2wg
cG9zc2libGUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjkyNjJdIHBj
aSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogc2V0dGluZyBhcyBib290IGRldmljZSAoVkdBIGxlZ2Fj
eSByZXNvdXJjZXMgbm90IGF2YWlsYWJsZSkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjkyNjJdIHZnYWFyYjogbG9hZGVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI5MzQ0XSBFREFDIE1DOiBWZXI6IDMuMC4wClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NDcxXSBSZWdpc3RlcmVkIGVmaXZhcnMgb3BlcmF0aW9u
cwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDE0NV0gTmV0TGFiZWw6
IEluaXRpYWxpemluZwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDE0
Nl0gTmV0TGFiZWw6ICBkb21haW4gaGFzaCBzaXplID0gMTI4ClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMwMTQ2XSBOZXRMYWJlbDogIHByb3RvY29scyA9IFVOTEFCRUxF
RCBDSVBTT3Y0IENBTElQU08KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzAxNTZdIE5ldExhYmVsOiAgdW5sYWJlbGVkIHRyYWZmaWMgYWxsb3dlZCBieSBkZWZhdWx0ClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwMjgzXSBjbG9ja3NvdXJjZTog
U3dpdGNoZWQgdG8gY2xvY2tzb3VyY2UgYXJjaF9zeXNfY291bnRlcgpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczODgwOV0gVkZTOiBEaXNrIHF1b3RhcyBkcXVvdF82LjYu
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczODgyNl0gVkZTOiBEcXVv
dC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDUxMiAob3JkZXIgMCwgNDA5NiBieXRlcykKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzg5NDJdIEFwcEFybW9yOiBBcHBB
cm1vciBGaWxlc3lzdGVtIEVuYWJsZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mzg5NzVdIHBucDogUG5QIEFDUEkgaW5pdApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczOTYwNF0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4
M2JmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NjA1XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzZmZm
ZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2MDddIHN5c3RlbSAwMDowMDog
W21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSBy
ZXNlcnZlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTYwOF0gc3lz
dGVtIDAwOjAwOiBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmIHdpbmRvd10gY291
bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM5NjA5XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmYg
d2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mzk2MTBdIHN5c3RlbSAwMDowMDogW21lbSAweDJmZmZmMDAwMDAwMC0weDJm
ZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTYxMV0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MzNmZmYw
MDAwMDAwLTB4MzNmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NjEyXSBzeXN0ZW0gMDA6MDA6IFtt
ZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVz
ZXJ2ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2MTZdIHN5c3Rl
bSAwMDowMDogUGx1ZyBhbmQgUGxheSBBQ1BJIGRldmljZSwgSURzIFBOUDBjMDIgKGFjdGl2ZSkK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk5MTBdIHBucDogUG5QIEFD
UEk6IGZvdW5kIDEgZGV2aWNlcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0MDg2NV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAyClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwOTMyXSBJUCBpZGVudHMgaGFzaCB0YWJsZSBlbnRy
aWVzOiAyNjIxNDQgKG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNTI1XSB0Y3BfbGlzdGVuX3BvcnRhZGRyX2hh
c2ggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDcsIDUyNDI4OCBieXRlcywgbGlu
ZWFyKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjgxNF0gVENQIGVz
dGFibGlzaGVkIGhhc2ggdGFibGUgZW50cmllczogNTI0Mjg4IChvcmRlcjogMTAsIDQxOTQzMDQg
Ynl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM4
OTZdIFRDUCBiaW5kIGhhc2ggdGFibGUgZW50cmllczogNjU1MzYgKG9yZGVyOiA4LCAxMDQ4NTc2
IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ0
MTU0XSBUQ1A6IEhhc2ggdGFibGVzIGNvbmZpZ3VyZWQgKGVzdGFibGlzaGVkIDUyNDI4OCBiaW5k
IDY1NTM2KQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NDIzOF0gVURQ
IGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5l
YXIpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ0NjQyXSBVRFAtTGl0
ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGlu
ZWFyKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTQxM10gTkVUOiBS
ZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1NDIwXSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDQ0ClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NDI3XSBwY2kgMDAwYzowMTowMC4w
OiBDTFMgbWlzbWF0Y2ggKDY0ICE9IDMyKSwgdXNpbmcgNjQgYnl0ZXMKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU1MDNdIFRyeWluZyB0byB1bnBhY2sgcm9vdGZzIGlt
YWdlIGFzIGluaXRyYW1mcy4uLgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE0NjM2OV0gRnJlZWluZyBpbml0cmQgbWVtb3J5OiAzNzE2NEsKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNDc5OTddIGh3IHBlcmZldmVudHM6IGVuYWJsZWQgd2l0aCBh
cm12OF9wbXV2M18wIFBNVSBkcml2ZXIsIDcgY291bnRlcnMgYXZhaWxhYmxlClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ4MDg0XSBrdm0gWzFdOiBJUEEgU2l6ZSBMaW1p
dDogNDggYml0cwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0ODEwNl0g
a3ZtIFsxXTogR0lDdjM6IG5vIEdJQ1YgcmVzb3VyY2UgZW50cnkKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNDgxMDddIGt2bSBbMV06IGRpc2FibGluZyBHSUN2MiBlbXVs
YXRpb24KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDgxMThdIGt2bSBb
MV06IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZSBlbmFibGVkClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ4NDYyXSBrdm0gWzFdOiB2Z2ljIGludGVycnVw
dCBJUlE5ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ4ODY5XSBrdm0g
WzFdOiBWSEUgbW9kZSBpbml0aWFsaXplZCBzdWNjZXNzZnVsbHkKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNDk3MDVdIEluaXRpYWxpc2Ugc3lzdGVtIHRydXN0ZWQga2V5
cmluZ3MKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDk3MTJdIEtleSB0
eXBlIGJsYWNrbGlzdCByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTQ5NzQ5XSB3b3JraW5nc2V0OiB0aW1lc3RhbXBfYml0cz00MiBtYXhfb3JkZXI9MjQg
YnVja2V0X29yZGVyPTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTA0
MTFdIHpidWQ6IGxvYWRlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1
MDUyOV0gaW50ZWdyaXR5OiBQbGF0Zm9ybSBLZXlyaW5nIGluaXRpYWxpemVkClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUwNTMxXSBLZXkgdHlwZSBhc3ltbWV0cmljIHJl
Z2lzdGVyZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTA1MzJdIEFz
eW1tZXRyaWMga2V5IHBhcnNlciAneDUwOScgcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE1MDUzOF0gQmxvY2sgbGF5ZXIgU0NTSSBnZW5lcmljIChic2cp
IGRyaXZlciB2ZXJzaW9uIDAuNCBsb2FkZWQgKG1ham9yIDI0OSkKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNTA1NzhdIGlvIHNjaGVkdWxlciBtcS1kZWFkbGluZSByZWdp
c3RlcmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUxNjAxXSBzaHBj
aHA6IFN0YW5kYXJkIEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjQK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTE3MTFdIGVmaWZiOiBwcm9i
aW5nIGZvciBlZmlmYgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MTcz
Ml0gZWZpZmI6IGZyYW1lYnVmZmVyIGF0IDB4MjAwMDAwMDAsIHVzaW5nIDE4NzZrLCB0b3RhbCAx
ODc1awpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MTczM10gZWZpZmI6
IG1vZGUgaXMgODAweDYwMHgzMiwgbGluZWxlbmd0aD0zMjAwLCBwYWdlcz0xClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUxNzMzXSBlZmlmYjogc2Nyb2xsaW5nOiByZWRy
YXcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTE3MzRdIGVmaWZiOiBU
cnVlY29sb3I6IHNpemU9ODo4Ojg6OCwgc2hpZnQ9MjQ6MTY6ODowClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMTUxODA2XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3Vy
IGZyYW1lIGJ1ZmZlciBkZXZpY2UgMTAweDM3ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTU3NjgzXSBmYjA6IEVGSSBWR0EgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1ODA0Nl0gaW5wdXQ6IFBvd2VyIEJ1dHRv
biBhcyAvZGV2aWNlcy9MTlhTWVNUTTowMC9MTlhTWUJVUzowMC9QTlAwQzBDOjAwL2lucHV0L2lu
cHV0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1ODA2MV0gQUNQSTog
UG93ZXIgQnV0dG9uIFtQV1JCXQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE2MjUzMF0gQUNQSSBHVERUOiBmb3VuZCAxIFNCU0EgZ2VuZXJpYyBXYXRjaGRvZyhzKS4KU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjMwNDldIFNlcmlhbDogODI1MC8x
NjU1MCBkcml2ZXIsIDQgcG9ydHMsIElSUSBzaGFyaW5nIGVuYWJsZWQKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjMzMTNdIFNlcmlhbDogQU1CQSBkcml2ZXIKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjMzNDddIG1zbV9zZXJpYWw6IGRyaXZl
ciBpbml0aWFsaXplZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MzM5
NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjM0MThdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjAuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZm
ZikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjM1ODRdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjM2MjBdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjM2ODRdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjQwNTldIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjEuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTY0MDcxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGlhcyA0
OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMTY0MTg1XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1
dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMTY0MjIzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86
IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTY0Mjg3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTY0NjA3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IG9wdGlv
biBtYXNrIDB4MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDYxOF0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChm
ZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE2NDY2NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYg
ZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2
NDc3NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50
cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDgx
MV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmll
cyBmb3IgcHJpcQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NTE0NV0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjUxNTVdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjMuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjUxOTldIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjUzMTFdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjUzNDZdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjU2NTBdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjQuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTY1NjU4XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGlhcyA0OC1i
aXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMTY1NzI2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86
IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTY1NzY5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFs
bG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTY1ODc4XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9j
YXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTY2MTgzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IG9wdGlvbiBt
YXNrIDB4MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjE5Ml0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0
dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2
NjI1Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50
cmllcyBmb3IgY21kcQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjMw
NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmll
cyBmb3IgZXZ0cQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjQxNF0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBm
b3IgcHJpcQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjcxOV0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjY3MjldIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjYuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjY3NjhdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjY4MzJdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjY4NzVdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjcyNjddIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjcuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTY3Mjc3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGlhcyA0OC1iaXQs
IG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTY3MzEzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTY3Mzc2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9j
YXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTY3NDE3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRl
ZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTcxNzQ4XSBtb3VzZWRldjogUFMvMiBtb3VzZSBkZXZpY2UgY29tbW9uIGZvciBhbGwg
bWljZQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MjY4OF0gcnRjLWVm
aSBydGMtZWZpLjA6IHJlZ2lzdGVyZWQgYXMgcnRjMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjE3MzE1Ml0gcnRjLWVmaSBydGMtZWZpLjA6IHNldHRpbmcgc3lzdGVtIGNs
b2NrIHRvIDIwMjEtMDktMTNUMDY6NDI6NTcgVVRDICgxNjMxNTE1Mzc3KQpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MzcyM10gbGVkdHJpZy1jcHU6IHJlZ2lzdGVyZWQg
dG8gaW5kaWNhdGUgYWN0aXZpdHkgb24gQ1BVcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjE3Mzk4N10gU01DQ0M6IFNPQ19JRDogSUQgPSBqZXAxMDY6MGExNjowMDAxIFJl
dmlzaW9uID0gMHgwMDAwMDBhMQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE3NDI0OV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxMApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MjI4MV0gU2VnbWVudCBSb3V0aW5nIHdpdGggSVB2
NgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MjI5NF0gbWlwNjogTW9i
aWxlIElQdjYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODIyOTZdIE5F
VDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xODIzMjFdIG1wbHNfZ3NvOiBNUExTIEdTTyBzdXBwb3J0ClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgyMzcxXSByZWdpc3RlcmVkIHRhc2tzdGF0
cyB2ZXJzaW9uIDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODIzNzNd
IExvYWRpbmcgY29tcGlsZWQtaW4gWC41MDkgY2VydGlmaWNhdGVzClNlcCAxMyAxMjoxMzowNCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4NjkxXSBMb2FkZWQgWC41MDkgY2VydCAnRGViaWFuIFNl
Y3VyZSBCb290IENBOiA2Y2NlY2U3ZTRjNmMwZDFmNjE0OWYzZGQyN2RmY2M1Y2JiNDE5ZWExJwpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwODcwMV0gTG9hZGVkIFguNTA5
IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBTaWduZXIgMjAyMSAtIGxpbnV4OiA0YjZlZjVhYmNh
NjY5ODI1MTc4ZTA1MmM4NDY2N2NjYmMwNTMxZjhjJwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIwODg1NF0genN3YXA6IGxvYWRlZCB1c2luZyBwb29sIGx6by96YnVkClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4OTU3XSBLZXkgdHlwZSAuX2Zz
Y3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
ODk1N10gS2V5IHR5cGUgLmZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIwODk1OF0gS2V5IHR5cGUgZnNjcnlwdC1wcm92aXNpb25pbmcgcmVn
aXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwODk5MF0gQXBw
QXJtb3I6IEFwcEFybW9yIHNoYTEgcG9saWN5IGhhc2hpbmcgZW5hYmxlZApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxODAwN10gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5
IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjE4NjYzXSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdTVVNFIExpbnV4IEVudGVy
cHJpc2UgU2VjdXJlIEJvb3QgQ0E6IDNkNGQ0MGNmOTM4NTM5MDI0YjFjZmM1YTEyZGVkZmU4YjE3
ZTc1NWYnClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE4NjY0XSBpbnRl
Z3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTg4MjRdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNl
cnQgJ0Nhbm9uaWNhbCBMdGQuIE1hc3RlciBDZXJ0aWZpY2F0ZSBBdXRob3JpdHk6IGFkOTE5OTBi
YzIyYWIxZjUxNzA0OGMyM2I2NjU1YTI2OGUzNDVhNjMnClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjE4ODI1XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNh
dGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTg5ODFd
IGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0ZlZG9yYSBTZWN1cmUgQm9vdCBDQTogZmRl
MzI1OTljMmQ2MWRiMWJmNTgwNzMzNWQ3YjIwZTRjZDk2M2I0MicKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMTg5ODJdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0
aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIx
ODk5NF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJv
ZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1
MycKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTg5OTVdIGludGVncml0
eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjIxOTAwOF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAn
TWljcm9zb2Z0IENvcnBvcmF0aW9uIFVFRkkgQ0EgMjAxMTogMTNhZGJmNDMwOWJkODI3MDljOGNk
NTRmMzE2ZWQ1MjI5ODhhMWJkNCcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMTkwMDhdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxOTAyMF0gaW50ZWdyaXR5OiBM
b2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTog
YTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjMzNzZdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMjM2NzRdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEw
NgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMzc0NV0gcGNpZXBvcnQg
MDAwZDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMQpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjIyNDAxM10gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBRVI6IGVu
YWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjI0MDU5XSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI0MjkzXSBwY2llcG9ydCAwMDBk
OjAwOjAyLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yMjQzMzZdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjQ1
NjldIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNDYxMV0gcGNpZXBvcnQgMDAwZDow
MDowNC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIyNDg0Ml0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBRVI6IGVuYWJsZWQg
d2l0aCBJUlEgMTA3ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI0OTA1
XSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCA1ClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI1MTY1XSBwY2llcG9ydCAwMDAxOjAwOjAx
LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMjUyMzNdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjU0OTddIHBj
aWVwb3J0IDAwMDA6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwOQpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNTU2Nl0gcGNpZXBvcnQgMDAwMjowMDowMS4w
OiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIyNTgzOF0gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJ
UlEgMTEwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI1ODg2XSBwY2ll
cG9ydCAwMDAyOjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA4ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2MTMyXSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFF
UjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yMjYxOTVdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjY0NzhdIHBjaWVwb3J0
IDAwMDM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNjUzMV0gcGNpZXBvcnQgMDAwMzowMDowNS4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgMTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMjY3NzRdIHBjaWVwb3J0IDAwMDM6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEx
MQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNjgzOF0gcGNpZXBvcnQg
MDAwNDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTEKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMjcxMDldIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQUVSOiBl
bmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyNzE1NV0gcGNpZXBvcnQgMDAwNDowMDowMy4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTIK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjczOThdIHBjaWVwb3J0IDAw
MDQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjIyNzQ0NV0gcGNpZXBvcnQgMDAwNDowMDowNS4wOiBBZGRpbmcg
dG8gaW9tbXUgZ3JvdXAgMTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
Mjc2ODhdIHBjaWVwb3J0IDAwMDQ6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNzc1Nl0gcGNpZXBvcnQgMDAw
NDowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yMjgwMjBdIHBjaWVwb3J0IDAwMDU6MDA6MDEuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDE1ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI4
Mjk0XSBwY2llcG9ydCAwMDA1OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjgzNDRdIHBjaWVwb3J0IDAwMDU6
MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE2ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjI4NTk0XSBwY2llcG9ydCAwMDA1OjAwOjA1LjA6IEFFUjogZW5hYmxl
ZCB3aXRoIElSUSAxMTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjg2
NDRdIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE3ClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI4ODg4XSBwY2llcG9ydCAwMDA1OjAw
OjA3LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yMzAyNDZdIEZyZWVpbmcgdW51c2VkIGtlcm5lbCBtZW1vcnk6IDU0NDBL
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzk4ODc5XSBDaGVja2VkIFcr
WCBtYXBwaW5nczogcGFzc2VkLCBubyBXK1ggcGFnZXMgZm91bmQKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4zOTg4ODVdIFJ1biAvaW5pdCBhcyBpbml0IHByb2Nlc3MKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zOTg4ODZdICAgd2l0aCBhcmd1bWVu
dHM6ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzk4ODg3XSAgICAgL2lu
aXQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zOTg4ODddICAgd2l0aCBl
bnZpcm9ubWVudDoKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zOTg4ODhd
ICAgICBIT01FPS8KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zOTg4ODld
ICAgICBURVJNPWxpbnV4ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzk4
ODkwXSAgICAgQk9PVF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04LWFybTY0ClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcyMzQ2XSBwcHNfY29yZTogTGludXhQUFMg
QVBJIHZlci4gMSByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNDcyMzQ4XSBwcHNfY29yZTogU29mdHdhcmUgdmVyLiA1LjMuNiAtIENvcHlyaWdodCAyMDA1
LTIwMDcgUm9kb2xmbyBHaW9tZXR0aSA8Z2lvbWV0dGlAbGludXguaXQ+ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDczMzg1XSBQVFAgY2xvY2sgc3VwcG9ydCByZWdpc3Rl
cmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDc4NDcwXSBTQ1NJIHN1
YnN5c3RlbSBpbml0aWFsaXplZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjQ3OTYzMF0gaWdiOiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgRHJpdmVyClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDc5NjMxXSBpZ2I6IENvcHlyaWdo
dCAoYykgMjAwNy0yMDE0IEludGVsIENvcnBvcmF0aW9uLgpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjQ3OTcyMl0gaWdiIDAwMDM6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDE4ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDgxMTg3XSBB
Q1BJOiBidXMgdHlwZSBVU0IgcmVnaXN0ZXJlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjQ4MTIwNl0gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZl
ciB1c2JmcwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4MTIxMV0gdXNi
Y29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciBodWIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODEyMzZdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGRl
dmljZSBkcml2ZXIgdXNiClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg0
NjA3XSBudm1lIDAwMGQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE5ClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg0ODg0XSBudm1lIG52bWUwOiBwY2kgZnVu
Y3Rpb24gMDAwZDowMTowMC4wClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NDg0OTM4XSBudm1lIDAwMGQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg1MzE5XSBudm1lIG52bWUxOiBwY2kg
ZnVuY3Rpb24gMDAwZDowMjowMC4wClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNDg1MzY1XSBudm1lIDAwMGQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIxClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg1NTg5XSBudm1lIG52bWUyOiBw
Y2kgZnVuY3Rpb24gMDAwZDowMzowMC4wClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNDg1NjMyXSBudm1lIDAwMGQ6MDQ6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIy
ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg1ODU3XSBudm1lIG52bWUz
OiBwY2kgZnVuY3Rpb24gMDAwZDowNDowMC4wClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNDkxNTMxXSBtcHQzc2FzIHZlcnNpb24gMzUuMTAwLjAwLjAwIGxvYWRlZApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MTY1MF0gbXB0M3NhcyAwMDBjOjAx
OjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyMwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjQ5MTk5MV0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBBZGRpbmcgdG8gaW9t
bXUgZ3JvdXAgMjQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTIwMDVd
IG1wdDNzYXNfY20wOiA2MyBCSVQgUENJIEJVUyBETUEgQUREUkVTU0lORyBTVVBQT1JURUQsIHRv
dGFsIG1lbSAoNjUzMDQ4MDAga0IpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNDkyNTY3XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDkyNTcxXSB4aGNpX2hjZCAwMDA0
OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMQpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MjU4OV0geGhjaV9oY2QgMDAw
NDowMzowMC4wOiBaZXJvaW5nIDY0Yml0IGJhc2UgcmVnaXN0ZXJzLCBleHBlY3RpbmcgZmF1bHQK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTM5MjldIHJhbmRvbTogZmFz
dCBpbml0IGRvbmUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTQ1ODJd
IG52bWUgbnZtZTA6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTU4NDNdIG52bWUgbnZtZTE6IDgwLzAvMCBkZWZh
dWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS40OTYzNDFdIG52bWUgbnZtZTM6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTY5ODNdIG52bWUgbnZtZTI6IDgw
LzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS40OTc4NDBdIGFzdCAwMDA0OjAyOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91
cCAxNApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5Nzg1NF0gIG52bWUw
bjE6ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk5MzkyXSAgbnZtZTFu
MTogcDEgcDIgcDMgcDQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTk1
MThdICBudm1lM24xOiBwMSBwMiBwMyBwNCBwNSBwNgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjY0OTI0Nl0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBoY2MgcGFyYW1zIDB4
MDE0MDUxY2YgaGNpIHZlcnNpb24gMHgxMDAgcXVpcmtzIDB4MDAwMDAwMTEwMDAwMDQxMApTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTU5Ml0gdXNiIHVzYjE6IE5ldyBV
U0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0xZDZiLCBpZFByb2R1Y3Q9MDAwMiwgYmNkRGV2aWNl
PSA1LjEwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjQ5NTkzXSB1c2Ig
dXNiMTogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVt
YmVyPTEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NDk1OTRdIHVzYiB1
c2IxOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29udHJvbGxlcgpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjY0OTU5NV0gdXNiIHVzYjE6IE1hbnVmYWN0dXJlcjogTGludXggNS4x
MC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42NDk1OTZdIHVzYiB1c2IxOiBTZXJpYWxOdW1iZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTY3NV0gY2hlY2tpbmcgZ2VuZXJpYyAoMjAw
MDAwMDAgMWQ1MDAwKSB2cyBodyAoMjAwMDAwMDAgMjAwMDAwMCkKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS42NDk2NzZdIGZiMDogc3dpdGNoaW5nIHRvIGFzdGRybWZiIGZy
b20gRUZJIFZHQQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTczMl0g
Q29uc29sZTogc3dpdGNoaW5nIHRvIGNvbG91ciBkdW1teSBkZXZpY2UgODB4MjUKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NDk3ODBdIGFzdCAwMDA0OjAyOjAwLjA6IFtk
cm1dIHBsYXRmb3JtIGhhcyBubyBJTyBzcGFjZSwgdHJ5aW5nIE1NSU8KU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NDk3ODZdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIFAy
QSBicmlkZ2UgZGlzYWJsZWQsIHVzaW5nIGRlZmF1bHQgY29uZmlndXJhdGlvbgpTZXAgMTMgMTI6
MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTc4N10gYXN0IDAwMDQ6MDI6MDAuMDogW2Ry
bV0gQVNUIDI1MDAgZGV0ZWN0ZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42NDk3OTFdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIEFuYWxvZyBWR0Egb25seQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTc5Ml0gYXN0IDAwMDQ6MDI6MDAuMDog
W2RybV0gZHJhbSBNQ0xLPTgwMCBNaHogdHlwZT0xIGJ1c193aWR0aD0xNgpTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTg0Ml0gW1RUTV0gWm9uZSAga2VybmVsOiBBdmFp
bGFibGUgZ3JhcGhpY3MgbWVtb3J5OiAzMjY1MjQwMCBLaUIKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS42NDk4NDNdIFtUVE1dIFpvbmUgICBkbWEzMjogQXZhaWxhYmxlIGdy
YXBoaWNzIG1lbW9yeTogMjA5NzE1MiBLaUIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS42NDk4NDRdIFtUVE1dIEluaXRpYWxpemluZyBwb29sIGFsbG9jYXRvcgpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY0OTg0N10gW1RUTV0gSW5pdGlhbGl6aW5n
IERNQSBwb29sIGFsbG9jYXRvcgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjY1MDAxOF0gaHViIDEtMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS42NTAwNTBdIFtkcm1dIEluaXRpYWxpemVkIGFzdCAwLjEuMCAyMDEy
MDIyOCBmb3IgMDAwNDowMjowMC4wIG9uIG1pbm9yIDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS43MzQzMDRdIGh1YiAxLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzM0NDY2XSB4aGNpX2hjZCAwMDA0OjAz
OjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNzM0NDY5XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lz
dGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjczNDQ3MV0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBIb3N0IHN1cHBvcnRzIFVT
QiAzLjAgU3VwZXJTcGVlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjcz
NDUwMF0gdXNiIHVzYjI6IFdlIGRvbid0IGtub3cgdGhlIGFsZ29yaXRobXMgZm9yIExQTSBmb3Ig
dGhpcyBob3N0LCBkaXNhYmxpbmcgTFBNLgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAxLjczNDUxMl0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0x
ZDZiLCBpZFByb2R1Y3Q9MDAwMywgYmNkRGV2aWNlPSA1LjEwClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNzM0NTEzXSB1c2IgdXNiMjogTmV3IFVTQiBkZXZpY2Ugc3RyaW5n
czogTWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS43MzQ1MTRdIHVzYiB1c2IyOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29u
dHJvbGxlcgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjczNDUxNV0gdXNi
IHVzYjI6IE1hbnVmYWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43MzQ1MTZdIHVzYiB1c2IyOiBTZXJpYWxO
dW1iZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjczNDYzOF0gaHViIDItMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS43MzQ2NDhdIGh1YiAyLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzY2OTQwXSBDb25zb2xlOiBzd2l0
Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMjQweDY3ClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzY2OTUxXSBpZ2IgMDAwMzowMjowMC4wOiBhZGRlZCBQ
SEMgb24gZXRoMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjc2Njk1Ml0g
aWdiIDAwMDM6MDI6MDAuMDogSW50ZWwoUikgR2lnYWJpdCBFdGhlcm5ldCBOZXR3b3JrIENvbm5l
Y3Rpb24KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43NjY5NTNdIGlnYiAw
MDAzOjAyOjAwLjA6IGV0aDA6IChQQ0llOjUuMEdiL3M6V2lkdGggeDIpIGQ4OjVlOmQzOjAyOmIx
OjkzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzY3MDM1XSBpZ2IgMDAw
MzowMjowMC4wOiBldGgwOiBQQkEgTm86IDEwNjMwMC0wMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS43NjcwMzZdIGlnYiAwMDAzOjAyOjAwLjA6IFVzaW5nIE1TSS1YIGlu
dGVycnVwdHMuIDggcnggcXVldWUocyksIDggdHggcXVldWUocykKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS43NjcxMTddIGlnYiAwMDAzOjAyOjAwLjE6IEFkZGluZyB0byBp
b21tdSBncm91cCAyNQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjc2OTcw
MF0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gZmIwOiBhc3Rkcm1mYiBmcmFtZSBidWZmZXIgZGV2
aWNlClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQ4Mjg3XSBtcHQzc2Fz
X2NtMDogQ3VycmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdl
IHNpemUgdG8gNGsKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDgzMDVd
IG1wdDNzYXNfY20wOiBNU0ktWCB2ZWN0b3JzIHN1cHBvcnRlZDogOTYKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDgzMDZdIAkgbm8gb2YgY29yZXM6IDgwLCBtYXhfbXNp
eF92ZWN0b3JzOiAtMQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg0ODMw
OF0gbXB0M3Nhc19jbTA6ICAwIDgwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODQ5NjI0XSBtcHQzc2FzX2NtMDogSGlnaCBJT1BzIHF1ZXVlcyA6IGRpc2FibGVkClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQ5NjI2XSBtcHQzc2FzMC1tc2l4MDog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDk2MjZdIG1wdDNzYXMwLW1zaXgxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2
NgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg0OTYyN10gbXB0M3NhczAt
bXNpeDI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY3ClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODQ5NjI4XSBtcHQzc2FzMC1tc2l4MzogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0NjgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MjhdIG1w
dDNzYXMwLW1zaXg0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2OQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg0OTYyOV0gbXB0M3NhczAtbXNpeDU6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNDcwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQ5
NjI5XSBtcHQzc2FzMC1tc2l4NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzEKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzBdIG1wdDNzYXMwLW1zaXg3OiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDQ3MgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjg0OTYzMV0gbXB0M3NhczAtbXNpeDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDczClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQ5NjMyXSBtcHQzc2FzMC1tc2l4
OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDk2MzJdIG1wdDNzYXMwLW1zaXgxMDogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0NzUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzNdIG1wdDNz
YXMwLW1zaXgxMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzYKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzNdIG1wdDNzYXMwLW1zaXgxMjogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0NzcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2
MzRdIG1wdDNzYXMwLW1zaXgxMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzgKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzRdIG1wdDNzYXMwLW1zaXgxNDogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0NzkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDk2MzVdIG1wdDNzYXMwLW1zaXgxNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODAK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzZdIG1wdDNzYXMwLW1z
aXgxNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDk2MzZdIG1wdDNzYXMwLW1zaXgxNzogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0ODIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzddIG1w
dDNzYXMwLW1zaXgxODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODMKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzddIG1wdDNzYXMwLW1zaXgxOTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0ODQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDk2MzhdIG1wdDNzYXMwLW1zaXgyMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODUKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2MzldIG1wdDNzYXMwLW1zaXgyMTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDk2MzldIG1wdDNzYXMwLW1zaXgyMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
ODcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDBdIG1wdDNzYXMw
LW1zaXgyMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODgKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDk2NDBdIG1wdDNzYXMwLW1zaXgyNDogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0ODkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDFd
IG1wdDNzYXMwLW1zaXgyNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTAKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDJdIG1wdDNzYXMwLW1zaXgyNjogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA0OTEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDk2NDJdIG1wdDNzYXMwLW1zaXgyNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTIKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDNdIG1wdDNzYXMwLW1zaXgy
ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDk2NDNdIG1wdDNzYXMwLW1zaXgyOTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0OTQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDRdIG1wdDNz
YXMwLW1zaXgzMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTUKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDRdIG1wdDNzYXMwLW1zaXgzMTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0OTYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2
NDVdIG1wdDNzYXMwLW1zaXgzMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTcKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDVdIG1wdDNzYXMwLW1zaXgzMzogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0OTgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDk2NDZdIG1wdDNzYXMwLW1zaXgzNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTkK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDZdIG1wdDNzYXMwLW1z
aXgzNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDk2NDddIG1wdDNzYXMwLW1zaXgzNjogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDhdIG1w
dDNzYXMwLW1zaXgzNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDIKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDhdIG1wdDNzYXMwLW1zaXgzODogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MDMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDk2NDldIG1wdDNzYXMwLW1zaXgzOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDQKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NDldIG1wdDNzYXMwLW1zaXg0MDog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDk2NTBdIG1wdDNzYXMwLW1zaXg0MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MDYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTFdIG1wdDNzYXMw
LW1zaXg0MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDcKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDk2NTFdIG1wdDNzYXMwLW1zaXg0MzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MDgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTJd
IG1wdDNzYXMwLW1zaXg0NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDkKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTJdIG1wdDNzYXMwLW1zaXg0NTogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDk2NTNdIG1wdDNzYXMwLW1zaXg0NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTEKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTNdIG1wdDNzYXMwLW1zaXg0
NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDk2NTRdIG1wdDNzYXMwLW1zaXg0ODogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MTMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTRdIG1wdDNz
YXMwLW1zaXg0OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTQKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTVdIG1wdDNzYXMwLW1zaXg1MDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MTUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2
NTZdIG1wdDNzYXMwLW1zaXg1MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTYKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTZdIG1wdDNzYXMwLW1zaXg1MjogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MTcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDk2NTddIG1wdDNzYXMwLW1zaXg1MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTgK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTddIG1wdDNzYXMwLW1z
aXg1NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDk2NThdIG1wdDNzYXMwLW1zaXg1NTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MjAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTldIG1w
dDNzYXMwLW1zaXg1NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjEKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NTldIG1wdDNzYXMwLW1zaXg1NzogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MjIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDk2NjBdIG1wdDNzYXMwLW1zaXg1ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjMKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjFdIG1wdDNzYXMwLW1zaXg1OTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDk2NjFdIG1wdDNzYXMwLW1zaXg2MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MjUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjJdIG1wdDNzYXMw
LW1zaXg2MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjYKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDk2NjJdIG1wdDNzYXMwLW1zaXg2MjogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MjcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjNd
IG1wdDNzYXMwLW1zaXg2MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjgKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjRdIG1wdDNzYXMwLW1zaXg2NDogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MjkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDk2NjRdIG1wdDNzYXMwLW1zaXg2NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzAKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjVdIG1wdDNzYXMwLW1zaXg2
NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDk2NjZdIG1wdDNzYXMwLW1zaXg2NzogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MzIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjZdIG1wdDNz
YXMwLW1zaXg2ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzMKU2VwIDEzIDEyOjEzOjA0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjddIG1wdDNzYXMwLW1zaXg2OTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MzQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2
NjddIG1wdDNzYXMwLW1zaXg3MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzUKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjhdIG1wdDNzYXMwLW1zaXg3MTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MzYKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDk2NjldIG1wdDNzYXMwLW1zaXg3MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzcK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NjldIG1wdDNzYXMwLW1z
aXg3MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzgKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDk2NzBdIG1wdDNzYXMwLW1zaXg3NDogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MzkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NzBdIG1w
dDNzYXMwLW1zaXg3NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDAKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NzFdIG1wdDNzYXMwLW1zaXg3NjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1NDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDk2NzJdIG1wdDNzYXMwLW1zaXg3NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDIKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NzJdIG1wdDNzYXMwLW1zaXg3ODog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDMKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDk2NzNdIG1wdDNzYXMwLW1zaXg3OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
NDQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2NzVdIG1wdDNzYXNf
Y20wOiBpb21lbSgweDAwMDAwMDAwNDAxNDAwMDApLCBtYXBwZWQoMHgoX19fX3B0cnZhbF9fX18p
KSwgc2l6ZSg2NTUzNikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDk2
NzVdIG1wdDNzYXNfY20wOiBpb3BvcnQoMHgwMDAwMDAwMDAwMDAwMDAwKSwgc2l6ZSgwKQpTZXAg
MTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg2NDg0OV0gaWdiIDAwMDM6MDI6MDAu
MTogYWRkZWQgUEhDIG9uIGV0aDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NjQ4NTFdIGlnYiAwMDAzOjAyOjAwLjE6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0
d29yayBDb25uZWN0aW9uClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODY0
ODUyXSBpZ2IgMDAwMzowMjowMC4xOiBldGgxOiAoUENJZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1
ZTpkMzowMjpiMTo5NApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg2NDkz
M10gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogUEJBIE5vOiAxMDYzMDAtMDAwClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODY0OTM0XSBpZ2IgMDAwMzowMjowMC4xOiBVc2lu
ZyBNU0ktWCBpbnRlcnJ1cHRzLiA4IHJ4IHF1ZXVlKHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAxMyAx
MjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODY1ODcxXSBpZ2IgMDAwMzowMjowMC4xIGVu
bzI6IHJlbmFtZWQgZnJvbSBldGgxClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuOTAyNDY4XSBpZ2IgMDAwMzowMjowMC4wIGVubzE6IHJlbmFtZWQgZnJvbSBldGgwClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTA3Nzc2XSBtcHQzc2FzX2NtMDogQ3Vy
cmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8g
NGsKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MDc3NzldIG1wdDNzYXNf
Y20wOiBzZW5kaW5nIGRpYWcgcmVzZXQgISEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMi4wMjYyNzddIHVzYiAxLTE6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmljZSBudW1iZXIg
MiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAyLjIy
MjU2Ml0gdXNiIDEtMTogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTQxM2MsIGlkUHJv
ZHVjdD0zMDFhLCBiY2REZXZpY2U9IDEuMDAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMi4yMjI1NjNdIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0xLCBQ
cm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDIuMjIyNTY0XSB1c2IgMS0xOiBQcm9kdWN0OiBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1v
dXNlClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjIyNTY1XSB1c2IgMS0x
OiBNYW51ZmFjdHVyZXI6IFBpeEFydApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAyLjIzMTU2NV0gaGlkOiByYXcgSElEIGV2ZW50cyBkcml2ZXIgKEMpIEppcmkgS29zaW5hClNl
cCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjQzNjI0XSB1c2Jjb3JlOiByZWdp
c3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVzYmhpZApTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAyLjI0MzYyNV0gdXNiaGlkOiBVU0IgSElEIGNvcmUgZHJpdmVyClNlcCAx
MyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjQ0ODc0XSBpbnB1dDogUGl4QXJ0IERl
bGwgTVMxMTYgVVNCIE9wdGljYWwgTW91c2UgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAw
OjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0xLzEtMToxLjAvMDAwMzo0MTNDOjMwMUEuMDAwMS9p
bnB1dC9pbnB1dDEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yNDQ5MjRd
IGhpZC1nZW5lcmljIDAwMDM6NDEzQzozMDFBLjAwMDE6IGlucHV0LGhpZHJhdzA6IFVTQiBISUQg
djEuMTEgTW91c2UgW1BpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlXSBvbiB1c2It
MDAwNDowMzowMC4wLTEvaW5wdXQwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDIuMzMwOTM4XSB1c2IgMi0zOiBuZXcgU3VwZXJTcGVlZCBHZW4gMSBVU0IgZGV2aWNlIG51bWJl
ciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIu
MzU5NDg5XSB1c2IgMi0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywgaWRQ
cm9kdWN0PTA2MjAsIGJjZERldmljZT05My4xMApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAyLjM1OTQ5MF0gdXNiIDItMzogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTEs
IFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMi4zNTk0OTFdIHVzYiAyLTM6IFByb2R1Y3Q6IFVTQjMuMSBIdWIKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zNTk0OTJdIHVzYiAyLTM6IE1hbnVmYWN0dXJlcjog
R2VuZXN5c0xvZ2ljClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzYyNTA2
XSBodWIgMi0zOjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5l
bDogWyAgICAyLjM2MzExOF0gaHViIDItMzoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi40OTAyNzBdIHVzYiAxLTI6IG5ldyBsb3ctc3Bl
ZWQgVVNCIGRldmljZSBudW1iZXIgMyB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTI6MTM6MDQgZGVi
aWFuIGtlcm5lbDogWyAgICAyLjcwMTU1NF0gdXNiIDEtMjogTmV3IFVTQiBkZXZpY2UgZm91bmQs
IGlkVmVuZG9yPTA0ZDksIGlkUHJvZHVjdD0xMjAzLCBiY2REZXZpY2U9IDIuODAKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi43MDE1NTVdIHVzYiAxLTI6IE5ldyBVU0IgZGV2
aWNlIHN0cmluZ3M6IE1mcj0wLCBQcm9kdWN0PTAsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzM4Njc1XSBpbnB1dDogSElEIDA0ZDk6MTIwMyBh
cyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTIv
MS0yOjEuMC8wMDAzOjA0RDk6MTIwMy4wMDAyL2lucHV0L2lucHV0MgpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAyLjgwMjMzMl0gaGlkLWdlbmVyaWMgMDAwMzowNEQ5OjEyMDMu
MDAwMjogaW5wdXQsaGlkcmF3MTogVVNCIEhJRCB2MS4xMSBLZXlib2FyZCBbSElEIDA0ZDk6MTIw
M10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAyLjg0NDYzNl0gaW5wdXQ6IEhJRCAwNGQ5OjEyMDMgU3lzdGVtIENvbnRyb2wg
YXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0y
LzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDMKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45MDYyODZdIGlucHV0OiBISUQgMDRkOToxMjAzIENvbnN1
bWVyIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzow
MC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDQKU2Vw
IDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45MDYzMDldIGhpZC1nZW5lcmljIDAw
MDM6MDREOToxMjAzLjAwMDM6IGlucHV0LGhpZHJhdzI6IFVTQiBISUQgdjEuMTEgRGV2aWNlIFtI
SUQgMDRkOToxMjAzXSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQxClNlcCAxMyAxMjoxMzow
NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTQ1Nzg2XSBtcHQzc2FzX2NtMDogZGlhZyByZXNldDog
U1VDQ0VTUwpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAyLjk4MDc5NV0gbXB0
M3Nhc19jbTA6IHNjYXR0ZXIgZ2F0aGVyOiBzZ2VfaW5fbWFpbl9tc2coMSksIHNnZV9wZXJfY2hh
aW4oNyksIHNnZV9wZXJfaW8oMTI4KSwgY2hhaW5zX3Blcl9pbygxOSkKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45ODE4ODFdIG1wdDNzYXNfY20wOiByZXF1ZXN0IHBvb2wo
MHgoX19fX3B0cnZhbF9fX18pKSAtIGRtYSgweGZmNTAwMDAwKTogZGVwdGgoNDIyNCksIGZyYW1l
X3NpemUoMTI4KSwgcG9vbF9zaXplKDUyOCBrQikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy4wMDE5NDldIG1wdDNzYXNfY20wOiBzZW5zZSBwb29sKDB4KF9fX19wdHJ2YWxf
X19fKSktIGRtYSgweGZlYjAwMDAwKTogZGVwdGgoMzk2MyksZWxlbWVudF9zaXplKDk2KSwgcG9v
bF9zaXplKDM3MSBrQikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wMDIw
NTBdIG1wdDNzYXNfY20wOiBjb25maWcgcGFnZSgweChfX19fcHRydmFsX19fXykpIC0gZG1hKDB4
ZmU5ZjYwMDApOiBzaXplKDUxMikKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAg
My4wMDIwNTFdIG1wdDNzYXNfY20wOiBBbGxvY2F0ZWQgcGh5c2ljYWwgbWVtb3J5OiBzaXplKDcy
Nzgga0IpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDAyMDUxXSBtcHQz
c2FzX2NtMDogQ3VycmVudCBDb250cm9sbGVyIFF1ZXVlIERlcHRoKDM5NjApLE1heCBDb250cm9s
bGVyIFF1ZXVlIERlcHRoKDQwOTYpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDMuMDAyMDUyXSBtcHQzc2FzX2NtMDogU2NhdHRlciBHYXRoZXIgRWxlbWVudHMgcGVyIElPKDEy
OCkKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wNDIyNzNdIHVzYiAxLTM6
IG5ldyBoaWdoLXNwZWVkIFVTQiBkZXZpY2UgbnVtYmVyIDQgdXNpbmcgeGhjaV9oY2QKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xOTU3MjJdIHVzYiAxLTM6IE5ldyBVU0Ig
ZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNWUzLCBpZFByb2R1Y3Q9MDYxMCwgYmNkRGV2aWNlPTkz
LjEwClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTk1NzIzXSB1c2IgMS0z
OiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9
MApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjE5NTcyNF0gdXNiIDEtMzog
UHJvZHVjdDogVVNCMi4wIEh1YgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAz
LjE5NTcyNV0gdXNiIDEtMzogTWFudWZhY3R1cmVyOiBHZW5lc3lzTG9naWMKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xOTk3NTddIGh1YiAxLTM6MS4wOiBVU0IgaHViIGZv
dW5kClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjAwMzQ5XSBodWIgMS0z
OjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjIwMjkxNF0gbXB0M3Nhc19jbTA6IF9iYXNlX2Rpc3BsYXlfZndwa2dfdmVyc2lvbjogY29t
cGxldGUKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMDMxOTddIG1wdDNz
YXNfY20wOiBMU0lTQVMzMDA4OiBGV1ZlcnNpb24oMTYuMDAuMTAuMDApLCBDaGlwUmV2aXNpb24o
MHgwMiksIEJpb3NWZXJzaW9uKDE1LjAwLjAxLjAwKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAzLjIwMzE5N10gbXB0M3Nhc19jbTA6IFByb3RvY29sPShJbml0aWF0b3IpLCBD
YXBhYmlsaXRpZXM9KFJhaWQsVExSLEVFRFAsU25hcHNob3QgQnVmZmVyLERpYWcgVHJhY2UgQnVm
ZmVyLFRhc2sgU2V0IEZ1bGwsTkNRKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjIwMzI0N10gc2NzaSBob3N0MDogRnVzaW9uIE1QVCBTQVMgSG9zdApTZXAgMTMgMTI6MTM6
MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjIwMzgxNV0gbXB0M3Nhc19jbTA6IHNlbmRpbmcgcG9y
dCBlbmFibGUgISEKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMzg2NzRd
IFBNOiBJbWFnZSBub3QgZm91bmQgKGNvZGUgLTIyKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAzLjI5MDE5M10gRVhUNC1mcyAobnZtZTNuMXAyKTogbW91bnRlZCBmaWxlc3lz
dGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAzLjMxOTcyNF0gTm90IGFjdGl2YXRpbmcgTWFuZGF0b3J5IEFj
Y2VzcyBDb250cm9sIGFzIC9zYmluL3RvbW95by1pbml0IGRvZXMgbm90IGV4aXN0LgpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjUxNDQwOV0gcmFuZG9tOiBzeXN0ZW1kOiB1
bmluaXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MTUxNTZdIHJhbmRvbTogc3lzdGVtZDogdW5pbml0aWFs
aXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJlYWQpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDMuNTE1MzY0XSByYW5kb206IHN5c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJh
bmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDog
WyAgICAzLjUzMTI4OV0gZnVzZTogaW5pdCAoQVBJIHZlcnNpb24gNy4zMikKU2VwIDEzIDEyOjEz
OjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MzI3NzZdIEVYVDQtZnMgKG52bWUzbjFwMik6IHJl
LW1vdW50ZWQuIE9wdHM6IGVycm9ycz1yZW1vdW50LXJvClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDMuNjI5ODU1XSBhY3BpLXRhZCBBQ1BJMDAwRTowMDogVW5zdXBwb3J0ZWQg
Y2FwYWJpbGl0aWVzClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjY4Mzgy
XSB4Z2VuZS1zbGltcHJvLWh3bW9uIEFQTUMwRDI5OjAwOiBBUE0gWC1HZW5lIFNvQyBIVyBtb25p
dG9yIGRyaXZlciByZWdpc3RlcmVkClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAg
IDMuNjc1MjU4XSBJUE1JIG1lc3NhZ2UgaGFuZGxlcjogdmVyc2lvbiAzOS4yClNlcCAxMyAxMjox
MzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjc3NzUyXSBpcG1pIGRldmljZSBpbnRlcmZhY2UK
U2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42ODM1MzJdIGlwbWlfc2k6IElQ
TUkgU3lzdGVtIEludGVyZmFjZSBkcml2ZXIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy42ODM3MzJdIGlwbWlfc2k6IFVuYWJsZSB0byBmaW5kIGFueSBTeXN0ZW0gSW50ZXJm
YWNlKHMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzU5MTE2XSBFWFQ0
LWZzIChudm1lM24xcDUpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9k
ZS4gT3B0czogKG51bGwpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzYw
OTE2XSBFWFQ0LWZzIChudm1lM24xcDMpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVk
IGRhdGEgbW9kZS4gT3B0czogKG51bGwpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuNzc0MzExXSBBZGRpbmcgMTAwMDQ0NGsgc3dhcCBvbiAvZGV2L252bWUzbjFwNC4gIFBy
aW9yaXR5Oi0yIGV4dGVudHM6MSBhY3Jvc3M6MTAwMDQ0NGsgU1NGUwpTZXAgMTMgMTI6MTM6MDQg
ZGViaWFuIGtlcm5lbDogWyAgICAzLjc3OTcwN10gRVhUNC1mcyAobnZtZTNuMXA2KTogbW91bnRl
ZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjc5NjcwMV0gaXBtaV9zc2lmOiBJUE1JIFNT
SUYgSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAz
LjgwMTMyM10gaXBtaV9zc2lmIGkyYy1BTVBDMDAwNDowMDogaXBtaV9zc2lmOiBUcnlpbmcgQUNQ
SS1zcGVjaWZpZWQgU1NJRiBpbnRlcmZhY2UgYXQgaTJjIGFkZHJlc3MgMHgxMCwgYWRhcHRlciBT
eW5vcHN5cyBEZXNpZ25XYXJlIEkyQyBhZGFwdGVyLCBzbGF2ZSBhZGRyZXNzIDB4MjAKU2VwIDEz
IDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MjY1OTJdIHNic2EtZ3dkdCBzYnNhLWd3
ZHQuMDogSW5pdGlhbGl6ZWQgd2l0aCAxMHMgdGltZW91dCBAIDI1MDAwMDAwIEh6LCBhY3Rpb249
MC4KU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44Mzc1NjRdIGlwbWlfc2k6
IElQTUkgU3lzdGVtIEludGVyZmFjZSBkcml2ZXIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44Mzg1OThdIHJhbmRvbTogY3JuZyBpbml0IGRvbmUKU2VwIDEzIDEyOjEzOjA0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44Mzg2MDBdIHJhbmRvbTogNyB1cmFuZG9tIHdhcm5pbmco
cykgbWlzc2VkIGR1ZSB0byByYXRlbGltaXRpbmcKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44Mzg5MDVdIHBzdG9yZTogVXNpbmcgY3Jhc2ggZHVtcCBjb21wcmVzc2lvbjog
ZGVmbGF0ZQpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjgzODkxM10gcHN0
b3JlOiBSZWdpc3RlcmVkIGVmaSBhcyBwZXJzaXN0ZW50IHN0b3JlIGJhY2tlbmQKU2VwIDEzIDEy
OjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44Mzk0NzNdIGlwbWlfc2k6IFVuYWJsZSB0byBm
aW5kIGFueSBTeXN0ZW0gSW50ZXJmYWNlKHMpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVs
OiBbICAgIDMuODQ0MDM1XSBjcnlwdGQ6IG1heF9jcHVfcWxlbiBzZXQgdG8gMTAwMApTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjg4MjA5N10gYXVkaXQ6IHR5cGU9MTQwMCBh
dWRpdCgxNjMxNTE1MzgwLjIwNDoyKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9m
aWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2ZmaWNlLXhwZGZpbXBv
cnQiIHBpZD0xMTQwIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjEzOjA0IGRlYmlh
biBrZXJuZWw6IFsgICAgMy44ODczMjZdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNTM4
MC4yMTI6Myk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9m
aWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJyZW9mZmljZS1zZW5kZG9jIiBwaWQ9MTE0NyBjb21t
PSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
ODk0NDI1XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MTUzODAuMjIwOjQpOiBhcHBhcm1v
cj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIg
bmFtZT0ibGlicmVvZmZpY2Utb29wc2xhc2giIHBpZD0xMTQxIGNvbW09ImFwcGFybW9yX3BhcnNl
ciIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44OTQ1OTVdIGF1ZGl0OiB0
eXBlPTE0MDAgYXVkaXQoMTYzMTUxNTM4MC4yMjA6NSk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJh
dGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsc2JfcmVsZWFz
ZSIgcGlkPTExMzcgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTI6MTM6MDQgZGViaWFu
IGtlcm5lbDogWyAgICAzLjkwNDU3Ml0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE1Mzgw
LjIyODo2KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2Zp
bGU9InVuY29uZmluZWQiIG5hbWU9Im52aWRpYV9tb2Rwcm9iZSIgcGlkPTExNDggY29tbT0iYXBw
YXJtb3JfcGFyc2VyIgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNTQ2
M10gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE1MzgwLjIyODo3KTogYXBwYXJtb3I9IlNU
QVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9
Im52aWRpYV9tb2Rwcm9iZS8va21vZCIgcGlkPTExNDggY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpT
ZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICAzLjkyMDUyMV0gYXVkaXQ6IHR5cGU9
MTQwMCBhdWRpdCgxNjMxNTE1MzgwLjI0NDo4KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9u
PSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Ii91c3IvYmluL21hbiIg
cGlkPTExNDIgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTI6MTM6MDQgZGViaWFuIGtl
cm5lbDogWyAgICAzLjkyMTEyOF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE1MzgwLjI0
NDo5KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9
InVuY29uZmluZWQiIG5hbWU9Im1hbl9maWx0ZXIiIHBpZD0xMTQyIGNvbW09ImFwcGFybW9yX3Bh
cnNlciIKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy45MjE5NTJdIGF1ZGl0
OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNTM4MC4yNDQ6MTApOiBhcHBhcm1vcj0iU1RBVFVTIiBv
cGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibWFuX2dy
b2ZmIiBwaWQ9MTE0MiBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMjoxMzowNCBkZWJp
YW4ga2VybmVsOiBbICAgIDQuMTExODM2XSBpcG1pX3NzaWYgaTJjLUFNUEMwMDA0OjAwOiBJUE1J
IG1lc3NhZ2UgaGFuZGxlcjogRm91bmQgbmV3IEJNQyAobWFuX2lkOiAweDAwM2MwYSwgcHJvZF9p
ZDogMHgwMjEyLCBkZXZfaWQ6IDB4MjApClNlcCAxMyAxMjoxMzowNCBkZWJpYW4ga2VybmVsOiBb
ICAgIDUuNzU2MTI5XSBtcHQzc2FzX2NtMDogaG9zdF9hZGQ6IGhhbmRsZSgweDAwMDEpLCBzYXNf
YWRkcigweDUwMDE0ODUwMDA3NzllMzApLCBwaHlzKDgpClNlcCAxMyAxMjoxMzowNCBkZWJpYW4g
a2VybmVsOiBbICAgIDcuOTE3NDM4XSBrYXVkaXRkX3ByaW50a19za2I6IDIgY2FsbGJhY2tzIHN1
cHByZXNzZWQKU2VwIDEzIDEyOjEzOjA0IGRlYmlhbiBrZXJuZWw6IFsgICAgNy45MTc0MzldIGF1
ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNTM4NC4yNDA6MTMpOiBhcHBhcm1vcj0iU1RBVFVT
IiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGli
cmVvZmZpY2Utc29mZmljZSIgcGlkPTExMzYgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMg
MTI6MTM6MDQgZGViaWFuIGtlcm5lbDogWyAgICA3LjkxODkzM10gYXVkaXQ6IHR5cGU9MTQwMCBh
dWRpdCgxNjMxNTE1Mzg0LjI0NDoxNCk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJv
ZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJyZW9mZmljZS1zb2ZmaWNl
Ly9ncGciIHBpZD0xMTM2IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjEzOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAxMC44OTgyNzhdIG1wdDNzYXNfY20wOiBwb3J0IGVuYWJsZTogU1VD
Q0VTUwpTZXAgMTMgMTI6MTM6MDggZGViaWFuIGtlcm5lbDogWyAgIDEyLjM3NDY5N10gaWdiIDAw
MDM6MDI6MDAuMSBlbm8yOiBpZ2I6IGVubzIgTklDIExpbmsgaXMgVXAgMTAwMCBNYnBzIEZ1bGwg
RHVwbGV4LCBGbG93IENvbnRyb2w6IFJYL1RYClNlcCAxMyAxMjoxMzowOCBkZWJpYW4ga2VybmVs
OiBbICAgMTIuMzc0ODY5XSBJUHY2OiBBRERSQ09ORihORVRERVZfQ0hBTkdFKTogZW5vMjogbGlu
ayBiZWNvbWVzIHJlYWR5ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBCb290aW5nIExpbnV4IG9uIHBoeXNpY2FsIENQVSAweDAwMDAxMjAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBMaW51eCB2
ZXJzaW9uIDUuMTAuMC04LWFybTY0IChkZWJpYW4ta2VybmVsQGxpc3RzLmRlYmlhbi5vcmcpIChn
Y2MtMTAgKERlYmlhbiAxMC4yLjEtNikgMTAuMi4xIDIwMjEwMTEwLCBHTlUgbGQgKEdOVSBCaW51
dGlscyBmb3IgRGViaWFuKSAyLjM1LjIpICMxIFNNUCBEZWJpYW4gNS4xMC40Ni00ICgyMDIxLTA4
LTAzKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZWZpOiBF
RkkgdjIuNzAgYnkgQW1lcmljYW4gTWVnYXRyZW5kcwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gZWZpOiBBQ1BJIDIuMD0weGVlOTAwMDAwIFNNQklPUyAzLjA9
MHhmMWIzZmY5OCBFU1JUPTB4ZTk0ZjhmOTggTU9LdmFyPTB4ZTk0ZjMwMDAgUk5HPTB4Zjc2NmU4
MTggTUVNUkVTRVJWRT0weGU5NjFmODE4IApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gZWZpOiBzZWVkaW5nIGVudHJvcHkgcG9vbApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gc2VjdXJlYm9vdDogU2VjdXJlIGJvb3QgZGlz
YWJsZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGVzcnQ6
IFJlc2VydmluZyBFU1JUIHNwYWNlIGZyb20gMHgwMDAwMDAwMGU5NGY4Zjk4IHRvIDB4MDAwMDAw
MDBlOTRmOGZkMC4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IEVhcmx5IHRhYmxlIGNoZWNrc3VtIHZlcmlmaWNhdGlvbiBkaXNhYmxlZApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUlNEUCAweDAwMDAw
MDAwRUU5MDAwMDAgMDAwMDI0ICh2MDIgQW1wZXJlKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogWFNEVCAweDAwMDAwMDAwRUU4RjAwMDAgMDAwMEE0
ICh2MDEgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGQUNQIDB4MDAwMDAwMDBF
RThEMDAwMCAwMDAxMTQgKHYwNiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUw
OSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IERT
RFQgMHgwMDAwMDAwMEVFODYwMDAwIDAxOUE5NSAodjAyIEFtcGVyZSBKYWRlICAgICAwMDAwMDAw
MSBJTlRMIDIwMjAwNzE3KQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogREJHMiAweDAwMDAwMDAwRUU4RTAwMDAgMDAwMDVDICh2MDAgQW1wZXJlIEFs
dHJhICAgIDAwMDAwMDAwIElOVEwgMjAxOTA1MDkpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBHVERUIDB4MDAwMDAwMDBFRThDMDAwMCAwMDAxMTAg
KHYwMyBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUwOSkKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNTRFQgMHgwMDAwMDAwMEVF
OEIwMDAwIDAwMDAyRCAodjAyIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMSBJTlRMIDIwMTkwNTA5
KQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRklE
VCAweDAwMDAwMDAwRUU4NTAwMDAgMDAwMDlDICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5
IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBTUENSIDB4MDAwMDAwMDBFRTg0MDAwMCAwMDAwNTAgKHYwMiBBTEFTS0EgQSBN
IEkgICAgMDEwNzIwMDkgQU1JICAwMDA1MDAwRikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEJHUlQgMHgwMDAwMDAwMEVFODMwMDAwIDAwMDAzOCAo
djAxIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTUNGRyAweDAwMDAwMDAwRUU4
MjAwMDAgMDAwMEFDICh2MDEgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIEFNUC4gMDEwMDAwMTMp
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBJT1JU
IDB4MDAwMDAwMDBFRTgxMDAwMCAwMDA2MTAgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAg
QU1QLiAwMTAwMDAxMykKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IFBQVFQgMHgwMDAwMDAwMEVFN0YwMDAwIDAwNkU2MCAodjAyIEFtcGVyZSBBbHRy
YSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogU0xJVCAweDAwMDAwMDAwRUU3RTAwMDAgMDAwMDJEICh2
MDEgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUIDB4MDAwMDAwMDBFRTdE
MDAwMCAwMDA2RDAgKHYwMyBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEFQSUMg
MHgwMDAwMDAwMEVFODAwMDAwIDAwMTlGNCAodjA1IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMyBB
TUkgIDAxMDAwMDEzKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogUENDVCAweDAwMDAwMDAwRUU3QzAwMDAgMDAwNTc2ICh2MDIgQW1wZXJlIEFsdHJh
ICAgIDAwMDAwMDAzIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBXU01UIDB4MDAwMDAwMDBFRTdCMDAwMCAwMDAwMjggKHYw
MSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZQRFQgMHgwMDAwMDAwMEVFN0Ew
MDAwIDAwMDA0NCAodjAxIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAxMDAwMDEzKQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1BDUjog
Y29uc29sZTogcGwwMTEsbW1pbzMyLDB4MTAwMDAyNjAwMDAwLDExNTIwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1JBVDogTm9kZSAwIFBYTSAw
IFttZW0gMHg4ODMwMDAwMC0weDg4M2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1JBVDogTm9kZSAwIFBYTSAwIFttZW0gMHg5MDAwMDAw
MC0weGZmZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogU1JBVDogTm9kZSAwIFBYTSAwIFttZW0gMHg4MDAwMDAwMDAwMC0weDgwMDdmZmZm
ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
U1JBVDogTm9kZSAwIFBYTSAwIFttZW0gMHg4MDEwMDAwMDAwMC0weDgwZmZmZmZmZmZmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gTlVNQTogTk9ERV9EQVRB
IFttZW0gMHg4MGZmODA4NGIwMC0weDgwZmY4MDg2ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gWm9uZSByYW5nZXM6ClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSAgICAgIFttZW0gMHgwMDAwMDAwMDg4MzAw
MDAwLTB4MDAwMDAwMDBmZmZmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdICAgRE1BMzIgICAgZW1wdHkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsICAgW21lbSAweDAwMDAwMDAxMDAwMDAwMDAtMHgw
MDAwMDgwZmZmZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gTW92YWJsZSB6b25lIHN0YXJ0IGZvciBlYWNoIG5vZGUKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEVhcmx5IG1lbW9yeSBub2RlIHJhbmdlcwpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21l
bSAweDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMDg4M2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAw
OTAwMDAwMDAtMHgwMDAwMDAwMDkxZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwOTIwMDAwMDAtMHgw
MDAwMDAwMDkzZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwOTQwMDAwMDAtMHgwMDAwMDAwMGVlMmU2
ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2Rl
ICAgMDogW21lbSAweDAwMDAwMDAwZWUyZTcwMDAtMHgwMDAwMDAwMGVlNzQwZmZmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAw
eDAwMDAwMDAwZWU3NDEwMDAtMHgwMDAwMDAwMGVlNzQ0ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3
NDUwMDAtMHgwMDAwMDAwMGVlNzRjZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3NGQwMDAtMHgwMDAw
MDAwMGVlNzRkZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3NGUwMDAtMHgwMDAwMDAwMGVlNzRmZmZm
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAg
MDogW21lbSAweDAwMDAwMDAwZWU3NTAwMDAtMHgwMDAwMDAwMGVlOTZmZmZmXQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAw
MDAwMDAwZWU5NzAwMDAtMHgwMDAwMDAwMGVlOTdmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU5ODAw
MDAtMHgwMDAwMDAwMGYwNjRmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZjA2NTAwMDAtMHgwMDAwMDAw
MGY3NjZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
ICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZjc2NzAwMDAtMHgwMDAwMDAwMGY3ODRmZmZmXQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDog
W21lbSAweDAwMDAwMDAwZjc4NTAwMDAtMHgwMDAwMDAwMGY3ZmRmZmZmXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAw
MDAwZjdmZTAwMDAtMHgwMDAwMDAwMGZmYzhlZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZmZjOGYwMDAt
MHgwMDAwMDAwMGZmYzhmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZmZjOTAwMDAtMHgwMDAwMDAwMGZm
ZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBu
b2RlICAgMDogW21lbSAweDAwMDAwODAwMDAwMDAwMDAtMHgwMDAwMDgwMDdmZmZmZmZmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21l
bSAweDAwMDAwODAxMDAwMDAwMDAtMHgwMDAwMDgwZmZmZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSW5pdG1lbSBzZXR1cCBub2RlIDAgW21lbSAw
eDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDgwZmZmZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gT24gbm9kZSAwIHRvdGFscGFnZXM6IDE2NzExOTM2
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25l
OiA3MTcyIHBhZ2VzIHVzZWQgZm9yIG1lbW1hcApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gICBETUEgem9uZTogMCBwYWdlcyByZXNlcnZlZApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEgem9uZTogNDU5MDA4IHBh
Z2VzLCBMSUZPIGJhdGNoOjYzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIE5vcm1hbCB6b25lOiAyNTM5NTIgcGFnZXMgdXNlZCBmb3IgbWVtbWFwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIE5vcm1hbCB6b25lOiAx
NjI1MjkyOCBwYWdlcywgTElGTyBiYXRjaDo2MwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gY21hOiBSZXNlcnZlZCA2NCBNaUIgYXQgMHgwMDAwMDAwMGZiYzAw
MDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2NpOiBw
cm9iaW5nIGZvciBjb25kdWl0IG1ldGhvZCBmcm9tIEFDUEkuClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2NpOiBQU0NJdjEuMSBkZXRlY3RlZCBpbiBmaXJt
d2FyZS4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6
IFVzaW5nIHN0YW5kYXJkIFBTQ0kgdjAuMiBmdW5jdGlvbiBJRHMKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IE1JR1JBVEVfSU5GT19UWVBFIG5vdCBz
dXBwb3J0ZWQuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
c2NpOiBTTUMgQ2FsbGluZyBDb252ZW50aW9uIHYxLjIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwIC0+IE5vZGUgMApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDMwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4NDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDUwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg1MDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4NjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg3MDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDgwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg4MDEwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDkwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhhMDAw
MCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YTAxMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweGIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHhiMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YzAwMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHhkMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZDAxMDAgLT4g
Tm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHhlMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGYwMTAwIC0+IE5vZGUgMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgxMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDEwMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTEwMDAwIC0+IE5v
ZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDEyMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIw
MTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMzAwMDAgLT4gTm9kZSAwClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MTQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAxMDAgLT4g
Tm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MTUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
NjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxNzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE4MDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxOTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDE5MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWEwMDAwIC0+IE5vZGUg
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDFiMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYzAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDFjMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MWQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFlMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MWUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZjAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgyMDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIxMDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgyMjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIy
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjMwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDI0MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNTAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDI1MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MjYwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI3MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MjcwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gcGVyY3B1OiBFbWJlZGRlZCAzMyBwYWdlcy9jcHUgczk1MTkyIHI4MTky
IGQzMTc4NCB1MTM1MTY4ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBwY3B1LWFsbG9jOiBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjggYWxsb2M9MzMqNDA5
NgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxv
YzogWzBdIDAwIFswXSAwMSBbMF0gMDIgWzBdIDAzIFswXSAwNCBbMF0gMDUgWzBdIDA2IFswXSAw
NyAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxs
b2M6IFswXSAwOCBbMF0gMDkgWzBdIDEwIFswXSAxMSBbMF0gMTIgWzBdIDEzIFswXSAxNCBbMF0g
MTUgClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFs
bG9jOiBbMF0gMTYgWzBdIDE3IFswXSAxOCBbMF0gMTkgWzBdIDIwIFswXSAyMSBbMF0gMjIgWzBd
IDIzIApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1h
bGxvYzogWzBdIDI0IFswXSAyNSBbMF0gMjYgWzBdIDI3IFswXSAyOCBbMF0gMjkgWzBdIDMwIFsw
XSAzMSAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUt
YWxsb2M6IFswXSAzMiBbMF0gMzMgWzBdIDM0IFswXSAzNSBbMF0gMzYgWzBdIDM3IFswXSAzOCBb
MF0gMzkgClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1
LWFsbG9jOiBbMF0gNDAgWzBdIDQxIFswXSA0MiBbMF0gNDMgWzBdIDQ0IFswXSA0NSBbMF0gNDYg
WzBdIDQ3IApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNw
dS1hbGxvYzogWzBdIDQ4IFswXSA0OSBbMF0gNTAgWzBdIDUxIFswXSA1MiBbMF0gNTMgWzBdIDU0
IFswXSA1NSAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBj
cHUtYWxsb2M6IFswXSA1NiBbMF0gNTcgWzBdIDU4IFswXSA1OSBbMF0gNjAgWzBdIDYxIFswXSA2
MiBbMF0gNjMgClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
Y3B1LWFsbG9jOiBbMF0gNjQgWzBdIDY1IFswXSA2NiBbMF0gNjcgWzBdIDY4IFswXSA2OSBbMF0g
NzAgWzBdIDcxIApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
cGNwdS1hbGxvYzogWzBdIDcyIFswXSA3MyBbMF0gNzQgWzBdIDc1IFswXSA3NiBbMF0gNzcgWzBd
IDc4IFswXSA3OSAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBHSUMgc3lzdGVtIHJl
Z2lzdGVyIENQVSBpbnRlcmZhY2UKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFZpcnR1YWxpemF0aW9uIEhvc3QgRXh0
ZW5zaW9ucwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BV
IGZlYXR1cmVzOiBrZXJuZWwgcGFnZSB0YWJsZSBpc29sYXRpb24gZm9yY2VkIE9OIGJ5IEtBU0xS
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVy
ZXM6IGRldGVjdGVkOiBLZXJuZWwgcGFnZSB0YWJsZSBpc29sYXRpb24gKEtQVEkpClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVj
dGVkOiBIYXJkd2FyZSBkaXJ0eSBiaXQgbWFuYWdlbWVudApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3BlY3RyZS12
NApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1
cmVzOiBkZXRlY3RlZDogQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gYWx0ZXJuYXRpdmVzOiBwYXRjaGluZyBrZXJuZWwgY29k
ZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQnVpbHQgMSB6
b25lbGlzdHMsIG1vYmlsaXR5IGdyb3VwaW5nIG9uLiAgVG90YWwgcGFnZXM6IDE2NDUwODEyClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBQb2xpY3kgem9uZTog
Tm9ybWFsClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBLZXJu
ZWwgY29tbWFuZCBsaW5lOiBCT09UX0lNQUdFPS9ib290L3ZtbGludXotNS4xMC4wLTgtYXJtNjQg
cm9vdD1VVUlEPTZjMDgzMjdjLTRiYmQtNDM0MS1hNDI3LWE3MmJjMTUzMzg5NSBybyBxdWlldApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2df
YnVmX2xlbiBpbmRpdmlkdWFsIG1heCBjcHUgY29udHJpYnV0aW9uOiA0MDk2IGJ5dGVzClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZf
bGVuIHRvdGFsIGNwdV9leHRyYSBjb250cmlidXRpb25zOiAzMjM1ODQgYnl0ZXMKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4g
bWluIHNpemU6IDEzMTA3MiBieXRlcwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbjogNTI0Mjg4IGJ5dGVzClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGVhcmx5IGxvZyBidWYg
ZnJlZTogMTE5MDE2KDkwJSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIERlbnRyeSBjYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDgzODg2MDggKG9yZGVyOiAx
NCwgNjcxMDg4NjQgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIElub2RlLWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogNDE5NDMwNCAo
b3JkZXI6IDEzLCAzMzU1NDQzMiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gbWVtIGF1dG8taW5pdDogc3RhY2s6b2ZmLCBoZWFwIGFs
bG9jOm9uLCBoZWFwIGZyZWU6b2ZmClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBzb2Z0d2FyZSBJTyBUTEI6IG1hcHBlZCBbbWVtIDB4MDAwMDAwMDBlYTJlNzAw
MC0weDAwMDAwMDAwZWUyZTcwMDBdICg2NE1CKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gTWVtb3J5OiAyNTUxNjMySy82Njg0Nzc0NEsgYXZhaWxhYmxlICgx
MTc3Nksga2VybmVsIGNvZGUsIDI0MzZLIHJ3ZGF0YSwgNzAwOEsgcm9kYXRhLCA1NDQwSyBpbml0
LCA1OThLIGJzcywgMTU4NTU1MksgcmVzZXJ2ZWQsIDY1NTM2SyBjbWEtcmVzZXJ2ZWQpClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByYW5kb206IGdldF9yYW5k
b21fdTY0IGNhbGxlZCBmcm9tIF9fa21lbV9jYWNoZV9jcmVhdGUrMHgzOC8weDU2MCB3aXRoIGNy
bmdfaW5pdD0wClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBT
TFVCOiBIV2FsaWduPTY0LCBPcmRlcj0wLTMsIE1pbk9iamVjdHM9MCwgQ1BVcz04MCwgTm9kZXM9
MQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZnRyYWNlOiBh
bGxvY2F0aW5nIDM4NTM1IGVudHJpZXMgaW4gMTUxIHBhZ2VzClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRlZCAxNTEgcGFnZXMgd2l0
aCA1IGdyb3VwcwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
cmN1OiBIaWVyYXJjaGljYWwgUkNVIGltcGxlbWVudGF0aW9uLgpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcmN1OiAJUkNVIHJlc3RyaWN0aW5nIENQVXMgZnJv
bSBOUl9DUFVTPTI1NiB0byBucl9jcHVfaWRzPTgwLgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gCVJ1ZGUgdmFyaWFudCBvZiBUYXNrcyBSQ1UgZW5hYmxlZC4K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIAlUcmFjaW5nIHZh
cmlhbnQgb2YgVGFza3MgUkNVIGVuYWJsZWQuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSByY3U6IFJDVSBjYWxjdWxhdGVkIHZhbHVlIG9mIHNjaGVkdWxlci1l
bmxpc3RtZW50IGRlbGF5IGlzIDI1IGppZmZpZXMuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IEFkanVzdGluZyBnZW9tZXRyeSBmb3IgcmN1X2Zhbm91
dF9sZWFmPTE2LCBucl9jcHVfaWRzPTgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBOUl9JUlFTOiA2NCwgbnJfaXJxczogNjQsIHByZWFsbG9jYXRlZCBpcnFz
OiAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2Mzog
R0lDOiBVc2luZyBzcGxpdCBFT0kvRGVhY3RpdmF0ZSBtb2RlClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogNjcyIFNQSXMgaW1wbGVtZW50ZWQKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiAwIEV4dGVu
ZGVkIFNQSXMgaW1wbGVtZW50ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEdJQ3YzOiBEaXN0cmlidXRvciBoYXMgbm8gUmFuZ2UgU2VsZWN0b3Igc3VwcG9y
dApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDE2
IFBQSXMgaW1wbGVtZW50ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEdJQ3YzOiBDUFUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA1YzAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAwIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDEgLT4gTm9kZSAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0g
MCAtPiBJVFMgMiAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAzIC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDQgLT4gTm9k
ZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQ
WE0gMCAtPiBJVFMgNSAtPiBOb2RlIDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA2IC0+IE5vZGUgMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDcgLT4g
Tm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMg
W21lbSAweDEwMDEwMDA0MDAwMC0weDEwMDEwMDA1ZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNDAwMDA6IGFsbG9jYXRl
ZCA4MTkyIERldmljZXMgQDgwMDAwMmMwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNo
ciAxKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4
MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBA
ODAwMDAyZDAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDYwMDAwLTB4
MTAwMTAwMDdmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gSVRTQDB4MDAwMDEwMDEwMDA2MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAy
ZjAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDYwMDAwOiBhbGxv
Y2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDMwMDAwMCAoZmxhdCwgZXN6
IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwODAwMDAtMHgxMDAxMDAwOWZmZmZdClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDgw
MDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDMyMDAwMCAoaW5kaXJlY3QsIGVzeiA4
LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQg
Q29sbGVjdGlvbnMgQDgwMDAwMzMwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEw
MDEwMDBhMDAwMC0weDEwMDEwMDBiZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwYTAwMDA6IGFsbG9jYXRlZCA4MTkyIERl
dmljZXMgQDgwMDAwMzUwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEw
MDBhMDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzNjAw
MDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGMwMDAwLTB4MTAwMTAwMGRm
ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4
MDAwMDEwMDEwMDBjMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzODAwMDAgKGlu
ZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAwOiBhbGxvY2F0ZWQgMzI3
NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM5MDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2
NEssIHNociAxKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTIFttZW0gMHgxMDAxMDAwZTAwMDAtMHgxMDAxMDAwZmZmZmZdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGUwMDAwOiBhbGxv
Y2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDNiMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRL
LCBzaHIgMSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElU
U0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlv
bnMgQDgwMDAwM2MwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDEwMDAw
MC0weDEwMDEwMDExZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUU0AweDAwMDAxMDAxMDAxMDAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgw
MDAwM2UwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEwMDAwMDog
YWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzZjAwMDAgKGZsYXQs
IGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTIwMDAwLTB4MTAwMTAwMTNmZmZmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEw
MDEyMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDA4MTAwMDAgKGluZGlyZWN0LCBl
c3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJy
dXB0IENvbGxlY3Rpb25zIEA4MDAwMDgyMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAx
KQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IHVz
aW5nIExQSSBwcm9wZXJ0eSB0YWJsZSBAMHgwMDAwMDgwMDAwODMwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogQ1BVMDogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4NDAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEkgR1REVDogZm91bmQgMSBtZW1vcnktbWFwcGVk
IHRpbWVyIGJsb2NrKHMpLgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gYXJjaF90aW1lcjogY3AxNSBhbmQgbW1pbyB0aW1lcihzKSBydW5uaW5nIGF0IDI1LjAw
TUh6IChwaHlzL3BoeXMpLgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gY2xvY2tzb3VyY2U6IGFyY2hfc3lzX2NvdW50ZXI6IG1hc2s6IDB4ZmZmZmZmZmZmZmZm
ZmYgbWF4X2N5Y2xlczogMHg1YzQwOTM5YjUsIG1heF9pZGxlX25zOiA0NDA3OTUyMDI2NDYgbnMK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNjaGVkX2Nsb2Nr
OiA1NiBiaXRzIGF0IDI1TUh6LCByZXNvbHV0aW9uIDQwbnMsIHdyYXBzIGV2ZXJ5IDQzOTgwNDY1
MTExMDBucwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDA1Ml0gQ29u
c29sZTogY29sb3VyIGR1bW15IGRldmljZSA4MHgyNQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDA3NF0gQUNQSTogQ29yZSByZXZpc2lvbiAyMDIwMDkyNQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDI2N10gQ2FsaWJyYXRpbmcgZGVsYXkg
bG9vcCAoc2tpcHBlZCksIHZhbHVlIGNhbGN1bGF0ZWQgdXNpbmcgdGltZXIgZnJlcXVlbmN5Li4g
NTAuMDAgQm9nb01JUFMgKGxwaj0xMDAwMDApClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMjY5XSBwaWRfbWF4OiBkZWZhdWx0OiA4MTkyMCBtaW5pbXVtOiA2NDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzMDJdIExTTTogU2VjdXJpdHkg
RnJhbWV3b3JrIGluaXRpYWxpemluZwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDMwN10gWWFtYTogZGlzYWJsZWQgYnkgZGVmYXVsdDsgZW5hYmxlIHdpdGggc3lzY3Rs
IGtlcm5lbC55YW1hLioKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAz
MjRdIEFwcEFybW9yOiBBcHBBcm1vciBpbml0aWFsaXplZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDMyNV0gVE9NT1lPIExpbnV4IGluaXRpYWxpemVkClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzY2XSBNb3VudC1jYWNoZSBoYXNoIHRh
YmxlIGVudHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzOTBdIE1vdW50cG9pbnQtY2Fj
aGUgaGFzaCB0YWJsZSBlbnRyaWVzOiAxMzEwNzIgKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBs
aW5lYXIpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMjU1XSByY3U6
IEhpZXJhcmNoaWNhbCBTUkNVIGltcGxlbWVudGF0aW9uLgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMTM5Ml0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwNDAwMDAg
ZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEz
OTNdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzk1XSBQbGF0Zm9ybSBNU0k6IElU
U0AweDEwMDEwMDA4MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTM5Nl0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOTdd
IFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzk4XSBQbGF0Zm9ybSBNU0k6IElUU0Aw
eDEwMDEwMDBlMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMTQwMF0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWlu
IGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDFdIFBs
YXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDA1XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAw
NDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDE0MDZdIFBDSS9NU0k6IElUU0AweDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwOF0gUENJL01TSTogSVRTQDB4
MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAxNDA5XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0
ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTJdIFBDSS9NU0k6
IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMTQxNF0gUENJL01TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFp
biBjcmVhdGVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDE1XSBQ
Q0kvTVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTZdIFBDSS9NU0k6IElUU0AweDEwMDEwMDEyMDAw
MCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MTQxOV0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDQwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDIxXSBmc2wtbWMgTVNJOiBJVFNA
MHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDE0MjJdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA4MDAwMCBkb21haW4g
Y3JlYXRlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyM10gZnNs
LW1jIE1TSTogSVRTQDB4MTAwMTAwMGEwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI0XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAw
YzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDE0MjVdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBlMDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyN10gZnNsLW1jIE1TSTog
SVRTQDB4MTAwMTAwMTAwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAxNDI4XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMjAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MzRd
IFJlbWFwcGluZyBhbmQgZW5hYmxpbmcgRUZJIHNlcnZpY2VzLgpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMzkwN10gc21wOiBCcmluZ2luZyB1cCBzZWNvbmRhcnkgQ1BV
cyAuLi4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ0MTZdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA0NDM1XSBHSUN2MzogQ1BVMTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwN2MwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA0NDUzXSBHSUN2MzogQ1BVMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA4NTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDQ0ODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDQ0OTRdIENQVTE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFhMDAwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU0MjJdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA1NDM3XSBHSUN2MzogQ1BVMjogZm91bmQgcmVkaXN0cmlidXRvciAxNDAwMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwNjQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA1NDU1XSBHSUN2MzogQ1BVMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA4NjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDU0ODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDU0OThdIENQVTI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU4OTRd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA1OTExXSBHSUN2MzogQ1BVMzogZm91bmQgcmVkaXN0cmlidXRvciAxYzAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA1OTI4XSBHSUN2MzogQ1BVMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDA4NzAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDU5NjFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDU5NzBdIENQVTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFjMDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY0
MjFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA2NDM1XSBHSUN2MzogQ1BVNDogZm91bmQgcmVkaXN0cmlidXRvciAx
MDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA2NDUzXSBHSUN2MzogQ1BVNDogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ODAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDY0ODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDY0OThdIENQVTQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEw
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDc0OTldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1ClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA3NTE1XSBHSUN2MzogQ1BVNTogZm91bmQgcmVkaXN0cmlidXRv
ciAxODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA3NTMzXSBHSUN2MzogQ1BVNTogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4OTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDc1NjddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDc1NzddIENQVTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDE4MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDc5MDRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2ClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3OTIwXSBHSUN2MzogQ1BVNjogZm91bmQgcmVkaXN0cmli
dXRvciAxNjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmMwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3OTM4XSBHSUN2MzogQ1BVNjogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4YTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDc5NzRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDc5ODRdIENQVTY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDE2MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDgyOTddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3ClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MzE1XSBHSUN2MzogQ1BVNzogZm91bmQgcmVkaXN0
cmlidXRvciAxZTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGMwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MzM0XSBHSUN2MzogQ1BVNzogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4YjAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDgzNjldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgzNzldIENQVTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDFlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDg3MDBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU4ClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzEyXSBHSUN2MzogQ1BVODogZm91bmQgcmVk
aXN0cmlidXRvciBhMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzYzAwMDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg3MzBdIEdJQ3YzOiBDUFU4OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhjMDAwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODc2OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwODc3OF0gQ1BVODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMGEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwOTA5Nl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTkKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDkxMTRdIEdJQ3YzOiBDUFU5OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDIyMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5YzAwMDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDkxMzNdIEdJQ3YzOiBDUFU5OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhkMDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTE2NF0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTE3NF0gQ1BVOTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMjIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwOTUwMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NTE1XSBHSUN2MzogQ1BVMTA6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgYzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDQwMDAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NTM0XSBHSUN2MzogQ1BVMTA6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGUwMDAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NTcwXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NTgxXSBDUFUxMDogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMGMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwOTg4MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEx
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5OTAwXSBHSUN2MzogQ1BV
MTE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGE0MDAw
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTkxOV0gR0lDdjM6IENQ
VTExOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhmMDAw
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTk1Ml0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTk2Ml0gQ1BVMTE6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI0MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyODddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUxMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDI5OV0gR0lD
djM6IENQVTEyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDgwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDM0MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDMxOF0gR0lD
djM6IENQVTEyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MDkwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDM1M10gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDM2NV0gQ1BVMTI6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA4MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA2ODddIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFUxMwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDcw
Nl0gR0lDdjM6IENQVTEzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIwMDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA5NDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3
MjVdIEdJQ3YzOiBDUFUxMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDA5MTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3
NThdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3NjldIENQ
VTEzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMDAwMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExMDgyXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVMTQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTEwOTZdIEdJQ3YzOiBDUFUxNDogZm91bmQgcmVkaXN0cmlidXRvciBlMDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA0YzAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTExMTddIEdJQ3YzOiBDUFUxNDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDA5MjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTExNTRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEx
NjZdIENQVTE0OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZTAwMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNDgzXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTE1MDNdIEdJQ3YzOiBDUFUxNTogZm91bmQgcmVkaXN0cmlidXRvciAyNjAwMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwYWMwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDExNTIzXSBHSUN2MzogQ1BVMTU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTMwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDExNTU3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDExNTY4XSBDUFUxNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjYwMDAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTg3
N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE2ClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDExODg3XSBHSUN2MzogQ1BVMTY6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMWMwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDExOTA4XSBHSUN2MzogQ1BVMTY6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDExOTQ2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDExOTU4XSBDUFUxNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MDIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMjI4Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE3ClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyMjkzXSBHSUN2MzogQ1BVMTc6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMjQwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyMzE1XSBHSUN2MzogQ1BVMTc6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTUwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyMzUzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDEyMzY1XSBDUFUxNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMDQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMjY4MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE4ClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNjkxXSBHSUN2MzogQ1BVMTg6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMCByZWdpb24gMDoweDAwMDAxMDAxMDAxNDAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI3MTNdIEdJQ3YzOiBDUFUxODogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NjAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI3NTFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTI3NjNdIENQVTE4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAwMDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDEzMDgzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTkKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMwOTVdIEdJQ3YzOiBDUFUxOTogZm91bmQg
cmVkaXN0cmlidXRvciA2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyYzAwMDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMxMTZdIEdJQ3YzOiBDUFUxOTogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NzAwMDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMxNTNdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMxNjZdIENQVTE5OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAwNjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDEzNDg0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM0OThdIEdJQ3YzOiBDUFUyMDog
Zm91bmQgcmVkaXN0cmlidXRvciAxMzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjAwMDAwClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNTE4XSBHSUN2MzogQ1BVMjA6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTgwMDAwClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNTUzXSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNTY1XSBDUFUyMDogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzg4MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTIxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzODk4XSBHSUN2Mzog
Q1BVMjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDgw
MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzkxOF0gR0lDdjM6
IENQVTIxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk5
MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzk1Ml0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzk2NF0gQ1BVMjE6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQyNzldIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFUyMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDI5NF0g
R0lDdjM6IENQVTIyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA2ODAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQzMTVd
IEdJQ3YzOiBDUFUyMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5YTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQzNTFd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQzNjNdIENQVTIy
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0Njc2XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMjMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTQ2OTNdIEdJQ3YzOiBDUFUyMzogZm91bmQgcmVkaXN0cmlidXRvciAxZDAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwODgwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE0NzE0XSBHSUN2MzogQ1BVMjM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwOWIwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE0NzQ4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0NzYw
XSBDUFUyMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWQwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTA3OF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1MDkyXSBHSUN2MzogQ1BVMjQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTEwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDU4MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNTExMl0gR0lDdjM6IENQVTI0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDljMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNTE0N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNTE2MF0gQ1BVMjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDExMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTU0OTNd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNTUxMF0gR0lDdjM6IENQVTI1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE5
MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3ODAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTU1MzFdIEdJQ3YzOiBDUFUyNTogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ZDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTU1NjRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTU1NzddIENQVTI1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAx
OTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE1ODg0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjYKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTU5MDBdIEdJQ3YzOiBDUFUyNjogZm91bmQgcmVkaXN0cmli
dXRvciAxNzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzAwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTIyXSBHSUN2MzogQ1BVMjY6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWUwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTU3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE1OTcwXSBDUFUyNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMTcwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNjI4NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI3ClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2MzAzXSBHSUN2MzogQ1BVMjc6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMWYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDkwMDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjMyNF0gR0lDdjM6IENQVTI3OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlmMDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjM1OV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjM3Ml0gQ1BVMjc6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDFmMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTY2ODJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyOApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjY5NF0gR0lDdjM6IENQVTI4OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIGIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQwMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjcxNl0gR0lDdjM6IENQVTI4OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEwMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjc1M10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjc2Nl0gQ1BVMjg6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDBiMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyODldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUy
OQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzMwOF0gR0lDdjM6IENQ
VTI5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhMDAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczMjldIEdJQ3YzOiBD
UFUyOTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMTAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczNjBdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczNzNdIENQVTI5OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3Njg0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVMzAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc2OThdIEdJ
Q3YzOiBDUFUzMDogZm91bmQgcmVkaXN0cmlidXRvciBkMDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA0ODAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc3MTldIEdJ
Q3YzOiBDUFUzMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBhMjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc3NTVdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc3NjldIENQVTMwOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MDg5XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVMzEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTgx
MDhdIEdJQ3YzOiBDUFUzMTogZm91bmQgcmVkaXN0cmlidXRvciAyNTAwMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwYTgwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4
MTMwXSBHSUN2MzogQ1BVMzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYTMwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4
MTYyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MTc2XSBD
UFUzMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjUwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODQ4N10gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTMyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4NTAwXSBHSUN2MzogQ1BVMzI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgOTAwMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwMzgwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4NTIzXSBHSUN2MzogQ1BVMzI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwYTQwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4NTU4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4
NTczXSBDUFUzMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDkwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODg4OV0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTMzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE4OTA3XSBHSUN2MzogQ1BVMzM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjEwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk4MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxODkzMF0gR0lDdjM6IENQVTMzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGE1MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxODk2NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxODk3OF0gQ1BVMzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIxMDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTky
ODVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxOTI5OV0gR0lDdjM6IENQVTM0OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IGYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxOTMyM10gR0lDdjM6IENQVTM0OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE2MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxOTM2MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxOTM3NV0gQ1BVMzQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDBmMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTk2OTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTcxMF0gR0lDdjM6IENQVTM1OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDI3MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBiMDAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk3MzNdIEdJQ3YzOiBDUFUzNTogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNzAwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk3NjhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTk3ODJdIENQVTM1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAyNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIwMDg3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzYKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAwOThdIEdJQ3YzOiBDUFUzNjogZm91bmQg
cmVkaXN0cmlidXRvciAzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyMDAwMDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxMjJdIEdJQ3YzOiBDUFUzNjogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhODAwMDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxNjBdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxNzVdIENQVTM2OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAwMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIwNDg2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzcKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0OTddIEdJQ3YzOiBDUFUzNzog
Zm91bmQgcmVkaXN0cmlidXRvciA1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyODAwMDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1MjFdIEdJQ3YzOiBDUFUzNzog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhOTAwMDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1NTldIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1NzRdIENQVTM3OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODg4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
MzgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA4OThdIEdJQ3YzOiBD
UFUzODogZm91bmQgcmVkaXN0cmlidXRvciAxMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxODAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA5MjRdIEdJQ3YzOiBD
UFUzODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYTAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA5NjJdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA5NzhdIENQVTM4OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMjg3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVMzkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjEyOTldIEdJ
Q3YzOiBDUFUzOTogZm91bmQgcmVkaXN0cmlidXRvciA3MDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAzMDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjEzMjNdIEdJ
Q3YzOiBDUFUzOTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBhYjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjEzNjBdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjEzNzZdIENQVTM5OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNjg0XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2
OThdIEdJQ3YzOiBDUFU0MDogZm91bmQgcmVkaXN0cmlidXRvciAxMjAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNWUwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIx
NzIyXSBHSUN2MzogQ1BVNDA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYWMwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIx
NzU1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNzcwXSBD
UFU0MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTIwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjA4N10gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTQxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIyMTAzXSBHSUN2MzogQ1BVNDE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWEwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDdlMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMjEyN10gR0lDdjM6IENQVTQxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGFkMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMjE2MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
MjE3NV0gQ1BVNDE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFhMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI0ODRdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMjQ5OV0gR0lDdjM6IENQVTQyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA2NjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjI1MjFdIEdJQ3YzOiBDUFU0MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBhZTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjI1NTZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjI1NzJdIENQVTQyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNDAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIy
ODg1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjI5MDFdIEdJQ3YzOiBDUFU0MzogZm91bmQgcmVkaXN0cmlidXRv
ciAxYzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIyOTI1XSBHSUN2MzogQ1BVNDM6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYWYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIyOTU4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIyOTczXSBDUFU0MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMWMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMzI4M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ0ClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjk2XSBHSUN2MzogQ1BVNDQ6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTAwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDU2MDAwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzMyMV0gR0lDdjM6IENQVTQ0OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIwMDAwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzM1Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMzM3MV0gQ1BVNDQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDEwMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjM2ODZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzcwMl0gR0lDdjM6IENQVTQ1OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDE4MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3NjAwMDAKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjM3MjZdIEdJQ3YzOiBDUFU0NTogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMTAwMDAKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjM3NTldIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjM3NzNdIENQVTQ1OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAxODAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI0MDg5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDYK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQxMDRdIEdJQ3YzOiBDUFU0
NjogZm91bmQgcmVkaXN0cmlidXRvciAxNjAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmUwMDAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0MTMwXSBHSUN2MzogQ1BV
NDY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjIwMDAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0MTY1XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0MTgwXSBDUFU0NjogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ4N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTQ3ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0NTA1XSBHSUN2
MzogQ1BVNDc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDhlMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDUyOV0gR0lD
djM6IENQVTQ3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGIzMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDU2NF0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDU4MF0gQ1BVNDc6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFlMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4NzZdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU0OApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDg4
OF0gR0lDdjM6IENQVTQ4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDNlMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDkx
M10gR0lDdjM6IENQVTQ4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGI0MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDk1
MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDk2Nl0gQ1BV
NDg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBhMDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUyODBdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU0OQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNTI5OV0gR0lDdjM6IENQVTQ5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIyMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA5ZTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjUzMjRdIEdJQ3YzOiBDUFU0OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBiNTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjUzNTldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUz
NzVdIENQVTQ5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NjgwXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjU2OTNdIEdJQ3YzOiBDUFU1MDogZm91bmQgcmVkaXN0cmlidXRvciBjMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA0NjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjU3MTddIEdJQ3YzOiBDUFU1MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBiNjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjU3NTJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjU3NjldIENQVTUwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYzAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDg3
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjYxMDZdIEdJQ3YzOiBDUFU1MTogZm91bmQgcmVkaXN0cmlidXRvciAy
NDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI2MTMxXSBHSUN2MzogQ1BVNTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjcwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI2MTYzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDI2MTc5XSBDUFU1MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MjQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNjQ4NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUyClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NDk1XSBHSUN2MzogQ1BVNTI6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgODAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzYwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NTIxXSBHSUN2MzogQ1BVNTI6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjgwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NTU5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI2NTc2XSBDUFU1MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMDgwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNjg5NV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUzClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2OTEzXSBHSUN2MzogQ1BVNTM6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjAwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk2MDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjkzOV0gR0lDdjM6IENQVTUzOiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI5MDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjk3M10gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjk5MF0gQ1BVNTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDIwMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjcyOTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMwM10gR0lDdjM6IENQVTU0OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIGUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRlMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMzMF0gR0lDdjM6IENQVTU0OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJhMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzM2N10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzM4NF0gQ1BVNTQ6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDBlMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2ODNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1
NQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzcwM10gR0lDdjM6IENQ
VTU1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhZTAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3MjldIEdJQ3YzOiBD
UFU1NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiYjAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3NjRdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3ODFdIENQVTU1OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MDgzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNTYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwOTRdIEdJ
Q3YzOiBDUFU1NjogZm91bmQgcmVkaXN0cmlidXRvciAyMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAxZTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxMjFdIEdJ
Q3YzOiBDUFU1NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBiYzAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxNThdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxNzZdIENQVTU2OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDgzXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNTcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg0
OTRdIEdJQ3YzOiBDUFU1NzogZm91bmQgcmVkaXN0cmlidXRvciA0MDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDAyNjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg1
MjFdIEdJQ3YzOiBDUFU1NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBiZDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg1
NTldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg1NzZdIENQ
VTU3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4ODg2XSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVNTgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjg4OTVdIEdJQ3YzOiBDUFU1ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwMTYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI4OTIzXSBHSUN2MzogQ1BVNTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYmUwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI4OTYxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4OTc5
XSBDUFU1ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDAwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTI4OF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU5ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5MzAwXSBHSUN2MzogQ1BVNTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNjAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwMmUwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5MzI3XSBHSUN2MzogQ1BVNTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYmYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5MzYzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5MzgyXSBDUFU1OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDYwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTY4OV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI5NzA0XSBHSUN2MzogQ1BVNjA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTMw
MTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDYyMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyOTczMV0gR0lDdjM6IENQVTYwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyOTc2OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyOTc4NF0gQ1BVNjA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEz
MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzAwOTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAzMDEwOF0gR0lDdjM6IENQVTYxOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDFiMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4MjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzAxMzNdIEdJQ3YzOiBDUFU2MTogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzAxNjddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzAxODVdIENQVTYxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxYjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDMwNDkxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjIKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1MDddIEdJQ3YzOiBDUFU2MjogZm91bmQgcmVk
aXN0cmlidXRvciAxNTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmEwMDAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNTM0XSBHSUN2MzogQ1BVNjI6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzIwMDAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNTY5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNTg2XSBDUFU2MjogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMTUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzMDg5MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYzClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwOTA5XSBHSUN2MzogQ1BVNjM6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMWQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDhhMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDkzNV0gR0lDdjM6IENQVTYzOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMzMDAwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDk2OF0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDk4NV0gQ1BVNjM6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDFkMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzEyODNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2
NApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI5OF0gR0lDdjM6IENQ
VTY0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDExMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1YTAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzEzMjVdIEdJQ3YzOiBD
UFU2NDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNDAw
MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzEzNjBdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzEzNzZdIENQVTY0OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNjg3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNjUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE3MDRdIEdJ
Q3YzOiBDUFU2NTogZm91bmQgcmVkaXN0cmlidXRvciAxOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwN2EwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNzMxXSBH
SUN2MzogQ1BVNjU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwYzUwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNzY2XSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNzgyXSBDUFU2NTog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTkwMTAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjA4N10gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTY2ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMy
MTAzXSBHSUN2MzogQ1BVNjY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTcwMTAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDcyMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MjEzMF0gR0lDdjM6IENQVTY2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMGM2MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MjE2N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjE4M10g
Q1BVNjY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE3MDEwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI0OTFdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFU2NwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMjUwOV0gR0lDdjM6IENQVTY3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFmMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA5MjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzI1MzddIEdJQ3YzOiBDUFU2NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBjNzAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzI1NzJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzI1ODhdIENQVTY3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZjAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyODkwXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI5MDJdIEdJQ3YzOiBDUFU2ODogZm91bmQgcmVkaXN0cmlidXRvciBiMDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA0MjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI5MzBdIEdJQ3YzOiBDUFU2ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBjODAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI5NjddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzI5ODNdIENQVTY4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYjAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMz
Mjg2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzMzMDVdIEdJQ3YzOiBDUFU2OTogZm91bmQgcmVkaXN0cmlidXRv
ciAyMzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTIwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMzMzMzXSBHSUN2MzogQ1BVNjk6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzkwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMzMzY1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDMzMzgxXSBDUFU2OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMjMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMzY4NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNjk3XSBHSUN2MzogQ1BVNzA6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGEwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNzI2XSBHSUN2MzogQ1BVNzA6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2EwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNzYyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMzNzc4XSBDUFU3MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMGQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNDA5Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcxClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0MTEyXSBHSUN2MzogQ1BVNzE6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMjUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFhMDAwMApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE0MF0gR0lDdjM6IENQVTcxOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNiMDAwMApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE3MV0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE4OF0gQ1BVNzE6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDI1MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzQ0OTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3MgpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDUwNF0gR0lDdjM6IENQVTcy
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDkwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNhMDAwMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDUzMl0gR0lDdjM6IENQVTcy
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNjMDAwMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDU3MF0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDU4OF0gQ1BVNzI6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA5MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4OTRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFU3MwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDkxMl0gR0lDdjM6
IENQVTczOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIxMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5
YTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5NDFdIEdJQ3Yz
OiBDUFU3MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBj
ZDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5NzNdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5OTFdIENQVTczOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MjgzXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNzQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUyOTdd
IEdJQ3YzOiBDUFU3NDogZm91bmQgcmVkaXN0cmlidXRvciBmMDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA1MjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUzMjZd
IEdJQ3YzOiBDUFU3NDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBjZTAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUzNjNd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUzODFdIENQVTc0
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZjAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NjkwXSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNzUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzU3MTBdIEdJQ3YzOiBDUFU3NTogZm91bmQgcmVkaXN0cmlidXRvciAyNzAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwYjIwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM1NzM5XSBHSUN2MzogQ1BVNzU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwY2YwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM1NzczXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1Nzkx
XSBDUFU3NTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjcwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjA5Ml0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc2ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM2MTAyXSBHSUN2MzogQ1BVNzY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMzAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwMjIwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM2MTMzXSBHSUN2MzogQ1BVNzY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwZDAwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM2MTcwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM2MTg5XSBDUFU3NjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDMwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjQ5MV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc3ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2NTAyXSBHSUN2MzogQ1BVNzc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNTAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMmEwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2NTMyXSBHSUN2MzogQ1BVNzc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDEwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2NTcwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM2NTg5XSBDUFU3NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDUw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
Njg4NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc4ClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM2ODk0XSBHSUN2MzogQ1BVNzg6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgMTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMWEwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM2OTI1XSBHSUN2MzogQ1BVNzg6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDIwMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM2OTYzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2OTgyXSBDUFU3ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMDEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNzI4N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc5ClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM3Mjk5XSBHSUN2MzogQ1BVNzk6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzIwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM3MzI5XSBHSUN2MzogQ1BVNzk6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDMwMDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM3MzY4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM3Mzg3XSBDUFU3OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMDcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNzQ0OF0gc21wOiBCcm91Z2h0IHVwIDEgbm9kZSwgODAgQ1BVcwpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNzQ2OF0gU01QOiBUb3RhbCBvZiA4MCBw
cm9jZXNzb3JzIGFjdGl2YXRlZC4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzc0NjldIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFByaXZpbGVnZWQgQWNjZXNzIE5ldmVy
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM3NDcwXSBDUFUgZmVhdHVy
ZXM6IGRldGVjdGVkOiBMU0UgYXRvbWljIGluc3RydWN0aW9ucwpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAzNzQ3MV0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogVXNlciBB
Y2Nlc3MgT3ZlcnJpZGUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzc0
NzJdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IDMyLWJpdCBFTDAgU3VwcG9ydApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNzQ3Ml0gQ1BVIGZlYXR1cmVzOiBkZXRlY3Rl
ZDogQ29tbW9uIG5vdCBQcml2YXRlIHRyYW5zbGF0aW9ucwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNzQ3M10gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogRGF0YSBjYWNo
ZSBjbGVhbiB0byBQb2ludCBvZiBQZXJzaXN0ZW5jZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzNzQ3NF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogUkFTIEV4dGVuc2lv
biBTdXBwb3J0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM3NDc0XSBD
UFUgZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIHRoZSBQb1Ugbm90IHJl
cXVpcmVkIGZvciBJL0QgY29oZXJlbmNlClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM3NDc1XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDUkMzMiBpbnN0cnVjdGlvbnMK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzc0NzVdIENQVSBmZWF0dXJl
czogZGV0ZWN0ZWQ6IFNwZWN1bGF0aXZlIFN0b3JlIEJ5cGFzc2luZyBTYWZlIChTU0JTKQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjA1OTkwMF0gQ1BVOiBBbGwgQ1BVKHMp
IHN0YXJ0ZWQgYXQgRUwyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ5
MDgwXSBub2RlIDAgZGVmZXJyZWQgcGFnZXMgaW5pdGlhbGlzZWQgaW4gNDg4bXMKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDkyMTZdIGRldnRtcGZzOiBpbml0aWFsaXpl
ZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0OTc5N10gUmVnaXN0ZXJl
ZCBjcDE1X2JhcnJpZXIgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NDk4MTldIFJlZ2lzdGVyZWQgc2V0ZW5kIGVtdWxhdGlvbiBoYW5kbGVy
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ5ODIxXSBLQVNMUiBlbmFi
bGVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ5ODgwXSBjbG9ja3Nv
dXJjZTogamlmZmllczogbWFzazogMHhmZmZmZmZmZiBtYXhfY3ljbGVzOiAweGZmZmZmZmZmLCBt
YXhfaWRsZV9uczogNzY0NTA0MTc4NTEwMDAwMCBucwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU0OTkzMl0gZnV0ZXggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3Jk
ZXI6IDksIDIwOTcxNTIgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NTAzNjRdIHBpbmN0cmwgY29yZTogaW5pdGlhbGl6ZWQgcGluY3RybCBzdWJz
eXN0ZW0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTA1MjVdIFNNQklP
UyAzLjMuMCBwcmVzZW50LgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1
MDUyOF0gRE1JOiBHSUdBQllURSBSMTUyLVAzMC0wMC9NUDMyLUFSMS0wMCwgQklPUyBGMTMgKFND
UDogMS41LjIwMjEwNDI2KSAwNS8yMS8yMDIxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNTUwNzU5XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDE2ClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUxMzQ2XSBETUE6IHByZWFsbG9jYXRl
ZCA0MDk2IEtpQiBHRlBfS0VSTkVMIHBvb2wgZm9yIGF0b21pYyBhbGxvY2F0aW9ucwpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MTQ3Ml0gRE1BOiBwcmVhbGxvY2F0ZWQg
NDA5NiBLaUIgR0ZQX0tFUk5FTHxHRlBfRE1BIHBvb2wgZm9yIGF0b21pYyBhbGxvY2F0aW9ucwpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MTU5Nl0gRE1BOiBwcmVhbGxv
Y2F0ZWQgNDA5NiBLaUIgR0ZQX0tFUk5FTHxHRlBfRE1BMzIgcG9vbCBmb3IgYXRvbWljIGFsbG9j
YXRpb25zClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUxNjAzXSBhdWRp
dDogaW5pdGlhbGl6aW5nIG5ldGxpbmsgc3Vic3lzIChkaXNhYmxlZCkKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTE2NDZdIGF1ZGl0OiB0eXBlPTIwMDAgYXVkaXQoMC41
NDg6MSk6IHN0YXRlPWluaXRpYWxpemVkIGF1ZGl0X2VuYWJsZWQ9MCByZXM9MQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MTczMV0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVy
ZWQgdGhlcm1hbCBnb3Zlcm5vciAnZmFpcl9zaGFyZScKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NTE3MzFdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292
ZXJub3IgJ2JhbmdfYmFuZycKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41
NTE3MzJdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3N0ZXBfd2lz
ZScKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTE3MzNdIHRoZXJtYWxf
c3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3VzZXJfc3BhY2UnClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUxNzM0XSB0aGVybWFsX3N5czogUmVnaXN0ZXJl
ZCB0aGVybWFsIGdvdmVybm9yICdwb3dlcl9hbGxvY2F0b3InClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNTUxNzg3XSBjcHVpZGxlOiB1c2luZyBnb3Zlcm5vciBsYWRkZXIK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTE3OTRdIGNwdWlkbGU6IHVz
aW5nIGdvdmVybm9yIG1lbnUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41
NTE4OTFdIERldGVjdGVkIDE1IFBDQyBTdWJzcGFjZXMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NTE5MDZdIFJlZ2lzdGVyaW5nIFBDQyBkcml2ZXIgYXMgTWFpbGJveCBj
b250cm9sbGVyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUxOTI1XSBo
dy1icmVha3BvaW50OiBmb3VuZCA2IGJyZWFrcG9pbnQgYW5kIDQgd2F0Y2hwb2ludCByZWdpc3Rl
cnMuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUyMzEyXSBBU0lEIGFs
bG9jYXRvciBpbml0aWFsaXNlZCB3aXRoIDMyNzY4IGVudHJpZXMKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC41NTIzMzZdIEFDUEk6IGJ1cyB0eXBlIFBDSSByZWdpc3RlcmVk
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUyMzM4XSBhY3BpcGhwOiBB
Q1BJIEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjUKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTIzODFdIFNlcmlhbDogQU1CQSBQTDAxMSBV
QVJUIGRyaXZlcgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1NDc0M10g
SHVnZVRMQiByZWdpc3RlcmVkIDEuMDAgR2lCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBh
Z2VzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTU0NzQ1XSBIdWdlVExC
IHJlZ2lzdGVyZWQgMzIuMCBNaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTQ3NDVdIEh1Z2VUTEIgcmVnaXN0
ZXJlZCAyLjAwIE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1NDc0Nl0gSHVnZVRMQiByZWdpc3RlcmVkIDY0
LjAgS2lCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjgyMzk0XSBBQ1BJOiBBZGRlZCBfT1NJKE1vZHVsZSBEZXZp
Y2UpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjgyMzk1XSBBQ1BJOiBB
ZGRlZCBfT1NJKFByb2Nlc3NvciBEZXZpY2UpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjgyMzk2XSBBQ1BJOiBBZGRlZCBfT1NJKDMuMCBfU0NQIEV4dGVuc2lvbnMpClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjgyMzk3XSBBQ1BJOiBBZGRlZCBf
T1NJKFByb2Nlc3NvciBBZ2dyZWdhdG9yIERldmljZSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42ODIzOThdIEFDUEk6IEFkZGVkIF9PU0koTGludXgtRGVsbC1WaWRlbykK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42ODI0MDBdIEFDUEk6IEFkZGVk
IF9PU0koTGludXgtTGVub3ZvLU5WLUhETUktQXVkaW8pClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjgyNDAxXSBBQ1BJOiBBZGRlZCBfT1NJKExpbnV4LUhQSS1IeWJyaWQt
R3JhcGhpY3MpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg5MzczXSBB
Q1BJOiAyIEFDUEkgQU1MIHRhYmxlcyBzdWNjZXNzZnVsbHkgYWNxdWlyZWQgYW5kIGxvYWRlZApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5MDczMV0gQUNQSTogSW50ZXJw
cmV0ZXIgZW5hYmxlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5MDcz
Ml0gQUNQSTogVXNpbmcgR0lDIGZvciBpbnRlcnJ1cHQgcm91dGluZwpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5MDc0MF0gQUNQSTogTUNGRyB0YWJsZSBkZXRlY3RlZCwg
OCBlbnRyaWVzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NTk2XSBB
Uk1IMDAxMTowMDogdHR5QU1BMCBhdCBNTUlPIDB4MTAwMDAyNjAwMDAwIChpcnEgPSAzNiwgYmFz
ZV9iYXVkID0gMCkgaXMgYSBTQlNBClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk5NjM4XSBwcmludGs6IGNvbnNvbGUgW3R0eUFNQTBdIGVuYWJsZWQKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA3NTFdIEFSTUgwMDExOjAxOiB0dHlBTUExIGF0
IE1NSU8gMHgxMDAwMDI2MjAwMDAgKGlycSA9IDM3LCBiYXNlX2JhdWQgPSAwKSBpcyBhIFNCU0EK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE4MThdIEFDUEk6IFBDSSBS
b290IEJyaWRnZSBbUENJMF0gKGRvbWFpbiAwMDBjIFtidXMgMDAtZmZdKQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTgyNV0gYWNwaSBQTlAwQTA4OjAwOiBfT1NDOiBP
UyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgt
VHlwZTNdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxODk0XSBhY3Bp
IFBOUDBBMDg6MDA6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVn
IFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAxOTU0XSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBD
SWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTk1
OF0gYWNwaSBQTlAwQTA4OjAwOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgzM2ZmZjAwMDAw
MDAtMHgzM2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmFhMmZmNzc3ZWE1
OApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjU4OV0gYWNwaSBQTlAw
QTA4OjAwOiBFQ0FNIGFyZWEgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gcmVz
ZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwMjU5Nl0gYWNwaSBQTlAwQTA4OjAwOiBFQ0FNIGF0IFttZW0gMHgzM2ZmZjAwMDAwMDAtMHgz
M2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMjY2Ml0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBjOjAwClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNjYzXSBwY2lfYnVzIDAwMGM6MDA6IHJv
b3QgYnVzIHJlc291cmNlIFttZW0gMHg0MDAwMDAwMC0weDRmZmZmZmZmIHdpbmRvd10KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDI2NjRdIHBjaV9idXMgMDAwYzowMDog
cm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDMwMDAwMDAwMDAwMC0weDMzZmZkZmZmZmZmZiB3aW5k
b3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNjY1XSBwY2lfYnVz
IDAwMGM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNjg1XSBwY2kgMDAwYzowMDowMC4wOiBbMWRlZjplMTAw
XSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAyNzMzXSBwY2kgMDAwYzowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBlIDAxIGNsYXNz
IDB4MDYwNDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNzQxXSBw
Y2kgMDAwYzowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNzU3XSBwY2kgMDAwYzowMDowMS4wOiBzdXBwb3J0cyBE
MSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjc1OF0gcGNpIDAw
MGM6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjgzNF0gYWNwaXBocDogU2xvdCBbMV0gcmVnaXN0
ZXJlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjg0N10gcGNpIDAw
MGM6MDE6MDAuMDogWzEwMDA6MDA5N10gdHlwZSAwMCBjbGFzcyAweDAxMDcwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMjg1Nl0gcGNpIDAwMGM6MDE6MDAuMDogcmVn
IDB4MTA6IFtpbyAgMHg0ZmZmODAwMC0weDRmZmY4MGZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMjg2NF0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MTQ6IFttZW0g
MHg0MDE0MDAwMC0weDQwMTRmZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMjg3Ml0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MWM6IFttZW0gMHg0MDEw
MDAwMC0weDQwMTNmZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMjg4MV0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg0MDAwMDAwMC0w
eDQwMGZmZmZmIHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAy
ODg1XSBwY2kgMDAwYzowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyOTgwXSBwY2kgMDAwYzowMTowMC4wOiBzdXBw
b3J0cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzA1Ml0g
cGNpX2J1cyAwMDBjOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMzA1Nl0gcGNpIDAwMGM6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAy
MDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMzA1OV0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NDAw
MDAwMDAtMHg0MDFmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDMwNjFdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDMwMDAwMDAw
MDAwMC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMzA2Ml0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDMwNjNdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzA2
NF0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMwNjVdIHBjaSAwMDBj
OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzA2Nl0gcGNpIDAwMGM6MDE6MDAu
MDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMGZmZmZmIHByZWZdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMDY4XSBwY2kgMDAwYzowMTowMC4w
OiBCQVIgMzogYXNzaWduZWQgW21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYgNjRiaXRdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMDczXSBwY2kgMDAwYzowMTowMC4w
OiBCQVIgMTogYXNzaWduZWQgW21lbSAweDQwMTQwMDAwLTB4NDAxNGZmZmYgNjRiaXRdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMDc4XSBwY2kgMDAwYzowMTowMC4w
OiBCQVIgMDogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAxMDBdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMDc5XSBwY2kgMDAwYzowMTowMC4wOiBCQVIgMDogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMTAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMzA4MF0gcGNpIDAwMGM6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVz
IDAxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzA4Ml0gcGNpIDAw
MGM6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzA4M10gcGNpIDAwMGM6MDA6
MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMwODRd
IHBjaV9idXMgMDAwYzowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25l
ZCwgdHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMzA4Nl0gcGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA0IFttZW0gMHg0
MDAwMDAwMC0weDRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDMwODddIHBjaV9idXMgMDAwYzowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzAwMDAw
MDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDMwODddIHBjaV9idXMgMDAwYzowMTogcmVzb3VyY2UgMSBbbWVtIDB4NDAw
MDAwMDAtMHg0MDFmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDMwODhdIHBjaV9idXMgMDAwYzowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4
MzAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAzMTk1XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTFdIChkb21haW4gMDAwZCBb
YnVzIDAwLWZmXSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMxOTld
IGFjcGkgUE5QMEEwODowMTogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0g
Q2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMzI2NF0gYWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBwbGF0Zm9ybSBkb2Vz
IG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzMyM10gYWNwaSBQTlAwQTA4OjAxOiBfT1ND
OiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDMzMjVdIGFjcGkgUE5QMEEwODowMTogTUNGRyBxdWlyazog
RUNBTSBhdCBbbWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1m
Zl0gd2l0aCAweGZmZmZhYTJmZjc3N2VhNTgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDM5NTFdIGFjcGkgUE5QMEEwODowMTogRUNBTSBhcmVhIFttZW0gMHgzN2ZmZjAw
MDAwMDAtMHgzN2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM5NTddIGFjcGkgUE5QMEEwODowMTogRUNBTSBh
dCBbbWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQwMTddIFBDSSBob3N0IGJyaWRn
ZSB0byBidXMgMDAwZDowMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
NDAxOF0gcGNpX2J1cyAwMDBkOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NTAwMDAwMDAt
MHg1ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA0MDE5XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzNDAwMDAw
MDAwMDAtMHgzN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwNDAyMF0gcGNpX2J1cyAwMDBkOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVz
IDAwLWZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDAzN10gcGNp
IDAwMGQ6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDA3Nl0gcGNpIDAwMGQ6MDA6MDEuMDog
WzFkZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNDA4M10gcGNpIDAwMGQ6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5k
ZWQgVGFncwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDA5OV0gcGNp
IDAwMGQ6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDQxMDBdIHBjaSAwMDBkOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20g
RDAgRDEgRDNob3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQxNTFd
IHBjaSAwMDBkOjAwOjAyLjA6IFsxZGVmOmUxMDJdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQxNzFdIHBjaSAwMDBkOjAwOjAy
LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA0MTcyXSBwY2kgMDAwZDowMDowMi4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0MjIwXSBwY2kgMDAwZDow
MDowMy4wOiBbMWRlZjplMTAzXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0MjQwXSBwY2kgMDAwZDowMDowMy4wOiBzdXBwb3J0
cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDI0MF0gcGNp
IDAwMGQ6MDA6MDMuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDI4Nl0gcGNpIDAwMGQ6MDA6MDQuMDogWzFk
ZWY6ZTEwNF0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwNDMwN10gcGNpIDAwMGQ6MDA6MDQuMDogc3VwcG9ydHMgRDEgRDIKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQzMDhdIHBjaSAwMDBkOjAwOjA0
LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDQzNzZdIGFjcGlwaHA6IFNsb3QgWzEtMl0gcmVnaXN0ZXJlZApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDM5Ml0gcGNpIDAwMGQ6MDE6
MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDQwNF0gcGNpIDAwMGQ6MDE6MDAuMDogcmVnIDB4MTA6
IFttZW0gMHg1MDMxMDAwMC0weDUwMzEzZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNDQyN10gcGNpIDAwMGQ6MDE6MDAuMDogcmVnIDB4MzA6IFttZW0g
MHg1MDMwMDAwMC0weDUwMzBmZmZmIHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA0NDM2XSBwY2kgMDAwZDowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdz
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0NTg3XSBhY3BpcGhwOiBT
bG90IFsyXSByZWdpc3RlcmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA0NjAxXSBwY2kgMDAwZDowMjowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEw
ODAyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0NjEyXSBwY2kgMDAw
ZDowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0NjMzXSBwY2kgMDAwZDowMjow
MC4wOiByZWcgMHgzMDogW21lbSAweDUwMjAwMDAwLTB4NTAyMGZmZmYgcHJlZl0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ2MzZdIHBjaSAwMDBkOjAyOjAwLjA6IGVu
YWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDQ3NzhdIGFjcGlwaHA6IFNsb3QgWzNdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDQ3OTNdIHBjaSAwMDBkOjAzOjAwLjA6IFs4MDg2OjBhNTRd
IHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDQ4MDRdIHBjaSAwMDBkOjAzOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4NTAxMTAwMDAt
MHg1MDExM2ZmZiA2NGJpdF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDQ4MjZdIHBjaSAwMDBkOjAzOjAwLjA6IHJlZyAweDMwOiBbbWVtIDB4NTAxMDAwMDAtMHg1MDEw
ZmZmZiBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDgyOV0g
cGNpIDAwMGQ6MDM6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDk3M10gYWNwaXBocDogU2xvdCBbNF0gcmVnaXN0ZXJl
ZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDk4N10gcGNpIDAwMGQ6
MDQ6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDk5OF0gcGNpIDAwMGQ6MDQ6MDAuMDogcmVnIDB4
MTA6IFttZW0gMHg1MDAxMDAwMC0weDUwMDEzZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNTAxOV0gcGNpIDAwMGQ6MDQ6MDAuMDogcmVnIDB4MzA6IFtt
ZW0gMHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA1MDIzXSBwY2kgMDAwZDowNDowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBU
YWdzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTQ3XSBwY2lfYnVz
IDAwMGQ6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA1MTQ5XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAw
LTB4MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNTE1MF0gcGNpIDAwMGQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6
ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNTE1Ml0gcGNpIDAwMGQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAx
MDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNTNdIHBjaSAw
MDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAy
XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1
MTU0XSBwY2kgMDAwZDowMDowMi4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAw
MGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24g
MTAwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTU1XSBwY2kg
MDAwZDowMDowMi4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0
byBbYnVzIDAyXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTE1Nl0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdl
IHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDEwMDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNTddIHBjaSAwMDBkOjAwOjAz
LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0g
dG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNTldIHBjaSAwMDBkOjAwOjAzLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDNdIGFkZF9zaXpl
IDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA1MTYwXSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAw
LTB4MGZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNTE2MV0gcGNpIDAwMGQ6MDA6MDQuMDogYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDA0XSBhZGRfc2l6
ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNTE2Ml0gcGNpIDAwMGQ6MDA6MDQuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAx
MDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNjZdIHBjaSAw
MDBkOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUwMDAwMDAwLTB4NTAxZmZmZmZd
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTY4XSBwY2kgMDAwZDow
MDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDAwMDAwMDAtMHgzNDAwMDAxZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUx
NjldIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUwMjAwMDAwLTB4
NTAzZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTcwXSBw
Y2kgMDAwZDowMDowMi4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDAyMDAwMDAtMHgz
NDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDUxNzFdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUw
NDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA1MTcyXSBwY2kgMDAwZDowMDowMy4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDA0
MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDUxNzNdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDUwNjAwMDAwLTB4NTA3ZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA1MTc0XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgzNDAwMDA2MDAwMDAtMHgzNDAwMDA3ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNzVdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA1MTc2XSBwY2kgMDAwZDowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDUxNzddIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTc4
XSBwY2kgMDAwZDowMDowMi4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxNzldIHBjaSAw
MDBkOjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTc5XSBwY2kgMDAwZDowMDowMy4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxODBdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA1MTgxXSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDUxODNdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1
MTg0XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxODVdIHBj
aSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MTg2XSBwY2kgMDAwZDowMDow
My4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxODddIHBjaSAwMDBkOjAwOjAyLjA6IEJB
UiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA1MTg4XSBwY2kgMDAwZDowMDowMi4wOiBCQVIgMTM6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDUxODhdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9y
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA1MTg5XSBwY2kgMDAwZDowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUxOTFd
IHBjaSAwMDBkOjAxOjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDAw
ZmZmZiBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTE5Ml0g
cGNpIDAwMGQ6MDE6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1MDAxMDAwMC0weDUwMDEz
ZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTE5N10g
cGNpIDAwMGQ6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTE5OF0gcGNpIDAwMGQ6MDA6MDEuMDogICBicmlkZ2Ug
d2luZG93IFttZW0gMHg1MDAwMDAwMC0weDUwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNTE5OV0gcGNpIDAwMGQ6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93
IFttZW0gMHgzNDAwMDAwMDAwMDAtMHgzNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUyMDFdIHBjaSAwMDBkOjAyOjAwLjA6IEJB
UiA2OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIwMl0gcGNpIDAwMGQ6MDI6MDAuMDogQkFS
IDA6IGFzc2lnbmVkIFttZW0gMHg1MDIxMDAwMC0weDUwMjEzZmZmIDY0Yml0XQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIwN10gcGNpIDAwMGQ6MDA6MDIuMDogUENJ
IGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwNTIwOF0gcGNpIDAwMGQ6MDA6MDIuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg1MDIwMDAw
MC0weDUwM2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIw
OV0gcGNpIDAwMGQ6MDA6MDIuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzNDAwMDAyMDAwMDAt
MHgzNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDUyMTFdIHBjaSAwMDBkOjAzOjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4
NTA0MDAwMDAtMHg1MDQwZmZmZiBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNTIxMl0gcGNpIDAwMGQ6MDM6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1
MDQxMDAwMC0weDUwNDEzZmZmIDY0Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNTIxN10gcGNpIDAwMGQ6MDA6MDMuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIxOF0gcGNpIDAwMGQ6MDA6
MDMuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIxOV0gcGNpIDAwMGQ6MDA6MDMuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUyMjFdIHBjaSAw
MDBkOjA0OjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAtMHg1MDYwZmZmZiBw
cmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIyMl0gcGNpIDAw
MGQ6MDQ6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1MDYxMDAwMC0weDUwNjEzZmZmIDY0
Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNTIyN10gcGNpIDAw
MGQ6MDA6MDQuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNTIyOF0gcGNpIDAwMGQ6MDA6MDQuMDogICBicmlkZ2Ugd2luZG93
IFttZW0gMHg1MDYwMDAwMC0weDUwN2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwNTIyOV0gcGNpIDAwMGQ6MDA6MDQuMDogICBicmlkZ2Ugd2luZG93IFttZW0g
MHgzNDAwMDA2MDAwMDAtMHgzNDAwMDA3ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDUyMzFdIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2Ug
NCBbbWVtIDB4NTAwMDAwMDAtMHg1ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA1MjMxXSBwY2lfYnVzIDAwMGQ6MDA6IHJlc291cmNlIDUgW21l
bSAweDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MjMyXSBwY2lfYnVzIDAwMGQ6MDE6IHJlc291cmNlIDEg
W21lbSAweDUwMDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA1MjMzXSBwY2lfYnVzIDAwMGQ6MDE6IHJlc291cmNlIDIgW21lbSAweDM0MDAw
MDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNTIzNF0gcGNpX2J1cyAwMDBkOjAyOiByZXNvdXJjZSAxIFttZW0g
MHg1MDIwMDAwMC0weDUwM2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNTIzNV0gcGNpX2J1cyAwMDBkOjAyOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDAyMDAw
MDAtMHgzNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDUyMzZdIHBjaV9idXMgMDAwZDowMzogcmVzb3VyY2UgMSBbbWVtIDB4NTA0
MDAwMDAtMHg1MDVmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDUyMzZdIHBjaV9idXMgMDAwZDowMzogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNDAwMDAwLTB4
MzQwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA1MjM3XSBwY2lfYnVzIDAwMGQ6MDQ6IHJlc291cmNlIDEgW21lbSAweDUwNjAwMDAw
LTB4NTA3ZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MjM4
XSBwY2lfYnVzIDAwMGQ6MDQ6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAw
MDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwNTM0N10gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kyXSAoZG9tYWluIDAwMDEgW2J1cyAw
MC1mZl0pClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA1MzUyXSBhY3Bp
IFBOUDBBMDg6MDI6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2Nr
UE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDU0MTZdIGFjcGkgUE5QMEEwODowMjogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qg
c3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDU0NzVdIGFjcGkgUE5QMEEwODowMjogX09TQzogT1Mg
bm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA1NDc3XSBhY3BpIFBOUDBBMDg6MDI6IE1DRkcgcXVpcms6IEVDQU0g
YXQgW21lbSAweDNiZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdp
dGggMHhmZmZmYWEyZmY3NzdlYTU4ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzA2MTA3XSBhY3BpIFBOUDBBMDg6MDI6IEVDQU0gYXJlYSBbbWVtIDB4M2JmZmYwMDAwMDAw
LTB4M2JmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2MTE0XSBhY3BpIFBOUDBBMDg6MDI6IEVDQU0gYXQgW21l
bSAweDNiZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2MTczXSBQQ0kgaG9zdCBicmlkZ2UgdG8g
YnVzIDAwMDE6MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDYxNzVd
IHBjaV9idXMgMDAwMTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDYwMDAwMDAwLTB4NmZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjE3
Nl0gcGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzgwMDAwMDAwMDAw
LTB4M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDYxNzZdIHBjaV9idXMgMDAwMTowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1m
Zl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDYxOTRdIHBjaSAwMDAx
OjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDYyMzddIHBjaSAwMDAxOjAwOjAxLjA6IFsxZGVm
OmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDYyNTBdIHBjaSAwMDAxOjAwOjAxLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRh
Z3MKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDYyNzZdIHBjaSAwMDAx
OjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA2Mjc3XSBwY2kgMDAwMTowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQx
IEQzaG90ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2MzY1XSBhY3Bp
cGhwOiBTbG90IFsxLTNdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDg0NTVdIHBjaV9idXMgMDAwMTowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg0NTddIHBjaSAwMDAxOjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4NDU4XSBwY2kgMDAwMTow
MDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHBy
ZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4NDU5XSBwY2kgMDAwMTowMDowMS4wOiBi
cmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRf
c2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwODQ2MV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVt
IDB4NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDg0NjJdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM4
MDAwMDAwMDAwMC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwODQ2M10gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDg0NjRdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwODQ2NV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg0NjZdIHBj
aSAwMDAxOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwODQ2N10gcGNpIDAwMDE6
MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwODQ2OV0gcGNpIDAwMDE6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwODQ3MF0gcGNpIDAwMDE6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgz
ODAwMDAwMDAwMDAtMHgzODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDg0NzNdIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNCBb
bWVtIDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA4NDczXSBwY2lfYnVzIDAwMDE6MDA6IHJlc291cmNlIDUgW21lbSAw
eDM4MDAwMDAwMDAwMC0weDNiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA4NDc0XSBwY2lfYnVzIDAwMDE6MDE6IHJlc291cmNlIDEgW21l
bSAweDYwMDAwMDAwLTB4NjAxZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA4NDc1XSBwY2lfYnVzIDAwMDE6MDE6IHJlc291cmNlIDIgW21lbSAweDM4MDAwMDAw
MDAwMC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwODU4MV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kzXSAoZG9tYWlu
IDAwMDAgW2J1cyAwMC1mZl0pClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA4NTg2XSBhY3BpIFBOUDBBMDg6MDM6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZp
ZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDg2NTBdIGFjcGkgUE5QMEEwODowMzogX09TQzogcGxhdGZv
cm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDg3MDhdIGFjcGkgUE5QMEEwODow
MzogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4NzEwXSBhY3BpIFBOUDBBMDg6MDM6IE1DRkcg
cXVpcms6IEVDQU0gYXQgW21lbSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZl0gZm9yIFti
dXMgMDAtZmZdIHdpdGggMHhmZmZmYWEyZmY3NzdlYTU4ClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA5MzM4XSBhY3BpIFBOUDBBMDg6MDM6IEVDQU0gYXJlYSBbbWVtIDB4
M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA5MzQ0XSBhY3BpIFBOUDBBMDg6MDM6
IEVDQU0gYXQgW21lbSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAt
ZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA5NDA0XSBQQ0kgaG9z
dCBicmlkZ2UgdG8gYnVzIDAwMDA6MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDk0MDVdIHBjaV9idXMgMDAwMDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDcw
MDAwMDAwLTB4N2ZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwOTQwNl0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4
M2MwMDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDk0MDddIHBjaV9idXMgMDAwMDowMDogcm9vdCBidXMgcmVzb3Vy
Y2UgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk0
MjRdIHBjaSAwMDAwOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk0NjddIHBjaSAwMDAwOjAw
OjAxLjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk0ODBdIHBjaSAwMDAwOjAwOjAxLjA6IGVuYWJsaW5n
IEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDk1
MDZdIHBjaSAwMDAwOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA5NTA3XSBwY2kgMDAwMDowMDowMS4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA5NTk1XSBhY3BpcGhwOiBTbG90IFsxLTRdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTE2ODVdIHBjaV9idXMgMDAwMDowMDogb24gTlVNQSBub2Rl
IDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE2ODZdIHBjaSAwMDAw
OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBh
ZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExNjg4
XSBwY2kgMDAwMDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZm
ZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAw
MDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExNjg5XSBwY2kgMDAw
MDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBb
YnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTY5MF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE0OiBh
c3NpZ25lZCBbbWVtIDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTE2OTFdIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxNTogYXNzaWdu
ZWQgW21lbSAweDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTY5Ml0gcGNpIDAwMDA6MDA6MDEuMDog
QkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTE2OTNdIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxMzogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxMTY5NF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTE2OTVdIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTY5
Nl0gcGNpIDAwMDA6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTY5OF0gcGNpIDAwMDA6MDA6MDEuMDogICBicmlk
Z2Ugd2luZG93IFttZW0gMHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMTY5OV0gcGNpIDAwMDA6MDA6MDEuMDogICBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE3MDFdIHBjaV9idXMgMDAwMDowMDog
cmVzb3VyY2UgNCBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExNzAyXSBwY2lfYnVzIDAwMDA6MDA6IHJlc291
cmNlIDUgW21lbSAweDNjMDAwMDAwMDAwMC0weDNmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExNzAzXSBwY2lfYnVzIDAwMDA6MDE6IHJl
c291cmNlIDEgW21lbSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzExNzA0XSBwY2lfYnVzIDAwMDA6MDE6IHJlc291cmNlIDIgW21l
bSAweDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTgwOF0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQ
Q0k0XSAoZG9tYWluIDAwMDIgW2J1cyAwMC1mZl0pClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzExODEyXSBhY3BpIFBOUDBBMDg6MDQ6IF9PU0M6IE9TIHN1cHBvcnRzIFtF
eHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4NzZdIGFjcGkgUE5QMEEwODowNDog
X09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcg
UE1FIExUUl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE5MzRdIGFj
cGkgUE5QMEEwODowNDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHld
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExOTM2XSBhY3BpIFBOUDBB
MDg6MDQ6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZm
ZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmYWEyZmY3NzdlYTU4ClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTY2XSBhY3BpIFBOUDBBMDg6MDQ6IEVDQU0g
YXJlYSBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAw
QzAyOjAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTczXSBhY3Bp
IFBOUDBBMDg6MDQ6IEVDQU0gYXQgW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZl0g
Zm9yIFtidXMgMDAtZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEy
NjMyXSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDI6MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTI2MzRdIHBjaV9idXMgMDAwMjowMDogcm9vdCBidXMgcmVzb3Vy
Y2UgW21lbSAweDAwODAwMDAwLTB4MGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMjYzNV0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNv
dXJjZSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI2MzZdIHBjaV9idXMgMDAwMjowMDogcm9v
dCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTI2NTJdIHBjaSAwMDAyOjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xh
c3MgMHgwNjAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI2OTVd
IHBjaSAwMDAyOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI3MzBdIHBjaSAwMDAyOjAwOjAx
LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzEyNzMxXSBwY2kgMDAwMjowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNzkyXSBwY2kgMDAwMjow
MDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyODI4XSBwY2kgMDAwMjowMDowNS4wOiBzdXBwb3J0
cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjgyOF0gcGNp
IDAwMDI6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjkxMV0gYWNwaXBocDogU2xvdCBbMS01XSBy
ZWdpc3RlcmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1MDI0XSBh
Y3BpcGhwOiBTbG90IFszLTJdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTcxMTZdIHBjaV9idXMgMDAwMjowMDogb24gTlVNQSBub2RlIDAKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcxMThdIHBjaSAwMDAyOjAwOjAxLjA6
IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAx
MDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTE5XSBwY2kgMDAw
MjowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0
IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTIwXSBwY2kgMDAwMjowMDowMS4w
OiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBh
ZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNzEyMV0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8g
IDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcxMjJdIHBjaSAwMDAyOjAwOjA1LjA6IGJyaWRnZSB3
aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTcxMjNdIHBjaSAwMDAyOjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTI2
XSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgwMDgwMDAwMC0weDAw
OWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzEyN10gcGNp
IDAwMDI6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE3MTI4XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgwMGEw
MDAwMC0weDAwYmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
NzEyOV0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMjAw
MDAwLTB4MjAwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE3MTMwXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzEzMV0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTMy
XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzEzMl0gcGNpIDAwMDI6
MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTM0XSBwY2kgMDAwMjowMDowNS4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzEzNV0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE3MTM2XSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNzEzNl0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3
MTM3XSBwY2kgMDAwMjowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTM5XSBwY2kgMDAwMjowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTQwXSBwY2kgMDAwMjowMDowMS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDIwMDAwMDAwMDAwMC0weDIwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzE0Ml0gcGNpIDAwMDI6MDA6MDUu
MDogUENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzE0NF0gcGNpIDAwMDI6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgw
MGEwMDAwMC0weDAwYmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzE0Nl0gcGNpIDAwMDI6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMDAwMDAy
MDAwMDAtMHgyMDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTcxNDhdIHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNCBbbWVtIDB4
MDA4MDAwMDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE3MTQ5XSBwY2lfYnVzIDAwMDI6MDA6IHJlc291cmNlIDUgW21lbSAweDIwMDAw
MDAwMDAwMC0weDIzZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE3MTUwXSBwY2lfYnVzIDAwMDI6MDE6IHJlc291cmNlIDEgW21lbSAweDAw
ODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE3MTUwXSBwY2lfYnVzIDAwMDI6MDE6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDAwMDAwMC0w
eDIwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzE1MV0gcGNpX2J1cyAwMDAyOjAyOiByZXNvdXJjZSAxIFttZW0gMHgwMGEwMDAw
MC0weDAwYmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzE1
Ml0gcGNpX2J1cyAwMDAyOjAyOiByZXNvdXJjZSAyIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAw
MDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTcyNjBdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJNV0gKGRvbWFpbiAwMDAzIFtidXMg
MDAtZmZdKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzI2NF0gYWNw
aSBQTlAwQTA4OjA1OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9j
a1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE3MzI3XSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90
IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3Mzg1XSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IE9T
IG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNzM4N10gYWNwaSBQTlAwQTA4OjA1OiBNQ0ZHIHF1aXJrOiBFQ0FN
IGF0IFttZW0gMHgyN2ZmZjAwMDAwMDAtMHgyN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3
aXRoIDB4ZmZmZmFhMmZmNzc3ZWE1OApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxODAyMF0gYWNwaSBQTlAwQTA4OjA1OiBFQ0FNIGFyZWEgW21lbSAweDI3ZmZmMDAwMDAw
MC0weDI3ZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAyNl0gYWNwaSBQTlAwQTA4OjA1OiBFQ0FNIGF0IFtt
ZW0gMHgyN2ZmZjAwMDAwMDAtMHgyN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODA3OV0gUENJIGhvc3QgYnJpZGdlIHRv
IGJ1cyAwMDAzOjAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MDgw
XSBwY2lfYnVzIDAwMDM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgxMDAwMDAwMC0weDFm
ZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgw
ODFdIHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI0MDAwMDAwMDAw
MC0weDI3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4MDgyXSBwY2lfYnVzIDAwMDM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAt
ZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MDk5XSBwY2kgMDAw
MzowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MTM5XSBwY2kgMDAwMzowMDowMS4wOiBbMWRl
ZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE4MTc1XSBwY2kgMDAwMzowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODE3Nl0gcGNpIDAwMDM6MDA6MDEu
MDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxODIzM10gcGNpIDAwMDM6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlw
ZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxODI1OV0gcGNpIDAwMDM6MDA6MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyNjBdIHBjaSAwMDAzOjAwOjA1LjA6IFBNRSMgc3Vw
cG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTgzMjddIGFjcGlwaHA6IFNsb3QgWzEtNl0gcmVnaXN0ZXJlZApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDQ1MF0gcGNpIDAwMDM6MDI6MDAuMDogWzgwODY6
MTUyMV0gdHlwZSAwMCBjbGFzcyAweDAyMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMDQ2NV0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgxMDAy
MDAwMC0weDEwMDNmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
MDQ4Ml0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg6IFtpbyAgMHgxZmZmODAyMC0weDFmZmY4
MDNmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDQ5MF0gcGNpIDAw
MDM6MDI6MDAuMDogcmVnIDB4MWM6IFttZW0gMHgxMDA0NDAwMC0weDEwMDQ3ZmZmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDYwMV0gcGNpIDAwMDM6MDI6MDAuMDog
UE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjA2MzhdIHBjaSAwMDAzOjAyOjAwLjA6IHJlZyAweDE4NDogW21l
bSAweDAwMDAwMDAwLTB4MDAwMDNmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjA2MzldIHBjaSAwMDAzOjAyOjAwLjA6IFZGKG4pIEJBUjAgc3Bh
Y2U6IFttZW0gMHgwMDAwMDAwMC0weDAwMDFmZmZmIDY0Yml0IHByZWZdIChjb250YWlucyBCQVIw
IGZvciA4IFZGcykKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA2NjBd
IHBjaSAwMDAzOjAyOjAwLjA6IHJlZyAweDE5MDogW21lbSAweDAwMDAwMDAwLTB4MDAwMDNmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA2NjFd
IHBjaSAwMDAzOjAyOjAwLjA6IFZGKG4pIEJBUjMgc3BhY2U6IFttZW0gMHgwMDAwMDAwMC0weDAw
MDFmZmZmIDY0Yml0IHByZWZdIChjb250YWlucyBCQVIzIGZvciA4IFZGcykKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA3NThdIHBjaSAwMDAzOjAyOjAwLjA6IDguMDAw
IEdiL3MgYXZhaWxhYmxlIFBDSWUgYmFuZHdpZHRoLCBsaW1pdGVkIGJ5IDUuMCBHVC9zIFBDSWUg
eDIgbGluayBhdCAwMDAzOjAwOjA1LjAgKGNhcGFibGUgb2YgMTYuMDAwIEdiL3Mgd2l0aCA1LjAg
R1QvcyBQQ0llIHg0IGxpbmspClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzIwODA1XSBwY2kgMDAwMzowMjowMC4xOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNsYXNzIDB4MDIw
MDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwODIwXSBwY2kgMDAw
MzowMjowMC4xOiByZWcgMHgxMDogW21lbSAweDEwMDAwMDAwLTB4MTAwMWZmZmZdClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwODM3XSBwY2kgMDAwMzowMjowMC4xOiBy
ZWcgMHgxODogW2lvICAweDFmZmY4MDAwLTB4MWZmZjgwMWZdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzIwODQ1XSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxYzogW21l
bSAweDEwMDQwMDAwLTB4MTAwNDNmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzIwOTUyXSBwY2kgMDAwMzowMjowMC4xOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQz
aG90IEQzY29sZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDk4NV0g
cGNpIDAwMDM6MDI6MDAuMTogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDk4Nl0g
cGNpIDAwMDM6MDI6MDAuMTogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAw
MWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTAwN10gcGNpIDAwMDM6MDI6MDAuMTogcmVnIDB4
MTkwOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTAwOF0gcGNpIDAwMDM6MDI6MDAuMTogVkYobikg
QkFSMyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRh
aW5zIEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMTEyOF0gcGNpX2J1cyAwMDAzOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTEzMF0gcGNpX2J1cyAwMDAzOjAwOiBtYXggYnVzIGRl
cHRoOiAxIHBjaV90cnlfbnVtOiAyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIxMTMzXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgxMDAw
MDAwMC0weDEwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
MTEzNF0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMDAw
MDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzIxMTM1XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFtt
ZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTEzNl0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4
MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTM3XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IG5v
IHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMTEzOF0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNz
aWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIxMTM5XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE0MF0g
cGNpIDAwMDM6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTQxXSBwY2kgMDAw
MzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzIxMTQzXSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzIxMTQ0XSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE0N10gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDA6IGFz
c2lnbmVkIFttZW0gMHgxMDIwMDAwMC0weDEwMjFmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyMTE1MF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDA6IGFzc2lnbmVk
IFttZW0gMHgxMDIyMDAwMC0weDEwMjNmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMTE1NF0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDM6IGFzc2lnbmVkIFttZW0g
MHgxMDI0MDAwMC0weDEwMjQzZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMTE1N10gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDc6IGFzc2lnbmVkIFttZW0gMHgyNDAw
MDAyMDAwMDAtMHgyNDAwMDAyMWZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjExNjFdIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAxMDogYXNzaWdu
ZWQgW21lbSAweDI0MDAwMDIyMDAwMC0weDI0MDAwMDIzZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE2NV0gcGNpIDAwMDM6MDI6MDAuMTog
QkFSIDM6IGFzc2lnbmVkIFttZW0gMHgxMDI0NDAwMC0weDEwMjQ3ZmZmXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE2OF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDc6
IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyNDAwMDAtMHgyNDAwMDAyNWZmZmYgNjRiaXQgcHJlZl0K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjExNzNdIHBjaSAwMDAzOjAy
OjAwLjE6IEJBUiAxMDogYXNzaWduZWQgW21lbSAweDI0MDAwMDI2MDAwMC0weDI0MDAwMDI3ZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE3
N10gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIw
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE3OF0gcGNpIDAwMDM6
MDI6MDAuMDogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjExNzldIHBjaSAwMDAzOjAyOjAwLjE6
IEJBUiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjExNzldIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzIxMTgwXSBwY2kgMDAwMzowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDJdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTgyXSBwY2kgMDAw
MzowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTgzXSBwY2kgMDAwMzowMDow
NS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE4NV0g
cGNpX2J1cyAwMDAzOjAwOiBOby4gMiB0cnkgdG8gYXNzaWduIHVuYXNzaWduZWQgcmVzClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTg3XSBwY2kgMDAwMzowMDowMS4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE4OF0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzIxMTg5XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMTE4OV0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIx
MTkwXSBwY2kgMDAwMzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTkyXSBwY2kgMDAwMzowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTkzXSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE5Nl0gcGNpIDAwMDM6MDI6MDAu
MDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE5Nl0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDI6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjExOTddIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjExOThdIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDAwMjBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTk5
XSBwY2kgMDAwMzowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMjAxXSBwY2kgMDAwMzowMDowNS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzIxMjAyXSBwY2kgMDAwMzowMDowNS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwNF0gcGNpX2J1cyAwMDAzOjAwOiBB
dXRvbWF0aWNhbGx5IGVuYWJsZWQgcGNpIHJlYWxsb2MsIGlmIHlvdSBoYXZlIHByb2JsZW0sIHRy
eSBib290aW5nIHdpdGggcGNpPXJlYWxsb2M9b2ZmClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzIxMjA1XSBwY2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDQgW21lbSAweDEw
MDAwMDAwLTB4MWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTIwNV0gcGNpX2J1cyAwMDAzOjAwOiByZXNvdXJjZSA1IFttZW0gMHgyNDAwMDAw
MDAwMDAtMHgyN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMTIwNl0gcGNpX2J1cyAwMDAzOjAxOiByZXNvdXJjZSAxIFttZW0gMHgxMDAw
MDAwMC0weDEwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
MTIwN10gcGNpX2J1cyAwMDAzOjAxOiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgy
NDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjEyMDhdIHBjaV9idXMgMDAwMzowMjogcmVzb3VyY2UgMSBbbWVtIDB4MTAyMDAwMDAt
MHgxMDNmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyMDld
IHBjaV9idXMgMDAwMzowMjogcmVzb3VyY2UgMiBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAw
M2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzIxMjc3XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTZdIChkb21haW4gMDAwNCBbYnVzIDAw
LWZmXSkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEyODFdIGFjcGkg
UE5QMEEwODowNjogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQ
TSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTM0N10gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBz
dXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTQwNV0gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBPUyBu
b3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjE0MDddIGFjcGkgUE5QMEEwODowNjogTUNGRyBxdWlyazogRUNBTSBh
dCBbbWVtIDB4MmJmZmYwMDAwMDAwLTB4MmJmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0
aCAweGZmZmZhYTJmZjc3N2VhNTgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjIwMzhdIGFjcGkgUE5QMEEwODowNjogRUNBTSBhcmVhIFttZW0gMHgyYmZmZjAwMDAwMDAt
MHgyYmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjIwNDRdIGFjcGkgUE5QMEEwODowNjogRUNBTSBhdCBbbWVt
IDB4MmJmZmYwMDAwMDAwLTB4MmJmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjIwOTVdIFBDSSBob3N0IGJyaWRnZSB0byBi
dXMgMDAwNDowMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjA5N10g
cGNpX2J1cyAwMDA0OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjAwMDAwMDAtMHgyZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMDk3
XSBwY2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyODAwMDAwMDAwMDAt
MHgyYmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMjA5OF0gcGNpX2J1cyAwMDA0OjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZm
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjExNV0gcGNpIDAwMDQ6
MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjE1M10gcGNpIDAwMDQ6MDA6MDEuMDogWzFkZWY6
ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMjE4MF0gcGNpIDAwMDQ6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjIxODBdIHBjaSAwMDA0OjAwOjAxLjA6
IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjIyMjNdIHBjaSAwMDA0OjAwOjAzLjA6IFsxZGVmOmUxMTNdIHR5cGUg
MDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjIyNTBdIHBjaSAwMDA0OjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMjUwXSBwY2kgMDAwNDowMDowMy4wOiBQTUUjIHN1cHBv
cnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIyMjk2XSBwY2kgMDAwNDowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyMzMyXSBwY2kg
MDAwNDowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMjMzM10gcGNpIDAwMDQ6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjQwNV0g
cGNpIDAwMDQ6MDE6MDAuMDogWzFhMDM6MTE1MF0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjQ1Ml0gcGNpIDAwMDQ6MDE6MDAu
MDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMjUwOF0gcGNpIDAwMDQ6MDE6MDAuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjI1MDldIHBjaSAwMDA0OjAxOjAwLjA6IFBN
RSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDIgRDNob3QgRDNjb2xkClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyNjA0XSBwY2lfYnVzIDAwMDQ6MDI6IGV4dGVuZGVkIGNv
bmZpZyBzcGFjZSBub3QgYWNjZXNzaWJsZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMjYyMl0gcGNpIDAwMDQ6MDI6MDAuMDogWzFhMDM6MjAwMF0gdHlwZSAwMCBjbGFz
cyAweDAzMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjY0MF0g
cGNpIDAwMDQ6MDI6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgyMDAwMDAwMC0weDIxZmZmZmZmXQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjY1MF0gcGNpIDAwMDQ6MDI6
MDAuMDogcmVnIDB4MTQ6IFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZmXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMjY2MF0gcGNpIDAwMDQ6MDI6MDAuMDogcmVnIDB4
MTg6IFtpbyAgMHgyZmZmODAwMC0weDJmZmY4MDdmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyMjcwNV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIHRv
IGVmaWZiClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIyNzQwXSBwY2kg
MDAwNDowMjowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMjc0MV0gcGNpIDAwMDQ6MDI6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjI4NTJdIHBjaSAwMDA0OjAzOjAwLjA6IFsxOTEyOjAwMTRdIHR5cGUgMDAgY2xhc3MgMHgw
YzAzMzAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjI4NzNdIHBjaSAw
MDA0OjAzOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4MjIyMDAwMDAtMHgyMjIwMWZmZiA2NGJpdF0K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjI5ODJdIHBjaSAwMDA0OjAz
OjAwLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTE2XSBwY2lfYnVzIDAwMDQ6MDA6IG9uIE5VTUEg
bm9kZSAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTE4XSBwY2kg
MDAwNDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0
Yml0IHByZWZdIHRvIFtidXMgMDEtMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTE5XSBwY2kgMDAwNDow
MDowMy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRk
X3NpemUgMTAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDEyMV0g
cGNpIDAwMDQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZm
ZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAw
MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDEyMl0gcGNpIDAwMDQ6
MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1
cyAwM10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjQxMjNdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5k
b3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTI0XSBwY2kgMDAwNDowMDowNS4wOiBi
cmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFti
dXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTI1XSBwY2kgMDAwNDowMDowNS4wOiBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAyMDAw
MDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyNDEyOF0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAw
MDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQx
MjldIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDI4MDAwMDAwMDAw
MC0weDI4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNDEzMF0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVt
IDB4MjMwMDAwMDAtMHgyMzFmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQxMzFdIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDI4
MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNDEzMl0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE0OiBhc3Np
Z25lZCBbbWVtIDB4MjMyMDAwMDAtMHgyMzNmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjQxMzNdIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxNTogYXNzaWduZWQg
W21lbSAweDI4MDAwMDQwMDAwMC0weDI4MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDEzNF0gcGNpIDAwMDQ6MDA6MDEuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjQxMzVdIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNDEzNl0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQxMzddIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDEzOF0g
cGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxMzhdIHBjaSAwMDA0OjAw
OjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE0MF0gcGNpIDAwMDQ6MDA6MDEuMDog
QkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjQxNDFdIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxMzogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyNDE0Ml0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjQxNDNdIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE0
NF0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxNDVdIHBjaSAwMDA0
OjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE0Nl0gcGNpIDAwMDQ6MDE6MDAu
MDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxNDddIHBjaSAwMDA0OjAxOjAwLjA6IEJB
UiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI0MTQ4XSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTM6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjQxNDldIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVt
IDB4MjAwMDAwMDAtMHgyMWZmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQxNTNdIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAxOiBhc3NpZ25lZCBbbWVtIDB4MjIw
MDAwMDAtMHgyMjAxZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQxNTZdIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MDA4MF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxNTddIHBjaSAw
MDA0OjAyOjAwLjA6IEJBUiAyOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwODBdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTU4XSBwY2kgMDAwNDowMTow
MC4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI0MTYyXSBwY2kgMDAwNDowMTowMC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI0MTY5XSBwY2kgMDAwNDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDEtMDJdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTcwXSBwY2kgMDAwNDowMDow
MS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTcyXSBwY2kgMDAwNDowMDowMS4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDAwMDAwMC0weDI4MDAwMDFmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE3NF0gcGNpIDAw
MDQ6MDM6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIzMDAxZmZmIDY0
Yml0XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE4M10gcGNpIDAw
MDQ6MDA6MDMuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyNDE4NF0gcGNpIDAwMDQ6MDA6MDMuMDogICBicmlkZ2Ugd2luZG93
IFttZW0gMHgyMzAwMDAwMC0weDIzMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNDE4Nl0gcGNpIDAwMDQ6MDA6MDMuMDogICBicmlkZ2Ugd2luZG93IFttZW0g
MHgyODAwMDAyMDAwMDAtMHgyODAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxODddIHBjaSAwMDA0OjAwOjA1LjA6IFBDSSBicmlk
Z2UgdG8gW2J1cyAwNF0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQx
ODldIHBjaSAwMDA0OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjMyMDAwMDAtMHgy
MzNmZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxOTBdIHBj
aSAwMDA0OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4Mjgw
MDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI0MTkyXSBwY2lfYnVzIDAwMDQ6MDA6IFNvbWUgUENJIGRldmljZSByZXNvdXJjZXMgYXJl
IHVuYXNzaWduZWQsIHRyeSBib290aW5nIHdpdGggcGNpPXJlYWxsb2MKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQxOTNdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2Ug
NCBbbWVtIDB4MjAwMDAwMDAtMHgyZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI0MTk0XSBwY2lfYnVzIDAwMDQ6MDA6IHJlc291cmNlIDUgW21l
bSAweDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MTk1XSBwY2lfYnVzIDAwMDQ6MDE6IHJlc291cmNlIDEg
W21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI0MTk2XSBwY2lfYnVzIDAwMDQ6MDE6IHJlc291cmNlIDIgW21lbSAweDI4MDAw
MDAwMDAwMC0weDI4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyNDE5N10gcGNpX2J1cyAwMDA0OjAyOiByZXNvdXJjZSAxIFttZW0g
MHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNDE5OF0gcGNpX2J1cyAwMDA0OjAzOiByZXNvdXJjZSAxIFttZW0gMHgyMzAwMDAwMC0w
eDIzMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDE5OV0g
cGNpX2J1cyAwMDA0OjAzOiByZXNvdXJjZSAyIFttZW0gMHgyODAwMDAyMDAwMDAtMHgyODAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQyMDBdIHBjaV9idXMgMDAwNDowNDogcmVzb3VyY2UgMSBbbWVtIDB4MjMyMDAwMDAtMHgyMzNm
ZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQyMDBdIHBjaV9i
dXMgMDAwNDowNDogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAwNWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MjI4
XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTddIChkb21haW4gMDAwNSBbYnVzIDAwLWZmXSkK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQyMzNdIGFjcGkgUE5QMEEw
ODowNzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdt
ZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyNDI5N10gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0
IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyNDM1NV0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBub3cgY29u
dHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjQzNTddIGFjcGkgUE5QMEEwODowNzogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVt
IDB4MmZmZmYwMDAwMDAwLTB4MmZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZm
ZmZhYTJmZjc3N2VhNTgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ5
ODhdIGFjcGkgUE5QMEEwODowNzogRUNBTSBhcmVhIFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZm
ZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjQ5OTRdIGFjcGkgUE5QMEEwODowNzogRUNBTSBhdCBbbWVtIDB4MmZm
ZmYwMDAwMDAwLTB4MmZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUwNTFdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAw
NTowMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTA1Ml0gcGNpX2J1
cyAwMDA1OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3
aW5kb3ddClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDUzXSBwY2lf
YnVzIDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyZmZm
ZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
NTA1NF0gcGNpX2J1cyAwMDA1OjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTA3MF0gcGNpIDAwMDU6MDA6MDAu
MDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNTExNV0gcGNpIDAwMDU6MDA6MDEuMDogWzFkZWY6ZTExMV0g
dHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNTE1MV0gcGNpIDAwMDU6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUxNTJdIHBjaSAwMDA1OjAwOjAxLjA6IFBNRSMg
c3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjUyMTNdIHBjaSAwMDA1OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xh
c3MgMHgwNjA0MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUyNDld
IHBjaSAwMDA1OjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI1MjUwXSBwY2kgMDAwNTowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1
MzAyXSBwY2kgMDAwNTowMDowNy4wOiBbMWRlZjplMTE3XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MzM4XSBwY2kgMDAwNTow
MDowNy4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNTMzOF0gcGNpIDAwMDU6MDA6MDcuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBE
M2hvdApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTQxNV0gYWNwaXBo
cDogU2xvdCBbMS03XSByZWdpc3RlcmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5NjIzXSBwY2lfYnVzIDAwMDU6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjI0XSBwY2kgMDAwNTowMDowMS4wOiBicmlk
Z2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTYyNV0gcGNpIDAwMDU6MDA6
MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVm
XSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTYyNl0gcGNpIDAwMDU6MDA6MDEuMDogYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMV0gYWRkX3Np
emUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43Mjk2MjhdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEw
MDAtMHgwZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI5NjI5XSBwY2kgMDAwNTowMDowNS4wOiBicmlkZ2Ugd2luZG93
IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDJdIGFkZF9z
aXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI5NjMwXSBwY2kgMDAwNTowMDowNS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgw
MDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWdu
IDEwMDAwMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTYzMV0gcGNp
IDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMg
MDNdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjk2MzJdIHBjaSAwMDA1OjAwOjA3LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4
MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGln
biAxMDAwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2MzNdIHBj
aSAwMDA1OjAwOjA3LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZd
IHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjM2XSBwY2kgMDAwNTowMDowMS4wOiBCQVIg
MTQ6IGFzc2lnbmVkIFttZW0gMHgzMDAwMDAwMC0weDMwMWZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTYzN10gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDE1OiBh
c3NpZ25lZCBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjM4XSBwY2kgMDAwNTowMDow
NS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY0MF0gcGNpIDAwMDU6MDA6MDUuMDog
QkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MmMwMDAwMjAwMDAwLTB4MmMwMDAwM2ZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjQwXSBwY2kg
MDAwNTowMDowNy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZm
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY0Ml0gcGNpIDAwMDU6
MDA6MDcuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5
NjQzXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY0M10gcGNpIDAw
MDU6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjQ0XSBwY2kgMDAwNTowMDow
NS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY0NV0gcGNpIDAwMDU6MDA6MDUuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI5NjQ2XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyOTY0N10gcGNpIDAwMDU6MDA6MDcuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI5NjQ5XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY1MF0gcGNp
IDAwMDU6MDA6MDcuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjUwXSBwY2kgMDAwNTow
MDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY1MV0gcGNpIDAwMDU6MDA6MDUuMDogQkFS
IDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjUyXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IG5v
IHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyOTY1M10gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNz
aWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI5NjU0XSBwY2kgMDAwNTowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjU2XSBwY2kgMDAwNTowMDowMS4w
OiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjU3XSBwY2kgMDAwNTowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY1OV0gcGNpIDAwMDU6
MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyOTY2MV0gcGNpIDAwMDU6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyOTY2Ml0gcGNpIDAwMDU6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgy
YzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2NjRdIHBjaSAwMDA1OjAwOjA3LjA6IFBDSSBicmlkZ2Ug
dG8gW2J1cyAwM10KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2NjZd
IHBjaSAwMDA1OjAwOjA3LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzA0MDAwMDAtMHgzMDVm
ZmZmZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2NjddIHBjaSAw
MDA1OjAwOjA3LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAw
NWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI5NjY5XSBwY2lfYnVzIDAwMDU6MDA6IHJlc291cmNlIDQgW21lbSAweDMwMDAwMDAwLTB4M2Zm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY3
MF0gcGNpX2J1cyAwMDA1OjAwOiByZXNvdXJjZSA1IFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyZmZm
ZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
OTY3MV0gcGNpX2J1cyAwMDA1OjAxOiByZXNvdXJjZSAxIFttZW0gMHgzMDAwMDAwMC0weDMwMWZm
ZmZmXQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTY3Ml0gcGNpX2J1
cyAwMDA1OjAxOiByZXNvdXJjZSAyIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2NzNd
IHBjaV9idXMgMDAwNTowMjogcmVzb3VyY2UgMSBbbWVtIDB4MzAyMDAwMDAtMHgzMDNmZmZmZl0K
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk2NzRdIHBjaV9idXMgMDAw
NTowMjogcmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwMjAwMDAwLTB4MmMwMDAwM2ZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5Njc0XSBwY2lf
YnVzIDAwMDU6MDM6IHJlc291cmNlIDEgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5Njc1XSBwY2lfYnVzIDAwMDU6MDM6
IHJlc291cmNlIDIgW21lbSAweDJjMDAwMDQwMDAwMC0weDJjMDAwMDVmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjE3OV0gaW9tbXU6IERl
ZmF1bHQgZG9tYWluIHR5cGU6IFRyYW5zbGF0ZWQgClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMyMjEzXSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6IFZHQSBkZXZpY2Ug
YWRkZWQ6IGRlY29kZXM9aW8rbWVtLG93bnM9bm9uZSxsb2Nrcz1ub25lClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjE0XSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6
IGJyaWRnZSBjb250cm9sIHBvc3NpYmxlClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMyMjE2XSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6IHNldHRpbmcgYXMgYm9vdCBk
ZXZpY2UgKFZHQSBsZWdhY3kgcmVzb3VyY2VzIG5vdCBhdmFpbGFibGUpClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjE3XSB2Z2FhcmI6IGxvYWRlZApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjMwNF0gRURBQyBNQzogVmVyOiAzLjAuMApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjQzNl0gUmVnaXN0ZXJlZCBl
Zml2YXJzIG9wZXJhdGlvbnMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzMxMTddIE5ldExhYmVsOiBJbml0aWFsaXppbmcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzMxMThdIE5ldExhYmVsOiAgZG9tYWluIGhhc2ggc2l6ZSA9IDEyOApTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzExOF0gTmV0TGFiZWw6ICBwcm90
b2NvbHMgPSBVTkxBQkVMRUQgQ0lQU092NCBDQUxJUFNPClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMzMTI4XSBOZXRMYWJlbDogIHVubGFiZWxlZCB0cmFmZmljIGFsbG93
ZWQgYnkgZGVmYXVsdApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzI1
N10gY2xvY2tzb3VyY2U6IFN3aXRjaGVkIHRvIGNsb2Nrc291cmNlIGFyY2hfc3lzX2NvdW50ZXIK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NzhdIFZGUzogRGlzayBx
dW90YXMgZHF1b3RfNi42LjAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDE3OTVdIFZGUzogRHF1b3QtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA1MTIgKG9yZGVyIDAs
IDQwOTYgYnl0ZXMpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxOTEw
XSBBcHBBcm1vcjogQXBwQXJtb3IgRmlsZXN5c3RlbSBFbmFibGVkClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxOTQ0XSBwbnA6IFBuUCBBQ1BJIGluaXQKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI1NzhdIHN5c3RlbSAwMDowMDogW21lbSAw
eDNiZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZl
ZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjU4MF0gc3lzdGVtIDAw
OjAwOiBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90
IGJlIHJlc2VydmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNTgx
XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmYgd2luZG93
XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDI1ODJdIHN5c3RlbSAwMDowMDogW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZm
ZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0MjU4M10gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MmJmZmYwMDAwMDAw
LTB4MmJmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNTg0XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgy
ZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI1ODVdIHN5c3RlbSAwMDow
MDogW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBi
ZSByZXNlcnZlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjU4Nl0g
c3lzdGVtIDAwOjAwOiBbbWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmIHdpbmRvd10g
Y291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQyNTkwXSBzeXN0ZW0gMDA6MDA6IFBsdWcgYW5kIFBsYXkgQUNQSSBkZXZpY2UsIElEcyBQ
TlAwYzAyIChhY3RpdmUpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQy
ODg1XSBwbnA6IFBuUCBBQ1BJOiBmb3VuZCAxIGRldmljZXMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDM4MTZdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkg
MgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0Mzg4Ml0gSVAgaWRlbnRz
IGhhc2ggdGFibGUgZW50cmllczogMjYyMTQ0IChvcmRlcjogOSwgMjA5NzE1MiBieXRlcywgbGlu
ZWFyKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTQ3Nl0gdGNwX2xp
c3Rlbl9wb3J0YWRkcl9oYXNoIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA3LCA1
MjQyODggYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDU3NjVdIFRDUCBlc3RhYmxpc2hlZCBoYXNoIHRhYmxlIGVudHJpZXM6IDUyNDI4OCAob3Jk
ZXI6IDEwLCA0MTk0MzA0IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ2ODQ2XSBUQ1AgYmluZCBoYXNoIHRhYmxlIGVudHJpZXM6IDY1NTM2IChv
cmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0NzEwNF0gVENQOiBIYXNoIHRhYmxlcyBjb25maWd1cmVkIChlc3RhYmxp
c2hlZCA1MjQyODggYmluZCA2NTUzNikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDcxOTFdIFVEUCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0
ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0NzU5NF0gVURQLUxpdGUgaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDgsIDEw
NDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDgzNzBdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODM3N10gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29s
IGZhbWlseSA0NApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODM4NV0g
cGNpIDAwMGM6MDE6MDAuMDogQ0xTIG1pc21hdGNoICg2NCAhPSAzMiksIHVzaW5nIDY0IGJ5dGVz
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NDYwXSBUcnlpbmcgdG8g
dW5wYWNrIHJvb3RmcyBpbWFnZSBhcyBpbml0cmFtZnMuLi4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNDkzMDBdIEZyZWVpbmcgaW5pdHJkIG1lbW9yeTogMzcxNjRLClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUwOTE0XSBodyBwZXJmZXZlbnRz
OiBlbmFibGVkIHdpdGggYXJtdjhfcG11djNfMCBQTVUgZHJpdmVyLCA3IGNvdW50ZXJzIGF2YWls
YWJsZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MTAwMl0ga3ZtIFsx
XTogSVBBIFNpemUgTGltaXQ6IDQ4IGJpdHMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNTEwMjVdIGt2bSBbMV06IEdJQ3YzOiBubyBHSUNWIHJlc291cmNlIGVudHJ5ClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUxMDI2XSBrdm0gWzFdOiBkaXNh
YmxpbmcgR0lDdjIgZW11bGF0aW9uClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTUxMDM3XSBrdm0gWzFdOiBHSUMgc3lzdGVtIHJlZ2lzdGVyIENQVSBpbnRlcmZhY2UgZW5h
YmxlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MTM3OV0ga3ZtIFsx
XTogdmdpYyBpbnRlcnJ1cHQgSVJROQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjE1MTc4OV0ga3ZtIFsxXTogVkhFIG1vZGUgaW5pdGlhbGl6ZWQgc3VjY2Vzc2Z1bGx5ClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUyNjI5XSBJbml0aWFsaXNlIHN5
c3RlbSB0cnVzdGVkIGtleXJpbmdzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTUyNjM3XSBLZXkgdHlwZSBibGFja2xpc3QgcmVnaXN0ZXJlZApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MjY3NV0gd29ya2luZ3NldDogdGltZXN0YW1wX2JpdHM9
NDIgbWF4X29yZGVyPTI0IGJ1Y2tldF9vcmRlcj0wClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTUzMzUyXSB6YnVkOiBsb2FkZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNTM0NjldIGludGVncml0eTogUGxhdGZvcm0gS2V5cmluZyBpbml0aWFs
aXplZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MzQ3MV0gS2V5IHR5
cGUgYXN5bW1ldHJpYyByZWdpc3RlcmVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTUzNDcxXSBBc3ltbWV0cmljIGtleSBwYXJzZXIgJ3g1MDknIHJlZ2lzdGVyZWQKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTM0NzhdIEJsb2NrIGxheWVyIFND
U0kgZ2VuZXJpYyAoYnNnKSBkcml2ZXIgdmVyc2lvbiAwLjQgbG9hZGVkIChtYWpvciAyNDkpClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUzNTE3XSBpbyBzY2hlZHVsZXIg
bXEtZGVhZGxpbmUgcmVnaXN0ZXJlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjE1NDUzOV0gc2hwY2hwOiBTdGFuZGFyZCBIb3QgUGx1ZyBQQ0kgQ29udHJvbGxlciBEcml2
ZXIgdmVyc2lvbjogMC40ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTU0
NjUyXSBlZmlmYjogcHJvYmluZyBmb3IgZWZpZmIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNTQ2NzFdIGVmaWZiOiBmcmFtZWJ1ZmZlciBhdCAweDIwMDAwMDAwLCB1c2lu
ZyAxODc2aywgdG90YWwgMTg3NWsKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNTQ2NzJdIGVmaWZiOiBtb2RlIGlzIDgwMHg2MDB4MzIsIGxpbmVsZW5ndGg9MzIwMCwgcGFn
ZXM9MQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1NDY3M10gZWZpZmI6
IHNjcm9sbGluZzogcmVkcmF3ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTU0Njc0XSBlZmlmYjogVHJ1ZWNvbG9yOiBzaXplPTg6ODo4OjgsIHNoaWZ0PTI0OjE2Ojg6MApT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1NDc0NF0gQ29uc29sZTogc3dp
dGNoaW5nIHRvIGNvbG91ciBmcmFtZSBidWZmZXIgZGV2aWNlIDEwMHgzNwpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MDU3Ml0gZmIwOiBFRkkgVkdBIGZyYW1lIGJ1ZmZl
ciBkZXZpY2UKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjA5MzVdIGlu
cHV0OiBQb3dlciBCdXR0b24gYXMgL2RldmljZXMvTE5YU1lTVE06MDAvTE5YU1lCVVM6MDAvUE5Q
MEMwQzowMC9pbnB1dC9pbnB1dDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNjA5NDldIEFDUEk6IFBvd2VyIEJ1dHRvbiBbUFdSQl0KU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNjUzOThdIEFDUEkgR1REVDogZm91bmQgMSBTQlNBIGdlbmVyaWMg
V2F0Y2hkb2cocykuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1OTIz
XSBTZXJpYWw6IDgyNTAvMTY1NTAgZHJpdmVyLCA0IHBvcnRzLCBJUlEgc2hhcmluZyBlbmFibGVk
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2MTgxXSBTZXJpYWw6IEFN
QkEgZHJpdmVyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2MjE0XSBt
c21fc2VyaWFsOiBkcml2ZXIgaW5pdGlhbGl6ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjYyNjJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogb3B0aW9u
IG1hc2sgMHgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2Mjg3XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZl
YXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTY2NDUzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBl
bnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2
NDkwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRy
aWVzIGZvciBldnRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2NTU0
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVz
IGZvciBwcmlxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY2OTI2XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjkzOF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuMS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NzA1M10gYXJtLXNtbXUtdjMg
YXJtLXNtbXUtdjMuMS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NzA5MF0gYXJtLXNtbXUtdjMgYXJt
LXNtbXUtdjMuMS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NzE1NF0gYXJtLXNtbXUtdjMgYXJtLXNt
bXUtdjMuMS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NzQ2NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuMi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjc0NzddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogaWFzIDQ4LWJp
dCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNjc1MjRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzog
YWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjc2MzVdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogYWxs
b2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjc2NzBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogYWxsb2Nh
dGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjgwMDNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogb3B0aW9uIG1h
c2sgMHgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY4MDEyXSBhcm0t
c21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1
cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY4
MDU3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRy
aWVzIGZvciBjbWRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY4MTY4
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVz
IGZvciBldnRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY4MjA0XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZv
ciBwcmlxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY4NTA4XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My40LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2ODUxN10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
NC5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2ODU4NV0gYXJtLXNtbXUtdjMgYXJt
LXNtbXUtdjMuNC5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2ODYyOV0gYXJtLXNtbXUtdjMgYXJtLXNt
bXUtdjMuNC5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2ODc0MV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuNC5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2OTA0OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
NS5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNjkwNTddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogaWFzIDQ4LWJpdCwg
b2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjkxMjRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxs
b2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjkxNjhdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxsb2Nh
dGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjkyODNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxsb2NhdGVk
IDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNjk1OTldIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogb3B0aW9uIG1hc2sg
MHgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY5NjA5XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My42LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVz
IDB4MDAwMDFmZmYpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY5NjQ3
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVz
IGZvciBjbWRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY5NzEyXSBh
cm0tc21tdS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZv
ciBldnRxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY5NzU2XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBw
cmlxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTcwMTM5XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My43LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MDE0OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNy5h
dXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MDE4NF0gYXJtLXNtbXUtdjMgYXJtLXNt
bXUtdjMuNy5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTI6
Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MDI0Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuNy5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MDI4N10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
Ny5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE3NDc4MF0gbW91c2VkZXY6IFBTLzIgbW91c2UgZGV2aWNl
IGNvbW1vbiBmb3IgYWxsIG1pY2UKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNzU3MjJdIHJ0Yy1lZmkgcnRjLWVmaS4wOiByZWdpc3RlcmVkIGFzIHJ0YzAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzYxODRdIHJ0Yy1lZmkgcnRjLWVmaS4wOiBz
ZXR0aW5nIHN5c3RlbSBjbG9jayB0byAyMDIxLTA5LTEzVDA3OjA4OjQyIFVUQyAoMTYzMTUxNjky
MikKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzY3NDZdIGxlZHRyaWct
Y3B1OiByZWdpc3RlcmVkIHRvIGluZGljYXRlIGFjdGl2aXR5IG9uIENQVXMKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzcwMTFdIFNNQ0NDOiBTT0NfSUQ6IElEID0gamVw
MTA2OjBhMTY6MDAwMSBSZXZpc2lvbiA9IDB4MDAwMDAwYTEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNzcyNzRdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkg
MTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODUyMjBdIFNlZ21lbnQg
Um91dGluZyB3aXRoIElQdjYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
ODUyMzRdIG1pcDY6IE1vYmlsZSBJUHY2ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTg1MjM1XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDE3ClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg1MjY5XSBtcGxzX2dzbzogTVBMUyBHU08g
c3VwcG9ydApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4NTMyMV0gcmVn
aXN0ZXJlZCB0YXNrc3RhdHMgdmVyc2lvbiAxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTg1MzIzXSBMb2FkaW5nIGNvbXBpbGVkLWluIFguNTA5IGNlcnRpZmljYXRlcwpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxMTY2OF0gTG9hZGVkIFguNTA5
IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBDQTogNmNjZWNlN2U0YzZjMGQxZjYxNDlmM2RkMjdk
ZmNjNWNiYjQxOWVhMScKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTE2
NzddIExvYWRlZCBYLjUwOSBjZXJ0ICdEZWJpYW4gU2VjdXJlIEJvb3QgU2lnbmVyIDIwMjEgLSBs
aW51eDogNGI2ZWY1YWJjYTY2OTgyNTE3OGUwNTJjODQ2NjdjY2JjMDUzMWY4YycKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTE4MzNdIHpzd2FwOiBsb2FkZWQgdXNpbmcg
cG9vbCBsem8vemJ1ZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxMTkz
N10gS2V5IHR5cGUgLl9mc2NyeXB0IHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yMTE5MzhdIEtleSB0eXBlIC5mc2NyeXB0IHJlZ2lzdGVyZWQKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTE5MzhdIEtleSB0eXBlIGZzY3J5cHQt
cHJvdmlzaW9uaW5nIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yMTE5NzJdIEFwcEFybW9yOiBBcHBBcm1vciBzaGExIHBvbGljeSBoYXNoaW5nIGVuYWJs
ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjA5OTZdIGludGVncml0
eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjIyMTcwMF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAn
U1VTRSBMaW51eCBFbnRlcnByaXNlIFNlY3VyZSBCb290IENBOiAzZDRkNDBjZjkzODUzOTAyNGIx
Y2ZjNWExMmRlZGZlOGIxN2U3NTVmJwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIyMTcwMV0gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRi
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIxODYyXSBpbnRlZ3JpdHk6
IExvYWRlZCBYLjUwOSBjZXJ0ICdDYW5vbmljYWwgTHRkLiBNYXN0ZXIgQ2VydGlmaWNhdGUgQXV0
aG9yaXR5OiBhZDkxOTkwYmMyMmFiMWY1MTcwNDhjMjNiNjY1NWEyNjhlMzQ1YTYzJwpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMTg2M10gaW50ZWdyaXR5OiBMb2FkaW5n
IFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjIyMDE4XSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdGZWRvcmEgU2Vj
dXJlIEJvb3QgQ0E6IGZkZTMyNTk5YzJkNjFkYjFiZjU4MDczMzVkN2IyMGU0Y2Q5NjNiNDInClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIyMDE5XSBpbnRlZ3JpdHk6IExv
YWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yMjIwMzJdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ01pY3Jv
c29mdCBXaW5kb3dzIFByb2R1Y3Rpb24gUENBIDIwMTE6IGE5MjkwMjM5OGUxNmM0OTc3OGNkOTBm
OTllNGY5YWUxN2M1NWFmNTMnClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjIyMDMyXSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjIwNDRdIGludGVncml0eTogTG9h
ZGVkIFguNTA5IGNlcnQgJ01pY3Jvc29mdCBDb3Jwb3JhdGlvbiBVRUZJIENBIDIwMTE6IDEzYWRi
ZjQzMDliZDgyNzA5YzhjZDU0ZjMxNmVkNTIyOTg4YTFiZDQnClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjIyMDQ1XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlm
aWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjIw
NTddIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ01pY3Jvc29mdCBXaW5kb3dzIFByb2R1
Y3Rpb24gUENBIDIwMTE6IGE5MjkwMjM5OGUxNmM0OTc3OGNkOTBmOTllNGY5YWUxN2M1NWFmNTMn
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2NDQyXSBwY2llcG9ydCAw
MDBjOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjI2NzM4XSBwY2llcG9ydCAwMDBjOjAwOjAxLjA6IEFFUjogZW5h
YmxlZCB3aXRoIElSUSAxMDYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
MjY4MDhdIHBjaWVwb3J0IDAwMGQ6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjcwNzRdIHBjaWVwb3J0IDAwMGQ6
MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIyNzExOF0gcGNpZXBvcnQgMDAwZDowMDowMi4wOiBBZGRpbmcgdG8g
aW9tbXUgZ3JvdXAgMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNzM1
Nl0gcGNpZXBvcnQgMDAwZDowMDowMi4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI3NDAxXSBwY2llcG9ydCAwMDBkOjAw
OjAzLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjI3NjM1XSBwY2llcG9ydCAwMDBkOjAwOjAzLjA6IEFFUjogZW5hYmxlZCB3
aXRoIElSUSAxMDcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjc2Nzhd
IHBjaWVwb3J0IDAwMGQ6MDA6MDQuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDQKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjc5MTNdIHBjaWVwb3J0IDAwMGQ6MDA6MDQu
MDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIyNzk3Nl0gcGNpZXBvcnQgMDAwMTowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUg
Z3JvdXAgNQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyODIzOV0gcGNp
ZXBvcnQgMDAwMTowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA4ClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI4MzA4XSBwY2llcG9ydCAwMDAwOjAwOjAxLjA6
IEFkZGluZyB0byBpb21tdSBncm91cCA2ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjI4NTcxXSBwY2llcG9ydCAwMDAwOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElS
USAxMDkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjg2NDBdIHBjaWVw
b3J0IDAwMDI6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDcKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjg5MTJdIHBjaWVwb3J0IDAwMDI6MDA6MDEuMDogQUVS
OiBlbmFibGVkIHdpdGggSVJRIDExMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIyODk2MV0gcGNpZXBvcnQgMDAwMjowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAg
OApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyOTIxNF0gcGNpZXBvcnQg
MDAwMjowMDowNS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEwClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjI5Mjk0XSBwY2llcG9ydCAwMDAzOjAwOjAxLjA6IEFkZGlu
ZyB0byBpb21tdSBncm91cCA5ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjI5NTY4XSBwY2llcG9ydCAwMDAzOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTEK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjk2MjNdIHBjaWVwb3J0IDAw
MDM6MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEwClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjI5ODY1XSBwY2llcG9ydCAwMDAzOjAwOjA1LjA6IEFFUjogZW5h
YmxlZCB3aXRoIElSUSAxMTEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
Mjk5MzBdIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDExClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMwMjAyXSBwY2llcG9ydCAwMDA0
OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yMzAyNDldIHBjaWVwb3J0IDAwMDQ6MDA6MDMuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDEyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMw
NDg5XSBwY2llcG9ydCAwMDA0OjAwOjAzLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTIKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzA1MzZdIHBjaWVwb3J0IDAwMDQ6
MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEzClNlcCAxMyAxMjozODo0NSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjMwNzc5XSBwY2llcG9ydCAwMDA0OjAwOjA1LjA6IEFFUjogZW5hYmxl
ZCB3aXRoIElSUSAxMTIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzA4
NDZdIHBjaWVwb3J0IDAwMDQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE0ClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMxMTEzXSBwY2llcG9ydCAwMDA1OjAw
OjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIzMTM4NF0gcGNpZXBvcnQgMDAwNTowMDowMS4wOiBBRVI6IGVuYWJsZWQg
d2l0aCBJUlEgMTEzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMxNDMz
XSBwY2llcG9ydCAwMDA1OjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNgpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIzMTY4MV0gcGNpZXBvcnQgMDAwNTowMDow
NS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjMxNzMzXSBwY2llcG9ydCAwMDA1OjAwOjA3LjA6IEFkZGluZyB0byBpb21t
dSBncm91cCAxNwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjIzMTk4MV0g
cGNpZXBvcnQgMDAwNTowMDowNy4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEzClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMzMzYwXSBGcmVlaW5nIHVudXNlZCBrZXJu
ZWwgbWVtb3J5OiA1NDQwSwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjM5
ODY1N10gQ2hlY2tlZCBXK1ggbWFwcGluZ3M6IHBhc3NlZCwgbm8gVytYIHBhZ2VzIGZvdW5kClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzk4NjYyXSBSdW4gL2luaXQgYXMg
aW5pdCBwcm9jZXNzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzk4NjYz
XSAgIHdpdGggYXJndW1lbnRzOgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAx
LjM5ODY2NF0gICAgIC9pbml0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
Mzk4NjY1XSAgIHdpdGggZW52aXJvbm1lbnQ6ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMzk4NjY1XSAgICAgSE9NRT0vClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMzk4NjY2XSAgICAgVEVSTT1saW51eApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAxLjM5ODY2N10gICAgIEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAt
OC1hcm02NApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ3MzEzM10gcHBz
X2NvcmU6IExpbnV4UFBTIEFQSSB2ZXIuIDEgcmVnaXN0ZXJlZApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjQ3MzEzNF0gcHBzX2NvcmU6IFNvZnR3YXJlIHZlci4gNS4zLjYg
LSBDb3B5cmlnaHQgMjAwNS0yMDA3IFJvZG9sZm8gR2lvbWV0dGkgPGdpb21ldHRpQGxpbnV4Lml0
PgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ3NDEwNV0gUFRQIGNsb2Nr
IHN1cHBvcnQgcmVnaXN0ZXJlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAx
LjQ3ODU5OV0gU0NTSSBzdWJzeXN0ZW0gaW5pdGlhbGl6ZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS40ODA0MzVdIGlnYjogSW50ZWwoUikgR2lnYWJpdCBFdGhlcm5ldCBO
ZXR3b3JrIERyaXZlcgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4MDQz
Nl0gaWdiOiBDb3B5cmlnaHQgKGMpIDIwMDctMjAxNCBJbnRlbCBDb3Jwb3JhdGlvbi4KU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODA4NDldIGlnYiAwMDAzOjAyOjAwLjA6
IEFkZGluZyB0byBpb21tdSBncm91cCAxOApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAxLjQ4MjA1M10gQUNQSTogYnVzIHR5cGUgVVNCIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODIwNzNdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3
IGludGVyZmFjZSBkcml2ZXIgdXNiZnMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS40ODIwNzldIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGludGVyZmFjZSBkcml2ZXIgaHVi
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDgyMTA2XSB1c2Jjb3JlOiBy
ZWdpc3RlcmVkIG5ldyBkZXZpY2UgZHJpdmVyIHVzYgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtl
cm5lbDogWyAgICAxLjQ4NDU2Nl0gbnZtZSAwMDBkOjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBn
cm91cCAxOQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4NDg0OF0gbnZt
ZSBudm1lMDogcGNpIGZ1bmN0aW9uIDAwMGQ6MDE6MDAuMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAxLjQ4NDkwM10gbnZtZSAwMDBkOjAyOjAwLjA6IEFkZGluZyB0byBpb21t
dSBncm91cCAyMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4NTMwMF0g
bnZtZSBudm1lMTogcGNpIGZ1bmN0aW9uIDAwMGQ6MDI6MDAuMApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjQ4NTM0N10gbnZtZSAwMDBkOjAzOjAwLjA6IEFkZGluZyB0byBp
b21tdSBncm91cCAyMQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4NTU5
MV0gbnZtZSBudm1lMjogcGNpIGZ1bmN0aW9uIDAwMGQ6MDM6MDAuMApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4NTYzN10gbnZtZSAwMDBkOjA0OjAwLjA6IEFkZGluZyB0
byBpb21tdSBncm91cCAyMgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4
NTg3M10gbnZtZSBudm1lMzogcGNpIGZ1bmN0aW9uIDAwMGQ6MDQ6MDAuMApTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MjExOV0gbXB0M3NhcyB2ZXJzaW9uIDM1LjEwMC4w
MC4wMCBsb2FkZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTIyMzdd
IG1wdDNzYXMgMDAwYzowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMjMKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTI2OTFdIG1wdDNzYXNfY20wOiA2MyBCSVQg
UENJIEJVUyBETUEgQUREUkVTU0lORyBTVVBQT1JURUQsIHRvdGFsIG1lbSAoNjUzMDQ3OTYga0Ip
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDkyNzk5XSB4aGNpX2hjZCAw
MDA0OjAzOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyNApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjQ5MzI1M10geGhjaV9oY2QgMDAwNDowMzowMC4wOiB4SENJIEhv
c3QgQ29udHJvbGxlcgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MzI1
N10geGhjaV9oY2QgMDAwNDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3NpZ25l
ZCBidXMgbnVtYmVyIDEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTMy
NzVdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogWmVyb2luZyA2NGJpdCBiYXNlIHJlZ2lzdGVycywg
ZXhwZWN0aW5nIGZhdWx0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk0
MTEyXSByYW5kb206IGZhc3QgaW5pdCBkb25lClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNDk2MzYwXSBudm1lIG52bWUxOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVl
dWVzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk2NTE3XSBudm1lIG52
bWUyOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVzClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNDk2NTI0XSBudm1lIG52bWUzOiA4MC8wLzAgZGVmYXVsdC9yZWFk
L3BvbGwgcXVldWVzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk2NTcz
XSBudm1lIG52bWUwOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVzClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk4MzkzXSBhc3QgMDAwNDowMjowMC4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS41MDAwMjddICBudm1lM24xOiBwMSBwMiBwMyBwNCBwNSBwNgpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjUwMDExMV0gIG52bWUxbjE6IHAxIHAyIHAzIHA0ClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTAwNDc2XSAgbnZtZTBuMToKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTAzNTddIHhoY2lfaGNkIDAwMDQ6MDM6MDAu
MDogaGNjIHBhcmFtcyAweDAxNDA1MWNmIGhjaSB2ZXJzaW9uIDB4MTAwIHF1aXJrcyAweDAwMDAw
MDExMDAwMDA0MTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTA3MDJd
IHVzYiB1c2IxOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MWQ2YiwgaWRQcm9kdWN0
PTAwMDIsIGJjZERldmljZT0gNS4xMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjY1MDcwM10gdXNiIHVzYjE6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0zLCBQcm9k
dWN0PTIsIFNlcmlhbE51bWJlcj0xClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjUwNzA0XSB1c2IgdXNiMTogUHJvZHVjdDogeEhDSSBIb3N0IENvbnRyb2xsZXIKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTA3MDVdIHVzYiB1c2IxOiBNYW51ZmFj
dHVyZXI6IExpbnV4IDUuMTAuMC04LWFybTY0IHhoY2ktaGNkClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNjUwNzA2XSB1c2IgdXNiMTogU2VyaWFsTnVtYmVyOiAwMDA0OjAz
OjAwLjAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTA4MjZdIGh1YiAx
LTA6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjUwODM4XSBodWIgMS0wOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjY1MDk5MF0geGhjaV9oY2QgMDAwNDowMzowMC4wOiB4SENJ
IEhvc3QgQ29udHJvbGxlcgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1
MDk5Ml0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3Np
Z25lZCBidXMgbnVtYmVyIDIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42
NTA5OTVdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogSG9zdCBzdXBwb3J0cyBVU0IgMy4wIFN1cGVy
U3BlZWQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEwMzNdIHVzYiB1
c2IyOiBXZSBkb24ndCBrbm93IHRoZSBhbGdvcml0aG1zIGZvciBMUE0gZm9yIHRoaXMgaG9zdCwg
ZGlzYWJsaW5nIExQTS4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEw
NDVdIHVzYiB1c2IyOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MWQ2YiwgaWRQcm9k
dWN0PTAwMDMsIGJjZERldmljZT0gNS4xMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAxLjY1MTA0Nl0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0zLCBQ
cm9kdWN0PTIsIFNlcmlhbE51bWJlcj0xClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNjUxMDQ3XSB1c2IgdXNiMjogUHJvZHVjdDogeEhDSSBIb3N0IENvbnRyb2xsZXIKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEwNDhdIHVzYiB1c2IyOiBNYW51
ZmFjdHVyZXI6IExpbnV4IDUuMTAuMC04LWFybTY0IHhoY2ktaGNkClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxMDQ5XSB1c2IgdXNiMjogU2VyaWFsTnVtYmVyOiAwMDA0
OjAzOjAwLjAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEwNzJdIGNo
ZWNraW5nIGdlbmVyaWMgKDIwMDAwMDAwIDFkNTAwMCkgdnMgaHcgKDIwMDAwMDAwIDIwMDAwMDAp
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxMDczXSBmYjA6IHN3aXRj
aGluZyB0byBhc3Rkcm1mYiBmcm9tIEVGSSBWR0EKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS42NTExMzZdIENvbnNvbGU6IHN3aXRjaGluZyB0byBjb2xvdXIgZHVtbXkgZGV2
aWNlIDgweDI1ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxMTg3XSBh
c3QgMDAwNDowMjowMC4wOiBbZHJtXSBwbGF0Zm9ybSBoYXMgbm8gSU8gc3BhY2UsIHRyeWluZyBN
TUlPClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxMTkzXSBhc3QgMDAw
NDowMjowMC4wOiBbZHJtXSBQMkEgYnJpZGdlIGRpc2FibGVkLCB1c2luZyBkZWZhdWx0IGNvbmZp
Z3VyYXRpb24KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTExOTRdIGFz
dCAwMDA0OjAyOjAwLjA6IFtkcm1dIEFTVCAyNTAwIGRldGVjdGVkClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxMTk4XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBBbmFs
b2cgVkdBIG9ubHkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEyMDBd
IGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIGRyYW0gTUNMSz04MDAgTWh6IHR5cGU9MSBidXNfd2lk
dGg9MTYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTEyMjhdIGh1YiAy
LTA6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjUxMjQ5XSBbVFRNXSBab25lICBrZXJuZWw6IEF2YWlsYWJsZSBncmFwaGljcyBtZW1vcnk6
IDMyNjUyMzk4IEtpQgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1MTI1
MF0gW1RUTV0gWm9uZSAgIGRtYTMyOiBBdmFpbGFibGUgZ3JhcGhpY3MgbWVtb3J5OiAyMDk3MTUy
IEtpQgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1MTI1MV0gW1RUTV0g
SW5pdGlhbGl6aW5nIHBvb2wgYWxsb2NhdG9yClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNjUxMjU1XSBbVFRNXSBJbml0aWFsaXppbmcgRE1BIHBvb2wgYWxsb2NhdG9yClNl
cCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjUxNDIwXSBodWIgMi0wOjEuMDog
NCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1
MTQ5M10gW2RybV0gSW5pdGlhbGl6ZWQgYXN0IDAuMS4wIDIwMTIwMjI4IGZvciAwMDA0OjAyOjAw
LjAgb24gbWlub3IgMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjc2NTI2
NV0gQ29uc29sZTogc3dpdGNoaW5nIHRvIGNvbG91ciBmcmFtZSBidWZmZXIgZGV2aWNlIDI0MHg2
NwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjc2NzkyNV0gYXN0IDAwMDQ6
MDI6MDAuMDogW2RybV0gZmIwOiBhc3Rkcm1mYiBmcmFtZSBidWZmZXIgZGV2aWNlClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzk2MTE1XSBpZ2IgMDAwMzowMjowMC4wOiBh
ZGRlZCBQSEMgb24gZXRoMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjc5
NjExNl0gaWdiIDAwMDM6MDI6MDAuMDogSW50ZWwoUikgR2lnYWJpdCBFdGhlcm5ldCBOZXR3b3Jr
IENvbm5lY3Rpb24KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTYxMTdd
IGlnYiAwMDAzOjAyOjAwLjA6IGV0aDA6IChQQ0llOjUuMEdiL3M6V2lkdGggeDIpIGQ4OjVlOmQz
OjAyOmIxOjkzClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzk2MTk5XSBp
Z2IgMDAwMzowMjowMC4wOiBldGgwOiBQQkEgTm86IDEwNjMwMC0wMDAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTYyMDBdIGlnYiAwMDAzOjAyOjAwLjA6IFVzaW5nIE1T
SS1YIGludGVycnVwdHMuIDggcnggcXVldWUocyksIDggdHggcXVldWUocykKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTYyNzddIGlnYiAwMDAzOjAyOjAwLjE6IEFkZGlu
ZyB0byBpb21tdSBncm91cCAyNQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAx
Ljg1MDgxMl0gaWdiIDAwMDM6MDI6MDAuMTogYWRkZWQgUEhDIG9uIGV0aDEKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTA4MTRdIGlnYiAwMDAzOjAyOjAwLjE6IEludGVs
KFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0d29yayBDb25uZWN0aW9uClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODUwODE1XSBpZ2IgMDAwMzowMjowMC4xOiBldGgxOiAoUENJ
ZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1ZTpkMzowMjpiMTo5NApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjg1MDg5N10gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogUEJBIE5v
OiAxMDYzMDAtMDAwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODUwODk4
XSBpZ2IgMDAwMzowMjowMC4xOiBVc2luZyBNU0ktWCBpbnRlcnJ1cHRzLiA4IHJ4IHF1ZXVlKHMp
LCA4IHR4IHF1ZXVlKHMpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODUx
ODY4XSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IHJlbmFtZWQgZnJvbSBldGgxClNlcCAxMyAxMjoz
ODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU0NzQ5XSBtcHQzc2FzX2NtMDogQ3VycmVudEhv
c3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8gNGsKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTQ3NjddIG1wdDNzYXNfY20wOiBN
U0ktWCB2ZWN0b3JzIHN1cHBvcnRlZDogOTYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTQ3NjhdIAkgbm8gb2YgY29yZXM6IDgwLCBtYXhfbXNpeF92ZWN0b3JzOiAtMQpT
ZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1NDc2OV0gbXB0M3Nhc19jbTA6
ICAwIDgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU2MDU5XSBtcHQz
c2FzX2NtMDogSGlnaCBJT1BzIHF1ZXVlcyA6IGRpc2FibGVkClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU2MDYwXSBtcHQzc2FzMC1tc2l4MDogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0NjUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjFd
IG1wdDNzYXMwLW1zaXgxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2NgpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1NjA2MV0gbXB0M3NhczAtbXNpeDI6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDY3ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODU2MDYyXSBtcHQzc2FzMC1tc2l4MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjgKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjNdIG1wdDNzYXMwLW1zaXg0OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2OQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDog
WyAgICAxLjg1NjA2M10gbXB0M3NhczAtbXNpeDU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDcw
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU2MDY0XSBtcHQzc2FzMC1t
c2l4NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NTYwNjRdIG1wdDNzYXMwLW1zaXg3OiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDQ3MgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1NjA2NV0gbXB0
M3NhczAtbXNpeDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDczClNlcCAxMyAxMjozODo0NSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU2MDY1XSBtcHQzc2FzMC1tc2l4OTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0NzQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYw
NjZdIG1wdDNzYXMwLW1zaXgxMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzUKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjddIG1wdDNzYXMwLW1zaXgxMTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0NzYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NTYwNjddIG1wdDNzYXMwLW1zaXgxMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzcK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjhdIG1wdDNzYXMwLW1z
aXgxMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NTYwNjhdIG1wdDNzYXMwLW1zaXgxNDogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0NzkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjldIG1w
dDNzYXMwLW1zaXgxNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODAKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNjldIG1wdDNzYXMwLW1zaXgxNjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0ODEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NTYwNzBdIG1wdDNzYXMwLW1zaXgxNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODIKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzFdIG1wdDNzYXMwLW1zaXgxODog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTYwNzFdIG1wdDNzYXMwLW1zaXgxOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
ODQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzJdIG1wdDNzYXMw
LW1zaXgyMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NTYwNzJdIG1wdDNzYXMwLW1zaXgyMTogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0ODYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzNd
IG1wdDNzYXMwLW1zaXgyMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODcKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzRdIG1wdDNzYXMwLW1zaXgyMzogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA0ODgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NTYwNzRdIG1wdDNzYXMwLW1zaXgyNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODkKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzVdIG1wdDNzYXMwLW1zaXgy
NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NTYwNzVdIG1wdDNzYXMwLW1zaXgyNjogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0OTEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzZdIG1wdDNz
YXMwLW1zaXgyNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTIKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzZdIG1wdDNzYXMwLW1zaXgyODogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0OTMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYw
NzddIG1wdDNzYXMwLW1zaXgyOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTQKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzhdIG1wdDNzYXMwLW1zaXgzMDogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0OTUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NTYwNzldIG1wdDNzYXMwLW1zaXgzMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTYK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwNzldIG1wdDNzYXMwLW1z
aXgzMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NTYwODBdIG1wdDNzYXMwLW1zaXgzMzogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0OTgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODBdIG1w
dDNzYXMwLW1zaXgzNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTkKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODFdIG1wdDNzYXMwLW1zaXgzNTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NTYwODFdIG1wdDNzYXMwLW1zaXgzNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDEKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODJdIG1wdDNzYXMwLW1zaXgzNzog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTYwODNdIG1wdDNzYXMwLW1zaXgzODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MDMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODNdIG1wdDNzYXMw
LW1zaXgzOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NTYwODRdIG1wdDNzYXMwLW1zaXg0MDogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MDUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODRd
IG1wdDNzYXMwLW1zaXg0MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDYKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODVdIG1wdDNzYXMwLW1zaXg0MjogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MDcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NTYwODZdIG1wdDNzYXMwLW1zaXg0MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDgKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODZdIG1wdDNzYXMwLW1zaXg0
NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NTYwODddIG1wdDNzYXMwLW1zaXg0NTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODddIG1wdDNz
YXMwLW1zaXg0NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTEKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODhdIG1wdDNzYXMwLW1zaXg0NzogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MTIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYw
ODhdIG1wdDNzYXMwLW1zaXg0ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTMKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwODldIG1wdDNzYXMwLW1zaXg0OTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MTQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NTYwOTBdIG1wdDNzYXMwLW1zaXg1MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTUK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTBdIG1wdDNzYXMwLW1z
aXg1MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NTYwOTFdIG1wdDNzYXMwLW1zaXg1MjogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MTcKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTFdIG1w
dDNzYXMwLW1zaXg1MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTgKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTJdIG1wdDNzYXMwLW1zaXg1NDogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MTkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NTYwOTNdIG1wdDNzYXMwLW1zaXg1NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjAKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTRdIG1wdDNzYXMwLW1zaXg1Njog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTYwOTRdIG1wdDNzYXMwLW1zaXg1NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MjIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTVdIG1wdDNzYXMw
LW1zaXg1ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NTYwOTZdIG1wdDNzYXMwLW1zaXg1OTogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MjQKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTZd
IG1wdDNzYXMwLW1zaXg2MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjUKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTddIG1wdDNzYXMwLW1zaXg2MTogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MjYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NTYwOThdIG1wdDNzYXMwLW1zaXg2MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjcKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOThdIG1wdDNzYXMwLW1zaXg2
MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NTYwOTldIG1wdDNzYXMwLW1zaXg2NDogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MjkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwOTldIG1wdDNz
YXMwLW1zaXg2NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzAKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDBdIG1wdDNzYXMwLW1zaXg2NjogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MzEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYx
MDFdIG1wdDNzYXMwLW1zaXg2NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzIKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDFdIG1wdDNzYXMwLW1zaXg2ODogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MzMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NTYxMDJdIG1wdDNzYXMwLW1zaXg2OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzQK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDNdIG1wdDNzYXMwLW1z
aXg3MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzUKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NTYxMDNdIG1wdDNzYXMwLW1zaXg3MTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MzYKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDRdIG1w
dDNzYXMwLW1zaXg3MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzcKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDRdIG1wdDNzYXMwLW1zaXg3MzogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MzgKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NTYxMDVdIG1wdDNzYXMwLW1zaXg3NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzkKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDZdIG1wdDNzYXMwLW1zaXg3NTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTYxMDZdIG1wdDNzYXMwLW1zaXg3NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
NDEKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDddIG1wdDNzYXMw
LW1zaXg3NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NTYxMDhdIG1wdDNzYXMwLW1zaXg3ODogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1NDMKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMDhd
IG1wdDNzYXMwLW1zaXg3OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDQKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMTBdIG1wdDNzYXNfY20wOiBpb21lbSgweDAw
MDAwMDAwNDAxNDAwMDApLCBtYXBwZWQoMHgoX19fX3B0cnZhbF9fX18pKSwgc2l6ZSg2NTUzNikK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYxMTBdIG1wdDNzYXNfY20w
OiBpb3BvcnQoMHgwMDAwMDAwMDAwMDAwMDAwKSwgc2l6ZSgwKQpTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjg2NTQ1MV0gaWdiIDAwMDM6MDI6MDAuMCBlbm8xOiByZW5hbWVk
IGZyb20gZXRoMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxMzI0M10g
bXB0M3Nhc19jbTA6IEN1cnJlbnRIb3N0UGFnZVNpemUgaXMgMDogU2V0dGluZyBkZWZhdWx0IGhv
c3QgcGFnZSBzaXplIHRvIDRrClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
OTEzMjQ1XSBtcHQzc2FzX2NtMDogc2VuZGluZyBkaWFnIHJlc2V0ICEhClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTY1MjU5XSB1c2IgMS0xOiBuZXcgbG93LXNwZWVkIFVT
QiBkZXZpY2UgbnVtYmVyIDIgdXNpbmcgeGhjaV9oY2QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMi4xNTgwOTVdIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZl
bmRvcj00MTNjLCBpZFByb2R1Y3Q9MzAxYSwgYmNkRGV2aWNlPSAxLjAwClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTU4MDk2XSB1c2IgMS0xOiBOZXcgVVNCIGRldmljZSBz
dHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTI6Mzg6NDUg
ZGViaWFuIGtlcm5lbDogWyAgICAyLjE1ODA5N10gdXNiIDEtMTogUHJvZHVjdDogRGVsbCBNUzEx
NiBVU0IgT3B0aWNhbCBNb3VzZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAy
LjE1ODA5OF0gdXNiIDEtMTogTWFudWZhY3R1cmVyOiBQaXhBcnQKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMi4xNjcyMzZdIGhpZDogcmF3IEhJRCBldmVudHMgZHJpdmVyIChD
KSBKaXJpIEtvc2luYQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAyLjE3OTIw
NV0gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciB1c2JoaWQKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xNzkyMDddIHVzYmhpZDogVVNCIEhJRCBj
b3JlIGRyaXZlcgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAyLjE4MDUyM10g
aW5wdXQ6IFBpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlIGFzIC9kZXZpY2VzL3Bj
aTAwMDQ6MDAvMDAwNDowMDowMy4wLzAwMDQ6MDM6MDAuMC91c2IxLzEtMS8xLTE6MS4wLzAwMDM6
NDEzQzozMDFBLjAwMDEvaW5wdXQvaW5wdXQxClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDIuMTgwNTcyXSBoaWQtZ2VuZXJpYyAwMDAzOjQxM0M6MzAxQS4wMDAxOiBpbnB1dCxo
aWRyYXcwOiBVU0IgSElEIHYxLjExIE1vdXNlIFtQaXhBcnQgRGVsbCBNUzExNiBVU0IgT3B0aWNh
bCBNb3VzZV0gb24gdXNiLTAwMDQ6MDM6MDAuMC0xL2lucHV0MApTZXAgMTMgMTI6Mzg6NDUgZGVi
aWFuIGtlcm5lbDogWyAgICAyLjI2MTkxNl0gdXNiIDItMzogbmV3IFN1cGVyU3BlZWQgR2VuIDEg
VVNCIGRldmljZSBudW1iZXIgMiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFu
IGtlcm5lbDogWyAgICAyLjI5MDQwMl0gdXNiIDItMzogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlk
VmVuZG9yPTA1ZTMsIGlkUHJvZHVjdD0wNjIwLCBiY2REZXZpY2U9OTMuMTAKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yOTA0MDNdIHVzYiAyLTM6IE5ldyBVU0IgZGV2aWNl
IHN0cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjkwNDA1XSB1c2IgMi0zOiBQcm9kdWN0OiBVU0IzLjEg
SHViClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjkwNDA2XSB1c2IgMi0z
OiBNYW51ZmFjdHVyZXI6IEdlbmVzeXNMb2dpYwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAyLjI5MzU0Ml0gaHViIDItMzoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEyOjM4
OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yOTQxNDddIGh1YiAyLTM6MS4wOiA0IHBvcnRzIGRl
dGVjdGVkClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuNDE3MjQ1XSB1c2Ig
MS0yOiBuZXcgbG93LXNwZWVkIFVTQiBkZXZpY2UgbnVtYmVyIDMgdXNpbmcgeGhjaV9oY2QKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi42MjUwODhdIHVzYiAxLTI6IE5ldyBV
U0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNGQ5LCBpZFByb2R1Y3Q9MTIwMywgYmNkRGV2aWNl
PSAyLjgwClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuNjI1MDg5XSB1c2Ig
MS0yOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MCwgUHJvZHVjdD0wLCBTZXJpYWxOdW1i
ZXI9MApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAyLjY2MzIxMF0gaW5wdXQ6
IEhJRCAwNGQ5OjEyMDMgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDow
MzowMC4wL3VzYjEvMS0yLzEtMjoxLjAvMDAwMzowNEQ5OjEyMDMuMDAwMi9pbnB1dC9pbnB1dDIK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi43MjUzMDBdIGhpZC1nZW5lcmlj
IDAwMDM6MDREOToxMjAzLjAwMDI6IGlucHV0LGhpZHJhdzE6IFVTQiBISUQgdjEuMTEgS2V5Ym9h
cmQgW0hJRCAwNGQ5OjEyMDNdIG9uIHVzYi0wMDA0OjAzOjAwLjAtMi9pbnB1dDAKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi43NjgxNzNdIGlucHV0OiBISUQgMDRkOToxMjAz
IFN5c3RlbSBDb250cm9sIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDowMDowMy4wLzAwMDQ6
MDM6MDAuMC91c2IxLzEtMi8xLTI6MS4xLzAwMDM6MDREOToxMjAzLjAwMDMvaW5wdXQvaW5wdXQz
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuODI1Mjc1XSBpbnB1dDogSElE
IDA0ZDk6MTIwMyBDb25zdW1lciBDb250cm9sIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDow
MDowMy4wLzAwMDQ6MDM6MDAuMC91c2IxLzEtMi8xLTI6MS4xLzAwMDM6MDREOToxMjAzLjAwMDMv
aW5wdXQvaW5wdXQ0ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuODI1MzA5
XSBoaWQtZ2VuZXJpYyAwMDAzOjA0RDk6MTIwMy4wMDAzOiBpbnB1dCxoaWRyYXcyOiBVU0IgSElE
IHYxLjExIERldmljZSBbSElEIDA0ZDk6MTIwM10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0
MQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAyLjk1Mjc1NF0gbXB0M3Nhc19j
bTA6IGRpYWcgcmVzZXQ6IFNVQ0NFU1MKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMi45NjEyNDFdIHVzYiAxLTM6IG5ldyBoaWdoLXNwZWVkIFVTQiBkZXZpY2UgbnVtYmVyIDQg
dXNpbmcgeGhjaV9oY2QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45ODc3
NjddIG1wdDNzYXNfY20wOiBzY2F0dGVyIGdhdGhlcjogc2dlX2luX21haW5fbXNnKDEpLCBzZ2Vf
cGVyX2NoYWluKDcpLCBzZ2VfcGVyX2lvKDEyOCksIGNoYWluc19wZXJfaW8oMTkpClNlcCAxMyAx
MjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTg4ODE2XSBtcHQzc2FzX2NtMDogcmVxdWVz
dCBwb29sKDB4KF9fX19wdHJ2YWxfX19fKSkgLSBkbWEoMHhmZjUwMDAwMCk6IGRlcHRoKDQyMjQp
LCBmcmFtZV9zaXplKDEyOCksIHBvb2xfc2l6ZSg1Mjgga0IpClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDMuMDA5MTUzXSBtcHQzc2FzX2NtMDogc2Vuc2UgcG9vbCgweChfX19f
cHRydmFsX19fXykpLSBkbWEoMHhmZWIwMDAwMCk6IGRlcHRoKDM5NjMpLGVsZW1lbnRfc2l6ZSg5
NiksIHBvb2xfc2l6ZSgzNzEga0IpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDMuMDA5MjU3XSBtcHQzc2FzX2NtMDogY29uZmlnIHBhZ2UoMHgoX19fX3B0cnZhbF9fX18pKSAt
IGRtYSgweGZlOWY2MDAwKTogc2l6ZSg1MTIpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDMuMDA5MjU4XSBtcHQzc2FzX2NtMDogQWxsb2NhdGVkIHBoeXNpY2FsIG1lbW9yeTog
c2l6ZSg3Mjc4IGtCKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjAwOTI1
OF0gbXB0M3Nhc19jbTA6IEN1cnJlbnQgQ29udHJvbGxlciBRdWV1ZSBEZXB0aCgzOTYwKSxNYXgg
Q29udHJvbGxlciBRdWV1ZSBEZXB0aCg0MDk2KQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAzLjAwOTI1OV0gbXB0M3Nhc19jbTA6IFNjYXR0ZXIgR2F0aGVyIEVsZW1lbnRzIHBl
ciBJTygxMjgpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTE4NzU2XSB1
c2IgMS0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywgaWRQcm9kdWN0PTA2
MTAsIGJjZERldmljZT05My4xMApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAz
LjExODc1OF0gdXNiIDEtMzogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9
MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4x
MTg3NTldIHVzYiAxLTM6IFByb2R1Y3Q6IFVTQjIuMCBIdWIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlh
biBrZXJuZWw6IFsgICAgMy4xMTg3NTldIHVzYiAxLTM6IE1hbnVmYWN0dXJlcjogR2VuZXN5c0xv
Z2ljClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTIyNzc3XSBodWIgMS0z
OjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAz
LjEyMzM4M10gaHViIDEtMzoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy4yMDk4OTJdIG1wdDNzYXNfY20wOiBfYmFzZV9kaXNwbGF5X2Z3
cGtnX3ZlcnNpb246IGNvbXBsZXRlClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAg
IDMuMjEwMjIyXSBtcHQzc2FzX2NtMDogTFNJU0FTMzAwODogRldWZXJzaW9uKDE2LjAwLjEwLjAw
KSwgQ2hpcFJldmlzaW9uKDB4MDIpLCBCaW9zVmVyc2lvbigxNS4wMC4wMS4wMCkKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMTAyMjNdIG1wdDNzYXNfY20wOiBQcm90b2Nv
bD0oSW5pdGlhdG9yKSwgQ2FwYWJpbGl0aWVzPShSYWlkLFRMUixFRURQLFNuYXBzaG90IEJ1ZmZl
cixEaWFnIFRyYWNlIEJ1ZmZlcixUYXNrIFNldCBGdWxsLE5DUSkKU2VwIDEzIDEyOjM4OjQ1IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy4yMTAyNzVdIHNjc2kgaG9zdDA6IEZ1c2lvbiBNUFQgU0FTIEhv
c3QKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMTA4NDhdIG1wdDNzYXNf
Y20wOiBzZW5kaW5nIHBvcnQgZW5hYmxlICEhClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVs
OiBbICAgIDMuMjM3NTYzXSBQTTogSW1hZ2Ugbm90IGZvdW5kIChjb2RlIC0yMikKU2VwIDEzIDEy
OjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yODU1MzNdIEVYVDQtZnMgKG52bWUzbjFwMik6
IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkK
U2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4zMTU2OTFdIE5vdCBhY3RpdmF0
aW5nIE1hbmRhdG9yeSBBY2Nlc3MgQ29udHJvbCBhcyAvc2Jpbi90b21veW8taW5pdCBkb2VzIG5v
dCBleGlzdC4KU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MTQyNDddIHJh
bmRvbTogc3lzdGVtZDogdW5pbml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJlYWQp
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTE0OTk3XSByYW5kb206IHN5
c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjUxNTE5Nl0gcmFuZG9tOiBzeXN0ZW1kOiB1
bmluaXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MzEwNzBdIGZ1c2U6IGluaXQgKEFQSSB2ZXJzaW9uIDcu
MzIpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTMyNTA5XSBFWFQ0LWZz
IChudm1lM24xcDIpOiByZS1tb3VudGVkLiBPcHRzOiBlcnJvcnM9cmVtb3VudC1ybwpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjYyNzE5OV0gYWNwaS10YWQgQUNQSTAwMEU6
MDA6IFVuc3VwcG9ydGVkIGNhcGFiaWxpdGllcwpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5l
bDogWyAgICAzLjY2MzgxOV0gSVBNSSBtZXNzYWdlIGhhbmRsZXI6IHZlcnNpb24gMzkuMgpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjY2NTQ5OV0geGdlbmUtc2xpbXByby1o
d21vbiBBUE1DMEQyOTowMDogQVBNIFgtR2VuZSBTb0MgSFcgbW9uaXRvciBkcml2ZXIgcmVnaXN0
ZXJlZApTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjY2ODAzM10gaXBtaSBk
ZXZpY2UgaW50ZXJmYWNlClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjc0
MzU1XSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVyClNlcCAxMyAxMjozODo0
NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjc0ODc4XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBh
bnkgU3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjc3MTQxMF0gaXBtaV9zc2lmOiBJUE1JIFNTSUYgSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMg
MTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjc3NjA1M10gaXBtaV9zc2lmIGkyYy1BTVBD
MDAwNDowMDogaXBtaV9zc2lmOiBUcnlpbmcgQUNQSS1zcGVjaWZpZWQgU1NJRiBpbnRlcmZhY2Ug
YXQgaTJjIGFkZHJlc3MgMHgxMCwgYWRhcHRlciBTeW5vcHN5cyBEZXNpZ25XYXJlIEkyQyBhZGFw
dGVyLCBzbGF2ZSBhZGRyZXNzIDB4MjAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy43Nzk1MTNdIEVYVDQtZnMgKG52bWUzbjFwNSk6IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRo
IG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy43Nzk2MTNdIEVYVDQtZnMgKG52bWUzbjFwMyk6IG1vdW50ZWQgZmlsZXN5
c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDEyOjM4OjQ1
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy43OTcyNjZdIEFkZGluZyAxMDAwNDQ0ayBzd2FwIG9uIC9k
ZXYvbnZtZTNuMXA0LiAgUHJpb3JpdHk6LTIgZXh0ZW50czoxIGFjcm9zczoxMDAwNDQ0ayBTU0ZT
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODE4MjY0XSBzYnNhLWd3ZHQg
c2JzYS1nd2R0LjA6IEluaXRpYWxpemVkIHdpdGggMTBzIHRpbWVvdXQgQCAyNTAwMDAwMCBIeiwg
YWN0aW9uPTAuClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODIzNTQ0XSBF
WFQ0LWZzIChudm1lM24xcDYpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEg
bW9kZS4gT3B0czogKG51bGwpClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMu
ODI3OTUzXSBwc3RvcmU6IFVzaW5nIGNyYXNoIGR1bXAgY29tcHJlc3Npb246IGRlZmxhdGUKU2Vw
IDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44Mjc5NjBdIHBzdG9yZTogUmVnaXN0
ZXJlZCBlZmkgYXMgcGVyc2lzdGVudCBzdG9yZSBiYWNrZW5kClNlcCAxMyAxMjozODo0NSBkZWJp
YW4ga2VybmVsOiBbICAgIDMuODU0MTg0XSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRlcmZhY2Ug
ZHJpdmVyClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODU0NTkxXSBpcG1p
X3NpOiBVbmFibGUgdG8gZmluZCBhbnkgU3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMgMTI6Mzg6
NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjg2NDAyOV0gY3J5cHRkOiBtYXhfY3B1X3FsZW4gc2V0
IHRvIDEwMDAKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44Nzg0MjRdIGF1
ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNjkyNS4yMDA6Mik6IGFwcGFybW9yPSJTVEFUVVMi
IG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJy
ZW9mZmljZS14cGRmaW1wb3J0IiBwaWQ9MTEzNyBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAx
MyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODc4NDQ3XSBhdWRpdDogdHlwZT0xNDAw
IGF1ZGl0KDE2MzE1MTY5MjUuMjAwOjMpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InBy
b2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibHNiX3JlbGVhc2UiIHBpZD0x
MTM1IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy44Nzk1NzBdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNjkyNS4yMDA6NCk6
IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNv
bmZpbmVkIiBuYW1lPSJsaWJyZW9mZmljZS1vb3BzbGFzaCIgcGlkPTExNDAgY29tbT0iYXBwYXJt
b3JfcGFyc2VyIgpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICAzLjg4MDExOV0g
YXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE2OTI1LjIwMDo1KTogYXBwYXJtb3I9IlNUQVRV
UyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Imxp
YnJlb2ZmaWNlLXNlbmRkb2MiIHBpZD0xMTQzIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEz
IDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44ODA3MDldIGF1ZGl0OiB0eXBlPTE0MDAg
YXVkaXQoMTYzMTUxNjkyNS4yMDA6Nik6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJv
ZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSIvdXNyL2Jpbi9tYW4iIHBpZD0x
MTQxIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjM4OjQ1IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy44ODA3MTJdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNjkyNS4yMDA6Nyk6
IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNv
bmZpbmVkIiBuYW1lPSJtYW5fZmlsdGVyIiBwaWQ9MTE0MSBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODgwNzE0XSBhdWRpdDogdHlw
ZT0xNDAwIGF1ZGl0KDE2MzE1MTY5MjUuMjAwOjgpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRp
b249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibWFuX2dyb2ZmIiBw
aWQ9MTE0MSBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDMuODgxMzIzXSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MTY5MjUuMjA0
OjkpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0i
dW5jb25maW5lZCIgbmFtZT0ibnZpZGlhX21vZHByb2JlIiBwaWQ9MTE0NiBjb21tPSJhcHBhcm1v
cl9wYXJzZXIiClNlcCAxMyAxMjozODo0NSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODgxMzI2XSBh
dWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MTY5MjUuMjA0OjEwKTogYXBwYXJtb3I9IlNUQVRV
UyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im52
aWRpYV9tb2Rwcm9iZS8va21vZCIgcGlkPTExNDYgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICA0LjA5Nzg1Nl0gcmFuZG9tOiBjcm5nIGlu
aXQgZG9uZQpTZXAgMTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICA0LjA5Nzg1OF0gcmFu
ZG9tOiA3IHVyYW5kb20gd2FybmluZyhzKSBtaXNzZWQgZHVlIHRvIHJhdGVsaW1pdGluZwpTZXAg
MTMgMTI6Mzg6NDUgZGViaWFuIGtlcm5lbDogWyAgICA0LjIzMTU4OF0gaXBtaV9zc2lmIGkyYy1B
TVBDMDAwNDowMDogSVBNSSBtZXNzYWdlIGhhbmRsZXI6IEZvdW5kIG5ldyBCTUMgKG1hbl9pZDog
MHgwMDNjMGEsIHByb2RfaWQ6IDB4MDIxMiwgZGV2X2lkOiAweDIwKQpTZXAgMTMgMTI6Mzg6NDcg
ZGViaWFuIGtlcm5lbDogWyAgICA1Ljc2NDYxNl0gbXB0M3Nhc19jbTA6IGhvc3RfYWRkOiBoYW5k
bGUoMHgwMDAxKSwgc2FzX2FkZHIoMHg1MDAxNDg1MDAwNzc5ZTMwKSwgcGh5cyg4KQpTZXAgMTMg
MTI6Mzg6NDkgZGViaWFuIGtlcm5lbDogWyAgICA3LjY3NzczMF0gaWdiIDAwMDM6MDI6MDAuMSBl
bm8yOiBpZ2I6IGVubzIgTklDIExpbmsgaXMgVXAgMTAwMCBNYnBzIEZ1bGwgRHVwbGV4LCBGbG93
IENvbnRyb2w6IFJYL1RYClNlcCAxMyAxMjozODo0OSBkZWJpYW4ga2VybmVsOiBbICAgIDcuNjc3
OTA4XSBJUHY2OiBBRERSQ09ORihORVRERVZfQ0hBTkdFKTogZW5vMjogbGluayBiZWNvbWVzIHJl
YWR5ClNlcCAxMyAxMjozODo1MiBkZWJpYW4ga2VybmVsOiBbICAgMTAuODkzMzEzXSBtcHQzc2Fz
X2NtMDogcG9ydCBlbmFibGU6IFNVQ0NFU1MKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEJvb3RpbmcgTGludXggb24gcGh5c2ljYWwgQ1BVIDB4MDAwMDEyMDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIExpbnV4IHZlcnNpb24gNS4xMC4wLTgtYXJtNjQgKGRlYmlhbi1rZXJuZWxAbGlzdHMuZGVi
aWFuLm9yZykgKGdjYy0xMCAoRGViaWFuIDEwLjIuMS02KSAxMC4yLjEgMjAyMTAxMTAsIEdOVSBs
ZCAoR05VIEJpbnV0aWxzIGZvciBEZWJpYW4pIDIuMzUuMikgIzEgU01QIERlYmlhbiA1LjEwLjQ2
LTQgKDIwMjEtMDgtMDMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBlZmk6IEVGSSB2Mi43MCBieSBBbWVyaWNhbiBNZWdhdHJlbmRzClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEFDUEkgMi4wPTB4ZWU5MDAwMDAg
U01CSU9TIDMuMD0weGYxYjNmZjk4IEVTUlQ9MHhlYmQ2MWY5OCBNT0t2YXI9MHhlOTYxNzAwMCBS
Tkc9MHhmNzY2ZTgxOCBNRU1SRVNFUlZFPTB4ZWJkNWEzOTggClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IHNlZWRpbmcgZW50cm9weSBwb29sClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBzZWN1cmVib290OiBTZWN1
cmUgYm9vdCBkaXNhYmxlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gZXNydDogUmVzZXJ2aW5nIEVTUlQgc3BhY2UgZnJvbSAweDAwMDAwMDAwZWJkNjFmOTgg
dG8gMHgwMDAwMDAwMGViZDYxZmQwLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogRWFybHkgdGFibGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2Fi
bGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBS
U0RQIDB4MDAwMDAwMDBFRTkwMDAwMCAwMDAwMjQgKHYwMiBBbXBlcmUpClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDBFRThG
MDAwMCAwMDAwQTQgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1JICAwMTAwMDAxMykK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZBQ1Ag
MHgwMDAwMDAwMEVFOEQwMDAwIDAwMDExNCAodjA2IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJ
TlRMIDIwMTkwNTA5KQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogRFNEVCAweDAwMDAwMDAwRUU4NjAwMDAgMDE5QTk1ICh2MDIgQW1wZXJlIEphZGUg
ICAgIDAwMDAwMDAxIElOVEwgMjAyMDA3MTcpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBEQkcyIDB4MDAwMDAwMDBFRThFMDAwMCAwMDAwNUMgKHYw
MCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUwOSkKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEdURFQgMHgwMDAwMDAwMEVFOEMw
MDAwIDAwMDExMCAodjAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1NEVCAw
eDAwMDAwMDAwRUU4QjAwMDAgMDAwMDJEICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIElO
VEwgMjAxOTA1MDkpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBGSURUIDB4MDAwMDAwMDBFRTg1MDAwMCAwMDAwOUMgKHYwMSBBTEFTS0EgQSBNIEkg
ICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQQ1IgMHgwMDAwMDAwMEVFODQwMDAwIDAwMDA1MCAodjAy
IEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDUwMDBGKQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQkdSVCAweDAwMDAwMDAwRUU4MzAw
MDAgMDAwMDM4ICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBNQ0ZHIDB4
MDAwMDAwMDBFRTgyMDAwMCAwMDAwQUMgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgQU1Q
LiAwMTAwMDAxMykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IElPUlQgMHgwMDAwMDAwMEVFODEwMDAwIDAwMDYxMCAodjAwIEFtcGVyZSBBbHRyYSAg
ICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogUFBUVCAweDAwMDAwMDAwRUU3RjAwMDAgMDA2RTYwICh2MDIg
QW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTTElUIDB4MDAwMDAwMDBFRTdFMDAw
MCAwMDAwMkQgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQgMHgw
MDAwMDAwMEVFN0QwMDAwIDAwMDZEMCAodjAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAu
IDAxMDAwMDEzKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogQVBJQyAweDAwMDAwMDAwRUU4MDAwMDAgMDAxOUY0ICh2MDUgQW1wZXJlIEFsdHJhICAg
IDAwMDAwMDAzIEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBQQ0NUIDB4MDAwMDAwMDBFRTdDMDAwMCAwMDA1NzYgKHYwMiBB
bXBlcmUgQWx0cmEgICAgMDAwMDAwMDMgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFdTTVQgMHgwMDAwMDAwMEVFN0IwMDAw
IDAwMDAyOCAodjAxIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRlBEVCAweDAw
MDAwMDAwRUU3QTAwMDAgMDAwMDQ0ICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAg
MDEwMDAwMTMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBTUENSOiBjb25zb2xlOiBwbDAxMSxtbWlvMzIsMHgxMDAwMDI2MDAwMDAsMTE1MjAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBO
b2RlIDAgUFhNIDAgW21lbSAweDg4MzAwMDAwLTB4ODgzZmZmZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21l
bSAweDkwMDAwMDAwLTB4ZmZmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDgwMDAwMDAwMDAw
LTB4ODAwN2ZmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDgwMTAwMDAwMDAwLTB4ODBmZmZm
ZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBOVU1B
OiBOT0RFX0RBVEEgW21lbSAweDgwZmY4MDg1YjAwLTB4ODBmZjgwODdmZmZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBab25lIHJhbmdlczoKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BICAgICAgW21lbSAweDAw
MDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMGZmZmZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEzMiAgICBlbXB0eQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBOb3JtYWwgICBbbWVtIDB4MDAwMDAwMDEw
MDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBNb3ZhYmxlIHpvbmUgc3RhcnQgZm9yIGVhY2ggbm9kZQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRWFybHkgbWVtb3J5IG5vZGUg
cmFuZ2VzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAwMDAwODgzZmZmZmZdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDAwMDA5MDAwMDAwMC0weDAwMDAwMDAwOTFmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5
MjAwMDAwMC0weDAwMDAwMDAwOTNmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5NDAwMDAwMC0weDAw
MDAwMDAwZWUyZTZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTJlNzAwMC0weDAwMDAwMDAwZWU3NDBm
ZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0MTAwMC0weDAwMDAwMDAwZWU3NDRmZmZdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDAwMDBlZTc0NTAwMC0weDAwMDAwMDAwZWU3NGNmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0
ZDAwMC0weDAwMDAwMDAwZWU3NGRmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZTAwMC0weDAwMDAw
MDAwZWU3NGZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc1MDAwMC0weDAwMDAwMDAwZWU5NmZmZmZd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDBlZTk3MDAwMC0weDAwMDAwMDAwZWU5N2ZmZmZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDBlZTk4MDAwMC0weDAwMDAwMDAwZjA2NGZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmMDY1MDAw
MC0weDAwMDAwMDAwZjc2NmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmNzY3MDAwMC0weDAwMDAwMDAw
Zjc4NGZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmNzg1MDAwMC0weDAwMDAwMDAwZjdmZGZmZmZdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBmN2ZlMDAwMC0weDAwMDAwMDAwZmZjOGVmZmZdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBmZmM4ZjAwMC0weDAwMDAwMDAwZmZjOGZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM5MDAwMC0w
eDAwMDAwMDAwZmZmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDAwMDAwMDAwMC0weDAwMDAwODAwN2Zm
ZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDA4MDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJbml0bWVtIHNldHVwIG5v
ZGUgMCBbbWVtIDB4MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBPbiBub2RlIDAgdG90YWxwYWdl
czogMTY3MTE5MzYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
ICAgRE1BIHpvbmU6IDcxNzIgcGFnZXMgdXNlZCBmb3IgbWVtbWFwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiAwIHBhZ2VzIHJlc2VydmVk
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25l
OiA0NTkwMDggcGFnZXMsIExJRk8gYmF0Y2g6NjMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDI1Mzk1MiBwYWdlcyB1c2VkIGZvciBt
ZW1tYXAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9y
bWFsIHpvbmU6IDE2MjUyOTI4IHBhZ2VzLCBMSUZPIGJhdGNoOjYzClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbWE6IFJlc2VydmVkIDY0IE1pQiBhdCAweDAw
MDAwMDAwZmJjMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHBzY2k6IHByb2JpbmcgZm9yIGNvbmR1aXQgbWV0aG9kIGZyb20gQUNQSS4KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFBTQ0l2MS4xIGRldGVj
dGVkIGluIGZpcm13YXJlLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gcHNjaTogVXNpbmcgc3RhbmRhcmQgUFNDSSB2MC4yIGZ1bmN0aW9uIElEcwpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogTUlHUkFURV9JTkZP
X1RZUEUgbm90IHN1cHBvcnRlZC4KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIHBzY2k6IFNNQyBDYWxsaW5nIENvbnZlbnRpb24gdjEuMgpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgxMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgzMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDQwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4NTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDUwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4NjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg3MDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4ODAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDgwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg5MDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweGEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhhMDEwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweGIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDAw
MCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YzAxMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweGQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHhkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZTAwMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHhmMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAxMDAgLT4g
Tm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEwMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MTAwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
MTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDExMDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxMjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTMwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxNDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDE0MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTUwMDAwIC0+IE5vZGUg
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDE2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTYwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNzAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDE3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MTgwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxODAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MTkwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFhMDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgxYjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFjMDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxZDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFk
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWUwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDFmMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWYwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDIwMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MjEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMTAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIyMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MjIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIzMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MjQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgyNDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI1MDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgyNjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI2MDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjcwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyNzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwZXJjcHU6IEVtYmVkZGVkIDMzIHBhZ2VzL2NwdSBz
OTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IHM5NTE5MiByODE5MiBkMzE3ODQgdTEzNTE2OCBh
bGxvYz0zMyo0MDk2ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBwY3B1LWFsbG9jOiBbMF0gMDAgWzBdIDAxIFswXSAwMiBbMF0gMDMgWzBdIDA0IFswXSAwNSBb
MF0gMDYgWzBdIDA3IApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gcGNwdS1hbGxvYzogWzBdIDA4IFswXSAwOSBbMF0gMTAgWzBdIDExIFswXSAxMiBbMF0gMTMg
WzBdIDE0IFswXSAxNSAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHBjcHUtYWxsb2M6IFswXSAxNiBbMF0gMTcgWzBdIDE4IFswXSAxOSBbMF0gMjAgWzBdIDIx
IFswXSAyMiBbMF0gMjMgClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBwY3B1LWFsbG9jOiBbMF0gMjQgWzBdIDI1IFswXSAyNiBbMF0gMjcgWzBdIDI4IFswXSAy
OSBbMF0gMzAgWzBdIDMxIApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gcGNwdS1hbGxvYzogWzBdIDMyIFswXSAzMyBbMF0gMzQgWzBdIDM1IFswXSAzNiBbMF0g
MzcgWzBdIDM4IFswXSAzOSAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIHBjcHUtYWxsb2M6IFswXSA0MCBbMF0gNDEgWzBdIDQyIFswXSA0MyBbMF0gNDQgWzBd
IDQ1IFswXSA0NiBbMF0gNDcgClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNDggWzBdIDQ5IFswXSA1MCBbMF0gNTEgWzBdIDUyIFsw
XSA1MyBbMF0gNTQgWzBdIDU1IApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDU2IFswXSA1NyBbMF0gNTggWzBdIDU5IFswXSA2MCBb
MF0gNjEgWzBdIDYyIFswXSA2MyAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA2NCBbMF0gNjUgWzBdIDY2IFswXSA2NyBbMF0gNjgg
WzBdIDY5IFswXSA3MCBbMF0gNzEgClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNzIgWzBdIDczIFswXSA3NCBbMF0gNzUgWzBdIDc2
IFswXSA3NyBbMF0gNzggWzBdIDc5IApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEdJ
QyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogVmlydHVhbGl6YXRp
b24gSG9zdCBFeHRlbnNpb25zClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBDUFUgZmVhdHVyZXM6IGtlcm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiBmb3JjZWQg
T04gYnkgS0FTTFIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEtlcm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiAoS1BU
SSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0
dXJlczogZGV0ZWN0ZWQ6IEhhcmR3YXJlIGRpcnR5IGJpdCBtYW5hZ2VtZW50ClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVk
OiBTcGVjdHJlLXY0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhbHRlcm5hdGl2ZXM6IHBhdGNoaW5n
IGtlcm5lbCBjb2RlClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBCdWlsdCAxIHpvbmVsaXN0cywgbW9iaWxpdHkgZ3JvdXBpbmcgb24uICBUb3RhbCBwYWdlczog
MTY0NTA4MTIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFBv
bGljeSB6b25lOiBOb3JtYWwKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEtlcm5lbCBjb21tYW5kIGxpbmU6IEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEw
LjAtOC1hcm02NCByb290PVVVSUQ9NmMwODMyN2MtNGJiZC00MzQxLWE0MjctYTcyYmMxNTMzODk1
IHJvIHF1aWV0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
cmludGs6IGxvZ19idWZfbGVuIGluZGl2aWR1YWwgbWF4IGNwdSBjb250cmlidXRpb246IDQwOTYg
Ynl0ZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50
azogbG9nX2J1Zl9sZW4gdG90YWwgY3B1X2V4dHJhIGNvbnRyaWJ1dGlvbnM6IDMyMzU4NCBieXRl
cwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBs
b2dfYnVmX2xlbiBtaW4gc2l6ZTogMTMxMDcyIGJ5dGVzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuOiA1MjQyODggYnl0ZXMK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogZWFy
bHkgbG9nIGJ1ZiBmcmVlOiAxMTkwMTYoOTAlKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gRGVudHJ5IGNhY2hlIGhhc2ggdGFibGUgZW50cmllczogODM4ODYw
OCAob3JkZXI6IDE0LCA2NzEwODg2NCBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVz
OiA0MTk0MzA0IChvcmRlcjogMTMsIDMzNTU0NDMyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBtZW0gYXV0by1pbml0OiBzdGFjazpv
ZmYsIGhlYXAgYWxsb2M6b24sIGhlYXAgZnJlZTpvZmYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNvZnR3YXJlIElPIFRMQjogbWFwcGVkIFttZW0gMHgwMDAw
MDAwMGRlZDkwMDAwLTB4MDAwMDAwMDBlMmQ5MDAwMF0gKDY0TUIpClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNZW1vcnk6IDI1NTE2MjhLLzY2ODQ3NzQ0SyBh
dmFpbGFibGUgKDExNzc2SyBrZXJuZWwgY29kZSwgMjQzNksgcndkYXRhLCA3MDA4SyByb2RhdGEs
IDU0NDBLIGluaXQsIDU5OEsgYnNzLCAxNTg1NTQ4SyByZXNlcnZlZCwgNjU1MzZLIGNtYS1yZXNl
cnZlZCkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJhbmRv
bTogZ2V0X3JhbmRvbV91NjQgY2FsbGVkIGZyb20gX19rbWVtX2NhY2hlX2NyZWF0ZSsweDM4LzB4
NTYwIHdpdGggY3JuZ19pbml0PTAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIFNMVUI6IEhXYWxpZ249NjQsIE9yZGVyPTAtMywgTWluT2JqZWN0cz0wLCBDUFVz
PTgwLCBOb2Rlcz0xClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBmdHJhY2U6IGFsbG9jYXRpbmcgMzg1MzUgZW50cmllcyBpbiAxNTEgcGFnZXMKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGZ0cmFjZTogYWxsb2NhdGVkIDE1
MSBwYWdlcyB3aXRoIDUgZ3JvdXBzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSByY3U6IEhpZXJhcmNoaWNhbCBSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IAlSQ1UgcmVzdHJpY3Rp
bmcgQ1BVcyBmcm9tIE5SX0NQVVM9MjU2IHRvIG5yX2NwdV9pZHM9ODAuClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAJUnVkZSB2YXJpYW50IG9mIFRhc2tzIFJD
VSBlbmFibGVkLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
CVRyYWNpbmcgdmFyaWFudCBvZiBUYXNrcyBSQ1UgZW5hYmxlZC4KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogUkNVIGNhbGN1bGF0ZWQgdmFsdWUgb2Yg
c2NoZWR1bGVyLWVubGlzdG1lbnQgZGVsYXkgaXMgMjUgamlmZmllcy4KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogQWRqdXN0aW5nIGdlb21ldHJ5IGZv
ciByY3VfZmFub3V0X2xlYWY9MTYsIG5yX2NwdV9pZHM9ODAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE5SX0lSUVM6IDY0LCBucl9pcnFzOiA2NCwgcHJlYWxs
b2NhdGVkIGlycXM6IDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEdJQ3YzOiBHSUM6IFVzaW5nIHNwbGl0IEVPSS9EZWFjdGl2YXRlIG1vZGUKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiA2NzIgU1BJcyBpbXBs
ZW1lbnRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lD
djM6IDAgRXh0ZW5kZWQgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IERpc3RyaWJ1dG9yIGhhcyBubyBSYW5nZSBTZWxl
Y3RvciBzdXBwb3J0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBHSUN2MzogMTYgUFBJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IENQVTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVjMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDAgLT4gTm9kZSAwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMg
MSAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IFNSQVQ6IFBYTSAwIC0+IElUUyAyIC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDMgLT4gTm9kZSAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJ
VFMgNCAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA1IC0+IE5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDYgLT4gTm9kZSAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAt
PiBJVFMgNyAtPiBOb2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDQwMDAwLTB4MTAwMTAwMDVmZmZmXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA0MDAw
MDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAyYzAwMDAgKGluZGlyZWN0LCBlc3ogOCwg
cHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENv
bGxlY3Rpb25zIEA4MDAwMDJkMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAx
MDAwNjAwMDAtMHgxMDAxMDAwN2ZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDYwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZp
Y2VzIEA4MDAwMDJmMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAw
NjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzAwMDAw
IChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDA4MDAwMC0weDEwMDEwMDA5ZmZm
Zl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAw
MDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwMzIwMDAwIChpbmRp
cmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA4MDAwMDogYWxsb2NhdGVkIDMyNzY4
IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzMzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRL
LCBzaHIgMSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElU
UyBbbWVtIDB4MTAwMTAwMGEwMDAwLTB4MTAwMTAwMGJmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBhMDAwMDogYWxsb2Nh
dGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzNTAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywg
c2hyIDEpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNA
MHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25z
IEA4MDAwMDM2MDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwYzAwMDAt
MHgxMDAxMDAwZGZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAw
MDM4MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwYzAwMDA6IGFs
bG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzkwMDAwIChmbGF0LCBl
c3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDBlMDAwMC0weDEwMDEwMDBmZmZmZl0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAw
ZTAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwM2IwMDAwIChpbmRpcmVjdCwgZXN6
IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBlMDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVw
dCBDb2xsZWN0aW9ucyBAODAwMDAzYzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4
MTAwMTAwMTAwMDAwLTB4MTAwMTAwMTFmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEwMDAwMDogYWxsb2NhdGVkIDgxOTIg
RGV2aWNlcyBAODAwMDAzZTAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAw
MTAwMTAwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDNm
MDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAxMjAwMDAtMHgxMDAxMDAx
M2ZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNA
MHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDgxMDAwMCAo
aW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAxMjAwMDA6IGFsbG9jYXRlZCAz
Mjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwODIwMDAwIChmbGF0LCBlc3ogMiwgcHN6
IDY0Sywgc2hyIDEpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBHSUN2MzogdXNpbmcgTFBJIHByb3BlcnR5IHRhYmxlIEAweDAwMDAwODAwMDA4MzAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBDUFUwOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg0MDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSSBHVERUOiBmb3VuZCAxIG1l
bW9yeS1tYXBwZWQgdGltZXIgYmxvY2socykuClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBhcmNoX3RpbWVyOiBjcDE1IGFuZCBtbWlvIHRpbWVyKHMpIHJ1bm5p
bmcgYXQgMjUuMDBNSHogKHBoeXMvcGh5cykuClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBjbG9ja3NvdXJjZTogYXJjaF9zeXNfY291bnRlcjogbWFzazogMHhm
ZmZmZmZmZmZmZmZmZiBtYXhfY3ljbGVzOiAweDVjNDA5MzliNSwgbWF4X2lkbGVfbnM6IDQ0MDc5
NTIwMjY0NiBucwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
c2NoZWRfY2xvY2s6IDU2IGJpdHMgYXQgMjVNSHosIHJlc29sdXRpb24gNDBucywgd3JhcHMgZXZl
cnkgNDM5ODA0NjUxMTEwMG5zClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDUzXSBDb25zb2xlOiBjb2xvdXIgZHVtbXkgZGV2aWNlIDgweDI1ClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDc2XSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIwMjAw
OTI1ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMjcyXSBDYWxpYnJh
dGluZyBkZWxheSBsb29wIChza2lwcGVkKSwgdmFsdWUgY2FsY3VsYXRlZCB1c2luZyB0aW1lciBm
cmVxdWVuY3kuLiA1MC4wMCBCb2dvTUlQUyAobHBqPTEwMDAwMCkKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAyNzRdIHBpZF9tYXg6IGRlZmF1bHQ6IDgxOTIwIG1pbmlt
dW06IDY0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMwN10gTFNN
OiBTZWN1cml0eSBGcmFtZXdvcmsgaW5pdGlhbGl6aW5nClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMzEzXSBZYW1hOiBkaXNhYmxlZCBieSBkZWZhdWx0OyBlbmFibGUg
d2l0aCBzeXNjdGwga2VybmVsLnlhbWEuKgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDMzMF0gQXBwQXJtb3I6IEFwcEFybW9yIGluaXRpYWxpemVkClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzMxXSBUT01PWU8gTGludXggaW5pdGlhbGl6
ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzNzJdIE1vdW50LWNh
Y2hlIGhhc2ggdGFibGUgZW50cmllczogMTMxMDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywg
bGluZWFyKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDM5N10gTW91
bnRwb2ludC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1
NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDEzMDRdIHJjdTogSGllcmFyY2hpY2FsIFNSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDQ2XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEw
MDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMTQ0OF0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNy
ZWF0ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NDldIFBsYXRm
b3JtIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDUxXSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEw
MDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMTQ1Ml0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0
ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NTNdIFBsYXRmb3Jt
IE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDU0XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEw
MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMTQ1NV0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAxMjAwMDAgZG9tYWluIGNyZWF0ZWQK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NTldIFBDSS9NU0k6IElU
U0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTQ2MV0gUENJL01TSTogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBj
cmVhdGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDYyXSBQQ0kv
TVNJOiBJVFNAMHgxMDAxMDAwODAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NjNdIFBDSS9NU0k6IElUU0AweDEwMDEwMDBhMDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQ2
Nl0gUENJL01TSTogSVRTQDB4MTAwMTAwMGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDY4XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAw
ZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDE0NzBdIFBDSS9NU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQ3MV0gUENJL01TSTogSVRTQDB4
MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAxNDc0XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9tYWluIGNy
ZWF0ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NzVdIGZzbC1t
YyBNU0k6IElUU0AweDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQ3Nl0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDgw
MDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxNDc4XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0NzldIGZzbC1tYyBNU0k6IElU
U0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTQ4MF0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFp
biBjcmVhdGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDgxXSBm
c2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0ODJdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEw
MDEyMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMTQ4OV0gUmVtYXBwaW5nIGFuZCBlbmFibGluZyBFRkkgc2VydmljZXMuClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAzOTYyXSBzbXA6IEJyaW5naW5nIHVwIHNl
Y29uZGFyeSBDUFVzIC4uLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NDE5MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDQyMTBdIEdJQ3YzOiBDUFUxOiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDFhMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YzAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDQyMjhdIEdJQ3YzOiBDUFUxOiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg1MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNDI2MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNDI3MF0gQ1BVMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MWEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNDc1M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ3NjldIEdJQ3YzOiBDUFUyOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDE0MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2NDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ3ODZdIEdJQ3YzOiBDUFUyOiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNDgyMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNDgzMl0gQ1BVMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMTQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNTcxM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTMKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU3MzBdIEdJQ3YzOiBDUFUzOiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDFjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4NDAwMDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU3NDldIEdJQ3YzOiBDUFUzOiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg3MDAwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTc4Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNTc5MV0gQ1BVMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMWMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNjEyNV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDYxMzldIEdJQ3YzOiBDUFU0OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDEwMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1NDAwMDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDYxNThdIEdJQ3YzOiBDUFU0OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg4MDAwMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjE5M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNjIwMl0gQ1BVNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMTAwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNjUzMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY1NDddIEdJQ3YzOiBDUFU1OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDE4MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3NDAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY1NjVdIEdJQ3YzOiBDUFU1OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg5MDAwMApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjYwMF0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjYxMF0gQ1BVNTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMTgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNjkyN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY5NDNdIEdJQ3YzOiBDUFU2OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDE2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2YzAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY5NjJdIEdJQ3YzOiBDUFU2OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhhMDAwMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjk5N10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzAwN10gQ1BVNjogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMTYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNzMyNV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDczNDNdIEdJQ3YzOiBDUFU3OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDFlMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4YzAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDczNjJdIEdJQ3YzOiBDUFU3OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhiMDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzM5N10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzQwN10gQ1BVNzogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMWUwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNzczMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTgK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDc3NDRdIEdJQ3YzOiBDUFU4
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNjMDAwMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzc2M10gR0lDdjM6IENQVTg6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGMwMDAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3ODAwXSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3ODEwXSBDUFU4OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwYTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MTExXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
OQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODEyOV0gR0lDdjM6IENQ
VTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDljMDAw
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODE0OF0gR0lDdjM6IENQ
VTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGQwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MTc5XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MTg5XSBDUFU5OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NTAzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMTAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg1MTZdIEdJQ3Yz
OiBDUFUxMDogZm91bmQgcmVkaXN0cmlidXRvciBjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0
NDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg1MzVdIEdJQ3Yz
OiBDUFUxMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4
ZTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg1NzFdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg1ODJdIENQVTEwOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTE3XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg5MzZd
IEdJQ3YzOiBDUFUxMTogZm91bmQgcmVkaXN0cmlidXRvciAyNDAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwYTQwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTU2
XSBHSUN2MzogQ1BVMTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwOGYwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTg5
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTk5XSBDUFUx
MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjQwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTMyOV0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTEyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA5MzQyXSBHSUN2MzogQ1BVMTI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgODAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwMzQwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA5MzYxXSBHSUN2MzogQ1BVMTI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwOTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA5Mzk3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NDA4
XSBDUFUxMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDgwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTc0MF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA5NzU4XSBHSUN2MzogQ1BVMTM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDk0MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwOTc3OF0gR0lDdjM6IENQVTEzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDkxMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwOTgxMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwOTgyMV0gQ1BVMTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIwMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAxMjdd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMDE0MV0gR0lDdjM6IENQVTE0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGUw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRjMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMDE2Ml0gR0lDdjM6IENQVTE0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkyMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMDE5OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMDIxMV0gQ1BVMTQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBl
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA1MTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMDUzMl0gR0lDdjM6IENQVTE1OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDI2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhYzAwMDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTA1NTJdIEdJQ3YzOiBDUFUxNTogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5MzAwMDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTA1ODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTA1OTddIENQVTE1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAyNjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwOTA4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTYKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA5MTldIEdJQ3YzOiBDUFUxNjogZm91bmQgcmVk
aXN0cmlidXRvciAyMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxYzAwMDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA5MzldIEdJQ3YzOiBDUFUxNjogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NDAwMDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA5NzddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTA5ODldIENQVTE2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAwMjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDExMzExXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTcKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzMjNdIEdJQ3YzOiBDUFUxNzogZm91
bmQgcmVkaXN0cmlidXRvciA0MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyNDAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzNDRdIEdJQ3YzOiBDUFUxNzogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NTAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzODJdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzOTRdIENQVTE3OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAwNDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDExNzEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTgK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTE3MjNdIEdJQ3YzOiBDUFUx
ODogZm91bmQgcmVkaXN0cmlidXRvciAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE0MDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTc0NF0gR0lDdjM6IENQVTE4OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk2MDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTc4M10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTc5NV0gQ1BVMTg6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDAwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTIxMTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUx
OQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjEyNF0gR0lDdjM6IENQ
VTE5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJjMDAw
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE0NV0gR0lDdjM6IENQ
VTE5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk3MDAw
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE4Ml0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE5NV0gQ1BVMTk6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA2MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1MTZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUyMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjUzMF0gR0lD
djM6IENQVTIwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEzMDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA2MDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1NTFdIEdJ
Q3YzOiBDUFUyMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDA5ODAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1ODZdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1OThdIENQVTIwOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTExXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVMjEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI5
MjddIEdJQ3YzOiBDUFUyMTogZm91bmQgcmVkaXN0cmlidXRvciAxYjAwMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwODAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEy
OTQ3XSBHSUN2MzogQ1BVMjE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwOTkwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEy
OTgwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTkyXSBD
UFUyMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWIwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzMwOF0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTIyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDEzMzIzXSBHSUN2MzogQ1BVMjI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTUwMDAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDY4MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxMzM0M10gR0lDdjM6IENQVTIyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDlhMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxMzM3OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
MzM5MV0gQ1BVMjI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE1MDAwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM3MDddIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxMzcyNF0gR0lDdjM6IENQVTIzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDAw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA4ODAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTM3NDVdIEdJQ3YzOiBDUFUyMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDA5YjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTM3ODBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTM3OTJdIENQVTIzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0
MTExXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjQKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTQxMjVdIEdJQ3YzOiBDUFUyNDogZm91bmQgcmVkaXN0cmlidXRv
ciAxMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTgwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE0MTQ2XSBHSUN2MzogQ1BVMjQ6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWMwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE0MTgxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE0MTk0XSBDUFUyNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMTEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNDUxNF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI1ClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0NTMwXSBHSUN2MzogQ1BVMjU6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTkwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc4MDAwMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDU1Ml0gR0lDdjM6IENQVTI1OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlkMDAwMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDU4NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNDU5OF0gQ1BVMjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDE5MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTUxMDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNgpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTEyNV0gR0lDdjM6IENQVTI2OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDE3MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3MDAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUxNDZdIEdJQ3YzOiBDUFUyNjogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ZTAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUxODJdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUxOTVdIENQVTI2OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAxNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NTEwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjcK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTU1MjhdIEdJQ3YzOiBDUFUy
NzogZm91bmQgcmVkaXN0cmlidXRvciAxZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTAwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NTUwXSBHSUN2MzogQ1BV
Mjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWYwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NTg1XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NTk4XSBDUFUyNzogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTkxMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTI4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTIyXSBHSUN2
MzogQ1BVMjg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
NDAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTQ0XSBHSUN2
MzogQ1BVMjg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTgxXSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1OTk0XSBDUFUyODogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjMxMl0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTI5ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2MzMw
XSBHSUN2MzogQ1BVMjk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjMwMDAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMGEwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjM1
Ml0gR0lDdjM6IENQVTI5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGExMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjM4
M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjM5Nl0gQ1BV
Mjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIzMDAwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY3MDhdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFUzMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNjcyMV0gR0lDdjM6IENQVTMwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGQwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDQ4MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNjc0NF0gR0lDdjM6IENQVTMwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGEyMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNjc4MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjc5
M10gQ1BVMzA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBkMDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcxMTBdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNzEyOV0gR0lDdjM6IENQVTMxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI1MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDBhODAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTcxNTFdIEdJQ3YzOiBDUFUzMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBhMzAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTcxODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTcxOThdIENQVTMxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNTAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NTA5
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTc1MjFdIEdJQ3YzOiBDUFUzMjogZm91bmQgcmVkaXN0cmlidXRvciA5
MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzODAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTc1NDRdIEdJQ3YzOiBDUFUzMjogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTc1ODBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTc1OTVdIENQVTMyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAw
OTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3OTE1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzMKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTc5MzNdIEdJQ3YzOiBDUFUzMzogZm91bmQgcmVkaXN0cmli
dXRvciAyMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTgwMDAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3OTU2XSBHSUN2MzogQ1BVMzM6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTUwMDAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3OTkwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE4MDA1XSBDUFUzMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMjEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODMwN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM0ClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MzIyXSBHSUN2MzogQ1BVMzQ6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTAwMDAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MzQ2XSBHSUN2MzogQ1BVMzQ6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTYwMDAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MzgzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4Mzk3XSBDUFUzNDogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMGYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODcxM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM1ClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NzM0XSBHSUN2MzogQ1BVMzU6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMjcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGIwMDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODc1N10gR0lDdjM6IENQVTM1OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE3MDAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODc5MV0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODgwNV0gQ1BVMzU6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDI3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTkxMTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUz
NgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTEyNF0gR0lDdjM6IENQ
VTM2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIwMDAw
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTE0OF0gR0lDdjM6IENQ
VTM2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE4MDAw
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTE4Nl0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTIwMV0gQ1BVMzY6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk1MTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUzNwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTUyNF0gR0lD
djM6IENQVTM3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDI4MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTU0OF0gR0lD
djM6IENQVTM3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGE5MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTU4Nl0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTYwMV0gQ1BVMzc6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk5MTZdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFUzOApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTky
Nl0gR0lDdjM6IENQVTM4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDE4MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTk1
Ml0gR0lDdjM6IENQVTM4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGFhMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTk5
MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDAwNl0gQ1BV
Mzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAxMDAwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAzMTNdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFUzOQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMDMyNV0gR0lDdjM6IENQVTM5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDcwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDMwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMDM1MF0gR0lDdjM6IENQVTM5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGFiMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMDM4Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDQw
M10gQ1BVMzk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA3MDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA3MTBdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyMDcyNV0gR0lDdjM6IENQVTQwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA1ZTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjA3NDddIEdJQ3YzOiBDUFU0MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBhYzAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjA3ODBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjA3OTVdIENQVTQwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMjAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMTEy
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjExMjhdIEdJQ3YzOiBDUFU0MTogZm91bmQgcmVkaXN0cmlidXRvciAx
YTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwN2UwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIxMTUwXSBHSUN2MzogQ1BVNDE6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYWQwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIxMTgzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIxMTk3XSBDUFU0MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MWEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMTUxMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQyClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNTI2XSBHSUN2MzogQ1BVNDI6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMTQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY2MDAwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTU0OV0gR0lDdjM6IENQVTQyOiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFlMDAwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTU4NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMTYwMF0gQ1BVNDI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDE0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjE5MTRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MwpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTkzMV0gR0lDdjM6IENQVTQzOiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDFjMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4NjAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE5NTVdIEdJQ3YzOiBDUFU0MzogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhZjAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE5ODhdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjIwMDJdIENQVTQzOiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAxYzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIyMzEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDQKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjIzMjddIEdJQ3YzOiBDUFU0NDog
Zm91bmQgcmVkaXN0cmlidXRvciAxMDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTYwMDAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMzUyXSBHSUN2MzogQ1BVNDQ6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjAwMDAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMzg3XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyNDAyXSBDUFU0NDogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjcxNF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTQ1ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyNzMwXSBHSUN2Mzog
Q1BVNDU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc2
MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjc1NF0gR0lDdjM6
IENQVTQ1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIx
MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjc4Nl0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjgwMV0gQ1BVNDU6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMxMTVdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU0NgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzEzMV0g
R0lDdjM6IENQVTQ2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA2ZTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMxNTZd
IEdJQ3YzOiBDUFU0NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBiMjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMxOTJd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMyMDhdIENQVTQ2
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNjAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNTE0XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNDcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjM1MzJdIEdJQ3YzOiBDUFU0NzogZm91bmQgcmVkaXN0cmlidXRvciAxZTAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwOGUwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDIzNTU3XSBHSUN2MzogQ1BVNDc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYjMwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDIzNTkyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjA4
XSBDUFU0NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzkxNV0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDIzOTI3XSBHSUN2MzogQ1BVNDg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYTAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwM2UwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDIzOTUzXSBHSUN2MzogQ1BVNDg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYjQwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDIzOTkwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI0MDA2XSBDUFU0ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGEwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDMxOF0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ5ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI0MzM2XSBHSUN2MzogQ1BVNDk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIw
MTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDllMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNDM2MV0gR0lDdjM6IENQVTQ5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI1MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNDM5Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNDQxMV0gQ1BVNDk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIy
MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ3MTRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNDcyN10gR0lDdjM6IENQVTUwOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIGMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNDc1Ml0gR0lDdjM6IENQVTUwOiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNDc4N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNDgwM10gQ1BVNTA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDBjMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUxMTldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTEzOF0gR0lDdjM6IENQVTUxOiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDI0MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhNjAwMDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUxNjRdIEdJQ3YzOiBDUFU1MTogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiNzAwMDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUxOTddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjUyMTNdIENQVTUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAyNDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1NTE2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTIKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU1MjhdIEdJQ3YzOiBDUFU1MjogZm91
bmQgcmVkaXN0cmlidXRvciA4MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzNjAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU1NTRdIEdJQ3YzOiBDUFU1MjogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiODAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU1OTJdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU2MDhdIENQVTUyOiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAwODAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1OTEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTMK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU5MzFdIEdJQ3YzOiBDUFU1
MzogZm91bmQgcmVkaXN0cmlidXRvciAyMDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTYwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1OTU3XSBHSUN2MzogQ1BV
NTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjkwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1OTkyXSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDA4XSBDUFU1MzogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjAwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjMxMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTU0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MzIzXSBHSUN2
MzogQ1BVNTQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
NGUwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MzUyXSBHSUN2
MzogQ1BVNTQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YmEwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2Mzg5XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NDA3XSBDUFU1NDogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjcxNV0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTU1ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NzM0
XSBHSUN2MzogQ1BVNTU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjYwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMGFlMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjc2
MF0gR0lDdjM6IENQVTU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGJiMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjc5
NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjgxMl0gQ1BV
NTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI2MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjcxMTVdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU1NgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNzEyNV0gR0lDdjM6IENQVTU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIwMTAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDFlMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNzE1Ml0gR0lDdjM6IENQVTU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGJjMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNzE5MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzIw
N10gQ1BVNTY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAyMDEwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc1MTZdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNzUyN10gR0lDdjM6IENQVTU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDQwMTAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDI2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNzU1NF0gR0lDdjM6IENQVTU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGJkMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNzU5Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNzYwOV0gQ1BVNTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA0MDEwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc5MTVd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1OApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNzkyNV0gR0lDdjM6IENQVTU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDAxNjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjc5NTNdIEdJQ3YzOiBDUFU1ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjc5OTFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjgwMDhdIENQVTU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMDAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
MzE5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTkKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjgzMzBdIEdJQ3YzOiBDUFU1OTogZm91bmQgcmVkaXN0cmlidXRv
ciA2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyZTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjgzNThdIEdJQ3YzOiBDUFU1OTogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjgzOTRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjg0MTJdIENQVTU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAwNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4NzE1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg3MzBdIEdJQ3YzOiBDUFU2MDogZm91bmQgcmVkaXN0
cmlidXRvciAxMzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjIwMDAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NzU2XSBHSUN2MzogQ1BVNjA6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzAwMDAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NzkzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI4ODEwXSBDUFU2MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMTMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyOTExMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYxClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MTI3XSBHSUN2MzogQ1BVNjE6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMWIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDgyMDAwMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTE1M10gR0lDdjM6IENQVTYxOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMxMDAwMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTE4N10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTIwNF0gQ1BVNjE6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDFiMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjk1MDVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MgpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTUyMF0gR0lDdjM6IENQVTYy
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2YTAwMDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjk1NDddIEdJQ3YzOiBDUFU2
MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMjAwMDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjk1ODJdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjk1OTldIENQVTYyOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5OTEwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNjMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjk5MjddIEdJQ3Yz
OiBDUFU2MzogZm91bmQgcmVkaXN0cmlidXRvciAxZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
OGEwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5OTU0XSBHSUN2
MzogQ1BVNjM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YzMwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5OTg3XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwMDA1XSBDUFU2MzogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDMxM10gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTY0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwMzI4
XSBHSUN2MzogQ1BVNjQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTEwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDVhMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDM1
NF0gR0lDdjM6IENQVTY0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGM0MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDM4
OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDQwNl0gQ1BV
NjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDExMDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA3MThdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU2NQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMDczNV0gR0lDdjM6IENQVTY1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE5MDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA3YTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzA3NjFdIEdJQ3YzOiBDUFU2NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBjNTAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzA3OTZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA4
MTJdIENQVTY1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxOTAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMTE1XSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzExMzJdIEdJQ3YzOiBDUFU2NjogZm91bmQgcmVkaXN0cmlidXRvciAxNzAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwNzIwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMxMTU5XSBHSUN2MzogQ1BVNjY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzYwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMxMTk2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMxMjEyXSBDUFU2NjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTcwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTUy
MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY3ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMxNTM4XSBHSUN2MzogQ1BVNjc6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MWYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDkyMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMTU2Nl0gR0lDdjM6IENQVTY3OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM3MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMTU5OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMTYxNl0gQ1BVNjc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDFmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzE5MTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2OApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTkyM10gR0lDdjM6IENQVTY4OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIGIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQyMDAwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTk1MV0gR0lDdjM6IENQVTY4OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM4MDAwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTk4N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMjAwNF0gQ1BVNjg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDBiMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzIzMTVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2OQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjMzNF0gR0lDdjM6IENQVTY5OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDIzMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhMjAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIzNjJdIEdJQ3YzOiBDUFU2OTogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjOTAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIzOTVdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI0MTJdIENQVTY5OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAyMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyNzA2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI3MjBdIEdJQ3YzOiBDUFU3MDog
Zm91bmQgcmVkaXN0cmlidXRvciBkMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0YTAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI3NDldIEdJQ3YzOiBDUFU3MDog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjYTAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI3ODVdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI4MDFdIENQVTcwOiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwZDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMTE0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
NzEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzMxMzRdIEdJQ3YzOiBD
UFU3MTogZm91bmQgcmVkaXN0cmlidXRvciAyNTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYWEw
MDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMTYyXSBHSUN2Mzog
Q1BVNzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2Iw
MDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMTk1XSBhcmNoX3Rp
bWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjEyXSBDUFU3MTogQm9vdGVk
IHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzUxNF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTcyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNTI3XSBH
SUN2MzogQ1BVNzI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwM2EwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNTU2XSBH
SUN2MzogQ1BVNzI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwY2MwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNTkzXSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNjEwXSBDUFU3Mjog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDkwMTAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzkxOV0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTczClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMz
OTM4XSBHSUN2MzogQ1BVNzM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjEwMTAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDlhMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
Mzk2N10gR0lDdjM6IENQVTczOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMGNkMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
Mzk5OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDAxNV0g
Q1BVNzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIxMDEwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQzMTZdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFU3NApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNDMzMV0gR0lDdjM6IENQVTc0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGYwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDUyMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNDM2MF0gR0lDdjM6IENQVTc0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGNlMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNDM5N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
NDQxNV0gQ1BVNzQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBmMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ3MTBdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNDczMV0gR0lDdjM6IENQVTc1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI3MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDBiMjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzQ3NjBdIEdJQ3YzOiBDUFU3NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBjZjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzQ3OTRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzQ4MTFdIENQVTc1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNzAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1
MTEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzYKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzUxMjRdIEdJQ3YzOiBDUFU3NjogZm91bmQgcmVkaXN0cmlidXRv
ciAzMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyMjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzUxNTVdIEdJQ3YzOiBDUFU3NjogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzUxOTNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzUyMTFdIENQVTc2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAwMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM1NTE4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzcKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1MjldIEdJQ3YzOiBDUFU3NzogZm91bmQgcmVkaXN0
cmlidXRvciA1MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyYTAwMDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1NjBdIEdJQ3YzOiBDUFU3NzogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMTAwMDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1OTddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzU2MTRdIENQVTc3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAwNTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM1OTI0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzgKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5MzRdIEdJQ3YzOiBDUFU3ODogZm91bmQg
cmVkaXN0cmlidXRvciAxMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxYTAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5NjRdIEdJQ3YzOiBDUFU3ODogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMjAwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYwMDJdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYwMjFdIENQVTc4OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAwMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM2MzEwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzkKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYzMjJdIEdJQ3YzOiBDUFU3OTog
Zm91bmQgcmVkaXN0cmlidXRvciA3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzMjAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYzNTJdIEdJQ3YzOiBDUFU3OTog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMzAwMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYzODldIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzY0MDldIENQVTc5OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDcxXSBzbXA6IEJyb3VnaHQgdXAgMSBub2RlLCA4MCBD
UFVzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDkzXSBTTVA6IFRv
dGFsIG9mIDgwIHByb2Nlc3NvcnMgYWN0aXZhdGVkLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzNjQ5NF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogUHJpdmlsZWdlZCBB
Y2Nlc3MgTmV2ZXIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzY0OTRd
IENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IExTRSBhdG9taWMgaW5zdHJ1Y3Rpb25zClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDk1XSBDUFUgZmVhdHVyZXM6IGRldGVj
dGVkOiBVc2VyIEFjY2VzcyBPdmVycmlkZQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzNjQ5Nl0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogMzItYml0IEVMMCBTdXBwb3J0
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDk3XSBDUFUgZmVhdHVy
ZXM6IGRldGVjdGVkOiBDb21tb24gbm90IFByaXZhdGUgdHJhbnNsYXRpb25zClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDk3XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVk
OiBEYXRhIGNhY2hlIGNsZWFuIHRvIFBvaW50IG9mIFBlcnNpc3RlbmNlClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDk4XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBS
QVMgRXh0ZW5zaW9uIFN1cHBvcnQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzY0OTldIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IERhdGEgY2FjaGUgY2xlYW4gdG8gdGhl
IFBvVSBub3QgcmVxdWlyZWQgZm9yIEkvRCBjb2hlcmVuY2UKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzY0OTldIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IENSQzMyIGlu
c3RydWN0aW9ucwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjUwMF0g
Q1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3BlY3VsYXRpdmUgU3RvcmUgQnlwYXNzaW5nIFNhZmUg
KFNTQlMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDU4OTkwXSBDUFU6
IEFsbCBDUFUocykgc3RhcnRlZCBhdCBFTDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC41NzczOTNdIG5vZGUgMCBkZWZlcnJlZCBwYWdlcyBpbml0aWFsaXNlZCBpbiA1MTZt
cwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NzUxOV0gZGV2dG1wZnM6
IGluaXRpYWxpemVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc4MDk0
XSBSZWdpc3RlcmVkIGNwMTVfYmFycmllciBlbXVsYXRpb24gaGFuZGxlcgpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3ODExNl0gUmVnaXN0ZXJlZCBzZXRlbmQgZW11bGF0
aW9uIGhhbmRsZXIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzgxMThd
IEtBU0xSIGVuYWJsZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41Nzgx
NzhdIGNsb2Nrc291cmNlOiBqaWZmaWVzOiBtYXNrOiAweGZmZmZmZmZmIG1heF9jeWNsZXM6IDB4
ZmZmZmZmZmYsIG1heF9pZGxlX25zOiA3NjQ1MDQxNzg1MTAwMDAwIG5zClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc4MjMwXSBmdXRleCBoYXNoIHRhYmxlIGVudHJpZXM6
IDMyNzY4IChvcmRlcjogOSwgMjA5NzE1MiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU3ODY3M10gcGluY3RybCBjb3JlOiBpbml0aWFsaXplZCBw
aW5jdHJsIHN1YnN5c3RlbQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3
ODgzMl0gU01CSU9TIDMuMy4wIHByZXNlbnQuClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNTc4ODM1XSBETUk6IEdJR0FCWVRFIFIxNTItUDMwLTAwL01QMzItQVIxLTAwLCBC
SU9TIEYxMyAoU0NQOiAxLjUuMjAyMTA0MjYpIDA1LzIxLzIwMjEKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC41NzkwNjddIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1p
bHkgMTYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41Nzk2NjFdIERNQTog
cHJlYWxsb2NhdGVkIDQwOTYgS2lCIEdGUF9LRVJORUwgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRp
b25zClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc5Nzg3XSBETUE6IHBy
ZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VSTkVMfEdGUF9ETUEgcG9vbCBmb3IgYXRvbWljIGFs
bG9jYXRpb25zClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc5OTExXSBE
TUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VSTkVMfEdGUF9ETUEzMiBwb29sIGZvciBh
dG9taWMgYWxsb2NhdGlvbnMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41
Nzk5MThdIGF1ZGl0OiBpbml0aWFsaXppbmcgbmV0bGluayBzdWJzeXMgKGRpc2FibGVkKQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3OTk1OF0gYXVkaXQ6IHR5cGU9MjAw
MCBhdWRpdCgwLjU3NjoxKTogc3RhdGU9aW5pdGlhbGl6ZWQgYXVkaXRfZW5hYmxlZD0wIHJlcz0x
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgwMDQ2XSB0aGVybWFsX3N5
czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVybm9yICdmYWlyX3NoYXJlJwpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDA0N10gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQg
dGhlcm1hbCBnb3Zlcm5vciAnYmFuZ19iYW5nJwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU4MDA0OF0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5v
ciAnc3RlcF93aXNlJwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDA0
OF0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAndXNlcl9zcGFjZScK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODAwNDldIHRoZXJtYWxfc3lz
OiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3Bvd2VyX2FsbG9jYXRvcicKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODAxMDRdIGNwdWlkbGU6IHVzaW5nIGdvdmVy
bm9yIGxhZGRlcgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDExMl0g
Y3B1aWRsZTogdXNpbmcgZ292ZXJub3IgbWVudQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU4MDIxMF0gRGV0ZWN0ZWQgMTUgUENDIFN1YnNwYWNlcwpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDIyNl0gUmVnaXN0ZXJpbmcgUENDIGRyaXZlciBh
cyBNYWlsYm94IGNvbnRyb2xsZXIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC41ODAyNDVdIGh3LWJyZWFrcG9pbnQ6IGZvdW5kIDYgYnJlYWtwb2ludCBhbmQgNCB3YXRjaHBv
aW50IHJlZ2lzdGVycy4KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODA2
MzldIEFTSUQgYWxsb2NhdG9yIGluaXRpYWxpc2VkIHdpdGggMzI3NjggZW50cmllcwpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDY2M10gQUNQSTogYnVzIHR5cGUgUENJ
IHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODA2NjRd
IGFjcGlwaHA6IEFDUEkgSG90IFBsdWcgUENJIENvbnRyb2xsZXIgRHJpdmVyIHZlcnNpb246IDAu
NQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDcwMV0gU2VyaWFsOiBB
TUJBIFBMMDExIFVBUlQgZHJpdmVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNTgzMDU3XSBIdWdlVExCIHJlZ2lzdGVyZWQgMS4wMCBHaUIgcGFnZSBzaXplLCBwcmUtYWxs
b2NhdGVkIDAgcGFnZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODMw
NTldIEh1Z2VUTEIgcmVnaXN0ZXJlZCAzMi4wIE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQg
MCBwYWdlcwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MzA2MF0gSHVn
ZVRMQiByZWdpc3RlcmVkIDIuMDAgTWlCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2Vz
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgzMDYwXSBIdWdlVExCIHJl
Z2lzdGVyZWQgNjQuMCBLaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTA2MDhdIEFDUEk6IEFkZGVkIF9PU0ko
TW9kdWxlIERldmljZSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTA2
MDldIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIERldmljZSkKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTA2MTBdIEFDUEk6IEFkZGVkIF9PU0koMy4wIF9TQ1AgRXh0
ZW5zaW9ucykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTA2MTFdIEFD
UEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIEFnZ3JlZ2F0b3IgRGV2aWNlKQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMDYxMl0gQUNQSTogQWRkZWQgX09TSShMaW51eC1E
ZWxsLVZpZGVvKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMDYxM10g
QUNQSTogQWRkZWQgX09TSShMaW51eC1MZW5vdm8tTlYtSERNSS1BdWRpbykKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTA2MTRdIEFDUEk6IEFkZGVkIF9PU0koTGludXgt
SFBJLUh5YnJpZC1HcmFwaGljcykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTc1MjhdIEFDUEk6IDIgQUNQSSBBTUwgdGFibGVzIHN1Y2Nlc3NmdWxseSBhY3F1aXJlZCBh
bmQgbG9hZGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODgxXSBB
Q1BJOiBJbnRlcnByZXRlciBlbmFibGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4ODgyXSBBQ1BJOiBVc2luZyBHSUMgZm9yIGludGVycnVwdCByb3V0aW5nClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODkwXSBBQ1BJOiBNQ0ZHIHRhYmxl
IGRldGVjdGVkLCA4IGVudHJpZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43Mjc2MzBdIEFSTUgwMDExOjAwOiB0dHlBTUEwIGF0IE1NSU8gMHgxMDAwMDI2MDAwMDAgKGly
cSA9IDM2LCBiYXNlX2JhdWQgPSAwKSBpcyBhIFNCU0EKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43Mjc2NzNdIHByaW50azogY29uc29sZSBbdHR5QU1BMF0gZW5hYmxlZApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODc4M10gQVJNSDAwMTE6MDE6
IHR0eUFNQTEgYXQgTU1JTyAweDEwMDAwMjYyMDAwMCAoaXJxID0gMzcsIGJhc2VfYmF1ZCA9IDAp
IGlzIGEgU0JTQQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg0MV0g
QUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kwXSAoZG9tYWluIDAwMGMgW2J1cyAwMC1mZl0pClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODQ3XSBhY3BpIFBOUDBBMDg6
MDA6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVu
dHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjk5MTRdIGFjcGkgUE5QMEEwODowMDogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBb
UENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43Mjk5NzNdIGFjcGkgUE5QMEEwODowMDogX09TQzogT1Mgbm93IGNvbnRy
b2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5OTc3XSBhY3BpIFBOUDBBMDg6MDA6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAw
eDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZm
ZDM4NzE1YTdlYTU4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwNjA0
XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXJlYSBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZm
ZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMwNjExXSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXQgW21lbSAweDMzZmZm
MDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwNjc3XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMGM6
MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA2NzldIHBjaV9idXMg
MDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2lu
ZG93XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDY4MF0gcGNpX2J1
cyAwMDBjOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzNmZmRm
ZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA2
ODFdIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA3MDBdIHBjaSAwMDBjOjAwOjAwLjA6
IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzA3NDhdIHBjaSAwMDBjOjAwOjAxLjA6IFsxZGVmOmUxMDFdIHR5
cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzA3NTZdIHBjaSAwMDBjOjAwOjAxLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA3NzFdIHBjaSAwMDBjOjAwOjAxLjA6
IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMw
NzcyXSBwY2kgMDAwYzowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwODQ5XSBhY3BpcGhwOiBTbG90
IFsxXSByZWdpc3RlcmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMw
ODYyXSBwY2kgMDAwYzowMTowMC4wOiBbMTAwMDowMDk3XSB0eXBlIDAwIGNsYXNzIDB4MDEwNzAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwODcxXSBwY2kgMDAwYzow
MTowMC4wOiByZWcgMHgxMDogW2lvICAweDRmZmY4MDAwLTB4NGZmZjgwZmZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwODc5XSBwY2kgMDAwYzowMTowMC4wOiByZWcg
MHgxNDogW21lbSAweDQwMTQwMDAwLTB4NDAxNGZmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwODg2XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxYzog
W21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMwODk2XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgzMDogW21lbSAw
eDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzA5MDBdIHBjaSAwMDBjOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA5OTRdIHBjaSAwMDBjOjAx
OjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzMxMDY2XSBwY2lfYnVzIDAwMGM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMDcwXSBwY2kgMDAwYzowMDowMS4wOiBicmlkZ2Ug
d2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFd
IGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMxMDczXSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVk
IFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMTA3NV0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVt
IDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMDc2XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczMTA3N10gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8g
YXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMxMDc4XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMTA3
OV0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMDgwXSBwY2kg
MDAwYzowMTowMC4wOiBCQVIgNjogYXNzaWduZWQgW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYg
cHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzEwODJdIHBjaSAw
MDBjOjAxOjAwLjA6IEJBUiAzOiBhc3NpZ25lZCBbbWVtIDB4NDAxMDAwMDAtMHg0MDEzZmZmZiA2
NGJpdF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzEwODddIHBjaSAw
MDBjOjAxOjAwLjA6IEJBUiAxOiBhc3NpZ25lZCBbbWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2
NGJpdF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzEwOTJdIHBjaSAw
MDBjOjAxOjAwLjA6IEJBUiAwOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDEwMF0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzEwOTNdIHBjaSAwMDBjOjAxOjAwLjA6
IEJBUiAwOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAxMDBdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMDk0XSBwY2kgMDAwYzowMDowMS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMx
MDk2XSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDQwMDAwMDAwLTB4
NDAxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMDk3XSBw
Y2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMDAwMDAwMDAwMC0weDMw
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMTA5OV0gcGNpX2J1cyAwMDBjOjAwOiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2VzIGFy
ZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGluZyB3aXRoIHBjaT1yZWFsbG9jClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMTAwXSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNl
IDQgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMTEwMV0gcGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA1IFtt
ZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMTEwMl0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAx
IFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMTEwM10gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzMDAw
MDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzEyMDldIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJMV0gKGRv
bWFpbiAwMDBkIFtidXMgMDAtZmZdKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMTIxM10gYWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRD
b25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMjc3XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IHBs
YXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxMzM2XSBhY3BpIFBOUDBB
MDg6MDE6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMTMzOF0gYWNwaSBQTlAwQTA4OjAxOiBN
Q0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZv
ciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmQzODcxNWE3ZWE1OApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMTk2MF0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGFyZWEgW21l
bSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMTk2N10gYWNwaSBQTlAwQTA4
OjAxOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVz
IDAwLWZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjAyNl0gUENJ
IGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBkOjAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMyMDI3XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0g
MHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzIwMjhdIHBjaV9idXMgMDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21l
bSAweDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMDI5XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJl
c291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMyMDQ2XSBwY2kgMDAwZDowMDowMC4wOiBbMWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYw
MDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMDg1XSBwY2kgMDAw
ZDowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMDkyXSBwY2kgMDAwZDowMDowMS4wOiBlbmFi
bGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMyMTA4XSBwY2kgMDAwZDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjEwOV0gcGNpIDAwMGQ6MDA6MDEuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMjE1OV0gcGNpIDAwMGQ6MDA6MDIuMDogWzFkZWY6ZTEwMl0gdHlwZSAwMSBjbGFzcyAw
eDA2MDQwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjE3OV0gcGNp
IDAwMGQ6MDA6MDIuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzIxODBdIHBjaSAwMDBkOjAwOjAyLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20g
RDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzIyMjdd
IHBjaSAwMDBkOjAwOjAzLjA6IFsxZGVmOmUxMDNdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzIyNDddIHBjaSAwMDBkOjAwOjAz
LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMyMjQ4XSBwY2kgMDAwZDowMDowMy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjk1XSBwY2kgMDAwZDow
MDowNC4wOiBbMWRlZjplMTA0XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMzE1XSBwY2kgMDAwZDowMDowNC4wOiBzdXBwb3J0
cyBEMSBEMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjMxNV0gcGNp
IDAwMGQ6MDA6MDQuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjM4M10gYWNwaXBocDogU2xvdCBbMS0yXSBy
ZWdpc3RlcmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyNDA1XSBw
Y2kgMDAwZDowMTowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyNDE3XSBwY2kgMDAwZDowMTowMC4w
OiByZWcgMHgxMDogW21lbSAweDUwMzEwMDAwLTB4NTAzMTNmZmYgNjRiaXRdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyNDQwXSBwY2kgMDAwZDowMTowMC4wOiByZWcg
MHgzMDogW21lbSAweDUwMzAwMDAwLTB4NTAzMGZmZmYgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzI0NDVdIHBjaSAwMDBkOjAxOjAwLjA6IGVuYWJsaW5nIEV4
dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI2MDBd
IGFjcGlwaHA6IFNsb3QgWzJdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzI2MTRdIHBjaSAwMDBkOjAyOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAg
Y2xhc3MgMHgwMTA4MDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI2
MjVdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2Zm
ZiA2NGJpdF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI2NDZdIHBj
aSAwMDBkOjAyOjAwLjA6IHJlZyAweDMwOiBbbWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVm
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjY1MF0gcGNpIDAwMGQ6
MDI6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczMjc4N10gYWNwaXBocDogU2xvdCBbM10gcmVnaXN0ZXJlZApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjgwMl0gcGNpIDAwMGQ6MDM6MDAuMDog
WzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczMjgxM10gcGNpIDAwMGQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0g
MHg1MDExMDAwMC0weDUwMTEzZmZmIDY0Yml0XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMjgzNF0gcGNpIDAwMGQ6MDM6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDEw
MDAwMC0weDUwMTBmZmZmIHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzMyODM4XSBwY2kgMDAwZDowMzowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTgwXSBhY3BpcGhwOiBTbG90IFs0
XSByZWdpc3RlcmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTk1
XSBwY2kgMDAwZDowNDowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMDA1XSBwY2kgMDAwZDowNDow
MC4wOiByZWcgMHgxMDogW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMDI2XSBwY2kgMDAwZDowNDowMC4wOiBy
ZWcgMHgzMDogW21lbSAweDUwMDAwMDAwLTB4NTAwMGZmZmYgcHJlZl0KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMwMzBdIHBjaSAwMDBkOjA0OjAwLjA6IGVuYWJsaW5n
IEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMx
NTNdIHBjaV9idXMgMDAwZDowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzMxNTVdIHBjaSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMTU3XSBwY2kgMDAwZDowMDowMS4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMg
MDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMzMTU4XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93
IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
MzE1OV0gcGNpIDAwMGQ6MDA6MDIuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZd
IHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzMxNjBdIHBjaSAwMDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAw
MTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAw
IGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzMxNjFdIHBjaSAwMDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4
MDAxZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMTYyXSBwY2kgMDAwZDowMDow
My4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3Np
emUgMTAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE2M10gcGNp
IDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2
NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE2NV0gcGNpIDAwMGQ6MDA6
MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAw
M10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzMxNjZdIHBjaSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cg
W2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMTY3XSBwY2kgMDAwZDowMDowNC4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMg
MDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMzMTY4XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93
IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
MzE3Ml0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAt
MHg1MDFmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxNzNd
IHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDAwMDAwMC0w
eDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMzE3NF0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4
NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzMxNzVdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAw
MDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczMzE3Nl0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25l
ZCBbbWVtIDB4NTA0MDAwMDAtMHg1MDVmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzMxNzddIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21l
bSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE3OF0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE0
OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzMxNzldIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNTogYXNz
aWduZWQgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE4MF0gcGNpIDAwMGQ6MDA6MDEu
MDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxODFdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczMzE4Ml0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzMxODNdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
MzE4NF0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxODRdIHBjaSAw
MDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE4NV0gcGNpIDAwMGQ6MDA6
MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxODZdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAx
MzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMzE4OF0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzMxODldIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczMzE5MF0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxOTFdIHBj
aSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE5Ml0gcGNpIDAwMGQ6
MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxOTJdIHBjaSAwMDBkOjAwOjAyLjA6IEJB
UiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzE5NF0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBu
byBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzMxOTRdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFz
c2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMzE5Nl0gcGNpIDAwMGQ6MDE6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDAw
MDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzMzMTk3XSBwY2kgMDAwZDowMTowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMDEw
MDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzMzMjAyXSBwY2kgMDAwZDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjAzXSBwY2kgMDAwZDowMDowMS4w
OiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjA1XSBwY2kgMDAwZDowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzIwNl0gcGNpIDAwMGQ6
MDI6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjA3XSBwY2kgMDAwZDow
MjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjEyXSBwY2kgMDAwZDow
MDowMi4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMzMjEzXSBwY2kgMDAwZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMzMjE0XSBwY2kgMDAwZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0
MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMzIxNl0gcGNpIDAwMGQ6MDM6MDAuMDogQkFSIDY6IGFzc2ln
bmVkIFttZW0gMHg1MDQwMDAwMC0weDUwNDBmZmZmIHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMzMjE3XSBwY2kgMDAwZDowMzowMC4wOiBCQVIgMDogYXNzaWdu
ZWQgW21lbSAweDUwNDEwMDAwLTB4NTA0MTNmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMzMjIyXSBwY2kgMDAwZDowMDowMy4wOiBQQ0kgYnJpZGdlIHRv
IFtidXMgMDNdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjIzXSBw
Y2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNDAwMDAwLTB4NTA1ZmZm
ZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjI0XSBwY2kgMDAw
ZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
MzIyNl0gcGNpIDAwMGQ6MDQ6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDYwMDAwMC0w
eDUwNjBmZmZmIHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMz
MjI3XSBwY2kgMDAwZDowNDowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNjEwMDAwLTB4
NTA2MTNmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMz
MjMxXSBwY2kgMDAwZDowMDowNC4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDRdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjMzXSBwY2kgMDAwZDowMDowNC4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDUwNjAwMDAwLTB4NTA3ZmZmZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjM0XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzIzNV0gcGNpX2J1cyAwMDBkOjAw
OiByZXNvdXJjZSA0IFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMyMzZdIHBjaV9idXMgMDAwZDowMDogcmVz
b3VyY2UgNSBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMyMzddIHBjaV9idXMgMDAwZDowMTog
cmVzb3VyY2UgMSBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzMyMzhdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMiBb
bWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMjM5XSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291
cmNlIDEgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMzMjQwXSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDIgW21lbSAw
eDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzI0MV0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAx
IFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMzI0MV0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAyIFttZW0gMHgzNDAw
MDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzMyNDJdIHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2UgMSBbbWVt
IDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzMyNDNdIHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNjAw
MDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMzMzUyXSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTJdIChkb21haW4g
MDAwMSBbYnVzIDAwLWZmXSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzMzNTZdIGFjcGkgUE5QMEEwODowMjogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmln
IEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMzQyMF0gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBwbGF0Zm9y
bSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzQ3OF0gYWNwaSBQTlAwQTA4OjAy
OiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM0ODFdIGFjcGkgUE5QMEEwODowMjogTUNGRyBx
dWlyazogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1
cyAwMC1mZl0gd2l0aCAweGZmZmZkMzg3MTVhN2VhNTgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzQxMDddIGFjcGkgUE5QMEEwODowMjogRUNBTSBhcmVhIFttZW0gMHgz
YmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQxMTNdIGFjcGkgUE5QMEEwODowMjog
RUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1m
Zl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQxNzFdIFBDSSBob3N0
IGJyaWRnZSB0byBidXMgMDAwMTowMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNDE3Ml0gcGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NjAw
MDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzM0MTczXSBwY2lfYnVzIDAwMDE6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgz
ODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNDE3NF0gcGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDE5
Ml0gcGNpIDAwMDE6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDIzNF0gcGNpIDAwMDE6MDA6
MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDI0N10gcGNpIDAwMDE6MDA6MDEuMDogZW5hYmxpbmcg
RXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDI3
M10gcGNpIDAwMDE6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQyNzRdIHBjaSAwMDAxOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVk
IGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzQzNjFdIGFjcGlwaHA6IFNsb3QgWzEtM10gcmVnaXN0ZXJlZApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjQ1MV0gcGNpX2J1cyAwMDAxOjAwOiBvbiBOVU1BIG5vZGUg
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjQ1M10gcGNpIDAwMDE6
MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDEwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY0NTRd
IHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZm
ZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAw
MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY0NTZdIHBjaSAwMDAx
OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFti
dXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDU4XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTQ6IGFz
c2lnbmVkIFttZW0gMHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNjQ1OV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25l
ZCBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDYwXSBwY2kgMDAwMTowMDowMS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjQ2MV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM2NDYyXSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNjQ2M10gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDYz
XSBwY2kgMDAwMTowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDY1XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDYwMDAwMDAwLTB4NjAxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM2NDY3XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDM4MDAwMDAwMDAwMC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjQ2OV0gcGNpX2J1cyAwMDAxOjAwOiBy
ZXNvdXJjZSA0IFttZW0gMHg2MDAwMDAwMC0weDZmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY0NzBdIHBjaV9idXMgMDAwMTowMDogcmVzb3Vy
Y2UgNSBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY0NzFdIHBjaV9idXMgMDAwMTowMTogcmVz
b3VyY2UgMSBbbWVtIDB4NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzY0NzFdIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMiBbbWVt
IDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NTc3XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BD
STNdIChkb21haW4gMDAwMCBbYnVzIDAwLWZmXSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzY1ODFdIGFjcGkgUE5QMEEwODowMzogX09TQzogT1Mgc3VwcG9ydHMgW0V4
dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjY0NV0gYWNwaSBQTlAwQTA4OjAzOiBf
T1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQ
TUUgTFRSXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjcwNF0gYWNw
aSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY3MDZdIGFjcGkgUE5QMEEw
ODowMzogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZm
ZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZkMzg3MTVhN2VhNTgKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzczMjhdIGFjcGkgUE5QMEEwODowMzogRUNBTSBh
cmVhIFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBD
MDI6MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzczMzVdIGFjcGkg
UE5QMEEwODowMzogRUNBTSBhdCBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBm
b3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzcz
OTRdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMDowMApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNzM5NV0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM3Mzk2XSBwY2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291
cmNlIFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzZmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzM5N10gcGNpX2J1cyAwMDAwOjAwOiByb290
IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNzQxNF0gcGNpIDAwMDA6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFz
cyAweDA2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzQ1Nl0g
cGNpIDAwMDA6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzQ2OV0gcGNpIDAwMDA6MDA6MDEu
MDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNzQ5NV0gcGNpIDAwMDA6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc0OTZdIHBjaSAwMDAwOjAwOjAxLjA6IFBN
RSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mzc1ODJdIGFjcGlwaHA6IFNsb3QgWzEtNF0gcmVnaXN0ZXJlZApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY3M10gcGNpX2J1cyAwMDAwOjAwOiBv
biBOVU1BIG5vZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY3
NF0gcGNpIDAwMDA6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRv
IFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mzk2NzVdIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAw
MDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFk
ZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2
NzZdIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5Njc4XSBwY2kgMDAwMDowMDowMS4w
OiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY3OV0gcGNpIDAwMDA6MDA6MDEuMDogQkFS
IDE1OiBhc3NpZ25lZCBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NjgwXSBwY2kgMDAw
MDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY4MV0gcGNpIDAwMDA6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NjgyXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczOTY4M10gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8g
YXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzM5Njg0XSBwY2kgMDAwMDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5Njg2XSBwY2kgMDAwMDowMDow
MS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5Njg3XSBwY2kgMDAwMDowMDowMS4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY4OV0gcGNpX2J1
cyAwMDAwOjAwOiByZXNvdXJjZSA0IFttZW0gMHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2OTBdIHBjaV9idXMgMDAw
MDowMDogcmVzb3VyY2UgNSBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRv
d10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2OTFdIHBjaV9idXMg
MDAwMDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2OTJdIHBjaV9idXMgMDAwMDowMTogcmVz
b3VyY2UgMiBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5Nzk1XSBBQ1BJOiBQQ0kgUm9v
dCBCcmlkZ2UgW1BDSTRdIChkb21haW4gMDAwMiBbYnVzIDAwLWZmXSkKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk3OTldIGFjcGkgUE5QMEEwODowNDogX09TQzogT1Mg
c3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5
cGUzXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTg2Ml0gYWNwaSBQ
TlAwQTA4OjA0OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBT
SFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczOTkyMF0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0ll
Q2FwYWJpbGl0eV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk5MjJd
IGFjcGkgUE5QMEEwODowNDogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAw
LTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZkMzg3MTVhN2VhNTgK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDA1NDhdIGFjcGkgUE5QMEEw
ODowNDogRUNBTSBhcmVhIFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmZdIHJlc2Vy
dmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDA1NTNdIGFjcGkgUE5QMEEwODowNDogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNm
ZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDA2MTJdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMjowMApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDYxNF0gcGNpX2J1cyAwMDAyOjAwOiByb290
IGJ1cyByZXNvdXJjZSBbbWVtIDB4MDA4MDAwMDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwNjE0XSBwY2lfYnVzIDAwMDI6MDA6IHJv
b3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDYxNV0gcGNpX2J1cyAw
MDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0MDYzMl0gcGNpIDAwMDI6MDA6MDAuMDogWzFkZWY6ZTExMF0g
dHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0MDY3NF0gcGNpIDAwMDI6MDA6MDEuMDogWzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAw
eDA2MDQwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDcwOV0gcGNp
IDAwMDI6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDA3MTBdIHBjaSAwMDAyOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20g
RDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDA3NzFd
IHBjaSAwMDAyOjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDA4MDZdIHBjaSAwMDAyOjAwOjA1
LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQwODA3XSBwY2kgMDAwMjowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwODg5XSBhY3BpcGhwOiBT
bG90IFsxLTVdIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDMwMDFdIGFjcGlwaHA6IFNsb3QgWzMtMl0gcmVnaXN0ZXJlZApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTA5M10gcGNpX2J1cyAwMDAyOjAwOiBvbiBOVU1BIG5v
ZGUgMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTA5NV0gcGNpIDAw
MDI6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFd
IGFkZF9zaXplIDEwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUw
OTZdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUwOTddIHBjaSAw
MDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRv
IFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MDk4XSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ug
d2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTA5OV0gcGNpIDAwMDI6MDA6MDUu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0
byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTEwMF0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDUxMDJdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAw
ODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ1MTAzXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAw
MDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDUxMDRdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1MTA1XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMDZdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ1MTA3XSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDUxMDhdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MTA5
XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMTFdIHBjaSAw
MDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MTEyXSBwY2kgMDAwMjowMDowNS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMTJdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ1MTEzXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDUxMTRdIHBjaSAwMDAyOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMTZdIHBjaSAwMDAyOjAw
OjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMTddIHBjaSAwMDAyOjAwOjAxLjA6
ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MTIwXSBwY2kg
MDAwMjowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ1MTIxXSBwY2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ1MTIzXSBwY2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTEyNV0gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJj
ZSA0IFttZW0gMHgwMDgwMDAwMC0weDBmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMjZdIHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNSBb
bWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUxMjddIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2Ug
MSBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDUxMjddIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjAw
MDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ1MTI4XSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDEgW21l
bSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ1MTI5XSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDIw
MDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0NTIzNF0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0k1XSAoZG9tYWlu
IDAwMDMgW2J1cyAwMC1mZl0pClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ1MjM5XSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZp
ZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDUzMDJdIGFjcGkgUE5QMEEwODowNTogX09TQzogcGxhdGZv
cm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDUzNjBdIGFjcGkgUE5QMEEwODow
NTogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MzYyXSBhY3BpIFBOUDBBMDg6MDU6IE1DRkcg
cXVpcms6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFti
dXMgMDAtZmZdIHdpdGggMHhmZmZmZDM4NzE1YTdlYTU4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ1OTg2XSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXJlYSBbbWVtIDB4
MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1OTkyXSBhY3BpIFBOUDBBMDg6MDU6
IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAt
ZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2MDQ0XSBQQ0kgaG9z
dCBicmlkZ2UgdG8gYnVzIDAwMDM6MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDYwNDZdIHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDEw
MDAwMDAwLTB4MWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc0NjA0Nl0gcGNpX2J1cyAwMDAzOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4
MjQwMDAwMDAwMDAwLTB4MjdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDYwNDddIHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3Vy
Y2UgW2J1cyAwMC1mZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDYw
NjRdIHBjaSAwMDAzOjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDYxMDRdIHBjaSAwMDAzOjAw
OjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDYxNDBdIHBjaSAwMDAzOjAwOjAxLjA6IHN1cHBvcnRz
IEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2MTQwXSBwY2kg
MDAwMzowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2MTk3XSBwY2kgMDAwMzowMDowNS4wOiBbMWRl
ZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ2MjI0XSBwY2kgMDAwMzowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NjIyNF0gcGNpIDAwMDM6MDA6MDUu
MDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0NjI5MF0gYWNwaXBocDogU2xvdCBbMS02XSByZWdpc3RlcmVkClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NDEzXSBwY2kgMDAwMzowMjow
MC4wOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNsYXNzIDB4MDIwMDAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NDI4XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxMDog
W21lbSAweDEwMDIwMDAwLTB4MTAwM2ZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ4NDQ0XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxODogW2lvICAweDFmZmY4
MDIwLTB4MWZmZjgwM2ZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4
NDUzXSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxYzogW21lbSAweDEwMDQ0MDAwLTB4MTAwNDdm
ZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NTY0XSBwY2kgMDAw
MzowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODYwMV0gcGNpIDAwMDM6MDI6MDAuMDogcmVn
IDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODYwMl0gcGNpIDAwMDM6MDI6MDAuMDogVkYo
bikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNv
bnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0ODYyM10gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTkwOiBbbWVtIDB4MDAwMDAwMDAt
MHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0ODYyNF0gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMyBzcGFjZTogW21lbSAweDAw
MDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjMgZm9yIDggVkZzKQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODcyMV0gcGNpIDAwMDM6MDI6
MDAuMDogOC4wMDAgR2IvcyBhdmFpbGFibGUgUENJZSBiYW5kd2lkdGgsIGxpbWl0ZWQgYnkgNS4w
IEdUL3MgUENJZSB4MiBsaW5rIGF0IDAwMDM6MDA6MDUuMCAoY2FwYWJsZSBvZiAxNi4wMDAgR2Iv
cyB3aXRoIDUuMCBHVC9zIFBDSWUgeDQgbGluaykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDg3NjddIHBjaSAwMDAzOjAyOjAwLjE6IFs4MDg2OjE1MjFdIHR5cGUgMDAg
Y2xhc3MgMHgwMjAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3
ODJdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDEwOiBbbWVtIDB4MTAwMDAwMDAtMHgxMDAxZmZm
Zl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3OTldIHBjaSAwMDAz
OjAyOjAwLjE6IHJlZyAweDE4OiBbaW8gIDB4MWZmZjgwMDAtMHgxZmZmODAxZl0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg4MDddIHBjaSAwMDAzOjAyOjAwLjE6IHJl
ZyAweDFjOiBbbWVtIDB4MTAwNDAwMDAtMHgxMDA0M2ZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDg5MTRdIHBjaSAwMDAzOjAyOjAwLjE6IFBNRSMgc3VwcG9ydGVk
IGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ4OTQ3XSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxODQ6IFttZW0gMHgwMDAwMDAwMC0w
eDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ4OTQ3XSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIwIHNwYWNlOiBbbWVtIDB4MDAw
MDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMCBmb3IgOCBWRnMpClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4OTY4XSBwY2kgMDAwMzowMjow
MC4xOiByZWcgMHgxOTA6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4OTY5XSBwY2kgMDAwMzowMjow
MC4xOiBWRihuKSBCQVIzIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBw
cmVmXSAoY29udGFpbnMgQkFSMyBmb3IgOCBWRnMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ5MDk0XSBwY2lfYnVzIDAwMDM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MDk2XSBwY2lfYnVzIDAwMDM6MDA6
IG1heCBidXMgZGVwdGg6IDEgcGNpX3RyeV9udW06IDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDkwOTldIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ5MTAwXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxMDFdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNDog
YXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ5MTAyXSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTU6IGFzc2ln
bmVkIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxMDNdIHBjaSAwMDAzOjAwOjAxLjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTA0XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDkxMDVdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2Ug
Zm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ5MTA2XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkx
MDddIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxMDhdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDkxMTBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTEzXSBwY2kgMDAwMzowMjowMC4w
OiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4MTAyMWZmZmZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTE2XSBwY2kgMDAwMzowMjowMC4xOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDEwMjIwMDAwLTB4MTAyM2ZmZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTE5XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMzogYXNz
aWduZWQgW21lbSAweDEwMjQwMDAwLTB4MTAyNDNmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ5MTIyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgNzogYXNzaWduZWQg
W21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDIxZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTEyN10gcGNpIDAwMDM6MDI6MDAuMDogQkFS
IDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjIwMDAwLTB4MjQwMDAwMjNmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTMxXSBwY2kgMDAw
MzowMjowMC4xOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQ0MDAwLTB4MTAyNDdmZmZdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTM0XSBwY2kgMDAwMzowMjow
MC4xOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDI0MDAwMC0weDI0MDAwMDI1ZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTEzOF0g
cGNpIDAwMDM6MDI6MDAuMTogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjYwMDAwLTB4
MjQwMDAwMjdmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ5MTQyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDAwMjBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTQz
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgw
MDIwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTE0NF0gcGNpIDAw
MDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTE0NV0gcGNpIDAwMDM6MDI6MDAuMTog
QkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNDZdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlk
Z2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkx
NDddIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgx
MDNmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNDldIHBj
aSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQw
MDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ5MTUwXSBwY2lfYnVzIDAwMDM6MDA6IE5vLiAyIHRyeSB0byBhc3NpZ24gdW5hc3NpZ25l
ZCByZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNTJdIHBjaSAw
MDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTUzXSBwY2kgMDAwMzowMDowMS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNTRdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ5MTU1XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDkxNTZdIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNTddIHBjaSAwMDAzOjAw
OjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNTldIHBjaSAwMDAzOjAwOjAxLjA6
ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTYxXSBwY2kg
MDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTYyXSBwY2kgMDAwMzowMjowMC4w
OiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTE2Ml0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0OTE2M10gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDkxNjRdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNjVdIHBjaSAwMDAzOjAwOjA1
LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNjddIHBjaSAwMDAzOjAwOjA1LjA6ICAg
YnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTY4XSBwY2lfYnVz
IDAwMDM6MDA6IEF1dG9tYXRpY2FsbHkgZW5hYmxlZCBwY2kgcmVhbGxvYywgaWYgeW91IGhhdmUg
cHJvYmxlbSwgdHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYz1vZmYKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxNjldIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2Ug
NCBbbWVtIDB4MTAwMDAwMDAtMHgxZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ5MTcwXSBwY2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDUgW21l
bSAweDI0MDAwMDAwMDAwMC0weDI3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MTcxXSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDEg
W21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ5MTcyXSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDIgW21lbSAweDI0MDAw
MDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0OTE3M10gcGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAxIFttZW0g
MHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0OTE3M10gcGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAyMDAw
MDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDkyNDBdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJNl0gKGRvbWFpbiAw
MDA0IFtidXMgMDAtZmZdKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
OTI0NF0gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcg
QVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ5MzA5XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IHBsYXRmb3Jt
IGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MzY3XSBhY3BpIFBOUDBBMDg6MDY6
IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTM2OV0gYWNwaSBQTlAwQTA4OjA2OiBNQ0ZHIHF1
aXJrOiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVz
IDAwLWZmXSB3aXRoIDB4ZmZmZmQzODcxNWE3ZWE1OApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0OTk5Ml0gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGFyZWEgW21lbSAweDJi
ZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTk5OF0gYWNwaSBQTlAwQTA4OjA2OiBF
Q0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZm
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MDA0M10gUENJIGhvc3Qg
YnJpZGdlIHRvIGJ1cyAwMDA0OjAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUwMDQ0XSBwY2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAw
MDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTAwNDVdIHBjaV9idXMgMDAwNDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI4
MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzUwMDQ2XSBwY2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNl
IFtidXMgMDAtZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwMDYy
XSBwY2kgMDAwNDowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwMDk5XSBwY2kgMDAwNDowMDow
MS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwMTI2XSBwY2kgMDAwNDowMDowMS4wOiBzdXBwb3J0cyBE
MSBEMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MDEyNl0gcGNpIDAw
MDQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MDE2OV0gcGNpIDAwMDQ6MDA6MDMuMDogWzFkZWY6
ZTExM10gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1MDE5NV0gcGNpIDAwMDQ6MDA6MDMuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTAxOTZdIHBjaSAwMDA0OjAwOjAzLjA6
IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTAyNDFdIHBjaSAwMDA0OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUg
MDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTAyNzZdIHBjaSAwMDA0OjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwMjc3XSBwY2kgMDAwNDowMDowNS4wOiBQTUUjIHN1cHBv
cnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUwMzQ5XSBwY2kgMDAwNDowMTowMC4wOiBbMWEwMzoxMTUwXSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwMzk3XSBwY2kg
MDAwNDowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzUwNDUzXSBwY2kgMDAwNDowMTowMC4wOiBzdXBwb3J0cyBEMSBE
MgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MDQ1M10gcGNpIDAwMDQ6
MDE6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTA1NDhdIHBjaV9idXMgMDAwNDowMjog
ZXh0ZW5kZWQgY29uZmlnIHNwYWNlIG5vdCBhY2Nlc3NpYmxlClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzUwNTY1XSBwY2kgMDAwNDowMjowMC4wOiBbMWEwMzoyMDAwXSB0
eXBlIDAwIGNsYXNzIDB4MDMwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUwNTgzXSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDIwMDAwMDAwLTB4
MjFmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwNTk0XSBw
Y2kgMDAwNDowMjowMC4wOiByZWcgMHgxNDogW21lbSAweDIyMDAwMDAwLTB4MjIwMWZmZmZdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwNjA0XSBwY2kgMDAwNDowMjow
MC4wOiByZWcgMHgxODogW2lvICAweDJmZmY4MDAwLTB4MmZmZjgwN2ZdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwNjQ4XSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDog
YXNzaWduZWQgdG8gZWZpZmIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTA2ODRdIHBjaSAwMDA0OjAyOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUwNjg1XSBwY2kgMDAwNDowMjowMC4wOiBQTUUjIHN1cHBv
cnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90IEQzY29sZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1MDc5NV0gcGNpIDAwMDQ6MDM6MDAuMDogWzE5MTI6MDAxNF0gdHlwZSAw
MCBjbGFzcyAweDBjMDMzMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
MDgxN10gcGNpIDAwMDQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgyMjIwMDAwMC0weDIyMjAx
ZmZmIDY0Yml0XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MDkyNV0g
cGNpIDAwMDQ6MDM6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwNTldIHBjaV9idXMgMDAwNDow
MDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTIwNjJdIHBjaSAwMDA0OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4
MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMS0wMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwNjNd
IHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBb
YnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUyMDY0XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAw
MC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDY1
XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZm
ZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA2Nl0gcGNpIDAwMDQ6MDA6MDUuMDog
YnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEw
MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwNjhdIHBjaSAwMDA0
OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQg
cHJlZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwNjldIHBjaSAwMDA0OjAwOjA1LjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDRdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzUyMDcyXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFtt
ZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MjA3M10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4
MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDc0XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTQ6IGFz
c2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIzMWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc1MjA3NV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25l
ZCBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4MjgwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDc2XSBwY2kgMDAwNDowMDowNS4wOiBC
QVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA3N10gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE1
OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAwNWZmZmZmIDY0Yml0IHByZWZd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDc4XSBwY2kgMDAwNDow
MDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA3OV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFS
IDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDgwXSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5v
IHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1MjA4MV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNz
aWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUyMDgyXSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA4Ml0g
cGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDg0XSBwY2kgMDAw
NDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA4NV0gcGNpIDAwMDQ6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDg2XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1MjA4N10gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8g
YXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzUyMDg4XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA4
OV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMDkwXSBwY2kg
MDAwNDowMTowMC4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZm
XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA5MV0gcGNpIDAwMDQ6
MDE6MDAuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwOTFdIHBjaSAwMDA0OjAxOjAwLjA6IEJB
UiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA5M10gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFz
c2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIxZmZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc1MjA5Nl0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDE6IGFzc2lnbmVk
IFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1MjEwMF0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBb
aW8gIHNpemUgMHgwMDgwXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
MjEwMV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MDA4MF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMDJdIHBj
aSAwMDA0OjAxOjAwLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMDZdIHBjaSAwMDA0OjAxOjAwLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTIxMTNdIHBjaSAwMDA0OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1
cyAwMS0wMl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMTRdIHBj
aSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZm
Zl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMTVdIHBjaSAwMDA0
OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUy
MTE3XSBwY2kgMDAwNDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDIzMDAwMDAwLTB4
MjMwMDFmZmYgNjRiaXRdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUy
MTI3XSBwY2kgMDAwNDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTI4XSBwY2kgMDAwNDowMDowMy4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTI5XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjEzMV0gcGNpIDAwMDQ6MDA6MDUu
MDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MjEzM10gcGNpIDAwMDQ6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgy
MzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc1MjEzNF0gcGNpIDAwMDQ6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDA0
MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTIxMzZdIHBjaV9idXMgMDAwNDowMDogU29tZSBQQ0kgZGV2aWNlIHJl
c291cmNlcyBhcmUgdW5hc3NpZ25lZCwgdHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjEzN10gcGNpX2J1cyAwMDA0OjAw
OiByZXNvdXJjZSA0IFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMzhdIHBjaV9idXMgMDAwNDowMDogcmVz
b3VyY2UgNSBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMzldIHBjaV9idXMgMDAwNDowMTog
cmVzb3VyY2UgMSBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTIxNDBdIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMiBb
bWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTQxXSBwY2lfYnVzIDAwMDQ6MDI6IHJlc291
cmNlIDEgW21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzUyMTQxXSBwY2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDEgW21lbSAw
eDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUyMTQyXSBwY2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDIwMDAw
MC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1MjE0M10gcGNpX2J1cyAwMDA0OjA0OiByZXNvdXJjZSAxIFttZW0gMHgyMzIw
MDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
MjE0NF0gcGNpX2J1cyAwMDA0OjA0OiByZXNvdXJjZSAyIFttZW0gMHgyODAwMDA0MDAwMDAtMHgy
ODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTIxNzFdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJN10gKGRvbWFpbiAwMDA1IFti
dXMgMDAtZmZdKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE3NV0g
YWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBD
bG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzUyMjM5XSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IHBsYXRmb3JtIGRvZXMg
bm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMjk3XSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6
IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1MjI5OV0gYWNwaSBQTlAwQTA4OjA3OiBNQ0ZHIHF1aXJrOiBF
Q0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZm
XSB3aXRoIDB4ZmZmZmQzODcxNWE3ZWE1OApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MjkyM10gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGFyZWEgW21lbSAweDJmZmZmMDAw
MDAwMC0weDJmZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjkzMF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGF0
IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1Mjk4Nl0gUENJIGhvc3QgYnJpZGdl
IHRvIGJ1cyAwMDA1OjAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUy
OTg3XSBwY2lfYnVzIDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMC0w
eDNmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTI5ODhdIHBjaV9idXMgMDAwNTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDJjMDAwMDAw
MDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzUyOTg5XSBwY2lfYnVzIDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMg
MDAtZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzMDA3XSBwY2kg
MDAwNTowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzMDQ3XSBwY2kgMDAwNTowMDowMS4wOiBb
MWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzUzMDg1XSBwY2kgMDAwNTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzA4Nl0gcGNpIDAwMDU6MDA6
MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1MzE0N10gcGNpIDAwMDU6MDA6MDUuMDogWzFkZWY6ZTExNV0g
dHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc1MzE4M10gcGNpIDAwMDU6MDA6MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMxODRdIHBjaSAwMDA1OjAwOjA1LjA6IFBNRSMg
c3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTMyMzZdIHBjaSAwMDA1OjAwOjA3LjA6IFsxZGVmOmUxMTddIHR5cGUgMDEgY2xh
c3MgMHgwNjA0MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMyNzFd
IHBjaSAwMDA1OjAwOjA3LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzUzMjcyXSBwY2kgMDAwNTowMDowNy4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUz
MzQ4XSBhY3BpcGhwOiBTbG90IFsxLTddIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NTc1NTVdIHBjaV9idXMgMDAwNTowMDogb24gTlVNQSBub2RlIDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NTddIHBjaSAwMDA1OjAw
OjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRf
c2l6ZSAxMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTU4XSBw
Y2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZm
IDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTU5XSBwY2kgMDAwNTow
MDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVz
IDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1NzU2MF0gcGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRv
dyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NjFdIHBjaSAwMDA1OjAwOjA1LjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1
cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NjJdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAw
MCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU3NTYzXSBwY2kgMDAwNTowMDowNy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZm
Zl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1NzU2NF0gcGNpIDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAw
MDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc1NzU2NV0gcGNpIDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAt
MHgwMDBmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NjhdIHBjaSAwMDA1OjAw
OjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTY5XSBwY2kgMDAwNTowMDowMS4w
OiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NzBdIHBj
aSAwMDA1OjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZm
ZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTcxXSBwY2kgMDAw
NTowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTc1NzJdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwNDAwMDAw
LTB4MzA1ZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTcz
XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDA0MDAwMDAt
MHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTc1NzRdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3
NTc1XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NzZdIHBj
aSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTc3XSBwY2kgMDAwNTowMDow
NS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1NzhdIHBjaSAwMDA1OjAwOjA3LjA6IEJB
UiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzU3NTc5XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTc1ODFdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9y
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU3NTgxXSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1ODJd
IHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTgzXSBwY2kgMDAwNTow
MDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1ODRdIHBjaSAwMDA1OjAwOjAxLjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTg1XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTc1ODZdIHBjaSAwMDA1OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1
cyAwMV0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1ODhdIHBjaSAw
MDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0K
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc1ODldIHBjaSAwMDA1OjAw
OjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTkx
XSBwY2kgMDAwNTowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NTkzXSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzU3NTk0XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDJjMDAwMDIwMDAwMC0weDJjMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NzU5Nl0gcGNpIDAwMDU6MDA6MDcuMDog
UENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc1NzU5OF0gcGNpIDAwMDU6MDA6MDcuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDQw
MDAwMC0weDMwNWZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
NzU5OV0gcGNpIDAwMDU6MDA6MDcuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDA0MDAw
MDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTc2MDFdIHBjaV9idXMgMDAwNTowMDogcmVzb3VyY2UgNCBbbWVtIDB4MzAw
MDAwMDAtMHgzZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzU3NjAyXSBwY2lfYnVzIDAwMDU6MDA6IHJlc291cmNlIDUgW21lbSAweDJjMDAwMDAw
MDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzU3NjAzXSBwY2lfYnVzIDAwMDU6MDE6IHJlc291cmNlIDEgW21lbSAweDMwMDAw
MDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3
NjA0XSBwY2lfYnVzIDAwMDU6MDE6IHJlc291cmNlIDIgW21lbSAweDJjMDAwMDAwMDAwMC0weDJj
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc1NzYwNV0gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAxIFttZW0gMHgzMDIwMDAwMC0w
eDMwM2ZmZmZmXQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NzYwNl0g
cGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAyIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTc2MDZdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMSBbbWVtIDB4MzA0MDAwMDAtMHgzMDVm
ZmZmZl0KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTc2MDddIHBjaV9i
dXMgMDAwNTowMzogcmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwMDg0
XSBpb21tdTogRGVmYXVsdCBkb21haW4gdHlwZTogVHJhbnNsYXRlZCAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjAxMThdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjog
VkdBIGRldmljZSBhZGRlZDogZGVjb2Rlcz1pbyttZW0sb3ducz1ub25lLGxvY2tzPW5vbmUKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjAxMTldIHBjaSAwMDA0OjAyOjAw
LjA6IHZnYWFyYjogYnJpZGdlIGNvbnRyb2wgcG9zc2libGUKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NjAxMjJdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogc2V0dGlu
ZyBhcyBib290IGRldmljZSAoVkdBIGxlZ2FjeSByZXNvdXJjZXMgbm90IGF2YWlsYWJsZSkKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjAxMjJdIHZnYWFyYjogbG9hZGVk
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwMjA0XSBFREFDIE1DOiBW
ZXI6IDMuMC4wClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwMzM2XSBS
ZWdpc3RlcmVkIGVmaXZhcnMgb3BlcmF0aW9ucwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc2MDk5OV0gTmV0TGFiZWw6IEluaXRpYWxpemluZwpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MTAwMF0gTmV0TGFiZWw6ICBkb21haW4gaGFzaCBzaXpl
ID0gMTI4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYxMDAwXSBOZXRM
YWJlbDogIHByb3RvY29scyA9IFVOTEFCRUxFRCBDSVBTT3Y0IENBTElQU08KU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjEwMTBdIE5ldExhYmVsOiAgdW5sYWJlbGVkIHRy
YWZmaWMgYWxsb3dlZCBieSBkZWZhdWx0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzYxMTQxXSBjbG9ja3NvdXJjZTogU3dpdGNoZWQgdG8gY2xvY2tzb3VyY2UgYXJjaF9z
eXNfY291bnRlcgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2OTcwM10g
VkZTOiBEaXNrIHF1b3RhcyBkcXVvdF82LjYuMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc2OTcyMF0gVkZTOiBEcXVvdC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDUx
MiAob3JkZXIgMCwgNDA5NiBieXRlcykKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Njk4MzVdIEFwcEFybW9yOiBBcHBBcm1vciBGaWxlc3lzdGVtIEVuYWJsZWQKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Njk4NjldIHBucDogUG5QIEFDUEkgaW5p
dApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc3MDQ5M10gc3lzdGVtIDAw
OjAwOiBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90
IGJlIHJlc2VydmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcwNDk1
XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmYgd2luZG93
XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NzA0OTZdIHN5c3RlbSAwMDowMDogW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZm
ZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc3MDQ5N10gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MjdmZmYwMDAwMDAw
LTB4MjdmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcwNDk4XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgy
YmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzA0OTldIHN5c3RlbSAwMDow
MDogW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBi
ZSByZXNlcnZlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc3MDUwMF0g
c3lzdGVtIDAwOjAwOiBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmIHdpbmRvd10g
Y291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzcwNTAxXSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZm
ZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NzA1MDVdIHN5c3RlbSAwMDowMDogUGx1ZyBhbmQgUGxheSBBQ1BJIGRl
dmljZSwgSURzIFBOUDBjMDIgKGFjdGl2ZSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NzA3OTVdIHBucDogUG5QIEFDUEk6IGZvdW5kIDEgZGV2aWNlcwpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAwLjc3MTc0NF0gTkVUOiBSZWdpc3RlcmVkIHByb3Rv
Y29sIGZhbWlseSAyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcxODA5
XSBJUCBpZGVudHMgaGFzaCB0YWJsZSBlbnRyaWVzOiAyNjIxNDQgKG9yZGVyOiA5LCAyMDk3MTUy
IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcz
NDAzXSB0Y3BfbGlzdGVuX3BvcnRhZGRyX2hhc2ggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAo
b3JkZXI6IDcsIDUyNDI4OCBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc3MzY5MV0gVENQIGVzdGFibGlzaGVkIGhhc2ggdGFibGUgZW50cmllczog
NTI0Mjg4IChvcmRlcjogMTAsIDQxOTQzMDQgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzQ3NzJdIFRDUCBiaW5kIGhhc2ggdGFibGUgZW50cmll
czogNjU1MzYgKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzc1MDMwXSBUQ1A6IEhhc2ggdGFibGVzIGNvbmZpZ3Vy
ZWQgKGVzdGFibGlzaGVkIDUyNDI4OCBiaW5kIDY1NTM2KQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc3NTExNV0gVURQIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9y
ZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzc1NTE5XSBVRFAtTGl0ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChv
cmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc3NjI5MF0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzc2Mjk2XSBORVQ6IFJlZ2lzdGVy
ZWQgcHJvdG9jb2wgZmFtaWx5IDQ0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzc2MzA0XSBwY2kgMDAwYzowMTowMC4wOiBDTFMgbWlzbWF0Y2ggKDY0ICE9IDMyKSwgdXNp
bmcgNjQgYnl0ZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzYzODBd
IFRyeWluZyB0byB1bnBhY2sgcm9vdGZzIGltYWdlIGFzIGluaXRyYW1mcy4uLgpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3NzY4OF0gRnJlZWluZyBpbml0cmQgbWVtb3J5
OiAzNzE2NEsKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzkzMjVdIGh3
IHBlcmZldmVudHM6IGVuYWJsZWQgd2l0aCBhcm12OF9wbXV2M18wIFBNVSBkcml2ZXIsIDcgY291
bnRlcnMgYXZhaWxhYmxlClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTc5
NDEyXSBrdm0gWzFdOiBJUEEgU2l6ZSBMaW1pdDogNDggYml0cwpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE3OTQzNF0ga3ZtIFsxXTogR0lDdjM6IG5vIEdJQ1YgcmVzb3Vy
Y2UgZW50cnkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzk0MzVdIGt2
bSBbMV06IGRpc2FibGluZyBHSUN2MiBlbXVsYXRpb24KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNzk0NDZdIGt2bSBbMV06IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGlu
dGVyZmFjZSBlbmFibGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTc5
Nzg3XSBrdm0gWzFdOiB2Z2ljIGludGVycnVwdCBJUlE5ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTgwMTk1XSBrdm0gWzFdOiBWSEUgbW9kZSBpbml0aWFsaXplZCBzdWNj
ZXNzZnVsbHkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODEwMjZdIElu
aXRpYWxpc2Ugc3lzdGVtIHRydXN0ZWQga2V5cmluZ3MKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xODEwMzRdIEtleSB0eXBlIGJsYWNrbGlzdCByZWdpc3RlcmVkClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgxMDcxXSB3b3JraW5nc2V0OiB0aW1l
c3RhbXBfYml0cz00MiBtYXhfb3JkZXI9MjQgYnVja2V0X29yZGVyPTAKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODE3MzFdIHpidWQ6IGxvYWRlZApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MTg0OV0gaW50ZWdyaXR5OiBQbGF0Zm9ybSBLZXly
aW5nIGluaXRpYWxpemVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgx
ODUxXSBLZXkgdHlwZSBhc3ltbWV0cmljIHJlZ2lzdGVyZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xODE4NTJdIEFzeW1tZXRyaWMga2V5IHBhcnNlciAneDUwOScgcmVn
aXN0ZXJlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MTg1OF0gQmxv
Y2sgbGF5ZXIgU0NTSSBnZW5lcmljIChic2cpIGRyaXZlciB2ZXJzaW9uIDAuNCBsb2FkZWQgKG1h
am9yIDI0OSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODE4OTddIGlv
IHNjaGVkdWxlciBtcS1kZWFkbGluZSByZWdpc3RlcmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTgyODk4XSBzaHBjaHA6IFN0YW5kYXJkIEhvdCBQbHVnIFBDSSBDb250
cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xODMwMDhdIGVmaWZiOiBwcm9iaW5nIGZvciBlZmlmYgpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MzAyOF0gZWZpZmI6IGZyYW1lYnVmZmVyIGF0IDB4MjAw
MDAwMDAsIHVzaW5nIDE4NzZrLCB0b3RhbCAxODc1awpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAxLjE4MzAyOV0gZWZpZmI6IG1vZGUgaXMgODAweDYwMHgzMiwgbGluZWxlbmd0
aD0zMjAwLCBwYWdlcz0xClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgz
MDMwXSBlZmlmYjogc2Nyb2xsaW5nOiByZWRyYXcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xODMwMzFdIGVmaWZiOiBUcnVlY29sb3I6IHNpemU9ODo4Ojg6OCwgc2hpZnQ9
MjQ6MTY6ODowClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgzMTAxXSBD
b25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMTAweDM3ClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg5MTU1XSBmYjA6IEVGSSBWR0Eg
ZnJhbWUgYnVmZmVyIGRldmljZQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE4OTYxNF0gaW5wdXQ6IFBvd2VyIEJ1dHRvbiBhcyAvZGV2aWNlcy9MTlhTWVNUTTowMC9MTlhT
WUJVUzowMC9QTlAwQzBDOjAwL2lucHV0L2lucHV0MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAxLjE4OTYyOV0gQUNQSTogUG93ZXIgQnV0dG9uIFtQV1JCXQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5Mzk4MF0gQUNQSSBHVERUOiBmb3VuZCAxIFNC
U0EgZ2VuZXJpYyBXYXRjaGRvZyhzKS4KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xOTQ0OTNdIFNlcmlhbDogODI1MC8xNjU1MCBkcml2ZXIsIDQgcG9ydHMsIElSUSBzaGFy
aW5nIGVuYWJsZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTQ3NTVd
IFNlcmlhbDogQU1CQSBkcml2ZXIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xOTQ3ODhdIG1zbV9zZXJpYWw6IGRyaXZlciBpbml0aWFsaXplZApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NDgzNV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5h
dXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xOTQ4NThdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogaWFzIDQ4LWJpdCwgb2Fz
IDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xOTUwMjRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2Nh
dGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xOTUwNjBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVk
IDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xOTUxMjNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1
NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xOTU1MDJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjEuYXV0bzogb3B0aW9uIG1hc2sgMHgw
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk1NTEzXSBhcm0tc21tdS12
MyBhcm0tc21tdS12My4xLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4
MDAwMDFmZmYpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk1NjI4XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZv
ciBjbWRxClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk1NjY1XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBl
dnRxClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk1NzI4XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlx
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk2MDI4XSBhcm0tc21tdS12
MyBhcm0tc21tdS12My4yLmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE5NjAzOV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRv
OiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NjA4N10gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NjE5OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
Mi5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NjIzM10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5h
dXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE5NjU2Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRv
OiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
OTY1NzZdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4
LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xOTY2MjFdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVk
IDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xOTY3MzJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDMy
NzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xOTY3NjddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2
IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
OTcwNjJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjQuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk3MDcxXSBhcm0tc21tdS12MyBh
cm0tc21tdS12My40LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAw
MDFmZmYpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk3MTQzXSBhcm0t
c21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBj
bWRxClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk3MTg3XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRx
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk3Mjk4XSBhcm0tc21tdS12
MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk3NjA5XSBhcm0tc21tdS12MyBh
cm0tc21tdS12My41LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAxLjE5NzYxOF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBp
YXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NzY4NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
NS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NzcyNl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5h
dXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE5NzgzN10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRv
OiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAxLjE5ODEzOV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBv
cHRpb24gbWFzayAweDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTgx
NDhdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJp
dCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xOTgxODddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1
NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xOTgyNTFdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDMyNzY4
IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
OTgyOTRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVu
dHJpZXMgZm9yIHByaXEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg2
NzRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjcuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk4NjgzXSBhcm0tc21tdS12MyBhcm0t
c21tdS12My43LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFm
ZmYpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk4NzE5XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRx
ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk4NzgwXSBhcm0tc21tdS12
MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTk4ODIyXSBhcm0tc21tdS12MyBh
cm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjAzMTg1XSBtb3VzZWRldjogUFMvMiBt
b3VzZSBkZXZpY2UgY29tbW9uIGZvciBhbGwgbWljZQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIwNDEyNF0gcnRjLWVmaSBydGMtZWZpLjA6IHJlZ2lzdGVyZWQgYXMgcnRj
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNDU4Ml0gcnRjLWVmaSBy
dGMtZWZpLjA6IHNldHRpbmcgc3lzdGVtIGNsb2NrIHRvIDIwMjEtMDktMTNUMDc6MjE6MDQgVVRD
ICgxNjMxNTE3NjY0KQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNTE2
M10gbGVkdHJpZy1jcHU6IHJlZ2lzdGVyZWQgdG8gaW5kaWNhdGUgYWN0aXZpdHkgb24gQ1BVcwpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNTQyNl0gU01DQ0M6IFNPQ19J
RDogSUQgPSBqZXAxMDY6MGExNjowMDAxIFJldmlzaW9uID0gMHgwMDAwMDBhMQpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNTY4Nl0gTkVUOiBSZWdpc3RlcmVkIHByb3Rv
Y29sIGZhbWlseSAxMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxMzY4
NV0gU2VnbWVudCBSb3V0aW5nIHdpdGggSVB2NgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIxMzY5OF0gbWlwNjogTW9iaWxlIElQdjYKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yMTM3MDBdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkg
MTcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTM3MjddIG1wbHNfZ3Nv
OiBNUExTIEdTTyBzdXBwb3J0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjEzNzc3XSByZWdpc3RlcmVkIHRhc2tzdGF0cyB2ZXJzaW9uIDEKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMTM3NzldIExvYWRpbmcgY29tcGlsZWQtaW4gWC41MDkgY2Vy
dGlmaWNhdGVzClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjQwMjAxXSBM
b2FkZWQgWC41MDkgY2VydCAnRGViaWFuIFNlY3VyZSBCb290IENBOiA2Y2NlY2U3ZTRjNmMwZDFm
NjE0OWYzZGQyN2RmY2M1Y2JiNDE5ZWExJwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAxLjI0MDIxMF0gTG9hZGVkIFguNTA5IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBTaWdu
ZXIgMjAyMSAtIGxpbnV4OiA0YjZlZjVhYmNhNjY5ODI1MTc4ZTA1MmM4NDY2N2NjYmMwNTMxZjhj
JwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MDM2NF0genN3YXA6IGxv
YWRlZCB1c2luZyBwb29sIGx6by96YnVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjQwNDU5XSBLZXkgdHlwZSAuX2ZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MDQ2MF0gS2V5IHR5cGUgLmZzY3J5cHQgcmVnaXN0
ZXJlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MDQ2MF0gS2V5IHR5
cGUgZnNjcnlwdC1wcm92aXNpb25pbmcgcmVnaXN0ZXJlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI0MDQ5M10gQXBwQXJtb3I6IEFwcEFybW9yIHNoYTEgcG9saWN5IGhh
c2hpbmcgZW5hYmxlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0OTU0
OV0gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjUwMDg4XSBpbnRlZ3JpdHk6IExvYWRlZCBY
LjUwOSBjZXJ0ICdTVVNFIExpbnV4IEVudGVycHJpc2UgU2VjdXJlIEJvb3QgQ0E6IDNkNGQ0MGNm
OTM4NTM5MDI0YjFjZmM1YTEyZGVkZmU4YjE3ZTc1NWYnClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjUwMDg4XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNh
dGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTAyNDhd
IGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0Nhbm9uaWNhbCBMdGQuIE1hc3RlciBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHk6IGFkOTE5OTBiYzIyYWIxZjUxNzA0OGMyM2I2NjU1YTI2OGUzNDVh
NjMnClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjUwMjQ5XSBpbnRlZ3Jp
dHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yNTA0MDRdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQg
J0ZlZG9yYSBTZWN1cmUgQm9vdCBDQTogZmRlMzI1OTljMmQ2MWRiMWJmNTgwNzMzNWQ3YjIwZTRj
ZDk2M2I0MicKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTA0MDRdIGlu
dGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MDQxN10gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkg
Y2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2
YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yNTA0MThdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTog
VUVGSTpkYgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MDQzMV0gaW50
ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IENvcnBvcmF0aW9uIFVFRkkgQ0Eg
MjAxMTogMTNhZGJmNDMwOWJkODI3MDljOGNkNTRmMzE2ZWQ1MjI5ODhhMWJkNCcKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTA0MzFdIGludGVncml0eTogTG9hZGluZyBY
LjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAxLjI1MDQ0Ml0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdp
bmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0Zjlh
ZTE3YzU1YWY1MycKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTQ4NDFd
IHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTUxMzhdIHBjaWVwb3J0IDAwMGM6MDA6MDEu
MDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAxLjI1NTIxN10gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUg
Z3JvdXAgMQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1NTQ5Ml0gcGNp
ZXBvcnQgMDAwZDowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU1NTM1XSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6
IEFkZGluZyB0byBpb21tdSBncm91cCAyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjU1Nzc2XSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFFUjogZW5hYmxlZCB3aXRoIElS
USAxMDcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTU4MTldIHBjaWVw
b3J0IDAwMGQ6MDA6MDMuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDMKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTYwNjBdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQUVS
OiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjI1NjEwMV0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAg
NApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1NjM0MV0gcGNpZXBvcnQg
MDAwZDowMDowNC4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjU2NDAyXSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFkZGlu
ZyB0byBpb21tdSBncm91cCA1ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjU2NjcwXSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDgK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTY3MzldIHBjaWVwb3J0IDAw
MDA6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDYKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yNTcwMDddIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQUVSOiBlbmFi
bGVkIHdpdGggSVJRIDEwOQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1
NzA3N10gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNwpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1NzM2NF0gcGNpZXBvcnQgMDAwMjow
MDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjU3NDEyXSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFkZGluZyB0byBp
b21tdSBncm91cCA4ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU3NjY3
XSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTc3MzJdIHBjaWVwb3J0IDAwMDM6MDA6
MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yNTgwMTNdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdp
dGggSVJRIDExMQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODA2NF0g
cGNpZXBvcnQgMDAwMzowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTgzMTJdIHBjaWVwb3J0IDAwMDM6MDA6MDUu
MDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAxLjI1ODM3N10gcGNpZXBvcnQgMDAwNDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUg
Z3JvdXAgMTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTg2NThdIHBj
aWVwb3J0IDAwMDQ6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODcwNF0gcGNpZXBvcnQgMDAwNDowMDowMy4w
OiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yNTg5NDldIHBjaWVwb3J0IDAwMDQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGgg
SVJRIDExMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODk5Nl0gcGNp
ZXBvcnQgMDAwNDowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTMKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTkyNDldIHBjaWVwb3J0IDAwMDQ6MDA6MDUuMDog
QUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAxLjI1OTMxNl0gcGNpZXBvcnQgMDAwNDowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3Jv
dXAgMTQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTk1ODBdIHBjaWVw
b3J0IDAwMDU6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE1ClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU5ODYxXSBwY2llcG9ydCAwMDA1OjAwOjAxLjA6IEFF
UjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yNTk5MTBdIHBjaWVwb3J0IDAwMDU6MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDE2ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjYwMTY4XSBwY2llcG9y
dCAwMDA1OjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjAyMTldIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDE3ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjYwNDcxXSBwY2llcG9ydCAwMDA1OjAwOjA3LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAx
MTMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjE4MzddIEZyZWVpbmcg
dW51c2VkIGtlcm5lbCBtZW1vcnk6IDU0NDBLClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNDIyNTQxXSBDaGVja2VkIFcrWCBtYXBwaW5nczogcGFzc2VkLCBubyBXK1ggcGFn
ZXMgZm91bmQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MjI1NDZdIFJ1
biAvaW5pdCBhcyBpbml0IHByb2Nlc3MKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS40MjI1NDddICAgd2l0aCBhcmd1bWVudHM6ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNDIyNTQ4XSAgICAgL2luaXQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS40MjI1NDldICAgd2l0aCBlbnZpcm9ubWVudDoKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS40MjI1NDldICAgICBIT01FPS8KU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS40MjI1NTBdICAgICBURVJNPWxpbnV4ClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDIyNTUxXSAgICAgQk9PVF9JTUFHRT0vYm9vdC92bWxp
bnV6LTUuMTAuMC04LWFybTY0ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NDk2MjU1XSBwcHNfY29yZTogTGludXhQUFMgQVBJIHZlci4gMSByZWdpc3RlcmVkClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk2MjU3XSBwcHNfY29yZTogU29mdHdhcmUg
dmVyLiA1LjMuNiAtIENvcHlyaWdodCAyMDA1LTIwMDcgUm9kb2xmbyBHaW9tZXR0aSA8Z2lvbWV0
dGlAbGludXguaXQ+ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk3MjY0
XSBQVFAgY2xvY2sgc3VwcG9ydCByZWdpc3RlcmVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNTAxNjk1XSBTQ1NJIHN1YnN5c3RlbSBpbml0aWFsaXplZApTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjUwNDU0Nl0gaWdiOiBJbnRlbChSKSBHaWdhYml0
IEV0aGVybmV0IE5ldHdvcmsgRHJpdmVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTA0NTQ3XSBpZ2I6IENvcHlyaWdodCAoYykgMjAwNy0yMDE0IEludGVsIENvcnBvcmF0
aW9uLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjUwNDg4OV0gaWdiIDAw
MDM6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE4ClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNTA1MjgwXSBBQ1BJOiBidXMgdHlwZSBVU0IgcmVnaXN0ZXJlZApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjUwNTI5OF0gdXNiY29yZTogcmVn
aXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciB1c2JmcwpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAxLjUwNTMwM10gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNl
IGRyaXZlciBodWIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDUzMjdd
IHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGRldmljZSBkcml2ZXIgdXNiClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA4NTU2XSBudm1lIDAwMGQ6MDE6MDAuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDE5ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTA4ODI1XSBudm1lIG52bWUwOiBwY2kgZnVuY3Rpb24gMDAwZDowMTowMC4wClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA4ODc1XSBudm1lIDAwMGQ6MDI6MDAuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDIwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNTA5MjY5XSBudm1lIG52bWUxOiBwY2kgZnVuY3Rpb24gMDAwZDowMjowMC4wClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA5MzE1XSBudm1lIDAwMGQ6MDM6MDAuMDog
QWRkaW5nIHRvIGlvbW11IGdyb3VwIDIxClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTA5NTYwXSBudm1lIG52bWUyOiBwY2kgZnVuY3Rpb24gMDAwZDowMzowMC4wClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA5NjA0XSBudm1lIDAwMGQ6MDQ6MDAu
MDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNTA5ODM3XSBudm1lIG52bWUzOiBwY2kgZnVuY3Rpb24gMDAwZDowNDowMC4wClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE0OTQ3XSBtcHQzc2FzIHZlcnNp
b24gMzUuMTAwLjAwLjAwIGxvYWRlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjUxNTA1N10gbXB0M3NhcyAwMDBjOjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAy
MwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjUxNTUyMV0gbXB0M3Nhc19j
bTA6IDYzIEJJVCBQQ0kgQlVTIERNQSBBRERSRVNTSU5HIFNVUFBPUlRFRCwgdG90YWwgbWVtICg2
NTMwNDgwMCBrQikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTU1NjVd
IHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDI0ClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE2MTEwXSB4aGNpX2hjZCAwMDA0OjAzOjAw
LjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTE2MTE0XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lzdGVy
ZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAxLjUxNjEzM10geGhjaV9oY2QgMDAwNDowMzowMC4wOiBaZXJvaW5nIDY0Yml0IGJhc2Ug
cmVnaXN0ZXJzLCBleHBlY3RpbmcgZmF1bHQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS41MTg2NjJdIHJhbmRvbTogZmFzdCBpbml0IGRvbmUKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS41MjA1MTZdIG52bWUgbnZtZTI6IDgwLzAvMCBkZWZhdWx0L3Jl
YWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjA2
NDNdIG52bWUgbnZtZTA6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjExNjddIGFzdCAwMDA0OjAyOjAwLjA6IEFk
ZGluZyB0byBpb21tdSBncm91cCAxNApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjUyMTYwMl0gbnZtZSBudm1lMTogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMTY3OV0gbnZtZSBudm1lMzog
ODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtl
cm5lbDogWyAgICAxLjUyNDE5OF0gIG52bWUwbjE6ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNTI1MDE4XSAgbnZtZTNuMTogcDEgcDIgcDMgcDQgcDUgcDYKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjUzNDVdICBudm1lMW4xOiBwMSBwMiBwMyBw
NApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY2MjI1OF0geGhjaV9oY2Qg
MDAwNDowMzowMC4wOiBoY2MgcGFyYW1zIDB4MDE0MDUxY2YgaGNpIHZlcnNpb24gMHgxMDAgcXVp
cmtzIDB4MDAwMDAwMTEwMDAwMDQxMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjY2MjUzOV0gdXNiIHVzYjE6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0xZDZi
LCBpZFByb2R1Y3Q9MDAwMiwgYmNkRGV2aWNlPSA1LjEwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjYyNTQwXSB1c2IgdXNiMTogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczog
TWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS42NjI1NDFdIHVzYiB1c2IxOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29udHJv
bGxlcgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY2MjU0Ml0gdXNiIHVz
YjE6IE1hbnVmYWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NjI1NDNdIHVzYiB1c2IxOiBTZXJpYWxOdW1i
ZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY2
MjY0MF0gaHViIDEtMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS42NjI2NTFdIGh1YiAxLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjYyNzg2XSB4aGNpX2hjZCAwMDA0OjAz
OjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNjYyNzg4XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lz
dGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAxLjY2Mjc5MF0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBIb3N0IHN1cHBvcnRzIFVT
QiAzLjAgU3VwZXJTcGVlZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3
NjM0OV0gdXNiIHVzYjI6IFdlIGRvbid0IGtub3cgdGhlIGFsZ29yaXRobXMgZm9yIExQTSBmb3Ig
dGhpcyBob3N0LCBkaXNhYmxpbmcgTFBNLgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAxLjY3NjM2Ml0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0x
ZDZiLCBpZFByb2R1Y3Q9MDAwMywgYmNkRGV2aWNlPSA1LjEwClNlcCAxMyAxMjo1MTowNyBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNjc2MzYzXSB1c2IgdXNiMjogTmV3IFVTQiBkZXZpY2Ugc3RyaW5n
czogTWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS42NzYzNjRdIHVzYiB1c2IyOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29u
dHJvbGxlcgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NjM2NF0gdXNi
IHVzYjI6IE1hbnVmYWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzYzNjVdIHVzYiB1c2IyOiBTZXJpYWxO
dW1iZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAx
LjY3NjM5MF0gY2hlY2tpbmcgZ2VuZXJpYyAoMjAwMDAwMDAgMWQ1MDAwKSB2cyBodyAoMjAwMDAw
MDAgMjAwMDAwMCkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzYzOTFd
IGZiMDogc3dpdGNoaW5nIHRvIGFzdGRybWZiIGZyb20gRUZJIFZHQQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NjQ1NV0gQ29uc29sZTogc3dpdGNoaW5nIHRvIGNvbG91
ciBkdW1teSBkZXZpY2UgODB4MjUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42NzY1MDZdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIHBsYXRmb3JtIGhhcyBubyBJTyBzcGFj
ZSwgdHJ5aW5nIE1NSU8KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzY1
MTNdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIFAyQSBicmlkZ2UgZGlzYWJsZWQsIHVzaW5nIGRl
ZmF1bHQgY29uZmlndXJhdGlvbgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAx
LjY3NjUxNF0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gQVNUIDI1MDAgZGV0ZWN0ZWQKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzY1MThdIGFzdCAwMDA0OjAyOjAwLjA6
IFtkcm1dIEFuYWxvZyBWR0Egb25seQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjY3NjUyMF0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gZHJhbSBNQ0xLPTgwMCBNaHogdHlw
ZT0xIGJ1c193aWR0aD0xNgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3
NjU1MF0gaHViIDItMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS42NzY1NzBdIFtUVE1dIFpvbmUgIGtlcm5lbDogQXZhaWxhYmxlIGdyYXBo
aWNzIG1lbW9yeTogMzI2NTI0MDAgS2lCClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNjc2NTcxXSBbVFRNXSBab25lICAgZG1hMzI6IEF2YWlsYWJsZSBncmFwaGljcyBtZW1v
cnk6IDIwOTcxNTIgS2lCClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc2
NTcyXSBbVFRNXSBJbml0aWFsaXppbmcgcG9vbCBhbGxvY2F0b3IKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS42NzY1NzddIFtUVE1dIEluaXRpYWxpemluZyBETUEgcG9vbCBh
bGxvY2F0b3IKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzY3NDFdIGh1
YiAyLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNjc2ODExXSBbZHJtXSBJbml0aWFsaXplZCBhc3QgMC4xLjAgMjAxMjAyMjggZm9y
IDAwMDQ6MDI6MDAuMCBvbiBtaW5vciAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNzkwNjE2XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBk
ZXZpY2UgMjQweDY3ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzkzMjc3
XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBmYjA6IGFzdGRybWZiIGZyYW1lIGJ1ZmZlciBkZXZp
Y2UKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44MjEyNjZdIGlnYiAwMDAz
OjAyOjAwLjA6IGFkZGVkIFBIQyBvbiBldGgwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODIxMjY3XSBpZ2IgMDAwMzowMjowMC4wOiBJbnRlbChSKSBHaWdhYml0IEV0aGVy
bmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjgyMTI2OV0gaWdiIDAwMDM6MDI6MDAuMDogZXRoMDogKFBDSWU6NS4wR2IvczpXaWR0aCB4
MikgZDg6NWU6ZDM6MDI6YjE6OTMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44MjEzNTBdIGlnYiAwMDAzOjAyOjAwLjA6IGV0aDA6IFBCQSBObzogMTA2MzAwLTAwMApTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjgyMTM1Ml0gaWdiIDAwMDM6MDI6MDAu
MDogVXNpbmcgTVNJLVggaW50ZXJydXB0cy4gOCByeCBxdWV1ZShzKSwgOCB0eCBxdWV1ZShzKQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjgyMTQxMV0gaWdiIDAwMDM6MDI6
MDAuMTogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDI1ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODc2OTU0XSBtcHQzc2FzX2NtMDogQ3VycmVudEhvc3RQYWdlU2l6ZSBpcyAw
OiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8gNGsKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NzY5NzFdIG1wdDNzYXNfY20wOiBNU0ktWCB2ZWN0b3JzIHN1
cHBvcnRlZDogOTYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzY5NzJd
IAkgbm8gb2YgY29yZXM6IDgwLCBtYXhfbXNpeF92ZWN0b3JzOiAtMQpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg3Njk3M10gbXB0M3Nhc19jbTA6ICAwIDgwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc4MjEyXSBtcHQzc2FzX2NtMDogSGlnaCBJ
T1BzIHF1ZXVlcyA6IGRpc2FibGVkClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODc4MjEzXSBtcHQzc2FzMC1tc2l4MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjUKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMTRdIG1wdDNzYXMwLW1zaXgx
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2NgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg3ODIxNV0gbXB0M3NhczAtbXNpeDI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDY3ClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc4MjE1XSBtcHQzc2Fz
MC1tc2l4MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjgKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NzgyMTZdIG1wdDNzYXMwLW1zaXg0OiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDQ2OQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3ODIxNl0g
bXB0M3NhczAtbXNpeDU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDcwClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc4MjE3XSBtcHQzc2FzMC1tc2l4NjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0NzEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NzgyMThdIG1wdDNzYXMwLW1zaXg3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MgpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3ODIxOF0gbXB0M3NhczAtbXNpeDg6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNDczClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODc4MjE5XSBtcHQzc2FzMC1tc2l4OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzQK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMTldIG1wdDNzYXMwLW1z
aXgxMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NzgyMjBdIG1wdDNzYXMwLW1zaXgxMTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0NzYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjBdIG1w
dDNzYXMwLW1zaXgxMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzcKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjFdIG1wdDNzYXMwLW1zaXgxMzogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0NzgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NzgyMjJdIG1wdDNzYXMwLW1zaXgxNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzkKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjJdIG1wdDNzYXMwLW1zaXgxNTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NzgyMjNdIG1wdDNzYXMwLW1zaXgxNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
ODEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjRdIG1wdDNzYXMw
LW1zaXgxNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODIKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NzgyMjRdIG1wdDNzYXMwLW1zaXgxODogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0ODMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjVd
IG1wdDNzYXMwLW1zaXgxOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODQKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjZdIG1wdDNzYXMwLW1zaXgyMDogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA0ODUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NzgyMjZdIG1wdDNzYXMwLW1zaXgyMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODYKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjddIG1wdDNzYXMwLW1zaXgy
MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NzgyMjddIG1wdDNzYXMwLW1zaXgyMzogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0ODgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjhdIG1wdDNz
YXMwLW1zaXgyNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODkKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMjhdIG1wdDNzYXMwLW1zaXgyNTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0OTAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44Nzgy
MjldIG1wdDNzYXMwLW1zaXgyNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTEKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzBdIG1wdDNzYXMwLW1zaXgyNzogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0OTIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NzgyMzBdIG1wdDNzYXMwLW1zaXgyODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTMK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzFdIG1wdDNzYXMwLW1z
aXgyOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NzgyMzFdIG1wdDNzYXMwLW1zaXgzMDogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0OTUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzJdIG1w
dDNzYXMwLW1zaXgzMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTYKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzJdIG1wdDNzYXMwLW1zaXgzMjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0OTcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NzgyMzNdIG1wdDNzYXMwLW1zaXgzMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTgKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzNdIG1wdDNzYXMwLW1zaXgzNDog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NzgyMzRdIG1wdDNzYXMwLW1zaXgzNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzVdIG1wdDNzYXMw
LW1zaXgzNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDEKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NzgyMzVdIG1wdDNzYXMwLW1zaXgzNzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzZd
IG1wdDNzYXMwLW1zaXgzODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDMKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzddIG1wdDNzYXMwLW1zaXgzOTogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MDQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NzgyMzddIG1wdDNzYXMwLW1zaXg0MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDUKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzhdIG1wdDNzYXMwLW1zaXg0
MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NzgyMzhdIG1wdDNzYXMwLW1zaXg0MjogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MDcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzldIG1wdDNz
YXMwLW1zaXg0MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDgKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NzgyMzldIG1wdDNzYXMwLW1zaXg0NDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MDkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44Nzgy
NDBdIG1wdDNzYXMwLW1zaXg0NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTAKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDFdIG1wdDNzYXMwLW1zaXg0NjogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MTEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NzgyNDFdIG1wdDNzYXMwLW1zaXg0NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTIK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDJdIG1wdDNzYXMwLW1z
aXg0ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NzgyNDJdIG1wdDNzYXMwLW1zaXg0OTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MTQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDNdIG1w
dDNzYXMwLW1zaXg1MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTUKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDRdIG1wdDNzYXMwLW1zaXg1MTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MTYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NzgyNDRdIG1wdDNzYXMwLW1zaXg1MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTcKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDVdIG1wdDNzYXMwLW1zaXg1Mzog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NzgyNDZdIG1wdDNzYXMwLW1zaXg1NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MTkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDZdIG1wdDNzYXMw
LW1zaXg1NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjAKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NzgyNDddIG1wdDNzYXMwLW1zaXg1NjogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MjEKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDdd
IG1wdDNzYXMwLW1zaXg1NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjIKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNDhdIG1wdDNzYXMwLW1zaXg1ODogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MjMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NzgyNDldIG1wdDNzYXMwLW1zaXg1OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjQKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTBdIG1wdDNzYXMwLW1zaXg2
MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NzgyNTBdIG1wdDNzYXMwLW1zaXg2MTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MjYKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTFdIG1wdDNz
YXMwLW1zaXg2MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjcKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTFdIG1wdDNzYXMwLW1zaXg2MzogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MjgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44Nzgy
NTJdIG1wdDNzYXMwLW1zaXg2NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjkKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTNdIG1wdDNzYXMwLW1zaXg2NTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MzAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NzgyNTNdIG1wdDNzYXMwLW1zaXg2NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzEK
U2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTRdIG1wdDNzYXMwLW1z
aXg2NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NzgyNTVdIG1wdDNzYXMwLW1zaXg2ODogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MzMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTVdIG1w
dDNzYXMwLW1zaXg2OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzQKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTZdIG1wdDNzYXMwLW1zaXg3MDogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MzUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NzgyNTddIG1wdDNzYXMwLW1zaXg3MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzYKU2VwIDEz
IDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNTddIG1wdDNzYXMwLW1zaXg3Mjog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzcKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NzgyNThdIG1wdDNzYXMwLW1zaXg3MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MzgKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNThdIG1wdDNzYXMw
LW1zaXg3NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzkKU2VwIDEzIDEyOjUxOjA3IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NzgyNTldIG1wdDNzYXMwLW1zaXg3NTogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1NDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNjBd
IG1wdDNzYXMwLW1zaXg3NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDEKU2VwIDEzIDEyOjUx
OjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNjBdIG1wdDNzYXMwLW1zaXg3NzogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1NDIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NzgyNjFdIG1wdDNzYXMwLW1zaXg3ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDMKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNjJdIG1wdDNzYXMwLW1zaXg3
OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NzgyNjNdIG1wdDNzYXNfY20wOiBpb21lbSgweDAwMDAwMDAwNDAxNDAwMDAp
LCBtYXBwZWQoMHgoX19fX3B0cnZhbF9fX18pKSwgc2l6ZSg2NTUzNikKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzgyNjRdIG1wdDNzYXNfY20wOiBpb3BvcnQoMHgwMDAw
MDAwMDAwMDAwMDAwKSwgc2l6ZSgwKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjg3OTA2MF0gaWdiIDAwMDM6MDI6MDAuMTogYWRkZWQgUEhDIG9uIGV0aDEKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzkwNjFdIGlnYiAwMDAzOjAyOjAwLjE6IElu
dGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0d29yayBDb25uZWN0aW9uClNlcCAxMyAxMjo1MTow
NyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc5MDYyXSBpZ2IgMDAwMzowMjowMC4xOiBldGgxOiAo
UENJZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1ZTpkMzowMjpiMTo5NApTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg3OTE0M10gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogUEJB
IE5vOiAxMDYzMDAtMDAwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc5
MTQ0XSBpZ2IgMDAwMzowMjowMC4xOiBVc2luZyBNU0ktWCBpbnRlcnJ1cHRzLiA4IHJ4IHF1ZXVl
KHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODgwMTI0XSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IHJlbmFtZWQgZnJvbSBldGgxClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDEuODkzMzIyXSBpZ2IgMDAwMzowMjowMC4wIGVu
bzE6IHJlbmFtZWQgZnJvbSBldGgwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDEuOTM1NjUyXSBtcHQzc2FzX2NtMDogQ3VycmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5n
IGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8gNGsKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS45MzU2NTRdIG1wdDNzYXNfY20wOiBzZW5kaW5nIGRpYWcgcmVzZXQgISEKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMS45OTMxNTFdIHVzYiAxLTE6IG5ldyBs
b3ctc3BlZWQgVVNCIGRldmljZSBudW1iZXIgMiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjE4MTkwM10gdXNiIDEtMTogTmV3IFVTQiBkZXZpY2Ug
Zm91bmQsIGlkVmVuZG9yPTQxM2MsIGlkUHJvZHVjdD0zMDFhLCBiY2REZXZpY2U9IDEuMDAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xODE5MDRdIHVzYiAxLTE6IE5ldyBV
U0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAx
MyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTgxOTA1XSB1c2IgMS0xOiBQcm9kdWN0
OiBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDIuMTgxOTA2XSB1c2IgMS0xOiBNYW51ZmFjdHVyZXI6IFBpeEFydApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjE5MTAwN10gaGlkOiByYXcgSElEIGV2ZW50
cyBkcml2ZXIgKEMpIEppcmkgS29zaW5hClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDIuMjAzMDAxXSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVz
YmhpZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjIwMzAwM10gdXNiaGlk
OiBVU0IgSElEIGNvcmUgZHJpdmVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAg
IDIuMjA0MzIwXSBpbnB1dDogUGl4QXJ0IERlbGwgTVMxMTYgVVNCIE9wdGljYWwgTW91c2UgYXMg
L2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0xLzEt
MToxLjAvMDAwMzo0MTNDOjMwMUEuMDAwMS9pbnB1dC9pbnB1dDEKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMi4yMDQzNjhdIGhpZC1nZW5lcmljIDAwMDM6NDEzQzozMDFBLjAw
MDE6IGlucHV0LGhpZHJhdzA6IFVTQiBISUQgdjEuMTEgTW91c2UgW1BpeEFydCBEZWxsIE1TMTE2
IFVTQiBPcHRpY2FsIE1vdXNlXSBvbiB1c2ItMDAwNDowMzowMC4wLTEvaW5wdXQwClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjg5Nzk1XSB1c2IgMi0zOiBuZXcgU3VwZXJT
cGVlZCBHZW4gMSBVU0IgZGV2aWNlIG51bWJlciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzE4MzMwXSB1c2IgMi0zOiBOZXcgVVNCIGRldmlj
ZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywgaWRQcm9kdWN0PTA2MjAsIGJjZERldmljZT05My4xMApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjMxODMzMV0gdXNiIDItMzogTmV3
IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zMTgzMzFdIHVzYiAyLTM6IFByb2R1
Y3Q6IFVTQjMuMSBIdWIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zMTgz
MzJdIHVzYiAyLTM6IE1hbnVmYWN0dXJlcjogR2VuZXN5c0xvZ2ljClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMzIxMzY2XSBodWIgMi0zOjEuMDogVVNCIGh1YiBmb3VuZApT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjMyMTk1NV0gaHViIDItMzoxLjA6
IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi40
NDkxMjhdIHVzYiAxLTI6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmljZSBudW1iZXIgMyB1c2luZyB4
aGNpX2hjZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjY2MDg5Nl0gdXNi
IDEtMjogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTA0ZDksIGlkUHJvZHVjdD0xMjAz
LCBiY2REZXZpY2U9IDIuODAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi42
NjA4OTddIHVzYiAxLTI6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0wLCBQcm9kdWN0PTAs
IFNlcmlhbE51bWJlcj0wClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDIuNjk5
MDEzXSBpbnB1dDogSElEIDA0ZDk6MTIwMyBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6
MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTIvMS0yOjEuMC8wMDAzOjA0RDk6MTIwMy4wMDAyL2lu
cHV0L2lucHV0MgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjc2MTIzMl0g
aGlkLWdlbmVyaWMgMDAwMzowNEQ5OjEyMDMuMDAwMjogaW5wdXQsaGlkcmF3MTogVVNCIEhJRCB2
MS4xMSBLZXlib2FyZCBbSElEIDA0ZDk6MTIwM10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0
MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAyLjgwMzk3OV0gaW5wdXQ6IEhJ
RCAwNGQ5OjEyMDMgU3lzdGVtIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAw
OjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9p
bnB1dC9pbnB1dDMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMi44NjUxNDJd
IGlucHV0OiBISUQgMDRkOToxMjAzIENvbnN1bWVyIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAw
NDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5
OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDQKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMi44NjUxNjVdIGhpZC1nZW5lcmljIDAwMDM6MDREOToxMjAzLjAwMDM6IGlucHV0LGhpZHJh
dzI6IFVTQiBISUQgdjEuMTEgRGV2aWNlIFtISUQgMDRkOToxMjAzXSBvbiB1c2ItMDAwNDowMzow
MC4wLTIvaW5wdXQxClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTcyNjM2
XSBtcHQzc2FzX2NtMDogZGlhZyByZXNldDogU1VDQ0VTUwpTZXAgMTMgMTI6NTE6MDcgZGViaWFu
IGtlcm5lbDogWyAgICAzLjAwMTEyNV0gdXNiIDEtMzogbmV3IGhpZ2gtc3BlZWQgVVNCIGRldmlj
ZSBudW1iZXIgNCB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDog
WyAgICAzLjAwNjYyOV0gbXB0M3Nhc19jbTA6IHNjYXR0ZXIgZ2F0aGVyOiBzZ2VfaW5fbWFpbl9t
c2coMSksIHNnZV9wZXJfY2hhaW4oNyksIHNnZV9wZXJfaW8oMTI4KSwgY2hhaW5zX3Blcl9pbygx
OSkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wMDc2ODNdIG1wdDNzYXNf
Y20wOiByZXF1ZXN0IHBvb2woMHgoX19fX3B0cnZhbF9fX18pKSAtIGRtYSgweGZmNTAwMDAwKTog
ZGVwdGgoNDIyNCksIGZyYW1lX3NpemUoMTI4KSwgcG9vbF9zaXplKDUyOCBrQikKU2VwIDEzIDEy
OjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wMjc4NDNdIG1wdDNzYXNfY20wOiBzZW5zZSBw
b29sKDB4KF9fX19wdHJ2YWxfX19fKSktIGRtYSgweGZlYjAwMDAwKTogZGVwdGgoMzk2MyksZWxl
bWVudF9zaXplKDk2KSwgcG9vbF9zaXplKDM3MSBrQikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4wMjc5NDNdIG1wdDNzYXNfY20wOiBjb25maWcgcGFnZSgweChfX19fcHRy
dmFsX19fXykpIC0gZG1hKDB4ZmU5ZjYwMDApOiBzaXplKDUxMikKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy4wMjc5NDNdIG1wdDNzYXNfY20wOiBBbGxvY2F0ZWQgcGh5c2lj
YWwgbWVtb3J5OiBzaXplKDcyNzgga0IpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuMDI3OTQ0XSBtcHQzc2FzX2NtMDogQ3VycmVudCBDb250cm9sbGVyIFF1ZXVlIERlcHRo
KDM5NjApLE1heCBDb250cm9sbGVyIFF1ZXVlIERlcHRoKDQwOTYpClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuMDI3OTQ1XSBtcHQzc2FzX2NtMDogU2NhdHRlciBHYXRoZXIg
RWxlbWVudHMgcGVyIElPKDEyOCkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
My4xNTg1NjRdIHVzYiAxLTM6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNWUzLCBp
ZFByb2R1Y3Q9MDYxMCwgYmNkRGV2aWNlPTkzLjEwClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDMuMTU4NTY1XSB1c2IgMS0zOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9
MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5l
bDogWyAgICAzLjE1ODU2Nl0gdXNiIDEtMzogUHJvZHVjdDogVVNCMi4wIEh1YgpTZXAgMTMgMTI6
NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjE1ODU2N10gdXNiIDEtMzogTWFudWZhY3R1cmVy
OiBHZW5lc3lzTG9naWMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNjI1
ODVdIGh1YiAxLTM6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2Vy
bmVsOiBbICAgIDMuMTYzMTkwXSBodWIgMS0zOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjIyODc3OF0gbXB0M3Nhc19jbTA6IF9iYXNl
X2Rpc3BsYXlfZndwa2dfdmVyc2lvbjogY29tcGxldGUKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4yMjkxMDVdIG1wdDNzYXNfY20wOiBMU0lTQVMzMDA4OiBGV1ZlcnNpb24o
MTYuMDAuMTAuMDApLCBDaGlwUmV2aXNpb24oMHgwMiksIEJpb3NWZXJzaW9uKDE1LjAwLjAxLjAw
KQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjIyOTEwNl0gbXB0M3Nhc19j
bTA6IFByb3RvY29sPShJbml0aWF0b3IpLCBDYXBhYmlsaXRpZXM9KFJhaWQsVExSLEVFRFAsU25h
cHNob3QgQnVmZmVyLERpYWcgVHJhY2UgQnVmZmVyLFRhc2sgU2V0IEZ1bGwsTkNRKQpTZXAgMTMg
MTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjIyOTE1Nl0gc2NzaSBob3N0MDogRnVzaW9u
IE1QVCBTQVMgSG9zdApTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjIyOTcy
NF0gbXB0M3Nhc19jbTA6IHNlbmRpbmcgcG9ydCBlbmFibGUgISEKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy4yNTc3NjddIFBNOiBJbWFnZSBub3QgZm91bmQgKGNvZGUgLTIy
KQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjMwNjE4Ml0gRVhUNC1mcyAo
bnZtZTNuMXAyKTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9w
dHM6IChudWxsKQpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjMzNTkzMV0g
Tm90IGFjdGl2YXRpbmcgTWFuZGF0b3J5IEFjY2VzcyBDb250cm9sIGFzIC9zYmluL3RvbW95by1p
bml0IGRvZXMgbm90IGV4aXN0LgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAz
LjUyNjMyMl0gcmFuZG9tOiBzeXN0ZW1kOiB1bmluaXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYg
Ynl0ZXMgcmVhZCkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MjcwNjZd
IHJhbmRvbTogc3lzdGVtZDogdW5pbml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJl
YWQpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTI3MjY5XSByYW5kb206
IHN5c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjU0MzAwOV0gZnVzZTogaW5pdCAoQVBJ
IHZlcnNpb24gNy4zMikKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41NDQ0
NzBdIEVYVDQtZnMgKG52bWUzbjFwMik6IHJlLW1vdW50ZWQuIE9wdHM6IGVycm9ycz1yZW1vdW50
LXJvClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjMyOTc3XSBhY3BpLXRh
ZCBBQ1BJMDAwRTowMDogVW5zdXBwb3J0ZWQgY2FwYWJpbGl0aWVzClNlcCAxMyAxMjo1MTowNyBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuNjc3NDI5XSB4Z2VuZS1zbGltcHJvLWh3bW9uIEFQTUMwRDI5
OjAwOiBBUE0gWC1HZW5lIFNvQyBIVyBtb25pdG9yIGRyaXZlciByZWdpc3RlcmVkClNlcCAxMyAx
Mjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjgxNjUxXSBJUE1JIG1lc3NhZ2UgaGFuZGxl
cjogdmVyc2lvbiAzOS4yClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjg1
MjQ0XSBpcG1pIGRldmljZSBpbnRlcmZhY2UKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy42OTEyMjBdIGlwbWlfc2k6IElQTUkgU3lzdGVtIEludGVyZmFjZSBkcml2ZXIKU2Vw
IDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42OTE0MzVdIGlwbWlfc2k6IFVuYWJs
ZSB0byBmaW5kIGFueSBTeXN0ZW0gSW50ZXJmYWNlKHMpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4g
a2VybmVsOiBbICAgIDMuNzc1MjA1XSBpcG1pX3NzaWY6IElQTUkgU1NJRiBJbnRlcmZhY2UgZHJp
dmVyClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzc5NDA2XSBFWFQ0LWZz
IChudm1lM24xcDUpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4g
T3B0czogKG51bGwpClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzc5NDU3
XSBpcG1pX3NzaWYgaTJjLUFNUEMwMDA0OjAwOiBpcG1pX3NzaWY6IFRyeWluZyBBQ1BJLXNwZWNp
ZmllZCBTU0lGIGludGVyZmFjZSBhdCBpMmMgYWRkcmVzcyAweDEwLCBhZGFwdGVyIFN5bm9wc3lz
IERlc2lnbldhcmUgSTJDIGFkYXB0ZXIsIHNsYXZlIGFkZHJlc3MgMHgyMApTZXAgMTMgMTI6NTE6
MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjc3OTQ2MF0gRVhUNC1mcyAobnZtZTNuMXAzKTogbW91
bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAg
MTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjc5NzE1Ml0gQWRkaW5nIDEwMDA0NDRr
IHN3YXAgb24gL2Rldi9udm1lM24xcDQuICBQcmlvcml0eTotMiBleHRlbnRzOjEgYWNyb3NzOjEw
MDA0NDRrIFNTRlMKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MTA0MTRd
IEVYVDQtZnMgKG52bWUzbjFwNik6IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0
YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
My44MzgzNjldIHBzdG9yZTogVXNpbmcgY3Jhc2ggZHVtcCBjb21wcmVzc2lvbjogZGVmbGF0ZQpT
ZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjgzODM3N10gcHN0b3JlOiBSZWdp
c3RlcmVkIGVmaSBhcyBwZXJzaXN0ZW50IHN0b3JlIGJhY2tlbmQKU2VwIDEzIDEyOjUxOjA3IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy44NDE0NzZdIGlwbWlfc2k6IElQTUkgU3lzdGVtIEludGVyZmFj
ZSBkcml2ZXIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NDE3MjNdIHNi
c2EtZ3dkdCBzYnNhLWd3ZHQuMDogSW5pdGlhbGl6ZWQgd2l0aCAxMHMgdGltZW91dCBAIDI1MDAw
MDAwIEh6LCBhY3Rpb249MC4KU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44
NDE3ODddIGlwbWlfc2k6IFVuYWJsZSB0byBmaW5kIGFueSBTeXN0ZW0gSW50ZXJmYWNlKHMpClNl
cCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuODU4MjkxXSBhdWRpdDogdHlwZT0x
NDAwIGF1ZGl0KDE2MzE1MTc2NjcuMTUyOjIpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249
InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2UteHBk
ZmltcG9ydCIgcGlkPTExMDMgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTI6NTE6MDcg
ZGViaWFuIGtlcm5lbDogWyAgICAzLjg1ODMxNl0gY3J5cHRkOiBtYXhfY3B1X3FsZW4gc2V0IHRv
IDEwMDAKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NTg0MzldIGF1ZGl0
OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNzY2Ny4xNTI6Myk6IGFwcGFybW9yPSJTVEFUVVMiIG9w
ZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJyZW9m
ZmljZS1zZW5kZG9jIiBwaWQ9MTEwNiBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMjo1
MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuODU4NjY4XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0
KDE2MzE1MTc2NjcuMTUyOjQpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVf
bG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibHNiX3JlbGVhc2UiIHBpZD0xMTAxIGNv
bW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
My44NTg4NzZdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNzY2Ny4xNTI6NSk6IGFwcGFy
bW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVk
IiBuYW1lPSJsaWJyZW9mZmljZS1vb3BzbGFzaCIgcGlkPTExMDQgY29tbT0iYXBwYXJtb3JfcGFy
c2VyIgpTZXAgMTMgMTI6NTE6MDcgZGViaWFuIGtlcm5lbDogWyAgICAzLjg1OTU1NV0gYXVkaXQ6
IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE3NjY3LjE1Mjo2KTogYXBwYXJtb3I9IlNUQVRVUyIgb3Bl
cmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Ii91c3IvYmlu
L21hbiIgcGlkPTExMDUgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTI6NTE6MDcgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjg1OTU1N10gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE3
NjY3LjE1Mjo3KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHBy
b2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im1hbl9maWx0ZXIiIHBpZD0xMTA1IGNvbW09ImFwcGFy
bW9yX3BhcnNlciIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NTk1NTld
IGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNzY2Ny4xNTI6OCk6IGFwcGFybW9yPSJTVEFU
VVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJt
YW5fZ3JvZmYiIHBpZD0xMTA1IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjUxOjA3
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NTk1NjFdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYz
MTUxNzY2Ny4xNTI6OSk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2Fk
IiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJudmlkaWFfbW9kcHJvYmUiIHBpZD0xMTA3IGNv
bW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEyOjUxOjA3IGRlYmlhbiBrZXJuZWw6IFsgICAg
My44NTk1NjNdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxNzY2Ny4xNTI6MTApOiBhcHBh
cm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5l
ZCIgbmFtZT0ibnZpZGlhX21vZHByb2JlLy9rbW9kIiBwaWQ9MTEwNyBjb21tPSJhcHBhcm1vcl9w
YXJzZXIiClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMuODc4OTI0XSByYW5k
b206IGNybmcgaW5pdCBkb25lClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDMu
ODc4OTI4XSByYW5kb206IDcgdXJhbmRvbSB3YXJuaW5nKHMpIG1pc3NlZCBkdWUgdG8gcmF0ZWxp
bWl0aW5nClNlcCAxMyAxMjo1MTowNyBkZWJpYW4ga2VybmVsOiBbICAgIDQuMDQwNjEwXSBpcG1p
X3NzaWYgaTJjLUFNUEMwMDA0OjAwOiBJUE1JIG1lc3NhZ2UgaGFuZGxlcjogRm91bmQgbmV3IEJN
QyAobWFuX2lkOiAweDAwM2MwYSwgcHJvZF9pZDogMHgwMjEyLCBkZXZfaWQ6IDB4MjApClNlcCAx
MyAxMjo1MTowOSBkZWJpYW4ga2VybmVsOiBbICAgIDUuNzg4ODY4XSBtcHQzc2FzX2NtMDogaG9z
dF9hZGQ6IGhhbmRsZSgweDAwMDEpLCBzYXNfYWRkcigweDUwMDE0ODUwMDA3NzllMzApLCBwaHlz
KDgpClNlcCAxMyAxMjo1MToxMSBkZWJpYW4ga2VybmVsOiBbICAgIDcuNzAyNjU3XSBpZ2IgMDAw
MzowMjowMC4xIGVubzI6IGlnYjogZW5vMiBOSUMgTGluayBpcyBVcCAxMDAwIE1icHMgRnVsbCBE
dXBsZXgsIEZsb3cgQ29udHJvbDogUlgvVFgKU2VwIDEzIDEyOjUxOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgNy43MDI4MzNdIElQdjY6IEFERFJDT05GKE5FVERFVl9DSEFOR0UpOiBlbm8yOiBsaW5r
IGJlY29tZXMgcmVhZHkKU2VwIDEzIDEyOjUxOjE0IGRlYmlhbiBrZXJuZWw6IFsgICAxMC45MTcx
NTNdIG1wdDNzYXNfY20wOiBwb3J0IGVuYWJsZTogU1VDQ0VTUwpTZXAgMTMgMTI6NTU6NTIgZGVi
aWFuIGtlcm5lbDogWyAgMjg5Ljk5OTI0MV0gZGV2aWNlLW1hcHBlcjogdWV2ZW50OiB2ZXJzaW9u
IDEuMC4zClNlcCAxMyAxMjo1NTo1MiBkZWJpYW4ga2VybmVsOiBbICAyODkuOTk5MzE3XSBkZXZp
Y2UtbWFwcGVyOiBpb2N0bDogNC40My4wLWlvY3RsICgyMDIwLTEwLTAxKSBpbml0aWFsaXNlZDog
ZG0tZGV2ZWxAcmVkaGF0LmNvbQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQm9vdGluZyBMaW51eCBvbiBwaHlzaWNhbCBDUFUgMHgwMDAwMTIwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gTGlu
dXggdmVyc2lvbiA1LjEwLjAtOC1hcm02NCAoZGViaWFuLWtlcm5lbEBsaXN0cy5kZWJpYW4ub3Jn
KSAoZ2NjLTEwIChEZWJpYW4gMTAuMi4xLTYpIDEwLjIuMSAyMDIxMDExMCwgR05VIGxkIChHTlUg
QmludXRpbHMgZm9yIERlYmlhbikgMi4zNS4yKSAjMSBTTVAgRGViaWFuIDUuMTAuNDYtNCAoMjAy
MS0wOC0wMykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGVm
aTogRUZJIHYyLjcwIGJ5IEFtZXJpY2FuIE1lZ2F0cmVuZHMKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGVmaTogQUNQSSAyLjA9MHhlZTkwMDAwMCBTTUJJT1Mg
My4wPTB4ZjFiM2ZmOTggRVNSVD0weGU5NGY2Zjk4IE1PS3Zhcj0weGU5NGRmMDAwIFJORz0weGY3
NjZlODE4IE1FTVJFU0VSVkU9MHhlOTYxZjY5OCAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIGVmaTogc2VlZGluZyBlbnRyb3B5IHBvb2wKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNlY3VyZWJvb3Q6IFNlY3VyZSBib290
IGRpc2FibGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBl
c3J0OiBSZXNlcnZpbmcgRVNSVCBzcGFjZSBmcm9tIDB4MDAwMDAwMDBlOTRmNmY5OCB0byAweDAw
MDAwMDAwZTk0ZjZmZDAuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBFYXJseSB0YWJsZSBjaGVja3N1bSB2ZXJpZmljYXRpb24gZGlzYWJsZWQKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFJTRFAgMHgw
MDAwMDAwMEVFOTAwMDAwIDAwMDAyNCAodjAyIEFtcGVyZSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFhTRFQgMHgwMDAwMDAwMEVFOEYwMDAwIDAw
MDBBNCAodjAxIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTUkgIDAxMDAwMDEzKQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRkFDUCAweDAwMDAw
MDAwRUU4RDAwMDAgMDAwMTE0ICh2MDYgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIElOVEwgMjAx
OTA1MDkpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBEU0RUIDB4MDAwMDAwMDBFRTg2MDAwMCAwMTlBOTUgKHYwMiBBbXBlcmUgSmFkZSAgICAgMDAw
MDAwMDEgSU5UTCAyMDIwMDcxNykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IERCRzIgMHgwMDAwMDAwMEVFOEUwMDAwIDAwMDA1QyAodjAwIEFtcGVy
ZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogR1REVCAweDAwMDAwMDAwRUU4QzAwMDAgMDAw
MTEwICh2MDMgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIElOVEwgMjAxOTA1MDkpClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTU0RUIDB4MDAwMDAw
MDBFRThCMDAwMCAwMDAwMkQgKHYwMiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgSU5UTCAyMDE5
MDUwOSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IEZJRFQgMHgwMDAwMDAwMEVFODUwMDAwIDAwMDA5QyAodjAxIEFMQVNLQSBBIE0gSSAgICAwMTA3
MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogU1BDUiAweDAwMDAwMDAwRUU4NDAwMDAgMDAwMDUwICh2MDIgQUxBU0tB
IEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDAwNTAwMEYpClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBCR1JUIDB4MDAwMDAwMDBFRTgzMDAwMCAwMDAw
MzggKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE1DRkcgMHgwMDAwMDAw
MEVFODIwMDAwIDAwMDBBQyAodjAxIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMSBBTVAuIDAxMDAw
MDEzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
SU9SVCAweDAwMDAwMDAwRUU4MTAwMDAgMDAwNjEwICh2MDAgQW1wZXJlIEFsdHJhICAgIDAwMDAw
MDAwIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBQUFRUIDB4MDAwMDAwMDBFRTdGMDAwMCAwMDZFNjAgKHYwMiBBbXBlcmUg
QWx0cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNMSVQgMHgwMDAwMDAwMEVFN0UwMDAwIDAwMDAy
RCAodjAxIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1JBVCAweDAwMDAwMDAw
RUU3RDAwMDAgMDAwNkQwICh2MDMgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAw
MTMpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBB
UElDIDB4MDAwMDAwMDBFRTgwMDAwMCAwMDE5RjQgKHYwNSBBbXBlcmUgQWx0cmEgICAgMDAwMDAw
MDMgQU1JICAwMTAwMDAxMykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IFBDQ1QgMHgwMDAwMDAwMEVFN0MwMDAwIDAwMDU3NiAodjAyIEFtcGVyZSBB
bHRyYSAgICAwMDAwMDAwMyBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogV1NNVCAweDAwMDAwMDAwRUU3QjAwMDAgMDAwMDI4
ICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGUERUIDB4MDAwMDAwMDBF
RTdBMDAwMCAwMDAwNDQgKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMTAwMDAx
MykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQ
Q1I6IGNvbnNvbGU6IHBsMDExLG1taW8zMiwweDEwMDAwMjYwMDAwMCwxMTUyMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQ6IE5vZGUgMCBQ
WE0gMCBbbWVtIDB4ODgzMDAwMDAtMHg4ODNmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQ6IE5vZGUgMCBQWE0gMCBbbWVtIDB4OTAw
MDAwMDAtMHhmZmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IFNSQVQ6IE5vZGUgMCBQWE0gMCBbbWVtIDB4ODAwMDAwMDAwMDAtMHg4MDA3
ZmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IFNSQVQ6IE5vZGUgMCBQWE0gMCBbbWVtIDB4ODAxMDAwMDAwMDAtMHg4MGZmZmZmZmZmZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE5VTUE6IE5PREVf
REFUQSBbbWVtIDB4ODBmZjgwODNiMDAtMHg4MGZmODA4NWZmZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFpvbmUgcmFuZ2VzOgpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEgICAgICBbbWVtIDB4MDAwMDAwMDA4
ODMwMDAwMC0weDAwMDAwMDAwZmZmZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIERNQTMyICAgIGVtcHR5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIE5vcm1hbCAgIFttZW0gMHgwMDAwMDAwMTAwMDAwMDAw
LTB4MDAwMDA4MGZmZmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIE1vdmFibGUgem9uZSBzdGFydCBmb3IgZWFjaCBub2RlClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBFYXJseSBtZW1vcnkgbm9kZSByYW5nZXMK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6
IFttZW0gMHgwMDAwMDAwMDg4MzAwMDAwLTB4MDAwMDAwMDA4ODNmZmZmZl0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAw
MDAwMDkwMDAwMDAwLTB4MDAwMDAwMDA5MWZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMDkyMDAwMDAw
LTB4MDAwMDAwMDA5M2ZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMDk0MDAwMDAwLTB4MDAwMDAwMDBl
ZTJlNmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAg
bm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGVlMmU3MDAwLTB4MDAwMDAwMDBlZTc0MGZmZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFtt
ZW0gMHgwMDAwMDAwMGVlNzQxMDAwLTB4MDAwMDAwMDBlZTc0NGZmZl0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAw
MGVlNzQ1MDAwLTB4MDAwMDAwMDBlZTc0Y2ZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGVlNzRkMDAwLTB4
MDAwMDAwMDBlZTc0ZGZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGVlNzRlMDAwLTB4MDAwMDAwMDBlZTc0
ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9k
ZSAgIDA6IFttZW0gMHgwMDAwMDAwMGVlNzUwMDAwLTB4MDAwMDAwMDBlZTk2ZmZmZl0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0g
MHgwMDAwMDAwMGVlOTcwMDAwLTB4MDAwMDAwMDBlZTk3ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGVl
OTgwMDAwLTB4MDAwMDAwMDBmMDY0ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGYwNjUwMDAwLTB4MDAw
MDAwMDBmNzY2ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGY3NjcwMDAwLTB4MDAwMDAwMDBmNzg0ZmZm
Zl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAg
IDA6IFttZW0gMHgwMDAwMDAwMGY3ODUwMDAwLTB4MDAwMDAwMDBmN2ZkZmZmZl0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgw
MDAwMDAwMGY3ZmUwMDAwLTB4MDAwMDAwMDBmZmM4ZWZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGZmYzhm
MDAwLTB4MDAwMDAwMDBmZmM4ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDAwMGZmYzkwMDAwLTB4MDAwMDAw
MDBmZmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
ICAgbm9kZSAgIDA6IFttZW0gMHgwMDAwMDgwMDAwMDAwMDAwLTB4MDAwMDA4MDA3ZmZmZmZmZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgbm9kZSAgIDA6
IFttZW0gMHgwMDAwMDgwMTAwMDAwMDAwLTB4MDAwMDA4MGZmZmZmZmZmZl0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEluaXRtZW0gc2V0dXAgbm9kZSAwIFtt
ZW0gMHgwMDAwMDAwMDg4MzAwMDAwLTB4MDAwMDA4MGZmZmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE9uIG5vZGUgMCB0b3RhbHBhZ2VzOiAxNjcx
MTkzNgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEg
em9uZTogNzE3MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDAgcGFnZXMgcmVzZXJ2ZWQKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDQ1OTAw
OCBwYWdlcywgTElGTyBiYXRjaDo2MwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gICBOb3JtYWwgem9uZTogMjUzOTUyIHBhZ2VzIHVzZWQgZm9yIG1lbW1hcApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBOb3JtYWwgem9u
ZTogMTYyNTI5MjggcGFnZXMsIExJRk8gYmF0Y2g6NjMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIGNtYTogUmVzZXJ2ZWQgNjQgTWlCIGF0IDB4MDAwMDAwMDBm
YmMwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNj
aTogcHJvYmluZyBmb3IgY29uZHVpdCBtZXRob2QgZnJvbSBBQ1BJLgpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogUFNDSXYxLjEgZGV0ZWN0ZWQgaW4g
ZmlybXdhcmUuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
c2NpOiBVc2luZyBzdGFuZGFyZCBQU0NJIHYwLjIgZnVuY3Rpb24gSURzClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2NpOiBNSUdSQVRFX0lORk9fVFlQRSBu
b3Qgc3VwcG9ydGVkLgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gcHNjaTogU01DIENhbGxpbmcgQ29udmVudGlvbiB2MS4yClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDEwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDMwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgzMDEwMCAt
PiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NDAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDQwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg1
MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAxMDAgLT4gTm9kZSAwClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDYwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHg2MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NzAwMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHg4MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ODAxMDAg
LT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDkwMDAwIC0+IE5vZGUgMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHg5MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
YTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMTAwIC0+IE5vZGUgMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHhiMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4YjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGMwMDAwIC0+IE5v
ZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4ZDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGQwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhlMDAwMCAtPiBOb2RlIDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4ZTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eGYwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxMDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDExMDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTEwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxMjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDEyMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTMwMDAwIC0+IE5vZGUg
MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDE0MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTQwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNTAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDE1MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MTYwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNjAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE3MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MTcwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxODAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE4MDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MTkwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgxOTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFhMDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWEwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxYjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFi
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWMwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDFkMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWQwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZTAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDFlMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MWYwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZjAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MjAwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMTAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIxMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MjIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgyMjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIzMDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgyNDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI0MDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjUwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyNTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDI2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjYwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDI3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIHBlcmNwdTogRW1iZWRkZWQgMzMgcGFnZXMvY3B1IHM5NTE5MiBy
ODE5MiBkMzE3ODQgdTEzNTE2OApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gcGNwdS1hbGxvYzogczk1MTkyIHI4MTkyIGQzMTc4NCB1MTM1MTY4IGFsbG9jPTMz
KjQwOTYKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUt
YWxsb2M6IFswXSAwMCBbMF0gMDEgWzBdIDAyIFswXSAwMyBbMF0gMDQgWzBdIDA1IFswXSAwNiBb
MF0gMDcgClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1
LWFsbG9jOiBbMF0gMDggWzBdIDA5IFswXSAxMCBbMF0gMTEgWzBdIDEyIFswXSAxMyBbMF0gMTQg
WzBdIDE1IApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNw
dS1hbGxvYzogWzBdIDE2IFswXSAxNyBbMF0gMTggWzBdIDE5IFswXSAyMCBbMF0gMjEgWzBdIDIy
IFswXSAyMyAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBj
cHUtYWxsb2M6IFswXSAyNCBbMF0gMjUgWzBdIDI2IFswXSAyNyBbMF0gMjggWzBdIDI5IFswXSAz
MCBbMF0gMzEgClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
Y3B1LWFsbG9jOiBbMF0gMzIgWzBdIDMzIFswXSAzNCBbMF0gMzUgWzBdIDM2IFswXSAzNyBbMF0g
MzggWzBdIDM5IApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
cGNwdS1hbGxvYzogWzBdIDQwIFswXSA0MSBbMF0gNDIgWzBdIDQzIFswXSA0NCBbMF0gNDUgWzBd
IDQ2IFswXSA0NyAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IHBjcHUtYWxsb2M6IFswXSA0OCBbMF0gNDkgWzBdIDUwIFswXSA1MSBbMF0gNTIgWzBdIDUzIFsw
XSA1NCBbMF0gNTUgClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBwY3B1LWFsbG9jOiBbMF0gNTYgWzBdIDU3IFswXSA1OCBbMF0gNTkgWzBdIDYwIFswXSA2MSBb
MF0gNjIgWzBdIDYzIApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gcGNwdS1hbGxvYzogWzBdIDY0IFswXSA2NSBbMF0gNjYgWzBdIDY3IFswXSA2OCBbMF0gNjkg
WzBdIDcwIFswXSA3MSAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHBjcHUtYWxsb2M6IFswXSA3MiBbMF0gNzMgWzBdIDc0IFswXSA3NSBbMF0gNzYgWzBdIDc3
IFswXSA3OCBbMF0gNzkgClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogR0lDIHN5c3Rl
bSByZWdpc3RlciBDUFUgaW50ZXJmYWNlClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBWaXJ0dWFsaXphdGlvbiBIb3N0
IEV4dGVuc2lvbnMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IENQVSBmZWF0dXJlczoga2VybmVsIHBhZ2UgdGFibGUgaXNvbGF0aW9uIGZvcmNlZCBPTiBieSBL
QVNMUgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogS2VybmVsIHBhZ2UgdGFibGUgaXNvbGF0aW9uIChLUFRJKQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBk
ZXRlY3RlZDogSGFyZHdhcmUgZGlydHkgYml0IG1hbmFnZW1lbnQKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFNwZWN0
cmUtdjQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBm
ZWF0dXJlczogZGV0ZWN0ZWQ6IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGFsdGVybmF0aXZlczogcGF0Y2hpbmcga2VybmVs
IGNvZGUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEJ1aWx0
IDEgem9uZWxpc3RzLCBtb2JpbGl0eSBncm91cGluZyBvbi4gIFRvdGFsIHBhZ2VzOiAxNjQ1MDgx
MgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gUG9saWN5IHpv
bmU6IE5vcm1hbApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
S2VybmVsIGNvbW1hbmQgbGluZTogQk9PVF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04LWFy
bTY0IHJvb3Q9VVVJRD02YzA4MzI3Yy00YmJkLTQzNDEtYTQyNy1hNzJiYzE1MzM4OTUgcm8gcXVp
ZXQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azog
bG9nX2J1Zl9sZW4gaW5kaXZpZHVhbCBtYXggY3B1IGNvbnRyaWJ1dGlvbjogNDA5NiBieXRlcwpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2df
YnVmX2xlbiB0b3RhbCBjcHVfZXh0cmEgY29udHJpYnV0aW9uczogMzIzNTg0IGJ5dGVzClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZf
bGVuIG1pbiBzaXplOiAxMzEwNzIgYnl0ZXMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW46IDUyNDI4OCBieXRlcwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBlYXJseSBsb2cg
YnVmIGZyZWU6IDExOTAxNig5MCUpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBEZW50cnkgY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA4Mzg4NjA4IChvcmRl
cjogMTQsIDY3MTA4ODY0IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBJbm9kZS1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDQxOTQz
MDQgKG9yZGVyOiAxMywgMzM1NTQ0MzIgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIG1lbSBhdXRvLWluaXQ6IHN0YWNrOm9mZiwgaGVh
cCBhbGxvYzpvbiwgaGVhcCBmcmVlOm9mZgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gc29mdHdhcmUgSU8gVExCOiBtYXBwZWQgW21lbSAweDAwMDAwMDAwZWEy
ZTcwMDAtMHgwMDAwMDAwMGVlMmU3MDAwXSAoNjRNQikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIE1lbW9yeTogMjU1MTYzMksvNjY4NDc3NDRLIGF2YWlsYWJs
ZSAoMTE3NzZLIGtlcm5lbCBjb2RlLCAyNDM2SyByd2RhdGEsIDcwMDhLIHJvZGF0YSwgNTQ0MEsg
aW5pdCwgNTk4SyBic3MsIDE1ODU1NTZLIHJlc2VydmVkLCA2NTUzNksgY21hLXJlc2VydmVkKQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcmFuZG9tOiBnZXRf
cmFuZG9tX3U2NCBjYWxsZWQgZnJvbSBfX2ttZW1fY2FjaGVfY3JlYXRlKzB4MzgvMHg1NjAgd2l0
aCBjcm5nX2luaXQ9MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gU0xVQjogSFdhbGlnbj02NCwgT3JkZXI9MC0zLCBNaW5PYmplY3RzPTAsIENQVXM9ODAsIE5v
ZGVzPTEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGZ0cmFj
ZTogYWxsb2NhdGluZyAzODUzNSBlbnRyaWVzIGluIDE1MSBwYWdlcwpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZnRyYWNlOiBhbGxvY2F0ZWQgMTUxIHBhZ2Vz
IHdpdGggNSBncm91cHMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHJjdTogSGllcmFyY2hpY2FsIFJDVSBpbXBsZW1lbnRhdGlvbi4KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogCVJDVSByZXN0cmljdGluZyBDUFVz
IGZyb20gTlJfQ1BVUz0yNTYgdG8gbnJfY3B1X2lkcz04MC4KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIAlSdWRlIHZhcmlhbnQgb2YgVGFza3MgUkNVIGVuYWJs
ZWQuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAJVHJhY2lu
ZyB2YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gcmN1OiBSQ1UgY2FsY3VsYXRlZCB2YWx1ZSBvZiBzY2hlZHVs
ZXItZW5saXN0bWVudCBkZWxheSBpcyAyNSBqaWZmaWVzLgpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcmN1OiBBZGp1c3RpbmcgZ2VvbWV0cnkgZm9yIHJjdV9m
YW5vdXRfbGVhZj0xNiwgbnJfY3B1X2lkcz04MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gTlJfSVJRUzogNjQsIG5yX2lycXM6IDY0LCBwcmVhbGxvY2F0ZWQg
aXJxczogMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lD
djM6IEdJQzogVXNpbmcgc3BsaXQgRU9JL0RlYWN0aXZhdGUgbW9kZQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDY3MiBTUElzIGltcGxlbWVudGVk
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMCBF
eHRlbmRlZCBTUElzIGltcGxlbWVudGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBHSUN2MzogRGlzdHJpYnV0b3IgaGFzIG5vIFJhbmdlIFNlbGVjdG9yIHN1
cHBvcnQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3Yz
OiAxNiBQUElzIGltcGxlbWVudGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBHSUN2MzogQ1BVMDogZm91bmQgcmVkaXN0cmlidXRvciAxMjAwMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwNWMwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMCAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAxIC0+IE5v
ZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDog
UFhNIDAgLT4gSVRTIDIgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMyAtPiBOb2RlIDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA0IC0+
IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JB
VDogUFhNIDAgLT4gSVRTIDUgLT4gTm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNiAtPiBOb2RlIDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA3
IC0+IE5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTIFttZW0gMHgxMDAxMDAwNDAwMDAtMHgxMDAxMDAwNWZmZmZdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQwMDAwOiBhbGxv
Y2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJjMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRL
LCBzaHIgMSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElU
U0AweDAwMDAxMDAxMDAwNDAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlv
bnMgQDgwMDAwMmQwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDA2MDAw
MC0weDEwMDEwMDA3ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgw
MDAwMmYwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA2MDAwMDog
YWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzMDAwMDAgKGZsYXQs
IGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDgwMDAwLTB4MTAwMTAwMDlmZmZmXQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEw
MDA4MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzMjAwMDAgKGluZGlyZWN0LCBl
c3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDgwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJy
dXB0IENvbGxlY3Rpb25zIEA4MDAwMDMzMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAx
KQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0g
MHgxMDAxMDAwYTAwMDAtMHgxMDAxMDAwYmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxvY2F0ZWQgODE5
MiBEZXZpY2VzIEA4MDAwMDM1MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAx
MDAxMDAwYTAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAw
MzYwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDBjMDAwMC0weDEwMDEw
MDBkZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElU
U0AweDAwMDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwMzgwMDAw
IChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBjMDAwMDogYWxsb2NhdGVk
IDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzOTAwMDAgKGZsYXQsIGVzeiAyLCBw
c3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIElUUyBbbWVtIDB4MTAwMTAwMGUwMDAwLTB4MTAwMTAwMGZmZmZmXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBlMDAwMDog
YWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzYjAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6
IDY0Sywgc2hyIDEpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBJVFNAMHgwMDAwMTAwMTAwMGUwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxl
Y3Rpb25zIEA4MDAwMDNjMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAx
MDAwMDAtMHgxMDAxMDAxMWZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2Vz
IEA4MDAwMDNlMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAxMDAw
MDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwM2YwMDAwIChm
bGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDEyMDAwMC0weDEwMDEwMDEzZmZmZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAx
MDAxMDAxMjAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwODEwMDAwIChpbmRpcmVj
dCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEyMDAwMDogYWxsb2NhdGVkIDMyNzY4IElu
dGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDA4MjAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBz
aHIgMSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3Yz
OiB1c2luZyBMUEkgcHJvcGVydHkgdGFibGUgQDB4MDAwMDA4MDAwMDgzMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IENQVTA6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwODQwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJIEdURFQ6IGZvdW5kIDEgbWVtb3J5LW1h
cHBlZCB0aW1lciBibG9jayhzKS4KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIGFyY2hfdGltZXI6IGNwMTUgYW5kIG1taW8gdGltZXIocykgcnVubmluZyBhdCAy
NS4wME1IeiAocGh5cy9waHlzKS4KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIGNsb2Nrc291cmNlOiBhcmNoX3N5c19jb3VudGVyOiBtYXNrOiAweGZmZmZmZmZm
ZmZmZmZmIG1heF9jeWNsZXM6IDB4NWM0MDkzOWI1LCBtYXhfaWRsZV9uczogNDQwNzk1MjAyNjQ2
IG5zClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAxXSBzY2hlZF9j
bG9jazogNTYgYml0cyBhdCAyNU1IeiwgcmVzb2x1dGlvbiA0MG5zLCB3cmFwcyBldmVyeSA0Mzk4
MDQ2NTExMTAwbnMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwNTNd
IENvbnNvbGU6IGNvbG91ciBkdW1teSBkZXZpY2UgODB4MjUKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwNzVdIEFDUEk6IENvcmUgcmV2aXNpb24gMjAyMDA5MjUKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAyNjddIENhbGlicmF0aW5nIGRl
bGF5IGxvb3AgKHNraXBwZWQpLCB2YWx1ZSBjYWxjdWxhdGVkIHVzaW5nIHRpbWVyIGZyZXF1ZW5j
eS4uIDUwLjAwIEJvZ29NSVBTIChscGo9MTAwMDAwKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDI2OV0gcGlkX21heDogZGVmYXVsdDogODE5MjAgbWluaW11bTogNjQw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzAyXSBMU006IFNlY3Vy
aXR5IEZyYW1ld29yayBpbml0aWFsaXppbmcKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAzMDddIFlhbWE6IGRpc2FibGVkIGJ5IGRlZmF1bHQ7IGVuYWJsZSB3aXRoIHN5
c2N0bCBrZXJuZWwueWFtYS4qClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMzI1XSBBcHBBcm1vcjogQXBwQXJtb3IgaW5pdGlhbGl6ZWQKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzMjZdIFRPTU9ZTyBMaW51eCBpbml0aWFsaXplZApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDM2Nl0gTW91bnQtY2FjaGUgaGFz
aCB0YWJsZSBlbnRyaWVzOiAxMzEwNzIgKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIp
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzkxXSBNb3VudHBvaW50
LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMxMDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRl
cywgbGluZWFyKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTI1M10g
cmN1OiBIaWVyYXJjaGljYWwgU1JDVSBpbXBsZW1lbnRhdGlvbi4KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOTNdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDQw
MDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxMzk0XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5Nl0gUGxhdGZvcm0gTVNJ
OiBJVFNAMHgxMDAxMDAwODAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDEzOTddIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGEwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
Mzk4XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5OV0gUGxhdGZvcm0gTVNJOiBJ
VFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDE0MDFdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMTAwMDAwIGRv
bWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDAy
XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwNl0gUENJL01TSTogSVRTQDB4MTAw
MTAwMDQwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAxNDA3XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDhdIFBDSS9NU0k6IElU
U0AweDEwMDEwMDA4MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTQxMF0gUENJL01TSTogSVRTQDB4MTAwMTAwMGEwMDAwIGRvbWFpbiBj
cmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDEzXSBQQ0kv
TVNJOiBJVFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTVdIFBDSS9NU0k6IElUU0AweDEwMDEwMDBlMDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQx
Nl0gUENJL01TSTogSVRTQDB4MTAwMTAwMTAwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDE3XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAx
MjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDE0MjBdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyMV0gZnNsLW1jIE1TSTog
SVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAxNDIyXSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwODAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjRd
IGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyNV0gZnNsLW1jIE1TSTogSVRTQDB4MTAw
MTAwMGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAxNDI2XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0
ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjddIGZzbC1tYyBN
U0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMTQyOF0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMTIwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
NDM0XSBSZW1hcHBpbmcgYW5kIGVuYWJsaW5nIEVGSSBzZXJ2aWNlcy4KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDM5MTBdIHNtcDogQnJpbmdpbmcgdXAgc2Vjb25kYXJ5
IENQVXMgLi4uClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA0MTE1XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNDEzM10gR0lDdjM6IENQVTE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWEwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDdjMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNDE1MV0gR0lDdjM6IENQVTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwODUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA0MTg0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA0MTkzXSBDUFUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYTAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA0NjY4
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNDY4NF0gR0lDdjM6IENQVTI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY0MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNDcwMV0gR0lDdjM6IENQVTI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwODYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA0NzM2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA0NzQ1XSBDUFUyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNDAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA1
NjY1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMwpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNTY4Ml0gR0lDdjM6IENQVTM6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MWMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDg0MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNTcwMF0gR0lDdjM6IENQVTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwODcwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA1NzMzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA1NzQyXSBDUFUzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAx
YzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA2MTQzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNjE1N10gR0lDdjM6IENQVTQ6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgMTAwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDU0MDAwMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNjE3Nl0gR0lDdjM6IENQVTQ6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwODgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA2MjExXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA2MjIxXSBDUFU0OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAxMDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA2NjY5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjY4NV0gR0lDdjM6IENQVTU6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMTgwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc0MDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjcwNF0gR0lDdjM6IENQVTU6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwODkwMDAwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA2NzM5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA2NzQ5XSBDUFU1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxODAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA3NzUwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNgpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzc2Nl0gR0lDdjM6IENQVTY6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDZjMDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzc4NV0gR0lDdjM6IENQVTY6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGEwMDAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3ODIxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA3ODMxXSBDUFU2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAxNjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA4MTQ2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNwpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODE2NF0gR0lDdjM6IENQVTc6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMWUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDhjMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODE4M10gR0lDdjM6IENQVTc6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGIwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MjE4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MjI4XSBDUFU3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAxZTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA4NTQxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVOApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODU1NF0gR0lDdjM6IENQVTg6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgYTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2MwMDAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NTcyXSBHSUN2MzogQ1BVODogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4YzAwMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2MTBdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2MjBdIENQVTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDBhMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDg5NDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU5ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTY3XSBHSUN2MzogQ1BVOTogZm91
bmQgcmVkaXN0cmlidXRvciAyMjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOWMwMDAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4OTg1XSBHSUN2MzogQ1BVOTogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ZDAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDkwMTddIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDkwMjddIENQVTk6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDIyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDkzNDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTM2Ml0gR0lDdjM6IENQVTEw
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ0MDAwMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTM4Ml0gR0lDdjM6IENQVTEw
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhlMDAwMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQxOF0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQyOV0gQ1BVMTA6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk3NTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFUxMQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTc3MV0gR0lDdjM6
IENQVTExOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBh
NDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk3OTBdIEdJQ3Yz
OiBDUFUxMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4
ZjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk4MjNdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk4MzRdIENQVTExOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwMTMyXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMTIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAxNDRd
IEdJQ3YzOiBDUFUxMjogZm91bmQgcmVkaXN0cmlidXRvciA4MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAzNDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAxNjRd
IEdJQ3YzOiBDUFUxMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5MDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyMDBd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyMTFdIENQVTEy
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwODAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwNTMzXSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMTMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA1NTJdIEdJQ3YzOiBDUFUxMzogZm91bmQgcmVkaXN0cmlidXRvciAyMDAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwOTQwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDEwNTcyXSBHSUN2MzogQ1BVMTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwOTEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDEwNjAzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwNjE1
XSBDUFUxMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjAwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDkzMV0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwOTQ1XSBHSUN2MzogQ1BVMTQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZTAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwNGMwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwOTY3XSBHSUN2MzogQ1BVMTQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwOTIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDExMDA0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDExMDE1XSBDUFUxNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGUwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTMzN10g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDExMzU4XSBHSUN2MzogQ1BVMTU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjYw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFjMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTM3OF0gR0lDdjM6IENQVTE1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkzMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTQxMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTQyMl0gQ1BVMTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI2
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTE3MzNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNgpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMTc0M10gR0lDdjM6IENQVTE2OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFjMDAwMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMTc2NF0gR0lDdjM6IENQVTE2OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk0MDAwMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMTgwMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMTgxNF0gQ1BVMTY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDAyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTIxMzBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNwpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE0Ml0gR0lDdjM6IENQVTE3OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI0MDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE2M10gR0lDdjM6IENQVTE3OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk1MDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjIwMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMjIxM10gQ1BVMTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDA0MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTMzNDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxOApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzM1Nl0gR0lDdjM6IENQVTE4OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMTQwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMzc4XSBHSUN2MzogQ1BVMTg6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTYwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNDE2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNDI4XSBDUFUxODogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMDAwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNDE0Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE5ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MTU1XSBHSUN2MzogQ1BVMTk6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgNjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMmMwMDAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MTc3XSBHSUN2MzogQ1BVMTk6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTcwMDAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MjEzXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MjI2XSBDUFUxOTogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMDYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNDUyOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0NTQyXSBHSUN2MzogQ1BV
MjA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDYwMDAw
MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDU2Ml0gR0lDdjM6IENQ
VTIwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk4MDAw
MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDU5N10gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDYwOV0gQ1BVMjA6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ5MjhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUyMQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDk0NF0gR0lD
djM6IENQVTIxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFiMDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA4MDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ5NjRdIEdJ
Q3YzOiBDUFUyMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDA5OTAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ5OTddIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUwMDldIENQVTIxOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYjAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MzI0XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVMjIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUz
MzldIEdJQ3YzOiBDUFUyMjogZm91bmQgcmVkaXN0cmlidXRvciAxNTAwMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNjgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1
MzU5XSBHSUN2MzogQ1BVMjI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwOWEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1
Mzk1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NDA4XSBD
UFUyMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTUwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTcyNV0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTIzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE1NzQyXSBHSUN2MzogQ1BVMjM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWQwMDAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDg4MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNTc2NF0gR0lDdjM6IENQVTIzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDliMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNTc5OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NTgxMF0gQ1BVMjM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFkMDAwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTYxMzVdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxNjE0OV0gR0lDdjM6IENQVTI0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDExMDAw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA1ODAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTYxNjldIEdJQ3YzOiBDUFUyNDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDA5YzAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTYyMDVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTYyMThdIENQVTI0OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMTAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2
NTM1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjUKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTY1NTJdIEdJQ3YzOiBDUFUyNTogZm91bmQgcmVkaXN0cmlidXRv
ciAxOTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE2NTczXSBHSUN2MzogQ1BVMjU6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWQwMDAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE2NjA2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE2NjE5XSBDUFUyNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMTkwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNjkyOV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI2ClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2OTQ1XSBHSUN2MzogQ1BVMjY6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDcwMDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjk2N10gR0lDdjM6IENQVTI2OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDllMDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzAwMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNzAxNV0gQ1BVMjY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDE3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTczMzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzM1MF0gR0lDdjM6IENQVTI3OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDFmMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5MDAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczNzJdIEdJQ3YzOiBDUFUyNzogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ZjAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc0MDZdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc0MTldIENQVTI3OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAxZjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE3NzMwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjgK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc3NDJdIEdJQ3YzOiBDUFUy
ODogZm91bmQgcmVkaXN0cmlidXRvciBiMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0MDAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc3NjVdIEdJQ3YzOiBDUFUy
ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMDAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc4MDJdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc4MTZdIENQVTI4OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MTM1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMjkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTgxNTNdIEdJQ3Yz
OiBDUFUyOTogZm91bmQgcmVkaXN0cmlidXRvciAyMzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
YTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MTc0XSBHSUN2
MzogQ1BVMjk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YTEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MjA2XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MjE5XSBDUFUyOTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODUyNl0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTMwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NTQw
XSBHSUN2MzogQ1BVMzA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZDAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwNDgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NTYy
XSBHSUN2MzogQ1BVMzA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYTIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NTk4
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NjEyXSBDUFUz
MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGQwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODkzMV0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTMxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE4OTUxXSBHSUN2MzogQ1BVMzE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjUwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMGE4MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODk3M10gR0lDdjM6IENQVTMxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGEzMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxOTAwNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTAx
OF0gQ1BVMzE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI1MDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkzMzBdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTM0M10gR0lDdjM6IENQVTMyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDkwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDM4MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTM2Nl0gR0lDdjM6IENQVTMyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGE0MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTQwMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxOTQxNl0gQ1BVMzI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA5MDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk3MzFd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTc0OV0gR0lDdjM6IENQVTMzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIx
MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5ODAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTk3NzJdIEdJQ3YzOiBDUFUzMzogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNTAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTk4MDZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTk4MjBdIENQVTMzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAy
MTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDIwMTI3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzQKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxNDFdIEdJQ3YzOiBDUFUzNDogZm91bmQgcmVkaXN0cmli
dXRvciBmMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1MDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxNjVdIEdJQ3YzOiBDUFUzNDogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjAyMDJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjAyMTddIENQVTM0OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAwZjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDIwNTM0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzUKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1NTRdIEdJQ3YzOiBDUFUzNTogZm91bmQgcmVk
aXN0cmlidXRvciAyNzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYjAwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNTc4XSBHSUN2MzogQ1BVMzU6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTcwMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjEzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNjI3XSBDUFUzNTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMjcwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyMDkzMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM2ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwOTQxXSBHSUN2MzogQ1BVMzY6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMjAwMDAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwOTY2XSBHSUN2MzogQ1BVMzY6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTgwMDAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMDAzXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMDE4XSBDUFUzNjogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMDMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMTMyN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM3
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMzM5XSBHSUN2MzogQ1BV
Mzc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMjgwMDAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMzYzXSBHSUN2MzogQ1BV
Mzc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTkwMDAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNDAxXSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNDE2XSBDUFUzNzogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDUwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTczMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTM4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNzQzXSBHSUN2
MzogQ1BVMzg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
MTgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNzY4XSBHSUN2
MzogQ1BVMzg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YWEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxODA2XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxODIxXSBDUFUzODogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjEzMF0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTM5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMTQy
XSBHSUN2MzogQ1BVMzk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNzAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwMzAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMTY3
XSBHSUN2MzogQ1BVMzk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYWIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMjAz
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMjE5XSBDUFUz
OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDcwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjUyOF0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDIyNTQyXSBHSUN2MzogQ1BVNDA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMTAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDVlMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMjU2NV0gR0lDdjM6IENQVTQwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGFjMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMjU5OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjYx
Ml0gQ1BVNDA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEyMDEwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5MjddIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyMjk0M10gR0lDdjM6IENQVTQxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFhMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA3ZTAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjI5NjZdIEdJQ3YzOiBDUFU0MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBhZDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjI5OTldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjMwMTRdIENQVTQxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYTAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzI4
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjMzNDRdIEdJQ3YzOiBDUFU0MjogZm91bmQgcmVkaXN0cmlidXRvciAx
NDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIzMzY4XSBHSUN2MzogQ1BVNDI6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYWUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIzNDAxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIzNDE2XSBDUFU0MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MTQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMzkzNV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQzClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzOTUzXSBHSUN2MzogQ1BVNDM6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMWMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDg2MDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzk3N10gR0lDdjM6IENQVTQzOiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFmMDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDAwOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyNDAyNF0gQ1BVNDM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDFjMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjQzMzVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDM0OV0gR0lDdjM6IENQVTQ0OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDEwMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1NjAwMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQzNzRdIEdJQ3YzOiBDUFU0NDogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMDAwMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ0MDldIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ0MjRdIENQVTQ0OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAxMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI0NzM0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDUKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ3NTBdIEdJQ3YzOiBDUFU0NTog
Zm91bmQgcmVkaXN0cmlidXRvciAxODAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzYwMDAwClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0Nzc0XSBHSUN2MzogQ1BVNDU6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjEwMDAwClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0ODA2XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0ODIyXSBDUFU0NTogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTgwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTEzNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTQ2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1MTUxXSBHSUN2Mzog
Q1BVNDY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDZl
MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTE3NV0gR0lDdjM6
IENQVTQ2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIy
MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTIxMl0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTIyN10gQ1BVNDY6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE2MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU1MzJdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU0NwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTU0OV0g
R0lDdjM6IENQVTQ3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFlMDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA4ZTAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU1NzNd
IEdJQ3YzOiBDUFU0NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBiMzAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU2MDhd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU2MjNdIENQVTQ3
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZTAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1OTMyXSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNDgKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjU5NDRdIEdJQ3YzOiBDUFU0ODogZm91bmQgcmVkaXN0cmlidXRvciBhMDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDAzZTAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjU5NjhdIEdJQ3YzOiBDUFU0ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBiNDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjYwMDVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjYwMjFd
IENQVTQ4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYTAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MzM1XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNDkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjYzNTJdIEdJQ3YzOiBDUFU0OTogZm91bmQgcmVkaXN0cmlidXRvciAyMjAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwOWUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI2Mzc3XSBHSUN2MzogQ1BVNDk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYjUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI2NDEyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI2NDI3XSBDUFU0OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjIwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjcyMl0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI2NzM1XSBHSUN2MzogQ1BVNTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYzAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI2NzU5XSBHSUN2MzogQ1BVNTA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI2Nzk0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI2ODEwXSBDUFU1MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGMw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NzEyM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUxClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI3MTQxXSBHSUN2MzogQ1BVNTE6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgMjQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGE2MDAwMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNzE2Nl0gR0lDdjM6IENQVTUxOiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI3MDAwMApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNzE5OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNzIxNV0gQ1BVNTE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDI0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjc1MjJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MgpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzUzM10gR0lDdjM6IENQVTUyOiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDM2MDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzU1OF0gR0lDdjM6IENQVTUyOiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI4MDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzU5Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNzYxM10gQ1BVNTI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDA4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjc5MzBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzk0N10gR0lDdjM6IENQVTUzOiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDIwMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5NjAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc5NzNdIEdJQ3YzOiBDUFU1MzogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiOTAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwMDddIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwMjNdIENQVTUzOiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAyMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI4MzI4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTQK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgzNDJdIEdJQ3YzOiBDUFU1
NDogZm91bmQgcmVkaXN0cmlidXRvciBlMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0ZTAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgzNjhdIEdJQ3YzOiBDUFU1
NDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiYTAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg0MDVdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg0MjJdIENQVTU0OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NzM4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNTUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg3NTddIEdJQ3Yz
OiBDUFU1NTogZm91bmQgcmVkaXN0cmlidXRvciAyNjAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
YWUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NzgyXSBHSUN2
MzogQ1BVNTU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YmIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4ODE3XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4ODM0XSBDUFU1NTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjYwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTEzMl0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTU2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MTQy
XSBHSUN2MzogQ1BVNTY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAxMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwMWUwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MTY4
XSBHSUN2MzogQ1BVNTY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYmMwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MjA2
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MjIzXSBDUFU1
NjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDIwMTAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTUzNV0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTU3ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5NTQ2XSBHSUN2MzogQ1BVNTc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNDAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwMjYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5NTcyXSBHSUN2MzogQ1BVNTc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYmQwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5NjEwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NjI3
XSBDUFU1NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDQwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTkyOF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5OTM3XSBHSUN2MzogQ1BVNTg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDE2MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyOTk2NF0gR0lDdjM6IENQVTU4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGJlMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMDAwMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MDAxOV0gQ1BVNTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAwMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAzMzFdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1OQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMDM0Ml0gR0lDdjM6IENQVTU5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDYwMTAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJlMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMDM3MF0gR0lDdjM6IENQVTU5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGJmMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMDQwNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMDQyNF0gQ1BVNTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA2MDEw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA3
MzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMDc0N10gR0lDdjM6IENQVTYwOiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDEzMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2MjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzA3NzNdIEdJQ3YzOiBDUFU2MDogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzA4MTBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzA4MjddIENQVTYwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAxMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMxMTM0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjEKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzExNTFdIEdJQ3YzOiBDUFU2MTogZm91bmQgcmVkaXN0
cmlidXRvciAxYjAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODIwMDAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMTc3XSBHSUN2MzogQ1BVNjE6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzEwMDAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMjEwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMxMjI3XSBDUFU2MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMWIwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMTUzMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYyClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNTQ3XSBHSUN2MzogQ1BVNjI6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMTUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDZhMDAwMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTU3NF0gR0lDdjM6IENQVTYyOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMyMDAwMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTYwOV0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTYyN10gQ1BVNjI6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDE1MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzE5MjZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MwpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTk0M10gR0lDdjM6IENQVTYz
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4YTAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE5NzBdIEdJQ3YzOiBDUFU2
MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMzAwMDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwMDNdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwMjBdIENQVTYzOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyMzI5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNjQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIzNDNdIEdJQ3Yz
OiBDUFU2NDogZm91bmQgcmVkaXN0cmlidXRvciAxMTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
NWEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyMzcwXSBHSUN2
MzogQ1BVNjQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YzQwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDA1XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDIyXSBDUFU2NDogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjc1Ml0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTY1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNzY5
XSBHSUN2MzogQ1BVNjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDdhMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjc5
Nl0gR0lDdjM6IENQVTY1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGM1MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjgz
MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjg0N10gQ1BV
NjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzMxNDRdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU2NgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMzE2MV0gR0lDdjM6IENQVTY2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE3MDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA3MjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzMxODhdIEdJQ3YzOiBDUFU2NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBjNjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzMyMjNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzMy
MzhdIENQVTY2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNzAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNTQ3XSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjcKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzM1NjVdIEdJQ3YzOiBDUFU2NzogZm91bmQgcmVkaXN0cmlidXRvciAxZjAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwOTIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMzNTkyXSBHSUN2MzogQ1BVNjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzcwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMzNjI3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMzNjQ1XSBDUFU2NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWYwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzkz
OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMzOTUyXSBHSUN2MzogQ1BVNjg6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
YjAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMzOTc5XSBHSUN2MzogQ1BVNjg6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzgwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM0MDE1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM0MDMxXSBDUFU2ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MGIwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzNDMzM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY5ClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0MzUyXSBHSUN2MzogQ1BVNjk6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMjMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEyMDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDM4MF0gR0lDdjM6IENQVTY5OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM5MDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDQxMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNDQyN10gQ1BVNjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDIzMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzQ3MzRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3MApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDc0N10gR0lDdjM6IENQVTcwOiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIGQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRhMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDc3Nl0gR0lDdjM6IENQVTcwOiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNhMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDgxM10gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDgyOV0gQ1BVNzA6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDBkMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzUxMzddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3MQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTE1N10gR0lDdjM6IENQVTcxOiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDI1MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhYTAwMDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUxODVdIEdJQ3YzOiBDUFU3MTog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjYjAwMDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUyMThdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUyMzRdIENQVTcxOiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAyNTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NTM4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
NzIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1NTFdIEdJQ3YzOiBD
UFU3MjogZm91bmQgcmVkaXN0cmlidXRvciA5MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzYTAw
MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1NzldIEdJQ3YzOiBD
UFU3MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjYzAw
MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU2MTddIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU2MzRdIENQVTcyOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1OTM0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNzMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5NTNdIEdJ
Q3YzOiBDUFU3MzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwOWEwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1OTgyXSBH
SUN2MzogQ1BVNzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwY2QwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDE0XSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDMxXSBDUFU3Mzog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMTAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjMyOF0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTc0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2
MzQyXSBHSUN2MzogQ1BVNzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZjAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNTIwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2
MzcyXSBHSUN2MzogQ1BVNzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwY2UwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2
NDA5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2NDI3XSBD
UFU3NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGYwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjc0Ml0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTc1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2NzYzXSBHSUN2MzogQ1BVNzU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjcwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMGIyMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNjc5MV0gR0lDdjM6IENQVTc1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGNmMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNjgyNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
Njg0M10gQ1BVNzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI3MDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzcxMzddIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNzE0N10gR0lDdjM6IENQVTc2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDMwMTAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIyMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNzE3N10gR0lDdjM6IENQVTc2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGQwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNzIxNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzNzIzM10gQ1BVNzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAzMDEw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzc1
MjddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NwpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNzUzOV0gR0lDdjM6IENQVTc3OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJhMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNzU3MF0gR0lDdjM6IENQVTc3OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGQxMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNzYwN10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNzYyNF0gQ1BVNzc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDA1MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzc5MzFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3OApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNzk0MV0gR0lDdjM6IENQVTc4OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFhMDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNzk3Ml0gR0lDdjM6IENQVTc4OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGQyMDAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODAwOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzODAyOF0gQ1BVNzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDAxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzgzMzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3OQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODM0NF0gR0lDdjM6IENQVTc5OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDcwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDMyMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODM3NF0gR0lDdjM6IENQVTc5OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGQzMDAwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODQxM10gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODQzMl0gQ1BVNzk6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDA3MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzg0OTFdIHNtcDogQnJvdWdodCB1cCAxIG5vZGUsIDgwIENQVXMKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzg1MTFdIFNNUDogVG90YWwgb2Yg
ODAgcHJvY2Vzc29ycyBhY3RpdmF0ZWQuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM4NTEyXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBQcml2aWxlZ2VkIEFjY2VzcyBO
ZXZlcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODUxM10gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogTFNFIGF0b21pYyBpbnN0cnVjdGlvbnMKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzg1MTNdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFVz
ZXIgQWNjZXNzIE92ZXJyaWRlClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM4NTE0XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiAzMi1iaXQgRUwwIFN1cHBvcnQKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzg1MTVdIENQVSBmZWF0dXJlczogZGV0
ZWN0ZWQ6IENvbW1vbiBub3QgUHJpdmF0ZSB0cmFuc2xhdGlvbnMKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzg1MTZdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IERhdGEg
Y2FjaGUgY2xlYW4gdG8gUG9pbnQgb2YgUGVyc2lzdGVuY2UKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzg1MTZdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFJBUyBFeHRl
bnNpb24gU3VwcG9ydApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjAzODUx
N10gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogRGF0YSBjYWNoZSBjbGVhbiB0byB0aGUgUG9VIG5v
dCByZXF1aXJlZCBmb3IgSS9EIGNvaGVyZW5jZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzODUxOF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogQ1JDMzIgaW5zdHJ1Y3Rp
b25zClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM4NTE4XSBDUFUgZmVh
dHVyZXM6IGRldGVjdGVkOiBTcGVjdWxhdGl2ZSBTdG9yZSBCeXBhc3NpbmcgU2FmZSAoU1NCUykK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wNjA5NzJdIENQVTogQWxsIENQ
VShzKSBzdGFydGVkIGF0IEVMMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjU4MDIwNF0gbm9kZSAwIGRlZmVycmVkIHBhZ2VzIGluaXRpYWxpc2VkIGluIDUyMG1zClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgwMzcyXSBkZXZ0bXBmczogaW5pdGlh
bGl6ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODA5NDZdIFJlZ2lz
dGVyZWQgY3AxNV9iYXJyaWVyIGVtdWxhdGlvbiBoYW5kbGVyClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNTgwOTY4XSBSZWdpc3RlcmVkIHNldGVuZCBlbXVsYXRpb24gaGFu
ZGxlcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MDk3MF0gS0FTTFIg
ZW5hYmxlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MTAzMF0gY2xv
Y2tzb3VyY2U6IGppZmZpZXM6IG1hc2s6IDB4ZmZmZmZmZmYgbWF4X2N5Y2xlczogMHhmZmZmZmZm
ZiwgbWF4X2lkbGVfbnM6IDc2NDUwNDE3ODUxMDAwMDAgbnMKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC41ODEwODNdIGZ1dGV4IGhhc2ggdGFibGUgZW50cmllczogMzI3Njgg
KG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTgxNTI2XSBwaW5jdHJsIGNvcmU6IGluaXRpYWxpemVkIHBpbmN0cmwg
c3Vic3lzdGVtClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgxNjg1XSBT
TUJJT1MgMy4zLjAgcHJlc2VudC4KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC41ODE2ODhdIERNSTogR0lHQUJZVEUgUjE1Mi1QMzAtMDAvTVAzMi1BUjEtMDAsIEJJT1MgRjEz
IChTQ1A6IDEuNS4yMDIxMDQyNikgMDUvMjEvMjAyMQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjU4MTkyMV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxNgpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MjUxN10gRE1BOiBwcmVhbGxv
Y2F0ZWQgNDA5NiBLaUIgR0ZQX0tFUk5FTCBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlvbnMKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODI2NDJdIERNQTogcHJlYWxsb2Nh
dGVkIDQwOTYgS2lCIEdGUF9LRVJORUx8R0ZQX0RNQSBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlv
bnMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODI3NjddIERNQTogcHJl
YWxsb2NhdGVkIDQwOTYgS2lCIEdGUF9LRVJORUx8R0ZQX0RNQTMyIHBvb2wgZm9yIGF0b21pYyBh
bGxvY2F0aW9ucwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4Mjc3NF0g
YXVkaXQ6IGluaXRpYWxpemluZyBuZXRsaW5rIHN1YnN5cyAoZGlzYWJsZWQpClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgyODE2XSBhdWRpdDogdHlwZT0yMDAwIGF1ZGl0
KDAuNTgwOjEpOiBzdGF0ZT1pbml0aWFsaXplZCBhdWRpdF9lbmFibGVkPTAgcmVzPTEKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODI5MDFdIHRoZXJtYWxfc3lzOiBSZWdp
c3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ2ZhaXJfc2hhcmUnClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNTgyOTAyXSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFs
IGdvdmVybm9yICdiYW5nX2JhbmcnClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNTgyOTAzXSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVybm9yICdzdGVw
X3dpc2UnClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgyOTAzXSB0aGVy
bWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVybm9yICd1c2VyX3NwYWNlJwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MjkwNF0gdGhlcm1hbF9zeXM6IFJlZ2lz
dGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAncG93ZXJfYWxsb2NhdG9yJwpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU4Mjk2NF0gY3B1aWRsZTogdXNpbmcgZ292ZXJub3IgbGFk
ZGVyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgyOTcxXSBjcHVpZGxl
OiB1c2luZyBnb3Zlcm5vciBtZW51ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNTgzMDY0XSBEZXRlY3RlZCAxNSBQQ0MgU3Vic3BhY2VzClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNTgzMDgwXSBSZWdpc3RlcmluZyBQQ0MgZHJpdmVyIGFzIE1haWxi
b3ggY29udHJvbGxlcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MzEw
MF0gaHctYnJlYWtwb2ludDogZm91bmQgNiBicmVha3BvaW50IGFuZCA0IHdhdGNocG9pbnQgcmVn
aXN0ZXJzLgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MzUxNF0gQVNJ
RCBhbGxvY2F0b3IgaW5pdGlhbGlzZWQgd2l0aCAzMjc2OCBlbnRyaWVzClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgzNTM3XSBBQ1BJOiBidXMgdHlwZSBQQ0kgcmVnaXN0
ZXJlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4MzUzOF0gYWNwaXBo
cDogQUNQSSBIb3QgUGx1ZyBQQ0kgQ29udHJvbGxlciBEcml2ZXIgdmVyc2lvbjogMC41ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTgzNTc2XSBTZXJpYWw6IEFNQkEgUEww
MTEgVUFSVCBkcml2ZXIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODU5
MzJdIEh1Z2VUTEIgcmVnaXN0ZXJlZCAxLjAwIEdpQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQg
MCBwYWdlcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjU4NTkzM10gSHVn
ZVRMQiByZWdpc3RlcmVkIDMyLjAgTWlCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2Vz
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTg1OTM0XSBIdWdlVExCIHJl
Z2lzdGVyZWQgMi4wMCBNaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41ODU5MzVdIEh1Z2VUTEIgcmVnaXN0ZXJl
ZCA2NC4wIEtpQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzU2N10gQUNQSTogQWRkZWQgX09TSShNb2R1bGUg
RGV2aWNlKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzU2OF0gQUNQ
STogQWRkZWQgX09TSShQcm9jZXNzb3IgRGV2aWNlKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxMzU2OV0gQUNQSTogQWRkZWQgX09TSSgzLjAgX1NDUCBFeHRlbnNpb25z
KQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzU3MF0gQUNQSTogQWRk
ZWQgX09TSShQcm9jZXNzb3IgQWdncmVnYXRvciBEZXZpY2UpClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzEzNTcxXSBBQ1BJOiBBZGRlZCBfT1NJKExpbnV4LURlbGwtVmlk
ZW8pClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNTczXSBBQ1BJOiBB
ZGRlZCBfT1NJKExpbnV4LUxlbm92by1OVi1IRE1JLUF1ZGlvKQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMzU3NF0gQUNQSTogQWRkZWQgX09TSShMaW51eC1IUEktSHli
cmlkLUdyYXBoaWNzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDUy
Nl0gQUNQSTogMiBBQ1BJIEFNTCB0YWJsZXMgc3VjY2Vzc2Z1bGx5IGFjcXVpcmVkIGFuZCBsb2Fk
ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjE4NzZdIEFDUEk6IElu
dGVycHJldGVyIGVuYWJsZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjE4NzddIEFDUEk6IFVzaW5nIEdJQyBmb3IgaW50ZXJydXB0IHJvdXRpbmcKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjE4ODVdIEFDUEk6IE1DRkcgdGFibGUgZGV0ZWN0
ZWQsIDggZW50cmllcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDYx
Nl0gQVJNSDAwMTE6MDA6IHR0eUFNQTAgYXQgTU1JTyAweDEwMDAwMjYwMDAwMCAoaXJxID0gMzYs
IGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMDY2MV0gcHJpbnRrOiBjb25zb2xlIFt0dHlBTUEwXSBlbmFibGVkClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMxNzYwXSBBUk1IMDAxMTowMTogdHR5QU1B
MSBhdCBNTUlPIDB4MTAwMDAyNjIwMDAwIChpcnEgPSAzNywgYmFzZV9iYXVkID0gMCkgaXMgYSBT
QlNBClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyODIyXSBBQ1BJOiBQ
Q0kgUm9vdCBCcmlkZ2UgW1BDSTBdIChkb21haW4gMDAwYyBbYnVzIDAwLWZmXSkKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI4MjldIGFjcGkgUE5QMEEwODowMDogX09T
QzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kg
SFBYLVR5cGUzXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjg5N10g
YWNwaSBQTlAwQTA4OjAwOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90
cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMjk1Nl0gYWNwaSBQTlAwQTA4OjAwOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FF
UiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzI5NjBdIGFjcGkgUE5QMEEwODowMDogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4MzNmZmYw
MDAwMDAwLTB4MzNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZjN2U3ZTVh
NmVhNTgKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM1ODRdIGFjcGkg
UE5QMEEwODowMDogRUNBTSBhcmVhIFttZW0gMHgzM2ZmZjAwMDAwMDAtMHgzM2ZmZmZmZmZmZmZd
IHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzM1OTJdIGFjcGkgUE5QMEEwODowMDogRUNBTSBhdCBbbWVtIDB4MzNmZmYwMDAwMDAw
LTB4MzNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzM2NTddIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwYzowMApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzY1OF0gcGNpX2J1cyAwMDBjOjAw
OiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NDAwMDAwMDAtMHg0ZmZmZmZmZiB3aW5kb3ddClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzNjU5XSBwY2lfYnVzIDAwMGM6
MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2ZmZGZmZmZmZmYg
d2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzY2MF0gcGNp
X2J1cyAwMDBjOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzY4MF0gcGNpIDAwMGM6MDA6MDAuMDogWzFkZWY6
ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMzcyOF0gcGNpIDAwMGM6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBj
bGFzcyAweDA2MDQwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzcz
Nl0gcGNpIDAwMGM6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzc1Ml0gcGNpIDAwMGM6MDA6MDEuMDogc3VwcG9y
dHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM3NTNdIHBj
aSAwMDBjOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM4MjhdIGFjcGlwaHA6IFNsb3QgWzFdIHJl
Z2lzdGVyZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM4NDJdIHBj
aSAwMDBjOjAxOjAwLjA6IFsxMDAwOjAwOTddIHR5cGUgMDAgY2xhc3MgMHgwMTA3MDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM4NTFdIHBjaSAwMDBjOjAxOjAwLjA6
IHJlZyAweDEwOiBbaW8gIDB4NGZmZjgwMDAtMHg0ZmZmODBmZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzM4NThdIHBjaSAwMDBjOjAxOjAwLjA6IHJlZyAweDE0OiBb
bWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzM4NjZdIHBjaSAwMDBjOjAxOjAwLjA6IHJlZyAweDFjOiBbbWVtIDB4
NDAxMDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzM4NzVdIHBjaSAwMDBjOjAxOjAwLjA6IHJlZyAweDMwOiBbbWVtIDB4NDAwMDAw
MDAtMHg0MDBmZmZmZiBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjczMzg3OV0gcGNpIDAwMGM6MDE6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzk3NV0gcGNpIDAwMGM6MDE6MDAuMDog
c3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQw
NDZdIHBjaV9idXMgMDAwYzowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQwNTBdIHBjaSAwMDBjOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3Np
emUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzQwNTNdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAw
eDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM0MDU1XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzMDAw
MDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQwNTZdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzM0MDU3XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzQwNTldIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0MDU5XSBwY2kg
MDAwYzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQwNjFdIHBjaSAwMDBjOjAx
OjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NDAwMDAwMDAtMHg0MDBmZmZmZiBwcmVmXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDA2Ml0gcGNpIDAwMGM6MDE6
MDAuMDogQkFSIDM6IGFzc2lnbmVkIFttZW0gMHg0MDEwMDAwMC0weDQwMTNmZmZmIDY0Yml0XQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDA2OF0gcGNpIDAwMGM6MDE6
MDAuMDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHg0MDE0MDAwMC0weDQwMTRmZmZmIDY0Yml0XQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDA3M10gcGNpIDAwMGM6MDE6
MDAuMDogQkFSIDA6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMTAwXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDA3NF0gcGNpIDAwMGM6MDE6MDAuMDogQkFSIDA6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDEwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQwNzVdIHBjaSAwMDBjOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8g
W2J1cyAwMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQwNzZdIHBj
aSAwMDBjOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4NDAwMDAwMDAtMHg0MDFmZmZm
Zl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQwNzddIHBjaSAwMDBj
OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0
MDc5XSBwY2lfYnVzIDAwMGM6MDA6IFNvbWUgUENJIGRldmljZSByZXNvdXJjZXMgYXJlIHVuYXNz
aWduZWQsIHRyeSBib290aW5nIHdpdGggcGNpPXJlYWxsb2MKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQwODBdIHBjaV9idXMgMDAwYzowMDogcmVzb3VyY2UgNCBbbWVt
IDB4NDAwMDAwMDAtMHg0ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM0MDgxXSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNlIDUgW21lbSAweDMw
MDAwMDAwMDAwMC0weDMzZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM0MDgyXSBwY2lfYnVzIDAwMGM6MDE6IHJlc291cmNlIDEgW21lbSAw
eDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM0MDgzXSBwY2lfYnVzIDAwMGM6MDE6IHJlc291cmNlIDIgW21lbSAweDMwMDAwMDAwMDAw
MC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNDE4OV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kxXSAoZG9tYWluIDAw
MGQgW2J1cyAwMC1mZl0pClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0
MTk0XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBB
U1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQyNTddIGFjcGkgUE5QMEEwODowMTogX09TQzogcGxhdGZvcm0g
ZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQzMTVdIGFjcGkgUE5QMEEwODowMTog
X09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0MzE4XSBhY3BpIFBOUDBBMDg6MDE6IE1DRkcgcXVp
cms6IEVDQU0gYXQgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZmZmZmZl0gZm9yIFtidXMg
MDAtZmZdIHdpdGggMHhmZmZmYzdlN2U1YTZlYTU4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM0OTQ0XSBhY3BpIFBOUDBBMDg6MDE6IEVDQU0gYXJlYSBbbWVtIDB4Mzdm
ZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0OTUxXSBhY3BpIFBOUDBBMDg6MDE6IEVD
QU0gYXQgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZd
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1MDEwXSBQQ0kgaG9zdCBi
cmlkZ2UgdG8gYnVzIDAwMGQ6MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzUwMTJdIHBjaV9idXMgMDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDUwMDAw
MDAwLTB4NWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNTAxMl0gcGNpX2J1cyAwMDBkOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzQw
MDAwMDAwMDAwLTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzUwMTNdIHBjaV9idXMgMDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W2J1cyAwMC1mZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUwMzBd
IHBjaSAwMDBkOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUwNjldIHBjaSAwMDBkOjAwOjAx
LjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzUwNzddIHBjaSAwMDBkOjAwOjAxLjA6IGVuYWJsaW5nIEV4
dGVuZGVkIFRhZ3MKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUwOTJd
IHBjaSAwMDBkOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM1MDkzXSBwY2kgMDAwZDowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1
MTQ0XSBwY2kgMDAwZDowMDowMi4wOiBbMWRlZjplMTAyXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1MTY0XSBwY2kgMDAwZDow
MDowMi4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNTE2NV0gcGNpIDAwMGQ6MDA6MDIuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBE
M2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTIxMl0gcGNpIDAw
MGQ6MDA6MDMuMDogWzFkZWY6ZTEwM10gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTIzMl0gcGNpIDAwMGQ6MDA6MDMuMDogc3Vw
cG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUyMzNd
IHBjaSAwMDBkOjAwOjAzLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUyNzhdIHBjaSAwMDBkOjAwOjA0LjA6
IFsxZGVmOmUxMDRdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzUyOTldIHBjaSAwMDBkOjAwOjA0LjA6IHN1cHBvcnRzIEQxIEQy
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1MzAwXSBwY2kgMDAwZDow
MDowNC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1MzY3XSBhY3BpcGhwOiBTbG90IFsxLTJdIHJlZ2lzdGVy
ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUzODNdIHBjaSAwMDBk
OjAxOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzUzOTVdIHBjaSAwMDBkOjAxOjAwLjA6IHJlZyAw
eDEwOiBbbWVtIDB4NTAzMTAwMDAtMHg1MDMxM2ZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzU0MThdIHBjaSAwMDBkOjAxOjAwLjA6IHJlZyAweDMwOiBb
bWVtIDB4NTAzMDAwMDAtMHg1MDMwZmZmZiBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNTQyMl0gcGNpIDAwMGQ6MDE6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQg
VGFncwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTU3MV0gYWNwaXBo
cDogU2xvdCBbMl0gcmVnaXN0ZXJlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNTU4Nl0gcGNpIDAwMGQ6MDI6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAw
eDAxMDgwMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTU5Nl0gcGNp
IDAwMGQ6MDI6MDAuMDogcmVnIDB4MTA6IFttZW0gMHg1MDIxMDAwMC0weDUwMjEzZmZmIDY0Yml0
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTYxOF0gcGNpIDAwMGQ6
MDI6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1NjIxXSBwY2kgMDAwZDowMjowMC4w
OiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzM1NzYzXSBhY3BpcGhwOiBTbG90IFszXSByZWdpc3RlcmVkClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1Nzc4XSBwY2kgMDAwZDowMzowMC4wOiBbODA4Njow
YTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM1Nzg5XSBwY2kgMDAwZDowMzowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMTEw
MDAwLTB4NTAxMTNmZmYgNjRiaXRdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM1ODEwXSBwY2kgMDAwZDowMzowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMTAwMDAwLTB4
NTAxMGZmZmYgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU4
MTRdIHBjaSAwMDBkOjAzOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NTddIGFjcGlwaHA6IFNsb3QgWzRdIHJlZ2lz
dGVyZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NzFdIHBjaSAw
MDBkOjA0OjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5ODJdIHBjaSAwMDBkOjA0OjAwLjA6IHJl
ZyAweDEwOiBbbWVtIDB4NTAwMTAwMDAtMHg1MDAxM2ZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYwMDNdIHBjaSAwMDBkOjA0OjAwLjA6IHJlZyAweDMw
OiBbbWVtIDB4NTAwMDAwMDAtMHg1MDAwZmZmZiBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNjAwN10gcGNpIDAwMGQ6MDQ6MDAuMDogZW5hYmxpbmcgRXh0ZW5k
ZWQgVGFncwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjEyOV0gcGNp
X2J1cyAwMDBkOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNjEzMV0gcGNpIDAwMGQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4
MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzYxMzNdIHBjaSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzYxMzRdIHBjaSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAw
eDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTM1XSBw
Y2kgMDAwZDowMDowMi4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1
cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNjEzNl0gcGNpIDAwMGQ6MDA6MDIuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAt
MHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2Fs
aWduIDEwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjEzOF0g
cGNpIDAwMGQ6MDA6MDIuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDFmZmZm
Zl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYxMzldIHBjaSAwMDBkOjAwOjAzLjA6IGJy
aWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTQwXSBwY2kgMDAwZDow
MDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHBy
ZWZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTQxXSBwY2kgMDAwZDowMDowMy4wOiBi
cmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAzXSBhZGRf
c2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNjE0Ml0gcGNpIDAwMGQ6MDA6MDQuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4
MTAwMC0weDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzYxNDNdIHBjaSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwNF0gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzYxNDRdIHBjaSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAw
eDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTQ4XSBw
Y2kgMDAwZDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMWZm
ZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjE0OV0gcGNpIDAw
MGQ6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM2MTUwXSBwY2kgMDAwZDowMDowMi4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg1MDIwMDAw
MC0weDUwM2ZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjE1
MV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzQwMDAwMjAwMDAw
LTB4MzQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM2MTUyXSBwY2kgMDAwZDowMDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0g
MHg1MDQwMDAwMC0weDUwNWZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNjE1NF0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzQw
MDAwNDAwMDAwLTB4MzQwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM2MTU0XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTQ6IGFzc2ln
bmVkIFttZW0gMHg1MDYwMDAwMC0weDUwN2ZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNjE1Nl0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE1OiBhc3NpZ25lZCBb
bWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTU3XSBwY2kgMDAwZDowMDowMS4wOiBCQVIg
MTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNjE1N10gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQg
dG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM2MTU4XSBwY2kgMDAwZDowMDowMi4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
NjE1OV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTYwXSBw
Y2kgMDAwZDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjE2MV0gcGNpIDAwMGQ6MDA6
MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTYyXSBwY2kgMDAwZDowMDowNC4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjE2M10gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM2MTY1XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNjE2Nl0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTY3
XSBwY2kgMDAwZDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjE2OF0gcGNpIDAwMGQ6
MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTY4XSBwY2kgMDAwZDowMDowMi4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjE2OV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM2MTcwXSBwY2kgMDAwZDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNjE3MV0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2
MTcyXSBwY2kgMDAwZDowMTowMC4wOiBCQVIgNjogYXNzaWduZWQgW21lbSAweDUwMDAwMDAwLTB4
NTAwMGZmZmYgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYx
NzRdIHBjaSAwMDBkOjAxOjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVtIDB4NTAwMTAwMDAtMHg1
MDAxM2ZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYx
NzldIHBjaSAwMDBkOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYxODBdIHBjaSAwMDBkOjAwOjAxLjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzYxODFdIHBjaSAwMDBkOjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MTgzXSBwY2kgMDAwZDowMjowMC4w
OiBCQVIgNjogYXNzaWduZWQgW21lbSAweDUwMjAwMDAwLTB4NTAyMGZmZmYgcHJlZl0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYxODRdIHBjaSAwMDBkOjAyOjAwLjA6
IEJBUiAwOiBhc3NpZ25lZCBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYxODldIHBjaSAwMDBkOjAwOjAyLjA6
IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzYxOTBdIHBjaSAwMDBkOjAwOjAyLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4NTAy
MDAwMDAtMHg1MDNmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzYxOTFdIHBjaSAwMDBkOjAwOjAyLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzQwMDAwMjAw
MDAwLTB4MzQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM2MTkzXSBwY2kgMDAwZDowMzowMC4wOiBCQVIgNjogYXNzaWduZWQgW21l
bSAweDUwNDAwMDAwLTB4NTA0MGZmZmYgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzYxOTRdIHBjaSAwMDBkOjAzOjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVt
IDB4NTA0MTAwMDAtMHg1MDQxM2ZmZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzYxOTldIHBjaSAwMDBkOjAwOjAzLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAw
M10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMDBdIHBjaSAwMDBk
OjAwOjAzLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4NTA0MDAwMDAtMHg1MDVmZmZmZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMDFdIHBjaSAwMDBkOjAwOjAz
LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzQwMDAwNDAwMDAwLTB4MzQwMDAwNWZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MjAzXSBw
Y2kgMDAwZDowNDowMC4wOiBCQVIgNjogYXNzaWduZWQgW21lbSAweDUwNjAwMDAwLTB4NTA2MGZm
ZmYgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMDRdIHBj
aSAwMDBkOjA0OjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVtIDB4NTA2MTAwMDAtMHg1MDYxM2Zm
ZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMDhdIHBj
aSAwMDBkOjAwOjA0LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwNF0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMTBdIHBjaSAwMDBkOjAwOjA0LjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzYyMTFdIHBjaSAwMDBkOjAwOjA0LjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MjEyXSBwY2lfYnVzIDAwMGQ6MDA6IHJlc291
cmNlIDQgW21lbSAweDUwMDAwMDAwLTB4NWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjIxM10gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA1
IFttZW0gMHgzNDAwMDAwMDAwMDAtMHgzN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjIxNF0gcGNpX2J1cyAwMDBkOjAxOiByZXNvdXJj
ZSAxIFttZW0gMHg1MDAwMDAwMC0weDUwMWZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNjIxNV0gcGNpX2J1cyAwMDBkOjAxOiByZXNvdXJjZSAyIFttZW0gMHgz
NDAwMDAwMDAwMDAtMHgzNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzYyMTZdIHBjaV9idXMgMDAwZDowMjogcmVzb3VyY2UgMSBb
bWVtIDB4NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzYyMTddIHBjaV9idXMgMDAwZDowMjogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAw
MjAwMDAwLTB4MzQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM2MjE4XSBwY2lfYnVzIDAwMGQ6MDM6IHJlc291cmNlIDEgW21lbSAw
eDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM2MjE4XSBwY2lfYnVzIDAwMGQ6MDM6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDQwMDAw
MC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNjIxOV0gcGNpX2J1cyAwMDBkOjA0OiByZXNvdXJjZSAxIFttZW0gMHg1MDYw
MDAwMC0weDUwN2ZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
NjIyMF0gcGNpX2J1cyAwMDBkOjA0OiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDA2MDAwMDAtMHgz
NDAwMDA3ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzYzMzNdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJMl0gKGRvbWFpbiAwMDAxIFti
dXMgMDAtZmZdKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjMzOF0g
YWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBD
bG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM2NDAyXSBhY3BpIFBOUDBBMDg6MDI6IF9PU0M6IHBsYXRmb3JtIGRvZXMg
bm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDYwXSBhY3BpIFBOUDBBMDg6MDI6IF9PU0M6
IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjQ2Ml0gYWNwaSBQTlAwQTA4OjAyOiBNQ0ZHIHF1aXJrOiBF
Q0FNIGF0IFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZm
XSB3aXRoIDB4ZmZmZmM3ZTdlNWE2ZWE1OApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNzA4OF0gYWNwaSBQTlAwQTA4OjAyOiBFQ0FNIGFyZWEgW21lbSAweDNiZmZmMDAw
MDAwMC0weDNiZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzA5NF0gYWNwaSBQTlAwQTA4OjAyOiBFQ0FNIGF0
IFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzE1NF0gUENJIGhvc3QgYnJpZGdl
IHRvIGJ1cyAwMDAxOjAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3
MTU1XSBwY2lfYnVzIDAwMDE6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHg2MDAwMDAwMC0w
eDZmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzcxNTZdIHBjaV9idXMgMDAwMTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDM4MDAwMDAw
MDAwMC0weDNiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM3MTU3XSBwY2lfYnVzIDAwMDE6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMg
MDAtZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MTc0XSBwY2kg
MDAwMTowMDowMC4wOiBbMWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MjE3XSBwY2kgMDAwMTowMDowMS4wOiBb
MWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM3MjMwXSBwY2kgMDAwMTowMDowMS4wOiBlbmFibGluZyBFeHRlbmRl
ZCBUYWdzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MjU3XSBwY2kg
MDAwMTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNzI1OF0gcGNpIDAwMDE6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzM0Nl0g
YWNwaXBocDogU2xvdCBbMS0zXSByZWdpc3RlcmVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM5NDM2XSBwY2lfYnVzIDAwMDE6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NDM3XSBwY2kgMDAwMTowMDowMS4w
OiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUg
MTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTQzOV0gcGNpIDAw
MDE6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJp
dCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTQ0MF0gcGNpIDAwMDE6MDA6MDEu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMV0g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43Mzk0NDJdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDYwMDAwMDAwLTB4NjAxZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM5NDQzXSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgzODAwMDAwMDAwMDAtMHgzODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk0NDRdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM5NDQ1XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mzk0NDZdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NDQ3
XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk0NDhdIHBjaSAw
MDAxOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mzk0NTBdIHBjaSAwMDAxOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mzk0NTFdIHBjaSAwMDAxOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NDUzXSBwY2lfYnVzIDAwMDE6MDA6IHJlc291cmNl
IDQgW21lbSAweDYwMDAwMDAwLTB4NmZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczOTQ1NF0gcGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA1IFtt
ZW0gMHgzODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTQ1NV0gcGNpX2J1cyAwMDAxOjAxOiByZXNvdXJjZSAx
IFttZW0gMHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjczOTQ1Nl0gcGNpX2J1cyAwMDAxOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzODAw
MDAwMDAwMDAtMHgzODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mzk1NjFdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJM10gKGRv
bWFpbiAwMDAwIFtidXMgMDAtZmZdKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczOTU2NV0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRD
b25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5NjI5XSBhY3BpIFBOUDBBMDg6MDM6IF9PU0M6IHBs
YXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJd
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM5Njg3XSBhY3BpIFBOUDBB
MDg6MDM6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTY4OV0gYWNwaSBQTlAwQTA4OjAzOiBN
Q0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmZdIGZv
ciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmM3ZTdlNWE2ZWE1OApTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0MDMxM10gYWNwaSBQTlAwQTA4OjAzOiBFQ0FNIGFyZWEgW21l
bSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDMxOV0gYWNwaSBQTlAwQTA4
OjAzOiBFQ0FNIGF0IFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmZdIGZvciBbYnVz
IDAwLWZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDM3OV0gUENJ
IGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDAwOjAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQwMzgwXSBwY2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0g
MHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDAzODFdIHBjaV9idXMgMDAwMDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21l
bSAweDNjMDAwMDAwMDAwMC0weDNmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwMzgzXSBwY2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJl
c291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQwMzk5XSBwY2kgMDAwMDowMDowMC4wOiBbMWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYw
MDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwNDQxXSBwY2kgMDAw
MDowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwNDU0XSBwY2kgMDAwMDowMDowMS4wOiBlbmFi
bGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQwNDgxXSBwY2kgMDAwMDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDQ4Ml0gcGNpIDAwMDA6MDA6MDEuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0MDU2OV0gYWNwaXBocDogU2xvdCBbMS00XSByZWdpc3RlcmVkClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjU5XSBwY2lfYnVzIDAwMDA6MDA6IG9uIE5VTUEg
bm9kZSAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjYxXSBwY2kg
MDAwMDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAw
MV0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
MjY2Ml0gcGNpIDAwMDA6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgw
MDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWdu
IDEwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjY2M10gcGNp
IDAwMDA6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0g
dG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI2NjVdIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAx
NDogYXNzaWduZWQgW21lbSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjY2XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTU6IGFz
c2lnbmVkIFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI2NjddIHBjaSAwMDAwOjAwOjAx
LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjY4XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDI2NjldIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQyNjcwXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDI2NzFdIHBjaSAwMDAwOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI2NzNdIHBjaSAwMDAwOjAwOjAxLjA6ICAg
YnJpZGdlIHdpbmRvdyBbbWVtIDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI2NzRdIHBjaSAwMDAwOjAwOjAxLjA6ICAgYnJpZGdl
IHdpbmRvdyBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjc2XSBwY2lfYnVzIDAwMDA6
MDA6IHJlc291cmNlIDQgW21lbSAweDcwMDAwMDAwLTB4N2ZmZmZmZmYgd2luZG93XQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjY3N10gcGNpX2J1cyAwMDAwOjAwOiBy
ZXNvdXJjZSA1IFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzZmZmZGZmZmZmZmYgd2luZG93XQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjY3OF0gcGNpX2J1cyAwMDAwOjAx
OiByZXNvdXJjZSAxIFttZW0gMHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjY3OV0gcGNpX2J1cyAwMDAwOjAxOiByZXNvdXJjZSAy
IFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI3ODJdIEFDUEk6IFBDSSBSb290IEJyaWRn
ZSBbUENJNF0gKGRvbWFpbiAwMDAyIFtidXMgMDAtZmZdKQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0Mjc4Nl0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBPUyBzdXBwb3J0
cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyODQ5XSBhY3BpIFBOUDBBMDg6
MDQ6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3Rw
bHVnIFBNRSBMVFJdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyOTA3
XSBhY3BpIFBOUDBBMDg6MDQ6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmls
aXR5XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjkwOV0gYWNwaSBQ
TlAwQTA4OjA0OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2Zm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmM3ZTdlNWE2ZWE1OApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzUzMl0gYWNwaSBQTlAwQTA4OjA0OiBF
Q0FNIGFyZWEgW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkg
UE5QMEMwMjowMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzUzOV0g
YWNwaSBQTlAwQTA4OjA0OiBFQ0FNIGF0IFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZm
ZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0MzU5OF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDAyOjAwClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzNTk5XSBwY2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJl
c291cmNlIFttZW0gMHgwMDgwMDAwMC0weDBmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM2MDBdIHBjaV9idXMgMDAwMjowMDogcm9vdCBidXMg
cmVzb3VyY2UgW21lbSAweDIwMDAwMDAwMDAwMC0weDIzZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzNjAxXSBwY2lfYnVzIDAwMDI6MDA6
IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQzNjE3XSBwY2kgMDAwMjowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAw
IGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQz
NjYwXSBwY2kgMDAwMjowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAw
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzNjk1XSBwY2kgMDAwMjow
MDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0MzY5Nl0gcGNpIDAwMDI6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBE
M2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0Mzc1N10gcGNpIDAw
MDI6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0Mzc5M10gcGNpIDAwMDI6MDA6MDUuMDogc3Vw
cG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM3OTNd
IHBjaSAwMDAyOjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM4NzVdIGFjcGlwaHA6IFNsb3QgWzEt
NV0gcmVnaXN0ZXJlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTk4
OF0gYWNwaXBocDogU2xvdCBbMy0yXSByZWdpc3RlcmVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ4MDgxXSBwY2lfYnVzIDAwMDI6MDA6IG9uIE5VTUEgbm9kZSAwClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4MDgyXSBwY2kgMDAwMjowMDow
MS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3Np
emUgMTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODA4M10gcGNp
IDAwMDI6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2
NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODA4NF0gcGNpIDAwMDI6MDA6
MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAw
MV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDgwODVdIHBjaSAwMDAyOjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cg
W2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4MDg2XSBwY2kgMDAwMjowMDowNS4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMg
MDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ4MDg3XSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ugd2luZG93
IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
ODA4OV0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MDA4MDAwMDAt
MHgwMDlmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDgwOTFd
IHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDIwMDAwMDAwMDAwMC0w
eDIwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc0ODA5Ml0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4
MDBhMDAwMDAtMHgwMGJmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDgwOTNdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDIwMDAw
MDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0ODA5NF0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDgwOTVdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
ODA5Nl0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDgwOTZdIHBjaSAw
MDAyOjAwOjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODA5OF0gcGNpIDAwMDI6MDA6
MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDgwOTldIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAx
MzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0ODEwMF0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDgxMDFdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0ODEwMV0gcGNpIDAwMDI6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODEwM10gcGNpIDAwMDI6MDA6MDEuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDgwMDAwMC0weDAwOWZmZmZmXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODEwNV0gcGNpIDAwMDI6MDA6MDEuMDogICBicmlk
Z2Ugd2luZG93IFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDgxMDddIHBjaSAwMDAyOjAw
OjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDgxMDhdIHBjaSAwMDAyOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDBhMDAwMDAtMHgwMGJmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDgxMTBdIHBjaSAwMDAyOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAw
MDAwMjAwMDAwLTB4MjAwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ4MTEyXSBwY2lfYnVzIDAwMDI6MDA6IHJlc291cmNlIDQgW21l
bSAweDAwODAwMDAwLTB4MGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0ODExMl0gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJjZSA1IFttZW0gMHgy
MDAwMDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0ODExM10gcGNpX2J1cyAwMDAyOjAxOiByZXNvdXJjZSAxIFttZW0g
MHgwMDgwMDAwMC0weDAwOWZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0ODExNF0gcGNpX2J1cyAwMDAyOjAxOiByZXNvdXJjZSAyIFttZW0gMHgyMDAwMDAwMDAw
MDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDgxMTVdIHBjaV9idXMgMDAwMjowMjogcmVzb3VyY2UgMSBbbWVtIDB4MDBh
MDAwMDAtMHgwMGJmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDgxMTZdIHBjaV9idXMgMDAwMjowMjogcmVzb3VyY2UgMiBbbWVtIDB4MjAwMDAwMjAwMDAwLTB4
MjAwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ4MjIxXSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTVdIChkb21haW4gMDAwMyBb
YnVzIDAwLWZmXSkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDgyMjZd
IGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0g
Q2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0ODI4OF0gYWNwaSBQTlAwQTA4OjA1OiBfT1NDOiBwbGF0Zm9ybSBkb2Vz
IG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODM0N10gYWNwaSBQTlAwQTA4OjA1OiBfT1ND
OiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDgzNDldIGFjcGkgUE5QMEEwODowNTogTUNGRyBxdWlyazog
RUNBTSBhdCBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1m
Zl0gd2l0aCAweGZmZmZjN2U3ZTVhNmVhNTgKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDg5NzVdIGFjcGkgUE5QMEEwODowNTogRUNBTSBhcmVhIFttZW0gMHgyN2ZmZjAw
MDAwMDAtMHgyN2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg5ODJdIGFjcGkgUE5QMEEwODowNTogRUNBTSBh
dCBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkwMzVdIFBDSSBob3N0IGJyaWRn
ZSB0byBidXMgMDAwMzowMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
OTAzNl0gcGNpX2J1cyAwMDAzOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MTAwMDAwMDAt
MHgxZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ5MDM3XSBwY2lfYnVzIDAwMDM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyNDAwMDAw
MDAwMDAtMHgyN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjc0OTAzOF0gcGNpX2J1cyAwMDAzOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVz
IDAwLWZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTA1NV0gcGNp
IDAwMDM6MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTA5NF0gcGNpIDAwMDM6MDA6MDEuMDog
WzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0OTEzMF0gcGNpIDAwMDM6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDkxMzFdIHBjaSAwMDAzOjAw
OjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDkxODddIHBjaSAwMDAzOjAwOjA1LjA6IFsxZGVmOmUxMTVd
IHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDkyMTRdIHBjaSAwMDAzOjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5MjE1XSBwY2kgMDAwMzowMDowNS4wOiBQTUUj
IHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ5MjgyXSBhY3BpcGhwOiBTbG90IFsxLTZdIHJlZ2lzdGVyZWQKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE0MDVdIHBjaSAwMDAzOjAyOjAwLjA6IFs4
MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTE0MjBdIHBjaSAwMDAzOjAyOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4
MTAwMjAwMDAtMHgxMDAzZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NTE0MzZdIHBjaSAwMDAzOjAyOjAwLjA6IHJlZyAweDE4OiBbaW8gIDB4MWZmZjgwMjAtMHgx
ZmZmODAzZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE0NDVdIHBj
aSAwMDAzOjAyOjAwLjA6IHJlZyAweDFjOiBbbWVtIDB4MTAwNDQwMDAtMHgxMDA0N2ZmZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE1NTZdIHBjaSAwMDAzOjAyOjAw
LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUxNTkzXSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxODQ6
IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUxNTk1XSBwY2kgMDAwMzowMjowMC4wOiBWRihuKSBCQVIw
IHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMg
QkFSMCBmb3IgOCBWRnMpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUx
NjE2XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxOTA6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAz
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUx
NjE3XSBwY2kgMDAwMzowMjowMC4wOiBWRihuKSBCQVIzIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAt
MHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMyBmb3IgOCBWRnMpClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUxNzEzXSBwY2kgMDAwMzowMjowMC4wOiA4
LjAwMCBHYi9zIGF2YWlsYWJsZSBQQ0llIGJhbmR3aWR0aCwgbGltaXRlZCBieSA1LjAgR1QvcyBQ
Q0llIHgyIGxpbmsgYXQgMDAwMzowMDowNS4wIChjYXBhYmxlIG9mIDE2LjAwMCBHYi9zIHdpdGgg
NS4wIEdUL3MgUENJZSB4NCBsaW5rKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc1MTc2MF0gcGNpIDAwMDM6MDI6MDAuMTogWzgwODY6MTUyMV0gdHlwZSAwMCBjbGFzcyAw
eDAyMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MTc3Nl0gcGNp
IDAwMDM6MDI6MDAuMTogcmVnIDB4MTA6IFttZW0gMHgxMDAwMDAwMC0weDEwMDFmZmZmXQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MTc5Ml0gcGNpIDAwMDM6MDI6MDAu
MTogcmVnIDB4MTg6IFtpbyAgMHgxZmZmODAwMC0weDFmZmY4MDFmXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MTgwMV0gcGNpIDAwMDM6MDI6MDAuMTogcmVnIDB4MWM6
IFttZW0gMHgxMDA0MDAwMC0weDEwMDQzZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1MTkwN10gcGNpIDAwMDM6MDI6MDAuMTogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEM2hvdCBEM2NvbGQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE5
NDBdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDE4NDogW21lbSAweDAwMDAwMDAwLTB4MDAwMDNm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE5
NDFdIHBjaSAwMDAzOjAyOjAwLjE6IFZGKG4pIEJBUjAgc3BhY2U6IFttZW0gMHgwMDAwMDAwMC0w
eDAwMDFmZmZmIDY0Yml0IHByZWZdIChjb250YWlucyBCQVIwIGZvciA4IFZGcykKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE5NjJdIHBjaSAwMDAzOjAyOjAwLjE6IHJl
ZyAweDE5MDogW21lbSAweDAwMDAwMDAwLTB4MDAwMDNmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTE5NjNdIHBjaSAwMDAzOjAyOjAwLjE6IFZG
KG4pIEJBUjMgc3BhY2U6IFttZW0gMHgwMDAwMDAwMC0weDAwMDFmZmZmIDY0Yml0IHByZWZdIChj
b250YWlucyBCQVIzIGZvciA4IFZGcykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTIwODFdIHBjaV9idXMgMDAwMzowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIwODNdIHBjaV9idXMgMDAwMzowMDogbWF4IGJ1
cyBkZXB0aDogMSBwY2lfdHJ5X251bTogMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MjA4Nl0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4
MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NTIwODddIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDI0MDAw
MDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc1MjA4OF0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDE0OiBhc3NpZ25l
ZCBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTIwODldIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNTogYXNzaWduZWQgW21l
bSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA5MF0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDEz
OiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTIwOTFdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRv
IGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MjA5Ml0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIw
OTNdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjA5NF0gcGNp
IDAwMDM6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1MjA5Nl0gcGNpIDAwMDM6MDA6MDEuMDogICBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgxMDAwMDAwMC0weDEwMWZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1MjA5N10gcGNpIDAwMDM6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMDBdIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAw
OiBhc3NpZ25lZCBbbWVtIDB4MTAyMDAwMDAtMHgxMDIxZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMDNdIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAwOiBhc3Np
Z25lZCBbbWVtIDB4MTAyMjAwMDAtMHgxMDIzZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTIxMDddIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAzOiBhc3NpZ25lZCBb
bWVtIDB4MTAyNDAwMDAtMHgxMDI0M2ZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTIxMTBdIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiA3OiBhc3NpZ25lZCBbbWVtIDB4
MjQwMDAwMjAwMDAwLTB4MjQwMDAwMjFmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTE0XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMTA6IGFz
c2lnbmVkIFttZW0gMHgyNDAwMDAyMjAwMDAtMHgyNDAwMDAyM2ZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMThdIHBjaSAwMDAzOjAyOjAw
LjE6IEJBUiAzOiBhc3NpZ25lZCBbbWVtIDB4MTAyNDQwMDAtMHgxMDI0N2ZmZl0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMjFdIHBjaSAwMDAzOjAyOjAwLjE6IEJB
UiA3OiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjQwMDAwLTB4MjQwMDAwMjVmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTI2XSBwY2kgMDAw
MzowMjowMC4xOiBCQVIgMTA6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyNjAwMDAtMHgyNDAwMDAy
N2ZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTIxMzBdIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MDAyMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxMzFdIHBjaSAw
MDAzOjAyOjAwLjA6IEJBUiAyOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwMjBdClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTMyXSBwY2kgMDAwMzowMjow
MC4xOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMTMyXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMjog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc1MjEzM10gcGNpIDAwMDM6MDA6MDUuMDogUENJIGJyaWRnZSB0byBb
YnVzIDAyXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjEzNV0gcGNp
IDAwMDM6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZm
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjEzNl0gcGNpIDAwMDM6
MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIx
MzhdIHBjaV9idXMgMDAwMzowMDogTm8uIDIgdHJ5IHRvIGFzc2lnbiB1bmFzc2lnbmVkIHJlcwpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE0MF0gcGNpIDAwMDM6MDA6
MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxNDBdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAx
MzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1MjE0MV0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDEzOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTIxNDJdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc1MjE0M10gcGNpIDAwMDM6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE0NV0gcGNpIDAwMDM6MDA6MDEuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgxMDAwMDAwMC0weDEwMWZmZmZmXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE0Nl0gcGNpIDAwMDM6MDA6MDEuMDogICBicmlk
Z2Ugd2luZG93IFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxNDhdIHBjaSAwMDAzOjAy
OjAwLjA6IEJBUiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxNDldIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAy
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzUyMTUwXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMjogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzUyMTUxXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
MjE1MV0gcGNpIDAwMDM6MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE1M10gcGNpIDAwMDM6MDA6MDUuMDogICBi
cmlkZ2Ugd2luZG93IFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MjE1NF0gcGNpIDAwMDM6MDA6MDUuMDogICBicmlkZ2Ug
d2luZG93IFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIxNTZdIHBjaV9idXMgMDAwMzow
MDogQXV0b21hdGljYWxseSBlbmFibGVkIHBjaSByZWFsbG9jLCBpZiB5b3UgaGF2ZSBwcm9ibGVt
LCB0cnkgYm9vdGluZyB3aXRoIHBjaT1yZWFsbG9jPW9mZgpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc1MjE1N10gcGNpX2J1cyAwMDAzOjAwOiByZXNvdXJjZSA0IFttZW0g
MHgxMDAwMDAwMC0weDFmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTIxNThdIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjQw
MDAwMDAwMDAwLTB4MjdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTIxNThdIHBjaV9idXMgMDAwMzowMTogcmVzb3VyY2UgMSBbbWVtIDB4
MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NTIxNjBdIHBjaV9idXMgMDAwMzowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjQwMDAwMDAwMDAw
LTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzUyMTYxXSBwY2lfYnVzIDAwMDM6MDI6IHJlc291cmNlIDEgW21lbSAweDEwMjAw
MDAwLTB4MTAzZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUy
MTYxXSBwY2lfYnVzIDAwMDM6MDI6IHJlc291cmNlIDIgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0
MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc1MjIyOF0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0k2XSAoZG9tYWluIDAwMDQgW2J1
cyAwMC1mZl0pClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyMjMyXSBh
Y3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENs
b2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTIyOTZdIGFjcGkgUE5QMEEwODowNjogX09TQzogcGxhdGZvcm0gZG9lcyBu
b3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTIzNTVdIGFjcGkgUE5QMEEwODowNjogX09TQzog
T1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzUyMzU3XSBhY3BpIFBOUDBBMDg6MDY6IE1DRkcgcXVpcms6IEVD
QU0gYXQgW21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZd
IHdpdGggMHhmZmZmYzdlN2U1YTZlYTU4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzUyOTgzXSBhY3BpIFBOUDBBMDg6MDY6IEVDQU0gYXJlYSBbbWVtIDB4MmJmZmYwMDAw
MDAwLTB4MmJmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUyOTg5XSBhY3BpIFBOUDBBMDg6MDY6IEVDQU0gYXQg
W21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzMDQwXSBQQ0kgaG9zdCBicmlkZ2Ug
dG8gYnVzIDAwMDQ6MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMw
NDJdIHBjaV9idXMgMDAwNDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDIwMDAwMDAwLTB4
MmZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
MzA0Ml0gcGNpX2J1cyAwMDA0OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjgwMDAwMDAw
MDAwLTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTMwNDNdIHBjaV9idXMgMDAwNDowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAw
MC1mZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMwNjBdIHBjaSAw
MDA0OjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMwOThdIHBjaSAwMDA0OjAwOjAxLjA6IFsx
ZGVmOmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTMxMjRdIHBjaSAwMDA0OjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzMTI1XSBwY2kgMDAwNDowMDow
MS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzUzMTY3XSBwY2kgMDAwNDowMDowMy4wOiBbMWRlZjplMTEzXSB0
eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzUzMTk0XSBwY2kgMDAwNDowMDowMy4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzE5NV0gcGNpIDAwMDQ6MDA6MDMuMDogUE1FIyBz
dXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1MzI0MF0gcGNpIDAwMDQ6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFz
cyAweDA2MDQwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzI3Nl0g
cGNpIDAwMDQ6MDA6MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTMyNzZdIHBjaSAwMDA0OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZy
b20gRDAgRDEgRDNob3QKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMz
NDldIHBjaSAwMDA0OjAxOjAwLjA6IFsxYTAzOjExNTBdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTMzOTddIHBjaSAwMDA0OjAx
OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTM0NTNdIHBjaSAwMDA0OjAxOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzNDU0XSBwY2kgMDAwNDowMTowMC4w
OiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90IEQzY29sZApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzU0OV0gcGNpX2J1cyAwMDA0OjAyOiBleHRlbmRl
ZCBjb25maWcgc3BhY2Ugbm90IGFjY2Vzc2libGUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTM1NjZdIHBjaSAwMDA0OjAyOjAwLjA6IFsxYTAzOjIwMDBdIHR5cGUgMDAg
Y2xhc3MgMHgwMzAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTM1
ODVdIHBjaSAwMDA0OjAyOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4MjAwMDAwMDAtMHgyMWZmZmZm
Zl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTM1OTVdIHBjaSAwMDA0
OjAyOjAwLjA6IHJlZyAweDE0OiBbbWVtIDB4MjIwMDAwMDAtMHgyMjAxZmZmZl0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTM2MDVdIHBjaSAwMDA0OjAyOjAwLjA6IHJl
ZyAweDE4OiBbaW8gIDB4MmZmZjgwMDAtMHgyZmZmODA3Zl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NTM2NDldIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAwOiBhc3NpZ25l
ZCB0byBlZmlmYgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzY4NV0g
cGNpIDAwMDQ6MDI6MDAuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTM2ODZdIHBjaSAwMDA0OjAyOjAwLjA6IFBNRSMgc3VwcG9ydGVkIGZy
b20gRDAgRDEgRDIgRDNob3QgRDNjb2xkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzUzNzk4XSBwY2kgMDAwNDowMzowMC4wOiBbMTkxMjowMDE0XSB0eXBlIDAwIGNsYXNz
IDB4MGMwMzMwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzODE5XSBw
Y2kgMDAwNDowMzowMC4wOiByZWcgMHgxMDogW21lbSAweDIyMjAwMDAwLTB4MjIyMDFmZmYgNjRi
aXRdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzUzOTI4XSBwY2kgMDAw
NDowMzowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA2Ml0gcGNpX2J1cyAwMDA0OjAwOiBvbiBO
VU1BIG5vZGUgMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA2NF0g
cGNpIDAwMDQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZm
ZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxLTAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEw
MDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA2Nl0gcGNpIDAw
MDQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDNd
IGFkZF9zaXplIDEwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUw
NjddIHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUwNjhdIHBjaSAw
MDA0OjAwOjAzLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRv
IFtidXMgMDNdIGFkZF9zaXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDY5XSBwY2kgMDAwNDowMDowNS4wOiBicmlkZ2Ug
d2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMTAwMApTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA3MF0gcGNpIDAwMDQ6MDA6MDUu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0
byBbYnVzIDA0XSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA3MV0gcGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTUwNzRdIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDIw
MDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU1MDc2XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyODAwMDAw
MDAwMDAtMHgyODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTUwNzddIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzU1MDc4XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgyODAwMDAyMDAwMDAtMHgyODAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUwNzldIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxNDog
YXNzaWduZWQgW21lbSAweDIzMjAwMDAwLTB4MjMzZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzU1MDgwXSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTU6IGFzc2ln
bmVkIFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUwODFdIHBjaSAwMDA0OjAwOjAxLjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDgyXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTUwODJdIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2Ug
Zm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzU1MDgzXSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUw
ODRdIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDg1XSBwY2kgMDAw
NDowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUwODddIHBjaSAwMDA0OjAwOjAx
LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDg4XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NTUwODldIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzU1MDg5XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTUwOTBdIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDkxXSBwY2kg
MDAwNDowMDowMy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUwOTJdIHBjaSAwMDA0OjAx
OjAwLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MDkzXSBwY2kgMDAwNDowMTowMC4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTA5NF0gcGNpIDAwMDQ6MDE6MDAuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzU1MDk1XSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQg
W21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzU1MDk5XSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMTogYXNzaWduZWQgW21lbSAw
eDIyMDAwMDAwLTB4MjIwMWZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzU1MTAyXSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDAwODBdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTAzXSBw
Y2kgMDAwNDowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDgw
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTEwNF0gcGNpIDAwMDQ6
MDE6MDAuMDogUENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1NTEwOF0gcGNpIDAwMDQ6MDE6MDAuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1NTExNl0gcGNpIDAwMDQ6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxLTAy
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTExN10gcGNpIDAwMDQ6
MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTExOF0gcGNpIDAwMDQ6MDA6MDEu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDAwMDAwMDAtMHgyODAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUxMjBdIHBj
aSAwMDA0OjAzOjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVtIDB4MjMwMDAwMDAtMHgyMzAwMWZm
ZiA2NGJpdF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUxMzBdIHBj
aSAwMDA0OjAwOjAzLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwM10KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTUxMzFdIHBjaSAwMDA0OjAwOjAzLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjMwMDAwMDAtMHgyMzFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTUxMzJdIHBjaSAwMDA0OjAwOjAzLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MjgwMDAwMjAwMDAwLTB4MjgwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTM0XSBwY2kgMDAwNDowMDowNS4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDRdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU1MTM2XSBwY2kgMDAwNDowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIzMjAwMDAw
LTB4MjMzZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTM3
XSBwY2kgMDAwNDowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDQwMDAwMC0w
eDI4MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1NTEzOV0gcGNpX2J1cyAwMDA0OjAwOiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2Vz
IGFyZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGluZyB3aXRoIHBjaT1yZWFsbG9jClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTQwXSBwY2lfYnVzIDAwMDQ6MDA6IHJlc291
cmNlIDQgW21lbSAweDIwMDAwMDAwLTB4MmZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTE0MV0gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA1
IFttZW0gMHgyODAwMDAwMDAwMDAtMHgyYmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTE0Ml0gcGNpX2J1cyAwMDA0OjAxOiByZXNvdXJj
ZSAxIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1NTE0M10gcGNpX2J1cyAwMDA0OjAxOiByZXNvdXJjZSAyIFttZW0gMHgy
ODAwMDAwMDAwMDAtMHgyODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTUxNDRdIHBjaV9idXMgMDAwNDowMjogcmVzb3VyY2UgMSBb
bWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTUxNDVdIHBjaV9idXMgMDAwNDowMzogcmVzb3VyY2UgMSBbbWVtIDB4MjMwMDAw
MDAtMHgyMzFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTUx
NDZdIHBjaV9idXMgMDAwNDowMzogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4Mjgw
MDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzU1MTQ2XSBwY2lfYnVzIDAwMDQ6MDQ6IHJlc291cmNlIDEgW21lbSAweDIzMjAwMDAwLTB4
MjMzZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTQ3XSBw
Y2lfYnVzIDAwMDQ6MDQ6IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDQwMDAwMC0weDI4MDAwMDVm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
NTE3NV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0k3XSAoZG9tYWluIDAwMDUgW2J1cyAwMC1m
Zl0pClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1MTc5XSBhY3BpIFBO
UDBBMDg6MDc6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0g
U2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTUyNDNdIGFjcGkgUE5QMEEwODowNzogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3Vw
cG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTUzMDFdIGFjcGkgUE5QMEEwODowNzogX09TQzogT1Mgbm93
IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzU1MzAzXSBhY3BpIFBOUDBBMDg6MDc6IE1DRkcgcXVpcms6IEVDQU0gYXQg
W21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGgg
MHhmZmZmYzdlN2U1YTZlYTU4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU1OTI2XSBhY3BpIFBOUDBBMDg6MDc6IEVDQU0gYXJlYSBbbWVtIDB4MmZmZmYwMDAwMDAwLTB4
MmZmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzU1OTMyXSBhY3BpIFBOUDBBMDg6MDc6IEVDQU0gYXQgW21lbSAw
eDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1OTg5XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVz
IDAwMDU6MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTU5OTBdIHBj
aV9idXMgMDAwNTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDMwMDAwMDAwLTB4M2ZmZmZm
ZmYgd2luZG93XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NTk5MV0g
cGNpX2J1cyAwMDA1OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4
MmZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NTU5OTJdIHBjaV9idXMgMDAwNTowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0K
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTYwMDhdIHBjaSAwMDA1OjAw
OjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTYwNTBdIHBjaSAwMDA1OjAwOjAxLjA6IFsxZGVmOmUx
MTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTYwODVdIHBjaSAwMDA1OjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU2MDg2XSBwY2kgMDAwNTowMDowMS4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzU2MTQ2XSBwY2kgMDAwNTowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAx
IGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU2
MTgyXSBwY2kgMDAwNTowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1NjE4M10gcGNpIDAwMDU6MDA6MDUuMDogUE1FIyBzdXBwb3J0
ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc1NjIzNV0gcGNpIDAwMDU6MDA6MDcuMDogWzFkZWY6ZTExN10gdHlwZSAwMSBjbGFzcyAweDA2
MDQwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NjI3MV0gcGNpIDAw
MDU6MDA6MDcuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTYyNzJdIHBjaSAwMDA1OjAwOjA3LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAg
RDEgRDNob3QKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTYzNDhdIGFj
cGlwaHA6IFNsb3QgWzEtN10gcmVnaXN0ZXJlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjc2MDU1NV0gcGNpX2J1cyAwMDA1OjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU1Nl0gcGNpIDAwMDU6MDA6MDEuMDog
YnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEw
MDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NThdIHBjaSAwMDA1
OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQg
cHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NTldIHBjaSAwMDA1OjAwOjAxLjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzYwNTYwXSBwY2kgMDAwNTowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAg
MHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU2MV0gcGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBh
ZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAwLjc2MDU2Ml0gcGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NjNd
IHBjaSAwMDA1OjAwOjA3LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBb
YnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzYwNTY0XSBwY2kgMDAwNTowMDowNy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAw
MC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwNTY1
XSBwY2kgMDAwNTowMDowNy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZm
ZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU2OF0gcGNpIDAwMDU6MDA6MDEuMDog
QkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0KU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NjldIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAx
NTogYXNzaWduZWQgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU3MF0gcGNpIDAwMDU6
MDA6MDUuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MzAyMDAwMDAtMHgzMDNmZmZmZl0KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NzFdIHBjaSAwMDA1OjAwOjA1
LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDJjMDAwMDIwMDAwMC0weDJjMDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU3Ml0g
cGNpIDAwMDU6MDA6MDcuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MzA0MDAwMDAtMHgzMDVm
ZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NzNdIHBjaSAw
MDA1OjAwOjA3LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDJjMDAwMDQwMDAwMC0weDJjMDAw
MDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc2MDU3NF0gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NzVdIHBj
aSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU3Nl0gcGNpIDAwMDU6
MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1NzddIHBjaSAwMDA1OjAwOjA1LjA6IEJB
UiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU3OF0gcGNpIDAwMDU6MDA6MDcuMDogQkFSIDEzOiBu
byBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NjA1NzldIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFz
c2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc2MDU4MV0gcGNpIDAwMDU6MDA6MDcuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1ODFd
IHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU4Ml0gcGNpIDAw
MDU6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1ODNdIHBjaSAwMDA1OjAwOjA1LjA6
IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU4NF0gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDEz
OiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NjA1ODVdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRv
IGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc2MDU4Nl0gcGNpIDAwMDU6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU4N10gcGNpIDAwMDU6MDA6
MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDAwMDAwMC0weDMwMWZmZmZmXQpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDU4OV0gcGNpIDAwMDU6MDA6MDEuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA1OTFdIHBjaSAw
MDA1OjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NjA1OTJdIHBjaSAwMDA1OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MzAyMDAwMDAtMHgzMDNmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NjA1OTRdIHBjaSAwMDA1OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MmMwMDAwMjAwMDAwLTB4MmMwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwNTk2XSBwY2kgMDAwNTowMDowNy4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYw
NTk3XSBwY2kgMDAwNTowMDowNy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwNDAwMDAwLTB4
MzA1ZmZmZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwNTk5XSBw
Y2kgMDAwNTowMDowNy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDQwMDAwMC0weDJj
MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc2MDYwMV0gcGNpX2J1cyAwMDA1OjAwOiByZXNvdXJjZSA0IFttZW0gMHgzMDAwMDAwMC0w
eDNmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NjA2MDJdIHBjaV9idXMgMDAwNTowMDogcmVzb3VyY2UgNSBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4
MmZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NjA2MDJdIHBjaV9idXMgMDAwNTowMTogcmVzb3VyY2UgMSBbbWVtIDB4MzAwMDAwMDAtMHgz
MDFmZmZmZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjA2MDNdIHBj
aV9idXMgMDAwNTowMTogcmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYw
NjA0XSBwY2lfYnVzIDAwMDU6MDI6IHJlc291cmNlIDEgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZm
ZmZdClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwNjA1XSBwY2lfYnVz
IDAwMDU6MDI6IHJlc291cmNlIDIgW21lbSAweDJjMDAwMDIwMDAwMC0weDJjMDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDYwNl0g
cGNpX2J1cyAwMDA1OjAzOiByZXNvdXJjZSAxIFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDYwN10gcGNpX2J1cyAwMDA1
OjAzOiByZXNvdXJjZSAyIFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjMwODBdIGlvbW11
OiBEZWZhdWx0IGRvbWFpbiB0eXBlOiBUcmFuc2xhdGVkIApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc2MzExNF0gcGNpIDAwMDQ6MDI6MDAuMDogdmdhYXJiOiBWR0EgZGV2
aWNlIGFkZGVkOiBkZWNvZGVzPWlvK21lbSxvd25zPW5vbmUsbG9ja3M9bm9uZQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MzExNV0gcGNpIDAwMDQ6MDI6MDAuMDogdmdh
YXJiOiBicmlkZ2UgY29udHJvbCBwb3NzaWJsZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAwLjc2MzExN10gcGNpIDAwMDQ6MDI6MDAuMDogdmdhYXJiOiBzZXR0aW5nIGFzIGJv
b3QgZGV2aWNlIChWR0EgbGVnYWN5IHJlc291cmNlcyBub3QgYXZhaWxhYmxlKQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MzExOF0gdmdhYXJiOiBsb2FkZWQKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjMyMDVdIEVEQUMgTUM6IFZlcjogMy4w
LjAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjMzMzJdIFJlZ2lzdGVy
ZWQgZWZpdmFycyBvcGVyYXRpb25zClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzYzOTk3XSBOZXRMYWJlbDogSW5pdGlhbGl6aW5nClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzYzOTk4XSBOZXRMYWJlbDogIGRvbWFpbiBoYXNoIHNpemUgPSAxMjgK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjM5OTldIE5ldExhYmVsOiAg
cHJvdG9jb2xzID0gVU5MQUJFTEVEIENJUFNPdjQgQ0FMSVBTTwpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc2NDAwOF0gTmV0TGFiZWw6ICB1bmxhYmVsZWQgdHJhZmZpYyBh
bGxvd2VkIGJ5IGRlZmF1bHQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NjQxNDBdIGNsb2Nrc291cmNlOiBTd2l0Y2hlZCB0byBjbG9ja3NvdXJjZSBhcmNoX3N5c19jb3Vu
dGVyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcyNjQ4XSBWRlM6IERp
c2sgcXVvdGFzIGRxdW90XzYuNi4wClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzcyNjY1XSBWRlM6IERxdW90LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogNTEyIChvcmRl
ciAwLCA0MDk2IGJ5dGVzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc3
Mjc4Ml0gQXBwQXJtb3I6IEFwcEFybW9yIEZpbGVzeXN0ZW0gRW5hYmxlZApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc3MjgxNl0gcG5wOiBQblAgQUNQSSBpbml0ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzczNDQyXSBzeXN0ZW0gMDA6MDA6IFtt
ZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVz
ZXJ2ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzM0NDRdIHN5c3Rl
bSAwMDowMDogW21lbSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxk
IG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc3
MzQ0NV0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmIHdp
bmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzczNDQ2XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyN2ZmZjAwMDAwMDAtMHgyN2Zm
ZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NzM0NDddIHN5c3RlbSAwMDowMDogW21lbSAweDJiZmZmMDAw
MDAwMC0weDJiZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc3MzQ0OF0gc3lzdGVtIDAwOjAwOiBbbWVt
IDB4MmZmZmYwMDAwMDAwLTB4MmZmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2Vy
dmVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzczNDQ5XSBzeXN0ZW0g
MDA6MDA6IFttZW0gMHgzM2ZmZjAwMDAwMDAtMHgzM2ZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBu
b3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzM0
NTFdIHN5c3RlbSAwMDowMDogW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZmZmZmZiB3aW5k
b3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAwLjc3MzQ1NF0gc3lzdGVtIDAwOjAwOiBQbHVnIGFuZCBQbGF5IEFDUEkgZGV2aWNlLCBJ
RHMgUE5QMGMwMiAoYWN0aXZlKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc3Mzc0NV0gcG5wOiBQblAgQUNQSTogZm91bmQgMSBkZXZpY2VzClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzc0NjYwXSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFt
aWx5IDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzQ3MjVdIElQIGlk
ZW50cyBoYXNoIHRhYmxlIGVudHJpZXM6IDI2MjE0NCAob3JkZXI6IDksIDIwOTcxNTIgYnl0ZXMs
IGxpbmVhcikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzYzMjhdIHRj
cF9saXN0ZW5fcG9ydGFkZHJfaGFzaCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjog
NywgNTI0Mjg4IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzc2NjE2XSBUQ1AgZXN0YWJsaXNoZWQgaGFzaCB0YWJsZSBlbnRyaWVzOiA1MjQyODgg
KG9yZGVyOiAxMCwgNDE5NDMwNCBieXRlcywgbGluZWFyKQpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAwLjc3NzY5N10gVENQIGJpbmQgaGFzaCB0YWJsZSBlbnRyaWVzOiA2NTUz
NiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Nzc5NTVdIFRDUDogSGFzaCB0YWJsZXMgY29uZmlndXJlZCAoZXN0
YWJsaXNoZWQgNTI0Mjg4IGJpbmQgNjU1MzYpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzc4MDQxXSBVRFAgaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDgs
IDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Nzg0NDVdIFVEUC1MaXRlIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA4
LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzc5MjE2XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDEKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzkyMjJdIE5FVDogUmVnaXN0ZXJlZCBwcm90
b2NvbCBmYW1pbHkgNDQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Nzky
MzBdIHBjaSAwMDBjOjAxOjAwLjA6IENMUyBtaXNtYXRjaCAoNjQgIT0gMzIpLCB1c2luZyA2NCBi
eXRlcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAwLjc3OTMwNl0gVHJ5aW5n
IHRvIHVucGFjayByb290ZnMgaW1hZ2UgYXMgaW5pdHJhbWZzLi4uClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMTgwNzY0XSBGcmVlaW5nIGluaXRyZCBtZW1vcnk6IDM3MTY0
SwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MjQ0OF0gaHcgcGVyZmV2
ZW50czogZW5hYmxlZCB3aXRoIGFybXY4X3BtdXYzXzAgUE1VIGRyaXZlciwgNyBjb3VudGVycyBh
dmFpbGFibGUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODI1MzVdIGt2
bSBbMV06IElQQSBTaXplIExpbWl0OiA0OCBiaXRzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTgyNTU3XSBrdm0gWzFdOiBHSUN2Mzogbm8gR0lDViByZXNvdXJjZSBlbnRy
eQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MjU1OF0ga3ZtIFsxXTog
ZGlzYWJsaW5nIEdJQ3YyIGVtdWxhdGlvbgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAxLjE4MjU2OV0ga3ZtIFsxXTogR0lDIHN5c3RlbSByZWdpc3RlciBDUFUgaW50ZXJmYWNl
IGVuYWJsZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODI5MThdIGt2
bSBbMV06IHZnaWMgaW50ZXJydXB0IElSUTkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xODMzMzFdIGt2bSBbMV06IFZIRSBtb2RlIGluaXRpYWxpemVkIHN1Y2Nlc3NmdWxs
eQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE4NDE2NF0gSW5pdGlhbGlz
ZSBzeXN0ZW0gdHJ1c3RlZCBrZXlyaW5ncwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAxLjE4NDE3MV0gS2V5IHR5cGUgYmxhY2tsaXN0IHJlZ2lzdGVyZWQKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODQyMTBdIHdvcmtpbmdzZXQ6IHRpbWVzdGFtcF9i
aXRzPTQyIG1heF9vcmRlcj0yNCBidWNrZXRfb3JkZXI9MApTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAxLjE4NDg2OV0gemJ1ZDogbG9hZGVkClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMTg0OTg4XSBpbnRlZ3JpdHk6IFBsYXRmb3JtIEtleXJpbmcgaW5p
dGlhbGl6ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODQ5OTBdIEtl
eSB0eXBlIGFzeW1tZXRyaWMgcmVnaXN0ZXJlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAxLjE4NDk5MV0gQXN5bW1ldHJpYyBrZXkgcGFyc2VyICd4NTA5JyByZWdpc3RlcmVk
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg0OTk3XSBCbG9jayBsYXll
ciBTQ1NJIGdlbmVyaWMgKGJzZykgZHJpdmVyIHZlcnNpb24gMC40IGxvYWRlZCAobWFqb3IgMjQ5
KQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE4NTAzOF0gaW8gc2NoZWR1
bGVyIG1xLWRlYWRsaW5lIHJlZ2lzdGVyZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xODYwNjFdIHNocGNocDogU3RhbmRhcmQgSG90IFBsdWcgUENJIENvbnRyb2xsZXIg
RHJpdmVyIHZlcnNpb246IDAuNApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
LjE4NjE3M10gZWZpZmI6IHByb2JpbmcgZm9yIGVmaWZiClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTg2MTkzXSBlZmlmYjogZnJhbWVidWZmZXIgYXQgMHgyMDAwMDAwMCwg
dXNpbmcgMTg3NmssIHRvdGFsIDE4NzVrClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTg2MTk0XSBlZmlmYjogbW9kZSBpcyA4MDB4NjAweDMyLCBsaW5lbGVuZ3RoPTMyMDAs
IHBhZ2VzPTEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODYxOTRdIGVm
aWZiOiBzY3JvbGxpbmc6IHJlZHJhdwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjE4NjE5Nl0gZWZpZmI6IFRydWVjb2xvcjogc2l6ZT04Ojg6ODo4LCBzaGlmdD0yNDoxNjo4
OjAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODYyNjhdIENvbnNvbGU6
IHN3aXRjaGluZyB0byBjb2xvdXIgZnJhbWUgYnVmZmVyIGRldmljZSAxMDB4MzcKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTIxMTFdIGZiMDogRUZJIFZHQSBmcmFtZSBi
dWZmZXIgZGV2aWNlClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTkyNTc0
XSBpbnB1dDogUG93ZXIgQnV0dG9uIGFzIC9kZXZpY2VzL0xOWFNZU1RNOjAwL0xOWFNZQlVTOjAw
L1BOUDBDMEM6MDAvaW5wdXQvaW5wdXQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTkyNTg4XSBBQ1BJOiBQb3dlciBCdXR0b24gW1BXUkJdClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMTk2OTA0XSBBQ1BJIEdURFQ6IGZvdW5kIDEgU0JTQSBnZW5l
cmljIFdhdGNoZG9nKHMpLgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5
NzQyNF0gU2VyaWFsOiA4MjUwLzE2NTUwIGRyaXZlciwgNCBwb3J0cywgSVJRIHNoYXJpbmcgZW5h
YmxlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5NzY3Nl0gU2VyaWFs
OiBBTUJBIGRyaXZlcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5Nzcw
OF0gbXNtX3NlcmlhbDogZHJpdmVyIGluaXRpYWxpemVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTk3NzU3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IG9w
dGlvbiBtYXNrIDB4MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5Nzc4
Ml0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0
IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjE5Nzk0OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBhbGxvY2F0ZWQgNjU1
MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
LjE5Nzk4NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBhbGxvY2F0ZWQgMzI3Njgg
ZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5
ODA0OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50
cmllcyBmb3IgcHJpcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5ODQx
OV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMS5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg0MzFdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjEuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZm
ZikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg1NDZdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjEuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg1ODNdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjEuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg2NDddIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjEuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTg5NTBdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjIuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTk4OTYyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IGlhcyA0
OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMTk5MDA5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1
dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMTk5MTIxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86
IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTk5MTU3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTk5NDkxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IG9wdGlv
biBtYXNrIDB4MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5OTUwMV0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChm
ZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
LjE5OTU0Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYg
ZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5
OTY1OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50
cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjE5OTY5
NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmll
cyBmb3IgcHJpcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIwMDAxOF0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNC5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDAwMjddIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjQuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDAwOTZdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjQuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDAxNTBdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjQuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDAyNjFdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjQuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDA1NzVdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjUuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjAwNTg2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IGlhcyA0OC1i
aXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjAwNjUyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86
IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjAwNjk2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IGFs
bG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjAwODA2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IGFsbG9j
YXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjAxMTE0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My42LmF1dG86IG9wdGlvbiBt
YXNrIDB4MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIwMTEyNV0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0
dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
MTE2NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50
cmllcyBmb3IgY21kcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIwMTIy
OV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmll
cyBmb3IgZXZ0cQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIwMTI3Ml0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBm
b3IgcHJpcQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIwMTY2MV0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNy5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDE2NzFdIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjcuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDE3MDddIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjcuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDE3NjldIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjcuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDE4MTBdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjcuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDYxNTNdIG1vdXNlZGV2OiBQUy8yIG1vdXNlIGRl
dmljZSBjb21tb24gZm9yIGFsbCBtaWNlClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjA3MDk3XSBydGMtZWZpIHJ0Yy1lZmkuMDogcmVnaXN0ZXJlZCBhcyBydGMwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA3NTU3XSBydGMtZWZpIHJ0Yy1lZmku
MDogc2V0dGluZyBzeXN0ZW0gY2xvY2sgdG8gMjAyMS0wOS0xM1QwNzozODoxNSBVVEMgKDE2MzE1
MTg2OTUpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4MTQ0XSBsZWR0
cmlnLWNwdTogcmVnaXN0ZXJlZCB0byBpbmRpY2F0ZSBhY3Rpdml0eSBvbiBDUFVzClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4NDA4XSBTTUNDQzogU09DX0lEOiBJRCA9
IGplcDEwNjowYTE2OjAwMDEgUmV2aXNpb24gPSAweDAwMDAwMGExClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4NjY2XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFt
aWx5IDEwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE2NzQwXSBTZWdt
ZW50IFJvdXRpbmcgd2l0aCBJUHY2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjE2NzUzXSBtaXA2OiBNb2JpbGUgSVB2NgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAxLjIxNjc1NV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxNwpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNjc3OV0gbXBsc19nc286IE1QTFMg
R1NPIHN1cHBvcnQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY4Mjld
IHJlZ2lzdGVyZWQgdGFza3N0YXRzIHZlcnNpb24gMQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjIxNjgzMl0gTG9hZGluZyBjb21waWxlZC1pbiBYLjUwOSBjZXJ0aWZpY2F0
ZXMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNDMxMzldIExvYWRlZCBY
LjUwOSBjZXJ0ICdEZWJpYW4gU2VjdXJlIEJvb3QgQ0E6IDZjY2VjZTdlNGM2YzBkMWY2MTQ5ZjNk
ZDI3ZGZjYzVjYmI0MTllYTEnClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjQzMTQ4XSBMb2FkZWQgWC41MDkgY2VydCAnRGViaWFuIFNlY3VyZSBCb290IFNpZ25lciAyMDIx
IC0gbGludXg6IDRiNmVmNWFiY2E2Njk4MjUxNzhlMDUyYzg0NjY3Y2NiYzA1MzFmOGMnClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjQzMzAzXSB6c3dhcDogbG9hZGVkIHVz
aW5nIHBvb2wgbHpvL3pidWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
NDM0MDFdIEtleSB0eXBlIC5fZnNjcnlwdCByZWdpc3RlcmVkClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjQzNDAyXSBLZXkgdHlwZSAuZnNjcnlwdCByZWdpc3RlcmVkClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjQzNDAzXSBLZXkgdHlwZSBmc2Ny
eXB0LXByb3Zpc2lvbmluZyByZWdpc3RlcmVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjQzNDMxXSBBcHBBcm1vcjogQXBwQXJtb3Igc2hhMSBwb2xpY3kgaGFzaGluZyBl
bmFibGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjUyNDcyXSBpbnRl
Z3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTMwMTFdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNl
cnQgJ1NVU0UgTGludXggRW50ZXJwcmlzZSBTZWN1cmUgQm9vdCBDQTogM2Q0ZDQwY2Y5Mzg1Mzkw
MjRiMWNmYzVhMTJkZWRmZThiMTdlNzU1ZicKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yNTMwMTJdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVG
STpkYgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MzE3M10gaW50ZWdy
aXR5OiBMb2FkZWQgWC41MDkgY2VydCAnQ2Fub25pY2FsIEx0ZC4gTWFzdGVyIENlcnRpZmljYXRl
IEF1dGhvcml0eTogYWQ5MTk5MGJjMjJhYjFmNTE3MDQ4YzIzYjY2NTVhMjY4ZTM0NWE2MycKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTMxNzRdIGludGVncml0eTogTG9h
ZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjI1MzMyOV0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnRmVkb3Jh
IFNlY3VyZSBCb290IENBOiBmZGUzMjU5OWMyZDYxZGIxYmY1ODA3MzM1ZDdiMjBlNGNkOTYzYjQy
JwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MzMyOV0gaW50ZWdyaXR5
OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjUzMzQyXSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdN
aWNyb3NvZnQgV2luZG93cyBQcm9kdWN0aW9uIFBDQSAyMDExOiBhOTI5MDIzOThlMTZjNDk3Nzhj
ZDkwZjk5ZTRmOWFlMTdjNTVhZjUzJwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjI1MzM0M10gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRi
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjUzMzU1XSBpbnRlZ3JpdHk6
IExvYWRlZCBYLjUwOSBjZXJ0ICdNaWNyb3NvZnQgQ29ycG9yYXRpb24gVUVGSSBDQSAyMDExOiAx
M2FkYmY0MzA5YmQ4MjcwOWM4Y2Q1NGYzMTZlZDUyMjk4OGExYmQ0JwpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MzM1Nl0gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNl
cnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjUzMzY4XSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdNaWNyb3NvZnQgV2luZG93cyBQ
cm9kdWN0aW9uIFBDQSAyMDExOiBhOTI5MDIzOThlMTZjNDk3NzhjZDkwZjk5ZTRmOWFlMTdjNTVh
ZjUzJwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI1Nzc0Nl0gcGNpZXBv
cnQgMDAwYzowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODA0M10gcGNpZXBvcnQgMDAwYzowMDowMS4wOiBBRVI6
IGVuYWJsZWQgd2l0aCBJUlEgMTA2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjU4MTEzXSBwY2llcG9ydCAwMDBkOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAx
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU4Mzc5XSBwY2llcG9ydCAw
MDBkOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yNTg0MjRdIHBjaWVwb3J0IDAwMGQ6MDA6MDIuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
NTg2NjRdIHBjaWVwb3J0IDAwMGQ6MDA6MDIuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODcwOV0gcGNpZXBvcnQgMDAw
ZDowMDowMy4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMwpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAxLjI1ODk0OF0gcGNpZXBvcnQgMDAwZDowMDowMy4wOiBBRVI6IGVuYWJs
ZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU4
OTkwXSBwY2llcG9ydCAwMDBkOjAwOjA0LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA0ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU5MjI5XSBwY2llcG9ydCAwMDBkOjAw
OjA0LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yNTkyODldIHBjaWVwb3J0IDAwMDE6MDA6MDEuMDogQWRkaW5nIHRvIGlv
bW11IGdyb3VwIDUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTk1NTRd
IHBjaWVwb3J0IDAwMDE6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwOApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI1OTYyM10gcGNpZXBvcnQgMDAwMDowMDow
MS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAxLjI1OTg4OV0gcGNpZXBvcnQgMDAwMDowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0
aCBJUlEgMTA5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU5OTYwXSBw
Y2llcG9ydCAwMDAyOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCA3ClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjYwMjQ1XSBwY2llcG9ydCAwMDAyOjAwOjAxLjA6
IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yNjAyOTVdIHBjaWVwb3J0IDAwMDI6MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdy
b3VwIDgKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjA1NDZdIHBjaWVw
b3J0IDAwMDI6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI2MDYxMl0gcGNpZXBvcnQgMDAwMzowMDowMS4wOiBB
ZGRpbmcgdG8gaW9tbXUgZ3JvdXAgOQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjI2MDg4OF0gcGNpZXBvcnQgMDAwMzowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEg
MTExClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjYwOTM5XSBwY2llcG9y
dCAwMDAzOjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxMApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI2MTE4NF0gcGNpZXBvcnQgMDAwMzowMDowNS4wOiBBRVI6
IGVuYWJsZWQgd2l0aCBJUlEgMTExClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjYxMjQ3XSBwY2llcG9ydCAwMDA0OjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAx
MQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI2MTUyM10gcGNpZXBvcnQg
MDAwNDowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEyClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjYxNTY5XSBwY2llcG9ydCAwMDA0OjAwOjAzLjA6IEFkZGlu
ZyB0byBpb21tdSBncm91cCAxMgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
LjI2MTgxM10gcGNpZXBvcnQgMDAwNDowMDowMy4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEy
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjYxODU5XSBwY2llcG9ydCAw
MDA0OjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxMwpTZXAgMTMgMTM6MDg6MTggZGVi
aWFuIGtlcm5lbDogWyAgICAxLjI2MjEwNV0gcGNpZXBvcnQgMDAwNDowMDowNS4wOiBBRVI6IGVu
YWJsZWQgd2l0aCBJUlEgMTEyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjYyMTc0XSBwY2llcG9ydCAwMDA0OjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNApT
ZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI2MjQzOV0gcGNpZXBvcnQgMDAw
NTowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTUKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yNjI3MTNdIHBjaWVwb3J0IDAwMDU6MDA6MDEuMDogQUVSOiBlbmFi
bGVkIHdpdGggSVJRIDExMwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI2
Mjc2Nl0gcGNpZXBvcnQgMDAwNTowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTYKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjMwMThdIHBjaWVwb3J0IDAwMDU6
MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMwpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAxLjI2MzA3MF0gcGNpZXBvcnQgMDAwNTowMDowNy4wOiBBZGRpbmcgdG8g
aW9tbXUgZ3JvdXAgMTcKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjMz
MTVdIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMwpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NDY4M10gRnJlZWluZyB1bnVzZWQg
a2VybmVsIG1lbW9yeTogNTQ0MEsKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS40MjMzNjFdIENoZWNrZWQgVytYIG1hcHBpbmdzOiBwYXNzZWQsIG5vIFcrWCBwYWdlcyBmb3Vu
ZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjQyMzM2Nl0gUnVuIC9pbml0
IGFzIGluaXQgcHJvY2VzcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjQy
MzM2OF0gICB3aXRoIGFyZ3VtZW50czoKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS40MjMzNjhdICAgICAvaW5pdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjQyMzM2OV0gICB3aXRoIGVudmlyb25tZW50OgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjQyMzM3MF0gICAgIEhPTUU9LwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjQyMzM3MF0gICAgIFRFUk09bGludXgKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMS40MjMzNzFdICAgICBCT09UX0lNQUdFPS9ib290L3ZtbGludXotNS4x
MC4wLTgtYXJtNjQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTc2ODJd
IHBwc19jb3JlOiBMaW51eFBQUyBBUEkgdmVyLiAxIHJlZ2lzdGVyZWQKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTc2ODRdIHBwc19jb3JlOiBTb2Z0d2FyZSB2ZXIuIDUu
My42IC0gQ29weXJpZ2h0IDIwMDUtMjAwNyBSb2RvbGZvIEdpb21ldHRpIDxnaW9tZXR0aUBsaW51
eC5pdD4KU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTg2OThdIFBUUCBj
bG9jayBzdXBwb3J0IHJlZ2lzdGVyZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS41MDMwOTldIFNDU0kgc3Vic3lzdGVtIGluaXRpYWxpemVkClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNTA1MDI3XSBpZ2I6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJu
ZXQgTmV0d29yayBEcml2ZXIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41
MDUwMjhdIGlnYjogQ29weXJpZ2h0IChjKSAyMDA3LTIwMTQgSW50ZWwgQ29ycG9yYXRpb24uClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA1NDE1XSBpZ2IgMDAwMzowMjow
MC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTgKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS41MDY1MjRdIEFDUEk6IGJ1cyB0eXBlIFVTQiByZWdpc3RlcmVkClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA2NTQzXSB1c2Jjb3JlOiByZWdpc3RlcmVk
IG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVzYmZzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNTA2NTQ4XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVy
IGh1YgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjUwNjU3MV0gdXNiY29y
ZTogcmVnaXN0ZXJlZCBuZXcgZGV2aWNlIGRyaXZlciB1c2IKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMS41MDg5OTNdIG52bWUgMDAwZDowMTowMC4wOiBBZGRpbmcgdG8gaW9t
bXUgZ3JvdXAgMTkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDkyNzVd
IG52bWUgbnZtZTA6IHBjaSBmdW5jdGlvbiAwMDBkOjAxOjAwLjAKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS41MDkzMzBdIG52bWUgMDAwZDowMjowMC4wOiBBZGRpbmcgdG8g
aW9tbXUgZ3JvdXAgMjAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDk3
MTVdIG52bWUgbnZtZTE6IHBjaSBmdW5jdGlvbiAwMDBkOjAyOjAwLjAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDk3NjVdIG52bWUgMDAwZDowMzowMC4wOiBBZGRpbmcg
dG8gaW9tbXUgZ3JvdXAgMjEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41
MTAwMDVdIG52bWUgbnZtZTI6IHBjaSBmdW5jdGlvbiAwMDBkOjAzOjAwLjAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTAwNTNdIG52bWUgMDAwZDowNDowMC4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgMjIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS41MTAyODZdIG52bWUgbnZtZTM6IHBjaSBmdW5jdGlvbiAwMDBkOjA0OjAwLjAKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTY1NTFdIG1wdDNzYXMgdmVyc2lvbiAzNS4x
MDAuMDAuMDAgbG9hZGVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE2
NjY5XSBtcHQzc2FzIDAwMGM6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIzClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE3MDkzXSBtcHQzc2FzX2NtMDogNjMg
QklUIFBDSSBCVVMgRE1BIEFERFJFU1NJTkcgU1VQUE9SVEVELCB0b3RhbCBtZW0gKDY1MzA0Nzky
IGtCKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjUxNzEwMV0geGhjaV9o
Y2QgMDAwNDowMzowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMjQKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTc2NTJdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogeEhD
SSBIb3N0IENvbnRyb2xsZXIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41
MTc2NTZdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogbmV3IFVTQiBidXMgcmVnaXN0ZXJlZCwgYXNz
aWduZWQgYnVzIG51bWJlciAxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTE3Njc1XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IFplcm9pbmcgNjRiaXQgYmFzZSByZWdpc3Rl
cnMsIGV4cGVjdGluZyBmYXVsdApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
LjUxODkwMl0gcmFuZG9tOiBmYXN0IGluaXQgZG9uZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjUyMDEwOF0gbnZtZSBudm1lMTogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xs
IHF1ZXVlcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMDg1Ml0gbnZt
ZSBudm1lMDogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMTc5MF0gbnZtZSBudm1lMjogODAvMC8wIGRlZmF1bHQv
cmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjUy
MTgyMl0gbnZtZSBudm1lMzogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMjg0NV0gYXN0IDAwMDQ6MDI6MDAuMDog
QWRkaW5nIHRvIGlvbW11IGdyb3VwIDE0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTIzNTU4XSAgbnZtZTFuMTogcDEgcDIgcDMgcDQKU2VwIDEzIDEzOjA4OjE4IGRlYmlh
biBrZXJuZWw6IFsgICAgMS41MjQzNTNdICBudm1lMG4xOgpTZXAgMTMgMTM6MDg6MTggZGViaWFu
IGtlcm5lbDogWyAgICAxLjUyNTIzM10gIG52bWUzbjE6IHAxIHAyIHAzIHA0IHA1IHA2ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0MzIxXSB4aGNpX2hjZCAwMDA0OjAz
OjAwLjA6IGhjYyBwYXJhbXMgMHgwMTQwNTFjZiBoY2kgdmVyc2lvbiAweDEwMCBxdWlya3MgMHgw
MDAwMDAxMTAwMDAwNDEwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0
NjcwXSB1c2IgdXNiMTogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlkUHJv
ZHVjdD0wMDAyLCBiY2REZXZpY2U9IDUuMTAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS42NzQ2NzFdIHVzYiB1c2IxOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9Mywg
UHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAxLjY3NDY3Ml0gdXNiIHVzYjE6IFByb2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0NjczXSB1c2IgdXNiMTogTWFu
dWZhY3R1cmVyOiBMaW51eCA1LjEwLjAtOC1hcm02NCB4aGNpLWhjZApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NDY3M10gdXNiIHVzYjE6IFNlcmlhbE51bWJlcjogMDAw
NDowMzowMC4wClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0Nzg1XSBo
dWIgMS0wOjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAxLjY3NDc5N10gaHViIDEtMDoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzQ5NDddIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDog
eEhDSSBIb3N0IENvbnRyb2xsZXIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42NzQ5NDldIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogbmV3IFVTQiBidXMgcmVnaXN0ZXJlZCwg
YXNzaWduZWQgYnVzIG51bWJlciAyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjc0OTUxXSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IEhvc3Qgc3VwcG9ydHMgVVNCIDMuMCBT
dXBlclNwZWVkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0OTk4XSB1
c2IgdXNiMjogV2UgZG9uJ3Qga25vdyB0aGUgYWxnb3JpdGhtcyBmb3IgTFBNIGZvciB0aGlzIGhv
c3QsIGRpc2FibGluZyBMUE0uClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
Njc1MDEwXSB1c2IgdXNiMjogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlk
UHJvZHVjdD0wMDAzLCBiY2REZXZpY2U9IDUuMTAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS42NzUwMTFdIHVzYiB1c2IyOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9
MywgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5l
bDogWyAgICAxLjY3NTAxMl0gdXNiIHVzYjI6IFByb2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVy
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc1MDEyXSB1c2IgdXNiMjog
TWFudWZhY3R1cmVyOiBMaW51eCA1LjEwLjAtOC1hcm02NCB4aGNpLWhjZApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTAxM10gdXNiIHVzYjI6IFNlcmlhbE51bWJlcjog
MDAwNDowMzowMC4wClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc1MDM2
XSBjaGVja2luZyBnZW5lcmljICgyMDAwMDAwMCAxZDUwMDApIHZzIGh3ICgyMDAwMDAwMCAyMDAw
MDAwKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTAzN10gZmIwOiBz
d2l0Y2hpbmcgdG8gYXN0ZHJtZmIgZnJvbSBFRkkgVkdBClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjc1MTAwXSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGR1bW15
IGRldmljZSA4MHgyNQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTE1
MV0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gcGxhdGZvcm0gaGFzIG5vIElPIHNwYWNlLCB0cnlp
bmcgTU1JTwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTE1OF0gYXN0
IDAwMDQ6MDI6MDAuMDogW2RybV0gUDJBIGJyaWRnZSBkaXNhYmxlZCwgdXNpbmcgZGVmYXVsdCBj
b25maWd1cmF0aW9uClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc1MTU5
XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBBU1QgMjUwMCBkZXRlY3RlZApTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTE2M10gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0g
QW5hbG9nIFZHQSBvbmx5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc1
MTY0XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBkcmFtIE1DTEs9ODAwIE1oeiB0eXBlPTEgYnVz
X3dpZHRoPTE2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc1MTkzXSBo
dWIgMi0wOjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAxLjY3NTIxMl0gW1RUTV0gWm9uZSAga2VybmVsOiBBdmFpbGFibGUgZ3JhcGhpY3MgbWVt
b3J5OiAzMjY1MjM5NiBLaUIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42
NzUyMTNdIFtUVE1dIFpvbmUgICBkbWEzMjogQXZhaWxhYmxlIGdyYXBoaWNzIG1lbW9yeTogMjA5
NzE1MiBLaUIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzUyMTRdIFtU
VE1dIEluaXRpYWxpemluZyBwb29sIGFsbG9jYXRvcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAxLjY3NTIxOV0gW1RUTV0gSW5pdGlhbGl6aW5nIERNQSBwb29sIGFsbG9jYXRv
cgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NTM4NF0gaHViIDItMDox
LjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42NzU0NTZdIFtkcm1dIEluaXRpYWxpemVkIGFzdCAwLjEuMCAyMDEyMDIyOCBmb3IgMDAwNDow
MjowMC4wIG9uIG1pbm9yIDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43
ODkyOTJdIENvbnNvbGU6IHN3aXRjaGluZyB0byBjb2xvdXIgZnJhbWUgYnVmZmVyIGRldmljZSAy
NDB4NjcKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTE5NTJdIGFzdCAw
MDA0OjAyOjAwLjA6IFtkcm1dIGZiMDogYXN0ZHJtZmIgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjgxOTg3OV0gaWdiIDAwMDM6MDI6MDAu
MDogYWRkZWQgUEhDIG9uIGV0aDAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44MTk4ODFdIGlnYiAwMDAzOjAyOjAwLjA6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0
d29yayBDb25uZWN0aW9uClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODE5
ODgyXSBpZ2IgMDAwMzowMjowMC4wOiBldGgwOiAoUENJZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1
ZTpkMzowMjpiMTo5MwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjgxOTk2
NF0gaWdiIDAwMDM6MDI6MDAuMDogZXRoMDogUEJBIE5vOiAxMDYzMDAtMDAwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODE5OTY1XSBpZ2IgMDAwMzowMjowMC4wOiBVc2lu
ZyBNU0ktWCBpbnRlcnJ1cHRzLiA4IHJ4IHF1ZXVlKHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODIwMDQzXSBpZ2IgMDAwMzowMjowMC4xOiBB
ZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMjUKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NzQ2ODldIGlnYiAwMDAzOjAyOjAwLjE6IGFkZGVkIFBIQyBvbiBldGgxClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc0NjkxXSBpZ2IgMDAwMzowMjowMC4xOiBJ
bnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg3NDY5Ml0gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTog
KFBDSWU6NS4wR2IvczpXaWR0aCB4MikgZDg6NWU6ZDM6MDI6YjE6OTQKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzQ3NzNdIGlnYiAwMDAzOjAyOjAwLjE6IGV0aDE6IFBC
QSBObzogMTA2MzAwLTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg3
NDc3NF0gaWdiIDAwMDM6MDI6MDAuMTogVXNpbmcgTVNJLVggaW50ZXJydXB0cy4gOCByeCBxdWV1
ZShzKSwgOCB0eCBxdWV1ZShzKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAx
Ljg3NTg0OV0gaWdiIDAwMDM6MDI6MDAuMCBlbm8xOiByZW5hbWVkIGZyb20gZXRoMApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg4MDEyNV0gbXB0M3Nhc19jbTA6IEN1cnJl
bnRIb3N0UGFnZVNpemUgaXMgMDogU2V0dGluZyBkZWZhdWx0IGhvc3QgcGFnZSBzaXplIHRvIDRr
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgwMTQxXSBtcHQzc2FzX2Nt
MDogTVNJLVggdmVjdG9ycyBzdXBwb3J0ZWQ6IDk2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODgwMTQyXSAJIG5vIG9mIGNvcmVzOiA4MCwgbWF4X21zaXhfdmVjdG9yczog
LTEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODAxNDNdIG1wdDNzYXNf
Y20wOiAgMCA4MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg4MTM4M10g
bXB0M3Nhc19jbTA6IEhpZ2ggSU9QcyBxdWV1ZXMgOiBkaXNhYmxlZApTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg4MTM4NF0gbXB0M3NhczAtbXNpeDA6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNDY1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgx
Mzg1XSBtcHQzc2FzMC1tc2l4MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjYKU2VwIDEzIDEz
OjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODEzODVdIG1wdDNzYXMwLW1zaXgyOiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDQ2NwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjg4MTM4Nl0gbXB0M3NhczAtbXNpeDM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY4ClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzg3XSBtcHQzc2FzMC1tc2l4
NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjkKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44ODEzODddIG1wdDNzYXMwLW1zaXg1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJR
IDQ3MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg4MTM4OF0gbXB0M3Nh
czAtbXNpeDY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDcxClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODgxMzg4XSBtcHQzc2FzMC1tc2l4NzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0NzIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODEzODld
IG1wdDNzYXMwLW1zaXg4OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MwpTZXAgMTMgMTM6MDg6
MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjg4MTM5MF0gbXB0M3NhczAtbXNpeDk6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDc0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODgxMzkwXSBtcHQzc2FzMC1tc2l4MTA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc1ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzkxXSBtcHQzc2FzMC1tc2l4MTE6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODgxMzkxXSBtcHQzc2FzMC1tc2l4MTI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDc3ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzkyXSBtcHQzc2Fz
MC1tc2l4MTM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc4ClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODgxMzkyXSBtcHQzc2FzMC1tc2l4MTQ6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDc5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzkz
XSBtcHQzc2FzMC1tc2l4MTU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgwClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk0XSBtcHQzc2FzMC1tc2l4MTY6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDgxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODgxMzk0XSBtcHQzc2FzMC1tc2l4MTc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgyClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk1XSBtcHQzc2FzMC1tc2l4
MTg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODgxMzk1XSBtcHQzc2FzMC1tc2l4MTk6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNDg0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk2XSBtcHQz
c2FzMC1tc2l4MjA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg1ClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk2XSBtcHQzc2FzMC1tc2l4MjE6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNDg2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgx
Mzk3XSBtcHQzc2FzMC1tc2l4MjI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg3ClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk3XSBtcHQzc2FzMC1tc2l4MjM6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNDg4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODgxMzk4XSBtcHQzc2FzMC1tc2l4MjQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg5
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxMzk5XSBtcHQzc2FzMC1t
c2l4MjU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkwClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODgxMzk5XSBtcHQzc2FzMC1tc2l4MjY6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNDkxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDAwXSBt
cHQzc2FzMC1tc2l4Mjc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkyClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDAxXSBtcHQzc2FzMC1tc2l4Mjg6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDkzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODgxNDAxXSBtcHQzc2FzMC1tc2l4Mjk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk0ClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDAyXSBtcHQzc2FzMC1tc2l4MzA6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODgxNDAyXSBtcHQzc2FzMC1tc2l4MzE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDk2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDAzXSBtcHQzc2Fz
MC1tc2l4MzI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk3ClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODgxNDA0XSBtcHQzc2FzMC1tc2l4MzM6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDk4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA0
XSBtcHQzc2FzMC1tc2l4MzQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk5ClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA1XSBtcHQzc2FzMC1tc2l4MzU6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTAwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODgxNDA1XSBtcHQzc2FzMC1tc2l4MzY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAxClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA2XSBtcHQzc2FzMC1tc2l4
Mzc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODgxNDA2XSBtcHQzc2FzMC1tc2l4Mzg6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTAzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA3XSBtcHQz
c2FzMC1tc2l4Mzk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA0ClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA4XSBtcHQzc2FzMC1tc2l4NDA6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTA1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgx
NDA4XSBtcHQzc2FzMC1tc2l4NDE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA2ClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDA5XSBtcHQzc2FzMC1tc2l4NDI6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTA3ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODgxNDEwXSBtcHQzc2FzMC1tc2l4NDM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA4
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDEwXSBtcHQzc2FzMC1t
c2l4NDQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODgxNDExXSBtcHQzc2FzMC1tc2l4NDU6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTEwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDExXSBt
cHQzc2FzMC1tc2l4NDY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTExClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDEyXSBtcHQzc2FzMC1tc2l4NDc6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTEyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODgxNDEyXSBtcHQzc2FzMC1tc2l4NDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTEzClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDEzXSBtcHQzc2FzMC1tc2l4NDk6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODgxNDE0XSBtcHQzc2FzMC1tc2l4NTA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTE1ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDE0XSBtcHQzc2Fz
MC1tc2l4NTE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE2ClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODgxNDE1XSBtcHQzc2FzMC1tc2l4NTI6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTE3ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDE2
XSBtcHQzc2FzMC1tc2l4NTM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE4ClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDE2XSBtcHQzc2FzMC1tc2l4NTQ6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTE5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODgxNDE3XSBtcHQzc2FzMC1tc2l4NTU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIwClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDE3XSBtcHQzc2FzMC1tc2l4
NTY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODgxNDE4XSBtcHQzc2FzMC1tc2l4NTc6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTIyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDE5XSBtcHQz
c2FzMC1tc2l4NTg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIzClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDIwXSBtcHQzc2FzMC1tc2l4NTk6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTI0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgx
NDIwXSBtcHQzc2FzMC1tc2l4NjA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI1ClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDIxXSBtcHQzc2FzMC1tc2l4NjE6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTI2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODgxNDIxXSBtcHQzc2FzMC1tc2l4NjI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI3
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDIyXSBtcHQzc2FzMC1t
c2l4NjM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODgxNDIzXSBtcHQzc2FzMC1tc2l4NjQ6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTI5ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDIzXSBt
cHQzc2FzMC1tc2l4NjU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMwClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDI0XSBtcHQzc2FzMC1tc2l4NjY6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTMxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODgxNDI1XSBtcHQzc2FzMC1tc2l4Njc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMyClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDI1XSBtcHQzc2FzMC1tc2l4Njg6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODgxNDI2XSBtcHQzc2FzMC1tc2l4Njk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTM0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDI3XSBtcHQzc2Fz
MC1tc2l4NzA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM1ClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODgxNDI3XSBtcHQzc2FzMC1tc2l4NzE6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTM2ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDI4
XSBtcHQzc2FzMC1tc2l4NzI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM3ClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDI5XSBtcHQzc2FzMC1tc2l4NzM6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTM4ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODgxNDI5XSBtcHQzc2FzMC1tc2l4NzQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM5ClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDMwXSBtcHQzc2FzMC1tc2l4
NzU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODgxNDMxXSBtcHQzc2FzMC1tc2l4NzY6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTQxClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDMxXSBtcHQz
c2FzMC1tc2l4Nzc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQyClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDMyXSBtcHQzc2FzMC1tc2l4Nzg6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTQzClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgx
NDMzXSBtcHQzc2FzMC1tc2l4Nzk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQ0ClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDM0XSBtcHQzc2FzX2NtMDogaW9tZW0o
MHgwMDAwMDAwMDQwMTQwMDAwKSwgbWFwcGVkKDB4KF9fX19wdHJ2YWxfX19fKSksIHNpemUoNjU1
MzYpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODgxNDM1XSBtcHQzc2Fz
X2NtMDogaW9wb3J0KDB4MDAwMDAwMDAwMDAwMDAwMCksIHNpemUoMCkKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MDQzMzddIGlnYiAwMDAzOjAyOjAwLjEgZW5vMjogcmVu
YW1lZCBmcm9tIGV0aDEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMS45Mzg2
NDldIG1wdDNzYXNfY20wOiBDdXJyZW50SG9zdFBhZ2VTaXplIGlzIDA6IFNldHRpbmcgZGVmYXVs
dCBob3N0IHBhZ2Ugc2l6ZSB0byA0awpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAg
ICAxLjkzODY1MV0gbXB0M3Nhc19jbTA6IHNlbmRpbmcgZGlhZyByZXNldCAhIQpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAxLjk5MjE0Ml0gdXNiIDEtMTogbmV3IGxvdy1zcGVl
ZCBVU0IgZGV2aWNlIG51bWJlciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDIuMTg1MDYyXSB1c2IgMS0xOiBOZXcgVVNCIGRldmljZSBmb3VuZCwg
aWRWZW5kb3I9NDEzYywgaWRQcm9kdWN0PTMwMWEsIGJjZERldmljZT0gMS4wMApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjE4NTA2M10gdXNiIDEtMTogTmV3IFVTQiBkZXZp
Y2Ugc3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDEzIDEzOjA4
OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xODUwNjRdIHVzYiAxLTE6IFByb2R1Y3Q6IERlbGwg
TVMxMTYgVVNCIE9wdGljYWwgTW91c2UKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMi4xODUwNjVdIHVzYiAxLTE6IE1hbnVmYWN0dXJlcjogUGl4QXJ0ClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTk0MTcyXSBoaWQ6IHJhdyBISUQgZXZlbnRzIGRyaXZl
ciAoQykgSmlyaSBLb3NpbmEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4y
MDYxNjVdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGludGVyZmFjZSBkcml2ZXIgdXNiaGlkClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjA2MTY2XSB1c2JoaWQ6IFVTQiBI
SUQgY29yZSBkcml2ZXIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yMDc0
NDNdIGlucHV0OiBQaXhBcnQgRGVsbCBNUzExNiBVU0IgT3B0aWNhbCBNb3VzZSBhcyAvZGV2aWNl
cy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTEvMS0xOjEuMC8w
MDAzOjQxM0M6MzAxQS4wMDAxL2lucHV0L2lucHV0MQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAyLjIwNzQ5Ml0gaGlkLWdlbmVyaWMgMDAwMzo0MTNDOjMwMUEuMDAwMTogaW5w
dXQsaGlkcmF3MDogVVNCIEhJRCB2MS4xMSBNb3VzZSBbUGl4QXJ0IERlbGwgTVMxMTYgVVNCIE9w
dGljYWwgTW91c2VdIG9uIHVzYi0wMDA0OjAzOjAwLjAtMS9pbnB1dDAKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yODQ3OThdIHVzYiAyLTM6IG5ldyBTdXBlclNwZWVkIEdl
biAxIFVTQiBkZXZpY2UgbnVtYmVyIDIgdXNpbmcgeGhjaV9oY2QKU2VwIDEzIDEzOjA4OjE4IGRl
YmlhbiBrZXJuZWw6IFsgICAgMi4zMTMzNjZdIHVzYiAyLTM6IE5ldyBVU0IgZGV2aWNlIGZvdW5k
LCBpZFZlbmRvcj0wNWUzLCBpZFByb2R1Y3Q9MDYyMCwgYmNkRGV2aWNlPTkzLjEwClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzEzMzY4XSB1c2IgMi0zOiBOZXcgVVNCIGRl
dmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjMxMzM2OV0gdXNiIDItMzogUHJvZHVjdDogVVNC
My4xIEh1YgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjMxMzM3MF0gdXNi
IDItMzogTWFudWZhY3R1cmVyOiBHZW5lc3lzTG9naWMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMi4zMTYzODFdIGh1YiAyLTM6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAx
MzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzE2OTkyXSBodWIgMi0zOjEuMDogNCBwb3J0
cyBkZXRlY3RlZApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjQ0NDEyNF0g
dXNiIDEtMjogbmV3IGxvdy1zcGVlZCBVU0IgZGV2aWNlIG51bWJlciAzIHVzaW5nIHhoY2lfaGNk
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNjUyMDU0XSB1c2IgMS0yOiBO
ZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDRkOSwgaWRQcm9kdWN0PTEyMDMsIGJjZERl
dmljZT0gMi44MApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjY1MjA1NV0g
dXNiIDEtMjogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTAsIFByb2R1Y3Q9MCwgU2VyaWFs
TnVtYmVyPTAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMi42ODkxNzddIGlu
cHV0OiBISUQgMDRkOToxMjAzIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDowMDowMy4wLzAw
MDQ6MDM6MDAuMC91c2IxLzEtMi8xLTI6MS4wLzAwMDM6MDREOToxMjAzLjAwMDIvaW5wdXQvaW5w
dXQyClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzUyMjI4XSBoaWQtZ2Vu
ZXJpYyAwMDAzOjA0RDk6MTIwMy4wMDAyOiBpbnB1dCxoaWRyYXcxOiBVU0IgSElEIHYxLjExIEtl
eWJvYXJkIFtISUQgMDRkOToxMjAzXSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQwClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzk1MTgyXSBpbnB1dDogSElEIDA0ZDk6
MTIwMyBTeXN0ZW0gQ29udHJvbCBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8w
MDA0OjAzOjAwLjAvdXNiMS8xLTIvMS0yOjEuMS8wMDAzOjA0RDk6MTIwMy4wMDAzL2lucHV0L2lu
cHV0MwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjg1NjE0MV0gaW5wdXQ6
IEhJRCAwNGQ5OjEyMDMgQ29uc3VtZXIgQ29udHJvbCBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAw
MDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTIvMS0yOjEuMS8wMDAzOjA0RDk6MTIwMy4w
MDAzL2lucHV0L2lucHV0NApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAyLjg1
NjE2NV0gaGlkLWdlbmVyaWMgMDAwMzowNEQ5OjEyMDMuMDAwMzogaW5wdXQsaGlkcmF3MjogVVNC
IEhJRCB2MS4xMSBEZXZpY2UgW0hJRCAwNGQ5OjEyMDNdIG9uIHVzYi0wMDA0OjAzOjAwLjAtMi9p
bnB1dDEKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45NzU2NDZdIG1wdDNz
YXNfY20wOiBkaWFnIHJlc2V0OiBTVUNDRVNTClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDIuOTg4MTI3XSB1c2IgMS0zOiBuZXcgaGlnaC1zcGVlZCBVU0IgZGV2aWNlIG51bWJl
ciA0IHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
MDEwNjQ4XSBtcHQzc2FzX2NtMDogc2NhdHRlciBnYXRoZXI6IHNnZV9pbl9tYWluX21zZygxKSwg
c2dlX3Blcl9jaGFpbig3KSwgc2dlX3Blcl9pbygxMjgpLCBjaGFpbnNfcGVyX2lvKDE5KQpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjAxMTY5NF0gbXB0M3Nhc19jbTA6IHJl
cXVlc3QgcG9vbCgweChfX19fcHRydmFsX19fXykpIC0gZG1hKDB4ZmY1MDAwMDApOiBkZXB0aCg0
MjI0KSwgZnJhbWVfc2l6ZSgxMjgpLCBwb29sX3NpemUoNTI4IGtCKQpTZXAgMTMgMTM6MDg6MTgg
ZGViaWFuIGtlcm5lbDogWyAgICAzLjAzMTg3NF0gbXB0M3Nhc19jbTA6IHNlbnNlIHBvb2woMHgo
X19fX3B0cnZhbF9fX18pKS0gZG1hKDB4ZmViMDAwMDApOiBkZXB0aCgzOTYzKSxlbGVtZW50X3Np
emUoOTYpLCBwb29sX3NpemUoMzcxIGtCKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAzLjAzMTk3M10gbXB0M3Nhc19jbTA6IGNvbmZpZyBwYWdlKDB4KF9fX19wdHJ2YWxfX19f
KSkgLSBkbWEoMHhmZTlmNjAwMCk6IHNpemUoNTEyKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAzLjAzMTk3NF0gbXB0M3Nhc19jbTA6IEFsbG9jYXRlZCBwaHlzaWNhbCBtZW1v
cnk6IHNpemUoNzI3OCBrQikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4w
MzE5NzVdIG1wdDNzYXNfY20wOiBDdXJyZW50IENvbnRyb2xsZXIgUXVldWUgRGVwdGgoMzk2MCks
TWF4IENvbnRyb2xsZXIgUXVldWUgRGVwdGgoNDA5NikKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4wMzE5NzZdIG1wdDNzYXNfY20wOiBTY2F0dGVyIEdhdGhlciBFbGVtZW50
cyBwZXIgSU8oMTI4KQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjE0NTU5
OF0gdXNiIDEtMzogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTA1ZTMsIGlkUHJvZHVj
dD0wNjEwLCBiY2REZXZpY2U9OTMuMTAKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy4xNDU2MDBdIHVzYiAxLTM6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0xLCBQcm9k
dWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAg
IDMuMTQ1NjAxXSB1c2IgMS0zOiBQcm9kdWN0OiBVU0IyLjAgSHViClNlcCAxMyAxMzowODoxOCBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuMTQ1NjAyXSB1c2IgMS0zOiBNYW51ZmFjdHVyZXI6IEdlbmVz
eXNMb2dpYwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjE0OTYxOV0gaHVi
IDEtMzoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy4xNTAyMjVdIGh1YiAxLTM6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjMyNzc2XSBtcHQzc2FzX2NtMDogX2Jhc2VfZGlzcGxh
eV9md3BrZ192ZXJzaW9uOiBjb21wbGV0ZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDog
WyAgICAzLjIzMzEwM10gbXB0M3Nhc19jbTA6IExTSVNBUzMwMDg6IEZXVmVyc2lvbigxNi4wMC4x
MC4wMCksIENoaXBSZXZpc2lvbigweDAyKSwgQmlvc1ZlcnNpb24oMTUuMDAuMDEuMDApClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjMzMTA0XSBtcHQzc2FzX2NtMDogUHJv
dG9jb2w9KEluaXRpYXRvciksIENhcGFiaWxpdGllcz0oUmFpZCxUTFIsRUVEUCxTbmFwc2hvdCBC
dWZmZXIsRGlhZyBUcmFjZSBCdWZmZXIsVGFzayBTZXQgRnVsbCxOQ1EpClNlcCAxMyAxMzowODox
OCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjMzMTUzXSBzY3NpIGhvc3QwOiBGdXNpb24gTVBUIFNB
UyBIb3N0ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjMzNzMxXSBtcHQz
c2FzX2NtMDogc2VuZGluZyBwb3J0IGVuYWJsZSAhIQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAzLjI2MDY5MV0gUE06IEltYWdlIG5vdCBmb3VuZCAoY29kZSAtMjIpClNlcCAx
MyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMzEzNzQwXSBFWFQ0LWZzIChudm1lM24x
cDIpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51
bGwpClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMzQzNzgyXSBOb3QgYWN0
aXZhdGluZyBNYW5kYXRvcnkgQWNjZXNzIENvbnRyb2wgYXMgL3NiaW4vdG9tb3lvLWluaXQgZG9l
cyBub3QgZXhpc3QuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTM0Mjc3
XSByYW5kb206IHN5c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyBy
ZWFkKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjUzNTAzMV0gcmFuZG9t
OiBzeXN0ZW1kOiB1bmluaXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MzUyMzRdIHJhbmRvbTogc3lzdGVt
ZDogdW5pbml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJlYWQpClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTUwODU2XSBmdXNlOiBpbml0IChBUEkgdmVyc2lv
biA3LjMyKQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjU1MjU5OF0gRVhU
NC1mcyAobnZtZTNuMXAyKTogcmUtbW91bnRlZC4gT3B0czogZXJyb3JzPXJlbW91bnQtcm8KU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42NDE5MjRdIGFjcGktdGFkIEFDUEkw
MDBFOjAwOiBVbnN1cHBvcnRlZCBjYXBhYmlsaXRpZXMKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy42ODI3MTZdIHhnZW5lLXNsaW1wcm8taHdtb24gQVBNQzBEMjk6MDA6IEFQ
TSBYLUdlbmUgU29DIEhXIG1vbml0b3IgZHJpdmVyIHJlZ2lzdGVyZWQKU2VwIDEzIDEzOjA4OjE4
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42ODU0MTFdIElQTUkgbWVzc2FnZSBoYW5kbGVyOiB2ZXJz
aW9uIDM5LjIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42ODgwMjFdIGlw
bWkgZGV2aWNlIGludGVyZmFjZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAz
LjY5NDM0OF0gaXBtaV9zaTogSVBNSSBTeXN0ZW0gSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMgMTM6
MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjY5NDc0OV0gaXBtaV9zaTogVW5hYmxlIHRvIGZp
bmQgYW55IFN5c3RlbSBJbnRlcmZhY2UocykKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy43NjYwNjhdIGlwbWlfc3NpZjogSVBNSSBTU0lGIEludGVyZmFjZSBkcml2ZXIKU2Vw
IDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy43NzAxMzFdIGlwbWlfc3NpZiBpMmMt
QU1QQzAwMDQ6MDA6IGlwbWlfc3NpZjogVHJ5aW5nIEFDUEktc3BlY2lmaWVkIFNTSUYgaW50ZXJm
YWNlIGF0IGkyYyBhZGRyZXNzIDB4MTAsIGFkYXB0ZXIgU3lub3BzeXMgRGVzaWduV2FyZSBJMkMg
YWRhcHRlciwgc2xhdmUgYWRkcmVzcyAweDIwClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVs
OiBbICAgIDMuNzc3NzA0XSBFWFQ0LWZzIChudm1lM24xcDUpOiBtb3VudGVkIGZpbGVzeXN0ZW0g
d2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51bGwpClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDMuNzc5MTk3XSBFWFQ0LWZzIChudm1lM24xcDMpOiBtb3VudGVkIGZp
bGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51bGwpClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODAyNDIyXSBFWFQ0LWZzIChudm1lM24xcDYpOiBt
b3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51bGwpClNl
cCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODA0MTM1XSBBZGRpbmcgMTAwMDQ0
NGsgc3dhcCBvbiAvZGV2L252bWUzbjFwNC4gIFByaW9yaXR5Oi0yIGV4dGVudHM6MSBhY3Jvc3M6
MTAwMDQ0NGsgU1NGUwpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjgzMjI4
NF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE4Njk4LjEyNDoyKTogYXBwYXJtb3I9IlNU
QVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9
ImxzYl9yZWxlYXNlIiBwaWQ9MTA4OSBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMzow
ODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODMyNzAzXSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0
KDE2MzE1MTg2OTguMTI0OjMpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVf
bG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2Utb29wc2xhc2giIHBp
ZD0xMDkyIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44MzQ0ODNdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxODY5OC4xMjQ6
NCk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1
bmNvbmZpbmVkIiBuYW1lPSJudmlkaWFfbW9kcHJvYmUiIHBpZD0xMDk1IGNvbW09ImFwcGFybW9y
X3BhcnNlciIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MzQ0ODddIGF1
ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxODY5OC4xMjQ6NSk6IGFwcGFybW9yPSJTVEFUVVMi
IG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJudmlk
aWFfbW9kcHJvYmUvL2ttb2QiIHBpZD0xMDk1IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEz
IDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MzU3NjhdIGF1ZGl0OiB0eXBlPTE0MDAg
YXVkaXQoMTYzMTUxODY5OC4xMjQ6Nik6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJv
ZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJyZW9mZmljZS14cGRmaW1w
b3J0IiBwaWQ9MTA5MSBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxMzowODoxOCBkZWJp
YW4ga2VybmVsOiBbICAgIDMuODM2MjUzXSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MTg2
OTguMTI4OjcpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJv
ZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2Utc2VuZGRvYyIgcGlkPTEwOTQgY29t
bT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAz
LjgzNjM5NF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTE4Njk4LjEyODo4KTogYXBwYXJt
b3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQi
IG5hbWU9Ii91c3IvYmluL21hbiIgcGlkPTEwOTMgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAg
MTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjgzNjM5N10gYXVkaXQ6IHR5cGU9MTQw
MCBhdWRpdCgxNjMxNTE4Njk4LjEyODo5KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJw
cm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im1hbl9maWx0ZXIiIHBpZD0x
MDkzIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6
IFsgICAgMy44MzYzOTldIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUxODY5OC4xMjg6MTAp
OiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5j
b25maW5lZCIgbmFtZT0ibWFuX2dyb2ZmIiBwaWQ9MTA5MyBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODQzMzc4XSBzYnNhLWd3ZHQg
c2JzYS1nd2R0LjA6IEluaXRpYWxpemVkIHdpdGggMTBzIHRpbWVvdXQgQCAyNTAwMDAwMCBIeiwg
YWN0aW9uPTAuClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODY4NjA5XSBj
cnlwdGQ6IG1heF9jcHVfcWxlbiBzZXQgdG8gMTAwMApTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtl
cm5lbDogWyAgICAzLjg3MTU5OF0gcHN0b3JlOiBVc2luZyBjcmFzaCBkdW1wIGNvbXByZXNzaW9u
OiBkZWZsYXRlClNlcCAxMyAxMzowODoxOCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODcxNjA1XSBw
c3RvcmU6IFJlZ2lzdGVyZWQgZWZpIGFzIHBlcnNpc3RlbnQgc3RvcmUgYmFja2VuZApTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjg3NzAzMl0gcmFuZG9tOiBjcm5nIGluaXQg
ZG9uZQpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjg3NzAzNV0gcmFuZG9t
OiA3IHVyYW5kb20gd2FybmluZyhzKSBtaXNzZWQgZHVlIHRvIHJhdGVsaW1pdGluZwpTZXAgMTMg
MTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAzLjg3NzMwNl0gaXBtaV9zaTogSVBNSSBTeXN0
ZW0gSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMgMTM6MDg6MTggZGViaWFuIGtlcm5lbDogWyAgICAz
Ljg3NzQ4N10gaXBtaV9zaTogVW5hYmxlIHRvIGZpbmQgYW55IFN5c3RlbSBJbnRlcmZhY2UocykK
U2VwIDEzIDEzOjA4OjE4IGRlYmlhbiBrZXJuZWw6IFsgICAgNC4wMzEwNjNdIGlwbWlfc3NpZiBp
MmMtQU1QQzAwMDQ6MDA6IElQTUkgbWVzc2FnZSBoYW5kbGVyOiBGb3VuZCBuZXcgQk1DIChtYW5f
aWQ6IDB4MDAzYzBhLCBwcm9kX2lkOiAweDAyMTIsIGRldl9pZDogMHgyMCkKU2VwIDEzIDEzOjA4
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgNS43ODg0MDRdIG1wdDNzYXNfY20wOiBob3N0X2FkZDog
aGFuZGxlKDB4MDAwMSksIHNhc19hZGRyKDB4NTAwMTQ4NTAwMDc3OWUzMCksIHBoeXMoOCkKU2Vw
IDEzIDEzOjA4OjIxIGRlYmlhbiBrZXJuZWw6IFsgICAgNy4wOTY2MjVdIGlnYiAwMDAzOjAyOjAw
LjEgZW5vMjogaWdiOiBlbm8yIE5JQyBMaW5rIGlzIFVwIDEwMDAgTWJwcyBGdWxsIER1cGxleCwg
RmxvdyBDb250cm9sOiBSWC9UWApTZXAgMTMgMTM6MDg6MjEgZGViaWFuIGtlcm5lbDogWyAgICA3
LjA5NjgwMl0gSVB2NjogQUREUkNPTkYoTkVUREVWX0NIQU5HRSk6IGVubzI6IGxpbmsgYmVjb21l
cyByZWFkeQpTZXAgMTMgMTM6MDg6MjUgZGViaWFuIGtlcm5lbDogWyAgIDEwLjkyODEzM10gbXB0
M3Nhc19jbTA6IHBvcnQgZW5hYmxlOiBTVUNDRVNTClNlcCAxMyAxMzoxMDozNiBkZWJpYW4ga2Vy
bmVsOiBbICAxNDMuNTM5MTIxXSBkZXZpY2UtbWFwcGVyOiB1ZXZlbnQ6IHZlcnNpb24gMS4wLjMK
U2VwIDEzIDEzOjEwOjM2IGRlYmlhbiBrZXJuZWw6IFsgIDE0My41MzkyMDVdIGRldmljZS1tYXBw
ZXI6IGlvY3RsOiA0LjQzLjAtaW9jdGwgKDIwMjAtMTAtMDEpIGluaXRpYWxpc2VkOiBkbS1kZXZl
bEByZWRoYXQuY29tClNlcCAxMyAxMzoxMDozOSBkZWJpYW4ga2VybmVsOiBbICAxNDYuNzM4MzE1
XSBTR0kgWEZTIHdpdGggQUNMcywgc2VjdXJpdHkgYXR0cmlidXRlcywgcmVhbHRpbWUsIHF1b3Rh
LCBubyBkZWJ1ZyBlbmFibGVkClNlcCAxMyAxMzoxMDozOSBkZWJpYW4ga2VybmVsOiBbICAxNDYu
NzQ1MDMzXSBKRlM6IG5UeEJsb2NrID0gODE5MiwgblR4TG9jayA9IDY1NTM2ClNlcCAxMyAxMzox
MDozOSBkZWJpYW4ga2VybmVsOiBbICAxNDYuNzcwNDY4XSBRTlg0IGZpbGVzeXN0ZW0gMC4yLjMg
cmVnaXN0ZXJlZC4KU2VwIDEzIDEzOjEwOjM5IGRlYmlhbiBrZXJuZWw6IFsgIDE0Ni44NDMwMzVd
IHJhaWQ2OiBuZW9ueDggICBnZW4oKSAxMjkzNCBNQi9zClNlcCAxMyAxMzoxMDozOSBkZWJpYW4g
a2VybmVsOiBbICAxNDYuOTExMDM1XSByYWlkNjogbmVvbng4ICAgeG9yKCkgMTAxMTUgTUIvcwpT
ZXAgMTMgMTM6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAgMTQ2Ljk3OTA0MV0gcmFpZDY6IG5lb254
NCAgIGdlbigpIDEyNzYwIE1CL3MKU2VwIDEzIDEzOjEwOjM5IGRlYmlhbiBrZXJuZWw6IFsgIDE0
Ny4wNDcwNDJdIHJhaWQ2OiBuZW9ueDQgICB4b3IoKSAxMDM2MiBNQi9zClNlcCAxMyAxMzoxMDo0
MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuMTE1MDQ2XSByYWlkNjogbmVvbngyICAgZ2VuKCkgMTIz
MTEgTUIvcwpTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtlcm5lbDogWyAgMTQ3LjE4MzA0OV0gcmFp
ZDY6IG5lb254MiAgIHhvcigpICA5NjI4IE1CL3MKU2VwIDEzIDEzOjEwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgIDE0Ny4yNTEwNTRdIHJhaWQ2OiBuZW9ueDEgICBnZW4oKSAgOTkzMiBNQi9zClNlcCAx
MyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuMzE5MDU3XSByYWlkNjogbmVvbngxICAg
eG9yKCkgIDg0NTYgTUIvcwpTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtlcm5lbDogWyAgMTQ3LjM4
NzA2Ml0gcmFpZDY6IGludDY0eDggIGdlbigpICA1OTU0IE1CL3MKU2VwIDEzIDEzOjEwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgIDE0Ny40NTUwNjVdIHJhaWQ2OiBpbnQ2NHg4ICB4b3IoKSAgMzU1MiBN
Qi9zClNlcCAxMyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuNTIzMDY2XSByYWlkNjog
aW50NjR4NCAgZ2VuKCkgIDY3ODggTUIvcwpTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtlcm5lbDog
WyAgMTQ3LjU5MTA3MV0gcmFpZDY6IGludDY0eDQgIHhvcigpICAzNTY4IE1CL3MKU2VwIDEzIDEz
OjEwOjQwIGRlYmlhbiBrZXJuZWw6IFsgIDE0Ny42NTkwNzRdIHJhaWQ2OiBpbnQ2NHgyICBnZW4o
KSAgNTc3MCBNQi9zClNlcCAxMyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuNzI3MDc5
XSByYWlkNjogaW50NjR4MiAgeG9yKCkgIDMxMDYgTUIvcwpTZXAgMTMgMTM6MTA6NDAgZGViaWFu
IGtlcm5lbDogWyAgMTQ3Ljc5NTA4NF0gcmFpZDY6IGludDY0eDEgIGdlbigpICA0NzM0IE1CL3MK
U2VwIDEzIDEzOjEwOjQwIGRlYmlhbiBrZXJuZWw6IFsgIDE0Ny44NjMwODddIHJhaWQ2OiBpbnQ2
NHgxICB4b3IoKSAgMjQ4NiBNQi9zClNlcCAxMyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAx
NDcuODYzMDg3XSByYWlkNjogdXNpbmcgYWxnb3JpdGhtIG5lb254OCBnZW4oKSAxMjkzNCBNQi9z
ClNlcCAxMyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuODYzMDg4XSByYWlkNjogLi4u
LiB4b3IoKSAxMDExNSBNQi9zLCBybXcgZW5hYmxlZApTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtl
cm5lbDogWyAgMTQ3Ljg2MzA4OV0gcmFpZDY6IHVzaW5nIG5lb24gcmVjb3ZlcnkgYWxnb3JpdGht
ClNlcCAxMyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuODY0OTg1XSB4b3I6IG1lYXN1
cmluZyBzb2Z0d2FyZSBjaGVja3N1bSBzcGVlZApTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtlcm5l
bDogWyAgMTQ3Ljg2NTU5N10gICAgOHJlZ3MgICAgICAgICAgIDogMTYxMzMgTUIvc2VjClNlcCAx
MyAxMzoxMDo0MCBkZWJpYW4ga2VybmVsOiBbICAxNDcuODY2MTA2XSAgICAzMnJlZ3MgICAgICAg
ICAgOiAxOTM2NyBNQi9zZWMKU2VwIDEzIDEzOjEwOjQwIGRlYmlhbiBrZXJuZWw6IFsgIDE0Ny44
NjY0NzBdICAgIGFybTY0X25lb24gICAgICA6IDI3MDgyIE1CL3NlYwpTZXAgMTMgMTM6MTA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgMTQ3Ljg2NjQ3MV0geG9yOiB1c2luZyBmdW5jdGlvbjogYXJtNjRf
bmVvbiAoMjcwODIgTUIvc2VjKQpTZXAgMTMgMTM6MTA6NDAgZGViaWFuIGtlcm5lbDogWyAgMTQ3
Ljg5MjgyMV0gQnRyZnMgbG9hZGVkLCBjcmMzMmM9Y3JjMzJjLWdlbmVyaWMKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEJvb3RpbmcgTGludXggb24gcGh5c2lj
YWwgQ1BVIDB4MDAwMDEyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIExpbnV4IHZlcnNpb24gNS4xMC4wLTgtYXJtNjQgKGRlYmlh
bi1rZXJuZWxAbGlzdHMuZGViaWFuLm9yZykgKGdjYy0xMCAoRGViaWFuIDEwLjIuMS02KSAxMC4y
LjEgMjAyMTAxMTAsIEdOVSBsZCAoR05VIEJpbnV0aWxzIGZvciBEZWJpYW4pIDIuMzUuMikgIzEg
U01QIERlYmlhbiA1LjEwLjQ2LTQgKDIwMjEtMDgtMDMpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEVGSSB2Mi43MCBieSBBbWVyaWNhbiBNZWdhdHJl
bmRzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEFD
UEkgMi4wPTB4ZWU5MDAwMDAgU01CSU9TIDMuMD0weGYxYjNmZjk4IEVTUlQ9MHhlOTRmNmY5OCBN
T0t2YXI9MHhlOTRkZjAwMCBSTkc9MHhmNzY2ZTgxOCBNRU1SRVNFUlZFPTB4ZTk2MWY2OTggClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IHNlZWRpbmcg
ZW50cm9weSBwb29sClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBzZWN1cmVib290OiBTZWN1cmUgYm9vdCBkaXNhYmxlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZXNydDogUmVzZXJ2aW5nIEVTUlQgc3BhY2UgZnJvbSAw
eDAwMDAwMDAwZTk0ZjZmOTggdG8gMHgwMDAwMDAwMGU5NGY2ZmQwLgpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRWFybHkgdGFibGUgY2hlY2tzdW0g
dmVyaWZpY2F0aW9uIGRpc2FibGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAwMDBFRTkwMDAwMCAwMDAwMjQgKHYwMiBBbXBl
cmUpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBY
U0RUIDB4MDAwMDAwMDBFRThGMDAwMCAwMDAwQTQgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAw
MDAgQU1JICAwMTAwMDAxMykKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IEZBQ1AgMHgwMDAwMDAwMEVFOEQwMDAwIDAwMDExNCAodjA2IEFtcGVyZSBB
bHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRFNEVCAweDAwMDAwMDAwRUU4NjAwMDAgMDE5QTk1
ICh2MDIgQW1wZXJlIEphZGUgICAgIDAwMDAwMDAxIElOVEwgMjAyMDA3MTcpClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBEQkcyIDB4MDAwMDAwMDBF
RThFMDAwMCAwMDAwNUMgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUw
OSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEdU
RFQgMHgwMDAwMDAwMEVFOEMwMDAwIDAwMDExMCAodjAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAw
MCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogU1NEVCAweDAwMDAwMDAwRUU4QjAwMDAgMDAwMDJEICh2MDIgQW1wZXJlIEFs
dHJhICAgIDAwMDAwMDAxIElOVEwgMjAxOTA1MDkpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGSURUIDB4MDAwMDAwMDBFRTg1MDAwMCAwMDAwOUMg
KHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQQ1IgMHgwMDAwMDAwMEVF
ODQwMDAwIDAwMDA1MCAodjAyIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDUwMDBG
KQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQkdS
VCAweDAwMDAwMDAwRUU4MzAwMDAgMDAwMDM4ICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5
IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBNQ0ZHIDB4MDAwMDAwMDBFRTgyMDAwMCAwMDAwQUMgKHYwMSBBbXBlcmUgQWx0
cmEgICAgMDAwMDAwMDEgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IElPUlQgMHgwMDAwMDAwMEVFODEwMDAwIDAwMDYxMCAo
djAwIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUFBUVCAweDAwMDAwMDAwRUU3
RjAwMDAgMDA2RTYwICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAwMTMp
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTTElU
IDB4MDAwMDAwMDBFRTdFMDAwMCAwMDAwMkQgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAg
QU1QLiAwMTAwMDAxMykKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IFNSQVQgMHgwMDAwMDAwMEVFN0QwMDAwIDAwMDZEMCAodjAzIEFtcGVyZSBBbHRy
YSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogQVBJQyAweDAwMDAwMDAwRUU4MDAwMDAgMDAxOUY0ICh2
MDUgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAzIEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBQQ0NUIDB4MDAwMDAwMDBFRTdD
MDAwMCAwMDA1NzYgKHYwMiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDMgQU1QLiAwMTAwMDAxMykK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFdTTVQg
MHgwMDAwMDAwMEVFN0IwMDAwIDAwMDAyOCAodjAxIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBB
TUkgIDAwMDEwMDEzKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogRlBEVCAweDAwMDAwMDAwRUU3QTAwMDAgMDAwMDQ0ICh2MDEgQUxBU0tBIEEgTSBJ
ICAgIDAxMDcyMDA5IEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUENSOiBjb25zb2xlOiBwbDAxMSxtbWlvMzIsMHgxMDAw
MDI2MDAwMDAsMTE1MjAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDg4MzAwMDAwLTB4ODgzZmZmZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFU
OiBOb2RlIDAgUFhNIDAgW21lbSAweDkwMDAwMDAwLTB4ZmZmZmZmZmZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAg
W21lbSAweDgwMDAwMDAwMDAwLTB4ODAwN2ZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDgw
MTAwMDAwMDAwLTB4ODBmZmZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBOVU1BOiBOT0RFX0RBVEEgW21lbSAweDgwZmY4MDg1YjAwLTB4ODBmZjgw
ODdmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBab25l
IHJhbmdlczoKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAg
RE1BICAgICAgW21lbSAweDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMGZmZmZmZmZmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEzMiAgICBlbXB0
eQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBOb3JtYWwg
ICBbbWVtIDB4MDAwMDAwMDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNb3ZhYmxlIHpvbmUgc3RhcnQgZm9y
IGVhY2ggbm9kZQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
RWFybHkgbWVtb3J5IG5vZGUgcmFuZ2VzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAw
MDAwODgzZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MDAwMDAwMC0weDAwMDAwMDAwOTFmZmZmZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDA5MjAwMDAwMC0weDAwMDAwMDAwOTNmZmZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDA5NDAwMDAwMC0weDAwMDAwMDAwZWUyZTZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTJlNzAw
MC0weDAwMDAwMDAwZWU3NDBmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0MTAwMC0weDAwMDAwMDAw
ZWU3NDRmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0NTAwMC0weDAwMDAwMDAwZWU3NGNmZmZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBlZTc0ZDAwMC0weDAwMDAwMDAwZWU3NGRmZmZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBlZTc0ZTAwMC0weDAwMDAwMDAwZWU3NGZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc1MDAwMC0w
eDAwMDAwMDAwZWU5NmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk3MDAwMC0weDAwMDAwMDAwZWU5
N2ZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk4MDAwMC0weDAwMDAwMDAwZjA2NGZmZmZdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDAwMDBmMDY1MDAwMC0weDAwMDAwMDAwZjc2NmZmZmZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBm
NzY3MDAwMC0weDAwMDAwMDAwZjc4NGZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmNzg1MDAwMC0weDAw
MDAwMDAwZjdmZGZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmN2ZlMDAwMC0weDAwMDAwMDAwZmZjOGVm
ZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM4ZjAwMC0weDAwMDAwMDAwZmZjOGZmZmZdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDAwMDBmZmM5MDAwMC0weDAwMDAwMDAwZmZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDAwMDAw
MDAwMC0weDAwMDAwODAwN2ZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDEwMDAwMDAwMC0weDAwMDAw
ODBmZmZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBJbml0bWVtIHNldHVwIG5vZGUgMCBbbWVtIDB4MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAwODBm
ZmZmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBP
biBub2RlIDAgdG90YWxwYWdlczogMTY3MTE5MzYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDcxNzIgcGFnZXMgdXNlZCBmb3IgbWVtbWFw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25l
OiAwIHBhZ2VzIHJlc2VydmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIERNQSB6b25lOiA0NTkwMDggcGFnZXMsIExJRk8gYmF0Y2g6NjMKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDI1Mzk1
MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDE2MjUyOTI4IHBhZ2VzLCBMSUZPIGJhdGNoOjYz
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbWE6IFJlc2Vy
dmVkIDY0IE1pQiBhdCAweDAwMDAwMDAwZmJjMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IHByb2JpbmcgZm9yIGNvbmR1aXQgbWV0aG9kIGZy
b20gQUNQSS4KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBz
Y2k6IFBTQ0l2MS4xIGRldGVjdGVkIGluIGZpcm13YXJlLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogVXNpbmcgc3RhbmRhcmQgUFNDSSB2MC4yIGZ1
bmN0aW9uIElEcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
cHNjaTogTUlHUkFURV9JTkZPX1RZUEUgbm90IHN1cHBvcnRlZC4KU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFNNQyBDYWxsaW5nIENvbnZlbnRpb24g
djEuMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEwMDAwIC0+IE5v
ZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgzMDAwMCAtPiBOb2RlIDAKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg2MDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg3MDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ODAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDgwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHg5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHhhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YjAwMDAgLT4g
Tm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHhjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YzAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGQwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHhkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4ZTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUwMTAwIC0+IE5vZGUg
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4ZjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEwMDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxMTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDExMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIwMDAwIC0+IE5vZGUg
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDEzMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTMwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDE0MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MTUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNTAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MTYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNzAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE3MDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgxODAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE5MDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTkwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxYTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFh
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDFjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWMwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDFkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MWUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZTAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MWYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMDAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MjEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgyMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIyMDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgyMzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIzMDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyNDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDI1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjUwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDI2MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4Mjcw
MDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNzAxMDAgLT4gTm9kZSAwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwZXJjcHU6IEVtYmVk
ZGVkIDMzIHBhZ2VzL2NwdSBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjgKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IHM5NTE5MiByODE5
MiBkMzE3ODQgdTEzNTE2OCBhbGxvYz0zMyo0MDk2ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMDAgWzBdIDAxIFswXSAwMiBbMF0g
MDMgWzBdIDA0IFswXSAwNSBbMF0gMDYgWzBdIDA3IApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDA4IFswXSAwOSBbMF0gMTAgWzBd
IDExIFswXSAxMiBbMF0gMTMgWzBdIDE0IFswXSAxNSAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSAxNiBbMF0gMTcgWzBdIDE4IFsw
XSAxOSBbMF0gMjAgWzBdIDIxIFswXSAyMiBbMF0gMjMgClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMjQgWzBdIDI1IFswXSAyNiBb
MF0gMjcgWzBdIDI4IFswXSAyOSBbMF0gMzAgWzBdIDMxIApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDMyIFswXSAzMyBbMF0gMzQg
WzBdIDM1IFswXSAzNiBbMF0gMzcgWzBdIDM4IFswXSAzOSAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA0MCBbMF0gNDEgWzBdIDQy
IFswXSA0MyBbMF0gNDQgWzBdIDQ1IFswXSA0NiBbMF0gNDcgClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNDggWzBdIDQ5IFswXSA1
MCBbMF0gNTEgWzBdIDUyIFswXSA1MyBbMF0gNTQgWzBdIDU1IApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDU2IFswXSA1NyBbMF0g
NTggWzBdIDU5IFswXSA2MCBbMF0gNjEgWzBdIDYyIFswXSA2MyAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA2NCBbMF0gNjUgWzBd
IDY2IFswXSA2NyBbMF0gNjggWzBdIDY5IFswXSA3MCBbMF0gNzEgClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNzIgWzBdIDczIFsw
XSA3NCBbMF0gNzUgWzBdIDc2IFswXSA3NyBbMF0gNzggWzBdIDc5IApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0
dXJlczogZGV0ZWN0ZWQ6IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRl
Y3RlZDogVmlydHVhbGl6YXRpb24gSG9zdCBFeHRlbnNpb25zClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGtlcm5lbCBwYWdlIHRhYmxl
IGlzb2xhdGlvbiBmb3JjZWQgT04gYnkgS0FTTFIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEtlcm5lbCBwYWdlIHRh
YmxlIGlzb2xhdGlvbiAoS1BUSSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEhhcmR3YXJlIGRpcnR5IGJpdCBtYW5h
Z2VtZW50ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUg
ZmVhdHVyZXM6IGRldGVjdGVkOiBTcGVjdHJlLXY0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhbHRl
cm5hdGl2ZXM6IHBhdGNoaW5nIGtlcm5lbCBjb2RlClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBCdWlsdCAxIHpvbmVsaXN0cywgbW9iaWxpdHkgZ3JvdXBpbmcg
b24uICBUb3RhbCBwYWdlczogMTY0NTA4MTIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIFBvbGljeSB6b25lOiBOb3JtYWwKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEtlcm5lbCBjb21tYW5kIGxpbmU6IEJPT1RfSU1BR0U9
L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1hcm02NCByb290PVVVSUQ9NmMwODMyN2MtNGJiZC00MzQx
LWE0MjctYTcyYmMxNTMzODk1IHJvIHF1aWV0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuIGluZGl2aWR1YWwgbWF4IGNwdSBj
b250cmlidXRpb246IDQwOTYgYnl0ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4gdG90YWwgY3B1X2V4dHJhIGNvbnRyaWJ1
dGlvbnM6IDMyMzU4NCBieXRlcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbiBtaW4gc2l6ZTogMTMxMDcyIGJ5dGVzClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZf
bGVuOiA1MjQyODggYnl0ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIHByaW50azogZWFybHkgbG9nIGJ1ZiBmcmVlOiAxMTkwMTYoOTAlKQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGVudHJ5IGNhY2hlIGhhc2ggdGFi
bGUgZW50cmllczogODM4ODYwOCAob3JkZXI6IDE0LCA2NzEwODg2NCBieXRlcywgbGluZWFyKQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSW5vZGUtY2FjaGUg
aGFzaCB0YWJsZSBlbnRyaWVzOiA0MTk0MzA0IChvcmRlcjogMTMsIDMzNTU0NDMyIGJ5dGVzLCBs
aW5lYXIpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBtZW0g
YXV0by1pbml0OiBzdGFjazpvZmYsIGhlYXAgYWxsb2M6b24sIGhlYXAgZnJlZTpvZmYKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNvZnR3YXJlIElPIFRMQjog
bWFwcGVkIFttZW0gMHgwMDAwMDAwMGVhMmU3MDAwLTB4MDAwMDAwMDBlZTJlNzAwMF0gKDY0TUIp
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNZW1vcnk6IDI1
NTE2MzJLLzY2ODQ3NzQ0SyBhdmFpbGFibGUgKDExNzc2SyBrZXJuZWwgY29kZSwgMjQzNksgcndk
YXRhLCA3MDA4SyByb2RhdGEsIDU0NDBLIGluaXQsIDU5OEsgYnNzLCAxNTg1NTQ0SyByZXNlcnZl
ZCwgNjU1MzZLIGNtYS1yZXNlcnZlZCkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIHJhbmRvbTogZ2V0X3JhbmRvbV91NjQgY2FsbGVkIGZyb20gX19rbWVtX2Nh
Y2hlX2NyZWF0ZSsweDM4LzB4NTYwIHdpdGggY3JuZ19pbml0PTAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNMVUI6IEhXYWxpZ249NjQsIE9yZGVyPTAtMywg
TWluT2JqZWN0cz0wLCBDUFVzPTgwLCBOb2Rlcz0xClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRpbmcgMzg1MzUgZW50cmllcyBpbiAx
NTEgcGFnZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGZ0
cmFjZTogYWxsb2NhdGVkIDE1MSBwYWdlcyB3aXRoIDUgZ3JvdXBzClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IEhpZXJhcmNoaWNhbCBSQ1UgaW1wbGVt
ZW50YXRpb24uClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBy
Y3U6IAlSQ1UgcmVzdHJpY3RpbmcgQ1BVcyBmcm9tIE5SX0NQVVM9MjU2IHRvIG5yX2NwdV9pZHM9
ODAuClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAJUnVkZSB2
YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gCVRyYWNpbmcgdmFyaWFudCBvZiBUYXNrcyBSQ1UgZW5hYmxlZC4K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogUkNVIGNh
bGN1bGF0ZWQgdmFsdWUgb2Ygc2NoZWR1bGVyLWVubGlzdG1lbnQgZGVsYXkgaXMgMjUgamlmZmll
cy4KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogQWRq
dXN0aW5nIGdlb21ldHJ5IGZvciByY3VfZmFub3V0X2xlYWY9MTYsIG5yX2NwdV9pZHM9ODAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE5SX0lSUVM6IDY0LCBu
cl9pcnFzOiA2NCwgcHJlYWxsb2NhdGVkIGlycXM6IDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBHSUM6IFVzaW5nIHNwbGl0IEVPSS9EZWFjdGl2
YXRlIG1vZGUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJ
Q3YzOiA2NzIgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gR0lDdjM6IDAgRXh0ZW5kZWQgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IERpc3RyaWJ1dG9y
IGhhcyBubyBSYW5nZSBTZWxlY3RvciBzdXBwb3J0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMTYgUFBJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IENQVTA6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMTIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVjMDAwMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDAg
LT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBT
UkFUOiBQWE0gMCAtPiBJVFMgMSAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAyIC0+IE5vZGUgMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRT
IDMgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBTUkFUOiBQWE0gMCAtPiBJVFMgNCAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA1IC0+IE5vZGUgMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4g
SVRTIDYgLT4gTm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNyAtPiBOb2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDQwMDAwLTB4MTAwMTAw
MDVmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRT
QDB4MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAyYzAwMDAg
KGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQwMDAwOiBhbGxvY2F0ZWQg
MzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDJkMDAwMCAoZmxhdCwgZXN6IDIsIHBz
eiA2NEssIHNociAxKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gSVRTIFttZW0gMHgxMDAxMDAwNjAwMDAtMHgxMDAxMDAwN2ZmZmZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDYwMDAwOiBh
bGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJmMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3og
NjRLLCBzaHIgMSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVj
dGlvbnMgQDgwMDAwMzAwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDA4
MDAwMC0weDEwMDEwMDA5ZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMg
QDgwMDAwMzIwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA4MDAw
MDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzMzAwMDAgKGZs
YXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGEwMDAwLTB4MTAwMTAwMGJmZmZmXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEw
MDEwMDBhMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzNTAwMDAgKGluZGlyZWN0
LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50
ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM2MDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNo
ciAxKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFtt
ZW0gMHgxMDAxMDAwYzAwMDAtMHgxMDAxMDAwZGZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAwOiBhbGxvY2F0ZWQg
ODE5MiBEZXZpY2VzIEA4MDAwMDM4MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIg
MSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAw
MDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgw
MDAwMzkwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDBlMDAwMC0weDEw
MDEwMDBmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IElUU0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwM2Iw
MDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBlMDAwMDogYWxsb2Nh
dGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzYzAwMDAgKGZsYXQsIGVzeiAy
LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTAwMDAwLTB4MTAwMTAwMTFmZmZmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEwMDAw
MDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzZTAwMDAgKGluZGlyZWN0LCBlc3ogOCwg
cHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENv
bGxlY3Rpb25zIEA4MDAwMDNmMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAx
MDAxMjAwMDAtMHgxMDAxMDAxM2ZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZp
Y2VzIEA4MDAwMDgxMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAx
MjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwODIwMDAw
IChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogdXNpbmcgTFBJIHByb3BlcnR5IHRhYmxlIEAweDAw
MDAwODAwMDA4MzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEdJQ3YzOiBDUFUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDg0MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
SSBHVERUOiBmb3VuZCAxIG1lbW9yeS1tYXBwZWQgdGltZXIgYmxvY2socykuClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhcmNoX3RpbWVyOiBjcDE1IGFuZCBt
bWlvIHRpbWVyKHMpIHJ1bm5pbmcgYXQgMjUuMDBNSHogKHBoeXMvcGh5cykuClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbG9ja3NvdXJjZTogYXJjaF9zeXNf
Y291bnRlcjogbWFzazogMHhmZmZmZmZmZmZmZmZmZiBtYXhfY3ljbGVzOiAweDVjNDA5MzliNSwg
bWF4X2lkbGVfbnM6IDQ0MDc5NTIwMjY0NiBucwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMV0gc2NoZWRfY2xvY2s6IDU2IGJpdHMgYXQgMjVNSHosIHJlc29sdXRp
b24gNDBucywgd3JhcHMgZXZlcnkgNDM5ODA0NjUxMTEwMG5zClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDUyXSBDb25zb2xlOiBjb2xvdXIgZHVtbXkgZGV2aWNlIDgw
eDI1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDc0XSBBQ1BJOiBD
b3JlIHJldmlzaW9uIDIwMjAwOTI1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMjY3XSBDYWxpYnJhdGluZyBkZWxheSBsb29wIChza2lwcGVkKSwgdmFsdWUgY2FsY3Vs
YXRlZCB1c2luZyB0aW1lciBmcmVxdWVuY3kuLiA1MC4wMCBCb2dvTUlQUyAobHBqPTEwMDAwMCkK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAyNjldIHBpZF9tYXg6IGRl
ZmF1bHQ6IDgxOTIwIG1pbmltdW06IDY0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDMwMl0gTFNNOiBTZWN1cml0eSBGcmFtZXdvcmsgaW5pdGlhbGl6aW5nClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzA3XSBZYW1hOiBkaXNhYmxlZCBi
eSBkZWZhdWx0OyBlbmFibGUgd2l0aCBzeXNjdGwga2VybmVsLnlhbWEuKgpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMyNF0gQXBwQXJtb3I6IEFwcEFybW9yIGluaXRp
YWxpemVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzI2XSBUT01P
WU8gTGludXggaW5pdGlhbGl6ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAzNjVdIE1vdW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMxMDcyIChvcmRlcjog
OCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDM4OV0gTW91bnRwb2ludC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDEzMTA3
MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDEyNDhdIHJjdTogSGllcmFyY2hpY2FsIFNSQ1UgaW1wbGVtZW50
YXRpb24uClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzkwXSBQbGF0
Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5MV0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAx
MDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDEzOTNdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVh
dGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzk0XSBQbGF0Zm9y
bSBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5NV0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAw
YzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDEzOTZdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVk
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzk4XSBQbGF0Zm9ybSBN
U0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMTM5OV0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAxMjAw
MDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDE0MDNdIFBDSS9NU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwNF0gUENJL01TSTogSVRTQDB4MTAw
MTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAxNDA1XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwODAwMDAgZG9tYWluIGNyZWF0ZWQK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDddIFBDSS9NU0k6IElU
U0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTQxMF0gUENJL01TSTogSVRTQDB4MTAwMTAwMGMwMDAwIGRvbWFpbiBj
cmVhdGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDEyXSBQQ0kv
TVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTNdIFBDSS9NU0k6IElUU0AweDEwMDEwMDEwMDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQx
NF0gUENJL01TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDE3XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAx
MDAwNDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDE0MThdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRl
ZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxOV0gZnNsLW1jIE1T
STogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAxNDIxXSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAg
ZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0
MjJdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyM10gZnNsLW1jIE1TSTogSVRTQDB4
MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAxNDI0XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNy
ZWF0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjZdIGZzbC1t
YyBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzMV0gUmVtYXBwaW5nIGFuZCBlbmFibGluZyBFRkkg
c2VydmljZXMuClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAzOTA2XSBz
bXA6IEJyaW5naW5nIHVwIHNlY29uZGFyeSBDUFVzIC4uLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNDQ5MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ1MDldIEdJQ3YzOiBDUFUxOiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDFhMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YzAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ1MjddIEdJQ3YzOiBDUFUxOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg1MDAwMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDU2MF0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDU2OV0gQ1BVMTogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMWEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNTI1NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUyNzBdIEdJQ3YzOiBDUFUy
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2NDAwMDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUyODddIEdJQ3YzOiBDUFUy
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg2MDAwMApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTMyM10gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTMzM10gQ1BVMjogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTU0Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU1NjBdIEdJQ3YzOiBD
UFUzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4NDAw
MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU1NzhdIEdJQ3YzOiBD
UFUzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg3MDAw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTYxMV0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTYyMF0gQ1BVMzogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTkyN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU5NDFdIEdJQ3Yz
OiBDUFU0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1
NDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU5NjBdIEdJQ3Yz
OiBDUFU0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg4
MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTk5NV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjAwNF0gQ1BVNDogQm9vdGVk
IHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjMzM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDYzNDldIEdJ
Q3YzOiBDUFU1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE4MDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA3NDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDYzNjddIEdJ
Q3YzOiBDUFU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MDg5MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjQwMl0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjQxMV0gQ1BVNTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjcyOF0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY3NDRd
IEdJQ3YzOiBDUFU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA2YzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDY3NjNd
IEdJQ3YzOiBDUFU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMDhhMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjc5OF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjgwOF0gQ1BVNjog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzEzM10gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDcx
NTFdIEdJQ3YzOiBDUFU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFlMDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA4YzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDcx
NzFdIEdJQ3YzOiBDUFU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDhiMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzIw
Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzIxNV0gQ1BV
NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzUyOV0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDc1NDJdIEdJQ3YzOiBDUFU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMDAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDNjMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NzU2MF0gR0lDdjM6IENQVTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwOGMwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3
NTk4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3NjA4XSBD
UFU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYTAwMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3OTMzXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVOQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNzk1MV0gR0lDdjM6IENQVTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDljMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNzk3MF0gR0lDdjM6IENQVTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwOGQwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA4MDAxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MDEx
XSBDUFU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MzI5XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVMTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDgzNDJdIEdJQ3YzOiBDUFUxMDogZm91bmQgcmVkaXN0cmlidXRvciBjMDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA0NDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDgzNjJdIEdJQ3YzOiBDUFUxMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA4ZTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDgzOThdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDg0MDldIENQVTEwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYzAwMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzMxXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDg3NTBdIEdJQ3YzOiBDUFUxMTogZm91bmQgcmVkaXN0cmlidXRvciAyNDAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTQwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA4NzY5XSBHSUN2MzogQ1BVMTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA4ODAyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA4ODEzXSBDUFUxMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjQw
MDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
OTEyN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEyClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA5MTM5XSBHSUN2MzogQ1BVMTI6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzQwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA5MTU5XSBHSUN2MzogQ1BVMTI6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA5MTk0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA5MjA2XSBDUFUxMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMDgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwOTUzOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEzClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NTU3XSBHSUN2MzogQ1BVMTM6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjAwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk0MDAwMApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTU3Nl0gR0lDdjM6IENQVTEzOiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkxMDAwMApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTYwOF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwOTYxOV0gQ1BVMTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDIwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDk5MzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTk0Nl0gR0lDdjM6IENQVTE0OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIGUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRjMDAwMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTk2OF0gR0lDdjM6IENQVTE0OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkyMDAwMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDAwNV0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDAxN10gQ1BVMTQ6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDBlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTAzMzNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDM1M10gR0lDdjM6IENQVTE1
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhYzAwMDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAzNzNdIEdJQ3YzOiBDUFUx
NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5MzAwMDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA0MDddIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA0MThdIENQVTE1OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwNzMwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3NDBdIEdJQ3Yz
OiBDUFUxNjogZm91bmQgcmVkaXN0cmlidXRvciAyMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAx
YzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3NjFdIEdJQ3Yz
OiBDUFUxNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5
NDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3OTldIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA4MTFdIENQVTE2OiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExMTMxXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMTcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTExNDJd
IEdJQ3YzOiBDUFUxNzogZm91bmQgcmVkaXN0cmlidXRvciA0MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAyNDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTExNjNd
IEdJQ3YzOiBDUFUxNzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5NTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEyMDFd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTEyMTNdIENQVTE3
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNTMyXSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMTgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTE1NDNdIEdJQ3YzOiBDUFUxODogZm91bmQgcmVkaXN0cmlidXRvciAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDE0MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTU2
NF0gR0lDdjM6IENQVTE4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDk2MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTYw
M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTYxNV0gQ1BV
MTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAwMDAwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTE5MzNdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFUxOQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMTk0NV0gR0lDdjM6IENQVTE5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDYwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDJjMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMTk2Nl0gR0lDdjM6IENQVTE5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDk3MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMjAwM10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjAx
Nl0gQ1BVMTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA2MDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIzMzNdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMjM0OF0gR0lDdjM6IENQVTIwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEzMDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA2MDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTIzNjhdIEdJQ3YzOiBDUFUyMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA5ODAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTI0MDNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTI0MTVdIENQVTIwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMzAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNzMy
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTI3NDhdIEdJQ3YzOiBDUFUyMTogZm91bmQgcmVkaXN0cmlidXRvciAx
YjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDEyNzY5XSBHSUN2MzogQ1BVMjE6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTkwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDEyODAxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDEyODEzXSBDUFUyMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MWIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMzEyOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIyClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMTQzXSBHSUN2MzogQ1BVMjI6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMTUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY4MDAwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzE2M10gR0lDdjM6IENQVTIyOiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlhMDAwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzE5OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMzIxMV0gQ1BVMjI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDE1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTM1MjddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMwpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzU0NF0gR0lDdjM6IENQVTIzOiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDFkMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4ODAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM1NjVdIEdJQ3YzOiBDUFUyMzogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5YjAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2MDBdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2MTJdIENQVTIzOiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAxZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDEzOTQyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjQKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM5NTddIEdJQ3YzOiBDUFUyNDog
Zm91bmQgcmVkaXN0cmlidXRvciAxMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTgwMDAwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzOTc3XSBHSUN2MzogQ1BVMjQ6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWMwMDAwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDEyXSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDI1XSBDUFUyNDogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDMzOF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTI1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MzU0XSBHSUN2Mzog
Q1BVMjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc4
MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDM3Nl0gR0lDdjM6
IENQVTI1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlk
MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQwOV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQyMl0gQ1BVMjU6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ3MzFdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFUyNgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDc0N10g
R0lDdjM6IENQVTI2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE3MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA3MDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ3Njld
IEdJQ3YzOiBDUFUyNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5ZTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ4MDRd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ4MTddIENQVTI2
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNzAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MTM2XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMjcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTUxNTRdIEdJQ3YzOiBDUFUyNzogZm91bmQgcmVkaXN0cmlidXRvciAxZjAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwOTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE1MTc1XSBHSUN2MzogQ1BVMjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwOWYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE1MjEwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MjIz
XSBDUFUyNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWYwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTUzM10gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI4ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1NTQ2XSBHSUN2MzogQ1BVMjg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYjAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwNDAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1NTY4XSBHSUN2MzogQ1BVMjg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1NjA1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE1NjE4XSBDUFUyODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGIwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTkzNF0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI5ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1OTUzXSBHSUN2MzogQ1BVMjk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjMw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNTk3NV0gR0lDdjM6IENQVTI5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGExMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNjAwNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNjAxOV0gQ1BVMjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIz
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTYzMzFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNjM0NV0gR0lDdjM6IENQVTMwOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIGQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ4MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNjM2Nl0gR0lDdjM6IENQVTMwOiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEyMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNjQwMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNjQxN10gQ1BVMzA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDBkMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTY3MzRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjc1NF0gR0lDdjM6IENQVTMxOiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDI1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhODAwMDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY3NzZdIEdJQ3YzOiBDUFUzMTogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMzAwMDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4MDldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4MjJdIENQVTMxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAyNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE3MTMzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzIKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcxNDVdIEdJQ3YzOiBDUFUzMjogZm91
bmQgcmVkaXN0cmlidXRvciA5MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzODAwMDAKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcxNjhdIEdJQ3YzOiBDUFUzMjogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNDAwMDAKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyMDRdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyMThdIENQVTMyOiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAwOTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE3NTM2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzMK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc1NTVdIEdJQ3YzOiBDUFUz
MzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTgwMDAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NTc4XSBHSUN2MzogQ1BV
MzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTUwMDAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NjEyXSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NjI3XSBDUFUzMzogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzkyOV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTM0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3OTQ0XSBHSUN2
MzogQ1BVMzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
NTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3OTY3XSBHSUN2
MzogQ1BVMzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YTYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MDA2XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MDIwXSBDUFUzNDogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODMzNV0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTM1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MzU1
XSBHSUN2MzogQ1BVMzU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjcwMDAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMGIwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODM3
OV0gR0lDdjM6IENQVTM1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGE3MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODQx
M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODQyOF0gQ1BV
MzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI3MDAwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg3MzJdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFUzNgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODc0M10gR0lDdjM6IENQVTM2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDMwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDIwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODc2OF0gR0lDdjM6IENQVTM2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGE4MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODgwNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODgy
MV0gQ1BVMzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAzMDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkxMzNdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTE0NV0gR0lDdjM6IENQVTM3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDUwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDI4MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTE2OF0gR0lDdjM6IENQVTM3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGE5MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTIwN10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxOTIyMV0gQ1BVMzc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA1MDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk1MzZd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTU0N10gR0lDdjM6IENQVTM4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE4MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTU3Ml0gR0lDdjM6IENQVTM4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFhMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTYxMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxOTYyNV0gQ1BVMzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAx
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTk5MzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTk0NF0gR0lDdjM6IENQVTM5OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDMwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTk2OV0gR0lDdjM6IENQVTM5OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFiMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMDAwNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyMDAyMl0gQ1BVMzk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDA3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjAzMjldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDM0NF0gR0lDdjM6IENQVTQwOiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDEyMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1ZTAwMDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAzNjZdIEdJQ3YzOiBDUFU0MDogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYzAwMDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAzOTldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0MTRdIENQVTQwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAxMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIwNzI5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDEKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA3NDZdIEdJQ3YzOiBDUFU0MTogZm91
bmQgcmVkaXN0cmlidXRvciAxYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwN2UwMDAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNzY5XSBHSUN2MzogQ1BVNDE6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYWQwMDAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODAyXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODE3XSBDUFU0MTogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMWEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMTEzNF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQy
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMTQ5XSBHSUN2MzogQ1BV
NDI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY2MDAw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTE3M10gR0lDdjM6IENQ
VTQyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFlMDAw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTIwOV0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTIyNF0gQ1BVNDI6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE1MzddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFU0MwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTU1NF0gR0lD
djM6IENQVTQzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA4NjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE1NzhdIEdJ
Q3YzOiBDUFU0MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBhZjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2MTFdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2MjZdIENQVTQzOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYzAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxOTMyXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNDQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE5
NDZdIEdJQ3YzOiBDUFU0NDogZm91bmQgcmVkaXN0cmlidXRvciAxMDAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNTYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIx
OTcyXSBHSUN2MzogQ1BVNDQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYjAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIy
MDA3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMDIyXSBD
UFU0NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjMzN10gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTQ1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIyMzUzXSBHSUN2MzogQ1BVNDU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTgwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDc2MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMjM3Nl0gR0lDdjM6IENQVTQ1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGIxMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMjQwOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
MjQyNF0gQ1BVNDU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE4MDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI3MzZdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMjc1Ml0gR0lDdjM6IENQVTQ2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA2ZTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjI3NzddIEdJQ3YzOiBDUFU0NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjI4MTNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjI4MzBdIENQVTQ2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNjAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIz
MTM1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDcKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjMxNTNdIEdJQ3YzOiBDUFU0NzogZm91bmQgcmVkaXN0cmlidXRv
ciAxZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGUwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIzMTc3XSBHSUN2MzogQ1BVNDc6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjMwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIzMjEyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIzMjI3XSBDUFU0NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMWUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMzUzNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ4ClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNTQ5XSBHSUN2MzogQ1BVNDg6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2UwMDAwClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNTc0XSBHSUN2MzogQ1BVNDg6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjQwMDAwClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjExXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIzNjI3XSBDUFU0ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMGEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMzkzOV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ5ClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzOTU3XSBHSUN2MzogQ1BVNDk6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMjIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDllMDAwMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzk4M10gR0lDdjM6IENQVTQ5OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI1MDAwMApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDAxNl0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDAzMl0gQ1BVNDk6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDIyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjQzMzRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDM0OF0gR0lDdjM6IENQVTUw
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ2MDAwMApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDM3M10gR0lDdjM6IENQVTUw
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI2MDAwMApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQwOF0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQyNV0gQ1BVNTA6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ3NDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFU1MQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDc2Ml0gR0lDdjM6
IENQVTUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBh
NjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ3ODddIEdJQ3Yz
OiBDUFU1MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBi
NzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4MjBdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4MzZdIENQVTUxOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1MTM2XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNTIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUxNDhd
IEdJQ3YzOiBDUFU1MjogZm91bmQgcmVkaXN0cmlidXRvciA4MDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAzNjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUxNzRd
IEdJQ3YzOiBDUFU1MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBiODAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUyMTJd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjUyMjldIENQVTUy
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwODAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NTQ0XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjU1NjJdIEdJQ3YzOiBDUFU1MzogZm91bmQgcmVkaXN0cmlidXRvciAyMDAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwOTYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI1NTg4XSBHSUN2MzogQ1BVNTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYjkwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI1NjIyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NjM5
XSBDUFU1MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjAwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTkzN10gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI1OTUxXSBHSUN2MzogQ1BVNTQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZTAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwNGUwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI1OTc5XSBHSUN2MzogQ1BVNTQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYmEwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI2MDE2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI2MDMzXSBDUFU1NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGUwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjM0MV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI2MzYyXSBHSUN2MzogQ1BVNTU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjYw
MTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFlMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNjM4N10gR0lDdjM6IENQVTU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJiMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNjQyM10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNjQ0MF0gQ1BVNTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI2
MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjY3MzldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NgpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjc0OV0gR0lDdjM6IENQVTU2OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFlMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjc3Nl0gR0lDdjM6IENQVTU2OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJjMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjgxNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNjgzMl0gQ1BVNTY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDAyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjcxMzBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NwpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzE0MV0gR0lDdjM6IENQVTU3OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI2MDAwMApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzE2OF0gR0lDdjM6IENQVTU3OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJkMDAwMApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzIwNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNzIyM10gQ1BVNTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDA0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjc1MzFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1OApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzU0MV0gR0lDdjM6IENQVTU4OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxNjAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc1NjldIEdJQ3YzOiBDUFU1ODogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZTAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2MDddIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2MjVdIENQVTU4OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAwMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI3OTMyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTkKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc5NDNdIEdJQ3YzOiBDUFU1OTog
Zm91bmQgcmVkaXN0cmlidXRvciA2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyZTAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc5NzBdIEdJQ3YzOiBDUFU1OTog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZjAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwMDZdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwMjVdIENQVTU5OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MzMwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
NjAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgzNDZdIEdJQ3YzOiBD
UFU2MDogZm91bmQgcmVkaXN0cmlidXRvciAxMzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjIw
MDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MzcyXSBHSUN2Mzog
Q1BVNjA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzAw
MDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDA5XSBhcmNoX3Rp
bWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDI3XSBDUFU2MDogQm9vdGVk
IHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODczNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTYxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NzUzXSBH
SUN2MzogQ1BVNjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDgyMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODc3OV0g
R0lDdjM6IENQVTYxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGMxMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODgxMV0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODgyOF0gQ1BVNjE6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkxMzZdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU2MgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
OTE1Ml0gR0lDdjM6IENQVTYyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA2YTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjkxNzldIEdJQ3YzOiBDUFU2MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBjMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjkyMTRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkyMzFd
IENQVTYyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NTM3XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNjMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjk1NTVdIEdJQ3YzOiBDUFU2MzogZm91bmQgcmVkaXN0cmlidXRvciAxZDAxMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwOGEwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5NTgxXSBHSUN2MzogQ1BVNjM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwYzMwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI5NjE1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5NjMyXSBDUFU2MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWQwMTAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTkzOV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI5OTU0XSBHSUN2MzogQ1BVNjQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTEw
MTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVhMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyOTk4MV0gR0lDdjM6IENQVTY0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM0MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzMDAxNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzMDAzM10gQ1BVNjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEx
MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzAzNDJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2NQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAzMDM1OV0gR0lDdjM6IENQVTY1OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDE5MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzAzODVdIEdJQ3YzOiBDUFU2NTogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0MjBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzA0MzddIENQVTY1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDMwNzQwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjYKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA3NTddIEdJQ3YzOiBDUFU2NjogZm91bmQgcmVk
aXN0cmlidXRvciAxNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzIwMDAwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNzg0XSBHSUN2MzogQ1BVNjY6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzYwMDAwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODE5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODM2XSBDUFU2NjogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMTcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzMTEzN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY3ClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMTU1XSBHSUN2MzogQ1BVNjc6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMWYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDkyMDAwMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTE4Ml0gR0lDdjM6IENQVTY3OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM3MDAwMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTIxN10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTIzNV0gQ1BVNjc6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDFmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzE1MzZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2
OApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTU0OV0gR0lDdjM6IENQ
VTY4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQyMDAw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTU3Nl0gR0lDdjM6IENQ
VTY4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM4MDAw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTYxM10gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTYzMF0gQ1BVNjg6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBiMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE5NDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFU2OQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTk2Ml0gR0lD
djM6IENQVTY5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDBhMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE5OTBdIEdJ
Q3YzOiBDUFU2OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBjOTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwMjNdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwNDBdIENQVTY5OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyMzM2XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNzAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIz
NTBdIEdJQ3YzOiBDUFU3MDogZm91bmQgcmVkaXN0cmlidXRvciBkMDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA0YTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIz
NzhdIEdJQ3YzOiBDUFU3MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBjYTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI0
MTVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI0MzJdIENQ
VTcwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNzMyXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVNzEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzI3NTFdIEdJQ3YzOiBDUFU3MTogZm91bmQgcmVkaXN0cmlidXRvciAyNTAxMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwYWEwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMyNzgwXSBHSUN2MzogQ1BVNzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwY2IwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMyODExXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMy
ODI4XSBDUFU3MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjUwMTAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzEyOV0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMzMTQyXSBHSUN2MzogQ1BVNzI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgOTAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwM2EwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMzMTcwXSBHSUN2MzogQ1BVNzI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2MwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMzMjA4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMzMjI2XSBDUFU3MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDkwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzUz
Nl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTczClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMzNTU1XSBHSUN2MzogQ1BVNzM6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MjEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDlhMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMzU4NF0gR0lDdjM6IENQVTczOiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNkMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMzYxN10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMzYzNF0gQ1BVNzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDIxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzM5MzRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzk0OF0gR0lDdjM6IENQVTc0OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIGYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUyMDAwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzk3OF0gR0lDdjM6IENQVTc0OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNlMDAwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDAxNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNDAzM10gQ1BVNzQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDBmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzQzMzldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDM2MF0gR0lDdjM6IENQVTc1OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDI3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBiMjAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQzODldIEdJQ3YzOiBDUFU3NTogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjZjAwMDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0MjRdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0NDNdIENQVTc1OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAyNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM0NzQxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzYKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ3NTJdIEdJQ3YzOiBDUFU3Njog
Zm91bmQgcmVkaXN0cmlidXRvciAzMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyMjAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ3ODJdIEdJQ3YzOiBDUFU3Njog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMDAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4MjBdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4MzhdIENQVTc2OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAwMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MTMxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
NzcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUxNDNdIEdJQ3YzOiBD
UFU3NzogZm91bmQgcmVkaXN0cmlidXRvciA1MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyYTAw
MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUxNzNdIEdJQ3YzOiBD
UFU3NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMTAw
MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUyMTFdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUyMjldIENQVTc3OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NTM3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNzgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1NDddIEdJ
Q3YzOiBDUFU3ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAxYTAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU1NzhdIEdJ
Q3YzOiBDUFU3ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBkMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU2MTZdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU2MzVdIENQVTc4OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1OTM3XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNzkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5
NDldIEdJQ3YzOiBDUFU3OTogZm91bmQgcmVkaXN0cmlidXRvciA3MDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDAzMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU5
NzhdIEdJQ3YzOiBDUFU3OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBkMzAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYw
MTddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYwMzZdIENQ
VTc5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDk5XSBzbXA6IEJyb3Vn
aHQgdXAgMSBub2RlLCA4MCBDUFVzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2MTIxXSBTTVA6IFRvdGFsIG9mIDgwIHByb2Nlc3NvcnMgYWN0aXZhdGVkLgpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjEyMl0gQ1BVIGZlYXR1cmVzOiBkZXRl
Y3RlZDogUHJpdmlsZWdlZCBBY2Nlc3MgTmV2ZXIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzYxMjNdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IExTRSBhdG9taWMgaW5z
dHJ1Y3Rpb25zClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MTIzXSBD
UFUgZmVhdHVyZXM6IGRldGVjdGVkOiBVc2VyIEFjY2VzcyBPdmVycmlkZQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjEyNF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDog
MzItYml0IEVMMCBTdXBwb3J0ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM2MTI1XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDb21tb24gbm90IFByaXZhdGUgdHJhbnNs
YXRpb25zClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MTI2XSBDUFUg
ZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIFBvaW50IG9mIFBlcnNpc3Rl
bmNlClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MTI3XSBDUFUgZmVh
dHVyZXM6IGRldGVjdGVkOiBSQVMgRXh0ZW5zaW9uIFN1cHBvcnQKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzYxMjddIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IERhdGEg
Y2FjaGUgY2xlYW4gdG8gdGhlIFBvVSBub3QgcmVxdWlyZWQgZm9yIEkvRCBjb2hlcmVuY2UKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYxMjhdIENQVSBmZWF0dXJlczog
ZGV0ZWN0ZWQ6IENSQzMyIGluc3RydWN0aW9ucwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNjEyOV0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3BlY3VsYXRpdmUgU3Rv
cmUgQnlwYXNzaW5nIFNhZmUgKFNTQlMpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDU4NTc2XSBDUFU6IEFsbCBDUFUocykgc3RhcnRlZCBhdCBFTDIKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDQ0NTRdIG5vZGUgMCBkZWZlcnJlZCBwYWdlcyBp
bml0aWFsaXNlZCBpbiA0ODRtcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjU0NDYwNF0gZGV2dG1wZnM6IGluaXRpYWxpemVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTQ1MTc1XSBSZWdpc3RlcmVkIGNwMTVfYmFycmllciBlbXVsYXRpb24gaGFu
ZGxlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NTE5Nl0gUmVnaXN0
ZXJlZCBzZXRlbmQgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDUxOThdIEtBU0xSIGVuYWJsZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NDUyNTZdIGNsb2Nrc291cmNlOiBqaWZmaWVzOiBtYXNrOiAweGZmZmZm
ZmZmIG1heF9jeWNsZXM6IDB4ZmZmZmZmZmYsIG1heF9pZGxlX25zOiA3NjQ1MDQxNzg1MTAwMDAw
IG5zClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ1MzA4XSBmdXRleCBo
YXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOSwgMjA5NzE1MiBieXRlcywgbGluZWFy
KQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NTczM10gcGluY3RybCBj
b3JlOiBpbml0aWFsaXplZCBwaW5jdHJsIHN1YnN5c3RlbQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjU0NTg5M10gU01CSU9TIDMuMy4wIHByZXNlbnQuClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ1ODk2XSBETUk6IEdJR0FCWVRFIFIxNTItUDMw
LTAwL01QMzItQVIxLTAwLCBCSU9TIEYxMyAoU0NQOiAxLjUuMjAyMTA0MjYpIDA1LzIxLzIwMjEK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDYxMjhdIE5FVDogUmVnaXN0
ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC41NDY3MTJdIERNQTogcHJlYWxsb2NhdGVkIDQwOTYgS2lCIEdGUF9LRVJORUwgcG9vbCBm
b3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNTQ2ODM4XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VSTkVMfEdGUF9ETUEg
cG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNTQ2OTYyXSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VSTkVMfEdG
UF9ETUEzMiBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlvbnMKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC41NDY5NjldIGF1ZGl0OiBpbml0aWFsaXppbmcgbmV0bGluayBzdWJz
eXMgKGRpc2FibGVkKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzAx
MV0gYXVkaXQ6IHR5cGU9MjAwMCBhdWRpdCgwLjU0NDoxKTogc3RhdGU9aW5pdGlhbGl6ZWQgYXVk
aXRfZW5hYmxlZD0wIHJlcz0xClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NTQ3MDk2XSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVybm9yICdmYWlyX3No
YXJlJwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzA5N10gdGhlcm1h
bF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnYmFuZ19iYW5nJwpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzA5OV0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVy
ZWQgdGhlcm1hbCBnb3Zlcm5vciAnc3RlcF93aXNlJwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU0NzA5OV0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zl
cm5vciAndXNlcl9zcGFjZScKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41
NDcxMDBdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3Bvd2VyX2Fs
bG9jYXRvcicKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDcxNTRdIGNw
dWlkbGU6IHVzaW5nIGdvdmVybm9yIGxhZGRlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU0NzE2MV0gY3B1aWRsZTogdXNpbmcgZ292ZXJub3IgbWVudQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzI1NF0gRGV0ZWN0ZWQgMTUgUENDIFN1YnNw
YWNlcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzI2OV0gUmVnaXN0
ZXJpbmcgUENDIGRyaXZlciBhcyBNYWlsYm94IGNvbnRyb2xsZXIKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC41NDcyODhdIGh3LWJyZWFrcG9pbnQ6IGZvdW5kIDYgYnJlYWtw
b2ludCBhbmQgNCB3YXRjaHBvaW50IHJlZ2lzdGVycy4KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NDc2ODZdIEFTSUQgYWxsb2NhdG9yIGluaXRpYWxpc2VkIHdpdGggMzI3
NjggZW50cmllcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzcwOF0g
QUNQSTogYnVzIHR5cGUgUENJIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDc3MTBdIGFjcGlwaHA6IEFDUEkgSG90IFBsdWcgUENJIENvbnRyb2xsZXIg
RHJpdmVyIHZlcnNpb246IDAuNQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjU0Nzc0N10gU2VyaWFsOiBBTUJBIFBMMDExIFVBUlQgZHJpdmVyClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNTUwMDkxXSBIdWdlVExCIHJlZ2lzdGVyZWQgMS4wMCBHaUIg
cGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NTAwOTJdIEh1Z2VUTEIgcmVnaXN0ZXJlZCAzMi4wIE1pQiBwYWdlIHNp
emUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjU1MDA5M10gSHVnZVRMQiByZWdpc3RlcmVkIDIuMDAgTWlCIHBhZ2Ugc2l6ZSwgcHJl
LWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NTUwMDk0XSBIdWdlVExCIHJlZ2lzdGVyZWQgNjQuMCBLaUIgcGFnZSBzaXplLCBwcmUtYWxsb2Nh
dGVkIDAgcGFnZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42NzczNDNd
IEFDUEk6IEFkZGVkIF9PU0koTW9kdWxlIERldmljZSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42NzczNDRdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIERldmljZSkK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42NzczNDVdIEFDUEk6IEFkZGVk
IF9PU0koMy4wIF9TQ1AgRXh0ZW5zaW9ucykKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42NzczNDZdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIEFnZ3JlZ2F0b3IgRGV2
aWNlKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY3NzM0N10gQUNQSTog
QWRkZWQgX09TSShMaW51eC1EZWxsLVZpZGVvKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY3NzM0OF0gQUNQSTogQWRkZWQgX09TSShMaW51eC1MZW5vdm8tTlYtSERNSS1B
dWRpbykKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42NzczNDldIEFDUEk6
IEFkZGVkIF9PU0koTGludXgtSFBJLUh5YnJpZC1HcmFwaGljcykKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42ODQzMjZdIEFDUEk6IDIgQUNQSSBBTUwgdGFibGVzIHN1Y2Nl
c3NmdWxseSBhY3F1aXJlZCBhbmQgbG9hZGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjg1Njc4XSBBQ1BJOiBJbnRlcnByZXRlciBlbmFibGVkClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg1Njc5XSBBQ1BJOiBVc2luZyBHSUMgZm9yIGludGVy
cnVwdCByb3V0aW5nClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg1Njg3
XSBBQ1BJOiBNQ0ZHIHRhYmxlIGRldGVjdGVkLCA4IGVudHJpZXMKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTQ0MDNdIEFSTUgwMDExOjAwOiB0dHlBTUEwIGF0IE1NSU8g
MHgxMDAwMDI2MDAwMDAgKGlycSA9IDM2LCBiYXNlX2JhdWQgPSAwKSBpcyBhIFNCU0EKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTQ0NDZdIHByaW50azogY29uc29sZSBb
dHR5QU1BMF0gZW5hYmxlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
NTU1M10gQVJNSDAwMTE6MDE6IHR0eUFNQTEgYXQgTU1JTyAweDEwMDAwMjYyMDAwMCAoaXJxID0g
MzcsIGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5NjYxMV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kwXSAoZG9tYWluIDAw
MGMgW2J1cyAwMC1mZl0pClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk2
NjE4XSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBB
U1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC42OTY2ODZdIGFjcGkgUE5QMEEwODowMDogX09TQzogcGxhdGZvcm0g
ZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTY3NDVdIGFjcGkgUE5QMEEwODowMDog
X09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk2NzUwXSBhY3BpIFBOUDBBMDg6MDA6IE1DRkcgcXVp
cms6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9yIFtidXMg
MDAtZmZdIHdpdGggMHhmZmZmYmM0Yzk0ZjRlYTU4ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjk3Mzc0XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXJlYSBbbWVtIDB4MzNm
ZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3MzgyXSBhY3BpIFBOUDBBMDg6MDA6IEVD
QU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3NDQ4XSBQQ0kgaG9zdCBi
cmlkZ2UgdG8gYnVzIDAwMGM6MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC42OTc0NDldIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDQwMDAw
MDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjY5NzQ1MF0gcGNpX2J1cyAwMDBjOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzAw
MDAwMDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTc0NTFdIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W2J1cyAwMC1mZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc0NzJd
IHBjaSAwMDBjOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc1MTldIHBjaSAwMDBjOjAwOjAx
LjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTc1MjddIHBjaSAwMDBjOjAwOjAxLjA6IGVuYWJsaW5nIEV4
dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc1NDNd
IHBjaSAwMDBjOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk3NTQzXSBwY2kgMDAwYzowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3
NjIwXSBhY3BpcGhwOiBTbG90IFsxXSByZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk3NjMzXSBwY2kgMDAwYzowMTowMC4wOiBbMTAwMDowMDk3XSB0eXBl
IDAwIGNsYXNzIDB4MDEwNzAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk3NjQyXSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxMDogW2lvICAweDRmZmY4MDAwLTB4NGZm
ZjgwZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3NjUwXSBwY2kg
MDAwYzowMTowMC4wOiByZWcgMHgxNDogW21lbSAweDQwMTQwMDAwLTB4NDAxNGZmZmYgNjRiaXRd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3NjU3XSBwY2kgMDAwYzow
MTowMC4wOiByZWcgMHgxYzogW21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYgNjRiaXRdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3NjY3XSBwY2kgMDAwYzowMTowMC4w
OiByZWcgMHgzMDogW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc2NzFdIHBjaSAwMDBjOjAxOjAwLjA6IGVuYWJs
aW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42
OTc3NjZdIHBjaSAwMDBjOjAxOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODM3XSBwY2lfYnVzIDAwMGM6MDA6IG9uIE5VTUEgbm9k
ZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODQxXSBwY2kgMDAw
YzowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0
IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODQ0XSBwY2kgMDAwYzowMDowMS4w
OiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg0Nl0gcGNpIDAwMGM6MDA6MDEuMDogQkFS
IDE1OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAwMWZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODQ3XSBwY2kgMDAw
YzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg0OF0gcGNpIDAwMGM6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODQ5XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5Nzg1MF0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8g
YXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNjk3ODUyXSBwY2kgMDAwYzowMTowMC4wOiBCQVIgNjogYXNzaWduZWQgW21lbSAweDQw
MDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTc4NTNdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAzOiBhc3NpZ25lZCBbbWVtIDB4NDAx
MDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTc4NThdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAxOiBhc3NpZ25lZCBbbWVtIDB4NDAx
NDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTc4NjRdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBubyBzcGFjZSBmb3IgW2lvICBz
aXplIDB4MDEwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc4NjRd
IHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAx
MDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODY2XSBwY2kgMDAw
YzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk3ODY3XSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk3ODY4XSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDMwMDAwMDAwMDAwMC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg3MF0gcGNpX2J1cyAwMDBjOjAwOiBTb21lIFBDSSBk
ZXZpY2UgcmVzb3VyY2VzIGFyZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGluZyB3aXRoIHBjaT1yZWFs
bG9jClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3ODcxXSBwY2lfYnVz
IDAwMGM6MDA6IHJlc291cmNlIDQgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg3Ml0gcGNpX2J1cyAwMDBj
OjAwOiByZXNvdXJjZSA1IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2ZmZGZmZmZmZmYgd2luZG93
XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg3M10gcGNpX2J1cyAw
MDBjOjAxOiByZXNvdXJjZSAxIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzg3NF0gcGNpX2J1cyAwMDBjOjAxOiByZXNv
dXJjZSAyIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc5ODBdIEFDUEk6IFBDSSBSb290
IEJyaWRnZSBbUENJMV0gKGRvbWFpbiAwMDBkIFtidXMgMDAtZmZdKQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzk4NV0gYWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBPUyBz
dXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlw
ZTNdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MDQ4XSBhY3BpIFBO
UDBBMDg6MDE6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNI
UENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4MTA3XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVD
YXBhYmlsaXR5XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODEwOV0g
YWNwaSBQTlAwQTA4OjAxOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAwMDAwMDAt
MHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmJjNGM5NGY0ZWE1OApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODczMl0gYWNwaSBQTlAwQTA4
OjAxOiBFQ0FNIGFyZWEgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZmZmZmZl0gcmVzZXJ2
ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
ODczOV0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2Zm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5ODc5OF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBkOjAwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4Nzk5XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3Qg
YnVzIHJlc291cmNlIFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg4MDBdIHBjaV9idXMgMDAwZDowMDogcm9v
dCBidXMgcmVzb3VyY2UgW21lbSAweDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4ODAxXSBwY2lfYnVzIDAw
MGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk4ODE4XSBwY2kgMDAwZDowMDowMC4wOiBbMWRlZjplMTAwXSB0
eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk4ODU2XSBwY2kgMDAwZDowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4ODY0XSBwY2kg
MDAwZDowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk4ODc5XSBwY2kgMDAwZDowMDowMS4wOiBzdXBwb3J0cyBEMSBE
MgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODg4MF0gcGNpIDAwMGQ6
MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODkzMV0gcGNpIDAwMGQ6MDA6MDIuMDogWzFkZWY6ZTEw
Ml0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5ODk1MV0gcGNpIDAwMGQ6MDA6MDIuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg5NTJdIHBjaSAwMDBkOjAwOjAyLjA6IFBN
RSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC42OTg5OTldIHBjaSAwMDBkOjAwOjAzLjA6IFsxZGVmOmUxMDNdIHR5cGUgMDEg
Y2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkw
MTldIHBjaSAwMDBkOjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk5MDIwXSBwY2kgMDAwZDowMDowMy4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk5MDY2XSBwY2kgMDAwZDowMDowNC4wOiBbMWRlZjplMTA0XSB0eXBlIDAxIGNsYXNzIDB4MDYw
NDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MDg2XSBwY2kgMDAw
ZDowMDowNC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5OTA4N10gcGNpIDAwMGQ6MDA6MDQuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBE
MSBEM2hvdApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTE1NF0gYWNw
aXBocDogU2xvdCBbMS0yXSByZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk5MTcwXSBwY2kgMDAwZDowMTowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNs
YXNzIDB4MDEwODAyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MTgy
XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMzEwMDAwLTB4NTAzMTNmZmYg
NjRiaXRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MjA2XSBwY2kg
MDAwZDowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMzAwMDAwLTB4NTAzMGZmZmYgcHJlZl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkyMTBdIHBjaSAwMDBkOjAx
OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC42OTkzNTldIGFjcGlwaHA6IFNsb3QgWzJdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkzNzNdIHBjaSAwMDBkOjAyOjAwLjA6IFs4
MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTkzODRdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4
NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTk0MDVdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDMwOiBbbWVtIDB4NTAyMDAw
MDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY5OTQwOV0gcGNpIDAwMGQ6MDI6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTU1MF0gYWNwaXBocDogU2xvdCBbM10g
cmVnaXN0ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTU2NV0g
cGNpIDAwMGQ6MDM6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTU3Nl0gcGNpIDAwMGQ6MDM6MDAu
MDogcmVnIDB4MTA6IFttZW0gMHg1MDExMDAwMC0weDUwMTEzZmZmIDY0Yml0XQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTU5N10gcGNpIDAwMGQ6MDM6MDAuMDogcmVn
IDB4MzA6IFttZW0gMHg1MDEwMDAwMC0weDUwMTBmZmZmIHByZWZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NjAxXSBwY2kgMDAwZDowMzowMC4wOiBlbmFibGluZyBF
eHRlbmRlZCBUYWdzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NzQz
XSBhY3BpcGhwOiBTbG90IFs0XSByZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjk5NzU3XSBwY2kgMDAwZDowNDowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAw
IGNsYXNzIDB4MDEwODAyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
NzY4XSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMDEwMDAwLTB4NTAwMTNm
ZmYgNjRiaXRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5Nzg5XSBw
Y2kgMDAwZDowNDowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMDAwMDAwLTB4NTAwMGZmZmYgcHJl
Zl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3OTNdIHBjaSAwMDBk
OjA0OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTk5MTVdIHBjaV9idXMgMDAwZDowMDogb24gTlVNQSBub2RlIDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5MTddIHBjaSAwMDBkOjAwOjAx
LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6
ZSAxMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTE5XSBwY2kg
MDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0
Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTIwXSBwY2kgMDAwZDowMDow
MS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAx
XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTkyMV0gcGNpIDAwMGQ6MDA6MDIuMDogYnJpZGdlIHdpbmRvdyBb
aW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5MjJdIHBjaSAwMDBkOjAwOjAyLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAw
Ml0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC42OTk5MjRdIHBjaSAwMDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAwMCBh
ZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
OTI1XSBwY2kgMDAwZDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0g
dG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5OTkyNl0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAx
MDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
OTkyN10gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgw
MDFmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5MjhdIHBjaSAwMDBkOjAwOjA0
LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6
ZSAxMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTI5XSBwY2kg
MDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0
Yml0IHByZWZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTMwXSBwY2kgMDAwZDowMDow
NC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDA0
XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTkzNF0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25l
ZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC42OTk5MzZdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21l
bSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTkzN10gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDE0
OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTk5MzhdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxNTogYXNz
aWduZWQgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTkzOV0gcGNpIDAwMGQ6MDA6MDMu
MDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA0MDAwMDAtMHg1MDVmZmZmZl0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NDBdIHBjaSAwMDBkOjAwOjAzLjA6IEJB
UiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk0MV0gcGNpIDAw
MGQ6MDA6MDQuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NDJdIHBjaSAwMDBkOjAw
OjA0LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk0
M10gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NDRdIHBjaSAwMDBk
OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk0NV0gcGNpIDAwMGQ6MDA6MDIu
MDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NDZdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTk0N10gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTk5NDhdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
OTk0OV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NDldIHBjaSAw
MDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk1Ml0gcGNpIDAwMGQ6MDA6
MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NTNdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAx
MzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjY5OTk1NF0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDEzOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTk5NTVdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY5OTk1NV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NTZdIHBj
aSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk1N10gcGNpIDAwMGQ6
MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk5NThdIHBjaSAwMDBkOjAwOjAxLjA6IEJB
UiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk1OV0gcGNpIDAwMGQ6MDE6MDAuMDogQkFSIDY6IGFz
c2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTYwXSBwY2kgMDAwZDowMTowMC4wOiBCQVIgMDogYXNz
aWduZWQgW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTY2XSBwY2kgMDAwZDowMDowMS4wOiBQQ0kgYnJpZGdl
IHRvIFtidXMgMDFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTY3
XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMDAwMDAwLTB4NTAx
ZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTY4XSBwY2kg
MDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAw
MDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY5OTk3MF0gcGNpIDAwMGQ6MDI6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDIwMDAw
MC0weDUwMjBmZmZmIHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk5OTcxXSBwY2kgMDAwZDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMjEwMDAw
LTB4NTAyMTNmZmYgNjRiaXRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk5OTc2XSBwY2kgMDAwZDowMDowMi4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTc3XSBwY2kgMDAwZDowMDowMi4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTc4XSBwY2kgMDAwZDowMDowMi4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk4MF0gcGNpIDAwMGQ6MDM6
MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDQwMDAwMC0weDUwNDBmZmZmIHByZWZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTgxXSBwY2kgMDAwZDowMzow
MC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNDEwMDAwLTB4NTA0MTNmZmYgNjRiaXRdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTg2XSBwY2kgMDAwZDowMDow
My4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk5OTg3XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk5OTg4XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAw
MDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTk5MF0gcGNpIDAwMGQ6MDQ6MDAuMDogQkFSIDY6IGFzc2lnbmVk
IFttZW0gMHg1MDYwMDAwMC0weDUwNjBmZmZmIHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk5OTkwXSBwY2kgMDAwZDowNDowMC4wOiBCQVIgMDogYXNzaWduZWQg
W21lbSAweDUwNjEwMDAwLTB4NTA2MTNmZmYgNjRiaXRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk5OTk1XSBwY2kgMDAwZDowMDowNC4wOiBQQ0kgYnJpZGdlIHRvIFti
dXMgMDRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTk2XSBwY2kg
MDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNjAwMDAwLTB4NTA3ZmZmZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTk3XSBwY2kgMDAwZDow
MDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk5
OV0gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA0IFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZm
IHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMDBdIHBj
aV9idXMgMDAwZDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzdmZmRmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMDFd
IHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMDJdIHBjaV9idXMgMDAw
ZDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAwMWZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMDAyXSBwY2lf
YnVzIDAwMGQ6MDI6IHJlc291cmNlIDEgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMDAzXSBwY2lfYnVzIDAwMGQ6MDI6
IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDAwNF0gcGNpX2J1cyAw
MDBkOjAzOiByZXNvdXJjZSAxIFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDAwNV0gcGNpX2J1cyAwMDBkOjAzOiByZXNv
dXJjZSAyIFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMDZdIHBjaV9idXMgMDAwZDow
NDogcmVzb3VyY2UgMSBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMDddIHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2Ug
MiBbbWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTE0XSBBQ1BJOiBQQ0kgUm9vdCBCcmlk
Z2UgW1BDSTJdIChkb21haW4gMDAwMSBbYnVzIDAwLWZmXSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDAxMTldIGFjcGkgUE5QMEEwODowMjogX09TQzogT1Mgc3VwcG9y
dHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDE4M10gYWNwaSBQTlAwQTA4
OjAyOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90
cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDI0
N10gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJp
bGl0eV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAyNDldIGFjcGkg
UE5QMEEwODowMjogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2Jm
ZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZiYzRjOTRmNGVhNTgKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA4NzhdIGFjcGkgUE5QMEEwODowMjog
RUNBTSBhcmVhIFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5
IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA4ODRd
IGFjcGkgUE5QMEEwODowMjogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZm
ZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDA5NDRdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMTowMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDk0NV0gcGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyBy
ZXNvdXJjZSBbbWVtIDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwOTQ2XSBwY2lfYnVzIDAwMDE6MDA6IHJvb3QgYnVz
IHJlc291cmNlIFttZW0gMHgzODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZmZmYgd2luZG93XQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDk0N10gcGNpX2J1cyAwMDAxOjAw
OiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDk2NF0gcGNpIDAwMDE6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAw
MCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MTAwOF0gcGNpIDAwMDE6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQw
MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTAyMF0gcGNpIDAwMDE6
MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMTA0N10gcGNpIDAwMDE6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDEwNDhdIHBjaSAwMDAxOjAwOjAx
LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDExMzVdIGFjcGlwaHA6IFNsb3QgWzEtM10gcmVnaXN0ZXJlZApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzIyNl0gcGNpX2J1cyAwMDAx
OjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwMzIyN10gcGNpIDAwMDE6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBm
ZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDMyMjhdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAw
eDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAw
MDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDMyMzBdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAw
LTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjMxXSBwY2kgMDAwMTow
MDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzIzM10gcGNpIDAwMDE6MDA6MDEu
MDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjMzXSBw
Y2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzIzNV0gcGNpIDAwMDE6MDA6
MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjM2XSBwY2kgMDAwMTowMDowMS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMzIzN10gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAzMjM4XSBwY2kgMDAwMTowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjM5XSBwY2kgMDAw
MTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDYwMDAwMDAwLTB4NjAxZmZmZmZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMjQxXSBwY2kgMDAwMTowMDow
MS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM4MDAwMDAwMDAwMC0weDM4MDAwMDFmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzI0M10g
cGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA0IFttZW0gMHg2MDAwMDAwMC0weDZmZmZmZmZmIHdp
bmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMyNDRdIHBjaV9i
dXMgMDAwMTowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4M2JmZmRmZmZmZmZm
IHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMyNDVdIHBj
aV9idXMgMDAwMTowMTogcmVzb3VyY2UgMSBbbWVtIDB4NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMyNDVdIHBjaV9idXMgMDAwMTow
MTogcmVzb3VyY2UgMiBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzMzUxXSBBQ1BJOiBQ
Q0kgUm9vdCBCcmlkZ2UgW1BDSTNdIChkb21haW4gMDAwMCBbYnVzIDAwLWZmXSkKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDMzNTVdIGFjcGkgUE5QMEEwODowMzogX09T
QzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kg
SFBYLVR5cGUzXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzQxOF0g
YWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90
cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMzQ3Nl0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FF
UiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDM0NzhdIGFjcGkgUE5QMEEwODowMzogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2ZmZmYw
MDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZiYzRjOTRm
NGVhNTgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQxMDFdIGFjcGkg
UE5QMEEwODowMzogRUNBTSBhcmVhIFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmZd
IHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDQxMDddIGFjcGkgUE5QMEEwODowMzogRUNBTSBhdCBbbWVtIDB4M2ZmZmYwMDAwMDAw
LTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDQxNjddIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMDowMApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDE2OF0gcGNpX2J1cyAwMDAwOjAw
OiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZmZiB3aW5kb3ddClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0MTY5XSBwY2lfYnVzIDAwMDA6
MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzZmZmZGZmZmZmZmYg
d2luZG93XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDE3MF0gcGNp
X2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDE4N10gcGNpIDAwMDA6MDA6MDAuMDogWzFkZWY6
ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwNDIzMl0gcGNpIDAwMDA6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBj
bGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDI0
NV0gcGNpIDAwMDA6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDI3MV0gcGNpIDAwMDA6MDA6MDEuMDogc3VwcG9y
dHMgRDEgRDIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQyNzJdIHBj
aSAwMDAwOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQzNjBdIGFjcGlwaHA6IFNsb3QgWzEtNF0g
cmVnaXN0ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjQ1MF0g
cGNpX2J1cyAwMDAwOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwNjQ1Ml0gcGNpIDAwMDA6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8g
IDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY0NTNdIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRnZSB3
aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDY0NTRdIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2NDU2
XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg3MDAwMDAwMC0weDcw
MWZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjQ1N10gcGNp
IDAwMDA6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2Mw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzA2NDU4XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjQ1OV0g
cGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2NDYwXSBwY2kgMDAw
MDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjQ2MV0gcGNpIDAwMDA6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2NDYyXSBwY2kgMDAwMDowMDowMS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2
NDYzXSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDcwMDAwMDAwLTB4
NzAxZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2NDY1XSBw
Y2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDNjMDAwMDAwMDAwMC0weDNj
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNjQ2N10gcGNpX2J1cyAwMDAwOjAwOiByZXNvdXJjZSA0IFttZW0gMHg3MDAwMDAwMC0w
eDdmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDY0NjhdIHBjaV9idXMgMDAwMDowMDogcmVzb3VyY2UgNSBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4
M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDY0NjldIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NzAwMDAwMDAtMHg3
MDFmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY0NjldIHBj
aV9idXMgMDAwMDowMTogcmVzb3VyY2UgMiBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2
NTczXSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTRdIChkb21haW4gMDAwMiBbYnVzIDAwLWZm
XSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY1NzddIGFjcGkgUE5Q
MEEwODowNDogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBT
ZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNjY0MF0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBw
b3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNjY5OF0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBPUyBub3cg
Y29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDY3MDFdIGFjcGkgUE5QMEEwODowNDogTUNGRyBxdWlyazogRUNBTSBhdCBb
bWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAw
eGZmZmZiYzRjOTRmNGVhNTgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDczMjJdIGFjcGkgUE5QMEEwODowNDogRUNBTSBhcmVhIFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgy
M2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDczMjldIGFjcGkgUE5QMEEwODowNDogRUNBTSBhdCBbbWVtIDB4
MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDczODldIFBDSSBob3N0IGJyaWRnZSB0byBidXMg
MDAwMjowMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzM5MF0gcGNp
X2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MDA4MDAwMDAtMHgwZmZmZmZm
ZiB3aW5kb3ddClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MzkxXSBw
Y2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgy
M2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwNzM5Ml0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzQwOF0gcGNpIDAwMDI6MDA6
MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzQ1MV0gcGNpIDAwMDI6MDA6MDEuMDogWzFkZWY6ZTEx
MV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNzQ4Nl0gcGNpIDAwMDI6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc0ODddIHBjaSAwMDAyOjAwOjAxLjA6IFBN
RSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDc1NDhdIHBjaSAwMDAyOjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEg
Y2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc1
ODNdIHBjaSAwMDAyOjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA3NTg0XSBwY2kgMDAwMjowMDowNS4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA3NjY2XSBhY3BpcGhwOiBTbG90IFsxLTVdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDk3NzldIGFjcGlwaHA6IFNsb3QgWzMtMl0gcmVnaXN0ZXJl
ZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTg3MV0gcGNpX2J1cyAw
MDAyOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxMTg3Ml0gcGNpIDAwMDI6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0w
eDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTE4NzNdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTE4NzVdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAw
MDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAw
MDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODc2XSBwY2kgMDAw
MjowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMl0g
YWRkX3NpemUgMTAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTg3
N10gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBm
ZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEw
MDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTg3OF0gcGNpIDAw
MDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8g
W2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4ODBdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxNDog
YXNzaWduZWQgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzExODgxXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTU6IGFzc2ln
bmVkIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4ODJdIHBjaSAwMDAyOjAwOjA1LjA6
IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODgzXSBwY2kgMDAwMjowMDowNS4wOiBCQVIg
MTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAzZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4ODRdIHBjaSAwMDAy
OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODg1XSBwY2kgMDAwMjowMDowMS4wOiBC
QVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4ODZdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzExODg3XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTE4ODhdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODg5
XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4OTBdIHBjaSAw
MDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODkxXSBwY2kgMDAwMjowMDowMS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4OTFdIHBjaSAwMDAyOjAwOjAxLjA6IFBDSSBi
cmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTE4OTNdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDA4MDAwMDAt
MHgwMDlmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE4OTVd
IHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4
MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzExODk3XSBwY2kgMDAwMjowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExODk4XSBwY2kgMDAwMjowMDow
NS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExOTAwXSBwY2kgMDAwMjowMDowNS4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMTkwMl0gcGNpX2J1
cyAwMDAyOjAwOiByZXNvdXJjZSA0IFttZW0gMHgwMDgwMDAwMC0weDBmZmZmZmZmIHdpbmRvd10K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE5MDNdIHBjaV9idXMgMDAw
MjowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRmZmZmZmZmIHdpbmRv
d10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE5MDNdIHBjaV9idXMg
MDAwMjowMTogcmVzb3VyY2UgMSBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE5MDRdIHBjaV9idXMgMDAwMjowMTogcmVz
b3VyY2UgMiBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExOTA1XSBwY2lfYnVzIDAwMDI6
MDI6IHJlc291cmNlIDEgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzExOTA2XSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNl
IDIgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjAxMl0gQUNQSTogUENJIFJvb3QgQnJp
ZGdlIFtQQ0k1XSAoZG9tYWluIDAwMDMgW2J1cyAwMC1mZl0pClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzEyMDE2XSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IE9TIHN1cHBv
cnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIwNzldIGFjcGkgUE5QMEEw
ODowNTogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hv
dHBsdWcgUE1FIExUUl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIx
MzhdIGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFi
aWxpdHldClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMTQwXSBhY3Bp
IFBOUDBBMDg6MDU6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3
ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmYmM0Yzk0ZjRlYTU4ClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNzY3XSBhY3BpIFBOUDBBMDg6MDU6
IEVDQU0gYXJlYSBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmXSByZXNlcnZlZCBi
eSBQTlAwQzAyOjAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNzcz
XSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZm
ZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzEyODI2XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDM6MDAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI4MjhdIHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMg
cmVzb3VyY2UgW21lbSAweDEwMDAwMDAwLTB4MWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjgyOF0gcGNpX2J1cyAwMDAzOjAwOiByb290IGJ1
cyByZXNvdXJjZSBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjdmZmRmZmZmZmZmIHdpbmRvd10KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI4MzBdIHBjaV9idXMgMDAwMzow
MDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTI4NDZdIHBjaSAwMDAzOjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUg
MDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTI4ODddIHBjaSAwMDAzOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0
MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI5MjNdIHBjaSAwMDAz
OjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzEyOTIzXSBwY2kgMDAwMzowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQx
IEQzaG90ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyOTgwXSBwY2kg
MDAwMzowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzMDA3XSBwY2kgMDAwMzowMDowNS4wOiBz
dXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzAw
OF0gcGNpIDAwMDM6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzA3NV0gYWNwaXBocDogU2xvdCBb
MS02XSByZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
MTk3XSBwY2kgMDAwMzowMjowMC4wOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNsYXNzIDB4MDIwMDAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1MjEyXSBwY2kgMDAwMzow
MjowMC4wOiByZWcgMHgxMDogW21lbSAweDEwMDIwMDAwLTB4MTAwM2ZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1MjI4XSBwY2kgMDAwMzowMjowMC4wOiByZWcg
MHgxODogW2lvICAweDFmZmY4MDIwLTB4MWZmZjgwM2ZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE1MjM3XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxYzogW21lbSAw
eDEwMDQ0MDAwLTB4MTAwNDdmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE1MzQ3XSBwY2kgMDAwMzowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQzaG90
IEQzY29sZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTM4NF0gcGNp
IDAwMDM6MDI6MDAuMDogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTM4Nl0gcGNp
IDAwMDM6MDI6MDAuMDogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZm
ZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTQwNl0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTkw
OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTQwN10gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFS
MyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5z
IEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
NTUwM10gcGNpIDAwMDM6MDI6MDAuMDogOC4wMDAgR2IvcyBhdmFpbGFibGUgUENJZSBiYW5kd2lk
dGgsIGxpbWl0ZWQgYnkgNS4wIEdUL3MgUENJZSB4MiBsaW5rIGF0IDAwMDM6MDA6MDUuMCAoY2Fw
YWJsZSBvZiAxNi4wMDAgR2IvcyB3aXRoIDUuMCBHVC9zIFBDSWUgeDQgbGluaykKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU1NTBdIHBjaSAwMDAzOjAyOjAwLjE6IFs4
MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTU1NjZdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDEwOiBbbWVtIDB4
MTAwMDAwMDAtMHgxMDAxZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTU1ODJdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDE4OiBbaW8gIDB4MWZmZjgwMDAtMHgx
ZmZmODAxZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU1OTFdIHBj
aSAwMDAzOjAyOjAwLjE6IHJlZyAweDFjOiBbbWVtIDB4MTAwNDAwMDAtMHgxMDA0M2ZmZl0KU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU2OTddIHBjaSAwMDAzOjAyOjAw
LjE6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NzMwXSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxODQ6
IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NzMxXSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIw
IHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMg
QkFSMCBmb3IgOCBWRnMpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
NzUyXSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxOTA6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAz
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
NzUzXSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIzIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAt
MHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMyBmb3IgOCBWRnMpClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODcwXSBwY2lfYnVzIDAwMDM6MDA6IG9u
IE5VTUEgbm9kZSAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODcx
XSBwY2lfYnVzIDAwMDM6MDA6IG1heCBidXMgZGVwdGg6IDEgcGNpX3RyeV9udW06IDIKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4NzVdIHBjaSAwMDAzOjAwOjAxLjA6
IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODc2XSBwY2kgMDAwMzowMDowMS4wOiBCQVIg
MTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4NzddIHBjaSAwMDAz
OjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODc5XSBwY2kgMDAwMzowMDow
NS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4Nzld
IHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODgwXSBwY2kgMDAwMzow
MDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4ODFdIHBjaSAwMDAzOjAwOjA1LjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODgyXSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTU4ODNdIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1
cyAwMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4ODVdIHBjaSAw
MDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU4ODddIHBjaSAwMDAzOjAw
OjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODg5
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4MTAy
MWZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODkzXSBwY2kg
MDAwMzowMjowMC4xOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjIwMDAwLTB4MTAyM2ZmZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODk2XSBwY2kgMDAwMzow
MjowMC4wOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQwMDAwLTB4MTAyNDNmZmZdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODk5XSBwY2kgMDAwMzowMjowMC4w
OiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDIxZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTkwM10gcGNp
IDAwMDM6MDI6MDAuMDogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjIwMDAwLTB4MjQw
MDAwMjNmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE1OTA4XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQ0
MDAwLTB4MTAyNDdmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
OTExXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDI0MDAw
MC0weDI0MDAwMDI1ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNTkxNV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDEwOiBhc3NpZ25lZCBbbWVt
IDB4MjQwMDAwMjYwMDAwLTB4MjQwMDAwMjdmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTE5XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE1OTIwXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFz
c2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNTkyMV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgwMDIwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTkyMl0g
cGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAy
MF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5MjNdIHBjaSAwMDAz
OjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTU5MjRdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTU5MjZdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTI3XSBwY2lfYnVzIDAwMDM6MDA6IE5vLiAyIHRyeSB0
byBhc3NpZ24gdW5hc3NpZ25lZCByZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTU5MjldIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTMw
XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5MzFdIHBjaSAw
MDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTMyXSBwY2kgMDAwMzowMDowNS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5MzNdIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBi
cmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTU5MzVdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAt
MHgxMDFmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5MzZd
IHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4
MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE1OTM4XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDAwMjBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTM5
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgw
MDIwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTk0MF0gcGNpIDAw
MDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTk0MF0gcGNpIDAwMDM6MDI6MDAuMTog
QkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NDFdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlk
Z2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5
NDNdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgx
MDNmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NDRdIHBj
aSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQw
MDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE1OTQ2XSBwY2lfYnVzIDAwMDM6MDA6IEF1dG9tYXRpY2FsbHkgZW5hYmxlZCBwY2kgcmVh
bGxvYywgaWYgeW91IGhhdmUgcHJvYmxlbSwgdHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYz1v
ZmYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NDddIHBjaV9idXMg
MDAwMzowMDogcmVzb3VyY2UgNCBbbWVtIDB4MTAwMDAwMDAtMHgxZmZmZmZmZiB3aW5kb3ddClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTQ4XSBwY2lfYnVzIDAwMDM6
MDA6IHJlc291cmNlIDUgW21lbSAweDI0MDAwMDAwMDAwMC0weDI3ZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTQ5XSBwY2lfYnVzIDAw
MDM6MDE6IHJlc291cmNlIDEgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1OTQ5XSBwY2lfYnVzIDAwMDM6MDE6IHJlc291
cmNlIDIgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTk1MF0gcGNpX2J1cyAwMDAzOjAy
OiByZXNvdXJjZSAxIFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTk1MV0gcGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAy
IFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYwMTddIEFDUEk6IFBDSSBSb290IEJyaWRn
ZSBbUENJNl0gKGRvbWFpbiAwMDA0IFtidXMgMDAtZmZdKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNjAyMV0gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBPUyBzdXBwb3J0
cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MDg3XSBhY3BpIFBOUDBBMDg6
MDY6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3Rw
bHVnIFBNRSBMVFJdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MTQ1
XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmls
aXR5XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjE0N10gYWNwaSBQ
TlAwQTA4OjA2OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmJjNGM5NGY0ZWE1OApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjc3NV0gYWNwaSBQTlAwQTA4OjA2OiBF
Q0FNIGFyZWEgW21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkg
UE5QMEMwMjowMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjc4MV0g
YWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZm
ZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjgzMl0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA0OjAwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2ODM0XSBwY2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJl
c291cmNlIFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY4MzVdIHBjaV9idXMgMDAwNDowMDogcm9vdCBidXMg
cmVzb3VyY2UgW21lbSAweDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2ODM2XSBwY2lfYnVzIDAwMDQ6MDA6
IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2ODUyXSBwY2kgMDAwNDowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAw
IGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
ODkwXSBwY2kgMDAwNDowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAw
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTE3XSBwY2kgMDAwNDow
MDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjkxOF0gcGNpIDAwMDQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBE
M2hvdApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk2MF0gcGNpIDAw
MDQ6MDA6MDMuMDogWzFkZWY6ZTExM10gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk4Nl0gcGNpIDAwMDQ6MDA6MDMuMDogc3Vw
cG9ydHMgRDEgRDIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5ODdd
IHBjaSAwMDA0OjAwOjAzLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcwMzJdIHBjaSAwMDA0OjAwOjA1LjA6
IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTcwNjhdIHBjaSAwMDA0OjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQy
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MDY5XSBwY2kgMDAwNDow
MDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTQxXSBwY2kgMDAwNDowMTowMC4wOiBbMWEwMzoxMTUw
XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE3MTg4XSBwY2kgMDAwNDowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MjQ0XSBwY2kgMDAwNDowMTow
MC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzI0NV0gcGNpIDAwMDQ6MDE6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEMiBE
M2hvdCBEM2NvbGQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTczNDBd
IHBjaV9idXMgMDAwNDowMjogZXh0ZW5kZWQgY29uZmlnIHNwYWNlIG5vdCBhY2Nlc3NpYmxlClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MzU3XSBwY2kgMDAwNDowMjow
MC4wOiBbMWEwMzoyMDAwXSB0eXBlIDAwIGNsYXNzIDB4MDMwMDAwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3Mzc1XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxMDog
W21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE3Mzg1XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxNDogW21lbSAweDIyMDAw
MDAwLTB4MjIwMWZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3
Mzk2XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxODogW2lvICAweDJmZmY4MDAwLTB4MmZmZjgw
N2ZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NDQwXSBwY2kgMDAw
NDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgdG8gZWZpZmIKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTc0NzZdIHBjaSAwMDA0OjAyOjAwLjA6IHN1cHBvcnRzIEQxIEQy
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NDc3XSBwY2kgMDAwNDow
MjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90IEQzY29sZApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzU4N10gcGNpIDAwMDQ6MDM6MDAuMDog
WzE5MTI6MDAxNF0gdHlwZSAwMCBjbGFzcyAweDBjMDMzMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNzYwOV0gcGNpIDAwMDQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0g
MHgyMjIwMDAwMC0weDIyMjAxZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNzcxOF0gcGNpIDAwMDQ6MDM6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEM2hvdCBEM2NvbGQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg4
NTFdIHBjaV9idXMgMDAwNDowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTg4NTRdIHBjaSAwMDA0OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMS0wMl0gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTg4NTVdIHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAw
eDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODU3XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDNdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE4ODU4XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0g
MHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2Fs
aWduIDEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg1OV0g
cGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFti
dXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTg4NjBdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAw
LTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg4NjFd
IHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZm
ZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODY0XSBwY2kgMDAwNDowMDowMS4wOiBC
QVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg2Nl0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDE1
OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODY3XSBwY2kgMDAwNDow
MDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIzMWZmZmZmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg2OF0gcGNpIDAwMDQ6MDA6MDMu
MDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4MjgwMDAwM2ZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODY5XSBw
Y2kgMDAwNDowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzIwMDAwMC0weDIzM2Zm
ZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg3MF0gcGNpIDAw
MDQ6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAw
NWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE4ODcxXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg3MV0gcGNp
IDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODcyXSBwY2kgMDAwNDow
MDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg3M10gcGNpIDAwMDQ6MDA6MDMuMDogQkFS
IDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODc0XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5v
IHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxODg3NV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNz
aWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE4ODc3XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg3N10g
cGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODc4XSBwY2kgMDAw
NDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg3OV0gcGNpIDAwMDQ6MDA6MDUuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4ODgwXSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6
IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxODg4MV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8g
YXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE4ODgyXSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgy
MDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxODg4M10gcGNpIDAwMDQ6MDE6MDAuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg4ODRdIHBj
aSAwMDA0OjAxOjAwLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg4NV0gcGNpIDAwMDQ6
MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIxZmZmZmZmXQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg4OV0gcGNpIDAwMDQ6MDI6MDAu
MDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODg5M10gcGNpIDAwMDQ6MDI6MDAuMDogQkFS
IDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDgwXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxODg5M10gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDI6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MDA4MF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTg4OTVdIHBjaSAwMDA0OjAxOjAwLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg4OTldIHBjaSAwMDA0OjAx
OjAwLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5MDZdIHBjaSAwMDA0OjAwOjAxLjA6
IFBDSSBicmlkZ2UgdG8gW2J1cyAwMS0wMl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTg5MDddIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTg5MDhdIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjgwMDAw
MDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE4OTExXSBwY2kgMDAwNDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQg
W21lbSAweDIzMDAwMDAwLTB4MjMwMDFmZmYgNjRiaXRdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE4OTIwXSBwY2kgMDAwNDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFti
dXMgMDNdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4OTIyXSBwY2kg
MDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4OTIzXSBwY2kgMDAwNDow
MDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODky
NF0gcGNpIDAwMDQ6MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODkyNl0gcGNpIDAwMDQ6MDA6MDUuMDogICBicmlk
Z2Ugd2luZG93IFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxODkyOF0gcGNpIDAwMDQ6MDA6MDUuMDogICBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5MzBdIHBjaV9idXMgMDAwNDowMDog
U29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25lZCwgdHJ5IGJvb3Rpbmcgd2l0
aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODkz
MV0gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA0IFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZm
IHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5MzFdIHBj
aV9idXMgMDAwNDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MmJmZmRmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5MzJd
IHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMSBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5MzNdIHBjaV9idXMgMDAw
NDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4OTM0XSBwY2lf
YnVzIDAwMDQ6MDI6IHJlc291cmNlIDEgW21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4OTM1XSBwY2lfYnVzIDAwMDQ6MDM6
IHJlc291cmNlIDEgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4OTM2XSBwY2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDIg
W21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODkzNl0gcGNpX2J1cyAwMDA0OjA0OiByZXNv
dXJjZSAxIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxODkzN10gcGNpX2J1cyAwMDA0OjA0OiByZXNvdXJjZSAyIFttZW0g
MHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTg5NjRdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJ
N10gKGRvbWFpbiAwMDA1IFtidXMgMDAtZmZdKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxODk2OV0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0
ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MDMzXSBhY3BpIFBOUDBBMDg6MDc6IF9P
U0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBN
RSBMVFJdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MDkxXSBhY3Bp
IFBOUDBBMDg6MDc6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTA5M10gYWNwaSBQTlAwQTA4
OjA3OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZm
ZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmJjNGM5NGY0ZWE1OApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTcxOF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGFy
ZWEgW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMw
MjowMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTcyNF0gYWNwaSBQ
TlAwQTA4OjA3OiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmZdIGZv
ciBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTc4
MF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA1OjAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE5NzgxXSBwY2lfYnVzIDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNl
IFttZW0gMHgzMDAwMDAwMC0weDNmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTk3ODJdIHBjaV9idXMgMDAwNTowMDogcm9vdCBidXMgcmVzb3Vy
Y2UgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NzgzXSBwY2lfYnVzIDAwMDU6MDA6IHJvb3Qg
YnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE5ODAwXSBwY2kgMDAwNTowMDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNz
IDB4MDYwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5ODQxXSBw
Y2kgMDAwNTowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5ODc3XSBwY2kgMDAwNTowMDowMS4w
OiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
OTg3N10gcGNpIDAwMDU6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTkzN10gcGNpIDAwMDU6MDA6
MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTk3NF0gcGNpIDAwMDU6MDA6MDUuMDogc3VwcG9ydHMg
RDEgRDIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk5NzRdIHBjaSAw
MDA1OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAwMjZdIHBjaSAwMDA1OjAwOjA3LjA6IFsxZGVm
OmUxMTddIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjAwNjJdIHBjaSAwMDA1OjAwOjA3LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMDYzXSBwY2kgMDAwNTowMDowNy4w
OiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzIwMTM5XSBhY3BpcGhwOiBTbG90IFsxLTddIHJlZ2lzdGVyZWQKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNDZdIHBjaV9idXMgMDAwNTow
MDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQzNDddIHBjaSAwMDA1OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZm
XSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI0MzQ4XSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgw
MDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAw
MCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI0MzUwXSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDM1MV0gcGNpIDAwMDU6MDA6
MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9z
aXplIDEwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNTJdIHBj
aSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYg
NjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNTNdIHBjaSAwMDA1OjAw
OjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMg
MDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI0MzU0XSBwY2kgMDAwNTowMDowNy4wOiBicmlkZ2Ugd2luZG93
IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDM1NV0gcGNpIDAwMDU6MDA6MDcuMDogYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVz
IDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNDM1Nl0gcGNpIDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAw
IGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQzNTldIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwMDAwMDAw
LTB4MzAxZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MzYx
XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDAwMDAwMDAt
MHgyYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjQzNjJdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAw
eDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI0MzYzXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAw
MDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjQzNjRdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxNDogYXNzaWdu
ZWQgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI0MzY1XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTU6IGFzc2lnbmVkIFtt
ZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNjZdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI0MzY3XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjQzNjddIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0
MzY4XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNjldIHBj
aSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MzcwXSBwY2kgMDAwNTowMDow
Ny4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNzJdIHBjaSAwMDA1OjAwOjA3LjA6IEJB
UiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI0MzczXSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjQzNzRdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9y
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI0Mzc1XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNzVd
IHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Mzc2XSBwY2kgMDAwNTow
MDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNzddIHBjaSAwMDA1OjAwOjAxLjA6
IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQzNzldIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzAw
MDAwMDAtMHgzMDFmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjQzODBdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MmMwMDAwMDAw
MDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI0MzgyXSBwY2kgMDAwNTowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDJdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Mzg0XSBwY2kgMDAw
NTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Mzg1XSBwY2kgMDAwNTowMDow
NS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDIwMDAwMC0weDJjMDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDM4N10g
cGNpIDAwMDU6MDA6MDcuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDM4OV0gcGNpIDAwMDU6MDA6MDcuMDogICBicmlkZ2Ug
d2luZG93IFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyNDM5MF0gcGNpIDAwMDU6MDA6MDcuMDogICBicmlkZ2Ugd2luZG93
IFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzOTJdIHBjaV9idXMgMDAwNTowMDogcmVz
b3VyY2UgNCBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0MzkzXSBwY2lfYnVzIDAwMDU6MDA6IHJlc291cmNl
IDUgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Mzk0XSBwY2lfYnVzIDAwMDU6MDE6IHJlc291
cmNlIDEgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI0Mzk1XSBwY2lfYnVzIDAwMDU6MDE6IHJlc291cmNlIDIgW21lbSAw
eDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDM5Nl0gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAx
IFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNDM5N10gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAyIFttZW0gMHgyYzAw
MDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjQzOThdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMSBbbWVt
IDB4MzA0MDAwMDAtMHgzMDVmZmZmZl0KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQzOThdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwNDAw
MDAwLTB4MmMwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI2ODY4XSBpb21tdTogRGVmYXVsdCBkb21haW4gdHlwZTogVHJhbnNsYXRl
ZCAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY5MDBdIHBjaSAwMDA0
OjAyOjAwLjA6IHZnYWFyYjogVkdBIGRldmljZSBhZGRlZDogZGVjb2Rlcz1pbyttZW0sb3ducz1u
b25lLGxvY2tzPW5vbmUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY5
MDJdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogYnJpZGdlIGNvbnRyb2wgcG9zc2libGUKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY5MDRdIHBjaSAwMDA0OjAyOjAw
LjA6IHZnYWFyYjogc2V0dGluZyBhcyBib290IGRldmljZSAoVkdBIGxlZ2FjeSByZXNvdXJjZXMg
bm90IGF2YWlsYWJsZSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY5
MDRdIHZnYWFyYjogbG9hZGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI2OTkyXSBFREFDIE1DOiBWZXI6IDMuMC4wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI3MTIyXSBSZWdpc3RlcmVkIGVmaXZhcnMgb3BlcmF0aW9ucwpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzc5NV0gTmV0TGFiZWw6IEluaXRpYWxpemlu
ZwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzc5Nl0gTmV0TGFiZWw6
ICBkb21haW4gaGFzaCBzaXplID0gMTI4ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI3Nzk3XSBOZXRMYWJlbDogIHByb3RvY29scyA9IFVOTEFCRUxFRCBDSVBTT3Y0IENB
TElQU08KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc4MDZdIE5ldExh
YmVsOiAgdW5sYWJlbGVkIHRyYWZmaWMgYWxsb3dlZCBieSBkZWZhdWx0ClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3OTM2XSBjbG9ja3NvdXJjZTogU3dpdGNoZWQgdG8g
Y2xvY2tzb3VyY2UgYXJjaF9zeXNfY291bnRlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNjQ0NF0gVkZTOiBEaXNrIHF1b3RhcyBkcXVvdF82LjYuMApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjQ2MV0gVkZTOiBEcXVvdC1jYWNoZSBoYXNo
IHRhYmxlIGVudHJpZXM6IDUxMiAob3JkZXIgMCwgNDA5NiBieXRlcykKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY1NzVdIEFwcEFybW9yOiBBcHBBcm1vciBGaWxlc3lz
dGVtIEVuYWJsZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY2MTBd
IHBucDogUG5QIEFDUEkgaW5pdApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNzIzNV0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZm
IHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM3MjM2XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzZmZmZjAwMDAwMDAtMHgz
ZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcyMzddIHN5c3RlbSAwMDowMDogW21lbSAweDIzZmZm
MDAwMDAwMC0weDIzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzIzOF0gc3lzdGVtIDAwOjAwOiBb
bWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJl
c2VydmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MjM5XSBzeXN0
ZW0gMDA6MDA6IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmYgd2luZG93XSBjb3Vs
ZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzcyNDFdIHN5c3RlbSAwMDowMDogW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZiB3
aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNzI0Ml0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNm
ZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MjQzXSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzN2ZmZjAw
MDAwMDAtMHgzN2ZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcyNDddIHN5c3RlbSAwMDowMDogUGx1
ZyBhbmQgUGxheSBBQ1BJIGRldmljZSwgSURzIFBOUDBjMDIgKGFjdGl2ZSkKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc1MzldIHBucDogUG5QIEFDUEk6IGZvdW5kIDEg
ZGV2aWNlcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczODQ1NV0gTkVU
OiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAyClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM4NTIwXSBJUCBpZGVudHMgaGFzaCB0YWJsZSBlbnRyaWVzOiAyNjIxNDQg
KG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQwMTE4XSB0Y3BfbGlzdGVuX3BvcnRhZGRyX2hhc2ggaGFzaCB0YWJs
ZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDcsIDUyNDI4OCBieXRlcywgbGluZWFyKQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDQwN10gVENQIGVzdGFibGlzaGVkIGhh
c2ggdGFibGUgZW50cmllczogNTI0Mjg4IChvcmRlcjogMTAsIDQxOTQzMDQgYnl0ZXMsIGxpbmVh
cikKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0ODhdIFRDUCBiaW5k
IGhhc2ggdGFibGUgZW50cmllczogNjU1MzYgKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5l
YXIpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzQ2XSBUQ1A6IEhh
c2ggdGFibGVzIGNvbmZpZ3VyZWQgKGVzdGFibGlzaGVkIDUyNDI4OCBiaW5kIDY1NTM2KQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTgzMF0gVURQIGhhc2ggdGFibGUg
ZW50cmllczogMzI3NjggKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyMjM1XSBVRFAtTGl0ZSBoYXNoIHRhYmxl
IGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzAwN10gTkVUOiBSZWdpc3RlcmVkIHBy
b3RvY29sIGZhbWlseSAxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQz
MDEzXSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDQ0ClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzMDIxXSBwY2kgMDAwYzowMTowMC4wOiBDTFMgbWlzbWF0
Y2ggKDY0ICE9IDMyKSwgdXNpbmcgNjQgYnl0ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDMwOTldIFRyeWluZyB0byB1bnBhY2sgcm9vdGZzIGltYWdlIGFzIGluaXRy
YW1mcy4uLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0NDIwMl0gRnJl
ZWluZyBpbml0cmQgbWVtb3J5OiAzNzE2NEsKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNDU4NzddIGh3IHBlcmZldmVudHM6IGVuYWJsZWQgd2l0aCBhcm12OF9wbXV2M18w
IFBNVSBkcml2ZXIsIDcgY291bnRlcnMgYXZhaWxhYmxlClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTQ1OTYyXSBrdm0gWzFdOiBJUEEgU2l6ZSBMaW1pdDogNDggYml0cwpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0NTk4NV0ga3ZtIFsxXTogR0lD
djM6IG5vIEdJQ1YgcmVzb3VyY2UgZW50cnkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNDU5ODZdIGt2bSBbMV06IGRpc2FibGluZyBHSUN2MiBlbXVsYXRpb24KU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDU5OTddIGt2bSBbMV06IEdJQyBzeXN0
ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZSBlbmFibGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTQ2MzM2XSBrdm0gWzFdOiB2Z2ljIGludGVycnVwdCBJUlE5ClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ2NzQxXSBrdm0gWzFdOiBWSEUgbW9k
ZSBpbml0aWFsaXplZCBzdWNjZXNzZnVsbHkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNDc1NzNdIEluaXRpYWxpc2Ugc3lzdGVtIHRydXN0ZWQga2V5cmluZ3MKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDc1ODFdIEtleSB0eXBlIGJsYWNrbGlz
dCByZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ3NjE3
XSB3b3JraW5nc2V0OiB0aW1lc3RhbXBfYml0cz00MiBtYXhfb3JkZXI9MjQgYnVja2V0X29yZGVy
PTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDgyODBdIHpidWQ6IGxv
YWRlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0ODQwMF0gaW50ZWdy
aXR5OiBQbGF0Zm9ybSBLZXlyaW5nIGluaXRpYWxpemVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTQ4NDAyXSBLZXkgdHlwZSBhc3ltbWV0cmljIHJlZ2lzdGVyZWQKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDg0MDJdIEFzeW1tZXRyaWMga2V5
IHBhcnNlciAneDUwOScgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjE0ODQwOV0gQmxvY2sgbGF5ZXIgU0NTSSBnZW5lcmljIChic2cpIGRyaXZlciB2ZXJz
aW9uIDAuNCBsb2FkZWQgKG1ham9yIDI0OSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNDg0NDhdIGlvIHNjaGVkdWxlciBtcS1kZWFkbGluZSByZWdpc3RlcmVkClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ5NDU1XSBzaHBjaHA6IFN0YW5kYXJk
IEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjQKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDk1NjddIGVmaWZiOiBwcm9iaW5nIGZvciBlZmlm
YgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0OTU4N10gZWZpZmI6IGZy
YW1lYnVmZmVyIGF0IDB4MjAwMDAwMDAsIHVzaW5nIDE4NzZrLCB0b3RhbCAxODc1awpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0OTU4OF0gZWZpZmI6IG1vZGUgaXMgODAw
eDYwMHgzMiwgbGluZWxlbmd0aD0zMjAwLCBwYWdlcz0xClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMTQ5NTg4XSBlZmlmYjogc2Nyb2xsaW5nOiByZWRyYXcKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDk1ODldIGVmaWZiOiBUcnVlY29sb3I6IHNp
emU9ODo4Ojg6OCwgc2hpZnQ9MjQ6MTY6ODowClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTQ5NjYxXSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZl
ciBkZXZpY2UgMTAweDM3ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTU1
NTcwXSBmYjA6IEVGSSBWR0EgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE1NTk5Ml0gaW5wdXQ6IFBvd2VyIEJ1dHRvbiBhcyAvZGV2aWNl
cy9MTlhTWVNUTTowMC9MTlhTWUJVUzowMC9QTlAwQzBDOjAwL2lucHV0L2lucHV0MApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1NjAwNl0gQUNQSTogUG93ZXIgQnV0dG9u
IFtQV1JCXQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MDM1NF0gQUNQ
SSBHVERUOiBmb3VuZCAxIFNCU0EgZ2VuZXJpYyBXYXRjaGRvZyhzKS4KU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjA4NjldIFNlcmlhbDogODI1MC8xNjU1MCBkcml2ZXIs
IDQgcG9ydHMsIElSUSBzaGFyaW5nIGVuYWJsZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjExMjBdIFNlcmlhbDogQU1CQSBkcml2ZXIKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNjExNTJdIG1zbV9zZXJpYWw6IGRyaXZlciBpbml0aWFsaXpl
ZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MTIwMV0gYXJtLXNtbXUt
djMgYXJtLXNtbXUtdjMuMC5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNjEyMjVdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0
bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjEzOTBdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjE0MjddIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjAuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjE0OTFdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAu
YXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNjE4NTddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjEuYXV0
bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTYxODY4XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0
OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTYxOTgyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRl
ZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTYyMDIwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCAz
Mjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTYyMDg0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCA2NTUz
NiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTYyNDAwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IG9wdGlvbiBtYXNrIDB4MApT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjQxMl0gYXJtLXNtbXUtdjMg
YXJtLXNtbXUtdjMuMi5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAw
MDAxZmZmKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjQ1OV0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3Ig
Y21kcQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjU3MF0gYXJtLXNt
bXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0
cQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjYwNV0gYXJtLXNtbXUt
djMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2Mjk0MF0gYXJtLXNtbXUtdjMg
YXJtLXNtbXUtdjMuMy5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNjI5NTBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzog
aWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjI5OTVdIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjMxMDVdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMu
YXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNjMxNDBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0
bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNjM0NDJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjQuYXV0bzog
b3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYz
NDUyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1i
aXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTYzNTE5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCA2
NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTYzNTYzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCAzMjc2
OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTYzNjc1XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBl
bnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYz
OTkwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDAwMF0gYXJtLXNtbXUtdjMgYXJt
LXNtbXUtdjMuNS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAx
ZmZmKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDA2Nl0gYXJtLXNt
bXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21k
cQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDExMF0gYXJtLXNtbXUt
djMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDIyMF0gYXJtLXNtbXUtdjMg
YXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDUyOF0gYXJtLXNtbXUtdjMgYXJt
LXNtbXUtdjMuNi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjQ1MzhdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogaWFz
IDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNjQ1NzZdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYu
YXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4xNjQ2NDBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0
bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4xNjQ2ODRdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzog
YWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjUwNjddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjcuYXV0bzogb3B0
aW9uIG1hc2sgMHgwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1MDc2
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQg
KGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTY1MTExXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCA2NTUz
NiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTY1MTcyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBl
bnRyaWVzIGZvciBldnRxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1
MjEzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRy
aWVzIGZvciBwcmlxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY5MjA5
XSBtb3VzZWRldjogUFMvMiBtb3VzZSBkZXZpY2UgY29tbW9uIGZvciBhbGwgbWljZQpTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MDE0N10gcnRjLWVmaSBydGMtZWZpLjA6
IHJlZ2lzdGVyZWQgYXMgcnRjMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE3MDYwOV0gcnRjLWVmaSBydGMtZWZpLjA6IHNldHRpbmcgc3lzdGVtIGNsb2NrIHRvIDIwMjEt
MDktMTNUMDg6MzI6MTcgVVRDICgxNjMxNTIxOTM3KQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjE3MTEzOV0gbGVkdHJpZy1jcHU6IHJlZ2lzdGVyZWQgdG8gaW5kaWNhdGUg
YWN0aXZpdHkgb24gQ1BVcwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3
MTQwMV0gU01DQ0M6IFNPQ19JRDogSUQgPSBqZXAxMDY6MGExNjowMDAxIFJldmlzaW9uID0gMHgw
MDAwMDBhMQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3MTY1NV0gTkVU
OiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjE3OTY3Ml0gU2VnbWVudCBSb3V0aW5nIHdpdGggSVB2NgpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjE3OTY4NV0gbWlwNjogTW9iaWxlIElQdjYKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzk2ODddIE5FVDogUmVnaXN0ZXJl
ZCBwcm90b2NvbCBmYW1pbHkgMTcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNzk3MTJdIG1wbHNfZ3NvOiBNUExTIEdTTyBzdXBwb3J0ClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMTc5NzYxXSByZWdpc3RlcmVkIHRhc2tzdGF0cyB2ZXJzaW9uIDEK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzk3NjNdIExvYWRpbmcgY29t
cGlsZWQtaW4gWC41MDkgY2VydGlmaWNhdGVzClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjA2MTAyXSBMb2FkZWQgWC41MDkgY2VydCAnRGViaWFuIFNlY3VyZSBCb290IENB
OiA2Y2NlY2U3ZTRjNmMwZDFmNjE0OWYzZGQyN2RmY2M1Y2JiNDE5ZWExJwpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjExMV0gTG9hZGVkIFguNTA5IGNlcnQgJ0RlYmlh
biBTZWN1cmUgQm9vdCBTaWduZXIgMjAyMSAtIGxpbnV4OiA0YjZlZjVhYmNhNjY5ODI1MTc4ZTA1
MmM4NDY2N2NjYmMwNTMxZjhjJwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIwNjI2NV0genN3YXA6IGxvYWRlZCB1c2luZyBwb29sIGx6by96YnVkClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA2MzcwXSBLZXkgdHlwZSAuX2ZzY3J5cHQgcmVnaXN0
ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjM3MF0gS2V5IHR5
cGUgLmZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIwNjM3MV0gS2V5IHR5cGUgZnNjcnlwdC1wcm92aXNpb25pbmcgcmVnaXN0ZXJlZApTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjQwMV0gQXBwQXJtb3I6IEFwcEFy
bW9yIHNoYTEgcG9saWN5IGhhc2hpbmcgZW5hYmxlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIxNTQzN10gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRl
OiBVRUZJOmRiClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE2MTIxXSBp
bnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdTVVNFIExpbnV4IEVudGVycHJpc2UgU2VjdXJl
IEJvb3QgQ0E6IDNkNGQ0MGNmOTM4NTM5MDI0YjFjZmM1YTEyZGVkZmU4YjE3ZTc1NWYnClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE2MTIyXSBpbnRlZ3JpdHk6IExvYWRp
bmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMTYyODJdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0Nhbm9uaWNh
bCBMdGQuIE1hc3RlciBDZXJ0aWZpY2F0ZSBBdXRob3JpdHk6IGFkOTE5OTBiYzIyYWIxZjUxNzA0
OGMyM2I2NjU1YTI2OGUzNDVhNjMnClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjE2MjgyXSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY0MzldIGludGVncml0eTog
TG9hZGVkIFguNTA5IGNlcnQgJ0ZlZG9yYSBTZWN1cmUgQm9vdCBDQTogZmRlMzI1OTljMmQ2MWRi
MWJmNTgwNzMzNWQ3YjIwZTRjZDk2M2I0MicKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yMTY0MzldIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVG
STpkYgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNjQ1Ml0gaW50ZWdy
aXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0Eg
MjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY0NTNdIGludGVncml0eTogTG9hZGluZyBY
LjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIxNjQ2NV0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IENv
cnBvcmF0aW9uIFVFRkkgQ0EgMjAxMTogMTNhZGJmNDMwOWJkODI3MDljOGNkNTRmMzE2ZWQ1MjI5
ODhhMWJkNCcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY0NjZdIGlu
dGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNjQ3N10gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkg
Y2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2
YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMjA3ODFdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjEwNzddIHBj
aWVwb3J0IDAwMGM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNgpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMTE0OV0gcGNpZXBvcnQgMDAwZDowMDowMS4w
OiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIyMTQxN10gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJ
UlEgMTA3ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIxNDU5XSBwY2ll
cG9ydCAwMDBkOjAwOjAyLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIxNzAxXSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFF
UjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yMjE3NDVdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjE5ODBdIHBjaWVwb3J0
IDAwMGQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjAyMV0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgNApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyMjI1Nl0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIyMzE1XSBwY2llcG9ydCAw
MDAxOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCA1ClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjIyNTg2XSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFFUjogZW5h
YmxlZCB3aXRoIElSUSAxMDgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
MjI2NTRdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDYKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjI5MjNdIHBjaWVwb3J0IDAwMDA6
MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwOQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIyMjk5MF0gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBZGRpbmcgdG8g
aW9tbXUgZ3JvdXAgNwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMzI2
Nl0gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIzMzE1XSBwY2llcG9ydCAwMDAyOjAw
OjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA4ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjIzNTY2XSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3
aXRoIElSUSAxMTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM2Mjld
IHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDkKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM5MDNdIHBjaWVwb3J0IDAwMDM6MDA6MDEu
MDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIyMzk1N10gcGNpZXBvcnQgMDAwMzowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUg
Z3JvdXAgMTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjQyMDBdIHBj
aWVwb3J0IDAwMDM6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNDI2NF0gcGNpZXBvcnQgMDAwNDowMDowMS4w
OiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yMjQ1NDBdIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGgg
SVJRIDExMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNDU4NV0gcGNp
ZXBvcnQgMDAwNDowMDowMy4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTIKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjQ4MjhdIHBjaWVwb3J0IDAwMDQ6MDA6MDMuMDog
QUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIyNDg3NV0gcGNpZXBvcnQgMDAwNDowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3Jv
dXAgMTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjUxMjVdIHBjaWVw
b3J0IDAwMDQ6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNTE5MV0gcGNpZXBvcnQgMDAwNDowMTowMC4wOiBB
ZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yMjU0NTZdIHBjaWVwb3J0IDAwMDU6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDE1ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI1NzQzXSBwY2llcG9y
dCAwMDA1OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjU3OTNdIHBjaWVwb3J0IDAwMDU6MDA6MDUuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDE2ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjI2MDQ5XSBwY2llcG9ydCAwMDA1OjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAx
MTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjYxMDRdIHBjaWVwb3J0
IDAwMDU6MDA6MDcuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE3ClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2MzU0XSBwY2llcG9ydCAwMDA1OjAwOjA3LjA6IEFFUjog
ZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMjc3MTFdIEZyZWVpbmcgdW51c2VkIGtlcm5lbCBtZW1vcnk6IDU0NDBLClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzg1NTI4XSBDaGVja2VkIFcrWCBtYXBwaW5nczog
cGFzc2VkLCBubyBXK1ggcGFnZXMgZm91bmQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4zODU1MzddIFJ1biAvaW5pdCBhcyBpbml0IHByb2Nlc3MKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zODU1MzldICAgd2l0aCBhcmd1bWVudHM6ClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzg1NTM5XSAgICAgL2luaXQKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zODU1NDBdICAgd2l0aCBlbnZpcm9ubWVudDoK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zODU1NDFdICAgICBIT01FPS8K
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4zODU1NDFdICAgICBURVJNPWxp
bnV4ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMzg1NTQyXSAgICAgQk9P
VF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04LWFybTY0ClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNDU5NzM5XSBwcHNfY29yZTogTGludXhQUFMgQVBJIHZlci4gMSBy
ZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDU5NzQwXSBw
cHNfY29yZTogU29mdHdhcmUgdmVyLiA1LjMuNiAtIENvcHlyaWdodCAyMDA1LTIwMDcgUm9kb2xm
byBHaW9tZXR0aSA8Z2lvbWV0dGlAbGludXguaXQ+ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNDYwNzEyXSBQVFAgY2xvY2sgc3VwcG9ydCByZWdpc3RlcmVkClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDY1MzA2XSBTQ1NJIHN1YnN5c3RlbSBpbml0
aWFsaXplZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ2NzAxMV0gaWdi
OiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgRHJpdmVyClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDY3MDEyXSBpZ2I6IENvcHlyaWdodCAoYykgMjAwNy0y
MDE0IEludGVsIENvcnBvcmF0aW9uLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjQ2NzM3Nl0gaWdiIDAwMDM6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE4ClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDY4NTMxXSBBQ1BJOiBidXMgdHlw
ZSBVU0IgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ2
ODU1Ml0gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciB1c2JmcwpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ2ODU1Nl0gdXNiY29yZTogcmVnaXN0
ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciBodWIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS40Njg1ODVdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGRldmljZSBkcml2ZXIg
dXNiClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcxMTA1XSBudm1lIDAw
MGQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE5ClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNDcxMzgyXSBudm1lIG52bWUwOiBwY2kgZnVuY3Rpb24gMDAwZDow
MTowMC4wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcxNDM2XSBudm1l
IDAwMGQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNDcxODEzXSBudm1lIG52bWUxOiBwY2kgZnVuY3Rpb24gMDAw
ZDowMjowMC4wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcxODYwXSBu
dm1lIDAwMGQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIxClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcyMTAxXSBudm1lIG52bWUyOiBwY2kgZnVuY3Rpb24g
MDAwZDowMzowMC4wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcyMTQ1
XSBudm1lIDAwMGQ6MDQ6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIyClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDcyMzc0XSBudm1lIG52bWUzOiBwY2kgZnVuY3Rp
b24gMDAwZDowNDowMC4wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDc4
NTY4XSBtcHQzc2FzIHZlcnNpb24gMzUuMTAwLjAwLjAwIGxvYWRlZApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjQ3ODY4M10gbXB0M3NhcyAwMDBjOjAxOjAwLjA6IEFkZGlu
ZyB0byBpb21tdSBncm91cCAyMwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjQ3OTA4NV0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMjQK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40NzkwOTBdIG1wdDNzYXNfY20w
OiA2MyBCSVQgUENJIEJVUyBETUEgQUREUkVTU0lORyBTVVBQT1JURUQsIHRvdGFsIG1lbSAoNjUz
MDQ4MDQga0IpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDc5NjY5XSB4
aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDc5NjczXSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5l
dyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMQpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ3OTY5Ml0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBa
ZXJvaW5nIDY0Yml0IGJhc2UgcmVnaXN0ZXJzLCBleHBlY3RpbmcgZmF1bHQKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODE1ODVdIHJhbmRvbTogZmFzdCBpbml0IGRvbmUK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODI3NzNdIG52bWUgbnZtZTA6
IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS40ODM1MzhdIG52bWUgbnZtZTE6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9s
bCBxdWV1ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODM4MzhdIG52
bWUgbnZtZTI6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODM5MTRdIG52bWUgbnZtZTM6IDgwLzAvMCBkZWZhdWx0
L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40
ODQ4MzddIGFzdCAwMDA0OjAyOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNApTZXAgMTMg
MTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4NjMzMV0gIG52bWUwbjE6ClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg3MjU0XSAgbnZtZTFuMTogcDEgcDIgcDMg
cDQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODc0ODVdICBudm1lM24x
OiBwMSBwMiBwMyBwNCBwNSBwNgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjYzNjgzMV0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBoY2MgcGFyYW1zIDB4MDE0MDUxY2YgaGNp
IHZlcnNpb24gMHgxMDAgcXVpcmtzIDB4MDAwMDAwMTEwMDAwMDQxMApTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjYzNzE2OV0gdXNiIHVzYjE6IE5ldyBVU0IgZGV2aWNlIGZv
dW5kLCBpZFZlbmRvcj0xZDZiLCBpZFByb2R1Y3Q9MDAwMiwgYmNkRGV2aWNlPSA1LjEwClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3MTcwXSB1c2IgdXNiMTogTmV3IFVT
QiBkZXZpY2Ugc3RyaW5nczogTWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42MzcxNzFdIHVzYiB1c2IxOiBQcm9kdWN0
OiB4SENJIEhvc3QgQ29udHJvbGxlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjYzNzE3Ml0gdXNiIHVzYjE6IE1hbnVmYWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQg
eGhjaS1oY2QKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42MzcxNzNdIHVz
YiB1c2IxOiBTZXJpYWxOdW1iZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjYzNzI4NV0gaHViIDEtMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42MzcyOTddIGh1YiAxLTA6MS4wOiA0IHBv
cnRzIGRldGVjdGVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3NDQ4
XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3NDUwXSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6
IG5ldyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMgpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjYzNzQ1Ml0geGhjaV9oY2QgMDAwNDowMzowMC4w
OiBIb3N0IHN1cHBvcnRzIFVTQiAzLjAgU3VwZXJTcGVlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjYzNzQ4NF0gdXNiIHVzYjI6IFdlIGRvbid0IGtub3cgdGhlIGFsZ29y
aXRobXMgZm9yIExQTSBmb3IgdGhpcyBob3N0LCBkaXNhYmxpbmcgTFBNLgpTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjYzNzQ5Nl0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNl
IGZvdW5kLCBpZFZlbmRvcj0xZDZiLCBpZFByb2R1Y3Q9MDAwMywgYmNkRGV2aWNlPSA1LjEwClNl
cCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3NDk3XSB1c2IgdXNiMjogTmV3
IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTMsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Mzc0OTddIHVzYiB1c2IyOiBQcm9k
dWN0OiB4SENJIEhvc3QgQ29udHJvbGxlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjYzNzQ5OF0gdXNiIHVzYjI6IE1hbnVmYWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJt
NjQgeGhjaS1oY2QKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Mzc0OTld
IHVzYiB1c2IyOiBTZXJpYWxOdW1iZXI6IDAwMDQ6MDM6MDAuMApTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjYzNzUyM10gY2hlY2tpbmcgZ2VuZXJpYyAoMjAwMDAwMDAgMWQ1
MDAwKSB2cyBodyAoMjAwMDAwMDAgMjAwMDAwMCkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS42Mzc1MjNdIGZiMDogc3dpdGNoaW5nIHRvIGFzdGRybWZiIGZyb20gRUZJIFZH
QQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjYzNzU4OF0gQ29uc29sZTog
c3dpdGNoaW5nIHRvIGNvbG91ciBkdW1teSBkZXZpY2UgODB4MjUKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS42Mzc2MzldIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIHBsYXRm
b3JtIGhhcyBubyBJTyBzcGFjZSwgdHJ5aW5nIE1NSU8KU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS42Mzc2NDVdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIFAyQSBicmlkZ2Ug
ZGlzYWJsZWQsIHVzaW5nIGRlZmF1bHQgY29uZmlndXJhdGlvbgpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjYzNzY0Nl0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gQVNUIDI1
MDAgZGV0ZWN0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Mzc2NTBd
IGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIEFuYWxvZyBWR0Egb25seQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjYzNzY1Ml0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gZHJh
bSBNQ0xLPTgwMCBNaHogdHlwZT0xIGJ1c193aWR0aD0xNgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjYzNzY4MV0gaHViIDItMDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Mzc3MDFdIFtUVE1dIFpvbmUgIGtlcm5l
bDogQXZhaWxhYmxlIGdyYXBoaWNzIG1lbW9yeTogMzI2NTI0MDIgS2lCClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3NzAyXSBbVFRNXSBab25lICAgZG1hMzI6IEF2YWls
YWJsZSBncmFwaGljcyBtZW1vcnk6IDIwOTcxNTIgS2lCClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjM3NzAyXSBbVFRNXSBJbml0aWFsaXppbmcgcG9vbCBhbGxvY2F0b3IK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Mzc3MDZdIFtUVE1dIEluaXRp
YWxpemluZyBETUEgcG9vbCBhbGxvY2F0b3IKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS42Mzc4NzJdIGh1YiAyLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjM3OTQzXSBbZHJtXSBJbml0aWFsaXplZCBhc3Qg
MC4xLjAgMjAxMjAyMjggZm9yIDAwMDQ6MDI6MDAuMCBvbiBtaW5vciAwClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzUxOTM0XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29s
b3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMjQweDY3ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNzU0NTk2XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBmYjA6IGFzdGRybWZi
IGZyYW1lIGJ1ZmZlciBkZXZpY2UKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS43ODMwMjldIGlnYiAwMDAzOjAyOjAwLjA6IGFkZGVkIFBIQyBvbiBldGgwClNlcCAxMyAxNDow
MjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzgzMDMwXSBpZ2IgMDAwMzowMjowMC4wOiBJbnRl
bChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjc4MzAzMl0gaWdiIDAwMDM6MDI6MDAuMDogZXRoMDogKFBD
SWU6NS4wR2IvczpXaWR0aCB4MikgZDg6NWU6ZDM6MDI6YjE6OTMKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS43ODMxMTNdIGlnYiAwMDAzOjAyOjAwLjA6IGV0aDA6IFBCQSBO
bzogMTA2MzAwLTAwMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjc4MzEx
NF0gaWdiIDAwMDM6MDI6MDAuMDogVXNpbmcgTVNJLVggaW50ZXJydXB0cy4gOCByeCBxdWV1ZShz
KSwgOCB0eCBxdWV1ZShzKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjc4
MzE5N10gaWdiIDAwMDM6MDI6MDAuMTogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDI1ClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQyNDQ5XSBtcHQzc2FzX2NtMDogQ3VycmVu
dEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8gNGsK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDI0NjZdIG1wdDNzYXNfY20w
OiBNU0ktWCB2ZWN0b3JzIHN1cHBvcnRlZDogOTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDI0NjddIAkgbm8gb2YgY29yZXM6IDgwLCBtYXhfbXNpeF92ZWN0b3JzOiAt
MQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg0MjQ2OF0gbXB0M3Nhc19j
bTA6ICAwIDgwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQzODE4XSBt
cHQzc2FzX2NtMDogSGlnaCBJT1BzIHF1ZXVlcyA6IGRpc2FibGVkClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODQzODE5XSBtcHQzc2FzMC1tc2l4MDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0NjUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4
MjBdIG1wdDNzYXMwLW1zaXgxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2NgpTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg0MzgyMF0gbXB0M3NhczAtbXNpeDI6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDY3ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODQzODIxXSBtcHQzc2FzMC1tc2l4MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjgKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MjFdIG1wdDNzYXMwLW1zaXg0
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2OQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg0MzgyMl0gbXB0M3NhczAtbXNpeDU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDcwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQzODIyXSBtcHQzc2Fz
MC1tc2l4NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzEKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDM4MjNdIG1wdDNzYXMwLW1zaXg3OiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDQ3MgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg0MzgyNF0g
bXB0M3NhczAtbXNpeDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDczClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODQzODI1XSBtcHQzc2FzMC1tc2l4OTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0NzQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDM4MjVdIG1wdDNzYXMwLW1zaXgxMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzUKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MjZdIG1wdDNzYXMwLW1zaXgxMTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDM4MjZdIG1wdDNzYXMwLW1zaXgxMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
NzcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MjddIG1wdDNzYXMw
LW1zaXgxMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzgKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDM4MjddIG1wdDNzYXMwLW1zaXgxNDogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0NzkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4Mjhd
IG1wdDNzYXMwLW1zaXgxNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODAKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MjldIG1wdDNzYXMwLW1zaXgxNjogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA0ODEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDM4MjldIG1wdDNzYXMwLW1zaXgxNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODIKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzBdIG1wdDNzYXMwLW1zaXgx
ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDM4MzBdIG1wdDNzYXMwLW1zaXgxOTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0ODQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzFdIG1wdDNz
YXMwLW1zaXgyMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODUKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzFdIG1wdDNzYXMwLW1zaXgyMTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0ODYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4
MzJdIG1wdDNzYXMwLW1zaXgyMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODcKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzNdIG1wdDNzYXMwLW1zaXgyMzogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0ODgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDM4MzNdIG1wdDNzYXMwLW1zaXgyNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODkK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzRdIG1wdDNzYXMwLW1z
aXgyNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDM4MzRdIG1wdDNzYXMwLW1zaXgyNjogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0OTEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzVdIG1w
dDNzYXMwLW1zaXgyNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTIKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzZdIG1wdDNzYXMwLW1zaXgyODogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0OTMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDM4MzZdIG1wdDNzYXMwLW1zaXgyOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTQKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzddIG1wdDNzYXMwLW1zaXgzMDog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDM4MzddIG1wdDNzYXMwLW1zaXgzMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
OTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzhdIG1wdDNzYXMw
LW1zaXgzMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTcKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDM4MzhdIG1wdDNzYXMwLW1zaXgzMzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0OTgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4Mzld
IG1wdDNzYXMwLW1zaXgzNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTkKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4MzldIG1wdDNzYXMwLW1zaXgzNTogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDM4NDBdIG1wdDNzYXMwLW1zaXgzNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDEKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDFdIG1wdDNzYXMwLW1zaXgz
NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDM4NDFdIG1wdDNzYXMwLW1zaXgzODogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MDMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDJdIG1wdDNz
YXMwLW1zaXgzOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDQKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDNdIG1wdDNzYXMwLW1zaXg0MDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MDUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4
NDNdIG1wdDNzYXMwLW1zaXg0MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDYKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDRdIG1wdDNzYXMwLW1zaXg0MjogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MDcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDM4NDRdIG1wdDNzYXMwLW1zaXg0MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDgK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDVdIG1wdDNzYXMwLW1z
aXg0NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDM4NDVdIG1wdDNzYXMwLW1zaXg0NTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDZdIG1w
dDNzYXMwLW1zaXg0NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTEKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDddIG1wdDNzYXMwLW1zaXg0NzogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MTIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDM4NDddIG1wdDNzYXMwLW1zaXg0ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTMKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDhdIG1wdDNzYXMwLW1zaXg0OTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDM4NDhdIG1wdDNzYXMwLW1zaXg1MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MTUKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NDldIG1wdDNzYXMw
LW1zaXg1MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTYKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDM4NTBdIG1wdDNzYXMwLW1zaXg1MjogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MTcKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTBd
IG1wdDNzYXMwLW1zaXg1MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTgKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTFdIG1wdDNzYXMwLW1zaXg1NDogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MTkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDM4NTJdIG1wdDNzYXMwLW1zaXg1NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjAKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTJdIG1wdDNzYXMwLW1zaXg1
NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDM4NTNdIG1wdDNzYXMwLW1zaXg1NzogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MjIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTNdIG1wdDNz
YXMwLW1zaXg1ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjMKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTRdIG1wdDNzYXMwLW1zaXg1OTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MjQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4
NTVdIG1wdDNzYXMwLW1zaXg2MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjUKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTVdIG1wdDNzYXMwLW1zaXg2MTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MjYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44NDM4NTZdIG1wdDNzYXMwLW1zaXg2MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjcK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NTZdIG1wdDNzYXMwLW1z
aXg2MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44NDM4NTddIG1wdDNzYXMwLW1zaXg2NDogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MjkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NThdIG1w
dDNzYXMwLW1zaXg2NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzAKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NThdIG1wdDNzYXMwLW1zaXg2NjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MzEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
NDM4NTldIG1wdDNzYXMwLW1zaXg2NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzIKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjBdIG1wdDNzYXMwLW1zaXg2ODog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NDM4NjBdIG1wdDNzYXMwLW1zaXg2OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MzQKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjFdIG1wdDNzYXMw
LW1zaXg3MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzUKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NDM4NjFdIG1wdDNzYXMwLW1zaXg3MTogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MzYKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjJd
IG1wdDNzYXMwLW1zaXg3MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzcKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjNdIG1wdDNzYXMwLW1zaXg3MzogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MzgKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44NDM4NjNdIG1wdDNzYXMwLW1zaXg3NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzkKU2Vw
IDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjRdIG1wdDNzYXMwLW1zaXg3
NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44NDM4NjVdIG1wdDNzYXMwLW1zaXg3NjogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1NDEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjVdIG1wdDNz
YXMwLW1zaXg3NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDIKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjZdIG1wdDNzYXMwLW1zaXg3ODogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1NDMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4
NjZdIG1wdDNzYXMwLW1zaXg3OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDQKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjhdIG1wdDNzYXNfY20wOiBpb21lbSgw
eDAwMDAwMDAwNDAxNDAwMDApLCBtYXBwZWQoMHgoX19fX3B0cnZhbF9fX18pKSwgc2l6ZSg2NTUz
NikKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NDM4NjldIG1wdDNzYXNf
Y20wOiBpb3BvcnQoMHgwMDAwMDAwMDAwMDAwMDAwKSwgc2l6ZSgwKQpTZXAgMTMgMTQ6MDI6MjAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg1NjAwMF0gaWdiIDAwMDM6MDI6MDAuMTogYWRkZWQgUEhD
IG9uIGV0aDEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTYwMDFdIGln
YiAwMDAzOjAyOjAwLjE6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0d29yayBDb25uZWN0
aW9uClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU2MDAyXSBpZ2IgMDAw
MzowMjowMC4xOiBldGgxOiAoUENJZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1ZTpkMzowMjpiMTo5
NApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1NjA4NF0gaWdiIDAwMDM6
MDI6MDAuMTogZXRoMTogUEJBIE5vOiAxMDYzMDAtMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODU2MDg1XSBpZ2IgMDAwMzowMjowMC4xOiBVc2luZyBNU0ktWCBpbnRl
cnJ1cHRzLiA4IHJ4IHF1ZXVlKHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU3MDUzXSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IHJlbmFtZWQg
ZnJvbSBldGgxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODc2MjA2XSBp
Z2IgMDAwMzowMjowMC4wIGVubzE6IHJlbmFtZWQgZnJvbSBldGgwClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuOTAxOTM1XSBtcHQzc2FzX2NtMDogQ3VycmVudEhvc3RQYWdl
U2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUgdG8gNGsKU2VwIDEzIDE0
OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MDE5MzddIG1wdDNzYXNfY20wOiBzZW5kaW5n
IGRpYWcgcmVzZXQgISEKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45NTE5
NDBdIHVzYiAxLTE6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmljZSBudW1iZXIgMiB1c2luZyB4aGNp
X2hjZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjE0NDU0N10gdXNiIDEt
MTogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTQxM2MsIGlkUHJvZHVjdD0zMDFhLCBi
Y2REZXZpY2U9IDEuMDAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xNDQ1
NDhdIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNl
cmlhbE51bWJlcj0wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTQ0NTQ5
XSB1c2IgMS0xOiBQcm9kdWN0OiBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTQ0NTUwXSB1c2IgMS0xOiBNYW51ZmFjdHVy
ZXI6IFBpeEFydApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjE1MzYwNl0g
aGlkOiByYXcgSElEIGV2ZW50cyBkcml2ZXIgKEMpIEppcmkgS29zaW5hClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMTY1NjQ5XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBp
bnRlcmZhY2UgZHJpdmVyIHVzYmhpZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAyLjE2NTY1MV0gdXNiaGlkOiBVU0IgSElEIGNvcmUgZHJpdmVyClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMTY2OTI1XSBpbnB1dDogUGl4QXJ0IERlbGwgTVMxMTYgVVNC
IE9wdGljYWwgTW91c2UgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDow
MzowMC4wL3VzYjEvMS0xLzEtMToxLjAvMDAwMzo0MTNDOjMwMUEuMDAwMS9pbnB1dC9pbnB1dDEK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xNjY5NzNdIGhpZC1nZW5lcmlj
IDAwMDM6NDEzQzozMDFBLjAwMDE6IGlucHV0LGhpZHJhdzA6IFVTQiBISUQgdjEuMTEgTW91c2Ug
W1BpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlXSBvbiB1c2ItMDAwNDowMzowMC4w
LTEvaW5wdXQwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjUyNTkzXSB1
c2IgMi0zOiBuZXcgU3VwZXJTcGVlZCBHZW4gMSBVU0IgZGV2aWNlIG51bWJlciAyIHVzaW5nIHho
Y2lfaGNkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjgxMDk5XSB1c2Ig
Mi0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywgaWRQcm9kdWN0PTA2MjAs
IGJjZERldmljZT05My4xMApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjI4
MTEwMF0gdXNiIDItMzogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9Miwg
U2VyaWFsTnVtYmVyPTAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yODEx
MDBdIHVzYiAyLTM6IFByb2R1Y3Q6IFVTQjMuMSBIdWIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMi4yODExMDFdIHVzYiAyLTM6IE1hbnVmYWN0dXJlcjogR2VuZXN5c0xvZ2lj
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjg0MTE3XSBodWIgMi0zOjEu
MDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjI4
NDcyOV0gaHViIDItMzoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMi40MTE5MjFdIHVzYiAxLTI6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmlj
ZSBudW1iZXIgMyB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAyLjYyMzU0MF0gdXNiIDEtMjogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTA0
ZDksIGlkUHJvZHVjdD0xMjAzLCBiY2REZXZpY2U9IDIuODAKU2VwIDEzIDE0OjAyOjIwIGRlYmlh
biBrZXJuZWw6IFsgICAgMi42MjM1NDFdIHVzYiAxLTI6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6
IE1mcj0wLCBQcm9kdWN0PTAsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDIuNjYxNzAzXSBpbnB1dDogSElEIDA0ZDk6MTIwMyBhcyAvZGV2aWNlcy9w
Y2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTIvMS0yOjEuMC8wMDAz
OjA0RDk6MTIwMy4wMDAyL2lucHV0L2lucHV0MgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAyLjcyMzk3OF0gaGlkLWdlbmVyaWMgMDAwMzowNEQ5OjEyMDMuMDAwMjogaW5wdXQs
aGlkcmF3MTogVVNCIEhJRCB2MS4xMSBLZXlib2FyZCBbSElEIDA0ZDk6MTIwM10gb24gdXNiLTAw
MDQ6MDM6MDAuMC0yL2lucHV0MApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAy
Ljc2NjY2N10gaW5wdXQ6IEhJRCAwNGQ5OjEyMDMgU3lzdGVtIENvbnRyb2wgYXMgL2RldmljZXMv
cGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAw
MzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi44Mjc5MzddIGlucHV0OiBISUQgMDRkOToxMjAzIENvbnN1bWVyIENvbnRyb2wg
YXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0y
LzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDQKU2VwIDEzIDE0OjAyOjIw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi44Mjc5NjBdIGhpZC1nZW5lcmljIDAwMDM6MDREOToxMjAz
LjAwMDM6IGlucHV0LGhpZHJhdzI6IFVTQiBISUQgdjEuMTEgRGV2aWNlIFtISUQgMDRkOToxMjAz
XSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQxClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDIuOTM5NDM0XSBtcHQzc2FzX2NtMDogZGlhZyByZXNldDogU1VDQ0VTUwpTZXAg
MTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjk2MzkyMV0gdXNiIDEtMzogbmV3IGhp
Z2gtc3BlZWQgVVNCIGRldmljZSBudW1iZXIgNCB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTQ6MDI6
MjAgZGViaWFuIGtlcm5lbDogWyAgICAyLjk3NDQ1OF0gbXB0M3Nhc19jbTA6IHNjYXR0ZXIgZ2F0
aGVyOiBzZ2VfaW5fbWFpbl9tc2coMSksIHNnZV9wZXJfY2hhaW4oNyksIHNnZV9wZXJfaW8oMTI4
KSwgY2hhaW5zX3Blcl9pbygxOSkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAg
Mi45NzU1MjhdIG1wdDNzYXNfY20wOiByZXF1ZXN0IHBvb2woMHgoX19fX3B0cnZhbF9fX18pKSAt
IGRtYSgweGZmNTAwMDAwKTogZGVwdGgoNDIyNCksIGZyYW1lX3NpemUoMTI4KSwgcG9vbF9zaXpl
KDUyOCBrQikKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi45OTU2NDhdIG1w
dDNzYXNfY20wOiBzZW5zZSBwb29sKDB4KF9fX19wdHJ2YWxfX19fKSktIGRtYSgweGZlYjAwMDAw
KTogZGVwdGgoMzk2MyksZWxlbWVudF9zaXplKDk2KSwgcG9vbF9zaXplKDM3MSBrQikKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi45OTU3NTBdIG1wdDNzYXNfY20wOiBjb25m
aWcgcGFnZSgweChfX19fcHRydmFsX19fXykpIC0gZG1hKDB4ZmU5ZjYwMDApOiBzaXplKDUxMikK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi45OTU3NTFdIG1wdDNzYXNfY20w
OiBBbGxvY2F0ZWQgcGh5c2ljYWwgbWVtb3J5OiBzaXplKDcyNzgga0IpClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTk1NzUyXSBtcHQzc2FzX2NtMDogQ3VycmVudCBDb250
cm9sbGVyIFF1ZXVlIERlcHRoKDM5NjApLE1heCBDb250cm9sbGVyIFF1ZXVlIERlcHRoKDQwOTYp
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTk1NzUyXSBtcHQzc2FzX2Nt
MDogU2NhdHRlciBHYXRoZXIgRWxlbWVudHMgcGVyIElPKDEyOCkKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMy4xMjEzMzRdIHVzYiAxLTM6IE5ldyBVU0IgZGV2aWNlIGZvdW5k
LCBpZFZlbmRvcj0wNWUzLCBpZFByb2R1Y3Q9MDYxMCwgYmNkRGV2aWNlPTkzLjEwClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTIxMzM1XSB1c2IgMS0zOiBOZXcgVVNCIGRl
dmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTQ6
MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjEyMTMzNl0gdXNiIDEtMzogUHJvZHVjdDogVVNC
Mi4wIEh1YgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjEyMTMzN10gdXNi
IDEtMzogTWFudWZhY3R1cmVyOiBHZW5lc3lzTG9naWMKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4xMjUzNzZdIGh1YiAxLTM6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAx
NDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTI1OTYwXSBodWIgMS0zOjEuMDogNCBwb3J0
cyBkZXRlY3RlZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjE5NjU2OV0g
bXB0M3Nhc19jbTA6IF9iYXNlX2Rpc3BsYXlfZndwa2dfdmVyc2lvbjogY29tcGxldGUKU2VwIDEz
IDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xOTY4OTJdIG1wdDNzYXNfY20wOiBMU0lT
QVMzMDA4OiBGV1ZlcnNpb24oMTYuMDAuMTAuMDApLCBDaGlwUmV2aXNpb24oMHgwMiksIEJpb3NW
ZXJzaW9uKDE1LjAwLjAxLjAwKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAz
LjE5Njg5Ml0gbXB0M3Nhc19jbTA6IFByb3RvY29sPShJbml0aWF0b3IpLCBDYXBhYmlsaXRpZXM9
KFJhaWQsVExSLEVFRFAsU25hcHNob3QgQnVmZmVyLERpYWcgVHJhY2UgQnVmZmVyLFRhc2sgU2V0
IEZ1bGwsTkNRKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjE5Njk0MV0g
c2NzaSBob3N0MDogRnVzaW9uIE1QVCBTQVMgSG9zdApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtl
cm5lbDogWyAgICAzLjE5NzUxNV0gbXB0M3Nhc19jbTA6IHNlbmRpbmcgcG9ydCBlbmFibGUgISEK
U2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMjQ0MzddIFBNOiBJbWFnZSBu
b3QgZm91bmQgKGNvZGUgLTIyKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAz
LjI3MzExOV0gRVhUNC1mcyAobnZtZTNuMXAyKTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3Jk
ZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAzLjMwMzQzMl0gTm90IGFjdGl2YXRpbmcgTWFuZGF0b3J5IEFjY2VzcyBDb250cm9s
IGFzIC9zYmluL3RvbW95by1pbml0IGRvZXMgbm90IGV4aXN0LgpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjUwNjE1N10gcmFuZG9tOiBzeXN0ZW1kOiB1bmluaXRpYWxpemVk
IHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy41MDY5MDddIHJhbmRvbTogc3lzdGVtZDogdW5pbml0aWFsaXplZCB1cmFuZG9t
IHJlYWQgKDE2IGJ5dGVzIHJlYWQpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDMuNTA3MTA4XSByYW5kb206IHN5c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgx
NiBieXRlcyByZWFkKQpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjUyMjgy
Nl0gZnVzZTogaW5pdCAoQVBJIHZlcnNpb24gNy4zMikKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy41MjQ1MzldIEVYVDQtZnMgKG52bWUzbjFwMik6IHJlLW1vdW50ZWQuIE9w
dHM6IGVycm9ycz1yZW1vdW50LXJvClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAg
IDMuNjEzNDM1XSBhY3BpLXRhZCBBQ1BJMDAwRTowMDogVW5zdXBwb3J0ZWQgY2FwYWJpbGl0aWVz
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjcxMDExXSB4Z2VuZS1zbGlt
cHJvLWh3bW9uIEFQTUMwRDI5OjAwOiBBUE0gWC1HZW5lIFNvQyBIVyBtb25pdG9yIGRyaXZlciBy
ZWdpc3RlcmVkClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjc1NTU0XSBJ
UE1JIG1lc3NhZ2UgaGFuZGxlcjogdmVyc2lvbiAzOS4yClNlcCAxMyAxNDowMjoyMCBkZWJpYW4g
a2VybmVsOiBbICAgIDMuNjc4NjIxXSBpcG1pIGRldmljZSBpbnRlcmZhY2UKU2VwIDEzIDE0OjAy
OjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy42ODQ4NTVdIGlwbWlfc2k6IElQTUkgU3lzdGVtIElu
dGVyZmFjZSBkcml2ZXIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy42ODUy
MTZdIGlwbWlfc2k6IFVuYWJsZSB0byBmaW5kIGFueSBTeXN0ZW0gSW50ZXJmYWNlKHMpClNlcCAx
MyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzU4Nzc5XSBFWFQ0LWZzIChudm1lM24x
cDMpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51
bGwpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzU5MjU2XSBFWFQ0LWZz
IChudm1lM24xcDUpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4g
T3B0czogKG51bGwpClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzc5OTY0
XSBBZGRpbmcgMTAwMDQ0NGsgc3dhcCBvbiAvZGV2L252bWUzbjFwNC4gIFByaW9yaXR5Oi0yIGV4
dGVudHM6MSBhY3Jvc3M6MTAwMDQ0NGsgU1NGUwpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5l
bDogWyAgICAzLjc4MzUzMF0gRVhUNC1mcyAobnZtZTNuMXA2KTogbW91bnRlZCBmaWxlc3lzdGVt
IHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjc4NTY0M10gaXBtaV9zc2lmOiBJUE1JIFNTSUYgSW50ZXJmYWNl
IGRyaXZlcgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjgwMTUxOV0gcHN0
b3JlOiBVc2luZyBjcmFzaCBkdW1wIGNvbXByZXNzaW9uOiBkZWZsYXRlClNlcCAxMyAxNDowMjoy
MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODAxNTI4XSBwc3RvcmU6IFJlZ2lzdGVyZWQgZWZpIGFz
IHBlcnNpc3RlbnQgc3RvcmUgYmFja2VuZApTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDog
WyAgICAzLjgwOTcxNF0gc2JzYS1nd2R0IHNic2EtZ3dkdC4wOiBJbml0aWFsaXplZCB3aXRoIDEw
cyB0aW1lb3V0IEAgMjUwMDAwMDAgSHosIGFjdGlvbj0wLgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFu
IGtlcm5lbDogWyAgICAzLjgxNDA5M10gaXBtaV9zc2lmIGkyYy1BTVBDMDAwNDowMDogaXBtaV9z
c2lmOiBUcnlpbmcgQUNQSS1zcGVjaWZpZWQgU1NJRiBpbnRlcmZhY2UgYXQgaTJjIGFkZHJlc3Mg
MHgxMCwgYWRhcHRlciBTeW5vcHN5cyBEZXNpZ25XYXJlIEkyQyBhZGFwdGVyLCBzbGF2ZSBhZGRy
ZXNzIDB4MjAKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MjI3MzJdIGNy
eXB0ZDogbWF4X2NwdV9xbGVuIHNldCB0byAxMDAwClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDMuODMzODY4XSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVy
ClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODM0MDk2XSBpcG1pX3NpOiBV
bmFibGUgdG8gZmluZCBhbnkgU3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMgMTQ6MDI6MjAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjg3MTcwNV0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTIx
OTQwLjE5NjoyKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHBy
b2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2ZmaWNlLXhwZGZpbXBvcnQiIHBpZD0xMTgy
IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy44NzIwMzhdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyMTk0MC4yMDA6Myk6IGFw
cGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZp
bmVkIiBuYW1lPSJsc2JfcmVsZWFzZSIgcGlkPTExNzkgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjg3MjYwNl0gYXVkaXQ6IHR5cGU9
MTQwMCBhdWRpdCgxNjMxNTIxOTQwLjIwMDo0KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9u
PSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2ZmaWNlLW9v
cHNsYXNoIiBwaWQ9MTE4NCBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNDowMjoyMCBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuODcyNjY0XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1
MjE5NDAuMjAwOjUpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIg
cHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2Utc2VuZGRvYyIgcGlkPTExODcg
Y29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjg3Mjk5NV0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTIxOTQwLjIwMDo2KTogYXBw
YXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmlu
ZWQiIG5hbWU9Ii91c3IvYmluL21hbiIgcGlkPTExODUgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpT
ZXAgMTMgMTQ6MDI6MjAgZGViaWFuIGtlcm5lbDogWyAgICAzLjg3Mjk5OF0gYXVkaXQ6IHR5cGU9
MTQwMCBhdWRpdCgxNjMxNTIxOTQwLjIwMDo3KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9u
PSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im1hbl9maWx0ZXIiIHBp
ZD0xMTg1IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44NzMwMDBdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyMTk0MC4yMDA6
OCk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1
bmNvbmZpbmVkIiBuYW1lPSJtYW5fZ3JvZmYiIHBpZD0xMTg1IGNvbW09ImFwcGFybW9yX3BhcnNl
ciIKU2VwIDEzIDE0OjAyOjIwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NzM4MDFdIGF1ZGl0OiB0
eXBlPTE0MDAgYXVkaXQoMTYzMTUyMTk0MC4yMDA6OSk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJh
dGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJudmlkaWFfbW9k
cHJvYmUiIHBpZD0xMTg5IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjAyOjIwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMy44NzM4MDVdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUy
MTk0MC4yMDA6MTApOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIg
cHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibnZpZGlhX21vZHByb2JlLy9rbW9kIiBwaWQ9MTE4
OSBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuOTEzNTAzXSByYW5kb206IGNybmcgaW5pdCBkb25lClNlcCAxMyAxNDowMjoyMCBkZWJp
YW4ga2VybmVsOiBbICAgIDMuOTEzNTA1XSByYW5kb206IDcgdXJhbmRvbSB3YXJuaW5nKHMpIG1p
c3NlZCBkdWUgdG8gcmF0ZWxpbWl0aW5nClNlcCAxMyAxNDowMjoyMCBkZWJpYW4ga2VybmVsOiBb
ICAgIDQuMDgxNzAzXSBpcG1pX3NzaWYgaTJjLUFNUEMwMDA0OjAwOiBJUE1JIG1lc3NhZ2UgaGFu
ZGxlcjogRm91bmQgbmV3IEJNQyAobWFuX2lkOiAweDAwM2MwYSwgcHJvZF9pZDogMHgwMjEyLCBk
ZXZfaWQ6IDB4MjApClNlcCAxMyAxNDowMjoyMiBkZWJpYW4ga2VybmVsOiBbICAgIDUuNzQ5NzU2
XSBtcHQzc2FzX2NtMDogaG9zdF9hZGQ6IGhhbmRsZSgweDAwMDEpLCBzYXNfYWRkcigweDUwMDE0
ODUwMDA3NzllMzApLCBwaHlzKDgpClNlcCAxMyAxNDowMjoyNCBkZWJpYW4ga2VybmVsOiBbICAg
IDcuNjg4MzUwXSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IGlnYjogZW5vMiBOSUMgTGluayBpcyBV
cCAxMDAwIE1icHMgRnVsbCBEdXBsZXgsIEZsb3cgQ29udHJvbDogUlgvVFgKU2VwIDEzIDE0OjAy
OjI0IGRlYmlhbiBrZXJuZWw6IFsgICAgNy42ODg1MjVdIElQdjY6IEFERFJDT05GKE5FVERFVl9D
SEFOR0UpOiBlbm8yOiBsaW5rIGJlY29tZXMgcmVhZHkKU2VwIDEzIDE0OjAyOjI3IGRlYmlhbiBr
ZXJuZWw6IFsgICAxMC44NzU5MzBdIG1wdDNzYXNfY20wOiBwb3J0IGVuYWJsZTogU1VDQ0VTUwpT
ZXAgMTMgMTQ6MjQ6MjAgZGViaWFuIGtlcm5lbDogWyAxMzI0LjU2Njg1Ml0gZGV2aWNlLW1hcHBl
cjogdWV2ZW50OiB2ZXJzaW9uIDEuMC4zClNlcCAxMyAxNDoyNDoyMCBkZWJpYW4ga2VybmVsOiBb
IDEzMjQuNTY2OTMzXSBkZXZpY2UtbWFwcGVyOiBpb2N0bDogNC40My4wLWlvY3RsICgyMDIwLTEw
LTAxKSBpbml0aWFsaXNlZDogZG0tZGV2ZWxAcmVkaGF0LmNvbQpTZXAgMTMgMTQ6MjQ6MjMgZGVi
aWFuIGtlcm5lbDogWyAxMzI4LjA5MTU2MV0gU0dJIFhGUyB3aXRoIEFDTHMsIHNlY3VyaXR5IGF0
dHJpYnV0ZXMsIHJlYWx0aW1lLCBxdW90YSwgbm8gZGVidWcgZW5hYmxlZApTZXAgMTMgMTQ6MjQ6
MjMgZGViaWFuIGtlcm5lbDogWyAxMzI4LjA5ODU2Nl0gSkZTOiBuVHhCbG9jayA9IDgxOTIsIG5U
eExvY2sgPSA2NTUzNgpTZXAgMTMgMTQ6MjQ6MjMgZGViaWFuIGtlcm5lbDogWyAxMzI4LjEyMzIw
NF0gUU5YNCBmaWxlc3lzdGVtIDAuMi4zIHJlZ2lzdGVyZWQuClNlcCAxMyAxNDoyNDoyMyBkZWJp
YW4ga2VybmVsOiBbIDEzMjguMTk4MzExXSByYWlkNjogbmVvbng4ICAgZ2VuKCkgMTI5ODYgTUIv
cwpTZXAgMTMgMTQ6MjQ6MjMgZGViaWFuIGtlcm5lbDogWyAxMzI4LjI2NjI5N10gcmFpZDY6IG5l
b254OCAgIHhvcigpIDEwMjAwIE1CL3MKU2VwIDEzIDE0OjI0OjIzIGRlYmlhbiBrZXJuZWw6IFsg
MTMyOC4zMzQyOTZdIHJhaWQ2OiBuZW9ueDQgICBnZW4oKSAxMjcwMiBNQi9zClNlcCAxMyAxNDoy
NDoyMyBkZWJpYW4ga2VybmVsOiBbIDEzMjguNDAyMjkyXSByYWlkNjogbmVvbng0ICAgeG9yKCkg
MTA2NTUgTUIvcwpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5lbDogWyAxMzI4LjQ3MDI5Ml0g
cmFpZDY6IG5lb254MiAgIGdlbigpIDEyMTg0IE1CL3MKU2VwIDEzIDE0OjI0OjI0IGRlYmlhbiBr
ZXJuZWw6IFsgMTMyOC41MzgyOTJdIHJhaWQ2OiBuZW9ueDIgICB4b3IoKSAgOTYyMCBNQi9zClNl
cCAxMyAxNDoyNDoyNCBkZWJpYW4ga2VybmVsOiBbIDEzMjguNjA2MjkwXSByYWlkNjogbmVvbngx
ICAgZ2VuKCkgIDk4MzcgTUIvcwpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5lbDogWyAxMzI4
LjY3NDI4N10gcmFpZDY6IG5lb254MSAgIHhvcigpICA4Mzg2IE1CL3MKU2VwIDEzIDE0OjI0OjI0
IGRlYmlhbiBrZXJuZWw6IFsgMTMyOC43NDIyODhdIHJhaWQ2OiBpbnQ2NHg4ICBnZW4oKSAgNTk1
NCBNQi9zClNlcCAxMyAxNDoyNDoyNCBkZWJpYW4ga2VybmVsOiBbIDEzMjguODEwMjkzXSByYWlk
NjogaW50NjR4OCAgeG9yKCkgIDM1NTIgTUIvcwpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5l
bDogWyAxMzI4Ljg3ODI4Ml0gcmFpZDY6IGludDY0eDQgIGdlbigpICA2Nzg4IE1CL3MKU2VwIDEz
IDE0OjI0OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOC45NDYyODhdIHJhaWQ2OiBpbnQ2NHg0ICB4
b3IoKSAgMzU2OCBNQi9zClNlcCAxMyAxNDoyNDoyNCBkZWJpYW4ga2VybmVsOiBbIDEzMjkuMDE0
MjgyXSByYWlkNjogaW50NjR4MiAgZ2VuKCkgIDU3NjkgTUIvcwpTZXAgMTMgMTQ6MjQ6MjQgZGVi
aWFuIGtlcm5lbDogWyAxMzI5LjA4MjI3OV0gcmFpZDY6IGludDY0eDIgIHhvcigpICAzMTA4IE1C
L3MKU2VwIDEzIDE0OjI0OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOS4xNTAyNzddIHJhaWQ2OiBp
bnQ2NHgxICBnZW4oKSAgNDczMiBNQi9zClNlcCAxMyAxNDoyNDoyNCBkZWJpYW4ga2VybmVsOiBb
IDEzMjkuMjE4Mjg1XSByYWlkNjogaW50NjR4MSAgeG9yKCkgIDI0ODYgTUIvcwpTZXAgMTMgMTQ6
MjQ6MjQgZGViaWFuIGtlcm5lbDogWyAxMzI5LjIxODI4Nl0gcmFpZDY6IHVzaW5nIGFsZ29yaXRo
bSBuZW9ueDggZ2VuKCkgMTI5ODYgTUIvcwpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5lbDog
WyAxMzI5LjIxODI4N10gcmFpZDY6IC4uLi4geG9yKCkgMTAyMDAgTUIvcywgcm13IGVuYWJsZWQK
U2VwIDEzIDE0OjI0OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOS4yMTgyODhdIHJhaWQ2OiB1c2lu
ZyBuZW9uIHJlY292ZXJ5IGFsZ29yaXRobQpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5lbDog
WyAxMzI5LjIyMDIzNl0geG9yOiBtZWFzdXJpbmcgc29mdHdhcmUgY2hlY2tzdW0gc3BlZWQKU2Vw
IDEzIDE0OjI0OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOS4yMjA4NDhdICAgIDhyZWdzICAgICAg
ICAgICA6IDE2MTI4IE1CL3NlYwpTZXAgMTMgMTQ6MjQ6MjQgZGViaWFuIGtlcm5lbDogWyAxMzI5
LjIyMTM1N10gICAgMzJyZWdzICAgICAgICAgIDogMTkzNTIgTUIvc2VjClNlcCAxMyAxNDoyNDoy
NCBkZWJpYW4ga2VybmVsOiBbIDEzMjkuMjIxNzE5XSAgICBhcm02NF9uZW9uICAgICAgOiAyNzE3
OSBNQi9zZWMKU2VwIDEzIDE0OjI0OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOS4yMjE3MjBdIHhv
cjogdXNpbmcgZnVuY3Rpb246IGFybTY0X25lb24gKDI3MTc5IE1CL3NlYykKU2VwIDEzIDE0OjI0
OjI0IGRlYmlhbiBrZXJuZWw6IFsgMTMyOS4yNDgxMTNdIEJ0cmZzIGxvYWRlZCwgY3JjMzJjPWNy
YzMyYy1nZW5lcmljClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBCb290aW5nIExpbnV4IG9uIHBoeXNpY2FsIENQVSAweDAwMDAxMjAwMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBMaW51eCB2ZXJz
aW9uIDUuMTAuMC04LWFybTY0IChkZWJpYW4ta2VybmVsQGxpc3RzLmRlYmlhbi5vcmcpIChnY2Mt
MTAgKERlYmlhbiAxMC4yLjEtNikgMTAuMi4xIDIwMjEwMTEwLCBHTlUgbGQgKEdOVSBCaW51dGls
cyBmb3IgRGViaWFuKSAyLjM1LjIpICMxIFNNUCBEZWJpYW4gNS4xMC40Ni00ICgyMDIxLTA4LTAz
KQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZWZpOiBFRkkg
djIuNzAgYnkgQW1lcmljYW4gTWVnYXRyZW5kcwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gZWZpOiBBQ1BJIDIuMD0weGVlOTAwMDAwIFNNQklPUyAzLjA9MHhm
MWIzZmY5OCBFU1JUPTB4ZTk0ZjZmOTggTU9LdmFyPTB4ZTk0ZGYwMDAgUk5HPTB4Zjc2NmU4MTgg
TUVNUkVTRVJWRT0weGU5NjFmNjk4IApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gZWZpOiBzZWVkaW5nIGVudHJvcHkgcG9vbApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gc2VjdXJlYm9vdDogU2VjdXJlIGJvb3QgZGlzYWJs
ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGVzcnQ6IFJl
c2VydmluZyBFU1JUIHNwYWNlIGZyb20gMHgwMDAwMDAwMGU5NGY2Zjk4IHRvIDB4MDAwMDAwMDBl
OTRmNmZkMC4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IEVhcmx5IHRhYmxlIGNoZWNrc3VtIHZlcmlmaWNhdGlvbiBkaXNhYmxlZApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUlNEUCAweDAwMDAwMDAw
RUU5MDAwMDAgMDAwMDI0ICh2MDIgQW1wZXJlKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogWFNEVCAweDAwMDAwMDAwRUU4RjAwMDAgMDAwMEE0ICh2
MDEgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGQUNQIDB4MDAwMDAwMDBFRThE
MDAwMCAwMDAxMTQgKHYwNiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUwOSkK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IERTRFQg
MHgwMDAwMDAwMEVFODYwMDAwIDAxOUE5NSAodjAyIEFtcGVyZSBKYWRlICAgICAwMDAwMDAwMSBJ
TlRMIDIwMjAwNzE3KQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogREJHMiAweDAwMDAwMDAwRUU4RTAwMDAgMDAwMDVDICh2MDAgQW1wZXJlIEFsdHJh
ICAgIDAwMDAwMDAwIElOVEwgMjAxOTA1MDkpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBHVERUIDB4MDAwMDAwMDBFRThDMDAwMCAwMDAxMTAgKHYw
MyBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUwOSkKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNTRFQgMHgwMDAwMDAwMEVFOEIw
MDAwIDAwMDAyRCAodjAyIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMSBJTlRMIDIwMTkwNTA5KQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRklEVCAw
eDAwMDAwMDAwRUU4NTAwMDAgMDAwMDlDICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFN
SSAgMDAwMTAwMTMpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTUENSIDB4MDAwMDAwMDBFRTg0MDAwMCAwMDAwNTAgKHYwMiBBTEFTS0EgQSBNIEkg
ICAgMDEwNzIwMDkgQU1JICAwMDA1MDAwRikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IEJHUlQgMHgwMDAwMDAwMEVFODMwMDAwIDAwMDAzOCAodjAx
IEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTUNGRyAweDAwMDAwMDAwRUU4MjAw
MDAgMDAwMEFDICh2MDEgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIEFNUC4gMDEwMDAwMTMpClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBJT1JUIDB4
MDAwMDAwMDBFRTgxMDAwMCAwMDA2MTAgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1Q
LiAwMTAwMDAxMykKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IFBQVFQgMHgwMDAwMDAwMEVFN0YwMDAwIDAwNkU2MCAodjAyIEFtcGVyZSBBbHRyYSAg
ICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogU0xJVCAweDAwMDAwMDAwRUU3RTAwMDAgMDAwMDJEICh2MDEg
QW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUIDB4MDAwMDAwMDBFRTdEMDAw
MCAwMDA2RDAgKHYwMyBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEFQSUMgMHgw
MDAwMDAwMEVFODAwMDAwIDAwMTlGNCAodjA1IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMyBBTUkg
IDAxMDAwMDEzKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogUENDVCAweDAwMDAwMDAwRUU3QzAwMDAgMDAwNTc2ICh2MDIgQW1wZXJlIEFsdHJhICAg
IDAwMDAwMDAzIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBXU01UIDB4MDAwMDAwMDBFRTdCMDAwMCAwMDAwMjggKHYwMSBB
TEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZQRFQgMHgwMDAwMDAwMEVFN0EwMDAw
IDAwMDA0NCAodjAxIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAxMDAwMDEzKQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1BDUjogY29u
c29sZTogcGwwMTEsbW1pbzMyLDB4MTAwMDAyNjAwMDAwLDExNTIwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1JBVDogTm9kZSAwIFBYTSAwIFtt
ZW0gMHg4ODMwMDAwMC0weDg4M2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogU1JBVDogTm9kZSAwIFBYTSAwIFttZW0gMHg5MDAwMDAwMC0w
eGZmZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogU1JBVDogTm9kZSAwIFBYTSAwIFttZW0gMHg4MDAwMDAwMDAwMC0weDgwMDdmZmZmZmZm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1JB
VDogTm9kZSAwIFBYTSAwIFttZW0gMHg4MDEwMDAwMDAwMC0weDgwZmZmZmZmZmZmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gTlVNQTogTk9ERV9EQVRBIFtt
ZW0gMHg4MGZmODA4NmIwMC0weDgwZmY4MDg4ZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gWm9uZSByYW5nZXM6ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSAgICAgIFttZW0gMHgwMDAwMDAwMDg4MzAwMDAw
LTB4MDAwMDAwMDBmZmZmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdICAgRE1BMzIgICAgZW1wdHkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsICAgW21lbSAweDAwMDAwMDAxMDAwMDAwMDAtMHgwMDAw
MDgwZmZmZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gTW92YWJsZSB6b25lIHN0YXJ0IGZvciBlYWNoIG5vZGUKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEVhcmx5IG1lbW9yeSBub2RlIHJhbmdlcwpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAw
eDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMDg4M2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwOTAw
MDAwMDAtMHgwMDAwMDAwMDkxZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwOTIwMDAwMDAtMHgwMDAw
MDAwMDkzZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwOTQwMDAwMDAtMHgwMDAwMDAwMGVlMmU2ZmZm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAg
MDogW21lbSAweDAwMDAwMDAwZWUyZTcwMDAtMHgwMDAwMDAwMGVlNzQwZmZmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAw
MDAwMDAwZWU3NDEwMDAtMHgwMDAwMDAwMGVlNzQ0ZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3NDUw
MDAtMHgwMDAwMDAwMGVlNzRjZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3NGQwMDAtMHgwMDAwMDAw
MGVlNzRkZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
ICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU3NGUwMDAtMHgwMDAwMDAwMGVlNzRmZmZmXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDog
W21lbSAweDAwMDAwMDAwZWU3NTAwMDAtMHgwMDAwMDAwMGVlOTZmZmZmXQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAw
MDAwZWU5NzAwMDAtMHgwMDAwMDAwMGVlOTdmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZWU5ODAwMDAt
MHgwMDAwMDAwMGYwNjRmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZjA2NTAwMDAtMHgwMDAwMDAwMGY3
NjZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBu
b2RlICAgMDogW21lbSAweDAwMDAwMDAwZjc2NzAwMDAtMHgwMDAwMDAwMGY3ODRmZmZmXQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21l
bSAweDAwMDAwMDAwZjc4NTAwMDAtMHgwMDAwMDAwMGY3ZmRmZmZmXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAw
ZjdmZTAwMDAtMHgwMDAwMDAwMGZmYzhlZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZmZjOGYwMDAtMHgw
MDAwMDAwMGZmYzhmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gICBub2RlICAgMDogW21lbSAweDAwMDAwMDAwZmZjOTAwMDAtMHgwMDAwMDAwMGZmZmZm
ZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2Rl
ICAgMDogW21lbSAweDAwMDAwODAwMDAwMDAwMDAtMHgwMDAwMDgwMDdmZmZmZmZmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBub2RlICAgMDogW21lbSAw
eDAwMDAwODAxMDAwMDAwMDAtMHgwMDAwMDgwZmZmZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSW5pdG1lbSBzZXR1cCBub2RlIDAgW21lbSAweDAw
MDAwMDAwODgzMDAwMDAtMHgwMDAwMDgwZmZmZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gT24gbm9kZSAwIHRvdGFscGFnZXM6IDE2NzExOTM2ClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiA3
MTcyIHBhZ2VzIHVzZWQgZm9yIG1lbW1hcApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gICBETUEgem9uZTogMCBwYWdlcyByZXNlcnZlZApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBETUEgem9uZTogNDU5MDA4IHBhZ2Vz
LCBMSUZPIGJhdGNoOjYzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIE5vcm1hbCB6b25lOiAyNTM5NTIgcGFnZXMgdXNlZCBmb3IgbWVtbWFwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIE5vcm1hbCB6b25lOiAxNjI1
MjkyOCBwYWdlcywgTElGTyBiYXRjaDo2MwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gY21hOiBSZXNlcnZlZCA2NCBNaUIgYXQgMHgwMDAwMDAwMGZiYzAwMDAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2NpOiBwcm9i
aW5nIGZvciBjb25kdWl0IG1ldGhvZCBmcm9tIEFDUEkuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2NpOiBQU0NJdjEuMSBkZXRlY3RlZCBpbiBmaXJtd2Fy
ZS4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFVz
aW5nIHN0YW5kYXJkIFBTQ0kgdjAuMiBmdW5jdGlvbiBJRHMKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IE1JR1JBVEVfSU5GT19UWVBFIG5vdCBzdXBw
b3J0ZWQuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwc2Np
OiBTTUMgQ2FsbGluZyBDb252ZW50aW9uIHYxLjIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MCAt
PiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDMwMTAwIC0+IE5vZGUg
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4NDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDUwMDAwIC0+
IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg1MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4NjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDYw
MTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg3MDAwMCAtPiBOb2RlIDAKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4NzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDgwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg4MDEwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDkwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhhMDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YTAxMDAgLT4gTm9kZSAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweGIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhi
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YzAwMDAgLT4gTm9kZSAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweGMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHhkMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZDAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHhlMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGYwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDEwMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTEwMDAwIC0+IE5vZGUg
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDEyMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIwMTAw
IC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMzAwMDAgLT4gTm9kZSAwClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDEzMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MTQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MTUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNjAw
MDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDEwMCAtPiBOb2RlIDAKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MTcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgxNzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE4MDAwMCAtPiBO
b2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxOTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE5
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWEwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDFiMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYzAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDFjMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MWQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFlMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MWUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZjAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgyMDAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIxMDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgyMjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIyMDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjMwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyMzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDI0MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDI1MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjYw
MDAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAxMDAgLT4gTm9kZSAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDI3MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MjcwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gcGVyY3B1OiBFbWJlZGRlZCAzMyBwYWdlcy9jcHUgczk1MTkyIHI4MTkyIGQz
MTc4NCB1MTM1MTY4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBwY3B1LWFsbG9jOiBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjggYWxsb2M9MzMqNDA5NgpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzog
WzBdIDAwIFswXSAwMSBbMF0gMDIgWzBdIDAzIFswXSAwNCBbMF0gMDUgWzBdIDA2IFswXSAwNyAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6
IFswXSAwOCBbMF0gMDkgWzBdIDEwIFswXSAxMSBbMF0gMTIgWzBdIDEzIFswXSAxNCBbMF0gMTUg
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9j
OiBbMF0gMTYgWzBdIDE3IFswXSAxOCBbMF0gMTkgWzBdIDIwIFswXSAyMSBbMF0gMjIgWzBdIDIz
IApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxv
YzogWzBdIDI0IFswXSAyNSBbMF0gMjYgWzBdIDI3IFswXSAyOCBbMF0gMjkgWzBdIDMwIFswXSAz
MSAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxs
b2M6IFswXSAzMiBbMF0gMzMgWzBdIDM0IFswXSAzNSBbMF0gMzYgWzBdIDM3IFswXSAzOCBbMF0g
MzkgClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFs
bG9jOiBbMF0gNDAgWzBdIDQxIFswXSA0MiBbMF0gNDMgWzBdIDQ0IFswXSA0NSBbMF0gNDYgWzBd
IDQ3IApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1h
bGxvYzogWzBdIDQ4IFswXSA0OSBbMF0gNTAgWzBdIDUxIFswXSA1MiBbMF0gNTMgWzBdIDU0IFsw
XSA1NSAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUt
YWxsb2M6IFswXSA1NiBbMF0gNTcgWzBdIDU4IFswXSA1OSBbMF0gNjAgWzBdIDYxIFswXSA2MiBb
MF0gNjMgClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1
LWFsbG9jOiBbMF0gNjQgWzBdIDY1IFswXSA2NiBbMF0gNjcgWzBdIDY4IFswXSA2OSBbMF0gNzAg
WzBdIDcxIApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNw
dS1hbGxvYzogWzBdIDcyIFswXSA3MyBbMF0gNzQgWzBdIDc1IFswXSA3NiBbMF0gNzcgWzBdIDc4
IFswXSA3OSAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBHSUMgc3lzdGVtIHJlZ2lz
dGVyIENQVSBpbnRlcmZhY2UKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFZpcnR1YWxpemF0aW9uIEhvc3QgRXh0ZW5z
aW9ucwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZl
YXR1cmVzOiBrZXJuZWwgcGFnZSB0YWJsZSBpc29sYXRpb24gZm9yY2VkIE9OIGJ5IEtBU0xSClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6
IGRldGVjdGVkOiBLZXJuZWwgcGFnZSB0YWJsZSBpc29sYXRpb24gKEtQVEkpClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVk
OiBIYXJkd2FyZSBkaXJ0eSBiaXQgbWFuYWdlbWVudApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3BlY3RyZS12NApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZlYXR1cmVz
OiBkZXRlY3RlZDogQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gYWx0ZXJuYXRpdmVzOiBwYXRjaGluZyBrZXJuZWwgY29kZQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQnVpbHQgMSB6b25l
bGlzdHMsIG1vYmlsaXR5IGdyb3VwaW5nIG9uLiAgVG90YWwgcGFnZXM6IDE2NDUwODEyClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBQb2xpY3kgem9uZTogTm9y
bWFsClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBLZXJuZWwg
Y29tbWFuZCBsaW5lOiBCT09UX0lNQUdFPS9ib290L3ZtbGludXotNS4xMC4wLTgtYXJtNjQgcm9v
dD1VVUlEPTZjMDgzMjdjLTRiYmQtNDM0MS1hNDI3LWE3MmJjMTUzMzg5NSBybyBxdWlldApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVm
X2xlbiBpbmRpdmlkdWFsIG1heCBjcHUgY29udHJpYnV0aW9uOiA0MDk2IGJ5dGVzClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVu
IHRvdGFsIGNwdV9leHRyYSBjb250cmlidXRpb25zOiAzMjM1ODQgYnl0ZXMKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4gbWlu
IHNpemU6IDEzMTA3MiBieXRlcwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbjogNTI0Mjg4IGJ5dGVzClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGVhcmx5IGxvZyBidWYgZnJl
ZTogMTE5MDE2KDkwJSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIERlbnRyeSBjYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDgzODg2MDggKG9yZGVyOiAxNCwg
NjcxMDg4NjQgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElub2RlLWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogNDE5NDMwNCAob3Jk
ZXI6IDEzLCAzMzU1NDQzMiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gbWVtIGF1dG8taW5pdDogc3RhY2s6b2ZmLCBoZWFwIGFsbG9j
Om9uLCBoZWFwIGZyZWU6b2ZmClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBzb2Z0d2FyZSBJTyBUTEI6IG1hcHBlZCBbbWVtIDB4MDAwMDAwMDBlYTJlNzAwMC0w
eDAwMDAwMDAwZWUyZTcwMDBdICg2NE1CKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gTWVtb3J5OiAyNTUxNjMySy82Njg0Nzc0NEsgYXZhaWxhYmxlICgxMTc3
Nksga2VybmVsIGNvZGUsIDI0MzZLIHJ3ZGF0YSwgNzAwOEsgcm9kYXRhLCA1NDQwSyBpbml0LCA1
OThLIGJzcywgMTU4NTU0MEsgcmVzZXJ2ZWQsIDY1NTM2SyBjbWEtcmVzZXJ2ZWQpClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByYW5kb206IGdldF9yYW5kb21f
dTY0IGNhbGxlZCBmcm9tIF9fa21lbV9jYWNoZV9jcmVhdGUrMHgzOC8weDU2MCB3aXRoIGNybmdf
aW5pdD0wClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTTFVC
OiBIV2FsaWduPTY0LCBPcmRlcj0wLTMsIE1pbk9iamVjdHM9MCwgQ1BVcz04MCwgTm9kZXM9MQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZnRyYWNlOiBhbGxv
Y2F0aW5nIDM4NTM1IGVudHJpZXMgaW4gMTUxIHBhZ2VzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRlZCAxNTEgcGFnZXMgd2l0aCA1
IGdyb3VwcwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcmN1
OiBIaWVyYXJjaGljYWwgUkNVIGltcGxlbWVudGF0aW9uLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcmN1OiAJUkNVIHJlc3RyaWN0aW5nIENQVXMgZnJvbSBO
Ul9DUFVTPTI1NiB0byBucl9jcHVfaWRzPTgwLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gCVJ1ZGUgdmFyaWFudCBvZiBUYXNrcyBSQ1UgZW5hYmxlZC4KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIAlUcmFjaW5nIHZhcmlh
bnQgb2YgVGFza3MgUkNVIGVuYWJsZWQuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSByY3U6IFJDVSBjYWxjdWxhdGVkIHZhbHVlIG9mIHNjaGVkdWxlci1lbmxp
c3RtZW50IGRlbGF5IGlzIDI1IGppZmZpZXMuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSByY3U6IEFkanVzdGluZyBnZW9tZXRyeSBmb3IgcmN1X2Zhbm91dF9s
ZWFmPTE2LCBucl9jcHVfaWRzPTgwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBOUl9JUlFTOiA2NCwgbnJfaXJxczogNjQsIHByZWFsbG9jYXRlZCBpcnFzOiAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogR0lD
OiBVc2luZyBzcGxpdCBFT0kvRGVhY3RpdmF0ZSBtb2RlClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogNjcyIFNQSXMgaW1wbGVtZW50ZWQKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiAwIEV4dGVuZGVk
IFNQSXMgaW1wbGVtZW50ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEdJQ3YzOiBEaXN0cmlidXRvciBoYXMgbm8gUmFuZ2UgU2VsZWN0b3Igc3VwcG9ydApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDE2IFBQ
SXMgaW1wbGVtZW50ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEdJQ3YzOiBDUFUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA1YzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAwIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDEgLT4gTm9kZSAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAt
PiBJVFMgMiAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAzIC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDQgLT4gTm9kZSAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0g
MCAtPiBJVFMgNSAtPiBOb2RlIDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA2IC0+IE5vZGUgMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDcgLT4gTm9k
ZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21l
bSAweDEwMDEwMDA0MDAwMC0weDEwMDEwMDA1ZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNDAwMDA6IGFsbG9jYXRlZCA4
MTkyIERldmljZXMgQDgwMDAwMmMwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAx
KQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAw
MDAyZDAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDYwMDAwLTB4MTAw
MTAwMDdmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTQDB4MDAwMDEwMDEwMDA2MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAyZjAw
MDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDYwMDAwOiBhbGxvY2F0
ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDMwMDAwMCAoZmxhdCwgZXN6IDIs
IHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwODAwMDAtMHgxMDAxMDAwOWZmZmZdClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDgwMDAw
OiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDMyMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBw
c3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29s
bGVjdGlvbnMgQDgwMDAwMzMwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEw
MDBhMDAwMC0weDEwMDEwMDBiZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwYTAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmlj
ZXMgQDgwMDAwMzUwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBh
MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzNjAwMDAg
KGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGMwMDAwLTB4MTAwMTAwMGRmZmZm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDBjMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzODAwMDAgKGluZGly
ZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAwOiBhbGxvY2F0ZWQgMzI3Njgg
SW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM5MDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEss
IHNociAxKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRT
IFttZW0gMHgxMDAxMDAwZTAwMDAtMHgxMDAxMDAwZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGUwMDAwOiBhbGxvY2F0
ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDNiMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBz
aHIgMSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0Aw
eDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMg
QDgwMDAwM2MwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDEwMDAwMC0w
eDEwMDEwMDExZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIElUU0AweDAwMDAxMDAxMDAxMDAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAw
M2UwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEwMDAwMDogYWxs
b2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzZjAwMDAgKGZsYXQsIGVz
eiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTIwMDAwLTB4MTAwMTAwMTNmZmZmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEy
MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDA4MTAwMDAgKGluZGlyZWN0LCBlc3og
OCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0
IENvbGxlY3Rpb25zIEA4MDAwMDgyMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IHVzaW5n
IExQSSBwcm9wZXJ0eSB0YWJsZSBAMHgwMDAwMDgwMDAwODMwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogQ1BVMDogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4NDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEkgR1REVDogZm91bmQgMSBtZW1vcnktbWFwcGVkIHRp
bWVyIGJsb2NrKHMpLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gYXJjaF90aW1lcjogY3AxNSBhbmQgbW1pbyB0aW1lcihzKSBydW5uaW5nIGF0IDI1LjAwTUh6
IChwaHlzL3BoeXMpLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gY2xvY2tzb3VyY2U6IGFyY2hfc3lzX2NvdW50ZXI6IG1hc2s6IDB4ZmZmZmZmZmZmZmZmZmYg
bWF4X2N5Y2xlczogMHg1YzQwOTM5YjUsIG1heF9pZGxlX25zOiA0NDA3OTUyMDI2NDYgbnMKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNjaGVkX2Nsb2NrOiA1
NiBiaXRzIGF0IDI1TUh6LCByZXNvbHV0aW9uIDQwbnMsIHdyYXBzIGV2ZXJ5IDQzOTgwNDY1MTEx
MDBucwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDA1MV0gQ29uc29s
ZTogY29sb3VyIGR1bW15IGRldmljZSA4MHgyNQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDA3M10gQUNQSTogQ29yZSByZXZpc2lvbiAyMDIwMDkyNQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDI2Nl0gQ2FsaWJyYXRpbmcgZGVsYXkgbG9v
cCAoc2tpcHBlZCksIHZhbHVlIGNhbGN1bGF0ZWQgdXNpbmcgdGltZXIgZnJlcXVlbmN5Li4gNTAu
MDAgQm9nb01JUFMgKGxwaj0xMDAwMDApClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMjY4XSBwaWRfbWF4OiBkZWZhdWx0OiA4MTkyMCBtaW5pbXVtOiA2NDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzMDFdIExTTTogU2VjdXJpdHkgRnJh
bWV3b3JrIGluaXRpYWxpemluZwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDMwNl0gWWFtYTogZGlzYWJsZWQgYnkgZGVmYXVsdDsgZW5hYmxlIHdpdGggc3lzY3RsIGtl
cm5lbC55YW1hLioKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzMjRd
IEFwcEFybW9yOiBBcHBBcm1vciBpbml0aWFsaXplZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDMyNl0gVE9NT1lPIExpbnV4IGluaXRpYWxpemVkClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzY2XSBNb3VudC1jYWNoZSBoYXNoIHRhYmxl
IGVudHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzOTBdIE1vdW50cG9pbnQtY2FjaGUg
aGFzaCB0YWJsZSBlbnRyaWVzOiAxMzEwNzIgKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5l
YXIpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMjUyXSByY3U6IEhp
ZXJhcmNoaWNhbCBTUkNVIGltcGxlbWVudGF0aW9uLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTM5NF0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOTZd
IFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxMzk3XSBQbGF0Zm9ybSBNU0k6IElUU0Aw
eDEwMDEwMDA4MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMTM5OV0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9tYWlu
IGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDBdIFBs
YXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDAxXSBQbGF0Zm9ybSBNU0k6IElUU0AweDEw
MDEwMDBlMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMTQwMl0gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNy
ZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDRdIFBsYXRm
b3JtIE1TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDA4XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwNDAw
MDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDE0MDldIFBDSS9NU0k6IElUU0AweDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxMF0gUENJL01TSTogSVRTQDB4MTAw
MTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAxNDExXSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTRdIFBDSS9NU0k6IElU
U0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMTQxN10gUENJL01TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBj
cmVhdGVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDE4XSBQQ0kv
TVNJOiBJVFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTldIFBDSS9NU0k6IElUU0AweDEwMDEwMDEyMDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQy
Ml0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDQwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDIzXSBmc2wtbWMgTVNJOiBJVFNAMHgx
MDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDE0MjVdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA4MDAwMCBkb21haW4gY3Jl
YXRlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyNl0gZnNsLW1j
IE1TSTogSVRTQDB4MTAwMTAwMGEwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI3XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwYzAw
MDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDE0MjldIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBlMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzMF0gZnNsLW1jIE1TSTogSVRT
QDB4MTAwMTAwMTAwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAxNDMxXSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMjAwMDAgZG9tYWlu
IGNyZWF0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MzddIFJl
bWFwcGluZyBhbmQgZW5hYmxpbmcgRUZJIHNlcnZpY2VzLgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMzkwNl0gc21wOiBCcmluZ2luZyB1cCBzZWNvbmRhcnkgQ1BVcyAu
Li4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ1OTBdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFUxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA0NjA4XSBHSUN2MzogQ1BVMTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAwMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwN2MwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA0NjI3XSBHSUN2MzogQ1BVMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDA4NTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDQ2NThdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ2
NjhdIENQVTE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFhMDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUxODRdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA1MTk5XSBHSUN2MzogQ1BVMjogZm91bmQgcmVkaXN0cmlidXRvciAxNDAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwNjQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA1MjE2XSBHSUN2MzogQ1BVMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA4NjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDUyNTFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDUyNjBdIENQVTI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDAwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU0NjFdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA1NDc4XSBHSUN2MzogQ1BVMzogZm91bmQgcmVkaXN0cmlidXRvciAxYzAwMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwODQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA1NDk1XSBHSUN2MzogQ1BVMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA4NzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDU1MjhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDU1MzddIENQVTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFjMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU4NDVd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA1ODU5XSBHSUN2MzogQ1BVNDogZm91bmQgcmVkaXN0cmlidXRvciAxMDAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA1ODc3XSBHSUN2MzogQ1BVNDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ODAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDU5MTJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDU5MjJdIENQVTQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEwMDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDYy
NDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA2MjYzXSBHSUN2MzogQ1BVNTogZm91bmQgcmVkaXN0cmlidXRvciAx
ODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA2MjgxXSBHSUN2MzogQ1BVNTogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4OTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDYzMTddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDYzMjddIENQVTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE4
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDY2NDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA2NjYyXSBHSUN2MzogQ1BVNjogZm91bmQgcmVkaXN0cmlidXRv
ciAxNjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA2NjgxXSBHSUN2MzogQ1BVNjogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4YTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDY3MTZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDY3MjZdIENQVTY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDE2MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDcwNDFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3MDU5XSBHSUN2MzogQ1BVNzogZm91bmQgcmVkaXN0cmli
dXRvciAxZTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGMwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3MDc3XSBHSUN2MzogQ1BVNzogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4YjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDcxMTJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDcxMjJdIENQVTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDFlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDc0NDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU4ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3NDU5XSBHSUN2MzogQ1BVODogZm91bmQgcmVkaXN0
cmlidXRvciBhMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzYzAwMDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDc0NzddIEdJQ3YzOiBDUFU4OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhjMDAwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzUxNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNzUyNV0gQ1BVODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMGEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNzg1MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTkKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDc4NzBdIEdJQ3YzOiBDUFU5OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDIyMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5YzAwMDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDc4ODhdIEdJQ3YzOiBDUFU5OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhkMDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzkxOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNzkyOV0gQ1BVOTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMjIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwODI0NV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MjU4XSBHSUN2MzogQ1BVMTA6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgYzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDQwMDAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4Mjc4XSBHSUN2MzogQ1BVMTA6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGUwMDAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MzE0XSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4MzI1XSBDUFUxMDogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMGMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwODY1MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTExClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjY5XSBHSUN2MzogQ1BVMTE6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGE0MDAwMApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODY4OV0gR0lDdjM6IENQVTEx
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhmMDAwMApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODcyMl0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwODczMl0gQ1BVMTE6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI0MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDkwNDRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFUxMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTA1Nl0gR0lDdjM6
IENQVTEyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDgwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDM0
MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTA3Nl0gR0lDdjM6
IENQVTEyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkw
MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTExMV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTEyMl0gQ1BVMTI6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA4MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk0NTRdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFUxMwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ3M10g
R0lDdjM6IENQVTEzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIwMDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA5NDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk0OTJd
IEdJQ3YzOiBDUFUxMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5MTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk1MjRd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk1MzVdIENQVTEz
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMDAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5ODQ4XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMTQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDk4NjJdIEdJQ3YzOiBDUFUxNDogZm91bmQgcmVkaXN0cmlidXRvciBlMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA0YzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDk4ODRdIEdJQ3YzOiBDUFUxNDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDA5MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDk5MjFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk5MzJd
IENQVTE0OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZTAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwMjUwXSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVMTUKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTAyNzBdIEdJQ3YzOiBDUFUxNTogZm91bmQgcmVkaXN0cmlidXRvciAyNjAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwYWMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwMjkwXSBHSUN2MzogQ1BVMTU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwOTMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwMzIzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDEwMzM1XSBDUFUxNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjYwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDg1MV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDEwODYyXSBHSUN2MzogQ1BVMTY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMWMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDEwODgyXSBHSUN2MzogQ1BVMTY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDEwOTIwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEwOTMyXSBDUFUxNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDIw
MDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
MTI1MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE3ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDExMjYxXSBHSUN2MzogQ1BVMTc6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMjQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDExMjgyXSBHSUN2MzogQ1BVMTc6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDExMzIwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDExMzMyXSBDUFUxNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMDQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxMTY1MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTE4ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNjYwXSBHSUN2MzogQ1BVMTg6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMCByZWdpb24gMDoweDAwMDAxMDAxMDAxNDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTE2ODJdIEdJQ3YzOiBDUFUxODogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTE3MjBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTE3MzJdIENQVTE4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAwMDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEyMDUxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTkKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIwNjNdIEdJQ3YzOiBDUFUxOTogZm91bmQgcmVk
aXN0cmlidXRvciA2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyYzAwMDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIwODRdIEdJQ3YzOiBDUFUxOTogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NzAwMDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIxMjBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTIxMzNdIENQVTE5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAwNjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDEyNDUwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI0NjVdIEdJQ3YzOiBDUFUyMDogZm91
bmQgcmVkaXN0cmlidXRvciAxMzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjAwMDAwClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNDg1XSBHSUN2MzogQ1BVMjA6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTgwMDAwClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNTIwXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyNTMyXSBDUFUyMDogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMTMwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMjg0NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIx
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODYwXSBHSUN2MzogQ1BV
MjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDgwMDAw
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjg4MF0gR0lDdjM6IENQ
VTIxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk5MDAw
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjkxM10gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjkyNV0gQ1BVMjE6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMyNDFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUyMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI1Nl0gR0lD
djM6IENQVTIyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA2ODAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMyNzddIEdJ
Q3YzOiBDUFUyMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDA5YTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMzMTFdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTMzMjNdIENQVTIyOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNjQzXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVMjMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2
NjBdIEdJQ3YzOiBDUFUyMzogZm91bmQgcmVkaXN0cmlidXRvciAxZDAwMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwODgwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEz
NjgxXSBHSUN2MzogQ1BVMjM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwOWIwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEz
NzE0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzNzI3XSBD
UFUyMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWQwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDA0NF0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTI0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE0MDU4XSBHSUN2MzogQ1BVMjQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTEwMDAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDU4MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNDA3OV0gR0lDdjM6IENQVTI0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDljMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNDExNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NDEyNl0gQ1BVMjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDExMDAwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ0NDVdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxNDQ2MV0gR0lDdjM6IENQVTI1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE5MDAw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA3ODAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTQ0ODJdIEdJQ3YzOiBDUFUyNTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ZDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTQ1MTZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQ1MjldIENQVTI1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxOTAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0
ODUwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjYKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTQ4NjZdIEdJQ3YzOiBDUFUyNjogZm91bmQgcmVkaXN0cmlidXRv
ciAxNzAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE0ODg4XSBHSUN2MzogQ1BVMjY6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE0OTIzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE0OTM2XSBDUFUyNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMTcwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNTI1MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI3ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MjY5XSBHSUN2MzogQ1BVMjc6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMWYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDkwMDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTI5MV0gR0lDdjM6IENQVTI3OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlmMDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTMyNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNTMzOV0gQ1BVMjc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDFmMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTU2NDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyOApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTY1OV0gR0lDdjM6IENQVTI4OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIGIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQwMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTY4Ml0gR0lDdjM6IENQVTI4OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEwMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTcxOV0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNTczMV0gQ1BVMjg6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDBiMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTYwNTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyOQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjA2OV0gR0lDdjM6IENQVTI5
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhMDAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTYwOTBdIEdJQ3YzOiBDUFUy
OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMTAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTYxMjNdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTYxMzZdIENQVTI5OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2NDQ2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMzAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY0NTldIEdJQ3Yz
OiBDUFUzMDogZm91bmQgcmVkaXN0cmlidXRvciBkMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0
ODAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY0ODFdIEdJQ3Yz
OiBDUFUzMDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBh
MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY1MThdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY1MzFdIENQVTMwOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2ODUwXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMzEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4Njld
IEdJQ3YzOiBDUFUzMTogZm91bmQgcmVkaXN0cmlidXRvciAyNTAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwYTgwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2ODkx
XSBHSUN2MzogQ1BVMzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYTMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2OTI0
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2OTM4XSBDUFUz
MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjUwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzI0NF0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTMyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3MjU3XSBHSUN2MzogQ1BVMzI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgOTAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwMzgwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3MjgwXSBHSUN2MzogQ1BVMzI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYTQwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE3MzE1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3MzI5
XSBDUFUzMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDkwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNzY1MV0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTMzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE3NjY5XSBHSUN2MzogQ1BVMzM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjEwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDk4MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNzY5MV0gR0lDdjM6IENQVTMzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGE1MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNzcyNl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNzc0MF0gQ1BVMzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIxMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTgwNDVd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODA1OV0gR0lDdjM6IENQVTM0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGYw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODA4M10gR0lDdjM6IENQVTM0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE2MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODEyMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODEzNV0gQ1BVMzQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBm
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTg0NTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxODQ3Ml0gR0lDdjM6IENQVTM1OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDI3MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBiMDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTg0OTVdIEdJQ3YzOiBDUFUzNTogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhNzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTg1MzBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTg1NDVdIENQVTM1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAyNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE4ODQ4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzYKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg4NThdIEdJQ3YzOiBDUFUzNjogZm91bmQgcmVk
aXN0cmlidXRvciAzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyMDAwMDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg4ODNdIEdJQ3YzOiBDUFUzNjogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhODAwMDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg5MjFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTg5MzZdIENQVTM2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAwMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE5MjQ5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzcKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkyNjBdIEdJQ3YzOiBDUFUzNzogZm91
bmQgcmVkaXN0cmlidXRvciA1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyODAwMDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkyODRdIEdJQ3YzOiBDUFUzNzogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhOTAwMDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkzMjJdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTkzMzddIENQVTM3OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAwNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE5NjUxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzgK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk2NjFdIEdJQ3YzOiBDUFUz
ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxODAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk2ODZdIEdJQ3YzOiBDUFUz
ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYTAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk3MjVdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTk3NDBdIENQVTM4OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwMDQ4XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMzkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAwNjBdIEdJQ3Yz
OiBDUFUzOTogZm91bmQgcmVkaXN0cmlidXRvciA3MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAz
MDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAwODVdIEdJQ3Yz
OiBDUFUzOTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBh
YjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxMjFdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAxMzddIENQVTM5OiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNDQ3XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0NjFd
IEdJQ3YzOiBDUFU0MDogZm91bmQgcmVkaXN0cmlidXRvciAxMjAxMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwNWUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNDg0
XSBHSUN2MzogQ1BVNDA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYWMwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNTE2
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwNTMxXSBDUFU0
MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTIwMTAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDg2M10gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTQxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDIwODc5XSBHSUN2MzogQ1BVNDE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWEwMTAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDdlMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMDkwMl0gR0lDdjM6IENQVTQxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGFkMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMDkzNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDk1
MF0gQ1BVNDE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFhMDEwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjEyNDldIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyMTI2NF0gR0lDdjM6IENQVTQyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA2NjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjEyODhdIEdJQ3YzOiBDUFU0MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBhZTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjEzMjNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjEzMzhdIENQVTQyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNDAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxNjUx
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjE2NjhdIEdJQ3YzOiBDUFU0MzogZm91bmQgcmVkaXN0cmlidXRvciAx
YzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIxNjkxXSBHSUN2MzogQ1BVNDM6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYWYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIxNzI0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIxNzM5XSBDUFU0MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MWMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMjA0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ0ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMDYzXSBHSUN2MzogQ1BVNDQ6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMTAwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDU2MDAwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjA4OV0gR0lDdjM6IENQVTQ0OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIwMDAwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjEyM10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMjEzOF0gQ1BVNDQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDEwMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjI0NDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjQ2NF0gR0lDdjM6IENQVTQ1OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDE4MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3NjAwMDAKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI0ODhdIEdJQ3YzOiBDUFU0NTogdXNpbmcg
YWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMTAwMDAKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI1MjBdIGFyY2hfdGltZXI6IEVuYWJsaW5n
IGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI1MzZdIENQVTQ1OiBCb290ZWQgc2Vjb25kYXJ5IHBy
b2Nlc3NvciAweDAwMDAxODAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIyODUxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDYKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI4NjddIEdJQ3YzOiBDUFU0Njog
Zm91bmQgcmVkaXN0cmlidXRvciAxNjAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmUwMDAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyODkyXSBHSUN2MzogQ1BVNDY6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjIwMDAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyOTI3XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyOTQyXSBDUFU0NjogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzI0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTQ3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjY4XSBHSUN2Mzog
Q1BVNDc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDhl
MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzI5Ml0gR0lDdjM6
IENQVTQ3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIz
MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzMyN10gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzM0Ml0gQ1BVNDc6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFlMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjM2NDldIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU0OApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzY2MV0g
R0lDdjM6IENQVTQ4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDNlMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzY4N10g
R0lDdjM6IENQVTQ4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGI0MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzcyNF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzc0MF0gQ1BVNDg6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBhMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQwNTFdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU0OQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NDA2OV0gR0lDdjM6IENQVTQ5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIyMDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA5ZTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQwOTVdIEdJQ3YzOiBDUFU0OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBiNTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQxMjldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQxNDVd
IENQVTQ5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0NDQ5XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNTAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjQ0NjJdIEdJQ3YzOiBDUFU1MDogZm91bmQgcmVkaXN0cmlidXRvciBjMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA0NjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjQ0ODddIEdJQ3YzOiBDUFU1MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBiNjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjQ1MjJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ1MzhdIENQVTUwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYzAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0ODQ5XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjQ4NjhdIEdJQ3YzOiBDUFU1MTogZm91bmQgcmVkaXN0cmlidXRvciAyNDAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI0ODkzXSBHSUN2MzogQ1BVNTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjcwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI0OTI2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI0OTQxXSBDUFU1MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjQw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NTI0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUyClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1MjYxXSBHSUN2MzogQ1BVNTI6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgODAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1Mjg3XSBHSUN2MzogQ1BVNTI6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjgwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI1MzI1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1MzQxXSBDUFU1MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMDgwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyNTY1Nl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUzClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1Njc0XSBHSUN2MzogQ1BVNTM6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjAwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk2MDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTcwMF0gR0lDdjM6IENQVTUzOiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI5MDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNTczNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNTc0OV0gQ1BVNTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDIwMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjYwNDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjA2M10gR0lDdjM6IENQVTU0OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIGUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRlMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjA5MF0gR0lDdjM6IENQVTU0OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJhMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjEyN10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjE0NF0gQ1BVNTQ6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDBlMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjY0NTZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjQ3Nl0gR0lDdjM6IENQVTU1
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhZTAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY1MDNdIEdJQ3YzOiBDUFU1
NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiYjAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY1MzddIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY1NTRdIENQVTU1OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2ODQyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNTYKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY4NTJdIEdJQ3Yz
OiBDUFU1NjogZm91bmQgcmVkaXN0cmlidXRvciAyMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAx
ZTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY4NzldIEdJQ3Yz
OiBDUFU1NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBi
YzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY5MTddIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY5MzVdIENQVTU2OiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3MjQyXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNTcKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjcyNTNd
IEdJQ3YzOiBDUFU1NzogZm91bmQgcmVkaXN0cmlidXRvciA0MDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAyNjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjcyODBd
IEdJQ3YzOiBDUFU1NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBiZDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjczMThd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjczMzVdIENQVTU3
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3NjQ1XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNTgKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
Mjc2NTRdIEdJQ3YzOiBDUFU1ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwMTYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3
NjgyXSBHSUN2MzogQ1BVNTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYmUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3
NzIxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI3NzM5XSBD
UFU1ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDAwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODA0Nl0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTU5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4MDU3XSBHSUN2MzogQ1BVNTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNjAxMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwMmUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4MDg0XSBHSUN2MzogQ1BVNTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwYmYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4MTIxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
MTM5XSBDUFU1OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDYwMTAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODQ0NF0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDI4NDU5XSBHSUN2MzogQ1BVNjA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTMwMTAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDYyMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyODQ4NV0gR0lDdjM6IENQVTYwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGMwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyODUyMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyODU0MF0gQ1BVNjA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDEzMDEw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg4
NDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyODg2NF0gR0lDdjM6IENQVTYxOiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDFiMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjg4OTBdIEdJQ3YzOiBDUFU2MTogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjg5MjFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjg5MzhdIENQVTYxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAxYjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI5MjQ5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjIKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkyNjVdIEdJQ3YzOiBDUFU2MjogZm91bmQgcmVkaXN0
cmlidXRvciAxNTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNmEwMDAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MjkxXSBHSUN2MzogQ1BVNjI6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzIwMDAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5MzI1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI5MzQzXSBDUFU2MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMTUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyOTY1M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYzClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NjcwXSBHSUN2MzogQ1BVNjM6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMWQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDhhMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTY5N10gR0lDdjM6IENQVTYzOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMzMDAwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTczMF0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTc0N10gQ1BVNjM6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDFkMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzAwNTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2NApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDA2Nl0gR0lDdjM6IENQVTY0
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDExMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1YTAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAwOTNdIEdJQ3YzOiBDUFU2
NDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNDAwMDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAxMjhdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzAxNDVdIENQVTY0OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNDQ1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNjUKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0NjJdIEdJQ3Yz
OiBDUFU2NTogZm91bmQgcmVkaXN0cmlidXRvciAxOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
N2EwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNDg5XSBHSUN2
MzogQ1BVNjU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YzUwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNTIzXSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwNTM5XSBDUFU2NTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTkwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDg1N10gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTY2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODcz
XSBHSUN2MzogQ1BVNjY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTcwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDcyMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDkw
MF0gR0lDdjM6IENQVTY2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGM2MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDkz
N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDk1M10gQ1BV
NjY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE3MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzEyNTNdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU2NwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMTI3MV0gR0lDdjM6IENQVTY3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFmMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA5MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzEyOThdIEdJQ3YzOiBDUFU2NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBjNzAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzEzMzNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzEz
NTFdIENQVTY3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZjAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxNjUyXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjgKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzE2NjRdIEdJQ3YzOiBDUFU2ODogZm91bmQgcmVkaXN0cmlidXRvciBiMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA0MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzE2OTJdIEdJQ3YzOiBDUFU2ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBjODAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzE3MjldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzE3NDZdIENQVTY4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYjAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyMDU1
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzIwNzRdIEdJQ3YzOiBDUFU2OTogZm91bmQgcmVkaXN0cmlidXRvciAy
MzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTIwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyMTAxXSBHSUN2MzogQ1BVNjk6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzkwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyMTM0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMyMTUxXSBDUFU2OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MjMwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMjQ0M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDU2XSBHSUN2MzogQ1BVNzA6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGEwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDg0XSBHSUN2MzogQ1BVNzA6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2EwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNTIxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyNTM4XSBDUFU3MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMGQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzMjg0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcxClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyODY4XSBHSUN2MzogQ1BVNzE6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjUwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFhMDAwMApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjg5N10gR0lDdjM6IENQVTcxOiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNiMDAwMApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjkyOV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMjk0NV0gQ1BVNzE6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDI1MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzMyNTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3MgpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzI2Ml0gR0lDdjM6IENQVTcyOiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDkwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNhMDAwMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzI5MV0gR0lDdjM6IENQVTcyOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNjMDAwMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzMyN10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzM0NV0gQ1BVNzI6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDA5MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzM2NTddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3
MwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzY3NV0gR0lDdjM6IENQ
VTczOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIxMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA5YTAw
MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzM3MDRdIEdJQ3YzOiBD
UFU3MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjZDAw
MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzM3MzhdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzM3NTVdIENQVTczOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0MDUyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNzQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQwNjZdIEdJ
Q3YzOiBDUFU3NDogZm91bmQgcmVkaXN0cmlidXRvciBmMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA1MjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQwOTVdIEdJ
Q3YzOiBDUFU3NDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBjZTAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQxMzJdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQxNDldIENQVTc0OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZjAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0NDUwXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNzUKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0
NzFdIEdJQ3YzOiBDUFU3NTogZm91bmQgcmVkaXN0cmlidXRvciAyNzAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwYjIwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0
NTAwXSBHSUN2MzogQ1BVNzU6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwY2YwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0
NTMzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0NTUxXSBD
UFU3NTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjcwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDg0N10gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTc2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM0ODU4XSBHSUN2MzogQ1BVNzY6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMzAxMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwMjIwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM0ODg4XSBHSUN2MzogQ1BVNzY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwZDAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM0OTI1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0
OTQyXSBDUFU3NjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDMwMTAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTI1MV0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM1MjYyXSBHSUN2MzogQ1BVNzc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgNTAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwMmEwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM1MjkzXSBHSUN2MzogQ1BVNzc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDEwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM1MzMxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM1MzQ5XSBDUFU3NzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDUwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNTY1
NV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM1NjY1XSBHSUN2MzogQ1BVNzg6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMWEwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM1Njk2XSBHSUN2MzogQ1BVNzg6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDIwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM1NzM0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM1NzUzXSBDUFU3ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MDEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzNjA0Nl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTc5ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDU4XSBHSUN2MzogQ1BVNzk6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzIwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDg4XSBHSUN2MzogQ1BVNzk6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwZDMwMDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MTI3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM2MTQ2XSBDUFU3OTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMDcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzNjIwN10gc21wOiBCcm91Z2h0IHVwIDEgbm9kZSwgODAgQ1BVcwpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIzMV0gU01QOiBUb3RhbCBvZiA4MCBwcm9j
ZXNzb3JzIGFjdGl2YXRlZC4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzYyMzJdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IFByaXZpbGVnZWQgQWNjZXNzIE5ldmVyClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MjMzXSBDUFUgZmVhdHVyZXM6
IGRldGVjdGVkOiBMU0UgYXRvbWljIGluc3RydWN0aW9ucwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzNjIzM10gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogVXNlciBBY2Nl
c3MgT3ZlcnJpZGUKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMzRd
IENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IDMyLWJpdCBFTDAgU3VwcG9ydApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIzNV0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDog
Q29tbW9uIG5vdCBQcml2YXRlIHRyYW5zbGF0aW9ucwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAzNjIzNV0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogRGF0YSBjYWNoZSBj
bGVhbiB0byBQb2ludCBvZiBQZXJzaXN0ZW5jZQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzNjIzNl0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogUkFTIEV4dGVuc2lvbiBT
dXBwb3J0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MjM3XSBDUFUg
ZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIHRoZSBQb1Ugbm90IHJlcXVp
cmVkIGZvciBJL0QgY29oZXJlbmNlClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2MjM3XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDUkMzMiBpbnN0cnVjdGlvbnMKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMzhdIENQVSBmZWF0dXJlczog
ZGV0ZWN0ZWQ6IFNwZWN1bGF0aXZlIFN0b3JlIEJ5cGFzc2luZyBTYWZlIChTU0JTKQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjA1ODYwNF0gQ1BVOiBBbGwgQ1BVKHMpIHN0
YXJ0ZWQgYXQgRUwyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ0ODU0
XSBub2RlIDAgZGVmZXJyZWQgcGFnZXMgaW5pdGlhbGlzZWQgaW4gNDg0bXMKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDQ5OTddIGRldnRtcGZzOiBpbml0aWFsaXplZApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NTU3NF0gUmVnaXN0ZXJlZCBj
cDE1X2JhcnJpZXIgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDU1OTVdIFJlZ2lzdGVyZWQgc2V0ZW5kIGVtdWxhdGlvbiBoYW5kbGVyClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ1NTk3XSBLQVNMUiBlbmFibGVk
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ1NjU3XSBjbG9ja3NvdXJj
ZTogamlmZmllczogbWFzazogMHhmZmZmZmZmZiBtYXhfY3ljbGVzOiAweGZmZmZmZmZmLCBtYXhf
aWRsZV9uczogNzY0NTA0MTc4NTEwMDAwMCBucwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU0NTcxMl0gZnV0ZXggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6
IDksIDIwOTcxNTIgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC41NDYxNDRdIHBpbmN0cmwgY29yZTogaW5pdGlhbGl6ZWQgcGluY3RybCBzdWJzeXN0
ZW0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDYzMDZdIFNNQklPUyAz
LjMuMCBwcmVzZW50LgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NjMw
OV0gRE1JOiBHSUdBQllURSBSMTUyLVAzMC0wMC9NUDMyLUFSMS0wMCwgQklPUyBGMTMgKFNDUDog
MS41LjIwMjEwNDI2KSAwNS8yMS8yMDIxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNTQ2NTM4XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDE2ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3MTE1XSBETUE6IHByZWFsbG9jYXRlZCA0
MDk2IEtpQiBHRlBfS0VSTkVMIHBvb2wgZm9yIGF0b21pYyBhbGxvY2F0aW9ucwpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzI0MV0gRE1BOiBwcmVhbGxvY2F0ZWQgNDA5
NiBLaUIgR0ZQX0tFUk5FTHxHRlBfRE1BIHBvb2wgZm9yIGF0b21pYyBhbGxvY2F0aW9ucwpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzM2NV0gRE1BOiBwcmVhbGxvY2F0
ZWQgNDA5NiBLaUIgR0ZQX0tFUk5FTHxHRlBfRE1BMzIgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRp
b25zClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3MzczXSBhdWRpdDog
aW5pdGlhbGl6aW5nIG5ldGxpbmsgc3Vic3lzIChkaXNhYmxlZCkKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC41NDc0MTJdIGF1ZGl0OiB0eXBlPTIwMDAgYXVkaXQoMC41NDQ6
MSk6IHN0YXRlPWluaXRpYWxpemVkIGF1ZGl0X2VuYWJsZWQ9MCByZXM9MQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzUwMF0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQg
dGhlcm1hbCBnb3Zlcm5vciAnZmFpcl9zaGFyZScKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDc1MDFdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJu
b3IgJ2JhbmdfYmFuZycKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDc1
MDJdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3N0ZXBfd2lzZScK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDc1MDNdIHRoZXJtYWxfc3lz
OiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJub3IgJ3VzZXJfc3BhY2UnClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3NTAzXSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0
aGVybWFsIGdvdmVybm9yICdwb3dlcl9hbGxvY2F0b3InClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTQ3NTU3XSBjcHVpZGxlOiB1c2luZyBnb3Zlcm5vciBsYWRkZXIKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDc1NjVdIGNwdWlkbGU6IHVzaW5n
IGdvdmVybm9yIG1lbnUKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDc2
NTldIERldGVjdGVkIDE1IFBDQyBTdWJzcGFjZXMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NDc2NzVdIFJlZ2lzdGVyaW5nIFBDQyBkcml2ZXIgYXMgTWFpbGJveCBjb250
cm9sbGVyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3Njk0XSBody1i
cmVha3BvaW50OiBmb3VuZCA2IGJyZWFrcG9pbnQgYW5kIDQgd2F0Y2hwb2ludCByZWdpc3RlcnMu
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ4MDc0XSBBU0lEIGFsbG9j
YXRvciBpbml0aWFsaXNlZCB3aXRoIDMyNzY4IGVudHJpZXMKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC41NDgwOThdIEFDUEk6IGJ1cyB0eXBlIFBDSSByZWdpc3RlcmVkClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ4MDk5XSBhY3BpcGhwOiBBQ1BJ
IEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjUKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDgxMzddIFNlcmlhbDogQU1CQSBQTDAxMSBVQVJU
IGRyaXZlcgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MDUwMV0gSHVn
ZVRMQiByZWdpc3RlcmVkIDEuMDAgR2lCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2Vz
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUwNTAyXSBIdWdlVExCIHJl
Z2lzdGVyZWQgMzIuMCBNaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTA1MDNdIEh1Z2VUTEIgcmVnaXN0ZXJl
ZCAyLjAwIE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MDUwNF0gSHVnZVRMQiByZWdpc3RlcmVkIDY0LjAg
S2lCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjc3OTIwXSBBQ1BJOiBBZGRlZCBfT1NJKE1vZHVsZSBEZXZpY2Up
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjc3OTIxXSBBQ1BJOiBBZGRl
ZCBfT1NJKFByb2Nlc3NvciBEZXZpY2UpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNjc3OTIyXSBBQ1BJOiBBZGRlZCBfT1NJKDMuMCBfU0NQIEV4dGVuc2lvbnMpClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjc3OTIzXSBBQ1BJOiBBZGRlZCBfT1NJ
KFByb2Nlc3NvciBBZ2dyZWdhdG9yIERldmljZSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC42Nzc5MjRdIEFDUEk6IEFkZGVkIF9PU0koTGludXgtRGVsbC1WaWRlbykKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzc5MjZdIEFDUEk6IEFkZGVkIF9P
U0koTGludXgtTGVub3ZvLU5WLUhETUktQXVkaW8pClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjc3OTI3XSBBQ1BJOiBBZGRlZCBfT1NJKExpbnV4LUhQSS1IeWJyaWQtR3Jh
cGhpY3MpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg0ODY0XSBBQ1BJ
OiAyIEFDUEkgQU1MIHRhYmxlcyBzdWNjZXNzZnVsbHkgYWNxdWlyZWQgYW5kIGxvYWRlZApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY4NjIxNF0gQUNQSTogSW50ZXJwcmV0
ZXIgZW5hYmxlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY4NjIxNV0g
QUNQSTogVXNpbmcgR0lDIGZvciBpbnRlcnJ1cHQgcm91dGluZwpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjY4NjIyM10gQUNQSTogTUNGRyB0YWJsZSBkZXRlY3RlZCwgOCBl
bnRyaWVzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk1MDAzXSBBUk1I
MDAxMTowMDogdHR5QU1BMCBhdCBNTUlPIDB4MTAwMDAyNjAwMDAwIChpcnEgPSAzNiwgYmFzZV9i
YXVkID0gMCkgaXMgYSBTQlNBClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk1MDQ3XSBwcmludGs6IGNvbnNvbGUgW3R0eUFNQTBdIGVuYWJsZWQKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTYxNTJdIEFSTUgwMDExOjAxOiB0dHlBTUExIGF0IE1N
SU8gMHgxMDAwMDI2MjAwMDAgKGlycSA9IDM3LCBiYXNlX2JhdWQgPSAwKSBpcyBhIFNCU0EKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTcyMjVdIEFDUEk6IFBDSSBSb290
IEJyaWRnZSBbUENJMF0gKGRvbWFpbiAwMDBjIFtidXMgMDAtZmZdKQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5NzIzMV0gYWNwaSBQTlAwQTA4OjAwOiBfT1NDOiBPUyBz
dXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlw
ZTNdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3MzAwXSBhY3BpIFBO
UDBBMDg6MDA6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNI
UENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk3MzU5XSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVD
YXBhYmlsaXR5XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5NzM2M10g
YWNwaSBQTlAwQTA4OjAwOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgzM2ZmZjAwMDAwMDAt
MHgzM2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmRkNzMyZDQzZWE1OApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5Nzk5Ml0gYWNwaSBQTlAwQTA4
OjAwOiBFQ0FNIGFyZWEgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gcmVzZXJ2
ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
ODAwMF0gYWNwaSBQTlAwQTA4OjAwOiBFQ0FNIGF0IFttZW0gMHgzM2ZmZjAwMDAwMDAtMHgzM2Zm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5ODA2NV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBjOjAwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MDY3XSBwY2lfYnVzIDAwMGM6MDA6IHJvb3Qg
YnVzIHJlc291cmNlIFttZW0gMHg0MDAwMDAwMC0weDRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTgwNjhdIHBjaV9idXMgMDAwYzowMDogcm9v
dCBidXMgcmVzb3VyY2UgW21lbSAweDMwMDAwMDAwMDAwMC0weDMzZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MDY4XSBwY2lfYnVzIDAw
MGM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk4MDg5XSBwY2kgMDAwYzowMDowMC4wOiBbMWRlZjplMTAwXSB0
eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk4MTM3XSBwY2kgMDAwYzowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MTQ1XSBwY2kg
MDAwYzowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk4MTYwXSBwY2kgMDAwYzowMDowMS4wOiBzdXBwb3J0cyBEMSBE
MgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODE2MV0gcGNpIDAwMGM6
MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODIzN10gYWNwaXBocDogU2xvdCBbMV0gcmVnaXN0ZXJl
ZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI1MV0gcGNpIDAwMGM6
MDE6MDAuMDogWzEwMDA6MDA5N10gdHlwZSAwMCBjbGFzcyAweDAxMDcwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI2MF0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4
MTA6IFtpbyAgMHg0ZmZmODAwMC0weDRmZmY4MGZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5ODI2N10gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MTQ6IFttZW0gMHg0
MDE0MDAwMC0weDQwMTRmZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5ODI3NV0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MWM6IFttZW0gMHg0MDEwMDAw
MC0weDQwMTNmZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY5ODI4NF0gcGNpIDAwMGM6MDE6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg0MDAwMDAwMC0weDQw
MGZmZmZmIHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4Mjg4
XSBwY2kgMDAwYzowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MzgzXSBwY2kgMDAwYzowMTowMC4wOiBzdXBwb3J0
cyBEMSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODQ1NV0gcGNp
X2J1cyAwMDBjOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5ODQ1OV0gcGNpIDAwMGM6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAw
MDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY5ODQ2M10gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NDAwMDAw
MDAtMHg0MDFmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg0
NjRdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDMwMDAwMDAwMDAw
MC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5ODQ2NV0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42
OTg0NjZdIHBjaSAwMDBjOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODQ2N10g
cGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg0NjhdIHBjaSAwMDBjOjAw
OjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODQ3MF0gcGNpIDAwMGM6MDE6MDAuMDog
QkFSIDY6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMGZmZmZmIHByZWZdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4NDcxXSBwY2kgMDAwYzowMTowMC4wOiBC
QVIgMzogYXNzaWduZWQgW21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYgNjRiaXRdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4NDc2XSBwY2kgMDAwYzowMTowMC4wOiBC
QVIgMTogYXNzaWduZWQgW21lbSAweDQwMTQwMDAwLTB4NDAxNGZmZmYgNjRiaXRdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4NDgxXSBwY2kgMDAwYzowMTowMC4wOiBC
QVIgMDogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAxMDBdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk4NDgyXSBwY2kgMDAwYzowMTowMC4wOiBCQVIgMDogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMTAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5ODQ4M10gcGNpIDAwMGM6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAx
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODQ4NV0gcGNpIDAwMGM6
MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZmXQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODQ4Nl0gcGNpIDAwMGM6MDA6MDEu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg0ODhdIHBj
aV9idXMgMDAwYzowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25lZCwg
dHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5ODQ4OV0gcGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA0IFttZW0gMHg0MDAw
MDAwMC0weDRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTg0OTBdIHBjaV9idXMgMDAwYzowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzAwMDAwMDAw
MDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTg0OTFdIHBjaV9idXMgMDAwYzowMTogcmVzb3VyY2UgMSBbbWVtIDB4NDAwMDAw
MDAtMHg0MDFmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg0
OTJdIHBjaV9idXMgMDAwYzowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk4NTk4XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTFdIChkb21haW4gMDAwZCBbYnVz
IDAwLWZmXSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTg2MDNdIGFj
cGkgUE5QMEEwODowMTogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xv
Y2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5ODY2Nl0gYWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5v
dCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODcyNV0gYWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBP
UyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC42OTg3MjddIGFjcGkgUE5QMEEwODowMTogTUNGRyBxdWlyazogRUNB
TSBhdCBbbWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0g
d2l0aCAweGZmZmZkZDczMmQ0M2VhNTgKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTkzNTJdIGFjcGkgUE5QMEEwODowMTogRUNBTSBhcmVhIFttZW0gMHgzN2ZmZjAwMDAw
MDAtMHgzN2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkzNTldIGFjcGkgUE5QMEEwODowMTogRUNBTSBhdCBb
bWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk0MThdIFBDSSBob3N0IGJyaWRnZSB0
byBidXMgMDAwZDowMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTQx
OV0gcGNpX2J1cyAwMDBkOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NTAwMDAwMDAtMHg1
ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
NDIwXSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzNDAwMDAwMDAw
MDAtMHgzN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5OTQyMV0gcGNpX2J1cyAwMDBkOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAw
LWZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTQzOF0gcGNpIDAw
MGQ6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTQ3N10gcGNpIDAwMGQ6MDA6MDEuMDogWzFk
ZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5OTQ4NF0gcGNpIDAwMGQ6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQg
VGFncwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTUwMF0gcGNpIDAw
MGQ6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTk1MDFdIHBjaSAwMDBkOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAg
RDEgRDNob3QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk1NTJdIHBj
aSAwMDBkOjAwOjAyLjA6IFsxZGVmOmUxMDJdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk1NzJdIHBjaSAwMDBkOjAwOjAyLjA6
IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
NTczXSBwY2kgMDAwZDowMDowMi4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NjIxXSBwY2kgMDAwZDowMDow
My4wOiBbMWRlZjplMTAzXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NjQxXSBwY2kgMDAwZDowMDowMy4wOiBzdXBwb3J0cyBE
MSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTY0MV0gcGNpIDAw
MGQ6MDA6MDMuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTY4N10gcGNpIDAwMGQ6MDA6MDQuMDogWzFkZWY6
ZTEwNF0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjY5OTcwOF0gcGNpIDAwMGQ6MDA6MDQuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3MDldIHBjaSAwMDBkOjAwOjA0LjA6
IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTk3NzZdIGFjcGlwaHA6IFNsb3QgWzEtMl0gcmVnaXN0ZXJlZApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTc5Ml0gcGNpIDAwMGQ6MDE6MDAu
MDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjY5OTgwNF0gcGNpIDAwMGQ6MDE6MDAuMDogcmVnIDB4MTA6IFtt
ZW0gMHg1MDMxMDAwMC0weDUwMzEzZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5OTgyN10gcGNpIDAwMGQ6MDE6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1
MDMwMDAwMC0weDUwMzBmZmZmIHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNjk5ODMxXSBwY2kgMDAwZDowMTowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTgxXSBhY3BpcGhwOiBTbG90
IFsyXSByZWdpc3RlcmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
OTk1XSBwY2kgMDAwZDowMjowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAy
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMDA2XSBwY2kgMDAwZDow
MjowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMDI3XSBwY2kgMDAwZDowMjowMC4w
OiByZWcgMHgzMDogW21lbSAweDUwMjAwMDAwLTB4NTAyMGZmZmYgcHJlZl0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAwMzBdIHBjaSAwMDBkOjAyOjAwLjA6IGVuYWJs
aW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDAxNzNdIGFjcGlwaHA6IFNsb3QgWzNdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDAxODhdIHBjaSAwMDBkOjAzOjAwLjA6IFs4MDg2OjBhNTRdIHR5
cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDAxOThdIHBjaSAwMDBkOjAzOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4NTAxMTAwMDAtMHg1
MDExM2ZmZiA2NGJpdF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAy
MjRdIHBjaSAwMDBkOjAzOjAwLjA6IHJlZyAweDMwOiBbbWVtIDB4NTAxMDAwMDAtMHg1MDEwZmZm
ZiBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDIyN10gcGNp
IDAwMGQ6MDM6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMDM3MV0gYWNwaXBocDogU2xvdCBbNF0gcmVnaXN0ZXJlZApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM4Nl0gcGNpIDAwMGQ6MDQ6
MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM5Nl0gcGNpIDAwMGQ6MDQ6MDAuMDogcmVnIDB4MTA6
IFttZW0gMHg1MDAxMDAwMC0weDUwMDEzZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMDQxN10gcGNpIDAwMGQ6MDQ6MDAuMDogcmVnIDB4MzA6IFttZW0g
MHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAwNDIxXSBwY2kgMDAwZDowNDowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdz
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTQ0XSBwY2lfYnVzIDAw
MGQ6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwNTQ2XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4
MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDU0OF0gcGNpIDAwMGQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAy
MDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMDU0OV0gcGNpIDAwMGQ6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAw
MDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAw
MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NTBdIHBjaSAwMDBk
OjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAyXSBh
ZGRfc2l6ZSAxMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTUx
XSBwY2kgMDAwZDowMDowMi4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZm
ZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAw
MDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTUzXSBwY2kgMDAw
ZDowMDowMi4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBb
YnVzIDAyXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDU1NF0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdp
bmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDEwMDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NTVdIHBjaSAwMDBkOjAwOjAzLjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8g
W2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NTZdIHBjaSAwMDBkOjAwOjAzLjA6IGJyaWRnZSB3
aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDEw
MDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwNTU3XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4
MGZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDU1OF0gcGNpIDAwMGQ6MDA6MDQuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAy
MDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMDU1OV0gcGNpIDAwMGQ6MDA6MDQuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAw
MDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAw
MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NjNdIHBjaSAwMDBk
OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUwMDAwMDAwLTB4NTAxZmZmZmZdClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTY1XSBwY2kgMDAwZDowMDow
MS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDAwMDAwMDAtMHgzNDAwMDAxZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NjZd
IHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUwMjAwMDAwLTB4NTAz
ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTY3XSBwY2kg
MDAwZDowMDowMi4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDAyMDAwMDAtMHgzNDAw
MDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDA1NjhdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDUwNDAw
MDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
NTY5XSBwY2kgMDAwZDowMDowMy4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgzNDAwMDA0MDAw
MDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDA1NzBdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNDogYXNzaWduZWQgW21l
bSAweDUwNjAwMDAwLTB4NTA3ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAwNTcxXSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgz
NDAwMDA2MDAwMDAtMHgzNDAwMDA3ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NzJdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAwNTczXSBwY2kgMDAwZDowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDA1NzRdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTc0XSBw
Y2kgMDAwZDowMDowMi4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NzVdIHBjaSAwMDBk
OjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTc2XSBwY2kgMDAwZDowMDowMy4wOiBC
QVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1NzddIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAwNTc4XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDA1ODBdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTgx
XSBwY2kgMDAwZDowMDowNC4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1ODJdIHBjaSAw
MDBkOjAwOjAzLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTgzXSBwY2kgMDAwZDowMDowMy4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1ODNdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAwNTg0XSBwY2kgMDAwZDowMDowMi4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDA1ODVdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
NTg2XSBwY2kgMDAwZDowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1ODddIHBj
aSAwMDBkOjAxOjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDAwZmZm
ZiBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDU4OF0gcGNp
IDAwMGQ6MDE6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1MDAxMDAwMC0weDUwMDEzZmZm
IDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDU5NF0gcGNp
IDAwMGQ6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwMDU5NV0gcGNpIDAwMGQ6MDA6MDEuMDogICBicmlkZ2Ugd2lu
ZG93IFttZW0gMHg1MDAwMDAwMC0weDUwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDU5Nl0gcGNpIDAwMGQ6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgzNDAwMDAwMDAwMDAtMHgzNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1OThdIHBjaSAwMDBkOjAyOjAwLjA6IEJBUiA2
OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDU5OV0gcGNpIDAwMGQ6MDI6MDAuMDogQkFSIDA6
IGFzc2lnbmVkIFttZW0gMHg1MDIxMDAwMC0weDUwMjEzZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYwNF0gcGNpIDAwMGQ6MDA6MDIuMDogUENJIGJy
aWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MDYwNV0gcGNpIDAwMGQ6MDA6MDIuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg1MDIwMDAwMC0w
eDUwM2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYwNl0g
cGNpIDAwMGQ6MDA6MDIuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzNDAwMDAyMDAwMDAtMHgz
NDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDA2MDhdIHBjaSAwMDBkOjAzOjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NTA0
MDAwMDAtMHg1MDQwZmZmZiBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMDYwOV0gcGNpIDAwMGQ6MDM6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1MDQx
MDAwMC0weDUwNDEzZmZmIDY0Yml0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMDYxNF0gcGNpIDAwMGQ6MDA6MDMuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYxNV0gcGNpIDAwMGQ6MDA6MDMu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYxNl0gcGNpIDAwMGQ6MDA6MDMuMDogICBi
cmlkZ2Ugd2luZG93IFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA2MTddIHBjaSAwMDBk
OjA0OjAwLjA6IEJBUiA2OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAtMHg1MDYwZmZmZiBwcmVm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYxOF0gcGNpIDAwMGQ6
MDQ6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHg1MDYxMDAwMC0weDUwNjEzZmZmIDY0Yml0
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDYyM10gcGNpIDAwMGQ6
MDA6MDQuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDYyNF0gcGNpIDAwMGQ6MDA6MDQuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHg1MDYwMDAwMC0weDUwN2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMDYyNV0gcGNpIDAwMGQ6MDA6MDQuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgz
NDAwMDA2MDAwMDAtMHgzNDAwMDA3ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDA2MjddIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2UgNCBb
bWVtIDB4NTAwMDAwMDAtMHg1ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAwNjI4XSBwY2lfYnVzIDAwMGQ6MDA6IHJlc291cmNlIDUgW21lbSAw
eDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzAwNjI5XSBwY2lfYnVzIDAwMGQ6MDE6IHJlc291cmNlIDEgW21l
bSAweDUwMDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAwNjMwXSBwY2lfYnVzIDAwMGQ6MDE6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDAw
MDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDYzMV0gcGNpX2J1cyAwMDBkOjAyOiByZXNvdXJjZSAxIFttZW0gMHg1
MDIwMDAwMC0weDUwM2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwMDYzMV0gcGNpX2J1cyAwMDBkOjAyOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDAyMDAwMDAt
MHgzNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDA2MzJdIHBjaV9idXMgMDAwZDowMzogcmVzb3VyY2UgMSBbbWVtIDB4NTA0MDAw
MDAtMHg1MDVmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA2
MzNdIHBjaV9idXMgMDAwZDowMzogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNDAwMDAwLTB4MzQw
MDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwNjM0XSBwY2lfYnVzIDAwMGQ6MDQ6IHJlc291cmNlIDEgW21lbSAweDUwNjAwMDAwLTB4
NTA3ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNjM1XSBw
Y2lfYnVzIDAwMGQ6MDQ6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDYwMDAwMC0weDM0MDAwMDdm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MDc0Ml0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kyXSAoZG9tYWluIDAwMDEgW2J1cyAwMC1m
Zl0pClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNzQ3XSBhY3BpIFBO
UDBBMDg6MDI6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0g
U2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDA4MTFdIGFjcGkgUE5QMEEwODowMjogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3Vw
cG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDA4NzBdIGFjcGkgUE5QMEEwODowMjogX09TQzogT1Mgbm93
IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAwODcyXSBhY3BpIFBOUDBBMDg6MDI6IE1DRkcgcXVpcms6IEVDQU0gYXQg
W21lbSAweDNiZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGgg
MHhmZmZmZGQ3MzJkNDNlYTU4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAxNTAxXSBhY3BpIFBOUDBBMDg6MDI6IEVDQU0gYXJlYSBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4
M2JmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzAxNTA3XSBhY3BpIFBOUDBBMDg6MDI6IEVDQU0gYXQgW21lbSAw
eDNiZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxNTY2XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVz
IDAwMDE6MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE1NjddIHBj
aV9idXMgMDAwMTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDYwMDAwMDAwLTB4NmZmZmZm
ZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTU2OF0g
cGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4
M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDE1NjldIHBjaV9idXMgMDAwMTowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0K
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE1ODZdIHBjaSAwMDAxOjAw
OjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE2MjldIHBjaSAwMDAxOjAwOjAxLjA6IFsxZGVmOmUx
MDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDE2NDJdIHBjaSAwMDAxOjAwOjAxLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE2NjhdIHBjaSAwMDAxOjAw
OjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAxNjY5XSBwY2kgMDAwMTowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQz
aG90ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxNzU2XSBhY3BpcGhw
OiBTbG90IFsxLTNdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDM4NDZdIHBjaV9idXMgMDAwMTowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM4NDddIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzODQ5XSBwY2kgMDAwMTowMDow
MS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZd
IHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzODUwXSBwY2kgMDAwMTowMDowMS4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6
ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMzg1Ml0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4
NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDM4NTNdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM4MDAw
MDAwMDAwMC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMzg1NF0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDM4NTVdIHBjaSAwMDAxOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
Mzg1Nl0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM4NTddIHBjaSAw
MDAxOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzg1OF0gcGNpIDAwMDE6MDA6
MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMzg2MF0gcGNpIDAwMDE6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0g
MHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwMzg2MV0gcGNpIDAwMDE6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzODAw
MDAwMDAwMDAtMHgzODAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDM4NjNdIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNCBbbWVt
IDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAzODY0XSBwY2lfYnVzIDAwMDE6MDA6IHJlc291cmNlIDUgW21lbSAweDM4
MDAwMDAwMDAwMC0weDNiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAzODY1XSBwY2lfYnVzIDAwMDE6MDE6IHJlc291cmNlIDEgW21lbSAw
eDYwMDAwMDAwLTB4NjAxZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAzODY2XSBwY2lfYnVzIDAwMDE6MDE6IHJlc291cmNlIDIgW21lbSAweDM4MDAwMDAwMDAw
MC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMzk3MF0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kzXSAoZG9tYWluIDAw
MDAgW2J1cyAwMC1mZl0pClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAz
OTc0XSBhY3BpIFBOUDBBMDg6MDM6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBB
U1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDQwMzhdIGFjcGkgUE5QMEEwODowMzogX09TQzogcGxhdGZvcm0g
ZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQwOTZdIGFjcGkgUE5QMEEwODowMzog
X09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0MDk4XSBhY3BpIFBOUDBBMDg6MDM6IE1DRkcgcXVp
cms6IEVDQU0gYXQgW21lbSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZl0gZm9yIFtidXMg
MDAtZmZdIHdpdGggMHhmZmZmZGQ3MzJkNDNlYTU4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA0NzI0XSBhY3BpIFBOUDBBMDg6MDM6IEVDQU0gYXJlYSBbbWVtIDB4M2Zm
ZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0NzMwXSBhY3BpIFBOUDBBMDg6MDM6IEVD
QU0gYXQgW21lbSAweDNmZmZmMDAwMDAwMC0weDNmZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0Nzg5XSBQQ0kgaG9zdCBi
cmlkZ2UgdG8gYnVzIDAwMDA6MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDQ3OTFdIHBjaV9idXMgMDAwMDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDcwMDAw
MDAwLTB4N2ZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNDc5MV0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4M2Mw
MDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDQ3OTJdIHBjaV9idXMgMDAwMDowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W2J1cyAwMC1mZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ4MDld
IHBjaSAwMDAwOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ4NTFdIHBjaSAwMDAwOjAwOjAx
LjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDQ4NjRdIHBjaSAwMDAwOjAwOjAxLjA6IGVuYWJsaW5nIEV4
dGVuZGVkIFRhZ3MKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ4OTFd
IHBjaSAwMDAwOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA0ODkyXSBwY2kgMDAwMDowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0
OTc4XSBhY3BpcGhwOiBTbG90IFsxLTRdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDcwNjhdIHBjaV9idXMgMDAwMDowMDogb24gTlVNQSBub2RlIDAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcwNzBdIHBjaSAwMDAwOjAw
OjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRf
c2l6ZSAxMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MDcxXSBw
Y2kgMDAwMDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZm
IDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MDcyXSBwY2kgMDAwMDow
MDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVz
IDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNzA3NF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE0OiBhc3Np
Z25lZCBbbWVtIDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDcwNzVdIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQg
W21lbSAweDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzA3Nl0gcGNpIDAwMDA6MDA6MDEuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDcwNzddIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwNzA3OF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDcwNzldIHBjaSAwMDAwOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzA4MF0g
cGNpIDAwMDA6MDA6MDEuMDogUENJIGJyaWRnZSB0byBbYnVzIDAxXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzA4Ml0gcGNpIDAwMDA6MDA6MDEuMDogICBicmlkZ2Ug
d2luZG93IFttZW0gMHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNzA4M10gcGNpIDAwMDA6MDA6MDEuMDogICBicmlkZ2Ugd2luZG93
IFttZW0gMHgzYzAwMDAwMDAwMDAtMHgzYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcwODVdIHBjaV9idXMgMDAwMDowMDogcmVz
b3VyY2UgNCBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MDg2XSBwY2lfYnVzIDAwMDA6MDA6IHJlc291cmNl
IDUgW21lbSAweDNjMDAwMDAwMDAwMC0weDNmZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MDg3XSBwY2lfYnVzIDAwMDA6MDE6IHJlc291
cmNlIDEgW21lbSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA3MDg4XSBwY2lfYnVzIDAwMDA6MDE6IHJlc291cmNlIDIgW21lbSAw
eDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzE5MV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0k0
XSAoZG9tYWluIDAwMDIgW2J1cyAwMC1mZl0pClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA3MTk1XSBhY3BpIFBOUDBBMDg6MDQ6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRl
bmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcyNThdIGFjcGkgUE5QMEEwODowNDogX09T
QzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1F
IExUUl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDczMTZdIGFjcGkg
UE5QMEEwODowNDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3MzE4XSBhY3BpIFBOUDBBMDg6
MDQ6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZm
Zl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmZGQ3MzJkNDNlYTU4ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3OTQzXSBhY3BpIFBOUDBBMDg6MDQ6IEVDQU0gYXJl
YSBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAy
OjAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3OTUwXSBhY3BpIFBO
UDBBMDg6MDQ6IEVDQU0gYXQgW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZl0gZm9y
IFtidXMgMDAtZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4MDA5
XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDI6MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDgwMTBdIHBjaV9idXMgMDAwMjowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W21lbSAweDAwODAwMDAwLTB4MGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwODAxMV0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDgwMTJdIHBjaV9idXMgMDAwMjowMDogcm9vdCBi
dXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDgwMjhdIHBjaSAwMDAyOjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3Mg
MHgwNjAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDgwNzBdIHBj
aSAwMDAyOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDgxMDVdIHBjaSAwMDAyOjAwOjAxLjA6
IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4
MTA2XSBwY2kgMDAwMjowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4MTY3XSBwY2kgMDAwMjowMDow
NS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzA4MjAyXSBwY2kgMDAwMjowMDowNS4wOiBzdXBwb3J0cyBE
MSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwODIwM10gcGNpIDAw
MDI6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwODI4OV0gYWNwaXBocDogU2xvdCBbMS01XSByZWdp
c3RlcmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEwNDAxXSBhY3Bp
cGhwOiBTbG90IFszLTJdIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTI0OTRdIHBjaV9idXMgMDAwMjowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI0OTVdIHBjaSAwMDAyOjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNDk2XSBwY2kgMDAwMjow
MDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHBy
ZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNDk4XSBwY2kgMDAwMjowMDowMS4wOiBi
cmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRf
c2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxMjQ5OV0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4
MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTI1MDBdIHBjaSAwMDAyOjAwOjA1LjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTI1MDFdIHBjaSAwMDAyOjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAw
eDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTAzXSBw
Y2kgMDAwMjowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgwMDgwMDAwMC0weDAwOWZm
ZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjUwNF0gcGNpIDAw
MDI6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzEyNTA1XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgwMGEwMDAw
MC0weDAwYmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjUw
Nl0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMjAwMDAw
LTB4MjAwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEyNTA3XSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
MjUwOF0gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTA5XSBw
Y2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjUwOV0gcGNpIDAwMDI6MDA6
MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTExXSBwY2kgMDAwMjowMDowNS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMjUxMl0gcGNpIDAwMDI6MDA6MDUuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzEyNTEzXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxMjUxM10gcGNpIDAwMDI6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTE0
XSBwY2kgMDAwMjowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNTE2XSBwY2kgMDAwMjowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzEyNTE4XSBwY2kgMDAwMjowMDowMS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDIwMDAwMDAwMDAwMC0weDIwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjUxOV0gcGNpIDAwMDI6MDA6MDUuMDog
UENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxMjUyMV0gcGNpIDAwMDI6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgwMGEw
MDAwMC0weDAwYmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
MjUyMl0gcGNpIDAwMDI6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMDAwMDAyMDAw
MDAtMHgyMDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTI1MjVdIHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNCBbbWVtIDB4MDA4
MDAwMDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzEyNTI1XSBwY2lfYnVzIDAwMDI6MDA6IHJlc291cmNlIDUgW21lbSAweDIwMDAwMDAw
MDAwMC0weDIzZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEyNTI2XSBwY2lfYnVzIDAwMDI6MDE6IHJlc291cmNlIDEgW21lbSAweDAwODAw
MDAwLTB4MDA5ZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEy
NTI3XSBwY2lfYnVzIDAwMDI6MDE6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDAwMDAwMC0weDIw
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxMjUyOF0gcGNpX2J1cyAwMDAyOjAyOiByZXNvdXJjZSAxIFttZW0gMHgwMGEwMDAwMC0w
eDAwYmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjUyOV0g
cGNpX2J1cyAwMDAyOjAyOiByZXNvdXJjZSAyIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTI2MzZdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJNV0gKGRvbWFpbiAwMDAzIFtidXMgMDAt
ZmZdKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjY0MF0gYWNwaSBQ
TlAwQTA4OjA1OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BN
IFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzEyNzAzXSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1
cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNzYxXSBhY3BpIFBOUDBBMDg6MDU6IF9PU0M6IE9TIG5v
dyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxMjc2M10gYWNwaSBQTlAwQTA4OjA1OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0
IFttZW0gMHgyN2ZmZjAwMDAwMDAtMHgyN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRo
IDB4ZmZmZmRkNzMyZDQzZWE1OApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxMzM5Ml0gYWNwaSBQTlAwQTA4OjA1OiBFQ0FNIGFyZWEgW21lbSAweDI3ZmZmMDAwMDAwMC0w
eDI3ZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMzM5OF0gYWNwaSBQTlAwQTA4OjA1OiBFQ0FNIGF0IFttZW0g
MHgyN2ZmZjAwMDAwMDAtMHgyN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzQ1MF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1
cyAwMDAzOjAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNDUyXSBw
Y2lfYnVzIDAwMDM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgxMDAwMDAwMC0weDFmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTM0NTJd
IHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI0MDAwMDAwMDAwMC0w
eDI3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzEzNDUzXSBwY2lfYnVzIDAwMDM6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNDcwXSBwY2kgMDAwMzow
MDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNTEwXSBwY2kgMDAwMzowMDowMS4wOiBbMWRlZjpl
MTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEzNTQ2XSBwY2kgMDAwMzowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzU0Nl0gcGNpIDAwMDM6MDA6MDEuMDog
UE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxMzYwNF0gcGNpIDAwMDM6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAw
MSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
MzYzMF0gcGNpIDAwMDM6MDA6MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTM2MzFdIHBjaSAwMDAzOjAwOjA1LjA6IFBNRSMgc3VwcG9y
dGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTM2OThdIGFjcGlwaHA6IFNsb3QgWzEtNl0gcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTgyMF0gcGNpIDAwMDM6MDI6MDAuMDogWzgwODY6MTUy
MV0gdHlwZSAwMCBjbGFzcyAweDAyMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNTgzNl0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgxMDAyMDAw
MC0weDEwMDNmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTg1
Ml0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg6IFtpbyAgMHgxZmZmODAyMC0weDFmZmY4MDNm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTg2MV0gcGNpIDAwMDM6
MDI6MDAuMDogcmVnIDB4MWM6IFttZW0gMHgxMDA0NDAwMC0weDEwMDQ3ZmZmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTk3MV0gcGNpIDAwMDM6MDI6MDAuMDogUE1F
IyBzdXBwb3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTYwMDhdIHBjaSAwMDAzOjAyOjAwLjA6IHJlZyAweDE4NDogW21lbSAw
eDAwMDAwMDAwLTB4MDAwMDNmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTYwMTBdIHBjaSAwMDAzOjAyOjAwLjA6IFZGKG4pIEJBUjAgc3BhY2U6
IFttZW0gMHgwMDAwMDAwMC0weDAwMDFmZmZmIDY0Yml0IHByZWZdIChjb250YWlucyBCQVIwIGZv
ciA4IFZGcykKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYwMzFdIHBj
aSAwMDAzOjAyOjAwLjA6IHJlZyAweDE5MDogW21lbSAweDAwMDAwMDAwLTB4MDAwMDNmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYwMzJdIHBj
aSAwMDAzOjAyOjAwLjA6IFZGKG4pIEJBUjMgc3BhY2U6IFttZW0gMHgwMDAwMDAwMC0weDAwMDFm
ZmZmIDY0Yml0IHByZWZdIChjb250YWlucyBCQVIzIGZvciA4IFZGcykKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYxMjhdIHBjaSAwMDAzOjAyOjAwLjA6IDguMDAwIEdi
L3MgYXZhaWxhYmxlIFBDSWUgYmFuZHdpZHRoLCBsaW1pdGVkIGJ5IDUuMCBHVC9zIFBDSWUgeDIg
bGluayBhdCAwMDAzOjAwOjA1LjAgKGNhcGFibGUgb2YgMTYuMDAwIEdiL3Mgd2l0aCA1LjAgR1Qv
cyBQQ0llIHg0IGxpbmspClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
MTc1XSBwY2kgMDAwMzowMjowMC4xOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNsYXNzIDB4MDIwMDAw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MTkwXSBwY2kgMDAwMzow
MjowMC4xOiByZWcgMHgxMDogW21lbSAweDEwMDAwMDAwLTB4MTAwMWZmZmZdClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MjA3XSBwY2kgMDAwMzowMjowMC4xOiByZWcg
MHgxODogW2lvICAweDFmZmY4MDAwLTB4MWZmZjgwMWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE2MjE1XSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxYzogW21lbSAw
eDEwMDQwMDAwLTB4MTAwNDNmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE2MzIyXSBwY2kgMDAwMzowMjowMC4xOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQzaG90
IEQzY29sZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM1NV0gcGNp
IDAwMDM6MDI6MDAuMTogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM1Nl0gcGNp
IDAwMDM6MDI6MDAuMTogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZm
ZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM3N10gcGNpIDAwMDM6MDI6MDAuMTogcmVnIDB4MTkw
OiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM3OF0gcGNpIDAwMDM6MDI6MDAuMTogVkYobikgQkFS
MyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5z
IEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
NjQ5OV0gcGNpX2J1cyAwMDAzOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjUwMF0gcGNpX2J1cyAwMDAzOjAwOiBtYXggYnVzIGRlcHRo
OiAxIHBjaV90cnlfbnVtOiAyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2NTAzXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgxMDAwMDAw
MC0weDEwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjUw
NV0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMDAwMDAw
LTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2NTA2XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0g
MHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjUwN10gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjQw
MDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE2NTA4XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNjUwOV0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2NTEwXSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjUxMF0gcGNp
IDAwMDM6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTEyXSBwY2kgMDAwMzow
MDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2NTEzXSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE2NTE1XSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI0
MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjUxOF0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDA6IGFzc2ln
bmVkIFttZW0gMHgxMDIwMDAwMC0weDEwMjFmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNjUyMV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDA6IGFzc2lnbmVkIFtt
ZW0gMHgxMDIyMDAwMC0weDEwMjNmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNjUyNF0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDM6IGFzc2lnbmVkIFttZW0gMHgx
MDI0MDAwMC0weDEwMjQzZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjUyN10gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDc6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAy
MDAwMDAtMHgyNDAwMDAyMWZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTY1MzFdIHBjaSAwMDAzOjAyOjAwLjA6IEJBUiAxMDogYXNzaWduZWQg
W21lbSAweDI0MDAwMDIyMDAwMC0weDI0MDAwMDIzZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjUzNl0gcGNpIDAwMDM6MDI6MDAuMTogQkFS
IDM6IGFzc2lnbmVkIFttZW0gMHgxMDI0NDAwMC0weDEwMjQ3ZmZmXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjUzOV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDc6IGFz
c2lnbmVkIFttZW0gMHgyNDAwMDAyNDAwMDAtMHgyNDAwMDAyNWZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY1NDNdIHBjaSAwMDAzOjAyOjAw
LjE6IEJBUiAxMDogYXNzaWduZWQgW21lbSAweDI0MDAwMDI2MDAwMC0weDI0MDAwMDI3ZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU0N10g
cGNpIDAwMDM6MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU0OF0gcGNpIDAwMDM6MDI6
MDAuMDogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY1NDldIHBjaSAwMDAzOjAyOjAwLjE6IEJB
UiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTY1NTBdIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBmYWlsZWQg
dG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2NTUxXSBwY2kgMDAwMzowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJd
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTUyXSBwY2kgMDAwMzow
MDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTUzXSBwY2kgMDAwMzowMDowNS4w
OiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDNmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU1NV0gcGNp
X2J1cyAwMDAzOjAwOiBOby4gMiB0cnkgdG8gYXNzaWduIHVuYXNzaWduZWQgcmVzClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTU3XSBwY2kgMDAwMzowMDowMS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjU1OF0gcGNpIDAwMDM6MDA6MDEuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2NTU5XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjU2MF0gcGNpIDAwMDM6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTYx
XSBwY2kgMDAwMzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTYyXSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE2NTY0XSBwY2kgMDAwMzowMDowMS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU2Nl0gcGNpIDAwMDM6MDI6MDAuMDog
QkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjU2N10gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDI6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTY1NjhdIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTY1NjhdIHBjaSAwMDAzOjAyOjAwLjE6IEJBUiAyOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6
ZSAweDAwMjBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTY5XSBw
Y2kgMDAwMzowMDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NTcxXSBwY2kgMDAwMzowMDowNS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE2NTcyXSBwY2kgMDAwMzowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU3NF0gcGNpX2J1cyAwMDAzOjAwOiBBdXRv
bWF0aWNhbGx5IGVuYWJsZWQgcGNpIHJlYWxsb2MsIGlmIHlvdSBoYXZlIHByb2JsZW0sIHRyeSBi
b290aW5nIHdpdGggcGNpPXJlYWxsb2M9b2ZmClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2NTc1XSBwY2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDQgW21lbSAweDEwMDAw
MDAwLTB4MWZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjU3Nl0gcGNpX2J1cyAwMDAzOjAwOiByZXNvdXJjZSA1IFttZW0gMHgyNDAwMDAwMDAw
MDAtMHgyN2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNjU3N10gcGNpX2J1cyAwMDAzOjAxOiByZXNvdXJjZSAxIFttZW0gMHgxMDAwMDAw
MC0weDEwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU3
N10gcGNpX2J1cyAwMDAzOjAxOiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAw
MDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTY1NzhdIHBjaV9idXMgMDAwMzowMjogcmVzb3VyY2UgMSBbbWVtIDB4MTAyMDAwMDAtMHgx
MDNmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY1NzldIHBj
aV9idXMgMDAwMzowMjogcmVzb3VyY2UgMiBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2Zm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
NjQ2XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTZdIChkb21haW4gMDAwNCBbYnVzIDAwLWZm
XSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY2NTBdIGFjcGkgUE5Q
MEEwODowNjogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBT
ZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjcxNV0gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBw
b3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjc3NF0gYWNwaSBQTlAwQTA4OjA2OiBfT1NDOiBPUyBub3cg
Y29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTY3NzZdIGFjcGkgUE5QMEEwODowNjogTUNGRyBxdWlyazogRUNBTSBhdCBb
bWVtIDB4MmJmZmYwMDAwMDAwLTB4MmJmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAw
eGZmZmZkZDczMmQ0M2VhNTgKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTc0MDJdIGFjcGkgUE5QMEEwODowNjogRUNBTSBhcmVhIFttZW0gMHgyYmZmZjAwMDAwMDAtMHgy
YmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTc0MDhdIGFjcGkgUE5QMEEwODowNjogRUNBTSBhdCBbbWVtIDB4
MmJmZmYwMDAwMDAwLTB4MmJmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc0NjBdIFBDSSBob3N0IGJyaWRnZSB0byBidXMg
MDAwNDowMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzQ2MV0gcGNp
X2J1cyAwMDA0OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjAwMDAwMDAtMHgyZmZmZmZm
ZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NDYyXSBw
Y2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyODAwMDAwMDAwMDAtMHgy
YmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzQ2M10gcGNpX2J1cyAwMDA0OjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzQ4MF0gcGNpIDAwMDQ6MDA6
MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzUxN10gcGNpIDAwMDQ6MDA6MDEuMDogWzFkZWY6ZTEx
MV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzU0NF0gcGNpIDAwMDQ6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc1NDVdIHBjaSAwMDA0OjAwOjAxLjA6IFBN
RSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTc1ODddIHBjaSAwMDA0OjAwOjAzLjA6IFsxZGVmOmUxMTNdIHR5cGUgMDEg
Y2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc2
MTRdIHBjaSAwMDA0OjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE3NjE0XSBwY2kgMDAwNDowMDowMy4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE3NjYwXSBwY2kgMDAwNDowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYw
NDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3Njk2XSBwY2kgMDAw
NDowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzY5N10gcGNpIDAwMDQ6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBE
MSBEM2hvdApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzc2OV0gcGNp
IDAwMDQ6MDE6MDAuMDogWzFhMDM6MTE1MF0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzgxN10gcGNpIDAwMDQ6MDE6MDAuMDog
ZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNzg3M10gcGNpIDAwMDQ6MDE6MDAuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc4NzNdIHBjaSAwMDA0OjAxOjAwLjA6IFBNRSMg
c3VwcG9ydGVkIGZyb20gRDAgRDEgRDIgRDNob3QgRDNjb2xkClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE3OTY5XSBwY2lfYnVzIDAwMDQ6MDI6IGV4dGVuZGVkIGNvbmZp
ZyBzcGFjZSBub3QgYWNjZXNzaWJsZQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNzk4Nl0gcGNpIDAwMDQ6MDI6MDAuMDogWzFhMDM6MjAwMF0gdHlwZSAwMCBjbGFzcyAw
eDAzMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAwNF0gcGNp
IDAwMDQ6MDI6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgyMDAwMDAwMC0weDIxZmZmZmZmXQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAxNF0gcGNpIDAwMDQ6MDI6MDAu
MDogcmVnIDB4MTQ6IFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZmXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAyNF0gcGNpIDAwMDQ6MDI6MDAuMDogcmVnIDB4MTg6
IFtpbyAgMHgyZmZmODAwMC0weDJmZmY4MDdmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxODA2OF0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIHRvIGVm
aWZiClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE4MTA0XSBwY2kgMDAw
NDowMjowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxODEwNV0gcGNpIDAwMDQ6MDI6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBE
MSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTgyMTZdIHBjaSAwMDA0OjAzOjAwLjA6IFsxOTEyOjAwMTRdIHR5cGUgMDAgY2xhc3MgMHgwYzAz
MzAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgyMzddIHBjaSAwMDA0
OjAzOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4MjIyMDAwMDAtMHgyMjIwMWZmZiA2NGJpdF0KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgzNDZdIHBjaSAwMDA0OjAzOjAw
LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDgwXSBwY2lfYnVzIDAwMDQ6MDA6IG9uIE5VTUEgbm9k
ZSAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDgzXSBwY2kgMDAw
NDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0
IHByZWZdIHRvIFtidXMgMDEtMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDg0XSBwY2kgMDAwNDowMDow
My4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3Np
emUgMTAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTQ4NV0gcGNp
IDAwMDQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2
NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTQ4N10gcGNpIDAwMDQ6MDA6
MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAw
M10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTk0ODhdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cg
W2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDg5XSBwY2kgMDAwNDowMDowNS4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMg
MDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE5NDkwXSBwY2kgMDAwNDowMDowNS4wOiBicmlkZ2Ugd2luZG93
IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAyMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
OTQ5M10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMDAt
MHgyMmZmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk0OTRd
IHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDI4MDAwMDAwMDAwMC0w
eDI4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxOTQ5NV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4
MjMwMDAwMDAtMHgyMzFmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTk0OTZdIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDI4MDAw
MDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxOTQ5N10gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE0OiBhc3NpZ25l
ZCBbbWVtIDB4MjMyMDAwMDAtMHgyMzNmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTk0OThdIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxNTogYXNzaWduZWQgW21l
bSAweDI4MDAwMDQwMDAwMC0weDI4MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTQ5OV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEz
OiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTk1MDBdIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRv
IGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxOTUwMV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1
MDJdIHBjaSAwMDA0OjAwOjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTUwM10gcGNp
IDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1MDNdIHBjaSAwMDA0OjAwOjA1
LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTUwNV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTk1MDZdIHBjaSAwMDA0OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxOTUwN10gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTk1MDhdIHBjaSAwMDA0OjAwOjA1LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTUwOV0g
cGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1MTBdIHBjaSAwMDA0OjAw
OjAzLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTUxMV0gcGNpIDAwMDQ6MDE6MDAuMDog
QkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1MTJdIHBjaSAwMDA0OjAxOjAwLjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE5NTEzXSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTk1MTRdIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAwOiBhc3NpZ25lZCBbbWVtIDB4
MjAwMDAwMDAtMHgyMWZmZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTk1MTddIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAxOiBhc3NpZ25lZCBbbWVtIDB4MjIwMDAw
MDAtMHgyMjAxZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1
MjFdIHBjaSAwMDA0OjAyOjAwLjA6IEJBUiAyOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDA4
MF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1MjJdIHBjaSAwMDA0
OjAyOjAwLjA6IEJBUiAyOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAwODBdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NTIzXSBwY2kgMDAwNDowMTowMC4w
OiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE5NTI3XSBwY2kgMDAwNDowMTowMC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIw
MDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE5NTM0XSBwY2kgMDAwNDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDEtMDJdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NTM1XSBwY2kgMDAwNDowMDowMS4w
OiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NTM2XSBwY2kgMDAwNDowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDAwMDAwMC0weDI4MDAwMDFmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTUzOF0gcGNpIDAwMDQ6
MDM6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIzMDAxZmZmIDY0Yml0
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTU0OF0gcGNpIDAwMDQ6
MDA6MDMuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxOTU0OV0gcGNpIDAwMDQ6MDA6MDMuMDogICBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgyMzAwMDAwMC0weDIzMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxOTU1MF0gcGNpIDAwMDQ6MDA6MDMuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgy
ODAwMDAyMDAwMDAtMHgyODAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTk1NTJdIHBjaSAwMDA0OjAwOjA1LjA6IFBDSSBicmlkZ2Ug
dG8gW2J1cyAwNF0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1NTRd
IHBjaSAwMDA0OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjMyMDAwMDAtMHgyMzNm
ZmZmZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1NTVdIHBjaSAw
MDA0OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAw
NWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE5NTU3XSBwY2lfYnVzIDAwMDQ6MDA6IFNvbWUgUENJIGRldmljZSByZXNvdXJjZXMgYXJlIHVu
YXNzaWduZWQsIHRyeSBib290aW5nIHdpdGggcGNpPXJlYWxsb2MKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTk1NThdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2UgNCBb
bWVtIDB4MjAwMDAwMDAtMHgyZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE5NTU5XSBwY2lfYnVzIDAwMDQ6MDA6IHJlc291cmNlIDUgW21lbSAw
eDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE5NTYwXSBwY2lfYnVzIDAwMDQ6MDE6IHJlc291cmNlIDEgW21l
bSAweDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE5NTYxXSBwY2lfYnVzIDAwMDQ6MDE6IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDAw
MDAwMC0weDI4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxOTU2MV0gcGNpX2J1cyAwMDA0OjAyOiByZXNvdXJjZSAxIFttZW0gMHgy
MDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxOTU2M10gcGNpX2J1cyAwMDA0OjAzOiByZXNvdXJjZSAxIFttZW0gMHgyMzAwMDAwMC0weDIz
MWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTU2M10gcGNp
X2J1cyAwMDA0OjAzOiByZXNvdXJjZSAyIFttZW0gMHgyODAwMDAyMDAwMDAtMHgyODAwMDAzZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1
NjRdIHBjaV9idXMgMDAwNDowNDogcmVzb3VyY2UgMSBbbWVtIDB4MjMyMDAwMDAtMHgyMzNmZmZm
Zl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1NjVdIHBjaV9idXMg
MDAwNDowNDogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAwNWZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NTkzXSBB
Q1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTddIChkb21haW4gMDAwNSBbYnVzIDAwLWZmXSkKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTk1OTddIGFjcGkgUE5QMEEwODow
NzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50
cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
OTY2MF0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQ
Q0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxOTcxOV0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBub3cgY29udHJv
bHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTk3MjFdIGFjcGkgUE5QMEEwODowNzogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4
MmZmZmYwMDAwMDAwLTB4MmZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZk
ZDczMmQ0M2VhNTgKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAzNDhd
IGFjcGkgUE5QMEEwODowNzogRUNBTSBhcmVhIFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZm
ZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjAzNTRdIGFjcGkgUE5QMEEwODowNzogRUNBTSBhdCBbbWVtIDB4MmZmZmYw
MDAwMDAwLTB4MmZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjA0MTBdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwNTow
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDQxMl0gcGNpX2J1cyAw
MDA1OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3aW5k
b3ddClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwNDEzXSBwY2lfYnVz
IDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyZmZmZGZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDQx
NF0gcGNpX2J1cyAwMDA1OjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDQzMF0gcGNpIDAwMDU6MDA6MDAuMDog
WzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyMDQ3Ml0gcGNpIDAwMDU6MDA6MDEuMDogWzFkZWY6ZTExMV0gdHlw
ZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMDUxMF0gcGNpIDAwMDU6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA1MTFdIHBjaSAwMDA1OjAwOjAxLjA6IFBNRSMgc3Vw
cG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjA1NzFdIHBjaSAwMDA1OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3Mg
MHgwNjA0MDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA2MDddIHBj
aSAwMDA1OjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzIwNjA4XSBwY2kgMDAwNTowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9t
IEQwIEQxIEQzaG90ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwNjYw
XSBwY2kgMDAwNTowMDowNy4wOiBbMWRlZjplMTE3XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwNjk3XSBwY2kgMDAwNTowMDow
Ny4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMDY5OF0gcGNpIDAwMDU6MDA6MDcuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hv
dApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDc3NF0gYWNwaXBocDog
U2xvdCBbMS03XSByZWdpc3RlcmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI0OTgxXSBwY2lfYnVzIDAwMDU6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0OTgyXSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ug
d2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMTAwMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDk4NF0gcGNpIDAwMDU6MDA6MDEu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0
byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDk4NV0gcGNpIDAwMDU6MDA6MDEuMDogYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQ5ODZdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAt
MHgwZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI0OTg3XSBwY2kgMDAwNTowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDJdIGFkZF9zaXpl
IDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI0OTg4XSBwY2kgMDAwNTowMDowNS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEw
MDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEw
MDAwMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDk4OV0gcGNpIDAw
MDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDNd
IGFkZF9zaXplIDEwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ5
OTBdIHBjaSAwMDA1OjAwOjA3LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ5OTFdIHBjaSAw
MDA1OjAwOjA3LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRv
IFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0OTk0XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTQ6
IGFzc2lnbmVkIFttZW0gMHgzMDAwMDAwMC0weDMwMWZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNDk5NV0gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDE1OiBhc3Np
Z25lZCBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0OTk2XSBwY2kgMDAwNTowMDowNS4w
OiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDk5N10gcGNpIDAwMDU6MDA6MDUuMDogQkFS
IDE1OiBhc3NpZ25lZCBbbWVtIDB4MmMwMDAwMjAwMDAwLTB4MmMwMDAwM2ZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0OTk4XSBwY2kgMDAw
NTowMDowNy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDk5OV0gcGNpIDAwMDU6MDA6
MDcuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDAw
XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAwMV0gcGNpIDAwMDU6
MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDAyXSBwY2kgMDAwNTowMDowNS4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAwMl0gcGNpIDAwMDU6MDA6MDUuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI1MDAzXSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNTAwNF0gcGNpIDAwMDU6MDA6MDcuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1
MDA2XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAwN10gcGNpIDAw
MDU6MDA6MDcuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDA4XSBwY2kgMDAwNTowMDow
NS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAwOV0gcGNpIDAwMDU6MDA6MDUuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI1MDEwXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyNTAxMF0gcGNpIDAwMDU6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI1MDExXSBwY2kgMDAwNTowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDEzXSBwY2kgMDAwNTowMDowMS4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDE1XSBwY2kgMDAwNTowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAxN10gcGNpIDAwMDU6MDA6
MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDAyXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNTAxOF0gcGNpIDAwMDU6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0g
MHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyNTAyMF0gcGNpIDAwMDU6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyYzAw
MDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjUwMjFdIHBjaSAwMDA1OjAwOjA3LjA6IFBDSSBicmlkZ2UgdG8g
W2J1cyAwM10KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUwMjNdIHBj
aSAwMDA1OjAwOjA3LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzA0MDAwMDAtMHgzMDVmZmZm
Zl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUwMjRdIHBjaSAwMDA1
OjAwOjA3LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1
MDI3XSBwY2lfYnVzIDAwMDU6MDA6IHJlc291cmNlIDQgW21lbSAweDMwMDAwMDAwLTB4M2ZmZmZm
ZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAyOF0g
cGNpX2J1cyAwMDA1OjAwOiByZXNvdXJjZSA1IFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyZmZmZGZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAy
OF0gcGNpX2J1cyAwMDA1OjAxOiByZXNvdXJjZSAxIFttZW0gMHgzMDAwMDAwMC0weDMwMWZmZmZm
XQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTAyOV0gcGNpX2J1cyAw
MDA1OjAxOiByZXNvdXJjZSAyIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUwMzBdIHBj
aV9idXMgMDAwNTowMjogcmVzb3VyY2UgMSBbbWVtIDB4MzAyMDAwMDAtMHgzMDNmZmZmZl0KU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjUwMzFdIHBjaV9idXMgMDAwNTow
MjogcmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwMjAwMDAwLTB4MmMwMDAwM2ZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDMyXSBwY2lfYnVz
IDAwMDU6MDM6IHJlc291cmNlIDEgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI1MDMzXSBwY2lfYnVzIDAwMDU6MDM6IHJl
c291cmNlIDIgW21lbSAweDJjMDAwMDQwMDAwMC0weDJjMDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzUyMl0gaW9tbXU6IERlZmF1
bHQgZG9tYWluIHR5cGU6IFRyYW5zbGF0ZWQgClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI3NTU1XSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6IFZHQSBkZXZpY2UgYWRk
ZWQ6IGRlY29kZXM9aW8rbWVtLG93bnM9bm9uZSxsb2Nrcz1ub25lClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTU2XSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6IGJy
aWRnZSBjb250cm9sIHBvc3NpYmxlClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI3NTU5XSBwY2kgMDAwNDowMjowMC4wOiB2Z2FhcmI6IHNldHRpbmcgYXMgYm9vdCBkZXZp
Y2UgKFZHQSBsZWdhY3kgcmVzb3VyY2VzIG5vdCBhdmFpbGFibGUpClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTU5XSB2Z2FhcmI6IGxvYWRlZApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzY0Nl0gRURBQyBNQzogVmVyOiAzLjAuMApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzc3N10gUmVnaXN0ZXJlZCBlZml2
YXJzIG9wZXJhdGlvbnMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg0
NjNdIE5ldExhYmVsOiBJbml0aWFsaXppbmcKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43Mjg0NjNdIE5ldExhYmVsOiAgZG9tYWluIGhhc2ggc2l6ZSA9IDEyOApTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODQ2NF0gTmV0TGFiZWw6ICBwcm90b2Nv
bHMgPSBVTkxBQkVMRUQgQ0lQU092NCBDQUxJUFNPClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI4NDczXSBOZXRMYWJlbDogIHVubGFiZWxlZCB0cmFmZmljIGFsbG93ZWQg
YnkgZGVmYXVsdApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODYwNF0g
Y2xvY2tzb3VyY2U6IFN3aXRjaGVkIHRvIGNsb2Nrc291cmNlIGFyY2hfc3lzX2NvdW50ZXIKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcxMTddIFZGUzogRGlzayBxdW90
YXMgZHF1b3RfNi42LjAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzcx
MzNdIFZGUzogRHF1b3QtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA1MTIgKG9yZGVyIDAsIDQw
OTYgYnl0ZXMpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MjQ5XSBB
cHBBcm1vcjogQXBwQXJtb3IgRmlsZXN5c3RlbSBFbmFibGVkClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM3Mjg0XSBwbnA6IFBuUCBBQ1BJIGluaXQKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc5MDldIHN5c3RlbSAwMDowMDogW21lbSAweDNi
ZmZmMDAwMDAwMC0weDNiZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzkxMF0gc3lzdGVtIDAwOjAw
OiBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJl
IHJlc2VydmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3OTExXSBz
eXN0ZW0gMDA6MDA6IFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmYgd2luZG93XSBj
b3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43Mzc5MTJdIHN5c3RlbSAwMDowMDogW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZm
ZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNzkxM10gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MmJmZmYwMDAwMDAwLTB4
MmJmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3OTE0XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyZmZm
ZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc5MTZdIHN5c3RlbSAwMDowMDog
W21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSBy
ZXNlcnZlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzkxN10gc3lz
dGVtIDAwOjAwOiBbbWVtIDB4MzdmZmYwMDAwMDAwLTB4MzdmZmZmZmZmZmZmIHdpbmRvd10gY291
bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM3OTIwXSBzeXN0ZW0gMDA6MDA6IFBsdWcgYW5kIFBsYXkgQUNQSSBkZXZpY2UsIElEcyBQTlAw
YzAyIChhY3RpdmUpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM4MjEy
XSBwbnA6IFBuUCBBQ1BJOiBmb3VuZCAxIGRldmljZXMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzkxMzldIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMgpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjczOTIwNF0gSVAgaWRlbnRzIGhh
c2ggdGFibGUgZW50cmllczogMjYyMTQ0IChvcmRlcjogOSwgMjA5NzE1MiBieXRlcywgbGluZWFy
KQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDgwMV0gdGNwX2xpc3Rl
bl9wb3J0YWRkcl9oYXNoIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA3LCA1MjQy
ODggYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDEwOTFdIFRDUCBlc3RhYmxpc2hlZCBoYXNoIHRhYmxlIGVudHJpZXM6IDUyNDI4OCAob3JkZXI6
IDEwLCA0MTk0MzA0IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQyMTcyXSBUQ1AgYmluZCBoYXNoIHRhYmxlIGVudHJpZXM6IDY1NTM2IChvcmRl
cjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc0MjQzMF0gVENQOiBIYXNoIHRhYmxlcyBjb25maWd1cmVkIChlc3RhYmxpc2hl
ZCA1MjQyODggYmluZCA2NTUzNikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDI1MTRdIFVEUCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0ODU3
NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
MjkxOV0gVURQLUxpdGUgaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDgsIDEwNDg1
NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDM2OTFdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzY5OF0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZh
bWlseSA0NApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzcwNV0gcGNp
IDAwMGM6MDE6MDAuMDogQ0xTIG1pc21hdGNoICg2NCAhPSAzMiksIHVzaW5nIDY0IGJ5dGVzClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzNzgwXSBUcnlpbmcgdG8gdW5w
YWNrIHJvb3RmcyBpbWFnZSBhcyBpbml0cmFtZnMuLi4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNDQ5NTBdIEZyZWVpbmcgaW5pdHJkIG1lbW9yeTogMzcxNjRLClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ2NTkxXSBodyBwZXJmZXZlbnRzOiBl
bmFibGVkIHdpdGggYXJtdjhfcG11djNfMCBQTVUgZHJpdmVyLCA3IGNvdW50ZXJzIGF2YWlsYWJs
ZQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0NjY3OF0ga3ZtIFsxXTog
SVBBIFNpemUgTGltaXQ6IDQ4IGJpdHMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNDY3MDBdIGt2bSBbMV06IEdJQ3YzOiBubyBHSUNWIHJlc291cmNlIGVudHJ5ClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ2NzAxXSBrdm0gWzFdOiBkaXNhYmxp
bmcgR0lDdjIgZW11bGF0aW9uClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTQ2NzEyXSBrdm0gWzFdOiBHSUMgc3lzdGVtIHJlZ2lzdGVyIENQVSBpbnRlcmZhY2UgZW5hYmxl
ZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0NzA1M10ga3ZtIFsxXTog
dmdpYyBpbnRlcnJ1cHQgSVJROQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE0NzQ2NF0ga3ZtIFsxXTogVkhFIG1vZGUgaW5pdGlhbGl6ZWQgc3VjY2Vzc2Z1bGx5ClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ4MzAxXSBJbml0aWFsaXNlIHN5c3Rl
bSB0cnVzdGVkIGtleXJpbmdzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MTQ4MzA4XSBLZXkgdHlwZSBibGFja2xpc3QgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE0ODM0Nl0gd29ya2luZ3NldDogdGltZXN0YW1wX2JpdHM9NDIg
bWF4X29yZGVyPTI0IGJ1Y2tldF9vcmRlcj0wClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTQ5MDEzXSB6YnVkOiBsb2FkZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNDkxMzFdIGludGVncml0eTogUGxhdGZvcm0gS2V5cmluZyBpbml0aWFsaXpl
ZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE0OTEzM10gS2V5IHR5cGUg
YXN5bW1ldHJpYyByZWdpc3RlcmVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTQ5MTM0XSBBc3ltbWV0cmljIGtleSBwYXJzZXIgJ3g1MDknIHJlZ2lzdGVyZWQKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNDkxNDBdIEJsb2NrIGxheWVyIFNDU0kg
Z2VuZXJpYyAoYnNnKSBkcml2ZXIgdmVyc2lvbiAwLjQgbG9hZGVkIChtYWpvciAyNDkpClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTQ5MTc4XSBpbyBzY2hlZHVsZXIgbXEt
ZGVhZGxpbmUgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE1MDIwMl0gc2hwY2hwOiBTdGFuZGFyZCBIb3QgUGx1ZyBQQ0kgQ29udHJvbGxlciBEcml2ZXIg
dmVyc2lvbjogMC40ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUwMzE0
XSBlZmlmYjogcHJvYmluZyBmb3IgZWZpZmIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNTAzMzRdIGVmaWZiOiBmcmFtZWJ1ZmZlciBhdCAweDIwMDAwMDAwLCB1c2luZyAx
ODc2aywgdG90YWwgMTg3NWsKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
NTAzMzVdIGVmaWZiOiBtb2RlIGlzIDgwMHg2MDB4MzIsIGxpbmVsZW5ndGg9MzIwMCwgcGFnZXM9
MQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MDMzNl0gZWZpZmI6IHNj
cm9sbGluZzogcmVkcmF3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTUw
MzM3XSBlZmlmYjogVHJ1ZWNvbG9yOiBzaXplPTg6ODo4OjgsIHNoaWZ0PTI0OjE2Ojg6MApTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE1MDQxMV0gQ29uc29sZTogc3dpdGNo
aW5nIHRvIGNvbG91ciBmcmFtZSBidWZmZXIgZGV2aWNlIDEwMHgzNwpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE1NjMyNV0gZmIwOiBFRkkgVkdBIGZyYW1lIGJ1ZmZlciBk
ZXZpY2UKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNTY3NThdIGlucHV0
OiBQb3dlciBCdXR0b24gYXMgL2RldmljZXMvTE5YU1lTVE06MDAvTE5YU1lCVVM6MDAvUE5QMEMw
QzowMC9pbnB1dC9pbnB1dDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
NTY3NzJdIEFDUEk6IFBvd2VyIEJ1dHRvbiBbUFdSQl0KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjExNjhdIEFDUEkgR1REVDogZm91bmQgMSBTQlNBIGdlbmVyaWMgV2F0
Y2hkb2cocykuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYxNjg3XSBT
ZXJpYWw6IDgyNTAvMTY1NTAgZHJpdmVyLCA0IHBvcnRzLCBJUlEgc2hhcmluZyBlbmFibGVkClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYxOTQwXSBTZXJpYWw6IEFNQkEg
ZHJpdmVyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYxOTczXSBtc21f
c2VyaWFsOiBkcml2ZXIgaW5pdGlhbGl6ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjIwMjFdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogb3B0aW9uIG1h
c2sgMHgwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYyMDQ0XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1
cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYy
MjA5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRy
aWVzIGZvciBjbWRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYyMjQ2
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVz
IGZvciBldnRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYyMzA5XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4wLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZv
ciBwcmlxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYyNjcwXSBhcm0t
c21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjY4MV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
MS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2Mjc5Nl0gYXJtLXNtbXUtdjMgYXJt
LXNtbXUtdjMuMS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MjgzM10gYXJtLXNtbXUtdjMgYXJtLXNt
bXUtdjMuMS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2Mjg5Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuMS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2MzIwMV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
Mi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNjMyMTNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogaWFzIDQ4LWJpdCwg
b2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNjMyNjBdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogYWxs
b2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjMzNzJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogYWxsb2Nh
dGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjM0MDddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjIuYXV0bzogYWxsb2NhdGVk
IDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNjM3MzNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogb3B0aW9uIG1hc2sg
MHgwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYzNzQzXSBhcm0tc21t
dS12MyBhcm0tc21tdS12My4zLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVz
IDB4MDAwMDFmZmYpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYzNzg4
XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVz
IGZvciBjbWRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYzODk5XSBh
cm0tc21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZv
ciBldnRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTYzOTM1XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My4zLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBw
cmlxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY0MjM0XSBhcm0tc21t
dS12MyBhcm0tc21tdS12My40LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDI0Ml0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNC5h
dXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDMxMF0gYXJtLXNtbXUtdjMgYXJtLXNt
bXUtdjMuNC5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDM1NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuNC5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDQ2NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
NC5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NDc5MF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5h
dXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNjQ4MDFdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogaWFzIDQ4LWJpdCwgb2Fz
IDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4xNjQ4NjhdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxsb2Nh
dGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4xNjQ5MTJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxsb2NhdGVk
IDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4xNjUwMjJdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjUuYXV0bzogYWxsb2NhdGVkIDY1
NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4xNjUzMzNdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogb3B0aW9uIG1hc2sgMHgw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1MzQzXSBhcm0tc21tdS12
MyBhcm0tc21tdS12My42LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4
MDAwMDFmZmYpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1MzgxXSBh
cm0tc21tdS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZv
ciBjbWRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1NDQ2XSBhcm0t
c21tdS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBl
dnRxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1NDkwXSBhcm0tc21t
dS12MyBhcm0tc21tdS12My42LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlx
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTY1ODcxXSBhcm0tc21tdS12
MyBhcm0tc21tdS12My43LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE2NTg4MF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNy5hdXRv
OiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NTkxNl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUt
djMuNy5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTQ6MzE6
NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NTk3OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMu
Ny5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE2NjAyMF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNy5h
dXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjE3MDU0MV0gbW91c2VkZXY6IFBTLzIgbW91c2UgZGV2aWNlIGNv
bW1vbiBmb3IgYWxsIG1pY2UKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
NzE0ODhdIHJ0Yy1lZmkgcnRjLWVmaS4wOiByZWdpc3RlcmVkIGFzIHJ0YzAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzE5NjVdIHJ0Yy1lZmkgcnRjLWVmaS4wOiBzZXR0
aW5nIHN5c3RlbSBjbG9jayB0byAyMDIxLTA5LTEzVDA5OjAxOjUxIFVUQyAoMTYzMTUyMzcxMSkK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzI1MjRdIGxlZHRyaWctY3B1
OiByZWdpc3RlcmVkIHRvIGluZGljYXRlIGFjdGl2aXR5IG9uIENQVXMKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xNzI3OTJdIFNNQ0NDOiBTT0NfSUQ6IElEID0gamVwMTA2
OjBhMTY6MDAwMSBSZXZpc2lvbiA9IDB4MDAwMDAwYTEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4xNzMwNDhdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTAK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODEwMTNdIFNlZ21lbnQgUm91
dGluZyB3aXRoIElQdjYKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODEw
MjZdIG1pcDY6IE1vYmlsZSBJUHY2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMTgxMDI3XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDE3ClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTgxMDUyXSBtcGxzX2dzbzogTVBMUyBHU08gc3Vw
cG9ydApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4MTEwMF0gcmVnaXN0
ZXJlZCB0YXNrc3RhdHMgdmVyc2lvbiAxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMTgxMTAyXSBMb2FkaW5nIGNvbXBpbGVkLWluIFguNTA5IGNlcnRpZmljYXRlcwpTZXAg
MTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNzQzNF0gTG9hZGVkIFguNTA5IGNl
cnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBDQTogNmNjZWNlN2U0YzZjMGQxZjYxNDlmM2RkMjdkZmNj
NWNiYjQxOWVhMScKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDc0NDNd
IExvYWRlZCBYLjUwOSBjZXJ0ICdEZWJpYW4gU2VjdXJlIEJvb3QgU2lnbmVyIDIwMjEgLSBsaW51
eDogNGI2ZWY1YWJjYTY2OTgyNTE3OGUwNTJjODQ2NjdjY2JjMDUzMWY4YycKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDc1OThdIHpzd2FwOiBsb2FkZWQgdXNpbmcgcG9v
bCBsem8vemJ1ZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNzY5OV0g
S2V5IHR5cGUgLl9mc2NyeXB0IHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMDc3MDBdIEtleSB0eXBlIC5mc2NyeXB0IHJlZ2lzdGVyZWQKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDc3MDFdIEtleSB0eXBlIGZzY3J5cHQtcHJv
dmlzaW9uaW5nIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMDc3MzBdIEFwcEFybW9yOiBBcHBBcm1vciBzaGExIHBvbGljeSBoYXNoaW5nIGVuYWJsZWQK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY3NjVdIGludGVncml0eTog
TG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIxNzMyNF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnU1VT
RSBMaW51eCBFbnRlcnByaXNlIFNlY3VyZSBCb290IENBOiAzZDRkNDBjZjkzODUzOTAyNGIxY2Zj
NWExMmRlZGZlOGIxN2U3NTVmJwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIxNzMyNF0gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE3NDg0XSBpbnRlZ3JpdHk6IExv
YWRlZCBYLjUwOSBjZXJ0ICdDYW5vbmljYWwgTHRkLiBNYXN0ZXIgQ2VydGlmaWNhdGUgQXV0aG9y
aXR5OiBhZDkxOTkwYmMyMmFiMWY1MTcwNDhjMjNiNjY1NWEyNjhlMzQ1YTYzJwpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNzQ4NV0gaW50ZWdyaXR5OiBMb2FkaW5nIFgu
NTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjE3NjQxXSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdGZWRvcmEgU2VjdXJl
IEJvb3QgQ0E6IGZkZTMyNTk5YzJkNjFkYjFiZjU4MDczMzVkN2IyMGU0Y2Q5NjNiNDInClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE3NjQxXSBpbnRlZ3JpdHk6IExvYWRp
bmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMTc2NTRdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ01pY3Jvc29m
dCBXaW5kb3dzIFByb2R1Y3Rpb24gUENBIDIwMTE6IGE5MjkwMjM5OGUxNmM0OTc3OGNkOTBmOTll
NGY5YWUxN2M1NWFmNTMnClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE3
NjU0XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTc2NjddIGludGVncml0eTogTG9hZGVk
IFguNTA5IGNlcnQgJ01pY3Jvc29mdCBDb3Jwb3JhdGlvbiBVRUZJIENBIDIwMTE6IDEzYWRiZjQz
MDliZDgyNzA5YzhjZDU0ZjMxNmVkNTIyOTg4YTFiZDQnClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjE3NjY3XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNh
dGU6IFVFRkk6ZGIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTc2Nzhd
IGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ01pY3Jvc29mdCBXaW5kb3dzIFByb2R1Y3Rp
b24gUENBIDIwMTE6IGE5MjkwMjM5OGUxNmM0OTc3OGNkOTBmOTllNGY5YWUxN2M1NWFmNTMnClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIyMDc5XSBwY2llcG9ydCAwMDBj
OjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjIyMzgwXSBwY2llcG9ydCAwMDBjOjAwOjAxLjA6IEFFUjogZW5hYmxl
ZCB3aXRoIElSUSAxMDYKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjI0
NTJdIHBjaWVwb3J0IDAwMGQ6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjI3MTNdIHBjaWVwb3J0IDAwMGQ6MDA6
MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIyMjc1OV0gcGNpZXBvcnQgMDAwZDowMDowMi4wOiBBZGRpbmcgdG8gaW9t
bXUgZ3JvdXAgMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjk5NV0g
cGNpZXBvcnQgMDAwZDowMDowMi4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIzMDM5XSBwY2llcG9ydCAwMDBkOjAwOjAz
LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjIzMjczXSBwY2llcG9ydCAwMDBkOjAwOjAzLjA6IEFFUjogZW5hYmxlZCB3aXRo
IElSUSAxMDcKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjMzMTRdIHBj
aWVwb3J0IDAwMGQ6MDA6MDQuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDQKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM1NDhdIHBjaWVwb3J0IDAwMGQ6MDA6MDQuMDog
QUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIyMzYxMF0gcGNpZXBvcnQgMDAwMTowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3Jv
dXAgNQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMzg4M10gcGNpZXBv
cnQgMDAwMTowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA4ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIzOTUzXSBwY2llcG9ydCAwMDAwOjAwOjAxLjA6IEFk
ZGluZyB0byBpb21tdSBncm91cCA2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjI0MjIzXSBwY2llcG9ydCAwMDAwOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAx
MDkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjQyOTBdIHBjaWVwb3J0
IDAwMDI6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDcKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMjQ1NjNdIHBjaWVwb3J0IDAwMDI6MDA6MDEuMDogQUVSOiBl
bmFibGVkIHdpdGggSVJRIDExMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyNDYxNF0gcGNpZXBvcnQgMDAwMjowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgOApT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNDg2NV0gcGNpZXBvcnQgMDAw
MjowMDowNS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEwClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjI0OTMxXSBwY2llcG9ydCAwMDAzOjAwOjAxLjA6IEFkZGluZyB0
byBpb21tdSBncm91cCA5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI1
MjAxXSBwY2llcG9ydCAwMDAzOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTEKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjUyNTRdIHBjaWVwb3J0IDAwMDM6
MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjI1NDkzXSBwY2llcG9ydCAwMDAzOjAwOjA1LjA6IEFFUjogZW5hYmxl
ZCB3aXRoIElSUSAxMTEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjU1
NTVdIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDExClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI1ODI2XSBwY2llcG9ydCAwMDA0OjAw
OjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yMjU4NzFdIHBjaWVwb3J0IDAwMDQ6MDA6MDMuMDogQWRkaW5nIHRvIGlv
bW11IGdyb3VwIDEyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2MTE0
XSBwY2llcG9ydCAwMDA0OjAwOjAzLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTIKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjYxNjVdIHBjaWVwb3J0IDAwMDQ6MDA6
MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDEzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjI2NDA4XSBwY2llcG9ydCAwMDA0OjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3
aXRoIElSUSAxMTIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjY0NzZd
IHBjaWVwb3J0IDAwMDQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE0ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2NzQxXSBwY2llcG9ydCAwMDA1OjAwOjAx
LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIyNzAxOF0gcGNpZXBvcnQgMDAwNTowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0
aCBJUlEgMTEzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI3MDY4XSBw
Y2llcG9ydCAwMDA1OjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNgpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNzMxNV0gcGNpZXBvcnQgMDAwNTowMDowNS4w
OiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjI3MzY3XSBwY2llcG9ydCAwMDA1OjAwOjA3LjA6IEFkZGluZyB0byBpb21tdSBn
cm91cCAxNwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNzYxNl0gcGNp
ZXBvcnQgMDAwNTowMDowNy4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTEzClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI4OTcxXSBGcmVlaW5nIHVudXNlZCBrZXJuZWwg
bWVtb3J5OiA1NDQwSwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQwNzQ4
MV0gQ2hlY2tlZCBXK1ggbWFwcGluZ3M6IHBhc3NlZCwgbm8gVytYIHBhZ2VzIGZvdW5kClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDA3NDg2XSBSdW4gL2luaXQgYXMgaW5p
dCBwcm9jZXNzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDA3NDg4XSAg
IHdpdGggYXJndW1lbnRzOgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQw
NzQ4OF0gICAgIC9pbml0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDA3
NDg5XSAgIHdpdGggZW52aXJvbm1lbnQ6ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNDA3NDkwXSAgICAgSE9NRT0vClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNDA3NDkwXSAgICAgVEVSTT1saW51eApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjQwNzQ5MV0gICAgIEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1h
cm02NApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4MDA1NF0gcHBzX2Nv
cmU6IExpbnV4UFBTIEFQSSB2ZXIuIDEgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjQ4MDA1NV0gcHBzX2NvcmU6IFNvZnR3YXJlIHZlci4gNS4zLjYgLSBD
b3B5cmlnaHQgMjAwNS0yMDA3IFJvZG9sZm8gR2lvbWV0dGkgPGdpb21ldHRpQGxpbnV4Lml0PgpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4MTEwOF0gUFRQIGNsb2NrIHN1
cHBvcnQgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4
NTE3M10gU0NTSSBzdWJzeXN0ZW0gaW5pdGlhbGl6ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS40ODg1NTJdIGlnYjogSW50ZWwoUikgR2lnYWJpdCBFdGhlcm5ldCBOZXR3
b3JrIERyaXZlcgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ4ODU1M10g
aWdiOiBDb3B5cmlnaHQgKGMpIDIwMDctMjAxNCBJbnRlbCBDb3Jwb3JhdGlvbi4KU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODg5MTNdIGlnYiAwMDAzOjAyOjAwLjA6IEFk
ZGluZyB0byBpb21tdSBncm91cCAxOApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjQ4OTMzNF0gQUNQSTogYnVzIHR5cGUgVVNCIHJlZ2lzdGVyZWQKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40ODkzNTFdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGlu
dGVyZmFjZSBkcml2ZXIgdXNiZnMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
MS40ODkzNTZdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGludGVyZmFjZSBkcml2ZXIgaHViClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDg5MzgwXSB1c2Jjb3JlOiByZWdp
c3RlcmVkIG5ldyBkZXZpY2UgZHJpdmVyIHVzYgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjQ5MjcyNl0gbnZtZSAwMDBkOjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91
cCAxOQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MzAxMF0gbnZtZSBu
dm1lMDogcGNpIGZ1bmN0aW9uIDAwMGQ6MDE6MDAuMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAxLjQ5MzA2Ml0gbnZtZSAwMDBkOjAyOjAwLjA6IEFkZGluZyB0byBpb21tdSBn
cm91cCAyMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MzQ0NF0gbnZt
ZSBudm1lMTogcGNpIGZ1bmN0aW9uIDAwMGQ6MDI6MDAuMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjQ5MzQ5MV0gbnZtZSAwMDBkOjAzOjAwLjA6IEFkZGluZyB0byBpb21t
dSBncm91cCAyMQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5MzcyNV0g
bnZtZSBudm1lMjogcGNpIGZ1bmN0aW9uIDAwMGQ6MDM6MDAuMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjQ5Mzc3MF0gbnZtZSAwMDBkOjA0OjAwLjA6IEFkZGluZyB0byBp
b21tdSBncm91cCAyMgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5Mzk5
OF0gbnZtZSBudm1lMzogcGNpIGZ1bmN0aW9uIDAwMGQ6MDQ6MDAuMApTZXAgMTMgMTQ6MzE6NTQg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjQ5OTU0OV0gbXB0M3NhcyB2ZXJzaW9uIDM1LjEwMC4wMC4w
MCBsb2FkZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40OTk2NjJdIG1w
dDNzYXMgMDAwYzowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMjMKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDAwMjVdIG1wdDNzYXNfY20wOiA2MyBCSVQgUENJ
IEJVUyBETUEgQUREUkVTU0lORyBTVVBQT1JURUQsIHRvdGFsIG1lbSAoNjUzMDQ4MDgga0IpClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTAwMzY5XSB4aGNpX2hjZCAwMDA0
OjAzOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyNApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjUwMDY2Nl0geGhjaV9oY2QgMDAwNDowMzowMC4wOiB4SENJIEhvc3Qg
Q29udHJvbGxlcgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjUwMDY3MV0g
eGhjaV9oY2QgMDAwNDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3NpZ25lZCBi
dXMgbnVtYmVyIDEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MDA2ODld
IHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogWmVyb2luZyA2NGJpdCBiYXNlIHJlZ2lzdGVycywgZXhw
ZWN0aW5nIGZhdWx0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTAyMjAx
XSByYW5kb206IGZhc3QgaW5pdCBkb25lClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTA0MTY3XSBudm1lIG52bWUxOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVz
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA0OTE4XSBudm1lIG52bWUy
OiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNTA1MDA5XSBudm1lIG52bWUwOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3Bv
bGwgcXVldWVzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA1MzM0XSBu
dm1lIG52bWUzOiA4MC8wLzAgZGVmYXVsdC9yZWFkL3BvbGwgcXVldWVzClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA1NzI0XSBhc3QgMDAwNDowMjowMC4wOiBBZGRpbmcg
dG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41
MDc3NTVdICBudm1lMW4xOiBwMSBwMiBwMyBwNApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjUwODcwN10gIG52bWUzbjE6IHAxIHAyIHAzIHA0IHA1IHA2ClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA4ODYyXSAgbnZtZTBuMToKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTczNDBdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDog
aGNjIHBhcmFtcyAweDAxNDA1MWNmIGhjaSB2ZXJzaW9uIDB4MTAwIHF1aXJrcyAweDAwMDAwMDEx
MDAwMDA0MTAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTc2ODddIHVz
YiB1c2IxOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MWQ2YiwgaWRQcm9kdWN0PTAw
MDIsIGJjZERldmljZT0gNS4xMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAx
LjY1NzY4OF0gdXNiIHVzYjE6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0zLCBQcm9kdWN0
PTIsIFNlcmlhbE51bWJlcj0xClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NjU3Njg5XSB1c2IgdXNiMTogUHJvZHVjdDogeEhDSSBIb3N0IENvbnRyb2xsZXIKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTc2OTBdIHVzYiB1c2IxOiBNYW51ZmFjdHVy
ZXI6IExpbnV4IDUuMTAuMC04LWFybTY0IHhoY2ktaGNkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjU3NjkxXSB1c2IgdXNiMTogU2VyaWFsTnVtYmVyOiAwMDA0OjAzOjAw
LjAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTc3OTBdIGh1YiAxLTA6
MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NjU3ODAyXSBodWIgMS0wOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjY1Nzk1M10geGhjaV9oY2QgMDAwNDowMzowMC4wOiB4SENJIEhv
c3QgQ29udHJvbGxlcgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1Nzk1
NV0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBuZXcgVVNCIGJ1cyByZWdpc3RlcmVkLCBhc3NpZ25l
ZCBidXMgbnVtYmVyIDIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTc5
NThdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogSG9zdCBzdXBwb3J0cyBVU0IgMy4wIFN1cGVyU3Bl
ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTc5OThdIHVzYiB1c2Iy
OiBXZSBkb24ndCBrbm93IHRoZSBhbGdvcml0aG1zIGZvciBMUE0gZm9yIHRoaXMgaG9zdCwgZGlz
YWJsaW5nIExQTS4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgwMTBd
IHVzYiB1c2IyOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MWQ2YiwgaWRQcm9kdWN0
PTAwMDMsIGJjZERldmljZT0gNS4xMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjY1ODAxMV0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0zLCBQcm9k
dWN0PTIsIFNlcmlhbE51bWJlcj0xClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjU4MDEyXSB1c2IgdXNiMjogUHJvZHVjdDogeEhDSSBIb3N0IENvbnRyb2xsZXIKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgwMTJdIHVzYiB1c2IyOiBNYW51ZmFj
dHVyZXI6IExpbnV4IDUuMTAuMC04LWFybTY0IHhoY2ktaGNkClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNjU4MDEzXSB1c2IgdXNiMjogU2VyaWFsTnVtYmVyOiAwMDA0OjAz
OjAwLjAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgwMzddIGNoZWNr
aW5nIGdlbmVyaWMgKDIwMDAwMDAwIDFkNTAwMCkgdnMgaHcgKDIwMDAwMDAwIDIwMDAwMDApClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjU4MDM4XSBmYjA6IHN3aXRjaGlu
ZyB0byBhc3Rkcm1mYiBmcm9tIEVGSSBWR0EKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS42NTgxMDldIENvbnNvbGU6IHN3aXRjaGluZyB0byBjb2xvdXIgZHVtbXkgZGV2aWNl
IDgweDI1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjU4MTYxXSBhc3Qg
MDAwNDowMjowMC4wOiBbZHJtXSBwbGF0Zm9ybSBoYXMgbm8gSU8gc3BhY2UsIHRyeWluZyBNTUlP
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjU4MTY4XSBhc3QgMDAwNDow
MjowMC4wOiBbZHJtXSBQMkEgYnJpZGdlIGRpc2FibGVkLCB1c2luZyBkZWZhdWx0IGNvbmZpZ3Vy
YXRpb24KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgxNjldIGFzdCAw
MDA0OjAyOjAwLjA6IFtkcm1dIEFTVCAyNTAwIGRldGVjdGVkClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNjU4MTczXSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBBbmFsb2cg
VkdBIG9ubHkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgxNzRdIGFz
dCAwMDA0OjAyOjAwLjA6IFtkcm1dIGRyYW0gTUNMSz04MDAgTWh6IHR5cGU9MSBidXNfd2lkdGg9
MTYKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NTgyMDNdIGh1YiAyLTA6
MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NjU4MjIzXSBbVFRNXSBab25lICBrZXJuZWw6IEF2YWlsYWJsZSBncmFwaGljcyBtZW1vcnk6IDMy
NjUyNDA0IEtpQgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1ODIyNF0g
W1RUTV0gWm9uZSAgIGRtYTMyOiBBdmFpbGFibGUgZ3JhcGhpY3MgbWVtb3J5OiAyMDk3MTUyIEtp
QgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1ODIyNF0gW1RUTV0gSW5p
dGlhbGl6aW5nIHBvb2wgYWxsb2NhdG9yClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNjU4MjI4XSBbVFRNXSBJbml0aWFsaXppbmcgRE1BIHBvb2wgYWxsb2NhdG9yClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjU4MzkyXSBodWIgMi0wOjEuMDogNCBw
b3J0cyBkZXRlY3RlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjY1ODQ2
M10gW2RybV0gSW5pdGlhbGl6ZWQgYXN0IDAuMS4wIDIwMTIwMjI4IGZvciAwMDA0OjAyOjAwLjAg
b24gbWlub3IgMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjc3MjI3N10g
Q29uc29sZTogc3dpdGNoaW5nIHRvIGNvbG91ciBmcmFtZSBidWZmZXIgZGV2aWNlIDI0MHg2NwpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjc3NDk0MF0gYXN0IDAwMDQ6MDI6
MDAuMDogW2RybV0gZmIwOiBhc3Rkcm1mYiBmcmFtZSBidWZmZXIgZGV2aWNlClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODAyOTk5XSBpZ2IgMDAwMzowMjowMC4wOiBhZGRl
ZCBQSEMgb24gZXRoMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjgwMzAw
MF0gaWdiIDAwMDM6MDI6MDAuMDogSW50ZWwoUikgR2lnYWJpdCBFdGhlcm5ldCBOZXR3b3JrIENv
bm5lY3Rpb24KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44MDMwMDFdIGln
YiAwMDAzOjAyOjAwLjA6IGV0aDA6IChQQ0llOjUuMEdiL3M6V2lkdGggeDIpIGQ4OjVlOmQzOjAy
OmIxOjkzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODAzMDgzXSBpZ2Ig
MDAwMzowMjowMC4wOiBldGgwOiBQQkEgTm86IDEwNjMwMC0wMDAKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44MDMwODRdIGlnYiAwMDAzOjAyOjAwLjA6IFVzaW5nIE1TSS1Y
IGludGVycnVwdHMuIDggcnggcXVldWUocyksIDggdHggcXVldWUocykKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44MDMxNjJdIGlnYiAwMDAzOjAyOjAwLjE6IEFkZGluZyB0
byBpb21tdSBncm91cCAyNQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1
ODA5Nl0gbXB0M3Nhc19jbTA6IEN1cnJlbnRIb3N0UGFnZVNpemUgaXMgMDogU2V0dGluZyBkZWZh
dWx0IGhvc3QgcGFnZSBzaXplIHRvIDRrClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODU4MTE0XSBtcHQzc2FzX2NtMDogTVNJLVggdmVjdG9ycyBzdXBwb3J0ZWQ6IDk2ClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU4MTE1XSAJIG5vIG9mIGNvcmVz
OiA4MCwgbWF4X21zaXhfdmVjdG9yczogLTEKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NTgxMTZdIG1wdDNzYXNfY20wOiAgMCA4MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjg1OTQ0M10gbXB0M3Nhc19jbTA6IEhpZ2ggSU9QcyBxdWV1ZXMgOiBk
aXNhYmxlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1OTQ0NV0gbXB0
M3NhczAtbXNpeDA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY1ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDQ1XSBtcHQzc2FzMC1tc2l4MTogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0NjYKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTk0
NDZdIG1wdDNzYXMwLW1zaXgyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2NwpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1OTQ0Nl0gbXB0M3NhczAtbXNpeDM6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDY4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODU5NDQ3XSBtcHQzc2FzMC1tc2l4NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjkKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTk0NDddIG1wdDNzYXMwLW1zaXg1
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg1OTQ0OF0gbXB0M3NhczAtbXNpeDY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDcxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDQ4XSBtcHQzc2Fz
MC1tc2l4NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzIKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMS44NTk0NDldIG1wdDNzYXMwLW1zaXg4OiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDQ3MwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1OTQ1MF0g
bXB0M3NhczAtbXNpeDk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc0ClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDUwXSBtcHQzc2FzMC1tc2l4MTA6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDc1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODU5NDUxXSBtcHQzc2FzMC1tc2l4MTE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc2ClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDUxXSBtcHQzc2FzMC1tc2l4MTI6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODU5NDUyXSBtcHQzc2FzMC1tc2l4MTM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDc4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDUyXSBtcHQzc2Fz
MC1tc2l4MTQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc5ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU5NDUzXSBtcHQzc2FzMC1tc2l4MTU6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDgwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU0
XSBtcHQzc2FzMC1tc2l4MTY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgxClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU0XSBtcHQzc2FzMC1tc2l4MTc6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDgyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODU5NDU1XSBtcHQzc2FzMC1tc2l4MTg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgzClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU2XSBtcHQzc2FzMC1tc2l4
MTk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODU5NDU2XSBtcHQzc2FzMC1tc2l4MjA6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNDg1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU3XSBtcHQz
c2FzMC1tc2l4MjE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg2ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU3XSBtcHQzc2FzMC1tc2l4MjI6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNDg3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5
NDU4XSBtcHQzc2FzMC1tc2l4MjM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg4ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDU5XSBtcHQzc2FzMC1tc2l4MjQ6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNDg5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODU5NDU5XSBtcHQzc2FzMC1tc2l4MjU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkw
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDYwXSBtcHQzc2FzMC1t
c2l4MjY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODU5NDYwXSBtcHQzc2FzMC1tc2l4Mjc6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNDkyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDYxXSBt
cHQzc2FzMC1tc2l4Mjg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkzClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDYxXSBtcHQzc2FzMC1tc2l4Mjk6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDk0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODU5NDYyXSBtcHQzc2FzMC1tc2l4MzA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk1ClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDYzXSBtcHQzc2FzMC1tc2l4MzE6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODU5NDYzXSBtcHQzc2FzMC1tc2l4MzI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDk3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY0XSBtcHQzc2Fz
MC1tc2l4MzM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk4ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU5NDY0XSBtcHQzc2FzMC1tc2l4MzQ6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDk5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY1
XSBtcHQzc2FzMC1tc2l4MzU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAwClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY1XSBtcHQzc2FzMC1tc2l4MzY6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTAxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODU5NDY2XSBtcHQzc2FzMC1tc2l4Mzc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAyClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY2XSBtcHQzc2FzMC1tc2l4
Mzg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODU5NDY3XSBtcHQzc2FzMC1tc2l4Mzk6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTA0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY4XSBtcHQz
c2FzMC1tc2l4NDA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA1ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY4XSBtcHQzc2FzMC1tc2l4NDE6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTA2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5
NDY5XSBtcHQzc2FzMC1tc2l4NDI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA3ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDY5XSBtcHQzc2FzMC1tc2l4NDM6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTA4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODU5NDcwXSBtcHQzc2FzMC1tc2l4NDQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA5
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDcxXSBtcHQzc2FzMC1t
c2l4NDU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTEwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODU5NDcxXSBtcHQzc2FzMC1tc2l4NDY6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTExClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDcyXSBt
cHQzc2FzMC1tc2l4NDc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTEyClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDcyXSBtcHQzc2FzMC1tc2l4NDg6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTEzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODU5NDczXSBtcHQzc2FzMC1tc2l4NDk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE0ClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc0XSBtcHQzc2FzMC1tc2l4NTA6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODU5NDc0XSBtcHQzc2FzMC1tc2l4NTE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTE2ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc1XSBtcHQzc2Fz
MC1tc2l4NTI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE3ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU5NDc1XSBtcHQzc2FzMC1tc2l4NTM6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTE4ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc2
XSBtcHQzc2FzMC1tc2l4NTQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE5ClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc3XSBtcHQzc2FzMC1tc2l4NTU6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTIwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODU5NDc3XSBtcHQzc2FzMC1tc2l4NTY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIxClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc4XSBtcHQzc2FzMC1tc2l4
NTc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODU5NDc4XSBtcHQzc2FzMC1tc2l4NTg6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTIzClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc5XSBtcHQz
c2FzMC1tc2l4NTk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI0ClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDc5XSBtcHQzc2FzMC1tc2l4NjA6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTI1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5
NDgwXSBtcHQzc2FzMC1tc2l4NjE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI2ClNlcCAxMyAx
NDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDgxXSBtcHQzc2FzMC1tc2l4NjI6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTI3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODU5NDgxXSBtcHQzc2FzMC1tc2l4NjM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI4
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDgyXSBtcHQzc2FzMC1t
c2l4NjQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODU5NDgyXSBtcHQzc2FzMC1tc2l4NjU6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTMwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDgzXSBt
cHQzc2FzMC1tc2l4NjY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMxClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg0XSBtcHQzc2FzMC1tc2l4Njc6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTMyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODU5NDg0XSBtcHQzc2FzMC1tc2l4Njg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMzClNlcCAx
MyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg1XSBtcHQzc2FzMC1tc2l4Njk6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODU5NDg1XSBtcHQzc2FzMC1tc2l4NzA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTM1ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg2XSBtcHQzc2Fz
MC1tc2l4NzE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM2ClNlcCAxMyAxNDozMTo1NCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODU5NDg3XSBtcHQzc2FzMC1tc2l4NzI6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTM3ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg4
XSBtcHQzc2FzMC1tc2l4NzM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM4ClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg4XSBtcHQzc2FzMC1tc2l4NzQ6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTM5ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODU5NDg5XSBtcHQzc2FzMC1tc2l4NzU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQwClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDg5XSBtcHQzc2FzMC1tc2l4
NzY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODU5NDkwXSBtcHQzc2FzMC1tc2l4Nzc6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTQyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDkxXSBtcHQz
c2FzMC1tc2l4Nzg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQzClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODU5NDkxXSBtcHQzc2FzMC1tc2l4Nzk6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTQ0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU5
NDkzXSBtcHQzc2FzX2NtMDogaW9tZW0oMHgwMDAwMDAwMDQwMTQwMDAwKSwgbWFwcGVkKDB4KF9f
X19wdHJ2YWxfX19fKSksIHNpemUoNjU1MzYpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODU5NDk0XSBtcHQzc2FzX2NtMDogaW9wb3J0KDB4MDAwMDAwMDAwMDAwMDAwMCks
IHNpemUoMCkKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NzI2NzFdIGln
YiAwMDAzOjAyOjAwLjE6IGFkZGVkIFBIQyBvbiBldGgxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODcyNjczXSBpZ2IgMDAwMzowMjowMC4xOiBJbnRlbChSKSBHaWdhYml0
IEV0aGVybmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg3MjY3NF0gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogKFBDSWU6NS4wR2IvczpX
aWR0aCB4MikgZDg6NWU6ZDM6MDI6YjE6OTQKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44NzI3NTVdIGlnYiAwMDAzOjAyOjAwLjE6IGV0aDE6IFBCQSBObzogMTA2MzAwLTAw
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3Mjc1Nl0gaWdiIDAwMDM6
MDI6MDAuMTogVXNpbmcgTVNJLVggaW50ZXJydXB0cy4gOCByeCBxdWV1ZShzKSwgOCB0eCBxdWV1
ZShzKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3Mzc0NF0gaWdiIDAw
MDM6MDI6MDAuMSBlbm8yOiByZW5hbWVkIGZyb20gZXRoMQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAxLjg5Mjc5M10gaWdiIDAwMDM6MDI6MDAuMCBlbm8xOiByZW5hbWVkIGZy
b20gZXRoMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNzExOV0gbXB0
M3Nhc19jbTA6IEN1cnJlbnRIb3N0UGFnZVNpemUgaXMgMDogU2V0dGluZyBkZWZhdWx0IGhvc3Qg
cGFnZSBzaXplIHRvIDRrClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTE3
MTIxXSBtcHQzc2FzX2NtMDogc2VuZGluZyBkaWFnIHJlc2V0ICEhClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuOTcyNjExXSB1c2IgMS0xOiBuZXcgbG93LXNwZWVkIFVTQiBk
ZXZpY2UgbnVtYmVyIDIgdXNpbmcgeGhjaV9oY2QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi4xNjUwNjNdIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRv
cj00MTNjLCBpZFByb2R1Y3Q9MzAxYSwgYmNkRGV2aWNlPSAxLjAwClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMTY1MDY0XSB1c2IgMS0xOiBOZXcgVVNCIGRldmljZSBzdHJp
bmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAyLjE2NTA2NV0gdXNiIDEtMTogUHJvZHVjdDogRGVsbCBNUzExNiBV
U0IgT3B0aWNhbCBNb3VzZQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAyLjE2
NTA2Nl0gdXNiIDEtMTogTWFudWZhY3R1cmVyOiBQaXhBcnQKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMi4xNzQxODNdIGhpZDogcmF3IEhJRCBldmVudHMgZHJpdmVyIChDKSBK
aXJpIEtvc2luYQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAyLjE4NjE2NF0g
dXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciB1c2JoaWQKU2VwIDEzIDE0
OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xODYxNjZdIHVzYmhpZDogVVNCIEhJRCBjb3Jl
IGRyaXZlcgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAyLjE4NzQ4NV0gaW5w
dXQ6IFBpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlIGFzIC9kZXZpY2VzL3BjaTAw
MDQ6MDAvMDAwNDowMDowMy4wLzAwMDQ6MDM6MDAuMC91c2IxLzEtMS8xLTE6MS4wLzAwMDM6NDEz
QzozMDFBLjAwMDEvaW5wdXQvaW5wdXQxClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDIuMTg3NTM0XSBoaWQtZ2VuZXJpYyAwMDAzOjQxM0M6MzAxQS4wMDAxOiBpbnB1dCxoaWRy
YXcwOiBVU0IgSElEIHYxLjExIE1vdXNlIFtQaXhBcnQgRGVsbCBNUzExNiBVU0IgT3B0aWNhbCBN
b3VzZV0gb24gdXNiLTAwMDQ6MDM6MDAuMC0xL2lucHV0MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAyLjI3MzI1OV0gdXNiIDItMzogbmV3IFN1cGVyU3BlZWQgR2VuIDEgVVNC
IGRldmljZSBudW1iZXIgMiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtl
cm5lbDogWyAgICAyLjMwMTczOV0gdXNiIDItMzogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVu
ZG9yPTA1ZTMsIGlkUHJvZHVjdD0wNjIwLCBiY2REZXZpY2U9OTMuMTAKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zMDE3NDBdIHVzYiAyLTM6IE5ldyBVU0IgZGV2aWNlIHN0
cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMzAxNzQwXSB1c2IgMi0zOiBQcm9kdWN0OiBVU0IzLjEgSHVi
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzAxNzQxXSB1c2IgMi0zOiBN
YW51ZmFjdHVyZXI6IEdlbmVzeXNMb2dpYwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAyLjMwNDc3NF0gaHViIDItMzoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zMDUzNjddIGh1YiAyLTM6MS4wOiA0IHBvcnRzIGRldGVj
dGVkClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNDMyNTkxXSB1c2IgMS0y
OiBuZXcgbG93LXNwZWVkIFVTQiBkZXZpY2UgbnVtYmVyIDMgdXNpbmcgeGhjaV9oY2QKU2VwIDEz
IDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi42NDQwNThdIHVzYiAxLTI6IE5ldyBVU0Ig
ZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNGQ5LCBpZFByb2R1Y3Q9MTIwMywgYmNkRGV2aWNlPSAy
LjgwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNjQ0MDYwXSB1c2IgMS0y
OiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MCwgUHJvZHVjdD0wLCBTZXJpYWxOdW1iZXI9
MApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAyLjY4MjMwOF0gaW5wdXQ6IEhJ
RCAwNGQ5OjEyMDMgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzow
MC4wL3VzYjEvMS0yLzEtMjoxLjAvMDAwMzowNEQ5OjEyMDMuMDAwMi9pbnB1dC9pbnB1dDIKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi43NDQ2ODVdIGhpZC1nZW5lcmljIDAw
MDM6MDREOToxMjAzLjAwMDI6IGlucHV0LGhpZHJhdzE6IFVTQiBISUQgdjEuMTEgS2V5Ym9hcmQg
W0hJRCAwNGQ5OjEyMDNdIG9uIHVzYi0wMDA0OjAzOjAwLjAtMi9pbnB1dDAKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi43ODcxODddIGlucHV0OiBISUQgMDRkOToxMjAzIFN5
c3RlbSBDb250cm9sIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDowMDowMy4wLzAwMDQ6MDM6
MDAuMC91c2IxLzEtMi8xLTI6MS4xLzAwMDM6MDREOToxMjAzLjAwMDMvaW5wdXQvaW5wdXQzClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuODQ4NjA3XSBpbnB1dDogSElEIDA0
ZDk6MTIwMyBDb25zdW1lciBDb250cm9sIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDowMDow
My4wLzAwMDQ6MDM6MDAuMC91c2IxLzEtMi8xLTI6MS4xLzAwMDM6MDREOToxMjAzLjAwMDMvaW5w
dXQvaW5wdXQ0ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuODQ4NjMxXSBo
aWQtZ2VuZXJpYyAwMDAzOjA0RDk6MTIwMy4wMDAzOiBpbnB1dCxoaWRyYXcyOiBVU0IgSElEIHYx
LjExIERldmljZSBbSElEIDA0ZDk6MTIwM10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0MQpT
ZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAyLjk1NjEwMl0gbXB0M3Nhc19jbTA6
IGRpYWcgcmVzZXQ6IFNVQ0NFU1MKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
Mi45ODQ1OTFdIHVzYiAxLTM6IG5ldyBoaWdoLXNwZWVkIFVTQiBkZXZpY2UgbnVtYmVyIDQgdXNp
bmcgeGhjaV9oY2QKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMi45OTExMTZd
IG1wdDNzYXNfY20wOiBzY2F0dGVyIGdhdGhlcjogc2dlX2luX21haW5fbXNnKDEpLCBzZ2VfcGVy
X2NoYWluKDcpLCBzZ2VfcGVyX2lvKDEyOCksIGNoYWluc19wZXJfaW8oMTkpClNlcCAxMyAxNDoz
MTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTkyMTg3XSBtcHQzc2FzX2NtMDogcmVxdWVzdCBw
b29sKDB4KF9fX19wdHJ2YWxfX19fKSkgLSBkbWEoMHhmZjUwMDAwMCk6IGRlcHRoKDQyMjQpLCBm
cmFtZV9zaXplKDEyOCksIHBvb2xfc2l6ZSg1Mjgga0IpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4g
a2VybmVsOiBbICAgIDMuMDEyNjkxXSBtcHQzc2FzX2NtMDogc2Vuc2UgcG9vbCgweChfX19fcHRy
dmFsX19fXykpLSBkbWEoMHhmZWIwMDAwMCk6IGRlcHRoKDM5NjMpLGVsZW1lbnRfc2l6ZSg5Niks
IHBvb2xfc2l6ZSgzNzEga0IpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
MDEyNzkwXSBtcHQzc2FzX2NtMDogY29uZmlnIHBhZ2UoMHgoX19fX3B0cnZhbF9fX18pKSAtIGRt
YSgweGZlOWY2MDAwKTogc2l6ZSg1MTIpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuMDEyNzkwXSBtcHQzc2FzX2NtMDogQWxsb2NhdGVkIHBoeXNpY2FsIG1lbW9yeTogc2l6
ZSg3Mjc4IGtCKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjAxMjc5MV0g
bXB0M3Nhc19jbTA6IEN1cnJlbnQgQ29udHJvbGxlciBRdWV1ZSBEZXB0aCgzOTYwKSxNYXggQ29u
dHJvbGxlciBRdWV1ZSBEZXB0aCg0MDk2KQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAzLjAxMjc5Ml0gbXB0M3Nhc19jbTA6IFNjYXR0ZXIgR2F0aGVyIEVsZW1lbnRzIHBlciBJ
TygxMjgpClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTQyMDk5XSB1c2Ig
MS0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywgaWRQcm9kdWN0PTA2MTAs
IGJjZERldmljZT05My4xMApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjE0
MjEwMF0gdXNiIDEtMzogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9Miwg
U2VyaWFsTnVtYmVyPTAKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNDIx
MDFdIHVzYiAxLTM6IFByb2R1Y3Q6IFVTQjIuMCBIdWIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4xNDIxMDJdIHVzYiAxLTM6IE1hbnVmYWN0dXJlcjogR2VuZXN5c0xvZ2lj
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTQ2MTIwXSBodWIgMS0zOjEu
MDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjE0
NjcyNV0gaHViIDEtMzoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMy4yMTM3NDNdIG1wdDNzYXNfY20wOiBfYmFzZV9kaXNwbGF5X2Z3cGtn
X3ZlcnNpb246IGNvbXBsZXRlClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
MjE0MDY3XSBtcHQzc2FzX2NtMDogTFNJU0FTMzAwODogRldWZXJzaW9uKDE2LjAwLjEwLjAwKSwg
Q2hpcFJldmlzaW9uKDB4MDIpLCBCaW9zVmVyc2lvbigxNS4wMC4wMS4wMCkKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMTQwNjddIG1wdDNzYXNfY20wOiBQcm90b2NvbD0o
SW5pdGlhdG9yKSwgQ2FwYWJpbGl0aWVzPShSYWlkLFRMUixFRURQLFNuYXBzaG90IEJ1ZmZlcixE
aWFnIFRyYWNlIEJ1ZmZlcixUYXNrIFNldCBGdWxsLE5DUSkKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMy4yMTQxMTddIHNjc2kgaG9zdDA6IEZ1c2lvbiBNUFQgU0FTIEhvc3QK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yMTQ2ODldIG1wdDNzYXNfY20w
OiBzZW5kaW5nIHBvcnQgZW5hYmxlICEhClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuMjQxMjAwXSBQTTogSW1hZ2Ugbm90IGZvdW5kIChjb2RlIC0yMikKU2VwIDEzIDE0OjMx
OjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yOTc2ODRdIEVYVDQtZnMgKG52bWUzbjFwMik6IG1v
dW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy4zMjY4OTZdIE5vdCBhY3RpdmF0aW5n
IE1hbmRhdG9yeSBBY2Nlc3MgQ29udHJvbCBhcyAvc2Jpbi90b21veW8taW5pdCBkb2VzIG5vdCBl
eGlzdC4KU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy41MjY1ODddIHJhbmRv
bTogc3lzdGVtZDogdW5pbml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJlYWQpClNl
cCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTI3MzQ2XSByYW5kb206IHN5c3Rl
bWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjUyNzU0OV0gcmFuZG9tOiBzeXN0ZW1kOiB1bmlu
aXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDE0OjMxOjU0IGRl
YmlhbiBrZXJuZWw6IFsgICAgMy41NDM0NDNdIGZ1c2U6IGluaXQgKEFQSSB2ZXJzaW9uIDcuMzIp
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTQ0ODc3XSBFWFQ0LWZzIChu
dm1lM24xcDIpOiByZS1tb3VudGVkLiBPcHRzOiBlcnJvcnM9cmVtb3VudC1ybwpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjY0MTI3M10gYWNwaS10YWQgQUNQSTAwMEU6MDA6
IFVuc3VwcG9ydGVkIGNhcGFiaWxpdGllcwpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDog
WyAgICAzLjY4MjM1NV0geGdlbmUtc2xpbXByby1od21vbiBBUE1DMEQyOTowMDogQVBNIFgtR2Vu
ZSBTb0MgSFcgbW9uaXRvciBkcml2ZXIgcmVnaXN0ZXJlZApTZXAgMTMgMTQ6MzE6NTQgZGViaWFu
IGtlcm5lbDogWyAgICAzLjY5MDkzMl0gSVBNSSBtZXNzYWdlIGhhbmRsZXI6IHZlcnNpb24gMzku
MgpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjY5NDYxNV0gaXBtaSBkZXZp
Y2UgaW50ZXJmYWNlClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzAxMjQ3
XSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVyClNlcCAxMyAxNDozMTo1NCBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuNzAxNzk1XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBhbnkg
U3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAz
Ljc3MjY3Ml0gRVhUNC1mcyAobnZtZTNuMXA1KTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3Jk
ZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTQ6MzE6NTQgZGViaWFuIGtlcm5l
bDogWyAgICAzLjc3Mjg2Ml0gRVhUNC1mcyAobnZtZTNuMXAzKTogbW91bnRlZCBmaWxlc3lzdGVt
IHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjc5NjI0Ml0gRVhUNC1mcyAobnZtZTNuMXA2KTogbW91bnRlZCBm
aWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTQ6
MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICAzLjgwODYwMl0gQWRkaW5nIDEwMDA0NDRrIHN3YXAg
b24gL2Rldi9udm1lM24xcDQuICBQcmlvcml0eTotMiBleHRlbnRzOjEgYWNyb3NzOjEwMDA0NDRr
IFNTRlMKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44MTEwNDJdIGlwbWlf
c3NpZjogSVBNSSBTU0lGIEludGVyZmFjZSBkcml2ZXIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy44MTU3NjRdIGlwbWlfc3NpZiBpMmMtQU1QQzAwMDQ6MDA6IGlwbWlfc3Np
ZjogVHJ5aW5nIEFDUEktc3BlY2lmaWVkIFNTSUYgaW50ZXJmYWNlIGF0IGkyYyBhZGRyZXNzIDB4
MTAsIGFkYXB0ZXIgU3lub3BzeXMgRGVzaWduV2FyZSBJMkMgYWRhcHRlciwgc2xhdmUgYWRkcmVz
cyAweDIwClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODM4MDcxXSBzYnNh
LWd3ZHQgc2JzYS1nd2R0LjA6IEluaXRpYWxpemVkIHdpdGggMTBzIHRpbWVvdXQgQCAyNTAwMDAw
MCBIeiwgYWN0aW9uPTAuClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODQ4
MzM2XSByYW5kb206IGNybmcgaW5pdCBkb25lClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDMuODQ4MzM5XSByYW5kb206IDcgdXJhbmRvbSB3YXJuaW5nKHMpIG1pc3NlZCBkdWUg
dG8gcmF0ZWxpbWl0aW5nClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODYy
MDc3XSBjcnlwdGQ6IG1heF9jcHVfcWxlbiBzZXQgdG8gMTAwMApTZXAgMTMgMTQ6MzE6NTQgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjg2NjU2MF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTIz
NzE0LjE5MjoyKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHBy
b2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2ZmaWNlLXhwZGZpbXBvcnQiIHBpZD0xMTE3
IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy44NjY2NTJdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyMzcxNC4xOTI6Myk6IGFw
cGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZp
bmVkIiBuYW1lPSJsaWJyZW9mZmljZS1zZW5kZG9jIiBwaWQ9MTEyMSBjb21tPSJhcHBhcm1vcl9w
YXJzZXIiClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODY2Njc2XSBwc3Rv
cmU6IFVzaW5nIGNyYXNoIGR1bXAgY29tcHJlc3Npb246IGRlZmxhdGUKU2VwIDEzIDE0OjMxOjU0
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NjY3MDVdIHBzdG9yZTogUmVnaXN0ZXJlZCBlZmkgYXMg
cGVyc2lzdGVudCBzdG9yZSBiYWNrZW5kClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuODY3OTY1XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MjM3MTQuMTkyOjQpOiBh
cHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25m
aW5lZCIgbmFtZT0ibHNiX3JlbGVhc2UiIHBpZD0xMTE0IGNvbW09ImFwcGFybW9yX3BhcnNlciIK
U2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NjgwMTJdIGF1ZGl0OiB0eXBl
PTE0MDAgYXVkaXQoMTYzMTUyMzcxNC4xOTI6NSk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlv
bj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJudmlkaWFfbW9kcHJv
YmUiIHBpZD0xMTIyIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjMxOjU0IGRlYmlh
biBrZXJuZWw6IFsgICAgMy44NjgwMTVdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyMzcx
NC4xOTI6Nik6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9m
aWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJudmlkaWFfbW9kcHJvYmUvL2ttb2QiIHBpZD0xMTIyIGNv
bW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAg
My44NzAxNDNdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyMzcxNC4xOTY6Nyk6IGFwcGFy
bW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVk
IiBuYW1lPSIvdXNyL2Jpbi9tYW4iIHBpZD0xMTIwIGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2Vw
IDEzIDE0OjMxOjU0IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NzAxNDZdIGF1ZGl0OiB0eXBlPTE0
MDAgYXVkaXQoMTYzMTUyMzcxNC4xOTY6OCk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0i
cHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJtYW5fZmlsdGVyIiBwaWQ9
MTEyMCBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVs
OiBbICAgIDMuODcwMTQ4XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MjM3MTQuMTk2Ojkp
OiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5j
b25maW5lZCIgbmFtZT0ibWFuX2dyb2ZmIiBwaWQ9MTEyMCBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODcwNDc4XSBhdWRpdDogdHlw
ZT0xNDAwIGF1ZGl0KDE2MzE1MjM3MTQuMTk2OjEwKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0
aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2ZmaWNl
LW9vcHNsYXNoIiBwaWQ9MTExOSBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNDozMTo1
NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODc4MDIyXSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRl
cmZhY2UgZHJpdmVyClNlcCAxMyAxNDozMTo1NCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODc4NDAz
XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBhbnkgU3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMg
MTQ6MzE6NTQgZGViaWFuIGtlcm5lbDogWyAgICA0LjA5MDg3NV0gaXBtaV9zc2lmIGkyYy1BTVBD
MDAwNDowMDogSVBNSSBtZXNzYWdlIGhhbmRsZXI6IEZvdW5kIG5ldyBCTUMgKG1hbl9pZDogMHgw
MDNjMGEsIHByb2RfaWQ6IDB4MDIxMiwgZGV2X2lkOiAweDIwKQpTZXAgMTMgMTQ6MzE6NTYgZGVi
aWFuIGtlcm5lbDogWyAgICA1Ljc2Njc4OF0gbXB0M3Nhc19jbTA6IGhvc3RfYWRkOiBoYW5kbGUo
MHgwMDAxKSwgc2FzX2FkZHIoMHg1MDAxNDg1MDAwNzc5ZTMwKSwgcGh5cyg4KQpTZXAgMTMgMTQ6
MzE6NTcgZGViaWFuIGtlcm5lbDogWyAgICA3LjE1NzA5Nl0gaWdiIDAwMDM6MDI6MDAuMSBlbm8y
OiBpZ2I6IGVubzIgTklDIExpbmsgaXMgVXAgMTAwMCBNYnBzIEZ1bGwgRHVwbGV4LCBGbG93IENv
bnRyb2w6IFJYL1RYClNlcCAxMyAxNDozMTo1NyBkZWJpYW4ga2VybmVsOiBbICAgIDcuMTU3Mjcy
XSBJUHY2OiBBRERSQ09ORihORVRERVZfQ0hBTkdFKTogZW5vMjogbGluayBiZWNvbWVzIHJlYWR5
ClNlcCAxMyAxNDozMjowMSBkZWJpYW4ga2VybmVsOiBbICAgMTAuODkyNjgxXSBtcHQzc2FzX2Nt
MDogcG9ydCBlbmFibGU6IFNVQ0NFU1MKU2VwIDEzIDE1OjA5OjIyIGRlYmlhbiBrZXJuZWw6IFsg
MjI1My45OTY2OTRdIGthdWRpdGRfcHJpbnRrX3NrYjogNCBjYWxsYmFja3Mgc3VwcHJlc3NlZApT
ZXAgMTMgMTU6MDk6MjIgZGViaWFuIGtlcm5lbDogWyAyMjUzLjk5NjY5N10gYXVkaXQ6IHR5cGU9
MTQwMCBhdWRpdCgxNjMxNTI1OTYyLjk1MToxNSk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlv
bj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJ2aXJ0LWFhLWhlbHBl
ciIgcGlkPTM2ODMgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6MDk6MjMgZGViaWFu
IGtlcm5lbDogWyAyMjU0LjA2MDcwMF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI1OTYz
LjAxNToxNik6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9m
aWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJ2aXJ0ZCIgcGlkPTM2ODkgY29tbT0iYXBwYXJtb3Jf
cGFyc2VyIgpTZXAgMTMgMTU6MDk6MjMgZGViaWFuIGtlcm5lbDogWyAyMjU0LjA2MTY3Ml0gYXVk
aXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI1OTYzLjAxOToxNyk6IGFwcGFybW9yPSJTVEFUVVMi
IG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJ2
aXJ0ZC8vcWVtdV9icmlkZ2VfaGVscGVyIiBwaWQ9MzY4OSBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxNTozMjowOSBkZWJpYW4ga2VybmVsOiBbIDM2MjAuMzA1NzQxXSBkZXZpY2UtbWFw
cGVyOiB1ZXZlbnQ6IHZlcnNpb24gMS4wLjMKU2VwIDEzIDE1OjMyOjA5IGRlYmlhbiBrZXJuZWw6
IFsgMzYyMC4zMDU4MzJdIGRldmljZS1tYXBwZXI6IGlvY3RsOiA0LjQzLjAtaW9jdGwgKDIwMjAt
MTAtMDEpIGluaXRpYWxpc2VkOiBkbS1kZXZlbEByZWRoYXQuY29tClNlcCAxMyAxNTozMjoxMiBk
ZWJpYW4ga2VybmVsOiBbIDM2MjMuNTU1MDEyXSBTR0kgWEZTIHdpdGggQUNMcywgc2VjdXJpdHkg
YXR0cmlidXRlcywgcmVhbHRpbWUsIHF1b3RhLCBubyBkZWJ1ZyBlbmFibGVkClNlcCAxMyAxNToz
MjoxMiBkZWJpYW4ga2VybmVsOiBbIDM2MjMuNTYxNzc0XSBKRlM6IG5UeEJsb2NrID0gODE5Miwg
blR4TG9jayA9IDY1NTM2ClNlcCAxMyAxNTozMjoxMiBkZWJpYW4ga2VybmVsOiBbIDM2MjMuNTg5
ODk1XSBRTlg0IGZpbGVzeXN0ZW0gMC4yLjMgcmVnaXN0ZXJlZC4KU2VwIDEzIDE1OjMyOjEyIGRl
YmlhbiBrZXJuZWw6IFsgMzYyMy42NjM2MzZdIHJhaWQ2OiBuZW9ueDggICBnZW4oKSAxMjg5MyBN
Qi9zClNlcCAxMyAxNTozMjoxMiBkZWJpYW4ga2VybmVsOiBbIDM2MjMuNzMxNjQxXSByYWlkNjog
bmVvbng4ICAgeG9yKCkgMTAwNzcgTUIvcwpTZXAgMTMgMTU6MzI6MTIgZGViaWFuIGtlcm5lbDog
WyAzNjIzLjc5OTYzNF0gcmFpZDY6IG5lb254NCAgIGdlbigpIDEyNjAzIE1CL3MKU2VwIDEzIDE1
OjMyOjEyIGRlYmlhbiBrZXJuZWw6IFsgMzYyMy44Njc2MzRdIHJhaWQ2OiBuZW9ueDQgICB4b3Io
KSAxMDUxOCBNQi9zClNlcCAxMyAxNTozMjoxMiBkZWJpYW4ga2VybmVsOiBbIDM2MjMuOTM1NjM1
XSByYWlkNjogbmVvbngyICAgZ2VuKCkgMTIyMjYgTUIvcwpTZXAgMTMgMTU6MzI6MTIgZGViaWFu
IGtlcm5lbDogWyAzNjI0LjAwMzYzOV0gcmFpZDY6IG5lb254MiAgIHhvcigpICA5NTA4IE1CL3MK
U2VwIDEzIDE1OjMyOjEzIGRlYmlhbiBrZXJuZWw6IFsgMzYyNC4wNzE2NDRdIHJhaWQ2OiBuZW9u
eDEgICBnZW4oKSAgOTg1NyBNQi9zClNlcCAxMyAxNTozMjoxMyBkZWJpYW4ga2VybmVsOiBbIDM2
MjQuMTM5NjM4XSByYWlkNjogbmVvbngxICAgeG9yKCkgIDgzMTYgTUIvcwpTZXAgMTMgMTU6MzI6
MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjIwNzYzN10gcmFpZDY6IGludDY0eDggIGdlbigpICA1
OTU2IE1CL3MKU2VwIDEzIDE1OjMyOjEzIGRlYmlhbiBrZXJuZWw6IFsgMzYyNC4yNzU2MzVdIHJh
aWQ2OiBpbnQ2NHg4ICB4b3IoKSAgMzU1MyBNQi9zClNlcCAxMyAxNTozMjoxMyBkZWJpYW4ga2Vy
bmVsOiBbIDM2MjQuMzQzNjM2XSByYWlkNjogaW50NjR4NCAgZ2VuKCkgIDY4MzMgTUIvcwpTZXAg
MTMgMTU6MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjQxMTYzNl0gcmFpZDY6IGludDY0eDQg
IHhvcigpICAzNTc2IE1CL3MKU2VwIDEzIDE1OjMyOjEzIGRlYmlhbiBrZXJuZWw6IFsgMzYyNC40
Nzk2MzJdIHJhaWQ2OiBpbnQ2NHgyICBnZW4oKSAgNTc2OSBNQi9zClNlcCAxMyAxNTozMjoxMyBk
ZWJpYW4ga2VybmVsOiBbIDM2MjQuNTQ3NjM2XSByYWlkNjogaW50NjR4MiAgeG9yKCkgIDMxMTMg
TUIvcwpTZXAgMTMgMTU6MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjYxNTYzNF0gcmFpZDY6
IGludDY0eDEgIGdlbigpICA0NzM0IE1CL3MKU2VwIDEzIDE1OjMyOjEzIGRlYmlhbiBrZXJuZWw6
IFsgMzYyNC42ODM2MzRdIHJhaWQ2OiBpbnQ2NHgxICB4b3IoKSAgMjQ4NiBNQi9zClNlcCAxMyAx
NTozMjoxMyBkZWJpYW4ga2VybmVsOiBbIDM2MjQuNjgzNjM0XSByYWlkNjogdXNpbmcgYWxnb3Jp
dGhtIG5lb254OCBnZW4oKSAxMjg5MyBNQi9zClNlcCAxMyAxNTozMjoxMyBkZWJpYW4ga2VybmVs
OiBbIDM2MjQuNjgzNjM1XSByYWlkNjogLi4uLiB4b3IoKSAxMDA3NyBNQi9zLCBybXcgZW5hYmxl
ZApTZXAgMTMgMTU6MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjY4MzYzNl0gcmFpZDY6IHVz
aW5nIG5lb24gcmVjb3ZlcnkgYWxnb3JpdGhtClNlcCAxMyAxNTozMjoxMyBkZWJpYW4ga2VybmVs
OiBbIDM2MjQuNjg1MzUwXSB4b3I6IG1lYXN1cmluZyBzb2Z0d2FyZSBjaGVja3N1bSBzcGVlZApT
ZXAgMTMgMTU6MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjY4NjIwMV0gICAgOHJlZ3MgICAg
ICAgICAgIDogMTE2MDAgTUIvc2VjClNlcCAxMyAxNTozMjoxMyBkZWJpYW4ga2VybmVsOiBbIDM2
MjQuNjg2OTA4XSAgICAzMnJlZ3MgICAgICAgICAgOiAxMzkxNSBNQi9zZWMKU2VwIDEzIDE1OjMy
OjEzIGRlYmlhbiBrZXJuZWw6IFsgMzYyNC42ODc0MTNdICAgIGFybTY0X25lb24gICAgICA6IDE5
NTMyIE1CL3NlYwpTZXAgMTMgMTU6MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjY4NzQxNF0g
eG9yOiB1c2luZyBmdW5jdGlvbjogYXJtNjRfbmVvbiAoMTk1MzIgTUIvc2VjKQpTZXAgMTMgMTU6
MzI6MTMgZGViaWFuIGtlcm5lbDogWyAzNjI0LjcxMDIxNV0gQnRyZnMgbG9hZGVkLCBjcmMzMmM9
Y3JjMzJjLWdlbmVyaWMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEJvb3RpbmcgTGludXggb24gcGh5c2ljYWwgQ1BVIDB4MDAwMDEyMDAwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIExpbnV4IHZl
cnNpb24gNS4xMC4wLTgtYXJtNjQgKGRlYmlhbi1rZXJuZWxAbGlzdHMuZGViaWFuLm9yZykgKGdj
Yy0xMCAoRGViaWFuIDEwLjIuMS02KSAxMC4yLjEgMjAyMTAxMTAsIEdOVSBsZCAoR05VIEJpbnV0
aWxzIGZvciBEZWJpYW4pIDIuMzUuMikgIzEgU01QIERlYmlhbiA1LjEwLjQ2LTQgKDIwMjEtMDgt
MDMpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEVG
SSB2Mi43MCBieSBBbWVyaWNhbiBNZWdhdHJlbmRzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEFDUEkgMi4wPTB4ZWU5MDAwMDAgU01CSU9TIDMuMD0w
eGYxYjNmZjk4IEVTUlQ9MHhlOTRmNmY5OCBNT0t2YXI9MHhlOTRkZjAwMCBSTkc9MHhmNzY2ZTgx
OCBNRU1SRVNFUlZFPTB4ZTk2MWY2OTggClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBlZmk6IHNlZWRpbmcgZW50cm9weSBwb29sClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBzZWN1cmVib290OiBTZWN1cmUgYm9vdCBkaXNh
YmxlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZXNydDog
UmVzZXJ2aW5nIEVTUlQgc3BhY2UgZnJvbSAweDAwMDAwMDAwZTk0ZjZmOTggdG8gMHgwMDAwMDAw
MGU5NGY2ZmQwLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogRWFybHkgdGFibGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2FibGVkClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAw
MDBFRTkwMDAwMCAwMDAwMjQgKHYwMiBBbXBlcmUpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDBFRThGMDAwMCAwMDAwQTQg
KHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1JICAwMTAwMDAxMykKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZBQ1AgMHgwMDAwMDAwMEVF
OEQwMDAwIDAwMDExNCAodjA2IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5
KQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRFNE
VCAweDAwMDAwMDAwRUU4NjAwMDAgMDE5QTk1ICh2MDIgQW1wZXJlIEphZGUgICAgIDAwMDAwMDAx
IElOVEwgMjAyMDA3MTcpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBEQkcyIDB4MDAwMDAwMDBFRThFMDAwMCAwMDAwNUMgKHYwMCBBbXBlcmUgQWx0
cmEgICAgMDAwMDAwMDAgSU5UTCAyMDE5MDUwOSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEdURFQgMHgwMDAwMDAwMEVFOEMwMDAwIDAwMDExMCAo
djAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1NEVCAweDAwMDAwMDAwRUU4
QjAwMDAgMDAwMDJEICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIElOVEwgMjAxOTA1MDkp
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGSURU
IDB4MDAwMDAwMDBFRTg1MDAwMCAwMDAwOUMgKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkg
QU1JICAwMDAxMDAxMykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IFNQQ1IgMHgwMDAwMDAwMEVFODQwMDAwIDAwMDA1MCAodjAyIEFMQVNLQSBBIE0g
SSAgICAwMTA3MjAwOSBBTUkgIDAwMDUwMDBGKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogQkdSVCAweDAwMDAwMDAwRUU4MzAwMDAgMDAwMDM4ICh2
MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBNQ0ZHIDB4MDAwMDAwMDBFRTgy
MDAwMCAwMDAwQUMgKHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgQU1QLiAwMTAwMDAxMykK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IElPUlQg
MHgwMDAwMDAwMEVFODEwMDAwIDAwMDYxMCAodjAwIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBB
TVAuIDAxMDAwMDEzKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogUFBUVCAweDAwMDAwMDAwRUU3RjAwMDAgMDA2RTYwICh2MDIgQW1wZXJlIEFsdHJh
ICAgIDAwMDAwMDAwIEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTTElUIDB4MDAwMDAwMDBFRTdFMDAwMCAwMDAwMkQgKHYw
MSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQgMHgwMDAwMDAwMEVFN0Qw
MDAwIDAwMDZEMCAodjAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQVBJQyAw
eDAwMDAwMDAwRUU4MDAwMDAgMDAxOUY0ICh2MDUgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAzIEFN
SSAgMDEwMDAwMTMpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBQQ0NUIDB4MDAwMDAwMDBFRTdDMDAwMCAwMDA1NzYgKHYwMiBBbXBlcmUgQWx0cmEg
ICAgMDAwMDAwMDMgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IFdTTVQgMHgwMDAwMDAwMEVFN0IwMDAwIDAwMDAyOCAodjAx
IEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRlBEVCAweDAwMDAwMDAwRUU3QTAw
MDAgMDAwMDQ0ICh2MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDEwMDAwMTMpClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUENSOiBj
b25zb2xlOiBwbDAxMSxtbWlvMzIsMHgxMDAwMDI2MDAwMDAsMTE1MjAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAg
W21lbSAweDg4MzAwMDAwLTB4ODgzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDkwMDAwMDAw
LTB4ZmZmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDgwMDAwMDAwMDAwLTB4ODAwN2ZmZmZm
ZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBT
UkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDgwMTAwMDAwMDAwLTB4ODBmZmZmZmZmZmZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBOVU1BOiBOT0RFX0RBVEEg
W21lbSAweDgwZmY4MDg3YjAwLTB4ODBmZjgwODlmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBab25lIHJhbmdlczoKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BICAgICAgW21lbSAweDAwMDAwMDAwODgzMDAw
MDAtMHgwMDAwMDAwMGZmZmZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gICBETUEzMiAgICBlbXB0eQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gICBOb3JtYWwgICBbbWVtIDB4MDAwMDAwMDEwMDAwMDAwMC0weDAw
MDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBNb3ZhYmxlIHpvbmUgc3RhcnQgZm9yIGVhY2ggbm9kZQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRWFybHkgbWVtb3J5IG5vZGUgcmFuZ2VzClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAwMDAwODgzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5
MDAwMDAwMC0weDAwMDAwMDAwOTFmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MjAwMDAwMC0weDAw
MDAwMDAwOTNmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5NDAwMDAwMC0weDAwMDAwMDAwZWUyZTZm
ZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDBlZTJlNzAwMC0weDAwMDAwMDAwZWU3NDBmZmZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDAwMDBlZTc0MTAwMC0weDAwMDAwMDAwZWU3NDRmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0
NTAwMC0weDAwMDAwMDAwZWU3NGNmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZDAwMC0weDAwMDAw
MDAwZWU3NGRmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZTAwMC0weDAwMDAwMDAwZWU3NGZmZmZd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDBlZTc1MDAwMC0weDAwMDAwMDAwZWU5NmZmZmZdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDBlZTk3MDAwMC0weDAwMDAwMDAwZWU5N2ZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk4MDAw
MC0weDAwMDAwMDAwZjA2NGZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmMDY1MDAwMC0weDAwMDAwMDAw
Zjc2NmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmNzY3MDAwMC0weDAwMDAwMDAwZjc4NGZmZmZdClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBmNzg1MDAwMC0weDAwMDAwMDAwZjdmZGZmZmZdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBmN2ZlMDAwMC0weDAwMDAwMDAwZmZjOGVmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM4ZjAwMC0w
eDAwMDAwMDAwZmZjOGZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM5MDAwMC0weDAwMDAwMDAwZmZm
ZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDA4MDAwMDAwMDAwMC0weDAwMDAwODAwN2ZmZmZmZmZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDA4MDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJbml0bWVtIHNldHVwIG5vZGUgMCBbbWVtIDB4
MDAwMDAwMDA4ODMwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBPbiBub2RlIDAgdG90YWxwYWdlczogMTY3MTE5MzYK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6
IDcxNzIgcGFnZXMgdXNlZCBmb3IgbWVtbWFwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiAwIHBhZ2VzIHJlc2VydmVkClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiA0NTkwMDggcGFn
ZXMsIExJRk8gYmF0Y2g6NjMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdICAgTm9ybWFsIHpvbmU6IDI1Mzk1MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDE2
MjUyOTI4IHBhZ2VzLCBMSUZPIGJhdGNoOjYzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBjbWE6IFJlc2VydmVkIDY0IE1pQiBhdCAweDAwMDAwMDAwZmJjMDAw
MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IHBy
b2JpbmcgZm9yIGNvbmR1aXQgbWV0aG9kIGZyb20gQUNQSS4KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFBTQ0l2MS4xIGRldGVjdGVkIGluIGZpcm13
YXJlLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTog
VXNpbmcgc3RhbmRhcmQgUFNDSSB2MC4yIGZ1bmN0aW9uIElEcwpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogTUlHUkFURV9JTkZPX1RZUEUgbm90IHN1
cHBvcnRlZC4KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBz
Y2k6IFNNQyBDYWxsaW5nIENvbnZlbnRpb24gdjEuMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgw
IC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDAgLT4gTm9kZSAwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAgLT4gTm9kZSAwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgzMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MzAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHg0MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDUwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHg2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
NjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHg3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4ODAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDgwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4OTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhhMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4YjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eGIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4YzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweGQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhkMDEwMCAtPiBO
b2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweGUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDAw
MCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAxMDAgLT4gTm9kZSAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDEwMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MTAwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAwMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDExMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MTIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMjAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDAwMCAtPiBOb2RlIDAKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MTMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgxNDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE0MDEwMCAtPiBO
b2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgxNTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2
MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTYwMTAwIC0+IE5vZGUgMApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDE3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMDAwIC0+
IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxODAxMDAgLT4gTm9kZSAwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDE5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MTkwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAwMDAgLT4gTm9kZSAw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MWIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYjAxMDAg
LT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFjMDAwMCAtPiBOb2RlIDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MWMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgxZDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFkMDEwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgxZTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDAw
MCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWYwMTAwIC0+IE5vZGUgMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgyMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDIwMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMDAwIC0+IE5v
ZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDIyMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjIw
MTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAwMDAgLT4gTm9kZSAwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDIzMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MjQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNDAxMDAgLT4g
Tm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MjUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgy
NjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI2MDEwMCAtPiBOb2RlIDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgyNzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBwZXJjcHU6IEVtYmVkZGVkIDMzIHBhZ2VzL2NwdSBzOTUxOTIgcjgxOTIg
ZDMxNzg0IHUxMzUxNjgKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHBjcHUtYWxsb2M6IHM5NTE5MiByODE5MiBkMzE3ODQgdTEzNTE2OCBhbGxvYz0zMyo0MDk2
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9j
OiBbMF0gMDAgWzBdIDAxIFswXSAwMiBbMF0gMDMgWzBdIDA0IFswXSAwNSBbMF0gMDYgWzBdIDA3
IApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxv
YzogWzBdIDA4IFswXSAwOSBbMF0gMTAgWzBdIDExIFswXSAxMiBbMF0gMTMgWzBdIDE0IFswXSAx
NSAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxs
b2M6IFswXSAxNiBbMF0gMTcgWzBdIDE4IFswXSAxOSBbMF0gMjAgWzBdIDIxIFswXSAyMiBbMF0g
MjMgClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFs
bG9jOiBbMF0gMjQgWzBdIDI1IFswXSAyNiBbMF0gMjcgWzBdIDI4IFswXSAyOSBbMF0gMzAgWzBd
IDMxIApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1h
bGxvYzogWzBdIDMyIFswXSAzMyBbMF0gMzQgWzBdIDM1IFswXSAzNiBbMF0gMzcgWzBdIDM4IFsw
XSAzOSAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUt
YWxsb2M6IFswXSA0MCBbMF0gNDEgWzBdIDQyIFswXSA0MyBbMF0gNDQgWzBdIDQ1IFswXSA0NiBb
MF0gNDcgClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1
LWFsbG9jOiBbMF0gNDggWzBdIDQ5IFswXSA1MCBbMF0gNTEgWzBdIDUyIFswXSA1MyBbMF0gNTQg
WzBdIDU1IApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNw
dS1hbGxvYzogWzBdIDU2IFswXSA1NyBbMF0gNTggWzBdIDU5IFswXSA2MCBbMF0gNjEgWzBdIDYy
IFswXSA2MyAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBj
cHUtYWxsb2M6IFswXSA2NCBbMF0gNjUgWzBdIDY2IFswXSA2NyBbMF0gNjggWzBdIDY5IFswXSA3
MCBbMF0gNzEgClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
Y3B1LWFsbG9jOiBbMF0gNzIgWzBdIDczIFswXSA3NCBbMF0gNzUgWzBdIDc2IFswXSA3NyBbMF0g
NzggWzBdIDc5IApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEdJQyBzeXN0ZW0gcmVn
aXN0ZXIgQ1BVIGludGVyZmFjZQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogVmlydHVhbGl6YXRpb24gSG9zdCBFeHRl
bnNpb25zClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUg
ZmVhdHVyZXM6IGtlcm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiBmb3JjZWQgT04gYnkgS0FTTFIK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJl
czogZGV0ZWN0ZWQ6IEtlcm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiAoS1BUSSkKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0
ZWQ6IEhhcmR3YXJlIGRpcnR5IGJpdCBtYW5hZ2VtZW50ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBTcGVjdHJlLXY0
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVy
ZXM6IGRldGVjdGVkOiBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBhbHRlcm5hdGl2ZXM6IHBhdGNoaW5nIGtlcm5lbCBjb2Rl
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBCdWlsdCAxIHpv
bmVsaXN0cywgbW9iaWxpdHkgZ3JvdXBpbmcgb24uICBUb3RhbCBwYWdlczogMTY0NTA4MTIKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFBvbGljeSB6b25lOiBO
b3JtYWwKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEtlcm5l
bCBjb21tYW5kIGxpbmU6IEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1hcm02NCBy
b290PVVVSUQ9NmMwODMyN2MtNGJiZC00MzQxLWE0MjctYTcyYmMxNTMzODk1IHJvIHF1aWV0ClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19i
dWZfbGVuIGluZGl2aWR1YWwgbWF4IGNwdSBjb250cmlidXRpb246IDQwOTYgYnl0ZXMKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9s
ZW4gdG90YWwgY3B1X2V4dHJhIGNvbnRyaWJ1dGlvbnM6IDMyMzU4NCBieXRlcwpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbiBt
aW4gc2l6ZTogMTMxMDcyIGJ5dGVzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuOiA1MjQyODggYnl0ZXMKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogZWFybHkgbG9nIGJ1ZiBm
cmVlOiAxMTkwMTYoOTAlKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gRGVudHJ5IGNhY2hlIGhhc2ggdGFibGUgZW50cmllczogODM4ODYwOCAob3JkZXI6IDE0
LCA2NzEwODg2NCBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gSW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA0MTk0MzA0IChv
cmRlcjogMTMsIDMzNTU0NDMyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBtZW0gYXV0by1pbml0OiBzdGFjazpvZmYsIGhlYXAgYWxs
b2M6b24sIGhlYXAgZnJlZTpvZmYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIHNvZnR3YXJlIElPIFRMQjogbWFwcGVkIFttZW0gMHgwMDAwMDAwMGVhMmU3MDAw
LTB4MDAwMDAwMDBlZTJlNzAwMF0gKDY0TUIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBNZW1vcnk6IDI1NDc3ODRLLzY2ODQ3NzQ0SyBhdmFpbGFibGUgKDEx
Nzc2SyBrZXJuZWwgY29kZSwgMjQzNksgcndkYXRhLCA3MDA4SyByb2RhdGEsIDU0NDBLIGluaXQs
IDU5OEsgYnNzLCAxNTg5Mzg0SyByZXNlcnZlZCwgNjU1MzZLIGNtYS1yZXNlcnZlZCkKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJhbmRvbTogZ2V0X3JhbmRv
bV91NjQgY2FsbGVkIGZyb20gX19rbWVtX2NhY2hlX2NyZWF0ZSsweDM4LzB4NTYwIHdpdGggY3Ju
Z19pbml0PTAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNM
VUI6IEhXYWxpZ249NjQsIE9yZGVyPTAtMywgTWluT2JqZWN0cz0wLCBDUFVzPTgwLCBOb2Rlcz0x
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFs
bG9jYXRpbmcgMzg1MzUgZW50cmllcyBpbiAxNTEgcGFnZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIGZ0cmFjZTogYWxsb2NhdGVkIDE1MSBwYWdlcyB3aXRo
IDUgZ3JvdXBzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBy
Y3U6IEhpZXJhcmNoaWNhbCBSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IAlSQ1UgcmVzdHJpY3RpbmcgQ1BVcyBmcm9t
IE5SX0NQVVM9MjU2IHRvIG5yX2NwdV9pZHM9ODAuClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAJUnVkZSB2YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gCVRyYWNpbmcgdmFy
aWFudCBvZiBUYXNrcyBSQ1UgZW5hYmxlZC4KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIHJjdTogUkNVIGNhbGN1bGF0ZWQgdmFsdWUgb2Ygc2NoZWR1bGVyLWVu
bGlzdG1lbnQgZGVsYXkgaXMgMjUgamlmZmllcy4KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIHJjdTogQWRqdXN0aW5nIGdlb21ldHJ5IGZvciByY3VfZmFub3V0
X2xlYWY9MTYsIG5yX2NwdV9pZHM9ODAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIE5SX0lSUVM6IDY0LCBucl9pcnFzOiA2NCwgcHJlYWxsb2NhdGVkIGlycXM6
IDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBH
SUM6IFVzaW5nIHNwbGl0IEVPSS9EZWFjdGl2YXRlIG1vZGUKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiA2NzIgU1BJcyBpbXBsZW1lbnRlZApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDAgRXh0ZW5k
ZWQgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gR0lDdjM6IERpc3RyaWJ1dG9yIGhhcyBubyBSYW5nZSBTZWxlY3RvciBzdXBwb3J0
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMTYg
UFBJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gR0lDdjM6IENQVTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMDAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDVjMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDAgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMSAtPiBOb2RlIDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAw
IC0+IElUUyAyIC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDMgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNCAtPiBOb2Rl
IDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBY
TSAwIC0+IElUUyA1IC0+IE5vZGUgMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gU1JBVDogUFhNIDAgLT4gSVRTIDYgLT4gTm9kZSAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNyAtPiBO
b2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBb
bWVtIDB4MTAwMTAwMDQwMDAwLTB4MTAwMTAwMDVmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVk
IDgxOTIgRGV2aWNlcyBAODAwMDAyYzAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hy
IDEpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgw
MDAwMTAwMTAwMDQwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4
MDAwMDJkMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwNjAwMDAtMHgx
MDAxMDAwN2ZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBJVFNAMHgwMDAwMTAwMTAwMDYwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJm
MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9j
YXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzAwMDAwIChmbGF0LCBlc3og
MiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBJVFMgW21lbSAweDEwMDEwMDA4MDAwMC0weDEwMDEwMDA5ZmZmZl0KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAw
MDA6IGFsbG9jYXRlZCA4MTkyIERldmljZXMgQDgwMDAwMzIwMDAwIChpbmRpcmVjdCwgZXN6IDgs
IHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA4MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBD
b2xsZWN0aW9ucyBAODAwMDAzMzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAw
MTAwMGEwMDAwLTB4MTAwMTAwMGJmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBhMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2
aWNlcyBAODAwMDAzNTAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAw
MGEwMDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM2MDAw
MCAoZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwYzAwMDAtMHgxMDAxMDAwZGZm
ZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgw
MDAwMTAwMTAwMGMwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDM4MDAwMCAoaW5k
aXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCAzMjc2
OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzkwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0
Sywgc2hyIDEpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJ
VFMgW21lbSAweDEwMDEwMDBlMDAwMC0weDEwMDEwMDBmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9j
YXRlZCA4MTkyIERldmljZXMgQDgwMDAwM2IwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEss
IHNociAxKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRT
QDB4MDAwMDEwMDEwMDBlMDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9u
cyBAODAwMDAzYzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTAwMDAw
LTB4MTAwMTAwMTFmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gSVRTQDB4MDAwMDEwMDEwMDEwMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAw
MDAzZTAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBh
bGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDNmMDAwMCAoZmxhdCwg
ZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAxMjAwMDAtMHgxMDAxMDAxM2ZmZmZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAw
MTIwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDgxMDAwMCAoaW5kaXJlY3QsIGVz
eiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAxMjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1
cHQgQ29sbGVjdGlvbnMgQDgwMDAwODIwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEp
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogdXNp
bmcgTFBJIHByb3BlcnR5IHRhYmxlIEAweDAwMDAwODAwMDA4MzAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBDUFUwOiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg0MDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSSBHVERUOiBmb3VuZCAxIG1lbW9yeS1tYXBwZWQg
dGltZXIgYmxvY2socykuClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBhcmNoX3RpbWVyOiBjcDE1IGFuZCBtbWlvIHRpbWVyKHMpIHJ1bm5pbmcgYXQgMjUuMDBN
SHogKHBoeXMvcGh5cykuClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBjbG9ja3NvdXJjZTogYXJjaF9zeXNfY291bnRlcjogbWFzazogMHhmZmZmZmZmZmZmZmZm
ZiBtYXhfY3ljbGVzOiAweDVjNDA5MzliNSwgbWF4X2lkbGVfbnM6IDQ0MDc5NTIwMjY0NiBucwpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gc2NoZWRfY2xvY2s6
IDU2IGJpdHMgYXQgMjVNSHosIHJlc29sdXRpb24gNDBucywgd3JhcHMgZXZlcnkgNDM5ODA0NjUx
MTEwMG5zClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDUzXSBDb25z
b2xlOiBjb2xvdXIgZHVtbXkgZGV2aWNlIDgweDI1ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDc1XSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIwMjAwOTI1ClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMjY4XSBDYWxpYnJhdGluZyBkZWxheSBs
b29wIChza2lwcGVkKSwgdmFsdWUgY2FsY3VsYXRlZCB1c2luZyB0aW1lciBmcmVxdWVuY3kuLiA1
MC4wMCBCb2dvTUlQUyAobHBqPTEwMDAwMCkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAyNzBdIHBpZF9tYXg6IGRlZmF1bHQ6IDgxOTIwIG1pbmltdW06IDY0MApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMwM10gTFNNOiBTZWN1cml0eSBG
cmFtZXdvcmsgaW5pdGlhbGl6aW5nClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMzA4XSBZYW1hOiBkaXNhYmxlZCBieSBkZWZhdWx0OyBlbmFibGUgd2l0aCBzeXNjdGwg
a2VybmVsLnlhbWEuKgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMy
NV0gQXBwQXJtb3I6IEFwcEFybW9yIGluaXRpYWxpemVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMzI3XSBUT01PWU8gTGludXggaW5pdGlhbGl6ZWQKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAzNjhdIE1vdW50LWNhY2hlIGhhc2ggdGFi
bGUgZW50cmllczogMTMxMDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDM5Ml0gTW91bnRwb2ludC1jYWNo
ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxp
bmVhcikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEyNTFdIHJjdTog
SGllcmFyY2hpY2FsIFNSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAxMzkxXSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5
M10gUGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOTRdIFBsYXRmb3JtIE1TSTogSVRT
QDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAxMzk1XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21h
aW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5Nl0g
UGxhdGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOThdIFBsYXRmb3JtIE1TSTogSVRTQDB4
MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAxMzk5XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4g
Y3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwMF0gUGxh
dGZvcm0gTVNJOiBJVFNAMHgxMDAxMDAxMjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MDRdIFBDSS9NU0k6IElUU0AweDEwMDEwMDA0
MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMTQwNV0gUENJL01TSTogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDA3XSBQQ0kvTVNJOiBJVFNAMHgx
MDAxMDAwODAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDE0MDhdIFBDSS9NU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRl
ZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxMV0gUENJL01TSTog
SVRTQDB4MTAwMTAwMGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAxNDEzXSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWlu
IGNyZWF0ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTRdIFBD
SS9NU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxNV0gUENJL01TSTogSVRTQDB4MTAwMTAwMTIwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
NDE4XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MTldIGZzbC1tYyBNU0k6IElUU0Aw
eDEwMDEwMDA2MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMTQyMV0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBj
cmVhdGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDIyXSBmc2wt
bWMgTVNJOiBJVFNAMHgxMDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjNdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBj
MDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMTQyNF0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI1XSBmc2wtbWMgTVNJOiBJ
VFNAMHgxMDAxMDAxMDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDE0MjZdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21h
aW4gY3JlYXRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzM10g
UmVtYXBwaW5nIGFuZCBlbmFibGluZyBFRkkgc2VydmljZXMuClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAzOTA0XSBzbXA6IEJyaW5naW5nIHVwIHNlY29uZGFyeSBDUFVz
IC4uLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDQyOF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDQ0NDZdIEdJQ3YzOiBDUFUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFhMDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA3YzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDQ0NjRdIEdJQ3YzOiBDUFUxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDg1MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNDQ5Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NDUwNV0gQ1BVMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTE5OV0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDUyMTRdIEdJQ3YzOiBDUFUyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA2NDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDUyMzFdIEdJQ3YzOiBDUFUyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDg2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNTI2N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNTI3Nl0gQ1BVMjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTQwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTQ4NV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDU1MDJdIEdJQ3YzOiBDUFUzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDAw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA4NDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDU1MTldIEdJQ3YzOiBDUFUzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMDg3MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNTU1Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNTU2MV0gQ1BVMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWMwMDAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTg2
N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDU4ODFdIEdJQ3YzOiBDUFU0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEw
MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1NDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDU5MDFdIEdJQ3YzOiBDUFU0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg4MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNTkzNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNTk0NV0gQ1BVNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAw
MDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NjI3M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDYyODldIEdJQ3YzOiBDUFU1OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDE4MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3NDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDYzMDddIEdJQ3YzOiBDUFU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDg5MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNjM0Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNjM1Ml0gQ1BVNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MTgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNjY2OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDY2ODRdIEdJQ3YzOiBDUFU2OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDE2MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2YzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDY3MDNdIEdJQ3YzOiBDUFU2OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhhMDAwMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwNjczOF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNjc0OF0gQ1BVNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMTYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNzA2Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDcwODBdIEdJQ3YzOiBDUFU3OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDFlMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4YzAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDcwOTldIEdJQ3YzOiBDUFU3OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDhiMDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzEzNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNzE0NF0gQ1BVNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMWUwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNzQ3Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTgKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDc0ODRdIEdJQ3YzOiBDUFU4OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIGEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNjMDAwMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzUwM10gR0lDdjM6IENQVTg6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGMwMDAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3NTQwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA3NTUwXSBDUFU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAwYTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA3ODcxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVOQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzg4OV0gR0lDdjM6IENQVTk6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDljMDAwMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzkwN10gR0lDdjM6IENQVTk6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGQwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3OTM4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3OTQ4XSBDUFU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAyMjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA4MjYzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDgyNzZdIEdJQ3YzOiBDUFUxMDogZm91
bmQgcmVkaXN0cmlidXRvciBjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0NDAwMDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDgyOTVdIEdJQ3YzOiBDUFUxMDogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ZTAwMDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDgzMzJdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDgzNDNdIENQVTEwOiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAwYzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA4NjY2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTEK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2ODVdIEdJQ3YzOiBDUFUx
MTogZm91bmQgcmVkaXN0cmlidXRvciAyNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTQwMDAw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzA0XSBHSUN2MzogQ1BV
MTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGYwMDAw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzM4XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzQ4XSBDUFUxMTogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTA2Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTEyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDc0XSBHSUN2
MzogQ1BVMTI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
MzQwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDkzXSBHSUN2
MzogQ1BVMTI6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
OTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MTI5XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MTQwXSBDUFUxMjogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTY3OV0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTEzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5Njk3
XSBHSUN2MzogQ1BVMTM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAwMDAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDk0MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTcx
N10gR0lDdjM6IENQVTEzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDkxMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTc0
OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTc2MF0gQ1BV
MTM6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIwMDAwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAwNzBdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFUxNApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMDA4NF0gR0lDdjM6IENQVTE0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGUwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDRjMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMDEwNl0gR0lDdjM6IENQVTE0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDkyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMDE0M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDE1
NF0gQ1BVMTQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBlMDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA0NzNdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxNQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMDQ5NF0gR0lDdjM6IENQVTE1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDBhYzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTA1MTNdIEdJQ3YzOiBDUFUxNTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA5MzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTA1NDddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTA1NThdIENQVTE1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwODY5
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTA4ODBdIEdJQ3YzOiBDUFUxNjogZm91bmQgcmVkaXN0cmlidXRvciAy
MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxYzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTA5MDFdIEdJQ3YzOiBDUFUxNjogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTA5MzhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTA5NTBdIENQVTE2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAw
MjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDExMjcxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTcKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTEyODJdIEdJQ3YzOiBDUFUxNzogZm91bmQgcmVkaXN0cmli
dXRvciA0MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAyNDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzMDRdIEdJQ3YzOiBDUFUxNzogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5NTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTEzNDJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTEzNTRdIENQVTE3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAwNDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDExNjcwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTgKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTE2ODBdIEdJQ3YzOiBDUFUxODogZm91bmQgcmVk
aXN0cmlidXRvciAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE0MDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTcwMl0gR0lDdjM6IENQVTE4OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk2MDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMTc0MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMTc1Ml0gQ1BVMTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDAwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTIwNzBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxOQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjA4Ml0gR0lDdjM6IENQVTE5OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJjMDAwMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjEwNF0gR0lDdjM6IENQVTE5OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk3MDAwMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE0MF0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjE1Ml0gQ1BVMTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDA2MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTI0NzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjQ4Nl0gR0lDdjM6IENQVTIwOiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDEzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2MDAwMDAKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1MDddIEdJQ3YzOiBDUFUyMDog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ODAwMDAKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1NDFdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1NTRdIENQVTIwOiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAxMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODY5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
MjEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI4ODVdIEdJQ3YzOiBD
UFUyMTogZm91bmQgcmVkaXN0cmlidXRvciAxYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODAw
MDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTA2XSBHSUN2Mzog
Q1BVMjE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTkw
MDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTM4XSBhcmNoX3Rp
bWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTUwXSBDUFUyMTogQm9vdGVk
IHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI2NF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTIyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMjc5XSBH
SUN2MzogQ1BVMjI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDY4MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI5OV0g
R0lDdjM6IENQVTIyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMDlhMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzMzNF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzM0Nl0gQ1BVMjI6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE1MDAwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2NjNdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFUyMwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
MzY4MV0gR0lDdjM6IENQVTIzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA4ODAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTM3MDFdIEdJQ3YzOiBDUFUyMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDA5YjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTM3MzVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM3NDdd
IENQVTIzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDY3XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVMjQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQwODJdIEdJQ3YzOiBDUFUyNDogZm91bmQgcmVkaXN0cmlidXRvciAxMTAwMDAgcmVn
aW9uIDA6MHgwMDAwMTAwMTAwNTgwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE0MTAyXSBHSUN2MzogQ1BVMjQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0
YWJsZSBAMHgwMDAwMDgwMDAwOWMwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE0MTM3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE0MTUwXSBDUFUyNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQ2N10g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI1ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE0NDgzXSBHSUN2MzogQ1BVMjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc4MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNDUwNF0gR0lDdjM6IENQVTI1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlkMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNDUzOF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNDU1MF0gQ1BVMjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTQ4NzFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNgpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNDg4N10gR0lDdjM6IENQVTI2OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDE3MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3MDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ5MDldIEdJQ3YzOiBDUFUyNjogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ZTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ5NDRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTQ5NTddIENQVTI2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxNzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1MjcwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjcKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTUyODhdIEdJQ3YzOiBDUFUyNzogZm91bmQgcmVk
aXN0cmlidXRvciAxZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTAwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MzA5XSBHSUN2MzogQ1BVMjc6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWYwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MzQ0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MzU4XSBDUFUyNzogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMWYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxNTY2OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI4ClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NjgyXSBHSUN2MzogQ1BVMjg6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDAwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NzA0XSBHSUN2MzogQ1BVMjg6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTAwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NzQxXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NzU0XSBDUFUyODogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMGIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNjA3MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI5
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2MDg5XSBHSUN2MzogQ1BV
Mjk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEwMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjExMF0gR0lDdjM6IENQ
VTI5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGExMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjE0MV0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjE1NV0gQ1BVMjk6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY0NjhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUzMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQ4Ml0gR0lD
djM6IENQVTMwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDQ4MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjUwM10gR0lD
djM6IENQVTMwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGEyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjUzOV0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjU1Ml0gQ1BVMzA6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBkMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4NzFdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFUzMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjg5
MF0gR0lDdjM6IENQVTMxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI1MDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDBhODAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY5
MTJdIEdJQ3YzOiBDUFUzMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBhMzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY5
NDRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY5NThdIENQ
VTMxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNTAwMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3MjcwXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVMzIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTcyODNdIEdJQ3YzOiBDUFUzMjogZm91bmQgcmVkaXN0cmlidXRvciA5MDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDAzODAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTczMDZdIEdJQ3YzOiBDUFUzMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBhNDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTczNDFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcz
NTVdIENQVTMyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwOTAwMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NjczXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMzMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTc2OTFdIEdJQ3YzOiBDUFUzMzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAwMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwOTgwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE3NzE0XSBHSUN2MzogQ1BVMzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTUwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE3NzQ4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE3NzYyXSBDUFUzMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMDAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODA2
OF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE4MDgyXSBHSUN2MzogQ1BVMzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
ZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE4MTA2XSBHSUN2MzogQ1BVMzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTYwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE4MTQzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE4MTU3XSBDUFUzNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MGYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODQ3NV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM1ClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4NDk2XSBHSUN2MzogQ1BVMzU6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgMjcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGIwMDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODUxOV0gR0lDdjM6IENQVTM1OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE3MDAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODU1NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxODU2OF0gQ1BVMzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDI3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTg4NzJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNgpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODg4Ml0gR0lDdjM6IENQVTM2OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIwMDAwMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODkwN10gR0lDdjM6IENQVTM2OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE4MDAwMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODk0NV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODk1OV0gQ1BVMzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDAzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTkyNzFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNwpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTI4M10gR0lDdjM6IENQVTM3OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI4MDAwMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTMwN10gR0lDdjM6IENQVTM3OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE5MDAwMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTM0NV0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTM2MF0gQ1BVMzc6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDA1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTk2OThdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUz
OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTcwOV0gR0lDdjM6IENQ
VTM4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE4MDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTczNF0gR0lDdjM6IENQ
VTM4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFhMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTc3Ml0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTc4N10gQ1BVMzg6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAxMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjAwNjldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUzOQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDA4MV0gR0lD
djM6IENQVTM5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDMwMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDEwNl0gR0lD
djM6IENQVTM5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGFiMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDE0Ml0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDE1OF0gQ1BVMzk6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0NzBdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDQ4
NV0gR0lDdjM6IENQVTQwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA1ZTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1
MDhdIEdJQ3YzOiBDUFU0MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBhYzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1
NDFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1NTZdIENQ
VTQwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMjAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODcxXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVNDEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjA4ODddIEdJQ3YzOiBDUFU0MTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAxMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwN2UwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIwOTA5XSBHSUN2MzogQ1BVNDE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwYWQwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIwOTQyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIw
OTU3XSBDUFU0MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMTAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTI2N10gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIxMjgzXSBHSUN2MzogQ1BVNDI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQwMTAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMTMwNl0gR0lDdjM6IENQVTQyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGFlMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMTM0MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMTM1Nl0gQ1BVNDI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDEw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2
NzBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MwpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMTY4N10gR0lDdjM6IENQVTQzOiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDFjMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4NjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjE3MTFdIEdJQ3YzOiBDUFU0MzogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhZjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjE3NDVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjE3NTldIENQVTQzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAxYzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIyMDcyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDQKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjIwODZdIEdJQ3YzOiBDUFU0NDogZm91bmQgcmVkaXN0
cmlidXRvciAxMDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTYwMDAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMTExXSBHSUN2MzogQ1BVNDQ6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjAwMDAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMTQ2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIyMTYyXSBDUFU0NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMTAwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMjQ3MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ1ClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyNDg3XSBHSUN2MzogQ1BVNDU6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMTgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc2MDAwMApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjUxMV0gR0lDdjM6IENQVTQ1OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIxMDAwMApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjU0M10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjU1N10gQ1BVNDU6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDE4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjI4NzNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjg4OV0gR0lDdjM6IENQVTQ2
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2ZTAwMDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5MTRdIEdJQ3YzOiBDUFU0
NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMjAwMDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5NTBdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5NjZdIENQVTQ2OiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjcxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNDcKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMyODldIEdJQ3Yz
OiBDUFU0NzogZm91bmQgcmVkaXN0cmlidXRvciAxZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
OGUwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzEzXSBHSUN2
MzogQ1BVNDc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YjMwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzQ4XSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzYzXSBDUFU0NzogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMzY3MF0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTQ4ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjgz
XSBHSUN2MzogQ1BVNDg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYTAxMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwM2UwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzA4
XSBHSUN2MzogQ1BVNDg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYjQwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzQ1
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzYxXSBDUFU0
ODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGEwMTAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDA3M10gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTQ5ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI0MDkxXSBHSUN2MzogQ1BVNDk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMTAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDllMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNDExN10gR0lDdjM6IENQVTQ5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMGI1MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNDE0OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDE2
NV0gQ1BVNDk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIyMDEwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ0NzFdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNDQ4NF0gR0lDdjM6IENQVTUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMTAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDQ2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNDUwOV0gR0lDdjM6IENQVTUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGI2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNDU0NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNDU2MF0gQ1BVNTA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDEwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4NzNd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1MQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNDg5Ml0gR0lDdjM6IENQVTUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0
MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhNjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjQ5MTddIEdJQ3YzOiBDUFU1MTogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiNzAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjQ5NTBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjQ5NjZdIENQVTUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAy
NDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI1MjY5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTIKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjUyODFdIEdJQ3YzOiBDUFU1MjogZm91bmQgcmVkaXN0cmli
dXRvciA4MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzNjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjUzMDddIEdJQ3YzOiBDUFU1MjogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiODAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjUzNDVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjUzNjJdIENQVTUyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAwODAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI1NjgyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTMKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjU3MDBdIEdJQ3YzOiBDUFU1MzogZm91bmQgcmVk
aXN0cmlidXRvciAyMDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTYwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NzI2XSBHSUN2MzogQ1BVNTM6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjkwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NzYwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1Nzc3XSBDUFU1MzogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMjAwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyNjA3MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU0ClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDg0XSBHSUN2MzogQ1BVNTQ6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGUwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MTExXSBHSUN2MzogQ1BVNTQ6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYmEwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MTQ4XSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MTY1XSBDUFU1NDogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMGUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjQ3OF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU1
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NDk3XSBHSUN2MzogQ1BV
NTU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFlMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjUyM10gR0lDdjM6IENQ
VTU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJiMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjU1OV0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjU3N10gQ1BVNTU6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI2MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjY4NzNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFU1NgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg4M10gR0lD
djM6IENQVTU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDFlMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjkxMV0gR0lD
djM6IENQVTU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGJjMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjk0OV0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjk2OF0gQ1BVNTY6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjcyNzZdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU1NwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzI4
N10gR0lDdjM6IENQVTU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDQwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDI2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMx
Nl0gR0lDdjM6IENQVTU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGJkMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzM1
M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzM3MV0gQ1BV
NTc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA0MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2NjZdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU1OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNzY3Nl0gR0lDdjM6IENQVTU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDAxNjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
Mjc3MDNdIEdJQ3YzOiBDUFU1ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBiZTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
Mjc3NDFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3NjBd
IENQVTU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMDAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MDY2XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNTkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjgwNzhdIEdJQ3YzOiBDUFU1OTogZm91bmQgcmVkaXN0cmlidXRvciA2MDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAyZTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjgxMDVdIEdJQ3YzOiBDUFU1OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBiZjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjgxNDFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjgxNjBdIENQVTU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNjAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDY1XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjg0ODBdIEdJQ3YzOiBDUFU2MDogZm91bmQgcmVkaXN0cmlidXRvciAxMzAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNjIwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI4NTA2XSBHSUN2MzogQ1BVNjA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI4NTQzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI4NTYxXSBDUFU2MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
ODg3MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYxClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI4ODg2XSBHSUN2MzogQ1BVNjE6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgMWIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDgyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyODkxMl0gR0lDdjM6IENQVTYxOiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGMxMDAwMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyODk0Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyODk2M10gQ1BVNjE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDFiMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjkyNjhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MgpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyOTI4NF0gR0lDdjM6IENQVTYyOiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDE1MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2YTAwMDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkzMTBdIEdJQ3YzOiBDUFU2MjogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMjAwMDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkzNDRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjkzNjJdIENQVTYyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAxNTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI5Njk1XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjMKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjk3MTNdIEdJQ3YzOiBDUFU2MzogZm91
bmQgcmVkaXN0cmlidXRvciAxZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGEwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NzM5XSBHSUN2MzogQ1BVNjM6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzMwMDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NzcyXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5Nzg5XSBDUFU2MzogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMWQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAzMDA3MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY0
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwMDg1XSBHSUN2MzogQ1BV
NjQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVhMDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDExM10gR0lDdjM6IENQ
VTY0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM0MDAw
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDE0N10gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDE2NV0gQ1BVNjQ6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDExMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0NzhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFU2NQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDQ5NV0gR0lD
djM6IENQVTY1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE5MDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA3YTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1MjJdIEdJ
Q3YzOiBDUFU2NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBjNTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1NTZdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1NzJdIENQVTY1OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODczXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNjYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA4
ODldIEdJQ3YzOiBDUFU2NjogZm91bmQgcmVkaXN0cmlidXRvciAxNzAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNzIwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMw
OTE3XSBHSUN2MzogQ1BVNjY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYzYwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMw
OTUyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwOTY4XSBD
UFU2NjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTcwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI2Nl0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTY3ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMxMjg0XSBHSUN2MzogQ1BVNjc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWYwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDkyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMTMxMl0gR0lDdjM6IENQVTY3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGM3MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMTM0N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MTM2NF0gQ1BVNjc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFmMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE2NjddIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMTY4MF0gR0lDdjM6IENQVTY4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGIwMTAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMTcwOF0gR0lDdjM6IENQVTY4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGM4MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMTc0NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMTc2Ml0gQ1BVNjg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBiMDEw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIw
NjldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2OQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMjA4N10gR0lDdjM6IENQVTY5OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDIzMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhMjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzIxMTVdIEdJQ3YzOiBDUFU2OTogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjOTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzIxNDhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzIxNjVdIENQVTY5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAyMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDMyNDcxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI0ODRdIEdJQ3YzOiBDUFU3MDogZm91bmQgcmVkaXN0
cmlidXRvciBkMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0YTAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI1MTNdIEdJQ3YzOiBDUFU3MDogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjYTAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI1NDldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzI1NjddIENQVTcwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAwZDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMyODc3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzEKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzI4OTddIEdJQ3YzOiBDUFU3MTogZm91bmQg
cmVkaXN0cmlidXRvciAyNTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYWEwMDAwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyOTI1XSBHSUN2MzogQ1BVNzE6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2IwMDAwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyOTU2XSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyOTczXSBDUFU3MTogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMjUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAzMzI3N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcyClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjg5XSBHSUN2MzogQ1BVNzI6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2EwMDAwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMzE5XSBHSUN2MzogQ1BVNzI6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2MwMDAwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMzU2XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMzc0XSBDUFU3MjogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMDkwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzY2OF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTczClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNjg3XSBHSUN2Mzog
Q1BVNzM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDlh
MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzcxNl0gR0lDdjM6
IENQVTczOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNk
MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzc0Nl0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzc2NF0gQ1BVNzM6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQwNjhdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU3NApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDA4Ml0g
R0lDdjM6IENQVTc0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDUyMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDExMV0g
R0lDdjM6IENQVTc0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGNlMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE0OF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE2NV0gQ1BVNzQ6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBmMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0ODBdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU3NQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
NDUwMV0gR0lDdjM6IENQVTc1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI3MDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDBiMjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzQ1MjldIEdJQ3YzOiBDUFU3NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBjZjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzQ1NjNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1Nzld
IENQVTc1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNzAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0ODc2XSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNzYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzQ4ODddIEdJQ3YzOiBDUFU3NjogZm91bmQgcmVkaXN0cmlidXRvciAzMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAyMjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzQ5MTddIEdJQ3YzOiBDUFU3NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBkMDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzQ5NTVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzQ5NzNdIENQVTc2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMzAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MjY0XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzcKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzUyNzVdIEdJQ3YzOiBDUFU3NzogZm91bmQgcmVkaXN0cmlidXRvciA1MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDAyYTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzUzMDVdIEdJQ3YzOiBDUFU3NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzUzNDNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzUzNjJdIENQVTc3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNTAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1
NjcyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzgKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzU2ODJdIEdJQ3YzOiBDUFU3ODogZm91bmQgcmVkaXN0cmlidXRv
ciAxMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxYTAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzU3MTNdIEdJQ3YzOiBDUFU3ODogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzU3NTFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzU3NzBdIENQVTc4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAwMTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2MDc0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzkKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYwODZdIEdJQ3YzOiBDUFU3OTogZm91bmQgcmVkaXN0
cmlidXRvciA3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAzMjAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYxMTZdIEdJQ3YzOiBDUFU3OTogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMzAwMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYxNTVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzYxNzRdIENQVTc5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAwNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDM2MjM1XSBzbXA6IEJyb3VnaHQgdXAgMSBub2RlLCA4MCBDUFVzClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MjU3XSBTTVA6IFRvdGFsIG9mIDgwIHBy
b2Nlc3NvcnMgYWN0aXZhdGVkLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzNjI1OF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogUHJpdmlsZWdlZCBBY2Nlc3MgTmV2ZXIK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyNTldIENQVSBmZWF0dXJl
czogZGV0ZWN0ZWQ6IExTRSBhdG9taWMgaW5zdHJ1Y3Rpb25zClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM2MjU5XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBVc2VyIEFj
Y2VzcyBPdmVycmlkZQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjI2
MF0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogMzItYml0IEVMMCBTdXBwb3J0ClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MjYxXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVk
OiBDb21tb24gbm90IFByaXZhdGUgdHJhbnNsYXRpb25zClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM2MjYxXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hl
IGNsZWFuIHRvIFBvaW50IG9mIFBlcnNpc3RlbmNlClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2MjYyXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBSQVMgRXh0ZW5zaW9u
IFN1cHBvcnQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyNjNdIENQ
VSBmZWF0dXJlczogZGV0ZWN0ZWQ6IERhdGEgY2FjaGUgY2xlYW4gdG8gdGhlIFBvVSBub3QgcmVx
dWlyZWQgZm9yIEkvRCBjb2hlcmVuY2UKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzYyNjNdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IENSQzMyIGluc3RydWN0aW9ucwpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjI2NF0gQ1BVIGZlYXR1cmVz
OiBkZXRlY3RlZDogU3BlY3VsYXRpdmUgU3RvcmUgQnlwYXNzaW5nIFNhZmUgKFNTQlMpClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDU4NjY1XSBDUFU6IEFsbCBDUFUocykg
c3RhcnRlZCBhdCBFTDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzQx
MzZdIG5vZGUgMCBkZWZlcnJlZCBwYWdlcyBpbml0aWFsaXNlZCBpbiA1MTJtcwpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NDI4NV0gZGV2dG1wZnM6IGluaXRpYWxpemVk
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc0ODczXSBSZWdpc3RlcmVk
IGNwMTVfYmFycmllciBlbXVsYXRpb24gaGFuZGxlcgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU3NDg5NV0gUmVnaXN0ZXJlZCBzZXRlbmQgZW11bGF0aW9uIGhhbmRsZXIK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzQ4OTddIEtBU0xSIGVuYWJs
ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzQ5NTRdIGNsb2Nrc291
cmNlOiBqaWZmaWVzOiBtYXNrOiAweGZmZmZmZmZmIG1heF9jeWNsZXM6IDB4ZmZmZmZmZmYsIG1h
eF9pZGxlX25zOiA3NjQ1MDQxNzg1MTAwMDAwIG5zClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTc1MDA2XSBmdXRleCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRl
cjogOSwgMjA5NzE1MiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU3NTQ0Ml0gcGluY3RybCBjb3JlOiBpbml0aWFsaXplZCBwaW5jdHJsIHN1YnN5
c3RlbQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NTYwMF0gU01CSU9T
IDMuMy4wIHByZXNlbnQuClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc1
NjAzXSBETUk6IEdJR0FCWVRFIFIxNTItUDMwLTAwL01QMzItQVIxLTAwLCBCSU9TIEYxMyAoU0NQ
OiAxLjUuMjAyMTA0MjYpIDA1LzIxLzIwMjEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC41NzU4MzRdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTYKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzY0MjJdIERNQTogcHJlYWxsb2NhdGVk
IDQwOTYgS2lCIEdGUF9LRVJORUwgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc2NTQ4XSBETUE6IHByZWFsbG9jYXRlZCA0
MDk2IEtpQiBHRlBfS0VSTkVMfEdGUF9ETUEgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc2NjczXSBETUE6IHByZWFsbG9j
YXRlZCA0MDk2IEtpQiBHRlBfS0VSTkVMfEdGUF9ETUEzMiBwb29sIGZvciBhdG9taWMgYWxsb2Nh
dGlvbnMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzY2ODBdIGF1ZGl0
OiBpbml0aWFsaXppbmcgbmV0bGluayBzdWJzeXMgKGRpc2FibGVkKQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NjcyMV0gYXVkaXQ6IHR5cGU9MjAwMCBhdWRpdCgwLjU3
NjoxKTogc3RhdGU9aW5pdGlhbGl6ZWQgYXVkaXRfZW5hYmxlZD0wIHJlcz0xClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc2ODA3XSB0aGVybWFsX3N5czogUmVnaXN0ZXJl
ZCB0aGVybWFsIGdvdmVybm9yICdmYWlyX3NoYXJlJwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU3NjgwOF0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zl
cm5vciAnYmFuZ19iYW5nJwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3
NjgwOF0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnc3RlcF93aXNl
JwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NjgwOV0gdGhlcm1hbF9z
eXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAndXNlcl9zcGFjZScKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzY4MTBdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVk
IHRoZXJtYWwgZ292ZXJub3IgJ3Bvd2VyX2FsbG9jYXRvcicKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC41NzY4NjhdIGNwdWlkbGU6IHVzaW5nIGdvdmVybm9yIGxhZGRlcgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3Njg3NV0gY3B1aWRsZTogdXNp
bmcgZ292ZXJub3IgbWVudQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3
Njk2OV0gRGV0ZWN0ZWQgMTUgUENDIFN1YnNwYWNlcwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU3Njk4NF0gUmVnaXN0ZXJpbmcgUENDIGRyaXZlciBhcyBNYWlsYm94IGNv
bnRyb2xsZXIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NzcwMDNdIGh3
LWJyZWFrcG9pbnQ6IGZvdW5kIDYgYnJlYWtwb2ludCBhbmQgNCB3YXRjaHBvaW50IHJlZ2lzdGVy
cy4KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41Nzc0MDJdIEFTSUQgYWxs
b2NhdG9yIGluaXRpYWxpc2VkIHdpdGggMzI3NjggZW50cmllcwpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjU3NzQyNV0gQUNQSTogYnVzIHR5cGUgUENJIHJlZ2lzdGVyZWQK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41Nzc0MjZdIGFjcGlwaHA6IEFD
UEkgSG90IFBsdWcgUENJIENvbnRyb2xsZXIgRHJpdmVyIHZlcnNpb246IDAuNQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3NzQ2NF0gU2VyaWFsOiBBTUJBIFBMMDExIFVB
UlQgZHJpdmVyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc5ODA0XSBI
dWdlVExCIHJlZ2lzdGVyZWQgMS4wMCBHaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFn
ZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41Nzk4MDVdIEh1Z2VUTEIg
cmVnaXN0ZXJlZCAzMi4wIE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjU3OTgwNl0gSHVnZVRMQiByZWdpc3Rl
cmVkIDIuMDAgTWlCIHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTc5ODA3XSBIdWdlVExCIHJlZ2lzdGVyZWQgNjQu
MCBLaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDcyODldIEFDUEk6IEFkZGVkIF9PU0koTW9kdWxlIERldmlj
ZSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcyOTBdIEFDUEk6IEFk
ZGVkIF9PU0koUHJvY2Vzc29yIERldmljZSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDcyOTBdIEFDUEk6IEFkZGVkIF9PU0koMy4wIF9TQ1AgRXh0ZW5zaW9ucykKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcyOTFdIEFDUEk6IEFkZGVkIF9P
U0koUHJvY2Vzc29yIEFnZ3JlZ2F0b3IgRGV2aWNlKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwNzI5Ml0gQUNQSTogQWRkZWQgX09TSShMaW51eC1EZWxsLVZpZGVvKQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzI5NF0gQUNQSTogQWRkZWQg
X09TSShMaW51eC1MZW5vdm8tTlYtSERNSS1BdWRpbykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDcyOTVdIEFDUEk6IEFkZGVkIF9PU0koTGludXgtSFBJLUh5YnJpZC1H
cmFwaGljcykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQyMjVdIEFD
UEk6IDIgQUNQSSBBTUwgdGFibGVzIHN1Y2Nlc3NmdWxseSBhY3F1aXJlZCBhbmQgbG9hZGVkClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NTcxXSBBQ1BJOiBJbnRlcnBy
ZXRlciBlbmFibGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NTcy
XSBBQ1BJOiBVc2luZyBHSUMgZm9yIGludGVycnVwdCByb3V0aW5nClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NTc5XSBBQ1BJOiBNQ0ZHIHRhYmxlIGRldGVjdGVkLCA4
IGVudHJpZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQyOTddIEFS
TUgwMDExOjAwOiB0dHlBTUEwIGF0IE1NSU8gMHgxMDAwMDI2MDAwMDAgKGlycSA9IDM2LCBiYXNl
X2JhdWQgPSAwKSBpcyBhIFNCU0EKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjQzNDJdIHByaW50azogY29uc29sZSBbdHR5QU1BMF0gZW5hYmxlZApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNTQ0M10gQVJNSDAwMTE6MDE6IHR0eUFNQTEgYXQg
TU1JTyAweDEwMDAwMjYyMDAwMCAoaXJxID0gMzcsIGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNjQ5N10gQUNQSTogUENJIFJv
b3QgQnJpZGdlIFtQQ0kwXSAoZG9tYWluIDAwMGMgW2J1cyAwMC1mZl0pClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NTAzXSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9T
IHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1U
eXBlM10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY1NzFdIGFjcGkg
UE5QMEEwODowMDogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcg
U0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjY2MzBdIGFjcGkgUE5QMEEwODowMDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJ
ZUNhcGFiaWxpdHldClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI2NjM0
XSBhY3BpIFBOUDBBMDg6MDA6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAw
MC0weDMzZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmYmVjODk5YzVlYTU4
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3MjU5XSBhY3BpIFBOUDBB
MDg6MDA6IEVDQU0gYXJlYSBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmXSByZXNl
cnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI3MjY3XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMz
ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI3MzMyXSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMGM6MDAKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjczMzNdIHBjaV9idXMgMDAwYzowMDogcm9v
dCBidXMgcmVzb3VyY2UgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzMzNF0gcGNpX2J1cyAwMDBjOjAwOiBy
b290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRv
d10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjczMzVdIHBjaV9idXMg
MDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjczNTVdIHBjaSAwMDBjOjAwOjAwLjA6IFsxZGVmOmUxMDBd
IHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjc0MDNdIHBjaSAwMDBjOjAwOjAxLjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3Mg
MHgwNjA0MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc0MTFdIHBj
aSAwMDBjOjAwOjAxLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43Mjc0MjZdIHBjaSAwMDBjOjAwOjAxLjA6IHN1cHBvcnRzIEQx
IEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NDI3XSBwY2kgMDAw
YzowMDowMS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTAyXSBhY3BpcGhwOiBTbG90IFsxXSByZWdpc3Rl
cmVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTE1XSBwY2kgMDAw
YzowMTowMC4wOiBbMTAwMDowMDk3XSB0eXBlIDAwIGNsYXNzIDB4MDEwNzAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTI0XSBwY2kgMDAwYzowMTowMC4wOiByZWcg
MHgxMDogW2lvICAweDRmZmY4MDAwLTB4NGZmZjgwZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI3NTMyXSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxNDogW21lbSAw
eDQwMTQwMDAwLTB4NDAxNGZmZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI3NTM5XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxYzogW21lbSAweDQwMTAw
MDAwLTB4NDAxM2ZmZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI3NTQ5XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDQwMDAwMDAwLTB4
NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc1
NTNdIHBjaSAwMDBjOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc2NDhdIHBjaSAwMDBjOjAxOjAwLjA6IHN1cHBv
cnRzIEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NzIwXSBw
Y2lfYnVzIDAwMGM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI3NzIzXSBwY2kgMDAwYzowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0g
MHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIw
MDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI3NzI2XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg0MDAw
MDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
NzcyOF0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAw
MDAwLTB4MzAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI3NzI5XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyNzczMF0gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NzMx
XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzczMl0gcGNpIDAwMGM6
MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NzM0XSBwY2kgMDAwYzowMTowMC4w
OiBCQVIgNjogYXNzaWduZWQgW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc3MzVdIHBjaSAwMDBjOjAxOjAwLjA6
IEJBUiAzOiBhc3NpZ25lZCBbbWVtIDB4NDAxMDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc3NDBdIHBjaSAwMDBjOjAxOjAwLjA6
IEJBUiAxOiBhc3NpZ25lZCBbbWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjc3NDVdIHBjaSAwMDBjOjAxOjAwLjA6
IEJBUiAwOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MDEwMF0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43Mjc3NDZdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDAxMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI3NzQ3XSBwY2kgMDAwYzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NzQ5XSBwY2kgMDAw
YzowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDQwMDAwMDAwLTB4NDAxZmZmZmZdClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NzUwXSBwY2kgMDAwYzowMDow
MS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMDAwMDAwMDAwMC0weDMwMDAwMDFmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzc1MV0g
cGNpX2J1cyAwMDBjOjAwOiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2VzIGFyZSB1bmFzc2lnbmVk
LCB0cnkgYm9vdGluZyB3aXRoIHBjaT1yZWFsbG9jClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI3NzUzXSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNlIDQgW21lbSAweDQw
MDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyNzc1NF0gcGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA1IFttZW0gMHgzMDAwMDAw
MDAwMDAtMHgzM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyNzc1NV0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAxIFttZW0gMHg0MDAw
MDAwMC0weDQwMWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
Nzc1Nl0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgz
MDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjc4NjJdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJMV0gKGRvbWFpbiAwMDBkIFti
dXMgMDAtZmZdKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNzg2Nl0g
YWNwaSBQTlAwQTA4OjAxOiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBD
bG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI3OTMwXSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IHBsYXRmb3JtIGRvZXMg
bm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3OTg4XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6
IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNzk5MF0gYWNwaSBQTlAwQTA4OjAxOiBNQ0ZHIHF1aXJrOiBF
Q0FNIGF0IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZm
XSB3aXRoIDB4ZmZmZmJlYzg5OWM1ZWE1OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyODYxOV0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGFyZWEgW21lbSAweDM3ZmZmMDAw
MDAwMC0weDM3ZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODYyNl0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGF0
IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODY4Nl0gUENJIGhvc3QgYnJpZGdl
IHRvIGJ1cyAwMDBkOjAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4
Njg3XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHg1MDAwMDAwMC0w
eDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjg2ODhdIHBjaV9idXMgMDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDM0MDAwMDAw
MDAwMC0weDM3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI4Njg5XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMg
MDAtZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4NzA2XSBwY2kg
MDAwZDowMDowMC4wOiBbMWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4NzQ1XSBwY2kgMDAwZDowMDowMS4wOiBb
MWRlZjplMTAxXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI4NzUzXSBwY2kgMDAwZDowMDowMS4wOiBlbmFibGluZyBFeHRlbmRl
ZCBUYWdzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4NzY4XSBwY2kg
MDAwZDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyODc2OV0gcGNpIDAwMGQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODgyMF0g
cGNpIDAwMGQ6MDA6MDIuMDogWzFkZWY6ZTEwMl0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODg0MF0gcGNpIDAwMGQ6MDA6MDIu
MDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjg4NDFdIHBjaSAwMDBkOjAwOjAyLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg4ODhdIHBjaSAwMDBkOjAw
OjAzLjA6IFsxZGVmOmUxMDNdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MDhdIHBjaSAwMDBkOjAwOjAzLjA6IHN1cHBvcnRz
IEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTA5XSBwY2kg
MDAwZDowMDowMy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTU0XSBwY2kgMDAwZDowMDowNC4wOiBbMWRl
ZjplMTA0XSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI4OTc1XSBwY2kgMDAwZDowMDowNC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODk3Nl0gcGNpIDAwMGQ6MDA6MDQu
MDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyOTA0NF0gYWNwaXBocDogU2xvdCBbMS0yXSByZWdpc3RlcmVkClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5MDYwXSBwY2kgMDAwZDowMTow
MC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5MDcxXSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgxMDog
W21lbSAweDUwMzEwMDAwLTB4NTAzMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5MDk1XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgzMDogW21lbSAw
eDUwMzAwMDAwLTB4NTAzMGZmZmYgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjkwOTldIHBjaSAwMDBkOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjkyNDhdIGFjcGlwaHA6IFNs
b3QgWzJdIHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjkyNjNdIHBjaSAwMDBkOjAyOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4
MDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjkyNzRdIHBjaSAwMDBk
OjAyOjAwLjA6IHJlZyAweDEwOiBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjkyOTVdIHBjaSAwMDBkOjAyOjAw
LjA6IHJlZyAweDMwOiBbbWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTI5OF0gcGNpIDAwMGQ6MDI6MDAuMDogZW5h
YmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyOTQ0MF0gYWNwaXBocDogU2xvdCBbM10gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyOTQ1Nl0gcGNpIDAwMGQ6MDM6MDAuMDogWzgwODY6MGE1NF0g
dHlwZSAwMCBjbGFzcyAweDAxMDgwMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyOTQ2Nl0gcGNpIDAwMGQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHg1MDExMDAwMC0w
eDUwMTEzZmZmIDY0Yml0XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
OTQ4OF0gcGNpIDAwMGQ6MDM6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDEwMDAwMC0weDUwMTBm
ZmZmIHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NDkxXSBw
Y2kgMDAwZDowMzowMC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjMzXSBhY3BpcGhwOiBTbG90IFs0XSByZWdpc3RlcmVk
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjQ4XSBwY2kgMDAwZDow
NDowMC4wOiBbODA4NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NjU4XSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgx
MDogW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI5NjgwXSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgzMDogW21l
bSAweDUwMDAwMDAwLTB4NTAwMGZmZmYgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mjk2ODNdIHBjaSAwMDBkOjA0OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRh
Z3MKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MDZdIHBjaV9idXMg
MDAwZDowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjk4MDhdIHBjaSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAt
MHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODA5XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXpl
IDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5ODExXSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEw
MDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEw
MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgxMl0gcGNpIDAw
MGQ6MDA6MDIuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJd
IGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4
MTNdIHBjaSAwMDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAx
MDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MTRdIHBjaSAw
MDBkOjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRv
IFtidXMgMDJdIGFkZF9zaXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODE1XSBwY2kgMDAwZDowMDowMy4wOiBicmlkZ2Ug
d2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgxNl0gcGNpIDAwMGQ6MDA6MDMu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0
byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgxN10gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUg
MTAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjk4MThdIHBjaSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAt
MHgwZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODE5XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDRdIGFkZF9zaXpl
IDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5ODIxXSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEw
MDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEw
MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgyNV0gcGNpIDAw
MGQ6MDA6MDEuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0K
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MjZdIHBjaSAwMDBkOjAw
OjAxLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgy
N10gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1
MDNmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MjhdIHBj
aSAwMDBkOjAwOjAyLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0
MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyOTgyOV0gcGNpIDAwMGQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA0
MDAwMDAtMHg1MDVmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjk4MzFdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDQw
MDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyOTgzMV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE0OiBhc3NpZ25lZCBb
bWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43Mjk4MzNdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAw
eDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgzM10gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBu
byBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mjk4MzRdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFz
c2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyOTgzNV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MzZd
IHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgzN10gcGNpIDAw
MGQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4MzhdIHBjaSAwMDBkOjAwOjAzLjA6
IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTgzOV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEz
OiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43Mjk4MzldIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRv
IGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyOTg0Ml0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4
NDNdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg0M10gcGNp
IDAwMGQ6MDA6MDMuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4NDRdIHBjaSAwMDBkOjAwOjAz
LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg0NV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mjk4NDZdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyOTg0N10gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjk4NDhdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg0OV0g
cGNpIDAwMGQ6MDE6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMDBm
ZmZmIHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODUxXSBw
Y2kgMDAwZDowMTowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMDEwMDAwLTB4NTAwMTNm
ZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODU2XSBw
Y2kgMDAwZDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODU3XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDUwMDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODU5XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg2MF0gcGNpIDAwMGQ6MDI6MDAuMDogQkFS
IDY6IGFzc2lnbmVkIFttZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODYxXSBwY2kgMDAwZDowMjowMC4wOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODY2XSBwY2kgMDAwZDowMDowMi4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI5ODY3XSBwY2kgMDAwZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMjAwMDAw
LTB4NTAzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODY5
XSBwY2kgMDAwZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDIwMDAwMC0w
eDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyOTg3MF0gcGNpIDAwMGQ6MDM6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1
MDQwMDAwMC0weDUwNDBmZmZmIHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5ODcxXSBwY2kgMDAwZDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUw
NDEwMDAwLTB4NTA0MTNmZmYgNjRiaXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5ODc2XSBwY2kgMDAwZDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODc3XSBwY2kgMDAwZDowMDow
My4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODc4XSBwY2kgMDAwZDowMDowMy4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg4MF0gcGNpIDAw
MGQ6MDQ6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDYwMDAwMC0weDUwNjBmZmZmIHBy
ZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODgxXSBwY2kgMDAw
ZDowNDowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNjEwMDAwLTB4NTA2MTNmZmYgNjRi
aXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5ODg2XSBwY2kgMDAw
ZDowMDowNC4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODg3XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDUwNjAwMDAwLTB4NTA3ZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI5ODg4XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDM0MDAwMDYwMDAwMC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg5MF0gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA0
IFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mjk4OTFdIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2UgNSBbbWVt
IDB4MzQwMDAwMDAwMDAwLTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4OTJdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMSBb
bWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43Mjk4OTNdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAw
MDAwMDAwLTB4MzQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODkzXSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDEgW21lbSAw
eDUwMjAwMDAwLTB4NTAzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI5ODk0XSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDIwMDAw
MC0weDM0MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyOTg5NV0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAxIFttZW0gMHg1MDQw
MDAwMC0weDUwNWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
OTg5Nl0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDA0MDAwMDAtMHgz
NDAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjk4OTddIHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2UgMSBbbWVtIDB4NTA2MDAwMDAt
MHg1MDdmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4OTdd
IHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAw
N2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMwMDA0XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTJdIChkb21haW4gMDAwMSBbYnVzIDAw
LWZmXSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzAwMDldIGFjcGkg
UE5QMEEwODowMjogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQ
TSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMDA3M10gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBz
dXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDEzMV0gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBu
b3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzAxMzNdIGFjcGkgUE5QMEEwODowMjogTUNGRyBxdWlyazogRUNBTSBh
dCBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0
aCAweGZmZmZiZWM4OTljNWVhNTgKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzA3NThdIGFjcGkgUE5QMEEwODowMjogRUNBTSBhcmVhIFttZW0gMHgzYmZmZjAwMDAwMDAt
MHgzYmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzA3NjRdIGFjcGkgUE5QMEEwODowMjogRUNBTSBhdCBbbWVt
IDB4M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzA4MjNdIFBDSSBob3N0IGJyaWRnZSB0byBi
dXMgMDAwMTowMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDgyNF0g
cGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NjAwMDAwMDAtMHg2ZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwODI1
XSBwY2lfYnVzIDAwMDE6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzODAwMDAwMDAwMDAt
MHgzYmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMDgyNl0gcGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZm
XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDg0M10gcGNpIDAwMDE6
MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDg4NV0gcGNpIDAwMDE6MDA6MDEuMDogWzFkZWY6
ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMDg5OF0gcGNpIDAwMDE6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFn
cwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDkyNV0gcGNpIDAwMDE6
MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzA5MjVdIHBjaSAwMDAxOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEg
RDNob3QKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzEwMTJdIGFjcGlw
aHA6IFNsb3QgWzEtM10gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMzEwMl0gcGNpX2J1cyAwMDAxOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzEwNF0gcGNpIDAwMDE6MDA6MDEuMDogYnJp
ZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxMDVdIHBjaSAwMDAxOjAw
OjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJl
Zl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMxMDZdIHBjaSAwMDAxOjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9z
aXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMzMTA4XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0g
MHg2MDAwMDAwMC0weDYwMWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMzEwOV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4Mzgw
MDAwMDAwMDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMzMTEwXSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMzExMV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMzMTEyXSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzExM10gcGNp
IDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMTE0XSBwY2kgMDAwMTow
MDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMzMTE2XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDYwMDAwMDAwLTB4NjAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMzMTE3XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM4
MDAwMDAwMDAwMC0weDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMzExOV0gcGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA0IFtt
ZW0gMHg2MDAwMDAwMC0weDZmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzMxMjBdIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNSBbbWVtIDB4
MzgwMDAwMDAwMDAwLTB4M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzMxMjFdIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMSBbbWVt
IDB4NjAwMDAwMDAtMHg2MDFmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzMxMjJdIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzgwMDAwMDAw
MDAwLTB4MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzMzMjI2XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTNdIChkb21haW4g
MDAwMCBbYnVzIDAwLWZmXSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzMyMzBdIGFjcGkgUE5QMEEwODowMzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmln
IEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczMzI5NF0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBwbGF0Zm9y
bSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzM1Ml0gYWNwaSBQTlAwQTA4OjAz
OiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMzNTRdIGFjcGkgUE5QMEEwODowMzogTUNGRyBx
dWlyazogRUNBTSBhdCBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1
cyAwMC1mZl0gd2l0aCAweGZmZmZiZWM4OTljNWVhNTgKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzM5NzddIGFjcGkgUE5QMEEwODowMzogRUNBTSBhcmVhIFttZW0gMHgz
ZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM5ODNdIGFjcGkgUE5QMEEwODowMzog
RUNBTSBhdCBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1m
Zl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQwNDJdIFBDSSBob3N0
IGJyaWRnZSB0byBidXMgMDAwMDowMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNDA0M10gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NzAw
MDAwMDAtMHg3ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzM0MDQ0XSBwY2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgz
YzAwMDAwMDAwMDAtMHgzZmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNDA0NV0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDA2
MV0gcGNpIDAwMDA6MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDEwM10gcGNpIDAwMDA6MDA6
MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDExNl0gcGNpIDAwMDA6MDA6MDEuMDogZW5hYmxpbmcg
RXh0ZW5kZWQgVGFncwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDE0
M10gcGNpIDAwMDA6MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzQxNDRdIHBjaSAwMDAwOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVk
IGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzQyMzBdIGFjcGlwaHA6IFNsb3QgWzEtNF0gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjMyMF0gcGNpX2J1cyAwMDAwOjAwOiBvbiBOVU1BIG5vZGUg
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjMyMl0gcGNpIDAwMDA6
MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDEwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYzMjNd
IHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZm
ZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAw
MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYzMjRdIHBjaSAwMDAw
OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFti
dXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MzI2XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTQ6IGFz
c2lnbmVkIFttZW0gMHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNjMyN10gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25l
ZCBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MzI4XSBwY2kgMDAwMDowMDowMS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjczNjMyOF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM2MzMwXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNjMzMV0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MzMy
XSBwY2kgMDAwMDowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MzM0XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM2MzM1XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDNjMDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjMzN10gcGNpX2J1cyAwMDAwOjAwOiBy
ZXNvdXJjZSA0IFttZW0gMHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYzMzhdIHBjaV9idXMgMDAwMDowMDogcmVzb3Vy
Y2UgNSBbbWVtIDB4M2MwMDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYzMzldIHBjaV9idXMgMDAwMDowMTogcmVz
b3VyY2UgMSBbbWVtIDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzYzNDBdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMiBbbWVt
IDB4M2MwMDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2NDQyXSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BD
STRdIChkb21haW4gMDAwMiBbYnVzIDAwLWZmXSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzY0NDZdIGFjcGkgUE5QMEEwODowNDogX09TQzogT1Mgc3VwcG9ydHMgW0V4
dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjUwOF0gYWNwaSBQTlAwQTA4OjA0OiBf
T1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQ
TUUgTFRSXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjU2Nl0gYWNw
aSBQTlAwQTA4OjA0OiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0K
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY1NjhdIGFjcGkgUE5QMEEw
ODowNDogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZm
ZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZiZWM4OTljNWVhNTgKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcxOTNdIGFjcGkgUE5QMEEwODowNDogRUNBTSBh
cmVhIFttZW0gMHgyM2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBD
MDI6MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcxOTldIGFjcGkg
UE5QMEEwODowNDogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBm
b3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzcy
NTddIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMjowMApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNzI1OV0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbbWVtIDB4MDA4MDAwMDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM3MjU5XSBwY2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJlc291
cmNlIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzI2MF0gcGNpX2J1cyAwMDAyOjAwOiByb290
IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNzI3N10gcGNpIDAwMDI6MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFz
cyAweDA2MDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzMxOV0g
cGNpIDAwMDI6MDA6MDEuMDogWzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzM1NF0gcGNpIDAwMDI6MDA6MDEu
MDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzczNTVdIHBjaSAwMDAyOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc0MTVdIHBjaSAwMDAyOjAw
OjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc0NTBdIHBjaSAwMDAyOjAwOjA1LjA6IHN1cHBvcnRz
IEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3NDUxXSBwY2kg
MDAwMjowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3NTMzXSBhY3BpcGhwOiBTbG90IFsxLTVdIHJl
Z2lzdGVyZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzk2NDZdIGFj
cGlwaHA6IFNsb3QgWzMtMl0gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc0MTczOF0gcGNpX2J1cyAwMDAyOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTc0MF0gcGNpIDAwMDI6MDA6MDEuMDog
YnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEw
MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NDFdIHBjaSAwMDAy
OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQg
cHJlZl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NDJdIHBjaSAwMDAyOjAwOjAxLjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQxNzQzXSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAg
MHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTc0NF0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBh
ZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0MTc0NV0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NDdd
IHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwODAwMDAwLTB4MDA5
ZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzQ4XSBwY2kg
MDAwMjowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAw
MDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDE3NTBdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwYTAw
MDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQx
NzUxXSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAyMDAw
MDAtMHgyMDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDE3NTJdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9y
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQxNzUyXSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBz
aXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NTNd
IHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzU1XSBwY2kgMDAwMjow
MDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NTZdIHBjaSAwMDAyOjAwOjA1LjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzU3XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDE3NThdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2Ug
Zm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQxNzU5XSBwY2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3
NjBdIHBjaSAwMDAyOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NjFdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDE3NjNdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzY1XSBwY2kgMDAwMjowMDowNS4w
OiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQxNzY2XSBwY2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDAw
YTAwMDAwLTB4MDBiZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQxNzY4XSBwY2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDIw
MDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0MTc3MF0gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJjZSA0IFttZW0gMHgw
MDgwMDAwMC0weDBmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDE3NzFdIHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjAwMDAw
MDAwMDAwLTB4MjNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDE3NzFdIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMSBbbWVtIDB4MDA4
MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDE3NzJdIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4
MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQxNzczXSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDEgW21lbSAweDAwYTAwMDAw
LTB4MDBiZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNzc0
XSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAw
MDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0MTg3OV0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0k1XSAoZG9tYWluIDAwMDMgW2J1cyAw
MC1mZl0pClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxODgzXSBhY3Bp
IFBOUDBBMDg6MDU6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2Nr
UE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDE5NDddIGFjcGkgUE5QMEEwODowNTogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qg
c3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDIwMDVdIGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mg
bm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQyMDA3XSBhY3BpIFBOUDBBMDg6MDU6IE1DRkcgcXVpcms6IEVDQU0g
YXQgW21lbSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdp
dGggMHhmZmZmYmVjODk5YzVlYTU4ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQyNjI5XSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXJlYSBbbWVtIDB4MjdmZmYwMDAwMDAw
LTB4MjdmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjM1XSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXQgW21l
bSAweDI3ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjg3XSBQQ0kgaG9zdCBicmlkZ2UgdG8g
YnVzIDAwMDM6MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI2ODhd
IHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDEwMDAwMDAwLTB4MWZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjY4
OV0gcGNpX2J1cyAwMDAzOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjQwMDAwMDAwMDAw
LTB4MjdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDI2OTBdIHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1m
Zl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI3MDZdIHBjaSAwMDAz
OjAwOjAwLjA6IFsxZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDI3NDZdIHBjaSAwMDAzOjAwOjAxLjA6IFsxZGVm
OmUxMTFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDI3ODJdIHBjaSAwMDAzOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNzgyXSBwY2kgMDAwMzowMDowMS4w
OiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQyODM5XSBwY2kgMDAwMzowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBl
IDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQyODY2XSBwY2kgMDAwMzowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0Mjg2N10gcGNpIDAwMDM6MDA6MDUuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0MjkzM10gYWNwaXBocDogU2xvdCBbMS02XSByZWdpc3RlcmVkClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MDU1XSBwY2kgMDAwMzowMjowMC4wOiBbODA4Njox
NTIxXSB0eXBlIDAwIGNsYXNzIDB4MDIwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1MDcxXSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDEwMDIw
MDAwLTB4MTAwM2ZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1
MDg3XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxODogW2lvICAweDFmZmY4MDIwLTB4MWZmZjgw
M2ZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MDk2XSBwY2kgMDAw
MzowMjowMC4wOiByZWcgMHgxYzogW21lbSAweDEwMDQ0MDAwLTB4MTAwNDdmZmZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MjA3XSBwY2kgMDAwMzowMjowMC4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0NTI0NF0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg0OiBbbWVt
IDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0NTI0NV0gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMCBzcGFj
ZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAg
Zm9yIDggVkZzKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTI2Nl0g
cGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTkwOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTI2N10g
cGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAw
MWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTM2NF0gcGNpIDAwMDM6MDI6MDAuMDogOC4wMDAg
R2IvcyBhdmFpbGFibGUgUENJZSBiYW5kd2lkdGgsIGxpbWl0ZWQgYnkgNS4wIEdUL3MgUENJZSB4
MiBsaW5rIGF0IDAwMDM6MDA6MDUuMCAoY2FwYWJsZSBvZiAxNi4wMDAgR2IvcyB3aXRoIDUuMCBH
VC9zIFBDSWUgeDQgbGluaykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDU0MTBdIHBjaSAwMDAzOjAyOjAwLjE6IFs4MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAw
MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU0MjZdIHBjaSAwMDAz
OjAyOjAwLjE6IHJlZyAweDEwOiBbbWVtIDB4MTAwMDAwMDAtMHgxMDAxZmZmZl0KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU0NDJdIHBjaSAwMDAzOjAyOjAwLjE6IHJl
ZyAweDE4OiBbaW8gIDB4MWZmZjgwMDAtMHgxZmZmODAxZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDU0NTFdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDFjOiBbbWVt
IDB4MTAwNDAwMDAtMHgxMDA0M2ZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDU1NTddIHBjaSAwMDAzOjAyOjAwLjE6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNo
b3QgRDNjb2xkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NTkxXSBw
Y2kgMDAwMzowMjowMC4xOiByZWcgMHgxODQ6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NTkyXSBw
Y2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIwIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAx
ZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMCBmb3IgOCBWRnMpClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NjEyXSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgx
OTA6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NjEzXSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBC
QVIzIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFp
bnMgQkFSMyBmb3IgOCBWRnMpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ1NzMzXSBwY2lfYnVzIDAwMDM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzM1XSBwY2lfYnVzIDAwMDM6MDA6IG1heCBidXMgZGVw
dGg6IDEgcGNpX3RyeV9udW06IDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDU3MzldIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMDAw
MDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1
NzQwXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAwMDAw
MDAtMHgyNDAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDU3NDFdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21l
bSAweDEwMjAwMDAwLTB4MTAzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ1NzQyXSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgy
NDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDU3NDNdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1NzQ0XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDU3NDVdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzQ1XSBw
Y2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3NDddIHBjaSAwMDAz
OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDU3NDhdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDU3NTBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzUyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMDogYXNz
aWduZWQgW21lbSAweDEwMjAwMDAwLTB4MTAyMWZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ1NzU2XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMDogYXNzaWduZWQg
W21lbSAweDEwMjIwMDAwLTB4MTAyM2ZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1NzU5XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMzogYXNzaWduZWQgW21lbSAw
eDEwMjQwMDAwLTB4MTAyNDNmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ1NzYyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAw
MDIwMDAwMC0weDI0MDAwMDIxZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0NTc2Nl0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDEwOiBhc3NpZ25l
ZCBbbWVtIDB4MjQwMDAwMjIwMDAwLTB4MjQwMDAwMjNmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzcxXSBwY2kgMDAwMzowMjowMC4xOiBC
QVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQ0MDAwLTB4MTAyNDdmZmZdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1Nzc0XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgNzog
YXNzaWduZWQgW21lbSAweDI0MDAwMDI0MDAwMC0weDI0MDAwMDI1ZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTc3OF0gcGNpIDAwMDM6MDI6
MDAuMTogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjYwMDAwLTB4MjQwMDAwMjdmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1Nzgy
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzgzXSBwY2kgMDAwMzow
MjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTc4NF0gcGNpIDAwMDM6MDI6MDAuMTog
QkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0NTc4NV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxl
ZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDU3ODZdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAw
Ml0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3ODhdIHBjaSAwMDAz
OjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3ODldIHBjaSAwMDAzOjAwOjA1
LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0
Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzkxXSBw
Y2lfYnVzIDAwMDM6MDA6IE5vLiAyIHRyeSB0byBhc3NpZ24gdW5hc3NpZ25lZCByZXMKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3OTNdIHBjaSAwMDAzOjAwOjAxLjA6
IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1NzkzXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZh
aWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDU3OTVdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2Ug
Zm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ1Nzk1XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3
OTZdIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU3OThdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDU3OTldIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1ODAxXSBwY2kgMDAwMzowMjowMC4w
OiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1ODAyXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0NTgwM10gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0NTgwNF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBz
aXplIDB4MDAyMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU4MDVd
IHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDU4MDZdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDU4MDddIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1ODA5XSBwY2lfYnVzIDAwMDM6MDA6IEF1
dG9tYXRpY2FsbHkgZW5hYmxlZCBwY2kgcmVhbGxvYywgaWYgeW91IGhhdmUgcHJvYmxlbSwgdHJ5
IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYz1vZmYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDU4MTBdIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2UgNCBbbWVtIDB4MTAw
MDAwMDAtMHgxZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ1ODExXSBwY2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDUgW21lbSAweDI0MDAwMDAw
MDAwMC0weDI3ZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ1ODExXSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDEgW21lbSAweDEwMDAw
MDAwLTB4MTAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1
ODEyXSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDIgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0NTgxM10gcGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAxIFttZW0gMHgxMDIwMDAwMC0w
eDEwM2ZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTgxNF0g
cGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDU4ODBdIEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJNl0gKGRvbWFpbiAwMDA0IFtidXMgMDAt
ZmZdKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NTg4NV0gYWNwaSBQ
TlAwQTA4OjA2OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BN
IFNlZ21lbnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ1OTUwXSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1
cHBvcnQgW1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2MDA4XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIG5v
dyBjb250cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0NjAxMF0gYWNwaSBQTlAwQTA4OjA2OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0
IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRo
IDB4ZmZmZmJlYzg5OWM1ZWE1OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0NjYzM10gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGFyZWEgW21lbSAweDJiZmZmMDAwMDAwMC0w
eDJiZmZmZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0NjYzOV0gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGF0IFttZW0g
MHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NjY4OV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1
cyAwMDA0OjAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2NjkwXSBw
Y2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMC0weDJmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDY2OTFd
IHBjaV9idXMgMDAwNDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI4MDAwMDAwMDAwMC0w
eDJiZmZkZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ2NjkyXSBwY2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2NzA5XSBwY2kgMDAwNDow
MDowMC4wOiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2NzQ2XSBwY2kgMDAwNDowMDowMS4wOiBbMWRlZjpl
MTExXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ2NzczXSBwY2kgMDAwNDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0Njc3NF0gcGNpIDAwMDQ6MDA6MDEuMDog
UE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0NjgxNl0gcGNpIDAwMDQ6MDA6MDMuMDogWzFkZWY6ZTExM10gdHlwZSAw
MSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
Njg0M10gcGNpIDAwMDQ6MDA6MDMuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDY4NDNdIHBjaSAwMDA0OjAwOjAzLjA6IFBNRSMgc3VwcG9y
dGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDY4ODhdIHBjaSAwMDA0OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgw
NjA0MDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDY5MjRdIHBjaSAw
MDA0OjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ2OTI1XSBwY2kgMDAwNDowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQw
IEQxIEQzaG90ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ2OTk3XSBw
Y2kgMDAwNDowMTowMC4wOiBbMWEwMzoxMTUwXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ3MDQ0XSBwY2kgMDAwNDowMTowMC4w
OiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ3MTAwXSBwY2kgMDAwNDowMTowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NzEwMV0gcGNpIDAwMDQ6MDE6MDAuMDogUE1F
IyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDcxOTZdIHBjaV9idXMgMDAwNDowMjogZXh0ZW5kZWQgY29u
ZmlnIHNwYWNlIG5vdCBhY2Nlc3NpYmxlClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQ3MjE0XSBwY2kgMDAwNDowMjowMC4wOiBbMWEwMzoyMDAwXSB0eXBlIDAwIGNsYXNz
IDB4MDMwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ3MjMyXSBw
Y2kgMDAwNDowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ3MjQyXSBwY2kgMDAwNDowMjow
MC4wOiByZWcgMHgxNDogW21lbSAweDIyMDAwMDAwLTB4MjIwMWZmZmZdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ3MjUyXSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgx
ODogW2lvICAweDJmZmY4MDAwLTB4MmZmZjgwN2ZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ3Mjk3XSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgdG8g
ZWZpZmIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDczMzNdIHBjaSAw
MDA0OjAyOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ3MzMzXSBwY2kgMDAwNDowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQw
IEQxIEQyIEQzaG90IEQzY29sZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0NzQ0NF0gcGNpIDAwMDQ6MDM6MDAuMDogWzE5MTI6MDAxNF0gdHlwZSAwMCBjbGFzcyAweDBj
MDMzMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NzQ2NV0gcGNpIDAw
MDQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHgyMjIwMDAwMC0weDIyMjAxZmZmIDY0Yml0XQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NzU3M10gcGNpIDAwMDQ6MDM6
MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3MDddIHBjaV9idXMgMDAwNDowMDogb24gTlVNQSBu
b2RlIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3MTBdIHBjaSAw
MDA0OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRi
aXQgcHJlZl0gdG8gW2J1cyAwMS0wMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3MTFdIHBjaSAwMDA0OjAw
OjAzLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRf
c2l6ZSAxMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzEyXSBw
Y2kgMDAwNDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZm
IDY0Yml0IHByZWZdIHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAw
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzEzXSBwY2kgMDAwNDow
MDowMy4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVz
IDAzXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0ODcxNF0gcGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdlIHdpbmRv
dyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3MTVdIHBjaSAwMDA0OjAwOjA1LjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1
cyAwNF0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDg3MTddIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAw
MCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ4NzIwXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAw
MC0weDIyZmZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODcy
MV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMDAwMDAw
LTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ4NzIyXSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0g
MHgyMzAwMDAwMC0weDIzMWZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0ODcyM10gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4Mjgw
MDAwMjAwMDAwLTB4MjgwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ4NzI0XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTQ6IGFzc2ln
bmVkIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0ODcyNV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBb
bWVtIDB4MjgwMDAwNDAwMDAwLTB4MjgwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzI2XSBwY2kgMDAwNDowMDowMS4wOiBCQVIg
MTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc0ODcyN10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQg
dG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ4NzI4XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
ODcyOV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzMwXSBw
Y2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODczMF0gcGNpIDAwMDQ6MDA6
MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzMyXSBwY2kgMDAwNDowMDowMS4wOiBC
QVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc0ODczM10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWls
ZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ4NzM0XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc0ODczNV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzM2
XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODczNl0gcGNpIDAwMDQ6
MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzM4XSBwY2kgMDAwNDowMTowMC4w
OiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODczOV0gcGNpIDAwMDQ6MDE6MDAuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDg3NDBdIHBjaSAwMDA0OjAxOjAwLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc0ODc0MV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0g
MHgyMDAwMDAwMC0weDIxZmZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0ODc0NV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHgyMjAw
MDAwMC0weDIyMDFmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
ODc0OF0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgw
MDgwXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc0OV0gcGNpIDAw
MDQ6MDI6MDAuMDogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDA4MF0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3NTBdIHBjaSAwMDA0OjAxOjAw
LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDg3NTRdIHBjaSAwMDA0OjAxOjAwLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDg3NjFdIHBjaSAwMDA0OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMS0wMl0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3NjNdIHBjaSAwMDA0OjAwOjAx
LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg3NjRdIHBjaSAwMDA0OjAwOjAxLjA6ICAg
YnJpZGdlIHdpbmRvdyBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzY2XSBwY2kgMDAw
NDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDIzMDAwMDAwLTB4MjMwMDFmZmYgNjRi
aXRdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4Nzc1XSBwY2kgMDAw
NDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ4Nzc2XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQ4Nzc4XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc3OV0gcGNpIDAwMDQ6MDA6MDUuMDogUENJIGJyaWRn
ZSB0byBbYnVzIDA0XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc4
MV0gcGNpIDAwMDQ6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMzIwMDAwMC0weDIz
M2ZmZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc4Ml0gcGNp
IDAwMDQ6MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAw
MDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDg3ODRdIHBjaV9idXMgMDAwNDowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUg
dW5hc3NpZ25lZCwgdHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc4Nl0gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA0
IFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDg3ODZdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2UgNSBbbWVt
IDB4MjgwMDAwMDAwMDAwLTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDg3ODddIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMSBb
bWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDg3ODhdIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAw
MDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ4Nzg5XSBwY2lfYnVzIDAwMDQ6MDI6IHJlc291cmNlIDEgW21lbSAw
eDIwMDAwMDAwLTB4MjJmZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ4NzkwXSBwY2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDEgW21lbSAweDIzMDAwMDAwLTB4
MjMxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ4NzkwXSBw
Y2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0
ODc5MV0gcGNpX2J1cyAwMDA0OjA0OiByZXNvdXJjZSAxIFttZW0gMHgyMzIwMDAwMC0weDIzM2Zm
ZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODc5Ml0gcGNpX2J1
cyAwMDA0OjA0OiByZXNvdXJjZSAyIFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDg4MTld
IEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJN10gKGRvbWFpbiAwMDA1IFtidXMgMDAtZmZdKQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0ODgyM10gYWNwaSBQTlAwQTA4
OjA3OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21l
bnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQ4ODg3XSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQg
W1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQ4OTQ1XSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IE9TIG5vdyBjb250
cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc0ODk0N10gYWNwaSBQTlAwQTA4OjA3OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0g
MHgyZmZmZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZm
ZmJlYzg5OWM1ZWE1OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTU3
MF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGFyZWEgW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZm
ZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc0OTU3Nl0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGF0IFttZW0gMHgyZmZm
ZjAwMDAwMDAtMHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTYzMl0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA1
OjAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5NjMzXSBwY2lfYnVz
IDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMC0weDNmZmZmZmZmIHdp
bmRvd10KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDk2MzRdIHBjaV9i
dXMgMDAwNTowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZk
ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5
NjM1XSBwY2lfYnVzIDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5NjUxXSBwY2kgMDAwNTowMDowMC4w
OiBbMWRlZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ5NjkzXSBwY2kgMDAwNTowMDowMS4wOiBbMWRlZjplMTExXSB0
eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ5NzMxXSBwY2kgMDAwNTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTczMl0gcGNpIDAwMDU6MDA6MDEuMDogUE1FIyBz
dXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc0OTc5Ml0gcGNpIDAwMDU6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFz
cyAweDA2MDQwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0OTgyOF0g
cGNpIDAwMDU6MDA6MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NDk4MjldIHBjaSAwMDA1OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZy
b20gRDAgRDEgRDNob3QKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDk4
ODFdIHBjaSAwMDA1OjAwOjA3LjA6IFsxZGVmOmUxMTddIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDk5MTddIHBjaSAwMDA1OjAw
OjA3LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzQ5OTE4XSBwY2kgMDAwNTowMDowNy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQz
aG90ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ5OTk0XSBhY3BpcGhw
OiBTbG90IFsxLTddIHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTQyMDFdIHBjaV9idXMgMDAwNTowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMDJdIHBjaSAwMDA1OjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjAzXSBwY2kgMDAwNTowMDow
MS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZd
IHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjA1XSBwY2kgMDAwNTowMDowMS4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6
ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAwLjc1NDIwNl0gcGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAw
MC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NTQyMDddIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3Np
emUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NTQyMDhdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAw
MTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24g
MTAwMDAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjA5XSBwY2kg
MDAwNTowMDowNy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAw
M10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
NDIxMF0gcGNpIDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgw
MDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWdu
IDEwMDAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDIxMV0gcGNp
IDAwMDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0g
dG8gW2J1cyAwM10gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMTRdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAx
NDogYXNzaWduZWQgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjE1XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTU6IGFz
c2lnbmVkIFttZW0gMHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMTZdIHBjaSAwMDA1OjAwOjA1
LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjE3XSBwY2kgMDAwNTowMDowNS4wOiBC
QVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMThdIHBjaSAw
MDA1OjAwOjA3LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjE5XSBwY2kgMDAwNTow
MDowNy4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQy
MjBdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjIxXSBwY2kgMDAw
NTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMjJdIHBjaSAwMDA1OjAwOjA1
LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjIzXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NTQyMjRdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzU0MjI0XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTQyMjddIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjI3XSBwY2kg
MDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMjhdIHBjaSAwMDA1OjAw
OjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjI5XSBwY2kgMDAwNTowMDowNS4wOiBCQVIg
MTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMzBdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzU0MjMxXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NTQyMzJdIHBjaSAwMDA1OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMzRdIHBjaSAwMDA1OjAwOjAxLjA6
ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0KU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyMzVdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjM3XSBwY2kgMDAwNTow
MDowNS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzU0MjM4XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzU0MjQwXSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJj
MDAwMDIwMDAwMC0weDJjMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjc1NDI0Ml0gcGNpIDAwMDU6MDA6MDcuMDogUENJIGJyaWRnZSB0
byBbYnVzIDAzXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDI0M10g
cGNpIDAwMDU6MDA6MDcuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDQwMDAwMC0weDMwNWZm
ZmZmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDI0NV0gcGNpIDAw
MDU6MDA6MDcuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NTQyNDddIHBjaV9idXMgMDAwNTowMDogcmVzb3VyY2UgNCBbbWVtIDB4MzAwMDAwMDAtMHgzZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjQ4
XSBwY2lfYnVzIDAwMDU6MDA6IHJlc291cmNlIDUgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZk
ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0
MjQ5XSBwY2lfYnVzIDAwMDU6MDE6IHJlc291cmNlIDEgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZm
ZmZdClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MjQ5XSBwY2lfYnVz
IDAwMDU6MDE6IHJlc291cmNlIDIgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDI1MF0g
cGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAxIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDI1MV0gcGNpX2J1cyAwMDA1
OjAyOiByZXNvdXJjZSAyIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyNTJdIHBjaV9i
dXMgMDAwNTowMzogcmVzb3VyY2UgMSBbbWVtIDB4MzA0MDAwMDAtMHgzMDVmZmZmZl0KU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQyNTNdIHBjaV9idXMgMDAwNTowMzog
cmVzb3VyY2UgMiBbbWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZmIDY0Yml0IHByZWZd
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU2NzIwXSBpb21tdTogRGVm
YXVsdCBkb21haW4gdHlwZTogVHJhbnNsYXRlZCAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTY3NTNdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogVkdBIGRldmljZSBh
ZGRlZDogZGVjb2Rlcz1pbyttZW0sb3ducz1ub25lLGxvY2tzPW5vbmUKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTY3NTRdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjog
YnJpZGdlIGNvbnRyb2wgcG9zc2libGUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTY3NTZdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogc2V0dGluZyBhcyBib290IGRl
dmljZSAoVkdBIGxlZ2FjeSByZXNvdXJjZXMgbm90IGF2YWlsYWJsZSkKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTY3NTddIHZnYWFyYjogbG9hZGVkClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU2ODQzXSBFREFDIE1DOiBWZXI6IDMuMC4wClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU2OTcyXSBSZWdpc3RlcmVkIGVm
aXZhcnMgb3BlcmF0aW9ucwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1
NzYyOV0gTmV0TGFiZWw6IEluaXRpYWxpemluZwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAwLjc1NzYzMF0gTmV0TGFiZWw6ICBkb21haW4gaGFzaCBzaXplID0gMTI4ClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NjMxXSBOZXRMYWJlbDogIHByb3Rv
Y29scyA9IFVOTEFCRUxFRCBDSVBTT3Y0IENBTElQU08KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43NTc2NDBdIE5ldExhYmVsOiAgdW5sYWJlbGVkIHRyYWZmaWMgYWxsb3dl
ZCBieSBkZWZhdWx0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3NzY4
XSBjbG9ja3NvdXJjZTogU3dpdGNoZWQgdG8gY2xvY2tzb3VyY2UgYXJjaF9zeXNfY291bnRlcgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2NjI1Nl0gVkZTOiBEaXNrIHF1
b3RhcyBkcXVvdF82LjYuMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2
NjI3Ml0gVkZTOiBEcXVvdC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDUxMiAob3JkZXIgMCwg
NDA5NiBieXRlcykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjYzODhd
IEFwcEFybW9yOiBBcHBBcm1vciBGaWxlc3lzdGVtIEVuYWJsZWQKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NjY0MjFdIHBucDogUG5QIEFDUEkgaW5pdApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2NzA0NV0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4
M2JmZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVk
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzY3MDQ2XSBzeXN0ZW0gMDA6
MDA6IFttZW0gMHgzZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3Qg
YmUgcmVzZXJ2ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjcwNDdd
IHN5c3RlbSAwMDowMDogW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZiB3aW5kb3dd
IGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc2NzA0OF0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZm
ZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzY3MDUwXSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyYmZmZjAwMDAwMDAt
MHgyYmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjcwNTFdIHN5c3RlbSAwMDowMDogW21lbSAweDJm
ZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2NzA1Ml0gc3lzdGVtIDAwOjAw
OiBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJl
IHJlc2VydmVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzY3MDUzXSBz
eXN0ZW0gMDA6MDA6IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmYgd2luZG93XSBj
b3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NjcwNTddIHN5c3RlbSAwMDowMDogUGx1ZyBhbmQgUGxheSBBQ1BJIGRldmljZSwgSURzIFBO
UDBjMDIgKGFjdGl2ZSkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Njcz
NDhdIHBucDogUG5QIEFDUEk6IGZvdW5kIDEgZGV2aWNlcwpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAwLjc2ODI5Ml0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAy
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzY4MzU3XSBJUCBpZGVudHMg
aGFzaCB0YWJsZSBlbnRyaWVzOiAyNjIxNDQgKG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5l
YXIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzY5OTUzXSB0Y3BfbGlz
dGVuX3BvcnRhZGRyX2hhc2ggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDcsIDUy
NDI4OCBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc3MDI0MV0gVENQIGVzdGFibGlzaGVkIGhhc2ggdGFibGUgZW50cmllczogNTI0Mjg4IChvcmRl
cjogMTAsIDQxOTQzMDQgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NzEzMjNdIFRDUCBiaW5kIGhhc2ggdGFibGUgZW50cmllczogNjU1MzYgKG9y
ZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzcxNTgxXSBUQ1A6IEhhc2ggdGFibGVzIGNvbmZpZ3VyZWQgKGVzdGFibGlz
aGVkIDUyNDI4OCBiaW5kIDY1NTM2KQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc3MTY2NV0gVURQIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA4LCAxMDQ4
NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzcyMDY5XSBVRFAtTGl0ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0
ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc3Mjg0MV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcyODQ4XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wg
ZmFtaWx5IDQ0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzcyODU2XSBw
Y2kgMDAwYzowMTowMC4wOiBDTFMgbWlzbWF0Y2ggKDY0ICE9IDMyKSwgdXNpbmcgNjQgYnl0ZXMK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NzI5MzFdIFRyeWluZyB0byB1
bnBhY2sgcm9vdGZzIGltYWdlIGFzIGluaXRyYW1mcy4uLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIxNDg5NF0gRnJlZWluZyBpbml0cmQgbWVtb3J5OiA0MTAxMksKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY1NzFdIGh3IHBlcmZldmVudHM6
IGVuYWJsZWQgd2l0aCBhcm12OF9wbXV2M18wIFBNVSBkcml2ZXIsIDcgY291bnRlcnMgYXZhaWxh
YmxlClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE2NjU2XSBrdm0gWzFd
OiBJUEEgU2l6ZSBMaW1pdDogNDggYml0cwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIxNjY3OF0ga3ZtIFsxXTogR0lDdjM6IG5vIEdJQ1YgcmVzb3VyY2UgZW50cnkKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTY2NzldIGt2bSBbMV06IGRpc2Fi
bGluZyBHSUN2MiBlbXVsYXRpb24KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMTY2OTBdIGt2bSBbMV06IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZSBlbmFi
bGVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE3MDI4XSBrdm0gWzFd
OiB2Z2ljIGludGVycnVwdCBJUlE5ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjE3NDM0XSBrdm0gWzFdOiBWSEUgbW9kZSBpbml0aWFsaXplZCBzdWNjZXNzZnVsbHkKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTgyNjhdIEluaXRpYWxpc2Ugc3lz
dGVtIHRydXN0ZWQga2V5cmluZ3MKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yMTgyNzZdIEtleSB0eXBlIGJsYWNrbGlzdCByZWdpc3RlcmVkClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjE4MzEzXSB3b3JraW5nc2V0OiB0aW1lc3RhbXBfYml0cz00
MiBtYXhfb3JkZXI9MjQgYnVja2V0X29yZGVyPTAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yMTg5NzldIHpidWQ6IGxvYWRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIxOTA5OV0gaW50ZWdyaXR5OiBQbGF0Zm9ybSBLZXlyaW5nIGluaXRpYWxp
emVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjE5MTAyXSBLZXkgdHlw
ZSBhc3ltbWV0cmljIHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yMTkxMDJdIEFzeW1tZXRyaWMga2V5IHBhcnNlciAneDUwOScgcmVnaXN0ZXJlZApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxOTEwOV0gQmxvY2sgbGF5ZXIgU0NT
SSBnZW5lcmljIChic2cpIGRyaXZlciB2ZXJzaW9uIDAuNCBsb2FkZWQgKG1ham9yIDI0OSkKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTkxNDldIGlvIHNjaGVkdWxlciBt
cS1kZWFkbGluZSByZWdpc3RlcmVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjIwMTQyXSBzaHBjaHA6IFN0YW5kYXJkIEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZl
ciB2ZXJzaW9uOiAwLjQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjAy
NTVdIGVmaWZiOiBwcm9iaW5nIGZvciBlZmlmYgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIyMDI3NV0gZWZpZmI6IGZyYW1lYnVmZmVyIGF0IDB4MjAwMDAwMDAsIHVzaW5n
IDE4NzZrLCB0b3RhbCAxODc1awpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyMDI3Nl0gZWZpZmI6IG1vZGUgaXMgODAweDYwMHgzMiwgbGluZWxlbmd0aD0zMjAwLCBwYWdl
cz0xClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIwMjc2XSBlZmlmYjog
c2Nyb2xsaW5nOiByZWRyYXcKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
MjAyNzddIGVmaWZiOiBUcnVlY29sb3I6IHNpemU9ODo4Ojg6OCwgc2hpZnQ9MjQ6MTY6ODowClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIwMzUwXSBDb25zb2xlOiBzd2l0
Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMTAweDM3ClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjI2MTkzXSBmYjA6IEVGSSBWR0EgZnJhbWUgYnVmZmVy
IGRldmljZQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyNjU5N10gaW5w
dXQ6IFBvd2VyIEJ1dHRvbiBhcyAvZGV2aWNlcy9MTlhTWVNUTTowMC9MTlhTWUJVUzowMC9QTlAw
QzBDOjAwL2lucHV0L2lucHV0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyNjYxMF0gQUNQSTogUG93ZXIgQnV0dG9uIFtQV1JCXQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIzMDkxM10gQUNQSSBHVERUOiBmb3VuZCAxIFNCU0EgZ2VuZXJpYyBX
YXRjaGRvZyhzKS4KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzE0MzBd
IFNlcmlhbDogODI1MC8xNjU1MCBkcml2ZXIsIDQgcG9ydHMsIElSUSBzaGFyaW5nIGVuYWJsZWQK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzE2ODddIFNlcmlhbDogQU1C
QSBkcml2ZXIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzE3MTldIG1z
bV9zZXJpYWw6IGRyaXZlciBpbml0aWFsaXplZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIzMTc2N10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBvcHRpb24g
bWFzayAweDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzE3OTFdIGFy
bS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVh
dHVyZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
MzE5NTldIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVu
dHJpZXMgZm9yIGNtZHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzE5
OTZdIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJp
ZXMgZm9yIGV2dHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzIwNjBd
IGFybS1zbW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMg
Zm9yIHByaXEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzI0MzldIGFy
bS1zbW11LXYzIGFybS1zbW11LXYzLjEuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMyNDUwXSBhcm0tc21tdS12MyBhcm0tc21tdS12
My4xLmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMyNTY0XSBhcm0tc21tdS12MyBh
cm0tc21tdS12My4xLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMyNjAxXSBhcm0tc21tdS12MyBhcm0t
c21tdS12My4xLmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMyNjY1XSBhcm0tc21tdS12MyBhcm0tc21t
dS12My4xLmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjMyOTcwXSBhcm0tc21tdS12MyBhcm0tc21tdS12
My4yLmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIzMjk4Ml0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBpYXMgNDgtYml0
LCBvYXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIzMzAyOV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBh
bGxvY2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIzMzE0MF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxv
Y2F0ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIzMzE3NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0
ZWQgNjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIzMzUwMl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBvcHRpb24gbWFz
ayAweDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzM1MTFdIGFybS1z
bW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVy
ZXMgMHgwMDAwMWZmZikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzM1
NTddIGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJp
ZXMgZm9yIGNtZHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzM2Njhd
IGFybS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMg
Zm9yIGV2dHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzM3MDRdIGFy
bS1zbW11LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9y
IHByaXEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzQwMTFdIGFybS1z
bW11LXYzIGFybS1zbW11LXYzLjQuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM0MDIwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My40
LmF1dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAx
MyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM0MDg4XSBhcm0tc21tdS12MyBhcm0t
c21tdS12My40LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM0MTMyXSBhcm0tc21tdS12MyBhcm0tc21t
dS12My40LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM0MjQzXSBhcm0tc21tdS12MyBhcm0tc21tdS12
My40LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM0NTQ3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My41
LmF1dG86IG9wdGlvbiBtYXNrIDB4MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIzNDU1OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBpYXMgNDgtYml0LCBv
YXMgNDgtYml0IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIzNDYyMl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxv
Y2F0ZWQgNjU1MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIzNDY2Nl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0
ZWQgMzI3NjggZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjIzNDc3NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQg
NjU1MzYgZW50cmllcyBmb3IgcHJpcQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIzNTA3NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBvcHRpb24gbWFzayAw
eDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzUwODRdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjYuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMg
MHgwMDAwMWZmZikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzUxMjJd
IGFybS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMg
Zm9yIGNtZHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzUxODZdIGFy
bS1zbW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9y
IGV2dHEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzUyMjldIGFybS1z
bW11LXYzIGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHBy
aXEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzU2MjNdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjcuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjM1NjMyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1
dG86IGlhcyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM1NjY4XSBhcm0tc21tdS12MyBhcm0tc21t
dS12My43LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM1NzI5XSBhcm0tc21tdS12MyBhcm0tc21tdS12
My43LmF1dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM1NzcwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43
LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjM5NzU1XSBtb3VzZWRldjogUFMvMiBtb3VzZSBkZXZpY2Ug
Y29tbW9uIGZvciBhbGwgbWljZQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjI0MDcwMl0gcnRjLWVmaSBydGMtZWZpLjA6IHJlZ2lzdGVyZWQgYXMgcnRjMApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MTE2M10gcnRjLWVmaSBydGMtZWZpLjA6IHNl
dHRpbmcgc3lzdGVtIGNsb2NrIHRvIDIwMjEtMDktMTNUMTA6MTA6MzcgVVRDICgxNjMxNTI3ODM3
KQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MTY5NF0gbGVkdHJpZy1j
cHU6IHJlZ2lzdGVyZWQgdG8gaW5kaWNhdGUgYWN0aXZpdHkgb24gQ1BVcwpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0MTk1OF0gU01DQ0M6IFNPQ19JRDogSUQgPSBqZXAx
MDY6MGExNjowMDAxIFJldmlzaW9uID0gMHgwMDAwMDBhMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI0MjIxNl0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAx
MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1MDI1MV0gU2VnbWVudCBS
b3V0aW5nIHdpdGggSVB2NgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1
MDI2NV0gbWlwNjogTW9iaWxlIElQdjYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yNTAyNjZdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTcKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTAyOTRdIG1wbHNfZ3NvOiBNUExTIEdTTyBz
dXBwb3J0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjUwMzUyXSByZWdp
c3RlcmVkIHRhc2tzdGF0cyB2ZXJzaW9uIDEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yNTAzNTVdIExvYWRpbmcgY29tcGlsZWQtaW4gWC41MDkgY2VydGlmaWNhdGVzClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjc2NzIwXSBMb2FkZWQgWC41MDkg
Y2VydCAnRGViaWFuIFNlY3VyZSBCb290IENBOiA2Y2NlY2U3ZTRjNmMwZDFmNjE0OWYzZGQyN2Rm
Y2M1Y2JiNDE5ZWExJwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3Njcy
OV0gTG9hZGVkIFguNTA5IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBTaWduZXIgMjAyMSAtIGxp
bnV4OiA0YjZlZjVhYmNhNjY5ODI1MTc4ZTA1MmM4NDY2N2NjYmMwNTMxZjhjJwpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3Njg4M10genN3YXA6IGxvYWRlZCB1c2luZyBw
b29sIGx6by96YnVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjc2OTgz
XSBLZXkgdHlwZSAuX2ZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjI3Njk4NF0gS2V5IHR5cGUgLmZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3Njk4NF0gS2V5IHR5cGUgZnNjcnlwdC1w
cm92aXNpb25pbmcgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjI3NzAxMl0gQXBwQXJtb3I6IEFwcEFybW9yIHNoYTEgcG9saWN5IGhhc2hpbmcgZW5hYmxl
ZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4NjA0N10gaW50ZWdyaXR5
OiBMb2FkaW5nIFguNTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjg2NjI0XSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdT
VVNFIExpbnV4IEVudGVycHJpc2UgU2VjdXJlIEJvb3QgQ0E6IDNkNGQ0MGNmOTM4NTM5MDI0YjFj
ZmM1YTEyZGVkZmU4YjE3ZTc1NWYnClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjg2NjI1XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODY3ODVdIGludGVncml0eTog
TG9hZGVkIFguNTA5IGNlcnQgJ0Nhbm9uaWNhbCBMdGQuIE1hc3RlciBDZXJ0aWZpY2F0ZSBBdXRo
b3JpdHk6IGFkOTE5OTBiYzIyYWIxZjUxNzA0OGMyM2I2NjU1YTI2OGUzNDVhNjMnClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjg2Nzg2XSBpbnRlZ3JpdHk6IExvYWRpbmcg
WC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yODY5NDNdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0ZlZG9yYSBTZWN1
cmUgQm9vdCBDQTogZmRlMzI1OTljMmQ2MWRiMWJmNTgwNzMzNWQ3YjIwZTRjZDk2M2I0MicKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODY5NDNdIGludGVncml0eTogTG9h
ZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjI4Njk1NV0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9z
b2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5
OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
ODY5NTZdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4Njk2OF0gaW50ZWdyaXR5OiBMb2Fk
ZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IENvcnBvcmF0aW9uIFVFRkkgQ0EgMjAxMTogMTNhZGJm
NDMwOWJkODI3MDljOGNkNTRmMzE2ZWQ1MjI5ODhhMWJkNCcKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yODY5NjldIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZp
Y2F0ZTogVUVGSTpkYgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4Njk4
MF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVj
dGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTEzODVdIHBjaWVwb3J0IDAw
MGM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yOTE2NzhdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQUVSOiBlbmFi
bGVkIHdpdGggSVJRIDEwNgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5
MTc0OF0gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMQpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5MjAwOF0gcGNpZXBvcnQgMDAwZDow
MDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjkyMDUyXSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFkZGluZyB0byBp
b21tdSBncm91cCAyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjkyMjg2
XSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTIzMzBdIHBjaWVwb3J0IDAwMGQ6MDA6
MDMuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yOTI1NjNdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdp
dGggSVJRIDEwNwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5MjYwNF0g
cGNpZXBvcnQgMDAwZDowMDowNC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5MjgzOV0gcGNpZXBvcnQgMDAwZDowMDowNC4w
OiBBRVI6IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjkyODk5XSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBn
cm91cCA1ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjkzMTY1XSBwY2ll
cG9ydCAwMDAxOjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDgKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTMyMzNdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDog
QWRkaW5nIHRvIGlvbW11IGdyb3VwIDYKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yOTM0OTJdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJR
IDEwOQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5MzU1OV0gcGNpZXBv
cnQgMDAwMjowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNwpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI5Mzg0MV0gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBRVI6
IGVuYWJsZWQgd2l0aCBJUlEgMTEwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjkzODg4XSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA4
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjk0MTM0XSBwY2llcG9ydCAw
MDAyOjAwOjA1LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yOTQyMDFdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
OTQ0NzJdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5NDUyNV0gcGNpZXBvcnQgMDAw
MzowMDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTAKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yOTQ3NjRdIHBjaWVwb3J0IDAwMDM6MDA6MDUuMDogQUVSOiBlbmFi
bGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5
NDgyNl0gcGNpZXBvcnQgMDAwNDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTEKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTUxMDFdIHBjaWVwb3J0IDAwMDQ6
MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI5NTE0Nl0gcGNpZXBvcnQgMDAwNDowMDowMy4wOiBBZGRpbmcgdG8g
aW9tbXUgZ3JvdXAgMTIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTUz
ODZdIHBjaWVwb3J0IDAwMDQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5NTQzN10gcGNpZXBvcnQgMDAwNDow
MDowNS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yOTU2ODFdIHBjaWVwb3J0IDAwMDQ6MDA6MDUuMDogQUVSOiBlbmFibGVk
IHdpdGggSVJRIDExMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjI5NTc0
OV0gcGNpZXBvcnQgMDAwNDowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTYwMTBdIHBjaWVwb3J0IDAwMDU6MDA6
MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE1ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjk2Mjg2XSBwY2llcG9ydCAwMDA1OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3
aXRoIElSUSAxMTMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTYzMzZd
IHBjaWVwb3J0IDAwMDU6MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE2ClNlcCAxMyAx
NTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjk2NTgzXSBwY2llcG9ydCAwMDA1OjAwOjA1
LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yOTY2MzZdIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDE3ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjk2ODg0XSBw
Y2llcG9ydCAwMDA1OjAwOjA3LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yOTgyMTJdIEZyZWVpbmcgdW51c2VkIGtlcm5l
bCBtZW1vcnk6IDU0NDBLClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDU5
NDA3XSBDaGVja2VkIFcrWCBtYXBwaW5nczogcGFzc2VkLCBubyBXK1ggcGFnZXMgZm91bmQKU2Vw
IDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40NTk0MTNdIFJ1biAvaW5pdCBhcyBp
bml0IHByb2Nlc3MKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40NTk0MTRd
ICAgd2l0aCBhcmd1bWVudHM6ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NDU5NDE1XSAgICAgL2luaXQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40
NTk0MTVdICAgd2l0aCBlbnZpcm9ubWVudDoKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS40NTk0MTZdICAgICBIT01FPS8KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS40NTk0MTddICAgICBURVJNPWxpbnV4ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNDU5NDE3XSAgICAgQk9PVF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04
LWFybTY0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTMyMDMyXSBwcHNf
Y29yZTogTGludXhQUFMgQVBJIHZlci4gMSByZWdpc3RlcmVkClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNTMyMDMzXSBwcHNfY29yZTogU29mdHdhcmUgdmVyLiA1LjMuNiAt
IENvcHlyaWdodCAyMDA1LTIwMDcgUm9kb2xmbyBHaW9tZXR0aSA8Z2lvbWV0dGlAbGludXguaXQ+
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTMzMDE3XSBQVFAgY2xvY2sg
c3VwcG9ydCByZWdpc3RlcmVkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTM3MjQ3XSBTQ1NJIHN1YnN5c3RlbSBpbml0aWFsaXplZApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjU0MDI1OV0gaWdiOiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5l
dHdvcmsgRHJpdmVyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTQwMjYw
XSBpZ2I6IENvcHlyaWdodCAoYykgMjAwNy0yMDE0IEludGVsIENvcnBvcmF0aW9uLgpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU0MDc0MV0gaWdiIDAwMDM6MDI6MDAuMDog
QWRkaW5nIHRvIGlvbW11IGdyb3VwIDE4ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTQxMDM3XSBBQ1BJOiBidXMgdHlwZSBVU0IgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU0MTA1NF0gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcg
aW50ZXJmYWNlIGRyaXZlciB1c2JmcwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjU0MTA2MF0gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciBodWIK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41NDEwODNdIHVzYmNvcmU6IHJl
Z2lzdGVyZWQgbmV3IGRldmljZSBkcml2ZXIgdXNiClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNTQ0NDgzXSBudm1lIDAwMGQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdy
b3VwIDE5ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTQ0NzczXSBudm1l
IG52bWUwOiBwY2kgZnVuY3Rpb24gMDAwZDowMTowMC4wClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNTQ0ODI4XSBudm1lIDAwMGQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDIwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTQ1MjIwXSBu
dm1lIG52bWUxOiBwY2kgZnVuY3Rpb24gMDAwZDowMjowMC4wClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNTQ1MjcwXSBudm1lIDAwMGQ6MDM6MDAuMDogQWRkaW5nIHRvIGlv
bW11IGdyb3VwIDIxClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTQ1NTEz
XSBudm1lIG52bWUyOiBwY2kgZnVuY3Rpb24gMDAwZDowMzowMC4wClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNTQ1NTU3XSBudm1lIDAwMGQ6MDQ6MDAuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDIyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTQ1
ODAxXSBudm1lIG52bWUzOiBwY2kgZnVuY3Rpb24gMDAwZDowNDowMC4wClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTUxMTEyXSBtcHQzc2FzIHZlcnNpb24gMzUuMTAwLjAw
LjAwIGxvYWRlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU1MTIxNV0g
bXB0M3NhcyAwMDBjOjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyMwpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU1MTU3OF0gbXB0M3Nhc19jbTA6IDYzIEJJVCBQ
Q0kgQlVTIERNQSBBRERSRVNTSU5HIFNVUFBPUlRFRCwgdG90YWwgbWVtICg2NTMwNDgxMiBrQikK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41NTE5MjFdIHhoY2lfaGNkIDAw
MDQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDI0ClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNTUyMjQ0XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kgSG9z
dCBDb250cm9sbGVyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTUyMjQ5
XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2lnbmVk
IGJ1cyBudW1iZXIgMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU1MjI2
N10geGhjaV9oY2QgMDAwNDowMzowMC4wOiBaZXJvaW5nIDY0Yml0IGJhc2UgcmVnaXN0ZXJzLCBl
eHBlY3RpbmcgZmF1bHQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41NTQ2
MzRdIHJhbmRvbTogZmFzdCBpbml0IGRvbmUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS41NTU3MzddIG52bWUgbnZtZTE6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1
ZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41NTU5MTZdIG52bWUgbnZt
ZTA6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS41NTYzOTFdIG52bWUgbnZtZTI6IDgwLzAvMCBkZWZhdWx0L3JlYWQv
cG9sbCBxdWV1ZXMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41NTcyNTld
IGFzdCAwMDA0OjAyOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAxNApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjU1ODQ3MV0gbnZtZSBudm1lMzogODAvMC8wIGRlZmF1
bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjU1OTI4NV0gIG52bWUxbjE6IHAxIHAyIHAzIHA0ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNTU5NDk5XSAgbnZtZTBuMToKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS41NjE1NjVdICBudm1lM24xOiBwMSBwMiBwMyBwNCBwNSBwNgpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjY5ODM4NF0geGhjaV9oY2QgMDAwNDowMzowMC4w
OiBoY2MgcGFyYW1zIDB4MDE0MDUxY2YgaGNpIHZlcnNpb24gMHgxMDAgcXVpcmtzIDB4MDAwMDAw
MTEwMDAwMDQxMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjY5ODY2NV0g
dXNiIHVzYjE6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0xZDZiLCBpZFByb2R1Y3Q9
MDAwMiwgYmNkRGV2aWNlPSA1LjEwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNjk4NjY3XSB1c2IgdXNiMTogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTMsIFByb2R1
Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42OTg2NjhdIHVzYiB1c2IxOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29udHJvbGxlcgpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjY5ODY2OF0gdXNiIHVzYjE6IE1hbnVmYWN0
dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS42OTg2NjldIHVzYiB1c2IxOiBTZXJpYWxOdW1iZXI6IDAwMDQ6MDM6
MDAuMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjY5ODc1M10gaHViIDEt
MDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42OTg3NjRdIGh1YiAxLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjk4ODk3XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kg
SG9zdCBDb250cm9sbGVyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjk4
OTAwXSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2ln
bmVkIGJ1cyBudW1iZXIgMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjY5
ODkwMl0geGhjaV9oY2QgMDAwNDowMzowMC4wOiBIb3N0IHN1cHBvcnRzIFVTQiAzLjAgU3VwZXJT
cGVlZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMjk3Ml0gdXNiIHVz
YjI6IFdlIGRvbid0IGtub3cgdGhlIGFsZ29yaXRobXMgZm9yIExQTSBmb3IgdGhpcyBob3N0LCBk
aXNhYmxpbmcgTFBNLgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMjk4
NV0gdXNiIHVzYjI6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0xZDZiLCBpZFByb2R1
Y3Q9MDAwMywgYmNkRGV2aWNlPSA1LjEwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNzEyOTg2XSB1c2IgdXNiMjogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZyPTMsIFBy
b2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS43MTI5ODddIHVzYiB1c2IyOiBQcm9kdWN0OiB4SENJIEhvc3QgQ29udHJvbGxlcgpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMjk4OF0gdXNiIHVzYjI6IE1hbnVm
YWN0dXJlcjogTGludXggNS4xMC4wLTgtYXJtNjQgeGhjaS1oY2QKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS43MTI5ODhdIHVzYiB1c2IyOiBTZXJpYWxOdW1iZXI6IDAwMDQ6
MDM6MDAuMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMzAxM10gY2hl
Y2tpbmcgZ2VuZXJpYyAoMjAwMDAwMDAgMWQ1MDAwKSB2cyBodyAoMjAwMDAwMDAgMjAwMDAwMCkK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS43MTMwMTRdIGZiMDogc3dpdGNo
aW5nIHRvIGFzdGRybWZiIGZyb20gRUZJIFZHQQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjcxMzA5MV0gQ29uc29sZTogc3dpdGNoaW5nIHRvIGNvbG91ciBkdW1teSBkZXZp
Y2UgODB4MjUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS43MTMxNDRdIGFz
dCAwMDA0OjAyOjAwLjA6IFtkcm1dIHBsYXRmb3JtIGhhcyBubyBJTyBzcGFjZSwgdHJ5aW5nIE1N
SU8KU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS43MTMxNTFdIGFzdCAwMDA0
OjAyOjAwLjA6IFtkcm1dIFAyQSBicmlkZ2UgZGlzYWJsZWQsIHVzaW5nIGRlZmF1bHQgY29uZmln
dXJhdGlvbgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMzE1MV0gYXN0
IDAwMDQ6MDI6MDAuMDogW2RybV0gQVNUIDI1MDAgZGV0ZWN0ZWQKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS43MTMxNTZdIGFzdCAwMDA0OjAyOjAwLjA6IFtkcm1dIEFuYWxv
ZyBWR0Egb25seQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMzE1N10g
YXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gZHJhbSBNQ0xLPTgwMCBNaHogdHlwZT0xIGJ1c193aWR0
aD0xNgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjcxMzE4N10gaHViIDIt
MDoxLjA6IFVTQiBodWIgZm91bmQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS43MTMxOTZdIGh1YiAyLTA6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNzEzMjQ4XSBbVFRNXSBab25lICBrZXJuZWw6IEF2YWlsYWJs
ZSBncmFwaGljcyBtZW1vcnk6IDMyNjUyNDA2IEtpQgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjcxMzI0OV0gW1RUTV0gWm9uZSAgIGRtYTMyOiBBdmFpbGFibGUgZ3JhcGhp
Y3MgbWVtb3J5OiAyMDk3MTUyIEtpQgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjcxMzI0OV0gW1RUTV0gSW5pdGlhbGl6aW5nIHBvb2wgYWxsb2NhdG9yClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzEzMjUyXSBbVFRNXSBJbml0aWFsaXppbmcgRE1B
IHBvb2wgYWxsb2NhdG9yClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuNzEz
NDUwXSBbZHJtXSBJbml0aWFsaXplZCBhc3QgMC4xLjAgMjAxMjAyMjggZm9yIDAwMDQ6MDI6MDAu
MCBvbiBtaW5vciAwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODI3Mjcx
XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMjQweDY3
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODI5OTMxXSBhc3QgMDAwNDow
MjowMC4wOiBbZHJtXSBmYjA6IGFzdGRybWZiIGZyYW1lIGJ1ZmZlciBkZXZpY2UKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTgzMDFdIGlnYiAwMDAzOjAyOjAwLjA6IGFk
ZGVkIFBIQyBvbiBldGgwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuODU4
MzAzXSBpZ2IgMDAwMzowMjowMC4wOiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsg
Q29ubmVjdGlvbgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1ODMwNF0g
aWdiIDAwMDM6MDI6MDAuMDogZXRoMDogKFBDSWU6NS4wR2IvczpXaWR0aCB4MikgZDg6NWU6ZDM6
MDI6YjE6OTMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44NTgzODVdIGln
YiAwMDAzOjAyOjAwLjA6IGV0aDA6IFBCQSBObzogMTA2MzAwLTAwMApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjg1ODM4Nl0gaWdiIDAwMDM6MDI6MDAuMDogVXNpbmcgTVNJ
LVggaW50ZXJydXB0cy4gOCByeCBxdWV1ZShzKSwgOCB0eCBxdWV1ZShzKQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjg1ODQ0OF0gaWdiIDAwMDM6MDI6MDAuMTogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDI1ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEu
OTEyMzEwXSBpZ2IgMDAwMzowMjowMC4xOiBhZGRlZCBQSEMgb24gZXRoMQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxMjMxMV0gaWdiIDAwMDM6MDI6MDAuMTogSW50ZWwo
UikgR2lnYWJpdCBFdGhlcm5ldCBOZXR3b3JrIENvbm5lY3Rpb24KU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS45MTIzMTJdIGlnYiAwMDAzOjAyOjAwLjE6IGV0aDE6IChQQ0ll
OjUuMEdiL3M6V2lkdGggeDIpIGQ4OjVlOmQzOjAyOmIxOjk0ClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuOTEyMzk0XSBpZ2IgMDAwMzowMjowMC4xOiBldGgxOiBQQkEgTm86
IDEwNjMwMC0wMDAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MTIzOTVd
IGlnYiAwMDAzOjAyOjAwLjE6IFVzaW5nIE1TSS1YIGludGVycnVwdHMuIDggcnggcXVldWUocyks
IDggdHggcXVldWUocykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MTMz
ODRdIGlnYiAwMDAzOjAyOjAwLjAgZW5vMTogcmVuYW1lZCBmcm9tIGV0aDAKU2VwIDEzIDE1OjQw
OjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MTUwODhdIG1wdDNzYXNfY20wOiBDdXJyZW50SG9z
dFBhZ2VTaXplIGlzIDA6IFNldHRpbmcgZGVmYXVsdCBob3N0IHBhZ2Ugc2l6ZSB0byA0awpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNTEwNl0gbXB0M3Nhc19jbTA6IE1T
SS1YIHZlY3RvcnMgc3VwcG9ydGVkOiA5NgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjkxNTEwN10gCSBubyBvZiBjb3JlczogODAsIG1heF9tc2l4X3ZlY3RvcnM6IC0xClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTE1MTA5XSBtcHQzc2FzX2NtMDog
IDAgODAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MTY0MDFdIG1wdDNz
YXNfY20wOiBIaWdoIElPUHMgcXVldWVzIDogZGlzYWJsZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMS45MTY0MDJdIG1wdDNzYXMwLW1zaXgwOiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDQ2NQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQwM10g
bXB0M3NhczAtbXNpeDE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY2ClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTE2NDAzXSBtcHQzc2FzMC1tc2l4MjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0NjcKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45
MTY0MDRdIG1wdDNzYXMwLW1zaXgzOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2OApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQwNF0gbXB0M3NhczAtbXNpeDQ6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNDY5ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuOTE2NDA1XSBtcHQzc2FzMC1tc2l4NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzAK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45MTY0MDZdIG1wdDNzYXMwLW1z
aXg2OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjkxNjQwNl0gbXB0M3NhczAtbXNpeDc6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNDcyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTE2NDA3XSBtcHQz
c2FzMC1tc2l4ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzMKU2VwIDEzIDE1OjQwOjQwIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS45MTY0MDhdIG1wdDNzYXMwLW1zaXg5OiBQQ0ktTVNJLVggZW5h
YmxlZDogSVJRIDQ3NApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQw
OF0gbXB0M3NhczAtbXNpeDEwOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3NQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQwOV0gbXB0M3NhczAtbXNpeDExOiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDQ3NgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjkxNjQxMF0gbXB0M3NhczAtbXNpeDEyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3NwpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxMF0gbXB0M3NhczAtbXNp
eDEzOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjkxNjQxMV0gbXB0M3NhczAtbXNpeDE0OiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDQ3OQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxMV0gbXB0
M3NhczAtbXNpeDE1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4MApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxMl0gbXB0M3NhczAtbXNpeDE2OiBQQ0ktTVNJLVgg
ZW5hYmxlZDogSVJRIDQ4MQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkx
NjQxM10gbXB0M3NhczAtbXNpeDE3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4MgpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxM10gbXB0M3NhczAtbXNpeDE4OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4MwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjkxNjQxNF0gbXB0M3NhczAtbXNpeDE5OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4
NApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxNF0gbXB0M3NhczAt
bXNpeDIwOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4NQpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjkxNjQxNV0gbXB0M3NhczAtbXNpeDIxOiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDQ4NgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxNl0g
bXB0M3NhczAtbXNpeDIyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4NwpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxNl0gbXB0M3NhczAtbXNpeDIzOiBQQ0ktTVNJ
LVggZW5hYmxlZDogSVJRIDQ4OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjkxNjQxN10gbXB0M3NhczAtbXNpeDI0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ4OQpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxOF0gbXB0M3NhczAtbXNpeDI1
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjkxNjQxOF0gbXB0M3NhczAtbXNpeDI2OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJR
IDQ5MQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQxOV0gbXB0M3Nh
czAtbXNpeDI3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5MgpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjkxNjQxOV0gbXB0M3NhczAtbXNpeDI4OiBQQ0ktTVNJLVggZW5h
YmxlZDogSVJRIDQ5MwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQy
MF0gbXB0M3NhczAtbXNpeDI5OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5NApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyMV0gbXB0M3NhczAtbXNpeDMwOiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDQ5NQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjkxNjQyMV0gbXB0M3NhczAtbXNpeDMxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5NgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyMl0gbXB0M3NhczAtbXNp
eDMyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5NwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjkxNjQyMl0gbXB0M3NhczAtbXNpeDMzOiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDQ5OApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyM10gbXB0
M3NhczAtbXNpeDM0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ5OQpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyM10gbXB0M3NhczAtbXNpeDM1OiBQQ0ktTVNJLVgg
ZW5hYmxlZDogSVJRIDUwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkx
NjQyNF0gbXB0M3NhczAtbXNpeDM2OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUwMQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyNV0gbXB0M3NhczAtbXNpeDM3OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDUwMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjkxNjQyNV0gbXB0M3NhczAtbXNpeDM4OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUw
MwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyNl0gbXB0M3NhczAt
bXNpeDM5OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUwNApTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjkxNjQyNl0gbXB0M3NhczAtbXNpeDQwOiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDUwNQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyN10g
bXB0M3NhczAtbXNpeDQxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUwNgpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyN10gbXB0M3NhczAtbXNpeDQyOiBQQ0ktTVNJ
LVggZW5hYmxlZDogSVJRIDUwNwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjkxNjQyOF0gbXB0M3NhczAtbXNpeDQzOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUwOApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQyOV0gbXB0M3NhczAtbXNpeDQ0
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUwOQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjkxNjQyOV0gbXB0M3NhczAtbXNpeDQ1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJR
IDUxMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzMF0gbXB0M3Nh
czAtbXNpeDQ2OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUxMQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjkxNjQzMF0gbXB0M3NhczAtbXNpeDQ3OiBQQ0ktTVNJLVggZW5h
YmxlZDogSVJRIDUxMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQz
MV0gbXB0M3NhczAtbXNpeDQ4OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUxMwpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzMl0gbXB0M3NhczAtbXNpeDQ5OiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDUxNApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjkxNjQzMl0gbXB0M3NhczAtbXNpeDUwOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUxNQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzM10gbXB0M3NhczAtbXNp
eDUxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUxNgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjkxNjQzM10gbXB0M3NhczAtbXNpeDUyOiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDUxNwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzNF0gbXB0
M3NhczAtbXNpeDUzOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUxOApTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzNV0gbXB0M3NhczAtbXNpeDU0OiBQQ0ktTVNJLVgg
ZW5hYmxlZDogSVJRIDUxOQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkx
NjQzNV0gbXB0M3NhczAtbXNpeDU1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUyMApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzNl0gbXB0M3NhczAtbXNpeDU2OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDUyMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjkxNjQzNl0gbXB0M3NhczAtbXNpeDU3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUy
MgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzN10gbXB0M3NhczAt
bXNpeDU4OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUyMwpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjkxNjQzOF0gbXB0M3NhczAtbXNpeDU5OiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDUyNApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzOF0g
bXB0M3NhczAtbXNpeDYwOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUyNQpTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQzOV0gbXB0M3NhczAtbXNpeDYxOiBQQ0ktTVNJ
LVggZW5hYmxlZDogSVJRIDUyNgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAx
LjkxNjQzOV0gbXB0M3NhczAtbXNpeDYyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUyNwpTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0MF0gbXB0M3NhczAtbXNpeDYz
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUyOApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAxLjkxNjQ0MV0gbXB0M3NhczAtbXNpeDY0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJR
IDUyOQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0MV0gbXB0M3Nh
czAtbXNpeDY1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzMApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0Ml0gbXB0M3NhczAtbXNpeDY2OiBQQ0ktTVNJLVggZW5h
YmxlZDogSVJRIDUzMQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0
M10gbXB0M3NhczAtbXNpeDY3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzMgpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0M10gbXB0M3NhczAtbXNpeDY4OiBQQ0kt
TVNJLVggZW5hYmxlZDogSVJRIDUzMwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjkxNjQ0NF0gbXB0M3NhczAtbXNpeDY5OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzNApT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0NF0gbXB0M3NhczAtbXNp
eDcwOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzNQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAxLjkxNjQ0NV0gbXB0M3NhczAtbXNpeDcxOiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDUzNgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0Nl0gbXB0
M3NhczAtbXNpeDcyOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzNwpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0Nl0gbXB0M3NhczAtbXNpeDczOiBQQ0ktTVNJLVgg
ZW5hYmxlZDogSVJRIDUzOApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkx
NjQ0N10gbXB0M3NhczAtbXNpeDc0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDUzOQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0OF0gbXB0M3NhczAtbXNpeDc1OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDU0MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDog
WyAgICAxLjkxNjQ0OF0gbXB0M3NhczAtbXNpeDc2OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDU0
MQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ0OV0gbXB0M3NhczAt
bXNpeDc3OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDU0MgpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAxLjkxNjQ0OV0gbXB0M3NhczAtbXNpeDc4OiBQQ0ktTVNJLVggZW5hYmxl
ZDogSVJRIDU0MwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ1MF0g
bXB0M3NhczAtbXNpeDc5OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDU0NApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ1MV0gbXB0M3Nhc19jbTA6IGlvbWVtKDB4MDAw
MDAwMDA0MDE0MDAwMCksIG1hcHBlZCgweChfX19fcHRydmFsX19fXykpLCBzaXplKDY1NTM2KQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAxLjkxNjQ1Ml0gbXB0M3Nhc19jbTA6
IGlvcG9ydCgweDAwMDAwMDAwMDAwMDAwMDApLCBzaXplKDApClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDEuOTI1OTQ3XSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IHJlbmFtZWQg
ZnJvbSBldGgxClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTczNzU0XSBt
cHQzc2FzX2NtMDogQ3VycmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9z
dCBwYWdlIHNpemUgdG8gNGsKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45
NzM3NTddIG1wdDNzYXNfY20wOiBzZW5kaW5nIGRpYWcgcmVzZXQgISEKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4wMjk3NjJdIHVzYiAxLTE6IG5ldyBsb3ctc3BlZWQgVVNC
IGRldmljZSBudW1iZXIgMiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtl
cm5lbDogWyAgICAyLjIyMjAzMl0gdXNiIDEtMTogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVu
ZG9yPTQxM2MsIGlkUHJvZHVjdD0zMDFhLCBiY2REZXZpY2U9IDEuMDAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yMjIwMzNdIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIHN0
cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxNTo0MDo0MCBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMjIyMDM0XSB1c2IgMS0xOiBQcm9kdWN0OiBEZWxsIE1TMTE2
IFVTQiBPcHRpY2FsIE1vdXNlClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDIu
MjIyMDM1XSB1c2IgMS0xOiBNYW51ZmFjdHVyZXI6IFBpeEFydApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAyLjIzMTA2OV0gaGlkOiByYXcgSElEIGV2ZW50cyBkcml2ZXIgKEMp
IEppcmkgS29zaW5hClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjQzMDky
XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVzYmhpZApTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjI0MzA5M10gdXNiaGlkOiBVU0IgSElEIGNv
cmUgZHJpdmVyClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjQ0MzI2XSBp
bnB1dDogUGl4QXJ0IERlbGwgTVMxMTYgVVNCIE9wdGljYWwgTW91c2UgYXMgL2RldmljZXMvcGNp
MDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0xLzEtMToxLjAvMDAwMzo0
MTNDOjMwMUEuMDAwMS9pbnB1dC9pbnB1dDEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMi4yNDQzNzBdIGhpZC1nZW5lcmljIDAwMDM6NDEzQzozMDFBLjAwMDE6IGlucHV0LGhp
ZHJhdzA6IFVTQiBISUQgdjEuMTEgTW91c2UgW1BpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2Fs
IE1vdXNlXSBvbiB1c2ItMDAwNDowMzowMC4wLTEvaW5wdXQwClNlcCAxMyAxNTo0MDo0MCBkZWJp
YW4ga2VybmVsOiBbICAgIDIuMzIyNDI3XSB1c2IgMi0zOiBuZXcgU3VwZXJTcGVlZCBHZW4gMSBV
U0IgZGV2aWNlIG51bWJlciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDIuMzUwOTYyXSB1c2IgMi0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRW
ZW5kb3I9MDVlMywgaWRQcm9kdWN0PTA2MjAsIGJjZERldmljZT05My4xMApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjM1MDk2NF0gdXNiIDItMzogTmV3IFVTQiBkZXZpY2Ug
c3RyaW5nczogTWZyPTEsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zNTA5NjVdIHVzYiAyLTM6IFByb2R1Y3Q6IFVTQjMuMSBI
dWIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zNTA5NjddIHVzYiAyLTM6
IE1hbnVmYWN0dXJlcjogR2VuZXN5c0xvZ2ljClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDIuMzUzOTgwXSBodWIgMi0zOjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTU6NDA6
NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjM1NDU4NV0gaHViIDItMzoxLjA6IDQgcG9ydHMgZGV0
ZWN0ZWQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi40ODE3NTNdIHVzYiAx
LTI6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmljZSBudW1iZXIgMyB1c2luZyB4aGNpX2hjZApTZXAg
MTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjY4OTAyNl0gdXNiIDEtMjogTmV3IFVT
QiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTA0ZDksIGlkUHJvZHVjdD0xMjAzLCBiY2REZXZpY2U9
IDIuODAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi42ODkwMjZdIHVzYiAx
LTI6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0wLCBQcm9kdWN0PTAsIFNlcmlhbE51bWJl
cj0wClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzI2MTg1XSBpbnB1dDog
SElEIDA0ZDk6MTIwMyBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAz
OjAwLjAvdXNiMS8xLTIvMS0yOjEuMC8wMDAzOjA0RDk6MTIwMy4wMDAyL2lucHV0L2lucHV0MgpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjc4OTg0NV0gaGlkLWdlbmVyaWMg
MDAwMzowNEQ5OjEyMDMuMDAwMjogaW5wdXQsaGlkcmF3MTogVVNCIEhJRCB2MS4xMSBLZXlib2Fy
ZCBbSElEIDA0ZDk6MTIwM10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0MApTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAyLjgzMjE1MF0gaW5wdXQ6IEhJRCAwNGQ5OjEyMDMg
U3lzdGVtIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDow
MzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDMK
U2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi44ODk3NjldIGlucHV0OiBISUQg
MDRkOToxMjAzIENvbnN1bWVyIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAw
OjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9p
bnB1dC9pbnB1dDQKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMi44ODk3ODld
IGhpZC1nZW5lcmljIDAwMDM6MDREOToxMjAzLjAwMDM6IGlucHV0LGhpZHJhdzI6IFVTQiBISUQg
djEuMTEgRGV2aWNlIFtISUQgMDRkOToxMjAzXSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQx
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDEzMjYzXSBtcHQzc2FzX2Nt
MDogZGlhZyByZXNldDogU1VDQ0VTUwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjAyMTc1Ml0gdXNiIDEtMzogbmV3IGhpZ2gtc3BlZWQgVVNCIGRldmljZSBudW1iZXIgNCB1
c2luZyB4aGNpX2hjZApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjA0ODI3
OF0gbXB0M3Nhc19jbTA6IHNjYXR0ZXIgZ2F0aGVyOiBzZ2VfaW5fbWFpbl9tc2coMSksIHNnZV9w
ZXJfY2hhaW4oNyksIHNnZV9wZXJfaW8oMTI4KSwgY2hhaW5zX3Blcl9pbygxOSkKU2VwIDEzIDE1
OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wNDkzMzJdIG1wdDNzYXNfY20wOiByZXF1ZXN0
IHBvb2woMHgoX19fX3B0cnZhbF9fX18pKSAtIGRtYSgweGZmNTAwMDAwKTogZGVwdGgoNDIyNCks
IGZyYW1lX3NpemUoMTI4KSwgcG9vbF9zaXplKDUyOCBrQikKU2VwIDEzIDE1OjQwOjQwIGRlYmlh
biBrZXJuZWw6IFsgICAgMy4wNjk1NTldIG1wdDNzYXNfY20wOiBzZW5zZSBwb29sKDB4KF9fX19w
dHJ2YWxfX19fKSktIGRtYSgweGZlYjAwMDAwKTogZGVwdGgoMzk2MyksZWxlbWVudF9zaXplKDk2
KSwgcG9vbF9zaXplKDM3MSBrQikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
My4wNjk2NThdIG1wdDNzYXNfY20wOiBjb25maWcgcGFnZSgweChfX19fcHRydmFsX19fXykpIC0g
ZG1hKDB4ZmU5ZjYwMDApOiBzaXplKDUxMikKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMy4wNjk2NTldIG1wdDNzYXNfY20wOiBBbGxvY2F0ZWQgcGh5c2ljYWwgbWVtb3J5OiBz
aXplKDcyNzgga0IpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDY5NjU5
XSBtcHQzc2FzX2NtMDogQ3VycmVudCBDb250cm9sbGVyIFF1ZXVlIERlcHRoKDM5NjApLE1heCBD
b250cm9sbGVyIFF1ZXVlIERlcHRoKDQwOTYpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVs
OiBbICAgIDMuMDY5NjYwXSBtcHQzc2FzX2NtMDogU2NhdHRlciBHYXRoZXIgRWxlbWVudHMgcGVy
IElPKDEyOCkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNzkxOTRdIHVz
YiAxLTM6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNWUzLCBpZFByb2R1Y3Q9MDYx
MCwgYmNkRGV2aWNlPTkzLjEwClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
MTc5MTk2XSB1c2IgMS0zOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0y
LCBTZXJpYWxOdW1iZXI9MApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjE3
OTE5N10gdXNiIDEtMzogUHJvZHVjdDogVVNCMi4wIEh1YgpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAzLjE3OTE5N10gdXNiIDEtMzogTWFudWZhY3R1cmVyOiBHZW5lc3lzTG9n
aWMKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xODMyMTRdIGh1YiAxLTM6
MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
MTgzODIxXSBodWIgMS0zOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjI3MDQwNV0gbXB0M3Nhc19jbTA6IF9iYXNlX2Rpc3BsYXlfZndw
a2dfdmVyc2lvbjogY29tcGxldGUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAg
My4yNzA3MzJdIG1wdDNzYXNfY20wOiBMU0lTQVMzMDA4OiBGV1ZlcnNpb24oMTYuMDAuMTAuMDAp
LCBDaGlwUmV2aXNpb24oMHgwMiksIEJpb3NWZXJzaW9uKDE1LjAwLjAxLjAwKQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjI3MDczMl0gbXB0M3Nhc19jbTA6IFByb3RvY29s
PShJbml0aWF0b3IpLCBDYXBhYmlsaXRpZXM9KFJhaWQsVExSLEVFRFAsU25hcHNob3QgQnVmZmVy
LERpYWcgVHJhY2UgQnVmZmVyLFRhc2sgU2V0IEZ1bGwsTkNRKQpTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjI3MDc4Ml0gc2NzaSBob3N0MDogRnVzaW9uIE1QVCBTQVMgSG9z
dApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjI3MTMzOF0gbXB0M3Nhc19j
bTA6IHNlbmRpbmcgcG9ydCBlbmFibGUgISEKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMy4zMDIyNTFdIFBNOiBJbWFnZSBub3QgZm91bmQgKGNvZGUgLTIyKQpTZXAgMTMgMTU6
NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjM1MTc1OV0gRVhUNC1mcyAobnZtZTNuMXAyKTog
bW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpT
ZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjM4NDE5NF0gTm90IGFjdGl2YXRp
bmcgTWFuZGF0b3J5IEFjY2VzcyBDb250cm9sIGFzIC9zYmluL3RvbW95by1pbml0IGRvZXMgbm90
IGV4aXN0LgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjUxNDcwNl0gcmFu
ZG9tOiBsdm1jb25maWc6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICg0IGJ5dGVzIHJlYWQp
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTgyNzIxXSByYW5kb206IHN5
c3RlbWQ6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjU4MzMyN10gcmFuZG9tOiBzeXN0ZW1kOiB1
bmluaXRpYWxpemVkIHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy42MDA2MTJdIGZ1c2U6IGluaXQgKEFQSSB2ZXJzaW9uIDcu
MzIpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNjAyNDcyXSBFWFQ0LWZz
IChudm1lM24xcDIpOiByZS1tb3VudGVkLiBPcHRzOiBlcnJvcnM9cmVtb3VudC1ybwpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjY5NzAwOV0gYWNwaS10YWQgQUNQSTAwMEU6
MDA6IFVuc3VwcG9ydGVkIGNhcGFiaWxpdGllcwpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5l
bDogWyAgICAzLjc0OTIzNF0geGdlbmUtc2xpbXByby1od21vbiBBUE1DMEQyOTowMDogQVBNIFgt
R2VuZSBTb0MgSFcgbW9uaXRvciBkcml2ZXIgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NDA6NDAgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjc1NTU3NV0gSVBNSSBtZXNzYWdlIGhhbmRsZXI6IHZlcnNpb24g
MzkuMgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjc1OTMwNF0gaXBtaSBk
ZXZpY2UgaW50ZXJmYWNlClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzY1
NjUwXSBpcG1pX3NpOiBJUE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVyClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzY1OTU3XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBh
bnkgU3lzdGVtIEludGVyZmFjZShzKQpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjgzNzYyM10gaXBtaV9zc2lmOiBJUE1JIFNTSUYgSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMg
MTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjg0MTk1M10gaXBtaV9zc2lmIGkyYy1BTVBD
MDAwNDowMDogaXBtaV9zc2lmOiBUcnlpbmcgQUNQSS1zcGVjaWZpZWQgU1NJRiBpbnRlcmZhY2Ug
YXQgaTJjIGFkZHJlc3MgMHgxMCwgYWRhcHRlciBTeW5vcHN5cyBEZXNpZ25XYXJlIEkyQyBhZGFw
dGVyLCBzbGF2ZSBhZGRyZXNzIDB4MjAKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy44NDM4MjBdIEVYVDQtZnMgKG52bWUzbjFwNSk6IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRo
IG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy44NDM4OTBdIEVYVDQtZnMgKG52bWUzbjFwMyk6IG1vdW50ZWQgZmlsZXN5
c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDE1OjQwOjQw
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NDk4MDNdIEFkZGluZyAxMDAwNDQ0ayBzd2FwIG9uIC9k
ZXYvbnZtZTNuMXA0LiAgUHJpb3JpdHk6LTIgZXh0ZW50czoxIGFjcm9zczoxMDAwNDQ0ayBTU0ZT
ClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODY0ODkwXSBFWFQ0LWZzIChu
dm1lM24xcDYpOiBtb3VudGVkIGZpbGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0
czogKG51bGwpClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODkyNzg5XSBz
YnNhLWd3ZHQgc2JzYS1nd2R0LjA6IEluaXRpYWxpemVkIHdpdGggMTBzIHRpbWVvdXQgQCAyNTAw
MDAwMCBIeiwgYWN0aW9uPTAuClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMu
ODk4MDk4XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1Mjc4NDAuMTU2OjIpOiBhcHBhcm1v
cj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIg
bmFtZT0ibHNiX3JlbGVhc2UiIHBpZD0xMDk4IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDEz
IDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44OTk1MDBdIGF1ZGl0OiB0eXBlPTE0MDAg
YXVkaXQoMTYzMTUyNzg0MC4xNTY6Myk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJv
ZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJyZW9mZmljZS1zZW5kZG9j
IiBwaWQ9MTEwNiBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4g
a2VybmVsOiBbICAgIDMuODk5Njc2XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1Mjc4NDAu
MTU2OjQpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmls
ZT0idW5jb25maW5lZCIgbmFtZT0ibGlidmlydGQiIHBpZD0xMTA0IGNvbW09ImFwcGFybW9yX3Bh
cnNlciIKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44OTk2NzldIGF1ZGl0
OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUyNzg0MC4xNTY6NSk6IGFwcGFybW9yPSJTVEFUVVMiIG9w
ZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsaWJ2aXJ0
ZC8vcWVtdV9icmlkZ2VfaGVscGVyIiBwaWQ9MTEwNCBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNl
cCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuODk5Nzc5XSBhdWRpdDogdHlwZT0x
NDAwIGF1ZGl0KDE2MzE1Mjc4NDAuMTU2OjYpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249
InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2UteHBk
ZmltcG9ydCIgcGlkPTExMDIgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6NDA6NDAg
ZGViaWFuIGtlcm5lbDogWyAgICAzLjkwMDkxMl0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMx
NTI3ODQwLjE1Njo3KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQi
IHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im52aWRpYV9tb2Rwcm9iZSIgcGlkPTExMDkgY29t
bT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAz
LjkwMDkxNF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI3ODQwLjE1Njo4KTogYXBwYXJt
b3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQi
IG5hbWU9Im52aWRpYV9tb2Rwcm9iZS8va21vZCIgcGlkPTExMDkgY29tbT0iYXBwYXJtb3JfcGFy
c2VyIgpTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwMTM3NV0gYXVkaXQ6
IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI3ODQwLjE1Njo5KTogYXBwYXJtb3I9IlNUQVRVUyIgb3Bl
cmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxpYnJlb2Zm
aWNlLW9vcHNsYXNoIiBwaWQ9MTEwMyBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNTo0
MDo0MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuOTA1NjAxXSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0
KDE2MzE1Mjc4NDAuMTYwOjEwKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxl
X2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9InZpcnQtYWEtaGVscGVyIiBwaWQ9MTEw
OCBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNTo0MDo0MCBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuOTA2NDExXSBwc3RvcmU6IFVzaW5nIGNyYXNoIGR1bXAgY29tcHJlc3Npb246IGRlZmxh
dGUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy45MDY0MTddIHBzdG9yZTog
UmVnaXN0ZXJlZCBlZmkgYXMgcGVyc2lzdGVudCBzdG9yZSBiYWNrZW5kClNlcCAxMyAxNTo0MDo0
MCBkZWJpYW4ga2VybmVsOiBbICAgIDMuOTI1MzQzXSBjcnlwdGQ6IG1heF9jcHVfcWxlbiBzZXQg
dG8gMTAwMApTZXAgMTMgMTU6NDA6NDAgZGViaWFuIGtlcm5lbDogWyAgICAzLjk0Mjk0Ml0gaXBt
aV9zaTogSVBNSSBTeXN0ZW0gSW50ZXJmYWNlIGRyaXZlcgpTZXAgMTMgMTU6NDA6NDAgZGViaWFu
IGtlcm5lbDogWyAgICAzLjk0MzA5Nl0gaXBtaV9zaTogVW5hYmxlIHRvIGZpbmQgYW55IFN5c3Rl
bSBJbnRlcmZhY2UocykKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgMy45NzAz
OTZdIHJhbmRvbTogY3JuZyBpbml0IGRvbmUKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6
IFsgICAgMy45NzAzOTldIHJhbmRvbTogNyB1cmFuZG9tIHdhcm5pbmcocykgbWlzc2VkIGR1ZSB0
byByYXRlbGltaXRpbmcKU2VwIDEzIDE1OjQwOjQwIGRlYmlhbiBrZXJuZWw6IFsgICAgNC4wOTE1
NjhdIGlwbWlfc3NpZiBpMmMtQU1QQzAwMDQ6MDA6IElQTUkgbWVzc2FnZSBoYW5kbGVyOiBGb3Vu
ZCBuZXcgQk1DIChtYW5faWQ6IDB4MDAzYzBhLCBwcm9kX2lkOiAweDAyMTIsIGRldl9pZDogMHgy
MCkKU2VwIDEzIDE1OjQwOjQyIGRlYmlhbiBrZXJuZWw6IFsgICAgNS44MjM0NDRdIG1wdDNzYXNf
Y20wOiBob3N0X2FkZDogaGFuZGxlKDB4MDAwMSksIHNhc19hZGRyKDB4NTAwMTQ4NTAwMDc3OWUz
MCksIHBoeXMoOCkKU2VwIDEzIDE1OjQwOjQzIGRlYmlhbiBrZXJuZWw6IFsgICAgNy4xODYyNjBd
IGlnYiAwMDAzOjAyOjAwLjEgZW5vMjogaWdiOiBlbm8yIE5JQyBMaW5rIGlzIFVwIDEwMDAgTWJw
cyBGdWxsIER1cGxleCwgRmxvdyBDb250cm9sOiBSWC9UWApTZXAgMTMgMTU6NDA6NDMgZGViaWFu
IGtlcm5lbDogWyAgICA3LjE4NjQzN10gSVB2NjogQUREUkNPTkYoTkVUREVWX0NIQU5HRSk6IGVu
bzI6IGxpbmsgYmVjb21lcyByZWFkeQpTZXAgMTMgMTU6NDA6NDcgZGViaWFuIGtlcm5lbDogWyAg
IDEwLjk0OTc4M10gbXB0M3Nhc19jbTA6IHBvcnQgZW5hYmxlOiBTVUNDRVNTClNlcCAxMyAxNTo0
NDowMCBkZWJpYW4ga2VybmVsOiBbICAyMDUuMzEwODI5XSBkZXZpY2UtbWFwcGVyOiB1ZXZlbnQ6
IHZlcnNpb24gMS4wLjMKU2VwIDEzIDE1OjQ0OjAwIGRlYmlhbiBrZXJuZWw6IFsgIDIwNS4zMTA5
MjBdIGRldmljZS1tYXBwZXI6IGlvY3RsOiA0LjQzLjAtaW9jdGwgKDIwMjAtMTAtMDEpIGluaXRp
YWxpc2VkOiBkbS1kZXZlbEByZWRoYXQuY29tClNlcCAxMyAxNTo0NDowNCBkZWJpYW4ga2VybmVs
OiBbICAyMDguNzgwMDYyXSBTR0kgWEZTIHdpdGggQUNMcywgc2VjdXJpdHkgYXR0cmlidXRlcywg
cmVhbHRpbWUsIHF1b3RhLCBubyBkZWJ1ZyBlbmFibGVkClNlcCAxMyAxNTo0NDowNCBkZWJpYW4g
a2VybmVsOiBbICAyMDguNzg3Mzc5XSBKRlM6IG5UeEJsb2NrID0gODE5MiwgblR4TG9jayA9IDY1
NTM2ClNlcCAxMyAxNTo0NDowNCBkZWJpYW4ga2VybmVsOiBbICAyMDguODE0NDAxXSBRTlg0IGZp
bGVzeXN0ZW0gMC4yLjMgcmVnaXN0ZXJlZC4KU2VwIDEzIDE1OjQ0OjA0IGRlYmlhbiBrZXJuZWw6
IFsgIDIwOC44ODk1OTJdIHJhaWQ2OiBuZW9ueDggICBnZW4oKSAxMjk4MSBNQi9zClNlcCAxMyAx
NTo0NDowNCBkZWJpYW4ga2VybmVsOiBbICAyMDguOTU3NTkwXSByYWlkNjogbmVvbng4ICAgeG9y
KCkgMTAxMzUgTUIvcwpTZXAgMTMgMTU6NDQ6MDQgZGViaWFuIGtlcm5lbDogWyAgMjA5LjAyNTU5
MF0gcmFpZDY6IG5lb254NCAgIGdlbigpIDEyNzU4IE1CL3MKU2VwIDEzIDE1OjQ0OjA0IGRlYmlh
biBrZXJuZWw6IFsgIDIwOS4wOTM1ODhdIHJhaWQ2OiBuZW9ueDQgICB4b3IoKSAxMDM0NyBNQi9z
ClNlcCAxMyAxNTo0NDowNCBkZWJpYW4ga2VybmVsOiBbICAyMDkuMTYxNTg5XSByYWlkNjogbmVv
bngyICAgZ2VuKCkgMTIxODIgTUIvcwpTZXAgMTMgMTU6NDQ6MDQgZGViaWFuIGtlcm5lbDogWyAg
MjA5LjIyOTU4N10gcmFpZDY6IG5lb254MiAgIHhvcigpICA5NjI3IE1CL3MKU2VwIDEzIDE1OjQ0
OjA0IGRlYmlhbiBrZXJuZWw6IFsgIDIwOS4yOTc1ODZdIHJhaWQ2OiBuZW9ueDEgICBnZW4oKSAg
OTgzNiBNQi9zClNlcCAxMyAxNTo0NDowNCBkZWJpYW4ga2VybmVsOiBbICAyMDkuMzY1NTg2XSBy
YWlkNjogbmVvbngxICAgeG9yKCkgIDgzNjcgTUIvcwpTZXAgMTMgMTU6NDQ6MDQgZGViaWFuIGtl
cm5lbDogWyAgMjA5LjQzMzU4N10gcmFpZDY6IGludDY0eDggIGdlbigpICA1OTY0IE1CL3MKU2Vw
IDEzIDE1OjQ0OjA0IGRlYmlhbiBrZXJuZWw6IFsgIDIwOS41MDE1ODZdIHJhaWQ2OiBpbnQ2NHg4
ICB4b3IoKSAgMzU1MSBNQi9zClNlcCAxMyAxNTo0NDowNCBkZWJpYW4ga2VybmVsOiBbICAyMDku
NTY5NTg0XSByYWlkNjogaW50NjR4NCAgZ2VuKCkgIDY3NzMgTUIvcwpTZXAgMTMgMTU6NDQ6MDQg
ZGViaWFuIGtlcm5lbDogWyAgMjA5LjYzNzU4NF0gcmFpZDY6IGludDY0eDQgIHhvcigpICAzNTgw
IE1CL3MKU2VwIDEzIDE1OjQ0OjA0IGRlYmlhbiBrZXJuZWw6IFsgIDIwOS43MDU1ODRdIHJhaWQ2
OiBpbnQ2NHgyICBnZW4oKSAgNTc2OCBNQi9zClNlcCAxMyAxNTo0NDowNSBkZWJpYW4ga2VybmVs
OiBbICAyMDkuNzczNTgxXSByYWlkNjogaW50NjR4MiAgeG9yKCkgIDMxMDMgTUIvcwpTZXAgMTMg
MTU6NDQ6MDUgZGViaWFuIGtlcm5lbDogWyAgMjA5Ljg0MTU4MF0gcmFpZDY6IGludDY0eDEgIGdl
bigpICA0NzMyIE1CL3MKU2VwIDEzIDE1OjQ0OjA1IGRlYmlhbiBrZXJuZWw6IFsgIDIwOS45MDk1
ODNdIHJhaWQ2OiBpbnQ2NHgxICB4b3IoKSAgMjQ4NiBNQi9zClNlcCAxMyAxNTo0NDowNSBkZWJp
YW4ga2VybmVsOiBbICAyMDkuOTA5NTgzXSByYWlkNjogdXNpbmcgYWxnb3JpdGhtIG5lb254OCBn
ZW4oKSAxMjk4MSBNQi9zClNlcCAxMyAxNTo0NDowNSBkZWJpYW4ga2VybmVsOiBbICAyMDkuOTA5
NTg0XSByYWlkNjogLi4uLiB4b3IoKSAxMDEzNSBNQi9zLCBybXcgZW5hYmxlZApTZXAgMTMgMTU6
NDQ6MDUgZGViaWFuIGtlcm5lbDogWyAgMjA5LjkwOTU4NV0gcmFpZDY6IHVzaW5nIG5lb24gcmVj
b3ZlcnkgYWxnb3JpdGhtClNlcCAxMyAxNTo0NDowNSBkZWJpYW4ga2VybmVsOiBbICAyMDkuOTEx
NTg2XSB4b3I6IG1lYXN1cmluZyBzb2Z0d2FyZSBjaGVja3N1bSBzcGVlZApTZXAgMTMgMTU6NDQ6
MDUgZGViaWFuIGtlcm5lbDogWyAgMjA5LjkxMjE5OF0gICAgOHJlZ3MgICAgICAgICAgIDogMTYx
MzggTUIvc2VjClNlcCAxMyAxNTo0NDowNSBkZWJpYW4ga2VybmVsOiBbICAyMDkuOTEyNzA2XSAg
ICAzMnJlZ3MgICAgICAgICAgOiAxOTM2MSBNQi9zZWMKU2VwIDEzIDE1OjQ0OjA1IGRlYmlhbiBr
ZXJuZWw6IFsgIDIwOS45MTMwNjhdICAgIGFybTY0X25lb24gICAgICA6IDI3Mzc5IE1CL3NlYwpT
ZXAgMTMgMTU6NDQ6MDUgZGViaWFuIGtlcm5lbDogWyAgMjA5LjkxMzA2OV0geG9yOiB1c2luZyBm
dW5jdGlvbjogYXJtNjRfbmVvbiAoMjczNzkgTUIvc2VjKQpTZXAgMTMgMTU6NDQ6MDUgZGViaWFu
IGtlcm5lbDogWyAgMjA5Ljk0MDE1OF0gQnRyZnMgbG9hZGVkLCBjcmMzMmM9Y3JjMzJjLWdlbmVy
aWMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEJvb3Rpbmcg
TGludXggb24gcGh5c2ljYWwgQ1BVIDB4MDAwMDEyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIExpbnV4IHZlcnNpb24gNS4xMC4w
LTgtYXJtNjQgKGRlYmlhbi1rZXJuZWxAbGlzdHMuZGViaWFuLm9yZykgKGdjYy0xMCAoRGViaWFu
IDEwLjIuMS02KSAxMC4yLjEgMjAyMTAxMTAsIEdOVSBsZCAoR05VIEJpbnV0aWxzIGZvciBEZWJp
YW4pIDIuMzUuMikgIzEgU01QIERlYmlhbiA1LjEwLjQ2LTQgKDIwMjEtMDgtMDMpClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEVGSSB2Mi43MCBieSBB
bWVyaWNhbiBNZWdhdHJlbmRzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBlZmk6IEFDUEkgMi4wPTB4ZWU5MDAwMDAgU01CSU9TIDMuMD0weGYxYjNmZjk4IEVT
UlQ9MHhlOTRmNmY5OCBNT0t2YXI9MHhlOTRkZjAwMCBSTkc9MHhmNzY2ZTgxOCBNRU1SRVNFUlZF
PTB4ZTk2MWY2OTggClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBlZmk6IHNlZWRpbmcgZW50cm9weSBwb29sClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBzZWN1cmVib290OiBTZWN1cmUgYm9vdCBkaXNhYmxlZApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZXNydDogUmVzZXJ2aW5nIEVT
UlQgc3BhY2UgZnJvbSAweDAwMDAwMDAwZTk0ZjZmOTggdG8gMHgwMDAwMDAwMGU5NGY2ZmQwLgpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRWFybHkg
dGFibGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2FibGVkClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAwMDBFRTkwMDAwMCAw
MDAwMjQgKHYwMiBBbXBlcmUpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDBFRThGMDAwMCAwMDAwQTQgKHYwMSBBbXBlcmUg
QWx0cmEgICAgMDAwMDAwMDAgQU1JICAwMTAwMDAxMykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZBQ1AgMHgwMDAwMDAwMEVFOEQwMDAwIDAwMDEx
NCAodjA2IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRFNEVCAweDAwMDAwMDAw
RUU4NjAwMDAgMDE5QTk1ICh2MDIgQW1wZXJlIEphZGUgICAgIDAwMDAwMDAxIElOVEwgMjAyMDA3
MTcpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBE
QkcyIDB4MDAwMDAwMDBFRThFMDAwMCAwMDAwNUMgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAw
MDAgSU5UTCAyMDE5MDUwOSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IEdURFQgMHgwMDAwMDAwMEVFOEMwMDAwIDAwMDExMCAodjAzIEFtcGVyZSBB
bHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1NEVCAweDAwMDAwMDAwRUU4QjAwMDAgMDAwMDJE
ICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIElOVEwgMjAxOTA1MDkpClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGSURUIDB4MDAwMDAwMDBF
RTg1MDAwMCAwMDAwOUMgKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAx
MykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQ
Q1IgMHgwMDAwMDAwMEVFODQwMDAwIDAwMDA1MCAodjAyIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAw
OSBBTUkgIDAwMDUwMDBGKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogQkdSVCAweDAwMDAwMDAwRUU4MzAwMDAgMDAwMDM4ICh2MDEgQUxBU0tBIEEg
TSBJICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBNQ0ZHIDB4MDAwMDAwMDBFRTgyMDAwMCAwMDAwQUMg
KHYwMSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IElPUlQgMHgwMDAwMDAwMEVF
ODEwMDAwIDAwMDYxMCAodjAwIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEz
KQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUFBU
VCAweDAwMDAwMDAwRUU3RjAwMDAgMDA2RTYwICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAw
IEFNUC4gMDEwMDAwMTMpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBTTElUIDB4MDAwMDAwMDBFRTdFMDAwMCAwMDAwMkQgKHYwMSBBbXBlcmUgQWx0
cmEgICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQgMHgwMDAwMDAwMEVFN0QwMDAwIDAwMDZEMCAo
djAzIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQVBJQyAweDAwMDAwMDAwRUU4
MDAwMDAgMDAxOUY0ICh2MDUgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAzIEFNSSAgMDEwMDAwMTMp
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBQQ0NU
IDB4MDAwMDAwMDBFRTdDMDAwMCAwMDA1NzYgKHYwMiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDMg
QU1QLiAwMTAwMDAxMykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IFdTTVQgMHgwMDAwMDAwMEVFN0IwMDAwIDAwMDAyOCAodjAxIEFMQVNLQSBBIE0g
SSAgICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogRlBEVCAweDAwMDAwMDAwRUU3QTAwMDAgMDAwMDQ0ICh2
MDEgQUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDEwMDAwMTMpClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUENSOiBjb25zb2xlOiBwbDAx
MSxtbWlvMzIsMHgxMDAwMDI2MDAwMDAsMTE1MjAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDg4MzAw
MDAwLTB4ODgzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDkwMDAwMDAwLTB4ZmZmZmZmZmZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFU
OiBOb2RlIDAgUFhNIDAgW21lbSAweDgwMDAwMDAwMDAwLTB4ODAwN2ZmZmZmZmZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAg
UFhNIDAgW21lbSAweDgwMTAwMDAwMDAwLTB4ODBmZmZmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBOVU1BOiBOT0RFX0RBVEEgW21lbSAweDgwZmY4
MDg1YjAwLTB4ODBmZjgwODdmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBab25lIHJhbmdlczoKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdICAgRE1BICAgICAgW21lbSAweDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAw
MGZmZmZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
ICBETUEzMiAgICBlbXB0eQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gICBOb3JtYWwgICBbbWVtIDB4MDAwMDAwMDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNb3ZhYmxl
IHpvbmUgc3RhcnQgZm9yIGVhY2ggbm9kZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gRWFybHkgbWVtb3J5IG5vZGUgcmFuZ2VzClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA4
ODMwMDAwMC0weDAwMDAwMDAwODgzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MDAwMDAwMC0weDAw
MDAwMDAwOTFmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MjAwMDAwMC0weDAwMDAwMDAwOTNmZmZm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDA5NDAwMDAwMC0weDAwMDAwMDAwZWUyZTZmZmZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDAwMDBlZTJlNzAwMC0weDAwMDAwMDAwZWU3NDBmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0
MTAwMC0weDAwMDAwMDAwZWU3NDRmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0NTAwMC0weDAwMDAw
MDAwZWU3NGNmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZDAwMC0weDAwMDAwMDAwZWU3NGRmZmZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDBlZTc0ZTAwMC0weDAwMDAwMDAwZWU3NGZmZmZdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDBlZTc1MDAwMC0weDAwMDAwMDAwZWU5NmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk3MDAw
MC0weDAwMDAwMDAwZWU5N2ZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk4MDAwMC0weDAwMDAwMDAw
ZjA2NGZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmMDY1MDAwMC0weDAwMDAwMDAwZjc2NmZmZmZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBmNzY3MDAwMC0weDAwMDAwMDAwZjc4NGZmZmZdClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBmNzg1MDAwMC0weDAwMDAwMDAwZjdmZGZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmN2ZlMDAwMC0w
eDAwMDAwMDAwZmZjOGVmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM4ZjAwMC0weDAwMDAwMDAwZmZj
OGZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM5MDAwMC0weDAwMDAwMDAwZmZmZmZmZmZdClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDA4MDAwMDAwMDAwMC0weDAwMDAwODAwN2ZmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDEw
MDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBJbml0bWVtIHNldHVwIG5vZGUgMCBbbWVtIDB4MDAwMDAwMDA4ODMw
MDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBPbiBub2RlIDAgdG90YWxwYWdlczogMTY3MTE5MzYKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDcxNzIgcGFnZXMg
dXNlZCBmb3IgbWVtbWFwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIERNQSB6b25lOiAwIHBhZ2VzIHJlc2VydmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiA0NTkwMDggcGFnZXMsIExJRk8gYmF0
Y2g6NjMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9y
bWFsIHpvbmU6IDI1Mzk1MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDE2MjUyOTI4IHBhZ2Vz
LCBMSUZPIGJhdGNoOjYzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBjbWE6IFJlc2VydmVkIDY0IE1pQiBhdCAweDAwMDAwMDAwZmJjMDAwMDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IHByb2JpbmcgZm9yIGNv
bmR1aXQgbWV0aG9kIGZyb20gQUNQSS4KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIHBzY2k6IFBTQ0l2MS4xIGRldGVjdGVkIGluIGZpcm13YXJlLgpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogVXNpbmcgc3RhbmRh
cmQgUFNDSSB2MC4yIGZ1bmN0aW9uIElEcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gcHNjaTogTUlHUkFURV9JTkZPX1RZUEUgbm90IHN1cHBvcnRlZC4KU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFNNQyBDYWxs
aW5nIENvbnZlbnRpb24gdjEuMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgwIC0+IE5vZGUgMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDEwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDEwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDIwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgzMDAwMCAt
PiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MzAxMDAgLT4gTm9kZSAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDQwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0
MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAwMDAgLT4gTm9kZSAwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHg2MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NjAxMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHg3MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ODAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDgwMTAwIC0+IE5vZGUgMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHg5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
OTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMDAwIC0+IE5vZGUgMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHhhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4YjAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGIwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4YzAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGQwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhkMDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4ZTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eGUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDEwMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMTAwIC0+
IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAwMDAgLT4gTm9kZSAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDExMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MTIwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMjAxMDAgLT4gTm9kZSAw
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MTMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAwMDAg
LT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBB
Q1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE0MDEwMCAtPiBOb2RlIDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6
IFBYTSAwIC0+IE1QSURSIDB4MTUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHgxNTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDAwMCAtPiBOb2Rl
IDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5V
TUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAg
LT4gTVBJRFIgMHgxNzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE3MDEw
MCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMDAwIC0+IE5vZGUgMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgxODAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDE5MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTkwMTAwIC0+IE5v
ZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDFhMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIw
MDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYjAxMDAgLT4gTm9kZSAwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDFjMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MWMwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAwMDAgLT4g
Tm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFkMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MWUwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
ZTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDAwMCAtPiBOb2RlIDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWYwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgyMDAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDEwMCAt
PiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMDAwIC0+IE5vZGUgMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgyMTAxMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDIyMDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjIwMTAwIC0+IE5vZGUg
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAwMDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDIzMDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMDAw
IC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNDAxMDAgLT4gTm9kZSAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDI1MDAwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MjUwMTAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAwMDAgLT4gTm9k
ZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI2MDEwMCAtPiBOb2RlIDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MjcwMDAwIC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNzAx
MDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBwZXJjcHU6IEVtYmVkZGVkIDMzIHBhZ2VzL2NwdSBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUx
NjgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxs
b2M6IHM5NTE5MiByODE5MiBkMzE3ODQgdTEzNTE2OCBhbGxvYz0zMyo0MDk2ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMDAgWzBd
IDAxIFswXSAwMiBbMF0gMDMgWzBdIDA0IFswXSAwNSBbMF0gMDYgWzBdIDA3IApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDA4IFsw
XSAwOSBbMF0gMTAgWzBdIDExIFswXSAxMiBbMF0gMTMgWzBdIDE0IFswXSAxNSAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSAxNiBb
MF0gMTcgWzBdIDE4IFswXSAxOSBbMF0gMjAgWzBdIDIxIFswXSAyMiBbMF0gMjMgClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMjQg
WzBdIDI1IFswXSAyNiBbMF0gMjcgWzBdIDI4IFswXSAyOSBbMF0gMzAgWzBdIDMxIApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDMy
IFswXSAzMyBbMF0gMzQgWzBdIDM1IFswXSAzNiBbMF0gMzcgWzBdIDM4IFswXSAzOSAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA0
MCBbMF0gNDEgWzBdIDQyIFswXSA0MyBbMF0gNDQgWzBdIDQ1IFswXSA0NiBbMF0gNDcgClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0g
NDggWzBdIDQ5IFswXSA1MCBbMF0gNTEgWzBdIDUyIFswXSA1MyBbMF0gNTQgWzBdIDU1IApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBd
IDU2IFswXSA1NyBbMF0gNTggWzBdIDU5IFswXSA2MCBbMF0gNjEgWzBdIDYyIFswXSA2MyAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFsw
XSA2NCBbMF0gNjUgWzBdIDY2IFswXSA2NyBbMF0gNjggWzBdIDY5IFswXSA3MCBbMF0gNzEgClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBb
MF0gNzIgWzBdIDczIFswXSA3NCBbMF0gNzUgWzBdIDc2IFswXSA3NyBbMF0gNzggWzBdIDc5IApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGlu
dGVyZmFjZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BV
IGZlYXR1cmVzOiBkZXRlY3RlZDogVmlydHVhbGl6YXRpb24gSG9zdCBFeHRlbnNpb25zClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGtl
cm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiBmb3JjZWQgT04gYnkgS0FTTFIKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6
IEtlcm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiAoS1BUSSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEhhcmR3YXJl
IGRpcnR5IGJpdCBtYW5hZ2VtZW50ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBTcGVjdHJlLXY0ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVk
OiBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBhbHRlcm5hdGl2ZXM6IHBhdGNoaW5nIGtlcm5lbCBjb2RlClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBCdWlsdCAxIHpvbmVsaXN0cywgbW9i
aWxpdHkgZ3JvdXBpbmcgb24uICBUb3RhbCBwYWdlczogMTY0NTA4MTIKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFBvbGljeSB6b25lOiBOb3JtYWwKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEtlcm5lbCBjb21tYW5kIGxp
bmU6IEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1hcm02NCByb290PVVVSUQ9NmMw
ODMyN2MtNGJiZC00MzQxLWE0MjctYTcyYmMxNTMzODk1IHJvIHF1aWV0ClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuIGluZGl2
aWR1YWwgbWF4IGNwdSBjb250cmlidXRpb246IDQwOTYgYnl0ZXMKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4gdG90YWwgY3B1
X2V4dHJhIGNvbnRyaWJ1dGlvbnM6IDMyMzU4NCBieXRlcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbiBtaW4gc2l6ZTogMTMx
MDcyIGJ5dGVzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
cmludGs6IGxvZ19idWZfbGVuOiA1MjQyODggYnl0ZXMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogZWFybHkgbG9nIGJ1ZiBmcmVlOiAxMTkwMTYo
OTAlKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGVudHJ5
IGNhY2hlIGhhc2ggdGFibGUgZW50cmllczogODM4ODYwOCAob3JkZXI6IDE0LCA2NzEwODg2NCBi
eXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gSW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA0MTk0MzA0IChvcmRlcjogMTMsIDMz
NTU0NDMyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBtZW0gYXV0by1pbml0OiBzdGFjazpvZmYsIGhlYXAgYWxsb2M6b24sIGhlYXAg
ZnJlZTpvZmYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNv
ZnR3YXJlIElPIFRMQjogbWFwcGVkIFttZW0gMHgwMDAwMDAwMGVhMmU3MDAwLTB4MDAwMDAwMDBl
ZTJlNzAwMF0gKDY0TUIpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBNZW1vcnk6IDI1NDc3ODRLLzY2ODQ3NzQ0SyBhdmFpbGFibGUgKDExNzc2SyBrZXJuZWwg
Y29kZSwgMjQzNksgcndkYXRhLCA3MDA4SyByb2RhdGEsIDU0NDBLIGluaXQsIDU5OEsgYnNzLCAx
NTg5MzkySyByZXNlcnZlZCwgNjU1MzZLIGNtYS1yZXNlcnZlZCkKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJhbmRvbTogZ2V0X3JhbmRvbV91NjQgY2FsbGVk
IGZyb20gX19rbWVtX2NhY2hlX2NyZWF0ZSsweDM4LzB4NTYwIHdpdGggY3JuZ19pbml0PTAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNMVUI6IEhXYWxpZ249
NjQsIE9yZGVyPTAtMywgTWluT2JqZWN0cz0wLCBDUFVzPTgwLCBOb2Rlcz0xClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRpbmcgMzg1
MzUgZW50cmllcyBpbiAxNTEgcGFnZXMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIGZ0cmFjZTogYWxsb2NhdGVkIDE1MSBwYWdlcyB3aXRoIDUgZ3JvdXBzClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IEhpZXJhcmNo
aWNhbCBSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSByY3U6IAlSQ1UgcmVzdHJpY3RpbmcgQ1BVcyBmcm9tIE5SX0NQVVM9MjU2
IHRvIG5yX2NwdV9pZHM9ODAuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAJUnVkZSB2YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gCVRyYWNpbmcgdmFyaWFudCBvZiBUYXNr
cyBSQ1UgZW5hYmxlZC4KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHJjdTogUkNVIGNhbGN1bGF0ZWQgdmFsdWUgb2Ygc2NoZWR1bGVyLWVubGlzdG1lbnQgZGVs
YXkgaXMgMjUgamlmZmllcy4KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIHJjdTogQWRqdXN0aW5nIGdlb21ldHJ5IGZvciByY3VfZmFub3V0X2xlYWY9MTYsIG5y
X2NwdV9pZHM9ODAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IE5SX0lSUVM6IDY0LCBucl9pcnFzOiA2NCwgcHJlYWxsb2NhdGVkIGlycXM6IDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBHSUM6IFVzaW5nIHNw
bGl0IEVPSS9EZWFjdGl2YXRlIG1vZGUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEdJQ3YzOiA2NzIgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDAgRXh0ZW5kZWQgU1BJcyBpbXBs
ZW1lbnRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lD
djM6IERpc3RyaWJ1dG9yIGhhcyBubyBSYW5nZSBTZWxlY3RvciBzdXBwb3J0ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMTYgUFBJcyBpbXBsZW1l
bnRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6
IENQVTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVj
MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDog
UFhNIDAgLT4gSVRTIDAgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMSAtPiBOb2RlIDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAyIC0+
IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JB
VDogUFhNIDAgLT4gSVRTIDMgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNCAtPiBOb2RlIDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA1
IC0+IE5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
U1JBVDogUFhNIDAgLT4gSVRTIDYgLT4gTm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNyAtPiBOb2RlIDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAw
MDQwMDAwLTB4MTAwMTAwMDVmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNl
cyBAODAwMDAyYzAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQw
MDAwOiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDJkMDAwMCAo
ZmxhdCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwNjAwMDAtMHgxMDAxMDAwN2ZmZmZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAw
MTAwMTAwMDYwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJmMDAwMCAoaW5kaXJl
Y3QsIGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJ
bnRlcnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzAwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywg
c2hyIDEpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMg
W21lbSAweDEwMDEwMDA4MDAwMC0weDEwMDEwMDA5ZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRl
ZCA4MTkyIERldmljZXMgQDgwMDAwMzIwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNo
ciAxKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4
MDAwMDEwMDEwMDA4MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBA
ODAwMDAzMzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGEwMDAwLTB4
MTAwMTAwMGJmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gSVRTQDB4MDAwMDEwMDEwMDBhMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAz
NTAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxv
Y2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM2MDAwMCAoZmxhdCwgZXN6
IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwYzAwMDAtMHgxMDAxMDAwZGZmZmZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMw
MDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDM4MDAwMCAoaW5kaXJlY3QsIGVzeiA4
LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIElUU0AweDAwMDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQg
Q29sbGVjdGlvbnMgQDgwMDAwMzkwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEw
MDEwMDBlMDAwMC0weDEwMDEwMDBmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCA4MTkyIERl
dmljZXMgQDgwMDAwM2IwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEw
MDBlMDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzYzAw
MDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTAwMDAwLTB4MTAwMTAwMTFm
ZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4
MDAwMDEwMDEwMDEwMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzZTAwMDAgKGlu
ZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBhbGxvY2F0ZWQgMzI3
NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDNmMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2
NEssIHNociAxKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTIFttZW0gMHgxMDAxMDAxMjAwMDAtMHgxMDAxMDAxM2ZmZmZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxv
Y2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDgxMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRL
LCBzaHIgMSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElU
U0AweDAwMDAxMDAxMDAxMjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlv
bnMgQDgwMDAwODIwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogdXNpbmcgTFBJIHByb3Bl
cnR5IHRhYmxlIEAweDAwMDAwODAwMDA4MzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBDUFUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMDg0MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSSBHVERUOiBmb3VuZCAxIG1lbW9yeS1tYXBwZWQgdGltZXIgYmxvY2so
cykuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhcmNoX3Rp
bWVyOiBjcDE1IGFuZCBtbWlvIHRpbWVyKHMpIHJ1bm5pbmcgYXQgMjUuMDBNSHogKHBoeXMvcGh5
cykuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbG9ja3Nv
dXJjZTogYXJjaF9zeXNfY291bnRlcjogbWFzazogMHhmZmZmZmZmZmZmZmZmZiBtYXhfY3ljbGVz
OiAweDVjNDA5MzliNSwgbWF4X2lkbGVfbnM6IDQ0MDc5NTIwMjY0NiBucwpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMV0gc2NoZWRfY2xvY2s6IDU2IGJpdHMgYXQg
MjVNSHosIHJlc29sdXRpb24gNDBucywgd3JhcHMgZXZlcnkgNDM5ODA0NjUxMTEwMG5zClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDU0XSBDb25zb2xlOiBjb2xvdXIg
ZHVtbXkgZGV2aWNlIDgweDI1ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDc2XSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIwMjAwOTI1ClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMjY5XSBDYWxpYnJhdGluZyBkZWxheSBsb29wIChza2lwcGVk
KSwgdmFsdWUgY2FsY3VsYXRlZCB1c2luZyB0aW1lciBmcmVxdWVuY3kuLiA1MC4wMCBCb2dvTUlQ
UyAobHBqPTEwMDAwMCkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAy
NzJdIHBpZF9tYXg6IGRlZmF1bHQ6IDgxOTIwIG1pbmltdW06IDY0MApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMwNV0gTFNNOiBTZWN1cml0eSBGcmFtZXdvcmsgaW5p
dGlhbGl6aW5nClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzEwXSBZ
YW1hOiBkaXNhYmxlZCBieSBkZWZhdWx0OyBlbmFibGUgd2l0aCBzeXNjdGwga2VybmVsLnlhbWEu
KgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMyOF0gQXBwQXJtb3I6
IEFwcEFybW9yIGluaXRpYWxpemVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMzI5XSBUT01PWU8gTGludXggaW5pdGlhbGl6ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAzNzBdIE1vdW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczog
MTMxMDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDM5NF0gTW91bnRwb2ludC1jYWNoZSBoYXNoIHRhYmxl
IGVudHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEyNTRdIHJjdTogSGllcmFyY2hpY2Fs
IFNSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAxMzk1XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRl
ZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTM5Nl0gUGxhdGZvcm0g
TVNJOiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDEzOThdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDgw
MDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxMzk5XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwMF0gUGxhdGZvcm0gTVNJ
OiBJVFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDE0MDFdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
NDAyXSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwNF0gUGxhdGZvcm0gTVNJOiBJ
VFNAMHgxMDAxMDAxMjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDE0MDhdIFBDSS9NU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4g
Y3JlYXRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwOV0gUENJ
L01TSTogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDEwXSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwODAwMDAg
ZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0
MTJdIFBDSS9NU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxNF0gUENJL01TSTogSVRTQDB4MTAwMTAw
MGMwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAxNDE3XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MThdIFBDSS9NU0k6IElUU0Aw
eDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMTQxOV0gUENJL01TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVh
dGVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDIyXSBmc2wtbWMg
TVNJOiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjNdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA2MDAw
MCBkb21haW4gY3JlYXRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MTQyNF0gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI2XSBmc2wtbWMgTVNJOiBJVFNA
MHgxMDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDE0MjddIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4g
Y3JlYXRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyOF0gZnNs
LW1jIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDI5XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAx
MDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDE0MzBdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzNl0gUmVtYXBwaW5nIGFu
ZCBlbmFibGluZyBFRkkgc2VydmljZXMuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAzOTA4XSBzbXA6IEJyaW5naW5nIHVwIHNlY29uZGFyeSBDUFVzIC4uLgpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDQ1Nl0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ0NzVd
IEdJQ3YzOiBDUFUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFhMDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA3YzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ0OTRd
IEdJQ3YzOiBDUFUxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMDg1MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDUyNV0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDUzNF0gQ1BVMTog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTE0OV0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUx
NjRdIEdJQ3YzOiBDUFUyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA2NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUx
ODFdIEdJQ3YzOiBDUFUyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDg2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTIx
Nl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTIyNV0gQ1BV
MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTQwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTQzM10gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU0NTBdIEdJQ3YzOiBDUFUzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA4NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU0NjhdIEdJQ3YzOiBDUFUzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMDg3MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NTUwMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTUxMF0g
Q1BVMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWMwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTgwNF0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDU4MThdIEdJQ3YzOiBDUFU0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA1NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDU4MzZdIEdJQ3YzOiBDUFU0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDg4MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNTg3MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTg4
MV0gQ1BVNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjIwOV0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDYyMjVdIEdJQ3YzOiBDUFU1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE4MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA3NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDYyNDNdIEdJQ3YzOiBDUFU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDg5MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNjI3OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NjI4N10gQ1BVNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTgwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjYwN10gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDY2MjNdIEdJQ3YzOiBDUFU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA2YzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDY2NDJdIEdJQ3YzOiBDUFU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDhhMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNjY3N10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNjY4N10gQ1BVNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzAxMV0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDcwMjldIEdJQ3YzOiBDUFU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFlMDAw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA4YzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDcwNDhdIEdJQ3YzOiBDUFU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMDhiMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNzA4M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNzA5M10gQ1BVNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMDAw
IFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzQw
M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDc0MTVdIEdJQ3YzOiBDUFU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNjMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNzQzNV0gR0lDdjM6IENQVTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGMwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA3NDcyXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA3NDgyXSBDUFU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYTAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3
ODExXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVOQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNzgyOV0gR0lDdjM6IENQVTk6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MjIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDljMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwNzg0OF0gR0lDdjM6IENQVTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGQwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDA3ODc5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA3ODkwXSBDUFU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAy
MjAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDA4MjA2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDgyMTldIEdJQ3YzOiBDUFUxMDogZm91bmQgcmVkaXN0cmli
dXRvciBjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDgyMzldIEdJQ3YzOiBDUFUxMDogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ZTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDgyNzVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDgyODZdIENQVTEwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAwYzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA4NjExXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTEKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2MzFdIEdJQ3YzOiBDUFUxMTogZm91bmQgcmVk
aXN0cmlidXRvciAyNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTQwMDAwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjUwXSBHSUN2MzogQ1BVMTE6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGYwMDAwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjgzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4Njk0XSBDUFUxMTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMjQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwOTAwNF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEyClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDE2XSBHSUN2MzogQ1BVMTI6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzQwMDAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDM3XSBHSUN2MzogQ1BVMTI6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTAwMDAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDcyXSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDgzXSBDUFUxMjogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMDgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwOTQxNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEz
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NDM0XSBHSUN2MzogQ1BV
MTM6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk0MDAw
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ1NF0gR0lDdjM6IENQ
VTEzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkxMDAw
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ4Nl0gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ5N10gQ1BVMTM6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk4MDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFUxNApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTgyM10gR0lD
djM6IENQVTE0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDRjMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg0NF0gR0lD
djM6IENQVTE0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MDkyMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg4MV0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg5Ml0gQ1BVMTQ6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyMTBdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFUxNQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDIz
MF0gR0lDdjM6IENQVTE1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDBhYzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAy
NTFdIEdJQ3YzOiBDUFUxNTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDA5MzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAy
ODRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyOTZdIENQ
VTE1OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAwMDAgWzB4NDEzZmQwYzFd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwODExXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVMTYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTA4MjJdIEdJQ3YzOiBDUFUxNjogZm91bmQgcmVkaXN0cmlidXRvciAyMDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDAxYzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTA4NDNdIEdJQ3YzOiBDUFUxNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDA5NDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTA4ODFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA4
OTNdIENQVTE2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAwMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExMjEyXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTEyMjNdIEdJQ3YzOiBDUFUxNzogZm91bmQgcmVkaXN0cmlidXRvciA0MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDAyNDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTEyNDRdIEdJQ3YzOiBDUFUxNzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA5NTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTEyODJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTEyOTRdIENQVTE3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNjA0
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTE2MTRdIEdJQ3YzOiBDUFUxODogZm91bmQgcmVkaXN0cmlidXRvciAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE0MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxMTYzNl0gR0lDdjM6IENQVTE4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMDk2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxMTY3NF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxMTY4N10gQ1BVMTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAwMDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIw
MTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxOQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMjAyNF0gR0lDdjM6IENQVTE5OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDYwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJjMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMjA0NV0gR0lDdjM6IENQVTE5OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk3MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMjA4MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxMjA5NF0gQ1BVMTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDA2MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTI0MTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMjQyNl0gR0lDdjM6IENQVTIwOiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDEzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2MDAwMDAKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI0NDddIEdJQ3YzOiBDUFUyMDogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ODAwMDAKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI0ODNdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTI0OTVdIENQVTIwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAxMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDEyODEyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjEKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI4MjldIEdJQ3YzOiBDUFUyMTogZm91bmQg
cmVkaXN0cmlidXRvciAxYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODAwMDAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODQ5XSBHSUN2MzogQ1BVMjE6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTkwMDAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODgyXSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODk0XSBDUFUyMTogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMWIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxMzIwNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIyClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMjIxXSBHSUN2MzogQ1BVMjI6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY4MDAwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI0Ml0gR0lDdjM6IENQVTIy
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlhMDAwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI3N10gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI5MF0gQ1BVMjI6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2MDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFUyMwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzYyNl0gR0lDdjM6
IENQVTIzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4
ODAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2NDddIEdJQ3Yz
OiBDUFUyMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5
YjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2ODFdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2OTNdIENQVTIzOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDA5XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMjQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQwMjNd
IEdJQ3YzOiBDUFUyNDogZm91bmQgcmVkaXN0cmlidXRvciAxMTAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwNTgwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDQ0
XSBHSUN2MzogQ1BVMjQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwOWMwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDc4
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDkyXSBDUFUy
NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQxMF0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTI1ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE0NDI2XSBHSUN2MzogQ1BVMjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkwMDAwIHJlZ2lvbiAw
OjB4MDAwMDEwMDEwMDc4MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNDQ0OF0gR0lDdjM6IENQVTI1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDlkMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNDQ4MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQ5
NF0gQ1BVMjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5MDAwMCBbMHg0MTNm
ZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ4MDVdIERldGVj
dGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyNgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNDgyMV0gR0lDdjM6IENQVTI2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE3MDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA3MDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTQ4NDNdIEdJQ3YzOiBDUFUyNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDA5ZTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTQ4NzhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTQ4OTFdIENQVTI2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNzAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MjA2
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTUyMjRdIEdJQ3YzOiBDUFUyNzogZm91bmQgcmVkaXN0cmlidXRvciAx
ZjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE1MjQ2XSBHSUN2MzogQ1BVMjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE1MjgxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE1Mjk0XSBDUFUyNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MWYwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxNTYwNl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI4ClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NjE4XSBHSUN2MzogQ1BVMjg6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDAwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1NjQxXSBHSUN2MzogQ1BVMjg6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTAwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1Njc4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDE1NjkxXSBDUFUyODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMGIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxNjAwN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI5ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2MDI2XSBHSUN2MzogQ1BVMjk6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEwMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjA0OF0gR0lDdjM6IENQVTI5OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGExMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjA3OV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjA5Ml0gQ1BVMjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDIzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTY0MDRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQxN10gR0lDdjM6IENQVTMwOiBm
b3VuZCByZWRpc3RyaWJ1dG9yIGQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ4MDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQzOV0gR0lDdjM6IENQVTMwOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEyMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQ3NV0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQ4OF0gQ1BVMzA6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDBkMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4MDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUz
MQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjgyOF0gR0lDdjM6IENQ
VTMxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhODAw
MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4NTFdIEdJQ3YzOiBD
UFUzMTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMzAw
MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4ODRdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4OThdIENQVTMxOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3MjA3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVMzIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyMjBdIEdJ
Q3YzOiBDUFUzMjogZm91bmQgcmVkaXN0cmlidXRvciA5MDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAzODAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyNDNdIEdJ
Q3YzOiBDUFUzMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBhNDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyNzhdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyOTNdIENQVTMyOiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwOTAwMDAgWzB4NDEzZmQwYzFdClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NjEwXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVMzMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc2
MjhdIEdJQ3YzOiBDUFUzMzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAwMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwOTgwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3
NjUxXSBHSUN2MzogQ1BVMzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYTUwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3
Njg1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3Njk4XSBD
UFUzMzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODAwNl0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTM0ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4MDIxXSBHSUN2MzogQ1BVMzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZjAwMDAgcmVnaW9u
IDA6MHgwMDAwMTAwMTAwNTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4MDQ1XSBHSUN2MzogQ1BVMzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJs
ZSBAMHgwMDAwMDgwMDAwYTYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDE4MDgzXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0g
ZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4
MDk3XSBDUFUzNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGYwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODQxMl0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM1ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDE4NDMyXSBHSUN2MzogQ1BVMzU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjcwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGIwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxODQ1Nl0gR0lDdjM6IENQVTM1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRp
bmcgdGFibGUgQDB4MDAwMDA4MDAwMGE3MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxODQ5MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBm
b3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxODUwNV0gQ1BVMzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI3MDAw
MCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg4
MDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNgpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxODgxOV0gR0lDdjM6IENQVTM2OiBmb3VuZCByZWRpc3RyaWJ1dG9y
IDMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxODg0NF0gR0lDdjM6IENQVTM2OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE4MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxODg4MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAxODg5Nl0gQ1BVMzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDAzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMTkyMDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNwpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTIxOV0gR0lDdjM6IENQVTM3OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI4MDAwMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTI0M10gR0lDdjM6IENQVTM3OiB1c2luZyBhbGxvY2F0
ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE5MDAwMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTI4MV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwg
d29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAxOTI5Nl0gQ1BVMzc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29y
IDB4MDAwMDA1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMTk2MTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTYyMV0gR0lDdjM6IENQVTM4OiBmb3VuZCBy
ZWRpc3RyaWJ1dG9yIDEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE4MDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTY0N10gR0lDdjM6IENQVTM4OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFhMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTY4NV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTcwMV0gQ1BVMzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDAxMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjAwMTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDAyMl0gR0lDdjM6IENQVTM5OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDMwMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDA0N10gR0lDdjM6IENQVTM5OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFiMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDA4NF0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDEwMF0gQ1BVMzk6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDA3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0MDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDQyMF0gR0lDdjM6IENQ
VTQwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1ZTAw
MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0NDNdIEdJQ3YzOiBD
UFU0MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYzAw
MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0NzZdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0OTFdIENQVTQwOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODEwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA4MjZdIEdJ
Q3YzOiBDUFU0MTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwN2UwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODQ5XSBH
SUN2MzogQ1BVNDE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwYWQwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODgyXSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODk2XSBDUFU0MTog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMTAwIFsweDQxM2ZkMGMxXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTIwOF0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTQyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIx
MjI0XSBHSUN2MzogQ1BVNDI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQwMTAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDY2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
MTI0N10gR0lDdjM6IENQVTQyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMGFlMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
MTI4M10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTI5OV0g
Q1BVNDI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDEwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2MTFdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFU0MwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyMTYyOV0gR0lDdjM6IENQVTQzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA4NjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjE2NTNdIEdJQ3YzOiBDUFU0MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBhZjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjE2ODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjE3MDBdIENQVTQzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYzAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMDEwXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjIwMjRdIEdJQ3YzOiBDUFU0NDogZm91bmQgcmVkaXN0cmlidXRvciAxMDAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNTYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIyMDQ5XSBHSUN2MzogQ1BVNDQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIyMDg0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDIyMDk5XSBDUFU0NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
MjQxMF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ1ClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDIyNDI2XSBHSUN2MzogQ1BVNDU6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgMTgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMjQ1MF0gR0lDdjM6IENQVTQ1OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIxMDAwMApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMjQ4Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyMjQ5N10gQ1BVNDU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDE4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjI4MTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjgyOV0gR0lDdjM6IENQVTQ2OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDE2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2ZTAwMDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI4NTNdIEdJQ3YzOiBDUFU0NjogdXNpbmcgYWxs
b2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMjAwMDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI4ODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxv
Y2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5MDRdIENQVTQ2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nl
c3NvciAweDAwMDAxNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDIzMjEyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDcKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMyMzBdIEdJQ3YzOiBDUFU0NzogZm91
bmQgcmVkaXN0cmlidXRvciAxZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGUwMDAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjU0XSBHSUN2MzogQ1BVNDc6IHVz
aW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjMwMDAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjg5XSBhcmNoX3RpbWVyOiBFbmFi
bGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzA0XSBDUFU0NzogQm9vdGVkIHNlY29uZGFy
eSBwcm9jZXNzb3IgMHgwMDAwMWUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyMzYwOV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ4
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjIyXSBHSUN2MzogQ1BV
NDg6IGZvdW5kIHJlZGlzdHJpYnV0b3IgYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2UwMDAw
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjQ3XSBHSUN2MzogQ1BV
NDg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjQwMDAw
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjg0XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzAwXSBDUFU0ODogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDAxM10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTQ5ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0MDMyXSBHSUN2
MzogQ1BVNDk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDllMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDA1N10gR0lD
djM6IENQVTQ5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGI1MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDA5M10gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDEwOF0gQ1BVNDk6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ0MDldIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU1MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQy
Ml0gR0lDdjM6IENQVTUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDQ2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ0
N10gR0lDdjM6IENQVTUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGI2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ4
Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ5OF0gQ1BV
NTA6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4MTJdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU1MQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNDgzMV0gR0lDdjM6IENQVTUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0MDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDBhNjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjQ4NTZdIEdJQ3YzOiBDUFU1MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBiNzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjQ4ODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ5
MDVdIENQVTUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNDAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1MjEwXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjUyMjJdIEdJQ3YzOiBDUFU1MjogZm91bmQgcmVkaXN0cmlidXRvciA4MDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDAzNjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjUyNDhdIEdJQ3YzOiBDUFU1MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBiODAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjUyODVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjUzMDNdIENQVTUyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwODAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NjE3
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjU2MzVdIEdJQ3YzOiBDUFU1MzogZm91bmQgcmVkaXN0cmlidXRvciAy
MDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI1NjYxXSBHSUN2MzogQ1BVNTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjkwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI1Njk1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDI1NzEyXSBDUFU1MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MjAwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyNjAxMl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU0ClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDI2XSBHSUN2MzogQ1BVNTQ6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGUwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDU0XSBHSUN2MzogQ1BVNTQ6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYmEwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2MDkwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI2MTA4XSBDUFU1NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMGUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyNjQxN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU1ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NDM3XSBHSUN2MzogQ1BVNTU6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFlMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjQ2M10gR0lDdjM6IENQVTU1OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJiMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjQ5OV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjUxNl0gQ1BVNTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDI2MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjY4MDVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NgpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjgxNV0gR0lDdjM6IENQVTU2OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFlMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg0Ml0gR0lDdjM6IENQVTU2OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJjMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg4MF0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg5OF0gQ1BVNTY6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDAyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjcyMDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1
NwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzIxOF0gR0lDdjM6IENQ
VTU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI2MDAw
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzI0NV0gR0lDdjM6IENQ
VTU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJkMDAw
MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzI4M10gYXJjaF90aW1l
cjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMwMF0gQ1BVNTc6IEJvb3RlZCBz
ZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2MDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBv
biBDUFU1OApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzYxOV0gR0lD
djM6IENQVTU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAx
NjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2NDddIEdJQ3Yz
OiBDUFU1ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBi
ZTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2ODVdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3MDNdIENQVTU4OiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MDEyXSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNTkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwMjRd
IEdJQ3YzOiBDUFU1OTogZm91bmQgcmVkaXN0cmlidXRvciA2MDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAyZTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwNTJd
IEdJQ3YzOiBDUFU1OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBiZjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwODhd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxMDZdIENQVTU5
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNjAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDExXSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNjAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
Mjg0MjZdIEdJQ3YzOiBDUFU2MDogZm91bmQgcmVkaXN0cmlidXRvciAxMzAxMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwNjIwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI4NDUyXSBHSUN2MzogQ1BVNjA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYzAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI4NDg5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NTA3
XSBDUFU2MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMwMTAwIFsweDQxM2Zk
MGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODgxOF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI4ODM0XSBHSUN2MzogQ1BVNjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMTAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDgyMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyODg2MV0gR0lDdjM6IENQVTYxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGMxMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAyODg5NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyODkxM10gQ1BVNjE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDEwMCBb
MHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkyMTJd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAyOTIyOF0gR0lDdjM6IENQVTYyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1
MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2YTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjkyNTRdIEdJQ3YzOiBDUFU2MjogdXNpbmcgYWxsb2NhdGVkIExQSSBw
ZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMjkyODldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91
bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjkzMDZdIENQVTYyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAx
NTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDI5NjAyXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjMKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMjk2MjBdIEdJQ3YzOiBDUFU2MzogZm91bmQgcmVkaXN0cmli
dXRvciAxZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGEwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NjQ3XSBHSUN2MzogQ1BVNjM6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzMwMDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5NjgwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDI5Njk3XSBDUFU2MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMWQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzMDAxMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY0ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwMDI2XSBHSUN2MzogQ1BVNjQ6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMTEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVhMDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDA1M10gR0lDdjM6IENQVTY0OiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM0MDAwMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDA4N10gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDEwNF0gQ1BVNjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDExMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzA0MTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2NQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDQzMF0gR0lDdjM6IENQVTY1OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIDE5MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YTAwMDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0NTddIEdJQ3YzOiBDUFU2NTog
dXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNTAwMDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0OTFdIGFyY2hfdGltZXI6IEVu
YWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1MDhdIENQVTY1OiBCb290ZWQgc2Vjb25k
YXJ5IHByb2Nlc3NvciAweDAwMDAxOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODEzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BV
NjYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA4MzBdIEdJQ3YzOiBD
UFU2NjogZm91bmQgcmVkaXN0cmlidXRvciAxNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzIw
MDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODU3XSBHSUN2Mzog
Q1BVNjY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzYw
MDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODkyXSBhcmNoX3Rp
bWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwOTA4XSBDUFU2NjogQm9vdGVk
IHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTIxN10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTY3ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMjM2XSBH
SUN2MzogQ1BVNjc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDkyMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI2M10g
R0lDdjM6IENQVTY3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGM3MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI5OF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTMxNV0gQ1BVNjc6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFmMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE2MDddIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU2OApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MTYyMF0gR0lDdjM6IENQVTY4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGIwMTAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDQyMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MTY0OF0gR0lDdjM6IENQVTY4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMGM4MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAz
MTY4NV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTcwMl0g
Q1BVNjg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBiMDEwMCBbMHg0MTNmZDBj
MV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwMTBdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFU2OQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMjAyOV0gR0lDdjM6IENQVTY5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDBhMjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzIwNTddIEdJQ3YzOiBDUFU2OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBjOTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzIwOTBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzIxMDddIENQVTY5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDEzXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI0MjZdIEdJQ3YzOiBDUFU3MDogZm91bmQgcmVkaXN0cmlidXRvciBkMDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA0YTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI0NTVdIEdJQ3YzOiBDUFU3MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBjYTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzI0OTBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzI1MDZdIENQVTcwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMy
ODIwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzEKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzI4MzldIEdJQ3YzOiBDUFU3MTogZm91bmQgcmVkaXN0cmlidXRv
ciAyNTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYWEwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMyODY3XSBHSUN2MzogQ1BVNzE6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2IwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMyODk5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDMyOTE3XSBDUFU3MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMjUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMzIwNV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcyClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjE4XSBHSUN2MzogQ1BVNzI6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2EwMDAwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjQ3XSBHSUN2MzogQ1BVNzI6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2MwMDAwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjg0XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMzMzAzXSBDUFU3MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMDkwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAzMzYxMV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTczClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNjI5XSBHSUN2MzogQ1BVNzM6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgMjEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDlhMDAwMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzY1OF0gR0lDdjM6IENQVTczOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNkMDAwMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzY5MV0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzcwOV0gQ1BVNzM6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDIxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMzQwMTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDAyNV0gR0lDdjM6IENQVTc0
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUyMDAwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDA1NF0gR0lDdjM6IENQVTc0
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNlMDAwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDA5MF0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDEwOF0gQ1BVNzQ6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0MTldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFU3NQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDQ0MF0gR0lDdjM6
IENQVTc1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBi
MjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0NjhdIEdJQ3Yz
OiBDUFU3NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBj
ZjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1MDNdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1MjFdIENQVTc1OiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0ODA3XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVNzYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4MThd
IEdJQ3YzOiBDUFU3NjogZm91bmQgcmVkaXN0cmlidXRvciAzMDEwMCByZWdpb24gMDoweDAwMDAx
MDAxMDAyMjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4NDhd
IEdJQ3YzOiBDUFU3NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDBkMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4ODVd
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5MDRdIENQVTc2
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMzAxMDAgWzB4NDEzZmQwYzFdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MjA3XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVNzcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzUyMTldIEdJQ3YzOiBDUFU3NzogZm91bmQgcmVkaXN0cmlidXRvciA1MDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDAyYTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzUyNDldIEdJQ3YzOiBDUFU3NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBkMTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzUyODddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUzMDZd
IENQVTc3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNTAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NjEzXSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNzgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzU2MjNdIEdJQ3YzOiBDUFU3ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAxYTAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzU2NTRdIEdJQ3YzOiBDUFU3ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBkMjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzU2OTJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MzU3MTFdIENQVTc4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDE0XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzYwMjZdIEdJQ3YzOiBDUFU3OTogZm91bmQgcmVkaXN0cmlidXRvciA3MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDAzMjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzYwNTZdIEdJQ3YzOiBDUFU3OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBkMzAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzYwOTVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMzYxMTRdIENQVTc5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2
MTc2XSBzbXA6IEJyb3VnaHQgdXAgMSBub2RlLCA4MCBDUFVzClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDM2MjAwXSBTTVA6IFRvdGFsIG9mIDgwIHByb2Nlc3NvcnMgYWN0
aXZhdGVkLgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIwMV0gQ1BV
IGZlYXR1cmVzOiBkZXRlY3RlZDogUHJpdmlsZWdlZCBBY2Nlc3MgTmV2ZXIKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMDFdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6
IExTRSBhdG9taWMgaW5zdHJ1Y3Rpb25zClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDM2MjAyXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBVc2VyIEFjY2VzcyBPdmVycmlk
ZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIwM10gQ1BVIGZlYXR1
cmVzOiBkZXRlY3RlZDogMzItYml0IEVMMCBTdXBwb3J0ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM2MjAzXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDb21tb24gbm90
IFByaXZhdGUgdHJhbnNsYXRpb25zClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2MjA0XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIFBv
aW50IG9mIFBlcnNpc3RlbmNlClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM2MjA1XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBSQVMgRXh0ZW5zaW9uIFN1cHBvcnQKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMDVdIENQVSBmZWF0dXJlczog
ZGV0ZWN0ZWQ6IERhdGEgY2FjaGUgY2xlYW4gdG8gdGhlIFBvVSBub3QgcmVxdWlyZWQgZm9yIEkv
RCBjb2hlcmVuY2UKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMDZd
IENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IENSQzMyIGluc3RydWN0aW9ucwpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIwN10gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDog
U3BlY3VsYXRpdmUgU3RvcmUgQnlwYXNzaW5nIFNhZmUgKFNTQlMpClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDU4NjA1XSBDUFU6IEFsbCBDUFUocykgc3RhcnRlZCBhdCBF
TDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDQ4MTldIG5vZGUgMCBk
ZWZlcnJlZCBwYWdlcyBpbml0aWFsaXNlZCBpbiA0ODRtcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjU0NDk2M10gZGV2dG1wZnM6IGluaXRpYWxpemVkClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ1NTM1XSBSZWdpc3RlcmVkIGNwMTVfYmFycmll
ciBlbXVsYXRpb24gaGFuZGxlcgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjU0NTU1Nl0gUmVnaXN0ZXJlZCBzZXRlbmQgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDU1NThdIEtBU0xSIGVuYWJsZWQKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDU2MTddIGNsb2Nrc291cmNlOiBqaWZmaWVz
OiBtYXNrOiAweGZmZmZmZmZmIG1heF9jeWNsZXM6IDB4ZmZmZmZmZmYsIG1heF9pZGxlX25zOiA3
NjQ1MDQxNzg1MTAwMDAwIG5zClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NTQ1NjY5XSBmdXRleCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOSwgMjA5NzE1
MiBieXRlcywgbGluZWFyKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0
NjA5OF0gcGluY3RybCBjb3JlOiBpbml0aWFsaXplZCBwaW5jdHJsIHN1YnN5c3RlbQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NjI1N10gU01CSU9TIDMuMy4wIHByZXNl
bnQuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ2MjYxXSBETUk6IEdJ
R0FCWVRFIFIxNTItUDMwLTAwL01QMzItQVIxLTAwLCBCSU9TIEYxMyAoU0NQOiAxLjUuMjAyMTA0
MjYpIDA1LzIxLzIwMjEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDY0
OTJdIE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTYKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC41NDcwODBdIERNQTogcHJlYWxsb2NhdGVkIDQwOTYgS2lCIEdG
UF9LRVJORUwgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNTQ3MjA0XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBf
S0VSTkVMfEdGUF9ETUEgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTQ3MzI5XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtp
QiBHRlBfS0VSTkVMfEdGUF9ETUEzMiBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlvbnMKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDczMzZdIGF1ZGl0OiBpbml0aWFsaXpp
bmcgbmV0bGluayBzdWJzeXMgKGRpc2FibGVkKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjU0NzM3NV0gYXVkaXQ6IHR5cGU9MjAwMCBhdWRpdCgwLjU0NDoxKTogc3RhdGU9
aW5pdGlhbGl6ZWQgYXVkaXRfZW5hYmxlZD0wIHJlcz0xClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTQ3NDYyXSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdv
dmVybm9yICdmYWlyX3NoYXJlJwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjU0NzQ2M10gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnYmFuZ19i
YW5nJwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzQ2NF0gdGhlcm1h
bF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnc3RlcF93aXNlJwpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzQ2NV0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVy
ZWQgdGhlcm1hbCBnb3Zlcm5vciAndXNlcl9zcGFjZScKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC41NDc0NjZdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292
ZXJub3IgJ3Bvd2VyX2FsbG9jYXRvcicKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC41NDc1MTldIGNwdWlkbGU6IHVzaW5nIGdvdmVybm9yIGxhZGRlcgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzUyNl0gY3B1aWRsZTogdXNpbmcgZ292ZXJub3Ig
bWVudQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjU0NzYyMF0gRGV0ZWN0
ZWQgMTUgUENDIFN1YnNwYWNlcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjU0NzYzNV0gUmVnaXN0ZXJpbmcgUENDIGRyaXZlciBhcyBNYWlsYm94IGNvbnRyb2xsZXIKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDc2NTRdIGh3LWJyZWFrcG9pbnQ6
IGZvdW5kIDYgYnJlYWtwb2ludCBhbmQgNCB3YXRjaHBvaW50IHJlZ2lzdGVycy4KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDgwMzFdIEFTSUQgYWxsb2NhdG9yIGluaXRp
YWxpc2VkIHdpdGggMzI3NjggZW50cmllcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjU0ODA1NF0gQUNQSTogYnVzIHR5cGUgUENJIHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NDgwNTVdIGFjcGlwaHA6IEFDUEkgSG90IFBsdWcg
UENJIENvbnRyb2xsZXIgRHJpdmVyIHZlcnNpb246IDAuNQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjU0ODA5M10gU2VyaWFsOiBBTUJBIFBMMDExIFVBUlQgZHJpdmVyClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTUwNDMzXSBIdWdlVExCIHJlZ2lz
dGVyZWQgMS4wMCBHaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NTA0MzRdIEh1Z2VUTEIgcmVnaXN0ZXJlZCAz
Mi4wIE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU1MDQzNV0gSHVnZVRMQiByZWdpc3RlcmVkIDIuMDAgTWlC
IHBhZ2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTUwNDM1XSBIdWdlVExCIHJlZ2lzdGVyZWQgNjQuMCBLaUIgcGFnZSBz
aXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42Nzc4MTJdIEFDUEk6IEFkZGVkIF9PU0koTW9kdWxlIERldmljZSkKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzc4MTNdIEFDUEk6IEFkZGVkIF9PU0koUHJv
Y2Vzc29yIERldmljZSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzc4
MTRdIEFDUEk6IEFkZGVkIF9PU0koMy4wIF9TQ1AgRXh0ZW5zaW9ucykKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42Nzc4MTVdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29y
IEFnZ3JlZ2F0b3IgRGV2aWNlKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjY3NzgxNl0gQUNQSTogQWRkZWQgX09TSShMaW51eC1EZWxsLVZpZGVvKQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY3NzgxOF0gQUNQSTogQWRkZWQgX09TSShMaW51eC1M
ZW5vdm8tTlYtSERNSS1BdWRpbykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC42Nzc4MTldIEFDUEk6IEFkZGVkIF9PU0koTGludXgtSFBJLUh5YnJpZC1HcmFwaGljcykKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42ODQ3MzldIEFDUEk6IDIgQUNQSSBB
TUwgdGFibGVzIHN1Y2Nlc3NmdWxseSBhY3F1aXJlZCBhbmQgbG9hZGVkClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg2MDkwXSBBQ1BJOiBJbnRlcnByZXRlciBlbmFibGVk
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjg2MDkxXSBBQ1BJOiBVc2lu
ZyBHSUMgZm9yIGludGVycnVwdCByb3V0aW5nClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjg2MDk4XSBBQ1BJOiBNQ0ZHIHRhYmxlIGRldGVjdGVkLCA4IGVudHJpZXMKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTQ3OTVdIEFSTUgwMDExOjAwOiB0
dHlBTUEwIGF0IE1NSU8gMHgxMDAwMDI2MDAwMDAgKGlycSA9IDM2LCBiYXNlX2JhdWQgPSAwKSBp
cyBhIFNCU0EKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTQ4MzldIHBy
aW50azogY29uc29sZSBbdHR5QU1BMF0gZW5hYmxlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5NTk0NF0gQVJNSDAwMTE6MDE6IHR0eUFNQTEgYXQgTU1JTyAweDEwMDAw
MjYyMDAwMCAoaXJxID0gMzcsIGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5NzAwM10gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQ
Q0kwXSAoZG9tYWluIDAwMGMgW2J1cyAwMC1mZl0pClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjk3MDA5XSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIHN1cHBvcnRzIFtF
eHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTcwNzhdIGFjcGkgUE5QMEEwODowMDog
X09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcg
UE1FIExUUl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTcxMzZdIGFj
cGkgUE5QMEEwODowMDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHld
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3MTQxXSBhY3BpIFBOUDBB
MDg6MDA6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZm
ZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmZGViNWFjZDRlYTU4ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3NzY3XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0g
YXJlYSBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAw
QzAyOjAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3Nzc0XSBhY3Bp
IFBOUDBBMDg6MDA6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0g
Zm9yIFtidXMgMDAtZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3
ODQwXSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMGM6MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC42OTc4NDJdIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3Vy
Y2UgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjY5Nzg0M10gcGNpX2J1cyAwMDBjOjAwOiByb290IGJ1cyByZXNv
dXJjZSBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc4NDRdIHBjaV9idXMgMDAwYzowMDogcm9v
dCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTc4NjVdIHBjaSAwMDBjOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xh
c3MgMHgwNjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc5MTNd
IHBjaSAwMDBjOjAwOjAxLjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTc5MjBdIHBjaSAwMDBjOjAwOjAx
LjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTc5MzZdIHBjaSAwMDBjOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk3OTM2XSBwY2kgMDAwYzowMDowMS4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNjk4MDEyXSBhY3BpcGhwOiBTbG90IFsxXSByZWdpc3RlcmVkClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MDI1XSBwY2kgMDAwYzowMTowMC4wOiBb
MTAwMDowMDk3XSB0eXBlIDAwIGNsYXNzIDB4MDEwNzAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk4MDM0XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxMDogW2lvICAw
eDRmZmY4MDAwLTB4NGZmZjgwZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk4MDQyXSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxNDogW21lbSAweDQwMTQwMDAwLTB4
NDAxNGZmZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4
MDQ5XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxYzogW21lbSAweDQwMTAwMDAwLTB4NDAxM2Zm
ZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MDU5XSBw
Y2kgMDAwYzowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJl
Zl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTgwNjNdIHBjaSAwMDBj
OjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC42OTgxNThdIHBjaSAwMDBjOjAxOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjI5XSBwY2lfYnVzIDAwMGM6
MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4MjMzXSBwY2kgMDAwYzowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjM2XSBw
Y2kgMDAwYzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMWZm
ZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODIzOF0gcGNpIDAw
MGM6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4MjM5XSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI0MF0gcGNp
IDAwMGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjQyXSBwY2kgMDAwYzow
MDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI0Ml0gcGNpIDAwMGM6MDA6MDEuMDogQkFS
IDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjQ0XSBwY2kgMDAwYzowMTowMC4wOiBCQVIgNjogYXNz
aWduZWQgW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTgyNDVdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAzOiBhc3Np
Z25lZCBbbWVtIDB4NDAxMDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTgyNTFdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAxOiBhc3Np
Z25lZCBbbWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTgyNTZdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBubyBz
cGFjZSBmb3IgW2lvICBzaXplIDB4MDEwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC42OTgyNTZdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDAxMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4MjU4XSBwY2kgMDAwYzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjU5XSBwY2kgMDAwYzowMDowMS4wOiAg
IGJyaWRnZSB3aW5kb3cgW21lbSAweDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk4MjYwXSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDMwMDAwMDAwMDAwMC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI2Ml0gcGNpX2J1cyAwMDBj
OjAwOiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2VzIGFyZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGlu
ZyB3aXRoIHBjaT1yZWFsbG9jClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4MjYzXSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNlIDQgW21lbSAweDQwMDAwMDAwLTB4NGZm
ZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI2
NF0gcGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA1IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2Zm
ZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
ODI2NV0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAxIFttZW0gMHg0MDAwMDAwMC0weDQwMWZm
ZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODI2Nl0gcGNpX2J1
cyAwMDBjOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTgzNzJd
IEFDUEk6IFBDSSBSb290IEJyaWRnZSBbUENJMV0gKGRvbWFpbiAwMDBkIFtidXMgMDAtZmZdKQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5ODM3Nl0gYWNwaSBQTlAwQTA4
OjAxOiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21l
bnRzIE1TSSBIUFgtVHlwZTNdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk4NDQwXSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQg
W1BDSWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk4NDk5XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IE9TIG5vdyBjb250
cm9scyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5ODUwMV0gYWNwaSBQTlAwQTA4OjAxOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0g
MHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZm
ZmRlYjVhY2Q0ZWE1OApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTEy
M10gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGFyZWEgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZm
ZmZmZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjY5OTEzMF0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGF0IFttZW0gMHgzN2Zm
ZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTE5MF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBk
OjAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MTkxXSBwY2lfYnVz
IDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdp
bmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkxOTJdIHBjaV9i
dXMgMDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDM0MDAwMDAwMDAwMC0weDM3ZmZk
ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5
MTkzXSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MjA5XSBwY2kgMDAwZDowMDowMC4w
OiBbMWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNjk5MjQ4XSBwY2kgMDAwZDowMDowMS4wOiBbMWRlZjplMTAxXSB0
eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk5MjU2XSBwY2kgMDAwZDowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5MjcxXSBwY2kgMDAwZDowMDowMS4w
OiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
OTI3Ml0gcGNpIDAwMGQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTMyM10gcGNpIDAwMGQ6MDA6
MDIuMDogWzFkZWY6ZTEwMl0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTM0M10gcGNpIDAwMGQ6MDA6MDIuMDogc3VwcG9ydHMg
RDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkzNDRdIHBjaSAw
MDBkOjAwOjAyLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTkzOTFdIHBjaSAwMDBkOjAwOjAzLjA6IFsxZGVm
OmUxMDNdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC42OTk0MTJdIHBjaSAwMDBkOjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NDEyXSBwY2kgMDAwZDowMDowMy4w
OiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNjk5NDU4XSBwY2kgMDAwZDowMDowNC4wOiBbMWRlZjplMTA0XSB0eXBl
IDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
Njk5NDc5XSBwY2kgMDAwZDowMDowNC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTQ4MF0gcGNpIDAwMGQ6MDA6MDQuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjY5OTU0N10gYWNwaXBocDogU2xvdCBbMS0yXSByZWdpc3RlcmVkClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5NTYzXSBwY2kgMDAwZDowMTowMC4wOiBbODA4Njow
YTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNjk5NTc1XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMzEw
MDAwLTB4NTAzMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNjk5NTk4XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMzAwMDAwLTB4
NTAzMGZmZmYgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk2
MDJdIHBjaSAwMDBkOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3NTJdIGFjcGlwaHA6IFNsb3QgWzJdIHJlZ2lz
dGVyZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3NjZdIHBjaSAw
MDBkOjAyOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3NzddIHBjaSAwMDBkOjAyOjAwLjA6IHJl
ZyAweDEwOiBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTk3OThdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDMw
OiBbbWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjY5OTgwMl0gcGNpIDAwMGQ6MDI6MDAuMDogZW5hYmxpbmcgRXh0ZW5k
ZWQgVGFncwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk0M10gYWNw
aXBocDogU2xvdCBbM10gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjY5OTk1OF0gcGNpIDAwMGQ6MDM6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFz
cyAweDAxMDgwMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk2OV0g
cGNpIDAwMGQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHg1MDExMDAwMC0weDUwMTEzZmZmIDY0
Yml0XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5OTk5MV0gcGNpIDAw
MGQ6MDM6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDEwMDAwMC0weDUwMTBmZmZmIHByZWZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNjk5OTk0XSBwY2kgMDAwZDowMzow
MC4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAwMTM3XSBhY3BpcGhwOiBTbG90IFs0XSByZWdpc3RlcmVkClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMTUxXSBwY2kgMDAwZDowNDowMC4wOiBbODA4
NjowYTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAwMTYyXSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgxMDogW21lbSAweDUw
MDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAwMTgzXSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMDAwMDAw
LTB4NTAwMGZmZmYgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDAxODddIHBjaSAwMDBkOjA0OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMTZdIHBjaV9idXMgMDAwZDowMDogb24g
TlVNQSBub2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMThd
IHBjaSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBb
YnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwMzE5XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAw
MC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzIw
XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZm
ZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDMyMV0gcGNpIDAwMGQ6MDA6MDIuMDog
YnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEw
MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMjJdIHBjaSAwMDBk
OjAwOjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQg
cHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMjNdIHBjaSAwMDBkOjAwOjAyLjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDJdIGFk
ZF9zaXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzAwMzI0XSBwY2kgMDAwZDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFtpbyAg
MHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDMyNV0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBh
ZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDMyN10gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMjhd
IHBjaSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBb
YnVzIDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwMzI5XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAw
MC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzMw
XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZm
ZmZmXSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDMzNF0gcGNpIDAwMGQ6MDA6MDEuMDog
QkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMzVdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAx
NTogYXNzaWduZWQgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDMzN10gcGNpIDAwMGQ6
MDA6MDIuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzMzhdIHBjaSAwMDBkOjAwOjAy
LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDMzOV0g
cGNpIDAwMGQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA0MDAwMDAtMHg1MDVm
ZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNDBdIHBjaSAw
MDBkOjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAw
MDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcwMDM0MV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAw
MDAtMHg1MDdmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAz
NDJdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDYwMDAw
MC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMDM0M10gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDAzNDRdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM0NV0g
cGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNDZdIHBjaSAwMDBkOjAw
OjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM0N10gcGNpIDAwMGQ6MDA6MDMuMDog
QkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNDhdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMDM0OF0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDAzNDldIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM1
Ml0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNTJdIHBjaSAwMDBk
OjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM1M10gcGNpIDAwMGQ6MDA6MDMu
MDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNTRdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMDM1NV0gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDAzNTZdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MDM1Nl0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDAzNTddIHBjaSAw
MDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM1OV0gcGNpIDAwMGQ6MDE6
MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzYwXSBwY2kgMDAwZDowMTow
MC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzY1XSBwY2kgMDAwZDowMDow
MS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzAwMzY3XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDUwMDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwMzY4XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAw
MDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMDM3MF0gcGNpIDAwMGQ6MDI6MDAuMDogQkFSIDY6IGFzc2lnbmVk
IFttZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAwMzcxXSBwY2kgMDAwZDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQg
W21lbSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzAwMzc2XSBwY2kgMDAwZDowMDowMi4wOiBQQ0kgYnJpZGdlIHRvIFti
dXMgMDJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzc3XSBwY2kg
MDAwZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzc4XSBwY2kgMDAwZDow
MDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM4
MF0gcGNpIDAwMGQ6MDM6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDQwMDAwMC0weDUw
NDBmZmZmIHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzgx
XSBwY2kgMDAwZDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNDEwMDAwLTB4NTA0
MTNmZmYgNjRiaXRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzg1
XSBwY2kgMDAwZDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzg2XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRn
ZSB3aW5kb3cgW21lbSAweDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzAwMzg4XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5k
b3cgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDM4OV0gcGNpIDAwMGQ6MDQ6MDAuMDog
QkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDYwMDAwMC0weDUwNjBmZmZmIHByZWZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzkwXSBwY2kgMDAwZDowNDowMC4wOiBC
QVIgMDogYXNzaWduZWQgW21lbSAweDUwNjEwMDAwLTB4NTA2MTNmZmYgNjRiaXRdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwMzk1XSBwY2kgMDAwZDowMDowNC4wOiBQ
Q0kgYnJpZGdlIHRvIFtidXMgMDRdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwMzk2XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNjAw
MDAwLTB4NTA3ZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAw
Mzk3XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDYwMDAw
MC0weDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMDM5OV0gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA0IFttZW0gMHg1MDAw
MDAwMC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDA0MDBdIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzQwMDAwMDAw
MDAwLTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDA0MDBdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NTAwMDAw
MDAtMHg1MDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA0
MDFdIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzAwNDAyXSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDEgW21lbSAweDUwMjAwMDAwLTB4
NTAzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNDAzXSBw
Y2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MDQwNF0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAxIFttZW0gMHg1MDQwMDAwMC0weDUwNWZm
ZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDQwNV0gcGNpX2J1
cyAwMDBkOjAzOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYg
NjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA0MDVd
IHBjaV9idXMgMDAwZDowNDogcmVzb3VyY2UgMSBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA0MDZdIHBjaV9idXMgMDAw
ZDowNDogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0
IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAwNTE0XSBBQ1BJ
OiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTJdIChkb21haW4gMDAwMSBbYnVzIDAwLWZmXSkKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDA1MThdIGFjcGkgUE5QMEEwODowMjog
X09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBN
U0kgSFBYLVR5cGUzXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMDU4
M10gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0ll
SG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwMDY0MV0gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBub3cgY29udHJvbHMg
W0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDA2NDNdIGFjcGkgUE5QMEEwODowMjogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2Jm
ZmYwMDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZkZWI1
YWNkNGVhNTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDEyNzNdIGFj
cGkgUE5QMEEwODowMjogRUNBTSBhcmVhIFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZm
ZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MDEyNzldIGFjcGkgUE5QMEEwODowMjogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAw
MDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDEzMzldIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMTowMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTM0MF0gcGNpX2J1cyAwMDAx
OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAxMzQxXSBwY2lfYnVzIDAw
MDE6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZm
ZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTM0Ml0g
cGNpX2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTM1OV0gcGNpIDAwMDE6MDA6MDAuMDogWzFk
ZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcwMTQwMl0gcGNpIDAwMDE6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAw
MSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcw
MTQxNV0gcGNpIDAwMDE6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMTQ0MV0gcGNpIDAwMDE6MDA6MDEuMDogc3Vw
cG9ydHMgRDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE0NDJd
IHBjaSAwMDAxOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDE1MzBdIGFjcGlwaHA6IFNsb3QgWzEt
M10gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzYy
MF0gcGNpX2J1cyAwMDAxOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwMzYyMl0gcGNpIDAwMDE6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBb
aW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM2MjNdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAw
MV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDM2MjRdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBh
ZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAz
NjI2XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg2MDAwMDAwMC0w
eDYwMWZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzYyN10g
cGNpIDAwMDE6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4
MzgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAzNjI4XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzYy
OV0gcGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzNjMwXSBwY2kg
MDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzYzMV0gcGNpIDAwMDE6MDA6MDEu
MDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzNjMyXSBwY2kgMDAwMTowMDowMS4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAzNjM0XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDYwMDAwMDAw
LTB4NjAxZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAzNjM1
XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM4MDAwMDAwMDAwMC0w
eDM4MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMzYzN10gcGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA0IFttZW0gMHg2MDAwMDAw
MC0weDZmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDM2MzhdIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzgwMDAwMDAwMDAw
LTB4M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDM2MzldIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMSBbbWVtIDB4NjAwMDAwMDAt
MHg2MDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM2NDBd
IHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzAzNzQ1XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTNdIChkb21haW4gMDAwMCBbYnVzIDAw
LWZmXSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDM3NDldIGFjcGkg
UE5QMEEwODowMzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQ
TSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwMzgxM10gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBz
dXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwMzg3MV0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBu
b3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDM4NzNdIGFjcGkgUE5QMEEwODowMzogTUNGRyBxdWlyazogRUNBTSBh
dCBbbWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0
aCAweGZmZmZkZWI1YWNkNGVhNTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MDQ0OTldIGFjcGkgUE5QMEEwODowMzogRUNBTSBhcmVhIFttZW0gMHgzZmZmZjAwMDAwMDAt
MHgzZmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MDQ1MDVdIGFjcGkgUE5QMEEwODowMzogRUNBTSBhdCBbbWVt
IDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ1NjVdIFBDSSBob3N0IGJyaWRnZSB0byBi
dXMgMDAwMDowMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDU2Nl0g
cGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA0NTY3
XSBwY2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzYzAwMDAwMDAwMDAt
MHgzZmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNDU2OF0gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZm
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDU4NV0gcGNpIDAwMDA6
MDA6MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDYyN10gcGNpIDAwMDA6MDA6MDEuMDogWzFkZWY6
ZTEwMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcwNDY0MF0gcGNpIDAwMDA6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFn
cwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNDY2N10gcGNpIDAwMDA6
MDA6MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDQ2NjddIHBjaSAwMDAwOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEg
RDNob3QKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDQ3NTVdIGFjcGlw
aHA6IFNsb3QgWzEtNF0gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNjg0NV0gcGNpX2J1cyAwMDAwOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjg0Nl0gcGNpIDAwMDA6MDA6MDEuMDogYnJp
ZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY4NDhdIHBjaSAwMDAwOjAw
OjAxLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJl
Zl0gdG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDY4NDldIHBjaSAwMDAwOjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9z
aXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzA2ODUwXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0g
MHg3MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNjg1Ml0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4M2Mw
MDAwMDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzA2ODUzXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcwNjg1M10gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzA2ODU1XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNjg1Nl0gcGNp
IDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA2ODU2XSBwY2kgMDAwMDow
MDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA2ODU4XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21l
bSAweDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA2ODYwXSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDNj
MDAwMDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNjg2Ml0gcGNpX2J1cyAwMDAwOjAwOiByZXNvdXJjZSA0IFtt
ZW0gMHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDY4NjNdIHBjaV9idXMgMDAwMDowMDogcmVzb3VyY2UgNSBbbWVtIDB4
M2MwMDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MDY4NjNdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMSBbbWVt
IDB4NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MDY4NjRdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMiBbbWVtIDB4M2MwMDAwMDAw
MDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzA2OTY4XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTRdIChkb21haW4g
MDAwMiBbYnVzIDAwLWZmXSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MDY5NzJdIGFjcGkgUE5QMEEwODowNDogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmln
IEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcwNzAzNV0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBwbGF0Zm9y
bSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzA5M10gYWNwaSBQTlAwQTA4OjA0
OiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDcwOTVdIGFjcGkgUE5QMEEwODowNDogTUNGRyBx
dWlyazogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1
cyAwMC1mZl0gd2l0aCAweGZmZmZkZWI1YWNkNGVhNTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MDc3MTldIGFjcGkgUE5QMEEwODowNDogRUNBTSBhcmVhIFttZW0gMHgy
M2ZmZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc3MjVdIGFjcGkgUE5QMEEwODowNDog
RUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1m
Zl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc3ODVdIFBDSSBob3N0
IGJyaWRnZSB0byBidXMgMDAwMjowMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcwNzc4Nl0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MDA4
MDAwMDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzA3Nzg3XSBwY2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgy
MDAwMDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcwNzc4OF0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzgw
NV0gcGNpIDAwMDI6MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzg0N10gcGNpIDAwMDI6MDA6
MDEuMDogWzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcwNzg4Ml0gcGNpIDAwMDI6MDA6MDEuMDogc3VwcG9ydHMg
RDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc4ODNdIHBjaSAw
MDAyOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDc5NDRdIHBjaSAwMDAyOjAwOjA1LjA6IFsxZGVm
OmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MDc5NzldIHBjaSAwMDAyOjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzA3OTgwXSBwY2kgMDAwMjowMDowNS4w
OiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzA4MDYzXSBhY3BpcGhwOiBTbG90IFsxLTVdIHJlZ2lzdGVyZWQKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTAxNzZdIGFjcGlwaHA6IFNsb3Qg
WzMtMl0gcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
MjI2OF0gcGNpX2J1cyAwMDAyOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxMjI2OV0gcGNpIDAwMDI6MDA6MDEuMDogYnJpZGdlIHdpbmRv
dyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIyNzFdIHBjaSAwMDAyOjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1
cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTIyNzJdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5k
b3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAw
MCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzEyMjczXSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZm
Zl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxMjI3NF0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAw
MDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxMjI3NV0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAt
MHgwMDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIyNzddIHBjaSAwMDAyOjAw
OjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjc5XSBwY2kgMDAwMjowMDowMS4w
OiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIyODBdIHBj
aSAwMDAyOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjgxXSBwY2kgMDAw
MjowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAz
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTIyODJdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjgzXSBwY2kg
MDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIyODRdIHBjaSAwMDAyOjAw
OjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjg1XSBwY2kgMDAwMjowMDowNS4wOiBCQVIg
MTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTIyODZdIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEyMjg3XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTIyODhdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjg5XSBw
Y2kgMDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIyOTBdIHBjaSAwMDAy
OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTIyOTJdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTIyOTNdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjk1XSBwY2kgMDAwMjowMDowNS4wOiBQQ0kgYnJpZGdl
IHRvIFtidXMgMDJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjk3
XSBwY2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDAwYTAwMDAwLTB4MDBi
ZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMjk4XSBwY2kg
MDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAw
MDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxMjMwMF0gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJjZSA0IFttZW0gMHgwMDgwMDAwMC0weDBm
ZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIz
MDFdIHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNm
ZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTIzMDJdIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMSBbbWVtIDB4MDA4MDAwMDAtMHgwMDlm
ZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTIzMDNdIHBjaV9i
dXMgMDAwMjowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMzA0
XSBwY2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDEgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyMzA1XSBwY2lfYnVzIDAw
MDI6MDI6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMjQxMV0gQUNQ
STogUENJIFJvb3QgQnJpZGdlIFtQQ0k1XSAoZG9tYWluIDAwMDMgW2J1cyAwMC1mZl0pClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEyNDE1XSBhY3BpIFBOUDBBMDg6MDU6
IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMg
TVNJIEhQWC1UeXBlM10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTI0
NzhdIGFjcGkgUE5QMEEwODowNTogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJ
ZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTI1MzZdIGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mgbm93IGNvbnRyb2xz
IFtBRVIgUENJZUNhcGFiaWxpdHldClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzEyNTM4XSBhY3BpIFBOUDBBMDg6MDU6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDI3
ZmZmMDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmZGVi
NWFjZDRlYTU4ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzMTY3XSBh
Y3BpIFBOUDBBMDg6MDU6IEVDQU0gYXJlYSBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZm
ZmZmXSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEzMTczXSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAw
MDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzEzMjI2XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDM6MDAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTMyMjddIHBjaV9idXMgMDAw
MzowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDEwMDAwMDAwLTB4MWZmZmZmZmYgd2luZG93
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzIyOF0gcGNpX2J1cyAw
MDAzOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjdmZmRmZmZm
ZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTMyMjld
IHBjaV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTMyNDVdIHBjaSAwMDAzOjAwOjAwLjA6IFsx
ZGVmOmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTMyODZdIHBjaSAwMDAzOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUg
MDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTMzMjFdIHBjaSAwMDAzOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzMzIyXSBwY2kgMDAwMzowMDowMS4wOiBQTUUjIHN1cHBv
cnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzEzMzc5XSBwY2kgMDAwMzowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNDA1XSBwY2kg
MDAwMzowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxMzQwNl0gcGNpIDAwMDM6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzQ3NF0g
YWNwaXBocDogU2xvdCBbMS02XSByZWdpc3RlcmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE1NTk2XSBwY2kgMDAwMzowMjowMC4wOiBbODA4NjoxNTIxXSB0eXBlIDAw
IGNsYXNzIDB4MDIwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
NjExXSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDEwMDIwMDAwLTB4MTAwM2Zm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NjI4XSBwY2kgMDAw
MzowMjowMC4wOiByZWcgMHgxODogW2lvICAweDFmZmY4MDIwLTB4MWZmZjgwM2ZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NjM2XSBwY2kgMDAwMzowMjowMC4wOiBy
ZWcgMHgxYzogW21lbSAweDEwMDQ0MDAwLTB4MTAwNDdmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE1NzQ3XSBwY2kgMDAwMzowMjowMC4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNTc4NV0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAt
MHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNTc4Nl0gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAw
MDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTgwN10gcGNpIDAwMDM6MDI6
MDAuMDogcmVnIDB4MTkwOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTgwOF0gcGNpIDAwMDM6MDI6
MDAuMDogVkYobikgQkFSMyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQg
cHJlZl0gKGNvbnRhaW5zIEJBUjMgZm9yIDggVkZzKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNTkwNF0gcGNpIDAwMDM6MDI6MDAuMDogOC4wMDAgR2IvcyBhdmFpbGFi
bGUgUENJZSBiYW5kd2lkdGgsIGxpbWl0ZWQgYnkgNS4wIEdUL3MgUENJZSB4MiBsaW5rIGF0IDAw
MDM6MDA6MDUuMCAoY2FwYWJsZSBvZiAxNi4wMDAgR2IvcyB3aXRoIDUuMCBHVC9zIFBDSWUgeDQg
bGluaykKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NTFdIHBjaSAw
MDAzOjAyOjAwLjE6IFs4MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAwMDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NjZdIHBjaSAwMDAzOjAyOjAwLjE6IHJl
ZyAweDEwOiBbbWVtIDB4MTAwMDAwMDAtMHgxMDAxZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTU5ODNdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDE4OiBbaW8g
IDB4MWZmZjgwMDAtMHgxZmZmODAxZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTU5OTFdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDFjOiBbbWVtIDB4MTAwNDAwMDAt
MHgxMDA0M2ZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYwOThd
IHBjaSAwMDAzOjAyOjAwLjE6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MTMxXSBwY2kgMDAwMzowMjow
MC4xOiByZWcgMHgxODQ6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MTMyXSBwY2kgMDAwMzowMjow
MC4xOiBWRihuKSBCQVIwIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBw
cmVmXSAoY29udGFpbnMgQkFSMCBmb3IgOCBWRnMpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2MTUzXSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxOTA6IFttZW0gMHgw
MDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2MTU0XSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIzIHNwYWNlOiBb
bWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMyBmb3Ig
OCBWRnMpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MjcyXSBwY2lf
YnVzIDAwMDM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2MjczXSBwY2lfYnVzIDAwMDM6MDA6IG1heCBidXMgZGVwdGg6IDEgcGNpX3Ry
eV9udW06IDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYyNzddIHBj
aSAwMDAzOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2Mjc4XSBwY2kgMDAw
MzowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAx
ZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTYyNzldIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAw
LTB4MTAzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2Mjgw
XSBwY2kgMDAwMzowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyMDAwMDAt
MHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTYyODFdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
MjgyXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYyODNdIHBj
aSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MjgzXSBwY2kgMDAwMzowMDow
NS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYyODVdIHBjaSAwMDAzOjAwOjAxLjA6IFBD
SSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTYyODZdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAw
MDAtMHgxMDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYy
ODhdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAw
LTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2MjkxXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAw
eDEwMjAwMDAwLTB4MTAyMWZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE2Mjk0XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjIw
MDAwLTB4MTAyM2ZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
Mjk3XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQwMDAwLTB4
MTAyNDNmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzAwXSBw
Y2kgMDAwMzowMjowMC4wOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0
MDAwMDIxZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjMwNV0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQw
MDAwMjIwMDAwLTB4MjQwMDAwMjNmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE2MzA5XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMzogYXNzaWdu
ZWQgW21lbSAweDEwMjQ0MDAwLTB4MTAyNDdmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2MzEyXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgNzogYXNzaWduZWQgW21l
bSAweDI0MDAwMDI0MDAwMC0weDI0MDAwMDI1ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjMxNl0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDEw
OiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAwMjYwMDAwLTB4MjQwMDAwMjdmZmZmIDY0Yml0IHByZWZd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzIxXSBwY2kgMDAwMzow
MjowMC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzIxXSBwY2kgMDAwMzowMjowMC4wOiBCQVIg
MjogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjMyMl0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNjMyM10gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MDAyMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTYzMjRdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzMjVdIHBjaSAwMDAzOjAwOjA1LjA6ICAg
YnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzMjddIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzI5XSBwY2lfYnVzIDAwMDM6
MDA6IE5vLiAyIHRyeSB0byBhc3NpZ24gdW5hc3NpZ25lZCByZXMKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTYzMzFdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2MzMxXSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTYzMzJdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzMzXSBw
Y2kgMDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzMzRdIHBjaSAwMDAz
OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTYzMzZdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTYzMzddIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MjQwMDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzM5XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2MzQwXSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2ln
biBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjM0MF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgwMDIwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM0MV0gcGNp
IDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzNDJdIHBjaSAwMDAzOjAw
OjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTYzNDNdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTYzNDVdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQw
MDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE2MzQ2XSBwY2lfYnVzIDAwMDM6MDA6IEF1dG9tYXRpY2FsbHkg
ZW5hYmxlZCBwY2kgcmVhbGxvYywgaWYgeW91IGhhdmUgcHJvYmxlbSwgdHJ5IGJvb3Rpbmcgd2l0
aCBwY2k9cmVhbGxvYz1vZmYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTYzNDhdIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2UgNCBbbWVtIDB4MTAwMDAwMDAtMHgxZmZm
ZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzQ4
XSBwY2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDUgW21lbSAweDI0MDAwMDAwMDAwMC0weDI3ZmZk
ZmZmZmZmZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
MzQ5XSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDEgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZm
ZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MzUwXSBwY2lfYnVz
IDAwMDM6MDE6IHJlc291cmNlIDIgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM1MV0g
cGNpX2J1cyAwMDAzOjAyOiByZXNvdXJjZSAxIFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjM1Ml0gcGNpX2J1cyAwMDAz
OjAyOiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY0MThdIEFDUEk6
IFBDSSBSb290IEJyaWRnZSBbUENJNl0gKGRvbWFpbiAwMDA0IFtidXMgMDAtZmZdKQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjQyMl0gYWNwaSBQTlAwQTA4OjA2OiBf
T1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1T
SSBIUFgtVHlwZTNdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NDg3
XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVI
b3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2NTQ2XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIG5vdyBjb250cm9scyBb
QUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjU0OF0gYWNwaSBQTlAwQTA4OjA2OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyYmZm
ZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmRlYjVh
Y2Q0ZWE1OApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzE3OF0gYWNw
aSBQTlAwQTA4OjA2OiBFQ0FNIGFyZWEgW21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZm
Zl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzE4NF0gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAw
MDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNzIzNV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA0OjAwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MjM2XSBwY2lfYnVzIDAwMDQ6
MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcyMzddIHBjaV9idXMgMDAw
NDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZm
ZiB3aW5kb3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MjM4XSBw
Y2lfYnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MjU1XSBwY2kgMDAwNDowMDowMC4wOiBbMWRl
ZjplMTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE3MjkyXSBwY2kgMDAwNDowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAx
IGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3
MzE5XSBwY2kgMDAwNDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNzMyMF0gcGNpIDAwMDQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0
ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNzM2Ml0gcGNpIDAwMDQ6MDA6MDMuMDogWzFkZWY6ZTExM10gdHlwZSAwMSBjbGFzcyAweDA2
MDQwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzM4OF0gcGNpIDAw
MDQ6MDA6MDMuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTczODldIHBjaSAwMDA0OjAwOjAzLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAg
RDEgRDNob3QKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc0MzRdIHBj
aSAwMDA0OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc0NzBdIHBjaSAwMDA0OjAwOjA1LjA6
IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3
NDcxXSBwY2kgMDAwNDowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NTQyXSBwY2kgMDAwNDowMTow
MC4wOiBbMWEwMzoxMTUwXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NTkwXSBwY2kgMDAwNDowMTowMC4wOiBlbmFibGluZyBF
eHRlbmRlZCBUYWdzClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NjQ1
XSBwY2kgMDAwNDowMTowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNzY0Nl0gcGNpIDAwMDQ6MDE6MDAuMDogUE1FIyBzdXBwb3J0ZWQg
ZnJvbSBEMCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTc3NDFdIHBjaV9idXMgMDAwNDowMjogZXh0ZW5kZWQgY29uZmlnIHNwYWNlIG5v
dCBhY2Nlc3NpYmxlClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3NzU4
XSBwY2kgMDAwNDowMjowMC4wOiBbMWEwMzoyMDAwXSB0eXBlIDAwIGNsYXNzIDB4MDMwMDAwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3Nzc2XSBwY2kgMDAwNDowMjow
MC4wOiByZWcgMHgxMDogW21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3Nzg2XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgx
NDogW21lbSAweDIyMDAwMDAwLTB4MjIwMWZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE3Nzk2XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxODogW2lvICAweDJm
ZmY4MDAwLTB4MmZmZjgwN2ZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE3ODQxXSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgdG8gZWZpZmIKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc4NzddIHBjaSAwMDA0OjAyOjAwLjA6
IHN1cHBvcnRzIEQxIEQyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3
ODc3XSBwY2kgMDAwNDowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90
IEQzY29sZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzk4N10gcGNp
IDAwMDQ6MDM6MDAuMDogWzE5MTI6MDAxNF0gdHlwZSAwMCBjbGFzcyAweDBjMDMzMApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAwOV0gcGNpIDAwMDQ6MDM6MDAuMDog
cmVnIDB4MTA6IFttZW0gMHgyMjIwMDAwMC0weDIyMjAxZmZmIDY0Yml0XQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODExN10gcGNpIDAwMDQ6MDM6MDAuMDogUE1FIyBz
dXBwb3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTkyNTBdIHBjaV9idXMgMDAwNDowMDogb24gTlVNQSBub2RlIDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkyNTNdIHBjaSAwMDA0OjAwOjAxLjA6
IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8g
W2J1cyAwMS0wMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkyNTVdIHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRn
ZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjU2XSBwY2kgMDAwNDowMDow
My4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZd
IHRvIFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjU3XSBwY2kgMDAwNDowMDowMy4wOiBicmlk
Z2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6
ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxOTI1OF0gcGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAw
MC0weDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTkyNTldIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwNF0gYWRkX3Np
emUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTkyNjBdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAw
MTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24g
MTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjY0XSBwY2kg
MDAwNDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZm
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI2NV0gcGNpIDAwMDQ6
MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5
MjY2XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0w
eDIzMWZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI2N10g
cGNpIDAwMDQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4
MjgwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE5MjY4XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgy
MzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxOTI2OV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAw
NDAwMDAwLTB4MjgwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE5MjcwXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxOTI3MF0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5
MjcxXSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI3Ml0gcGNpIDAw
MDQ6MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjczXSBwY2kgMDAwNDowMDow
NS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI3NF0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE5Mjc2XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNw
YWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxOTI3N10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWdu
IFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE5Mjc4XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI3OV0gcGNp
IDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjgwXSBwY2kgMDAwNDow
MDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI4MV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFS
IDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MjgyXSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTQ6IGFz
c2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxOTI4M10gcGNpIDAwMDQ6MDE6MDAuMDogQkFSIDEzOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTkyODNdIHBjaSAwMDA0OjAxOjAwLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgxMDAwXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
OTI4NV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0w
eDIxZmZmZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI4OF0g
cGNpIDAwMDQ6MDI6MDAuMDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHgyMjAwMDAwMC0weDIyMDFm
ZmZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI5Ml0gcGNpIDAw
MDQ6MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDgwXQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTI5M10gcGNpIDAwMDQ6MDI6MDAuMDog
QkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDA4MF0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkyOTRdIHBjaSAwMDA0OjAxOjAwLjA6IFBDSSBicmlk
Z2UgdG8gW2J1cyAwMl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTky
OThdIHBjaSAwMDA0OjAxOjAwLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgy
MmZmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzMDVdIHBj
aSAwMDA0OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMS0wMl0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzMDZdIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdl
IHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTkzMDddIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MzA5XSBwY2kgMDAwNDowMzowMC4wOiBC
QVIgMDogYXNzaWduZWQgW21lbSAweDIzMDAwMDAwLTB4MjMwMDFmZmYgNjRiaXRdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MzE5XSBwY2kgMDAwNDowMDowMy4wOiBQ
Q0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE5MzIwXSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIzMDAw
MDAwLTB4MjMxZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5
MzIxXSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDIwMDAw
MC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxOTMyM10gcGNpIDAwMDQ6MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0
XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTMyNV0gcGNpIDAwMDQ6
MDA6MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTMyNl0gcGNpIDAwMDQ6MDA6MDUu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzMjhdIHBj
aV9idXMgMDAwNDowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25lZCwg
dHJ5IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxOTMyOV0gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA0IFttZW0gMHgyMDAw
MDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTkzMzBdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjgwMDAwMDAw
MDAwLTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTkzMzFdIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMSBbbWVtIDB4MjAwMDAw
MDAtMHgyMmZmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkz
MzJdIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4Mjgw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE5MzMzXSBwY2lfYnVzIDAwMDQ6MDI6IHJlc291cmNlIDEgW21lbSAweDIwMDAwMDAwLTB4
MjJmZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MzM0XSBw
Y2lfYnVzIDAwMDQ6MDM6IHJlc291cmNlIDEgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5MzM0XSBwY2lfYnVzIDAwMDQ6
MDM6IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTMzNV0gcGNpX2J1
cyAwMDA0OjA0OiByZXNvdXJjZSAxIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTMzNl0gcGNpX2J1cyAwMDA0OjA0OiBy
ZXNvdXJjZSAyIFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTkzNjRdIEFDUEk6IFBDSSBS
b290IEJyaWRnZSBbUENJN10gKGRvbWFpbiAwMDA1IFtidXMgMDAtZmZdKQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTM2OF0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBP
UyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgt
VHlwZTNdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE5NDMyXSBhY3Bp
IFBOUDBBMDg6MDc6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVn
IFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE5NDkwXSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBD
SWVDYXBhYmlsaXR5XQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxOTQ5
Ml0gYWNwaSBQTlAwQTA4OjA3OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAw
MDAtMHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmRlYjVhY2Q0ZWE1
OApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDExOF0gYWNwaSBQTlAw
QTA4OjA3OiBFQ0FNIGFyZWEgW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gcmVz
ZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMDEyNF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgy
ZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcyMDE4MV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA1OjAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMTgyXSBwY2lfYnVzIDAwMDU6MDA6IHJv
b3QgYnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMC0weDNmZmZmZmZmIHdpbmRvd10KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAxODNdIHBjaV9idXMgMDAwNTowMDog
cm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5k
b3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMTg0XSBwY2lfYnVz
IDAwMDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjAwXSBwY2kgMDAwNTowMDowMC4wOiBbMWRlZjplMTEw
XSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzIwMjQxXSBwY2kgMDAwNTowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNz
IDB4MDYwNDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjc3XSBw
Y2kgMDAwNTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyMDI3OF0gcGNpIDAwMDU6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJv
bSBEMCBEMSBEM2hvdApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDMz
OF0gcGNpIDAwMDU6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDM3NF0gcGNpIDAwMDU6MDA6
MDUuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjAzNzVdIHBjaSAwMDA1OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNo
b3QKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA0MjZdIHBjaSAwMDA1
OjAwOjA3LjA6IFsxZGVmOmUxMTddIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjA0NjJdIHBjaSAwMDA1OjAwOjA3LjA6IHN1cHBv
cnRzIEQxIEQyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwNDYzXSBw
Y2kgMDAwNTowMDowNy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwNTM5XSBhY3BpcGhwOiBTbG90IFsxLTdd
IHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NDZd
IHBjaV9idXMgMDAwNTowMDogb24gTlVNQSBub2RlIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjQ3NDhdIHBjaSAwMDA1OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lv
ICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzQ5XSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ug
d2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFd
IGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI0NzUwXSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFtt
ZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRk
X2FsaWduIDEwMDAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc1
MV0gcGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRv
IFtidXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQ3NTJdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAw
MDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFk
ZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3
NTNdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAw
ZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzU0XSBwY2kgMDAwNTowMDowNy4w
OiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUg
MTAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc1NV0gcGNpIDAw
MDU6MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJp
dCBwcmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc1Nl0gcGNpIDAwMDU6MDA6MDcu
MDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwM10g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjQ3NTldIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQg
W21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzI0NzYxXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0g
MHgyYzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NjFdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxNDog
YXNzaWduZWQgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI0NzYzXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTU6IGFzc2ln
bmVkIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NjRdIHBjaSAwMDA1OjAwOjA3LjA6
IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzY1XSBwY2kgMDAwNTowMDowNy4wOiBCQVIg
MTU6IGFzc2lnbmVkIFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NjZdIHBjaSAwMDA1
OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzY3XSBwY2kgMDAwNTowMDowMS4wOiBC
QVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NjhdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI0NzY4XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MjQ3NjldIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Nzcw
XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NzJdIHBjaSAw
MDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzczXSBwY2kgMDAwNTowMDowNy4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NzRdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAx
Mzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI0Nzc1XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0
byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjQ3NzZdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0
Nzc2XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXpl
IDB4MTAwMF0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NzddIHBj
aSAwMDA1OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3NzldIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjQ3ODBdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzgyXSBwY2kgMDAwNTowMDowNS4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDJdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI0Nzg0XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMjAwMDAw
LTB4MzAzZmZmZmZdClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Nzg1
XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDIwMDAwMC0w
eDJjMDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyNDc4N10gcGNpIDAwMDU6MDA6MDcuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc4OV0gcGNpIDAwMDU6MDA6
MDcuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc5MF0gcGNpIDAwMDU6MDA6MDcuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3OTNdIHBjaV9i
dXMgMDAwNTowMDogcmVzb3VyY2UgNCBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3aW5kb3dd
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NzkzXSBwY2lfYnVzIDAw
MDU6MDA6IHJlc291cmNlIDUgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5k
b3ddClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Nzk0XSBwY2lfYnVz
IDAwMDU6MDE6IHJlc291cmNlIDEgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0Nzk1XSBwY2lfYnVzIDAwMDU6MDE6IHJl
c291cmNlIDIgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc5Nl0gcGNpX2J1cyAwMDA1
OjAyOiByZXNvdXJjZSAxIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDc5N10gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJj
ZSAyIFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3OTddIHBjaV9idXMgMDAwNTowMzog
cmVzb3VyY2UgMSBbbWVtIDB4MzA0MDAwMDAtMHgzMDVmZmZmZl0KU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MjQ3OThdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMiBb
bWVtIDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3MjczXSBpb21tdTogRGVmYXVsdCBkb21haW4g
dHlwZTogVHJhbnNsYXRlZCAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjczMDddIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogVkdBIGRldmljZSBhZGRlZDogZGVjb2Rl
cz1pbyttZW0sb3ducz1ub25lLGxvY2tzPW5vbmUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjczMDhdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogYnJpZGdlIGNvbnRy
b2wgcG9zc2libGUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjczMTBd
IHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogc2V0dGluZyBhcyBib290IGRldmljZSAoVkdBIGxl
Z2FjeSByZXNvdXJjZXMgbm90IGF2YWlsYWJsZSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjczMTFdIHZnYWFyYjogbG9hZGVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI3NDAwXSBFREFDIE1DOiBWZXI6IDMuMC4wClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI3NTMzXSBSZWdpc3RlcmVkIGVmaXZhcnMgb3BlcmF0
aW9ucwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODIyNV0gTmV0TGFi
ZWw6IEluaXRpYWxpemluZwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
ODIyNl0gTmV0TGFiZWw6ICBkb21haW4gaGFzaCBzaXplID0gMTI4ClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4MjI3XSBOZXRMYWJlbDogIHByb3RvY29scyA9IFVOTEFC
RUxFRCBDSVBTT3Y0IENBTElQU08KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjgyMzddIE5ldExhYmVsOiAgdW5sYWJlbGVkIHRyYWZmaWMgYWxsb3dlZCBieSBkZWZhdWx0
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4MzY3XSBjbG9ja3NvdXJj
ZTogU3dpdGNoZWQgdG8gY2xvY2tzb3VyY2UgYXJjaF9zeXNfY291bnRlcgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjg4Ml0gVkZTOiBEaXNrIHF1b3RhcyBkcXVvdF82
LjYuMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjg5OV0gVkZTOiBE
cXVvdC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDUxMiAob3JkZXIgMCwgNDA5NiBieXRlcykK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcwMTVdIEFwcEFybW9yOiBB
cHBBcm1vciBGaWxlc3lzdGVtIEVuYWJsZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzcwNDldIHBucDogUG5QIEFDUEkgaW5pdApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczNzY3NF0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4M2JmZmYwMDAwMDAw
LTB4M2JmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3Njc1XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgz
ZmZmZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc2NzZdIHN5c3RlbSAwMDow
MDogW21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBi
ZSByZXNlcnZlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzY3N10g
c3lzdGVtIDAwOjAwOiBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmIHdpbmRvd10g
Y291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM3Njc4XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZm
ZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43Mzc2NzldIHN5c3RlbSAwMDowMDogW21lbSAweDJmZmZmMDAwMDAwMC0w
eDJmZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzY4MF0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MzNm
ZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3NjgyXSBzeXN0ZW0gMDA6MDA6
IFttZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUg
cmVzZXJ2ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc2ODVdIHN5
c3RlbSAwMDowMDogUGx1ZyBhbmQgUGxheSBBQ1BJIGRldmljZSwgSURzIFBOUDBjMDIgKGFjdGl2
ZSkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mzc5NzhdIHBucDogUG5Q
IEFDUEk6IGZvdW5kIDEgZGV2aWNlcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczODg4OV0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAyClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM4OTU1XSBJUCBpZGVudHMgaGFzaCB0YWJsZSBl
bnRyaWVzOiAyNjIxNDQgKG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQwNTU0XSB0Y3BfbGlzdGVuX3BvcnRhZGRy
X2hhc2ggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDcsIDUyNDI4OCBieXRlcywg
bGluZWFyKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MDg0Ml0gVENQ
IGVzdGFibGlzaGVkIGhhc2ggdGFibGUgZW50cmllczogNTI0Mjg4IChvcmRlcjogMTAsIDQxOTQz
MDQgYnl0ZXMsIGxpbmVhcikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDE5MjNdIFRDUCBiaW5kIGhhc2ggdGFibGUgZW50cmllczogNjU1MzYgKG9yZGVyOiA4LCAxMDQ4
NTc2IGJ5dGVzLCBsaW5lYXIpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzQyMTgyXSBUQ1A6IEhhc2ggdGFibGVzIGNvbmZpZ3VyZWQgKGVzdGFibGlzaGVkIDUyNDI4OCBi
aW5kIDY1NTM2KQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MjI2NV0g
VURQIGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBs
aW5lYXIpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQyNjY5XSBVRFAt
TGl0ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywg
bGluZWFyKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MzQ0OF0gTkVU
OiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQzNDU0XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDQ0ClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzNDYyXSBwY2kgMDAwYzowMTow
MC4wOiBDTFMgbWlzbWF0Y2ggKDY0ICE9IDMyKSwgdXNpbmcgNjQgYnl0ZXMKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM1MzhdIFRyeWluZyB0byB1bnBhY2sgcm9vdGZz
IGltYWdlIGFzIGluaXRyYW1mcy4uLgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjE4NjI4Ml0gRnJlZWluZyBpbml0cmQgbWVtb3J5OiA0MTAxMksKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODc5OTRdIGh3IHBlcmZldmVudHM6IGVuYWJsZWQgd2l0
aCBhcm12OF9wbXV2M18wIFBNVSBkcml2ZXIsIDcgY291bnRlcnMgYXZhaWxhYmxlClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg4MDgwXSBrdm0gWzFdOiBJUEEgU2l6ZSBM
aW1pdDogNDggYml0cwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjE4ODEw
Ml0ga3ZtIFsxXTogR0lDdjM6IG5vIEdJQ1YgcmVzb3VyY2UgZW50cnkKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODgxMDNdIGt2bSBbMV06IGRpc2FibGluZyBHSUN2MiBl
bXVsYXRpb24KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODgxMTRdIGt2
bSBbMV06IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZSBlbmFibGVkClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg4NDU5XSBrdm0gWzFdOiB2Z2ljIGludGVy
cnVwdCBJUlE5ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTg4ODg1XSBr
dm0gWzFdOiBWSEUgbW9kZSBpbml0aWFsaXplZCBzdWNjZXNzZnVsbHkKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODk3MTNdIEluaXRpYWxpc2Ugc3lzdGVtIHRydXN0ZWQg
a2V5cmluZ3MKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xODk3MjFdIEtl
eSB0eXBlIGJsYWNrbGlzdCByZWdpc3RlcmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMTg5NzU5XSB3b3JraW5nc2V0OiB0aW1lc3RhbXBfYml0cz00MiBtYXhfb3JkZXI9
MjQgYnVja2V0X29yZGVyPTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4x
OTA0MjddIHpidWQ6IGxvYWRlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjE5MDU0NV0gaW50ZWdyaXR5OiBQbGF0Zm9ybSBLZXlyaW5nIGluaXRpYWxpemVkClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTkwNTQ3XSBLZXkgdHlwZSBhc3ltbWV0cmlj
IHJlZ2lzdGVyZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTA1NDhd
IEFzeW1tZXRyaWMga2V5IHBhcnNlciAneDUwOScgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjE5MDU1NF0gQmxvY2sgbGF5ZXIgU0NTSSBnZW5lcmljIChi
c2cpIGRyaXZlciB2ZXJzaW9uIDAuNCBsb2FkZWQgKG1ham9yIDI0OSkKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTA1OTRdIGlvIHNjaGVkdWxlciBtcS1kZWFkbGluZSBy
ZWdpc3RlcmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTkxNTcwXSBz
aHBjaHA6IFN0YW5kYXJkIEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAw
LjQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTE2ODJdIGVmaWZiOiBw
cm9iaW5nIGZvciBlZmlmYgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5
MTcwMV0gZWZpZmI6IGZyYW1lYnVmZmVyIGF0IDB4MjAwMDAwMDAsIHVzaW5nIDE4NzZrLCB0b3Rh
bCAxODc1awpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5MTcwMl0gZWZp
ZmI6IG1vZGUgaXMgODAweDYwMHgzMiwgbGluZWxlbmd0aD0zMjAwLCBwYWdlcz0xClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTkxNzAzXSBlZmlmYjogc2Nyb2xsaW5nOiBy
ZWRyYXcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4xOTE3MDRdIGVmaWZi
OiBUcnVlY29sb3I6IHNpemU9ODo4Ojg6OCwgc2hpZnQ9MjQ6MTY6ODowClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMTkxNzc4XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29s
b3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMTAweDM3ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMTk3NjgwXSBmYjA6IEVGSSBWR0EgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5ODA4OF0gaW5wdXQ6IFBvd2VyIEJ1
dHRvbiBhcyAvZGV2aWNlcy9MTlhTWVNUTTowMC9MTlhTWUJVUzowMC9QTlAwQzBDOjAwL2lucHV0
L2lucHV0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjE5ODEwMl0gQUNQ
STogUG93ZXIgQnV0dG9uIFtQV1JCXQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIwMjQ2OV0gQUNQSSBHVERUOiBmb3VuZCAxIFNCU0EgZ2VuZXJpYyBXYXRjaGRvZyhzKS4K
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDI5OTRdIFNlcmlhbDogODI1
MC8xNjU1MCBkcml2ZXIsIDQgcG9ydHMsIElSUSBzaGFyaW5nIGVuYWJsZWQKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDMyNTFdIFNlcmlhbDogQU1CQSBkcml2ZXIKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDMyODNdIG1zbV9zZXJpYWw6IGRy
aXZlciBpbml0aWFsaXplZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
MzMzMl0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBvcHRpb24gbWFzayAweDAKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDMzNTZdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjAuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAw
MWZmZikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDM1MjRdIGFybS1z
bW11LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNt
ZHEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDM1NjBdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDM2MjNdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDM5ODhdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjEuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjAzOTk5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGlh
cyA0OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA0MTEzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4x
LmF1dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjA0MTUwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1
dG86IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjA0MjE0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86
IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjA0NTUyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IG9w
dGlvbiBtYXNrIDB4MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNDU2
NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0
IChmZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIwNDYxM10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1
MzYgZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIwNDcyNF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgMzI3Njgg
ZW50cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
NDc2MF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50
cmllcyBmb3IgcHJpcQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNTEw
MV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDUxMTFdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjMuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZm
ZikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDUxNTZdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDUyNjZdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDUzMDFdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDU1OTddIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjQuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjA1NjA2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGlhcyA0
OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjA1NjczXSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1
dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjA1NzE3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86
IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjA1ODI3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjA2MTI3XSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IG9wdGlv
biBtYXNrIDB4MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjEzN10g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChm
ZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIwNjIwMV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYg
ZW50cmllcyBmb3IgY21kcQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
NjI0NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50
cmllcyBmb3IgZXZ0cQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjM1
NV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmll
cyBmb3IgcHJpcQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNjY3NV0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDY2ODRdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjYuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDY3MjJdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDY3ODZdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDY4MjldIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDcyMDRdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjcuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjA3MjEzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGlhcyA0OC1i
aXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjA3MjQ5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86
IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjA3MzExXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFs
bG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjA3MzUyXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9j
YXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjExODU5XSBtb3VzZWRldjogUFMvMiBtb3VzZSBkZXZpY2UgY29tbW9uIGZvciBh
bGwgbWljZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxMjgyMV0gcnRj
LWVmaSBydGMtZWZpLjA6IHJlZ2lzdGVyZWQgYXMgcnRjMApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAxLjIxMzI5MF0gcnRjLWVmaSBydGMtZWZpLjA6IHNldHRpbmcgc3lzdGVt
IGNsb2NrIHRvIDIwMjEtMDktMTNUMTA6MjA6MjkgVVRDICgxNjMxNTI4NDI5KQpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxMzg2M10gbGVkdHJpZy1jcHU6IHJlZ2lzdGVy
ZWQgdG8gaW5kaWNhdGUgYWN0aXZpdHkgb24gQ1BVcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIxNDEyNF0gU01DQ0M6IFNPQ19JRDogSUQgPSBqZXAxMDY6MGExNjowMDAx
IFJldmlzaW9uID0gMHgwMDAwMDBhMQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjIxNDM4Ml0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxMApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjQ0NF0gU2VnbWVudCBSb3V0aW5nIHdpdGgg
SVB2NgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjQ1Nl0gbWlwNjog
TW9iaWxlIElQdjYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjI0NThd
IE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTcKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yMjI0ODBdIG1wbHNfZ3NvOiBNUExTIEdTTyBzdXBwb3J0ClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjIyNTI5XSByZWdpc3RlcmVkIHRhc2tz
dGF0cyB2ZXJzaW9uIDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjI1
MzFdIExvYWRpbmcgY29tcGlsZWQtaW4gWC41MDkgY2VydGlmaWNhdGVzClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjQ4NzE1XSBMb2FkZWQgWC41MDkgY2VydCAnRGViaWFu
IFNlY3VyZSBCb290IENBOiA2Y2NlY2U3ZTRjNmMwZDFmNjE0OWYzZGQyN2RmY2M1Y2JiNDE5ZWEx
JwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0ODcyNF0gTG9hZGVkIFgu
NTA5IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBTaWduZXIgMjAyMSAtIGxpbnV4OiA0YjZlZjVh
YmNhNjY5ODI1MTc4ZTA1MmM4NDY2N2NjYmMwNTMxZjhjJwpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI0ODg4MF0genN3YXA6IGxvYWRlZCB1c2luZyBwb29sIGx6by96YnVk
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjQ4OTczXSBLZXkgdHlwZSAu
X2ZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjI0ODk3NF0gS2V5IHR5cGUgLmZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjI0ODk3NV0gS2V5IHR5cGUgZnNjcnlwdC1wcm92aXNpb25pbmcg
cmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI0OTAwNV0g
QXBwQXJtb3I6IEFwcEFybW9yIHNoYTEgcG9saWN5IGhhc2hpbmcgZW5hYmxlZApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1ODA2OV0gaW50ZWdyaXR5OiBMb2FkaW5nIFgu
NTA5IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjU4NzYzXSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdTVVNFIExpbnV4IEVu
dGVycHJpc2UgU2VjdXJlIEJvb3QgQ0E6IDNkNGQ0MGNmOTM4NTM5MDI0YjFjZmM1YTEyZGVkZmU4
YjE3ZTc1NWYnClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjU4NzY0XSBp
bnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTg5MjNdIGludGVncml0eTogTG9hZGVkIFguNTA5
IGNlcnQgJ0Nhbm9uaWNhbCBMdGQuIE1hc3RlciBDZXJ0aWZpY2F0ZSBBdXRob3JpdHk6IGFkOTE5
OTBiYzIyYWIxZjUxNzA0OGMyM2I2NjU1YTI2OGUzNDVhNjMnClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjU4OTI0XSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlm
aWNhdGU6IFVFRkk6ZGIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTkw
ODFdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0ZlZG9yYSBTZWN1cmUgQm9vdCBDQTog
ZmRlMzI1OTljMmQ2MWRiMWJmNTgwNzMzNWQ3YjIwZTRjZDk2M2I0MicKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTkwODFdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBj
ZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjI1OTA5M10gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3Mg
UHJvZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1
YWY1MycKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNTkwOTRdIGludGVn
cml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI1OTEwNl0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2Vy
dCAnTWljcm9zb2Z0IENvcnBvcmF0aW9uIFVFRkkgQ0EgMjAxMTogMTNhZGJmNDMwOWJkODI3MDlj
OGNkNTRmMzE2ZWQ1MjI5ODhhMWJkNCcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yNTkxMDZdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpk
YgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI1OTExN10gaW50ZWdyaXR5
OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAx
MTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjM1NTNdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDog
QWRkaW5nIHRvIGlvbW11IGdyb3VwIDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yNjM4NTFdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJR
IDEwNgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2MzkyNV0gcGNpZXBv
cnQgMDAwZDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMQpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NDE4NV0gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBRVI6
IGVuYWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjY0MjMwXSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAy
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY0NDg3XSBwY2llcG9ydCAw
MDBkOjAwOjAyLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yNjQ1MzBdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
NjQ3NjZdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NDgwOF0gcGNpZXBvcnQgMDAw
ZDowMDowNC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI2NTA0NF0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBRVI6IGVuYWJs
ZWQgd2l0aCBJUlEgMTA3ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY1
MTA1XSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCA1ClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY1MzY0XSBwY2llcG9ydCAwMDAxOjAw
OjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yNjU0MzFdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQWRkaW5nIHRvIGlv
bW11IGdyb3VwIDYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjU2OTZd
IHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwOQpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NTc2NF0gcGNpZXBvcnQgMDAwMjowMDow
MS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAxLjI2NjAzNl0gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0
aCBJUlEgMTEwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY2MDg1XSBw
Y2llcG9ydCAwMDAyOjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA4ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY2MzQ3XSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6
IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS4yNjY0MTJdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdy
b3VwIDkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjY2ODJdIHBjaWVw
b3J0IDAwMDM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NjczN10gcGNpZXBvcnQgMDAwMzowMDowNS4wOiBB
ZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yNjY5NzZdIHBjaWVwb3J0IDAwMDM6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJR
IDExMQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NzAzOF0gcGNpZXBv
cnQgMDAwNDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTEKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjczMDRdIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQUVS
OiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjI2NzM0OV0gcGNpZXBvcnQgMDAwNDowMDowMy4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAg
MTIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjc1ODldIHBjaWVwb3J0
IDAwMDQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NzYzOV0gcGNpZXBvcnQgMDAwNDowMDowNS4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgMTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yNjc4ODFdIHBjaWVwb3J0IDAwMDQ6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEx
MgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2Nzk1MF0gcGNpZXBvcnQg
MDAwNDowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yNjgyMTBdIHBjaWVwb3J0IDAwMDU6MDA6MDEuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDE1ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjY4NDk0XSBwY2llcG9ydCAwMDA1OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNjg1NDddIHBjaWVwb3J0IDAw
MDU6MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE2ClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjY4Nzk0XSBwY2llcG9ydCAwMDA1OjAwOjA1LjA6IEFFUjogZW5h
YmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
Njg4NDZdIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE3ClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY5MDk4XSBwY2llcG9ydCAwMDA1
OjAwOjA3LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yNzA0NThdIEZyZWVpbmcgdW51c2VkIGtlcm5lbCBtZW1vcnk6IDU0
NDBLClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDM1MzI5XSBDaGVja2Vk
IFcrWCBtYXBwaW5nczogcGFzc2VkLCBubyBXK1ggcGFnZXMgZm91bmQKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MzUzMzVdIFJ1biAvaW5pdCBhcyBpbml0IHByb2Nlc3MK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MzUzMzZdICAgd2l0aCBhcmd1
bWVudHM6ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDM1MzM3XSAgICAg
L2luaXQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MzUzMzhdICAgd2l0
aCBlbnZpcm9ubWVudDoKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MzUz
MzldICAgICBIT01FPS8KU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MzUz
MzldICAgICBURVJNPWxpbnV4ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NDM1MzQwXSAgICAgQk9PVF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04LWFybTY0ClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA4NDI0XSBwcHNfY29yZTogTGludXhQ
UFMgQVBJIHZlci4gMSByZWdpc3RlcmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTA4NDI1XSBwcHNfY29yZTogU29mdHdhcmUgdmVyLiA1LjMuNiAtIENvcHlyaWdodCAy
MDA1LTIwMDcgUm9kb2xmbyBHaW9tZXR0aSA8Z2lvbWV0dGlAbGludXguaXQ+ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA5NDUxXSBQVFAgY2xvY2sgc3VwcG9ydCByZWdp
c3RlcmVkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE0MTEzXSBTQ1NJ
IHN1YnN5c3RlbSBpbml0aWFsaXplZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjUxNzMzN10gaWdiOiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgRHJpdmVy
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE3MzM4XSBpZ2I6IENvcHly
aWdodCAoYykgMjAwNy0yMDE0IEludGVsIENvcnBvcmF0aW9uLgpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjUxNzg3OF0gQUNQSTogYnVzIHR5cGUgVVNCIHJlZ2lzdGVyZWQK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTc4OTZdIHVzYmNvcmU6IHJl
Z2lzdGVyZWQgbmV3IGludGVyZmFjZSBkcml2ZXIgdXNiZnMKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS41MTc5MDFdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGludGVyZmFj
ZSBkcml2ZXIgaHViClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTE3OTI4
XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBkZXZpY2UgZHJpdmVyIHVzYgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUxODQ5NV0gaWdiIDAwMDM6MDI6MDAuMDogQWRkaW5n
IHRvIGlvbW11IGdyb3VwIDE4ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTIwNDg0XSBudm1lIDAwMGQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE5ClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTIwNzYyXSBudm1lIG52bWUwOiBwY2kg
ZnVuY3Rpb24gMDAwZDowMTowMC4wClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNTIwODE0XSBudm1lIDAwMGQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIwClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTIxMjAyXSBudm1lIG52bWUxOiBw
Y2kgZnVuY3Rpb24gMDAwZDowMjowMC4wClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTIxMjUwXSBudm1lIDAwMGQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIx
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTIxNDgwXSBudm1lIG52bWUy
OiBwY2kgZnVuY3Rpb24gMDAwZDowMzowMC4wClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNTIxNTI1XSBudm1lIDAwMGQ6MDQ6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDIyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTIxNzU1XSBudm1lIG52
bWUzOiBwY2kgZnVuY3Rpb24gMDAwZDowNDowMC4wClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNTI3NTc0XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IEFkZGluZyB0byBpb21t
dSBncm91cCAyMwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyNzg3M10g
bXB0M3NhcyB2ZXJzaW9uIDM1LjEwMC4wMC4wMCBsb2FkZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS41Mjc4NzhdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogeEhDSSBIb3N0
IENvbnRyb2xsZXIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41Mjc4ODRd
IHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogbmV3IFVTQiBidXMgcmVnaXN0ZXJlZCwgYXNzaWduZWQg
YnVzIG51bWJlciAxClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTI3OTAy
XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IFplcm9pbmcgNjRiaXQgYmFzZSByZWdpc3RlcnMsIGV4
cGVjdGluZyBmYXVsdApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyNzk4
NV0gbXB0M3NhcyAwMDBjOjAxOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyNApTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyOTg2Ml0gcmFuZG9tOiBmYXN0IGluaXQg
ZG9uZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUzMTQwNF0gbnZtZSBu
dm1lMDogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjUzMTk0OF0gbnZtZSBudm1lMTogODAvMC8wIGRlZmF1bHQvcmVh
ZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUzMjAx
Ml0gbnZtZSBudm1lMzogODAvMC8wIGRlZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUzMzE3MF0gbnZtZSBudm1lMjogODAvMC8wIGRl
ZmF1bHQvcmVhZC9wb2xsIHF1ZXVlcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjUzNDQzMF0gYXN0IDAwMDQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE0ClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTM0OTI4XSAgbnZtZTBuMToKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MzUzNjNdICBudm1lM24xOiBwMSBw
MiBwMyBwNCBwNSBwNgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjUzNTU2
MF0gIG52bWUxbjE6IHAxIHAyIHAzIHA0ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNjc0MDE0XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IGhjYyBwYXJhbXMgMHgwMTQwNTFj
ZiBoY2kgdmVyc2lvbiAweDEwMCBxdWlya3MgMHgwMDAwMDAxMTAwMDAwNDEwClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0Mjk3XSB1c2IgdXNiMTogTmV3IFVTQiBkZXZp
Y2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlkUHJvZHVjdD0wMDAyLCBiY2REZXZpY2U9IDUuMTAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzQyOTldIHVzYiB1c2IxOiBO
ZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MywgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NDI5OV0gdXNiIHVzYjE6IFBy
b2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNjc0MzAwXSB1c2IgdXNiMTogTWFudWZhY3R1cmVyOiBMaW51eCA1LjEwLjAtOC1h
cm02NCB4aGNpLWhjZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NDMw
MV0gdXNiIHVzYjE6IFNlcmlhbE51bWJlcjogMDAwNDowMzowMC4wClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0NDE3XSBodWIgMS0wOjEuMDogVVNCIGh1YiBmb3VuZApT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3NDQyOF0gaHViIDEtMDoxLjA6
IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42
NzQ1NjBdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogeEhDSSBIb3N0IENvbnRyb2xsZXIKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzQ1NjJdIHhoY2lfaGNkIDAwMDQ6MDM6
MDAuMDogbmV3IFVTQiBidXMgcmVnaXN0ZXJlZCwgYXNzaWduZWQgYnVzIG51bWJlciAyClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc0NTY1XSB4aGNpX2hjZCAwMDA0OjAz
OjAwLjA6IEhvc3Qgc3VwcG9ydHMgVVNCIDMuMCBTdXBlclNwZWVkClNlcCAxMyAxNTo1MDozMiBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjg4MTIzXSB1c2IgdXNiMjogV2UgZG9uJ3Qga25vdyB0aGUg
YWxnb3JpdGhtcyBmb3IgTFBNIGZvciB0aGlzIGhvc3QsIGRpc2FibGluZyBMUE0uClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjg4MTM2XSB1c2IgdXNiMjogTmV3IFVTQiBk
ZXZpY2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlkUHJvZHVjdD0wMDAzLCBiY2REZXZpY2U9IDUu
MTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42ODgxMzddIHVzYiB1c2Iy
OiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MywgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9
MQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY4ODEzOF0gdXNiIHVzYjI6
IFByb2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNjg4MTM4XSB1c2IgdXNiMjogTWFudWZhY3R1cmVyOiBMaW51eCA1LjEwLjAt
OC1hcm02NCB4aGNpLWhjZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY4
ODEzOV0gdXNiIHVzYjI6IFNlcmlhbE51bWJlcjogMDAwNDowMzowMC4wClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjg4MjA1XSBodWIgMi0wOjEuMDogVVNCIGh1YiBmb3Vu
ZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY4ODIyMV0gaHViIDItMDox
LjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS42ODgzMDJdIG1wdDNzYXNfY20wOiA2MyBCSVQgUENJIEJVUyBETUEgQUREUkVTU0lORyBTVVBQ
T1JURUQsIHRvdGFsIG1lbSAoNjUzMDQ4MDQga0IpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNjg4ODcxXSBjaGVja2luZyBnZW5lcmljICgyMDAwMDAwMCAxZDUwMDApIHZz
IGh3ICgyMDAwMDAwMCAyMDAwMDAwKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAxLjY4ODg3M10gZmIwOiBzd2l0Y2hpbmcgdG8gYXN0ZHJtZmIgZnJvbSBFRkkgVkdBClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjg4OTI3XSBDb25zb2xlOiBzd2l0Y2hp
bmcgdG8gY29sb3VyIGR1bW15IGRldmljZSA4MHgyNQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAxLjY4ODk3OV0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gcGxhdGZvcm0gaGFz
IG5vIElPIHNwYWNlLCB0cnlpbmcgTU1JTwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAxLjY4ODk4NV0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gUDJBIGJyaWRnZSBkaXNhYmxl
ZCwgdXNpbmcgZGVmYXVsdCBjb25maWd1cmF0aW9uClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuNjg4OTg2XSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBBU1QgMjUwMCBkZXRl
Y3RlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY4ODk5MF0gYXN0IDAw
MDQ6MDI6MDAuMDogW2RybV0gQW5hbG9nIFZHQSBvbmx5ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjg4OTkxXSBhc3QgMDAwNDowMjowMC4wOiBbZHJtXSBkcmFtIE1DTEs9
ODAwIE1oeiB0eXBlPTEgYnVzX3dpZHRoPTE2ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNjg5MDI2XSBbVFRNXSBab25lICBrZXJuZWw6IEF2YWlsYWJsZSBncmFwaGljcyBt
ZW1vcnk6IDMyNjUyNDAyIEtpQgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAx
LjY4OTAyN10gW1RUTV0gWm9uZSAgIGRtYTMyOiBBdmFpbGFibGUgZ3JhcGhpY3MgbWVtb3J5OiAy
MDk3MTUyIEtpQgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjY4OTAyOF0g
W1RUTV0gSW5pdGlhbGl6aW5nIHBvb2wgYWxsb2NhdG9yClNlcCAxMyAxNTo1MDozMiBkZWJpYW4g
a2VybmVsOiBbICAgIDEuNjg5MDMxXSBbVFRNXSBJbml0aWFsaXppbmcgRE1BIHBvb2wgYWxsb2Nh
dG9yClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjg5MjQ1XSBbZHJtXSBJ
bml0aWFsaXplZCBhc3QgMC4xLjAgMjAxMjAyMjggZm9yIDAwMDQ6MDI6MDAuMCBvbiBtaW5vciAw
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODAyOTEzXSBDb25zb2xlOiBz
d2l0Y2hpbmcgdG8gY29sb3VyIGZyYW1lIGJ1ZmZlciBkZXZpY2UgMjQweDY3ClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODA1NTc1XSBhc3QgMDAwNDowMjowMC4wOiBbZHJt
XSBmYjA6IGFzdGRybWZiIGZyYW1lIGJ1ZmZlciBkZXZpY2UKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44MzM0NzJdIGlnYiAwMDAzOjAyOjAwLjA6IGFkZGVkIFBIQyBvbiBl
dGgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODMzNDc0XSBpZ2IgMDAw
MzowMjowMC4wOiBJbnRlbChSKSBHaWdhYml0IEV0aGVybmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjgzMzQ3NV0gaWdiIDAwMDM6MDI6
MDAuMDogZXRoMDogKFBDSWU6NS4wR2IvczpXaWR0aCB4MikgZDg6NWU6ZDM6MDI6YjE6OTMKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44MzM1NTZdIGlnYiAwMDAzOjAyOjAw
LjA6IGV0aDA6IFBCQSBObzogMTA2MzAwLTAwMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAxLjgzMzU1N10gaWdiIDAwMDM6MDI6MDAuMDogVXNpbmcgTVNJLVggaW50ZXJydXB0
cy4gOCByeCBxdWV1ZShzKSwgOCB0eCBxdWV1ZShzKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAxLjgzMzYxNl0gaWdiIDAwMDM6MDI6MDAuMTogQWRkaW5nIHRvIGlvbW11IGdy
b3VwIDI1ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg5ODcwXSBtcHQz
c2FzX2NtMDogQ3VycmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBw
YWdlIHNpemUgdG8gNGsKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODk4
ODhdIG1wdDNzYXNfY20wOiBNU0ktWCB2ZWN0b3JzIHN1cHBvcnRlZDogOTYKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODk4ODldIAkgbm8gb2YgY29yZXM6IDgwLCBtYXhf
bXNpeF92ZWN0b3JzOiAtMQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjg4
OTg5MV0gbXB0M3Nhc19jbTA6ICAwIDgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODkxMjE4XSBtcHQzc2FzX2NtMDogSGlnaCBJT1BzIHF1ZXVlcyA6IGRpc2FibGVkClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODkxMjIwXSBtcHQzc2FzMC1tc2l4
MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44OTEyMjFdIG1wdDNzYXMwLW1zaXgxOiBQQ0ktTVNJLVggZW5hYmxlZDogSVJR
IDQ2NgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjg5MTIyMV0gbXB0M3Nh
czAtbXNpeDI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY3ClNlcCAxMyAxNTo1MDozMiBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODkxMjIyXSBtcHQzc2FzMC1tc2l4MzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0NjgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjJd
IG1wdDNzYXMwLW1zaXg0OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ2OQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjg5MTIyM10gbXB0M3NhczAtbXNpeDU6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDcwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODkxMjI0XSBtcHQzc2FzMC1tc2l4NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzEKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjRdIG1wdDNzYXMwLW1zaXg3OiBQ
Q0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAxLjg5MTIyNV0gbXB0M3NhczAtbXNpeDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDcz
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuODkxMjI1XSBtcHQzc2FzMC1t
c2l4OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44OTEyMjZdIG1wdDNzYXMwLW1zaXgxMDogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0NzUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjddIG1w
dDNzYXMwLW1zaXgxMTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzYKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjddIG1wdDNzYXMwLW1zaXgxMjogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0NzcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
OTEyMjhdIG1wdDNzYXMwLW1zaXgxMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzgKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjhdIG1wdDNzYXMwLW1zaXgxNDog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44OTEyMjldIG1wdDNzYXMwLW1zaXgxNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
ODAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMjldIG1wdDNzYXMw
LW1zaXgxNjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODEKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44OTEyMzBdIG1wdDNzYXMwLW1zaXgxNzogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA0ODIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzFd
IG1wdDNzYXMwLW1zaXgxODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODMKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzFdIG1wdDNzYXMwLW1zaXgxOTogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA0ODQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44OTEyMzJdIG1wdDNzYXMwLW1zaXgyMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODUKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzJdIG1wdDNzYXMwLW1zaXgy
MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44OTEyMzNdIG1wdDNzYXMwLW1zaXgyMjogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA0ODcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzRdIG1wdDNz
YXMwLW1zaXgyMzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0ODgKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzRdIG1wdDNzYXMwLW1zaXgyNDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0ODkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEy
MzVdIG1wdDNzYXMwLW1zaXgyNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTAKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzVdIG1wdDNzYXMwLW1zaXgyNjogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA0OTEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44OTEyMzZdIG1wdDNzYXMwLW1zaXgyNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTIK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzZdIG1wdDNzYXMwLW1z
aXgyODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44OTEyMzddIG1wdDNzYXMwLW1zaXgyOTogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA0OTQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzddIG1w
dDNzYXMwLW1zaXgzMDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTUKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzhdIG1wdDNzYXMwLW1zaXgzMTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA0OTYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
OTEyMzldIG1wdDNzYXMwLW1zaXgzMjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTcKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyMzldIG1wdDNzYXMwLW1zaXgzMzog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0OTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44OTEyNDBdIG1wdDNzYXMwLW1zaXgzNDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0
OTkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDBdIG1wdDNzYXMw
LW1zaXgzNTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44OTEyNDFdIG1wdDNzYXMwLW1zaXgzNjogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDFd
IG1wdDNzYXMwLW1zaXgzNzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDIKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDJdIG1wdDNzYXMwLW1zaXgzODogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MDMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44OTEyNDNdIG1wdDNzYXMwLW1zaXgzOTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDQKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDRdIG1wdDNzYXMwLW1zaXg0
MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44OTEyNDRdIG1wdDNzYXMwLW1zaXg0MTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MDYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDVdIG1wdDNz
YXMwLW1zaXg0MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDcKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDVdIG1wdDNzYXMwLW1zaXg0MzogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MDgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEy
NDZdIG1wdDNzYXMwLW1zaXg0NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MDkKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDZdIG1wdDNzYXMwLW1zaXg0NTogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44OTEyNDddIG1wdDNzYXMwLW1zaXg0NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTEK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDddIG1wdDNzYXMwLW1z
aXg0NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44OTEyNDhdIG1wdDNzYXMwLW1zaXg0ODogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MTMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDldIG1w
dDNzYXMwLW1zaXg0OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTQKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNDldIG1wdDNzYXMwLW1zaXg1MDogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MTUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
OTEyNTBdIG1wdDNzYXMwLW1zaXg1MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTYKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTBdIG1wdDNzYXMwLW1zaXg1Mjog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44OTEyNTFdIG1wdDNzYXMwLW1zaXg1MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MTgKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTJdIG1wdDNzYXMw
LW1zaXg1NDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MTkKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44OTEyNTJdIG1wdDNzYXMwLW1zaXg1NTogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MjAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTNd
IG1wdDNzYXMwLW1zaXg1NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjEKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTNdIG1wdDNzYXMwLW1zaXg1NzogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1MjIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44OTEyNTRdIG1wdDNzYXMwLW1zaXg1ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjMKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTVdIG1wdDNzYXMwLW1zaXg1
OTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44OTEyNTVdIG1wdDNzYXMwLW1zaXg2MDogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1MjUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTZdIG1wdDNz
YXMwLW1zaXg2MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjYKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTddIG1wdDNzYXMwLW1zaXg2MjogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA1MjcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEy
NTddIG1wdDNzYXMwLW1zaXg2MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MjgKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNThdIG1wdDNzYXMwLW1zaXg2NDogUENJ
LU1TSS1YIGVuYWJsZWQ6IElSUSA1MjkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS44OTEyNThdIG1wdDNzYXMwLW1zaXg2NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzAK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNTldIG1wdDNzYXMwLW1z
aXg2NjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44OTEyNjBdIG1wdDNzYXMwLW1zaXg2NzogUENJLU1TSS1YIGVuYWJsZWQ6
IElSUSA1MzIKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjBdIG1w
dDNzYXMwLW1zaXg2ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzMKU2VwIDEzIDE1OjUwOjMy
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjFdIG1wdDNzYXMwLW1zaXg2OTogUENJLU1TSS1Y
IGVuYWJsZWQ6IElSUSA1MzQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
OTEyNjFdIG1wdDNzYXMwLW1zaXg3MDogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzUKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjJdIG1wdDNzYXMwLW1zaXg3MTog
UENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzYKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44OTEyNjNdIG1wdDNzYXMwLW1zaXg3MjogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1
MzcKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjNdIG1wdDNzYXMw
LW1zaXg3MzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1MzgKU2VwIDEzIDE1OjUwOjMyIGRlYmlh
biBrZXJuZWw6IFsgICAgMS44OTEyNjRdIG1wdDNzYXMwLW1zaXg3NDogUENJLU1TSS1YIGVuYWJs
ZWQ6IElSUSA1MzkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjRd
IG1wdDNzYXMwLW1zaXg3NTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDAKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjVdIG1wdDNzYXMwLW1zaXg3NjogUENJLU1T
SS1YIGVuYWJsZWQ6IElSUSA1NDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS44OTEyNjZdIG1wdDNzYXMwLW1zaXg3NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDIKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjZdIG1wdDNzYXMwLW1zaXg3
ODogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA1NDMKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS44OTEyNjddIG1wdDNzYXMwLW1zaXg3OTogUENJLU1TSS1YIGVuYWJsZWQ6IElS
USA1NDQKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44OTEyNjhdIG1wdDNz
YXNfY20wOiBpb21lbSgweDAwMDAwMDAwNDAxNDAwMDApLCBtYXBwZWQoMHgoX19fX3B0cnZhbF9f
X18pKSwgc2l6ZSg2NTUzNikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44
OTEyNjldIG1wdDNzYXNfY20wOiBpb3BvcnQoMHgwMDAwMDAwMDAwMDAwMDAwKSwgc2l6ZSgwKQpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjkwMTkxNF0gaWdiIDAwMDM6MDI6
MDAuMTogYWRkZWQgUEhDIG9uIGV0aDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS45MDE5MTVdIGlnYiAwMDAzOjAyOjAwLjE6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQg
TmV0d29yayBDb25uZWN0aW9uClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEu
OTAxOTE2XSBpZ2IgMDAwMzowMjowMC4xOiBldGgxOiAoUENJZTo1LjBHYi9zOldpZHRoIHgyKSBk
ODo1ZTpkMzowMjpiMTo5NApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAxLjkw
MTk5OF0gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogUEJBIE5vOiAxMDYzMDAtMDAwClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTAxOTk5XSBpZ2IgMDAwMzowMjowMC4xOiBV
c2luZyBNU0ktWCBpbnRlcnJ1cHRzLiA4IHJ4IHF1ZXVlKHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAx
MyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTAyOTkxXSBpZ2IgMDAwMzowMjowMC4w
IGVubzE6IHJlbmFtZWQgZnJvbSBldGgwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuOTI0NTA4XSBpZ2IgMDAwMzowMjowMC4xIGVubzI6IHJlbmFtZWQgZnJvbSBldGgxClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDEuOTQ5MzczXSBtcHQzc2FzX2NtMDog
Q3VycmVudEhvc3RQYWdlU2l6ZSBpcyAwOiBTZXR0aW5nIGRlZmF1bHQgaG9zdCBwYWdlIHNpemUg
dG8gNGsKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45NDkzNzRdIG1wdDNz
YXNfY20wOiBzZW5kaW5nIGRpYWcgcmVzZXQgISEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi4wMTIzNzJdIHVzYiAxLTE6IG5ldyBsb3ctc3BlZWQgVVNCIGRldmljZSBudW1i
ZXIgMiB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAy
LjIwNDY2Nl0gdXNiIDEtMTogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTQxM2MsIGlk
UHJvZHVjdD0zMDFhLCBiY2REZXZpY2U9IDEuMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi4yMDQ2NjddIHVzYiAxLTE6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0x
LCBQcm9kdWN0PTIsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDIuMjA0NjY4XSB1c2IgMS0xOiBQcm9kdWN0OiBEZWxsIE1TMTE2IFVTQiBPcHRpY2Fs
IE1vdXNlClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjA0NjY5XSB1c2Ig
MS0xOiBNYW51ZmFjdHVyZXI6IFBpeEFydApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAyLjIxMzc0MV0gaGlkOiByYXcgSElEIGV2ZW50cyBkcml2ZXIgKEMpIEppcmkgS29zaW5h
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjI1NzMxXSB1c2Jjb3JlOiBy
ZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVzYmhpZApTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAyLjIyNTczMl0gdXNiaGlkOiBVU0IgSElEIGNvcmUgZHJpdmVyClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuMjI3MDM4XSBpbnB1dDogUGl4QXJ0
IERlbGwgTVMxMTYgVVNCIE9wdGljYWwgTW91c2UgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0
OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEvMS0xLzEtMToxLjAvMDAwMzo0MTNDOjMwMUEuMDAw
MS9pbnB1dC9pbnB1dDEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yMjcw
ODddIGhpZC1nZW5lcmljIDAwMDM6NDEzQzozMDFBLjAwMDE6IGlucHV0LGhpZHJhdzA6IFVTQiBI
SUQgdjEuMTEgTW91c2UgW1BpeEFydCBEZWxsIE1TMTE2IFVTQiBPcHRpY2FsIE1vdXNlXSBvbiB1
c2ItMDAwNDowMzowMC4wLTEvaW5wdXQwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBb
ICAgIDIuMzEzMDI0XSB1c2IgMi0zOiBuZXcgU3VwZXJTcGVlZCBHZW4gMSBVU0IgZGV2aWNlIG51
bWJlciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDIuMzQxNTkyXSB1c2IgMi0zOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDVlMywg
aWRQcm9kdWN0PTA2MjAsIGJjZERldmljZT05My4xMApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAyLjM0MTU5M10gdXNiIDItMzogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZy
PTEsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi4zNDE1OTRdIHVzYiAyLTM6IFByb2R1Y3Q6IFVTQjMuMSBIdWIKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4zNDE1OTVdIHVzYiAyLTM6IE1hbnVmYWN0dXJl
cjogR2VuZXN5c0xvZ2ljClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuMzQ0
NjA5XSBodWIgMi0zOjEuMDogVVNCIGh1YiBmb3VuZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtl
cm5lbDogWyAgICAyLjM0NTIxN10gaHViIDItMzoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi40NzIzNTJdIHVzYiAxLTI6IG5ldyBsb3ct
c3BlZWQgVVNCIGRldmljZSBudW1iZXIgMyB1c2luZyB4aGNpX2hjZApTZXAgMTMgMTU6NTA6MzIg
ZGViaWFuIGtlcm5lbDogWyAgICAyLjY4MzY1OV0gdXNiIDEtMjogTmV3IFVTQiBkZXZpY2UgZm91
bmQsIGlkVmVuZG9yPTA0ZDksIGlkUHJvZHVjdD0xMjAzLCBiY2REZXZpY2U9IDIuODAKU2VwIDEz
IDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi42ODM2NjBdIHVzYiAxLTI6IE5ldyBVU0Ig
ZGV2aWNlIHN0cmluZ3M6IE1mcj0wLCBQcm9kdWN0PTAsIFNlcmlhbE51bWJlcj0wClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzIwNzcyXSBpbnB1dDogSElEIDA0ZDk6MTIw
MyBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8x
LTIvMS0yOjEuMC8wMDAzOjA0RDk6MTIwMy4wMDAyL2lucHV0L2lucHV0MgpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAyLjc4NDQxMV0gaGlkLWdlbmVyaWMgMDAwMzowNEQ5OjEy
MDMuMDAwMjogaW5wdXQsaGlkcmF3MTogVVNCIEhJRCB2MS4xMSBLZXlib2FyZCBbSElEIDA0ZDk6
MTIwM10gb24gdXNiLTAwMDQ6MDM6MDAuMC0yL2lucHV0MApTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAyLjgyNjc0MV0gaW5wdXQ6IEhJRCAwNGQ5OjEyMDMgU3lzdGVtIENvbnRy
b2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDowMzowMC4wL3VzYjEv
MS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDMKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi44ODgzNjhdIGlucHV0OiBISUQgMDRkOToxMjAzIENv
bnN1bWVyIENvbnRyb2wgYXMgL2RldmljZXMvcGNpMDAwNDowMC8wMDA0OjAwOjAzLjAvMDAwNDow
MzowMC4wL3VzYjEvMS0yLzEtMjoxLjEvMDAwMzowNEQ5OjEyMDMuMDAwMy9pbnB1dC9pbnB1dDQK
U2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMi44ODgzOTJdIGhpZC1nZW5lcmlj
IDAwMDM6MDREOToxMjAzLjAwMDM6IGlucHV0LGhpZHJhdzI6IFVTQiBISUQgdjEuMTEgRGV2aWNl
IFtISUQgMDRkOToxMjAzXSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQxClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDIuOTg3ODYyXSBtcHQzc2FzX2NtMDogZGlhZyByZXNl
dDogU1VDQ0VTUwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjAyMjg3N10g
bXB0M3Nhc19jbTA6IHNjYXR0ZXIgZ2F0aGVyOiBzZ2VfaW5fbWFpbl9tc2coMSksIHNnZV9wZXJf
Y2hhaW4oNyksIHNnZV9wZXJfaW8oMTI4KSwgY2hhaW5zX3Blcl9pbygxOSkKU2VwIDEzIDE1OjUw
OjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wMjM5NDhdIG1wdDNzYXNfY20wOiByZXF1ZXN0IHBv
b2woMHgoX19fX3B0cnZhbF9fX18pKSAtIGRtYSgweGZmNTAwMDAwKTogZGVwdGgoNDIyNCksIGZy
YW1lX3NpemUoMTI4KSwgcG9vbF9zaXplKDUyOCBrQikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy4wMjQzNTddIHVzYiAxLTM6IG5ldyBoaWdoLXNwZWVkIFVTQiBkZXZpY2Ug
bnVtYmVyIDQgdXNpbmcgeGhjaV9oY2QKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMy4wNDQzNDVdIG1wdDNzYXNfY20wOiBzZW5zZSBwb29sKDB4KF9fX19wdHJ2YWxfX19fKSkt
IGRtYSgweGZlYjAwMDAwKTogZGVwdGgoMzk2MyksZWxlbWVudF9zaXplKDk2KSwgcG9vbF9zaXpl
KDM3MSBrQikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wNDQ0NDhdIG1w
dDNzYXNfY20wOiBjb25maWcgcGFnZSgweChfX19fcHRydmFsX19fXykpIC0gZG1hKDB4ZmU5ZjYw
MDApOiBzaXplKDUxMikKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wNDQ0
NDldIG1wdDNzYXNfY20wOiBBbGxvY2F0ZWQgcGh5c2ljYWwgbWVtb3J5OiBzaXplKDcyNzgga0Ip
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDQ0NDUwXSBtcHQzc2FzX2Nt
MDogQ3VycmVudCBDb250cm9sbGVyIFF1ZXVlIERlcHRoKDM5NjApLE1heCBDb250cm9sbGVyIFF1
ZXVlIERlcHRoKDQwOTYpClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDQ0
NDUwXSBtcHQzc2FzX2NtMDogU2NhdHRlciBHYXRoZXIgRWxlbWVudHMgcGVyIElPKDEyOCkKU2Vw
IDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNzc4MjddIHVzYiAxLTM6IE5ldyBV
U0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNWUzLCBpZFByb2R1Y3Q9MDYxMCwgYmNkRGV2aWNl
PTkzLjEwClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTc3ODI4XSB1c2Ig
MS0zOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1i
ZXI9MApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjE3NzgyOV0gdXNiIDEt
MzogUHJvZHVjdDogVVNCMi4wIEh1YgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjE3NzgzMF0gdXNiIDEtMzogTWFudWZhY3R1cmVyOiBHZW5lc3lzTG9naWMKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xODE4NDhdIGh1YiAxLTM6MS4wOiBVU0IgaHVi
IGZvdW5kClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTgyNDU0XSBodWIg
MS0zOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAzLjI0NTQ4N10gbXB0M3Nhc19jbTA6IF9iYXNlX2Rpc3BsYXlfZndwa2dfdmVyc2lvbjog
Y29tcGxldGUKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yNDU4MTRdIG1w
dDNzYXNfY20wOiBMU0lTQVMzMDA4OiBGV1ZlcnNpb24oMTYuMDAuMTAuMDApLCBDaGlwUmV2aXNp
b24oMHgwMiksIEJpb3NWZXJzaW9uKDE1LjAwLjAxLjAwKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAzLjI0NTgxNF0gbXB0M3Nhc19jbTA6IFByb3RvY29sPShJbml0aWF0b3Ip
LCBDYXBhYmlsaXRpZXM9KFJhaWQsVExSLEVFRFAsU25hcHNob3QgQnVmZmVyLERpYWcgVHJhY2Ug
QnVmZmVyLFRhc2sgU2V0IEZ1bGwsTkNRKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAzLjI0NTg2NF0gc2NzaSBob3N0MDogRnVzaW9uIE1QVCBTQVMgSG9zdApTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjI0NjQzMF0gbXB0M3Nhc19jbTA6IHNlbmRpbmcg
cG9ydCBlbmFibGUgISEKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4yNzY5
MjNdIFBNOiBJbWFnZSBub3QgZm91bmQgKGNvZGUgLTIyKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFu
IGtlcm5lbDogWyAgICAzLjMzNDI3OV0gRVhUNC1mcyAobnZtZTNuMXAyKTogbW91bnRlZCBmaWxl
c3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTU6NTA6
MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjM2NjYyNl0gTm90IGFjdGl2YXRpbmcgTWFuZGF0b3J5
IEFjY2VzcyBDb250cm9sIGFzIC9zYmluL3RvbW95by1pbml0IGRvZXMgbm90IGV4aXN0LgpTZXAg
MTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjQ5NTI0OV0gcmFuZG9tOiBsdm1jb25m
aWc6IHVuaW5pdGlhbGl6ZWQgdXJhbmRvbSByZWFkICg0IGJ5dGVzIHJlYWQpClNlcCAxMyAxNTo1
MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTYzNjEwXSByYW5kb206IHN5c3RlbWQ6IHVuaW5p
dGlhbGl6ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjU2NDIwNF0gcmFuZG9tOiBzeXN0ZW1kOiB1bmluaXRpYWxpemVk
IHVyYW5kb20gcmVhZCAoMTYgYnl0ZXMgcmVhZCkKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy41ODE0MjldIGZ1c2U6IGluaXQgKEFQSSB2ZXJzaW9uIDcuMzIpClNlcCAxMyAx
NTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuNTgzMDQ4XSBFWFQ0LWZzIChudm1lM24xcDIp
OiByZS1tb3VudGVkLiBPcHRzOiBlcnJvcnM9cmVtb3VudC1ybwpTZXAgMTMgMTU6NTA6MzIgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjY2NjczM10gYWNwaS10YWQgQUNQSTAwMEU6MDA6IFVuc3VwcG9y
dGVkIGNhcGFiaWxpdGllcwpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjcw
NDMzMV0geGdlbmUtc2xpbXByby1od21vbiBBUE1DMEQyOTowMDogQVBNIFgtR2VuZSBTb0MgSFcg
bW9uaXRvciBkcml2ZXIgcmVnaXN0ZXJlZApTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAzLjcwOTE1OF0gSVBNSSBtZXNzYWdlIGhhbmRsZXI6IHZlcnNpb24gMzkuMgpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjcxMjA4MV0gaXBtaSBkZXZpY2UgaW50ZXJm
YWNlClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzE4NTg2XSBpcG1pX3Np
OiBJUE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2Vy
bmVsOiBbICAgIDMuNzE4ODc3XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBhbnkgU3lzdGVtIElu
dGVyZmFjZShzKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjc5NjU0Nl0g
RVhUNC1mcyAobnZtZTNuMXA1KTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRh
IG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAz
Ljc5NzkyM10gRVhUNC1mcyAobnZtZTNuMXAzKTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3Jk
ZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAzLjgxMDgxMF0gaXBtaV9zc2lmOiBJUE1JIFNTSUYgSW50ZXJmYWNlIGRyaXZlcgpT
ZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjgxNTAwNV0gaXBtaV9zc2lmIGky
Yy1BTVBDMDAwNDowMDogaXBtaV9zc2lmOiBUcnlpbmcgQUNQSS1zcGVjaWZpZWQgU1NJRiBpbnRl
cmZhY2UgYXQgaTJjIGFkZHJlc3MgMHgxMCwgYWRhcHRlciBTeW5vcHN5cyBEZXNpZ25XYXJlIEky
QyBhZGFwdGVyLCBzbGF2ZSBhZGRyZXNzIDB4MjAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44MTk5MDNdIEVYVDQtZnMgKG52bWUzbjFwNik6IG1vdW50ZWQgZmlsZXN5c3Rl
bSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDE1OjUwOjMyIGRl
YmlhbiBrZXJuZWw6IFsgICAgMy44NDQzNjhdIEFkZGluZyAxMDAwNDQ0ayBzd2FwIG9uIC9kZXYv
bnZtZTNuMXA0LiAgUHJpb3JpdHk6LTIgZXh0ZW50czoxIGFjcm9zczoxMDAwNDQ0ayBTU0ZTClNl
cCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODQ0NTg4XSBzYnNhLWd3ZHQgc2Jz
YS1nd2R0LjA6IEluaXRpYWxpemVkIHdpdGggMTBzIHRpbWVvdXQgQCAyNTAwMDAwMCBIeiwgYWN0
aW9uPTAuClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODQ5MjExXSByYW5k
b206IGNybmcgaW5pdCBkb25lClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMu
ODQ5MjE0XSByYW5kb206IDcgdXJhbmRvbSB3YXJuaW5nKHMpIG1pc3NlZCBkdWUgdG8gcmF0ZWxp
bWl0aW5nClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODUzNTA5XSBhdWRp
dDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1Mjg0MzIuMTMyOjIpOiBhcHBhcm1vcj0iU1RBVFVTIiBv
cGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVv
ZmZpY2Utc2VuZGRvYyIgcGlkPTEwOTYgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6
NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjg1MzUxMl0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRp
dCgxNjMxNTI4NDMyLjEzMjozKTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxl
X2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9ImxzYl9yZWxlYXNlIiBwaWQ9MTA5MCBj
b21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAg
IDMuODUzNTE1XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1Mjg0MzIuMTM2OjQpOiBhcHBh
cm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5l
ZCIgbmFtZT0ibnZpZGlhX21vZHByb2JlIiBwaWQ9MTA5OCBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODUzNTE3XSBhdWRpdDogdHlw
ZT0xNDAwIGF1ZGl0KDE2MzE1Mjg0MzIuMTM2OjUpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRp
b249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibnZpZGlhX21vZHBy
b2JlLy9rbW9kIiBwaWQ9MTA5OCBjb21tPSJhcHBhcm1vcl9wYXJzZXIiClNlcCAxMyAxNTo1MDoz
MiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODUzNTE5XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2
MzE1Mjg0MzIuMTM2OjYpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9h
ZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlicmVvZmZpY2UteHBkZmltcG9ydCIgcGlk
PTEwOTIgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5l
bDogWyAgICAzLjg1MzU0MV0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI4NDMyLjEzNjo3
KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVu
Y29uZmluZWQiIG5hbWU9ImxpYnZpcnRkIiBwaWQ9MTA5NCBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODUzNTQzXSBhdWRpdDogdHlw
ZT0xNDAwIGF1ZGl0KDE2MzE1Mjg0MzIuMTM2OjgpOiBhcHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRp
b249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25maW5lZCIgbmFtZT0ibGlidmlydGQvL3Fl
bXVfYnJpZGdlX2hlbHBlciIgcGlkPTEwOTQgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMg
MTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjg1NTYxMF0gYXVkaXQ6IHR5cGU9MTQwMCBh
dWRpdCgxNjMxNTI4NDMyLjEzNjo5KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9m
aWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Ii91c3IvYmluL21hbiIgcGlkPTEw
OTUgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDog
WyAgICAzLjg1NTYxMl0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTI4NDMyLjEzNjoxMCk6
IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNv
bmZpbmVkIiBuYW1lPSJtYW5fZmlsdGVyIiBwaWQ9MTA5NSBjb21tPSJhcHBhcm1vcl9wYXJzZXIi
ClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVsOiBbICAgIDMuODY3MzU0XSBpcG1pX3NpOiBJ
UE1JIFN5c3RlbSBJbnRlcmZhY2UgZHJpdmVyClNlcCAxMyAxNTo1MDozMiBkZWJpYW4ga2VybmVs
OiBbICAgIDMuODY3ODQ4XSBpcG1pX3NpOiBVbmFibGUgdG8gZmluZCBhbnkgU3lzdGVtIEludGVy
ZmFjZShzKQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjg2OTMxOV0gY3J5
cHRkOiBtYXhfY3B1X3FsZW4gc2V0IHRvIDEwMDAKU2VwIDEzIDE1OjUwOjMyIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMy44ODQwMThdIHBzdG9yZTogVXNpbmcgY3Jhc2ggZHVtcCBjb21wcmVzc2lvbjog
ZGVmbGF0ZQpTZXAgMTMgMTU6NTA6MzIgZGViaWFuIGtlcm5lbDogWyAgICAzLjg4NDAyNV0gcHN0
b3JlOiBSZWdpc3RlcmVkIGVmaSBhcyBwZXJzaXN0ZW50IHN0b3JlIGJhY2tlbmQKU2VwIDEzIDE1
OjUwOjMyIGRlYmlhbiBrZXJuZWw6IFsgICAgNC4wODEwMTddIGlwbWlfc3NpZiBpMmMtQU1QQzAw
MDQ6MDA6IElQTUkgbWVzc2FnZSBoYW5kbGVyOiBGb3VuZCBuZXcgQk1DIChtYW5faWQ6IDB4MDAz
YzBhLCBwcm9kX2lkOiAweDAyMTIsIGRldl9pZDogMHgyMCkKU2VwIDEzIDE1OjUwOjM0IGRlYmlh
biBrZXJuZWw6IFsgICAgNS43OTgyMTldIG1wdDNzYXNfY20wOiBob3N0X2FkZDogaGFuZGxlKDB4
MDAwMSksIHNhc19hZGRyKDB4NTAwMTQ4NTAwMDc3OWUzMCksIHBoeXMoOCkKU2VwIDEzIDE1OjUw
OjM2IGRlYmlhbiBrZXJuZWw6IFsgICAgNy43ODE5MDhdIGlnYiAwMDAzOjAyOjAwLjEgZW5vMjog
aWdiOiBlbm8yIE5JQyBMaW5rIGlzIFVwIDEwMDAgTWJwcyBGdWxsIER1cGxleCwgRmxvdyBDb250
cm9sOiBSWC9UWApTZXAgMTMgMTU6NTA6MzYgZGViaWFuIGtlcm5lbDogWyAgICA3Ljc4MjA4MF0g
SVB2NjogQUREUkNPTkYoTkVUREVWX0NIQU5HRSk6IGVubzI6IGxpbmsgYmVjb21lcyByZWFkeQpT
ZXAgMTMgMTU6NTA6MzkgZGViaWFuIGtlcm5lbDogWyAgIDEwLjk0MDUzNV0gbXB0M3Nhc19jbTA6
IHBvcnQgZW5hYmxlOiBTVUNDRVNTClNlcCAxMyAxNzoxNDo1NCBkZWJpYW4ga2VybmVsOiBbIDUw
NjYuNzkxNjMyXSBrYXVkaXRkX3ByaW50a19za2I6IDcgY2FsbGJhY2tzIHN1cHByZXNzZWQKU2Vw
IDEzIDE3OjE0OjU0IGRlYmlhbiBrZXJuZWw6IFsgNTA2Ni43OTE2MzRdIGF1ZGl0OiB0eXBlPTE0
MDAgYXVkaXQoMTYzMTUzMzQ5NC4yNzA6MTgpOiBhcHBhcm1vcj0iQUxMT1dFRCIgb3BlcmF0aW9u
PSJvcGVuIiBwcm9maWxlPSJsaWJyZW9mZmljZS1zb2ZmaWNlIiBuYW1lPTJGNjg2RjZENjUyRjYz
Njk2RDJGNkU3NjZENjUyRjU1NzM2NTcyNzMyRjQ0NjU2NjYxNzU2Qzc0MkY0MTcwNzA0NDYxNzQ2
MTJGNTI2RjYxNkQ2OTZFNjcyRjRENjk2MzcyNkY3MzZGNjY3NDJGNTc2OTZFNjQ2Rjc3NzMyRjUz
NjU2RTY0NTQ2RjJGNEQ2MTY5NkMyMDUyNjU2MzY5NzA2OTY1NkU3NDJFNEQ0MTUwNDk0RDYxNjk2
QyBwaWQ9MjU2OCBjb21tPSJzb2ZmaWNlLmJpbiIgcmVxdWVzdGVkX21hc2s9IndyIiBkZW5pZWRf
bWFzaz0id3IiIGZzdWlkPTEwMDAgb3VpZD0wClNlcCAxMyAxNzoxNDo1NCBkZWJpYW4ga2VybmVs
OiBbIDUwNjYuNzkxNjQxXSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MzM0OTQuMjcwOjE5
KTogYXBwYXJtb3I9IkFMTE9XRUQiIG9wZXJhdGlvbj0iZmlsZV9sb2NrIiBwcm9maWxlPSJsaWJy
ZW9mZmljZS1zb2ZmaWNlIiBuYW1lPTJGNjg2RjZENjUyRjYzNjk2RDJGNkU3NjZENjUyRjU1NzM2
NTcyNzMyRjQ0NjU2NjYxNzU2Qzc0MkY0MTcwNzA0NDYxNzQ2MTJGNTI2RjYxNkQ2OTZFNjcyRjRE
Njk2MzcyNkY3MzZGNjY3NDJGNTc2OTZFNjQ2Rjc3NzMyRjUzNjU2RTY0NTQ2RjJGNEQ2MTY5NkMy
MDUyNjU2MzY5NzA2OTY1NkU3NDJFNEQ0MTUwNDk0RDYxNjk2QyBwaWQ9MjU2OCBjb21tPSJzb2Zm
aWNlLmJpbiIgcmVxdWVzdGVkX21hc2s9IndrIiBkZW5pZWRfbWFzaz0id2siIGZzdWlkPTEwMDAg
b3VpZD0wClNlcCAxMyAxNzoxNDo1NCBkZWJpYW4ga2VybmVsOiBbIDUwNjYuOTY1MzM5XSBhdWRp
dDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MzM0OTQuNDQ2OjIwKTogYXBwYXJtb3I9IkFMTE9XRUQi
IG9wZXJhdGlvbj0ib3BlbiIgcHJvZmlsZT0ibGlicmVvZmZpY2Utc29mZmljZSIgbmFtZT0yRjY4
NkY2RDY1MkY2MzY5NkQyRjZFNzY2RDY1MkY1NTczNjU3MjczMkY0NDY1NjY2MTc1NkM3NDJGNDE3
MDcwNDQ2MTc0NjEyRjUyNkY2MTZENjk2RTY3MkY0RDY5NjM3MjZGNzM2RjY2NzQyRjU3Njk2RTY0
NkY3NzczMkY1MzY1NkU2NDU0NkYyRjRENjE2OTZDMjA1MjY1NjM2OTcwNjk2NTZFNzQyRTRENDE1
MDQ5NEQ2MTY5NkMgcGlkPTI1NjggY29tbT0ic29mZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJy
IiBkZW5pZWRfbWFzaz0iciIgZnN1aWQ9MTAwMCBvdWlkPTAKU2VwIDEzIDE3OjE0OjU0IGRlYmlh
biBrZXJuZWw6IFsgNTA2Ny4xNDA2NjFdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUzMzQ5
NC42MTg6MjEpOiBhcHBhcm1vcj0iQUxMT1dFRCIgb3BlcmF0aW9uPSJvcGVuIiBwcm9maWxlPSJs
aWJyZW9mZmljZS1zb2ZmaWNlIiBuYW1lPTJGNjg2RjZENjUyRjYzNjk2RDJGNkU3NjZENjUyRjU1
NzM2NTcyNzMyRjQ0NjU2NjYxNzU2Qzc0MkY0MTcwNzA0NDYxNzQ2MTJGNTI2RjYxNkQ2OTZFNjcy
RjRENjk2MzcyNkY3MzZGNjY3NDJGNTc2OTZFNjQ2Rjc3NzMyRjUzNjU2RTY0NTQ2RjJGMkU3RTZD
NkY2MzZCMkU0RDYxNjk2QzIwNTI2NTYzNjk3MDY5NjU2RTc0MkU0RDQxNTA0OTRENjE2OTZDMjMg
cGlkPTI1NjggY29tbT0ic29mZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJ3cmMiIGRlbmllZF9t
YXNrPSJ3cmMiIGZzdWlkPTEwMDAgb3VpZD0wClNlcCAxMyAxNzoxNTowMiBkZWJpYW4ga2VybmVs
OiBbIDUwNzUuMTYyNjk2XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MzM1MDIuNjQyOjIy
KTogYXBwYXJtb3I9IkFMTE9XRUQiIG9wZXJhdGlvbj0ib3BlbiIgcHJvZmlsZT0ibGlicmVvZmZp
Y2Utc29mZmljZSIgbmFtZT0yRjY4NkY2RDY1MkY2MzY5NkQyRjZFNzY2RDY1MkY1NTczNjU3Mjcz
MkY0NDY1NjY2MTc1NkM3NDJGNDE3MDcwNDQ2MTc0NjEyRjUyNkY2MTZENjk2RTY3MkY0RDY5NjM3
MjZGNzM2RjY2NzQyRjU3Njk2RTY0NkY3NzczMkY1MzY1NkU2NDU0NkYyRjJFN0U2QzZGNjM2QjJF
NEQ2MTY5NkMyMDUyNjU2MzY5NzA2OTY1NkU3NDJFNEQ0MTUwNDk0RDYxNjk2QzIzIHBpZD0yNTY4
IGNvbW09InNvZmZpY2UuYmluIiByZXF1ZXN0ZWRfbWFzaz0iciIgZGVuaWVkX21hc2s9InIiIGZz
dWlkPTEwMDAgb3VpZD0wClNlcCAxMyAxNzoxNTowMiBkZWJpYW4ga2VybmVsOiBbIDUwNzUuMTYz
MDA0XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1MzM1MDIuNjQyOjIzKTogYXBwYXJtb3I9
IkFMTE9XRUQiIG9wZXJhdGlvbj0idW5saW5rIiBwcm9maWxlPSJsaWJyZW9mZmljZS1zb2ZmaWNl
IiBuYW1lPTJGNjg2RjZENjUyRjYzNjk2RDJGNkU3NjZENjUyRjU1NzM2NTcyNzMyRjQ0NjU2NjYx
NzU2Qzc0MkY0MTcwNzA0NDYxNzQ2MTJGNTI2RjYxNkQ2OTZFNjcyRjRENjk2MzcyNkY3MzZGNjY3
NDJGNTc2OTZFNjQ2Rjc3NzMyRjUzNjU2RTY0NTQ2RjJGMkU3RTZDNkY2MzZCMkU0RDYxNjk2QzIw
NTI2NTYzNjk3MDY5NjU2RTc0MkU0RDQxNTA0OTRENjE2OTZDMjMgcGlkPTI1NjggY29tbT0ic29m
ZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJkIiBkZW5pZWRfbWFzaz0iZCIgZnN1aWQ9MTAwMCBv
dWlkPTAKU2VwIDEzIDE3OjE2OjUwIGRlYmlhbiBrZXJuZWw6IFsgNTE4My4zMDUzOTNdIGF1ZGl0
OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUzMzYxMC43ODY6MjQpOiBhcHBhcm1vcj0iQUxMT1dFRCIg
b3BlcmF0aW9uPSJvcGVuIiBwcm9maWxlPSJsaWJyZW9mZmljZS1zb2ZmaWNlIiBuYW1lPSIvaG9t
ZS9jaW0vbnZtZS8kUmVjeWNsZS5CaW4vUy0xLTUtMjEtMTE3NTI1NTI1MS0xOTM4MzA4NDg0LTI2
MDQzMjMzMzMtNTAwL2Rlc2t0b3AuaW5pIiBwaWQ9MjYzOSBjb21tPSJzb2ZmaWNlLmJpbiIgcmVx
dWVzdGVkX21hc2s9IndyIiBkZW5pZWRfbWFzaz0id3IiIGZzdWlkPTEwMDAgb3VpZD0wClNlcCAx
MyAxNzoxNjo1MCBkZWJpYW4ga2VybmVsOiBbIDUxODMuMzA1NDE5XSBhdWRpdDogdHlwZT0xNDAw
IGF1ZGl0KDE2MzE1MzM2MTAuNzg2OjI1KTogYXBwYXJtb3I9IkFMTE9XRUQiIG9wZXJhdGlvbj0i
ZmlsZV9sb2NrIiBwcm9maWxlPSJsaWJyZW9mZmljZS1zb2ZmaWNlIiBuYW1lPSIvaG9tZS9jaW0v
bnZtZS8kUmVjeWNsZS5CaW4vUy0xLTUtMjEtMTE3NTI1NTI1MS0xOTM4MzA4NDg0LTI2MDQzMjMz
MzMtNTAwL2Rlc2t0b3AuaW5pIiBwaWQ9MjYzOSBjb21tPSJzb2ZmaWNlLmJpbiIgcmVxdWVzdGVk
X21hc2s9IndrIiBkZW5pZWRfbWFzaz0id2siIGZzdWlkPTEwMDAgb3VpZD0wClNlcCAxMyAxNzox
Njo1MCBkZWJpYW4ga2VybmVsOiBbIDUxODMuMzQxMzY0XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0
KDE2MzE1MzM2MTAuODIyOjI2KTogYXBwYXJtb3I9IkFMTE9XRUQiIG9wZXJhdGlvbj0ib3BlbiIg
cHJvZmlsZT0ibGlicmVvZmZpY2Utc29mZmljZSIgbmFtZT0iL2hvbWUvY2ltL252bWUvJFJlY3lj
bGUuQmluL1MtMS01LTIxLTExNzUyNTUyNTEtMTkzODMwODQ4NC0yNjA0MzIzMzMzLTUwMC9kZXNr
dG9wLmluaSIgcGlkPTI2MzkgY29tbT0ic29mZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJyIiBk
ZW5pZWRfbWFzaz0iciIgZnN1aWQ9MTAwMCBvdWlkPTAKU2VwIDEzIDE3OjE2OjUwIGRlYmlhbiBr
ZXJuZWw6IFsgNTE4My40MzgyOTJdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUzMzYxMC45
MTg6MjcpOiBhcHBhcm1vcj0iQUxMT1dFRCIgb3BlcmF0aW9uPSJvcGVuIiBwcm9maWxlPSJsaWJy
ZW9mZmljZS1zb2ZmaWNlIiBuYW1lPSIvaG9tZS9jaW0vbnZtZS8kUmVjeWNsZS5CaW4vUy0xLTUt
MjEtMTE3NTI1NTI1MS0xOTM4MzA4NDg0LTI2MDQzMjMzMzMtNTAwLy5+bG9jay5kZXNrdG9wLmlu
aSMiIHBpZD0yNjM5IGNvbW09InNvZmZpY2UuYmluIiByZXF1ZXN0ZWRfbWFzaz0id3JjIiBkZW5p
ZWRfbWFzaz0id3JjIiBmc3VpZD0xMDAwIG91aWQ9MApTZXAgMTMgMTc6MTY6NTQgZGViaWFuIGtl
cm5lbDogWyA1MTg2LjUyNzQxMV0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTMzNjE0LjAw
NjoyOCk6IGFwcGFybW9yPSJBTExPV0VEIiBvcGVyYXRpb249Im9wZW4iIHByb2ZpbGU9ImxpYnJl
b2ZmaWNlLXNvZmZpY2UiIG5hbWU9Ii9ob21lL2NpbS9udm1lLyRSZWN5Y2xlLkJpbi9TLTEtNS0y
MS0xMTc1MjU1MjUxLTE5MzgzMDg0ODQtMjYwNDMyMzMzMy01MDAvLn5sb2NrLmRlc2t0b3AuaW5p
IyIgcGlkPTI2MzkgY29tbT0ic29mZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJyIiBkZW5pZWRf
bWFzaz0iciIgZnN1aWQ9MTAwMCBvdWlkPTAKU2VwIDEzIDE3OjE2OjU0IGRlYmlhbiBrZXJuZWw6
IFsgNTE4Ni41Mjc1NTZdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTUzMzYxNC4wMDY6Mjkp
OiBhcHBhcm1vcj0iQUxMT1dFRCIgb3BlcmF0aW9uPSJ1bmxpbmsiIHByb2ZpbGU9ImxpYnJlb2Zm
aWNlLXNvZmZpY2UiIG5hbWU9Ii9ob21lL2NpbS9udm1lLyRSZWN5Y2xlLkJpbi9TLTEtNS0yMS0x
MTc1MjU1MjUxLTE5MzgzMDg0ODQtMjYwNDMyMzMzMy01MDAvLn5sb2NrLmRlc2t0b3AuaW5pIyIg
cGlkPTI2MzkgY29tbT0ic29mZmljZS5iaW4iIHJlcXVlc3RlZF9tYXNrPSJkIiBkZW5pZWRfbWFz
az0iZCIgZnN1aWQ9MTAwMCBvdWlkPTAKU2VwIDEzIDE3OjU1OjI5IGRlYmlhbiBrZXJuZWw6IFsg
NzUwMS42Mjc2NTJdICBudm1lMm4xOiBwMSBwMgpTZXAgMTMgMTc6NTY6MDUgZGViaWFuIGtlcm5l
bDogWyA3NTM3Ljg3NjYwN10gIG52bWUybjE6IHAxIHAyClNlcCAxMyAxNzo1ODo1MiBkZWJpYW4g
a2VybmVsOiBbIDc3MDQuNzExMDk1XSAgbnZtZTJuMTogcDEgcDIKU2VwIDEzIDE4OjA2OjE0IGRl
YmlhbiBrZXJuZWw6IFsgODE0Ny4yODU5MjJdIEVYVDQtZnMgKG52bWUybjFwMSk6IG1vdW50ZWQg
ZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkKU2VwIDEzIDE4
OjA2OjE5IGRlYmlhbiBrZXJuZWw6IFsgODE1MS41NDIyNzddIEVYVDQtZnMgKG52bWUybjFwMik6
IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEJvb3RpbmcgTGlu
dXggb24gcGh5c2ljYWwgQ1BVIDB4MDAwMDEyMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIExpbnV4IHZlcnNpb24gNS4xMC4wLTgt
YXJtNjQgKGRlYmlhbi1rZXJuZWxAbGlzdHMuZGViaWFuLm9yZykgKGdjYy0xMCAoRGViaWFuIDEw
LjIuMS02KSAxMC4yLjEgMjAyMTAxMTAsIEdOVSBsZCAoR05VIEJpbnV0aWxzIGZvciBEZWJpYW4p
IDIuMzUuMikgIzEgU01QIERlYmlhbiA1LjEwLjQ2LTQgKDIwMjEtMDgtMDMpClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBlZmk6IEVGSSB2Mi43MCBieSBBbWVy
aWNhbiBNZWdhdHJlbmRzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBlZmk6IEFDUEkgMi4wPTB4ZWU5MDAwMDAgU01CSU9TIDMuMD0weGYxYjNmZjk4IEVTUlQ9
MHhlOTU2MWY5OCBNT0t2YXI9MHhlOTU1YzAwMCBSTkc9MHhmNzY2ZTgxOCBNRU1SRVNFUlZFPTB4
ZTlkNTg4MTggClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBl
Zmk6IHNlZWRpbmcgZW50cm9weSBwb29sClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBzZWN1cmVib290OiBTZWN1cmUgYm9vdCBkaXNhYmxlZApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gZXNydDogUmVzZXJ2aW5nIEVTUlQg
c3BhY2UgZnJvbSAweDAwMDAwMDAwZTk1NjFmOTggdG8gMHgwMDAwMDAwMGU5NTYxZmQwLgpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRWFybHkgdGFi
bGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2FibGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAwMDBFRTkwMDAwMCAwMDAw
MjQgKHYwMiBBbXBlcmUpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDBFRThGMDAwMCAwMDAwQTQgKHYwMSBBbXBlcmUgQWx0
cmEgICAgMDAwMDAwMDAgQU1JICAwMTAwMDAxMykKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IEZBQ1AgMHgwMDAwMDAwMEVFOEQwMDAwIDAwMDExNCAo
djA2IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogRFNEVCAweDAwMDAwMDAwRUU4
NjAwMDAgMDE5QTk1ICh2MDIgQW1wZXJlIEphZGUgICAgIDAwMDAwMDAxIElOVEwgMjAyMDA3MTcp
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBEQkcy
IDB4MDAwMDAwMDBFRThFMDAwMCAwMDAwNUMgKHYwMCBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDAg
SU5UTCAyMDE5MDUwOSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IEdURFQgMHgwMDAwMDAwMEVFOEMwMDAwIDAwMDExMCAodjAzIEFtcGVyZSBBbHRy
YSAgICAwMDAwMDAwMCBJTlRMIDIwMTkwNTA5KQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogU1NEVCAweDAwMDAwMDAwRUU4QjAwMDAgMDAwMDJEICh2
MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAxIElOVEwgMjAxOTA1MDkpClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBGSURUIDB4MDAwMDAwMDBFRTg1
MDAwMCAwMDAwOUMgKHYwMSBBTEFTS0EgQSBNIEkgICAgMDEwNzIwMDkgQU1JICAwMDAxMDAxMykK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNQQ1Ig
MHgwMDAwMDAwMEVFODQwMDAwIDAwMDA1MCAodjAyIEFMQVNLQSBBIE0gSSAgICAwMTA3MjAwOSBB
TUkgIDAwMDUwMDBGKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogQkdSVCAweDAwMDAwMDAwRUU4MzAwMDAgMDAwMDM4ICh2MDEgQUxBU0tBIEEgTSBJ
ICAgIDAxMDcyMDA5IEFNSSAgMDAwMTAwMTMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBNQ0ZHIDB4MDAwMDAwMDBFRTgyMDAwMCAwMDAwQUMgKHYw
MSBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDEgQU1QLiAwMTAwMDAxMykKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IElPUlQgMHgwMDAwMDAwMEVFODEw
MDAwIDAwMDYxMCAodjAwIEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogUFBUVCAw
eDAwMDAwMDAwRUU3RjAwMDAgMDA2RTYwICh2MDIgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAwIEFN
UC4gMDEwMDAwMTMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTTElUIDB4MDAwMDAwMDBFRTdFMDAwMCAwMDAwMkQgKHYwMSBBbXBlcmUgQWx0cmEg
ICAgMDAwMDAwMDAgQU1QLiAwMTAwMDAxMykKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IFNSQVQgMHgwMDAwMDAwMEVFN0QwMDAwIDAwMDZEMCAodjAz
IEFtcGVyZSBBbHRyYSAgICAwMDAwMDAwMCBBTVAuIDAxMDAwMDEzKQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogQVBJQyAweDAwMDAwMDAwRUU4MDAw
MDAgMDAxOUY0ICh2MDUgQW1wZXJlIEFsdHJhICAgIDAwMDAwMDAzIEFNSSAgMDEwMDAwMTMpClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBQQ0NUIDB4
MDAwMDAwMDBFRTdDMDAwMCAwMDA1NzYgKHYwMiBBbXBlcmUgQWx0cmEgICAgMDAwMDAwMDMgQU1Q
LiAwMTAwMDAxMykKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IFdTTVQgMHgwMDAwMDAwMEVFN0IwMDAwIDAwMDAyOCAodjAxIEFMQVNLQSBBIE0gSSAg
ICAwMTA3MjAwOSBBTUkgIDAwMDEwMDEzKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogRlBEVCAweDAwMDAwMDAwRUU3QTAwMDAgMDAwMDQ0ICh2MDEg
QUxBU0tBIEEgTSBJICAgIDAxMDcyMDA5IEFNSSAgMDEwMDAwMTMpClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUENSOiBjb25zb2xlOiBwbDAxMSxt
bWlvMzIsMHgxMDAwMDI2MDAwMDAsMTE1MjAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDg4MzAwMDAw
LTB4ODgzZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhNIDAgW21lbSAweDkwMDAwMDAwLTB4ZmZmZmZmZmZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBO
b2RlIDAgUFhNIDAgW21lbSAweDgwMDAwMDAwMDAwLTB4ODAwN2ZmZmZmZmZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBTUkFUOiBOb2RlIDAgUFhN
IDAgW21lbSAweDgwMTAwMDAwMDAwLTB4ODBmZmZmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBOVU1BOiBOT0RFX0RBVEEgW21lbSAweDgwZmY4MDg1
YjAwLTB4ODBmZjgwODdmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBab25lIHJhbmdlczoKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdICAgRE1BICAgICAgW21lbSAweDAwMDAwMDAwODgzMDAwMDAtMHgwMDAwMDAwMGZm
ZmZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gICBE
TUEzMiAgICBlbXB0eQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gICBOb3JtYWwgICBbbWVtIDB4MDAwMDAwMDEwMDAwMDAwMC0weDAwMDAwODBmZmZmZmZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBNb3ZhYmxlIHpv
bmUgc3RhcnQgZm9yIGVhY2ggbm9kZQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gRWFybHkgbWVtb3J5IG5vZGUgcmFuZ2VzClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA4ODMw
MDAwMC0weDAwMDAwMDAwODgzZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MDAwMDAwMC0weDAwMDAw
MDAwOTFmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDA5MjAwMDAwMC0weDAwMDAwMDAwOTNmZmZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAw
OiBbbWVtIDB4MDAwMDAwMDA5NDAwMDAwMC0weDAwMDAwMDAwZWUyZTFmZmZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAw
MDAwMDBlZTJlMjAwMC0weDAwMDAwMDAwZWU3NDVmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0NjAw
MC0weDAwMDAwMDAwZWU3NDlmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0YTAwMC0weDAwMDAwMDAw
ZWU3NGNmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAg
IG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTc0ZDAwMC0weDAwMDAwMDAwZWU3NGRmZmZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBb
bWVtIDB4MDAwMDAwMDBlZTc0ZTAwMC0weDAwMDAwMDAwZWU3NGZmZmZdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAw
MDBlZTc1MDAwMC0weDAwMDAwMDAwZWU5NmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk3MDAwMC0w
eDAwMDAwMDAwZWU5N2ZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBlZTk4MDAwMC0weDAwMDAwMDAwZjA2
NGZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5v
ZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmMDY1MDAwMC0weDAwMDAwMDAwZjc2NmZmZmZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVt
IDB4MDAwMDAwMDBmNzY3MDAwMC0weDAwMDAwMDAwZjc4NGZmZmZdClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBm
Nzg1MDAwMC0weDAwMDAwMDAwZjdmZGZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmN2ZlMDAwMC0weDAw
MDAwMDAwZmZjOGVmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM4ZjAwMC0weDAwMDAwMDAwZmZjOGZm
ZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUg
ICAwOiBbbWVtIDB4MDAwMDAwMDBmZmM5MDAwMC0weDAwMDAwMDAwZmZmZmZmZmZdClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4
MDAwMDA4MDAwMDAwMDAwMC0weDAwMDAwODAwN2ZmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSAgIG5vZGUgICAwOiBbbWVtIDB4MDAwMDA4MDEwMDAw
MDAwMC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBJbml0bWVtIHNldHVwIG5vZGUgMCBbbWVtIDB4MDAwMDAwMDA4ODMwMDAw
MC0weDAwMDAwODBmZmZmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBPbiBub2RlIDAgdG90YWxwYWdlczogMTY3MTE5MzYKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgRE1BIHpvbmU6IDcxNzIgcGFnZXMgdXNl
ZCBmb3IgbWVtbWFwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSAgIERNQSB6b25lOiAwIHBhZ2VzIHJlc2VydmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSAgIERNQSB6b25lOiA0NTkwMDggcGFnZXMsIExJRk8gYmF0Y2g6
NjMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFs
IHpvbmU6IDI1Mzk1MiBwYWdlcyB1c2VkIGZvciBtZW1tYXAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdICAgTm9ybWFsIHpvbmU6IDE2MjUyOTI4IHBhZ2VzLCBM
SUZPIGJhdGNoOjYzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBjbWE6IFJlc2VydmVkIDY0IE1pQiBhdCAweDAwMDAwMDAwZmJjMDAwMDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IHByb2JpbmcgZm9yIGNvbmR1
aXQgbWV0aG9kIGZyb20gQUNQSS4KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIHBzY2k6IFBTQ0l2MS4xIGRldGVjdGVkIGluIGZpcm13YXJlLgpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcHNjaTogVXNpbmcgc3RhbmRhcmQg
UFNDSSB2MC4yIGZ1bmN0aW9uIElEcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gcHNjaTogTUlHUkFURV9JTkZPX1RZUEUgbm90IHN1cHBvcnRlZC4KU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBzY2k6IFNNQyBDYWxsaW5n
IENvbnZlbnRpb24gdjEuMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgwIC0+IE5vZGUgMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHgxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDEwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMDEwMCAtPiBOb2RlIDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDIwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgzMDAwMCAtPiBO
b2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MzAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDQwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHg0MDEw
MCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NTAwMDAgLT4gTm9kZSAwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDUwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIg
MHg2MDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4NjAxMDAgLT4gTm9kZSAw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDcwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHg3MDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4ODAwMDAgLT4g
Tm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDgwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHg5MDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4OTAx
MDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGEwMDAwIC0+IE5vZGUgMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JB
VDogUFhNIDAgLT4gTVBJRFIgMHhhMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4YjAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGIwMTAwIC0+IE5vZGUg
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhjMDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4YzAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGQwMDAwIC0+
IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhkMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4ZTAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweGUw
MTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHhmMDAwMCAtPiBOb2RlIDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4ZjAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElE
UiAweDEwMDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTAwMTAwIC0+IE5v
ZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTog
TlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMTAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0g
MCAtPiBNUElEUiAweDExMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTIw
MDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAw
MF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxMjAxMDAgLT4gTm9kZSAwClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBT
UkFUOiBQWE0gMCAtPiBNUElEUiAweDEzMDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1Q
SURSIDB4MTMwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxNDAwMDAgLT4g
Tm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJ
OiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE0MDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBY
TSAwIC0+IE1QSURSIDB4MTUwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgx
NTAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE2MDAwMCAtPiBOb2RlIDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6
IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTYwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4g
TVBJRFIgMHgxNzAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDE3MDEwMCAt
PiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFD
UEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTgwMDAwIC0+IE5vZGUgMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDog
UFhNIDAgLT4gTVBJRFIgMHgxODAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAw
eDE5MDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MTkwMTAwIC0+IE5vZGUg
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVN
QTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYTAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAt
PiBNUElEUiAweDFhMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MWIwMDAw
IC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
QUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxYjAxMDAgLT4gTm9kZSAwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFU
OiBQWE0gMCAtPiBNUElEUiAweDFjMDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURS
IDB4MWMwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZDAwMDAgLT4gTm9k
ZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBO
VU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFkMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAw
IC0+IE1QSURSIDB4MWUwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgxZTAx
MDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDFmMDAwMCAtPiBOb2RlIDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNS
QVQ6IFBYTSAwIC0+IE1QSURSIDB4MWYwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJ
RFIgMHgyMDAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIwMDEwMCAtPiBO
b2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6
IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjEwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhN
IDAgLT4gTVBJRFIgMHgyMTAxMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDIy
MDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjIwMTAwIC0+IE5vZGUgMApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTog
U1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyMzAwMDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQWE0gMCAtPiBN
UElEUiAweDIzMDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4MjQwMDAwIC0+
IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQUNQ
STogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNDAxMDAgLT4gTm9kZSAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1BOiBTUkFUOiBQ
WE0gMCAtPiBNUElEUiAweDI1MDAwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+IE1QSURSIDB4
MjUwMTAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNjAwMDAgLT4gTm9kZSAw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBBQ1BJOiBOVU1B
OiBTUkFUOiBQWE0gMCAtPiBNUElEUiAweDI2MDEwMCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEFDUEk6IE5VTUE6IFNSQVQ6IFBYTSAwIC0+
IE1QSURSIDB4MjcwMDAwIC0+IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gQUNQSTogTlVNQTogU1JBVDogUFhNIDAgLT4gTVBJRFIgMHgyNzAxMDAg
LT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBw
ZXJjcHU6IEVtYmVkZGVkIDMzIHBhZ2VzL2NwdSBzOTUxOTIgcjgxOTIgZDMxNzg0IHUxMzUxNjgK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6
IHM5NTE5MiByODE5MiBkMzE3ODQgdTEzNTE2OCBhbGxvYz0zMyo0MDk2ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMDAgWzBdIDAx
IFswXSAwMiBbMF0gMDMgWzBdIDA0IFswXSAwNSBbMF0gMDYgWzBdIDA3IApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDA4IFswXSAw
OSBbMF0gMTAgWzBdIDExIFswXSAxMiBbMF0gMTMgWzBdIDE0IFswXSAxNSAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSAxNiBbMF0g
MTcgWzBdIDE4IFswXSAxOSBbMF0gMjAgWzBdIDIxIFswXSAyMiBbMF0gMjMgClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gMjQgWzBd
IDI1IFswXSAyNiBbMF0gMjcgWzBdIDI4IFswXSAyOSBbMF0gMzAgWzBdIDMxIApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDMyIFsw
XSAzMyBbMF0gMzQgWzBdIDM1IFswXSAzNiBbMF0gMzcgWzBdIDM4IFswXSAzOSAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA0MCBb
MF0gNDEgWzBdIDQyIFswXSA0MyBbMF0gNDQgWzBdIDQ1IFswXSA0NiBbMF0gNDcgClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0gNDgg
WzBdIDQ5IFswXSA1MCBbMF0gNTEgWzBdIDUyIFswXSA1MyBbMF0gNTQgWzBdIDU1IApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gcGNwdS1hbGxvYzogWzBdIDU2
IFswXSA1NyBbMF0gNTggWzBdIDU5IFswXSA2MCBbMF0gNjEgWzBdIDYyIFswXSA2MyAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHBjcHUtYWxsb2M6IFswXSA2
NCBbMF0gNjUgWzBdIDY2IFswXSA2NyBbMF0gNjggWzBdIDY5IFswXSA3MCBbMF0gNzEgClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwY3B1LWFsbG9jOiBbMF0g
NzIgWzBdIDczIFswXSA3NCBbMF0gNzUgWzBdIDc2IFswXSA3NyBbMF0gNzggWzBdIDc5IApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVy
ZmFjZQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogVmlydHVhbGl6YXRpb24gSG9zdCBFeHRlbnNpb25zClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGtlcm5l
bCBwYWdlIHRhYmxlIGlzb2xhdGlvbiBmb3JjZWQgT04gYnkgS0FTTFIKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEtl
cm5lbCBwYWdlIHRhYmxlIGlzb2xhdGlvbiAoS1BUSSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IEhhcmR3YXJlIGRp
cnR5IGJpdCBtYW5hZ2VtZW50ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBTcGVjdHJlLXY0ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBB
Uk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBhbHRlcm5hdGl2ZXM6IHBhdGNoaW5nIGtlcm5lbCBjb2RlClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBCdWlsdCAxIHpvbmVsaXN0cywgbW9iaWxp
dHkgZ3JvdXBpbmcgb24uICBUb3RhbCBwYWdlczogMTY0NTA4MTIKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFBvbGljeSB6b25lOiBOb3JtYWwKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEtlcm5lbCBjb21tYW5kIGxpbmU6
IEJPT1RfSU1BR0U9L2Jvb3Qvdm1saW51ei01LjEwLjAtOC1hcm02NCByb290PVVVSUQ9NmMwODMy
N2MtNGJiZC00MzQxLWE0MjctYTcyYmMxNTMzODk1IHJvIHF1aWV0ClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmludGs6IGxvZ19idWZfbGVuIGluZGl2aWR1
YWwgbWF4IGNwdSBjb250cmlidXRpb246IDQwOTYgYnl0ZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogbG9nX2J1Zl9sZW4gdG90YWwgY3B1X2V4
dHJhIGNvbnRyaWJ1dGlvbnM6IDMyMzU4NCBieXRlcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwMDAwMF0gcHJpbnRrOiBsb2dfYnVmX2xlbiBtaW4gc2l6ZTogMTMxMDcy
IGJ5dGVzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBwcmlu
dGs6IGxvZ19idWZfbGVuOiA1MjQyODggYnl0ZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIHByaW50azogZWFybHkgbG9nIGJ1ZiBmcmVlOiAxMTkwMTYoOTAl
KQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gRGVudHJ5IGNh
Y2hlIGhhc2ggdGFibGUgZW50cmllczogODM4ODYwOCAob3JkZXI6IDE0LCA2NzEwODg2NCBieXRl
cywgbGluZWFyKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRyaWVzOiA0MTk0MzA0IChvcmRlcjogMTMsIDMzNTU0
NDMyIGJ5dGVzLCBsaW5lYXIpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMDAwXSBtZW0gYXV0by1pbml0OiBzdGFjazpvZmYsIGhlYXAgYWxsb2M6b24sIGhlYXAgZnJl
ZTpvZmYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHNvZnR3
YXJlIElPIFRMQjogbWFwcGVkIFttZW0gMHgwMDAwMDAwMGVhMmUyMDAwLTB4MDAwMDAwMDBlZTJl
MjAwMF0gKDY0TUIpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAw
XSBNZW1vcnk6IDI1NDc3NjRLLzY2ODQ3NzQ0SyBhdmFpbGFibGUgKDExNzc2SyBrZXJuZWwgY29k
ZSwgMjQzNksgcndkYXRhLCA3MDA4SyByb2RhdGEsIDU0NDBLIGluaXQsIDU5OEsgYnNzLCAxNTg5
NDE2SyByZXNlcnZlZCwgNjU1MzZLIGNtYS1yZXNlcnZlZCkKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIHJhbmRvbTogZ2V0X3JhbmRvbV91NjQgY2FsbGVkIGZy
b20gX19rbWVtX2NhY2hlX2NyZWF0ZSsweDM4LzB4NTYwIHdpdGggY3JuZ19pbml0PTAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNMVUI6IEhXYWxpZ249NjQs
IE9yZGVyPTAtMywgTWluT2JqZWN0cz0wLCBDUFVzPTgwLCBOb2Rlcz0xClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBmdHJhY2U6IGFsbG9jYXRpbmcgMzg1MzUg
ZW50cmllcyBpbiAxNTEgcGFnZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIGZ0cmFjZTogYWxsb2NhdGVkIDE1MSBwYWdlcyB3aXRoIDUgZ3JvdXBzClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSByY3U6IEhpZXJhcmNoaWNh
bCBSQ1UgaW1wbGVtZW50YXRpb24uClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSByY3U6IAlSQ1UgcmVzdHJpY3RpbmcgQ1BVcyBmcm9tIE5SX0NQVVM9MjU2IHRv
IG5yX2NwdV9pZHM9ODAuClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDAwXSAJUnVkZSB2YXJpYW50IG9mIFRhc2tzIFJDVSBlbmFibGVkLgpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gCVRyYWNpbmcgdmFyaWFudCBvZiBUYXNrcyBS
Q1UgZW5hYmxlZC4KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBd
IHJjdTogUkNVIGNhbGN1bGF0ZWQgdmFsdWUgb2Ygc2NoZWR1bGVyLWVubGlzdG1lbnQgZGVsYXkg
aXMgMjUgamlmZmllcy4KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIHJjdTogQWRqdXN0aW5nIGdlb21ldHJ5IGZvciByY3VfZmFub3V0X2xlYWY9MTYsIG5yX2Nw
dV9pZHM9ODAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIE5S
X0lSUVM6IDY0LCBucl9pcnFzOiA2NCwgcHJlYWxsb2NhdGVkIGlycXM6IDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBHSUM6IFVzaW5nIHNwbGl0
IEVPSS9EZWFjdGl2YXRlIG1vZGUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDAwMDBdIEdJQ3YzOiA2NzIgU1BJcyBpbXBsZW1lbnRlZApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IDAgRXh0ZW5kZWQgU1BJcyBpbXBsZW1l
bnRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6
IERpc3RyaWJ1dG9yIGhhcyBubyBSYW5nZSBTZWxlY3RvciBzdXBwb3J0ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogMTYgUFBJcyBpbXBsZW1lbnRl
ZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gR0lDdjM6IENQ
VTA6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTIwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVjMDAw
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDogUFhN
IDAgLT4gSVRTIDAgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgMSAtPiBOb2RlIDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyAyIC0+IE5v
ZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JBVDog
UFhNIDAgLT4gSVRTIDMgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNCAtPiBOb2RlIDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIFNSQVQ6IFBYTSAwIC0+IElUUyA1IC0+
IE5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gU1JB
VDogUFhNIDAgLT4gSVRTIDYgLT4gTm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBTUkFUOiBQWE0gMCAtPiBJVFMgNyAtPiBOb2RlIDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMDQw
MDAwLTB4MTAwMTAwMDVmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDA0MDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBA
ODAwMDAyYzAwMDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMDQwMDAw
OiBhbGxvY2F0ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDJkMDAwMCAoZmxh
dCwgZXN6IDIsIHBzeiA2NEssIHNociAxKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwNjAwMDAtMHgxMDAxMDAwN2ZmZmZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAw
MTAwMDYwMDAwOiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDJmMDAwMCAoaW5kaXJlY3Qs
IGVzeiA4LCBwc3ogNjRLLCBzaHIgMSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwNjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRl
cnJ1cHQgQ29sbGVjdGlvbnMgQDgwMDAwMzAwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hy
IDEpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21l
bSAweDEwMDEwMDA4MDAwMC0weDEwMDEwMDA5ZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwODAwMDA6IGFsbG9jYXRlZCA4
MTkyIERldmljZXMgQDgwMDAwMzIwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAx
KQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDA4MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAw
MDAzMzAwMDAgKGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMGEwMDAwLTB4MTAw
MTAwMGJmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0g
SVRTQDB4MDAwMDEwMDEwMDBhMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzNTAw
MDAgKGluZGlyZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGEwMDAwOiBhbGxvY2F0
ZWQgMzI3NjggSW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDM2MDAwMCAoZmxhdCwgZXN6IDIs
IHBzeiA2NEssIHNociAxKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
MDAwMF0gSVRTIFttZW0gMHgxMDAxMDAwYzAwMDAtMHgxMDAxMDAwZGZmZmZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMGMwMDAw
OiBhbGxvY2F0ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDM4MDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBw
c3ogNjRLLCBzaHIgMSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAw
MDBdIElUU0AweDAwMDAxMDAxMDAwYzAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29s
bGVjdGlvbnMgQDgwMDAwMzkwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFMgW21lbSAweDEwMDEw
MDBlMDAwMC0weDEwMDEwMDBmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDAwMDBdIElUU0AweDAwMDAxMDAxMDAwZTAwMDA6IGFsbG9jYXRlZCA4MTkyIERldmlj
ZXMgQDgwMDAwM2IwMDAwIChpbmRpcmVjdCwgZXN6IDgsIHBzeiA2NEssIHNociAxKQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAwMDEwMDEwMDBl
MDAwMDogYWxsb2NhdGVkIDMyNzY4IEludGVycnVwdCBDb2xsZWN0aW9ucyBAODAwMDAzYzAwMDAg
KGZsYXQsIGVzeiAyLCBwc3ogNjRLLCBzaHIgMSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDAwMDBdIElUUyBbbWVtIDB4MTAwMTAwMTAwMDAwLTB4MTAwMTAwMTFmZmZm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRTQDB4MDAw
MDEwMDEwMDEwMDAwMDogYWxsb2NhdGVkIDgxOTIgRGV2aWNlcyBAODAwMDAzZTAwMDAgKGluZGly
ZWN0LCBlc3ogOCwgcHN6IDY0Sywgc2hyIDEpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTAwMDAwOiBhbGxvY2F0ZWQgMzI3Njgg
SW50ZXJydXB0IENvbGxlY3Rpb25zIEA4MDAwMDNmMDAwMCAoZmxhdCwgZXN6IDIsIHBzeiA2NEss
IHNociAxKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gSVRT
IFttZW0gMHgxMDAxMDAxMjAwMDAtMHgxMDAxMDAxM2ZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBJVFNAMHgwMDAwMTAwMTAwMTIwMDAwOiBhbGxvY2F0
ZWQgODE5MiBEZXZpY2VzIEA4MDAwMDgxMDAwMCAoaW5kaXJlY3QsIGVzeiA4LCBwc3ogNjRLLCBz
aHIgMSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAwMDBdIElUU0Aw
eDAwMDAxMDAxMDAxMjAwMDA6IGFsbG9jYXRlZCAzMjc2OCBJbnRlcnJ1cHQgQ29sbGVjdGlvbnMg
QDgwMDAwODIwMDAwIChmbGF0LCBlc3ogMiwgcHN6IDY0Sywgc2hyIDEpClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBHSUN2MzogdXNpbmcgTFBJIHByb3BlcnR5
IHRhYmxlIEAweDAwMDAwODAwMDA4MzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDAwMDBdIEdJQ3YzOiBDUFUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDg0MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMDAwMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwMDAwMF0gQUNQSSBHVERUOiBmb3VuZCAxIG1lbW9yeS1tYXBwZWQgdGltZXIgYmxvY2socyku
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBhcmNoX3RpbWVy
OiBjcDE1IGFuZCBtbWlvIHRpbWVyKHMpIHJ1bm5pbmcgYXQgMjUuMDBNSHogKHBoeXMvcGh5cyku
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDAwXSBjbG9ja3NvdXJj
ZTogYXJjaF9zeXNfY291bnRlcjogbWFzazogMHhmZmZmZmZmZmZmZmZmZiBtYXhfY3ljbGVzOiAw
eDVjNDA5MzliNSwgbWF4X2lkbGVfbnM6IDQ0MDc5NTIwMjY0NiBucwpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDAwMF0gc2NoZWRfY2xvY2s6IDU2IGJpdHMgYXQgMjVN
SHosIHJlc29sdXRpb24gNDBucywgd3JhcHMgZXZlcnkgNDM5ODA0NjUxMTEwMG5zClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMDU1XSBDb25zb2xlOiBjb2xvdXIgZHVt
bXkgZGV2aWNlIDgweDI1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAw
MDc3XSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIwMjAwOTI1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDAwMjcxXSBDYWxpYnJhdGluZyBkZWxheSBsb29wIChza2lwcGVkKSwg
dmFsdWUgY2FsY3VsYXRlZCB1c2luZyB0aW1lciBmcmVxdWVuY3kuLiA1MC4wMCBCb2dvTUlQUyAo
bHBqPTEwMDAwMCkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDAyNzNd
IHBpZF9tYXg6IGRlZmF1bHQ6IDgxOTIwIG1pbmltdW06IDY0MApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDMwN10gTFNNOiBTZWN1cml0eSBGcmFtZXdvcmsgaW5pdGlh
bGl6aW5nClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAwMzEyXSBZYW1h
OiBkaXNhYmxlZCBieSBkZWZhdWx0OyBlbmFibGUgd2l0aCBzeXNjdGwga2VybmVsLnlhbWEuKgpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMDMyOV0gQXBwQXJtb3I6IEFw
cEFybW9yIGluaXRpYWxpemVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAwMzMxXSBUT01PWU8gTGludXggaW5pdGlhbGl6ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDAzNzFdIE1vdW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMx
MDcyIChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGluZWFyKQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAwMDM5Nl0gTW91bnRwb2ludC1jYWNoZSBoYXNoIHRhYmxlIGVu
dHJpZXM6IDEzMTA3MiAob3JkZXI6IDgsIDEwNDg1NzYgYnl0ZXMsIGxpbmVhcikKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDEyNjBdIHJjdTogSGllcmFyY2hpY2FsIFNS
Q1UgaW1wbGVtZW50YXRpb24uClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxNDA0XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3JlYXRlZApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwNV0gUGxhdGZvcm0gTVNJ
OiBJVFNAMHgxMDAxMDAwNjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDE0MDddIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAw
IGRvbWFpbiBjcmVhdGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAx
NDA4XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQwOV0gUGxhdGZvcm0gTVNJOiBJ
VFNAMHgxMDAxMDAwYzAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMDE0MTFdIFBsYXRmb3JtIE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRv
bWFpbiBjcmVhdGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDEy
XSBQbGF0Zm9ybSBNU0k6IElUU0AweDEwMDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxM10gUGxhdGZvcm0gTVNJOiBJVFNA
MHgxMDAxMDAxMjAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDE0MTddIFBDSS9NU0k6IElUU0AweDEwMDEwMDA0MDAwMCBkb21haW4gY3Jl
YXRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQxOF0gUENJL01T
STogSVRTQDB4MTAwMTAwMDYwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDAxNDE5XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwODAwMDAgZG9t
YWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjFd
IFBDSS9NU0k6IElUU0AweDEwMDEwMDBhMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQyM10gUENJL01TSTogSVRTQDB4MTAwMTAwMGMw
MDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDAxNDI1XSBQQ0kvTVNJOiBJVFNAMHgxMDAxMDAwZTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDE0MjddIFBDSS9NU0k6IElUU0AweDEw
MDEwMDEwMDAwMCBkb21haW4gY3JlYXRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwMTQyOF0gUENJL01TSTogSVRTQDB4MTAwMTAwMTIwMDAwIGRvbWFpbiBjcmVhdGVk
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDMxXSBmc2wtbWMgTVNJ
OiBJVFNAMHgxMDAxMDAwNDAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDE0MzJdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDA2MDAwMCBk
b21haW4gY3JlYXRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQz
M10gZnNsLW1jIE1TSTogSVRTQDB4MTAwMTAwMDgwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDM1XSBmc2wtbWMgTVNJOiBJVFNAMHgx
MDAxMDAwYTAwMDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDE0MzZdIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDBjMDAwMCBkb21haW4gY3Jl
YXRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQzN10gZnNsLW1j
IE1TSTogSVRTQDB4MTAwMTAwMGUwMDAwIGRvbWFpbiBjcmVhdGVkClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDAxNDM4XSBmc2wtbWMgTVNJOiBJVFNAMHgxMDAxMDAxMDAw
MDAgZG9tYWluIGNyZWF0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDE0MzldIGZzbC1tYyBNU0k6IElUU0AweDEwMDEwMDEyMDAwMCBkb21haW4gY3JlYXRlZApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwMTQ0NV0gUmVtYXBwaW5nIGFuZCBl
bmFibGluZyBFRkkgc2VydmljZXMuClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDAzOTIwXSBzbXA6IEJyaW5naW5nIHVwIHNlY29uZGFyeSBDUFVzIC4uLgpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDU4OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hl
IG9uIENQVTEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ2MDhdIEdJ
Q3YzOiBDUFUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFhMDAwMCByZWdpb24gMDoweDAwMDAxMDAx
MDA3YzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDQ2MjZdIEdJ
Q3YzOiBDUFUxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MDg1MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDY1OV0gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNDY2OF0gQ1BVMTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTE4NF0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUyMDBd
IEdJQ3YzOiBDUFUyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE0MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDA2NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDUyMThd
IEdJQ3YzOiBDUFUyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMDg2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTI1NF0g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTI2M10gQ1BVMjog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTQwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTQ3Ml0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU0
ODldIEdJQ3YzOiBDUFUzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDAwMCByZWdpb24gMDoweDAw
MDAxMDAxMDA4NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDU1
MDZdIEdJQ3YzOiBDUFUzOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMDg3MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTUz
OV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTU0OF0gQ1BV
MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWMwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTg0M10gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU4NTddIEdJQ3YzOiBDUFU0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDA1NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MDU4NzZdIEdJQ3YzOiBDUFU0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMDg4MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NTkxMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNTkyMV0g
Q1BVNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMDAwIFsweDQxM2ZkMGMx
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjI0OV0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDYyNjVdIEdJQ3YzOiBDUFU1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE4MDAwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA3NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMDYyODNdIEdJQ3YzOiBDUFU1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUg
QDB4MDAwMDA4MDAwMDg5MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNjMxOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVy
cmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjMy
OV0gQ1BVNTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTgwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNjY1OF0gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDY2NzRdIEdJQ3YzOiBDUFU2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE2MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA2YzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMDY2OTNdIEdJQ3YzOiBDUFU2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMDhhMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAwNjcyOV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAw
NjczOF0gQ1BVNjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTYwMDAwIFsweDQx
M2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzA0OV0gRGV0
ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDcwNjddIEdJQ3YzOiBDUFU3OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFlMDAwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA4YzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMDcwODZdIEdJQ3YzOiBDUFU3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDhiMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAwNzEyMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAwNzEzMl0gQ1BVNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWUwMDAwIFsw
eDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwNzQ0Nl0g
RGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDc0NTldIEdJQ3YzOiBDUFU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGEwMDAw
IHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDNjMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwNzQ3OF0gR0lDdjM6IENQVTg6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGMwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDA3NTE2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA3NTI2XSBDUFU4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwYTAwMDAg
WzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA3ODQ3
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVOQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNzg2NV0gR0lDdjM6IENQVTk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDljMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAwNzg4M10gR0lDdjM6IENQVTk6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGQwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDA3OTE1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDA3OTI1XSBDUFU5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMjAw
MDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4
MjQwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgyNTNdIEdJQ3YzOiBDUFUxMDogZm91bmQgcmVkaXN0cmlidXRv
ciBjMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA0NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgyNzNdIEdJQ3YzOiBDUFUxMDogdXNpbmcgYWxsb2NhdGVkIExQ
SSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA4ZTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMDgzMDldIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmth
cm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMDgzMjBdIENQVTEwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAw
MDAwYzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDA4NjU0XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTEKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDg2NzNdIEdJQ3YzOiBDUFUxMTogZm91bmQgcmVkaXN0
cmlidXRvciAyNDAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYTQwMDAwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NjkyXSBHSUN2MzogQ1BVMTE6IHVzaW5nIGFsbG9j
YXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOGYwMDAwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA4NzI1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2Nh
bCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDA4NzM2XSBDUFUxMTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNz
b3IgMHgwMDAwMjQwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAwOTA0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEyClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDYxXSBHSUN2MzogQ1BVMTI6IGZvdW5k
IHJlZGlzdHJpYnV0b3IgODAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwMzQwMDAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MDgxXSBHSUN2MzogQ1BVMTI6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTAwMDAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MTE3XSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5MTI4XSBDUFUxMjogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMDgwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAwOTQ0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTEzClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDA5NDY3XSBHSUN2MzogQ1BVMTM6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjAwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDk0MDAwMApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTQ4N10gR0lDdjM6IENQVTEz
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDkxMDAwMApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTUxOV0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTUzMF0gQ1BVMTM6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIwMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMDk4NTJdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFUxNApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg2Nl0gR0lDdjM6
IENQVTE0OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDRj
MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTg4OF0gR0lDdjM6
IENQVTE0OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDky
MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTkyNV0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAwOTkzNl0gQ1BVMTQ6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBlMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyNTRdIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFUxNQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMDI3NF0g
R0lDdjM6IENQVTE1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI2MDAwMCByZWdpb24gMDoweDAwMDAx
MDAxMDBhYzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAyOTVd
IEdJQ3YzOiBDUFUxNTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAw
ODAwMDA5MzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAzMjld
IGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0
MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTAzNDBdIENQVTE1
OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNjAwMDAgWzB4NDEzZmQwYzFdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEwNjQ3XSBEZXRlY3RlZCBQSVBU
IEktY2FjaGUgb24gQ1BVMTYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA2NThdIEdJQ3YzOiBDUFUxNjogZm91bmQgcmVkaXN0cmlidXRvciAyMDAwMCByZWdpb24gMDow
eDAwMDAxMDAxMDAxYzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA2NzhdIEdJQ3YzOiBDUFUxNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDA5NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTA3MTZdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTA3Mjhd
IENQVTE2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMjAwMDAgWzB4NDEzZmQw
YzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExMDUxXSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVMTcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTEwNjJdIEdJQ3YzOiBDUFUxNzogZm91bmQgcmVkaXN0cmlidXRvciA0MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAyNDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTEwODRdIEdJQ3YzOiBDUFUxNzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA5NTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTExMjJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTExMzRdIENQVTE3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNDAwMDAgWzB4
NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDExNDUwXSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMTgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTE0NjBdIEdJQ3YzOiBDUFUxODogZm91bmQgcmVkaXN0cmlidXRvciAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMDE0MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTQ4M10gR0lDdjM6IENQVTE4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMDk2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMTUyMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxMTUzNF0gQ1BVMTg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAwMDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTIwNTZd
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUxOQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMjA2OF0gR0lDdjM6IENQVTE5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDYw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDJjMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMjA5MF0gR0lDdjM6IENQVTE5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDk3MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMjEyN10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxMjE0MF0gQ1BVMTk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA2
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTI0NTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUyMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxMjQ2Nl0gR0lDdjM6IENQVTIwOiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDEzMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2MDAwMDAKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTI0ODddIEdJQ3YzOiBDUFUyMDogdXNpbmcgYWxsb2NhdGVk
IExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5ODAwMDAKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTI1MjJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdv
cmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMTI1MzRdIENQVTIwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAw
eDAwMDAxMzAwMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDEyODUwXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjEKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTI4NjZdIEdJQ3YzOiBDUFUyMTogZm91bmQgcmVk
aXN0cmlidXRvciAxYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwODAwMDAwClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyODg3XSBHSUN2MzogQ1BVMjE6IHVzaW5nIGFs
bG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOTkwMDAwClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTIxXSBhcmNoX3RpbWVyOiBFbmFibGluZyBs
b2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDEyOTMzXSBDUFUyMTogQm9vdGVkIHNlY29uZGFyeSBwcm9j
ZXNzb3IgMHgwMDAwMWIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxMzI0OF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTIyClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDEzMjY0XSBHSUN2MzogQ1BVMjI6IGZv
dW5kIHJlZGlzdHJpYnV0b3IgMTUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDY4MDAwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzI4NV0gR0lDdjM6IENQVTIyOiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMDlhMDAwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzMyMV0gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzMzM10gQ1BVMjI6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDE1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2NDhdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUy
MwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxMzY2NV0gR0lDdjM6IENQ
VTIzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFkMDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDA4ODAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM2ODZdIEdJQ3YzOiBD
UFUyMzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDA5YjAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM3MTldIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTM3MzJdIENQVTIzOiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxZDAwMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDUxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVMjQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQwNjVdIEdJ
Q3YzOiBDUFUyNDogZm91bmQgcmVkaXN0cmlidXRvciAxMTAwMDAgcmVnaW9uIDA6MHgwMDAwMTAw
MTAwNTgwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MDg2XSBH
SUN2MzogQ1BVMjQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgw
MDAwOWMwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MTIxXSBh
cmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4
MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0MTM0XSBDUFUyNDog
Qm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTEwMDAwIFsweDQxM2ZkMGMxXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDQ1MF0gRGV0ZWN0ZWQgUElQVCBJ
LWNhY2hlIG9uIENQVTI1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE0
NDY2XSBHSUN2MzogQ1BVMjU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTkwMDAwIHJlZ2lvbiAwOjB4
MDAwMDEwMDEwMDc4MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NDQ4OF0gR0lDdjM6IENQVTI1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4
MDAwMDA4MDAwMDlkMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NDUyMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0
dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNDUzNF0g
Q1BVMjU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE5MDAwMCBbMHg0MTNmZDBj
MV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTQ4NDRdIERldGVjdGVk
IFBJUFQgSS1jYWNoZSBvbiBDUFUyNgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNDg2MF0gR0lDdjM6IENQVTI2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE3MDAwMCByZWdp
b24gMDoweDAwMDAxMDAxMDA3MDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQ4ODJdIEdJQ3YzOiBDUFUyNjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDA5ZTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTQ5MTddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTQ5MzBdIENQVTI2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNzAwMDAgWzB4
NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE1MjQ1XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVMjcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTUyNjNdIEdJQ3YzOiBDUFUyNzogZm91bmQgcmVkaXN0cmlidXRvciAxZjAw
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1Mjg1XSBHSUN2MzogQ1BVMjc6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwOWYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1MzIwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE1MzMyXSBDUFUyNzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWYw
MDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAx
NTY0Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI4ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NjU1XSBHSUN2MzogQ1BVMjg6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgYjAwMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNDAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1Njc3XSBHSUN2MzogQ1BVMjg6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDE1NzE1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDE1NzI4XSBDUFUyODogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMGIwMDAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAxNjA0OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTI5ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE2MDY4XSBHSUN2MzogQ1BVMjk6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjMwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGEwMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjA5MF0gR0lDdjM6IENQVTI5OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGExMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjEyMV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxNjEzNF0gQ1BVMjk6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDIzMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMTY0NDNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQ1Nl0gR0lDdjM6IENQVTMwOiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIGQwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDQ4MDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjQ3OF0gR0lDdjM6IENQVTMwOiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGEyMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjUxM10gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjUyN10gQ1BVMzA6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDBkMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMTY4NDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzMQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxNjg2NV0gR0lDdjM6IENQVTMx
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI1MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDBhODAwMDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY4ODhdIEdJQ3YzOiBDUFUz
MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhMzAwMDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY5MjFdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTY5MzVdIENQVTMxOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3MjQxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVMzIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyNTNdIEdJQ3Yz
OiBDUFUzMjogZm91bmQgcmVkaXN0cmlidXRvciA5MDAwMCByZWdpb24gMDoweDAwMDAxMDAxMDAz
ODAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTcyNzddIEdJQ3Yz
OiBDUFUzMjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBh
NDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczMTJdIGFyY2hf
dGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTczMjddIENQVTMyOiBCb290
ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwOTAwMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NjU2XSBEZXRlY3RlZCBQSVBUIEktY2Fj
aGUgb24gQ1BVMzMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTc2NzVd
IEdJQ3YzOiBDUFUzMzogZm91bmQgcmVkaXN0cmlidXRvciAyMTAwMDAgcmVnaW9uIDA6MHgwMDAw
MTAwMTAwOTgwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3Njk3
XSBHSUN2MzogQ1BVMzM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAw
MDgwMDAwYTUwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NzMy
XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAx
NDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE3NzQ1XSBDUFUz
MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjEwMDAwIFsweDQxM2ZkMGMxXQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODA0OV0gRGV0ZWN0ZWQgUElQ
VCBJLWNhY2hlIG9uIENQVTM0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE4MDY0XSBHSUN2MzogQ1BVMzQ6IGZvdW5kIHJlZGlzdHJpYnV0b3IgZjAwMDAgcmVnaW9uIDA6
MHgwMDAwMTAwMTAwNTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE4MDg4XSBHSUN2MzogQ1BVMzQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBA
MHgwMDAwMDgwMDAwYTYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDE4MTI1XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJy
YXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDE4MTM5
XSBDUFUzNDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGYwMDAwIFsweDQxM2Zk
MGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxODQ1M10gRGV0ZWN0
ZWQgUElQVCBJLWNhY2hlIG9uIENQVTM1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDE4NDczXSBHSUN2MzogQ1BVMzU6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjcwMDAwIHJl
Z2lvbiAwOjB4MDAwMDEwMDEwMGIwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODQ5N10gR0lDdjM6IENQVTM1OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcg
dGFibGUgQDB4MDAwMDA4MDAwMGE3MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODUzMF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3Ig
QVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAxODU0NF0gQ1BVMzU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDI3MDAwMCBb
MHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMTg4NDld
IERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODg2MF0gR0lDdjM6IENQVTM2OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDMw
MDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDIwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODg4NV0gR0lDdjM6IENQVTM2OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBl
bmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE4MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxODkyMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3Vu
ZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAxODkzN10gQ1BVMzY6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDAz
MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MTkyNDldIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzNwpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTI2MV0gR0lDdjM6IENQVTM3OiBmb3VuZCByZWRpc3RyaWJ1
dG9yIDUwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI4MDAwMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTI4NV0gR0lDdjM6IENQVTM3OiB1c2luZyBhbGxvY2F0ZWQg
TFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGE5MDAwMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTMyM10gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29y
a2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjAxOTMzOV0gQ1BVMzc6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4
MDAwMDA1MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMTk2NTBdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTY2MV0gR0lDdjM6IENQVTM4OiBmb3VuZCByZWRp
c3RyaWJ1dG9yIDEwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDE4MDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTY4N10gR0lDdjM6IENQVTM4OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFhMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAxOTcyNV0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAxOTc0MV0gQ1BVMzg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDAxMDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjAwNTFdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFUzOQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDA2M10gR0lDdjM6IENQVTM5OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDcwMDAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDMwMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDA4OF0gR0lDdjM6IENQVTM5OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGFiMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDEyNF0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDE0MF0gQ1BVMzk6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDA3MDAwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjA0NDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0MApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMDQ2Ml0gR0lDdjM6IENQVTQw
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEyMDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA1ZTAwMDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA0ODVdIEdJQ3YzOiBDUFU0
MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBhYzAwMDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1MThdIGFyY2hfdGltZXI6
IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA1MzNdIENQVTQwOiBCb290ZWQgc2Vj
b25kYXJ5IHByb2Nlc3NvciAweDAwMDAxMjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODQ3XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24g
Q1BVNDEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjA4NjRdIEdJQ3Yz
OiBDUFU0MTogZm91bmQgcmVkaXN0cmlidXRvciAxYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAw
N2UwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwODg4XSBHSUN2
MzogQ1BVNDE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAw
YWQwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwOTIwXSBhcmNo
X3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIwOTM0XSBDUFU0MTogQm9v
dGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMWEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTI0OF0gRGV0ZWN0ZWQgUElQVCBJLWNh
Y2hlIG9uIENQVTQyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIxMjYz
XSBHSUN2MzogQ1BVNDI6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMTQwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDY2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTI4
N10gR0lDdjM6IENQVTQyOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGFlMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTMy
Ml0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMTMzNl0gQ1BV
NDI6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDE0MDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE2NTBdIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU0MwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAyMTY2N10gR0lDdjM6IENQVTQzOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDFjMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDA4NjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjE2OTFdIEdJQ3YzOiBDUFU0MzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBhZjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjE3MjRdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjE3
MzldIENQVTQzOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxYzAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIyMDUzXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMjIwNjddIEdJQ3YzOiBDUFU0NDogZm91bmQgcmVkaXN0cmlidXRvciAxMDAxMDAg
cmVnaW9uIDA6MHgwMDAwMTAwMTAwNTYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIyMDkzXSBHSUN2MzogQ1BVNDQ6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGlu
ZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIyMTI3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZv
ciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDIyMTQzXSBDUFU0NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTAwMTAw
IFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjQ1
Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDIyNDY4XSBHSUN2MzogQ1BVNDU6IGZvdW5kIHJlZGlzdHJpYnV0b3Ig
MTgwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDc2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMjQ5Ml0gR0lDdjM6IENQVTQ1OiB1c2luZyBhbGxvY2F0ZWQgTFBJ
IHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGIxMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMjUyNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fy
b3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyMjUzOV0gQ1BVNDU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAw
MDE4MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMjI4NTRdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU0NgpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyMjg3MF0gR0lDdjM6IENQVTQ2OiBmb3VuZCByZWRpc3Ry
aWJ1dG9yIDE2MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA2ZTAwMDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI4OTVdIEdJQ3YzOiBDUFU0NjogdXNpbmcgYWxsb2Nh
dGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiMjAwMDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjI5MzFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2Fs
IHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjI5NDddIENQVTQ2OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3Nv
ciAweDAwMDAxNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDIzMjUzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNDcKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjMyNzFdIEdJQ3YzOiBDUFU0NzogZm91bmQg
cmVkaXN0cmlidXRvciAxZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGUwMDAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMjk2XSBHSUN2MzogQ1BVNDc6IHVzaW5n
IGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjMwMDAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzMxXSBhcmNoX3RpbWVyOiBFbmFibGlu
ZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzMzQ2XSBDUFU0NzogQm9vdGVkIHNlY29uZGFyeSBw
cm9jZXNzb3IgMHgwMDAwMWUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjAyMzY1MF0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTQ4ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjYyXSBHSUN2MzogQ1BVNDg6
IGZvdW5kIHJlZGlzdHJpYnV0b3IgYTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2UwMDAwClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNjg4XSBHSUN2MzogQ1BVNDg6
IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjQwMDAwClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzI1XSBhcmNoX3RpbWVyOiBF
bmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDIzNzQwXSBDUFU0ODogQm9vdGVkIHNlY29u
ZGFyeSBwcm9jZXNzb3IgMHgwMDAwMGEwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDA1M10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQ
VTQ5ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI0MDcyXSBHSUN2Mzog
Q1BVNDk6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMjIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDll
MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDA5N10gR0lDdjM6
IENQVTQ5OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGI1
MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDEzMl0gYXJjaF90
aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDE0OF0gQ1BVNDk6IEJvb3Rl
ZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDIyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ0NDldIERldGVjdGVkIFBJUFQgSS1jYWNo
ZSBvbiBDUFU1MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ2Ml0g
R0lDdjM6IENQVTUwOiBmb3VuZCByZWRpc3RyaWJ1dG9yIGMwMTAwIHJlZ2lvbiAwOjB4MDAwMDEw
MDEwMDQ2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDQ4OF0g
R0lDdjM6IENQVTUwOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4
MDAwMGI2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDUyM10g
YXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQx
ODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNDUzOV0gQ1BVNTA6
IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBjMDEwMCBbMHg0MTNmZDBjMV0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ4NTVdIERldGVjdGVkIFBJUFQg
SS1jYWNoZSBvbiBDUFU1MQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NDg3NF0gR0lDdjM6IENQVTUxOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI0MDEwMCByZWdpb24gMDow
eDAwMDAxMDAxMDBhNjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ4OTldIEdJQ3YzOiBDUFU1MTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAw
eDAwMDAwODAwMDBiNzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjQ5MzJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJh
dHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjQ5NDhd
IENQVTUxOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNDAxMDAgWzB4NDEzZmQw
YzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1MjUyXSBEZXRlY3Rl
ZCBQSVBUIEktY2FjaGUgb24gQ1BVNTIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUyNjRdIEdJQ3YzOiBDUFU1MjogZm91bmQgcmVkaXN0cmlidXRvciA4MDEwMCByZWdp
b24gMDoweDAwMDAxMDAxMDAzNjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUyOTBdIEdJQ3YzOiBDUFU1MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRh
YmxlIEAweDAwMDAwODAwMDBiODAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjUzMjhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFS
TSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4w
MjUzNDRdIENQVTUyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwODAxMDAgWzB4
NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI1NjU4XSBE
ZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNTMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjU2NzddIEdJQ3YzOiBDUFU1MzogZm91bmQgcmVkaXN0cmlidXRvciAyMDAx
MDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOTYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1NzAyXSBHSUN2MzogQ1BVNTM6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVu
ZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYjkwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI1NzM3XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5k
IGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuMDI1NzUzXSBDUFU1MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMjAw
MTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
NjA1MV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU0ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI2MDY1XSBHSUN2MzogQ1BVNTQ6IGZvdW5kIHJlZGlzdHJpYnV0
b3IgZTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNGUwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI2MDkzXSBHSUN2MzogQ1BVNTQ6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYmEwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI2MTMwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI2MTQ3XSBDUFU1NDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMGUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyNjQ1N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTU1ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI2NDc3XSBHSUN2MzogQ1BVNTU6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMjYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMGFlMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjUwNF0gR0lDdjM6IENQVTU1OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJiMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjU0MF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAyNjU1N10gQ1BVNTU6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDI2MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjY4NTVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NgpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg2Nl0gR0lDdjM6IENQVTU2OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDIwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDFlMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjg5Ml0gR0lDdjM6IENQVTU2OiB1c2lu
ZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJjMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjkzMF0gYXJjaF90aW1lcjogRW5hYmxp
bmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNjk0OF0gQ1BVNTY6IEJvb3RlZCBzZWNvbmRhcnkg
cHJvY2Vzc29yIDB4MDAwMDAyMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjcyNDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU1NwpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzI1N10gR0lDdjM6IENQVTU3
OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDQwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDI2MDAwMApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzI4NF0gR0lDdjM6IENQVTU3
OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGJkMDAwMApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMyMV0gYXJjaF90aW1lcjog
RW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzMzN10gQ1BVNTc6IEJvb3RlZCBzZWNv
bmRhcnkgcHJvY2Vzc29yIDB4MDAwMDA0MDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2NDddIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBD
UFU1OApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyNzY1N10gR0lDdjM6
IENQVTU4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDAxNjAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc2ODVdIEdJQ3YzOiBD
UFU1ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBiZTAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3MjNdIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjc3NDFdIENQVTU4OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMDAxMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4MDQ5XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNTkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwNjFdIEdJ
Q3YzOiBDUFU1OTogZm91bmQgcmVkaXN0cmlidXRvciA2MDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAyZTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgwODhdIEdJ
Q3YzOiBDUFU1OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBiZjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxMjVdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjgxNDNdIENQVTU5OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNjAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NDUyXSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNjAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjg0
NjZdIEdJQ3YzOiBDUFU2MDogZm91bmQgcmVkaXN0cmlidXRvciAxMzAxMDAgcmVnaW9uIDA6MHgw
MDAwMTAwMTAwNjIwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
NDkzXSBHSUN2MzogQ1BVNjA6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgw
MDAwMDgwMDAwYzAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4
NTMwXSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1
bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI4NTQ4XSBD
UFU2MDogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTMwMTAwIFsweDQxM2ZkMGMx
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAyODg1NV0gRGV0ZWN0ZWQg
UElQVCBJLWNhY2hlIG9uIENQVTYxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDI4ODcyXSBHSUN2MzogQ1BVNjE6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWIwMTAwIHJlZ2lv
biAwOjB4MDAwMDEwMDEwMDgyMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyODg5OF0gR0lDdjM6IENQVTYxOiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFi
bGUgQDB4MDAwMDA4MDAwMGMxMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAyODkzMl0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJN
IGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAy
ODk0OV0gQ1BVNjE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFiMDEwMCBbMHg0
MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMjkyNTNdIERl
dGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2MgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjAyOTI2OV0gR0lDdjM6IENQVTYyOiBmb3VuZCByZWRpc3RyaWJ1dG9yIDE1MDEw
MCByZWdpb24gMDoweDAwMDAxMDAxMDA2YTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjkyOTZdIEdJQ3YzOiBDUFU2MjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5k
aW5nIHRhYmxlIEAweDAwMDAwODAwMDBjMjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMjkzMzFdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQg
Zm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC4wMjkzNDldIENQVTYyOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAxNTAx
MDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDI5
NjUxXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjMKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC4wMjk2NjhdIEdJQ3YzOiBDUFU2MzogZm91bmQgcmVkaXN0cmlidXRv
ciAxZDAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwOGEwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI5Njk2XSBHSUN2MzogQ1BVNjM6IHVzaW5nIGFsbG9jYXRlZCBM
UEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzMwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDI5NzI5XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3Jr
YXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDI5NzQ3XSBDUFU2MzogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgw
MDAwMWQwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjAzMDA1Nl0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTY0ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwMDcxXSBHSUN2MzogQ1BVNjQ6IGZvdW5kIHJlZGlz
dHJpYnV0b3IgMTEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDVhMDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDA5OV0gR0lDdjM6IENQVTY0OiB1c2luZyBhbGxv
Y2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGM0MDAwMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDEzNF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9j
YWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjAzMDE1MV0gQ1BVNjQ6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz
c29yIDB4MDAwMDExMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC4wMzA0NDVdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU2NQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMDQ2Ml0gR0lDdjM6IENQVTY1OiBmb3Vu
ZCByZWRpc3RyaWJ1dG9yIDE5MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDA3YTAwMDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA0OTBdIEdJQ3YzOiBDUFU2NTogdXNp
bmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjNTAwMDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1MjRdIGFyY2hfdGltZXI6IEVuYWJs
aW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA1MzldIENQVTY1OiBCb290ZWQgc2Vjb25kYXJ5
IHByb2Nlc3NvciAweDAwMDAxOTAxMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDMwODQ2XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNjYK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzA4NjNdIEdJQ3YzOiBDUFU2
NjogZm91bmQgcmVkaXN0cmlidXRvciAxNzAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwNzIwMDAw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwODkxXSBHSUN2MzogQ1BV
NjY6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwYzYwMDAw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwOTI2XSBhcmNoX3RpbWVy
OiBFbmFibGluZyBsb2NhbCB3b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMwOTQyXSBDUFU2NjogQm9vdGVkIHNl
Y29uZGFyeSBwcm9jZXNzb3IgMHgwMDAwMTcwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI1Ml0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9u
IENQVTY3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMxMjcwXSBHSUN2
MzogQ1BVNjc6IGZvdW5kIHJlZGlzdHJpYnV0b3IgMWYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEw
MDkyMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTI5OF0gR0lD
djM6IENQVTY3OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAw
MGM3MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTMzM10gYXJj
aF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTM1MF0gQ1BVNjc6IEJv
b3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDFmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzE2NTBdIERldGVjdGVkIFBJUFQgSS1j
YWNoZSBvbiBDUFU2OApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTY2
M10gR0lDdjM6IENQVTY4OiBmb3VuZCByZWRpc3RyaWJ1dG9yIGIwMTAwIHJlZ2lvbiAwOjB4MDAw
MDEwMDEwMDQyMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTY5
MV0gR0lDdjM6IENQVTY4OiB1c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAw
MDA4MDAwMGM4MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTcy
OF0gYXJjaF90aW1lcjogRW5hYmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0g
MTQxODA0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMTc0Nl0gQ1BV
Njg6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vzc29yIDB4MDAwMDBiMDEwMCBbMHg0MTNmZDBjMV0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIwNTldIERldGVjdGVkIFBJ
UFQgSS1jYWNoZSBvbiBDUFU2OQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMjA3OF0gR0lDdjM6IENQVTY5OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDIzMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDBhMjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzIxMDZdIEdJQ3YzOiBDUFU2OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBjOTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzIxMzddIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzIx
NTRdIENQVTY5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyMzAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyNDQ3XSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzI0NjBdIEdJQ3YzOiBDUFU3MDogZm91bmQgcmVkaXN0cmlidXRvciBkMDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDA0YTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzI0OTBdIEdJQ3YzOiBDUFU3MDogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBjYTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzI1MjVdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzI1NDJdIENQVTcwOiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwZDAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMyODU3
XSBEZXRlY3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzI4NzZdIEdJQ3YzOiBDUFU3MTogZm91bmQgcmVkaXN0cmlidXRvciAy
NTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwYWEwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyOTA1XSBHSUN2MzogQ1BVNzE6IHVzaW5nIGFsbG9jYXRlZCBMUEkg
cGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2IwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMyOTM4XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3b3JrYXJv
dW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuMDMyOTU1XSBDUFU3MTogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3IgMHgwMDAw
MjUwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjAzMzI1N10gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTcyClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjY5XSBHSUN2MzogQ1BVNzI6IGZvdW5kIHJlZGlzdHJp
YnV0b3IgOTAxMDAgcmVnaW9uIDA6MHgwMDAwMTAwMTAwM2EwMDAwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMjk4XSBHSUN2MzogQ1BVNzI6IHVzaW5nIGFsbG9jYXRl
ZCBMUEkgcGVuZGluZyB0YWJsZSBAMHgwMDAwMDgwMDAwY2MwMDAwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzMzM2XSBhcmNoX3RpbWVyOiBFbmFibGluZyBsb2NhbCB3
b3JrYXJvdW5kIGZvciBBUk0gZXJyYXR1bSAxNDE4MDQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDMzMzU0XSBDUFU3MjogQm9vdGVkIHNlY29uZGFyeSBwcm9jZXNzb3Ig
MHgwMDAwMDkwMTAwIFsweDQxM2ZkMGMxXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjAzMzY1OV0gRGV0ZWN0ZWQgUElQVCBJLWNhY2hlIG9uIENQVTczClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDMzNjc3XSBHSUN2MzogQ1BVNzM6IGZvdW5kIHJl
ZGlzdHJpYnV0b3IgMjEwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDlhMDAwMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzcwNl0gR0lDdjM6IENQVTczOiB1c2luZyBh
bGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNkMDAwMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzczOV0gYXJjaF90aW1lcjogRW5hYmxpbmcg
bG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzMzc1N10gQ1BVNzM6IEJvb3RlZCBzZWNvbmRhcnkgcHJv
Y2Vzc29yIDB4MDAwMDIxMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC4wMzQwNDZdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3NApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDA2MV0gR0lDdjM6IENQVTc0OiBm
b3VuZCByZWRpc3RyaWJ1dG9yIGYwMTAwIHJlZ2lvbiAwOjB4MDAwMDEwMDEwMDUyMDAwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDA5MF0gR0lDdjM6IENQVTc0OiB1
c2luZyBhbGxvY2F0ZWQgTFBJIHBlbmRpbmcgdGFibGUgQDB4MDAwMDA4MDAwMGNlMDAwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDEyN10gYXJjaF90aW1lcjogRW5h
YmxpbmcgbG9jYWwgd29ya2Fyb3VuZCBmb3IgQVJNIGVycmF0dW0gMTQxODA0MApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDE0NV0gQ1BVNzQ6IEJvb3RlZCBzZWNvbmRh
cnkgcHJvY2Vzc29yIDB4MDAwMDBmMDEwMCBbMHg0MTNmZDBjMV0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ0NTNdIERldGVjdGVkIFBJUFQgSS1jYWNoZSBvbiBDUFU3
NQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNDQ3NF0gR0lDdjM6IENQ
VTc1OiBmb3VuZCByZWRpc3RyaWJ1dG9yIDI3MDEwMCByZWdpb24gMDoweDAwMDAxMDAxMDBiMjAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1MDNdIEdJQ3YzOiBD
UFU3NTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAwMDBjZjAw
MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1MzddIGFyY2hfdGlt
ZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgwNDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ1NTRdIENQVTc1OiBCb290ZWQg
c2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAyNzAxMDAgWzB4NDEzZmQwYzFdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM0ODUzXSBEZXRlY3RlZCBQSVBUIEktY2FjaGUg
b24gQ1BVNzYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4NjRdIEdJ
Q3YzOiBDUFU3NjogZm91bmQgcmVkaXN0cmlidXRvciAzMDEwMCByZWdpb24gMDoweDAwMDAxMDAx
MDAyMjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ4OTRdIEdJ
Q3YzOiBDUFU3NjogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAwMDAwODAw
MDBkMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5MzJdIGFy
Y2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVtIDE0MTgw
NDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzQ5NTBdIENQVTc2OiBC
b290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMzAxMDAgWzB4NDEzZmQwYzFdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1MjU5XSBEZXRlY3RlZCBQSVBUIEkt
Y2FjaGUgb24gQ1BVNzcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUy
NzBdIEdJQ3YzOiBDUFU3NzogZm91bmQgcmVkaXN0cmlidXRvciA1MDEwMCByZWdpb24gMDoweDAw
MDAxMDAxMDAyYTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUz
MDBdIEdJQ3YzOiBDUFU3NzogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxlIEAweDAw
MDAwODAwMDBkMTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUz
MzhdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBlcnJhdHVt
IDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzUzNTddIENQ
VTc3OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNTAxMDAgWzB4NDEzZmQwYzFd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM1NjUxXSBEZXRlY3RlZCBQ
SVBUIEktY2FjaGUgb24gQ1BVNzgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU2NjFdIEdJQ3YzOiBDUFU3ODogZm91bmQgcmVkaXN0cmlidXRvciAxMDEwMCByZWdpb24g
MDoweDAwMDAxMDAxMDAxYTAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU2OTJdIEdJQ3YzOiBDUFU3ODogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5nIHRhYmxl
IEAweDAwMDAwODAwMDBkMjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzU3MzBdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9yIEFSTSBl
cnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzU3
NDldIENQVTc4OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwMTAxMDAgWzB4NDEz
ZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MDUxXSBEZXRl
Y3RlZCBQSVBUIEktY2FjaGUgb24gQ1BVNzkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzYwNjNdIEdJQ3YzOiBDUFU3OTogZm91bmQgcmVkaXN0cmlidXRvciA3MDEwMCBy
ZWdpb24gMDoweDAwMDAxMDAxMDAzMjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzYwOTNdIEdJQ3YzOiBDUFU3OTogdXNpbmcgYWxsb2NhdGVkIExQSSBwZW5kaW5n
IHRhYmxlIEAweDAwMDAwODAwMDBkMzAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC4wMzYxMzJdIGFyY2hfdGltZXI6IEVuYWJsaW5nIGxvY2FsIHdvcmthcm91bmQgZm9y
IEFSTSBlcnJhdHVtIDE0MTgwNDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC4wMzYxNTFdIENQVTc5OiBCb290ZWQgc2Vjb25kYXJ5IHByb2Nlc3NvciAweDAwMDAwNzAxMDAg
WzB4NDEzZmQwYzFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2MjE0
XSBzbXA6IEJyb3VnaHQgdXAgMSBub2RlLCA4MCBDUFVzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuMDM2MjM1XSBTTVA6IFRvdGFsIG9mIDgwIHByb2Nlc3NvcnMgYWN0aXZh
dGVkLgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIzNl0gQ1BVIGZl
YXR1cmVzOiBkZXRlY3RlZDogUHJpdmlsZWdlZCBBY2Nlc3MgTmV2ZXIKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyMzddIENQVSBmZWF0dXJlczogZGV0ZWN0ZWQ6IExT
RSBhdG9taWMgaW5zdHJ1Y3Rpb25zClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuMDM2MjM3XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBVc2VyIEFjY2VzcyBPdmVycmlkZQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjIzOF0gQ1BVIGZlYXR1cmVz
OiBkZXRlY3RlZDogMzItYml0IEVMMCBTdXBwb3J0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuMDM2MjM5XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBDb21tb24gbm90IFBy
aXZhdGUgdHJhbnNsYXRpb25zClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
MDM2MjM5XSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBEYXRhIGNhY2hlIGNsZWFuIHRvIFBvaW50
IG9mIFBlcnNpc3RlbmNlClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuMDM2
MjQwXSBDUFUgZmVhdHVyZXM6IGRldGVjdGVkOiBSQVMgRXh0ZW5zaW9uIFN1cHBvcnQKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyNDFdIENQVSBmZWF0dXJlczogZGV0
ZWN0ZWQ6IERhdGEgY2FjaGUgY2xlYW4gdG8gdGhlIFBvVSBub3QgcmVxdWlyZWQgZm9yIEkvRCBj
b2hlcmVuY2UKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC4wMzYyNDFdIENQ
VSBmZWF0dXJlczogZGV0ZWN0ZWQ6IENSQzMyIGluc3RydWN0aW9ucwpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjAzNjI0Ml0gQ1BVIGZlYXR1cmVzOiBkZXRlY3RlZDogU3Bl
Y3VsYXRpdmUgU3RvcmUgQnlwYXNzaW5nIFNhZmUgKFNTQlMpClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuMDU4NzEzXSBDUFU6IEFsbCBDUFUocykgc3RhcnRlZCBhdCBFTDIK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjA5NTRdIG5vZGUgMCBkZWZl
cnJlZCBwYWdlcyBpbml0aWFsaXNlZCBpbiA1MDBtcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU2MTA5OV0gZGV2dG1wZnM6IGluaXRpYWxpemVkClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTYxNjg0XSBSZWdpc3RlcmVkIGNwMTVfYmFycmllciBl
bXVsYXRpb24gaGFuZGxlcgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2
MTcwNl0gUmVnaXN0ZXJlZCBzZXRlbmQgZW11bGF0aW9uIGhhbmRsZXIKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjE3MDhdIEtBU0xSIGVuYWJsZWQKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjE3NjhdIGNsb2Nrc291cmNlOiBqaWZmaWVzOiBt
YXNrOiAweGZmZmZmZmZmIG1heF9jeWNsZXM6IDB4ZmZmZmZmZmYsIG1heF9pZGxlX25zOiA3NjQ1
MDQxNzg1MTAwMDAwIG5zClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTYx
ODIxXSBmdXRleCBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOSwgMjA5NzE1MiBi
eXRlcywgbGluZWFyKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MjI2
MV0gcGluY3RybCBjb3JlOiBpbml0aWFsaXplZCBwaW5jdHJsIHN1YnN5c3RlbQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MjQyM10gU01CSU9TIDMuMy4wIHByZXNlbnQu
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTYyNDI2XSBETUk6IEdJR0FC
WVRFIFIxNTItUDMwLTAwL01QMzItQVIxLTAwLCBCSU9TIEYxMyAoU0NQOiAxLjUuMjAyMTA0MjYp
IDA1LzIxLzIwMjEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjI2NThd
IE5FVDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTYKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC41NjMyNTRdIERNQTogcHJlYWxsb2NhdGVkIDQwOTYgS2lCIEdGUF9L
RVJORUwgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNTYzMzgwXSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBHRlBfS0VS
TkVMfEdGUF9ETUEgcG9vbCBmb3IgYXRvbWljIGFsbG9jYXRpb25zClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNTYzNTA1XSBETUE6IHByZWFsbG9jYXRlZCA0MDk2IEtpQiBH
RlBfS0VSTkVMfEdGUF9ETUEzMiBwb29sIGZvciBhdG9taWMgYWxsb2NhdGlvbnMKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjM1MTJdIGF1ZGl0OiBpbml0aWFsaXppbmcg
bmV0bGluayBzdWJzeXMgKGRpc2FibGVkKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjU2MzU1M10gYXVkaXQ6IHR5cGU9MjAwMCBhdWRpdCgwLjU2MDoxKTogc3RhdGU9aW5p
dGlhbGl6ZWQgYXVkaXRfZW5hYmxlZD0wIHJlcz0xClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTYzNjQyXSB0aGVybWFsX3N5czogUmVnaXN0ZXJlZCB0aGVybWFsIGdvdmVy
bm9yICdmYWlyX3NoYXJlJwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2
MzY0M10gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnYmFuZ19iYW5n
JwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MzY0NF0gdGhlcm1hbF9z
eXM6IFJlZ2lzdGVyZWQgdGhlcm1hbCBnb3Zlcm5vciAnc3RlcF93aXNlJwpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MzY0NV0gdGhlcm1hbF9zeXM6IFJlZ2lzdGVyZWQg
dGhlcm1hbCBnb3Zlcm5vciAndXNlcl9zcGFjZScKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC41NjM2NDVdIHRoZXJtYWxfc3lzOiBSZWdpc3RlcmVkIHRoZXJtYWwgZ292ZXJu
b3IgJ3Bvd2VyX2FsbG9jYXRvcicKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC41NjM3MDBdIGNwdWlkbGU6IHVzaW5nIGdvdmVybm9yIGxhZGRlcgpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MzcwOF0gY3B1aWRsZTogdXNpbmcgZ292ZXJub3IgbWVu
dQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2MzgwM10gRGV0ZWN0ZWQg
MTUgUENDIFN1YnNwYWNlcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjU2
MzgxOF0gUmVnaXN0ZXJpbmcgUENDIGRyaXZlciBhcyBNYWlsYm94IGNvbnRyb2xsZXIKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjM4MzddIGh3LWJyZWFrcG9pbnQ6IGZv
dW5kIDYgYnJlYWtwb2ludCBhbmQgNCB3YXRjaHBvaW50IHJlZ2lzdGVycy4KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjQyMzBdIEFTSUQgYWxsb2NhdG9yIGluaXRpYWxp
c2VkIHdpdGggMzI3NjggZW50cmllcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjU2NDI1NF0gQUNQSTogYnVzIHR5cGUgUENJIHJlZ2lzdGVyZWQKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjQyNTVdIGFjcGlwaHA6IEFDUEkgSG90IFBsdWcgUENJ
IENvbnRyb2xsZXIgRHJpdmVyIHZlcnNpb246IDAuNQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjU2NDI5NF0gU2VyaWFsOiBBTUJBIFBMMDExIFVBUlQgZHJpdmVyClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNTY2Njk2XSBIdWdlVExCIHJlZ2lzdGVy
ZWQgMS4wMCBHaUIgcGFnZSBzaXplLCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC41NjY2OTddIEh1Z2VUTEIgcmVnaXN0ZXJlZCAzMi4w
IE1pQiBwYWdlIHNpemUsIHByZS1hbGxvY2F0ZWQgMCBwYWdlcwpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjU2NjY5OF0gSHVnZVRMQiByZWdpc3RlcmVkIDIuMDAgTWlCIHBh
Z2Ugc2l6ZSwgcHJlLWFsbG9jYXRlZCAwIHBhZ2VzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNTY2Njk4XSBIdWdlVExCIHJlZ2lzdGVyZWQgNjQuMCBLaUIgcGFnZSBzaXpl
LCBwcmUtYWxsb2NhdGVkIDAgcGFnZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC42OTQxNDldIEFDUEk6IEFkZGVkIF9PU0koTW9kdWxlIERldmljZSkKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTQxNTFdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vz
c29yIERldmljZSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42OTQxNTFd
IEFDUEk6IEFkZGVkIF9PU0koMy4wIF9TQ1AgRXh0ZW5zaW9ucykKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC42OTQxNTJdIEFDUEk6IEFkZGVkIF9PU0koUHJvY2Vzc29yIEFn
Z3JlZ2F0b3IgRGV2aWNlKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjY5
NDE1M10gQUNQSTogQWRkZWQgX09TSShMaW51eC1EZWxsLVZpZGVvKQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjY5NDE1NF0gQUNQSTogQWRkZWQgX09TSShMaW51eC1MZW5v
dm8tTlYtSERNSS1BdWRpbykKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC42
OTQxNTVdIEFDUEk6IEFkZGVkIF9PU0koTGludXgtSFBJLUh5YnJpZC1HcmFwaGljcykKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MDExNDNdIEFDUEk6IDIgQUNQSSBBTUwg
dGFibGVzIHN1Y2Nlc3NmdWxseSBhY3F1aXJlZCBhbmQgbG9hZGVkClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNTAzXSBBQ1BJOiBJbnRlcnByZXRlciBlbmFibGVkClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzAyNTA0XSBBQ1BJOiBVc2luZyBH
SUMgZm9yIGludGVycnVwdCByb3V0aW5nClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzAyNTEyXSBBQ1BJOiBNQ0ZHIHRhYmxlIGRldGVjdGVkLCA4IGVudHJpZXMKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTEzNjRdIEFSTUgwMDExOjAwOiB0dHlB
TUEwIGF0IE1NSU8gMHgxMDAwMDI2MDAwMDAgKGlycSA9IDM2LCBiYXNlX2JhdWQgPSAwKSBpcyBh
IFNCU0EKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTE0MDhdIHByaW50
azogY29uc29sZSBbdHR5QU1BMF0gZW5hYmxlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxMjUxOV0gQVJNSDAwMTE6MDE6IHR0eUFNQTEgYXQgTU1JTyAweDEwMDAwMjYy
MDAwMCAoaXJxID0gMzcsIGJhc2VfYmF1ZCA9IDApIGlzIGEgU0JTQQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcxMzU4Nl0gQUNQSTogUENJIFJvb3QgQnJpZGdlIFtQQ0kw
XSAoZG9tYWluIDAwMGMgW2J1cyAwMC1mZl0pClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzEzNTkzXSBhY3BpIFBOUDBBMDg6MDA6IF9PU0M6IE9TIHN1cHBvcnRzIFtFeHRl
bmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJIEhQWC1UeXBlM10KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTM2NjBdIGFjcGkgUE5QMEEwODowMDogX09T
QzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhvdHBsdWcgU0hQQ0hvdHBsdWcgUE1F
IExUUl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTM3MjBdIGFjcGkg
UE5QMEEwODowMDogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtBRVIgUENJZUNhcGFiaWxpdHldClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzEzNzI0XSBhY3BpIFBOUDBBMDg6
MDA6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZm
Zl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmYWQ5ZmFlNDdlYTU4ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MzU0XSBhY3BpIFBOUDBBMDg6MDA6IEVDQU0gYXJl
YSBbbWVtIDB4MzNmZmYwMDAwMDAwLTB4MzNmZmZmZmZmZmZmXSByZXNlcnZlZCBieSBQTlAwQzAy
OjAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0MzYyXSBhY3BpIFBO
UDBBMDg6MDA6IEVDQU0gYXQgW21lbSAweDMzZmZmMDAwMDAwMC0weDMzZmZmZmZmZmZmZl0gZm9y
IFtidXMgMDAtZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0NDI4
XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMGM6MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTQ0MjldIHBjaV9idXMgMDAwYzowMDogcm9vdCBidXMgcmVzb3VyY2Ug
W21lbSAweDQwMDAwMDAwLTB4NGZmZmZmZmYgd2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcxNDQzMF0gcGNpX2J1cyAwMDBjOjAwOiByb290IGJ1cyByZXNvdXJj
ZSBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzNmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ0MzFdIHBjaV9idXMgMDAwYzowMDogcm9vdCBi
dXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTQ0NTFdIHBjaSAwMDBjOjAwOjAwLjA6IFsxZGVmOmUxMDBdIHR5cGUgMDAgY2xhc3Mg
MHgwNjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ1MDFdIHBj
aSAwMDBjOjAwOjAxLjA6IFsxZGVmOmUxMDFdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ1MDldIHBjaSAwMDBjOjAwOjAxLjA6
IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTQ1MjRdIHBjaSAwMDBjOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0NTI1XSBwY2kgMDAwYzowMDowMS4wOiBQTUUj
IHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE0NjAzXSBhY3BpcGhwOiBTbG90IFsxXSByZWdpc3RlcmVkClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0NjE2XSBwY2kgMDAwYzowMTowMC4wOiBbMTAw
MDowMDk3XSB0eXBlIDAwIGNsYXNzIDB4MDEwNzAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE0NjI2XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxMDogW2lvICAweDRm
ZmY4MDAwLTB4NGZmZjgwZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE0NjMzXSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxNDogW21lbSAweDQwMTQwMDAwLTB4NDAx
NGZmZmYgNjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0NjQx
XSBwY2kgMDAwYzowMTowMC4wOiByZWcgMHgxYzogW21lbSAweDQwMTAwMDAwLTB4NDAxM2ZmZmYg
NjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0NjUwXSBwY2kg
MDAwYzowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ2NTRdIHBjaSAwMDBjOjAx
OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MTQ3NDhdIHBjaSAwMDBjOjAxOjAwLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0ODIxXSBwY2lfYnVzIDAwMGM6MDA6
IG9uIE5VTUEgbm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0
ODI0XSBwY2kgMDAwYzowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAw
MGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24g
MTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0ODI3XSBwY2kg
MDAwYzowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDgyOV0gcGNpIDAwMGM6
MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzAwMDAwMDAwMDAwLTB4MzAwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0
ODMwXSBwY2kgMDAwYzowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDgzMV0gcGNpIDAw
MGM6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0ODMyXSBwY2kgMDAwYzowMDow
MS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDgzM10gcGNpIDAwMGM6MDA6MDEuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzE0ODM1XSBwY2kgMDAwYzowMTowMC4wOiBCQVIgNjogYXNzaWdu
ZWQgW21lbSAweDQwMDAwMDAwLTB4NDAwZmZmZmYgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTQ4MzZdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAzOiBhc3NpZ25l
ZCBbbWVtIDB4NDAxMDAwMDAtMHg0MDEzZmZmZiA2NGJpdF0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTQ4NDFdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAxOiBhc3NpZ25l
ZCBbbWVtIDB4NDAxNDAwMDAtMHg0MDE0ZmZmZiA2NGJpdF0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTQ4NDZdIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBubyBzcGFj
ZSBmb3IgW2lvICBzaXplIDB4MDEwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTQ4NDddIHBjaSAwMDBjOjAxOjAwLjA6IEJBUiAwOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDAxMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0
ODQ5XSBwY2kgMDAwYzowMDowMS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0ODUxXSBwY2kgMDAwYzowMDowMS4wOiAgIGJy
aWRnZSB3aW5kb3cgW21lbSAweDQwMDAwMDAwLTB4NDAxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0ODUyXSBwY2kgMDAwYzowMDowMS4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDMwMDAwMDAwMDAwMC0weDMwMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDg1M10gcGNpX2J1cyAwMDBjOjAw
OiBTb21lIFBDSSBkZXZpY2UgcmVzb3VyY2VzIGFyZSB1bmFzc2lnbmVkLCB0cnkgYm9vdGluZyB3
aXRoIHBjaT1yZWFsbG9jClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE0
ODU0XSBwY2lfYnVzIDAwMGM6MDA6IHJlc291cmNlIDQgW21lbSAweDQwMDAwMDAwLTB4NGZmZmZm
ZmYgd2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDg1NV0g
cGNpX2J1cyAwMDBjOjAwOiByZXNvdXJjZSA1IFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzM2ZmZGZm
ZmZmZmYgd2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDg1
Nl0gcGNpX2J1cyAwMDBjOjAxOiByZXNvdXJjZSAxIFttZW0gMHg0MDAwMDAwMC0weDQwMWZmZmZm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDg1N10gcGNpX2J1cyAw
MDBjOjAxOiByZXNvdXJjZSAyIFttZW0gMHgzMDAwMDAwMDAwMDAtMHgzMDAwMDAxZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTQ5NjZdIEFD
UEk6IFBDSSBSb290IEJyaWRnZSBbUENJMV0gKGRvbWFpbiAwMDBkIFtidXMgMDAtZmZdKQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNDk3MF0gYWNwaSBQTlAwQTA4OjAx
OiBfT1NDOiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRz
IE1TSSBIUFgtVHlwZTNdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1
MDM1XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BD
SWVIb3RwbHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE1MDk0XSBhY3BpIFBOUDBBMDg6MDE6IF9PU0M6IE9TIG5vdyBjb250cm9s
cyBbQUVSIFBDSWVDYXBhYmlsaXR5XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNTA5Nl0gYWNwaSBQTlAwQTA4OjAxOiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgz
N2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmFk
OWZhZTQ3ZWE1OApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTcyM10g
YWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGFyZWEgW21lbSAweDM3ZmZmMDAwMDAwMC0weDM3ZmZmZmZm
ZmZmZl0gcmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNTcyOV0gYWNwaSBQTlAwQTA4OjAxOiBFQ0FNIGF0IFttZW0gMHgzN2ZmZjAw
MDAwMDAtMHgzN2ZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNTc5MF0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDBkOjAw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1NzkxXSBwY2lfYnVzIDAw
MGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHg1MDAwMDAwMC0weDVmZmZmZmZmIHdpbmRv
d10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU3OTJdIHBjaV9idXMg
MDAwZDowMDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDM0MDAwMDAwMDAwMC0weDM3ZmZkZmZm
ZmZmZiB3aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1Nzkz
XSBwY2lfYnVzIDAwMGQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODEwXSBwY2kgMDAwZDowMDowMC4wOiBb
MWRlZjplMTAwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE1ODQ5XSBwY2kgMDAwZDowMDowMS4wOiBbMWRlZjplMTAxXSB0eXBl
IDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE1ODU3XSBwY2kgMDAwZDowMDowMS4wOiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE1ODczXSBwY2kgMDAwZDowMDowMS4wOiBz
dXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTg3
NF0gcGNpIDAwMGQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNTkyNV0gcGNpIDAwMGQ6MDA6MDIu
MDogWzFkZWY6ZTEwMl0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNTk0NV0gcGNpIDAwMGQ6MDA6MDIuMDogc3VwcG9ydHMgRDEg
RDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5NDZdIHBjaSAwMDBk
OjAwOjAyLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTU5OTRdIHBjaSAwMDBkOjAwOjAzLjA6IFsxZGVmOmUx
MDNdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MTYwMTRdIHBjaSAwMDBkOjAwOjAzLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MDE1XSBwY2kgMDAwZDowMDowMy4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2MDYyXSBwY2kgMDAwZDowMDowNC4wOiBbMWRlZjplMTA0XSB0eXBlIDAx
IGNsYXNzIDB4MDYwNDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2
MDgyXSBwY2kgMDAwZDowMDowNC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjA4M10gcGNpIDAwMGQ6MDA6MDQuMDogUE1FIyBzdXBwb3J0
ZWQgZnJvbSBEMCBEMSBEM2hvdApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcxNjE1MV0gYWNwaXBocDogU2xvdCBbMS0yXSByZWdpc3RlcmVkClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2MTY3XSBwY2kgMDAwZDowMTowMC4wOiBbODA4NjowYTU0
XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE2MTc5XSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMzEwMDAw
LTB4NTAzMTNmZmYgNjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2MjAyXSBwY2kgMDAwZDowMTowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMzAwMDAwLTB4NTAz
MGZmZmYgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYyMDZd
IHBjaSAwMDBkOjAxOjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzNjddIGFjcGlwaHA6IFNsb3QgWzJdIHJlZ2lzdGVy
ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzODJdIHBjaSAwMDBk
OjAyOjAwLjA6IFs4MDg2OjBhNTRdIHR5cGUgMDAgY2xhc3MgMHgwMTA4MDIKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTYzOTNdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAw
eDEwOiBbbWVtIDB4NTAyMTAwMDAtMHg1MDIxM2ZmZiA2NGJpdF0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTY0MTRdIHBjaSAwMDBkOjAyOjAwLjA6IHJlZyAweDMwOiBb
bWVtIDB4NTAyMDAwMDAtMHg1MDIwZmZmZiBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNjQxN10gcGNpIDAwMGQ6MDI6MDAuMDogZW5hYmxpbmcgRXh0ZW5kZWQg
VGFncwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU1Nl0gYWNwaXBo
cDogU2xvdCBbM10gcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcxNjU3MV0gcGNpIDAwMGQ6MDM6MDAuMDogWzgwODY6MGE1NF0gdHlwZSAwMCBjbGFzcyAw
eDAxMDgwMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjU4Ml0gcGNp
IDAwMGQ6MDM6MDAuMDogcmVnIDB4MTA6IFttZW0gMHg1MDExMDAwMC0weDUwMTEzZmZmIDY0Yml0
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjYwM10gcGNpIDAwMGQ6
MDM6MDAuMDogcmVnIDB4MzA6IFttZW0gMHg1MDEwMDAwMC0weDUwMTBmZmZmIHByZWZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NjA3XSBwY2kgMDAwZDowMzowMC4w
OiBlbmFibGluZyBFeHRlbmRlZCBUYWdzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE2NzUwXSBhY3BpcGhwOiBTbG90IFs0XSByZWdpc3RlcmVkClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2NzY1XSBwY2kgMDAwZDowNDowMC4wOiBbODA4Njow
YTU0XSB0eXBlIDAwIGNsYXNzIDB4MDEwODAyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2Nzc1XSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgxMDogW21lbSAweDUwMDEw
MDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzE2Nzk2XSBwY2kgMDAwZDowNDowMC4wOiByZWcgMHgzMDogW21lbSAweDUwMDAwMDAwLTB4
NTAwMGZmZmYgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY4
MDBdIHBjaSAwMDBkOjA0OjAwLjA6IGVuYWJsaW5nIEV4dGVuZGVkIFRhZ3MKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5MjNdIHBjaV9idXMgMDAwZDowMDogb24gTlVN
QSBub2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5MjVdIHBj
aSAwMDBkOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVz
IDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2OTI3XSBwY2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTI4XSBw
Y2kgMDAwZDowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZm
XSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjkyOV0gcGNpIDAwMGQ6MDA6MDIuMDogYnJp
ZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDEwMDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5MzBdIHBjaSAwMDBkOjAw
OjAyLjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJl
Zl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5MzFdIHBjaSAwMDBkOjAwOjAyLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAxZmZmZmZdIHRvIFtidXMgMDJdIGFkZF9z
aXplIDEwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzE2OTMzXSBwY2kgMDAwZDowMDowMy4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgx
MDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcxNjkzNF0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAzXSBhZGRf
c2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNjkzNV0gcGNpIDAwMGQ6MDA6MDMuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MDAxMDAwMDAtMHgwMDFmZmZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAwMDAwIGFkZF9hbGln
biAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5MzddIHBj
aSAwMDBkOjAwOjA0LjA6IGJyaWRnZSB3aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVz
IDA0XSBhZGRfc2l6ZSAxMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2OTM4XSBwY2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0w
eDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxp
Z24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTM5XSBw
Y2kgMDAwZDowMDowNC4wOiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZm
XSB0byBbYnVzIDA0XSBhZGRfc2l6ZSAxMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk0M10gcGNpIDAwMGQ6MDA6MDEuMDogQkFS
IDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAwMDAwMDAtMHg1MDFmZmZmZl0KU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NDRdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxNTog
YXNzaWduZWQgW21lbSAweDM0MDAwMDAwMDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk0NV0gcGNpIDAwMGQ6MDA6
MDIuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTAyMDAwMDAtMHg1MDNmZmZmZl0KU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NDZdIHBjaSAwMDBkOjAwOjAyLjA6
IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk0N10gcGNp
IDAwMGQ6MDA6MDMuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA0MDAwMDAtMHg1MDVmZmZm
Zl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NDhdIHBjaSAwMDBk
OjAwOjAzLjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcx
Njk0OV0gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDE0OiBhc3NpZ25lZCBbbWVtIDB4NTA2MDAwMDAt
MHg1MDdmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NTBd
IHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxNTogYXNzaWduZWQgW21lbSAweDM0MDAwMDYwMDAwMC0w
eDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNjk1MV0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lv
ICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5
NTJdIHBjaSAwMDBkOjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUg
MHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk1M10gcGNp
IDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NTRdIHBjaSAwMDBkOjAwOjAy
LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk1NV0gcGNpIDAwMGQ6MDA6MDMuMDogQkFS
IDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MTY5NTZdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFpbGVk
IHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxNjk1N10gcGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3Ig
W2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTY5NThdIHBjaSAwMDBkOjAwOjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk2MF0g
cGNpIDAwMGQ6MDA6MDQuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NjFdIHBjaSAwMDBkOjAw
OjA0LjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk2Ml0gcGNpIDAwMGQ6MDA6MDMuMDog
QkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NjJdIHBjaSAwMDBkOjAwOjAzLjA6IEJBUiAxMzogZmFp
bGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNjk2M10gcGNpIDAwMGQ6MDA6MDIuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTY5NjRdIHBjaSAwMDBkOjAwOjAyLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk2
NV0gcGNpIDAwMGQ6MDA6MDEuMDogQkFSIDEzOiBubyBzcGFjZSBmb3IgW2lvICBzaXplIDB4MTAw
MF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTY5NjZdIHBjaSAwMDBk
OjAwOjAxLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgxMDAwXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk2N10gcGNpIDAwMGQ6MDE6MDAu
MDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDAwMDAwMC0weDUwMDBmZmZmIHByZWZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTY4XSBwY2kgMDAwZDowMTowMC4w
OiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwMDEwMDAwLTB4NTAwMTNmZmYgNjRiaXRdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTczXSBwY2kgMDAwZDowMDowMS4w
OiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzE2OTc1XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUw
MDAwMDAwLTB4NTAxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE2OTc2XSBwY2kgMDAwZDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDAw
MDAwMC0weDM0MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcxNjk3OF0gcGNpIDAwMGQ6MDI6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFtt
ZW0gMHg1MDIwMDAwMC0weDUwMjBmZmZmIHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2OTc5XSBwY2kgMDAwZDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21l
bSAweDUwMjEwMDAwLTB4NTAyMTNmZmYgNjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzE2OTgzXSBwY2kgMDAwZDowMDowMi4wOiBQQ0kgYnJpZGdlIHRvIFtidXMg
MDJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTg1XSBwY2kgMDAw
ZDowMDowMi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwMjAwMDAwLTB4NTAzZmZmZmZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTg2XSBwY2kgMDAwZDowMDow
Mi4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZmZiA2
NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk4OF0g
cGNpIDAwMGQ6MDM6MDAuMDogQkFSIDY6IGFzc2lnbmVkIFttZW0gMHg1MDQwMDAwMC0weDUwNDBm
ZmZmIHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTg4XSBw
Y2kgMDAwZDowMzowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDUwNDEwMDAwLTB4NTA0MTNm
ZmYgNjRiaXRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTkzXSBw
Y2kgMDAwZDowMDowMy4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDNdClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTk0XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3
aW5kb3cgW21lbSAweDUwNDAwMDAwLTB4NTA1ZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzE2OTk2XSBwY2kgMDAwZDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cg
W21lbSAweDM0MDAwMDQwMDAwMC0weDM0MDAwMDVmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNjk5OF0gcGNpIDAwMGQ6MDQ6MDAuMDogQkFS
IDY6IGFzc2lnbmVkIFttZW0gMHg1MDYwMDAwMC0weDUwNjBmZmZmIHByZWZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE2OTk5XSBwY2kgMDAwZDowNDowMC4wOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDUwNjEwMDAwLTB4NTA2MTNmZmYgNjRiaXRdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MDAzXSBwY2kgMDAwZDowMDowNC4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDRdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE3MDA0XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDUwNjAwMDAw
LTB4NTA3ZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MDA2
XSBwY2kgMDAwZDowMDowNC4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM0MDAwMDYwMDAwMC0w
eDM0MDAwMDdmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzAwN10gcGNpX2J1cyAwMDBkOjAwOiByZXNvdXJjZSA0IFttZW0gMHg1MDAwMDAw
MC0weDVmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MTcwMDhdIHBjaV9idXMgMDAwZDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzQwMDAwMDAwMDAw
LTB4MzdmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTcwMDldIHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMSBbbWVtIDB4NTAwMDAwMDAt
MHg1MDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcwMTBd
IHBjaV9idXMgMDAwZDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwMDAwMDAwLTB4MzQwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzE3MDExXSBwY2lfYnVzIDAwMGQ6MDI6IHJlc291cmNlIDEgW21lbSAweDUwMjAwMDAwLTB4NTAz
ZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MDEyXSBwY2lf
YnVzIDAwMGQ6MDI6IHJlc291cmNlIDIgW21lbSAweDM0MDAwMDIwMDAwMC0weDM0MDAwMDNmZmZm
ZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzAx
Ml0gcGNpX2J1cyAwMDBkOjAzOiByZXNvdXJjZSAxIFttZW0gMHg1MDQwMDAwMC0weDUwNWZmZmZm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzAxM10gcGNpX2J1cyAw
MDBkOjAzOiByZXNvdXJjZSAyIFttZW0gMHgzNDAwMDA0MDAwMDAtMHgzNDAwMDA1ZmZmZmYgNjRi
aXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcwMTRdIHBj
aV9idXMgMDAwZDowNDogcmVzb3VyY2UgMSBbbWVtIDB4NTA2MDAwMDAtMHg1MDdmZmZmZl0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcwMTVdIHBjaV9idXMgMDAwZDow
NDogcmVzb3VyY2UgMiBbbWVtIDB4MzQwMDAwNjAwMDAwLTB4MzQwMDAwN2ZmZmZmIDY0Yml0IHBy
ZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3MTI0XSBBQ1BJOiBQ
Q0kgUm9vdCBCcmlkZ2UgW1BDSTJdIChkb21haW4gMDAwMSBbYnVzIDAwLWZmXSkKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTcxMjhdIGFjcGkgUE5QMEEwODowMjogX09T
QzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kg
SFBYLVR5cGUzXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzE5M10g
YWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90
cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcxNzI1Ml0gYWNwaSBQTlAwQTA4OjAyOiBfT1NDOiBPUyBub3cgY29udHJvbHMgW0FF
UiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MTcyNTVdIGFjcGkgUE5QMEEwODowMjogTUNGRyBxdWlyazogRUNBTSBhdCBbbWVtIDB4M2JmZmYw
MDAwMDAwLTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAweGZmZmZhZDlmYWU0
N2VhNTgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTc4ODRdIGFjcGkg
UE5QMEEwODowMjogRUNBTSBhcmVhIFttZW0gMHgzYmZmZjAwMDAwMDAtMHgzYmZmZmZmZmZmZmZd
IHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MTc4OTFdIGFjcGkgUE5QMEEwODowMjogRUNBTSBhdCBbbWVtIDB4M2JmZmYwMDAwMDAw
LTB4M2JmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MTc5NTJdIFBDSSBob3N0IGJyaWRnZSB0byBidXMgMDAwMTowMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzk1M10gcGNpX2J1cyAwMDAxOjAw
OiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NjAwMDAwMDAtMHg2ZmZmZmZmZiB3aW5kb3ddClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzE3OTU0XSBwY2lfYnVzIDAwMDE6
MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzODAwMDAwMDAwMDAtMHgzYmZmZGZmZmZmZmYg
d2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzk1NV0gcGNp
X2J1cyAwMDAxOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxNzk3MV0gcGNpIDAwMDE6MDA6MDAuMDogWzFkZWY6
ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjcxODAxNF0gcGNpIDAwMDE6MDA6MDEuMDogWzFkZWY6ZTEwMV0gdHlwZSAwMSBj
bGFzcyAweDA2MDQwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODAy
Nl0gcGNpIDAwMDE6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcxODA1M10gcGNpIDAwMDE6MDA6MDEuMDogc3VwcG9y
dHMgRDEgRDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgwNTRdIHBj
aSAwMDAxOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MTgxNDJdIGFjcGlwaHA6IFNsb3QgWzEtM10g
cmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDIzNV0g
cGNpX2J1cyAwMDAxOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyMDIzN10gcGNpIDAwMDE6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBbaW8g
IDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAyMzhdIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3
aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMV0g
YWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjAyMzldIHBjaSAwMDAxOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBhZGRf
YWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjQx
XSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg2MDAwMDAwMC0weDYw
MWZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDI0Ml0gcGNp
IDAwMDE6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4Mzgw
MDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIwMjQzXSBwY2kgMDAwMTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNp
emUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDI0NF0g
cGNpIDAwMDE6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjQ1XSBwY2kgMDAw
MTowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMDI0Nl0gcGNpIDAwMDE6MDA6MDEuMDog
QkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjQ3XSBwY2kgMDAwMTowMDowMS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIw
MjQ5XSBwY2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDYwMDAwMDAwLTB4
NjAxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIwMjUwXSBw
Y2kgMDAwMTowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDM4MDAwMDAwMDAwMC0weDM4
MDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMDI1Ml0gcGNpX2J1cyAwMDAxOjAwOiByZXNvdXJjZSA0IFttZW0gMHg2MDAwMDAwMC0w
eDZmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjAyNTNdIHBjaV9idXMgMDAwMTowMDogcmVzb3VyY2UgNSBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4
M2JmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjAyNTRdIHBjaV9idXMgMDAwMTowMTogcmVzb3VyY2UgMSBbbWVtIDB4NjAwMDAwMDAtMHg2
MDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAyNTVdIHBj
aV9idXMgMDAwMTowMTogcmVzb3VyY2UgMiBbbWVtIDB4MzgwMDAwMDAwMDAwLTB4MzgwMDAwMWZm
ZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIw
MzYyXSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTNdIChkb21haW4gMDAwMCBbYnVzIDAwLWZm
XSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjAzNjZdIGFjcGkgUE5Q
MEEwODowMzogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFTUE0gQ2xvY2tQTSBT
ZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMDQzMF0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBwbGF0Zm9ybSBkb2VzIG5vdCBzdXBw
b3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyMDQ4OV0gYWNwaSBQTlAwQTA4OjAzOiBfT1NDOiBPUyBub3cg
Y29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjA0OTFdIGFjcGkgUE5QMEEwODowMzogTUNGRyBxdWlyazogRUNBTSBhdCBb
bWVtIDB4M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0gd2l0aCAw
eGZmZmZhZDlmYWU0N2VhNTgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MjExMTddIGFjcGkgUE5QMEEwODowMzogRUNBTSBhcmVhIFttZW0gMHgzZmZmZjAwMDAwMDAtMHgz
ZmZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MjExMjRdIGFjcGkgUE5QMEEwODowMzogRUNBTSBhdCBbbWVtIDB4
M2ZmZmYwMDAwMDAwLTB4M2ZmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjExODRdIFBDSSBob3N0IGJyaWRnZSB0byBidXMg
MDAwMDowMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTE4NV0gcGNp
X2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4NzAwMDAwMDAtMHg3ZmZmZmZm
ZiB3aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIxMTg2XSBw
Y2lfYnVzIDAwMDA6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgzYzAwMDAwMDAwMDAtMHgz
ZmZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMTE4N10gcGNpX2J1cyAwMDAwOjAwOiByb290IGJ1cyByZXNvdXJjZSBbYnVzIDAwLWZmXQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTIwNF0gcGNpIDAwMDA6MDA6
MDAuMDogWzFkZWY6ZTEwMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI0Nl0gcGNpIDAwMDA6MDA6MDEuMDogWzFkZWY6ZTEw
MV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyMTI1OV0gcGNpIDAwMDA6MDA6MDEuMDogZW5hYmxpbmcgRXh0ZW5kZWQgVGFncwpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMTI4Nl0gcGNpIDAwMDA6MDA6
MDEuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjEyODZdIHBjaSAwMDAwOjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNo
b3QKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjEzNzNdIGFjcGlwaHA6
IFNsb3QgWzEtNF0gcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMzQ2M10gcGNpX2J1cyAwMDAwOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMzQ2NV0gcGNpIDAwMDA6MDA6MDEuMDogYnJpZGdl
IHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjM0NjZdIHBjaSAwMDAwOjAwOjAx
LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0g
dG8gW2J1cyAwMV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjM0NjddIHBjaSAwMDAwOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXpl
IDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzIzNDY5XSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHg3
MDAwMDAwMC0weDcwMWZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyMzQ3MF0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4M2MwMDAw
MDAwMDAwLTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzIzNDcxXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjcyMzQ3Ml0gcGNpIDAwMDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIz
NDczXSBwY2kgMDAwMDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMzQ3NF0gcGNpIDAw
MDA6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzIzNDc1XSBwY2kgMDAwMDowMDow
MS4wOiBQQ0kgYnJpZGdlIHRvIFtidXMgMDFdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzIzNDc3XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAw
eDcwMDAwMDAwLTB4NzAxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzIzNDc4XSBwY2kgMDAwMDowMDowMS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDNjMDAw
MDAwMDAwMC0weDNjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyMzQ4MF0gcGNpX2J1cyAwMDAwOjAwOiByZXNvdXJjZSA0IFttZW0g
MHg3MDAwMDAwMC0weDdmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjM0ODFdIHBjaV9idXMgMDAwMDowMDogcmVzb3VyY2UgNSBbbWVtIDB4M2Mw
MDAwMDAwMDAwLTB4M2ZmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MjM0ODJdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMSBbbWVtIDB4
NzAwMDAwMDAtMHg3MDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MjM0ODNdIHBjaV9idXMgMDAwMDowMTogcmVzb3VyY2UgMiBbbWVtIDB4M2MwMDAwMDAwMDAw
LTB4M2MwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzIzNTg3XSBBQ1BJOiBQQ0kgUm9vdCBCcmlkZ2UgW1BDSTRdIChkb21haW4gMDAw
MiBbYnVzIDAwLWZmXSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjM1
OTFdIGFjcGkgUE5QMEEwODowNDogX09TQzogT1Mgc3VwcG9ydHMgW0V4dGVuZGVkQ29uZmlnIEFT
UE0gQ2xvY2tQTSBTZWdtZW50cyBNU0kgSFBYLVR5cGUzXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyMzY1NF0gYWNwaSBQTlAwQTA4OjA0OiBfT1NDOiBwbGF0Zm9ybSBk
b2VzIG5vdCBzdXBwb3J0IFtQQ0llSG90cGx1ZyBTSFBDSG90cGx1ZyBQTUUgTFRSXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyMzcxM10gYWNwaSBQTlAwQTA4OjA0OiBf
T1NDOiBPUyBub3cgY29udHJvbHMgW0FFUiBQQ0llQ2FwYWJpbGl0eV0KU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjM3MTVdIGFjcGkgUE5QMEEwODowNDogTUNGRyBxdWly
azogRUNBTSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAw
MC1mZl0gd2l0aCAweGZmZmZhZDlmYWU0N2VhNTgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MjQzNDNdIGFjcGkgUE5QMEEwODowNDogRUNBTSBhcmVhIFttZW0gMHgyM2Zm
ZjAwMDAwMDAtMHgyM2ZmZmZmZmZmZmZdIHJlc2VydmVkIGJ5IFBOUDBDMDI6MDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQzNDhdIGFjcGkgUE5QMEEwODowNDogRUNB
TSBhdCBbbWVtIDB4MjNmZmYwMDAwMDAwLTB4MjNmZmZmZmZmZmZmXSBmb3IgW2J1cyAwMC1mZl0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ0MDddIFBDSSBob3N0IGJy
aWRnZSB0byBidXMgMDAwMjowMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjcyNDQwOF0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MDA4MDAw
MDAtMHgwZmZmZmZmZiB3aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzI0NDA5XSBwY2lfYnVzIDAwMDI6MDA6IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAw
MDAwMDAwMDAtMHgyM2ZmZGZmZmZmZmYgd2luZG93XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjcyNDQxMF0gcGNpX2J1cyAwMDAyOjAwOiByb290IGJ1cyByZXNvdXJjZSBb
YnVzIDAwLWZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDQyN10g
cGNpIDAwMDI6MDA6MDAuMDogWzFkZWY6ZTExMF0gdHlwZSAwMCBjbGFzcyAweDA2MDAwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyNDQ3MF0gcGNpIDAwMDI6MDA6MDEu
MDogWzFkZWY6ZTExMV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAwLjcyNDUwNV0gcGNpIDAwMDI6MDA6MDEuMDogc3VwcG9ydHMgRDEg
RDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ1MDZdIHBjaSAwMDAy
OjAwOjAxLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjQ1NjddIHBjaSAwMDAyOjAwOjA1LjA6IFsxZGVmOmUx
MTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjQ2MDJdIHBjaSAwMDAyOjAwOjA1LjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI0NjAzXSBwY2kgMDAwMjowMDowNS4wOiBQ
TUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzI0Njg3XSBhY3BpcGhwOiBTbG90IFsxLTVdIHJlZ2lzdGVyZWQKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjY3OTldIGFjcGlwaHA6IFNsb3QgWzMt
Ml0gcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyODg5
MV0gcGNpX2J1cyAwMDAyOjAwOiBvbiBOVU1BIG5vZGUgMApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjcyODg5Ml0gcGNpIDAwMDI6MDA6MDEuMDogYnJpZGdlIHdpbmRvdyBb
aW8gIDB4MTAwMC0weDBmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDEwMDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg4OTRdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRn
ZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAw
MV0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mjg4OTVdIHBjaSAwMDAyOjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cg
W21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDFdIGFkZF9zaXplIDIwMDAwMCBh
ZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4
ODk2XSBwY2kgMDAwMjowMDowNS4wOiBicmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0g
dG8gW2J1cyAwMl0gYWRkX3NpemUgMTAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjcyODg5N10gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAx
MDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBwcmVmXSB0byBbYnVzIDAyXSBhZGRfc2l6ZSAyMDAwMDAg
YWRkX2FsaWduIDEwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
ODg5OF0gcGNpIDAwMDI6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgw
MDBmZmZmZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MDBdIHBjaSAwMDAyOjAwOjAx
LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwODAwMDAwLTB4MDA5ZmZmZmZdClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTAyXSBwY2kgMDAwMjowMDowMS4wOiBC
QVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMDAwMDAtMHgyMDAwMDAxZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MDJdIHBjaSAw
MDAyOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTA0XSBwY2kgMDAwMjow
MDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAyMDAwMDAtMHgyMDAwMDAzZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5
MDRdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEw
MDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTA1XSBwY2kgMDAw
MjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MDZdIHBjaSAwMDAyOjAwOjA1
LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTA3XSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6
IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43Mjg5MDldIHBjaSAwMDAyOjAwOjA1LjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI4OTEwXSBwY2kgMDAwMjowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
Mjg5MTBdIHBjaSAwMDAyOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTExXSBwY2kg
MDAwMjowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MTJdIHBjaSAwMDAyOjAw
OjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mjg5MTRdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MDA4MDAwMDAtMHgwMDlmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43Mjg5MTZdIHBjaSAwMDAyOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAw
MDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI4OTE4XSBwY2kgMDAwMjowMDowNS4wOiBQQ0kgYnJpZGdlIHRv
IFtidXMgMDJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTE5XSBw
Y2kgMDAwMjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZm
ZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTIxXSBwY2kgMDAw
MjowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNm
ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcy
ODkyM10gcGNpX2J1cyAwMDAyOjAwOiByZXNvdXJjZSA0IFttZW0gMHgwMDgwMDAwMC0weDBmZmZm
ZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MjRd
IHBjaV9idXMgMDAwMjowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjNmZmRm
ZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5
MjVdIHBjaV9idXMgMDAwMjowMTogcmVzb3VyY2UgMSBbbWVtIDB4MDA4MDAwMDAtMHgwMDlmZmZm
Zl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjg5MjZdIHBjaV9idXMg
MDAwMjowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjAwMDAwMDAwMDAwLTB4MjAwMDAwMWZmZmZmIDY0
Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTI3XSBw
Y2lfYnVzIDAwMDI6MDI6IHJlc291cmNlIDEgW21lbSAweDAwYTAwMDAwLTB4MDBiZmZmZmZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI4OTI3XSBwY2lfYnVzIDAwMDI6
MDI6IHJlc291cmNlIDIgW21lbSAweDIwMDAwMDIwMDAwMC0weDIwMDAwMDNmZmZmZiA2NGJpdCBw
cmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTAzM10gQUNQSTog
UENJIFJvb3QgQnJpZGdlIFtQQ0k1XSAoZG9tYWluIDAwMDMgW2J1cyAwMC1mZl0pClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5MDM4XSBhY3BpIFBOUDBBMDg6MDU6IF9P
U0M6IE9TIHN1cHBvcnRzIFtFeHRlbmRlZENvbmZpZyBBU1BNIENsb2NrUE0gU2VnbWVudHMgTVNJ
IEhQWC1UeXBlM10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MjkxMDFd
IGFjcGkgUE5QMEEwODowNTogX09TQzogcGxhdGZvcm0gZG9lcyBub3Qgc3VwcG9ydCBbUENJZUhv
dHBsdWcgU0hQQ0hvdHBsdWcgUE1FIExUUl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MjkxNjBdIGFjcGkgUE5QMEEwODowNTogX09TQzogT1Mgbm93IGNvbnRyb2xzIFtB
RVIgUENJZUNhcGFiaWxpdHldClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzI5MTYyXSBhY3BpIFBOUDBBMDg6MDU6IE1DRkcgcXVpcms6IEVDQU0gYXQgW21lbSAweDI3ZmZm
MDAwMDAwMC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdIHdpdGggMHhmZmZmYWQ5ZmFl
NDdlYTU4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzI5NzkwXSBhY3Bp
IFBOUDBBMDg6MDU6IEVDQU0gYXJlYSBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZm
XSByZXNlcnZlZCBieSBQTlAwQzAyOjAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzI5Nzk2XSBhY3BpIFBOUDBBMDg6MDU6IEVDQU0gYXQgW21lbSAweDI3ZmZmMDAwMDAw
MC0weDI3ZmZmZmZmZmZmZl0gZm9yIFtidXMgMDAtZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzI5ODQ5XSBQQ0kgaG9zdCBicmlkZ2UgdG8gYnVzIDAwMDM6MDAKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4NTBdIHBjaV9idXMgMDAwMzow
MDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDEwMDAwMDAwLTB4MWZmZmZmZmYgd2luZG93XQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcyOTg1MV0gcGNpX2J1cyAwMDAz
OjAwOiByb290IGJ1cyByZXNvdXJjZSBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4MjdmZmRmZmZmZmZm
IHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4NTJdIHBj
aV9idXMgMDAwMzowMDogcm9vdCBidXMgcmVzb3VyY2UgW2J1cyAwMC1mZl0KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk4NjldIHBjaSAwMDAzOjAwOjAwLjA6IFsxZGVm
OmUxMTBdIHR5cGUgMDAgY2xhc3MgMHgwNjAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43Mjk5MDldIHBjaSAwMDAzOjAwOjAxLjA6IFsxZGVmOmUxMTFdIHR5cGUgMDEg
Y2xhc3MgMHgwNjA0MDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43Mjk5
NDVdIHBjaSAwMDAzOjAwOjAxLjA6IHN1cHBvcnRzIEQxIEQyClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzI5OTQ1XSBwY2kgMDAwMzowMDowMS4wOiBQTUUjIHN1cHBvcnRl
ZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMwMDA0XSBwY2kgMDAwMzowMDowNS4wOiBbMWRlZjplMTE1XSB0eXBlIDAxIGNsYXNzIDB4MDYw
NDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMwMDMwXSBwY2kgMDAw
MzowMDowNS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczMDAzMF0gcGNpIDAwMDM6MDA6MDUuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBEMCBE
MSBEM2hvdApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMDA5OF0gYWNw
aXBocDogU2xvdCBbMS02XSByZWdpc3RlcmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMyMjIwXSBwY2kgMDAwMzowMjowMC4wOiBbODA4NjoxNTIxXSB0eXBlIDAwIGNs
YXNzIDB4MDIwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjM1
XSBwY2kgMDAwMzowMjowMC4wOiByZWcgMHgxMDogW21lbSAweDEwMDIwMDAwLTB4MTAwM2ZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjUyXSBwY2kgMDAwMzow
MjowMC4wOiByZWcgMHgxODogW2lvICAweDFmZmY4MDIwLTB4MWZmZjgwM2ZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyMjYxXSBwY2kgMDAwMzowMjowMC4wOiByZWcg
MHgxYzogW21lbSAweDEwMDQ0MDAwLTB4MTAwNDdmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMyMzcyXSBwY2kgMDAwMzowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBm
cm9tIEQwIEQzaG90IEQzY29sZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczMjQwOV0gcGNpIDAwMDM6MDI6MDAuMDogcmVnIDB4MTg0OiBbbWVtIDB4MDAwMDAwMDAtMHgw
MDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczMjQxMV0gcGNpIDAwMDM6MDI6MDAuMDogVkYobikgQkFSMCBzcGFjZTogW21lbSAweDAwMDAw
MDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJlZl0gKGNvbnRhaW5zIEJBUjAgZm9yIDggVkZzKQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjQzMl0gcGNpIDAwMDM6MDI6MDAu
MDogcmVnIDB4MTkwOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAwM2ZmZiA2NGJpdCBwcmVmXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjQzM10gcGNpIDAwMDM6MDI6MDAu
MDogVkYobikgQkFSMyBzcGFjZTogW21lbSAweDAwMDAwMDAwLTB4MDAwMWZmZmYgNjRiaXQgcHJl
Zl0gKGNvbnRhaW5zIEJBUjMgZm9yIDggVkZzKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczMjUyOV0gcGNpIDAwMDM6MDI6MDAuMDogOC4wMDAgR2IvcyBhdmFpbGFibGUg
UENJZSBiYW5kd2lkdGgsIGxpbWl0ZWQgYnkgNS4wIEdUL3MgUENJZSB4MiBsaW5rIGF0IDAwMDM6
MDA6MDUuMCAoY2FwYWJsZSBvZiAxNi4wMDAgR2IvcyB3aXRoIDUuMCBHVC9zIFBDSWUgeDQgbGlu
aykKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI1NzZdIHBjaSAwMDAz
OjAyOjAwLjE6IFs4MDg2OjE1MjFdIHR5cGUgMDAgY2xhc3MgMHgwMjAwMDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI1OTJdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAw
eDEwOiBbbWVtIDB4MTAwMDAwMDAtMHgxMDAxZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzI2MDhdIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDE4OiBbaW8gIDB4
MWZmZjgwMDAtMHgxZmZmODAxZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzI2MTddIHBjaSAwMDAzOjAyOjAwLjE6IHJlZyAweDFjOiBbbWVtIDB4MTAwNDAwMDAtMHgx
MDA0M2ZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI3MjRdIHBj
aSAwMDAzOjAyOjAwLjE6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDNob3QgRDNjb2xkClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyNzU3XSBwY2kgMDAwMzowMjowMC4x
OiByZWcgMHgxODQ6IFttZW0gMHgwMDAwMDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyNzU4XSBwY2kgMDAwMzowMjowMC4x
OiBWRihuKSBCQVIwIHNwYWNlOiBbbWVtIDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVm
XSAoY29udGFpbnMgQkFSMCBmb3IgOCBWRnMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMyNzc5XSBwY2kgMDAwMzowMjowMC4xOiByZWcgMHgxOTA6IFttZW0gMHgwMDAw
MDAwMC0weDAwMDAzZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMyNzgwXSBwY2kgMDAwMzowMjowMC4xOiBWRihuKSBCQVIzIHNwYWNlOiBbbWVt
IDB4MDAwMDAwMDAtMHgwMDAxZmZmZiA2NGJpdCBwcmVmXSAoY29udGFpbnMgQkFSMyBmb3IgOCBW
RnMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTA2XSBwY2lfYnVz
IDAwMDM6MDA6IG9uIE5VTUEgbm9kZSAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMyOTA3XSBwY2lfYnVzIDAwMDM6MDA6IG1heCBidXMgZGVwdGg6IDEgcGNpX3RyeV9u
dW06IDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5MTBdIHBjaSAw
MDAzOjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTEyXSBwY2kgMDAwMzow
MDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAwMDAwMDAtMHgyNDAwMDAxZmZm
ZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5
MTNdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxNDogYXNzaWduZWQgW21lbSAweDEwMjAwMDAwLTB4
MTAzZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTE0XSBw
Y2kgMDAwMzowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgy
NDAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzI5MTVdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTE2
XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5MTddIHBjaSAw
MDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTE3XSBwY2kgMDAwMzowMDowNS4w
OiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5MTldIHBjaSAwMDAzOjAwOjAxLjA6IFBDSSBi
cmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzI5MjBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MTAwMDAwMDAt
MHgxMDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5MjJd
IHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAwMDAwMDAwLTB4
MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMyOTI1XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEw
MjAwMDAwLTB4MTAyMWZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzMyOTI4XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMDogYXNzaWduZWQgW21lbSAweDEwMjIwMDAw
LTB4MTAyM2ZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTMx
XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMzogYXNzaWduZWQgW21lbSAweDEwMjQwMDAwLTB4MTAy
NDNmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTM1XSBwY2kg
MDAwMzowMjowMC4wOiBCQVIgNzogYXNzaWduZWQgW21lbSAweDI0MDAwMDIwMDAwMC0weDI0MDAw
MDIxZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczMjkzOV0gcGNpIDAwMDM6MDI6MDAuMDogQkFSIDEwOiBhc3NpZ25lZCBbbWVtIDB4MjQwMDAw
MjIwMDAwLTB4MjQwMDAwMjNmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMyOTQzXSBwY2kgMDAwMzowMjowMC4xOiBCQVIgMzogYXNzaWduZWQg
W21lbSAweDEwMjQ0MDAwLTB4MTAyNDdmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMyOTQ2XSBwY2kgMDAwMzowMjowMC4xOiBCQVIgNzogYXNzaWduZWQgW21lbSAw
eDI0MDAwMDI0MDAwMC0weDI0MDAwMDI1ZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjk1MF0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDEwOiBh
c3NpZ25lZCBbbWVtIDB4MjQwMDAwMjYwMDAwLTB4MjQwMDAwMjdmZmZmIDY0Yml0IHByZWZdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTU1XSBwY2kgMDAwMzowMjow
MC4wOiBCQVIgMjogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTU1XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjog
ZmFpbGVkIHRvIGFzc2lnbiBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczMjk1Nl0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgwMDIwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMjk1N10gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lv
ICBzaXplIDB4MDAyMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5
NThdIHBjaSAwMDAzOjAwOjA1LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5NTldIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJp
ZGdlIHdpbmRvdyBbbWVtIDB4MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzI5NjFdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjQwMDAwMjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTYyXSBwY2lfYnVzIDAwMDM6MDA6
IE5vLiAyIHRyeSB0byBhc3NpZ24gdW5hc3NpZ25lZCByZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43MzI5NjRdIHBjaSAwMDAzOjAwOjAxLjA6IEJBUiAxMzogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMyOTY2XSBwY2kgMDAwMzowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24g
W2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzI5NjZdIHBjaSAwMDAzOjAwOjA1LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAw
eDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTY4XSBwY2kg
MDAwMzowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5NjldIHBjaSAwMDAzOjAw
OjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43MzI5NzBdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MTAwMDAwMDAtMHgxMDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzI5NzJdIHBjaSAwMDAzOjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQw
MDAwMDAwMDAwLTB4MjQwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzMyOTc0XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogbm8gc3Bh
Y2UgZm9yIFtpbyAgc2l6ZSAweDAwMjBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMyOTc0XSBwY2kgMDAwMzowMjowMC4wOiBCQVIgMjogZmFpbGVkIHRvIGFzc2lnbiBb
aW8gIHNpemUgMHgwMDIwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
Mjk3NV0gcGNpIDAwMDM6MDI6MDAuMTogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgw
MDIwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjk3Nl0gcGNpIDAw
MDM6MDI6MDAuMTogQkFSIDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDAyMF0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5NzddIHBjaSAwMDAzOjAwOjA1
LjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzI5NzldIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4
MTAyMDAwMDAtMHgxMDNmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzI5ODBdIHBjaSAwMDAzOjAwOjA1LjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjQwMDAw
MjAwMDAwLTB4MjQwMDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzMyOTgxXSBwY2lfYnVzIDAwMDM6MDA6IEF1dG9tYXRpY2FsbHkgZW5h
YmxlZCBwY2kgcmVhbGxvYywgaWYgeW91IGhhdmUgcHJvYmxlbSwgdHJ5IGJvb3Rpbmcgd2l0aCBw
Y2k9cmVhbGxvYz1vZmYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzI5
ODJdIHBjaV9idXMgMDAwMzowMDogcmVzb3VyY2UgNCBbbWVtIDB4MTAwMDAwMDAtMHgxZmZmZmZm
ZiB3aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTgzXSBw
Y2lfYnVzIDAwMDM6MDA6IHJlc291cmNlIDUgW21lbSAweDI0MDAwMDAwMDAwMC0weDI3ZmZkZmZm
ZmZmZiB3aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTg0
XSBwY2lfYnVzIDAwMDM6MDE6IHJlc291cmNlIDEgW21lbSAweDEwMDAwMDAwLTB4MTAxZmZmZmZd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMyOTg1XSBwY2lfYnVzIDAw
MDM6MDE6IHJlc291cmNlIDIgW21lbSAweDI0MDAwMDAwMDAwMC0weDI0MDAwMDFmZmZmZiA2NGJp
dCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjk4Nl0gcGNp
X2J1cyAwMDAzOjAyOiByZXNvdXJjZSAxIFttZW0gMHgxMDIwMDAwMC0weDEwM2ZmZmZmXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMjk4N10gcGNpX2J1cyAwMDAzOjAy
OiByZXNvdXJjZSAyIFttZW0gMHgyNDAwMDAyMDAwMDAtMHgyNDAwMDAzZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzMwNTVdIEFDUEk6IFBD
SSBSb290IEJyaWRnZSBbUENJNl0gKGRvbWFpbiAwMDA0IFtidXMgMDAtZmZdKQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzA1OV0gYWNwaSBQTlAwQTA4OjA2OiBfT1ND
OiBPUyBzdXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBI
UFgtVHlwZTNdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzMTI1XSBh
Y3BpIFBOUDBBMDg6MDY6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3Rw
bHVnIFNIUENIb3RwbHVnIFBNRSBMVFJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzMzMTg0XSBhY3BpIFBOUDBBMDg6MDY6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVS
IFBDSWVDYXBhYmlsaXR5XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
MzE4Nl0gYWNwaSBQTlAwQTA4OjA2OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAw
MDAwMDAtMHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmFkOWZhZTQ3
ZWE1OApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczMzgxN10gYWNwaSBQ
TlAwQTA4OjA2OiBFQ0FNIGFyZWEgW21lbSAweDJiZmZmMDAwMDAwMC0weDJiZmZmZmZmZmZmZl0g
cmVzZXJ2ZWQgYnkgUE5QMEMwMjowMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczMzgyMl0gYWNwaSBQTlAwQTA4OjA2OiBFQ0FNIGF0IFttZW0gMHgyYmZmZjAwMDAwMDAt
MHgyYmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczMzg2OV0gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA0OjAwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzODcwXSBwY2lfYnVzIDAwMDQ6MDA6
IHJvb3QgYnVzIHJlc291cmNlIFttZW0gMHgyMDAwMDAwMC0weDJmZmZmZmZmIHdpbmRvd10KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzM4NzFdIHBjaV9idXMgMDAwNDow
MDogcm9vdCBidXMgcmVzb3VyY2UgW21lbSAweDI4MDAwMDAwMDAwMC0weDJiZmZkZmZmZmZmZiB3
aW5kb3ddClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzODcyXSBwY2lf
YnVzIDAwMDQ6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzODg5XSBwY2kgMDAwNDowMDowMC4wOiBbMWRlZjpl
MTEwXSB0eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzMzOTI1XSBwY2kgMDAwNDowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNs
YXNzIDB4MDYwNDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzMzOTUy
XSBwY2kgMDAwNDowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAwLjczMzk1M10gcGNpIDAwMDQ6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQg
ZnJvbSBEMCBEMSBEM2hvdApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
Mzk5Nl0gcGNpIDAwMDQ6MDA6MDMuMDogWzFkZWY6ZTExM10gdHlwZSAwMSBjbGFzcyAweDA2MDQw
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDAyM10gcGNpIDAwMDQ6
MDA6MDMuMDogc3VwcG9ydHMgRDEgRDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzQwMjNdIHBjaSAwMDA0OjAwOjAzLjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEg
RDNob3QKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQwNjldIHBjaSAw
MDA0OjAwOjA1LjA6IFsxZGVmOmUxMTVdIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQxMDVdIHBjaSAwMDA0OjAwOjA1LjA6IHN1
cHBvcnRzIEQxIEQyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0MTA2
XSBwY2kgMDAwNDowMDowNS4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0MTc5XSBwY2kgMDAwNDowMTowMC4w
OiBbMWEwMzoxMTUwXSB0eXBlIDAxIGNsYXNzIDB4MDYwNDAwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM0MjI2XSBwY2kgMDAwNDowMTowMC4wOiBlbmFibGluZyBFeHRl
bmRlZCBUYWdzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0MjgyXSBw
Y2kgMDAwNDowMTowMC4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNDI4M10gcGNpIDAwMDQ6MDE6MDAuMDogUE1FIyBzdXBwb3J0ZWQgZnJv
bSBEMCBEMSBEMiBEM2hvdCBEM2NvbGQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzQzNzhdIHBjaV9idXMgMDAwNDowMjogZXh0ZW5kZWQgY29uZmlnIHNwYWNlIG5vdCBh
Y2Nlc3NpYmxlClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0Mzk1XSBw
Y2kgMDAwNDowMjowMC4wOiBbMWEwMzoyMDAwXSB0eXBlIDAwIGNsYXNzIDB4MDMwMDAwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0NDE0XSBwY2kgMDAwNDowMjowMC4w
OiByZWcgMHgxMDogW21lbSAweDIwMDAwMDAwLTB4MjFmZmZmZmZdClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0NDI0XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxNDog
W21lbSAweDIyMDAwMDAwLTB4MjIwMWZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzM0NDM0XSBwY2kgMDAwNDowMjowMC4wOiByZWcgMHgxODogW2lvICAweDJmZmY4
MDAwLTB4MmZmZjgwN2ZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0
NDc5XSBwY2kgMDAwNDowMjowMC4wOiBCQVIgMDogYXNzaWduZWQgdG8gZWZpZmIKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzQ1MTVdIHBjaSAwMDA0OjAyOjAwLjA6IHN1
cHBvcnRzIEQxIEQyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM0NTE1
XSBwY2kgMDAwNDowMjowMC4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQyIEQzaG90IEQz
Y29sZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDYyN10gcGNpIDAw
MDQ6MDM6MDAuMDogWzE5MTI6MDAxNF0gdHlwZSAwMCBjbGFzcyAweDBjMDMzMApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDY0OF0gcGNpIDAwMDQ6MDM6MDAuMDogcmVn
IDB4MTA6IFttZW0gMHgyMjIwMDAwMC0weDIyMjAxZmZmIDY0Yml0XQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNDc1N10gcGNpIDAwMDQ6MDM6MDAuMDogUE1FIyBzdXBw
b3J0ZWQgZnJvbSBEMCBEM2hvdCBEM2NvbGQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43MzU4OTFdIHBjaV9idXMgMDAwNDowMDogb24gTlVNQSBub2RlIDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU4OTRdIHBjaSAwMDA0OjAwOjAxLjA6IGJy
aWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1
cyAwMS0wMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU4OTVdIHBjaSAwMDA0OjAwOjAzLjA6IGJyaWRnZSB3
aW5kb3cgW2lvICAweDEwMDAtMHgwZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAxMDAwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1ODk2XSBwY2kgMDAwNDowMDowMy4w
OiBicmlkZ2Ugd2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRv
IFtidXMgMDNdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1ODk3XSBwY2kgMDAwNDowMDowMy4wOiBicmlkZ2Ug
d2luZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMWZmZmZmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAx
MDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNTg5OF0gcGNpIDAwMDQ6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0w
eDBmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDEwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzU4OTldIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21l
bSAweDAwMTAwMDAwLTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwNF0gYWRkX3NpemUg
MjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzU5MDBdIHBjaSAwMDA0OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAw
MDAwLTB4MDAwZmZmZmZdIHRvIFtidXMgMDRdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAw
MDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTA0XSBwY2kgMDAw
NDowMDowMS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkwNV0gcGNpIDAwMDQ6MDA6
MDEuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZm
IDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTA2
XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzAwMDAwMC0weDIz
MWZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkwOF0gcGNp
IDAwMDQ6MDA6MDMuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwMjAwMDAwLTB4Mjgw
MDAwM2ZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM1OTA4XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTQ6IGFzc2lnbmVkIFttZW0gMHgyMzIw
MDAwMC0weDIzM2ZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
NTkwOV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDE1OiBhc3NpZ25lZCBbbWVtIDB4MjgwMDAwNDAw
MDAwLTB4MjgwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzM1OTEwXSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZv
ciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
LjczNTkxMV0gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTEy
XSBwY2kgMDAwNDowMDowMy4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAw
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkxM10gcGNpIDAwMDQ6
MDA6MDMuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTE0XSBwY2kgMDAwNDowMDowNS4w
OiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkxNV0gcGNpIDAwMDQ6MDA6MDUuMDogQkFSIDEzOiBm
YWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzM1OTE3XSBwY2kgMDAwNDowMDowMS4wOiBCQVIgMTM6IG5vIHNwYWNl
IGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjczNTkxN10gcGNpIDAwMDQ6MDA6MDEuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtp
byAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1
OTE4XSBwY2kgMDAwNDowMDowNS4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgx
MDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkxOV0gcGNpIDAw
MDQ6MDA6MDUuMDogQkFSIDEzOiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTIwXSBwY2kgMDAwNDowMDow
My4wOiBCQVIgMTM6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkyMV0gcGNpIDAwMDQ6MDA6MDMuMDogQkFSIDEz
OiBmYWlsZWQgdG8gYXNzaWduIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM1OTIyXSBwY2kgMDAwNDowMTowMC4wOiBCQVIgMTQ6IGFzc2ln
bmVkIFttZW0gMHgyMDAwMDAwMC0weDIyZmZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjczNTkyM10gcGNpIDAwMDQ6MDE6MDAuMDogQkFSIDEzOiBubyBzcGFjZSBm
b3IgW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzU5MjRdIHBjaSAwMDA0OjAxOjAwLjA6IEJBUiAxMzogZmFpbGVkIHRvIGFzc2lnbiBbaW8g
IHNpemUgMHgxMDAwXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTky
NV0gcGNpIDAwMDQ6MDI6MDAuMDogQkFSIDA6IGFzc2lnbmVkIFttZW0gMHgyMDAwMDAwMC0weDIx
ZmZmZmZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkyOV0gcGNp
IDAwMDQ6MDI6MDAuMDogQkFSIDE6IGFzc2lnbmVkIFttZW0gMHgyMjAwMDAwMC0weDIyMDFmZmZm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkzMl0gcGNpIDAwMDQ6
MDI6MDAuMDogQkFSIDI6IG5vIHNwYWNlIGZvciBbaW8gIHNpemUgMHgwMDgwXQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTkzM10gcGNpIDAwMDQ6MDI6MDAuMDogQkFS
IDI6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MDA4MF0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzU5MzRdIHBjaSAwMDA0OjAxOjAwLjA6IFBDSSBicmlkZ2Ug
dG8gW2J1cyAwMl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5Mzhd
IHBjaSAwMDA0OjAxOjAwLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZm
ZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NDVdIHBjaSAw
MDA0OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMS0wMl0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NDddIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdp
bmRvdyBbbWVtIDB4MjAwMDAwMDAtMHgyMmZmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMC43MzU5NDhdIHBjaSAwMDA0OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBb
bWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTUwXSBwY2kgMDAwNDowMzowMC4wOiBCQVIg
MDogYXNzaWduZWQgW21lbSAweDIzMDAwMDAwLTB4MjMwMDFmZmYgNjRiaXRdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTU5XSBwY2kgMDAwNDowMDowMy4wOiBQQ0kg
YnJpZGdlIHRvIFtidXMgMDNdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM1OTYxXSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDIzMDAwMDAw
LTB4MjMxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTYy
XSBwY2kgMDAwNDowMDowMy4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDI4MDAwMDIwMDAwMC0w
eDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNTk2NF0gcGNpIDAwMDQ6MDA6MDUuMDogUENJIGJyaWRnZSB0byBbYnVzIDA0XQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTk2NV0gcGNpIDAwMDQ6MDA6
MDUuMDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTk2N10gcGNpIDAwMDQ6MDA6MDUuMDog
ICBicmlkZ2Ugd2luZG93IFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQg
cHJlZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NjldIHBjaV9i
dXMgMDAwNDowMDogU29tZSBQQ0kgZGV2aWNlIHJlc291cmNlcyBhcmUgdW5hc3NpZ25lZCwgdHJ5
IGJvb3Rpbmcgd2l0aCBwY2k9cmVhbGxvYwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNTk3MF0gcGNpX2J1cyAwMDA0OjAwOiByZXNvdXJjZSA0IFttZW0gMHgyMDAwMDAw
MC0weDJmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43MzU5NzFdIHBjaV9idXMgMDAwNDowMDogcmVzb3VyY2UgNSBbbWVtIDB4MjgwMDAwMDAwMDAw
LTB4MmJmZmRmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43MzU5NzJdIHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMSBbbWVtIDB4MjAwMDAwMDAt
MHgyMmZmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzU5NzJd
IHBjaV9idXMgMDAwNDowMTogcmVzb3VyY2UgMiBbbWVtIDB4MjgwMDAwMDAwMDAwLTB4MjgwMDAw
MWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM1OTczXSBwY2lfYnVzIDAwMDQ6MDI6IHJlc291cmNlIDEgW21lbSAweDIwMDAwMDAwLTB4MjJm
ZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTc0XSBwY2lf
YnVzIDAwMDQ6MDM6IHJlc291cmNlIDEgW21lbSAweDIzMDAwMDAwLTB4MjMxZmZmZmZdClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM1OTc1XSBwY2lfYnVzIDAwMDQ6MDM6
IHJlc291cmNlIDIgW21lbSAweDI4MDAwMDIwMDAwMC0weDI4MDAwMDNmZmZmZiA2NGJpdCBwcmVm
XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTk3Nl0gcGNpX2J1cyAw
MDA0OjA0OiByZXNvdXJjZSAxIFttZW0gMHgyMzIwMDAwMC0weDIzM2ZmZmZmXQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNTk3N10gcGNpX2J1cyAwMDA0OjA0OiByZXNv
dXJjZSAyIFttZW0gMHgyODAwMDA0MDAwMDAtMHgyODAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzYwMDRdIEFDUEk6IFBDSSBSb290
IEJyaWRnZSBbUENJN10gKGRvbWFpbiAwMDA1IFtidXMgMDAtZmZdKQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjAwOV0gYWNwaSBQTlAwQTA4OjA3OiBfT1NDOiBPUyBz
dXBwb3J0cyBbRXh0ZW5kZWRDb25maWcgQVNQTSBDbG9ja1BNIFNlZ21lbnRzIE1TSSBIUFgtVHlw
ZTNdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2MDczXSBhY3BpIFBO
UDBBMDg6MDc6IF9PU0M6IHBsYXRmb3JtIGRvZXMgbm90IHN1cHBvcnQgW1BDSWVIb3RwbHVnIFNI
UENIb3RwbHVnIFBNRSBMVFJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzM2MTMxXSBhY3BpIFBOUDBBMDg6MDc6IF9PU0M6IE9TIG5vdyBjb250cm9scyBbQUVSIFBDSWVD
YXBhYmlsaXR5XQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjEzNF0g
YWNwaSBQTlAwQTA4OjA3OiBNQ0ZHIHF1aXJrOiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAt
MHgyZmZmZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXSB3aXRoIDB4ZmZmZmFkOWZhZTQ3ZWE1OApT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjc2M10gYWNwaSBQTlAwQTA4
OjA3OiBFQ0FNIGFyZWEgW21lbSAweDJmZmZmMDAwMDAwMC0weDJmZmZmZmZmZmZmZl0gcmVzZXJ2
ZWQgYnkgUE5QMEMwMjowMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjcz
Njc3MF0gYWNwaSBQTlAwQTA4OjA3OiBFQ0FNIGF0IFttZW0gMHgyZmZmZjAwMDAwMDAtMHgyZmZm
ZmZmZmZmZmZdIGZvciBbYnVzIDAwLWZmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAwLjczNjgyN10gUENJIGhvc3QgYnJpZGdlIHRvIGJ1cyAwMDA1OjAwClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2ODI4XSBwY2lfYnVzIDAwMDU6MDA6IHJvb3Qg
YnVzIHJlc291cmNlIFttZW0gMHgzMDAwMDAwMC0weDNmZmZmZmZmIHdpbmRvd10KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzY4MjldIHBjaV9idXMgMDAwNTowMDogcm9v
dCBidXMgcmVzb3VyY2UgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2ODMwXSBwY2lfYnVzIDAw
MDU6MDA6IHJvb3QgYnVzIHJlc291cmNlIFtidXMgMDAtZmZdClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzM2ODQ4XSBwY2kgMDAwNTowMDowMC4wOiBbMWRlZjplMTEwXSB0
eXBlIDAwIGNsYXNzIDB4MDYwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDAuNzM2ODkyXSBwY2kgMDAwNTowMDowMS4wOiBbMWRlZjplMTExXSB0eXBlIDAxIGNsYXNzIDB4
MDYwNDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM2OTI4XSBwY2kg
MDAwNTowMDowMS4wOiBzdXBwb3J0cyBEMSBEMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAwLjczNjkyOV0gcGNpIDAwMDU6MDA6MDEuMDogUE1FIyBzdXBwb3J0ZWQgZnJvbSBE
MCBEMSBEM2hvdApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNjk4OV0g
cGNpIDAwMDU6MDA6MDUuMDogWzFkZWY6ZTExNV0gdHlwZSAwMSBjbGFzcyAweDA2MDQwMApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjczNzAyNV0gcGNpIDAwMDU6MDA6MDUu
MDogc3VwcG9ydHMgRDEgRDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
MzcwMjZdIHBjaSAwMDA1OjAwOjA1LjA6IFBNRSMgc3VwcG9ydGVkIGZyb20gRDAgRDEgRDNob3QK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcwNzhdIHBjaSAwMDA1OjAw
OjA3LjA6IFsxZGVmOmUxMTddIHR5cGUgMDEgY2xhc3MgMHgwNjA0MDAKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43MzcxMTRdIHBjaSAwMDA1OjAwOjA3LjA6IHN1cHBvcnRz
IEQxIEQyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MTE1XSBwY2kg
MDAwNTowMDowNy4wOiBQTUUjIHN1cHBvcnRlZCBmcm9tIEQwIEQxIEQzaG90ClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzM3MTkxXSBhY3BpcGhwOiBTbG90IFsxLTddIHJl
Z2lzdGVyZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDEzOThdIHBj
aV9idXMgMDAwNTowMDogb24gTlVNQSBub2RlIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDE0MDBdIHBjaSAwMDA1OjAwOjAxLjA6IGJyaWRnZSB3aW5kb3cgW2lvICAw
eDEwMDAtMHgwZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAxMDAwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDAxXSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2lu
ZG93IFttZW0gMHgwMDEwMDAwMC0weDAwMGZmZmZmIDY0Yml0IHByZWZdIHRvIFtidXMgMDFdIGFk
ZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQxNDAyXSBwY2kgMDAwNTowMDowMS4wOiBicmlkZ2Ugd2luZG93IFttZW0g
MHgwMDEwMDAwMC0weDAwMGZmZmZmXSB0byBbYnVzIDAxXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2Fs
aWduIDEwMDAwMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQwM10g
cGNpIDAwMDU6MDA6MDUuMDogYnJpZGdlIHdpbmRvdyBbaW8gIDB4MTAwMC0weDBmZmZdIHRvIFti
dXMgMDJdIGFkZF9zaXplIDEwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDE0MDVdIHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAw
LTB4MDAwZmZmZmYgNjRiaXQgcHJlZl0gdG8gW2J1cyAwMl0gYWRkX3NpemUgMjAwMDAwIGFkZF9h
bGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MDZd
IHBjaSAwMDA1OjAwOjA1LjA6IGJyaWRnZSB3aW5kb3cgW21lbSAweDAwMTAwMDAwLTB4MDAwZmZm
ZmZdIHRvIFtidXMgMDJdIGFkZF9zaXplIDIwMDAwMCBhZGRfYWxpZ24gMTAwMDAwClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDA3XSBwY2kgMDAwNTowMDowNy4wOiBi
cmlkZ2Ugd2luZG93IFtpbyAgMHgxMDAwLTB4MGZmZl0gdG8gW2J1cyAwM10gYWRkX3NpemUgMTAw
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQwOF0gcGNpIDAwMDU6
MDA6MDcuMDogYnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZiA2NGJpdCBw
cmVmXSB0byBbYnVzIDAzXSBhZGRfc2l6ZSAyMDAwMDAgYWRkX2FsaWduIDEwMDAwMApTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQwOV0gcGNpIDAwMDU6MDA6MDcuMDog
YnJpZGdlIHdpbmRvdyBbbWVtIDB4MDAxMDAwMDAtMHgwMDBmZmZmZl0gdG8gW2J1cyAwM10gYWRk
X3NpemUgMjAwMDAwIGFkZF9hbGlnbiAxMDAwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDE0MTJdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxNDogYXNzaWduZWQgW21l
bSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDAuNzQxNDEzXSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTU6IGFzc2lnbmVkIFttZW0gMHgy
YzAwMDAwMDAwMDAtMHgyYzAwMDAxZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MTRdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxNDogYXNz
aWduZWQgW21lbSAweDMwMjAwMDAwLTB4MzAzZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDAuNzQxNDE1XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTU6IGFzc2lnbmVk
IFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MTZdIHBjaSAwMDA1OjAwOjA3LjA6IEJB
UiAxNDogYXNzaWduZWQgW21lbSAweDMwNDAwMDAwLTB4MzA1ZmZmZmZdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDE3XSBwY2kgMDAwNTowMDowNy4wOiBCQVIgMTU6
IGFzc2lnbmVkIFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJlZl0K
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MThdIHBjaSAwMDA1OjAw
OjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDE5XSBwY2kgMDAwNTowMDowMS4wOiBCQVIg
MTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MjBdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzogbm8g
c3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzQxNDIxXSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3Np
Z24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MC43NDE0MjJdIHBjaSAwMDA1OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6
ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDIyXSBw
Y2kgMDAwNTowMDowNy4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAw
MF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MjVdIHBjaSAwMDA1
OjAwOjA3LjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDI1XSBwY2kgMDAwNTowMDowNy4wOiBC
QVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MjZdIHBjaSAwMDA1OjAwOjA1LjA6IEJBUiAxMzog
bm8gc3BhY2UgZm9yIFtpbyAgc2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQxNDI3XSBwY2kgMDAwNTowMDowNS4wOiBCQVIgMTM6IGZhaWxlZCB0byBh
c3NpZ24gW2lvICBzaXplIDB4MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NDE0MjhdIHBjaSAwMDA1OjAwOjAxLjA6IEJBUiAxMzogbm8gc3BhY2UgZm9yIFtpbyAg
c2l6ZSAweDEwMDBdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDI4
XSBwY2kgMDAwNTowMDowMS4wOiBCQVIgMTM6IGZhaWxlZCB0byBhc3NpZ24gW2lvICBzaXplIDB4
MTAwMF0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0MzBdIHBjaSAw
MDA1OjAwOjAxLjA6IFBDSSBicmlkZ2UgdG8gW2J1cyAwMV0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDE0MzFdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRv
dyBbbWVtIDB4MzAwMDAwMDAtMHgzMDFmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NDE0MzNdIHBjaSAwMDA1OjAwOjAxLjA6ICAgYnJpZGdlIHdpbmRvdyBbbWVt
IDB4MmMwMDAwMDAwMDAwLTB4MmMwMDAwMWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDM1XSBwY2kgMDAwNTowMDowNS4wOiBQQ0kgYnJp
ZGdlIHRvIFtidXMgMDJdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQx
NDM3XSBwY2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDMwMjAwMDAwLTB4
MzAzZmZmZmZdClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDM4XSBw
Y2kgMDAwNTowMDowNS4wOiAgIGJyaWRnZSB3aW5kb3cgW21lbSAweDJjMDAwMDIwMDAwMC0weDJj
MDAwMDNmZmZmZiA2NGJpdCBwcmVmXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAwLjc0MTQ0MF0gcGNpIDAwMDU6MDA6MDcuMDogUENJIGJyaWRnZSB0byBbYnVzIDAzXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQ0Ml0gcGNpIDAwMDU6MDA6MDcu
MDogICBicmlkZ2Ugd2luZG93IFttZW0gMHgzMDQwMDAwMC0weDMwNWZmZmZmXQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQ0M10gcGNpIDAwMDU6MDA6MDcuMDogICBi
cmlkZ2Ugd2luZG93IFttZW0gMHgyYzAwMDA0MDAwMDAtMHgyYzAwMDA1ZmZmZmYgNjRiaXQgcHJl
Zl0KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0NDVdIHBjaV9idXMg
MDAwNTowMDogcmVzb3VyY2UgNCBbbWVtIDB4MzAwMDAwMDAtMHgzZmZmZmZmZiB3aW5kb3ddClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDQ2XSBwY2lfYnVzIDAwMDU6
MDA6IHJlc291cmNlIDUgW21lbSAweDJjMDAwMDAwMDAwMC0weDJmZmZkZmZmZmZmZiB3aW5kb3dd
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDQ3XSBwY2lfYnVzIDAw
MDU6MDE6IHJlc291cmNlIDEgW21lbSAweDMwMDAwMDAwLTB4MzAxZmZmZmZdClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQxNDQ4XSBwY2lfYnVzIDAwMDU6MDE6IHJlc291
cmNlIDIgW21lbSAweDJjMDAwMDAwMDAwMC0weDJjMDAwMDFmZmZmZiA2NGJpdCBwcmVmXQpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQ0OV0gcGNpX2J1cyAwMDA1OjAy
OiByZXNvdXJjZSAxIFttZW0gMHgzMDIwMDAwMC0weDMwM2ZmZmZmXQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc0MTQ1MF0gcGNpX2J1cyAwMDA1OjAyOiByZXNvdXJjZSAy
IFttZW0gMHgyYzAwMDAyMDAwMDAtMHgyYzAwMDAzZmZmZmYgNjRiaXQgcHJlZl0KU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDE0NTBdIHBjaV9idXMgMDAwNTowMzogcmVz
b3VyY2UgMSBbbWVtIDB4MzA0MDAwMDAtMHgzMDVmZmZmZl0KU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMC43NDE0NTJdIHBjaV9idXMgMDAwNTowMzogcmVzb3VyY2UgMiBbbWVt
IDB4MmMwMDAwNDAwMDAwLTB4MmMwMDAwNWZmZmZmIDY0Yml0IHByZWZdClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQzOTU2XSBpb21tdTogRGVmYXVsdCBkb21haW4gdHlw
ZTogVHJhbnNsYXRlZCAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM5
OTBdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogVkdBIGRldmljZSBhZGRlZDogZGVjb2Rlcz1p
byttZW0sb3ducz1ub25lLGxvY2tzPW5vbmUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDM5OTFdIHBjaSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogYnJpZGdlIGNvbnRyb2wg
cG9zc2libGUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NDM5OTRdIHBj
aSAwMDA0OjAyOjAwLjA6IHZnYWFyYjogc2V0dGluZyBhcyBib290IGRldmljZSAoVkdBIGxlZ2Fj
eSByZXNvdXJjZXMgbm90IGF2YWlsYWJsZSkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMC43NDM5OTRdIHZnYWFyYjogbG9hZGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDAuNzQ0MDc4XSBFREFDIE1DOiBWZXI6IDMuMC4wClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ0MjEwXSBSZWdpc3RlcmVkIGVmaXZhcnMgb3BlcmF0aW9u
cwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NDkwN10gTmV0TGFiZWw6
IEluaXRpYWxpemluZwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc0NDkw
OF0gTmV0TGFiZWw6ICBkb21haW4gaGFzaCBzaXplID0gMTI4ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDAuNzQ0OTA4XSBOZXRMYWJlbDogIHByb3RvY29scyA9IFVOTEFCRUxF
RCBDSVBTT3Y0IENBTElQU08KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43
NDQ5MThdIE5ldExhYmVsOiAgdW5sYWJlbGVkIHRyYWZmaWMgYWxsb3dlZCBieSBkZWZhdWx0ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzQ1MDQ1XSBjbG9ja3NvdXJjZTog
U3dpdGNoZWQgdG8gY2xvY2tzb3VyY2UgYXJjaF9zeXNfY291bnRlcgpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzU1M10gVkZTOiBEaXNrIHF1b3RhcyBkcXVvdF82LjYu
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1MzU3MF0gVkZTOiBEcXVv
dC1jYWNoZSBoYXNoIHRhYmxlIGVudHJpZXM6IDUxMiAob3JkZXIgMCwgNDA5NiBieXRlcykKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTM2ODVdIEFwcEFybW9yOiBBcHBB
cm1vciBGaWxlc3lzdGVtIEVuYWJsZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMC43NTM3MThdIHBucDogUG5QIEFDUEkgaW5pdApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAwLjc1NDM0NF0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4M2JmZmYwMDAwMDAwLTB4
M2JmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MzQ1XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgzZmZm
ZjAwMDAwMDAtMHgzZmZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQzNDZdIHN5c3RlbSAwMDowMDog
W21lbSAweDIzZmZmMDAwMDAwMC0weDIzZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSBy
ZXNlcnZlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDM0OF0gc3lz
dGVtIDAwOjAwOiBbbWVtIDB4MjdmZmYwMDAwMDAwLTB4MjdmZmZmZmZmZmZmIHdpbmRvd10gY291
bGQgbm90IGJlIHJlc2VydmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAu
NzU0MzQ5XSBzeXN0ZW0gMDA6MDA6IFttZW0gMHgyYmZmZjAwMDAwMDAtMHgyYmZmZmZmZmZmZmYg
d2luZG93XSBjb3VsZCBub3QgYmUgcmVzZXJ2ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMC43NTQzNTBdIHN5c3RlbSAwMDowMDogW21lbSAweDJmZmZmMDAwMDAwMC0weDJm
ZmZmZmZmZmZmZiB3aW5kb3ddIGNvdWxkIG5vdCBiZSByZXNlcnZlZApTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NDM1MV0gc3lzdGVtIDAwOjAwOiBbbWVtIDB4MzNmZmYw
MDAwMDAwLTB4MzNmZmZmZmZmZmZmIHdpbmRvd10gY291bGQgbm90IGJlIHJlc2VydmVkClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU0MzUyXSBzeXN0ZW0gMDA6MDA6IFtt
ZW0gMHgzN2ZmZjAwMDAwMDAtMHgzN2ZmZmZmZmZmZmYgd2luZG93XSBjb3VsZCBub3QgYmUgcmVz
ZXJ2ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQzNTZdIHN5c3Rl
bSAwMDowMDogUGx1ZyBhbmQgUGxheSBBQ1BJIGRldmljZSwgSURzIFBOUDBjMDIgKGFjdGl2ZSkK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTQ2NDZdIHBucDogUG5QIEFD
UEk6IGZvdW5kIDEgZGV2aWNlcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAw
Ljc1NTU4NF0gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAyClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU1NjQ4XSBJUCBpZGVudHMgaGFzaCB0YWJsZSBlbnRy
aWVzOiAyNjIxNDQgKG9yZGVyOiA5LCAyMDk3MTUyIGJ5dGVzLCBsaW5lYXIpClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU3MjM5XSB0Y3BfbGlzdGVuX3BvcnRhZGRyX2hh
c2ggaGFzaCB0YWJsZSBlbnRyaWVzOiAzMjc2OCAob3JkZXI6IDcsIDUyNDI4OCBieXRlcywgbGlu
ZWFyKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1NzUyN10gVENQIGVz
dGFibGlzaGVkIGhhc2ggdGFibGUgZW50cmllczogNTI0Mjg4IChvcmRlcjogMTAsIDQxOTQzMDQg
Ynl0ZXMsIGxpbmVhcikKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NTg2
MDldIFRDUCBiaW5kIGhhc2ggdGFibGUgZW50cmllczogNjU1MzYgKG9yZGVyOiA4LCAxMDQ4NTc2
IGJ5dGVzLCBsaW5lYXIpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU4
ODY3XSBUQ1A6IEhhc2ggdGFibGVzIGNvbmZpZ3VyZWQgKGVzdGFibGlzaGVkIDUyNDI4OCBiaW5k
IDY1NTM2KQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc1ODk1M10gVURQ
IGhhc2ggdGFibGUgZW50cmllczogMzI3NjggKG9yZGVyOiA4LCAxMDQ4NTc2IGJ5dGVzLCBsaW5l
YXIpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzU5MzU2XSBVRFAtTGl0
ZSBoYXNoIHRhYmxlIGVudHJpZXM6IDMyNzY4IChvcmRlcjogOCwgMTA0ODU3NiBieXRlcywgbGlu
ZWFyKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAwLjc2MDEyOV0gTkVUOiBS
ZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDAuNzYwMTM1XSBORVQ6IFJlZ2lzdGVyZWQgcHJvdG9jb2wgZmFtaWx5IDQ0ClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDAuNzYwMTQzXSBwY2kgMDAwYzowMTowMC4w
OiBDTFMgbWlzbWF0Y2ggKDY0ICE9IDMyKSwgdXNpbmcgNjQgYnl0ZXMKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMC43NjAyMTldIFRyeWluZyB0byB1bnBhY2sgcm9vdGZzIGlt
YWdlIGFzIGluaXRyYW1mcy4uLgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIwMjk0N10gRnJlZWluZyBpbml0cmQgbWVtb3J5OiA0MTAxMksKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMDQ2NjRdIGh3IHBlcmZldmVudHM6IGVuYWJsZWQgd2l0aCBh
cm12OF9wbXV2M18wIFBNVSBkcml2ZXIsIDcgY291bnRlcnMgYXZhaWxhYmxlClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA0NzUxXSBrdm0gWzFdOiBJUEEgU2l6ZSBMaW1p
dDogNDggYml0cwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwNDc3M10g
a3ZtIFsxXTogR0lDdjM6IG5vIEdJQ1YgcmVzb3VyY2UgZW50cnkKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMDQ3NzRdIGt2bSBbMV06IGRpc2FibGluZyBHSUN2MiBlbXVs
YXRpb24KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDQ3ODVdIGt2bSBb
MV06IEdJQyBzeXN0ZW0gcmVnaXN0ZXIgQ1BVIGludGVyZmFjZSBlbmFibGVkClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA1MTQ2XSBrdm0gWzFdOiB2Z2ljIGludGVycnVw
dCBJUlE5ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA1NTc2XSBrdm0g
WzFdOiBWSEUgbW9kZSBpbml0aWFsaXplZCBzdWNjZXNzZnVsbHkKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMDY0MTldIEluaXRpYWxpc2Ugc3lzdGVtIHRydXN0ZWQga2V5
cmluZ3MKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDY0MjZdIEtleSB0
eXBlIGJsYWNrbGlzdCByZWdpc3RlcmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjA2NDYzXSB3b3JraW5nc2V0OiB0aW1lc3RhbXBfYml0cz00MiBtYXhfb3JkZXI9MjQg
YnVja2V0X29yZGVyPTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDcx
NTBdIHpidWQ6IGxvYWRlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIw
NzI3MF0gaW50ZWdyaXR5OiBQbGF0Zm9ybSBLZXlyaW5nIGluaXRpYWxpemVkClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA3MjcxXSBLZXkgdHlwZSBhc3ltbWV0cmljIHJl
Z2lzdGVyZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDcyNzJdIEFz
eW1tZXRyaWMga2V5IHBhcnNlciAneDUwOScgcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjIwNzI3OF0gQmxvY2sgbGF5ZXIgU0NTSSBnZW5lcmljIChic2cp
IGRyaXZlciB2ZXJzaW9uIDAuNCBsb2FkZWQgKG1ham9yIDI0OSkKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yMDczMTZdIGlvIHNjaGVkdWxlciBtcS1kZWFkbGluZSByZWdp
c3RlcmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4Mjk4XSBzaHBj
aHA6IFN0YW5kYXJkIEhvdCBQbHVnIFBDSSBDb250cm9sbGVyIERyaXZlciB2ZXJzaW9uOiAwLjQK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDg0MTBdIGVmaWZiOiBwcm9i
aW5nIGZvciBlZmlmYgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwODQz
Ml0gZWZpZmI6IGZyYW1lYnVmZmVyIGF0IDB4MjAwMDAwMDAsIHVzaW5nIDE4NzZrLCB0b3RhbCAx
ODc1awpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIwODQzM10gZWZpZmI6
IG1vZGUgaXMgODAweDYwMHgzMiwgbGluZWxlbmd0aD0zMjAwLCBwYWdlcz0xClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4NDMzXSBlZmlmYjogc2Nyb2xsaW5nOiByZWRy
YXcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMDg0MzVdIGVmaWZiOiBU
cnVlY29sb3I6IHNpemU9ODo4Ojg6OCwgc2hpZnQ9MjQ6MTY6ODowClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjA4NTA1XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3Vy
IGZyYW1lIGJ1ZmZlciBkZXZpY2UgMTAweDM3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjE0MzY2XSBmYjA6IEVGSSBWR0EgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNDcxMF0gaW5wdXQ6IFBvd2VyIEJ1dHRv
biBhcyAvZGV2aWNlcy9MTlhTWVNUTTowMC9MTlhTWUJVUzowMC9QTlAwQzBDOjAwL2lucHV0L2lu
cHV0MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIxNDcyM10gQUNQSTog
UG93ZXIgQnV0dG9uIFtQV1JCXQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIxOTEzNV0gQUNQSSBHVERUOiBmb3VuZCAxIFNCU0EgZ2VuZXJpYyBXYXRjaGRvZyhzKS4KU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTk2NTldIFNlcmlhbDogODI1MC8x
NjU1MCBkcml2ZXIsIDQgcG9ydHMsIElSUSBzaGFyaW5nIGVuYWJsZWQKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTk5MjhdIFNlcmlhbDogQU1CQSBkcml2ZXIKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMTk5NjNdIG1zbV9zZXJpYWw6IGRyaXZl
ciBpbml0aWFsaXplZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMDAx
Ml0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMC5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjAwMzVdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjAuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZm
ZikKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjAyMDNdIGFybS1zbW11
LXYzIGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjAyNDBdIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjAzMDNdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjAuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjA2NjNdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjEuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjIwNjc0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGlhcyA0
OC1iaXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjIwNzg5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1
dG86IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjIwODI2XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86
IGFsbG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjIwODkwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My4xLmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjIxMjA5XSBhcm0tc21tdS12MyBhcm0tc21tdS12My4yLmF1dG86IG9wdGlv
biBtYXNrIDB4MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMTIyMF0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChm
ZWF0dXJlcyAweDAwMDAxZmZmKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIyMTI2OF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYg
ZW50cmllcyBmb3IgY21kcQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIy
MTM3OV0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50
cmllcyBmb3IgZXZ0cQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMTQx
NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMi5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmll
cyBmb3IgcHJpcQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMTc2Ml0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuMy5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjE3NzJdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjMuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjE4MTddIGFybS1zbW11LXYz
IGFybS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjE5MjhdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjE5NjNdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjMuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjIyNjVdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjQuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjIyMjc0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGlhcyA0OC1i
aXQsIG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuMjIyMzQxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86
IGFsbG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjIyMzg1XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFs
bG9jYXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjIyNDk0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My40LmF1dG86IGFsbG9j
YXRlZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjIyNzkxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My41LmF1dG86IG9wdGlvbiBt
YXNrIDB4MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjgwMF0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBpYXMgNDgtYml0LCBvYXMgNDgtYml0IChmZWF0
dXJlcyAweDAwMDAxZmZmKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIy
Mjg2NF0gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50
cmllcyBmb3IgY21kcQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMjkw
N10gYXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgMzI3NjggZW50cmll
cyBmb3IgZXZ0cQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMzAxNl0g
YXJtLXNtbXUtdjMgYXJtLXNtbXUtdjMuNS5hdXRvOiBhbGxvY2F0ZWQgNjU1MzYgZW50cmllcyBm
b3IgcHJpcQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyMzMxMl0gYXJt
LXNtbXUtdjMgYXJtLXNtbXUtdjMuNi5hdXRvOiBvcHRpb24gbWFzayAweDAKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjMzMjJdIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjYuYXV0bzogaWFzIDQ4LWJpdCwgb2FzIDQ4LWJpdCAoZmVhdHVyZXMgMHgwMDAwMWZmZikKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjMzNjBdIGFybS1zbW11LXYzIGFy
bS1zbW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIGNtZHEKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM0MjRdIGFybS1zbW11LXYzIGFybS1z
bW11LXYzLjYuYXV0bzogYWxsb2NhdGVkIDMyNzY4IGVudHJpZXMgZm9yIGV2dHEKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM0NjhdIGFybS1zbW11LXYzIGFybS1zbW11
LXYzLjYuYXV0bzogYWxsb2NhdGVkIDY1NTM2IGVudHJpZXMgZm9yIHByaXEKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMjM4NjVdIGFybS1zbW11LXYzIGFybS1zbW11LXYz
LjcuYXV0bzogb3B0aW9uIG1hc2sgMHgwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjIzODc0XSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGlhcyA0OC1iaXQs
IG9hcyA0OC1iaXQgKGZlYXR1cmVzIDB4MDAwMDFmZmYpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjIzOTEwXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFs
bG9jYXRlZCA2NTUzNiBlbnRyaWVzIGZvciBjbWRxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuMjIzOTcxXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9j
YXRlZCAzMjc2OCBlbnRyaWVzIGZvciBldnRxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuMjI0MDEzXSBhcm0tc21tdS12MyBhcm0tc21tdS12My43LmF1dG86IGFsbG9jYXRl
ZCA2NTUzNiBlbnRyaWVzIGZvciBwcmlxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuMjI4NjE5XSBtb3VzZWRldjogUFMvMiBtb3VzZSBkZXZpY2UgY29tbW9uIGZvciBhbGwg
bWljZQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIyOTYxMV0gcnRjLWVm
aSBydGMtZWZpLjA6IHJlZ2lzdGVyZWQgYXMgcnRjMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAxLjIzMDA5M10gcnRjLWVmaSBydGMtZWZpLjA6IHNldHRpbmcgc3lzdGVtIGNs
b2NrIHRvIDIwMjEtMDktMTRUMDU6MDA6MDggVVRDICgxNjMxNTk1NjA4KQpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIzMDY0MV0gbGVkdHJpZy1jcHU6IHJlZ2lzdGVyZWQg
dG8gaW5kaWNhdGUgYWN0aXZpdHkgb24gQ1BVcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjIzMDkwOV0gU01DQ0M6IFNPQ19JRDogSUQgPSBqZXAxMDY6MGExNjowMDAxIFJl
dmlzaW9uID0gMHgwMDAwMDBhMQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjIzMTE2N10gTkVUOiBSZWdpc3RlcmVkIHByb3RvY29sIGZhbWlseSAxMApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIzOTMwOF0gU2VnbWVudCBSb3V0aW5nIHdpdGggSVB2
NgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjIzOTMyMl0gbWlwNjogTW9i
aWxlIElQdjYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzkzMjRdIE5F
VDogUmVnaXN0ZXJlZCBwcm90b2NvbCBmYW1pbHkgMTcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yMzkzNDhdIG1wbHNfZ3NvOiBNUExTIEdTTyBzdXBwb3J0ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjM5Mzk3XSByZWdpc3RlcmVkIHRhc2tzdGF0
cyB2ZXJzaW9uIDEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yMzk0MDBd
IExvYWRpbmcgY29tcGlsZWQtaW4gWC41MDkgY2VydGlmaWNhdGVzClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuMjY1NTkyXSBMb2FkZWQgWC41MDkgY2VydCAnRGViaWFuIFNl
Y3VyZSBCb290IENBOiA2Y2NlY2U3ZTRjNmMwZDFmNjE0OWYzZGQyN2RmY2M1Y2JiNDE5ZWExJwpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NTYwM10gTG9hZGVkIFguNTA5
IGNlcnQgJ0RlYmlhbiBTZWN1cmUgQm9vdCBTaWduZXIgMjAyMSAtIGxpbnV4OiA0YjZlZjVhYmNh
NjY5ODI1MTc4ZTA1MmM4NDY2N2NjYmMwNTMxZjhjJwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAxLjI2NTc1OF0genN3YXA6IGxvYWRlZCB1c2luZyBwb29sIGx6by96YnVkClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjY1ODUxXSBLZXkgdHlwZSAuX2Zz
Y3J5cHQgcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2
NTg1Ml0gS2V5IHR5cGUgLmZzY3J5cHQgcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAxLjI2NTg1M10gS2V5IHR5cGUgZnNjcnlwdC1wcm92aXNpb25pbmcgcmVn
aXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI2NTg4Ml0gQXBw
QXJtb3I6IEFwcEFybW9yIHNoYTEgcG9saWN5IGhhc2hpbmcgZW5hYmxlZApTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3NDkzMF0gaW50ZWdyaXR5OiBMb2FkaW5nIFguNTA5
IGNlcnRpZmljYXRlOiBVRUZJOmRiClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuMjc1NTYxXSBpbnRlZ3JpdHk6IExvYWRlZCBYLjUwOSBjZXJ0ICdTVVNFIExpbnV4IEVudGVy
cHJpc2UgU2VjdXJlIEJvb3QgQ0E6IDNkNGQ0MGNmOTM4NTM5MDI0YjFjZmM1YTEyZGVkZmU4YjE3
ZTc1NWYnClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjc1NTYyXSBpbnRl
Z3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNhdGU6IFVFRkk6ZGIKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNzU3MjNdIGludGVncml0eTogTG9hZGVkIFguNTA5IGNl
cnQgJ0Nhbm9uaWNhbCBMdGQuIE1hc3RlciBDZXJ0aWZpY2F0ZSBBdXRob3JpdHk6IGFkOTE5OTBi
YzIyYWIxZjUxNzA0OGMyM2I2NjU1YTI2OGUzNDVhNjMnClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjc1NzIzXSBpbnRlZ3JpdHk6IExvYWRpbmcgWC41MDkgY2VydGlmaWNh
dGU6IFVFRkk6ZGIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNzU4ODFd
IGludGVncml0eTogTG9hZGVkIFguNTA5IGNlcnQgJ0ZlZG9yYSBTZWN1cmUgQm9vdCBDQTogZmRl
MzI1OTljMmQ2MWRiMWJmNTgwNzMzNWQ3YjIwZTRjZDk2M2I0MicKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yNzU4ODFdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0
aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3
NTg5NF0gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJv
ZHVjdGlvbiBQQ0EgMjAxMTogYTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1
MycKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yNzU4OTRdIGludGVncml0
eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjI3NTkwN10gaW50ZWdyaXR5OiBMb2FkZWQgWC41MDkgY2VydCAn
TWljcm9zb2Z0IENvcnBvcmF0aW9uIFVFRkkgQ0EgMjAxMTogMTNhZGJmNDMwOWJkODI3MDljOGNk
NTRmMzE2ZWQ1MjI5ODhhMWJkNCcKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yNzU5MDhdIGludGVncml0eTogTG9hZGluZyBYLjUwOSBjZXJ0aWZpY2F0ZTogVUVGSTpkYgpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI3NTkxOV0gaW50ZWdyaXR5OiBM
b2FkZWQgWC41MDkgY2VydCAnTWljcm9zb2Z0IFdpbmRvd3MgUHJvZHVjdGlvbiBQQ0EgMjAxMTog
YTkyOTAyMzk4ZTE2YzQ5Nzc4Y2Q5MGY5OWU0ZjlhZTE3YzU1YWY1MycKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODAzMzVdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yODA2MzFdIHBjaWVwb3J0IDAwMGM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEw
NgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4MDcwNF0gcGNpZXBvcnQg
MDAwZDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjI4MDk3MF0gcGNpZXBvcnQgMDAwZDowMDowMS4wOiBBRVI6IGVu
YWJsZWQgd2l0aCBJUlEgMTA3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
MjgxMDE0XSBwY2llcG9ydCAwMDBkOjAwOjAyLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjgxMjU5XSBwY2llcG9ydCAwMDBk
OjAwOjAyLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDcKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yODEzMDVdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODE1
NDVdIHBjaWVwb3J0IDAwMGQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwNwpTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4MTU4N10gcGNpZXBvcnQgMDAwZDow
MDowNC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAxLjI4MTgyNF0gcGNpZXBvcnQgMDAwZDowMDowNC4wOiBBRVI6IGVuYWJsZWQg
d2l0aCBJUlEgMTA3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjgxODg0
XSBwY2llcG9ydCAwMDAxOjAwOjAxLjA6IEFkZGluZyB0byBpb21tdSBncm91cCA1ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjgyMTQ5XSBwY2llcG9ydCAwMDAxOjAwOjAx
LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMDgKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS4yODIyMjBdIHBjaWVwb3J0IDAwMDA6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODI0NzhdIHBj
aWVwb3J0IDAwMDA6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEwOQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4MjU0N10gcGNpZXBvcnQgMDAwMjowMDowMS4w
OiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgNwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAxLjI4MjgyN10gcGNpZXBvcnQgMDAwMjowMDowMS4wOiBBRVI6IGVuYWJsZWQgd2l0aCBJ
UlEgMTEwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjgyODc4XSBwY2ll
cG9ydCAwMDAyOjAwOjA1LjA6IEFkZGluZyB0byBpb21tdSBncm91cCA4ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjgzMTI3XSBwY2llcG9ydCAwMDAyOjAwOjA1LjA6IEFF
UjogZW5hYmxlZCB3aXRoIElSUSAxMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMS4yODMxOTVdIHBjaWVwb3J0IDAwMDM6MDA6MDEuMDogQWRkaW5nIHRvIGlvbW11IGdyb3Vw
IDkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODM0NjRdIHBjaWVwb3J0
IDAwMDM6MDA6MDEuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMQpTZXAgMTQgMTA6MzA6MTEg
ZGViaWFuIGtlcm5lbDogWyAgICAxLjI4MzUxOF0gcGNpZXBvcnQgMDAwMzowMDowNS4wOiBBZGRp
bmcgdG8gaW9tbXUgZ3JvdXAgMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS4yODM3NjFdIHBjaWVwb3J0IDAwMDM6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDEx
MQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4MzgyNV0gcGNpZXBvcnQg
MDAwNDowMDowMS4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTEKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS4yODQxMDldIHBjaWVwb3J0IDAwMDQ6MDA6MDEuMDogQUVSOiBl
bmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjI4NDE1NV0gcGNpZXBvcnQgMDAwNDowMDowMy4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTIK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODQ0MDBdIHBjaWVwb3J0IDAw
MDQ6MDA6MDMuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjI4NDQ0OF0gcGNpZXBvcnQgMDAwNDowMDowNS4wOiBBZGRpbmcg
dG8gaW9tbXUgZ3JvdXAgMTMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4y
ODQ2OTNdIHBjaWVwb3J0IDAwMDQ6MDA6MDUuMDogQUVSOiBlbmFibGVkIHdpdGggSVJRIDExMgpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjI4NDc2NV0gcGNpZXBvcnQgMDAw
NDowMTowMC4wOiBBZGRpbmcgdG8gaW9tbXUgZ3JvdXAgMTQKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMS4yODUwMjhdIHBjaWVwb3J0IDAwMDU6MDA6MDEuMDogQWRkaW5nIHRv
IGlvbW11IGdyb3VwIDE1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjg1
MzE0XSBwY2llcG9ydCAwMDA1OjAwOjAxLjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODUzNjZdIHBjaWVwb3J0IDAwMDU6
MDA6MDUuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuMjg1NjA5XSBwY2llcG9ydCAwMDA1OjAwOjA1LjA6IEFFUjogZW5hYmxl
ZCB3aXRoIElSUSAxMTMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS4yODU2
NjNdIHBjaWVwb3J0IDAwMDU6MDA6MDcuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE3ClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuMjg1OTExXSBwY2llcG9ydCAwMDA1OjAw
OjA3LjA6IEFFUjogZW5hYmxlZCB3aXRoIElSUSAxMTMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS4yODcyODldIEZyZWVpbmcgdW51c2VkIGtlcm5lbCBtZW1vcnk6IDU0NDBL
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDI2MTU5XSBDaGVja2VkIFcr
WCBtYXBwaW5nczogcGFzc2VkLCBubyBXK1ggcGFnZXMgZm91bmQKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS40MjYxNjVdIFJ1biAvaW5pdCBhcyBpbml0IHByb2Nlc3MKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MjYxNjZdICAgd2l0aCBhcmd1bWVu
dHM6ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDI2MTY3XSAgICAgL2lu
aXQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MjYxNjddICAgd2l0aCBl
bnZpcm9ubWVudDoKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MjYxNjhd
ICAgICBIT01FPS8KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS40MjYxNjld
ICAgICBURVJNPWxpbnV4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDI2
MTY5XSAgICAgQk9PVF9JTUFHRT0vYm9vdC92bWxpbnV6LTUuMTAuMC04LWFybTY0ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNDk5ODExXSBwcHNfY29yZTogTGludXhQUFMg
QVBJIHZlci4gMSByZWdpc3RlcmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNDk5ODEzXSBwcHNfY29yZTogU29mdHdhcmUgdmVyLiA1LjMuNiAtIENvcHlyaWdodCAyMDA1
LTIwMDcgUm9kb2xmbyBHaW9tZXR0aSA8Z2lvbWV0dGlAbGludXguaXQ+ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTAwODM2XSBQVFAgY2xvY2sgc3VwcG9ydCByZWdpc3Rl
cmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTA4ODcwXSBpZ2I6IElu
dGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0d29yayBEcml2ZXIKU2VwIDE0IDEwOjMwOjExIGRl
YmlhbiBrZXJuZWw6IFsgICAgMS41MDg4NzFdIGlnYjogQ29weXJpZ2h0IChjKSAyMDA3LTIwMTQg
SW50ZWwgQ29ycG9yYXRpb24uClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTA5MjEzXSBTQ1NJIHN1YnN5c3RlbSBpbml0aWFsaXplZApTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAxLjUxMDE4N10gaWdiIDAwMDM6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11
IGdyb3VwIDE4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTExMDI4XSBB
Q1BJOiBidXMgdHlwZSBVU0IgcmVnaXN0ZXJlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjUxMTA0N10gdXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZl
ciB1c2JmcwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjUxMTA1Ml0gdXNi
Y29yZTogcmVnaXN0ZXJlZCBuZXcgaW50ZXJmYWNlIGRyaXZlciBodWIKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MTEwNzVdIHVzYmNvcmU6IHJlZ2lzdGVyZWQgbmV3IGRl
dmljZSBkcml2ZXIgdXNiClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTEy
MDgxXSBudm1lIDAwMGQ6MDE6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDE5ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTEyMzYzXSBudm1lIG52bWUwOiBwY2kgZnVu
Y3Rpb24gMDAwZDowMTowMC4wClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
NTEyNDE3XSBudm1lIDAwMGQ6MDI6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTEyODIyXSBudm1lIG52bWUxOiBwY2kg
ZnVuY3Rpb24gMDAwZDowMjowMC4wClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNTEyODczXSBudm1lIDAwMGQ6MDM6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIxClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTEzMTIzXSBudm1lIG52bWUyOiBw
Y2kgZnVuY3Rpb24gMDAwZDowMzowMC4wClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuNTEzMTc1XSBudm1lIDAwMGQ6MDQ6MDAuMDogQWRkaW5nIHRvIGlvbW11IGdyb3VwIDIy
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTEzNDEyXSBudm1lIG52bWUz
OiBwY2kgZnVuY3Rpb24gMDAwZDowNDowMC4wClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuNTIwMzk0XSBtcHQzc2FzIHZlcnNpb24gMzUuMTAwLjAwLjAwIGxvYWRlZApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMDUyNF0gbXB0M3NhcyAwMDBjOjAx
OjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91cCAyMwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAxLjUyMDg1MV0gbXB0M3Nhc19jbTA6IDYzIEJJVCBQQ0kgQlVTIERNQSBBRERS
RVNTSU5HIFNVUFBPUlRFRCwgdG90YWwgbWVtICg2NTMwNDc4MCBrQikKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjExODRdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogQWRk
aW5nIHRvIGlvbW11IGdyb3VwIDI0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuNTIxNDYwXSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTIxNDY1XSB4aGNpX2hjZCAwMDA0
OjAzOjAwLjA6IG5ldyBVU0IgYnVzIHJlZ2lzdGVyZWQsIGFzc2lnbmVkIGJ1cyBudW1iZXIgMQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyMTQ4M10geGhjaV9oY2QgMDAw
NDowMzowMC4wOiBaZXJvaW5nIDY0Yml0IGJhc2UgcmVnaXN0ZXJzLCBleHBlY3RpbmcgZmF1bHQK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjE4MzBdIHJhbmRvbTogZmFz
dCBpbml0IGRvbmUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjM1MzZd
IG52bWUgbnZtZTE6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjQwNDRdIG52bWUgbnZtZTA6IDgwLzAvMCBkZWZh
dWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS41MjQxNTNdIG52bWUgbnZtZTI6IDgwLzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS41MjQxNTddIG52bWUgbnZtZTM6IDgw
LzAvMCBkZWZhdWx0L3JlYWQvcG9sbCBxdWV1ZXMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS41MjY2NTVdIGFzdCAwMDA0OjAyOjAwLjA6IEFkZGluZyB0byBpb21tdSBncm91
cCAxNApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjUyNzI1OF0gIG52bWUx
bjE6IHAxIHAyIHAzIHA0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNTI3
NDcwXSAgbnZtZTJuMTogcDEgcDIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAg
MS41Mjc3MDZdICBudm1lMG4xOgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAx
LjUyNzg4MV0gIG52bWUzbjE6IHAxIHAyIHAzIHA0IHA1IHA2ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuNjc4NjQ2XSB4aGNpX2hjZCAwMDA0OjAzOjAwLjA6IGhjYyBwYXJh
bXMgMHgwMTQwNTFjZiBoY2kgdmVyc2lvbiAweDEwMCBxdWlya3MgMHgwMDAwMDAxMTAwMDAwNDEw
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc4OTk2XSB1c2IgdXNiMTog
TmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlkUHJvZHVjdD0wMDAyLCBiY2RE
ZXZpY2U9IDUuMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Nzg5OThd
IHVzYiB1c2IxOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MywgUHJvZHVjdD0yLCBTZXJp
YWxOdW1iZXI9MQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3ODk5OV0g
dXNiIHVzYjE6IFByb2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuNjc4OTk5XSB1c2IgdXNiMTogTWFudWZhY3R1cmVyOiBMaW51
eCA1LjEwLjAtOC1hcm02NCB4aGNpLWhjZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAxLjY3OTAwMF0gdXNiIHVzYjE6IFNlcmlhbE51bWJlcjogMDAwNDowMzowMC4wClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5MTA5XSBodWIgMS0wOjEuMDogVVNC
IGh1YiBmb3VuZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTEyMV0g
aHViIDEtMDoxLjA6IDQgcG9ydHMgZGV0ZWN0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMS42NzkyNzJdIHhoY2lfaGNkIDAwMDQ6MDM6MDAuMDogeEhDSSBIb3N0IENvbnRy
b2xsZXIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42NzkyNzRdIHhoY2lf
aGNkIDAwMDQ6MDM6MDAuMDogbmV3IFVTQiBidXMgcmVnaXN0ZXJlZCwgYXNzaWduZWQgYnVzIG51
bWJlciAyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5Mjc3XSB4aGNp
X2hjZCAwMDA0OjAzOjAwLjA6IEhvc3Qgc3VwcG9ydHMgVVNCIDMuMCBTdXBlclNwZWVkClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5MzI0XSB1c2IgdXNiMjogV2UgZG9u
J3Qga25vdyB0aGUgYWxnb3JpdGhtcyBmb3IgTFBNIGZvciB0aGlzIGhvc3QsIGRpc2FibGluZyBM
UE0uClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5MzM2XSB1c2IgdXNi
MjogTmV3IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTFkNmIsIGlkUHJvZHVjdD0wMDAzLCBi
Y2REZXZpY2U9IDUuMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Nzkz
MzddIHVzYiB1c2IyOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MywgUHJvZHVjdD0yLCBT
ZXJpYWxOdW1iZXI9MQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTMz
N10gdXNiIHVzYjI6IFByb2R1Y3Q6IHhIQ0kgSG9zdCBDb250cm9sbGVyClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5MzM4XSB1c2IgdXNiMjogTWFudWZhY3R1cmVyOiBM
aW51eCA1LjEwLjAtOC1hcm02NCB4aGNpLWhjZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjY3OTMzOV0gdXNiIHVzYjI6IFNlcmlhbE51bWJlcjogMDAwNDowMzowMC4wClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5MzYzXSBjaGVja2luZyBnZW5l
cmljICgyMDAwMDAwMCAxZDUwMDApIHZzIGh3ICgyMDAwMDAwMCAyMDAwMDAwKQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTM2NF0gZmIwOiBzd2l0Y2hpbmcgdG8gYXN0
ZHJtZmIgZnJvbSBFRkkgVkdBClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
Njc5NDI3XSBDb25zb2xlOiBzd2l0Y2hpbmcgdG8gY29sb3VyIGR1bW15IGRldmljZSA4MHgyNQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTQ3Nl0gYXN0IDAwMDQ6MDI6
MDAuMDogW2RybV0gcGxhdGZvcm0gaGFzIG5vIElPIHNwYWNlLCB0cnlpbmcgTU1JTwpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTQ4M10gYXN0IDAwMDQ6MDI6MDAuMDog
W2RybV0gUDJBIGJyaWRnZSBkaXNhYmxlZCwgdXNpbmcgZGVmYXVsdCBjb25maWd1cmF0aW9uClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5NDg0XSBhc3QgMDAwNDowMjow
MC4wOiBbZHJtXSBBU1QgMjUwMCBkZXRlY3RlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjY3OTQ4OF0gYXN0IDAwMDQ6MDI6MDAuMDogW2RybV0gQW5hbG9nIFZHQSBvbmx5
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5NDg5XSBhc3QgMDAwNDow
MjowMC4wOiBbZHJtXSBkcmFtIE1DTEs9ODAwIE1oeiB0eXBlPTEgYnVzX3dpZHRoPTE2ClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuNjc5NTE4XSBodWIgMi0wOjEuMDogVVNC
IGh1YiBmb3VuZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTUzOV0g
W1RUTV0gWm9uZSAga2VybmVsOiBBdmFpbGFibGUgZ3JhcGhpY3MgbWVtb3J5OiAzMjY1MjM5MCBL
aUIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Nzk1MzldIFtUVE1dIFpv
bmUgICBkbWEzMjogQXZhaWxhYmxlIGdyYXBoaWNzIG1lbW9yeTogMjA5NzE1MiBLaUIKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Nzk1NDBdIFtUVE1dIEluaXRpYWxpemlu
ZyBwb29sIGFsbG9jYXRvcgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3
OTU0NF0gW1RUTV0gSW5pdGlhbGl6aW5nIERNQSBwb29sIGFsbG9jYXRvcgpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjY3OTcxMF0gaHViIDItMDoxLjA6IDQgcG9ydHMgZGV0
ZWN0ZWQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS42Nzk3MzhdIFtkcm1d
IEluaXRpYWxpemVkIGFzdCAwLjEuMCAyMDEyMDIyOCBmb3IgMDAwNDowMjowMC4wIG9uIG1pbm9y
IDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTM1ODNdIENvbnNvbGU6
IHN3aXRjaGluZyB0byBjb2xvdXIgZnJhbWUgYnVmZmVyIGRldmljZSAyNDB4NjcKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS43OTYyNDNdIGFzdCAwMDA0OjAyOjAwLjA6IFtk
cm1dIGZiMDogYXN0ZHJtZmIgZnJhbWUgYnVmZmVyIGRldmljZQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAxLjgyNDE3MV0gaWdiIDAwMDM6MDI6MDAuMDogYWRkZWQgUEhDIG9u
IGV0aDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44MjQxNzNdIGlnYiAw
MDAzOjAyOjAwLjA6IEludGVsKFIpIEdpZ2FiaXQgRXRoZXJuZXQgTmV0d29yayBDb25uZWN0aW9u
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODI0MTc0XSBpZ2IgMDAwMzow
MjowMC4wOiBldGgwOiAoUENJZTo1LjBHYi9zOldpZHRoIHgyKSBkODo1ZTpkMzowMjpiMTo5MwpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjgyNDI1Nl0gaWdiIDAwMDM6MDI6
MDAuMDogZXRoMDogUEJBIE5vOiAxMDYzMDAtMDAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODI0MjU3XSBpZ2IgMDAwMzowMjowMC4wOiBVc2luZyBNU0ktWCBpbnRlcnJ1
cHRzLiA4IHJ4IHF1ZXVlKHMpLCA4IHR4IHF1ZXVlKHMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODI0MzQyXSBpZ2IgMDAwMzowMjowMC4xOiBBZGRpbmcgdG8gaW9tbXUg
Z3JvdXAgMjUKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44Nzg2MjBdIGln
YiAwMDAzOjAyOjAwLjE6IGFkZGVkIFBIQyBvbiBldGgxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODc4NjIxXSBpZ2IgMDAwMzowMjowMC4xOiBJbnRlbChSKSBHaWdhYml0
IEV0aGVybmV0IE5ldHdvcmsgQ29ubmVjdGlvbgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg3ODYyMl0gaWdiIDAwMDM6MDI6MDAuMTogZXRoMTogKFBDSWU6NS4wR2IvczpX
aWR0aCB4MikgZDg6NWU6ZDM6MDI6YjE6OTQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44Nzg3MDRdIGlnYiAwMDAzOjAyOjAwLjE6IGV0aDE6IFBCQSBObzogMTA2MzAwLTAw
MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3ODcwNV0gaWdiIDAwMDM6
MDI6MDAuMTogVXNpbmcgTVNJLVggaW50ZXJydXB0cy4gOCByeCBxdWV1ZShzKSwgOCB0eCBxdWV1
ZShzKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjg3OTc0OV0gaWdiIDAw
MDM6MDI6MDAuMSBlbm8yOiByZW5hbWVkIGZyb20gZXRoMQpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAxLjg4NDA0MF0gbXB0M3Nhc19jbTA6IEN1cnJlbnRIb3N0UGFnZVNpemUg
aXMgMDogU2V0dGluZyBkZWZhdWx0IGhvc3QgcGFnZSBzaXplIHRvIDRrClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg0MDU4XSBtcHQzc2FzX2NtMDogTVNJLVggdmVjdG9y
cyBzdXBwb3J0ZWQ6IDk2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg0
MDU4XSAJIG5vIG9mIGNvcmVzOiA4MCwgbWF4X21zaXhfdmVjdG9yczogLTEKU2VwIDE0IDEwOjMw
OjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODQwNTldIG1wdDNzYXNfY20wOiAgMCA4MApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjg4NTM4M10gbXB0M3Nhc19jbTA6IEhp
Z2ggSU9QcyBxdWV1ZXMgOiBkaXNhYmxlZApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAxLjg4NTM4NF0gbXB0M3NhczAtbXNpeDA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY1
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzg1XSBtcHQzc2FzMC1t
c2l4MTogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NjYKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMS44ODUzODVdIG1wdDNzYXMwLW1zaXgyOiBQQ0ktTVNJLVggZW5hYmxlZDog
SVJRIDQ2NwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjg4NTM4Nl0gbXB0
M3NhczAtbXNpeDM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDY4ClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzg2XSBtcHQzc2FzMC1tc2l4NDogUENJLU1TSS1YIGVu
YWJsZWQ6IElSUSA0NjkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODUz
ODddIG1wdDNzYXMwLW1zaXg1OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MApTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjg4NTM4OF0gbXB0M3NhczAtbXNpeDY6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDcxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODg1Mzg4XSBtcHQzc2FzMC1tc2l4NzogUENJLU1TSS1YIGVuYWJsZWQ6IElSUSA0NzIKU2Vw
IDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS44ODUzODldIG1wdDNzYXMwLW1zaXg4
OiBQQ0ktTVNJLVggZW5hYmxlZDogSVJRIDQ3MwpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAxLjg4NTM4OV0gbXB0M3NhczAtbXNpeDk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDc0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1MzkwXSBtcHQzc2Fz
MC1tc2l4MTA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc1ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODg1MzkwXSBtcHQzc2FzMC1tc2l4MTE6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDc2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzkx
XSBtcHQzc2FzMC1tc2l4MTI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc3ClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1MzkxXSBtcHQzc2FzMC1tc2l4MTM6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDc4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODg1MzkyXSBtcHQzc2FzMC1tc2l4MTQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDc5ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1MzkzXSBtcHQzc2FzMC1tc2l4
MTU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODg1MzkzXSBtcHQzc2FzMC1tc2l4MTY6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNDgxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk0XSBtcHQz
c2FzMC1tc2l4MTc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDgyClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk0XSBtcHQzc2FzMC1tc2l4MTg6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNDgzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1
Mzk1XSBtcHQzc2FzMC1tc2l4MTk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg0ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk1XSBtcHQzc2FzMC1tc2l4MjA6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNDg1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODg1Mzk2XSBtcHQzc2FzMC1tc2l4MjE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg2
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk2XSBtcHQzc2FzMC1t
c2l4MjI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODg1Mzk3XSBtcHQzc2FzMC1tc2l4MjM6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNDg4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk4XSBt
cHQzc2FzMC1tc2l4MjQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDg5ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk4XSBtcHQzc2FzMC1tc2l4MjU6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNDkwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODg1Mzk5XSBtcHQzc2FzMC1tc2l4MjY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkxClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1Mzk5XSBtcHQzc2FzMC1tc2l4Mjc6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDkyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODg1NDAwXSBtcHQzc2FzMC1tc2l4Mjg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NDkzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDAwXSBtcHQzc2Fz
MC1tc2l4Mjk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk0ClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODg1NDAxXSBtcHQzc2FzMC1tc2l4MzA6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNDk1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDAx
XSBtcHQzc2FzMC1tc2l4MzE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk2ClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDAyXSBtcHQzc2FzMC1tc2l4MzI6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNDk3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODg1NDAzXSBtcHQzc2FzMC1tc2l4MzM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk4ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA0XSBtcHQzc2FzMC1tc2l4
MzQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNDk5ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODg1NDA0XSBtcHQzc2FzMC1tc2l4MzU6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTAwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA1XSBtcHQz
c2FzMC1tc2l4MzY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAxClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA1XSBtcHQzc2FzMC1tc2l4Mzc6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTAyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1
NDA2XSBtcHQzc2FzMC1tc2l4Mzg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTAzClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA2XSBtcHQzc2FzMC1tc2l4Mzk6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTA0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODg1NDA3XSBtcHQzc2FzMC1tc2l4NDA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA1
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA4XSBtcHQzc2FzMC1t
c2l4NDE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODg1NDA4XSBtcHQzc2FzMC1tc2l4NDI6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTA3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA5XSBt
cHQzc2FzMC1tc2l4NDM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTA4ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDA5XSBtcHQzc2FzMC1tc2l4NDQ6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTA5ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODg1NDEwXSBtcHQzc2FzMC1tc2l4NDU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTEwClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDExXSBtcHQzc2FzMC1tc2l4NDY6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTExClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODg1NDExXSBtcHQzc2FzMC1tc2l4NDc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTEyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDEyXSBtcHQzc2Fz
MC1tc2l4NDg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTEzClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODg1NDEyXSBtcHQzc2FzMC1tc2l4NDk6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTE0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDEz
XSBtcHQzc2FzMC1tc2l4NTA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE1ClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE0XSBtcHQzc2FzMC1tc2l4NTE6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTE2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODg1NDE0XSBtcHQzc2FzMC1tc2l4NTI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE3ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE1XSBtcHQzc2FzMC1tc2l4
NTM6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTE4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODg1NDE2XSBtcHQzc2FzMC1tc2l4NTQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTE5ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE2XSBtcHQz
c2FzMC1tc2l4NTU6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIwClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE3XSBtcHQzc2FzMC1tc2l4NTY6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTIxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1
NDE3XSBtcHQzc2FzMC1tc2l4NTc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTIyClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE4XSBtcHQzc2FzMC1tc2l4NTg6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTIzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODg1NDE5XSBtcHQzc2FzMC1tc2l4NTk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI0
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDE5XSBtcHQzc2FzMC1t
c2l4NjA6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODg1NDIwXSBtcHQzc2FzMC1tc2l4NjE6IFBDSS1NU0ktWCBlbmFibGVk
OiBJUlEgNTI2ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDIwXSBt
cHQzc2FzMC1tc2l4NjI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI3ClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDIxXSBtcHQzc2FzMC1tc2l4NjM6IFBDSS1NU0kt
WCBlbmFibGVkOiBJUlEgNTI4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEu
ODg1NDIxXSBtcHQzc2FzMC1tc2l4NjQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTI5ClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDIyXSBtcHQzc2FzMC1tc2l4NjU6
IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDEuODg1NDIzXSBtcHQzc2FzMC1tc2l4NjY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEg
NTMxClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI0XSBtcHQzc2Fz
MC1tc2l4Njc6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTMyClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDEuODg1NDI0XSBtcHQzc2FzMC1tc2l4Njg6IFBDSS1NU0ktWCBlbmFi
bGVkOiBJUlEgNTMzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI1
XSBtcHQzc2FzMC1tc2l4Njk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM0ClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI1XSBtcHQzc2FzMC1tc2l4NzA6IFBDSS1N
U0ktWCBlbmFibGVkOiBJUlEgNTM1ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDEuODg1NDI2XSBtcHQzc2FzMC1tc2l4NzE6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM2ClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI3XSBtcHQzc2FzMC1tc2l4
NzI6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM3ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2Vy
bmVsOiBbICAgIDEuODg1NDI3XSBtcHQzc2FzMC1tc2l4NzM6IFBDSS1NU0ktWCBlbmFibGVkOiBJ
UlEgNTM4ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI4XSBtcHQz
c2FzMC1tc2l4NzQ6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTM5ClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDI4XSBtcHQzc2FzMC1tc2l4NzU6IFBDSS1NU0ktWCBl
bmFibGVkOiBJUlEgNTQwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1
NDI5XSBtcHQzc2FzMC1tc2l4NzY6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQxClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDMwXSBtcHQzc2FzMC1tc2l4Nzc6IFBD
SS1NU0ktWCBlbmFibGVkOiBJUlEgNTQyClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDEuODg1NDMwXSBtcHQzc2FzMC1tc2l4Nzg6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQz
ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDMxXSBtcHQzc2FzMC1t
c2l4Nzk6IFBDSS1NU0ktWCBlbmFibGVkOiBJUlEgNTQ0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDEuODg1NDMyXSBtcHQzc2FzX2NtMDogaW9tZW0oMHgwMDAwMDAwMDQwMTQw
MDAwKSwgbWFwcGVkKDB4KF9fX19wdHJ2YWxfX19fKSksIHNpemUoNjU1MzYpClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDEuODg1NDMzXSBtcHQzc2FzX2NtMDogaW9wb3J0KDB4
MDAwMDAwMDAwMDAwMDAwMCksIHNpemUoMCkKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMS44OTMxOTFdIGlnYiAwMDAzOjAyOjAwLjAgZW5vMTogcmVuYW1lZCBmcm9tIGV0aDAK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMS45NDI1MzddIG1wdDNzYXNfY20w
OiBDdXJyZW50SG9zdFBhZ2VTaXplIGlzIDA6IFNldHRpbmcgZGVmYXVsdCBob3N0IHBhZ2Ugc2l6
ZSB0byA0awpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAxLjk0MjUzOV0gbXB0
M3Nhc19jbTA6IHNlbmRpbmcgZGlhZyByZXNldCAhIQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAxLjk5NzA1MV0gdXNiIDEtMTogbmV3IGxvdy1zcGVlZCBVU0IgZGV2aWNlIG51
bWJlciAyIHVzaW5nIHhoY2lfaGNkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAg
IDIuMTg5Mzg2XSB1c2IgMS0xOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9NDEzYywg
aWRQcm9kdWN0PTMwMWEsIGJjZERldmljZT0gMS4wMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAyLjE4OTM4N10gdXNiIDEtMTogTmV3IFVTQiBkZXZpY2Ugc3RyaW5nczogTWZy
PTEsIFByb2R1Y3Q9MiwgU2VyaWFsTnVtYmVyPTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJu
ZWw6IFsgICAgMi4xODkzODhdIHVzYiAxLTE6IFByb2R1Y3Q6IERlbGwgTVMxMTYgVVNCIE9wdGlj
YWwgTW91c2UKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4xODkzODldIHVz
YiAxLTE6IE1hbnVmYWN0dXJlcjogUGl4QXJ0ClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDIuMTk4NDYxXSBoaWQ6IHJhdyBISUQgZXZlbnRzIGRyaXZlciAoQykgSmlyaSBLb3Np
bmEKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yMTA0NDhdIHVzYmNvcmU6
IHJlZ2lzdGVyZWQgbmV3IGludGVyZmFjZSBkcml2ZXIgdXNiaGlkClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDIuMjEwNDQ5XSB1c2JoaWQ6IFVTQiBISUQgY29yZSBkcml2ZXIK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4yMTE3NDRdIGlucHV0OiBQaXhB
cnQgRGVsbCBNUzExNiBVU0IgT3B0aWNhbCBNb3VzZSBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAw
MDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNiMS8xLTEvMS0xOjEuMC8wMDAzOjQxM0M6MzAxQS4w
MDAxL2lucHV0L2lucHV0MQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjIx
MTc5Ml0gaGlkLWdlbmVyaWMgMDAwMzo0MTNDOjMwMUEuMDAwMTogaW5wdXQsaGlkcmF3MDogVVNC
IEhJRCB2MS4xMSBNb3VzZSBbUGl4QXJ0IERlbGwgTVMxMTYgVVNCIE9wdGljYWwgTW91c2VdIG9u
IHVzYi0wMDA0OjAzOjAwLjAtMS9pbnB1dDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMi4yOTc3MDFdIHVzYiAyLTM6IG5ldyBTdXBlclNwZWVkIEdlbiAxIFVTQiBkZXZpY2Ug
bnVtYmVyIDIgdXNpbmcgeGhjaV9oY2QKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsg
ICAgMi4zMjYxODddIHVzYiAyLTM6IE5ldyBVU0IgZGV2aWNlIGZvdW5kLCBpZFZlbmRvcj0wNWUz
LCBpZFByb2R1Y3Q9MDYyMCwgYmNkRGV2aWNlPTkzLjEwClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDIuMzI2MTg5XSB1c2IgMi0zOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBN
ZnI9MSwgUHJvZHVjdD0yLCBTZXJpYWxOdW1iZXI9MApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAyLjMyNjE4OV0gdXNiIDItMzogUHJvZHVjdDogVVNCMy4xIEh1YgpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjMyNjE5MF0gdXNiIDItMzogTWFudWZhY3R1
cmVyOiBHZW5lc3lzTG9naWMKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi4z
MjkyMDZdIGh1YiAyLTM6MS4wOiBVU0IgaHViIGZvdW5kClNlcCAxNCAxMDozMDoxMSBkZWJpYW4g
a2VybmVsOiBbICAgIDIuMzI5ODE3XSBodWIgMi0zOjEuMDogNCBwb3J0cyBkZXRlY3RlZApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjQ1NzAzMl0gdXNiIDEtMjogbmV3IGxv
dy1zcGVlZCBVU0IgZGV2aWNlIG51bWJlciAzIHVzaW5nIHhoY2lfaGNkClNlcCAxNCAxMDozMDox
MSBkZWJpYW4ga2VybmVsOiBbICAgIDIuNjY4Mzc4XSB1c2IgMS0yOiBOZXcgVVNCIGRldmljZSBm
b3VuZCwgaWRWZW5kb3I9MDRkOSwgaWRQcm9kdWN0PTEyMDMsIGJjZERldmljZT0gMi44MApTZXAg
MTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjY2ODM3OV0gdXNiIDEtMjogTmV3IFVT
QiBkZXZpY2Ugc3RyaW5nczogTWZyPTAsIFByb2R1Y3Q9MCwgU2VyaWFsTnVtYmVyPTAKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi43MDU0OTNdIGlucHV0OiBISUQgMDRkOTox
MjAzIGFzIC9kZXZpY2VzL3BjaTAwMDQ6MDAvMDAwNDowMDowMy4wLzAwMDQ6MDM6MDAuMC91c2Ix
LzEtMi8xLTI6MS4wLzAwMDM6MDREOToxMjAzLjAwMDIvaW5wdXQvaW5wdXQyClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDIuNzY5MDk1XSBoaWQtZ2VuZXJpYyAwMDAzOjA0RDk6
MTIwMy4wMDAyOiBpbnB1dCxoaWRyYXcxOiBVU0IgSElEIHYxLjExIEtleWJvYXJkIFtISUQgMDRk
OToxMjAzXSBvbiB1c2ItMDAwNDowMzowMC4wLTIvaW5wdXQwClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDIuODExNDYyXSBpbnB1dDogSElEIDA0ZDk6MTIwMyBTeXN0ZW0gQ29u
dHJvbCBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0OjAzOjAwLjAvdXNi
MS8xLTIvMS0yOjEuMS8wMDAzOjA0RDk6MTIwMy4wMDAzL2lucHV0L2lucHV0MwpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjg3MzA0OV0gaW5wdXQ6IEhJRCAwNGQ5OjEyMDMg
Q29uc3VtZXIgQ29udHJvbCBhcyAvZGV2aWNlcy9wY2kwMDA0OjAwLzAwMDQ6MDA6MDMuMC8wMDA0
OjAzOjAwLjAvdXNiMS8xLTIvMS0yOjEuMS8wMDAzOjA0RDk6MTIwMy4wMDAzL2lucHV0L2lucHV0
NApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAyLjg3MzA3M10gaGlkLWdlbmVy
aWMgMDAwMzowNEQ5OjEyMDMuMDAwMzogaW5wdXQsaGlkcmF3MjogVVNCIEhJRCB2MS4xMSBEZXZp
Y2UgW0hJRCAwNGQ5OjEyMDNdIG9uIHVzYi0wMDA0OjAzOjAwLjAtMi9pbnB1dDEKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMi45ODA1NDFdIG1wdDNzYXNfY20wOiBkaWFnIHJl
c2V0OiBTVUNDRVNTClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDA5MDMy
XSB1c2IgMS0zOiBuZXcgaGlnaC1zcGVlZCBVU0IgZGV2aWNlIG51bWJlciA0IHVzaW5nIHhoY2lf
aGNkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMDE1MDYwXSBtcHQzc2Fz
X2NtMDogc2NhdHRlciBnYXRoZXI6IHNnZV9pbl9tYWluX21zZygxKSwgc2dlX3Blcl9jaGFpbig3
KSwgc2dlX3Blcl9pbygxMjgpLCBjaGFpbnNfcGVyX2lvKDE5KQpTZXAgMTQgMTA6MzA6MTEgZGVi
aWFuIGtlcm5lbDogWyAgICAzLjAxNjExN10gbXB0M3Nhc19jbTA6IHJlcXVlc3QgcG9vbCgweChf
X19fcHRydmFsX19fXykpIC0gZG1hKDB4ZmY1MDAwMDApOiBkZXB0aCg0MjI0KSwgZnJhbWVfc2l6
ZSgxMjgpLCBwb29sX3NpemUoNTI4IGtCKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDog
WyAgICAzLjAzNjcxMV0gbXB0M3Nhc19jbTA6IHNlbnNlIHBvb2woMHgoX19fX3B0cnZhbF9fX18p
KS0gZG1hKDB4ZmViMDAwMDApOiBkZXB0aCgzOTYzKSxlbGVtZW50X3NpemUoOTYpLCBwb29sX3Np
emUoMzcxIGtCKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjAzNjgxMV0g
bXB0M3Nhc19jbTA6IGNvbmZpZyBwYWdlKDB4KF9fX19wdHJ2YWxfX19fKSkgLSBkbWEoMHhmZTlm
NjAwMCk6IHNpemUoNTEyKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjAz
NjgxMl0gbXB0M3Nhc19jbTA6IEFsbG9jYXRlZCBwaHlzaWNhbCBtZW1vcnk6IHNpemUoNzI3OCBr
QikKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4wMzY4MTJdIG1wdDNzYXNf
Y20wOiBDdXJyZW50IENvbnRyb2xsZXIgUXVldWUgRGVwdGgoMzk2MCksTWF4IENvbnRyb2xsZXIg
UXVldWUgRGVwdGgoNDA5NikKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4w
MzY4MTNdIG1wdDNzYXNfY20wOiBTY2F0dGVyIEdhdGhlciBFbGVtZW50cyBwZXIgSU8oMTI4KQpT
ZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjE2NjU0N10gdXNiIDEtMzogTmV3
IFVTQiBkZXZpY2UgZm91bmQsIGlkVmVuZG9yPTA1ZTMsIGlkUHJvZHVjdD0wNjEwLCBiY2REZXZp
Y2U9OTMuMTAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNjY1NDldIHVz
YiAxLTM6IE5ldyBVU0IgZGV2aWNlIHN0cmluZ3M6IE1mcj0xLCBQcm9kdWN0PTIsIFNlcmlhbE51
bWJlcj0wClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMTY2NTQ5XSB1c2Ig
MS0zOiBQcm9kdWN0OiBVU0IyLjAgSHViClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuMTY2NTUwXSB1c2IgMS0zOiBNYW51ZmFjdHVyZXI6IEdlbmVzeXNMb2dpYwpTZXAgMTQg
MTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjE3MDU2OF0gaHViIDEtMzoxLjA6IFVTQiBo
dWIgZm91bmQKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy4xNzExNzJdIGh1
YiAxLTM6MS4wOiA0IHBvcnRzIGRldGVjdGVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDMuMjM3NjgxXSBtcHQzc2FzX2NtMDogX2Jhc2VfZGlzcGxheV9md3BrZ192ZXJzaW9u
OiBjb21wbGV0ZQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjIzODAxMV0g
bXB0M3Nhc19jbTA6IExTSVNBUzMwMDg6IEZXVmVyc2lvbigxNi4wMC4xMC4wMCksIENoaXBSZXZp
c2lvbigweDAyKSwgQmlvc1ZlcnNpb24oMTUuMDAuMDEuMDApClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDMuMjM4MDEyXSBtcHQzc2FzX2NtMDogUHJvdG9jb2w9KEluaXRpYXRv
ciksIENhcGFiaWxpdGllcz0oUmFpZCxUTFIsRUVEUCxTbmFwc2hvdCBCdWZmZXIsRGlhZyBUcmFj
ZSBCdWZmZXIsVGFzayBTZXQgRnVsbCxOQ1EpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDMuMjM4MDYxXSBzY3NpIGhvc3QwOiBGdXNpb24gTVBUIFNBUyBIb3N0ClNlcCAxNCAx
MDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMjM4NjM3XSBtcHQzc2FzX2NtMDogc2VuZGlu
ZyBwb3J0IGVuYWJsZSAhIQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjI2
NTUyMF0gUE06IEltYWdlIG5vdCBmb3VuZCAoY29kZSAtMjIpClNlcCAxNCAxMDozMDoxMSBkZWJp
YW4ga2VybmVsOiBbICAgIDMuMzIyOTg2XSBFWFQ0LWZzIChudm1lM24xcDIpOiBtb3VudGVkIGZp
bGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4gT3B0czogKG51bGwpClNlcCAxNCAxMDoz
MDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuMzU1MjI5XSBOb3QgYWN0aXZhdGluZyBNYW5kYXRv
cnkgQWNjZXNzIENvbnRyb2wgYXMgL3NiaW4vdG9tb3lvLWluaXQgZG9lcyBub3QgZXhpc3QuClNl
cCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNDg0OTM2XSByYW5kb206IGx2bWNv
bmZpZzogdW5pbml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDQgYnl0ZXMgcmVhZCkKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy41NTc0NTNdIHJhbmRvbTogc3lzdGVtZDogdW5p
bml0aWFsaXplZCB1cmFuZG9tIHJlYWQgKDE2IGJ5dGVzIHJlYWQpClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuNTU4MDQzXSByYW5kb206IHN5c3RlbWQ6IHVuaW5pdGlhbGl6
ZWQgdXJhbmRvbSByZWFkICgxNiBieXRlcyByZWFkKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAzLjU3NTMzOV0gZnVzZTogaW5pdCAoQVBJIHZlcnNpb24gNy4zMikKU2VwIDE0
IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy41NzcxNjVdIEVYVDQtZnMgKG52bWUzbjFw
Mik6IHJlLW1vdW50ZWQuIE9wdHM6IGVycm9ycz1yZW1vdW50LXJvClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuNjc1MjA3XSBhY3BpLXRhZCBBQ1BJMDAwRTowMDogVW5zdXBw
b3J0ZWQgY2FwYWJpbGl0aWVzClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMu
NzE0MjY4XSB4Z2VuZS1zbGltcHJvLWh3bW9uIEFQTUMwRDI5OjAwOiBBUE0gWC1HZW5lIFNvQyBI
VyBtb25pdG9yIGRyaXZlciByZWdpc3RlcmVkClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVs
OiBbICAgIDMuNzIyMzI1XSBJUE1JIG1lc3NhZ2UgaGFuZGxlcjogdmVyc2lvbiAzOS4yClNlcCAx
NCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuNzI1NTY3XSBpcG1pIGRldmljZSBpbnRl
cmZhY2UKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy43MzIwMTFdIGlwbWlf
c2k6IElQTUkgU3lzdGVtIEludGVyZmFjZSBkcml2ZXIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBr
ZXJuZWw6IFsgICAgMy43MzI0NDJdIGlwbWlfc2k6IFVuYWJsZSB0byBmaW5kIGFueSBTeXN0ZW0g
SW50ZXJmYWNlKHMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODA3NDE1
XSBpcG1pX3NzaWY6IElQTUkgU1NJRiBJbnRlcmZhY2UgZHJpdmVyClNlcCAxNCAxMDozMDoxMSBk
ZWJpYW4ga2VybmVsOiBbICAgIDMuODEwODIzXSBpcG1pX3NzaWYgaTJjLUFNUEMwMDA0OjAwOiBp
cG1pX3NzaWY6IFRyeWluZyBBQ1BJLXNwZWNpZmllZCBTU0lGIGludGVyZmFjZSBhdCBpMmMgYWRk
cmVzcyAweDEwLCBhZGFwdGVyIFN5bm9wc3lzIERlc2lnbldhcmUgSTJDIGFkYXB0ZXIsIHNsYXZl
IGFkZHJlc3MgMHgyMApTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjgyNzc0
OV0gRVhUNC1mcyAobnZtZTNuMXA1KTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBk
YXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAg
ICAzLjgyNzk2Ml0gRVhUNC1mcyAobnZtZTNuMXAzKTogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGgg
b3JkZXJlZCBkYXRhIG1vZGUuIE9wdHM6IChudWxsKQpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtl
cm5lbDogWyAgICAzLjgzNzA1N10gQWRkaW5nIDEwMDA0NDRrIHN3YXAgb24gL2Rldi9udm1lM24x
cDQuICBQcmlvcml0eTotMiBleHRlbnRzOjEgYWNyb3NzOjEwMDA0NDRrIFNTRlMKU2VwIDE0IDEw
OjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44NjE1MjFdIEVYVDQtZnMgKG52bWUzbjFwNik6
IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBtb2RlLiBPcHRzOiAobnVsbCkK
U2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy44ODg3MDNdIGlwbWlfc2k6IElQ
TUkgU3lzdGVtIEludGVyZmFjZSBkcml2ZXIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6
IFsgICAgMy44ODg5MDFdIGlwbWlfc2k6IFVuYWJsZSB0byBmaW5kIGFueSBTeXN0ZW0gSW50ZXJm
YWNlKHMpClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBbICAgIDMuODkyOTY3XSBwc3Rv
cmU6IFVzaW5nIGNyYXNoIGR1bXAgY29tcHJlc3Npb246IGRlZmxhdGUKU2VwIDE0IDEwOjMwOjEx
IGRlYmlhbiBrZXJuZWw6IFsgICAgMy44OTI5NzVdIHBzdG9yZTogUmVnaXN0ZXJlZCBlZmkgYXMg
cGVyc2lzdGVudCBzdG9yZSBiYWNrZW5kClNlcCAxNCAxMDozMDoxMSBkZWJpYW4ga2VybmVsOiBb
ICAgIDMuOTAzNzY4XSBhdWRpdDogdHlwZT0xNDAwIGF1ZGl0KDE2MzE1OTU2MTEuMTY4OjIpOiBh
cHBhcm1vcj0iU1RBVFVTIiBvcGVyYXRpb249InByb2ZpbGVfbG9hZCIgcHJvZmlsZT0idW5jb25m
aW5lZCIgbmFtZT0ibGlicmVvZmZpY2Utc2VuZGRvYyIgcGlkPTExMTcgY29tbT0iYXBwYXJtb3Jf
cGFyc2VyIgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNDUwN10gYXVk
aXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTk1NjExLjE2ODozKTogYXBwYXJtb3I9IlNUQVRVUyIg
b3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im52aWRp
YV9tb2Rwcm9iZSIgcGlkPTExMTkgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNDUxMF0gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgx
NjMxNTk1NjExLjE2ODo0KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xv
YWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im52aWRpYV9tb2Rwcm9iZS8va21vZCIgcGlk
PTExMTkgY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5l
bDogWyAgICAzLjkwNDY1N10gYXVkaXQ6IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTk1NjExLjE2ODo1
KTogYXBwYXJtb3I9IlNUQVRVUyIgb3BlcmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVu
Y29uZmluZWQiIG5hbWU9Ii91c3IvYmluL21hbiIgcGlkPTExMTUgY29tbT0iYXBwYXJtb3JfcGFy
c2VyIgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNDY2MF0gYXVkaXQ6
IHR5cGU9MTQwMCBhdWRpdCgxNjMxNTk1NjExLjE2ODo2KTogYXBwYXJtb3I9IlNUQVRVUyIgb3Bl
cmF0aW9uPSJwcm9maWxlX2xvYWQiIHByb2ZpbGU9InVuY29uZmluZWQiIG5hbWU9Im1hbl9maWx0
ZXIiIHBpZD0xMTE1IGNvbW09ImFwcGFybW9yX3BhcnNlciIKU2VwIDE0IDEwOjMwOjExIGRlYmlh
biBrZXJuZWw6IFsgICAgMy45MDQ2NjJdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTU5NTYx
MS4xNjg6Nyk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9m
aWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJtYW5fZ3JvZmYiIHBpZD0xMTE1IGNvbW09ImFwcGFybW9y
X3BhcnNlciIKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy45MDQ5ODRdIGF1
ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTU5NTYxMS4xNjg6OCk6IGFwcGFybW9yPSJTVEFUVVMi
IG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJ2aXJ0
LWFhLWhlbHBlciIgcGlkPTExMTggY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTQgMTA6MzA6
MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNTA4NV0gY3J5cHRkOiBtYXhfY3B1X3FsZW4gc2V0
IHRvIDEwMDAKU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy45MDU1MDNdIHNi
c2EtZ3dkdCBzYnNhLWd3ZHQuMDogSW5pdGlhbGl6ZWQgd2l0aCAxMHMgdGltZW91dCBAIDI1MDAw
MDAwIEh6LCBhY3Rpb249MC4KU2VwIDE0IDEwOjMwOjExIGRlYmlhbiBrZXJuZWw6IFsgICAgMy45
MDU1MjJdIGF1ZGl0OiB0eXBlPTE0MDAgYXVkaXQoMTYzMTU5NTYxMS4xNzI6OSk6IGFwcGFybW9y
PSJTVEFUVVMiIG9wZXJhdGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBu
YW1lPSJsaWJyZW9mZmljZS1vb3BzbGFzaCIgcGlkPTExMTMgY29tbT0iYXBwYXJtb3JfcGFyc2Vy
IgpTZXAgMTQgMTA6MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkwNzAyN10gYXVkaXQ6IHR5
cGU9MTQwMCBhdWRpdCgxNjMxNTk1NjExLjE3MjoxMCk6IGFwcGFybW9yPSJTVEFUVVMiIG9wZXJh
dGlvbj0icHJvZmlsZV9sb2FkIiBwcm9maWxlPSJ1bmNvbmZpbmVkIiBuYW1lPSJsc2JfcmVsZWFz
ZSIgcGlkPTExMDggY29tbT0iYXBwYXJtb3JfcGFyc2VyIgpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICAzLjkxNDM3OV0gcmFuZG9tOiBjcm5nIGluaXQgZG9uZQpTZXAgMTQgMTA6
MzA6MTEgZGViaWFuIGtlcm5lbDogWyAgICAzLjkxNDM4MV0gcmFuZG9tOiA3IHVyYW5kb20gd2Fy
bmluZyhzKSBtaXNzZWQgZHVlIHRvIHJhdGVsaW1pdGluZwpTZXAgMTQgMTA6MzA6MTEgZGViaWFu
IGtlcm5lbDogWyAgICA0LjA1OTIwNF0gaXBtaV9zc2lmIGkyYy1BTVBDMDAwNDowMDogSVBNSSBt
ZXNzYWdlIGhhbmRsZXI6IEZvdW5kIG5ldyBCTUMgKG1hbl9pZDogMHgwMDNjMGEsIHByb2RfaWQ6
IDB4MDIxMiwgZGV2X2lkOiAweDIwKQpTZXAgMTQgMTA6MzA6MTMgZGViaWFuIGtlcm5lbDogWyAg
ICA1Ljc5MDg2MF0gbXB0M3Nhc19jbTA6IGhvc3RfYWRkOiBoYW5kbGUoMHgwMDAxKSwgc2FzX2Fk
ZHIoMHg1MDAxNDg1MDAwNzc5ZTMwKSwgcGh5cyg4KQpTZXAgMTQgMTA6MzA6MTUgZGViaWFuIGtl
cm5lbDogWyAgICA3LjgxMDQ5MF0gaWdiIDAwMDM6MDI6MDAuMSBlbm8yOiBpZ2I6IGVubzIgTklD
IExpbmsgaXMgVXAgMTAwMCBNYnBzIEZ1bGwgRHVwbGV4LCBGbG93IENvbnRyb2w6IFJYL1RYClNl
cCAxNCAxMDozMDoxNSBkZWJpYW4ga2VybmVsOiBbICAgIDcuODEwNjYyXSBJUHY2OiBBRERSQ09O
RihORVRERVZfQ0hBTkdFKTogZW5vMjogbGluayBiZWNvbWVzIHJlYWR5ClNlcCAxNCAxMDozMDox
OCBkZWJpYW4ga2VybmVsOiBbICAgMTAuOTI5MDcxXSBtcHQzc2FzX2NtMDogcG9ydCBlbmFibGU6
IFNVQ0NFU1MKU2VwIDE0IDExOjEwOjM5IGRlYmlhbiBrZXJuZWw6IFsgMjQzMy4zOTQ0MDldIHVz
YiAyLTMuMTogbmV3IFN1cGVyU3BlZWQgR2VuIDEgVVNCIGRldmljZSBudW1iZXIgMyB1c2luZyB4
aGNpX2hjZApTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAyNDMzLjQzMzc5OF0gdXNi
IDItMy4xOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDc4MSwgaWRQcm9kdWN0PTU1
ODEsIGJjZERldmljZT0gMS4wMApTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAyNDMz
LjQzMzgwMl0gdXNiIDItMy4xOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MSwgUHJvZHVj
dD0yLCBTZXJpYWxOdW1iZXI9MwpTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAyNDMz
LjQzMzgwNF0gdXNiIDItMy4xOiBQcm9kdWN0OiAgU2FuRGlzayAzLjJHZW4xClNlcCAxNCAxMTox
MDozOSBkZWJpYW4ga2VybmVsOiBbIDI0MzMuNDMzODA3XSB1c2IgMi0zLjE6IE1hbnVmYWN0dXJl
cjogIFVTQgpTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAyNDMzLjQzMzgwOV0gdXNi
IDItMy4xOiBTZXJpYWxOdW1iZXI6IDA0MDFlYWFlNGY2ZWM5ZmE4OTlkOTBlYjZhZTgzOTE3MjNm
NzMyM2M1OWNkYjdiYmQ1M2EwZDYzYWI1YmNlNmRmMWZkMDAwMDAwMDAwMDAwMDAwMDAwMDA2MDRl
MWY1YjAwOGU2NzE4ODE1NTgxMDcxY2FiNDVlOQpTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5l
bDogWyAyNDMzLjQ0NjEyMF0gdXNiLXN0b3JhZ2UgMi0zLjE6MS4wOiBVU0IgTWFzcyBTdG9yYWdl
IGRldmljZSBkZXRlY3RlZApTZXAgMTQgMTE6MTA6MzkgZGViaWFuIGtlcm5lbDogWyAyNDMzLjQ0
NjI5M10gc2NzaSBob3N0MTogdXNiLXN0b3JhZ2UgMi0zLjE6MS4wClNlcCAxNCAxMToxMDozOSBk
ZWJpYW4ga2VybmVsOiBbIDI0MzMuNDQ2MzkwXSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRl
cmZhY2UgZHJpdmVyIHVzYi1zdG9yYWdlClNlcCAxNCAxMToxMDozOSBkZWJpYW4ga2VybmVsOiBb
IDI0MzMuNDQ4MTQ2XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVh
cwpTZXAgMTQgMTE6MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjQ2OTAwM10gc2NzaSAxOjA6
MDowOiBEaXJlY3QtQWNjZXNzICAgICAgVVNCICAgICAgU2FuRGlzayAzLjJHZW4xIDEuMDAgUFE6
IDAgQU5TSTogNgpTZXAgMTQgMTE6MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjQ3MzIxMl0g
c2NzaSAxOjA6MDowOiBBdHRhY2hlZCBzY3NpIGdlbmVyaWMgc2cwIHR5cGUgMApTZXAgMTQgMTE6
MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjQ3NTk2NF0gc2QgMTowOjA6MDogW3NkYV0gMTIw
MTc2NjQwIDUxMi1ieXRlIGxvZ2ljYWwgYmxvY2tzOiAoNjEuNSBHQi81Ny4zIEdpQikKU2VwIDE0
IDExOjEwOjQxIGRlYmlhbiBrZXJuZWw6IFsgMjQzNC40Nzc3NDldIHNkIDE6MDowOjA6IFtzZGFd
IFdyaXRlIFByb3RlY3QgaXMgb2ZmClNlcCAxNCAxMToxMDo0MSBkZWJpYW4ga2VybmVsOiBbIDI0
MzQuNDc3NzUzXSBzZCAxOjA6MDowOiBbc2RhXSBNb2RlIFNlbnNlOiA0MyAwMCAwMCAwMApTZXAg
MTQgMTE6MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjQ3ODg5OV0gc2QgMTowOjA6MDogW3Nk
YV0gV3JpdGUgY2FjaGU6IGRpc2FibGVkLCByZWFkIGNhY2hlOiBlbmFibGVkLCBkb2Vzbid0IHN1
cHBvcnQgRFBPIG9yIEZVQQpTZXAgMTQgMTE6MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjUz
MDEyNl0gIHNkYTogc2RhMQpTZXAgMTQgMTE6MTA6NDEgZGViaWFuIGtlcm5lbDogWyAyNDM0LjUz
NDM2OV0gc2QgMTowOjA6MDogW3NkYV0gQXR0YWNoZWQgU0NTSSByZW1vdmFibGUgZGlzawpTZXAg
MTQgMTE6MTI6MjMgZGViaWFuIGtlcm5lbDogWyAyNTM2LjQ4Mzg0Ml0gc2RhOiBkZXRlY3RlZCBj
YXBhY2l0eSBjaGFuZ2UgZnJvbSA2MTUzMDQzOTY4MCB0byAwClNlcCAxNCAxMToxMjoyMyBkZWJp
YW4ga2VybmVsOiBbIDI1MzcuNDA1NzkyXSB1c2IgMi0zLjE6IFVTQiBkaXNjb25uZWN0LCBkZXZp
Y2UgbnVtYmVyIDMKU2VwIDE0IDExOjEzOjEwIGRlYmlhbiBrZXJuZWw6IFsgMjU4My41NDgzNjld
IHVzYiAyLTMuMTogbmV3IFN1cGVyU3BlZWQgR2VuIDEgVVNCIGRldmljZSBudW1iZXIgNCB1c2lu
ZyB4aGNpX2hjZApTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtlcm5lbDogWyAyNTgzLjU3NjU4M10g
dXNiIDItMy4xOiBOZXcgVVNCIGRldmljZSBmb3VuZCwgaWRWZW5kb3I9MDc4MSwgaWRQcm9kdWN0
PTU1ODEsIGJjZERldmljZT0gMS4wMApTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtlcm5lbDogWyAy
NTgzLjU3NjU4N10gdXNiIDItMy4xOiBOZXcgVVNCIGRldmljZSBzdHJpbmdzOiBNZnI9MSwgUHJv
ZHVjdD0yLCBTZXJpYWxOdW1iZXI9MwpTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtlcm5lbDogWyAy
NTgzLjU3NjU4OV0gdXNiIDItMy4xOiBQcm9kdWN0OiAgU2FuRGlzayAzLjJHZW4xClNlcCAxNCAx
MToxMzoxMCBkZWJpYW4ga2VybmVsOiBbIDI1ODMuNTc2NTkxXSB1c2IgMi0zLjE6IE1hbnVmYWN0
dXJlcjogIFVTQgpTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtlcm5lbDogWyAyNTgzLjU3NjU5M10g
dXNiIDItMy4xOiBTZXJpYWxOdW1iZXI6IDA0MDFlYWFlNGY2ZWM5ZmE4OTlkOTBlYjZhZTgzOTE3
MjNmNzMyM2M1OWNkYjdiYmQ1M2EwZDYzYWI1YmNlNmRmMWZkMDAwMDAwMDAwMDAwMDAwMDAwMDA2
MDRlMWY1YjAwOGU2NzE4ODE1NTgxMDcxY2FiNDVlOQpTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtl
cm5lbDogWyAyNTgzLjU4MTg1N10gdXNiLXN0b3JhZ2UgMi0zLjE6MS4wOiBVU0IgTWFzcyBTdG9y
YWdlIGRldmljZSBkZXRlY3RlZApTZXAgMTQgMTE6MTM6MTAgZGViaWFuIGtlcm5lbDogWyAyNTgz
LjU4MjA3Nl0gc2NzaSBob3N0MTogdXNiLXN0b3JhZ2UgMi0zLjE6MS4wClNlcCAxNCAxMToxMzox
MSBkZWJpYW4ga2VybmVsOiBbIDI1ODQuNjEzNTgzXSBzY3NpIDE6MDowOjA6IERpcmVjdC1BY2Nl
c3MgICAgICBVU0IgICAgICBTYW5EaXNrIDMuMkdlbjEgMS4wMCBQUTogMCBBTlNJOiA2ClNlcCAx
NCAxMToxMzoxMSBkZWJpYW4ga2VybmVsOiBbIDI1ODQuNjEzOTYxXSBzZCAxOjA6MDowOiBBdHRh
Y2hlZCBzY3NpIGdlbmVyaWMgc2cwIHR5cGUgMApTZXAgMTQgMTE6MTM6MTEgZGViaWFuIGtlcm5l
bDogWyAyNTg0LjYxNDE3OV0gc2QgMTowOjA6MDogW3NkYV0gMTIwMTc2NjQwIDUxMi1ieXRlIGxv
Z2ljYWwgYmxvY2tzOiAoNjEuNSBHQi81Ny4zIEdpQikKU2VwIDE0IDExOjEzOjExIGRlYmlhbiBr
ZXJuZWw6IFsgMjU4NC42MTYwMThdIHNkIDE6MDowOjA6IFtzZGFdIFdyaXRlIFByb3RlY3QgaXMg
b2ZmClNlcCAxNCAxMToxMzoxMSBkZWJpYW4ga2VybmVsOiBbIDI1ODQuNjE2MDIxXSBzZCAxOjA6
MDowOiBbc2RhXSBNb2RlIFNlbnNlOiA0MyAwMCAwMCAwMApTZXAgMTQgMTE6MTM6MTEgZGViaWFu
IGtlcm5lbDogWyAyNTg0LjYxNzE3Ml0gc2QgMTowOjA6MDogW3NkYV0gV3JpdGUgY2FjaGU6IGRp
c2FibGVkLCByZWFkIGNhY2hlOiBlbmFibGVkLCBkb2Vzbid0IHN1cHBvcnQgRFBPIG9yIEZVQQpT
ZXAgMTQgMTE6MTM6MTEgZGViaWFuIGtlcm5lbDogWyAyNTg0LjY0ODg0M10gIHNkYTogc2RhMQpT
ZXAgMTQgMTE6MTM6MTEgZGViaWFuIGtlcm5lbDogWyAyNTg0LjY1MzA3NF0gc2QgMTowOjA6MDog
W3NkYV0gQXR0YWNoZWQgU0NTSSByZW1vdmFibGUgZGlzawpTZXAgMTQgMTE6MTg6MTYgZGViaWFu
IGtlcm5lbDogWyAyODkwLjM2MjY1OV0gc2RhOiBkZXRlY3RlZCBjYXBhY2l0eSBjaGFuZ2UgZnJv
bSA2MTUzMDQzOTY4MCB0byAwClNlcCAxNCAxMToxODoxOSBkZWJpYW4ga2VybmVsOiBbIDI4OTIu
NzYwMTEwXSB1c2IgMi0zLjE6IFVTQiBkaXNjb25uZWN0LCBkZXZpY2UgbnVtYmVyIDQK
--000000000000f7287605cbee41cf--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 06:55:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 06:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186158.334876 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2LZ-0001Ro-To; Tue, 14 Sep 2021 06:55:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186158.334876; Tue, 14 Sep 2021 06:55:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2LZ-0001Rh-Qs; Tue, 14 Sep 2021 06:55:13 +0000
Received: by outflank-mailman (input) for mailman id 186158;
 Tue, 14 Sep 2021 06:55:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ2LZ-0001Rb-1S
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 06:55:13 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b428bd44-1528-11ec-b3ff-12813bfff9fa;
 Tue, 14 Sep 2021 06:55:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b428bd44-1528-11ec-b3ff-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631602511;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=P2t+bIunCdVFJ1eBQHXRxajT+xdHm1HFCzoc/FgzRPs=;
  b=ZlYkjaTfuBR+cogQeGjjRjeYv91YmBHtO2xWVwJMOnYQSKpI4Ciqf4uR
   MJnC/DMmEcTYACG0dOG72i/8X91Z7KaG0KvOq2iFM0PbQs9QJnuFL1MmD
   E+blRKa+hE3rFKiXqef54xFoDQkk+uaGipQkY8eLqSaG2DrOX0Fet85jA
   4=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: k45HzrNVK9XlVF27Oj0HLc6o+xaJ1sBlYNbkp2++kraoqpoUCGLzINDYRBXTlNQ0ZHug6hEs/3
 9H53tJnr/m+m+fgBD2lTNNqmh6pGXF0lxTvf6LEdB8JVrRXxTxOzOwav42IsTbfWdF4piZ2cO1
 Uug8JjB8VYF5BYexjncD8CCNfKFKJwKZ5qcgLU0pM3S6H4H34JZP+H43lEzHbAYi7rIJsKQr9S
 VIF2KfwD10gw9OHv77mmxubJK1xOlViPhJYGAgP3oSGZxjH0cMtvu3jPsSwQ32KD2eUgOJAGsh
 CyayJQGHkfZSQkPKJNbbPmkT
X-SBRS: 5.1
X-MesageID: 53056487
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:DQylUKDUSoV9zBVW/zfkw5YqxClBgxIJ4kV8jS/XYbTApDgn32AHx
 jceWWCAaauONmr8Kdp1Odzi909X7JDdmoJjQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0E47wobVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/qByihfdd9
 4h0sJWIWAkqN47jneckTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHv6SuIEEjGhYasZmIs7af
 tcJa2VWUjfQai1wIwksIoMGtbL97pX4W2IB8w/EzUYt2EDMyCRh3b6rN8DaEvSaSMMQkkuGq
 2bu+2XiHgpcJNGZ0SCC8H+nmqnIhyyTcIAYGaC89/VqqEaO3WFVAxoTPWZXutHg1BT4AYgGb
 RVJpGx+9sDe6XBHUPHedjaih1K74ycZUuJBHe4zyy6IyZPLtlPx6nc/ctJRVDA3nJZoHmVwj
 QLYw4yB6S9H6+LOGCnEnluAhXbrY3FEczVaDcMRZVZdu7HeTJcPYgUjpzqJOJW8iMH8URr0y
 iqDxMTVr+RO1ZNXv0lXEFauvt5NmnQrZlVujuk0djj8hu+cWGJCT9b0gWU3Fd4acO6koqCp5
 RDoYfRyCdwz4WylznTREI3h441FF97aaWaB0DaD7rEK9ii3+m7LQL28FApWfR8zWu5dIGeBS
 BaK5Wt5uc8CVFP3PPQfS9/gVKwXIV3ISI2Nugb8NYEVPPCctWavoUlTWKJn9zuxyRN9zf5mY
 s7znATFJS9yNJmLBQGeHo81+bQq2jo/1SXUQ5X6xA6gyr2QeDieTrJtDbdERrlRAHqsrFqH/
 tBBGdGNzhkDAuTybjOOqdwYLEwQLGh9Dpfz8pQFeumGKwtgOWcgF/6Onu9xJ907x/xYxrXS4
 3Wwek5E016j13fJHhqHNyJ4Y7T1UJch8X9iZX4wPUyl0mQIaJq06PtNbIM+eLQqrbQxzfN9Q
 /QfVd+HB/BDFmbO9zgHNMGvp41+bhW7wwmJOnP9MjQ4epdhQS3P+8PlIVSzpHVfUHLvuJJn8
 bO61w7dTZ4SfChYDZ7bOKC10le8nXkBg+YuDUHGFcZeJRf3+49wJi2v0vJue5MQKQ/Ozyex3
 hqNBUtKvvHEpoI4/YWbha2AqIv1QeJyElADQjve5LeycyLb4nCi0clLV+PRJWLRU2b9+aODY
 +RJzq6jbK1bzQgS64csQax2ya8e5sf0o+4IxwtpK3zHclC3B+4yOXKBx8RO6vVAy7IxVdFag
 a5TFg224Ym0Bf4=
IronPort-HdrOrdr: A9a23:QPM/3qNhWTTsEcBcTq+jsMiBIKoaSvp037BN7SFMoH1uHPBw+P
 rDoB1273XJYVUqM03I8OroUJVoKkmyyXca2+QsAYs=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="53056487"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VzBRiQl1nVyXINJyAtRFLvL8z4Q5q/sj1Tid39KJ7jmJl++mu3cupVhtDvNIIBega+QnJbOpy33FR67FR5wFbChLcJ52r/z2NU/K6DjF4Z2uTr+RGwBiESIolIofP9DWfaIYZ9KO0I4I+BEhuwSd1Q2UieujepJ7s735+p+iP68lEctpDjFJL0+9F5JzCvL9HDgc2KQXyijxt19DtPD7FS8kiCrguUF1ggfKl+9cvmg9xX+atSN5lydxgcPkBSWmufndR6TJQKhr2/UgR+QKaX6gJjXWo4yR8pULljVgaIWaVUTAUN3zD0MO0s2lOLZzpoGWMXwAVVbxJCFSWe9TVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cMmlJddhrJKH0sobqoQ4EX7wVdGnGn9RVUOEOYK1QW4=;
 b=S/3RPKZOyEjeWowXTD+O3u78Oqh7ESzucZFDwzYeW0fif2++xWuwwqKxkZE5PgmHTCGaioflsqzv32Y3rOBI489hEUiHXzLOyFCFwTzwtjGEY58/Nzz0Yl5z7ZTnCmEoB16yrVOBnLdBaaJR9aU8Dnj1/MbNE1Ogg88/qIC+d054hoiuNzbFFAJS29I8gZDHruAhBlWDjjqSZ9FdCHjguvubim//Yi3lNFKUvZjjh6kEY6bd+ivqvew1YA0sdSx5jenbQCCGLhZ/VwlJoMbAid27aD3sEu2omyxjpai7AMm13bq3/U4PaHZQhBiEFnGbg7Ui4akQcUZfVrx5G4LkdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cMmlJddhrJKH0sobqoQ4EX7wVdGnGn9RVUOEOYK1QW4=;
 b=RIL+tvEcU4iBrLItRAaXPkPq47NbDWVo7DHiPK88B3mg8rG0mWMLHyFi3eQrsf6R9Z192zZaVEpB8W5+Pj7L1YoFtbfYSfa62I127JhHNfeoGg/Dz40XC4BVpHJGX0gVL7Pu70jBHaXP0PukjQINZYicyktJ4F26d1wQRuyZ7IM=
Date: Tue, 14 Sep 2021 08:55:02 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
Message-ID: <YUBHRhfLDoEHH7k9@MacBook-Air-de-Roger.local>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
 <7c97de87-9687-a088-406b-33faf037244a@suse.com>
 <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
 <2959d91e-1048-a4dd-7f79-ec1d14b81d02@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2959d91e-1048-a4dd-7f79-ec1d14b81d02@suse.com>
X-ClientProxiedBy: LO4P123CA0502.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1ab::21) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 88cd08b4-3718-45ce-8c6d-08d9774c9681
X-MS-TrafficTypeDiagnostic: DM5PR03MB3372:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB33721358C7A4C5035C3B9FD78FDA9@DM5PR03MB3372.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 3gQnfJaDu240E1UhhHwof9oq1Rf3ZYQZ0hRf+EJVJaKPiOUnrq8xjlsMQvoc+LjMaGSaZhUOa6Q6VxydD6mH4gXeX+MDvNpjFn7S6x5Jm7KXrfpFKzOqtJiErmpmVXJahgFnaiOFHtKNd70oZBcH5eFphgzE6HPylK0Qq/MLH0jWVbkCrfgV4OAzwFczgsV80DG4l5hzRnd6nUqCdblyLJ0O7bT+wvxr20srIv5JXph7IDTO9bWdfGkfdEnQhbyrtEkJZi03W6iSpdD6CNmKr0PCXgYbW0HzK+oGb3/+MF26bMpG+7LmU0YwEy/sGC+eH7/H5uGylyhO7Y6SWh6PFd0cDTNxJCKifUDg00bwBmFGL5cvHZUSzXtemtpHFhl1OW8neiUqqDetOqozIS1DhZ2dmiiLBXPABUByqEhqRhi+kBgq/FSlnvvI3YMp2+rS7jCN6tUYQG2vGJXC2XiAOgDV7LhSHBbphrkM7SUq+0GnOiux/gpOPqjzCRxy41hzkMb4GWS5ktUPymA4Zp/wC4Micc4KLPxh0GYMjE4sO2nTcc3uZ4/cUoth0g8DC+dGH9C8Ic2ox0xyWSRsKBUPt4ZPNl5Z89TABc9+rmKs/J4WgA+H+t2vy7JrYemjBVkVxWvExNeQTlCw1dME0YaxnQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(366004)(346002)(136003)(6496006)(316002)(54906003)(8936002)(478600001)(85182001)(956004)(83380400001)(5660300002)(4326008)(26005)(38100700002)(86362001)(2906002)(53546011)(9686003)(8676002)(66946007)(66476007)(66556008)(6486002)(6916009)(6666004)(186003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?c1UzMmV3akxvRFZ0MXNnTnhtSVp4aVFkR1Q4cFpHZmNISUc0SGxQY1hFQTlI?=
 =?utf-8?B?dUJWNVJBZC9ha2NUdlBjWEphcGlGTnhyQkFvdXVEcXlQS3hYR29lUmNVV1d1?=
 =?utf-8?B?WlUvMWRYM3lVZXpNNFUvK0FLVndMNHVPK1hQZDVPT3FsaXF3a0lvaG5HWWc3?=
 =?utf-8?B?VUZhbUFwTy95UVpFZFJNaWt3RDBNTXpKUksxWEZPSmNUZUczR2VDOVY4RHRI?=
 =?utf-8?B?d2ZKaUd4QnIvVitDcmt6UVZjc2NtR0RsaUd6emhzNldkQURlMkRFeXVPMDJt?=
 =?utf-8?B?SzYzeFVGTWFXeXkrVUl3bzRqdVBXQ3ljUEM2bkNiTG93VGZpcXRnamRGS0dq?=
 =?utf-8?B?Y1Z2aU8yaWRaNityT1ZPdUp0WCtRVGpvQ2hpMzBUN0Z3cFFQdk4vZ3dnbjQ0?=
 =?utf-8?B?T29QK2FtRExJQkxsTUZxbzloeFBoR2hTR1lFd0VIN0VEeXFaOFhIOFlXa0c4?=
 =?utf-8?B?OE9rdnU1NE5rekNRVWYwY09ZcjhqR0YzbXhTNVlNa3Y1TzYrSVIramZFcE1n?=
 =?utf-8?B?ZW9PY1NRUnlWdEdna2NBMUlHUU1LQ1pzMlZlL3l0VGpJYmdIeEJmRExNUXJI?=
 =?utf-8?B?T0VqbXdqZHdrdHpPNndlQU9rQk1zOFFoejcyRk1tZGZUU0lBWkt2VzRGTnZI?=
 =?utf-8?B?ay9rNVQ3TGloWXM3c0pVVlpKTnk0OXVod21TU3B1UnNuUDNJc05jczJLQTBl?=
 =?utf-8?B?RmJibXJWZVFUcEpsVkNTS3ljdlQ1V0JxQjN6QkxMdnpQS1NmMS8rYVdnL0N6?=
 =?utf-8?B?Y1FNVnJQdWs1RTlmVERBLzFCTUhTVllCZFhSejh0ZzYxV1grOG54cVltTzF5?=
 =?utf-8?B?ZGFNd2xDTVkwZXg1OHJyUjVuSldFYUN1NmppOWIvUmFPYk52OVBkMis0UEhs?=
 =?utf-8?B?MDV0ZG9WcnJTdEJ3REh5T1pJdkJaUk11WUU0WlhxNXo2dDM0emFxUGNNNjk0?=
 =?utf-8?B?eitLMGJBMk41RWFGUlQ0eXoxa0FMTzkvUUlaSTE3ek5Kcmh2c1pZYVFDdERn?=
 =?utf-8?B?TXlhY1FiTzg0VldYMEU2ZXRwZW13UW5BcDE0WFVWTkZVbnBuL0hvMEhnSnVw?=
 =?utf-8?B?ZWlEVmRRaXJJWW5Jc0RtOFhvaU9qTU9kdGlLVUp0bkVaRHdKMncwRUNORENG?=
 =?utf-8?B?RnNvbzZsSGlnSGFYTjhEYitHMytxdDh4SFN3cVZGbS90dDNoOUVZSGJ4TElR?=
 =?utf-8?B?YUpxSkNWakdjQmpPUElNNkZRVG1BbEZwYTEzTVpXM3YvY1ZPT1RNR2wxT2FN?=
 =?utf-8?B?VXlJU2VMVWNmdWtnVlQ5ZTJLU3h2RmdOQXE2SU9EWGt6azNPZjhzZjhxNVFp?=
 =?utf-8?B?cjliMnNORE9Ud0pzdmh2Q1JqWkEvL1ZiMC8rcFB1N3pjc3NTa3Z5REVlMCtp?=
 =?utf-8?B?MGxvZG1ZQ3duZmg4Q3Nia3Q5VUhydFd6Y1pSVEhzT2dncWczOXBRNzdqU3JW?=
 =?utf-8?B?WnhKWlFYMW55OFh0VjNFU2ZSK3Jtc2RrakVURW53TjBldURrRjBnTnJCb1Q0?=
 =?utf-8?B?R2xlbThnVUxYMHJuL3l3MzcvYmMxNFpPUlNaYkFzditmODRXMExGblNqSUpx?=
 =?utf-8?B?TzFWU20wbTB4ak81M21mRUZDUWRPYjU2bXMzMU40UTQrUzN0dURNZ2FvN1FR?=
 =?utf-8?B?UnB4SGgwUFU0YnRuMW02S0Jza1FaTHhWeks0U1d0WStLSVRmbGJ6UGpkSEJI?=
 =?utf-8?B?VlpSTEhGdWJJOHJSOEZYdVhTaWRKdmdmajBpaVpFQUdJVHYvUWk5ejFSVG1j?=
 =?utf-8?Q?isJdSang88exzPjRvxkWxGXDP7ye8uivQPbaSFZ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 88cd08b4-3718-45ce-8c6d-08d9774c9681
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 06:55:07.8103
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IvBjAwuJGUc3XIg5tZROIsgdB0OSXlAiE4xvfF+8+JITOOzY4Pv/XhXDPlTo2ADUdakxTaJUBEno+SmHSYXKIA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3372
X-OriginatorOrg: citrix.com

On Mon, Sep 13, 2021 at 05:10:54PM +0200, Jan Beulich wrote:
> On 13.09.2021 16:33, Roger Pau Monné wrote:
> > On Mon, Sep 13, 2021 at 04:05:15PM +0200, Jan Beulich wrote:
> >> On 13.09.2021 15:37, Roger Pau Monné wrote:
> >>>> --- a/xen/arch/x86/Makefile
> >>>> +++ b/xen/arch/x86/Makefile
> >>>> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
> >>>>  
> >>>>  ifneq ($(build_id_linker),)
> >>>>  notes_phdrs = --notes
> >>>> +# Determine whether to engage a workaround for GNU ld 2.37.
> >>>> +build-id-ld-good = $(shell echo 'void test(void) {}' \
> >>>> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - 2>.check.err \
> >>>> +                           && $(LD) -T check.lds -o .check.elf .check.o 2>>.check.err \
> >>>> +                           && echo y)
> >>>
> >>> Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
> >>> then use is here?
> >>>
> >>> We already have compiler and assembler checks in x86/Kconfig, so it
> >>> would seem more natural to place it there.
> >>
> >> The question of whether to record tool chain capabilities in .config
> >> is still pending. I'm not convinced this is a good idea, Andrew keeps
> >> shouting me out for that, and an actual discussion doesn't really
> >> happen. Yet unlike back at the time when I first raised my concern,
> >> Anthony meanwhile supports me in at least the question (to Andrew) of
> >> when such a discussion would have happened: Neither of us is aware,
> >> yet Andrew claims it did happen, but so far didn't point out where
> >> one could read about what was discussed and decided there.
> >>
> >> For the few uses we've accumulated I gave (if at all) an ack for
> >> things happening under some sort of pressure, with the request that
> >> aformentioned discussion would happen subsequently (and, depending on
> >> outcome, these would be converted to another approach if need be). I
> >> have meanwhile realized that it was a mistake to allow such things in
> >> on this basis - the more of them we gain, the more I'm hearing "we've
> >> already got some".
> > 
> > I see, that's not an ideal situation from a review PoV, as we don't
> > have a clear placement for those and that will just cause confusion
> > (and more work since there are potentially two places to check).
> > 
> > What's the benefit of placing the checks in Kconfig instead of the
> > Makefiles?
> > 
> > As I understand Kconfig is run only once, while the Makefile could run
> > the check multiple times.
> > 
> > The downfall of having them in .config is that .config could suddenly
> > change as tools are updated or as it's moved around different systems
> > (yet .config already contains specific compiler version information).
> 
> I should have added in the earlier reply: Besides the pros and cons
> there is, to me at least, also the abstract question of whether such
> information belongs in .config in the first place. To me this file
> has always been a record of build-meister decisions only. Quite
> different from the output of userland ./configure, where checking
> various aspects of the environment is the primary goal.

Well, userland ./configure is also used for setting build time options
for the tools (we have a bunch of --{disable/enable}-foo), so I think
the line is quite fuzzy there also.

I think it would be better if we could split Kconfig settings into
.config and .tools for example, to clearly differentiate between
user-selected options vs environment information, and then it would
also be fairly easy to regenerate .tools on every build. Sadly I don't
think Kconfig is capable of this in it's current state.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 07:14:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 07:14:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186169.334888 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2eR-000404-LR; Tue, 14 Sep 2021 07:14:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186169.334888; Tue, 14 Sep 2021 07:14:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2eR-0003zx-GT; Tue, 14 Sep 2021 07:14:43 +0000
Received: by outflank-mailman (input) for mailman id 186169;
 Tue, 14 Sep 2021 07:14:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ2eP-0003zr-UL
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 07:14:42 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 19734cd8-8010-43e7-882a-049f542c931f;
 Tue, 14 Sep 2021 07:14:40 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 81228200CD;
 Tue, 14 Sep 2021 07:14:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4514413E4A;
 Tue, 14 Sep 2021 07:14:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id FaZqD99LQGHjeQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 07:14:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19734cd8-8010-43e7-882a-049f542c931f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631603679; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=DJa19hTde/qOBR3+nSTkyFQnZuNkLZ/gxm3JnyWLTd4=;
	b=gtvxpid3ZJ95Elfw4sDXBjJL9nuVLbcX2SEv2PyQlsp/qCTiUQQK70VBUsCilwMcYLol8o
	hWOTxyXRXH9bmA/3oNXZ81Mq7xzTFbYukzC5RpnJF25eqtLq8hdPZVCMI6qalYGbZAP3Jp
	Ffph88dzBGjGmkz/ba4E4mhTk3fYHjA=
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, Mike Rapoport <rppt@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
 Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
 linux-kernel@vger.kernel.org, xen-devel <xen-devel@lists.xenproject.org>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
From: Juergen Gross <jgross@suse.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
Message-ID: <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
Date: Tue, 14 Sep 2021 09:14:38 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YT9I+Xs9wOPVCIVd@mail-itl>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="l4jfD4hpX9iAQ8lDxD2znK3npbtEVhEGs"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--l4jfD4hpX9iAQ8lDxD2znK3npbtEVhEGs
Content-Type: multipart/mixed; boundary="JTBVfVXW6uAbagWOQ54K6n3VW5XYFcVGN";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, Mike Rapoport <rppt@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
 Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
 linux-kernel@vger.kernel.org, xen-devel <xen-devel@lists.xenproject.org>
Message-ID: <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
References: <YT9I+Xs9wOPVCIVd@mail-itl>
In-Reply-To: <YT9I+Xs9wOPVCIVd@mail-itl>

--JTBVfVXW6uAbagWOQ54K6n3VW5XYFcVGN
Content-Type: multipart/mixed;
 boundary="------------8E27746F541E8C0209A0A9BF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8E27746F541E8C0209A0A9BF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 13.09.21 14:50, Marek Marczykowski-G=C3=B3recki wrote:
> Hi,
>=20
> Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the=

> kernel version.
> Test environment:
>   - Xen 4.14.2
>   - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
>   - Linux 5.13.13, confirmed also on 5.14
>=20
> The crash happens only if the initramfs has earlycpio with microcode.
> I don't have a serial console, but I've got a photo with crash message
> (from Xen, Linux doesn't managed to print anything):
> https://user-images.githubusercontent.com/726704/133084966-5038f37e-001=
b-4688-9f90-83d09be3dc2d.jpg
>=20
> Transcription of some of it:
>=20
>      mapping kernel into physical memory
>      about to get started
>      (XEN) Pagetable walk from ffffffff82810888:
>      (XEN)  L4[0x1ff] =3D 0000000332815067 0000000000002815
>      (XEN)  L3[0x1fe] =3D 0000000332816067 0000000000002816
>      (XEN)  L2[0x014] =3D 0000000334018067 0000000000004018
>      (XEN)  L1[0x010] =3D 0000000332810067 0000000000002810
>      (XEN) domain_crash_sync called from entry.S: fault at ffff82d04033=
e790 x86_64/entry.S#domain_crash_page_fault
>      (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
>      (XEN) ----[ Xen-4.14.2  x86_64  debug=3Dn  Not tainted ]----
>      (XEN) CPU:    0
>      (XEN) RIP:    e033:[<0000000000000000>]

The domain's run state seems to be completely clobbered.

Did you try to boot the kernel with "earlyprintk=3Dxen" to get some idea
how far it progressed?

I could imagine that doing the early reservations after the call of
e820__memory_setup() is problematic, as Xen PV guests have a hook in
this function performing some rather extended actions.

I'm not sure the call of early_reserve_memory() can be moved just before
the e820__memory_setup() call. If this is possibel it should be done
IMO, if not then the reservations which have been at the start of
setup_arch() might need to go there again.


Juergen

>=20
> I've bisected it down to the commit a799c2bd29d19c565f37fa038b31a0a1d44=
d0e4d
>=20
>      x86/setup: Consolidate early memory reservations
>=20
>      The early reservations of memory areas used by the firmware, bootl=
oader,
>      kernel text and data are spread over setup_arch(). Moreover, some =
of them
>      happen *after* memblock allocations, e.g trim_platform_memory_rang=
es() and
>      trim_low_memory_range() are called after reserve_real_mode() that =
allocates
>      memory.
>=20
>      There was no corruption of these memory regions because memblock a=
lways
>      allocates memory either from the end of memory (in top-down mode) =
or above
>      the kernel image (in bottom-up mode). However, the bottom up mode =
is going
>      to be updated to span the entire memory [1] to avoid limitations c=
aused by
>      KASLR.
>=20
>      Consolidate early memory reservations in a dedicated function to i=
mprove
>      robustness against future changes. Having the early reservations i=
n one
>      place also makes it clearer what memory must be reserved before me=
mblock
>      allocations are allowed.
>=20
>      Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
>      Signed-off-by: Borislav Petkov <bp@suse.de>
>      Reviewed-by: Baoquan He <bhe@redhat.com>
>      Acked-by: Borislav Petkov <bp@suse.de>
>      Acked-by: David Hildenbrand <david@redhat.com>
>      Link: [1] https://lore.kernel.org/lkml/20201217201214.3414100-2-gu=
ro@fb.com
>      Link: https://lkml.kernel.org/r/20210302100406.22059-2-rppt@kernel=
=2Eorg
>=20
> Since this seems to affect Xen boot only, I'm copying xen-devel too.
>=20
> Any ideas?
>=20


--------------8E27746F541E8C0209A0A9BF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8E27746F541E8C0209A0A9BF--

--JTBVfVXW6uAbagWOQ54K6n3VW5XYFcVGN--

--l4jfD4hpX9iAQ8lDxD2znK3npbtEVhEGs
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFAS94FAwAAAAAACgkQsN6d1ii/Ey+T
Owf+MNxstSzsfIUq7cY1rZlbOnJ4MVHw8NBm78S+Yk8qV3nHUSQanDnRK++r7clui0jkFr498TyU
8hX5c1BU1161R4z5blwxRelx8Wc5QnYd8682n57UA+zgZhYq3lGxo5jYF19Vf9clmkHwP2ppab51
R41dZpf/xVXhNunPGMXWgviaWUeFRTwSKWm09OJcoL76zdTn64dO12qW9ppnqyBHAQPNGss5HSLP
93t+rQ4RxFkg7Zo6I1HZUUyEC+sr2RCZyfUue4W0Ea8MavhugdqVjZ1rKpbMFJj8eyMXR996Yxrg
OM/PqGn/Q1JqJWLPcsvCUn30eItOck7z0I5YfIM9AQ==
=UJPY
-----END PGP SIGNATURE-----

--l4jfD4hpX9iAQ8lDxD2znK3npbtEVhEGs--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 07:29:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 07:29:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186179.334899 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2t3-0005j2-0a; Tue, 14 Sep 2021 07:29:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186179.334899; Tue, 14 Sep 2021 07:29:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ2t2-0005iv-TO; Tue, 14 Sep 2021 07:29:48 +0000
Received: by outflank-mailman (input) for mailman id 186179;
 Tue, 14 Sep 2021 07:29:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ2t1-0005ip-30
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 07:29:47 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 83102463-03ba-4f55-a9b9-950082d133ce;
 Tue, 14 Sep 2021 07:29:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83102463-03ba-4f55-a9b9-950082d133ce
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631604584;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=6Ms2wkGWXtw/1UqTUOgMRL2u4IZIASPlYnlYFIaSUHE=;
  b=TotGWtPjlR+SIlQ3t81e8Fakcyg91xPY1KF5HTSL9QxL4UtplflAxY1V
   uQydR0ZMPDxvK+8Z1UD9Z7Hg6YKHwX2hSmj5XieQCfty2DziqTeaHAlvG
   e6WB6ZGNqECQDWfatTVxdockDH63jU/UmTpt0ZqPXMNOq8DAGtE6OXJFL
   A=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: KlRtiMKY5cso7l0MHzs8szn2l2JVCYyOrXieB59tck8ytrHSlbWQswemxurKBxglDUo6Jv2ZRI
 sZqYA5l5pPvCNsMUwm9FQRKk2mJt20OmWOLJmcxmCi7mjdHrbSqPjhTnaOizjCvZ8co8zTftlK
 cUaFDB0KD1yyt0hg8yxL+pjguWKciD5U75bTNPubNw+d+Vyr176nDlJ9uVKYO7A3U5dpNPtNEJ
 hFZpMVax3OxMlirvx0KV7yGUTaO3CCg5XORvieHaZ0kZmR/+MilfbtTI8X8YMkDVjDEgYuvats
 J6+3fW1IKSGmK5p+gw39CCko
X-SBRS: 5.1
X-MesageID: 52244988
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:3w9W0q8ooOcxYHUFmsi9DrUDnnmTJUtcMsCJ2f8bNWPcYEJGY0x3z
 TEcCmjXPa2MZ2KgLogkaoqx9hxQvMXXytA1TlRtrHg8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79g3tYx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPgs7
 ex35LCtRT4KP6rKsflFUSAbHHphaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFhm1s2pEeTZ4yY
 eIfewtjUQ2QZydjZA8aIcIAmdiWlnbgJmgwRFW9+vNsvjm7IBZK+KfpGMrYfJqNX8o9tlaVo
 CfK8nr0BjkeNceD0nyV/3S0nOjNkCjnHoUIG9WQ9PRnnVmSzWw7EwANWB2wpvzRt6Klc4sBc
 QpOoHNo9PVsshzwJjXgY/GmiCOhhyRNd8dcKs477wek1/LW2kWWXXdRG1atd+caWN8KqS0Cj
 wHSxomyWG0z79V5WlrGqezF9mra1Tw9aDZYPH5aF1NtD8zL/dlr5i8jWOqPB0JcYjfdIjj23
 znChyw3nbx7YSUjhvjjoAyvb95BoPH0ou8JCuf/BTnNAuBRPtfNi2mUBb/zt6ooEWphZgPd1
 EXoYuDHhAz0MX1oqBFhvc1XRO35jxp6DNEsqQE2RMRwn9hc01WiYZpR8FlDGaudCe5dIWWBS
 BaK4Wt5vcYPVFP3Pf4fS9/gUKwCkPm/fekJo9iJN7KilLAqL1TZlMyvDGbNt13QfL8Eyv1mY
 sbHLp/3VB73y81PlVKLegvU6pdyrggWzmLPX5HriROh1LuVfnmOTrkZdlCJa4gEAGms+W05K
 v5TaJmHzQtxSur7bnWF+IIfNwlSf3M6GYr3u4pccevaelhqH2QoCvnwx7I9etM6w/QJx7mQp
 nztCFVFzFffhGHcLVnYYH5Ud76yD410qmg2PHJwMA/wiWQje4um8IwWa4AzIes87OVmwPMtF
 6sFdsyMD+5hUDPC/zhBP5DxoJY7LEaghB6UPjrjaz86JsYySwvM897iXw3u6ChRUXbn6Zpg+
 +Wtj1qJT4ACSgJuCNftRMiul17h72IAnO9SXlfTJoUBckvb74U3eTf6ieU6Ip9QJEyblCeaz
 QufHTwRufLJ/90u6NDMiK2J89WpHu95EhYIFmXX9+/rZyzT/27lyo5cSueYOzvaUTqsqqmlY
 OxUydD6MeEGwwkW49YtTe4zwPJs/cbrqp9b0h9gTSfCYFmcA799JmWLgJtUvapXy74F4Qa7V
 ypjIDWB1Wll7C89LGMsGQ==
IronPort-HdrOrdr: A9a23:7YCuA6+3GiFC2hMV9bxuk+FLdb1zdoMgy1knxilNoENuHPBwxv
 rAoB1E73PJYVYqOE3Jmbi7Sc+9qFfnhONICO4qTMuftWjdyRGVxeRZjLcKrAeQfhEWmtQtsZ
 uINpIOd+EYbmIK/foSgjPIa+rIqePvmMvD6Ja8vhUdPj2CKZsQlDuRYjzrY3GeLzM2fKbReq
 Dsgfau8FGbCAoqh4mAdzU4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kLEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 bxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72zeBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlJXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbZrmGuhHjXkV1RUsZiRtixZJGbAfqFCgL3V79FupgE686NCr/Zv2Evp9/oGOtF5Dq
 r/Q/1VfBwndL5gUUtHPpZ1fSKAMB2Fffv9ChPhHb3ZLtByB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52244988"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nten3braosCehQW+2h1iyRct6b6CASKHrcCeH2xLjtauiK9AWxMBpSyifa+sQKTh9re/AdwMfJHbDqsavNQtP8aEuPzXF4OYfdHZbI7Ccypa/tWSjgDYB1d0jmeiMi7QM/u4/F4Ink7D5BALWghd5JLDoMJBNqFIyjku+tyCcEB2svkEhj/HHeUyTqDT70C/VJWPW5hj6XNTbsBECjyPHOYA+IY/yxPl4oc8bb/h3uQABLY7qh8xX0fF6ZocyVqZKJDTy6+SF9NEL6aE5I3LVoBJH0s7C6cr2B4HcqbwhcOGMuCzhkmq90SIsP28R9oNnuzwV9nHBmEKNwRceTNeDA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dAY+uZGV42lpJtZXtU0lvoXV0hBjXdyR9G7jZEG1zJw=;
 b=DLaIVI9FIFDI/x4AeSp8/9z+iO5AGreVsVZqdDRZo5E7/j0fy3DSeH0NHRPM1lNZCm1zDlfJAsDDRGlN1yOSQc8VFUSQE/rSuWg28Xv4lVOKw32SNVFSP+sdaPysbYkb37YbjU9EkGKqDfWNXChk7jfaZzMnVIKd9f17REJc1K5hCCW1gd4a/HKtahAum9A8Gh58Sd76vjMi9Ef5+455TgEKwOJCSjLh1mxBAZnX8y5ZlSJ0ndSmAcJt/Q5MdFK4R5DBYHuvCWf5HBNu1N1TE/DKA6VFviVoMx9sqnic/JD8SQYY0Q0Gn24KNFCSEMT8n1Th5Pc0lOT/pQAw3hT4Eg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dAY+uZGV42lpJtZXtU0lvoXV0hBjXdyR9G7jZEG1zJw=;
 b=eP9K0+iEWQtO4JZ+DWT8jfBfih1Ca407qhKYlLarJgZAAvul4GiMCka621iDwRpBe89iLY6NMWv312qhL69ZrBcTblenEV3IFt9fkNEu3my9PIxPr9Q/eRvieOTsPSnS8p7bcjteB0wKriHk3k5A3tYekSqxy8/fvW97VJ+wR7M=
Date: Tue, 14 Sep 2021 09:29:36 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: conditionalize workaround for build issue with GNU
 ld 2.37
Message-ID: <YUBPYBGsgK6nwgnf@MacBook-Air-de-Roger.local>
References: <4efe38f8-16a7-ad66-04dd-d55aadd57166@suse.com>
 <YT9UADtkz00JJkuB@MacBook-Air-de-Roger.local>
 <7c97de87-9687-a088-406b-33faf037244a@suse.com>
 <YT9hHPpt+SRU/Cv0@MacBook-Air-de-Roger.local>
 <4c8ddad3-e01f-06b0-1544-cffb477641b5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4c8ddad3-e01f-06b0-1544-cffb477641b5@suse.com>
X-ClientProxiedBy: LO4P123CA0099.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:191::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c2868e34-cf9e-4550-b2ce-08d977516a34
X-MS-TrafficTypeDiagnostic: DM6PR03MB4220:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4220432C45A19F2028071F5D8FDA9@DM6PR03MB4220.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dBBUY5GsIYAYW488tpdqqgoa/aRtZcpV0jvmEiVmbHwiGv25RBWEVEyWaoKj/MTrZOueik1PsQc47LySlL408HCACGbxF2znWk3etWZvMDulf4+NSlviejA64R0fpuDP+kdllz2eJc8QUYGQUnTpL77lGMmSoKqLirrpdPkFRi6E8xlBg/OztPuKr6ZkNzpORmEs3QDgtB7hy0a2oO1tTPJg4fUARuSkOfPsi7bmlD1SWqihL3L9DvzUJIXfhxd9wHYJT9TtPwohc9MyNDyYY7uQ+hYb01yB9t0GDM6KoEi+pynf2pQl5FSPw2sgJLH+G6JD2zmF9OK3QY5aDUCaxZEE/0cwJcDv7+7OyGiRARp+YILxvNMnya5UDacjhc8DN9sjSrC5HyzOt3IDFmgv6SXAwmDy9t2Tzia88jD2uq+LOKBCshDoxqRFIlA+XYIOCmMsD8nNv0GMU2uHBBVOBoGJ80Gih35w2bRYbnLP2HuEzT/sV+OFpHSauA9lI5Tdiw0czMSPGQbhPsSR8XEkVOZRC0iZLQAn7svwvBTyDikg04jlH9Gvcf4JMb4xp0UF3ROOXYJzl12SArNQ0nXwdrouQPQHbqrH80O6/8C94/brotFNd7Qvajfvlb7IqdWuOrYyFHFepyi+TXqYGzXV0FDhrDNbR/zUUW7Kmtt8gqXrh8wkg1jCFGJLC/pHX4OzUhiHmxA55cHMUmFBABfu0JBs2zGAErfEqjqSIAq3YN8=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(136003)(39860400002)(396003)(366004)(478600001)(8936002)(2906002)(66476007)(66946007)(54906003)(66556008)(4326008)(26005)(316002)(86362001)(6916009)(6666004)(186003)(5660300002)(83380400001)(6496006)(6486002)(9686003)(53546011)(38100700002)(956004)(966005)(8676002)(85182001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aE5oWW1DSXFTN1AwY3V0K1FqK2ZPUE14dzRIRGhIMDhjTEx2ZHU4R3doMi9L?=
 =?utf-8?B?UkNyYTIwclgzRlNKalUxQnB4cTgrUEN2TitXdkVoVUdNanZwcC8zd1UycEVC?=
 =?utf-8?B?ZzlCaGk1YTJMNGtDS1FYZXNCbURuNFlPQkcyWkZERDBaQ2dDMnRwVXNNNi9v?=
 =?utf-8?B?dGluQnF3WGpwSTVqNUpoNFB4a1BCNzI5Rm42MnpPbVJLdGYwQ3lmc3lvZXBl?=
 =?utf-8?B?TXB2SVNRSHVhYi9kOGNTRTRiWjFMWmw4K0ZkS3VUeC9jdWoyZko3ckVzejJ4?=
 =?utf-8?B?RnRCZm9RMmZheHhQVDkwREFoRlp0UW81MitVbGUyamFXeDI0TXlYSVlZam1o?=
 =?utf-8?B?RGM3YTF2WUNzSVhmRGdzV0pFL0k3OHlvUkZlUHhvcXpIUW9qTENWdHNjZlcx?=
 =?utf-8?B?MDByNEtGMEc5Z01yZzNyVWw4MXpCakVTOW9GVXM4ZkNRNE15V2dKZmR6Z3kv?=
 =?utf-8?B?WWdLcHR4aEJNVmIyMlVXb0pWYmFubjBXa1Y1Z2dJSXZLRmdLWkx1dHhraktR?=
 =?utf-8?B?eUdxNmN4aXdJVjJ3WTNCQTRTcGZiT0hYazBtUUF6OGtlb0lXZVlJTjZvUkZL?=
 =?utf-8?B?c2REc2tUKzBCOE9XSjJ1RWFPajhTQmh5aFdsU0VNZ2xOQnJaVGZDZkVhUHpt?=
 =?utf-8?B?RTZHVTROem9ORVM1Sm9lZ0pNZzAvdC9Dc0hKWk8xQWxQRTJGSmFyV0VYcjVx?=
 =?utf-8?B?NnR5NTJOOS9KM3FQM0NyaWF6N0k2OUR5MVN2cHhBSENwOUJuMTFUcGoyTm1E?=
 =?utf-8?B?UHRTZmlEYVJVaTl3MnpnQjNNS1BlQ3J3MzVyQXBiMkJSbDU4TVNPajlhbnl5?=
 =?utf-8?B?eDYvSjdvaDlYcHlSdGpQRi9ndWdtVEplQXhJVGovS2E1RnkyTU53eW82ME9H?=
 =?utf-8?B?R01HZ2NNZnBkZVRqOE4raENEKytpbGZ0SlRWcVYyOWdxcm5mbHd0dXkwMXpI?=
 =?utf-8?B?M2lCSDNiSGNMc24xOVZrUHFSVWJROTFuTHFaSW9mL3hEMkRxM05HZnVyb1N0?=
 =?utf-8?B?QjNObEU2cnBDSlppTVpFWWJ4L3JyYjlmMkdzUGtnNTM3a0tyZEQraEN1bEp1?=
 =?utf-8?B?SDB4QmJWNDN6Zm5wN09landhSElHRTlNakZEQ0RXNDFLWTM1Sk9TTXVadElK?=
 =?utf-8?B?WFNlcklDSmJyY0hnZE95YitDQWYvaTRPcWtZRXNJYmFuNGF2Ni83clFwWGhQ?=
 =?utf-8?B?LzJkb1lzUDlOU2VVN0p3ZzROMGhEcUNuZVZoR0VOWnNFbTc4cWQ4VWUzaXN1?=
 =?utf-8?B?eGlTU0NEYmhhL2hZd2RvWEFJRVZwTHJ5L2I1M054MXQyWnorNmFJRm9hS1Bv?=
 =?utf-8?B?SXNGSnE2QjhCQmxROCtMa3E0S2Zma2QzUktySTYrNmR6V2Qvd1VSdDZIQUVa?=
 =?utf-8?B?Q2VPYlZqbDFHQlZQRFJITDVGMXp0VGYwSWdPVWVuZG1Pa2d2YzB0cElZRmU4?=
 =?utf-8?B?aThxM3J3WFhTNHRyMjBBdklrYzY1dnFhaDB6aEI5RUQ0SUlWdWdRZXBWOUs0?=
 =?utf-8?B?eS9PRjJWbmdYcXNCelloNmtqTEVXbmRFTEo4L3FRL3NFNGFpUzM4VjR6OVcr?=
 =?utf-8?B?emhEaTUrM2VqS28vbzF0cThRVlpZaXhDZGZLL3BrUHY0N3dKY3ZkdjRvZW9G?=
 =?utf-8?B?UEtKc24zVFZIWWFtVWthcDdzV2ZWNXhyVW5WZmRRVjJPeVZYUktWbytLNGkz?=
 =?utf-8?B?NC9iOHMrbTVNeEFnVWhFaUExdHlTNGRWbmpKdWMwOUZHV25lOXBUaVpvYnQv?=
 =?utf-8?Q?WJxDfW3Q1I4mi3bsHlwkSSlAVeZd5kUTUl5iR+1?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c2868e34-cf9e-4550-b2ce-08d977516a34
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 07:29:40.8551
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6et7Ecfv3M12zRRiy2GQqSbPr1fVTLz2qtRM6FdrxPWJ0CNZqO4BMf+7IW6Opdzj+x6d6sf7FMNZiV/kOTC09g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4220
X-OriginatorOrg: citrix.com

On Mon, Sep 13, 2021 at 05:07:54PM +0200, Jan Beulich wrote:
> On 13.09.2021 16:33, Roger Pau Monné wrote:
> > On Mon, Sep 13, 2021 at 04:05:15PM +0200, Jan Beulich wrote:
> >> On 13.09.2021 15:37, Roger Pau Monné wrote:
> >>> On Thu, Sep 09, 2021 at 04:35:49PM +0200, Jan Beulich wrote:
> >>>> I haven't been able to find an environment where I could actually try
> >>>> with lld (ld.lld); all testing was with GNU ld (ld.bfd).
> >>>
> >>> Thanks for fixing this. I've been able to test with LLVM ld and the
> >>> workaround is fine.
> >>
> >> Oh, good, thanks for trying this out.
> >>
> >>>> --- a/xen/arch/x86/Makefile
> >>>> +++ b/xen/arch/x86/Makefile
> >>>> @@ -92,10 +92,16 @@ efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
> >>>>  
> >>>>  ifneq ($(build_id_linker),)
> >>>>  notes_phdrs = --notes
> >>>> +# Determine whether to engage a workaround for GNU ld 2.37.
> >>>> +build-id-ld-good = $(shell echo 'void test(void) {}' \
> >>>> +                           | $(CC) $(XEN_CFLAGS) -o .check.o -c -x c - 2>.check.err \
> >>>> +                           && $(LD) -T check.lds -o .check.elf .check.o 2>>.check.err \
> >>>> +                           && echo y)
> >>>
> >>> Do we want to make this a Kconfig option (ie: LD_UNQUOTED_DASH) and
> >>> then use is here?
> >>>
> >>> We already have compiler and assembler checks in x86/Kconfig, so it
> >>> would seem more natural to place it there.
> >>
> >> The question of whether to record tool chain capabilities in .config
> >> is still pending. I'm not convinced this is a good idea, Andrew keeps
> >> shouting me out for that, and an actual discussion doesn't really
> >> happen. Yet unlike back at the time when I first raised my concern,
> >> Anthony meanwhile supports me in at least the question (to Andrew) of
> >> when such a discussion would have happened: Neither of us is aware,
> >> yet Andrew claims it did happen, but so far didn't point out where
> >> one could read about what was discussed and decided there.
> >>
> >> For the few uses we've accumulated I gave (if at all) an ack for
> >> things happening under some sort of pressure, with the request that
> >> aformentioned discussion would happen subsequently (and, depending on
> >> outcome, these would be converted to another approach if need be). I
> >> have meanwhile realized that it was a mistake to allow such things in
> >> on this basis - the more of them we gain, the more I'm hearing "we've
> >> already got some".
> > 
> > I see, that's not an ideal situation from a review PoV, as we don't
> > have a clear placement for those and that will just cause confusion
> > (and more work since there are potentially two places to check).
> > 
> > What's the benefit of placing the checks in Kconfig instead of the
> > Makefiles?
> > 
> > As I understand Kconfig is run only once, while the Makefile could run
> > the check multiple times.
> 
> Right - as many times as a directory would be entered for building,
> times the number of evaluations of a respective variable.
> 
> > The downfall of having them in .config is that .config could suddenly
> > change as tools are updated or as it's moved around different systems
> > (yet .config already contains specific compiler version information).
> 
> Correct: Tool chain specific information may get updated, but then
> further options may get silently turned off. Plus to update tool
> chain specific information there needs to be a trigger to invoke
> kconfig in the first place. Merely relying on make dependencies is
> not enough there. Iirc we don't have any means in place yet to
> actually enforce this even when there's no other reason to run
> kconfig in the course of re-building a previously built tree.

Indeed, we would need something to trigger the (re)evaluation of
Kconfig options on every run, regardless of whether they have been
set, unless there's already some magic for options using cc-option or
shell test macros that does it. Linux will surely have the same
problem with this, and they recommend to use Kconfig to check for
compiler capabilities [0].

My opinion would be to go for Kconfig because it's IMO cleaner to
represent the options there rather than mixed into Makefiles, and
should also prove to be faster regarding build times due to the single
evaluation. We can always bring the environment related issues to the
Kconfig developers in order to try to find a solution for it.

Thanks, Roger.

[0] https://www.kernel.org/doc/html/latest/kbuild/kconfig-language.html#adding-features-that-need-compiler-support


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 07:40:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 07:40:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186190.334910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ33E-00089h-3k; Tue, 14 Sep 2021 07:40:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186190.334910; Tue, 14 Sep 2021 07:40:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ33E-00089a-0G; Tue, 14 Sep 2021 07:40:20 +0000
Received: by outflank-mailman (input) for mailman id 186190;
 Tue, 14 Sep 2021 07:40:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ33B-00089U-Ut
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 07:40:17 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4159eba7-7f1a-4d3d-ab25-f05b7f08ac28;
 Tue, 14 Sep 2021 07:40:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4159eba7-7f1a-4d3d-ab25-f05b7f08ac28
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631605216;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=fxiKxw4TGOhF9oQQ/YHtDk0Qky1JfQs0RV5J2w14iVA=;
  b=BvxzB4wJpsQn9+pqjgEzYa+NJ9HPNm2QLjTuuBbYMAUUd3zcxdQ8A2Vr
   io7MHMW2nlCKZ9jFok1fAoZor8+zxkSk7tt9SfzePWv/isI8dA8PJuAqA
   BNuxluUmoHQUGLTUlsCgDQBQ5b0YXRlcyE2UVTSyi/UNbMrXYoU1/6al8
   Y=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 3DWOVslmPQoMsJLPs7/4u6pBxqoK/sIGXSBv/4wma1PjFeXWnbVKc4JtgT8Y+ArlfPB6z7xQU+
 KPSPZ+T13ZQssFNqEUs7d1qL5gevTeU1agH9g0Wnb/E5zGB4BH9ThMaFux5Tvk4w3F/YihvH8v
 AD23kDgqBdUX72lPl8jpWRjDBal3O/LOlkBM9Kk+TwyLhl2+w+Zn/xhKMwQlxyL3nE06jFKgGx
 JOtfTqArIyvP0oCrdjvGJdfXl4JgSQTl7vsd4N70Pf/YKP9ADlqKX+doPihwFJN4+6Fi+T7YRe
 NW7SZyl0uFKEtAj2Xf0GCAsG
X-SBRS: 5.1
X-MesageID: 53059401
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:2gegnKmfNejnTAUCBUXHUETo5gyfIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJKX2jUO/jZa2WkfdolYYW09RtQ6pGBmoJqTABppXtgFCMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09c02YHR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 PhgkIWSQy0NBbP3qMhCeQVFLQVQJKITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpr3p0fQa6HD
 yYfQTxNTRLvSSNzAXwsK89khveFoXvTbxQN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wt
 m/Aumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3ntZoilCOwm0YCDUNSEC25/K+jyaDt8l3c
 hJOvHB09O5rqRLtHoKVswCETGCskzM5QvhaOckB9jq95vX+/xunIkYdQWsUADA5j/PaVQDGx
 3fQwYiwX2cy4eHKIZ6O3uzL9mLpYED5OUdHPHVdHFVfu7EPtalu1kqnczp1LEKiYjQZ8xnLy
 jaWpWAVg7wJhKbnPI3qoAia31pASnXPJzPZBzk7vEr+tWuVh6b/PuREDGQ3Ct4acO6koqGp5
 iRspiRnxLlm4WuxeMmxfQnwNOvxu6bt3MLgba5HQMB6qmXFF4+LVoFM+jBuTHpU3jI/UWaxO
 if74FoJjLcKZSfCRfInMuqZVpVxpYC9RIuNaxwhRocXCnSHXFTcp38GiI/59z2FrXXAZolkZ
 c7AIJrzVC1DYUmlpRLvL9ogPXYQ7nlW7UvYRIzhzgTh1rybZXWPTqwCPkfIZec8hJ5oai2Jm
 zqGH8fVmRhZTsPkZSzbrdwaIVwQdCBpDpHqsc1HMOWEJ1M+Sm0mDvbQx5InepBkwPsJxruZo
 CnlVx8K0kf7iF3GNR6ONiJpZoTwUMsttnk8JyEtYwqlgiBxfYa14a4DXJIrZr17pvd7xPt5Q
 qBdKcWNC/hCUBrd/DEZYcWvpYBubk3z1wmPIzCkcH40eJs5H17F/drtfw3O8igSD3Xo6Zti8
 uP4jg6CGMgNXQVvCsrSecmD9VLpsChPgv92UmvJPsJXJBfm/r91JnGjlfQwOcwNd0nOn2PIy
 waMDB4EjuDRuItposLRjKWJoor1QetzGk1WQzvS4bqsbHSI+2OixclLUfqSfCCbX2Txof3wa
 eJQxvD6EfsGgFcV7NYsT+c1lfozt4n1urtX7gV4B3GaPV2kB4RpLmSCwcQS5LZGwaVUuFfuV
 0+CkjWA1W5l5C8x/IYtGTcY
IronPort-HdrOrdr: A9a23:EcJGiaqYuJ7vy8U24eQfAJ8aV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NCZLXTbUQqTXfpfBO7ZrQEIdBeOlNK1uZ
 0QFpSWTeeAcWSS7vyKkTVQcexQueVvmZrA7Yy1rwYPPHFXguNbnn9E426gYzNLrWJ9dPwE/f
 Snl656T23KQwVpUi33PAhJY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX232oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iBnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDA4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWArqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocWTbqjVQGagoBT+q3oYpxqdS32BnTq+/blnQS+pUoJjHfxn6ck7zA9HJFUcegM2w
 2LCNUvqFniJvVmGZ6VP91xM/dfPFa9Ny4kAFjiUmgPK5t3Tk4li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="53059401"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GfD+BQPsdTQCm2coeDGSxHLGRNXvH4ZbI3ReugqmZpzwICjWuxL5kdN8RMtViVyDTGVxEzJcJszT2KMEkuf77cBos4zGZITIqqT0gsY/XABJ76Tc5cJPgSiSH6PLP4z1WApbL1NdCJrkWQWYUTGHkg6mywKn4BkrJBsO/jr3a0PZLQGPzXQ234Sa1K4hAg8loLDK4jc+AV9fpqOyXAMg5Hnc9zy7C49e2Wacx0ybixsoEz8StP4jUy4n9UOnp1NotshXznBBF++uSv9YRJLgjVegyDf5BJ4i1kWDihjSeKIAw9/LRhUVvHhPn+dTUdIdJBI1xtb4E9RC5+qQZbtEDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Uz/3wz4bM261vF5JSn7CkFIYZz9cSOziYznu+yNe6cE=;
 b=Ctjn5Db3FoxU0cfONiSCCG7MjxRDY+8wPEh9glJLYF+v4qRRrwj20AM0BF4dzE9GR7YCy9gBz47QGHvqPca3G6u6e3h47r2FNV0lysNbAhnT64a0LGLsSaIjF+Hr/ND1HZ7fYNixrcmPU3pOllb1h4T7C6rqRupyFNhVsCBVPSKTBJl41f+l4qDfaN12rDY7cooGwGX971u2vDDE/e3UVYEFuk3vb4Y/EH6oZesrsoIuwWeASp7mTZD5sQ+bm8BGwfA8usP3YF//TQJuU2QqseHfz9J8Be9LCuOhGN3ZWX5Y61A5iYvDHoMkawH5CdTdYmnmXTUcEM4TLF1FLu/BpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Uz/3wz4bM261vF5JSn7CkFIYZz9cSOziYznu+yNe6cE=;
 b=mvTIZMIC4Gxh4MZsaqxbxlEWYMZ5QN7ASOZEiXw89AaYVTU++LcCIWbMS6q8aZ3dou+drHYRJCSiGE5Q+CcbLn/Wr8pv2gela+VLpuPO/pog+G+WE00eBB7/yBTy2JmyMTDM7QebeyZLEZQXfspyPcuJsJ7s/ZlcE5Tmykqdr+E=
Date: Tue, 14 Sep 2021 09:40:08 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/ACPI: ignore processors which cannot be brought
 online
Message-ID: <YUBR2NqDJeucPYGH@MacBook-Air-de-Roger.local>
References: <370f1ebf-b8d6-70f2-b9c5-d642f710c550@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <370f1ebf-b8d6-70f2-b9c5-d642f710c550@suse.com>
X-ClientProxiedBy: LO4P123CA0058.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:153::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2629d739-bd18-4335-d990-08d97752e351
X-MS-TrafficTypeDiagnostic: DM5PR03MB2969:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2969DF590339CB3D6E9D0E628FDA9@DM5PR03MB2969.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:345;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: JhqUc+ifHgXxqVLHOjDfr+7CKt6tw072aVTW/vcWbV+8KoOsnZnOk7l+cYOKKVQuiRn1wXCXsrjGVT5zSjLuDGoDFUhTYnJVChs/9AYfQPo4Rm6zvwNYsyNo6El+ZkBgcjCY5nU6+Isgo/evXUlOH9SMaPWri0+J8nJ79+lbjki2bKgOkRpL9GD5gSuQg8OS4sW3Z81m58eOgSXxR+mziL3ZO7oyZ4Ev5ZjAXkmxgFYy4LPczWmGZ/IT3NxxA5Xu4M6j/CkzMzcwuDuVZ9lvIHodLrAsEKRgyHj46I4XSOthUB0ILZrti+CVVjXMvFBjSGJliTc8VF2VRl0eNZRMpVgURmUTyi6RT1ybwFoy2J+FCLR5lesdPKK+7+GZWuk+Qqv/tcrx7iddSQ/bpK9KxmwsR2N2anplRxFIEj02Lm24RnBYRm7HH6RSM775Abycgy/SzY1lQwAlbVCuxJwTAYFTahGSxkhuxoHClAdq4oLcZe3+Vw4GoX2vVTHX+DjO6/eFfHXTXo6/LS+jGkluL5I1e6sG3VrrPYDu5AMSQgqMISA5r2PF89AZ3fPpUM7Fz54Eqb6oyW4safv1cuQi+pN7SsgcBfvcyWlQIcVaBbDh5o0VLu0GIIhkCGWCvQ7L3s+2wU9MpI8M1B1+Y9SJww==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(136003)(39860400002)(346002)(376002)(396003)(2906002)(4326008)(66946007)(316002)(26005)(38100700002)(6916009)(66476007)(956004)(83380400001)(8676002)(8936002)(54906003)(66556008)(186003)(6496006)(478600001)(6486002)(6666004)(5660300002)(85182001)(86362001)(9686003)(4744005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eEptdjczN0hlVHpTN1pzdmJMbWFvd1ZMcnZiMnNWZnNyOWNSWlppWThkTFBX?=
 =?utf-8?B?aDlpMnRuUmI1RkVka2V2ZE9NYlNVK2dDN3dOQmYxRkJZTmVZMTdNQ2xTR3FE?=
 =?utf-8?B?MHVwWmlEa292WjNwdFUwdDlRc20xQ2VHS2tlVGtsRWxQa3E4RnlNUFEyWUdl?=
 =?utf-8?B?dFdWeHVqeFpWR1dGT252Zzd4dHYxOVdCQWpERkFMV3RZVitEZ2s4Q2RxcUIx?=
 =?utf-8?B?Q2wwNFJxYURJRFUvekdWeFYxRFFqa2x5K1RSTlBDSWJENFI5eUI4L0VHTGFk?=
 =?utf-8?B?bEh6WE0zeVhxMGgxOWlFbDA0S2t5R0U1QXFSdmVjNHA3M3RPY05LSytRZ1Va?=
 =?utf-8?B?QlpCMGluYVJwUm9US3RGb0dOZTFoUjRZTHFLaWw1QkdvbklNWFJHSFIzUmFp?=
 =?utf-8?B?Wm54U3dZa05kMGdmYU1lYkxRbDFWcjFtTkpEWkM0dUQ0N2VveERsbUdlaml1?=
 =?utf-8?B?STdIN0VRNXNXbWhJSmZUd3hrakQ1RStWekJxUHE1QkhSTFU1TmVsNFFIQldM?=
 =?utf-8?B?WHdubEJrNzdwc1dTSm9ycXJKTXJGaXVhOWd5dlNCZWY5MnY5dW1RdFN6VlVx?=
 =?utf-8?B?RWN1VlZLUU5hYzRTS2kvSHZEYkh4Y3I4Y1o0WGs1dzVOa0IxODEvdHpTTDFD?=
 =?utf-8?B?QnRFVEJkV2JIWGJadlUyZDlDeHFnTUJqNFdTd3RIMU5xZFlJYm04V09FU0tX?=
 =?utf-8?B?RUVPM2JPUm9ZUTg5dUlTTmFvcUJNa1RpZUtTYVFkaTFGN1c4QVpBY0wwc2Ro?=
 =?utf-8?B?cDV1dlJoaXlTT1RBSTM3U1k2THdSaE42YzhINFVxQUhha0s3Y0VJdUF3OEtE?=
 =?utf-8?B?dUI2NXE0QUZ2RXlNZDdhRWFmSEJsRG1XMFNVc1pkRThmd2VzR0VMR2hEWFBV?=
 =?utf-8?B?eWVrakNrc1BnSUxma081R2JhZlZSNCtheFdZTlJWc3ZsakRLOTVvR1A1TnIx?=
 =?utf-8?B?UkNjSHFrVC9wZFlGYnhMTWxYVnZnTnlnTisyRjVkUzZkdmx3OXFxNW4yTmYz?=
 =?utf-8?B?ZFVlU0JWM01wQU1DYlRqWWVkeWZwaHRPTExqN2NxRXd0Sk9WaUpnbXlaOWls?=
 =?utf-8?B?YUVNSGNpMWtYR1BBbGt0YVQrS1RiWWpJcDhiUnBabHd3UTlPR1RsaFJJS3px?=
 =?utf-8?B?THJsaFdWVEhmVWc3QzE2RTdLZ1hHOWlpSTR0WFcvWlR6ODFDL2U1eHRWODN2?=
 =?utf-8?B?UzR5T3gycTNhTU1MbXZKN0EySmVqbEVIdkhpdHN6SkRxd3BrZHZqNWlTaW94?=
 =?utf-8?B?bXYwZ1hyMk9vSmJML0JQaURpdWU0SGhoVVN0WUhZSkNwak1BS2Q4T29iR1l5?=
 =?utf-8?B?OGFBYXRlQ0ZUTjJHSG1scGtibE9PQUtoaUdmUGNBZUhnRmx5bzFmZmZKNTFH?=
 =?utf-8?B?WjV5WVhKMEx3aFdhUkNzTDRuUFl3dXB3cHFadGsvTWl6SmhiYndqNEY3NGhG?=
 =?utf-8?B?ekVrSjI5U3M2L1hSYktpREFadVhUTmZPYktiTEJmNWpWeFllTDF6NHovSEtp?=
 =?utf-8?B?Sjd1TVFKcTZvQVBJb29IbXFMNlBpM3p2NmF3d1ZpWHlmbTlQK0NaNkx5ZEs4?=
 =?utf-8?B?TzhESzJoR21RSHpwbXBsV05uVlFmRmduTU9FV0lBYTExcEFBdjNqc3hER3Na?=
 =?utf-8?B?ZlNxM1htMGVveG5qTmEwMWE2eHdnZEVDeHNUTnNaN2JLdlRTZTNHaGZ6OGpN?=
 =?utf-8?B?ZlE5d1FXaGlHeW9pUGMxZkozL0xVUnMrMDQ4MzB6cklBYVBBWTJFQmZqZjZ2?=
 =?utf-8?Q?rYx2QwDj7WPgdxQ2/PIPkXBMMK5qvSi0/1gT0o4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2629d739-bd18-4335-d990-08d97752e351
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 07:40:13.5238
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +DK3R1RBDCpVqZ1LSMyZF0d6G1+nxIiuzhHvRgXxtAQczYrt2lqRqLZzL4HAJFEf7OTy6zEICgm4S25XIwMs/g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2969
X-OriginatorOrg: citrix.com

On Mon, Sep 13, 2021 at 09:54:01AM +0200, Jan Beulich wrote:
> ACPI 6.3 introduced a flag allowing to tell MADT entries describing
> hotpluggable processors from ones which are simply placeholders (often
> used by firmware writers to simplify handling there).
> 
> Inspired by a Linux patch by Mario Limonciello <mario.limonciello@amd.com>.
> 
> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 07:53:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 07:53:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186200.334920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3G9-0001KQ-AH; Tue, 14 Sep 2021 07:53:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186200.334920; Tue, 14 Sep 2021 07:53:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3G9-0001KJ-6r; Tue, 14 Sep 2021 07:53:41 +0000
Received: by outflank-mailman (input) for mailman id 186200;
 Tue, 14 Sep 2021 07:53:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ3G7-0001KB-Bm
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 07:53:39 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id de58f810-1530-11ec-b402-12813bfff9fa;
 Tue, 14 Sep 2021 07:53:37 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18E7e5jJ015214; 
 Tue, 14 Sep 2021 07:53:36 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b2m9x0jjh-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 07:53:36 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3374.eurprd03.prod.outlook.com (2603:10a6:803:18::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Tue, 14 Sep
 2021 07:53:32 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 07:53:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de58f810-1530-11ec-b402-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PpdJe9EgWa7qxdPijNACSixn5e2pNqf2OisONkqSzs7Cje7S0Z+CLbh6+1NRqXbv6VIwbbqDExKaDHgDOhDSkPqtUHlGJXpps0qpmveOiQvscCtULgaAsjVqKR1N/GAe8tDwabFFyWOqvA8ja1K485wJkcmK99GYZo9ZvLxPdnEBbRGuXTgr5euCwJkDwBulS4Z3IWcG0n7hZe/DL5ku1E6bSUt9fCVx5quuKbtB40yHkOeTvH3Pih8Sgrw0ILuctp8RnDO46pmH3VHbUzEiYAGzyIrYsZKTIVWDh0cgtuFlrtKMiHyQ6Ra8J/ou0ztVzF19WSzMGX+z8x7TJLRPjw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wlmZPGeXN4ZuQgFXCANZ+TPGqQoW8sCwPr77bYDd17I=;
 b=ZkMMu8H04bEYBmQB6arFwwgDfQBaX0aKfZYkkLUaiS7LwvM+/WW1ZiIqiei1BXcjVTN47VY/iWO1JeIRNQCSRLs+kLq2bqhgCK+3OzJcin5I6Z2dztQEAUGzg1wN3VKSkXamzgvq3whjyaUN+DaiOz+N9YzMFWd6BF86l7Io3cK9pfmBP8gHk13x8LYInTVMsDWE4CB2+bAYJxBexMathyU7lN/QWYsERIiRTRPqXdXNeY6mceQm0XNxFddvTmPtO1s1Br74ehrzpmBSDKOm0bWlEy7icqcKXgCO2JfvaY3nk3rY6mNIpPIqPxGFZUlXfe2oMNlnEMl2MoDJFYn7hw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wlmZPGeXN4ZuQgFXCANZ+TPGqQoW8sCwPr77bYDd17I=;
 b=4wzQc7xUM9mCt8a4YEpgaPr5WuDX4BoApRdUi0sZ9y4UUVvyYCzeZ7mWhJl2SkVln1aOPMHHaCm0slWesUaI+9J1L1RdozR89fiQVoyM1g+11J1ju2pEgRSRJsXI2W3+tFFFcN6FBkDKlhL+wSQJEPmH91ig+RVIewdjrvoVl+QXb6kJ14sEE8FWnxxPal5x4EecikvjN6atlpwhoH2Zdx380IKRryCIX5HD+W2P7DN/OfVkhsbXCRVTBeXUyoDDqgXnV6tz0uDGeRfhUlyS585sii53xL3TsulEpHxRcsUcg0AoEsOjN0qINJZOdyCgEwPZ5DEoxuituHkohpkfEQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXqT2cTKPAnBLuBkixTqpTrxO8Dg==
Date: Tue, 14 Sep 2021 07:53:32 +0000
Message-ID: <0053165a-7adc-5f93-4715-1ff888a533cc@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
In-Reply-To: 
 <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 48a73d50-1f5e-4d00-cb9e-08d97754bfac
x-ms-traffictypediagnostic: VI1PR0302MB3374:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB33743EDDCEE7635D3FA68C8EE7DA9@VI1PR0302MB3374.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 yd9U1XD/Pi2zRXhGFwFsCoRLOjTFTjuFyekBpiaTUwzGyN8YbEUAWqWNtIsonVK5FwsQ60DmJT7vOfqwcfvhqSdeSk1BXK/1b/m1QzWojhDaNKA0Xu3sE39ss9qruefYOnnuNSbFWFdCTaX+1QBqy0qgtSOYoiEyZ9M18AXK9NPFbKUHVWQW1DsJ5A4HvX9GonGTvfhtHu9evybZ7aXCmuKBrEU0HI2k4N3gtta1qg12YUlhzEA0D3rcrN49YNrSyRfMN7ngRi0bRiRKLcgR/iULed+fAbWgqzkTMetBNvZIlsIWeW855VLtBFpyhgG1QcTisBxFXCqETEEdLFIG2DcgSLEJbj5Hw/NQxkVh+6jqfEj83Fdfdf4iUAcMsHyOYRO5kZFEYui/8xsjZApvqH1f2y4Hqvnqwe4OkgS+sS4sVq7ZCBBgWU9suoZM1TTpyKT/4vFe5Czokn65Dr2tM8XogLjv3qWPJiD/PxRZKGXx4sb4+q3Mm6E3NFguUj/eAEUr37Tv7Dx3RtuxEnX+ybi/obSvwXGA9HYSqOPZTTOL3OAMxFjiJuxcUGGCG55t2dKvGfpAazFy/FDjz4N5JQhztLpUG7z5MgVFw7TTbVRQ+pgBRMhKeZzL8w58sfZk6ONmaNfu2568/Gr12IpFbfNhEIZRsSoaz+DaoC71lHHv/Y4nHM0jD0mrtURBUjVqgSZuXU6igcKm4MzyxNwowEMBRJeIO5L/64c2RfNwvleZvg/6w1R7/J15P8q6Iu+c
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31696002)(8936002)(31686004)(5660300002)(55236004)(186003)(2906002)(66446008)(4744005)(6512007)(91956017)(66476007)(6486002)(38070700005)(36756003)(66946007)(122000001)(86362001)(71200400001)(54906003)(38100700002)(316002)(110136005)(6506007)(64756008)(508600001)(8676002)(53546011)(26005)(76116006)(107886003)(66556008)(2616005)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NWlrTG00eXMxVWFob2hIbzd6WGJWQzhBc2wxdzZJZ29nanptMnpjTmdiL1ZI?=
 =?utf-8?B?K2lJMjZWUEFvZENQQ3NQaTlHdnN0QU8rMTNrMU83WlNIbXh1ZmxxY1p1WGtE?=
 =?utf-8?B?dm4rTGNxOWNNZHlLQnpmU08rVzgzVkMvRmxtWmVtOGpzeVF5K3dvbUUrSzd0?=
 =?utf-8?B?ZVdLM05CTm01SGJuMnJQbGo1aEhBTVpBWkczYXZCQnZsazBNaTFZWEx4NjA5?=
 =?utf-8?B?dE5XZkd0d3QyUFpUMEx5RDVlU0w5aVVhKzhOZHBMSTJTSnVaemw0UU1OY1cy?=
 =?utf-8?B?NXduZ09BM1o3bmsxb2tBUVViTG9PVGh1b2o2Y2NIWEdZWms0WXVSQXN3cU9M?=
 =?utf-8?B?UWc5R2N3aVBDdGt4RFdtTW5NQmg2YlJnOWxvS1hVN3ZubkNuOEJqWnlqT2U5?=
 =?utf-8?B?a0tIay9MV21kV1htVW9IcmpQUy9sNytKTXc2NkZka3BwVW40UGJ0WmtpMk4r?=
 =?utf-8?B?aE9kZ211OG5KSEdWd3pleW1NS2U2S2JvK2dJNFdKLzRYZ2RjVUxZWGt3aWd6?=
 =?utf-8?B?QkFhd05DaUJmczNKU1FyWEVaLy80NFhSbW5leHVWeHpIbXNWYkw4aEhQazJs?=
 =?utf-8?B?S083aVFZYlJNR1JzcWJYRkljNWczMXMzRXJRQzFGd2V6WmNROTZiSFpEQ2l5?=
 =?utf-8?B?d2ZhdW5NNFpFeW8xd1EvL1dZTmU0cFZPRndXaHJLR21vSjVLMEc2eGJQOVJ2?=
 =?utf-8?B?SUZkaWozYnV4WGZqQ2xLcFIydlkzQm44Q3pWeXhCZEgrVkE3YkpNOGdXTjdD?=
 =?utf-8?B?a0lhT3RYM1lwMzZzWXQzZmZPZkc2UlE1YmxacFR6VEozQ0V6WTFzWFVUTlpS?=
 =?utf-8?B?dWJmdUNZUThrQlE3NHhSelZ1d1pKQTNBS2JpWmhJTDcrQUZoa1lzMUJnWE5X?=
 =?utf-8?B?S3doYVRCRi94MlltZWJTOHVWM29aUVdLRC8rZW4xR0lKbytGVVJ3ZDRJK2hK?=
 =?utf-8?B?MGtOelNjVU95clNHTDZoL2lxU1hiOFpwSXdXZU4vdGd0T1d5eCtDVG4vbDJu?=
 =?utf-8?B?NGVncncxWGZ1REJRRXpqQ3RuZHo0MXd0R0VGRG1kTnA5MzU1ak9YejljaGta?=
 =?utf-8?B?UUp6MGdjUjN4OWNaMUlpbWl6YVA0cXp1bWpPMmZxSFlVWHpvbTBPdE1hcEtF?=
 =?utf-8?B?OWcxUEJrSWZwdlRrb0dZaXl2YU9tS1AyYml1amlzMFBuaDljcW9ZcGQ1ejBw?=
 =?utf-8?B?M0szZDM0N2NJTDBrRXNGRVdkU1RSaDhtcFN6b0h5aTNkcWtocFY1cksvWTVr?=
 =?utf-8?B?SkYzdmpMbGVSbkpyeFVXS2pYUjd1Nm9jMnNna0pPMnB4Y3RINDk0MThZaHVM?=
 =?utf-8?B?VmtyVDJFWUNrakNISk1ITko3b1NLUFN2KzVWa0FNclE2Mlozb1Y4UUVVUi9F?=
 =?utf-8?B?S0pKK215TG5RMTNiNElZM2E4c0ZQRnpmZmMwVlJWRW50cmVtOU0yc0phZFRt?=
 =?utf-8?B?U0VFYno0Y1VEbzdMZ2RiL3J3U2wzYkx5cmcrditJVTVkTGg5dFdHQjYxMGJF?=
 =?utf-8?B?MTJKV0Z3SEZUczhaOHF3WVFzajlVVkFOV1ZnOGN4bStCalQ3Q1I2OFZiTHdN?=
 =?utf-8?B?cnNQT0haWmZib2lQMStXck1SKzBPZ3JDRURLR2J3UlBCWWRIeGlsWnZaY0E0?=
 =?utf-8?B?OGgvUjFSRHg4bkZmRHFiSEhQZTE2Ky9tS2tiMmxWYld1UVEzeFdnakZLaEtE?=
 =?utf-8?B?S3F5cit2cmVKa1FyT1g2dzBTV3BLOHoxck4zdmRDbTNUVkZXOGJ3a25memx1?=
 =?utf-8?B?amtwK2pwenZjNEJ3NHJZWTU2d1pQVTdYZkJuRFNDVFAwRUNnNHY3OXRJWWU2?=
 =?utf-8?B?cGEwa0sxdGIvaHlqbStNZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <016E1436C005B34EA075979FBE1F3B50@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 48a73d50-1f5e-4d00-cb9e-08d97754bfac
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 07:53:32.5400
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: AZcTP7lFl6lGJb8bKhYdXCp5fBjtiqpMPmaKniSI6F+jOHu15hyQFdgt1zDB4RhfeG9/bq/fRq9iNrOmMiyxgq7N5DYweiIBFCa7ZK0YdjDTFY0O9XZzsbYBUH5d6stX
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3374
X-Proofpoint-ORIG-GUID: n2sYSzv3i6TQ5i3KW59qKzxQsDKT9o2S
X-Proofpoint-GUID: n2sYSzv3i6TQ5i3KW59qKzxQsDKT9o2S
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_02,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 malwarescore=0 phishscore=0 suspectscore=0 adultscore=0 mlxlogscore=999
 clxscore=1015 bulkscore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140046

SGksIFJhaHVsIQ0KDQpPbiAxOS4wOC4yMSAxNTowMiwgUmFodWwgU2luZ2ggd3JvdGU6DQo+IFhF
TiBkdXJpbmcgYm9vdCB3aWxsIHJlYWQgdGhlIFBDSSBkZXZpY2UgdHJlZSBub2RlIOKAnHJlZ+KA
nSBwcm9wZXJ0eQ0KPiBhbmQgd2lsbCBtYXAgdGhlIFBDSSBjb25maWcgc3BhY2UgdG8gdGhlIFhF
TiBtZW1vcnkuDQpbc25pcF0NCj4gK2ludCBwY2lfaG9zdF9jb21tb25fcHJvYmUoc3RydWN0IGR0
X2RldmljZV9ub2RlICpkZXYsDQo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgIGludCBlY2Ft
X3JlZ19pZHgpDQo+ICt7DQo+ICsgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlOw0K
PiArICAgIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnOw0KPiArICAgIGludCBlcnI7DQo+
ICsNCg0KU2hvdWxkIHdlIGNoZWNrIGlmIHRoZSBicmlkZ2UgaXMgdG8gYmUgcGFzc2VkIHRocm91
Z2g/IEUuZy4NCg0KIMKgwqDCoCBpZiAoIGR0X2RldmljZV9mb3JfcGFzc3Rocm91Z2goZGV2KSAp
DQoNCiDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsNCg0KVGhhbmtzLA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 07:59:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 07:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186211.334932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3LE-00027F-Ug; Tue, 14 Sep 2021 07:58:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186211.334932; Tue, 14 Sep 2021 07:58:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3LE-000278-Qp; Tue, 14 Sep 2021 07:58:56 +0000
Received: by outflank-mailman (input) for mailman id 186211;
 Tue, 14 Sep 2021 07:58:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ3LD-000272-Sd
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 07:58:56 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4aa556b4-7c11-4efb-839d-4a4e8b0d6f98;
 Tue, 14 Sep 2021 07:58:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4aa556b4-7c11-4efb-839d-4a4e8b0d6f98
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631606334;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=XrDpap8pmKJKkv9I71CpW+Ut00pu7KKxgmuVvD69BXw=;
  b=HHzdORNLuciBdKibzhagYpMRv4ElzhhHMXGKuJMlPoM8AjNZ2bGXccjy
   Lzwmu5cnTa2CKgxAbWtpZbe6iv8yb/HTaJqO4E/EeXlnJv6cFItmQ17yH
   T7QGoZ1K6rin8pNYsN/oV3UJGgTys6pe22MQYCnpnaBAyoik5C7GEUfjc
   g=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: Y8eriGLJivOF6AdwkHQTYDT0uyp0zxMLOgklhYt7Qcep5cxc/Wak1gtFfHG1BgSjGXXnzLkRiD
 sgGrrbRqxODqyjfDVndCIhhd8D5dgZsQ/gUY5sD2oQ2cVa+L6MrXGurcWFHnXVDfSaFQN+PauZ
 NJtbd5KyTX1OVvQRGA+MecXLCWB2j5kEdl74eU+lJdCluFpzEasNr8G+BEQoREP0BujLeY0ha1
 3pvOqjgTIYZpZFiQq2kq8ZmoD+3afcZZSPWZz/anQCcTF808N1CBQucon99EBTASh2dO/HyFf8
 duF3RQpVDnWuKxMaDlldwCE8
X-SBRS: 5.1
X-MesageID: 54423534
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:7DLP7qil4/+B+z50mAOoD/zVX1617xcKZh0ujC45NGQN5FlHY01je
 htvWGuAb/mDM2qnft1waIXko05T6J/XxtUxHgpkrShkEyob9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8x3YDga++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1tuoWvexs1OZfJxvoaUxhdNxFVBaRZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9t2pwURquFO
 6L1bxJFMRCeb0Z0JmspAZ08oPaw2HKuMB9h/Qf9Sa0fvDGIkV0ZPKLWGMrYfJmGSNtYmm6cp
 3na5CLpDxcCLtudxDGZtHW2iYfnmSL9RZJXHrm//f1CnlKe3CoQBQcQWF/9puO24ma6WtRCO
 wkX9zAooKwa6kOmVJ/+Uge+rXrCuQQTM/JZA+c95SmOx7DS7gLfCmtsZiBcbNUsnM4tTjUxk
 FaC2djvbRRDubiURHTb0rqxqjq2ETcZJikJYipsZQgC5cPqrMcsjxbMZtF5GaWxg5v+HjSY/
 tyRhHFg3fNJ15dNjvjluwCc696xmnTXZi46/xrlQn6i1AYneomlZomM2XPj5OkVee51UWK9U
 Gg4d9m2tb5VVMvWyX3RH43hD5nyuK3Ua2S0bUpHWsB7rmXzoSbLkZV4vWkmTHqFJProbtMAj
 KX7ggpX+JYbF3+jd6Yfj2mZWplykPSI+TgIUJnpgjtyjnpZL1TvEMJGPxf4M4XRfK8Ey/1XB
 HtjWZzwZUv28Iw+pNZMewv47YLHOwhknT+DLXwE8/hX+eXHPyPEIVv0GHCPcvo4/Mu5nekhy
 P4Gb5Hi40wGCIXWO3CLmaZOfQFiBSVqXvje9p0IHtNv1yI7QQnN/deKmuh/E2Gk9owI/tr1E
 oaVARMBkwuj2iSccG1nqBlLMdvSYHq2llpiVQQENle0wXkzJ4Gp6aYUbZwserc7sudkyJZJo
 zMtIq1s29xDFWbK/Sozd574oNAwfRinn1vWbSGkfCI+b9hrQAmQoo3oeQ7m9S8vCCurtJRh/
 +38h12DGZdTFR5/CMv2ae60yw/jt3Yqh+8vDVDDJcNeeRuw/dEyeTDxlPI+P+oFNQ7HmmmBz
 w+TDBpB/bvNroY5/cPnn6eBq4v1QeJyElADRzvQ7KqsNDmc9W2mmNcSXOGNdDHbdWX15KT9O
 rkFk6CiaKUKxQ8YvZB9HrBnybMFy+Hu/7IKnB55GHjrbkiwDu8yKHexwsQS5LZGwaVUuFXqV
 xvXqMVaI7iAJOjsDEUVeFg+du2G2PwZxmvS4PAyLBmo7SN75uPaA0BbPh3Kgy1BNrpldogix
 L556sIR7gW+jDssM8qH0X8IpzjdcCRYXvV1rIweDa/qlhEvmwNLbpHrAyPr5I2CNodXOU4wL
 z7I3KfPitywHKYZn6bfwZQV4ddguA==
IronPort-HdrOrdr: A9a23:RljdXKG0+dnTIdvMpLqFcpHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HmBEClewKnyXcT2/htAV7CZnichILMFu9fBOTZsl/d8kHFh4tgPO
 JbAtRD4b7LfClHZKTBkXCF+r8bqbHtmsDY5pav854ud3ATV0gJ1XYGNu/xKDwReOApP+tcKH
 LKjfA32AZINE5nJPiTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1SvV
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfpWoCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8DLeiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6NpuTgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeQv003MwmMm9yUkqp/FWGmLeXLzEO91a9Mwc/U/WuonhrdCsT9Tpd+CQd9k1wgq7VBaM0oN
 gsCZ4Y5o2mePVmGp6VNN1xMvdfNVa9NC4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="54423534"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FlqjMiNDO6xOk0y91ueXrfzcfjKks958L0WdoeH+hxZ4EUu7XWV0ctBYIQPMdAH12cMkR/Fn/9LsYOyi3sUJiTk3brkjR73T8FDj/GVs+63o5eWm2v47Zqy/BHNYjimk3svV1JI5Ydy+BbwueXX7m+c/XiEG92IckwQDC0RANp/tNUxIzyc2fcnEQ4v1xDTMsh0zGVYlb3IqfzDQsl+fGjalODkSlibDZt2Dn7vK6ztX6BfnikBzYc+fKTiEaCFiF3JbHxA330ifo5YPb8dvtCxWxfV2rpvkuZfqsNxbcLFe6H0QXjv8W0LHUCU3XhJb6e5BfrmVrCUy6wKZg4KeLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XrDpap8pmKJKkv9I71CpW+Ut00pu7KKxgmuVvD69BXw=;
 b=Cz/IvXD78ZF9QjV7exz7Ylo7prDl8j2wIy7J8ZIF26AqD0nkzLqINMOVYCZsx+3bXp1hc8tMaLIIJL4D0Uy3j3fLBHSPAf1iJMuju/RlEbrN0r5Pd4o1zXjsDfHdNfAud6YWxjLfbOe0HBz2/QV6jawTFFKFJikMMmgKpsiCn6hkGJ7hAgDQwnoiGIR1BADEFqjOwhat1kNSSlvQrmn39IcKPAtLjDGqIPNUDEs/sxQsgRmMmu9xC2oDUk2aiKdBWSs+7kocWrMxQ8JzZUnYRAHQAESjUMd/YfZE9l1b3MeJ922QfLaTKwpA+CNfpkCiQE2V3v2Cz6EqAsNa8OsRzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XrDpap8pmKJKkv9I71CpW+Ut00pu7KKxgmuVvD69BXw=;
 b=ZU2V8WMb9WJM1cg6mFxCgIfbeDf3BhGzeUK3V3hC12FOTVGlBhU7+4tAGI51bv8iqfkBGSBCuoQ+uYV47SL3cYoUb4+0vxtFydmqabbfaN0AXglVHzBtNbIWZNxo4YDBEPgRHyb8Do7OKs8+ft8Ct7TYKj1LebWM/B8GqCUHEVE=
Date: Tue, 14 Sep 2021 09:58:46 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Xu Wang <vulab@iscas.ac.cn>
CC: <Damien.LeMoal@wdc.com>, <konrad.wilk@oracle.com>, <axboe@kernel.dk>,
	<xen-devel@lists.xenproject.org>, <linux-block@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] xen-blkback: Remove needless request_queue NULL
 pointer check
Message-ID: <YUBWNkD3lECHrhAp@MacBook-Air-de-Roger.local>
References: <20210910035918.24907-1-vulab@iscas.ac.cn>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210910035918.24907-1-vulab@iscas.ac.cn>
X-ClientProxiedBy: LO4P123CA0117.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:192::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 200dd00c-22ad-4f3b-31fd-08d977557de4
X-MS-TrafficTypeDiagnostic: DM6PR03MB4220:
X-Microsoft-Antispam-PRVS: <DM6PR03MB422044049DBC424663C4D83E8FDA9@DM6PR03MB4220.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: kD1dkEX/OIhgH5Q8Yk8YaC4GuVeMqE1CKJTsmXurRGIEi4hx3BXFK7TKKEAL5XWUw8Jkvp+6olQ1brs8sOO4YX/pmJVs1Df+bvVAxAHz+Un8za3d7JtDLBm2igpgXtOnh6coJ3vpLFZO3FuXcZLDu/3z2O7JhmajojFNrVPdOQ5KJZSZgAuOKRL/zOoXE1VNAQz7vfNPLZKoJsZcFAdFI31Np7RqIvjXZ3aWQmLjo/qVOXvhouootiA25PGe9Lpn3sAfP+PZ/+5KSnP2ZuQWQzzVgXos3pRk0mYLWzlKzT4xQ1oPSvONiiW0ptDOePEmagh9h5EUeIvvaYR1KxU0DPrFQaWHNu1yRM9jkAPSpqpzhMGtVpBHPaAXMYpUbI3rynouehxRjPGIdIj/YYE6iaR4RCFQTESVrvpANAKGiIWtbKxRGa2f8MjUCN8MXlCSv+jox039V2QxbIWSGLTgKNtXwMpfQBsh4b1O9mQ76XsLiPWG22MbposmCyqX0xl56pxxKQ2qt93ZkB+XY+V6+lxLfndKc6ISIBQ3CWpG+beHmePh2uqOsnqbkctQulyq0FE38iLxXLKl7zeoRJBYBUV4mLhqOS32fs4lf+2xIsFgRgeKnO0n3VXlf6iuVBITOMa8REsau05c2P7IxNnNUw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(39860400002)(136003)(346002)(376002)(83380400001)(6496006)(6486002)(4744005)(9686003)(6666004)(186003)(5660300002)(8676002)(85182001)(38100700002)(956004)(66946007)(66556008)(66476007)(478600001)(8936002)(2906002)(26005)(316002)(6916009)(86362001)(4326008);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RzlBK0lRMHdta1dWMThadEM4QzJ3NmZaeFJGQWdzUS9HbzhyUUxCZGxYNXBy?=
 =?utf-8?B?QnQ3VHlkeERrM09JL2hBUzhBd1hYUFAyRVNFNUlyZURCZWdYMkQwcFZpcmMy?=
 =?utf-8?B?UkpkNlZ3YU1qeENhRkJxMU9qaUpLQW9kQnlxa24zUFFEZ2FFL21JbUsrRjVO?=
 =?utf-8?B?R2JKRDQwSnNGakJ1aXNEeUlJekJrTWhDSzVvTTZJUnIxVGE2ZU9SMWNWZG1Z?=
 =?utf-8?B?d0ZHdUI3K3hWVDhxd3FKVnYrU3duM3ZjM3k1dmF5YVRCS3NYdE93RzJuVlVH?=
 =?utf-8?B?eklBQzkyMmc4RDc0bGlnay9nV0J5UFpUaEgzYkhuWk53eDBYNUovb1NiNGdN?=
 =?utf-8?B?MlNWeVhUK2JhdnlrWS9zbjhaRVZEZ0tlYVNtRmVTWHZCZmJvSXdSQ3MyWnBV?=
 =?utf-8?B?clA2RG5JTno3c3p4ZGdZdmFOSGs4M1A2dWlzeldBTFpNdFhhdTV1WHdPcUJH?=
 =?utf-8?B?R044QlNjVjVSUDE3eXF6RTNTYkV2M3MwMzRXRlRua2hXWnl4bURkRHdhVGRO?=
 =?utf-8?B?VW96dXE3WXBQTE9GajJLV0FYVW0wZS8vVU1jaC9YSWZMckp0dnpveUdZMVFk?=
 =?utf-8?B?bFlQRVNpWThlVm0rS2Z6UlREVTFNQTlWczhOb2E0aGV0KzNkNnNnd3dJWnpj?=
 =?utf-8?B?NFc0OFNYWnFLdXViazNpbDR1SmFtMTVtM3hVMW1KT0U4RGczcVRub2M2ZFJQ?=
 =?utf-8?B?MFVTekorUnhBYmREWThycTRPNGVUbVZ2L2tHdklwQmxyMnlxTEFWcFl4ZW04?=
 =?utf-8?B?ZTBtdm9rdlBzNEU5SjZWekd4aG9ZRURZTVBGTWR6UnJBTkhXN0VSY3g1aDlk?=
 =?utf-8?B?WmhTUVU0aVdQVVFRYktaN0doeVN1eWRKNktoZ0tYOHo3ZDVROVAvMm9tSUoy?=
 =?utf-8?B?VnVJdHBIT2JlaGVqSW5qYjc2QzBTTUIzV0g5ajFKWVUxbnJvQzZDWW04blFx?=
 =?utf-8?B?MjhVTU10NVdwUUpUQ2NPdHFxM0czOGhVQjdYMjQrRVRNbUtheG50alFsMnVv?=
 =?utf-8?B?eTZSN0NQYzFFUGpFWldhNHRDa2lFY0MvTitBSEwwREx5Tm1jVnZnbENHQ3BT?=
 =?utf-8?B?VVp2aTJ4MzF2K3I4eWp5QzlUb21jcmNoU0Y0MDVZS2J1ZVovS1E2RWo3TnRK?=
 =?utf-8?B?enhnSXQ5d2ZidGlpZ0hObWh3V01HWlg2VW1GdHJ2SEFwbDVaWjhqYWVnb1FI?=
 =?utf-8?B?ZkNqN3huemI3QnBSTWFaOTVZZE00NWN6YWpxTVArMkwvcGZXQ21Na3FwSkRX?=
 =?utf-8?B?cFhBRWR5aURXK0RORzhrRlUwM1dpd0R3YnVQSGNJWVRKZzRRdEtBaXRLczRv?=
 =?utf-8?B?TkNCU2RhNGZ0d2szVFNrSSttbUxMVHlDWXZMbGxNQVB6UW5aMTRRcFg5ZElS?=
 =?utf-8?B?dVlNYlNIdFJ2OUdCclJwempIb0hMTS9vZ0RydjBKU2MvVDA4bGY0ZjdmZWFF?=
 =?utf-8?B?eHVkaGZKVVZrbkRhcUR6RTJNTTJHMW56Tk5FVXhFN0JkZnRaVHE3SVRIZDFN?=
 =?utf-8?B?Wld4dkt1TGFMQWNkcUNvUXp2Y1ZwUWRxWlpZSGNWS3huSWgwcTQzSGltRXk3?=
 =?utf-8?B?MmF5Tll3RS8vb1hKakZjRmdldFBqTU5vRnpDU1lrMkI1Vmh3cnlxNk1pUWlS?=
 =?utf-8?B?aisyNjVIdnY3Vk4vNkNhZTdEeXJOV0xRVFk1STJDaUh6UmN2TXpjb2VjbDlH?=
 =?utf-8?B?Y1QvWXIwbmEyVXlqZ1lUbXZacUpiZ3BCT0crMEsyVDRRQjFJVGpGQUs3K0Rk?=
 =?utf-8?Q?r6lpcHzVBOD/bgBnviw3ZP5kImRYDj18gHpQUz8?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 200dd00c-22ad-4f3b-31fd-08d977557de4
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 07:58:51.8310
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: j3U5A3nWbujbaU3HBXAXahv/3j6GwDQmoWCfCKBLFfIZK2+5Z/kUBNFgV8VbxCwVnIyta7FGy5EsO9LbXLUBuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4220
X-OriginatorOrg: citrix.com

On Fri, Sep 10, 2021 at 03:59:18AM +0000, Xu Wang wrote:
> The request_queue pointer returned from bdev_get_queue() shall
> never be NULL, so the null check is unnecessary, just remove it.

There are other places in the code where the return of bdev_get_queue
is checked to not be NULL, like __blkdev_issue_discard. Should those
also be changed?

> Signed-off-by: Xu Wang <vulab@iscas.ac.cn>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:04:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:04:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186226.334942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3Qx-00047i-VU; Tue, 14 Sep 2021 08:04:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186226.334942; Tue, 14 Sep 2021 08:04:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3Qx-00047b-SX; Tue, 14 Sep 2021 08:04:51 +0000
Received: by outflank-mailman (input) for mailman id 186226;
 Tue, 14 Sep 2021 08:04:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ3Qx-00047V-AG
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:04:51 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0fc4bf0f-e16f-4b37-8c19-097088afecd8;
 Tue, 14 Sep 2021 08:04:50 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 93C0A2209C;
 Tue, 14 Sep 2021 08:04:49 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 679DD13E55;
 Tue, 14 Sep 2021 08:04:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id B39pF6FXQGEPFwAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 08:04:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fc4bf0f-e16f-4b37-8c19-097088afecd8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631606689; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=xZv9emHYY+yxYSbveEAhPUqXsMH0BbgkpDDj1kucJH4=;
	b=F5uSC0rzCyAjPcq0hyMXgPO/L88HDxSapjV6r9Nk5T3to3fO2GkqMVFXlsZsWaMGy9jxAd
	PnSqgOtK7dtJco7h2QphWJlzSKVN+mmyfnLfpGn11GU5BSVe2uU/kfrUctUYg6GV4D7Nad
	oruLvkD/TY7Bmvx/yj2wzHP2owRwIhA=
Subject: Re: [PATCH] xen/pvcalls: backend can be a module
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <c5dd67c8-9292-d91b-dfe6-1ee397fa261e@suse.com>
Date: Tue, 14 Sep 2021 10:04:48 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="GXCKBqAPGfAuVlhONvxXkTU8QgRAYAYnS"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--GXCKBqAPGfAuVlhONvxXkTU8QgRAYAYnS
Content-Type: multipart/mixed; boundary="6kZ4hmri5TMIbfx8DVJmz0r56Iu4DOcX5";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <c5dd67c8-9292-d91b-dfe6-1ee397fa261e@suse.com>
Subject: Re: [PATCH] xen/pvcalls: backend can be a module
References: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>
In-Reply-To: <54a6070c-92bb-36a3-2fc0-de9ccca438c5@suse.com>

--6kZ4hmri5TMIbfx8DVJmz0r56Iu4DOcX5
Content-Type: multipart/mixed;
 boundary="------------1BD2A8630832860143C174B8"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------1BD2A8630832860143C174B8
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 14:17, Jan Beulich wrote:
> It's not clear to me why only the frontend has been tristate. Switch th=
e
> backend to be, too.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Pushed to xen/tip.git for-linus-5.15


Juergen

--------------1BD2A8630832860143C174B8
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------1BD2A8630832860143C174B8--

--6kZ4hmri5TMIbfx8DVJmz0r56Iu4DOcX5--

--GXCKBqAPGfAuVlhONvxXkTU8QgRAYAYnS
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFAV6AFAwAAAAAACgkQsN6d1ii/Ey8G
lwgAiF+Gq9LN2li0UkCKbMs+kfUngAhO0fpVOO6d++eE147kQKivgp8G3q+7oC2eRzbLjBK5uor6
NMn7JK8bMq067/f5/CYVi1VuGH2uoC090JOgMWssFNzg0x3nlbXONWq5UGpJz+JY10qLTB1l6r+N
rmyCJRO3dHTlW4WJkZu/WPTDLhNn6EzkJ2yUz7S4jT9xlVU39x2i3e8o22M2O4GNIqhYz5VMF0RL
tB1lYYKEq/+t/pTliRhLD/OG4EXPQvw8K86hiMX6yVillnO1OENv69nhm7kcscQDSowJrlj6IMa8
hbsyKos31ku5hzk+aJVAMlF4gLJ9tkJf9+9nJYjftg==
=nGWZ
-----END PGP SIGNATURE-----

--GXCKBqAPGfAuVlhONvxXkTU8QgRAYAYnS--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:06:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:06:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186234.334954 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3SR-0004iG-Ct; Tue, 14 Sep 2021 08:06:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186234.334954; Tue, 14 Sep 2021 08:06:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3SR-0004i9-8t; Tue, 14 Sep 2021 08:06:23 +0000
Received: by outflank-mailman (input) for mailman id 186234;
 Tue, 14 Sep 2021 08:06:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ3SQ-0004i1-9n
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:06:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5cb712a2-1c21-4082-a7a2-0f1dffc82983;
 Tue, 14 Sep 2021 08:06:21 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-fjOQO6OIMrm66bvaL-KDeA-2;
 Tue, 14 Sep 2021 10:06:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 08:06:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 08:06:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0020.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 14 Sep 2021 08:06:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cb712a2-1c21-4082-a7a2-0f1dffc82983
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631606780;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=p6cVXwCgI3ZhoJUMSTwlj63aGkUFMebjA8TT2q2fUqc=;
	b=CZsGhWsQCHzTZts4WIqtILSZV23s4aQBCZoEa2EJrF37DlkVANZ3pR3pG9bQArj4BTYb6N
	dA06wDqb+GUrFbMoa/e3BVU+fYFDbLRjYAHfdFJZJZmLGCZl+dwtkvC15efg3GnBDFbRDy
	/iC/GqIFvRgWQZIcqInxH3Ed2099zfU=
X-MC-Unique: fjOQO6OIMrm66bvaL-KDeA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eS25SORDsF2HhD4kf64rJbrV6H8CKkMug7zdnjd+InzFa5BBPTnUHa+IBPoQRmRnj9oeLhF23LLLJE46aMkCmbe51pBnx57CKe7hP+10De6amxDol78tSEXDJSOXdiNga2xT9ptE25iR4jpfFpPV696pTaeO4ntAHUvLeyk8xBjXu98UKTGzgudM58ThF+o+BCceKXbazSlI+L77lrb9deFlhF3rM1LzUyErpkNyrrcl9MIAnVSu1fhlqWKeoACBtWpyAYVxNw7H34L3N+Ws+0JnNUGQZTHHZTgua8mxNwRaJvp8IyE1wPTAcfNvCURAWKpix+coxUtH2s5pgQD30g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p6cVXwCgI3ZhoJUMSTwlj63aGkUFMebjA8TT2q2fUqc=;
 b=aRzOSkJbmm/j07GCGKYi9BU0HERWkgh3A2k+N7wpg1rBBS2Zr6/EBnZ5ZsK6aoa0SOae/qjLKpZFdDqHk9dmom5z7LGRczy9GYUPRJiDPCIiVLpSZtyqEaATX9msJuAMZebbi3TXP4moyzpmwZxMRohDbS5EIqG/Z+HCOQBsBekLi2XtDbJoRo5kzJpe3rkWifmK7Hn5vITxhUrQk7+sZaXifCMXVjxLcIuVrBKW9g4FhQWA7pLKLUZGXLTbUST/lOgy0Ak32lyxpoUVg/QHfjBDwbWSZiNSLtNRCtyYAw4xLauOhdKn+5xkTjVow4mIr+dc2oWibBk20kRkk+wWlw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Oleksandr <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
 <c006550f-4fe7-29c5-f370-dc67bfac2b95@suse.com>
 <f9e95fd9-d63e-675b-0236-653afee617b1@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <083817dc-05f3-bfd1-b9cf-1855ec9d21a8@suse.com>
Date: Tue, 14 Sep 2021 10:06:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f9e95fd9-d63e-675b-0236-653afee617b1@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0020.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d1493fdb-7398-47a1-4593-08d9775686eb
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7039AAF610968F79F9B19885B3DA9@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qA8gRaTtcgSV6QqOs6BFPRY8j818iituA7PJFLqE8myNq/6qOLx6JcjqH8rPqbA/GsoUA0IOBcwfvxivwEnwz4BzeODOTTdEl9FdtL79Q5FIsPjHtppUKteQakaOivD2MMK8dGtAtkCqRFm6ZL/FfgfCqUjCN3GKHC/6EeAyYVNqzUK/icjjprhwixAXY5Vsy4vTptMnR6rZCxbS2wLnpb/veiyjap3bR6dOgwTvCNdPA+TJFg8lxTrRKhRr8kwHM8vlvtcoIFuwSut2OQy0eLrNGvSUNWJETV1MB/F4jTYPpEc2poSmlDU+PdosCiulk143Q/1UoY7uucoj6f6wfIKe9GaYJccg5dS+dhVgYivvDC5xJG/qP9Q/1u/+2PKwbt/q4Y7fTXrvNT70FrJMKFaanwlc36BhTvyer3SN3n9VAu1GNWHh7qyuhNEbaxefePiV4ac3WeiW08tHOZWIk1yk4LMwMAE9GLExNjGzWzAc8II5m4MlvvWemaMitm+UlI66GGBRu5c15t5ZVwtbrP9noZ6M9aDQ445hjztkYNo4CZSO+gcflNnE7gP3P3IbXwhERMKJhiS1UHZz+6dVWnKfzGTpCCYVMz2wc4br3Y1FQzC2Lz7jQDwWuQjYji1+vbRp55vZjJefeA+uLyJxByVq6J35HdACpOi3OP547yDCHiJA7gcxNyLiFixahW3kphlrgzal+67lWfVCZMDdCTTviMVthL9Gf8pCg2ZmsTA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(346002)(396003)(136003)(376002)(39860400002)(5660300002)(186003)(2616005)(31686004)(36756003)(6486002)(86362001)(8676002)(4326008)(26005)(478600001)(16576012)(316002)(2906002)(66556008)(66476007)(6916009)(38100700002)(31696002)(956004)(83380400001)(8936002)(54906003)(7416002)(53546011)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eG1ETmtjQWhhZmtFOG1GUXNhYkpLOEwrME1Cd0hiVUlJUkh6TkZHU2hlaFhk?=
 =?utf-8?B?ZTBlNHBhSlhLVHVscXNGYTJjUUJqSm4ybSs1QytVSnpjUjlLOGJPUkRoMVZZ?=
 =?utf-8?B?dElaRXJNMzBKcFg3TkR2RFE0bGFWakl3UVpMY2FtMjJrRmVvUjhTbXRiamVn?=
 =?utf-8?B?dThGeXB5VWRoK3V6VG45K1o2Ym83TC9CdEVPMXRmdnJrTTlXYWlqMVNLYW91?=
 =?utf-8?B?MytzRmdJRXVtSjM2aG1sWXpaeXlybElHMTRudFhhenBieThIVUk0SmhuSVA4?=
 =?utf-8?B?L2srTFBxZVpBa0ZIMEVmZWNLTmhvaFg0aDJwWlNKSVNmWmlFOU4zVEs1S1Zt?=
 =?utf-8?B?dkkxVmtQZk16WDBUMkJ3eEplZ1IxRXc3RWtHVGFJYll6citmeVNKWHJRVzQw?=
 =?utf-8?B?M0x0UDBsNGhPLzNXWUQzUlBHV1RSL0lDaXZQeWFWMUlyTEZhZzkvSDZiY0U1?=
 =?utf-8?B?TlB2dFZMeTFraWFlTDV6endEMkVORTE5c2JXV3dFK0gxbDh5TVFpcDUyQzdX?=
 =?utf-8?B?WnlWa3JrcmdIR2ZSVEZldUpld0NES3E4V3Z3QnBHUHlGM0NQL29KWGhQN2ll?=
 =?utf-8?B?bWVuR1gvOVZVL3Q2aGx2Z0tVS2trekVuelpqdzBpREFScGJ3Q3FkU0JLK0dM?=
 =?utf-8?B?MGJQc01pSHRNWUNSRlE0VVpHWXFBS0JKZUxHTEtkcWdMU295eGYxZzJHZ2RH?=
 =?utf-8?B?eGNHL0k1RmVMQzZuQXVUMUpHRmd1S2ZydHdyM2N5WVd3b2duYndqd3JSZ05n?=
 =?utf-8?B?VzJOUlRLbFpHYXFqYUtLZm15cnZPblF6cWtwZVE1SHJuUWFNMkpBVTdSbVBD?=
 =?utf-8?B?MjI4YUpxc1kyRUZ5TCt0aGVUMG91NVdaNFhKdUVUOFZrSEZXdzQrZXpjckth?=
 =?utf-8?B?OExOcHR6UHZHNWEyeGlEUmI1QWlaUVZOWkRtNFlLcUdyZTRIOUlvalBaai9I?=
 =?utf-8?B?K3dleWZOYzdXUmFPNE1KQ3IvK2NvMjZnTmpSbGcvSXd6UEZWOWx6SHpka3Vk?=
 =?utf-8?B?OGJvc3I0eXhTMFJJKzI4V1Q2cGh2emVpZ2NSTjFQNXk0SENrdUVYbGtIYjIw?=
 =?utf-8?B?emk5U2Zld0dFNU16emF1Qi9USC91M1drK1M3U1ozUjVsYjA2MlpOTjd2dVRl?=
 =?utf-8?B?MEZuZSsxMzNSSkRReE5FK0Z4TG9VMmFPTHh3RlZrWXhFYnBSWktUTU9wZDNR?=
 =?utf-8?B?dTNyQ2RDQjNKRUxGRGhXY29NYjZpS2V2d1hFSjg1U0FjbG80WE1NSkQwcTFW?=
 =?utf-8?B?VXQ1VWJ0OFZBTWIyRXRwQmxCM2dlSTI0Y0x4V25qS3cwNnY4RmwxdCtkTGVr?=
 =?utf-8?B?cFdlQUcxSUlqOUZON25DOHZHZjJuWDNCWHhCTFNsL3ptV3RtQ25VbFlUc3ZP?=
 =?utf-8?B?Z25mZVV4Z0l5MWp3V3hVajFLUmsxWGVHeDVXVTJZeVJYVE9CaVErRzdkVXBB?=
 =?utf-8?B?Z0FWSHJpZHZqZ0x5dFJtWWJNUHIwOEtFMkZENTZ1TUMzRURqQlU5R0h5QWU5?=
 =?utf-8?B?RXNmd2FoYkh2MElTbFZkMkRXVVdURmZiU0cycVVDMjF4dkIvVlI3dDE4bWor?=
 =?utf-8?B?ZWwrSXJsTnJ2M1BYd3huaWp6Q0tnc2t6T2lxb3FxUEkzY3J1MzgzZTlLdnVQ?=
 =?utf-8?B?QSsrMjU1cEdxcUVENmU1Qk42ZVppSVVxdlZPdUk1Mm1veWlVa1VEQkVKMlJG?=
 =?utf-8?B?bW5QSHdpczBSU3pyTkwraUo0VDlLWGl2UXNwQTVKWHJtdnFneHhidFE2N0lO?=
 =?utf-8?Q?dPytJVjMFAm0wLj99jcQjPRS9A1ip6M39v5Cf6o?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d1493fdb-7398-47a1-4593-08d9775686eb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 08:06:16.5943
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cLyVRsTB5KqeBchpVbkeQXhaHizl0AtVZagEwNcp7lPet9nVWqa/I8uS1QkW8BnVwhPXAZa7IxzgS/aO+eBk/A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 13.09.2021 21:57, Oleksandr wrote:
> 
> On 13.09.21 09:17, Jan Beulich wrote:
> 
> Hi Jan
> 
>> On 10.09.2021 09:52, Jan Beulich wrote:
>>> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>>>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>>>        */
>>>>       if ( p2m_is_foreign(pte.p2m.type) )
>>>>       {
>>>> -        mfn_t mfn = lpae_get_mfn(pte);
>>>> -
>>>>           ASSERT(mfn_valid(mfn));
>>>>           put_page(mfn_to_page(mfn));
>>>>       }
>>>> +
>>>> +#ifdef CONFIG_GRANT_TABLE
>>>> +    /*
>>>> +     * Check whether we deal with grant table page. As the grant table page
>>>> +     * is xen_heap page and its entry has known p2m type, detect it and mark
>>>> +     * the stored GFN as invalid.
>>>> +     */
>>>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>>>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>>>> +#endif
>>> I take it the write done is benign for other Xen heap pages? I suppose
>>> this would want making very explicit, as such an assumption is easy to
>>> go stale by entirely unrelated changes.
>>>
>>> I also wonder whether you really mean to include p2m_ram_ro pages here
>>> as well.
>> Actually I've meanwhile realized I should put my question here quite
>> differently: Aren't you effectively introducing an M2P here for Arm,
>> except that you artificially limit it to the Xen heap pages needed for
>> the grant table?
> 
> Difficult to say, it might indeed look a bit close to M2P, so the answer 
> to your question is more yes than no. But, I didn't have a plan to 
> introduce M2P on Arm. It turned out that stashing GFN into page_info (as 
> was suggested) avoided huge lookups, so we have got MFN->GFN in the end. 
> The purpose of this patch was just to fix a potential issue with 
> remapping grant-table frame on architecture without the M2P (Arm).

I understand this is the immediate goal. I wonder though if it's helpful
to make this a special case when it can (I think) easily be made general.
But of course there may be (perhaps Arm-specific) aspects which I'm
simply unaware of.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:14:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:14:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186245.334964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3Zn-0006Hi-5Z; Tue, 14 Sep 2021 08:13:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186245.334964; Tue, 14 Sep 2021 08:13:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3Zn-0006Hb-2U; Tue, 14 Sep 2021 08:13:59 +0000
Received: by outflank-mailman (input) for mailman id 186245;
 Tue, 14 Sep 2021 08:13:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r2So=OE=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQ3Zl-0006HV-7J
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:13:57 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.67]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 06a01939-8216-4ff2-9ec1-584f953997b8;
 Tue, 14 Sep 2021 08:13:54 +0000 (UTC)
Received: from DB8PR03CA0005.eurprd03.prod.outlook.com (2603:10a6:10:be::18)
 by AM9PR08MB6867.eurprd08.prod.outlook.com (2603:10a6:20b:305::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 08:13:51 +0000
Received: from DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:be:cafe::b3) by DB8PR03CA0005.outlook.office365.com
 (2603:10a6:10:be::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 14 Sep 2021 08:13:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT044.mail.protection.outlook.com (10.152.21.167) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 08:13:50 +0000
Received: ("Tessian outbound e27daf245730:v103");
 Tue, 14 Sep 2021 08:13:50 +0000
Received: from 10064435747b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 332A3621-72CE-4AB1-915D-21EA221845BE.1; 
 Tue, 14 Sep 2021 08:13:45 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 10064435747b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 14 Sep 2021 08:13:45 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4657.eurprd08.prod.outlook.com (2603:10a6:20b:cf::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Tue, 14 Sep
 2021 08:13:43 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Tue, 14 Sep 2021
 08:13:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06a01939-8216-4ff2-9ec1-584f953997b8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=A1eAOtC3pk2epVyElQaImZ6r7HoGTFqWbmPybf/bh2M=;
 b=Fs7gBpms9ovJ/v2CcKLVo4RvZRYXbN0eQsegsvg72LB61HwRpyZcNknL7/cex4ZJubDGEqJ0mT+XjpBpIEzjDthM9KU8hjFZg1g1sRuM0jFoE9GxOWNCXqYPfQCNsQB5DLVqgqpYxe0d4zV8c4BDKZQuDLXgHxTDMu30lPilJbI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 527c5467429b26f3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rvt6BTXOY+B8/YOTsjq2x34GU7l0FMw/uH4BJEAyX2majIedFZZ2XuGk32p9VylnblZKL0NoM4MAvQ4M3NpndMbe5NvPu/yQsR/34ZXhtC1lOAVaVKxFK+hrHe7QsI3ht4BXjQuzNRbGqXVu/B/MhQIxlt5//bpDuoEB741NLfa1vO1dPkSeM9fhEwWWBa0YPv1B8Zx1xBqApRv9lHxUi2Zc9B+TNxFTpTOGAeC6qYRbfwdq2ql17gzJV/IkBJ8c+JeNKxNT5E0CuZdhY2CPRY6EaNiK+JgN5Ptqz3r/1ZHFVIV58dwgnka3BLMr70098NbrczwQjC3E1MEr1nlM4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=A1eAOtC3pk2epVyElQaImZ6r7HoGTFqWbmPybf/bh2M=;
 b=hRaMCsgl1CONssp2u+y2Swiz0G6B2sGbT1WDaL54VNS4sock3LJJgsHjsAyS4XjOw/OhTPZGq+F72M8ae6GJX2zh7bycWimGS1DwyB65Ep4KjpHJQmstnHz5cqT+xBx0S4UyEYNTd0LZ3tHVGFPO6jn0+LDIi/+2ZjBkkeGp2Fy9ss1vqdMCqKs4CVSfZV52mxNbZulLQYx76uH3DnkzXaQqlXim4DoD03K+b3IdPR4uK7iiwFEWk3ZG9zuX1yiTzS9CQAS6EISFNWLt4EcZAcJmlYwx1YVzfKIWuREhnJcqCY1eu8RjaO1l9g8b9YzWznyf54PWp6/lXx4hcAoMNg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=A1eAOtC3pk2epVyElQaImZ6r7HoGTFqWbmPybf/bh2M=;
 b=Fs7gBpms9ovJ/v2CcKLVo4RvZRYXbN0eQsegsvg72LB61HwRpyZcNknL7/cex4ZJubDGEqJ0mT+XjpBpIEzjDthM9KU8hjFZg1g1sRuM0jFoE9GxOWNCXqYPfQCNsQB5DLVqgqpYxe0d4zV8c4BDKZQuDLXgHxTDMu30lPilJbI=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Topic: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Index: AQHXlPK6hMBa2cV9wU+RB+r01dTauqubssEAgAekNIA=
Date: Tue, 14 Sep 2021 08:13:42 +0000
Message-ID: <9FFB1675-1D02-49E2-A340-AA33A98C50B8@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
 <d7da2d5c-fdc0-d72c-789c-c6ba75412f3f@xen.org>
In-Reply-To: <d7da2d5c-fdc0-d72c-789c-c6ba75412f3f@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 021fbcd4-6e11-4d1a-e365-08d9775795d1
x-ms-traffictypediagnostic: AM6PR08MB4657:|AM9PR08MB6867:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB6867CB56B980DA4A803E93E5FCDA9@AM9PR08MB6867.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 w50sxf+uN7q4RKV6X2vuVTq55iVHyiaPEsPBwOZla7O22pT9iBz8CXcteDxMYQYqkmKmronRpFeLYAucCqV3Zn1aUvvNDz4+LX0RFeHqV1RWkEri+rmoP/XIw8Ze2OlKc8G+agOgZgAoXHvoYbyvT/n+aGlMmxuM/5r480dfB28atkMLxVN4yjmHvhmjv8c9ZWGZT6Nnhs1mf5rjn6YboEADT63BACl2i6DFmHpSFUnPPgf+VPDQ5Hsnyb0PwPFSanjrbtO81GMU6dacrH/CO3mS5r04f7L+bP6LDoYj683b5r1Zxd5+SRrnTworL+ciNJgVSGaFEi3LBD4K61089LlHYzHlAo4KfM3FRPbVYh3bjQHA84CSJmKdG/jNhehR+dd9646haRxxvJdnyk0Hfu4uM3BQwzsNdTG+E3vAd6bycsZKQjlVfqxWhPOhd+NiYn7q1hm+4zfKp0EVTocjoSkIJum89T0dWyjOgWWvst/nmI8vYD2GkD/+8046AWKisrrBbwwNFriQs8cYlOuOOtgndP9hGsVOIr4wh7PHour4c6ELpzWA0b078txiKqOtC63yZ9R0/lK2tpoNtlZGOj7aI3eBnccrzn4MkGhBtySjdHHYCkSc54Jat1o8yh7LqKLiTUrKaXT29c2MzbEsZx8VazcTADpXLWufqJaKTS/nT5lHAwwK7W8z07JMai5Qe37qOIir9TBEN1WaIWWeYgdKn4ln0pZduPq0C4yss7SrZLgmMEmKQ6gF1Mg52nP6I4cOfc0I0f51NTnS34jdeuEUkK8EcEyzO4kvTRIwd3VVRxIdmfhDjLWI/dTTme4O7kHGt2qWowDV2CWXkBTeNA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(346002)(366004)(136003)(36756003)(33656002)(316002)(86362001)(122000001)(91956017)(38100700002)(64756008)(66556008)(66476007)(4326008)(6916009)(76116006)(66946007)(83380400001)(8676002)(66446008)(38070700005)(6512007)(71200400001)(5660300002)(2906002)(478600001)(6486002)(54906003)(26005)(2616005)(6506007)(53546011)(8936002)(186003)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F2A846229AB536429D87080EB32B528E@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4657
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	4f2072a7-9c84-4db0-a98b-08d977579102
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	muhwbhBxZ7n+gnlG0sWyV4WAVN4ujiklXIhN5/qTxXFqxlKYbPG0kLtoYpE2nl4GFnZcCafrKQu5fIbrjohOk9/mxkboe5Eqj1sRQKvfxGckwoGTFSMcxIr/8ros+Frj3nBy7j83u5s8NBeOEXXH3JRTMxFqv6bLi7txeHngpKIu30RuPfNH31w240l5x4iuzFRqFsdrFIMqol4b/f9fPeUWS26LC0NFegWxkiZuE1kCHX6nOFBrQEDIVUBpv+HNkacus20IrV0AIv2tOJLyAf9wsRBOEvQ/egVqmKB/ec6hP1B/pfo8ZDkkmk3zaPvyghOLwC91MvTvp6anVwhbKga78qoB3N5th1w/nRBhzHQfKEUNiJif0IhRZ+Kamf2uYHbme+PHAhkFR7Xr3x+DSOmhCstpmjf3XJJyu2/V8VgCGsZSJJuMYSSLgzQwaMgscgPUdx02kGvXkYoN2xTqiBwrUxBGcOeCupbZLh5EnlKFLrNRyS9hEZk+0V2Hoj5XK3Ipnj1yULQq0G7BeRYeyDzdKvqni14TjvIz5N4pEphg9pSn7nuF1wOpWMRWEqBWcGNTLMdb+0UOEWznsPtvChk/Cm2fsTIQp2yYujPfdzXG4Z+8oOTfQiO1NV/s7uvqV2JUsRSCobJiOXJpjcpcZEeiaBwOVf0c6bv6PWOXq2I+bQ/wjFEWWn/tt9wFXq7ksTm/gleK+lJLic+AUqcS7wv/mK/AroIx7A3rE3Hte+6hIFyJhz1HqiHs11OYkVJch9zaKboAo5nqKpgxKSDYkyREsMo9oJV3Kat7LwLqrrrNn8rz1cH1GWu1xCm+nrco
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(346002)(376002)(46966006)(36840700001)(356005)(36756003)(6506007)(82740400003)(5660300002)(81166007)(2906002)(70586007)(2616005)(107886003)(70206006)(6486002)(86362001)(4326008)(47076005)(54906003)(8936002)(316002)(83380400001)(36860700001)(33656002)(26005)(82310400003)(336012)(478600001)(186003)(8676002)(53546011)(6862004)(6512007)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 08:13:50.8966
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 021fbcd4-6e11-4d1a-e365-08d9775795d1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6867

Hi Julien,

> On 9 Sep 2021, at 12:32 pm, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Rahul,
>=20
> On 19/08/2021 13:02, Rahul Singh wrote:
>> Add support for PCI ecam operations to access the PCI
>> configuration space.
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>>  xen/arch/arm/pci/Makefile           |  1 +
>>  xen/arch/arm/pci/ecam.c             | 63 +++++++++++++++++++++++++++++
>>  xen/arch/arm/pci/pci-access.c       | 53 ++++++++++++++++++++++++
>>  xen/arch/arm/pci/pci-host-common.c  | 13 +++++-
>>  xen/arch/arm/pci/pci-host-generic.c |  8 +++-
>>  xen/include/asm-arm/pci.h           | 32 +++++++++++++++
>>  6 files changed, 167 insertions(+), 3 deletions(-)
>>  create mode 100644 xen/arch/arm/pci/ecam.c
>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
>> index f3d97f859e..6f32fbbe67 100644
>> --- a/xen/arch/arm/pci/Makefile
>> +++ b/xen/arch/arm/pci/Makefile
>> @@ -2,3 +2,4 @@ obj-y +=3D pci.o
>>  obj-y +=3D pci-access.o
>>  obj-y +=3D pci-host-generic.o
>>  obj-y +=3D pci-host-common.o
>> +obj-y +=3D ecam.o
>> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
>> new file mode 100644
>> index 0000000000..91c691b41f
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/ecam.c
>> @@ -0,0 +1,63 @@
>> +/*
>> + * Copyright (C) 2021 Arm Ltd.
>> + *
>> + * Based on Linux drivers/pci/ecam.c
>> + * Copyright 2016 Broadcom
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>=
.
>> + */
>> +
>> +#include <xen/pci.h>
>> +#include <xen/sched.h>
>> +
>> +/*
>> + * Function to implement the pci_ops ->map_bus method.
>> + */
>> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>> +                                      uint32_t sbdf, uint32_t where)
>> +{
>> +    const struct pci_config_window *cfg =3D bridge->sysdata;
>> +    unsigned int devfn_shift =3D cfg->ops->bus_shift - 8;
>> +    void __iomem *base;
>> +
>> +    pci_sbdf_t sbdf_t =3D (pci_sbdf_t) sbdf ;
>=20
> AFAICT, pci_sbdf is an union between a 32-bit and a structure. So please =
don't use the cast and use the 32-bit field to assign the value.
>=20
> Also, there is an extra space before ';'.

Ok. As per your below comment I will remove the sbdf_t completely.
>=20
>=20
>> +    unsigned int busn =3D sbdf_t.bus;
>> +
>> +    if ( busn < cfg->busn_start || busn > cfg->busn_end )
>> +        return NULL;
>> +
>> +    busn -=3D cfg->busn_start;
>> +    base =3D cfg->win + (busn << cfg->ops->bus_shift);
>> +
>> +    return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + w=
here;
>=20
> How about using PCI_DEVFN2(sbdf)? This would allow you to drop the use of=
 sbdf_t completely (sbdf_t.bus could be replaced with PCI_BUS(sdbf)).
Make sense. I will modify as per your request.
>=20
>> +}
>> +
>> +/* ECAM ops */
>> +const struct pci_ecam_ops pci_generic_ecam_ops =3D {
>> +    .bus_shift  =3D 20,
>> +    .pci_ops    =3D {
>> +        .map_bus                =3D pci_ecam_map_bus,
>> +        .read                   =3D pci_generic_config_read,
>> +        .write                  =3D pci_generic_config_write,
>> +    }
>> +};
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * tab-width: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access=
.c
>> index b938047c03..f39f6a3a38 100644
>> --- a/xen/arch/arm/pci/pci-access.c
>> +++ b/xen/arch/arm/pci/pci-access.c
>> @@ -15,6 +15,59 @@
>>   */
>>    #include <xen/pci.h>
>> +#include <asm/io.h>
>> +
>> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sb=
df,
>> +                            uint32_t reg, uint32_t len, uint32_t *value=
)
>> +{
>> +    void __iomem *addr =3D bridge->ops->map_bus(bridge, sbdf, reg);
>=20
> Please add a newline here.

Ack.=20
>=20
>> +    if (!addr) {
>=20
> You seem to use a mix of Xen and Linux coding style. If the file contains=
 mostly Xen code, then we should use the former.

Ok. I will fix the coding style in next version.
=20
Regards,
Rahul=


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:30:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:30:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186258.334976 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3pl-0000Pr-Ow; Tue, 14 Sep 2021 08:30:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186258.334976; Tue, 14 Sep 2021 08:30:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3pl-0000Pk-Lc; Tue, 14 Sep 2021 08:30:29 +0000
Received: by outflank-mailman (input) for mailman id 186258;
 Tue, 14 Sep 2021 08:30:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ3pj-0000Pe-N5
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:30:27 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 02985ea0-1536-11ec-b404-12813bfff9fa;
 Tue, 14 Sep 2021 08:30:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 79B681FDC4;
 Tue, 14 Sep 2021 08:30:25 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2078A13E4F;
 Tue, 14 Sep 2021 08:30:25 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 4e8UBKFdQGEQJAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 08:30:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02985ea0-1536-11ec-b404-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631608225; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=g+KNvYYwaq+QEYe4lscLRpP6P8hWsK6xLT06mLWFFy0=;
	b=TG+vwFzGTUtTvrX/L1U2bzHlTuslgNe9ZqZbhgaGy+hV3Hd8Q8SB5mK0yeHkSv2qZAghV3
	JAT13QXtAn/4EolJ6hTjBjBlJ5LXsfTsyc107dh/quA6C7PH7o+09HZB5Z5dynM1JAhSgJ
	3O1MBNlhAjMDnN7cHhq+XenBqk7LS3w=
Subject: Re: [PATCH 00/12] swiotlb-xen: fixes and adjustments
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <fea8a42c-2034-3804-8773-b12dbe1bd34d@suse.com>
Date: Tue, 14 Sep 2021 10:30:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="mul7AtqHhilVdnxoSPn6gYltOKuN2CqxE"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--mul7AtqHhilVdnxoSPn6gYltOKuN2CqxE
Content-Type: multipart/mixed; boundary="bp05TFxV3hx78iI4yWYuxQGYo8tmA9Gip";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Message-ID: <fea8a42c-2034-3804-8773-b12dbe1bd34d@suse.com>
Subject: Re: [PATCH 00/12] swiotlb-xen: fixes and adjustments
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
In-Reply-To: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>

--bp05TFxV3hx78iI4yWYuxQGYo8tmA9Gip
Content-Type: multipart/mixed;
 boundary="------------9BE04ABB14BCE0CC8A5548AA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------9BE04ABB14BCE0CC8A5548AA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 14:03, Jan Beulich wrote:
> The primary intention really was the last patch, there you go ...
>=20
> 01: swiotlb-xen: avoid double free
> 02: swiotlb-xen: fix late init retry
> 03: swiotlb-xen: maintain slab count properly
> 04: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
> 05: swiotlb-xen: suppress certain init retries
> 06: swiotlb-xen: limit init retries
> 07: swiotlb-xen: drop leftover __ref
> 08: swiotlb-xen: arrange to have buffer info logged
> 09: swiotlb-xen: drop DEFAULT_NSLABS
> 10: xen-pcifront: this module is PV-only
> 11: xen/pci-swiotlb: reduce visibility of symbols
> 12: swiotlb-xen: this is PV-only on x86

Pushed patches 1-3 and 5-9 to xen/tip.git for-linus-5.15


Juergen

--------------9BE04ABB14BCE0CC8A5548AA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------9BE04ABB14BCE0CC8A5548AA--

--bp05TFxV3hx78iI4yWYuxQGYo8tmA9Gip--

--mul7AtqHhilVdnxoSPn6gYltOKuN2CqxE
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFAXaAFAwAAAAAACgkQsN6d1ii/Ey9J
Rgf8CZuLijUcQXUJRcWR7j2iGubwF/Qdjn37T0tXpiVwgPelW7hKm+uMgq7sdDhv6OQtuY8bVmX6
OlGFxfmfW1XnzzDAurby2GEbkVLTZskovlUvLRdKlO/X+pusyc5fp/cRhzgjTYgRQFB8SocbhVpJ
v+eRIMYi3mgXCjjR5W3zq/jqHbCPiEiW/n5VlKq4IEZnn3iNzKg3MBiwqbfG0/hgV5XxuIQazuR8
jH/w60PMvpGvvDyWqkn1GfcmST69RKyvAqcX0FYkzKf02kKc49myWBueS+Wh5rmupDe3iPclBfk2
347ZZI8oxRsWFjMdozkY8iDqX31rRLKlK/MuP7zoPQ==
=oQYH
-----END PGP SIGNATURE-----

--mul7AtqHhilVdnxoSPn6gYltOKuN2CqxE--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:31:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:31:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186268.334987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3r6-00011G-4C; Tue, 14 Sep 2021 08:31:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186268.334987; Tue, 14 Sep 2021 08:31:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3r6-000119-0s; Tue, 14 Sep 2021 08:31:52 +0000
Received: by outflank-mailman (input) for mailman id 186268;
 Tue, 14 Sep 2021 08:31:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ3r5-000112-I4
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:31:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b63575e9-a921-4937-b210-41388abd894b;
 Tue, 14 Sep 2021 08:31:50 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-ZBfO--oEPHC6hRiAsiZb9A-1; Tue, 14 Sep 2021 10:31:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 08:31:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 08:31:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0142.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 08:31:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b63575e9-a921-4937-b210-41388abd894b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631608309;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=M09F3jpJ2XrXXO2NKOicKlAkmmQYFljFuPYwTdMbazE=;
	b=ZkmCsLm7JwECIiU/4Y95l+fvBAPy9v043AwxaBqNJsyCKGFkBSo1JJw08JG+GVXnUU99Zr
	NHdU5x3CROxKEUNoAd/Ubh+B531GRM2uh0FVZdtFeY4U+z5suWj10mThKqry+ongfKSiQF
	BC/91qLNL5ExyA9KC9NfqxT1tetKIhw=
X-MC-Unique: ZBfO--oEPHC6hRiAsiZb9A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=a9Tg3SO0ALScl1ojgXIVd4nbp8CY8B6Pj5owYYLMg/lsUK6YUdpn0q+cU5S6/eoiuLF0gpUVHa/l6M1ogW/MIdvy36tKTghfM3e6M8ar+ATF6//1G1DGj74Pc1cE8MeFfVW4Ok2GkTgcNIUuDqXVm5nNbImknaLNdGs9xdu9CSmwh4voGid1gzBkbdcP2aFzHYaPvgcueJtfpMXyHjjD7GMtW/h7bjxf1Kz5TvIdOss6Uap5MwjcPzGJMvQKwS26PJu1uWRXZz2zJIHd8Ciauobygnyb03rcT3uPNvwKNLb9J490f/8q5Pns89Eo6Xa/7732kkkiGiAFMPGJFaqkWw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=c1PgFZ/1KSPVZt+CxDUU17pwkTnnxg+QY9W8NaCFS90=;
 b=DYiRDSmCvbae79ugt8nzpWmvy9SoM/urrGSWbm2BX7bEf3ciiEMSSDnHHJ5Nm2bg+Bzdys2iToOkDjgHNZHTqNoEmgbPp61KRjuL/rBuMnODb5xs0NZbKKntyPnCeJPZSzS2wyHvQbm9SjpIGKPWhFjiOOpcP5+MgxWvXfCqDRY1pOq/jEg2plsRDkRX5+W0p/rk2UoqjsoRIPNk29HTkBS/pQAu3+g/+zxGOzF9WqH2mkUFa8osLSWiXyzp7m+soxOcHZtNWsY4EctSB2fFNN9CxHqX2/W29t3TCKiFiE4ATLgntVInIn39v2JXlQb9oiKBRqZY8/9c+4trLMxCOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Oleksandr <olekstysh@gmail.com>
CC: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
 <c286febc-b7b4-5aa4-5331-f0c2f6a17000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a0ad861c-d8a8-82f0-023b-26c3fd4d50cb@suse.com>
Date: Tue, 14 Sep 2021 10:31:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c286febc-b7b4-5aa4-5331-f0c2f6a17000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0142.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 04ef6c99-1fae-4fcf-1eb9-08d9775a163a
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2445070EAF0E55A11FF60D35B3DA9@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YgV+wAsvqKAAIPeAV3FMoYas33lOvrWrdOlEicB1pDFShzguXdpXbE9VinE6C5GgbQrG5BGRVwV7pqpFzCcrNu+yRFSWiZqG0DBsDsNBgghqzDTs9Qdp0nPLv870OziDgvQ+CCiEh6YACS9V+r06YhALWFMEi93cowKu4uMjQ8Re8/5KypTD41Sq4hiWy85OIwxdmjXEOkafCTmzPqS5+qeBDi6k2xkMFvuuE7dUD/maXDRhuwQNEi91S9atLAbXRJfKwmo78UhdnhMv1InH+cEq54pEHEYZfyUZjMW6gmbk5NyjjQQGvdbyjmz1IrpJPf1JfKQpS1v2l0wOLThmx/7mH+dQMq60KUzBDg/yVZGwrk8AQ79dw3mJWKA++AYiAbBmYvqhrIi3cGLnlyeFuPNsVtYhW3GzLK53/Yf2gsVl/BosnPKHWY3YD0wp0qJIjjsJuPHhOzDBtgdgo2mTTN4pXiLYUAAKnJpwKl6xZA7a9wgPoVEymYvZHMODMxgBbN9PoyOS67Q14KoMqyWxRVC54UHADsQJvbN3mMNGdqzTo0XIDt870uYS7H0GyRW7DM85drooGpxbw/LzkLaGEWHnXg+Kgr4KrjtncwKpkBTHnj6Pom6P3WiMBDYRckpBvGJzNQ4fNPndfS2KqdBARqeCyBvlB10ON1Xr5Itpd4lN6GZCaSn3li3F6EQ+gQNCty0NRk1eEbZjBiqQcLTvm4cRt4d4srt/d0aT0MMdM0PJtBRN4EW52IJcRfPbXSLNlGzztnVq8Ev0CdenhJzyAg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(396003)(136003)(346002)(36756003)(31686004)(5660300002)(66946007)(2906002)(66556008)(38100700002)(7416002)(2616005)(956004)(66476007)(6486002)(86362001)(4326008)(16576012)(54906003)(31696002)(8936002)(316002)(83380400001)(6916009)(26005)(478600001)(186003)(8676002)(53546011)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?LjT3xM41PbJWTGxVDlm9cCaS3tsngm9SM5OtnFsVdye3+Rr925AXJQ937XYv?=
 =?us-ascii?Q?y6xUrllr/fJakx/dixJAf5nv1ThqVF1q+H/9Xavlv2iJkuZFegS7R/40xr5s?=
 =?us-ascii?Q?FM0WnbI5Ohfqp+7TYRWJW6iUVDP0q/KA+h5QViCYAKRA0fhENvyucbW6IQvu?=
 =?us-ascii?Q?oUXUxYkGNWGZQbtTk6ufserEAJDFKwkKa4VlNl9S4hPVi4LkS5maxl5PFUoY?=
 =?us-ascii?Q?3o1BT/tU147B0Ya5DIU8w+p3o74aR3zBslmjN0CaNSEgKSUEAEM9iobtkF5p?=
 =?us-ascii?Q?3z5JtmVi5CBM3w+cOXaYEp2010p6wmsDu7GFqedcRX+hOd0ZQ8pZYtOXOOjv?=
 =?us-ascii?Q?r/uax22vbWNgdJ84XmwIn8W/ol5Xfpa4OjOY/GT05W0GrdZDW5VOsGH3fDud?=
 =?us-ascii?Q?U8LzMcpdlU8MmKdw8M9kOnSp3A1N8eWVw7FIQVY0EwAQa+bYMocLCCzG2S7x?=
 =?us-ascii?Q?4Nforq0dVY0DXD71DLGSZWK34OlYpzsZRTQ+O/ETEu7uEFtKN02LTTaVKeQC?=
 =?us-ascii?Q?t1RPQ0+Gcjra+WaDcJudCYep/ao9RxbSh2YTtUgKK5bkrrmy25iqy7lOZaiN?=
 =?us-ascii?Q?VoTnhAieZLyF5ygPmtyCEGrRNsZf3jPCpcVpaWFCF4G1jQjM5npybL4ywK6W?=
 =?us-ascii?Q?cmaT0cFc2+9T87JN8MsSULQxJ1E1MtELwWKNq/nkXEydd9iQJX8OnnY5/Il5?=
 =?us-ascii?Q?heh+sALKKec9F+cWfBz9Chib7f2DuEdyxKxNarQ7/7GdfelEddoPPXVQJIyr?=
 =?us-ascii?Q?wBa/sEyldtMy1UV/9x75Vw1YHagZxbjxlRPcHTmKDVWFUKWZ7dOrw9XoIBpj?=
 =?us-ascii?Q?G/ERTSfnyaRCmtIk9h8VHumO/fUxrTUuXQ88FoIMA4lWs4V6QJ4aKAYahRZ9?=
 =?us-ascii?Q?54L7GI8o3R53jLsupBT8nv/n/EwxKitkRHorYlnBph7VUkX8LcTGHF/XyhJM?=
 =?us-ascii?Q?kd3oia5WvLC8OdXDzkQpBWH5WdFaDPt+9eBYVqVbSE+CdOSFUi0yjK9CoXYs?=
 =?us-ascii?Q?bD8JprTb/NjhcbPMCmAJD4RYfrNe3EPqR7NSoypWytErNDjP5v6LbIQQYPvT?=
 =?us-ascii?Q?uQGR7wU7Vjwws++AYeS9syTuTKJCuxhX3is+MP9CTXNGACDdDhGOaiA/O9Sy?=
 =?us-ascii?Q?NZV7bDGtTBEpD5daadtGsiOsEyr94uO66E5n6dVskV5p5xteEyPXs7qbMWUp?=
 =?us-ascii?Q?9Ng9Hl76hdpC+sKUCB/ZY8pQ0o9WGBH681pi0BSTRLY1Ytr4n2NwNfBevnMk?=
 =?us-ascii?Q?YgrmpIPwZusYpFfoMD195TLgywnhq1doQNEc7jLsFwyxQsUpPD7ZxYx3RZDM?=
 =?us-ascii?Q?3UjhlihrU5eS/jRJ0CW8Qj/6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 04ef6c99-1fae-4fcf-1eb9-08d9775a163a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 08:31:45.4454
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +dHAuFlbXdsm7K43r4sJFSUWuLZovGglJxpy5R8Ujzyep0+V/DyGA6wS5MsUzLGFHxjylGOvwuXmlPhVAm7bQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

On 13.09.2021 19:09, Oleksandr wrote:
> On 10.09.21 10:52, Jan Beulich wrote:
>> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>>        */
>>>       if ( p2m_is_foreign(pte.p2m.type) )
>>>       {
>>> -        mfn_t mfn =3D lpae_get_mfn(pte);
>>> -
>>>           ASSERT(mfn_valid(mfn));
>>>           put_page(mfn_to_page(mfn));
>>>       }
>>> +
>>> +#ifdef CONFIG_GRANT_TABLE
>>> +    /*
>>> +     * Check whether we deal with grant table page. As the grant table=
 page
>>> +     * is xen_heap page and its entry has known p2m type, detect it an=
d mark
>>> +     * the stored GFN as invalid.
>>> +     */
>>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>>> +#endif
>> I take it the write done is benign for other Xen heap pages? I suppose
>> this would want making very explicit, as such an assumption is easy to
>> go stale by entirely unrelated changes.
>=20
> Yes, I don't see what bad could happen if we would clear this field for=20
> non grant table pages. Except grant table pages I could detect only one=20
> page passed this check here which is, I assume, shared_info page. All=20
> pages have this field initialized with INVALID_GFN. But *only* grant=20
> table pages have this field in use. So, I think, no one will suffer. I=20
> will add a comment. Or you meant something more than just a comment?

The answer here goes hand in hand with the pending question of whether
you wouldn't better generally introduce recording of the GFN (and hence
effectively an M2P): The less special casing here, the better imo. The
more special casing here, the more justification / explanation is imo
needed in the comment.

>>> --- a/xen/include/asm-arm/mm.h
>>> +++ b/xen/include/asm-arm/mm.h
>>> @@ -39,7 +39,15 @@ struct page_info
>>>           /* Page is in use: ((count_info & PGC_count_mask) !=3D 0). */
>>>           struct {
>>>               /* Type reference count and various PGT_xxx flags and fie=
lds. */
>>> -            unsigned long type_info;
>>> +            unsigned long type_info:4;
>>> +
>>> +            /*
>>> +             * Stored GFN if page is used for grant table frame
>>> +             * (bits 59(27)-0).
>> Are these bit numbers correct? I thought Arm, like x86, counted bits
>> from low to high (unlike PPC, for example)?
>=20
> I think, these are correct.=C2=A0 Yes, Little Endian is used.

Well, the low 4 bits are used by type_info here. Therefore I'm having
trouble seeing what the 0 refers to.

>>> @@ -94,12 +102,12 @@ struct page_info
>>>   #define PG_shift(idx)   (BITS_PER_LONG - (idx))
>>>   #define PG_mask(x, idx) (x ## UL << PG_shift(idx))
>>>  =20
>>> -#define PGT_none          PG_mask(0, 1)  /* no special uses of this pa=
ge   */
>>> -#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?    =
     */
>>> -#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.            =
     */
>>> +#define PGT_none          (0UL << 3)  /* no special uses of this page =
  */
>>> +#define PGT_writable_page (1UL << 3)  /* has writable mappings?       =
  */
>>> +#define PGT_type_mask     (1UL << 3)
>>>  =20
>>>    /* Count of uses of this frame as its current type. */
>>> -#define PGT_count_width   PG_shift(2)
>>> +#define PGT_count_width   1
>>>   #define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>> Leaving just a single bit seems pretty risky to me, and I can't see
>> why you do so. You need 52 bits on Arm64. Which means you have 12
>> bits left. Why not use them in full? Even on Arm32 you have 3 bits
>> available for the count afaict.
>=20
> Only 1 bit in the type_info field is really used on Arm, but anyway ...
>=20
>=20
>>
>> Also there's a disconnect here: If anyone wanted to change the number
>> of bits used for the various fields, each such change should require
>> an adjustment in as few independent places as possible. That is, there
>> shouldn't be multiple uses of literal 4 further up, and there also
>> shouldn't be a hidden connection between that 4 and the PGT_* values
>> here. I think you'd better express the GFN as such a PGT_* construct
>> as well. And you better would keep using PG_mask() and PG_shift().
>=20
> ... I think, this indeed makes sense. If got your request correctly, we=20
> can avoid disconnect here
> and reserve 3-bit field for count for both Arm32 and Arm64. The struct=20
> page_info's type_info field remains untouched.
>=20
>=20
> diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
> index ded74d2..8b73e1c 100644
> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -99,8 +99,14 @@ struct page_info
>  =C2=A0#define PGT_type_mask=C2=A0=C2=A0=C2=A0=C2=A0 PG_mask(1, 1)=C2=A0 =
/* Bits 31 or=20
> 63.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
>=20
>  =C2=A0 /* Count of uses of this frame as its current type. */
> -#define PGT_count_width=C2=A0=C2=A0 PG_shift(2)
> -#define PGT_count_mask=C2=A0=C2=A0=C2=A0 ((1UL<<PGT_count_width)-1)
> +#define PGT_count_base=C2=A0=C2=A0=C2=A0 PG_shift(4)
> +#define PGT_count_mask=C2=A0=C2=A0=C2=A0 PG_mask(7, 4)
> +
> +/* Stored in bits [27:0] or [59:0] GFN if page is used for grant table=20
> frame */
> +#define PGT_gfn_width=C2=A0=C2=A0=C2=A0=C2=A0 PG_shift(4)
> +#define PGT_gfn_mask=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ((1UL<<PGT_gfn_width)=
-1)
> +
> +#define PGT_INVALID_FRAME_GFN=C2=A0=C2=A0 _gfn(PGT_gfn_mask)
>=20
>  =C2=A0 /* Cleared when the owning guest 'frees' this page. */
>  =C2=A0#define _PGC_allocated=C2=A0=C2=A0=C2=A0 PG_shift(1)
> @@ -163,6 +169,22 @@ extern unsigned long xenheap_base_pdx;
>=20
>  =C2=A0#define maddr_get_owner(ma) (page_get_owner(maddr_to_page((ma))))
>=20
> +#define page_get_frame_gfn(_p) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 gfn_t gfn_ =3D _gfn((_p)->u.inuse.type_info & PGT_gfn=
_mask);=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : g=
fn_;=C2=A0=C2=A0 \
> +})
> +
> +#define page_set_frame_gfn(_p, _gfn) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 gfn_t gfn_ =3D gfn_eq(_gfn, INVALID_GFN) ?=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 PGT_INVALID_FRAME_GFN : _gfn;=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 (_p)->u.inuse.type_info &=3D ~PGT_gfn_mask;=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 (_p)->u.inuse.type_info |=3D gfn_x(gfn_);=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +})
>=20
> [snip]
>=20
> Is it close to what you keep in mind?

Yes. Just to be sure - did you check that moving the count up
from starting at bit 0 is compatible with all present uses? At
least on x86 I think we have uses where it is assumed to occupy
the bottom so many bits (and the same also for PGC_count_mask).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:32:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:32:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186277.334998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3rx-0001dN-Dn; Tue, 14 Sep 2021 08:32:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186277.334998; Tue, 14 Sep 2021 08:32:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3rx-0001dF-AT; Tue, 14 Sep 2021 08:32:45 +0000
Received: by outflank-mailman (input) for mailman id 186277;
 Tue, 14 Sep 2021 08:32:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ3rw-0001d3-4O
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:32:44 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5427d412-1536-11ec-b404-12813bfff9fa;
 Tue, 14 Sep 2021 08:32:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5427d412-1536-11ec-b404-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631608363;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=ZezQPooQbQlS0umoJVJdwI8mPZjcPE0V5DvMh+Oe/K4=;
  b=SAbt+qScwP6Nh9nsChqjFwcRH7z8Is3aMT19GxbrrdxnfCsbT47iTBf0
   084LDjKgy1QXzgcSF2K8BbsT0s66TBNJ6zXiOmMcPtTSTu/4QPbminsyQ
   TSOyBdcp13Cxe89MnSMbAOXjoNbY3u2DTyXxYPXEG7uo4o3IhZh2hRGN2
   8=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: cvoL9MUQLVgbPbZQ1EON2NXTngTlhnZMZvV9P0JHsrjO95o4Kc0jVK2qBGyp4YoYNnilRHHN/s
 pKorCaSGcbgHEZfO1XgcsCHYCdUKAh6pDgETNDfqQfyAW+ckVhqmSbiiwuE9XSQmkjQefd0qAg
 CWvTuf8oXTyJ/ynzF6ZlDE7mNQamAu+4Bid3kDRF6CU5p3Cm2I7KJGMhs1xNwrzMyhvRdaRFpA
 8UA0smURnM2+8e9Gex1fFxOWM28+kNag6cRnTjNqH1hgktT0pj0CoOdb1FzcAMKNyeCSPLzPxQ
 rxdci4R7WcDCiX7r/jpL9LO4
X-SBRS: 5.1
X-MesageID: 52248973
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:9xEIlarOdfLmYXYsGQI/CI2ADkReBmLvYhIvgKrLsJaIsI4StFCzt
 garIBmDPK2CM2v2KNx3bt/j9hxQ65fSx4MySwVp/HsyHilGp5uZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0IHlW1rlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnb2VWS12O7X3pM8cc0UFGh5FbfJcw6CSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFJkYtXx6iynQEN4tQIzZQrWM7thdtNs1rp0UR6qON
 pVCAdZpRDTqbAZrF00vM7IVnvXypXvUTgNg8U3A8MLb5ECMlVcsgdABKuH9YceWTM9YmkKZo
 GPu/GnjBBwectuFxlKt9nOqm/+Kni7hXo8WPKO3++Qsg1CJwGEXThoMWjOTsfS/z0KzRd9bA
 0gV4TY167g/8lSxSdvwVAH+p2SL1jYeUddNF+wx6CmW17HZpQ2eAwAsTDFbb8c9nNQrXjFs3
 ViM9/vjAiZuq/uSUm6H8amPriKaPjIcJmsPIyQDSGM4D8LL+d9pyEiVF5A6TfDz3oad9SzML
 y6ijQ0ureoWlfU3/KSboQ3OvAn8+YLyUVtgjunIZV5J/j+Vdab8OdfxswOGvKofRGqKZgLe5
 ylfwqBy+MhLVMvUxXLXGI3hCZn0v67tDdHKvbJ483DNHRyW8ni/dMh75DhkLS+F2e5VJGe0P
 Cc/Ve5XjaK/3UdGj4csOOpd6OxwlMAM8OgJsdiOMrJzjmBZLlPvwc2XTRf4M5rRfK0QrE3CE
 c3DLZbE4Ykm5VRPk2PtGrZ1PU4D7SEi32LDLa3GI+Cc+ePGPha9EO5dWHPXN7xRxP7U8W39r
 ocEX+PXmko3bQELSnSOmWLlBQtRdiZT6FGfg5E/S9Nv1SI9RDh6WqOOmOh4E2Gn9owM/tr1E
 riGchYw4HL0hGHdKBXMbXZmabj1Wo14o259NispVWtEEVB6CWp2xKtAJZYxY5c98+lvkax9Q
 /UfIp3SCfVTUDXXvT8aaMCl/oBlcR2qgyOIPjakP2djL8IxGVSR94+2ZBbr+QkPEjGz6Zk0r
 Yq/216JWpEEXQljUprbMar901OrsHEBs+tuRE+UcMJLcUDh/dEyeSz8h/M6Oe8WLhDHymfI3
 gqaG05A9+LMv5U04J/CgqXd99WlFO53H0x7GWjH7OnpaXmGrzT7mYIZCbSGZzHQUm/w6Z6OX
 +QNwqGuKuADkXZLr5F4T+Rhw5Uh6oa9vLRd1AllQinGNgz5FrN6L3Ca9sBTrakRlKRBsA67V
 0/TqNlXPbKFZJHsHFILfVd3a+2C0bcfmyXI7ORzK0L/vXcl8L2CWERUHh+NlC0Cc+clbNJ7m
 b8s6JwM9giyqhs2KdLX3Clb+lOFImEET6h65IoRB5Xmi1Zzx1xPCXAG5vQaPH1bhw1wD3QX
IronPort-HdrOrdr: A9a23:WW820arnXNcFu8vVyO3oKwcaV5vPL9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ
 0QDpSWa+eAc2SS7/yKmTVQeuxIqLLskNHK9JbjJjVWPHlXgslbnnhE422gYytLrWd9dP4E/M
 323Ls6m9PsQwVdUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZuzU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDk1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXo90fLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplWy2/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 ggMCjl3ocXTbqmVQGbgoE2q+bcHEjbXy32DnTqg/blkgS/xxtCvg4lLM92pAZ2yHtycegB2w
 3+CNUaqFh5dL5jUUtMPpZwfSKJMB2+ffvtChPaHb21LtBOB5ryw6SHlYndotvaP6A18A==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52248973"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=awWTx/jNK4zh1zBWnqhfovflqthpRwZz/2u4F+SH8fy7n6QwSKuQSaK+VbmyrTQoWsT1cTZRytbl/ITne2w4hsjYJO6jsgalygw84+YV5pZeMId94mZoHpah/n+bZQ0U3UliySduBN1oG8OPKeMcEdIfJYVPuHz45z0ZQUIpXXgyx8aNxohJLtR7GP1ZXS+DhJREN/+di2qzYlDQk750V0CpOFRh+B+loUBTM37IlLT9fBg3CoLBJ5aIoC8e29wfbVWOERcFyyxfRZc3WxMW8oJDrDai572uvBv8vav6C5SdhjMTMI7LFAsjagtRrwr0q4gkUQRBElYiXHIimejWMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wrQC+1b7iEYcqIUfkjJLrWIXe0PichpEoEMz0g+BwNM=;
 b=fjtdxbag+iVHLVaL1uQoMs6JDoqjuhwMLwZjKZeTZAAMjTBphd8yMZmOG2sH75pBg1WK+th0YUlUfgtylJyYfsDHiOJsmBPJjdITccrMaHBo34/yqo1sUqV4J0RhCkIr58150WLET5xj/nuElN5lfbQWiAa8XC94Be6NDgMHp8THNsrby7n+HBVSxjQNYFzjw/sPR/LaG/C9XxHCsNwJjp7RzJY+BVBWTZLXCC0yNgsdBoZUJg9oL1VUu4mRJDh3l/KxQpStxVRtrqRYXAV5V89bxDUD76Taj/6vJKQZp+1nLjZYf6paKGBiTrog/c/+CFxpiY0Qbifpv2I80VA8vA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wrQC+1b7iEYcqIUfkjJLrWIXe0PichpEoEMz0g+BwNM=;
 b=vOX1CYcoVuCYjxJZYI3Z23+qPyUXdzNOgpjFV98t6jMhMDvOXskz1hCMOA+uNmeXXi3cKQmeYDDFaKXRiilGCxNQiqt2DjRjChPqf3AFyvOkzOzLKmS5n6hnrtjLUAjt9+TNtifc/PXB3+/Qc87teAALPV9ZvCMVRlfRMufFB8w=
Date: Tue, 14 Sep 2021 10:32:36 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Juergen Gross <jgross@suse.com>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
Message-ID: <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
X-ClientProxiedBy: LO4P123CA0422.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18b::13) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1fb221f9-8cc5-4098-28d2-08d9775a36ea
X-MS-TrafficTypeDiagnostic: DM4PR03MB5998:
X-Microsoft-Antispam-PRVS: <DM4PR03MB59980174E82C64E38A8624328FDA9@DM4PR03MB5998.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: VaRrgc3YzXsET2YHohp8d+P9Sw8aQ/KXZwKSDxV2Uu0RDjZhUpqUjEIGoWSoJ/FDKLk+5MfKZFtuU7IugC7mAjlzACZhR3ESc1RHZXG1/yZf9LGu3tqszwQifOlqHlXTwHmPrCSj82lRSWWo6PEPgLKZQSvOnJVcuuy8d0YOAg1I3N20foBnFNw7Hdzt05tI4484wV7X6LdbVTC0vOgS4fSVW3gepYFL/D/hxnz7f/QYivlOBhZZ06lRHQ3ugm8VClRZpVwuKisVJ1LB7AFHDaPP3RNkwBscGvV3g5Oz8HBxZcVYHxYWfBCU2Awl8SvWQBAt85cWfOxp6bDOuF9byQmXaGnmwUJxGnq/Vy84cZIHX4O/MCn9sm9cm3zfa20XjDlFA5ImSreqZZbhXP0s+Y01/Ta46QtHk8xsa8y/M+R/wAaGJkxDLjxXNOOnHglMBkK6fICNCEDQ+8UeFjumvZfGKi7uu4ZML+PXQz75Nco97elh4SavBOWgMN7KuWVTdUki1nCXnRxFSS4nTG42utMPJdVOazWh7ET/+buIbm2q4cIV3fzhtmIPGhXeL5rCRkqLSDTdzBcSGWq9Al74Ub5OyYquoL1j5pQqKgQWIMbGBWT3B9OWfh9LMg9OIIo3+N4T0ynWLbRIDCAODzcFWw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(366004)(136003)(39860400002)(396003)(4326008)(6486002)(38100700002)(6666004)(26005)(9686003)(2906002)(8676002)(6496006)(54906003)(66556008)(478600001)(66476007)(956004)(66946007)(85182001)(186003)(316002)(8936002)(86362001)(6916009)(5660300002)(83380400001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aGN6WGxUSzltaTVUcXhFZGVuOUdudERxNWVXRmIxeWRJR3lKVUMvckxjeUdh?=
 =?utf-8?B?VTY3M1dZUVdYZkRraXN1ZzV1L2lCUHVPSnNxOUpNcWNXNmRmb2xOalJ5cXBS?=
 =?utf-8?B?WU1XVENKbTRvckN4RG5yWWw2YkZiNGJaa2NKbHZWVUd0Q0JiaTJ1L21BVGYy?=
 =?utf-8?B?TXdLNzB2bFpUUFRyVkJab1dlQUVtSG5tY1Yydy9udHhhU0UrUlYxZzFQaXpW?=
 =?utf-8?B?bmE1VHpLQjE0ci80Y01abndiSmJ2UGVhbC93cFovMG9DeUVXUk5WTjg2TDEy?=
 =?utf-8?B?blRmSlZ5UFQwVUxlQzljTGJnTkkrM2N5MjZiTUpYR1RFUHgwSUJScEFPTWVk?=
 =?utf-8?B?cURuUkR0YlJCZWNMV2Nod0NvbnRNMGxRN3c4dkhzZ0xlKzNzWDkra2xNTFZx?=
 =?utf-8?B?RlRWbyszczErbUk3MWo4TjFTWVJMYis4ZENYY1NJK1R2U21aTGJCWkc2WGtX?=
 =?utf-8?B?R1JEUHRWcTZyeXcvUVRVNjMzaDBqTE9vd2NsUW9CUktjaG4zTW9wdDU1V2dN?=
 =?utf-8?B?UzgxeVJiTThrMllick5RYmVJSk5xZXYvQkRqWWl4eUdvbGh2RWNiNml3S1Ja?=
 =?utf-8?B?RFhOeWRiRmlpZ3VrakZXenQvQUVuamxYVFVHaXhhTEJxWVJHd1ZvbWpIRVE1?=
 =?utf-8?B?WDNueXZoNldJRHZweXdobWNEbnpyMmdGVGpEcWlIUU5NNHgyQ2xXNWdVMkw2?=
 =?utf-8?B?RzdLRUpaWkJQeWVFYkYrSzAvL3VVd3FyQTRqdUpzeFZ2N2pJdlRZa3pYTEpP?=
 =?utf-8?B?SkNjS0luN0JGcEFHUWlxOG0veCsxSW9DNFo4a0ZxVjVaak5PQm9xdzlPV2VM?=
 =?utf-8?B?Q0ZFNGt1dGc2c09HWUpRR2xTUHVzT2R6NXFVbXF4UFlwc0ZmeU1VWHQ2ZU4y?=
 =?utf-8?B?a2NlRmdsaDdud1NqMjI4V0pRWGpmUFFET1o3MW9lKytSU3RIS2F2dFBRdUo1?=
 =?utf-8?B?ejRJc0lIUzZiN0grT0N2VkNlRmlsTnBtWUNMT1U2WXlBY2QrVmtYdDExQWhn?=
 =?utf-8?B?WGlIVWFPOHRWT2xWUkNaZWJ2UmlxZWs2NFdDakFFc2ptUUFkaktqd2k4dlQ1?=
 =?utf-8?B?Rzk2bDNoZUlvUkxQd0FLL1lHZktZMDIxbHNLWGxuU3lpTWhSU3p3U0tUYmNl?=
 =?utf-8?B?b1FGbzNuZ0hBeFN6blhhM3BCUnNHTFl3V1RtZkN4a2J3S1BjN2dOcU9jOTgw?=
 =?utf-8?B?aVBDM1N5ZFRvYmlYWUdqYTRZM3pBRmc1L0xNY3hhalRxQWpFWUFxbnpVS1JP?=
 =?utf-8?B?UjdZSmdyOXRxY3NUOThMWkErQXY3bHN5ZUprZnAvakQ4L3lBOHBvZm9wYWtU?=
 =?utf-8?B?SWtkRmVWVG9iZ25SQWNObitYQ3hFZ1JyVDZkM2gxZ29wSG9xdFhsc09NYVN5?=
 =?utf-8?B?MWY1TjFPNUQ2UnVNcFllbmYrRG9taEg5TFhXZDZVUUkyc0RFZUVCVHI0MGhF?=
 =?utf-8?B?d0pXaUtObi9kZXo1dWFqMCthbHkyaG56QWZjNzM0a1l4NjFuYnRjaVNSeVlh?=
 =?utf-8?B?aEZhakc2d3pMR0djNW9hMFFPYkNNNWU2Q1JUZ3Z3ZjdlNmZFVmxRTFl3V0dB?=
 =?utf-8?B?aXFwU09Yb2hZQ0Z0Z2RTY1A1dDVmZk5adUJ4aFBLS3VuWlJldXhyTUd5SjVw?=
 =?utf-8?B?YjlOQlV0NnBHU05uQ2RRTTcyNmdLYThjelpqemxrTFpWMzVyRnFTTGpFSzRi?=
 =?utf-8?B?anlnRHB3cGlYUTRXVy9Xays4QWQxWnAwOGhKeVVQSFA2ZkZDUExkWWtraHNH?=
 =?utf-8?Q?ZyYw56iLjSHDrocjfRcqVEiCRMqwv4GylWls66V?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 1fb221f9-8cc5-4098-28d2-08d9775a36ea
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 08:32:40.3679
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Wg3IqW9xrUbNlHaxYIFTRUi27UYPRzI7Yel5HTt24AEGCbKd1+DptdhhNo4nXCgWohEndo6GAFE3ChT5SlPUyw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB5998
X-OriginatorOrg: citrix.com

On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
> In order to try to debug hypervisor side breakage from XSA-378 I found
> myself urged to finally give PVH Dom0 a try. Sadly things didn't work
> quite as expected. In the course of investigating these issues I actually
> spotted one piece of PV Dom0 breakage as well, a fix for which is also
> included here.
> 
> There are two immediate remaining issues (also mentioned in affected
> patches):
> 
> 1) It is not clear to me how PCI device reporting is to work. PV Dom0
>    reports devices as they're discovered, including ones the hypervisor
>    may not have been able to discover itself (ones on segments other
>    than 0 or hotplugged ones). The respective hypercall, however, is
>    inaccessible to PVH Dom0. Depending on the answer to this, either
>    the hypervisor will need changing (to permit the call) or patch 2
>    here will need further refinement.

I would rather prefer if we could limit the hypercall usage to only
report hotplugged segments to Xen. Then Xen would have to scan the
segment when reported and add any devices found.

Such hypercall must be used before dom0 tries to access any device, as
otherwise the BARs won't be mapped in the second stage translation and
the traps for the MCFG area won't be setup either.

> 
> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
>    console) when in a non-default mode (i.e. not 80x25 text), as the
>    necessary information (in particular about VESA-bases LFB modes) is
>    not communicated. On the hypervisor side this looks like deliberate
>    behavior, but it is unclear to me what the intentions were towards
>    an alternative model. (X may be able to access the screen depending
>    on whether it has a suitable driver besides the presently unusable
>    /dev/fb<N> based one.)

I had to admit most of my boxes are headless servers, albeit I have
one NUC I can use to test gfx stuff, so I don't really use gfx output
with Xen.

As I understand such information is fetched from the BIOS and passed
into Xen, which should then hand it over to the dom0 kernel?

I guess the only way for Linux dom0 kernel to fetch that information
would be to emulate the BIOS or drop into realmode and issue the BIOS
calls?

Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
info using the PV EFI interface?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:33:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:33:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186288.335008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3sx-0002NJ-Qw; Tue, 14 Sep 2021 08:33:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186288.335008; Tue, 14 Sep 2021 08:33:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3sx-0002NC-Ny; Tue, 14 Sep 2021 08:33:47 +0000
Received: by outflank-mailman (input) for mailman id 186288;
 Tue, 14 Sep 2021 08:33:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rbOp=OE=linux.ibm.com=rppt@srs-us1.protection.inumbo.net>)
 id 1mQ3sw-0002Mx-71
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:33:46 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.156.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 471392ab-cb59-486f-9816-50ed4797f3a9;
 Tue, 14 Sep 2021 08:33:45 +0000 (UTC)
Received: from pps.filterd (m0098409.ppops.net [127.0.0.1])
 by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18E8DDVp003164;
 Tue, 14 Sep 2021 04:33:30 -0400
Received: from pps.reinject (localhost [127.0.0.1])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3b2n91v8qr-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 04:33:30 -0400
Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1])
 by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18E5xNPG014160;
 Tue, 14 Sep 2021 04:33:30 -0400
Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com
 [169.51.49.99])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3b2n91v8pu-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 04:33:30 -0400
Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1])
 by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18E8XOWD005452;
 Tue, 14 Sep 2021 08:33:27 GMT
Received: from b06avi18626390.portsmouth.uk.ibm.com
 (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192])
 by ppma04ams.nl.ibm.com with ESMTP id 3b0m39rvyu-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 08:33:27 +0000
Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com
 [9.149.105.59])
 by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP
 id 18E8Sthk59048334
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Tue, 14 Sep 2021 08:28:55 GMT
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 1501EA4057;
 Tue, 14 Sep 2021 08:33:24 +0000 (GMT)
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 3D5C7A405F;
 Tue, 14 Sep 2021 08:33:23 +0000 (GMT)
Received: from linux.ibm.com (unknown [9.145.48.104])
 by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTPS;
 Tue, 14 Sep 2021 08:33:23 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 471392ab-cb59-486f-9816-50ed4797f3a9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc :
 subject : message-id : references : content-type : in-reply-to :
 content-transfer-encoding : mime-version; s=pp1;
 bh=pKxQpYGx7FCYqXPJeZcEZp0s2mCsJ/vTkMebph8UpHg=;
 b=c/jFDeff74opNPuhZivqJl24DeEXWtHpxq0scPesE73bTGItp2la9ee5Le8rh2WDqhK8
 19KvhsAHO5kIig7f3OtmztBotyUNdOjPpHcqNkQ3aoekjW6ssWFDj8sYyZRYyGZEVeJp
 W+KIFryQLSaPQLEYn80iFdI3jcSn3aukG2CQEjrLUBIWs8KA8ldRz5hq0JYSy3S03fav
 9eLKLDhoIapuWcwXS0GzUKMmu5bY38f0ZyI/PY8HPiQJeiePP1FRVQKFiR/WEcMzXNJG
 dTFSWKJ+Qj6/VfFT0S70+1k7A/Gg0Vbjd9Oqc/5OVdajPAHX5Q4/5htM421/ItjqzpKF 4g== 
Date: Tue, 14 Sep 2021 11:33:21 +0300
From: Mike Rapoport <rppt@linux.ibm.com>
To: Juergen Gross <jgross@suse.com>
Cc: Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
        Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
        linux-kernel@vger.kernel.org,
        xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
Message-ID: <YUBeUZdduCAbOQJN@linux.ibm.com>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
 <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: Fek_4RKUSmSBac3314u_qg2oT-UwR5fB
X-Proofpoint-GUID: 1n5L13yxXZwxMDPxGVuAivJeUC3QPW11
Content-Transfer-Encoding: 8bit
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475
 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 spamscore=0
 adultscore=0 mlxscore=0 malwarescore=0 impostorscore=0 bulkscore=0
 phishscore=0 priorityscore=1501 suspectscore=0 mlxlogscore=999
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140027

On Tue, Sep 14, 2021 at 09:14:38AM +0200, Juergen Gross wrote:
> On 13.09.21 14:50, Marek Marczykowski-Grecki wrote:
> > Hi,
> > 
> > Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the
> > kernel version.
> > Test environment:
> >   - Xen 4.14.2
> >   - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
> >   - Linux 5.13.13, confirmed also on 5.14
> > 
> > The crash happens only if the initramfs has earlycpio with microcode.
> > I don't have a serial console, but I've got a photo with crash message
> > (from Xen, Linux doesn't managed to print anything):
> > https://user-images.githubusercontent.com/726704/133084966-5038f37e-001b-4688-9f90-83d09be3dc2d.jpg
> > 
> > Transcription of some of it:
> > 
> >      mapping kernel into physical memory
> >      about to get started
> >      (XEN) Pagetable walk from ffffffff82810888:
> >      (XEN)  L4[0x1ff] = 0000000332815067 0000000000002815
> >      (XEN)  L3[0x1fe] = 0000000332816067 0000000000002816
> >      (XEN)  L2[0x014] = 0000000334018067 0000000000004018
> >      (XEN)  L1[0x010] = 0000000332810067 0000000000002810
> >      (XEN) domain_crash_sync called from entry.S: fault at ffff82d04033e790 x86_64/entry.S#domain_crash_page_fault
> >      (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
> >      (XEN) ----[ Xen-4.14.2  x86_64  debug=n  Not tainted ]----
> >      (XEN) CPU:    0
> >      (XEN) RIP:    e033:[<0000000000000000>]
> 
> The domain's run state seems to be completely clobbered.
> 
> Did you try to boot the kernel with "earlyprintk=xen" to get some idea
> how far it progressed?
> 
> I could imagine that doing the early reservations after the call of
> e820__memory_setup() is problematic, as Xen PV guests have a hook in
> this function performing some rather extended actions.

Right, among them it may relocate initrd:

https://elixir.bootlin.com/linux/latest/source/arch/x86/xen/setup.c#L872
 
and this may cause the reported crash.

> I'm not sure the call of early_reserve_memory() can be moved just before
> the e820__memory_setup() call. If this is possibel it should be done
> IMO, if not then the reservations which have been at the start of
> setup_arch() might need to go there again.

early_reserve_memory() can be moved to the beginning of setup_arch().

Anther possibility is to move initrd relocation out of xen_setup_memory()
and maybe even integrate it somehow in reserve_initrd().

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:39:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:39:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186303.335020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3yH-0003OP-GC; Tue, 14 Sep 2021 08:39:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186303.335020; Tue, 14 Sep 2021 08:39:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ3yH-0003OI-CM; Tue, 14 Sep 2021 08:39:17 +0000
Received: by outflank-mailman (input) for mailman id 186303;
 Tue, 14 Sep 2021 08:39:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ3yG-0003O9-6f
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:39:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3dcb82d0-1537-11ec-b404-12813bfff9fa;
 Tue, 14 Sep 2021 08:39:15 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-7ixzqw4GPyKPIDlMrlTjbw-1; Tue, 14 Sep 2021 10:39:13 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 08:39:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 08:39:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0032.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 08:39:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dcb82d0-1537-11ec-b404-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631608754;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HrAVZXNh4bCJ752ooATAW733Fmm6XrzqI9uo9s4JpAY=;
	b=KXV1i/JNzBT2p6pgz49dz8z7kyJ4rq6JudwwXTDFPWG/HqmWT0Fm/vSlxvRxnW7PlVHjJP
	EsovSeCW3CK8GVF6E6Y97YsyntabioTzNtD6Fl4yhc7XEz7DANZ1NLpaitHrhfZEW/V5rz
	AEZJxY1x8mmvJPbebM720GA65C5R3bI=
X-MC-Unique: 7ixzqw4GPyKPIDlMrlTjbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ybu5fnseQZNF/MCFwuOkw2QhBRdjHeK1S4D4DR/PBfTSOADFAMR6AsXkJBVfHR2MwBrL9Vf25VSSb9yI28vRT3CxdPymhIEdvknl6/bxwttzXj26jQFKVav+Bog+Pt5VKiBN5yqkHCOuGW41y2ewkHF6Kh5PkMG8btGkjTMsVPtxHVQGQORwlKQC9T/4fxszq8LcUErRu71DkwBUjneiRUuCa2iZnMmpKypjya7WFGfDFT4oxKB8tub4otXxhTavQO2Sn7NHbK4VaEt2xHkjTPs2o25EIsdoliUAUdcvWpU6+nwGbQgHEpqCtmxHzK5+krH6KqBBcf2p/QdDYB6sGQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+wuUFNJGxVbJ34h7Y6G7CvvRLqWX2FD6F/vSJ208lEc=;
 b=R7mHUVaxYDwN3++87QHu35r26yNwg/grJNfBZD0D1yx+VwHgH7mO/pKTE5NLDAMtOqaiypbdEBy900bEgidhWWOUyXIxZahQRsf+KLd3/ZbCHmLhGy176gqffvCfEhmQG+UQyo+/oSg7vXTIL7GbbnMWiIyh6xxLPNmntMrtOJzsHExtqn4tIwo95p+Zs8pFPnVnCw6ecQ8N1yh3UnvHMH+RuHkjFpxiKraSK2NpCJd2B0C0SyNXf7mgCKUuRxBk807/2lcsNo0YyDQMaDcYs5Q0YEDpvF4UZ/yt21SFHT77c2FJbVdNI04UXcEVmrhFmAb2q4pjl09Iro3/A3evPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: linux.ibm.com; dkim=none (message not signed)
 header.d=none;linux.ibm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
To: Juergen Gross <jgross@suse.com>,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?= <marmarek@invisiblethingslab.com>
CC: Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@suse.de>,
 Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
 linux-kernel@vger.kernel.org, xen-devel <xen-devel@lists.xenproject.org>,
 Mike Rapoport <rppt@linux.ibm.com>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
 <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ed6b9999-7a7b-31db-9e88-efdeecefa45b@suse.com>
Date: Tue, 14 Sep 2021 10:39:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2P264CA0032.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 12f5a466-d40c-4cc4-d809-08d9775b201d
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2445AD25DD0FD06D6BD1A9D9B3DA9@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fIMuygzkZvvBzjuC6V3opgqW0OLrFnwpTOAZNc3tCSJgq7LK8XimuGxm2+Yav8SetnKFs/9ccsWiCZQUsaLJZ6ohvof20ZhPrU4JJ1AJulGRW+EyFc51IytCZv8ivOwyHl54tfDjphFljPouSDyG/SBwBho7kKM7RHrd+JUxOQF2+aw1xbWiVIwYRk01J6mzcJ6cg9+Ry0LVuVHHlQn/KkU14P4BQIAFXOGDMr2/XUrI6RvvXpvmvS3KpnPhsi9+6o3YZGrXPSYWnWa65K6xfk9U/XrNZpPKw6xEUruJNj6SEuuej9hRF+lg4Pbb3DI3scNZb2dcaIuv/OurYseW60Agicn9OvUW3iqDyBRroBFWZWRj6om804Mv71klNEk7SY0OY+hu/PI9U0Not/5t6fBEC0qmGC43Ey/jYwmfyNjvMmB3Qb2bMT4i2u/ZupLQQ+hpCob5yhTdKd0E9zc+f/DSdisJQjXFiNDb55VCSjWEmdRkp5Iil+EDxwXK8x3URNsGnujQoVuzscIemkp1xX4TZq7SQoIw+y5rJ6ygFzewB9XXyT2Ayip+8NZ02CZIXzXy4k1Z+kpYEGpdzVUY35oqYcoOdGUvn/Fi7Jir5/ZuVQBez3MGqRC0jN916i9PMeJD658CnOBjPtJ7Bkma0iGI6WcwsKWwqiCLUNP2dFA1R3syBzQhyBhjangkA8ipssOr3QbbWHBvQwPFhvfym3iEjv+4jeCvb4twFm+KZYk/AHnHC/1eUHjNWWZ2CkrEXEztUaiaWZiuGWuzbjfI5CYTkllYXYSV0cXd2iNRvU4UPP5arDhHZzdO/txBIREV8zl4HWsYr8kt2PczZB0hOg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(396003)(136003)(346002)(36756003)(66574015)(31686004)(5660300002)(66946007)(2906002)(66556008)(38100700002)(2616005)(956004)(66476007)(6486002)(86362001)(4326008)(16576012)(54906003)(31696002)(110136005)(8936002)(316002)(83380400001)(26005)(478600001)(966005)(186003)(8676002)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?U1mDhRc7AJe8ZyO5FYxNVDTmpq9GrqbhAxz0gdmhyfZ8V5U9L95Ph6jOW4KL?=
 =?us-ascii?Q?nt9VchFIgWoV71E2bFpZ8OYp2teLCVuTsZsfotDf5/dmtdSnzQGmB1mib+yr?=
 =?us-ascii?Q?7Kmoa7Foa/6Dx14xaiaPnCnxOUZADwCnk5isVqAWA3rGlZFCQ0c8U2Du1a+5?=
 =?us-ascii?Q?DVnYz1xHSH18bVOQUuIzcuMu2RkA0Ya5PXqIN0N9+h9h0+JlQtXm4RZlY0qS?=
 =?us-ascii?Q?goeTsDYDj6KrbuXP0Kis7LhlKstJPjcNmBNycr34Brg20lZgHg/livu6mZYD?=
 =?us-ascii?Q?87jeKremBrks9lPOzZvRfb/z4O8N1BCPVAV8KcaciG5v+AKYB7jVmMShpbgk?=
 =?us-ascii?Q?zwsWKmkTV51PtTEIKfVLLUONtZ33qcWuTSP9PV4bmuNh32KCOVuzaOYrjTVS?=
 =?us-ascii?Q?wPLH99ih6I+zYV/C/CPVaq8LDJ0Bhl06jvGj8yhpuhSybLTx8m4VkXVuwRt6?=
 =?us-ascii?Q?i1iy+3IH4sKSRa3pbi1DzTvYHoh4l02YfPHDYk15uEsI7gV2Ig+dhShyH1RK?=
 =?us-ascii?Q?cnqPI4RXlGQ4//p5etMvJIKtCs80v/pZukzzPlp6ZJ9ASbE5KIoDdhZTXLaS?=
 =?us-ascii?Q?OzyWeJDsbQG2MWgH0nbS7B02K5ghS/DthP/twv2WJpNLTor6H7eKOEjGZYIY?=
 =?us-ascii?Q?mw8nQPBveo5O5zMLjETfzGApcG5qn5Fdlg84cb+qj5xQzrjjE/mn1tovga8I?=
 =?us-ascii?Q?UnS52hXiKzQtJA9Q7hWq0tUpiuOxtKJqdY3/3RgS4gL2i7CUWWaw5eUrOy8e?=
 =?us-ascii?Q?on73FKfD9y6SnWMnqyEFz3+CRKRf5svBuP07CJRhjCRunO62en+kgVwaKvWY?=
 =?us-ascii?Q?ojUpEpQLHtsjANg9TGVza5dQDSHxYmjuUMuPr9+zHW5oOAjsROu97KS661Lm?=
 =?us-ascii?Q?0inBr9mb3NSWYPYtESuSfzcuovUEvnZ0FJeiLH1J/SoLWPaGcFnVh2uBh9Su?=
 =?us-ascii?Q?0ae55jNYH8C5fpz7cnJHLx6i0zGDhDME1HrTtX2vdlwnmbLqo7QDlcAnOzBo?=
 =?us-ascii?Q?t6AyF3NdnZVL+9s/X5kEG21PVejPxeS3GXFONwjxczZrWPXM6ESFMSXZwRM4?=
 =?us-ascii?Q?XOckGQcLCGNclZOGpauLeAub2XETxpY322fIkCiIPzOiwvKVhFyrQ173pUks?=
 =?us-ascii?Q?4GgCQEvkOM/kQKIanpYb/2CWPIBS+42Dnc23W5Q0zkszbEslhTu+SAp4nMSB?=
 =?us-ascii?Q?aO9W4kpVYS4FAKtF03Lo4J8NRCFXP+bUqeqVaQ1/5Bm8BwtCMnwMM6/J0VuE?=
 =?us-ascii?Q?2c2376FYvR7guIVcHBPC93o8wk461smNz900Onuum+d1eWr+sbSyS74gDk8b?=
 =?us-ascii?Q?fgBvGlO6XnuYw0Q3r8cTPLy8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 12f5a466-d40c-4cc4-d809-08d9775b201d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 08:39:11.5301
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3rppAkQOdYwbrLgt5z44IKy/+xDDqo9HUhN1BzUtbU6t+AOonysvbBoBE0FODIQuijbWedt3PZcz6WYEwR4ZwA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

On 14.09.2021 09:14, Juergen Gross wrote:
> On 13.09.21 14:50, Marek Marczykowski-G=C3=B3recki wrote:
>> Hi,
>>
>> Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the
>> kernel version.
>> Test environment:
>>   - Xen 4.14.2
>>   - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
>>   - Linux 5.13.13, confirmed also on 5.14
>>
>> The crash happens only if the initramfs has earlycpio with microcode.
>> I don't have a serial console, but I've got a photo with crash message
>> (from Xen, Linux doesn't managed to print anything):
>> https://user-images.githubusercontent.com/726704/133084966-5038f37e-001b=
-4688-9f90-83d09be3dc2d.jpg
>>
>> Transcription of some of it:
>>
>>      mapping kernel into physical memory
>>      about to get started
>>      (XEN) Pagetable walk from ffffffff82810888:
>>      (XEN)  L4[0x1ff] =3D 0000000332815067 0000000000002815
>>      (XEN)  L3[0x1fe] =3D 0000000332816067 0000000000002816
>>      (XEN)  L2[0x014] =3D 0000000334018067 0000000000004018
>>      (XEN)  L1[0x010] =3D 0000000332810067 0000000000002810
>>      (XEN) domain_crash_sync called from entry.S: fault at ffff82d04033e=
790 x86_64/entry.S#domain_crash_page_fault
>>      (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
>>      (XEN) ----[ Xen-4.14.2  x86_64  debug=3Dn  Not tainted ]----
>>      (XEN) CPU:    0
>>      (XEN) RIP:    e033:[<0000000000000000>]
>=20
> The domain's run state seems to be completely clobbered.
>=20
> Did you try to boot the kernel with "earlyprintk=3Dxen" to get some idea
> how far it progressed?

I guess without my "xen/x86: allow "earlyprintk=3Dxen" to work for PV Dom0"
"earlyprintk=3Dxen" would need to be accompanied by "console=3Dxenboot". I
have not tried whether this helps, this is merely a guess from having
looked at the code relatively recently.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 08:42:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 08:42:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186312.335030 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ40x-0004l7-TK; Tue, 14 Sep 2021 08:42:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186312.335030; Tue, 14 Sep 2021 08:42:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ40x-0004l0-QN; Tue, 14 Sep 2021 08:42:03 +0000
Received: by outflank-mailman (input) for mailman id 186312;
 Tue, 14 Sep 2021 08:42:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ40x-0004ks-7z
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 08:42:03 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a164eb7e-1537-11ec-b404-12813bfff9fa;
 Tue, 14 Sep 2021 08:42:02 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6A6401FDC4;
 Tue, 14 Sep 2021 08:42:01 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 291DD13E4F;
 Tue, 14 Sep 2021 08:42:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id sMF6CFlgQGE4KgAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 08:42:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a164eb7e-1537-11ec-b404-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631608921; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=xPouFTvUSxde8wBIN4mbuwBIT5eFtCgb2f3gPa9oKwA=;
	b=fFpbWhGXaIJcgrTvJMcAfJA2LoW1SmE4HLDBdYBLf/w1tNW+AKkRq+ZjHvSdd7GQu0+Dzk
	+eQc4mRGCsTtvMA2MDmE8kMPPznh0XifF83067GBKTF8uC710IdG8GoDQ4uxFbGsFTH3GW
	aHTOX19CdE9lIVjWUG158jaRkclxXek=
To: Mike Rapoport <rppt@linux.ibm.com>
Cc: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@suse.de>, Thomas Gleixner <tglx@linutronix.de>,
 x86@kernel.org, linux-kernel@vger.kernel.org,
 xen-devel <xen-devel@lists.xenproject.org>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
 <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
 <YUBeUZdduCAbOQJN@linux.ibm.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
Message-ID: <3680dfc8-d7a4-7b10-0070-eb476f55bcc3@suse.com>
Date: Tue, 14 Sep 2021 10:42:00 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YUBeUZdduCAbOQJN@linux.ibm.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ilb5mPSY6AWHgPesUzx2TavFJnFwZ81vl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ilb5mPSY6AWHgPesUzx2TavFJnFwZ81vl
Content-Type: multipart/mixed; boundary="ae01mqCaNnJ5aUgKVDKNJmIxZNb4zZNYO";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Mike Rapoport <rppt@linux.ibm.com>
Cc: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@suse.de>, Thomas Gleixner <tglx@linutronix.de>,
 x86@kernel.org, linux-kernel@vger.kernel.org,
 xen-devel <xen-devel@lists.xenproject.org>
Message-ID: <3680dfc8-d7a4-7b10-0070-eb476f55bcc3@suse.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
References: <YT9I+Xs9wOPVCIVd@mail-itl>
 <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
 <YUBeUZdduCAbOQJN@linux.ibm.com>
In-Reply-To: <YUBeUZdduCAbOQJN@linux.ibm.com>

--ae01mqCaNnJ5aUgKVDKNJmIxZNb4zZNYO
Content-Type: multipart/mixed;
 boundary="------------E3D07A4408E216413FF53383"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------E3D07A4408E216413FF53383
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 14.09.21 10:33, Mike Rapoport wrote:
> On Tue, Sep 14, 2021 at 09:14:38AM +0200, Juergen Gross wrote:
>> On 13.09.21 14:50, Marek Marczykowski-G=C3=B3recki wrote:
>>> Hi,
>>>
>>> Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing t=
he
>>> kernel version.
>>> Test environment:
>>>    - Xen 4.14.2
>>>    - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
>>>    - Linux 5.13.13, confirmed also on 5.14
>>>
>>> The crash happens only if the initramfs has earlycpio with microcode.=

>>> I don't have a serial console, but I've got a photo with crash messag=
e
>>> (from Xen, Linux doesn't managed to print anything):
>>> https://user-images.githubusercontent.com/726704/133084966-5038f37e-0=
01b-4688-9f90-83d09be3dc2d.jpg
>>>
>>> Transcription of some of it:
>>>
>>>       mapping kernel into physical memory
>>>       about to get started
>>>       (XEN) Pagetable walk from ffffffff82810888:
>>>       (XEN)  L4[0x1ff] =3D 0000000332815067 0000000000002815
>>>       (XEN)  L3[0x1fe] =3D 0000000332816067 0000000000002816
>>>       (XEN)  L2[0x014] =3D 0000000334018067 0000000000004018
>>>       (XEN)  L1[0x010] =3D 0000000332810067 0000000000002810
>>>       (XEN) domain_crash_sync called from entry.S: fault at ffff82d04=
033e790 x86_64/entry.S#domain_crash_page_fault
>>>       (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
>>>       (XEN) ----[ Xen-4.14.2  x86_64  debug=3Dn  Not tainted ]----
>>>       (XEN) CPU:    0
>>>       (XEN) RIP:    e033:[<0000000000000000>]
>>
>> The domain's run state seems to be completely clobbered.
>>
>> Did you try to boot the kernel with "earlyprintk=3Dxen" to get some id=
ea
>> how far it progressed?
>>
>> I could imagine that doing the early reservations after the call of
>> e820__memory_setup() is problematic, as Xen PV guests have a hook in
>> this function performing some rather extended actions.
>=20
> Right, among them it may relocate initrd:
>=20
> https://elixir.bootlin.com/linux/latest/source/arch/x86/xen/setup.c#L87=
2
>  =20
> and this may cause the reported crash.
>=20
>> I'm not sure the call of early_reserve_memory() can be moved just befo=
re
>> the e820__memory_setup() call. If this is possibel it should be done
>> IMO, if not then the reservations which have been at the start of
>> setup_arch() might need to go there again.
>=20
> early_reserve_memory() can be moved to the beginning of setup_arch().

IMO this should be the preferred fix. I will write a patch to do that.

> Anther possibility is to move initrd relocation out of xen_setup_memory=
()
> and maybe even integrate it somehow in reserve_initrd().

This would be rather complicated as xen_setup_memory() is changing the
memory map from one large memory chunk to match the memory map of the
host in case the system is running as dom0 (the need to do so has
historical reasons, changing that is no option). The initrd needs to be
moved in case it is using memory which is conflicting with the new
layout.


Juergen

--------------E3D07A4408E216413FF53383
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------E3D07A4408E216413FF53383--

--ae01mqCaNnJ5aUgKVDKNJmIxZNb4zZNYO--

--ilb5mPSY6AWHgPesUzx2TavFJnFwZ81vl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFAYFgFAwAAAAAACgkQsN6d1ii/Ey+h
mQgAi/OAAYr7BdRGH/RZDoPLz7fsoqtr0Mu7XgC5fjwZt8HlagWMA75VQaiH18wANcinlvINcwfM
0aqkBGugG9t+NBr4byO5C9UBLj/c3OYQzy0BYO9atduWdrBHxf2kYky0DAiHF+t8oh8uTBFj1pbt
os5URpuzOF2Cr4WU9GVOI6wW1ledwm5mw8PxLJ7kepTj1G5aPicatKBPOw3Xm0VbVzCMvyOx8ym5
eEa9alqxM9OjjEViu8Erxd7SfgKaAstQ5VQp03CHqVSUkKTHhy+qdpkyb0UNW9o+suV+FxJSYxSz
o7u7gAGZlb1GXF7h/ceHtWsxp5qjRx+EKfH8S2ucGA==
=bzrN
-----END PGP SIGNATURE-----

--ilb5mPSY6AWHgPesUzx2TavFJnFwZ81vl--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 09:03:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 09:03:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186322.335041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4Lm-0007RD-Nc; Tue, 14 Sep 2021 09:03:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186322.335041; Tue, 14 Sep 2021 09:03:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4Lm-0007R6-KI; Tue, 14 Sep 2021 09:03:34 +0000
Received: by outflank-mailman (input) for mailman id 186322;
 Tue, 14 Sep 2021 09:03:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ4Ll-0007R0-J1
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 09:03:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a25830b0-153a-11ec-b405-12813bfff9fa;
 Tue, 14 Sep 2021 09:03:32 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2058.outbound.protection.outlook.com [104.47.0.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-mi1iXnl2PI-17Kl0tCCJxA-2;
 Tue, 14 Sep 2021 11:03:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 14 Sep
 2021 09:03:25 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 09:03:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0060.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Tue, 14 Sep 2021 09:03:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a25830b0-153a-11ec-b405-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631610211;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=H9qvZHgpP4wC+33f5cI/t8e1yMNPzcANf4auAbyuyDY=;
	b=MKHEdoP9wxRVnw2jShttH3LVWLe2Gt1Zkh9X4BHcCBBj86VMB73axUW2/MjIiqb1ggJwyV
	K5hBnselFJyz62CKOzYUpQTHHPupdOEZIAkJbOkKA6qqYZBSbyZ6BNovQs+LGIN1zKubLp
	02WILRnuKOnLWmnhu1bEGCzFV1agx3U=
X-MC-Unique: mi1iXnl2PI-17Kl0tCCJxA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aGIbY7rmsEl3VFNdXBGzDzjBjHPciqJmGMCPqMzI3ihkiV3r0gp1dkWydrczcU2hykMnVXiNYgqjc49cctf+pC1ijgC7PF3gV0tDSARuOHhFJY43+qQsU/um03l0UEgyUROEVW36kYonkT9JuE8QufyigqsKpzt/ndPU6LQ86lPcfnbkW5N0jpGj/sj61rvey9ophhvsXXY2izVNWrvHXpmaJFPk5wjnbJecKbbUBY6CvpUA8nGRYqsR1PQBxXadFjLLmbELixZGgfCiA1ktU84PuJDiVdmUTqHZBPr+dzhiwJKLZhoxQwTgMARpiXH7RS7ipnlARlS7utFmbWJd6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4Wkq6JSIVgMF8sknRxAM07SqcyyinocsLxyGuOdW5Zg=;
 b=LsyGglzlAPAhJl9G+HQGC4gTyOFBPpSWB33iZVQ+m17+RBpF2E0puK/RyWpLb/ID+zl/15BIt5w+8EPdCiiQqvHmZNFkm2INpScXBWNRv+vJSDQnEdA9ty2sHCZt1aEQHhwhi3GsnXS2MqAOCzK9BUr5+RQ6S8L2y54Pjp2X9qpnhl7S/PqSVIMLa/J1mdcWB/YZJ8n7GndrPZxxQSps4ak7nHUqn+yjuFDnIxyFbPSQ0mPZRddXao5mxGyv4tR6foxfzzbTXVLET5RnIaXPzj4L7BhmXF39i35lQKIdlFEoL0NGUlW0/Hi6pw3LGVeI9FNy3pHWOEPPUd8D9Da5Kg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
Date: Tue, 14 Sep 2021 11:03:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0060.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6b263a4c-9b5c-478b-0bdf-08d9775e8282
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233535318B326EE760374439B3DA9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xqjmwAmz8zxviBjQ+6XNu+yWpToqzEsGPsfKn6WuW0Qp+ltxsteJN3TRZ2KZToARDrrvw9vzHwCIZVXRMQiGKh9+CRDwDP+aULKpqLeq66U0ZqoyVGe2xo3XYtlQIt2y+Q5miUKBth1CzhvwSqY3NNmhnYDm4ezFXjFnj5Ic5q8uAf/MAdqachxtFklFu+zS25xzR3OIh1zb3H9rtLSNa1Bls0sHzozTsaHCHp0wJdQsYZ+Bg1yJav0BCVsSd/YTxp2c50GMcZnr9o0lqF6haJ8lnD01Fbu+OJhHRM/cOqKMp+5/DMAkiwosbv1Q8J0PGO++sGi1ZNvxYs/vpOEMkzuAvGDUUPS/3LHh+Xyi0N7j8Mpzhdde6cXyQbo+3RBmx0V4UFGV1iQmNsdFR6OFc3Zv5CqBefHbwkBv8MKkqF8lyN56Sxbqmg46OZr7CIkZb+bK7082sDy8VfnBRRP+pdRm8xJadIIBNOonb8iz3wntaWfyg4JdrTOOawjEGJt57nQaZXmFLmKOAXgmwKPSyM2mKZ4XN43fLoQcnQMXBtpIL3hXJ4dkKslpG+rv0ARGFL1bDoYA82ATHuMzhPm83XjJyAA3hdqHhHeUdQhAde1+VKrT7Nx58UX5P7PjcykJOwfcNpLpkUmQgnVgSSC0wlwPhZoQJcX314/u4DCUssDrLfjTHsTSYhkwusrFaRrc67MHW84jReXLEpRvBv1GIOs5avzyEcadJusVIiLC9CM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(136003)(346002)(39860400002)(396003)(31686004)(6916009)(31696002)(38100700002)(53546011)(478600001)(26005)(6486002)(36756003)(186003)(8936002)(8676002)(2906002)(4326008)(5660300002)(83380400001)(86362001)(66946007)(66476007)(66556008)(54906003)(2616005)(316002)(16576012)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?FiLW94DEoSHlznonFQyTTdlUgDUGoucSBqdmXqOuXmhnr3IHzdBPval155kI?=
 =?us-ascii?Q?af9JZZfoZ5oA1bB8bNm5GWMJg2b6PaQB+TlMi8bC6ERqlumnBliPMIuMpWrl?=
 =?us-ascii?Q?SKKRftvSmM+9jaRWu7YyGyLb/9ZD47szaU5bF4002eQL/7wRH+dMN/iDIuCw?=
 =?us-ascii?Q?7srqW0zpC7iJxjR1LB/Oic1qda6lXpzRJv/eWjzE0aiTPYH/bXQ8H1OGejFT?=
 =?us-ascii?Q?0x3bYVjagkkx91YwDKgRlhqjhlfKNkew1cPMp0Fq11GWR19m/b63hQ/u6IJV?=
 =?us-ascii?Q?CDHjrVrnPpGbsTJOpcVZi9y1FBknuH9X5yUpbnp85k1/GFsX8cwegZhim16P?=
 =?us-ascii?Q?FJjQWPzRmUrKrMttBdyxPmxMbo7VIi2DBRNRLAFrzj1gTTIdfNCDcIKuwJGd?=
 =?us-ascii?Q?dqVShsIG0fwGU2qad1eQ5R3SgW+a54QBADDGrXDikizEoHNSUm8Zo2C2UZa2?=
 =?us-ascii?Q?hvxfTw/Ffi/08u5jIVZ/g6tv4bYLXkW9id/iA15EWgviXqY2G+hmaY04r0Lu?=
 =?us-ascii?Q?q73avxZrbQK8GaDzkH0Cgt67Z+PrTDMfuIexP93XWr0elexT4EoGGv31y04B?=
 =?us-ascii?Q?4atYsEzNQi35OcrXtFxA9jWnCjP4tWQe/Vrag4yoggIVEJrkfr+3LZwpDicz?=
 =?us-ascii?Q?h1OhiU3l2r94UFyX60iCpdVmDPPZTZLePRf9Jh4zzHd/PVbmQPZgabPB20K2?=
 =?us-ascii?Q?1m2phHlck6u8FDBgkIQwhhQtjbbZJBsFIUyZDd80WKORub8cNa1+4wvqDe9k?=
 =?us-ascii?Q?+RZLtka46T02F0+IeZ9XhQL4kUIDy1IXZd+1xndOXYXao6+u8z2/eFrOcxx9?=
 =?us-ascii?Q?kznee/CtHBs2Gb6nw1guaiR4FEHrZ+b2LmpIopPxz3FUFgl/k+ha97JsI/Yk?=
 =?us-ascii?Q?CEFWxI/Y5Xm/Z7DpLt94AZvLpGDpSU4uT30qy+tXFqxIkpQiHgYnXvz+mRqP?=
 =?us-ascii?Q?NJA1ql2pJ5owvr+ueajSmdyY8MolEGvqrpRZkf11bWt9fvzMUh5y6Ga04+0A?=
 =?us-ascii?Q?Fk+0PeNSvVq9OlCBvYosOqjJoGsKVLrsU+QGDSrOn2MgLbqhf1H3B7mAdjJb?=
 =?us-ascii?Q?pOyqQfc0Xhf+obpNtNdq3NQdHE5y43mTSyxhpQ3Jym6BLkTijp7n0cZSYLhq?=
 =?us-ascii?Q?hNSZyV3skNJJrCno7SDoYUSUwBCTR+p+3C0jN+NnfpMQlbf9ltytYtRlfoRv?=
 =?us-ascii?Q?wLE0qDtDfFzSxmb07nDhL7n9cznW9sdwBAg14ZIEMba3vfHbrQUt/0MeFOgw?=
 =?us-ascii?Q?FfNzdVcss1R4vYTZADAod9XlJo7s12XIf7A4aRROlRNB1+zDHG18Vcbladlh?=
 =?us-ascii?Q?gi35TDoOjFF3OZseBguhO3Kr?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6b263a4c-9b5c-478b-0bdf-08d9775e8282
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 09:03:25.1247
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lRx+2laN24xk1SqO72Ez6FFFP0qvuRgkFeYObezdwhQNN2Lh4NOm45tg8oZ2T9gJva3eLKae9KiqGOJo9k595g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 14.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
>> In order to try to debug hypervisor side breakage from XSA-378 I found
>> myself urged to finally give PVH Dom0 a try. Sadly things didn't work
>> quite as expected. In the course of investigating these issues I actuall=
y
>> spotted one piece of PV Dom0 breakage as well, a fix for which is also
>> included here.
>>
>> There are two immediate remaining issues (also mentioned in affected
>> patches):
>>
>> 1) It is not clear to me how PCI device reporting is to work. PV Dom0
>>    reports devices as they're discovered, including ones the hypervisor
>>    may not have been able to discover itself (ones on segments other
>>    than 0 or hotplugged ones). The respective hypercall, however, is
>>    inaccessible to PVH Dom0. Depending on the answer to this, either
>>    the hypervisor will need changing (to permit the call) or patch 2
>>    here will need further refinement.
>=20
> I would rather prefer if we could limit the hypercall usage to only
> report hotplugged segments to Xen. Then Xen would have to scan the
> segment when reported and add any devices found.
>=20
> Such hypercall must be used before dom0 tries to access any device, as
> otherwise the BARs won't be mapped in the second stage translation and
> the traps for the MCFG area won't be setup either.

This might work if hotplugging would only ever be of segments, and not
of individual devices. Yet the latter is, I think, a common case (as
far as hotplugging itself is "common").

Also don't forget about SR-IOV VFs - they would typically not be there
when booting. They would materialize when the PF driver initializes
the device. This is, I think, something that can be dealt with by
intercepting writes to the SR-IOV capability. But I wonder whether
there might be other cases where devices become "visible" only while
the Dom0 kernel is already running.

>> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
>>    console) when in a non-default mode (i.e. not 80x25 text), as the
>>    necessary information (in particular about VESA-bases LFB modes) is
>>    not communicated. On the hypervisor side this looks like deliberate
>>    behavior, but it is unclear to me what the intentions were towards
>>    an alternative model. (X may be able to access the screen depending
>>    on whether it has a suitable driver besides the presently unusable
>>    /dev/fb<N> based one.)
>=20
> I had to admit most of my boxes are headless servers, albeit I have
> one NUC I can use to test gfx stuff, so I don't really use gfx output
> with Xen.
>=20
> As I understand such information is fetched from the BIOS and passed
> into Xen, which should then hand it over to the dom0 kernel?

That's how PV Dom0 learns of the information, yes. See
fill_console_start_info(). (I'm in the process of eliminating the
need for some of the "fetch from BIOS" in Xen right now, but that's
not going to get us as far as being able to delete that code, no
matter how much in particular Andrew would like that to happen.)

> I guess the only way for Linux dom0 kernel to fetch that information
> would be to emulate the BIOS or drop into realmode and issue the BIOS
> calls?

Native Linux gets this information passed from the boot loader, I think
(except in the EFI case, as per below).

> Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
> info using the PV EFI interface?

There it's EFI boot services functions which can be invoked before
leaving boot services (in the native case). Aiui the PVH entry point
lives logically past any EFI boot services interaction, and hence
using them is not an option (if there was EFI firmware present in Dom0
in the first place, which I consider difficult all by itself - this
can't be the physical system's firmware, but I also don't see where
virtual firmware would be taken from).

There is no PV EFI interface to obtain video information. With the
needed information getting passed via start_info, PV has no need for
such, and I would be hesitant to add a fundamentally redundant
interface for PVH. The more that the information needed isn't EFI-
specific at all.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 09:11:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 09:11:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186331.335053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4TC-0000Yx-HV; Tue, 14 Sep 2021 09:11:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186331.335053; Tue, 14 Sep 2021 09:11:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4TC-0000Yq-DR; Tue, 14 Sep 2021 09:11:14 +0000
Received: by outflank-mailman (input) for mailman id 186331;
 Tue, 14 Sep 2021 09:11:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ4TB-0000Yg-1x
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 09:11:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fd0a933d-9d45-4c7a-a713-3abea8683d1d;
 Tue, 14 Sep 2021 09:11:11 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2057.outbound.protection.outlook.com [104.47.2.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-ENN272z4NSqmdlIA60AU3Q-1; Tue, 14 Sep 2021 11:11:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2448.eurprd04.prod.outlook.com (2603:10a6:800:52::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 09:11:08 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 09:11:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0005.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 09:11:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd0a933d-9d45-4c7a-a713-3abea8683d1d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631610670;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lRSbwAvJszhlt1oSYrz+SNQDPj7Z+tBJanyHUCx4ZNo=;
	b=jU9ubkUXp17p/Ayb1fik88U/IDMnO1EqZOopEdmpD3JNkj0Cw0lu3Wn3SAOrMf2MIySuT1
	qFRKcHFDQa0Cdh4M5VLdorNKDzXZl2xiUL9HZNsRzUCiZ+Wdb3bhl8PZfr/nCX4M1gK96N
	i3tVB2LXeYf1M7+LYGgXxgvgJAWDhco=
X-MC-Unique: ENN272z4NSqmdlIA60AU3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=J59eIivuc4qVroPkdUYDq79upH69aDnlR4vVkunpz1oEFMMcPUP0jQtHyfqKId8tZKAElBfkyqPcdV1ACP55XgdBY6Xg2Q9S4Rss/7TZv1L1k8iSxG255U2D//M47nsAqyvHSHx1+/8OYkpe74ZrUGFnlUDv1srkN88mBaMwZlEIlr5Bcvq7wHTLpBiIw5X3fxR8f3/6cYhe20VZEcUlIamSQJyor3pJa+3HHlI/LHozed5uEg4Ab2rcYqLEFGianRIBJ/6m2UZw5PADZaMxFiSgaLO4LXn++wXxLFKvmPH55+PCnoGZS2LRkYEEJ49Ze7d8zg2EbEqWO9Gia7GKzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lRSbwAvJszhlt1oSYrz+SNQDPj7Z+tBJanyHUCx4ZNo=;
 b=Np6siSC8NP7aoig05HnzItV2+UttGQ/N3WCuTXyrEx1TY3mgl4BR+4snqqTlNkFbUsa9wbNVmJrBqRHYLv3oEswFu7sbrGYSGqdgMHo1hSktfww5BujwsWkY2o9B6QEb12YcpfMyCuUANdOXw+LyopggN4BtyuEGkEQ4XzIngAnQYtfuc+H0MSp+SLix+DTSfryqw0NNciNxJcgE39Tu7qm2+Xgs4MirLYXCOMJ5BR9YeESH5TpbWmHSPq51CIz3BlCrE2GZcOXYSitCP3UUQQBmT4iABGguDhAN6BudijT2NPHhrZiZ9J8yiO6oAvOTh2koY9dmKk6UCrsn78eqMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
 <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s>
 <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com>
 <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9819a6e9-93d5-e62a-7b4a-ffc2ecd996dc@suse.com>
Date: Tue, 14 Sep 2021 11:11:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0005.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::17)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 635036e4-15b6-4be4-ced3-08d9775f965e
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2448:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24483B96DD16067785BF7C29B3DA9@VI1PR0401MB2448.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Xr1L8G4nsHsYGVO3yL71gJWvaOPRLZL54uPfjvJXE7L9p6CkeFSR658oMuVsIrIcdqIWwjrVxp+O1HCssOKPXqrTiq3bd66lwrKLMeqqeL4kMu2aPU14aNulvUX9V5rV5U6WbvQctSNTRTIjvBqha+DLkMy9KkXxjnLTVah35ZvfnVw8F5mXitrfq30Oqj4uD0VnfhGVdCHlZ4TEVTQDLC8mU9vAMzAoaJXakVP/EoW77XC40abeLS/8EKJUe0x1bkv00NrYrvrheiyl7x6U02+Gi5s90sB9IycHbxFnCeM0qL0ITVCmD5UTiWDIbt0LRkzk/X3qpFU+Qe10SgyFhU1MI7Siq6OywZUn71UPwP5XMrGl0VpEyMMF8ypWJEU4a/4nOQdjm4Fa+STRQUHhjvikzarfKxa3WeuXq9I8wo2Kh45yQMGw9A88fccH1XDCIlG0d5k/9wYW5x4+PCiqxEDwdnuosyj14FqpiMmRWvRFQvMrAqZZdhp7tpUpQIyXFu32PFv4xj7Qn+H7yzzOuqk5rOI4fL7GRgb5K/Pv+9kL1jDS6qkbJckNxl3L3J7I4qH8bloviLusuNcYbvfifSiCzGTF+vSW7Z8wVOKab4ZGwEUc2OwNNag3Br50gFVjzgF9tzGGvcaJojzWmNhVZxWMMVWgWfz/o4SUqbgUcWBVokqSKeeHh69ZRHgY6SnPiifLD8zxxth2kuwYhb+4ursUIZz3YD9CajWuXpen7H1NpRz64LGisnP3LA9btr+g
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(376002)(366004)(396003)(136003)(39860400002)(66946007)(66476007)(86362001)(66556008)(53546011)(31686004)(36756003)(26005)(478600001)(83380400001)(5660300002)(6486002)(956004)(2616005)(316002)(16576012)(8936002)(2906002)(54906003)(38100700002)(186003)(6916009)(8676002)(4326008)(31696002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aHIwQUpyRUZUUnRlbldMVTJGaUQ1aXhoMExXbXZjU294NmdHdzRVMk4wNmZN?=
 =?utf-8?B?Y3RrUGlpRHo1N1AzMU5iKzNsSXFhT0lDZDU1ak5SeGFHSkpQSWR0ZS9Bc0ha?=
 =?utf-8?B?Yk9KSlIxMFJlVXhmdGxudjRrWWlLU1lqNGlkWHFYZVlOQ0dvZmYwNGo2Tkdq?=
 =?utf-8?B?TUJRay9iSklVVzBPZjhhMWVaNnMvNzlOSURReHdZN0JVVmFRWGdyNWdaRlR1?=
 =?utf-8?B?djRIZmVSbE55QzVlWi9TV2puOTFnWVdlZXk1bnVYamdVUWJyS1JkV3RsN0tT?=
 =?utf-8?B?OUpyNWZ3ZCs3ZysvU1BjbjQ4VndOKzBSNDJzS2p0NDFQd3l6d3NHUmhMQURi?=
 =?utf-8?B?YWRCREtPWmRZTTdjOFNYeWlYNlc4MnB4SXNuYmF0b0xPRE1UbDRwaWNsMWFi?=
 =?utf-8?B?YkVxZXlxaGUvOWEyaGM3TGQxdDV6UWdvbURrTmhpaThxUVY2ZTdnMFpZcllv?=
 =?utf-8?B?YVp1anN3bjJWYWJaSTRhdDZaTHVSWVVMcjNDSW8vMVkyWmQyMFFjTmwzTmw1?=
 =?utf-8?B?VzV6M3pQMHlWaUs5cE1yUGFKaVNlL0lnMGpaOHUwemhKMEgvb0h6dDBnalBJ?=
 =?utf-8?B?S3FtWk95ZTNkOFNHVCtZdHN1RmFNQzRYek5ZYVQ2MEs0R2F3bkVRWU5QeU55?=
 =?utf-8?B?c3pTd0pRVkQ1eWpJYlhSdXZZUTlWZkYzZlNscmFUREtETWhTdS84ZUtGWW0x?=
 =?utf-8?B?OVA5OHE5OXAwbUlIeXZPbDRMR0xqU2pYUE5iMDJkN1VLdURTQ3ZvL2I4RFBV?=
 =?utf-8?B?VnY2UktlcGNyUWFjaEF1d2xHZEtPRGVYemF0RlExalRhc2xNNi9hZnNqcG1h?=
 =?utf-8?B?MThSREhNMzl3UEwzYkViay94OXA3TU5vS1ExQkVpM1prYjZvWFJLalM5Y05h?=
 =?utf-8?B?RlNlUmN3bjNuYVFINXZjSkFiclozU3FhTTJ0ZE5GMVllbU1XUXZYdHFGZnk2?=
 =?utf-8?B?U1V6Nnpzd0NEVUhEc0F5clVaSzBIUGl0OGdpeTVQL1F3NVZHWnN6VmVBM29W?=
 =?utf-8?B?blhkUThtWE1rTlRIWXRzMEZtRG5EZTA0OEYxTjd4NjFSSjlaMUJjVHJTVmdj?=
 =?utf-8?B?ZWRJTDVuVFdCakRaMFkxbjZuNDJSUjdvQkVFUkswbzJ2QkdxTFFvOXFCTHIr?=
 =?utf-8?B?YVA5ZURKY2phU0V4SWw4YjROME1tSVpUUnd4M2I4bk0xamErTS82WU5WWXJa?=
 =?utf-8?B?RUx3b24xVkRKdXNJak5JSlJCSHE3NVI2dFd0WHp2aDV2ZW1HRDJRSGJjMDJV?=
 =?utf-8?B?MkU3MXFTWWw1aldsdkFZdFNZZE9OL3ZHaExhcXZTV2tiMndVMDVTbWZ0TWJU?=
 =?utf-8?B?VFFuS0dOczA1YXZ0ODEwQ20yVkg1VEFPTWdpSnBVZE5YUWdWSnlvcXN4NGI3?=
 =?utf-8?B?TC9rcVJtL1V5UEl3Y0s3aThNYkJjRVhsRkc5d2hleWhjQTZhTGZybVoxYnNF?=
 =?utf-8?B?RU5YV050Mi9ZYjJraEdkOEdrczM0UzVTaUJBYlNqSXlRcy9XTlhic296YWNI?=
 =?utf-8?B?dHNrZjkvOUJ6YmpscXRjZ05jMjR2aldvTlE4eVJYZ0lJQTFMSUJJYk1XSFJS?=
 =?utf-8?B?OHQrQ0I1blMvQ0s4eVBmcSttZk54WEh2ZHBxOUNIMVBpZ3RHRGtxQlV0UStE?=
 =?utf-8?B?eUNxL0UwQ2lCRzV5ZjJSN0VWNlhVTnJaUGhYNmRVTjMzendtWU9MY2x6ZElK?=
 =?utf-8?B?MkIxdE41eE40NUoxZEE0a0ZReUdRNXY4NzZJbW9EN2d5UDc3Q2R0aTZ3Yk40?=
 =?utf-8?Q?DuFGocYpke4supAyxyH9tMmtC65RbjxyO32he0K?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 635036e4-15b6-4be4-ced3-08d9775f965e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 09:11:07.9988
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OMRLpRtWPy9GhjJXRo7Xkgtazw7iE657sXD/ZbVbG3Lf7PAbBZ473CAGJb7MzgNM6bYOMfj2uvjAVEqseCYS3Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2448

On 13.09.2021 22:31, Stefano Stabellini wrote:
> On Mon, 13 Sep 2021, Jan Beulich wrote:
>> On 11.09.2021 01:14, Stefano Stabellini wrote:
>>> On Tue, 7 Sep 2021, Jan Beulich wrote:
>>>> While the hypervisor hasn't been enforcing this, we would still better
>>>> avoid issuing requests with GFNs not aligned to the requested order.
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>> ---
>>>> I wonder how useful it is to include the alignment in the panic()
>>>> message.
>>>
>>> Not very useful given that it is static. I don't mind either way but you
>>> can go ahead and remove it if you prefer (and it would make the line
>>> shorter.)
>>>
>>>
>>>> I further wonder how useful it is to wrap "bytes" in
>>>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
>>>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
>>>> properly").
>>>
>>> This one I would keep, to make sure to print out the same amount passed
>>> to memblock_alloc.
>>
>> Oh - if I was to drop it from the printk(), I would have been meaning to
>> also drop it there. If it's useless, then it's useless everywhere.
> 
> That's fine too

Thanks, I'll see about dropping that then.

Another Arm-related question has occurred to me: Do you actually
mind the higher-than-necessary alignment there? If so, a per-arch
definition of the needed alignment would need introducing. Maybe
that could default to PAGE_SIZE, allowing Arm and alike to get away
without explicitly specifying a value ...

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 09:27:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 09:27:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186340.335064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4iL-0002Bb-SP; Tue, 14 Sep 2021 09:26:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186340.335064; Tue, 14 Sep 2021 09:26:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4iL-0002BU-Ov; Tue, 14 Sep 2021 09:26:53 +0000
Received: by outflank-mailman (input) for mailman id 186340;
 Tue, 14 Sep 2021 09:26:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ4iK-0002BK-2I; Tue, 14 Sep 2021 09:26:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ4iJ-0001XX-RR; Tue, 14 Sep 2021 09:26:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ4iJ-00050B-Gt; Tue, 14 Sep 2021 09:26:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ4iJ-0003ig-GO; Tue, 14 Sep 2021 09:26:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+OxctLPZJPOHB1gCL1LgfgNSs4gNUTQcF3ZghXXTJ5I=; b=45hk5JjvTvwZZ/sWUsuH98Fa/R
	O9tF1z1m6ep5Qo1RBX+vNFGn6i3t1q1brHTLUBILpL4V5c92mjsK+H9D4yH7W90Sm0n2yx7AtAHBt
	vbd6iEACEpr+SjZT3OzZ2S3S9u2bfbvghmKkZYbA4JXZm6WSwFZRnJo5PodWJ3M2qvxk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164973-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164973: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=7d79344d4fa44e520e6e89f8fed9a27d3d554a9b
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 09:26:51 +0000

flight 164973 qemu-mainline real [real]
flight 164979 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164973/
http://logs.test-lab.xenproject.org/osstest/logs/164979/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                7d79344d4fa44e520e6e89f8fed9a27d3d554a9b
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    2 days
Failing since        164967  2021-09-13 13:06:52 Z    0 days    2 attempts
Testing same since   164973  2021-09-13 23:09:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>
  Lara Lazier <laramglazier@gmail.com>
  Markus Armbruster <armbru@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 461 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 09:41:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 09:41:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186351.335077 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4wG-0004jq-C0; Tue, 14 Sep 2021 09:41:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186351.335077; Tue, 14 Sep 2021 09:41:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ4wG-0004jj-99; Tue, 14 Sep 2021 09:41:16 +0000
Received: by outflank-mailman (input) for mailman id 186351;
 Tue, 14 Sep 2021 09:41:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ4wF-0004jc-J6
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 09:41:15 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b130ae0-b80e-4de6-90fe-18fb070c4870;
 Tue, 14 Sep 2021 09:41:13 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id AF5EA220CC;
 Tue, 14 Sep 2021 09:41:12 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 61C0313E55;
 Tue, 14 Sep 2021 09:41:12 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 1VE8FjhuQGFuSwAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 09:41:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b130ae0-b80e-4de6-90fe-18fb070c4870
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631612472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=30u9QxljtHQChxZ06/YlGhJs1iq5r8Bx3qVFUI1IaBI=;
	b=iyMgnQgHJFFJjFjFAPR5PynTg/uuBNPUBW7HSkJDvAHUqb2vI8NVwufZ9B8pw1W8RwbV21
	sY9ZPbqzyYgOU8rd59OHmzpkABuHi4r4SbS2XZaPS0OidXTQJHOpHEsK3oqq72VAHugVPv
	X+5HfzQQuA8wHQ0cTH5y8G4eeOVFOdw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	stable@vger.kernel.org,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH] x86/setup: call early_reserve_memory() earlier
Date: Tue, 14 Sep 2021 11:41:08 +0200
Message-Id: <20210914094108.22482-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
reservations") introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately the
call of early_reserve_memory() is done too late for Xen dom0, as in
some cases a Xen hook called by e820__memory_setup() will need those
memory reservations to have happened already.

Move the call of early_reserve_memory() to the beginning of
setup_arch() in order to avoid such problems.

Cc: stable@vger.kernel.org
Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/kernel/setup.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f164141116..f369c51ec580 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 
 void __init setup_arch(char **cmdline_p)
 {
+	/*
+	 * Do some memory reservations *before* memory is added to
+	 * memblock, so memblock allocations won't overwrite it.
+	 * Do it after early param, so we could get (unlikely) panic from
+	 * serial.
+	 *
+	 * After this point everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not
+	 * RAM in e820. All other memory is free game.
+	 */
+	early_reserve_memory();
+
 #ifdef CONFIG_X86_32
 	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
 
@@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:00:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:00:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186363.335088 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5F4-0007MS-Uf; Tue, 14 Sep 2021 10:00:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186363.335088; Tue, 14 Sep 2021 10:00:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5F4-0007ML-Rh; Tue, 14 Sep 2021 10:00:42 +0000
Received: by outflank-mailman (input) for mailman id 186363;
 Tue, 14 Sep 2021 10:00:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ5F3-0007MF-HI
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:00:41 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d251b8e5-a86a-466e-8d31-0166cb5a04ec;
 Tue, 14 Sep 2021 10:00:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d251b8e5-a86a-466e-8d31-0166cb5a04ec
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631613639;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=F9sPTkw0mJCRNPcVyWrmjj75j6YyGRmrGKs1xZxnygA=;
  b=TGq2PwOuNc3xYTgG/ukkkH/1ljMUpkCgavNfRVQ2DDkLXvR4C1e1EF+r
   ZxEl93WdvvZ9XbdRzfBBHMeTMWKR3FNcvXA5sy8MghFbDFSbgyiBM6Kg7
   nLj8CiahJzMnwEZEdsoGx1NdXt+6RcjcxNk0wfdv86Ow0g9/csZkSk4uQ
   s=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: /Q/3amSaS/fBh/unwhtU8GMhDcF8NNutRKzimd20FSwjthReyMxynzEJTeREIp7pyyxGWIg6HU
 2YxnhvmcwSfi8IUoXEIezP5VPbu7nyEwymDOf+2MZMIsllQ9VAE3L3oXGqaZ2X/d6/ZBOWqYrT
 ZsTesiXxa8+gHi6jGZXyE5uM2kFFbcocTkjMkpCA7SsYjw/UeT1Y4COAdgO+e3pp+ROCiMKPZE
 DHD6423+EjMPbwK+YKA+877HOEzjU0LF4Mjh7Vm/Eorz5564D+uCUisL3i5m6ePcBeAwfIFzhz
 0fCUYWPJ2fv56kVKosg13DqL
X-SBRS: 5.1
X-MesageID: 52677731
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fgxo5K1/MasCEyzPaPbD5dl2kn2cJEfYwER7XKvMYLTBsI5bpzVSm
 GIYWTrTMv2PNmb3eogkOY7ip09VvpKAx9RqHVBlpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrJh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhko1y2
 NQKvpGMRFkzDIHrtcdAV15AOnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIUGg2hv3Zkm8fD2X
 uBAaxp1Sir5X0MfH0kUDZRmuMmIvyyqG9FfgA3M/vdmi4TJ9yRh3b6oPNfLd9iiQcROgl3et
 m/A537+ABwRKJqY0zXt2nCmi/LLnCj7cJkPD7D+/flv6GB/3URKVkdQDwHi56Dk1AjuALqzN
 nD45AIj8Ksi9QugQ+XwAQeKjFmpnUMYHMJ5RrhSBB629kbE3+qILjFaFWcbM4F36JBeqS8Cj
 QDSzoiwbdB7mPjMEyvMqO3Mxd+nEXVNdQc/iTk4oRzpCjUJiLo6iA7GBv1nGbS85jEeMWCtm
 23WxMTSarN6sCLq60lZ1Quc695PjsKQJuLQ2ukwdjj5hj6VnKb/O+SVBaHztJ6s1rp1q2W8U
 IUswZDCvIji8q1hZATSGb5QTdlFFt6uMSHGgE4HIqTNAw+Foib5FagJuWkWDB4wbq4sJG+4C
 GeO6Fg5zMICYxOXgVpfPtvZ5zIClvO7S7wIl5n8M7JzX3SGXFTbpXo/ORHJhDmFfYpFufhXB
 Kp3uP2EVB4yIa9m0CC3V6Ea17oqzTo53mTdWdbwyBHP7FZUTCT9pW4tPATcY+YnwrmDpQmJo
 d9TO9HTk0dUUfHkYzmR+okWdAhYIX8+DJHwispWau/cfVY2RDB/U6fckeE7Zohov6VJjeOUr
 HuzbVBVlQjkjnrdJATUNn06MOHzXYxypG4QNDA3OQr6wGAqZIuitf9Nd5Y+cbQ92vZkyPp4E
 6sMd8maW6wdQTXb4TUNK5L6qdU6JhisgAuPOQujYSQ+IME8F1CYpIe8c1K2picUDyexuc8vm
 JGa11vWEcgZWgBvLMfKc/bznVm/imcQxbBpVEzSL9gNJEi1qNp2Kzb8h+McKt0XLUmR3SOT0
 gubDEtKpeTJpINpotDFibrd8tWsGup6WEFbA3Pa/fC9MiyDpjivxopJUeCpezHBVTyrpPX+N
 LsNl/ysYucamFtqspZnF+c5xK0z0NLjurtGw1k2B37MdVmqVuttL3TuMROjbUGRKmu1YTeLZ
 38=
IronPort-HdrOrdr: A9a23:46ZM7K2WVtoCWEMJ4TUOtwqjBVRyeYIsimQD101hICG9Lfb3qy
 n+ppsmPEHP5Ar5OEtBpTiBUJPwJ0800aQFnLX5XI3SJjUO3VHIEGgM1/qG/9SNIVybygcZ79
 YdT0EcMqyAMbEZt7eD3ODQKb9Jq7PrgcPY55av854ud3AQV0gJ1XYGNu/xKDwOeOApP+tfKH
 LKjfA32AZINE5nJPiTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1SvV
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfpWoCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8DLeiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6NpuTgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeQv003MwmMm9yUkqp/FWGmLeXLzEO91a9Mwc/U/WuonhrdCsT9Tpd+CQd9k1wgq7VBaM0oN
 gsCZ4Y5o2mePVmGp6VNN1xMvdfNVa9NC4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52677731"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nGuqkWfNwPm1HqBrYFEN7LA4iuC+8ydsZy/Fq6sQV5KT0bu7KXGTDpok22yllty3yJFe1hbBD9AcpPgU4E/jwzBdOQnVppIBSdSBMfIseK6MvSWp4XyL1kp5D/s05f7JBzuGEtcteYIz0HmGxuPqDLG0Gjc9lwfkhEe/42nDCRI1BpaRLAcBTtxW80pig6GTOPfpVeeE03tFXzZ1O6JIaYRaGJgV66NplCCuXC7NlnisBSrZQhlScglGIFn2DGS5ucF+Mp11bKT8uXmfZXaRVWccQ+wXEmuNDG/duoUMQy6Ba2sbJK4B/OxKTMu1+10080BOKjXJDR9xhfqodyZ65A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=G/nGGELEptmYClFcx/LRVXVpbY27RoS9J+An/txk4a0=;
 b=KdyVLNA1x87IfId71rOzAgH/b8Rnab4dnzkaB3iy2bTf+4IvsM+FcWhaRDmyk+EVYYwhAg8awg5vU+w3jp/eKK1ku9BBiq4BHnIf4eAAVwOiFZY55XfKHOpnO5To6LOb0uJJON1NIWk4dD+eQtVhnV9t9loXsvmbmEJtdRrD8D8PUwj7wnHTsYU/iDQLmjwWcqk7YWK++j27PW7gzFgUFsRIgCWBSNxsDMQfKkCPXju6k7efzONBYVcKWnu36Jf+/U/iToqt/htY88VE3DsSAx4LqSXLOlBPs3veWdWvGP4YBf9p4kwtqi2XhlCzbhK+NiOnJGemgd40DJgFdDcDOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G/nGGELEptmYClFcx/LRVXVpbY27RoS9J+An/txk4a0=;
 b=QXRIOo99inXkqHI0/94CdjkOLjH99S5E3/GROqc5pc3mDmqiTYM4p3jDmPmZ4Z/q04pafa+ARV0sDK+DgZWOWufkbDPDES0XwDrVT2GIug1Op06soZueTvpM/ccvbhokJrYIZXPzU0oG9OpYc3RBbr2NscSCwPH+K9nCTIbJmMc=
Date: Tue, 14 Sep 2021 12:00:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
Message-ID: <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
X-ClientProxiedBy: LO2P265CA0368.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a3::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b3b073d-0636-4e5e-536d-08d977667ece
X-MS-TrafficTypeDiagnostic: DM6PR03MB3484:
X-Microsoft-Antispam-PRVS: <DM6PR03MB34843E26BA3956ACA7FE3A808FDA9@DM6PR03MB3484.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: wdGmu3mD4wF78v3CZ5OIVSPp7UoK5iRsJzd6cYUoM4WnLTx/f7iVAJDQeu4E1OwDM+/ZgSuO513lg6pDkyR59PhRK+7h8MMnhX7QNr0ccwGtkLxLHlyaP0pjsw4FeEZrEFu8D+1GhAEJQEGST7oLxBb2u2mdw2uYwf3lY1X1KLgTUVyRTn3BWwIQM5rtp+ocT8tJLFsh6P6vUg79A6e0YNX3Tc3vvofvX7zE/BMlZlwM7/SYAx7qKEhbKq0LgIu74uAFaZHgJAd+1yT9VBFjJYKYNIpCnVBQ+0vjLquE7DVjOR/ar0NXXXUaNFN+eQDe7HrZr2HQEcy4agWg+jzhHijYR8f/XtxDLczPJHYwOAY3i/cfA15I8+RWW/knYTcPoorJHCodtP/sjbeXom2g+v+y0m9Rcf4QGuOnYJUjhTcqEWuGQ/t32gVu/xaYukcrQYi04gevxDUCjuwtkk1oFfYjAmCa9Uv6s5QIdcodU7bfXMj8RLRgO57sojpP/gkb+SflUdyDSR9KNtK76UeSAaTohgrPHBLim/QocgWhAHzOzd8DoElnjLptpT8Z0yu6QY4X0sInESxOwzQY4cppFwKz5ESbBouMqSoPZOKQ8WMQHhz7FgWMmkc+dnQxdJvcyFl9UDoy6z1vxcnMqj9RJyRH+nP8lJBmARVreM5OZIopL1oJlTC8nkhkr6yCOppCROLGefmr/woP+uBnEXGnVw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(83380400001)(26005)(478600001)(9686003)(86362001)(38100700002)(186003)(316002)(5660300002)(6916009)(66476007)(8676002)(6486002)(15650500001)(4326008)(85182001)(66556008)(6666004)(66946007)(8936002)(6496006)(956004)(2906002)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Znc0WHFadWdBdWh1c21Tdk53SlZrWUlmSGo2WnlIYXI2NXhaOFErZTBPY3Uy?=
 =?utf-8?B?Q2tFUTZOcy9tdHRVNGszdjV4b0R5bUpzdGlpUXp5bk4rL05BUlpwOGpGR3Qx?=
 =?utf-8?B?ajlJMlU0R2c5QXlDTXZjZEx5T3hsWis0WXFSVVh3dkgza3FKNURqd1hlVGFl?=
 =?utf-8?B?Skg1bkNSbktubG1BMVpNYS9FM25CUVFpWmZxK2I5VGFCeTRrYzV1RTdCOFRu?=
 =?utf-8?B?V0F6ZWpjcm5jOGs0YUMrR0ZaYWtVVlB5c3hJYXZtVm9KMHBWamVscHpqSDZQ?=
 =?utf-8?B?ekpKKzJwOFZNY2hrYTZtZm9uYUdqNzNqaVFvMmdjaEp0NUZPQ3RCQ2NUTjlJ?=
 =?utf-8?B?VXhyYy9rNGRSNDJBeWR0MGRRbFpKRUJwZXZtT1pza0doWDRNOS9kUkY0MlJu?=
 =?utf-8?B?S0hIcWFJM0lIL1h2WlNPbEpndWZMQkNINDZWTjhYSTN1R1d0Nmk1d1hONnJB?=
 =?utf-8?B?WXJJMWxINEtSL0tmcWxYK0RxeGVwNWdSYkVzRDYra0N1bCt0Nmg5YnBHVDlO?=
 =?utf-8?B?QlRVZUNGbWpJTEFFdjlISlNVTDVCTE9mR2p4dlNtQ0U1TTVQVWJpck1IbFFM?=
 =?utf-8?B?VzJNN2pWM2tTcmhmK0VpUW9rS0MxT0R4V0U5VFNpcmlLTS90RE0zUkFXcmJC?=
 =?utf-8?B?eVVLeXkya2VrN2orbHVBSmt6cmk2YmJyekI4OE1iOVJnY2VjSXovZUhwNjdx?=
 =?utf-8?B?MStPK0swN3d1K0JjSklYVjNwTnBzUDJTU29qenljS21hU2RQTjRKK1pTdSt1?=
 =?utf-8?B?YzV6M0p5NUtCS0poN1NMc0tOTDB5dmZxT2Z2WEFSQTI2dElvMmlaUkk0UUlS?=
 =?utf-8?B?bTh5a05KaEsxbVQ4SFNuUjlQNkcvdXhRc3JCeEl5My9QbGh6eHV5NzBLZi9m?=
 =?utf-8?B?VXFtWFNXWGFubVB3S3ZZMW1qSVJMTGNxZUlTTyswWncxdjcyd3lya1pjb1hj?=
 =?utf-8?B?bHJjZ0Mwc1JjMmZld0tyWFRKRkZCWjNqTkNReWZ5N0J1ZXpqRzA0UUlNRUVi?=
 =?utf-8?B?OGRyTURzWVlRclFzSU10SXZMZmltQ21aOEFZZXk2TmtDalozdDJUSnljWGJj?=
 =?utf-8?B?RUY4am5yRUxLZTlzR2U4b3llbEhCMFZhNVc1cHdzd2JZVVRldmdXdkszMzBz?=
 =?utf-8?B?cHNJRVpHUDd0Sm04TUh4c01ZeWt4dDVET3NvbGI4ZTFWVWZnODhGZlBxYkVI?=
 =?utf-8?B?S2t3Q0RsMFhxL1paOGdyN2w4akRPNmhvUFRwUmFjRkFrd3VEKzhyUlBITTZP?=
 =?utf-8?B?bDV3bHo1dEU5ZHNMT2J6RVFyUDNiVEhacXN1R0JwMy9JVThIZDJlak5WVjN6?=
 =?utf-8?B?NXdQWEdrd2lFZmFvNWVJbFJVS0lZVlBYQzhsZzlLOVM3em52ZzVLN1FacjJV?=
 =?utf-8?B?Wm9QdVNxRldvajVKMFdDaU1zSFJjN3htUTBsYm1Qc1lwWUJKM1BuTTY2V1Vu?=
 =?utf-8?B?MGE5eVBQZTQwNUxxSExqR3BDanN2V29OUlFIRnEyTlh1OHB1VUhKUVdsRW5G?=
 =?utf-8?B?aEFhMmxSTlJhWDV3SDNTNG5vTHRmUjkxamRHZFdZejE1Y01Ha0JXR2s4VWhZ?=
 =?utf-8?B?NWUwazVHbG95cW5FUnVGcHVVZFZiNjFkcmZVMnhtM0FhZCtTSEdxMVhoWFZZ?=
 =?utf-8?B?RWxLTDNWT2lKRXdCQmM4YlFpOExtRXY3NTlXd3RNWUM2ekx3Rktwd05IL1NP?=
 =?utf-8?B?OXRyU1JzK2E4Rm5INmVDN2NGMjZ4b0JRYTRDR0t2L3Eyc1czeGFEWER2UHYz?=
 =?utf-8?Q?YkXh9MG901fFeoHaE9RBpIPaGNkFRZQ65Z3OBoP?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b3b073d-0636-4e5e-536d-08d977667ece
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 10:00:35.4176
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8vAFlwZdhsbDxnThxK2ZLvMWHLWFvnZsMi0nPIy9wIAX69MKkRQydFnOpHd3fUmWIjtpKI7QVVM/M0uY9p8vQw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB3484
X-OriginatorOrg: citrix.com

On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
> console) are associated with DomXEN, not Dom0. This means that while
> looking for overlapping BARs such devices cannot be found on Dom0's
> list of devices; DomXEN's list also needs to be scanned.

Thanks for looking into this, I certainly didn't take hidden devices
into account for vPCI dom0.

> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
>      would make the diff difficult to read. At this point I'd merely
>      like to gather input towards possible better approaches to solve
>      the issue (not the least because quite possibly there are further
>      places needing changing).

I have a couple of questions, AFAICT we currently hide the serial
console and/or the VGA adapter if it's in use by Xen.

I wonder whether we need to add vPCI handlers for those:
setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
device because of the temporary override of pdev->domain done in
_setup_hwdom_pci_devices, but I think that for hidden devices we
shouldn't add vPCI handlers. We should instead block PCI config space
access from dom0 to the device so that it doesn't mess with Xen usage.

It's also not clear why does Xen want to have those hidden devices
partly controlled by dom0, as it would seem to me that dom0 interfering
with hidden devices in use by Xen can only lead to trouble.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:03:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186373.335100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5Ha-0007yt-CN; Tue, 14 Sep 2021 10:03:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186373.335100; Tue, 14 Sep 2021 10:03:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5Ha-0007ym-90; Tue, 14 Sep 2021 10:03:18 +0000
Received: by outflank-mailman (input) for mailman id 186373;
 Tue, 14 Sep 2021 10:03:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ5HZ-0007yg-Ih
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:03:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f0d3d52-d74c-45bb-aaaa-33639998b12b;
 Tue, 14 Sep 2021 10:03:16 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-3TH0sG1pP_6Oj4xn8ib1sQ-1;
 Tue, 14 Sep 2021 12:03:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 10:03:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 10:03:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0115.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 10:03:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f0d3d52-d74c-45bb-aaaa-33639998b12b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631613795;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kbVzIVpBB61/hP9zrzJNjmoAVsNi4+VAMLBrf30cmuU=;
	b=bfrRYkcFfqMLZKUACVBEF7Ibb/9gG4R/OxGQjskQ9l5XBqrQrXsiBq/560ifeKyRdBcJaB
	ZDHJ/wDnI/xbCMx18FvbcTr9wtVUZf2LPf85eggrRg7KRcmTiGRYE8mJ9fP2o16oBfRWaT
	uK7jlLlCpFCjSLprpRERfKnlVuRpbic=
X-MC-Unique: 3TH0sG1pP_6Oj4xn8ib1sQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z3PySqpBAmR2HV4T55YbBaYLvt01cuer0C10+1K6byE8ABKYj8DZuWADoJmzqoAcTbDIYy0IqynZitRrinRjYfQg5ZjdSVMseY/e6x/5kHFOcZbgRsNsmHh9dvXKkhun4hThTyXNuBCHbixA+1YSA2GigosS1Gmv+s7Ct9z4MCKbJYmZUL3aZLIJUC8g8rI9Hhc1OJOTvBifsUtdeova5pzouq5jepELPrh0JtWP6Ip0+sQ4DNbROnI/Ypt/u5h7/nIZ68VwwlzJRCp3qDo88ps5wTRJUjFUkDRZL3zPIC8/dVzBs3IhIWSo0kc3/eL0FU0IxJynNRfdBfriW/MRGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EqBK0cRHouKDe5d/+3aETfwWbIfY1VeZMfnMmP8EH4U=;
 b=R+0ue9VshgzLTJZ73Bm5ZM8ZVkOgt+uO1xfp7CbHAnSlO98pYpR9x8K2YVZNhsUQ3k/CWahU+HjKDxCZdyHJl2xzauNGfQPmX85YgO3H5NPFRfQEylhH4l6mF01ahXnP33kd7kDbVouFAAK9W3m8Yfak42o5wB7Js7M0OJGCvQtP3uLK9ZqBOrc74BcWZ5ETUWa6tBfI0rTA80edjOzN7QydEg6uvIzYiK1UIf6te+q+veprMct7YbJQXJ5TGiF0Mxy6Nib7bY1XgL3sYrNf5cTuab0QX4ZGptHyL9IE3KC3sbxgpN3CiQA2ZeyIC4MRwWL0gx+HJ+2507B65bZk1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
To: Juergen Gross <jgross@suse.com>
CC: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel@lists.xenproject.org,
 x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210914094108.22482-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
Date: Tue, 14 Sep 2021 12:03:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210914094108.22482-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0115.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9f949dfa-944d-46d8-63f0-08d97766dc86
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68610DAC24243B423B90D33CB3DA9@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	AVHLEHqXXO7cD2boiJp4xDg0bKys7q9Ga4+yWTfBpr4RQxTKXqiwn9Q9tqttestWnsmDHDEw9mnNMI/agrQ8sXInuzwsjy5D1nmUWeXhAa1rWQzhgxLKDN96fN1PHSIcu9/nAvnJ4XAPEC9xQm9x2sMAOBR1nBo20KAk+E4Jr7Cd8S7QrI/2QfZKNDWiAeTREACVCLPV+z3jgTJ810N7EgUCnrAhXzjT+El53QUQgNinOxtjclQsNIy7cdd6643FNAK+qAPhFodpQGt2WhqijO614YbxDXCw4Z+L1OV7bImfbITYXaUr9HqHsl7HbBjdxyU9tkIiMVa3tZbVtRZUZxZ8jh9f0Bm/jjUpkW01FxcTiABvkmWiK5fBrnBNPpr/58mf0uoxa79QgUsgc8I/9LPH1F7XfiYkI1k/OXvwd2455fYd4FolGjLYMjrTmCbUylhRpaoe8KOvZpHvBgywV0JPMHPxJLIfwlsGYqDxvllR4gzEaJwBpUE1lTQ2JTLR9AiAumm/+khdV1JrPyf03IOb9R3q5hwy2xUU1q5bqYRT/HhWskQ9J2LG/Sqdn0mphZZ4PXGbZKAru0p78J834VsOlCMznHA5ZZTtUtA0/h2MFuYVb1UVYpwsq6hbM/83/qcFqcf/VVwmXwIJnQ7zdbDiUan64VOigCW+YY+Ssi6+Vb1ZxxGtf3FnYDO4iQQLp5qUYVwjtKCQmfm+SmcxlJrfwL8AltIw7Ot2zkBUlimi+l6pNZB4l+7KmBR8/KCq
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(136003)(366004)(346002)(39860400002)(478600001)(5660300002)(186003)(83380400001)(26005)(6636002)(2616005)(86362001)(8676002)(37006003)(36756003)(38100700002)(956004)(8936002)(31696002)(6486002)(2906002)(54906003)(53546011)(66574015)(316002)(16576012)(31686004)(66946007)(6862004)(66476007)(4326008)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?4bsuIGN23zE62g4XGv3OYz1WrkL/Tqx/3cZf+DvNaCIPQUtNyS/eKhlkHxLV?=
 =?us-ascii?Q?FvbMcvOAqEjfmRutQsoTIYBVka8y099ZfpV4+yBy829nImdN/ULjrA89rWpW?=
 =?us-ascii?Q?uKCeuJJxNPBvhX5ovbJzTWHwyRxUZV36PTlFvs6YKeXkevHNTZte7YdEdM7o?=
 =?us-ascii?Q?z6tqlofd3mI3TCVkL4tka72OS61i0sQTy2wXQjMTdkpi/pkpk5LjZDar/6RC?=
 =?us-ascii?Q?XS5p6tQhP8LJpA6ZNXCuV/v6fY1lDjotsYG7GtFHsDkj8+Xd0RqWbDtsCXOI?=
 =?us-ascii?Q?dSQVdTMZnMTmrCsNnK99EXhU3e4T3HJoOjnuxZYSTOfu4RWYOobvSl9CUjUl?=
 =?us-ascii?Q?KLvSHa2z6T0uXsASv2WlJ88yq9fYNwck+8PEUkCExZxnvRyX9P5Vb7EX6xJy?=
 =?us-ascii?Q?Lc2pySGcRCZQDYfRnDH6EJfaKxZY46V7OlGojttYpV9gXA4bDEHtJoYSKbH0?=
 =?us-ascii?Q?RenTCx5OeSO3y488gVRdywskIkklte7UYxYFmYUVOHwosSXlBX2yAL3WnQoZ?=
 =?us-ascii?Q?V7ijhJhE71fu1OQNH+UCkFYSg2ljiYkKBlEHdxWTOTGyn07v2zCbev5RwfvR?=
 =?us-ascii?Q?yuiPeP7FcAn96zddHXsuR5ol8qTa0Rwe6QoDIQsX7HBqUnOpp1+TUk+iha5m?=
 =?us-ascii?Q?HZzWi6SU6TsaDhuRURFLTsDmLMwNACsROou5zeaTeApZdjf6nZz3ZWvVo9Ok?=
 =?us-ascii?Q?QeCcK3AChY/2fIACmRkLSLgdnf/yG131XR4RqUs9teqZLy79B6ngDU9IMa87?=
 =?us-ascii?Q?k5sNLjdScFpuB8u91Xj7herYup598vysQCKNdath/BxLigZoP2sIyXihTiqj?=
 =?us-ascii?Q?Kp9u7xIkD6xRXC3Ggsob4QFTf9Bgo6dzdGBCqlurc32jhDSPT7R8TiZsQaew?=
 =?us-ascii?Q?aPOowntwF5xuzLQ+EvOIEQY7t2i09KiATe6i61ENW3t1WhnpBz8lhT4kkZgr?=
 =?us-ascii?Q?XeqwZ2qAj0/qS9wuohSTW0/H3A9rf7ACVkHCvKKBssUZWvhWPf2HZY7Adu9F?=
 =?us-ascii?Q?5Qu/FvYI9FL8sYDs7Shqeeid+pts4hgq9oBfd+dVcZaoomCJagpvzxJ38SYv?=
 =?us-ascii?Q?zP/Ttg8AO48It4MEoViK6BLy7psZu39o4LxlDL2dL/FUSxCo6b/NhXjUlT6b?=
 =?us-ascii?Q?GEkC3c1lk+oZzpnliBhM4H1XO/Ak94NNo6eR3V/730z+pEDVLoF+M9A3ZFtF?=
 =?us-ascii?Q?JfoVmdQd9wSbpkC7cWooLGCdp0CdvihzlfDlqmN+TaXSjEy79LbemtGz+Vin?=
 =?us-ascii?Q?RWJdgSe8WSx/TTzpZ2x+BODDaf+h2WG5sLWoN8yHsvGoz/45TYvndwXXbunI?=
 =?us-ascii?Q?jeiLB+Yi86JviVSvidR5ujD6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f949dfa-944d-46d8-63f0-08d97766dc86
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 10:03:12.1859
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9E8pt84bA3zBI531mdFxixUuI0E+cAwwcPEeujDAH0G1L/upd8MsFg7RDMMVVv9c6h/2PNcP7jN92oGoTw/UAQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 14.09.2021 11:41, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
>=20
> Move the call of early_reserve_memory() to the beginning of
> setup_arch() in order to avoid such problems.
>=20
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservatio=
ns")
> Reported-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab=
.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  arch/x86/kernel/setup.c | 24 ++++++++++++------------
>  1 file changed, 12 insertions(+), 12 deletions(-)
>=20
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..f369c51ec580 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsi=
gned long v, void *p)
> =20
>  void __init setup_arch(char **cmdline_p)
>  {
> +	/*
> +	 * Do some memory reservations *before* memory is added to
> +	 * memblock, so memblock allocations won't overwrite it.
> +	 * Do it after early param, so we could get (unlikely) panic from
> +	 * serial.

Hmm, this part of the comment is not only stale now, but gets actively
undermined. No idea how likely such a panic() would be, and hence how
relevant it is to retain this particular property.

Jan

> +	 * After this point everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not
> +	 * RAM in e820. All other memory is free game.
> +	 */
> +	early_reserve_memory();
> +
>  #ifdef CONFIG_X86_32
>  	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
> =20
> @@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
> =20
>  	parse_early_param();
> =20
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:04:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:04:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186383.335111 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5IK-0000CU-Qk; Tue, 14 Sep 2021 10:04:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186383.335111; Tue, 14 Sep 2021 10:04:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5IK-0000CN-My; Tue, 14 Sep 2021 10:04:04 +0000
Received: by outflank-mailman (input) for mailman id 186383;
 Tue, 14 Sep 2021 10:04:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ5II-0000C7-Vv
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:04:03 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1546e550-1543-11ec-b410-12813bfff9fa;
 Tue, 14 Sep 2021 10:04:01 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18E9hJfH014551; 
 Tue, 14 Sep 2021 10:03:58 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b2rke882w-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 10:03:57 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6689.eurprd03.prod.outlook.com (2603:10a6:20b:2db::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Tue, 14 Sep
 2021 10:03:53 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 10:03:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1546e550-1543-11ec-b410-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QEA1inzZSWTODo01bYxbpGrT2ZlC0iAtIhHF+nIdvEdxX+gS9ZZAQYoGDCbPRCT3gsnKUzXIlKu2AhRhv5ctRC2PIjxCGTRlgkCENkDoXnLTk+lx4idyFpPyhH98aa56A6T7A6tuTphqtMw07GqG5XD0LVJj6Dk3qHHuSRRq5lzPGWJm1qb2pikFXO3tPMA6VYKCFwpkmahj+k6krVrdYzrrcIM3/KjZa4W9UbxJ2djTHffHCtxkjzTOf/cFSbJbLeKGoKBQv7S7SA3l3ll0s8I/IPR/xxP9UgBgM/aBfny0jo327eSIODtb4iAoQz8seT5YGHLH9cIIXt0V/3IqPQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lN/64qMQ4AnI7Ukjq15Om4A7DOiJB8jZ57Q02MbLoLM=;
 b=WWaIeAnXB8eDaXA29IYvNgKZ1FsvrgVceaMW3uhAiH96IonvMcckaoL2+Zny6c5PPrg4doX9MfMSYMpm1aDcjM7fzLoIeefi6zMjbqDq7FI2kAAPyb9p+KPMc/EBQ1cw9fiMfgRKSqxKuBxdTTv1FouoHQbiel1YIjQT5n403r0UDHjvFw3V3fJmWfyHV2wtkbbpZy8IWhoc1Rkls6ND5kC476PR5O8AWmOcjX6ir42h9IAO4QGayuQYCp2QdoT6/jYXLARdvZm7NvqguLBeYp7L9XDw/H1mB4E/Co3HWWhsfdTAso9NaQQL99la0x1GocCv+9HYDMAj+tPQxV1Vxg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lN/64qMQ4AnI7Ukjq15Om4A7DOiJB8jZ57Q02MbLoLM=;
 b=fiABSu4qX7p6QVlH4g7ZrUpFw0V+3ORzSFI9UB8R6ynTZ3x2pdanwA58vCuF9v28YVRPMhp+CgNY2nv9s3C186ZbBsuZNZ2kfwKrP1Ww8KjHFk0szdxhY5+EHTJWcE+ykLn6jGnbssSFq3Jm6sIvbadHMfnykMS8M/vefYyc5+AqxAPam6CvbtRgF1EsMBBEHx0UlxfHtc7fjqVrunxrCrDiDBctpZFaN9HeobvtyPFTYijv9tPaV1SPLtPTpQFAHkq6IqdmL+v9ypJIi35wDZ0V/H4xDFcMQ6kjE4LF1+CGGLWbqNfeCNxZOBddOeT4vvhXkVuXt77RUW1aB9S2Pw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien.grall.oss@gmail.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: Oleksandr Andrushchenko <andr2000@gmail.com>,
        xen-devel
	<xen-devel@lists.xenproject.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAARZwCAAFsJAIAAFA4AgAO3eICAAc7agA==
Date: Tue, 14 Sep 2021 10:03:53 +0000
Message-ID: <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
 <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
 <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
In-Reply-To: <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 097feea7-d11d-4634-8dd1-08d97766f565
x-ms-traffictypediagnostic: AM9PR03MB6689:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB66899A63B48EFBE8C6EB78D8E7DA9@AM9PR03MB6689.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6430;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 HjLyIIfEZCXJ/fljvMbzgLjhmG5HHHX59u69UMVMusk2evOsOdxQQyIwq75vJz45FPeOK31gXbQGu7Hx8XJAt9KgudUYlhbwBjCKxjIaOfDQ4XZ+17+QrELGttuW/AzR8bM1sFiwvpOY/5dI4DyDo6pKw109svpjLY7WB4s1SADEI1cUR7HuGGeru2NtyyKJbNGv2BURuSDL3d2X2tARFlYQ7/bixQGrVeazANL39/Ltu9DCfqZeSEoF0+O2+1p4EMkKNACmflkw+tTmOAC4U10TU2ynD8BoQ8CAXZ1uwxoWgy23JRrYfGshLE+DGlS6Dq0+ivsr6wkRckvCgeo22yIsf929v0nPEhJf4lSLVLdEXF8nbepVZtyzM4py8GmQ7aXGUc7qho6PiXmXEc59lJmN7XfYftffnSQDOSddl2WdxW1Gm2z+EIYmMpAI6AJ66QPPdPE/6Dv1HI5fNZTIUgFBO3S4ZBiw9wpBJ2GGnwJeNEMSAF4+dQizuQincThx2d+/S0GzPP0AhOzLOXGAlJ5BFCGUEni2AkxlhB3QastG647zkF6T+gGLbESpn2x6rublILzfiXnvteZPcqYVaBQPm8Dv3gvdC/chw92x7PQjZs91PhoYypcwHuVV2eQyYcnW4IPx9SWH/h8YhRja3WixJY8k7dqrzAX+jB4klJRcq24DYrXvSDFuocbD6XWkFX3BCuJZ77NXLGP+uecscDTuP5VK5Pr3/jzVWFdvUHPZdWRcoe4vbJM7nqI9QLbT
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(39860400002)(346002)(136003)(376002)(396003)(31696002)(5660300002)(122000001)(66556008)(110136005)(64756008)(86362001)(66476007)(76116006)(38100700002)(36756003)(8936002)(66946007)(71200400001)(91956017)(55236004)(6512007)(53546011)(2616005)(6506007)(6486002)(30864003)(54906003)(2906002)(316002)(38070700005)(31686004)(186003)(8676002)(66446008)(83380400001)(4326008)(26005)(478600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Z3B3K1VGK0hWT3BMYlFvaXNZVXZYaWdzOWg2MDZlRG4yVVcwamI2Nms5UlB3?=
 =?utf-8?B?S3h4WVRNTUtaNEVXelc1TlM0SGxRVFMwOGhDWnJxU3JMaXF3N2RqM1VBMkd3?=
 =?utf-8?B?dmk5eHAwSTZRMTlUR3pmT0pTNU9hdnFGUGZ3OGFXd29WZm9YL0xzbnN1Q1h2?=
 =?utf-8?B?UllEcXQ0OVcrS1RoM25waEpPaEw2SW1aYVRhaWE1TGV1OFRUMkFlYmlBSDZ6?=
 =?utf-8?B?dGY3YUtWMXhyV1M2aENUMFBUL1d3YzEyT2NGK0xmRklZc3dPM1FaT0Rub0Fw?=
 =?utf-8?B?L09aczg3cStDQlFEQUh5VnZqS3V6WWJWL0s3blJEV1lQOEZaTCtUd2J5SGFT?=
 =?utf-8?B?TnhwaGV6eEY1TVJnUys1SXFwb0huYUt0b3pKV3FOV2JqYzVNMFBVODdHTW9H?=
 =?utf-8?B?VTVDWHA5SFY1dzdLVWx6eVh5ZGdiaEZicWE3NnVCMDlIT0s3a0VyYmp3bitW?=
 =?utf-8?B?UFQ1d1RaK0Zqc1dycHNUVjRxbEtUYXJPblVxVnZMeU5ZVDBNQ29tdHFsMzMz?=
 =?utf-8?B?c3ZlblpEaG0zb2daa2Z2Mm1Gdm55M0Y5OFZZTmV5Q2VLdko4WUJacy9iR0tw?=
 =?utf-8?B?b3RXU3BtYURJRks3b1o2K2RueXR3RGVtOEpVRGk2MUc2RHpXbHhlQW5tR1hU?=
 =?utf-8?B?TlJxWGZRbkFUV1pYQ0Vxcm9JWWQ1d3VEcGljNzUxcFJoYzZxdDZlNmlHY3FG?=
 =?utf-8?B?OFQrTnNQdVJYOXVEKzJaY0NCMzV1dkpvZnd5UkI0RXRxQWZWWkFWV1RJOXVC?=
 =?utf-8?B?MjQyczl1aVRjeXZ4VGw5WUdVelVCRU90NmJMR2NidUFqclY5dU1sZER5UXNx?=
 =?utf-8?B?VTNVcjExclNMQnB0WXU3TitUS1lrUk1CclZhcU1ocjBETW9rVTRQY05CbDFW?=
 =?utf-8?B?Lzk3anFETmdJVHpFaU9La2phdjB4VFZIa20xNVllb1MxN29pOStvb0p1Q3VQ?=
 =?utf-8?B?eDJZZnBCTGlQaWE0OUh1cGIrUmhwc2Z6NTVJTmxNR0hxSHV4RG5OajBneFE5?=
 =?utf-8?B?ZlRzSlZVdGU4K3BjS29qNXB6eFdxakE0VXdJVDY5aGE0ZWtVRUszR0JVU05K?=
 =?utf-8?B?OUpiVm1FYVVFS2t3QWVYRHdtU2xTb0xLbFl5WHIrRHFjVlRYTTlNTm1hajZ3?=
 =?utf-8?B?b3V3V3FMTGhkRFpCTjZpUE4zNXRYbDdGT0Z2WnJtaXJTaXBheXNvRkliVjBG?=
 =?utf-8?B?L2VWQkMxNXQ3MmNqVS9tbkNRT3gzS3hDT2VKUFNDdFpJSmh1NTE1MC9DUDJG?=
 =?utf-8?B?SkEwbmpOWkxDRWFZTlJxVU1IUlBJamVNSDZ4a3dKTksyYmorcks3TGZvakNv?=
 =?utf-8?B?S3BmaGtteHcrbTNLYzNtVXhJNFdyN3R2MDNxdFl5VTd5WjU0d3JrOWcvMmVZ?=
 =?utf-8?B?SHVBNXJZeExmajlTcnBmMlR2d2xjUmc3YnV6WlUvcys4RkJHTE5wclFjNGR3?=
 =?utf-8?B?YnZJZzhyNmhaUHBkOURHOUdCaW5US25mVVBsMHVnSzRKL2JERUpNd0x4aUJM?=
 =?utf-8?B?eWc2bG0yREltTlphWjF3MEUxeTlURlJlQ2I5SzljV0FOZ1U1ZEZ5bG9zUmFX?=
 =?utf-8?B?QmkxY0lLb2c1WTNSc2F3M2EzL2NrdkczOUNVRitLMkl6cnRRTXJnVFVEbWxz?=
 =?utf-8?B?M09IMFhuaU1qdHJUSHNJdUM0ZzMzZmRDaDRyVmlCckpmR2crcTl1MU9DN3FT?=
 =?utf-8?B?OWgvT0xKOFpqaE1rWG55ZEQrRXcxbW5uN1M0THdJYS9lWGh5LzduVGNBSDdY?=
 =?utf-8?B?MHhGTGNmRlBhTkhXZWVKUVdjV0ZVV1IxREhFSlRRNHlMQk5nLzg5VmZSSFVX?=
 =?utf-8?B?OTNTS1loUU1nTGZoWU51QT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F68A49ED8D8E5A4EB1107D7ED4837B45@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 097feea7-d11d-4634-8dd1-08d97766f565
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 10:03:53.5993
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3N/vuHBUpC+j9TJkjzORUAGxMcK/KuEOo245bP9Ksi+DwADtzH4fyFD0lfmwVbBNjggmr8yThFXaToSrLSW2NNQwtysfUfGCz3wLajOCJepFmAUiAM0VR1hW2a+nYy8B
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6689
X-Proofpoint-GUID: RnXwRAgZC6SYjzg8jhU_0MD9Lb5IZ-c2
X-Proofpoint-ORIG-GUID: RnXwRAgZC6SYjzg8jhU_0MD9Lb5IZ-c2
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_02,2021-09-09_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 lowpriorityscore=0 priorityscore=1501 mlxscore=0 phishscore=0
 clxscore=1015 bulkscore=0 spamscore=0 malwarescore=0 suspectscore=0
 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109140061

DQo+DQo+Pg0KPj4gwqDCoMKgIElmIHNvLCBpdCB3b3VsZCBiZSBiZXR0ZXIgaWYgdGhlIE1NSU8g
cmVnaW9uIGluIHF1ZXN0aW9uIHdhcyBuZXZlcg0KPj4gwqDCoMKgIG1hcHBlZCBpbnRvIERvbTAg
YXQgYWxsIHJhdGhlciBoYXZpbmcgdG8gdW5tYXAgaXQuDQo+Pg0KPiBPaywgSSdsbCBkbyB0aGF0
DQoNClNvcnJ5IGZvciBwYXN0aW5nIGhlcmUgcXVpdGUgYSBiaWcgcGF0Y2gsIGJ1dCBJIGZlZWwg
SSBuZWVkIGNsYXJpZmljYXRpb24gaWYgdGhpcyBpcw0KDQp0aGUgd2F5IHdlIHdhbnQgaXQuIFBs
ZWFzZSBub3RlIEkgaGFkIHRvIG1vZGlmeSBzZXR1cC5oLg0KDQoNCiBGcm9tIDZlZWU5NmJjMDQ2
ZWZiNDFlYzI1Zjg3MzYyYjFmNmU5NzNhNGU2YzIgTW9uIFNlcCAxNyAwMDowMDowMCAyMDAxDQpG
cm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBh
bS5jb20+DQpEYXRlOiBUdWUsIDE0IFNlcCAyMDIxIDEyOjE0OjQzICswMzAwDQpTdWJqZWN0OiBb
UEFUQ0hdIEZpeGVzOiBhNTdkYzg0ZGE1ZmQgKCJ4ZW4vYXJtOiBEbyBub3QgbWFwIFBDSSBFQ0FN
IHNwYWNlIHRvDQogwqBEb21haW4tMCdzIHAybSIpDQoNClNpZ25lZC1vZmYtYnk6IE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCi0tLQ0K
IMKgeGVuL2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jwqDCoMKgwqDCoMKgwqAgfCAzNyArKysrKysr
KysrKysrLS0tLS0tLS0NCiDCoHhlbi9hcmNoL2FybS9wY2kvZWNhbS5jwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCB8IDExICsrKy0tLS0NCiDCoHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMgfCA1MyArKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0NCiDCoHhlbi9pbmNsdWRlL2Fz
bS1hcm0vcGNpLmjCoMKgwqDCoMKgwqDCoMKgwqAgfCAxOCArKysrLS0tLS0tDQogwqB4ZW4vaW5j
bHVkZS9hc20tYXJtL3NldHVwLmjCoMKgwqDCoMKgwqDCoCB8IDEzICsrKysrKysrDQogwqA1IGZp
bGVzIGNoYW5nZWQsIDg2IGluc2VydGlvbnMoKyksIDQ2IGRlbGV0aW9ucygtKQ0KDQpkaWZmIC0t
Z2l0IGEveGVuL2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jIGIveGVuL2FyY2gvYXJtL2RvbWFpbl9i
dWlsZC5jDQppbmRleCA3NmY1YjUxMzI4MGMuLmI0YmZkYTlkNWI1YSAxMDA2NDQNCi0tLSBhL3hl
bi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KKysrIGIveGVuL2FyY2gvYXJtL2RvbWFpbl9idWls
ZC5jDQpAQCAtMTAsNyArMTAsNiBAQA0KIMKgI2luY2x1ZGUgPGFzbS9yZWdzLmg+DQogwqAjaW5j
bHVkZSA8eGVuL2Vycm5vLmg+DQogwqAjaW5jbHVkZSA8eGVuL2Vyci5oPg0KLSNpbmNsdWRlIDx4
ZW4vZGV2aWNlX3RyZWUuaD4NCiDCoCNpbmNsdWRlIDx4ZW4vbGliZmR0L2xpYmZkdC5oPg0KIMKg
I2luY2x1ZGUgPHhlbi9ndWVzdF9hY2Nlc3MuaD4NCiDCoCNpbmNsdWRlIDx4ZW4vaW9jYXAuaD4N
CkBAIC00NywxMiArNDYsNiBAQCBzdGF0aWMgaW50IF9faW5pdCBwYXJzZV9kb20wX21lbShjb25z
dCBjaGFyICpzKQ0KIMKgfQ0KIMKgY3VzdG9tX3BhcmFtKCJkb20wX21lbSIsIHBhcnNlX2RvbTBf
bWVtKTsNCg0KLXN0cnVjdCBtYXBfcmFuZ2VfZGF0YQ0KLXsNCi3CoMKgwqAgc3RydWN0IGRvbWFp
biAqZDsNCi3CoMKgwqAgcDJtX3R5cGVfdCBwMm10Ow0KLX07DQotDQogwqAvKiBPdmVycmlkZSBt
YWNyb3MgZnJvbSBhc20vcGFnZS5oIHRvIG1ha2UgdGhlbSB3b3JrIHdpdGggbWZuX3QgKi8NCiDC
oCN1bmRlZiB2aXJ0X3RvX21mbg0KIMKgI2RlZmluZSB2aXJ0X3RvX21mbih2YSkgX21mbihfX3Zp
cnRfdG9fbWZuKHZhKSkNCkBAIC0xMjI4LDkgKzEyMjEsOCBAQCBzdGF0aWMgaW50IF9faW5pdCBt
YXBfZHRfaXJxX3RvX2RvbWFpbihjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCiDC
oMKgwqDCoCByZXR1cm4gMDsNCiDCoH0NCg0KLXN0YXRpYyBpbnQgX19pbml0IG1hcF9yYW5nZV90
b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQotwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgdTY0IGFkZHIsIHU2NCBsZW4sDQotwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdm9pZCAqZGF0YSkN
CitpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9u
b2RlICpkZXYsDQorwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIHU2NCBhZGRyLCB1NjQgbGVuLCB2b2lkICpkYXRhKQ0KIMKgew0KIMKg
wqDCoMKgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSAqbXJfZGF0YSA9IGRhdGE7DQogwqDCoMKgwqAg
c3RydWN0IGRvbWFpbiAqZCA9IG1yX2RhdGEtPmQ7DQpAQCAtMTI1Nyw4ICsxMjQ5LDEwIEBAIHN0
YXRpYyBpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2Rldmlj
ZV9ub2RlICpkZXYsDQogwqDCoMKgwqDCoMKgwqDCoCB9DQogwqDCoMKgwqAgfQ0KDQotwqDCoMKg
IGlmICggbmVlZF9tYXBwaW5nICYmIChkZXZpY2VfZ2V0X2NsYXNzKGRldikgPT0gREVWSUNFX1BD
SSkgKQ0KLcKgwqDCoMKgwqDCoMKgIG5lZWRfbWFwcGluZyA9IHBjaV9ob3N0X2JyaWRnZV9uZWVk
X3AybV9tYXBwaW5nKGQsIGRldiwgYWRkciwgbGVuKTsNCisjaWZkZWYgQ09ORklHX0hBU19QQ0kN
CivCoMKgwqAgaWYgKCAoZGV2aWNlX2dldF9jbGFzcyhkZXYpID09IERFVklDRV9QQ0kpICYmICFt
cl9kYXRhLT5tYXBfcGNpX2JyaWRnZSApDQorwqDCoMKgwqDCoMKgwqAgbmVlZF9tYXBwaW5nID0g
ZmFsc2U7DQorI2VuZGlmDQoNCiDCoMKgwqDCoCBpZiAoIG5lZWRfbWFwcGluZyApDQogwqDCoMKg
wqAgew0KQEAgLTEyOTMsNyArMTI4NywxMSBAQCBzdGF0aWMgaW50IF9faW5pdCBtYXBfZGV2aWNl
X2NoaWxkcmVuKHN0cnVjdCBkb21haW4gKmQsDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBz
dHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHAybV90eXBl
X3QgcDJtdCkNCiDCoHsNCi3CoMKgwqAgc3RydWN0IG1hcF9yYW5nZV9kYXRhIG1yX2RhdGEgPSB7
IC5kID0gZCwgLnAybXQgPSBwMm10IH07DQorwqDCoMKgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSBt
cl9kYXRhID0gew0KK8KgwqDCoMKgwqDCoMKgIC5kID0gZCwNCivCoMKgwqDCoMKgwqDCoCAucDJt
dCA9IHAybXQsDQorwqDCoMKgwqDCoMKgwqAgLm1hcF9wY2lfYnJpZGdlID0gZmFsc2UNCivCoMKg
wqAgfTsNCiDCoMKgwqDCoCBpbnQgcmV0Ow0KDQogwqDCoMKgwqAgaWYgKCBkdF9kZXZpY2VfdHlw
ZV9pc19lcXVhbChkZXYsICJwY2kiKSApDQpAQCAtMTQyNSw3ICsxNDIzLDExIEBAIHN0YXRpYyBp
bnQgX19pbml0IGhhbmRsZV9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGR0X2Rldmlj
ZV9ub2RlICpkZXYsDQogwqDCoMKgwqAgLyogR2l2ZSBwZXJtaXNzaW9uIGFuZCBtYXAgTU1JT3Mg
Ki8NCiDCoMKgwqDCoCBmb3IgKCBpID0gMDsgaSA8IG5hZGRyOyBpKysgKQ0KIMKgwqDCoMKgIHsN
Ci3CoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSA9IHsgLmQgPSBk
LCAucDJtdCA9IHAybXQgfTsNCivCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEg
bXJfZGF0YSA9IHsNCivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC5kID0gZCwNCivCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgIC5wMm10ID0gcDJtdCwNCivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIC5tYXBf
cGNpX2JyaWRnZSA9IGZhbHNlDQorwqDCoMKgwqDCoMKgwqAgfTsNCiDCoMKgwqDCoMKgwqDCoMKg
IHJlcyA9IGR0X2RldmljZV9nZXRfYWRkcmVzcyhkZXYsIGksICZhZGRyLCAmc2l6ZSk7DQogwqDC
oMKgwqDCoMKgwqDCoCBpZiAoIHJlcyApDQogwqDCoMKgwqDCoMKgwqDCoCB7DQpAQCAtMjU5NCw3
ICsyNTk2LDE0IEBAIHN0YXRpYyBpbnQgX19pbml0IGNvbnN0cnVjdF9kb20wKHN0cnVjdCBkb21h
aW4gKmQpDQogwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gcmM7DQoNCiDCoMKgwqDCoCBpZiAoIGFj
cGlfZGlzYWJsZWQgKQ0KK8KgwqDCoCB7DQogwqDCoMKgwqDCoMKgwqDCoCByYyA9IHByZXBhcmVf
ZHRiX2h3ZG9tKGQsICZraW5mbyk7DQorI2lmZGVmIENPTkZJR19IQVNfUENJDQorwqDCoMKgwqDC
oMKgwqAgaWYgKCByYyA8IDAgKQ0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIHJjOw0K
K8KgwqDCoMKgwqDCoMKgIHJjID0gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKGQsIHAybV9tbWlv
X2RpcmVjdF9jKTsNCisjZW5kaWYNCivCoMKgwqAgfQ0KIMKgwqDCoMKgIGVsc2UNCiDCoMKgwqDC
oMKgwqDCoMKgIHJjID0gcHJlcGFyZV9hY3BpKGQsICZraW5mbyk7DQoNCmRpZmYgLS1naXQgYS94
ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQppbmRleCBk
MzJlZmI3ZmNiZDAuLmUwOGI5YzY5MDliNiAxMDA2NDQNCi0tLSBhL3hlbi9hcmNoL2FybS9wY2kv
ZWNhbS5jDQorKysgYi94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0KQEAgLTUyLDE3ICs1MiwxNCBA
QCBzdGF0aWMgaW50IHBjaV9lY2FtX3JlZ2lzdGVyX21taW9faGFuZGxlcihzdHJ1Y3QgZG9tYWlu
ICpkLA0KIMKgwqDCoMKgIHJldHVybiAwOw0KIMKgfQ0KDQotc3RhdGljIGludCBwY2lfZWNhbV9u
ZWVkX3AybV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsDQotwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0cnVj
dCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCi3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRk
ciwgdWludDY0X3QgbGVuKQ0KK3N0YXRpYyBib29sIHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmco
c3RydWN0IGRvbWFpbiAqZCwNCivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJp
ZGdlICpicmlkZ2UsDQorwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkcikNCiDCoHsNCiDC
oMKgwqDCoCBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0YTsN
Cg0KLcKgwqDCoCBpZiAoICFpc19oYXJkd2FyZV9kb21haW4oZCkgKQ0KLcKgwqDCoMKgwqDCoMKg
IHJldHVybiB0cnVlOw0KLQ0KIMKgwqDCoMKgIC8qDQotwqDCoMKgwqAgKiBXZSBkbyBub3Qgd2Fu
dCBFQ0FNIGFkZHJlc3Mgc3BhY2UgdG8gYmUgbWFwcGVkIGluIGRvbWFpbidzIHAybSwNCivCoMKg
wqDCoCAqIFdlIGRvIG5vdCB3YW50IEVDQU0gYWRkcmVzcyBzcGFjZSB0byBiZSBtYXBwZWQgaW4g
RG9tYWluLTAncyBwMm0sDQogwqDCoMKgwqDCoCAqIHNvIHdlIGNhbiB0cmFwIGFjY2VzcyB0byBp
dC4NCiDCoMKgwqDCoMKgICovDQogwqDCoMKgwqAgcmV0dXJuIGNmZy0+cGh5c19hZGRyICE9IGFk
ZHI7DQpkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYyBiL3hl
bi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMNCmluZGV4IGMwNGJlNjM2NDUyZC4uNzQw
NzdkZWM4YzcyIDEwMDY0NA0KLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24u
Yw0KKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KQEAgLTI1LDYgKzI1
LDcgQEANCiDCoCNpbmNsdWRlIDx4ZW4vaW5pdC5oPg0KIMKgI2luY2x1ZGUgPHhlbi9wY2kuaD4N
CiDCoCNpbmNsdWRlIDxhc20vcGNpLmg+DQorI2luY2x1ZGUgPGFzbS9zZXR1cC5oPg0KIMKgI2lu
Y2x1ZGUgPHhlbi9yd2xvY2suaD4NCiDCoCNpbmNsdWRlIDx4ZW4vc2NoZWQuaD4NCiDCoCNpbmNs
dWRlIDx4ZW4vdm1hcC5oPg0KQEAgLTMzNSwyNSArMzM2LDUxIEBAIGludCBwY2lfaG9zdF9pdGVy
YXRlX2JyaWRnZXMoc3RydWN0IGRvbWFpbiAqZCwNCiDCoMKgwqDCoCByZXR1cm4gMDsNCiDCoH0N
Cg0KLWJvb2wgcGNpX2hvc3RfYnJpZGdlX25lZWRfcDJtX21hcHBpbmcoc3RydWN0IGRvbWFpbiAq
ZCwNCi3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKm5vZGUs
DQotwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWludDY0X3QgbGVuKQ0KK2ludCBf
X2luaXQgcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKHN0cnVjdCBkb21haW4gKmQsIHAybV90eXBl
X3QgcDJtdCkNCiDCoHsNCiDCoMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2U7
DQorwqDCoMKgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSBtcl9kYXRhID0gew0KK8KgwqDCoMKgwqDC
oMKgIC5kID0gZCwNCivCoMKgwqDCoMKgwqDCoCAucDJtdCA9IHAybXQsDQorwqDCoMKgwqDCoMKg
wqAgLm1hcF9wY2lfYnJpZGdlID0gdHJ1ZQ0KK8KgwqDCoCB9Ow0KDQorwqDCoMKgIC8qDQorwqDC
oMKgwqAgKiBGb3IgZWFjaCBQQ0kgaG9zdCBicmlkZ2Ugd2UgbmVlZCB0byBvbmx5IG1hcCB0aG9z
ZSByYW5nZXMNCivCoMKgwqDCoCAqIHdoaWNoIGFyZSB1c2VkIGJ5IERvbWFpbi0wIHRvIHByb3Bl
cmx5IGluaXRpYWxpemUgdGhlIGJyaWRnZSwNCivCoMKgwqDCoCAqIGUuZy4gd2UgZG8gbm90IHdh
bnQgdG8gbWFwIEVDQU0gY29uZmlndXJhdGlvbiBzcGFjZSB3aGljaCBsaXZlcyBpbg0KK8KgwqDC
oMKgICogInJlZyIgb3IgImFzc2lnbmVkLWFkZHJlc3NlcyIgZGV2aWNlIHRyZWUgcHJvcGVydHku
DQorwqDCoMKgwqAgKiBOZWl0aGVyIHdlIHdhbnQgdG8gbWFwIGFueSBvZiB0aGUgTU1JTyByYW5n
ZXMgZm91bmQgaW4gdGhlwqAgInJhbmdlcyINCivCoMKgwqDCoCAqIGRldmljZSB0cmVlIHByb3Bl
cnR5Lg0KK8KgwqDCoMKgICovDQogwqDCoMKgwqAgbGlzdF9mb3JfZWFjaF9lbnRyeSggYnJpZGdl
LCAmcGNpX2hvc3RfYnJpZGdlcywgbm9kZSApDQogwqDCoMKgwqAgew0KLcKgwqDCoMKgwqDCoMKg
IGlmICggYnJpZGdlLT5kdF9ub2RlICE9IG5vZGUgKQ0KLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
Y29udGludWU7DQotDQotwqDCoMKgwqDCoMKgwqAgaWYgKCAhYnJpZGdlLT5vcHMtPm5lZWRfcDJt
X21hcHBpbmcgKQ0KLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIHRydWU7DQotDQotwqDC
oMKgwqDCoMKgwqAgcmV0dXJuIGJyaWRnZS0+b3BzLT5uZWVkX3AybV9tYXBwaW5nKGQsIGJyaWRn
ZSwgYWRkciwgbGVuKTsNCivCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25v
ZGUgKmRldiA9IGJyaWRnZS0+ZHRfbm9kZTsNCivCoMKgwqDCoMKgwqDCoCBpbnQgaTsNCisNCivC
oMKgwqDCoMKgwqDCoCBmb3IgKCBpID0gMDsgaSA8IGR0X251bWJlcl9vZl9hZGRyZXNzKGRldik7
IGkrKyApDQorwqDCoMKgwqDCoMKgwqAgew0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdWludDY0
X3QgYWRkciwgc2l6ZTsNCivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGludCBlcnI7DQorDQorwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJlc3MoZGV2LCBpLCAm
YWRkciwgJnNpemUpOw0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaWYgKCBlcnIgKQ0KK8KgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgew0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwcmlu
dGsoWEVOTE9HX0VSUiAiVW5hYmxlIHRvIHJldHJpZXZlIGFkZHJlc3MgJXUgZm9yICVzXG4iLA0K
K8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGksIGR0X25vZGVf
ZnVsbF9uYW1lKGRldikpOw0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4g
ZXJyOw0KK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfQ0KKw0KK8KgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgaWYgKCBicmlkZ2UtPm9wcy0+bmVlZF9wMm1fbWFwcGluZyhkLCBicmlkZ2UsIGFkZHIpICkN
CivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHsNCivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgZXJyID0gbWFwX3JhbmdlX3RvX2RvbWFpbihkZXYsIGFkZHIsIHNpemUsICZtcl9kYXRhKTsN
CivCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaWYgKCBlcnIgKQ0KK8KgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiBlcnI7DQorwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCB9DQorwqDCoMKgwqDCoMKgwqAgfQ0KIMKgwqDCoMKgIH0NCi3CoMKgwqAgcHJpbnRr
KFhFTkxPR19FUlIgIlVuYWJsZSB0byBmaW5kIFBDSSBicmlkZ2UgZm9yICVzIHNlZ21lbnQgJWQs
IGFkZHIgJWx4XG4iLA0KLcKgwqDCoMKgwqDCoMKgwqDCoMKgIG5vZGUtPmZ1bGxfbmFtZSwgYnJp
ZGdlLT5zZWdtZW50LCBhZGRyKTsNCi3CoMKgwqAgcmV0dXJuIHRydWU7DQorDQorwqDCoMKgIHJl
dHVybiAwOw0KIMKgfQ0KDQogwqAvKg0KZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0v
cGNpLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oDQppbmRleCA5YzI4YTRiZGM0YjcuLjk3
ZmJhYWMwMTM3MCAxMDA2NDQNCi0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCisrKyBi
L3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCkBAIC0yMSw2ICsyMSw4IEBADQoNCiDCoCNpZmRl
ZiBDT05GSUdfSEFTX1BDSQ0KDQorI2luY2x1ZGUgPGFzbS9wMm0uaD4NCisNCiDCoCNkZWZpbmUg
cGNpX3RvX2RldihwY2lkZXYpICgmKHBjaWRldiktPmFyY2guZGV2KQ0KIMKgI2RlZmluZSBQUklf
cGNpICIlMDR4OiUwMng6JTAyeC4ldSINCg0KQEAgLTgyLDggKzg0LDkgQEAgc3RydWN0IHBjaV9v
cHMgew0KIMKgwqDCoMKgIGludCAoKnJlZ2lzdGVyX21taW9faGFuZGxlcikoc3RydWN0IGRvbWFp
biAqZCwNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KIMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCBjb25zdCBzdHJ1Y3QgbW1pb19oYW5kbGVyX29wcyAqb3BzKTsNCi3CoMKgwqAgaW50ICgq
bmVlZF9wMm1fbWFwcGluZykoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IHBjaV9ob3N0X2JyaWRn
ZSAqYnJpZGdlLA0KLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCB1aW50NjRfdCBhZGRyLCB1aW50NjRfdCBsZW4pOw0KK8KgwqDCoCBib29sICgq
bmVlZF9wMm1fbWFwcGluZykoc3RydWN0IGRvbWFpbiAqZCwNCivCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgcGNpX2hvc3RfYnJp
ZGdlICpicmlkZ2UsDQorwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkcik7DQogwqB9Ow0KDQogwqAvKg0KQEAgLTExNywx
OSArMTIwLDEwIEBAIHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqcGNpX2ZpbmRfaG9zdF9icmlkZ2Vf
bm9kZShzdHJ1Y3QgZGV2aWNlICpkZXYpOw0KIMKgaW50IHBjaV9ob3N0X2l0ZXJhdGVfYnJpZGdl
cyhzdHJ1Y3QgZG9tYWluICpkLA0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW50ICgqY2xiKShzdHJ1Y3QgZG9tYWluICpkLA0KIMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSkpOw0K
LWJvb2wgcGNpX2hvc3RfYnJpZGdlX25lZWRfcDJtX21hcHBpbmcoc3RydWN0IGRvbWFpbiAqZCwN
Ci3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKm5vZGUsDQot
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgdWludDY0X3QgYWRkciwgdWludDY0X3QgbGVuKTsNCitpbnQgcGNp
X2hvc3RfYnJpZGdlX21hcHBpbmdzKHN0cnVjdCBkb21haW4gKmQsIHAybV90eXBlX3QgcDJtdCk7
DQogwqAjZWxzZcKgwqAgLyohQ09ORklHX0hBU19QQ0kqLw0KDQogwqBzdHJ1Y3QgYXJjaF9wY2lf
ZGV2IHsgfTsNCg0KLXN0YXRpYyBpbmxpbmUgYm9vbA0KLXBjaV9ob3N0X2JyaWRnZV9uZWVkX3Ay
bV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsDQotwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgZHRfZGV2
aWNlX25vZGUgKm5vZGUsDQotwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1aW50NjRfdCBhZGRyLCB1aW50NjRfdCBsZW4pDQot
ew0KLcKgwqDCoCByZXR1cm4gdHJ1ZTsNCi19DQogwqAjZW5kaWbCoCAvKiFDT05GSUdfSEFTX1BD
SSovDQogwqAjZW5kaWYgLyogX19BUk1fUENJX0hfXyAqLw0KZGlmZiAtLWdpdCBhL3hlbi9pbmNs
dWRlL2FzbS1hcm0vc2V0dXAuaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vc2V0dXAuaA0KaW5kZXgg
YzRiNmFmNjAyOTk1Li5hMWMzMWMwYmIwMjQgMTAwNjQ0DQotLS0gYS94ZW4vaW5jbHVkZS9hc20t
YXJtL3NldHVwLmgNCisrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vc2V0dXAuaA0KQEAgLTIsNiAr
Miw4IEBADQogwqAjZGVmaW5lIF9fQVJNX1NFVFVQX0hfDQoNCiDCoCNpbmNsdWRlIDxwdWJsaWMv
dmVyc2lvbi5oPg0KKyNpbmNsdWRlIDxhc20vcDJtLmg+DQorI2luY2x1ZGUgPHhlbi9kZXZpY2Vf
dHJlZS5oPg0KDQogwqAjZGVmaW5lIE1JTl9GRFRfQUxJR04gOA0KIMKgI2RlZmluZSBNQVhfRkRU
X1NJWkUgU1pfMk0NCkBAIC03Niw2ICs3OCwxNCBAQCBzdHJ1Y3QgYm9vdGluZm8gew0KIMKgI2Vu
ZGlmDQogwqB9Ow0KDQorc3RydWN0IG1hcF9yYW5nZV9kYXRhDQorew0KK8KgwqDCoCBzdHJ1Y3Qg
ZG9tYWluICpkOw0KK8KgwqDCoCBwMm1fdHlwZV90IHAybXQ7DQorwqDCoMKgIC8qIFNldCBpZiBt
YXBwaW5ncyBmb3IgUENJIGhvc3QgYnJpZGdlcyBtdXN0IG5vdCBiZSBza2lwcGVkLiAqLw0KK8Kg
wqDCoCBib29sIG1hcF9wY2lfYnJpZGdlOw0KK307DQorDQogwqBleHRlcm4gc3RydWN0IGJvb3Rp
bmZvIGJvb3RpbmZvOw0KDQogwqBleHRlcm4gZG9taWRfdCBtYXhfaW5pdF9kb21pZDsNCkBAIC0x
MjMsNiArMTMzLDkgQEAgdm9pZCBkZXZpY2VfdHJlZV9nZXRfcmVnKGNvbnN0IF9fYmUzMiAqKmNl
bGwsIHUzMiBhZGRyZXNzX2NlbGxzLA0KIMKgdTMyIGRldmljZV90cmVlX2dldF91MzIoY29uc3Qg
dm9pZCAqZmR0LCBpbnQgbm9kZSwNCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgY29uc3QgY2hhciAqcHJvcF9uYW1lLCB1MzIgZGZsdCk7DQoNCitpbnQg
bWFwX3JhbmdlX3RvX2RvbWFpbihjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCivC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHU2NCBhZGRyLCB1
NjQgbGVuLCB2b2lkICpkYXRhKTsNCisNCiDCoCNlbmRpZg0KIMKgLyoNCiDCoCAqIExvY2FsIHZh
cmlhYmxlczoNCi0tIA0KMi4yNS4xDQoNCg0KV2l0aCB0aGUgcGF0Y2ggYWJvdmUgSSBoYXZlIHRo
ZSBmb2xsb3dpbmcgbG9nIGluIERvbWFpbi0wOg0KDQpnZW5lcmljLWFybXY4LXh0LWRvbTAgbG9n
aW46IHJvb3QNCnJvb3RAZ2VuZXJpYy1hcm12OC14dC1kb20wOn4jIChYRU4pICoqKiBTZXJpYWwg
aW5wdXQgdG8gWGVuICh0eXBlICdDVFJMLWEnIHRocmVlIHRpbWVzIHRvIHN3aXRjaCBpbnB1dCkN
CihYRU4pID09PT0gUENJIGRldmljZXMgPT09PQ0KKFhFTikgPT09PSBzZWdtZW50IDAwMDAgPT09
PQ0KKFhFTikgMDAwMDowMzowMC4wIC0gZDAgLSBub2RlIC0xDQooWEVOKSAwMDAwOjAyOjAyLjAg
LSBkMCAtIG5vZGUgLTENCihYRU4pIDAwMDA6MDI6MDEuMCAtIGQwIC0gbm9kZSAtMQ0KKFhFTikg
MDAwMDowMjowMC4wIC0gZDAgLSBub2RlIC0xDQooWEVOKSAwMDAwOjAxOjAwLjAgLSBkMCAtIG5v
ZGUgLTENCihYRU4pIDAwMDA6MDA6MDAuMCAtIGQwIC0gbm9kZSAtMQ0KKFhFTikgKioqIFNlcmlh
bCBpbnB1dCB0byBET00wICh0eXBlICdDVFJMLWEnIHRocmVlIHRpbWVzIHRvIHN3aXRjaCBpbnB1
dCkNCg0Kcm9vdEBnZW5lcmljLWFybXY4LXh0LWRvbTA6fiMgbW9kcHJvYmUgZTEwMDBlDQpbwqDC
oCA0Ni4xMDQ3MjldIGUxMDAwZTogSW50ZWwoUikgUFJPLzEwMDAgTmV0d29yayBEcml2ZXINClvC
oMKgIDQ2LjEwNTQ3OV0gZTEwMDBlOiBDb3B5cmlnaHQoYykgMTk5OSAtIDIwMTUgSW50ZWwgQ29y
cG9yYXRpb24uDQpbwqDCoCA0Ni4xMDcyOTddIGUxMDAwZSAwMDAwOjAzOjAwLjA6IGVuYWJsaW5n
IGRldmljZSAoMDAwMCAtPiAwMDAyKQ0KKFhFTikgbWFwIFtlMDAwMCwgZTAwMWZdIC0+IDB4ZTAw
MDAgZm9yIGQwDQooWEVOKSBtYXAgW2UwMDIwLCBlMDAzZl0gLT4gMHhlMDAyMCBmb3IgZDANClvC
oMKgIDQ2LjE3ODUxM10gZTEwMDBlIDAwMDA6MDM6MDAuMDogSW50ZXJydXB0IFRocm90dGxpbmcg
UmF0ZSAoaW50cy9zZWMpIHNldCB0byBkeW5hbWljIGNvbnNlcnZhdGl2ZSBtb2RlDQpbwqDCoCA0
Ni4xODk2NjhdIHBjaV9tc2lfc2V0dXBfbXNpX2lycXMNClvCoMKgIDQ2LjE5MTAxNl0gbndsX2Nv
bXBvc2VfbXNpX21zZyBtc2cgYXQgZmU0NDAwMDANCihYRU4pIHRyYXBzLmM6MjAxNDpkMHYwIEhT
Uj0weDAwMDAwMDkzODEwMDQ3IHBjPTB4ZmZmZjgwMDAxMDRiNGIwMCBndmE9MHhmZmZmODAwMDEw
ZmE1MDAwIGdwYT0weDAwMDAwMGUwMDQwMDAwDQpbwqDCoCA0Ni4yMDA0NTVdIFVuaGFuZGxlZCBm
YXVsdCBhdCAweGZmZmY4MDAwMTBmYTUwMDANCg0KW3NuaXBdDQoNClvCoMKgIDQ2LjIzMzA3OV0g
Q2FsbCB0cmFjZToNClvCoMKgIDQ2LjIzMzU1OV3CoCBfX3BjaV93cml0ZV9tc2lfbXNnKzB4NzAv
MHgxODANClvCoMKgIDQ2LjIzNDE0OV3CoCBwY2lfbXNpX2RvbWFpbl93cml0ZV9tc2crMHgyOC8w
eDMwDQpbwqDCoCA0Ni4yMzQ4NjldwqAgbXNpX2RvbWFpbl9hY3RpdmF0ZSsweDVjLzB4ODgNCg0K
IEZyb20gdGhlIGFib3ZlIHlvdSBjYW4gc2VlIHRoYXQgQkFScyBhcmUgbWFwcGVkIGZvciBEb21h
aW4tMCBub3cNCg0Kb25seSB3aGVuIGFuIGFzc2lnbmVkIFBDSSBkZXZpY2UgZ2V0cyBlbmFibGVk
IGluIERvbWFpbi0wLg0KDQpBbm90aGVyIHRoaW5nIHRvIG5vdGUgaXMgdGhhdCB3ZSBjcmFzaCBv
biBNU0ktWCBhY2Nlc3MgYXMgQkFScyBhcmUgbWFwcGVkDQoNCnRvIHRoZSBkb21haW4gd2hpbGUg
ZW5hYmxpbmcgbWVtb3J5IGRlY29kaW5nIGluIHRoZSBDT01NQU5EIHJlZ2lzdGVyLA0KDQpidXQg
TVNJLVggYXJlIGhhbmRsZWQgZGlmZmVyZW50bHksIGUuZy4gd2UgaGF2ZSBNU0ktWCBob2xlcyBp
biB0aGUgbWFwcGluZ3MuDQoNClRoaXMgaXMgYmVjYXVzZSBiZWZvcmUgdGhpcyBjaGFuZ2UgdGhl
IHdob2xlIFBDSSBhcGVydHVyZSB3YXMgbWFwcGVkIGludG8NCg0KRG9tYWluLTAgYW5kIGl0IGlz
IG5vdC4gVGh1cywgTVNJLVggaG9sZXMgYXJlIGxlZnQgdW5tYXBwZWQgbm93IGFuZCB0aGVyZQ0K
DQp3YXMgbm8gbmVlZCB0byBkbyBzbywgZS5nLiB0aGV5IHdlcmUgYWx3YXlzIG1hcHBlZCBpbnRv
IERvbWFpbi0wIGFuZA0KDQplbXVsYXRlZCBmb3IgZ3Vlc3RzLg0KDQpQbGVhc2Ugbm90ZSB0aGF0
IG9uZSBjYW5ub3QgdXNlIHhsIHBjaS1hdHRhY2ggaW4gdGhpcyBjYXNlIHRvIGF0dGFjaCB0aGUg
UENJIGRldmljZQ0KDQppbiBxdWVzdGlvbiB0byBEb21haW4tMCBhcyAocGxlYXNlIHNlZSB0aGUg
bG9nKSB0aGF0IGRldmljZSBpcyBhbHJlYWR5IGF0dGFjaGVkLg0KDQpOZWl0aGVyIGl0IGNhbiBi
ZSBkZXRhY2hlZCBhbmQgcmUtYXR0YWNoZWQuIFNvLCB3aXRob3V0IG1hcHBpbmcgTVNJLVggaG9s
ZXMgZm9yDQoNCkRvbWFpbi0wIHRoZSBkZXZpY2UgYmVjb21lcyB1bnVzYWJsZSBpbiBEb21haW4t
MC4gQXQgdGhlIHNhbWUgdGltZSB0aGUgZGV2aWNlDQoNCmNhbiBiZSBzdWNjZXNzZnVsbHkgcGFz
c2VkIHRvIERvbVUuDQoNCg0KSnVsaWVuLCBTdGVmYW5vISBQbGVhc2UgbGV0IG1lIGtub3cgaG93
IGNhbiB3ZSBwcm9jZWVkIHdpdGggdGhpcy4NCg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:12:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186394.335122 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5QJ-0001rD-Rq; Tue, 14 Sep 2021 10:12:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186394.335122; Tue, 14 Sep 2021 10:12:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ5QJ-0001r6-N5; Tue, 14 Sep 2021 10:12:19 +0000
Received: by outflank-mailman (input) for mailman id 186394;
 Tue, 14 Sep 2021 10:12:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ5QJ-0001r0-AP
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:12:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a5d31518-03ec-4631-9705-0a1817a4bd98;
 Tue, 14 Sep 2021 10:12:18 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-VW3AQTw4M9ihvmoOn2TSnw-1; Tue, 14 Sep 2021 12:12:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2333.eurprd04.prod.outlook.com (2603:10a6:800:28::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Tue, 14 Sep
 2021 10:12:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 10:12:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0110.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 14 Sep 2021 10:12:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5d31518-03ec-4631-9705-0a1817a4bd98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631614337;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=tWRvUAxYIVKLmt3Hj6rF4ecxp2VuTUJBR8ODHqBGIPE=;
	b=VOhKnUc+emKMNVuXv5jE8hXhi4OMXSwi3CMwFR7WLNEXh260ABkahkR0yFC07nNs0Bb20D
	GBA1RNlBm57cBClNHtu8LUf7bWvdP4oAJZv+kEDa/KV9gGKvQzIkPQj2NlqhL4S+RVzZsy
	8/1wKQ+D7GXJwvgNwogCg4v1qin8x/c=
X-MC-Unique: VW3AQTw4M9ihvmoOn2TSnw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BGGtxP4zvItdyzcIOrIYSts2nMULKasb6YJLUOyKmDzwzpcVVG1ILwq0QytFjjG4oKOU1CgFHrdx9SCDoNGXIpT0jB/AUe5LP26f+bQRXeuWE7QEufvQ8235Sz+wbB1b4v+cILpdhb4Ww6m2vWfQwObcX67iNjruOcawhsK+9630RUIhYe32h89RzQhG40zjGVN7AUMKK3UzMj6f9ilsv5+d1Q2ZIVA9JKUamtGR87QOgDbWzUmMvZf4T0PglmL2PcHMKPJ3YhGo5/vX1+EsPnMlEAk0ns7530w0k2lc7p9ECisMHYA2VgTqjuvzeE0eRgejSGQR4rxy1lcp1AaCXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ByWGbETjwnvuTkWHHaztDcj1odn98swWY3HS7XHbdmg=;
 b=QYgUEZH/gCFaVTmMhVGvCASuZDfTnpngQWvXeIGjedh+E5dbdRE0ts/uNVK45s9XJqSI6h4GjeAup9jKyEjVJha0MlcjGvDphHMJmIEnmrwmGehIp6XTN7y/K+b4GQDZQxaGgOwMAN1AesrnJ+HcSxuhDqAyDybrqKbOwT1NXciY2L3Kdu718WkbLblYMXnNDLX8jqbeMLzdaHRC/jqhiZ2YBbMLT7WpSTRQ4U6EBJ+WAhwtxkRzeOWczaupHg9tMkRDxU44ejEUGxUHqW4gC5TUxaehvongLGk6cVc3V2S7NlbtjLBvGFamGlwJ5WZNkZ8hNinbUj2lq5C6gfgwFQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
Date: Tue, 14 Sep 2021 12:12:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0110.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1741f2ff-b045-4ddc-129d-08d977681f53
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2333407FB17A95846FA8DC48B3DA9@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	q+q10zAIFVyNALzLimufK+8uVwVgKrezRZD6iTFjC4VZUl8UL2y1udlTqDL2Psxo32jPT36uAjkMiTW4sSAM3CYigAYdnn6TBVyuf6Umm/Z39Y/RfThp5MJkI2UmNKQF8+XDF0SkGTefc8eZayB8mJ8m5V9xfLNI1nvvSfLt4v2s/9j/n9t5senxXuSRAT3VZiPzLYR9G/E+mhRio395ccMoCHw4Jr2nDCC7QhaQMG+jmrEhRa/ZUaTXlFmQPjQCPl/JM9cw39opwu13VKs7fRTHoEKqIam4H+ZM5ZoZhSFWS2Jb78yYJTZ/D+X5smVmx0IWj0ByMcwubrOnisEKiZcYnkiBDxsI2N2heyMgOXfwMC0s8gPiJHx5zYH+ZP11aKc8d5rBflOrZfAkrf74xVCATC4cGT7/C6GIMzVgIqnE8461Vl/qOpj2r4iUX5son6QU2+Lso62It3KSX5x7RjdNqJfB1Qv7du0ghklHuFVP+LWZJ0/YcIbMaDlVLTZEJAFDWVkJW6OqozfPpOeZTbcnUPlQDaM5pcELi1zUxLzq75J8qm64AEbIYUSua8rNTxP3pi5/HJS7aci3x6pJGJdZLomuOH0berpzpgP/rVejByhi2t8NOzV3d/8FyfllfjFB+zSSRZlG15hhCENr/SJ0U00xDaPMGu8RlQRW0uzKrgk9AHwEYetAc04tkkZXknooopAOYV+0OKC/K+awKgB8bay0kOb5iQ+8mtHyD1TEPXbiqVMcl4twk0Y573M+
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(376002)(396003)(346002)(366004)(478600001)(6916009)(2906002)(36756003)(66946007)(5660300002)(16576012)(316002)(38100700002)(6486002)(15650500001)(66556008)(956004)(66476007)(86362001)(4326008)(186003)(2616005)(31686004)(26005)(8676002)(8936002)(53546011)(83380400001)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?/5JybJXbeRMq+d8d70wbLDN4/fQHYdqandxvH7b+T0HiLCS852R6PkYAmhrq?=
 =?us-ascii?Q?n1XP54zqCAvjNfIRCXbvoD4iozH/G7xux3kfsrJCwurDefrf6PHQCHrrf7BP?=
 =?us-ascii?Q?IRU+086ZBoWZOTD4X92KxFdPwz1n8DDh7HyYmDXLwSJjb9SSANevi2lD2Ntk?=
 =?us-ascii?Q?rDIVKITxXpOIgC5NT3sYYd4xzS2+LxWrUp6kVq+7ZEpa4Qe2HKzndkJthTkP?=
 =?us-ascii?Q?fJZF5PFKBfhzXUVNKZsxP+534R3W7cuRryxbxWDmwnOHtVXifAMtghD1RUOA?=
 =?us-ascii?Q?MrxxHb4Vn0adM77xey33p+Eq+xk8xoH3G3eJhVd5Pr4lXYS+Gh8Vi1mgjV5Z?=
 =?us-ascii?Q?pOornRDKcvtsE/zi4OdBm/+Yz1OqfAhCRN659bwg34LqcNa/fzB31II+Q5A3?=
 =?us-ascii?Q?FioqtN+V9x8Eth3ZiBp2uS7Pd/VtOmlko9GW4VowFmxw7H3HSjD7vFyzVCc7?=
 =?us-ascii?Q?Lj6jTz33X4bLqIt21EnPSDfsSEwUZl9oVFFHKjMqFmKDdoGSSaTxDsdzI2BP?=
 =?us-ascii?Q?dcx66MeWPRjAzNwAqa2up6/dCecggvHduETx7fnnJbBHUItY/T4LVWdhgcBn?=
 =?us-ascii?Q?kWEyK8wbl336jg7/ay1g2jDgqBzmJWBEGIrZ1IN9gktB9RJhQh2VEd7u3Mn9?=
 =?us-ascii?Q?pBzkN/RdRRIFP0JKmEU0uSPx9lTa4pBHoD+ckY+Qzl/kK6T1Jh3g+nTClzBk?=
 =?us-ascii?Q?cCVoEW+lyi6QyeBRel3gWQwVnNyHrt3O4wwEIMIlPTjr/kH5ltFUcHnBNV5i?=
 =?us-ascii?Q?7dHE4xgzFzcA1i1/VKjN6f61dFvATkiSLKYpAUDRKF703wZHMm1eHN5jnq3v?=
 =?us-ascii?Q?LKUFonPztLMMQ61KVLWnf4qVh5nkawtUVvvYPm4tbnz128VpO5uB1gEgr+um?=
 =?us-ascii?Q?8RbqhVsxaBg41tYxYvwpgxI+fQOsO/qLVnZtqZBSKDswCaDlyhSpDTrIFE19?=
 =?us-ascii?Q?VjG1E/itHHqjKxUA3oKRMtcBJWFKk6PZDjVxjqms0bkvNCD8Y/mOUEwOn3u0?=
 =?us-ascii?Q?CnmY7ePrIJh91c3Wy2zNEsovzuqWqmHOTFQCzdENjhig0+ReVSjmMgWaE/a0?=
 =?us-ascii?Q?G8d8q6skEA3mWhWhJsYpO6S+yZao17kfZQL4smJOA69kGZsAzqZOPbXLMLON?=
 =?us-ascii?Q?WZxOeuI/JgSdLQIOFxTVDR9JCPVr+f4WhzVm5o5NuMQl42QW+lep11NeY0pg?=
 =?us-ascii?Q?wmHlbT4VATay8+hT5j3kP6y7MPl7TV/3xZ8I1s4HYwVIUSRyKssxsTmOYeYd?=
 =?us-ascii?Q?zCo+VlIPL4WmiwiQOo19KoE+fWF8BKtOiw+gt1CfEsJ/rJKWltZAux+/H7rL?=
 =?us-ascii?Q?5ziiCzdccEr2MaLRjaV6Q+Ii?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1741f2ff-b045-4ddc-129d-08d977681f53
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 10:12:13.6687
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ri9BO19Fz4Affk9Lut/YYDCPYd88DlP+KsjlqxV0FFGg3hY5/D0s/7nC1q5cpDLSnmSeVSQ4xUN44YzhmNcbZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

On 14.09.2021 12:00, Roger Pau Monn=C3=A9 wrote:
> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
>> console) are associated with DomXEN, not Dom0. This means that while
>> looking for overlapping BARs such devices cannot be found on Dom0's
>> list of devices; DomXEN's list also needs to be scanned.
>=20
> Thanks for looking into this, I certainly didn't take hidden devices
> into account for vPCI dom0.
>=20
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
>>      would make the diff difficult to read. At this point I'd merely
>>      like to gather input towards possible better approaches to solve
>>      the issue (not the least because quite possibly there are further
>>      places needing changing).
>=20
> I have a couple of questions, AFAICT we currently hide the serial
> console and/or the VGA adapter if it's in use by Xen.
>=20
> I wonder whether we need to add vPCI handlers for those:
> setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
> device because of the temporary override of pdev->domain done in
> _setup_hwdom_pci_devices, but I think that for hidden devices we
> shouldn't add vPCI handlers. We should instead block PCI config space
> access from dom0 to the device so that it doesn't mess with Xen usage.

The answer to this follows (I think) from the one below.

> It's also not clear why does Xen want to have those hidden devices
> partly controlled by dom0, as it would seem to me that dom0 interfering
> with hidden devices in use by Xen can only lead to trouble.

Dom0 can't interfere as long as it can only read from the device.
Restricting accesses to reads is one of the purposes of "hiding"
(the other is to make it impossible to assign these to a DomU). Not
allowing Dom0 to read from such devices would lead to wrong PCI
device discovery - some devices would be missing (which may or may
not be merely a cosmetic issue). If the missing device is a multi-
function one at function 0, other devices in the same slot may also
not be found by Dom0 (depending on whether it looks at non-zero
function numbers in this case).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:49:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:49:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186411.335132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ60N-0005cE-K5; Tue, 14 Sep 2021 10:49:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186411.335132; Tue, 14 Sep 2021 10:49:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ60N-0005c7-HF; Tue, 14 Sep 2021 10:49:35 +0000
Received: by outflank-mailman (input) for mailman id 186411;
 Tue, 14 Sep 2021 10:49:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v6J8=OE=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mQ60M-0005c1-Nb
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:49:34 +0000
Received: from out1-smtp.messagingengine.com (unknown [66.111.4.25])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1fc0724e-6372-4ee7-8913-11cf3edeec0d;
 Tue, 14 Sep 2021 10:49:32 +0000 (UTC)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailout.nyi.internal (Postfix) with ESMTP id 79C745C0212;
 Tue, 14 Sep 2021 06:49:32 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Tue, 14 Sep 2021 06:49:32 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 14 Sep 2021 06:49:30 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1fc0724e-6372-4ee7-8913-11cf3edeec0d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=tHeJnF
	1k6ZyHrxR5t5ZIhno68yHo7xZ7224tLpR6g/I=; b=APRgfJAdR+9vVcfDaslR2x
	5+B8MflgQL6X4jjhWtV4ne9qsJmsiuX4sZmXojOFaqfhbI7CVfqShb8eN+U5V9Ta
	EZOEMFo0kLQwaqaOAWlaAJqcYZi0uL8t8QQZfTd++gkAyTKQI2H34KlCHnJys1mv
	0L0l2l+p9qxg3ofLU2J6fV42fGlB1RB/kJl2qKJulO2baFkDiCTYxqcmu/PLIc5N
	xtuz61NqBziez18OaJJH6/YFBZGJzEuLTFrxzSuJNLdLg14DyugG7KUd8+RQznkX
	kbhU4v4ejXiozBECA+6tZfw7tJJaDj9blxxblKMjeBkiF+SL4D/KuOlvtmHqc+Fw
	==
X-ME-Sender: <xms:O35AYb8RvWQBkRZ6Mh-KmCgSn2H2pHuiAjuLxhLmIMoMisgOADNnpg>
    <xme:O35AYXtEfAhPQG0YW5bj7pZFisfR80uL8T3miWOJGApUeR6t7vRknII2ns4PA55Jl
    hH9pOFNOnJ3pQ>
X-ME-Received: <xmr:O35AYZACNAUmJma3BROdGO2R1Ps9ipSreOOHL2-HkksunEHuu1ldTYlqM-XJdcmfPG9YkELajcbs9GoxIO8UvXN8kNRD2fV7>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudegledgfeduucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeetveff
    iefghfekhffggeeffffhgeevieektedthfehveeiheeiiedtudegfeetffenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:O35AYXfjFXMBjdJLsWzx6d4CBg0OZNW6A6nwNoRqa9ZwG4d1woANTg>
    <xmx:O35AYQOIpsrWxQHr1MDWDrrFdH1PoUWbaA1cIHp0kwIodM5EPgyIwQ>
    <xmx:O35AYZk3xWTDMCBrzCaUvM__NwPg3lchBp5Qy1i3WfOxpJdM5DDogw>
    <xmx:PH5AYcBkHmQFWFAwvg_Nmjyzc2-ciSOPgMcdlsJyEz9kSJmYZhGvjA>
Date: Tue, 14 Sep 2021 12:49:27 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, stable@vger.kernel.org
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUB+N+bVrd4TCt9X@mail-itl>
References: <20210914094108.22482-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="Ng3gYZ7HeyL14wKB"
Content-Disposition: inline
In-Reply-To: <20210914094108.22482-1-jgross@suse.com>


--Ng3gYZ7HeyL14wKB
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 14 Sep 2021 12:49:27 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, stable@vger.kernel.org
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier

On Tue, Sep 14, 2021 at 11:41:08AM +0200, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
>=20
> Move the call of early_reserve_memory() to the beginning of
> setup_arch() in order to avoid such problems.
>=20
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservatio=
ns")
> Reported-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab=
=2Ecom>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Thanks, it works!

Tested-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>

> ---
>  arch/x86/kernel/setup.c | 24 ++++++++++++------------
>  1 file changed, 12 insertions(+), 12 deletions(-)
>=20
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..f369c51ec580 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsi=
gned long v, void *p)
> =20
>  void __init setup_arch(char **cmdline_p)
>  {
> +	/*
> +	 * Do some memory reservations *before* memory is added to
> +	 * memblock, so memblock allocations won't overwrite it.
> +	 * Do it after early param, so we could get (unlikely) panic from
> +	 * serial.
> +	 *
> +	 * After this point everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not
> +	 * RAM in e820. All other memory is free game.
> +	 */
> +	early_reserve_memory();
> +
>  #ifdef CONFIG_X86_32
>  	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
> =20
> @@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
> =20
>  	parse_early_param();
> =20
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> --=20
> 2.26.2
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--Ng3gYZ7HeyL14wKB
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmFAfjcACgkQ24/THMrX
1yyiDAf/cbECScwjsEcDuWCxieb4bXydSDNTKkPbdFll3q6+HMw1aHcvhY/WBej7
xt4vOb/dbqLjRBSPH3w3vCVw8r2zGk8QjhojMNN/8uRqkDpax/KsutFiEs8XsfTC
3NME49YGNCXocNGHIVF79/PNZpKpPIHN40CA/B8r5/OqM6DEBKt9KNWDX7Z9fb/o
s/NdqU/1hFWswunAL87hJ3e/XU8pu1mxP794sAq8XRCwBQjlKoBrdHDBF6QlLau8
Q2qx8U9y7sWNsf4hsGXkgcjLvy7R+zOCvzzXtF4m0SkueVPOm4ilUG4/65X+kcy5
3i+rBfYZ41Qb8csPFnFJOPud0tuTdQ==
=mhfL
-----END PGP SIGNATURE-----

--Ng3gYZ7HeyL14wKB--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 10:51:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 10:51:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186421.335143 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ620-0006wE-VU; Tue, 14 Sep 2021 10:51:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186421.335143; Tue, 14 Sep 2021 10:51:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ620-0006w7-SR; Tue, 14 Sep 2021 10:51:16 +0000
Received: by outflank-mailman (input) for mailman id 186421;
 Tue, 14 Sep 2021 10:51:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v6J8=OE=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mQ620-0006w1-0t
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 10:51:16 +0000
Received: from out1-smtp.messagingengine.com (unknown [66.111.4.25])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id aec54248-1549-11ec-b415-12813bfff9fa;
 Tue, 14 Sep 2021 10:51:15 +0000 (UTC)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailout.nyi.internal (Postfix) with ESMTP id C474A5C0069;
 Tue, 14 Sep 2021 06:51:14 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute3.internal (MEProxy); Tue, 14 Sep 2021 06:51:14 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 14 Sep 2021 06:51:12 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aec54248-1549-11ec-b415-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=K0U22i
	YdgaWPJNYzZWHJRK3jLpUaQw4t+41HSf01KMU=; b=Vf8j1Sqc4VqqlWcLbEaj6e
	OlwA80NMgML4FPQNtCOl/k0C8blxk68lpZv1yTNxoweFpLAU/oseEyGUTf+czHxL
	eTjLLJcmM6KYS/71/s4oBM+68gslak72kkYdGf5wyETyzlhsqPNiXkvYaCFHNUzV
	xyDcheXSCX5ysn7s9dz5kT2yoqhl90Hp+MEhhov0FfNdALacreTOolfgOlKE6LZo
	OJdnqLwCLint6ZF6L/8Ts8DOkbzQKt3AAA3wMds47X3zU5fPOg/wCDB4WeydkpCw
	28juCdhN4KsvgNzL4drJwQ1JAPG6RRu1QfljsuEa7xm1lypSGMfSgvfRI8nHWhYg
	==
X-ME-Sender: <xms:oX5AYf4LpqYIAaN3wt7fmbcNfrwpYbTvy_XA3HauBlY3xHzdkywQbQ>
    <xme:oX5AYU5rl709ozD3fsoxpy_wCKzjx_ouDHPqFA6OVm2Jdx4KWrPHUJNk2HHH13b1B
    AWK-2iAMVfzug>
X-ME-Received: <xmr:oX5AYWccKUsxHBLKxh_GF6lEdY945rsy5YalLh4p0BGrLLcwnjydmlA9L_atxbx48yIJsMtLpNuJ8HBnijve7aKx7FhLkUE1>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudegledgfedvucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefhfeff
    hfetgeffueekleehhfetkeetgedttdfhiedvteekjedufffgudeiffefjeenucffohhmrg
    hinhepghhithhhuhgsuhhsvghrtghonhhtvghnthdrtghomhenucevlhhushhtvghrufhi
    iigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhish
    hisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:oX5AYQIO3bVKwNtOVQk8behQdR2uoxB7Df4lMJvK-lFStIDadI9kmA>
    <xmx:oX5AYTKH0Os9gtQmhLFx-Yf3Dad1hZ8ekJ8iZckUXRSNinTnNxjvFw>
    <xmx:oX5AYZwYP0DhWEzJMmbA9CfR-VK1wA2a5SgJKy50oP8mpq2ss0jPuw>
    <xmx:on5AYb9sxvbsSbhut3PLLQWQP4lEzh3PMNnmu0x6iHffTfKP8MKaLw>
Date: Tue, 14 Sep 2021 12:51:09 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>, Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@suse.de>, Thomas Gleixner <tglx@linutronix.de>,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	xen-devel <xen-devel@lists.xenproject.org>,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)
Message-ID: <YUB+nbuIgI4vLilj@mail-itl>
References: <YT9I+Xs9wOPVCIVd@mail-itl>
 <923de2f5-3d33-4c02-acec-739fdaf2ae05@suse.com>
 <ed6b9999-7a7b-31db-9e88-efdeecefa45b@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="9IO9vSHELR/sHIWS"
Content-Disposition: inline
In-Reply-To: <ed6b9999-7a7b-31db-9e88-efdeecefa45b@suse.com>


--9IO9vSHELR/sHIWS
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 14 Sep 2021 12:51:09 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Juergen Gross <jgross@suse.com>, Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@suse.de>, Thomas Gleixner <tglx@linutronix.de>,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	xen-devel <xen-devel@lists.xenproject.org>,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: Re: Linux 5.13+ as Xen dom0 crashes on Ryzen CPU (ucode loading
 related?)

On Tue, Sep 14, 2021 at 10:39:10AM +0200, Jan Beulich wrote:
> On 14.09.2021 09:14, Juergen Gross wrote:
> > On 13.09.21 14:50, Marek Marczykowski-G=C3=B3recki wrote:
> >> Hi,
> >>
> >> Since 5.13, the Xen (PV) dom0 crashes on boot, before even printing the
> >> kernel version.
> >> Test environment:
> >>   - Xen 4.14.2
> >>   - AMD Ryzen 5 4500U (reported also on AMD Ryzen 7 4750U)
> >>   - Linux 5.13.13, confirmed also on 5.14
> >>
> >> The crash happens only if the initramfs has earlycpio with microcode.
> >> I don't have a serial console, but I've got a photo with crash message
> >> (from Xen, Linux doesn't managed to print anything):
> >> https://user-images.githubusercontent.com/726704/133084966-5038f37e-00=
1b-4688-9f90-83d09be3dc2d.jpg
> >>
> >> Transcription of some of it:
> >>
> >>      mapping kernel into physical memory
> >>      about to get started
> >>      (XEN) Pagetable walk from ffffffff82810888:
> >>      (XEN)  L4[0x1ff] =3D 0000000332815067 0000000000002815
> >>      (XEN)  L3[0x1fe] =3D 0000000332816067 0000000000002816
> >>      (XEN)  L2[0x014] =3D 0000000334018067 0000000000004018
> >>      (XEN)  L1[0x010] =3D 0000000332810067 0000000000002810
> >>      (XEN) domain_crash_sync called from entry.S: fault at ffff82d0403=
3e790 x86_64/entry.S#domain_crash_page_fault
> >>      (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
> >>      (XEN) ----[ Xen-4.14.2  x86_64  debug=3Dn  Not tainted ]----
> >>      (XEN) CPU:    0
> >>      (XEN) RIP:    e033:[<0000000000000000>]
> >=20
> > The domain's run state seems to be completely clobbered.
> >=20
> > Did you try to boot the kernel with "earlyprintk=3Dxen" to get some idea
> > how far it progressed?
>=20
> I guess without my "xen/x86: allow "earlyprintk=3Dxen" to work for PV Dom=
0"
> "earlyprintk=3Dxen" would need to be accompanied by "console=3Dxenboot". I
> have not tried whether this helps, this is merely a guess from having
> looked at the code relatively recently.

This boot was with "earlyprintk=3Dxen" already, but I didn't know
about "console=3Dxenboot".
Anyway, it seems it isn't relevant anymore.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--9IO9vSHELR/sHIWS
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmFAfp0ACgkQ24/THMrX
1yxj3QgAklzrVFkqulCCimAJBOIO9E4Yn5L9JXT1DXGQ0RH446/gqRWmYU7+Atu2
6r4ZLfsYMIJm0FCKt3DTHvqFr9rWxbb1dOq4HVhLsid3Yp0zJZM39uhRUxKPopG3
rgmE7V7A98pX4RQIw9FGiMiFmUryiAwvMgUI9afdQ1RF71gQwwE1pH8tDgsmMV86
Te2ubFkmYakwszLdT4buG8Bwpc6s+rDcOzgIyruIgYLY20wNHEZm71TjNwqWUr8h
h52671MMCj1/o9uTKPkiqHDSR6EUIwphTyDzKoAMCI5IdND+iIvSmjwqFj8idnNP
ifatm5BSF9Knqy82Gt8CLJnTbkzBJg==
=2izk
-----END PGP SIGNATURE-----

--9IO9vSHELR/sHIWS--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:06:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:06:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186431.335155 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Gh-0000C1-BE; Tue, 14 Sep 2021 11:06:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186431.335155; Tue, 14 Sep 2021 11:06:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Gh-0000Bu-7b; Tue, 14 Sep 2021 11:06:27 +0000
Received: by outflank-mailman (input) for mailman id 186431;
 Tue, 14 Sep 2021 11:06:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ6Gf-0000Bo-Gb
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:06:25 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eee6afe6-4360-4252-a4bb-6faef91203c9;
 Tue, 14 Sep 2021 11:06:24 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 932FE21F51;
 Tue, 14 Sep 2021 11:06:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3A5E013D3F;
 Tue, 14 Sep 2021 11:06:23 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id U4NADC+CQGFKewAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 11:06:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eee6afe6-4360-4252-a4bb-6faef91203c9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631617583; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=5FxNj08CZ7D7xF1jTkAJfDCxzu7SlZTTuvd1FJwKeVk=;
	b=onsgYDREoW47skJj4r49hw+zmvw/8XSZgHOpYdh4JRLlFeDkFQ1RqqVx/JCZa6a5rQEltI
	K7sbB7ijciaQ0LtxT/tj/NtzNgQJ2LJ/JCBYvUbZWdNFu1yErmcDa+ZO1gRZ/vYVAmMmDF
	kW5kgqtxW1smaCkQWoMe5niFADJDIx8=
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
To: Jan Beulich <jbeulich@suse.com>
Cc: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel@lists.xenproject.org,
 x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>
Date: Tue, 14 Sep 2021 13:06:22 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="PYYcy5EsIvN2Mh4E4aFREQNEmA4Z3j9xW"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--PYYcy5EsIvN2Mh4E4aFREQNEmA4Z3j9xW
Content-Type: multipart/mixed; boundary="t0VTqccQOdjCg6wBQWftT76NFO9b4tjpL";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 stable@vger.kernel.org,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel@lists.xenproject.org,
 x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
In-Reply-To: <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>

--t0VTqccQOdjCg6wBQWftT76NFO9b4tjpL
Content-Type: multipart/mixed;
 boundary="------------262AC67502117700DCC747A9"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------262AC67502117700DCC747A9
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 14.09.21 12:03, Jan Beulich wrote:
> On 14.09.2021 11:41, Juergen Gross wrote:
>> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
>> reservations") introduced early_reserve_memory() to do all needed
>> initial memblock_reserve() calls in one function. Unfortunately the
>> call of early_reserve_memory() is done too late for Xen dom0, as in
>> some cases a Xen hook called by e820__memory_setup() will need those
>> memory reservations to have happened already.
>>
>> Move the call of early_reserve_memory() to the beginning of
>> setup_arch() in order to avoid such problems.
>>
>> Cc: stable@vger.kernel.org
>> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reserva=
tions")
>> Reported-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethings=
lab.com>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   arch/x86/kernel/setup.c | 24 ++++++++++++------------
>>   1 file changed, 12 insertions(+), 12 deletions(-)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 79f164141116..f369c51ec580 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, u=
nsigned long v, void *p)
>>  =20
>>   void __init setup_arch(char **cmdline_p)
>>   {
>> +	/*
>> +	 * Do some memory reservations *before* memory is added to
>> +	 * memblock, so memblock allocations won't overwrite it.
>> +	 * Do it after early param, so we could get (unlikely) panic from
>> +	 * serial.
>=20
> Hmm, this part of the comment is not only stale now, but gets actively
> undermined. No idea how likely such a panic() would be, and hence how
> relevant it is to retain this particular property.

Ah, right.

The alternative would be to split it up again. Let's let the x86
maintainers decide which way is the better one.


Juergen

>=20
> Jan
>=20
>> +	 * After this point everything still needed from the boot loader or
>> +	 * firmware or kernel text should be early reserved or marked not
>> +	 * RAM in e820. All other memory is free game.
>> +	 */
>> +	early_reserve_memory();
>> +
>>   #ifdef CONFIG_X86_32
>>   	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
>>  =20
>> @@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
>>  =20
>>   	parse_early_param();
>>  =20
>> -	/*
>> -	 * Do some memory reservations *before* memory is added to
>> -	 * memblock, so memblock allocations won't overwrite it.
>> -	 * Do it after early param, so we could get (unlikely) panic from
>> -	 * serial.
>> -	 *
>> -	 * After this point everything still needed from the boot loader or
>> -	 * firmware or kernel text should be early reserved or marked not
>> -	 * RAM in e820. All other memory is free game.
>> -	 */
>> -	early_reserve_memory();
>> -
>>   #ifdef CONFIG_MEMORY_HOTPLUG
>>   	/*
>>   	 * Memory used by the kernel cannot be hot-removed because Linux
>>
>=20


--------------262AC67502117700DCC747A9
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------262AC67502117700DCC747A9--

--t0VTqccQOdjCg6wBQWftT76NFO9b4tjpL--

--PYYcy5EsIvN2Mh4E4aFREQNEmA4Z3j9xW
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFAgi4FAwAAAAAACgkQsN6d1ii/Ey8P
bQf/bl8cCERGWocCyopsKc1tUuNDgMDxHRw4XwHm/SbBMOXOg1uVMrJ9Ysquf5rCZcn2zHocJplH
PQTvkKRNIr+1NCxdO90lu1tPNBLj0yT6JUk5n/bPEC+EVWTCYU+Eh0t5KNV1W0evto4OrMRU+oQK
R3zN/DfQQts3IxQxfjWA8rOff6HtRgtNEybiJh5OI/00tmIdr8qigad9J1W5s1iFHezLU2RbgjLj
6CBmrKVZdjatj942oBCN/xS1AdvtJHHlheptPSh/gdcFUl0FcOkhDh+C25/A3w9ev1ChLa+P9l46
Wlbop6C+gMGcgSpcpJaU3ivxNbdvze8UQcLardyC3g==
=uz1U
-----END PGP SIGNATURE-----

--PYYcy5EsIvN2Mh4E4aFREQNEmA4Z3j9xW--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:08:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:08:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186441.335165 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Id-0000zc-Q0; Tue, 14 Sep 2021 11:08:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186441.335165; Tue, 14 Sep 2021 11:08:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Id-0000zV-N7; Tue, 14 Sep 2021 11:08:27 +0000
Received: by outflank-mailman (input) for mailman id 186441;
 Tue, 14 Sep 2021 11:08:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mQ6Id-0000zP-8s
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:08:27 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mQ6Id-0003l8-6V
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:08:27 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mQ6Id-0005eB-3x
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:08:27 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mQ6IZ-0005Qw-66; Tue, 14 Sep 2021 12:08:23 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=afkqex+7zKLHh6fQx5wK/YDyb189/2Rwt4TBVhKWsqg=; b=Ennu79YusTvQErUjSQwHzwLExG
	mQ1T9zNdrN9hovo+cznyD3GLaWMyN+La0VMmS6iwOcHsNOKMxKBvy35Wv9w4DKR6hmno7zpttgIw7
	Cr40keGaRaF45aotat78WzwuLpjdX9hzgoOn/e+8LUxhRYur9Qx4XTuuH/CgN+gcActQ=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24896.33446.713051.453627@mariner.uk.xensource.com>
Date: Tue, 14 Sep 2021 12:08:22 +0100
To: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
Cc: xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>,
    Anthony PERARD <anthony.perard@citrix.com>
Subject: Re: [PATCH 1/1] tools/xl: Fix regular expression for autobaloon mode detection based on xen command line.
In-Reply-To: <9dc667d2017e6e69877a2b9859041c1f7d965899.1631555561.git.isaikin-dmitry@yandex.ru>
References: <cover.1631555561.git.isaikin-dmitry@yandex.ru>
	<9dc667d2017e6e69877a2b9859041c1f7d965899.1631555561.git.isaikin-dmitry@yandex.ru>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Dmitry Isaikin writes ("[PATCH 1/1] tools/xl: Fix regular expression for autobaloon mode detection based on xen command line."):
> The case of specifying a negative size was handled incorrectly.

Thank you.  The change looks good to me.  Did you test that after your
change, the functionality works ?

Ian.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:14:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:14:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186451.335177 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6OG-0002QR-GF; Tue, 14 Sep 2021 11:14:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186451.335177; Tue, 14 Sep 2021 11:14:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6OG-0002QK-CS; Tue, 14 Sep 2021 11:14:16 +0000
Received: by outflank-mailman (input) for mailman id 186451;
 Tue, 14 Sep 2021 11:14:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r2So=OE=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQ6OE-0002QE-Rv
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:14:15 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e33a84e1-154c-11ec-b419-12813bfff9fa;
 Tue, 14 Sep 2021 11:14:12 +0000 (UTC)
Received: from AM6PR04CA0050.eurprd04.prod.outlook.com (2603:10a6:20b:f0::27)
 by AM9PR08MB6772.eurprd08.prod.outlook.com (2603:10a6:20b:305::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 11:14:10 +0000
Received: from AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:f0:cafe::4d) by AM6PR04CA0050.outlook.office365.com
 (2603:10a6:20b:f0::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16 via Frontend
 Transport; Tue, 14 Sep 2021 11:14:10 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT046.mail.protection.outlook.com (10.152.16.164) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 11:14:10 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Tue, 14 Sep 2021 11:14:09 +0000
Received: from 6228a7da4f50.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 23A08488-FB4E-436B-B420-272C5FEB8648.1; 
 Tue, 14 Sep 2021 11:14:00 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6228a7da4f50.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 14 Sep 2021 11:14:00 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB3303.eurprd08.prod.outlook.com (2603:10a6:209:40::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep
 2021 11:13:56 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Tue, 14 Sep 2021
 11:13:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e33a84e1-154c-11ec-b419-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nUx3nDRy5r09fvd9vCo8jssIROw0IePISh+2WIu/zwo=;
 b=OpBStSh0Df9SRxA4DroPLD093+9BqbvLfeL96pvd8i83q9eQpeeyovuc7IPvCjkAA6saxlCbyNoeGE38GEvaGrlMupSgwYITU8lM8IdqljtaRNtd5eV6C6fdeY3kYC+lEqIGX4++5YG4OV48rtwJjxxa1rM9EnuAiQtB0FuZC2o=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 83a366cc42947a45
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RWgdsmFXFP1EkeW7GoAloHlKSn4pM8Ef7P4ng/s8NCaXxvXjOhMgxdWG0zNSEwqYAlr/yeldrQ6fWiKg8YE0tFpq2uyT11bWtAPg73nSZRvBbd8rUtP7YOY5loHZ0t+nIPjtEaYZxPLUtSaeZumt6kB1ifxwmyyorKaH1nMOZ7SzpH6So5GO7WwilzqWzaeo9BouPMk1K+rzjOrLmXz/yia9YD6dLSgrS7nIqbxDuPRvSEkJIqcDp6/RcLEB9iZs7xmBA5i91qKPtfr8qPaGRa9vPILFcExULeEgENEdR/VdamtK5IUan/w+OfiV9I1CxEHM/z96I8bDDzI4SLpXOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nUx3nDRy5r09fvd9vCo8jssIROw0IePISh+2WIu/zwo=;
 b=VecawD+Xl6D7J7PrKWVWgu0dQqPaSJZkHKI67JeqCu0/LWxc0vmY4g1qR/tVJ3RBIlzWwZGoMmeAVOymRsr+yHuct1AIx2Vi9Y+bLvQfG/k/Adkr2XCFQcmpxWsDSBmTjl4eUNmz8rEF8L9eX58t6WZlUmyoIGAvn0sTLOic62EmQneXhcjl4nnDj6KNnuaid/PEjzvvG7FFBGkYvfxGxWpmNhAqzUsF3kxNq6kV3eljFRXJ61mYj9n52PxwWmfTED/bsq9vSOKxDk22r+Yr4BhKCSu16cmJvlWfXtIpfvk9JQ9DbNQLcDu/r65C9WuBZtaM9bSi4YMVeVXzabBm7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nUx3nDRy5r09fvd9vCo8jssIROw0IePISh+2WIu/zwo=;
 b=OpBStSh0Df9SRxA4DroPLD093+9BqbvLfeL96pvd8i83q9eQpeeyovuc7IPvCjkAA6saxlCbyNoeGE38GEvaGrlMupSgwYITU8lM8IdqljtaRNtd5eV6C6fdeY3kYC+lEqIGX4++5YG4OV48rtwJjxxa1rM9EnuAiQtB0FuZC2o=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Topic: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Index: AQHXlPK6hMBa2cV9wU+RB+r01dTauquceRAAgAcQPwA=
Date: Tue, 14 Sep 2021 11:13:55 +0000
Message-ID: <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3b6b5a8b-c25b-4ca7-7ca0-08d97770c6c6
x-ms-traffictypediagnostic: AM6PR08MB3303:|AM9PR08MB6772:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB677278120470D8029BE60EBDFCDA9@AM9PR08MB6772.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 oboz8NAZ8wekEy6+ns9yqPt1Rjt3+DTWrUPqg3NShGCIQV8N47vOCKgFcUUf+LToRGrYen5cukJgXZff70lLN8xNfOFPaUQfOZ2/b3RuS4JrsvMFZeZ4qxb9+Ml+Ew4zolCWNbey2zP0lgMjZedCRiQzyAbwfbLITU1DFGVGmfNPZGJjFyeuXtpbzXP67pprkVmXUJaksskixuA/xh38dTEgfUeKi6z35+ORXhzbxCFAUvFdbucB7mcQ9z/yOIPjfi1XnkMKnf8QNDhOCG/6y/nSDrkNpgZeIzUBmTaJAj1MvHnyGNkfqrAN5c+oONeRscmTR1NmIOj4HEqvrCZVDWQiIBO/0bP18FS9WEEEwRefIcFlO2IF1jtJvxP0Dt6Qm3HepKw4SmeF8L0UxVWL0j5NA+sLoq6lKBvYDKk3ZZ1A9QgSklcAEpTFv50+uRBSqXOwB4E4JRjPwZgoYLCKuuXGI+cH92Kx2nebtYRRdkr89ZFjYsCRpktLtXPUOm8lSju4kSSGs0eSVFlfl111c5IFGJuvAgU8n7tHoZVQidj3ATiO/QPEG0KKoN7otEuKvhdzdlUbxm+NNd04roITGFceFungB4LORE2Pj8+CIo/AwJLfUIgnWdzdXXu/zY6sBSrnePnZS1wJt/U9I7vqlqPRACnfsfLElVNmMc3sccIPDR6tYgLMeuYmkzPI9kUlucWGBfRYwWS1HMMpbyke+WSK7rIzo1M8L38vQ/Lj3FklWti+2ulyX6qNFaaMHI13jnlU/XECmLkrWa+sImzecAeAHwo9qV/6A3FaK2mUx7LzS+Y/nsziggLGePpZ7C3QTAmbADprizJ29u4X8k64VPmvbMovbYAfHtrAaSpXdktdks8btmeiw/BdOqTVJ2Oa
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(366004)(136003)(346002)(396003)(39850400004)(64756008)(83380400001)(26005)(30864003)(66556008)(66946007)(66446008)(38100700002)(2616005)(5660300002)(6506007)(53546011)(122000001)(66476007)(33656002)(966005)(71200400001)(6512007)(186003)(6486002)(478600001)(6916009)(54906003)(4326008)(316002)(8936002)(8676002)(2906002)(38070700005)(91956017)(76116006)(86362001)(36756003)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <11466A9DCD1755458982681856D161ED@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3303
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0051b47a-ac42-473b-11b4-08d97770be35
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4CUlPLuaBf37DejQUNBpP25E5tTPLgrCzv+eTmBL0pABcojWck6GyYbv3TOYjzs+wbWOePNGfv3BhRCzq2KN8rz2QDkz7KGSiSlZn3nMLWWtCXW17gcG80PkSzIjsw+yPERnydC1o1poQTIz47Uk7xThn26xsH+mRvD6a7s3Ral7Cc5cNVjgqTZ8X36UdYJqpEg+62ttSZUVmKo6qyAHOSbZx5pf3dwTnFCkRJkDF1Cg9WGV9g1Jd4TC8PU1sIO/hRQ5qM9Q5dmYqUmZ4clgHJjq8+bFPczpIgpKA2Cc7/4bw01NLIhCxSKeG2VKodpFLnaRfKhUX2qKrKAcG1ia7OgECevMazVRDv75f6vKpe9u8lvjteTuwIAS+pQc+9tS9g5vDQXez+sZOYG6hsAM9/UdeVrvF/axJM1Q9hRCOFnUDhHTwnhWdI3sXKZM0RX5HgF1xQ+JgJS6yym/Uq8TyEt3cOPtUYdI/f2g2j1BoLQtZUH96e9V5JQcDYQgofurTevU2B9LlPF571FdQ+4cug8XyZT4/dMKS3qsJhlrJAcA2Dybypwz6dtTSyRg7sfjRlWDNoFL9Pp2Ps0lsauJHTp5S2Z8tX9tX9VHTIV7Bl/iz19xiUZbbxEe936bei3uttmkotPdRszX+VUCKSXsj9NVaRCNNNjHPbWzYg1wzlFmL8AwajmH5rk5pKOv6xcLdrYvvtyLPDqtDy9/Bnb6YG6UXHRBhIx/C/XJKmsMvRIDF3PD0pSobzLVbGR5S+jpk3HL7oW+9EaCifalkb5vGSYGiieNAOvffCj14lC4HHYIVDzbnXiFPOfHEvm33XEwI+I1LDybN2xLvgLsurdpfcqvOFqPKF0JnDlbpE3eF24=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(396003)(39850400004)(136003)(346002)(36840700001)(46966006)(26005)(36860700001)(5660300002)(186003)(478600001)(4326008)(356005)(6862004)(2906002)(33656002)(47076005)(6506007)(53546011)(8676002)(8936002)(82310400003)(83380400001)(107886003)(966005)(86362001)(336012)(6486002)(30864003)(2616005)(82740400003)(70586007)(70206006)(316002)(6512007)(54906003)(36756003)(81166007)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 11:14:10.3765
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3b6b5a8b-c25b-4ca7-7ca0-08d97770c6c6
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6772

SGkgU3RlZmFubywNCg0KPiBPbiAxMCBTZXAgMjAyMSwgYXQgMTI6MjEgYW0sIFN0ZWZhbm8gU3Rh
YmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4gd3JvdGU6DQo+IA0KPiBPbiBUaHUsIDE5
IEF1ZyAyMDIxLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+IEFkZCBzdXBwb3J0IGZvciBQQ0kgZWNh
bSBvcGVyYXRpb25zIHRvIGFjY2VzcyB0aGUgUENJDQo+PiBjb25maWd1cmF0aW9uIHNwYWNlLg0K
Pj4gDQo+PiBTaWduZWQtb2ZmLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2luZ2hAYXJtLmNvbT4N
Cj4+IC0tLQ0KPj4geGVuL2FyY2gvYXJtL3BjaS9NYWtlZmlsZSAgICAgICAgICAgfCAgMSArDQo+
PiB4ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyAgICAgICAgICAgICB8IDYzICsrKysrKysrKysrKysr
KysrKysrKysrKysrKysrDQo+PiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1hY2Nlc3MuYyAgICAgICB8
IDUzICsrKysrKysrKysrKysrKysrKysrKysrKw0KPj4geGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9z
dC1jb21tb24uYyAgfCAxMyArKysrKy0NCj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtZ2Vu
ZXJpYy5jIHwgIDggKysrLQ0KPj4geGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCAgICAgICAgICAg
fCAzMiArKysrKysrKysrKysrKysNCj4+IDYgZmlsZXMgY2hhbmdlZCwgMTY3IGluc2VydGlvbnMo
KyksIDMgZGVsZXRpb25zKC0pDQo+PiBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3Bj
aS9lY2FtLmMNCj4+IA0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUg
Yi94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlDQo+PiBpbmRleCBmM2Q5N2Y4NTllLi42ZjMyZmJi
ZTY3IDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4gKysrIGIv
eGVuL2FyY2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4gQEAgLTIsMyArMiw0IEBAIG9iai15ICs9IHBj
aS5vDQo+PiBvYmoteSArPSBwY2ktYWNjZXNzLm8NCj4+IG9iai15ICs9IHBjaS1ob3N0LWdlbmVy
aWMubw0KPj4gb2JqLXkgKz0gcGNpLWhvc3QtY29tbW9uLm8NCj4+ICtvYmoteSArPSBlY2FtLm8N
Cj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyBiL3hlbi9hcmNoL2FybS9w
Y2kvZWNhbS5jDQo+PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPj4gaW5kZXggMDAwMDAwMDAwMC4u
OTFjNjkxYjQxZg0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9l
Y2FtLmMNCj4+IEBAIC0wLDAgKzEsNjMgQEANCj4+ICsvKg0KPj4gKyAqIENvcHlyaWdodCAoQykg
MjAyMSBBcm0gTHRkLg0KPj4gKyAqDQo+PiArICogQmFzZWQgb24gTGludXggZHJpdmVycy9wY2kv
ZWNhbS5jDQo+PiArICogQ29weXJpZ2h0IDIwMTYgQnJvYWRjb20NCj4+ICsgKg0KPj4gKyAqIFRo
aXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0ZSBpdCBhbmQv
b3IgbW9kaWZ5DQo+PiArICogaXQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJhbCBQ
dWJsaWMgTGljZW5zZSB2ZXJzaW9uIDIgYXMNCj4+ICsgKiBwdWJsaXNoZWQgYnkgdGhlIEZyZWUg
U29mdHdhcmUgRm91bmRhdGlvbi4NCj4+ICsgKg0KPj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBkaXN0
cmlidXRlZCBpbiB0aGUgaG9wZSB0aGF0IGl0IHdpbGwgYmUgdXNlZnVsLA0KPj4gKyAqIGJ1dCBX
SVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9m
DQo+PiArICogTUVSQ0hBTlRBQklMSVRZIG9yIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQ
T1NFLiAgU2VlIHRoZQ0KPj4gKyAqIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIGZvciBtb3Jl
IGRldGFpbHMuDQo+PiArICoNCj4+ICsgKiBZb3Ugc2hvdWxkIGhhdmUgcmVjZWl2ZWQgYSBjb3B5
IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZQ0KPj4gKyAqIGFsb25nIHdpdGggdGhp
cyBwcm9ncmFtLiAgSWYgbm90LCBzZWUgPGh0dHA6Ly93d3cuZ251Lm9yZy9saWNlbnNlcy8+Lg0K
Pj4gKyAqLw0KPj4gKw0KPj4gKyNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiArI2luY2x1ZGUgPHhl
bi9zY2hlZC5oPg0KPj4gKw0KPj4gKy8qDQo+PiArICogRnVuY3Rpb24gdG8gaW1wbGVtZW50IHRo
ZSBwY2lfb3BzIC0+bWFwX2J1cyBtZXRob2QuDQo+PiArICovDQo+PiArdm9pZCBfX2lvbWVtICpw
Y2lfZWNhbV9tYXBfYnVzKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+ICsgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHNiZGYsIHVpbnQzMl90
IHdoZXJlKQ0KPiANCj4gQ29kZSBzdHlsZTogYWxpZ25tZW50DQpBY2suDQo+IA0KPiANCj4+ICt7
DQo+PiArICAgIGNvbnN0IHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnID0gYnJpZGdlLT5z
eXNkYXRhOw0KPj4gKyAgICB1bnNpZ25lZCBpbnQgZGV2Zm5fc2hpZnQgPSBjZmctPm9wcy0+YnVz
X3NoaWZ0IC0gODsNCj4gDQo+IElzIGl0IGEgZ3VhcmFudGVlIHRoYXQgZGV2Zm5fc2hpZnQgPT0g
YnVzX3NoaWZ0IC0gOCwgb3IgaXMgaXQganVzdCBzbw0KPiBmb3IgRUNBTT8NCg0KWWVzIGl0IGlz
IGd1YXJhbnRlZSB0aGF0ICJkZXZmbl9zaGlmdCA9PSBidXNfc2hpZnQgLSA44oCdIGFzIGJ1cyBu
dW1iZXIgd2lsbCBiZSA4IGJpdCBhbHdheXMuDQpidXNfc2hpZnQgY2FuIGJlIGRpZmZlcmVudCBi
YXNlZCBvbiB2ZW5kb3IuDQoNCmh0dHBzOi8vZWxpeGlyLmJvb3RsaW4uY29tL2xpbnV4L2xhdGVz
dC9zb3VyY2UvZHJpdmVycy9wY2kvZWNhbS5jI0wxNzINCmh0dHBzOi8vZWxpeGlyLmJvb3RsaW4u
Y29tL2xpbnV4L2xhdGVzdC9zb3VyY2UvZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktdGh1bmRl
ci1wZW0uYyNMMzANCg0KPiANCj4gDQo+PiArICAgIHZvaWQgX19pb21lbSAqYmFzZTsNCj4+ICsN
Cj4+ICsgICAgcGNpX3NiZGZfdCBzYmRmX3QgPSAocGNpX3NiZGZfdCkgc2JkZiA7DQo+PiArICAg
IHVuc2lnbmVkIGludCBidXNuID0gc2JkZl90LmJ1czsNCj4+ICsNCj4+ICsgICAgaWYgKCBidXNu
IDwgY2ZnLT5idXNuX3N0YXJ0IHx8IGJ1c24gPiBjZmctPmJ1c25fZW5kICkNCj4gDQo+IEdlbnVp
bmUgcXVlc3Rpb246IHNob3VsZCBpdCBiZSBidXNuID49IGNmZy0+YnVzbl9lbmQgPyAgSSBkb24n
dCBrbm93IGlmDQo+IHRoZSByYW5nZSBpbmNsdWRlcyBidXNuX2VuZCBvciBub3QuDQoNCmNmZy0+
YnVzX2VuZCBpbmNsdWRlcyB0aGUgdmFsaWQgbGFzdCBidXMgbnVtYmVyLiAgImJ1c24gPiBjZmct
PmJ1c25fZW5kIiBpcyB2YWxpZCBjaGVjayANCnRvIGNvbmZpcm0gaWYgd2UgYXJlIG5vdCBhY2Nl
c3NpbmcgcGNpIGRldmljZSBvdXRzaWRlIHRoZSB2YWxpZCBidXMgbnVtYmVyLg0KDQo+IA0KPiAN
Cj4+ICsgICAgICAgIHJldHVybiBOVUxMOw0KPj4gKw0KPj4gKyAgICBidXNuIC09IGNmZy0+YnVz
bl9zdGFydDsNCj4+ICsgICAgYmFzZSA9IGNmZy0+d2luICsgKGJ1c24gPDwgY2ZnLT5vcHMtPmJ1
c19zaGlmdCk7DQo+PiArDQo+PiArICAgIHJldHVybiBiYXNlICsgKFBDSV9ERVZGTihzYmRmX3Qu
ZGV2LCBzYmRmX3QuZm4pIDw8IGRldmZuX3NoaWZ0KSArIHdoZXJlOw0KPj4gK30NCj4gDQo+IEkg
dW5kZXJzdGFuZCB0aGF0IHRoZSBhcm0zMiBwYXJ0IGlzIG5vdCBpbXBsZW1lbnRlZCBhbmQgbm90
IHBhcnQgb2YgdGhpcw0KPiBzZXJpZXMsIHRoYXQncyBmaW5lLiBIb3dldmVyIGlmIHRoZSBwbGFu
IGlzIHRoYXQgYXJtMzIgd2lsbCBkeW5hbWljYWxseQ0KPiBtYXAgZWFjaCBidXMgaW5kaXZpZHVh
bGx5LCB0aGVuIEkgaW1hZ2luZSB0aGlzIGZ1bmN0aW9uIHdpbGwgaGF2ZSBhbg0KPiBpb3JlbWFw
IGluIHRoZSBhcm0zMiB2ZXJzaW9uLiBXaGljaCBtZWFucyB0aGF0IHdlIGFsc28gbmVlZCBhbg0K
PiB1bm1hcF9idXMgY2FsbCBpbiBzdHJ1Y3QgcGNpX29wcy4gSSB1bmRlcnN0YW5kIHRoYXQgcGNp
X2VjYW1fdW5tYXBfYnVzDQo+IHdvdWxkIGJlIGEgTk9QIHRvZGF5IGZvciBhcm02NCwgYnV0IEkg
dGhpbmsgaXQgbWFrZXMgc2Vuc2UgdG8gaGF2ZSBpdCBpZg0KPiB3ZSB3YW50IHRoZSBBUEkgdG8g
YmUgZ2VuZXJpYy4NCg0KQXMgcGVyIG15IHVuZGVyc3RhbmRpbmcgd2UgZG9u4oCZdCBuZWVkIHBj
aV9lY2FtX3VubWFwX2J1cyguLikgYXMgSSBkb27igJl0IHNlZSBhbnkgdXNlIGNhc2UgdG8gdW5t
YXAgdGhlIA0KYnVzIGR5bmFtaWNhbGx5LiBXZSBjYW4gYWRkIHRoZSBzdXBwb3J0IGZvciBwZXJf
YnVzX21hcHBpbmcgZm9yIEFSTTMyIG9uY2Ugd2UgaW1wbGVtZW50IGFybTMyIHBhcnQuDQpMZXQg
bWUga25vdyB5b3VyIHZpZXcgb24gdGhpcy4NCg0KPiANCj4gDQo+PiArLyogRUNBTSBvcHMgKi8N
Cj4+ICtjb25zdCBzdHJ1Y3QgcGNpX2VjYW1fb3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0K
Pj4gKyAgICAuYnVzX3NoaWZ0ICA9IDIwLA0KPj4gKyAgICAucGNpX29wcyAgICA9IHsNCj4+ICsg
ICAgICAgIC5tYXBfYnVzICAgICAgICAgICAgICAgID0gcGNpX2VjYW1fbWFwX2J1cywNCj4+ICsg
ICAgICAgIC5yZWFkICAgICAgICAgICAgICAgICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3JlYWQs
DQo+PiArICAgICAgICAud3JpdGUgICAgICAgICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZp
Z193cml0ZSwNCj4+ICsgICAgfQ0KPj4gK307DQo+PiArDQo+PiArLyoNCj4+ICsgKiBMb2NhbCB2
YXJpYWJsZXM6DQo+PiArICogbW9kZTogQw0KPj4gKyAqIGMtZmlsZS1zdHlsZTogIkJTRCINCj4+
ICsgKiBjLWJhc2ljLW9mZnNldDogNA0KPj4gKyAqIHRhYi13aWR0aDogNA0KPj4gKyAqIGluZGVu
dC10YWJzLW1vZGU6IG5pbA0KPj4gKyAqIEVuZDoNCj4+ICsgKi8NCj4+IGRpZmYgLS1naXQgYS94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1hY2Nlc3MuYyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWFjY2Vz
cy5jDQo+PiBpbmRleCBiOTM4MDQ3YzAzLi5mMzlmNmEzYTM4IDEwMDY0NA0KPj4gLS0tIGEveGVu
L2FyY2gvYXJtL3BjaS9wY2ktYWNjZXNzLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNp
LWFjY2Vzcy5jDQo+PiBAQCAtMTUsNiArMTUsNTkgQEANCj4+ICAqLw0KPj4gDQo+PiAjaW5jbHVk
ZSA8eGVuL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDxhc20vaW8uaD4NCj4+ICsNCj4+ICtpbnQgcGNp
X2dlbmVyaWNfY29uZmlnX3JlYWQoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1aW50
MzJfdCBzYmRmLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50MzJfdCByZWcs
IHVpbnQzMl90IGxlbiwgdWludDMyX3QgKnZhbHVlKQ0KPj4gK3sNCj4+ICsgICAgdm9pZCBfX2lv
bWVtICphZGRyID0gYnJpZGdlLT5vcHMtPm1hcF9idXMoYnJpZGdlLCBzYmRmLCByZWcpOw0KPj4g
KyAgICBpZiAoIWFkZHIpIHsNCj4+ICsgICAgICAgICp2YWx1ZSA9IH4wOw0KPiANCj4gSXMgdGhp
cyBhIHN0YW5kYXJkIGVycm9yPyBJZiBzbywgSSB0aGluayB3ZSBzaG91bGQgZGVmaW5lIGl0IHdp
dGggYQ0KPiBtYWNybyAoZS5nLiBJTlZBTElEX1BBRERSKS4NCg0KT2suDQo+IA0KPiANCj4+ICsg
ICAgICAgIHJldHVybiAtRU5PREVWOw0KPj4gKyAgICB9DQo+PiArDQo+PiArICAgIHN3aXRjaCAo
bGVuKQ0KPj4gKyAgICB7DQo+PiArICAgIGNhc2UgMToNCj4+ICsgICAgICAgICp2YWx1ZSA9IHJl
YWRiKGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7DQo+PiArICAgIGNhc2UgMjoNCj4+ICsgICAg
ICAgICp2YWx1ZSA9IHJlYWR3KGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7DQo+PiArICAgIGNh
c2UgNDoNCj4+ICsgICAgICAgICp2YWx1ZSA9IHJlYWRsKGFkZHIpOw0KPj4gKyAgICAgICAgYnJl
YWs7DQo+PiArICAgIGRlZmF1bHQ6DQo+PiArICAgICAgICBCVUcoKTsNCj4gDQo+IEEgQlVHIGhl
cmUgaXMgaGFyc2ggYmVjYXVzZSBpdCBjb3VsZCBiZSBwb3RlbnRpYWxseSBndWVzdC10cmlnZ2Vy
ZWQuIEFuDQo+IEFTU0VSVCB3b3VsZCBiZSBiZXR0ZXIuDQoNCk9rLg0KPiANCj4gDQo+PiArICAg
IH0NCj4+ICsNCj4+ICsgICAgcmV0dXJuIDA7DQo+PiArfQ0KPj4gKw0KPj4gK2ludCBwY2lfZ2Vu
ZXJpY19jb25maWdfd3JpdGUoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1aW50MzJf
dCBzYmRmLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50MzJfdCByZWcsIHVp
bnQzMl90IGxlbiwgdWludDMyX3QgdmFsdWUpDQo+PiArew0KPj4gKyAgICB2b2lkIF9faW9tZW0g
KmFkZHIgPSBicmlkZ2UtPm9wcy0+bWFwX2J1cyhicmlkZ2UsIHNiZGYsIHJlZyk7DQo+PiArICAg
IGlmICghYWRkcikNCj4+ICsgICAgICAgIHJldHVybiAtRU5PREVWOw0KPj4gKw0KPj4gKyAgICBz
d2l0Y2ggKGxlbikNCj4+ICsgICAgew0KPj4gKyAgICBjYXNlIDE6DQo+PiArICAgICAgICB3cml0
ZWIodmFsdWUsIGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7DQo+PiArICAgIGNhc2UgMjoNCj4+
ICsgICAgICAgIHdyaXRldyh2YWx1ZSwgYWRkcik7DQo+PiArICAgICAgICBicmVhazsNCj4+ICsg
ICAgY2FzZSA0Og0KPj4gKyAgICAgICAgd3JpdGVsKHZhbHVlLCBhZGRyKTsNCj4+ICsgICAgICAg
IGJyZWFrOw0KPj4gKyAgICBkZWZhdWx0Og0KPj4gKyAgICAgICAgQlVHKCk7DQo+IA0KPiBTYW1l
IGhlcmUNCk9rLg0KPiANCj4gDQo+PiArICAgIH0NCj4+ICsNCj4+ICsgICAgcmV0dXJuIDA7DQo+
PiArfQ0KPj4gDQo+PiBzdGF0aWMgdWludDMyX3QgcGNpX2NvbmZpZ19yZWFkKHBjaV9zYmRmX3Qg
c2JkZiwgdW5zaWduZWQgaW50IHJlZywNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgaW50IGxlbikNCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL3Bj
aS1ob3N0LWNvbW1vbi5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KPj4g
aW5kZXggOWRkOWIwMjI3MS4uYzU4MjUyN2U5MiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2Fy
bS9wY2kvcGNpLWhvc3QtY29tbW9uLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtY29tbW9uLmMNCj4+IEBAIC02OCw2ICs2OCw3IEBAIHN0YXRpYyB2b2lkIHBjaV9lY2FtX2Zy
ZWUoc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmcpDQo+PiB9DQo+PiANCj4+IHN0YXRpYyBz
dHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmdlbl9wY2lfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25v
ZGUgKmRldiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgY29uc3Qgc3RydWN0IHBjaV9lY2FtX29wcyAqb3BzLA0KPj4gICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGludCBlY2FtX3JlZ19pZHgpDQo+PiB7DQo+PiAg
ICAgaW50IGVycjsNCj4+IEBAIC05Niw2ICs5Nyw3IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2NvbmZp
Z193aW5kb3cgKmdlbl9wY2lfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+IA0K
Pj4gICAgIGNmZy0+cGh5c19hZGRyID0gYWRkcjsNCj4+ICAgICBjZmctPnNpemUgPSBzaXplOw0K
Pj4gKyAgICBjZmctPm9wcyA9IG9wczsNCj4+IA0KPj4gICAgIC8qDQo+PiAgICAgICogT24gNjQt
Yml0IHN5c3RlbXMsIHdlIGRvIGEgc2luZ2xlIGlvcmVtYXAgZm9yIHRoZSB3aG9sZSBjb25maWcg
c3BhY2UNCj4+IEBAIC0xMTEsNiArMTEzLDEzIEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2NvbmZpZ193
aW5kb3cgKmdlbl9wY2lfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+ICAgICBw
cmludGsoIkVDQU0gYXQgW21lbSAlbHgtJWx4XSBmb3IgW2J1cyAleC0leF0gXG4iLGNmZy0+cGh5
c19hZGRyLA0KPj4gICAgICAgICAgICAgY2ZnLT5waHlzX2FkZHIgKyBjZmctPnNpemUgLSAxLCBj
ZmctPmJ1c25fc3RhcnQsIGNmZy0+YnVzbl9lbmQpOw0KPj4gDQo+PiArICAgIGlmICggb3BzLT5p
bml0ICkNCj4+ICsgICAgew0KPj4gKyAgICAgICAgZXJyID0gb3BzLT5pbml0KGNmZyk7DQo+PiAr
ICAgICAgICBpZiAoZXJyKQ0KPj4gKyAgICAgICAgICAgIGdvdG8gZXJyX2V4aXQ7DQo+PiArICAg
IH0NCj4+ICsNCj4+ICAgICByZXR1cm4gY2ZnOw0KPj4gDQo+PiBlcnJfZXhpdF9yZW1hcDoNCj4+
IEBAIC0yMTYsNiArMjI1LDcgQEAgc3RhdGljIGludCBwY2lfYnVzX2ZpbmRfZG9tYWluX25yKHN0
cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2KQ0KPj4gfQ0KPj4gDQo+PiBpbnQgcGNpX2hvc3RfY29t
bW9uX3Byb2JlKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4gKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBjaV9lY2FtX29wcyAqb3BzLA0KPj4gICAgICAgICAg
ICAgICAgICAgICAgICAgICBpbnQgZWNhbV9yZWdfaWR4KQ0KPj4gew0KPj4gICAgIHN0cnVjdCBw
Y2lfaG9zdF9icmlkZ2UgKmJyaWRnZTsNCj4+IEBAIC0yMjcsNyArMjM3LDcgQEAgaW50IHBjaV9o
b3N0X2NvbW1vbl9wcm9iZShzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+ICAgICAgICAg
cmV0dXJuIC1FTk9NRU07DQo+PiANCj4+ICAgICAvKiBQYXJzZSBhbmQgbWFwIG91ciBDb25maWd1
cmF0aW9uIFNwYWNlIHdpbmRvd3MgKi8NCj4+IC0gICAgY2ZnID0gZ2VuX3BjaV9pbml0KGRldiwg
ZWNhbV9yZWdfaWR4KTsNCj4+ICsgICAgY2ZnID0gZ2VuX3BjaV9pbml0KGRldiwgb3BzLCBlY2Ft
X3JlZ19pZHgpOw0KPj4gICAgIGlmICggIWNmZyApDQo+PiAgICAgew0KPj4gICAgICAgICBlcnIg
PSAtRU5PTUVNOw0KPj4gQEAgLTIzNiw2ICsyNDYsNyBAQCBpbnQgcGNpX2hvc3RfY29tbW9uX3By
b2JlKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4gDQo+PiAgICAgYnJpZGdlLT5kdF9u
b2RlID0gZGV2Ow0KPj4gICAgIGJyaWRnZS0+c3lzZGF0YSA9IGNmZzsNCj4+ICsgICAgYnJpZGdl
LT5vcHMgPSAmb3BzLT5wY2lfb3BzOw0KPj4gICAgIGJyaWRnZS0+YnVzX3N0YXJ0ID0gY2ZnLT5i
dXNuX3N0YXJ0Ow0KPj4gICAgIGJyaWRnZS0+YnVzX2VuZCA9IGNmZy0+YnVzbl9lbmQ7DQo+PiAN
Cj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMuYyBiL3hl
bi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtZ2VuZXJpYy5jDQo+PiBpbmRleCAxM2QwZjdmOTk5Li4y
ZDY1MmU4OTEwIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5l
cmljLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtZ2VuZXJpYy5jDQo+PiBA
QCAtMjMsMjAgKzIzLDI0IEBADQo+PiAjaW5jbHVkZSA8YXNtL3BjaS5oPg0KPj4gDQo+PiBzdGF0
aWMgY29uc3Qgc3RydWN0IGR0X2RldmljZV9tYXRjaCBnZW5fcGNpX2R0X21hdGNoW10gPSB7DQo+
PiAtICAgIHsgLmNvbXBhdGlibGUgPSAicGNpLWhvc3QtZWNhbS1nZW5lcmljIiB9LA0KPj4gKyAg
ICB7IC5jb21wYXRpYmxlID0gInBjaS1ob3N0LWVjYW0tZ2VuZXJpYyIsDQo+PiArICAgICAgLmRh
dGEgPSAgICAgICAmcGNpX2dlbmVyaWNfZWNhbV9vcHMgfSwNCj4+ICsNCj4+ICAgICB7IH0sDQo+
PiB9Ow0KPj4gDQo+PiBzdGF0aWMgaW50IGdlbl9wY2lfZHRfaW5pdChzdHJ1Y3QgZHRfZGV2aWNl
X25vZGUgKmRldiwgY29uc3Qgdm9pZCAqZGF0YSkNCj4+IHsNCj4+ICAgICBjb25zdCBzdHJ1Y3Qg
ZHRfZGV2aWNlX21hdGNoICpvZl9pZDsNCj4+ICsgICAgY29uc3Qgc3RydWN0IHBjaV9lY2FtX29w
cyAqb3BzOw0KPj4gDQo+PiAgICAgb2ZfaWQgPSBkdF9tYXRjaF9ub2RlKGdlbl9wY2lfZHRfbWF0
Y2gsIGRldi0+ZGV2Lm9mX25vZGUpOw0KPj4gKyAgICBvcHMgPSAoc3RydWN0IHBjaV9lY2FtX29w
cyAqKSBvZl9pZC0+ZGF0YTsNCj4+IA0KPj4gICAgIHByaW50ayhYRU5MT0dfSU5GTyAiRm91bmQg
UENJIGhvc3QgYnJpZGdlICVzIGNvbXBhdGlibGU6JXMgXG4iLA0KPj4gICAgICAgICAgICBkdF9u
b2RlX2Z1bGxfbmFtZShkZXYpLCBvZl9pZC0+Y29tcGF0aWJsZSk7DQo+PiANCj4+IC0gICAgcmV0
dXJuIHBjaV9ob3N0X2NvbW1vbl9wcm9iZShkZXYsIDApOw0KPj4gKyAgICByZXR1cm4gcGNpX2hv
c3RfY29tbW9uX3Byb2JlKGRldiwgb3BzLCAwKTsNCj4+IH0NCj4+IA0KPj4gRFRfREVWSUNFX1NU
QVJUKHBjaV9nZW4sICJQQ0kgSE9TVCBHRU5FUklDIiwgREVWSUNFX1BDSSkNCj4+IGRpZmYgLS1n
aXQgYS94ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9wY2ku
aA0KPj4gaW5kZXggNThhNTFlNzI0ZS4uMjI4NjYyNDRkMiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9p
bmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgN
Cj4+IEBAIC0zNyw2ICszNyw3IEBAIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyB7DQo+PiAgICAg
dWludDhfdCAgICAgICAgIGJ1c25fc3RhcnQ7DQo+PiAgICAgdWludDhfdCAgICAgICAgIGJ1c25f
ZW5kOw0KPj4gICAgIHZvaWQgX19pb21lbSAgICAqd2luOw0KPj4gKyAgICBjb25zdCBzdHJ1Y3Qg
ICAgcGNpX2VjYW1fb3BzICpvcHM7DQo+PiB9Ow0KPj4gDQo+PiAvKg0KPj4gQEAgLTUwLDEwICs1
MSw0MSBAQCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlIHsNCj4+ICAgICB1OCBidXNfc3RhcnQ7ICAg
ICAgICAgICAgICAgICAgICAvKiBCdXMgc3RhcnQgb2YgdGhpcyBicmlkZ2UuICovDQo+PiAgICAg
dTggYnVzX2VuZDsgICAgICAgICAgICAgICAgICAgICAgLyogQnVzIGVuZCBvZiB0aGlzIGJyaWRn
ZS4gKi8NCj4+ICAgICB2b2lkICpzeXNkYXRhOyAgICAgICAgICAgICAgICAgICAvKiBQb2ludGVy
IHRvIHRoZSBjb25maWcgc3BhY2Ugd2luZG93Ki8NCj4+ICsgICAgY29uc3Qgc3RydWN0IHBjaV9v
cHMgKm9wczsNCj4+IH07DQo+PiANCj4+ICtzdHJ1Y3QgcGNpX29wcyB7DQo+PiArICAgIHZvaWQg
X19pb21lbSAqKCptYXBfYnVzKShzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQz
Ml90IHNiZGYsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50MzJfdCBvZmZz
ZXQpOw0KPj4gKyAgICBpbnQgKCpyZWFkKShzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2Us
IHVpbnQzMl90IHNiZGYsDQo+PiArICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMy
X3QgbGVuLCB1aW50MzJfdCAqdmFsdWUpOw0KPj4gKyAgICBpbnQgKCp3cml0ZSkoc3RydWN0IHBj
aV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1aW50MzJfdCBzYmRmLA0KPj4gKyAgICAgICAgICAgICAg
ICAgdWludDMyX3QgcmVnLCB1aW50MzJfdCBsZW4sIHVpbnQzMl90IHZhbHVlKTsNCj4+ICt9Ow0K
Pj4gKw0KPj4gKy8qDQo+PiArICogc3RydWN0IHRvIGhvbGQgcGNpIG9wcyBhbmQgYnVzIHNoaWZ0
IG9mIHRoZSBjb25maWcgd2luZG93DQo+PiArICogZm9yIGEgUENJIGNvbnRyb2xsZXIuDQo+PiAr
ICovDQo+PiArc3RydWN0IHBjaV9lY2FtX29wcyB7DQo+PiArICAgIHVuc2lnbmVkIGludCAgICAg
ICAgICAgIGJ1c19zaGlmdDsNCj4+ICsgICAgc3RydWN0IHBjaV9vcHMgICAgICAgICAgcGNpX29w
czsNCj4+ICsgICAgaW50ICgqaW5pdCkoc3RydWN0IHBjaV9jb25maWdfd2luZG93ICopOw0KPiAN
Cj4gSXMgdGhpcyBsYXN0IG1lbWJlciBvZiB0aGUgc3RydWN0IG5lZWRlZC91c2VkPw0KDQpZZXMg
SXQgd2lsbCBiZSB1c2VkIGJ5IHNvbWUgdmVuZG9yIHNwZWNpZmljIGJvYXJkKCBOMVNEUCApLiBQ
bGVhc2UgY2hlY2sgYmVsb3cuDQpodHRwczovL2dpdC5saW5hcm8ub3JnL2xhbmRpbmctdGVhbXMv
d29ya2luZy9hcm0vbjFzZHAtcGNpZS1xdWlyay5naXQvY29tbWl0Lz9pZD0wNGI3ZTc2ZDBmZTY0
ODFhODAzZjU4ZTU0ZTAwOGExNDg5ZDcxM2E1DQogDQpSZWdhcmRzLA0KUmFodWw=


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:15:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:15:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186461.335188 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6PT-00034M-0B; Tue, 14 Sep 2021 11:15:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186461.335188; Tue, 14 Sep 2021 11:15:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6PS-00034F-T1; Tue, 14 Sep 2021 11:15:30 +0000
Received: by outflank-mailman (input) for mailman id 186461;
 Tue, 14 Sep 2021 11:15:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ6PR-000347-1g
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:15:29 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1cc70b71-6668-4588-919c-afa071cce71e;
 Tue, 14 Sep 2021 11:15:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cc70b71-6668-4588-919c-afa071cce71e
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631618127;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=fbwPSSqtThBA55/yBXof+39Y2XSi/PWHDb/MFGDwcuQ=;
  b=V4srH+W/b5oOybe+as23mZfN8c/wHsncH08S3Q/CCnh3Vs0YR6Berz2e
   sjkN33aRdrK+tRzE7Cx+pDy+a9sRWp0mBykXCn6O8atiGS/7jsqvgcYpW
   juAK1zRexOcCYXWl4qVZWfE7GZONH8cL0H4RAEfQwveXAn6sDfC0cP155
   k=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: SGBRNgBbIYNKrgd4gVtK1JMQ8wjoUnqbS0HgjD+ezjUM6V+dTukYjFMmRwCXsXV1sic8R/JFu1
 ASTJyz7Bq0eGBqFstwpQIbJ1q+UkNKpYvL+ejivCn0UkE35Wjv0obRmuuNy+n5vNPQygkGrzrw
 XYv/Sfh8RSqFKHsZUjmnr9bsoC2JHHWNjG2wTtsH4/qvIjz8h/ewilwL79sR8VdTWaGD+HDuSl
 VcpyFsIkapWHD+7kGNRK23Sqsf354g/4Vvx5TTC8gxDtDTHwqnbbN8HYzDemDCb9ddq73TWNfU
 nmdFbliqv4E5995I48qiCz29
X-SBRS: 5.1
X-MesageID: 53073338
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Fh3PN6KXqp8i0gJZFE+RJpIlxSXFcZb7ZxGr2PjKsXjdYENS1TcGm
 GAcCzvUO/vcMWr2fIh1O43j9htT6pOHmt8wGgNlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6x7Zg6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB22z9tv9
 9tduKaAEyEjffHynMIwFDRxRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2XvocHh2xr1qiiG96Df
 uknZSRmTy/7TABTG00vFpk6lfm30yyXnzpw9wvO+PtfD3Lo5BdqzLHnPd7Re9qLbcZYhECVo
 iTB5WuRKhUbMsGPjDmI6HShgsfRki7hHoEfDru18rhtmlL77nweDlgaWEW2pdG9i1WiQJRPJ
 koM4C0soKMuskuxQbHVXRe1vXqFtR40QMdLHqsx7wTl4qjb+QGCHUAfUyVMLtchsaceTzsw1
 0TPmMj1Hztxq76EYXWH/7yQoHW5Pi19EIMZTXZaF01fuYCl+dxtyEKUJjp+LEKrpuKlEi/Pz
 SCHlhhkob9Q0fAP8Yj80HmS1lpAuaP1oh4JChT/Bzz+t1kiOd/5OORE+nCAsq0Rd9/xok2p+
 SFewpnAtrhm4YSlyXTVKNjhCo1F8Bps3Nf0uldpA5Bp3DCk4XfLkWt4sWwmeRsB3irpf1bUj
 K7vVeF5v8Q70JiCN/Yfj2eN5yMCl/SIKDgdfqqIBueim7AoHON9wM2LWXN8Iki3yBR8+U3AB
 XtrWZn1VitLYUiW5BG3W/0cwdcW+8zK/kuKHcqT503+idK2PSfJIZ9YYArmRr1ot8us/VSKm
 +uzwuPXkn2zpsWlOXKJmWPSRHhXRUUG6Wfe8JAOLb/ffVM9QQnMyZb5mNscRmCspIwM/s/g9
 XChQE5Ijl35gHzMMwKRbX5/LrjoWP5CQbgTZETA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:PauX/q6C/SVmTWJBawPXwMbXdLJyesId70hD6qkRc3Bom6mj/P
 xG88516faZslgssRMb+exoSZPgfZq0z/cci+Qs1NyZLWrbUQWTXeRfxLqn7zr8GzDvss5xvJ
 0QF5SW0eeAb2RHsQ==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="53073338"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fc0LxzpHkMOEA6wIlUcuxiamtt8XpwZCbSHELpUl2dqbTUBvTx/zHJrGaiEJ3eec/yqd2f1qjSeNohejfsTWxxXWUhxQx18QGOuYhBFNB4+tRRJTfN2wiH7bRPwWt/viYfCmSoei2wsTv0vrtCJZStfsmEiYwUJDbVeYYR+zvMC+47uv6VFe731SOocy4gcYOu01aTDDCHzvl1pgJL9rzvYq6B6Kyr3ealakMnKrPNc2L7yA1CoaEIF1qdmaqkn4w/ISu+N9x7+1z57/egrZ9pI52cSSVU6HZbanxR9wBKsliGCCUsDyovy5kmlboPEWeSgah17c8TMdyTpAGfK86A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vJODM7L9NIrwaWiYPjspGce8Y9XQfJBgNDO92ziqp/g=;
 b=GWAaCu2ErDvoA5Zgp8sZagsze47GWnOlokNouUsI4AVgUtmmKvqNx/3aSGGmEu7eOqj+F/rXMIjMNtV9zKQ6hvVcsXVXzb32bsN+QrevtQo9yjHlqOKhQYI61poRPRKZBaaJBBNJIsRaoYX1UlaeuCmWU2YYi90jhMxuUBFXSkZsmeRyXj4+NDT43OIwPk8pL8uVpMyr8V1RwpCwjWqiBMBIWXIeKu4nEF4aEkj21Ay2G4rPH2zuIHA2YlXwPKMXjoPXaRZZuFBt+P4P9lbvtW4VRWoHtRkwTucGGHR/Y4DJLL9FxBPvunH5FIM/4t0Msuv2dlUxqJxDRb10pBNyiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vJODM7L9NIrwaWiYPjspGce8Y9XQfJBgNDO92ziqp/g=;
 b=pyhAnFZ8TtyAE6C2KZVzMOosBMqd1JJ9HnKdVIpyDpSFLMZ+snItRLcJR4/S0A3H4cSc6+eP/jhUsDg6lMOMLizyFJReTtZhhCyZfiSdAW7Cqn+/7QT9fpnRANOvhSS5K650/ZuUrhC0BF4i1vOILcI5D5IlHfyohgxsAQI+77w=
Date: Tue, 14 Sep 2021 13:15:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Juergen Gross <jgross@suse.com>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
Message-ID: <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
X-ClientProxiedBy: LO4P123CA0093.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:191::8) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9b524273-2c64-47a8-5434-08d97770eec4
X-MS-TrafficTypeDiagnostic: DM5PR03MB2922:
X-Microsoft-Antispam-PRVS: <DM5PR03MB29229691A65D6FCCD1DDAE098FDA9@DM5PR03MB2922.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: eveKJ6ioOjnnX1y/OE4RGiWMHM/f763rHyXHi+cEmawMKbDrWQt09Lokh8eoSzmcFnBDuxEitcQH2nMfJYvfthZ3C7RSB/W4dDv5FLDQSmnc9EWjPdbB+RLz/AtQFAStiq7ZJa1lt1a+RNKC89nGjilTxc6W3lrxDeCLDR7uKqXsgTmGkkgegmrG3MEC6py2X25u8gBcZqYIicAYUQBo3Ggng8peMaE2sHdZzge5nixEr1DscEn3IEVJvX7l8DvBlkJsi4eDdGoK2cO+FKMpWwdNK4Ygpy9CC+fvyxKzwaATFuy0JhrKJ27YUpcKnCfAFxjIk6fRYjfpn5ymMEOjqdzXtkrJyqR6F3DsacPg5+3cXL4HPbQIYzRFyJnfd91zlFxaKt3yRXrrZ15hpbQYyPSNqQ3RQQhVcN/5uBiFBQjnRpIw+R2a+57Tr7LX72YTBFrf//9dhI1DkIM5UFEMOnInUd2658qT/LtgAEygHKz7G8n7ejEziu8uyuajMvdluxlDe2SJ6dx5y7/vvXoMB+XEw+6BZ6eaS9YLSyBeG9poonkLQotkjNEDxUEA+xEH8NituEdYbtQ8BunMHjvqTk1Ngmay7N0NjATcBixzfy8AuGP5QnBdmfDLUy5kPimJBC/MPwmU3uzArzMe2SOVVbKmhP7FN50cLLBZNVLNecgWSh3ctRD/zmtVJgqEg1IETfKFqJQnhRFiYW7DTmyamzQ1OPJPkbS1anJIk2AIFwCsc95Jo7uqvAjG2NQcaGscohAoNsC0MGqsdcwcXBNT3wtbXwBUAPZyKcig0apDnV4=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(366004)(396003)(136003)(2906002)(6666004)(86362001)(6496006)(5660300002)(85182001)(6916009)(4326008)(83380400001)(8936002)(966005)(6486002)(66556008)(478600001)(316002)(9686003)(66946007)(54906003)(8676002)(66476007)(186003)(956004)(38100700002)(53546011)(26005)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?amlYNU5lL1BDZ2s1QmJubFJzUGhxaFVMdmVFZCtrUmtiODc4QWFxcndiU2Mv?=
 =?utf-8?B?Q0Z2dWFHQXQzZjRuZG5ER3Vyb29yVU4yT1NOc2tnUjNRZnhVWTJrejViTVo5?=
 =?utf-8?B?cFY3ekVJcUlXWVhuM1hucGxsbkk5T29iOXpwS0l1VVhnU29JYkFKVDlVQzl1?=
 =?utf-8?B?L2I3RFBqeThISU5Jc2NRR1FjaDA5R0ZPTlNPQkxiSDRqYThZM0VmU01ZcFBF?=
 =?utf-8?B?clFaSTVudVFIUEp3ZFRCQU43YVZkZEhFeWxkVnBSVERkMy9XcG9CT0NpTEFB?=
 =?utf-8?B?bk1hMkFkeDUwaDBseWxXMm1OSzkrNndLWE44Ykg5bDZsU05lMUZybUUwdGpk?=
 =?utf-8?B?eWxVNGtvN08rZHAvRzR3c01tRTI1SlZvU3llMXZvS1V1OU9RaythNHAwR2xD?=
 =?utf-8?B?cG1jelpmaDFVNDZkMVJZbTB6WGtJMXhCUWh4a1pkZSt1NGJHb1cvQ290WDla?=
 =?utf-8?B?VlgxSDBubG5FMiswOVcrZWV6allvSkhTY2VIRlR5bVdsUmUxdGtqcXZpR0E5?=
 =?utf-8?B?ZERCUnZKSi9FZjdHZFRvUFQwTHBlOGY5bTNYVUVuZEgyZXUrNWZQRmNQNDFR?=
 =?utf-8?B?ZS83S05vQmZ5YkhQYWNYK1JkcnlyWVNYUzRpdlhWcjVZcnVQa3JnRTY1aHd5?=
 =?utf-8?B?VmdxMDVaRzNOMXpOQWNNQy9lcGlEU0ZLMDhDTU1uSy9xcnEzL3dqZVhHcFk0?=
 =?utf-8?B?WEFPRCsycGhHc1BjSm1HNVFFd294L1luOW9udm10WHZETmZQSEFIVUUwc0lZ?=
 =?utf-8?B?MEZMbWd1cmE0d2J6RUhtd2lkRzdRd290bitpbDVBQnBiVHFMUXIxTkovbElq?=
 =?utf-8?B?ZlpWam5QY01LcFY2dUZJVDA4R3ZoYWRzUkFnakZwcEdVbW9KTVpvUVZmRG9Y?=
 =?utf-8?B?bm8xQ3o3TjJwWlV1SEtHQTA0Y0Fnbk02L0RyQ21vSDRXZW5qc0IrdUZoamtl?=
 =?utf-8?B?QnM0dVZZYXp3RVlYTTF4cVZmYm5ITm9JNytzTlB4VFMzNkdYa1lYUW1rQ0NQ?=
 =?utf-8?B?NkoyTUI3djZMSzlSTjhTdjlPQ0pvbzlPbUZQUXc0YkNYT1JNT3NTQ29WQVhp?=
 =?utf-8?B?dld1UC9XU3k3TTM0S3lQZ2tYRHNVaFVwTUd4YXhML3BWZXZmaGVOc3VuNTJD?=
 =?utf-8?B?NUYrR0JWZEZHNjZ4R2RwWmxuTTFiVmQ2L3ZzdFpmRHJsUHlScVVuV3lRaVJC?=
 =?utf-8?B?MnRzZXdnVzZ5NzBSNVlFdkJaMnpNS2Zpbk5HNXN6cFhVMWN2ZXd0bmRJMTVF?=
 =?utf-8?B?YkpySlZNTjhpbUlJMm1kdS9HZEZWdkFxTzVPSlFKVzRWTG1qNGg1MFJPMkxI?=
 =?utf-8?B?M25RQzZpNi9QTWFvdzNUQXQyVmpEaFp0MzJRd0pEejN4Y3R4aTNEOU5Ob25o?=
 =?utf-8?B?L3FSSEhKMW1GaE1ObzBPcjNuYStaNGhSWWdiUktHVmNkUXE5SkFZdzdsS0RS?=
 =?utf-8?B?cENTbitLRXVmdlFwc2FDbzB5NXEwYVowWlNFQVgydllZckpRaURmdmIrNFRS?=
 =?utf-8?B?RkZucGNsQTUrUk90Sm81T0FtODhJeG82c0craEFUVCtqSXBKR0FEeFJiRXAv?=
 =?utf-8?B?bDF1bnBJaUczZkhzN3E3eGl5VDBDdUZhUFBGcXlVbGdINGhEeUhHRGpLZmF5?=
 =?utf-8?B?bDhXSmpqNmtTMHcvbVhtalMva1hFdWVXQ0VnV3cvVElSOXBMYW9xWERjK1ZX?=
 =?utf-8?B?ZzdLTUhVUG9CZmxQNHg4c0ZjbUZnM1crM3ludndWbnJTSTRrOER3NlFZYWJF?=
 =?utf-8?Q?S+H4njhYesT/b+H0L1DeCnn+j/bgU1GAk9Fe/Y2?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9b524273-2c64-47a8-5434-08d97770eec4
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 11:15:17.6970
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: buX4q2KgBa+ck5wZZv/i7p959I2i8O+Ne6iDNA7jBjmOG9yB+4WRWHBpxjJZBzDMBdLic0GddyhZRG5X79AS4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2922
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
> On 14.09.2021 10:32, Roger Pau Monné wrote:
> > On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
> >> In order to try to debug hypervisor side breakage from XSA-378 I found
> >> myself urged to finally give PVH Dom0 a try. Sadly things didn't work
> >> quite as expected. In the course of investigating these issues I actually
> >> spotted one piece of PV Dom0 breakage as well, a fix for which is also
> >> included here.
> >>
> >> There are two immediate remaining issues (also mentioned in affected
> >> patches):
> >>
> >> 1) It is not clear to me how PCI device reporting is to work. PV Dom0
> >>    reports devices as they're discovered, including ones the hypervisor
> >>    may not have been able to discover itself (ones on segments other
> >>    than 0 or hotplugged ones). The respective hypercall, however, is
> >>    inaccessible to PVH Dom0. Depending on the answer to this, either
> >>    the hypervisor will need changing (to permit the call) or patch 2
> >>    here will need further refinement.
> > 
> > I would rather prefer if we could limit the hypercall usage to only
> > report hotplugged segments to Xen. Then Xen would have to scan the
> > segment when reported and add any devices found.
> > 
> > Such hypercall must be used before dom0 tries to access any device, as
> > otherwise the BARs won't be mapped in the second stage translation and
> > the traps for the MCFG area won't be setup either.
> 
> This might work if hotplugging would only ever be of segments, and not
> of individual devices. Yet the latter is, I think, a common case (as
> far as hotplugging itself is "common").

Right, I agree to use hypercalls to report either hotplugged segments
or devices. However I would like to avoid mandating usage of the
hypercall for non-hotplug stuff, as then OSes not having hotplug
support don't really need to care about making use of those
hypercalls.

> Also don't forget about SR-IOV VFs - they would typically not be there
> when booting. They would materialize when the PF driver initializes
> the device. This is, I think, something that can be dealt with by
> intercepting writes to the SR-IOV capability.

My plan was to indeed trap SR-IOV capability accesses, see:

https://lore.kernel.org/xen-devel/20180717094830.54806-1-roger.pau@citrix.com/

I just don't have time ATM to continue this work.

> But I wonder whether
> there might be other cases where devices become "visible" only while
> the Dom0 kernel is already running.

I would consider those kind of hotplug devices, and hence would
require the use of the hypercall in order to notify Xen about them.

> >> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
> >>    console) when in a non-default mode (i.e. not 80x25 text), as the
> >>    necessary information (in particular about VESA-bases LFB modes) is
> >>    not communicated. On the hypervisor side this looks like deliberate
> >>    behavior, but it is unclear to me what the intentions were towards
> >>    an alternative model. (X may be able to access the screen depending
> >>    on whether it has a suitable driver besides the presently unusable
> >>    /dev/fb<N> based one.)
> > 
> > I had to admit most of my boxes are headless servers, albeit I have
> > one NUC I can use to test gfx stuff, so I don't really use gfx output
> > with Xen.
> > 
> > As I understand such information is fetched from the BIOS and passed
> > into Xen, which should then hand it over to the dom0 kernel?
> 
> That's how PV Dom0 learns of the information, yes. See
> fill_console_start_info(). (I'm in the process of eliminating the
> need for some of the "fetch from BIOS" in Xen right now, but that's
> not going to get us as far as being able to delete that code, no
> matter how much in particular Andrew would like that to happen.)
> 
> > I guess the only way for Linux dom0 kernel to fetch that information
> > would be to emulate the BIOS or drop into realmode and issue the BIOS
> > calls?
> 
> Native Linux gets this information passed from the boot loader, I think
> (except in the EFI case, as per below).
> 
> > Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
> > info using the PV EFI interface?
> 
> There it's EFI boot services functions which can be invoked before
> leaving boot services (in the native case). Aiui the PVH entry point
> lives logically past any EFI boot services interaction, and hence
> using them is not an option (if there was EFI firmware present in Dom0
> in the first place, which I consider difficult all by itself - this
> can't be the physical system's firmware, but I also don't see where
> virtual firmware would be taken from).
> 
> There is no PV EFI interface to obtain video information. With the
> needed information getting passed via start_info, PV has no need for
> such, and I would be hesitant to add a fundamentally redundant
> interface for PVH. The more that the information needed isn't EFI-
> specific at all.

I think our only option is to expand the HVM start info information to
convey that data from Xen into dom0.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:22:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:22:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186473.335199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Vj-0004d2-NF; Tue, 14 Sep 2021 11:21:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186473.335199; Tue, 14 Sep 2021 11:21:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6Vj-0004cv-KD; Tue, 14 Sep 2021 11:21:59 +0000
Received: by outflank-mailman (input) for mailman id 186473;
 Tue, 14 Sep 2021 11:21:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ6Vi-0004cp-5C
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:21:58 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f83264de-154d-11ec-b41a-12813bfff9fa;
 Tue, 14 Sep 2021 11:21:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f83264de-154d-11ec-b41a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631618516;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=yFDuDpGX3FxW0U8Y86hsKjigs5Vsu3utb0sb2h8ni4g=;
  b=B66f5eJIy4GFBfXHjqoYpvawx9jKulxJi4cuXqhLNRJJrGiPOs+gcEsp
   4gFyAN8Fr5YFp5TEHLuPouMMlIBDUPLP0eXsHBKyAWd4uwDKCTAvl5npJ
   gqUpiwWcjneZ/f6dAXT6Ll/bklPotXgma8xHOWAXfm5+qIyATUWWBV/BR
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 4Xf0DRu0XSE7ntekASjZ6FQ/RmMFPTkSU7fYY5lEQagZ62xt37j89x84oyUDPD7MEvXjTM22H6
 nOEMLD1U8KRWJqbGaDt4Z02WCj6zAtaUga9WLGFBvqTGqdJ6iDv2q1PR2Sz51xUyeQY/TAoaig
 Ip/oul5GbZLEGs5wPYvE6xgFfeSaZlQJo6tPb4tAkdJf01Zh/WNQJfKgCk/dRtw9MHu3XbxwFi
 r9d2ObIp7JGPQc8kQZbUi8p2Y5cYyR4dO57kmCt2dGfzhN1TPQ7obu3I5UbCtSnXk08uMxmN0w
 Lk8/qFXT07NPoay2I2IF2stw
X-SBRS: 5.1
X-MesageID: 53073777
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:lQKRoq0yFoziIfHJiPbD5dl2kn2cJEfYwER7XKvMYLTBsI5bpzwEz
 2ceD2iGM//fYjGneIsjO9+/9E1SuJeHmoRmQQI9pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrJh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhjohP8
 MxyvqKLZgYFN6Tgl94FWhMbKnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQAg2xv2Z8m8fD2Z
 MEpdBpFUT36MhRRYnAKS7Q7p/mTryyqG9FfgA3M/vdmi4TJ9yRh3b6oPNfLd9iiQcROgl3et
 m/A537+ABwRKJqY0zXt2nCmi/LLnCj7cJkPD7D+/flv6GB/3URKVkdQDwHi56Dk1AjuALqzN
 nD45AI/9ZYK7XOgcOLSXjGxqVKLhAYVUZlpRrhSBB629kbE3+qILjFaFWcbM4F36JBeqS8Cj
 QDSzoiwbdB7mPjMEyvMqO3Mxd+nEXVNdQc/iTk4oRzpCjUJiLo6iA7GBv1nGbS85jEeMWCtm
 23WxMTSarN6sCLq60lZ1Quc695PjsKQJuLQ2ukwdjj5hj6VnKb/O+SVBaHztJ6s1rp1q2W8U
 IUswZDCvIji8q1hZATSGb5QTdlFFt6uMSHGgE4HIqTNAw+Foib5FagJuWkWDB4wbq4sJG+4C
 GeO6Fg5zMICYxOXgVpfPtvZ5zIClvO7S7wIl5n8M7JzX3SGXFTbpXo/ORHJhDmFfYpFufhXB
 Kp3uP2EVB4yIa9m0CC3V6Ea17oqzTo53mTdWdbwyBHP7FZUTCf9pW4tPATcY+YnwrmDpQmJo
 d9TO9HTk0dUUfHkYzmR+okWdAhYIX8+DJHwispWau/cfVY2RDB/U6fckeE7Zohov6VJjeOUr
 HuzbVBVlQjkjnrdJATUNn06MOHzXYxypG4QNDA3OQr6wGAqZIuitf9Nd5Y+cbQ92vZkyPp4E
 6sMd8maW6wdQTXb4TUNK5L6qdU6JhisgAuPOQujYSQ+IME8F1CYpIe8c1K2picUDyexuc8vm
 JGa11vWEcgZWgBvLMfKc/bznVm/imcQxbBpVEzSL9gNJEi1qNp2Kzb8h+McKt0XLUmR3SOT0
 gubDEtKpeTJpINpotDFibrd8tWsGup6WEFbA3Pa/fC9MiyDpjivxopJUeCpezHBVTyrpPX+N
 LsNl/ysYucamFtqspZnF+c5xK0z0NLjurtGw1k2B37MdVmqVuttL3TuMROjbUGRKmu1YTeLZ
 38=
IronPort-HdrOrdr: A9a23:uY7mVKOVwU5y0MBcTsWjsMiBIKoaSvp037BN7TEXdfU1SL39qy
 nKpp8mPHDP5Ar5NEtOpTniAsm9qBHnm6KdiLN5Vd3OYOCMggqVBbAnwYz+wyDxXw3Sn9QtsJ
 uIqpIOa+EY22IK7/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="53073777"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ip+BZh5OMZw91euM1yQ1cig2b0/Wt1EBiS4FfPte/kMvRyrPnGqLU37wTsufvodK3Y1Ub9Ohe0FNPgEqMA2mxRPkq48u0SJ5FPdyxhNh7ZdWlFhVLY7RInvC5F7GWuNrZVpowoUW1zzFiyFJ4F7/fpdW5jp2jUskyaeQbmb2P9phdfRRZ95HMPES21pf2mjwEmd7GrP+Y3quMuA71Qos7MlYbcqFpgEFrTbS+eUo9s9ybqomDz2bf3JyMBc/1Asms2UI00xMdqAohPiblMYlgyu0/6ixV+ZuMKm7kNdE7mqGTfQzKst1LOpis4Rm28RNZwk2/ZFeCKXeHcKMCukjmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=v8d851AzpOadqYBOj3pG+NduZOrVaMGqY78DgbM8UTk=;
 b=G53/UXDDGUOuniks7JNHsi4+Y7lacsYwqmgmqNg9eyaupmbw54BhLhIczya8vfCxZQppH1UvEHmYV+dF09UeT7XNtSy1+2b+I3htcFz3F1vuCskP4XuRavAR/+XKAOFch0Z4GLe2/UOL8jL1gT0V2hdyHSPRWUEcRm19zkZqLRIQW1mTosZOZn0rf7mRkZrtDkkeJ8ZlAIPef2jPn5522Mk5Mrid3aYEjtyxedigSSbxZthzDp+siS7ouZB6rlwt7fXlvn+GAICAwTBMs/BoZ9TdbvCby00tZjc2MRCeHjtgHuZiEUKvT4mAXkzjqaJZ6DMnrlqPQukBHi8poottOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v8d851AzpOadqYBOj3pG+NduZOrVaMGqY78DgbM8UTk=;
 b=dWlYANq/LjpvIdDmj+0sHl5AVzwDawJG3ThhLoa10n5rXeNrio58ngftwO2Hb4mYO9EVQ2wzcRojl4tgqaLGsemVaPHWARw37PHbWxFXG/eJiC5XaCKepjud9krUxhPwME2CyL8BXceVKPkAZl+QHUBwFSR9uHoS8YsdMI+W1K4=
Date: Tue, 14 Sep 2021 13:21:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
Message-ID: <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
 <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
X-ClientProxiedBy: LO4P123CA0307.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::6) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 039438ee-abc4-48e1-a8b4-08d97771d5ad
X-MS-TrafficTypeDiagnostic: DM4PR03MB6064:
X-Microsoft-Antispam-PRVS: <DM4PR03MB6064AB6BCC6715D19C9105E98FDA9@DM4PR03MB6064.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 84eR1qz8T78xv6Q8ra0uDUbkiGsLK2m0aYJwBlQnqiF/ywkenMMtYRJEdvBcHeC+3zK1D81sDHisBog076MLEluTWkJmzZMyKoC5n2+49XXMIZMETI3wq8QaETAMDdMf6lJbnbv9IWPafOZ0QazPOwAG3NKGbddM5N1ii2q1dsQgeMxAMAlB1Aho2muq2y8czXkiCd3JezU49dpQ4KH/4qPle9LTURPYxkHa9ti/7mjKBNz7rRKwNkD3oD6Jhq5GQiWIiDXL7mSwznBtsfh6jOKB9KBONpGQA9dBZywsMHlUTAciTNiHi03b1c8JyCjdddEbwOdQiLJEWc4Vr/wRGtmowuqebA6nSetxJ6j/XDjFGsUGM1TjcKCKTXDWGoAC4+BPm7NoBcOFC+a7vyHeblH9W/ieWB1hP6+7I73SrBMTlkjGKC2FiCWKHHL+5TmT+vrXTxasFSP0mKBTtkkHsJBYOyoyVWYAndnkz8kqOAwZ7ll/w4pXM01uYTQgu7FFx/w6u8CXu27FrQEJyJir83ryCo7roEppxQQGTdEV+E/vrvyKtueR4oBoYYD0GAUIOPVyh4/UdK69McZEnc4Qm0ifDTf2uOTR+6bOrfqTTjodw2LemS9UKt+wKB0xs4TrJF+pBqRK0rUL6LyXjYiK3Q==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(39860400002)(366004)(136003)(83380400001)(6486002)(53546011)(316002)(86362001)(26005)(6916009)(6496006)(8936002)(66476007)(6666004)(186003)(66946007)(478600001)(66556008)(5660300002)(8676002)(4326008)(2906002)(9686003)(85182001)(38100700002)(956004)(15650500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RzcyZ3BmTXdKdmdNbTRrWUlXdS9wSnNPWkNTMktWd0tTMGdLcWlsS0l4Z2xH?=
 =?utf-8?B?cDh4QVZma1hYUXFBR0crNWpHbC9aK2lYSFNNRUdCanErZzVwd1R6ZERYd015?=
 =?utf-8?B?Zmk2ZlZnMXg2SUs3U0lRVXBlUFFQSytJNHdMOWlOamlJbTU1cCtZU2tMcm1o?=
 =?utf-8?B?b3l6cHlVcGJYZStGYS83UXdERGgxVEFrcE9oV3FrVnJaSnk0bUdUK0FuOWEv?=
 =?utf-8?B?cVRqZ0RZZkFBOWF2dUNVbGdBWnB0SXYxaUMreTQwSGhGRVhsZGV6MlZVN1ps?=
 =?utf-8?B?emdhYWNRU2lIbi8wZWVpZjNDK045aXpEVTU1UHZXMUFoa2Y4eW5qQnZJVmwv?=
 =?utf-8?B?UHJUOTByMHB2aTRWdWg5MGVuTU5IWFlJWFMwUFVFeTRmd2U0dXV4NHdWTGJH?=
 =?utf-8?B?bWpFVlNwNzNudEN5aWlPZUErcHFjNC9Wd3l0VWd1VWJ6NG9sSWg5NzNua0JM?=
 =?utf-8?B?c3hveDRZdmxURzBacWdvbm05VXpSdEVseTZMSDg5SHBsNktSWU9nN1NJYzFZ?=
 =?utf-8?B?S0RaUVE2eEo5ZENFejhWcjVZNXFEZnlaK3UrdWg2OTUvMnUzRmtTTmFCb0F0?=
 =?utf-8?B?ZTdreGR0MktZRFBLMUZyTGpKMUZKTENtbUxPcWhVWGNCSU43d3UvZzB3dUcx?=
 =?utf-8?B?RGV3NnE0TW4xOVRzajdwM3NzcW05bkJEVC9hdUh0aW83VWpSSWZrUnYzTGZz?=
 =?utf-8?B?a2U3ZzhKNlhEbFJqS1VzdnJpNFFUY0kyRlhRME5NTWhTQlRVRTV1ZlVYc0VL?=
 =?utf-8?B?SXFzWWk4T2F1bUNKa1NhQmN4dkVteGQwN0VZSHBmalZSN0M4aCtZSFdVcFhN?=
 =?utf-8?B?OVVsM25qK1BUZlg0UnE1OFBLZUhDQUppelprbWpVUE92cmo4V3cwZEdzdElX?=
 =?utf-8?B?cEQ3L0xEV0d6QTlpVVRiY2I1Ym1oWGZQdS9keEhGTFVTK0llaFg0ZG9CWDly?=
 =?utf-8?B?eVZrYkkwTldFa213NmIzeGErcnh4TElxcFg4RG4zSXNITlpqN2hJVHluUlA1?=
 =?utf-8?B?NEc4TC8xaWhPU2xWTFJZemtlRi93UFFJSkYwZmphdGhTRHUwNTQ1U0ZTRGRr?=
 =?utf-8?B?TllweWRvYk1KSE13cG12MCt2YlFkOFVjY01RMmF4RGdKUDFtVWJqcGQ0TC9I?=
 =?utf-8?B?RkNmdGtld2JGaVhSZCt5ZTRFamUxWUlBVUt6ZFlqdTVZWGZtYWZ3N3BZdkJM?=
 =?utf-8?B?ZTEwK1FaUExqNjNQQmpISVVTdnMvaTRyQmpIMVYwbDBVc2VLczh3UVdsR1BC?=
 =?utf-8?B?ellZbnRFNkYxdStuSUxnRU4xWk5SdHFkbkdxV0R2eTIzV2FXQi9nVWRXbytS?=
 =?utf-8?B?aS9DZDNDb0ExejdJYTBWYlN3c0tIb0JwdjJ1Z3dTVFptcGFwRjI3MDFjNEg0?=
 =?utf-8?B?M2k4Q2l2ZzNaaUFoWEJxZml0dS8xTElmS2J4Rlg3Q0dGVjJQV0R5SXNpcWZJ?=
 =?utf-8?B?SVc3QXJ1RDlZV2h0ZGtQUnZWeGJKcWdRcElUYmk4MERidEFIZm04c3RJaTZB?=
 =?utf-8?B?SWRMREFZVUdGbWVraXpWT2EwT1dpTlJYVlNlS0w0QVkrb09LWngzV1pyZHpN?=
 =?utf-8?B?bi9mSmFrMGx4dkt1U3B4MDhzSjI0OGY5cEtOVW9vOVBadWQwQnRuUHFMVFVN?=
 =?utf-8?B?YUVCVzcrek9ENUJLbnRCdWJ4NjhQVTZTcktJS2Radnh6YUJyeHZTZGxIVlJk?=
 =?utf-8?B?Sis5N3VLUmpUT2hqZmtmZmFIVGVSREpWekx5VlhIb1NaV2NnSU42cjR1aE5q?=
 =?utf-8?Q?Ou4hRHMv0sb7Z0hHxeRJwG8gwWIEGHrIgwj/suc?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 039438ee-abc4-48e1-a8b4-08d97771d5ad
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 11:21:45.1515
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HbbBfuqJAOoyOfF7UnrWd8d3ihSuCxFkEWUp3C48XSqPzAqYh+pdvQY9EOJxShR1ic1iR78f6qnpZfdQA6VnYQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6064
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
> On 14.09.2021 12:00, Roger Pau Monné wrote:
> > On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
> >> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
> >> console) are associated with DomXEN, not Dom0. This means that while
> >> looking for overlapping BARs such devices cannot be found on Dom0's
> >> list of devices; DomXEN's list also needs to be scanned.
> > 
> > Thanks for looking into this, I certainly didn't take hidden devices
> > into account for vPCI dom0.
> > 
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
> >>      would make the diff difficult to read. At this point I'd merely
> >>      like to gather input towards possible better approaches to solve
> >>      the issue (not the least because quite possibly there are further
> >>      places needing changing).
> > 
> > I have a couple of questions, AFAICT we currently hide the serial
> > console and/or the VGA adapter if it's in use by Xen.
> > 
> > I wonder whether we need to add vPCI handlers for those:
> > setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
> > device because of the temporary override of pdev->domain done in
> > _setup_hwdom_pci_devices, but I think that for hidden devices we
> > shouldn't add vPCI handlers. We should instead block PCI config space
> > access from dom0 to the device so that it doesn't mess with Xen usage.
> 
> The answer to this follows (I think) from the one below.
> 
> > It's also not clear why does Xen want to have those hidden devices
> > partly controlled by dom0, as it would seem to me that dom0 interfering
> > with hidden devices in use by Xen can only lead to trouble.
> 
> Dom0 can't interfere as long as it can only read from the device.
> Restricting accesses to reads is one of the purposes of "hiding"
> (the other is to make it impossible to assign these to a DomU). Not
> allowing Dom0 to read from such devices would lead to wrong PCI
> device discovery - some devices would be missing (which may or may
> not be merely a cosmetic issue). If the missing device is a multi-
> function one at function 0, other devices in the same slot may also
> not be found by Dom0 (depending on whether it looks at non-zero
> function numbers in this case).

Hm, indeed seems possible that missing function 0 the whole device is
skipped.

Maybe we need a special vPCI handling for those devices that just
allows reads but not writes, and that doesn't maps the BARs into dom0
p2m? Or could the BARs potentially be shared between multiple
functions on the same device?

This also makes me wonder why they have to be added to the IOMMU, is
that related to device groups and the fact that Xen is not clever
enough to figure whether devices belong to the same IOMMU group and
thus must be assigned together?

Thanks Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186485.335209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6u3-0007RZ-Sl; Tue, 14 Sep 2021 11:47:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186485.335209; Tue, 14 Sep 2021 11:47:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ6u3-0007RS-Px; Tue, 14 Sep 2021 11:47:07 +0000
Received: by outflank-mailman (input) for mailman id 186485;
 Tue, 14 Sep 2021 11:47:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ6u2-0007RI-G3; Tue, 14 Sep 2021 11:47:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ6u2-0004Y5-7V; Tue, 14 Sep 2021 11:47:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ6u1-0004IQ-Rg; Tue, 14 Sep 2021 11:47:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ6u1-0000qt-RI; Tue, 14 Sep 2021 11:47:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=l3cX34B3YmmkgeNcAAYIY5ApANA7+0L85/S2VmTTpPM=; b=nopOFBTZ9q0oMwpny3394hwGX/
	ta9Wh2QxjhndhxnYvAyjWd0rmSQtW/Ejn+tIF1us9jcMm/RJoo3+ZPgMNf/cHZ+rZP20eu6+yr1S7
	aizLhgpeUT6auqp9vFDN+cgR+L9sGfVhN9+zp5J9l8SbzkB+tLHZuQO7RFbrbf0MvVy8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164978-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164978: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=9b49c2c6d3d958c24ca9af5bac2abb06e4475a6a
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 11:47:05 +0000

flight 164978 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164978/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              9b49c2c6d3d958c24ca9af5bac2abb06e4475a6a
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  431 days
Failing since        151818  2020-07-11 04:18:52 Z  430 days  421 attempts
Testing same since   164978  2021-09-14 04:20:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73299 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 11:58:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 11:58:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186497.335224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ75C-0000dQ-1Q; Tue, 14 Sep 2021 11:58:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186497.335224; Tue, 14 Sep 2021 11:58:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ75B-0000dJ-TI; Tue, 14 Sep 2021 11:58:37 +0000
Received: by outflank-mailman (input) for mailman id 186497;
 Tue, 14 Sep 2021 11:58:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ75A-0000dD-C0
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 11:58:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 161a8738-1553-11ec-b41f-12813bfff9fa;
 Tue, 14 Sep 2021 11:58:34 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2055.outbound.protection.outlook.com [104.47.4.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-xkeiV7F7Pz6XsvTxq7Likw-3; Tue, 14 Sep 2021 13:58:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4847.eurprd04.prod.outlook.com (2603:10a6:803:54::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep
 2021 11:58:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 11:58:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0006.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 11:58:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 161a8738-1553-11ec-b41f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631620713;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=x8h8vrdylyoR9o51554wlmESqyDSMuN77HtVq4cvB2k=;
	b=VxjeEKoka1/xDQr/I9oOz6vLmTlrirHHLJSkDO+1gRkxu9DFlKZPOHHubRBl8FxlGamST5
	WqHXt5uuL8MSpzQV2ZE57T/wJyyZSUMjEP1Xj0b2ap9Mg3AfwqsU6f5hXmGaLdJUeZPuq1
	E8oDu5gGM0Y+5pmublDtJ+q7LR4pOYI=
X-MC-Unique: xkeiV7F7Pz6XsvTxq7Likw-3
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g1UASAHL7YZ3pmzMjn0SklA8/FQkxLfydjd+IspXFdtf5m6OJBZdw0Ba/NLvQ+DHylHzf67mt0Ekv/9sPB2j5OxnITFHwoKnsO7xeJCDgXD64PG4eTvz2dkM8s4KvvjElzBuhsfnPQkiK+f42N6lcHCKmpXYHGC4qbzBs10r01fbLmj6xNkDzRGqi6gwLUu+6qyAtlbtMa28rtFPRVV798YC2x2IcbGxjXE3SXFdYgon4hsZu6z9v/45JU8/zx7NaVhR2R+2pQM2PQfW5lo2DfHGO7BgDrhiB0Jni1MLYI+4llmBxgEqW4Fk28/wYwz+Rap1MqerDy0w9vupSYzc7w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=22xZutEVb7YW8bswfCVj8AKyuuIOWtilaoRxFERMWqs=;
 b=FHgtVmjW/EoNRQfoxyXfHvRCojLkBZatSpXutA1Iwbz/cUkxB6xDDKLJxAqu2/lDw+efYTRLU9eZ9+VaIWMzdZxV9ewt+GozO3vQGYi+sEPXB8OjOKVF/5qwt/5/W8aRmIKoORmhoOc2pF5QXhAWhEM4PUmpJwJxrN7a81tCXex6+k4jh024NShvDy7vyz3qeyfO1iUwg3jbtlgDsyvKcNLmf5UjIa6j6cvxNDkqQLba/uBhwVH3nKWoGYNQbuII9jDjqvEscFQpojx6aB/igZnc4XCrxc1tjigU5Ah8iX8fLHZYGJuJqN9ktVCZD4sisvflRDpI35OBCGU/0A6CKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
 <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
Date: Tue, 14 Sep 2021 13:58:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2P264CA0006.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::18)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dbb7c1ad-f466-4052-1f41-08d97776f86b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4847:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB484789E7F32119DF40BAB3A2B3DA9@VI1PR04MB4847.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CYKmB3vn0zkYTPhXAdyHTOB2SALaWXVvfHDLdZi3a5+GuW6ZaEl0RjsdjfiBFRfdHdpW5QhKqrNxYAq2IA15lKYn2Gow2Q4wyOb97VsqT3eCbbSjzT8iiMXuG5Nms5AzmEH7aMEKzZ8LDf2bIuVCVX3HPSunmPQxRa8bcPIhGvcQO2z0VsvXl0gwIDxWLYViQhbhx+714c6404mNd+aV4HWUmG0ae1RK5B0ij/4Nnzn0j/OIx29s+unU8TFPOv/2JX6oyJz908fV+PSturxboq8eKhl/q8tuXwBu33AA+PVXFkKOZOH37O8BR1Rco4tzrhIkeH1rbam0l34OVcI1kJh5bJpeBnVUEBP/gPqA5phC1aF1ZTNDr3SIxxgn2P4e+x/f1/lmIJUSCFjpbW/gpJN7W35w1uq20mj9f5tIz0E1KnLVPZrgthx/uBQkMJfMqzZCnZOdMrczLKip60+ffEtLH/DC4x6Oxb0vzDj6JIDbZHwGtF2c5M7dbuS/AZZEmCpNL8WZUMc6vMbf2lJCQeDHD9QcV7dx5au+CYd83q9MTgf2WNwyj7pD5ygoSWfq0LIhUas3acJkuT/HjSRtH7u0OsBd+H/1h3RAeX7HTtmeqNRqjuH1Cs1FK8NINhqjjDy3GgD4lVHxchRna3B84NcKtB+bd6vd3I2kjNVvEmRWx+gGtNqAjUmjIEf+DnrjyVO+V+ALoAF9OAAsLH4weDkjh5y0tgZeRUQLIgtfo50TcB7GeeOSKlj0xUaL4UvGlLEcQ0SJ7ErkVB2CoMiW41MP1WBdcPsit96gEk8wfs6CA/pifbIb34Ke42sg90Nn
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(346002)(376002)(186003)(478600001)(38100700002)(4326008)(956004)(86362001)(36756003)(8676002)(2616005)(83380400001)(31696002)(5660300002)(966005)(16576012)(316002)(53546011)(8936002)(54906003)(2906002)(26005)(6916009)(66946007)(6486002)(66476007)(66556008)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?s/EcSBBRZvZypQAUQXfCZ6iWsMaz4v84ci/jHV4rXU9YOEIJ86S6ZEKXozoq?=
 =?us-ascii?Q?vvzKhkzseyRDK2iYLeRaZJf5Q92jL+ym6ZDoKYSoeyCiaEsdmkmC17TaGg9R?=
 =?us-ascii?Q?GbLya3CBEFzx2J8M8dA5DaGgb5BPz9HYTu1xwqA/HbmMC19ofH/M50lkrjAR?=
 =?us-ascii?Q?5WYQq571rBRFcERSLkhLui7zmmeRcWh/fjPAjTHfAsK/qF1loI6f1g2GNOAl?=
 =?us-ascii?Q?K1+UbpBXwK6Ozxt33vS/H3mkYeDL3fcUIQOqYoKeXhV5xWat/72i2YEuZXXA?=
 =?us-ascii?Q?Qef148xucSIKagTNwvFsguc71UBsZuEYCfz01hF36v/llGqAvRHiY0JUVLJ1?=
 =?us-ascii?Q?moFUZWubf6uxISKiz1pp/hZsyIZk8x4da8ARbyhqNCU+P7QocfyTlkbrDVUK?=
 =?us-ascii?Q?swCLSCehA7lXXGYKZ35EDVcmKHbqm4spjoBqsw0lHqAIxmuwuhpZCQJ/Satb?=
 =?us-ascii?Q?cserW8aEJEs8utfUCIOjbU1st9kY3Aepf8TPjGiSL0xgCj1Ck64bW32Bkskm?=
 =?us-ascii?Q?Kygb1Amc+56cTbz7nnpf1aJg56vEojnQfp0nBpurwDk+ca5cTH+4yD+grdSH?=
 =?us-ascii?Q?xtk23D2YLgDmHYA+bwjJqnnJCo6Du//g/J44+VsMq0oLBCB9ZdEyNBm0brcq?=
 =?us-ascii?Q?zvwICfl90tmgnDVxj6PCR/XDWANj160K6LVxEaeI+9GmNa0mj/WDtptBDVWb?=
 =?us-ascii?Q?1uuH0PUZuNgM2U+/whovfOvWB/ZdZvGNh1vm6H43+P+/7KkncLZncWFRyJs5?=
 =?us-ascii?Q?Pu+vR6jWbBbjgpK3/u1SSZ9nQW+CZOY6fCIriQzOLmSb1ndpVhL/5Z4Tvf6x?=
 =?us-ascii?Q?hWlO+2hiqMkyFURnfbDBRG+HNfHkjMpw5AmfgPgzcN5w6YI2BRv3M9H6Cy+N?=
 =?us-ascii?Q?98DFfbPcsNKzF9zKQb2miMdj1E8w0/ckHvA1z7LvOERtTaUXJHCCPLZ1L/9f?=
 =?us-ascii?Q?PNlwesepKAG4Vv/Lvn94PsV8BK+ulK7e17tbB7OJuLv/5j7W8siBLuNr3h2O?=
 =?us-ascii?Q?BAE5Ntuvh2liP/txkbKcyruuGhV5mrHu/77cHojsOTFeeM9MbByD6On+NZoj?=
 =?us-ascii?Q?xvx/CwvKNyuSs2rBWc2DspdM2v5vHjKb1KMoh8iUiQwfJJH/pVxVQ6yrXbH9?=
 =?us-ascii?Q?ztWlEABRh8lHqIG5RBfa0VUZYf1Oer0g7vYpNshAuJL4wi0NrgpYRVmijI69?=
 =?us-ascii?Q?8xz1ruEYv2NWAqsPcICImxNsZ6qZPM9HtQQ64Fh8ueCP2tI913YnLUhYXqKA?=
 =?us-ascii?Q?b7WPcCroFKm4YRpCnhjdpyKiB4l/BUkPz0128oAQi3rfUTHzeouoCx1X2K8u?=
 =?us-ascii?Q?P5NbVPIuKf14b5nuh743+IqO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dbb7c1ad-f466-4052-1f41-08d97776f86b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 11:58:30.8661
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SrpIztEEIRW0KSxvy0FhmuV/2c6KDC/SGPdXFPoTMSXmXYQrF7fbuiOjQeE5mbrAYdFUHsgrCdtdLVAAvUNZuw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4847

On 14.09.2021 13:15, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
>> On 14.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
>>>> In order to try to debug hypervisor side breakage from XSA-378 I found
>>>> myself urged to finally give PVH Dom0 a try. Sadly things didn't work
>>>> quite as expected. In the course of investigating these issues I actua=
lly
>>>> spotted one piece of PV Dom0 breakage as well, a fix for which is also
>>>> included here.
>>>>
>>>> There are two immediate remaining issues (also mentioned in affected
>>>> patches):
>>>>
>>>> 1) It is not clear to me how PCI device reporting is to work. PV Dom0
>>>>    reports devices as they're discovered, including ones the hyperviso=
r
>>>>    may not have been able to discover itself (ones on segments other
>>>>    than 0 or hotplugged ones). The respective hypercall, however, is
>>>>    inaccessible to PVH Dom0. Depending on the answer to this, either
>>>>    the hypervisor will need changing (to permit the call) or patch 2
>>>>    here will need further refinement.
>>>
>>> I would rather prefer if we could limit the hypercall usage to only
>>> report hotplugged segments to Xen. Then Xen would have to scan the
>>> segment when reported and add any devices found.
>>>
>>> Such hypercall must be used before dom0 tries to access any device, as
>>> otherwise the BARs won't be mapped in the second stage translation and
>>> the traps for the MCFG area won't be setup either.
>>
>> This might work if hotplugging would only ever be of segments, and not
>> of individual devices. Yet the latter is, I think, a common case (as
>> far as hotplugging itself is "common").
>=20
> Right, I agree to use hypercalls to report either hotplugged segments
> or devices. However I would like to avoid mandating usage of the
> hypercall for non-hotplug stuff, as then OSes not having hotplug
> support don't really need to care about making use of those
> hypercalls.
>=20
>> Also don't forget about SR-IOV VFs - they would typically not be there
>> when booting. They would materialize when the PF driver initializes
>> the device. This is, I think, something that can be dealt with by
>> intercepting writes to the SR-IOV capability.
>=20
> My plan was to indeed trap SR-IOV capability accesses, see:
>=20
> https://lore.kernel.org/xen-devel/20180717094830.54806-1-roger.pau@citrix=
.com/
>=20
> I just don't have time ATM to continue this work.
>=20
>> But I wonder whether
>> there might be other cases where devices become "visible" only while
>> the Dom0 kernel is already running.
>=20
> I would consider those kind of hotplug devices, and hence would
> require the use of the hypercall in order to notify Xen about them.

So what does this mean for the one patch? Should drivers/xen/pci.c
then be built for PVH (and then have logic added to filter boot
time device discovery), or should I restrict this to be PV-only (and
PVH would get some completely different logic added later)?

>>>> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
>>>>    console) when in a non-default mode (i.e. not 80x25 text), as the
>>>>    necessary information (in particular about VESA-bases LFB modes) is
>>>>    not communicated. On the hypervisor side this looks like deliberate
>>>>    behavior, but it is unclear to me what the intentions were towards
>>>>    an alternative model. (X may be able to access the screen depending
>>>>    on whether it has a suitable driver besides the presently unusable
>>>>    /dev/fb<N> based one.)
>>>
>>> I had to admit most of my boxes are headless servers, albeit I have
>>> one NUC I can use to test gfx stuff, so I don't really use gfx output
>>> with Xen.
>>>
>>> As I understand such information is fetched from the BIOS and passed
>>> into Xen, which should then hand it over to the dom0 kernel?
>>
>> That's how PV Dom0 learns of the information, yes. See
>> fill_console_start_info(). (I'm in the process of eliminating the
>> need for some of the "fetch from BIOS" in Xen right now, but that's
>> not going to get us as far as being able to delete that code, no
>> matter how much in particular Andrew would like that to happen.)
>>
>>> I guess the only way for Linux dom0 kernel to fetch that information
>>> would be to emulate the BIOS or drop into realmode and issue the BIOS
>>> calls?
>>
>> Native Linux gets this information passed from the boot loader, I think
>> (except in the EFI case, as per below).
>>
>>> Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
>>> info using the PV EFI interface?
>>
>> There it's EFI boot services functions which can be invoked before
>> leaving boot services (in the native case). Aiui the PVH entry point
>> lives logically past any EFI boot services interaction, and hence
>> using them is not an option (if there was EFI firmware present in Dom0
>> in the first place, which I consider difficult all by itself - this
>> can't be the physical system's firmware, but I also don't see where
>> virtual firmware would be taken from).
>>
>> There is no PV EFI interface to obtain video information. With the
>> needed information getting passed via start_info, PV has no need for
>> such, and I would be hesitant to add a fundamentally redundant
>> interface for PVH. The more that the information needed isn't EFI-
>> specific at all.
>=20
> I think our only option is to expand the HVM start info information to
> convey that data from Xen into dom0.

PHV doesn't use the ordinary start_info, does it?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:05:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:05:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186510.335238 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7BV-0002A6-18; Tue, 14 Sep 2021 12:05:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186510.335238; Tue, 14 Sep 2021 12:05:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7BU-00029z-U5; Tue, 14 Sep 2021 12:05:08 +0000
Received: by outflank-mailman (input) for mailman id 186510;
 Tue, 14 Sep 2021 12:05:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ7BT-00029t-Ub
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:05:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0daa36a9-059c-4116-909d-2e4c3ec1a0fe;
 Tue, 14 Sep 2021 12:05:06 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-4IjHtp5PPZmW7IgbjmQ6Jg-1; Tue, 14 Sep 2021 14:05:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5904.eurprd04.prod.outlook.com (2603:10a6:803:e6::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 12:05:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 12:05:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0121.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend
 Transport; Tue, 14 Sep 2021 12:05:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0daa36a9-059c-4116-909d-2e4c3ec1a0fe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631621105;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6iBG0dT/ga0ORRPW+vBvUu3OPQkhNIVEXGDSvBG49Oc=;
	b=YzkdoWZjMU0j19aYk977msuE2rbF78vIXJajtvQik1vzNLjATEDGg5EAFU1Nt/rMFppENJ
	ewyNhkjNN/ovdami9dW7g+TVEIrhTUcgegMuparR22/HdUmhzcOndeGcQw8dWFfZDKCthl
	PJ466KIsqjXuzLDi34NSbke/AnWfBrw=
X-MC-Unique: 4IjHtp5PPZmW7IgbjmQ6Jg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Qxc9L2XgCI0K+84fi/NnCVJVu2SxW7Zb8r7TH+HEXozed/0RZOYz13U4/KVgg83ahQR3Bo0Kqm5td8ctVdFf+XaZ3dEJRzuR9JabfQGKZrb+dsIOUiuJqbz8J+u18uuu0fJLop2BKCWuDfnoE4TAAf1qO8UD9ZNBIRVKT42SnlP6v9CiwB5fI6dcVejEe7qM4J/+zztqNRxW3RK6/+w1p3lAk340xr2luLwUBPGgVvM37S4Ac7fMu3PyXuSDYqBvYhpa8YRT36/g1MDfCBoSRc5JrFowM6Z8lHISYX5uOI5iz2fEFEddmOPrrQVOj7uQ8y4rlK62ehPBQGE9WyPDJg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vjJDOYTtIahsSojHg6DggSb0G+WVqbSXp5whCbCigC0=;
 b=Fn3T7OLuF2J+XTKbXSN/9vOEeA19AklD3R/9aMxbwt9Avbpk65ZlIXZgAb7sO/Pi3GyBpJ02rHj0oxl+EwFGFFgqtRdy7zrHWZqX0FqHIhKhV4bKDdS/iu6xbHGcQY+009WsMOUKtstBM+x0y7CKnPR0eR5mN7rWkPoG4W/uMYM7sjrVDJnWepOe7Pdh5rIVL3dT6nhbhs0Pa+TDSp4yDMC6TedN2vcNORVHaoOYkDEcB+M6QyyuwX7NSPfBMG18iC2q2CAVCoysQmtcj/BCTb8SPE7qkoK+b12aZGxQIiQrr7yULuoWUZrlThVQruVb+XFoDa6qfbyPD8jkBdV2Jg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
 <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
 <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <df432c93-b722-6006-c3dc-7f511b377bcc@suse.com>
Date: Tue, 14 Sep 2021 14:05:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR01CA0121.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f5e390d4-1ca1-46e8-a676-08d97777e223
X-MS-TrafficTypeDiagnostic: VI1PR04MB5904:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB59043E0EBA8EA28DF3B51953B3DA9@VI1PR04MB5904.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LfSTwb7t0/t3Px5PJ9J482ttlzLkB3ON2UyY2vMyNXg/e0HNEdX14Ab8a6+dZ1k/Lp55RDZYVte52xm4TSd9GpOtPpNU0ICU3iP2QJCXP/ru8UjIIu6HBwFOAskjn/wQRp96YmILzXRaTI3oQRhyxeigBcz7iG6m9XPsHlaa+/lsaYGOyae4y8yCKv6MQBVb2kdH2CqjN9YQs0ludPI4fiNVJNiFy0Db/lN6hVGBxiR0ANe9gVGx9KUkIBv835yQHxNfvN+AE2WfSlmiC96mLkdjXhygbrCHLwpzL8AqFJPsy0A2LykyiwFXZ+RZpgqjSt+Y30Mkw8FCqBagW2RaQJBz8eN82LNdFtBE2B/D5dRrXEHN7WGj+6jwDiU8uCoCQKkYlDLRF+BY38fHfrd0EkL8BXC9rKMHdC12jdx+Ef5B2LFbyTi3Wmlk/H72Q/N0kS1V9ycMIkyTRsobclVnqmzldzx/GqqMXX+S9Du7IMJ0oaEPbJ/JisuFYvDXmTlpD+rJcEF5+MXJgnu6SMBXqSu03mkpualnJE6WmPcGpOdK6hjgSMZKQtdz5HYE/u4y7+YdMNUD8j75JgsBcys8CzHQKvpaL6DZvklf3XdUzNlEFyMBErH3g68LuVFVCXIDABdp1VDQn5Jq+/K8KOOzZ6Vc0tw/+64oRyRsDTiZMy+fEDC3/rbEAi14YuCPHelcIqspGosXaotJ2FvbogXnBw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(366004)(376002)(39860400002)(396003)(346002)(31696002)(8936002)(8676002)(38100700002)(316002)(2616005)(478600001)(5660300002)(6916009)(16576012)(66946007)(36756003)(4326008)(186003)(53546011)(66476007)(6486002)(26005)(2906002)(15650500001)(83380400001)(86362001)(31686004)(956004)(66556008)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?SYvZJIXgZysMxrWTNt7eed5u9qULJJRKgekrRL6UJ4fZtGP3DdSy5zxHctXO?=
 =?us-ascii?Q?fEwU62sP5y1GOvjEi9k9zQszML7cp+LG1fp5CNrZJ6qs7CDrK/JU8XQ3IbBv?=
 =?us-ascii?Q?f5AqgGrR/VOg6ZLz+Y2WboX3ab6rOBa+Ki195AR7Ji/T0U/97/A/fQSA3Ul0?=
 =?us-ascii?Q?2ME5kRiM4KODQBEPdF1RL0rFHKhPDidafxFQ4QJGfMiGoAwYmjXE8TlWfpEI?=
 =?us-ascii?Q?z5XUHPl1dRq4GNfdFMxGd1gkew85W9h3Isz+RVhAdbCk8mQriVwyzGZaJqWR?=
 =?us-ascii?Q?cVxDLoKEfhLLvAICUWBJHkX+HMMv+JF1N0H4Khdsv1inCpI7ptyqmi0q8c3D?=
 =?us-ascii?Q?17V6Q9eoKxvIdg46Ngr7o2o86j+EPBfb8J8G0GQ+VtQqTprvUyEKQE7ECw35?=
 =?us-ascii?Q?O981P2ao+pluh8WG1MWaNTCCaYPQwnPyRr+T9dr/RaTPoje2yNXKTafisWLb?=
 =?us-ascii?Q?Aw0w+1IzQ1+0zirIRzeww/9QDthsox76NhCW3jeRI/S/56fT/OGZx6OR2dyg?=
 =?us-ascii?Q?SQNTgB762XmPd6odSrDe/F1im5eof17U/wOBP3V5s06QgiUhwRo2Do/dwVUE?=
 =?us-ascii?Q?/z39z6d+CL7ox9B+3paGYZVQsTMnlirCXRlJMLu45tbvGdLP5Gfdnm8Wb9IZ?=
 =?us-ascii?Q?kwQxptUbYnlMOVo2m0VoDtW91T3QH5upLAPn4KlVEaGF0+xPEoYlswiZti1u?=
 =?us-ascii?Q?t67OSzVfIjRKFnLOXaqzdmH9jnkDwiee6k4UnlD8BS/fbFfPCSUpMTEHxDdZ?=
 =?us-ascii?Q?fOrIgvTXBMWLf7GcFnbjjAdZQiNYt0G4I3gjpGl7bcXtrEAym1OdpAHAux9Y?=
 =?us-ascii?Q?WqxwA4tWNW6D6YyhVWDH9zm8tDaOQlwAcmsJTtEU+jN+9kVkRI5ngbt/jziW?=
 =?us-ascii?Q?aTQJqZ8Rb8cWFuSheH6nKW2/NS70frBtWANLs0hzWP0wKj3PHc+HVlgoauCI?=
 =?us-ascii?Q?5j825mwc5LsiP7sFopgXv+wkphsy+nll22R+qHZ28r69WflALwb4kFkVW/XV?=
 =?us-ascii?Q?Q7KJ6BCC3giXa3kObwSowvqXJHfZm9qBmU6/wvjztDAlF6WLsxS5Zz5CWR7p?=
 =?us-ascii?Q?jw2lfJv8D843s4lafk4Hxgx4+17Mi6KA4bVBJeFzIklrGlfjbjw7ZgHqlgRY?=
 =?us-ascii?Q?2hN2M4p4TMxGC6vmXbnVb5Q2V7YgdxvNBrU6Or7QipsqnyvY6QyUlPzo+jKc?=
 =?us-ascii?Q?zAsdEDYhQmr61yV9qlaCIZ0Zkvr8UkCOoiVc7nEK2iXfPl91JLde/5vRhiTg?=
 =?us-ascii?Q?/6DGhS9GYEMDk3H2C3NbpIL1lbUMj2vjs6PsHbAEKawUM97phVjw72FSEo8Y?=
 =?us-ascii?Q?R+JZg99mTkhuuMSpIIcxFuIh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f5e390d4-1ca1-46e8-a676-08d97777e223
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 12:05:02.9795
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mPu8cnB7SqBNt/Usvn/DRxSxAjs9lFClFv3vhO1fC411dpp6NNzIAQVrVUEW7qEqhCT9sj500DSQBYbCOsSLdQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5904

On 14.09.2021 13:21, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
>> On 14.09.2021 12:00, Roger Pau Monn=C3=A9 wrote:
>>> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
>>>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
>>>> console) are associated with DomXEN, not Dom0. This means that while
>>>> looking for overlapping BARs such devices cannot be found on Dom0's
>>>> list of devices; DomXEN's list also needs to be scanned.
>>>
>>> Thanks for looking into this, I certainly didn't take hidden devices
>>> into account for vPCI dom0.
>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>> ---
>>>> RFC: Patch intentionally mis-formatted, as the necessary re-indentatio=
n
>>>>      would make the diff difficult to read. At this point I'd merely
>>>>      like to gather input towards possible better approaches to solve
>>>>      the issue (not the least because quite possibly there are further
>>>>      places needing changing).
>>>
>>> I have a couple of questions, AFAICT we currently hide the serial
>>> console and/or the VGA adapter if it's in use by Xen.
>>>
>>> I wonder whether we need to add vPCI handlers for those:
>>> setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
>>> device because of the temporary override of pdev->domain done in
>>> _setup_hwdom_pci_devices, but I think that for hidden devices we
>>> shouldn't add vPCI handlers. We should instead block PCI config space
>>> access from dom0 to the device so that it doesn't mess with Xen usage.
>>
>> The answer to this follows (I think) from the one below.
>>
>>> It's also not clear why does Xen want to have those hidden devices
>>> partly controlled by dom0, as it would seem to me that dom0 interfering
>>> with hidden devices in use by Xen can only lead to trouble.
>>
>> Dom0 can't interfere as long as it can only read from the device.
>> Restricting accesses to reads is one of the purposes of "hiding"
>> (the other is to make it impossible to assign these to a DomU). Not
>> allowing Dom0 to read from such devices would lead to wrong PCI
>> device discovery - some devices would be missing (which may or may
>> not be merely a cosmetic issue). If the missing device is a multi-
>> function one at function 0, other devices in the same slot may also
>> not be found by Dom0 (depending on whether it looks at non-zero
>> function numbers in this case).
>=20
> Hm, indeed seems possible that missing function 0 the whole device is
> skipped.
>=20
> Maybe we need a special vPCI handling for those devices that just
> allows reads but not writes, and that doesn't maps the BARs into dom0
> p2m?

Not sure about mapping. They could be mapped r/o. And they may
actually need mapping for multi-function devices, but I guess for
such devices to actually function properly then there would be
more work required.

> Or could the BARs potentially be shared between multiple
> functions on the same device?

I don't think that's allowed (or would even work) - if anything
(like in general) small BARs may share a page, but without overlaps.

> This also makes me wonder why they have to be added to the IOMMU, is
> that related to device groups and the fact that Xen is not clever
> enough to figure whether devices belong to the same IOMMU group and
> thus must be assigned together?

I'm actually not sure whether they need to be added to the IOMMU.
With multi-function devices in mind, back at the time I though it
would make more sense that way. But I may have been wrong, and
hence we may want to revisit this. Problem with such changes is
that they may cause rare issues on very specific systems, which we
may have no way noticing even during a full release cycle.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:29:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:29:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186520.335249 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7Yu-0004vo-0c; Tue, 14 Sep 2021 12:29:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186520.335249; Tue, 14 Sep 2021 12:29:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7Yt-0004vh-T5; Tue, 14 Sep 2021 12:29:19 +0000
Received: by outflank-mailman (input) for mailman id 186520;
 Tue, 14 Sep 2021 12:29:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ7Yt-0004vX-18; Tue, 14 Sep 2021 12:29:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ7Ys-0005GU-Q1; Tue, 14 Sep 2021 12:29:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ7Ys-00061L-H9; Tue, 14 Sep 2021 12:29:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ7Ys-0002pN-Gf; Tue, 14 Sep 2021 12:29:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5E57nrd4fmcCfZzi4AVIYFCTu353uDgt/kn51G/DsM8=; b=fVhbSJonGjSmz7s0jQpI19rUZY
	MHGPaliabaTSL/0TWoihnv/jEDZDUk9ZpJENnY51u/pSKY/d2aR5mfMnq/CCce4HvtqO2Aa42Oxoz
	2T3WTlVLymv7TVYVzf436WMaIaDkobLAzQQsmhsmJRGQ/wdcueY4nFiM/HJF1J2p+5Rc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164976-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164976: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a89bcd9737757e4d671783588a6041a84a5e1754
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 12:29:18 +0000

flight 164976 xen-unstable real [real]
flight 164981 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164976/
http://logs.test-lab.xenproject.org/osstest/logs/164981/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  a89bcd9737757e4d671783588a6041a84a5e1754
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    3 days
Failing since        164951  2021-09-12 00:14:36 Z    2 days    5 attempts
Testing same since   164976  2021-09-14 01:38:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit a89bcd9737757e4d671783588a6041a84a5e1754
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Wed Jul 7 09:15:31 2021 +0200

    tools/go: honor append build flags
    
    Make the go build use APPEND_{C/LD}FLAGS when necessary, just like
    other parts of the build.
    
    Reported-by: Ting-Wei Lan <lantw44@gmail.com>
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Acked-by: Nick Rosbrook <rosbrookn@ainfosec.com>
    Acked-by: Ian Jackson <iwj@xenproject.org>

commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:32:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:32:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186531.335263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7bm-0006KI-Kf; Tue, 14 Sep 2021 12:32:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186531.335263; Tue, 14 Sep 2021 12:32:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7bm-0006KB-Gc; Tue, 14 Sep 2021 12:32:18 +0000
Received: by outflank-mailman (input) for mailman id 186531;
 Tue, 14 Sep 2021 12:32:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jeVa=OE=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQ7bl-0006K3-1y
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:32:17 +0000
Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f7518b5-ac5e-40b6-a3b3-1dd7b1fad4d5;
 Tue, 14 Sep 2021 12:32:14 +0000 (UTC)
Received: by mail-ej1-x635.google.com with SMTP id h9so28568950ejs.4
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 05:32:14 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id h10sm5616975edf.85.2021.09.14.05.32.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 Sep 2021 05:32:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f7518b5-ac5e-40b6-a3b3-1dd7b1fad4d5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=Zu26EcxSPuA7jpEqSDVCkAl6vo2I+Y3ZS09kAqjLKZ4=;
        b=Rk0AVVB8IdWySleepo3/4SbCEfMxlygxqGT1QlKu9CiPmdO1ghpjezBAkA/RdtMzLp
         99RP2zLow301AuNaYWEG0G4scQ4LcqVdoGkWzkHcWRrP76IOWMRdrTEgkT1b0MpoIzIi
         jeNsdRisHfXtGfbJGkBo8uzwOUhTDjfX9hR4PSCHaooQWmuE/l0TI5FZ4JxkGWbItymt
         sqhNFvwopjeFbSzCD747H7zbIyIOCeVMusILPpRdus36c48ZU6rYR23HtxNBZjfMqJxp
         0IsC2brAW1+lFiCGFmMif7C+tjo2YxC/ZVoseUr5Q8ZrCa5cnawsY1HY49Mkp21o+syl
         ku7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=Zu26EcxSPuA7jpEqSDVCkAl6vo2I+Y3ZS09kAqjLKZ4=;
        b=DFsgWYR6M5DZ1FZJPpPv4DyRt6B0VFml4u7dPModRmI/bkD4MvxR950EqY5CAh3JXz
         oH4KqTjrz4iz2J3ta13QzDR8SbQlZ0Nz3IQdK49dfF2sNYx/cg6TXlP0gwVnRzWMrCoF
         6OJK9pk6/6BtVFX28uPRVfEZ6vn0ChaimUrOs93y/k+zCLCNirRbAzf33CLuImg0Cnht
         0EhicW1T0qs7xyi+r/kjsEgLh0bll9Od30uGPAxkYQOSmwd4uqqvAmFDotoXgZdDsV1u
         0jjeTn6bFJECSTOokpQgxJ/So/Nwrh632hNkZ50w1mGszG55EJNDk922O56PkempEioe
         sqDw==
X-Gm-Message-State: AOAM5337SkNzvFxZ10M4x4QLW2/bpdM4F269zPUO6cNJ8VjAmBSpeCDl
	huMspiC5oJfvbJ7cGYStyrOcqVBDs2I=
X-Google-Smtp-Source: ABdhPJz4a8xzITQbQXlcGQhN305ctSA3Vz3/qTIYhmJdb+IWJI9zLYYp5wJkydROuTwCX0G8NE4iag==
X-Received: by 2002:a17:906:410c:: with SMTP id j12mr18577953ejk.161.1631622732785;
        Tue, 14 Sep 2021 05:32:12 -0700 (PDT)
Subject: Re: [RFC PATCH] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631228688-30347-1-git-send-email-olekstysh@gmail.com>
 <398b436a-984a-d6b2-bf88-33e994c95c55@suse.com>
 <c286febc-b7b4-5aa4-5331-f0c2f6a17000@gmail.com>
 <a0ad861c-d8a8-82f0-023b-26c3fd4d50cb@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <de043a0e-e5b1-7f1b-d7c5-6fcc220048cf@gmail.com>
Date: Tue, 14 Sep 2021 15:32:11 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <a0ad861c-d8a8-82f0-023b-26c3fd4d50cb@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 14.09.21 11:31, Jan Beulich wrote:

Hi Jan

> On 13.09.2021 19:09, Oleksandr wrote:
>> On 10.09.21 10:52, Jan Beulich wrote:
>>> On 10.09.2021 01:04, Oleksandr Tyshchenko wrote:
>>>> @@ -731,11 +733,19 @@ static void p2m_put_l3_page(const lpae_t pte)
>>>>         */
>>>>        if ( p2m_is_foreign(pte.p2m.type) )
>>>>        {
>>>> -        mfn_t mfn = lpae_get_mfn(pte);
>>>> -
>>>>            ASSERT(mfn_valid(mfn));
>>>>            put_page(mfn_to_page(mfn));
>>>>        }
>>>> +
>>>> +#ifdef CONFIG_GRANT_TABLE
>>>> +    /*
>>>> +     * Check whether we deal with grant table page. As the grant table page
>>>> +     * is xen_heap page and its entry has known p2m type, detect it and mark
>>>> +     * the stored GFN as invalid.
>>>> +     */
>>>> +    if ( p2m_is_ram(pte.p2m.type) && is_xen_heap_mfn(mfn) )
>>>> +        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
>>>> +#endif
>>> I take it the write done is benign for other Xen heap pages? I suppose
>>> this would want making very explicit, as such an assumption is easy to
>>> go stale by entirely unrelated changes.
>> Yes, I don't see what bad could happen if we would clear this field for
>> non grant table pages. Except grant table pages I could detect only one
>> page passed this check here which is, I assume, shared_info page. All
>> pages have this field initialized with INVALID_GFN. But *only* grant
>> table pages have this field in use. So, I think, no one will suffer. I
>> will add a comment. Or you meant something more than just a comment?
> The answer here goes hand in hand with the pending question of whether
> you wouldn't better generally introduce recording of the GFN (and hence
> effectively an M2P): The less special casing here, the better imo. The
> more special casing here, the more justification / explanation is imo
> needed in the comment.

I got your point. I think, I am not the right person to judge whether 
the Xen on Arm needs / wants complete M2P support or not (this is a 
question to the maintainers and the community),
but I can't recollect a strong desire or critical need for implementing 
it every time such discussion arose and IIRC only arguments against it. 
Also this activity is not in my list of priorities to do on Xen on Arm. 
But, I agree that having M2P would prevent this (with grant table) and 
similar situations. So, I will add more justification / explanation for 
the current patch.


>
>>>> --- a/xen/include/asm-arm/mm.h
>>>> +++ b/xen/include/asm-arm/mm.h
>>>> @@ -39,7 +39,15 @@ struct page_info
>>>>            /* Page is in use: ((count_info & PGC_count_mask) != 0). */
>>>>            struct {
>>>>                /* Type reference count and various PGT_xxx flags and fields. */
>>>> -            unsigned long type_info;
>>>> +            unsigned long type_info:4;
>>>> +
>>>> +            /*
>>>> +             * Stored GFN if page is used for grant table frame
>>>> +             * (bits 59(27)-0).
>>> Are these bit numbers correct? I thought Arm, like x86, counted bits
>>> from low to high (unlike PPC, for example)?
>> I think, these are correct.  Yes, Little Endian is used.
> Well, the low 4 bits are used by type_info here. Therefore I'm having
> trouble seeing what the 0 refers to.

Now I see that comment is not precise. Probably I should have written 
that "60-bit(28-bit) field is used to store the GFN ..."


>
>>>> @@ -94,12 +102,12 @@ struct page_info
>>>>    #define PG_shift(idx)   (BITS_PER_LONG - (idx))
>>>>    #define PG_mask(x, idx) (x ## UL << PG_shift(idx))
>>>>    
>>>> -#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
>>>> -#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>>>> -#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>>>> +#define PGT_none          (0UL << 3)  /* no special uses of this page   */
>>>> +#define PGT_writable_page (1UL << 3)  /* has writable mappings?         */
>>>> +#define PGT_type_mask     (1UL << 3)
>>>>    
>>>>     /* Count of uses of this frame as its current type. */
>>>> -#define PGT_count_width   PG_shift(2)
>>>> +#define PGT_count_width   1
>>>>    #define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>>> Leaving just a single bit seems pretty risky to me, and I can't see
>>> why you do so. You need 52 bits on Arm64. Which means you have 12
>>> bits left. Why not use them in full? Even on Arm32 you have 3 bits
>>> available for the count afaict.
>> Only 1 bit in the type_info field is really used on Arm, but anyway ...
>>
>>
>>> Also there's a disconnect here: If anyone wanted to change the number
>>> of bits used for the various fields, each such change should require
>>> an adjustment in as few independent places as possible. That is, there
>>> shouldn't be multiple uses of literal 4 further up, and there also
>>> shouldn't be a hidden connection between that 4 and the PGT_* values
>>> here. I think you'd better express the GFN as such a PGT_* construct
>>> as well. And you better would keep using PG_mask() and PG_shift().
>> ... I think, this indeed makes sense. If got your request correctly, we
>> can avoid disconnect here
>> and reserve 3-bit field for count for both Arm32 and Arm64. The struct
>> page_info's type_info field remains untouched.
>>
>>
>> diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
>> index ded74d2..8b73e1c 100644
>> --- a/xen/include/asm-arm/mm.h
>> +++ b/xen/include/asm-arm/mm.h
>> @@ -99,8 +99,14 @@ struct page_info
>>    #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or
>> 63.                 */
>>
>>     /* Count of uses of this frame as its current type. */
>> -#define PGT_count_width   PG_shift(2)
>> -#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>> +#define PGT_count_base    PG_shift(4)
>> +#define PGT_count_mask    PG_mask(7, 4)
>> +
>> +/* Stored in bits [27:0] or [59:0] GFN if page is used for grant table
>> frame */
>> +#define PGT_gfn_width     PG_shift(4)
>> +#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
>> +
>> +#define PGT_INVALID_FRAME_GFN   _gfn(PGT_gfn_mask)
>>
>>     /* Cleared when the owning guest 'frees' this page. */
>>    #define _PGC_allocated    PG_shift(1)
>> @@ -163,6 +169,22 @@ extern unsigned long xenheap_base_pdx;
>>
>>    #define maddr_get_owner(ma) (page_get_owner(maddr_to_page((ma))))
>>
>> +#define page_get_frame_gfn(_p) ({                               \
>> +    gfn_t gfn_ = _gfn((_p)->u.inuse.type_info & PGT_gfn_mask);  \
>> +    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
>> +})
>> +
>> +#define page_set_frame_gfn(_p, _gfn) ({                         \
>> +    gfn_t gfn_ = gfn_eq(_gfn, INVALID_GFN) ?                    \
>> +                 PGT_INVALID_FRAME_GFN : _gfn;                  \
>> +    (_p)->u.inuse.type_info &= ~PGT_gfn_mask;                   \
>> +    (_p)->u.inuse.type_info |= gfn_x(gfn_);                     \
>> +})
>>
>> [snip]
>>
>> Is it close to what you keep in mind?
> Yes. Just to be sure - did you check that moving the count up
> from starting at bit 0 is compatible with all present uses? At
> least on x86 I think we have uses where it is assumed to occupy
> the bottom so many bits (and the same also for PGC_count_mask).

ok, thank you. Yes, I checked that (on Arm). I could find only one place 
which would need updating, which was the share_xen_page_with_guest().


>
> Jan
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:36:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:36:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186541.335274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fS-00070Z-5c; Tue, 14 Sep 2021 12:36:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186541.335274; Tue, 14 Sep 2021 12:36:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fS-00070S-1h; Tue, 14 Sep 2021 12:36:06 +0000
Received: by outflank-mailman (input) for mailman id 186541;
 Tue, 14 Sep 2021 12:36:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ7fQ-00070M-JX
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:36:04 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1bcbe46a-b677-481e-ace2-8554af700768;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D74BB220BC;
 Tue, 14 Sep 2021 12:36:02 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6A4FB13342;
 Tue, 14 Sep 2021 12:36:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id n0JYGDKXQGE7MAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 12:36:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bcbe46a-b677-481e-ace2-8554af700768
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631622962; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=M1dIWQCkg38Q5PQjLGTJ1A9BSt19LN0OahLQLwcJiqc=;
	b=EIZG1R3W/2cd0m/AJaopGED7B2/odtslsTU+XcJg1pDVocGqA8FYZjlCx7jnOOOD7lVDAR
	PuSZWDud5jubQVJm6SsF1GcehcxMqxGuREzWuBiy2guLGacfCPQ03iV1FjEYoL4IKEO9V2
	D0DLuJP81RAXpV91a3pxeiXHxUQ5CzI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH RFC 0/4] remove libxenctrl usage from xenstored
Date: Tue, 14 Sep 2021 14:35:56 +0200
Message-Id: <20210914123600.1626-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Xenstored is using libxenctrl for only one purpose: to get information
about state of domains.

This RFC patch series is removing that dependency by introducing a new
stable interface which can be used by xenstored instead.

I have chosen to add a new hypercall with sub-commands. This can be
changed, of course, but I thought this would be a good step towards
stable control interfaces for the Xen tools.

I have done only very little testing (basically normal domain creation
and deletion). In case my approach is being accepted I'll do a more
thorough test.

What is missing right now is some cleanup (e.g. the need to allow
the dominfo operation of the domctl hypercall for a stubdom based
Xenstore could be removed). For Mini-OS some more work is needed to
remove the dependency to libxenctrl for xenstore-stubdom, as Mini-OS
can't be configured to use libxenevent and libxengnttab, but not
libxenctrl today. Again, I'll do that cleanup in case the series is
generally accepted.

Juergen Gross (4):
  xen: add a domain unique id to each domain
  xen: add bitmap to indicate per-domain state changes
  xen: add new stable control hypercall
  tools/xenstore: use new stable interface instead of libxenctrl

 tools/flask/policy/modules/dom0.te  |   2 +-
 tools/xenstore/Makefile             |   3 +-
 tools/xenstore/xenstored_control.c  |  14 +-
 tools/xenstore/xenstored_domain.c   | 219 +++++++++++++++-------------
 xen/arch/arm/traps.c                |   1 +
 xen/arch/x86/hvm/hypercall.c        |   1 +
 xen/arch/x86/hypercall.c            |   1 +
 xen/arch/x86/pv/hypercall.c         |   1 +
 xen/common/Makefile                 |   1 +
 xen/common/control.c                |  52 +++++++
 xen/common/domain.c                 |  76 ++++++++++
 xen/common/event_channel.c          |  11 +-
 xen/include/Makefile                |   1 +
 xen/include/public/control.h        |  80 ++++++++++
 xen/include/public/xen.h            |   1 +
 xen/include/xen/event.h             |   6 +
 xen/include/xen/hypercall.h         |   5 +
 xen/include/xen/sched.h             |   7 +
 xen/include/xsm/dummy.h             |  14 ++
 xen/include/xsm/xsm.h               |   6 +
 xen/xsm/dummy.c                     |   1 +
 xen/xsm/flask/hooks.c               |   6 +
 xen/xsm/flask/policy/access_vectors |   2 +
 23 files changed, 402 insertions(+), 109 deletions(-)
 create mode 100644 xen/common/control.c
 create mode 100644 xen/include/public/control.h

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:36:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:36:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186542.335285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fU-0007HC-Cz; Tue, 14 Sep 2021 12:36:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186542.335285; Tue, 14 Sep 2021 12:36:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fU-0007H1-99; Tue, 14 Sep 2021 12:36:08 +0000
Received: by outflank-mailman (input) for mailman id 186542;
 Tue, 14 Sep 2021 12:36:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ7fT-0007Eq-4s
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:36:07 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5388358e-1558-11ec-b421-12813bfff9fa;
 Tue, 14 Sep 2021 12:36:05 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3AC06220EA;
 Tue, 14 Sep 2021 12:36:04 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id F3D8113F2C;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id KAwKOjOXQGE7MAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 12:36:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5388358e-1558-11ec-b421-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631622964; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EH+dU3e+z7VJuNtk1nknzKQyPzqsv/7YUiwjiyb6J/Y=;
	b=TAlRcuSOmk3CTnhnWvKPvzACQ7Z/l+R/61XeosOOYScQ8OsPEoN/cJtEFpSfb2XQBsR1JQ
	Je4hMLDeXehqBLqwUo9bY4QCdnKRmkQ+VlSSeAL4HtSp1/tYGJF7hwGGkgcAEOWnFLOMor
	EI0btNofF2tB5b5ozK7zgzAIhlb/LsY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH RFC 4/4] tools/xenstore: use new stable interface instead of libxenctrl
Date: Tue, 14 Sep 2021 14:36:00 +0200
Message-Id: <20210914123600.1626-5-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210914123600.1626-1-jgross@suse.com>
References: <20210914123600.1626-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Xenstore is using libxenctrl only for obtaining state information
about domains.

Use the new stable interface XEN_CONTROL_OP_get_state_changed_domain
instead.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/xenstore/Makefile            |   3 +-
 tools/xenstore/xenstored_control.c |  14 +-
 tools/xenstore/xenstored_domain.c  | 219 ++++++++++++++++-------------
 3 files changed, 130 insertions(+), 106 deletions(-)

diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile
index 292b478fa1..b78e29470b 100644
--- a/tools/xenstore/Makefile
+++ b/tools/xenstore/Makefile
@@ -11,6 +11,7 @@ CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += -I./include
 CFLAGS += $(CFLAGS_libxenevtchn)
 CFLAGS += $(CFLAGS_libxenctrl)
+CFLAGS += $(CFLAGS_libxencall)
 CFLAGS += $(CFLAGS_libxenguest)
 CFLAGS += $(CFLAGS_libxentoolcore)
 CFLAGS += -DXEN_LIB_STORED="\"$(XEN_LIB_STORED)\""
@@ -70,7 +71,7 @@ endif
 $(XENSTORED_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
 
 xenstored: $(XENSTORED_OBJS)
-	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
+	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxencall) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
 
 xenstored.a: $(XENSTORED_OBJS)
 	$(AR) cr $@ $^
diff --git a/tools/xenstore/xenstored_control.c b/tools/xenstore/xenstored_control.c
index 7b4300ef77..d767aea0f9 100644
--- a/tools/xenstore/xenstored_control.c
+++ b/tools/xenstore/xenstored_control.c
@@ -594,6 +594,13 @@ void lu_read_state(void)
 	void *ctx = talloc_new(NULL); /* Work context for subfunctions. */
 	struct xs_state_preamble *pre;
 
+	/*
+	 * We may have missed the VIRQ_DOM_EXC notification and a domain may
+	 * have died while we were live-updating. So check all the domains are
+	 * still alive. This will pre-initialize all domain structures.
+	 */
+	check_domains();
+
 	syslog(LOG_INFO, "live-update: read state\n");
 	lu_get_dump_state(&state);
 	if (state.size == 0)
@@ -634,13 +641,6 @@ void lu_read_state(void)
 	lu_close_dump_state(&state);
 
 	talloc_free(ctx);
-
-	/*
-	 * We may have missed the VIRQ_DOM_EXC notification and a domain may
-	 * have died while we were live-updating. So check all the domains are
-	 * still alive.
-	 */
-	check_domains();
 }
 
 static const char *lu_activate_binary(const void *ctx)
diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
index 47e9107c14..2801e4b24f 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -31,11 +31,12 @@
 #include "xenstored_transaction.h"
 #include "xenstored_watch.h"
 
+#include <xencall.h>
 #include <xenevtchn.h>
-#include <xenctrl.h>
+#include <xen/control.h>
 #include <xen/grant_table.h>
 
-static xc_interface **xc_handle;
+static xencall_handle **xc_handle;
 xengnttab_handle **xgt_handle;
 static evtchn_port_t virq_port;
 
@@ -50,6 +51,7 @@ struct domain
 
 	/* The id of this domain */
 	unsigned int domid;
+	uint64_t unique_id;
 
 	/* Event channel port */
 	evtchn_port_t port;
@@ -238,54 +240,114 @@ static int destroy_domain(void *_domain)
 	return 0;
 }
 
-static bool get_domain_info(unsigned int domid, xc_dominfo_t *dominfo)
+static bool get_domain_info(struct xen_control_changed_domain *dominfo)
 {
-	return xc_domain_getinfo(*xc_handle, domid, 1, dominfo) == 1 &&
-	       dominfo->domid == domid;
+	struct xen_control_changed_domain *buf;
+	int ret;
+
+	buf = xencall_alloc_buffer(*xc_handle, sizeof(*buf));
+	if (!buf)
+		return false;
+	ret = xencall2(*xc_handle, __HYPERVISOR_control_op,
+		       XEN_CONTROL_OP_get_state_changed_domain,
+		       (unsigned long)buf);
+	*dominfo = *buf;
+	xencall_free_buffer(*xc_handle, buf);
+
+	return ret == 0;
 }
 
-void check_domains(void)
+static struct domain *find_domain_struct(unsigned int domid)
+{
+	struct domain *i;
+
+	list_for_each_entry(i, &domains, list) {
+		if (i->domid == domid)
+			return i;
+	}
+	return NULL;
+}
+
+static struct domain *alloc_domain(unsigned int domid)
 {
-	xc_dominfo_t dominfo;
 	struct domain *domain;
+
+	domain = talloc_zero(talloc_autofree_context(), struct domain);
+	if (!domain) {
+		errno = ENOMEM;
+		return NULL;
+	}
+
+	domain->domid = domid;
+	domain->generation = generation;
+	domain->introduced = false;
+
+	talloc_set_destructor(domain, destroy_domain);
+
+	list_add(&domain->list, &domains);
+
+	return domain;
+}
+
+static void domain_drop(struct domain *domain)
+{
 	struct connection *conn;
+
+	/* domain is a talloc child of domain->conn. */
+	conn = domain->conn;
+	domain->conn = NULL;
+	talloc_unlink(talloc_autofree_context(), conn);
+}
+
+void check_domains(void)
+{
+	struct xen_control_changed_domain dominfo;
+	struct domain *domain;
 	int notify = 0;
-	bool dom_valid;
 
- again:
-	list_for_each_entry(domain, &domains, list) {
-		dom_valid = get_domain_info(domain->domid, &dominfo);
-		if (!domain->introduced) {
-			if (!dom_valid) {
+	while (get_domain_info(&dominfo)) {
+		domain = find_domain_struct(dominfo.domid);
+
+		if (!dominfo.state) {
+			if (domain && !domain->introduced)
 				talloc_free(domain);
-				goto again;
-			}
 			continue;
 		}
-		if (dom_valid) {
-			if ((dominfo.crashed || dominfo.shutdown)
-			    && !domain->shutdown) {
-				domain->shutdown = true;
-				notify = 1;
-			}
-			/*
-			 * On Restore, we may have been unable to remap the
-			 * interface and the port. As we don't know whether
-			 * this was because of a dying domain, we need to
-			 * check if the interface and port are still valid.
-			 */
-			if (!dominfo.dying && domain->port &&
-			    domain->interface)
+
+		if (domain && domain->unique_id &&
+		    domain->unique_id != dominfo.unique_id) {
+			if (domain->conn)
+				domain_drop(domain);
+			else
+				talloc_free(domain);
+			domain = NULL;
+		}
+
+		if (!domain) {
+			domain = alloc_domain(dominfo.domid);
+			if (!domain)
 				continue;
+			domain->unique_id = dominfo.unique_id;
 		}
-		if (domain->conn) {
-			/* domain is a talloc child of domain->conn. */
-			conn = domain->conn;
-			domain->conn = NULL;
-			talloc_unlink(talloc_autofree_context(), conn);
-			notify = 0; /* destroy_domain() fires the watch */
-			goto again;
+
+		if ((dominfo.state & XEN_CONTROL_CHANGEDDOM_STATE_SHUTDOWN) &&
+		    !domain->shutdown) {
+			domain->shutdown = true;
+			notify = 1;
 		}
+
+		/*
+		 * On Restore, we may have been unable to remap the
+		 * interface and the port. As we don't know whether
+		 * this was because of a dying domain, we need to
+		 * check if the interface and port are still valid.
+		 */
+		if (!(dominfo.state & XEN_CONTROL_CHANGEDDOM_STATE_DYING) &&
+		    domain->port && domain->interface)
+			continue;
+
+		if (domain->conn)
+			domain_drop(domain);
 	}
 
 	if (notify)
@@ -323,46 +385,6 @@ static char *talloc_domain_path(void *context, unsigned int domid)
 	return talloc_asprintf(context, "/local/domain/%u", domid);
 }
 
-static struct domain *find_domain_struct(unsigned int domid)
-{
-	struct domain *i;
-
-	list_for_each_entry(i, &domains, list) {
-		if (i->domid == domid)
-			return i;
-	}
-	return NULL;
-}
-
-static struct domain *alloc_domain(const void *context, unsigned int domid)
-{
-	struct domain *domain;
-
-	domain = talloc(context, struct domain);
-	if (!domain) {
-		errno = ENOMEM;
-		return NULL;
-	}
-
-	domain->domid = domid;
-	domain->generation = generation;
-	domain->introduced = false;
-
-	talloc_set_destructor(domain, destroy_domain);
-
-	list_add(&domain->list, &domains);
-
-	return domain;
-}
-
-static struct domain *find_or_alloc_domain(const void *ctx, unsigned int domid)
-{
-	struct domain *domain;
-
-	domain = find_domain_struct(domid);
-	return domain ? : alloc_domain(ctx, domid);
-}
-
 static int new_domain(struct domain *domain, int port, bool restore)
 {
 	int rc;
@@ -443,9 +465,13 @@ static struct domain *introduce_domain(const void *ctx,
 	struct xenstore_domain_interface *interface;
 	bool is_master_domain = (domid == xenbus_master_domid());
 
-	domain = find_or_alloc_domain(ctx, domid);
-	if (!domain)
+	check_domains();
+
+	domain = find_domain_struct(domid);
+	if (!domain) {
+		errno = ENOENT;
 		return NULL;
+	}
 
 	if (!domain->introduced) {
 		interface = is_master_domain ? xenbus_map()
@@ -650,7 +676,7 @@ int do_reset_watches(struct connection *conn, struct buffered_data *in)
 
 static int close_xc_handle(void *_handle)
 {
-	xc_interface_close(*(xc_interface**)_handle);
+	xencall_close(*(xencall_handle **)_handle);
 	return 0;
 }
 
@@ -741,11 +767,11 @@ void domain_init(int evtfd)
 {
 	int rc;
 
-	xc_handle = talloc(talloc_autofree_context(), xc_interface*);
+	xc_handle = talloc(talloc_autofree_context(), xencall_handle *);
 	if (!xc_handle)
 		barf_perror("Failed to allocate domain handle");
 
-	*xc_handle = xc_interface_open(0,0,0);
+	*xc_handle = xencall_open(NULL, 0);
 	if (!*xc_handle)
 		barf_perror("Failed to open connection to hypervisor");
 
@@ -821,7 +847,6 @@ void domain_entry_inc(struct connection *conn, struct node *node)
  * count (used for testing whether a node permission is older than a domain).
  *
  * Return values:
- * -1: error
  *  0: domain has higher generation count (it is younger than a node with the
  *     given count), or domain isn't existing any longer
  *  1: domain is older than the node
@@ -829,7 +854,6 @@ void domain_entry_inc(struct connection *conn, struct node *node)
 static int chk_domain_generation(unsigned int domid, uint64_t gen)
 {
 	struct domain *d;
-	xc_dominfo_t dominfo;
 
 	if (!xc_handle && domid == 0)
 		return 1;
@@ -838,11 +862,9 @@ static int chk_domain_generation(unsigned int domid, uint64_t gen)
 	if (d)
 		return (d->generation <= gen) ? 1 : 0;
 
-	if (!get_domain_info(domid, &dominfo))
-		return 0;
-
-	d = alloc_domain(NULL, domid);
-	return d ? 1 : -1;
+	check_domains();
+	d = find_domain_struct(domid);
+	return (d && d->generation <= gen) ? 1 : 0;
 }
 
 /*
@@ -855,8 +877,6 @@ int domain_adjust_node_perms(struct node *node)
 	int ret;
 
 	ret = chk_domain_generation(node->perms.p[0].id, node->generation);
-	if (ret < 0)
-		return errno;
 
 	/* If the owner doesn't exist any longer give it to priv domain. */
 	if (!ret)
@@ -867,8 +887,6 @@ int domain_adjust_node_perms(struct node *node)
 			continue;
 		ret = chk_domain_generation(node->perms.p[i].id,
 					    node->generation);
-		if (ret < 0)
-			return errno;
 		if (!ret)
 			node->perms.p[i].perms |= XS_PERM_IGNORE;
 	}
@@ -1300,16 +1318,21 @@ void read_state_connection(const void *ctx, const void *state)
 	} else {
 		domain = introduce_domain(ctx, sc->spec.ring.domid,
 					  sc->spec.ring.evtchn, true);
-		if (!domain)
+		if (!domain) {
+			/* Domain vanished during LM? */
+			if (errno = ENOENT)
+				return;
 			barf("domain allocation error");
+		}
 
 		if (sc->spec.ring.tdomid != DOMID_INVALID) {
-			tdomain = find_or_alloc_domain(ctx,
-						       sc->spec.ring.tdomid);
-			if (!tdomain)
+			tdomain = find_domain_struct(sc->spec.ring.tdomid);
+			if (!tdomain && errno != ENOENT)
 				barf("target domain allocation error");
-			talloc_reference(domain->conn, tdomain->conn);
-			domain->conn->target = tdomain->conn;
+			if (tdomain) {
+				talloc_reference(domain->conn, tdomain->conn);
+				domain->conn->target = tdomain->conn;
+			}
 		}
 		conn = domain->conn;
 	}
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:36:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:36:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186543.335296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fW-0007Zu-Oy; Tue, 14 Sep 2021 12:36:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186543.335296; Tue, 14 Sep 2021 12:36:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fW-0007Zh-LP; Tue, 14 Sep 2021 12:36:10 +0000
Received: by outflank-mailman (input) for mailman id 186543;
 Tue, 14 Sep 2021 12:36:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ7fV-00070M-Gg
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:36:09 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id be5117b7-fc42-49a6-9ea0-c31c1553aa9d;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2DD48220E6;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DD6DF13F2C;
 Tue, 14 Sep 2021 12:36:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id KE6gNDKXQGE7MAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 12:36:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be5117b7-fc42-49a6-9ea0-c31c1553aa9d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631622963; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vq2/ecChHTKwpSB6rnjbRKzVPD40UedVXsB5/I4nS/M=;
	b=rU1nkO5IUXnR1p4uMZduS7O0st8GM0JMYHMkpri5KqLTaYTMa6gSxJsH4yW7+LfNRxNr1s
	+03OjL4fZErgmcWaUwL4M2/FRmzlKIy/JFUFm60Jdmb1vG1sPbF2GSwMv2PGbDM6h9yvLA
	M7ymz/Nl2KiiuBPS97+K842aHK/Lk70=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH RFC 1/4] xen: add a domain unique id to each domain
Date: Tue, 14 Sep 2021 14:35:57 +0200
Message-Id: <20210914123600.1626-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210914123600.1626-1-jgross@suse.com>
References: <20210914123600.1626-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Xenstore is referencing domains by their domid, but reuse of a domid
can lead to the situation that Xenstore can't tell whether a domain
with that domid has been deleted and created again without Xenstore
noticing the domain is a new one now.

Add a global domain creation unique id which is updated when creating
a new domain, and store that value in struct domain of the new domain.
The global unique id is initialized with the system time and updates
are done via the xorshift algorithm which is used for pseudo random
number generation, too (see https://en.wikipedia.org/wiki/Xorshift).

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/common/domain.c     | 16 ++++++++++++++++
 xen/include/xen/sched.h |  3 +++
 2 files changed, 19 insertions(+)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..755349b93f 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -84,6 +84,9 @@ vcpu_info_t dummy_vcpu_info;
 
 bool __read_mostly vmtrace_available;
 
+/* Unique domain identifier, protected by domctl lock. */
+static uint64_t unique_id;
+
 static void __domain_finalise_shutdown(struct domain *d)
 {
     struct vcpu *v;
@@ -473,6 +476,18 @@ static void _domain_destroy(struct domain *d)
     free_domain_struct(d);
 }
 
+static uint64_t get_unique_id(void)
+{
+    uint64_t x = unique_id ? : NOW();
+
+    x ^= x << 13;
+    x ^= x >> 7;
+    x ^= x << 17;
+    unique_id = x;
+
+    return x;
+}
+
 static int sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
     bool hvm = config->flags & XEN_DOMCTL_CDF_hvm;
@@ -552,6 +567,7 @@ struct domain *domain_create(domid_t domid,
 
     /* Sort out our idea of is_system_domain(). */
     d->domain_id = domid;
+    d->unique_id = get_unique_id();
 
     /* Debug sanity. */
     ASSERT(is_system_domain(d) ? config == NULL : config != NULL);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404..b827c5af8d 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -362,6 +362,9 @@ struct domain
     domid_t          domain_id;
 
     unsigned int     max_vcpus;
+
+    uint64_t         unique_id;       /* Unique domain identifier */
+
     struct vcpu    **vcpu;
 
     shared_info_t   *shared_info;     /* shared data area */
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:36:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:36:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186544.335307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fc-0007yo-4x; Tue, 14 Sep 2021 12:36:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186544.335307; Tue, 14 Sep 2021 12:36:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fc-0007yc-1U; Tue, 14 Sep 2021 12:36:16 +0000
Received: by outflank-mailman (input) for mailman id 186544;
 Tue, 14 Sep 2021 12:36:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ7fa-00070M-Gg
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:36:14 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0816215a-03ed-47af-b250-f3bce5de19ac;
 Tue, 14 Sep 2021 12:36:04 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7F8F9220E7;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3637413342;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id EHaWCzOXQGE7MAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 12:36:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0816215a-03ed-47af-b250-f3bce5de19ac
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631622963; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gmpX0RV1KtNAd1QGZZ8I7TEoSv8qu+7eNCb6tbvk9tE=;
	b=h3Z9wwfBYUyrkUqm9vPejr3MHUGnNFISxmcAAQEWYJ+PVYI2Kgz35r4YNLWCzXkrGniN+M
	xSYPhZUr/BRnzTiRUxqfDi1GGHoTGCwpwyzQTb1P66bZQGIcwDB4/YBC1SDQN7S9G0MmVH
	6uoW9xA8LX7E/2V1Lfu1AiSvVSalaLs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH RFC 2/4] xen: add bitmap to indicate per-domain state changes
Date: Tue, 14 Sep 2021 14:35:58 +0200
Message-Id: <20210914123600.1626-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210914123600.1626-1-jgross@suse.com>
References: <20210914123600.1626-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a bitmap with one bit per possible domid indicating the respective
domain has changed its state (created, deleted, dying, crashed,
shutdown).

Registering the VIRQ_DOM_EXC event will result in setting the bits for
all existing domains and resetting all other bits.

Resetting a bit will be done in a future patch.

This information is needed for Xenstore to keep track of all domains.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/common/domain.c        | 22 ++++++++++++++++++++++
 xen/common/event_channel.c |  2 ++
 xen/include/xen/sched.h    |  2 ++
 3 files changed, 26 insertions(+)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index 755349b93f..14a427e2ef 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -87,6 +87,22 @@ bool __read_mostly vmtrace_available;
 /* Unique domain identifier, protected by domctl lock. */
 static uint64_t unique_id;
 
+static DECLARE_BITMAP(dom_state_changed, DOMID_MASK + 1);
+
+void domain_reset_states(void)
+{
+    struct domain *d;
+
+    bitmap_zero(dom_state_changed, DOMID_MASK + 1);
+
+    rcu_read_lock(&domlist_read_lock);
+
+    for_each_domain ( d )
+        set_bit(d->domain_id, dom_state_changed);
+
+    rcu_read_unlock(&domlist_read_lock);
+}
+
 static void __domain_finalise_shutdown(struct domain *d)
 {
     struct vcpu *v;
@@ -101,6 +117,7 @@ static void __domain_finalise_shutdown(struct domain *d)
             return;
 
     d->is_shut_down = 1;
+    set_bit(d->domain_id, dom_state_changed);
     if ( (d->shutdown_code == SHUTDOWN_suspend) && d->suspend_evtchn )
         evtchn_send(d, d->suspend_evtchn);
     else
@@ -720,6 +737,8 @@ struct domain *domain_create(domid_t domid,
         rcu_assign_pointer(domain_hash[DOMAIN_HASH(domid)], d);
         spin_unlock(&domlist_update_lock);
 
+        set_bit(d->domain_id, dom_state_changed);
+
         memcpy(d->handle, config->handle, sizeof(d->handle));
     }
 
@@ -954,6 +973,7 @@ int domain_kill(struct domain *d)
         /* Mem event cleanup has to go here because the rings 
          * have to be put before we call put_domain. */
         vm_event_cleanup(d);
+        set_bit(d->domain_id, dom_state_changed);
         put_domain(d);
         send_global_virq(VIRQ_DOM_EXC);
         /* fallthrough */
@@ -1141,6 +1161,8 @@ static void complete_domain_destroy(struct rcu_head *head)
 
     xfree(d->vcpu);
 
+    set_bit(d->domain_id, dom_state_changed);
+
     _domain_destroy(d);
 
     send_global_virq(VIRQ_DOM_EXC);
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index da88ad141a..e2a416052b 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -1218,6 +1218,8 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         rc = evtchn_bind_virq(&bind_virq, 0);
         if ( !rc && __copy_to_guest(arg, &bind_virq, 1) )
             rc = -EFAULT; /* Cleaning up here would be a mess! */
+        if ( !rc && bind_virq.virq == VIRQ_DOM_EXC )
+            domain_reset_states();
         break;
     }
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index b827c5af8d..2ae26bc50e 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -730,6 +730,8 @@ void domain_resume(struct domain *d);
 
 int domain_soft_reset(struct domain *d, bool resuming);
 
+void domain_reset_states(void);
+
 int vcpu_start_shutdown_deferral(struct vcpu *v);
 void vcpu_end_shutdown_deferral(struct vcpu *v);
 
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:36:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:36:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186545.335318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fh-0008QF-Cr; Tue, 14 Sep 2021 12:36:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186545.335318; Tue, 14 Sep 2021 12:36:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7fh-0008Q6-9P; Tue, 14 Sep 2021 12:36:21 +0000
Received: by outflank-mailman (input) for mailman id 186545;
 Tue, 14 Sep 2021 12:36:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8exS=OE=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQ7ff-00070M-Gi
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:36:19 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fe3c330b-c808-4ed7-90ee-e9b4efad6c37;
 Tue, 14 Sep 2021 12:36:04 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id EDEF9220E9;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8981613342;
 Tue, 14 Sep 2021 12:36:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 0IUlIDOXQGE7MAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 14 Sep 2021 12:36:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe3c330b-c808-4ed7-90ee-e9b4efad6c37
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631622963; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+ePWQiFmb8Y2hRzTVgAeltcrXsQ9l+i0Wvg5OPdAaKw=;
	b=ENmoeESBNEgQsESshQLy23eluDlQgcGMVMebX2ukzx9cicUzcZ55hXw2Hm8hm9S2QmUXYX
	d+/AzjdPJzw159w9uu27qMLW812t35MSVKqgR64g/OBO+xgPeS8RJmcnQweFqeU9o8yYTl
	0P9H2QKGx4G4Rj1QR8zb0qew770+a2E=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH RFC 3/4] xen: add new stable control hypercall
Date: Tue, 14 Sep 2021 14:35:59 +0200
Message-Id: <20210914123600.1626-4-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210914123600.1626-1-jgross@suse.com>
References: <20210914123600.1626-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The sysctl and domctl hypercalls are not stable, so tools using those
need to be in sync with the hypervisor.

In order to decouple (some) tools from the hypervisor add a new stable
hypercall __HYPERVISOR_control_op with (for now) two sub-options:

- XEN_CONTROL_OP_get_version for retrieving the max version of the new
  hypercall supported by the hypervisor
- XEN_CONTROL_OP_get_state_changed_domain for retrieving some state
  data of a domain which changed state (this is needed by Xenstore).
  The returned state just contains the domid, the domain unique id,
  and some flags (existing, shutdown, dying).

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/flask/policy/modules/dom0.te  |  2 +-
 xen/arch/arm/traps.c                |  1 +
 xen/arch/x86/hvm/hypercall.c        |  1 +
 xen/arch/x86/hypercall.c            |  1 +
 xen/arch/x86/pv/hypercall.c         |  1 +
 xen/common/Makefile                 |  1 +
 xen/common/control.c                | 52 +++++++++++++++++++
 xen/common/domain.c                 | 38 ++++++++++++++
 xen/common/event_channel.c          |  9 +++-
 xen/include/Makefile                |  1 +
 xen/include/public/control.h        | 80 +++++++++++++++++++++++++++++
 xen/include/public/xen.h            |  1 +
 xen/include/xen/event.h             |  6 +++
 xen/include/xen/hypercall.h         |  5 ++
 xen/include/xen/sched.h             |  2 +
 xen/include/xsm/dummy.h             | 14 +++++
 xen/include/xsm/xsm.h               |  6 +++
 xen/xsm/dummy.c                     |  1 +
 xen/xsm/flask/hooks.c               |  6 +++
 xen/xsm/flask/policy/access_vectors |  2 +
 20 files changed, 227 insertions(+), 3 deletions(-)
 create mode 100644 xen/common/control.c
 create mode 100644 xen/include/public/control.h

diff --git a/tools/flask/policy/modules/dom0.te b/tools/flask/policy/modules/dom0.te
index 0a63ce15b6..5c5e4af56c 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -11,7 +11,7 @@ allow dom0_t xen_t:xen {
 	mtrr_del mtrr_read microcode physinfo quirk writeconsole readapic
 	writeapic privprofile nonprivprofile kexec firmware sleep frequency
 	getidle debug getcpuinfo heap pm_op mca_op lockprof cpupool_op
-	getscheduler setscheduler hypfs_op
+	getscheduler setscheduler hypfs_op control_op
 };
 allow dom0_t xen_t:xen2 {
 	resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 219ab3c3fb..78802a5660 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1397,6 +1397,7 @@ static arm_hypercall_t arm_hypercall_table[] = {
 #ifdef CONFIG_IOREQ_SERVER
     HYPERCALL(dm_op, 3),
 #endif
+    HYPERCALL(control_op, 2),
 };
 
 #ifndef NDEBUG
diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 261d8ee8a4..007fcf5cb0 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -154,6 +154,7 @@ static const struct {
 #ifdef CONFIG_HYPFS
     HYPERCALL(hypfs_op),
 #endif
+    HYPERCALL(control_op),
     HYPERCALL(arch_1)
 };
 
diff --git a/xen/arch/x86/hypercall.c b/xen/arch/x86/hypercall.c
index 2370d31d3f..53523bac43 100644
--- a/xen/arch/x86/hypercall.c
+++ b/xen/arch/x86/hypercall.c
@@ -74,6 +74,7 @@ const hypercall_args_t hypercall_args_table[NR_hypercalls] =
     ARGS(hvm_op, 2),
     ARGS(dm_op, 3),
     ARGS(hypfs_op, 5),
+    ARGS(control_op, 2),
     ARGS(mca, 1),
     ARGS(arch_1, 1),
 };
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index 9765e674cf..8f81c1e645 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -99,6 +99,7 @@ const pv_hypercall_table_t pv_hypercall_table[] = {
 #ifdef CONFIG_HYPFS
     HYPERCALL(hypfs_op),
 #endif
+    HYPERCALL(control_op),
     HYPERCALL(mca),
 #ifndef CONFIG_PV_SHIM_EXCLUSIVE
     HYPERCALL(arch_1),
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 54de70d422..3c44def563 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -1,6 +1,7 @@
 obj-$(CONFIG_ARGO) += argo.o
 obj-y += bitmap.o
 obj-$(CONFIG_HYPFS_CONFIG) += config_data.o
+obj-y += control.o
 obj-$(CONFIG_CORE_PARKING) += core_parking.o
 obj-y += cpu.o
 obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
diff --git a/xen/common/control.c b/xen/common/control.c
new file mode 100644
index 0000000000..f92178d461
--- /dev/null
+++ b/xen/common/control.c
@@ -0,0 +1,52 @@
+/******************************************************************************
+ *
+ * control.c
+ *
+ * Entry point of the stable __HYPERVISOR_control_op hypercall.
+ */
+#include <xen/err.h>
+#include <xen/event.h>
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <public/control.h>
+
+long do_control_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    int ret = 0;
+
+    if ( xsm_control_op(XSM_OTHER, cmd) )
+        return -EPERM;
+
+    switch ( cmd )
+    {
+    case XEN_CONTROL_OP_get_version:
+        if ( !guest_handle_is_null(arg) )
+            return -EINVAL;
+
+        ret = XEN_CONTROL_VERSION;
+        break;
+
+    case XEN_CONTROL_OP_get_state_changed_domain:
+    {
+        struct xen_control_changed_domain info = { };
+
+        if ( get_global_virq_handler(VIRQ_DOM_EXC) != current->domain )
+            return -EPERM;
+
+        ret = domain_get_dom_state_changed(&info);
+        if ( ret )
+            break;
+
+        if ( copy_to_guest(arg, &info, 1) )
+            ret = -EFAULT;
+
+        break;
+    }
+
+    default:
+        ret = -EOPNOTSUPP;
+        break;
+    }
+
+    return ret;
+}
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 14a427e2ef..d6d729c485 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -36,6 +36,7 @@
 #include <asm/debugger.h>
 #include <asm/p2m.h>
 #include <asm/processor.h>
+#include <public/control.h>
 #include <public/sched.h>
 #include <public/sysctl.h>
 #include <public/vcpu.h>
@@ -103,6 +104,43 @@ void domain_reset_states(void)
     rcu_read_unlock(&domlist_read_lock);
 }
 
+int domain_get_dom_state_changed(struct xen_control_changed_domain *info)
+{
+    unsigned int dom;
+    struct domain *d;
+
+    while ( (dom = find_first_bit(dom_state_changed, DOMID_MASK + 1)) <
+            DOMID_FIRST_RESERVED )
+    {
+        d = rcu_lock_domain_by_id(dom);
+
+        if ( test_and_clear_bit(dom, dom_state_changed) )
+        {
+            info->domid = dom;
+            if ( d )
+            {
+                info->state = XEN_CONTROL_CHANGEDDOM_STATE_EXIST;
+                if ( d->is_shut_down )
+                    info->state |= XEN_CONTROL_CHANGEDDOM_STATE_SHUTDOWN;
+                if ( d->is_dying == DOMDYING_dead )
+                    info->state |= XEN_CONTROL_CHANGEDDOM_STATE_DYING;
+                info->unique_id = d->unique_id;
+
+                rcu_unlock_domain(d);
+            }
+
+            return 0;
+        }
+
+        if ( d )
+        {
+            rcu_unlock_domain(d);
+        }
+    }
+
+    return -ENOENT;
+}
+
 static void __domain_finalise_shutdown(struct domain *d)
 {
     struct vcpu *v;
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index e2a416052b..b5f377c76f 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -894,11 +894,16 @@ static struct domain *global_virq_handlers[NR_VIRQS] __read_mostly;
 
 static DEFINE_SPINLOCK(global_virq_handlers_lock);
 
-void send_global_virq(uint32_t virq)
+struct domain *get_global_virq_handler(uint32_t virq)
 {
     ASSERT(virq_is_global(virq));
 
-    send_guest_global_virq(global_virq_handlers[virq] ?: hardware_domain, virq);
+    return global_virq_handlers[virq] ?: hardware_domain;
+}
+
+void send_global_virq(uint32_t virq)
+{
+    send_guest_global_virq(get_global_virq_handler(virq), virq);
 }
 
 int set_global_virq_handler(struct domain *d, uint32_t virq)
diff --git a/xen/include/Makefile b/xen/include/Makefile
index 95daa8a289..adf61f40c3 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -4,6 +4,7 @@ compat-arch-$(CONFIG_X86) := x86_32
 
 headers-y := \
     compat/arch-$(compat-arch-y).h \
+    compat/control.h \
     compat/elfnote.h \
     compat/event_channel.h \
     compat/features.h \
diff --git a/xen/include/public/control.h b/xen/include/public/control.h
new file mode 100644
index 0000000000..0b2a032e96
--- /dev/null
+++ b/xen/include/public/control.h
@@ -0,0 +1,80 @@
+/******************************************************************************
+ * Xen Control Hypercall
+ *
+ * Copyright (c) 2021, SUSE Software Solutions Germany GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __XEN_PUBLIC_CONTROL_H__
+#define __XEN_PUBLIC_CONTROL_H__
+
+#include "xen.h"
+
+/*
+ * Definitions for the __HYPERVISOR_control_op hypercall.
+ */
+
+/* Highest version number of the control interface currently defined. */
+#define XEN_CONTROL_VERSION      1
+
+/*
+ * Hypercall operations.
+ */
+
+/*
+ * XEN_CONTROL_OP_get_version
+ *
+ * Read highest interface version supported by the hypervisor.
+ *
+ * arg: NULL
+ *
+ * Possible return values:
+ * >0: highest supported interface version
+ * <0: negative Xen errno value
+ */
+#define XEN_CONTROL_OP_get_version                  0
+
+/*
+ * XEN_CONTROL_OP_get_state_changed_domain
+ *
+ * Get information about a domain having changed state and reset the state
+ * change indicator for that domain. This function is usable only by a domain
+ * having registered the VIRQ_DOM_EXC event (normally Xenstore).
+ *
+ * arg: XEN_GUEST_HANDLE(struct xen_control_changed_domain)
+ *
+ * Possible return values:
+ * 0: success
+ * <0 : negative Xen errno value
+ */
+#define XEN_CONTROL_OP_get_state_changed_domain     1
+struct xen_control_changed_domain {
+    domid_t domid;
+    uint16_t state;
+#define XEN_CONTROL_CHANGEDDOM_STATE_EXIST     0x0001  /* Domain is existing. */
+#define XEN_CONTROL_CHANGEDDOM_STATE_SHUTDOWN  0x0002  /* Shutdown finished. */
+#define XEN_CONTROL_CHANGEDDOM_STATE_DYING     0x0004  /* Domain dying. */
+    uint32_t pad1;           /* Returned as 0. */
+    uint64_t unique_id;      /* Unique domain identifier. */
+    uint64_t pad2[6];        /* Returned as 0. */
+};
+
+#endif /* __XEN_PUBLIC_CONTROL_H__ */
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index e373592c33..1344fbcc36 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -131,6 +131,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
 #define __HYPERVISOR_xenpmu_op            40
 #define __HYPERVISOR_dm_op                41
 #define __HYPERVISOR_hypfs_op             42
+#define __HYPERVISOR_control_op           43
 
 /* Architecture-specific hypercall definitions. */
 #define __HYPERVISOR_arch_0               48
diff --git a/xen/include/xen/event.h b/xen/include/xen/event.h
index 21c95e14fd..c770f29e8e 100644
--- a/xen/include/xen/event.h
+++ b/xen/include/xen/event.h
@@ -43,6 +43,12 @@ void send_guest_global_virq(struct domain *d, uint32_t virq);
  */
 int set_global_virq_handler(struct domain *d, uint32_t virq);
 
+/*
+ * get_global_virq_handler: Get domain handling a global virq.
+ *  @virq:     Virtual IRQ number (VIRQ_*), must be global
+ */
+struct domain *get_global_virq_handler(uint32_t virq);
+
 /*
  * send_guest_pirq:
  *  @d:        Domain to which physical IRQ should be sent
diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 3771487a30..1876e0b26f 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -160,6 +160,11 @@ do_hypfs_op(
     unsigned long arg4);
 #endif
 
+extern long
+do_control_op(
+    unsigned int cmd,
+    XEN_GUEST_HANDLE_PARAM(void) arg);
+
 #ifdef CONFIG_COMPAT
 
 extern int
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 2ae26bc50e..2795906a8f 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -24,6 +24,7 @@
 #include <xen/vpci.h>
 #include <xen/wait.h>
 #include <public/xen.h>
+#include <public/control.h>
 #include <public/domctl.h>
 #include <public/sysctl.h>
 #include <public/vcpu.h>
@@ -731,6 +732,7 @@ void domain_resume(struct domain *d);
 int domain_soft_reset(struct domain *d, bool resuming);
 
 void domain_reset_states(void);
+int domain_get_dom_state_changed(struct xen_control_changed_domain *info);
 
 int vcpu_start_shutdown_deferral(struct vcpu *v);
 void vcpu_end_shutdown_deferral(struct vcpu *v);
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 214b5408b1..e3c9b27acc 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -17,6 +17,7 @@
 
 #include <xen/sched.h>
 #include <xsm/xsm.h>
+#include <public/control.h>
 #include <public/hvm/params.h>
 
 /* Cannot use BUILD_BUG_ON here because the expressions we check are not
@@ -157,6 +158,19 @@ static XSM_INLINE int xsm_sysctl(XSM_DEFAULT_ARG int cmd)
     return xsm_default_action(action, current->domain, NULL);
 }
 
+static XSM_INLINE int xsm_control_op(XSM_DEFAULT_ARG uint32_t cmd)
+{
+    XSM_ASSERT_ACTION(XSM_OTHER);
+    switch ( cmd )
+    {
+    case XEN_CONTROL_OP_get_version:
+    case XEN_CONTROL_OP_get_state_changed_domain:
+        return xsm_default_action(XSM_XS_PRIV, current->domain, NULL);
+    default:
+        return xsm_default_action(XSM_PRIV, current->domain, NULL);
+    }
+}
+
 static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG uint32_t clear)
 {
     XSM_ASSERT_ACTION(XSM_HOOK);
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 16b90be2c5..21965d3df9 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -125,6 +125,7 @@ struct xsm_operations {
 
     int (*page_offline)(uint32_t cmd);
     int (*hypfs_op)(void);
+    int (*control_op)(uint32_t cmd);
 
     long (*do_xsm_op) (XEN_GUEST_HANDLE_PARAM(void) op);
 #ifdef CONFIG_COMPAT
@@ -539,6 +540,11 @@ static inline int xsm_hypfs_op(xsm_default_t def)
     return xsm_ops->hypfs_op();
 }
 
+static inline int xsm_control_op(xsm_default_t def, uint32_t cmd)
+{
+    return xsm_ops->control_op(cmd);
+}
+
 static inline long xsm_do_xsm_op (XEN_GUEST_HANDLE_PARAM(void) op)
 {
     return xsm_ops->do_xsm_op(op);
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index de44b10130..0190463523 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -104,6 +104,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)
 
     set_to_dummy_if_null(ops, page_offline);
     set_to_dummy_if_null(ops, hypfs_op);
+    set_to_dummy_if_null(ops, control_op);
     set_to_dummy_if_null(ops, hvm_param);
     set_to_dummy_if_null(ops, hvm_control);
     set_to_dummy_if_null(ops, hvm_param_altp2mhvm);
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 1465db125a..5966e35c5c 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1181,6 +1181,11 @@ static inline int flask_hypfs_op(void)
     return domain_has_xen(current->domain, XEN__HYPFS_OP);
 }
 
+static inline int flask_control_op(void)
+{
+    return domain_has_xen(current->domain, XEN__CONTROL_OP);
+}
+
 static int flask_add_to_physmap(struct domain *d1, struct domain *d2)
 {
     return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP);
@@ -1815,6 +1820,7 @@ static struct xsm_operations flask_ops = {
 
     .page_offline = flask_page_offline,
     .hypfs_op = flask_hypfs_op,
+    .control_op = flask_control_op,
     .hvm_param = flask_hvm_param,
     .hvm_control = flask_hvm_param,
     .hvm_param_altp2mhvm = flask_hvm_param_altp2mhvm,
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index 6359c7fc87..3ca91ac23c 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -69,6 +69,8 @@ class xen
     cpupool_op
 # hypfs hypercall
     hypfs_op
+# control hypercall
+    control_op
 # XEN_SYSCTL_scheduler_op with XEN_DOMCTL_SCHEDOP_getinfo, XEN_SYSCTL_sched_id, XEN_DOMCTL_SCHEDOP_getvcpuinfo
     getscheduler
 # XEN_SYSCTL_scheduler_op with XEN_DOMCTL_SCHEDOP_putinfo, XEN_DOMCTL_SCHEDOP_putvcpuinfo
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 12:41:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 12:41:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186576.335329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7kU-0002e4-6U; Tue, 14 Sep 2021 12:41:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186576.335329; Tue, 14 Sep 2021 12:41:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ7kU-0002dx-3U; Tue, 14 Sep 2021 12:41:18 +0000
Received: by outflank-mailman (input) for mailman id 186576;
 Tue, 14 Sep 2021 12:41:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ7kT-0002dr-36
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 12:41:17 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6724807b-f095-49b9-8634-29d647b03a77;
 Tue, 14 Sep 2021 12:41:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6724807b-f095-49b9-8634-29d647b03a77
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631623275;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=SUrijeXNVLZ7C/wfJwbta0cDiWslSHEV8cFFg/6P7wI=;
  b=ZJv2Smd1IwQPuThOG/aqRkkKZhf4Mt9HruoTj0KeWceR5suTTWINQ4Rm
   piIM9nXlW2bzxUbo2vAfvnOm6Tlei3LMSDwfBuhh7moSRGSYRt6GKzywW
   1SZo1HbBAr4eDhKgTMOF3oLBB3+THPE+xBuSqTxth6Q3lBozdQtAmAr7G
   o=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: QTZxJnGufd4l0xrPjj0NRz41Wgfh6cBGcMdq2xC+ZU25WCKXrJNeW1Pxyh6IJPtkl4RBtDwU9F
 DT5bFgbBRvECF2RKOX989Ptm2pGfLQ9Fzewhp2Ox3x53Fp1COM8LepRFk5hd7EqwEgPNMbK/TN
 psgmWz/PCUwT4ME6u/afh4tBpZ/oJzNYQP7XAESOuR6iwx/umt46IGuv+zM1QhVTzX7Hj1A0PW
 giyRU6/S/W1K0OsaqaRkRMtKfFfiYR1KY9xX0nRkR6b88Y6ERPz/lnHH6vMqk2cYEFYmPlT7bg
 3klFld9A9om1RDN/WLho5+48
X-SBRS: 5.1
X-MesageID: 52661756
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:GZ/FLqoWHKaT7yZjuo5a4P5nLOleBmKPYhIvgKrLsJaIsI4StFCzt
 garIBmPM6mCZjDxfthxPo7n/RgG7ZTcnYMwTQBkrSs0EigS8ZuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0IHlW1rlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnbKLVxlqOvCUobQAcytlS31BJ6sBqaCSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFJkYtXx6iynQEN4tQIzZQrWM7thdtNs1rp0UQ6iHO
 JdHAdZpRE7RaiBsAkg4M79g2/2ulHavWiBjh03A8MLb5ECMlVcsgdABKuH9YceWTM9YmkKZo
 GPu/GnjBBwectuFxlKt9nOqm/+Kni7hXo8WPKO3++Qsg1CJwGEXThoMWjOTsfS/z0KzRd9bA
 0gV4TY167g/8lSxSdvwVAH+p2SL1jYeUddNF+wx6CmW17HZpQ2eAwAsTDFbb8c9nNQrXjFs3
 ViM9/vjAiZuq/uSUm6H8amPriKaPjIcJmsPIyQDSGM4D8LL+d9pyEiVF5A6TfDz3oad9SzML
 y6ighMgmfYX0JYyh7ibz22f3w22oN+YUVtgjunIZV5J/j+Vdab8OdfxtAmEsqgZRGqKZgLe5
 ylfwqBy+MhLVMvUxXLXGI3hCZn0v67tDdHKvbJ483DNHRyW8ni/dMh75DhkLS+F2e5VJGe0P
 Cc/Ve5XjaK/3UdGj4csOOpd6OxwlMAM8OgJsNiOMLKihbArKGe6ENlGPxL44owUuBFEfVsD1
 XKnnSCEVydy5UNPl2Heegvg+eVzmnBWKZ37HMimp/hY7VZuTCHMEupUWLd/Rss48LmFsG3oH
 yV3bpDRoyizpNbWO3GNmaZKdAhiBSFiWfje9pwGHsbec1EOMDxwVJfsLUYJJtUNc1J9zbyTo
 BlQmyZwlTLCuJEwAV7WMysyNOy2Bs8XQLBSFXVEAGtEEkMLOO6HxKwea4E2bf8g8ulixuRzV
 P4LZ4OLBfEnd9gN0211gUDVoNMweRK1qxiJOib5MjEzc4Q5H17C+8P+fxup/y4LV3Llucw7q
 rym9wXaXZtcGFgyUJeIMKqinwGroHwQuONuRE+UcNNdT1rhrdpxICvrg/5pf8xVcUffxiGX3
 hq9CAsDobWfuJc89dTE3PjWr4qgH+ZkMFBdGm3XseS/OSXApzLxyo5cSueYOzvaUTqsqqmlY
 OxUydD6MeEGwwkW49YtTe4zwPtntdX1prJcwgB1J1nxbgymWuF6P32L/chTrakRlLVXjhS7B
 xCU8d5ANLTXZM68SAwNJBAoZ/io3O0PnmWA9uw8JUj36XMl/LeDVkkObRCAhDYEcelwOYIhh
 +wgpNQX+0q0jR9zaoSKiSVd9mKtKH0cUvp46sFGUdGz0gd7mEtfZZH8CzPt5MDdYtpBBUAmP
 zuIifeQnL9b3EfDLyI+GHWlMTCxXnjSVMSmFGM/Gmk=
IronPort-HdrOrdr: A9a23:5Hwc6aB271b39C/lHejesseALOsnbusQ8zAXPh9KJyC9I/b2qy
 nxppgmPEfP+UossHFJo6HlBEDyewKiyXcV2/heAV7GZmOGhILGFvAb0WKP+UyDJ8S6zJ8h6U
 4CSdk+NDSTNykAsS+S2mDReLtBsbq6GeKT9J3jJhxWPGZXgtRbnn5E43GgYytLrWd9dP8EPa
 vZwvACiyureHwRYMj+LGICRfL/q9rCk4+jSQIaBjY8gTP+zg+A2frfKVy1zx0eWzRAzfMJ6m
 7eiTH04a2lrrWS1gLc7WnO9J5b8eGRiOerRfb8z/T9GA+czTpAV74RHYFqewpF+d1H3Wxa1O
 UkZS1QZ/ibpUmhJV1d6iGdpTUImAxemkMKj2XowUfLkIjBXzQ9BNNGhYVFNjXky2dIhqAg7I
 t7m1uDsZxZFBXBmzm4wePpeVVFqmqYyEBSy9L6qRRkINQjgXtq3NAiFQpuYec9NTO/54Y9HO
 Z0CsbAoP5QbFOBdnjc+nJi2dq2Qx0Ib1q7q2U5y4eoOgJt7TpEJoojtbsit2ZF8Ih4R4hP5u
 zCPKgtnLZSTtUOZaY4AOsaW8O4BmHEXBqJaQupUBvaPbBCP2iIp4/84b0z6u3vcJsUzIEqkJ
 CEVF9Dr2Y9d0/nFMXL1pxW9RLGRnm7QF3Wu41jzok8vqe5SKvgMCWFRlxrm8y8o+8HCsmeQP
 q3MII+OY6qEYIvI/cB4+TaYegnFZAzarxmhj8LYSP5niuQEPyYigXySoenGIbQ
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52661756"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TuztZ2yH4zUyAlzu1dNi2duolQlh4BUaatOU9bN6YYBcVUFhuguKN/quekcL+ASke0GN8LxLG3qYdOVmARMstGfAERTAxC5RtasTK+uz8A/6lYLAKats6fVCzO8VmAC62zeqbMWA3xoRsCd6U/xf4vHp0AHGUqLUUyB1L6nTHEyhAIcBifQnwtddAY08MvYcnETXQSj2/qPIv5lZH3jY8piG1jyVHRtPAjupLBrJVf1RvMcciPTQEn6TWpJe0L5lv2niJCmeYwYwPt+wtyRvSKDF5SC6T0wRbX6WIVgiRFZ9S/9IoXB7AdbZ1nA/e0qBf6c+MxYUBNxyp6pdvODeFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nmoya7Wy3/dlPb46qHqs9Pc/h7Q4JGS/lDNSIMdu+4o=;
 b=EYtE/PyPNi4PWDX7UgwMePtzJyealh66xG21V/8teMH6x8h7xzIGSfgvbbe9S9nwHckGvKbV70do5J0iaJxgzQLpIegQbVveAx1dpwKP/LWoe/30PHdMB646Lg+H8Va+gNjTr5Qc0wwSFZOscDJ9ml7fYStgks9z7K51YoNl4hMA5gWwtEN4uzH5jiCCAiWgL8on+s69+AdvsEtiWcAO9sWLuTTTmkXjsoXU+TZ+qxyIRfaMta84BADMxF243skAx8XRw5aLRIZaTiwKfvelwUl0mlmdjbwJk8PcYpPqd5rnQH8gjXvQNBHrbasDBxPkHxQhqGdcNIDjbxvjjrkrKw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nmoya7Wy3/dlPb46qHqs9Pc/h7Q4JGS/lDNSIMdu+4o=;
 b=Cl1LhmoKMRsT9um/OY0/AUTnv3c6JLfcRw1G9Z04+69GDqxS06YhriREr1Gy84AxutA+9+Ay3H5VRU1Nbmt1HAXds7Q+x6XFBovyqc29QYl7olJycPqSwSTcwCK6SzV+46wogFm6cxtQpzHXVZktp6zOSskIL3ZXYdI052yi4s8=
Date: Tue, 14 Sep 2021 14:41:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Juergen Gross <jgross@suse.com>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
Message-ID: <YUCYYmqQMR5NCTyz@MacBook-Air-de-Roger.local>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
 <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
 <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
X-ClientProxiedBy: LO2P265CA0144.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9f::36) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0b221a01-64f5-40ff-dec7-08d9777ceee4
X-MS-TrafficTypeDiagnostic: DM4PR03MB6000:
X-Microsoft-Antispam-PRVS: <DM4PR03MB60008D71C1A46F423871DE398FDA9@DM4PR03MB6000.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: ryzELLkwsBhjXq6/R5rMlgU5BHm+AlbdfzcyBiHz5DvG2ev35vLTpxy9o1QfKpNpwrCiH9BTMb97SMbJ7bGdJVlX+M9kksDMp2uBy0LYnD4bjO+XDSfhbTN9UNUFShBu3V9zsrnVvao8R+/vELO/uqZ57zddxRTAbvtiXEU1zQui6BhgjqFg1JayprBlzrhfu2nE3AJBm7vZ2dncI8eu9n9htz+NOf/g/vDwOOwpXpKLQzjnKEtT5R5j5jZ8Cce0kF7qJniVWq7IjroM3iRO9DEg3eXO4XzgO662B8vrX6fyJ6QmU57W7EHdf0b5c7Wqsm0Oe87aVdQlvTf4IYm1bOd2ozGwQT33Tb1+5D6/DHejpW7PI5bemROZ5xKsFRQORz94AKGc5jfxGvPdMmkeyLza1xx/xtcNOJ1M/GP16++vNH5Ss2Aibf5w9c2K/Jo4zAfbzGGV/ZpfKYKlCznvRCwuex0wLhku2IQGTHo17NlrXYCBa8J4DpsE8GlyNePP9oddDUJj1L2wjtc/FziOGoM6iuATd/7cOIfOCcGe1gCqYg38P7qlcwwpDLfw8Vxwu5e/zw215F6o8iiGQZO79Fn9rjVua0p10kKVvaeM6fJkck3iWUFeEDbBFEK8Zm0iS3IN7LQTWgtG4fKdMwbmM3FFTozQstPC5LzWqcWTr6591vO05xy8jZUV9iIWyjoVNKg+lC77rZCkOMk0l9UvmUgOEEUi1HbBkqXOXP80tBY=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(366004)(346002)(39860400002)(85182001)(6496006)(6666004)(5660300002)(45080400002)(478600001)(6486002)(86362001)(9686003)(38100700002)(26005)(66946007)(966005)(6916009)(4326008)(2906002)(53546011)(186003)(8676002)(54906003)(66556008)(66476007)(8936002)(83380400001)(956004)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b092cTdDR2pGR2NsbHpnMk5IdjR4RVRGUk1rdFhvSnNvSXJpQ0NBMzBrdE9u?=
 =?utf-8?B?Qk1naE9yUmF3UEdFN3QvdTlMMWNGTnRiVm5hTFBnNWl4UUM2SkV5UmNVcXZZ?=
 =?utf-8?B?OWg2dFd0V1VNbkZxRnhCcE9pK0tPeUF6ajBVY0FiRGczeE9vVnVrOG5GQVZr?=
 =?utf-8?B?aSsxVEJUYVVMY1FxUEdIeWZ4Vk5UZkxHTFFrbG84aFlSWkJEVzhCVFpjUUMy?=
 =?utf-8?B?Sm8vZDlSeER1WWwvZGE5MjVCUkowYTc3eVZ5RHVQbEF1aTNTd1ZQOWF4MWZz?=
 =?utf-8?B?eWdSeDYzQU9xcVlKSHM2NGhmZ1RRVDF5eGM2c2lidUloaHJ3cXYzRjFySHJX?=
 =?utf-8?B?bzNWejBmdHBLMWk4eFVDY1k2TFc2R3Y4Z2pCU2piZVd1NXlmRlpFaEtXNVkw?=
 =?utf-8?B?bndtazN3eStwVzdRZlE5U2RXRVh0R05wbkhKaHNHekd1Ym1rbnBpMGNSb294?=
 =?utf-8?B?RGxGOHFVMlp3WDZ4RnNIbUdaSXJHcjZwR1B1RWtPYkpLR0FWSUh2YUYxKzky?=
 =?utf-8?B?clVsWFBZQWUwQWMrSU9tRUNtYkFlWEpLZE0xakpJbmFibzRFS2pNNFMwQXlU?=
 =?utf-8?B?RCtOVHVLc1haSGhXL2NMdHpsbWxURXZ3aUdJQlNzMk1Ua0lDak9EUVNZZ0dt?=
 =?utf-8?B?S3h3eVpQSkFTNS9NcUF0WmprcFUyVk9vdTRhcXlXMTVkRXEySk9UbHBjT2VK?=
 =?utf-8?B?djBaVXJxa0MzbllBNjR2NGUwd2pLVWRsVDladWZOSFNDK3FpbXFSckhMQnRW?=
 =?utf-8?B?bGl0OFRMUW9FdmZ3elZZQzlQY1hqMjR0cVBOOVJFZGpGWm5zZFZlc2I3UXhy?=
 =?utf-8?B?V0k4Ui8xT0hwOGN0a1VtL29SVC93U0o4YUhDRmpiZTM5NHZYNFFIRVpsb2ZT?=
 =?utf-8?B?bGY2RmYzMlFxRFpWUjFTU3FEdWsvKzdIZDhhZkM2Rk9xZ1A0NlBKZG1FTGo3?=
 =?utf-8?B?OGg1NG1hZitLMFExWkxkTVRZbjFnekJEazRRK01WbElnYndjZkovaSs5YXZF?=
 =?utf-8?B?L08wY2NabTZEV1U0Z0ZzcS9CZmgyYm9teEcvVHlIaHVaL0pVVDVqS0VGelRT?=
 =?utf-8?B?YWVLaHZuWGRVWEtlSGh5WWtPWHdmSURPYlRJM3BuNVlIWnRwRlR6S21DQmI5?=
 =?utf-8?B?TG9XdW1tYXhLWVpibTVRVDhYMStJMFNhaGtMNkRDVHVKZm9ha1RMVmhMT1po?=
 =?utf-8?B?MGxUbEwzelNwelNCN3VwMVFwbzUxa3luUGw5cy82RE5CbnVidS9HUTVYNWRq?=
 =?utf-8?B?UmkwYnpzenkzYUVFbjJwdmM5UTIyejdhTnhUaXNqRU5RWlVFSklnTTRVZmVu?=
 =?utf-8?B?NUFraTRMenBoMFRjSUlJN296anRoL0w2WnhMSDF6NFFTV3E3K3M4K3hiY0I1?=
 =?utf-8?B?Tm9PYmg1aExFOGx1WW5Xc0UxbXM1Q08xUFZhbzV2TE92MXMwSGxGdkM5Z1pV?=
 =?utf-8?B?Unc5YlV0VW5GYXF4VGpqWnhuem4vS0RGWlFOeGwreGhkNmhmNWhCREVLMk1Y?=
 =?utf-8?B?bWdNemMzZjV4Z1pOVVZvaGlkcmkrN1ZoNm9meE5qMzlrbkU2N3lGUlJEbURi?=
 =?utf-8?B?Vk1wZnRoMjFJK0ZIc1hhZ1gwL1JlYUN6RFVVVm5rRURqQit3SjZwbjBpT0dP?=
 =?utf-8?B?NzRTKys1R1BMaVRHek5pZDNubnowV05sdzRaVksza0dBaFlmSFBBUC9ZOGF0?=
 =?utf-8?B?eGYxczlkVWZKZ1B5MzcvVTdZMExRQzgzWitjcUIrdmxVTWpxdmluajJXdVps?=
 =?utf-8?Q?haksXOl4/itSnkwJ4cvGvJvE1DeQ3bnD2N8pstf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0b221a01-64f5-40ff-dec7-08d9777ceee4
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 12:41:11.8372
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ODiZFPEydN3nmF4EwQ47Fn1ncR5FksUQH+HBV4LMgU+ySQcp9jji66WUdF8QAZQbS9J9G62bV982uAYhT2NaTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6000
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 01:58:29PM +0200, Jan Beulich wrote:
> On 14.09.2021 13:15, Roger Pau Monné wrote:
> > On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
> >> On 14.09.2021 10:32, Roger Pau Monné wrote:
> >>> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
> >>>> In order to try to debug hypervisor side breakage from XSA-378 I found
> >>>> myself urged to finally give PVH Dom0 a try. Sadly things didn't work
> >>>> quite as expected. In the course of investigating these issues I actually
> >>>> spotted one piece of PV Dom0 breakage as well, a fix for which is also
> >>>> included here.
> >>>>
> >>>> There are two immediate remaining issues (also mentioned in affected
> >>>> patches):
> >>>>
> >>>> 1) It is not clear to me how PCI device reporting is to work. PV Dom0
> >>>>    reports devices as they're discovered, including ones the hypervisor
> >>>>    may not have been able to discover itself (ones on segments other
> >>>>    than 0 or hotplugged ones). The respective hypercall, however, is
> >>>>    inaccessible to PVH Dom0. Depending on the answer to this, either
> >>>>    the hypervisor will need changing (to permit the call) or patch 2
> >>>>    here will need further refinement.
> >>>
> >>> I would rather prefer if we could limit the hypercall usage to only
> >>> report hotplugged segments to Xen. Then Xen would have to scan the
> >>> segment when reported and add any devices found.
> >>>
> >>> Such hypercall must be used before dom0 tries to access any device, as
> >>> otherwise the BARs won't be mapped in the second stage translation and
> >>> the traps for the MCFG area won't be setup either.
> >>
> >> This might work if hotplugging would only ever be of segments, and not
> >> of individual devices. Yet the latter is, I think, a common case (as
> >> far as hotplugging itself is "common").
> > 
> > Right, I agree to use hypercalls to report either hotplugged segments
> > or devices. However I would like to avoid mandating usage of the
> > hypercall for non-hotplug stuff, as then OSes not having hotplug
> > support don't really need to care about making use of those
> > hypercalls.
> > 
> >> Also don't forget about SR-IOV VFs - they would typically not be there
> >> when booting. They would materialize when the PF driver initializes
> >> the device. This is, I think, something that can be dealt with by
> >> intercepting writes to the SR-IOV capability.
> > 
> > My plan was to indeed trap SR-IOV capability accesses, see:
> > 
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fxen-devel%2F20180717094830.54806-1-roger.pau%40citrix.com%2F&amp;data=04%7C01%7Croger.pau%40citrix.com%7C35d2502d0128484e229e08d97777087f%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637672175399546062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=sSeE%2F4wEo5%2Fplkj2yH%2B1kpHi5c15lxJxeUxx6Cbyr4s%3D&amp;reserved=0
> > 
> > I just don't have time ATM to continue this work.
> > 
> >> But I wonder whether
> >> there might be other cases where devices become "visible" only while
> >> the Dom0 kernel is already running.
> > 
> > I would consider those kind of hotplug devices, and hence would
> > require the use of the hypercall in order to notify Xen about them.
> 
> So what does this mean for the one patch? Should drivers/xen/pci.c
> then be built for PVH (and then have logic added to filter boot
> time device discovery), or should I restrict this to be PV-only (and
> PVH would get some completely different logic added later)?

I think we can reuse the same hypercalls for PVH, and maybe the same
code in Linux. For PVH we just need to be careful to make the
hypercalls before attempting to access the BARs (or the PCI
configuration space for the device) since there won't be any traps
setup, and BARs won't be mapped on the p2m.

It might be easier for Linux to just report every device it finds to
Xen, like it's currently done for PV dom0, instead of filtering on
whether the device has been hotplugged.

> >>>> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
> >>>>    console) when in a non-default mode (i.e. not 80x25 text), as the
> >>>>    necessary information (in particular about VESA-bases LFB modes) is
> >>>>    not communicated. On the hypervisor side this looks like deliberate
> >>>>    behavior, but it is unclear to me what the intentions were towards
> >>>>    an alternative model. (X may be able to access the screen depending
> >>>>    on whether it has a suitable driver besides the presently unusable
> >>>>    /dev/fb<N> based one.)
> >>>
> >>> I had to admit most of my boxes are headless servers, albeit I have
> >>> one NUC I can use to test gfx stuff, so I don't really use gfx output
> >>> with Xen.
> >>>
> >>> As I understand such information is fetched from the BIOS and passed
> >>> into Xen, which should then hand it over to the dom0 kernel?
> >>
> >> That's how PV Dom0 learns of the information, yes. See
> >> fill_console_start_info(). (I'm in the process of eliminating the
> >> need for some of the "fetch from BIOS" in Xen right now, but that's
> >> not going to get us as far as being able to delete that code, no
> >> matter how much in particular Andrew would like that to happen.)
> >>
> >>> I guess the only way for Linux dom0 kernel to fetch that information
> >>> would be to emulate the BIOS or drop into realmode and issue the BIOS
> >>> calls?
> >>
> >> Native Linux gets this information passed from the boot loader, I think
> >> (except in the EFI case, as per below).
> >>
> >>> Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
> >>> info using the PV EFI interface?
> >>
> >> There it's EFI boot services functions which can be invoked before
> >> leaving boot services (in the native case). Aiui the PVH entry point
> >> lives logically past any EFI boot services interaction, and hence
> >> using them is not an option (if there was EFI firmware present in Dom0
> >> in the first place, which I consider difficult all by itself - this
> >> can't be the physical system's firmware, but I also don't see where
> >> virtual firmware would be taken from).
> >>
> >> There is no PV EFI interface to obtain video information. With the
> >> needed information getting passed via start_info, PV has no need for
> >> such, and I would be hesitant to add a fundamentally redundant
> >> interface for PVH. The more that the information needed isn't EFI-
> >> specific at all.
> > 
> > I think our only option is to expand the HVM start info information to
> > convey that data from Xen into dom0.
> 
> PHV doesn't use the ordinary start_info, does it?

No, it's HVM start info as described in:

xen/include/public/arch-x86/hvm/start_info.h

We have already extended it once to add a memory map, we could extend
it another time to add the video information.

Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:10:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:10:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186607.335340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8CW-0006FD-J4; Tue, 14 Sep 2021 13:10:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186607.335340; Tue, 14 Sep 2021 13:10:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8CW-0006F6-Ek; Tue, 14 Sep 2021 13:10:16 +0000
Received: by outflank-mailman (input) for mailman id 186607;
 Tue, 14 Sep 2021 13:10:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ8CV-0006Ew-FH; Tue, 14 Sep 2021 13:10:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ8CV-00060j-9f; Tue, 14 Sep 2021 13:10:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ8CU-0008LD-Q5; Tue, 14 Sep 2021 13:10:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQ8CU-0002tW-Pc; Tue, 14 Sep 2021 13:10:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iKO1e+nyRgvBT3w5PLS+9eznmsDHDr23IL/JjGnN9gg=; b=UI3pv/JuCgUcXfwvF3H7AYm0aA
	33GO44NpRFuhS/T0Mx3yNysSmMGMwdEviC5tjMJ47o4wRIQd2dOvNdjocA8EHfVOny7IkP4OsX2rV
	HVZt7TMUPciHNN7bRttFMDnuIsI7tRtfYmTPqWpHqp75ZLGe0hva1KJucHOZC/dJ60fA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164977-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164977: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d0ee23f9d78be5531c4b055ea424ed0b489dfe9b
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 13:10:14 +0000

flight 164977 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164977/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                d0ee23f9d78be5531c4b055ea424ed0b489dfe9b
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  409 days
Failing since        152366  2020-08-01 20:49:34 Z  408 days  711 attempts
Testing same since   164977  2021-09-14 03:20:39 Z    0 days    1 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255658 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:35:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:35:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186625.335365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8aD-0000nz-1a; Tue, 14 Sep 2021 13:34:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186625.335365; Tue, 14 Sep 2021 13:34:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8aC-0000nq-Ts; Tue, 14 Sep 2021 13:34:44 +0000
Received: by outflank-mailman (input) for mailman id 186625;
 Tue, 14 Sep 2021 13:30:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WCGQ=OE=fedoraproject.org=pjp@srs-us1.protection.inumbo.net>)
 id 1mQ8WI-0000TI-6n
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:30:42 +0000
Received: from sonic314-19.consmr.mail.sg3.yahoo.com (unknown [106.10.240.143])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9e780da9-ceb6-4470-862f-3368c9ce0b0f;
 Tue, 14 Sep 2021 13:30:39 +0000 (UTC)
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic314.consmr.mail.sg3.yahoo.com with HTTP; Tue, 14 Sep 2021 13:30:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e780da9-ceb6-4470-862f-3368c9ce0b0f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1631626237; bh=0Nh/Y6xPV4CIgLPlomK2v31ad896Cuh8tuUla8gXYJA=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=WBgabCv7VqlSscHaoyJAnNULqsBKn/hLYKbCkeq7f7M0Ro2NVT6j6bU3WY3ygqB06Ve9Uku7tsiXpbWAc3xY1d1qcY4yF3kPIInABh2G8+pHJU9yBkjVr2MbYqTczkDsen2ScfTnBZrdRS37XgINASlCWjFsUdG407WF5yndA0DAP2gBnyBs2+fBSt4AYj53F6rHNTOUmPb+ZskDBfRrGHM4ma6r3sgeP/GUie+zxHhEbVmSp+vqklCUlz8q0o8VcjwXi34O+RYPRzjfuW+HfxY+dDNMKOgrlqveZe+W5Q/LYe28YuojVOlQZ7kZx3ZJbS+mBoLclo0V1D9WKmnCdw==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1631626237; bh=SporCpWUHBPKtAiDhTZuOydwhJPEm0S5V10VKjyr79z=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=Lz/5jw/cd45EIRsRswLO/KZqCbn/IuavDVPIIGcItSAkXBxdu8Pn3DaohlECSAsZzOo+d1YH5NWdOe5c9Hnz35ZIFEaBWbbZGp1mvOV7Vz8u0zm0AXtNfit8pUH6z5eHhVF14C7FHz9RS2+Oq8eJmhORGEywo/GTCBqd9q9trl20pEp4iU/ZmEFDeVomG5yaGA0CqdzSrzztVCHaDs96dTgzEZDHDfzN5gnuq+YojIBMsT1IJ799Hqk2m3aog+5uRKT8W89pyQWMgKLuPSMdqkXPFotIv4deWoKbVfI5UxDlnyj3QAVVyzB3HYt2bbkkTlZ84o3E4LM9DM7UK4kQRw==
X-YMail-OSG: mLlyaboVM1mrhSXnTyTjnjQoxs4Qgg55vR6LHFZ5jDzt0tKCU1qo9a7OSSpJT11
 E1sWGG.iFXwp3Hqx86Tctpm0zpMK_I6APui2eXcNQ91gzCiye0xw85TVtAMqGADTDAxsivZqS6aR
 TCZftZs2SYB7M5zXd8YBj.7gh.kuiSPD.mVgB4.zODc7fSswwsEVTuqwyEMkG.bnKFkdzhzvSeZG
 HjGbmupCeAHoDCe4ByHQkE1jrn8xoaHVGOiivWp_7y.tHPkE56rpc8_y9spE10XrbMyF8WhrjpXO
 JaIfcZzgUyFlort1Lla_canu2n1Sgw5xT6MgWWwa_ZWb6J0NWM3oX.GJllIyxW9Mw4zo8_3GZV0S
 AUQ_i8tGaJXN6gnm6Qo4pP0ZWBXt9dbDvI0M7u5gQWRQ4XKSqu7TKPNPuL8l71ZFGhvb2flpWSTx
 pUuZ5r1oJtfH979NOMGk9FCSkjPi8NaHiTqGaF6uaIGN7ob4f8gm9maBTOi95VpFZGjaxzWmtxdJ
 McfGhFH5.yGbNtcDueQ8FVZfnmzgKyq2Vn549KZNzBS7YQNajAsiFj6acA1qDMSHDhpsf7MpbGIJ
 h.tNENxZtUIvpSyL6nAVu6AZZRPUQ5ull64ZJx.xPHBSFcCUoY_OxCfYNC85vWMgjGTBUJo.SKpF
 v7ZM70Z6O73AcoTH6jmqKQlbJa5SWPpeBjsS8whGyPJwssm0Tkn9hcs_sOhzHZYPPm_Sft0SgLpi
 0Bhl.csJDwu9uO6NbyGOM3TicHmEvrIjeg3ASEjGJFpEfv2M2wakU87Wan0XNJSsppNh4FirhvEO
 hUKxR4Bu2eveVBRI3a4qXF2QysBAWThaKdTLUkvaR4wtnoI9MUSU.EfUV2DFFK4USeOjrJYqJJz_
 MnvVTxPaU.qWlk0XHJrVa6pjEJCCP.gHpWerZUGGZfD5z_oRjg_vOgw7PeLDPm.8YElY_fysH5Lq
 NzYpA4ZW0FIYu1q617oWatc_cBhyZo4wJAR.YrcvlQfK00HT21ZSk.Kh1Y6tqa_w2TL4JVUkXweQ
 rPAbxIgS3QrhnoOdvYTswx.JmlibaMMPsBoXLYfA.wKr7JTdBXVQVA2Ep03Wq2OmxOT6aiW71WVm
 G72HL_3xIiNSRLhI7Z0jLvn0.eSuJrpcFC_NoJ8BExTHg4tOwn2E64CdW9EhFLg6M6mmUPPkzV2E
 yOsp8gJ.4wMXxC.dSTa8m4NzLgknCMSrtn0T4bTb897riI.mUX7ZdKzn.LrRTGGR0G2CYdTvnFxt
 3yAZsT0ObwJhhWfA_6guW28QLQ00CmSrj8mOtewkXTHHjmEfUeFUGsfzN2DsA91biSLUNMrr2Vwb
 kJkBctRUhkdUQzl1h31i3.Q1zIMdK8R4IdJICpxZpTsAEZBP6PEzYCTGO7RleDo7uUfYwLGYZdBD
 Ue2tTnmH2kA2pdPwx6LlEqpthATG2FMZFSB.Y5tI_yNQ8FxL4tYrvcyglMmSJSyVERx8znF58tUb
 510s67KW3yODK4aZKMqtrl4fpo.MxQQKiTrR_KW5W4gjuS2bM.W5w0u5BTxHanyCkZAU6DkKtyks
 QjF7y6X4YtD_uDBHD1mM3f.N9kZUInSlPlFnVZKSowSQtFpLkklzf4SthWjDXtTkOpY9OyIX_Rui
 5XrNHhuNTQlHWTuit2H7ZbJwQxC_2PeLFFbVhEmuSPMczjHZjkiPuy2NKlJ7oun6oWS7cXxOcB8Z
 OzHQF5GdRSLSsinzsZi3Oel2Ox7xYnJN1goI82Bz_jKoeBMjTD1yFgU1RyRbkn.zEd2ezm8zv5Bk
 fMhk6OyFyqWMmcN51m5tKHdrAn56UiOnpYoM2garlCkv9NSNhMgw_Lh58gkHk4WMBvupBmGItvP0
 BJLXmnwbcjHe5TqovDVP_eFzDCG31EvpKUBywg1WoMAOuJEO_JBjcuhsjj0DaHwsTLM4a39Fb8gg
 2q.5PEDUsDU_anowV42IQ6KhqtZ0N8VNbqw7xqwWFF01tcxN5Psy356obfXp7ghIl2KQpPZWf8MV
 KJBWSHE_n2Eqop.HfxCzzj7NSwVPag6bWcLMcuSyR3j8AjTEte2iseamCZCKnG41DSveXpC7XkSH
 kITanbdJBIITXhVNWLKTERKJS9WSPAjHtytE2kfQCIULoRWaY79KV0EQ7F2Mo0jC9tRifR3RJgi6
 b3qZwHAUKaDG1ryKDlPlMQmLG4GBPwBnpBrIItklixg0lFr1WjhtU5UInVL2piBEr2aB_k8xYlEQ
 mSWnZK11AOB9bkXojOwxF3AyVOaoRl60DWzmAQLNewn6STqDFa.PsugMJXuQyKRwyMKHGuTngFFG
 H9pOC5eGHZN0fBedKgh37FQoLRWya4OQPBTw7t2L4qk7lTzBi4lAYJomQGW1tZ0blrftW1D8trEM
 14Fo-
X-Sonic-MF: <pjp@fedoraproject.org>
Date: Tue, 14 Sep 2021 13:30:27 +0000 (UTC)
From: P J P <pjp@fedoraproject.org>
Reply-To: P J P <pj.pandit@yahoo.co.in>
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@redhat.com>, 
	=?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= <berrange@redhat.com>
Cc: "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, 
	Thomas Huth <thuth@redhat.com>, 
	"Michael S. Tsirkin" <mst@redhat.com>, 
	Markus Armbruster <armbru@redhat.com>, 
	Paolo Bonzini <pbonzini@redhat.com>, 
	Eduardo Habkost <ehabkost@redhat.com>, 
	=?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <f4bug@amsat.org>, 
	Eric Blake <eblake@redhat.com>, 
	Richard Henderson <richard.henderson@linaro.org>, 
	"qemu-block@nongnu.org" <qemu-block@nongnu.org>, 
	Peter Maydell <peter.maydell@linaro.org>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <798304472.4432617.1631626227208@mail.yahoo.com>
In-Reply-To: <YTnhy/Wo5M1qVs1k@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com> <YTnhy/Wo5M1qVs1k@redhat.com>
Subject: Re: [RFC PATCH 00/10] security: Introduce
 qemu_security_policy_taint() API
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Mailer: WebService/1.1.18924 YMailNorrin
Content-Length: 4762

Hello Philippe, all

>On Thursday, 9 September, 2021, 03:58:40 pm IST, Daniel P. Berrang=C3=A9 <=
berrange@redhat.com> wrote:
>On Thu, Sep 09, 2021 at 01:20:14AM +0200, Philippe Mathieu-Daud=C3=A9 wrot=
e:
>> This series is experimental! The goal is to better limit the
>> boundary of what code is considerated security critical, and
>> what is less critical (but still important!).
>>
>> This approach was quickly discussed few months ago with Markus
>> then Daniel. Instead of classifying the code on a file path
>> basis (see [1]), we insert (runtime) hints into the code
>> (which survive code movement). Offending unsafe code can
>> taint the global security policy. By default this policy
>> is 'none': the current behavior. It can be changed on the
>> command line to 'warn' to display warnings, and to 'strict'
>> to prohibit QEMU running with a tainted policy.
>

* Thanks so much for restarting this thread. I've been at it intermittently=
 last few
=C2=A0 months, thinking about how could we annotate the source/module objec=
ts.

=C2=A0 =C2=A0-> [*] https://lists.gnu.org/archive/html/qemu-devel/2020-07/m=
sg04642.html

* Last time we discussed about having both compile and run time options for=
 users
=C2=A0 to be able to select the qualified objects/backends/devices as desir=
ed.

* To confirm: How/Where is the security policy defined? Is it device/module=
 specific OR QEMU project wide?

> IOW, the reporting via QAPI introspetion is much more important
> for libvirt and mgmt apps, than any custom cli arg / printfs
> at the QEMU level.
>

* True, while it makes sense to have a solution that is conversant with
=C2=A0 the management/libvirt layers, It'll be great if we could address qe=
mu/cli
=C2=A0 other use cases too.

>it feels like we need
>=C2=A0 'secure': 'bool'

* Though we started the (above [*]) discussion with '--security' option in =
mind,
=C2=A0 I wonder if 'secure' annotation is much specific. And if we could wi=
den its scope.
--- x ---


Source annotations: I've been thinking over following approaches
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

1) Segregate the QEMU sources under

=C2=A0 =C2=A0 =C2=A0 ../staging/ =C2=A0 =C2=A0 =C2=A0<=3D devel/experimenta=
l, not for production usage
=C2=A0 =C2=A0 =C2=A0 ../src/ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<=3D good fo=
r production usage, hence security relevant
=C2=A0 =C2=A0 =C2=A0 ../deprecated/ =C2=A0 <=3D Bad for production usage, n=
ot security relevant=C2=A0

=C2=A0 =C2=A0- This is similar to Linux staging drivers' tree.
=C2=A0 =C2=A0- Staging drivers are not considered for production usage and =
hence CVE allocation.
=C2=A0 =C2=A0- At build time by default we only build sources under ../src/=
 tree.
=C2=A0 =C2=A0- But we can still have options to build /staging/ and /deprec=
ated/ trees. =C2=A0
=C2=A0 =C2=A0- It's readily understandable to end users.

2) pkgconfig(1) way:
=C2=A0 =C2=A0- If we could define per device/backend a configuration (.pc) =
file which is then used
=C2=A0 =C2=A0 =C2=A0at build/run time to decide which sources are suitable =
for the build or usage.

=C2=A0 =C2=A0- I'm trying to experiment with this.

3) We annotate QEMU devices/backends/modules with macros which define its s=
tatus.
=C2=A0 =C2=A0It can then be used at build/run times to decide if it's suita=
ble for usage.
=C2=A0 =C2=A0For ex:

=C2=A0 =C2=A0 $ cat annotsrc.h

=C2=A0 =C2=A0 #include <inttypes.h>

=C2=A0 =C2=A0 enum SRCSTATUS {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 DEVEL =3D 0,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 STAGING,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 PRODUCTION,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 DEPRECATED
=C2=A0 =C2=A0 };

=C2=A0 =C2=A0 uint8_t get_src_status(void);
=C2=A0 =C2=A0 =3D=3D=3D

=C2=A0 =C2=A0 $ cat libx.c

=C2=A0 =C2=A0 #include <annotsrc.h>

=C2=A0 =C2=A0 #define SRC_STATUS DEPRECATED

=C2=A0 =C2=A0 uint8_t
=C2=A0 =C2=A0 get_src_status(void)
=C2=A0 =C2=A0 {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return SRC_STATUS;
=C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =3D=3D=3D

=C2=A0 =C2=A0 $ cat testlibx.c

=C2=A0 =C2=A0 #include <stdio.h>
=C2=A0 =C2=A0 #include <annotsrc.h>

=C2=A0 =C2=A0 int
=C2=A0 =C2=A0 main (int argc, char *argv[])
=C2=A0 =C2=A0 {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 printf("LibX status: %d\n", get_src_status());
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
=C2=A0 =C2=A0 }
--- x ---



* Approach 3) above is similar to the _security_policy_taint() API,
=C2=A0 but works at the source/object file level, rather than specific 'str=
uct type' field.
=C2=A0=C2=A0

* Does adding a field to struct type (ex. DeviceClass) scale to all objects=
/modules/backends etc?
=C2=A0 Does it have any limitations to include/cover other sources/objects?


* I'd really appreciate any feedback/inputs/suggestions you may have.


Thank you.
---
=C2=A0 -P J P
http://feedmug.com


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:35:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:35:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186627.335354 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8a8-0000XT-O5; Tue, 14 Sep 2021 13:34:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186627.335354; Tue, 14 Sep 2021 13:34:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8a8-0000XM-L0; Tue, 14 Sep 2021 13:34:40 +0000
Received: by outflank-mailman (input) for mailman id 186627;
 Tue, 14 Sep 2021 13:34:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ8a8-0000XG-AX
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:34:40 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81f80df7-1560-11ec-b430-12813bfff9fa;
 Tue, 14 Sep 2021 13:34:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81f80df7-1560-11ec-b430-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631626478;
  h=from:to:cc:subject:date:message-id:
   content-transfer-encoding:mime-version;
  bh=i4EvmQcjVvidWdZBfCt/WMR/N1cBEj5zAawqggHS+J0=;
  b=Jhb6jAJ5J0lxRyed8BK77EOvn/iF+/i0QqOa0P1DpVz6VAlBy9yMcB8k
   yzaM+S0butvA/enom7G/2sUxicvNjKw3KIW0hnR0eBPuDJhZyiVvA/LJh
   5KkYvOz3lnyPs2OIZ+YhWeS/j6yanM8tB71fp4TarLdHsoE6NfwO+aTHD
   Q=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 1OfiyB8GONqkJmMwpEGY+dk+vgcH+h4eAV0YqFH4dqJLFwFmQCP6DZ1veTQ0b63r1uHR4K8IbM
 gEE9saq58E3IMhheYd/SND9NKoZxoLDh9/JMTOxbTC/O+Zj3lVDE18xgrVAjLAyYhY+bXMCX+l
 dFreqe/E5GXc1OU/Of+eJCokVGrheh/1uyrywE7qlJvttdC7gvYKZ/IiJo/OtkenI9fYog1eNi
 UM45HDSMS2gU6Tae1fzjvt9VGMUXoKS3z2GL/p+oIcqrzHhwZz0iwFA9oOFISwPrYoVY/QzOnJ
 W1Gpy4EVZq9CboJgBK4wS7j6
X-SBRS: 5.1
X-MesageID: 52668333
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:mUsWoa0T+6k6RZTvJfbD5e12kn2cJEfYwER7XKvMYLTBsI5bp2ZWn
 GdNDGyGPayDZDCneYgkbY2xoRsEv8WByt4wSAI5pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrJh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhhtlBx
 vVU6qaJTyQ3HaHOmugHaiZ5OnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIUGgmdr3Zsm8fD2R
 vFIVyFMfgz8bz5ePxQWKK8lk8z4ryyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1XlEuFo
 mPN/0ziHwoXcteYzFKt4n+qw+PCgy7/cIYTD6GjsO5nhkWJwW4eAwFQUkG0ycRVkWbnBYgZc
 RZNvHNz8+5iryRHU+URQTWhqU6KngdDYeNWMPMZ5AOPxrDt3CewUz1soiF6VDA2iCMnbWV0j
 QbUzoy2W2UHXK69Ei3Gp+zNxd+mEW1MdzZTO3VcJecQy4S7+OkOYgTzosGP+UJfpub8Hy35i
 xuOpTI371n4pZ9Wj/jnlbwrbjTFm3QocuLXzl6MNo5GxlkgDGJAW2BOwQKFhRqnBNzFJmRtR
 FBex6CjABkmVPlhbhBhpdnh+pnztp5p1xWH2jZS82QJrWzxqxZPg6gJuGwWyLhV3jYsJmayP
 R67VfJ5z55PJnq6BZKbkKroUJ9C8EQULvy8Dqq8RoMXOvBZLVbblAkzNR/49z28yyAEzPBgU
 ap3hO7xVB72/4w8l2HoLwrcuJd2rh0DKZT7HsyilEX7juPFPRZ4i94taTOzUwzw14vdyC39+
 NdDLcqajRJZVez1eC7M9oAPa1sNKBAG6Vre9aS7r8aPfVhrHn8PEfjUze9zcoBphf0NxOzJ4
 mu8SglTz1+m3S/LLgCDa3ZCbrLzXMkg8SJnbHJ0ZVv4iWI+ZYuP7bsEc8dld7cQ6+E+n+V/S
 OMIepvcD60XGCjH4TkUcbL0sJdmKEawnQuLMif8OGo/cpdsShbn4NjhegezpiACAjDu7Zk1o
 qG61xOdSp0GHlwwAMHTYfOp7lWwoXlCx74iAxqWeoFeIRy+/pJrJir9iu4MD/sNcRiTlCGH0
 wu2AAsDobWfqYEC79SU17uPqJ2kErUiExMCTXXb97u/KQLT4nGnnd1bSO+NcD3QCDH09aGla
 bkHxv3wKqRazlNDso46GLd316MuodDoouYCnAhjGXzKaXWtC69hfSbajZUe6PUVy+8LoxayV
 2KO5sJeaOeAN87SGVIMIBYoM7aY3vYOlziOtfk4LS0WPsOsEGZrhamKAySxtQ==
IronPort-HdrOrdr: A9a23:lWtUdaw9XVadDQp8yAwdKrPxyeskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjlfZq6z+8O3WBxB8bYYOCCggWVxe5ZnO3fKlHbak/DH41mpN
 hdmspFeaTN5DFB5K6QimnIcKdf/DDuytHVuQ609QYLcegFUdAE0+8vYTzrb3GeCTM2TqYRJd
 653I5qtjCgcXMYYoCSAWQEZfHKo5numIj9aRALKhY74E3W5AnYpYLSIly95FMzQjlPybAt/S
 zslBH43Lyqt7WexgXH32HewpxKkJ/Ky8dFBuaLls8JQw+cwDqAVcBEYfmvrTo1qOag5BIDl8
 TNmQ4pO4BJ53bYbgiO0G3Q8jil9Axrx27pyFeej3emi9f+XigGB81Igp8cWgfF6mI71esMnp
 5j7ia8jd56HBnAlCPy65zjTBdxjHe5pnIkjKo6k2Ffa40Dc7VcxLZvs3+9KK1wXB4S1bpXU9
 WHVKrnlbRrmBKhHjzkV1BUsZ2Rti9ZJGbCfqAA0vblpAS+0koJi3fxaaQk7wU9HakGOul5Dt
 L/Q9BVfYF1P74rhJ1GdZQ8qLOMexTwqDL3QSuvyAfcZek600ykke+D3Fxy3pDsRKA1
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52668333"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jFMCRB1kOh7dLOBtL31Nh+L8vojJ/48uwhdSfjfCaCyyu8pQWVE++L/4GC8ojEoNLrAq1I76djgKfFjAGHHOGj8GVS5NXpRKFogif1gZcKjghRoHSa0Jhze+lc9TZm+Qh1fZay9sC02PM59g66sAPJLi/tMleLsMEX8eAfPB9VvMhoBqBLWnbmb/cTe60QbYKdpo6zqf36bFbe/PR7zEpMbO9bxx99y1z3mv4IDe2JFs84hOeIJP/M7vh0rPDNQnu92iRFBc7vDRkMc5yDpeSjRkr6gGBK6pYzNdSAC871bzZE2kMj0h44/fHrQE92sEV1jdzSOETRbZN3cNP407mg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EgUxGtlAMtrN3bVygrNwcyHj5WdASjCPNTGzebnP1Rs=;
 b=daDBGq2XJrnx2dkFXFamnQJjPFbH5Tu7X0vN7+Sk2xqUcSqqSdPNClKK+xMDFxh0XywyjtipvND6qPdHsybeQn4xi0HnFrZL1fkmFYE84v5hc3+5NuYVZRmcwRsJx5ZXhJqPZMaE3aa0ahjjor0Qh14rv2kavIkghM8JR2ZmE6RfSZHtmG75EUKMKfjGzjmZfTlkVx+ov84dJPpR9nxVKxAhvr49giDF4S5vXmovIQB/lOhZPF3R21HiH2bosYR1+3V2hLemTwfdLLbnhwiWcOFFvPGyBWAqbGEDqPTpScZx7DXgPPKLlxGtuxp982Smq85d3vGBbnQLO1oCk4K/0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EgUxGtlAMtrN3bVygrNwcyHj5WdASjCPNTGzebnP1Rs=;
 b=QOvkMEUicwiq1R7kq3i0ilVOws+NuTc2EOpJcYXtDyOxAFsU6EDDK1f89xu2qXfzPp9Rk82q0zrwUUTcUipBtX2iHZlLzJb/49Mi2a+H9cICsGtxCgybjqJKw0FC57mzrg3mR7RuBF3gdPMxOHGY9qgut6ngvm80OUAgBkf2j/E=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH] x86/p2m: fix xenmem_add_to_physmap_one double page removal
Date: Tue, 14 Sep 2021 15:34:19 +0200
Message-ID: <20210914133419.8407-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0176.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: da482aef-e323-41a1-7ed5-08d9778460ce
X-MS-TrafficTypeDiagnostic: DM5PR03MB3212:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB3212F3A985DF13E3B4D9CEC48FDA9@DM5PR03MB3212.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1122;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 04QVqnRBpZAoqMcFMeEnhCgw/jrnJErbISECN3iYJWQEUPKpWgdJQKMce8SV95ibYoptCNJMlg1cdBWPaL/cyrtRzBm7zgk7rxSlVGr7wLdzpoFSh1eWdCfNMOmAOLW5KLFqQr8zuWFfE2/5a0VA90WJfK1qDyrwRC9Vfb9j/n5HgN7oOjGq8YQ9CcBPB+cbqH8R/MYMDURvaxOHVEOlUI61BWF7slaYoFtwUi2T7BWsvSI8rLRrCQb7OmcB2avu02q15fSuSZ8BLJLUCQ5FQd5IPZY/I8ygTd9wibHc2wKpup+nOtMY5+eyttCZIM12fCtEFkdEkve0LpUlHIdkn9Eq+lm4Ft1WYkuYZ63mxjE5tSYvcbjHm74H8Y+cy/hA9zH+7YuGk7w8ckevoN0tOczFUeIjn1CKm2OzX2o6PUN0cC3+TIIsm22o2eB1sQigfhV3pQQNEks09dPWojg/vfeVyfkijZVToXUcQfVac8WDGufsgOvfJIlD5DrZ9NpiMDumr4pWMFYPkGpdKSxjnBNYB+clQ44wIV4KXcWKiwcuM6lLmj8Ar/h0rgZA/vHHOdkvK8bZhVp19nUxjj0r2wrU7ARp3sUM8v+MrOAcg9a/vxWNZssh0NCY3NZFRPONcy8280z+ouC356zr3Gmd+Q==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(346002)(396003)(366004)(136003)(26005)(54906003)(5660300002)(86362001)(83380400001)(6486002)(186003)(316002)(6916009)(6666004)(1076003)(38100700002)(66476007)(6496006)(8936002)(2906002)(8676002)(478600001)(4326008)(66946007)(2616005)(956004)(66556008)(36756003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RU51a2dZWWhFek00VVhCdmVQTjFyMS8vQThUa1RibEkrTUxTUkI1STNPS1Nj?=
 =?utf-8?B?SGVIWXQvbllVYjdvTGh6dVpIRlF6dlVyNk1JdTVNSlFROGJTNjFtc2hWQ0Rs?=
 =?utf-8?B?RVkxSm01dGw1ZGhVZS8vRlA5OFF3MUlwc2VaWWttTXgvM2o4dzQ3eGNVU1Fp?=
 =?utf-8?B?dXpJZXo2N2hCc25Bdld4eEJDUTdJSmxUOTU5U01oQ1pBWERQelZIZk51ZGNF?=
 =?utf-8?B?UGY4ZXdrS1VnTmVZQkVjR01VcW90WHNSOEZBQy92UmVVSFpmYlF1RWZxc3A0?=
 =?utf-8?B?eUlrZ3N4cWdUY2RmdmFhOThjYndYTmJ4UDdueXdaTDJRVHhZK1kyNTREdkRZ?=
 =?utf-8?B?SXFpbzE2clRsSGl1cHJMMGNMMCtLMmRTWEE1VFkwdXk4dXNnL2dlVjcrS1dS?=
 =?utf-8?B?OVNvczFlWXZKZjgrR2taWkxUWnJRdGsxa3lROXlMdk9TSEQwbUtDRGhBYnFD?=
 =?utf-8?B?a1I0Ym10bk02MDBhYmZob1J2ME5ONCs3WTBLZXBMZjdySXBhS0YwWHJjT253?=
 =?utf-8?B?SGZiSm8vWEs2aDJEbzVXK0o4Rm5TejdseGxTblpYcnRaMWJiM0NQVXowQlIx?=
 =?utf-8?B?K1c0UnhETlh3T1lmOFVSNHBpdERpZnU0WlZYRUdia1lHR3hTUm1qdWRaR3NW?=
 =?utf-8?B?Tm1sdVJ1cTVINHhuS2w0MU02a1l6cXpUZThSNzFJZzVQOHp1dCtLM2FXTnpw?=
 =?utf-8?B?aXZ3alhmdzRqK3ZucVQ2YmZnSE4zcUZrV20ycGRqeXg2eE9DV3V1UEh4TDlx?=
 =?utf-8?B?TTBQaXYxSEVVYkdrTlVvYlp6eHVvbGxFMTU3Wm9ZeUJLaENBSCs3YmplNDRs?=
 =?utf-8?B?MEswS0VmbXIvRm8yS01IUU1jVC9DRFpGVjNTMEtPSFpwWVBheWM3a0ljYmdh?=
 =?utf-8?B?Nm1FQitFR2VBZEMxQkk2cUZHRyt2Y0lKbzJNWFVTRmd2bGJhbFM3dFRNZVhv?=
 =?utf-8?B?c1FMTG1lNUFxcy9lbDBXTXUvajM2S0Y2dnZEK01kdHo4K3gvV2oxK25jZmgr?=
 =?utf-8?B?UW9GWTFpbXBlYTFKTlU3K0IyL1lXUjFmcXo0YjFPWDl5QUJHRWt6UkpuQlFu?=
 =?utf-8?B?U21HNWFnNE1kNUwyQnBOMUxoK2VuQVZzVlRPVkFBTnVRc0tqa0QvM3Nvem1w?=
 =?utf-8?B?Tkl5RDFIR21ZVC84TVNJR2hMRS9LMFFiaVJzWm9Pd2xnaGk3NVBYMUZuM0J5?=
 =?utf-8?B?RkVlSWJRTGp4aWdpcXF0bmcwVDVNV3FLVG9YUmtiVWM2K2Z3RkE3SkdMdHp5?=
 =?utf-8?B?ZG10dm83SEVuSklvVVhtUkdXcWRjeDJjd2xueGZDUll2S0kyMUVxVHpqWk1B?=
 =?utf-8?B?bGtFRUZyQU5oSmF2MDJOUEMwdUh3U2EyZ0J0YVgvbER2V29tTERQMTZNUmdT?=
 =?utf-8?B?SjFVOHNzZ043YnJRdkhDV3Z6NUNSc3k0cU9ONCtjcHcrNjVreTE5TGtaa0VV?=
 =?utf-8?B?UnoxYUxkOFphTGRaV2Y1Nyttbk9SSG5FLzVJZCtQaXcxckZHaWdwbzhPZ3dN?=
 =?utf-8?B?ZFRqUHJwQndnNFN1VWF2SzhsVGh6aDhTZFZUcEtTMGJ0Vm5MYURQb0VvazlI?=
 =?utf-8?B?aHVNanZmcmJBY0Nld2c4MWRCMlFrc29rWEhpd1BETzFUTkFFeXlzUjlsVVph?=
 =?utf-8?B?ekdhb3JjVE1QOVRJNERqbXFtR2VjUjNyQUNFQ3JFWEFYWXZYaDN3TndLMXJx?=
 =?utf-8?B?YWZ1TWxlS2RsaXNhSEJmSCtTL3oxQnNzOVJ4RldYK0Y3L09qbndlN0VpdDI5?=
 =?utf-8?Q?nsPLT/cafVuBjtkC0zqFh3DsO4ffuRjWMTpZIV0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: da482aef-e323-41a1-7ed5-08d9778460ce
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 13:34:29.5290
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Ulpw85xBQ1+04plxbYgb75y/6751vwH5XpYiDyPf8OHesSwkUR/RjW7XVE5Dn8MV/ztiAOz5JuWnOBCN4sVqkg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3212
X-OriginatorOrg: citrix.com

If the new gfn matches the previous one (ie: gfn == old_gpfn)
xenmem_add_to_physmap_one will issue a duplicated call to
guest_physmap_remove_page with the same gfn, because the
get_gpfn_from_mfn call has been moved by commit f8582da041 to be
performed before the original page is removed. This leads to the
second guest_physmap_remove_page failing, which was not the case
before commit f8582da041.

Fix this by adding a check that prevents a second call to
guest_physmap_remove_page if the previous one has already removed the
backing page from that gfn.

Fixes: f8582da041 ('x86/mm: pull a sanity check earlier in xenmem_add_to_physmap_one()')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/mm/p2m.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 674a6f4fe9..2bd4d37286 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2813,7 +2813,7 @@ int xenmem_add_to_physmap_one(
     }
 
     /* Unmap from old location, if any. */
-    if ( !rc && old_gpfn != INVALID_M2P_ENTRY )
+    if ( !rc && old_gpfn != INVALID_M2P_ENTRY && !gfn_eq(_gfn(old_gpfn), gpfn) )
         rc = guest_physmap_remove_page(d, _gfn(old_gpfn), mfn, PAGE_ORDER_4K);
 
     /* Map at new location. */
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:39:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:39:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186647.335375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ek-0001xC-Ki; Tue, 14 Sep 2021 13:39:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186647.335375; Tue, 14 Sep 2021 13:39:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ek-0001x5-HZ; Tue, 14 Sep 2021 13:39:26 +0000
Received: by outflank-mailman (input) for mailman id 186647;
 Tue, 14 Sep 2021 13:39:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8ei-0001wz-Ul
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:25 +0000
Received: from mail-pg1-x52b.google.com (unknown [2607:f8b0:4864:20::52b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9f49761c-f8e8-4b81-ab60-b9240809ca7a;
 Tue, 14 Sep 2021 13:39:23 +0000 (UTC)
Received: by mail-pg1-x52b.google.com with SMTP id h3so12754319pgb.7
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:23 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f49761c-f8e8-4b81-ab60-b9240809ca7a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=BHnwRGLPqH2A09ZXeDyMyxAcljYFX6n5VugCn85EoaM=;
        b=pjWLs+F/UNZaQ69Ds31Zi6HGFXPIFegbFqnIPgAg/e7eumR7muKpnjQsK4h4ATjgb2
         by7YiKtMcGNpi+ZXill2CZ9odlQEVqCHILMVlph6YQFKC8hHwp1QpE3r+RuSLxydvPYZ
         eGAqVhTH+xBk/V/LWNCB2iXK6oGjpeCm6U1nP3C+AFVZg+TFR+itqQRDs0wgk7bN3jJj
         IMVP9rnCIplN4weSfeBPFdD9eNEmq4L1JXa8SxhKOq/LaXmZsBWdbgaymudmjb7zfL2E
         DKrQ17H/rltwz4VcgDwdq+noncaSP4fVO9efd8m4H7efVZrJSaN580PDrm6EaBFxrg5t
         ohzA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=BHnwRGLPqH2A09ZXeDyMyxAcljYFX6n5VugCn85EoaM=;
        b=oUXUjJGkcW+3hI4FuXCdcUhk3ohvLpYzrlzaqnd/YxCQ+KH27bPuvNIUDtO1i99A2J
         UwD9jcj7vxvTsVlkzYOerhZc9OEIcJ3Lk6qgxzOV2vVYZx++3qglrbH/Lrbwp/dYBLWb
         jaM7CB4FmJWqaHONv3K9z28BW7jT4ZG8+NoVApOgCwTldyj+DUkUSE1r6lMTY6Uk2e1g
         /lvma+eXE7Q9dwKVn34jwHV+ImGx+3tBXISOk8nBNaRZGe1d6YTBqMDuVj7jIPA5Ttv9
         04jab08KPNeO+Mo4SC4IfLQRB03z8ZpBWUHkFYTyzKwj2oUrOHhkwWptqG7BSz8ou9Nu
         PzlQ==
X-Gm-Message-State: AOAM533xOdssshy+0AW3kji325tKvmBpfJOuZ+1P9yiDpZM6Jz6/zCfE
	lZJ64XV1nLe//lqrH1732dQ=
X-Google-Smtp-Source: ABdhPJysSi/ePEIpmop1TOOZL/EDUeVswpP9kBW06aQK1cw/R5aOGjFREOrMllhH8ENnzQcpU6ys4A==
X-Received: by 2002:a63:3d4a:: with SMTP id k71mr15619461pga.276.1631626762730;
        Tue, 14 Sep 2021 06:39:22 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 00/12] x86/Hyper-V: Add Hyper-V Isolation VM support
Date: Tue, 14 Sep 2021 09:39:01 -0400
Message-Id: <20210914133916.1440931-1-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Hyper-V provides two kinds of Isolation VMs. VBS(Virtualization-based
security) and AMD SEV-SNP unenlightened Isolation VMs. This patchset
is to add support for these Isolation VM support in Linux.

The memory of these vms are encrypted and host can't access guest
memory directly. Hyper-V provides new host visibility hvcall and
the guest needs to call new hvcall to mark memory visible to host
before sharing memory with host. For security, all network/storage
stack memory should not be shared with host and so there is bounce
buffer requests.

Vmbus channel ring buffer already plays bounce buffer role because
all data from/to host needs to copy from/to between the ring buffer
and IO stack memory. So mark vmbus channel ring buffer visible.

There are two exceptions - packets sent by vmbus_sendpacket_
pagebuffer() and vmbus_sendpacket_mpb_desc(). These packets
contains IO stack memory address and host will access these memory.
So add allocation bounce buffer support in vmbus for these packets.

For SNP isolation VM, guest needs to access the shared memory via
extra address space which is specified by Hyper-V CPUID HYPERV_CPUID_
ISOLATION_CONFIG. The access physical address of the shared memory
should be bounce buffer memory GPA plus with shared_gpa_boundary
reported by CPUID.

This patchset is based on the 5.15-rc1.

Change since v4:
	- Hide hv_mark_gpa_visibility() and set memory visibility via
	  set_memory_encrypted/decrypted() 
	- Change gpadl handle in netvsc and uio driver from u32 to
	  struct vmbus_gpadl.
	- Change vmbus_establish_gpadl()'s gpadl_handle parameter
	  to vmbus_gpadl data structure.
	- Remove hv_get_simp(), hv_get_siefp()  hv_get_synint_*()
	  helper function. Move the logic into hv_get/set_register().
	- Use scsi_dma_map/unmap() instead of dma_map/unmap_sg() in storvsc driver.
	- Allocate rx/tx ring buffer via alloc_pages() in Isolation VM  

Change since V3:
	- Initalize GHCB page in the cpu init callbac.
	- Change vmbus_teardown_gpadl() parameter in order to
	  mask the memory back to non-visible to host.
	- Merge hv_ringbuffer_post_init() into hv_ringbuffer_init().
	- Keep Hyper-V bounce buffer size as same as AMD SEV VM
	- Use dma_map_sg() instead of dm_map_page() in the storvsc driver.

Change since V2:
       - Drop x86_set_memory_enc static call and use platform check
         in the __set_memory_enc_dec() to run platform callback of
	 set memory encrypted or decrypted.

Change since V1:
       - Introduce x86_set_memory_enc static call and so platforms can
         override __set_memory_enc_dec() with their implementation
       - Introduce sev_es_ghcb_hv_call_simple() and share code
         between SEV and Hyper-V code.
       - Not remap monitor pages in the non-SNP isolation VM
       - Make swiotlb_init_io_tlb_mem() return error code and return
         error when dma_map_decrypted() fails.

Change since RFC V4:
       - Introduce dma map decrypted function to remap bounce buffer
          and provide dma map decrypted ops for platform to hook callback.        
       - Split swiotlb and dma map decrypted change into two patches
       - Replace vstart with vaddr in swiotlb changes.

Change since RFC v3:
       - Add interface set_memory_decrypted_map() to decrypt memory and
         map bounce buffer in extra address space
       - Remove swiotlb remap function and store the remap address
         returned by set_memory_decrypted_map() in swiotlb mem data structure.
       - Introduce hv_set_mem_enc() to make code more readable in the __set_memory_enc_dec().

Change since RFC v2:
       - Remove not UIO driver in Isolation VM patch
       - Use vmap_pfn() to replace ioremap_page_range function in
       order to avoid exposing symbol ioremap_page_range() and
       ioremap_page_range()
       - Call hv set mem host visibility hvcall in set_memory_encrypted/decrypted()
       - Enable swiotlb force mode instead of adding Hyper-V dma map/unmap hook
       - Fix code style

Tianyu Lan (12):
  x86/hyperv: Initialize GHCB page in Isolation VM
  x86/hyperv: Initialize shared memory boundary in the Isolation VM.
  x86/hyperv: Add new hvcall guest address host visibility  support
  Drivers: hv: vmbus: Mark vmbus ring buffer visible to host in
    Isolation VM
  x86/hyperv: Add Write/Read MSR registers via ghcb page
  x86/hyperv: Add ghcb hvcall support for SNP VM
  Drivers: hv: vmbus: Add SNP support for VMbus channel initiate 
    message
  Drivers: hv : vmbus: Initialize VMbus ring buffer for Isolation VM
  x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM
  hyperv/IOMMU: Enable swiotlb bounce buffer for Isolation VM
  scsi: storvsc: Add Isolation VM support for storvsc driver
  net: netvsc: Add Isolation VM support for netvsc driver

 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |  78 ++++++--
 arch/x86/hyperv/ivm.c              | 282 ++++++++++++++++++++++++++
 arch/x86/include/asm/hyperv-tlfs.h |  17 ++
 arch/x86/include/asm/mshyperv.h    |  62 ++++--
 arch/x86/include/asm/sev.h         |   6 +
 arch/x86/kernel/cpu/mshyperv.c     |   5 +
 arch/x86/kernel/sev-shared.c       |  63 +++---
 arch/x86/mm/mem_encrypt.c          |   3 +-
 arch/x86/mm/pat/set_memory.c       |  19 +-
 arch/x86/xen/pci-swiotlb-xen.c     |   3 +-
 drivers/hv/Kconfig                 |   1 +
 drivers/hv/channel.c               |  73 ++++---
 drivers/hv/connection.c            |  96 ++++++++-
 drivers/hv/hv.c                    |  85 ++++++--
 drivers/hv/hv_common.c             |  12 ++
 drivers/hv/hyperv_vmbus.h          |   2 +
 drivers/hv/ring_buffer.c           |  55 ++++--
 drivers/hv/vmbus_drv.c             |   4 +
 drivers/iommu/hyperv-iommu.c       |  60 ++++++
 drivers/net/hyperv/hyperv_net.h    |  12 +-
 drivers/net/hyperv/netvsc.c        | 304 +++++++++++++++++++++++++++--
 drivers/net/hyperv/netvsc_drv.c    |   1 +
 drivers/net/hyperv/rndis_filter.c  |   2 +
 drivers/scsi/storvsc_drv.c         |  24 ++-
 drivers/uio/uio_hv_generic.c       |  20 +-
 include/asm-generic/hyperv-tlfs.h  |   1 +
 include/asm-generic/mshyperv.h     |  17 +-
 include/linux/hyperv.h             |  19 +-
 include/linux/swiotlb.h            |   6 +
 kernel/dma/swiotlb.c               |  41 +++-
 31 files changed, 1204 insertions(+), 171 deletions(-)
 create mode 100644 arch/x86/hyperv/ivm.c

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:39:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:39:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186648.335386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8eq-0002Fn-1F; Tue, 14 Sep 2021 13:39:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186648.335386; Tue, 14 Sep 2021 13:39:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ep-0002Fe-UM; Tue, 14 Sep 2021 13:39:31 +0000
Received: by outflank-mailman (input) for mailman id 186648;
 Tue, 14 Sep 2021 13:39:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8en-0001wz-Qo
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:29 +0000
Received: from mail-pj1-x1032.google.com (unknown [2607:f8b0:4864:20::1032])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7a7994b4-e450-4c6e-96eb-880b16d83ea2;
 Tue, 14 Sep 2021 13:39:26 +0000 (UTC)
Received: by mail-pj1-x1032.google.com with SMTP id
 lb1-20020a17090b4a4100b001993f863df2so2799411pjb.5
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:26 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a7994b4-e450-4c6e-96eb-880b16d83ea2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=QrhS3LTuU3+XNI0p3h2fXGmudzlD8JRBXhiK5DozhlE=;
        b=o/165tggS7dVhrFX6GAOmcFAAgPlx1idPlHms0MswSiWTbPGiIx2vuWrvhXVMR6XkU
         WnL7lHtu2y5YInUkOyc6YYbJ+YyMZOyaX6VetzB8rAhjcwH7L1jcLGhpqoJPd3iTqUQ1
         dNQjwCQDWy/PlCBNgPOOFGfLa2wHkiAyAEiHBsGTMmFG840H1ZDmeTA745j4Xd7kZSJv
         rxrqlUztv9pv74ZIlwPiVGtpvnJTTCB3hGp2H3qKrMxHv8B5zJ4HRQjUdyYF67qJUBZD
         vqO4oTXLK/H1sWmy174msIvzDi3RamnaVJYMCU5gkVx926Wxg7ezDeybpNyhXJEPyAHm
         AB6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=QrhS3LTuU3+XNI0p3h2fXGmudzlD8JRBXhiK5DozhlE=;
        b=rgdP94Gcra59jrXn80Q7Vw2MgH7j8Qf1zY7NUt6fbAiWyUC6B+g7h00HWZgkA1u/MI
         baoFvh3gHz97Jke/JtMenB0Ekh0uC1rej9SSlbJSgyKkC2yR/CrC58FvFPDqsfxUYMn5
         HTjrnCyIXhhaojZ7ZF5QqM4+lj+owylTebxv7GdjWZmSsenjXWE+9siGnWb3aD+Zp+zM
         Mh3H81sEG9jPKUWlA8TvEFxzjEZedWieh3cCRfwN7h07yC01tn4kT50V3IBEQULI93Tv
         0UfcX7/AJnm7HBc7Qd4rk3a9QAlOQdEf2V8OJPCbfWl60rLCw+OTFS46RC/0YtIgS5WE
         viyA==
X-Gm-Message-State: AOAM530kybuoWCUC/KDhD/t6E+RhN8kod7dpzIeLUuCci2RSNG1EsVOH
	HebbA5Lg/DXR5RVjgT//McU=
X-Google-Smtp-Source: ABdhPJxcVcMgZPPk9T3dmVUHKa5l0JwMCgOBrxv8XyGGHB4KduVicPNXQOZXfeM3vdDo5Yrr754YSw==
X-Received: by 2002:a17:90b:1bd1:: with SMTP id oa17mr2215152pjb.82.1631626766123;
        Tue, 14 Sep 2021 06:39:26 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 01/12] x86/hyperv: Initialize GHCB page in Isolation VM
Date: Tue, 14 Sep 2021 09:39:02 -0400
Message-Id: <20210914133916.1440931-2-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Hyperv exposes GHCB page via SEV ES GHCB MSR for SNP guest
to communicate with hypervisor. Map GHCB page for all
cpus to read/write MSR register and submit hvcall request
via ghcb page.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Fix typo comment

Chagne since v3:
        * Rename ghcb_base to hv_ghcb_pg and move it out of
	  struct ms_hyperv_info.
	* Allocate hv_ghcb_pg before cpuhp_setup_state() and leverage
	  hv_cpu_init() to initialize ghcb page.
---
 arch/x86/hyperv/hv_init.c       | 68 +++++++++++++++++++++++++++++----
 arch/x86/include/asm/mshyperv.h |  4 ++
 arch/x86/kernel/cpu/mshyperv.c  |  3 ++
 include/asm-generic/mshyperv.h  |  1 +
 4 files changed, 69 insertions(+), 7 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 708a2712a516..a7e922755ad1 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -20,6 +20,7 @@
 #include <linux/kexec.h>
 #include <linux/version.h>
 #include <linux/vmalloc.h>
+#include <linux/io.h>
 #include <linux/mm.h>
 #include <linux/hyperv.h>
 #include <linux/slab.h>
@@ -36,12 +37,42 @@ EXPORT_SYMBOL_GPL(hv_current_partition_id);
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
+void __percpu **hv_ghcb_pg;
+
 /* Storage to save the hypercall page temporarily for hibernation */
 static void *hv_hypercall_pg_saved;
 
 struct hv_vp_assist_page **hv_vp_assist_page;
 EXPORT_SYMBOL_GPL(hv_vp_assist_page);
 
+static int hyperv_init_ghcb(void)
+{
+	u64 ghcb_gpa;
+	void *ghcb_va;
+	void **ghcb_base;
+
+	if (!hv_isolation_type_snp())
+		return 0;
+
+	if (!hv_ghcb_pg)
+		return -EINVAL;
+
+	/*
+	 * GHCB page is allocated by paravisor. The address
+	 * returned by MSR_AMD64_SEV_ES_GHCB is above shared
+	 * memory boundary and map it here.
+	 */
+	rdmsrl(MSR_AMD64_SEV_ES_GHCB, ghcb_gpa);
+	ghcb_va = memremap(ghcb_gpa, HV_HYP_PAGE_SIZE, MEMREMAP_WB);
+	if (!ghcb_va)
+		return -ENOMEM;
+
+	ghcb_base = (void **)this_cpu_ptr(hv_ghcb_pg);
+	*ghcb_base = ghcb_va;
+
+	return 0;
+}
+
 static int hv_cpu_init(unsigned int cpu)
 {
 	union hv_vp_assist_msr_contents msr = { 0 };
@@ -85,7 +116,7 @@ static int hv_cpu_init(unsigned int cpu)
 		}
 	}
 
-	return 0;
+	return hyperv_init_ghcb();
 }
 
 static void (*hv_reenlightenment_cb)(void);
@@ -177,6 +208,14 @@ static int hv_cpu_die(unsigned int cpu)
 {
 	struct hv_reenlightenment_control re_ctrl;
 	unsigned int new_cpu;
+	void **ghcb_va;
+
+	if (hv_ghcb_pg) {
+		ghcb_va = (void **)this_cpu_ptr(hv_ghcb_pg);
+		if (*ghcb_va)
+			memunmap(*ghcb_va);
+		*ghcb_va = NULL;
+	}
 
 	hv_common_cpu_die(cpu);
 
@@ -366,10 +405,16 @@ void __init hyperv_init(void)
 		goto common_free;
 	}
 
+	if (hv_isolation_type_snp()) {
+		hv_ghcb_pg = alloc_percpu(void *);
+		if (!hv_ghcb_pg)
+			goto free_vp_assist_page;
+	}
+
 	cpuhp = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:online",
 				  hv_cpu_init, hv_cpu_die);
 	if (cpuhp < 0)
-		goto free_vp_assist_page;
+		goto free_ghcb_page;
 
 	/*
 	 * Setup the hypercall page and enable hypercalls.
@@ -383,10 +428,8 @@ void __init hyperv_init(void)
 			VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX,
 			VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
 			__builtin_return_address(0));
-	if (hv_hypercall_pg == NULL) {
-		wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
-		goto remove_cpuhp_state;
-	}
+	if (hv_hypercall_pg == NULL)
+		goto clean_guest_os_id;
 
 	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 	hypercall_msr.enable = 1;
@@ -456,8 +499,11 @@ void __init hyperv_init(void)
 	hv_query_ext_cap(0);
 	return;
 
-remove_cpuhp_state:
+clean_guest_os_id:
+	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
 	cpuhp_remove_state(cpuhp);
+free_ghcb_page:
+	free_percpu(hv_ghcb_pg);
 free_vp_assist_page:
 	kfree(hv_vp_assist_page);
 	hv_vp_assist_page = NULL;
@@ -559,3 +605,11 @@ bool hv_is_isolation_supported(void)
 {
 	return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE;
 }
+
+DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
+
+bool hv_isolation_type_snp(void)
+{
+	return static_branch_unlikely(&isolation_type_snp);
+}
+EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index adccbc209169..37739a277ac6 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -11,6 +11,8 @@
 #include <asm/paravirt.h>
 #include <asm/mshyperv.h>
 
+DECLARE_STATIC_KEY_FALSE(isolation_type_snp);
+
 typedef int (*hyperv_fill_flush_list_func)(
 		struct hv_guest_mapping_flush_list *flush,
 		void *data);
@@ -39,6 +41,8 @@ extern void *hv_hypercall_pg;
 
 extern u64 hv_current_partition_id;
 
+extern void __percpu **hv_ghcb_pg;
+
 int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
 int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
 int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags);
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index e095c28d27ae..b09ade389040 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -316,6 +316,9 @@ static void __init ms_hyperv_init_platform(void)
 
 		pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n",
 			ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
+
+		if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP)
+			static_branch_enable(&isolation_type_snp);
 	}
 
 	if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) {
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index c1ab6a6e72b5..0924bbd8458e 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -237,6 +237,7 @@ bool hv_is_hyperv_initialized(void);
 bool hv_is_hibernation_supported(void);
 enum hv_isolation_type hv_get_isolation_type(void);
 bool hv_is_isolation_supported(void);
+bool hv_isolation_type_snp(void);
 void hyperv_cleanup(void);
 bool hv_query_ext_cap(u64 cap_query);
 #else /* CONFIG_HYPERV */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:39:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:39:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186649.335397 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8et-0002Yv-9S; Tue, 14 Sep 2021 13:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186649.335397; Tue, 14 Sep 2021 13:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8et-0002Yo-5x; Tue, 14 Sep 2021 13:39:35 +0000
Received: by outflank-mailman (input) for mailman id 186649;
 Tue, 14 Sep 2021 13:39:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8es-0001wz-Qw
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:34 +0000
Received: from mail-pl1-x62c.google.com (unknown [2607:f8b0:4864:20::62c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3789e736-6a54-4fee-bbb4-fa76cd6f82cf;
 Tue, 14 Sep 2021 13:39:29 +0000 (UTC)
Received: by mail-pl1-x62c.google.com with SMTP id w6so8208421pll.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:29 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3789e736-6a54-4fee-bbb4-fa76cd6f82cf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=YUwTqwYZOOkMzYqyDRP5u7DXlTVFM/o7xmj81RyBU+U=;
        b=Zy+W6lLKWmt7luEwpTRuKO/55L+1pJhM+U2Y08JCjUl/Sx3GrJY7QhjVoeYU8XS0sZ
         VYkNN1yYSgJpcM/oeMCO+0ImLroJ2E6tKqOy3G3BQc+A+XteLhWOhUNt04iKCz4oLukL
         ZXdeQ2jx1nvY0B/yoIhbppgpWtNXrdn1B8PukHkeaXer6DbeBQQ6LMeN6zIn0UbmFUqA
         XFkU1p/3fDU6Rv88cbsucwIK2FeuA5bSFEOuD0P01iDmKeSieEZM0MwidbNyW+kC5LvM
         reTUrry6bD4yBiSWg+2gSNkw7hmnvgvjy6KZtQZkdkIbi9CfdzA5mje9Mgsm1Myh/rpm
         yczA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=YUwTqwYZOOkMzYqyDRP5u7DXlTVFM/o7xmj81RyBU+U=;
        b=fkTE9sCk7UZIaOE3Fkrq0YAxUSmzjn4SFyIT/099Rs+ezUmQIun/5Xi7NrwpWEMfJL
         ALOo/HBwPk6F6Znat5AJsHwKIqabMfs/CkqJlOgPVLUnb1Er18cKW8tGQAIoKkBX4CF4
         Q1vaRIQG6wlF9u7w/rLCfisrGRvLboTcvCMrDHh1BvZVuzKMJwLCsYunrlukI23bgyWX
         R7ZTvZxNHDvrFpPJj5EnW2GSurbI2bf4usDqBqO3X286gbsBlVUw5wzF01oRV5zj++pZ
         PLfu2HSefoHUqeteO1IWGlecZZAdj23Z3smPTzlNxO5YM5zxNE2ThDivbEGFH4eH64Tb
         uWmw==
X-Gm-Message-State: AOAM533nR4jru6DD1asRxM1+8dY2cYvaB9PPWfl1btuftbU3tlnubJJ8
	wJ9CEmr7EVQz749XF2X7jJ0=
X-Google-Smtp-Source: ABdhPJx44lS4Ai2yXXn8V591ghM4am9GxTdD+xMKN2ARZV8UrnEqqpomOlRQvCNWc6041rvSAnTYOg==
X-Received: by 2002:a17:90a:9404:: with SMTP id r4mr2180743pjo.240.1631626769029;
        Tue, 14 Sep 2021 06:39:29 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 02/12] x86/hyperv: Initialize shared memory boundary in the Isolation VM.
Date: Tue, 14 Sep 2021 09:39:03 -0400
Message-Id: <20210914133916.1440931-3-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Hyper-V exposes shared memory boundary via cpuid
HYPERV_CPUID_ISOLATION_CONFIG and store it in the
shared_gpa_boundary of ms_hyperv struct. This prepares
to share memory with host for SNP guest.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Rename reserve field.

Change since v3:
	* user BIT_ULL to get shared_gpa_boundary
	* Rename field Reserved* to reserved
---
 arch/x86/kernel/cpu/mshyperv.c |  2 ++
 include/asm-generic/mshyperv.h | 12 +++++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index b09ade389040..4794b716ec79 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -313,6 +313,8 @@ static void __init ms_hyperv_init_platform(void)
 	if (ms_hyperv.priv_high & HV_ISOLATION) {
 		ms_hyperv.isolation_config_a = cpuid_eax(HYPERV_CPUID_ISOLATION_CONFIG);
 		ms_hyperv.isolation_config_b = cpuid_ebx(HYPERV_CPUID_ISOLATION_CONFIG);
+		ms_hyperv.shared_gpa_boundary =
+			BIT_ULL(ms_hyperv.shared_gpa_boundary_bits);
 
 		pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n",
 			ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index 0924bbd8458e..e04efb87fee5 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -35,7 +35,17 @@ struct ms_hyperv_info {
 	u32 max_vp_index;
 	u32 max_lp_index;
 	u32 isolation_config_a;
-	u32 isolation_config_b;
+	union {
+		u32 isolation_config_b;
+		struct {
+			u32 cvm_type : 4;
+			u32 reserved1 : 1;
+			u32 shared_gpa_boundary_active : 1;
+			u32 shared_gpa_boundary_bits : 6;
+			u32 reserved2 : 20;
+		};
+	};
+	u64 shared_gpa_boundary;
 };
 extern struct ms_hyperv_info ms_hyperv;
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:39:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:39:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186650.335408 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ey-0002wm-IN; Tue, 14 Sep 2021 13:39:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186650.335408; Tue, 14 Sep 2021 13:39:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ey-0002wV-F1; Tue, 14 Sep 2021 13:39:40 +0000
Received: by outflank-mailman (input) for mailman id 186650;
 Tue, 14 Sep 2021 13:39:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8ex-0001wz-R0
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:39 +0000
Received: from mail-pj1-x102c.google.com (unknown [2607:f8b0:4864:20::102c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2e51e196-b0fe-4698-a9cd-225e048bc0d9;
 Tue, 14 Sep 2021 13:39:32 +0000 (UTC)
Received: by mail-pj1-x102c.google.com with SMTP id oc9so8895861pjb.4
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:32 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e51e196-b0fe-4698-a9cd-225e048bc0d9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=iJLckU3abLu4a4Vn3DJ3KB7tPATL+qkVDgrKdWDNQvA=;
        b=fzFgKrgO3jR5OKFVzEwIfXK3KnTUpULKrPF1M+i/SNeZQDHvH52YDOWEoP4MQb1ozB
         +oNlHnH4gt/YrymRtTUrhVGA30YglWkcl2xkpeHfyB0Ic1vejz3UUxhTDRKfUqgp/HNN
         eQXM4aWS/XLaXBYpkvaCYSaW3dXCFEJUZGhqoZZlPKXdCGKbNvC6RMO9b6vH5Kze8cof
         w4hjDa73RlAeZRTFwZFgpYwyp7E56g8iDihxR5KWFZAHj35Qjhu4nFdLWoTi9ZEEaPj+
         SOVkuFXNs1puyXoaXf2zO/cDECY6Ac/eRl7xItk7LRaW7iYZ6GlolkoDeF5TcexsMPxH
         CFRQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=iJLckU3abLu4a4Vn3DJ3KB7tPATL+qkVDgrKdWDNQvA=;
        b=kw8pUFCpi7GigK03gvmT2h9OdPqlJc8fjRBzGkjQuM4PtJfPPvsIAgIiP1EvyZdaiL
         KDYAZ+w+wqCg8m+v1DudiNPfUaeZ/PPTEw0o/CSLT66iiq6+MGxsGdPQtzhJjm0q+/xm
         c5sfoToHscNcrKcQUy5vHN0+o4R3BLG4fIT0CJBj2TI/ppriAF7htN/WhyknzPCwgOwt
         UET5FvXOiMfiqXmw0/V8ffolIEclX2/WW2yqEYEj32a/p76MZyCXFCJl3lTsyUhlJ6a/
         J/8FccbTKS+sb9w45aWtmv4GaRZRtiARGYEhF5MI7LAhODHtl2qHIBOwWm03IHtzwTsF
         JAXQ==
X-Gm-Message-State: AOAM530NwUhYAhK+flvTsS7FXEK1giNxTmvHrzROG3XYMiwWMEGSHxna
	4emwoqne7iVFl1g7QHqwJ6o=
X-Google-Smtp-Source: ABdhPJzvBiQY8jpYU+5kaM8wZ2SZ6gM/MP0CdPvROt1+d+Wt4/7tv+dIMdviYBvG4czVMjty3Drajw==
X-Received: by 2002:a17:90b:a42:: with SMTP id gw2mr2163041pjb.26.1631626771780;
        Tue, 14 Sep 2021 06:39:31 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 03/12] x86/hyperv: Add new hvcall guest address host visibility  support
Date: Tue, 14 Sep 2021 09:39:04 -0400
Message-Id: <20210914133916.1440931-4-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Add new hvcall guest address host visibility support to mark
memory visible to host. Call it inside set_memory_decrypted
/encrypted(). Add HYPERVISOR feature check in the
hv_is_isolation_supported() to optimize in non-virtualization
environment.

Acked-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
        * Fix typo in the comment
	* Make hv_mark_gpa_visibility() to be a static function
	* Merge __hv_set_mem_host_visibility() and hv_set_mem_host_visibility()

Change since v3:
	* Fix error code handle in the __hv_set_mem_host_visibility().
	* Move HvCallModifySparseGpaPageHostVisibility near to enum
	  hv_mem_host_visibility.

Change since v2:
       * Rework __set_memory_enc_dec() and call Hyper-V and AMD function
         according to platform check.

Change since v1:
       * Use new staic call x86_set_memory_enc to avoid add Hyper-V
         specific check in the set_memory code.
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   6 ++
 arch/x86/hyperv/ivm.c              | 105 +++++++++++++++++++++++++++++
 arch/x86/include/asm/hyperv-tlfs.h |  17 +++++
 arch/x86/include/asm/mshyperv.h    |   2 +-
 arch/x86/mm/pat/set_memory.c       |  19 ++++--
 include/asm-generic/hyperv-tlfs.h  |   1 +
 include/asm-generic/mshyperv.h     |   1 +
 8 files changed, 146 insertions(+), 7 deletions(-)
 create mode 100644 arch/x86/hyperv/ivm.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 48e2c51464e8..5d2de10809ae 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
-obj-y			:= hv_init.o mmu.o nested.o irqdomain.o
+obj-y			:= hv_init.o mmu.o nested.o irqdomain.o ivm.o
 obj-$(CONFIG_X86_64)	+= hv_apic.o hv_proc.o
 
 ifdef CONFIG_X86_64
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index a7e922755ad1..d57df6825527 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -603,6 +603,12 @@ EXPORT_SYMBOL_GPL(hv_get_isolation_type);
 
 bool hv_is_isolation_supported(void)
 {
+	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
+		return false;
+
+	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
+		return false;
+
 	return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE;
 }
 
diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
new file mode 100644
index 000000000000..79e7fb83472a
--- /dev/null
+++ b/arch/x86/hyperv/ivm.c
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hyper-V Isolation VM interface with paravisor and hypervisor
+ *
+ * Author:
+ *  Tianyu Lan <Tianyu.Lan@microsoft.com>
+ */
+
+#include <linux/hyperv.h>
+#include <linux/types.h>
+#include <linux/bitfield.h>
+#include <linux/slab.h>
+#include <asm/io.h>
+#include <asm/mshyperv.h>
+
+/*
+ * hv_mark_gpa_visibility - Set pages visible to host via hvcall.
+ *
+ * In Isolation VM, all guest memory is encrypted from host and guest
+ * needs to set memory visible to host via hvcall before sharing memory
+ * with host.
+ */
+static int hv_mark_gpa_visibility(u16 count, const u64 pfn[],
+			   enum hv_mem_host_visibility visibility)
+{
+	struct hv_gpa_range_for_visibility **input_pcpu, *input;
+	u16 pages_processed;
+	u64 hv_status;
+	unsigned long flags;
+
+	/* no-op if partition isolation is not enabled */
+	if (!hv_is_isolation_supported())
+		return 0;
+
+	if (count > HV_MAX_MODIFY_GPA_REP_COUNT) {
+		pr_err("Hyper-V: GPA count:%d exceeds supported:%lu\n", count,
+			HV_MAX_MODIFY_GPA_REP_COUNT);
+		return -EINVAL;
+	}
+
+	local_irq_save(flags);
+	input_pcpu = (struct hv_gpa_range_for_visibility **)
+			this_cpu_ptr(hyperv_pcpu_input_arg);
+	input = *input_pcpu;
+	if (unlikely(!input)) {
+		local_irq_restore(flags);
+		return -EINVAL;
+	}
+
+	input->partition_id = HV_PARTITION_ID_SELF;
+	input->host_visibility = visibility;
+	input->reserved0 = 0;
+	input->reserved1 = 0;
+	memcpy((void *)input->gpa_page_list, pfn, count * sizeof(*pfn));
+	hv_status = hv_do_rep_hypercall(
+			HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY, count,
+			0, input, &pages_processed);
+	local_irq_restore(flags);
+
+	if (hv_result_success(hv_status))
+		return 0;
+	else
+		return -EFAULT;
+}
+
+/*
+ * hv_set_mem_host_visibility - Set specified memory visible to host.
+ *
+ * In Isolation VM, all guest memory is encrypted from host and guest
+ * needs to set memory visible to host via hvcall before sharing memory
+ * with host. This function works as wrap of hv_mark_gpa_visibility()
+ * with memory base and size.
+ */
+int hv_set_mem_host_visibility(unsigned long kbuffer, int pagecount, bool visible)
+{
+	enum hv_mem_host_visibility visibility = visible ?
+			VMBUS_PAGE_VISIBLE_READ_WRITE : VMBUS_PAGE_NOT_VISIBLE;
+	u64 *pfn_array;
+	int ret = 0;
+	int i, pfn;
+
+	if (!hv_is_isolation_supported() || !hv_hypercall_pg)
+		return 0;
+
+	pfn_array = kmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
+	if (!pfn_array)
+		return -ENOMEM;
+
+	for (i = 0, pfn = 0; i < pagecount; i++) {
+		pfn_array[pfn] = virt_to_hvpfn((void *)kbuffer + i * HV_HYP_PAGE_SIZE);
+		pfn++;
+
+		if (pfn == HV_MAX_MODIFY_GPA_REP_COUNT || i == pagecount - 1) {
+			ret = hv_mark_gpa_visibility(pfn, pfn_array,
+						     visibility);
+			if (ret)
+				goto err_free_pfn_array;
+			pfn = 0;
+		}
+	}
+
+ err_free_pfn_array:
+	kfree(pfn_array);
+	return ret;
+}
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 2322d6bd5883..381e88122a5f 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -276,6 +276,23 @@ enum hv_isolation_type {
 #define HV_X64_MSR_TIME_REF_COUNT	HV_REGISTER_TIME_REF_COUNT
 #define HV_X64_MSR_REFERENCE_TSC	HV_REGISTER_REFERENCE_TSC
 
+/* Hyper-V memory host visibility */
+enum hv_mem_host_visibility {
+	VMBUS_PAGE_NOT_VISIBLE		= 0,
+	VMBUS_PAGE_VISIBLE_READ_ONLY	= 1,
+	VMBUS_PAGE_VISIBLE_READ_WRITE	= 3
+};
+
+/* HvCallModifySparseGpaPageHostVisibility hypercall */
+#define HV_MAX_MODIFY_GPA_REP_COUNT	((PAGE_SIZE / sizeof(u64)) - 2)
+struct hv_gpa_range_for_visibility {
+	u64 partition_id;
+	u32 host_visibility:2;
+	u32 reserved0:30;
+	u32 reserved1;
+	u64 gpa_page_list[HV_MAX_MODIFY_GPA_REP_COUNT];
+} __packed;
+
 /*
  * Declare the MSR used to setup pages used to communicate with the hypervisor.
  */
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 37739a277ac6..ede440f9a1e2 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -192,7 +192,7 @@ struct irq_domain *hv_create_pci_msi_domain(void);
 int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector,
 		struct hv_interrupt_entry *entry);
 int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
-
+int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible);
 #else /* CONFIG_HYPERV */
 static inline void hyperv_init(void) {}
 static inline void hyperv_setup_mmu_ops(void) {}
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index ad8a5c586a35..1e4a0882820a 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -29,6 +29,8 @@
 #include <asm/proto.h>
 #include <asm/memtype.h>
 #include <asm/set_memory.h>
+#include <asm/hyperv-tlfs.h>
+#include <asm/mshyperv.h>
 
 #include "../mm_internal.h"
 
@@ -1980,15 +1982,11 @@ int set_memory_global(unsigned long addr, int numpages)
 				    __pgprot(_PAGE_GLOBAL), 0);
 }
 
-static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+static int __set_memory_enc_pgtable(unsigned long addr, int numpages, bool enc)
 {
 	struct cpa_data cpa;
 	int ret;
 
-	/* Nothing to do if memory encryption is not active */
-	if (!mem_encrypt_active())
-		return 0;
-
 	/* Should not be working on unaligned addresses */
 	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
 		addr &= PAGE_MASK;
@@ -2023,6 +2021,17 @@ static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
 	return ret;
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+	if (hv_is_isolation_supported())
+		return hv_set_mem_host_visibility(addr, numpages, !enc);
+
+	if (mem_encrypt_active())
+		return __set_memory_enc_pgtable(addr, numpages, enc);
+
+	return 0;
+}
+
 int set_memory_encrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, true);
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index 56348a541c50..8ed6733d5146 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -158,6 +158,7 @@ struct ms_hyperv_tsc_page {
 #define HVCALL_RETARGET_INTERRUPT		0x007e
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
+#define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db
 
 /* Extended hypercalls */
 #define HV_EXT_CALL_QUERY_CAPABILITIES		0x8001
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index e04efb87fee5..cb529c85c0ad 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -254,6 +254,7 @@ bool hv_query_ext_cap(u64 cap_query);
 static inline bool hv_is_hyperv_initialized(void) { return false; }
 static inline bool hv_is_hibernation_supported(void) { return false; }
 static inline void hyperv_cleanup(void) {}
+static inline bool hv_is_isolation_supported(void) { return false; }
 #endif /* CONFIG_HYPERV */
 
 #endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:39:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:39:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186654.335420 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8f9-0003cj-1V; Tue, 14 Sep 2021 13:39:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186654.335420; Tue, 14 Sep 2021 13:39:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8f8-0003cW-TJ; Tue, 14 Sep 2021 13:39:50 +0000
Received: by outflank-mailman (input) for mailman id 186654;
 Tue, 14 Sep 2021 13:39:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8f7-0001wz-RQ
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:49 +0000
Received: from mail-pf1-x42a.google.com (unknown [2607:f8b0:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0c1cf805-08fe-4e35-9547-26d1d010eea5;
 Tue, 14 Sep 2021 13:39:35 +0000 (UTC)
Received: by mail-pf1-x42a.google.com with SMTP id j16so12255448pfc.2
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:35 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c1cf805-08fe-4e35-9547-26d1d010eea5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=02yTUUUdT/7RUZ1OmKs0RzX0RhFgmF+tfZ6adAfAN0Q=;
        b=ArjBR52nkO4bzAaOf5wvfC4rn632GJGm1maXaaJ0TgHHpjedDqm+WJBv+1VeIaZgH8
         5flQwVivXRx6pq+gRBaVASuOXo4ymVynu6o0WucDl6BnBxy/91jZ+nwMIgdhlKICaUAN
         fL2xHyxJgQNN+aDYvnsnscMMXj3E3OC8kfXdFSGOUc0+WouUasML3nSV8Cf8i+dRrNsL
         SBZXPNecPo4cfaG1kwB4eYB7mQqvkhfeKDY6slY3Csa2hCRcX2Jhc3s7mqeRwF6/hgpa
         VplUwV00fbi/jQIiWIZD05ygm/4jJojIcYosTkxoDhhh7QGJk2WMeezVJnuWZVbEcLJ2
         7OGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=02yTUUUdT/7RUZ1OmKs0RzX0RhFgmF+tfZ6adAfAN0Q=;
        b=adH0pCISCGDrI3QbxvZ4akEWsxwYW7CPESIhHZGIMhaKt96J7om7vPgKenfJ1/ZrFq
         WqeKGzVfdD6EvVHEbs/dGFeNXrZ7e7cnPCETN1E/iViAY7HWRR6/2H34H1TQQ6LwdRlb
         WLAAyE5w+6dBJGHFT7dgoJPRKiV8MUGiaP6FLmybCj+GcAiFAgO6da7F+jm2DLZoWjxR
         kR1/N7h6KhJAmiVCPeJYFv3Zk4lpynbjfR3dlLDDDDwlTy4SHCErzWLuAw8yniWd3k6Y
         x5p97oPD8uur47Sh+FRIzgB0vzPJ5bVEMZNGyOaAiGfnPav6FWMVsgdl2Oh4BEB7IT6g
         Srgg==
X-Gm-Message-State: AOAM531Z8PlRy5qXZXvnW+oYr2dRDwMHMn5PCbhMgrRS2j430bBmCeWf
	NOMD2OCFUFG9YSKJATKjHLY=
X-Google-Smtp-Source: ABdhPJzPDtIgzdi7XXhZ9rLdA7pHLUxlkJdK1Y4rWJkJJlHJjwPXRjwfsPAQ3lITRYtRblyGoAlIXQ==
X-Received: by 2002:a63:6544:: with SMTP id z65mr15696367pgb.325.1631626774304;
        Tue, 14 Sep 2021 06:39:34 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 04/12] Drivers: hv: vmbus: Mark vmbus ring buffer visible to host in Isolation VM
Date: Tue, 14 Sep 2021 09:39:05 -0400
Message-Id: <20210914133916.1440931-5-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Mark vmbus ring buffer visible with set_memory_decrypted() when
establish gpadl handle.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change sincv v4
	* Change gpadl handle in netvsc and uio driver from u32 to
	  struct vmbus_gpadl.
	* Change vmbus_establish_gpadl()'s gpadl_handle parameter
	  to vmbus_gpadl data structure.

Change since v3:
	* Change vmbus_teardown_gpadl() parameter and put gpadl handle,
	  buffer and buffer size in the struct vmbus_gpadl.
---
 drivers/hv/channel.c            | 54 ++++++++++++++++++++++++---------
 drivers/net/hyperv/hyperv_net.h |  5 +--
 drivers/net/hyperv/netvsc.c     | 17 ++++++-----
 drivers/uio/uio_hv_generic.c    | 20 ++++++------
 include/linux/hyperv.h          | 12 ++++++--
 5 files changed, 71 insertions(+), 37 deletions(-)

diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index f3761c73b074..cf419eb1de77 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -17,6 +17,7 @@
 #include <linux/hyperv.h>
 #include <linux/uio.h>
 #include <linux/interrupt.h>
+#include <linux/set_memory.h>
 #include <asm/page.h>
 #include <asm/mshyperv.h>
 
@@ -456,7 +457,7 @@ static int create_gpadl_header(enum hv_gpadl_type type, void *kbuffer,
 static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
 				   enum hv_gpadl_type type, void *kbuffer,
 				   u32 size, u32 send_offset,
-				   u32 *gpadl_handle)
+				   struct vmbus_gpadl *gpadl)
 {
 	struct vmbus_channel_gpadl_header *gpadlmsg;
 	struct vmbus_channel_gpadl_body *gpadl_body;
@@ -474,6 +475,15 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
 	if (ret)
 		return ret;
 
+	ret = set_memory_decrypted((unsigned long)kbuffer,
+				   HVPFN_UP(size));
+	if (ret) {
+		dev_warn(&channel->device_obj->device,
+			 "Failed to set host visibility for new GPADL %d.\n",
+			 ret);
+		return ret;
+	}
+
 	init_completion(&msginfo->waitevent);
 	msginfo->waiting_channel = channel;
 
@@ -537,7 +547,10 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
 	}
 
 	/* At this point, we received the gpadl created msg */
-	*gpadl_handle = gpadlmsg->gpadl;
+	gpadl->gpadl_handle = gpadlmsg->gpadl;
+	gpadl->buffer = kbuffer;
+	gpadl->size = size;
+
 
 cleanup:
 	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
@@ -549,6 +562,11 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
 	}
 
 	kfree(msginfo);
+
+	if (ret)
+		set_memory_encrypted((unsigned long)kbuffer,
+				     HVPFN_UP(size));
+
 	return ret;
 }
 
@@ -561,10 +579,10 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
  * @gpadl_handle: some funky thing
  */
 int vmbus_establish_gpadl(struct vmbus_channel *channel, void *kbuffer,
-			  u32 size, u32 *gpadl_handle)
+			  u32 size, struct vmbus_gpadl *gpadl)
 {
 	return __vmbus_establish_gpadl(channel, HV_GPADL_BUFFER, kbuffer, size,
-				       0U, gpadl_handle);
+				       0U, gpadl);
 }
 EXPORT_SYMBOL_GPL(vmbus_establish_gpadl);
 
@@ -639,6 +657,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	struct vmbus_channel_open_channel *open_msg;
 	struct vmbus_channel_msginfo *open_info = NULL;
 	struct page *page = newchannel->ringbuffer_page;
+	struct vmbus_gpadl gpadl;
 	u32 send_pages, recv_pages;
 	unsigned long flags;
 	int err;
@@ -675,7 +694,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 		goto error_clean_ring;
 
 	/* Establish the gpadl for the ring buffer */
-	newchannel->ringbuffer_gpadlhandle = 0;
+	newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0;
 
 	err = __vmbus_establish_gpadl(newchannel, HV_GPADL_RING,
 				      page_address(newchannel->ringbuffer_page),
@@ -701,7 +720,8 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	open_msg->header.msgtype = CHANNELMSG_OPENCHANNEL;
 	open_msg->openid = newchannel->offermsg.child_relid;
 	open_msg->child_relid = newchannel->offermsg.child_relid;
-	open_msg->ringbuffer_gpadlhandle = newchannel->ringbuffer_gpadlhandle;
+	open_msg->ringbuffer_gpadlhandle
+		= newchannel->ringbuffer_gpadlhandle.gpadl_handle;
 	/*
 	 * The unit of ->downstream_ringbuffer_pageoffset is HV_HYP_PAGE and
 	 * the unit of ->ringbuffer_send_offset (i.e. send_pages) is PAGE, so
@@ -759,8 +779,8 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 error_free_info:
 	kfree(open_info);
 error_free_gpadl:
-	vmbus_teardown_gpadl(newchannel, newchannel->ringbuffer_gpadlhandle);
-	newchannel->ringbuffer_gpadlhandle = 0;
+	vmbus_teardown_gpadl(newchannel, &newchannel->ringbuffer_gpadlhandle);
+	newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0;
 error_clean_ring:
 	hv_ringbuffer_cleanup(&newchannel->outbound);
 	hv_ringbuffer_cleanup(&newchannel->inbound);
@@ -806,7 +826,7 @@ EXPORT_SYMBOL_GPL(vmbus_open);
 /*
  * vmbus_teardown_gpadl -Teardown the specified GPADL handle
  */
-int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)
+int vmbus_teardown_gpadl(struct vmbus_channel *channel, struct vmbus_gpadl *gpadl)
 {
 	struct vmbus_channel_gpadl_teardown *msg;
 	struct vmbus_channel_msginfo *info;
@@ -825,7 +845,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)
 
 	msg->header.msgtype = CHANNELMSG_GPADL_TEARDOWN;
 	msg->child_relid = channel->offermsg.child_relid;
-	msg->gpadl = gpadl_handle;
+	msg->gpadl = gpadl->gpadl_handle;
 
 	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
 	list_add_tail(&info->msglistentry,
@@ -859,6 +879,12 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)
 	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
 
 	kfree(info);
+
+	ret = set_memory_encrypted((unsigned long)gpadl->buffer,
+				   HVPFN_UP(gpadl->size));
+	if (ret)
+		pr_warn("Fail to set mem host visibility in GPADL teardown %d.\n", ret);
+
 	return ret;
 }
 EXPORT_SYMBOL_GPL(vmbus_teardown_gpadl);
@@ -896,6 +922,7 @@ void vmbus_reset_channel_cb(struct vmbus_channel *channel)
 static int vmbus_close_internal(struct vmbus_channel *channel)
 {
 	struct vmbus_channel_close_channel *msg;
+	struct vmbus_gpadl gpadl;
 	int ret;
 
 	vmbus_reset_channel_cb(channel);
@@ -933,9 +960,8 @@ static int vmbus_close_internal(struct vmbus_channel *channel)
 	}
 
 	/* Tear down the gpadl for the channel's ring buffer */
-	else if (channel->ringbuffer_gpadlhandle) {
-		ret = vmbus_teardown_gpadl(channel,
-					   channel->ringbuffer_gpadlhandle);
+	else if (channel->ringbuffer_gpadlhandle.gpadl_handle) {
+		ret = vmbus_teardown_gpadl(channel, &channel->ringbuffer_gpadlhandle);
 		if (ret) {
 			pr_err("Close failed: teardown gpadl return %d\n", ret);
 			/*
@@ -944,7 +970,7 @@ static int vmbus_close_internal(struct vmbus_channel *channel)
 			 */
 		}
 
-		channel->ringbuffer_gpadlhandle = 0;
+		channel->ringbuffer_gpadlhandle.gpadl_handle = 0;
 	}
 
 	if (!ret)
diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index bc48855dff10..315278a7cf88 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -1075,14 +1075,15 @@ struct netvsc_device {
 	/* Receive buffer allocated by us but manages by NetVSP */
 	void *recv_buf;
 	u32 recv_buf_size; /* allocated bytes */
-	u32 recv_buf_gpadl_handle;
+	struct vmbus_gpadl recv_buf_gpadl_handle;
 	u32 recv_section_cnt;
 	u32 recv_section_size;
 	u32 recv_completion_cnt;
 
 	/* Send buffer allocated by us */
 	void *send_buf;
-	u32 send_buf_gpadl_handle;
+	u32 send_buf_size;
+	struct vmbus_gpadl send_buf_gpadl_handle;
 	u32 send_section_cnt;
 	u32 send_section_size;
 	unsigned long *send_section_map;
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index 7bd935412853..1f87e570ed2b 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -278,9 +278,9 @@ static void netvsc_teardown_recv_gpadl(struct hv_device *device,
 {
 	int ret;
 
-	if (net_device->recv_buf_gpadl_handle) {
+	if (net_device->recv_buf_gpadl_handle.gpadl_handle) {
 		ret = vmbus_teardown_gpadl(device->channel,
-					   net_device->recv_buf_gpadl_handle);
+					   &net_device->recv_buf_gpadl_handle);
 
 		/* If we failed here, we might as well return and have a leak
 		 * rather than continue and a bugchk
@@ -290,7 +290,7 @@ static void netvsc_teardown_recv_gpadl(struct hv_device *device,
 				   "unable to teardown receive buffer's gpadl\n");
 			return;
 		}
-		net_device->recv_buf_gpadl_handle = 0;
+		net_device->recv_buf_gpadl_handle.gpadl_handle = 0;
 	}
 }
 
@@ -300,9 +300,9 @@ static void netvsc_teardown_send_gpadl(struct hv_device *device,
 {
 	int ret;
 
-	if (net_device->send_buf_gpadl_handle) {
+	if (net_device->send_buf_gpadl_handle.gpadl_handle) {
 		ret = vmbus_teardown_gpadl(device->channel,
-					   net_device->send_buf_gpadl_handle);
+					   &net_device->send_buf_gpadl_handle);
 
 		/* If we failed here, we might as well return and have a leak
 		 * rather than continue and a bugchk
@@ -312,7 +312,7 @@ static void netvsc_teardown_send_gpadl(struct hv_device *device,
 				   "unable to teardown send buffer's gpadl\n");
 			return;
 		}
-		net_device->send_buf_gpadl_handle = 0;
+		net_device->send_buf_gpadl_handle.gpadl_handle = 0;
 	}
 }
 
@@ -380,7 +380,7 @@ static int netvsc_init_buf(struct hv_device *device,
 	memset(init_packet, 0, sizeof(struct nvsp_message));
 	init_packet->hdr.msg_type = NVSP_MSG1_TYPE_SEND_RECV_BUF;
 	init_packet->msg.v1_msg.send_recv_buf.
-		gpadl_handle = net_device->recv_buf_gpadl_handle;
+		gpadl_handle = net_device->recv_buf_gpadl_handle.gpadl_handle;
 	init_packet->msg.v1_msg.
 		send_recv_buf.id = NETVSC_RECEIVE_BUFFER_ID;
 
@@ -463,6 +463,7 @@ static int netvsc_init_buf(struct hv_device *device,
 		ret = -ENOMEM;
 		goto cleanup;
 	}
+	net_device->send_buf_size = buf_size;
 
 	/* Establish the gpadl handle for this buffer on this
 	 * channel.  Note: This call uses the vmbus connection rather
@@ -482,7 +483,7 @@ static int netvsc_init_buf(struct hv_device *device,
 	memset(init_packet, 0, sizeof(struct nvsp_message));
 	init_packet->hdr.msg_type = NVSP_MSG1_TYPE_SEND_SEND_BUF;
 	init_packet->msg.v1_msg.send_send_buf.gpadl_handle =
-		net_device->send_buf_gpadl_handle;
+		net_device->send_buf_gpadl_handle.gpadl_handle;
 	init_packet->msg.v1_msg.send_send_buf.id = NETVSC_SEND_BUFFER_ID;
 
 	trace_nvsp_send(ndev, init_packet);
diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c
index 652fe2547587..548243dcd895 100644
--- a/drivers/uio/uio_hv_generic.c
+++ b/drivers/uio/uio_hv_generic.c
@@ -58,11 +58,11 @@ struct hv_uio_private_data {
 	atomic_t refcnt;
 
 	void	*recv_buf;
-	u32	recv_gpadl;
+	struct vmbus_gpadl recv_gpadl;
 	char	recv_name[32];	/* "recv_4294967295" */
 
 	void	*send_buf;
-	u32	send_gpadl;
+	struct vmbus_gpadl send_gpadl;
 	char	send_name[32];
 };
 
@@ -179,15 +179,15 @@ hv_uio_new_channel(struct vmbus_channel *new_sc)
 static void
 hv_uio_cleanup(struct hv_device *dev, struct hv_uio_private_data *pdata)
 {
-	if (pdata->send_gpadl) {
-		vmbus_teardown_gpadl(dev->channel, pdata->send_gpadl);
-		pdata->send_gpadl = 0;
+	if (pdata->send_gpadl.gpadl_handle) {
+		vmbus_teardown_gpadl(dev->channel, &pdata->send_gpadl);
+		pdata->send_gpadl.gpadl_handle = 0;
 		vfree(pdata->send_buf);
 	}
 
-	if (pdata->recv_gpadl) {
-		vmbus_teardown_gpadl(dev->channel, pdata->recv_gpadl);
-		pdata->recv_gpadl = 0;
+	if (pdata->recv_gpadl.gpadl_handle) {
+		vmbus_teardown_gpadl(dev->channel, &pdata->recv_gpadl);
+		pdata->recv_gpadl.gpadl_handle = 0;
 		vfree(pdata->recv_buf);
 	}
 }
@@ -303,7 +303,7 @@ hv_uio_probe(struct hv_device *dev,
 
 	/* put Global Physical Address Label in name */
 	snprintf(pdata->recv_name, sizeof(pdata->recv_name),
-		 "recv:%u", pdata->recv_gpadl);
+		 "recv:%u", pdata->recv_gpadl.gpadl_handle);
 	pdata->info.mem[RECV_BUF_MAP].name = pdata->recv_name;
 	pdata->info.mem[RECV_BUF_MAP].addr
 		= (uintptr_t)pdata->recv_buf;
@@ -324,7 +324,7 @@ hv_uio_probe(struct hv_device *dev,
 	}
 
 	snprintf(pdata->send_name, sizeof(pdata->send_name),
-		 "send:%u", pdata->send_gpadl);
+		 "send:%u", pdata->send_gpadl.gpadl_handle);
 	pdata->info.mem[SEND_BUF_MAP].name = pdata->send_name;
 	pdata->info.mem[SEND_BUF_MAP].addr
 		= (uintptr_t)pdata->send_buf;
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index ddc8713ce57b..a9e0bc3b1511 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -803,6 +803,12 @@ struct vmbus_device {
 
 #define VMBUS_DEFAULT_MAX_PKT_SIZE 4096
 
+struct vmbus_gpadl {
+	u32 gpadl_handle;
+	u32 size;
+	void *buffer;
+};
+
 struct vmbus_channel {
 	struct list_head listentry;
 
@@ -822,7 +828,7 @@ struct vmbus_channel {
 	bool rescind_ref; /* got rescind msg, got channel reference */
 	struct completion rescind_event;
 
-	u32 ringbuffer_gpadlhandle;
+	struct vmbus_gpadl ringbuffer_gpadlhandle;
 
 	/* Allocated memory for ring buffer */
 	struct page *ringbuffer_page;
@@ -1192,10 +1198,10 @@ extern int vmbus_sendpacket_mpb_desc(struct vmbus_channel *channel,
 extern int vmbus_establish_gpadl(struct vmbus_channel *channel,
 				      void *kbuffer,
 				      u32 size,
-				      u32 *gpadl_handle);
+				      struct vmbus_gpadl *gpadl);
 
 extern int vmbus_teardown_gpadl(struct vmbus_channel *channel,
-				     u32 gpadl_handle);
+				     struct vmbus_gpadl *gpadl);
 
 void vmbus_reset_channel_cb(struct vmbus_channel *channel);
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:40:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:40:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186664.335431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8fJ-0004PE-Aq; Tue, 14 Sep 2021 13:40:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186664.335431; Tue, 14 Sep 2021 13:40:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8fJ-0004Of-7g; Tue, 14 Sep 2021 13:40:01 +0000
Received: by outflank-mailman (input) for mailman id 186664;
 Tue, 14 Sep 2021 13:39:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8fH-0001wz-Rd
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:39:59 +0000
Received: from mail-pg1-x52f.google.com (unknown [2607:f8b0:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a8072179-f310-4121-9ce6-f13ea5cbfd79;
 Tue, 14 Sep 2021 13:39:37 +0000 (UTC)
Received: by mail-pg1-x52f.google.com with SMTP id w8so12730199pgf.5
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:37 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8072179-f310-4121-9ce6-f13ea5cbfd79
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=zUkXn4sp/qQgiPEGSRnTvw2J9RtbphVDtIRuiQqhe5I=;
        b=qZtlo7fWNwe59y7IrBkgFI8skL03Pkvbnc/q8fXggTmEI9tn+Ho4t22cbkbvbJovOT
         mWtn7+Cl7NIOidFVYUEmp1/a18j9Ui2v0aZn2WlO1fIZSVhlpWJn2wLOEVZJU65hpDKS
         TbjN1BhZEYIH/Vptm0SF+Dg3jmqz1QMh7FL6rNTY+16gkezB327gktsNZG8tMQQLvlOk
         KwGOvjQ6llHrapWMeKShRAOx/cjevc0rB28Cv4aPicR1r2d3Ncwkv4mIgfHZ7oE7UyLw
         4xcNunfiZFvZIp1vvt3X8dea3yvRpEHd3BkS9EhKCO9Pv71tFLWLFB8cf8kjQ0T1R9cq
         0jjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=zUkXn4sp/qQgiPEGSRnTvw2J9RtbphVDtIRuiQqhe5I=;
        b=N6Ym97YxD5KbTwNkhhYaS0W4BRPuzd6vKqP/hve5UXhXpcMfZV5WLDicW2XyKZj5E7
         dEZQnY4hXfuWIHoDFN/h0RksKx101LUuyHoqwluu4v6cPAE++p45uTOfVwnECSrzuqm9
         9h89BDlHk5K24FVxb7lSi4Z9xfA2ebWgozzcBXGUPJTJZDEFiGTTzxmAMwOWQd1y2q+/
         iKGEhOOvtIILf4/+RXYKnBFOsiHEvHAywDITDzRcI54iNO2XIVinSJmYWng7ndKOgqKQ
         eN6X9+n2AfBFWGWMkXuYhdJePKHEy/t8sYPxejyoCh+5VFJ0rglkyzEJGgUxjQeEtdoA
         fv0w==
X-Gm-Message-State: AOAM531uHPCGyNf+3FOAejN9RgWCWLIS0yJ1NsYnZpNHHv6rswpFQL7w
	+5vjAUXzXHslZrWTiF52v+Edm4oDOdavEw==
X-Google-Smtp-Source: ABdhPJzmO9j1fgt6LuREDGB93sw+XgWPBQLWOatojtm5X4J7XEc1TUjh25t15N4hYiTqkBTF0edZnw==
X-Received: by 2002:a63:740e:: with SMTP id p14mr15442707pgc.329.1631626777043;
        Tue, 14 Sep 2021 06:39:37 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 05/12] x86/hyperv: Add Write/Read MSR registers via ghcb page
Date: Tue, 14 Sep 2021 09:39:06 -0400
Message-Id: <20210914133916.1440931-6-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

Hyperv provides GHCB protocol to write Synthetic Interrupt
Controller MSR registers in Isolation VM with AMD SEV SNP
and these registers are emulated by hypervisor directly.
Hyperv requires to write SINTx MSR registers twice. First
writes MSR via GHCB page to communicate with hypervisor
and then writes wrmsr instruction to talk with paravisor
which runs in VMPL0. Guest OS ID MSR also needs to be set
via GHCB page.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	 * Remove hv_get_simp(), hv_get_siefp()  hv_get_synint_*()
	   helper function. Move the logic into hv_get/set_register().

Change since v3:
         * Pass old_msg_type to hv_signal_eom() as parameter.
	 * Use HV_REGISTER_* marcro instead of HV_X64_MSR_*
	 * Add hv_isolation_type_snp() weak function.
	 * Add maros to set syinc register in ARM code.

Change since v1:
         * Introduce sev_es_ghcb_hv_call_simple() and share code
	   between SEV and Hyper-V code.

Fix for hyperv: Add Write/Read MSR registers via ghcb page
---
 arch/x86/hyperv/hv_init.c       |  36 +++--------
 arch/x86/hyperv/ivm.c           | 103 ++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h |  56 ++++++++++++-----
 arch/x86/include/asm/sev.h      |   6 ++
 arch/x86/kernel/sev-shared.c    |  63 +++++++++++--------
 drivers/hv/hv.c                 |  77 +++++++++++++++++++-----
 drivers/hv/hv_common.c          |   6 ++
 include/asm-generic/mshyperv.h  |   2 +
 8 files changed, 266 insertions(+), 83 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index d57df6825527..a16a83e46a30 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -37,7 +37,7 @@ EXPORT_SYMBOL_GPL(hv_current_partition_id);
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
-void __percpu **hv_ghcb_pg;
+union hv_ghcb __percpu **hv_ghcb_pg;
 
 /* Storage to save the hypercall page temporarily for hibernation */
 static void *hv_hypercall_pg_saved;
@@ -406,7 +406,7 @@ void __init hyperv_init(void)
 	}
 
 	if (hv_isolation_type_snp()) {
-		hv_ghcb_pg = alloc_percpu(void *);
+		hv_ghcb_pg = alloc_percpu(union hv_ghcb *);
 		if (!hv_ghcb_pg)
 			goto free_vp_assist_page;
 	}
@@ -424,6 +424,9 @@ void __init hyperv_init(void)
 	guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, guest_id);
 
+	/* Hyper-V requires to write guest os id via ghcb in SNP IVM. */
+	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, guest_id);
+
 	hv_hypercall_pg = __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START,
 			VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX,
 			VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
@@ -501,6 +504,7 @@ void __init hyperv_init(void)
 
 clean_guest_os_id:
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
+	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
 	cpuhp_remove_state(cpuhp);
 free_ghcb_page:
 	free_percpu(hv_ghcb_pg);
@@ -522,6 +526,7 @@ void hyperv_cleanup(void)
 
 	/* Reset our OS id */
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
+	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
 
 	/*
 	 * Reset hypercall page reference before reset the page,
@@ -592,30 +597,3 @@ bool hv_is_hyperv_initialized(void)
 	return hypercall_msr.enable;
 }
 EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
-
-enum hv_isolation_type hv_get_isolation_type(void)
-{
-	if (!(ms_hyperv.priv_high & HV_ISOLATION))
-		return HV_ISOLATION_TYPE_NONE;
-	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
-}
-EXPORT_SYMBOL_GPL(hv_get_isolation_type);
-
-bool hv_is_isolation_supported(void)
-{
-	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
-		return false;
-
-	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
-		return false;
-
-	return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE;
-}
-
-DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
-
-bool hv_isolation_type_snp(void)
-{
-	return static_branch_unlikely(&isolation_type_snp);
-}
-EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
index 79e7fb83472a..5439723446c9 100644
--- a/arch/x86/hyperv/ivm.c
+++ b/arch/x86/hyperv/ivm.c
@@ -6,12 +6,115 @@
  *  Tianyu Lan <Tianyu.Lan@microsoft.com>
  */
 
+#include <linux/types.h>
+#include <linux/bitfield.h>
 #include <linux/hyperv.h>
 #include <linux/types.h>
 #include <linux/bitfield.h>
 #include <linux/slab.h>
+#include <asm/svm.h>
+#include <asm/sev.h>
 #include <asm/io.h>
 #include <asm/mshyperv.h>
+#include <asm/hypervisor.h>
+
+union hv_ghcb {
+	struct ghcb ghcb;
+} __packed __aligned(HV_HYP_PAGE_SIZE);
+
+void hv_ghcb_msr_write(u64 msr, u64 value)
+{
+	union hv_ghcb *hv_ghcb;
+	void **ghcb_base;
+	unsigned long flags;
+
+	if (!hv_ghcb_pg)
+		return;
+
+	WARN_ON(in_nmi());
+
+	local_irq_save(flags);
+	ghcb_base = (void **)this_cpu_ptr(hv_ghcb_pg);
+	hv_ghcb = (union hv_ghcb *)*ghcb_base;
+	if (!hv_ghcb) {
+		local_irq_restore(flags);
+		return;
+	}
+
+	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
+	ghcb_set_rax(&hv_ghcb->ghcb, lower_32_bits(value));
+	ghcb_set_rdx(&hv_ghcb->ghcb, upper_32_bits(value));
+
+	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 1, 0))
+		pr_warn("Fail to write msr via ghcb %llx.\n", msr);
+
+	local_irq_restore(flags);
+}
+
+void hv_ghcb_msr_read(u64 msr, u64 *value)
+{
+	union hv_ghcb *hv_ghcb;
+	void **ghcb_base;
+	unsigned long flags;
+
+	/* Check size of union hv_ghcb here. */
+	BUILD_BUG_ON(sizeof(union hv_ghcb) != HV_HYP_PAGE_SIZE);
+
+	if (!hv_ghcb_pg)
+		return;
+
+	WARN_ON(in_nmi());
+
+	local_irq_save(flags);
+	ghcb_base = (void **)this_cpu_ptr(hv_ghcb_pg);
+	hv_ghcb = (union hv_ghcb *)*ghcb_base;
+	if (!hv_ghcb) {
+		local_irq_restore(flags);
+		return;
+	}
+
+	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
+	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 0, 0))
+		pr_warn("Fail to read msr via ghcb %llx.\n", msr);
+	else
+		*value = (u64)lower_32_bits(hv_ghcb->ghcb.save.rax)
+			| ((u64)lower_32_bits(hv_ghcb->ghcb.save.rdx) << 32);
+	local_irq_restore(flags);
+}
+
+enum hv_isolation_type hv_get_isolation_type(void)
+{
+	if (!(ms_hyperv.priv_high & HV_ISOLATION))
+		return HV_ISOLATION_TYPE_NONE;
+	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
+}
+EXPORT_SYMBOL_GPL(hv_get_isolation_type);
+
+/*
+ * hv_is_isolation_supported - Check system runs in the Hyper-V
+ * isolation VM.
+ */
+bool hv_is_isolation_supported(void)
+{
+	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
+		return false;
+
+	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
+		return false;
+
+	return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE;
+}
+
+DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
+
+/*
+ * hv_isolation_type_snp - Check system runs in the AMD SEV-SNP based
+ * isolation VM.
+ */
+bool hv_isolation_type_snp(void)
+{
+	return static_branch_unlikely(&isolation_type_snp);
+}
 
 /*
  * hv_mark_gpa_visibility - Set pages visible to host via hvcall.
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index ede440f9a1e2..165423e8b67a 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -11,25 +11,14 @@
 #include <asm/paravirt.h>
 #include <asm/mshyperv.h>
 
+union hv_ghcb;
+
 DECLARE_STATIC_KEY_FALSE(isolation_type_snp);
 
 typedef int (*hyperv_fill_flush_list_func)(
 		struct hv_guest_mapping_flush_list *flush,
 		void *data);
 
-static inline void hv_set_register(unsigned int reg, u64 value)
-{
-	wrmsrl(reg, value);
-}
-
-static inline u64 hv_get_register(unsigned int reg)
-{
-	u64 value;
-
-	rdmsrl(reg, value);
-	return value;
-}
-
 #define hv_get_raw_timer() rdtsc_ordered()
 
 void hyperv_vector_handler(struct pt_regs *regs);
@@ -41,7 +30,7 @@ extern void *hv_hypercall_pg;
 
 extern u64 hv_current_partition_id;
 
-extern void __percpu **hv_ghcb_pg;
+extern union hv_ghcb  __percpu **hv_ghcb_pg;
 
 int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
 int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
@@ -193,6 +182,8 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector,
 		struct hv_interrupt_entry *entry);
 int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
 int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible);
+void hv_ghcb_msr_write(u64 msr, u64 value);
+void hv_ghcb_msr_read(u64 msr, u64 *value);
 #else /* CONFIG_HYPERV */
 static inline void hyperv_init(void) {}
 static inline void hyperv_setup_mmu_ops(void) {}
@@ -209,9 +200,46 @@ static inline int hyperv_flush_guest_mapping_range(u64 as,
 {
 	return -1;
 }
+
+static inline void hv_ghcb_msr_write(u64 msr, u64 value) {}
+static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {}
 #endif /* CONFIG_HYPERV */
 
+static inline void hv_set_register(unsigned int reg, u64 value);
 
 #include <asm-generic/mshyperv.h>
 
+static inline bool hv_is_synic_reg(unsigned int reg)
+{
+	if ((reg >= HV_REGISTER_SCONTROL) &&
+	    (reg <= HV_REGISTER_SINT15))
+		return true;
+	return false;
+}
+
+static inline u64 hv_get_register(unsigned int reg)
+{
+	u64 value;
+
+	if (hv_is_synic_reg(reg) && hv_isolation_type_snp())
+		hv_ghcb_msr_read(reg, &value);
+	else
+		rdmsrl(reg, value);
+	return value;
+}
+
+static inline void hv_set_register(unsigned int reg, u64 value)
+{
+	if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) {
+		hv_ghcb_msr_write(reg, value);
+
+		/* Write proxy bit via wrmsl instruction */
+		if (reg >= HV_REGISTER_SINT0 &&
+		    reg <= HV_REGISTER_SINT15)
+			wrmsrl(reg, value | 1 << 20);
+	} else {
+		wrmsrl(reg, value);
+	}
+}
+
 #endif
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index fa5cd05d3b5b..60bfdbd141b1 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -81,12 +81,18 @@ static __always_inline void sev_es_nmi_complete(void)
 		__sev_es_nmi_complete();
 }
 extern int __init sev_es_efi_map_ghcbs(pgd_t *pgd);
+extern enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
+				   u64 exit_code, u64 exit_info_1,
+				   u64 exit_info_2);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
 static inline int sev_es_setup_ap_jump_table(struct real_mode_header *rmh) { return 0; }
 static inline void sev_es_nmi_complete(void) { }
 static inline int sev_es_efi_map_ghcbs(pgd_t *pgd) { return 0; }
+static inline enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
+				   u64 exit_code, u64 exit_info_1,
+				   u64 exit_info_2) { return ES_VMM_ERROR; }
 #endif
 
 #endif
diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
index 9f90f460a28c..dd7f37de640b 100644
--- a/arch/x86/kernel/sev-shared.c
+++ b/arch/x86/kernel/sev-shared.c
@@ -94,10 +94,9 @@ static void vc_finish_insn(struct es_em_ctxt *ctxt)
 	ctxt->regs->ip += ctxt->insn.length;
 }
 
-static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
-					  struct es_em_ctxt *ctxt,
-					  u64 exit_code, u64 exit_info_1,
-					  u64 exit_info_2)
+enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
+				   u64 exit_code, u64 exit_info_1,
+				   u64 exit_info_2)
 {
 	enum es_result ret;
 
@@ -109,29 +108,45 @@ static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
 	ghcb_set_sw_exit_info_1(ghcb, exit_info_1);
 	ghcb_set_sw_exit_info_2(ghcb, exit_info_2);
 
-	sev_es_wr_ghcb_msr(__pa(ghcb));
 	VMGEXIT();
 
-	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) == 1) {
-		u64 info = ghcb->save.sw_exit_info_2;
-		unsigned long v;
-
-		info = ghcb->save.sw_exit_info_2;
-		v = info & SVM_EVTINJ_VEC_MASK;
-
-		/* Check if exception information from hypervisor is sane. */
-		if ((info & SVM_EVTINJ_VALID) &&
-		    ((v == X86_TRAP_GP) || (v == X86_TRAP_UD)) &&
-		    ((info & SVM_EVTINJ_TYPE_MASK) == SVM_EVTINJ_TYPE_EXEPT)) {
-			ctxt->fi.vector = v;
-			if (info & SVM_EVTINJ_VALID_ERR)
-				ctxt->fi.error_code = info >> 32;
-			ret = ES_EXCEPTION;
-		} else {
-			ret = ES_VMM_ERROR;
-		}
-	} else {
+	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) == 1)
+		ret = ES_VMM_ERROR;
+	else
 		ret = ES_OK;
+
+	return ret;
+}
+
+static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
+				   struct es_em_ctxt *ctxt,
+				   u64 exit_code, u64 exit_info_1,
+				   u64 exit_info_2)
+{
+	unsigned long v;
+	enum es_result ret;
+	u64 info;
+
+	sev_es_wr_ghcb_msr(__pa(ghcb));
+
+	ret = sev_es_ghcb_hv_call_simple(ghcb, exit_code, exit_info_1,
+					 exit_info_2);
+	if (ret == ES_OK)
+		return ret;
+
+	info = ghcb->save.sw_exit_info_2;
+	v = info & SVM_EVTINJ_VEC_MASK;
+
+	/* Check if exception information from hypervisor is sane. */
+	if ((info & SVM_EVTINJ_VALID) &&
+	    ((v == X86_TRAP_GP) || (v == X86_TRAP_UD)) &&
+	    ((info & SVM_EVTINJ_TYPE_MASK) == SVM_EVTINJ_TYPE_EXEPT)) {
+		ctxt->fi.vector = v;
+		if (info & SVM_EVTINJ_VALID_ERR)
+			ctxt->fi.error_code = info >> 32;
+		ret = ES_EXCEPTION;
+	} else {
+		ret = ES_VMM_ERROR;
 	}
 
 	return ret;
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index e83507f49676..dee1a96bc535 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -8,6 +8,7 @@
  */
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/io.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/slab.h>
@@ -136,17 +137,24 @@ int hv_synic_alloc(void)
 		tasklet_init(&hv_cpu->msg_dpc,
 			     vmbus_on_msg_dpc, (unsigned long) hv_cpu);
 
-		hv_cpu->synic_message_page =
-			(void *)get_zeroed_page(GFP_ATOMIC);
-		if (hv_cpu->synic_message_page == NULL) {
-			pr_err("Unable to allocate SYNIC message page\n");
-			goto err;
-		}
+		/*
+		 * Synic message and event pages are allocated by paravisor.
+		 * Skip these pages allocation here.
+		 */
+		if (!hv_isolation_type_snp()) {
+			hv_cpu->synic_message_page =
+				(void *)get_zeroed_page(GFP_ATOMIC);
+			if (hv_cpu->synic_message_page == NULL) {
+				pr_err("Unable to allocate SYNIC message page\n");
+				goto err;
+			}
 
-		hv_cpu->synic_event_page = (void *)get_zeroed_page(GFP_ATOMIC);
-		if (hv_cpu->synic_event_page == NULL) {
-			pr_err("Unable to allocate SYNIC event page\n");
-			goto err;
+			hv_cpu->synic_event_page =
+				(void *)get_zeroed_page(GFP_ATOMIC);
+			if (hv_cpu->synic_event_page == NULL) {
+				pr_err("Unable to allocate SYNIC event page\n");
+				goto err;
+			}
 		}
 
 		hv_cpu->post_msg_page = (void *)get_zeroed_page(GFP_ATOMIC);
@@ -201,16 +209,35 @@ void hv_synic_enable_regs(unsigned int cpu)
 	/* Setup the Synic's message page */
 	simp.as_uint64 = hv_get_register(HV_REGISTER_SIMP);
 	simp.simp_enabled = 1;
-	simp.base_simp_gpa = virt_to_phys(hv_cpu->synic_message_page)
-		>> HV_HYP_PAGE_SHIFT;
+
+	if (hv_isolation_type_snp()) {
+		hv_cpu->synic_message_page
+			= memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT,
+				   HV_HYP_PAGE_SIZE, MEMREMAP_WB);
+		if (!hv_cpu->synic_message_page)
+			pr_err("Fail to map syinc message page.\n");
+	} else {
+		simp.base_simp_gpa = virt_to_phys(hv_cpu->synic_message_page)
+			>> HV_HYP_PAGE_SHIFT;
+	}
 
 	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
 
 	/* Setup the Synic's event page */
 	siefp.as_uint64 = hv_get_register(HV_REGISTER_SIEFP);
 	siefp.siefp_enabled = 1;
-	siefp.base_siefp_gpa = virt_to_phys(hv_cpu->synic_event_page)
-		>> HV_HYP_PAGE_SHIFT;
+
+	if (hv_isolation_type_snp()) {
+		hv_cpu->synic_event_page =
+			memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT,
+				 HV_HYP_PAGE_SIZE, MEMREMAP_WB);
+
+		if (!hv_cpu->synic_event_page)
+			pr_err("Fail to map syinc event page.\n");
+	} else {
+		siefp.base_siefp_gpa = virt_to_phys(hv_cpu->synic_event_page)
+			>> HV_HYP_PAGE_SHIFT;
+	}
 
 	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
 
@@ -257,30 +284,48 @@ int hv_synic_init(unsigned int cpu)
  */
 void hv_synic_disable_regs(unsigned int cpu)
 {
+	struct hv_per_cpu_context *hv_cpu
+		= per_cpu_ptr(hv_context.cpu_context, cpu);
 	union hv_synic_sint shared_sint;
 	union hv_synic_simp simp;
 	union hv_synic_siefp siefp;
 	union hv_synic_scontrol sctrl;
 
+
 	shared_sint.as_uint64 = hv_get_register(HV_REGISTER_SINT0 +
 					VMBUS_MESSAGE_SINT);
 
 	shared_sint.masked = 1;
 
+
+
 	/* Need to correctly cleanup in the case of SMP!!! */
 	/* Disable the interrupt */
 	hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT,
 				shared_sint.as_uint64);
 
 	simp.as_uint64 = hv_get_register(HV_REGISTER_SIMP);
+	/*
+	 * In Isolation VM, sim and sief pages are allocated by
+	 * paravisor. These pages also will be used by kdump
+	 * kernel. So just reset enable bit here and keep page
+	 * addresses.
+	 */
 	simp.simp_enabled = 0;
-	simp.base_simp_gpa = 0;
+	if (hv_isolation_type_snp())
+		memunmap(hv_cpu->synic_message_page);
+	else
+		simp.base_simp_gpa = 0;
 
 	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
 
 	siefp.as_uint64 = hv_get_register(HV_REGISTER_SIEFP);
 	siefp.siefp_enabled = 0;
-	siefp.base_siefp_gpa = 0;
+
+	if (hv_isolation_type_snp())
+		memunmap(hv_cpu->synic_event_page);
+	else
+		siefp.base_siefp_gpa = 0;
 
 	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
 
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index c0d9048a4112..1fc82d237161 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -249,6 +249,12 @@ bool __weak hv_is_isolation_supported(void)
 }
 EXPORT_SYMBOL_GPL(hv_is_isolation_supported);
 
+bool __weak hv_isolation_type_snp(void)
+{
+	return false;
+}
+EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
+
 void __weak hv_setup_vmbus_handler(void (*handler)(void))
 {
 }
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index cb529c85c0ad..94750bafd4cc 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -24,6 +24,7 @@
 #include <linux/cpumask.h>
 #include <linux/nmi.h>
 #include <asm/ptrace.h>
+#include <asm/mshyperv.h>
 #include <asm/hyperv-tlfs.h>
 
 struct ms_hyperv_info {
@@ -54,6 +55,7 @@ extern void  __percpu  **hyperv_pcpu_output_arg;
 
 extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr);
 extern u64 hv_do_fast_hypercall8(u16 control, u64 input8);
+extern bool hv_isolation_type_snp(void);
 
 /* Helper functions that provide a consistent pattern for checking Hyper-V hypercall status. */
 static inline int hv_result(u64 status)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:40:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:40:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186671.335442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8fS-0005ht-QQ; Tue, 14 Sep 2021 13:40:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186671.335442; Tue, 14 Sep 2021 13:40:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8fS-0005hg-N6; Tue, 14 Sep 2021 13:40:10 +0000
Received: by outflank-mailman (input) for mailman id 186671;
 Tue, 14 Sep 2021 13:40:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8fR-0001wz-SB
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:09 +0000
Received: from mail-pj1-x102f.google.com (unknown [2607:f8b0:4864:20::102f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 25a0bd9d-8c5d-4bd4-8d96-9a0e7cc43ac6;
 Tue, 14 Sep 2021 13:39:40 +0000 (UTC)
Received: by mail-pj1-x102f.google.com with SMTP id
 lb1-20020a17090b4a4100b001993f863df2so2799919pjb.5
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:40 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25a0bd9d-8c5d-4bd4-8d96-9a0e7cc43ac6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=azCewkuzRAkBZyiDOjmA0vSW/QH/3obR4Ytj+SPfQzQ=;
        b=YK5mVlypHJRSS7afMsq7DyxTNlQ/p9SKLwx+cOOkVvxOGeuM6eWoauMwXmPc1oqD+h
         8e/7mt+/7qYq08phSAIbvSPhlho5l6bHdYmjrjK0P59fIP9AAVx6da+9UhivbazZeao7
         hkI2XuYefI+84XSLv5sDvueSTlukH2kXJ0Bo0iKwh+5F/fDZNQK+wmNjmUUu8Uuiu0ht
         b+s3OSft1wHLaAepa21DDSmakrwSRefK8GEX0khdKFbF/DlzqOOFmYHIZBetHsCFHxn8
         r46AcAy8J11/ApUlIPT+c5gLWcT9l6OhimerwJRDl7pJyHQqUxUDREPhPiA5a1Miq6DQ
         Eslw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=azCewkuzRAkBZyiDOjmA0vSW/QH/3obR4Ytj+SPfQzQ=;
        b=V+HuCxcKoiIzqmMhXv6qTOn2IWoBbB3AcDqWD2yP3YGzcK1D8m18HTK9T3fFkkrwr4
         KrQfrxwEMxxS2VfN9wQv6ygr43TpKi8vZjJZGyfqCpfFUnxqVBJ+O+n1xSpyiAS3d0OL
         FnJr4DFVBmXa4Ka/9IwpMXyV9azqGzktQp/99L3Aw9nG8wfu+qrheEGrnB0GGVloAfVZ
         +5gdvQAfKVkLF6Fl8PZME6YSUF3udC0bOMKWk9/Zbz4VYTtJIC59atKJuDYy5jbCxbrE
         hzIBVP6e68A4RVj6wGatA39kq+Znxn3QWd6FZBNcjFKLBvX56evf1X9QK2brA8ESYZSl
         8O+w==
X-Gm-Message-State: AOAM533Z93PB4UOEX4Aab50Ri1eh4r7NKGDzFMNjoKa09vu2nGC78Spa
	l6kip0yXQsRFRda1UX2PoTs=
X-Google-Smtp-Source: ABdhPJztnF9gFB+MKR5DdaKIrIGDyLZ4ZBz+QRP7UkoIajfwGN+xMzTLqQUKzjlgKwa664+jV4M3Pw==
X-Received: by 2002:a17:903:1207:b0:138:e2f9:6c98 with SMTP id l7-20020a170903120700b00138e2f96c98mr15512885plh.11.1631626779382;
        Tue, 14 Sep 2021 06:39:39 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 06/12] x86/hyperv: Add ghcb hvcall support for SNP VM
Date: Tue, 14 Sep 2021 09:39:07 -0400
Message-Id: <20210914133916.1440931-7-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

hyperv provides ghcb hvcall to handle VMBus
HVCALL_SIGNAL_EVENT and HVCALL_POST_MESSAGE
msg in SNP Isolation VM. Add such support.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v3:
	* Add hv_ghcb_hypercall() stub function to avoid
	  compile error for ARM.
---
 arch/x86/hyperv/ivm.c          | 74 ++++++++++++++++++++++++++++++++++
 drivers/hv/connection.c        |  6 ++-
 drivers/hv/hv.c                |  8 +++-
 drivers/hv/hv_common.c         |  6 +++
 include/asm-generic/mshyperv.h |  1 +
 5 files changed, 93 insertions(+), 2 deletions(-)

diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
index 5439723446c9..dfdac3a40036 100644
--- a/arch/x86/hyperv/ivm.c
+++ b/arch/x86/hyperv/ivm.c
@@ -18,10 +18,84 @@
 #include <asm/mshyperv.h>
 #include <asm/hypervisor.h>
 
+#define GHCB_USAGE_HYPERV_CALL	1
+
 union hv_ghcb {
 	struct ghcb ghcb;
+	struct {
+		u64 hypercalldata[509];
+		u64 outputgpa;
+		union {
+			union {
+				struct {
+					u32 callcode        : 16;
+					u32 isfast          : 1;
+					u32 reserved1       : 14;
+					u32 isnested        : 1;
+					u32 countofelements : 12;
+					u32 reserved2       : 4;
+					u32 repstartindex   : 12;
+					u32 reserved3       : 4;
+				};
+				u64 asuint64;
+			} hypercallinput;
+			union {
+				struct {
+					u16 callstatus;
+					u16 reserved1;
+					u32 elementsprocessed : 12;
+					u32 reserved2         : 20;
+				};
+				u64 asunit64;
+			} hypercalloutput;
+		};
+		u64 reserved2;
+	} hypercall;
 } __packed __aligned(HV_HYP_PAGE_SIZE);
 
+u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_size)
+{
+	union hv_ghcb *hv_ghcb;
+	void **ghcb_base;
+	unsigned long flags;
+	u64 status;
+
+	if (!hv_ghcb_pg)
+		return -EFAULT;
+
+	WARN_ON(in_nmi());
+
+	local_irq_save(flags);
+	ghcb_base = (void **)this_cpu_ptr(hv_ghcb_pg);
+	hv_ghcb = (union hv_ghcb *)*ghcb_base;
+	if (!hv_ghcb) {
+		local_irq_restore(flags);
+		return -EFAULT;
+	}
+
+	hv_ghcb->ghcb.protocol_version = GHCB_PROTOCOL_MAX;
+	hv_ghcb->ghcb.ghcb_usage = GHCB_USAGE_HYPERV_CALL;
+
+	hv_ghcb->hypercall.outputgpa = (u64)output;
+	hv_ghcb->hypercall.hypercallinput.asuint64 = 0;
+	hv_ghcb->hypercall.hypercallinput.callcode = control;
+
+	if (input_size)
+		memcpy(hv_ghcb->hypercall.hypercalldata, input, input_size);
+
+	VMGEXIT();
+
+	hv_ghcb->ghcb.ghcb_usage = 0xffffffff;
+	memset(hv_ghcb->ghcb.save.valid_bitmap, 0,
+	       sizeof(hv_ghcb->ghcb.save.valid_bitmap));
+
+	status = hv_ghcb->hypercall.hypercalloutput.callstatus;
+
+	local_irq_restore(flags);
+
+	return status;
+}
+
 void hv_ghcb_msr_write(u64 msr, u64 value)
 {
 	union hv_ghcb *hv_ghcb;
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 5e479d54918c..8820ae68f20f 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -447,6 +447,10 @@ void vmbus_set_event(struct vmbus_channel *channel)
 
 	++channel->sig_events;
 
-	hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT, channel->sig_event);
+	if (hv_isolation_type_snp())
+		hv_ghcb_hypercall(HVCALL_SIGNAL_EVENT, &channel->sig_event,
+				NULL, sizeof(channel->sig_event));
+	else
+		hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT, channel->sig_event);
 }
 EXPORT_SYMBOL_GPL(vmbus_set_event);
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index dee1a96bc535..5644ba2bfa5c 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -98,7 +98,13 @@ int hv_post_message(union hv_connection_id connection_id,
 	aligned_msg->payload_size = payload_size;
 	memcpy((void *)aligned_msg->payload, payload, payload_size);
 
-	status = hv_do_hypercall(HVCALL_POST_MESSAGE, aligned_msg, NULL);
+	if (hv_isolation_type_snp())
+		status = hv_ghcb_hypercall(HVCALL_POST_MESSAGE,
+				(void *)aligned_msg, NULL,
+				sizeof(*aligned_msg));
+	else
+		status = hv_do_hypercall(HVCALL_POST_MESSAGE,
+				aligned_msg, NULL);
 
 	/* Preemption must remain disabled until after the hypercall
 	 * so some other thread can't get scheduled onto this cpu and
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index 1fc82d237161..7be173a99f27 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -289,3 +289,9 @@ void __weak hyperv_cleanup(void)
 {
 }
 EXPORT_SYMBOL_GPL(hyperv_cleanup);
+
+u64 __weak hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_size)
+{
+	return HV_STATUS_INVALID_PARAMETER;
+}
+EXPORT_SYMBOL_GPL(hv_ghcb_hypercall);
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index 94750bafd4cc..a0ec607a2fd6 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -250,6 +250,7 @@ bool hv_is_hibernation_supported(void);
 enum hv_isolation_type hv_get_isolation_type(void);
 bool hv_is_isolation_supported(void);
 bool hv_isolation_type_snp(void);
+u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_size);
 void hyperv_cleanup(void);
 bool hv_query_ext_cap(u64 cap_query);
 #else /* CONFIG_HYPERV */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:47:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:47:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186687.335452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8m5-0007Al-It; Tue, 14 Sep 2021 13:47:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186687.335452; Tue, 14 Sep 2021 13:47:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8m5-0007Ae-FD; Tue, 14 Sep 2021 13:47:01 +0000
Received: by outflank-mailman (input) for mailman id 186687;
 Tue, 14 Sep 2021 13:46:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8fl-0001wz-So
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:29 +0000
Received: from mail-pg1-x532.google.com (unknown [2607:f8b0:4864:20::532])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 41ee5e18-fc5d-4791-9e59-19ee5069d3d7;
 Tue, 14 Sep 2021 13:39:42 +0000 (UTC)
Received: by mail-pg1-x532.google.com with SMTP id r2so12725885pgl.10
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:42 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41ee5e18-fc5d-4791-9e59-19ee5069d3d7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=04R/nF7ZQn0L4lmwG9K7NZhI1RIoqJtAembHuZpzcoE=;
        b=cXWjfh70bA6/tTG3cRi7I0j1g+p2Lm3CJSGXyCEtns1z4ew1JMJSouaTXxuUkcmAAF
         pD2ztmNCM/EaSPUeDDaq6UxsWFBxK9H35yFnlW25xXgoMBjP2i0pScCK83QZEclf8ecy
         N9PLUvbIXVRHq4/y1x9TK184cTltER/wXcMXRfCw3qyn5rb59Xp4R9JHz9uej2V7BUzJ
         4nbgXuO4P+lxZbUEZEs/1u05P/Zm7T9ezY8W5GtsXzTaY96NJ/PhksjnQl9x9HnadNWx
         cTQCObr34jF7frx/WSelSqFMtM9MhrWIvfcA85EYhYphGuBONqJDe9RZDWb/6vKzX5CK
         TR2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=04R/nF7ZQn0L4lmwG9K7NZhI1RIoqJtAembHuZpzcoE=;
        b=xbK8sNR9lALL8t5RW+5eQocFepqs38aoTNCOArRHMJ0iAZ6P6kGf7R2dWMjz24/xyA
         ufV0fLfKhh++ooFMDNc0/RyzwTeu7BirQrIWPYeO6JGIwKNtEafubd1o0vM9xSeRUjCK
         FRszI4p9bdu4yyDJoMQylsWRhYo/mlSJxDWM+yGehO3QBvASTgEPB2tPoRLWpEIPky3F
         ZOldbAUqX7AduFcxJ+Wl8XlHHzDpIX9NFEyIjQ9Hh9wnW+mw0G93TDtRzxRcHMIT3orF
         cSFW7Rcpwa0MllldUiTf+hPA6AStY+TLzyevOZhJWNkBWyyprRF3Nu+VRyjJiKyAYl4d
         33wQ==
X-Gm-Message-State: AOAM532eQmKRMwQiAyCqkWL3VlA5a6Q14o3mhW4G8aLk3Eb5aQkBC6Ub
	KWmXkOinW+LjJwuZDhg6vv0=
X-Google-Smtp-Source: ABdhPJyVP18hjlg38ErK5oPFLiCLyC2tXuvAHanGQz+Hb5O3TLtyTYhzBvzQ3suhx9t9fxq63nkGDQ==
X-Received: by 2002:a63:7112:: with SMTP id m18mr15630957pgc.93.1631626782143;
        Tue, 14 Sep 2021 06:39:42 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 07/12] Drivers: hv: vmbus: Add SNP support for VMbus channel initiate  message
Date: Tue, 14 Sep 2021 09:39:08 -0400
Message-Id: <20210914133916.1440931-8-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

The monitor pages in the CHANNELMSG_INITIATE_CONTACT msg are shared
with host in Isolation VM and so it's necessary to use hvcall to set
them visible to host. In Isolation VM with AMD SEV SNP, the access
address should be in the extra space which is above shared gpa
boundary. So remap these pages into the extra address(pa +
shared_gpa_boundary).

Introduce monitor_pages_original[] in the struct vmbus_connection
to store monitor page virtual address returned by hv_alloc_hyperv_
zeroed_page() and free monitor page via monitor_pages_original in
the vmbus_disconnect(). The monitor_pages[] is to used to access
monitor page and it is initialized to be equal with monitor_pages_
original. The monitor_pages[] will be overridden in the isolation VM
with va of extra address. Introduce monitor_pages_pa[] to store
monitor pages' physical address and use it to populate pa in the
initiate msg.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Introduce monitor_pages_pa[] to store monitor pages' physical
	  address and use it to populate pa in the initiate msg.
	* Move code of mapping moniter pages in extra address into
	  vmbus_connect().

Change since v3:
	* Rename monitor_pages_va with monitor_pages_original
	* free monitor page via monitor_pages_original and
	  monitor_pages is used to access monitor page.

Change since v1:
        * Not remap monitor pages in the non-SNP isolation VM.
---
 drivers/hv/connection.c   | 90 ++++++++++++++++++++++++++++++++++++---
 drivers/hv/hyperv_vmbus.h |  2 +
 2 files changed, 86 insertions(+), 6 deletions(-)

diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 8820ae68f20f..edd8f7dd169f 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -19,6 +19,8 @@
 #include <linux/vmalloc.h>
 #include <linux/hyperv.h>
 #include <linux/export.h>
+#include <linux/io.h>
+#include <linux/set_memory.h>
 #include <asm/mshyperv.h>
 
 #include "hyperv_vmbus.h"
@@ -102,8 +104,9 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
 		vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID;
 	}
 
-	msg->monitor_page1 = virt_to_phys(vmbus_connection.monitor_pages[0]);
-	msg->monitor_page2 = virt_to_phys(vmbus_connection.monitor_pages[1]);
+	msg->monitor_page1 = vmbus_connection.monitor_pages_pa[0];
+	msg->monitor_page2 = vmbus_connection.monitor_pages_pa[1];
+
 	msg->target_vcpu = hv_cpu_number_to_vp_number(VMBUS_CONNECT_CPU);
 
 	/*
@@ -216,6 +219,65 @@ int vmbus_connect(void)
 		goto cleanup;
 	}
 
+	vmbus_connection.monitor_pages_original[0]
+		= vmbus_connection.monitor_pages[0];
+	vmbus_connection.monitor_pages_original[1]
+		= vmbus_connection.monitor_pages[1];
+	vmbus_connection.monitor_pages_pa[0]
+		= virt_to_phys(vmbus_connection.monitor_pages[0]);
+	vmbus_connection.monitor_pages_pa[1]
+		= virt_to_phys(vmbus_connection.monitor_pages[1]);
+
+	if (hv_is_isolation_supported()) {
+		vmbus_connection.monitor_pages_pa[0] +=
+			ms_hyperv.shared_gpa_boundary;
+		vmbus_connection.monitor_pages_pa[1] +=
+			ms_hyperv.shared_gpa_boundary;
+
+		ret = set_memory_decrypted((unsigned long)
+					   vmbus_connection.monitor_pages[0],
+					   1);
+		ret |= set_memory_decrypted((unsigned long)
+					    vmbus_connection.monitor_pages[1],
+					    1);
+		if (ret)
+			goto cleanup;
+
+		/*
+		 * Isolation VM with AMD SNP needs to access monitor page via
+		 * address space above shared gpa boundary.
+		 */
+		if (hv_isolation_type_snp()) {
+			vmbus_connection.monitor_pages[0]
+				= memremap(vmbus_connection.monitor_pages_pa[0],
+					   HV_HYP_PAGE_SIZE,
+					   MEMREMAP_WB);
+			if (!vmbus_connection.monitor_pages[0]) {
+				ret = -ENOMEM;
+				goto cleanup;
+			}
+
+			vmbus_connection.monitor_pages[1]
+				= memremap(vmbus_connection.monitor_pages_pa[1],
+					   HV_HYP_PAGE_SIZE,
+					   MEMREMAP_WB);
+			if (!vmbus_connection.monitor_pages[1]) {
+				ret = -ENOMEM;
+				goto cleanup;
+			}
+		}
+
+		/*
+		 * Set memory host visibility hvcall smears memory
+		 * and so zero monitor pages here.
+		 */
+		memset(vmbus_connection.monitor_pages[0], 0x00,
+		       HV_HYP_PAGE_SIZE);
+		memset(vmbus_connection.monitor_pages[1], 0x00,
+		       HV_HYP_PAGE_SIZE);
+
+	}
+
 	msginfo = kzalloc(sizeof(*msginfo) +
 			  sizeof(struct vmbus_channel_initiate_contact),
 			  GFP_KERNEL);
@@ -303,10 +365,26 @@ void vmbus_disconnect(void)
 		vmbus_connection.int_page = NULL;
 	}
 
-	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[0]);
-	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[1]);
-	vmbus_connection.monitor_pages[0] = NULL;
-	vmbus_connection.monitor_pages[1] = NULL;
+	if (hv_is_isolation_supported()) {
+		memunmap(vmbus_connection.monitor_pages[0]);
+		memunmap(vmbus_connection.monitor_pages[1]);
+
+		set_memory_encrypted((unsigned long)
+			vmbus_connection.monitor_pages_original[0],
+			1);
+		set_memory_encrypted((unsigned long)
+			vmbus_connection.monitor_pages_original[1],
+			1);
+	}
+
+	hv_free_hyperv_page((unsigned long)
+		vmbus_connection.monitor_pages_original[0]);
+	hv_free_hyperv_page((unsigned long)
+		vmbus_connection.monitor_pages_original[1]);
+	vmbus_connection.monitor_pages_original[0] =
+		vmbus_connection.monitor_pages[0] = NULL;
+	vmbus_connection.monitor_pages_original[1] =
+		vmbus_connection.monitor_pages[1] = NULL;
 }
 
 /*
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 42f3d9d123a1..560cba916d1d 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -240,6 +240,8 @@ struct vmbus_connection {
 	 * is child->parent notification
 	 */
 	struct hv_monitor_page *monitor_pages[2];
+	void *monitor_pages_original[2];
+	unsigned long monitor_pages_pa[2];
 	struct list_head chn_msg_list;
 	spinlock_t channelmsg_lock;
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:47:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:47:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186698.335464 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mG-0007Yw-RX; Tue, 14 Sep 2021 13:47:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186698.335464; Tue, 14 Sep 2021 13:47:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mG-0007YE-OE; Tue, 14 Sep 2021 13:47:12 +0000
Received: by outflank-mailman (input) for mailman id 186698;
 Tue, 14 Sep 2021 13:47:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8gP-0001wz-Tz
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:41:09 +0000
Received: from mail-pl1-x630.google.com (unknown [2607:f8b0:4864:20::630])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e1b20c67-e05b-4253-910b-5c3d6b9844ff;
 Tue, 14 Sep 2021 13:39:54 +0000 (UTC)
Received: by mail-pl1-x630.google.com with SMTP id n18so8199569plp.7
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:54 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1b20c67-e05b-4253-910b-5c3d6b9844ff
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=LkmXj6KxjUvBaz8z0a8dNOFjY1uabCCopAH/jkVX7K0=;
        b=H0oAe82GjXdTdE+tPELa3975j1MGt/XfBwybJVWm9rXqcL3sZBlP4N3dVxqejuMmCR
         fTxH2exCC64AO/wTOZvdNd7kolrZ/irkjYbpTtFRFS52HpEJiYpKx7tvDB22KmzcCOv7
         c9DS1KJW86b06AqUV3XvHCJsPD24m0nR2St4Gm7JJGLUJisEPWa/XBbIzSmSmqfr13m7
         8k2iUw5lq4EvEwGop+B0nMUmtEgs3N2LwGcdbL1UtdAgaL9kn7ZyWDVSGWcSLQCnunTP
         f35mOoI/LXthKpV4WOGUGrXrLzkkUj0AyvgnRnOwf8rvVBG3C6mESM16QWBrymuP71V7
         1ddg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=LkmXj6KxjUvBaz8z0a8dNOFjY1uabCCopAH/jkVX7K0=;
        b=l1cv9P85CrKZMgyqGgPGyY2W8XxgU25LdyxOpOvSA9KjsxoAyAMOX+eycZ1NOpGhz1
         f4ZXdJTqvq7VDTIo+Jh2k6zObUYMwZd9CAsVncwVy62rZne8UsgE6s+1TiFA7JWV4dh3
         HDAZlT17ymCSi6dJSnGjH4+8K6G3SrwNsTOXtE5Y652Fct08jhbwXMOGESCBCthx+Eu5
         nZDbUYrnpIPJuZWhWditkTf4XeCCxY0f9BjnyvpQwua2e0qerZOTovyF22fBWm6H3CIt
         nqm8SiLe4kBb23mhrhuwGiNYoumKF0JHzdud+0u3Qk1vjOpeVkPDdfHvlGO5Bo1aqkm3
         VGDA==
X-Gm-Message-State: AOAM530fYqm9o5JpJnHB042fGb6LgulULOAgkMxNj1tD8cGOpEvDSjH9
	x86P9Y1l/I7HwAPGNIS+elI=
X-Google-Smtp-Source: ABdhPJzd50JbAXqZtpTD7xpue98Virh8fDhY7VdwfyRuohNOtMS2fLTQ6Ka6cqdkhWO9Ql38QPYNrQ==
X-Received: by 2002:a17:902:da89:b0:13b:7d3d:59e9 with SMTP id j9-20020a170902da8900b0013b7d3d59e9mr14413893plx.41.1631626794081;
        Tue, 14 Sep 2021 06:39:54 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc driver
Date: Tue, 14 Sep 2021 09:39:13 -0400
Message-Id: <20210914133916.1440931-13-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

In Isolation VM, all shared memory with host needs to mark visible
to host via hvcall. vmbus_establish_gpadl() has already done it for
netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
pagebuffer() stills need to be handled. Use DMA API to map/umap
these memory during sending/receiving packet and Hyper-V swiotlb
bounce buffer dma address will be returned. The swiotlb bounce buffer
has been masked to be visible to host during boot up.

Allocate rx/tx ring buffer via alloc_pages() in Isolation VM and map
these pages via vmap(). After calling vmbus_establish_gpadl() which
marks these pages visible to host, unmap these pages to release the
virtual address mapped with physical address below shared_gpa_boundary
and map them in the extra address space via vmap_pfn().

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Allocate rx/tx ring buffer via alloc_pages() in Isolation VM
	* Map pages after calling vmbus_establish_gpadl().
	* set dma_set_min_align_mask for netvsc driver.

Change since v3:
	* Add comment to explain why not to use dma_map_sg()
	* Fix some error handle.
---
 drivers/net/hyperv/hyperv_net.h   |   7 +
 drivers/net/hyperv/netvsc.c       | 287 +++++++++++++++++++++++++++++-
 drivers/net/hyperv/netvsc_drv.c   |   1 +
 drivers/net/hyperv/rndis_filter.c |   2 +
 include/linux/hyperv.h            |   5 +
 5 files changed, 296 insertions(+), 6 deletions(-)

diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index 315278a7cf88..87e8c74398a5 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -164,6 +164,7 @@ struct hv_netvsc_packet {
 	u32 total_bytes;
 	u32 send_buf_index;
 	u32 total_data_buflen;
+	struct hv_dma_range *dma_range;
 };
 
 #define NETVSC_HASH_KEYLEN 40
@@ -1074,6 +1075,8 @@ struct netvsc_device {
 
 	/* Receive buffer allocated by us but manages by NetVSP */
 	void *recv_buf;
+	struct page **recv_pages;
+	u32 recv_page_count;
 	u32 recv_buf_size; /* allocated bytes */
 	struct vmbus_gpadl recv_buf_gpadl_handle;
 	u32 recv_section_cnt;
@@ -1082,6 +1085,8 @@ struct netvsc_device {
 
 	/* Send buffer allocated by us */
 	void *send_buf;
+	struct page **send_pages;
+	u32 send_page_count;
 	u32 send_buf_size;
 	struct vmbus_gpadl send_buf_gpadl_handle;
 	u32 send_section_cnt;
@@ -1731,4 +1736,6 @@ struct rndis_message {
 #define RETRY_US_HI	10000
 #define RETRY_MAX	2000	/* >10 sec */
 
+void netvsc_dma_unmap(struct hv_device *hv_dev,
+		      struct hv_netvsc_packet *packet);
 #endif /* _HYPERV_NET_H */
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index 1f87e570ed2b..7d5254bf043e 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -20,6 +20,7 @@
 #include <linux/vmalloc.h>
 #include <linux/rtnetlink.h>
 #include <linux/prefetch.h>
+#include <linux/gfp.h>
 
 #include <asm/sync_bitops.h>
 #include <asm/mshyperv.h>
@@ -150,11 +151,33 @@ static void free_netvsc_device(struct rcu_head *head)
 {
 	struct netvsc_device *nvdev
 		= container_of(head, struct netvsc_device, rcu);
+	unsigned int alloc_unit;
 	int i;
 
 	kfree(nvdev->extension);
-	vfree(nvdev->recv_buf);
-	vfree(nvdev->send_buf);
+
+	if (nvdev->recv_pages) {
+		alloc_unit = (nvdev->recv_buf_size /
+			nvdev->recv_page_count) >> PAGE_SHIFT;
+
+		vunmap(nvdev->recv_buf);
+		for (i = 0; i < nvdev->recv_page_count; i++)
+			__free_pages(nvdev->recv_pages[i], alloc_unit);
+	} else {
+		vfree(nvdev->recv_buf);
+	}
+
+	if (nvdev->send_pages) {
+		alloc_unit = (nvdev->send_buf_size /
+			nvdev->send_page_count) >> PAGE_SHIFT;
+
+		vunmap(nvdev->send_buf);
+		for (i = 0; i < nvdev->send_page_count; i++)
+			__free_pages(nvdev->send_pages[i], alloc_unit);
+	} else {
+		vfree(nvdev->send_buf);
+	}
+
 	kfree(nvdev->send_section_map);
 
 	for (i = 0; i < VRSS_CHANNEL_MAX; i++) {
@@ -330,6 +353,108 @@ int netvsc_alloc_recv_comp_ring(struct netvsc_device *net_device, u32 q_idx)
 	return nvchan->mrc.slots ? 0 : -ENOMEM;
 }
 
+void *netvsc_alloc_pages(struct page ***pages_array, unsigned int *array_len,
+			 unsigned long size)
+{
+	struct page *page, **pages, **vmap_pages;
+	unsigned long pg_count = size >> PAGE_SHIFT;
+	int alloc_unit = MAX_ORDER_NR_PAGES;
+	int i, j, vmap_page_index = 0;
+	void *vaddr;
+
+	if (pg_count < alloc_unit)
+		alloc_unit = 1;
+
+	/* vmap() accepts page array with PAGE_SIZE as unit while try to
+	 * allocate high order pages here in order to save page array space.
+	 * vmap_pages[] is used as input parameter of vmap(). pages[] is to
+	 * store allocated pages and map them later.
+	 */
+	vmap_pages = kmalloc_array(pg_count, sizeof(*vmap_pages), GFP_KERNEL);
+	if (!vmap_pages)
+		return NULL;
+
+retry:
+	*array_len = pg_count / alloc_unit;
+	pages = kmalloc_array(*array_len, sizeof(*pages), GFP_KERNEL);
+	if (!pages)
+		goto cleanup;
+
+	for (i = 0; i < *array_len; i++) {
+		page = alloc_pages(GFP_KERNEL | __GFP_ZERO,
+				   get_order(alloc_unit << PAGE_SHIFT));
+		if (!page) {
+			/* Try allocating small pages if high order pages are not available. */
+			if (alloc_unit == 1) {
+				goto cleanup;
+			} else {
+				memset(vmap_pages, 0,
+				       sizeof(*vmap_pages) * vmap_page_index);
+				vmap_page_index = 0;
+
+				for (j = 0; j < i; j++)
+					__free_pages(pages[j], alloc_unit);
+
+				kfree(pages);
+				alloc_unit = 1;
+				goto retry;
+			}
+		}
+
+		pages[i] = page;
+		for (j = 0; j < alloc_unit; j++)
+			vmap_pages[vmap_page_index++] = page++;
+	}
+
+	vaddr = vmap(vmap_pages, vmap_page_index, VM_MAP, PAGE_KERNEL);
+	kfree(vmap_pages);
+
+	*pages_array = pages;
+	return vaddr;
+
+cleanup:
+	for (j = 0; j < i; j++)
+		__free_pages(pages[i], alloc_unit);
+
+	kfree(pages);
+	kfree(vmap_pages);
+	return NULL;
+}
+
+static void *netvsc_map_pages(struct page **pages, int count, int alloc_unit)
+{
+	int pg_count = count * alloc_unit;
+	struct page *page;
+	unsigned long *pfns;
+	int pfn_index = 0;
+	void *vaddr;
+	int i, j;
+
+	if (!pages)
+		return NULL;
+
+	pfns = kcalloc(pg_count, sizeof(*pfns), GFP_KERNEL);
+	if (!pfns)
+		return NULL;
+
+	for (i = 0; i < count; i++) {
+		page = pages[i];
+		if (!page) {
+			pr_warn("page is not available %d.\n", i);
+			return NULL;
+		}
+
+		for (j = 0; j < alloc_unit; j++) {
+			pfns[pfn_index++] = page_to_pfn(page++) +
+				(ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
+		}
+	}
+
+	vaddr = vmap_pfn(pfns, pg_count, PAGE_KERNEL_IO);
+	kfree(pfns);
+	return vaddr;
+}
+
 static int netvsc_init_buf(struct hv_device *device,
 			   struct netvsc_device *net_device,
 			   const struct netvsc_device_info *device_info)
@@ -337,7 +462,7 @@ static int netvsc_init_buf(struct hv_device *device,
 	struct nvsp_1_message_send_receive_buffer_complete *resp;
 	struct net_device *ndev = hv_get_drvdata(device);
 	struct nvsp_message *init_packet;
-	unsigned int buf_size;
+	unsigned int buf_size, alloc_unit;
 	size_t map_words;
 	int i, ret = 0;
 
@@ -350,7 +475,14 @@ static int netvsc_init_buf(struct hv_device *device,
 		buf_size = min_t(unsigned int, buf_size,
 				 NETVSC_RECEIVE_BUFFER_SIZE_LEGACY);
 
-	net_device->recv_buf = vzalloc(buf_size);
+	if (hv_isolation_type_snp())
+		net_device->recv_buf =
+			netvsc_alloc_pages(&net_device->recv_pages,
+					   &net_device->recv_page_count,
+					   buf_size);
+	else
+		net_device->recv_buf = vzalloc(buf_size);
+
 	if (!net_device->recv_buf) {
 		netdev_err(ndev,
 			   "unable to allocate receive buffer of size %u\n",
@@ -375,6 +507,27 @@ static int netvsc_init_buf(struct hv_device *device,
 		goto cleanup;
 	}
 
+	if (hv_isolation_type_snp()) {
+		alloc_unit = (buf_size / net_device->recv_page_count)
+				>> PAGE_SHIFT;
+
+		/* Unmap previous virtual address and map pages in the extra
+		 * address space(above shared gpa boundary) in Isolation VM.
+		 */
+		vunmap(net_device->recv_buf);
+		net_device->recv_buf =
+			netvsc_map_pages(net_device->recv_pages,
+					 net_device->recv_page_count,
+					 alloc_unit);
+		if (!net_device->recv_buf) {
+			netdev_err(ndev,
+				   "unable to allocate receive buffer of size %u\n",
+				   buf_size);
+			ret = -ENOMEM;
+			goto cleanup;
+		}
+	}
+
 	/* Notify the NetVsp of the gpadl handle */
 	init_packet = &net_device->channel_init_pkt;
 	memset(init_packet, 0, sizeof(struct nvsp_message));
@@ -456,13 +609,21 @@ static int netvsc_init_buf(struct hv_device *device,
 	buf_size = device_info->send_sections * device_info->send_section_size;
 	buf_size = round_up(buf_size, PAGE_SIZE);
 
-	net_device->send_buf = vzalloc(buf_size);
+	if (hv_isolation_type_snp())
+		net_device->send_buf =
+			netvsc_alloc_pages(&net_device->send_pages,
+					   &net_device->send_page_count,
+					   buf_size);
+	else
+		net_device->send_buf = vzalloc(buf_size);
+
 	if (!net_device->send_buf) {
 		netdev_err(ndev, "unable to allocate send buffer of size %u\n",
 			   buf_size);
 		ret = -ENOMEM;
 		goto cleanup;
 	}
+
 	net_device->send_buf_size = buf_size;
 
 	/* Establish the gpadl handle for this buffer on this
@@ -478,6 +639,27 @@ static int netvsc_init_buf(struct hv_device *device,
 		goto cleanup;
 	}
 
+	if (hv_isolation_type_snp()) {
+		alloc_unit = (buf_size / net_device->send_page_count)
+				>> PAGE_SHIFT;
+
+		/* Unmap previous virtual address and map pages in the extra
+		 * address space(above shared gpa boundary) in Isolation VM.
+		 */
+		vunmap(net_device->send_buf);
+		net_device->send_buf =
+			netvsc_map_pages(net_device->send_pages,
+					 net_device->send_page_count,
+					 alloc_unit);
+		if (!net_device->send_buf) {
+			netdev_err(ndev,
+				   "unable to allocate receive buffer of size %u\n",
+				   buf_size);
+			ret = -ENOMEM;
+			goto cleanup;
+		}
+	}
+
 	/* Notify the NetVsp of the gpadl handle */
 	init_packet = &net_device->channel_init_pkt;
 	memset(init_packet, 0, sizeof(struct nvsp_message));
@@ -768,7 +950,7 @@ static void netvsc_send_tx_complete(struct net_device *ndev,
 
 	/* Notify the layer above us */
 	if (likely(skb)) {
-		const struct hv_netvsc_packet *packet
+		struct hv_netvsc_packet *packet
 			= (struct hv_netvsc_packet *)skb->cb;
 		u32 send_index = packet->send_buf_index;
 		struct netvsc_stats *tx_stats;
@@ -784,6 +966,7 @@ static void netvsc_send_tx_complete(struct net_device *ndev,
 		tx_stats->bytes += packet->total_bytes;
 		u64_stats_update_end(&tx_stats->syncp);
 
+		netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
 		napi_consume_skb(skb, budget);
 	}
 
@@ -948,6 +1131,87 @@ static void netvsc_copy_to_send_buf(struct netvsc_device *net_device,
 		memset(dest, 0, padding);
 }
 
+void netvsc_dma_unmap(struct hv_device *hv_dev,
+		      struct hv_netvsc_packet *packet)
+{
+	u32 page_count = packet->cp_partial ?
+		packet->page_buf_cnt - packet->rmsg_pgcnt :
+		packet->page_buf_cnt;
+	int i;
+
+	if (!hv_is_isolation_supported())
+		return;
+
+	if (!packet->dma_range)
+		return;
+
+	for (i = 0; i < page_count; i++)
+		dma_unmap_single(&hv_dev->device, packet->dma_range[i].dma,
+				 packet->dma_range[i].mapping_size,
+				 DMA_TO_DEVICE);
+
+	kfree(packet->dma_range);
+}
+
+/* netvsc_dma_map - Map swiotlb bounce buffer with data page of
+ * packet sent by vmbus_sendpacket_pagebuffer() in the Isolation
+ * VM.
+ *
+ * In isolation VM, netvsc send buffer has been marked visible to
+ * host and so the data copied to send buffer doesn't need to use
+ * bounce buffer. The data pages handled by vmbus_sendpacket_pagebuffer()
+ * may not be copied to send buffer and so these pages need to be
+ * mapped with swiotlb bounce buffer. netvsc_dma_map() is to do
+ * that. The pfns in the struct hv_page_buffer need to be converted
+ * to bounce buffer's pfn. The loop here is necessary because the
+ * entries in the page buffer array are not necessarily full
+ * pages of data.  Each entry in the array has a separate offset and
+ * len that may be non-zero, even for entries in the middle of the
+ * array.  And the entries are not physically contiguous.  So each
+ * entry must be individually mapped rather than as a contiguous unit.
+ * So not use dma_map_sg() here.
+ */
+static int netvsc_dma_map(struct hv_device *hv_dev,
+		   struct hv_netvsc_packet *packet,
+		   struct hv_page_buffer *pb)
+{
+	u32 page_count =  packet->cp_partial ?
+		packet->page_buf_cnt - packet->rmsg_pgcnt :
+		packet->page_buf_cnt;
+	dma_addr_t dma;
+	int i;
+
+	if (!hv_is_isolation_supported())
+		return 0;
+
+	packet->dma_range = kcalloc(page_count,
+				    sizeof(*packet->dma_range),
+				    GFP_KERNEL);
+	if (!packet->dma_range)
+		return -ENOMEM;
+
+	for (i = 0; i < page_count; i++) {
+		char *src = phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
+					 + pb[i].offset);
+		u32 len = pb[i].len;
+
+		dma = dma_map_single(&hv_dev->device, src, len,
+				     DMA_TO_DEVICE);
+		if (dma_mapping_error(&hv_dev->device, dma)) {
+			kfree(packet->dma_range);
+			return -ENOMEM;
+		}
+
+		packet->dma_range[i].dma = dma;
+		packet->dma_range[i].mapping_size = len;
+		pb[i].pfn = dma >> HV_HYP_PAGE_SHIFT;
+		pb[i].offset = offset_in_hvpage(dma);
+		pb[i].len = len;
+	}
+
+	return 0;
+}
+
 static inline int netvsc_send_pkt(
 	struct hv_device *device,
 	struct hv_netvsc_packet *packet,
@@ -988,14 +1252,24 @@ static inline int netvsc_send_pkt(
 
 	trace_nvsp_send_pkt(ndev, out_channel, rpkt);
 
+	packet->dma_range = NULL;
 	if (packet->page_buf_cnt) {
 		if (packet->cp_partial)
 			pb += packet->rmsg_pgcnt;
 
+		ret = netvsc_dma_map(ndev_ctx->device_ctx, packet, pb);
+		if (ret) {
+			ret = -EAGAIN;
+			goto exit;
+		}
+
 		ret = vmbus_sendpacket_pagebuffer(out_channel,
 						  pb, packet->page_buf_cnt,
 						  &nvmsg, sizeof(nvmsg),
 						  req_id);
+
+		if (ret)
+			netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
 	} else {
 		ret = vmbus_sendpacket(out_channel,
 				       &nvmsg, sizeof(nvmsg),
@@ -1003,6 +1277,7 @@ static inline int netvsc_send_pkt(
 				       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
 	}
 
+exit:
 	if (ret == 0) {
 		atomic_inc_return(&nvchan->queue_sends);
 
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 382bebc2420d..c3dc884b31e3 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2577,6 +2577,7 @@ static int netvsc_probe(struct hv_device *dev,
 	list_add(&net_device_ctx->list, &netvsc_dev_list);
 	rtnl_unlock();
 
+	dma_set_min_align_mask(&dev->device, HV_HYP_PAGE_SIZE - 1);
 	netvsc_devinfo_put(device_info);
 	return 0;
 
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index f6c9c2a670f9..448fcc325ed7 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -361,6 +361,8 @@ static void rndis_filter_receive_response(struct net_device *ndev,
 			}
 		}
 
+		netvsc_dma_unmap(((struct net_device_context *)
+			netdev_priv(ndev))->device_ctx, &request->pkt);
 		complete(&request->wait_event);
 	} else {
 		netdev_err(ndev,
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index c94c534a944e..81e58dd582dc 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -1597,6 +1597,11 @@ struct hyperv_service_callback {
 	void (*callback)(void *context);
 };
 
+struct hv_dma_range {
+	dma_addr_t dma;
+	u32 mapping_size;
+};
+
 #define MAX_SRV_VER	0x7ffffff
 extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *icmsghdrp, u8 *buf, u32 buflen,
 				const int *fw_version, int fw_vercnt,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:47:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:47:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186699.335469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mH-0007ct-5n; Tue, 14 Sep 2021 13:47:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186699.335469; Tue, 14 Sep 2021 13:47:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mH-0007bW-0z; Tue, 14 Sep 2021 13:47:13 +0000
Received: by outflank-mailman (input) for mailman id 186699;
 Tue, 14 Sep 2021 13:47:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8gA-0001wz-TU
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:54 +0000
Received: from mail-pf1-x436.google.com (unknown [2607:f8b0:4864:20::436])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 12e9ab66-ee91-44c8-8903-43d60f23f701;
 Tue, 14 Sep 2021 13:39:50 +0000 (UTC)
Received: by mail-pf1-x436.google.com with SMTP id m26so12258771pff.3
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:50 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12e9ab66-ee91-44c8-8903-43d60f23f701
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ChpyrJrtZ0GWASO9SJSkVtTi3NWXtvZFVFa0kxYT1A8=;
        b=HE+Mpp7NrG39OPwhSHFYcEZgDEVLplripW8SNkjWBluPNhIPHnO4Zc3m25/+SMv9ox
         teu8xjn6xSwxQBB952yLTWC0vcJbuUUEVkiGOR8FS9DSekELD6QzR1hT2fU+xQdK7Uul
         Cj41tmdrUxQIfllJ/7zBfNfb/Gindx6VmnHKCORnCWA39fPeFGYGFF7S6OD3J97lxNxs
         DVoG0FIayXg0fvP17onaBJXLG3VHM82NewgI22FMQXLITV20NYgiHQLQz5oTFXLXtY7G
         0u7GQsCFLuYkCnMhkeKP27UJxUm0AOfwxhp2ZeUejXbuePKmYRrXRlyyAbznfW2hDEGT
         74nw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ChpyrJrtZ0GWASO9SJSkVtTi3NWXtvZFVFa0kxYT1A8=;
        b=XyQ0aBjdmSoN5L2JpD66Z+aUTTAs4PCkCIH8SgQeui8Ue+ACxvyrpewxBek1c2SxMl
         rmDUIPiO0EAuK6mmsbvHjTVt5HuVy9bFON12F95HGdOwE01YqQAXJqcn8n9fzELGhsWY
         xq1xH36QkraxN4QUtEgX9QQaZs/dBJhJdo0xjxkLTuPUxAeolcD7MW+J1RZXi5bqEfa7
         +GYpUMRtH5XI9pcdEidGP1RsOC8lisn7vBnTFDqz1dIz69ju6zDkiepfS3hFi0EDbXgr
         nrSac8JHM0hfuDL5CcWUUlQyLnoBCaFZ0rlsHyQY+/Bi18unufQzDTqHRUfZLT/c5LFv
         RPkA==
X-Gm-Message-State: AOAM5338npscI/bM8nlncQV/1dg2oA5ClG7JvEZIdpiPwxcC8C1NwvWZ
	+0l4yeXHt7rMRquGY+ctd2c=
X-Google-Smtp-Source: ABdhPJz7GLWCu0uNZSZf3h3ggTXiZShd7wRNgOpywDn3xOoYCN0POtH6uJq1e060FQC3kbngt9TJWQ==
X-Received: by 2002:a62:b414:0:b029:32e:3ef0:7735 with SMTP id h20-20020a62b4140000b029032e3ef07735mr4848036pfn.61.1631626789107;
        Tue, 14 Sep 2021 06:39:49 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 10/12] hyperv/IOMMU: Enable swiotlb bounce buffer for Isolation VM
Date: Tue, 14 Sep 2021 09:39:11 -0400
Message-Id: <20210914133916.1440931-11-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

hyperv Isolation VM requires bounce buffer support to copy
data from/to encrypted memory and so enable swiotlb force
mode to use swiotlb bounce buffer for DMA transaction.

In Isolation VM with AMD SEV, the bounce buffer needs to be
accessed via extra address space which is above shared_gpa_boundary
(E.G 39 bit address line) reported by Hyper-V CPUID ISOLATION_CONFIG.
The access physical address will be original physical address +
shared_gpa_boundary. The shared_gpa_boundary in the AMD SEV SNP
spec is called virtual top of memory(vTOM). Memory addresses below
vTOM are automatically treated as private while memory above
vTOM is treated as shared.

Hyper-V initalizes swiotlb bounce buffer and default swiotlb
needs to be disabled. pci_swiotlb_detect_override() and
pci_swiotlb_detect_4gb() enable the default one. To override
the setting, hyperv_swiotlb_detect() needs to run before
these detect functions which depends on the pci_xen_swiotlb_
init(). Make pci_xen_swiotlb_init() depends on the hyperv_swiotlb
_detect() to keep the order.

Swiotlb bounce buffer code calls set_memory_decrypted()
to mark bounce buffer visible to host and map it in extra
address space via memremap. Populate the shared_gpa_boundary
(vTOM) via swiotlb_unencrypted_base variable.

The map function memremap() can't work in the early place
hyperv_iommu_swiotlb_init() and so initialize swiotlb bounce
buffer in the hyperv_iommu_swiotlb_later_init().

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
       * Use swiotlb_unencrypted_base variable to pass shared_gpa_
         boundary and map bounce buffer inside swiotlb code.

Change since v3:
       * Get hyperv bounce bufffer size via default swiotlb
       bounce buffer size function and keep default size as
       same as the one in the AMD SEV VM.
---
 arch/x86/include/asm/mshyperv.h |  2 ++
 arch/x86/mm/mem_encrypt.c       |  3 +-
 arch/x86/xen/pci-swiotlb-xen.c  |  3 +-
 drivers/hv/vmbus_drv.c          |  3 ++
 drivers/iommu/hyperv-iommu.c    | 60 +++++++++++++++++++++++++++++++++
 include/linux/hyperv.h          |  1 +
 6 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 165423e8b67a..2d22f29f90c9 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -182,6 +182,8 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector,
 		struct hv_interrupt_entry *entry);
 int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
 int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible);
+void *hv_map_memory(void *addr, unsigned long size);
+void hv_unmap_memory(void *addr);
 void hv_ghcb_msr_write(u64 msr, u64 value);
 void hv_ghcb_msr_read(u64 msr, u64 *value);
 #else /* CONFIG_HYPERV */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index ff08dc463634..e2db0b8ed938 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -30,6 +30,7 @@
 #include <asm/processor-flags.h>
 #include <asm/msr.h>
 #include <asm/cmdline.h>
+#include <asm/mshyperv.h>
 
 #include "mm_internal.h"
 
@@ -202,7 +203,7 @@ void __init sev_setup_arch(void)
 	phys_addr_t total_mem = memblock_phys_mem_size();
 	unsigned long size;
 
-	if (!sev_active())
+	if (!sev_active() && !hv_is_isolation_supported())
 		return;
 
 	/*
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
index 54f9aa7e8457..43bd031aa332 100644
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -4,6 +4,7 @@
 
 #include <linux/dma-map-ops.h>
 #include <linux/pci.h>
+#include <linux/hyperv.h>
 #include <xen/swiotlb-xen.h>
 
 #include <asm/xen/hypervisor.h>
@@ -91,6 +92,6 @@ int pci_xen_swiotlb_init_late(void)
 EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
 
 IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
-		  NULL,
+		  hyperv_swiotlb_detect,
 		  pci_xen_swiotlb_init,
 		  NULL);
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 392c1ac4f819..b0be287e9a32 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -23,6 +23,7 @@
 #include <linux/cpu.h>
 #include <linux/sched/task_stack.h>
 
+#include <linux/dma-map-ops.h>
 #include <linux/delay.h>
 #include <linux/notifier.h>
 #include <linux/panic_notifier.h>
@@ -2078,6 +2079,7 @@ struct hv_device *vmbus_device_create(const guid_t *type,
 	return child_device_obj;
 }
 
+static u64 vmbus_dma_mask = DMA_BIT_MASK(64);
 /*
  * vmbus_device_register - Register the child device
  */
@@ -2118,6 +2120,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
 	}
 	hv_debug_add_dev_dir(child_device_obj);
 
+	child_device_obj->device.dma_mask = &vmbus_dma_mask;
 	return 0;
 
 err_kset_unregister:
diff --git a/drivers/iommu/hyperv-iommu.c b/drivers/iommu/hyperv-iommu.c
index e285a220c913..a8ac2239de0f 100644
--- a/drivers/iommu/hyperv-iommu.c
+++ b/drivers/iommu/hyperv-iommu.c
@@ -13,14 +13,22 @@
 #include <linux/irq.h>
 #include <linux/iommu.h>
 #include <linux/module.h>
+#include <linux/hyperv.h>
+#include <linux/io.h>
 
 #include <asm/apic.h>
 #include <asm/cpu.h>
 #include <asm/hw_irq.h>
 #include <asm/io_apic.h>
+#include <asm/iommu.h>
+#include <asm/iommu_table.h>
 #include <asm/irq_remapping.h>
 #include <asm/hypervisor.h>
 #include <asm/mshyperv.h>
+#include <asm/swiotlb.h>
+#include <linux/dma-map-ops.h>
+#include <linux/dma-direct.h>
+#include <linux/set_memory.h>
 
 #include "irq_remapping.h"
 
@@ -36,6 +44,9 @@
 static cpumask_t ioapic_max_cpumask = { CPU_BITS_NONE };
 static struct irq_domain *ioapic_ir_domain;
 
+static unsigned long hyperv_io_tlb_size;
+static void *hyperv_io_tlb_start;
+
 static int hyperv_ir_set_affinity(struct irq_data *data,
 		const struct cpumask *mask, bool force)
 {
@@ -337,4 +348,53 @@ static const struct irq_domain_ops hyperv_root_ir_domain_ops = {
 	.free = hyperv_root_irq_remapping_free,
 };
 
+static void __init hyperv_iommu_swiotlb_init(void)
+{
+	/*
+	 * Allocate Hyper-V swiotlb bounce buffer at early place
+	 * to reserve large contiguous memory.
+	 */
+	hyperv_io_tlb_size = swiotlb_size_or_default();
+	hyperv_io_tlb_start = memblock_alloc(
+		hyperv_io_tlb_size, PAGE_SIZE);
+
+	if (!hyperv_io_tlb_start) {
+		pr_warn("Fail to allocate Hyper-V swiotlb buffer.\n");
+		return;
+	}
+}
+
+int __init hyperv_swiotlb_detect(void)
+{
+	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
+		return 0;
+
+	if (!hv_is_isolation_supported())
+		return 0;
+
+	/*
+	 * Enable swiotlb force mode in Isolation VM to
+	 * use swiotlb bounce buffer for dma transaction.
+	 */
+	swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
+	swiotlb_force = SWIOTLB_FORCE;
+	return 1;
+}
+
+static void __init hyperv_iommu_swiotlb_later_init(void)
+{
+	/*
+	 * Swiotlb bounce buffer needs to be mapped in extra address
+	 * space. Map function doesn't work in the early place and so
+	 * call swiotlb_late_init_with_tbl() here.
+	 */
+	if (swiotlb_late_init_with_tbl(hyperv_io_tlb_start,
+				       hyperv_io_tlb_size >> IO_TLB_SHIFT))
+		panic("Fail to initialize hyperv swiotlb.\n");
+}
+
+IOMMU_INIT_FINISH(hyperv_swiotlb_detect,
+		  NULL, hyperv_iommu_swiotlb_init,
+		  hyperv_iommu_swiotlb_later_init);
+
 #endif
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index a9e0bc3b1511..bb1a1519b93a 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -1739,6 +1739,7 @@ int hyperv_write_cfg_blk(struct pci_dev *dev, void *buf, unsigned int len,
 int hyperv_reg_block_invalidate(struct pci_dev *dev, void *context,
 				void (*block_invalidate)(void *context,
 							 u64 block_mask));
+int __init hyperv_swiotlb_detect(void);
 
 struct hyperv_pci_block_ops {
 	int (*read_block)(struct pci_dev *dev, void *buf, unsigned int buf_len,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:47:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:47:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186731.335486 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mg-0000We-He; Tue, 14 Sep 2021 13:47:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186731.335486; Tue, 14 Sep 2021 13:47:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8mg-0000WT-El; Tue, 14 Sep 2021 13:47:38 +0000
Received: by outflank-mailman (input) for mailman id 186731;
 Tue, 14 Sep 2021 13:47:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8gF-0001wz-Tk
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:59 +0000
Received: from mail-pg1-x535.google.com (unknown [2607:f8b0:4864:20::535])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 45bdebf3-7175-4c1a-a337-030cfe2d2541;
 Tue, 14 Sep 2021 13:39:52 +0000 (UTC)
Received: by mail-pg1-x535.google.com with SMTP id s11so12736189pgr.11
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:52 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45bdebf3-7175-4c1a-a337-030cfe2d2541
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=V6GY3z4JvJviWkSRCk5kJjmxao2kOnqCUtugNfHg/pw=;
        b=dDHEgjjhONvetT5YclBkax+X6EY3U8rrUfLy/aimO6iFEaJzZte34OHy3Gmk7JZfPE
         Pggybi9YLY9iIVSeqJqx7clgxMGVMEP8TtLaHzOW6LPUCpK91pv50TaSb2AEO9aMpLBd
         5vNa0hpfM90xmm+vOpPn4FaKRXi5urhJjU86eJpvLASlL+/dKBZuE22QOsg1W6uzTiyo
         kH5MFhd5SbmeVWhmueq7+NQylhNDj3Eh9b63XJbTIpiudQB42/28UOhDAfQf8/ph6yE7
         dcIqRYrSNXK3RNbQ3Xe+HFePVpn7E6KSoYLVPniJoegZD+jyC822quIHXse/2XucEYnb
         Llbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=V6GY3z4JvJviWkSRCk5kJjmxao2kOnqCUtugNfHg/pw=;
        b=NVzID8JQEFGwD5FdpeHVWi2FUyz7Iah69wvu0QCO9AFr+ZwdaiK7E0O5rrWqL7XiyT
         l0uLhpRGQkWZaFkl+lwpIcz8z3/TbHYKvpiJS2UEVIepz0ilFw4vasLwaad9D1wM8yYj
         5CPY+ogvzHIRMLaPAqAcC1TaobdKZ6UEC325soQeG9/G42V3AitMZbsoWnOMHTl0HnRZ
         vKGJBaFgARZirGbBywrNmEnJIFMu7rInPJDX4xgoMoMjzPSrnmCb1m/HBh714iKxzCm5
         R5v02F8+9I+LF9tkc75MNcx+NB1kArULmf84h3l8w7v75ZeYhoAILlmYj44VkTs86ZI/
         uvjw==
X-Gm-Message-State: AOAM530zkzTiBG3CyZBi86NhCG8bN4lc4ZWIUUbhk1jogL7aFn1YEhBy
	kwy/lHay1kEQzv5jjh/pRqs=
X-Google-Smtp-Source: ABdhPJyd7B/cycREjmjwvYi/L8C41okdW25JkBh0jV2R9467KDZelN884wrT/mvAlGHMBU2oT9UWmg==
X-Received: by 2002:a63:ec06:: with SMTP id j6mr15654053pgh.259.1631626791699;
        Tue, 14 Sep 2021 06:39:51 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 11/12] scsi: storvsc: Add Isolation VM support for storvsc driver
Date: Tue, 14 Sep 2021 09:39:12 -0400
Message-Id: <20210914133916.1440931-12-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

In Isolation VM, all shared memory with host needs to mark visible
to host via hvcall. vmbus_establish_gpadl() has already done it for
storvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
mpb_desc() still needs to be handled. Use DMA API(scsi_dma_map/unmap)
to map these memory during sending/receiving packet and return swiotlb
bounce buffer dma address. In Isolation VM, swiotlb  bounce buffer is
marked to be visible to host and the swiotlb force mode is enabled.

Set device's dma min align mask to HV_HYP_PAGE_SIZE - 1 in order to
keep the original data offset in the bounce buffer.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* use scsi_dma_map/unmap() instead of dma_map/unmap_sg()
	* Add deleted comments back.
	* Fix error calculation of  hvpnfs_to_add

Change since v3:
	* Rplace dma_map_page with dma_map_sg()
	* Use for_each_sg() to populate payload->range.pfn_array.
	* Remove storvsc_dma_map macro
---
 drivers/hv/vmbus_drv.c     |  1 +
 drivers/scsi/storvsc_drv.c | 24 +++++++++++++++---------
 include/linux/hyperv.h     |  1 +
 3 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index b0be287e9a32..9c53f823cde1 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -2121,6 +2121,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
 	hv_debug_add_dev_dir(child_device_obj);
 
 	child_device_obj->device.dma_mask = &vmbus_dma_mask;
+	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
 	return 0;
 
 err_kset_unregister:
diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index ebbbc1299c62..d10b450bcf0c 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -21,6 +21,8 @@
 #include <linux/device.h>
 #include <linux/hyperv.h>
 #include <linux/blkdev.h>
+#include <linux/dma-mapping.h>
+
 #include <scsi/scsi.h>
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_host.h>
@@ -1322,6 +1324,7 @@ static void storvsc_on_channel_callback(void *context)
 					continue;
 				}
 				request = (struct storvsc_cmd_request *)scsi_cmd_priv(scmnd);
+				scsi_dma_unmap(scmnd);
 			}
 
 			storvsc_on_receive(stor_device, packet, request);
@@ -1735,7 +1738,6 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 	struct hv_host_device *host_dev = shost_priv(host);
 	struct hv_device *dev = host_dev->dev;
 	struct storvsc_cmd_request *cmd_request = scsi_cmd_priv(scmnd);
-	int i;
 	struct scatterlist *sgl;
 	unsigned int sg_count;
 	struct vmscsi_request *vm_srb;
@@ -1817,10 +1819,11 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 	payload_sz = sizeof(cmd_request->mpb);
 
 	if (sg_count) {
-		unsigned int hvpgoff, hvpfns_to_add;
 		unsigned long offset_in_hvpg = offset_in_hvpage(sgl->offset);
 		unsigned int hvpg_count = HVPFN_UP(offset_in_hvpg + length);
-		u64 hvpfn;
+		struct scatterlist *sg;
+		unsigned long hvpfn, hvpfns_to_add;
+		int j, i = 0;
 
 		if (hvpg_count > MAX_PAGE_BUFFER_COUNT) {
 
@@ -1834,8 +1837,11 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 		payload->range.len = length;
 		payload->range.offset = offset_in_hvpg;
 
+		sg_count = scsi_dma_map(scmnd);
+		if (sg_count < 0)
+			return SCSI_MLQUEUE_DEVICE_BUSY;
 
-		for (i = 0; sgl != NULL; sgl = sg_next(sgl)) {
+		for_each_sg(sgl, sg, sg_count, j) {
 			/*
 			 * Init values for the current sgl entry. hvpgoff
 			 * and hvpfns_to_add are in units of Hyper-V size
@@ -1845,10 +1851,9 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 			 * even on other than the first sgl entry, provided
 			 * they are a multiple of PAGE_SIZE.
 			 */
-			hvpgoff = HVPFN_DOWN(sgl->offset);
-			hvpfn = page_to_hvpfn(sg_page(sgl)) + hvpgoff;
-			hvpfns_to_add =	HVPFN_UP(sgl->offset + sgl->length) -
-						hvpgoff;
+			hvpfn = HVPFN_DOWN(sg_dma_address(sg));
+			hvpfns_to_add = HVPFN_UP(sg_dma_address(sg) +
+						 sg_dma_len(sg)) - hvpfn;
 
 			/*
 			 * Fill the next portion of the PFN array with
@@ -1858,7 +1863,7 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 			 * the PFN array is filled.
 			 */
 			while (hvpfns_to_add--)
-				payload->range.pfn_array[i++] =	hvpfn++;
+				payload->range.pfn_array[i++] = hvpfn++;
 		}
 	}
 
@@ -2002,6 +2007,7 @@ static int storvsc_probe(struct hv_device *device,
 	stor_device->vmscsi_size_delta = sizeof(struct vmscsi_win8_extension);
 	spin_lock_init(&stor_device->lock);
 	hv_set_drvdata(device, stor_device);
+	dma_set_min_align_mask(&device->device, HV_HYP_PAGE_SIZE - 1);
 
 	stor_device->port_number = host->host_no;
 	ret = storvsc_connect_to_vsp(device, storvsc_ringbuffer_size, is_fc);
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index bb1a1519b93a..c94c534a944e 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -1274,6 +1274,7 @@ struct hv_device {
 
 	struct vmbus_channel *channel;
 	struct kset	     *channels_kset;
+	struct device_dma_parameters dma_parms;
 
 	/* place holder to keep track of the dir for hv device in debugfs */
 	struct dentry *debug_dir;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:47:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:47:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186739.335496 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ms-000107-QR; Tue, 14 Sep 2021 13:47:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186739.335496; Tue, 14 Sep 2021 13:47:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8ms-000100-N4; Tue, 14 Sep 2021 13:47:50 +0000
Received: by outflank-mailman (input) for mailman id 186739;
 Tue, 14 Sep 2021 13:47:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8g0-0001wz-TF
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:44 +0000
Received: from mail-pg1-x530.google.com (unknown [2607:f8b0:4864:20::530])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3c7a927e-7bd9-435b-9bc2-5a5258cff843;
 Tue, 14 Sep 2021 13:39:47 +0000 (UTC)
Received: by mail-pg1-x530.google.com with SMTP id k24so12733905pgh.8
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:47 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c7a927e-7bd9-435b-9bc2-5a5258cff843
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=owa2epIC/5mze9MxnGBTWuFjw26S5l9iHD474a0FPZY=;
        b=YPR0RRcFGHUBaUpbP9gJUeI5hFz/Ykc2vEDRk21Y+Hx2ak94WCKoxp0mcESjgioo67
         zQ0QtdVf0Bv2thg+qZaXcGTuwv8ZB/9ONE9jPeCXqMp2Inclm/3Hhux53T4BLmW7uz90
         j7APkz+ekg0qxBWYqZuPr9Uy9XzMQeYfkCqJqS/Wzvh1ec7KuAQZ7pi6wk63IJlMzJ4i
         2oz+AnJOtCDleuDhptkGHliv7R41SodRqvAJ7f3Fb1eGVT1WzZv2aCriGL1Fh2H9+Gg/
         t9SRIkFCG/H7bZf6edW4ThgnW3aT2vnhMToatGUVRBJ7o+6iRnTqJSTDYkSghGECv/mb
         un1g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=owa2epIC/5mze9MxnGBTWuFjw26S5l9iHD474a0FPZY=;
        b=4FC8UNG3+65FAT1OnlVRHpXgHNLx5GlbmOU0EZi0WGAZl4D6AiF0l4C9Jn45o2d1uI
         /x6DbLytVX8nzlFYzKZHLqUwl8OkZS5yccBFhYsgK+TPy11zFfihXdXHR31OW92RUDcv
         duo7qEZq465jYSd6F1E+3TWO+nRabTrqdlXM9XQtNQSu7QxEprSABRkNr7XKzyqrzyfr
         8/DhdbH7jk3sHaS2m79qAsbjpk1YOaq2b1+1QMnn9GX+PMUbSTeVde42nJ6Ubu5oYuV7
         ptaDj4gD4KvsmYAp17q9hkBoOZ7+tVgkNaV7h9v0nuOMRufs9XBHgKRq/c1Vtavt4MaO
         6fag==
X-Gm-Message-State: AOAM533HV5SV1M4SoxD9dW9KOQE3Rz/uiCZfJjmF53ujtHktdma1xGdN
	Ia3Cr88++E92YcS8CV60axY=
X-Google-Smtp-Source: ABdhPJxXTtvlP911RHquYd4PWMjAFBqIUg4/wztNKPqgo3aV/dGiro8zblLWz9doF4YohiyUAH/bGw==
X-Received: by 2002:a05:6a00:2449:b0:43c:4a5e:55a6 with SMTP id d9-20020a056a00244900b0043c4a5e55a6mr4918302pfj.43.1631626786861;
        Tue, 14 Sep 2021 06:39:46 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 09/12] x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM
Date: Tue, 14 Sep 2021 09:39:10 -0400
Message-Id: <20210914133916.1440931-10-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

In Isolation VM with AMD SEV, bounce buffer needs to be accessed via
extra address space which is above shared_gpa_boundary
(E.G 39 bit address line) reported by Hyper-V CPUID ISOLATION_CONFIG.
The access physical address will be original physical address +
shared_gpa_boundary. The shared_gpa_boundary in the AMD SEV SNP
spec is called virtual top of memory(vTOM). Memory addresses below
vTOM are automatically treated as private while memory above
vTOM is treated as shared.

Expose swiotlb_unencrypted_base for platforms to set unencrypted
memory base offset and call memremap() to map bounce buffer in the
swiotlb code, store map address and use the address to copy data
from/to swiotlb bounce buffer.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Expose swiotlb_unencrypted_base to set unencrypted memory
	  offset.
	* Use memremap() to map bounce buffer if swiotlb_unencrypted_
	  base is set.

Change since v1:
	* Make swiotlb_init_io_tlb_mem() return error code and return
          error when dma_map_decrypted() fails.
---
 include/linux/swiotlb.h |  6 ++++++
 kernel/dma/swiotlb.c    | 41 +++++++++++++++++++++++++++++++++++------
 2 files changed, 41 insertions(+), 6 deletions(-)

diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b0cb2a9973f4..4998ed44ae3d 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -72,6 +72,9 @@ extern enum swiotlb_force swiotlb_force;
  * @end:	The end address of the swiotlb memory pool. Used to do a quick
  *		range check to see if the memory was in fact allocated by this
  *		API.
+ * @vaddr:	The vaddr of the swiotlb memory pool. The swiotlb
+ *		memory pool may be remapped in the memory encrypted case and store
+ *		virtual address for bounce buffer operation.
  * @nslabs:	The number of IO TLB blocks (in groups of 64) between @start and
  *		@end. For default swiotlb, this is command line adjustable via
  *		setup_io_tlb_npages.
@@ -91,6 +94,7 @@ extern enum swiotlb_force swiotlb_force;
 struct io_tlb_mem {
 	phys_addr_t start;
 	phys_addr_t end;
+	void *vaddr;
 	unsigned long nslabs;
 	unsigned long used;
 	unsigned int index;
@@ -185,4 +189,6 @@ static inline bool is_swiotlb_for_alloc(struct device *dev)
 }
 #endif /* CONFIG_DMA_RESTRICTED_POOL */
 
+extern phys_addr_t swiotlb_unencrypted_base;
+
 #endif /* __LINUX_SWIOTLB_H */
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 87c40517e822..9e30cc4bd872 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -50,6 +50,7 @@
 #include <asm/io.h>
 #include <asm/dma.h>
 
+#include <linux/io.h>
 #include <linux/init.h>
 #include <linux/memblock.h>
 #include <linux/iommu-helper.h>
@@ -72,6 +73,8 @@ enum swiotlb_force swiotlb_force;
 
 struct io_tlb_mem io_tlb_default_mem;
 
+phys_addr_t swiotlb_unencrypted_base;
+
 /*
  * Max segment that we can provide which (if pages are contingous) will
  * not be bounced (unless SWIOTLB_FORCE is set).
@@ -175,7 +178,7 @@ void __init swiotlb_update_mem_attributes(void)
 	memset(vaddr, 0, bytes);
 }
 
-static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
+static int swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
 				    unsigned long nslabs, bool late_alloc)
 {
 	void *vaddr = phys_to_virt(start);
@@ -196,13 +199,34 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
 		mem->slots[i].orig_addr = INVALID_PHYS_ADDR;
 		mem->slots[i].alloc_size = 0;
 	}
+
+	if (set_memory_decrypted((unsigned long)vaddr, bytes >> PAGE_SHIFT))
+		return -EFAULT;
+
+	/*
+	 * Map memory in the unencrypted physical address space when requested
+	 * (e.g. for Hyper-V AMD SEV-SNP Isolation VMs).
+	 */
+	if (swiotlb_unencrypted_base) {
+		phys_addr_t paddr = __pa(vaddr) + swiotlb_unencrypted_base;
+
+		vaddr = memremap(paddr, bytes, MEMREMAP_WB);
+		if (!vaddr) {
+			pr_err("Failed to map the unencrypted memory.\n");
+			return -ENOMEM;
+		}
+	}
+
 	memset(vaddr, 0, bytes);
+	mem->vaddr = vaddr;
+	return 0;
 }
 
 int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 {
 	struct io_tlb_mem *mem = &io_tlb_default_mem;
 	size_t alloc_size;
+	int ret;
 
 	if (swiotlb_force == SWIOTLB_NO_FORCE)
 		return 0;
@@ -217,7 +241,11 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 		panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
 		      __func__, alloc_size, PAGE_SIZE);
 
-	swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+	ret = swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
+	if (ret) {
+		memblock_free(__pa(mem), alloc_size);
+		return ret;
+	}
 
 	if (verbose)
 		swiotlb_print_info();
@@ -304,7 +332,7 @@ int
 swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
 {
 	struct io_tlb_mem *mem = &io_tlb_default_mem;
-	unsigned long bytes = nslabs << IO_TLB_SHIFT;
+	int ret;
 
 	if (swiotlb_force == SWIOTLB_NO_FORCE)
 		return 0;
@@ -318,8 +346,9 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
 	if (!mem->slots)
 		return -ENOMEM;
 
-	set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
-	swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+	ret = swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
+	if (ret)
+		return ret;
 
 	swiotlb_print_info();
 	swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
@@ -371,7 +400,7 @@ static void swiotlb_bounce(struct device *dev, phys_addr_t tlb_addr, size_t size
 	phys_addr_t orig_addr = mem->slots[index].orig_addr;
 	size_t alloc_size = mem->slots[index].alloc_size;
 	unsigned long pfn = PFN_DOWN(orig_addr);
-	unsigned char *vaddr = phys_to_virt(tlb_addr);
+	unsigned char *vaddr = mem->vaddr + tlb_addr - mem->start;
 	unsigned int tlb_offset, orig_addr_offset;
 
 	if (orig_addr == INVALID_PHYS_ADDR)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:48:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:48:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186746.335508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8n2-0001Wg-74; Tue, 14 Sep 2021 13:48:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186746.335508; Tue, 14 Sep 2021 13:48:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8n2-0001WV-3o; Tue, 14 Sep 2021 13:48:00 +0000
Received: by outflank-mailman (input) for mailman id 186746;
 Tue, 14 Sep 2021 13:47:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ8fq-0001wz-Sx
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:40:34 +0000
Received: from mail-pg1-x52d.google.com (unknown [2607:f8b0:4864:20::52d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb471e53-b4b5-4916-b9e9-d353826a2057;
 Tue, 14 Sep 2021 13:39:45 +0000 (UTC)
Received: by mail-pg1-x52d.google.com with SMTP id w8so12730577pgf.5
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 06:39:45 -0700 (PDT)
Received: from ubuntu-Virtual-Machine.corp.microsoft.com
 ([2001:4898:80e8:7:6ea2:a529:4af3:5057])
 by smtp.gmail.com with ESMTPSA id v13sm10461234pfm.16.2021.09.14.06.39.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 06:39:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb471e53-b4b5-4916-b9e9-d353826a2057
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=/usMlQBuF7F2k7aIrpXXc/KIZyi4prmIZLY9hJqrLaI=;
        b=L7DFk/u0wH9b/iGIHpdByH28JFlOCJvQJXIcQEQHfMZXJGvU9AkaMwEr1HnbxliyPL
         kfkbVqsa3rXNVuWDdCe7Bc1f248xQkyj17Gq8RLbwe1bkJrwpIjU2pIVT66S/RvRtCJ+
         XqFp3mMkYzBghBFtb2kQFehpWcmMo4tserFm8EZKV6vENVhy3H9Gs/VmnaI1kPljNwfI
         q5UnZA04pfDmnJfxVMpg0DvIeRkONnlOalgt3Cd3iCECHRA2yCzKdEcsr84DqJJ9hiQL
         0oWKvFTKQvk4YIQRsjK06WsNDVD9CME0l3HHBXaK/AC0I+rdqCj2QaJqIkNYv2L3xIx2
         Iejw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=/usMlQBuF7F2k7aIrpXXc/KIZyi4prmIZLY9hJqrLaI=;
        b=y1zkA6lqIMBmLFQE/smKc5Ed9fGUYgBuo8cWymDP+HU1JQUSMf1Qxm+6dOC7ypzzQS
         Q51olGoyIJuWVjxEek6WLn/6yp2xmFfV6SRfON62CBMPmeJ9nQjT57LEfAoNRY8pnZLO
         6ZOhJKF9KbI+itDGy6t3qug/3MqUQ1f9bezELzy5OZbJcrOnoDCquHT2ZskN0K1ufT0G
         NtC/DBCghKsnp3p2a6Qxf5Eg5y8kHnIywu5luHJuPV9llcQaVkOOcxC92TyZA5C8bxnD
         dDV4/d9EuR9EcE65dJQrhum6ePyvyOZtlud5E03zcUkiil7nAsNLxLkP6jaAN0JMF/BH
         tQAg==
X-Gm-Message-State: AOAM532PUJFvHuN1Nfwrt+uCTD8fkxg699vKe/oWHc7OO+PB2HmCnIS5
	9wp/Nk/jS/qbipawBDWI1qE=
X-Google-Smtp-Source: ABdhPJwtcm6RMZWi5bhD5h9QY6d/jTIRK78GsCcyNNGg4YhuGPJvGaV4TuX2VKOGUZcAcHGS6Xympw==
X-Received: by 2002:a62:7985:0:b0:437:36f1:d0df with SMTP id u127-20020a627985000000b0043736f1d0dfmr4833203pfc.52.1631626784499;
        Tue, 14 Sep 2021 06:39:44 -0700 (PDT)
From: Tianyu Lan <ltykernel@gmail.com>
To: kys@microsoft.com,
	haiyangz@microsoft.com,
	sthemmin@microsoft.com,
	wei.liu@kernel.org,
	decui@microsoft.com,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	x86@kernel.org,
	hpa@zytor.com,
	dave.hansen@linux.intel.com,
	luto@kernel.org,
	peterz@infradead.org,
	konrad.wilk@oracle.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	joro@8bytes.org,
	will@kernel.org,
	davem@davemloft.net,
	kuba@kernel.org,
	jejb@linux.ibm.com,
	martin.petersen@oracle.com,
	gregkh@linuxfoundation.org,
	arnd@arndb.de,
	hch@lst.de,
	m.szyprowski@samsung.com,
	robin.murphy@arm.com,
	brijesh.singh@amd.com,
	Tianyu.Lan@microsoft.com,
	thomas.lendacky@amd.com,
	pgonda@google.com,
	akpm@linux-foundation.org,
	kirill.shutemov@linux.intel.com,
	rppt@kernel.org,
	sfr@canb.auug.org.au,
	aneesh.kumar@linux.ibm.com,
	saravanand@fb.com,
	krish.sadhukhan@oracle.com,
	xen-devel@lists.xenproject.org,
	tj@kernel.org,
	rientjes@google.com,
	michael.h.kelley@microsoft.com
Cc: iommu@lists.linux-foundation.org,
	linux-arch@vger.kernel.org,
	linux-hyperv@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-scsi@vger.kernel.org,
	netdev@vger.kernel.org,
	vkuznets@redhat.com,
	parri.andrea@gmail.com,
	dave.hansen@intel.com
Subject: [PATCH V5 08/12] Drivers: hv : vmbus: Initialize VMbus ring buffer for Isolation VM
Date: Tue, 14 Sep 2021 09:39:09 -0400
Message-Id: <20210914133916.1440931-9-ltykernel@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210914133916.1440931-1-ltykernel@gmail.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Tianyu Lan <Tianyu.Lan@microsoft.com>

VMbus ring buffer are shared with host and it's need to
be accessed via extra address space of Isolation VM with
AMD SNP support. This patch is to map the ring buffer
address in extra address space via vmap_pfn(). Hyperv set
memory host visibility hvcall smears data in the ring buffer
and so reset the ring buffer memory to zero after mapping.

Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
---
Change since v4:
	* Use PFN_DOWN instead of HVPFN_DOWN in the hv_ringbuffer_init()

Change since v3:
	* Remove hv_ringbuffer_post_init(), merge map
	operation for Isolation VM into hv_ringbuffer_init()
	* Call hv_ringbuffer_init() after __vmbus_establish_gpadl().
---
 drivers/hv/Kconfig       |  1 +
 drivers/hv/channel.c     | 19 +++++++-------
 drivers/hv/ring_buffer.c | 55 ++++++++++++++++++++++++++++++----------
 3 files changed, 53 insertions(+), 22 deletions(-)

diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
index d1123ceb38f3..dd12af20e467 100644
--- a/drivers/hv/Kconfig
+++ b/drivers/hv/Kconfig
@@ -8,6 +8,7 @@ config HYPERV
 		|| (ARM64 && !CPU_BIG_ENDIAN))
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR if X86
+	select VMAP_PFN
 	help
 	  Select this option to run Linux as a Hyper-V client operating
 	  system.
diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index cf419eb1de77..ec847bd14119 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -684,15 +684,6 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	if (!newchannel->max_pkt_size)
 		newchannel->max_pkt_size = VMBUS_DEFAULT_MAX_PKT_SIZE;
 
-	err = hv_ringbuffer_init(&newchannel->outbound, page, send_pages, 0);
-	if (err)
-		goto error_clean_ring;
-
-	err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
-				 recv_pages, newchannel->max_pkt_size);
-	if (err)
-		goto error_clean_ring;
-
 	/* Establish the gpadl for the ring buffer */
 	newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0;
 
@@ -704,6 +695,16 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	if (err)
 		goto error_clean_ring;
 
+	err = hv_ringbuffer_init(&newchannel->outbound,
+				 page, send_pages, 0);
+	if (err)
+		goto error_free_gpadl;
+
+	err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages],
+				 recv_pages, newchannel->max_pkt_size);
+	if (err)
+		goto error_free_gpadl;
+
 	/* Create and init the channel open message */
 	open_info = kzalloc(sizeof(*open_info) +
 			   sizeof(struct vmbus_channel_open_channel),
diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
index 2aee356840a2..5e014d23a7ad 100644
--- a/drivers/hv/ring_buffer.c
+++ b/drivers/hv/ring_buffer.c
@@ -17,6 +17,8 @@
 #include <linux/vmalloc.h>
 #include <linux/slab.h>
 #include <linux/prefetch.h>
+#include <linux/io.h>
+#include <asm/mshyperv.h>
 
 #include "hyperv_vmbus.h"
 
@@ -183,8 +185,10 @@ void hv_ringbuffer_pre_init(struct vmbus_channel *channel)
 int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
 		       struct page *pages, u32 page_cnt, u32 max_pkt_size)
 {
-	int i;
 	struct page **pages_wraparound;
+	unsigned long *pfns_wraparound;
+	u64 pfn;
+	int i;
 
 	BUILD_BUG_ON((sizeof(struct hv_ring_buffer) != PAGE_SIZE));
 
@@ -192,23 +196,48 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
 	 * First page holds struct hv_ring_buffer, do wraparound mapping for
 	 * the rest.
 	 */
-	pages_wraparound = kcalloc(page_cnt * 2 - 1, sizeof(struct page *),
-				   GFP_KERNEL);
-	if (!pages_wraparound)
-		return -ENOMEM;
+	if (hv_isolation_type_snp()) {
+		pfn = page_to_pfn(pages) +
+			PFN_DOWN(ms_hyperv.shared_gpa_boundary);
+
+		pfns_wraparound = kcalloc(page_cnt * 2 - 1,
+			sizeof(unsigned long), GFP_KERNEL);
+		if (!pfns_wraparound)
+			return -ENOMEM;
+
+		pfns_wraparound[0] = pfn;
+		for (i = 0; i < 2 * (page_cnt - 1); i++)
+			pfns_wraparound[i + 1] = pfn + i % (page_cnt - 1) + 1;
 
-	pages_wraparound[0] = pages;
-	for (i = 0; i < 2 * (page_cnt - 1); i++)
-		pages_wraparound[i + 1] = &pages[i % (page_cnt - 1) + 1];
+		ring_info->ring_buffer = (struct hv_ring_buffer *)
+			vmap_pfn(pfns_wraparound, page_cnt * 2 - 1,
+				 PAGE_KERNEL);
+		kfree(pfns_wraparound);
 
-	ring_info->ring_buffer = (struct hv_ring_buffer *)
-		vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, PAGE_KERNEL);
+		if (!ring_info->ring_buffer)
+			return -ENOMEM;
+
+		/* Zero ring buffer after setting memory host visibility. */
+		memset(ring_info->ring_buffer, 0x00, PAGE_SIZE * page_cnt);
+	} else {
+		pages_wraparound = kcalloc(page_cnt * 2 - 1,
+					   sizeof(struct page *),
+					   GFP_KERNEL);
+
+		pages_wraparound[0] = pages;
+		for (i = 0; i < 2 * (page_cnt - 1); i++)
+			pages_wraparound[i + 1] =
+				&pages[i % (page_cnt - 1) + 1];
 
-	kfree(pages_wraparound);
+		ring_info->ring_buffer = (struct hv_ring_buffer *)
+			vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP,
+				PAGE_KERNEL);
 
+		kfree(pages_wraparound);
+		if (!ring_info->ring_buffer)
+			return -ENOMEM;
+	}
 
-	if (!ring_info->ring_buffer)
-		return -ENOMEM;
 
 	ring_info->ring_buffer->read_index =
 		ring_info->ring_buffer->write_index = 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 13:57:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 13:57:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186760.335519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8vn-0003W2-5D; Tue, 14 Sep 2021 13:57:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186760.335519; Tue, 14 Sep 2021 13:57:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ8vn-0003Vv-1Y; Tue, 14 Sep 2021 13:57:03 +0000
Received: by outflank-mailman (input) for mailman id 186760;
 Tue, 14 Sep 2021 13:57:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ8nN-0007XF-S7
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 13:48:22 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1f259fc8-bb5d-42f2-b1bf-0603caf6fecb;
 Tue, 14 Sep 2021 13:47:56 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18ECrhTg015245; 
 Tue, 14 Sep 2021 13:47:53 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b2sbn8vcj-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 13:47:53 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3715.eurprd03.prod.outlook.com (2603:10a6:208:43::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Tue, 14 Sep
 2021 13:47:46 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 13:47:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f259fc8-bb5d-42f2-b1bf-0603caf6fecb
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HC1XQFKpSH9tYWMISbPCt1iGPDtED/RS6Q1TBwq3Z78PGSEyheYDu0kj/JjlEzIrrrLWWjusDNtxozqJF7R7sVwLDUCNTf5on28KBL/PwLYfgZgrilWNL3/7j7XmI780q6N2YW6Q2fqPoDVrvlugJKgkgIal4YBi4gzMaD5iXPWYcHJO0y2awhVLLi0tH7k8/IwQvMc1MgZ9ou4/B2yF/QW+PogYvqn4cfosP8NN//wiis3QicKh80jbMrsjnIjXOfT3oFXqpj2haOL83jnzW3TUvddTQf2HSlAEqlaVQ/nVkf3F80PEKsRW2cEeLVvy2i2mztTMSN+GVgztgY+p3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OlQJl4qkgARb6YazDMBXh0b5AfvGQIxurXlJnPkf7M0=;
 b=FVls4KvVt5nCbOIFo6nuSI1JDcCW/OmqcZ1pdb/8nND77oIg4kVUVsPB4utYV2ScN5N2lwkN5nQNnSLoPVkGhCTPPIPN+37CkQYngQ4987xDkBhIor9Vb3O7CcJrI05kEk/xJiPdz/6oniJogNsCp1h0S12zfNUnF+TT+FSaxVisPUoW8YcQezwDLxoRwFUNegtO2XKzyudQbxeXwjoJUDewqlti+jDEh8qGQEU49ZD8t5ljGkbE34mlZx0FkOt06wtg1zxxyvc9iza1Ioxf20jpBmhFv5RNA4NqrVKqTZ7NdIWeR4NiKXEjMvHDxAd834EPq2FMpcol9A1cMjUt7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OlQJl4qkgARb6YazDMBXh0b5AfvGQIxurXlJnPkf7M0=;
 b=0GXfrMqW/Y8eR3rrKEzj4VIhbz504WCs+Jh0taemdWQZi9S4zH3di2HNTibBHhhC26rYJSlqimEl+GoIxto+8HHCvNVtMKpVQ/OJFTYsf7iUvZsghXT8etGTcJIcG4hk70cmaOaSkZWufw9AxNsPHKm/SyPV1a6tKFAmGgMSkNANNtcIVO6KmOfiIBj23BRXpp3vpUyJ5zisGoy9jYykKTxsnLw9fxVqDIkLmhXJ03gdqsUNEUAlXgVFZYuD3lM/JdBpkGkFjbkQwkf9UvpaMxoW6/14fpVVQB1JoK3pbsVqjh0++RcLr4o+fS09Q/B3AolpNcYD5Mvg7cSiKrLO8Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Julien Grall <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>
CC: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: 
 AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4ACAABanAIAAAwIAgAABgYCAAAHPAIAAAcgAgAABzYCABkt1gA==
Date: Tue, 14 Sep 2021 13:47:46 +0000
Message-ID: <a4343670-2d9a-cec2-f093-aa3b0a5e0657@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
 <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
 <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
 <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org>
 <52da7955-ead7-f404-6e51-7d1ee7bc3142@epam.com>
In-Reply-To: <52da7955-ead7-f404-6e51-7d1ee7bc3142@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: cbbfd751-a326-4b07-0e7e-08d977863be0
x-ms-traffictypediagnostic: AM0PR03MB3715:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB37159DBD13BA4EC5718E48B5E7DA9@AM0PR03MB3715.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 jdxEoQif0vZzQ42w1kkKgWQpUavAOx4eCT+K/33FEo/RllGt4TcU0ePiXNNSJHlEypSEXfjNEOHdWjjozNPfrnXT9T5/0XRn/tsHaP/pyDxew0aSvDnzGL8KYv27HX+WdkgjcncHj4v4lzBNAwmx1o2ZzXkFByAhE/s+ELBE4Emy5q+20XEZHNiQzqdufW31YqcGVgi/WpaZAhKG5ACMqCNUJPtVf06/zMibnKWvsZ4mA2MhYom3DXGrNAjlGjtW7TsknJVfd8T1X4IavNv94dH5rOyLySlt/cKeB1pqngluOsRkv4wtAAqKCoPxHtc82f5YB1K5op1qp28aFj8uDsxBe+KfuicozEX4VEFvsXhwygpWuBDkkrPF3oEq4UxHFiZo77VtO2xh6unZGwY1Xz7/eOj5TLFgX5vv2/DS+LY5nTfEHYLn9YBnlJD7TArTlN8gM7dWek9h21/l5upmOiOTQkubOAbSb9x7hgboJAflmKWR/cx+yCLNaEOLGRWnvLjQ6D6f4G5hgR9FhKfmLNihfwmXeJeDcmQFRJPG4NIXfKYXGqKIwBDdT444dZA3HRr1eP3WzapZUQiuEqt8mY55XKuWER8QzJ5nBhK6IRqTi/wOUuQI4xr76pTt+f7a1ASYybslxWwdZcmPp4BAQcQQaXpaQMqGlSIlT+eBIla1usf/gbgOxUU1Xuw+HspHZqPMaPWBF+8TxA+bQHqCCq7wAFE4P+lQ1RZtZlJlg1lWtjQe7cF7gxlmzVjeP+jd
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(376002)(136003)(396003)(39860400002)(346002)(110136005)(5660300002)(54906003)(6512007)(4326008)(316002)(2906002)(186003)(2616005)(31686004)(38070700005)(478600001)(83380400001)(36756003)(122000001)(8676002)(76116006)(66476007)(8936002)(66556008)(66446008)(64756008)(71200400001)(6486002)(66946007)(86362001)(31696002)(26005)(91956017)(55236004)(38100700002)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?R0srMlB1MGRpTjNRMGdOait4Q2JPSmVnNzZWYmxqTmpxV2ZhcGpZTWdtVFpp?=
 =?utf-8?B?TmpEL0pId3EreHcvTzBuYVlMT3IvdTV6MnF6SGxYOUdPbEJudHJ2aE5BZkJt?=
 =?utf-8?B?emFBcVA3eFhPVDQ1NlhmeFlWdE9PeFE1YUNEZkdueEZLZVhzVW0zb2ZkUFk2?=
 =?utf-8?B?ZjQ3cjhKNHh2aWkySG5Ic3VwamNiMDRzSHU0OWtyTXlJSG5rY0tJK0dNYWJT?=
 =?utf-8?B?Rm5nTExPcHFMQnp0cUpUMXVvU1hwRE9tcXd6dHRWakxtTVhId09yUnRCS3Ux?=
 =?utf-8?B?K1R1cmRPbDN6UkMwSGF4V0Q0QkhpbS9qQnc1YTFUV2VpQy92S1c0TkV5QkQ3?=
 =?utf-8?B?M1hpS3ZhM2JBOHZWaEVmbkMrWElMRHl4Y0JyTm42T1piODJBbXg4SldVNGZU?=
 =?utf-8?B?aS8xOEt5U0I4ZDJsY0VJaFp2N0FMU2tpMnhwTlE1ZE9kMWJ3azJrc3d6Ylp3?=
 =?utf-8?B?VWRYVzZqV1pVYll3eHZ6S1NXQmRpNHhiSGc2ZGhUT3dlczB1dEYvY1N0OEM2?=
 =?utf-8?B?cURJSnRtTW56aEhCd0twS0FwUG5PUWZVY2xqejNsd3BqMDFkN255Ulp6Qjc1?=
 =?utf-8?B?bDh5SENyclpiOWt4K2w3ZUtVSlpxYXFSR043cmYxLzBQazdoamRyQnd0dVdz?=
 =?utf-8?B?MDhPenZGYmNDWnNhMmVBMFZGajNjbkZNTGdvOEJDVVpFRFlPWnNZeGhQSGsy?=
 =?utf-8?B?T05acEVaYTNRQUtnR0N5cUZBeFZFVnFkK2h3a2x0YUIvczJScEM1MlRFL2lY?=
 =?utf-8?B?ZWpPWW40SXVzL0VtU0JyRUxQSlk5TXhlc3NUTi9LNUJUNDBTaGdKamZZdHh5?=
 =?utf-8?B?SjZEaUkxVktEd252cVVscHo5RGZ1d3ZzUlJndm03ZFp1aHJ3eDFpY0tCTWFs?=
 =?utf-8?B?bkJmc3FNWlgzSEZsdlF5ZlFlUkZzR3U3VjhSS1cxNlpzc2w4ZWxPWitzOC9S?=
 =?utf-8?B?ejNWdGU5K085WVpXRlhDZndveWJKUmMyV2VQTklxa2dGbTVaRjh2Rmw3NGJW?=
 =?utf-8?B?QkpTUmhpWTVQT1NIcExKZzJRSUg0TFhlUEdLZmhkQnFST1pVSXUvZFNQUThR?=
 =?utf-8?B?ZW8xUTY3VlQwNDdPN050SzlzYkRHM0FFcm5lckt2dExDS1VzbzVvQ05SOWhz?=
 =?utf-8?B?dGU4VXYyL3U0VVpBT0hwWnRVaTUycHV3QURNUGRvYlZHZjZYMGt3eEtveUl1?=
 =?utf-8?B?d25Ib2l6STk3Sm1ia1h5Q3ZlZ28vTXZpNWlOWHNEcnd6Um12djIwdTgrU3RR?=
 =?utf-8?B?aXJPZnNPc1k0K0xCazlJancyZ3FZTzlZNHZSdHF3SUp6eHY1UzZLd25xOXJ1?=
 =?utf-8?B?V1lRcG1vQUw1eUZ4Wi9JcGFiM3UwRC9SSWk3R3RLRVNsMU1YRGpQbXJ4RGtD?=
 =?utf-8?B?VWx0UldjbUlhMmI0UVEwNlRsUGkySnV4YVZIQjB3VU05N0kyYXROcUN1ekZR?=
 =?utf-8?B?K1h2N2NEZHBJVXhiTk40V0p2VURRSzZMdm1SSHNBRnBPekhheXBTaU13c3Vx?=
 =?utf-8?B?UlNDazFnSEVNbmJhcGRBNmJzWXpxS01qdStpL1ZHbjV0aGh6eUdFV1dpOGdK?=
 =?utf-8?B?clhaQWY4SUVUM2xCQXRkd05xMEY2N0xYaXlOTTJXSFhxeVV4QWpmREIzOG5h?=
 =?utf-8?B?L0sxaDJHTHh1ekhHOHZqRHNKZERmb3lGV3JTVDZ3Z2dGVXBhVE84ckJVQ0Ny?=
 =?utf-8?B?dU9DN0pYc2JaTDlWdHR6YStKbjNSZjduSVkvTUFjUStNRUVONElBRlpBbVJJ?=
 =?utf-8?B?U1VIdGFpWkp3RUwrSi9Wa2ZqcFE0SzR4cks2Sk91QUZQUHlzM0V3OXJEd2RM?=
 =?utf-8?B?ZlBLRE5qTXBIbmpDVmEwZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C67697F451F1824AB8EB18EE6F3A578D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cbbfd751-a326-4b07-0e7e-08d977863be0
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 13:47:46.2024
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: gjH7mrJgDW03tToE5WOsW4u+mV+a6srzEKV5/84p0QovFBmvnlqmZEHsXDsh3raEuoBViqor6+KzUPrt0GBppGJFdksdNAW6mnt15MJHajpwI+YiVs6uFiq4vxaBW9St
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3715
X-Proofpoint-ORIG-GUID: -I5pmvBhh9NKO4VMjLOMUCPacGLE65HU
X-Proofpoint-GUID: -I5pmvBhh9NKO4VMjLOMUCPacGLE65HU
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_03,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 mlxlogscore=999
 bulkscore=0 clxscore=1015 adultscore=0 mlxscore=0 malwarescore=0
 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140086

DQo+PiBXaGF0IHlvdSB3YW50IHRvIGtub3cgaWYgaG93IG1hbnkgdGltZXMgcmVnaXN0ZXJfbW1p
b19oYW5kbGVyKCkgd2lsbCBiZSBjYWxsZWQgZnJvbSBkb21haW5fdnBjaV9pbml0KCkuDQo+Pg0K
Pj4gWW91IGludHJvZHVjZWQgYSBmdW5jdGlvbiBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoKSB0
aGF0IHdpbGwgd2FsayBvdmVyIHRoZSBicmlkZ2VzIGFuZCB0aGVuIGNhbGwgdGhlIGNhbGxiYWNr
IHZwY2lfc2V0dXBfbW1pb19oYW5kbGVyKCkuIFNvIHlvdSBjb3VsZCBpbnRyb2R1Y2UgYSBuZXcg
Y2FsbGJhY2sgdGhhdCB3aWxsIHJldHVybiAxIGlmIGJyaWRnZS0+b3BzLT5yZWdpc3Rlcl9tbWlv
X2hhbmRsZXIgaXMgbm90IE5VTEwgb3IgMC4NCj4NCj4gT2ssIGNsZWFyLiBTb21ldGhpbmcgbGlr
ZToNCj4NCj4gwqDCoMKgIGlmICggKHJjID0gZG9tYWluX3ZnaWNfcmVnaXN0ZXIoZCwgJmNvdW50
KSkgIT0gMCApDQo+IMKgwqDCoMKgwqDCoMKgIGdvdG8gZmFpbDsNCj4NCj4gwqDCoMKgICpmaW5k
IG91dCBob3cgbWFueSBicmlkZ2VzIGFuZCB1cGRhdGUgY291bnQqDQo+DQo+DQo+IMKgwqDCoCBp
ZiAoIChyYyA9IGRvbWFpbl9pb19pbml0KGQsIGNvdW50ICsgTUFYX0lPX0hBTkRMRVIpKSAhPSAw
ICkNCj4gwqDCoMKgwqDCoMKgwqAgZ290byBmYWlsOw0KPg0KSSBoYXZlIHRoZSBmb2xsb3dpbmcg
Y29kZSBub3c6DQoNCmludCBkb21haW5fdnBjaV9nZXRfbnVtX21taW9faGFuZGxlcnMoc3RydWN0
IGRvbWFpbiAqZCkNCnsNCiDCoMKgwqAgaW50IGNvdW50Ow0KDQogwqDCoMKgIGlmICggaXNfaGFy
ZHdhcmVfZG9tYWluKGQpICkNCiDCoMKgwqDCoMKgwqDCoCAvKiBGb3IgZWFjaCBQQ0kgaG9zdCBi
cmlkZ2UncyBjb25maWd1cmF0aW9uIHNwYWNlLiAqLw0KIMKgwqDCoMKgwqDCoMKgIGNvdW50ICs9
IHBjaV9ob3N0X2dldF9udW1fYnJpZGdlcygpOw0KIMKgwqDCoCBlbHNlDQogwqDCoMKgwqDCoMKg
wqAgLyoNCiDCoMKgwqDCoMKgwqDCoMKgICogVlBDSV9NU0lYX01FTV9OVU0gaGFuZGxlcnMgZm9y
IE1TSS1YIHRhYmxlcyBwZXIgZWFjaCBQQ0kgZGV2aWNlDQogwqDCoMKgwqDCoMKgwqDCoCAqIGJl
aW5nIHBhc3NlZCB0aHJvdWdoLiBNYXhpbXVtIG51bWJlciBvZiBzdXBwb3J0ZWQgZGV2aWNlcw0K
IMKgwqDCoMKgwqDCoMKgwqAgKiBpcyAzMiBhcyB2aXJ0dWFsIGJ1cyB0b3BvbG9neSBlbXVsYXRl
cyB0aGUgZGV2aWNlcyBhcyBlbWJlZGRlZA0KIMKgwqDCoMKgwqDCoMKgwqAgKiBlbmRwb2ludHMu
DQogwqDCoMKgwqDCoMKgwqDCoCAqICsxIGZvciBhIHNpbmdsZSBlbXVsYXRlZCBob3N0IGJyaWRn
ZSdzIGNvbmZpZ3VyYXRpb24gc3BhY2UuICovDQogwqDCoMKgwqDCoMKgwqAgY291bnQgPSBWUENJ
X01TSVhfTUVNX05VTSAqIDMyICsgMTsNCiDCoMKgwqAgcmV0dXJuIGNvdW50Ow0KfQ0KDQpQbGVh
c2Ugbm90ZSB0aGF0IHdlIGNhbm5vdCB0ZWxsIGhvdyBtYW55IFBDSWUgZGV2aWNlcyBhcmUgZ29p
bmcgdG8gYmUgcGFzc2VkIHRocm91Z2gNCg0KU28sIHdvcnN0IGNhc2UgZm9yIERvbVUgaXMgZ29p
bmcgdG8gYmUgNjUgdG8gd2hhdCB3ZSBhbHJlYWR5IGhhdmUuLi4NCg0KVGhpcyBzb3VuZHMgc2Nh
cnkgYSBiaXQgYXMgbW9zdCBwcm9iYWJseSB3ZSB3b24ndCBwYXNzIHRocm91Z2ggMzIgZGV2aWNl
cyBtb3N0IG9mIHRoZQ0KDQp0aW1lLCBidXQgd2lsbCBtYWtlIGQtPmFyY2gudm1taW8uaGFuZGxl
cnMgYWxtb3N0IDQgdGltZXMgYmlnZ2VyIHRoZW4gaXQgaXMgbm93Lg0KDQpUaGlzIG1heSBoYXZl
IGluZmx1ZW5jZSBvbiB0aGUgTU1JTyBoYW5kbGVycyBwZXJmb3JtYW5jZS4uLg0KDQpUaGFua3Ms
DQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:22:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:22:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186790.335530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9KD-00077i-80; Tue, 14 Sep 2021 14:22:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186790.335530; Tue, 14 Sep 2021 14:22:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9KD-00077b-40; Tue, 14 Sep 2021 14:22:17 +0000
Received: by outflank-mailman (input) for mailman id 186790;
 Tue, 14 Sep 2021 14:22:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQ9KB-00077F-Hf
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 14:22:15 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5441d78c-5b15-47fa-b41c-0f106c12f905;
 Tue, 14 Sep 2021 14:22:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5441d78c-5b15-47fa-b41c-0f106c12f905
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631629334;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=9BmDetHKCQXTnm4i/GilQXirlKvyh5ZfduiM9Mo3+tU=;
  b=G9JWjHy5/5+P3pBDaX6tRi33MEkt1/yyCk/eTasB3VnYjh76+VpO8EDy
   AHxbRKkdshS2SbkJYvuEXN8VT7dtrtbyPi5CP9xWI9hsoxOkA+MDUlQUe
   KooOS3a4BWgxWPKUMKBOE7/O0ZS59xYmaiXRJzCZJZuXLP8OkS4x5ct7d
   o=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: rdj6xcF5TXdN6VUWrTJUw5+NrOS/HQVoAywAn0CynvTvLVRH3RAHeijlfGvVCaWYANbntBMpib
 b9oTXHu5kDjyhs5VCNu9RyI1kckZElh8eP2hlDM4NZRBG4sxIy5UB0PZVIEcEn3qDUY5CyYdeK
 Y8l9cBkQkdlX3FNxTbcs0eOzan8eExRLeeByU5AyDAMxwzwolYpJJM8pfdduIah/O3EN5gDvIG
 6VfbVLo/UxJUpQty3VzOmNrV1HLnJv+xMjm+4LFpQ5fJlZBTv4jmcUXaUD8RsHLbkxxEhisczP
 UAA+AuMVSBU4hCxHnlj0APTI
X-SBRS: 5.1
X-MesageID: 52702693
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QfTsGKLznZ0uVsKpFE+R+5IlxSXFcZb7ZxGr2PjKsXjdYENSg2EFn
 zEeWWqPP//eYWf1ctAjaYvn9RkC78PSzt42TlFlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6x7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB20vO507
 Il0uaaxEw0HAbaQwu0QFAlhRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2Xv4AAhmlh3aiiG977V
 ok6OSgycS6QSDpEAUowN40Rhcen0yyXnzpw9wvO+PtfD3Lo5Bx81v3hPcTYfvSORN5JhQCIq
 2Te5WP7DxoGctuFxlKt8Hihm+vOliPTQ58JGfuz8fsCvbGI7jVNUltMDwL9+KTnzB7lMz5CF
 6AK0hcNk60WqHWndNinVTOHuyLetxNbS+MFRoXW9zqxJrroDxexXzZfF2MQN4N47afaVhRxi
 QTYxIqB6ShH9eTPECPDrO/8QSaaZHBNRVLucxPoWufsDzPLm4g1khuHZdJqCqfdYjbdSGyon
 mziQMTTgdwuYS83O0eTpguvb9GE/MGhousJCuP/BDzNAuRRPtLNWmBQwQKHhcus1a7AJrV7g
 JThpyR4xLpfZX1qvHfWKNjh4Znzv6rVWNEiqQc3QvHNCAhBC1b8JNsNsVmS1W9CM9oeeC+BX
 aMgkVoKv/du0I+RRfYvOeqZUp1ypYC5TIiNfq2EP7JmP8kqHCfarX4GWKJl9z20+KTaufpkY
 snznAfFJStyNJmLOxLtG75GgON0n3tnrY4RLLiipymaPXOlTCf9YZ8OMUeUb/B/66WBoQ7P9
 M1YOdfMwBJaONASqAGOmWLKBVxVf3U9G77srMlbKryKLgZ8QTlzAP7N27IxPYdimv0NxOvP+
 3i8XG5eyUb+2iKbeVnbNCg7ZeO9R4t7oFI6ITcoYQSi1U88bNv996wYbZY2I+UqrbQx0f5uQ
 vAZUMycGfATGC/f8jEQYMCl/oxvfRimnyyUOC+hbGRtdpJsXVWRqNTlYhHu5G8FCS/u7Zkyp
 Lip1wX6R5sfRls9UJaKOaz3l17o5CoTguN/WUfMM+J/QkS0/dg4MTH1g982P9oIdUfJyAyF2
 lvEGhwfv+TM/dM4qYGbmaCeoo61OOJiBU4GTXLD5LO7OCSGrGquxYhMDLSBcTzHDT6m/ayjY
 aNezu3mMe1Bl1FP6tIuH7FuxKM4xt3uu74FkVg0QCSVNwymWuF6P32L/chTrakclLZWtDy/V
 l+L5tQHa66CP9noEQJJKQcoBghZOSr4RtUGASwJHXjH
IronPort-HdrOrdr: A9a23:d64gLak9pdlzOlSGaDxaC4J/H/TpDfO2imdD5ihNYBxZY6Wkfp
 +V88jzhCWZtN9OYhwdcLC7WZVpQRvnhPlICK0qTM2ftW7dyRaVxeBZnPDfKljbdREWmdQtt5
 uIH5IObeEYSGIK8foSgzPIYurIouP3iZxA7N22pxwGLXAIGtFdBkVCe36m+yVNNXd77PECZf
 yhD6R81l6dkSN9VLXFOpBJZZmIm/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTuj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZsA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQvZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv0nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLVU5nghBgt/DWQZAVwIv/fKXJy//B9kgIm00yR9nFohPD2xRw7hdYAo5ot3Z
 WzDk0nrsAIciYsV9MOOA42e7rBNoX8e2O+DIusGyWTKEgmAQOEl3el2sR/2AmVEKZ4uKfa3q
 6xFm9liQ==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52702693"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aN9SdOWQBI7himeFWWuql3zozSG5OlxnxX3b1CzAhM6cZqKIxGEfw4JFAhAU8im8GPEnT54VpX0VnOe8An8LCwOiJpvm8ZggVp/Lnw5eOwUW2Fy5f+k9PkqESPSARP2R9Py3JWue60uj5puNfhRyk5zBKMFq1jQ2mTwiw17N1rTBKpsIYDGlOhl5PVwiQiGEKkf37JuOj20YmDzsG53fUZ/hkRDCNG1xXhLBMRAv0PiW0AwXANY2elQ+4BKOQ+P0fYXtUYZLrrmpxowZtdrtzbvsiIagViNWNqcE3oRf/oS1xjQTNxCL5DNR1bGxMZtW9GY8ngVx+RU68aoeaKEQVg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XicimkWAG4CtDkeE7vKpUgbHBafeFsFVkiFg74ln4wU=;
 b=R6BeZxWKraIWYi1AU4xHP3VdT2wNGJjEdgRg1ChQiMc1YC679FCedAcGY1EXW6LhUEOeVNqhNmhexnboUhTTy5ZXrN9dXvx/iRcAoTOIDHIrW1z8Zt3W4fCGWSaD7ks0MC/Y3gL7MsWljbnAhfYIoVd3UJXVXM37mgVpzHPkcpiIkNREI0NgfsFxXo77NQ1lQuee+S+JxvdkZI+iLpmd3fCXH/SfRgl4cVxQYaiAa9hE1JM4IEObol8ISfGTmwMkf37TO0d2czPPJBmNiGSYe1XXcu1HOzJUOKdZamCbEwgPaifAbuHB7C1dGQxYE4Un0489fTKd23yxz3SNh00hzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XicimkWAG4CtDkeE7vKpUgbHBafeFsFVkiFg74ln4wU=;
 b=svnySeXtcyWJ1HWOALxP37Tfrlrk/VTm0nGBxgQ0b7trnMFDErbc3IPiTWxOSeuKU+GokRiv6Q6ROHbmlKhLMn9gTCKLDIuPu5aI8dgrUpHuXVKjYG63bc+CQNxy2dhKGm15zgd7TNhEehbajsis/YVDRNR4hlcyi8JGBjYMEKQ=
Date: Tue, 14 Sep 2021 16:22:07 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
Message-ID: <YUCwD5RWj0WKZZUc@MacBook-Air-de-Roger.local>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
 <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
 <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
 <df432c93-b722-6006-c3dc-7f511b377bcc@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <df432c93-b722-6006-c3dc-7f511b377bcc@suse.com>
X-ClientProxiedBy: LO2P265CA0089.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8::29) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d66fd560-d261-476c-2bc2-08d9778b0a8f
X-MS-TrafficTypeDiagnostic: DM5PR03MB3068:
X-Microsoft-Antispam-PRVS: <DM5PR03MB3068F9238678611B493A6F738FDA9@DM5PR03MB3068.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 5Vg4aCD/Bn6kjBwxi0HWmDQ4Ro9z4Lgpk9IpzmJQRzDrnyjaQETYhsuJzipjo3wssr4YUFuhLIdybR7OWlqg7Vv/bWketCVvnVEBTXDl3rnusdZlz2Y3QnIYMyd7BBPY54ifL+dSLb1+jLCl5X1jgJ78VwOW4oq8oH7EXxDb6q80V2vVltXuJu1rURkZfNobAU36TsiTn8yYgOSIMPkBIEIKVQQHyQFXnU1oikhSBeXFqva9D/yDRPDMwxt82H9OcFFbXULT8k8xoQeDOj74BA7EFTK+2zPFx6IFHZt109YlkdwiNLtMS+S6l4aFpAs89f7F3JRGYR1FrkrJbIDDq292ytk37KW0hrfnAxvrp/uMZqQ/o5OjpHFGgJ0kEqAvu2qQ89G8ZfaWJ5QWQfca7wVvxpV5rQfcGHsR6IKZby+eSgr9QjsGw3zleOu+gdVREJMguhbCsl3Coovfb+8+RQwcjrKKSdXxCUYFlpclAOV9FlEtWUU2bpULd8zh6ZXYTjSc1Fq228G4+w1NTAFuHQ/+6HmBEIyaWFRLW+GSxixGkrnHR4bEzYT1Y4QXTnpwy0lqD1i1CSMc5igDckLWQPCh2kBqLM5SIYZA9FA+5uzAmngkHJkAn6FHwmoS21AdZHozQbudoMR06MhOmdoMQw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(38100700002)(66556008)(2906002)(8936002)(66476007)(9686003)(86362001)(15650500001)(186003)(956004)(66946007)(316002)(6916009)(6666004)(6496006)(508600001)(85182001)(8676002)(5660300002)(6486002)(83380400001)(53546011)(4326008);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cmdSdDRmQmFTcVZOZERtcGNVeTI3WlRxc0ZvRlBLRXU4Sm1kbDlWRHF3TjZK?=
 =?utf-8?B?cytnZkJVTHExV1BGNkFId2VYanVKcXVBcVhkUE5CV1hXakNJSi9qV3JhYmJX?=
 =?utf-8?B?ZzdveXhJa29Ba2Fhc2poMUtkWGh3WXBaZjlJRWxlVllQSEdoZWQ2OGU0b2cz?=
 =?utf-8?B?ZjNDdkxsaFByN2NFdy9ibnpyYTY3MUhLdncvVFBsMjZwRFNlei9ka29YeGta?=
 =?utf-8?B?dHpBOXV2M1Yxc2R6WEduWDNRdnZmbU1aZ04xZ1FxeFBrempGN0hsWjVuRE82?=
 =?utf-8?B?V1RZTEpjTktISWllYnRBYjR5eGZxU0UrTFBIc1dVUXcrVDIrd0tZM003NmVh?=
 =?utf-8?B?YUdlR3Y0dWptVFBhaEpERml2Q1NvQzdRcHl2c2ViVnl0Vm0yTzV5ckhqOEpv?=
 =?utf-8?B?bVVKTnprSkltNkQxbkRHdWVURGVxa3ZTdS9ZNlduU1lVeE1MOGZ0OTF0Q1o0?=
 =?utf-8?B?Q1c3MGs2N2FSeWxEZzVzRUZEUW9vZGZRTGRBR054bEh1elpLaVNhSEJyT0tK?=
 =?utf-8?B?WmdacFNDeWFyRHRQelIxTDNDWGhVVkNsRjhwV2dyTVM1bmdBdHN1dXh1TndB?=
 =?utf-8?B?bm9rRVVpdzNLdUNQK20zSkl4TW9hQzFOREt2aXR5alV1SG11cFZReDRYZWpI?=
 =?utf-8?B?OXFLWk0wTnozTXUrdDZQbVF5R05UNEhZOG5yRnYzRlFVUnBzY0d5cWJHb2xp?=
 =?utf-8?B?M09IOXJza3JBYkkrNEgzaGp2clMrbDJIWitjekllWmwzSDRmZWxWYnZYUWMx?=
 =?utf-8?B?TFN0OFE2Q1FLOXl6L0VkMkRKR0tnK1FjWmRGKzFHRVhzWnJYOE9BOGVGcExQ?=
 =?utf-8?B?S3ZWZzlnU1BZdnVOU1ovb0lYb1o2N0ZONDM0ZUZhVEJ3cGVJVm5xODZrdVdE?=
 =?utf-8?B?ZzIzQzVlVnlYc1FXdzRTNHpTZVhHMXI5bVIyTUE5R0NlZmVCa0laa0lTVlQ0?=
 =?utf-8?B?N2V6WFRIbTdtWERwKzdtS0RTWjIxUEs3Vk9NbnI4M1M4SldYY3ZEZTZrWHBi?=
 =?utf-8?B?bTY1MUNUZmtLcEhYVTgwQ1JuTGJidUI0Z0NIWFZJclpsdGZ5bmtxZHdva3J4?=
 =?utf-8?B?TVFWQWlzaWk0ZWVsVllDUE1qNG9nM2VEdWlrczNWZWxZRFUvM3JkQ2RhV0tT?=
 =?utf-8?B?R0JPRDVab2hWeUZ3WFRMTVNqNDBybGZWZzhqT2QwMFRyQ0pZdW9UZUhTckkx?=
 =?utf-8?B?SzJNY1pOb2g2NVlNNDZka0tUcnJCODhwQlR4SnVMbzFUSWNhMzc0NUFGQWxh?=
 =?utf-8?B?Y1BPWWRzeDE0Rnh6NU5PVzc2N0xSSDFhMXFzb1RzQ29nYmQwaGlHRWg4d3ZD?=
 =?utf-8?B?TWphTHFSVlVFeXRVZm1ZS3FIZm0xalV4TUd6aFc3UWxZZnF2ZHd3L2pwUWFV?=
 =?utf-8?B?MzZpTGtwNUxYOUFNV29ndTNpSUJ1M1doaFUxZ2ZzRDhlbHJnUXpxSVV1WGhv?=
 =?utf-8?B?VDhJQmFMUnN1cmhPYjhmWWE4aDhKeCtIWWQ3N25xZTVLVWR1bSs2aFFXNE43?=
 =?utf-8?B?dnY5Yyt0a015SjFSTTNzOUYwdEdUbXo0NHY1bEFybTJoM2tQUGZCL05xSmZP?=
 =?utf-8?B?SHR3UDhodVI3TFZqUTROTDlNOEhDT1hyZm9zcDM3c2tMK054Rld6VUJza2da?=
 =?utf-8?B?aGFkMVNtV2ZYcnVwOTh3TkNXOHJ3c0lHbGpnTW8zU3FwVzlpVnQwbytaYU5Z?=
 =?utf-8?B?Ukh2ZFBnSytHQWZzYWduL29MdW44VXJzb1RsbE9OZ3Q1MlY1YWlqZERNeDJB?=
 =?utf-8?Q?m2bgVOYDUBpKmvqt6bi+m3LpCovPkCuWzqZeOAs?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d66fd560-d261-476c-2bc2-08d9778b0a8f
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 14:22:11.1826
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XBJJbnGKgqSBDTZCSnRA0tl+4OHDczxPdk/NZCk0N6QBQSshs4B4LyjARTlEkUMJz51XxnYMUCOkaLdqq/cE3A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3068
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 02:05:01PM +0200, Jan Beulich wrote:
> On 14.09.2021 13:21, Roger Pau Monné wrote:
> > On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
> >> On 14.09.2021 12:00, Roger Pau Monné wrote:
> >>> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
> >>>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
> >>>> console) are associated with DomXEN, not Dom0. This means that while
> >>>> looking for overlapping BARs such devices cannot be found on Dom0's
> >>>> list of devices; DomXEN's list also needs to be scanned.
> >>>
> >>> Thanks for looking into this, I certainly didn't take hidden devices
> >>> into account for vPCI dom0.
> >>>
> >>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>> ---
> >>>> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
> >>>>      would make the diff difficult to read. At this point I'd merely
> >>>>      like to gather input towards possible better approaches to solve
> >>>>      the issue (not the least because quite possibly there are further
> >>>>      places needing changing).
> >>>
> >>> I have a couple of questions, AFAICT we currently hide the serial
> >>> console and/or the VGA adapter if it's in use by Xen.
> >>>
> >>> I wonder whether we need to add vPCI handlers for those:
> >>> setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
> >>> device because of the temporary override of pdev->domain done in
> >>> _setup_hwdom_pci_devices, but I think that for hidden devices we
> >>> shouldn't add vPCI handlers. We should instead block PCI config space
> >>> access from dom0 to the device so that it doesn't mess with Xen usage.
> >>
> >> The answer to this follows (I think) from the one below.
> >>
> >>> It's also not clear why does Xen want to have those hidden devices
> >>> partly controlled by dom0, as it would seem to me that dom0 interfering
> >>> with hidden devices in use by Xen can only lead to trouble.
> >>
> >> Dom0 can't interfere as long as it can only read from the device.
> >> Restricting accesses to reads is one of the purposes of "hiding"
> >> (the other is to make it impossible to assign these to a DomU). Not
> >> allowing Dom0 to read from such devices would lead to wrong PCI
> >> device discovery - some devices would be missing (which may or may
> >> not be merely a cosmetic issue). If the missing device is a multi-
> >> function one at function 0, other devices in the same slot may also
> >> not be found by Dom0 (depending on whether it looks at non-zero
> >> function numbers in this case).
> > 
> > Hm, indeed seems possible that missing function 0 the whole device is
> > skipped.
> > 
> > Maybe we need a special vPCI handling for those devices that just
> > allows reads but not writes, and that doesn't maps the BARs into dom0
> > p2m?
> 
> Not sure about mapping. They could be mapped r/o. And they may
> actually need mapping for multi-function devices, but I guess for
> such devices to actually function properly then there would be
> more work required.

I'm also slightly puzzled as to why ehci-dbgp uses pci_hide_device
while ns16550 uses pci_ro_device instead.

Is this because the PCI device used by ehci-dbgp must be shared with
dom0 for other USB ports to be usable, and hence dom0 will need read
and write access to the device PCI config space and BARs?

Note that pci_hide_device is less restrictive than pci_ro_device, as
it doesn't mark the device as RO.

That would seem quite risky, as it's likely that dom0 will perform
some kind of reset of the USB device and thus the console will be
lost.

> > Or could the BARs potentially be shared between multiple
> > functions on the same device?
> 
> I don't think that's allowed (or would even work) - if anything
> (like in general) small BARs may share a page, but without overlaps.

BARs sharing a page should already be handled correctly by vPCI.

> > This also makes me wonder why they have to be added to the IOMMU, is
> > that related to device groups and the fact that Xen is not clever
> > enough to figure whether devices belong to the same IOMMU group and
> > thus must be assigned together?
> 
> I'm actually not sure whether they need to be added to the IOMMU.
> With multi-function devices in mind, back at the time I though it
> would make more sense that way. But I may have been wrong, and
> hence we may want to revisit this. Problem with such changes is
> that they may cause rare issues on very specific systems, which we
> may have no way noticing even during a full release cycle.

I guess it depends on whether the device is to be used by dom0, see
my question above about ehci-dbgp for example.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:24:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:24:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186800.335540 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9ME-0007oW-Mx; Tue, 14 Sep 2021 14:24:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186800.335540; Tue, 14 Sep 2021 14:24:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9ME-0007oP-K5; Tue, 14 Sep 2021 14:24:22 +0000
Received: by outflank-mailman (input) for mailman id 186800;
 Tue, 14 Sep 2021 14:24:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pQww=OE=epam.com=prvs=98919bdb76=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQ9MD-0007oJ-Kc
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 14:24:21 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7363ea56-1567-11ec-b43a-12813bfff9fa;
 Tue, 14 Sep 2021 14:24:20 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18EDwUrC018725;
 Tue, 14 Sep 2021 14:24:14 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2053.outbound.protection.outlook.com [104.47.14.53])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b2qp4sbbg-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 14 Sep 2021 14:24:14 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4577.eurprd03.prod.outlook.com (2603:10a6:208:c1::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep
 2021 14:24:10 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 14:24:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7363ea56-1567-11ec-b43a-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vd5jllhtNaGZPhYrqBKLXsTRdsj3StCRczJeXlsurJUYNp9NXtMId6KcpMZH9DHuVhv0/n46ExxKDVVaJW1S0+H5TGcgXbDg6rPJ7VNhw6acXIJFuEa1VK62CxXY5aLeFYE+d7NGyRmYbWcjOTTw9GRWEqTTa/ffEtCm24dYfZsmSzlFlNgFVn54XFnQeZFtKM5d/1Pv9NC8UmdVw5Dxbf++A8Klv4M5Jdi8EcVJz+V/IN7zviwZyGYH++4FEm4C8HSx5L7G8MHjVzRIiR/ei4mwiC6yJZ3+S9SVqJIbZdAAHrnmzaPunHFAJwS75lO1q/tybIbeW44dRuG7VxkMog==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FZ8QhArfeYPBGE9lk/pv9xQcsubY+TFE+qbmLG6HSek=;
 b=CpiHIatoFmxR3k+0D6DSJHj521RjvncNlB6mzGQ0qFkOxkm0AeynWRhlju1il3HVfTyuHgGIuqKXYEvqwzV17TEwuuks5Zqw9YeFerREUFm0hdLMRUk00hjVIIUMEMzPMMvXn6/Q7bHJjyPGntAGHAzaKlv1p1riaBoV6CZCKSMJmBAgFAnvsh5hhJVeelfBwRWL9+SCOy2tFd42Mbv8IdWbnbBGJw9il2orPMJRaOG/BnAl2+nYXxAKZRKfg9TGHXB23K6FZ+HHFTVBIO8cXwmALw2/ZSkl/KhZZ/h8RJn+agholICk+DAZxhxiS8V2dUILHSNk5fctwi+VVnj38g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FZ8QhArfeYPBGE9lk/pv9xQcsubY+TFE+qbmLG6HSek=;
 b=0ETvuKK+AptJF4bG9lfMS2o2zE3wNheb0bxnOCwUbLPjeMNOpn6WD13ZY8SzX5DpvpYuhn5THbuYr59jQdx4YIB13TcQ+mYj0mnEnBb//5sdlGc9//X6U9unogyoA9e5EQHn6zeLZyYthU8F7xUX7YAZeW5+3vusCvx4DXNEeqNBK7p4DBqsTCshm+j+89lAG+vq+gl7PkkdFcxfzjGwrqldz4UPD4RpNtPpDcfEHsLaq7bWTIzt1ikW3GKHy+dkh1cJ+HsJ9LpkHSgzViOEsNudG+YfIkXZroKidc4QE4yQJknBiyfANGjVEu12D79Of/70Dm3OxfHLYUTqCHBe1Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Rahul Singh
	<rahul.singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKudvyQAgAXn/AA=
Date: Tue, 14 Sep 2021 14:24:10 +0000
Message-ID: <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a004ef87-ac4b-429a-7985-08d9778b51ee
x-ms-traffictypediagnostic: AM0PR03MB4577:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4577A73FD229484106A42C21E7DA9@AM0PR03MB4577.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Hl4VbB50cC2oV+4PzMe5ex2Q/vRR63KeYhqMqP1s1qStz1MBhaq3g6T+VPbOgz7570EEYuRJb9bXvvV65lXHhHaDig2nhv1ILlv+QZX08eD+46Sq7XTl5AMfAa8G74Ob8tGT/kkzC+jidgd0YQIGeKKWkOl1VpbSMh5zhq3/gQ1zWtV0k63qETolN2ikvOJ98QWht2+5SiwB/nV1l2QSOpFXF1/P9uRUsMo2TN/2fIc8MskKHCb4jKj3fSyGXpYgf8yaKQkdGwAlfblyiGTu8xVvhPaYPEgwX9i7aepCRST9EVwUXfvF0+rCMxbh6qN1AZEpJA/YEl5B69/4NQToZS2hdzqGbIHbWgWjyrgFpJvrgF5JtIUSjty2LmCcH8YAKlT9mgKFCE6p5ne7F27fS+EPVSTyO7z+AHilOnLafgURU8tcXH4dZY9GzWAEltkB33l6wyqHhs+Y/u5mGEoOFKCruxhkJaG+JTECJPfxPnZ+0OSIy/s+mHUSVD7RegV8+Ql4Fwb7jXLjqGOGC+6lfCIryUoHTsQiAkNadVuQ7s76xvcais/bZmZxQepMXSyWOzbmr+J4vglMmCA4mEZYeEOPnsofGPUzTAljDZZwfrEMjFxAN28M4sTW1JUzsx4WtG6QMTa8U53y229Joyeq6mXuFHR4/iX2Ywbhgy3RAclMEpzNOrHhC9S93OKyTrSHyv1csEb9q93N/3QDkxpoGYf/vuOjv6D5dhuzMJsk4SmUgj/0f6+F9X96o0bGvgle
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(366004)(346002)(396003)(376002)(31686004)(478600001)(26005)(86362001)(5660300002)(31696002)(4744005)(2906002)(4326008)(6486002)(71200400001)(122000001)(6512007)(66446008)(66946007)(8676002)(76116006)(6506007)(64756008)(91956017)(55236004)(38070700005)(110136005)(2616005)(316002)(36756003)(186003)(66556008)(66476007)(54906003)(38100700002)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aXhCS2kxWG5uWUN1eHdabXlYZHZxb1hmenBoSkRhN0s2MlhGK0hmc3JTRmNC?=
 =?utf-8?B?TzVFV1pacGxBdzloS1BXbytDZXpST0hSdXp0N1phNXhuNmhDQmFFL3FXcnlt?=
 =?utf-8?B?enpsSUNHaldJY1J1NHIzYWd6TmZsdURCaWlMTzN0YWdacmxncDFsTVpYb0k3?=
 =?utf-8?B?QXAvbmFRbStYeFQwc1NFWmpLeWc0VXVaWHAybG9wNlM5VkdLNnV2elRPeDdt?=
 =?utf-8?B?azNJVkZzdVJmZ01YZzdMcS9oSElJNURDQThBQUNFckpLYWlVdlVtUmtFcHRu?=
 =?utf-8?B?dFJDZFJYaUxpbStjZDE4d09Sd0JYb2NVV0hwOC9reUtOcVY1R0ViMENvcjRG?=
 =?utf-8?B?NmNSU001c3ZJeWZSVXFJaHRTcTdsaWRuVzRtcklkbWd6NVlLNks1eTNvVHpT?=
 =?utf-8?B?SGlGWTlIZXQ3NTNSZEdidWthSkF6NFp2Q0hHbXhaQXhLYTJJSWVPQWdBc3Bz?=
 =?utf-8?B?OFJNSUdEd1d4a2VTNkg0MDYyYjZPT0ltY0hNa20wQ21RaHQzVDNkVndEZEVi?=
 =?utf-8?B?azdnVE1qNFpNbWoxcVo0bG43bDZJVGVBU2dtV09ZK1VhbktOTmNIWjFqeEJo?=
 =?utf-8?B?enliM3lqcElTZ3lwTWNBdGNEWkREVFl1WjBlOEZhdDBRRFRVK2JUc3NEMjBm?=
 =?utf-8?B?b3p0M084bklhK3JvSWV5UHg3QUs5RjN6bXlRWVJtdGxVRS9iRzBZVDZwL0du?=
 =?utf-8?B?RmpwRlBNU3doclp2c0twT0tzd1dhRHlybGdLOFh5ZjI3aG9FaExJbE0vMFpz?=
 =?utf-8?B?dmw2cTc0MzRmVzE4Y3BUSU45NXJ0UWorano5eFNiWG4ydXRyY1YyWW1WM2dr?=
 =?utf-8?B?Nk0yWTRmY1BIU1djR3psVDlPUFhyYTFtSU9Id0VGa0tyNHFjZERKOG9NNWxl?=
 =?utf-8?B?NXhHVThlaFNTWjBlRnhrRm5wNVBWMHg1UHhFczN6TklmQ3hyUzBPc2lrNmxm?=
 =?utf-8?B?ZUFJM1YvNG5oQndsNjhHZlAvZlAyT0VBYWdVeWV1VWxOWVdHWURCQW4zSXov?=
 =?utf-8?B?Yk8vNlVDSWpNRWgwY3k0YkltQVFGNHBISkhjMVpyWVd4VjhzeS8yclVCcURG?=
 =?utf-8?B?ZkR2eDJnVHBvenVuUHNqVTgxUHFRZSt5eXBQSzVKckppRjcwYVd2SkdHV3R0?=
 =?utf-8?B?MjJvdnJkZFBFSEp3SGFGNEp3VkFUN1JGLytocDRqNUg0TFo4RmM2VEs5TGRi?=
 =?utf-8?B?TlZoOSs1WEtJc2VaT015anc3WUZSUjBHNm1FVjBUL1hKVVlDeVl1YWJlN3RL?=
 =?utf-8?B?aHQxdlZkZjZ4U3J0UHVGVWZWVWYzeXE0MEpGaWxRTE91QVJHSmVCbGxWaWtU?=
 =?utf-8?B?UDV4R3pWNk9RcDJta0dGTG91NGpLa2xWSEZPNVNmN1Q2U0pkdVFwclByNklS?=
 =?utf-8?B?OS9xSlRzSkpqV0cyQkc1REsxV2puN29yL0FCaXFSS0hpNmFJbDBBL0RWR0Y0?=
 =?utf-8?B?UkhaTDhERGdmTXFEQXV4Z2RVTy9oU3c3UWdqSmZZVWNIb3J0c1RLY1gwbktN?=
 =?utf-8?B?enJMaXFoRlNVSDR1bWpRcFF0Wm5Jam9IeFZxN0NUK2o1MWgvalJmVHpIWTZa?=
 =?utf-8?B?Y0lEQVVlTUpMOU54KzAycWFyeDZVZEhxVVpiUjk1cWU5anpkRXE3bjBMU0JH?=
 =?utf-8?B?TUk5cFAyQmJaOWpmb3RYSGMrOG9lMXRMTFAyQUdNanRhUndVU0FMMXBUYW1N?=
 =?utf-8?B?ZW5udXJaMDZXbnFSL2Fka1RHbW95QkpqTlp3SEZUNGM2SGZsMnh0aHJsNFBi?=
 =?utf-8?B?NEJqQWlQNG9EdGIwR3JpcUc5emhpZTFlZmVneGNpV1dIRjd0T2NFTGh6RUx6?=
 =?utf-8?B?dCthYTd3S2pHWXY0RWt0QT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <BDE03ED3F5F89A4A8F5C97D4FFA4D04C@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a004ef87-ac4b-429a-7985-08d9778b51ee
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 14:24:10.7143
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: oLVWpVT5ABBx7rnkeAeIN7F6yfdU+xjVoQuFiQJbgDVY2xWkZOAijFGpIhISKZlojKXZnEWEdnlAng1JZe+iheQ/2gKDOvJrxpOzKJkPzPygal2gIdldP/nacq5ZdNCO
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4577
X-Proofpoint-GUID: UAg8ZOV1Amfvid24SK2wHPVkF-z6c9YM
X-Proofpoint-ORIG-GUID: UAg8ZOV1Amfvid24SK2wHPVkF-z6c9YM
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_03,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 impostorscore=0 suspectscore=0 spamscore=0 mlxscore=0 bulkscore=0
 phishscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 malwarescore=0
 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140089

DQp9DQo+PiAgIA0KPj4gK3N0YXRpYyBpbnQgcGNpX2VjYW1fcmVnaXN0ZXJfbW1pb19oYW5kbGVy
KHN0cnVjdCBkb21haW4gKmQsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4gKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBtbWlvX2hhbmRsZXJf
b3BzICpvcHMpDQo+PiArew0KPj4gKyAgICBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9
IGJyaWRnZS0+c3lzZGF0YTsNCj4+ICsNCj4+ICsgICAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQs
IG9wcywgY2ZnLT5waHlzX2FkZHIsIGNmZy0+c2l6ZSwgTlVMTCk7DQo+PiArICAgIHJldHVybiAw
Ow0KPj4gK30NCj4gR2l2ZW4gdGhhdCBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgaXMgZ2VuZXJp
YyAoaXQgaXMgbm90IHNwZWNpZmljIHRvDQo+IG9uZSBicmlkZ2UpLCBJIHdvbmRlciBpZiB3ZSBl
dmVuIG5lZWQgdGhlIC5yZWdpc3Rlcl9tbWlvX2hhbmRsZXINCj4gY2FsbGJhY2sgaGVyZS4NCj4N
Cj4gSW4gZmFjdCwgcGNpX2hvc3RfYnJpZGdlLT5zeXNkYXRhIGRvZXNuJ3QgZXZlbiBuZWVkIHRv
IGJlIGEgdm9pZCosIGl0DQo+IGNvdWxkIGJlIGEgc3RydWN0IHBjaV9jb25maWdfd2luZG93Kiwg
cmlnaHQ/DQoNClJhaHVsLCB0aGlzIGFjdHVhbGx5IG1heSBjaGFuZ2UgeW91ciBzZXJpZXMuDQoN
CkRvIHlvdSB0aGluayB3ZSBjYW4gZG8gdGhhdD8NCg0KPg0KPiBXZSBjb3VsZCBzaW1wbHkgY2Fs
bDoNCj4NCj4gcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsIG9wcywgY2ZnLT5waHlzX2FkZHIsIGNm
Zy0+c2l6ZSwgTlVMTCk7DQo+DQo+IGZvciBlYWNoIGJyaWRnZSBkaXJlY3RseSBmcm9tIGRvbWFp
bl92cGNpX2luaXQgPw0KDQpJZiBSYWh1bCBjaGFuZ2VzIHRoZSBBUEkgdGhlbiB3ZSBjYW4gcHJv
YmFibHkgZG8gdGhhdC4NCg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:33:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:33:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186810.335552 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9Uc-0000y0-Fj; Tue, 14 Sep 2021 14:33:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186810.335552; Tue, 14 Sep 2021 14:33:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9Uc-0000xt-Ch; Tue, 14 Sep 2021 14:33:02 +0000
Received: by outflank-mailman (input) for mailman id 186810;
 Tue, 14 Sep 2021 14:33:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQ9Ua-0000xn-Bt
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 14:33:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f409a10-b09c-447b-84dd-8110f40ca7c1;
 Tue, 14 Sep 2021 14:32:59 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-YOGgfKp5OkGebasYqmF7yw-1; Tue, 14 Sep 2021 16:32:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3774.eurprd04.prod.outlook.com (2603:10a6:803:1f::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep
 2021 14:32:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 14:32:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0108.eurprd03.prod.outlook.com (2603:10a6:208:69::49) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Tue, 14 Sep 2021 14:32:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f409a10-b09c-447b-84dd-8110f40ca7c1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631629978;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KCHr+iu/Fh/OEFPHBp/vLAtGuSF7lejGesqvViIC/XM=;
	b=Q6P/zlCaWSnacPoUvQM0UnInmbgocoV4uQhQHYX+dZainu4z6DYxhToxpm1teteM2GrhME
	cKD0JeRkEN4I7k9e64pVqw1mLayhsVeupwhH+ku0phoE0c+BOEUq1hMgfaSHd27G6APCXp
	b/BUCkQbrVjkKNt5grup1k3yWEsX/HY=
X-MC-Unique: YOGgfKp5OkGebasYqmF7yw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IksFiJu2gI1huiGiGxJi3JHP/lBgOU2HTS6OoVY2lDs875Beib6WvGpNR0l0OKoJt4aw1Cr6QRN6RW7BtezY8AuRx52RzujZL4XUhVL19caiyglPPmcB+KdTfYwLVhGmEJDH+Ubenf0ABYNZrJmueUmIQDvNxRQCkKmfujdage8aOK5KpEWrLHN78xg1SqLGu9lWZH/u6DOk5wCarKj+A9NQCdNPK6WvQpmuR/HOzamMAG7vbgDG1/kvTGoH+2uVHO6sGtNB+GuXycT/6CCXm3n+GO/BV4Ogx0ZKN/trpPf1JEZHzeTfVHNBJJuZvAeWz3mY4YWkvMxarV0d+robHw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KCHr+iu/Fh/OEFPHBp/vLAtGuSF7lejGesqvViIC/XM=;
 b=ZJ3ALdgKUk/I8+gGQ5BqaNWLaoc5AGTOXsNaM4tET342LekYnJb1mSLKRTQj8dNnz4HhNBmoWIW7WK6gwFyNqlxFV6rD/Mevmpp1gVhbiL0cCZ77wLJixSUxJBSO1wGgkC7bdHDJ2pKqzU5PaEUuXObGGK0nsF0s0vVDfoBVsmt3U4scviWsbZl77ulCBmkoPND7iHoJf3JJPxFGPE01Q6lZyMk6nQwZ26RHwp7bYZcCtrsTrp9M68MTFNuPWtT16F1nIi/lneKxCGA7fnsAiyCHfvyoKJjPokbrYMGpDD4tRmrEc977pijOaQ7XJ4cKJUbkGlNNqenZLrDNKgZ1uA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/p2m: fix xenmem_add_to_physmap_one double page
 removal
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210914133419.8407-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <96c7ffdd-57b4-15f6-7fb5-560c6ed0504e@suse.com>
Date: Tue, 14 Sep 2021 16:32:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210914133419.8407-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0108.eurprd03.prod.outlook.com
 (2603:10a6:208:69::49) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fe8e8932-b536-4b84-6053-08d9778c8a86
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3774:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3774CF0F9F01CA315730E8EDB3DA9@VI1PR0402MB3774.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	p/hOsC4FvBaK1btt1YKggbR3Pi5js/uhi2aSuRHOBpVykryni4SnbAMTFhcZs63RB8fVAw+L+cu51fvb7CqiMvqDkzWMRM77NCvLmiGHEzp5yc/3vLZMk1sdS8CvPb9STwHxoepAdDx2mygo4ZN2vXScWi1Tb55OKgdxoZrduLztuI8nhM7JbULm0ZVVklt9rPoFHaAyK9okBih+PLQuTMff3jfH4DpdoHWvTOS4BfCDGyoAc8X0ueTr5MTPLhEzwQKLCyZDMX80J9Em8d+lyCsMXPXGo4Rcm7Gv9BaX35A0eu82dQAOZd8W6JFAwOUCn4DCixnIFjoth+ZBPlVkf47WcH3sHvd8zHcK7GavIclUmU4IjsYHRbYKihpdSFWdFMLPxyC8SAC2CEIJqDIzURBApgsrlU1vg3cGI0Aqs3UDafcJbWiOD6eb30JKS8kpW3Vm3LGDZ6HB+YNZRQx5G1/r6I4eZ6e8VV8jiyjIsXH/QAoNBpSdX9A9QGGwXehnUP9d2Ami7T9kqOCJ+Qld5VLas6fAR9EIGuChb8yJjC0cF1ihwqsmEUoNvDQSjTlDq3XMGOG+biSwdjJ7/RH1Gu40aAWNLNVF3fyKSR5vvt6PI0SCCdaMouhKYTHF0g3IX3tP7I/elQbfryKQzdflV4LeiBplLOw0mhmcmbzHtOLtV2SZ5KeamZAUouBAiK0/JFcJ2vEzYEONevHvWym/aVd4rlensWTLHb8gXtHBC3M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(39860400002)(346002)(366004)(136003)(53546011)(5660300002)(186003)(66556008)(8676002)(66946007)(8936002)(478600001)(36756003)(31696002)(16576012)(316002)(26005)(54906003)(6486002)(86362001)(2906002)(4326008)(6916009)(83380400001)(31686004)(38100700002)(66476007)(2616005)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?d2FCVWlDTVpRUHhaZmMxQmRlVWhtWDl1MWJXZHAwclRIQUZpV0JhMC85VDlD?=
 =?utf-8?B?L2J2VTNWMXFQNnFJY1ZLbktmMm9rN2hMSDUxRUlQSk1obVRiSEpvTXM4NW5Z?=
 =?utf-8?B?SXNrdStGckxMbDdNVG5GUnk2M3gzOXVYNWh0dzkyZTFZcFpRWGF3eUdhNWxN?=
 =?utf-8?B?ZHpzRVVlOG9xUnB2WmtxSU1Md3pkb0NvQmwwVFYxVTFrRWM1UjNLeloybDhn?=
 =?utf-8?B?b29rbXFoWTVUcC95UUhHUk16Vjg5TTF6empvS3p1NDVCWW5pT3B1RG9IOTl3?=
 =?utf-8?B?SnVDclV5OWdHK3lONmpaVmdiSHUyN3hMQ3ZwRmdlZjFhNUZkMThoMWp0Q3Fy?=
 =?utf-8?B?ajlORlV5WnQ1YklKZjhmWDg2dlR1R0FKQXlLWjkwL0dOZXIvWXlJVU90TCtv?=
 =?utf-8?B?T2YxaVhIdGdmZjhZb2ozNVJoKytRWXVsNTBUQUJOU3VEbzM5a2l1WVJPM0cr?=
 =?utf-8?B?N3YwU1RFaktwMFpaRnBGa0lpanpGTmpOQjJOejkrREZCM0MxZ0JmS2tCMzdU?=
 =?utf-8?B?cG8zS1h5elI1dFhOMFRkYTFlcXlFZGJuWEVhdkdvNDExZzNrc3M1cXFWc1R1?=
 =?utf-8?B?N1hSTFR0M1lkUXdQd3FaUW95RUpZRWdiL001SmdOV2QxamhDaERRcG1XR3hG?=
 =?utf-8?B?WHdjNW5QbjIzNkxsUU9aUFlycUxxZkU4dTh4MkNxZzBOSXlESGhrcnBweTFZ?=
 =?utf-8?B?bHpmcXpyK3ByNHg4V2w5Q09veEZpS2RyaVRQVmo5R3lwc252M0huc05FdGNt?=
 =?utf-8?B?K3dxL2ZiUy9KdEEyT0pidUhPa0lWVE9LVTVSNWtoOGVxdXVZTFI2Nm9aNFJD?=
 =?utf-8?B?ZFhiNFVLU3A5d1E1SUl3OHhGaTQ2allLLzdLLzR5MFNlSXYxbU8vczA5RC9Y?=
 =?utf-8?B?LzZhRXd4anlObHFJQml0bEorUTErWlZhSnk3LzduSENHcUliL1dGQ2JiYmxN?=
 =?utf-8?B?Vk5PRzN6ZGZBYzlTTmV2b0RrQ1RvZzFSTnIvc3IrVC9FenhGZjdNaWxMWUUz?=
 =?utf-8?B?a1NTbU9uWTQ4Rk9RTWN2WU5IOVNzb0NRTks1Z2pKMVVFeUNuTitXaFErZllt?=
 =?utf-8?B?T3E5UDZJMTVQMmtoS1RialFNcHFUZENYNlUzOVNuaU5RaHNteXBRVjlaeUc2?=
 =?utf-8?B?V1dtM3lFRzNKSXpUbit6NnVqaHZwa093Q2ZiRFVWZDB5M0NHL1FzczJmZzZo?=
 =?utf-8?B?N1E0RGE4dVlYZytVVVJoeGJEcXpydlZsdldLVmp1citFWnFjT0ZuaXJ0MXRy?=
 =?utf-8?B?dlA0dnhzLzUvUlNFa0tKbEIvY0RiaWU4N3BaRVNzbTREMnVEOGdwV1VnWmow?=
 =?utf-8?B?clRwblIwcGRmRTlzdG8zTkg1b24vVk9aODdsKzJTZFBWeTN4Um1hVWZzcDlk?=
 =?utf-8?B?SVVRK3JleU9FWDVzK2NyM2JuakNONVVvN0YrZkVpMGhTVWp1TjJyRU1Gb09h?=
 =?utf-8?B?c2ZCTkxId3V0aTVqQlY4WFNwYjk0aUF6UFV3SW1aS3Z1NCtMTFNsVDdXRFh2?=
 =?utf-8?B?N2Z2cmhQV09BaSt5K3NtRnplS3p0Y3RYOEUwOGFSUm1NZm1WY2s5R3Q4U2FN?=
 =?utf-8?B?cXNCNnJGQno1Ry95VWpWYlBFMU81aUFMayswcVRTZVFpSzZvcCtZMmUrcVh6?=
 =?utf-8?B?OG5mRzhGTTIzTFk4dXgwbDVjRkJTa083UnR6cldIQjlJb1pYaWFzTXFISlg5?=
 =?utf-8?B?TUk0MHoyM1dBOWVnSFJRdG1jcEprQ1pxdUpuZmx4dnJiMVVwZ2ljaVVXMmVv?=
 =?utf-8?Q?dOOCWN5QvukzRF1fAg/3/mswSkFuhklcnRgpY5W?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fe8e8932-b536-4b84-6053-08d9778c8a86
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 14:32:55.3924
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gytMOlG1GMXym19vJ8O2ukFs5p88toxC4+q1PDR5mUDN21jGlHT4SKASBbrbIin/JC12jV+hDHf9Hve2RUovoA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3774

On 14.09.2021 15:34, Roger Pau Monne wrote:
> If the new gfn matches the previous one (ie: gfn == old_gpfn)

It took me a while to realize that you probably mean "gpfn" here.

> xenmem_add_to_physmap_one will issue a duplicated call to
> guest_physmap_remove_page with the same gfn, because the

Considering the local variable of this name, may I suggest to
upper-case GFN in this case?

> get_gpfn_from_mfn call has been moved by commit f8582da041 to be
> performed before the original page is removed. This leads to the
> second guest_physmap_remove_page failing, which was not the case
> before commit f8582da041.
> 
> Fix this by adding a check that prevents a second call to
> guest_physmap_remove_page if the previous one has already removed the
> backing page from that gfn.

Same here (if this remains; see below).

> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -2813,7 +2813,7 @@ int xenmem_add_to_physmap_one(
>      }
>  
>      /* Unmap from old location, if any. */
> -    if ( !rc && old_gpfn != INVALID_M2P_ENTRY )
> +    if ( !rc && old_gpfn != INVALID_M2P_ENTRY && !gfn_eq(_gfn(old_gpfn), gpfn) )
>          rc = guest_physmap_remove_page(d, _gfn(old_gpfn), mfn, PAGE_ORDER_4K);
>  
>      /* Map at new location. */
> 

It feels unbalanced to suppress the remove, but keep the add in
this case. Wouldn't we be better off skipping both, or short-
circuiting the effective no-op even earlier? I recall considering
to install a shortcut, but it didn't seem worth it. But now that
you've found actual breakage, perhaps this need reconsidering.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:34:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:34:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186820.335563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9Vj-0001ZW-RU; Tue, 14 Sep 2021 14:34:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186820.335563; Tue, 14 Sep 2021 14:34:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9Vj-0001ZP-OK; Tue, 14 Sep 2021 14:34:11 +0000
Received: by outflank-mailman (input) for mailman id 186820;
 Tue, 14 Sep 2021 14:34:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=puvF=OE=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mQ9Vi-0001ZJ-ID
 for xen-devel@lists.xen.org; Tue, 14 Sep 2021 14:34:10 +0000
Received: from mail-wr1-x42d.google.com (unknown [2a00:1450:4864:20::42d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 651f7dc0-b134-46d1-b641-8337acd90c9e;
 Tue, 14 Sep 2021 14:34:09 +0000 (UTC)
Received: by mail-wr1-x42d.google.com with SMTP id t8so15559955wrq.4
 for <xen-devel@lists.xen.org>; Tue, 14 Sep 2021 07:34:09 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id o10sm11893120wrc.16.2021.09.14.07.34.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 07:34:07 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 4C04D1FF96;
 Tue, 14 Sep 2021 15:34:06 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 651f7dc0-b134-46d1-b641-8337acd90c9e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=7N1WlynKaMd1gFi/xUXdVFiOHnGdvDIif7OrtCJw3zg=;
        b=QV2U3GUMEOTeYY1UtMJEW0TsLBNd4RMF9KqLCTqgC2r2CrHbjvrTa5MbBNF552jrIT
         CvPCeJbaC7sWRU66aNmU7DzQd7uIXij0M+MIqUq0MTLh47kDc6+rOdvGcMyHHUjKVzXa
         6i0nxGdF1Cp1qS8kTdqT9udhhE5+NoqqouTq0n8oYmGvqMfVzBA6bVF+nrYVQ0rtBG3y
         hlgxqF+03SxxLDAm0esQM6Ang3ecPR6RKGvQc1NlziSRwidTQ2n0lbhwmD3aEbM6f66F
         GR31klbS8YFH8inRbtYnTnYSZPGkyXz+4X1br2fm9w53SIInGORXPq14MRBYkU7XLUHd
         YFTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=7N1WlynKaMd1gFi/xUXdVFiOHnGdvDIif7OrtCJw3zg=;
        b=eV5acPjBHbnCA/t+8swbu+2j3YdyaJxtAr3vKHrddm/azOErzgHKF3fQR70yVZdD/B
         DvkOSoik4++Vg/Tb3hdIVehrzhJtlZhh4BQhVLPFvMpw2NCMTBOLS3d3n9x6Gt2DMRA5
         BsWVUG/gFTgUYpHroRFm7aK6bcJgegYEA0hH2599hlcGKMWIEmp6eM1VDrkNFsbhH2BY
         K3ffPrLe+ckxGffNINOsOiN2ov2aetpxvUHU9y6514v5bBXbVZug10QJZdPM2dhUsHog
         BL8ezUAtbdsG2DjZSNzufSIbRhUVMtj7BvuvYIYmW68VUEta5d7eRmWB4wGT+qP27TZm
         5Icg==
X-Gm-Message-State: AOAM53181O1x+4+aLAvPDhwuUAnljkJZwHivqqbEyjIoHftrtNx1kJvS
	fhcRGJGWiGldVUQKznYkV7G3kA==
X-Google-Smtp-Source: ABdhPJyHckofqWY2VKh07GWmlJ0xv6a0kbRJroIzDTqKuO5X5O/0cTMWoOwx0Ri+ylwwUXwuDsuoNg==
X-Received: by 2002:adf:b789:: with SMTP id s9mr19237422wre.315.1631630048147;
        Tue, 14 Sep 2021 07:34:08 -0700 (PDT)
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
 <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Stefano Stabellini <stefano.stabellini@xilinx.com>
Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>, Stefan Hajnoczi
 <stefanha@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
 virtio-dev@lists.oasis-open.org, Arnd Bergmann <arnd.bergmann@linaro.org>,
 Viresh Kumar <viresh.kumar@linaro.org>, Jan Kiszka
 <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>,
 pratikp@quicinc.com, Srivatsa Vaddagiri <vatsa@codeaurora.org>,
 Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier
 <mathieu.poirier@linaro.org>, Wei.Chen@arm.com, olekstysh@gmail.com,
 Oleksandr_Tyshchenko@epam.com, Bertrand.Marquis@arm.com,
 Artem_Mygaiev@epam.com, julien@xen.org, jgross@suse.com, paul@xen.org,
 xen-devel@lists.xen.org, Elena Afanasova <eafanasova@gmail.com>
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Date: Tue, 14 Sep 2021 15:25:22 +0100
In-reply-to: <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
Message-ID: <878rzzusyp.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Stefano Stabellini <stefano.stabellini@xilinx.com> writes:

> On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
>> > the second is how many context switches are involved in a transaction.
>> > Of course with all things there is a trade off. Things involving the
>> > very tightest latency would probably opt for a bare metal backend which
>> > I think would imply hypervisor knowledge in the backend binary.
>>=20
>> In configuration phase of virtio device, the latency won't be a big matt=
er.
>> In device operations (i.e. read/write to block devices), if we can
>> resolve 'mmap' issue, as Oleksandr is proposing right now, the only issu=
e is
>> how efficiently we can deliver notification to the opposite side. Right?
>> And this is a very common problem whatever approach we would take.
>>=20
>> Anyhow, if we do care the latency in my approach, most of virtio-proxy-
>> related code can be re-implemented just as a stub (or shim?) library
>> since the protocols are defined as RPCs.
>> In this case, however, we would lose the benefit of providing "single bi=
nary"
>> BE.
>> (I know this is is an arguable requirement, though.)
>
> In my experience, latency, performance, and security are far more
> important than providing a single binary.
>
> In my opinion, we should optimize for the best performance and security,
> then be practical on the topic of hypervisor agnosticism. For instance,
> a shared source with a small hypervisor-specific component, with one
> implementation of the small component for each hypervisor, would provide
> a good enough hypervisor abstraction. It is good to be hypervisor
> agnostic, but I wouldn't go extra lengths to have a single binary.

I agree it shouldn't be a primary goal although a single binary working
with helpers to bridge the gap would make a cool demo. The real aim of
agnosticism is avoid having multiple implementations of the backend
itself for no other reason than a change in hypervisor.

> I cannot picture a case where a BE binary needs to be moved between
> different hypervisors and a recompilation is impossible (BE, not FE).
> Instead, I can definitely imagine detailed requirements on IRQ latency
> having to be lower than 10us or bandwidth higher than 500 MB/sec.
>
> Instead of virtio-proxy, my suggestion is to work together on a common
> project and common source with others interested in the same problem.
>
> I would pick something like kvmtool as a basis. It doesn't have to be
> kvmtools, and kvmtools specifically is GPL-licensed, which is
> unfortunate because it would help if the license was BSD-style for ease
> of integration with Zephyr and other RTOSes.

This does imply making some choices, especially the implementation
language. However I feel that C is really the lowest common denominator
here and I get the sense that people would rather avoid it if they could
given the potential security implications of a bug prone back end. This
is what is prompting interest in Rust.

> As long as the project is open to working together on multiple
> hypervisors and deployment models then it is fine. For instance, the
> shared source could be based on OpenAMP kvmtool [1] (the original
> kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
> kvmtool was created to add support for hypervisor-less virtio but they
> are very open to hypervisors too. It could be a good place to add a Xen
> implementation, a KVM fatqueue implementation, a Jailhouse
> implementation, etc. -- work together toward the common goal of a single
> BE source (not binary) supporting multiple different deployment models.
>
>
> [1] https://github.com/OpenAMP/kvmtool


--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:42:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:42:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186831.335573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9d8-0003EI-Nb; Tue, 14 Sep 2021 14:41:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186831.335573; Tue, 14 Sep 2021 14:41:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9d8-0003EB-KU; Tue, 14 Sep 2021 14:41:50 +0000
Received: by outflank-mailman (input) for mailman id 186831;
 Tue, 14 Sep 2021 14:41:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2lzK=OE=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQ9d7-0003E5-J5
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 14:41:49 +0000
Received: from mail-pg1-x531.google.com (unknown [2607:f8b0:4864:20::531])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fe03bd0f-9439-4bae-ab97-c32904d42fbb;
 Tue, 14 Sep 2021 14:41:48 +0000 (UTC)
Received: by mail-pg1-x531.google.com with SMTP id n18so12903714pgm.12
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 07:41:48 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 w5sm10372503pfu.160.2021.09.14.07.41.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 Sep 2021 07:41:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe03bd0f-9439-4bae-ab97-c32904d42fbb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=0tFcS0bgrni7T/UyH5tgM3kB2fvxQVZyvDvr4x9Nn0E=;
        b=UcCCLIQhvZq8Ncnr+BAJoJFu9Ixa/D+XUjmhbU7HIvc+hRj87USaT6BE1yy+iBq9Dg
         QGiVMbGMV+1FXGsEkdF095w1rXTPmXSBy/GzNBUQxpd9dOzN44hTusQVw+uvgN3pUog6
         vEZMdKSx10jM/eMjPmLE/1HA0l7wWv8mmMdUGuhqsKvWo5d9FFKvEl6dUqe0Cmi7wFvc
         IqUUL7XX9h+4FMxeygEw4BAnE8Jjme2tf0Xn3pAOyegWOofflTw3yvlM5BEDZhc2M9rn
         c1s9RDJ0MDor9d8Ut53EaH5xQoqBopBFHh0ewA7REHTL2u6atXEGxwqc8AvAaeBreZaC
         OxUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=0tFcS0bgrni7T/UyH5tgM3kB2fvxQVZyvDvr4x9Nn0E=;
        b=wM6aL/HKGjipaeiKXnuCRdHNCIG+lf1VacjycBrKNTH6c2B5stQZXhwWcBZM92EOzl
         2OwqBqr1mn8z6Cjpmjyg/CnCEABP0YQnH3fQ1Ajaq7pVcYTnixOJc8GhZ7J4RWFiL3L+
         C/evOOXnLbYJSP5j0WERwYDaypRf8DGwVxATnshrJkng1Sw71fZoj+s6lTXHwbnU9q7O
         OM63kDnZwS2yuAzDfEjAZdy5gHbPNrNXgDtpG95/aHhPVrcmJzctX39JlYnTtW6vmhn7
         PL8uPYx1jgl9nhgEHIrSAUyZoXf++tsB7JQ9I8B0DdMErBgTyvauu4hCnD27M4P353eu
         ff3w==
X-Gm-Message-State: AOAM530S8C5MkKf/D0alXIGU4gSAC7dwykRN1qYq8E3gkzSTvKuy3BHC
	OYZ0Dg0QugwJErKg72QIxnU=
X-Google-Smtp-Source: ABdhPJylfNPyIQEAHW7Jn9/cFbcvLCTICR8qzXoHOEsrI/lcIc/0eSlT+n5cpDvBA5GlFRMw2tbP5w==
X-Received: by 2002:a05:6a00:2405:b0:3e1:9f65:9703 with SMTP id z5-20020a056a00240500b003e19f659703mr5132461pfh.6.1631630507544;
        Tue, 14 Sep 2021 07:41:47 -0700 (PDT)
Subject: Re: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support
To: Michael Kelley <mikelley@microsoft.com>, Christoph Hellwig <hch@lst.de>
Cc: KY Srinivasan <kys@microsoft.com>, Haiyang Zhang
 <haiyangz@microsoft.com>, Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
 "will@kernel.org" <will@kernel.org>, "tglx@linutronix.de"
 <tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>,
 "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
 "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>, "pgonda@google.com"
 <pgonda@google.com>, "martin.b.radev@gmail.com" <martin.b.radev@gmail.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>, "hannes@cmpxchg.org"
 <hannes@cmpxchg.org>, "aneesh.kumar@linux.ibm.com"
 <aneesh.kumar@linux.ibm.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "linux-arm-kernel@lists.infradead.org"
 <linux-arm-kernel@lists.infradead.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "rientjes@google.com" <rientjes@google.com>,
 "ardb@kernel.org" <ardb@kernel.org>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210827172114.414281-1-ltykernel@gmail.com>
 <20210830120036.GA22005@lst.de>
 <MWHPR21MB15933503E7C324167CB4132CD7CC9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <20210902075939.GB14986@lst.de>
 <MWHPR21MB1593060DCFD854FDA14604D3D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <9b2f6372-3173-cc5c-81d2-365c8f09ef6f@gmail.com>
Date: Tue, 14 Sep 2021 22:41:32 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB1593060DCFD854FDA14604D3D7CE9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi Michael and Christoph:
       I just sent out V5 patchset. I use alloc_pages() to allocate rx/tx
ring buffer in Isolation VM and use vmap() to map rx/tx buffer first
because the vmbus_establish_gpadl() still needs to va of low end memory
to initialize gpadl buffer. After calling vmbus_establish_gpadl(), the
va returned by vmap will be unmapped to release virtual address space 
which will not be used in the following code and then map these pages in 
the extra address space above shared_gpa_boundary via vmap_pfn(). Please
have a look.

https://lkml.org/lkml/2021/9/14/672

Thanks.

On 9/2/2021 11:57 PM, Michael Kelley wrote:
> From: Christoph Hellwig <hch@lst.de> Sent: Thursday, September 2, 2021 1:00 AM
>>
>> On Tue, Aug 31, 2021 at 05:16:19PM +0000, Michael Kelley wrote:
>>> As a quick overview, I think there are four places where the
>>> shared_gpa_boundary must be applied to adjust the guest physical
>>> address that is used.  Each requires mapping a corresponding
>>> virtual address range.  Here are the four places:
>>>
>>> 1)  The so-called "monitor pages" that are a core communication
>>> mechanism between the guest and Hyper-V.  These are two single
>>> pages, and the mapping is handled by calling memremap() for
>>> each of the two pages.  See Patch 7 of Tianyu's series.
>>
>> Ah, interesting.
>>
>>> 3)  The network driver send and receive buffers.  vmap_phys_range()
>>> should work here.
>>
>> Actually it won't.  The problem with these buffers is that they are
>> physically non-contiguous allocations.
> 
> Indeed you are right.  These buffers are allocated with vzalloc().
> 
>> We really have two sensible options:
>>
>>   1) use vmap_pfn as in the current series.  But in that case I think
>>      we should get rid of the other mapping created by vmalloc.  I
>>      though a bit about finding a way to apply the offset in vmalloc
>>      itself, but I think it would be too invasive to the normal fast
>>      path.  So the other sub-option would be to allocate the pages
>>      manually (maybe even using high order allocations to reduce TLB
>>      pressure) and then remap them
> 
> What's the benefit of getting rid of the other mapping created by
> vmalloc if it isn't referenced?  Just page table space?  The default sizes
> are a 16 Meg receive buffer and a 1 Meg send buffer for each VMbus
> channel used by netvsc, and usually the max number of channels
> is 8.  So there's 128 Meg of virtual space to be saved on the receive
> buffers,  which could be worth it.
> 
> Allocating the pages manually is also an option, but we have to
> be careful about high order allocations.  While typically these buffers
> are allocated during system boot, these synthetic NICs can be hot
> added and removed while the VM is running.   The channel count
> can also be changed while the VM is running.  So multiple 16 Meg
> receive buffer allocations may need to be done after the system has
> been running a long time.
> 
>>   2) do away with the contiguous kernel mapping entirely.  This means
>>      the simple memcpy calls become loops over kmap_local_pfn.  As
>>      I just found out for the send side that would be pretty easy,
>>      but the receive side would be more work.  We'd also need to check
>>      the performance implications.
> 
> Doing away with the contiguous kernel mapping entirely seems like
> it would result in fairly messy code to access the buffer.  What's the
> benefit of doing away with the mapping?  I'm not an expert on the
> netvsc driver, but decoding the incoming packets is already fraught
> with complexities because of the nature of the protocol with Hyper-V.
> The contiguous kernel mapping at least keeps the basics sane.
> 
>>
>>> 4) The swiotlb memory used for bounce buffers.  vmap_phys_range()
>>> should work here as well.
>>
>> Or memremap if it works for 1.
>>
>>> Case #2 above does unusual mapping.  The ring buffer consists of a ring
>>> buffer header page, followed by one or more pages that are the actual
>>> ring buffer.  The pages making up the actual ring buffer are mapped
>>> twice in succession.  For example, if the ring buffer has 4 pages
>>> (one header page and three ring buffer pages), the contiguous
>>> virtual mapping must cover these seven pages:  0, 1, 2, 3, 1, 2, 3.
>>> The duplicate contiguous mapping allows the code that is reading
>>> or writing the actual ring buffer to not be concerned about wrap-around
>>> because writing off the end of the ring buffer is automatically
>>> wrapped-around by the mapping.  The amount of data read or
>>> written in one batch never exceeds the size of the ring buffer, and
>>> after a batch is read or written, the read or write indices are adjusted
>>> to put them back into the range of the first mapping of the actual
>>> ring buffer pages.  So there's method to the madness, and the
>>> technique works pretty well.  But this kind of mapping is not
>>> amenable to using vmap_phys_range().
>>
>> Hmm.  Can you point me to where this is mapped?  Especially for the
>> classic non-isolated case where no vmap/vmalloc mapping is involved
>> at all?
> 
> The existing code is in hv_ringbuffer_init() in drivers/hv/ring_buffer.c.
> The code hasn't changed in a while, so any recent upstream code tree
> is valid to look at.  The memory pages are typically allocated
> in vmbus_alloc_ring() in drivers/hv/channel.c.
> 
> Michael
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 14:49:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 14:49:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186841.335585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9kY-00046W-Gm; Tue, 14 Sep 2021 14:49:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186841.335585; Tue, 14 Sep 2021 14:49:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQ9kY-00046P-Dk; Tue, 14 Sep 2021 14:49:30 +0000
Received: by outflank-mailman (input) for mailman id 186841;
 Tue, 14 Sep 2021 14:49:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=puvF=OE=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mQ9kW-00046J-P7
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 14:49:28 +0000
Received: from mail-wm1-x32b.google.com (unknown [2a00:1450:4864:20::32b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 10ec073a-bd73-47ab-a3f6-6f5dc3a0a8a8;
 Tue, 14 Sep 2021 14:49:27 +0000 (UTC)
Received: by mail-wm1-x32b.google.com with SMTP id g128so5070506wma.5
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 07:49:27 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id j21sm1424161wmj.40.2021.09.14.07.49.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 14 Sep 2021 07:49:25 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id A1A001FF96;
 Tue, 14 Sep 2021 15:49:24 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10ec073a-bd73-47ab-a3f6-6f5dc3a0a8a8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=VsSaQm/zbSQNMCoZhBkWVXe+j2FkR4F2DupdJvdq+Ak=;
        b=Do4fypaueveDIn2KEf3d+A6bgMGNQaekYViEjI1dvQGqZEPvprL+44xWwK28pqvFZp
         X8dnggO/KCM3P0Hw58FOJmiSLbrwfjsYF7XGllx5SfPgxJKv3IYy6KdsFz50ctDjbdV/
         cSsO1OHxLYaV+pWd0Ap9GeUGDeARdauqGssczxk2aOqUiU52SDhZxtiJnogGMdZUA0wq
         PEL/nl22pIXqsEKHrk0r6U+rYrzj05fMC3vcWzent4NWOC0YE6PDlQc+5V8zJ3qJbQ0A
         m1aYAAQ/EWs6Tq1fPK6sWFFbrXj6s2dhsfNPLJ2pooqEqGXCfTBEKkmU/hgeCe1LtvIm
         eW6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=VsSaQm/zbSQNMCoZhBkWVXe+j2FkR4F2DupdJvdq+Ak=;
        b=Q4z89XCvTMKCYk0zl2Oi6OQjZkyOLLAPFTxk6PvlSd4tLLbQFgIR42W/510rxf53o2
         Vy2XB9ilONqkpwd29cinFtlGcjxsO954byxlVJo6YgqVgRLr0S8Q4ElOFXF0gcgi7Uyd
         Kh0M5tjgpvf8QCAf8kKAMNqnqw+JrzAXyu4d3HVo4J7+raMlgNuN8mPFOhf5uPVbMK8x
         5Ka6hh2RX4yj6xFKR6lOb+a/LgGD9Tb5phj6cLRR+6ja4XRVS/8D+NCQXrWG5jB+Kr28
         fWsQfCPI2Gp2BdTuoIDCCCiw+ltLZVsg306eOBXCOZnGeEvoxDH2V2ZCYmT75i8BEjsU
         CM4Q==
X-Gm-Message-State: AOAM5311saXCCB3J6EioOh2pGvOyJePmbRHUVqchuxXDqlay815igKYb
	EPDGlACLqKn3veIsMz2IWmnQwg==
X-Google-Smtp-Source: ABdhPJy9XlURfdc6gZhXaT9U9m1ZEmaW8K2p/LoflX2P8gyg4HKxT764dGNgv/2/O3kTtMhPD3VKcQ==
X-Received: by 2002:a1c:1d42:: with SMTP id d63mr2712677wmd.55.1631630966835;
        Tue, 14 Sep 2021 07:49:26 -0700 (PDT)
References: <87lf40vay1.fsf@linaro.org>
 <abfa4f44-8c56-af3f-485e-41b58e790d92@citrix.com>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>, Viresh Kumar
 <viresh.kumar@linaro.org>, Wei Liu <wl@xen.org>,
 "rust-vmm@lists.opendev.org" <rust-vmm@lists.opendev.org>,
 xen-devel@lists.xenproject.org, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
 <olekstysh@gmail.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>,
 Doug Goldstein <cardoe@cardoe.com>, Demi Marie Obenour
 <demi@invisiblethingslab.com>, Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
Date: Tue, 14 Sep 2021 15:44:01 +0100
In-reply-to: <abfa4f44-8c56-af3f-485e-41b58e790d92@citrix.com>
Message-ID: <874kanus97.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Andrew Cooper <andrew.cooper3@citrix.com> writes:

> On 13/09/2021 13:44, Alex Benn=C3=A9e wrote:
>> Hi,
>>
>> As we consider the next cycle for Project Stratos I would like to make
>> some more progress on hypervisor agnosticism for our virtio backends.
>> While we have implemented a number of virtio vhost-user backends using C
>> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
>> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
>> backends does it make sense to do some enabling work in rust-vmm to
>> support Xen?
>>
>> There are two chunks of work I can think of:
>>
>>   1. Enough of libxl/hypervisor interface to implement an IOREQ end poin=
t.
>
> No libxl here at all.
>
> As of Xen 4.15, there are enough stable interfaces to implement simple
> IOERQ servers.
>
> https://github.com/xapi-project/varstored/commit/fde707c59f7a189e1d4e97c1=
a4ee1a2d0c378ad1
> was the commit where I removed the final unstable interface from
> varstored (terrible name) which is a dom0 backend for UEFI secure
> variable handling.=C2=A0 As such, it also serves as a (not totally simple)
> reference of an IOERQ server.
>
>
> There are a few bits and pieces of rust going on within Xen, and a whole
> load of plans.=C2=A0 Also, there is a lot of interest from other downstre=
ams
> in being able to write Rust backends.
>
> We've got a placeholder xen and xen-sys crates, and placeholder work for
> supporting cross-compile as x86 PV and PVH stubdomains.

Are these in the rust-vmm project is elsewhere?

> The want to have a simple IOREQ server compiled either as a dom0
> backend, or as a PV or PVH stubdomains influences some of the design
> decisions early on, but they're all no-brainers for the longevity of the
> work.

Just to clarify nomenclature is a PVH stubdomain what I'm referring to
as a bare metal backend, i.e: a unikernel or RTOS image that implements
the backend without having to transition between some sort of userspace
and it's supporting kernel?=20

> I started work on trying to reimplement varstored entirely in Rust as a
> hackathon project, although ran out of time trying to make hypercall
> buffers work (there is a bug with Box and non-global allocators causing
> rustc to hit an assert().=C2=A0 In the short term, we'll have to implement
> hypercall buffers in a slightly more irritating way).
>
> Furthermore, stick to the stable hypercalls only.=C2=A0 Xen's C libraries=
 are
> disaster for cross-version compatibility, and you absolutely do not want
> to recompile your rust program just to run it against a different
> version of the hypervisor.=C2=A0 The plan is to start with simple IOREQ
> servers, which are on fully stable interfaces, then stabilise further
> hypercalls as necessary to expand functionality.

Are the hypercalls mediated by a kernel layer or are you making direct
HVC calls (on ARM) to the hypervisor from userspace?

Where would I look in the Xen code to find the hypercalls that are
considered stable and won't change between versions?

> It's high time the Xen Rust working group (which has been talked about
> for several years now) actually forms...

Indeed part of the purpose of this email was to smoke out those who are
interested in the intersection of Xen, Rust and VirtIO ;-)

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:10:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:10:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186855.335595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQA56-0007T6-9F; Tue, 14 Sep 2021 15:10:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186855.335595; Tue, 14 Sep 2021 15:10:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQA56-0007Sz-5M; Tue, 14 Sep 2021 15:10:44 +0000
Received: by outflank-mailman (input) for mailman id 186855;
 Tue, 14 Sep 2021 15:10:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dJAl=OE=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mQA54-0007St-1W
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:10:42 +0000
Received: from mail-lj1-x22f.google.com (unknown [2a00:1450:4864:20::22f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 19907586-bc48-436a-a132-5724f3c74417;
 Tue, 14 Sep 2021 15:10:40 +0000 (UTC)
Received: by mail-lj1-x22f.google.com with SMTP id f2so24538957ljn.1
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 08:10:40 -0700 (PDT)
Received: from localhost ([178.151.124.169])
 by smtp.gmail.com with ESMTPSA id z13sm1380486ljo.37.2021.09.14.08.10.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 14 Sep 2021 08:10:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19907586-bc48-436a-a132-5724f3c74417
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=+K8MjtcnJXfpNCRFyMCdqNnPKrcaeccqs3rzyDDn0vU=;
        b=IaMh1odM3L3VhmQ1m9aeZoRN0SQqI1qWvuDoZWK+kGl0yYbfmDEi7D0j2oLe14UX2F
         rpjUo3lsxP5Ys1zoUPhUxacNkXBGHhKp/VWLcn3VaR1IZbD/wkmOEHytFa0NiO1XV0aE
         O1GlE5Y2v46M+6SUc2NpFvRok3ihp3e/rfYBP921FLdN2x2npuwD6bPz9uzlrrgVeYu9
         m+iuYgMs/d2E2Nt8Vhc0NLSoGg6tG917T7wzGKqoVsqJs2NYLv7TzW6UM3OszHB4dLXY
         pfJbHa0qySjo55aBIDzrsH27D5ycEHqJnZOlX1euImWJ1nEKFy+itmgG9ixUnlRxJzzs
         +oDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=+K8MjtcnJXfpNCRFyMCdqNnPKrcaeccqs3rzyDDn0vU=;
        b=IF8tHeJbOlqKlOBfh8QWJ+8XagKx41ctcYrwoW/V3PwBXhsXqqBnVnBKEXiTrQH8s1
         Sh6jvUoSD8x+Mpuqs5HbwQzXZLFcI5k2FXiokUZ84Mk3q5ptEvS8DM7zDEn0lRSaQDB2
         vV8CpPbS6cS0O+TSQyn6Yq7WPg9ir4vN4Fo3V/CbheSHBkT1iHqbUJn+xzwDCwNYb8VT
         bBB9zfeWLkFt0l8eOvS95Q5ndzggOMEMdQc1F2P0oo7TJ6wid/vziqLayNimYWByOaSI
         ZpvR+uYUuSPI5P28y6ISYKlNYUw7r4i8DRjGapRt9JJBAoJgdfj4N0pN4bZCyY2OPnIi
         ekSg==
X-Gm-Message-State: AOAM531pdKCAUnIOJcU7o60G32urXtcLLb2jIrCssNxJ5s1Y/nrmj3CU
	nc+SAn7vRJusRf0nKlaMUIwWSJEBRjTLcw==
X-Google-Smtp-Source: ABdhPJzf6vLVIOTdmCQu5RbgpN6oC/S/5GVuhLuSFjpLP9tVLZKCDmzH0tCMHS1catjmI7HUw5NlUA==
X-Received: by 2002:a05:651c:1410:: with SMTP id u16mr15749408lje.435.1631632239102;
        Tue, 14 Sep 2021 08:10:39 -0700 (PDT)
From: Roman Skakun <rm.skakun@gmail.com>
X-Google-Original-From: Roman Skakun <Roman_Skakun@epam.com>
To: xen-devel@lists.xenproject.org,
	linux-mips@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	iommu@lists.linux-foundation.org
Cc: Roman Skakun <rm.skakun@gmail.com>,
	Andrii Anisov <andrii_anisov@epam.com>,
	Roman Skakun <roman_skakun@epam.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christoph Hellwig <hch@lst.de>,
	Marek Szyprowski <m.szyprowski@samsung.com>,
	Robin Murphy <robin.murphy@arm.com>,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	"Maciej W. Rozycki" <macro@orcam.me.uk>,
	Muchun Song <songmuchun@bytedance.com>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	Mike Rapoport <rppt@kernel.org>,
	Will Deacon <will@kernel.org>
Subject: [PATCH] swiotlb: set IO TLB segment size via cmdline
Date: Tue, 14 Sep 2021 18:10:11 +0300
Message-Id: <20210914151016.3174924-1-Roman_Skakun@epam.com>
X-Mailer: git-send-email 2.27.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Roman Skakun <roman_skakun@epam.com>

It is possible when default IO TLB size is not
enough to fit a long buffers as described here [1].

This patch makes a way to set this parameter
using cmdline instead of recompiling a kernel.

[1] https://www.xilinx.com/support/answers/72694.html

Signed-off-by: Roman Skakun <roman_skakun@epam.com>
---
 .../admin-guide/kernel-parameters.txt         |  5 +-
 arch/mips/cavium-octeon/dma-octeon.c          |  2 +-
 arch/powerpc/platforms/pseries/svm.c          |  2 +-
 drivers/xen/swiotlb-xen.c                     |  7 +--
 include/linux/swiotlb.h                       |  1 +
 kernel/dma/swiotlb.c                          | 51 ++++++++++++++-----
 6 files changed, 48 insertions(+), 20 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 91ba391f9b32..f842a523a485 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5558,8 +5558,9 @@
 			it if 0 is given (See Documentation/admin-guide/cgroup-v1/memory.rst)
 
 	swiotlb=	[ARM,IA-64,PPC,MIPS,X86]
-			Format: { <int> | force | noforce }
-			<int> -- Number of I/O TLB slabs
+			Format: { <slabs> [,<io_tlb_segment_size>] [,force | noforce]​ }
+			<slabs> -- Number of I/O TLB slabs
+			<io_tlb_segment_size> -- Max IO TLB segment size
 			force -- force using of bounce buffers even if they
 			         wouldn't be automatically used by the kernel
 			noforce -- Never use bounce buffers (for debugging)
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index df70308db0e6..446c73bc936e 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -237,7 +237,7 @@ void __init plat_swiotlb_setup(void)
 		swiotlbsize = 64 * (1<<20);
 #endif
 	swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
-	swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
+	swiotlb_nslabs = ALIGN(swiotlb_nslabs, swiotlb_io_seg_size());
 	swiotlbsize = swiotlb_nslabs << IO_TLB_SHIFT;
 
 	octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 87f001b4c4e4..2a1f09c722ac 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -47,7 +47,7 @@ void __init svm_swiotlb_init(void)
 	unsigned long bytes, io_tlb_nslabs;
 
 	io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
-	io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
+	io_tlb_nslabs = ALIGN(io_tlb_nslabs, swiotlb_io_seg_size());
 
 	bytes = io_tlb_nslabs << IO_TLB_SHIFT;
 
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 643fe440c46e..0fc9c6cb6815 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -110,12 +110,13 @@ static int xen_swiotlb_fixup(void *buf, unsigned long nslabs)
 	int dma_bits;
 	dma_addr_t dma_handle;
 	phys_addr_t p = virt_to_phys(buf);
+	unsigned long tlb_segment_size = swiotlb_io_seg_size();
 
-	dma_bits = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT) + PAGE_SHIFT;
+	dma_bits = get_order(tlb_segment_size << IO_TLB_SHIFT) + PAGE_SHIFT;
 
 	i = 0;
 	do {
-		int slabs = min(nslabs - i, (unsigned long)IO_TLB_SEGSIZE);
+		int slabs = min(nslabs - i, (unsigned long)tlb_segment_size);
 
 		do {
 			rc = xen_create_contiguous_region(
@@ -153,7 +154,7 @@ static const char *xen_swiotlb_error(enum xen_swiotlb_err err)
 	return "";
 }
 
-#define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, IO_TLB_SEGSIZE)
+#define DEFAULT_NSLABS		ALIGN(SZ_64M >> IO_TLB_SHIFT, swiotlb_io_seg_size())
 
 int __ref xen_swiotlb_init(void)
 {
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b0cb2a9973f4..35c3ffeda9fa 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -59,6 +59,7 @@ void swiotlb_sync_single_for_cpu(struct device *dev, phys_addr_t tlb_addr,
 		size_t size, enum dma_data_direction dir);
 dma_addr_t swiotlb_map(struct device *dev, phys_addr_t phys,
 		size_t size, enum dma_data_direction dir, unsigned long attrs);
+unsigned long swiotlb_io_seg_size(void);
 
 #ifdef CONFIG_SWIOTLB
 extern enum swiotlb_force swiotlb_force;
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 87c40517e822..6b505206fc13 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -72,6 +72,11 @@ enum swiotlb_force swiotlb_force;
 
 struct io_tlb_mem io_tlb_default_mem;
 
+/*
+ * Maximum IO TLB segment size.
+ */
+static unsigned long io_tlb_seg_size = IO_TLB_SEGSIZE;
+
 /*
  * Max segment that we can provide which (if pages are contingous) will
  * not be bounced (unless SWIOTLB_FORCE is set).
@@ -81,15 +86,30 @@ static unsigned int max_segment;
 static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
 
 static int __init
-setup_io_tlb_npages(char *str)
+setup_io_tlb_params(char *str)
 {
+	unsigned long tmp;
+
 	if (isdigit(*str)) {
-		/* avoid tail segment of size < IO_TLB_SEGSIZE */
-		default_nslabs =
-			ALIGN(simple_strtoul(str, &str, 0), IO_TLB_SEGSIZE);
+		default_nslabs = simple_strtoul(str, &str, 0);
 	}
 	if (*str == ',')
 		++str;
+
+	/* get max IO TLB segment size */
+	if (isdigit(*str)) {
+		tmp = simple_strtoul(str, &str, 0);
+		if (tmp)
+			io_tlb_seg_size = ALIGN(tmp, IO_TLB_SEGSIZE);
+	}
+	if (*str == ',')
+		++str;
+
+	/* update io_tlb_nslabs after applying a new segment size and
+	 * avoid tail segment of size < IO TLB segment size
+	 */
+	default_nslabs = ALIGN(default_nslabs, io_tlb_seg_size);
+
 	if (!strcmp(str, "force"))
 		swiotlb_force = SWIOTLB_FORCE;
 	else if (!strcmp(str, "noforce"))
@@ -97,7 +117,7 @@ setup_io_tlb_npages(char *str)
 
 	return 0;
 }
-early_param("swiotlb", setup_io_tlb_npages);
+early_param("swiotlb", setup_io_tlb_params);
 
 unsigned int swiotlb_max_segment(void)
 {
@@ -118,6 +138,11 @@ unsigned long swiotlb_size_or_default(void)
 	return default_nslabs << IO_TLB_SHIFT;
 }
 
+unsigned long swiotlb_io_seg_size(void)
+{
+	return io_tlb_seg_size;
+}
+
 void __init swiotlb_adjust_size(unsigned long size)
 {
 	/*
@@ -128,7 +153,7 @@ void __init swiotlb_adjust_size(unsigned long size)
 	if (default_nslabs != IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT)
 		return;
 	size = ALIGN(size, IO_TLB_SIZE);
-	default_nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+	default_nslabs = ALIGN(size >> IO_TLB_SHIFT, io_tlb_seg_size);
 	pr_info("SWIOTLB bounce buffer size adjusted to %luMB", size >> 20);
 }
 
@@ -147,7 +172,7 @@ void swiotlb_print_info(void)
 
 static inline unsigned long io_tlb_offset(unsigned long val)
 {
-	return val & (IO_TLB_SEGSIZE - 1);
+	return val & (io_tlb_seg_size - 1);
 }
 
 static inline unsigned long nr_slots(u64 val)
@@ -192,7 +217,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
 
 	spin_lock_init(&mem->lock);
 	for (i = 0; i < mem->nslabs; i++) {
-		mem->slots[i].list = IO_TLB_SEGSIZE - io_tlb_offset(i);
+		mem->slots[i].list = io_tlb_seg_size - io_tlb_offset(i);
 		mem->slots[i].orig_addr = INVALID_PHYS_ADDR;
 		mem->slots[i].alloc_size = 0;
 	}
@@ -261,7 +286,7 @@ int
 swiotlb_late_init_with_default_size(size_t default_size)
 {
 	unsigned long nslabs =
-		ALIGN(default_size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE);
+		ALIGN(default_size >> IO_TLB_SHIFT, io_tlb_seg_size);
 	unsigned long bytes;
 	unsigned char *vstart = NULL;
 	unsigned int order;
@@ -522,7 +547,7 @@ static int swiotlb_find_slots(struct device *dev, phys_addr_t orig_addr,
 			alloc_size - (offset + ((i - index) << IO_TLB_SHIFT));
 	}
 	for (i = index - 1;
-	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+	     io_tlb_offset(i) != io_tlb_seg_size - 1 &&
 	     mem->slots[i].list; i--)
 		mem->slots[i].list = ++count;
 
@@ -600,7 +625,7 @@ static void swiotlb_release_slots(struct device *dev, phys_addr_t tlb_addr)
 	 * with slots below and above the pool being returned.
 	 */
 	spin_lock_irqsave(&mem->lock, flags);
-	if (index + nslots < ALIGN(index + 1, IO_TLB_SEGSIZE))
+	if (index + nslots < ALIGN(index + 1, io_tlb_seg_size))
 		count = mem->slots[index + nslots].list;
 	else
 		count = 0;
@@ -620,7 +645,7 @@ static void swiotlb_release_slots(struct device *dev, phys_addr_t tlb_addr)
 	 * available (non zero)
 	 */
 	for (i = index - 1;
-	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 && mem->slots[i].list;
+	     io_tlb_offset(i) != io_tlb_seg_size - 1 && mem->slots[i].list;
 	     i--)
 		mem->slots[i].list = ++count;
 	mem->used -= nslots;
@@ -698,7 +723,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
 
 size_t swiotlb_max_mapping_size(struct device *dev)
 {
-	return ((size_t)IO_TLB_SIZE) * IO_TLB_SEGSIZE;
+	return ((size_t)IO_TLB_SIZE) * io_tlb_seg_size;
 }
 
 bool is_swiotlb_active(struct device *dev)
-- 
2.27.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:13:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:13:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186862.335606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQA8E-0008Av-Ra; Tue, 14 Sep 2021 15:13:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186862.335606; Tue, 14 Sep 2021 15:13:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQA8E-0008Ao-Oh; Tue, 14 Sep 2021 15:13:58 +0000
Received: by outflank-mailman (input) for mailman id 186862;
 Tue, 14 Sep 2021 15:13:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQA8E-0008Ai-Au
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:13:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 664d4aa8-82aa-4b2f-aa87-8e5c825fb0e1;
 Tue, 14 Sep 2021 15:13:56 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-gmrR_YGDP-my1J0cNaWr6g-1;
 Tue, 14 Sep 2021 17:13:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 14 Sep
 2021 15:13:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 15:13:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM9P192CA0024.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:21d::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 15:13:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 664d4aa8-82aa-4b2f-aa87-8e5c825fb0e1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631632436;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=eUEbQ511uBH4HywxbmIrZZ/yrHO5uYXLo2/mlkVIRZs=;
	b=hDmUP210wMIryaLe6Q3jwIxldjYeC2icPwsG3UwHGkYVZixWp69Ra9RsY5OgUCqw7n7bXQ
	VaKao+3s9ItnLnDyXSYeFIgLxxLwi6QbCx6lTHzTuxEhWdQJqiiZZz5N0gdwkPLAIuXUz0
	HqXJbVbXLlPS5NSaurr05BWtht/b0sI=
X-MC-Unique: gmrR_YGDP-my1J0cNaWr6g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SDk7/3jEJxafXnInRIatcNCb9ospsr2UZEnP4MNzQgTxNh0Zkn/nRJ8e2ATR2mE2pEs2xoxlw66pAX5aWsTvhN/5Q7bQ6YTfsb+iN9qWgzyoPkbcBnXrqJGxpSIoRoq7IaoH3FCu9CF9wEt0keXfhE6Lw9ehd2EbTz46H4zS3kvTNj+8AxpSYTKOnluAeTII4lTdTj6hRQpM4Hr6OyUrtTQOT0yP8sDh45u0+W1YZ2zY9eX6wAk753hnIbJqTp+BSgE3fdVmnHMTJFTMOPvqmqdDQoot5HCkGEZN6RHS8C1BeZToZuX8yiyNZKlrIhgBZbjXOIfa+6A+kvtGl1Axvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ECD752V6eG6nfPVInGP4QXCeIopzoJTg6NlXkgkzu0M=;
 b=EBgpvGVvMvkHssBLZxzlbx/uL7AW0kM+JS0pHZe+ZxihM6sSiQdIK0vNBR2MMLDRuJcas/insmT2jj2KdOjNkPPNLB0JjFRbAGroHg4VWoqD/fZ6P1kjxQ8jKP94WX4xRYbPivCUhDwF4PZeC5SALD+j1niJujZ3cqEHbUI/kKPAg+XXy3kfMdQTYl3SHj2L0E9I56Cn0GNv2H6Wx296CZRNaS1NQfxPHOgCYJlCX++mb+d4fGZXnnupJAaZkbAOjjic7hbnWrTWLU1W03Gc5mzfeUGNetLd6bDtUhi8UEWcIB9O0g292aLai0XIV6nW1myXLFwkKzHspv0bS/iXVw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
 <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
 <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
 <YUCYYmqQMR5NCTyz@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <13dd8509-330b-4571-086a-7a3ed9c22c82@suse.com>
Date: Tue, 14 Sep 2021 17:13:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUCYYmqQMR5NCTyz@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM9P192CA0024.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:20b:21d::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5adbe3f1-dc66-4920-9049-08d9779243cc
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23354197C41F9E71E890D952B3DA9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	niMXTS/KNXbkB+wjze2R1IVFVOHfeFeUehENw3LpWc34kQZ/uDwbBYDyylTZSHN6Af2CWhGO/m8VaFaHKfzA7OCWPDWqRXw6lHK4EO7/RFfS4m6Q6eNxvjMyZtsnV9QzRAy67M8EsnHX48sMnWS+7gEMbnp4nP58ytzNFdg0d7icj93eucSz40ch4dC/CRMMSVJuGdm8tLmrmB9H3I1lmlQciFBxONE/dJYQd97qUakX1RhJy7kyV5WXzyTPq8hcXc2XioKsg76DKqaY1SeIoJLqfQ81CMBtpiw76HUleOvzWb4Eww9mIL+IBepbm6wQgLmj3t7nB0YNFd9trTpeO16kS3woMCS21xvx90dkaEZLjzzUMziYlnERLwxX8MsyAh/1tOumTIqIrdPCYQo9QcjmyRCq66OkV1oGfWQ3qGInVFSJ/DDALo1ua1E4KDiI00Egw1wGEj2IL5U9EBk/Ub4UU4SE+48V2Dh21CZbiBt9774DIsnc3b5Evms8PpY+9bXMqFmOO/qSICF9/vXwmFzJOZnoZu1g0l08NlR95O4S3w6wOV5pLUtFFB4dFrllJG5HEYOS1wAuhmHKMrYozw18JVlm3hhRTgxBniCtc2wzi/iyPbGWJM50ALMoKV5Sv4ZU/rkZ4zot653pTir4PgvwGRUFKO3R6Z1nEg7Q3sARDvlPYLEsm8pATRJesAWKHWxgWZoUPrmkNVaSQaN8YnXNBwTfJdlXFO1eoFgcZYYFWRiBqqeK5bwtgS/H5r97sRg87q4UB9igWTGL0AATwObzep+6+7b0sXjGCe0yhVQUZXOPvZaXe9G4QC7qWb41vCUUKB/2/oQy9pgYJa4kfQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(136003)(366004)(39860400002)(346002)(31686004)(6916009)(26005)(31696002)(53546011)(38100700002)(6486002)(36756003)(45080400002)(186003)(8936002)(478600001)(2906002)(8676002)(4326008)(5660300002)(83380400001)(86362001)(66556008)(966005)(54906003)(2616005)(316002)(16576012)(66946007)(66476007)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?at9P6pCwgpCB5svNX/CO8os8thqC+oDosEeqZhCCWPGnq8Ianqz9YTumYWAz?=
 =?us-ascii?Q?pYfJFotsxwQVWlqDxvdA5qM9/g/xXjC2IJiFkAxGfOoaOglIVdDY66yxkCfP?=
 =?us-ascii?Q?iLKyAxPszcWk58xcg2s5o888Ot86GzyS13EHoaa5Fw86Vl9IydpOSa248ycE?=
 =?us-ascii?Q?0mF4EwCGNmHUEj4zo/5jD9ueHs+UTqmuEC0GZTfpoFKYIIkdmxLEHOt5Rchh?=
 =?us-ascii?Q?3QwgRdjHqUVamepONRBcOB8t/NokX5wgaTIm663jpfUETLAEaU+FCVKB8tUe?=
 =?us-ascii?Q?TIikDlPSFPtecMyApBPMWc1UNEnTQncixpXEcWuw9QoPtxUz7Cm158vQ9tZo?=
 =?us-ascii?Q?qu4WTnaAclCGL4XHb6Pt8yhype4ASP7xXsBWRVGsUrt+kAUkPGM6yChh4Mi5?=
 =?us-ascii?Q?gIvQG39g2f2g+HoLGzZY+49QQgN2FpIY0UKWXFihHoWLg61uVbhP/rRqppwV?=
 =?us-ascii?Q?dX4z7274+N9ER3v44BpTfIpWhsDR8n4ZKh3stNV3wgkKl3qaP3TQYlLCj48C?=
 =?us-ascii?Q?XPTiWO+fqR20RyHWao5/m+DB6l61FIKV9ixOqYIQ6QjAn03U6sE4itryNmLG?=
 =?us-ascii?Q?AJad+hYx6rs9+0icI/5NPGrg+bw1yCGYxtHE55PLUTB3joCwtM4LuZNJYrI8?=
 =?us-ascii?Q?TfEQO36EMMh/Bf4HSlVpVw7013jdjtKpTQxTcSQKulDFdG9oK85AHa5N19uZ?=
 =?us-ascii?Q?A6egGZPeHywWwsP6u9YbsnraixnKliksUxCliL1fRi03rqMwueY5pPC2Skxt?=
 =?us-ascii?Q?QGyjbzoelPs3AZbTQV20gslIMCJZ6Xpr9IMSbbMYJhOQc4H/FedgM1y6vRPR?=
 =?us-ascii?Q?WnSQ4UpfujbGA2Rj3hJ4bfpovykTSr+ifnPLod+WbygwXny3uMN/d9aqQk0V?=
 =?us-ascii?Q?DbWdLxcEHJzEUf/ZrpiMKRADHNNRBNMZ7aLE9WAFAQjY2tVQMpcDt8hmZbmz?=
 =?us-ascii?Q?PHrPklCbKyXXwqkJqw9koiJjTCg6ajLZ+eUY3M+qcZj/wae8X5/hnsb/Y8vG?=
 =?us-ascii?Q?S2bI0/19kswmUucSx60f7xdnSHasM0qs/CURuecuIzce43plvsgkvCS79hWk?=
 =?us-ascii?Q?gZocLPs9OCJMTCyr9oUnvN8SkU2Dd1W1gay3JXC0z514rKjAk/qOelaRAeMf?=
 =?us-ascii?Q?LCn2OIW9NnMhcHINTyqdlifvSrVFREkLFumbUqv4IEf7w8JD5IL+vOnyd7b9?=
 =?us-ascii?Q?h5T/oolHgA5Ypnz5K/fMmP/dP17VfLBEx/LX8cnwLrsb6Z8LR2N3Fr44opTN?=
 =?us-ascii?Q?cOf1ZBlgR0yfaWkYjif+PEsdE6BTJtKyLpGOnMsRXw3QEWY9iU9eQlT89909?=
 =?us-ascii?Q?Z994Ebuoov2NjYtiilYZq/MD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5adbe3f1-dc66-4920-9049-08d9779243cc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 15:13:53.7275
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fRgwcyn0rXPUAjv56cYjTyvN4jc/Whq6BcRTH2XT8UX/14PdqMPKPyvYJls7GRbkf/0lH+D0IWJtNkSE1hQCbA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 14.09.2021 14:41, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 14, 2021 at 01:58:29PM +0200, Jan Beulich wrote:
>> On 14.09.2021 13:15, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
>>>> On 14.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>>>> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
>>>>>> In order to try to debug hypervisor side breakage from XSA-378 I fou=
nd
>>>>>> myself urged to finally give PVH Dom0 a try. Sadly things didn't wor=
k
>>>>>> quite as expected. In the course of investigating these issues I act=
ually
>>>>>> spotted one piece of PV Dom0 breakage as well, a fix for which is al=
so
>>>>>> included here.
>>>>>>
>>>>>> There are two immediate remaining issues (also mentioned in affected
>>>>>> patches):
>>>>>>
>>>>>> 1) It is not clear to me how PCI device reporting is to work. PV Dom=
0
>>>>>>    reports devices as they're discovered, including ones the hypervi=
sor
>>>>>>    may not have been able to discover itself (ones on segments other
>>>>>>    than 0 or hotplugged ones). The respective hypercall, however, is
>>>>>>    inaccessible to PVH Dom0. Depending on the answer to this, either
>>>>>>    the hypervisor will need changing (to permit the call) or patch 2
>>>>>>    here will need further refinement.
>>>>>
>>>>> I would rather prefer if we could limit the hypercall usage to only
>>>>> report hotplugged segments to Xen. Then Xen would have to scan the
>>>>> segment when reported and add any devices found.
>>>>>
>>>>> Such hypercall must be used before dom0 tries to access any device, a=
s
>>>>> otherwise the BARs won't be mapped in the second stage translation an=
d
>>>>> the traps for the MCFG area won't be setup either.
>>>>
>>>> This might work if hotplugging would only ever be of segments, and not
>>>> of individual devices. Yet the latter is, I think, a common case (as
>>>> far as hotplugging itself is "common").
>>>
>>> Right, I agree to use hypercalls to report either hotplugged segments
>>> or devices. However I would like to avoid mandating usage of the
>>> hypercall for non-hotplug stuff, as then OSes not having hotplug
>>> support don't really need to care about making use of those
>>> hypercalls.
>>>
>>>> Also don't forget about SR-IOV VFs - they would typically not be there
>>>> when booting. They would materialize when the PF driver initializes
>>>> the device. This is, I think, something that can be dealt with by
>>>> intercepting writes to the SR-IOV capability.
>>>
>>> My plan was to indeed trap SR-IOV capability accesses, see:
>>>
>>> https://nam04.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flor=
e.kernel.org%2Fxen-devel%2F20180717094830.54806-1-roger.pau%40citrix.com%2F=
&amp;data=3D04%7C01%7Croger.pau%40citrix.com%7C35d2502d0128484e229e08d97777=
087f%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637672175399546062%7CUnkn=
own%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV=
CI6Mn0%3D%7C1000&amp;sdata=3DsSeE%2F4wEo5%2Fplkj2yH%2B1kpHi5c15lxJxeUxx6Cby=
r4s%3D&amp;reserved=3D0
>>>
>>> I just don't have time ATM to continue this work.
>>>
>>>> But I wonder whether
>>>> there might be other cases where devices become "visible" only while
>>>> the Dom0 kernel is already running.
>>>
>>> I would consider those kind of hotplug devices, and hence would
>>> require the use of the hypercall in order to notify Xen about them.
>>
>> So what does this mean for the one patch? Should drivers/xen/pci.c
>> then be built for PVH (and then have logic added to filter boot
>> time device discovery), or should I restrict this to be PV-only (and
>> PVH would get some completely different logic added later)?
>=20
> I think we can reuse the same hypercalls for PVH, and maybe the same
> code in Linux. For PVH we just need to be careful to make the
> hypercalls before attempting to access the BARs (or the PCI
> configuration space for the device) since there won't be any traps
> setup, and BARs won't be mapped on the p2m.
>=20
> It might be easier for Linux to just report every device it finds to
> Xen, like it's currently done for PV dom0, instead of filtering on
> whether the device has been hotplugged.

Okay. I'll leave the Linux patch as is then and instead make a Xen
patch to actually let through the necessary function(s) in
hvm_physdev_op().

>>>>>> 2) Dom0, unlike in the PV case, cannot access the screen (to use as =
a
>>>>>>    console) when in a non-default mode (i.e. not 80x25 text), as the
>>>>>>    necessary information (in particular about VESA-bases LFB modes) =
is
>>>>>>    not communicated. On the hypervisor side this looks like delibera=
te
>>>>>>    behavior, but it is unclear to me what the intentions were toward=
s
>>>>>>    an alternative model. (X may be able to access the screen dependi=
ng
>>>>>>    on whether it has a suitable driver besides the presently unusabl=
e
>>>>>>    /dev/fb<N> based one.)
>>>>>
>>>>> I had to admit most of my boxes are headless servers, albeit I have
>>>>> one NUC I can use to test gfx stuff, so I don't really use gfx output
>>>>> with Xen.
>>>>>
>>>>> As I understand such information is fetched from the BIOS and passed
>>>>> into Xen, which should then hand it over to the dom0 kernel?
>>>>
>>>> That's how PV Dom0 learns of the information, yes. See
>>>> fill_console_start_info(). (I'm in the process of eliminating the
>>>> need for some of the "fetch from BIOS" in Xen right now, but that's
>>>> not going to get us as far as being able to delete that code, no
>>>> matter how much in particular Andrew would like that to happen.)
>>>>
>>>>> I guess the only way for Linux dom0 kernel to fetch that information
>>>>> would be to emulate the BIOS or drop into realmode and issue the BIOS
>>>>> calls?
>>>>
>>>> Native Linux gets this information passed from the boot loader, I thin=
k
>>>> (except in the EFI case, as per below).
>>>>
>>>>> Is that an issue on UEFI also, or there dom0 can fetch the framebuffe=
r
>>>>> info using the PV EFI interface?
>>>>
>>>> There it's EFI boot services functions which can be invoked before
>>>> leaving boot services (in the native case). Aiui the PVH entry point
>>>> lives logically past any EFI boot services interaction, and hence
>>>> using them is not an option (if there was EFI firmware present in Dom0
>>>> in the first place, which I consider difficult all by itself - this
>>>> can't be the physical system's firmware, but I also don't see where
>>>> virtual firmware would be taken from).
>>>>
>>>> There is no PV EFI interface to obtain video information. With the
>>>> needed information getting passed via start_info, PV has no need for
>>>> such, and I would be hesitant to add a fundamentally redundant
>>>> interface for PVH. The more that the information needed isn't EFI-
>>>> specific at all.
>>>
>>> I think our only option is to expand the HVM start info information to
>>> convey that data from Xen into dom0.
>>
>> PHV doesn't use the ordinary start_info, does it?
>=20
> No, it's HVM start info as described in:
>=20
> xen/include/public/arch-x86/hvm/start_info.h
>=20
> We have already extended it once to add a memory map, we could extend
> it another time to add the video information.

Okay, I'll try to make a(nother) patch along these lines. Since there's
a DomU counterpart in PV's start_info - where does that information get
passed for PVH? (I'm mainly wondering whether there's another approach
to consider.)

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:16:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:16:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186869.335618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAAZ-0000Mk-8j; Tue, 14 Sep 2021 15:16:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186869.335618; Tue, 14 Sep 2021 15:16:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAAZ-0000Md-5n; Tue, 14 Sep 2021 15:16:23 +0000
Received: by outflank-mailman (input) for mailman id 186869;
 Tue, 14 Sep 2021 15:16:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQAAY-0000MX-Cb
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:16:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d7a9532c-d93f-4499-ad5f-9b66a47ea524;
 Tue, 14 Sep 2021 15:16:21 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2057.outbound.protection.outlook.com [104.47.1.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-8HoWLGSgO7mgBzjrM5Rt4w-1; Tue, 14 Sep 2021 17:16:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 14 Sep
 2021 15:16:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 15:16:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0011.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 14 Sep 2021 15:16:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7a9532c-d93f-4499-ad5f-9b66a47ea524
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631632580;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7lBNw3GpzbaauOe9fp6xbD5cH6EpZcG6XxCsx0N7VeU=;
	b=Yrs/0RSxh/q2sX9MCdTirXdnv/1D1p3znEqv+hIK9eWQesoyjeHhqw7sLOb5jIZECLPWBz
	74al3y8uI5kd3OjGSjuXdlI7sy5gMGQPUwd1FgImvWstvj/dZ//YBl/8WdB/IQ+yy8laaq
	v/xGvC8ICJFUJBMVuOLEdeaMNuuyp9I=
X-MC-Unique: 8HoWLGSgO7mgBzjrM5Rt4w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IEcVrm8TeQKyMUzDbnpgBDmJWEddILKWPjVPsR+StOT0T2LgkrLrmQzyuU4lnJVAe8sP9HD9T5xntQNFd+UoMcyxc+7byrdxBrLdMXyKv5Nik20qQM8AHeBS869NbYoeahUFH+z+IlaoenBL/Ld3nJnevFsFQENZTp3rwKr4rxw2wcHkKngFKxOevrq1leNGpUyMo/MI8fL2LsMwedzD/594BrY+Sd4Ibdz4dmqHfj1/0kY3krX5hJsJsPPgauEJ4FM8wB+usB3F/HOiZv2aFXUdCpy0CBjbLKCrXwM6MIh450WdtoMOhmg2inu7kBJXhZIpNhbgPZDjKkMckLHhrg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=RpA+BPFEAho6gwLyuh4KFPPGvMqgp8OhzuIrcaCHD5k=;
 b=KBqoeg6eVCIfzr2NLUwosfdcsT1J41FMuYpzGs3UbN1OciDerDxttBSc7xsnGaF2vXuzH+bPhEzv/c69W+SxVHxOHnsBffA8IkT1pQiseHDOZw/IjuEz0xmWCKq9UOItGP9Q4hecgfwTXyHfjgQ12T6DnoFG2uR/zrPp2W5OBFZ7CAlM7AnEIttj8hU6JRFR9pkmtXOwSDWNiD78Pm0LqWHI28/Zbsq8qHSaBY/yizDu1mxY6ab4KHAnweJAcRA6pWJfguf9P44Ys3+6Pa7MNIddUcaQgI+4V8yqFmyYJ5JBCva0birXon1iuJPTxnQ5O7hXAOgKSzZ9Y0iUG9c/5w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
 <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
 <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
 <df432c93-b722-6006-c3dc-7f511b377bcc@suse.com>
 <YUCwD5RWj0WKZZUc@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d8b1b328-7887-cde4-dbe4-bdc43a1d5147@suse.com>
Date: Tue, 14 Sep 2021 17:16:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUCwD5RWj0WKZZUc@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P192CA0011.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 34906484-4f48-4ee3-4e0c-08d977929a1b
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2335E5F142E5216B6661483AB3DA9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0OUNfLr3ndnyhFsNOOIi0eJlPETi0DLr4XUrT2muQODHTJKIykM8wsIjkt5zWvnTYQptmBoskjmVVHXkAGLlt9mOuOCV1q0JyqoTVrUlbxS/dKXy91DIwJId1VU0UmuOGYM5o+4L+eOGHNu29YWL1OEAsfruJ/vQegEaFZKJm8R4bCktpWZUbUCYu1BPveiK4JVAZFO+oJ7GAb/l3rkeZbXkqYisZ7WBg0oFZdZrhms7EHf8PKnr7Dh93oAkPY5f57LDv1UCp8g6oEH04GNyw7hRziHPXKmciFYTrMO9THW1jmKP/2dTE0wE4Aqo7tFs59W5dZPIQJaSOVjecVyUtr7iqDPF3V6XI2Iu6siqk1x01uzYQbzwWofgsSTF4CScDy0azTzbKChtqL6hpTtXHf3AQD8fRu+XmOINNJp+PtuZ9r7EAHO79teJPEE1WrOMNVSL3+UF4AYLRnEU68ibHjDK0l8oK5lZaZlivIM893+FbkxbNhIdlM//BTzvYEODd2mTLJV1FPlAyXKHUB5Z9U+BXXEuLsfu6unD2dD+luCW70SPxbhUutdo3HfdCFw0KmVbUiLcUW8QsfOcg/U+wdPRTY5oPelJ8uz18mRaOYFtP2ZJqedmG+tlJOKjA11LTBKXY4NwsIijrrg52x3I4GYsSuU0KTRNPqJMPxUtj/OkkM2h8LZsZroClCFKma245fUd6KlegDlRhqsmbXXlPXjnlBw7xmsdq4ORR3RDComhDV19AlAxjU+LNQLld2Lh
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(136003)(396003)(376002)(366004)(5660300002)(83380400001)(86362001)(8676002)(4326008)(956004)(66556008)(66946007)(66476007)(2616005)(316002)(16576012)(38100700002)(15650500001)(6486002)(36756003)(26005)(31686004)(6916009)(31696002)(53546011)(2906002)(186003)(478600001)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?aH3SOYFuTBDVrjyxBugqdiQ1F5Gg3ORqSTfP1vKodnim2/qotz82WkuOGrYF?=
 =?us-ascii?Q?IRCIi9CM2T6n//vpeTAVPVdcLw8n3JDfxOD0VXAkYFcM9WtIHQmymv7YqevE?=
 =?us-ascii?Q?XpYLkwfH3ynq2U+lA4lJIy67NGhQCEL1P2yGu7Y2XMDKMSIGvE3ilxOhlO9d?=
 =?us-ascii?Q?/Hu5xWMn4JN3vrDONrZN2tGWYcYZW6zx6TyEyMbj9LTCcohUHOYGmanqpLz8?=
 =?us-ascii?Q?qV0iLmwsY2tUpyVXZDD8spwT5NlSM6mqRQgNFNqnYiS+0XMhlkzxfy8BHUkV?=
 =?us-ascii?Q?3H4mCGBco/2NJC7nDL6LkJxHe4CtXoQFMAiSUEmLXLXywSyI9wbo2YLtMrF2?=
 =?us-ascii?Q?V5KYL7nvyBlK+mVrfMGhEYItNQI00TGiPcttkl5icKw8g7g/sKL7a6ABy4wE?=
 =?us-ascii?Q?OBIc1paZkuu4n7dbi3bvAoEwo2NgcXxLS4WsKRTdSP2GXRBs+yCw7oVMcfYx?=
 =?us-ascii?Q?D0pvQfQsYgKv1Nxy3wC0pFPpohwyhEDPC2p0jvMsG1PCIdE7hRwXZdyxb6aG?=
 =?us-ascii?Q?+vt6/HRTE3z4kAKo3I33F9RNImq8VAt9BWNlPS8p1RdMRFCSayhBbwIJvh51?=
 =?us-ascii?Q?0eYwYzd7ZWzetd/+7qzkDgXzYTbYTOVe2SStI4ezDZbOev+V5SN/5OpQArhE?=
 =?us-ascii?Q?imzRIg4PpO1wlYlWpf8MxYaqZkQ/gC/Lwd46s4AvkjJfVW3KBiyw+GYsA2hS?=
 =?us-ascii?Q?zdEItPkrFLBZYH8V39I4gp0HsTq7n8YqBr/eU6O3ZaJpuK/s9a6R6NYaAmqV?=
 =?us-ascii?Q?/M/j/d72K7OL0HuyHikux7/Q6iV31ZxCIMG7FTHeZyVbi1MW3LEUcixLE15z?=
 =?us-ascii?Q?8543tL6Lb2SSFmRqRNXiqeUFnF7hcI/XGI6iB6R0f/A5hQZhotBvKQjXlHEc?=
 =?us-ascii?Q?HEdDTbdN3bXoyarBJh7bfJRBcPXWeosPE5m5vKRpQEOPg4DhbqcCWAonduJ6?=
 =?us-ascii?Q?rwFR2X3I6nKKT+Dd8hUWEom6J4AuRF7CTkm1vo9l1XQv73zz3KHHhYP3vIte?=
 =?us-ascii?Q?c8rOyhKnTm55n0SyUDbPbJBt0Eey36xOMWJ19LjxPQSDIdJLNx3vx5/anHhb?=
 =?us-ascii?Q?16BPVUA1qRaREbinlYYafd5rTmf25nGafFUbqXopC3zH5DwivjwTywgW1Mrw?=
 =?us-ascii?Q?qUBbSKlyZdQzHwnnj7pRKKSf+ubTmdvtpQO332PMtpCfaxySu8c9zOt8Bm4J?=
 =?us-ascii?Q?35T+3xcDOzkTOEaUqZ9nBKvcg3WPejIJPoy1k+zY/k2caDCndh5ox0XBXXRe?=
 =?us-ascii?Q?Z1bGZIUY7E6LimeqkQ81RQXT/L7RSOxElK8g/WyXM9Fidj5Nurg7svKqo8/1?=
 =?us-ascii?Q?l46CleVd8r9E8z3oYoznaeyk?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 34906484-4f48-4ee3-4e0c-08d977929a1b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 15:16:18.5316
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GhfBy44hdnjRMkzcOMHE+QxpziyhPGR5G5JCjdwbb0hzBKfoSWg7j2BB1xfe/v7Ir4K/E0tibtvgjquVVb9c4g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 14.09.2021 16:22, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 14, 2021 at 02:05:01PM +0200, Jan Beulich wrote:
>> On 14.09.2021 13:21, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
>>>> On 14.09.2021 12:00, Roger Pau Monn=C3=A9 wrote:
>>>>> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
>>>>>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
>>>>>> console) are associated with DomXEN, not Dom0. This means that while
>>>>>> looking for overlapping BARs such devices cannot be found on Dom0's
>>>>>> list of devices; DomXEN's list also needs to be scanned.
>>>>>
>>>>> Thanks for looking into this, I certainly didn't take hidden devices
>>>>> into account for vPCI dom0.
>>>>>
>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>> ---
>>>>>> RFC: Patch intentionally mis-formatted, as the necessary re-indentat=
ion
>>>>>>      would make the diff difficult to read. At this point I'd merely
>>>>>>      like to gather input towards possible better approaches to solv=
e
>>>>>>      the issue (not the least because quite possibly there are furth=
er
>>>>>>      places needing changing).
>>>>>
>>>>> I have a couple of questions, AFAICT we currently hide the serial
>>>>> console and/or the VGA adapter if it's in use by Xen.
>>>>>
>>>>> I wonder whether we need to add vPCI handlers for those:
>>>>> setup_one_hwdom_device will attempt to add vPCI handlers to the hidde=
n
>>>>> device because of the temporary override of pdev->domain done in
>>>>> _setup_hwdom_pci_devices, but I think that for hidden devices we
>>>>> shouldn't add vPCI handlers. We should instead block PCI config space
>>>>> access from dom0 to the device so that it doesn't mess with Xen usage=
.
>>>>
>>>> The answer to this follows (I think) from the one below.
>>>>
>>>>> It's also not clear why does Xen want to have those hidden devices
>>>>> partly controlled by dom0, as it would seem to me that dom0 interferi=
ng
>>>>> with hidden devices in use by Xen can only lead to trouble.
>>>>
>>>> Dom0 can't interfere as long as it can only read from the device.
>>>> Restricting accesses to reads is one of the purposes of "hiding"
>>>> (the other is to make it impossible to assign these to a DomU). Not
>>>> allowing Dom0 to read from such devices would lead to wrong PCI
>>>> device discovery - some devices would be missing (which may or may
>>>> not be merely a cosmetic issue). If the missing device is a multi-
>>>> function one at function 0, other devices in the same slot may also
>>>> not be found by Dom0 (depending on whether it looks at non-zero
>>>> function numbers in this case).
>>>
>>> Hm, indeed seems possible that missing function 0 the whole device is
>>> skipped.
>>>
>>> Maybe we need a special vPCI handling for those devices that just
>>> allows reads but not writes, and that doesn't maps the BARs into dom0
>>> p2m?
>>
>> Not sure about mapping. They could be mapped r/o. And they may
>> actually need mapping for multi-function devices, but I guess for
>> such devices to actually function properly then there would be
>> more work required.
>=20
> I'm also slightly puzzled as to why ehci-dbgp uses pci_hide_device
> while ns16550 uses pci_ro_device instead.
>=20
> Is this because the PCI device used by ehci-dbgp must be shared with
> dom0 for other USB ports to be usable, and hence dom0 will need read
> and write access to the device PCI config space and BARs?
>=20
> Note that pci_hide_device is less restrictive than pci_ro_device, as
> it doesn't mark the device as RO.
>=20
> That would seem quite risky, as it's likely that dom0 will perform
> some kind of reset of the USB device and thus the console will be
> lost.

There's actually a protocol to prevent exactly that: See PHYSDEVOP_dbgp_op.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:29:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:29:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186884.335629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAN3-0002Fp-Jr; Tue, 14 Sep 2021 15:29:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186884.335629; Tue, 14 Sep 2021 15:29:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAN3-0002Fi-GP; Tue, 14 Sep 2021 15:29:17 +0000
Received: by outflank-mailman (input) for mailman id 186884;
 Tue, 14 Sep 2021 15:29:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=V0d6=OE=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQAN2-0002Fc-BH
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:29:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b761ff87-b241-4d19-bbba-f9bf6e2748bd;
 Tue, 14 Sep 2021 15:29:15 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-G5eDJQa8NLqkuyWUKAJ4qg-2; Tue, 14 Sep 2021 17:29:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Tue, 14 Sep
 2021 15:29:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4500.019; Tue, 14 Sep 2021
 15:29:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0051.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 14 Sep 2021 15:29:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b761ff87-b241-4d19-bbba-f9bf6e2748bd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631633354;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hhWzPqc+baNtksScInoyMTGZMgEM+jvg4iBfu2+k9DA=;
	b=Ci7BohGNFAqwR6KZtF+hLRGGq6Vo5ezkQ0d/lxK644/4bXw7G1wip08Z/9i3qaez55vbRS
	VKOBnOAYEQiY5Umvgb5zM0WkkekceV1fsW5hvB933XopjtXzMxjbokMMkQhDpPzoRJKAPu
	8fOyHYr5z3CBVgP9yU2werzZ3WMvtPo=
X-MC-Unique: G5eDJQa8NLqkuyWUKAJ4qg-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UBAA5TupC446mVSVjKo9V+J/Iu2eC6Fb9hV9F7LdCa/pTPM1LXOq3n7wdIh/W6iKArAJegcUaGTVUFwXKmlJ1vAwO0ulUp3u4AzA/3JfCtTv+Ns/q2J9H8P00HOrWojtu0LTVrPPEL3GcP+ydQJkVcAfW581SvJTdg3hjIuTrQ9fQi/vQO4E0OZHSdIG1piC015SXDF5HOPlUeGDSnOOKuel1gmHgL7wLhKNtepAwlTKj6ibcWaO9FQCrkhzIJf1fOl+2viAxrwkkz99Utk4vL9DqEkuPtp/3Wn7ACyxbTI5IAeRSUg2d51XUSqA4XocQ7nbc627yErH0R91ZFTk5g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hhWzPqc+baNtksScInoyMTGZMgEM+jvg4iBfu2+k9DA=;
 b=QJXBGHRJoxVYSgQKR75ZqtQBu2JWN2ijj7/zhZhbrHSbInJOTW8wp8HgnVgJZ3os0DY5EQ++b4XgpKsUB8vWoVhMwmQfPHOwb++lBH7KPqcomUWK+bjJykWjWLv3+l9078TL25bjJ+A/ESIDB4hidja/xztszSGjyCDorde6oWh7RAx9/o9A/lVCp5P/oPvUAXAAnUPTIus96qxT2LrYnwQZjnb6/VxOT2kZFZnYUdAwuxWeyywkE8RD6/Xnkt44oi6zYOISTow27UQIayqgcn1bGRXoO18PfL2CxfaC/wadWMOJBx2MnnqdJ8VrrbewgTRVlPpriMLqeMUHyzs3cg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.linux-foundation.org; dkim=none (message not
 signed) header.d=none;lists.linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Roman Skakun <rm.skakun@gmail.com>
Cc: Andrii Anisov <andrii_anisov@epam.com>,
 Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>,
 Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
 Michael Ellerman <mpe@ellerman.id.au>,
 Benjamin Herrenschmidt <benh@kernel.crashing.org>,
 Paul Mackerras <paulus@samba.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Christoph Hellwig <hch@lst.de>, Marek Szyprowski <m.szyprowski@samsung.com>,
 Robin Murphy <robin.murphy@arm.com>, "Paul E. McKenney"
 <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>,
 Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, "Maciej W. Rozycki"
 <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>,
 Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra
 <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>,
 Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
 xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
 linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
Date: Tue, 14 Sep 2021 17:29:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210914151016.3174924-1-Roman_Skakun@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0051.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 074dfc55-95fb-49c7-e925-08d977946657
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2334A539B4FCAEFF4E2FA551B3DA9@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	RYoU3P+Y0RnTn+IGj2fzDOBGc0v8jOjAO4pN2pmc16lKx818yevzS30dMvhdz8ceOWjTOkEM2fwnNDUASKTZayhy9rTCD6v1kleeE95zvxxPwyMpZh1lfXPQ3JKpsnonX5LNEh3Vu1fbmXdFeqqKzz25vgPzSSbLV/OwqEQknaLDrzwnfTgeznDqJWjxlsVcC9sYHoDrLtB49Fvsh6IlCpGm8/wpByf/wdbjoTMl9Pt+dRkWZugoeRByDq0FzpAimYt+VNZPTzTqOVlw8On5eQ3VpG/OSAk4ObykCcm8oUPoV4SzRiyhIfLq9/DBSQuXKAaHQd1wPwq97F68XgxDHeVH/PSatmhRN8/zckvBgT1UYVfP5jSY+my4czYnYpTWo6CblQ9i/fB/cSMhISvjqOC6hmc+3k0UNmIGSEQtdhhh+HTFaiKuXSYOwOQ4cRE06uiboKxXkhyKGbpdhRcoNKsQ+HTDZcTGkgoaVFampZVCZRNALVZqsKyZwoVX7qnJasU0zOUq5sEAyOPU0L+WsdKd5UniZpYJQuStlHxc8ljXn64I8NDE5XHFvJwXo4d1KeKcB76a/WB8j8+PuBbjjxFwSOSnW2HnDzuxjHdSjzNTl2IjkMcsMKSTv4PT4BEpED6FeRw0QTmFwVZHmyvw+2GLuz4rdzFWqyrzzf+pGwLNmzz6/02MjNVBc0sZJMVKLIgzSuLM4PKzYwoda1zxRzGztL+b4kMyNTKFStZFPzGStXTgpwAuWdUTGqW5YCw00LM+SoDKifedQtHjkZnHZyVI8YPbDerPxyC8d4Dv4deopMYuAlO12AuSlGEIRTCc
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(366004)(396003)(136003)(376002)(6916009)(86362001)(66946007)(966005)(16576012)(38100700002)(186003)(316002)(5660300002)(6486002)(83380400001)(7406005)(7416002)(478600001)(54906003)(66476007)(66556008)(4326008)(31696002)(2616005)(53546011)(36756003)(956004)(2906002)(8676002)(31686004)(26005)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aXpaZmNvUjZVNWNjVGhZcVJFZzJqR25PcGR4bjgvcmFQMDZ4a1pCdFZVNDN1?=
 =?utf-8?B?d0dlenhXMXp4cVpaT283UGNtc0NPZXpHY05SRmtxc2RTZ3B5NElLaGhocXVL?=
 =?utf-8?B?SWszUmY5MDZaNlNtajljYmZMMU1MWDl4UnZwSHk1QUEvbXBBa1Q2cWtHRDBL?=
 =?utf-8?B?TjdFUEVpNThPVFNxMkc1ZVk0NHpFcDNYdTY3Y2pOSkNUWmJNQWkvNStld2V6?=
 =?utf-8?B?R3J1YmFDaWtvZVVVc2ZXakJRYXlFUjZGSU1MYmdaQXZjTDJ1NS9Ra3ZGelNa?=
 =?utf-8?B?TW5WR2JIR0Q0ZUgxaXRHaWcvaytsRUpVQUk4SFl0Vnl4QjQxQUk1VGhBZGg2?=
 =?utf-8?B?ZW54dmJCMGtVOWdpUVNDdUg1eXNsMEZNY0N3S1Z2Rko0RWcxdnhKY0lnOU5L?=
 =?utf-8?B?WXA3a1ZTdUhNQUFHNmpjMFZiMUhsUjZtVzN2L0ZzOGR3VzBEaXU1NHJnTWo3?=
 =?utf-8?B?aGJxbFMvN2dFVXl2T2tQRXpHUUY0K3JFby9JUzBnM3dvRkgza2YzSS9vTUtI?=
 =?utf-8?B?WjhSQ2Z1aHBGREJHekRucm1lUFVuMWEzY0NYdFdJNVE0alg1YjhDRXRFWHBx?=
 =?utf-8?B?UW4yTHk4QnVFKzNjOUE0eFZLQ29TYkNMSHdHekpzSFdPNWw5cXBMYjdwK3F1?=
 =?utf-8?B?dG8rdjlrUm51RDBnUGV2a3QyQUFoWHRyWFdQUHdGYVRyZ09rZTJkSHVVT2RQ?=
 =?utf-8?B?K1Fuc0FBTFprMUJXTzhpbktDLzNNeWkzVDkzV0NaTFNDTTd2THMwZ3BTcFZh?=
 =?utf-8?B?TVVYL25rMVFyTFdzSDVJOU1PVWRxTHJ1OUQyaTQ2dVNSNVBHd1RPMm50OUpR?=
 =?utf-8?B?WDcxNWhKbHRDREgzdHNzbGlESkZsNU5ZU2VDandSQ2o0OGlOWG54K3pDZ1Jr?=
 =?utf-8?B?Wm5qc3d6Q05oY0pUVlo5bm05azROL3lDNWRKckIxdUM1TmtWYzVMakF3NVVo?=
 =?utf-8?B?d2dteUN5Skt1N09KSndoZG9vRW5oUVE2blgrSmFuYTIvc3ZVQ3AwRytYa0Jh?=
 =?utf-8?B?aU1tVmRsMEdxN0NsL0ZJai9UNHBwU0hHTTJ0STZWaFlWV2RmZGZ0SmtpYkdh?=
 =?utf-8?B?TXYyOTJwdnVJVXI1RUN5OEdIT0E5Vk1iZTZab3pJaTUxR0JaN2tSNlM0MjN3?=
 =?utf-8?B?cEdEWmlCSWk2cXlGNVV4eFlGNHhYdThpV1BXWW9pRzh5cTRhUEI4M1pYVU9x?=
 =?utf-8?B?REFJSUlva05mTERKRkw3d0Y4QTZQS2hQSFdkUkRBQXJZcG95T2Q0VkhzL1dJ?=
 =?utf-8?B?NmxzRjA0VGcrU3Y5VDVKSjA1UFBQNHdBYXF5SUNrdzBiZ2F1cGQrODR5ZnRK?=
 =?utf-8?B?ZW80K0J0aFh4ODBCU0VkbXZqeXlhQW1WcndSRW53UVlZaUduWFJrSWhwcmJN?=
 =?utf-8?B?Q21XcDk0Um5lNTVHdnpKeXdjcU4wV3Q3YjhyL3JyZ2NFdUdLbUZBOEpBTG1I?=
 =?utf-8?B?bC9LcmZEblJwYlBQd2hBS2Fza0pRSHVlSkN1THdoSFJKUXBDa28yR0s5dXpr?=
 =?utf-8?B?V0FmWDhtZ2s1aHJxSFVsUTJRWUg4aktIV282MU0xeWhKL2tFV3h5V1JUNXVx?=
 =?utf-8?B?SUNSaEdVdFIvTTkra1pxSld4VFAxZ05XMklTTU1wSGhPSWM3c3hCdkJXVjlr?=
 =?utf-8?B?UzhOdXJwSTdhNkxCYlBHVi9vV3dpUis5dFpOd2VreFlJK3N0L241OHFkMEx1?=
 =?utf-8?B?QWpZODJpUFBXLzRHZEZUeHJpdXh5bUgva3JZS25HV3hMdmszelQ2VVRjbThp?=
 =?utf-8?Q?EHtteRfnHGiG4VVIJeSMsg07Np7Q1JnvbOEPUBb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 074dfc55-95fb-49c7-e925-08d977946657
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 15:29:10.7377
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kXdQssFPz9rIrTB1YSXP35DiSNUdVE4BfSOJKHu3+j7Ku5TB+6FgftUS2/Mi3A3sAouc4CUWJBS2Nck1t6XQaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

On 14.09.2021 17:10, Roman Skakun wrote:
> From: Roman Skakun <roman_skakun@epam.com>
> 
> It is possible when default IO TLB size is not
> enough to fit a long buffers as described here [1].
> 
> This patch makes a way to set this parameter
> using cmdline instead of recompiling a kernel.
> 
> [1] https://www.xilinx.com/support/answers/72694.html

I'm not convinced the swiotlb use describe there falls under "intended
use" - mapping a 1280x720 framebuffer in a single chunk? (As an aside,
the bottom of this page is also confusing, as following "Then we can
confirm the modified swiotlb size in the boot log:" there is a log
fragment showing the same original size of 64Mb.

> --- a/arch/mips/cavium-octeon/dma-octeon.c
> +++ b/arch/mips/cavium-octeon/dma-octeon.c
> @@ -237,7 +237,7 @@ void __init plat_swiotlb_setup(void)
>  		swiotlbsize = 64 * (1<<20);
>  #endif
>  	swiotlb_nslabs = swiotlbsize >> IO_TLB_SHIFT;
> -	swiotlb_nslabs = ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
> +	swiotlb_nslabs = ALIGN(swiotlb_nslabs, swiotlb_io_seg_size());

In order to be sure to catch all uses like this one (including ones
which make it upstream in parallel to yours), I think you will want
to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.

> @@ -81,15 +86,30 @@ static unsigned int max_segment;
>  static unsigned long default_nslabs = IO_TLB_DEFAULT_SIZE >> IO_TLB_SHIFT;
>  
>  static int __init
> -setup_io_tlb_npages(char *str)
> +setup_io_tlb_params(char *str)
>  {
> +	unsigned long tmp;
> +
>  	if (isdigit(*str)) {
> -		/* avoid tail segment of size < IO_TLB_SEGSIZE */
> -		default_nslabs =
> -			ALIGN(simple_strtoul(str, &str, 0), IO_TLB_SEGSIZE);
> +		default_nslabs = simple_strtoul(str, &str, 0);
>  	}
>  	if (*str == ',')
>  		++str;
> +
> +	/* get max IO TLB segment size */
> +	if (isdigit(*str)) {
> +		tmp = simple_strtoul(str, &str, 0);
> +		if (tmp)
> +			io_tlb_seg_size = ALIGN(tmp, IO_TLB_SEGSIZE);

>From all I can tell io_tlb_seg_size wants to be a power of 2. Merely
aligning to a multiple of IO_TLB_SEGSIZE isn't going to be enough.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:30:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:30:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186890.335639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAOb-0003Yn-Vk; Tue, 14 Sep 2021 15:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186890.335639; Tue, 14 Sep 2021 15:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAOb-0003Yg-Sg; Tue, 14 Sep 2021 15:30:53 +0000
Received: by outflank-mailman (input) for mailman id 186890;
 Tue, 14 Sep 2021 15:30:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SMEH=OE=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mQAOZ-0003YX-Oq
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:30:51 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bd686c22-1570-11ec-b44e-12813bfff9fa;
 Tue, 14 Sep 2021 15:30:50 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 375B567373; Tue, 14 Sep 2021 17:30:47 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd686c22-1570-11ec-b44e-12813bfff9fa
Date: Tue, 14 Sep 2021 17:30:46 +0200
From: Christoph Hellwig <hch@lst.de>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roman Skakun <rm.skakun@gmail.com>,
	Andrii Anisov <andrii_anisov@epam.com>,
	Roman Skakun <roman_skakun@epam.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christoph Hellwig <hch@lst.de>,
	Marek Szyprowski <m.szyprowski@samsung.com>,
	Robin Murphy <robin.murphy@arm.com>,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	"Maciej W. Rozycki" <macro@orcam.me.uk>,
	Muchun Song <songmuchun@bytedance.com>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
	xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
Message-ID: <20210914153046.GB815@lst.de>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com> <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Sep 14, 2021 at 05:29:07PM +0200, Jan Beulich wrote:
> I'm not convinced the swiotlb use describe there falls under "intended
> use" - mapping a 1280x720 framebuffer in a single chunk? (As an aside,
> the bottom of this page is also confusing, as following "Then we can
> confirm the modified swiotlb size in the boot log:" there is a log
> fragment showing the same original size of 64Mb.

It doesn't.  We also do not add hacks to the kernel for whacky out
of tree modules.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 15:49:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 15:49:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186906.335651 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAgn-0005T8-IU; Tue, 14 Sep 2021 15:49:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186906.335651; Tue, 14 Sep 2021 15:49:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAgn-0005T1-EG; Tue, 14 Sep 2021 15:49:41 +0000
Received: by outflank-mailman (input) for mailman id 186906;
 Tue, 14 Sep 2021 15:49:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=a0c3=OE=microsoft.com=haiyangz@srs-us1.protection.inumbo.net>)
 id 1mQAgm-0005Sv-OC
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 15:49:40 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.8])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5e8c3fc8-1573-11ec-b452-12813bfff9fa;
 Tue, 14 Sep 2021 15:49:39 +0000 (UTC)
Received: from MN2PR21MB1295.namprd21.prod.outlook.com (2603:10b6:208:3e::25)
 by MN2PR21MB1167.namprd21.prod.outlook.com (2603:10b6:208:fe::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.1; Tue, 14 Sep
 2021 15:49:29 +0000
Received: from MN2PR21MB1295.namprd21.prod.outlook.com
 ([fe80::d804:7493:8e3d:68d3]) by MN2PR21MB1295.namprd21.prod.outlook.com
 ([fe80::d804:7493:8e3d:68d3%9]) with mapi id 15.20.4478.015; Tue, 14 Sep 2021
 15:49:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e8c3fc8-1573-11ec-b452-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IVpYj/cHMGnqw3aZqz3CXub/GCyOE2J+snAOvN13PN21RYy+XKkUABQQgY9faJh3oIhurbFh8GVXyzWq7ysq5bSFE9jIss4vYw2r6IzDbjZaOCRmYe7yvbk4OHX52wTdxmh4bwj4XbTZTGdvL2T0jQ3crvPcSOjlbuivofTA+DTNvpD1LfHoeBCX2600QIis1DsonUDjwfRdcMa3V73EQRF7DgmaSNWFt9H4xYo5lkdoSEpz3a4FdgV6jDYMTbh1IoBz8PVDyBGsd0dme6CS2BlSj1UjNRZZ0y57AKMx2onrgR0irV/nzAJkOfEkdABExuPLrqETpa5pzMBDxGyYQg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=F1OIXYjt1pXx+hzAgzU8rYjUG5YvunZwpM+CWxD9qNA=;
 b=A0gw6NlIUkBR2crB4hPIXSukhrZZo5gTUw8lReDD40jzWxeecOlQuElVzasG0f0PenKOn+KqekhvQ3r69AMCqiQ0Czuw2QZMqYrnY7A8KnDvdeajn3qlw0FeES2RlFy0cs/JJGHUMMYJ1mqkqZNYvVC1nI6d28ihqX7JqXWj7aP6pzakKe0BPhG0hyDPF94EVsgiDOXg80Vp2VvGmekkqYZqeT2OCNun9It1AEAgf6vCFEqGPonmHN/zr+kiO0NmykBfMsmqTyhWRZt4YdwiQYz+ZBqJOudCkidmjJiZ3gDPpl6vqL6PL5bJrho8b4nc+UpmeKNWBqLpO4VVWhEmUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=F1OIXYjt1pXx+hzAgzU8rYjUG5YvunZwpM+CWxD9qNA=;
 b=aHx6tePt+bfF9MDFHVxp06gNV+r5+IlTaCjIuaEUdLR9q15DKvn/c91O5froQl2Bz+UOQkts25lPvRjEOj8peFlNIrDPpKWTdiFa4qDxJAFUq3LTN4YxvnXS6HCdmBP4veQPtsX2Lw3XQpwMHezvtHrl4DdPSzt7F9eTpjKY2wo=
From: Haiyang Zhang <haiyangz@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>, "wei.liu@kernel.org"
	<wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>, "tglx@linutronix.de"
	<tglx@linutronix.de>, "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de"
	<bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com"
	<hpa@zytor.com>, "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>,
	Michael Kelley <mikelley@microsoft.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
Thread-Topic: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
 netvsc driver
Thread-Index: AQHXqW4FQ8MnKzPhAECYP0JKGb2C7qujrPRw
Date: Tue, 14 Sep 2021 15:49:29 +0000
Message-ID:
 <MN2PR21MB12953A434D67350B4C80C2D8CADA9@MN2PR21MB1295.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-13-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=ec0b8299-6ace-4e10-940d-c2bfa0583e06;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T15:48:05Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 31614527-6ad5-46a5-dfcc-08d977973cd4
x-ms-traffictypediagnostic: MN2PR21MB1167:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MN2PR21MB116751FFC2720429D6E376A9CADA9@MN2PR21MB1167.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5797;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 wSJCRkd0cIPW5RiB6Hy8Wq6KnUDIjL3h8musDztMljzwi3S4UOrYjSbWAX1KY4xrj8RHG4RxTY9fuzE/Pn0/tt7Zuk39d+9t6tz2kfSF6Tt84Apizls2wXUQvRp0omWDuMtTV1LMJK2+kxreafrZIamRjHliYh57I140ZXj6DtxN/Q7RPb61rYuD59qKHoFk+uDzp77AvLT8ZcVjFXDhUiuaWWedtSpssDBtov0rRSUHLnW06zLUa116qRCfC8isF5ztHIN7ROy7tvEa0U02TvsDDZH8vrZIDrOA51i1BrYZOus8V2jdsZm2EbArOtc1bcO3aUOS1Vx/e2ICU99J3T+cE8SQE63Tzfy5quZXIiTm/J8oPleuCGuma6+wG2VShOVcBz35pw3cCdTRzF2CmGW76u/a9ZthScCx3Q2r/ZZ218fmTkCP13HexIQitwKJT7mL0XeWSrqj6hM5t2sYh53PmmwzXiDY3SmBYYjke2WPXXqlLrg+U4yZDjwQyrevRUiGTvYudcBi2fRMPFHwe8KfKNBdyo5QbNba/f34OwvhaSZMrqiyXXI3hA2iAENZH/8iWi3a4OREPoN9RhL4VQQS6bzt40PaKfQDTLhdVP2Pt+y4+WMv5QcnREqFMTc7G08EJOf3SU8D1y804oE06ZGwVZYl4a3cJZ2osm6V+ICb+JnDchMSr9kfqD0l5BfMXkJwy1eDd+h0byEyqt4i4c84qqRyNGE7n7tMqKHxyr8=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR21MB1295.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(33656002)(55016002)(9686003)(6506007)(508600001)(53546011)(26005)(921005)(54906003)(38100700002)(76116006)(122000001)(71200400001)(5660300002)(4326008)(186003)(82950400001)(66476007)(86362001)(110136005)(82960400001)(52536014)(8936002)(38070700005)(2906002)(8990500004)(316002)(83380400001)(7406005)(7416002)(64756008)(66556008)(66446008)(10290500003)(66946007)(7696005)(6636002)(8676002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?VbT88gCNa2ZyW6pwzrLrtQuTO6cYgL+U6HzYbJCr1dL0GDoiBcGMdjEUJ2Vy?=
 =?us-ascii?Q?agTdnqM44iiB49YKumLaafY7Na7ZqkUYBBog+poLzvFVKmHGELz0ThYDVxUA?=
 =?us-ascii?Q?1OLzRK1z84a369F0ynOtDUsXFixwMKvIQAlPLCqvcfR29ElpahU0ZzAEkArD?=
 =?us-ascii?Q?z6nYGmqR3AED/WU/eS2ITwXIbUI76iS0bVWk0j0wv0chfiof3s+w1YRu4oai?=
 =?us-ascii?Q?UTaWt/MKmPBsYQS9pS+kShafgw9N1HUOU0cvHZOceelBOV+naLz8TmRE3EjP?=
 =?us-ascii?Q?YFW21N/d1ECqhu0jCzr50Aec0sdl0DLmsDqDHt6UukRSramMmR0zYiKLW11X?=
 =?us-ascii?Q?DAaF2oZyb4Q1sFIR8TW/fd66c7XAwgejZiSU9bEPIho1n7eN7BkcDrx2pZy7?=
 =?us-ascii?Q?NUwvIhzVYnYMn0mYH/HfUCMZ8PBtRZnO6fFEoTJ3SKeYPY3ySbKYhY0qW2XC?=
 =?us-ascii?Q?QmV/K7+3eBf5JAZY6T4FvplEzu3RlOL0MO0bbwLifI+wsBpTxnI+JDm5lae/?=
 =?us-ascii?Q?dy/LA54thkcXlnQcZwd/woHuhjXcyCKoaMDk5oxKba3E579urw1bVTBFDteN?=
 =?us-ascii?Q?Elc0EAGxtIUT/5M17mlVQo8DXc8kPToL3ShQuw/oXi1mWsPVUw4MRgIQtth7?=
 =?us-ascii?Q?eGTAhP4R+xLDAIb0GPbyK6S4fI/4k4Vg0rxpsnEDjhQnoBp9OcKMFWM1e0md?=
 =?us-ascii?Q?5fN2aEIbTv9oTRm3Wcnxdsn4rXJRZsukGJiK1yIuubdLREIF1WfQ2oPgj7D3?=
 =?us-ascii?Q?7oGWz/lsw2/pcKKQja3ZdNVoYm3DEwl3razbo+1IO0u7lPys6A9EMIoTd7L2?=
 =?us-ascii?Q?qZOmMFCu69PAmRuBtONteSDKCzJFcnGb+2DEseIukuSCKb3egenv5T4cVYz1?=
 =?us-ascii?Q?MQ5ULFcnLRaS1lKI+8qqL3j5e5vSY3VFKRVbtPoph5OfihjsS5QsiIx1pCN7?=
 =?us-ascii?Q?c+g36cdxL/wEv4PYNe9SM37lV5ufw7iUTaDL9C5CvNRRAvMp+JzxkCillp3Z?=
 =?us-ascii?Q?FGoo7m9dZ3ysfxtl27vazGUUl8HkadesHKQYdb6iCR31WTqPbthp7kgRkeZY?=
 =?us-ascii?Q?BwWeNCswSLWfJ66vAIDGmc3WR3gm23tmFQcnX7pWbY+inINiOnYISzC4DYja?=
 =?us-ascii?Q?m2n1OCrnDtp+NJBlj6B8qBpF/yeJqMWngWZiWU9WupLX/kHPeGsL3oX9e1kn?=
 =?us-ascii?Q?x3lnupCzCVLG0jkuuDVKqwb1zm24UMg8ubTgpdNjkBLskqEb4/CpJTJpNnpj?=
 =?us-ascii?Q?9kKrEpO/WfJ8eqaJo4J6GVPbrh+saoO1pZr/P/AfzxHaruq6Ww1OJBdK+4UU?=
 =?us-ascii?Q?k4x/jMEmn2EKUOgrHt4g7hxR?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR21MB1295.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 31614527-6ad5-46a5-dfcc-08d977973cd4
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 15:49:29.1987
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 7/zdvCskn8y0eguwZA7zusKyl7sZe+BazC1uD+5HblPJjRGcyQGn6dxnrXQ9O2+us+43B83NY99FRAlb///ZXg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR21MB1167



> -----Original Message-----
> From: Tianyu Lan <ltykernel@gmail.com>
> Sent: Tuesday, September 14, 2021 9:39 AM
> To: KY Srinivasan <kys@microsoft.com>; Haiyang Zhang
> <haiyangz@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> wei.liu@kernel.org; Dexuan Cui <decui@microsoft.com>; tglx@linutronix.de;
> mingo@redhat.com; bp@alien8.de; x86@kernel.org; hpa@zytor.com;
> dave.hansen@linux.intel.com; luto@kernel.org; peterz@infradead.org;
> konrad.wilk@oracle.com; boris.ostrovsky@oracle.com; jgross@suse.com;
> sstabellini@kernel.org; joro@8bytes.org; will@kernel.org;
> davem@davemloft.net; kuba@kernel.org; jejb@linux.ibm.com;
> martin.petersen@oracle.com; gregkh@linuxfoundation.org; arnd@arndb.de;
> hch@lst.de; m.szyprowski@samsung.com; robin.murphy@arm.com;
> brijesh.singh@amd.com; Tianyu Lan <Tianyu.Lan@microsoft.com>;
> thomas.lendacky@amd.com; pgonda@google.com; akpm@linux-foundation.org;
> kirill.shutemov@linux.intel.com; rppt@kernel.org; sfr@canb.auug.org.au;
> aneesh.kumar@linux.ibm.com; saravanand@fb.com;
> krish.sadhukhan@oracle.com; xen-devel@lists.xenproject.org;
> tj@kernel.org; rientjes@google.com; Michael Kelley
> <mikelley@microsoft.com>
> Cc: iommu@lists.linux-foundation.org; linux-arch@vger.kernel.org; linux-
> hyperv@vger.kernel.org; linux-kernel@vger.kernel.org; linux-
> scsi@vger.kernel.org; netdev@vger.kernel.org; vkuznets
> <vkuznets@redhat.com>; parri.andrea@gmail.com; dave.hansen@intel.com
> Subject: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
> netvsc driver
>=20
> From: Tianyu Lan <Tianyu.Lan@microsoft.com>
>=20
> In Isolation VM, all shared memory with host needs to mark visible
> to host via hvcall. vmbus_establish_gpadl() has already done it for
> netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
> pagebuffer() stills need to be handled. Use DMA API to map/umap
> these memory during sending/receiving packet and Hyper-V swiotlb
> bounce buffer dma address will be returned. The swiotlb bounce buffer
> has been masked to be visible to host during boot up.
>=20
> Allocate rx/tx ring buffer via alloc_pages() in Isolation VM and map
> these pages via vmap(). After calling vmbus_establish_gpadl() which
> marks these pages visible to host, unmap these pages to release the
> virtual address mapped with physical address below shared_gpa_boundary
> and map them in the extra address space via vmap_pfn().
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	* Allocate rx/tx ring buffer via alloc_pages() in Isolation VM
> 	* Map pages after calling vmbus_establish_gpadl().
> 	* set dma_set_min_align_mask for netvsc driver.
>=20
> Change since v3:
> 	* Add comment to explain why not to use dma_map_sg()
> 	* Fix some error handle.
> ---

Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>

Thank you!


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 16:05:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 16:05:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186916.335662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAvc-0008SO-0w; Tue, 14 Sep 2021 16:05:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186916.335662; Tue, 14 Sep 2021 16:04:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAvb-0008SH-TY; Tue, 14 Sep 2021 16:04:59 +0000
Received: by outflank-mailman (input) for mailman id 186916;
 Tue, 14 Sep 2021 16:04:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQAva-0008SB-80
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 16:04:58 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81185a3e-1575-11ec-b454-12813bfff9fa;
 Tue, 14 Sep 2021 16:04:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81185a3e-1575-11ec-b454-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631635496;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=o89xmoAng9uJ125V3mShyKJrpaoF7fXbYluT9p9XaKg=;
  b=C9DBD16OJGfxQoie3fWu0wGvBhX2hsvMYcTjHmJFPk1R41E4KCGXSgtP
   jtAD+FWyriBHBdur6yS1QfgrL+9O/R4UVu7LDAhu+vGF8H/0o/NgrNAB0
   ZqLLGKN9PxelfME2i8o+idGdIFGCXRuirmIB2FTly/s955Xm3roh31VZU
   o=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: gryE06ybVrYslK1/jhVVBP+yMptAdKC/AZrIlPKAvSiKiF0U+s4JCd2zqzaoMaiQSkvn4GxVst
 VhLVXpF/reQoRV/7SQolxQdLxatxBLP9F6JvzzepSA8m4YKxl9O8IfNJu0yq6RWMfbXOEF1Sco
 7kZdtB0vOsFxd4LPHPdV+PABSHkcKpJT5zTbjApTtslyYGa1OCKjoqGDLzX1AvCgAjBq2/h2FF
 Y8hLhgfCdSAa6dnhYL31zdyLraEtVQ+sl9v/pg54endrEVa1UHqZr2RNisrfm+1OTWz6cWdkOP
 yRmwAH+m3iiEaOAHNT0UXAXB
X-SBRS: 5.1
X-MesageID: 52714329
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:nbuj9KDH7xGk8BVW/wPkw5YqxClBgxIJ4kV8jS/XYbTApDomhjZUy
 2IXC2/Vbv+CazOhftt+btvloR5Su5DdnYUxQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0E47wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/jSi4k+pV9
 MhxtJGWbAYJJaiWlbkyTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHv+VvIMDhmZYasZmFtbDf
 ZRJTgNVaBntUjxhY30zMpFipbL97pX4W2IB8w/EzUYt2EDDwQo03LXzPd79ft2RWd4Tjkuev
 njB/WnyHlcdLtP34TiP/2+oh+TPtTjmQ49UH7q9nsOGm3XKmDZVUkdPEwLm/7/p0SZSRu6zN
 WQS5xsil4wO6XCqQ//YXj2jpSSJ4zEDDo84//IB1O2d9kbFy1/HXTFZH2EeNoVOWNweHmNxh
 wTQ9z/9LXk26uTEFyjFnluBhW7qYUAowXk+iTjopOfvy+Lqpp06xjnLR8xqeEJepoyoQWysq
 9xmQS5XulnysSLp//7glbwkq2j1znQscuLTzl+JNo5CxlklDLNJn6TytTDmAQ9ode51tGVtW
 UTofeDEvYji6rnWz0SwrBglRun1t55pzhWF2QMH82YdG8SFpCf4INE4DMBWD0Z1KMcUEQIFk
 2eK4lg52XOnB1PzNfUfS9voU6wClPG8ffy4BqG8RocfOfBZKV7YlByCkGbNhggBZmB3yvphU
 XpaGO7xZUsn5VNPl2bvHLZBjuZwn0jTBwr7HPjG8vhu6pLHDFa9QrYZKlqeKOc/6aKPugLO9
 NhDccCNzn1ivCfWO0E7KKYfcgIHK2YVH5fzp5AFf+KPOFM+SmogF+XQ0fUqfIk8x/ZZkeLB/
 3ecXE5EyQWg2S2beFvSMn0zOqnyWZtfrG4gOXB+N1ifxHV+M52k670SdsVrcOB/pvBj1/N9U
 9IMZ96EXqZUUj3C9jlENcv9oYVueQ6FnwWLOyb5MjEzc4Q5H17C+8P+fxup/y4LV3Llucw7q
 rym9wXaXZtcGFgyUJeIMKqilgrjs2IcleR+W1rzDuNSIEi8opJ3LyHRj+MsJ51eIxv02TbHh
 R2dBg0VpLeRrtZtosXJn62Ns6ygD/B6QhhBB2De4Lu7aXva826kzdMSWeqEZ2mABmb9+aHkb
 uRJ1fDsdvYAmQ8S4YZ7Fr9qy4M45sfu+OAGnlg1Qi2TYgT5EK5kL1mHwdJL5/9EybJusAerX
 l6Cp4tBMrKTNcK5SFMcKWLJtAhYOS34TtUK0ckIHQ==
IronPort-HdrOrdr: A9a23:bBtLvaDDdh1wvSLlHeg2sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHPlOkPQs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ
 0QD5SWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHlXgslbnnlE422gYytLrWd9dP4E/M
 323Ls5m9PsQwVcUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZvzU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDj1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyEfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW92/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 ghMCjl3ocUTbqmVQGagoE2q+bcG0jbXy32DXTqg/blkwS/xxtCvg8lLM92pAZ3yHtycegC2w
 3+CNUbqFh5dL5gUUtMPpZzfSKJMB25ffvtChPbHb21LtBNB5ryw6SHlIndotvaPqA18A==
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52714329"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dAvpsIDa5pwN4+z/49IA4U3b+GF/C15ns2+NoJhblvMEX3/8ClXm+CQp4HKMkZbn5WI1DVf6TDJ8W22MGEHqfL81k4OD5jjPfUnKumjls5cUHHjHXyAPG3y+9nao5EqqoZeBJfHsSQKX1Ebk2LQq5ziwkfZng4N4M5fMsT+3kMVpqwVo7ZxH/B9rWDyJ3ljgEHq918I176CLkDukullVmgrRzx5PWr8hqBHsFP9pbgti/gZvxsQC9CQG5sw0o6PwtDmPqSTVtVZpCzEPTnbSFSmiUndqoinnMb+BF92eTfBogX5QJSfN1m/2QJwAJEzPc42G/LiCSYYOOjd7kVzhDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MQpun+hJTHP5TYfFSveNxQaob6FXDOnfGmGUw6bbQMg=;
 b=QqD+Zd76/goXnlmIq54qZvvGDjikjS3AlTJcKNNLoW7pTV1K2oeUCfhaVkG8P2it/GWY5B+rw/rr1DgXMiYQ0K43jthfHWGuFDC/yC6MMTo4AXDYs6fGfdLUq6BxUEdUmQ5Remg+NZuXZCgd86Kh3upyU3+h7w/7YKBj6roTc1Ea/yydDmxO2ts7nQJNtWRB0IPCatKSj0QrMqqLHUUUA24/YDsFH+vBfAkRh88KvsfRE2TQ3eGoLrzTnw20yENXX/sy+d0o4hr7R6G3DPNnMo8qj1p9Nb8+njEI5cwtGZQ81qvsXUkZdaX+zpDyEMW/Jw/WRaZoQ8SpF126diK1rA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MQpun+hJTHP5TYfFSveNxQaob6FXDOnfGmGUw6bbQMg=;
 b=dKW3a1uNZ/E7ti/tLdH1hE8TtXbyTbEQxbWDo7yo8T0MXQfBHbFd/VmuEyHcK8jUMxcWrbyeVhfy3+g2j+YQUahIwqGP8geOpsJBcTv0TgR4qwAHu/9HpV68ub0Kb/Sk97V9EgJSpqdaeYaPtKNkT9XCcQJmRK0T6xUdFEjz5KA=
Date: Tue, 14 Sep 2021 18:04:48 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()
Message-ID: <YUDIIA9Yk2CIYcWM@MacBook-Air-de-Roger.local>
References: <6285981f-683f-a043-7025-993613eaea7c@suse.com>
 <YUByvkKO+NlIjB2m@MacBook-Air-de-Roger.local>
 <45e8ef36-3f7a-5cd7-e640-61e1c6d63903@suse.com>
 <YUCFxJnDoaVNgHiu@MacBook-Air-de-Roger.local>
 <df432c93-b722-6006-c3dc-7f511b377bcc@suse.com>
 <YUCwD5RWj0WKZZUc@MacBook-Air-de-Roger.local>
 <d8b1b328-7887-cde4-dbe4-bdc43a1d5147@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d8b1b328-7887-cde4-dbe4-bdc43a1d5147@suse.com>
X-ClientProxiedBy: LO2P123CA0106.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:139::21) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d7fe926-7249-49d7-1791-08d9779963bb
X-MS-TrafficTypeDiagnostic: DM6PR03MB4682:
X-Microsoft-Antispam-PRVS: <DM6PR03MB4682163CBF3E8FA33BE097148FDA9@DM6PR03MB4682.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: GKdfoE/NtUakt09vGojh7L64RxJFOQ+x3TidQ7aDnCN6YQZnmehPLMAGNBTGco4GEPxnIAuiRsf7wE5MZkA0sM5nSvlsfOT+aeYsuv8hyneMi3Mmbu1wJl/6M4ZUwLhYvsZUHxrh/fD8+dK2RuKZQsUTvq5Jn1uHXgQEF1fSwDJ75bMZoj/96jv0Nh+ch4PhWPt8p6md18LMi52YyXErMxgDLVUJZuiJjRdGdSTtiN2rs4Iheb7wW36EmmgrKg6/DIuSvcf4Ae8HM2ITCCe8in5F9VgWSpZG2tg0qlXoKjCFxrViiEc3tYhK61pCdJcvbqEMEl5WnnLvS9DeduFy1qftPKxRroHMQFU0jE3H3+ZJ4SzBep/Mc0wquMt7lYYwneAQkSLXtWyXHrpp6kj4TJOFktEYnDA+oqLmu5wGZWaIXEEU1VdIQggItcXwoY7SPgne0LHAwLAjeNHXRc5PJjMGFfpjJDaXzJLhwRIQiQSb/XT1BlVKf64dXgDbRdLpku/UgqcWUBPfXZE+URSQyjXC5wzp2hbgL5eXEz7sA8osxhcl32J9K96Cdpoe9OP3a1a6Ax0nIq96hkJJMnelVtlhfXM4xtyK7QjGNDQouigUmiJGZGR+EJ2ftAHASo96aH8myTqCUZ1IgfxX2uNUiQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(376002)(366004)(346002)(39860400002)(9686003)(53546011)(956004)(478600001)(6496006)(15650500001)(66476007)(8676002)(38100700002)(86362001)(6666004)(26005)(186003)(83380400001)(66946007)(6486002)(8936002)(4326008)(316002)(2906002)(85182001)(6916009)(66556008)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UWpyMGd2SGhkQkdhMkdxR1pETFdZcGdrWHJHeElnRkRlRUtjUFAzTVV0Vmlq?=
 =?utf-8?B?SEIxcitlYzBGbXBHYkp5V3NJTEJGUFIvQURFNTFmRHJodElldXBUeEVpOVhT?=
 =?utf-8?B?Ym1hODdMdU44SlF6Um44OFRRMkNndlN6Tlk2aXo1NzgxdDBmMzV4bWU0YU5q?=
 =?utf-8?B?TURkb3lkQS80cmdaMnpOaUd5VEpxSTVFck55dHU0SStRUUxQaW9RYkxrTndh?=
 =?utf-8?B?TzNMUFpRMjBRb2xWRFFsSTBFQlhBNy9nSm4ycGNJYVRzekxiQjlkTDIrSFdY?=
 =?utf-8?B?eWlBdWd1dkROWGlWQkRzb3B2U2Y3UUhMa0FWc1FCRW1XL0l2TzN6Y2s1ZEJ6?=
 =?utf-8?B?Yy9lUzVhaTdRWmtpck5UWFV6S1Z4RERueDByTnc1MnRuWFM5VXNCaFJ4MFNl?=
 =?utf-8?B?cFdHTkZjeXJSRGx0Ui91Z1JJYTBXM1l5bURTdENMWitqZjBwUnhwSmRkcFpM?=
 =?utf-8?B?WnZwNG9Wb1RwRG9CN1JnZXF2RHlzUU1CcmtHTldqM2VzRDgvTG4ydGY2V2ZD?=
 =?utf-8?B?UXhUdGZDeW9jUHh2ZityYkloQUVVNXF3clVZRHhYVjhLYWNZZjNyWlhxaG9z?=
 =?utf-8?B?U0JZNVNKc2t0dDVrS0VyZ2tkWUoyR085UDlpZ1NmbHNtVU50T0VET1NGR1A4?=
 =?utf-8?B?K3lOaWU5MnJ3cnFqcHd0MkFGeWt0UG5qTEI1ZCtmdjNlYzc5aWR4aVJrb0oy?=
 =?utf-8?B?VXh2T1pCeWV1c2ZOeHovY0lQVWtnVi9GSG9Sd1dzWEMzazY3d28zK3cydFBx?=
 =?utf-8?B?S1BWS0EzRGVmcFhvbE44TGVLZXZYVWwvc0RhaVJsS3FpeGduNDZPb2Y1TEc2?=
 =?utf-8?B?eXdvcExLK2M2dGtURktUWkhLMG0vNjlwNnZwc0VQaEo0OER0K1VuTWpmSmRz?=
 =?utf-8?B?SXEyWml3V001ZDdhMGlEVkhvTytIakoreVVXOTkyeGRmV1pONTVpZm52aEEz?=
 =?utf-8?B?UTZQejhrY05Ed2NPMVBLL3o0ZEpCMEJwOURFd2IzbUw4Mit6clhDODlZNDRC?=
 =?utf-8?B?amorNTV4bEgwS013YnhOUEZ4b2FVajRESWk0SDZKMy96MkZnd3drc0RmbHVL?=
 =?utf-8?B?b3hpYXhMUE5yOTNzTUtNQTNnYjNTYUVpZXZmRTBNbGhvWkJqOU8zRDZwRUJJ?=
 =?utf-8?B?Um50SmVWNWVUcnlkNmNrcjZhak9PQm1POW00VEljU2dvK3hGL3NpL2YrdlNk?=
 =?utf-8?B?aFpNeDFKN0J3dnFab0d2UTl5MEJyeWtrc2dIQVVCL0F2NlNTRzl5VlI0SnJC?=
 =?utf-8?B?ZzBYYVEvSzdueFlQYWhEejFZamg2cjdaMWIraGZXRFhDMjhXVUhBVnFQNG9T?=
 =?utf-8?B?aGZYVFFybWpMbXo5M3htRFFIdWMyOGZpUkJDZjNacUZNZmx4cmhJbmtESm9N?=
 =?utf-8?B?RWN1UjB5NUNPRG54U3pnWmlkYjlhQklyTEZab2JxU0ROOXM3d2VKdUFKRFpX?=
 =?utf-8?B?dXlWdDBOS0h2L3VYanhQU3ZkSmxKWFhRMjJRc1NzWmFFbGNmT00wSTRoVmlY?=
 =?utf-8?B?N0NIclRudWFhcnVoSnNhRjI1ZHY0aEY4MklvR05ldHF3MnZHbW12UUthMUM1?=
 =?utf-8?B?b2RCblZQWnkxSFRHb3E4aVcxSi83ZDMyekZCMUpYMFJEMTgzMmFjK0tUT2tQ?=
 =?utf-8?B?Z3hZelNTbzAwdWZxQ3gyd3kvQUxoMXlOaHIzYnF4c1N5RjlidTdMYzVsNGdV?=
 =?utf-8?B?U1J1Z25VaURiRzE2U01aY2tudDQxdFFNaVNXQlhTRGhWV1paUURlbDNZUnA5?=
 =?utf-8?Q?o2+ukTumQFAypVAP1/rrGeOEe/4UxTKDyXppY+2?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d7fe926-7249-49d7-1791-08d9779963bb
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 16:04:53.8081
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lJnGsoDTdbvMyYvGaRbU4/Wt4+77pz6uBFUG1Vc/qzOUjfG0x+XaVxw70bW5+bOx+TKHcVAzQ1Hcw/Zct3Nqbw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4682
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 05:16:17PM +0200, Jan Beulich wrote:
> On 14.09.2021 16:22, Roger Pau Monné wrote:
> > On Tue, Sep 14, 2021 at 02:05:01PM +0200, Jan Beulich wrote:
> >> On 14.09.2021 13:21, Roger Pau Monné wrote:
> >>> On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
> >>>> On 14.09.2021 12:00, Roger Pau Monné wrote:
> >>>>> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
> >>>>>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
> >>>>>> console) are associated with DomXEN, not Dom0. This means that while
> >>>>>> looking for overlapping BARs such devices cannot be found on Dom0's
> >>>>>> list of devices; DomXEN's list also needs to be scanned.
> >>>>>
> >>>>> Thanks for looking into this, I certainly didn't take hidden devices
> >>>>> into account for vPCI dom0.
> >>>>>
> >>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>>>> ---
> >>>>>> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
> >>>>>>      would make the diff difficult to read. At this point I'd merely
> >>>>>>      like to gather input towards possible better approaches to solve
> >>>>>>      the issue (not the least because quite possibly there are further
> >>>>>>      places needing changing).
> >>>>>
> >>>>> I have a couple of questions, AFAICT we currently hide the serial
> >>>>> console and/or the VGA adapter if it's in use by Xen.
> >>>>>
> >>>>> I wonder whether we need to add vPCI handlers for those:
> >>>>> setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
> >>>>> device because of the temporary override of pdev->domain done in
> >>>>> _setup_hwdom_pci_devices, but I think that for hidden devices we
> >>>>> shouldn't add vPCI handlers. We should instead block PCI config space
> >>>>> access from dom0 to the device so that it doesn't mess with Xen usage.
> >>>>
> >>>> The answer to this follows (I think) from the one below.
> >>>>
> >>>>> It's also not clear why does Xen want to have those hidden devices
> >>>>> partly controlled by dom0, as it would seem to me that dom0 interfering
> >>>>> with hidden devices in use by Xen can only lead to trouble.
> >>>>
> >>>> Dom0 can't interfere as long as it can only read from the device.
> >>>> Restricting accesses to reads is one of the purposes of "hiding"
> >>>> (the other is to make it impossible to assign these to a DomU). Not
> >>>> allowing Dom0 to read from such devices would lead to wrong PCI
> >>>> device discovery - some devices would be missing (which may or may
> >>>> not be merely a cosmetic issue). If the missing device is a multi-
> >>>> function one at function 0, other devices in the same slot may also
> >>>> not be found by Dom0 (depending on whether it looks at non-zero
> >>>> function numbers in this case).
> >>>
> >>> Hm, indeed seems possible that missing function 0 the whole device is
> >>> skipped.
> >>>
> >>> Maybe we need a special vPCI handling for those devices that just
> >>> allows reads but not writes, and that doesn't maps the BARs into dom0
> >>> p2m?
> >>
> >> Not sure about mapping. They could be mapped r/o. And they may
> >> actually need mapping for multi-function devices, but I guess for
> >> such devices to actually function properly then there would be
> >> more work required.
> > 
> > I'm also slightly puzzled as to why ehci-dbgp uses pci_hide_device
> > while ns16550 uses pci_ro_device instead.
> > 
> > Is this because the PCI device used by ehci-dbgp must be shared with
> > dom0 for other USB ports to be usable, and hence dom0 will need read
> > and write access to the device PCI config space and BARs?
> > 
> > Note that pci_hide_device is less restrictive than pci_ro_device, as
> > it doesn't mark the device as RO.
> > 
> > That would seem quite risky, as it's likely that dom0 will perform
> > some kind of reset of the USB device and thus the console will be
> > lost.
> 
> There's actually a protocol to prevent exactly that: See PHYSDEVOP_dbgp_op.

OK, so dom0 should be allowed write access to hidden devices config
space and BARs, while RO devices don't. That means that for hidden devices we
need to intercept PCI config space accesses and use vPCI for PVH.

For RO devices we already allow read-only access to it's config space
when using vPCI, albeit it might be nice to also add BARs as RO to the
guest physmap if they are enabled.

Will look into sorting this out.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 16:06:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 16:06:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186923.335673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAwa-0000ar-Aj; Tue, 14 Sep 2021 16:06:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186923.335673; Tue, 14 Sep 2021 16:06:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQAwa-0000ak-7d; Tue, 14 Sep 2021 16:06:00 +0000
Received: by outflank-mailman (input) for mailman id 186923;
 Tue, 14 Sep 2021 16:05:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r2So=OE=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQAwY-0000aY-IB
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 16:05:58 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a54f30f8-1575-11ec-b455-12813bfff9fa;
 Tue, 14 Sep 2021 16:05:57 +0000 (UTC)
Received: from AS9PR06CA0179.eurprd06.prod.outlook.com (2603:10a6:20b:45c::9)
 by AM4PR0802MB2177.eurprd08.prod.outlook.com (2603:10a6:200:5d::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 14 Sep
 2021 16:05:47 +0000
Received: from VE1EUR03FT034.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::ce) by AS9PR06CA0179.outlook.office365.com
 (2603:10a6:20b:45c::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Tue, 14 Sep 2021 16:05:47 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT034.mail.protection.outlook.com (10.152.18.85) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 16:05:46 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Tue, 14 Sep 2021 16:05:44 +0000
Received: from a25c6d213c1b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 35C77C70-4A32-4C71-99F8-3FC9AC704217.1; 
 Tue, 14 Sep 2021 16:05:32 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a25c6d213c1b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 14 Sep 2021 16:05:32 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1809.eurprd08.prod.outlook.com (2603:10a6:203:3a::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep
 2021 16:05:28 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Tue, 14 Sep 2021
 16:05:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a54f30f8-1575-11ec-b455-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4RDD4gxU08S1uSaApCZk1px4OvUrzzI9gUt7p+1J4WA=;
 b=PhTpTdpX/kpRLCNh7jFqdyvj+bdlzxLW5/9ced5j2UNsfbBkHNHkm8JWoxpxi6w7MZcKB2sIX5qeHjFDcIx1X+sJnr9PJcqqYG36Rb087IK8gg2rislFXmthwN69GDv0KoHGbWYmzFvx0eQeAzQNB4hHcXcm+gJy9ka7neq9IQE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5308ce0caadccc43
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UOqH3mtdphovGdjuFvLw+wjIWRe7r6pepUAcaHJohJJQfqpEY1zFXnfTPLvsCBfvU1S4E4eUffFEVdsVT3KSN5o8JIxL4VuWjHt1zsxIcM6a8+dte0dAFdM9sf4O6TbKRUy1GfAx17OlwJ9BxVjnLvbT9ZhkMxbP/P3JCeZBE8FXHXpL0KTBahURLQWI4DtQQ7Af34CYG0sli3khDjeSmVHNzfFxNiTy+rxigtsE1VFrdZ2CVIyPxiBKLg9OArpRCE5UX1/dZ+BRrc4h0i6M4FnNxxyn3BbAzH9JbM6oHw9y8DsnuSqvl625IECYu1ON7mvrY7k43u8uz4SxtPaPGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4RDD4gxU08S1uSaApCZk1px4OvUrzzI9gUt7p+1J4WA=;
 b=AokXm3PhhP2M62dx4eUria7rzD8iP0e2DVDqb9oPf+YY6xCnhAoLBvEILjy6ojiUm2WgGj3QvPQlzqghEKriHoSMbKvem0XFrHmROuCWOaNRjUsq9Vg/hgO0A/SEjO8rUSbQFhk0Vh9KXeBJprNuFd6Yk7WgtH0b2sDWlnhtzp3nN5LcbmHwwNqoMYuaRSxqhqjQpqWC6LpwZ0DUT0Y8FH6mjXdrvFazSYXXSk4a52xetWtOia9LiY/xEQMylWj57jGXRE1iP0dKC8DRFlecf45v8hTTEEjiUarh7QYAtG1LxL2XX2kcSBlhf1W+prgVr1qP+AuuIWp20dr5jUsxrg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4RDD4gxU08S1uSaApCZk1px4OvUrzzI9gUt7p+1J4WA=;
 b=PhTpTdpX/kpRLCNh7jFqdyvj+bdlzxLW5/9ced5j2UNsfbBkHNHkm8JWoxpxi6w7MZcKB2sIX5qeHjFDcIx1X+sJnr9PJcqqYG36Rb087IK8gg2rislFXmthwN69GDv0KoHGbWYmzFvx0eQeAzQNB4hHcXcm+gJy9ka7neq9IQE=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Index: AQHXlPLKFxZ9ODgqA0ysGgNkOt1JX6ucfp8AgAdcI4A=
Date: Tue, 14 Sep 2021 16:05:28 +0000
Message-ID: <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 7fe8395b-1cf2-428a-eb53-08d97799839a
x-ms-traffictypediagnostic: AM5PR0801MB1809:|AM4PR0802MB2177:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM4PR0802MB217724208F6F26CA00C1A7A4FCDA9@AM4PR0802MB2177.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 muLPpNuf3BY600gaGrs5pwoVRihM3LtTZfr3ScsOIIKVJ02FmPh2ww03OEysUiWdk32PGqUpTz4O6wBjyoAgR55AlxRbatG2twlhQ6nWXPeA1odzsYgfvhYrhWsV2vQJtyppV4FkEJGHntpgnvw6ckXSsMnl+7ED6kT3utRGpDzXQpo+PMbFIhgmGSZ42ardQE1kiIMtVetPf0ybN10GFPkaWobfdslabQkTkCm6jCRCP0ii3Dvn1kcq+2WwxNu1Hn9DZ3biH8IRx+pQDYFsM5hLf+xp8z1yOx61s0/sgMQxgwa2v+F4ZynHxdNwipjwiv43ZNVMeWl46qugvfPkC87LmEuymA3zV+JwZykRejDSGRJPkyF0Xronmb8NKrwpmqAj15G40pRS5iX33p1nR8AFTw287ngReqEArfsLZV0EElc+7jU3A4/OYijZXgxJsgdyErIU6+WH1FEv6antn3O1qYmNLZMUmY89+bXPZNoJ4udHccMNsbUlSDkuVjQHxfXGV/qh3nUddGS0pcBQTzpwpOIdlNb0xsBiblljr+q7eBd0SNe0pSDcheNjAS/nnGNRWZLox6tTbt83xQVOg+tsZn16EMVeLCIzFj167Fbm6ICNH7M4Ctd9Od+VZhERPbSnvYj+cO1rnUhjljoNNfcRxJuL7/A7fyyhA9W0oQ+bB5sh1R8fMi7ED7smlyEbVLycG+qCaWZIn+W+MwwBE8BClFwJqp4t9jJqjN955ktTQhpxCtREkqPCXnhdRi7c
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(6512007)(6916009)(38070700005)(186003)(2906002)(5660300002)(38100700002)(8936002)(76116006)(66946007)(6486002)(66556008)(316002)(86362001)(66446008)(508600001)(2616005)(122000001)(8676002)(66476007)(83380400001)(64756008)(6506007)(53546011)(91956017)(36756003)(26005)(4326008)(71200400001)(33656002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <53EA1C7039DE7B4FB3281D651EE92634@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1809
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0841bfda-d605-4a2c-decf-08d977997889
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rEiso4WJVNF/bXgCvUrhnqAzW4nRjJKFEBPJsbiXAAX9Yp1mvGls5aK5Qf6HnvdODXsqAulM5ntesyrkL4robNkcTMp3BbzNCiC8bj7R6BG8/794UNc59C1q60H/bGhFMsiy/iHf2+en2PtBcc6NV7xARKLsum4vdKqFUAb7LBOwe/Lgo6AopSLEaNvdYkkx4o6OU6L1ASFaA4zMuCs76IS9HeWsEwjXVq47pKA5Jg2z3bD8KkuH0/No0CxWDncFEl+/F/QO0WQ0FkqUm3UIxtXvHX0OyM+MW37oBHGmLknNVHU1s3eb9Ep1OUka2sEiexRAcsQPIwqHIE8qAGNw9dGFCjyNOU3vJPdfaDTe2sVtNWNBxFOnUMgkgGYk0BGxm9ykXqcwhbMR3j6JzaDSSI5Gd7XNB7ZjWxDHSbuFl+OryvcnilHFYKPwxko0pQG/HBqiCxG+LB1R8BCt5KqffS76IO3Ltcq/QD8Z8zwIcKFOKg6SYLh+iDZkxKhTXabDCs3snoVXKndQIUDWpsrrHqnr1Xh2pxHgdvYfoBI0ZxKYwVlYEJ/v2U0tyFrj07tHC8L0pT+ou8SkB6ne/qjW9bQv8Bbvq9kzJAHdV21F8LIOWg9YOQFnnOaQgoN8Ofb4XUxp5F/VHY4nhsUXeczfYFrn3Qa7IF1GrSQltzsvpyBc2iEpHo6FtNgMMIdunP41U0qZ8dKhGtSnTBhzSv1E+Q==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(6486002)(36756003)(6862004)(53546011)(33656002)(47076005)(81166007)(2616005)(508600001)(2906002)(336012)(8936002)(70586007)(70206006)(82310400003)(36860700001)(6512007)(86362001)(107886003)(83380400001)(26005)(356005)(8676002)(6506007)(316002)(54906003)(5660300002)(186003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 16:05:46.9866
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7fe8395b-1cf2-428a-eb53-08d97799839a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT034.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2177

SGkgU3RlZmFubywNCg0KPiBPbiAxMCBTZXAgMjAyMSwgYXQgMTI6NDEgYW0sIFN0ZWZhbm8gU3Rh
YmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4gd3JvdGU6DQo+IA0KPiBPbiBUaHUsIDE5
IEF1ZyAyMDIxLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+IEltcGxlbWVudCBnZW5lcmljIHBjaSBh
Y2Nlc3MgZnVuY3Rpb25zIHRvIHJlYWQvd3JpdGUgdGhlIGNvbmZpZ3VyYXRpb24NCj4+IHNwYWNl
Lg0KPj4gDQo+PiBTaWduZWQtb2ZmLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2luZ2hAYXJtLmNv
bT4NCj4+IC0tLQ0KPj4geGVuL2FyY2gvYXJtL3BjaS9wY2ktYWNjZXNzLmMgICAgICB8IDMxICsr
KysrKysrKysrKysrKysrKysrKysrKysrKysrLQ0KPj4geGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9z
dC1jb21tb24uYyB8IDE5ICsrKysrKysrKysrKysrKysrKw0KPj4geGVuL2luY2x1ZGUvYXNtLWFy
bS9wY2kuaCAgICAgICAgICB8ICAyICsrDQo+PiAzIGZpbGVzIGNoYW5nZWQsIDUxIGluc2VydGlv
bnMoKyksIDEgZGVsZXRpb24oLSkNCj4+IA0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9w
Y2kvcGNpLWFjY2Vzcy5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktYWNjZXNzLmMNCj4+IGluZGV4
IGYzOWY2YTNhMzguLmI5NGRlM2MzYWMgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vcGNp
L3BjaS1hY2Nlc3MuYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktYWNjZXNzLmMNCj4+
IEBAIC03MiwxMiArNzIsNDEgQEAgaW50IHBjaV9nZW5lcmljX2NvbmZpZ193cml0ZShzdHJ1Y3Qg
cGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYsDQo+PiBzdGF0aWMgdWludDMy
X3QgcGNpX2NvbmZpZ19yZWFkKHBjaV9zYmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywNCj4+
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGxlbikNCj4+IHsN
Cj4+IC0gICAgcmV0dXJuIH4wVTsNCj4+ICsgICAgdWludDMyX3QgdmFsID0gR0VOTUFTSygwLCBs
ZW4gKiA4KTsNCj4gDQo+IFRoaXMgc2VlbXMgdG8gYmUgYW5vdGhlciBkZWZhdWx0IGVycm9yIHZh
bHVlIHRoYXQgaXQgd291bGQgYmUgYmV0dGVyIHRvDQo+IGRlZmluZSB3aXRoIGl0cyBvd24gbWFj
cm8NCg0KVGhpcyBkZWZhdWx0IGVycm9yIGlzIHVzZWQgb25jZSBkbyB5b3Ugd2FudCB0byBtZSBk
ZWZpbmUgYXMgbWFjcm8uICANCg0KPiANCj4gDQo+PiArICAgIHN0cnVjdCBwY2lfaG9zdF9icmlk
Z2UgKmJyaWRnZSA9IHBjaV9maW5kX2hvc3RfYnJpZGdlKHNiZGYuc2VnLCBzYmRmLmJ1cyk7DQo+
PiArDQo+PiArICAgIGlmICggdW5saWtlbHkoIWJyaWRnZSkgKQ0KPj4gKyAgICB7DQo+PiArICAg
ICAgICBwcmludGsoWEVOTE9HX0VSUiAiVW5hYmxlIHRvIGZpbmQgYnJpZGdlIGZvciAiUFJJX3Bj
aSJcbiIsDQo+PiArICAgICAgICAgICAgICAgIHNiZGYuc2VnLCBzYmRmLmJ1cywgc2JkZi5kZXYs
IHNiZGYuZm4pOw0KPiANCj4gWW91IGFyZSBub3QgYWN0dWFsbHkgcHJpbnRpbmcgc2JkZi5zZWcs
IHNiZGYuYnVzLCBzYmRmLmRldiwgc2JkZi5mbiA/DQoNClllcyBJIGFtIHByaW50aW5nIHdpdGgg
4oCcUFJJX3BjaSIuDQo+IA0KPiANCj4+ICsgICAgICAgIHJldHVybiB2YWw7DQo+PiArICAgIH0N
Cj4+ICsNCj4+ICsgICAgaWYgKCB1bmxpa2VseSghYnJpZGdlLT5vcHMtPnJlYWQpICkNCj4+ICsg
ICAgICAgIHJldHVybiB2YWw7DQo+PiArDQo+PiArICAgIGJyaWRnZS0+b3BzLT5yZWFkKGJyaWRn
ZSwgKHVpbnQzMl90KSBzYmRmLnNiZGYsIHJlZywgbGVuLCAmdmFsKTsNCj4gDQo+IFdvdWxkIGl0
IG1ha2Ugc2Vuc2UgdG8gbWFrZSB0aGUgaW50ZXJmYWNlIHRha2UgYSBwY2lfc2JkZl90IGRpcmVj
dGx5DQo+IGluc3RlYWQgb2YgY2FzdGluZyB0byB1aW50MzJfdCBhbmQgYmFjaz8NCg0KcGNpX3Ni
ZGZfdCBpcyBkZWZpbmVkIGluICJ4ZW4vcGNpLmjigJ0gYW5kICJ4ZW4vcGNpLmjigJ0gaW5jbHVk
ZXMgImFzbS1hcm0vcGNpLmjigJ0uIA0KSWYgSSBtb2RpZnkgdGhlIGZ1bmN0aW9uIGFyZ3VtZW50
IGluICJhc20tYXJtL3BjaS5o4oCdIHRvIHVzZSBwY2lfc2JkZl90ICBJIHdpbGwgZ2V0IGNvbXBp
bGF0aW9uIGVycm9yLg0KDQo+IA0KPiANCj4+ICsgICAgcmV0dXJuIHZhbDsNCj4+IH0NCj4+IA0K
Pj4gc3RhdGljIHZvaWQgcGNpX2NvbmZpZ193cml0ZShwY2lfc2JkZl90IHNiZGYsIHVuc2lnbmVk
IGludCByZWcsDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBs
ZW4sIHVpbnQzMl90IHZhbCkNCj4+IHsNCj4+ICsgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAq
YnJpZGdlID0gcGNpX2ZpbmRfaG9zdF9icmlkZ2Uoc2JkZi5zZWcsIHNiZGYuYnVzKTsNCj4+ICsN
Cj4+ICsgICAgaWYgKCB1bmxpa2VseSghYnJpZGdlKSApDQo+PiArICAgIHsNCj4+ICsgICAgICAg
IHByaW50ayhYRU5MT0dfRVJSICJVbmFibGUgdG8gZmluZCBicmlkZ2UgZm9yICJQUklfcGNpIlxu
IiwNCj4+ICsgICAgICAgICAgICAgICAgc2JkZi5zZWcsIHNiZGYuYnVzLCBzYmRmLmRldiwgc2Jk
Zi5mbik7DQo+IA0KPiBzYW1lIGhlcmUNClllcyBJIGFtIHByaW50aW5nIHdpdGgg4oCcUFJJX3Bj
aSIuDQo+ICANCg0KUmVnYXJkcw0KUmFodWwNCg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 16:28:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 16:28:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186948.335683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQBI1-0003Vd-6I; Tue, 14 Sep 2021 16:28:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186948.335683; Tue, 14 Sep 2021 16:28:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQBI1-0003VW-3J; Tue, 14 Sep 2021 16:28:09 +0000
Received: by outflank-mailman (input) for mailman id 186948;
 Tue, 14 Sep 2021 16:28:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K1wc=OE=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQBHz-0003VQ-A4
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 16:28:07 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78fbfa76-8fe8-42d6-9ae1-394887b083be;
 Tue, 14 Sep 2021 16:28:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78fbfa76-8fe8-42d6-9ae1-394887b083be
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631636885;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=0oDu8A4E1nBudlkblNkn/OQ6raVkygEBFAyYDouYCeE=;
  b=dJD9VqYZJsz+DEu31N5FOZQPscz1Zz+Dr76wqPE9JXIGevLhtzW+NlPE
   /IFtcWw5xoPxIWr8ljdTXa3U9Mpln6ZBJVH7QomhjxoK0YvEojhEFFZ4N
   bMeGq4hzP/m5cIfws/i2a9dHyVVeZF/qCNcIm62HjQcdbDyqro5wAT6UW
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: sTpUpzu+P8oaQRNY5oXHzA/ElGr3kPGrlXi84A4uofqfk6PPHYpYRY7ciF7GoNVfqz+Z1L6hmw
 VA1YjAxELHjmcmVbt6UXOHEGm+2aA6uMndc/v9lBuB66SS65ASavL6xJzznfdLiDJtjx7tJaTn
 WO0qYQaMVSv+oRLdL8yIxXTySoYjLDxGJ1T/9xFX2zWNG/515XaacX6oylv+o763LOUhi9KcMC
 EzfYoGoBKRUaWBGkNyW/kl0I0BR+l3wd2WUVEhfMLCOPd6InzZfzTf5ejiNUtktPik7scXSyrO
 FXsLZebhKJQw2kJigpir6FsL
X-SBRS: 5.1
X-MesageID: 52717053
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:xFRCm6JbAf6ysWIsFE+RRpIlxSXFcZb7ZxGr2PjKsXjdYENS0TYAy
 WYWUGiGOv3ZYDf3L9pzbNvn90oH75LTnNBiTARlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6x7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2kwvdPl
 dtg6qbpF18uBrfQxroEWkRHRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2Xv4ABg29t3aiiG970Q
 vsgMjxGYCjcICRSYFUuFpsYpcKn0yyXnzpw9wvO+PtfD3Lo5BdqzLHnPd7Re9qLbcZYhECVo
 iTB5WuRKhUbMsGPjDmI6HShgsfRki7hHoEfDru18rhtmlL77nweDlgaWEW2pdG9i1WiQJRPJ
 koM4C0soKMuskuxQbHVXRe1vXqFtR40QMdLHqsx7wTl4qjb+QGCHUAfUyVMLtchsaceTzsw1
 0TPmMj1Hztxq76EYXWH/7yQoHW5Pi19EIMZTXZaF01fuYCl+dxtyEKUJjp+LEKrpvCoPg+tz
 R3VlnFg37Irl9JVibyyoE+S1lpAuaP1oh4JChT/Bzz+t14oO9H7PuRE+nCAsq0Rd9/xok2p+
 SFewpnAtrhm4YSlyXTVKNjhCo1F8Bps3Nf0uldpA5Bp3DCk4XfLkWt4sWwmeRsB3irpf1bUj
 K7vVeF5v8Q70JiCN/Yfj2eN5yICl/aI+TPNDKy8Uza2SsItHDJrBQk3DaJq44wIrKTKuftuU
 Xt8WZ33ZUv29Iw9lGbmLwvj+eZzm0jSOl8/tbiklk/6gNJylVa+SKsfMUvmUwzKxPrf+23oH
 yJkH5LSkX13CbSmCgGOqNJ7BQ1afBATWMGtw+QKJ7HrH+aTMDx4YxMn6eh6ININcmU8vrqgw
 0xRrWcDlAOk1CGYcF7TAp2hAZu2NatCQbsAFXVEFX6j2mQ5YJbp66EadpAteqIg+vAlxvlxJ
 8Tpse3cahiWYjiYqTkbc7fnq4luKEaiiQ6UZnL3azkjZZ9wAQfO/4a8LAfo8SAPCAuxtNc//
 OL8hl+KH8JbSlQwFtvSZdKu00i14SoXltVtUhaaOdJUYkjtrtRncnSjkv8tLsgQAhzf3T/Gh
 R2OCBIVqLCV8Y84+dXEn46eqIKtH7csF0ZWBTCDv723KTPb7iyoxooZCLSEejXUVWXV/qS+Z
 LoKk6GgYaNfxFsT6thyCbdmy6469uDDnb4Cw1Q2BmjPYnSqFqhkfiuM0/5Qu/Af3bReowa3B
 B6Co4EIJbWTNcr5O1cNPw55PP+b3PQZlzSOv/Q4JEL2uH1+8LadCBgAOhCNjGpWLadvMZNjy
 uAk4ZZE5wu6gxssE9CHkiELqDjcci1eC/0q5sMAHYvmqgs30VUTM5XTBxj/7IyLd9gRYFIhJ
 SWZhfaairlRrqYYn6HfyZQZMTJhuKkz
IronPort-HdrOrdr: A9a23:Mb7tLKkYhYJRdnw8k13bjLfa11rpDfIi3DAbv31ZSRFFG/Fw9v
 rDoB1/73TJYVkqN03I9ervBEDjexPhHO9OgLX5VI3KNGOKhILCFvAA0WKN+UyEJwTOssJbyK
 d8Y+xfJbTLfDxHZB/BkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52717053"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LR+UjHDak1cqYTDLICD7KCXbaMD+tUGN2bO0ZU0HScW5bizOffYJtDwIvAKscoBrBPL/XNZrvOR8fsWhNAhGgSnCjjQISTWagZKR0aIi2Bx2q2QUA5VO7KnF4xAdhiFvG5TvYXjB9ajnpTknvZ3MmEbma0Dvi9esf8MC723H0tFW47wtdJGm58pvC/0a8hd5xRP6Qb1J1C74blZ0LJ2+EFDepOvUBYGQjaB3lqpTup28L1E4xtZCeAJnJ0HUP8qR/3sbpTayeAy/ea22e3SJ10LvTRCruCIWgwvqdiN4LDsVbJNsYmGEEQlbB+3ko6AHc8kEuh8jf5GBJqxV0llCGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/FUPmgE0iKT7YxUURvlZlq/MtNcRKfBk8QhYO56li0I=;
 b=Oih5P4AE+atSEkVprqyWU7MDlGbx8Wp7CZG4u3HibtR78LpTjcfafewYKJck4vq4QaGfsmtPauzknUCXYAPlU7Jq9NnoWMdC3BYdQ9LJ6+N2mxuTSrWG1dlr61J6kk5num02a5qt07vKWPV10HH2WPmPpdIciBXP7M/AQF6Hbsrw53TIsogTm0qQWdIVgLCrdxi1jMV2LUHunKfftSgmCmMtuuczQM9+1yMgYLTtaTbQRrb9eTYAi+DRJqShwzN6L+L0yistVvMsXDGMm+RXKTIoGSROzlKBXWnwlFQnv7zSFg+NFYRYQ44n+pJy2bSzUgBh0VpbwJKRsUZGqiOrVA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/FUPmgE0iKT7YxUURvlZlq/MtNcRKfBk8QhYO56li0I=;
 b=lVmHx1AUwybUpn2L10TQ4TTOMA5p9+kgoCaeGvo36TprCb0DfyAgRy0OWQonxQuBklr7ZKp2j0ZlgO5b8zVZ/X+nNp56kRTTjjfc2JKiKn5X9Srw8Fa+OZPKrlFiTEDal4D+UQPGvGKS7lsCG8wdC8l30hY8Cd5HOIaR8U4ITQY=
Date: Tue, 14 Sep 2021 18:27:51 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Juergen Gross <jgross@suse.com>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
Message-ID: <YUDNh9r8nYOxB76n@MacBook-Air-de-Roger.local>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
 <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
 <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
 <YUCYYmqQMR5NCTyz@MacBook-Air-de-Roger.local>
 <13dd8509-330b-4571-086a-7a3ed9c22c82@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <13dd8509-330b-4571-086a-7a3ed9c22c82@suse.com>
X-ClientProxiedBy: LO2P265CA0316.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a4::16) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5f5cc8a3-6cff-4a90-9cf4-08d9779c9be0
X-MS-TrafficTypeDiagnostic: DM5PR03MB2844:
X-Microsoft-Antispam-PRVS: <DM5PR03MB2844FD8C963E972EBA22AABA8FDA9@DM5PR03MB2844.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: v76C3P8DenRctbHtidp9JEWnr81wtHuRBu/paS4bPExq75RpA3AwoLBu5yXgDOrPgMsVApa71R0IVofTGmjBgcOFxOs4hQF51G5oYry9wGWvADLfx0iC4SS/FCkRq89w5EsEIKt3ryYRZREC/tIdbQi4m5dQmF2PlTS2MEy4TJAoq4sHYxDifGisy2KN1Clvi2XODAsxJXNJ92p/9IqCq/soPEVSWDzcVVWsGAJDd6PTF9YneLCMUwOKUEv74ZoMZiOAh+QdmeYfjnvVYXgeyOYRPBDskDQnto0JhAgz+SVM07MTST4BPNrLRxyHIeoZXUAZ6lwOgK/3/GLENZ+vd4LAWQAxvUMav0KdArj80CS7kNi+XzTRLzCtSm/96M7xFEzkgBcEXwpvnLQWcSNFsHMVfQAb+SZKfoJ753AOTp4YyGOzG6kQ8QH1DHqZbV2zfqZwQEzPjrXDswmhWAf3OfXSsQomLQMRcJSPo7MRPQ6NAtzWU71dIFG6/9Kyo6ZJFSeNZqNmDuXIPtSRyIbvy053+HoGEu9R5Ww9lMTdIMjIZs0jyZ4YqcojqmEbgZEVVNWUti73U3V5Pz9BT+lhhjI8MeM+W6O1+N4tHDfNhDs0IPdApSUbNHa4sYjBZ1lLAxdCqIgxtC8fIyW8W0M7/g==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(85182001)(478600001)(83380400001)(5660300002)(6486002)(8676002)(4326008)(53546011)(54906003)(66946007)(956004)(2906002)(186003)(26005)(38100700002)(66556008)(66476007)(9686003)(86362001)(8936002)(6916009)(316002)(6496006)(6666004);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aFBOWlVLcXBIa3d0MWVaSzc3Z0xTenh6T3NkUlNTajlWclNRZjRRdk84ckhM?=
 =?utf-8?B?c01WaUdrb0oxdGk1TUpaM2Jka21zUW9mTlQxWGdlZy9QTmhKWlI0RE9vTThK?=
 =?utf-8?B?bTFpQjFuTjJRL0dORndOdmtFV0txRGZkMnRicTR6Qk5yYTNsR2lMZlQ4SFZv?=
 =?utf-8?B?ZEEwbHBEa0xTSHdTSkFvUEttN2NQUFVqQWZmWGlhMnNGbjZsNSs3d3ArZlpt?=
 =?utf-8?B?V09QTW11VG5XNUZBTGE0M29OaTd1UWVSaFFzT2VtYmozQnNLY0srVFpSMFpU?=
 =?utf-8?B?aG9TQklKRDA3RjFiZ1ZKcG5XeGxBNmhZejkwUUo0SW5URzRoZ013OGIzMWI2?=
 =?utf-8?B?dnROZDJxN2V2T0s5d0U3S0pJb1pVYzcwSzYveTRUejJ5bHhQcWNOTnNLUGRP?=
 =?utf-8?B?R1NJU1VnSm9TMmxMekpDWWhZaVR3NmVBTmF3cUkrOVY0ZFN1eUpOYjdTa3Bp?=
 =?utf-8?B?ZGxZT3Z1YlZqUXdJWi9IZUIzSUJJZG1tcUl2YTA3Vm5Ub1hybWJMeDhsMmdJ?=
 =?utf-8?B?U1N0K1NSWlNYVm15Z0ViUWxYVDc0RGFGQmgxUndETC95U1Z5Zy9CRTBuTmU5?=
 =?utf-8?B?SnQraklkQTVtQVIvaHJlUXh4T2EwOUJRUDFIREttNnp1Y2drVGdIU3VsZjNa?=
 =?utf-8?B?RGFvM2NPT05YZ3VHSmloUHgzQkpXZUw4a1JwL0w5K1llMENOUkt2ZDJkcCtP?=
 =?utf-8?B?emFweFRadEV0Y3d2UU10Q093OG9YdG1pTDVVYThNTG1QRFVXOXVYVnkzZnNB?=
 =?utf-8?B?bWtlQ2RIem9TdnpyM0Z5NzN5OGlZeS9EYThXeVZ1dm10TVJEYmcxSzJzSld4?=
 =?utf-8?B?bi9KWkhQejRUNHk1dTlFVGVnUWplY3dVaXBmaUh6YkhRNUhBRGVZUmxRU0VF?=
 =?utf-8?B?QzJ1STBFUnJJbUUzUGZycXhDTzN3dzVGdEJqd1duZ1p2bkNTK3RHUENNU2xt?=
 =?utf-8?B?aVZTTWszSEpERkZad0tNaGJtaXhxOVQ1MmU5RWJyMCtxdVplZUFmQXIyVmtG?=
 =?utf-8?B?NDNnWnhJVG1sQUxWZGNvYUJ1aE5YV0VqbzFXR0NITEZVWnBkOGtBVk52UU5z?=
 =?utf-8?B?ekFVMkx6U3p5ekV2YWtHZ285T1pGZDdIdXBnYmpINnNTN2VnRzU4Mks1RnRt?=
 =?utf-8?B?dEU3SjUvc3ltYzZqeE9YTDJYZVN3azZVTTErcDk0RHhNUmpXaVJCdFA5VmJa?=
 =?utf-8?B?UXdrdnYwNTRqR2h3a2w2RCtSb3BNZ0M2U1Mrc0g1TjJVcUwydmIvMlhuaWlU?=
 =?utf-8?B?VWx6bGpURVN4bGNhRmVsN2R6c3ZFUFNrNXJ3WDNBdDdaMkxZNHhKMm5mMHhi?=
 =?utf-8?B?SGRkUVcvMlNNckNpYjdaejFMNDV3OEdFYmVpRUcvajRUVlFrMDN1bHlnczNa?=
 =?utf-8?B?WlJuVXcwTWN4NjdKS0RtdE5GZlpwRlZkODdMbjFXQ0ptR1kyY3pNbFd4OTJX?=
 =?utf-8?B?MmhSLzFDL3NCazdYaWgwZjhKT2RNeE0rbTNXU05OcFVlbkg2NVdrcjFReWxu?=
 =?utf-8?B?cFpLWDIreVFUMXlNdnEvOVdDYk80QlFoS1VmS01UNUJqbjlEbWxmb0hDVEVE?=
 =?utf-8?B?R1BPcjNJN1h4dGc2WHZ0bzFCRzZ6MW45Y1pYcFF0bDl2Y0pNQjlBeHpKWGJS?=
 =?utf-8?B?WS83YjVLQ3cvTGxiWkQ5dnVHK2M5b0ZZR0cxNG1tblZ0dFlCNVpWaEN1c29j?=
 =?utf-8?B?U1dMM2UraGwyaXVwalNWL3JXOFcrc24xUE9GVHhUY3Y0blo3VFJqVWtrTXNh?=
 =?utf-8?Q?Ad8XJ8sL/l34ys4acnUZvt5TKgzSLSIO3zEzvTq?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5f5cc8a3-6cff-4a90-9cf4-08d9779c9be0
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 16:27:56.5260
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KDf2PF6qnty9LloXnEYG65aUYSZRufH+U9ObhYAJZx5kfHRm8cAIcdAALmMJ7wm34fCi3MJJ6zjr6KMb79Xj8g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2844
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 05:13:52PM +0200, Jan Beulich wrote:
> On 14.09.2021 14:41, Roger Pau Monné wrote:
> > On Tue, Sep 14, 2021 at 01:58:29PM +0200, Jan Beulich wrote:
> >> On 14.09.2021 13:15, Roger Pau Monné wrote:
> >>> On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
> >>>> On 14.09.2021 10:32, Roger Pau Monné wrote:
> >>>>> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
> >>>>>> 2) Dom0, unlike in the PV case, cannot access the screen (to use as a
> >>>>>>    console) when in a non-default mode (i.e. not 80x25 text), as the
> >>>>>>    necessary information (in particular about VESA-bases LFB modes) is
> >>>>>>    not communicated. On the hypervisor side this looks like deliberate
> >>>>>>    behavior, but it is unclear to me what the intentions were towards
> >>>>>>    an alternative model. (X may be able to access the screen depending
> >>>>>>    on whether it has a suitable driver besides the presently unusable
> >>>>>>    /dev/fb<N> based one.)
> >>>>>
> >>>>> I had to admit most of my boxes are headless servers, albeit I have
> >>>>> one NUC I can use to test gfx stuff, so I don't really use gfx output
> >>>>> with Xen.
> >>>>>
> >>>>> As I understand such information is fetched from the BIOS and passed
> >>>>> into Xen, which should then hand it over to the dom0 kernel?
> >>>>
> >>>> That's how PV Dom0 learns of the information, yes. See
> >>>> fill_console_start_info(). (I'm in the process of eliminating the
> >>>> need for some of the "fetch from BIOS" in Xen right now, but that's
> >>>> not going to get us as far as being able to delete that code, no
> >>>> matter how much in particular Andrew would like that to happen.)
> >>>>
> >>>>> I guess the only way for Linux dom0 kernel to fetch that information
> >>>>> would be to emulate the BIOS or drop into realmode and issue the BIOS
> >>>>> calls?
> >>>>
> >>>> Native Linux gets this information passed from the boot loader, I think
> >>>> (except in the EFI case, as per below).
> >>>>
> >>>>> Is that an issue on UEFI also, or there dom0 can fetch the framebuffer
> >>>>> info using the PV EFI interface?
> >>>>
> >>>> There it's EFI boot services functions which can be invoked before
> >>>> leaving boot services (in the native case). Aiui the PVH entry point
> >>>> lives logically past any EFI boot services interaction, and hence
> >>>> using them is not an option (if there was EFI firmware present in Dom0
> >>>> in the first place, which I consider difficult all by itself - this
> >>>> can't be the physical system's firmware, but I also don't see where
> >>>> virtual firmware would be taken from).
> >>>>
> >>>> There is no PV EFI interface to obtain video information. With the
> >>>> needed information getting passed via start_info, PV has no need for
> >>>> such, and I would be hesitant to add a fundamentally redundant
> >>>> interface for PVH. The more that the information needed isn't EFI-
> >>>> specific at all.
> >>>
> >>> I think our only option is to expand the HVM start info information to
> >>> convey that data from Xen into dom0.
> >>
> >> PHV doesn't use the ordinary start_info, does it?
> > 
> > No, it's HVM start info as described in:
> > 
> > xen/include/public/arch-x86/hvm/start_info.h
> > 
> > We have already extended it once to add a memory map, we could extend
> > it another time to add the video information.
> 
> Okay, I'll try to make a(nother) patch along these lines. Since there's
> a DomU counterpart in PV's start_info - where does that information get
> passed for PVH? (I'm mainly wondering whether there's another approach
> to consider.)

We don't pass the video information at all for PVH, neither in domU or
dom0 modes if that's what you mean. Not sure what video information we
could pass for domU anyway, as that would be a PV framebuffer that
would need setup ATM. Maybe we could at some point provide some kind
of emulated or passed through card.

The information contained in start_info that's needed for PVH is
passed using hvm params, just like it's done for plain HVM guests. We
could pass the video information in a hvm param I guess, but it would
require stealing guest memory to store it (and mark as reserved in
the memory map). Not sure that's better than expanding HVM start info.

Maybe there's another hypercall interface I'm missing we could use to
propagate that information to dom0?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 16:44:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 16:44:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186958.335695 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQBXJ-0005ue-Hm; Tue, 14 Sep 2021 16:43:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186958.335695; Tue, 14 Sep 2021 16:43:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQBXJ-0005uX-Ej; Tue, 14 Sep 2021 16:43:57 +0000
Received: by outflank-mailman (input) for mailman id 186958;
 Tue, 14 Sep 2021 16:43:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xqqe=OE=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mQBXI-0005uR-6w
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 16:43:56 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f2af4720-157a-11ec-b457-12813bfff9fa;
 Tue, 14 Sep 2021 16:43:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2af4720-157a-11ec-b457-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631637834;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=Ackbs8ldT2lfoTN6n5zVtWeptYPWxyqD2XRJg4SRF/w=;
  b=a9+RizmGNpZBAsjUaOc3ceufUAhxngfyV0haaxTIj8Acptm5kRGejYgB
   Lc2132Na8tFSOpJuDKxXaJd93OUtMKgj3HxQvUWrtwM4UPJibVyhXg8l7
   gqS1ggWi9nUpbfTKegp59td6R3+unbgk/48AJjc3hitGGGfqHvOT7zII7
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: ImFavC2M3FRwXOJDiwoSOeVLlj+33Xz5eey6YhaFhap9F2kmL3fPZXu7C5faq5uFq2lS9YiJjJ
 idmGNFOXJsao7+HBTbZ/JyeU5qB7Ul62m65cqfWNByEZf8fQNAwEGIvvhwKAMAq5p2g3TsWc3I
 FdZX/1ttXD1kdefX/++X5slXAG7bZV23VJHlZS15R6yvBjgHcgIR30YIQLZ6SmWXJvZtOHUGe5
 VvrMeb+wV+oGug93MMP7kOg6vEuxVLQ5b9ZUohhDhNrhI83VDVNHx3kK6QhGIhcwwZLNpwxqxZ
 qs0Xn1zvet5jX8Xus+MUcpDf
X-SBRS: 5.1
X-MesageID: 52694767
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:N8dsEanaqXaa5/Lj3FtzCtvo5gy5IURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIdUW6Fb6qDa2qjL9l+bYmz801VvsOBytAwGgdrrnw3FSMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09c02YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 OpdjJ2vSj9zBPbnxrkPWisGHQtRM6ITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBOrmIoIZ/Ep8wD/QC/E4aZvCX7/L9ZlT2zJYasVmQKyAN
 pZGN2UHgBLoRhMTN20VUI0Hrvb0lkP7YyR7sX+JjP9ii4TU5FMoi+W8WDbPQfSVQe1Fk0Deo
 XjJl0z7CBwHMN2UyRKe72mhwOTImEvTSI8UUbG16PNuqFmS3XAITg0bU0Ohpvu0gVL4XMhQQ
 2QV5SgGvaU07FasTNT2Q1u/unHsg/IHc4MOSatgsljLk/eKpVbCboQZctJfQOM2jc4pRgRw7
 ASiwY77DA4xvpe6R33Io994sgiO1TgpwX4qPHFfFFZUvIa9+enfnTqUEY0yS/fdYsndXGiqm
 mHU9nBWa6A70JZTv5hX62wrlN5FSnLhdQcz+gyfdWas9AoRiGWNNtHwtASzARqtKu+kori9U
 JoswJP2AAMmV8jleMmxrAMlRu/BCxGtamG0vLKXN8N9nwlBAlb6FWyq3N2bGKuPGpxVEdMOS
 BSI0T69GbcJZCf6BUOJS9vpVqzGMpQM5fy6D6uJP7Kik7BadROd/TEGWKJj9zm2yyARfVUEE
 c7DK66EVC9CYYw+lWbeb7pNgNcDm3FlrUuOFM+T8vhS+efHDJJjYexeawXmgyFQxP7snTg5B
 P4Ea5LRmkkACbanCsQVmKZKRW03wbEALcieg6RqmiSre2KKwUktVK3cx60PYYtgk/gHn+vE5
 CjlCERZ1ED+lTvMLgDTMiJvb7bmXJBeq3MnPHNzYQb0iiZ7OYv/vr0Cc5YXfKU88LAxx/BDU
 PRYKd6LBe5CS2qb9m1FP4X9toFraD+imRmKY3i+eDE6cpM5H17J99bocxHB7i4LCibr58Iyr
 6f5jlHQQIYZRhQkB8HTMar9w1S0tHkbueRzQ0qXfYUDJBSyqNBncnWjgOU2LscALQT46gGbj
 wvGUw0FoeTtopMu9IWbj66zsIr0QfB1GVBXHjeH4O/uZzXa5Geq3aRJTP2MIWLGTGrx9aivO
 bdVwvX7PKFVlVpGqdMhQbNizKZ47Nrzvb5KiA9jGSyTPVisD7phJFiA3NVO6fIRluMI51PuV
 xLd4MReNJWIJNjhQQwYKwcSZ+ie0e0Zx2vJ5vMvLUSmvCJ68dJri6mJ08VgXMCFEIZIDQ==
IronPort-HdrOrdr: A9a23:JFLz/qBUJJmqMCflHemg55DYdb4zR+YMi2TC1yhKJyC9Ffbo8/
 xG/c5rsyMc5wxwZJhNo7y90cq7MBbhHPxOkOos1N6ZNWGM0gaVxelZnOzfKlbbehEWmNQz6U
 4ZSdkdNOHN
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52694767"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich
	<jbeulich@suse.com>
Subject: [PATCH v2] tools/libxl: Correctly align the ACPI tables
Date: Tue, 14 Sep 2021 17:43:23 +0100
Message-ID: <20210914164323.14904-1-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The memory allocator currently calculates alignment in libxl's virtual
address space, rather than guest physical address space. This results
in the FACS being commonly misaligned.

Furthermore, the allocator has several other bugs.

The opencoded align-up calculation is currently susceptible to a bug
that occurs in the corner case that the buffer is already aligned to
begin with. In that case, an align-sized memory hole is introduced.

The while loop is dead logic because its effects are entirely and
unconditionally overwritten immediately after it.

Rework the memory allocator to align in guest physical address space
instead of libxl's virtual memory and improve the calculation, drop
errant extra page in allocated buffer for ACPI tables, and give some
of the variables better names/types.

Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>

v2: Rewrite completely, to align in guest physical address space
---
 tools/libs/light/libxl_x86_acpi.c | 47 +++++++++++++------------------
 1 file changed, 19 insertions(+), 28 deletions(-)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 3eca1c7a9f..8b6dee2e05 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -20,6 +20,7 @@
 
  /* Number of pages holding ACPI tables */
 #define NUM_ACPI_PAGES 16
+#define ALIGN(p, a) (((p) + ((a) - 1)) & ~((a) - 1))
 
 struct libxl_acpi_ctxt {
     struct acpi_ctxt c;
@@ -28,10 +29,10 @@ struct libxl_acpi_ctxt {
     unsigned int page_shift;
 
     /* Memory allocator */
-    unsigned long alloc_base_paddr;
-    unsigned long alloc_base_vaddr;
-    unsigned long alloc_currp;
-    unsigned long alloc_end;
+    unsigned long guest_start;
+    unsigned long guest_curr;
+    unsigned long guest_end;
+    void *buf;
 };
 
 extern const unsigned char dsdt_pvh[];
@@ -43,8 +44,7 @@ static unsigned long virt_to_phys(struct acpi_ctxt *ctxt, void *v)
     struct libxl_acpi_ctxt *libxl_ctxt =
         CONTAINER_OF(ctxt, struct libxl_acpi_ctxt, c);
 
-    return (((unsigned long)v - libxl_ctxt->alloc_base_vaddr) +
-            libxl_ctxt->alloc_base_paddr);
+    return libxl_ctxt->guest_start + (v - libxl_ctxt->buf);
 }
 
 static void *mem_alloc(struct acpi_ctxt *ctxt,
@@ -58,20 +58,16 @@ static void *mem_alloc(struct acpi_ctxt *ctxt,
     if (align < 16)
         align = 16;
 
-    s = (libxl_ctxt->alloc_currp + align) & ~((unsigned long)align - 1);
+    s = ALIGN(libxl_ctxt->guest_curr, align);
     e = s + size - 1;
 
     /* TODO: Reallocate memory */
-    if ((e < s) || (e >= libxl_ctxt->alloc_end))
+    if ((e < s) || (e >= libxl_ctxt->guest_end))
         return NULL;
 
-    while (libxl_ctxt->alloc_currp >> libxl_ctxt->page_shift != 
-           e >> libxl_ctxt->page_shift)
-        libxl_ctxt->alloc_currp += libxl_ctxt->page_size;
+    libxl_ctxt->guest_curr = e;
 
-    libxl_ctxt->alloc_currp = e;
-
-    return (void *)s;
+    return libxl_ctxt->buf + (s - libxl_ctxt->guest_start);
 }
 
 static void acpi_mem_free(struct acpi_ctxt *ctxt,
@@ -163,7 +159,6 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     struct acpi_config config = {0};
     struct libxl_acpi_ctxt libxl_ctxt;
     int rc = 0, acpi_pages_num;
-    void *acpi_pages;
     unsigned long page_mask;
 
     if (b_info->type != LIBXL_DOMAIN_TYPE_PVH)
@@ -186,19 +181,17 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
     config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
     /* Pages to hold ACPI tables */
-    acpi_pages =  libxl__malloc(gc, (NUM_ACPI_PAGES + 1) *
-                                libxl_ctxt.page_size);
+    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES *
+                                   libxl_ctxt.page_size);
 
     /*
      * Set up allocator memory.
      * Start next to acpi_info page to avoid fracturing e820.
      */
-    libxl_ctxt.alloc_base_paddr = ACPI_INFO_PHYSICAL_ADDRESS +
-        libxl_ctxt.page_size;
-    libxl_ctxt.alloc_base_vaddr = libxl_ctxt.alloc_currp =
-        (unsigned long)acpi_pages;
-    libxl_ctxt.alloc_end = (unsigned long)acpi_pages +
-        (NUM_ACPI_PAGES * libxl_ctxt.page_size);
+    libxl_ctxt.guest_start = libxl_ctxt.guest_curr = libxl_ctxt.guest_end =
+        ACPI_INFO_PHYSICAL_ADDRESS + libxl_ctxt.page_size;
+
+    libxl_ctxt.guest_end += NUM_ACPI_PAGES * libxl_ctxt.page_size;
 
     /* Build the tables. */
     rc = acpi_build_tables(&libxl_ctxt.c, &config);
@@ -208,10 +201,8 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     }
 
     /* Calculate how many pages are needed for the tables. */
-    acpi_pages_num =
-        ((libxl_ctxt.alloc_currp - (unsigned long)acpi_pages)
-         >> libxl_ctxt.page_shift) +
-        ((libxl_ctxt.alloc_currp & page_mask) ? 1 : 0);
+    acpi_pages_num = (ALIGN(libxl_ctxt.guest_curr, libxl_ctxt.page_size) -
+                      libxl_ctxt.guest_start) >> libxl_ctxt.page_shift;
 
     dom->acpi_modules[0].data = (void *)config.rsdp;
     dom->acpi_modules[0].length = 64;
@@ -231,7 +222,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     dom->acpi_modules[1].length = 4096;
     dom->acpi_modules[1].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS;
 
-    dom->acpi_modules[2].data = acpi_pages;
+    dom->acpi_modules[2].data = libxl_ctxt.buf;
     dom->acpi_modules[2].length = acpi_pages_num  << libxl_ctxt.page_shift;
     dom->acpi_modules[2].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS +
         libxl_ctxt.page_size;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 17:23:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 17:23:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186968.335706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQC9b-000235-Km; Tue, 14 Sep 2021 17:23:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186968.335706; Tue, 14 Sep 2021 17:23:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQC9b-00022y-Gw; Tue, 14 Sep 2021 17:23:31 +0000
Received: by outflank-mailman (input) for mailman id 186968;
 Tue, 14 Sep 2021 17:23:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W5lM=OE=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mQC9a-00022s-EK
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 17:23:30 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f18c24c1-9f0b-4d65-a525-6b9d5abda0ec;
 Tue, 14 Sep 2021 17:23:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f18c24c1-9f0b-4d65-a525-6b9d5abda0ec
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631640208;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=Iqz4ShWtEm+izaVvkJ/yoQ+XF0ghWvDCZvP5xa0LC3k=;
  b=fwZqpmEflNSD2RqV+3NKe5fayyTpO9nrcts5MjhzcLIWJCBxNPTYr9ld
   xoQ+VjJzgoXZ8scWROR4EcRFlL2eVWYTVpXMxPypIlTvvbRQNPaoXDMow
   L4qwCs/PBSUOOJSjqTu4C7VXvvms+ESeNwrQGxbfsXhNui9FDabiC8+6L
   Q=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: j15wnaleKkOGmh2IjJOYOPSnnww2v0J44Pj/DoL3zsyrETbWGvcd813HssyJMQ3/HK/6ugYRWM
 s5WZ/MIo0OjvJGX2WyS2JeGRFcGZs4CW+DOo49xCIIiYgQsBS4qXeSgdsily26/fcVobR7UAq7
 /hhpcCEBeij2YrFqbWS7AlSRmoDbcvn5yJXOTz9SPDaYdAfmTfFAj1OW8DlZvRrDH8iQGfp9m5
 3Zgb0rdcgCgYQ64RMQGI4Uu9MeGQAOK/b3JKlwHsrwJMqVJJRBvEEAAHUlM8h6KZcIc94BEhaX
 davOt3cy5Z7TVLVSzYYTf5l6
X-SBRS: 5.1
X-MesageID: 52301933
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:7r+Z0Km423y4SNhGpAabSpjo5gxQIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJNUWzXbv2MZzf0Ktl2YYW/oEgBv57WyoQ2G1dv+CthECMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09c02YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 I9Gr66xSyAoBK7JueoXXR5JSAtEbJQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQKyFP
 5NIMmEHgBLofzcXZHslNakCrcT1tnn4fGxTuQK8qv9ii4TU5FMoi+W8WDbPQfSLWsd9jkuev
 njB/WnyHlcdLtP34SqI9Degi/HCmQv/WZkOD/uo+/hymlqRy2cPThoMWjOTo/O0l0q/UNJ3M
 FEP92wlqq1a3FOvZsnwWVu/unHsg/IHc4MOSatgsljLk/eKpVbCboQZctJfQIYtrv03Rxsr7
 FyMvt3jCzt+65evGEvIo994sgiO1TgpwX4qPHFfFFZUvIa9+enfnTqUEY0yS/fdYsndXGiqm
 mHU9nBWa6A70JZTv5hX62wrlN5FSnLhdQcz+gyfdWas9AoRiGWNNtHwtASzARqtKu+kori9U
 JoswJP2AAMmV8jleMmxrAMlRu/BCxGtamG0vLKXN8N9nwlBAlb6FWyq3N2bGKuPGpxVEdMOS
 BSI0T69GbcJZCf6BUOJS9vpVqzGMpQM5fy6D6uJP7Kik7BadROd/TEGWKJj9zm2yyARfVUEE
 c7DK66EVC9CYYw+lWbeb7pNgNcDm3FlrUuOFM+T8vhS+efHDJJjYexeawXmgyFQxP7snTg5B
 P4Ea5LRmkkACbanCsQVmKZKRW03wbEALcmeg+Rcd/KZIxogH2ckCvTLxqgmdZAjlKNQ/tokN
 FnkMqOB4Fag13DBNyuQbXVvNOHmUZpl9CppNi0wJ1e4nXMkZN/3vqsYcpI2e5gh9fBikqEoH
 6VUJZ3YD6QdUCnD9hQccYL58N5oeiO0iF/cJCGiejU+IcJtHlSb5t/+cwLz3yASFS7r59Amq
 rit21qDE5oOTghvFujMb/erww/jtHQRgrsqDUDJPsNSaAPn940zc379ifo+IsctLxTfx2TFi
 1bKUElA/eSU+t076tjEg6yAvryFKeomExoIBXTf4Ja3KTLeojipz7hfXbvaZjvaTm71pvmvP
 L0H0/HmPfQbt19WqI4gQa1zxKcz6taz9b9XygNoQCfCY1ixU+4yJ3CH2Y9Et7FXx68fsgyzA
 xrd9t5fMLSPGcXkDF9Oe1Z1MrXdjakZymvI8PA4AETm/ysmrrOIXHJbMwSIlCEAfqB+N5kow
 Lt5tcMbg+BlZsHG7jpSYvhoylmx
IronPort-HdrOrdr: A9a23:k1zRfqwscXDG6XdgXDu1KrPwIr1zdoMgy1knxilNoH1uHvBw8v
 rEoB1173DJYVoqNk3I++rhBEDwexLhHPdOiOF6UItKNzOW21dAQrsSiLfK8nnNHDD/6/4Y9Y
 oISdkbNDQoNykZsfrH
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52301933"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IEpLDkPqHRiAd2p0V/W1yHPZSaeP+o+5MdJ77WtjleKV2LYeqf2yxWaYX8Yxc+RYQPO5v5GQvNtwYGAQ26hJj9d8HadAxdwc+Ifebp9qKVOq8pl3wA5/AyD3p4IbNZ4SvhJpbie1uF+O4V2rfhZFXYMgdxWaw0fRPrtuyqQzKs9QP6jcVTnaXeks5UsRYyoUhEw2r9poILYKOWO+oGIbTZmKFkWwmB9h+y7T2k4WDlVVuLRrekK4WTa97J9xzuo0Hrtz65ShkTLqWAi6ZydqruP364AkoWI3+qESEkm3FIgefMiz0WICS5BK/7YifZuT/EU24JuFhOuxI64+E5ROfw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kWgIW2qbylxRzv9ou/AJZm+kYnZkcCza3B6+Wu8228Y=;
 b=I2OIIV0KJuoAckAYCP3RLtYMAEDcTW5gMMqKUICg9ljqn6hvXcNCCmVxBTpGJtXxUDkkjDAtKzD8P7VFZBRaREwEFWPxcL1UQP81f7mzytotWmH8sZ6Y9MGmUZBcYBrXmiD88/3pTme5lq0CsbsZVMFyNU+fjhF9yI6qbPTaWza/nFfp99Hn6iYbW6QeZGQeUfLxe0y7vKvCz8QI1nc0Ok8mmtMONxY1nk6DW/JVp4gLGL6Lc26EUndcvIwxmNzBz4MMaLOe8AL6tsxuin6uRXjKXJ+Y6faukJrLn8OKVUBy3lw6Lt64StZLDfBTCPHyz1DaSSNfE5KyOk+p/F1aVw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kWgIW2qbylxRzv9ou/AJZm+kYnZkcCza3B6+Wu8228Y=;
 b=FpqSsa/4OPM7ofrLHtGJLy+kLbsFy61O2YKdbwQXUGIVHSvkQyiINu4rmL5/uuydBFJTqhT2y+naF6hu7fqlZZpgW2EDCWNd01sb7fTCPmH1kD0U/IQXF/5tYam3WiQ/PAk1X1mrQI9v2hLVGQoe+GNAThqz2VVq28asoRPvtjQ=
To: Kevin Stefanov <kevin.stefanov@citrix.com>, Xen-devel
	<xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>
References: <20210914164323.14904-1-kevin.stefanov@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2] tools/libxl: Correctly align the ACPI tables
Message-ID: <3ac67acd-3c37-4e82-341a-99d499d84350@citrix.com>
Date: Tue, 14 Sep 2021 18:23:17 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210914164323.14904-1-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0234.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:b::30) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 174f5a6b-77b1-4ea4-9a75-08d977a45bb0
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5584:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB558457250925E2323211E204BADA9@SJ0PR03MB5584.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: TC1V0pPMnst/fr4tD7fGU1JlUvgMGqZIlMCobr+iR5pNSYUO8QP6HOvEWjwU/R+2xrN1/X12WP+Y/r8ESDIkJMpekShInP0Fokdk+E8fC+oAPnfhd6YpzZUuK+aGhJ6sSGEQJ2/iLmTQxNbB/nUeZ/X5uzjPk7TTAYj8qNGD3Q1IzUh5M7MEvHZEWV6yrO5/oAuyqZR8eml0lU+hNRM/BcCs9OGQMsOAfVJiFM6rny4WQzmnlBLlsomItQA1v4J6tQ5qpemsxfiD5vU0kmeB2/CslK3Z/LfoBBh5fuwJw/A3x8NL6MYmg/3Jlgz33oZhjgBYfKoDHqonvmIxdF9GYV5+9+e9b1lAnF2GiwF28hxVPmg7XCTsyrfIGHJ3/EwNZesWC0cgbPqta3sboJd4GKf5KwiYQwKu8bN9iH+Wcrsafo/+zYzcHykEvsqNieXmWwUzpZirSdw9KgPU4GoPKCLZXYVUJUcWx7Ggg0vanD8pc4K0xT8muuSVWKFxnYHXdWjvIpFCRrEZysd4FTK3LM5XuUsz9te+VaJFLBBqX/J5DG2T2sUr6MBxxSq4sOFADYaQwL2/USMS8kRMeOXAgoqd4v4RX40UELGEA63nShWPWeOQPTGQjUKTXqBZ9JQi54NBQebCwa7uqte3PXnE631yYs0ckKoVNYjnn5pSjJwDRIL1QsUCwWC2xQOYA+BYYDDCxd+ZbL7u9gEzRw+h0VCKAdplY3L+46Q2sa893Q0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(396003)(346002)(376002)(366004)(6486002)(956004)(2616005)(6666004)(8676002)(54906003)(110136005)(83380400001)(16576012)(55236004)(4326008)(66556008)(26005)(31686004)(38100700002)(36756003)(86362001)(5660300002)(53546011)(31696002)(8936002)(2906002)(186003)(478600001)(316002)(66946007)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NDgzZ29SZ3ZOb1dRR2l5Ui9yaDhrNWk5Q0xMYnVZb0Q5QzJpWmdDOWpLdlds?=
 =?utf-8?B?K0xWclRua0pwY01qQWlmaWFSKytDV1RKNURwUVlYZWFMWmk3SWhiOHFSZ0Vq?=
 =?utf-8?B?di9KT1d0bnF1empRNE4wZUFiUkF2NWZqZUgyS3NOQk5hNHpUOXliczNHdWEw?=
 =?utf-8?B?cDBwUGprb0hoUTk0YmhGaCszSVRNNFI5ak5zRnZkVGxvSmF0L1p6SlVjbEg2?=
 =?utf-8?B?NkdzMWM3UTVjR1c0QWMzcUlWK2o4Z3BRZ3EySUdNMHMxOVI1K3FEN1p1U25B?=
 =?utf-8?B?M28xM010K3MvalJENW12enNlK0dYc0c3akVMYkN4SXlpM1Fpd0toSEtKMDI0?=
 =?utf-8?B?cGorNWMzRmRhNXNVTEswVXNDajlINmI1VHFFSURXamJpek9QVUl4eEFmT0xJ?=
 =?utf-8?B?V01QZTEyVVBSYWRQT3NWT0VaY2NtYmhmTVZnTE9ScWdxcGZIakRUVU1DcGVM?=
 =?utf-8?B?c2ErenBqRDFLVjI4S1Z5dWdQOEhKcjhSYlQwV2dxZ1QxVFNZNjBsdkdYcDhW?=
 =?utf-8?B?b0VYcmJJbWh0MGNaZUdWK1hLSGpEc0ZxTDdxTnhuKzdIUXRnZXM2Tnlrcm43?=
 =?utf-8?B?V2xSUVVkNGJCcDR5Z3c0TkpRaGJPWGF3aTlDd0lXb2Qrc1dXWE1LL0l4eUcw?=
 =?utf-8?B?alZDV0tRc2xuTVVyMXJUNnJUdUUxWnREOEV0U2MvcmNSMFRQc0NTVVFwS01a?=
 =?utf-8?B?Yng1Ry8xTlJFVjRhMjJLK1Exc2pyVlRqYmNJOVhiSHc5TU8weWR4OElSVDRt?=
 =?utf-8?B?K1B2N01rSzFzVUw5MmxUaFppaFdNTG53NDM5UjJ3WVZlUTh2WE4wd3dKNGxy?=
 =?utf-8?B?K3pCUFlVZU1IY3VJcW1tQTRtOTkwLzk3SEdZRmYrZlk2RUZOdWNYb3pwMEI3?=
 =?utf-8?B?c2hialROeVRJNkNzVmhGeXMrV0lBc1JOQmV0YjJaVUtXeG91NTJDMjY1RDc0?=
 =?utf-8?B?NEZmOWRXMDFaRk9pSVlPVkg4LytjTjdWWkZUMVBjN0podUNRSUVxSWsvTVpW?=
 =?utf-8?B?TEZDL0wxa2RXeC9VMklwWHdpaUdYV05XdXhMWUtyZ1hjekJsS0lBR3RKQTdH?=
 =?utf-8?B?VnVsVm5hL0lJNC9yVDZFTlZKcmRudlBsODdFSitPdENNNENUd1R5Ni8rV2c5?=
 =?utf-8?B?Y1Q4NDFzOGdJS0lsVjdCNDNCcE9FRzBHT0VTQ1c0Y2hNaFBuaFV3aElHZUdk?=
 =?utf-8?B?UVlXK3NOL3JVcVV2K3ZldlpJekpsRm9KOFZtOFQ1NlAwSzNVYytTTGZ1NTYr?=
 =?utf-8?B?TDdybDZaTnNlUVZLbFU4RzF6dXVEb2lWMmlpM1BGOFgwUjl4UUdjYTlFRjc3?=
 =?utf-8?B?U3B5YXQzTnNmM0lTYnlQNlBVQXkzd3lMMG13YkxvUUhVUG9Ub3FIMVBUdTBq?=
 =?utf-8?B?VitVc1Uva0tOTU93RCtSUmdQbnJkeVZtdFlEaTVVUTI4YmYxZllkSjhRcjJx?=
 =?utf-8?B?TGxZVHN4dzFGaXFtdmVEaENrd2lPWjJYc29ycFJnTGpKNXVUTi9wbGtwVjZk?=
 =?utf-8?B?RUJkTGFTNi9kZmN0QXV2VHUxT0tjMVY3TUdQVGg0TWNlMlh2dkR4clBUZG9y?=
 =?utf-8?B?a1hOMnRXMHozMHZXeGk3V0JMR3VRUXZRb004Q1BLLzI1OXpjZnJkaWRKWjU1?=
 =?utf-8?B?YTdJaEVoaU1icmh1MjFTU3B6WXhlcVo4aVpaLzVGdHpySEVWWmFrWnhsTnhv?=
 =?utf-8?B?cmVkS1dkdU5FeTdmNDFITjJ0Y0kvZlFjY0FySHFkQTRxZ0o4L2dEUEo4RGdG?=
 =?utf-8?Q?GUB3OiNStTMAWzGtoeY/ngTTHuSp71RpYuvCa+C?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 174f5a6b-77b1-4ea4-9a75-08d977a45bb0
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 17:23:24.8007
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /322zbDwGVhk1hu/NXLuwORMWXE4jZ6UWJj0N4Quu3fUoCe3kGnIyggo0Rm1FmOfuod9jUflv/HUrrq5UiHet8O3gHe1w9gZNTFjgXkJWd4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5584
X-OriginatorOrg: citrix.com

On 14/09/2021 17:43, Kevin Stefanov wrote:
> The memory allocator currently calculates alignment in libxl's virtual
> address space, rather than guest physical address space. This results
> in the FACS being commonly misaligned.
>
> Furthermore, the allocator has several other bugs.
>
> The opencoded align-up calculation is currently susceptible to a bug
> that occurs in the corner case that the buffer is already aligned to
> begin with. In that case, an align-sized memory hole is introduced.
>
> The while loop is dead logic because its effects are entirely and
> unconditionally overwritten immediately after it.
>
> Rework the memory allocator to align in guest physical address space
> instead of libxl's virtual memory and improve the calculation, drop
> errant extra page in allocated buffer for ACPI tables, and give some
> of the variables better names/types.
>
> Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
> ---
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Ian Jackson <iwj@xenproject.org>
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Jan Beulich <jbeulich@suse.com>
>
> v2: Rewrite completely, to align in guest physical address space

It appears that patchew isn't running properly right now, but ...

> diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x=
86_acpi.c
> index 3eca1c7a9f..8b6dee2e05 100644
> --- a/tools/libs/light/libxl_x86_acpi.c
> +++ b/tools/libs/light/libxl_x86_acpi.c
> @@ -208,10 +201,8 @@ int libxl__dom_load_acpi(libxl__gc *gc,
>      }
> =20
>      /* Calculate how many pages are needed for the tables. */
> -    acpi_pages_num =3D
> -        ((libxl_ctxt.alloc_currp - (unsigned long)acpi_pages)
> -         >> libxl_ctxt.page_shift) +
> -        ((libxl_ctxt.alloc_currp & page_mask) ? 1 : 0);
> +    acpi_pages_num =3D (ALIGN(libxl_ctxt.guest_curr, libxl_ctxt.page_siz=
e) -
> +                      libxl_ctxt.guest_start) >> libxl_ctxt.page_shift;


... this hunk means page_mask lost its only user, and is now a
written-but-not-used variable, which some compilers will warn about.

The fix is easy, and just to drop that local variable too, which is
another 2 lines deleted in the resulting patch.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 17:59:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 17:59:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186980.335717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQCie-0005jW-MP; Tue, 14 Sep 2021 17:59:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186980.335717; Tue, 14 Sep 2021 17:59:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQCie-0005jP-JD; Tue, 14 Sep 2021 17:59:44 +0000
Received: by outflank-mailman (input) for mailman id 186980;
 Tue, 14 Sep 2021 17:59:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQCic-0005jF-RX; Tue, 14 Sep 2021 17:59:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQCic-0003De-LV; Tue, 14 Sep 2021 17:59:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQCic-00048R-AY; Tue, 14 Sep 2021 17:59:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQCic-0007Fx-9v; Tue, 14 Sep 2021 17:59:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+Th5ZNqakdaYEP70iUz2+afsaz5P4/SNK43BICqSvRQ=; b=0p8OKKo70tfE55PkhavUeglQHD
	z/EyXmfXlrEGnCxCL4voMCS+xXlDAWmJ42KiFb2FXpnKgo1gpIex3SlnnmNQy67b/LNeTYFzT/FfL
	KpTwrbkjlxpEyQ93ItQfg0PKBsT1MdG+LhV2QVrSSPLQu/xQm7PwwcWaiCixWgR3GiOY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164980-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164980: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-libvirt-raw:xen-boot:fail:heisenbug
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=7d79344d4fa44e520e6e89f8fed9a27d3d554a9b
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 17:59:42 +0000

flight 164980 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164980/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw  8 xen-boot                   fail pass in 164973

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail in 164973 like 164950
 test-armhf-armhf-libvirt-raw 14 migrate-support-check fail in 164973 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                7d79344d4fa44e520e6e89f8fed9a27d3d554a9b
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    2 days
Failing since        164967  2021-09-13 13:06:52 Z    1 days    3 attempts
Testing same since   164973  2021-09-13 23:09:51 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Berrangé <berrange@redhat.com>
  Lara Lazier <laramglazier@gmail.com>
  Markus Armbruster <armbru@redhat.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 461 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 18:43:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 18:43:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186994.335731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQDOM-0002Uy-Hx; Tue, 14 Sep 2021 18:42:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186994.335731; Tue, 14 Sep 2021 18:42:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQDOM-0002Ur-Ek; Tue, 14 Sep 2021 18:42:50 +0000
Received: by outflank-mailman (input) for mailman id 186994;
 Tue, 14 Sep 2021 18:42:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W5lM=OE=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mQDOL-0002Ul-4Y
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 18:42:49 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8dfd0770-158b-11ec-b46a-12813bfff9fa;
 Tue, 14 Sep 2021 18:42:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8dfd0770-158b-11ec-b46a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631644967;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=0VXfCH/cdwJgEQKo0+jzzSzMuUH2pcOWCTzTFLRlO/A=;
  b=DpEiu69cx5dH6I35m7tdg7ezDA4pUjpt7gP5s0Qtuqq/o/v+Bs285Z4Y
   2mgvK8CF1HKu0feKhBKEhgo7ul4TeFdZCJsi8rTCpE1KVeOQsMutw8wcr
   8easfDlDsIWn+Je7OcMi3kXsRjUZTzdDaRziZrv4gvzh4dzBIs3hP/8FZ
   0=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 6sjF8AzFtevgUfDVSvNxXKIcVYvBzHgsclBvj4RQ0Jde8Q0e5GotGtXyX7k431i++OzU76r2Og
 Rq5lfb9fuPdR7V5bIUtimGn2P3FKRcPoY5ocU7Yb2EYzUwR9H4JSBjXtV1BTv6SPY8hvz9wbxj
 hFzqnH3n0TdyLNFLVsF0aLC+chjizymVYKVcVN3iYJihImHsZxUsNHtn+hiBdxvfQJmtis7S29
 WdUatjU5y7TaQLUHMPf5JLtIh4rRz02SCTZJoq8QOc2cdsGG8G7/RxiG/xR3XjTlqiC84d4MV8
 Q0R/gaFmP8xE3KDPMYiStYqd
X-SBRS: 5.1
X-MesageID: 52729877
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:pmlzuKJr0Fbgj5+4FE+R6JMlxSXFcZb7ZxGr2PjKsXjdYENS1TEAz
 mYcXjjQPKuPYDfxett/YY+w9xkF7JKBmtZjS1NlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6x7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2lgflr0
 88cr6eLFwoqLLXOp+MZeiBxRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gpq2ZofTamBD
 yYfQRdgPCrdPxFmBlQ4IpZ9juXyo1T/cyIN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wv
 H/d4yHnBxQyMN2E1SHD9WqhgOPCg2X8Qo16PK218LtmjUOewkQXCQYKTh2rrP+hkEm8VtlDb
 UsO9UIGpKw/5AqhQ9/7UhCQpH+CtwQbHd1KHIUS7QiRyqvZ/kCBAWkeTzNbQNgnssYsQnotz
 FDht83oHzF0sPuWVHeU7J+QrDW7Iy9TKnUNDQcPRBEJ5NTLq4gpghXCCN1kFcadgtDwGBnxx
 S6Ltyk0g7gPjc8N2L6/9FqBiDWpzrDMRxQw7x/aXUqk6B14f4+vY4G06Vnd4u1EJYzfRV6E1
 FAfh9KX8OcVSJ2AigSKWKAJHaq1/LCBMTvVm1kpGIMunxyq/3OgZpxB+zFWK0JgM8JCcjjsC
 GfIpQ5f7ZlUemC2ZKV6Z4awDcUC3ankFNL1EPvTa7JzjoNZLVHduns0PAjJgj6rwBNEfbwD1
 YmzXdiFF20AWL5c92S7RPUF04AswgkH7DaGLXzk9ChLwYZydVbMF+xcbQrRMb5ghE+XiF6Kq
 IcEbqNm3z0aCbenM3eNqeb/OHhXdSBTOHzglyBAmgdvyCJdEWc9Arf6xbo7cuSJdIwEy7+Vo
 hlRtqJeoWcTZEEryy3RMRiPi5u1BP6TSE7X2wR2Yz5EPFB5Pe6SAF83LcdfQFXe3LULIQRIo
 xw5lyOoWKUnptPvoG91UHUAhNY6KETDafymZnL4CNTAQ3KQb1OQoYK1Fuce3AIPEjC2paMDT
 06Ij1iAKafvsz9KVZ6MANr2lgvZlSFExIpaAhuZSvEOKR6E2NU7dETMYgoffphkxePrnWDBi
 W57wH4w+IHwnmPC2IKV3PvV89jwSLIW84gzNzCz0Ita/BLypwKL6YRBTPyJbXbaUmb187+lf
 uJb07f3N/hvobqAm9MU/29Dwf1s6t3xiaVdywg4TnzHY07yUuFrI2Wc3NkJvapIn+cLtQyzU
 0OJ299bJbTWZ5+1TA9PfFIoPraZyPUZujjO9vBpck/00zB6oeicWkJIMhjS1CEEdOlpMJkoy
 PsKsdIN71DtkQIjN9uL13gG92mFInEafb8gs5UWXN3ihgYxkwkQap3AEC7mppqIbowUYEUtJ
 zaVgovEhqhdmRWeIyZiSyCV0LME15oUuR1MwFsTHHizm4LI1q0twRlc0TUrVQAJnB9J5P1+Z
 zpwPEpvKKTQozox3JpfX3qhEh1qDQGC/hCj0EMAkWDUQhX6VmHJK2Fha++B8FpArjBZdzlfu
 rqZ1HzkQXDhe8Sohnk+XktsqvrCS91t91KdxJD7TprdR5RqMyD4hqKOZHYTr0q1CMw8s0TLu
 O128bsicqb8LyMR//U2BoTyOW78k/xYyLiumc1cwZ4=
IronPort-HdrOrdr: A9a23:VHadHaHOfrlqKWIlpLqFeJHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HkBEDyewKiyXcT2/hsAV7CZniahILMFu9fBOTZskXd8kHFh4lgPO
 JbAtJD4b7LfChHZKTBkXCF+r8bqbHtmsDY5pas854ud3APV0gJ1XYJNu/xKDwReOApP+taKH
 PR3Ls9m9L2Ek5nEPhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2mdfVmGJ6VMt1xN/dfOla9My4kD1jiVWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,292,1624334400"; 
   d="scan'208";a="52729877"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lKeA8jGULhnI/BfzO0c5rLLIVneHty2Ep2ofAZeUBvLoEhbKvWOAOEYmiLuM7dFuzlldQtraAt5axHlKpsUaGiHRjoMT6vOCN6CAQDqcIX4HF4AzOLEpvBevC+Btrw2BusQZU5XJ6WJ38/H4mdIZCMHUC9de9m3QVQY0iP+BkVCFYq5SH1rKNlseYzkeuIpRz1S/hG7LUYQzaAbw9DUNtEdTn3wNDWf8qUJlzP1GnhYeZDRG1n2jfyjLe1yjbUhBJo7twuzAXN2M3IFG3ScWBfAJ7+W0rp3UfM4zVjnlKA/+yGsGKJBlGGjh1ZeKaMLFzLrQTdfPOKUdYg5Q/Ahbvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/jvNcSqAoSjgZPWvDVhXor5ok7D0z6pQ9NRd4aZj5tI=;
 b=kuz1NkkdOQf8BQGV6mjrztFNw0zX5BTeKejcw1XJeggOtUnenZazI3R0Gseo3pIgHp2bl8sAqm1bV/HsfxQHzyDl+lBfLMkN3WYh1Ym7YhCfIRgZKQRdcApUIK4qk2z3gO3qohG6xfxF4VTpW2v1L8TY90ThioxDBnr0a06VuZ2juUaexwEId8OiTEhHHBdBgfclNnm6ZKgbqFLqN2Ob7F0OevPDz+3IFyeNwdz5V6sf1O5iOkGdeSo16SJouyAtdgoU1eGDALihuMJ1vsv9CrGfotjM2Q57Y40IL029YxWFg39ugkT+gIYQAxf87a9KoT4BHiEWWUGuVuFGeMrWhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/jvNcSqAoSjgZPWvDVhXor5ok7D0z6pQ9NRd4aZj5tI=;
 b=Y5PuBUbA8NxiKG2zFrvyUadmiGVDAXm5wKsKTskvetTI8iSzLrauJt/tKU/4p1eqZoOz51WyNqfLqPuFrx2AY9yeDnnLTUAhc/q8IEK0i94ClH6kThSe7aHjyIgg85Gb7+8OHuR3Zt6zvnALS8t0TT1SotOeweVzJa1DDYatr2E=
To: =?UTF-8?Q?Alex_Benn=c3=a9e?= <alex.bennee@linaro.org>
CC: Mathieu Poirier <mathieu.poirier@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>, Wei Liu <wl@xen.org>, "rust-vmm@lists.opendev.org"
	<rust-vmm@lists.opendev.org>, <xen-devel@lists.xenproject.org>, Stratos
 Mailing List <stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
	<olekstysh@gmail.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>,
	Doug Goldstein <cardoe@cardoe.com>, Demi Marie Obenour
	<demi@invisiblethingslab.com>, =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
	<marmarek@invisiblethingslab.com>
References: <87lf40vay1.fsf@linaro.org>
 <abfa4f44-8c56-af3f-485e-41b58e790d92@citrix.com> <874kanus97.fsf@linaro.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
Message-ID: <188afb35-54c1-9a52-19f1-867cea4487ea@citrix.com>
Date: Tue, 14 Sep 2021 19:42:34 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <874kanus97.fsf@linaro.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0410.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::14) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 11120564-d092-4927-70d8-08d977af6f96
X-MS-TrafficTypeDiagnostic: BYAPR03MB3431:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3431F3E8A65E70245183E3E7BADA9@BYAPR03MB3431.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: IlNDNrt83RbE/nu4YpUaSfKUHgOXchsqBMSL3V0gYeXPuFBuRAPhoJaGdpVgPe823m+6qB4KyLQXPJnvtgpDHIhtVkKGw64KSVW6mJ5I4k05Y3k5x6y1SDh9pvJzd/Cp8Q0IA1hrszscfuctkOlz+7KbtkQ7WUuSokVanBuhv3NAK5v8yBHj49GP1nPaRe/UxEgogqAseSnOrH62+TEgqS+YABruO3SDk4naWtxnSLV3DYJImUv7OXWN/Gu2aeAdN7b5Wn9Fu6PK9QgyEZNz2mDDvZ42+VbpO80IYsY9aENOupvvKJn59aPz8vexct3VG+oGAM7qy3vpV1OtAXz1l5o6JQOggKv/8g2gsbQqDrhoUbpc8c+m8ylgdbFNvQZuW4pR/Rh4k8H28d5e9pxoukDkV5k52aFuGM5Qsa1ZsU6tjfKo0g3PMuEplU7JGFqaovnGN/cKmkKqbBmp8e/DRwUblCWJQ/8e+29p5a105C325aG4ilaE4Bw8qsNwbJOmlipj2ha1brNbb2BevAI3mN2aabGTXnoVcINAwQ97GmHQi38Xf9rpVH+SXKC/SA0mPVYgJQgAZlzv4wjUiJyBum9gjcVtieFttnkUauejfHDNz2i8ew7O0HlYMMbqzn1nTAKf6xz5rUpIiorJWax4pL44CwPAptvpOMRPqgVIql0EeaF968ke/URRbmQWchKk+xjJJzsOEvjnEmtD7fp3EetkKbz3L1kyGySN23OfgRdgsp2YSE3PmkMbrv7HcJcASa/Q5i7jy9Wv7zYj1YNU5PgaTqaGI7IsaR8gYL5vqrZ/oNjrh7EUz0D8sA2YSpzlP1cysGbJ/581c3CvnrUmvtgZQ4yOnJnDDbJJ2D5PVwM=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(376002)(39860400002)(346002)(396003)(186003)(38100700002)(31686004)(316002)(5660300002)(26005)(36756003)(45080400002)(6916009)(966005)(86362001)(55236004)(66946007)(2906002)(53546011)(8676002)(478600001)(66556008)(54906003)(66476007)(66574015)(16576012)(31696002)(2616005)(956004)(7416002)(6486002)(4326008)(8936002)(6666004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZzVxM2dWdkl4cjhvOHhpcnY1V1NOR0hRVDNoMHJpRkFXanV3b0FYYTV6dEdu?=
 =?utf-8?B?M1ZYZk5WT3IzdlQ2VXRDQVdadk14UlpUdS9HRTZOdCsvNlNIekU5WDhrUWtk?=
 =?utf-8?B?UzhiWElsaVkxaEFjVW8vcVRLMDJzUXRBVXd4QVJxUnYxV2NEMWhWYUZQaTBX?=
 =?utf-8?B?UGJPTmNpMXVhaHZtZG12anVjdDQvcmpROE5yRW1qcGcyWElBMWFpcFpBNjE5?=
 =?utf-8?B?ZGpiNEdOb1JzeEtMOXZNYlBQMExlTjFGVVV5Q0pWZUphUXZsM0JYTWV5UHMy?=
 =?utf-8?B?N0tsOWorTG8zeUc0QU5JSUZKVFNXRzBZTmpDZm9JVUI0bzRybW1XNXFEZXZ4?=
 =?utf-8?B?RnNDQzhWcTFJbkVjUC9veVVnaVg0VzJWdHRGcnB1czE0dktGN3VtTk5KSWZB?=
 =?utf-8?B?aDMxQ1p5U2VoQ05hWGtqdnN1TGxXamNVZXdqc0k0K2RGT2FqSFk4RHBBM0JX?=
 =?utf-8?B?eS96bTE3RVNyUmpWNE9MUnBlMWh6L2hZTzBLb25MUjFHTHRlaHZSdVI4amJT?=
 =?utf-8?B?Q0d3MEVtWjU3OGMxV05SS2R4U1d6MUR2ZGpYWWFMVG5sdVRiOGhlNXoxMGlC?=
 =?utf-8?B?U1Y5aEFFOFRFOUNBcTZsY2MxK0RWWEhzQUJaTW1FN2Jja2IrR0x6N2poZm10?=
 =?utf-8?B?VTRLK2FmZjZHK1NoMEljbTZEaVNjSFNHc2FXT2VEUkZPVVA3MExEVFJZZi9z?=
 =?utf-8?B?SDBqSFQ5Q2QrUVQ3Q1RCVUtWeTExbzExUFpDSlpaWHRLaTdtVk5TUU5NWlcx?=
 =?utf-8?B?V1dJZGFUK0h3eGxnT3NqYnYyUERYaGQrbkxUSUxVVU1jeDc3WDdINFl4ZjU1?=
 =?utf-8?B?LzZtelM0TGNldms4NHRxamY4NUE4anpTTHQ3YnhkZHY0MHNxQVh5SnozVFJ3?=
 =?utf-8?B?WUxSVDI4NXl1Q0NoUDBRcjhhWHNGa05ZaDFmdGc4d0d1U2dKaS8vU1VrL0JE?=
 =?utf-8?B?UmQ2Ym9pR2FYQ1p6c1MyNXZqNnM5RVloNTg0M3EzMy91TCtidjdxSFRVaXVD?=
 =?utf-8?B?U2hVNkY3VVZwM3hFay9VZzk0UFhOY3NRY1JDSnlvSnYyM3pPd093VHBWclA5?=
 =?utf-8?B?WU45Vk95S3JtQzE4alI5cHJCVGJod1ZNYllQMEZFWTZKNHZmU2Fta0g5cmRk?=
 =?utf-8?B?OGQwWVdMM3NLbGVUeEVkdElMUUxEd3dlaVJaYWxmRUl6NVN3OU1YSWc5bHk1?=
 =?utf-8?B?OGVVczlxNFNIb1d2d1kzQWcvQ2txcTJXN0JpRXo4RVlrQmYyUmNpZ2ZpaHRM?=
 =?utf-8?B?TlgwajR5TWU4S25jeHhwMVoyNU9UTlVKMTBUcGtyQng0RU9mQTd0c3B4R21w?=
 =?utf-8?B?TkFYMURFSHdaaFppKzZ3YjluV0F4RW1BT1lybzdscWtYK3Z1SG5MR2ZLY3JR?=
 =?utf-8?B?aHdLOTA4UXo5OXVad0h0YUFpVmxmNllHa1NWZ1ZQOEtxSWlKWnVibjR2SGxn?=
 =?utf-8?B?YjNZS2FUL2JjbVg0eTFSWmtINzRNUXpTbEtpM1NSSVlKcDRFVVdQMk8wVjVL?=
 =?utf-8?B?OENTbXlTcm94RHdxLzdPVFA3Tit0V3JBUHpLbFhlZTBudE1vdm1ncW1QTnFT?=
 =?utf-8?B?ZENWWitQVkpOa0FlUFJYc1FTazQrQkxpYzJ5cVlNNlMwR3ZVNmZRbmFPTDE4?=
 =?utf-8?B?d21NdVhZOHZ4T1JVdmF5TUFsY3dwSkhoVXNDdUV4V2t4YzhwVEdZR0xCNG9E?=
 =?utf-8?B?dG5XOHVpK3ZleXFlVFBWNjlMZ3E5cFA4T3kzeVFlTWdLdWREbjBhL2cwRWcz?=
 =?utf-8?Q?n+GogUBE+m7SDAGBsUEGgJ5Pja6Zdv/MrYSgU9l?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 11120564-d092-4927-70d8-08d977af6f96
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 18:42:42.6357
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AB6/z1f+Xb4EgpgzI4omzWeJ4BF/FSt3hYZfwllzczX/0b9mlsrBD4LpwGLTYWStO77zM78K5fQArNDY7Ivt/F/xYt4AIIyjqK9ANacYQ2U=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3431
X-OriginatorOrg: citrix.com

On 14/09/2021 15:44, Alex Benn=C3=A9e wrote:
> Andrew Cooper <andrew.cooper3@citrix.com> writes:
>
>> On 13/09/2021 13:44, Alex Benn=C3=A9e wrote:
>>> Hi,
>>>
>>> As we consider the next cycle for Project Stratos I would like to make
>>> some more progress on hypervisor agnosticism for our virtio backends.
>>> While we have implemented a number of virtio vhost-user backends using =
C
>>> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
>>> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
>>> backends does it make sense to do some enabling work in rust-vmm to
>>> support Xen?
>>>
>>> There are two chunks of work I can think of:
>>>
>>>   1. Enough of libxl/hypervisor interface to implement an IOREQ end poi=
nt.
>> No libxl here at all.
>>
>> As of Xen 4.15, there are enough stable interfaces to implement simple
>> IOERQ servers.
>>
>> https://nam04.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith=
ub.com%2Fxapi-project%2Fvarstored%2Fcommit%2Ffde707c59f7a189e1d4e97c1a4ee1a=
2d0c378ad1&amp;data=3D04%7C01%7CAndrew.Cooper3%40citrix.com%7C08a3fe14704a4=
d6888cf08d9778ee5b2%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C6376722779=
05441489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi=
I6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3D%2B1pKhIuqzCGkgYD4snd6jnxjoEJzr=
CgUdol%2FfA2kwOk%3D&amp;reserved=3D0
>> was the commit where I removed the final unstable interface from
>> varstored (terrible name) which is a dom0 backend for UEFI secure
>> variable handling.=C2=A0 As such, it also serves as a (not totally simpl=
e)
>> reference of an IOERQ server.
>>
>>
>> There are a few bits and pieces of rust going on within Xen, and a whole
>> load of plans.=C2=A0 Also, there is a lot of interest from other downstr=
eams
>> in being able to write Rust backends.
>>
>> We've got a placeholder xen and xen-sys crates, and placeholder work for
>> supporting cross-compile as x86 PV and PVH stubdomains.
> Are these in the rust-vmm project is elsewhere?

https://crates.io/crates/xen-sys

When I say placeholder, I really do mean placeholder.

To start this work meaningfully, we'd want to make a repo (or several)
in the xen-project organisation on github or gitlab (we have both, for
reasons), and set these as the upstream of the xen and xen-sys crates.

>> The want to have a simple IOREQ server compiled either as a dom0
>> backend, or as a PV or PVH stubdomains influences some of the design
>> decisions early on, but they're all no-brainers for the longevity of the
>> work.
> Just to clarify nomenclature is a PVH stubdomain what I'm referring to
> as a bare metal backend, i.e: a unikernel or RTOS image that implements
> the backend without having to transition between some sort of userspace
> and it's supporting kernel?

I think so, yes, although calling it "bare metal" seems misleading for
something which is a VM targetted at a specific hypervisor...


>> I started work on trying to reimplement varstored entirely in Rust as a
>> hackathon project, although ran out of time trying to make hypercall
>> buffers work (there is a bug with Box and non-global allocators causing
>> rustc to hit an assert().=C2=A0 In the short term, we'll have to impleme=
nt
>> hypercall buffers in a slightly more irritating way).
>>
>> Furthermore, stick to the stable hypercalls only.=C2=A0 Xen's C librarie=
s are
>> disaster for cross-version compatibility, and you absolutely do not want
>> to recompile your rust program just to run it against a different
>> version of the hypervisor.=C2=A0 The plan is to start with simple IOREQ
>> servers, which are on fully stable interfaces, then stabilise further
>> hypercalls as necessary to expand functionality.
> Are the hypercalls mediated by a kernel layer or are you making direct
> HVC calls (on ARM) to the hypervisor from userspace?

For a dom0 backends irrespective of architecture, you need to issue
ioctl()'s on the appropriate kernel device.

For a PV/PVH stubdom, you should make a call into the hypercall_page
https://xenbits.xen.org/docs/latest/guest-guide/x86/hypercall-abi.html
because Intel and AMD used different instructions for their equivalent
of HVC.

ARM doesn't have the hypercall page ABI, so I'd expect the hypercall
implementation to expand to HVC directly.


In terms of rust APIs, we'd want a crate which has target-specific
implementations so the caller need not worry about the implementation
details in the common case.

>
> Where would I look in the Xen code to find the hypercalls that are
> considered stable and won't change between versions?

I'm afraid that's mostly in developers heads right now.

For a first pass, you can look for __XEN_TOOLS__=C2=A0 (This is mis-named,
and ought to be called __XEN_UNSTABLE_INTERFACE__, because...) but be
aware that some things currently tagged __XEN_TOOLS__ are incorrect and
are in fact stable.

As a first pass, assume everything is unstable.=C2=A0 The things contained
within libxendevicemodel and libxenforeignmem are stable and were
specifically made so to try and get simple IOREQ server functionality
done and stable.

Almost everything else, particularly concerning the toolstack
operations, is unstable.=C2=A0 There is 15 years of organic growth and
dubious decisions here, and they need unpicking carefully.=C2=A0 We've got
some hypercalls which look like they're unstable, but are actually
stable (as they were exposed to guests), and therefore have ridiculous
interfaces.

The "ABI v2" work is massive and complicated, and picking at some of the
corners based on "what is needed to make new $FOO work" is a good way to
make some inroads.

>> It's high time the Xen Rust working group (which has been talked about
>> for several years now) actually forms...
> Indeed part of the purpose of this email was to smoke out those who are
> interested in the intersection of Xen, Rust and VirtIO ;-)

The conversation has come up quite a few times in the past, but mostly
by people who are also busy with other things.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 20:44:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 20:44:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187010.335741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQFIF-00068S-HB; Tue, 14 Sep 2021 20:44:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187010.335741; Tue, 14 Sep 2021 20:44:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQFIF-00068L-E7; Tue, 14 Sep 2021 20:44:39 +0000
Received: by outflank-mailman (input) for mailman id 187010;
 Tue, 14 Sep 2021 20:44:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jeVa=OE=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQFIE-00068F-63
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 20:44:38 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21744033-2fad-4d17-8d18-c364a8c09127;
 Tue, 14 Sep 2021 20:44:36 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id x6so257295wrv.13
 for <xen-devel@lists.xenproject.org>; Tue, 14 Sep 2021 13:44:36 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id k6sm993474wmo.37.2021.09.14.13.44.33
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 14 Sep 2021 13:44:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21744033-2fad-4d17-8d18-c364a8c09127
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=GoZcnLD9m4N5gmlIbm/gTIsyYg9LzB8Uf3nV7DR5ekM=;
        b=ZNjP0eQIp2w0o0AZAfIaJg4kkVe8XY4Ywq6flQ5pJQtgwGd0l60WMm1TXeWMEXZ3dH
         tb1jWTijtIBo7CnaHoEIeFEEx2gsDxjlbApoe2e31Os1q7tLN7r1yyhvE5/sZPeJUMBS
         UrxO0wfKOTEMEMvJKNI0CB2lR9GZAZdfgAbSxaG+q4F+Ln+/B6gig9507i/0B/O97cdJ
         aLrqUmwp5Yu3SLYdit5AKzvVfBXcRCBj5YQ8YzQfsuv2/+95Up6jtGuZ+f15RD+8hINh
         Hv4+46IgYQSFsiKjDp9vAcAIXPSBWPVcP1WQAIfOxg5x++f5aMa3gSsHU2MrWaUoTlQh
         g/Yw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=GoZcnLD9m4N5gmlIbm/gTIsyYg9LzB8Uf3nV7DR5ekM=;
        b=lxEGncDmmnD412WJrUOgOr5p9uLiY9yxEImiGLJPw6C2iUCpwHQmb9O6vySzV4puoO
         oU6EZwfOrM4eZ94CMJsF0vcMNSk8d80GTKjWAQI11SH6xUnkaSjEeell/2xVXpeHp+ch
         vwQi1IKzev6IcHYkZdTD1vDLRZrs1YMwwsv/rbCQylb9H4e4/lqIo5R0FAtxQxODHohs
         4INahhn0mj3KMzx6I92rA8vVwKJ4gEr9pP+YWZDn/R0bj0U0i4XNVou6HpPyPbEuTXtv
         U63HN3lSuDhsYj3cnzRc73CiBjtstgJ7B6A6F4Zy6RWAUOeLArscaCtiP5TysBXop/ko
         j+Aw==
X-Gm-Message-State: AOAM531LiU7CBYynJACtRiXnSvEOoyrelp9uhTL/zFROCVFIDIAyamkd
	tJUcOefe+hhlQMBUmgYYFvSIeWzxEc0=
X-Google-Smtp-Source: ABdhPJxvkGIyo//M06CXfKTh9hza52v2LJrPQ4HUdo3pNaZeSW1u9gsSsHN2XCmd62H7kggu4ctS5w==
X-Received: by 2002:adf:c501:: with SMTP id q1mr1160367wrf.150.1631652274788;
        Tue, 14 Sep 2021 13:44:34 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [RFC PATCH V2] xen/gnttab: Store frame GFN in struct page_info on Arm
Date: Tue, 14 Sep 2021 23:44:05 +0300
Message-Id: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Rework Arm implementation to store grant table frame GFN
in struct page_info directly instead of keeping it in
standalone status/shared arrays.

To cover 64-bit/40-bit IPA on Arm64/Arm32 we need the space
to hold 52-bit/28-bit value respectively. In order to not
grow the size of struct page_info borrow the required amount
of bits from type_info's count portion which current context
won't suffer (currently only 1 bit is used on Arm).

Introduce corresponding PGT_* constructs and access macros.
Update existing gnttab macros to deal with GFN value according
to new location. Also update the use of count portion on Arm
in share_xen_page_with_guest().

Update the P2M code to clean said GFN portion when putting
a reference on the grant table page in p2m_put_l3_page().
The added check is based on the assumption that grant table page
is the xen_heap page and its entry has p2m_ram_rw type, which
is correct. However, this check is not entirely precise and we
might end up clearing the GFN portion for other xen_heap pages
with the same p2m_type. But, this action is considered as
harmless, since only grant table pages really use that portion.

And for everything to work correctly introduce arch-specific
macros to be called from alloc_xenheap_pages()/free_xenheap_pages()
which purposes on Arm are to clear the GFN portion before use and
make sure the portion is cleared after use, on x86 these are
just stubs.

This patch is intended to fix the potential issue on Arm
which might happen when remapping grant-table frame.
A guest (or the toolstack) will unmap the grant-table frame
using XENMEM_remove_physmap. This is a generic hypercall,
so on x86, we are relying on the fact the M2P entry will
be cleared on removal. For architecture without the M2P,
the GFN would still be present in the grant frame/status
array. So on the next call to map the page, we will end up to
request the P2M to remove whatever mapping was the given GFN.
This could well be another mapping.

Besides that, this patch simplifies arch code on Arm by
removing arrays and corresponding management code and
as the result gnttab_init_arch/gnttab_destroy_arch helpers
and struct grant_table_arch become useless and can be
dropped globally.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
You can find the related discussions at:
https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
https://lore.kernel.org/xen-devel/1631228688-30347-1-git-send-email-olekstysh@gmail.com/

! Please note, there is still unresolved locking question here for which
I failed to find a suitable solution !

According to the internal conversation:
Now the GFN field in the struct page_info is accessed from
gnttab_set_frame_gfn() in the grant table code and from page_set_frame_gfn()
in the P2M code (the former uses the latter).

We need to prevent the concurrent access to this field. But, we cannot grab
the grant lock from the P2M code because we will introduce a lock inversion.
The page_set_frame_gfn() will be called from the P2M code with the p2m lock held
and then acquire the grant table lock. The gnttab_map_frame() will do the inverse.

Changes RFC1 -> RFC2:
 - update patch description
 - add/update comments in code
 - clarify check in p2m_put_l3_page()
 - introduce arch_alloc_xenheap_page() and arch_free_xenheap_page()
   and drop page_arch_init()
 - add ASSERT to gnttab_shared_page() and gnttab_status_page()
 - rework changes to Arm's struct page_info: do not split type_info,
   allocate GFN portion by reducing count portion, create corresponding
   PGT_* construct, etc
 - update page_get_frame_gfn() and page_set_frame_gfn()
 - update the use of count portion on Arm
 - drop the leading underscore in the macro parameter names
---
 xen/arch/arm/mm.c                 |  8 ++++--
 xen/arch/arm/p2m.c                | 21 ++++++++++++---
 xen/common/grant_table.c          |  9 -------
 xen/common/page_alloc.c           |  6 +++++
 xen/include/asm-arm/grant_table.h | 57 +++++++++++++++------------------------
 xen/include/asm-arm/mm.h          | 36 ++++++++++++++++++++++---
 xen/include/asm-x86/grant_table.h |  5 ----
 xen/include/asm-x86/mm.h          |  4 +++
 8 files changed, 87 insertions(+), 59 deletions(-)

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index eea926d..1857af4 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1376,14 +1376,18 @@ unsigned long domain_get_maximum_gpfn(struct domain *d)
 void share_xen_page_with_guest(struct page_info *page, struct domain *d,
                                enum XENSHARE_flags flags)
 {
+    unsigned long type_info;
+
     if ( page_get_owner(page) == d )
         return;
 
     spin_lock(&d->page_alloc_lock);
 
     /* The incremented type count pins as writable or read-only. */
-    page->u.inuse.type_info =
-        (flags == SHARE_ro ? PGT_none : PGT_writable_page) | 1;
+    type_info = page->u.inuse.type_info & ~(PGT_type_mask | PGT_count_mask);
+    page->u.inuse.type_info = type_info |
+        (flags == SHARE_ro ? PGT_none : PGT_writable_page) |
+        (1UL << PGT_count_base);
 
     page_set_owner(page, d);
     smp_wmb(); /* install valid domain ptr before updating refcnt. */
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index eff9a10..a2b5597 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -718,8 +718,10 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
  * TODO: Handle superpages, for now we only take special references for leaf
  * pages (specifically foreign ones, which can't be super mapped today).
  */
-static void p2m_put_l3_page(const lpae_t pte)
+static void p2m_put_l3_page(struct p2m_domain *p2m, const lpae_t pte)
 {
+    mfn_t mfn = lpae_get_mfn(pte);
+
     ASSERT(p2m_is_valid(pte));
 
     /*
@@ -731,11 +733,22 @@ static void p2m_put_l3_page(const lpae_t pte)
      */
     if ( p2m_is_foreign(pte.p2m.type) )
     {
-        mfn_t mfn = lpae_get_mfn(pte);
-
         ASSERT(mfn_valid(mfn));
         put_page(mfn_to_page(mfn));
     }
+
+#ifdef CONFIG_GRANT_TABLE
+    /*
+     * Check whether we deal with grant table page. As the grant table page
+     * is xen_heap page and its entry has known p2m type, detect it and mark
+     * the stored GFN as invalid. Although this check is not precise and we
+     * might end up updating this for other xen_heap pages, this action is
+     * harmless to these pages since only grant table pages have this field
+     * in use. So, at worst, unnecessary action might be performed.
+     */
+    if ( (pte.p2m.type == p2m_ram_rw) && is_xen_heap_mfn(mfn) )
+        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
+#endif
 }
 
 /* Free lpae sub-tree behind an entry */
@@ -768,7 +781,7 @@ static void p2m_free_entry(struct p2m_domain *p2m,
         p2m->stats.mappings[level]--;
         /* Nothing to do if the entry is a super-page. */
         if ( level == 3 )
-            p2m_put_l3_page(entry);
+            p2m_put_l3_page(p2m, entry);
         return;
     }
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index e80f8d0..fd8d7e3 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -93,8 +93,6 @@ struct grant_table {
 
     /* Domain to which this struct grant_table belongs. */
     const struct domain *domain;
-
-    struct grant_table_arch arch;
 };
 
 unsigned int __read_mostly opt_max_grant_frames = 64;
@@ -1981,14 +1979,9 @@ int grant_table_init(struct domain *d, int max_grant_frames,
 
     grant_write_lock(gt);
 
-    ret = gnttab_init_arch(gt);
-    if ( ret )
-        goto unlock;
-
     /* gnttab_grow_table() allocates a min number of frames, so 0 is okay. */
     ret = gnttab_grow_table(d, 0);
 
- unlock:
     grant_write_unlock(gt);
 
  out:
@@ -3894,8 +3887,6 @@ grant_table_destroy(
     if ( t == NULL )
         return;
 
-    gnttab_destroy_arch(t);
-
     for ( i = 0; i < nr_grant_frames(t); i++ )
         free_xenheap_page(t->shared_raw[i]);
     xfree(t->shared_raw);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 958ba0c..00371e7 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2204,7 +2204,10 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
         return NULL;
 
     for ( i = 0; i < (1u << order); i++ )
+    {
         pg[i].count_info |= PGC_xen_heap;
+        arch_alloc_xenheap_page(&pg[i]);
+    }
 
     return page_to_virt(pg);
 }
@@ -2222,7 +2225,10 @@ void free_xenheap_pages(void *v, unsigned int order)
     pg = virt_to_page(v);
 
     for ( i = 0; i < (1u << order); i++ )
+    {
         pg[i].count_info &= ~PGC_xen_heap;
+        arch_free_xenheap_page(&pg[i]);
+    }
 
     free_heap_pages(pg, order, true);
 }
diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h
index 0ce77f9..479339d 100644
--- a/xen/include/asm-arm/grant_table.h
+++ b/xen/include/asm-arm/grant_table.h
@@ -11,11 +11,6 @@
 #define INITIAL_NR_GRANT_FRAMES 1U
 #define GNTTAB_MAX_VERSION 1
 
-struct grant_table_arch {
-    gfn_t *shared_gfn;
-    gfn_t *status_gfn;
-};
-
 static inline void gnttab_clear_flags(struct domain *d,
                                       unsigned int mask, uint16_t *addr)
 {
@@ -46,35 +41,11 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
 #define gnttab_dom0_frames()                                             \
     min_t(unsigned int, opt_max_grant_frames, PFN_DOWN(_etext - _stext))
 
-#define gnttab_init_arch(gt)                                             \
-({                                                                       \
-    unsigned int ngf_ = (gt)->max_grant_frames;                          \
-    unsigned int nsf_ = grant_to_status_frames(ngf_);                    \
-                                                                         \
-    (gt)->arch.shared_gfn = xmalloc_array(gfn_t, ngf_);                  \
-    (gt)->arch.status_gfn = xmalloc_array(gfn_t, nsf_);                  \
-    if ( (gt)->arch.shared_gfn && (gt)->arch.status_gfn )                \
-    {                                                                    \
-        while ( ngf_-- )                                                 \
-            (gt)->arch.shared_gfn[ngf_] = INVALID_GFN;                   \
-        while ( nsf_-- )                                                 \
-            (gt)->arch.status_gfn[nsf_] = INVALID_GFN;                   \
-    }                                                                    \
-    else                                                                 \
-        gnttab_destroy_arch(gt);                                         \
-    (gt)->arch.shared_gfn ? 0 : -ENOMEM;                                 \
-})
-
-#define gnttab_destroy_arch(gt)                                          \
-    do {                                                                 \
-        XFREE((gt)->arch.shared_gfn);                                    \
-        XFREE((gt)->arch.status_gfn);                                    \
-    } while ( 0 )
-
 #define gnttab_set_frame_gfn(gt, st, idx, gfn)                           \
     do {                                                                 \
-        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =    \
-            (gfn);                                                       \
+        struct page_info *pg_ = (st) ? gnttab_status_page(gt, idx)       \
+                                     : gnttab_shared_page(gt, idx);      \
+        page_set_frame_gfn(pg_, gfn);                                    \
     } while ( 0 )
 
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
@@ -82,11 +53,25 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
         : gnttab_shared_gfn(NULL, gt, idx);                              \
 })
 
-#define gnttab_shared_gfn(d, t, i)                                       \
-    (((i) >= nr_grant_frames(t)) ? INVALID_GFN : (t)->arch.shared_gfn[i])
+#define gnttab_shared_page(t, i) ({                                      \
+    ASSERT((t)->shared_raw[i]);                                          \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->shared_raw[i])));                \
+})
+
+#define gnttab_status_page(t, i) ({                                      \
+    ASSERT((t)->status[i]);                                              \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->status[i])));                    \
+})
 
-#define gnttab_status_gfn(d, t, i)                                       \
-    (((i) >= nr_status_frames(t)) ? INVALID_GFN : (t)->arch.status_gfn[i])
+#define gnttab_shared_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_shared_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
+
+#define gnttab_status_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_status_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
 
 #define gnttab_need_iommu_mapping(d)                    \
     (is_domain_direct_mapped(d) && is_iommu_enabled(d))
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index ded74d2..dd425d6 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -98,9 +98,18 @@ struct page_info
 #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
 #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
 
- /* Count of uses of this frame as its current type. */
-#define PGT_count_width   PG_shift(2)
-#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
+ /* 3-bit count of uses of this frame as its current type. */
+#define PGT_count_base    PG_shift(4)
+#define PGT_count_mask    PG_mask(7, 4)
+
+/*
+ * Stored in bits [27:0] or [59:0] GFN if page is used for grant table frame.
+ * This only valid for the xenheap pages.
+ */
+#define PGT_gfn_width     PG_shift(4)
+#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
+
+#define PGT_INVALID_FRAME_GFN   _gfn(PGT_gfn_mask)
 
  /* Cleared when the owning guest 'frees' this page. */
 #define _PGC_allocated    PG_shift(1)
@@ -163,6 +172,27 @@ extern unsigned long xenheap_base_pdx;
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+#define page_get_frame_gfn(p) ({                                \
+    gfn_t gfn_ = _gfn((p)->u.inuse.type_info & PGT_gfn_mask);   \
+    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
+})
+
+#define page_set_frame_gfn(p, gfn) ({                           \
+    gfn_t gfn_ = gfn_eq(gfn, INVALID_GFN) ?                     \
+                 PGT_INVALID_FRAME_GFN : gfn;                   \
+    (p)->u.inuse.type_info &= ~PGT_gfn_mask;                    \
+    (p)->u.inuse.type_info |= gfn_x(gfn_);                      \
+})
+
+/*
+ * As the struct page_info representing the xen_heap page can contain
+ * the grant table frame GFN on Arm we need to clear it beforehand and
+ * make sure it is not still set when freeing a page.
+ */
+#define arch_alloc_xenheap_page(p)   page_set_frame_gfn(p, INVALID_GFN)
+#define arch_free_xenheap_page(p) \
+    BUG_ON(!gfn_eq(page_get_frame_gfn(p), INVALID_GFN))
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 /* PDX of the first page in the frame table. */
 extern unsigned long frametable_base_pdx;
diff --git a/xen/include/asm-x86/grant_table.h b/xen/include/asm-x86/grant_table.h
index 84e3296..0eb018f 100644
--- a/xen/include/asm-x86/grant_table.h
+++ b/xen/include/asm-x86/grant_table.h
@@ -14,9 +14,6 @@
 
 #define INITIAL_NR_GRANT_FRAMES 1U
 
-struct grant_table_arch {
-};
-
 static inline int create_grant_host_mapping(uint64_t addr, mfn_t frame,
                                             unsigned int flags,
                                             unsigned int cache_flags)
@@ -35,8 +32,6 @@ static inline int replace_grant_host_mapping(uint64_t addr, mfn_t frame,
     return replace_grant_pv_mapping(addr, frame, new_addr, flags);
 }
 
-#define gnttab_init_arch(gt) 0
-#define gnttab_destroy_arch(gt) do {} while ( 0 )
 #define gnttab_set_frame_gfn(gt, st, idx, gfn) do {} while ( 0 )
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
     mfn_t mfn_ = (st) ? gnttab_status_mfn(gt, idx)                       \
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index cb90527..04d8704 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -327,6 +327,10 @@ struct page_info
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+/* No arch-specific actions are needed for the xen_heap page */
+#define arch_alloc_xenheap_page(p)   do {} while ( 0 )
+#define arch_free_xenheap_page(p)    do {} while ( 0 )
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 extern unsigned long max_page;
 extern unsigned long total_pages;
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 21:18:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 21:18:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187020.335753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQFoX-0001Hc-6z; Tue, 14 Sep 2021 21:18:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187020.335753; Tue, 14 Sep 2021 21:18:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQFoX-0001HV-3T; Tue, 14 Sep 2021 21:18:01 +0000
Received: by outflank-mailman (input) for mailman id 187020;
 Tue, 14 Sep 2021 21:18:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L61H=OE=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQFoW-0001HP-0w
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 21:18:00 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3c227a00-15a1-11ec-b478-12813bfff9fa;
 Tue, 14 Sep 2021 21:17:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8CC7E61183;
 Tue, 14 Sep 2021 21:17:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c227a00-15a1-11ec-b478-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631654278;
	bh=UQRLGHS6xf6GmOuTj/HQn8nm9cEbre+JvMXAL0ON+oY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HZSjZbN2ZZyyvB7ePeAYwklJBux399JsFKsdhJN6j2v2JC9tz+8KXhn961n9I8fSI
	 7E9I+Foi8FmQ4o82NUpOzS4Fqc72qqIh8xONnfjXyWx473ACwH84NRLvD+CbJit4g6
	 pNXdJITIWK2uv/cNSialnHsfb0c8dTtu7u6ICDalqMGNnnZr9lsJquN2VASCvBatVz
	 gRakVWj6/RKpAdwWKs8brlUhlvGnuBlyovgsWRQA5SbHmlxaqYmKvM9ux7rin4LemX
	 Ys3AbNle8xhhEsHNERbB2a63Y1Cw1cGJrXKozndqum8qDHwkr44bma4eOOPcE/9kG6
	 1Vidzltb1/v3w==
Date: Tue, 14 Sep 2021 14:17:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Andrew Cooper <andrew.cooper3@citrix.com>
cc: =?UTF-8?Q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>, 
    Mathieu Poirier <mathieu.poirier@linaro.org>, 
    Viresh Kumar <viresh.kumar@linaro.org>, Wei Liu <wl@xen.org>, 
    "rust-vmm@lists.opendev.org" <rust-vmm@lists.opendev.org>, 
    xen-devel@lists.xenproject.org, 
    Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
    Oleksandr Tyshchenko <olekstysh@gmail.com>, 
    Stefano Stabellini <stefano.stabellini@xilinx.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Demi Marie Obenour <demi@invisiblethingslab.com>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
In-Reply-To: <188afb35-54c1-9a52-19f1-867cea4487ea@citrix.com>
Message-ID: <alpine.DEB.2.21.2109141416160.21985@sstabellini-ThinkPad-T480s>
References: <87lf40vay1.fsf@linaro.org> <abfa4f44-8c56-af3f-485e-41b58e790d92@citrix.com> <874kanus97.fsf@linaro.org> <188afb35-54c1-9a52-19f1-867cea4487ea@citrix.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-634525-1631654278=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-634525-1631654278=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 14 Sep 2021, Andrew Cooper wrote:
> On 14/09/2021 15:44, Alex Bennée wrote:
> > Andrew Cooper <andrew.cooper3@citrix.com> writes:
> >
> >> On 13/09/2021 13:44, Alex Bennée wrote:
> >>> Hi,
> >>>
> >>> As we consider the next cycle for Project Stratos I would like to make
> >>> some more progress on hypervisor agnosticism for our virtio backends.
> >>> While we have implemented a number of virtio vhost-user backends using C
> >>> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
> >>> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
> >>> backends does it make sense to do some enabling work in rust-vmm to
> >>> support Xen?
> >>>
> >>> There are two chunks of work I can think of:
> >>>
> >>>   1. Enough of libxl/hypervisor interface to implement an IOREQ end point.
> >> No libxl here at all.
> >>
> >> As of Xen 4.15, there are enough stable interfaces to implement simple
> >> IOERQ servers.
> >>
> >> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fxapi-project%2Fvarstored%2Fcommit%2Ffde707c59f7a189e1d4e97c1a4ee1a2d0c378ad1&amp;data=04%7C01%7CAndrew.Cooper3%40citrix.com%7C08a3fe14704a4d6888cf08d9778ee5b2%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637672277905441489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2B1pKhIuqzCGkgYD4snd6jnxjoEJzrCgUdol%2FfA2kwOk%3D&amp;reserved=0
> >> was the commit where I removed the final unstable interface from
> >> varstored (terrible name) which is a dom0 backend for UEFI secure
> >> variable handling.  As such, it also serves as a (not totally simple)
> >> reference of an IOERQ server.
> >>
> >>
> >> There are a few bits and pieces of rust going on within Xen, and a whole
> >> load of plans.  Also, there is a lot of interest from other downstreams
> >> in being able to write Rust backends.
> >>
> >> We've got a placeholder xen and xen-sys crates, and placeholder work for
> >> supporting cross-compile as x86 PV and PVH stubdomains.
> > Are these in the rust-vmm project is elsewhere?
> 
> https://crates.io/crates/xen-sys
> 
> When I say placeholder, I really do mean placeholder.
> 
> To start this work meaningfully, we'd want to make a repo (or several)
> in the xen-project organisation on github or gitlab (we have both, for
> reasons), and set these as the upstream of the xen and xen-sys crates.
> 
> >> The want to have a simple IOREQ server compiled either as a dom0
> >> backend, or as a PV or PVH stubdomains influences some of the design
> >> decisions early on, but they're all no-brainers for the longevity of the
> >> work.
> > Just to clarify nomenclature is a PVH stubdomain what I'm referring to
> > as a bare metal backend, i.e: a unikernel or RTOS image that implements
> > the backend without having to transition between some sort of userspace
> > and it's supporting kernel?
> 
> I think so, yes, although calling it "bare metal" seems misleading for
> something which is a VM targetted at a specific hypervisor...
> 
> 
> >> I started work on trying to reimplement varstored entirely in Rust as a
> >> hackathon project, although ran out of time trying to make hypercall
> >> buffers work (there is a bug with Box and non-global allocators causing
> >> rustc to hit an assert().  In the short term, we'll have to implement
> >> hypercall buffers in a slightly more irritating way).
> >>
> >> Furthermore, stick to the stable hypercalls only.  Xen's C libraries are
> >> disaster for cross-version compatibility, and you absolutely do not want
> >> to recompile your rust program just to run it against a different
> >> version of the hypervisor.  The plan is to start with simple IOREQ
> >> servers, which are on fully stable interfaces, then stabilise further
> >> hypercalls as necessary to expand functionality.
> > Are the hypercalls mediated by a kernel layer or are you making direct
> > HVC calls (on ARM) to the hypervisor from userspace?
> 
> For a dom0 backends irrespective of architecture, you need to issue
> ioctl()'s on the appropriate kernel device.
> 
> For a PV/PVH stubdom, you should make a call into the hypercall_page
> https://xenbits.xen.org/docs/latest/guest-guide/x86/hypercall-abi.html
> because Intel and AMD used different instructions for their equivalent
> of HVC.
> 
> ARM doesn't have the hypercall page ABI, so I'd expect the hypercall
> implementation to expand to HVC directly.

See for example arch/arm64/xen/hypercall.S in Linux
--8323329-634525-1631654278=:21985--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 22:00:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 22:00:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187032.335764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQGT1-0005ee-G6; Tue, 14 Sep 2021 21:59:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187032.335764; Tue, 14 Sep 2021 21:59:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQGT1-0005eX-DB; Tue, 14 Sep 2021 21:59:51 +0000
Received: by outflank-mailman (input) for mailman id 187032;
 Tue, 14 Sep 2021 21:59:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H4Zq=OE=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mQGSz-0005eR-Hw
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 21:59:49 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 13eefe68-15a7-11ec-b47c-12813bfff9fa;
 Tue, 14 Sep 2021 21:59:48 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A4A70220D6;
 Tue, 14 Sep 2021 21:59:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 556AB13B49;
 Tue, 14 Sep 2021 21:59:47 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id A3ynEVMbQWErRQAAMHmgww
 (envelope-from <dfaggioli@suse.com>); Tue, 14 Sep 2021 21:59:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13eefe68-15a7-11ec-b47c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631656787; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=DT2mo4YQXCWOZfAQDx5c+qpw2Ym6EeOsdPLQJRc1VzY=;
	b=hXZBP/SmVpzFNB8Uxt3WoFP2AMF7Nt+PLzYKQ2qsefBF/yx1/7LFLCLIvVNVWH7xIQKFSp
	/szUW80UpMLTg2vZ+si3PbEpgRqTvagBDIfKIoYoXvpW+AxG+Z9p5SdJYPLYhkVG69rhNm
	Mas2DRWCCTUpuQXKU0pIUonoIRWBk6E=
Message-ID: <3ec7f4bc4859f18aa6514634640ef343e21d9f08.camel@suse.com>
Subject: Re: [PATCH v2] xen/sched: fix sched_move_domain() for domain
 without vcpus
From: Dario Faggioli <dfaggioli@suse.com>
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>, Bertrand Marquis
	 <bertrand.marquis@arm.com>
Date: Tue, 14 Sep 2021 23:59:46 +0200
In-Reply-To: <20210908130440.24605-1-jgross@suse.com>
References: <20210908130440.24605-1-jgross@suse.com>
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-3wODEDxs4cxVYALalTL0"
User-Agent: Evolution 3.40.3 (by Flathub.org) 
MIME-Version: 1.0


--=-3wODEDxs4cxVYALalTL0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 2021-09-08 at 15:04 +0200, Juergen Gross wrote:
> In case a domain is created with a cpupool other than Pool-0
> specified
> it will be moved to that cpupool before any vcpus are allocated.
>=20
> This will lead to a NULL pointer dereference in sched_move_domain().
>=20
> Fix that by tolerating vcpus not being allocated yet.
>=20
> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between
> cpupools with different granularity")
> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-3wODEDxs4cxVYALalTL0
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFBG1IACgkQFkJ4iaW4
c+7d4BAAzaro6gaPS24vkVSea4BZAp8J0v3MUm5CkdKJJ8yyYAIoaawwRKtYeziN
RGDWfp1fP5dLwwohmZAACdxsfidid7qVhmOQxtwId1McMpdwsL6OjEveEvHT8rwQ
KXz2pBTopQk7qiErSkVvUCMxvvN5fg9AIjNrz9+8Pxc7LAMu6NtLS/V6yE8xhh+W
CLsvZrDqU/34z9Bm0hm1SgNxROxrYb2Kt7VKUbqz6peic6/rTkd1LctS0G613BZ5
/L+DTQJ+4ILyjfReGNMyCg4HzbXgpVFfs3A3fock/2z8hH/gOybsnZUxJf7IkwuZ
bVceVxSnEJqPLRzK+dOmnsKf5gfXrYMlOwVdu954TEq/yOzEuDPdmv4kAWoB0wqn
ohDbTm13RF+YZUTPMxEpdXIY3JKEECUGIP31tQVs0CGXp9IYHeNJAxCu1WA3iGwJ
Pahr9qQEZ2fKQ++ER7eABR7hS9VEr7An9qq2pVo3S7qKia/k5l+daQ0KGPYgonFY
Y6Ngs5CrAMUcfFzHYrdqCpSux8k1DYSsXAxg+3+d02v7gHYRVbrNmqvLkE6+2qOL
DFGtUjf/12+tYn6ecHcPD+hCg/QoeaF7hKd9MLg+CgorQhe8UNLIT77M/NssZ5YA
kTEZ0lXhQfUuDq3RfMKnH3tRHEwufFTDZ2m/RYBXtfA7Lew3fy0=
=qpA8
-----END PGP SIGNATURE-----

--=-3wODEDxs4cxVYALalTL0--



From xen-devel-bounces@lists.xenproject.org Tue Sep 14 22:40:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 22:40:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187043.335775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQH6f-0002O7-Qd; Tue, 14 Sep 2021 22:40:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187043.335775; Tue, 14 Sep 2021 22:40:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQH6f-0002O0-NH; Tue, 14 Sep 2021 22:40:49 +0000
Received: by outflank-mailman (input) for mailman id 187043;
 Tue, 14 Sep 2021 22:40:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L61H=OE=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQH6d-0002Nu-Md
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 22:40:47 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cd2ce9ee-15ac-11ec-b47c-12813bfff9fa;
 Tue, 14 Sep 2021 22:40:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8FD7C6115B;
 Tue, 14 Sep 2021 22:40:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd2ce9ee-15ac-11ec-b47c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631659245;
	bh=XLwkarAxi2F06UZdHeqR6DMKqo7ziOd96BMK/KLTjBE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UlRmuMm1l6iR4HPyhywQgOXy+jw+NcKRu4FCU+e4zSjk59oh/9RS0kAE04pUHWHRI
	 kko3/dgCGDOmwwNxg/Ur0ghJV+VvQ1o1OCnSbbRi8gQbqiFnE1spoLwjDDTsYrm3nf
	 eNfYTzcO5twYhDTorTWjTRVXxx1z5F+96oBzxJ0YGmqfzWJomEQAAazw2ICDcrp1IX
	 GNcwuvBgT1QbDXsMnnLxGGivwUJcsfx0JDUHJBzfYpdDaqVCL54VxKcyD0Tg8OL3tY
	 JZEKiIHQTcqYBpzFeb742UnT757aJ7B8la8WX/l3qNd6zfg72ECViMmNDYkQPp8Ss3
	 Jomd/QbNZnktw==
Date: Tue, 14 Sep 2021 15:40:44 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, jbeulich@suse.com
Subject: Re: [PATCH v1 08/14] xen:arm: Implement pci access functions
In-Reply-To: <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
Message-ID: <alpine.DEB.2.21.2109141522380.21985@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s> <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-480397342-1631658371=:21985"
Content-ID: <alpine.DEB.2.21.2109141537470.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-480397342-1631658371=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109141537471.21985@sstabellini-ThinkPad-T480s>

+Jan for the header include question at the bottom


On Tue, 14 Sep 2021, Rahul Singh wrote:
> Hi Stefano,
> 
> > On 10 Sep 2021, at 12:41 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 19 Aug 2021, Rahul Singh wrote:
> >> Implement generic pci access functions to read/write the configuration
> >> space.
> >> 
> >> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> >> ---
> >> xen/arch/arm/pci/pci-access.c      | 31 +++++++++++++++++++++++++++++-
> >> xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++++++++++
> >> xen/include/asm-arm/pci.h          |  2 ++
> >> 3 files changed, 51 insertions(+), 1 deletion(-)
> >> 
> >> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> >> index f39f6a3a38..b94de3c3ac 100644
> >> --- a/xen/arch/arm/pci/pci-access.c
> >> +++ b/xen/arch/arm/pci/pci-access.c
> >> @@ -72,12 +72,41 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
> >>                                 unsigned int len)
> >> {
> >> -    return ~0U;
> >> +    uint32_t val = GENMASK(0, len * 8);
> > 
> > This seems to be another default error value that it would be better to
> > define with its own macro
> 
> This default error is used once do you want to me define as macro.  

Yes. A macro is good even if you are going to use it once because it
also serves as documentation for the error. For instance:

/* PCI host bridge not found */
#define PCI_ERR_NOTFOUND(len) GENMASK(0, len * 8)

really helps with the explanation of what the error is about.


> >> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> >> +
> >> +    if ( unlikely(!bridge) )
> >> +    {
> >> +        printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n",
> >> +                sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn);
> > 
> > You are not actually printing sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn ?
> 
> Yes I am printing with “PRI_pci".

Sorry I missed it!


> >> +        return val;
> >> +    }
> >> +
> >> +    if ( unlikely(!bridge->ops->read) )
> >> +        return val;
> >> +
> >> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);
> > 
> > Would it make sense to make the interface take a pci_sbdf_t directly
> > instead of casting to uint32_t and back?
> 
> pci_sbdf_t is defined in "xen/pci.h” and "xen/pci.h” includes "asm-arm/pci.h”. 
> If I modify the function argument in "asm-arm/pci.h” to use pci_sbdf_t  I will get compilation error.

This is unfortunate. One way around it is to make the appended change to
xen/include/xen/pci.h and then simply add:

typedef union pci_sbdf pci_sbdf_t;

to xen/include/asm-arm/pci.h.

Jan do you have any better suggestions?


diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 8e3d4d9454..ae8d48135b 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -41,7 +41,7 @@
 #define PCI_SBDF3(s,b,df) \
     ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
 
-typedef union {
+union pci_sbdf {
     uint32_t sbdf;
     struct {
         union {
@@ -60,7 +60,9 @@ typedef union {
         };
         uint16_t                seg;
     };
-} pci_sbdf_t;
+};
+
+typedef union pci_sbdf pci_sbdf_t;
 
 struct pci_dev_info {
     /*
--8323329-480397342-1631658371=:21985--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 23:07:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 23:07:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187053.335786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQHVp-0004xM-SS; Tue, 14 Sep 2021 23:06:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187053.335786; Tue, 14 Sep 2021 23:06:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQHVp-0004xF-PR; Tue, 14 Sep 2021 23:06:49 +0000
Received: by outflank-mailman (input) for mailman id 187053;
 Tue, 14 Sep 2021 23:06:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L61H=OE=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQHVp-0004x9-16
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 23:06:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5181069a-aead-4e99-8b32-975d834425c8;
 Tue, 14 Sep 2021 23:06:47 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D6AA603E5;
 Tue, 14 Sep 2021 23:06:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5181069a-aead-4e99-8b32-975d834425c8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631660807;
	bh=v7cDB2XwLuvGLjxlVPf6/TIa+X5CHj9IPBciXatkZuY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=M3Q5Fuj50l+1Tg0pyP9WaogOkrYtn6TEZxC4N0zinGNIRVOzLNKjgoBWUmipDrRrl
	 LHjKFBLjzSomZh+7+05sycP0tNUMZMS5105LSBWQPjD/63EG5N7Vi74qLcpv3H1uOl
	 0ARzqoWb1YCsygGfV1koScmgxCIOs3BVp4cBBuOxFjxHGduvakm4I4vNtZ1xodV7jQ
	 eHsOvR3WKXJUUSkOkrs2lDJ64JYQsKEYm8lmOo4pnWgesQVTQoZ81JvWMEflbtFi8h
	 khNF39KpmZ3KXnqFlgTWmsQ8vSssmtgE9i89gJHQSqYuOGGz3MV/42TIskdXvKFgqP
	 mxapdtiZHh4Ow==
Date: Tue, 14 Sep 2021 16:06:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
In-Reply-To: <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
Message-ID: <alpine.DEB.2.21.2109141543090.21985@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s> <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1558274019-1631659690=:21985"
Content-ID: <alpine.DEB.2.21.2109141549020.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1558274019-1631659690=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109141549021.21985@sstabellini-ThinkPad-T480s>

On Tue, 14 Sep 2021, Rahul Singh wrote:
> >> +        return NULL;
> >> +
> >> +    busn -= cfg->busn_start;
> >> +    base = cfg->win + (busn << cfg->ops->bus_shift);
> >> +
> >> +    return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
> >> +}
> > 
> > I understand that the arm32 part is not implemented and not part of this
> > series, that's fine. However if the plan is that arm32 will dynamically
> > map each bus individually, then I imagine this function will have an
> > ioremap in the arm32 version. Which means that we also need an
> > unmap_bus call in struct pci_ops. I understand that pci_ecam_unmap_bus
> > would be a NOP today for arm64, but I think it makes sense to have it if
> > we want the API to be generic.
> 
> As per my understanding we don’t need pci_ecam_unmap_bus(..) as I don’t see any use case to unmap the 
> bus dynamically. We can add the support for per_bus_mapping for ARM32 once we implement arm32 part.
> Let me know your view on this.

In the patch titled "xen/arm: PCI host bridge discovery within XEN on
ARM" there is the following in-code comment:

* On 64-bit systems, we do a single ioremap for the whole config space
* since we have enough virtual address range available.  On 32-bit, we
* ioremap the config space for each bus individually.
*
* As of now only 64-bit is supported 32-bit is not supported.


So I take it that on arm32 we don't have enough virtual address range
available, therefore we cannot ioremap the whole range. Instead, we'll
have to ioremap the config space of each bus individually.

I assumed that the idea was to call ioremap and iounmap dynamically,
otherwise the total amount of virtual address range required would still
be the same. I also thought that the dynamic mapping function, the one
which would end up calling ioremap on arm32, would be pci_ecam_map_bus.
If so, then we are missing the corresponding unmapping function, the one
that would call iounmap on arm32 and do nothing on arm64, called before
returning from pci_generic_config_read and pci_generic_config_write.

As always, I am not asking for the arm32 implementation, but if we are
introducing internal APIs, it would be good that they are consistent.



> >> @@ -50,10 +51,41 @@ struct pci_host_bridge {
> >>     u8 bus_start;                    /* Bus start of this bridge. */
> >>     u8 bus_end;                      /* Bus end of this bridge. */
> >>     void *sysdata;                   /* Pointer to the config space window*/
> >> +    const struct pci_ops *ops;
> >> };
> >> 
> >> +struct pci_ops {
> >> +    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> +                             uint32_t offset);
> >> +    int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> +                uint32_t reg, uint32_t len, uint32_t *value);
> >> +    int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> +                 uint32_t reg, uint32_t len, uint32_t value);
> >> +};
> >> +
> >> +/*
> >> + * struct to hold pci ops and bus shift of the config window
> >> + * for a PCI controller.
> >> + */
> >> +struct pci_ecam_ops {
> >> +    unsigned int            bus_shift;
> >> +    struct pci_ops          pci_ops;
> >> +    int (*init)(struct pci_config_window *);
> > 
> > Is this last member of the struct needed/used?
> 
> Yes It will be used by some vendor specific board( N1SDP ). Please check below.
> https://git.linaro.org/landing-teams/working/arm/n1sdp-pcie-quirk.git/commit/?id=04b7e76d0fe6481a803f58e54e008a1489d713a5

OK. I don't doubt that there might be bridge-specific initializations,
but we already have things like:

DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
.dt_match = gen_pci_dt_match,
.init = gen_pci_dt_init,
DT_DEVICE_END

The question is do we actually need two different vendor-specific init
functions? One is driven by device tree, e.g. ZynqMP is calling
gen_pci_dt_init. The other one is pci_ecam_ops->init, which is called
from the following chain:

DT_DEVICE_START -> gen_pci_dt_init -> pci_host_common_probe ->
gen_pci_init -> pci_generic_ecam_ops.init

What's the difference between gen_pci_dt_init and pci_ecam_ops.init in
terms of purpose?

I am happy to have pci_ecam_ops.init if it serves a different purpose
from DT_DEVICE_START.init. In that case we might want to add an in-code
comment so that an engineer would know where to add vendor-specific code
(whether to DT_DEVICE_START.init or to pci_ecam_ops.init).
--8323329-1558274019-1631659690=:21985--


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 23:17:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 23:17:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187066.335797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQHgT-0006cQ-TY; Tue, 14 Sep 2021 23:17:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187066.335797; Tue, 14 Sep 2021 23:17:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQHgT-0006cJ-QF; Tue, 14 Sep 2021 23:17:49 +0000
Received: by outflank-mailman (input) for mailman id 187066;
 Tue, 14 Sep 2021 23:17:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pzMZ=OE=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mQHgS-0006cD-6s
 for xen-devel@lists.xenproject.org; Tue, 14 Sep 2021 23:17:48 +0000
Received: from hera.aquilenet.fr (unknown [2a0c:e300::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c39529d0-a703-408e-911a-c82fd71502a0;
 Tue, 14 Sep 2021 23:17:45 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by hera.aquilenet.fr (Postfix) with ESMTP id ED04127B;
 Wed, 15 Sep 2021 01:17:43 +0200 (CEST)
Received: from hera.aquilenet.fr ([127.0.0.1])
 by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 3XfStTJK8Dom; Wed, 15 Sep 2021 01:17:43 +0200 (CEST)
Received: from begin (unknown [IPv6:2a01:cb19:956:1b00:de41:a9ff:fe47:ec49])
 by hera.aquilenet.fr (Postfix) with ESMTPSA id 537014A;
 Wed, 15 Sep 2021 01:17:43 +0200 (CEST)
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mQHgM-00DvMY-DX;
 Wed, 15 Sep 2021 01:17:42 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c39529d0-a703-408e-911a-c82fd71502a0
X-Virus-Scanned: Debian amavisd-new at aquilenet.fr
Date: Wed, 15 Sep 2021 01:17:42 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] mini-os: xenbus: support large messages
Message-ID: <20210914231742.pxpac2lhp2kk5ook@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210818152610.6114-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)
X-Spamd-Bar: --
Authentication-Results: hera.aquilenet.fr
X-Rspamd-Server: hera
X-Rspamd-Queue-Id: ED04127B
X-Spamd-Result: default: False [-2.50 / 15.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 HAS_ORG_HEADER(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 RCVD_NO_TLS_LAST(0.10)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MID_RHS_NOT_FQDN(0.50)[];
	 BAYES_HAM(-3.00)[100.00%]

Hello,

Thanks for having worked on this!

Juergen Gross, le mer. 18 août 2021 17:26:10 +0200, a ecrit:
> +static void xenbus_read_data(char *buf, unsigned int len)
> +{
> +    unsigned int off = 0;
> +    unsigned int prod;
> +    unsigned int size;
> +    int notify;
> +
> +    while (off != len)
> +    {
> +        if (xenstore_buf->rsp_prod == xenstore_buf->rsp_cons)
> +            wait_event(xb_waitq,
> +                       xenstore_buf->rsp_prod != xenstore_buf->rsp_cons);

The if is redundant since wait_event already tests it.

> +        prod = xenstore_buf->rsp_prod;
> +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod);
> +        size = min(len - off, prod - xenstore_buf->rsp_cons);
> +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
> +                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size);
> +        off += size;
> +        notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE ==
> +                  xenstore_buf->rsp_prod);

This looks odd to me?  We want to notify as soon as the ring is empty,
which can happen at any place in the ring right?

Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after*
the rsp_cons increase.

> +        rmb();

rmb() must be placed before memcpy_from_ring, to make sure that the data
we read from the buffer is up-to-date according to the read we made from
rsp_prod.

The rest seems ok to me.

Samuel


From xen-devel-bounces@lists.xenproject.org Tue Sep 14 23:52:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 14 Sep 2021 23:52:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187082.335812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIE8-0002Rr-JM; Tue, 14 Sep 2021 23:52:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187082.335812; Tue, 14 Sep 2021 23:52:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIE8-0002Rk-GH; Tue, 14 Sep 2021 23:52:36 +0000
Received: by outflank-mailman (input) for mailman id 187082;
 Tue, 14 Sep 2021 23:52:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIE7-0002Ra-FU; Tue, 14 Sep 2021 23:52:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIE7-0000zL-5w; Tue, 14 Sep 2021 23:52:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIE6-0007ST-Qu; Tue, 14 Sep 2021 23:52:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIE6-000326-QQ; Tue, 14 Sep 2021 23:52:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YEDTtY+w06zFJ393096g5v6776Ortjm2UWGIXW3q+j0=; b=gWJvHgo7+o4VnCzEgl8U7Hgfw6
	DOL6TkLOpLnxf8UPzvIjn1TjWOi4aWOpOP5W2LcTomFgX4ZfZ0xB6RResyrlziC9hUaUx0zeM2wuj
	lMEAfn3zv6u46L/pEiim9TYsl1ceV+h2pYpwsX7TAYO/AWqq8SRuxMexMbhuA9NDF3mA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164983-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164983: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a89bcd9737757e4d671783588a6041a84a5e1754
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 14 Sep 2021 23:52:34 +0000

flight 164983 xen-unstable real [real]
flight 164986 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164983/
http://logs.test-lab.xenproject.org/osstest/logs/164986/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  a89bcd9737757e4d671783588a6041a84a5e1754
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    4 days
Failing since        164951  2021-09-12 00:14:36 Z    2 days    6 attempts
Testing same since   164976  2021-09-14 01:38:33 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit a89bcd9737757e4d671783588a6041a84a5e1754
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Wed Jul 7 09:15:31 2021 +0200

    tools/go: honor append build flags
    
    Make the go build use APPEND_{C/LD}FLAGS when necessary, just like
    other parts of the build.
    
    Reported-by: Ting-Wei Lan <lantw44@gmail.com>
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Acked-by: Nick Rosbrook <rosbrookn@ainfosec.com>
    Acked-by: Ian Jackson <iwj@xenproject.org>

commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 00:13:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 00:13:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187096.335825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIYI-0005bq-IK; Wed, 15 Sep 2021 00:13:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187096.335825; Wed, 15 Sep 2021 00:13:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIYI-0005bj-FC; Wed, 15 Sep 2021 00:13:26 +0000
Received: by outflank-mailman (input) for mailman id 187096;
 Wed, 15 Sep 2021 00:13:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIYG-0005bZ-P8; Wed, 15 Sep 2021 00:13:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIYG-00020c-HU; Wed, 15 Sep 2021 00:13:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIYG-0008CV-4L; Wed, 15 Sep 2021 00:13:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQIYG-0003OM-3x; Wed, 15 Sep 2021 00:13:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=W3vEYOYIQXaI1grrQdzedtdxjuwPIHCu23V++VMHjHk=; b=Ux80vmjOXb5qX5cDQmRgulBoB5
	+OZJRgI0htXa+gsyzkWhzCQrHMhUZ+ppxdzvtZBJRPDaPBAXO1W+2iZsQz6ejuomDQscwGGsHpB26
	ILw3iS430X4VAzLVjw7B4ZA1j/ZLwtO/PSMurWlPDRxa3a/y2o/xejlNEos+9TcBx2kE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164984-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164984: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:guest-start:fail:heisenbug
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d0ee23f9d78be5531c4b055ea424ed0b489dfe9b
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 00:13:24 +0000

flight 164984 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164984/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-xsm      14 guest-start      fail in 164977 pass in 164984
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 164977

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-amd64-amd64-freebsd11-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                d0ee23f9d78be5531c4b055ea424ed0b489dfe9b
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  410 days
Failing since        152366  2020-08-01 20:49:34 Z  409 days  712 attempts
Testing same since   164977  2021-09-14 03:20:39 Z    0 days    2 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255658 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 00:25:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 00:25:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187105.335840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIk0-0007BZ-Mn; Wed, 15 Sep 2021 00:25:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187105.335840; Wed, 15 Sep 2021 00:25:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIk0-0007BS-Jv; Wed, 15 Sep 2021 00:25:32 +0000
Received: by outflank-mailman (input) for mailman id 187105;
 Wed, 15 Sep 2021 00:25:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQIjz-0007BM-PY
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 00:25:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cde7055d-c39d-48fb-80e7-cf04772d8b92;
 Wed, 15 Sep 2021 00:25:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BFA376121D;
 Wed, 15 Sep 2021 00:25:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cde7055d-c39d-48fb-80e7-cf04772d8b92
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631665530;
	bh=1r6We/XbbEX1xmXeEkmuZ8/CtRbvBr5pJq4VhaF9Ju4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Fk+8NHEsEDFEBUUNkcCN+HWntwq6tEMUwzP8Ca7r0uDJoVY6YNUfOSmVM+UELoUcr
	 QuMTedNtYpf9I3X37yB09ttqeRpjcM+6n5uypd/CyAH6FfHw4vgXRxOZGTBhVLIK2O
	 BCZIYqRa6Ix5HQ+xGR4wxsw0IqSOV1VHGf+7OP4G0S2Hjzo95cF730Kf67xzgsMHz2
	 6G8YW93MhvhM6MWQM3vg/h9U+t79hDC/iFsxUz4t2xBURK0rT6S1/5Za/yf0YtIzQP
	 ifY3tofOkS4bBdkH/wyCMllRLs5enEE/Oz3kmCTZRajiGYMC4K1iHqg2xxruSpisLN
	 JNbAaPcZ7dvuQ==
Date: Tue, 14 Sep 2021 17:25:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Julien Grall <julien@xen.org>, 
    "sstabellini@kernel.org" <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Rahul Singh <rahul.singh@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for hardware
 domain
In-Reply-To: <a4343670-2d9a-cec2-f093-aa3b0a5e0657@epam.com>
Message-ID: <alpine.DEB.2.21.2109141716170.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-10-andr2000@gmail.com> <247bd41c-98e6-f898-8216-e36b22158636@xen.org> <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com> <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com> <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org> <7263f268-7f90-e561-4679-f78185c88cd8@epam.com> <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org> <52da7955-ead7-f404-6e51-7d1ee7bc3142@epam.com>
 <a4343670-2d9a-cec2-f093-aa3b0a5e0657@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-735770519-1631665135=:21985"
Content-ID: <alpine.DEB.2.21.2109141719190.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-735770519-1631665135=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109141719191.21985@sstabellini-ThinkPad-T480s>

On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
> >> What you want to know if how many times register_mmio_handler() will be called from domain_vpci_init().
> >>
> >> You introduced a function pci_host_iterate_bridges() that will walk over the bridges and then call the callback vpci_setup_mmio_handler(). So you could introduce a new callback that will return 1 if bridge->ops->register_mmio_handler is not NULL or 0.
> >
> > Ok, clear. Something like:
> >
> >     if ( (rc = domain_vgic_register(d, &count)) != 0 )
> >         goto fail;
> >
> >     *find out how many bridges and update count*
> >
> >
> >     if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 )
> >         goto fail;
> >
> I have the following code now:
> 
> int domain_vpci_get_num_mmio_handlers(struct domain *d)
> {
>      int count;

count is incremented but not initialized


>      if ( is_hardware_domain(d) )
>          /* For each PCI host bridge's configuration space. */
>          count += pci_host_get_num_bridges();
>      else
>          /*
>           * VPCI_MSIX_MEM_NUM handlers for MSI-X tables per each PCI device
>           * being passed through. Maximum number of supported devices
>           * is 32 as virtual bus topology emulates the devices as embedded
>           * endpoints.
>           * +1 for a single emulated host bridge's configuration space. */
>          count = VPCI_MSIX_MEM_NUM * 32 + 1;
>      return count;
> }
> 
> Please note that we cannot tell how many PCIe devices are going to be passed through
> 
> So, worst case for DomU is going to be 65 to what we already have...
> 
> This sounds scary a bit as most probably we won't pass through 32 devices most of the
> 
> time, but will make d->arch.vmmio.handlers almost 4 times bigger then it is now.
> 
> This may have influence on the MMIO handlers performance...

I am OK with that given that it doesn't affect performance until you
actually start creating too many virtual devices for the DomU. In other
words, find_mmio_handler restricts the search to vmmio->num_entries, so
as long as most entries are allocated but unused, we should be fine.
--8323329-735770519-1631665135=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 00:36:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 00:36:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187113.335851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIuE-0000Hm-MK; Wed, 15 Sep 2021 00:36:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187113.335851; Wed, 15 Sep 2021 00:36:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQIuE-0000Hb-JC; Wed, 15 Sep 2021 00:36:06 +0000
Received: by outflank-mailman (input) for mailman id 187113;
 Wed, 15 Sep 2021 00:36:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQIuD-0000HU-9s
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 00:36:05 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a0b6d54e-1a06-44f3-8ea1-cee554687c5c;
 Wed, 15 Sep 2021 00:36:04 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4C5666121E;
 Wed, 15 Sep 2021 00:36:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0b6d54e-1a06-44f3-8ea1-cee554687c5c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631666163;
	bh=5/ccFTamaZvBXeAJx3CTBWIVIcgIHELyJe4Bn2xdoXI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EOK6KsAXGxYQPMs1jmFZJ/ZQeOI533u1tKLhSWaucgYJZ8dqY25vHr2fNrk7/WwT3
	 W9U5+3ZXyVAu/8BcIie5sb+FriMs2rshFIR2z1KBr3Kn6Up7XzF9w/HQzhe6inmOA+
	 Bs1mQXaOwX7kLPUYG3vOkjLIBst+72dovIjz7HvpuH46qiBvf+/dA1YZUCfwKjx9sH
	 TRPp7/xoWoQqU+A0soPlMpPEkaan5EhPvWmB1nrD+4Trxaf/hBcUalTwuIj/gJ67VP
	 ogv2NIPVeDELT9mJSXdPFTsOI3F3y61UaMH2hAtk+O98/Of+pPr7yBLJO0kQJCWIpr
	 HmpuXmmjV9zfQ==
Date: Tue, 14 Sep 2021 17:36:02 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Julien Grall <julien.grall.oss@gmail.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
In-Reply-To: <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
Message-ID: <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com> <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com> <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com> <684b3534-40eb-add7-f46e-c6258904757b@xen.org> <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s> <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com> <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-250997749-1631666163=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-250997749-1631666163=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
> With the patch above I have the following log in Domain-0:
> 
> generic-armv8-xt-dom0 login: root
> root@generic-armv8-xt-dom0:~# (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input)
> (XEN) ==== PCI devices ====
> (XEN) ==== segment 0000 ====
> (XEN) 0000:03:00.0 - d0 - node -1
> (XEN) 0000:02:02.0 - d0 - node -1
> (XEN) 0000:02:01.0 - d0 - node -1
> (XEN) 0000:02:00.0 - d0 - node -1
> (XEN) 0000:01:00.0 - d0 - node -1
> (XEN) 0000:00:00.0 - d0 - node -1
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> 
> root@generic-armv8-xt-dom0:~# modprobe e1000e
> [   46.104729] e1000e: Intel(R) PRO/1000 Network Driver
> [   46.105479] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
> [   46.107297] e1000e 0000:03:00.0: enabling device (0000 -> 0002)
> (XEN) map [e0000, e001f] -> 0xe0000 for d0
> (XEN) map [e0020, e003f] -> 0xe0020 for d0
> [   46.178513] e1000e 0000:03:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
> [   46.189668] pci_msi_setup_msi_irqs
> [   46.191016] nwl_compose_msi_msg msg at fe440000
> (XEN) traps.c:2014:d0v0 HSR=0x00000093810047 pc=0xffff8000104b4b00 gva=0xffff800010fa5000 gpa=0x000000e0040000
> [   46.200455] Unhandled fault at 0xffff800010fa5000
> 
> [snip]
> 
> [   46.233079] Call trace:
> [   46.233559]  __pci_write_msi_msg+0x70/0x180
> [   46.234149]  pci_msi_domain_write_msg+0x28/0x30
> [   46.234869]  msi_domain_activate+0x5c/0x88
> 
>  From the above you can see that BARs are mapped for Domain-0 now
> 
> only when an assigned PCI device gets enabled in Domain-0.
> 
> Another thing to note is that we crash on MSI-X access as BARs are mapped
> 
> to the domain while enabling memory decoding in the COMMAND register,
> 
> but MSI-X are handled differently, e.g. we have MSI-X holes in the mappings.
> 
> This is because before this change the whole PCI aperture was mapped into
> 
> Domain-0 and it is not. Thus, MSI-X holes are left unmapped now and there
> 
> was no need to do so, e.g. they were always mapped into Domain-0 and
> 
> emulated for guests.
> 
> Please note that one cannot use xl pci-attach in this case to attach the PCI device
> 
> in question to Domain-0 as (please see the log) that device is already attached.
> 
> Neither it can be detached and re-attached. So, without mapping MSI-X holes for
> 
> Domain-0 the device becomes unusable in Domain-0. At the same time the device
> 
> can be successfully passed to DomU.
> 
> 
> Julien, Stefano! Please let me know how can we proceed with this.

What was the plan for MSI-X in Dom0?

Given that Dom0 interacts with a virtual-ITS and virtual-GIC rather than
a physical-ITS and physical-GIC, I imagine that it wasn't correct for
Dom0 to write to the real MSI-X table directly?

Shouldn't Dom0 get emulated MSI-X tables like any DomU?

Otherwise, if Dom0 is expected to have the real MSI-X tables mapped, then
I would suggest to map them at the same time as the BARs. But I am
thinking that Dom0 should get emulated MSI-X tables, not physical MSI-X
tables.
--8323329-250997749-1631666163=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:12:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:12:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187130.335889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJSy-0003AI-MJ; Wed, 15 Sep 2021 01:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187130.335889; Wed, 15 Sep 2021 01:12:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJSy-0003AB-JZ; Wed, 15 Sep 2021 01:12:00 +0000
Received: by outflank-mailman (input) for mailman id 187130;
 Wed, 15 Sep 2021 01:11:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wSyZ=OF=canb.auug.org.au=sfr@srs-us1.protection.inumbo.net>)
 id 1mQJSv-0003A4-Av
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:11:59 +0000
Received: from ozlabs.org (unknown [203.11.71.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 034c0daa-db69-469f-82ae-b5a1a8a1d088;
 Wed, 15 Sep 2021 01:11:53 +0000 (UTC)
Received: from authenticated.ozlabs.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by mail.ozlabs.org (Postfix) with ESMTPSA id 4H8Mcj3v97z9sVq;
 Wed, 15 Sep 2021 11:11:49 +1000 (AEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 034c0daa-db69-469f-82ae-b5a1a8a1d088
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canb.auug.org.au;
	s=201702; t=1631668309;
	bh=bsbgThutFzSZRIz+iIoRjpMDBwlNKjj7zE08iVOcKm8=;
	h=Date:From:To:Cc:Subject:From;
	b=GQ4N5kqUphB1adToSZjO4dbkkhtqdhVMR1notEhX3u97X4ssGnPn32VdHHedKmWWZ
	 +0arIz8siPieVOqQ9B8UoyuvnI9qof9E/VsYqh0s7iptj81Y3Sf6IOEnXgnQcnvTWe
	 Iqpb8rQB1QY8tcTO8bSgUv9SgmHu4eZ1u3vjWfX7dVMKQ5oMMRH+UDLpum9RZ71y4J
	 tXm5t5h61g6B8uq7XAl5712Hv1R0LTw/sCerjChEP91tZVcDX78AxBmi5y5rhyT9jm
	 POzBGNG8OcIbKmZ+JjhZrURJo6DypINO7/3gakwIgigLbNMz9suc8wTPtr4hXgglOz
	 dkWum8cvmiU1Q==
Date: Wed, 15 Sep 2021 11:11:48 +1000
From: Stephen Rothwell <sfr@canb.auug.org.au>
To: Juergen Gross <jgross@suse.com>, Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Xen Devel
 <xen-devel@lists.xenproject.org>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Linux Next
 Mailing List <linux-next@vger.kernel.org>
Subject: linux-next: build failure after merge of the xen-tip tree
Message-ID: <20210915111148.54ddb83d@canb.auug.org.au>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/G6hcjoL2RPpb8JI4cIlnBY_";
 protocol="application/pgp-signature"; micalg=pgp-sha256

--Sig_/G6hcjoL2RPpb8JI4cIlnBY_
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Hi all,

After merging the xen-tip tree, today's linux-next build (x86_64
allmodconfig) failed like this:

drivers/base/power/trace.c: In function 'generate_pm_trace':
drivers/base/power/trace.c:170:10: error: 'return' with a value, in functio=
n returning void [-Werror=3Dreturn-type]
  170 |   return 0;
      |          ^
drivers/base/power/trace.c:163:6: note: declared here
  163 | void generate_pm_trace(const void *tracedata, unsigned int user)
      |      ^~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Caused by commit

  38b6c7137e13 ("PM: base: power: don't try to use non-existing RTC for sto=
ring data")

I used the xen-tip tree from next-20210914 for today.
--=20
Cheers,
Stephen Rothwell

--Sig_/G6hcjoL2RPpb8JI4cIlnBY_
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEENIC96giZ81tWdLgKAVBC80lX0GwFAmFBSFQACgkQAVBC80lX
0Gwr+wgAn46P/kdwFCQD2pKCXXk05+r+qWnuFMr5KwM2MfId4x5n2PVvQI3g5uPm
dLBHzZMllBDegIdFjtA4y1LDWf9YVKZa+XIwhUnOO1OUFnNR18rB3oLTAWFhIvIt
8FTtXQx30QfzUMrJ8yQfmvmswT1POEKINkQ9fSKH9XKNTo/yghs4CwEQM2UGjPr2
mUDvk+aguzW/YGRK0WCR+ZsahpQ9pj+DydM7DTPRaDJG3Ok1G0NYwXPP0W23A61D
ErwiFq/IPeb8o7c+mfDJG6o5AoW+xy+VKBnNt8UsERO0r2Bg7yZEZvn/TEtehLKS
UJ+ep5+DR8uotIVWaEQQfh5zjz67Uw==
=c0eW
-----END PGP SIGNATURE-----

--Sig_/G6hcjoL2RPpb8JI4cIlnBY_--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:19:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:19:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187139.335901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJaB-00042x-JN; Wed, 15 Sep 2021 01:19:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187139.335901; Wed, 15 Sep 2021 01:19:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJaB-00042q-GK; Wed, 15 Sep 2021 01:19:27 +0000
Received: by outflank-mailman (input) for mailman id 187139;
 Wed, 15 Sep 2021 01:19:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rlG+=OF=intel.com=lingshan.zhu@srs-us1.protection.inumbo.net>)
 id 1mQJaA-00042k-QC
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:19:26 +0000
Received: from mga05.intel.com (unknown [192.55.52.43])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7e5287fd-8a56-44b3-bf23-6a2235857950;
 Wed, 15 Sep 2021 01:19:22 +0000 (UTC)
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
 by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 14 Sep 2021 18:19:21 -0700
Received: from zhoushua-mobl1.ccr.corp.intel.com (HELO [10.255.30.237])
 ([10.255.30.237])
 by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 14 Sep 2021 18:19:14 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e5287fd-8a56-44b3-bf23-6a2235857950
X-IronPort-AV: E=McAfee;i="6200,9189,10107"; a="307729552"
X-IronPort-AV: E=Sophos;i="5.85,292,1624345200"; 
   d="scan'208";a="307729552"
X-IronPort-AV: E=Sophos;i="5.85,292,1624345200"; 
   d="scan'208";a="544371674"
Message-ID: <186c330e-be42-4c49-545c-8f73573b5869@intel.com>
Date: Wed, 15 Sep 2021 09:19:12 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.0.3
Subject: Re: [PATCH V10 01/18] perf/core: Use static_call to optimize
 perf_guest_info_callbacks
Content-Language: en-US
To: Sean Christopherson <seanjc@google.com>
Cc: peterz@infradead.org, pbonzini@redhat.com, bp@alien8.de,
 vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com,
 joro@8bytes.org, kan.liang@linux.intel.com, ak@linux.intel.com,
 wei.w.wang@intel.com, eranian@google.com, liuxiangdong5@huawei.com,
 linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org,
 like.xu.linux@gmail.com, boris.ostrvsky@oracle.com,
 Like Xu <like.xu@linux.intel.com>, Will Deacon <will@kernel.org>,
 Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
 Nick Hu <nickhu@andestech.com>, Paul Walmsley <paul.walmsley@sifive.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 xen-devel@lists.xenproject.org
References: <20210806133802.3528-1-lingshan.zhu@intel.com>
 <20210806133802.3528-2-lingshan.zhu@intel.com> <YSfykbECnC6J02Yk@google.com>
From: "Zhu, Lingshan" <lingshan.zhu@intel.com>
In-Reply-To: <YSfykbECnC6J02Yk@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 8/27/2021 3:59 AM, Sean Christopherson wrote:
> TL;DR: Please don't merge this patch, it's broken and is also built on a shoddy
>         foundation that I would like to fix.
Hi Sean,Peter, Paolo

I will send out an V11 which drops this patch since it's buggy, and Sean 
is working on fix this.
Does this sound good?

Thanks,
Zhu Lingshan
>
> On Fri, Aug 06, 2021, Zhu Lingshan wrote:
>> diff --git a/kernel/events/core.c b/kernel/events/core.c
>> index 464917096e73..e466fc8176e1 100644
>> --- a/kernel/events/core.c
>> +++ b/kernel/events/core.c
>> @@ -6489,9 +6489,18 @@ static void perf_pending_event(struct irq_work *entry)
>>    */
>>   struct perf_guest_info_callbacks *perf_guest_cbs;
>>   
>> +/* explicitly use __weak to fix duplicate symbol error */
>> +void __weak arch_perf_update_guest_cbs(void)
>> +{
>> +}
>> +
>>   int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
>>   {
>> +	if (WARN_ON_ONCE(perf_guest_cbs))
>> +		return -EBUSY;
>> +
>>   	perf_guest_cbs = cbs;
>> +	arch_perf_update_guest_cbs();
> This is horribly broken, it fails to cleanup the static calls when KVM unregisters
> the callbacks, which happens when the vendor module, e.g. kvm_intel, is unloaded.
> The explosion doesn't happen until 'kvm' is unloaded because the functions are
> implemented in 'kvm', i.e. the use-after-free is deferred a bit.
>
>    BUG: unable to handle page fault for address: ffffffffa011bb90
>    #PF: supervisor instruction fetch in kernel mode
>    #PF: error_code(0x0010) - not-present page
>    PGD 6211067 P4D 6211067 PUD 6212063 PMD 102b99067 PTE 0
>    Oops: 0010 [#1] PREEMPT SMP
>    CPU: 0 PID: 1047 Comm: rmmod Not tainted 5.14.0-rc2+ #460
>    Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
>    RIP: 0010:0xffffffffa011bb90
>    Code: Unable to access opcode bytes at RIP 0xffffffffa011bb66.
>    Call Trace:
>     <NMI>
>     ? perf_misc_flags+0xe/0x50
>     ? perf_prepare_sample+0x53/0x6b0
>     ? perf_event_output_forward+0x67/0x160
>     ? kvm_clock_read+0x14/0x30
>     ? kvm_sched_clock_read+0x5/0x10
>     ? sched_clock_cpu+0xd/0xd0
>     ? __perf_event_overflow+0x52/0xf0
>     ? handle_pmi_common+0x1f2/0x2d0
>     ? __flush_tlb_all+0x30/0x30
>     ? intel_pmu_handle_irq+0xcf/0x410
>     ? nmi_handle+0x5/0x260
>     ? perf_event_nmi_handler+0x28/0x50
>     ? nmi_handle+0xc7/0x260
>     ? lock_release+0x2b0/0x2b0
>     ? default_do_nmi+0x6b/0x170
>     ? exc_nmi+0x103/0x130
>     ? end_repeat_nmi+0x16/0x1f
>     ? lock_release+0x2b0/0x2b0
>     ? lock_release+0x2b0/0x2b0
>     ? lock_release+0x2b0/0x2b0
>     </NMI>
>    Modules linked in: irqbypass [last unloaded: kvm]
>
> Even more fun, the existing perf_guest_cbs framework is also broken, though it's
> much harder to get it to fail, and probably impossible to get it to fail without
> some help.  The issue is that perf_guest_cbs is global, which means that it can
> be nullified by KVM (during module unload) while the callbacks are being accessed
> by a PMI handler on a different CPU.
>
> The bug has escaped notice because all dererfences of perf_guest_cbs follow the
> same "perf_guest_cbs && perf_guest_cbs->is_in_guest()" pattern, and AFAICT the
> compiler never reload perf_guest_cbs in this sequence.  The compiler does reload
> perf_guest_cbs for any future dereferences, but the ->is_in_guest() guard all but
> guarantees the PMI handler will win the race, e.g. to nullify perf_guest_cbs,
> KVM has to completely exit the guest and teardown down all VMs before it can be
> unloaded.
>
> But with a help, e.g. RAED_ONCE(perf_guest_cbs), unloading kvm_intel can trigger
> a NULL pointer derference, e.g. this tweak
>
> diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> index 1eb45139fcc6..202e5ad97f82 100644
> --- a/arch/x86/events/core.c
> +++ b/arch/x86/events/core.c
> @@ -2954,7 +2954,7 @@ unsigned long perf_misc_flags(struct pt_regs *regs)
>   {
>          int misc = 0;
>
> -       if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
> +       if (READ_ONCE(perf_guest_cbs) && READ_ONCE(perf_guest_cbs)->is_in_guest()) {
>                  if (perf_guest_cbs->is_user_mode())
>                          misc |= PERF_RECORD_MISC_GUEST_USER;
>                  else
>
>
> while spamming module load/unload leads to:
>
>    BUG: kernel NULL pointer dereference, address: 0000000000000000
>    #PF: supervisor read access in kernel mode
>    #PF: error_code(0x0000) - not-present page
>    PGD 0 P4D 0
>    Oops: 0000 [#1] PREEMPT SMP
>    CPU: 6 PID: 1825 Comm: stress Not tainted 5.14.0-rc2+ #459
>    Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
>    RIP: 0010:perf_misc_flags+0x1c/0x70
>    Call Trace:
>     perf_prepare_sample+0x53/0x6b0
>     perf_event_output_forward+0x67/0x160
>     __perf_event_overflow+0x52/0xf0
>     handle_pmi_common+0x207/0x300
>     intel_pmu_handle_irq+0xcf/0x410
>     perf_event_nmi_handler+0x28/0x50
>     nmi_handle+0xc7/0x260
>     default_do_nmi+0x6b/0x170
>     exc_nmi+0x103/0x130
>     asm_exc_nmi+0x76/0xbf
>
>
> The good news is that I have a series that should fix both the existing NULL pointer
> bug and mostly obviate the need for static calls.  The bad news is that my approach,
> making perf_guest_cbs per-CPU, likely complicates turning these into static calls,
> though I'm guessing it's still a solvable problem.
>
> Tangentially related, IMO we should make architectures opt-in to getting
> perf_guest_cbs and nuke all of the code in the below files.  Except for arm,
> which recently lost KVM support, it's all a bunch of useless copy-paste code that
> serves no purpose and just complicates cleanups like this.
>
>>   arch/arm/kernel/perf_callchain.c   | 16 +++++++-----
>>   arch/csky/kernel/perf_callchain.c  |  4 +--
>>   arch/nds32/kernel/perf_event_cpu.c | 16 +++++++-----
>>   arch/riscv/kernel/perf_callchain.c |  4 +--



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:22:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:22:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187148.335911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJdQ-0005Nx-2N; Wed, 15 Sep 2021 01:22:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187148.335911; Wed, 15 Sep 2021 01:22:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJdP-0005Nq-Vj; Wed, 15 Sep 2021 01:22:47 +0000
Received: by outflank-mailman (input) for mailman id 187148;
 Wed, 15 Sep 2021 01:22:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQJdO-0005Nk-TY
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:22:46 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ab744a35-d748-4359-99d0-8875380d7aa1;
 Wed, 15 Sep 2021 01:22:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3A5C561164;
 Wed, 15 Sep 2021 01:22:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab744a35-d748-4359-99d0-8875380d7aa1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631668965;
	bh=34pMuOrExtoHtRZ9Gf9/zwAtQwy7VI+WAiaN+pkvJA4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iog5jdpmbLkNrJwTUHixTrBW8dJcX7Nn22lx0XXX9dTsxliTHJZPCqfbCu+mcTt/C
	 96OHH/AYMexmxb6Exzr8I0Ep5SzNufmbFz3AMA+yVe1H6wTSqDDl52/q2mV2KCymsL
	 a3oVie8VcPZmw3Lsmtf02bzSJgIUhuhf1oetKtaKh8QKbzcTn58UUbhJLMpZazA3cS
	 iDbXkndq5gAgXYLe4R/fvTiiEW2k14BcNsmOE4/40XNE0dTexSnbGAv8RxFCYrxt5p
	 h1iNmHzbfD4O0TQu9l9utoq4hPIOEIRaFjiPf95dZgCeTNNdN+y/zl2Uor8w7kmsTY
	 gInqIPxhlCiHQ==
Date: Tue, 14 Sep 2021 18:22:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
In-Reply-To: <9819a6e9-93d5-e62a-7b4a-ffc2ecd996dc@suse.com>
Message-ID: <alpine.DEB.2.21.2109141817150.21985@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <397bf325-f81e-e104-6142-e8c9c4955475@suse.com> <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s> <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com> <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
 <9819a6e9-93d5-e62a-7b4a-ffc2ecd996dc@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 14 Sep 2021, Jan Beulich wrote:
> On 13.09.2021 22:31, Stefano Stabellini wrote:
> > On Mon, 13 Sep 2021, Jan Beulich wrote:
> >> On 11.09.2021 01:14, Stefano Stabellini wrote:
> >>> On Tue, 7 Sep 2021, Jan Beulich wrote:
> >>>> While the hypervisor hasn't been enforcing this, we would still better
> >>>> avoid issuing requests with GFNs not aligned to the requested order.
> >>>>
> >>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>> ---
> >>>> I wonder how useful it is to include the alignment in the panic()
> >>>> message.
> >>>
> >>> Not very useful given that it is static. I don't mind either way but you
> >>> can go ahead and remove it if you prefer (and it would make the line
> >>> shorter.)
> >>>
> >>>
> >>>> I further wonder how useful it is to wrap "bytes" in
> >>>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
> >>>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
> >>>> properly").
> >>>
> >>> This one I would keep, to make sure to print out the same amount passed
> >>> to memblock_alloc.
> >>
> >> Oh - if I was to drop it from the printk(), I would have been meaning to
> >> also drop it there. If it's useless, then it's useless everywhere.
> > 
> > That's fine too
> 
> Thanks, I'll see about dropping that then.
> 
> Another Arm-related question has occurred to me: Do you actually
> mind the higher-than-necessary alignment there? If so, a per-arch
> definition of the needed alignment would need introducing. Maybe
> that could default to PAGE_SIZE, allowing Arm and alike to get away
> without explicitly specifying a value ...

Certainly a patch like that could be good. Given that it is only one
allocation I was assuming that the higher-than-necessary alignment
wouldn't be a problem worth addressing (and I cannot completely rule out
that one day we might have to use XENMEM_exchange on ARM too).


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:32:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:32:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187156.335922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJmn-0006uz-W0; Wed, 15 Sep 2021 01:32:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187156.335922; Wed, 15 Sep 2021 01:32:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQJmn-0006us-Ss; Wed, 15 Sep 2021 01:32:29 +0000
Received: by outflank-mailman (input) for mailman id 187156;
 Wed, 15 Sep 2021 01:32:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQJmm-0006um-HD
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:32:28 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c8f4742e-15c4-11ec-b48a-12813bfff9fa;
 Wed, 15 Sep 2021 01:32:27 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 86E4260FC0;
 Wed, 15 Sep 2021 01:32:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8f4742e-15c4-11ec-b48a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631669546;
	bh=O0mnt61XzFZv3YViYVdAusceV0LVpem+l+yeZjoFIqI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=sS6YyoqIA2RwTdXvF+ptBb092bimPh1pTtr1ZY5CBDl1DoAIRkxYmbNAmdRGF/8qd
	 ZFzRJ4eXm+oU/y/wDfCMgDa8kbefPpEeDVjTS6hOE5AR6pW6Lb3SeS3Oqgcn8g6yG1
	 3AzCxCQoB+i7Uz7IhogKMOfYys0+NuSybBVM17Hlf8v2+doSi3YE+Q6t7FbK/PuiYF
	 a/6SPdlhNy+wobyNfxG8Y+2MTDHotbqriQ50NoETZrhtweDp5Jxn8C50WPInhEYgsF
	 q5HMGNEudXFsM2a3HaZ/fFdNuHHFUcnmFO0ErS+nUmaV+TIf6itNnI3jZVXrL7NvUg
	 /DUd12+jmROeQ==
Date: Tue, 14 Sep 2021 18:32:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Sai Kiran Kumar Reddy Y <ysaikiran1997@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    kannan@cimware.in
Subject: Re: Xen Booting Problem on ARM Machine
In-Reply-To: <CAEsO4uwpiDO2QoqLRTOxpsYM9YSPAsbX0P=gagdEy21pk1VsPQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.2109141825330.21985@sstabellini-ThinkPad-T480s>
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol> <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org> <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com> <alpine.DEB.2.21.2109131437030.10523@sstabellini-ThinkPad-T480s>
 <CAEsO4uwpiDO2QoqLRTOxpsYM9YSPAsbX0P=gagdEy21pk1VsPQ@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1943811075-1631669172=:21985"
Content-ID: <alpine.DEB.2.21.2109141831220.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1943811075-1631669172=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109141831221.21985@sstabellini-ThinkPad-T480s>

Something is off. When you enabled earlyprintk in Xen, you should see
something like this at boot time:
https://marc.info/?l=xen-devel&m=158829968025334

All the Xen logs starting with "(XEN)" on the serial. Do you have access
to the serial of the machine? Without it, it is going to be hard to
debug.


On Tue, 14 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
> In the folder "/var/log", there's a file called "xen", which is empty. As far as the boot logs are concerned, I don't see any debug
> messages related to xen. I am attaching the log files, "kern.txt" and "boot.txt"
> 
> On Tue, Sep 14, 2021 at 3:08 AM Stefano Stabellini <sstabellini@kernel.org> wrote:
>       On Mon, 13 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
>       > On Fri, Sep 10, 2021 at 7:30 PM Julien Grall <julien@xen.org> wrote:
>       >
>       >
>       >       On 08/09/2021 11:43, Sai Kiran wrote:
>       >       > Hello,
>       >
>       >       Hi,
>       >
>       >       Thank you for the report. Moving the discussion back to xen-devel
>       >       because this looks like a potential issue in the UEFI stub in Xen.
>       >
>       >       > I have Xen-4.15.0 on an ARM Machine, with Debian 11 installed on it.
>       >
>       >       Would you be able to give more details on the Arm machine you are using?
>       >       Also, are you using ACPI or DT to boot?
>       >
>       >
>       >    Sai >> DT . ACPI configuration is disabled in Boot settings    
>       >
>       >       >  I
>       >       > am able to do “make world” and “make install”, after “./configure”, as
>       >       > specified in README file. When I reboot the system, I get the following
>       >       > message:
>       >       >
>       >       > Warning: All 128 bootinfo mem banks exhausted.
>       >       >
>       >       > Warning: All 128 bootinfo mem banks exhausted.
>       >
>       >       Hmmm... This means that you have more than 128 memory regions described
>       >       in the EFI memory map. That's quite a lot.
>       >
>       >       Although, this should be harmless as it means Xen will not use the extra
>       >       memory banks.
>       >
>       >       >
>       >       > Cannot exit boot services: ErrCode: 0x8000000000000002
>       >
>       >       This means EFI_INVALID_PARAMETER. We have code to retry because AFAICT
>       >       ExitBootServices() may sometime fails (I have CCed Jan may have more
>       >       idea what's happening).
>       >
>       >       Would you be able to provide more details on the UEFI firmware you are
>       >       using? Is it EDK2 or U-boot?
>       >
>       > Sai >>  EDK2 
>       >       Also, do you know if Linux is boot on the same system?
>       >
>       > Sai >> Yes 
>       >       However, AFAICT, the error message would not prevent Xen to continue
>       >       booting. So you may get stuck later in the boot process.
>       >
>       >       My suggestion would be to enable earlyprintk for your platform. You can
>       >       setup it up from the menuconfig in "Debugging Options".
>       >
>       > Sai >> Yes, I have enabled earlyprintk.
>       > I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h) value to 256, from 128. The error message is no longer seen,
>       but the device
>       > is stuck in the boot process.
> 
>       Could you please post the boot logs now that you enabled earlyprintk?
>       Ideally not a camera picture but a textual copy/paste from the target
>       serial?
> 
>       Earlyprintk is pretty verbose, we should be able to figure out where it
>       gets stuck.
> 
> 
> 
--8323329-1943811075-1631669172=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:51:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:51:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187164.335933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQK4p-0000wQ-J3; Wed, 15 Sep 2021 01:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187164.335933; Wed, 15 Sep 2021 01:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQK4p-0000wJ-Fj; Wed, 15 Sep 2021 01:51:07 +0000
Received: by outflank-mailman (input) for mailman id 187164;
 Wed, 15 Sep 2021 01:51:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQK4n-0000wD-SK
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:51:05 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8d64c143-a73d-48cb-a750-c28fd2b12926;
 Wed, 15 Sep 2021 01:51:03 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id CDF8061211;
 Wed, 15 Sep 2021 01:51:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d64c143-a73d-48cb-a750-c28fd2b12926
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631670662;
	bh=z0W70Z4JOetSa7g8CfvjNwgRWC4qCwo9bmhv6sh9Bxs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DhJ6GOb1z9RF8MoKZiSnxeXdJKNkyxJ7YyiEn0qQubMzT9FZtSjTJUt/KUr+QwVKI
	 +frjDgFVeOdxWz9IZc7wervGNYWJ63gSfCERzcvLkSnA46r3ypahZL2U/4IHNXcntT
	 /DqRf5MOYuMRlQ77Uv0SNMsaDJzboDdlLxIhocqe0R7UgiEK7ZBy99L8v0w5K3bX72
	 sJ48QhBWhdiCuNKFPKWm/Qjfkv8EiwrcYFd4jtxEycZyiFEiLrwMg1PFndT/7dOiPM
	 zl0gJLYf7yi4Oohaksg8j8XTXXQSSgFUXRaPI/wIqRfum+bBu2e2ldXr9TpWz6n+3M
	 stAUSLvG5uH7w==
Date: Tue, 14 Sep 2021 18:51:01 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: rm.skakun@gmail.com
cc: Jan Beulich <jbeulich@suse.com>, Roman Skakun <rm.skakun@gmail.com>, 
    Andrii Anisov <andrii_anisov@epam.com>, 
    Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>, 
    Thomas Bogendoerfer <tsbogend@alpha.franken.de>, 
    Michael Ellerman <mpe@ellerman.id.au>, 
    Benjamin Herrenschmidt <benh@kernel.crashing.org>, 
    Paul Mackerras <paulus@samba.org>, 
    Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Marek Szyprowski <m.szyprowski@samsung.com>, 
    Robin Murphy <robin.murphy@arm.com>, 
    "Paul E. McKenney" <paulmck@kernel.org>, 
    Randy Dunlap <rdunlap@infradead.org>, 
    Andrew Morton <akpm@linux-foundation.org>, 
    Thomas Gleixner <tglx@linutronix.de>, 
    "Maciej W. Rozycki" <macro@orcam.me.uk>, 
    Muchun Song <songmuchun@bytedance.com>, 
    Viresh Kumar <viresh.kumar@linaro.org>, 
    Peter Zijlstra <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>, 
    Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>, 
    xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org, 
    linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org, 
    linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, hch@lst.de
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
In-Reply-To: <20210914153046.GB815@lst.de>
Message-ID: <alpine.DEB.2.21.2109141838290.21985@sstabellini-ThinkPad-T480s>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com> <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <20210914153046.GB815@lst.de>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 14 Sep 2021, Christoph Hellwig wrote:
> On Tue, Sep 14, 2021 at 05:29:07PM +0200, Jan Beulich wrote:
> > I'm not convinced the swiotlb use describe there falls under "intended
> > use" - mapping a 1280x720 framebuffer in a single chunk? (As an aside,
> > the bottom of this page is also confusing, as following "Then we can
> > confirm the modified swiotlb size in the boot log:" there is a log
> > fragment showing the same original size of 64Mb.
> 
> It doesn't.  We also do not add hacks to the kernel for whacky out
> of tree modules.

Also, Option 1 listed in the webpage seems to be a lot better. Any
reason you can't do that? Because that option both solves the problem
and increases performance.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 01:54:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 01:54:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187172.335945 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQK7w-0001ag-1P; Wed, 15 Sep 2021 01:54:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187172.335945; Wed, 15 Sep 2021 01:54:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQK7v-0001aZ-Uk; Wed, 15 Sep 2021 01:54:19 +0000
Received: by outflank-mailman (input) for mailman id 187172;
 Wed, 15 Sep 2021 01:54:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQK7u-0001aT-P5
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 01:54:18 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d61b2c30-15c7-11ec-b48b-12813bfff9fa;
 Wed, 15 Sep 2021 01:54:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 268A3600AA;
 Wed, 15 Sep 2021 01:54:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d61b2c30-15c7-11ec-b48b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631670857;
	bh=MqDbmijwQbGSZNaO6bnRJxyOp/mjLub5Df6jQIZhwdo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fdw1xpnMmAZA7cwc4JwkDdxOK1Ood93KCC1NKZQMNArWCxBySC/614vrjkBhJ8tgP
	 Q9O9Ws4O8XX8tfEd/WOeTy75REHHK3vsBsO2HUf7aVGOo0iSeSotEZx7svfOdXVl9j
	 z95jluv5tsVxH0Lm8wezkptaCUavlWBE258hUHmDnldgy3EC+6wzollKQ0C7uwEdlW
	 qWEBluOz+xjG8yZWtXkLM1SDRkUT2dbTs9+M+6vNejCz3shyumHTdpEEQECyDz/EMb
	 CBCt9Q5Nms7OaO5uhbfocfIgWAfCssLrv6PQy1FJtY1KifnwJHYMX0S8eGWmSYaQg5
	 d57v3bdIK516w==
Date: Tue, 14 Sep 2021 18:54:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
In-Reply-To: <alpine.DEB.2.21.2109141817150.21985@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109141853510.21985@sstabellini-ThinkPad-T480s>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com> <397bf325-f81e-e104-6142-e8c9c4955475@suse.com> <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s> <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com> <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
 <9819a6e9-93d5-e62a-7b4a-ffc2ecd996dc@suse.com> <alpine.DEB.2.21.2109141817150.21985@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 14 Sep 2021, Stefano Stabellini wrote:
> On Tue, 14 Sep 2021, Jan Beulich wrote:
> > On 13.09.2021 22:31, Stefano Stabellini wrote:
> > > On Mon, 13 Sep 2021, Jan Beulich wrote:
> > >> On 11.09.2021 01:14, Stefano Stabellini wrote:
> > >>> On Tue, 7 Sep 2021, Jan Beulich wrote:
> > >>>> While the hypervisor hasn't been enforcing this, we would still better
> > >>>> avoid issuing requests with GFNs not aligned to the requested order.
> > >>>>
> > >>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > >>>> ---
> > >>>> I wonder how useful it is to include the alignment in the panic()
> > >>>> message.
> > >>>
> > >>> Not very useful given that it is static. I don't mind either way but you
> > >>> can go ahead and remove it if you prefer (and it would make the line
> > >>> shorter.)
> > >>>
> > >>>
> > >>>> I further wonder how useful it is to wrap "bytes" in
> > >>>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
> > >>>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
> > >>>> properly").
> > >>>
> > >>> This one I would keep, to make sure to print out the same amount passed
> > >>> to memblock_alloc.
> > >>
> > >> Oh - if I was to drop it from the printk(), I would have been meaning to
> > >> also drop it there. If it's useless, then it's useless everywhere.
> > > 
> > > That's fine too
> > 
> > Thanks, I'll see about dropping that then.
> > 
> > Another Arm-related question has occurred to me: Do you actually
> > mind the higher-than-necessary alignment there? If so, a per-arch
> > definition of the needed alignment would need introducing. Maybe
> > that could default to PAGE_SIZE, allowing Arm and alike to get away
> > without explicitly specifying a value ...
> 
> Certainly a patch like that could be good. Given that it is only one
> allocation I was assuming that the higher-than-necessary alignment
> wouldn't be a problem worth addressing (and I cannot completely rule out
> that one day we might have to use XENMEM_exchange on ARM too).

Also this code is currently #ifdef CONFIG_X86


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 02:40:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 02:40:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187181.335955 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQKqq-0007Iv-7h; Wed, 15 Sep 2021 02:40:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187181.335955; Wed, 15 Sep 2021 02:40:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQKqq-0007Io-4o; Wed, 15 Sep 2021 02:40:44 +0000
Received: by outflank-mailman (input) for mailman id 187181;
 Wed, 15 Sep 2021 02:40:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U58Y=OF=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mQKqo-0007Ii-6Z
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 02:40:42 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 50cee92a-15ce-11ec-b4d0-12813bfff9fa;
 Wed, 15 Sep 2021 02:40:40 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 18F2eWxa031805
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 14 Sep 2021 22:40:38 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 18F2eWfB031804;
 Tue, 14 Sep 2021 19:40:32 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50cee92a-15ce-11ec-b4d0-12813bfff9fa
Date: Tue, 14 Sep 2021 19:40:32 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Tue, Sep 07, 2021 at 05:57:10PM +0200, Jan Beulich wrote:
> On 07.09.2021 17:03, Elliott Mitchell wrote:
> >  Could be this system is in an
> > intergenerational hole, and some spot in the PVH/HVM code makes an
> > assumption of the presence of NPT guarantees presence of an operational
> > IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
> > code, some portion of the IOMMU code might be checking for presence of
> > NPT instead of presence of IOMMU.
> 
> This is all very speculative; I consider what you suspect not very likely,
> but also not entirely impossible. This is not the least because for a
> long time we've been running without shared page tables on AMD.
> 
> I'm afraid without technical data and without knowing how to repro, I
> don't see a way forward here.

Downtimes are very expensive even for lower-end servers.  Plus there is
the issue the system wasn't meant for development and thus never had
appropriate setup done.

Experimentation with a system of similar age suggested another candidate.
System has a conventional BIOS.  Might some dependancies on the presence
of UEFI snuck into the NPT code?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Wed Sep 15 03:29:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 03:29:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187189.335967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQLbl-0003TO-Ps; Wed, 15 Sep 2021 03:29:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187189.335967; Wed, 15 Sep 2021 03:29:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQLbl-0003TH-Mp; Wed, 15 Sep 2021 03:29:13 +0000
Received: by outflank-mailman (input) for mailman id 187189;
 Wed, 15 Sep 2021 03:29:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=26R+=OF=xilinx.com=stefanos@srs-us1.protection.inumbo.net>)
 id 1mQLbj-0003TB-TN
 for xen-devel@lists.xen.org; Wed, 15 Sep 2021 03:29:12 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (unknown
 [40.107.220.54]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 16a4f29c-15d5-11ec-b4d4-12813bfff9fa;
 Wed, 15 Sep 2021 03:29:10 +0000 (UTC)
Received: from DM5PR12CA0050.namprd12.prod.outlook.com (2603:10b6:3:103::12)
 by BYAPR02MB5157.namprd02.prod.outlook.com (2603:10b6:a03:64::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 03:29:05 +0000
Received: from DM3NAM02FT025.eop-nam02.prod.protection.outlook.com
 (2603:10b6:3:103:cafe::4c) by DM5PR12CA0050.outlook.office365.com
 (2603:10b6:3:103::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Wed, 15 Sep 2021 03:29:05 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 DM3NAM02FT025.mail.protection.outlook.com (10.13.4.112) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 03:29:05 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Tue, 14 Sep 2021 20:29:03 -0700
Received: from smtp.xilinx.com (172.19.127.96) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Tue, 14 Sep 2021 20:29:03 -0700
Received: from [10.23.120.123] (port=59358 helo=localhost)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <stefano.stabellini@xilinx.com>)
 id 1mQLbb-0005bf-2T; Tue, 14 Sep 2021 20:29:03 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16a4f29c-15d5-11ec-b4d4-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LeIu1K7IWEioBlGJ3Oyl2SMlQaPVNkEK6hsjR/vLYx/OaXHeG9l+eFIKguiuc9LKb7KtSWKawV9LocgqzVi/JnIziXH6xF4KMQP4127HrsuWFd94JnxInhLW+IVl7DbMGwq3rtvEbrAuwyR+GWN6hmxNly5lEyy/3074DFfNwv2/Ik+8CBOdiWFUF4jGMtWPyu/tIbRmMY0c1SOneICsjlKQZt9o4YlrjGsXEk6q00EqX52nOsvEkOLGFSMQD+ZkvPmPdffS4bOPNKSwvOk8duPPm4SKpXUdsArsOa/Y5PXKNBweHKmmJ5G1x9tD02sEmf0TC9PZ1SZdjXGn3tT4sQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=N7Tl2WRijlXWZr4aIJxLSFdeZd+kL7VyxyPMG7+Sp00=;
 b=S3CB7EocoRGnm/pCf04OWCLZrKGFVkgU8ZYOJ30H1tUpKC5pubNOk8/56rZvNi+s9iWIpQbsHjcwUUafM5Cg1xXL41zlsPPkck5EnVaabq89FVH+lL2FQH3hpjgDDr+Etn12nDB9YSKI+zFzheYxcgakICol3+YQntR10YVeW5peuJc0xY0c4rRpqcfLVddq4IF7hqRcjpxizu8h8nbLEdusstYAHoGfjqRslb0P4Z5VoMPeXIqXp0q75S59EgVP/MnoPYefdMrPhpl7de4mng4STNesI7GqTujogm6KlkvUmm46OYjtAXJrKXE9c68JeqdCEYelkNFXRQ+ki6lRpQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=quicinc.com smtp.mailfrom=xilinx.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=N7Tl2WRijlXWZr4aIJxLSFdeZd+kL7VyxyPMG7+Sp00=;
 b=KaA+QS5JNSJgvdObvsChGapVEmGfUhjdOtK6DOf/lRCXWikCf03a1FvSXmpt0KZGu9aWVCRWzTAX/IEjpz5NiiTIXbaXdJvgOgeypzvsiWXzDbi7jxdbe8wXnaO3/zgoT0ya1jdsiuFTJgxWXhPlpy0mPLONvjLn/JwDj+KaijU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; quicinc.com; dkim=none (message not signed)
 header.d=none;quicinc.com; dmarc=pass action=none header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
Date: Tue, 14 Sep 2021 20:29:02 -0700
From: Stefano Stabellini <stefano.stabellini@xilinx.com>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Trilok Soni <quic_tsoni@quicinc.com>
CC: Stefano Stabellini <stefano.stabellini@xilinx.com>, AKASHI Takahiro
	<takahiro.akashi@linaro.org>, <paul@xen.org>, Stratos Mailing List
	<stratos-dev@op-lists.linaro.org>, <virtio-dev@lists.oasis-open.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Jan Kiszka <jan.kiszka@siemens.com>,
	Arnd Bergmann <arnd.bergmann@linaro.org>, <jgross@suse.com>,
	<julien@xen.org>, Carl van Schaik <cvanscha@qti.qualcomm.com>,
	<Bertrand.Marquis@arm.com>, Stefan Hajnoczi <stefanha@redhat.com>,
	<Artem_Mygaiev@epam.com>, <xen-devel@lists.xen.org>, <olekstysh@gmail.com>,
	<Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO
 backends
In-Reply-To: <e9df2014-b691-1ee4-7fcd-03912d0aed59@quicinc.com>
Message-ID: <alpine.DEB.2.21.2109141127500.21985@sstabellini-ThinkPad-T480s>
References: <87v94ldrqq.fsf@linaro.org> <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s> <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org> <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org> <20210906022356.GD40187@laputa>
 <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s> <e9df2014-b691-1ee4-7fcd-03912d0aed59@quicinc.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e198edbf-544c-4c33-9159-08d977f8f889
X-MS-TrafficTypeDiagnostic: BYAPR02MB5157:
X-Microsoft-Antispam-PRVS:
	<BYAPR02MB515783CCE042A8F0477F9CE1A0DB9@BYAPR02MB5157.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Jsa4pVwidVDbBVcMI/6ijrZRo0iUCfVUT1/x3/beXYsC9ThqCg8zONMfNQf4FYUpjI1PEXlZmazRJA+Bz7xcCmQInOuQjg6yDtY5mb7i4kxKBUvkGNHkLSLolfY6treIATV7cStUPUxXMHdZ8hdYqWK6cigBKC8aEo635lN4kYLmcuOTXOLaCFDMvtroUNjJPZuzxarF3QX37V7Gr0QdFt60cVI6++tOCjzLc+O+yDAHoQVdaFGi/WKtTJ08vZBrmwSxapAoSjhiVmzqC6EBuCvcunCllXtFl7EYFcrfYm50APOC44T8pb3y7Q37lcVztUJOMwX1rC+TYn/y3oxjJLB2ohChsD4cg2D7G3UPOw8Tdypy8DIlorix2NVigJTAU+38kROvSkUmQInNR4/Ic3RhxcqMMSyAT/yQ1trBwQgZcmJvEjZsP2PR7nOzCu+j4MyM41EMfnLiTQ7vZpgeD8Wt2XSNLfHGtZg+cZ95CN8mOG43r9YCP+ZcNGKqRFvN2nP68H8mlcfg1rU1a6T3EO4u+rkvEj27o7fWdEep8FzZdhxBcIXBor3J6g1lrNvAcReeNyiL3u7eoChZrbP9dwurWB8o1GDPD4QB4VMDAJt8rJ9rVCDj7QsgRzXIvJOQ/l92hF0t5+Dpwu6+CbU4VvqE+kmvNgKU+exZ6+U1VVGJCucUv1rJMeTJz2knUK0ojIKkEU8A0o9d8ht4i4Z9Ug==
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(7916004)(4636009)(396003)(346002)(39850400004)(376002)(136003)(36840700001)(46966006)(356005)(47076005)(33716001)(8676002)(26005)(45080400002)(7636003)(426003)(2906002)(7416002)(9686003)(82310400003)(4326008)(336012)(5660300002)(70586007)(36906005)(83380400001)(6916009)(54906003)(8936002)(316002)(36860700001)(82740400003)(478600001)(70206006)(186003)(44832011)(9786002)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 03:29:05.4076
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e198edbf-544c-4c33-9159-08d977f8f889
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DM3NAM02FT025.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR02MB5157

On Tue, 14 Sep 2021, Trilok Soni wrote:
> On 9/13/2021 4:51 PM, Stefano Stabellini via Stratos-dev wrote:
> > On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
> > > > the second is how many context switches are involved in a transaction.
> > > > Of course with all things there is a trade off. Things involving the
> > > > very tightest latency would probably opt for a bare metal backend which
> > > > I think would imply hypervisor knowledge in the backend binary.
> > > 
> > > In configuration phase of virtio device, the latency won't be a big
> > > matter.
> > > In device operations (i.e. read/write to block devices), if we can
> > > resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue
> > > is
> > > how efficiently we can deliver notification to the opposite side. Right?
> > > And this is a very common problem whatever approach we would take.
> > > 
> > > Anyhow, if we do care the latency in my approach, most of virtio-proxy-
> > > related code can be re-implemented just as a stub (or shim?) library
> > > since the protocols are defined as RPCs.
> > > In this case, however, we would lose the benefit of providing "single
> > > binary"
> > > BE.
> > > (I know this is is an arguable requirement, though.)
> > 
> > In my experience, latency, performance, and security are far more
> > important than providing a single binary.
> > 
> > In my opinion, we should optimize for the best performance and security,
> > then be practical on the topic of hypervisor agnosticism. For instance,
> > a shared source with a small hypervisor-specific component, with one
> > implementation of the small component for each hypervisor, would provide
> > a good enough hypervisor abstraction. It is good to be hypervisor
> > agnostic, but I wouldn't go extra lengths to have a single binary. I
> > cannot picture a case where a BE binary needs to be moved between
> > different hypervisors and a recompilation is impossible (BE, not FE).
> > Instead, I can definitely imagine detailed requirements on IRQ latency
> > having to be lower than 10us or bandwidth higher than 500 MB/sec.
> > 
> > Instead of virtio-proxy, my suggestion is to work together on a common
> > project and common source with others interested in the same problem.
> > 
> > I would pick something like kvmtool as a basis. It doesn't have to be
> > kvmtools, and kvmtools specifically is GPL-licensed, which is
> > unfortunate because it would help if the license was BSD-style for ease
> > of integration with Zephyr and other RTOSes.
> > 
> > As long as the project is open to working together on multiple
> > hypervisors and deployment models then it is fine. For instance, the
> > shared source could be based on OpenAMP kvmtool [1] (the original
> > kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
> > kvmtool was created to add support for hypervisor-less virtio but they
> > are very open to hypervisors too. It could be a good place to add a Xen
> > implementation, a KVM fatqueue implementation, a Jailhouse
> > implementation, etc. -- work together toward the common goal of a single
> > BE source (not binary) supporting multiple different deployment models.
> 
> I have my reservations on using "kvmtool" to do any development here.
> "kvmtool" can't be used on the products and it is just a tool for the
> developers.
>
> The benefit of the solving problem w/ rust-vmm is that some of the crates from
> this project can be utilized for the real product. Alex has mentioned that
> "rust-vmm" today has some KVM specific bits but the rust-vmm community is
> already discussing to remove or re-org them in such a way that other
> Hypervisors can fit in.
> 
> Microsoft has Hyper-V implementation w/ cloud-hypervisor which uses some of
> the rust-vmm components as well and they had shown interest to add the Hyper-V
> support in the "rust-vmm" project as well. I don't know the current progress
> but they had proven it it "cloud-hypervisor" project.
> 
> "rust-vmm" project's license will work as well for most of the project
> developments and I see that "CrosVM" is shipping in the products as well.

Most things in open source start as a developers tool before they become
part of a product :)

I am concerned about how "embeddable" rust-vmm is going to be. Do you
think it would be possible to run it against an RTOS together with other
apps written in C?

Let me make a realistic example. You can imagine a Zephyr instance with
simple toolstack functionalities written in C (starting/stopping VMs).
One might want to add a virtio backend to it. I am not familiar enough
with Rust and rust-vmm to know if it would be feasible and "easy" to run
a rust-vmm backend as a Zephyr app.

A C project of the size of kvmtool, but BSD-licensed, could run on
Zephyr with only a little porting effort using the POSIX compatibility
layer. I think that would be ideal. Anybody aware of a project
fulfilling these requirements?


If we have to give up the ability to integrate with an RTOS, then I
think QEMU could be the leading choice because is still the main
reference implementation for virtio.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 04:50:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 04:50:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187198.335978 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQMsC-00048l-Jh; Wed, 15 Sep 2021 04:50:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187198.335978; Wed, 15 Sep 2021 04:50:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQMsC-00048e-GZ; Wed, 15 Sep 2021 04:50:16 +0000
Received: by outflank-mailman (input) for mailman id 187198;
 Wed, 15 Sep 2021 04:50:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tFB7=OF=epam.com=prvs=9892afeb01=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQMsA-00048Y-Dj
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 04:50:14 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ca53ba9d-58b0-4775-a3f6-4da87a6a5be6;
 Wed, 15 Sep 2021 04:50:13 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F4nCam016051;
 Wed, 15 Sep 2021 04:50:09 GMT
Received: from eur02-he1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b3a4ng020-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 15 Sep 2021 04:50:09 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6067.eurprd03.prod.outlook.com (2603:10a6:208:15e::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 04:50:06 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 04:50:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca53ba9d-58b0-4775-a3f6-4da87a6a5be6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lYFbiSrE68zkQMFUiblj/BkwSqmrdZuRN5GgC9qnzFFecjj+XpJtC8glmvbWYuiMk9AZ8yq2SLf0UeUQzwK0dv92yhHDdo4SfmNJqGnzCXvmFSMGRQpHRS5j+bUQeG1F5ZxBHppKCiaAuekFeYSVyPwSgm6G8YEAPU/Q/5iy6xhdJCeEaKCYZGpKSz6KT3FgT2DmKz2YkhQ3dAAxmY8LTDAw4eU7vioCww1OShWOkoDWWJi2WO8LXK9zJPX4S75ta+Ei59D3ynvsjpe0gj9Y8keGh+Yl5lB+oJTvfaAEh/apFatSyQxfUpp+afrrdAOqptm9/KF0ueWlHOyBYV2dqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YFEIHkrqVx3dzdXzYlJfTvgvrLVAsNPXYEvsCpIhXB4=;
 b=defh8b78xl8NtjsLGCqX2Dn2QCrGKlvJ5ztywNY9NfEKdpA52YHXErNLk7frRIVTyhT29F6Fb9uzQBvsJ2BSWmxqa5V/sauRlrjz+BdbnSO0IeW3Qrx32qmiPuXqWRDYLyLjv7w/OKczpnS5FWtCS21fRBJnpGWVvs3Pf5OqsLefFCHLG70QC4IAulO/KkNHlXepNt3j/hosYzFt2t0XxPXmlSsqH75jfc5OYizlo2zCEF+T6rKvwYaYYSMV8RyneB3iC1KxTflQx4TCsPcGzi165E4AWc2XlH3/M0NlrtYKMn3/Uk5OBlykafTjMEsNKOj00CYht9Yhz3R13/AvKQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YFEIHkrqVx3dzdXzYlJfTvgvrLVAsNPXYEvsCpIhXB4=;
 b=T4q49g6557HI+fwE2gJiejbRSSYWo8PBn5jRMGKHdIumrRLtTIR16k+/Ck3T8jD9ukLgDDziTjKzh+DzKJue8H9dFVCD7TSZ38LZ2EqvoIeZxIrLw/+KejEVqVpKX2s9mWo9LaYrlTrnYzPrCW+l6A5Q6YvMuX8q2KsqFBX0v67zPlab+CXauc4nu7X9R8EWMYwFw032tyzMLt4228Eko8h2iF4sv8BEmolEDJ+mSTzrnTNYKr0p2bdVmguIuBx6YztVuLM9n/V4kzjoRJw1qb2oMcySe5L3I0e7o3MrnbbLeQUO44VP+lBOqTLwgc1k4GB55rObTZMO9RN8yxVQ2A==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: 
 AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKucAxCAgAEt4ACAABanAIAAAwIAgAABgYCAAAHPAIAAAcgAgAABzYCABkt1gIAAsi4AgABJ7oA=
Date: Wed, 15 Sep 2021 04:50:06 +0000
Message-ID: <4e5db370-76ac-d72f-f641-0124518b17c6@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <247bd41c-98e6-f898-8216-e36b22158636@xen.org>
 <8db7ab42-d361-5b20-c648-7af9d0cdaad9@epam.com>
 <d7ecd474-fe0a-2bca-717e-cb82c89358ea@xen.org>
 <ad8ef71c-a834-daff-7ad8-b7c3f718a3e2@epam.com>
 <6810eefb-b6d0-9557-7bdd-80ac381e467b@xen.org>
 <7263f268-7f90-e561-4679-f78185c88cd8@epam.com>
 <0eb861db-eaf4-0919-4f54-64f448f1cdfe@xen.org>
 <52da7955-ead7-f404-6e51-7d1ee7bc3142@epam.com>
 <a4343670-2d9a-cec2-f093-aa3b0a5e0657@epam.com>
 <alpine.DEB.2.21.2109141716170.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109141716170.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8abe54ef-2e29-4f1b-ce32-08d978044a0e
x-ms-traffictypediagnostic: AM0PR03MB6067:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB606752F5352498D34467433DE7DB9@AM0PR03MB6067.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 U0hrj6JuWIDyklB5QYetOqerMWksV4qEOZysAJ2qxV+WWYM++Fcfj5ZiJOjtbM5bUZOkqGpu7S5cYlNF+FWpNd/jKr9wb2//yVeHKrn1KswXCV7iVigV6YVzRbGEnGt41N/MKecqzpOv1xAED0UwBKXUFFPn/C0Fr9B5urVw2VRgvxjGvB6RYlnneU9PhcZvBNFkMWvnDXZy5wOlPRldeGNqAfaYYGmdEf5lkRgA2J473frP+mw7VTlgmduAmhpvBMY54h3EHK5tYbCggNXp7IrPF1R5so77EKfWSkc5eu0emymQn469XfH2pMDZJa2vkKzH+BkC02ZxXY72I7cvS/jPXOskeW65A05eD6OAf5bvFf7ldIIK1b4tsTCgoyDjJYMLKtpDA1wyAXdV/zopSKuzbm3x/fbcECElo8ufZ6Eaoskt4yZ0UhY1V0cTpb5BzzL+LBiQQx8eSeZhw+rDBgY7H1PF6vYBZB1A9sczC/MwemNiJp/9T/jaCyHWWBkp1t8/l7mVlh6Hpbh95iEncxq+wGXbflwXcnvuxkY/QLelTh7E+oCVzX0gPQSKUjFV0KMelqbbAWcrThYHnrTcBm/+9TXRoKKC5u8vuwnlqj+r2tXopUvlQyhjYtLUI4mljJn13RJ8pHV+QjohW1VU+GNdp2//BeW8ojVSY4TX65pJuvoABHrczPPoipyjw9Rrg3fNTWw/iRJI1D8/gMNu9SnlHYrWRWOEn+ZfTtUwIRvaMhK7OPmYhxZepjoH1Y9D
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(31686004)(66946007)(38070700005)(6486002)(64756008)(66556008)(66446008)(76116006)(83380400001)(66476007)(54906003)(71200400001)(8676002)(6512007)(86362001)(91956017)(2616005)(508600001)(5660300002)(8936002)(4326008)(53546011)(2906002)(6916009)(55236004)(6506007)(122000001)(38100700002)(26005)(31696002)(36756003)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?cDdZdUI5QU1Ld2R0NkpYU3pRakovNmFBK0VRN00xOXNlaDFJcXVtTXRQaXpZ?=
 =?utf-8?B?T3luNlY5MnVaR1N6d2cyaGMwbzgreGxXUlZldnM5eDNMemxGeWVaamdSMXJy?=
 =?utf-8?B?bmxtNGs2cnhFUWt2d3NPWXhodWxZQVVtZkFSaytlTEJpdG5CYzJSeVVqTE5r?=
 =?utf-8?B?VVpWeXl1ZlZWSGlVV2JhTkg1dEF4VytEZGlTTmVUOGgzMVRwV1RJZWRVWFZv?=
 =?utf-8?B?OWEyLzVGQzVlMDNvVmRySmtyOGxZOGpWaEpUL29RSUtlSjdEcXl3K1pLTTkz?=
 =?utf-8?B?MUpIRVpraVI3VVlPaDFmK2dDTzFTbzhqUHN6REhCVlNWaFZmVytpNEFWd2pv?=
 =?utf-8?B?Umw3SXBMVE9HTHFwYXZDbko3KzB5K0FwRTZ0dlBxZmtMTDBUWXZwVGdEeWVs?=
 =?utf-8?B?elFmUWhiYzdQak5PYVFSRlRvanRILzJTOVArVVZvTERZWEFWWHdZSXFDeWRX?=
 =?utf-8?B?NnVPSWRRWXN4NG1GT2QzTk9ULzRrbytKTUFxUDR5UVgxT2pLL0FObC9QNy95?=
 =?utf-8?B?OGFiS3dxZ2pTb2YzbngwWENTU2U4aFVFT2pVUjROTEJFeXlhZ2lXWmV1SEla?=
 =?utf-8?B?NGV6WUFrQlZmaXdIU0FxWEhML1FKWTVrN0pkb2xGYW55cGYrWmRiZWhuaWdV?=
 =?utf-8?B?aXVCcW9RMkMxdlcvQWJxZTVIWWdHWXpmU2l5ZUJUSy8weHA4Y0VjcDdPaXgx?=
 =?utf-8?B?OWQwY3NldzlaTnhKeGpOTUtIODBuWkdNNUs1RG9rVHZyUkw5UWp6bUo0ejM0?=
 =?utf-8?B?dDBTZXF3MDg1QjdxNmNPVTNwaTRuSkRpWUhJVGh2THRMb1pvK0ttMXpiaWtD?=
 =?utf-8?B?b0k0OUpCdDQ4dDByb2JqU1FZOG1vdjBDeVp4bWFwMklrQ0V4L2Mxbm1PL2s1?=
 =?utf-8?B?QTZUQnNXWWt4R1ZmYXV0TE5Ja0tKZVRHd05TY3BJZEQveXdWaEpDV0FNR2FT?=
 =?utf-8?B?bWV0bTB3dkR2emd5NWkvQVgzWDRQY1pKeGxtN2lqTU9ibHlSN0dKd3F6OGRJ?=
 =?utf-8?B?bU5Mc1k5VXhhaW1EZ2Yyd3FRYW5ycFIwNk51VkkwUXJzM2NzcWFjNE1aLzlw?=
 =?utf-8?B?VEFGc3RaK3VTR1haYmlpdXNuMkM3WDF1Zm9GdUZERmN1c1dXTXZMZmFXT0Fy?=
 =?utf-8?B?UVYrcGQ5UUlQRlFlNEw0TGhSK3hBbUs0ckNHaDlQQVFQSWh2bHREdDFFSnJ0?=
 =?utf-8?B?Wml3V1d4WmllS2U5cGhSdlQwSGk3K1RFSHVhdDZPKzF0U0JXYk9sSU1RTWxU?=
 =?utf-8?B?dUowNzg1UThPeU9LNmNCTDNZZTN3SWdxR2x3YkFJdE9nT0lDTHhkTXlqT2R1?=
 =?utf-8?B?aVlaZnlpVWRaMWI0R2U0c0d4cEhLZDNGLzlzUFJyRXpubkROcmNXL2E5TEZY?=
 =?utf-8?B?ekhGTHlmcFRNQTNTTkdCWGJBamFPQTE0SUVUNXFQeW1Id3Uvajh4bFcyc2tV?=
 =?utf-8?B?a2VRWm9naVpHZnYrSkdXSnErVTdrUGQwWVE0aDBidXdzQ2RqeVdaSWZ2Sk5L?=
 =?utf-8?B?SkJkcS8vT0Z0OFlyWVpyY2tUZS9lTTQ4clpYRTZRSUNnd0lFa3ZSaFlWV0RI?=
 =?utf-8?B?aTIvUmZZdnBPSkdWQU5FNEZSZ2ZFOFVUTjR5U2hFNklpV0hNNmZwR25JUDlB?=
 =?utf-8?B?Q0l5cmpZOGtyM2EyKzZGK3hTbDJRWkhnTDJFcXJPOVBwSUxsUzNsNjMrMS90?=
 =?utf-8?B?Q3BOTmJ3Wmo5UCtid0ZiWDdZM01DZDZpcXFpemEvTmdSVS9FajRYczdYeE5R?=
 =?utf-8?B?Tnlvc0k4L1lMdXgwVVNta05NZHp0YlVlS2RKa2RxUDQ1K3RSSG9Oc0dTbjB3?=
 =?utf-8?B?TFplb2VRNk1CSUhoZUVVZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <77B8F6412F73A04C94A3004C389D98D9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8abe54ef-2e29-4f1b-ce32-08d978044a0e
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 04:50:06.5809
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: RaWENQe2/FKxxFSmxfk/i6Aai6Y9DdDxKGG9G9bnk2F+drbuLlk/LOi9Kz5xlzkSrTMMoLSDBg2K0AwuVG7Pk9a4jI7NAddolqRbelf0fdaLhkj7GAC5Z1U0ZOmbOfEq
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6067
X-Proofpoint-ORIG-GUID: JE7cnbYOniPaHcYiKUpltSoIIoBXpG8c
X-Proofpoint-GUID: JE7cnbYOniPaHcYiKUpltSoIIoBXpG8c
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_10,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0
 lowpriorityscore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 phishscore=0
 spamscore=0 malwarescore=0 suspectscore=0 clxscore=1015 priorityscore=1501
 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109150029

DQpPbiAxNS4wOS4yMSAwMzoyNSwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUdWUs
IDE0IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4gV2hhdCB5
b3Ugd2FudCB0byBrbm93IGlmIGhvdyBtYW55IHRpbWVzIHJlZ2lzdGVyX21taW9faGFuZGxlcigp
IHdpbGwgYmUgY2FsbGVkIGZyb20gZG9tYWluX3ZwY2lfaW5pdCgpLg0KPj4+Pg0KPj4+PiBZb3Ug
aW50cm9kdWNlZCBhIGZ1bmN0aW9uIHBjaV9ob3N0X2l0ZXJhdGVfYnJpZGdlcygpIHRoYXQgd2ls
bCB3YWxrIG92ZXIgdGhlIGJyaWRnZXMgYW5kIHRoZW4gY2FsbCB0aGUgY2FsbGJhY2sgdnBjaV9z
ZXR1cF9tbWlvX2hhbmRsZXIoKS4gU28geW91IGNvdWxkIGludHJvZHVjZSBhIG5ldyBjYWxsYmFj
ayB0aGF0IHdpbGwgcmV0dXJuIDEgaWYgYnJpZGdlLT5vcHMtPnJlZ2lzdGVyX21taW9faGFuZGxl
ciBpcyBub3QgTlVMTCBvciAwLg0KPj4+IE9rLCBjbGVhci4gU29tZXRoaW5nIGxpa2U6DQo+Pj4N
Cj4+PiAgwqDCoMKgIGlmICggKHJjID0gZG9tYWluX3ZnaWNfcmVnaXN0ZXIoZCwgJmNvdW50KSkg
IT0gMCApDQo+Pj4gIMKgwqDCoMKgwqDCoMKgIGdvdG8gZmFpbDsNCj4+Pg0KPj4+ICDCoMKgwqAg
KmZpbmQgb3V0IGhvdyBtYW55IGJyaWRnZXMgYW5kIHVwZGF0ZSBjb3VudCoNCj4+Pg0KPj4+DQo+
Pj4gIMKgwqDCoCBpZiAoIChyYyA9IGRvbWFpbl9pb19pbml0KGQsIGNvdW50ICsgTUFYX0lPX0hB
TkRMRVIpKSAhPSAwICkNCj4+PiAgwqDCoMKgwqDCoMKgwqAgZ290byBmYWlsOw0KPj4+DQo+PiBJ
IGhhdmUgdGhlIGZvbGxvd2luZyBjb2RlIG5vdzoNCj4+DQo+PiBpbnQgZG9tYWluX3ZwY2lfZ2V0
X251bV9tbWlvX2hhbmRsZXJzKHN0cnVjdCBkb21haW4gKmQpDQo+PiB7DQo+PiAgIMKgwqDCoCBp
bnQgY291bnQ7DQo+IGNvdW50IGlzIGluY3JlbWVudGVkIGJ1dCBub3QgaW5pdGlhbGl6ZWQNCkV4
Y2Vzc2l2ZSBjbGVhbnVwIGJlZm9yZSBzZW5kaW5nIDspDQo+DQo+DQo+PiAgIMKgwqDCoCBpZiAo
IGlzX2hhcmR3YXJlX2RvbWFpbihkKSApDQo+PiAgIMKgwqDCoMKgwqDCoMKgIC8qIEZvciBlYWNo
IFBDSSBob3N0IGJyaWRnZSdzIGNvbmZpZ3VyYXRpb24gc3BhY2UuICovDQo+PiAgIMKgwqDCoMKg
wqDCoMKgIGNvdW50ICs9IHBjaV9ob3N0X2dldF9udW1fYnJpZGdlcygpOw0KPj4gICDCoMKgwqAg
ZWxzZQ0KPj4gICDCoMKgwqDCoMKgwqDCoCAvKg0KPj4gICDCoMKgwqDCoMKgwqDCoMKgICogVlBD
SV9NU0lYX01FTV9OVU0gaGFuZGxlcnMgZm9yIE1TSS1YIHRhYmxlcyBwZXIgZWFjaCBQQ0kgZGV2
aWNlDQo+PiAgIMKgwqDCoMKgwqDCoMKgwqAgKiBiZWluZyBwYXNzZWQgdGhyb3VnaC4gTWF4aW11
bSBudW1iZXIgb2Ygc3VwcG9ydGVkIGRldmljZXMNCj4+ICAgwqDCoMKgwqDCoMKgwqDCoCAqIGlz
IDMyIGFzIHZpcnR1YWwgYnVzIHRvcG9sb2d5IGVtdWxhdGVzIHRoZSBkZXZpY2VzIGFzIGVtYmVk
ZGVkDQo+PiAgIMKgwqDCoMKgwqDCoMKgwqAgKiBlbmRwb2ludHMuDQo+PiAgIMKgwqDCoMKgwqDC
oMKgwqAgKiArMSBmb3IgYSBzaW5nbGUgZW11bGF0ZWQgaG9zdCBicmlkZ2UncyBjb25maWd1cmF0
aW9uIHNwYWNlLiAqLw0KPj4gICDCoMKgwqDCoMKgwqDCoCBjb3VudCA9IFZQQ0lfTVNJWF9NRU1f
TlVNICogMzIgKyAxOw0KPj4gICDCoMKgwqAgcmV0dXJuIGNvdW50Ow0KPj4gfQ0KPj4NCj4+IFBs
ZWFzZSBub3RlIHRoYXQgd2UgY2Fubm90IHRlbGwgaG93IG1hbnkgUENJZSBkZXZpY2VzIGFyZSBn
b2luZyB0byBiZSBwYXNzZWQgdGhyb3VnaA0KPj4NCj4+IFNvLCB3b3JzdCBjYXNlIGZvciBEb21V
IGlzIGdvaW5nIHRvIGJlIDY1IHRvIHdoYXQgd2UgYWxyZWFkeSBoYXZlLi4uDQo+Pg0KPj4gVGhp
cyBzb3VuZHMgc2NhcnkgYSBiaXQgYXMgbW9zdCBwcm9iYWJseSB3ZSB3b24ndCBwYXNzIHRocm91
Z2ggMzIgZGV2aWNlcyBtb3N0IG9mIHRoZQ0KPj4NCj4+IHRpbWUsIGJ1dCB3aWxsIG1ha2UgZC0+
YXJjaC52bW1pby5oYW5kbGVycyBhbG1vc3QgNCB0aW1lcyBiaWdnZXIgdGhlbiBpdCBpcyBub3cu
DQo+Pg0KPj4gVGhpcyBtYXkgaGF2ZSBpbmZsdWVuY2Ugb24gdGhlIE1NSU8gaGFuZGxlcnMgcGVy
Zm9ybWFuY2UuLi4NCj4gSSBhbSBPSyB3aXRoIHRoYXQgZ2l2ZW4gdGhhdCBpdCBkb2Vzbid0IGFm
ZmVjdCBwZXJmb3JtYW5jZSB1bnRpbCB5b3UNCj4gYWN0dWFsbHkgc3RhcnQgY3JlYXRpbmcgdG9v
IG1hbnkgdmlydHVhbCBkZXZpY2VzIGZvciB0aGUgRG9tVS4gSW4gb3RoZXINCj4gd29yZHMsIGZp
bmRfbW1pb19oYW5kbGVyIHJlc3RyaWN0cyB0aGUgc2VhcmNoIHRvIHZtbWlvLT5udW1fZW50cmll
cywgc28NCj4gYXMgbG9uZyBhcyBtb3N0IGVudHJpZXMgYXJlIGFsbG9jYXRlZCBidXQgdW51c2Vk
LCB3ZSBzaG91bGQgYmUgZmluZS4NCg0KT2ssIGZpbmUsIHNvIEknbGwgaGF2ZSB0aGlzIGNoYW5n
ZSBhcyBhYm92ZSBpbiB2Mi4NCg0KVGhhbmtzLA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 04:58:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 04:58:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187206.335989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQN0X-00050y-GV; Wed, 15 Sep 2021 04:58:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187206.335989; Wed, 15 Sep 2021 04:58:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQN0X-00050r-Cn; Wed, 15 Sep 2021 04:58:53 +0000
Received: by outflank-mailman (input) for mailman id 187206;
 Wed, 15 Sep 2021 04:58:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQN0W-00050h-Kv; Wed, 15 Sep 2021 04:58:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQN0W-0005NZ-Ak; Wed, 15 Sep 2021 04:58:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQN0V-00037i-Ud; Wed, 15 Sep 2021 04:58:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQN0V-0007hK-Tw; Wed, 15 Sep 2021 04:58:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1LbMmWbYUAX2/ZTjyO073rNrvzSMW7OloOod8vYtrHw=; b=ltCA3Lz06spC/0bLRdKlXBiMWx
	zZgKOAUiyE47t8qMXlSYxg/aK2TBt2OVbjeHomrOQDm3+H96+MabVWZAtOypimPezqmf6mefzwRAD
	KTf0U7MrYsdwpUdBKdIq+lLxZNt8E4VCnY22BZeS6mr5vyvMeSqTi9cEXYat1eoCb8HI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164989-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164989: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=487975df53b5298316b594550c79934d646701bd
X-Osstest-Versions-That:
    xen=a89bcd9737757e4d671783588a6041a84a5e1754
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 04:58:51 +0000

flight 164989 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164989/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  487975df53b5298316b594550c79934d646701bd
baseline version:
 xen                  a89bcd9737757e4d671783588a6041a84a5e1754

Last test of basis   164966  2021-09-13 13:02:44 Z    1 days
Testing same since   164989  2021-09-15 01:01:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Penny Zheng <penny.zheng@arm.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a89bcd9737..487975df53  487975df53b5298316b594550c79934d646701bd -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 05:30:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 05:30:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187216.336003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQNVU-00012u-TO; Wed, 15 Sep 2021 05:30:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187216.336003; Wed, 15 Sep 2021 05:30:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQNVU-00012n-Q8; Wed, 15 Sep 2021 05:30:52 +0000
Received: by outflank-mailman (input) for mailman id 187216;
 Wed, 15 Sep 2021 05:30:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tFB7=OF=epam.com=prvs=9892afeb01=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQNVT-00012f-Jk
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 05:30:51 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3bbad2bd-cda1-4014-a247-9113b253ef43;
 Wed, 15 Sep 2021 05:30:49 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18F5J8gT003443; 
 Wed, 15 Sep 2021 05:30:45 GMT
Received: from eur01-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2053.outbound.protection.outlook.com [104.47.1.53])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b3ajy00sk-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 15 Sep 2021 05:30:44 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7347.eurprd03.prod.outlook.com (2603:10a6:20b:268::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 05:30:41 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 05:30:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bbad2bd-cda1-4014-a247-9113b253ef43
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VOH3gO/KC0caIx72XxhR+ZUu6I6/zZJ5pX34lC7MeKah4ZKP+ZgAho/pZ4RbfTwBBCAfSABtkXXUhqpTX7IpBG/Bo2iXw1CpHBJf3pB7ETJgttveWxPtNzFYxAZwn6dMIh6uJm85F3bwvTKRCob+pfR5GmBQZGFox2SKlquHiVl/MUS9SVDlfA4TZhctTz161m3bmCH/aOqmta9UYBJRY9AgXwmff++qzFyF9SKq3eiL81bjrg8lxhBzAmOtFmR/78g3Vew0D8TB/EZZk66YlvN+UA5pBoictPVpt42e/0evoIWy+D72ALrnh9fFOpYieZKogOFHkCYNb3Anom7xgg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+Js1ULUN0xbIjgVEqSHLxfhmwuL2sC6W2PAFSur+PnA=;
 b=bZyX9739LCjkvSgQfwDoXY0vNwe7+HZLL9JQn225rR+GYup7W6kjNQCyprwQYTiOCTFUnJucqcEn9XdDr2vDp/BL0RJd4OLWX/Cpw+TkI2JQ0cecBG3t2qbkhS38oDdfyV4q4QEDm+A3ayEenaGUpzJnddI9sy/bxIPSwBIxnYujnTEgh8da58Z/GngI9hWwXHyogGcOSleSIJkZyUcG7d0/r7gNt1z3dyJr1/Y5oDNczT1lH1bmx7RdP0Nh9+kn+wdl+sowc/nlxYxpzjkC5vv8UtACV0GPCJRmzsTeDiTQZNfWiVWtLomD7wNS031LDISlfuy8lUgm+BDe3275Ow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+Js1ULUN0xbIjgVEqSHLxfhmwuL2sC6W2PAFSur+PnA=;
 b=kdaGYcjnD4mvVhFB8A3C3/nGUUV8MsIPnlOLketTD56PDENnPREsfqmjxkDtp2hk6swuQPuXcAvNSsz7GDSadVIjRH5ordSnhgTrjSTiU7hzFyUZWscmun3vawBWPwezUHQxJQx2OPUIwjkWKxZ9mSYa7r98NfB/uvN624Cuuh1XnJ3F5BhfDXaAM7c6owRuM5edoO2EcF4Wd4/OZf2TH41VMqqrmDRP/rt6NbGmC/YmXd9cI7T7dFVd5n9ZQ4KRr4D2JyTNt7SExjyG+DYrGx2p76qnn/9qEu6FGRjHmxg2CQDyCFLnXon09fzF24P/rMeqYuVmkKIhF1SRqZF0Qg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKudvyQAgAXn/ACAAP1GAA==
Date: Wed, 15 Sep 2021 05:30:40 +0000
Message-ID: <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
 <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
In-Reply-To: <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 640899f4-d5b5-41a4-c6f5-08d97809f514
x-ms-traffictypediagnostic: AM9PR03MB7347:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB73476C7EC569150D63C11041E7DB9@AM9PR03MB7347.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 yV0qHo9iWYvN9XqPXgAWKKt2M2BfKzYIiqFBXCG0YtXmGxrMymMCTl1spWSLMAv7PNRXcJROIyRya/mAgDN97kKbL58HfvyZYg6O4ptG60stgyB34wEcNdw31CbfItgV3i18hDGawCeagGCmszdYwO4x7gfwO0YUJVUZXQcIAFNNqTKqt0CJRM5oDt5aCTPnQAWGYQF7lbTWWmmJvLU6aJA8XF6E4aX+0Pk3LLvMrIfNSP2a8pjqKTHmxx52SdF7o+S23NKT6T26fD470htA51YXLHhmlWh/QAJy38UL6jc1zvvpqbIaxocooSLFvd517JbHFa98Pc8rYd9url4iDqApxROM0cBAWK2bTJBpVfGki1TtAcjynPGLOSkUTNSnDsfp67kCNQyHNxuDdV6TeNrLsuoaXfVKLaLEK2kLUfDFT1ynIsqLS+pQ26XzIIj+LiFKMnCMF+WJxCJ5DXgNJ0AD/aEIgTopderSKL1h+eA5FYSMduA3n4K4AOVlVj1PmAvkmrIUm/8jr35KbqDMh4K7ywZvLbTIUUMBMZZyS+LZnQ3RFdQcZd8BLw6C+yKXb9Oce+smOgZ2+Nsj/yFGZOtz4hnje37wO/PSpxAp86+am0EnRR0SHIAIK/XYGcgnJ2EmJKL45tb5Pb1mlBQmScdOjoF5OYzgZBAPshWBt5RQFKcHkZTYkkX3OsshcpUEx94b1z3we3JH3JvUtHCu+wTa/KeMPVEpgji9ivThQ2FNLLUT5R1S8iubTWJEZcjp
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(366004)(36756003)(38070700005)(6916009)(31696002)(2616005)(5660300002)(4744005)(8936002)(54906003)(6486002)(71200400001)(478600001)(122000001)(186003)(31686004)(2906002)(83380400001)(4326008)(66946007)(53546011)(91956017)(76116006)(86362001)(316002)(66556008)(6512007)(66476007)(6506007)(64756008)(38100700002)(66446008)(55236004)(26005)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?QlA3SjZEQitwUEprMEwxcGFtQmJYK3VZMDg4MGh6a2xYU1JMQmh6bHpkeG13?=
 =?utf-8?B?MmNZK1k1WXBpZEZqME55V2Z0Z1dVdStyVGhQek96OUNWOGpJNEJCMWxFdVZ4?=
 =?utf-8?B?aXlMTnhLYm0wc1BRa0xqSDM5R21NcndwbENUOVBic09zVUN2ZGF2T2NVU0xG?=
 =?utf-8?B?MlZ6c1BXWnhtMVRJM2pPelE4RDBHSkZianJQdVhtMElTTXJvMHVRUWhFbGVn?=
 =?utf-8?B?Tlo2Q3VFelBpMUpFZmRqZm51TEUwZnNZMmVaa1hYQ0I5cmdRa01CbklsSTIx?=
 =?utf-8?B?b1FBMzdGS1lkNDdEZEM5OXBweklWQXNxdUZscVdIbVV1bUlvK2pKYmRvS0J0?=
 =?utf-8?B?YStXZE52TGRmRS9OOUg2anFWQUdYdk1sMm5BYUpGL1VkRDN6elE5MFNQMnF2?=
 =?utf-8?B?Q0FVb1VBUC9WcjVMSkh1V3JqMkQ4bUZ4bXlCdVpEWDVRTjFQcFJKNnlWV1BU?=
 =?utf-8?B?eHQwd1A4SW5SOFNlcTNTbFJKS2lWUVQ1RjZ1MTl3VXFJTDRoN1FQc3pKUTFB?=
 =?utf-8?B?MGpSdWVWK2xUUEhIR3ExcFQzdmlCOGx5NDE2bzJPY2Z1bGs0NmUwWWJtTUtR?=
 =?utf-8?B?WU5xQUtvK2owbWR0MUdwM2ZiV0I1NW1LMEhLalE0ZFcwTGdZYXFEYWxFUkZL?=
 =?utf-8?B?bzNodXBIZmIrcXI2aUVJeWF3U2lEeHkyNXp5K2UxTDNVcUl1ZEE3SElBeHNO?=
 =?utf-8?B?ampUbzl1RnJhRXRwcGFkWDBPRzBMSDI4eisxZFYvSDByR3NRZEp2UTFQWWRp?=
 =?utf-8?B?QjZHOXozdmFwRVpWSTRmZkhWWlhHQkZDdXA3bllXcUFFZGxsOFVyQnpscURQ?=
 =?utf-8?B?bDlhQjUwYXByZ0FLWlJPUUNiNkEyUEN1ZmdJTk5UVVAraVV0VTRZRkY5bDRW?=
 =?utf-8?B?dVNvRjg4blI5eXV5OU9CNS82bW90RnZiYlEwU0xlNWc3ZFQ4L0U1czJ5aEVS?=
 =?utf-8?B?cnpIV2pITnlRV2xGY3ZKVmZQU2FjTmZFNVZvb2xHY2k1Zi9HSjIrRUpZVjJ6?=
 =?utf-8?B?MGZRREJIN2FzUzhRN3dUSWovbzM3UXpkWGFBalNEcSszZXp1UlhlYithbUtE?=
 =?utf-8?B?T2R5TE1VNEJyVjJtREwyMW1uWU42YXE1RytZTDBKRjVLZlVIU2M1YnNDbTZS?=
 =?utf-8?B?SHhucXRHS05TVjVIcysvZmFMaVVCaW9BOWVjL0kxTzFORm02VjVlMFU0L01r?=
 =?utf-8?B?UkExalpHYWNaUjAreFJnaXR0ZUpLM000MlVjNVcveEY1M1ExdFBXRXBHQVY5?=
 =?utf-8?B?Q05wMWZmZGNCaXV0VlovQVIrWFdwM0Z1VlZIdG5Fc0h3VTFKa2FXREY4ek13?=
 =?utf-8?B?VEpaQTRvb3lLSUlKdWhQYzE2MFFFQVdVMUYvY1RFdlFwVW9rWTZDWWNOTS8v?=
 =?utf-8?B?V092dFYxd3RZeHVKT05ZY2puRTJycWJvVk45Z2J6SjRQR0ZsckYvd2ZwL2Zx?=
 =?utf-8?B?TVRySUh1eFQwNGlPSFJCdlhVMUZyUzRsUDVrQS9KYitrNjBzQ2FUMzAzRzZk?=
 =?utf-8?B?MGlSSWZHTlVTZGk2VTRDckFIU2ZOK2lYKy83ckVDcHBvU2hBYnQzUTI4T1Bs?=
 =?utf-8?B?aUloZmhJSUFpY3d4RTdJNmN2UDJrb3d4cjVDdDFWQTY4MUhqUSt5cGpZWUNj?=
 =?utf-8?B?cG9MQzk0VVF2TmxJNXhVQ0x6U1dHcHRtKzViL3pIZk44RG5vMzFkVk1xMmY3?=
 =?utf-8?B?bkIxMkM3VEJMb2M0MFJrSUhFd1JxM1VaL29wRjhRelBkMEFxVmRZTWROV3hT?=
 =?utf-8?B?ZmtxU2pwM2ZaWHBYeWdNdXNqeHZpRGZzM3AyV09ldHJ2cTRidDB4eml5eG1j?=
 =?utf-8?B?Ukd5eTcybUx6NHQ2QUxUdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <E3985653E616B44781E933B07CF65C39@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 640899f4-d5b5-41a4-c6f5-08d97809f514
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 05:30:40.9771
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 5jqA4yRO0srcIBkwMm7pa2N8riZfkV/PjjGcw67yDXVSJcl3TIftzjDi5ji5K2jPb4xgsfQ9Zf+zqICsFfJA6qsX6jPEeIYdltSF/PXERKX7YIHvrEFvB5Beo8az0VBq
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7347
X-Proofpoint-GUID: 9PLTroHSpHmZmUcbpcFekuHXRX__9MSY
X-Proofpoint-ORIG-GUID: 9PLTroHSpHmZmUcbpcFekuHXRX__9MSY
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_10,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015
 phishscore=0 adultscore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0
 priorityscore=1501 impostorscore=0 mlxlogscore=999 bulkscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109150034

SGksIFJhaHVsIQ0KDQpPbiAxNC4wOS4yMSAxNzoyNCwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28g
d3JvdGU6DQo+DQo+IH0NCj4+PiDCoCArc3RhdGljIGludCBwY2lfZWNhbV9yZWdpc3Rlcl9tbWlv
X2hhbmRsZXIoc3RydWN0IGRvbWFpbiAqZCwNCj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+Pj4gK8KgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgY29uc3Qgc3RydWN0IG1taW9faGFuZGxlcl9vcHMgKm9wcykNCj4+PiArew0KPj4+
ICvCoMKgwqAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmcgPSBicmlkZ2UtPnN5c2RhdGE7
DQo+Pj4gKw0KPj4+ICvCoMKgwqAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsIG9wcywgY2ZnLT5w
aHlzX2FkZHIsIGNmZy0+c2l6ZSwgTlVMTCk7DQo+Pj4gK8KgwqDCoCByZXR1cm4gMDsNCj4+PiAr
fQ0KPj4gR2l2ZW4gdGhhdCBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgaXMgZ2VuZXJpYyAoaXQg
aXMgbm90IHNwZWNpZmljIHRvDQo+PiBvbmUgYnJpZGdlKSwgSSB3b25kZXIgaWYgd2UgZXZlbiBu
ZWVkIHRoZSAucmVnaXN0ZXJfbW1pb19oYW5kbGVyDQo+PiBjYWxsYmFjayBoZXJlLg0KPj4NCj4+
IEluIGZhY3QsIHBjaV9ob3N0X2JyaWRnZS0+c3lzZGF0YSBkb2Vzbid0IGV2ZW4gbmVlZCB0byBi
ZSBhIHZvaWQqLCBpdA0KPj4gY291bGQgYmUgYSBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cqLCBy
aWdodD8NCj4NCj4gUmFodWwsIHRoaXMgYWN0dWFsbHkgbWF5IGNoYW5nZSB5b3VyIHNlcmllcy4N
Cj4NCj4gRG8geW91IHRoaW5rIHdlIGNhbiBkbyB0aGF0Pw0KPg0KVGhpcyBpcyB0aGUgb25seSBj
aGFuZ2UgcmVxdWVzdGVkIHRoYXQgbGVmdCB1bmFuc3dlcmVkIGJ5IG5vdy4NCg0KV2lsbCBpdCBi
ZSBwb3NzaWJsZSB0aGF0IHlvdSBjaGFuZ2UgdGhlIEFQSSBhY2NvcmRpbmdseSwgc28gSSBjYW4N
Cg0KaW1wbGVtZW50IGFzIFN0ZWZhbm8gc3VnZ2VzdHM/DQoNClRoYW5rcywNCg0KT2xla3NhbmRy
DQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 05:36:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 05:36:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187224.336014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQNaQ-0001hs-Gk; Wed, 15 Sep 2021 05:35:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187224.336014; Wed, 15 Sep 2021 05:35:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQNaQ-0001hl-DZ; Wed, 15 Sep 2021 05:35:58 +0000
Received: by outflank-mailman (input) for mailman id 187224;
 Wed, 15 Sep 2021 05:35:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tFB7=OF=epam.com=prvs=9892afeb01=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQNaP-0001hf-8B
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 05:35:57 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cca51e80-15e6-11ec-b4d9-12813bfff9fa;
 Wed, 15 Sep 2021 05:35:56 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18F5D6DE025402; 
 Wed, 15 Sep 2021 05:35:53 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b39ff06v0-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 15 Sep 2021 05:35:53 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3472.eurprd03.prod.outlook.com (2603:10a6:803:25::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.26; Wed, 15 Sep
 2021 05:35:48 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 05:35:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cca51e80-15e6-11ec-b4d9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gO+mc9O8Kx7jYbjiJiD3w87d5prvrEeTv6KFMVHRWKIKYQFMlN+Yv7TiLZbDUEvg3CUryfTGktzQVonEshCyeEQIL0MwDFguuaOclHyzTkfshgeGcitKQGAChR5J6ebodxbrAsL9aej72+MSMgppPr1B5V1WniyMtQEp7lEvbEuZcdUxFiDVbY8MdRmqjKzfxbqPjrK7xG9bgj7BRK4XOuwqaJlA2M7qF4jcMUWsAhIIJ9PJyK5c+/yPbvLnl/i5WBJbsqoIsewwyq8hVSSDwhTMOozNWs8hHoCA4FlKUIG6oe4Nye6vQVdP2+Ix58C9hSTqFkaTP0LsT3VEFQBmOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vPpE+tz+LC0WamEI+NnBHQ3+T2W7EesEGE/fp236j10=;
 b=FIZiGAvSfY7bK2d6HDLeRtaP57MmkxuMvzDbT2DSiBZNDAITpVio22zwD2gJ6TI9u0tuwrXK8pJB0fCW1DEqa1fqrdxUHVqbo3fA2doGkqHDqFq8We6jsJO635Iu9jJhZ9brmF12E3fVqJ7/6PEEdImPac3br/+8mpX10ypmYrmfbsCHUtohodvdU3QRwN4mR8jKjM54k1JuNYb/5xvq/K2MtUPiqjmgnH6OUGb/O1rrIpG9JCigDtmUHuL1++LM1s1ww0LmIP8pt2zK328v7GCwKBkxYkRu6dNVQ7vjY3PHCG2lOTRM9IqAC9KyDS7C1zbMucFmX+nIm/HJF0ZoBg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vPpE+tz+LC0WamEI+NnBHQ3+T2W7EesEGE/fp236j10=;
 b=FwBtxYOZHKdn9MbSVzn317TVlxr77Lz4BpyLwOZmYjzqotrLEgzz9djADhNOrWI1/6jNzZ4kex8rOxMJQ0rl5VBIKR3QmGJGffueAzRHcq3uZsB6fHuBqPLSOyMv9COx+SEDEaS9w0iZ5qa8P+UDt7WCfJ56sxDqXt5d3Cw8MCB8qGiwG+4IfwP92AmMjfkpuFwYrYNh7al5es/u2GL5sQo3xC6cPmPpxAqdTDZOyNzz99OiWxnVTYfgXHopw72Z3i1NgLNcfRSIAVlo48yWD7z4OEEsvp3f/+pFjS+L1x3aaHhPZP+t4gEvIU7acuYmTEISox9Hdmo9l17Jy2NB3A==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Rahul Singh
	<rahul.singh@arm.com>
CC: Julien Grall <julien.grall.oss@gmail.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        xen-devel <xen-devel@lists.xenproject.org>,
        Oleksandr
 Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAARZwCAAFsJAIAAFA4AgAO3eICAAc7agIAA860AgABTv4A=
Date: Wed, 15 Sep 2021 05:35:48 +0000
Message-ID: <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
 <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
 <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
 <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b5070c9f-9e1b-4bcd-42f2-08d9780aac28
x-ms-traffictypediagnostic: VI1PR0302MB3472:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB34723F08504615DB0F5B46F4E7DB9@VI1PR0302MB3472.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 S2YlYZ62Do98mBAVopKsmT8HUoJm38R34lWNyeYNsNU9tz3SIL9G6jBHqbrj2Uie6EfNZHeJPixvWl3jHpuPuiR86px9iAlheJE2BfT8X70fmKzIzkxrkGxjpqXRinmaU5j0MfDakn1RDnGaWEV/yy39SXBxtsPi9Vmy42ZQrn9uCDhUmur82kmtcy8WBc380fmTblrUzkft89Znv4ByifA17WJkZgxs5bWffVB9s32Y2Mi0mPRAM7f4+b2S+Ez/kyFZdeUmOyLgk9PwAch9twnML5rhXshH1j7ebWCB8+TW2PnHTnF8L/xeVg78+3baNzFYF22fcNioqdo8UqEBGdJe1oNNO8a6UL5T9YHEbun4ASY/+kA+cLVgxRguaeFPovOOkDxXL4cgCwOg0AriQmYUKM7PSpCWK6bJdHGzn4kVf05cpF3/9bZwb4uei0hgvUfZGp0I9C4qYezusWXRDGfA8Kr0JSJk+tTCRhz4xdi1k3/55FgTlx+81dIqlZogFsYmjWrWQ1M+OlB2YalzpK4Tyz3c4YUhXWuUeWS7AoUW56j8GBOBQcpn/2Xf9sBaEzkChil2y5aOEk5mFxHRSGcCKVAY6L8oqVFL/fgoShpYhiih6aXIrxsNOegxeHLkxnplkuaXnBq+K9/HwlhB1M1iEAdj9tBEw1Tj0SjPQrSHQheLqibCXpnfWG1s6zy5CdGFOQEPbr8PQuWr8FAffeTKcv8DxO5ImD4q/6y8frbiDeC4mo+JaHXNGCaIBF8E
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(39860400002)(376002)(366004)(396003)(136003)(8676002)(478600001)(316002)(66946007)(71200400001)(5660300002)(36756003)(26005)(122000001)(31686004)(6512007)(66476007)(83380400001)(64756008)(38100700002)(31696002)(55236004)(91956017)(53546011)(66556008)(6506007)(8936002)(54906003)(66446008)(2616005)(86362001)(76116006)(6486002)(38070700005)(4326008)(2906002)(110136005)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RGQ5L1Rack9mcVA2QmZFUFBsNG1hSk04dnRMbFIzeEpDdkd6VkhpcEIwSTE2?=
 =?utf-8?B?NUJRb2NFbnpwT0YyaXBzM0M3TEdpSTNJaWVCaTY5QXNzVFViUFZSVnY4TXp6?=
 =?utf-8?B?NzRvSzl1VmdNbWpNVkMzczB0WWFtZTBWTnpZSmcvTUJRSGltMk1SVGxVWXRK?=
 =?utf-8?B?VzU4WG1NSnhaMHBGK3JpNzFCSk9mZHJmdS9NV2d1bmx1R2RNM050S2kyU1lx?=
 =?utf-8?B?Ym8rZklKV1Z2OW5lVHNmSCtLeDRES2g0WUUvZklKS0xSWk9pVnViQ002aTZC?=
 =?utf-8?B?d1dkaWV1VEw3VGQ4eXJKT2JSK1JUeWxvRWhYa082bUc1Qm1FblpkdlRLcWpy?=
 =?utf-8?B?TDZrcUVPNkJJeFpJdTZUU1BpMDcveVlHK3BFbFk2QkJDM0RRN29GblRlWks5?=
 =?utf-8?B?bHhYaUNDMjF5NWswUEppZDBHb0lRaWlEL2x4R0kwME44OTdKWW82REcxRTJZ?=
 =?utf-8?B?dUxDb1VJTnZDdXpvVmNycjlVMHZLeEJPV1FSenRHazZ5NmdmNWU4NWFCd2VH?=
 =?utf-8?B?REEzSW1PdmNGZUZvcEFKQktpNWpQUDNjaDZMdElIR0hyNlZqcS9DZEdIY3VR?=
 =?utf-8?B?ekE4RFkxM0dPeU1sbXpGZGF2bWZ3aHlKVVRpMzRxSTAya2VrWWFaakdTSXhD?=
 =?utf-8?B?a0lPenVCdzVmZjdnN1ljVkE2Y2ZzdDBjZUxTT0JTQXZxazU4QzMzTTFMaTV1?=
 =?utf-8?B?RUNCWW9OVUk5RkVPcU5pVDdBUjNRVzhKa2U3bmF0dkZvQkRxNjJ0VjhHVkpT?=
 =?utf-8?B?Sy9DVnFRQ0pSRFVLcEU4dW1KTXlFRHFCY3lyNUJ0ZGFQc0FEa3Y3WVNyR05D?=
 =?utf-8?B?N3pRcXowZjVFSE5UaTZDUmZPOUh1RUl4Q1ViMDVxRGUzaURsak15L05YSzcx?=
 =?utf-8?B?YmdjNy9pcm93a3JHUmxoc0tGU1kvd3RPUEVEd09BODZ3UUxBQ3BwNjNxaEJi?=
 =?utf-8?B?cjZnRGgvWTVTbURHOXNpMGZmV2dZMW1DVW1CN3BNS3ErU0FKWWI5U1NubGIv?=
 =?utf-8?B?SW1EZk1zQWR1ZXRBOUxuQ1V2RVJEVGhOOFZsN3MyK0lnZWZvZk02MkE2RWRK?=
 =?utf-8?B?eUp4V2NqeEFPK25aQmsyelZpUHAxd1dqUG5wQXcrOHdNUnlVcFVXRTU3ZDRY?=
 =?utf-8?B?VE5DbGhqdHE3K0I2VHJaYWVFQ0JiZmJGU21wR251ODJvdUVFOHcxSnB4TlVJ?=
 =?utf-8?B?bDRMTzVqUEF6QUQxcWtvaUNaZUZpWHAxVmM0a3lKcG5sMzQvL0x6SUFWSEtw?=
 =?utf-8?B?YnozenJ4eCtJVDRSdVRTSC9LcVVwWXg2VkkvV1VvaFBoTjFROEZjMlFHRFQ1?=
 =?utf-8?B?Y25vWkxlaTV2ZXdqc01PKzZNU3BzbjcyTHA2aUo5bVRmNldIaG91Z0VXcFlL?=
 =?utf-8?B?Q1d6Zyt4SWpNaFAxeFBLZmRFOEVWTjUwN3VsdVVEOW9CTlhBL00xSzZKN0Ny?=
 =?utf-8?B?QTdMMm5hSENIc1ZlQjdLT0w1Rm9UbE1pOGtBUTEwWUs0L0dHVmw0R0dCeFdH?=
 =?utf-8?B?d2ZrY0s1WThFOWhBUGpPdldoWEVNWWVkcUN5c2hrR0NheW82djVIWndCZ2Nt?=
 =?utf-8?B?Z1RaYkhlOFhPOTkvNTdHNW11YitSZ24rQy9jVjVUVy9xSklzZFZDeEdGV2ZH?=
 =?utf-8?B?dlc3dGRzYi9IZXM5VE90WWlvenhoYmlIVm4zTkl0Skg0cDRCT1RDRFVOa0JG?=
 =?utf-8?B?UWFRbWFHTlRjQ01EYlV5NDBSZytlVk4wL0t6LzBBL0pSakY0ajREM1o3UG04?=
 =?utf-8?B?V2cyRjF6UEtJOXZYVk1aZXRFck0vRitJVFNuazduVmlGMjFtY0FPTjhwTmxi?=
 =?utf-8?B?bE8rZS9QQjVycFAyMnJuUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <902E2E3EF1FCC44AA10C63136363F766@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5070c9f-9e1b-4bcd-42f2-08d9780aac28
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 05:35:48.1806
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ULPz0j6W+S1eBSYzxzNyAS3GQB/Ygpj4YInngC7fjTP/sa475nUYzqb72W+Lhe69lSnz3jXHF+aDEuGeN1OmtUP88AM6k5VgJdFFIPWDV2jJQGVoWQIrMIARgnfP4iPP
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3472
X-Proofpoint-GUID: IgkAErY10S5bHY-ghGWHjmbI1RDyXjMT
X-Proofpoint-ORIG-GUID: IgkAErY10S5bHY-ghGWHjmbI1RDyXjMT
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-14_10,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0
 clxscore=1015 bulkscore=0 impostorscore=0 spamscore=0 phishscore=0
 mlxlogscore=999 priorityscore=1501 mlxscore=0 suspectscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109150035

SGksIFN0ZWZhbm8sIFJhaHVsIQ0KDQpPbiAxNS4wOS4yMSAwMzozNiwgU3RlZmFubyBTdGFiZWxs
aW5pIHdyb3RlOg0KPiBPbiBUdWUsIDE0IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byB3cm90ZToNCj4+IFdpdGggdGhlIHBhdGNoIGFib3ZlIEkgaGF2ZSB0aGUgZm9sbG93aW5nIGxv
ZyBpbiBEb21haW4tMDoNCj4+DQo+PiBnZW5lcmljLWFybXY4LXh0LWRvbTAgbG9naW46IHJvb3QN
Cj4+IHJvb3RAZ2VuZXJpYy1hcm12OC14dC1kb20wOn4jIChYRU4pICoqKiBTZXJpYWwgaW5wdXQg
dG8gWGVuICh0eXBlICdDVFJMLWEnIHRocmVlIHRpbWVzIHRvIHN3aXRjaCBpbnB1dCkNCj4+IChY
RU4pID09PT0gUENJIGRldmljZXMgPT09PQ0KPj4gKFhFTikgPT09PSBzZWdtZW50IDAwMDAgPT09
PQ0KPj4gKFhFTikgMDAwMDowMzowMC4wIC0gZDAgLSBub2RlIC0xDQo+PiAoWEVOKSAwMDAwOjAy
OjAyLjAgLSBkMCAtIG5vZGUgLTENCj4+IChYRU4pIDAwMDA6MDI6MDEuMCAtIGQwIC0gbm9kZSAt
MQ0KPj4gKFhFTikgMDAwMDowMjowMC4wIC0gZDAgLSBub2RlIC0xDQo+PiAoWEVOKSAwMDAwOjAx
OjAwLjAgLSBkMCAtIG5vZGUgLTENCj4+IChYRU4pIDAwMDA6MDA6MDAuMCAtIGQwIC0gbm9kZSAt
MQ0KPj4gKFhFTikgKioqIFNlcmlhbCBpbnB1dCB0byBET00wICh0eXBlICdDVFJMLWEnIHRocmVl
IHRpbWVzIHRvIHN3aXRjaCBpbnB1dCkNCj4+DQo+PiByb290QGdlbmVyaWMtYXJtdjgteHQtZG9t
MDp+IyBtb2Rwcm9iZSBlMTAwMGUNCj4+IFvCoMKgIDQ2LjEwNDcyOV0gZTEwMDBlOiBJbnRlbChS
KSBQUk8vMTAwMCBOZXR3b3JrIERyaXZlcg0KPj4gW8KgwqAgNDYuMTA1NDc5XSBlMTAwMGU6IENv
cHlyaWdodChjKSAxOTk5IC0gMjAxNSBJbnRlbCBDb3Jwb3JhdGlvbi4NCj4+IFvCoMKgIDQ2LjEw
NzI5N10gZTEwMDBlIDAwMDA6MDM6MDAuMDogZW5hYmxpbmcgZGV2aWNlICgwMDAwIC0+IDAwMDIp
DQo+PiAoWEVOKSBtYXAgW2UwMDAwLCBlMDAxZl0gLT4gMHhlMDAwMCBmb3IgZDANCj4+IChYRU4p
IG1hcCBbZTAwMjAsIGUwMDNmXSAtPiAweGUwMDIwIGZvciBkMA0KPj4gW8KgwqAgNDYuMTc4NTEz
XSBlMTAwMGUgMDAwMDowMzowMC4wOiBJbnRlcnJ1cHQgVGhyb3R0bGluZyBSYXRlIChpbnRzL3Nl
Yykgc2V0IHRvIGR5bmFtaWMgY29uc2VydmF0aXZlIG1vZGUNCj4+IFvCoMKgIDQ2LjE4OTY2OF0g
cGNpX21zaV9zZXR1cF9tc2lfaXJxcw0KPj4gW8KgwqAgNDYuMTkxMDE2XSBud2xfY29tcG9zZV9t
c2lfbXNnIG1zZyBhdCBmZTQ0MDAwMA0KPj4gKFhFTikgdHJhcHMuYzoyMDE0OmQwdjAgSFNSPTB4
MDAwMDAwOTM4MTAwNDcgcGM9MHhmZmZmODAwMDEwNGI0YjAwIGd2YT0weGZmZmY4MDAwMTBmYTUw
MDAgZ3BhPTB4MDAwMDAwZTAwNDAwMDANCj4+IFvCoMKgIDQ2LjIwMDQ1NV0gVW5oYW5kbGVkIGZh
dWx0IGF0IDB4ZmZmZjgwMDAxMGZhNTAwMA0KPj4NCj4+IFtzbmlwXQ0KPj4NCj4+IFvCoMKgIDQ2
LjIzMzA3OV0gQ2FsbCB0cmFjZToNCj4+IFvCoMKgIDQ2LjIzMzU1OV3CoCBfX3BjaV93cml0ZV9t
c2lfbXNnKzB4NzAvMHgxODANCj4+IFvCoMKgIDQ2LjIzNDE0OV3CoCBwY2lfbXNpX2RvbWFpbl93
cml0ZV9tc2crMHgyOC8weDMwDQo+PiBbwqDCoCA0Ni4yMzQ4NjldwqAgbXNpX2RvbWFpbl9hY3Rp
dmF0ZSsweDVjLzB4ODgNCj4+DQo+PiAgIEZyb20gdGhlIGFib3ZlIHlvdSBjYW4gc2VlIHRoYXQg
QkFScyBhcmUgbWFwcGVkIGZvciBEb21haW4tMCBub3cNCj4+DQo+PiBvbmx5IHdoZW4gYW4gYXNz
aWduZWQgUENJIGRldmljZSBnZXRzIGVuYWJsZWQgaW4gRG9tYWluLTAuDQo+Pg0KPj4gQW5vdGhl
ciB0aGluZyB0byBub3RlIGlzIHRoYXQgd2UgY3Jhc2ggb24gTVNJLVggYWNjZXNzIGFzIEJBUnMg
YXJlIG1hcHBlZA0KPj4NCj4+IHRvIHRoZSBkb21haW4gd2hpbGUgZW5hYmxpbmcgbWVtb3J5IGRl
Y29kaW5nIGluIHRoZSBDT01NQU5EIHJlZ2lzdGVyLA0KPj4NCj4+IGJ1dCBNU0ktWCBhcmUgaGFu
ZGxlZCBkaWZmZXJlbnRseSwgZS5nLiB3ZSBoYXZlIE1TSS1YIGhvbGVzIGluIHRoZSBtYXBwaW5n
cy4NCj4+DQo+PiBUaGlzIGlzIGJlY2F1c2UgYmVmb3JlIHRoaXMgY2hhbmdlIHRoZSB3aG9sZSBQ
Q0kgYXBlcnR1cmUgd2FzIG1hcHBlZCBpbnRvDQo+Pg0KPj4gRG9tYWluLTAgYW5kIGl0IGlzIG5v
dC4gVGh1cywgTVNJLVggaG9sZXMgYXJlIGxlZnQgdW5tYXBwZWQgbm93IGFuZCB0aGVyZQ0KPj4N
Cj4+IHdhcyBubyBuZWVkIHRvIGRvIHNvLCBlLmcuIHRoZXkgd2VyZSBhbHdheXMgbWFwcGVkIGlu
dG8gRG9tYWluLTAgYW5kDQo+Pg0KPj4gZW11bGF0ZWQgZm9yIGd1ZXN0cy4NCj4+DQo+PiBQbGVh
c2Ugbm90ZSB0aGF0IG9uZSBjYW5ub3QgdXNlIHhsIHBjaS1hdHRhY2ggaW4gdGhpcyBjYXNlIHRv
IGF0dGFjaCB0aGUgUENJIGRldmljZQ0KPj4NCj4+IGluIHF1ZXN0aW9uIHRvIERvbWFpbi0wIGFz
IChwbGVhc2Ugc2VlIHRoZSBsb2cpIHRoYXQgZGV2aWNlIGlzIGFscmVhZHkgYXR0YWNoZWQuDQo+
Pg0KPj4gTmVpdGhlciBpdCBjYW4gYmUgZGV0YWNoZWQgYW5kIHJlLWF0dGFjaGVkLiBTbywgd2l0
aG91dCBtYXBwaW5nIE1TSS1YIGhvbGVzIGZvcg0KPj4NCj4+IERvbWFpbi0wIHRoZSBkZXZpY2Ug
YmVjb21lcyB1bnVzYWJsZSBpbiBEb21haW4tMC4gQXQgdGhlIHNhbWUgdGltZSB0aGUgZGV2aWNl
DQo+Pg0KPj4gY2FuIGJlIHN1Y2Nlc3NmdWxseSBwYXNzZWQgdG8gRG9tVS4NCj4+DQo+Pg0KPj4g
SnVsaWVuLCBTdGVmYW5vISBQbGVhc2UgbGV0IG1lIGtub3cgaG93IGNhbiB3ZSBwcm9jZWVkIHdp
dGggdGhpcy4NCj4gV2hhdCB3YXMgdGhlIHBsYW4gZm9yIE1TSS1YIGluIERvbTA/DQpJdCBqdXN0
IHdvcmtlZCBiZWNhdXNlIHdlIG1hcHBlZCBldmVyeXRoaW5nDQo+DQo+IEdpdmVuIHRoYXQgRG9t
MCBpbnRlcmFjdHMgd2l0aCBhIHZpcnR1YWwtSVRTIGFuZCB2aXJ0dWFsLUdJQyByYXRoZXIgdGhh
bg0KPiBhIHBoeXNpY2FsLUlUUyBhbmQgcGh5c2ljYWwtR0lDLCBJIGltYWdpbmUgdGhhdCBpdCB3
YXNuJ3QgY29ycmVjdCBmb3INCj4gRG9tMCB0byB3cml0ZSB0byB0aGUgcmVhbCBNU0ktWCB0YWJs
ZSBkaXJlY3RseT8NCj4NCj4gU2hvdWxkbid0IERvbTAgZ2V0IGVtdWxhdGVkIE1TSS1YIHRhYmxl
cyBsaWtlIGFueSBEb21VPw0KPg0KPiBPdGhlcndpc2UsIGlmIERvbTAgaXMgZXhwZWN0ZWQgdG8g
aGF2ZSB0aGUgcmVhbCBNU0ktWCB0YWJsZXMgbWFwcGVkLCB0aGVuDQo+IEkgd291bGQgc3VnZ2Vz
dCB0byBtYXAgdGhlbSBhdCB0aGUgc2FtZSB0aW1lIGFzIHRoZSBCQVJzLiBCdXQgSSBhbQ0KPiB0
aGlua2luZyB0aGF0IERvbTAgc2hvdWxkIGdldCBlbXVsYXRlZCBNU0ktWCB0YWJsZXMsIG5vdCBw
aHlzaWNhbCBNU0ktWA0KPiB0YWJsZXMuDQoNClllcywgaXQgc2VlbXMgbW9yZSB0aGFuIHJlYXNv
bmFibGUgdG8gZW5hYmxlIGVtdWxhdGlvbiBmb3IgRG9tYWluLTANCg0KYXMgd2VsbC4gT3RoZXIg
dGhhbiB0aGF0LCBTdGVmYW5vLCBkbyB5b3UgdGhpbmsgd2UgYXJlIGdvb2QgdG8gZ28gd2l0aA0K
DQp0aGUgY2hhbmdlcyBJIGRpZCBpbiBvcmRlciB0byB1bm1hcCBldmVyeXRoaW5nIGZvciBEb21h
aW4tMD8NCg0KUmFodWwsIGl0IHNlZW1zIHdlIHdpbGwgbmVlZCBhIGNoYW5nZSB0byB2UENJL01T
SS1YIHNvIHdlIGNhbiBhbHNvDQoNCnRyYXAgRG9tYWluLTAgZm9yIE1TSS1YIHRhYmxlcy4NCg0K
VGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 06:05:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 06:05:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187233.336025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQO2j-0005Gf-Uq; Wed, 15 Sep 2021 06:05:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187233.336025; Wed, 15 Sep 2021 06:05:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQO2j-0005GY-Rk; Wed, 15 Sep 2021 06:05:13 +0000
Received: by outflank-mailman (input) for mailman id 187233;
 Wed, 15 Sep 2021 06:05:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQO2i-0005GS-6F
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 06:05:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e223580e-15ea-11ec-b4db-12813bfff9fa;
 Wed, 15 Sep 2021 06:05:11 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-55XKYy4pMzClJ4mJWTVE9w-1; Wed, 15 Sep 2021 08:05:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4445.eurprd04.prod.outlook.com (2603:10a6:803:6e::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 06:05:07 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 06:05:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Wed, 15 Sep 2021 06:05:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e223580e-15ea-11ec-b4db-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631685910;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2E+WEwgOtIX1OCyW2woDo9w/2jvsNO81sEG/PAJhb0s=;
	b=NP1AksTXgCODrw0yfrWK+V51mKetBQG9LJcD5auj3v051Gyw+lBzJqb0qvn1ukWXaSqAgw
	hXRk1YxjL9BlvWDkDWiSo/LwJo43swdSb4iBBF8slhu1tRm+OoLhK6DrqhesR3Jw8R1GNu
	Tcx1r4FFbziWqeHU79hlyEBk+jkGqK4=
X-MC-Unique: 55XKYy4pMzClJ4mJWTVE9w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ndnpmZQDOzwdHTWxxKt9tGLto0MuJ/MR4F0oWyCX16cTWs5aeYVTMVmzZGm1AGq7T0EZzikyUptFmzlUupDOXvnAZESjNsBHmgPZJ+7SHmy6KnfysxvkE6gJA/Y3vc9IXMb1EsjmByckZMc8w74+vR90btEokk6ljKLbu8XWcnRUQzBhlW6FCtRUTvuGIeMU/ndhSIyXXALchGRvzZpuumlfrpfa5kD2M0vp8kJEvVaQ0Jf6oMeQP2R1uO0g9r2koTRrC5oM1l826gTmY2CbNpq/Kj9nyigRnKRWL1RR2gwBfDoBZ/lff4+YSw0PUd0tzgnMFpPPbhy/cy30Vuqh2Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2E+WEwgOtIX1OCyW2woDo9w/2jvsNO81sEG/PAJhb0s=;
 b=cG3kG6kgW3leC7tXviqr4Ni3hWqF4v1MopQIMJG6KVTLASUAjEchWMPyujj0zX2gy452s7uYAVI2mnx18mMPGEAsGU8e7dCwORRBCTsgQm/SlmY1NyNasFvh61xeoYiBUxWD8KqWuyxO2oyT1IlzJu44HhlvP73YeIhXBqqzS4G9qEzUpHtl2NEWKms2+ndwH04vTsOaqIzzuQGfqGVNPjUmou017f7acejqwgInM26QfGkNkjZQFIjzCZKocuw/I2wFf5c5jDHTHKFL8VYjP6ykXmBqTppIZjCwBYGSfdAuKaBQ9MUrtaNzDtnSpu6RrFnRHTg7Iql+CLq3JeE73w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
Date: Wed, 15 Sep 2021 08:05:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e5dd8198-443a-4a11-cc3b-08d9780ec4a0
X-MS-TrafficTypeDiagnostic: VI1PR04MB4445:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4445C63ED0FA59CFD2F82EF0B3DB9@VI1PR04MB4445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ai/YzDbBWPD3fZMnseFVN+xdIh6DxAmYTUfZ+agkEheEkHBDNpd5BjDK/SJr/16pEnLMDh1XyYLylhT92gsuu6bFxJ53/Do/WaaZ1c0zYBWT2hXmXw9VQIksWu/L8W/KOO9/RgaDVozcRPfXmYgY0EbaeimPhYO0wRHBUUoUIZpbE34KZmUuxhWRWFNZbxmINHRiK3m0IOaNt7asZsq+Sd0ByjEBSXigcLSGgPCfVMPwvbgqkuPW9iciPvN006jzrAZ4ydVU7gp355zciyyqJK6X+ddfL+TPZT1bp/eQPecAu5wlzLfnDZ+xO60ZrjmNeDELVhOQrw8sNRLj3DNI1tKf8rspCJtOrSmtJyzqETBaBGJXg/tVXqo6UQuVzEctKW+OnP05aXzmoMErw2/Zzpb3nHV/IVCGojtaea0OmX8eBadLBV6eVNBvLuoeAKOkSiUrKlkGOEkh4zBQXxaHx0Bx8BqmVIXAxbZO3aqp03fRNKkxPWOogRpEaKJj/cSlKD7xGtndlQvyaG68DOZupbLp+1M6Bd/Dx/xY/8skT0bUk3ttRykxjVOS3di2nSbHacr0QfDc3IYSTQMyKvG0aZKknOX4FFkE41WXnYAOVT87atD0REJx33k7JXONRchutFtZGb5pvqILDI4hhl05F+NGonrdLCaDgVobWJRaiI/LyNaY06MOd0D6oynJ1r1ua016AuMHzv/7T23dR6UGXNI+JD1NpB/v+k0RiTyI6fk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(4326008)(66946007)(31696002)(8676002)(31686004)(6486002)(508600001)(38100700002)(53546011)(316002)(66556008)(66476007)(8936002)(3480700007)(2616005)(5660300002)(86362001)(16576012)(186003)(36756003)(26005)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cmNjTmkwa1ZidTJZK3l3MUtoT0xHMlFhWitRRCs0c1F1eFJnR0h4WUxVNWR2?=
 =?utf-8?B?eXNYL3lCUE9uSklUdFJXV1RrZ09lQzRjRFRGd0VNK3UwNlBtNklZTlVoazRk?=
 =?utf-8?B?QkFrb1NtUVZ1VEVESlV1RnVjbTV2V202Z29KcXRTbTZNV0Y4bGxBT1pkQURz?=
 =?utf-8?B?ZW9wc2RNLy95aEN5VVZ3czRsQ0ZSYkFoOGZtZzZOSi8zOHdVWnZBRHV1aG03?=
 =?utf-8?B?RDZQZmhubmZXZ3owMmJmQUdlcXhFZDMyY3lRM2NtM2NmaXBkSDdPcmVmZHlD?=
 =?utf-8?B?djJDZjBHQ1JxQVZEV202Y1hqUnFtVzE4VVczc2ZZeGF6MDEwZXluUTBiMHVw?=
 =?utf-8?B?M3hTY05tSWxhVmYrY0ZQUUxQOVRmSVVIVmEvRDdsT2wrMldGcVhJeWQ1eXRu?=
 =?utf-8?B?SEpBN2Zzb3BWQnRYRnYwSVBmdUxIWjMvcVM3WXo0elUzRG51d0kyekxzZElX?=
 =?utf-8?B?OWF2R0F6c29NOUNwbnZRUTdmYm45cjUxWERjY0NiV3cxRlFxU1Z2K3hEUVM0?=
 =?utf-8?B?SHYxOWpLQmFTeW9JSEhtUE9OblNnV1lmTHRhVm9QSHFlK0h4OGpkblh6aUVw?=
 =?utf-8?B?ZFF1NnU0cXFydmxyWWV5b3BjQVhjaUtKNnNkMjlkbklSeEg3cXJOZU1wVmwr?=
 =?utf-8?B?WnFCcGlFVlJLdGV0UkpZSlhjWGI0emRyUU13cm1EVHVuTjRZVXk4MEg3M2Y4?=
 =?utf-8?B?QVJlUm42QmlSMVg1L0RZK1VYNG5ya3dRN1VDSm9QQ0ZsY2tnZlFOK21CR2ts?=
 =?utf-8?B?dUpNUnV6WnBTYldEQjl6cHFHZUhhSDNsVzJhT3NkTFl6RDRaZnBGZE9MUk1O?=
 =?utf-8?B?ekNtZlp3WnRTaTB4QnpFOTFjMlJncG1WUE41VURDZHpGOWJPMnAzcXVXcWVF?=
 =?utf-8?B?QURxR1l4NEk5aVMwK2YvT0FaVkJ0amxFbFM3OVFFUHQxM3BwVFZSM0hXaS9K?=
 =?utf-8?B?M3lKTGt0S1NFdFU0R002aDhTRGtYWmJTSW8ybWx2S3lSNmpCaUdDVlArWTNx?=
 =?utf-8?B?MGlYYWJ3bXBMcHZVZFYrVlg5dVdTQUJzRndkOVVEbkNiSXpkV1ByN3kwcXZl?=
 =?utf-8?B?NVBDQU1EZ0pxblFMK0o3UHVwUUlNYjdpeG1VTW5NZFJGZVE2aVcyMWJFMXBo?=
 =?utf-8?B?L1dmSUpuZXFiaExzdTNNRE52TXhhLzJ4WDhTeENyVTY3Vys5U3JsNE05ai9a?=
 =?utf-8?B?L3M0NTZoWlNJQVlETmtwWll2SGtPeVhuYTlIZ2lhWXJ1KzYwb0U3OGVEQXpS?=
 =?utf-8?B?clQyZ2NSOE4valh1RnpFRjEvQldjenF4bEZQTk5TL003SHBtRGFXVFp2SmlE?=
 =?utf-8?B?enBOTnlPRUxlNGMrMTJuUStVcG12Z3k1VzFGQkE5VlJVK2UxRFBHK05BNFZY?=
 =?utf-8?B?NndUcWZqa0YxTmUycnp4VzdLS3RNUVk3M2d2VUtPVjRwaWpxMFI4UENYTzN5?=
 =?utf-8?B?R0NlS2tSOXlFcmxxRkszMTVUcUsrOVZwUXVFOHhWWFhQaHliU25CRTEwLy90?=
 =?utf-8?B?ZU5Cbm1sZURtRGlzVitnczZkekN2M3JjNHVXOUoxVVRjaWN2M1BsQnc0R2lz?=
 =?utf-8?B?WWZOdzJiL2UySDY1UEUycWdNZmkzaHVueHhRYktFOWJwd0JjUzQvRFljWHZ1?=
 =?utf-8?B?ZThvalkxajVVcVVNaTlnYTBwTll0UmxvQXZtVFNPWVUvV1RSTEs1aXA0V2V2?=
 =?utf-8?B?elhhZ2RmRUNJeEs2anFVNVV0NitBazA3WE8xTFJJdnB2aDBLRHpYazVMMHBX?=
 =?utf-8?Q?wkI5T/e4cgVdQ+deb2d9t+lxO9wWgQhM4GvJsC/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e5dd8198-443a-4a11-cc3b-08d9780ec4a0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 06:05:07.5142
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /XmOHC6lN2IUqjf47JGNLw8p8/lTwSbsG5MwpjVXq6AP5RbHJxGNyHUL2/kB5KXtEqcaDiCrdMyus/cBtfFUWA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4445

On 15.09.2021 04:40, Elliott Mitchell wrote:
> On Tue, Sep 07, 2021 at 05:57:10PM +0200, Jan Beulich wrote:
>> On 07.09.2021 17:03, Elliott Mitchell wrote:
>>>  Could be this system is in an
>>> intergenerational hole, and some spot in the PVH/HVM code makes an
>>> assumption of the presence of NPT guarantees presence of an operational
>>> IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
>>> code, some portion of the IOMMU code might be checking for presence of
>>> NPT instead of presence of IOMMU.
>>
>> This is all very speculative; I consider what you suspect not very likely,
>> but also not entirely impossible. This is not the least because for a
>> long time we've been running without shared page tables on AMD.
>>
>> I'm afraid without technical data and without knowing how to repro, I
>> don't see a way forward here.
> 
> Downtimes are very expensive even for lower-end servers.  Plus there is
> the issue the system wasn't meant for development and thus never had
> appropriate setup done.
> 
> Experimentation with a system of similar age suggested another candidate.
> System has a conventional BIOS.  Might some dependancies on the presence
> of UEFI snuck into the NPT code?

I can't think of any such, but as all of this is very nebulous I can't
really rule out anything.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 07:51:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 07:51:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187242.336036 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPgx-0007n1-Cw; Wed, 15 Sep 2021 07:50:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187242.336036; Wed, 15 Sep 2021 07:50:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPgx-0007mu-9p; Wed, 15 Sep 2021 07:50:51 +0000
Received: by outflank-mailman (input) for mailman id 187242;
 Wed, 15 Sep 2021 07:50:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wWxD=OF=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQPgv-0007mn-O4
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 07:50:49 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a3713586-15f9-11ec-b4e7-12813bfff9fa;
 Wed, 15 Sep 2021 07:50:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3713586-15f9-11ec-b4e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631692248;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=MKup7DxC8nYEi3wI/1H/6Z7GE/ElfkgyRNFUzYgNg2c=;
  b=gEqqe+ylUzxh7HOqcxkomTR9f6OUw3F+rWF+FVifnFI2nFnNsEK0fLh9
   k66I1mCaqixr6PLgV/L+ejYLbkDxlbSHg7RDZE1YUY3vWKzRSXJFkUtPw
   /T10APdsZM2dOLss5OoPzWKoMZnKvWoWWpSIklEU/l/+pY4/oaSTRt7PD
   8=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: RMhcBG6rE2+MJHp3oUswB2K9suNxkGEWPj8FNTZLEDEx+XLAQssuHTUSxJXjRakQQNNlIJW6SD
 JlPLWpWMVo+IxpY9fOxkzfJ7n/bW5CS0tC2uTe1vW6SAb4WEm3/U+z8ROd9/Lift3XtQ/iiS69
 iNExidkDaO9cjYPoG8yX8O31dXOxkBRbMDWHbfGwaTgwDxIXP7cR7gGK9HUfdJ8ClpMv8ABrSt
 zl2ipgFiUggWoz89Na+lbEUL5JJM2GVqpnzk6fS1TBDwomgOoa7xR8C2tbci7gSO8EeFH9J9US
 ZCIVqpKchgkUQEyXoLQT3Lsw
X-SBRS: 5.1
X-MesageID: 53164880
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:5/0Mva6zZ62rqkrl3QpuUQxRtAbAchMFZxGqfqrLsTDasY5as4F+v
 jAaDTuOaPuOZmChLdwgO9/i9U8B7JWAx4VqHVRkr3gwHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo29Mw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Zl
 /BJp5erZ1kTAajcqsgMAl4fVDt7BPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWxp2JweTa62i
 8wxNSExZUr4bD90KloLCLMBoPuUoVu4SmgNwL6SjfVuuDWCpOBr65D9PdyQdtGUSMF9mkeDu
 nmA72n/GgsdNtGU1XyC6H3ErvDLtTP2XsQVDrLQ3vxgjUCXx2cTIAYLTlb9qv684nNSQPoGd
 RZSoHB36/Fvqgr7FbERQiFUvlbbmxoOUMF+TdcF7RG/5ruFpEHeHG09G2sphMMdiCMmedA7/
 gbXxIq0VGMw7uT9pWG1rejP/GjrUcQBBSpbP3ZVE1FdizX2iNxr1nryosBf/LlZZzEfMQr5x
 SyD5AM6jq8a5SLg//TmpQ2b695AS56gc+LU2uk1djn+hu+aTNT8D2BN1bQ9xaweRGp+ZgPd1
 EXoY+DEsIgz4WilzURhutnh+Y1FAd7fYVUwZnY0RPEcG8mFoSb/Lei8HhknfBoB3jk4lc/BP
 xaI5FI5CG57F3q2d65nC79d+OxzlvOIKDgRbdiNNoAmSsEoLGevpXgyDWbNjzGFuBV9yskXZ
 MbEGftA+F5HUMyLOhLtHLxDuVLqrwhjrV7uqWfTlEj+iuvCOyfOEN/o8jKmN4gE0U9Nmy2Mm
 /53PMqW0RRPFur4Zyjc64kIKl4Wa3M8APjLRwZ/LIZv+yJqRzMsDeH/27Qkd9A3lqhZjL6Qr
 Hq8RlVZ2Bz0gniecVeGbXVqabXOW5djrC1kYXxwbAjwg3VzM5yy6Ko/docseeV1/uJU0vMpH
 eIOfN+NA6oTR22fqSgdd5T0sKdraA+v2VCVJyOgbTVmJ8xgSgXF98XKZAzq8CVSXCO7udFn+
 ++r1x/BQIpFTANnVZ6EZPWqxlK3nH4chOMtABeYfogNIB3hqdE4JTbwg/k7J9A3BS/CnjbKh
 RyLBRo4pPXWp9Nn+tf+mq3Z/ZyiFPFzHxQGEjCDv6q2LyTT4kGq3ZREDLSTZTnYWW75pPeia
 OFSw62uOfELhg8X4Y91ErItxqMi/dr/4bRdy108TnnMal2qDJJmI2WHgpYT5vEcmOcBtFvkQ
 F+L9/lbJa6NaZHsH1MmLQY4aviOiKMPkT7I4PVpeEj36UebJlZcvZm+6/VUtBFgEQ==
IronPort-HdrOrdr: A9a23:+qf486oUIdLl+Ygn9zyLawwaV5uwL9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ
 0QDpSWa+eAc2SS7/yKmTVQeuxIqLLskNHK9JbjJjVWPHlXgslbnnhE422gYytLrWd9dP4E/M
 323Ls6m9PsQwVcUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZvzU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDj1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyEfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW92/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 ghMCjl3ocUTbqmVQGagoE2q+bcG0jbXy32DXTqg/blkwS/xxtCvg8lLM92pAZ3yHtycegC2w
 xoWp4Y4Y2mdfVmHp6VMt1xNvdfOla9MS4kD1jiU2gPNJt3c04l+KSHq4nc2omRCeg1Jd0J6d
 L8bG8=
X-IronPort-AV: E=Sophos;i="5.85,294,1624334400"; 
   d="scan'208";a="53164880"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kbu/hLyaA7s+UP3oXCDkiDrrwmEuPuDQtpEvZJ6xtbvtFoZH7lCbu7sSXyKL+TSz9XxBWoFgSGWRLFSNyWjNB4FRiBqD3GYHlzPIVg5EWukKFX1zDSX3UPHJFolkzsFkliLWKwEz/oE6R73/z0iPoK00CTV0IQwPnzwQTDjt1IKkc+ILZj3q9K7qK/i6/50RqRbCnXLVr9qqegEaP8kuUGaQY5Fgxo048IoQbZmYgnX/IOUCtrNRT4X0/TGAFFWbEyUNkQhcWCvDKDemICcxdd8M7yf8y9qhufnyDh6AWGOivcLAMSrYsXgVy3opcHyB2i8ErXb7OO8TPzWGL0fKpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5W6DN5j2PLL8WQ1KBd6ZY06LmsFB+DZCdaeQe9CMJAE=;
 b=Z+oowZrbZDuZLJud6z58dJY77NWEFa/3OU0ABy7qQ8NWNrsdFoczsCqg4/fhX7gqxs8qHw3CK75id19iVGKtlSEXxsX0nF6lTQx0M9fYb/480w6IF74Of575BFzDCpfugYNHYtZZow/pCWgRYAEH2TX2/MHAVdWvSPTznlu0MNvsOhJgaB0QrJtBdPX1BgEBR5m2Uex4v9ZcKiBYJCt6tZo0w4huV3AbcCyLPBso5pcMS0wIQ7/yv6enCoSOYQD0h1TeA34uIAlrsrKJFjFemfjoGxaPapAR0SYxKdmYPfHbexkXHvOYWdLxKrAFyzdbLXRWRCkN0n5j/zQeXvdQhQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5W6DN5j2PLL8WQ1KBd6ZY06LmsFB+DZCdaeQe9CMJAE=;
 b=gpav5XTel8oKFS5I3aTblv6AzotN3JR9apB6Mh4xbDtZrs40Wjp/NeLLq3yRfCES+9M71XGyc4TP0Alhnm8A6k7RDRN2FzWqzmKhGe9Lh9E7CA+IqXBKDNe4lHlnMwqBI9rTaBxKebE5dnLFztHLfCOtRh5osWjYJB7d5EKXlao=
Date: Wed, 15 Sep 2021 09:50:39 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] x86/p2m: fix xenmem_add_to_physmap_one double page
 removal
Message-ID: <YUGlz9A+MbbzB/W1@MacBook-Air-de-Roger.local>
References: <20210914133419.8407-1-roger.pau@citrix.com>
 <96c7ffdd-57b4-15f6-7fb5-560c6ed0504e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <96c7ffdd-57b4-15f6-7fb5-560c6ed0504e@suse.com>
X-ClientProxiedBy: LO4P123CA0309.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::8) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cbb2f7f3-62b4-4c1d-498c-08d9781d8611
X-MS-TrafficTypeDiagnostic: DM4PR03MB6096:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM4PR03MB6096945CE6DB994B3CE1A15B8FDB9@DM4PR03MB6096.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: abEklmOk1CziIYszf/9L2pGIL3swMyOipl4M6DvIR5PXdo29aGE0k0nxJdj2Hc72dWzvfndaIqMnh7TH3fHQI1XQmJN3O0LIuooXFS5f1a1jcSe5tAVctbBaKu3Qn/M4Jpny2ghK400z1VYYuzRCQUjKAZMUmKyS8fdk1nW2ZsTyflv4i3tcXnwRaM+piqs2EL9E1gtf5g2giPEOcbEz1qaB7XPXjQaH3Pz41xWZc4mDP/JPNJw8Pz19S6RojW7jehtOvwK78DngTLekgfNSpxibebP6hzfuk0/zeaMqc31lFuYZo+zLtCRXsg0XsABfjO33Hu1BTi9CQpPVdLxZ/sroCUzsJKfg0QwO1OSrc+uz1AZ3NwO8j07LEDR1LdS7eahl+vkphx+cmu1x1ZHkDgROHfpVhRoT4Rz0XQtKOJuBxkL1WwYi3cez84YpU2nSWSUUJ5THRYv6TcyHhhA0ejIY74W+wHR7x1KymC6fRlAyEVIjb7U4cM9bKRortVEKqU4+gml+XuzoXfcdkU3Huymr+mq1DVH3vf9ptXRyoHxU56PWej7jbeuooyWZDvswCrrJM2Ko3tY+9H8g/0Jsufs3tX4i3b6VHICGPYN7PlSB6hy5YyghszjapfcQS5Iqn/hbYB7iTHCvz/BC2ka18w==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(39860400002)(346002)(366004)(136003)(38100700002)(86362001)(9686003)(6496006)(83380400001)(26005)(6916009)(478600001)(5660300002)(316002)(956004)(66476007)(186003)(8676002)(54906003)(85182001)(6666004)(4326008)(66556008)(6486002)(8936002)(66946007)(2906002)(53546011);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?T2FlbWg0NFZXWVNBWExsZmd4Z0xKblNldU5NVVNpclNaMjZicitIUUQzQXNM?=
 =?utf-8?B?NlhlbXBxMEdzNXd6Z3FuUEcvYkxuTTJmQ0hmVlZId0dDWGJ6ZGVLTC9ubitB?=
 =?utf-8?B?K0tCOXZUOWpGWHVkSmM0UjNlcWh5a21jV2lTdWF1NlVNbkdGZkVzdVp3MkJw?=
 =?utf-8?B?MDFncjVzMm8zT2djdXdQNnliUURJUGtkSUtCdXF2aVdyQlRpM3lMcGlHMW44?=
 =?utf-8?B?UFB1Z3BkeDh5ZGErMlUzdVkzOFZFUk1Nbk10UHhpS0NjQ2hFWUZ4SDdVcVdJ?=
 =?utf-8?B?Vm5ybnIwUEUyeVo4clFFL09vM09lNlFFNDJhL1lHdGJ5Szk3TE5ZcEV4NGJP?=
 =?utf-8?B?VFBZb3loNERuRUtDS1ZjZFZOOXBVTDJLUGhRNDN0cHl5ekRDNjVEZXpCZGN6?=
 =?utf-8?B?NHBHUklOMEZ1RUFHSElWZ1kyZW9TQWFmTk5BdEJBMW5UWlB5WHZjY09QN1Rj?=
 =?utf-8?B?QnFuNUpNNGYwcnVSRmowQnZyMm53dkplYXhnY0NaRmtuSE9FeWd5NmkwVVNs?=
 =?utf-8?B?ZVNpOGR4eGZabmFuR0VhVnVFNE03NWw3NEJ4TGpTNW9xbXltUUdGY1FTa2R3?=
 =?utf-8?B?V1RjOU5TYmpFK00vRVVnYXIzczN1cGx3d3dvRjFlWHRZdERvTlA3ektiTURi?=
 =?utf-8?B?VFo2Z2dWck11bU5RMmFEbUhEUm1JdkluYmVENUxmRmthTGY5a21WMG84MHhR?=
 =?utf-8?B?S2I2UFlsUUUzOEwyRnNYTFczWHBUL3BBTnlQemJlaU9DcnNUYWs4MGVlWUpS?=
 =?utf-8?B?cE9NQXpKTVFIR1NtVVhXN0FsRHJLWkR4NjQrVk93Sk1qbjY5OGxDc2drL3RR?=
 =?utf-8?B?OC9rZnlvSDRoRWNYYzluQ083WG05and0NW56c3ZEajhOV3BlZVA2Wm5ncTFM?=
 =?utf-8?B?cjIySU9tZ1ZoejUvQXlyQnFQeGkyMHNxNXloRFdQQUIyY2h4YTQyTWRyeHNS?=
 =?utf-8?B?UEk5bE5KSWlOVDlzeDFrcWt2YU9Kb1pUMmRqdDBhMVhrV2I1NUtDNVR0ZDYw?=
 =?utf-8?B?cW9JYUo2S2tGM1RHMzk2N29NUmV4REZ6NldCalMvYkVTYUZZY2N6VjRBMmxq?=
 =?utf-8?B?VlJMaHp4WklFZ3ZEaWlOMi9HeXVyZHpWYWtTaFlrbWx5dmtNUHczT3R3MFZG?=
 =?utf-8?B?dkFoSG5lZnQ4Vk5EWjVvbEdiTjRIVlVJK0liaFpCM2l0L3YraEROZEtCSk5r?=
 =?utf-8?B?U25Xa0QwNEp2SDJTNG14VFkyWlRnUW9aZ1M4SGpnWkF2eEpnZkZrSUVJclQx?=
 =?utf-8?B?ZGRKenVwTUdnUXJwdUJ6bWNDaVBEOXFCMjZTaDFDZzJ4SVZYZ2U5Y2g5amxN?=
 =?utf-8?B?NHBpdEJiOVJETjBNMzBsR2h1TFNxL1NpREQ2UUlaZkowcXNiV0l5MGNweldj?=
 =?utf-8?B?LzZlTjM0dnlmSENPN0RnUlRaaXZyRFZkREhwVG9vUG1jUDJacDdXa2o1ZHhJ?=
 =?utf-8?B?S3IvM1lobm04K1FQY2VTNXN1dmdBS2FWMk5KNy9MdXpwNWdyRjdxMENJTVhR?=
 =?utf-8?B?NEp2Y0JOMnBad1lxalR3b1J1Sm1oSXFhSjRBMTA0RnpEMnFRclN0Z0U5K1dh?=
 =?utf-8?B?YnlsRFhtTGlHU2hpRnFQaFVJU0FZVktVbGo3OW11ZlN0S0NBamx3U05KRDhC?=
 =?utf-8?B?cXhIeE1mQnhIWGo2blh4NEJEVGk2ZDVWVU5YYVlXYzRORVdaTlorMmtKc0dO?=
 =?utf-8?B?aWNPeDVRMC9IVGxvSlVyN0Z3NHdHbXB3bm1RNEFBUDFMNDBZZ3Q5MjZXSnBy?=
 =?utf-8?Q?zYqGHDSgrkgGgsjemoFEGvzCySJXoFnj13BwVzX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cbb2f7f3-62b4-4c1d-498c-08d9781d8611
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 07:50:45.0660
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iI9N07NtCLpHbddgxcxtqxJCXe2RWKMgLGPzHiNNBqtefDQzJwgZWAiEWdc1AevlQzMte6R7rZb3BE/sRTEfnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6096
X-OriginatorOrg: citrix.com

On Tue, Sep 14, 2021 at 04:32:53PM +0200, Jan Beulich wrote:
> On 14.09.2021 15:34, Roger Pau Monne wrote:
> > If the new gfn matches the previous one (ie: gfn == old_gpfn)
> 
> It took me a while to realize that you probably mean "gpfn" here.

Indeed. The variable names could probably do with some disambiguation.

> > xenmem_add_to_physmap_one will issue a duplicated call to
> > guest_physmap_remove_page with the same gfn, because the
> 
> Considering the local variable of this name, may I suggest to
> upper-case GFN in this case?
> 
> > get_gpfn_from_mfn call has been moved by commit f8582da041 to be
> > performed before the original page is removed. This leads to the
> > second guest_physmap_remove_page failing, which was not the case
> > before commit f8582da041.
> > 
> > Fix this by adding a check that prevents a second call to
> > guest_physmap_remove_page if the previous one has already removed the
> > backing page from that gfn.
> 
> Same here (if this remains; see below).
> 
> > --- a/xen/arch/x86/mm/p2m.c
> > +++ b/xen/arch/x86/mm/p2m.c
> > @@ -2813,7 +2813,7 @@ int xenmem_add_to_physmap_one(
> >      }
> >  
> >      /* Unmap from old location, if any. */
> > -    if ( !rc && old_gpfn != INVALID_M2P_ENTRY )
> > +    if ( !rc && old_gpfn != INVALID_M2P_ENTRY && !gfn_eq(_gfn(old_gpfn), gpfn) )
> >          rc = guest_physmap_remove_page(d, _gfn(old_gpfn), mfn, PAGE_ORDER_4K);
> >  
> >      /* Map at new location. */
> > 
> 
> It feels unbalanced to suppress the remove, but keep the add in
> this case.

Well, there's a remove further up which is not skipped.

> Wouldn't we be better off skipping both, or short-
> circuiting the effective no-op even earlier? I recall considering
> to install a shortcut, but it didn't seem worth it. But now that
> you've found actual breakage, perhaps this need reconsidering.

Sure, just didn't think this corner case was worth adding a short
circuit. Let me send v2 with that approach if that's what you
prefer.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 07:54:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 07:54:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187250.336047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPkc-0008R7-U0; Wed, 15 Sep 2021 07:54:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187250.336047; Wed, 15 Sep 2021 07:54:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPkc-0008R0-QR; Wed, 15 Sep 2021 07:54:38 +0000
Received: by outflank-mailman (input) for mailman id 187250;
 Wed, 15 Sep 2021 07:54:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tFB7=OF=epam.com=prvs=9892afeb01=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQPkb-0008Qu-58
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 07:54:37 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 32cbf40c-2671-4095-a3fe-58f061578ea2;
 Wed, 15 Sep 2021 07:54:36 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18F7o3Sd005841; 
 Wed, 15 Sep 2021 07:54:33 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b3c8ug3et-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 15 Sep 2021 07:54:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3374.eurprd03.prod.outlook.com (2603:10a6:803:18::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 07:54:30 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 07:54:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32cbf40c-2671-4095-a3fe-58f061578ea2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XzbNATN7UDSdM9fRW5sHGqBg25Isk8q1RQymYyrRc+1rtXmytE4IKklXAwd+sW52sMvrOnPoux4rUOJN9DH6tkIMSd9d1NY8li2QqzS0xnpkuxMGMBDJxg/WL8594Ql3Lczd4v7OuVBz7berQ/1ZjiBTLBDlfoLyiYyYtQjAoNIIwFu/Zy5ZpE89DkmthrjWxnlhcBLoNrcet5G+w0qJFP0ZAnFtr27olEDMoey3jQ3CWDItl8N86Ru8VYKHVBth/bJwyOQJSb4FNYPdS5F8XhFCS+8KUTgyc+B0xvc7SBiQkxJs7rcaP6ffbypT3IsufnHU73lVlC1T/jeNJWvijw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZCa6er99XD9a2CQzg8T0mQjKx6vdmU9JwbkP0jMKadQ=;
 b=mOKM+rcnRsMlXO8v44yJHo5WXzVxpO6dbzfwteNo9/ynD0QXHKGHzskxlr+1ceYfGjZgdRf6ucdph+79eiD67IwTGyPWj/Bkzh6N+7JyF0O6gLHmvjDAP1licUBcHKuMFLne+Q8T/GvpfnDbw+AjEWAMI6jMddGpbiGbkvEQBGosguvvI/No0uw4hU15fUVohuFi4U+K2BeTWtbzqSAWdVwOgeaGIDTjOkfZQaqpvj+uzHQLb8dkF49TYWrD9/N3MidOW4V/mUPnHKkExnHCMh5C3w0m2puLdwD++6u/ojeWmkdJkEwIKiftuaNBZ9zhumIT8iFuy9DtjGlOBGTOZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZCa6er99XD9a2CQzg8T0mQjKx6vdmU9JwbkP0jMKadQ=;
 b=qnRA7Ypv0I2/21XF+rZpRHUmIVlmbLnX1/+vc9RcWr6c6eSjfq2y4oO6Yx5NH5EmaXl9HCOB7SLrkQxr0fRZaN5DhtaEyUZ/kfT9sGn+6qEsxIw1CF6sXob/9U0EndBd/5+DkFbC/iIwxVoj8BBo235d5sfl7Y29/PbyvyRARmd1vttcCmc5TXBStc7qtPouPrZI4z9LzLmtQg9qbdV0qkBm7jCHvj8xLsBGIxRqAqwMf3DDfKqHOmgEdqAzBVEnwpAzMNqztrOfKc9gF/1wJOjbrW/B2nI9ACGSEZynS7uW/0boNriU1MqVwQwfZDLiOHZmAjOzuO6ZZUuMeHbpHA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Index: AQHXqgbplAv3ejwcPEaBVpikglszUA==
Date: Wed, 15 Sep 2021 07:54:30 +0000
Message-ID: <6841a3c5-2565-9fee-8776-19d5c4a68fea@epam.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s>
 <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
In-Reply-To: <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f876b05f-41a4-40f9-35b5-08d9781e0c60
x-ms-traffictypediagnostic: VI1PR0302MB3374:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB33748616B877C03512A223C3E7DB9@VI1PR0302MB3374.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5516;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 bxqWkarGUDFYBgNEUj/0zL4+K2nrFQhGBFiQtdHTe4GpFg1Qhp0JWPtgeQjyEABbPIsLspMQAeJqv8s6jtVDCSaVnz8C50fQRoyvf8hqRZaeVc9EovdFw34c+hE0C0lGwIrt4IwND8x16mtsCpUssHPTgXTmHNjO+YOknd1MUlP+fxnICncErqizrbmV1ccTK/c975ruPQlT8UbPjYs8Jwg6ld2F32bz9GtyZZMnoLWg4Uo2pbgFTCAyDj76oo8BLaUOYWzhDWstdvUw+r4X0bJNpH1I0ret7DAucZWGziO28zMVkrpy8vTqbRKGZnpydnbIRG1vRaBWL8Y7QXlVwkgQaUp7Y2ntJ17XgwxXLRyIKAACd71ZJG2VMqMGd/7ynk/EFZryYemugFMNFnQT8vCsvAnvNgudI5RbbME9fP7QvJqMZvfoY78d3wZdaCXGN4pWPsSERQcxV88gUByqKA7j9YwAa5NEyyMTXyzwwXBvO6AGbJTExFH+Lc3QbyqWN1AAcxh2Ww9JjfrBjaZdWcRce9m9TOO+wVuahMS16QqI8O6dJTcg5jRWF7n4GyPiaj5nzFELPF4mLLbyk8sIjBpRyrOueW/tAUVluV/n4rGEpvMymYlYc1HkJEMH4of4iTrm1GCdrhh7nqQGPZTFX8S2sM+vLZRniLheooEQlxhM7CQVFTeM5cGX8aRczNOzybmPVW3UYOOCLzNAce4QZ5h04f5eTJY6pZO5cK/hPb1HU9+bjvirWSLvpIazfUjE
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31696002)(8936002)(31686004)(2616005)(2906002)(66556008)(5660300002)(55236004)(66446008)(186003)(38070700005)(6486002)(4744005)(36756003)(66476007)(6512007)(122000001)(64756008)(66946007)(71200400001)(86362001)(8676002)(54906003)(38100700002)(316002)(6506007)(110136005)(508600001)(107886003)(76116006)(4326008)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SGdzc0hObFhKeGwyV2ZFSzByb3JQY0lwN3JHbFdTeGZFc0ErNThQbFgveC9r?=
 =?utf-8?B?QXd0Q0U4ZDlQbDEzcERXYnU4L1RXd25YQXc5eWQxWm12enlPSTZTb2VrZURQ?=
 =?utf-8?B?ZnlUQ3VxbFRJTDVWNDczTXRGYTY0ZW1wUkUrWFQ3dW12SXFUOTNFdFJJUkxi?=
 =?utf-8?B?SzFaZlVCVitmSmJiK0VnakVqUmV3aUVGbldZNEFkeSt5L2JNK041WmFKb2dr?=
 =?utf-8?B?M0FtU0laSUY2NTA4TmMzd0p6VU9MNlIzVDNZTTYzT09sbUZKQmpLNTF5MmFk?=
 =?utf-8?B?T2tWVG10alhJUXZHakxIYXBpUkdNTnRoOEtmazJmTk5lV0UzOGpQaEpIcHFM?=
 =?utf-8?B?T2cyUVc5YVlGZGtBU0FqTDVlUUtVSGQ5NFArejZ1Z0xNTjQrRU5PUUYxdUdW?=
 =?utf-8?B?eWRrb2lWeXRUcFZrQnAweHdwcDM4MEpIdmI5QXVLdlBnVEJyd0JLTXFTNVRi?=
 =?utf-8?B?dVhYY2VFS2txdEJ1UHRIVWdWZXgwWGRLdmpBN3RLaytYcllzZGlCVzNFRzd2?=
 =?utf-8?B?aFk4YXVoSnpmRTBMdEcyMW9nejR4cVVLcndueWw1QVNxcFVkTHRSeCs0ZzRN?=
 =?utf-8?B?VHMyRlJNb1VhOFFGekpmV0Jhc0kvcFF1akF0aXJRbXFVVTNqSEtjeHhjV3RH?=
 =?utf-8?B?TW9BN3hweUJNWHY0cUpUekMxanRTQ3ZQSTFPNVRYZU1aamJ4ZXFnaDI3amZn?=
 =?utf-8?B?VkJPU2p1bzVoMy9oaXU2aTlrcDlDb2Z2OWN4UTY5azltR1FkaWE3RmhYZ0Z3?=
 =?utf-8?B?ZUxIejErTFZrSE1DWTVqWi9FZUs3UlVJLzFaY1UwOGtKMHYwMlNyNmtSa1ZL?=
 =?utf-8?B?TFlBeG8vQ0Q2VUJWcmQ3Z1pIWEtiZE1CMFFiNlFuR3lBOGgvTE04eG5oMDh3?=
 =?utf-8?B?cVZBaVIzTnQ5M2piWVRQYm5VOG5ZVXVFUitNa1RibFNYK3Y1Ry80a0l0MVQw?=
 =?utf-8?B?VEhXN0IrT3RiWmI3cXQ0eEpOcnlwZC9UNHVxRmpuREZ3RkpsTnMzcHZKMzZH?=
 =?utf-8?B?Y2ZOZ25HSFBGajAvT1doRm43bGFHbEw2YldwdStWOFRKd0xGMzFSV3BKMHM2?=
 =?utf-8?B?Y0lNUTdjaSt2ZjQvWGZPcXZyR1E5b2NJa3RjTEJ4RkI2eHEyaGZmRmhxakNZ?=
 =?utf-8?B?K25IM25WcWhPRDFOdmsyZ3U1TVE4YWkwSTl5cHJ4SHM1REtxZFp1NkNMZTRY?=
 =?utf-8?B?Z1N6Wi8vUHVNbmEralN0OTlYNUdldUM5b2cwUjRteW9aM21UUHROWm82ZUgy?=
 =?utf-8?B?OXVEMDBNc2F0TksrTDZVcXBjUlVnV3BXZTBsd0syd211R2JZeVJrdmU1U2FY?=
 =?utf-8?B?elNHUUswSzQwamxEa2t0ZXloRUlpL0RTWXRDemtWWkZNeG5rNVJEQlppQytk?=
 =?utf-8?B?MHdLRUw2MjRyNlAzZ3l0Nk9JS3hiMWt2Qk1ZQmxqUEdJY0swUW9IdTdxS0ox?=
 =?utf-8?B?d3BCelF4b3BSb3RjazZac1hqU3hEN29mc0dFazN5MHRkZVV4T29KTUFQc0J1?=
 =?utf-8?B?b1RudVJhbE9RaEFta04zMGhNMjloNnV0Z1M2MFVlaTlpMHhoRUVaQURxZVVN?=
 =?utf-8?B?U3dJVmlNeDVKYmtuamFOaGk2V2lFTjNvRCtkcCsvU1JXaHlrREJ0cEJUcG01?=
 =?utf-8?B?YWdOSFJkY1UzWVgyblVkeE12L0E5UW5UVytTeitTejJRSmFTMFRpeTltRVFw?=
 =?utf-8?B?REZJUFdDVmVUckc3MUt4bDVhT3AvM1BVTXV2UXRrTUZzbHVuam1DMzkxQllk?=
 =?utf-8?B?UE5lRXE2VFZBU1ZVOFpWRXJKSmNRekFXS1A3aHJJZ2RVVjFVTllGSEt0STdW?=
 =?utf-8?B?MTNraitveHJrczJmTFFoQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <EA1AEDEEAE0CD246B446D1066061BA2D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f876b05f-41a4-40f9-35b5-08d9781e0c60
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 07:54:30.0803
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sWf4f0mS0lNPkLFiA0VexDbIYzkIX+II7a/gH3J7b9kAy164ZwgGrQNRy880mW5xVX/Dg1eli08j+4noXZ+Vst2N9K8L6U2wqcUCccZl79khHsSYLFSzggHH4xvOl6gs
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3374
X-Proofpoint-ORIG-GUID: -n0GZ274pfsABakNjRUNRlrhxkxBZzM5
X-Proofpoint-GUID: -n0GZ274pfsABakNjRUNRlrhxkxBZzM5
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-15_01,2021-09-14_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0
 priorityscore=1501 phishscore=0 clxscore=1015 spamscore=0 mlxlogscore=851
 lowpriorityscore=0 suspectscore=0 mlxscore=0 impostorscore=0
 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109030001 definitions=main-2109150050

SGksIFJhaHVsIQ0KPj4+IHN0YXRpYyB2b2lkIHBjaV9jb25maWdfd3JpdGUocGNpX3NiZGZfdCBz
YmRmLCB1bnNpZ25lZCBpbnQgcmVnLA0KPj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCBsZW4sIHVpbnQzMl90IHZhbCkNCj4+PiB7DQo+Pj4gKyAgICBzdHJ1Y3Qg
cGNpX2hvc3RfYnJpZGdlICpicmlkZ2UgPSBwY2lfZmluZF9ob3N0X2JyaWRnZShzYmRmLnNlZywg
c2JkZi5idXMpOw0KPj4+ICsNCj4+PiArICAgIGlmICggdW5saWtlbHkoIWJyaWRnZSkgKQ0KPj4+
ICsgICAgew0KPj4+ICsgICAgICAgIHByaW50ayhYRU5MT0dfRVJSICJVbmFibGUgdG8gZmluZCBi
cmlkZ2UgZm9yICJQUklfcGNpIlxuIiwNCj4+PiArICAgICAgICAgICAgICAgIHNiZGYuc2VnLCBz
YmRmLmJ1cywgc2JkZi5kZXYsIHNiZGYuZm4pOw0KPj4gc2FtZSBoZXJlDQo+IFllcyBJIGFtIHBy
aW50aW5nIHdpdGgg4oCcUFJJX3BjaSIuDQoNCnZQQ0kgYW5kIHRoZSByZXN0IGFyZSB3aWRlbHkg
dXNpbmcNCg0KIMKgwqDCoMKgwqDCoMKgIHByaW50aygiJXBwXG4iLMKgICZzYmRmKTsNClNvLCBJ
IHRoaW5rIGlmIHdlIGhhdmUgU0JERiB0aGVuIGl0IGlzIGJldHRlciB0byB1c2UgJXBwIGluc3Rl
YWQgb2YgdHJ5aW5nIHRvIHVuZm9sZCBpdCBtYW51YWxseS4NCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 07:58:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 07:58:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187260.336057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPoI-0000od-GU; Wed, 15 Sep 2021 07:58:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187260.336057; Wed, 15 Sep 2021 07:58:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPoI-0000oW-DY; Wed, 15 Sep 2021 07:58:26 +0000
Received: by outflank-mailman (input) for mailman id 187260;
 Wed, 15 Sep 2021 07:58:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQPoH-0000oM-I7; Wed, 15 Sep 2021 07:58:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQPoH-0000LM-Be; Wed, 15 Sep 2021 07:58:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQPoG-0006wV-Q8; Wed, 15 Sep 2021 07:58:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQPoG-0001pR-Pg; Wed, 15 Sep 2021 07:58:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iYlED1r3jq+gS+oMO7+yosdlr9puJ/kBOSYYDlmxFh8=; b=1lK0dvk+17T8TnWp/8ymntGq1Z
	U2LaGDHOh+tnKWgddnA9RbX/MJuQTVv1Fy9w91IDdXok0SvFTwrqI0Jb3z4D58STFBeeIObr2Hwr4
	BNmdB3IsdmyzcwM+Aza8qGLhynbSz9KYld7/fYB6QAgV0szPTdtuO/x7qqi8zoJH0ZSY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164985-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164985: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=4c9af1ea1457782cf0adb293179335ef6de942aa
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 07:58:24 +0000

flight 164985 qemu-mainline real [real]
flight 164991 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164985/
http://logs.test-lab.xenproject.org/osstest/logs/164991/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                4c9af1ea1457782cf0adb293179335ef6de942aa
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    3 days
Failing since        164967  2021-09-13 13:06:52 Z    1 days    4 attempts
Testing same since   164985  2021-09-14 18:09:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Lara Lazier <laramglazier@gmail.com>
  Marc Zyngier <maz@kernel.org>
  Markus Armbruster <armbru@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1087 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:04:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:04:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187274.336072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPuY-0002nk-JM; Wed, 15 Sep 2021 08:04:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187274.336072; Wed, 15 Sep 2021 08:04:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPuY-0002nd-FA; Wed, 15 Sep 2021 08:04:54 +0000
Received: by outflank-mailman (input) for mailman id 187274;
 Wed, 15 Sep 2021 08:04:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQPuW-0002nX-Sv
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:04:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df169c4b-1fa0-4bd3-a69a-f7f12dd4c625;
 Wed, 15 Sep 2021 08:04:51 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-14-X9jyxgW-PO2ucaABi9Wqwg-1; Wed, 15 Sep 2021 10:04:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3392.eurprd04.prod.outlook.com (2603:10a6:803:7::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 08:04:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 08:04:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0003.eurprd09.prod.outlook.com (2603:10a6:101:16::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 08:04:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df169c4b-1fa0-4bd3-a69a-f7f12dd4c625
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631693090;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=r132eiCb14LF3oKvpumOcwCR1jNghL/zwsEyh0UIrSY=;
	b=iM7zKMbi3pHhB8x1etviWeebo+m/0R5KzFgIwi+fNau7X0x0XU6EqW2XiM9GRCuJ5vdJag
	Wgg00vwypfuP0z+UZ4k2ZclnKjpY982pXATqmY6tXwr8fRYPvByGk/qA5lPy4wUSh4hWWZ
	ZKWYcpjHRNTpNXGrQNRJoyOLaDE2+lg=
X-MC-Unique: X9jyxgW-PO2ucaABi9Wqwg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YMjeqgtxZOzecfOl9XVohA844BHun1geGRBwe4PoCp11H1PpoYWP0swF8TS52ACHsyTEJDoS2VLIZUrEmUfDtwZ45RCWJU/ERwW9uVmvbCqqw3iQj3jKYJifJM3zfzhJ42Lsg1wVjQOmHQZDLx3CQjpH58pxM+/XoGynIWwKhnOwBJbo+D280E2mzR82FGnBnlVeyW9wHM79d2VazMHvI/TAcuA3V7Z9wjGY0aOG31rBam4WXHvKcEwtup2jwMz1MFzS+HUxnl3u6LYaFPyDMVE6agmLC4nCZe2FIHYyUAH+M+VovnBTnk8rMG4VnmSry/yTunuxYWZ5i2CIFfo7Fw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=r132eiCb14LF3oKvpumOcwCR1jNghL/zwsEyh0UIrSY=;
 b=Vx5EZs817tlPtQPk6h1iejvDcSvCzI0UPrdKoOUXXPwrhrbVbs+lJcqcYSNJy5sa/rAH3LjHRo/VOCgONvoVub8Yf3ClWRnZVnC0PaHJUhjSteZVemvMIZ5T4UXDO+fQrfoA/rIDZi1P8kPkclxPRqARhycE6BGXLn3uJhUQ7YXneyRnGcj+nr9xuu6lLqsV9bnGRpgJ4ZF+A8yk2Z/AFvu3iYSMB7HPgCTcGf5lLoWBHlvOhUBbP0raMoeTBg/mt7Y2r2mlijOldKzu5pzYRPptNlv5q0xGSPZQAHzLFrSNItfy9o/uiHo2HmkJ2rUf2en9GBGIrstWxn8l2S8mZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Penny Zheng <penny.zheng@arm.com>, Bertrand.Marquis@arm.com,
 Wei.Chen@arm.com, xen-devel@lists.xenproject.org, julien@xen.org
References: <20210910025215.1671073-1-penny.zheng@arm.com>
 <20210910025215.1671073-6-penny.zheng@arm.com>
 <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com>
 <alpine.DEB.2.21.2109100916270.10523@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d2e87c7a-08e0-3cca-fe4a-6278d6939591@suse.com>
Date: Wed, 15 Sep 2021 10:04:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109100916270.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0003.eurprd09.prod.outlook.com
 (2603:10a6:101:16::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1cb274df-df84-40e3-84de-08d9781f7c57
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3392:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB33927FD29567B1915AC436FCB3DB9@VI1PR0402MB3392.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6Upn39ntDiY6yaAuSbcZsITtJXckDJm0su9ezkgPSIEmjAOdADPhZDQWxndmMzBN6/exAK43AuyToxWUcPGKJhYlWa2JCH3+pHRwb4DxXcz3c+N7TrMFi87mnnLRkF8RSW2RmM1VjMsZaCpPFpkPnawS4Bp5tpdcCFvyDCcznCCOc1iCJzkGm3TA6w34rMOCL0/pSW3DuJN2qrvEpQXptM9Q0i/A4yvkv+gmRlyfIFzdTabVuc4qxaZapPCE2bn3C6KxPcOqyPjn/MZfCQ5dEtShFD2Tl0RSLNfFPvyOTDBNv/Vad7rhTKQKLr8GkJY21AFexY+kRLkRPnVOm5JFh1K/qPJqhZuelqZbe/Y2OMp44TPs0EDcngW5bXcF2N5yOHB+E30vfwccaVEcrRbiTpbfCLWKYxhSZ1CCUFnKL7YllZj2VTDhRu0UrAc2IQ/9v7TQOOKzQ9o26OYl3EvUL17Ix1yKMR7Pln7rjrwohvvSNwLIuH1rtJTsLtFGPbU9UDLdFL7QRMh5m/4paNgebu6aJkVoJIMys5cL+oFkUcTB46zYxNzwoScu3tbtCUOkZZSFzwtjv8InW9nW+fHZuoacxmEa7HsIzvpBx4qOSAIWoosNdeJBApbbw4oNvBnKmf0YPcr4rMc9FJKU9AwHWDI4tEFGiwb6ztyU8aPT1EQWy6b+H6JA+L/6u2IF/EnO/zC4tvrfSW3yT/SukluH1uc+U3liwVB6AsDevbcHr7Q=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(2906002)(66556008)(86362001)(186003)(6916009)(31686004)(508600001)(6486002)(83380400001)(31696002)(53546011)(4326008)(316002)(26005)(16576012)(66476007)(8676002)(956004)(36756003)(8936002)(38100700002)(5660300002)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K0FXOUx6aTZyRFlzakFHMDM5NVgrL0JVUEllRFNVNWdZUzVkRXNzU0RzbEZz?=
 =?utf-8?B?blZhd0FkS2pNQmVSWWpYRXVGOEkrdlRxSnlLSzNTMDJJcElQMGo0TmVpN0cv?=
 =?utf-8?B?TWdVWkVrZnZ4cmVONk53YnFyM1pWaFdFbEc1ekhCVFVTK0YrbUVKUjg1UmVn?=
 =?utf-8?B?ODFCSWt6VWw3MW1ZTkNpZDYrSjZQT2pDdVUwM2tFUittS0doNGZJbVVLZTJB?=
 =?utf-8?B?VmVSRDZqV2RBVTc4UGkwYkd2bE9NZkJSUDdKUm9aUUVnWCtYVHdCbHRiSDB4?=
 =?utf-8?B?NEdUN0ZGMWt6eGpxVENwTUR0RFhBVEdBVzFZZSsvUkVZMFNNRmVQclZIR0tP?=
 =?utf-8?B?aGVyeExrZEcvQUdNTzVEcWxqT0d4V24zVHNkT1pCc3lRMDk2WFg4ZXA1d1My?=
 =?utf-8?B?OXUrc2x6Z3NjaTJid2s2VDJqb0lPTDlvS0x1WmcvZGc2ZURrSEZGN283NWQ3?=
 =?utf-8?B?b2NFS0llQXp3MzdjRmZZclVPdWVCbHZHTE5XdzBLRzd1ei9KRkhUcDRvM3ln?=
 =?utf-8?B?aEdvRDZxclMvM29CU0tGeFlmZDR6ZC9HeEhzaVRGcjJxUE5wZ3BVVWVXOUR5?=
 =?utf-8?B?UWR1MlFXeVBudDV3aENueFQ4czI4SWxqSDJoUnVRbEZMSk9RZVJ0VmRmeTgr?=
 =?utf-8?B?VjMzMUJUajZPdUxBd0VyRXZQbGxTcys3MUg5WmlobThxQ0F1Q2VsYm9PTkRj?=
 =?utf-8?B?ZzUrWGVXWVRhYTVuM2RlZjdsTHp4bStkb053a245bHhYM3VjRnd0WGVPSkFi?=
 =?utf-8?B?eDQ1NXl6a1NaMFRya0RSZVJ2YVhWcG0rV0dVTk5heG5SdE85QWk4dSt1a3Z3?=
 =?utf-8?B?bmttODNPd3paVXBiZ3VGWEpYS3pld3Rhc0FOaElZUURZZnZ6NUQycDZlM0Vx?=
 =?utf-8?B?ZkJhUzV2TGV1ZGlweHVPOXNGazNqQzBpT2F5YjlCNXRKOTBBOVE0THFmZWhH?=
 =?utf-8?B?Y3FvQ0owWVlrUjcvZjVwc2xyR0FRbVhQS3lQT05OSmhySmtJUWxkZS9vcldR?=
 =?utf-8?B?V20xUm5yREw4R0hyampFQ21rOFBUZFh4UXE4cWRPRU8rdWkzWnltYnkxWEk3?=
 =?utf-8?B?MjRuU3lEVXdjVTNqbjREd3pSUDNvVUxwVGE0OTdTaUpMaFpFN0pGWTA3dUVF?=
 =?utf-8?B?OExVQkx0VzFWRC90L1ZPOTF0bG1xVXQ1NDRHZURlRVlzbk1BLzVjcEdYM3JM?=
 =?utf-8?B?NTk2Z2tyeGhBdzJDYlBjczc0TU5VZVkydUFsLytESFlNR25ReUsvcmRCQk1J?=
 =?utf-8?B?cVlodWpIV3ViUXNRREtNaVRqb1V3K09STGV1Vjg2a0VWRWl0R1l3cWNwdmph?=
 =?utf-8?B?eGlMaXpSa0xSdUhXSDRUVUhZM25OUE9zc0kxT1N2YklkOHFpb1ZodVJ6WG5W?=
 =?utf-8?B?VUtVdllZVzVadmdNcTZMYzNTNGJUUklTNUwyc3VhTlJTVEI0Tzl1NGZGYjFK?=
 =?utf-8?B?Nk9IbUUzc2tMeS9nMzlBWTVVcitTOUFwZVNFZldRTisrM2pZczB2OUhnQ1k5?=
 =?utf-8?B?V0o0cWFzNmdaWWNGaUtuMER3WXg2TzUwcDZaODgzMG1sMkIyNjVpaXdkSWta?=
 =?utf-8?B?M1piR1lSR3NtMGxwZWRiclNzOWNCUG05dUpkamhnOENQNUhDYWxEMDMwcFFP?=
 =?utf-8?B?MWxabEZZbkhMWFFlNWdVUzdnQzJ3UVJOTUhySi9LOGUxN281V2ljOG5DWkhB?=
 =?utf-8?B?anZIaXNrZFowY2oxNWJGSWZoZTN4ci9RNEN0Q1V2UmlNcXNxOWU3cmRkVmR5?=
 =?utf-8?Q?OY/HnKusTAHOFNo/Il0zR4zV8fcrjK9312vPNqG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1cb274df-df84-40e3-84de-08d9781f7c57
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:04:47.6724
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: f3tQHYXsaOWkoBK7/N9we58RcGZ/YH1YCyHXsd4zh+eOFBF0wAxSNS8VVDb63RvLxV8oo98xkuoMRqrZI1tAhg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3392

On 10.09.2021 18:23, Stefano Stabellini wrote:
> On Fri, 10 Sep 2021, Jan Beulich wrote:
>> On 10.09.2021 04:52, Penny Zheng wrote:
>>> In order to deal with the trouble of count-to-order conversion when page number
>>> is not in a power-of-two, this commit re-define assign_pages for nr pages and
>>> assign_page for original page with a single order.
>>>
>>> Backporting confusion could be helped by altering the order of assign_pages
>>> parameters, such that the compiler would point out that adjustments at call
>>> sites are needed.
>>
>> Thanks, this now takes care of my primary concern. However, I (now?) have
>> another (I thought I would have mentioned this before):
>>
>>> --- a/xen/common/page_alloc.c
>>> +++ b/xen/common/page_alloc.c
>>> @@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
>>>  
>>>  
>>>  int assign_pages(
>>> -    struct domain *d,
>>>      struct page_info *pg,
>>> -    unsigned int order,
>>> +    unsigned long nr,
>>
>> If this really is to be "unsigned long" (and not "unsigned int"), then...
> 
> Thanks for spotting this. I think it makes sense to use "unsigned int
> nr" here.

I see you've made the change while committing, but the subsequent patch
then would have needed adjustment as well: It's now silently truncating
an "unsigned long" value to "unsigned int". It was the splitting that's
now needed there _anyway_ that made me wonder whether the patch here
really is worthwhile to have. But of course acquire_domstatic_pages()
could for now also simply reject too large values ...

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:05:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:05:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187275.336083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPug-00037p-VD; Wed, 15 Sep 2021 08:05:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187275.336083; Wed, 15 Sep 2021 08:05:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPug-00037i-RZ; Wed, 15 Sep 2021 08:05:02 +0000
Received: by outflank-mailman (input) for mailman id 187275;
 Wed, 15 Sep 2021 08:05:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wWxD=OF=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQPuf-000370-GW
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:05:01 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9fa1e35e-15fb-11ec-b4e7-12813bfff9fa;
 Wed, 15 Sep 2021 08:05:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9fa1e35e-15fb-11ec-b4e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631693100;
  h=from:to:cc:subject:date:message-id:
   content-transfer-encoding:mime-version;
  bh=0MKXOtuPLz/CVTk0nxd0ruLUtS08Q8L9fwLdhUWEhMI=;
  b=MUgRRvzfPjFA8vPL8zuw45G1IqO+ntj9sAbJcCxe+VBcNMXQPvumsy7q
   xjboiW3ftNZOleBTupKYUOdSB6CcOuuk0txcwqUrDabb2WiciFfv1Yt4K
   by0q01OrAa5ZCHtsW6agNCBmO8HUVnyvKyYmzX6w/nf+s882jfgQG1Bk9
   E=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: dYo5ACvcKxygtG2bTpTTtE31Gdykrtcf3FYMQbzc+vKIvFPrdaMJMU5CqGAE9+NkovO4FPdzio
 dmzaoH0ISIlwXpRG6kNbqF1tMNrngCGW/6yppHSU/AF9FWgtETrnGeR3QaCXAsLvBOc69R0v+f
 IMXZINmGWZRu51fxd+FBa3vDow1aq03aT4wC+c/PowKMR2dRBFENU71NPdzNFtcmpPIz5EiuZ/
 8NDwMLIkOOGjPZ1xqiTtbw6Pnt+EF/5cuiUd+3Tlu6SuVNh/DEEWe7hLYSbEdLYROTZONsJI6c
 z7qhJ3fLUCWRJqJ8mZBcQGxm
X-SBRS: 5.1
X-MesageID: 52750017
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:nN1NIah/3uJyoh+VSTqfLh7rX161XhcKZh0ujC45NGQN5FlHY01je
 htvWGHXa6reZGLxcoh0Ptm+pE0AsMXVz4BjSAI9ry1gRC4b9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8x3IDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1RrbiyCj4FBpH2kdoQeRkBKAh7P45JreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9t3J8TRa6HO
 KL1bxI+SzX7SBR9a25HJ7cetbyZvXz8TTxx/Qf9Sa0fvDGIkV0ZPKLWGMXRUsyHQ4NShEnwj
 mDM8nn9AxoaHMeC0jfD+XWp7sffkCW+VI8MGbmQ8v9xnEbV1mEVEAcRV1awvb++kEHWZj5EA
 xVKoGx09/F0rRH1CImmN/GlnJKalgIEf9gTKsEC1DuE9Kf+41+iL20GbTEUPbTKq/QKqSwWO
 k6hxo2yXGE+7+LLEBpx5Z/P8mjjYnF9wXsqIHZeFFpbuYGLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3m7ESnQkSGcICuTrsoCNtVgRiG2NPdXABb3nARBodtjxor6p5
 iRspiRmxLpSZaxhbQTUKAn3IF1M2xpjGGeN6bKIN8J6nwlBBlb5JdwAiN2ADB4xb67ohgMFk
 GeM4FgMtfe/zVOBbLNtYpLZNijZ5fG7TrzYugTvRoMWOPBZLVbflAk3PBL49z29wSAEzPBkU
 b/GIJnEMJrvIfk+pNZAb7xGiuFDK+FX7T67eK0XODz8jePFNSLKEOxcWLZMB8hghJ65TMzu2
 483H+OByglFUf24ZS/S8IUJKksNI2R9Dpfzw/G7vMbaSua/MG1+WfLX3507fIlpw/ZcmuvSp
 ynvUU5E0lvvw3bALFzSOHxkbbruW7d5rG46YnNwbQr5hSB7bNb99robers2YaIjqL5pw8lrQ
 qRXYM6HGPlOFGjKomxPcZnnoYV+Xx23ngbSbTG9aT0ycsc4FQzE89PpZCX18ywKAnblvMcyu
 eT4hAjaXYACV0JpC8OPMKCjyFa4vH48nuNuXhSXfokPKRu0qIUzcn7/lP46Jc0IOC7v/DrC2
 lbEGwocqMnMv5QxrIvDi5ebotr7COB5BEdbQTXWtO7kKSnA82O/6oZcS+LULyvFXWb59aj+N
 +VYy/bwbK8OkFpQ6tcuFr9qyeQ15sf1pq8cxQNhRS2ZY1OuA7JmA3+HwcgQ6fEdmu4H4VO7C
 hCV591XGbSVI8e0QlceKT0sYvmHyfxJyCLZ6u44IRmi6SJ6lFZdvZ6+4/VYZPRhEYZI
IronPort-HdrOrdr: A9a23:qnp/LK9lUvKu2s++UZ5uk+E9db1zdoMgy1knxilNoENuHfBwxv
 rDoB1E73LJYVYqOU3Jmbi7Sc29qBTnhORICOgqTMyftWzd1ldAQ7sSi7cKrweQeREWldQtqp
 uIEZIOceEYZGIS5a2RgHjaYqgdKbG8gdyVbIzlvhBQpG9RGsRdB1ATMHfnLqQ6fngMObMJUL
 6nouZXrTupfnoaKuy9G3k+RuDG4/nGjojvbxIqDwMurFDmt0Lk1JfKVzyjmjsOWTJGxrkvtU
 DDjgzC/62m99W20AXV2WP/54lf3PHh1txALsqRjdV9EESnti+YIKBaH5GStjE8p++irH4sjd
 n3uh8le/9+7nvAF1vF1yfF6k3F6nID+nXiwViXjT/IusriXg83DMJHmMZwbgbZw1BIhqAz7I
 t7m0ai87ZHBxLJmyrwo/LSUQtxq0ayqX0+1cYOkn1kV5cEYrM5l/1RwKpsKuZBIMvG0vFlLA
 E3Z/uspsq+MGnqI0wxh1MfgeBFBR8Ib1K7qktrgL3j79EZpgE886O0rPZv7Evoz6hNO6Ws0d
 60RpiApIs+P/P+UpgNd9vpOfHHclAlYSi8eV56cm6XXJ3uBRr22uvKCfMOlaaXRKA=
X-IronPort-AV: E=Sophos;i="5.85,294,1624334400"; 
   d="scan'208";a="52750017"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dgUPpPy/Puj5IkCcKEKeP+RpArtBF5TM3gjJaXUYqDHFXMC/3HWXBIPUY+7L3UAh4uc8GgbrzQh9MfOYWnv05fDxWaDRGg4zmZxpeQdHjUmQbfVnMqlAEmgyC2fQuTNoSYz3E6B54SDphrnc3Xj/6CJyAU5QFi6FtgDDiJ8P0jdJsxC6/J6/2VRJfwz/yDTmvTVp+INPjYKyP0jmMXsfBo/CnbIu4O8TLO0k8W/RakMEAuiIQyBh5fJwvdAgSFH6tMrMTIg8HQc5LGRvufilkkBhvYdV0pb2ej6+a/pPCjitY+33O5jH7RqXEkyE7ryYlbqVhDKOUp/Aw7ExO69E9Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aGqy4nQE3HdDQKTl4mOJt2/JdDVwHm0kVMcZrpQQVcs=;
 b=QZyWV/qDICjutggGo8ipksRBO1WltzasKUOst1cM1S5cId9bTBdC3LJ7Ub0zNtl6VzGoJ4csMiZa0OZLFK5eoe/i9SK9cLWjsD/JMhjQ/fmmLR8VUhInF/g9P3AAgo/8+xURV7Ab8SKXjZ9yc+6oQ6gcqIMTfmLbOW5D0uhL8o/IPlBfwBNACN/uDAx0ewABKOTESclDUyExvfgyi40Alp8mlmstdQmK7NSEB6MgVfYw6iF03Gvx1xyzu9nIVW7teVxIVtD+v5VJVyXmDTVVHNb5DBHLpHxinZ5h/mkPqYre+tBo3JtucDVi3Xtj/o9RR+h9geYznIACzcNJJ25vZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aGqy4nQE3HdDQKTl4mOJt2/JdDVwHm0kVMcZrpQQVcs=;
 b=S5ZT3WSyOei/M5Roje+NRSyAMT7MfEmUZwnjsqSpMXH6YiToh109wmFrtp634S1VMtBTyrWmBaJWHIGSK4mfe9/0NM/o9hM+KI13PSaW1zFitxP5wdknj6MFTv9vKcl0p5dmErDuTOaWv95/wYDULs4CNOo30s7YKGALv7osv9Y=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH v2] x86/p2m: fix xenmem_add_to_physmap_one double page removal
Date: Wed, 15 Sep 2021 10:03:42 +0200
Message-ID: <20210915080342.21346-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0102.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:c::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9c4a3501-11a5-4e0d-d2d1-08d9781f8176
X-MS-TrafficTypeDiagnostic: DM5PR03MB2636:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2636C744EC5EFF57428F73828FDB9@DM5PR03MB2636.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:655;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: OY8LCeB2VmP6T7oy0TFh4RGommL+zpaq7XKQ9aeSirIKUp23BvSMAxh5ZpgVgnJUcB93N/uT3AjKNJXwEujaL28IFZcirciVp7KbFUBJo47Fn9lkxQEDZg2IlbqeqItspcqC4JEvxgnkTDYiCp7QG21Aan/ZlKm4bseY0+OASb6WV0tq/SEQY7xpxQW/AacR4Gtv8uU3l24izg6gjdSfSyYcBYXfj1tz6YrGV7xCIF12QzOgKs8anLN/oMOSLTi66/S/XRYBVTNddS+xWwr6se7b5STMZejYC5fNMqQXN0i5wRlgZabwpR0MKJ4XQ/D+tg9S7rs+dnOyCuqwn6bTRsE75t/7pIxAw/VxTR5VOKR7DlRVjDsWE9AeUNmcNgZsLlhe3GgRv/0Ot655a4FeX7tUQMizhhfZkmbiJY/n04FOzyKUFhlaj8ronUHUYI2eCTEqvm3IAksxPRxU8kqckKOOaLq2KjRcqqL734lPic6THx0emkRe5NsCUQiAs1kw7L0GLeE5A83EXHESIUuW4Ghk2sl9o3QhAOd31MHf82QdsCJ5Ept15uFrbeYfYu/sXJBnCG5VMiI1mA8+wZACU/LZVrqWOv9vLjUKWnK7PxRJ4QMaRh5VETbqB7fE7EyDKnbl4ZnQFhGA+VAGg84r8g==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(396003)(376002)(136003)(39860400002)(366004)(2616005)(54906003)(5660300002)(316002)(1076003)(6916009)(8936002)(956004)(83380400001)(8676002)(26005)(478600001)(2906002)(6666004)(66476007)(36756003)(66556008)(38100700002)(186003)(4326008)(6496006)(6486002)(86362001)(66946007);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGxabG94ZkZSQysyZzk0OWlzRkNOQW1SRXBrcDJ6czBFbG94VG1ybStMWDhY?=
 =?utf-8?B?b3RkdXluaUJNNlJDRzNCZk5QUmlrdzZROWQ5cmNKSEwwRElvZm9oc0ZUWmpJ?=
 =?utf-8?B?NDk5b3B5emthb054ajcwYXAzNkMyanVucXdrRkhhQUFIQ1BibUdXTDg2Y1JN?=
 =?utf-8?B?YXVyUW1wc0FFbEVoQWh2a3RjcnhGSTZRSTgwUDJsY2daYlFpU3VBWDBmbmhL?=
 =?utf-8?B?R1ZQUVF0S0JrSHNwY3dvQnlFakxCUU5yOGxCeVIvYTM0ZkFtMGY5b2ZiaUd4?=
 =?utf-8?B?TSt1WEpzSy81ZVlBYlljUS9FR0toYmRMZEsyU3FiOHdzZ0JsUWI1MXVXV1Nj?=
 =?utf-8?B?NmRlMFdLamhMdWdNaEd6UnBWNEVYQlhHbDFvNTQrb0NrS0x1VUROTU9LS1R4?=
 =?utf-8?B?WVQreDNsSUJhUkgxa21PMVU3dVZhVjBWN3IyaFlOQlZ6bWdaekNPQVpNMld4?=
 =?utf-8?B?dkNybGJGdHpwT3JQRWdJbWJQRnZEOFIxdjFhblQySCtvcS82YVloZXU4bFB4?=
 =?utf-8?B?NnBrcXdGS3NqR20yRHN5by9Pb1loSWJ3SlFqRlZlaTRhWFZvSWZPell3MmNT?=
 =?utf-8?B?UWpYWE9ucVpZeUwrMGVNWjhaTklyRDNVUFBGTXRkTWdRT2dRZi9CY3gzSGRW?=
 =?utf-8?B?SzhoUXVaQ2doam9scGNIbDZSWGtmQWZhazRHMGRQeEYxNFRUbXhDRGFaWTFX?=
 =?utf-8?B?RkVDM1o5b1YxZ2VDYWtHREo4dlVvdHJjQ3cxbU9pWXN0cjEwMlVNQnJKcy9q?=
 =?utf-8?B?NGs4djlEbElwRjBPQWhFb3JEMzlFN1BkUkxITE1hOGJKZFdpdUhNUzNHNkpt?=
 =?utf-8?B?T01IOHNWZ1E3RXhRYXhuV0MwcDV0TDNYeEE1Q1RxeU1JaUZDSW9nOUduTzBH?=
 =?utf-8?B?SWZlN05iSlhSS1lyd1Q2Myt2SkhZcTE0cmpTdThDdkduTnVQUVFKU2NzVkFr?=
 =?utf-8?B?cDczcGJFMk1tZHY0SFd4c2ltSEl0dENwL290KzVEU1htcUVIdXdjZ3IrZGdO?=
 =?utf-8?B?NU1Sd1YxYlk0VGc3YVhXcG5QYm56dEFoZFNaS1RZZDNIS2dkVWozYnVHamxu?=
 =?utf-8?B?ZERkSGp4cjJrKzE2czRickZOVEorVHhwTnBQcGlsODF1V0l6ZzlnVnJQendD?=
 =?utf-8?B?dG1xeitucEFGQ0dwdlZudHFCemUzRGpTVTVBLzRQbFhOWHdFSTVBYklYVDBn?=
 =?utf-8?B?eWRsZFJaTTQ0MUhFTHJkaDUxRmpDWmFsbEVQWmxWU3R5TURQNEkxV0VMbEtB?=
 =?utf-8?B?QW0rdGZaZEdCYWdqQ0xYOGpURWtBcC83b2toNEp6R01PTHNqYTJXRFhHM05p?=
 =?utf-8?B?VCtOVWhXTllVdHRzSkZWTDNGWWsxQ3ZaU1RQYnFZM0F1RE81U0xWWjZnK3hT?=
 =?utf-8?B?ZWxkT0ZYSE5CR09RTGIzT0ZNV2tUalZDS0M1VytqNEJnTU53YUJSUjBOWUt2?=
 =?utf-8?B?aFNKS09WVXBjcmxzTkNCeFNwdWJGMVB0NmdSTGM4d3A3c2NnZWw5NG13K3Ex?=
 =?utf-8?B?bjRDSXRtcDFIRmVPMWdCYmJwV3VQdVYwdS8rbUdZMzdOWlVWVWdybkRtVTl4?=
 =?utf-8?B?WmZhSXRxK1pVTnlkZXp4MDdwRkFJTG5UZXFrNFBDcE5WWUcxL1Z4QmFUV0Uy?=
 =?utf-8?B?TnJpVHFOWkRVdThFZ1Fzc1lhRmZZYlNEbUdZQ05CbzFtcExicmxRUTl3a0JK?=
 =?utf-8?B?RjJSUXVQc1NSVUlGSWF4S0JKK0lNZ2NaZXYybFBrc1p2S2pTTTZkeHZha1la?=
 =?utf-8?Q?mHgOhgiBPmgT63faEMYq6E4nkj2/OssOvG0BqcA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c4a3501-11a5-4e0d-d2d1-08d9781f8176
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:04:56.2934
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EuAqGZC81YIGSNncTzgoC3wMFhEGrdY4uit9h4z5VybWLv/knuYWhZlsnbYW/izDp6DpXi3quFcfXBEZQsmsAw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2636
X-OriginatorOrg: citrix.com

If the new gfn matches the previous one (ie: gpfn == old_gpfn)
xenmem_add_to_physmap_one will issue a duplicated call to
guest_physmap_remove_page with the same guest frame number, because
the get_gpfn_from_mfn call has been moved by commit f8582da041 to be
performed before the original page is removed. This leads to the
second guest_physmap_remove_page failing, which was not the case
before commit f8582da041.

Add a shortcut to skip modifying the p2m if the mapping is already as
requested.

Fixes: f8582da041 ('x86/mm: pull a sanity check earlier in xenmem_add_to_physmap_one()')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Short circuit and skip modifying the p2m.
---
 xen/arch/x86/mm/p2m.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 674a6f4fe9..bcdc5c7014 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2799,6 +2799,13 @@ int xenmem_add_to_physmap_one(
         goto put_all;
     }
 
+    if ( gfn_eq(_gfn(old_gpfn), gpfn) )
+    {
+        /* Nothing to do, mapping is already as requested. */
+        ASSERT(mfn_eq(prev_mfn, mfn));
+        goto put_all;
+    }
+
     /* Remove previously mapped page if it was present. */
     if ( p2mt == p2m_mmio_direct )
         rc = -EPERM;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:10:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:10:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.186978.336093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPzo-0004y6-LK; Wed, 15 Sep 2021 08:10:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 186978.336093; Wed, 15 Sep 2021 08:10:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQPzo-0004xy-Fw; Wed, 15 Sep 2021 08:10:20 +0000
Received: by outflank-mailman (input) for mailman id 186978;
 Tue, 14 Sep 2021 17:38:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Bbdn=OE=quicinc.com=quic_tsoni@srs-us1.protection.inumbo.net>)
 id 1mQCNm-0003o9-Hn
 for xen-devel@lists.xen.org; Tue, 14 Sep 2021 17:38:10 +0000
Received: from alexa-out-sd-02.qualcomm.com (unknown [199.106.114.39])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 820cb494-aa54-47ca-a38b-356928eb3106;
 Tue, 14 Sep 2021 17:38:09 +0000 (UTC)
Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145])
 by alexa-out-sd-02.qualcomm.com with ESMTP; 14 Sep 2021 10:38:06 -0700
Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231])
 by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 14 Sep 2021 10:38:05 -0700
Received: from [10.110.72.52] (10.80.80.8) by nasanex01a.na.qualcomm.com
 (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.922.7; Tue, 14 Sep 2021
 10:38:04 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 820cb494-aa54-47ca-a38b-356928eb3106
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim;
  t=1631641089; x=1663177089;
  h=subject:to:cc:references:from:message-id:date:
   mime-version:in-reply-to:content-transfer-encoding;
  bh=LxUYjTCuKp+eIbODD4m5XJrEET+FLPtks/iYvJ1ETf0=;
  b=TbQxYx6+6Q1f7FbnXYXHDQR4s+WGQx1N6tN1NkG9hZgl7TJN9ITUcgZN
   PevcAK6+It6y8j+mbljpfe+R4psE1qA92AiUieWvlqiUJwVKC5a/0pOGv
   5vlZoTnH+7nPxDmOc5Skm5+DyTN9+6xojdAYj4lq4Ro47OkNlwcI7hoFN
   M=;
X-QCInternal: smtphost
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
To: Stefano Stabellini <stefano.stabellini@xilinx.com>, AKASHI Takahiro
	<takahiro.akashi@linaro.org>
CC: <paul@xen.org>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	<virtio-dev@lists.oasis-open.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jan Kiszka <jan.kiszka@siemens.com>, Arnd Bergmann
	<arnd.bergmann@linaro.org>, <jgross@suse.com>, <julien@xen.org>, "Carl van
 Schaik" <cvanscha@qti.qualcomm.com>, <Bertrand.Marquis@arm.com>, "Stefan
 Hajnoczi" <stefanha@redhat.com>, <Artem_Mygaiev@epam.com>,
	<xen-devel@lists.xen.org>, <olekstysh@gmail.com>,
	<Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
 <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
From: Trilok Soni <quic_tsoni@quicinc.com>
Message-ID: <e9df2014-b691-1ee4-7fcd-03912d0aed59@quicinc.com>
Date: Tue, 14 Sep 2021 10:38:04 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.80.80.8]
X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To
 nasanex01a.na.qualcomm.com (10.52.223.231)


Hello,

On 9/13/2021 4:51 PM, Stefano Stabellini via Stratos-dev wrote:
> On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
>>> the second is how many context switches are involved in a transaction.
>>> Of course with all things there is a trade off. Things involving the
>>> very tightest latency would probably opt for a bare metal backend which
>>> I think would imply hypervisor knowledge in the backend binary.
>>
>> In configuration phase of virtio device, the latency won't be a big matter.
>> In device operations (i.e. read/write to block devices), if we can
>> resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue is
>> how efficiently we can deliver notification to the opposite side. Right?
>> And this is a very common problem whatever approach we would take.
>>
>> Anyhow, if we do care the latency in my approach, most of virtio-proxy-
>> related code can be re-implemented just as a stub (or shim?) library
>> since the protocols are defined as RPCs.
>> In this case, however, we would lose the benefit of providing "single binary"
>> BE.
>> (I know this is is an arguable requirement, though.)
> 
> In my experience, latency, performance, and security are far more
> important than providing a single binary.
> 
> In my opinion, we should optimize for the best performance and security,
> then be practical on the topic of hypervisor agnosticism. For instance,
> a shared source with a small hypervisor-specific component, with one
> implementation of the small component for each hypervisor, would provide
> a good enough hypervisor abstraction. It is good to be hypervisor
> agnostic, but I wouldn't go extra lengths to have a single binary. I
> cannot picture a case where a BE binary needs to be moved between
> different hypervisors and a recompilation is impossible (BE, not FE).
> Instead, I can definitely imagine detailed requirements on IRQ latency
> having to be lower than 10us or bandwidth higher than 500 MB/sec.
> 
> Instead of virtio-proxy, my suggestion is to work together on a common
> project and common source with others interested in the same problem.
> 
> I would pick something like kvmtool as a basis. It doesn't have to be
> kvmtools, and kvmtools specifically is GPL-licensed, which is
> unfortunate because it would help if the license was BSD-style for ease
> of integration with Zephyr and other RTOSes.
> 
> As long as the project is open to working together on multiple
> hypervisors and deployment models then it is fine. For instance, the
> shared source could be based on OpenAMP kvmtool [1] (the original
> kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
> kvmtool was created to add support for hypervisor-less virtio but they
> are very open to hypervisors too. It could be a good place to add a Xen
> implementation, a KVM fatqueue implementation, a Jailhouse
> implementation, etc. -- work together toward the common goal of a single
> BE source (not binary) supporting multiple different deployment models.

I have my reservations on using "kvmtool" to do any development here. 
"kvmtool" can't be used on the products and it is just a tool for the 
developers.

The benefit of the solving problem w/ rust-vmm is that some of the 
crates from this project can be utilized for the real product. Alex has 
mentioned that "rust-vmm" today has some KVM specific bits but the 
rust-vmm community is already discussing to remove or re-org them in 
such a way that other Hypervisors can fit in.

Microsoft has Hyper-V implementation w/ cloud-hypervisor which uses some 
of the rust-vmm components as well and they had shown interest to add 
the Hyper-V support in the "rust-vmm" project as well. I don't know the 
current progress but they had proven it it "cloud-hypervisor" project.

"rust-vmm" project's license will work as well for most of the project 
developments and I see that "CrosVM" is shipping in the products as well.


---Trilok Soni



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:17:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:17:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187298.336105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQ65-0005dn-9p; Wed, 15 Sep 2021 08:16:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187298.336105; Wed, 15 Sep 2021 08:16:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQ65-0005dg-6o; Wed, 15 Sep 2021 08:16:49 +0000
Received: by outflank-mailman (input) for mailman id 187298;
 Wed, 15 Sep 2021 08:16:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQ63-0005da-OH
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:16:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5c6fb6b5-e8b3-4ee3-ac00-99e385e32d58;
 Wed, 15 Sep 2021 08:16:46 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-Lkv8nHc_MQS5zaBK1l5jag-1; Wed, 15 Sep 2021 10:16:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7087.eurprd04.prod.outlook.com (2603:10a6:800:12a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 15 Sep
 2021 08:16:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 08:16:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0027.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 08:16:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c6fb6b5-e8b3-4ee3-ac00-99e385e32d58
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631693805;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xtYTEz4egySuYKDee2kzOyxpiadVw/pEzzeIm62TtmQ=;
	b=CZUuHhFejkQx8cbN4fdNRgCiF1O8B4reJtDLhoezcePaC9qot+Sc1TgYfsYMaBE0btoYeN
	r7VyE1NdkytBgTE3ubIIIg/Uq+qTatH/zogv6IFj2t1siYp2eLm6i4TugzQ+/I7JBcc6P1
	h+cp5rovwyww4tzgkl9oSgwa9vTPzXg=
X-MC-Unique: Lkv8nHc_MQS5zaBK1l5jag-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NneLDN7yQzu0YqIqW2FH7NtgM3BIZdAYCuVH8uc2XTl7cuOIVvk5WtFTsA6KHbCAlYzGrkUobP8b9OD3cx2+TDjeyj8N5Ug3Vn4gZHu60rhC6BO4X0MQCyPCfcQyk+9mXDb/UTXxGckjZsOyMQWNuXz9tMCePUOi7RiwyVcTIIYFsXsxyub6Mp7bZaJ7FjEUOPAOf/EQcp3qnSA/aoGaXi541fV0H6ens77vkj24hap8r0HogpiBNK9fPRsrXIFqUiTVANSq/6VHt+axmp0s+Byn+1TkglVdmx5TTqkyxlkhAWs/g3/mrc/UGnIqepVSTMbKMSSefuODgKdFGo6izQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xtYTEz4egySuYKDee2kzOyxpiadVw/pEzzeIm62TtmQ=;
 b=nD8sVxD1d9jNVf5LHVFJTvf8jS7NVBWXGRAj5eixHl3PllU01B9SrG94+lUys7aWFQbO9hgosqp7erwdjrD5DDy+Hpu7e0rW2cCoEFJFVKPAIG1l1xRDWiDy+/e/XtNYzcYJVok42HdCDL9M3nJcxe1PjJbSfhsf+3S2PEeuJD6FzFgqqy8MWgfn2bMbyRaFj8U1OuOtdAOKCNXkt7avAT/6qbbelxZ4p+K2ujwC0+m+1rEM3QLmALkqyaa/njiALu2MKnlX391XrPeoSUWHRMIhFrE3eFRuS9Ng9FVpShcITEtZghaaGH4q1GuvlTb+CLrIbwb8kRu7cYG8bKuf2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] x86/p2m: fix xenmem_add_to_physmap_one double page
 removal
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210915080342.21346-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1d70ff4d-bef2-1865-27c3-27d17c15d17f@suse.com>
Date: Wed, 15 Sep 2021 10:16:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210915080342.21346-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P195CA0027.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dd279e79-082f-4e72-a10c-08d97821270e
X-MS-TrafficTypeDiagnostic: VI1PR04MB7087:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7087A99ACBBCAEC78CF53DAAB3DB9@VI1PR04MB7087.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fZoP9+2IIk1sam/J0GuiLmskhu3K4fkE3AIUULbE4VQnck6rg/aZW1KWSq0oqjbuUdfQYkebHoTCI0ffMy+Vzcm38AxSmmoo23H1mWj6BL5zr/es8zenXmdgp4wro6WOA1/903a0ILaXc5BYsM+CfcVtxEn4dUQsmBt+ChlCiGCWN0P++lbwPWe24RCKgmLeUkkJw6DumAvbFm3Et2tDcvyxSFtlp+Kzpo1f1Q9a+RKH7faBlbuNMowj8ILFbWh2F50GBYMtnZGzWnQg6NbTnHMaOP4lANCHkWziLNvtR8b9w4uYH50D4Xj1YUR/NHlD0Pyc/mXIJR8uCmexAbuYvOXAL/h5jMHibzMIOytXAXh73pAFHHKiIvP/FUM/tcO5P+aDToCZfMHrWn8gVp6h5B1murCxh0BC3TXPOJ9L9JwErQMD8ZXdOJQKxmyNMu0kR8VCxodKNUfD1eGq+pGPorUjG05ATioRurApUq70iIOvD3Kqb3qta0tZwT12GnZsCEgJqoCffFRnSfloXMWSEn7cqqZmJppxAqQy1meznbHiAsil3KWekZy/SyMHM6aEVoHKDzdQV56y5rGUtM5l20b4anrK/9VJwr6vm0Pwfyrob0SWMzg/F++q5hCTmqdfrChf9cwyptDrIA8kX/ztQbTbPw76pPq8ZHyyUeo/eFP6oiUhuLGHwuPsHsKNBHBRLCaEie999V3YOEplXnY5KrXMe0O0xvF4K6YJnkVU+uIKGQpP7wg96Esf6q9Ym+jE
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(366004)(346002)(376002)(186003)(6486002)(36756003)(5660300002)(31686004)(8676002)(2616005)(86362001)(4326008)(16576012)(53546011)(316002)(66476007)(66556008)(2906002)(6916009)(38100700002)(8936002)(31696002)(54906003)(83380400001)(478600001)(956004)(26005)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z0RNNWlpUTUyMFU2N2N5eFJaaUljS2E3a2NVWENUK1JJcEdmUFNvdDlkQ00x?=
 =?utf-8?B?emE3WUllcGdmNkg3MFVPWWg4SjBja2h2cE96d3lNL25lUEZEdTZROXVyRzBW?=
 =?utf-8?B?RXAxQlU4YnREZzI4eVBvSmxZK3NJWDJXUC8yVEJ1ZE9LbGlWd211c2JpZzBU?=
 =?utf-8?B?T3BrRENEQTEyNDBZSlhmampvMGg0akhISC9tVG1pa0VGTGN0ZG1YTHVDZ0J1?=
 =?utf-8?B?cmJRN2k1MlowVVZ1N2YycFlMUVRQVThxdXJwQnR2Rm5wSmxONm9vZGMrMEs0?=
 =?utf-8?B?NjM1WW1yVXlBMlFLWkdkSXZ5TkpqclNMeFprMkc0MjYwYkwxSVpIRkpoeko0?=
 =?utf-8?B?YlF2VTlBRmdNVVJVWjlCak1KSVdJbW43QmNkOXlPVTVnMHFqMjg2c1ExZXlJ?=
 =?utf-8?B?dHRndkZDT0h3aCsyM0tEQXZTTU1lSitHUXhVcXc2YmdwOWQ5QWd6NG5Yb05P?=
 =?utf-8?B?UmxETUt3emJuSVZZNWVZVExYZlZpSW9SK0Z1K1ZLd3dUbHFjLzlVdm9xRktX?=
 =?utf-8?B?bkZDOXVZS0dJc1RHZ3dwR2dxNWVzWFk5ZHdnY1dMa21vNVJkZURmYklTbTI4?=
 =?utf-8?B?ZkZFUERFdC9NaEtCSDhybWp0Uk5QS3VZdnhrNThnY2xKNHVQZ2E3OFcwb3RZ?=
 =?utf-8?B?bnFyVjJLQTlFN3poWFR3dzA2a1VZS1hJam93Wk8wV2JTTUFvNndUeXMxTmw1?=
 =?utf-8?B?QS82L2NIWUVna0NUeE9jc2psK3crNEs3UDA2UE1SblkydDd3WklHMlZ1Zm9x?=
 =?utf-8?B?N2owVDl3bDJ4Y21EbE41L25kMUcyR2J4QlhJS2JRaGJDRFdBOFN5TzJGdjE4?=
 =?utf-8?B?c1RBeEw0ekhUUEpzUlJsRmZHaE12TUh5NkN0emZOUERySU1xREZudERlOE41?=
 =?utf-8?B?WS9GQ0Z3Z3E2b0xDWUVxMHJJbDN5MklleE50Tm9VVXlOM2s3alpCbExiR0tN?=
 =?utf-8?B?S3lKS1FJYTVCOWNLZmh2NHAxV2kxNjdwWnh4YnI1ajROa25XTGJmWnVtc3lw?=
 =?utf-8?B?V1M4K0tZb3RtNzdNS25tWFlIbk9SZGE3YkRmZDdxUFBhZU5iK21xS3UzNHl0?=
 =?utf-8?B?ZzYvb0NHQUJMWklkZzlwR21TV2t4TmxyQzFEbk1FN21JNmswblZ2THRFWEFs?=
 =?utf-8?B?V1VXaDZodmFBZUg2WHlMalhyRlp2Zmk3ekIzYUFuS1RBM0JiaTh3dFNLeUdW?=
 =?utf-8?B?Um1iY0o0bEp5eDNzSEF1dnJZZ3o5RGFGKzMwTjgzZEFjSllka3lJcDRxdG5H?=
 =?utf-8?B?U3E5bXhGSmtzeHdvaWVsODRoODJLUnVtYXp4emd6Y01oMjdwbkZZYTdRd3Fv?=
 =?utf-8?B?NDZhYjNVM0hWY3Q0WFl2c01UdFJZdVhMWjZmNW1wczlIeUprMVFpMkVIU3FU?=
 =?utf-8?B?QWFJRWVSR2trZDNmNjc0ZUVTaWhVUm90Wk1pR3BsZDU5MmxDZ0hDSnVoWWx1?=
 =?utf-8?B?cmp6dVpYQXpYb0tQUE1CcVRDaHZMVkZ5NWgrbUN1cVViVStoaUFvNzZEVTBF?=
 =?utf-8?B?RDdTMWU2b0lBL2tlTjVqOTB4TnBDd09TSnZDRUlJYzhLMVBLalJwRjR4K28r?=
 =?utf-8?B?VTIzTERhSkVOTFRMbVQySXRHUGExYk4zbWprOENUTEw5aE5IRzRJT2NncTBC?=
 =?utf-8?B?dC8xZFJBUmRGaTZneUU1RUlYRTFxaXVQM25yY2ZFN0pJeFBiYjJMY3c3enAr?=
 =?utf-8?B?VjVzMXFmOGdlYWI0WFY5cUpxcHJkYlRwYXJXNVNDQTZoZm5UUzVnc0g5Z3ZU?=
 =?utf-8?Q?qiwpHcruAunhvYWZ7d2bmhSiWRCppU5s9NYQKQW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dd279e79-082f-4e72-a10c-08d97821270e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:16:43.5009
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mUWtoqU5EyzWEL/iixc/bG94fNcdrflUau2fU/9IrgofYG6wW00LDsL/wSfhXrwQoE2srQQQZkypr43kM1cseQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7087

On 15.09.2021 10:03, Roger Pau Monne wrote:
> If the new gfn matches the previous one (ie: gpfn == old_gpfn)
> xenmem_add_to_physmap_one will issue a duplicated call to
> guest_physmap_remove_page with the same guest frame number, because
> the get_gpfn_from_mfn call has been moved by commit f8582da041 to be
> performed before the original page is removed. This leads to the
> second guest_physmap_remove_page failing, which was not the case
> before commit f8582da041.
> 
> Add a shortcut to skip modifying the p2m if the mapping is already as
> requested.

I've meanwhile had further thoughts here - not sure in how far they
affect what to do (including whether to go back to v1, with the
description's 1st paragraph adjusted as per above):

> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -2799,6 +2799,13 @@ int xenmem_add_to_physmap_one(
>          goto put_all;
>      }
>  
> +    if ( gfn_eq(_gfn(old_gpfn), gpfn) )
> +    {
> +        /* Nothing to do, mapping is already as requested. */
> +        ASSERT(mfn_eq(prev_mfn, mfn));
> +        goto put_all;
> +    }

The mapping may not be "already as requested" because of p2m type or
p2m access. Thoughts? (At the very least the new check would likely
want to move after the p2m_mmio_direct one.)

I've also meanwhile realized that it was a different form of short
circuiting that I had been thinking about before: XENMAPSPACE_gmfn's
idx == gpfn.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:21:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:21:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187307.336115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQAn-0007CU-0t; Wed, 15 Sep 2021 08:21:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187307.336115; Wed, 15 Sep 2021 08:21:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQAm-0007CN-UC; Wed, 15 Sep 2021 08:21:40 +0000
Received: by outflank-mailman (input) for mailman id 187307;
 Wed, 15 Sep 2021 08:21:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wWxD=OF=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQQAm-0007CH-2V
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:21:40 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 089bfe83-7ffe-4b35-afb4-701ecab992c0;
 Wed, 15 Sep 2021 08:21:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 089bfe83-7ffe-4b35-afb4-701ecab992c0
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631694099;
  h=from:to:cc:subject:date:message-id:
   content-transfer-encoding:mime-version;
  bh=RH7xwGIdol7s79gGtxpS5ozKt+VBklOPd/I/V+y5fd0=;
  b=X6irJydKbZULHQaETjUTVaHzs/OBWiBqrjrhTM8CznoL9RbS0RQt28qT
   qTG5DFwSWXOUlE6y2GLQqgK/89HWBRr0vHA0/GQ+sCHeJcbEAsnT4oJAS
   5W2z00SQ9uAK/u3lX0J959O4sxxc7WDYCl3UWpvfisdRRxj6TIeiwRLxo
   8=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: kSRl9OQFdxrrQfG2ta0ZSHY5zughj5udQSqXxHFWAF6Um5TH9nmjKFj2lP+ex5Ao7V/lha02V/
 xstB7ou/fJ55K1UI5eldUkrRhRqmpNaJWO2yke/vQWxb8PnjCBirbVfEACAfkHFdeN8dRQWnzx
 CDz9nW7QYKmIxzaoh5D3iNRUmhQM7c+8K7aEXh64xnss6P+T/gu96Gb60RQPgrzQ5kc/e2//iB
 amI2cVj5F5k2Q66SQdgez1vsqcTVkmfDG3hjKsav2tH2SaieJaMfPigdcCypRz0GakvSCLQR/v
 WDV/yJs4YVoNrT0ZPZdBnznG
X-SBRS: 5.1
X-MesageID: 52774695
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:9HxskqgyBTYcm42gslW486NtX161XhcKZh0ujC45NGQN5FlHY01je
 htvXz/TOfuOMGChKN8na4i+9RkPuMODzoIyTAA6/yFkH34b9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8x3IDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /11n82xSFd5OpHru84ZXDsHMz5lA/BZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9t3J8RQaiPO
 qL1bxJvL0THYDlfPmtQI7A5xdexlETFXS1h/Qf9Sa0fvDGIkV0ZPKLWGMXRUsyHQ4NShEnwj
 mDM8nn9AxoaHMeC0jfD+XWp7sffkCW+VI8MGbmQ8v9xnEbV1mEVEAcRV1awvb++kEHWZj5EA
 xVKoGx09/F0rRH1CImmN/GlnJKaliY2CttRQ70+1Cau4K7bwVm5GXM6EyEUPbTKq/QKqSwWO
 k6hxo2yXGE+7+LLEBpx5Z/P8mjjYnF9wXsqIHZeFFpbuYGLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3m7ESnQkSGcICuTrsoCNtVgRiG2NPdXABb3nARBodtjxor6p5
 iRspiRmxLpSZaxhbQTUKAn3IF1M2xpjGGeN6bKIN8J6nwlBBlb5JdwAiN2ADB4xb67ohgMFk
 GeM4FgMtfe/zVOBbLNtYpLZNijZ5fG7TrzYugTvRoMWOPBZLVbflAk3PBL49z29wSAEzPBkU
 b/GIJnEMJrvIfk+pNZAb7xGiuFDK+FX7T67eK0XODz8jePFNSLKEOxcWLZMB8hghJ65TMzu2
 483H+OByglFUf24ZS/S8IUJKksNI2R9Dpfzw/G7vMbaSua/MG1+WfLX3507fIlpw/ZcmuvSp
 ynvUU5E0lvvw3bALFzSOHxkbbruW7d5rG46YnNwbQr5hSB7bNb99robers2YaIjqL5pw8lrQ
 qRXYM6HGPlOFGjKomxPcZnnoYV+Xx23ngbSbTG9aT0ycsc4FQzE89PpZCX18ywKAnblvMcyu
 eT4hAjaXYACV0JpC8OPMKCjyFa4vH48nuNuXhSXfokPKRu0qIUzcn7/lP46Jc0IOC7v/DrC2
 lbEGwocqMnMv5QxrIvDi5ebotr7COB5BEdbQTXWtO7kKSnA82O/6oZcS+LULyvFXWb59aj+N
 +VYy/bwbK8OkFpQ6tcuFr9qyeQ15sf1pq8cxQNhRS2ZY1OuA7JmA3+HwcgQ6fEdmu4H4VO7C
 hCV591XGbSVI8e0QlceKT0sYvmHyfxJyCLZ6u44IRmi6SJ6lFZdvZ6+4/VYZPRhEYZI
IronPort-HdrOrdr: A9a23:Jg93S6p3Yk0KIlJrURikTQMaV5vOL9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBfhHPlOkPMs1NaZLXLbUQ6TQr2KgrGSpQEIdxeOlNK1tp
 0QDJSWaueAdGSS5PySiGLTc6dCsae6GciT9JvjJhxWPGZXgs9bnmJE4lHxKDwKeOAKP+tzKL
 Osou584xawc3Ueacq2QlEDQuj4vtXO0LbrewQPCRIL4BSHyWrA0s+2LzGomjMlFx9fy7Yr9m
 bI1yT/+6WYqvm+jjvRzXXa4Zh6kMbojvFDGMuPoM4ILSiEsHfjWK1RH5m5+BwlquCm71gn1P
 HKvhcbJsx2r0jce2mkyCGdrTXI4XIL0TvP2FWYiXzsrYjSXzQhEfdMgopfb1/w91cglMsU6t
 ME40up875sST/QliX04NbFEztwkFCvnHYkmekPy1RCTIolbqNLp4B3xjIbLH45JlO71GkbKp
 gsMCmFj8wmM29yLkqp8lWH+ebcH0jaRXy9Mw0/Us/86UkgoJk29Tpd+CSz9k1whq7VcKM0l9
 gsBJ4Y4I2mfvVmHJ6VO91xMvdfcla9OS4kEFjiaGgPR5t3c04klfbMkcAIDaeRCds18Kc=
X-IronPort-AV: E=Sophos;i="5.85,294,1624334400"; 
   d="scan'208";a="52774695"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xsfg29d6nQijV227xIrtff5jRdj9mQ+u8w9kiMivZUVyHVzwWmP/oExKxxMBrZGbvv9cJLvHVTzsCSTZOUhRaXKiqszuaBQS3JuOnknaYGMwK5EZHlhkKBPrGytW8f1QcO6wjHUgkQAwiaNuJ62sJMWjcFAYwi2uE5nejZHjiZ0e+P2I1tE+WOF4B2Sld8qy0LfptwtCPAn6/uFlc8VihdRwlT6cuBD1f877cnMeF+rDabz4BKm11l/qMYQ6QsWkaTiQ4jmGOASxxK4DIk/T7UaJGnJZEU1t/ye3P+mTeGkbsyp2a+zZiconpj9JLrjf7cQqSsbq8YJa966SQotgfA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=l1g5PWVzKl8h5wZ1rimfreosePf43AUvwJvgZD4ri1w=;
 b=TA993vvhtn1dq2PNMBYQwisuzLuBcDGh7OJMsGJVUZ5YC+JbczO11GysNXGpYh/HaOMf0Z5D7j/c8nnbSMuvpByZxkUiSkqc+JoY/rgwNxRVZqKQZ9iKpqX+rA50xeqEpsM7lDUjOT5pV0/1DK5SIqU2u9RzWWaDF1RB+6O6U4lU1FF3lkz89XhCThreUFnvwzgabCszhxNJ2xB5q2MWI7GdDztpjnsbULS6K0vsCgwH35jaZkWHLKKcOaYPorsHPsQCvPL4iYWqEiafXJOvJXRjsbm8tFFtjYAmltRstyA+uAzB378Qo8JrU/Mrx9mnwSjaFGq8MTO5Z5TCwqb39w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=l1g5PWVzKl8h5wZ1rimfreosePf43AUvwJvgZD4ri1w=;
 b=DE1L9J6RJKSe6VTzK6Cu9YMa1MyyWJvkD3omZhB3cb4OK2Z/aN8/E4lFmaEEus4z6TVEcdd2PBZv6rC4dnDi1cXwtf4j9P9PUZo3M6YHhBxrBG6oCGXNOmmMoZkCMeHrxNfi011n9On7p8YQWIRwds9tBRVP48ESvN3Jv+fHeYI=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH] x86: quote section names when defining them in linker script
Date: Wed, 15 Sep 2021 10:21:23 +0200
Message-ID: <20210915082123.21795-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0447.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:e::27) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e36cd702-6cc6-4a32-4ff5-08d97821d1bf
X-MS-TrafficTypeDiagnostic: DM6PR03MB4539:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB453970FB39B5E62758338C7D8FDB9@DM6PR03MB4539.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Y6SL6WtJU+VvNMN+5gIJQeK95rkFfg1vOc8LX5+J1y8oXtkiPnzpVC4NhH5KWxTYtKE0KTq4cY5Qm+mj2aNpdAs/vO2Ag/rLJpGUfPokFAWincZdHV1WLncLr8o/fQYSl/F40MUQJRly1h2aq6cg0ZSMlOwP2bNYPypVgvaZMQQe0J6nkK5LFw9nvQbtNFm7FwMXx/XeD1Ngsuf7WXpu3msGhbos68vB8XZRaHW9c/04vyFy/28sN+PSJnMoia51vrZz1MAhaCb/oDaz60fpqemiLkNBHIq9aYtGA0EEWfvmzMPWo8O6g33ARZrB+bPAKMJH9823yrv3l3sSjgA6TgZxMQKFfvdKhZIMoKeCLh6Yzzns28l9Tvt841MM0U6zuFWz/60o9bJVDqA7DxMNc1hxmuDsGCg0OeLzKM/3phzNPVb2DXrNhEfJ/YwJ2m4IzkhSO7EpbYsK19S4sw8MwYjfaZXwy/vZefj7LyupfJpF5/enRXApKg6BgoPk79WtS9oW5tGXBQtylMl2X/Cyv28xldLWVRrfn5rZC+a5DC9oywicycdC5XLvB4I7dzx41r9YwfyYbxOz6Ya3i+PvfpruPKxA1Mxa/mU8Ap5PyuQgMB89yVEUjkt3Jhs/BJ7MguIKz78ufn1vMzI/uabzuKq3dxVSw4bx2XntihVnC3+jo0ygVsLki3kuK+liv0uZdzY8Tr/Qmkj8+/ih3MkHeuX7DbPXhG9gQ9XzYmI3FLE=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(136003)(366004)(376002)(396003)(83380400001)(2616005)(2906002)(478600001)(956004)(316002)(1076003)(54906003)(36756003)(8676002)(6486002)(4326008)(66946007)(38100700002)(6666004)(6496006)(186003)(66556008)(66476007)(26005)(5660300002)(966005)(6916009)(86362001)(8936002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RzExb241YWRzeTU1Q3JYbW9nUXB0WmRMUUhjTU5BNy9tU2VxU0dkZlF6b0ll?=
 =?utf-8?B?YWJOcFFUa1dZK083cm1LdHNEMDgzeFA1VFNRMW5IaURDOEtLOU1sMXFhQnFy?=
 =?utf-8?B?QmJkN1M4KytlVFp2OHRIS0hONjR2YkwyeDd2Vk1Xc2pLTUNRdmFGcUZHWmYw?=
 =?utf-8?B?QmplT1J2MWlDUDdlaTk0MGVOYVFiWVNEVjBCOGNTbWdsZHVWODg2cEdUc0Rn?=
 =?utf-8?B?aVpQSEtpK3lvV1U5Vm9RblVKUTI5ZjFhb1RrWEt5d0p6WnhLTnRJajdtYmFD?=
 =?utf-8?B?QmVTMUhYQnRmaW5ZcHhpVjJlQVRTcWpMQWlaYzh5SjJBSEFONXNDZ1pPS0Ur?=
 =?utf-8?B?ZFJjUGVXV1JEUXRsS2xrRHNaaks3d21HaG8zY1lGOCtOS2Jvb1laTDc4YUJp?=
 =?utf-8?B?UFdDQ1VzcFVSZDViWWVGQkJtc0w0WWNRZVBzZTBzbDNrYlVKYkdhTkpNcWVN?=
 =?utf-8?B?UjVKWEcwR0N1VzRsaGhkdzhLdklTUWs5VzBPRzZuSmttK0g0d2FZelNkeTdv?=
 =?utf-8?B?a0kwbDMxMWEvTzB0cnc0R3JIZS93SzlIY0lqTDJlSVh0WE1sbHFDYnhIR0Vz?=
 =?utf-8?B?UURiYndoYThJM3pNdUU5YUNtSVZXUzRwS2UzUllBSDhsQ0lUZWdtWFpTMlhU?=
 =?utf-8?B?ZWh1NmlVcEcvRkdDV2lINXdXTWgzd3RNQXpqY1IwdjQxczlKNzd0cHZyeEYz?=
 =?utf-8?B?d2pUTHJhaFd3UC81Ry9GV0ppVHVvYUtodFpjTjU2b085NXVML2thdFN2a014?=
 =?utf-8?B?Mk1TbjZRMXZPVHE4R0lsMTRsYmlXQ0ltODFTNHIwbGJuL0E0ZHpkSER5VWVC?=
 =?utf-8?B?NEJDNFlYeXdFMUFLUjVYOXRIWE5PTXBmR2l4SjZHNCtaZk5mS0lLaEdNbmVp?=
 =?utf-8?B?VjdYeE5wczA3NEpZSHNMSjVLWmtDTWd2Tjh1YnRid3F4UytlbTlCNUV5emoz?=
 =?utf-8?B?NTBaM1lQN0pPZklJck1MT3RZcnNKMUt3TVJtQzdVMHQ0RStERjhEZmtuVm0x?=
 =?utf-8?B?U2l6L3lsUWJJWHdScVo2VUNxZENDYWRxZDdPWlVJbmdVTlhlbnVFRkVkN1pr?=
 =?utf-8?B?aFRsUUUwbmV5VDhHYWRVZCtRR2NFVTEwM0VWQVcxd2U3bi82OTIyODE1bFM5?=
 =?utf-8?B?NzgzWkNBWlV0cEFpRFR1ZE4wWjhRa1d4RFVWVkF6ZXI3cHplSTVxRE11L0p3?=
 =?utf-8?B?ZUE4eERiNzRFbWF3dHRibTVDaTA2VVRvc1RTOHhQNU5GUlBuZXc0bEdEOE16?=
 =?utf-8?B?cmlMNksxQURGS2ZLZE9mU3c3ZUhCdnBtNlpycDJaL24rM0dyOGdaUW5rRndh?=
 =?utf-8?B?OW5vck1LOWtKSmJqanhIOWxnSjFUY3F3ay80S2ppOHI0OTRnb2JDb2hjaHlU?=
 =?utf-8?B?cWJtYlFOdW1OcjFCcCtVR0NXYWh3MlE2UEdjYVg2ODlLdGduZnNRclA1Nmtk?=
 =?utf-8?B?a21kZ21mSFk5WmU3dzhDWU9WTHZIOTY2dWxHR2lSU0xUQWJMOEVaT0R0dEtR?=
 =?utf-8?B?YmZVYllXUDllNkVwcS9iM0s0VjV2RHJ5eGs3c2VpczZQalZjZWU2ekhYVzBY?=
 =?utf-8?B?bWJ6Wll2OGZpTWcrMldtZ2xlUUNpTEdGa0RiMFRsSWxtdEpkcGtWbTR2SklB?=
 =?utf-8?B?STJzbTg4UWdkZnM3YTVoRENuTnJ5bzdwdyttN05qLyttV2d5ZmJ6R2lTb29O?=
 =?utf-8?B?ZFlnaitIVjNIZlBhcmdjOWJRQlEvZ2VBUDBqS3BuMC83QTZxUy9JdUlkTjQ3?=
 =?utf-8?Q?v4PH8j4JqMP/cUMSs2FOzxGMpftaKIgEXQIMMun?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e36cd702-6cc6-4a32-4ff5-08d97821d1bf
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:21:29.9866
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7ojFPCZLhb+HBL/RavGFyxbEVL29PLRrSEr781eYkVh7AZo0EzEYE4K6rI65jNsuTYvUWFgG5TMHn2rInfLRaQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4539
X-OriginatorOrg: citrix.com

LLVM ld seems to require section names to be quoted at both definition
and when referencing them for a match to happen, or else we get the
following errors:

ld: error: xen.lds:45: undefined section ".text"
ld: error: xen.lds:69: undefined section ".rodata"
ld: error: xen.lds:104: undefined section ".note.gnu.build-id"
[...]

The original fix for GNU ld 2.37 only quoted the section name when
referencing it in the ADDR function. Fix by also quoting the section
names when declaring them.

Fixes: 58ad654ebce7 ("x86: work around build issue with GNU ld 2.37")
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
This seems to make all versions that we test in gitlab happy:
https://gitlab.com/xen-project/people/royger/xen/-/pipelines/370610794
Plus the FreeBSD Cirrus-CI:
https://cirrus-ci.com/build/6690836768358400

I'm slightly worried that LLVM ld requires quotes to be present always
in order to match, but I can confirm such quotes are not present in
the sections names in the output binary.
---
 xen/arch/x86/xen.lds.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 955d5cf4a0..11b1da2154 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -18,7 +18,7 @@ ENTRY(efi_start)
 #else /* !EFI */
 
 #define FORMAT "elf64-x86-64"
-#define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
+#define DECL_SECTION(x) #x : AT(ADDR(#x) - __XEN_VIRT_START)
 
 ENTRY(start_pa)
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:22:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:22:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187310.336127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQB8-0007ei-A2; Wed, 15 Sep 2021 08:22:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187310.336127; Wed, 15 Sep 2021 08:22:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQB8-0007eZ-6t; Wed, 15 Sep 2021 08:22:02 +0000
Received: by outflank-mailman (input) for mailman id 187310;
 Wed, 15 Sep 2021 08:22:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQB7-0007d8-1k
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:22:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fe920a5e-15fd-11ec-b4e7-12813bfff9fa;
 Wed, 15 Sep 2021 08:21:59 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-XztqRFtDOe230osyglwgdQ-2; Wed, 15 Sep 2021 10:21:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3533.eurprd04.prod.outlook.com (2603:10a6:803:b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 08:21:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 08:21:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P251CA0025.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:21b::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 08:21:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe920a5e-15fd-11ec-b4e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631694118;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UobxhOwENG75OeE9t5Q7pyFYU1cTvsFYR6nUP5ypraQ=;
	b=mKGn4FI8lHWmq3cXsUMXWqjQO/2lYtmVgpS/VHXVoraUFxajISVhzh7XB9OhkuZwtLiBB8
	U0nGGcc3+MndW18pwwX9zO+1725HARd8TUz/jY06UUJQ7+ApQl6HMgWy/bUqezkEeKIgnu
	iBCJQQfzltbOunnm5v9XEJRdVRAUM/c=
X-MC-Unique: XztqRFtDOe230osyglwgdQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JtplVii/RcT+W/ApW5sjDYxxShFu3TAjpZiHxikgWQQjTGRndryFI0lqX6NM5HybbcH2B02q9SlvFbwMwRLvy5GxcnSWsjQ0gsfaJ5R3AhGKVoNggCfjUe57hGjTGPg13/ZFu4L9wby34Muu2KznqGJ74VM7IFk+yxx7QiNTpBm66cUtLGFiD3Lhmh/SDEqxPr/ejzNfa0j7AMUFWfwvGt0sYhDb09HNSwFSAI5Z6/dZywQ1R8WkFUdch7Kl82uioTZi6nCfu9Yd0c3C3HkylNX3kZGwA2tEI6RnBLEMtk1nqLkf4fWJvROFU1ZDI3jizVuaAjm4smCuNCaKje4kTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UobxhOwENG75OeE9t5Q7pyFYU1cTvsFYR6nUP5ypraQ=;
 b=oT48ao4h7wcOiqJhWzurR70L79Js49e4CPl+E+U982MsL3/yUrx/C/gx2MC1+Xpw7l2sZSwKY77V9UKnuWWoHXRDk9rjh7wfgdkgvW6ao27J7cU9X7OlMteTNyeqYRIK1Uk5T0L1aIhGYGvDgreB8+vJ780UOmKJSSTVDXUL9iH5TzpAGqZ1TlTnpaU6IScrJ3RAClE3mggUuPI39z9AFD3RZgbxtd34K6C86V4kJZAAwdWNkC+pome/9Zt3pQLPL7BfexocYG6Q2F7c7g2pTibYTjn7kH5jLRn8UJ08n4bloHsJTQHIo2dJItopesAsITujZSVAesbvefVBkpA5xA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 04/12] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <588b3e6d-2682-160c-468e-44ca4867a570@suse.com>
 <397bf325-f81e-e104-6142-e8c9c4955475@suse.com>
 <alpine.DEB.2.21.2109101613130.10523@sstabellini-ThinkPad-T480s>
 <d868bbcc-e800-ed30-3524-a30a5feb7e5a@suse.com>
 <alpine.DEB.2.21.2109131328130.10523@sstabellini-ThinkPad-T480s>
 <9819a6e9-93d5-e62a-7b4a-ffc2ecd996dc@suse.com>
 <alpine.DEB.2.21.2109141817150.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109141853510.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b93284dc-782e-b455-96fb-1c890ee42560@suse.com>
Date: Wed, 15 Sep 2021 10:21:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109141853510.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P251CA0025.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:20b:21b::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1d013969-4b6a-41af-a227-08d97821dff4
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3533:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3533B29884D67E82D906994BB3DB9@VI1PR0402MB3533.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+d0tehk5Pq3p9hWLi97Y01skDmu1tNto69AjBpUgr84r3y1buV07jyM//eD/tO7VgEovxDrRBg4QODc8ct0ZSgZbWP0ROWy4G23q52ZmAMRprO37+HQd8k8Giu3ruKscOIlQJBD5uBEgIyzn+otDojRyJKitnQUXfKyO2HRjKpIuaOPJDyA44Mzs/UT33Y7neK/HRfXJmTXrdOPxaEC1gH0qkg3gjXJAQy0R1r+eqd7TGJnd1vKTpXk6cp6uDXP6r3bK6UIcfX+z1WSE6kRx37XuLrgwSLIJKmUgmU9SxabtzYVXkuYGXYxMgl0K/ll8B2GEQMENRV1IzUcK/tUndFIe+UtR+5NosaLR2laFfel677zM1H35+ZHZXzJbTARx+JwCoSpUxMmaX1nW72p0vvjlHuhTyklqSsIeC0WhFdaqzvvlmZ9viFRXTNhZwxL2Fc+9KvFG05e/xETNZQchrgB6ZYo+6o5umwtwbng2zq2reiSoW/i0fMCAUEGj/GUHWlrXM7AwEOOVn2/WqD1CGN1/MIWeeRhE71MWsDj3IaQn0z6ihkDTCwLe/WvHcqnUGt41goqlh3QGr84072ibvWSoFioc610+qrTZ9CG1NFBZEWyx9wDdXPlHG2fukgorhnDstXgUW93ZHiKjUUtCIwblNBqYd5Nf3xq9kEbncDEzqmqgUsg3kPNvNG3+bE+m52CriKc4ZAbkGvZHMHEV4Z48lFflTswrprY+wVhHZXGPMD4j/bjwzDdPBCvBtK8s
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(136003)(346002)(396003)(39860400002)(366004)(956004)(26005)(66476007)(66556008)(316002)(2906002)(4326008)(54906003)(2616005)(8936002)(66946007)(5660300002)(478600001)(83380400001)(31696002)(6916009)(8676002)(6486002)(186003)(38100700002)(36756003)(86362001)(53546011)(31686004)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?emFkSVZwRDdTVzhCVGZjR3gvV3dxdzMySTZaTVRwR2U5NVZHNjZRaVN5SjRw?=
 =?utf-8?B?clZteHNJNjJBaVNLRzNGYnFGM1cwNit3VzUyaEJaVkY5NUtRcnFUMmNmWGl0?=
 =?utf-8?B?MzM0N3ZOQjFaMDM5SWx2S1BpSmE1QkRyZjRMR0dRV3d5bS83TlAvaWVzQngr?=
 =?utf-8?B?NnVka0N3UktpaExpa3JId2RnRlFCeDFjOXl0eEZPWElkZVpHVWhyUW16K0xK?=
 =?utf-8?B?ZnZhSXlYaU1IUm5uSTRrZlhJV2ppVjRHWTZKU05YcCtuaDYwN09jM1ZXZVBZ?=
 =?utf-8?B?alhhT2Zzd2N1SFBmVUREMy8yVElvcXZaSmNxZmR1elErVXFjZ29xSDVIVGZs?=
 =?utf-8?B?QU9HYzZjc25nbjRHd2x4b3JlZ3JwYXZ4NklNcVFrQzY0c0I3dk1ucElJQWQ2?=
 =?utf-8?B?bnF3SWtlYkF5U1lHUVZkYUVId1dOOVRzMFh2cUl5Vi9lWHZGcGd3RWxvck1k?=
 =?utf-8?B?UXFVYUFjV3lCdWRVeTRsaGhWc3lRdHhrb0NySWdZeVhMcHNIYVZKUXZYaEJD?=
 =?utf-8?B?NDhENk5KL29Ta201MGlGK0dEUURpdHhmcGx0U0xCenpkcWJtaG56eVBzdVZN?=
 =?utf-8?B?Y0t3clJlSU5lSS82cnpid001WGRnbEozVVB3NEp4Nkt2eDNFRE03a3E5WEpv?=
 =?utf-8?B?VkVFbHFPSko0ZzJHNUVvTFdDWXArUGM4Wmt4c1FSdDFrUVVuOFBTTVRsdERI?=
 =?utf-8?B?M3Z2elYzY2cxSDczenBvSkVOMVQ2dC9jaExHazcxcVJOemlZMGZRaXBOT1l2?=
 =?utf-8?B?ekk5QTBTZ0YveE5yN2QyY3Zqd2RGcGpVbmtzWWQ1NW5ObUwwREZWUm00cUlq?=
 =?utf-8?B?RnF4Vjg3RStlenBWRm93NGU5a080MmNnUnllSGg2eU9OTHZxUlZxY3h5Qkd4?=
 =?utf-8?B?TnZoaEhEdUFIV0tjZWpHWHp2QkZQZ3QzTENEOWFtdkJjMUZUdjFZclNVeE9q?=
 =?utf-8?B?cklLb09wNFR4YXZ4K2ZCVlR6c1ZKS1RuQW8xR1c4L0JkNHVVVDZOaW43b0Vt?=
 =?utf-8?B?dlhXZTdTQWdKN2V3MXhSMUxhajA3NmRSZ0hCcEp6M0g0aDh6MlNCcU1KY0Rv?=
 =?utf-8?B?YjhVOE9hMjBNcDV2aXJQVjcvSVEvQVgwc1pHVy9RZ0ordGgzNmpVVlJyTzVK?=
 =?utf-8?B?bmsxSHllL1UyNUtaUTh5YzFJRFR2Vzd6TE5GbzArRzRZL0ZSYnB3TldsVzJn?=
 =?utf-8?B?UGRBc1FaeG9lYWFXYjk1c2t3SC9HK3lxMjE4cjVCbE9mRi9LL3NqTkdlZUZU?=
 =?utf-8?B?K1FDSjVWcEZNa1BkbjVMRXY2UjRTblljQlNKL3M5eGpqQ0pqTHlrMWRkY3FJ?=
 =?utf-8?B?c25aSkkvYlYyL0crRE0xR01UeG8zdjVseElQbXNjN21XUGJFZWtwWUNkMVhR?=
 =?utf-8?B?R1Y0bUJrTFJML2FjTzhidUwvL25MRitoS002a3R2NE45d2FQc0Jvc0VkcTZB?=
 =?utf-8?B?d0RjL3FBTGcwV1QxVXRwR0tWckdZTEhlc3ZYSTBaV2VXR3AyaDRmZ090UmFx?=
 =?utf-8?B?OFNobGE2YjhyNSt0VENKZU10MU00N0pVRGthczF1RGVhZ1VOc0xUeVM5bjQ5?=
 =?utf-8?B?TVdqMWltdHQ0ZnVKVnVYZzR1Wmt6VEl0d3ZNQTJUeDhKN1IrVTVub1kzbHBJ?=
 =?utf-8?B?ZDNWVDNGR2xoRDRjYUdNM0oyTnphU3dGbUNoQVJrZkozK0w2VUNGMElxeEZs?=
 =?utf-8?B?c0dsMFBEdHhXQjlDMnoyeUFmeVVhYTZFNjB0WTFmRVkxOGhqMW15U0ZpVW5w?=
 =?utf-8?Q?6FnXBAypB+Lw49Y2v6F+O3n9DDMSqP5zYfrSUJ8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d013969-4b6a-41af-a227-08d97821dff4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:21:53.7110
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9SxqIiD3QfobtHq7o0hgxSQEqjvHuLX9E61LtN/o9E4Ozk1ee7+oyNYZ9UdE4fUl3G8qedJTmU+t1PReRZVN6Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3533

On 15.09.2021 03:54, Stefano Stabellini wrote:
> On Tue, 14 Sep 2021, Stefano Stabellini wrote:
>> On Tue, 14 Sep 2021, Jan Beulich wrote:
>>> On 13.09.2021 22:31, Stefano Stabellini wrote:
>>>> On Mon, 13 Sep 2021, Jan Beulich wrote:
>>>>> On 11.09.2021 01:14, Stefano Stabellini wrote:
>>>>>> On Tue, 7 Sep 2021, Jan Beulich wrote:
>>>>>>> While the hypervisor hasn't been enforcing this, we would still better
>>>>>>> avoid issuing requests with GFNs not aligned to the requested order.
>>>>>>>
>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>> ---
>>>>>>> I wonder how useful it is to include the alignment in the panic()
>>>>>>> message.
>>>>>>
>>>>>> Not very useful given that it is static. I don't mind either way but you
>>>>>> can go ahead and remove it if you prefer (and it would make the line
>>>>>> shorter.)
>>>>>>
>>>>>>
>>>>>>> I further wonder how useful it is to wrap "bytes" in
>>>>>>> PAGE_ALIGN(), when it is a multiple of a segment's size anyway (or at
>>>>>>> least was supposed to be, prior to "swiotlb-xen: maintain slab count
>>>>>>> properly").
>>>>>>
>>>>>> This one I would keep, to make sure to print out the same amount passed
>>>>>> to memblock_alloc.
>>>>>
>>>>> Oh - if I was to drop it from the printk(), I would have been meaning to
>>>>> also drop it there. If it's useless, then it's useless everywhere.
>>>>
>>>> That's fine too
>>>
>>> Thanks, I'll see about dropping that then.
>>>
>>> Another Arm-related question has occurred to me: Do you actually
>>> mind the higher-than-necessary alignment there? If so, a per-arch
>>> definition of the needed alignment would need introducing. Maybe
>>> that could default to PAGE_SIZE, allowing Arm and alike to get away
>>> without explicitly specifying a value ...
>>
>> Certainly a patch like that could be good. Given that it is only one
>> allocation I was assuming that the higher-than-necessary alignment
>> wouldn't be a problem worth addressing (and I cannot completely rule out
>> that one day we might have to use XENMEM_exchange on ARM too).
> 
> Also this code is currently #ifdef CONFIG_X86

Oh, good point. When writing the patch I did take this into consideration,
but I had managed to forget that aspect in the meantime. No adjustment to
this effect needed then.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:29:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:29:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187324.336138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQIF-0000CF-48; Wed, 15 Sep 2021 08:29:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187324.336138; Wed, 15 Sep 2021 08:29:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQIE-0000C8-W6; Wed, 15 Sep 2021 08:29:22 +0000
Received: by outflank-mailman (input) for mailman id 187324;
 Wed, 15 Sep 2021 08:29:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQID-0000By-9W
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:29:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 055d2386-15ff-11ec-b4e7-12813bfff9fa;
 Wed, 15 Sep 2021 08:29:20 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-c0oMk0xyNli3FlPz0zxCpw-2; Wed, 15 Sep 2021 10:29:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2608.eurprd04.prod.outlook.com (2603:10a6:800:4f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 08:29:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 08:29:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0233.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 08:29:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 055d2386-15ff-11ec-b4e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631694559;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NrPN8UOMJbX5DXknjPHRqIXzQeCP7d2OwvcDucIFrx4=;
	b=Riw8cljiwLjO2F7hOerP4xeOX3feo5/YjSR0iz7HIZywTezyZPysmmGrI2db3qzEB1DQtu
	P2i2OhW3u74igVd6hhcajzLoxbyaZ/VsvWk4XC5+1WZ7COCoHyGkI4XiKfnqsEbpAToUU6
	bPwws+8dQLvELFbUAW1pyNKGjlP+S/w=
X-MC-Unique: c0oMk0xyNli3FlPz0zxCpw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BKR3KyojnGCablw2P/AcNxOpYnZXAK6fS6jSoWj6MFpypqrjMDe1nO4cs0n1KZva92g/ib3HoxyCeB54y/40a23EPS42LxZtayOQZdnscPN4RSSEHs8QaqzR2WQY6wu5Xor1jKw7KwqkA2Y2LPjfjXORxMZS+bZbfJmlU6PSNnKbq9G+RE4wKpIgilVHAk3DPPzKT4U7bDrDIotvqxRwtG/fnBtHKsmy2qCIp3O/Ig9eHTcso9Ji9TV/9Mto/GYGZa2O7QLDTXSIYsy9uu1sRunEUqoVNgJUOEk2r+K1ykAWkMKhwonHbw6qFXDBS+JgFJ3RqahAYiehvg0WvkFrVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2YQi9jhheTBMVlHNg08LfTgwAJJiCRcBKae449KfxBk=;
 b=Hi6xPT9QK1XfTKwlte+h/B2i7CJgJ4I6EoAa5E31KxPJ+aY6BWFQ2DentiABLPk8Fmt4hU22eDB8w8CaHl/ODwJfWBhN9OrlAQGMIhVZuS7tSGDa4YvwHFZDS+8hoGa4zbPdLpoLT148ENlTV6DroaMKIvnx9q9nqBgPpGJcXTRgDoT0h9Ry9QKbQzkZUvXNpBUpegDWwTbQ1bik2qiat4GXrElx3s+VaKyg+vzfYPkBR9DpGigKW8AimFCCKLU+W7C2afiuw66fcRxO44w+4dTIRWJgTKHRFzYk9wQiAz/vAqnKRkD/Z8OKD9clqoT+XliyUVt3BCaR3LKFTpAaOg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <YUBeJLvWXukyGSFf@MacBook-Air-de-Roger.local>
 <050f71c8-227f-4f78-1ec5-394ba9fba9c1@suse.com>
 <YUCEQFpdc+3kUqQj@MacBook-Air-de-Roger.local>
 <40ea4ec8-483b-9191-a233-f2916f22c131@suse.com>
 <YUCYYmqQMR5NCTyz@MacBook-Air-de-Roger.local>
 <13dd8509-330b-4571-086a-7a3ed9c22c82@suse.com>
 <YUDNh9r8nYOxB76n@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0d011380-292b-49a2-f900-ead44e8a0bc0@suse.com>
Date: Wed, 15 Sep 2021 10:29:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUDNh9r8nYOxB76n@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0233.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2ec2ffe3-c71d-44f4-269c-08d97822e68c
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2608:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260816DF26796CB7281234BAB3DB9@VI1PR0401MB2608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0HYb8vP2tP/qAjMZT2+S0se7lxMdbie9SaiioTKc6Q4GSWZK8YKr0LFV5YbDMy2YeBc79PYLdgjr5Mrc0D8QEkz1/P/69BJnCuE3gq4xNOcxxEwjLqKjKRCUpCttPv+V4vSqtEfKS2mEzvfb5xJZBpeBAggV4601p6zbXeHgaW5+2guGH+G3bpQMzGKlA7cJgMix/3H+/+BJyHdY6ZqNhXqWMAoITzol2BgWoeNu7e150uMgaHnQPO+IPxkRGU8TZVLy5VMBeHSFnuFBiGWE9BlGV5Vls4ZU8Pm1G7syTbUyxL72mpUSbPzIpe8GueAIRDUwqb7h5dc6IztT3tsdjIJFDv3GDsv0I416tvKgF6JnHrZwkauWFp2U4I7G7VsqH61bpCSn3DjOjFAG8N+fjSVhQVvWXY4Fwm8FD2F10CbWNWkjQaM51ofP1ilu67TTWba/KQIycDVkxBjxAiOqxd0TdmBWeBh3p2d3f94KH39EnoCsmjRdKhwLpEPD8VfL2/lyo76V/WxYNac+vpCuwcEw2/0/iKU2UXxkDTgQ3LBFmmPbdxw+wSQnz4KDHPphvGhZ8QyhvV+nxVCD60Pnp8J4254cwea/Tsb4P1eBriJujWxdoqq/PMOsi201hBJM56DEX0zOSGZGL+cfETYqFN/UeHnrZxqiF2KB3qbIpGutW9WJBqIRRP0c+y+ls8bUbwKIM4UNHqwxfHbBk+KaE1hSmGbW6D8UyXIOOMLfwOwuvIkAMwLh6uuh8CSDzzfk
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(36756003)(53546011)(31686004)(66946007)(66556008)(66476007)(5660300002)(6486002)(31696002)(86362001)(6916009)(186003)(2906002)(4326008)(54906003)(83380400001)(316002)(2616005)(8676002)(956004)(38100700002)(508600001)(26005)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?QN24YgvqTzKhfeBjXJO7hy6DEoo+7J2npBMCPiXFc4NVN5qQ0/X9X2h3D9ni?=
 =?us-ascii?Q?w+3U9gN+I5rruUL40m8jYLjEyOBl7lIEM4y/ZpWgbeTOmZNPPWcH77T1NlWw?=
 =?us-ascii?Q?3VnaEZ+6Ly42OiysZJ7k6x2ppqPGuGhyZADOnTJmVxxMbqTmj7EamoVwZ0WF?=
 =?us-ascii?Q?VgeOKoe3bLrn94vNAFU2/qHMlWhsobVw599E/nYPfZlzezOz5PQvSY1LgT+f?=
 =?us-ascii?Q?8obvUlcBumWr4HSWn+bjRS0u0/vFU4o72SF095ZGjojrd3dFS5VkZymyuMZU?=
 =?us-ascii?Q?P4Pkl0ql+fpo0TBSZlNTQB6e8OhmaxEgh+dQiiLXv3U7b72XpatoGhaYDO9m?=
 =?us-ascii?Q?GLcuVtPWKJHnYNkgZvo6c1YOfYa7wua1O2K6rI4OQDcCJ4fGGHeDCuG3x0RM?=
 =?us-ascii?Q?73ZVjITgEvFnJgEuW7HEliJhplT6xhLxSXNiCxb7NnBpvp7i5qEirhRnO0Gj?=
 =?us-ascii?Q?4S8QD40GfOMbCZaCcAS+z17ztkLL40rebKkvy3ltM9FgTXO2ZVksH2/o2cFb?=
 =?us-ascii?Q?wtmsWPYAQpz7juZ7+G6nAOkse1/6sNPGFejaowGF3rVzlJWhZBcRJJNHnzsl?=
 =?us-ascii?Q?z4KzmohCQHP+07c5nxy5ACQbc6VU/ACR867fm/S/B/s5vaX7onP7BRLfsvLd?=
 =?us-ascii?Q?y3wbVTHz6lhHKWFCClm3VczbV2cM3BWkBihwxeykxfAPCbVwoBwoSUpi6KZI?=
 =?us-ascii?Q?OURY+nciBEfs61hhI1YW0ZC+9lEY8vcwDP81kZAx1JndpiCGIY01bdYGzQQD?=
 =?us-ascii?Q?cJETs2i3sKGHg2Xg70nV3+5WoemyfM4MEMtVi2ZK6g6DtKD48yP2+lMwPySM?=
 =?us-ascii?Q?KY9lPyu9Wt6p1t34lqaTasip1d+i2gq8V0wgbnBG1JpBWqoZ2sUdIpSkstC6?=
 =?us-ascii?Q?/YErLXxy44Fk2seQ4Znd7gyw3d9d8LsUPQ1ADnIjOEgaiJK6IE3+lceAuupu?=
 =?us-ascii?Q?wHWiFd6eEf/WZBmcUzafpUS0ooXCMsh4bhjjRwZsCtEJso8q2ULybJYF/kNE?=
 =?us-ascii?Q?b2h7SHX6aSTxJGKrpZJKrGisStLEp9I+APSG9sYy+0VyUQgOsiUnGMYdSN+c?=
 =?us-ascii?Q?5FEMeeNdDRcd5vYEwfoHbu9YrqnjalmX7W653gCaTrID+o+5ViCuRbuE4LeO?=
 =?us-ascii?Q?coGKLkxSoWVl2mNVCYvYh5XST0ecP8J2fNZI8a9ToX5V5VsqX2OjQ+pmVTIZ?=
 =?us-ascii?Q?rWYzbx+UsZJDPw4cpFDWePr+8pcxcQ/qeNxos6I3dBbuIOFo5Cxv3AEPPuY4?=
 =?us-ascii?Q?bflNhgS4bpasZwoeWzyoZZ0Pcfqw8P9Fz/R3mNDzecBsV1ieSKsnxaflqMxl?=
 =?us-ascii?Q?krg7D//M0RbfUWWiLs/J7Ev7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ec2ffe3-c71d-44f4-269c-08d97822e68c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:29:14.2671
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TeqoU17UZS0+1G+bCf5CgfAJB9NDnrpI2SW8Y4r6y4Ny5GwRlc0jxtaO7gs15ppqi1jzEsL9/CR5BUx3pMpdeQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2608

On 14.09.2021 18:27, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 14, 2021 at 05:13:52PM +0200, Jan Beulich wrote:
>> On 14.09.2021 14:41, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 14, 2021 at 01:58:29PM +0200, Jan Beulich wrote:
>>>> On 14.09.2021 13:15, Roger Pau Monn=C3=A9 wrote:
>>>>> On Tue, Sep 14, 2021 at 11:03:23AM +0200, Jan Beulich wrote:
>>>>>> On 14.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>>>>>> On Tue, Sep 07, 2021 at 12:04:34PM +0200, Jan Beulich wrote:
>>>>>>>> 2) Dom0, unlike in the PV case, cannot access the screen (to use a=
s a
>>>>>>>>    console) when in a non-default mode (i.e. not 80x25 text), as t=
he
>>>>>>>>    necessary information (in particular about VESA-bases LFB modes=
) is
>>>>>>>>    not communicated. On the hypervisor side this looks like delibe=
rate
>>>>>>>>    behavior, but it is unclear to me what the intentions were towa=
rds
>>>>>>>>    an alternative model. (X may be able to access the screen depen=
ding
>>>>>>>>    on whether it has a suitable driver besides the presently unusa=
ble
>>>>>>>>    /dev/fb<N> based one.)
>>>>>>>
>>>>>>> I had to admit most of my boxes are headless servers, albeit I have
>>>>>>> one NUC I can use to test gfx stuff, so I don't really use gfx outp=
ut
>>>>>>> with Xen.
>>>>>>>
>>>>>>> As I understand such information is fetched from the BIOS and passe=
d
>>>>>>> into Xen, which should then hand it over to the dom0 kernel?
>>>>>>
>>>>>> That's how PV Dom0 learns of the information, yes. See
>>>>>> fill_console_start_info(). (I'm in the process of eliminating the
>>>>>> need for some of the "fetch from BIOS" in Xen right now, but that's
>>>>>> not going to get us as far as being able to delete that code, no
>>>>>> matter how much in particular Andrew would like that to happen.)
>>>>>>
>>>>>>> I guess the only way for Linux dom0 kernel to fetch that informatio=
n
>>>>>>> would be to emulate the BIOS or drop into realmode and issue the BI=
OS
>>>>>>> calls?
>>>>>>
>>>>>> Native Linux gets this information passed from the boot loader, I th=
ink
>>>>>> (except in the EFI case, as per below).
>>>>>>
>>>>>>> Is that an issue on UEFI also, or there dom0 can fetch the framebuf=
fer
>>>>>>> info using the PV EFI interface?
>>>>>>
>>>>>> There it's EFI boot services functions which can be invoked before
>>>>>> leaving boot services (in the native case). Aiui the PVH entry point
>>>>>> lives logically past any EFI boot services interaction, and hence
>>>>>> using them is not an option (if there was EFI firmware present in Do=
m0
>>>>>> in the first place, which I consider difficult all by itself - this
>>>>>> can't be the physical system's firmware, but I also don't see where
>>>>>> virtual firmware would be taken from).
>>>>>>
>>>>>> There is no PV EFI interface to obtain video information. With the
>>>>>> needed information getting passed via start_info, PV has no need for
>>>>>> such, and I would be hesitant to add a fundamentally redundant
>>>>>> interface for PVH. The more that the information needed isn't EFI-
>>>>>> specific at all.
>>>>>
>>>>> I think our only option is to expand the HVM start info information t=
o
>>>>> convey that data from Xen into dom0.
>>>>
>>>> PHV doesn't use the ordinary start_info, does it?
>>>
>>> No, it's HVM start info as described in:
>>>
>>> xen/include/public/arch-x86/hvm/start_info.h
>>>
>>> We have already extended it once to add a memory map, we could extend
>>> it another time to add the video information.
>>
>> Okay, I'll try to make a(nother) patch along these lines. Since there's
>> a DomU counterpart in PV's start_info - where does that information get
>> passed for PVH? (I'm mainly wondering whether there's another approach
>> to consider.)
>=20
> We don't pass the video information at all for PVH, neither in domU or
> dom0 modes if that's what you mean. Not sure what video information we
> could pass for domU anyway, as that would be a PV framebuffer that
> would need setup ATM. Maybe we could at some point provide some kind
> of emulated or passed through card.
>=20
> The information contained in start_info that's needed for PVH is
> passed using hvm params, just like it's done for plain HVM guests.

This is what I was referring to; I'm sorry for having been unclear.
It's no video _mode_ information, but information on hot to get at
the console.

> We
> could pass the video information in a hvm param I guess, but it would
> require stealing guest memory to store it (and mark as reserved in
> the memory map). Not sure that's better than expanding HVM start info.

I don't think it would be; a param doesn't seem a good fit here,
and I have to admit I'm not even convinced its a good fit for
xenstore and console coordinates (that's fine for HVM; the only
reason I can see for PVH to use the same is the expectation of
the line between both to become increasingly blurred).

> Maybe there's another hypercall interface I'm missing we could use to
> propagate that information to dom0?

I don't think there is; if anything we'd have to add something.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:45:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:45:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187334.336148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQXS-0002c8-JC; Wed, 15 Sep 2021 08:45:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187334.336148; Wed, 15 Sep 2021 08:45:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQXS-0002c1-Fr; Wed, 15 Sep 2021 08:45:06 +0000
Received: by outflank-mailman (input) for mailman id 187334;
 Wed, 15 Sep 2021 08:45:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wWxD=OF=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQQXS-0002bv-1b
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:45:06 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 38aa068a-1601-11ec-b4e8-12813bfff9fa;
 Wed, 15 Sep 2021 08:45:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38aa068a-1601-11ec-b4e8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631695504;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=ZwAgTdU9pJk4/za7hKNn6YsI/jx/DgVo1m7OBlYdgX4=;
  b=Cy2cVgZKZCz7ODT/tsHH9IefuXmrp6NdPa+cO9vzXzSq/kYTgwgofJPI
   jogeH4fkjriC2r3Gm+rI+UYk4oYsVXP6/+9+tVQbnQUv+5iSnckqfuuKk
   utN2zp2xpY1irONTK5fxBVJDbSyj9KucQwpCntrI6y5I5DZ8qgU0ptXMU
   E=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 1Q8r67l/Ojve8Ys2or7LqILnPrtt/pRDoFpAiIZnkhK+yT2QsYbvIxgS1M501+BEJlmBi7xk7i
 HPjA0tnjlBVHUx9ooVbfWTKPSjqFcoH/POcdHPVT3QXyxFoyfM5sCWUpN+pPQ6Upu4Gn3Owy5y
 oiZA212eAkPCGq7Fg4Hc5l3fTnfYk79Wap2/8xtxkZG4xHP4vT0pkBHro9NBTuQdn5crtjSfiG
 c36y5yi+YF5bLsRTBg0iKIfR2+874ceXlOgDNmx2iCfejbmgob5cpZIPS5tkbkOkEKx0nq5t/W
 Z2lZVQCsJ6TVskKt0QcWTmQG
X-SBRS: 5.1
X-MesageID: 52352312
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:tlr1ZqDdqOialxVW/xfkw5YqxClBgxIJ4kV8jS/XYbTApGsq0TdTn
 DMfUWvTP6vYZzOmfY91O9vk8RsOuJTTyYNlQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0E87wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/ojqLpMB72
 dl2uJnuYy4uZ7X1uLgmakwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHv+RuIUDhW1YasZmE+mHa
 tFGOAdTXAnFQSNwFnk0BbMPg7L97pX4W2IB8w/EzUYt2EDDwQo03LXzPd79ft2RWd4Tjkuev
 njB/WnyHlcdLtP34SWB2mKhgKnIhyyTcIAYGaC89/VqqEaO3WFVAxoTPWZXutHg1BT4AYgGb
 RVJpGx+9sDe6XBHUPHUUB+DsHSVsiURXoR6Oupj8DitlpHttlPx6nc/ctJRVDA3nJZoHmV3h
 wDWwYqB6S9H6+LOGCnEnluAhXbrY3FEczVaDcMRZVZdu7HeTJcPYgUjpzqJOJW8iMH8URr0y
 iqDxMTVr+RO1ZNXv0lXEFauvt5NmnQrZlVujuk0djj8hu+cWGJDT9b2gbQ8xawcRLt1tnHb4
 BA5dzG2tYji961hcRBhps1XTdlFAN7ea1XhbaNHRcF9p1xBBVb6Jd04DM5CyLdBbZ9fJG6Bj
 L77kgJN/p5DVEZGnocuONnZNije9oC5TY6NfqmNNrJmO8EtHCfarHAGTRPBhAjFzRlz+ZzTz
 L/GKK5A+15BUv85pNd3Ls9AuYIWKtcWnj+KHsGnkE38iNJzphe9EN84DbdHVchghIusqwTJ6
 ddPccyMzhRUSurlZSfLt4UUKDg3wbITX/gacuRbKbyOJBRIAmYkB6ODyL8tYdU9za9Uiv3J7
 je2XUoBkAjzgnjOKAOrbHF/aeywAcYj/CxjZSF8b0y133UDYJq06PtNfZUAYrR6pvdoyuR5T
 qdZdpzYUOhPUDnO5x8UcYL58N55bB2uiA/XZ3ilbTEzcoROXQvM/tO4LALj+DNXVni8tNcko
 q3m3QTeGMJRSwNnBcfQSfSu01Lu4iRNxLMsBxPFe4ABdl/t/Y5mLz3KosU2e8xcew/ewja61
 hqNBUtKr+f6vIJoosLCgrqJrtn1HrImTFZaBWTS8Z2/KTLeoji42YZFXeuFIWLdWWfz9Pnwb
 OlZ1aihYvgOnVIMuItgCbd7i6k54oK39bNdyw1lGlTNbkiqVew8ciXXg5EXu/0f3KJdtCu3R
 lmLq4tTNri+Mc/4FEIceVg+ZeOZ2PBIwjTf4JzZ+qkhCPObKFZfbXhvAg==
IronPort-HdrOrdr: A9a23:/U9uraEyUulc7cz6pLqFdZHXdLJyesId70hD6qkvc3Jom52j+P
 xGws526faVslYssHFJo6HnBEClewKgyXcV2/hqAV7GZmjbUQSTXeRfBOfZslnd8mjFh5JgPM
 RbAtlD4b/LfCBHZK/BiWHSebtQo6jkgdCVbKXlvgxQpGlRGt9dBmxCe3+m+yNNNW577c1TLu
 vi2iMLnUvsRV0nKuCAQlUVVenKoNPG0LrgfB49HhYirC2Dlymh5rLWGwWRmk52aUIB/Z4StU
 z+1yDp7KSqtP+2jjfaym/o9pxT3P/s0MFKCsCggtUcbh/slgGrToJ8XKDqhkF4nMifrHIR1P
 XcqRYpOMp+r1vXY2GOuBPonzLt1T4/gkWSgWOwsD/Gm4jUVTg6A81OicZyaR3C8Xctu9l6ze
 Ziw3+Zn4A/N2KBoA3No/zzEz16nEu9pnQv1cQJiWZEbIcYYLhN6aQC4UJuFosaFi6S0vFnLA
 BXNrCf2B9qSyLeU5iA1VMfhOBEH05DUCtue3Jy+fB8iFNt7TdEJ0hx/r1Xop5PzuN6d3Hoj9
 60RpiAr4s+PPP+W5gNctvpcfHHeVAlfii8RV56AW6XX53vaEi94aIe3t0OlZaXkdozvcIPpK
 g=
X-IronPort-AV: E=Sophos;i="5.85,294,1624334400"; 
   d="scan'208";a="52352312"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LbvsJ7enWwaavjjULKBD7idImaK3WPB0tOVRhALCCoCWwoQ7RlkNJ8j/KGhPT9B3hyfQkVz7C2h881bBhPKaHNI/yKpCP4ZSQZIEQTDfuI5B3cIsNn4zEKS27xW1naSmp4NHpmrMD4yEdaejuXb0VMJ44cxO3C1ZzuvEFxviIX/0P2TJcMPvrK9W9le1EndHVCUyCUuilf5WifkAk2tFSufOG4D/9wLoblxnJLfkGU+A+IJUtlyOaK6U6trJoIVGqlsalaqYRctfp/CpDTUAXpHD4dc8po7pYvnV3RGHB30wOZZiGwC0KExftr37eZV7nrfYXYgwtmsfBAYhcR2pug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QYj6oQaFjDX2lnu9xCgHFKYYTC1RRp4yRVSQUssYdMQ=;
 b=lgAt53q1U/Nb/0mf6ZGo/8lzHSZZjPe4vM0SipFhTz14kEi+P5ycoGozEN9r1pSaJuCCRR72KY4xdm30SMPS0Kgtwb2RsIiliKzrKaqHeV+J7WQRk/0NpAYDlC7hrR9/BaNjbxlqswYbXLBP+L6G7hvgaWqQA2QKvFmYM+JySFQHkmlcuxZT8SjG+XBBVwIlaehlRzXW4XhCPxtwWe3j7/VyHNo3WgA8ZnTMGveQ7b18QOxgAwCTPhlkdjrl4vNsG0L0YvWI6rRjYLJFg13uUQsQgZ7DfF52sPiLTXAF2XrIZqL5QqE4JArHfoHibqkNNClE1Cfhm5Ihyt2PHp1gKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QYj6oQaFjDX2lnu9xCgHFKYYTC1RRp4yRVSQUssYdMQ=;
 b=bk60A8uS7ScU54wfDrDz/w8g2bPM444mNruPwFipYCxDdWGLnqFeotJ57+odEyCyj78VAUpLYiuqhKBp/Jl8J4DTZs+yrznAxv5UIin53tAA2dZXcLlBl2RROQ55kEIjAM4BM7iiKriYkkzViMHi0S3PpzTNHOX47DFUqsN6G2I=
Date: Wed, 15 Sep 2021 10:44:54 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2] x86/p2m: fix xenmem_add_to_physmap_one double page
 removal
Message-ID: <YUGyhk0uZQMf9c4s@MacBook-Air-de-Roger.local>
References: <20210915080342.21346-1-roger.pau@citrix.com>
 <1d70ff4d-bef2-1865-27c3-27d17c15d17f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <1d70ff4d-bef2-1865-27c3-27d17c15d17f@suse.com>
X-ClientProxiedBy: LO2P123CA0101.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:139::16) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 064d84b0-459f-4ee2-685f-08d97825193e
X-MS-TrafficTypeDiagnostic: DM6PR03MB3947:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB3947374217596DC9C5D3548E8FDB9@DM6PR03MB3947.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 5r+Hyx9J5nBNJzmPPDLcP/VQOP99adJkVXULM6GF85nRpX9kW7DdXuTwVNsB4/K9wDE9i6oPPhmB6qcWlPixWWaseIW1+jbiPQ9HGfLxHj8l0uHcwnmjIODWwecAxev9lztWEFk+KrMf9PcqKp1X8pZrWcLuEeUhhiTUGkZ1LmDAhhNxNI+CTAM2C6IKhOZAnj4iaLv4rLLtM28zKevtQZ5f0+kBHhOYz3JXRdLbf2XbdMQeDGo2X+XDWSG6WH2XhygRZJri1yEZU2O1rP8MxhKn4Y2ZFe9uiDDygrecJkyrntApx8KxzCH7H9fNaXE0AdkMA17MgAuqZKSoel1yOo6P9Cnrz8s/3VpaJ1BAspq8PFQ2+ujVOIyMfnep67Puq9GMOIrR6Quq23pHeBMVV/oZn6vlZvFX1j9CjoVT9O85DkeyKCcCfLYfFnIDltOjaaxiXiOI7bg++CaH0ml/0oPhbfTkKV1cPmoITPnY+LXzM8e3QR1GNPkrG7MBaSdyXek01tslSxdjD4xHF16x43V5y/wI7fCArOA5SRsWn4hhRvOcvJnK3eMggYp0GBKbmkTrKCkaYAjdjSOg1A6zcFHFTebk/BCjUNzIvMVcLp7cUkppVCrha91ItFNN6JlhIhkBJcYoTn9GJdAbgHwB1A==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(396003)(136003)(366004)(39860400002)(8676002)(54906003)(316002)(83380400001)(9686003)(6916009)(53546011)(6486002)(6496006)(4326008)(85182001)(5660300002)(6666004)(66556008)(66476007)(186003)(66946007)(2906002)(956004)(478600001)(38100700002)(86362001)(8936002)(26005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TG95cUZpQWhONVVIODhvakxlMmFla3IrZTBxTzM1VWFhU0w4VnBMRitiZXcr?=
 =?utf-8?B?MVk0Y2R6S1pZSXJBOWVRL2EwMEpHbHNFT3p3Qm1JR1B5QnRTbTdVby9mZ1g1?=
 =?utf-8?B?T2dUTXR1WE9hSk5ndjQzcVFhOXlaNTJJUXQ1U01IQ0E4TThBczhobGh6SzFH?=
 =?utf-8?B?eDJhZERJcVFZeERTOWRORCttSmxEcTVVVzgrbm1SUCtWdFFMS3hkMTllM2x6?=
 =?utf-8?B?UTV1RXIzakR0V2prd1ZqUFhJcktnTkp2NTNGOTM2a2YxOFVtTGFuUC8rSGhr?=
 =?utf-8?B?SjA3TG56MndwVVY4WU1jSUduZUFLRHE4MEZUMkpxTXZLcHV5MlNjeUJlVjNs?=
 =?utf-8?B?TlB4MElVQnRrQ1YxdFdkUTBqbi9XeWNBZlRDS0k3T084c0x4UXhLWUhMR2d2?=
 =?utf-8?B?N29OUDVFa2d4VDRLRHI3akc5TllOMnlwS1ZZWjltNTlrZ3JoWDZ1VXhycVhL?=
 =?utf-8?B?V3NUT1BXOGpWanNLdGdJNE1LUTg3UG44MWV2TXBndWRNaURyR3FnSzlzSFho?=
 =?utf-8?B?bWlSNDZURkVBcVhvNjhtYU4vMTBiSGF5VDY4SVBUY2ZPekxwMzRuWVJnTEJL?=
 =?utf-8?B?MUo2ZTR5amNNeUVJMFJhTXdDMXZ1cXNTN0RhVkpaMmh4M29aRG1Wb2F3Vm1P?=
 =?utf-8?B?RnJ2V3BwSkxoNmdkSlhLSlhjQzRZS1lJb1doREJmVHhCY2EyZDY4MG9qM0cr?=
 =?utf-8?B?Z0szUk9RR2lBenhMV2JOb3h3TXduRkVMek5oVkM1WnVsdkNVKzM2eE1jbzNk?=
 =?utf-8?B?czAxK05mVWZYSGltcis0cVRobmNScGhpa2MxelZPSW0yeitneTJaWmJ4TFlQ?=
 =?utf-8?B?ZzJPb2VzL2hMSFhvQytHMWtiZVgzZlovaTR1ZVFnS0UyUHhQWVVNRVVzKzlP?=
 =?utf-8?B?RCt5Sm5kVkF4OUVqRjNZd3dZYTBJSHVseFhJdk9jb3l5c014WUQ4TU11YjNK?=
 =?utf-8?B?RTlGU2J5WTEwam5RQXQxQ3VsU1RvWHlPSWV0cWVRMWo3eWtCbCtPQjNzM2tv?=
 =?utf-8?B?ZjlzQXY4NmxVQ0c1UFRYTWVPallBYld0NU5xVTRwWm9DUU5NRXZEOGk4ZlZv?=
 =?utf-8?B?MXhmTmRlMlZGSmh1T0ZiRmdlVXk2S09SSnNnSHgrNGFlQXRLd3lTN3NQbkEy?=
 =?utf-8?B?SkkybmFJakJMV3RDRlRUdGRBQmxtejNjWnFVQjhOL2FZdFEzN20wazdnNHJ3?=
 =?utf-8?B?b0l6M2pldm5RSlArc1k1ZmpWeWtGSFN6VXE4NGhHKzFLUCtUMnMxSUpkT3A3?=
 =?utf-8?B?RWorWEdYVXdVVXpOU0c3OXRna2FPNXpEeGNEUHBYNHIyRzU4QkRSZDlieE0w?=
 =?utf-8?B?YldUZ2tSWnFYc0RTQS9vUUFBUVNDcjVOaXhXbnM0SU5lRmx3ZDh2TVRNNXFh?=
 =?utf-8?B?NEFPcHdEYUNzd2VhWVBBZW9Mb0RHcXVXZWtZR3RORVYxdmNZL09ieklUQ1Fy?=
 =?utf-8?B?QzBXcWJqTkNaOFlHT0xVZzZoVE81Nlh0UlVPd1BhUS9VNEpkcDBHczVpUUVx?=
 =?utf-8?B?M2cwS3EyVHVCTS82TG5pWU1uOFp5eGU0QTJUdEtlZUp3UjYxUUhiQy9uVWNx?=
 =?utf-8?B?M1FsMUs3dnVRdENoTDhSRURNcmdvZzNLaTgwdUFNM2JqUUMxUHhxWVJ6dUtG?=
 =?utf-8?B?eXNibVhMQ3hBSTh4WVpkbElwaTlIazVRTXlGa0JLclBQWG8rM1Y4c1Y5MXAx?=
 =?utf-8?B?ODQzZ0d4OW1ETDNoekowTDBQam1SbmpNbUNVc2ZjSEJjYTUyWW5TRjZZR00z?=
 =?utf-8?Q?Hmu+vC6LcvOh4RwEbGhKr6Ei92yqOmpzzK/Ztkk?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 064d84b0-459f-4ee2-685f-08d97825193e
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:44:58.4280
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vGHoWiff+HrpNl5xY06bqkMFkoEuOOTKp+BD1lvQbBKmtahyUlJkSybW/s96Swa9ydFudKuC7SZHTliZqPp1Aw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB3947
X-OriginatorOrg: citrix.com

On Wed, Sep 15, 2021 at 10:16:41AM +0200, Jan Beulich wrote:
> On 15.09.2021 10:03, Roger Pau Monne wrote:
> > If the new gfn matches the previous one (ie: gpfn == old_gpfn)
> > xenmem_add_to_physmap_one will issue a duplicated call to
> > guest_physmap_remove_page with the same guest frame number, because
> > the get_gpfn_from_mfn call has been moved by commit f8582da041 to be
> > performed before the original page is removed. This leads to the
> > second guest_physmap_remove_page failing, which was not the case
> > before commit f8582da041.
> > 
> > Add a shortcut to skip modifying the p2m if the mapping is already as
> > requested.
> 
> I've meanwhile had further thoughts here - not sure in how far they
> affect what to do (including whether to go back to v1, with the
> description's 1st paragraph adjusted as per above):
> 
> > --- a/xen/arch/x86/mm/p2m.c
> > +++ b/xen/arch/x86/mm/p2m.c
> > @@ -2799,6 +2799,13 @@ int xenmem_add_to_physmap_one(
> >          goto put_all;
> >      }
> >  
> > +    if ( gfn_eq(_gfn(old_gpfn), gpfn) )
> > +    {
> > +        /* Nothing to do, mapping is already as requested. */
> > +        ASSERT(mfn_eq(prev_mfn, mfn));
> > +        goto put_all;
> > +    }
> 
> The mapping may not be "already as requested" because of p2m type or
> p2m access. Thoughts? (At the very least the new check would likely
> want to move after the p2m_mmio_direct one.)

Is the type really relevant for the caller? If the mapping is already
setup as requested, I don't think it makes sense to return -EPERM if
the type is mmio. I'm also unsure whether we can get into that state
in the first place.

I'm unsure regarding the access, I would say changing the access type
should be done by other means rather that replying on
xenmem_add_to_physmap.

v1 was indeed more similar to the previous behavior IMO, so it's
likely a safer approach.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 08:51:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 08:51:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187342.336160 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQdH-00047b-8Y; Wed, 15 Sep 2021 08:51:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187342.336160; Wed, 15 Sep 2021 08:51:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQdH-00047U-5V; Wed, 15 Sep 2021 08:51:07 +0000
Received: by outflank-mailman (input) for mailman id 187342;
 Wed, 15 Sep 2021 08:51:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQdF-00047O-Q1
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 08:51:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3117ad0e-2e44-44e1-9db5-d36df0179b26;
 Wed, 15 Sep 2021 08:51:04 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-hcS_bbY4MMiu4JECqCsohA-1; Wed, 15 Sep 2021 10:51:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 08:51:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 08:51:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0237.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 08:50:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3117ad0e-2e44-44e1-9db5-d36df0179b26
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631695863;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=o1r1mvd/U+UK2xcDF8uzoLqEn2rfVHv9tFZu8OzyBY0=;
	b=e6vJoZZNxnUUbQ7okPl3FMeyW/TyhFp9MgwiAZ1w6d3OXW9KNACiXa4H7a7c4ehz5WHv2U
	tGbJQp48qyi61PlZiLV/a9qpdC7V3j0fQthC1dnqJKD7mudVqBsP3P4Myxu5vnliEquCo+
	VSecyJsO3DoUSS2SgFujhiTl598nYDs=
X-MC-Unique: hcS_bbY4MMiu4JECqCsohA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nrqptWP5yG/BOIDI+QeSl14nPF7WDggOrQydE+jsmvGyQG36HThP96fsKlIw6tQQu13sptv9eSYVCuvx1fYAbJO86ckTvKBAyl1QXvI5URNvjcgsrA8EPMS0NTPzZQWM7dVvUyGBZshNh0MeTj7xcEUL/BVdASkOV90cJd7RwBVzNWJdB0UQ2Qnifxh0u5bHfp0A/A55utQV0a5Ff6QA/RLXKkmKXkmRUdLVxqbQuptDKnA+KuO7/Ay1xj5SmTQU+e+OlqUjHRUeHRwDVIeTr0hvnuS96rid6WNXBRLeTbtPJDhiRx7Scu7hkiPQ71C7FPLCPL1DPIvGs32dMaguIQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3XNe/JhcU0nN+6cWPJW+M2KdPbtv7prVQ7AwVqLTYJk=;
 b=cCYN4CVcFnbmwkfldTnsWMe1ZSWwQFJuMYfrl5bGPOzG0wo6sj+bDLC7TSO5Oi89Yc3Y7+9p3eIhSM/V+5+EmDqOVsChaI74h6aXiHwDIO4Gm4Y/JkFIYP0WfR8jQWrnUqKFBxp2OHo52BWNDvcyBSLqSnlJg/Ki1FPQBU+smW5DYqQ9aY/hN9H8+HZXndUd49YphNvQ72eSmFZxSZMZaTWcE5+WYYkrW+jfImrROsA9wWt1SKuI1ig4MQjrfhL7oRyl26lNhZG7A2MWilOE8n8hZKpSl1OHj0LRnV36yWt2MBo3JjNfI6nmdzoDqEQEmoXv9lcP/BwlCbWIPYhYbQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86: quote section names when defining them in linker
 script
To: Roger Pau Monne <roger.pau@citrix.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210915082123.21795-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6a71f281-2bcf-910c-91ed-881de3f78439@suse.com>
Date: Wed, 15 Sep 2021 10:50:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210915082123.21795-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0237.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a376f78d-f146-4176-5f4e-08d97825f106
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233569596860FEB473558BA2B3DB9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	P1x3cPybpQFnqvnfL94MjIanQskP/hVkG5CzPmWLW7T1JnMBl1C4nUVMPhyaXntBywVnPjp1gHbyMX/DjOf8DiT3q9meJ+EMx9xZ8/tjDjiTG9tZT6lK7iLKvo1vphzhdfClmK1uPJtZgweM70QbqoeHLdTh2NSTKCGnB2dV9NFFngQ1TegsCN2qE83PLWzgjZshmUlwJd8QFHx0cPXKhm11c9cbsgE9AftmYKqIdB5KgwQLvm65Q1AG6vbFjaGrlYn96feSXKs+FH8ief4b6FZcA9tkcVgHp+e3uCbVqWoRgBmP0lUJKgtsfZ6dKWoTGmovMxColEoPieBR8qkaaUNDKdmCwyfl6To6I8ge02CEcnK4em+1nhcZ3RN3QkIIHGxx1WdqisPEsjyGSY2Itv715/r7f6wpXXoPHFAO+ygSBbnQVcSRIt7W9KjUUUOlQPcpRVZJy4IwZ04hABbkDE96qflei8uah1DFktvSj9E3xbzHTEgRa6BfDaBDIfw/IJ8ckITBYhu0o5BxxyQMCk/yJ9P3p3Cc5LCfvz0La34Kch9t20/Sj7r0IaCX3KYbl9Ti/88nRIxGuT/xbEwxt/IVFaXTFxM5ghG6QgJr1ErgnnUvu0bZVkdO5huA9UvNS1+0fp80B6kQ3vXnXLRrtfDjCmH7a6VSIGjM+auganvk6MUnAFJbEbB8FAJt9DAUcRHzYSGHUwz/7O540GdcTzWKFBEP+OMZ8JcflSERmCrwDMRBS2ZU3HXL1qvtv7P7
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(136003)(396003)(376002)(366004)(83380400001)(5660300002)(86362001)(478600001)(186003)(4326008)(956004)(66556008)(66476007)(54906003)(8676002)(66946007)(2616005)(16576012)(316002)(26005)(36756003)(6486002)(6916009)(31686004)(38100700002)(53546011)(31696002)(2906002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?dsTFLP13viThDnU92IQHNaWnbaCKK/PI04wOFjKQu0tYR+naQp/Mt9Blj9Nk?=
 =?us-ascii?Q?c+PorNgNI5YXsdKu8CRuhM35gmXn0+gpMQj7Q3qevDKM/QiIk0uPSJSG9/ik?=
 =?us-ascii?Q?cuuTOlcFO50dp6I2xO5OJ0dz1i2s/RHAxAIOuFL4+/zjQpou6AOjPVNp5xWK?=
 =?us-ascii?Q?dpeRFDwDE2ZVwmiKpCdvlebLDUptQEjcAeAopXWkfmhp0IjxPNgpREk2yngY?=
 =?us-ascii?Q?ugJfw463b0W1OKzLI5NTvlQlbWS7bb+BTg59nC8otbD1pK2xAaRV1I0VcFv2?=
 =?us-ascii?Q?t+g80FeUXNt+eT9WzJRVcdM92UIEZr5V7PAzthZItOKgnHpoD/DdrUqIoOFw?=
 =?us-ascii?Q?i/qEnPgb7JVfzyECnQJ4U52ItVSxSeY4nKlIaNyyg1WnxkXWfoAhGULmTbwL?=
 =?us-ascii?Q?7W68wiViUwz4B5FVC76PeZ7a1wiPox2IQ5RqiK515pW6tMyqSEKfFmfKwiMy?=
 =?us-ascii?Q?5EYFbLcY4xXJZi0ffMnWJ+tvHdB+RihZ0LTg6yaZ/i/laiZc8pMRA/J4ZJvj?=
 =?us-ascii?Q?GfiDwhme2rg7AMO0L5zBHxurekI9vfDwzIRmnHvwU4CxbiI3CJUMI7yR5zyO?=
 =?us-ascii?Q?nTM1zOCkrFbtFlXB09CdZG2XG8X4f1kgArJW4uEXjLh0755W+DevSkpvq51s?=
 =?us-ascii?Q?EuxtwQVfZtLpNTOg5riuZDDbtqMB9cpScKKuAs711M5MEngpK7FomfuBg38w?=
 =?us-ascii?Q?blhImOZgOLDDJH0qXJb3g7l7lAsaFZ9r/mp8BCkRYsU9NWAuhkK05pv7R92E?=
 =?us-ascii?Q?+Gm3ksI6ylP5F9JTY3S4qLQOV8F4LqW5FAihZluvn5RNg81saSMD/6nHFYXR?=
 =?us-ascii?Q?VhPimHDO5Fmdhv0yjTtOIagSoOHVRM+S7GZIV7ClstfGvjjFgvI0aIFThgmC?=
 =?us-ascii?Q?d6LFSMnOq49q/6G6+KVBw7K0A1oTlN2DafEOi9rS97K3cjvFxEKe9nXy+v7j?=
 =?us-ascii?Q?T7FW9FuHp93ZblQRolNk3po2QtpMyOh4kIAWC+M1EWHT529IUFu4gDWWj6gt?=
 =?us-ascii?Q?AW5biViEKiWjWFnh1SD5TT816IJsOdSposnAaP34KurRkVLvGQGl1DZ56NPq?=
 =?us-ascii?Q?XmW6d+ttUVcWhJq3AsSf+y19X01suRrYwv41mHyUTuHyJNY/aSSgNc+bxP/D?=
 =?us-ascii?Q?3e+Y7uAgqua8gGn4OwV9ChawpfX8fTn9+AhDL+uv5/R6Drp9WPXDl/DnIMS4?=
 =?us-ascii?Q?fCr2oVDjJJYHp+h5x+hZnY+BWWH9wV7DVJuWlKRbYwemcynQKbxCM2d9u2SU?=
 =?us-ascii?Q?Uq1HOK/legC5r1zek0nSrj/zb/ESlZ3uesNpD9QFONChBAWwV+jTt+Lx1Mf2?=
 =?us-ascii?Q?0pCN6IFgyuUNEoTpOIRCGhlJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a376f78d-f146-4176-5f4e-08d97825f106
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 08:51:00.3305
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eG4Wo4lofcuXaKz2bae+0tjFzjkdEmxYAhzQpgiC3MdnfBBuHOWG2MWqixCoEmGb6YYVR0sJSu/6X60Z9APmaQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 15.09.2021 10:21, Roger Pau Monne wrote:
> LLVM ld seems to require section names to be quoted at both definition
> and when referencing them for a match to happen, or else we get the
> following errors:
>=20
> ld: error: xen.lds:45: undefined section ".text"
> ld: error: xen.lds:69: undefined section ".rodata"
> ld: error: xen.lds:104: undefined section ".note.gnu.build-id"
> [...]
>=20
> The original fix for GNU ld 2.37 only quoted the section name when
> referencing it in the ADDR function. Fix by also quoting the section
> names when declaring them.
>=20
> Fixes: 58ad654ebce7 ("x86: work around build issue with GNU ld 2.37")
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> I'm slightly worried that LLVM ld requires quotes to be present always
> in order to match, but I can confirm such quotes are not present in
> the sections names in the output binary.

They're "drop in replacement" statement clearly is too ambitious.

> --- a/xen/arch/x86/xen.lds.S
> +++ b/xen/arch/x86/xen.lds.S
> @@ -18,7 +18,7 @@ ENTRY(efi_start)
>  #else /* !EFI */
> =20
>  #define FORMAT "elf64-x86-64"
> -#define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
> +#define DECL_SECTION(x) #x : AT(ADDR(#x) - __XEN_VIRT_START)

Indeed much neater an adjustment. Thanks!

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:10:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:10:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187351.336171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQvo-0006eF-TZ; Wed, 15 Sep 2021 09:10:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187351.336171; Wed, 15 Sep 2021 09:10:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQvo-0006e8-QU; Wed, 15 Sep 2021 09:10:16 +0000
Received: by outflank-mailman (input) for mailman id 187351;
 Wed, 15 Sep 2021 09:10:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQvm-0006e2-Qj
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:10:14 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bbaf3d04-1604-11ec-b4ec-12813bfff9fa;
 Wed, 15 Sep 2021 09:10:13 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-P2XbVVMzMGuKONAOHg-auQ-1; Wed, 15 Sep 2021 11:10:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 09:10:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:10:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM9P195CA0017.EURP195.PROD.OUTLOOK.COM (2603:10a6:20b:21f::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 09:10:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbaf3d04-1604-11ec-b4ec-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697012;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=WC674Lr8KsLW7ShRRnmjydlZkAKLLT/FjGzCzUM9sJ4=;
	b=IAoIy8U5uAvc8dpaefCwZ/dDmVXduCsOHh68xNSkDrrOgREiP1wOzSX2+se07uIlseXPTJ
	vgEPNFJBl3NRX/ooWhHW05r8+9z/EqNzRWA5KCseKhJvYZV0xt+MDJYfXZmp25YmDDNJua
	Y7gIaDrEAH5Q8z7D3YGutueM1XNDs2Y=
X-MC-Unique: P2XbVVMzMGuKONAOHg-auQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gVWUrUp1TYjtPQFzuVt2qNl55YqTgYU+Eybr4rjIZEiXV55l+VgT2xaaTJ7Hr2hPfkbXI+EAGZO9jbou4NeBLrrcEwifL1GZG3Jgjqlb2rV02Ae+AKwCUsHCrJHgpRYL294NwLU+KcsukucNjmIplljrFf/EbYgwGP1s2miwmA7/g2UqqKGaAHE2kZQGk+cF9pi6bP9/W+FLu4NmccjKBuzPD5dAH/mXRxuJi0w/1T/XjRQiRW36UQfFNxDe60GZ+h5AwpVQI2h1I8xgplfO5JHQ2X7q77FCvvx7UbbufP3+iizj77GRhz4MQk0DbH+1Hn/7nNcMF1HjdwrR/AeWdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WC674Lr8KsLW7ShRRnmjydlZkAKLLT/FjGzCzUM9sJ4=;
 b=jiT6GBDq70s2fzwUMRNFbO1fRaHGn6f8Dpap/D4pHyh2PdECYGYfvS1c3mH03VJOl3afK16CsoN4PMykfiL0WVWi6k9YD0GUPE34yzr1psu+UJ9WlCW2BFKPNQcTI7GZDAav6pOiWaGrjApzFSE1hG63KcVYH/AhnITR/MLLxYOCAM25/8R6me+4RZ1AaLPkg5seZvws/rgzWHrpfIzIYNihgICjydZ/mbfleAJq4jTvsZw9RYH+m7ndGEoZTmgxMreN29ogrNV5MnXUWoCNZaHKrfcbegUTzF42AZeN4vC4aD3ThSeA6fla8zd1PwK2mEajPb65/TNeXhkVi/1mzg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: IOMMU/PCI: respect device specifics
Message-ID: <b232a5d5-5054-e305-a260-a0562570f033@suse.com>
Date: Wed, 15 Sep 2021 11:10:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM9P195CA0017.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:20b:21f::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 05e927e6-cd63-433e-8428-08d978289dc2
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6862CD7EB13554287BAD3EBEB3DB9@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	k3JgEyqxwEWwqgGB+EUEkFE3dB+O3d2OLTe5BeDJiVORVEhHesvk2SSm6iczNlQmbiAYoGXt445yQcBqN56XW3cqsTa13ocCkO0eMTAHJvcepqn9bdl8Fma/UUVHBCZfl9Fm5ZYDKXepDdrFxGksniNdUMgf9g1VnFLwQUjcSQuflDo8BpZ4ajRWoKb74GHlxtsRdhnY8lsDbw9wJa0A6RNWtlfMll1TmxHprjezD6GDJ2VtoWucZRWuXV4eYQWXfl9JEBLqAy24As0YH4Oht6ECI8/3wwJ/Vpw2Nik53vrBL21c9DZ7ThhwXPTuc+GOr+z5L7GXHsZM6WlzUAQ3DDZug4lRHrc3AvDIkmcP8m3RQIN7AfT0w552vxeuTHki682DQ1q+UpY2jti1aGjCcnGaPglbJy1s1VmFwjBk4PLNxPXSInsBS2MjpXJHIL4Ay3lFXuDazOpv6T2WqNt24qEtxK3gYHPWCvUR/9bDb35pR0cAl+MF0QebglmnYcH4106bpbJjRYjhqheHGvxvXR6d/tyNyZypkaMB79naYjAaTgBwLPJaF3QiBJvqfIiXQhJ8QTuUqiiyzk/QJkIrHLXWhi+gL9EtH8dvSpip2ts1rIH6yc5PdaDXJNn1WZ6ij0HgloJqzp2XFYKH2JdfKojGXP2YOguCyBN1KBGwQn6gp/gQX3UxX+n7DDxcoMypwVsLLVbxgxZJL/qGSCMmIG4G+9lwU4if9N99iafgnaJi3RnZ9aaxn1SWXGUFBO2m
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(376002)(346002)(36756003)(956004)(2616005)(86362001)(83380400001)(8936002)(31696002)(2906002)(31686004)(4744005)(66946007)(66476007)(4326008)(66556008)(26005)(5660300002)(186003)(8676002)(478600001)(6486002)(38100700002)(54906003)(16576012)(316002)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Tmo3cHIzTXJjT05TMll6TmZuSm9UQkErQ3kvcENvemswVDlsU3AwSkFzcGpX?=
 =?utf-8?B?YW1sSHF6WS9JVVRkY3VSVlFSOURBREx2c1cybkl0c0pOSzFGZGlGVUpUYnNz?=
 =?utf-8?B?Nms2T0x1VlE5MUJrZWEvUXZ4b0xvelJHNHlqL2ZpeGk3bU8vME0zdEpDeXh2?=
 =?utf-8?B?QXBYdVVnSFBBV1NSNUpINFZiWDV3dCs4TklScGc5WHR5YjFnZlUvR2tNRmZQ?=
 =?utf-8?B?ZXd1ak1JNG1YMTJYeG5XelpxVXppTi9VbVNjWmc4bDNNRHlpeTdscUlZR05E?=
 =?utf-8?B?RlVybEtaRGxpTVo2T096cFJqbGpZQnJ3S0VjZWVneUluUFBiNVNYSGdJWUIr?=
 =?utf-8?B?NGF4ZXRmeXVTTm5WeG9FR1Nra2R6bmt0RzRkZCtnUHVuUjREN0lyZktWUE5k?=
 =?utf-8?B?cjR0Vnk0Tk1QNkczUjJReFJuSDl2aFRjRXlEUklzbjNsYm94RENDWlVzc2V0?=
 =?utf-8?B?UmdRNjRXcmhCb09JSlhqVnJ0S0ZSUlJZNkdFdzRSR1R0Y0dQQUNCSWtFRHVX?=
 =?utf-8?B?VVVjNHhZUVI4RVhWb0tibmRGWnhMekhxY1JXbVQ0N0Era0dhSm0ycTFEc01q?=
 =?utf-8?B?aTQ3djZZMlE5OEtreEM2L3pra0pWS1N4Tlc0SXduUDZHMXZpclhROStwL0Jj?=
 =?utf-8?B?RlpGMUJHNHZya28rem0xd3pLYXdva2c1ekw2b081Vitqd1prRGtkejhaeU8x?=
 =?utf-8?B?b0M0VkN5SXVHQ1YzUk1wMFJiUGZjTlJha212c0RLUHJKOHFpdzJDVkZYcXRK?=
 =?utf-8?B?L1BrWENIc0VzQmt6WmZpMVJtd01HbE15ajlnTksyUTA2Zm13QWxreVlOKzlZ?=
 =?utf-8?B?K2tEc3E4K2JBUGlpcG9aT2kwWHlVQkt5WExxNDR5WXlyRHVHUmFhaXMrRjRx?=
 =?utf-8?B?Tk5ocXVzR0ZsTXBZTVE2SzVJOFhRMWp6emUxTDBsMU5YaXpveHE5eGpOd1Mx?=
 =?utf-8?B?MDE2ZEgrOEIrUEhkL0syQzJWbHBtNkNuaDl6RE1LMkFleWdqeGdsbzBVY0Q5?=
 =?utf-8?B?NU00VGQ0WUZPYkF3Z3I0Zi9qQkYvbjNnOG1PYm83QkR0TStGSHQxV1VsNGl5?=
 =?utf-8?B?YXhqeE8zM1VVQ2N2ZlhzVEp4WS9oZVJmODZIbzJjenNvUGZSMXYvOWdGMjlU?=
 =?utf-8?B?bFdUODhIcWhmczNOYXNDRUZKYk9EZmc0ME1CcGhJRHVjY2pyRitTQkNvRklJ?=
 =?utf-8?B?WU05WVhrOEN0dW1CWHhMM09oYXg2dVVaVXdwQlo3TmlBbHdXVWprdzNEcG9Y?=
 =?utf-8?B?cXdUb3lnbEFLTFlCQldkWFM2MlFseUpRUUJuOUtuOEFSc0YwUzMxTUJxZ3JG?=
 =?utf-8?B?Yld0NFAvamJQN1ZhSU9BZFhCRjk1TXZ1MDNzdUVOMG1EdDVpcjU2T3NrUHJl?=
 =?utf-8?B?Skd3aGEweEdWenEvckJxKzB0cnA0NmtTa1dZNFFpQUFhZG82NVFZY2x6M3R6?=
 =?utf-8?B?MUIrRm9hQkNadjg0THRCaGRLWVBPYWRlMVJPbHp2bzFENEFzZDJaa1pSWnRP?=
 =?utf-8?B?YWtVNUVYaVh6aDZPWmJaU0lEQkJpSWV1WUhwTXd3WjVJZENySjFUcDJWZ0hr?=
 =?utf-8?B?WEtBNy9FOWVtRnF4bFRBaWNXanUzRWMxeGFUbEJPVGJ2dXI0bTJYR0JjeHlB?=
 =?utf-8?B?NjlmSzNxWVdoNGkweEdocmUyTmFDK1NSYmFZYUNyNXNPVmFJeEI5bUhkaFJV?=
 =?utf-8?B?UDVRd1gyOW1TSW1Wa1V6anVNOGRZUlhFS2lJa3ZKZWlJVHJCc2lHZnJocTBa?=
 =?utf-8?Q?zYtCh5NjgzREsTARPO+iPF/G0u9HuCjJQp4ebBS?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 05e927e6-cd63-433e-8428-08d978289dc2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:10:09.1440
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gHQkEwbBzQMS0JtoX025kX4Bbz/lLFt7xgsWcMY4ZS7CB34hg68aGLP9+zsT3qSCH1IypH6bPBiT3mR0zAuO2A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

While making the first change here it occurred to me that the recent
vPCI-related discussion about hidden devices has some relevance also
elsewhere. In the course I then came to also notice a phantom device
related quirk.

1: VT-d: defer "no DRHD" check when (un)mapping devices
2: VT-d: consider hidden devices when unmapping
3: VT-d: skip IOMMU bitmap cleanup for phantom devices
4: AMD/IOMMU: consider hidden devices when flushing device I/O TLBs

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:11:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:11:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187362.336190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQwp-0007L0-CS; Wed, 15 Sep 2021 09:11:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187362.336190; Wed, 15 Sep 2021 09:11:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQwp-0007Kt-98; Wed, 15 Sep 2021 09:11:19 +0000
Received: by outflank-mailman (input) for mailman id 187362;
 Wed, 15 Sep 2021 09:11:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQwo-0007Ki-Jf
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:11:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9f00e41d-fca5-4e3d-a136-9f313cba18be;
 Wed, 15 Sep 2021 09:11:17 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-SsfKgIQ6M0q1Vlnzgn8GlQ-1; Wed, 15 Sep 2021 11:11:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 09:11:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:11:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0112.eurprd02.prod.outlook.com (2603:10a6:20b:28c::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 09:11:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f00e41d-fca5-4e3d-a136-9f313cba18be
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697076;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=itTu3oZc5MzNRF0hPw+x0ZHFualAlO0fWetEbSE/YRM=;
	b=fMfiRHa3C+H+Ts7MAvs3p2s02JtHlchllv+czA/TMfMh5AAvX61qKeuRp7B0zBjRJDm3dK
	0JLnn+w3jSCI+ZjchKCKTH2l5fF4rAooR4uUnpI9U+WUREjwWPiTvzlu8MmbmYEwLeN3en
	miSG2X9Nmbyh+x0D+Jt18G1DgWNnpb4=
X-MC-Unique: SsfKgIQ6M0q1Vlnzgn8GlQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GUc0YNJeZy1VJNO/GICQqgMVsntCw60nlJAF5r41cbCbj9aTmpGeBh8mt491AlTeYhm0dIPt+oME0BLIgPCVAC2WmayTQgH9spLY85i5/0QdutOimLqQoKNQJ2qjkEALVaFblrnFzq4k7+ZndGNdoSw14Xhqg4P967vpaMy5NFLd+CC6cLJ60SWfjv65ZSxC8FGbnLoXK3JR2/2FBmv1C+96p25jQRAyv644k0YvuWbhvjT+Jw2Q/RHhhP3eUNlqR7g3mZjjfcfg7elRZ09ACeMOZuJx4X99FSVT4Nzoqq9uAmljiBcanW36Hrp5mBmVr7nAyR9a7c+l0gN8RhrxCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=itTu3oZc5MzNRF0hPw+x0ZHFualAlO0fWetEbSE/YRM=;
 b=S30ghGvivsQP+PT8AZ8BqF62/hZM3ab6K3JB7GA91DVfbYKJ5B/5Bni/nIlAhbpctz+ATeOx9PCKKRlXCAW0I/uYurHbBZwZV5Vcly2PiLCnl2dAFG+btuBUq0dQAoJtycvkvKQ1BKPkzprbDBIJuPSt29trtmm7vcf1+9SlOQ1ALLPsqp8g9mIRErRT1s8Ez6XcFh6oQ0updPxArr0M4LFlgnSai6LttYy364SutWQanFwojebtD5dd8dWYnByynZVYtkM02tiVZUJKLIbZwj2CcEpqDCTXQKWavwLnCizQPFT8VZWhxgjgcsw+M4uRLEgCauHlb0R4mo5yM1XMlA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/4] IOMMU/PCI: respect device specifics
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Date: Wed, 15 Sep 2021 11:11:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0112.eurprd02.prod.outlook.com
 (2603:10a6:20b:28c::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 32e84125-85bd-4f90-4d1c-08d97828c4f4
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6862D08096D78431C82656DAB3DB9@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Hw9ItL3XIFX+3/13ifd5/MzUgdvgtumoT5oznJLF8fHF8Y3+561/Z+ZG9vnzJ/NHT2gIUL7SvJTc5kaMcu1wZmVhB6rTz25QFj5zWP4LQHJKpjl9RkQ7MTjy+FZa/SMS1N2T2f4r9yuWn+meZSKrurA4GJLBE64YWEQbxMi0T7hKTvwvQBGXK6oZhvshVsVm2BS75vpKv3gTrhPD67hFok/dI/0XkiOHI3RsQ441fPYAxxCd8Xfo5FIjXsK7xZEDc25SBPKw/MHcNMUA6UoBD6pc1c7YvpFa8PI/C9fM6ktnqdiGaK65ZF++a57JsvUvlCkyYncDLb8jFsNdlemfcI9QmLylvOzdPFWDF87N/C+YY8/hFpyueG8KX3yvT9QHHFJXpwIDof+t71+gg4OFYxPRh81V/puANFIs0YNnS4tGcj/smGiaYGIz9JqAlTmRJBvvIfMFM5eIT0/2h+QNg7OtdEA/TKschp1ad4lWjEhdF24OuY16g7wY/1boIVcIHB3gI/gkqB7j4shX8THmXNTD+/AS+LeeOi3kXxhahoXpYx710xoNzTsjo+Cg6VJW3bcBVngtBuLCNYftolWbwBc3vc12bZH6szccmNfu17121J15JioPAeDKHv+bZWKzU170iSjWUCmQy6asoiRUG4KiUZTSWb/ySqHOxEre/Tg/twNv705HrLMFJIBB7vga/8df4fu4OnB28xI9VRIxnyc6K7DFa3rmpSYtQ6bmQIXhDTiiGIctFK9/3RnJdXvV
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(376002)(346002)(36756003)(956004)(2616005)(86362001)(83380400001)(8936002)(31696002)(2906002)(31686004)(4744005)(66946007)(66476007)(4326008)(66556008)(26005)(5660300002)(186003)(8676002)(478600001)(6486002)(38100700002)(54906003)(16576012)(316002)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cUhVY0ZVc2ovdk5sUGo0eHJHcCtTaEVKQVJ0dEdLazNCN1R2eWw3Z0hRRWxZ?=
 =?utf-8?B?U3U5c2NsTUZXclZaRzBCQ3RSRFA5VnpWWlkvaWcrRHdqWURCdlpXd1p4bTBq?=
 =?utf-8?B?eUdkcDF3N1o0U2FRSGJRd2hxRkMxUEVPc2ZCZktWaFJUQktSQ1hRVWwyaG8z?=
 =?utf-8?B?TG5XZFUyTkduL3pCbHhOM3lzTllPTm41cVo3Wk01RzA4U212VUtaZUlNbXNL?=
 =?utf-8?B?SFo5eGIzUDE3SXR0RDN0LzFaMG9kQ25ES0tVT1pnNGhvVExwM1lsZENOM0pR?=
 =?utf-8?B?alVLMVVVc04wRmJBendRZHJOVGRhWWVzNE42R1A3cmRQeXRHQXVvTG01OFd2?=
 =?utf-8?B?VkVlU3N5bGpaclZmZ2t1d3d3Mit5UlNuQ0wzSkR3bmRFMk84ZiswK0ZBOTh2?=
 =?utf-8?B?MnU2NmNtU25wOG92cUV4Mzd6eEpmZ1dhclh0ZVZFeHAzbzJBYUZ2TFdwVDRh?=
 =?utf-8?B?YXZ1NVo2ckhFSlc5OFFqSGE0WlkvOXpxek5YR3kzOE5ISHRtam5vOFF2Nm8r?=
 =?utf-8?B?bHBmaGlzU2JxaGZGMHJsd25CY2dsaThZSCtYTUVLdUtpckxMZVF3eDRhK2lO?=
 =?utf-8?B?UzRaemVkUFNOSC8rUHRmaWsvQjdiNXBZL2pIWlp5UHJMT3RHWWNHaDJqVHpY?=
 =?utf-8?B?dmM3dVZYUk5tdUVzZkwyaUtOUnZMMloraVY2TmlWVnp4T3c3T0RHRE83SE1V?=
 =?utf-8?B?SXhpVTVUelNZZ29LRklvKzIvNC9Ya3c5aE9nUkhZODQrbWlETENMM2tiZDU2?=
 =?utf-8?B?c1UySUJQSENiRk5NSVpJa0hpTkNvQ0Vwai9hZllGOWJReHMxWDJQZjR1SWli?=
 =?utf-8?B?SzlvN1lCK3FOQm4xSTR5VGRpVklEd2RORGFxQ3VYZXpMU1ZjbGRCbmUrOU85?=
 =?utf-8?B?WHR0dzQrUUNsb29KT3hHZmRvdHZ2TTJoQ0xoWHp0OEtpN1ptU0luakRzbVNK?=
 =?utf-8?B?ZnBQQUdjSVhwV0NFTWNWTHV5Zi8rUlJVKzJHNTduSmZOaUtGQ3NyRlhtREIx?=
 =?utf-8?B?Y1ZvLzBkUkhaRUNoelJ2TXJYYkxPVkNNNU1qZDZuUjdWdXZxRm4zNTNjZW9a?=
 =?utf-8?B?eHpYNDF3Y2JtWjdFL2I4ejhLelNQd0Q0dlJucGF3bWtKUUxDWHpTaEVMZXdS?=
 =?utf-8?B?YVAvbVk0UHFtYWlYN1NJKzJMVGFHSDE3ai93bGkyNGhlZTJCV0tEY0hxeFFS?=
 =?utf-8?B?d1FtR3ErYktGblJmU0xrVlVicGlGZkVheGgxZUFrMVQvbS9EV0ZjWnl5M0Jh?=
 =?utf-8?B?d1N2ZHNpeVZiaWlWMjJKblUweTVsYW1DUzRucHJjdUhKVkdUYmZ4eFltSHFO?=
 =?utf-8?B?MTd6ZklDQVJzQ2dITEVmWmhIUFNLckNmV0YxZXdSVnVjUXhSSjRrcStJSzJy?=
 =?utf-8?B?QzZIM2FkTld5K0xPS3NLMENJcmNrTmI0QVd1Y2t1MjVGNi9Wc1RtRm10NzZG?=
 =?utf-8?B?M09qYWN0QzBWbzBEUDhudzlpTm93V05qMGozSlRIRkhqVE1kWTVQUkRGZjdV?=
 =?utf-8?B?NnBmbXhscHQ4ck9pTkFya244ZHZoTjNBbWZLbk4zQ1JIbFdySGtzRGV5YjZJ?=
 =?utf-8?B?S3hvWEVqVmlWQnNJUWtSZjRKKys3ZTByVUp0SEhzc1p0WTJ4RWRac0pGRkJ0?=
 =?utf-8?B?eExSdDZlb2hDZzhvajZKUnl4TXBmbE9VdkkzT0dOemZDSVllRmJvUlhKc2Ji?=
 =?utf-8?B?Um5KZk9SUWNEKytXeDFNT21FKzZEbDAvOFIyODJlRnBHRHdHWU5yREdXc0l1?=
 =?utf-8?Q?8vCgsv65zcE4KmMWBsFCt0dp7dtSSjEjeAU/MHA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 32e84125-85bd-4f90-4d1c-08d97828c4f4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:11:14.9264
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8jitjNEf0jHX26jCAGbr3medho67ZEnfq9Nf0MPt/qxkTG+uIqrzBRXd2WnwBJZpUrg542JDUtns7Z26F+3iFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

While making the first change here it occurred to me that the recent
vPCI-related discussion about hidden devices has some relevance also
elsewhere. In the course I then came to also notice a phantom device
related quirk.

1: VT-d: defer "no DRHD" check when (un)mapping devices
2: VT-d: consider hidden devices when unmapping
3: VT-d: skip IOMMU bitmap cleanup for phantom devices
4: AMD/IOMMU: consider hidden devices when flushing device I/O TLBs

Jan

(I'm sorry for the resend, but I thought I would better add the
previously missing tag.)



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:12:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:12:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187373.336209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQxp-000852-Pd; Wed, 15 Sep 2021 09:12:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187373.336209; Wed, 15 Sep 2021 09:12:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQxp-00084v-Mh; Wed, 15 Sep 2021 09:12:21 +0000
Received: by outflank-mailman (input) for mailman id 187373;
 Wed, 15 Sep 2021 09:12:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQxo-00084l-4H
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:12:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 88ed0497-ce9b-4681-86e7-8b084330d285;
 Wed, 15 Sep 2021 09:12:19 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-PV71D0OkNracoxx4ys6C6w-1; Wed, 15 Sep 2021 11:12:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 09:12:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:12:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR05CA0035.eurprd05.prod.outlook.com (2603:10a6:205::48) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 09:12:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88ed0497-ce9b-4681-86e7-8b084330d285
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697138;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=V/tbiUwwg7Omg2atvKwkwb61QSkxuOMN4oMJkXQ5UrY=;
	b=AgkEgSA3u0KOLRp/wV82xAP5lKnDRHEMflkNJ/pQLF2g4IvNq6Ws+LbC2PBJSdaV+z0QQq
	Wj4uR7I+lOD54Pbzh4TJU1dZ2H93pwqPTOqlxgu9iZlF5+jKgeWjwTJFUH/MC/z2hnpZLm
	RIkZmh1WG5+tVL3SIJGT7mgcbopKX54=
X-MC-Unique: PV71D0OkNracoxx4ys6C6w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DoxJu4O3kBhUwA1+Al1FtezdmH+x/dkBK+PxRhrJbBcEite+YB5phhidKBdz+bfTtApAZnjMnIZMuFOVuKbKXr0b18Aa/DH5sNVzvIkRm7W2njqs77OobJeGnXi3756dsrIdbdObbEjWoMxLEdn8ypSEl0KKmJfz2uPSo/aNFvMyLN2cRUB9XPFI6rcxfNKktQUVTDnscqFneO8/6XhPOtXpq7IflxYFFIXPpTEA1+InsaCbXr1WCVYZnM7L1qFarVWvEJEX60FbZ+6Xdt7h2VRN0ulcicOxAiq3rm2vmUdKvGKX/b6LhbHr/2xXFGsXEitpUFsgBRE+OeSizLVTpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V/tbiUwwg7Omg2atvKwkwb61QSkxuOMN4oMJkXQ5UrY=;
 b=J2mI4pjQqc7dMjKuSy1Zx4GijafkfEK4O5znFMStR3qZPS8KZQJK4TzTfbKPn062R9phTgSarJngcD5YULpa9KDeGvAFmLSmgwgC9qVwJvoxgPx+0eeXm/aS87HZc6HUpeKy2i9FTizKHaww0gXcDqArdwR2xl8NQopBCyPPVr71XnPptjlZkfmOfi/sDEaFMPe+Ph3kRyEvLHNnVZ5CdE8rAiZ6MBWtUe5WqykLoS0wc8fNtBMYAzKpia3+pfj1OlNWWXrP0h/ZLIS+7ulXsBIkIrqA0kTcyz6w5JFKzqT5kath97El/O8rgFCvMLufXvviVItD7Bu4pELZ5NLG4A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/4] VT-d: defer "no DRHD" check when (un)mapping devices
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Message-ID: <afd4520f-7966-eec6-b1dd-9e0c12dc8836@suse.com>
Date: Wed, 15 Sep 2021 11:12:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR05CA0035.eurprd05.prod.outlook.com (2603:10a6:205::48)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fb7040f1-58e0-4f76-6c61-08d97828e998
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6862EBBAEEA3A876527242CDB3DB9@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8wa41EkcSi/iLP6H9l0G/u6Uz5E5snSIurSnGN2FoqWNkbcI/WtQRBH2i0iWqCgSGL4QFChsNGvuHDRVx5qWrVY3gUWdJfdKkP/VwDj7yG2amRgN6IXnL4LlJtnf+Xms2yB2IjwL0uUrkQgZkcv9ZL45gvpwTa91sb2AZqauOgrBhbKy9z3bf+4yxhzHZuiRwI4LGXVCyrNJdcESpmqyPyw8gpgEM3meviYPisvWtTTDSpeISHe0os/qMgqBN+TN7tzyrCEFP6D2SVS8n8yBuPuuujMeaWfOVXIZjGNVc0M7nf6PhLG5Kb4TEav5yQzv/Gq7tRHVjyp5rOPP6+RQIpiQSSttshLPPgjz8ebmq6J0eqGIa+lD407bWRovLTSlLzRDPLM0y3vypWGCCh1SDmEvhM7+8E9GAGRQ8+l9PrJ4bqNnqa/+X0Op/fGFXvCabl30C0Ma4jYmqnmYynR5JIfQrzeBsT5mWZB3/0oKcMVEx/rJEGe128I58luJOWheJC65q4T1l7x4xyPrin7hl9/dEscZqOpZeo50an/mI6+MjxaQrEVoadgSdNH2xjfg8alKftGhWDvhsk+kh72/aAv2Wm0GLircDIsThj7HcJ2aDTTCfBqVXD2fCksNQ1JImkyVOSUmeJ178mRq7HlBcfvopfjzSVMX6mcFd6qaepEgIcLyVoX/PJnnXsWQXvvzNUOuDunfhFklB0gWCLfG70ZLbVa0JCzPRhUN7Hzx/cq6YljfchdalKJKU5OIyOhp
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(396003)(136003)(376002)(346002)(36756003)(956004)(2616005)(86362001)(83380400001)(8936002)(31696002)(2906002)(31686004)(66946007)(66476007)(4326008)(66556008)(26005)(5660300002)(186003)(8676002)(478600001)(6486002)(38100700002)(54906003)(16576012)(316002)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dFZ2N0xNVWpoMWVlcWFZeUhlWW5Xem93QjR2UzllWDcxZTQ2dUthUXYrNWNh?=
 =?utf-8?B?VXRicGRLVERRZUd1YUx4VWh4S3Nlb0pUckg1UEtSM3AxajFYbERYazZzWHJK?=
 =?utf-8?B?UElLTHQ3WGVGbWJoUHN0YXk4K3hiSmxkRHd2MjdnWmtmQThNRkw5cXcreHoz?=
 =?utf-8?B?MUlJdEVHeUhHM0JMakVGQ1EzSnkvVnZBQkdRQ1ZHUVN5UG4wR1o0TXBEcS9m?=
 =?utf-8?B?akYwN3ZBWXo0RjBUek04N09ORXkzNVcxWjJYK09qa1VnLzRyZC9MclJNVzY2?=
 =?utf-8?B?WXRubUxQdUQ3K1l4TUQ1eFRaUWk3TVNpcVpKTDl2Q0l0aW1uNE8zUjFCTFA3?=
 =?utf-8?B?OGdITVNib25leEloeS8xTjdNbWtMZ0l6Skw5Z2ltS1dTbS9DcDNYYmt4bFN2?=
 =?utf-8?B?UnJ3bFRzUTE5RW5HbTdndnF4U1Nmd1pWSzZ6TlpBcHJlQXJRSHRaZ1lTU2FJ?=
 =?utf-8?B?djR5SDByNTNpTFg5c3dLbmxkQTM1KzNzS05GNUFWSThVVEhhSGQ4bFdSUjNE?=
 =?utf-8?B?K0MyOCtnNnBlRWxrQUM0dzZuZllWdnhleHhmZk83djY2dGErOWduVDBJUkds?=
 =?utf-8?B?dEZmOVlzTVR6RjVvRzE2bGwwSFlNUVRTZUJtZm5CMzFSdytaWkVmOEhJWVI0?=
 =?utf-8?B?YWg2bXI3clNXUG4rOGtmNGY5dkFqT08xN2lteVl3dk5ZbDBhMHAzU3p4eWo3?=
 =?utf-8?B?ZjhFbjNNa0VZVWxLOUlQL004U09Wcy9MUHpIYzlrK0NRcFVERkdsdnIyaUl2?=
 =?utf-8?B?cm9HZWN6cG1rc1NMNGROUmE2VU9oY3hhQjc1YUtFV0o2dUpkVUdDY0JrR3ZQ?=
 =?utf-8?B?MHBEK1piU2tYbzFkakRsV3h1YitiQzh2bkJEZ1BQU2hETzJHUVdBVm0vTWZO?=
 =?utf-8?B?M3g3NCtKWXhiUHl6eXJ2QUNqUkk4bzNZbVhYN0lHQmYwQUk5TlhnZktWbjVu?=
 =?utf-8?B?eGFWdWFJNE1DbUROYzJJODM4TzV3TUpzdE5ucTRXQlppSEQra3ZCelhiWnF3?=
 =?utf-8?B?aEdvMlBkeFpkQjBUMTZtMHZVNndjeVBLaTg4WUxUVUZIMW10dGVvcG92Ylc3?=
 =?utf-8?B?UnBVTlc0L1JoWUFxV3pVVDl2NFZ5TEQzWU1zSEwxc1VKN3ZWZ0RjVEFDbDRJ?=
 =?utf-8?B?Y0dBZ2NHc3NYZzFJTTIvQ1I4dUF2bnQ4bG10OU5aOXZGR3V0RUhvVjNvQXBF?=
 =?utf-8?B?Q01mRmVQajh2Nm9UeHRxTG5WOXFxOXN6RUNHRmNQQkk0d0hHRS9DTUY5TkZ5?=
 =?utf-8?B?RmZhZ2dzcGJhdmxNM1I5NVFtaHB0eC9XMnl4OFVGcjdjeSttK0NGZWtwL0JE?=
 =?utf-8?B?WkZxb0xXTFFCRUI0b2RVQ0ErUkhpdnZqVTl1bXVqQXVHZHZqQ25ISHdiSWcz?=
 =?utf-8?B?OHdoUTNQMVgzQVU5MlVmd2VjRi84N2RlTE45S01Qb25tdklwTmhOVmErc2VT?=
 =?utf-8?B?MTZ1aCtFdWVhYjB5WjFZd3V2aWR5ankvdG5LL2pNQ01zOHNCTUxwSzY0R0FF?=
 =?utf-8?B?bUZMU3ZyN3dPZSsyR1VneTVBSHRjbTB2SmRESWxUOC9yZUhINFdqZWljNmFC?=
 =?utf-8?B?SDVnWFljb0F5b0N0R2J2Ujc0cGxhMmtLY0FZVy9ERllxaWJrQmh5L0l5cmRG?=
 =?utf-8?B?Tld4QlZ6SzkrNGpLL3VkbGVQU1d4Y2k5RGJaMGN0Q05GQWlTY0FnaFJMU0ww?=
 =?utf-8?B?SXVpU3RCUm9jd2Y0QlRwTWlFWXZvTjZmd2xKdnVwOWE3N1ZNZlJyM3ZWVWZE?=
 =?utf-8?Q?jEYy68EGI7J04EVJU8XUJdKuq0Tmv2CbY+NmA1X?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fb7040f1-58e0-4f76-6c61-08d97828e998
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:12:16.3550
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2C3DQbQQUSfj381+o7dsR+d52WpPrWN89NZu0BYtO2gTmTEArHDPG9peT10NAlivFqKlTE4cQsdZwJ6/XcXw4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

If devices are to be skipped anyway (which is the case in particular for
host bridges), there's no point complaining about a missing DRHD (and
hence a missing association with an IOMMU).

While there convert assignments to initializers and constify "drhd"
local variables.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1460,14 +1460,10 @@ static int domain_context_unmap(struct d
 static int domain_context_mapping(struct domain *domain, u8 devfn,
                                   struct pci_dev *pdev)
 {
-    struct acpi_drhd_unit *drhd;
+    const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
     int ret = 0;
     u8 seg = pdev->seg, bus = pdev->bus, secbus;
 
-    drhd = acpi_find_matched_drhd_unit(pdev);
-    if ( !drhd )
-        return -ENODEV;
-
     /*
      * Generally we assume only devices from one node to get assigned to a
      * given guest.  But even if not, by replacing the prior value here we
@@ -1476,7 +1472,7 @@ static int domain_context_mapping(struct
      * this or other devices may be penalized then, but some would also be
      * if we left other than NUMA_NO_NODE untouched here.
      */
-    if ( drhd->iommu->node != NUMA_NO_NODE )
+    if ( drhd && drhd->iommu->node != NUMA_NO_NODE )
         dom_iommu(domain)->node = drhd->iommu->node;
 
     ASSERT(pcidevs_locked());
@@ -1497,6 +1493,9 @@ static int domain_context_mapping(struct
         break;
 
     case DEV_TYPE_PCIe_ENDPOINT:
+        if ( !drhd )
+            return -ENODEV;
+
         if ( iommu_debug )
             printk(VTDPREFIX "%pd:PCIe: map %pp\n",
                    domain, &PCI_SBDF3(seg, bus, devfn));
@@ -1508,6 +1507,9 @@ static int domain_context_mapping(struct
         break;
 
     case DEV_TYPE_PCI:
+        if ( !drhd )
+            return -ENODEV;
+
         if ( iommu_debug )
             printk(VTDPREFIX "%pd:PCI: map %pp\n",
                    domain, &PCI_SBDF3(seg, bus, devfn));
@@ -1651,17 +1653,12 @@ int domain_context_unmap_one(
 static int domain_context_unmap(struct domain *domain, u8 devfn,
                                 struct pci_dev *pdev)
 {
-    struct acpi_drhd_unit *drhd;
-    struct vtd_iommu *iommu;
+    const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
+    struct vtd_iommu *iommu = drhd ? drhd->iommu : NULL;
     int ret;
     u8 seg = pdev->seg, bus = pdev->bus, tmp_bus, tmp_devfn, secbus;
     int found = 0;
 
-    drhd = acpi_find_matched_drhd_unit(pdev);
-    if ( !drhd )
-        return -ENODEV;
-    iommu = drhd->iommu;
-
     switch ( pdev->type )
     {
     case DEV_TYPE_PCI_HOST_BRIDGE:
@@ -1676,6 +1673,9 @@ static int domain_context_unmap(struct d
         return 0;
 
     case DEV_TYPE_PCIe_ENDPOINT:
+        if ( !iommu )
+            return -ENODEV;
+
         if ( iommu_debug )
             printk(VTDPREFIX "%pd:PCIe: unmap %pp\n",
                    domain, &PCI_SBDF3(seg, bus, devfn));
@@ -1686,6 +1686,9 @@ static int domain_context_unmap(struct d
         break;
 
     case DEV_TYPE_PCI:
+        if ( !iommu )
+            return -ENODEV;
+
         if ( iommu_debug )
             printk(VTDPREFIX "%pd:PCI: unmap %pp\n",
                    domain, &PCI_SBDF3(seg, bus, devfn));



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:13:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:13:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187381.336219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQyU-0000Eh-3T; Wed, 15 Sep 2021 09:13:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187381.336219; Wed, 15 Sep 2021 09:13:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQyU-0000Ea-0N; Wed, 15 Sep 2021 09:13:02 +0000
Received: by outflank-mailman (input) for mailman id 187381;
 Wed, 15 Sep 2021 09:13:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQyS-0000EM-Lk
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:13:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1ed06cdc-1605-11ec-b4ec-12813bfff9fa;
 Wed, 15 Sep 2021 09:12:59 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-KL9Vz8YcNlqdHYbQZdp6-w-1; Wed, 15 Sep 2021 11:12:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4605.eurprd04.prod.outlook.com (2603:10a6:803:65::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 09:12:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:12:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR05CA0009.eurprd05.prod.outlook.com (2603:10a6:205::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 09:12:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ed06cdc-1605-11ec-b4ec-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697178;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/83ivPXoMWZK0DfJ0iAElyITmi87CEwTYL/CkVUUj8s=;
	b=Cc0S76Pgw7Xx9WhPjKbIeA/msdNf8NxNQGHTO7a11i9L8RwXMObvDT/0ztRm7/ozxrtDOb
	qbxUBAqCifmDTyJnDjwzzL9KCKwZzAwmVQ7Hp7QQYtsYoY+g+38XWjiBFQYMx7QcA64k9W
	NXMCoO2Jendr9ON3Plgn1rEqvn6VvdM=
X-MC-Unique: KL9Vz8YcNlqdHYbQZdp6-w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fmo4aJFhHb5z7pPEqc2GAtj+S1xtd9BW2LEyOEttEoofkxpSIIKuXUmDTYdjKM6DlCt+ewJjitwqyxIBX0AZQ+kaQHWXPA3Y+bsQzg4Rk5sFfUrIESGwG8pWM1ZLgGSJy8jgsr9ho00oaApM49u1YVHVCGNMc1X8jvZPBjT0KMnUwdiU5jO60nkM/wTBqyBFGlrrvlLje4m2N5h/jUxhOYOiJ+GF54jqY9bdzn5+4uXkFVOI292bFvXazUnBGQlm6VOHkYOLzi6cBLee/Yxs9GKXIV4pa1ulVQH+HFzkE3L7pj8noWUQX6pk8GgsbxHCTfEOFISA0W6CLXxqhuyc3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/83ivPXoMWZK0DfJ0iAElyITmi87CEwTYL/CkVUUj8s=;
 b=BcSHQKEWuqj/X9b6xvanuDEmFQkUsmYBodHk4Hh8yr3AFlFD13Cg71CGZetPVibgIPtRXCGiROsa8m7TAVG2unwu+18c/NFEc2rm3BeDwxAjMWALtfnNfyTjNKstHk/ko9kwaVLCSycyPQGGbxtR9lCNYZs+sH1r/Uo26wvgh4wsGAEinfZcrJQoKa9uBxP+GLN3gYCi1xzxDFdcE51G3JQjXwb/lhEr/UjGKu8WERPI82EFz0p4KzDdTrSiHtREZa5Da7KgrK4r7BfzzohZFGWJA7HvDHFQUVE9GO9BRWwEX3hg8sjObs4c+yV6dZyVA0xrbQ4Os9/Lzr6imWu1Vw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/4] VT-d: consider hidden devices when unmapping
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Message-ID: <5cf392cf-09a2-b23c-2a25-610c64f20707@suse.com>
Date: Wed, 15 Sep 2021 11:12:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR05CA0009.eurprd05.prod.outlook.com (2603:10a6:205::22)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 642123a2-c9b4-46ae-6e17-08d978290152
X-MS-TrafficTypeDiagnostic: VI1PR04MB4605:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4605AD7B26D394D073B3A4A9B3DB9@VI1PR04MB4605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8PCAqCp7QC3Nmn0avbgNteqcuwg/I5Oi3mqHVHYZIh24OwBgi67u0vS8fe1AFKYsG9Cm/qRAqUWGx8bkHuSmluHZRvpKJtv6iZ0n3htwamiJjKPbNUf2F29WWzqxJKHMVdkMYtPi4V90FUCuLX0LtR9KXP3LJy3ktVN3NxJkHSFIitntcWXUog2MR35cDqpmP5x9vjeO7OGjj2MMMON+Wd7qhhITuhRVjlDGR4GQccDmgnxgcQXZbBAAQpO2cP8R/EFX+gDPL6HNECJnlf360DcTr6rL+1TBx8yVw9yNsnGK/63O5tPBM1f0XZq4zpEYJrkQHwXMRpviKEsXmOL3LWMeaUfqXGGA+a89RCi/ONiPilD8SGTt7pCRCx0ycERy4mTodkGPoqovxKHSEJIaPtf7E2nWuZbh7snSkyxgo9cm2n2fMTwpNEfPbWJp4zW1JdOILUAa+qp3bLTcKi9yuyMnHuXL+nfZV5ubt8zpcWms1jmxNdgczlcKhsMb/D1XhfuAeaKmPZg8VX5VvzRyennMd3rb4u/J7ZIvzGsV5sfNh39ghStO8YhHBBfhFqdrhHXUFwDeDHWKh5vaI5Qd6rUwsxKBReHq1c+2Lkhz0l3rSbmnf2V/j//Ami4PskCFoUGSLvXxE2BUtvlua8nmOkP6QvfPBk3id3gnrFqKR3SSQLNjSPbTgucC5O4dGtBBkew9bqNYTwPOvBw0ZOPPc1D4yWguv3v1fZqie0I8N6Q=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(136003)(39860400002)(396003)(2616005)(38100700002)(5660300002)(956004)(86362001)(6486002)(478600001)(66946007)(4326008)(316002)(16576012)(6916009)(54906003)(66476007)(66556008)(31696002)(186003)(26005)(8676002)(2906002)(8936002)(36756003)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L29zdmU1bnpXRkkwZE00ZWoxNS9UN2FWYnJpL3VmTVhxRjBmVklqRmU0OFhF?=
 =?utf-8?B?Z2IvZ3JtZ2VjenZPQkNJNnJKTG4xNW9SQVVObWFFVHdnODRxem4ySjc4dWVX?=
 =?utf-8?B?M0Y5Q2JvWXNiM1RtNjdtMzE5TklVb2UxL1hDWVdzaFhjcFpjc2ZRNUFnSHl6?=
 =?utf-8?B?UTNraHdZL2xmUWZzRTdxS0FUbnU3YTBLeVQ5V0RiVWZibUFqS0U3aDE1eisr?=
 =?utf-8?B?N0Jtay9ZSHRyV2E0WU10R0lmZ0RVNy90MkF0ZmNVVjR3ZmZIM1BJaGFTcTRl?=
 =?utf-8?B?VFhsMFNPWlpTMm4yTUl6bVFwN0QvbWNQM0s2ZzVUWmkwRlhrK2Qzc0VkbFBI?=
 =?utf-8?B?ZWRXZGswWmtTTEg3eGl1bllpZXFjSGVyeDN3SzhXTUdwNnQranN1R3l6bDFr?=
 =?utf-8?B?NUdBVWlydE5mY0RuNzVCOFByZUw0aE1KbGdjYm1pV1dHSHA1aUhHblVpV1pt?=
 =?utf-8?B?TkduVkFER0wrSDR6WXUwU0ZVZnloenk0eDRLZXBITFViSXk3S3ZxTVZGN3Jq?=
 =?utf-8?B?L1AzRmQ0TEwzbmczT2N2ODlYVHVSNXN3cGFjdEMrUURiT0VIME1Kc3RSUjFR?=
 =?utf-8?B?cU5VM2R4RGt1OTBDV1hRV1BlOW1GZ0w0ZnVjNmZKUDdUcVdTYTdFWXd4NW1o?=
 =?utf-8?B?UGpuTXRxZG93SEsrazlTNVh6cDNYK3JYTXZ1eEFVY3dUTmt2NVpRT3RGSDk1?=
 =?utf-8?B?dmIrT1NGdW1WSlZqaXkzRHlVRFA1SS9LUnR2MjJZbGQ5M2k0TW15ZXAxMzB4?=
 =?utf-8?B?UmMvclU1SldkL0pnWEh5THVFZDZ1cG9saUN0SHhHc1cyY3ZlVDJKL0M3SEJj?=
 =?utf-8?B?S1JZWm5sTFhrUGM4Qk1Cb0wyZ2lpemw2WHdZeW85S3Vpb1FWQ2JKRXpwOFlk?=
 =?utf-8?B?OE00eXdFWUQ5aVhNaVRPR3VDV0FReWNoQmFFaTQ0TlJLNjJhYjdQY3FOWTBN?=
 =?utf-8?B?Z25YZXZtZU9PRmtvNllSc1I0N1ptUkszcVFjVUJtSE9PZkZ2WnQvRVIzaERo?=
 =?utf-8?B?cFREOFV0TC9WSm1sSTNjUWxEWEV3QzhhVWdaREh3Y0I1RVM1YmprQlJ1dWlE?=
 =?utf-8?B?ZWtmRXpHakhmV1lyWkQrODIwRE4vOGVxYWhCejJSM1owRFlYbFVONUtVL28w?=
 =?utf-8?B?Tlc4ZEJtSVNpLzRpVXBpUXAzSlNnYk1UYkExa0JmQW5aSjE0dlh2SWlZYmdG?=
 =?utf-8?B?SFdZV3dTWjBZd05qS09LMHEvRkdPQ2VnU1hGei9NWlpDbnNQWk54T3liQWlk?=
 =?utf-8?B?ajVWWXZZK1grTE1lWXFCMThqeURicThMS1VYWUg4VzV5VmVObG1VSG95RnFq?=
 =?utf-8?B?b2d3MFYvMjNxd0lNT0tvMjFwVUFrZFlCWXFwV0ZSdlZwRlY4aXR3WDhqUTVo?=
 =?utf-8?B?WnJJbmtmc1hWamduUlNROTZENTV2TzE4ZU5kZVlEYVNyWVdMa3RCaVZoSHBZ?=
 =?utf-8?B?N1FUSnUvK2hsNE44NmI2cDU4LzBFOVgvMTQwUU8zTmFkbTI2a0Z1UmxMME8x?=
 =?utf-8?B?dkxqUnVuMFZXbFpSSzVzU2xsUkpUT0wrbklYV3lwZVdIUjZBZTl2cnkzYzE5?=
 =?utf-8?B?dXViL2prejVwdGw0T05TQytBSDV1S1lBUzlPV2RuZC9nVmRia3E3WDNFVm1I?=
 =?utf-8?B?UXV6bkhHMVV0dEFQYzRYS2Zka1QzUFRuaURPa29sZFZxNU4zWE83YTVheURU?=
 =?utf-8?B?cFk0dGYwSGMyYVZqTFJ4dHlRMjU3SUh6YmVqQU1DbHgyMEhUb2R1elZTZG1y?=
 =?utf-8?Q?xhsmtLPiCL3N1nmIB6wjIPadBfuAJL8ogfBrFj9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 642123a2-c9b4-46ae-6e17-08d978290152
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:12:56.1517
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eZ6/Of8FrrXKMK2D3Sd0BBq9yCilaEf2KewdtyO7iSh17n6kJ1v2pJFvEjPJWEAAm9TBOm1i3gRktrbHfERjyQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4605

Whether to clear an IOMMU's bit in the domain's bitmap should depend on
all devices the domain can control. For the hardware domain this
includes hidden devices, which are associated with DomXEN.

While touching related logic, convert "found" to "bool".

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1657,7 +1657,7 @@ static int domain_context_unmap(struct d
     struct vtd_iommu *iommu = drhd ? drhd->iommu : NULL;
     int ret;
     u8 seg = pdev->seg, bus = pdev->bus, tmp_bus, tmp_devfn, secbus;
-    int found = 0;
+    bool found;
 
     switch ( pdev->type )
     {
@@ -1737,23 +1737,33 @@ static int domain_context_unmap(struct d
         return ret;
 
     /*
-     * if no other devices under the same iommu owned by this domain,
-     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmp
+     * If no other devices under the same iommu owned by this domain,
+     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmap.
      */
-    for_each_pdev ( domain, pdev )
+    for ( found = false; ; domain = dom_xen )
     {
-        if ( pdev->seg == seg && pdev->bus == bus && pdev->devfn == devfn )
-            continue;
-
-        drhd = acpi_find_matched_drhd_unit(pdev);
-        if ( drhd && drhd->iommu == iommu )
+        for_each_pdev ( domain, pdev )
         {
-            found = 1;
-            break;
+            if ( pdev->seg == seg && pdev->bus == bus && pdev->devfn == devfn )
+                continue;
+
+            drhd = acpi_find_matched_drhd_unit(pdev);
+            if ( drhd && drhd->iommu == iommu )
+            {
+                found = true;
+                break;
+            }
         }
+
+        /*
+         * Hidden devices are associated with DomXEN but usable by the
+         * hardware domain. Hence they need considering here as well.
+         */
+        if ( found || !is_hardware_domain(domain) )
+            break;
     }
 
-    if ( found == 0 )
+    if ( !found )
     {
         clear_bit(iommu->index, &dom_iommu(domain)->arch.vtd.iommu_bitmap);
         cleanup_domid_map(domain, iommu);



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:13:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:13:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187389.336231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQyu-0000t6-FU; Wed, 15 Sep 2021 09:13:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187389.336231; Wed, 15 Sep 2021 09:13:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQyu-0000sz-CR; Wed, 15 Sep 2021 09:13:28 +0000
Received: by outflank-mailman (input) for mailman id 187389;
 Wed, 15 Sep 2021 09:13:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQyt-0000is-5r
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:13:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2b7fb60e-1605-11ec-b4ec-12813bfff9fa;
 Wed, 15 Sep 2021 09:13:21 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-v3woC4z0MwWLON-TAE0nyA-1; Wed, 15 Sep 2021 11:13:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4605.eurprd04.prod.outlook.com (2603:10a6:803:65::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 09:13:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:13:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR05CA0147.eurprd05.prod.outlook.com (2603:10a6:207:3::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 09:13:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b7fb60e-1605-11ec-b4ec-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697200;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=32ebYhQya8W1RrUzbbQIk60a77c15WTRh4/23DDxt8s=;
	b=GEfoYrpzH6+wUCKjZaLEnGB/YHaK4jHBzfiovhP4TfWQbysmSfZficSntMm4aQEM4XJKKO
	aHd+mcOJ41EdGVoGkFYKnSaQbQ5BFOdcsFnAWy/Y3wYNQ726309+Wpa1Yb3mc3V38965J6
	gnK6hO+uqJ/kUIfG6lxz/6fqq+CELpY=
X-MC-Unique: v3woC4z0MwWLON-TAE0nyA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GwRwISbZXLP3toO46IJntMnZTALYvIh2Nj/kLOpH1vKwiqSmrh3NQJonKMwv4QTmJEucidi7aaKQqFlKC7n2adKKQVWLpsW0Y8a4/AhQEtXmBEcMk656rcRKvnvj48F/LBqWI6XLJpk/GVQS5Eyqj2rgizBhyE1o/v9ezqqqKzdIgpu6H4HjCodNdChcWrkyo9oAKylKRo5x6CHEyItfcFKOF7RLR0ZrjjYxzlAsYIxEHkSndiqlsb84FzbASCp9ABL2NeQOTTdHWnQUk7+F1Y6qFdS/noupRXjaHGR8PLZejkZIto1lASP4sfJKGP2cIchQuiWuPZBBSTJ737YVag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=32ebYhQya8W1RrUzbbQIk60a77c15WTRh4/23DDxt8s=;
 b=TJORSQ5y9HVkQeOC/H0aqi91f4KSXzWYzac5VRxRMDr/8J53fJwg+nJp8qwinmlt5m0W241SyYpo5F4qFn1HMpY6bxzq8ULOtNRiHJJSQqbYY9Ba662zYaa+dK0ZbEy+38s/H+Z/LWdkfdICjMjldSPH21t8ebrtiEcIS2zPHRP1SN6di7gcUeOK1r40AQOKuNgs7n3qezb9210s7OO94cN34qCxdF4t7TV3VUkotCO8pTCY3saO7eXqZRmShKNtm4/Hcnyq0vxguUIb4OXZSTJ5348ndN2EoixrFzOTGDx6YC9bxBdcul3+CyFDk3Tz8CeRIOXlEfP6Hw2eS5lF4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/4] VT-d: skip IOMMU bitmap cleanup for phantom devices
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Message-ID: <57c75e32-df31-c886-b2b2-86d0f82a57d0@suse.com>
Date: Wed, 15 Sep 2021 11:13:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR05CA0147.eurprd05.prod.outlook.com
 (2603:10a6:207:3::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a1cf6ae7-7974-45dd-ea38-08d978290e66
X-MS-TrafficTypeDiagnostic: VI1PR04MB4605:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB46054FD28FB1E7C7A796BBC0B3DB9@VI1PR04MB4605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UcUyzyXHSPmodqNl3UsUYxFvt4tIaT11ttbfrzxQS4tdgIIFaK8HCsy60MX/UAEFCrwz4C7PEqJQQjD2BQ3XfwEOIFw8rleTLtZjfABusPXvcSmnzM1xfQKAzDU14/wllmhkYqU2rtONV9D1/fR8bmKDK59aX3eyQ8RPZTXP0z/ZXsfJavh6HLpx+IvM9tIq54djPbREzlKCAMqagv6B4VXlOGddrQ44cstZUy4Xrwakp7kadVN2vxz1DG2cbaS60ucOSMRrRmVoTYSbvBTfxfLK0ChkMJBVA4AkRMhlcBuesC/RiiEK8qy78Fg60yjfU6Y9NUdcR84dB8sgO/4jZOQ1MajonIt6uPwDgoSI5AKkU/4WchOCv7SBlAPt0J967rIZ4LIyL4mIzu4PthyqAhOsBwASa2zLBcw8E9sfgCFzl0oVAzhyNXzxXeiYqIbF+FySm8LXo42xmT/GIpDB8SSFXU2sbfu6qd5sZTt2vHXgIjsgIid3tRhpr9GjrfQZKHYYkKa4xKUM0O3H9n3xNVaYwPtawDWVPa8eo/uB11ggz+nFKEN+balQCzr3gChMBFNWfwhdFz0PXgWGUKchJMfQ++zvLwBx5m7J4QOI+FQXzSi+YnIf2maKXfd35Sv4vJQUWC74VLTQRTF90E0wxnnxF96CJa8KzAJIxNTtGF6y29i3PYv4LepMvIMUqHCErHrwdQIDDg+NWpU5EyMnnFNz0tfH60h3TaGI3BYH52M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(136003)(39860400002)(396003)(2616005)(38100700002)(5660300002)(956004)(86362001)(6486002)(478600001)(66946007)(4326008)(316002)(16576012)(6916009)(54906003)(66476007)(66556008)(31696002)(186003)(26005)(8676002)(4744005)(2906002)(8936002)(36756003)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UU4wc2hpT1pyT29STnp2bndUTm9iT0g0SFJGZmoxTGFNWDF3ZzdyZFN6OVNV?=
 =?utf-8?B?dFA4MCt1bGh5alBxY3dMY1IxNjVXYXFFUFRFUmhPakhxRURuTkRsalp0TE5B?=
 =?utf-8?B?Mjl2bXZxcGRrR2N5aHNpSmYxeWE5SDFaWmVQR1ovVDgxbmFyMWFvZkxKR052?=
 =?utf-8?B?MmpoMXlZM2pjR3FKTTNISi9vbUdzRDR1Y0F3elZLNWxtbUpkd24rODAzZHZl?=
 =?utf-8?B?dThtZnpsYlZsQUJRK2VBazRrd2hrcUw4N1ZvSlVKR1pOaDFIRlplQWY5V2Yy?=
 =?utf-8?B?cndRa0NGQWV5V0lQazgrdEFYY0tjTlFXeHFNTy9SUE9XTzZ2L1BFTG05bnQy?=
 =?utf-8?B?N2J2Vk0raXFBUmlXSisvZkVTOHNJWm9MOUtrQjhnSDYzOXM2b0JWSlRselY3?=
 =?utf-8?B?OHZJay9qRWJ1Ry9VUjZZNXR5UlZ3a2FMTUczcW5rUmQ2U0NFMzd5Y3lGSkJh?=
 =?utf-8?B?Mm5pVTIyNld2b0FXdWN6WlFVV0tKUFN0L0JDT2VjYTlpVExkM21sNWJsNHVN?=
 =?utf-8?B?dHVkV3A3SmtJOVJEL3ZYUDF0NERsWHp6MkRqZlhUSFZRUTNYUXRHTmI5TGhw?=
 =?utf-8?B?K2k3Ym5LU2tIZGRNdEZVY2lzUTNZM3RsRnlUdE4yRjdhOHVwUGQzeDc2d3FR?=
 =?utf-8?B?azZUbUNic0xFR1Q3d2puMTJCd0JIYVFqTitRRmE0elg4d25xWVhwbFo0QjhQ?=
 =?utf-8?B?dWZVQmMzQ0dUUFVWY08reDhvYjZvNWZoSnZSaHI4QTJuRllsUEhudEVMT2Jj?=
 =?utf-8?B?ZDJIVWJqNnBtM3FOUCtQWEFoZHVvVUJmMFNkWEVrbEJjaUZ3a1lXZFhGbkpT?=
 =?utf-8?B?bFhHN0lHQ0ZPcnNRRFVFbUZrdi9TNVpwdEJWa0Y5cU1VeTR5QTdGeERGbTVC?=
 =?utf-8?B?L3Q0OWFlaTRNMWladmpzcnRQQ2VVMnE3N1dYV21hL1dNQ1J3Tmlmd1YxUlQ3?=
 =?utf-8?B?blpzUlc5LzBXdjBMaGY3Zm5qY20yWFBQbjZMVmVuMW5Sa1lJc2dvTTNqc1Er?=
 =?utf-8?B?bk5WL0NpT2wzWm1ldDJuSm9HRkJwWUtaUFlYT0N0V0xBUjhZbEpvQUt6SnYy?=
 =?utf-8?B?SGxwTWZWdktUMzB3TUc0M25QSEd5VGJ1TWV0U3lZNllZL1BDV0trMWVjeENi?=
 =?utf-8?B?c1pLZ1FUUWxLRGN3VUFJUGlSeWplT0VHWTVoaVdVcGwzZXUwdUhOL09pczVV?=
 =?utf-8?B?QzQrajREOEtGR2piUkpTRmc4Um5DdXM5ek5kVFR4L1dQNC9zNlB4WVhNckxm?=
 =?utf-8?B?Ui9zMzZjVUE4VFFkUDBQYU03alRjcVZnZTVwdGx1RHJDTUhLclFGU0lVRWl3?=
 =?utf-8?B?b3RUdFZiQmZYUk51S2YyTHVSZWhzNllHR1NYZGJac1ZpaTZaazUwZm1tS3dV?=
 =?utf-8?B?UHBKSkZDQ1FkMmpTM2l6c2NwK0dBKyt1Y3JqczRUNURhU08rSDRMMHFrTlIv?=
 =?utf-8?B?Qko0TWRzZHNaLzVid2JMNG9EVlRJNDc5ZENOSG5RQmlkM2ZMWjNITlppY2dn?=
 =?utf-8?B?ZFNXazR5TzNvYmloRVFmc2tZT1BSZW55NkpzSGNpMUxXMGhubjZXaVJqZGFW?=
 =?utf-8?B?V1IwWHRvTzJacUd6Mkt6L0dHazZyejZvQjlzL0dkcUwwdURBKzIrZlZmUDJ1?=
 =?utf-8?B?UzJNdG9pcEl0VGpIZVVQQXE3UG4weU5UTVpOQ0xncDNIZEh2b0lhaWlVdzBT?=
 =?utf-8?B?cGRXTnNud2M0VHZubHhZYWcxSyt3TUJxTkVHZlR2K0R6SVMyVDE4QlBOMjRY?=
 =?utf-8?Q?qeCV2/ADKOEVhDXR/GrKYkasJ1NdKzZjYlceDb9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a1cf6ae7-7974-45dd-ea38-08d978290e66
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:13:18.0783
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UKtU3fci0huM32s8PK1/Er8KbcFWG+KyjxkpT3Qo5rSorEgdBFwf4v1xfamLYCyllnDTSNCLx9GaNhZCv2nMmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4605

Doing the cleanup also for phantom devices is at best redundant with
doing it for the corresponding real device. I couldn't force myself into
checking all the code paths whether it really is: It seems better to
explicitly skip this step in such cases.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1733,7 +1733,7 @@ static int domain_context_unmap(struct d
         return -EINVAL;
     }
 
-    if ( ret || QUARANTINE_SKIP(domain) )
+    if ( ret || QUARANTINE_SKIP(domain) || pdev->devfn != devfn )
         return ret;
 
     /*



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:13:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:13:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187397.336242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQzM-0001U2-Of; Wed, 15 Sep 2021 09:13:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187397.336242; Wed, 15 Sep 2021 09:13:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQQzM-0001Tv-La; Wed, 15 Sep 2021 09:13:56 +0000
Received: by outflank-mailman (input) for mailman id 187397;
 Wed, 15 Sep 2021 09:13:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQQzL-0001Tf-AZ
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:13:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7117321c-b748-403b-98c6-112c689792df;
 Wed, 15 Sep 2021 09:13:54 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2057.outbound.protection.outlook.com [104.47.10.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-bNj072P4M025VJlUB7dzEw-1; Wed, 15 Sep 2021 11:13:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7375.eurprd04.prod.outlook.com (2603:10a6:800:1a8::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 09:13:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:13:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR0202CA0008.eurprd02.prod.outlook.com (2603:10a6:200:89::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Wed, 15 Sep 2021 09:13:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7117321c-b748-403b-98c6-112c689792df
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631697233;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TPv3uK8bqSIgwLCsg7oLnpJv0fedDK5iubeaeJBmDWs=;
	b=apgQ/xiIRJ0zyjt+w3QK2sdsw5TXBCosQ/8YFdcvXDK85xuvpsEBCgAyj5pkU+3B4tEWb9
	B+/ncF/fpl+t4gPBTAcH9yRYO3SFhlaD+JuUoZ3goeIl85SCfdjazXBiLmW8oGGPc0yPoX
	Qvr17P+ZHAQiKvG1lgpWcKnGfhYC2kQ=
X-MC-Unique: bNj072P4M025VJlUB7dzEw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bCG3KgvFWoKRGLk0vzMIsx6uAGdCo0wHOGocck3QvFHmCkhNtZ7zCYwaXl2K6J8Jw06UrtvJF0tYoAEmv7vsvy5JgUERf/2+h7hKRbXX7b05fXqSnMV+vopRi6YTADfBbKZ7haX1YMPywPLNOgXOPeJKcrr5qgHmc1ZhCqjZATXQwh2J7GNuaX8+V39w+qVNRynAA0XWePZXpZLB10lX4z79wcwlLjKQmlYoOS/founwdUk39HG4rYqbPLIfaoXV/bPtKRcZrrhas53iFtx3ZWYlKQ6yI1ZdMYdUUxMLWpmZ8FyQbP4o5AyejB0m9uc7dIIAkMJ+h3CyRXHcUlUPOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TPv3uK8bqSIgwLCsg7oLnpJv0fedDK5iubeaeJBmDWs=;
 b=O3qt+/t6uVIQEydyIxxfTrz6Cc3rbvFgfDVJTbiFGxL1eKfkgsqy5bX+smjlsq4fUunGKtSoFG5M24yqZOoRPtYknB8ZJ4C6VyHza2tUcq1hVQcrrEsmT21G3ukj2VafqDIpSMVF/G3yOCPBgLDt8MNW+OnSF4QeMTgGLQZmGct8f57kyC/RW5hRdMUgx84TGnE2KNaqOjp+iIOsGGM3h3Clen0MMuHRjGL77+qFInmuScXEfKpiXqPvePA0G8DupMzqK38GASZSDUelItBwOihUZ22lZ1j70xUHrbfQ1gPSpupI7fCG3Gsvycv2DX1k8AZlE6uZ+MZoKEY+PRfdhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 4/4] AMD/IOMMU: consider hidden devices when flushing device
 I/O TLBs
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Message-ID: <f40604e9-f41e-4d51-011f-d507ec711904@suse.com>
Date: Wed, 15 Sep 2021 11:13:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR0202CA0008.eurprd02.prod.outlook.com
 (2603:10a6:200:89::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 32bb6961-04e3-4b0e-7339-08d9782921a9
X-MS-TrafficTypeDiagnostic: VE1PR04MB7375:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB737556DCF6F0A5357240A718B3DB9@VE1PR04MB7375.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DhtTAYESLKi3B+4UjjZ38SYd9YqjPIDhybF4lJHfWEq3bZmkcvioP22GQYzIZYkBgrolpfYOqtohmh8JYEv7ogwsQXdM0NdBBeijGDRBJnT6xq/pbbqzQIeDkELZf8u08XY0wTt2I1P76AXambUHr4EFuIno3haQrcQcou8hPrk0d/uox38zSt+/29TWCLw9Y4XlCXTgLT/X1w2KCeL/PyfmnySMc0y+O3GOS4+U7luLxudHZi3slhJ1oNDRxaOFhjmxhIGr6tyUAAhzeD9dtUk78gVFgfHVn3tFm/sKkUPLk/pcYtdJI6Bv/P3LvrisJGVgtlLqLcJ5GNoR1GhJ7cK1hsurKNirqkUBdv0w+h9O3bvm0Xs5CF/Ws+dgFSaTBlok/6C0efspCR5IGGET7qAJjvprWM60lVHDHhyIyT9kuUM8codxs6UZh2xBMa9BDE0o+FKsbXbhbmGoGwpgZT/NNLS4M0fu1Ri91Hg9gfXKDUhWXFaAV+/fMDbMMxgT2aYzt5UA/8n0eaeIf4xt36jt812Y9RmMdcERGi/R8mTyFeAs3bACXbYpqNVAegrDOmjVq0HBwEGtDbY4a19pUPe9Eb61pSCPnTYwsnCI/mIQDfOf7ofheFp2nVc6F+7XaGdM92L+uOrspEp/fZn2Lk82GZlNvZfuHJBsCufR5elWr05fLLCtMCLOhoN4DXMNSn0mn20ZnRE276u9uFLNtwgyWrFte3J7r0sHZAdoEJY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(366004)(136003)(376002)(346002)(38100700002)(2616005)(26005)(54906003)(83380400001)(5660300002)(4326008)(8936002)(316002)(6916009)(31686004)(16576012)(8676002)(66946007)(6486002)(86362001)(31696002)(66476007)(66556008)(2906002)(956004)(186003)(36756003)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RUtBRXJzeWhFZUptZGQrQktScHZ1VjYvMlYrdTZyY2FISDBCZHZwRzlqaHNW?=
 =?utf-8?B?VUhkaTFWNzZBR2Rkc0RzMU93UHE5Y3N1eWJMdDlrZ2NBN1JQRis5T2xpUXhr?=
 =?utf-8?B?Y29iL3kwRmFlY01Ia1VmZ0NZYi9DMXcwSFRocnFTZldmTUF2Ukw5U3I1K3dL?=
 =?utf-8?B?ZG9mMW9BN2NCd1Y0WWlRbjM3MzllaDFLaUdLMktuUTVZZEZWaHludHBDSGFI?=
 =?utf-8?B?eGJ1eW9WMThDRXB5WGFudmZLT1JQNGVrQ3RWKy82Z0Q4RWlhTWRMRVFCbmpQ?=
 =?utf-8?B?Zitpb0xpaTJVd2g4bTRxbXpkSjI2amlCcTlmaWxsK0l0M3ZhU2J1Q053RFFm?=
 =?utf-8?B?ckNwbFJPRC9aRTZDbDNYK0RiVzYxdUlacmVpcFV3YzNvcFphaGxzVzBVckFr?=
 =?utf-8?B?NmtlM1hWTWNHSkx5SFFqaVBCZlRDQW9hZDVCRm1LOCtMOFRaYWk0aFJwY1Bh?=
 =?utf-8?B?TGZjc0swNGJDeklWZDhyc3NMcm93WGc5YjhtT2tVQzBlWUlPZy93RDNxSTRC?=
 =?utf-8?B?M291eUp2aDVybGVGMTh0TXZyb2RMKzMxQVY3bllnamJFUWRmVzZwbHNqaCta?=
 =?utf-8?B?SnNKdE5vMEZtVmJBSS9RTXpwcXJ1V1pibm45VkwzeW9jWUN4UVRzTEVySmtB?=
 =?utf-8?B?K2Yzeld2b2JmTmhjR01OQm5CaHd0WXpjcnBnT29DUTVzQ0Y1NVBLZjZUQUJp?=
 =?utf-8?B?M0FkR1VnMWFmOWk1ZzMzQjZOMzBYbHhVeXpPTDRiRXA4WDJIMk5DU2tHNm5C?=
 =?utf-8?B?cUltVUJzbXFtL00xWHhTMTRnTEc4WC9PaVloTzgwK3Q0NTVlakRUa1U0aHJ6?=
 =?utf-8?B?TW11YzEwZTNTRlZ6T3FoZnM3eWcrazh6blB3REdkNVNkUEJveDZ5L3lsT0ZW?=
 =?utf-8?B?NGdJNzlqMkRtU05VeWtxUmV3aVgzKzBqck9jRnVJS2JzU2k4K0ZMRGg1S01X?=
 =?utf-8?B?NGFQMkYzRDdqVjVObXRidDhNSGhlKzFUeDZwR0dHZWNUQ3BMUGk4bE1JekZh?=
 =?utf-8?B?OG9TdW53Rzd0OTFpaUgzZTU4MU1FblhRdzY2M01EUndoUG1ROHVoN2w0UXYr?=
 =?utf-8?B?aWx2OVdBUXc3NnRHVkwyRVZoZzRFZXFiOUsrSWJndFF5ckU5elFGQ3hGT0Zo?=
 =?utf-8?B?OEVSdVhuOWw4TWZKU0V4Y0tzUDdDZy8wcEs4eWE1dTRjSWlVUWdwMjFZcXJw?=
 =?utf-8?B?RjV4RjVoWFRDZ0N1ZGpQQnpzTXg2YkZTMC9nN1FKZVZmZU9BWHhDMXkxYjQr?=
 =?utf-8?B?citkTDg3VS83YktMNHJmRVhHNHNSempNbThmcDdwU0ZHTG04NVoxT21OUWU2?=
 =?utf-8?B?Z1lvTnJ1dDF4K01hdDNuUzB5N1UzcjFyTDZuZFJ5ZjB1dXQ2bXZIYXU4YTdF?=
 =?utf-8?B?LytGdFozWTVlMWNVLzRMT3Q5ZGRTUXFBQkVtckRIUHV4L0FQeXJwdWtReHpW?=
 =?utf-8?B?cVdwbjhVTTAyZERoWkM3Z0RYeWl5TmRpTHlWVDFRTGJzME93a3FoYVVRU0pz?=
 =?utf-8?B?S2xIS3hlOHArMjdreG14Z0pjdm84NHY3TDFnclNHOXBOWGE0VnNKNDlpVnR2?=
 =?utf-8?B?SzNhb2hiWHkwdzFadzNXM2E2VXYvZ2tLV2IwQjhWcks1NmxXZ2N2L21kQjVV?=
 =?utf-8?B?V2VFbXZmZjJVRnhFN0RTcWt4YmEwOVExSDYrMEZGZkpmU2tFUk52b3hrZDFZ?=
 =?utf-8?B?SStRdGMwMkJGQ1JWRDkvNmNMN3ZxYXdUSCtsM0Z2U2pSb3dqNjU3VFdicCtK?=
 =?utf-8?Q?3hEkXufHvtyjlmXApzIrjBSVRM5UYW28VySMySb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 32bb6961-04e3-4b0e-7339-08d9782921a9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:13:50.3950
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: b1SoOOnzcswpQT+qOlCAapvNcGZ6v6sDofaCXqgnEZaa745w7ejKn69l0zmZPF6axcy1bqMS7/9e/4zgVJdaPg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7375

Hidden devices are associated with DomXEN but usable by the
hardware domain. Hence they need flushing as well when all devices are
to have flushes invoked.

While there drop a redundant ATS-enabled check and constify the first
parameter of the involved function.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu_cmd.c
+++ b/xen/drivers/passthrough/amd/iommu_cmd.c
@@ -308,14 +308,11 @@ void amd_iommu_flush_iotlb(u8 devfn, con
     flush_command_buffer(iommu, iommu_dev_iotlb_timeout);
 }
 
-static void amd_iommu_flush_all_iotlbs(struct domain *d, daddr_t daddr,
+static void amd_iommu_flush_all_iotlbs(const struct domain *d, daddr_t daddr,
                                        unsigned int order)
 {
     struct pci_dev *pdev;
 
-    if ( !ats_enabled )
-        return;
-
     for_each_pdev( d, pdev )
     {
         u8 devfn = pdev->devfn;
@@ -343,7 +340,16 @@ static void _amd_iommu_flush_pages(struc
     }
 
     if ( ats_enabled )
+    {
         amd_iommu_flush_all_iotlbs(d, daddr, order);
+
+        /*
+         * Hidden devices are associated with DomXEN but usable by the
+         * hardware domain. Hence they need dealing with here as well.
+         */
+        if ( is_hardware_domain(d) )
+            amd_iommu_flush_all_iotlbs(dom_xen, daddr, order);
+    }
 }
 
 void amd_iommu_flush_all_pages(struct domain *d)



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:31:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:31:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187406.336252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRFx-000444-0L; Wed, 15 Sep 2021 09:31:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187406.336252; Wed, 15 Sep 2021 09:31:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRFw-00043x-Ta; Wed, 15 Sep 2021 09:31:04 +0000
Received: by outflank-mailman (input) for mailman id 187406;
 Wed, 15 Sep 2021 09:31:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQRFw-00043r-Fw
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:31:04 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a49aedae-1607-11ec-b4ef-12813bfff9fa;
 Wed, 15 Sep 2021 09:31:03 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2059.outbound.protection.outlook.com [104.47.0.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-JSC065HkMgeJSIW6PzTGHw-1; Wed, 15 Sep 2021 11:31:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5902.eurprd04.prod.outlook.com (2603:10a6:803:ed::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 09:30:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:30:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0067.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Wed, 15 Sep 2021 09:30:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a49aedae-1607-11ec-b4ef-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631698262;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2mzhn0mvrPIwJZv0btGOpnNToczTHAHpNNtCJ5kQjZk=;
	b=XiT3Z3uKDnokYZSs1+xRmjBLQxIPwj4y+Fekv8Qg2V6wmsC/eVKcbqWqYuri216mC5eiBq
	2TIf7sVXaqTuGFXJ/o3zYlnewcv+zAseku/4EbzKK5ZsZuIACN1W4aKWIjy4lAu/q9KjMu
	H0dMQ3+lpVHc6rdCSm40preqjIO5f+w=
X-MC-Unique: JSC065HkMgeJSIW6PzTGHw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lpzmGlMzE/tm+YiQjUNksTO4HTTEowf5CViqdMkeYrC0Btp0Z8PmkwjEknVXmZRVKR94KUeFqgVGOOQ2EVBE6kzSA2tgt9OQzdXZ7onX9tZ2iXtbkdh1pSmEV0ZCmpYN9ag6gSKo5NKuG0739MPWUKMbkoX3hIJa3Mvq0Fo2Iw9BqD/dxEoxwpSBp0/d+lEt3DWFw8PlVnkQhx+9ZsCmGWGi/J5DaI78JfDUc1Rpx1+F0uKxWyDN4bD1muxvwNHBPFvAHxu8CPo7QIQ7M/IXf/zmVKUyPlSyuFRnb+IA28lD3tpZymwF9Nx01fZdabxpK5KzwNSnuusW6/wH7AJsGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2mzhn0mvrPIwJZv0btGOpnNToczTHAHpNNtCJ5kQjZk=;
 b=Ol0YpJf/1hOG2JSDOIyqDx6F4rWw6SKQ33NP3+tnPvqF2K+wkVq+QwGGUVccUnBrbTBJ/w5PMMKDO+0qkHR1yD72HfBd3D6Y901fkVsJMmzx/vVWk9hR//KW0a/VvlD3dVd0EVj74o+TXTTAeyEGVBmZIfceyada0COYliJ76y46DfjHN9oaHxLLMcWmSTCBIrk3xGWVIw+tbTk7GQVT0UFTpm7/bQZTKboy4bTJVz5vkAxHdq7cTSQT7Kk/LVbT/QEOCWu3vwrx6UaU2umP3EktsqsRFK7awzeJrZp1llKkF/WY4vBF3o4dzVnEICA08TwBKjMn2NaOSZnPNo8LOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] tools/libxl: Correctly align the ACPI tables
To: Kevin Stefanov <kevin.stefanov@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210914164323.14904-1-kevin.stefanov@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f5a68380-16f1-5383-5cfa-85a9e02ccf8d@suse.com>
Date: Wed, 15 Sep 2021 11:30:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210914164323.14904-1-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0067.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b69ef38-e56e-4407-a105-08d9782b86b3
X-MS-TrafficTypeDiagnostic: VI1PR04MB5902:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5902AC064D74CD65267D06C7B3DB9@VI1PR04MB5902.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2201;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	r6MPjQH2WsguQRqN9ES+0Y2NZj+dbl+RlUXi9k8WWJbse4z9+YQDXJ5DK1dmN4a9sfIgqFllYijHrAI2spYanpcinZ3CIpQNl7CWaP3ES/4gvaw0BGuGfChPvKk2IXLBiVBcqbRG9n/2TnCZQ08jkBp3hMkPjmksPLagpt7XK5QFkEmEzWIbccxVVmcPPuoDDC3A0fbiXX/dMz1uxqiVlxw1wNbKVJpoKPV/jUDtVsGiicZaR1jn+Wttc9HtttoUfx+1yg9FS++IRNv40l684uyPbb3kt4LeFLwo7q5bMyAX8DG7Z0GefGbv3CyTV8aXNzLp8RayxoNDO7cwo9qdiI5SELho1wgIaD6f8ezcYOgBU9y074mFDVxyZsFhwKgOR57jzGjGQoKTRpNqfpzWRP73rhAIcun2X6el7a0O5LnSTLekKeCozOfMbSlDSyHarZfEbxQtFM6wEUyHFEB9jBBrZkn19pK8zuccHu482Bflgs4KW+NhzvJDuwMuShXxAcsiuZGmHxth8zwA+WmQumMa7H8IbdpJv6Y5u0ZRm24NYPfmSSZ1zYumXz+j2UZZ8Fap6VFQfECsr8E2/KbMROl5gI0xxIEvKvSqCvpDXL8ziG8qBVjkwdwcY+0f2eUP2szAhkA0a3bx66nCHiPYeJUkzmqwJzDZBM0v7ah9wstkYEB20gD0emnmgrSE1H1HgH1hSvoQj0y3rQO7VY9DpdvENVvyPvA/n1utqz7z1I9ZQLdYZ7OvuDiqp27lTKiP
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(376002)(136003)(366004)(346002)(478600001)(16576012)(36756003)(26005)(5660300002)(66946007)(4744005)(316002)(66476007)(66556008)(6916009)(2906002)(31686004)(186003)(8936002)(53546011)(54906003)(31696002)(4326008)(6486002)(2616005)(8676002)(956004)(86362001)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NzBmWlBwdFI2QnovTHRlRC9UbWlTbnR5Q1FWejJoTG9RbkQ1VzJFWGNiZGdW?=
 =?utf-8?B?NW5YMTk2K01wMnZaY0JSQkQrdWJLNFpaS1FDcy9Ja1dxWmg3R01GUFVabjMw?=
 =?utf-8?B?VE94RXVHR2VCOWgzK3VwK2htUUdYUHZjZWRiaFVyQ3hZUC9mNWNNelZFTEVT?=
 =?utf-8?B?K1c2aXlBYUN0dm10NVNVOE0rbzBndDlHaFUybDJteFY3dFBUSGpyNzQxSHBz?=
 =?utf-8?B?RVkvaHZ5U2RYeGpDbXpXcGI0V2dXRm9ZeEpQbnNHLzNEeWtXcE9ldHZEQ0Ez?=
 =?utf-8?B?bGtweXZPL2hTWEczNmpHbS9PbXVNM0FtRjJxS2ZiYnlVRTZHMlBDQ2lkeDJz?=
 =?utf-8?B?VkxoU0NBL3dGSDB5YlRZZWlSdU55akQ0a3FXdEpXWWIzZmRoajBOcmRnSExD?=
 =?utf-8?B?UTJzUUczSDhJR2JOUXNGMkloWWdMMDgvV2dnU0E3SkhJbEtEQURHcUZFY1c3?=
 =?utf-8?B?cmJWUjZXcVZMSVJDR1kzeHBoSGY0aW5WZk1QK09CRFBFc3RjZnJOTk5JcDRE?=
 =?utf-8?B?dGlzc0p0TEp1Ri8wUzBOODQ2OURLcWlhOFkrVy80azJhblpZOEovTis0UXJ6?=
 =?utf-8?B?Y2lwMUlES00rQ1BHUHFSblVOYStVM01CWHBOVjlEN1IycEZzTUdIMXo0cG5L?=
 =?utf-8?B?VDhlQVBDRVE1eWFuTWZ6QnF1cGdlNTNNMm5mbjRLYUl4SCs5djJBVFFDYjFt?=
 =?utf-8?B?MjFQbGVBcm5iQm1mOWx4endDRXA2OFdjSERrOERSTituZFhmdnBTVkJVMk80?=
 =?utf-8?B?cFJZK0k1TExiSU81VzFtbmRPdmJieHBQbWVnZng2M0FVTVZzT0RMUzRQUFo3?=
 =?utf-8?B?RmZGcnNTb3JYWmwzT3lNNndwNEdBMldvWkNpL1FwcjNIa1lnNUY4eFkxdTcr?=
 =?utf-8?B?QkZhYjI1SGhrL3dnS0tNL3pkazU5SlUxVzQxSUdQSW93aFB6QXhlZno1VUNJ?=
 =?utf-8?B?NWd2cnVNQ3BTb3pCS0JPV2ZPajVjcVlMQTlrQU5iM1FVNk9EUnFXcHNtSUJ4?=
 =?utf-8?B?SlNaSlMzQ096bVVVTmxxeTF6TjhScmtDRjFqZStIQWQwNmJwQzdvY0c3SkV0?=
 =?utf-8?B?SUNDZ1VkeWFXckEvS2FOUlBIZzB1T1lDWDk1a01jSzdrM3cxYlhHWmRZL09q?=
 =?utf-8?B?Vi9jRklTdUIxc0FTUkt1ZnJSb0JhaEJaK2lzSytFdVNncitmL0VIODJ6ZWhO?=
 =?utf-8?B?aFlSOVRJcG5TaFp4MkVjUE5FK3dtK09QYzVKNEEwRE9aQjY2bkt4QW4wRkJQ?=
 =?utf-8?B?NkllQ29XVDJ3TW02MkpJSk9sMTdVVk5hL0szMTJXWjBYMnM0WENlSC9pVGJU?=
 =?utf-8?B?a2RiMXVtZStVakQ0SGJVekRGbGFPVkpjTkMvTFhVM0JPUENDQnkwMzR6THVW?=
 =?utf-8?B?ZktBOWZZTkdPeGRQd2Z4TXZEamxta0ZRQmVRM040WnJLUTMxR1d5RUs3R080?=
 =?utf-8?B?bXVBU0cyZHZkeWxpRzNnOFBmRjFpN0dTeGJua2JiMC9yWEdWZHV2Rndwb0Zw?=
 =?utf-8?B?L1VqSkdqM1RRcHZ4aWdmVU1xdTAwc1cvUi9uaGZuZnVxb1ZhS1I1NXgxUmNw?=
 =?utf-8?B?Nm53UzQzaTJsU3hBc0JQSXZ4UnNSRUFqUjVHUHFkRDlPVGVxSVE2TzY2RHly?=
 =?utf-8?B?UnVPMEpZODJ5eW9ha05rcUwwRnBEOTFBOFIrdVZKTHlMS2xvNlNKbzFHaHY5?=
 =?utf-8?B?dk1GVVQ1S1RqcmZiL0lZc2tLOUxDemlCWnlTbDBzeEV4QVdhMEhLV0tNVHo3?=
 =?utf-8?Q?KtytnNewQrfK1acn260tui2XYe6mPtHhfG8iklW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b69ef38-e56e-4407-a105-08d9782b86b3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:30:58.9481
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 61d64PImSLQYSzVFqlyT+3o3Kdg+MJ43iQsSQfcGyrLHWFB4vflsJvV3K6V2LnWc6Byh+iAK09I770QOuZGJpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5902

On 14.09.2021 18:43, Kevin Stefanov wrote:
> --- a/tools/libs/light/libxl_x86_acpi.c
> +++ b/tools/libs/light/libxl_x86_acpi.c
> @@ -20,6 +20,7 @@
>  
>   /* Number of pages holding ACPI tables */
>  #define NUM_ACPI_PAGES 16
> +#define ALIGN(p, a) (((p) + ((a) - 1)) & ~((a) - 1))

Wouldn't this better live in xen-tools/libs.h?

> @@ -163,7 +159,6 @@ int libxl__dom_load_acpi(libxl__gc *gc,
>      struct acpi_config config = {0};
>      struct libxl_acpi_ctxt libxl_ctxt;
>      int rc = 0, acpi_pages_num;
> -    void *acpi_pages;
>      unsigned long page_mask;

With, as pointed out by Andrew, this now dead variable also removed
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:49:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:49:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187416.336263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRXn-0005tr-Je; Wed, 15 Sep 2021 09:49:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187416.336263; Wed, 15 Sep 2021 09:49:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRXn-0005tk-Gc; Wed, 15 Sep 2021 09:49:31 +0000
Received: by outflank-mailman (input) for mailman id 187416;
 Wed, 15 Sep 2021 09:49:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQRXl-0005te-Ts
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:49:29 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 374727e2-160a-11ec-b4f0-12813bfff9fa;
 Wed, 15 Sep 2021 09:49:28 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2058.outbound.protection.outlook.com [104.47.14.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-TR9Ib79VM-iNZFqqfQwUKA-1;
 Wed, 15 Sep 2021 11:49:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 09:49:24 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 09:49:24 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0240.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 09:49:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 374727e2-160a-11ec-b4f0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631699367;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YVYF1+4759JGKnDVGjvlOJsaq7w4NBqK3wjGpNJ2IXU=;
	b=hDWFh0UqxY6lmlN0Ai44zR6kWBbxn3VUSQRqfKu4bmzosa1J3WYPYmURA5PnS7wihtm30t
	dJziJoOYhBza4sTjiGMev9HfDgk4dC9m7xfagTqFGWf1CRLN2DgcMGHmQll7sfBD42NseE
	jsN6sD1n/36GmVXSpZuHaGCnKpoe5Ro=
X-MC-Unique: TR9Ib79VM-iNZFqqfQwUKA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PUXmSFaHrVpiQa7NM4thzhP7jpU3cWYR8y+0CRYFfY0BM5mIfD3dUnGWbJXCPMsN9VKzMj1Dyfu9tgSSYJYysZ2zHiOhPHDutGtaDCKw2M14oUqa1Fa/ZD6tktU81pcwyAheNElpK6Lf1dUCnmrTI2aRW0SESMdPS3aaLJNTGCl58Cf4ojx8wL5X5EJBPQbJ2KBzBlf5xpP5R48u7TIaXpsQbFxz4fyuJ1lD7/CSIuhTHlgZICi/huqejX5B1Rf0blo9X7bEW33adbdyc5SGpFpjPfarh0mMqUANZ+zs2TS6XX+lZsWnTen6pwDFezt9e6gH7sw5NfnBn7GS7Niv7A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=a2AgMpcRbS959oxBQSIAFBZVmflgjKd8eH6KLnV2Gxg=;
 b=FZecZNIZNrsBOn9rZn0Kuy6AGqxXen16n9Qth8JEaWvCAkNsQ8vNpWbdzkqnPxTOz4r9d+vmILTfsYYNAmANWX7Qzp0AJ2bfaJIcx6y3SFxRXijIiAGluQ+v9DkFAHSiPc8LIx8bbB4bFhkA2lK72ZRIBpVVM0Bplz/RYeR1jioBbAhyHBTPX8uoERQhaAt/aAsp746RUQ3L3qdcA6mBerl+S4m+mrpstcDQT/F8DqgVFeqZ8lZZk8PMSuFyse9vxd48nfzwgTIXeQz1Qp75GcETv5kiWndRaoMYi1/zfyR+UCHpsZbMtI6hfeHfJQYFrgg8f6N6I7o9EqPsvXAnAA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] x86/p2m: fix xenmem_add_to_physmap_one double page
 removal
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20210915080342.21346-1-roger.pau@citrix.com>
 <1d70ff4d-bef2-1865-27c3-27d17c15d17f@suse.com>
 <YUGyhk0uZQMf9c4s@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <98dd86d5-a864-a95d-6fa7-f0c70abb5a84@suse.com>
Date: Wed, 15 Sep 2021 11:49:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUGyhk0uZQMf9c4s@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0240.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ec6e1760-1189-423f-1e41-08d9782e195a
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233536AC87C3A3438101B4BFB3DB9@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	n+v9q9qc2GkhUH1SkkY2HgKmQ/oV2HW28q+ntFN4yQv07d/DFIcwmc7IdCvtUC7We2/JszzffuLf/Sdez9rgG1kimdGXEoP5nxgqHks5C6qVVfBnLvIiCHqcFSyRcgdroXW76G5Vcnt7P1yV1Kr5JnzqM2u+EFIe3x56MLWDsOeUj2PhUrfzZjs0npNTjRLbc9BUH5QGWsRTEcSrNvC983AYOsXuM1ySS3c4PkVgJtaJGsZI5ykCFl4lAZa264gxmpwP1ei33tV7/oetgWT0JbbLZXxxhUDghJtwJyAxLu85PKUPVcrnz5M2TJIpdZFZ9n8Oc8T4SXKJAJDY3QpSnT1ALQBcCabvpPTyg7TGa/hpdATD4QbQlrKJhGoVj72X99R89LyuYrfwWM8CCvTeos0OJEYTYYphOm3xnUQ8Z2yVoADEpty+jWLCXqM9AsSXJh0R+8/wTWZF1fC330Uqinu8/fIfj1N2NUOUa79tpihjEOAvjbUMcn9jpziR7sNp1w+CTdXWueIub07lbL/hOr6VFFAOMHZJfpHz6BNu2ADiuu1FRCFGkNpez+Lbu8vvw58BEShmqk2dlEYfr19eU8gDmIUJPQySjzRT0/ZkVIL3Y2kD3NcF034SQGMNU5MuLSSRg/4Ht/jug5BnOjrzWFHnWwchTObvn6V5uUVghk5WzdjZvOt3+8NQD6K3+h0AUzFediUjBnw2BwQ8+Q3wTVavePC5RnlwdRbHTO0wfP4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(376002)(366004)(136003)(346002)(5660300002)(86362001)(83380400001)(478600001)(186003)(4326008)(66946007)(956004)(66556008)(66476007)(54906003)(8676002)(2616005)(316002)(16576012)(6486002)(26005)(36756003)(53546011)(6916009)(31686004)(38100700002)(31696002)(2906002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?E0QaFGKZjS80XmiVMoFJ/jVhTdzZ9HE21dW9pKFjPudlV6St6gvg8Rx+qGNy?=
 =?us-ascii?Q?WMZOWMm1Z8BqquGWsvOoNjnLvq3fRPcN2lsWH47LMYh1UuKcsU0BiAqkznJL?=
 =?us-ascii?Q?V6AMYHGk4XlyM2mDVLuriLw7bvyp6Q7sOxA2Z1AZGGIaN6l9g5e4YZkUxQpi?=
 =?us-ascii?Q?UGZTIeU1hazwcs8zO+mjxP2zFfPUEt0pkLNK2xP0N+xI1LeLwD2HFhH8jiSW?=
 =?us-ascii?Q?Opg/1qyRe2Oauw6NXzp2RRy0vbY5GwSfAxJnanE+CSQ/t2F6KDmdGTBCGccB?=
 =?us-ascii?Q?MjcNPcsmOKIuuQpQ0d0yRVkUm0e0UQ3zSfbBB21mJR7NS12E2V7+Iw+nvKIj?=
 =?us-ascii?Q?8uHo2IS84yv7ijb1LVFBjeRWweC0R/7pxMdIG/FzYw6bQcNpWRHnWF7xvMH8?=
 =?us-ascii?Q?i3enEgmDrcTKnSiPcF0z/uMS+J0knvjR2wL0kLYKFgUaOj3fHAvClhBYOXn6?=
 =?us-ascii?Q?oCqgQcE+ILq1JeGFAvro1mEpu5nMP+H0Q/YEB4iOx4r0kG04Ah0j4YEXMDgX?=
 =?us-ascii?Q?FMfHchPz8Ig1M9dsCPksBoPBX1Gypvo0KapDW9eUsehgSmmKp3TEtp7CrYUx?=
 =?us-ascii?Q?GEoerFuMzLtQTsHA4JNUtyNYQHDXB9u86qaljBYMnQynsLaWotlBc9dF+1oX?=
 =?us-ascii?Q?XtFdz3d9LmFpLJV94pw8thQtuDFU10G3sHD4Z3laR+DeZ3moLZJt2TZ7tbzv?=
 =?us-ascii?Q?EkJ76isTi38/tXziGHHbJYwkCG5thGPA/N4YwpiucdgV7I2BVGmr67Kyop31?=
 =?us-ascii?Q?DuT8FaVRtYnTf3OVBwh6ckzNZSbBVI8iqZOxWWVg4VQ7NZIioh01lTNjcZnc?=
 =?us-ascii?Q?6fezkZnjYgsGkBrsO5b2S4/hbDpUuW+D1RjVpHOElgIKRKjNtMWBG+m2HW3v?=
 =?us-ascii?Q?1ItQ+98PilbAAfc/+4fcvWCDuAx9RrrWybhxDoFR1viQ2wVmj1FMgDs5l8HT?=
 =?us-ascii?Q?rzh01nxQNj0f6wiuYkbuHhk3QQidxZ2qa2WHbx+4XL/ZjNs0LqGqOnBRTt+x?=
 =?us-ascii?Q?EvKh8DQOwQWsxO9H62Ry+yc0ITfR7w3dQZHSVF5EPAP1uiT8dmI1Gc/bChM0?=
 =?us-ascii?Q?DQcTzkaD6GiSH+/PNUB9jvHSJqVhhGi4u3jkZRWuayXbCzlGS021qlT3do4S?=
 =?us-ascii?Q?Kr1iIx16j666ZFGuLOb/fQaCmnEGb4kdtWKipIf+PTDxl+FJqGrfW5jyHZlD?=
 =?us-ascii?Q?1tuBCYMgAvdo0J2u594u8ko/LUOYNj+zLFtsbqEmdtxAu5Wp8wpQqZcguhTY?=
 =?us-ascii?Q?GYLY3zTNqT8So99b9DdpuMT6vki/wbHyqyYtWCxiQN2EPmup+TI+zeZ6soSm?=
 =?us-ascii?Q?vZphxjePCAi1d2n10s8s79Og?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec6e1760-1189-423f-1e41-08d9782e195a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 09:49:23.9980
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HjnIvA3/8BcfGFpqf9PAGIRwQWiClz9RdGV1iEeLwQAYfydl9f88XoMc0WK47D0as9Dm3PddPlAPGIWkhlZJCA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 15.09.2021 10:44, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 15, 2021 at 10:16:41AM +0200, Jan Beulich wrote:
>> On 15.09.2021 10:03, Roger Pau Monne wrote:
>>> If the new gfn matches the previous one (ie: gpfn =3D=3D old_gpfn)
>>> xenmem_add_to_physmap_one will issue a duplicated call to
>>> guest_physmap_remove_page with the same guest frame number, because
>>> the get_gpfn_from_mfn call has been moved by commit f8582da041 to be
>>> performed before the original page is removed. This leads to the
>>> second guest_physmap_remove_page failing, which was not the case
>>> before commit f8582da041.
>>>
>>> Add a shortcut to skip modifying the p2m if the mapping is already as
>>> requested.
>>
>> I've meanwhile had further thoughts here - not sure in how far they
>> affect what to do (including whether to go back to v1, with the
>> description's 1st paragraph adjusted as per above):
>>
>>> --- a/xen/arch/x86/mm/p2m.c
>>> +++ b/xen/arch/x86/mm/p2m.c
>>> @@ -2799,6 +2799,13 @@ int xenmem_add_to_physmap_one(
>>>          goto put_all;
>>>      }
>>> =20
>>> +    if ( gfn_eq(_gfn(old_gpfn), gpfn) )
>>> +    {
>>> +        /* Nothing to do, mapping is already as requested. */
>>> +        ASSERT(mfn_eq(prev_mfn, mfn));
>>> +        goto put_all;
>>> +    }
>>
>> The mapping may not be "already as requested" because of p2m type or
>> p2m access. Thoughts? (At the very least the new check would likely
>> want to move after the p2m_mmio_direct one.)
>=20
> Is the type really relevant for the caller? If the mapping is already
> setup as requested, I don't think it makes sense to return -EPERM if
> the type is mmio. I'm also unsure whether we can get into that state
> in the first place.

mmio perhaps indeed can't occur (because of the earlier
get_page_from_mfn()), but in general the type might very well be
relevant: The seemingly benign change might e.g. change logdirty
to ram_rw. Whether that's for good or bad is a different matter.

> I'm unsure regarding the access, I would say changing the access type
> should be done by other means rather that replying on
> xenmem_add_to_physmap.

It _should_, yes, but there might be callers relying on it
changing as a side effect here. (There might also be callers
abusing the call for this purpose.)

> v1 was indeed more similar to the previous behavior IMO, so it's
> likely a safer approach.

Okay, I'll commit v1 with the slightly adjusted description then.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:53:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187424.336275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRbn-0007Gj-41; Wed, 15 Sep 2021 09:53:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187424.336275; Wed, 15 Sep 2021 09:53:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRbn-0007Gc-18; Wed, 15 Sep 2021 09:53:39 +0000
Received: by outflank-mailman (input) for mailman id 187424;
 Wed, 15 Sep 2021 09:53:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IFyV=OF=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mQRbl-0007GV-GM
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 09:53:37 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ca052d36-160a-11ec-b4f1-12813bfff9fa;
 Wed, 15 Sep 2021 09:53:33 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C29C6D;
 Wed, 15 Sep 2021 02:53:33 -0700 (PDT)
Received: from [10.57.23.30] (unknown [10.57.23.30])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 896283F5A1;
 Wed, 15 Sep 2021 02:53:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca052d36-160a-11ec-b4f1-12813bfff9fa
Subject: Re: [XEN PATCH v7 00/51] xen: Build system improvements, now with
 out-of-tree build!
To: Anthony PERARD <anthony.perard@citrix.com>
References: <20210824105038.1257926-1-anthony.perard@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <5419c010-c860-42a5-aa6c-9ba328b0c7c6@arm.com>
Date: Wed, 15 Sep 2021 11:53:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210824105038.1257926-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi Anthony,

On 24.08.2021 12:49, Anthony PERARD wrote:
> Patch series available in this git branch:
> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v7
> 
> v7:
>     Out-of-tree build!
> 
>     This mean many more patches. Everything after patch 27 is new.
> 
>     There's a few new patch before that, but otherwise are rework of v6.
> 
> v6:
>     This new version of the series get's us very close to be able to do
>     out-of-tree build, but I've stop at been able to use Linux's `fixdep` and
>     getting rid of those annoying *.d2 files generation.
> 
>     There a few patch left from the previous round within the first 8 patches
>     (mixed with new or replaced patches). All the rest is new.
> 
>     The main patch is probably "build: build everything from the root dir, use
>     obj=$subdir" which has still quite a few changes left. I might be still be
>     able to break it down even more if needed to help with review (and might
>     allow me to better explain some changes in it).
> 
> v5:
> https://lore.kernel.org/xen-devel/20200421161208.2429539-1-anthony.perard@citrix.com/
> - few changes detailed in patch notes.
> - 1 new patch
> 
> Hi,
> 
> I have work toward building Xen (the hypervisor) with Linux's build system,
> Kbuild.
> 
> The main reason for that is to be able to have out-of-tree build. It's annoying
> when a build fail because of the pvshim. Other benefit is a much faster
> rebuild, and `make clean` doesn't take ages, and better dependencies to figure
> out what needs to be rebuild.
> 
I wanted to let you know that I tested out-of-tree build using v7 series on arm64/arm32 and it works.
Also I tested it with Yocto building Xen(to check if it does not break the builds) and
it also works. Rebuilding/cleaning is really much quicker. Great work.

> This new version of the series get's us very close to be able to do out-of-tree
> build, but I've stop at been able to use Linux's `fixdep` and getting rid of
> those annoying *.d2 files.
> 
> Cheers,
> 
Cheers,
Michal

> Anthony PERARD (51):
>   build: introduce cpp_flags macro
>   build: use if_changed on built_in.o
>   build: use if_changed_rules with %.o:%.c targets
>   build: factorise generation of the linker scripts
>   x86/mm: avoid building multiple .o from a single .c file
>   build,include: rework compat-build-source.py
>   build,include: rework compat-build-header.py
>   build: fix clean targets when subdir-y is used
>   build: use subdir-y in test/Makefile
>   build,arm: move LDFLAGS change to arch.mk
>   build: move make option changes check earlier
>   build: avoid building arm/arm/*/head.o twice
>   build: convert binfile use to if_changed
>   xen: move include/asm-* to arch/*/include/asm
>   build,riscv: tell the build system about riscv64/head.S
>   build: generate "include/xen/compile.h" with if_changed
>   build: set XEN_BUILD_EFI earlier
>   build: fix $(TARGET).efi creation in arch/arm
>   build: fix arch/x86/note.o rule
>   build: avoid re-executing the main Makefile by introducing build.mk
>   build: set ALL_OBJS to main Makefile; move prelink.o to main Makefile
>   build: clean common temporary files from root makefile
>   build,include: remove arch-*/*.h public header listing from
>     headers*.chk
>   build: prepare to always invoke $(MAKE) from xen/, use $(obj)
>   build: rework test/livepatch/Makefile
>   build: build everything from the root dir, use obj=$subdir
>   build: introduce if_changed_deps
>   build: rename __LINKER__ to LINKER_SCRIPT
>   build: add an other explicite rules to not build $(XEN_ROOT)/.config
>   build: hook kconfig into xen build system
>   xen/tools/kconfig: fix build with -Wdeclaration-after-statement
>   build: Remove KBUILD_ specific from Makefile.host
>   build: handle always-y and hostprogs-always-y
>   build: start building the tools with the main makefiles
>   build: Add headers path to CFLAGS once for all archs
>   build: generate x86's asm-macros.h with filechk
>   build: clean-up "clean" rules of duplication
>   build: use main rune to build host binary x86's mkelf32 and mkreloc
>   build: rework coverage and ubsan CFLAGS handling
>   build: fix dependencies in arch/x86/boot
>   build,x86: remove the need for build32.mk
>   build: grab common EFI source files in arch specific dir
>   build: replace $(BASEDIR) by $(objtree)
>   build: add $(srctree) in few key places
>   build: rework cloc recipe
>   build: replace $(BASEDIR) by $(srctree)
>   build: Rework "clean" to clean from the root dir
>   build: Rework "headers*.chk" prerequisite in include/
>   build: adding out-of-tree support to the xen build
>   build: specify source tree in include/ for prerequisite
>   build: add %.E targets
> 
>  .gitignore                                    |   7 +-
>  MAINTAINERS                                   |  37 +-
>  tools/include/Makefile                        |   2 +-
>  tools/misc/xen-access.c                       |   4 +-
>  tools/tests/vhpet/Makefile                    |   2 +-
>  xen/Kconfig                                   |   4 +-
>  xen/Makefile                                  | 408 +++++++++++-------
>  xen/Rules.mk                                  | 293 ++++++++-----
>  xen/arch/arm/Makefile                         |  62 +--
>  xen/arch/arm/README.LinuxPrimitives           |  10 +-
>  xen/arch/arm/arch.mk                          |  12 +-
>  xen/arch/arm/arm32/Makefile                   |   1 +
>  xen/arch/arm/arm32/head.S                     |   2 +-
>  xen/arch/arm/arm64/Makefile                   |   2 +
>  xen/arch/arm/arm64/head.S                     |   2 +-
>  xen/arch/arm/efi/Makefile                     |   9 +
>  .../asm-arm => arch/arm/include/asm}/acpi.h   |   0
>  .../arm/include/asm}/alternative.h            |   0
>  .../asm-arm => arch/arm/include/asm}/altp2m.h |   0
>  .../arm/include/asm}/arm32/atomic.h           |   0
>  .../arm/include/asm}/arm32/bitops.h           |   0
>  .../arm/include/asm}/arm32/bug.h              |   0
>  .../arm/include/asm}/arm32/cmpxchg.h          |   0
>  .../arm/include/asm}/arm32/flushtlb.h         |   0
>  .../arm/include/asm}/arm32/insn.h             |   0
>  .../arm/include/asm}/arm32/io.h               |   0
>  .../arm/include/asm}/arm32/macros.h           |   0
>  .../arm/include/asm}/arm32/mm.h               |   0
>  .../arm/include/asm}/arm32/page.h             |   0
>  .../arm/include/asm}/arm32/processor.h        |   0
>  .../arm/include/asm}/arm32/sysregs.h          |   0
>  .../arm/include/asm}/arm32/system.h           |   0
>  .../arm/include/asm}/arm32/traps.h            |   0
>  .../arm/include/asm}/arm32/vfp.h              |   0
>  .../arm/include/asm}/arm64/atomic.h           |   0
>  .../arm/include/asm}/arm64/bitops.h           |   0
>  .../arm/include/asm}/arm64/brk.h              |   0
>  .../arm/include/asm}/arm64/bug.h              |   0
>  .../arm/include/asm}/arm64/cmpxchg.h          |   0
>  .../arm/include/asm}/arm64/efibind.h          |   0
>  .../arm/include/asm}/arm64/flushtlb.h         |   0
>  .../arm/include/asm}/arm64/hsr.h              |   0
>  .../arm/include/asm}/arm64/insn.h             |   0
>  .../arm/include/asm}/arm64/io.h               |   0
>  .../arm/include/asm}/arm64/macros.h           |   0
>  .../arm/include/asm}/arm64/mm.h               |   0
>  .../arm/include/asm}/arm64/page.h             |   0
>  .../arm/include/asm}/arm64/processor.h        |   0
>  .../arm/include/asm}/arm64/sysregs.h          |   0
>  .../arm/include/asm}/arm64/system.h           |   0
>  .../arm/include/asm}/arm64/traps.h            |   0
>  .../arm/include/asm}/arm64/vfp.h              |   0
>  .../arm/include/asm}/asm_defns.h              |   0
>  .../asm-arm => arch/arm/include/asm}/atomic.h |   0
>  .../asm-arm => arch/arm/include/asm}/bitops.h |   0
>  .../asm-arm => arch/arm/include/asm}/bug.h    |   0
>  .../arm/include/asm}/byteorder.h              |   0
>  .../asm-arm => arch/arm/include/asm}/cache.h  |   0
>  .../arm/include/asm}/cadence-uart.h           |   0
>  .../asm-arm => arch/arm/include/asm}/config.h |   2 +-
>  .../asm-arm => arch/arm/include/asm}/cpregs.h |   0
>  .../arm/include/asm}/cpuerrata.h              |   0
>  .../arm/include/asm}/cpufeature.h             |   0
>  .../arm/include/asm}/current.h                |   0
>  .../arm/include/asm}/debugger.h               |   0
>  .../asm-arm => arch/arm/include/asm}/delay.h  |   0
>  .../asm-arm => arch/arm/include/asm}/desc.h   |   0
>  .../asm-arm => arch/arm/include/asm}/device.h |   0
>  .../asm-arm => arch/arm/include/asm}/div64.h  |   0
>  .../asm-arm => arch/arm/include/asm}/domain.h |   0
>  .../arm/include/asm}/domain_build.h           |   0
>  .../arm/include/asm}/early_printk.h           |   0
>  .../arm/include/asm}/efibind.h                |   0
>  .../asm-arm => arch/arm/include/asm}/elf.h    |   0
>  .../asm-arm => arch/arm/include/asm}/event.h  |   0
>  .../arm/include/asm}/exynos4210-uart.h        |   0
>  .../arm/include/asm}/flushtlb.h               |   0
>  .../asm-arm => arch/arm/include/asm}/gic.h    |   0
>  .../arm/include/asm}/gic_v3_defs.h            |   0
>  .../arm/include/asm}/gic_v3_its.h             |   0
>  .../arm/include/asm}/grant_table.h            |   0
>  .../arm/include/asm}/guest_access.h           |   0
>  .../arm/include/asm}/guest_atomics.h          |   0
>  .../arm/include/asm}/guest_walk.h             |   0
>  .../arm/include/asm}/hardirq.h                |   0
>  .../asm-arm => arch/arm/include/asm}/hsr.h    |   0
>  .../arm/include/asm}/hypercall.h              |   0
>  .../asm-arm => arch/arm/include/asm}/init.h   |   0
>  .../asm-arm => arch/arm/include/asm}/insn.h   |   0
>  .../asm-arm => arch/arm/include/asm}/io.h     |   0
>  .../asm-arm => arch/arm/include/asm}/iocap.h  |   0
>  .../asm-arm => arch/arm/include/asm}/iommu.h  |   0
>  .../arm/include/asm}/iommu_fwspec.h           |   0
>  .../asm-arm => arch/arm/include/asm}/ioreq.h  |   0
>  .../asm-arm => arch/arm/include/asm}/irq.h    |   0
>  .../asm-arm => arch/arm/include/asm}/kernel.h |   0
>  .../arm/include/asm}/livepatch.h              |   0
>  .../asm-arm => arch/arm/include/asm}/lpae.h   |   0
>  .../asm-arm => arch/arm/include/asm}/macros.h |   0
>  .../arm/include/asm}/mem_access.h             |   0
>  .../asm-arm => arch/arm/include/asm}/mm.h     |   0
>  .../asm-arm => arch/arm/include/asm}/mmio.h   |   0
>  .../arm/include/asm}/monitor.h                |   0
>  .../arm/include/asm}/new_vgic.h               |   0
>  .../asm-arm => arch/arm/include/asm}/nospec.h |   0
>  .../asm-arm => arch/arm/include/asm}/numa.h   |   0
>  .../asm-arm => arch/arm/include/asm}/p2m.h    |   0
>  .../arm/include/asm}/page-bits.h              |   0
>  .../asm-arm => arch/arm/include/asm}/page.h   |   0
>  .../asm-arm => arch/arm/include/asm}/paging.h |   0
>  .../asm-arm => arch/arm/include/asm}/pci.h    |   0
>  .../asm-arm => arch/arm/include/asm}/percpu.h |   0
>  .../asm-arm => arch/arm/include/asm}/perfc.h  |   0
>  .../arm/include/asm}/perfc_defn.h             |   0
>  .../arm/include/asm}/pl011-uart.h             |   0
>  .../arm/include/asm}/platform.h               |   0
>  .../arm/include/asm}/platforms/exynos5.h      |   0
>  .../arm/include/asm}/platforms/midway.h       |   0
>  .../arm/include/asm}/platforms/omap5.h        |   0
>  .../arm/include/asm}/platforms/vexpress.h     |   0
>  .../asm}/platforms/xilinx-zynqmp-eemi.h       |   0
>  .../arm/include/asm}/processor.h              |   0
>  .../arm/include/asm}/procinfo.h               |   0
>  .../asm-arm => arch/arm/include/asm}/psci.h   |   0
>  .../asm-arm => arch/arm/include/asm}/random.h |   0
>  .../asm-arm => arch/arm/include/asm}/regs.h   |   0
>  .../arm/include/asm}/scif-uart.h              |   0
>  .../asm-arm => arch/arm/include/asm}/setup.h  |   0
>  .../arm/include/asm}/short-desc.h             |   0
>  .../asm-arm => arch/arm/include/asm}/smccc.h  |   0
>  .../asm-arm => arch/arm/include/asm}/smp.h    |   0
>  .../arm/include/asm}/softirq.h                |   0
>  .../arm/include/asm}/spinlock.h               |   0
>  .../asm-arm => arch/arm/include/asm}/string.h |   0
>  .../arm/include/asm}/sysregs.h                |   0
>  .../asm-arm => arch/arm/include/asm}/system.h |   0
>  .../arm/include/asm}/tee/optee_msg.h          |   0
>  .../arm/include/asm}/tee/optee_rpc_cmd.h      |   0
>  .../arm/include/asm}/tee/optee_smc.h          |   0
>  .../arm/include/asm}/tee/tee.h                |   0
>  .../asm-arm => arch/arm/include/asm}/time.h   |   0
>  .../asm-arm => arch/arm/include/asm}/trace.h  |   0
>  .../asm-arm => arch/arm/include/asm}/traps.h  |   0
>  .../asm-arm => arch/arm/include/asm}/types.h  |   0
>  .../asm-arm => arch/arm/include/asm}/vfp.h    |   0
>  .../arm/include/asm}/vgic-emul.h              |   0
>  .../asm-arm => arch/arm/include/asm}/vgic.h   |   0
>  .../arm/include/asm}/vm_event.h               |   0
>  .../asm-arm => arch/arm/include/asm}/vpl011.h |   0
>  .../asm-arm => arch/arm/include/asm}/vpsci.h  |   0
>  .../asm-arm => arch/arm/include/asm}/vreg.h   |   0
>  .../asm-arm => arch/arm/include/asm}/vtimer.h |   0
>  .../arm/include/asm}/xenoprof.h               |   0
>  xen/arch/arm/smpboot.c                        |   2 +-
>  xen/arch/arm/vpsci.c                          |   2 +-
>  xen/arch/riscv/arch.mk                        |   1 -
>  .../riscv/include/asm}/config.h               |   0
>  xen/arch/riscv/riscv64/Makefile               |   1 +
>  xen/arch/x86/Makefile                         | 204 ++++-----
>  xen/arch/x86/Rules.mk                         |   4 +-
>  xen/arch/x86/arch.mk                          |  54 ++-
>  xen/arch/x86/boot/Makefile                    |  62 ++-
>  xen/arch/x86/boot/build32.mk                  |  40 --
>  xen/arch/x86/efi/Makefile                     |  15 +-
>  .../asm-x86 => arch/x86/include/asm}/acpi.h   |   0
>  .../x86/include/asm}/alternative-asm.h        |   0
>  .../x86/include/asm}/alternative.h            |   0
>  .../asm-x86 => arch/x86/include/asm}/altp2m.h |   0
>  .../asm-x86 => arch/x86/include/asm}/amd.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/apic.h   |   0
>  .../x86/include/asm}/apicdef.h                |   0
>  .../x86/include/asm}/asm-defns.h              |   0
>  .../x86/include/asm}/asm_defns.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/atomic.h |   0
>  .../asm-x86 => arch/x86/include/asm}/bitops.h |   0
>  .../asm-x86 => arch/x86/include/asm}/bug.h    |   0
>  .../x86/include/asm}/byteorder.h              |   0
>  .../x86/include/asm}/bzimage.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/cache.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/compat.h |   0
>  .../asm-x86 => arch/x86/include/asm}/config.h |   0
>  .../x86/include/asm}/cpufeature.h             |   0
>  .../x86/include/asm}/cpufeatures.h            |   0
>  .../x86/include/asm}/cpufeatureset.h          |   0
>  .../asm-x86 => arch/x86/include/asm}/cpuid.h  |   0
>  .../x86/include/asm}/cpuidle.h                |   0
>  .../x86/include/asm}/current.h                |   2 +-
>  .../x86/include/asm}/debugger.h               |   0
>  .../x86/include/asm}/debugreg.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/delay.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/desc.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/device.h |   0
>  .../asm-x86 => arch/x86/include/asm}/div64.h  |   0
>  .../x86/include/asm}/dom0_build.h             |   0
>  .../asm-x86 => arch/x86/include/asm}/domain.h |   2 +-
>  .../asm-x86 => arch/x86/include/asm}/e820.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/edd.h    |   0
>  .../x86/include/asm}/efibind.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/elf.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/event.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/fixmap.h |   0
>  .../x86/include/asm}/flushtlb.h               |   0
>  .../x86/include/asm}/genapic.h                |   0
>  .../x86/include/asm}/grant_table.h            |   0
>  .../asm-x86 => arch/x86/include/asm}/guest.h  |   0
>  .../x86/include/asm}/guest/hyperv-hcall.h     |   0
>  .../x86/include/asm}/guest/hyperv-tlfs.h      |   0
>  .../x86/include/asm}/guest/hyperv.h           |   0
>  .../x86/include/asm}/guest/hypervisor.h       |   0
>  .../x86/include/asm}/guest/pvh-boot.h         |   0
>  .../x86/include/asm}/guest/xen-hcall.h        |   0
>  .../x86/include/asm}/guest/xen.h              |   0
>  .../x86/include/asm}/guest_access.h           |   0
>  .../x86/include/asm}/guest_atomics.h          |   0
>  .../x86/include/asm}/guest_pt.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/hap.h    |   0
>  .../x86/include/asm}/hardirq.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/hpet.h   |   0
>  .../x86/include/asm}/hvm/asid.h               |   0
>  .../x86/include/asm}/hvm/cacheattr.h          |   0
>  .../x86/include/asm}/hvm/domain.h             |   0
>  .../x86/include/asm}/hvm/emulate.h            |   0
>  .../x86/include/asm}/hvm/grant_table.h        |   0
>  .../x86/include/asm}/hvm/guest_access.h       |   0
>  .../x86/include/asm}/hvm/hvm.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/hvm/io.h |   0
>  .../x86/include/asm}/hvm/ioreq.h              |   0
>  .../x86/include/asm}/hvm/irq.h                |   0
>  .../x86/include/asm}/hvm/monitor.h            |   0
>  .../x86/include/asm}/hvm/nestedhvm.h          |   0
>  .../x86/include/asm}/hvm/save.h               |   0
>  .../x86/include/asm}/hvm/support.h            |   0
>  .../x86/include/asm}/hvm/svm/asid.h           |   0
>  .../x86/include/asm}/hvm/svm/emulate.h        |   0
>  .../x86/include/asm}/hvm/svm/intr.h           |   0
>  .../x86/include/asm}/hvm/svm/nestedsvm.h      |   0
>  .../x86/include/asm}/hvm/svm/svm.h            |   0
>  .../x86/include/asm}/hvm/svm/svmdebug.h       |   0
>  .../x86/include/asm}/hvm/svm/vmcb.h           |   0
>  .../x86/include/asm}/hvm/trace.h              |   0
>  .../x86/include/asm}/hvm/vcpu.h               |   0
>  .../x86/include/asm}/hvm/vioapic.h            |   0
>  .../x86/include/asm}/hvm/viridian.h           |   0
>  .../x86/include/asm}/hvm/vlapic.h             |   0
>  .../x86/include/asm}/hvm/vm_event.h           |   0
>  .../x86/include/asm}/hvm/vmx/vmcs.h           |   0
>  .../x86/include/asm}/hvm/vmx/vmx.h            |   0
>  .../x86/include/asm}/hvm/vmx/vvmx.h           |   0
>  .../x86/include/asm}/hvm/vpic.h               |   0
>  .../x86/include/asm}/hvm/vpt.h                |   0
>  .../x86/include/asm}/hypercall.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/i387.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/init.h   |   0
>  .../x86/include/asm}/invpcid.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/io.h     |   0
>  .../x86/include/asm}/io_apic.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/iocap.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/iommu.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/ioreq.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/irq.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/ldt.h    |   0
>  .../x86/include/asm}/livepatch.h              |   0
>  .../x86/include/asm}/mach-default/bios_ebda.h |   0
>  .../x86/include/asm}/mach-default/io_ports.h  |   0
>  .../include/asm}/mach-default/irq_vectors.h   |   0
>  .../include/asm}/mach-default/mach_mpparse.h  |   0
>  .../include/asm}/mach-default/mach_mpspec.h   |   0
>  .../x86/include/asm}/mach-generic/mach_apic.h |   0
>  .../include/asm}/mach-generic/mach_mpparse.h  |   0
>  .../x86/include/asm}/machine_kexec.h          |   0
>  .../x86/include/asm}/mc146818rtc.h            |   0
>  .../asm-x86 => arch/x86/include/asm}/mce.h    |   0
>  .../x86/include/asm}/mem_access.h             |   0
>  .../x86/include/asm}/mem_paging.h             |   0
>  .../x86/include/asm}/mem_sharing.h            |   0
>  .../x86/include/asm}/microcode.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/mm.h     |   0
>  .../x86/include/asm}/monitor.h                |   0
>  .../asm-x86 => arch/x86/include/asm}/mpspec.h |   0
>  .../x86/include/asm}/mpspec_def.h             |   0
>  .../asm-x86 => arch/x86/include/asm}/msi.h    |   0
>  .../x86/include/asm}/msr-index.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/msr.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/mtrr.h   |   0
>  .../x86/include/asm}/multicall.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/mwait.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/nmi.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/nops.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/nospec.h |   0
>  .../asm-x86 => arch/x86/include/asm}/numa.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/p2m.h    |   0
>  .../x86/include/asm}/page-bits.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/page.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/paging.h |   0
>  .../asm-x86 => arch/x86/include/asm}/pci.h    |   0
>  .../asm-x86 => arch/x86/include/asm}/percpu.h |   0
>  .../asm-x86 => arch/x86/include/asm}/perfc.h  |   0
>  .../x86/include/asm}/perfc_defn.h             |   0
>  .../x86/include/asm}/processor.h              |   0
>  .../asm-x86 => arch/x86/include/asm}/psr.h    |   0
>  .../x86/include/asm}/pv/domain.h              |   0
>  .../x86/include/asm}/pv/grant_table.h         |   0
>  .../asm-x86 => arch/x86/include/asm}/pv/mm.h  |   0
>  .../x86/include/asm}/pv/shim.h                |   0
>  .../x86/include/asm}/pv/traps.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/random.h |   0
>  .../asm-x86 => arch/x86/include/asm}/regs.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/setup.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/shadow.h |   0
>  .../asm-x86 => arch/x86/include/asm}/shared.h |   0
>  .../asm-x86 => arch/x86/include/asm}/smp.h    |   0
>  .../x86/include/asm}/softirq.h                |   0
>  .../x86/include/asm}/spec_ctrl.h              |   0
>  .../x86/include/asm}/spec_ctrl_asm.h          |   0
>  .../x86/include/asm}/spinlock.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/string.h |   0
>  .../asm-x86 => arch/x86/include/asm}/system.h |   0
>  .../asm-x86 => arch/x86/include/asm}/tboot.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/time.h   |   0
>  .../asm-x86 => arch/x86/include/asm}/trace.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/traps.h  |   0
>  .../asm-x86 => arch/x86/include/asm}/types.h  |   0
>  .../x86/include/asm}/uaccess.h                |   0
>  .../x86/include/asm}/unaligned.h              |   0
>  .../x86/include/asm}/vm_event.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/vpmu.h   |   0
>  .../x86/include/asm}/x86-defns.h              |   0
>  .../x86/include/asm}/x86-vendors.h            |   0
>  .../x86/include/asm}/x86_64/efibind.h         |   0
>  .../x86/include/asm}/x86_64/elf.h             |   0
>  .../x86/include/asm}/x86_64/page.h            |   0
>  .../x86/include/asm}/x86_64/regs.h            |   0
>  .../x86/include/asm}/x86_64/system.h          |   0
>  .../x86/include/asm}/x86_64/uaccess.h         |   0
>  .../x86/include/asm}/x86_emulate.h            |   2 +-
>  .../x86/include/asm}/xenoprof.h               |   0
>  .../asm-x86 => arch/x86/include/asm}/xstate.h |   0
>  xen/arch/x86/mm/Makefile                      |   9 -
>  xen/arch/x86/mm/guest_walk.c                  |   3 -
>  xen/arch/x86/mm/guest_walk_2.c                |   2 +
>  xen/arch/x86/mm/guest_walk_3.c                |   2 +
>  xen/arch/x86/mm/guest_walk_4.c                |   2 +
>  xen/arch/x86/mm/hap/Makefile                  |  15 +-
>  xen/arch/x86/mm/hap/guest_walk.c              |   3 -
>  xen/arch/x86/mm/hap/guest_walk_2.c            |   2 +
>  xen/arch/x86/mm/hap/guest_walk_3.c            |   2 +
>  xen/arch/x86/mm/hap/guest_walk_4.c            |   2 +
>  xen/arch/x86/mm/shadow/Makefile               |   9 -
>  xen/arch/x86/mm/shadow/guest_2.c              |   2 +
>  xen/arch/x86/mm/shadow/guest_3.c              |   2 +
>  xen/arch/x86/mm/shadow/guest_4.c              |   2 +
>  xen/arch/x86/mm/shadow/multi.c                |   3 -
>  xen/build.mk                                  |  87 ++++
>  xen/common/Makefile                           |  14 +-
>  xen/common/efi/runtime.c                      |   2 +-
>  xen/common/libelf/Makefile                    |   4 +-
>  xen/common/libfdt/Makefile                    |   8 +-
>  xen/common/page_alloc.c                       |   2 +-
>  xen/include/Makefile                          |  81 ++--
>  xen/include/xen/acpi.h                        |   5 +-
>  xen/include/xen/bitmap.h                      |   2 +-
>  xen/scripts/Kbuild.include                    |  65 ++-
>  xen/scripts/Kconfig.include                   |   2 +-
>  xen/scripts/Makefile.clean                    |  40 +-
>  xen/{tools/kconfig => scripts}/Makefile.host  |  29 +-
>  xen/scripts/mkmakefile                        |  17 +
>  xen/test/Makefile                             |  16 +-
>  xen/test/livepatch/Makefile                   | 213 ++++-----
>  xen/tools/Makefile                            |  14 +-
>  xen/tools/compat-build-header.py              |  44 +-
>  xen/tools/compat-build-source.py              |   8 +-
>  xen/tools/fixdep.c                            | 404 +++++++++++++++++
>  xen/tools/kconfig/Makefile.kconfig            | 106 -----
>  xen/tools/kconfig/confdata.c                  |   2 +-
>  xen/tools/symbols.c                           |  18 +-
>  xen/xsm/flask/Makefile                        |  45 +-
>  xen/xsm/flask/policy/mkaccess_vector.sh       |   7 +-
>  xen/xsm/flask/ss/Makefile                     |   5 +-
>  378 files changed, 1572 insertions(+), 978 deletions(-)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/acpi.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/alternative.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/altp2m.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/atomic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/bitops.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/bug.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/cmpxchg.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/flushtlb.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/insn.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/io.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/macros.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/mm.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/page.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/processor.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/sysregs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/system.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/traps.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm32/vfp.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/atomic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/bitops.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/brk.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/bug.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/cmpxchg.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/efibind.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/flushtlb.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/hsr.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/insn.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/io.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/macros.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/mm.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/page.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/processor.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/sysregs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/system.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/traps.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/arm64/vfp.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/asm_defns.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/atomic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/bitops.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/bug.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/byteorder.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/cache.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/cadence-uart.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/config.h (99%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/cpregs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/cpuerrata.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/cpufeature.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/current.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/debugger.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/delay.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/desc.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/device.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/div64.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/domain.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/domain_build.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/early_printk.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/efibind.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/elf.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/event.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/exynos4210-uart.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/flushtlb.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/gic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/gic_v3_defs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/gic_v3_its.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/grant_table.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/guest_access.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/guest_atomics.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/guest_walk.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/hardirq.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/hsr.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/hypercall.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/init.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/insn.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/io.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/iocap.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/iommu.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/iommu_fwspec.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/ioreq.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/irq.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/kernel.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/livepatch.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/lpae.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/macros.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/mem_access.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/mm.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/mmio.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/monitor.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/new_vgic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/nospec.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/numa.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/p2m.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/page-bits.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/page.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/paging.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/pci.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/percpu.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/perfc.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/perfc_defn.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/pl011-uart.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platform.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platforms/exynos5.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platforms/midway.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platforms/omap5.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platforms/vexpress.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/platforms/xilinx-zynqmp-eemi.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/processor.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/procinfo.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/psci.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/random.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/regs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/scif-uart.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/setup.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/short-desc.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/smccc.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/smp.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/softirq.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/spinlock.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/string.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/sysregs.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/system.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/tee/optee_msg.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/tee/optee_rpc_cmd.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/tee/optee_smc.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/tee/tee.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/time.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/trace.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/traps.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/types.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vfp.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vgic-emul.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vgic.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vm_event.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vpl011.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vpsci.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vreg.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/vtimer.h (100%)
>  rename xen/{include/asm-arm => arch/arm/include/asm}/xenoprof.h (100%)
>  rename xen/{include/asm-riscv => arch/riscv/include/asm}/config.h (100%)
>  create mode 100644 xen/arch/riscv/riscv64/Makefile
>  delete mode 100644 xen/arch/x86/boot/build32.mk
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/acpi.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/alternative-asm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/alternative.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/altp2m.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/amd.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/apic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/apicdef.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/asm-defns.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/asm_defns.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/atomic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/bitops.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/bug.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/byteorder.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/bzimage.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cache.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/compat.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/config.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cpufeature.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cpufeatures.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cpufeatureset.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cpuid.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/cpuidle.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/current.h (99%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/debugger.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/debugreg.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/delay.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/desc.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/device.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/div64.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/dom0_build.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/domain.h (99%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/e820.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/edd.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/efibind.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/elf.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/event.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/fixmap.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/flushtlb.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/genapic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/grant_table.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/hyperv-hcall.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/hyperv-tlfs.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/hyperv.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/hypervisor.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/pvh-boot.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/xen-hcall.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest/xen.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest_access.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest_atomics.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/guest_pt.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hap.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hardirq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hpet.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/asid.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/cacheattr.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/domain.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/emulate.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/grant_table.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/guest_access.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/hvm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/io.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/ioreq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/irq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/monitor.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/nestedhvm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/save.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/support.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/asid.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/emulate.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/intr.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/nestedsvm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/svm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/svmdebug.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/svm/vmcb.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/trace.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vcpu.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vioapic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/viridian.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vlapic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vm_event.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vmx/vmcs.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vmx/vmx.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vmx/vvmx.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vpic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hvm/vpt.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/hypercall.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/i387.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/init.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/invpcid.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/io.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/io_apic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/iocap.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/iommu.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/ioreq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/irq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/ldt.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/livepatch.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-default/bios_ebda.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-default/io_ports.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-default/irq_vectors.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-default/mach_mpparse.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-default/mach_mpspec.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-generic/mach_apic.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mach-generic/mach_mpparse.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/machine_kexec.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mc146818rtc.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mce.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mem_access.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mem_paging.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mem_sharing.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/microcode.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/monitor.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mpspec.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mpspec_def.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/msi.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/msr-index.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/msr.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mtrr.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/multicall.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/mwait.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/nmi.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/nops.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/nospec.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/numa.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/p2m.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/page-bits.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/page.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/paging.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pci.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/percpu.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/perfc.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/perfc_defn.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/processor.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/psr.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pv/domain.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pv/grant_table.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pv/mm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pv/shim.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/pv/traps.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/random.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/regs.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/setup.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/shadow.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/shared.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/smp.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/softirq.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/spec_ctrl.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/spec_ctrl_asm.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/spinlock.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/string.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/system.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/tboot.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/time.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/trace.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/traps.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/types.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/uaccess.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/unaligned.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/vm_event.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/vpmu.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86-defns.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86-vendors.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/efibind.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/elf.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/page.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/regs.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/system.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_64/uaccess.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/x86_emulate.h (89%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/xenoprof.h (100%)
>  rename xen/{include/asm-x86 => arch/x86/include/asm}/xstate.h (100%)
>  create mode 100644 xen/arch/x86/mm/guest_walk_2.c
>  create mode 100644 xen/arch/x86/mm/guest_walk_3.c
>  create mode 100644 xen/arch/x86/mm/guest_walk_4.c
>  create mode 100644 xen/arch/x86/mm/hap/guest_walk_2.c
>  create mode 100644 xen/arch/x86/mm/hap/guest_walk_3.c
>  create mode 100644 xen/arch/x86/mm/hap/guest_walk_4.c
>  create mode 100644 xen/arch/x86/mm/shadow/guest_2.c
>  create mode 100644 xen/arch/x86/mm/shadow/guest_3.c
>  create mode 100644 xen/arch/x86/mm/shadow/guest_4.c
>  create mode 100644 xen/build.mk
>  rename xen/{tools/kconfig => scripts}/Makefile.host (88%)
>  create mode 100755 xen/scripts/mkmakefile
>  create mode 100644 xen/tools/fixdep.c
>  delete mode 100644 xen/tools/kconfig/Makefile.kconfig
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 09:59:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 09:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187434.336286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRhX-00087d-Sd; Wed, 15 Sep 2021 09:59:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187434.336286; Wed, 15 Sep 2021 09:59:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRhX-00087W-PT; Wed, 15 Sep 2021 09:59:35 +0000
Received: by outflank-mailman (input) for mailman id 187434;
 Wed, 15 Sep 2021 09:59:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQRhV-00087M-Nk; Wed, 15 Sep 2021 09:59:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQRhV-00030i-Jf; Wed, 15 Sep 2021 09:59:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQRhV-0003EF-BZ; Wed, 15 Sep 2021 09:59:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQRhV-0004bq-B8; Wed, 15 Sep 2021 09:59:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BJ8N+wvGN7xQPMtvn2byw9Nso/jluF76F5ucJy7afmg=; b=qfN5pvfHtcoRe6Hf/355odezBR
	eFOasBU0ruAlFzHkjvt/L9l3w/oKgsNt+Qsn+4JuXK6Cgwouzlr1/JhQLuqAx6v2o9gCsbfM1Od4K
	5IkYBs4gPExmtB6pV1BRwmSBywr8MRgzJMpOu67wmQFB2KHSvfsHm3gjUzpi1wW0nuVs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164994-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 164994: all pass - PUSHED
X-Osstest-Versions-This:
    xen=487975df53b5298316b594550c79934d646701bd
X-Osstest-Versions-That:
    xen=6d45368a0a89e01a3a01d156af61fea565db96cc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 09:59:33 +0000

flight 164994 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164994/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  487975df53b5298316b594550c79934d646701bd
baseline version:
 xen                  6d45368a0a89e01a3a01d156af61fea565db96cc

Last test of basis   164955  2021-09-12 09:19:39 Z    3 days
Testing same since   164994  2021-09-15 09:19:43 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6d45368a0a..487975df53  487975df53b5298316b594550c79934d646701bd -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 10:06:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 10:06:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187443.336300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRo2-0001BP-LU; Wed, 15 Sep 2021 10:06:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187443.336300; Wed, 15 Sep 2021 10:06:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQRo2-0001BI-Hj; Wed, 15 Sep 2021 10:06:18 +0000
Received: by outflank-mailman (input) for mailman id 187443;
 Wed, 15 Sep 2021 10:06:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQRo1-0001BC-J5
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 10:06:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 90347723-160c-11ec-b4f5-12813bfff9fa;
 Wed, 15 Sep 2021 10:06:16 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2053.outbound.protection.outlook.com [104.47.0.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-cZJSWAaCMu2gAYE6nDkJEw-1; Wed, 15 Sep 2021 12:06:13 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 10:06:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 10:06:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P195CA0091.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:86::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 10:06:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90347723-160c-11ec-b4f5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631700375;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4q+qC6paFk7/bef8SZIkopLvKrBsYyd+6Xvwu8ov/lg=;
	b=iyszW3yaiKanrePj2IAPaqCE+WWIEforvGk+9U4YCdqXo0sAV/LneFCVXg/MaoYT0u7SQe
	c70Ji8ZD5NhXM9DLnlJFAm3o/yi4HIf1nDZKLaozUZux76Ms0ITlvXOD9tomBxM7bMaNv9
	KHL0ju7M0/DzfZ+SBPHLFR6Lp2ScS/I=
X-MC-Unique: cZJSWAaCMu2gAYE6nDkJEw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KrCeiHYwwCeRVy3hZNIIj0k+45s1N+JwBRym8JqNaB37Rb9o6dCDwC0wn9RDi430VkHxkppCT3Cg1ScO48c3bSFIs4RYyC1KQhGhyMvZG7clmevQwj1Z7RKGXLPCYA2dszop7tPu8IfzLhrVLMuNqhKObBE5G1aW9C6EzwbokntdHnM+S4lrllJXENGrQvqNv8Ag1jqdfio8N0hSH1cYNq1iNA5g89r5tTEgwKM7cLg3j+Q7MgYHxoN3XEUmMp6KKjFOlU08zoYHCEjRFUdLbASt/5wP8/RnT/c8vhj5WIqMb5EeKmBXOcNEU7wk5tyZGBXP1zKWbNljvnGap80xgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4q+qC6paFk7/bef8SZIkopLvKrBsYyd+6Xvwu8ov/lg=;
 b=HixSVFkmjkvxm5FqD6qfsh5oAN1vvq5k+s8KfDyi5wbUZijs9E7J7fXpaxHDTcxWAYRDSiWb54rhctzxsLTdGu+nEWN1MuwyJz3XFvdvZbOQ88DLSS3IqmFYbQ3Nf3b1D7Onw5EVGzWvUSH3qWRv5ahED52y3eqxysZljjUkUeP2iIpcYAA8qUL75vFyesrMFOBI1WabljbhUfC7nc3XOXN6qhhUaspTOXb5J0lIEz1cCCJO+9KoiLdFQY6Dbsv/fF5oOCF1WXaCYrSzm7SODXDLL9gC+sLGWLbVzln8PA7WsvDAaOanePJozVm5jzJeELUIxFwyw/yJXg9dgk3heQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH V2] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b6744333-4d43-ef24-0f9b-b5cd54680660@suse.com>
Date: Wed, 15 Sep 2021 12:06:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P195CA0091.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:209:86::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ecb48d85-4a34-4b8e-6e4b-08d9783071c5
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB715297E410A35B94ED708EFEB3DB9@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wYGEIuOAHypYyZN9S7cDbfCsJDYlhazQJmRhxw/LvkxaWjjC22BzoKIPnzBsBE/civkhz+CQwapiS9zZfZt86UbcixLWaElo3JmkFFe3W7jw/Q2Ij5RlWupOE2byo9HTeK/SjfEvWX2Kh/rzX9vq01mxA69gxhwgc011JhljCjHBdnwwsXmNRsCCtQZZbTSvq5m0tCfH7JsS7p2zJJMCqbQe+3Q5ku0yc/Fxgfk3HS3iz+8Ki31gZ42TJxPjgDzsFbqrAOlDa/Tiw4vZWF4Eob1eOBwygkyPV8SAHg2mbPHc+lvw4hf+N/xbCbYmtS3OYpmNpF6sBW0UKalhoppcF2+WEXYcVvzVWvZfgvt+yOWnhYALQAGkHoz3Ck2z9Nw3oTwS2FEjm3Iir2F7HGIdIIin1UkQP9yrsF6Wl0lN5GE0FEn79RJ6gJFkPbCj9zax8zB8SljCCSK2Mok/JnameZQkDWyJ84kTiUolpOY3FFUrVkaqzdfB9qkD95T40w5JbyLttPZRBHig8VsMHUcNlZ78VdPinMpXMcMT1df/ZQr7trqunb3GYsEHxPw3sG54MNvx8Mh8NS8kD9eu5EXVfAQLlxI/QdO+IOmUFOGErko8fR7zMoObu0yZc5zCAIFxCikwfdpC8sa8Eg3VjFq/e3Xj43lJMiABY7x7UftaDVkOYLu64UQhWpsBNnR22iBQhbirEVNeXpzETphhEl3MDwKKUs1Ah9+srnnQaw4M2k7p4RtyWSsw1ht/EqwcKEfH
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(366004)(376002)(136003)(346002)(5660300002)(4326008)(86362001)(83380400001)(8676002)(956004)(66476007)(54906003)(316002)(16576012)(66946007)(2616005)(66556008)(7416002)(36756003)(6486002)(53546011)(26005)(38100700002)(31686004)(31696002)(6916009)(2906002)(478600001)(186003)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZUQvVUlLRlBuMTdlRUk1U2hlRnlDajliTTNNeXNwT2ltVXZpdUpLUlZtc2R0?=
 =?utf-8?B?cnVtZlBiYnIwN0JTRnh3Y0pScGsvUk1xMGRQZCtaK1NDZ0NQVk5WcEUvK0JD?=
 =?utf-8?B?WWRhWWtnM2hZSzBwMXVMaUpXdFE5Q2JJY2FoZENpTGNYM01jL25WWkNmZCtR?=
 =?utf-8?B?aUZKMTF6WUk4ZEFDamZpT2hpSHVUQmdWTWxRS05CRnJpa00rUyt1bUxXZVkx?=
 =?utf-8?B?d2ZWYlpCSURCLzBTN2IwTE5nT1lZTmNub1ZIRURsdlc0YjdUMTVTVXFxTkNE?=
 =?utf-8?B?STV2MWsrTnpYQ0V0dU12Q1pmTVFkNEtFaXlXdG0rMlRWeUxrcDJxbmJBY2pS?=
 =?utf-8?B?RW1lV1MyWGNNMXdNL0FIbThaUFN4WjA3Mm5pZjZLdmdhZ0UrNzFhVnBYcGVa?=
 =?utf-8?B?YzFBdDFUaXFBb3FXWmlRVVZRdHNXbmVuamwrNkJjVHpkNEpPcy9MTzNjaWdZ?=
 =?utf-8?B?YVBwdE40Q213bkROZmUvR09VZ0Z0U1BXZHhXb1JOQnlycVkxUWhocGlidnVx?=
 =?utf-8?B?d2FYT1pzRkdxZkc3R3l4K1pMSUZBQ2pwb1gwZVduVTZqUE44ZkdmdDIxZFpE?=
 =?utf-8?B?YmplQ2p4cEpXcklla1cxZ1BSVFdPaTgxVlI0RU1KQ0tPUU1EK2w2N3ZhWElS?=
 =?utf-8?B?N0o4MHBjckk5REFJU1FBVmFacG00bFpwNERYeEhFb3RmYmozcGhndU5EeW10?=
 =?utf-8?B?KzJsVWpLWUZFZVdMaDlOZitZeUk1ZFg5ZkIrb3IyUHVZcFNaMHZmQ3UvT0Jz?=
 =?utf-8?B?WlkweE53WU1FOEdtWHdjZ05uZWptc2RadXlqcm1SenQ0SHdYeDJmaUhJK0hG?=
 =?utf-8?B?eTMyNElyaE5CNW4wY2JpbkptSXNmMXhjaDVzYlFZV3NNZHpiTE1WUlFOVFcw?=
 =?utf-8?B?S0pZUVI4eVBNU0VWRk1Kd2FZNjQybU1mWHVDU0pjZGlNVDZQNnFoOWpIVGVu?=
 =?utf-8?B?ODU4S1QxZzB4SnZzUEpyZjZSZG1JQXNkYWZhTko2bWY0cFhzM3ZsaXVzUlN5?=
 =?utf-8?B?RytWbFRkd2RFdCtkRmVublBxbEdlSWZXczd4cytLa2FyU1hraGNzWlN3OHI5?=
 =?utf-8?B?N2RzWDZubmg0bHlReFhRQVBVL3lKTDQ4QWw2Rkh0QjdOZWNkTXBHQWRBQjh2?=
 =?utf-8?B?WktmQisxejZhR3NBMVFQUjVubk9RRERSYUtvQmFTd1lUOVdQZ0dLK1dIYTNG?=
 =?utf-8?B?RW0rMktXMFRONERTVmU2TEYyTXRNdUFGaEZWWlFwcjV4YmErSlZXQmYzcisw?=
 =?utf-8?B?UVd4TVF3VXNrVlMyMUVmYkJudFBVZGY0bllHTDEvOExYZmdwSVY3MFcyMHFU?=
 =?utf-8?B?MWFDdFk0VjVFVU5wOFAyY0pUZVIzbUp1WDA3b0FyZFlsNEZmVEprZldrSzVk?=
 =?utf-8?B?OUZ6cnBHaUV0cXA4aFVTOEljWjRCczlaTmozZGlZdkc1bG96aHN0c3pwV0Ux?=
 =?utf-8?B?Zy94SHBVVU14SEprSTFrRmo0WEhWUlFjc3daN0xITkx5S2JycUN5a2ZsWk1a?=
 =?utf-8?B?RThxcnQwOXhpeGVPU0Y2Zm05bjdZS3N0YWdEL0Y2RDM4bkIrMU1mMjNiZjhO?=
 =?utf-8?B?N25DV3hGb05CZ3RjRnpMWkFTUktXdTVPRTVNUWhyZG5PR0xlNS9jM1h0M25Y?=
 =?utf-8?B?UlJ4dS9lWFZoMjJEOWRvYUNaM3ZNQjY3UjlYRTFybG9RbG9VT1A2SE50MTFs?=
 =?utf-8?B?QTFVbThNalV0OWxJSk82R3RyTW84Uy9XczB4Ym43b0h6ZEtJUTNHbVB0STVD?=
 =?utf-8?Q?pUGH+Wv0FmSMHt7fP6H0bcHqAn+LGZ7bIjfimPN?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ecb48d85-4a34-4b8e-6e4b-08d9783071c5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 10:06:11.3032
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2d1g9fU3hSml/FVQVfYR9mRMMev1FAzPmIprzMI3kEzIAcRMMlFK7vmskE568GvHupo27advKCMsKoSGsJuCyA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

On 14.09.2021 22:44, Oleksandr Tyshchenko wrote:
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -1376,14 +1376,18 @@ unsigned long domain_get_maximum_gpfn(struct domain *d)
>  void share_xen_page_with_guest(struct page_info *page, struct domain *d,
>                                 enum XENSHARE_flags flags)
>  {
> +    unsigned long type_info;
> +
>      if ( page_get_owner(page) == d )
>          return;
>  
>      spin_lock(&d->page_alloc_lock);
>  
>      /* The incremented type count pins as writable or read-only. */
> -    page->u.inuse.type_info =
> -        (flags == SHARE_ro ? PGT_none : PGT_writable_page) | 1;
> +    type_info = page->u.inuse.type_info & ~(PGT_type_mask | PGT_count_mask);
> +    page->u.inuse.type_info = type_info |
> +        (flags == SHARE_ro ? PGT_none : PGT_writable_page) |
> +        (1UL << PGT_count_base);

Just as a note: If this was x86 code, I'd request the redundant
PGT_count_base to be dropped. Constructs like the above is what we
have MASK_INSR() for.

> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2204,7 +2204,10 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>          return NULL;
>  
>      for ( i = 0; i < (1u << order); i++ )
> +    {
>          pg[i].count_info |= PGC_xen_heap;
> +        arch_alloc_xenheap_page(&pg[i]);
> +    }
>  
>      return page_to_virt(pg);
>  }
> @@ -2222,7 +2225,10 @@ void free_xenheap_pages(void *v, unsigned int order)
>      pg = virt_to_page(v);
>  
>      for ( i = 0; i < (1u << order); i++ )
> +    {
>          pg[i].count_info &= ~PGC_xen_heap;
> +        arch_free_xenheap_page(&pg[i]);
> +    }
>  
>      free_heap_pages(pg, order, true);
>  }

You look to only be adjusting the !SEPARATE_XENHEAP instances of the
functions. Isn't 32-bit Arm using SEPARATE_XENHEAP?

> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -98,9 +98,18 @@ struct page_info
>  #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>  #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>  
> - /* Count of uses of this frame as its current type. */
> -#define PGT_count_width   PG_shift(2)
> -#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
> + /* 3-bit count of uses of this frame as its current type. */
> +#define PGT_count_base    PG_shift(4)
> +#define PGT_count_mask    PG_mask(7, 4)
> +
> +/*
> + * Stored in bits [27:0] or [59:0] GFN if page is used for grant table frame.

I don't know enough Arm details to tell whether this is properly
one bit more than the maximum number of physical address bits.
Without the extra bit you wouldn't be able to tell apart a
guest specified GFN matching the value of PGT_INVALID_FRAME_GFN
from an entry which was set from INVALID_GFN.

> + * This only valid for the xenheap pages.
> + */
> +#define PGT_gfn_width     PG_shift(4)
> +#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)

Any reason you don't use PG_mask() here? Any open-coding is prone
to people later making mistakes.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 10:35:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 10:35:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187451.336311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSFi-0004du-Rc; Wed, 15 Sep 2021 10:34:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187451.336311; Wed, 15 Sep 2021 10:34:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSFi-0004dn-OM; Wed, 15 Sep 2021 10:34:54 +0000
Received: by outflank-mailman (input) for mailman id 187451;
 Wed, 15 Sep 2021 10:34:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQSFh-0004dd-Cx; Wed, 15 Sep 2021 10:34:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQSFh-0003gC-4b; Wed, 15 Sep 2021 10:34:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQSFg-0004b7-Nw; Wed, 15 Sep 2021 10:34:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQSFg-0000qG-NR; Wed, 15 Sep 2021 10:34:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pP9qwtw5eM+itMRNol8//CuH4tqOJNp/lIIkKNpLsic=; b=o8GhaPJwDLQLfQ9E/hqH55Akje
	L11lFD7nzyjrGasLwbC6XyYXWTSFgf/dUiMmZtdgW7/GGDjpz9XaaRKZbVDI7icQal2UOAIsKh6/0
	Qdy8qtj23bTRRhtvnmHKW4jzADeqR6lYGcGsIRr4muokL0ir8woibYJqqfivI9/N2KYw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164990-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 164990: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=0cb4550305e5a798439f06c3023da00e0e3b5b7c
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 10:34:52 +0000

flight 164990 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164990/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              0cb4550305e5a798439f06c3023da00e0e3b5b7c
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  432 days
Failing since        151818  2020-07-11 04:18:52 Z  431 days  422 attempts
Testing same since   164990  2021-09-15 04:20:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73485 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 10:46:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 10:46:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187462.336325 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSQn-0006H4-2s; Wed, 15 Sep 2021 10:46:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187462.336325; Wed, 15 Sep 2021 10:46:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSQm-0006Gx-W4; Wed, 15 Sep 2021 10:46:20 +0000
Received: by outflank-mailman (input) for mailman id 187462;
 Wed, 15 Sep 2021 10:46:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LStB=OF=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQSQl-0006Gr-80
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 10:46:19 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.58]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 155e62e8-588f-4bb9-a083-482e3a0a5509;
 Wed, 15 Sep 2021 10:46:15 +0000 (UTC)
Received: from AM6PR0502CA0049.eurprd05.prod.outlook.com
 (2603:10a6:20b:56::26) by VI1PR0802MB2429.eurprd08.prod.outlook.com
 (2603:10a6:800:ba::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 10:46:14 +0000
Received: from AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:56:cafe::8) by AM6PR0502CA0049.outlook.office365.com
 (2603:10a6:20b:56::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Wed, 15 Sep 2021 10:46:14 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT003.mail.protection.outlook.com (10.152.16.149) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 10:46:13 +0000
Received: ("Tessian outbound 8e26f7114b75:v103");
 Wed, 15 Sep 2021 10:46:12 +0000
Received: from 3ba332429c71.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D8BC9BC8-49DD-4066-BD88-52E93A6E70F0.1; 
 Wed, 15 Sep 2021 10:46:04 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3ba332429c71.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 Sep 2021 10:46:04 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1874.eurprd08.prod.outlook.com (2603:10a6:203:49::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 10:46:00 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 10:45:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 155e62e8-588f-4bb9-a083-482e3a0a5509
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oGKW/gOUVgzzW5DtTrQAoowlwet+WhTAaJlhFN8KcOQ=;
 b=MxAHddygq4WnAPubJ1aCLS2yNQAyLBSeE5kbwdN3d1gPcI2YAQWUdB7q6uXcuqC87BC0NfWDbTN7peeJBbPxif0a0oWoLw9I3iM+pLiDwWWn0dT36IHIoVNIZswcQOj3jBluZWDXMS8ldd9vM7TMa/Qx8oKLmQ3alsKpLOxsFYQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2f58c133a6dece66
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SZqvBZthUKsvayXUhMcteLFnKCAMEdZimPqmdc24dG8UoIFcvODoh4Rx/0NaIV+tIas7BHTfv3Bqp3yDuGjmvriKJoK2/MlkGA4nxYOqTfu+JZ2tN1l2jW+y7l1Sk9WMMpD3tM9iEWaSYZaNpBUqRJeGa+q8RFHeIRhPMLcdDGZyTZNnGba1QuNUBqOQmPQI19Zb+HZuskrdVJPEgfwolbDuan6/o/cH+VNfrvq+/UrTWT4vju5/wbxUgffLCCfAvHaGsqlkh++p13q0m4cCwSE1I8B0zfbwoi1FlBBJFQF+EYtwI5DDQC7O/qsxMiKaswO/+Mu4MHStPwDGaAujrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oGKW/gOUVgzzW5DtTrQAoowlwet+WhTAaJlhFN8KcOQ=;
 b=gsdt5OOdRaCIvreOvXvOQRk7mJrNQS9hOYl8GsiXdWQrRfCa8VhxTHb8mT+ZpZOt5vR3mG4diDu816I0ZyOpi3hdisFOIrWa/XtkbON1v46yJGhPkXJOULdy3BOJ0e5+T92RFe18MS2UvTdINp8BukxQW/RIDzFgHI2LtOKHWitknB5fDOeFY28jdwmLTQUpjqQcXKEoGD4lpoa3LwNQiglD122Px4e9/0N4EAQRrKYwQqwtq1Vl+6QXvV+WtcWxI/qDdZ2pPz/5nQNiHQgWZWPwk9CfuZfw/ThsoUMbapjsyXaTnvMAnVlM201LstoDYhT7IhguJylGKm+EvW1QAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oGKW/gOUVgzzW5DtTrQAoowlwet+WhTAaJlhFN8KcOQ=;
 b=MxAHddygq4WnAPubJ1aCLS2yNQAyLBSeE5kbwdN3d1gPcI2YAQWUdB7q6uXcuqC87BC0NfWDbTN7peeJBbPxif0a0oWoLw9I3iM+pLiDwWWn0dT36IHIoVNIZswcQOj3jBluZWDXMS8ldd9vM7TMa/Qx8oKLmQ3alsKpLOxsFYQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ57xrO/H5MzpEqKB3+2EFX1LqudvyUAgAXn+wCAAP1GAIAAWBqA
Date: Wed, 15 Sep 2021 10:45:59 +0000
Message-ID: <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
 <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
 <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
In-Reply-To: <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ee5798ec-4000-40cc-3b3d-08d9783609b2
x-ms-traffictypediagnostic: AM5PR0801MB1874:|VI1PR0802MB2429:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0802MB2429FAA595D4A5CD9EECE9B1FCDB9@VI1PR0802MB2429.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 TSyFtXjiXlZ96HTrFqTKF89wK5JvskoIsUHK5Il3n9iF1I1N+vk7MM6NVieJGv+4CNpAQ8OmfQO8x72ghWgsDPGGlyknLnCSB8VtICVAz2nA1SxoUcrSZI9sgsw8F8DFSV407mpLGD7CcJIz9d2CQ0cvfegqyV6MlER8ds++OMHYQhdKF2Nye5hjKIC1XjkdQJbcwZIhHhl4Rz4PgOa/nmnRBGTKSIlC6a2KT2VxhUWnbM2qgxM27h5BQx4/KevLKCSq6XdZrC8OrijWyEZ2L8Ev2i3SzJZrW26tl2JNIXW7XzXAWDvPuNkHMxiq0jlwa0KQx4DhZkpPjel26+4k0K5DjWpiNAurkes+fm+0/o/HxzQ4+NmyyyNQ3d/5NpcQZNNNP4bhvU3rOP76/uRXrB8LNxbkrjKrtSlwxabWILrN/UTFeKjH9F6ZwPenzymokhgBA6wnYaL7WdWNqU+gGYbRmgGyYn4sxEmK0Qtq4b3P+c/MHmmP1qvskP7UimMVGmVIPBnOqaOVs2zSPSuFdO7tD53Du3XSy828X6k6CQ8KFG7eo9yUK/+cgi3uGN22mEFtsQ5TV7EmsI87ofFa5b3FJmULXC9ITjo1wl9SyoODaQLWddblrLnhjiEcneqHjaWt4BnFDwVpyBq+8J3gt6k63xa49nZ/WZuTn7lw0ea5SJTmIAm13P2pzTJepHWCOw4aqyJuh75b1NrmmrfuacrmzyPNkt+EDVHRdTbrWyI//q1xmQWu0uWlzQG4sT59PJtrR1fP00HWGGzCyPIE2oG2IA2SqhXoi3J1ULwyQDVrVZ3YuZFB25NGtsX9EdpUkipDzxTURojvvP8biLFm7w==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(366004)(346002)(396003)(39860400002)(478600001)(38100700002)(83380400001)(2616005)(66946007)(66476007)(38070700005)(186003)(26005)(71200400001)(76116006)(966005)(5660300002)(64756008)(66556008)(66446008)(6512007)(8936002)(36756003)(91956017)(6506007)(53546011)(4326008)(2906002)(122000001)(6486002)(54906003)(6916009)(8676002)(316002)(86362001)(33656002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7BAB166D3F43E242B61ADD1EC0E0A731@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1874
Original-Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8cfa7acd-1ffd-4300-8b71-08d97836018e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GqOg/gFCAAwWnGH5N56v56ur2Fmj8eW5SZQlpHnvX1sLjuaD+lt505KEUNwYb4c8P7Z9FDMXG2Jahi825+xLxTjG+q3I26Fzko5hwRSvMXuAJmxcZGxZm8yybCnwRUXmHIM8mqvr/wvyprOZgsDsTUyJvp0b5XqzWsHUXFd+asu8V6BMgfOP5yZNKcvZof4R2iOhOgfI1y9gqxr1e+cg+Ap5DDdNjbku9GjCvtH3CkNp9Zn12dO5/F6cgh+r7gg/zHprubCsxpUe1mghSqybhq4k/RT0isiogzjZ/mCNk1Q44RF8Cjq4ViirEM9Aee6XgN56bzj5Gkuxw2BuuM3RKm3pDuWFCW4pAg5cAZyTylZopwdDFTAN6/ofwejS4NrZNLU3dEcCtrvsPsGPf1A0Fa+5lQY6hiCrEuaQa2bg2y3+25C94fxBFZUDnuE8HK1OLugSljXnEyPixd1UvgTpxXOeutoqfKORQr2hLeCJ/kxV9el5iotXj5BI0MBkWdyXDVAZSs8DRd5dScVv8hoi87ZOkRHWYCI6KPXxAb7BDmhcn7U//3ZSe19cYwN6MuwekM28Ybx3eF70e1yrq+fE+Rm4CtEedVLXM4tNWG2Ym0NA9lD8nW4Ooe3I3E8dkY79XbISN3PAviSqS7USxnZyP/vzhQ+InXY9yozuyaxkAjVcCcTFYtpEDP5IAX98ERN25ZTDDK8+FLPrDMUvlHGY02lhLwcE4+TRi8cT3s8uwy9lghygmfPpRvMl358OMbaxW0kDVHoexi6V6XEIrM7sxFD5lH3IGJH/FsI6NUAvMRY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(39860400002)(396003)(346002)(376002)(36840700001)(46966006)(82310400003)(47076005)(2906002)(2616005)(6862004)(82740400003)(6506007)(6512007)(107886003)(5660300002)(4326008)(356005)(966005)(316002)(6486002)(81166007)(70206006)(33656002)(70586007)(54906003)(478600001)(36756003)(8676002)(336012)(86362001)(36860700001)(186003)(26005)(53546011)(83380400001)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 10:46:13.5288
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ee5798ec-4000-40cc-3b3d-08d9783609b2
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2429

Hi Oleksandr, Stefano,

> On 15 Sep 2021, at 6:30 am, Oleksandr Andrushchenko <Oleksandr_Andrushche=
nko@epam.com> wrote:
>=20
> Hi, Rahul!
>=20
> On 14.09.21 17:24, Oleksandr Andrushchenko wrote:
>>=20
>> }
>>>>   +static int pci_ecam_register_mmio_handler(struct domain *d,
>>>> +                                          struct pci_host_bridge *bri=
dge,
>>>> +                                          const struct mmio_handler_o=
ps *ops)
>>>> +{
>>>> +    struct pci_config_window *cfg =3D bridge->sysdata;
>>>> +
>>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
>>>> +    return 0;
>>>> +}
>>> Given that struct pci_config_window is generic (it is not specific to
>>> one bridge), I wonder if we even need the .register_mmio_handler
>>> callback here.
>>>=20
>>> In fact,pci_host_bridge->sysdata doesn't even need to be a void*, it
>>> could be a struct pci_config_window*, right?
>>=20
>> Rahul, this actually may change your series.
>>=20
>> Do you think we can do that?
>>=20
> This is the only change requested that left unanswered by now.
>=20
> Will it be possible that you change the API accordingly, so I can
>=20
> implement as Stefano suggests?

We need pci_host_bridge->sysdata as void* in case we need to implement the =
new non-ecam PCI controller in XEN.
Please have a look once in Linux code[1] how bridge->sysdata will be used. =
struct pci_config_window is used only for=20
ecam supported host controller. Different PCI host controller will have dif=
ferent PCI interface to access the PCI controller.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree=
/drivers/pci/controller/pcie-rcar-host.c#n309

I think we need bridge->sysdata in future to implement the new PCI controll=
er.

Regards,
Rahul
=20
>=20
> Thanks,
>=20
> Oleksandr



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 10:48:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 10:48:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187470.336336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSSZ-0006zg-Fl; Wed, 15 Sep 2021 10:48:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187470.336336; Wed, 15 Sep 2021 10:48:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSSZ-0006zZ-CW; Wed, 15 Sep 2021 10:48:11 +0000
Received: by outflank-mailman (input) for mailman id 187470;
 Wed, 15 Sep 2021 10:48:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LStB=OF=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQSSY-0006zP-3m
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 10:48:10 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.47]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 69b8cdf4-1612-11ec-b4f9-12813bfff9fa;
 Wed, 15 Sep 2021 10:48:08 +0000 (UTC)
Received: from AM6PR02CA0005.eurprd02.prod.outlook.com (2603:10a6:20b:6e::18)
 by DB8PR08MB5449.eurprd08.prod.outlook.com (2603:10a6:10:119::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 10:48:07 +0000
Received: from AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:6e:cafe::38) by AM6PR02CA0005.outlook.office365.com
 (2603:10a6:20b:6e::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 15 Sep 2021 10:48:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT043.mail.protection.outlook.com (10.152.17.43) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 10:48:06 +0000
Received: ("Tessian outbound 8e26f7114b75:v103");
 Wed, 15 Sep 2021 10:48:06 +0000
Received: from b29202494266.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 99EC4606-F558-48D7-AB27-FB5685A2A599.1; 
 Wed, 15 Sep 2021 10:47:55 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b29202494266.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 Sep 2021 10:47:55 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4966.eurprd08.prod.outlook.com (2603:10a6:20b:d4::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 10:47:53 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 10:47:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69b8cdf4-1612-11ec-b4f9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zQetbcv3diWg4Fav5cK9sH+6bAAy1ZfmP9iOA6LOBeU=;
 b=CfNZ5LRq+y0Pb8cK+fpu1jB36D046cP4iKp4wcU7eeiK8HOrOFPKeLz+ZryUlRLyZdjnkT10L2TKJpRrIixiBohqZzNDM2FGW/ZGqeqfYP+svYwwG4RMve3TypLzjOlYOucCCTzeWRW1CsRQS8FPY862Ulldp9waE4abrGWm0OQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c3ca78460d6d950f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dF4phKe/BdpqZAFtwMSsajXg8Tik0ohllPLPpjSDuvArY5uNAq4aW9MrbGgknhym+f+9vpSMaxcF1lQRyPCKGL2X+N9twSOIkLoameM3oZlYNo56yOPxIMnMhNugAun0zOUzPXkJMISdxZR6dlmLKz4Elz51pDxubpv0TbtNCifdFUstlNjjBulXMt6tY0vy9n7c/ZQCs6sROZZ0H3pMJxM3cL2n1l6flOeETeAnsyN6TEMfJewG0qiCoTw0CDqub3rZy9jN/SFxdplu8V23OmyJKa1RT3mcCrBqZkae1UIZvpUkf1O/kGmmwvJgQbHTRHC4ea/byH+SvfPtHgNDHw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zQetbcv3diWg4Fav5cK9sH+6bAAy1ZfmP9iOA6LOBeU=;
 b=k0r3uy7SP+Lfll6HUoHcZ8WR5nvtyumZgf+bu2NTuPF9JlKj8UorNklTQR/CuHMUO4OhlfklUFUNpFx3NGDI3OOAr3SC6CUtckXBOmumI1sHo964GqTTuu/oFre8eTsWCoL34OmNOB+xNaUNaJVWQKnlKZj1a/zobb6Wx/ADujSAnGhdai5IlhXcwVeOa0t6wxYlCEpeVhaDoDVTp5nePM/dgd/dsah9YoUTO3Q/xc/tm/YmsSW+oD+ixTXcX7BN8h14+rq1A6ruGmbq4XMhTbOkl18B+ZSCXBEW87EItv/5Bhc2rGp0UuJ9iMJquVKxWzfI9LF9h1TZzBpoXrFnmA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zQetbcv3diWg4Fav5cK9sH+6bAAy1ZfmP9iOA6LOBeU=;
 b=CfNZ5LRq+y0Pb8cK+fpu1jB36D046cP4iKp4wcU7eeiK8HOrOFPKeLz+ZryUlRLyZdjnkT10L2TKJpRrIixiBohqZzNDM2FGW/ZGqeqfYP+svYwwG4RMve3TypLzjOlYOucCCTzeWRW1CsRQS8FPY862Ulldp9waE4abrGWm0OQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v1 08/14] xen:arm: Implement pci access functions
Thread-Index: AQHXlPLKFxZ9ODgqA0ysGgNkOt1JX6ucfp8AgAdcI4CAAQksAIAAMHGA
Date: Wed, 15 Sep 2021 10:47:53 +0000
Message-ID: <511F9F8E-B6A2-4A37-A892-600107031227@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <c90c3088a592b41c477a0026446294a3b9422f76.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091639360.10523@sstabellini-ThinkPad-T480s>
 <E37D41BC-1ACB-4779-9A8F-379F0E7BCB1A@arm.com>
 <6841a3c5-2565-9fee-8776-19d5c4a68fea@epam.com>
In-Reply-To: <6841a3c5-2565-9fee-8776-19d5c4a68fea@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 23ae8aa3-84b2-476c-5990-08d978364d22
x-ms-traffictypediagnostic: AM6PR08MB4966:|DB8PR08MB5449:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB54494BB03A0EFD5CABA08EC0FCDB9@DB8PR08MB5449.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +lgj27PQyRcX/4Vtey5hLVB78U3B9LvZ4tI34VH+CuGuXhXBtZveXUhQsJ76H+B/C0Pn/KK5FMWlpKQfBOiYr/AqDUWSgTj7Mv5a/PnJEr7wbdudbyYOJQ7hNPOTnWSOqXdnD6z4ScQnX6SSM2Sq87dIWBLvxY7H0vnVkABsIQ0rGRCebM29v7NdrHqGcyCP64A39/oJ4En3svQY5HKui+DvT5sGAllLVSUIzszYEItq/RuEx0x98+E7+Zv17W9YkZsOjYgR1fxlm9eqNhzizTuxw1lW8WIfoGuX5EIPTjiG8aMziGQvRrkhSeL0pg11PGfWDKQCx9+q09GaUiww10WuiymfBcEZ7K2L9TSTx1fNSbfbIOkDeInPsO7bIqWyEAY/eMx7AufZ/jmfubG/CY0iFVTmQJ/lTzKSZaeKYJaveTeQqRJaIgq/VabGj+ZFqylF7x+2gr4w1aAlrAZszIVjatvq509Q/9Zf6wNqsCCWwHcXn+3tgma8j9emUdQry0Fls/+KS7BRkSWZHs6jnLQphAbv5kS6yLnd0bC4PtL+ErFKrjLj8LgfzeeXiL8EqHbSctHvDK6fONMvI07Z5P8LDuJiH/Bv+aczYVrXR6t3UMks23lNnyiMvD782s3Pj9WxT6k6pFrQkFcma9l0ZAVSN9m1dlkWL7c38wcUT4D4kFbKAVXXEsc+7Qt8GMh8NOyaG1dvMyStYMXnYbLG8eo9ROEJTX84l7C2k0EkxsjODKphWDuDZ3PejuY6TTq9
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(122000001)(6512007)(66946007)(26005)(4326008)(8676002)(76116006)(4744005)(186003)(6486002)(71200400001)(66556008)(64756008)(66476007)(33656002)(38100700002)(66446008)(38070700005)(2616005)(53546011)(36756003)(316002)(6506007)(508600001)(86362001)(6916009)(8936002)(54906003)(2906002)(5660300002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <4A18542D7239A5498B8BADFC15D847C0@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4966
Original-Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a54e5ddb-931b-4e7a-318b-08d97836457e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tfBEbpCwc6YNJUGBwqqzFxuLP45qJ58jnQbqU68wy0YRE+aG1zhDX32t2YdkrElW9nBMEFjiSM9LAyjl4cUwxMkEQnyi1moHh0+GjAU9DDlxFjPicfPeSMqaK4yM7bK+JDSpzvN23r3TKP9EoVcu2mMcX4hqokYJkf1VBBeA0V2aqJlmlBOhFd/pA7l4oYFzUnXMwUvt5Fru6CQJIfD9FOZLcc36YKHKZOWJhrmUQplmx9Cb9Vnxg6yLSUkZmjDD3HCgCklXO9FwfSI9h7+9UEl78Sn75cfFo23bMD9PWQWjX5sLAZ+8yZm0YYxXrKk0tMoYrO80ZJZnus84ogALnbo0rZtAnefG2TxaanqCWSoEVqnRgzJy9tZwsoR19PZXJUryB0ljYcVKTMadJS0UkkIG/CpnKCozpt7Uu7QOTRD7qMVk54WY5iTT6Kn2v7Fo+Q+DZeUte7BQCyetk/jjLNiwiyDMrQwR+iZeasMVX3gBAKpqE3kRTAHxylgu1oK0BZTvdcpT0YCcaZJJOI6T5nSXnXylEk8V7obQoF4dqVSTjWUq5FtkB99su3eaMMnQ6RATE0njeF6nYU/SjLqFdvIq8dsgjwDH+fy3yqUlaAOshewErC1+EEoEXd1RnJhxQ6hHhajuTgzeHPs7CHN9vGFHM3b0aEsP/q9lOl98iaXxjrHY20rqR9HzBuQjZFfpEfea4w9wDWW9GJ8MCk7YQg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(336012)(26005)(508600001)(8936002)(186003)(5660300002)(6512007)(82310400003)(107886003)(4744005)(6486002)(8676002)(70206006)(70586007)(53546011)(6506007)(2616005)(316002)(4326008)(47076005)(33656002)(36756003)(54906003)(86362001)(36860700001)(81166007)(2906002)(356005)(6862004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 10:48:06.6690
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 23ae8aa3-84b2-476c-5990-08d978364d22
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5449

SGkgT2xla3NhbmRyLA0KDQo+IE9uIDE1IFNlcCAyMDIxLCBhdCA4OjU0IGFtLCBPbGVrc2FuZHIg
QW5kcnVzaGNoZW5rbyA8T2xla3NhbmRyX0FuZHJ1c2hjaGVua29AZXBhbS5jb20+IHdyb3RlOg0K
PiANCj4gSGksIFJhaHVsIQ0KPj4+PiBzdGF0aWMgdm9pZCBwY2lfY29uZmlnX3dyaXRlKHBjaV9z
YmRmX3Qgc2JkZiwgdW5zaWduZWQgaW50IHJlZywNCj4+Pj4gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1bnNpZ25lZCBpbnQgbGVuLCB1aW50MzJfdCB2YWwpDQo+Pj4+IHsNCj4+Pj4gKyAg
ICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UgPSBwY2lfZmluZF9ob3N0X2JyaWRnZShz
YmRmLnNlZywgc2JkZi5idXMpOw0KPj4+PiArDQo+Pj4+ICsgICAgaWYgKCB1bmxpa2VseSghYnJp
ZGdlKSApDQo+Pj4+ICsgICAgew0KPj4+PiArICAgICAgICBwcmludGsoWEVOTE9HX0VSUiAiVW5h
YmxlIHRvIGZpbmQgYnJpZGdlIGZvciAiUFJJX3BjaSJcbiIsDQo+Pj4+ICsgICAgICAgICAgICAg
ICAgc2JkZi5zZWcsIHNiZGYuYnVzLCBzYmRmLmRldiwgc2JkZi5mbik7DQo+Pj4gc2FtZSBoZXJl
DQo+PiBZZXMgSSBhbSBwcmludGluZyB3aXRoIOKAnFBSSV9wY2kiLg0KPiANCj4gdlBDSSBhbmQg
dGhlIHJlc3QgYXJlIHdpZGVseSB1c2luZw0KPiANCj4gICAgICAgICBwcmludGsoIiVwcFxuIiwg
ICZzYmRmKTsNCj4gU28sIEkgdGhpbmsgaWYgd2UgaGF2ZSBTQkRGIHRoZW4gaXQgaXMgYmV0dGVy
IHRvIHVzZSAlcHAgaW5zdGVhZCBvZiB0cnlpbmcgdG8gdW5mb2xkIGl0IG1hbnVhbGx5Lg0KDQpP
ay4gSSB3aWxsIHVzZSB0aGUgJXBwIGZvciBwcmludGluZyB0aGUgU0JERi4NCg0KUmVnYXJkcywN
ClJhaHVsDQoNCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 10:48:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 10:48:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187476.336351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSTF-0007cE-RG; Wed, 15 Sep 2021 10:48:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187476.336351; Wed, 15 Sep 2021 10:48:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSTF-0007c7-NL; Wed, 15 Sep 2021 10:48:53 +0000
Received: by outflank-mailman (input) for mailman id 187476;
 Wed, 15 Sep 2021 10:48:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XwCH=OF=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQSTE-0007Z2-RT
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 10:48:52 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 804632c8-1612-11ec-b4f9-12813bfff9fa;
 Wed, 15 Sep 2021 10:48:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 9A0EF2220E;
 Wed, 15 Sep 2021 10:48:45 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7661113C2F;
 Wed, 15 Sep 2021 10:48:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 7bKnG43PQWEmBAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 15 Sep 2021 10:48:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 804632c8-1612-11ec-b4f9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631702925; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=iTKSaRtAvb4MPwNlHHAs/1gnbRIWxnDB5cbkIAO1GYk=;
	b=t/rojrg2wrxx4pRgBWYGWnJ2Ze410a7OmVESnyHsEw5REFzBJObacrRkBjT+2UP7rqPRbY
	zBEkWKtm4HWZeH7du5fRizxnFtJWmqoDX5BO5eqNKcmF74RzIIM+UB0jtY9BK9kP9rOM7b
	kOu98XwmIq2F8Baq8wo98+MgEI4Jfa8=
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] mini-os: xenbus: support large messages
Message-ID: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
Date: Wed, 15 Sep 2021 12:48:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210914231742.pxpac2lhp2kk5ook@begin>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB
Content-Type: multipart/mixed; boundary="dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
Message-ID: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
Subject: Re: [PATCH] mini-os: xenbus: support large messages
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
In-Reply-To: <20210914231742.pxpac2lhp2kk5ook@begin>

--dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB
Content-Type: multipart/mixed;
 boundary="------------4AF975DB5AFDAF92EAD2B51A"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------4AF975DB5AFDAF92EAD2B51A
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 15.09.21 01:17, Samuel Thibault wrote:
> Hello,
>=20
> Thanks for having worked on this!
>=20
> Juergen Gross, le mer. 18 ao=C3=BBt 2021 17:26:10 +0200, a ecrit:
>> +static void xenbus_read_data(char *buf, unsigned int len)
>> +{
>> +    unsigned int off =3D 0;
>> +    unsigned int prod;
>> +    unsigned int size;
>> +    int notify;
>> +
>> +    while (off !=3D len)
>> +    {
>> +        if (xenstore_buf->rsp_prod =3D=3D xenstore_buf->rsp_cons)
>> +            wait_event(xb_waitq,
>> +                       xenstore_buf->rsp_prod !=3D xenstore_buf->rsp_=
cons);
>=20
> The if is redundant since wait_event already tests it.

Ah, right.

>=20
>> +        prod =3D xenstore_buf->rsp_prod;
>> +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, =
prod);
>> +        size =3D min(len - off, prod - xenstore_buf->rsp_cons);
>> +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
>> +                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), s=
ize);
>> +        off +=3D size;
>> +        notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D=3D=

>> +                  xenstore_buf->rsp_prod);
>=20
> This looks odd to me?  We want to notify as soon as the ring is empty,
> which can happen at any place in the ring right?

No, we want to notify if the ring was full and is about to gain some
space again, as the other side was probably not able to put all data
in and is now waiting for more space to become available.

>=20
> Linux' code uses (intf->rsp_prod - cons >=3D XENSTORE_RING_SIZE), *afte=
r*
> the rsp_cons increase.
>=20
>> +        rmb();
>=20
> rmb() must be placed before memcpy_from_ring, to make sure that the dat=
a
> we read from the buffer is up-to-date according to the read we made fro=
m
> rsp_prod.

Ah, yes. Thanks for spotting that one!

>=20
> The rest seems ok to me.

Thanks,


Juergen

--------------4AF975DB5AFDAF92EAD2B51A
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------4AF975DB5AFDAF92EAD2B51A--

--dCjWhK2Ru03rTrBBaj7QlVmsqxnePxHnB--

--a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFBz4wFAwAAAAAACgkQsN6d1ii/Ey+l
dQf6ApRTwp9ksBqlRW19px1aRTw+8f8odlRkxEUfzEj0AFav0/wyCygTBCSdWev93acVbAF2Uo+T
czuUxY6dWAiPdxrYS0XzfoG8C+Pq8TMDmxt+nl582VSMwDxLJzYIkZmwh0897je9jr86m1x5a2lS
EnpBiZn0UJvBiYTn5cP6ocFWA6at5vW88fT9UmWX6NBwguH9u9y4Y9ZV7Wv8unJKyNjtsQ7mKGgA
PL5+2yxM8Mwxvv1WCCT7fee7AYF//gpX/CYyZVe2nw5Chjf64NvoLJBLnVIMMGrn6SH46xx+ba3H
cE81xv80CGJsiJWZJf+8hHt6mD0lQ1Aj1uWsLbKbCg==
=Isq2
-----END PGP SIGNATURE-----

--a6FiRo5rvrV8rM1QpL0b6BtntmpTnTYqB--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:00:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:00:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187488.336362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSek-0001fl-3h; Wed, 15 Sep 2021 11:00:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187488.336362; Wed, 15 Sep 2021 11:00:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSej-0001fe-VY; Wed, 15 Sep 2021 11:00:45 +0000
Received: by outflank-mailman (input) for mailman id 187488;
 Wed, 15 Sep 2021 11:00:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3xTB=OF=alien8.de=bp@srs-us1.protection.inumbo.net>)
 id 1mQSeY-0001fY-KQ
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:00:44 +0000
Received: from mail.skyhub.de (unknown [2a01:4f8:190:11c2::b:1457])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c6a073c2-51c7-4451-8008-8a06cc411f2f;
 Wed, 15 Sep 2021 11:00:32 +0000 (UTC)
Received: from zn.tnic (p200300ec2f0d07000c3d48728178681f.dip0.t-ipconnect.de
 [IPv6:2003:ec:2f0d:700:c3d:4872:8178:681f])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 9F7E51EC0493;
 Wed, 15 Sep 2021 13:00:26 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c6a073c2-51c7-4451-8008-8a06cc411f2f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=dkim;
	t=1631703626;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VhruTfx6a1vn9bJbkqwEzxTY41J4MJGsLlefXt9yGLg=;
	b=AoZxQDh0ovGCZp36TVoMqssSPPir4S4CLUTje3ga2Sl4kb+jKndFyxzIQAdJXNZ2UdIJ6s
	kfLUqWGA4EkHlLtO7vdZeGBKfn+T4gB2KoAzSvk/LcfcYioMbDg/OWsjZTxwkOGmMUGJ43
	XiSdKkF3s8/MlzVxKAN0eN5YeTY07vg=
Date: Wed, 15 Sep 2021 13:00:20 +0200
From: Borislav Petkov <bp@alien8.de>
To: Juergen Gross <jgross@suse.com>
Cc: Jan Beulich <jbeulich@suse.com>, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
	Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, Mike Rapoport <rppt@kernel.org>
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUHSRKubsGT2Jvur@zn.tnic>
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
 <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>

You forgot to Cc Mike, lemme add him.

And drop stable@ too.

On Tue, Sep 14, 2021 at 01:06:22PM +0200, Juergen Gross wrote:
> On 14.09.21 12:03, Jan Beulich wrote:
> > On 14.09.2021 11:41, Juergen Gross wrote:
> > > Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> > > reservations") introduced early_reserve_memory() to do all needed
> > > initial memblock_reserve() calls in one function. Unfortunately the
> > > call of early_reserve_memory() is done too late for Xen dom0, as in
> > > some cases a Xen hook called by e820__memory_setup() will need those
> > > memory reservations to have happened already.
> > > 
> > > Move the call of early_reserve_memory() to the beginning of
> > > setup_arch() in order to avoid such problems.
> > > 
> > > Cc: stable@vger.kernel.org
> > > Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
> > > Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > > Signed-off-by: Juergen Gross <jgross@suse.com>
> > > ---
> > >   arch/x86/kernel/setup.c | 24 ++++++++++++------------
> > >   1 file changed, 12 insertions(+), 12 deletions(-)
> > > 
> > > diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> > > index 79f164141116..f369c51ec580 100644
> > > --- a/arch/x86/kernel/setup.c
> > > +++ b/arch/x86/kernel/setup.c
> > > @@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
> > >   void __init setup_arch(char **cmdline_p)
> > >   {
> > > +	/*
> > > +	 * Do some memory reservations *before* memory is added to
> > > +	 * memblock, so memblock allocations won't overwrite it.
> > > +	 * Do it after early param, so we could get (unlikely) panic from
> > > +	 * serial.
> > 
> > Hmm, this part of the comment is not only stale now, but gets actively
> > undermined. No idea how likely such a panic() would be, and hence how
> > relevant it is to retain this particular property.
> 
> Ah, right.
> 
> The alternative would be to split it up again. Let's let the x86
> maintainers decide which way is the better one.
> 
> 
> Juergen
> 
> > 
> > Jan
> > 
> > > +	 * After this point everything still needed from the boot loader or
> > > +	 * firmware or kernel text should be early reserved or marked not
> > > +	 * RAM in e820. All other memory is free game.
> > > +	 */
> > > +	early_reserve_memory();
> > > +
> > >   #ifdef CONFIG_X86_32
> > >   	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
> > > @@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
> > >   	parse_early_param();
> > > -	/*
> > > -	 * Do some memory reservations *before* memory is added to
> > > -	 * memblock, so memblock allocations won't overwrite it.
> > > -	 * Do it after early param, so we could get (unlikely) panic from
> > > -	 * serial.
> > > -	 *
> > > -	 * After this point everything still needed from the boot loader or
> > > -	 * firmware or kernel text should be early reserved or marked not
> > > -	 * RAM in e820. All other memory is free game.
> > > -	 */
> > > -	early_reserve_memory();
> > > -
> > >   #ifdef CONFIG_MEMORY_HOTPLUG
> > >   	/*
> > >   	 * Memory used by the kernel cannot be hot-removed because Linux
> > > 
> > 
> 






-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:05:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:05:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187496.336373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSjB-0002KV-KT; Wed, 15 Sep 2021 11:05:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187496.336373; Wed, 15 Sep 2021 11:05:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSjB-0002KO-HS; Wed, 15 Sep 2021 11:05:21 +0000
Received: by outflank-mailman (input) for mailman id 187496;
 Wed, 15 Sep 2021 11:05:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FsSg=OF=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mQSjA-0002KH-CT
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:05:20 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4e4d5784-787e-4d0c-80b7-682dde088413;
 Wed, 15 Sep 2021 11:05:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e4d5784-787e-4d0c-80b7-682dde088413
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631703918;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=PSimCOqPBV3WkzV1SpJJGf/2wNBZRc8NQJZZe89DJIA=;
  b=FVCUWYDuKHmN+NmL6J8oI0h5OMWuYl/Fs7fyA590ukwxYUVEqJAXXiY6
   N3EP5ahgFzMm/pUQn3NTFozmKWur48fhSjmBIXxLuu5/a4Ss3Ydi7LO3s
   /X63diR2QE+vYCmAG42Fv8od4ldM/SsvfpO609iFgnUWFVrcNbFio9bDg
   0=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: bPfa+JX68P+FQ2i+8rREjHkiLeAGj/D0W/9qsgX/O4jKf8u8jZ9boBcJT53wnlyPeI6h8TUQ9Q
 FAedE4oj1iyR6vIbLZ6Bteqijxq25QBVKpHP4LnYVYzBfqqIr6GFJZZV2Tgvo8JWkwgisWuU0X
 0OsDjrFgvZpWi6lJwK20vKNsIQlho4ToytJWzin7nKt9JhyHEpv0WA8NMKjQqzwhBw4HQMG6it
 PaP8IHmgrppdTz5ts2Xxr/8f1L4+BAAQGR2DM65RuYg73d9P/Ih1oOvXdFs6dxMs2dOvRr8cDP
 QqsGrZTSqiWgwMnJru8IsWXf
X-SBRS: 5.1
X-MesageID: 54540432
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:YnX5561TGKDvjAMT/vbD5QJ2kn2cJEfYwER7XKvMYLTBsI5bpzxTy
 jBLXzuFO6veMGejedFwOouz8RhS7MfTn9I2SwNppC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrNh2eaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhueh01
 P9vsbiJZywEZYnsh9YGdUl4HHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t2p0SRaqFP
 aL1bxJ3TxH8cjBMO20vGc0Fpe6nuHr8XQRh/Qf9Sa0fvDGIkV0ZPKLWGNvKePSaSMNNhEGaq
 2nauWPjDXkyNtaS1D6E+XKEnfLUkGXwX4d6PK218LtmjUOewkQXCQYKTh2rrP+hkEm8VtlDb
 UsO9UIGsqwa5EGtCN7nUHWFTGWs50BGHYAKSqtjtV/LmvG8Dxul6nYsaAZEdPYEsOsPaxcHj
 XupgOHyI2xPv+jAIZ6CzYt4vQ9eKABMczRbOH5bElVcizXwiNps1UOUF76PBIbw14esSG+qm
 1hmuQBj3+17sCId60msEbkraRqXr57VR0Ye4gzNVwpJBSsoOdb4O+REBbXdhMuszbp1rHHa5
 xDoeODEtYji6K1hcgTXHo3h+5nzu5643MX02wIHInXY323FF4SfkWVsDNdWfhYB3iEsImSBX
 aMukVkJuM870IWCNPcqC25ONyja5fe5Tom0PhwlRvFPfoJwZGe6ENJGPBXLt10BZHMEyPllU
 b/CKJ7EJS9DVcxPkWrnL89AgORD7n1vmgvuqWXTkk3PPUy2PyXOF9/o8TKmM4gE0U9ziF6Jq
 o0Oa5TTkEw3vS+XSnC/zLP/5GsidBATLZv3t9ZWZqiEJA9nE3smEPjf3fUqfIkNokifvrugE
 qiVVhAKxVzhq2fALAnWOHlvZKm2BcR0rG4hPDxqNlGtgiBxbYGq5aYZVp02Ybh4q7Azka8qF
 6EIK5eaH/BCajXb4DBBP5Pzm5NvKUawjgWUMiv7PDVmJ8x8RxbE88PPdxf08HVcFTK+sMYz+
 uXy1g7STZcZaR5lCcLaNKCmw1+r5CBPk+NuRUrYZNJUfRy0ooRtLiXwiN4xIt0NdkqflmfLi
 V7ODE5B9+fXooIz/N3Yvoy+rt+kQ7lkA05XP2jH9rLqZyPUyXWunN1bW+GScDGDCG6toPe+Z
 f9Yxu3XOeEcmAoYqJJ1FrtmwP5s59broLMGnA1oEG+SMgauA7JkZHKHwdNOputGwboA4Vm6X
 UeG+997P7SVOZy6TA5NdVR9NunTh+sJnjTy7OguJBSo7SB6y7OLTEFOMkTekydaNrZ0bNsoz
 OpJVBT6MOBjZs7G6uq7sx0=
IronPort-HdrOrdr: A9a23:aM3cfKytOG+1TWofyA9kKrPxMegkLtp133Aq2lEZdPULSKGlfp
 GV9sjziyWetN9IYgBTpTlEAtj3fZquz+8B3WBxB8bhYOCCggeVxe5ZnPXf6hHbMxTk8ORQz7
 1tfsFFeYbN5DRB/L3HCUyDYqUdKbq8geWVbIXlvgZQpGhRAskOgXYbe2TrazwPeORfP/oE/d
 inl7p6TlKbCDMqh6+AdwQ4tp341qL2feXdEGI77nAcmXOzZAeTmfjHOind8gYVVjtOxbs40G
 7ejgD13KWstPmn1gTRzSv2445NkNXs88BEDsCXkMQZQw+c7zqAVcBMSreGtDAwrPqO7E8wkN
 7Qix08P8J1gkmhGF2dkF/AygPk2DYr52TDzUaCgXz/hMTlSDUxG6N69OVkWyqcx1srudl91K
 5R5nmepopzBQ7Nmy7sjuK4My2C23DE/UbLXoYo/jxieLpbTIUUgZ0U/UtTHptFNCXm6Lo/GO
 0rN83Y7OY+SyLUU5iM1FMP/PWcGlAIWjuWSEkLvcKYlxJMmmpi8kcezMsD2l8d6ZMUUfB/lr
 H5G5UtsIsLYt4dbKp7CutEa9CwEHbxTRXFN3/XCUj7FZsAJ2nGp/fMkecIDdmRCcg1JaYJ6c
 T8uRJjxD8PknvVeJOzNUhwg0PwqGbUZ0Wg9igR3ek8y+rBrHyCC1zbdLlkqbrVnxwlOLySZx
 5oUKgmW8MLZVGebrqh/zeOAaW6ekNuDfH8TL4AKhOzSvyiEPyhigQtGMyjaoYFVwxUBV8XJ0
 FzHwQbFf8wo3xDAEWIySQ5c0mdMXDCwQ==
X-IronPort-AV: E=Sophos;i="5.85,295,1624334400"; 
   d="scan'208";a="54540432"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZWMVBjbAbJZS3ZN+bq/y5Ae9tRp9TUMySIsBeDe8GCPKPToz505yNU7zmMlUXYhGGwLTIjrifND/YmlaF/szM1O5SL3qdKVmSCpW7gMH1V4NfzIGwzE4fYV5D2BoOeDGFlXbcTibZx6Epcr6/piOSK91SzJVEH8345nWTAnE7iT9xzufZz5pnCUbgBcS5/9h1onOSotYKBK+Q9tx5cBKW6hZQDMijicw1Pu2qe1r/1QnGRaYa2Kqdmdbzqg12eX/s12Gp7WbZYgueOZ8anhhK6t/TW8HxOtGMGLOhyQDTbCQUbPkIfC50/00A28hiBLhWrHqOYryY2BYSTrrg7xs/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/8NYyk/vjXp+GbEJC9szBDZ+NqHZduKItzBrrbkmeJo=;
 b=GQgfk3EPRVwDfgbEVAdT9QCoBzk1tSs2pyDUZF38r36CN/J7k7pRFkZQqI1qqhruOgVO7kmjjTP6dZ/LivJMO0uIrwZ1Ahp5/SoF50H9wRVT5T+mYLa+7pfpz9gHzPcjAJR3qKR7WAYdK52HC+MD1kJ0poToLP3CsInEa41wR3lt4+x5BU81QUtTjC4hzHh3s3l5gX2TLE3r148JH7X8oH+sMuqNfzIAUdZtpQewFpc/5VkfN8L/0UeNydtc/2zpN+secfAl1E0OdMdxwnp9z2FPk3LxbojTOwkTpzDLPSAgC1Kd8/JWcmrb6Ht5kXEHkdVkLYO+sSb/Grao1+wH7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/8NYyk/vjXp+GbEJC9szBDZ+NqHZduKItzBrrbkmeJo=;
 b=Jari9oYXafIBMr0UF9QlF9vYcS3zBxTASORD7t998iza0+tGxYEG9TjqvWcpJb079RbbfGInsCmUWQnj+vcXEIFjoqxFnumk00PGvhShhtVQTP3e3++iIzBpfixJJjLhBya69bSfU4JbIf8DJf5LjfRs/Hcaj+xr/w2D3n6imD8=
To: Jan Beulich <jbeulich@suse.com>, Kevin Stefanov
	<kevin.stefanov@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210914164323.14904-1-kevin.stefanov@citrix.com>
 <f5a68380-16f1-5383-5cfa-85a9e02ccf8d@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2] tools/libxl: Correctly align the ACPI tables
Message-ID: <fb7c68c5-3fe0-7e61-fc9e-829345d471b3@citrix.com>
Date: Wed, 15 Sep 2021 12:05:08 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f5a68380-16f1-5383-5cfa-85a9e02ccf8d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0013.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:a6::25) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 84c8ec89-653b-45a3-9e63-08d97838b182
X-MS-TrafficTypeDiagnostic: BY5PR03MB5345:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB53452BCDD4D984277A851A7EBADB9@BY5PR03MB5345.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 57RREdzM6Zeg/nzp0/34hmey2XbdeBFB2oXC3bNcaI7v0+3SN5tDdvq7nu07Pa/mILSrzb4jCveX5TrC9+MBmit/oAAEswYWZcQ2sD6vWQxNdpz989iO5kYPKh0XuE8g/H1nMXOj7IaNEka7rw7XExdqskUqWi9LdSFF/O76zwbUCgQ6LKhfQf7upogjhinccw+CVueNAxUNBNjHR1LwLcsf4OemLvQ9CNq8tUqIEu/pIKQBDwX/CRalxDA6Pe+TaXkOxWY5OzCqRAqWaqSZ3BacVF09GrqHsp3ffpdvDOUPAnKej7W3AsI9ILIFDwRMPojxbM/Opc8ITVBpN39gFoaxJd1P0Fn8ip89QRBiYBRvyP805OTXFFA4oM44Gajtb3Hp4AlrNt2yMpWVZz3GzZwRDhPoOk9yLCaUV2B2bPebzz/vYs6xf4Ev49/Cs60hxh786xUrJlyGp4qv/YvNaddzA41FdrVDdOMSgtCALOiT6eaAz8RGZ/1Y+uxGFyVuMOisTdSN4vM/d3DUFqc8k2Cw54JHWah7PTx3qf5e9mQTx4++zWAulX1Jxu+DRHebbnkdQYAuCi51OSlHS2ddkKolS+u7cUh7e9q8jxNXfKLeVOOqQzFDcExM9g2Gmx9M3f/ij/yVp4tCeGh3KS/TgOcXXydMxY1P4GC8NTrJNqm4DFbRVzPglIb1y60//XtpXrLXebVG0yNaV2MvJTbW9k6AWwK+wtKa+7AlCvAcaHY=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(39860400002)(136003)(396003)(376002)(16576012)(316002)(31686004)(36756003)(54906003)(110136005)(31696002)(4326008)(86362001)(5660300002)(478600001)(6636002)(6666004)(2906002)(66476007)(55236004)(4744005)(66946007)(66556008)(6486002)(956004)(2616005)(26005)(8676002)(186003)(8936002)(38100700002)(53546011)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MHh2VXNmVlhBK3Z0dCtFMDV6YXdjelRwMlN1VW0zVm9salpwVlJsOE1RUmlW?=
 =?utf-8?B?bEh4Lzk5c0VGSXV1WHpaY2lVK2JxM1FDa1NGQjNPOWpsTks2d1ZNWkV6bGlF?=
 =?utf-8?B?ZGFGQlVERERQQ1pTbGIvZ1dpNm05YlpDQXhFb0Q3Y25FYXZHSjlQOC9scklv?=
 =?utf-8?B?L0JaYjhGYS9EN3NMUXBHeHN5TDVyQjg0bXNTL3NKNXVwQkNxSGJUWFJTb001?=
 =?utf-8?B?VGtBUUErT2RPVlRTWHhUbC9YWGh6dmtwWUJkNU1sZFFmTmZQM0RYYm1BMTA5?=
 =?utf-8?B?Snc0aFlGVHFtSkZXcVU2L1paMFliTXhvN0RMMS9rcVBTTDBrck8yTDd1ZGNu?=
 =?utf-8?B?cEFCL0FhaFFuaVpzeDg4TkU4Zy80aTNLMGhWVTlrQko2NDNtMm13SWIxTmlY?=
 =?utf-8?B?R0Vmd0FzS0dUKzJ4QnpYZ2V2Wi9xcFpkbnhCNFdUTjJ2OVlGRWd5VnQ0Z3dy?=
 =?utf-8?B?VUhkdnhJaHFaNEQzVVE5d2d5MUZGQWVFWkQ0QngxM0Y4dmhFakhBOUtiY2Ju?=
 =?utf-8?B?TWVQWVNjcWxQVDRDQ0RwSmx6Um1vWE9HbHVibEEyUE1MVHdUbjdMK0hmWmJz?=
 =?utf-8?B?WERrejh0KzRnZmRucFVEREJreVMvQ0luM2NZVGRPRHBXRnFrV3RBV1BhV0ZV?=
 =?utf-8?B?NGN3UUd5cHZhYkcwTXJab29SOUZ1MVRrQlZHeHFtMUo5SE1aR2h0TTVwdlli?=
 =?utf-8?B?ZFJPUTFxaVZ6bzZYbDBLMzdVeFVyZEtjTEF4OWU5LzVlaFNLeUtyL3RaRnM2?=
 =?utf-8?B?cUNPajhhRGFiSEltYmJpemRCOEF0eUJhY25IM0hsaVN3UzRob3B5bVN1dmo4?=
 =?utf-8?B?MVBVTm5JNnFNLytCcVRzUTF1eEJLNysxd2k0VWFDR1F4ZU5QUjZsc2kyZFdq?=
 =?utf-8?B?VFJvSlhyZlE0RXJrNW8wS3docEkrcHU2Ykp3Z3BiSzIwc29QZGxmbFJPanRU?=
 =?utf-8?B?SWFQV2FWMDcycmVYN2JLTUhRcHFudmNYT2ZKT0RkeUhiRjh2UHFsVHBJQjho?=
 =?utf-8?B?N1NZWFltcWFFbWJJQnlRRjJvRzBaQnNsV1pVamV0UjBybTR4Vk5tT1NDRUUz?=
 =?utf-8?B?QkZCUEVkVVRZRmZsSkZ4SE5hU1JsZE9VRVB2eTJXRXZDaSsxdEMyRnRJSkJB?=
 =?utf-8?B?ZHRGbWNBMytCcEFsdHArUUNqRmlCT2hDZFlIejJBcFBpeFdMTnFOcjJEMmo3?=
 =?utf-8?B?eFRHQTJ0QVJicFQrRkl3ejFRL1Y1RTFMME5OTHNUeEFGaXZBNjBlWVAvaXdP?=
 =?utf-8?B?ZEZ5UEJCMHZzNzhtWk8welBjTXFXMjRWTXl1Qko0Y1RDeVIyalg5Tk54Vzla?=
 =?utf-8?B?R0lWNjVtMENQeTZ6aXBGZm9aTFNWSmxHdWZ5dExWRkJ2N1RnemFqdVk4c0xz?=
 =?utf-8?B?SkRMNnFZemhEcm9YQlozZ0V3RXFmelNBaVd0QXdNc1FaZzRtb2FBMURPUkY2?=
 =?utf-8?B?WTZ3UU1EVWZUQUVDMmZMUEU0ZnUzMlU2UVdrUXFtSXhQcC9IVER5OTVubVk1?=
 =?utf-8?B?SEs2bjAyUlBVZVJSNERqNjRhOEtvTGc2TXJpWlE5c3B4eEZraFVDS3F4Q2lm?=
 =?utf-8?B?cmJxZlVNdDVZM0ZXZFo0bHk2RDRNcm56VC9RWEhXZDk4R1lKR3ZLMXh2azZq?=
 =?utf-8?B?cDc1dHJkUDBCQjdCd2JMNVlBTXUyMWl6dFprbTZNYW01K0JGK2g2ckhHV0NJ?=
 =?utf-8?B?MzhpUnZuSzJaUUxWd3VxbGpqdC9CbEg0b3JUVGY1M210WWY4RXNrbFZCam1Q?=
 =?utf-8?Q?2l701FE8p5NzXSnCqAOv71sbuW0bDRVEfKvWc8j?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 84c8ec89-653b-45a3-9e63-08d97838b182
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 11:05:14.2544
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: z3Dxidzncesm9H6JV58QfKMaHqGKaBqEerXvFLPmy+ss4Pn50/1F0yw0K/eeykBPqA03c8a8D1Pi+cURE+tmJ7ia3rDlU6Bf4eUMg9CvKMY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5345
X-OriginatorOrg: citrix.com

On 15/09/2021 10:30, Jan Beulich wrote:
> On 14.09.2021 18:43, Kevin Stefanov wrote:
>> --- a/tools/libs/light/libxl_x86_acpi.c
>> +++ b/tools/libs/light/libxl_x86_acpi.c
>> @@ -20,6 +20,7 @@
>> =20
>>   /* Number of pages holding ACPI tables */
>>  #define NUM_ACPI_PAGES 16
>> +#define ALIGN(p, a) (((p) + ((a) - 1)) & ~((a) - 1))
> Wouldn't this better live in xen-tools/libs.h?

In this case, not really.=C2=A0 That file doesn't exist on all versions thi=
s
bugfix needs backporting to, and there is an unknown chance of collision
in older trees.

This would be a whole lot easier if ROUNDUP() wasn't in a dubious state...

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:16:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:16:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187506.336388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSu6-0003w7-Ny; Wed, 15 Sep 2021 11:16:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187506.336388; Wed, 15 Sep 2021 11:16:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSu6-0003w0-KJ; Wed, 15 Sep 2021 11:16:38 +0000
Received: by outflank-mailman (input) for mailman id 187506;
 Wed, 15 Sep 2021 11:16:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XwCH=OF=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQSu5-0003tQ-BE
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:16:37 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id aef0d088-d611-48ba-b794-2221fe00e3e2;
 Wed, 15 Sep 2021 11:16:30 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 25B0A2221F;
 Wed, 15 Sep 2021 11:16:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E6BCB13C37;
 Wed, 15 Sep 2021 11:16:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 7evhNgzWQWEHFAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 15 Sep 2021 11:16:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aef0d088-d611-48ba-b794-2221fe00e3e2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631704589; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=AjV0yLQDNp7wCmyLlXRUm2R34qvQRC0Ch6O/l5NYfDc=;
	b=OxfedwDzTzFCJ0byFYdt0blxi/vXV3CzSgJ3/wNV3jzZZvtjKTDJbJEOtaQX91Z1d4EXX8
	YFNF0GOcjAzRpvPpPhUQiXIka4YPB/4R4Vfyauk1k1nlqn5lzQde12eNifiqJGDBH+RM0C
	Gke2pA1mEBRgVzmnU7T0Pzx1Rrxxy3Q=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2] xenbus: support large messages
Date: Wed, 15 Sep 2021 13:16:27 +0200
Message-Id: <20210915111627.19486-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Today the implementation of the xenbus protocol in Mini-OS will only
allow to transfer the complete message to or from the ring page buffer.
This is limiting the maximum message size to lower values as the xenbus
protocol normally would allow.

Change that by allowing to transfer the xenbus message in chunks as
soon as they are available.

Avoid crashing Mini-OS in case of illegal data read from the ring
buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- drop redundant if (Samuel Thibault)
- move rmb() (Samuel Thibault)
---
 xenbus/xenbus.c | 211 ++++++++++++++++++++++++++++--------------------
 1 file changed, 123 insertions(+), 88 deletions(-)

diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
index 23de61e..ac016e9 100644
--- a/xenbus/xenbus.c
+++ b/xenbus/xenbus.c
@@ -29,6 +29,7 @@
 #include <xen/hvm/params.h>
 #include <mini-os/spinlock.h>
 #include <mini-os/xmalloc.h>
+#include <mini-os/semaphore.h>
 
 #define min(x,y) ({                       \
         typeof(x) tmpx = (x);                 \
@@ -46,6 +47,7 @@
 static struct xenstore_domain_interface *xenstore_buf;
 static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
 DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
+static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
 
 xenbus_event_queue xenbus_events;
 static struct watch {
@@ -231,75 +233,104 @@ char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_
 }
 
 
+static void xenbus_read_data(char *buf, unsigned int len)
+{
+    unsigned int off = 0;
+    unsigned int prod;
+    unsigned int size;
+    int notify;
+
+    while (off != len)
+    {
+        wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons);
+
+        prod = xenstore_buf->rsp_prod;
+        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod);
+        size = min(len - off, prod - xenstore_buf->rsp_cons);
+
+        rmb();   /* Make sure data read from ring is ordered with rsp_prod. */
+        memcpy_from_ring(xenstore_buf->rsp, buf + off,
+                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size);
+        off += size;
+        notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE ==
+                  xenstore_buf->rsp_prod);
+        xenstore_buf->rsp_cons += size;
+        wmb();
+        if (notify)
+            notify_remote_via_evtchn(xenbus_evtchn);
+    }
+}
+
 static void xenbus_thread_func(void *ign)
 {
     struct xsd_sockmsg msg;
-    unsigned prod = xenstore_buf->rsp_prod;
+    char *data;
 
     for (;;) {
-        wait_event(xb_waitq, prod != xenstore_buf->rsp_prod);
-        while (1) {
-            prod = xenstore_buf->rsp_prod;
-            DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons,
-                  xenstore_buf->rsp_prod);
-            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < sizeof(msg))
-                break;
-            rmb();
-            memcpy_from_ring(xenstore_buf->rsp, &msg,
-                             MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                             sizeof(msg));
-            DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),
-                  xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id);
-
-            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons <
-                sizeof(msg) + msg.len)
-                break;
-
-            DEBUG("Message is good.\n");
-
-            if (msg.type == XS_WATCH_EVENT) {
-                struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
-                xenbus_event_queue *events = NULL;
-                char *data = (char*)event + sizeof(*event);
-                struct watch *watch;
-
-                memcpy_from_ring(xenstore_buf->rsp, data,
-                    MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(msg)),
-                    msg.len);
-
-                event->path = data;
-                event->token = event->path + strlen(event->path) + 1;
-
-                mb();
-                xenstore_buf->rsp_cons += msg.len + sizeof(msg);
-
-                for (watch = watches; watch; watch = watch->next)
-                    if (!strcmp(watch->token, event->token)) {
-                        events = watch->events;
-                        break;
-                    }
-
-                if (events) {
-                    event->next = *events;
-                    *events = event;
-                    wake_up(&xenbus_watch_queue);
-                } else {
-                    printk("unexpected watch token %s\n", event->token);
-                    free(event);
+        xenbus_read_data((char *)&msg, sizeof(msg));
+        DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),
+              xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id);
+
+        if (msg.len > XENSTORE_PAYLOAD_MAX) {
+            printk("Xenstore violates protocol, message longer than allowed.\n");
+            return;
+        }
+
+        if (msg.type == XS_WATCH_EVENT) {
+            struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
+            xenbus_event_queue *events = NULL;
+            struct watch *watch;
+            char *c;
+            int zeroes = 0;
+
+            data = (char*)event + sizeof(*event);
+            xenbus_read_data(data, msg.len);
+
+            for (c = data; c < data + msg.len; c++)
+                if (!*c)
+                    zeroes++;
+            if (zeroes != 2) {
+                printk("Xenstore: illegal watch event data\n");
+                free(event);
+                continue;
+            }
+
+            event->path = data;
+            event->token = event->path + strlen(event->path) + 1;
+
+            for (watch = watches; watch; watch = watch->next)
+                if (!strcmp(watch->token, event->token)) {
+                    events = watch->events;
+                    break;
                 }
+
+            if (events) {
+                event->next = *events;
+                *events = event;
+                wake_up(&xenbus_watch_queue);
             } else {
-                req_info[msg.req_id].reply = malloc(sizeof(msg) + msg.len);
-                memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_id].reply,
-                                 MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                                 msg.len + sizeof(msg));
-                mb();
-                xenstore_buf->rsp_cons += msg.len + sizeof(msg);
-                wake_up(&req_info[msg.req_id].waitq);
+                printk("Xenstore: unexpected watch token %s\n", event->token);
+                free(event);
             }
 
-            wmb();
-            notify_remote_via_evtchn(xenbus_evtchn);
+            continue;
+        }
+
+        data = malloc(sizeof(msg) + msg.len);
+        memcpy(data, &msg, sizeof(msg));
+        xenbus_read_data(data + sizeof(msg), msg.len);
+
+        if (msg.req_id >= NR_REQS || !req_info[msg.req_id].in_use) {
+            printk("Xenstore: illegal request id %d\n", msg.req_id);
+            free(data);
+            continue;
         }
+
+        DEBUG("Message is good.\n");
+
+        req_info[msg.req_id].reply = data;
+
+        wake_up(&req_info[msg.req_id].waitq);
     }
 }
 
@@ -451,36 +482,40 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id,
 
     cur_req = &header_req;
 
-    BUG_ON(len > XENSTORE_RING_SIZE);
-    /* Wait for the ring to drain to the point where we can send the
-       message. */
-    prod = xenstore_buf->req_prod;
-    if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE) 
-    {
-        /* Wait for there to be space on the ring */
-        DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n",
-                prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE);
-        wait_event(xb_waitq,
-                xenstore_buf->req_prod + len - xenstore_buf->req_cons <=
-                XENSTORE_RING_SIZE);
-        DEBUG("Back from wait.\n");
-        prod = xenstore_buf->req_prod;
-    }
+    BUG_ON(len > XENSTORE_PAYLOAD_MAX);
 
-    /* We're now guaranteed to be able to send the message without
-       overflowing the ring.  Do so. */
+    /* Make sure we are the only thread trying to write. */
+    down(&xb_write_sem);
+
+    /* Send the message in chunks using free ring space when available. */
     total_off = 0;
     req_off = 0;
-    while (total_off < len) 
+    while (total_off < len)
     {
+        prod = xenstore_buf->req_prod;
+        if (prod - xenstore_buf->req_cons >= XENSTORE_RING_SIZE)
+        {
+            /* Send evtchn to notify remote */
+            notify_remote_via_evtchn(xenbus_evtchn);
+
+            /* Wait for there to be space on the ring */
+            DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", prod,
+                  len - total_off, xenstore_buf->req_cons, XENSTORE_RING_SIZE);
+            wait_event(xb_waitq,
+                       prod - xenstore_buf->req_cons < XENSTORE_RING_SIZE);
+            DEBUG("Back from wait.\n");
+        }
+
         this_chunk = min(cur_req->len - req_off,
-                XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod));
+                         XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod));
+        this_chunk = min(this_chunk,
+                         xenstore_buf->req_cons + XENSTORE_RING_SIZE - prod);
         memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod),
-                (char *)cur_req->data + req_off, this_chunk);
+               (char *)cur_req->data + req_off, this_chunk);
         prod += this_chunk;
         req_off += this_chunk;
         total_off += this_chunk;
-        if (req_off == cur_req->len) 
+        if (req_off == cur_req->len)
         {
             req_off = 0;
             if (cur_req == &header_req)
@@ -488,20 +523,20 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id,
             else
                 cur_req++;
         }
+
+        /* Remote must see entire message before updating indexes */
+        wmb();
+        xenstore_buf->req_prod = prod;
     }
 
+    /* Send evtchn to notify remote */
+    notify_remote_via_evtchn(xenbus_evtchn);
+
     DEBUG("Complete main loop of xb_write.\n");
     BUG_ON(req_off != 0);
     BUG_ON(total_off != len);
-    BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE);
-
-    /* Remote must see entire message before updating indexes */
-    wmb();
-
-    xenstore_buf->req_prod += len;
 
-    /* Send evtchn to notify remote */
-    notify_remote_via_evtchn(xenbus_evtchn);
+    up(&xb_write_sem);
 }
 
 /* Send a mesasge to xenbus, in the same fashion as xb_write, and
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:20:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:20:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187516.336399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSxl-0005XC-D9; Wed, 15 Sep 2021 11:20:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187516.336399; Wed, 15 Sep 2021 11:20:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQSxl-0005X5-9C; Wed, 15 Sep 2021 11:20:25 +0000
Received: by outflank-mailman (input) for mailman id 187516;
 Wed, 15 Sep 2021 11:20:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uklH=OF=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mQSxk-0005Wz-Hx
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:20:24 +0000
Received: from mail3-relais-sop.national.inria.fr (unknown [192.134.164.104])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ac53fdfe-4486-4ce5-94bc-4f2a534ee4ad;
 Wed, 15 Sep 2021 11:20:22 +0000 (UTC)
Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin)
 ([194.199.1.36])
 by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 15 Sep 2021 13:20:20 +0200
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mQSxg-00F8lH-Bk;
 Wed, 15 Sep 2021 13:20:20 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac53fdfe-4486-4ce5-94bc-4f2a534ee4ad
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AO0s7kKurlqaXSTNPnfL9oYNI7skDttV00zEX?=
 =?us-ascii?q?/kB9WHVpm5Oj5ruTdaUgpGbJYWgqKQkdcIa7WZVoJkmyyXcV2/h0AV7GZmPbUQ?=
 =?us-ascii?q?STQr2KgbGSoQEIeBeSygcH78ddmsFFYbWaMbEdt7eY3ODSKbodKbe8gcWVrNab?=
 =?us-ascii?q?5HMoaQFudq16qyZwDQufHkAefnggObMJULGZovddrz27YHINc4CHBn0IRfXKo8?=
 =?us-ascii?q?SjruOBXSI7?=
X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; 
   d="scan'208";a="393037543"
Date: Wed, 15 Sep 2021 13:20:20 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] mini-os: xenbus: support large messages
Message-ID: <20210915112020.iy6vqn4qcvtbohyi@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
 <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mer. 15 sept. 2021 12:48:44 +0200, a ecrit:
> On 15.09.21 01:17, Samuel Thibault wrote:
> > > +        prod = xenstore_buf->rsp_prod;
> > > +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons, prod);
> > > +        size = min(len - off, prod - xenstore_buf->rsp_cons);
> > > +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
> > > +                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons), size);
> > > +        off += size;
> > > +        notify = (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE ==
> > > +                  xenstore_buf->rsp_prod);
> > 
> > This looks odd to me?  We want to notify as soon as the ring is empty,
> > which can happen at any place in the ring right?
> 
> No, we want to notify if the ring was full and is about to gain some
> space again, as the other side was probably not able to put all data
> in and is now waiting for more space to become available.

Ok, that said, the producer may fill the ring between this test and
the rsp_cons update, and thus the producer will go sleep and here the
consumer will not notice it and thus never notify it.

So we really need to make the test after the rsp_cons update, like Linux
does:

> > Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after*
> > the rsp_cons increase.

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:45:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:45:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187525.336413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTLa-0008BX-Cf; Wed, 15 Sep 2021 11:45:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187525.336413; Wed, 15 Sep 2021 11:45:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTLa-0008BQ-93; Wed, 15 Sep 2021 11:45:02 +0000
Received: by outflank-mailman (input) for mailman id 187525;
 Wed, 15 Sep 2021 11:45:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQTLZ-0008BG-B2; Wed, 15 Sep 2021 11:45:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQTLZ-0004w2-4V; Wed, 15 Sep 2021 11:45:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQTLY-0007OU-Qq; Wed, 15 Sep 2021 11:45:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQTLY-0003AW-QQ; Wed, 15 Sep 2021 11:45:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dHwsC80KoOrZQBSTWh5A/vrr072g7s4HUXIq3AFikPI=; b=lFr/MZKD8zwOKcdWf+4OWo8Far
	4a81P2CzsfP2GEmWXolAX0vPESLAQS7DPyIvZVLgjGb/AEw9SArYp+myd3+BKGTeCMSSPA6SSV+F3
	fJY98ZIT/bsZ1Ml8cjmW9SBBNVycsGAjxRRtZ65A3MgvFQv7v0gsikRmDzNL+rYk97z8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164987-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164987: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a89bcd9737757e4d671783588a6041a84a5e1754
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 11:45:00 +0000

flight 164987 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164987/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164983
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164983

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a89bcd9737757e4d671783588a6041a84a5e1754
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    4 days
Failing since        164951  2021-09-12 00:14:36 Z    3 days    7 attempts
Testing same since   164976  2021-09-14 01:38:33 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit a89bcd9737757e4d671783588a6041a84a5e1754
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Wed Jul 7 09:15:31 2021 +0200

    tools/go: honor append build flags
    
    Make the go build use APPEND_{C/LD}FLAGS when necessary, just like
    other parts of the build.
    
    Reported-by: Ting-Wei Lan <lantw44@gmail.com>
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Acked-by: Nick Rosbrook <rosbrookn@ainfosec.com>
    Acked-by: Ian Jackson <iwj@xenproject.org>

commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 11:55:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 11:55:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187534.336428 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTVl-0001KV-Gd; Wed, 15 Sep 2021 11:55:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187534.336428; Wed, 15 Sep 2021 11:55:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTVl-0001KO-CW; Wed, 15 Sep 2021 11:55:33 +0000
Received: by outflank-mailman (input) for mailman id 187534;
 Wed, 15 Sep 2021 11:55:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tFB7=OF=epam.com=prvs=9892afeb01=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQTVj-0001KI-S3
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 11:55:32 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3529b484-06ef-42ef-aa8d-de39ec1c8679;
 Wed, 15 Sep 2021 11:55:30 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18FAwY3x017285; 
 Wed, 15 Sep 2021 11:55:28 GMT
Received: from eur03-db5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2059.outbound.protection.outlook.com [104.47.10.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b3d53rsnd-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 15 Sep 2021 11:55:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5940.eurprd03.prod.outlook.com (2603:10a6:208:15a::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 11:55:25 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 11:55:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3529b484-06ef-42ef-aa8d-de39ec1c8679
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PspGxfzzQLPU4lVATu3MAx+j3BmD/YDGx1RE0oblgM+SMpjfxBZh+uB9YGPdar+TN38sz2NmSu38pFWWeoxuxn732E2r0cEpr19AcPl4Bi43vcflLYfzFL13WoY9Urz6N5MsWNB5iQ32cXa0HS8pAz8JRdHk12bmIv7IthuS35kQ5YTt0KKQAcSuRe9Cx6vywEJusipS7vFXWodXms2Iz2KVp3PmIO4QpSsJvHZTszoCy6vAjWdz5YmyxCFBwqAHp7KVykBM13jnjXJxAp8Cw1k82RmvBQP0h0CquGAjWrIFzJhgzSDClPDWMONY77LdiTVjyWzItEO6KDZMHMT/dg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aJ4furlTlTUdKoH9lwvuuuT37pwVAddPVfoym+Z5CPc=;
 b=OycMEN1WeAHLjHjQGco6wdY/jaNRYI9hMhBU72zfI8VQ8DBqTWeusyt267zpOxyldHyuB8J7K4D9wQc8vLtLcyPVy4MQkGZB/CvDFJ2gCv9aMH5MwLXGwjuM2miHslitnjDO+xkgXsOTbgMfddyNkCjFH9/GMlDVSnuJuLt9QzgQh5v1BJD0B83CG8w5GJOAtp6B0R49XhLRVopKhnukP119GJ866+VWhERG+ttxOS4iZYmG63y75Ai1M45KipO1OsIiuvUcIOizq3IyIx5W65YwTCHFth6pWhvtSZNPJzNa37gBCZnLGgUehsgjsl4QVzKnXIqCTAvRSsOXks6DsQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aJ4furlTlTUdKoH9lwvuuuT37pwVAddPVfoym+Z5CPc=;
 b=u5h31lRJx5Z6KaX/CpjP4weg+kY7KhgmjUJLmoOgf0Ef76fowNylc2clg8AgCXWdXeMcxv/YCabVm8mR1+xEa5gaA7NrBC33GtnRGDPgm9n2mMJZp4FCaB9TEj0C6bYC/BV6Ey8C+C0ILQ0gezil0BElV3LzMu1TH37Mf6eSc+0tmGrg7JAvMnlp78HQ5nmpjtci1Y+5jiFIVtifIR0FtgDip3CG7wtabKGUV9I1Ac3UqG/Lu6WkVhYrAlxu64fAWV/G13fB78dinf6g++0LlZkQwuQCyAsaRdzryGcJ24rCyKMlpW+HT7G1mTCNt7dRf8q8RU7Ku0rQwTnkw8vtkA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKudvyQAgAXn/ACAAP1GAIAAWBmAgAATZQA=
Date: Wed, 15 Sep 2021 11:55:25 +0000
Message-ID: <067006ee-5390-dc68-c10e-acf236c0e69d@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
 <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
 <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
 <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
In-Reply-To: <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2725d1da-eae2-45ac-28ba-08d9783fb463
x-ms-traffictypediagnostic: AM0PR03MB5940:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5940D3BC8FC1911BBE94F126E7DB9@AM0PR03MB5940.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 1fzv5gbNgOc2ZaV8tYOu+Bv5r/+W0vCUUtm26DXRKcSl7xZ0ajk791VAJnlYBMk/sUqtvclfptvaluu5BqZvvJDCmUSFC9Z5PkErkNX9u7YMlzu1NTKjtyfm7okWBowdt3C7NteC8KHicCV7kk6lBtqL5oeTv0pKwHcKXYbmDEFALD6g75umbWDb9T4B40YfLMRtkEt3Ko6XyURYZWJgbCaPoDDmUeExOtgVOJkMAdXeUpTPegCt5MFYSAoIRyVCu5Ek/BF/IG0TWjClQ/obq14DpG8/Cy92eIlgNvwCsh6h9ni5zo2YuJ7tfjRxarUvPbS0s6KWvsoL4BhMck+/y/11h6F/4s1gORW059YdZYxctf0QG88uF5YxvRfxM0wzAgiw44T6q5/qWkItQrnLJJelP9+3JUIWqtZfIvtN6ROBIWCfs/oRWkt4e1kkJEV/4+4rNR1OOWlbwcKjbiSO/2JvibnkehZ8xWXG4BUocnnEgaxV6mIUyMbXSnEalvIcRq/2ECMniphj62LTgcE3+gvPK5aBOaau/gtAGiQlNM/NQoLH1bcwCkQAswCY0RNyF3T5jRvSEzXtCrhupGJvydN3ax9qVBIDeF3ZNT9tYURnog0pmYhtNZDBlO0FgBl8nZ0KZPj8fY1yCfHBg7CAXiOqUDHAZA6kKxBcvs8y1wPzSQXZ0vZkRfoDEDNZOExiWU4aJKwlGJeqLzti890/MhRmbmihLwPDyT0iCDnXv/BWp9UTXzMK4XMG8XtO/uUFj/kDnrvl6uqyJAu7xoFC8CWFOhvHrQ9kKSrySlXlazHEdkTqU0qyEsOWhMrSCijX7xn2B29azOnu868hy+zqQw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(376002)(39860400002)(346002)(396003)(136003)(2616005)(186003)(966005)(76116006)(36756003)(66476007)(6506007)(64756008)(4326008)(2906002)(54906003)(26005)(6486002)(31696002)(31686004)(316002)(478600001)(8936002)(38100700002)(83380400001)(5660300002)(6512007)(53546011)(66946007)(66446008)(6916009)(8676002)(86362001)(55236004)(38070700005)(71200400001)(66556008)(122000001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?V0xJdDNBTEpuTU5lV1JsdGZEQld5Sks1VTVnRlpaUENTemVWYTVuUWdESi96?=
 =?utf-8?B?MENTTVlmZXVpdFpkV1N2OW9BTXAvSmluTnI3OWNkNHQwUTJKOUxOaGFJNjJC?=
 =?utf-8?B?VHBRNmJKZFVuL0FDYUVhSkxUU2w1S2E4WkZOKy94c2V0ZDVYOFQ4Unp2MGlw?=
 =?utf-8?B?aEllWlhuMklZN3N6K3lqY0lCK2JPbjRHWUIwSDZkd3hMNDZ5blc2bDV2ZVRN?=
 =?utf-8?B?ZkdrNEJqK211YWhhU0REc3ZsR3hVdWRZN0FETzdZWUJWK24rdERvLy9VZ20z?=
 =?utf-8?B?OUo2VjV3bGxUMGxtUzBJTzFJSVpPaUZJRkVqQ1U4dlRDTGtmKytOTzNBN05k?=
 =?utf-8?B?cVlkNE0yWTlRc2RiRVN6N010LzlQSlRiODJtZlQ4VnBCeDY5ZEo1STBPUDBD?=
 =?utf-8?B?ZjlieHVBTGgrVitrTWlmdEJSYnd0YlJlalVjOGVid2FHSmp3RVRpUXlSa0dy?=
 =?utf-8?B?TXNjdFhkN05WUlRkTHNtcHJzTHVYU3dpQ2U5eTh1bXZsWHFySFU1bDlqdlEz?=
 =?utf-8?B?OUxMc0Jhcm92bTFMQjJ6SUxrTFE2UlJ5YnZaU1dzeGJaaldweDJZK1U4WVVn?=
 =?utf-8?B?cHZuSlRaeU1wdWZFWU1MU05nSU1UWUE4OFY2cWZTWGVJL0ljWjlhb04vbzJO?=
 =?utf-8?B?Nml0bHUybGRoemVENVVJclo4MzRsRURvVUR0UXVoNDR0N0VZdCsxcUtKN1do?=
 =?utf-8?B?MkNlZFhLbFViOFllNWFyNTlGMnVDM2xuMXNHMCs2aFVwQmdwc1lFTE11OHBT?=
 =?utf-8?B?SkJHWVNCLzRWcE54M0JnRDJKVXh5bXpGaVFzWUU0cWdhY1pZbTVLV2poK1lI?=
 =?utf-8?B?TTczeVFGbi9jRFlXVkwwTEs4NG1TUzdmUGU0L050d2c5Nzg4VDlRMHB0UGVP?=
 =?utf-8?B?U3ZFNDdmTVFobHNmVy9JQ1R2Z2pIY1ZEQnV6U2MwV2JSOTNlNURCVm9pWjlr?=
 =?utf-8?B?ZCtNMmdJZ1lhYVFGK1lzWDRHYXlDaDlWQWlIMzc3VThpNHp2R2hUUUIzbGNi?=
 =?utf-8?B?bFNzanBmaTd0MGk0a3VoVVV6ZG53dG1abzJ6Y0ZYdi9TWEQzc1ZpWU95Um8v?=
 =?utf-8?B?Q0tKRDFSdkN3ODh1am1pZ2JOYVFXSXp3dUFSV0pnYndKNzh0dlZmaXpUSkEz?=
 =?utf-8?B?a09SMnZPcEY0TUJ0d3hSZzRqTUlsY1lYQzZ3YWYvZ3lKb04zaGh5aUVCbEVZ?=
 =?utf-8?B?L2RyQjNQSU5WbVBhbmtlem9FNGdpVXh6STduMUxhcnA2REs1OTU2QmpqWk5h?=
 =?utf-8?B?WHR2alVjYVc2UEE1OE1kSVFVV255eENVMkxMWUZXTHdZM1Q4NkloaHNabEtw?=
 =?utf-8?B?bExkYm9yays3ejdwNk1Cb0pWU2o5dDlhR1pKbjVWSi9jLzRKallnQm43R2Zi?=
 =?utf-8?B?cklxa3Q5SmRsNE1aemZ6V3V6QXBFcmp2S3Z4RDJWOVo0Z3dGOWVEaTEveWlZ?=
 =?utf-8?B?N3NKUFVtcFM5NDFET2hxZSt3UlI5L0tRc1pZRlhwYUJzVkVEeEdEQk5ZenJ3?=
 =?utf-8?B?NXppZk5nalQ2QlNBN09mMWw0U2JucG9DTkNuTURjRktKYkRsSnptb0hGYzJV?=
 =?utf-8?B?MUFyY2dXZkgvbmR5bzIrdzhyeVlDK0xPZktvZGhpUTg4TGxZSWpCT0JiejBL?=
 =?utf-8?B?bVR6bFlPL1pLc21iVm5WTHc4SVRSVkgwS2ZROGZJVlBCaFM5N1RCRWNaTWZs?=
 =?utf-8?B?aVVNVmhkcjhUb1ZHSm55bFFtOTZnb0RvQ1ZqVmh0dTduUWQ3S0dtd3RubEV3?=
 =?utf-8?B?bDNoejNUU2NZTDF0UzVxc3VCMDhoaHBTcmZkSlIzMW45bk5iK1o1dUR3dEpm?=
 =?utf-8?B?Yk9JaHhiVmk2QUJKRHIyZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <EC16366D5AEEB942AC26FB36F80C5697@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2725d1da-eae2-45ac-28ba-08d9783fb463
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 11:55:25.3041
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rUvuohleohhyv4KgK4ZjUDPb/kl0cXrIgqoAZTGB7Kl2aVPiHtOpgXE9pGH5Jvs2XZ+yIwEw9A9Y/8NhW4JGlc3BE/y/Zpf68zLZ4f0zoFqDCihkffzSExutwgtl535y
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5940
X-Proofpoint-ORIG-GUID: cbZMibfbrbxj0G02uT6UJSaovG4nRGDI
X-Proofpoint-GUID: cbZMibfbrbxj0G02uT6UJSaovG4nRGDI
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-15_03,2021-09-15_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 bulkscore=0
 impostorscore=0 clxscore=1015 mlxlogscore=999 priorityscore=1501
 mlxscore=0 lowpriorityscore=0 suspectscore=0 phishscore=0 spamscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109150077

DQpPbiAxNS4wOS4yMSAxMzo0NSwgUmFodWwgU2luZ2ggd3JvdGU6DQo+IEhpIE9sZWtzYW5kciwg
U3RlZmFubywNCj4NCj4+IE9uIDE1IFNlcCAyMDIxLCBhdCA2OjMwIGFtLCBPbGVrc2FuZHIgQW5k
cnVzaGNoZW5rbyA8T2xla3NhbmRyX0FuZHJ1c2hjaGVua29AZXBhbS5jb20+IHdyb3RlOg0KPj4N
Cj4+IEhpLCBSYWh1bCENCj4+DQo+PiBPbiAxNC4wOS4yMSAxNzoyNCwgT2xla3NhbmRyIEFuZHJ1
c2hjaGVua28gd3JvdGU6DQo+Pj4gfQ0KPj4+Pj4gICAgK3N0YXRpYyBpbnQgcGNpX2VjYW1fcmVn
aXN0ZXJfbW1pb19oYW5kbGVyKHN0cnVjdCBkb21haW4gKmQsDQo+Pj4+PiArICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJp
ZGdlLA0KPj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBtbWlvX2hhbmRsZXJfb3BzICpvcHMpDQo+Pj4+PiArew0KPj4+Pj4gKyAgICBz
dHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0YTsNCj4+Pj4+ICsN
Cj4+Pj4+ICsgICAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsIG9wcywgY2ZnLT5waHlzX2FkZHIs
IGNmZy0+c2l6ZSwgTlVMTCk7DQo+Pj4+PiArICAgIHJldHVybiAwOw0KPj4+Pj4gK30NCj4+Pj4g
R2l2ZW4gdGhhdCBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgaXMgZ2VuZXJpYyAoaXQgaXMgbm90
IHNwZWNpZmljIHRvDQo+Pj4+IG9uZSBicmlkZ2UpLCBJIHdvbmRlciBpZiB3ZSBldmVuIG5lZWQg
dGhlIC5yZWdpc3Rlcl9tbWlvX2hhbmRsZXINCj4+Pj4gY2FsbGJhY2sgaGVyZS4NCj4+Pj4NCj4+
Pj4gSW4gZmFjdCxwY2lfaG9zdF9icmlkZ2UtPnN5c2RhdGEgZG9lc24ndCBldmVuIG5lZWQgdG8g
YmUgYSB2b2lkKiwgaXQNCj4+Pj4gY291bGQgYmUgYSBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cq
LCByaWdodD8NCj4+PiBSYWh1bCwgdGhpcyBhY3R1YWxseSBtYXkgY2hhbmdlIHlvdXIgc2VyaWVz
Lg0KPj4+DQo+Pj4gRG8geW91IHRoaW5rIHdlIGNhbiBkbyB0aGF0Pw0KPj4+DQo+PiBUaGlzIGlz
IHRoZSBvbmx5IGNoYW5nZSByZXF1ZXN0ZWQgdGhhdCBsZWZ0IHVuYW5zd2VyZWQgYnkgbm93Lg0K
Pj4NCj4+IFdpbGwgaXQgYmUgcG9zc2libGUgdGhhdCB5b3UgY2hhbmdlIHRoZSBBUEkgYWNjb3Jk
aW5nbHksIHNvIEkgY2FuDQo+Pg0KPj4gaW1wbGVtZW50IGFzIFN0ZWZhbm8gc3VnZ2VzdHM/DQo+
IFdlIG5lZWQgcGNpX2hvc3RfYnJpZGdlLT5zeXNkYXRhIGFzIHZvaWQqIGluIGNhc2Ugd2UgbmVl
ZCB0byBpbXBsZW1lbnQgdGhlIG5ldyBub24tZWNhbSBQQ0kgY29udHJvbGxlciBpbiBYRU4uDQo+
IFBsZWFzZSBoYXZlIGEgbG9vayBvbmNlIGluIExpbnV4IGNvZGVbMV0gaG93IGJyaWRnZS0+c3lz
ZGF0YSB3aWxsIGJlIHVzZWQuIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyBpcyB1c2VkIG9ubHkg
Zm9yDQo+IGVjYW0gc3VwcG9ydGVkIGhvc3QgY29udHJvbGxlci4gRGlmZmVyZW50IFBDSSBob3N0
IGNvbnRyb2xsZXIgd2lsbCBoYXZlIGRpZmZlcmVudCBQQ0kgaW50ZXJmYWNlIHRvIGFjY2VzcyB0
aGUgUENJIGNvbnRyb2xsZXIuDQo+DQo+IFsxXSBodHRwczovL3VybGRlZmVuc2UuY29tL3YzL19f
aHR0cHM6Ly9naXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5lbC9naXQvdG9ydmFsZHMv
bGludXguZ2l0L3RyZWUvZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2llLXJjYXItaG9zdC5jKm4z
MDlfXztJdyEhR0ZfMjlkYmNRSVVCUEEhbWJJX2l1dS1sYVlRb1VuMzZrS2YzejJINEF5eFI0SjhD
NTlDY0tiMjFwTGxkeVZuRGFLYmdKU1FoWjRsaUVud25BZTJ1eks4T0EkIFtnaXRbLl1rZXJuZWxb
Ll1vcmddDQo+DQo+IEkgdGhpbmsgd2UgbmVlZCBicmlkZ2UtPnN5c2RhdGEgaW4gZnV0dXJlIHRv
IGltcGxlbWVudCB0aGUgbmV3IFBDSSBjb250cm9sbGVyLg0KPg0KPiBSZWdhcmRzLA0KPiBSYWh1
bA0KU3RlZmFubywgZG9lcyBpdCBzb3VuZCByZWFzb25hYmxlIHRoZW4gdG8ga2VlcCB0aGUgYWJv
dmUgY29kZSBhcyBpcz8NCj4gICANCj4+IFRoYW5rcywNCj4+DQo+PiBPbGVrc2FuZHI=


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 12:03:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 12:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187544.336443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTdg-000315-I1; Wed, 15 Sep 2021 12:03:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187544.336443; Wed, 15 Sep 2021 12:03:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTdg-00030y-Ev; Wed, 15 Sep 2021 12:03:44 +0000
Received: by outflank-mailman (input) for mailman id 187544;
 Wed, 15 Sep 2021 12:03:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XwCH=OF=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQTdf-0002yV-8Y
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 12:03:43 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e30529ed-5b9f-496f-bba0-b9d1997f84a0;
 Wed, 15 Sep 2021 12:03:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 0F69A20187;
 Wed, 15 Sep 2021 12:03:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DDABD13C29;
 Wed, 15 Sep 2021 12:03:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id sIa1NBfhQWFYLgAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 15 Sep 2021 12:03:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e30529ed-5b9f-496f-bba0-b9d1997f84a0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631707416; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=ENhmOeCtsm5hfqnNK954lymn6nLMXDrcKR3qancrAyk=;
	b=JbRqSckOJdoT1BrjCsXvigpfRJUYL2Wj9ROLqtKkvTLGYrlfvmyIS/tHSvvKx/dfX0Ma1S
	UF9ZFAjxmMcrNt6QbgMQ0lAAuZADFJO78nYCNBE8Dx3wN4z4lQgxmvIeclDz2W2Kn3/dMF
	lQBYvEqjc7HmdSy7wtLybB51/WxVjv8=
Subject: Re: [PATCH] mini-os: xenbus: support large messages
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
 <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
 <20210915112020.iy6vqn4qcvtbohyi@begin>
From: Juergen Gross <jgross@suse.com>
Message-ID: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com>
Date: Wed, 15 Sep 2021 14:03:35 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210915112020.iy6vqn4qcvtbohyi@begin>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA
Content-Type: multipart/mixed; boundary="CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
Message-ID: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com>
Subject: Re: [PATCH] mini-os: xenbus: support large messages
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
 <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
 <20210915112020.iy6vqn4qcvtbohyi@begin>
In-Reply-To: <20210915112020.iy6vqn4qcvtbohyi@begin>

--CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp
Content-Type: multipart/mixed;
 boundary="------------A0BD0CE41CC0BB783A29960C"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A0BD0CE41CC0BB783A29960C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 15.09.21 13:20, Samuel Thibault wrote:
> Juergen Gross, le mer. 15 sept. 2021 12:48:44 +0200, a ecrit:
>> On 15.09.21 01:17, Samuel Thibault wrote:
>>>> +        prod =3D xenstore_buf->rsp_prod;
>>>> +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons=
, prod);
>>>> +        size =3D min(len - off, prod - xenstore_buf->rsp_cons);
>>>> +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
>>>> +                         MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),=
 size);
>>>> +        off +=3D size;
>>>> +        notify =3D (xenstore_buf->rsp_cons + XENSTORE_RING_SIZE =3D=
=3D
>>>> +                  xenstore_buf->rsp_prod);
>>>
>>> This looks odd to me?  We want to notify as soon as the ring is empty=
,
>>> which can happen at any place in the ring right?
>>
>> No, we want to notify if the ring was full and is about to gain some
>> space again, as the other side was probably not able to put all data
>> in and is now waiting for more space to become available.
>=20
> Ok, that said, the producer may fill the ring between this test and
> the rsp_cons update, and thus the producer will go sleep and here the
> consumer will not notice it and thus never notify it.
>=20
> So we really need to make the test after the rsp_cons update, like Linu=
x
> does:
>=20
>>> Linux' code uses (intf->rsp_prod - cons >=3D XENSTORE_RING_SIZE), *af=
ter*
>>> the rsp_cons increase.

Oh, you are right, of course. How could I overlook this?

Thanks,


Juergen

--------------A0BD0CE41CC0BB783A29960C
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A0BD0CE41CC0BB783A29960C--

--CMMeocb5gxwlK7fdJkh0AgWtsvXXjTXOp--

--ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFB4RcFAwAAAAAACgkQsN6d1ii/Ey8A
ywf/RgULYRjT5cSCcoBw6DISFWGq+0kG0JV/52VpYOvEuHQ4QB2jfdSW2FnhiI+zi7AhhmurF2fI
W9VacbvdcDjQU6YLWXVRGQkirLAIg9LPZxtE8XTU9kZ0GS+PKsfAuKvm0YmmNVzT9tltsPMRiivL
aqzHi4sdioQhpBxMi2uUpTqZhoX1k2T3uokEA/WkGOQpaWFxsVUuy+9uwu8hndtJY07leHazIpun
X5oH5XuBoJOOhH9XvRfXOZuZ6Deq+WbzKnyZLY2eCeOM9L5uwK6On/awQe+zuRgvVZ1LTKZ0JxXS
wgIfraKDHxZ77q1+acbEVg706lemRhj2Xw3PHzPcCA==
=0vWU
-----END PGP SIGNATURE-----

--ScL0Va8R8fzjJJz3Nv9BCmmjeBa35z0nA--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 12:20:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 12:20:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187553.336454 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTti-0005X0-W5; Wed, 15 Sep 2021 12:20:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187553.336454; Wed, 15 Sep 2021 12:20:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQTti-0005Wt-T2; Wed, 15 Sep 2021 12:20:18 +0000
Received: by outflank-mailman (input) for mailman id 187553;
 Wed, 15 Sep 2021 12:20:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uklH=OF=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mQTti-0005Wn-9w
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 12:20:18 +0000
Received: from mail2-relais-roc.national.inria.fr (unknown [192.134.164.83])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4819fa30-161f-11ec-b508-12813bfff9fa;
 Wed, 15 Sep 2021 12:20:16 +0000 (UTC)
Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin)
 ([194.199.1.36])
 by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 15 Sep 2021 14:20:13 +0200
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mQTtd-00FA9l-7I;
 Wed, 15 Sep 2021 14:20:13 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4819fa30-161f-11ec-b508-12813bfff9fa
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A0bEInaoqM1KGkjgH0DbTKQkaV5oweYIsimQD?=
 =?us-ascii?q?101hICG9Ffb5qynOppUmPHDP4gr5NEtQ++xoW5PvfZqjz+8X3WB5B97LNzUO01?=
 =?us-ascii?q?HFEGgN1+bfKnHbdBHDyg=3D=3D?=
X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; 
   d="scan'208";a="528588671"
Date: Wed, 15 Sep 2021 14:20:13 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] mini-os: xenbus: support large messages
Message-ID: <20210915122013.5fswso7poab5iyzq@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20210818152610.6114-1-jgross@suse.com>
 <20210914231742.pxpac2lhp2kk5ook@begin>
 <2304b885-434c-8a19-78e0-14637a4adee3@suse.com>
 <20210915112020.iy6vqn4qcvtbohyi@begin>
 <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20e4fa88-74a8-35a0-dffa-fd5e5765fe0d@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mer. 15 sept. 2021 14:03:35 +0200, a ecrit:
> On 15.09.21 13:20, Samuel Thibault wrote:
> > So we really need to make the test after the rsp_cons update, like Linux
> > does:
> > 
> > > > Linux' code uses (intf->rsp_prod - cons >= XENSTORE_RING_SIZE), *after*
> > > > the rsp_cons increase.
> 
> Oh, you are right, of course. How could I overlook this?

One person alone will always overlook something, that's why we make
code review, to have several pairs of eyes check whether we may have
forgotten something :)

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 12:42:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 12:42:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187562.336469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUEw-00088F-Sd; Wed, 15 Sep 2021 12:42:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187562.336469; Wed, 15 Sep 2021 12:42:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUEw-000888-Oo; Wed, 15 Sep 2021 12:42:14 +0000
Received: by outflank-mailman (input) for mailman id 187562;
 Wed, 15 Sep 2021 12:42:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUEv-000882-RZ
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 12:42:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 58cf45ee-1622-11ec-b50d-12813bfff9fa;
 Wed, 15 Sep 2021 12:42:12 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-to2CDSwFO_m2MK3wGzJ1AA-1;
 Wed, 15 Sep 2021 14:42:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2448.eurprd04.prod.outlook.com (2603:10a6:800:52::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 12:42:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 12:42:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Wed, 15 Sep 2021 12:42:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58cf45ee-1622-11ec-b50d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631709731;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=eR0V1NG0NMLIk9uijdWykWXvWlreOAGUhZ7ZSgPcfFY=;
	b=ceKjgrMRzWZXZQVt09n6SbsQ3362FaLBgCsqSykUr+G3fZGIdpp8ZETzrCKfuRh71glB6j
	7zPBFqYWtQDos9qAnbdmOfdf6PhAXK1Tx/CqMgFiQgcLrePmYQoI9fmKt28M/aobwdJ0q0
	SM+8hCqb/mrZUMTadn1c2JJuJEq1N34=
X-MC-Unique: to2CDSwFO_m2MK3wGzJ1AA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RJStghtHSBspysOYcruAV3eqfNj85sH6F2ozjP3LwdQsD0+F3FI9hJIWlU0NfA/t22mHU5WKwPd56tODvB7aoDCzBilShkt6Rvi6q0BXPSf2ug9PUnKpxLKBRn7MpDjM7XBxrbXIueHUmRxoVkPzyjfn3AHL2iyBQ6Ca5Wsq1qZhxO9juCJrhtrm7K4jvU/HCYzhG6gj/7e2jJv4gHprxAFamPXLvOaepc89bvQiwBiv19dVYNwKE+PNM+YRFrGdP9aX8L6M9bSG3sK0Vj04fIP+qVPLqX8Ff926e2J6Yk2jb5blAzT0USuCai6kOcnqihksOO2gBzuJh9YOjgDDMg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eR0V1NG0NMLIk9uijdWykWXvWlreOAGUhZ7ZSgPcfFY=;
 b=l2cycB1TTsaWj9kUjiHZ1XI4l6+pG7Cu9xlRevQB6xAtW64c/2r1G8t+FUdvRDmOKm6jJSdcfnrPR1hi/veJYoAJQCHv9FTkVxP21yM5+P1peBvJj/Tr4BCXDfqpXkgOh7QiraDw5Paj82b4z8BPCV6n9SPRDoVRj8In4cicustBj3w61GbErObB1Ldpc37wptI/9PbpdfT6j9sIq2FqasMYzz2E/vLq9sCFjknjj7XmOgXENjyd1NwmqPSS7YAETd/OkPNcSCObmZT8Qn76r0sV5GKbMlO59VcJpMKQ+AC/AlZiYj8uiYu0Cf3sQVEXFfD+jMsuEyTOZus9DA/Mxw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Ping: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
From: Jan Beulich <jbeulich@suse.com>
To: Kevin Tian <kevin.tian@intel.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
Message-ID: <9fb1d554-8fcc-fc0c-2b0c-52756001d36d@suse.com>
Date: Wed, 15 Sep 2021 14:42:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6bf5ec3b-295b-4f72-aa2c-08d978463b29
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2448:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24487D1D3105409D7347626EB3DB9@VI1PR0401MB2448.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1013;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cl9ildsgYYKtex/Oqtrbebt68Ed4ABHhf4708QsuSyMEG9aIY8S3i5wzsPnpjyiawGIvAczYMbJcBq73vz2f/QF5dVavE8udmdjnIhrKjtMtAW04+xi9LKorKhjCqt8pLwW70WrJvN6m0SMvlpAJCHslyjl/9qGVAJsEBhnhVdKdzAJbBh5OhZ32dvsyZUrSgQHLPlkLneJgu262RGME25PEfXYrRoYwzgc3tLNh3daA90ODShzljsq8F61HrVSPD6sRlqsyxuDW59q0JM1aKlLHFnUfCdq2sgDtvb/b/wDuAVhzHYZI/FBA0CKjiEGjhG0bgf2RNv+Tcbm5CpGxLiK3qySBLVr+tFtGDu4zRqasH50Gm2Nna4vVW0twMIf5d+nwjU0DWNhAC9EF+fjeHwD2Y8Atp4eMuyMrcW4UK67+ldiYUg/tEmOxEOGGuNOaMlpUgG6GT/IsgvVSH5i6TvlZqDMXeHhDbmYYxj07FfR6q0h3teN4oiEryux4W9GzFNB+orD55gIxFGhGLOSYmg3S3MKuWkIGSrEHTCLsdBYYJJ+f82QTXmotn935JBZywKcdkBd1xjlMABBjsNKvcr596+YTidP7Y91ff92/7TNeBPFAeONiyNXZqn8bN9UiUrUyC/nKJzGNar6sfyr/fkQCO2d7cDBqBdble43jBb+UylUCvFg3W05Wi+0abOgK1nEBxw2vMjs+5/WPfGbIaG0F1rMqpoTLNotHatnQEoU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(39860400002)(346002)(396003)(136003)(4326008)(2616005)(186003)(36756003)(66476007)(956004)(2906002)(54906003)(26005)(6486002)(31696002)(31686004)(316002)(16576012)(478600001)(8936002)(38100700002)(5660300002)(53546011)(66946007)(6916009)(8676002)(86362001)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Y1dBbHBUamlQMjhhNjBqa0x3cnlNWnVFOHVMOUgyZ0loQ0UzdjZ0U1FUWVY1?=
 =?utf-8?B?bnNDSEtQY2hZenZWUWhMZDhidkx3MStseW44SWlpSE5WcWRpdlFoVmcveDhL?=
 =?utf-8?B?WmJBL3NEcUxGYWdtQjhlK0I2L205Y0JRVDlPcGFCVXQ5aHBRVloyNVFsR1l4?=
 =?utf-8?B?RE1LeXdqYlBlUWk2cFpnMzNWdnZsWi9vb25xSllhaVQyK2JjNG1GVHRVcTZk?=
 =?utf-8?B?bUU4eVhIMlZ5eGlVdU9mMFJzaSttTGlVZjh2cExxazRwMXFNTmFFVy9talAz?=
 =?utf-8?B?TXR6akpkSWFFMXE2VlltSTBvbDdUdkdrN1QwcFhRMW0yUVROTSt5VWJTc2Y3?=
 =?utf-8?B?NjNHbDl3S0FESHUwWkp4dnZ3cXpFRk00MkxoeFJxWDFLRDVFYlZYMkdxMzU4?=
 =?utf-8?B?ZnpLckpUVy93NW1BVkVCTk9USlp2RTZpOHZybWZvVE9yei80c3dtUUUrT1dJ?=
 =?utf-8?B?YW53UmJOSHZYOWNiUHpXUDBFanl1YXJYLzlyU09aZk9DWndKTkNIVU9LTm10?=
 =?utf-8?B?UGE3NFUwbnlDdkpERGR6VVU3WWxwaXU1QXdYd1dLNkYwZ1V2S1B4eXp0VlNk?=
 =?utf-8?B?Z2pHNmd6SGdHZlNDbThQWi9GL0lEejJyd055MUNKYUFLSHlFRXREaVhla0hZ?=
 =?utf-8?B?YWpSRjg0WmdWZVllM1lVYWxJemxjN0I1eFZ1aE92cW15Q0pTM0JJeHhIOS9W?=
 =?utf-8?B?S05BVWtLZ1pzR25nZzl4MXRIQjVwNlA5ZmxCVWx1TEE0b1E3SFBscTRGNHd5?=
 =?utf-8?B?WVFselU1RUg2MGVvVHE1SE5MU2JNZWRUak5RL2IzOVgrQWgzR2pQR0hqdHl6?=
 =?utf-8?B?TmxoTGplUEhTaVJ6bFVnajhYRWtFdWNVR2p0dWFhb1RnSlNNeWFFR08vemNC?=
 =?utf-8?B?TmtyUHk3TE16WW1FN1MveFRvWkNobEVSeklpSUs4bzh3ODlXdm5VeEVvb2hL?=
 =?utf-8?B?UTdFNnBjNUJlRC95RlkwNnp4MGt2R0RnQnF6U2wzV1phTFdpcHpDTHhEUzFn?=
 =?utf-8?B?bEx6S3FuR0xSQ29JazhVTGIzbnAwRDNOYVdSMVZ5ZC9zek5DS1ZiVlFxS1cx?=
 =?utf-8?B?MXRSMkp4K1NxckJNcCtLK21OL2M2N2dkMlBUelNlRldsajIzNGRxRm1oUDFr?=
 =?utf-8?B?VDF1clJBbWh2NXlNaVBRdFpHcGkwNjdPeTRSeERXVFlVMWloUGhDY0twZ0p6?=
 =?utf-8?B?M2E4ZFh6dGFYdnBDN3RoTnJHZ3VxVnhDYWU0a0FnaUk2TVA5TU52TWd5Rlgr?=
 =?utf-8?B?S2MxQ2dlVlllc3I2eDJDaDJJRnNHaW5WN2I4eVdmZFJFR2VBcWVvdGF5MzBw?=
 =?utf-8?B?dVY2ZkFsblJJaVVZMC84Q0VPVm0wZlJQRi8ydzREckdVYW4vRGtXYkhFOWVq?=
 =?utf-8?B?RHhWOHZCUUFrazFzaEFLUncvUStZMFIrRTlKR1UvVXdQbHZnRzNETTllTkxR?=
 =?utf-8?B?bnI5a2RVMVd0MlBnODJIVG0wTDU0OUMyelZSeWZaR0t5UFlsQUUvTXdESVFW?=
 =?utf-8?B?d3FqZTZHUTNtRGxOTUpqalVkSkZIaWU2TEpSMkdOSjhrZ0gxamxUN0diMmdh?=
 =?utf-8?B?UExDaUIySGtnUE5DN2lBVnBvZTlmSGtHYndQakRYV20yUzVHSXhxRkZEM3Vv?=
 =?utf-8?B?TUJFOVMrNjQ1MTJRM05WNFRib25GUFJ1VzhHcHpHaDliOXJlM1d2NlR6cmlR?=
 =?utf-8?B?cUhsSzBZMnVRaklJSjhwd0hDczZwRnpmYmxqUGdnQ1lWL2p5MEV5czdFMGZF?=
 =?utf-8?Q?yVA74DwdhOvpcbHqa7LJeS+t9rfHQU0kOzpi1HD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6bf5ec3b-295b-4f72-aa2c-08d978463b29
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 12:42:08.6755
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: a9u4HktpFU/HqsCpLCjkXHIOW4KQOf0xAaGntSf9bCTwKaPLdYA5Xt+TJfCjikcVZVCb+ldzIW2KnxLPIeOvBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2448

Kevin,

On 24.08.2021 16:27, Jan Beulich wrote:
> map_domain_page() et al never fail; no need to check their return values
> against NULL, and no need to carry dead printk()s.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

may I get an ack (or otherwise) on this patch please? Ideally also
for some other VT-d specific ones in this series? I'd really like
to get in at least some parts of this series, before submitting an
even larger v2.

Thanks, Jan

> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -530,12 +530,6 @@ static void amd_dump_page_table_level(st
>          return;
>  
>      table_vaddr = __map_domain_page(pg);
> -    if ( table_vaddr == NULL )
> -    {
> -        printk("AMD IOMMU failed to map domain page %"PRIpaddr"\n",
> -                page_to_maddr(pg));
> -        return;
> -    }
>  
>      for ( index = 0; index < PTE_PER_TABLE_SIZE; index++ )
>      {
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2800,12 +2800,6 @@ static void vtd_dump_page_table_level(pa
>          return;
>  
>      pt_vaddr = map_vtd_domain_page(pt_maddr);
> -    if ( pt_vaddr == NULL )
> -    {
> -        printk(VTDPREFIX " failed to map domain page %"PRIpaddr"\n",
> -               pt_maddr);
> -        return;
> -    }
>  
>      next_level = level - 1;
>      for ( i = 0; i < PTE_NUM; i++ )
> --- a/xen/drivers/passthrough/vtd/utils.c
> +++ b/xen/drivers/passthrough/vtd/utils.c
> @@ -106,11 +106,6 @@ void print_vtd_entries(struct vtd_iommu
>      }
>  
>      root_entry = (struct root_entry *)map_vtd_domain_page(iommu->root_maddr);
> -    if ( root_entry == NULL )
> -    {
> -        printk("    root_entry == NULL\n");
> -        return;
> -    }
>  
>      printk("    root_entry[%02x] = %"PRIx64"\n", bus, root_entry[bus].val);
>      if ( !root_present(root_entry[bus]) )
> @@ -123,11 +118,6 @@ void print_vtd_entries(struct vtd_iommu
>      val = root_entry[bus].val;
>      unmap_vtd_domain_page(root_entry);
>      ctxt_entry = map_vtd_domain_page(val);
> -    if ( ctxt_entry == NULL )
> -    {
> -        printk("    ctxt_entry == NULL\n");
> -        return;
> -    }
>  
>      val = ctxt_entry[devfn].lo;
>      printk("    context[%02x] = %"PRIx64"_%"PRIx64"\n",
> @@ -151,11 +141,6 @@ void print_vtd_entries(struct vtd_iommu
>      do
>      {
>          l = map_vtd_domain_page(val);
> -        if ( l == NULL )
> -        {
> -            printk("    l%u == NULL\n", level);
> -            break;
> -        }
>          l_index = get_level_index(gmfn, level);
>          pte.val = l[l_index];
>          unmap_vtd_domain_page(l);
> 
> 



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 12:49:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 12:49:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187570.336480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQULi-0000Wh-LK; Wed, 15 Sep 2021 12:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187570.336480; Wed, 15 Sep 2021 12:49:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQULi-0000Wa-IC; Wed, 15 Sep 2021 12:49:14 +0000
Received: by outflank-mailman (input) for mailman id 187570;
 Wed, 15 Sep 2021 12:49:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQULg-0000WO-M1; Wed, 15 Sep 2021 12:49:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQULg-00063R-FP; Wed, 15 Sep 2021 12:49:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQULg-0002Yx-2v; Wed, 15 Sep 2021 12:49:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQULg-0002RU-2S; Wed, 15 Sep 2021 12:49:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7gdPJi47gNaiHLUIcNHEufjzzF8Ji02f1g1nBp8P4Js=; b=D1gGljB5uq+Z+GsyyIe8Nx5ZnI
	io+4Jvg0smzoTiNAxPErxDpZbYHJsybPvBEmXJjfmmVL1XKE6lElomujyV+Cj1FKsM4hIDPtqrAtA
	OK/cj8nYin1MbBdhu1QF1ewR9i1pFmFxVZJA7Vmn8OU2mq9XPeKGjcQOzevf4HrO/RCM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164988-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164988: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=3ca706c189db861b2ca2019a0901b94050ca49d8
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 12:49:12 +0000

flight 164988 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164988/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-amd64-amd64-freebsd11-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                3ca706c189db861b2ca2019a0901b94050ca49d8
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  410 days
Failing since        152366  2020-08-01 20:49:34 Z  409 days  713 attempts
Testing same since   164988  2021-09-15 00:40:46 Z    0 days    1 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255804 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:10:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:10:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187581.336494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUgJ-0003tj-JZ; Wed, 15 Sep 2021 13:10:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187581.336494; Wed, 15 Sep 2021 13:10:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUgJ-0003tc-Fn; Wed, 15 Sep 2021 13:10:31 +0000
Received: by outflank-mailman (input) for mailman id 187581;
 Wed, 15 Sep 2021 13:10:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQUgI-0003tB-4h; Wed, 15 Sep 2021 13:10:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQUgH-0006RM-UB; Wed, 15 Sep 2021 13:10:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQUgH-0003bj-KJ; Wed, 15 Sep 2021 13:10:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQUgH-00084K-Jn; Wed, 15 Sep 2021 13:10:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=vxHldcaSvDssRNTU0RZUxd212RAOmIBBJlS38qTcScc=; b=Eo33T2VHaO9erbMh3v3GNLh8W5
	yVUjBQS+yC/rSfvEAwXt8nk+HmUhBVMozZd9baWFArVS3kdfy2SZKpFhu/+YVSBIz8xPqoWr7IPeV
	fQczBwbEHwJ+CeO3mYt5Q1IhkoZOqbML+ViaESFfkPSMfCW1aY0vgodnw93X+BmlHx9c=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164995-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164995: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=6254920587c33bcc7ab884e6c9a11cfc0d5867ab
X-Osstest-Versions-That:
    xen=487975df53b5298316b594550c79934d646701bd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 13:10:29 +0000

flight 164995 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164995/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  6254920587c33bcc7ab884e6c9a11cfc0d5867ab
baseline version:
 xen                  487975df53b5298316b594550c79934d646701bd

Last test of basis   164989  2021-09-15 01:01:51 Z    0 days
Testing same since   164995  2021-09-15 10:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   487975df53..6254920587  6254920587c33bcc7ab884e6c9a11cfc0d5867ab -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:23:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:23:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187591.336511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUsp-0005XS-QS; Wed, 15 Sep 2021 13:23:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187591.336511; Wed, 15 Sep 2021 13:23:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUsp-0005XL-N1; Wed, 15 Sep 2021 13:23:27 +0000
Received: by outflank-mailman (input) for mailman id 187591;
 Wed, 15 Sep 2021 13:23:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUso-0005XF-F6
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:23:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 19ee43bb-1628-11ec-b517-12813bfff9fa;
 Wed, 15 Sep 2021 13:23:25 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-OqPvAyUvOmOlduviU5dPyA-1; Wed, 15 Sep 2021 15:23:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5742.eurprd04.prod.outlook.com (2603:10a6:803:e5::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 13:23:20 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:23:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0206.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 13:23:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19ee43bb-1628-11ec-b517-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712204;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=eF/F0iAs7jvag1La3RcSAjevpSwKJ/5NYKXkpAFvqX0=;
	b=M0bKyIBd0gEoT3Z41jeOR+cp+MMB4k/QCA/GoOre7D+AcxNc3LDqK2RLZRg8AoWCLfLw8y
	8p2jKBSMEK5i6vCy3j7QGtf5xCKFBhkt8UbT6S97XwF8OTWt46dqniatN2UWEjharLZPuI
	PLSVJ9p71+xy+s5ooAgbMEGc81N2HIY=
X-MC-Unique: OqPvAyUvOmOlduviU5dPyA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NXGsIy23xMAQ91uNmtHTTxnwQiMF3+T039jffcZ6z45hMtZa9r4p641/A00HnEHzn+Rhl+ohQ2XgFmnsj9JsXMSdhbwHcCoS2cQGqKsVzWfntDut7OzgkYaobpOVAa92SBJz/HGOsxRdOR7VwULp63erSKrCjqhzBVtEO06hGtSppAYTge42uzs4l0SnsHTUC+Y/NauWKr/451K7tOnAHw5bTUR3p+IGuHQ56Yl6Cs503MUDi4n71JyYlKCb3OE7uXAT3YbtZHkltkdAk09gEfIjvG5jEfQG4kkix7mIpcE5MKS5iG8AiCi94hL75k+Ctn1nFepd2m0TC9LWe5H4Aw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eF/F0iAs7jvag1La3RcSAjevpSwKJ/5NYKXkpAFvqX0=;
 b=U+gmcNReOYWGevQg0peT/vRTPEBGRLkjvRj8/XqPxzI164DVWIeV5gFepSUaoJ5Vz797e6aYvYRoYZoSAdFd48qse43lLc1VGVyDo8To7mCnxgsZItX35fXvYDzAuUIh2QdU8asYOHxkDyzxKjq+doumDOwolBBtBoNUxGANy5gcZjoigco6WSpLuPXIvJ1nYcoUU+xd8HRhRRl/g+istph+1VJ3qRSjD8SdUQA5Sda9YrMtWhx0RGlhMCP4hAHI4PXGjRih0q3w/LhGqSJKekZvxhjbzX2WlWM21sS6UKETdMtwnQ0vMINZtNwmnnKfZFFL+tP25oQCL0kfxYVk9w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/7] x86/boot: (mostly) video mode handling adjustments
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Date: Wed, 15 Sep 2021 15:23:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0206.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b0d788c5-aa2a-4402-fdee-08d9784bfc6c
X-MS-TrafficTypeDiagnostic: VI1PR04MB5742:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5742839EF87993615D0758B9B3DB9@VI1PR04MB5742.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2887;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+fHAD63yd61/S4Z8we5p1EU7mEzWNLgeqBXfiNjrncyLKmHK/vwpx529iDt/dEPfWlNI55vs4hgyR1LEzHfocVhgoXK2SKEIcmWTmfUVWlC3Euz+gyLzXnrKaKsfGBncikvHD3bncy0pd6/RP3hTNhOfD8w6UREFXcZ6pA0FS61wYGipmkV4UrSc+EbfJdQLtPQlaHS4jGkXxZ9jggsajKoObipkL5hSEKrb0pJqVEuroHADcyxG0wjQUjpCQlTt2jO4uymvu1zcL3MOh+bHYMNbC1+4mGAH5fqv8Ckx2At1df+vrS9Axj1xryiV9CeJHsOQEUlkNmlIWxnRrtWY5+CV0iIU4fTftp3Cd/6GcwamRXRvE/LXglH7+SpEpDxAmVPG/6XUp/9pONaY8PsNscMf1GqILf7xQckS9uioU6irGJZiqEuDurEVKxDZjwfPantp6eqx96oLuLHYBvtN+6GZVSK5Ive6fP0sQbPszSqe0BfI3ezk/p4USUGtFhMeGuBG+0hZ9Okz6KxhEwm/UeoyD/q6E0kYbIKKgOolIA+72yftbatSpR1p9aIQDAqZI1O+BsI/xT3KNc69mLxfHbuULPEdZcdo0DvD9Q2oi4nAkNUuh3hkmDWb6a+WNgFGV1XGYYB/06PekRg/dlXmOhS9N3apePJfOxjomDwoEtWdYJlAzv+tcOXDaAQ/Zti6+3NGUbPNt5MwHHLDnygdxUyg6qR3LU256uoRC07S7YNdLLu0915nXjoP9M5Sk2cJ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(39860400002)(396003)(346002)(136003)(366004)(316002)(66946007)(86362001)(478600001)(6916009)(186003)(16576012)(2616005)(36756003)(31696002)(66556008)(2906002)(66476007)(6486002)(558084003)(31686004)(4326008)(54906003)(5660300002)(26005)(38100700002)(8676002)(8936002)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RXpGTXZuYTA0ditFRTByd2Uyb0RrQ0dVOXl5UzU3aEI4SXdSd2RQQm1IT0hI?=
 =?utf-8?B?bXQveU9UdkFML2RkbURVcG5GOHkxTy94NlRqS0VTZmVRKzFIeDI1SFM3bC84?=
 =?utf-8?B?TGFOcmRiRmdoeDkvbFFUbVRyVGdoT1FQSzVUaGZiMDFmT091SEkrSkNMYndO?=
 =?utf-8?B?NFBoOGQ4b21aYjd6dXQxSlFlMTA3eTFyVzNHa1RiTHp1dFRDNm9BVVluSE1D?=
 =?utf-8?B?OG1rMDBOSHFIUytxMVEyeVVqd09DbENoQlFtYk1XaHpqZHdxdUozbDVWYmYx?=
 =?utf-8?B?MjlXc3VXdGFSN21PVVE0VG91Z2ZpNGZvbGhHTVYyV0hiSURjNlBzZ3hIaktn?=
 =?utf-8?B?bXBxdEZSbGpOa1c4bUgrTXowdXl6WERHM0JDVUtyZkt1MVFjNFpEZUFJT1Vx?=
 =?utf-8?B?SHpaWGdvVWpsajl6Z0FJMzZrQU0rZjNwM0tCZ0l4ZUovbFZnazl3SGF3aEdx?=
 =?utf-8?B?bnMyRXEvOExiOFF5eE9DMi83Wlp6Smw2UlIrYzZzRUNDTzByWHgwWXQ2T0I1?=
 =?utf-8?B?cU9ydnVIVUljY2kzYmg3N0QwZ2t2QVArTlBpTVBvZWxvcUFWZUd3WEtkZThP?=
 =?utf-8?B?c3hnRHd5SVdCVHpESWhmdDNzLytmajNDSjZUTHJTb3ZjdnQxeURDdm15QlQ1?=
 =?utf-8?B?RjczU0lCWEdLbXB1eVJDUWhQOU10djBPRmhMeW9oY3Jxa09CQ1BOenhzYllT?=
 =?utf-8?B?V1NVYWF0UkZKQTJ6OGZoRWhUQkc3UHl1bThydGxkVTBrQ2ZMTkdjNWhMQWdL?=
 =?utf-8?B?YWxWdHE5Rm1kQWdiUnRqMzZDeWM3OWw3Nm5DQmxTWmRDTEoxNHRGR01VOTQy?=
 =?utf-8?B?emdTNVIxV0MwSzhqcE01OVpXV3hXeWV2YTloR2dmTnkwcDBuNXRadXZ1N2F2?=
 =?utf-8?B?SUNUejl0VFJYTStSOVo3ZVc4NE5oVXdFWHRrU3d3QThjMTlPNWhDcjNQRGFC?=
 =?utf-8?B?NlJhU010eGJVVFBZWnFiQUZ0aFl1R0dTUWVZTjd6dHJTY05saDNWanR6c05u?=
 =?utf-8?B?YlIyWmZ0Mm1qNUJ0QWtKNkNpVktmcFlNZUd6N0JJQXhnd3BBZXp6ZnBDdWJa?=
 =?utf-8?B?QjY2SHpSUno3ZUNEYU1pWHF4Q2t3OFIxWXArTzdUbUtsOFFCUFFjVjFNTyt5?=
 =?utf-8?B?dVlrdDEzZmkrOXVMaXRxb2tPM1ZvYXpZRWw4UVp4VTJnbnZHSzIvWkVtdlpE?=
 =?utf-8?B?WE94WThaZjI2U0xaZ01lNVhXVzlBa2N2RUJ3dHQvTVVtWDd4R0lIVDZ4ang4?=
 =?utf-8?B?enZNYzk4QXkyUWErbDlkOENKdE5mZ0FLU3ZnbkROek5QK3VvcGVwekJZYjRw?=
 =?utf-8?B?RjR4blNtTVpmWmk3Mm13aUZrdUZBUU1VcERId1ZpbkhobzcwRkVBdWJweHdn?=
 =?utf-8?B?d3hpU0tFL3ZwbU9CdHBlOHFncU8xV2VvQU1sTDM5aERvZDZZcUFkQWFXVWNM?=
 =?utf-8?B?QUFyTzBSQjZDdWo3M3VSSjhnN29kOWJDWWROL3lWUU9IRHdNRVZLMmw2QVVR?=
 =?utf-8?B?OHBaWE80dFlEekRkVFQrWjh3Z3REK21HZThIOUR0L2twNFZrYXpEanBjOUdi?=
 =?utf-8?B?VGZhdllPeEk1ajNsSGhENTJySkdGbzBwZ1p1bnJvNWExUE9WcGdoQlFrMy9o?=
 =?utf-8?B?clZGc054c0ZMVi9WZEozWVZrOHp0elk1eXN6dnpoazVobGFMQ3hHdGdDeWts?=
 =?utf-8?B?T0RGVXpxT29Wbk5OV01QZ2RKS2Uva0hSVStRK1l3c3NiRnVRZXdYSE9MQy8y?=
 =?utf-8?Q?5W0kPXRFoG5J/k+C1/t/T77jBmte6398gsE9x6Y?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b0d788c5-aa2a-4402-fdee-08d9784bfc6c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:23:20.3127
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VGv5IBC/6P/D21PYiV93u5w4WQic+VtoJzy8O0u0xaRWHlmP1gTK9/NGBc0fnS5ql1t6204e8KnP8W5poKjyeA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5742

1: make "vga=current" work with graphics modes
2: obtain video info from boot loader
3: EFI: retrieve EDID
4: simplify mode_table
5: fold branches in video handling code
6: fold/replace moves in video handling code
7: fold two MOVs into an ADD

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:24:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:24:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187598.336523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUtk-00068P-4K; Wed, 15 Sep 2021 13:24:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187598.336523; Wed, 15 Sep 2021 13:24:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUtk-00068G-1B; Wed, 15 Sep 2021 13:24:24 +0000
Received: by outflank-mailman (input) for mailman id 187598;
 Wed, 15 Sep 2021 13:24:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUti-000684-Fg
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:24:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7a26eb15-bc4d-4ce0-9cc3-917bcdf9bc33;
 Wed, 15 Sep 2021 13:24:21 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-_7mfDszgOBmVqh1jW8B5GA-1; Wed, 15 Sep 2021 15:24:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 13:24:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:24:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0208.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 13:24:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a26eb15-bc4d-4ce0-9cc3-917bcdf9bc33
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712260;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vmvh23AF/QYwECie+A4cxpmvY3OAqvVGprF/crpbyog=;
	b=DFrZWHNZtWLHQmtRtbEfKw+tHB3oXI0npL0gKD1OFwttXWyD6JyY7gNG6IZYlhKcppFj3c
	OvO9Eyh4Zpr8Ud1LYk71p2O3dBMYUoh0Bo4sSa3ejxwav6TMx6drNv7qR7bGI1r4c7f9jd
	DzOu1wVs5EgJEjPsiMM6oOiWt3A+Dqk=
X-MC-Unique: _7mfDszgOBmVqh1jW8B5GA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ih1x9nB/MQE09L7Z/FeVoxuOL969/1TEiGBcgGI2OWNqRZ9c4dGf/3UiIcLbms+tddWrio5VnVwFy65g58h481xcvfBgQcBHdConT6PbSH9rN8hrtp/sbLCTthw1lZdk36IDto8Bqfw71JAg+lnKFzff1fRgCQF+XZ2Qp+c5JzMa1ZGwh8VhFN26m4TCfbUlg5O/ic0ZakuGkeQ+2ChZLde0MofFtJEo+/CCybtVIjCg9wzC+E8RoMhKBPUws9g8204aOVdQEXb1g/LYBASEUe9gnVk/yI06YuglsuO6Y7fG5mR9kS+C8B2ZOdUc6uZGrE1OHUzK3aWpGI8VcJYdug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vmvh23AF/QYwECie+A4cxpmvY3OAqvVGprF/crpbyog=;
 b=fAD8aYqNM8+WWnZWDcW9ZCBnDgwjUcCpvwP9m2wCkxswEABZJWgLHaoh/UjCuRzbyaYg0/rDBFlAXIVK/Fak1+t1Npdkb62LsGbsfg9Gyb7TZoYQnc53/OUid8Qa9Ea2q3+GAHFKDyoKJKYLjGH7TGT6dqXK25DEAwmtZteWYZKkD5bcJlCF8XL5fxJwBd6IGTmJEuZOFnCtUfe6jnPjNMTkCZQZkBPlZam5mn/1fkN/W6LyxOJ8e4emdvT5R1/JKNteA9mcgrqo70gL0Z45amJg1pM8kllhe2oNyWPfDvRaL0lDChJFFkt3oJMOGjmelEhlemvoR/raJkt3w8c/jA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 1/7] x86/boot: make "vga=current" work with graphics modes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <126bf1d2-64dc-ba0d-8133-27f29c942b0b@suse.com>
Date: Wed, 15 Sep 2021 15:24:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0208.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6496a335-2fe3-4390-ebd3-08d9784c1f06
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435273B9F400A83BEDC02BCEB3DB9@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	L7U4ZEBb903AOkmC2oHkfqAmh2jS2FaoxJPsRsnnxN4tKvPol1vmUm7LlZ+JGP9nA5SfBROBakKxU1dWxWnLrllx6t+gyjw36Cr2n2QGP8D2l9TT/o+8LjMV73Tqtd7cNbzxpsVhG/2mrjUQmnKlOWsbcqTjJ2EH5fp9py6qUMEEL75oaHk9GuSMYr3IEJlfL/13OXJR12RCLvraHS3xVbU4gf3I6sCB/zuhnVJZk8ZlPVj5DCZ9iUD0ZlnSEeAFo1TcPOYQI2KBt2dXzCZybYg+OIE3zARqKz7pVu7klefOyCqwNQ6xPnhdwHpzh38bSIVPIefWnVYB8ARFvXS0JCeTox6y9p/Ujgq2nNP6Y48VCDTaMnWiNzMmp5I5/FEB0EX7Ucm1Vxf+N/B4kRaWwxT3ctehVbas6lhdTZVQXbKyJw7kGLbntXxrrx2AUMhgFKZydviCIy+aJ+htvbXzb1t7tNipp9yrEMZveLdL9r2N+plTDK6bOCtJSfEbSWbz9fKTW+svHavNKb2JuS7iWMQXbniZnpQ9bOwKRsnu0OsZzPcxSXOlMYNHjItP1qEa35fLb1qph40CB8vvp1W4OMaaYxTfqiw3UoOfES/8XeXZd4hCd+kDpooP2Of8lUouU/8EnjdopUK5ehNDQAnf5qdD7+MW0Zqw3j6VFKZrHucRX46L1yEvrRH1nBMIVc3rhq1tERhyT/8HY68lBPhKD+MY1Hkexnt9bgso1CqS3c8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(346002)(366004)(396003)(376002)(31696002)(316002)(4326008)(16576012)(26005)(6486002)(38100700002)(8936002)(36756003)(956004)(66946007)(54906003)(66476007)(66556008)(5660300002)(2616005)(6916009)(8676002)(2906002)(186003)(86362001)(478600001)(83380400001)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eUpYSlBYSXlMWGRXRUJGUjFJSjdZeEJJMERmNVhHL3BVUTVjWm02VGdEN21n?=
 =?utf-8?B?WFBqK201TXdLamtrOURyNVBaZVBwdXdTcXVpOWZXTzlPS3hEVCtSSWhJdE1h?=
 =?utf-8?B?Q0ErOTdtQ3FyTW0xK0J2RUc1dHJGRk5oYmI3ZDJlcHdNRFFUVmdnSG9zNktS?=
 =?utf-8?B?TDFrb0tkUTM0RmpWelZsQ3d3NDRIV05PaHhtOXMzeHpUdlVBWS9FSndyZTBS?=
 =?utf-8?B?aWFKRmlZNlRKYWFjeHdkV1NkZHNKdVZ3c0QzTHhYbWpFUWc2MFN1ZGZtcDJ2?=
 =?utf-8?B?SGdlWkEwQ1Yveks4TTc1a1Z3WTdDQUVDQ1JYSHRkMkFCdm5nMzJiVnBROWNy?=
 =?utf-8?B?TWFuMWN1UFpvc2lRMzN1c05Nb1Q3ZVowclZtdWNCRTU0ajAvVHhxb21UdDJi?=
 =?utf-8?B?WndxOS9hUUVldW9zczhDY0xwVTZ6TmM3dmE3U1ppUy9aNXZzK0RPSVZWTUxE?=
 =?utf-8?B?U1hwKzIvbEFSVFJOY3NOL0M5Q29DYk9lWElxSWxHNlkrbmg1MlFSQmVpbjl5?=
 =?utf-8?B?S2JESkJlVU0rdFBoT2FoU29UMnYxNDBtOUJpdlRhTERrQUVibEtRUm94bG5I?=
 =?utf-8?B?QnFhblBFUTAxcE92dXFjaUFJWWJpOStJcGVzUDg3eGh6d3oxeENqdHl0MXVi?=
 =?utf-8?B?cVBIbWpBdDF3dndzSzJXNUxmRFA5VXVnVG5iYzJkb3Q5TVZFcStsYmJYN2o3?=
 =?utf-8?B?dXVpM2ErSmdlajNUNTV3NWVYSkEyRm1CZlJRZndhZHpnRFFKTTNJaHd1c1Q1?=
 =?utf-8?B?SVk0ZVlWVlFMVTNYZk9Od2ZQWWdoK2JwN0xaaCtMZW5zd2FMaktkVDZpcGtC?=
 =?utf-8?B?UEM5QjFodjk5VnNVRUxSMXZhVXFTSTVGQVNBZnhCT0h6R2ZUVVdjVllUOEYz?=
 =?utf-8?B?R09iZlhwWTBLZWZYUkZCalhiN3RFNDdZQ09SajQ2clduL0pKcmF5bWd4dkhG?=
 =?utf-8?B?ZXptL2NHZmdocktNR09oZUJ2VUZCMys1elRPcWphZWJwT01QRlZlQTVrK2JC?=
 =?utf-8?B?VHlFSUNQUXpWM2U3a05IREFMMmtVVllKZ0VveGY1Ylk2LzFWTDdVSG41dmdQ?=
 =?utf-8?B?d2ZZSTJwN1dSOWhwMERuS3hDcC9KTFFOT1ZOQTFaWndZVjVURXZzQVZXOGxv?=
 =?utf-8?B?MjU0c2tEYnJjM2tsTlpoMk9IRmEza2h1OURIVmdkNm9uejdHemV0WTJlenNs?=
 =?utf-8?B?cFdpbUZUdURHcmZsUit5SE5TSFFBZXRIQytkRDNYeWVnMnd5WDJGd21SS04w?=
 =?utf-8?B?Q0RKSGlnTStGYjJLZUN6bHJkT0k0N2RTaU1JTDQ3Z3k1SHUvbHQrRW1ScmNn?=
 =?utf-8?B?K2ZZRVZJZlRVNjN6elo4MThjaHdIcTlnYkhBallHdnZ1Y1kvSzU4aGZ0VzZX?=
 =?utf-8?B?RC96NXpxVDZUQkFCZlVvb2dFQmZIYUYrcS8zZS9ha2hXNHdyRWEzZFdHM0U0?=
 =?utf-8?B?dm4xM0J5Y1FuSk1YTkVVZFdEVnlLcjlSdW9VSWVTRWxWWXNMaURlZnhZNjNU?=
 =?utf-8?B?TU9WN2d3cFd1Vmx6MmpKWVVhTWZXODZubWZiQzlrMTNDWllOSTlkV3l5R0dp?=
 =?utf-8?B?T1dEa3J1Nk5OOGhSSk16U0s2Y3lDT2ZEYzROd1RjNXEyN2dXVkFrTHJUdWVL?=
 =?utf-8?B?MmZXZ0FoOG4wQXlTTjlPVGE0cDg2Q29ibEh0azZLTHBMVmJYMElTLzlWb0kv?=
 =?utf-8?B?dmgzTExadXptd0ZCY1kwamVGUlFiOTB4bnFFNFBDNkpCclZvV2R6bGlXZFBa?=
 =?utf-8?Q?YYrzSaE8SYoQzAprelGnOSfJ/4RuzXQy3mVzHYc?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6496a335-2fe3-4390-ebd3-08d9784c1f06
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:24:18.3870
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YigxjAvzhojjwbvS7sTcgbNWWxCK6ftLBN7YUr7N1CEc2ofdEufzH6ka6A0Z8mnO2EvxKjnSw2QzIsf330ouiA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

GrUB2 can be told to leave the screen in the graphics mode it has been
using (or any other one), via "set gfxpayload=keep" (or suitable
variants thereof). In this case we can avoid doing another mode switch
ourselves. This in particular avoids possibly setting the screen to a
less desirable mode: On one of my test systems the set of modes
reported available by the VESA BIOS depends on whether the interposed
KVM switch has that machine set as the active one. If it's not active,
only modes up to 1024x768 get reported, while when active 1280x1024
modes are also included. For things to always work with an explicitly
specified mode (via the "vga=" option), that mode therefore needs be a
1024x768 one.

For some reason this only works for me with "multiboot2" (and
"module2"); "multiboot" (and "module") still forces the screen into text
mode, despite my reading of the sources suggesting otherwise.

For starters I'm limiting this to graphics modes; I do think this ought
to also work for text modes, but
- I can't tell whether GrUB2 can set any text mode other than 80x25
  (I've only found plain "text" to be valid as a "gfxpayload" setting),
- I'm uncertain whether supporting that is worth it, since I'm uncertain
  how many people would be running their systems/screens in text mode,
- I'd like to limit the amount of code added to the realmode trampoline.

For starters I'm also limiting mode information retrieval to raw BIOS
accesses. This will allow things to work (in principle) also with other
boot environments where a graphics mode can be left in place. The
downside is that this then still is dependent upon switching back to
real mode, so retrieving the needed information from multiboot info is
likely going to be desirable down the road.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm not convinced boot_vid_mode really needs setting here; I'm doing so
mainly because setvesabysize also does.
---
v2: Use 0x9b instead of 0x99 for attributes check: I think the value
    used by check_vesa also wants to be converted, to match vesa2's.
    (Perhaps the value wants to become a #define, albeit before doing so
    I'd question the requirement of the mode being a color one.)

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -575,7 +575,6 @@ set14:  movw    $0x1111, %ax
         movb    $0x01, %ah              # Define cursor scan lines 11-12
         movw    $0x0b0c, %cx
         int     $0x10
-set_current:
         stc
         ret
 
@@ -693,6 +692,39 @@ vga_modes:
         .word   VIDEO_80x60, 0x50,0x3c,0        # 80x60
 vga_modes_end:
 
+# If the current mode is a VESA graphics one, obtain its parameters.
+set_current:
+        leaw    vesa_glob_info, %di
+        movw    $0x4f00, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        movw    $0x4f03, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        leaw    vesa_mode_info, %di     # Get mode information structure
+        movw    %bx, %cx
+        movw    $0x4f01, %ax
+        int     $0x10
+        cmpw    $0x004f, %ax
+        jne     .Lsetc_done
+
+        movb    (%di), %al              # Check mode attributes
+        andb    $0x9b, %al
+        cmpb    $0x9b, %al
+        jne     .Lsetc_done             # Doh! No linear frame buffer
+
+        movb    $1, bootsym(graphic_mode)
+        movw    %bx, bootsym(boot_vid_mode)
+        movw    %bx, bootsym(video_mode)
+
+.Lsetc_done:
+        stc
+        ret
+
 # Detect VESA modes.
 vesa_modes:
         movw    %di, %bp                # BP=original mode table end



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:25:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:25:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187605.336533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUuP-0006iu-Dv; Wed, 15 Sep 2021 13:25:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187605.336533; Wed, 15 Sep 2021 13:25:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUuP-0006in-Am; Wed, 15 Sep 2021 13:25:05 +0000
Received: by outflank-mailman (input) for mailman id 187605;
 Wed, 15 Sep 2021 13:25:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUuN-0006iX-Gp
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:25:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 546d14db-1628-11ec-b51b-12813bfff9fa;
 Wed, 15 Sep 2021 13:25:02 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-2zwv82RQO8y2_KMNFs9QPw-1; Wed, 15 Sep 2021 15:25:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 13:24:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:24:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0063.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Wed, 15 Sep 2021 13:24:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 546d14db-1628-11ec-b51b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712301;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lVFjmh6RimoQpMTq+9aSlOOru1h888tl/hV8Nfxcv+U=;
	b=UoI5eWcTEuPelH+UjG/6jBbt0/fjMQV7be6GXemrRrjeoFMMMiDmfxm4zS/q+Yo7KcdvJG
	O/7Q00yFsEPYnCAvgWCHFX7SUGQlhpVM26QTMoVhirx5N9T1rYIKlB0KtD/ZXOKrP/IP9H
	I99y5Nrx/IK+bL94b+Aqs8sDrWzQTQE=
X-MC-Unique: 2zwv82RQO8y2_KMNFs9QPw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=c10jtRtCSc/aamTWBhR/uKwKGjDpCtEy7tA/aT8/f0t4I7XIBZ4ccs/DnJwxcSCCu/K232fsVOXCPNizv4V3pfBfOsLgfKxrNQQkL5zv4y2K9XM4FLt15rVUgZJX56e4xO8GsIz11ZCz8ccOfqoXuC94bm4jmd5oqfl/Qq9T8lUc+dRh+Z5X927S3/bNawVXWwGNsbSRUneTvpnb9xkA4n4RJnrAWUlUVpbo6Cc4kkLzKjEm92T9jXM7MOZNBueXlxQOd74uhCmreQrUVdEJ/RLcaz2eTHsRqV8vSjcjJlLnwAfEhVXEXhCYGErVbLuQ4bZkZ2DyggX0u5wLR9ttjA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lVFjmh6RimoQpMTq+9aSlOOru1h888tl/hV8Nfxcv+U=;
 b=KPY/lMCh8V7A0Kd972Y1gL4crxtgK6JJ8bqTj4o30pRNCjI18/tH9KHpSs/RDa6+6UCX8A2y4Ou6AA07yYp/LzsiHjOfa3ig94Q0lgFDwsnQMzrfK/YLD6reW0BEcM9c+XaQimhL+7jMogxJXwkjvnqBmrn2wfA1kWZDOw8UUHqAZu3WPCV+d+ZJ3O5qV+NZiODtnTvQOh76ZKrvE2A1tkyir8K9LrkckCGzQNvE4kTAyGFMsXxbZw9NWsRMfDjyHfuKkcx3K7272vyilndVdsjWfEULHSaaqt3zg9QEa8TBcSdQOg+KRK5cvv5xzXWalMwnwW3sGPOuJTTcpYtezA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 2/7] x86/boot: obtain video info from boot loader
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <bc7fb3a7-ee1f-8859-f1e0-a6cb04fa9d33@suse.com>
Date: Wed, 15 Sep 2021 15:24:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0063.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1eaf84d3-140d-415f-ff02-08d9784c3704
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435210F95093A46215530010B3DB9@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VAZ3eqvwgtXtQl2XGV/x/51BXeOhmq7cY2CYfvlwNPfbfNRxVEupdQfDTQ+BJLLaEBVgGxiQQHCihV/0GiN51Dl6vaqCyz0R3MB6xjjZJtjrb5v53aM8j4esn3recxyH/4Y1CUhgC7XJgRuSdT2P6lBxoTNrgTfqp00pSpMQmvnvXZ4EpcEMHAyXZjulVHyJHnu0wNXl8UuRCu3O7C4oGbPIbQu8+StRykYhHX4Fk8dvD9921bQxcSVznSy8xaCuTLNKKMMhPXEdWXrKCiaTEXCR4dGWuQHXsEsXlaP8mWZZixTCS9H6BvI4UMynWhE6mAMZ+BFolDexRoGrH185bZ6xN21a7jL8IIl6bIK2vV6TLXKWKPHoyDhsXVnNsdEmbaw025s5lQ2K4UWXu9293fTUmF6uuwAJ4cD2Nfjiraj8edbiOPhtq9Dx+A1jz1bt76EKytRArFxXaWSwKfJ9VYkPzopziJUtKzhJhy6tmC0Nnb5zmIRSEcfk3UNBRSQhWl+haCYwjP6gYGL8E/SYdDeTZOx/CqIp44MMdFGhtl2FJTBhxzFfrZJzDTTm+6wMBgvLUk792SLJAZvqXEAbTcr6fxz5e+exz6X310py6oqFK6MzQ2h/MEN9Oc1jMpNJkcMo4zWSV3v3492o6pmrHDtn17il6m9c7Bwz+Wa7PT3G/KCU/v2h4bicIKiuLQCw+kfgtN8G/YC8lYeWbG+qcUZVSUP5VDPCD+u7lZTRTzA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(346002)(366004)(396003)(376002)(31696002)(316002)(4326008)(16576012)(26005)(6486002)(38100700002)(8936002)(36756003)(956004)(66946007)(54906003)(66476007)(66556008)(5660300002)(2616005)(6916009)(8676002)(2906002)(186003)(86362001)(478600001)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WWlTQzU2cVZQM1VGZHdtZThtdk14OG9JM0luQWtnVnRLTWQ0MVdjUjU1Wkp0?=
 =?utf-8?B?RVlCb1FZdlQvdGhVaUlqNmsrdmZ3L0JGU2h3RDN1UlE0aFFDSEJlZi9ucEVY?=
 =?utf-8?B?QlVrSWpOYjBhcklPSWVqZFZ5M2E1akxwUHFiZVUyV280WVpUTUY2d3pBMyto?=
 =?utf-8?B?RjFCdHdaQ2FXRHY5TG5CM2NYVmRkTWU1SUoyczlkRjM1MHlpZXhlMzZzdHBv?=
 =?utf-8?B?YTFvNEN0NHAyVUVSOWFqL1A1Sk5QR25VRFBMWFo0MWQzMlFlK3RyVGFNRkVy?=
 =?utf-8?B?azJzbkFSb1ZWZmJFV0FTRGwzdXpyZ1JpMVd6aHY2WGZFM09RcytURVI5Y3I5?=
 =?utf-8?B?dERQZWYrNUhLNXE0MWRrZlFvL1l4Sm1GVENlc3FrT1NpMS8wNnJ0NXdsNEJp?=
 =?utf-8?B?Q0pONjRsQklWbDhrelhKeTVsRENkSzJsKzVsMGE4QlMrNUhhc1dOYnQxVzlF?=
 =?utf-8?B?UGRCOEM2bXhGdEJxRmdGMFlvU1B4N0lqVkF0clNUQ2d6T2ZQYWpUcVh0NjEz?=
 =?utf-8?B?WUJtUDM0K0FmU2Z0RS9tR3U5N3hwc3NRYW80aHhXdWhGYm5xYmRveGRjb0FZ?=
 =?utf-8?B?STNPcGtmdmZ6L3J5ck1aRG9qc2J1ZjZIK2FpTWJwaHdoSGZFOWV2dWpoYkl2?=
 =?utf-8?B?c284UnkxSmN4K0ZNVm1aTnpLbnc1eS8xVnpwUnFlOUY2L2E0UVo2eStlaS9G?=
 =?utf-8?B?WHppZStsc29WTktzeTJjWWVPVUxuTWErcnFnTnBmcUZOWSs1RkFsR0gzMWth?=
 =?utf-8?B?WUk1NHY0Z0dRRWt1eGUyMmpnbUhXS2xzdnNEMUJ0TjR0dTcxa3JCYjFNWWxN?=
 =?utf-8?B?WFhHRXN6eWZubVpLNmdtWFVFL21sMndERDZ3MkdzVWJTUzdsZ2VzTEJUZUFy?=
 =?utf-8?B?UGV6NVVBVjkwV3UzSWdnYVkybGR0YzVuUUJPejBzWTdaMUsrQ0ZadEZOL2Fq?=
 =?utf-8?B?YTNXcGRPQ1lOK2N5Mk1wQXhiclA2eHJTYjFDcEpNOVcwWFFrbmdnMjV2TzEx?=
 =?utf-8?B?WklkTGFvVXBQL3kzTWQvQXVncWtGbThHYnVFZnN3MGxNdEtBeWxISGFGSnkr?=
 =?utf-8?B?eWVqVXVmWWRRSndDWkdCWjZoQldpMEhoWGNHbjRxZ0g1dXBwT1FISXNPRStu?=
 =?utf-8?B?MlpuN0IwOGREWVdmSUo2TGR2RitXSlFuczZPOGxxWERQTTRFYlM2S2FlaWJn?=
 =?utf-8?B?N2lRMUVMRVZNM3ZXVDJrRDc2UkhDSHJDdllVTnpFZi9kNXA1T0cyTVpTTUh2?=
 =?utf-8?B?ekdvVXJEeHYxeEtyc1liaHUzV1pDUlc5OTluYW5HOXNnei9wWFdIblgvcWMw?=
 =?utf-8?B?WER0aTl2M2xXQ3BCODZaaW9SYUZiUkFqQWFlYTM5b1U4U1VQYWMxZ0VtRU4r?=
 =?utf-8?B?M2tLbUJpLzhPZ05ybCsxV2Jta1c4bGRqS3NvQWl4NGQxVW1KWExyZWpaNGoy?=
 =?utf-8?B?czN3NVlVY0ppbUFPQy9Zei9oTWE1WmhzZEp1d2hzYUtkNVgzVWt2aHJRNXVo?=
 =?utf-8?B?NVZiRzJiQ292TkgzZ3g2SFVzUkhhOG5mSjFGSXc4QXNnU2x5Uk1pdlJBQmU2?=
 =?utf-8?B?Zm1LUi9Yc2FkOERETFhlb3ljUmhEQ3ZoOUtQaGovL1RtdzN2YlE1ZC95OFI4?=
 =?utf-8?B?YWhzTmFmWUQ2L255bU9qbk5ZYVRYTUNNOXd4K011cFpzdzRkWGR0emZaRTNB?=
 =?utf-8?B?b083aUFxQUpDaVRkS2F5ekJONEVZcTU5ZUh5L1dTMDN0Uk1pMG05L3d3djU4?=
 =?utf-8?Q?5hgq6YLTvnzREtwqjE5NjgVbXp4+MOclBvwwQpD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1eaf84d3-140d-415f-ff02-08d9784c3704
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:24:58.6197
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EEXiPZt9/9HCnCrR+vSUhr+qj6BhlyWEJowYkVfON7/N+Y5gt+P657Daf40ZvgSljybZzb/vAzZJGLXredBXWw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

With MB2 the boot loader may provide this information, allowing us to
obtain it without needing to enter real mode (assuming we don't need to
set a new mode from "vga=", but can instead inherit the one the
bootloader may have established).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -4,7 +4,7 @@ DEFS_H_DEPS = defs.h $(BASEDIR)/include/
 
 CMDLINE_DEPS = $(DEFS_H_DEPS) video.h
 
-RELOC_DEPS = $(DEFS_H_DEPS) \
+RELOC_DEPS = $(DEFS_H_DEPS) video.h \
 	     $(BASEDIR)/include/generated/autoconf.h \
 	     $(BASEDIR)/include/xen/kconfig.h \
 	     $(BASEDIR)/include/xen/multiboot.h \
--- a/xen/arch/x86/boot/defs.h
+++ b/xen/arch/x86/boot/defs.h
@@ -53,6 +53,7 @@ typedef unsigned int u32;
 typedef unsigned long long u64;
 typedef unsigned int size_t;
 typedef u8 uint8_t;
+typedef u16 uint16_t;
 typedef u32 uint32_t;
 typedef u64 uint64_t;
 
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -562,12 +562,18 @@ trampoline_setup:
         mov     %esi, sym_esi(xen_phys_start)
         mov     %esi, sym_esi(trampoline_xen_phys_start)
 
-        mov     sym_esi(trampoline_phys), %ecx
-
         /* Get bottom-most low-memory stack address. */
+        mov     sym_esi(trampoline_phys), %ecx
         add     $TRAMPOLINE_SPACE,%ecx
 
+#ifdef CONFIG_VIDEO
+        lea     sym_esi(boot_vid_info), %edx
+#else
+        xor     %edx, %edx
+#endif
+
         /* Save Multiboot / PVH info struct (after relocation) for later use. */
+        push    %edx                /* Boot video info to be filled from MB2. */
         push    %ecx                /* Bottom-most low-memory stack address. */
         push    %ebx                /* Multiboot / PVH information address. */
         push    %eax                /* Magic number. */
--- a/xen/arch/x86/boot/reloc.c
+++ b/xen/arch/x86/boot/reloc.c
@@ -14,9 +14,10 @@
 
 /*
  * This entry point is entered from xen/arch/x86/boot/head.S with:
- *   - 0x4(%esp) = MAGIC,
- *   - 0x8(%esp) = INFORMATION_ADDRESS,
- *   - 0xc(%esp) = TOPMOST_LOW_MEMORY_STACK_ADDRESS.
+ *   - 0x04(%esp) = MAGIC,
+ *   - 0x08(%esp) = INFORMATION_ADDRESS,
+ *   - 0x0c(%esp) = TOPMOST_LOW_MEMORY_STACK_ADDRESS.
+ *   - 0x10(%esp) = BOOT_VIDEO_INFO_ADDRESS.
  */
 asm (
     "    .text                         \n"
@@ -32,6 +33,39 @@ asm (
 #include "../../../include/xen/kconfig.h"
 #include <public/arch-x86/hvm/start_info.h>
 
+#ifdef CONFIG_VIDEO
+# include "video.h"
+
+/* VESA control information */
+struct __packed vesa_ctrl_info {
+    uint8_t signature[4];
+    uint16_t version;
+    uint32_t oem_name;
+    uint32_t capabilities;
+    uint32_t mode_list;
+    uint16_t mem_size;
+    /* We don't use any further fields. */
+};
+
+/* VESA 2.0 mode information */
+struct vesa_mode_info {
+    uint16_t attrib;
+    uint8_t window[14]; /* We don't use the individual fields. */
+    uint16_t bytes_per_line;
+    uint16_t width;
+    uint16_t height;
+    uint8_t cell_width;
+    uint8_t cell_height;
+    uint8_t nr_planes;
+    uint8_t depth;
+    uint8_t memory[5]; /* We don't use the individual fields. */
+    struct boot_video_colors colors;
+    uint8_t direct_color;
+    uint32_t base;
+    /* We don't use any further fields. */
+};
+#endif /* CONFIG_VIDEO */
+
 #define get_mb2_data(tag, type, member)   (((multiboot2_tag_##type##_t *)(tag))->member)
 #define get_mb2_string(tag, type, member) ((u32)get_mb2_data(tag, type, member))
 
@@ -146,7 +180,7 @@ static multiboot_info_t *mbi_reloc(u32 m
     return mbi_out;
 }
 
-static multiboot_info_t *mbi2_reloc(u32 mbi_in)
+static multiboot_info_t *mbi2_reloc(uint32_t mbi_in, uint32_t video_out)
 {
     const multiboot2_fixed_t *mbi_fix = _p(mbi_in);
     const multiboot2_memory_map_t *mmap_src;
@@ -154,6 +188,9 @@ static multiboot_info_t *mbi2_reloc(u32
     module_t *mbi_out_mods = NULL;
     memory_map_t *mmap_dst;
     multiboot_info_t *mbi_out;
+#ifdef CONFIG_VIDEO
+    struct boot_video_info *video = NULL;
+#endif
     u32 ptr;
     unsigned int i, mod_idx = 0;
 
@@ -254,17 +291,64 @@ static multiboot_info_t *mbi2_reloc(u32
             ++mod_idx;
             break;
 
+#ifdef CONFIG_VIDEO
+        case MULTIBOOT2_TAG_TYPE_VBE:
+            if ( video_out )
+            {
+                const struct vesa_ctrl_info *ci;
+                const struct vesa_mode_info *mi;
+
+                video = _p(video_out);
+                ci = (void *)get_mb2_data(tag, vbe, vbe_control_info);
+                mi = (void *)get_mb2_data(tag, vbe, vbe_mode_info);
+
+                if ( ci->version >= 0x0200 && (mi->attrib & 0x9b) == 0x9b )
+                {
+                    video->capabilities = ci->capabilities;
+                    video->lfb_linelength = mi->bytes_per_line;
+                    video->lfb_width = mi->width;
+                    video->lfb_height = mi->height;
+                    video->lfb_depth = mi->depth;
+                    video->lfb_base = mi->base;
+                    video->lfb_size = ci->mem_size;
+                    video->colors = mi->colors;
+                    video->vesa_attrib = mi->attrib;
+                }
+
+                video->vesapm.seg = get_mb2_data(tag, vbe, vbe_interface_seg);
+                video->vesapm.off = get_mb2_data(tag, vbe, vbe_interface_off);
+            }
+            break;
+
+        case MULTIBOOT2_TAG_TYPE_FRAMEBUFFER:
+            if ( (get_mb2_data(tag, framebuffer, framebuffer_type) !=
+                  MULTIBOOT2_FRAMEBUFFER_TYPE_RGB) )
+            {
+                video_out = 0;
+                video = NULL;
+            }
+            break;
+#endif /* CONFIG_VIDEO */
+
         case MULTIBOOT2_TAG_TYPE_END:
-            return mbi_out;
+            goto end; /* Cannot "break;" here. */
 
         default:
             break;
         }
 
+ end:
+
+#ifdef CONFIG_VIDEO
+    if ( video )
+        video->orig_video_isVGA = 0x23;
+#endif
+
     return mbi_out;
 }
 
-void * __stdcall reloc(u32 magic, u32 in, u32 trampoline)
+void *__stdcall reloc(uint32_t magic, uint32_t in, uint32_t trampoline,
+                      uint32_t video_info)
 {
     alloc = trampoline;
 
@@ -274,7 +358,7 @@ void * __stdcall reloc(u32 magic, u32 in
         return mbi_reloc(in);
 
     case MULTIBOOT2_BOOTLOADER_MAGIC:
-        return mbi2_reloc(in);
+        return mbi2_reloc(in, video_info);
 
     case XEN_HVM_START_MAGIC_VALUE:
         if ( IS_ENABLED(CONFIG_PVH_GUEST) )
--- a/xen/arch/x86/boot/video.h
+++ b/xen/arch/x86/boot/video.h
@@ -28,4 +28,45 @@
 /* The "recalculate timings" flag */
 #define VIDEO_RECALC        0x8000
 
+#ifndef __ASSEMBLY__
+struct boot_video_info {
+    uint8_t  orig_x;             /* 0x00 */
+    uint8_t  orig_y;             /* 0x01 */
+    uint8_t  orig_video_mode;    /* 0x02 */
+    uint8_t  orig_video_cols;    /* 0x03 */
+    uint8_t  orig_video_lines;   /* 0x04 */
+    uint8_t  orig_video_isVGA;   /* 0x05 */
+    uint16_t orig_video_points;  /* 0x06 */
+
+    /* VESA graphic mode -- linear frame buffer */
+    uint32_t capabilities;       /* 0x08 */
+    uint16_t lfb_linelength;     /* 0x0c */
+    uint16_t lfb_width;          /* 0x0e */
+    uint16_t lfb_height;         /* 0x10 */
+    uint16_t lfb_depth;          /* 0x12 */
+    uint32_t lfb_base;           /* 0x14 */
+    uint32_t lfb_size;           /* 0x18 */
+    union {
+        struct {
+            uint8_t  red_size;   /* 0x1c */
+            uint8_t  red_pos;    /* 0x1d */
+            uint8_t  green_size; /* 0x1e */
+            uint8_t  green_pos;  /* 0x1f */
+            uint8_t  blue_size;  /* 0x20 */
+            uint8_t  blue_pos;   /* 0x21 */
+            uint8_t  rsvd_size;  /* 0x22 */
+            uint8_t  rsvd_pos;   /* 0x23 */
+        };
+        struct boot_video_colors {
+            uint8_t  rgbr[8];
+        } colors;
+    };
+    struct {
+        uint16_t seg;            /* 0x24 */
+        uint16_t off;            /* 0x26 */
+    } vesapm;
+    uint16_t vesa_attrib;        /* 0x28 */
+};
+#endif /* __ASSEMBLY__ */
+
 #endif /* __BOOT_VIDEO_H__ */
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -515,35 +515,7 @@ static void __init setup_max_pdx(unsigne
 static struct e820map __initdata boot_e820;
 
 #ifdef CONFIG_VIDEO
-struct boot_video_info {
-    u8  orig_x;             /* 0x00 */
-    u8  orig_y;             /* 0x01 */
-    u8  orig_video_mode;    /* 0x02 */
-    u8  orig_video_cols;    /* 0x03 */
-    u8  orig_video_lines;   /* 0x04 */
-    u8  orig_video_isVGA;   /* 0x05 */
-    u16 orig_video_points;  /* 0x06 */
-
-    /* VESA graphic mode -- linear frame buffer */
-    u32 capabilities;       /* 0x08 */
-    u16 lfb_linelength;     /* 0x0c */
-    u16 lfb_width;          /* 0x0e */
-    u16 lfb_height;         /* 0x10 */
-    u16 lfb_depth;          /* 0x12 */
-    u32 lfb_base;           /* 0x14 */
-    u32 lfb_size;           /* 0x18 */
-    u8  red_size;           /* 0x1c */
-    u8  red_pos;            /* 0x1d */
-    u8  green_size;         /* 0x1e */
-    u8  green_pos;          /* 0x1f */
-    u8  blue_size;          /* 0x20 */
-    u8  blue_pos;           /* 0x21 */
-    u8  rsvd_size;          /* 0x22 */
-    u8  rsvd_pos;           /* 0x23 */
-    u16 vesapm_seg;         /* 0x24 */
-    u16 vesapm_off;         /* 0x26 */
-    u16 vesa_attrib;        /* 0x28 */
-};
+# include "boot/video.h"
 extern struct boot_video_info boot_vid_info;
 #endif
 
--- a/xen/include/xen/multiboot2.h
+++ b/xen/include/xen/multiboot2.h
@@ -158,6 +158,59 @@ typedef struct {
     multiboot2_memory_map_t entries[];
 } multiboot2_tag_mmap_t;
 
+typedef struct
+{
+    uint32_t type;
+    uint32_t size;
+    uint16_t vbe_mode;
+    uint16_t vbe_interface_seg;
+    uint16_t vbe_interface_off;
+    uint16_t vbe_interface_len;
+    uint8_t vbe_control_info[512];
+    uint8_t vbe_mode_info[256];
+} multiboot2_tag_vbe_t;
+
+typedef struct
+{
+    uint8_t red;
+    uint8_t green;
+    uint8_t blue;
+} multiboot2_color_t;
+
+typedef struct
+{
+    uint32_t type;
+    uint32_t size;
+    uint64_t framebuffer_addr;
+    uint32_t framebuffer_pitch;
+    uint32_t framebuffer_width;
+    uint32_t framebuffer_height;
+    uint8_t framebuffer_bpp;
+#define MULTIBOOT2_FRAMEBUFFER_TYPE_INDEXED  0
+#define MULTIBOOT2_FRAMEBUFFER_TYPE_RGB      1
+#define MULTIBOOT2_FRAMEBUFFER_TYPE_EGA_TEXT 2
+    uint8_t framebuffer_type;
+    uint16_t reserved;
+
+    union
+    {
+        struct
+        {
+            uint16_t framebuffer_palette_num_colors;
+            multiboot2_color_t framebuffer_palette[];
+        };
+        struct
+        {
+            uint8_t framebuffer_red_field_position;
+            uint8_t framebuffer_red_mask_size;
+            uint8_t framebuffer_green_field_position;
+            uint8_t framebuffer_green_mask_size;
+            uint8_t framebuffer_blue_field_position;
+            uint8_t framebuffer_blue_mask_size;
+        };
+    };
+} multiboot2_tag_framebuffer_t;
+
 typedef struct {
     u32 type;
     u32 size;



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:25:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:25:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187609.336544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUuh-0007Cj-Pf; Wed, 15 Sep 2021 13:25:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187609.336544; Wed, 15 Sep 2021 13:25:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUuh-0007Cc-Md; Wed, 15 Sep 2021 13:25:23 +0000
Received: by outflank-mailman (input) for mailman id 187609;
 Wed, 15 Sep 2021 13:25:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUug-0007CB-Ma
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:25:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e84f385-7855-48db-bf06-9ed834f07ee0;
 Wed, 15 Sep 2021 13:25:21 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-3-SdYf92MjM0uvDqMip3mpRg-1; Wed, 15 Sep 2021 15:25:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 13:25:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:25:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0029.eurprd09.prod.outlook.com (2603:10a6:102:b7::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 13:25:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e84f385-7855-48db-bf06-9ed834f07ee0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712320;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/VDoYITGSl1EhuEmr7q/46YlB0q7RVPdBu0DpYzl+Hk=;
	b=BRplxwogbz8hyMhlkztT6U6Vtp5q56JxWNRD+F81jGAxRtqF7dzYDEv4DDEjUFDzkJARw9
	dqzu/ghqtH4uJsh4XPJ5nufc+QziurWKN0O244UbvbLKdw1VIJ6lGR2DlbCuKh9hl1ejSg
	kRcpXteednMemq3RqBAeOQid9fA5C1w=
X-MC-Unique: SdYf92MjM0uvDqMip3mpRg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XknlVbhfh2oaVramCtNzdgetx9vQx5mr7+VHnvEFWOg3xAxu77DZOPY0QmQ3AbKgsvns850zjEKDLYBdGCibc8/IYgi1ugarE6osReFers+qyAmR1MkPhBniY+133UEn6200fvCG7F/o6TICM79M4mXOvbWokoJjy3HztRTtE/3j+s+O3f5rDfJ8+MydYSXQASeoBYADz9Y4gNcBgDvqRvb8wfOCPQIcMIRGyvWjmzauYlsvm25vLqmO+fhsKpQv12SZ9RPoXFLo2RfPUOKhGZTSdT6ZuVBuw5B1hbRk117KLP3rCzfUNZqvdhvEVSO8mrk10pw4CC2T+WTASPC0bQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/VDoYITGSl1EhuEmr7q/46YlB0q7RVPdBu0DpYzl+Hk=;
 b=TISPcPUTZLaz8b5ZBEqpSU/i//CcYiJQgvtQ7wadY2eoke2IJQftnznnmOj446+srLm0Lq7zoUqtlnfY24Jjll9L0Y3wAnsOLstB8dDjWUz+933oVfHFtuMVdqci8gW08f3+NFqJw/Weu1r7udpfJ1Gqsy+461Sc3VRWy4QoA2p3F0h99TlCGq1Vcet8T8SHm81iOZJFuGa5QmBspon9t4yCku3AGweGXEq5ODNeY05EdyVC+IOzgz29CyyQ5OQ/yP5lkk18Wysp4yGr88P3lIpqe1PMaRDIOsRHb4FTvkcdke1SAIurOGLMgtej0VPT2fDMKlHN+KEd9zA/Df3Wzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 3/7] x86/EFI: retrieve EDID
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <7c06c5e8-324a-e402-680d-131693a37ce3@suse.com>
Date: Wed, 15 Sep 2021 15:25:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0029.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8aa65c14-25b9-41c6-6a3e-08d9784c42a2
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435242FA3D549E4754CE998AB3DB9@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	T+wOxeSo8FIw3phuKg+zNSc8SQUVUAVUrMjuF1SPl0IIbYB/gFpRuevISorZqnVN++HU4MMRyewpucicxLhsm7IuiydCmAxvVcUFRxDNXkS2VPBsyA40vcWcd63QOEC7eoDvIPV+rjjumMprsoI9eX3O3UDaoVqwNpDAWuoWaDnUsOTexST8F5aXFJGKxnQYRnwAd2nKL49M3XnQznQrOn6x2UGDJn+xRZleBZttyhZzm1weEwsOtXoaQTNtXdSDa2BEYPMFNSpH35KsUngPI8WVOn3nodplHlOFQQDx+jbAHiGXaiM4VRaa+Rm7ornJVQUbByFrZNYTk+1S/YqRKTkGfJKkn2vvR8KDR0atjiHMtLYK2uUgH1tb02joA4i31tfW4R6+tqi6vPCg2mLr0G9GWhuBiU3cATwg1mkiKpXVv1NqDxpnywkXsb3yqqGIUyNU9St7FECKJw0lIEbjrurE3PrwPlurIykHZ0Hp5t2r60oNECuPcF5Uegeq/FduxGkToq0sRQAVVo+62J0qYQMoMFXYrzX2dAtf9PbOJRGmwvv4Kv/RGERq/N4Y4IVJqzp4M3lvfZNmGX2SAMJwZoSPvBsqN/QBdZLqYq2CSjZyzMqQvQnyq7AM+4RFDslRjv8GJGPl17YRZaofhgQxfvcm4ZGfozcYhUpAE6r0o5nImc5lHlqhdJhvC3Isn0OUi9U2zpDJLe4TGkYqjiclLVpYkQMiwhwega3jfsUo+e4zQGEB7OMcebLq7xbM4+fkRlCdMOCqSLA25I/qcWOsFw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(346002)(366004)(396003)(376002)(31696002)(316002)(4326008)(16576012)(26005)(6486002)(38100700002)(8936002)(36756003)(956004)(66946007)(54906003)(66476007)(66556008)(5660300002)(2616005)(6916009)(8676002)(2906002)(186003)(86362001)(478600001)(83380400001)(31686004)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cHBaeHorRFVaM1JYZCtHbzVJQ2xTaGN4V29lYVpEZFVhN2hQQnBGK2JUK1BI?=
 =?utf-8?B?bFkzNi9yRnBlTlhLelVDV1dvSXRGSEYvYWRham1ZazdSd3JGVW00eVMrcWJB?=
 =?utf-8?B?UEVzRDc2bTRpVVhCSnk5OUtaRnN2WXMxODQ2UGU3YXJ1M1JnME5wWFErSkdQ?=
 =?utf-8?B?VnJDcUt3RzJMVE1DaE9RTzQrb091YjRId29WdVRXeVA2ZzZqWlNxMXRyN0Iy?=
 =?utf-8?B?Q3RnZFhoWnVBMGt3RDQ0aHBSM0RsWEdoZVdiZHlUMnk5Mzh3Q2lqcVdOM0Zq?=
 =?utf-8?B?SW5XYnRDUHYvcGRzQm1VeXl2bE1rWVo0N2UxUmxkRWZmeXJLQkFSRCt5Y2t3?=
 =?utf-8?B?Q0R6RXNUdGtDQmdjWmdjNHYvL0Myb1RpUmJVbjQ1SGsyLy9UakVRQlNuLzBD?=
 =?utf-8?B?aFZGQko3RFJrVzV6cHZQcDFmMXBBU1k5ZjBvNy9tSzZDUWltWFpHMnpGNkx3?=
 =?utf-8?B?Z082a3hQQVJzYWtzUm1jTnlvdFJNN2dxRFFkTm54aElzbEd0UGFJd0JwemxY?=
 =?utf-8?B?QkdXbnYrUUd0a01LOFBlYTloY09tWmc3aUkyb0tlekVPTlZQYSs2ZlFQMWdn?=
 =?utf-8?B?anFUMDBCWDdLaVhEVmNHa2JWcXlPOHdnQm5lUjRUTXkveUMwSVQ2Tm9GS25r?=
 =?utf-8?B?YUxuYTF6OEJjeXZSS2xNK2lIaUpLNElDK0xIcHFmZjhDVXIwbFgwVnlXY2py?=
 =?utf-8?B?ZjM2Vk9KNlJsWDNmTHdRV0ExbXU0cTRML0o3dUxpaTkzNkZIUWhHZzlIazBo?=
 =?utf-8?B?YTBRYjR0TERmdlRzaThhUHRTS0ZNa2NUdlhObWcrT3lmN1dmSmIxb1E5SXM2?=
 =?utf-8?B?VW9nYTgwYmVZbGZpQ0F3Q0hQM3BqL2VNZXdqVGFOZTBFVVlvUkxkVTNMNXVQ?=
 =?utf-8?B?U1ZvVkl0c1RBZDlGSlhJZnI5VVRWa1QxLzhFc3NMZnBJaUlpWElWN2NiVE9k?=
 =?utf-8?B?Y1FyamtITXB4UDhmVHlyUEY2dEJxZ09WQy9IdURjQUxLYUR3eUdlZnN2VHhH?=
 =?utf-8?B?LytFUkJoQ2FrRzkvMmt2R1NicjlJL01zaVBJQ1crR2drRjVucGlOcEZzdkNt?=
 =?utf-8?B?VjhGSEZ3c3kxMHFqNjZ5UXRXMi9SK0J6ZGkzK2xmZlZoUlVPcG53UURvLyt0?=
 =?utf-8?B?Tjk3NVA4bU5sRlVhVjRDa2phME02UlVoWHZyR1VsVzJQZTZuZ1ZQZnQ2d28v?=
 =?utf-8?B?MnZWcnFoNUEwSk5rVHY2UGlWcmpXQmtPcEI0ekpzZXFpWXZFOGZybHNXK2ly?=
 =?utf-8?B?ZkVURzVYaExDQWNSNUIvVGpocU9aYnhIVHVvNStTT2F4WEdaVUx4QTIydE41?=
 =?utf-8?B?WFVJMUJpeGMxbTErcWVqMzNscVh5OHE5YjlJUElCRmJxUUpiUWpKNCtZVzBD?=
 =?utf-8?B?d2ZJc3dvcmV2Q2JvVWxESTVXZzBab1dHc2RzZmp6NkxyT1c3K2lBYXpmeUtJ?=
 =?utf-8?B?TEMzZ29TN1lSb1diM1dQWVFqRmRlK2swREY2MEtMaWdYR2tiZmpBMm5hWFFT?=
 =?utf-8?B?VEx5OVRLVkhyazNsbHFEY1huOTh4Q1F4d0UvZ3BKTS9jdUlScTA2UVVlVmdF?=
 =?utf-8?B?am8raGtxYTQvRktpNjJDMUs1R0Z1Rnl5WWRZaG53b3UvOVVXVnZ6QW1UcElu?=
 =?utf-8?B?bEJUT0VIVm1HYXNKM2pUN1hnK0VyLzljb1E4OFlsM3dudXBLSXV2aHlhcnJH?=
 =?utf-8?B?MG5ycWc3c2V4TTJWVXNBRTlzYVJkMVVkcXdrZWJ6b3hhbjdmQU9ueTkxb0Zw?=
 =?utf-8?Q?iOr/B06AGBH2HempGW0ISsOvkE67OsvWdzyOcpd?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8aa65c14-25b9-41c6-6a3e-08d9784c42a2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:25:18.1323
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: V8YPv2hWZef89OpHv4zLKa90/MBjP/4OXjVZ3N7eOK1MbYK+ZZPVqjWcFMhmvPjmoYsasyqElZyBbFilqZhT5A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

When booting directly from EFI, obtaining this information from EFI is
the only possible way. And even when booting with a boot loader
interposed, it's more clean not to use legacy BIOS calls for this
purpose. (The downside being that there are no "capabilities" that we
can retrieve the EFI way.)

To achieve this we need to propagate the handle used to obtain the
EFI_GRAPHICS_OUTPUT_PROTOCOL instance for further obtaining an
EFI_EDID_*_PROTOCOL instance, which has been part of the spec since 2.5.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Setting boot_edid_caps to zero isn't desirable, but arbitrarily setting
one or both of the two low bits also doesn't seem appropriate.

GrUB also checks an "agp-internal-edid" variable. As I haven't been able
to find any related documentation, and as GrUB being happy about the
variable being any size (rather than at least / precisely 128 bytes),
I didn't follow that route.
---
v2: New.

--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -430,6 +430,10 @@ static void __init efi_arch_edd(void)
 {
 }
 
+static void __init efi_arch_edid(EFI_HANDLE gop_handle)
+{
+}
+
 static void __init efi_arch_memory_setup(void)
 {
 }
--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -922,7 +922,14 @@ store_edid:
         pushw   %dx
         pushw   %di
 
-        cmpb    $1, bootsym(opt_edid)   # EDID disabled on cmdline (edid=no)?
+        movb    bootsym(opt_edid), %al
+        cmpw    $0x1313, bootsym(boot_edid_caps) # Data already retrieved?
+        je      .Lcheck_edid
+        cmpb    $2, %al                 # EDID forced on cmdline (edid=force)?
+        jne     .Lno_edid
+
+.Lcheck_edid:
+        cmpb    $1, %al                 # EDID disabled on cmdline (edid=no)?
         je      .Lno_edid
 
         leaw    vesa_glob_info, %di
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -568,6 +568,49 @@ static void __init efi_arch_video_init(E
 #endif
 }
 
+#ifdef CONFIG_VIDEO
+static bool __init copy_edid(const void *buf, unsigned int size)
+{
+    /*
+     * Be conservative - for both undersized and oversized blobs it is unclear
+     * what to actually do with them. The more that unlike the VESA BIOS
+     * interface we also have no associated "capabilities" value (which might
+     * carry a hint as to possible interpretation).
+     */
+    if ( size != ARRAY_SIZE(boot_edid_info) )
+        return false;
+
+    memcpy(boot_edid_info, buf, size);
+    boot_edid_caps = 0;
+
+    return true;
+}
+#endif
+
+static void __init efi_arch_edid(EFI_HANDLE gop_handle)
+{
+#ifdef CONFIG_VIDEO
+    static EFI_GUID __initdata active_guid = EFI_EDID_ACTIVE_PROTOCOL_GUID;
+    static EFI_GUID __initdata discovered_guid = EFI_EDID_DISCOVERED_PROTOCOL_GUID;
+    EFI_EDID_ACTIVE_PROTOCOL *active_edid;
+    EFI_EDID_DISCOVERED_PROTOCOL *discovered_edid;
+    EFI_STATUS status;
+
+    status = efi_bs->OpenProtocol(gop_handle, &active_guid,
+                                  (void **)&active_edid, efi_ih, NULL,
+                                  EFI_OPEN_PROTOCOL_GET_PROTOCOL);
+    if ( status == EFI_SUCCESS &&
+         copy_edid(active_edid->Edid, active_edid->SizeOfEdid) )
+        return;
+
+    status = efi_bs->OpenProtocol(gop_handle, &discovered_guid,
+                                  (void **)&discovered_edid, efi_ih, NULL,
+                                  EFI_OPEN_PROTOCOL_GET_PROTOCOL);
+    if ( status == EFI_SUCCESS )
+        copy_edid(discovered_edid->Edid, discovered_edid->SizeOfEdid);
+#endif
+}
+
 static void __init efi_arch_memory_setup(void)
 {
     unsigned int i;
@@ -729,6 +772,7 @@ static void __init efi_arch_flush_dcache
 void __init efi_multiboot2(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 {
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop;
+    EFI_HANDLE gop_handle;
     UINTN cols, gop_mode = ~0, rows;
 
     __set_bit(EFI_BOOT, &efi_flags);
@@ -742,11 +786,15 @@ void __init efi_multiboot2(EFI_HANDLE Im
                            &cols, &rows) == EFI_SUCCESS )
         efi_arch_console_init(cols, rows);
 
-    gop = efi_get_gop();
+    gop = efi_get_gop(&gop_handle);
 
     if ( gop )
+    {
         gop_mode = efi_find_gop_mode(gop, 0, 0, 0);
 
+        efi_arch_edid(gop_handle);
+    }
+
     efi_arch_edd();
     efi_arch_cpu();
 
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -131,7 +131,7 @@ static bool match_guid(const EFI_GUID *g
 
 static void efi_init(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable);
 static void efi_console_set_mode(void);
-static EFI_GRAPHICS_OUTPUT_PROTOCOL *efi_get_gop(void);
+static EFI_GRAPHICS_OUTPUT_PROTOCOL *efi_get_gop(EFI_HANDLE *gop_handle);
 static UINTN efi_find_gop_mode(EFI_GRAPHICS_OUTPUT_PROTOCOL *gop,
                                UINTN cols, UINTN rows, UINTN depth);
 static void efi_tables(void);
@@ -747,7 +747,7 @@ static void __init efi_console_set_mode(
         StdOut->SetMode(StdOut, best);
 }
 
-static EFI_GRAPHICS_OUTPUT_PROTOCOL __init *efi_get_gop(void)
+static EFI_GRAPHICS_OUTPUT_PROTOCOL __init *efi_get_gop(EFI_HANDLE *gop_handle)
 {
     EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *mode_info;
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
@@ -772,7 +772,10 @@ static EFI_GRAPHICS_OUTPUT_PROTOCOL __in
             continue;
         status = gop->QueryMode(gop, gop->Mode->Mode, &info_size, &mode_info);
         if ( !EFI_ERROR(status) )
+        {
+            *gop_handle = handles[i];
             break;
+        }
     }
     if ( handles )
         efi_bs->FreePool(handles);
@@ -1219,6 +1222,7 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SY
     if ( use_cfg_file )
     {
         EFI_FILE_HANDLE dir_handle;
+        EFI_HANDLE gop_handle;
         UINTN depth, cols, rows, size;
 
         size = cols = rows = depth = 0;
@@ -1227,7 +1231,7 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SY
                                &cols, &rows) == EFI_SUCCESS )
             efi_arch_console_init(cols, rows);
 
-        gop = efi_get_gop();
+        gop = efi_get_gop(&gop_handle);
 
         /* Get the file system interface. */
         dir_handle = get_parent_handle(loaded_image, &file_name);
@@ -1364,12 +1368,15 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SY
         dir_handle->Close(dir_handle);
 
         if ( gop && !base_video )
+        {
             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
+
+            efi_arch_edid(gop_handle);
+        }
     }
 
     efi_arch_edd();
 
-    /* XXX Collect EDID info. */
     efi_arch_cpu();
 
     efi_tables();
--- a/xen/include/efi/efiprot.h
+++ b/xen/include/efi/efiprot.h
@@ -724,5 +724,52 @@ struct _EFI_GRAPHICS_OUTPUT_PROTOCOL {
   EFI_GRAPHICS_OUTPUT_PROTOCOL_BLT         Blt;
   EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE        *Mode;
 };
+
+/*
+ * EFI EDID Discovered Protocol
+ * UEFI Specification Version 2.5 Section 11.9
+ */
+#define EFI_EDID_DISCOVERED_PROTOCOL_GUID \
+    { 0x1C0C34F6, 0xD380, 0x41FA, { 0xA0, 0x49, 0x8a, 0xD0, 0x6C, 0x1A, 0x66, 0xAA} }
+
+typedef struct _EFI_EDID_DISCOVERED_PROTOCOL {
+    UINT32   SizeOfEdid;
+    UINT8   *Edid;
+} EFI_EDID_DISCOVERED_PROTOCOL;
+
+/*
+ * EFI EDID Active Protocol
+ * UEFI Specification Version 2.5 Section 11.9
+ */
+#define EFI_EDID_ACTIVE_PROTOCOL_GUID \
+    { 0xBD8C1056, 0x9F36, 0x44EC, { 0x92, 0xA8, 0xA6, 0x33, 0x7F, 0x81, 0x79, 0x86} }
+
+typedef struct _EFI_EDID_ACTIVE_PROTOCOL {
+    UINT32   SizeOfEdid;
+    UINT8   *Edid;
+} EFI_EDID_ACTIVE_PROTOCOL;
+
+/*
+ * EFI EDID Override Protocol
+ * UEFI Specification Version 2.5 Section 11.9
+ */
+#define EFI_EDID_OVERRIDE_PROTOCOL_GUID \
+    { 0x48ECB431, 0xFB72, 0x45C0, { 0xA9, 0x22, 0xF4, 0x58, 0xFE, 0x04, 0x0B, 0xD5} }
+
+INTERFACE_DECL(_EFI_EDID_OVERRIDE_PROTOCOL);
+
+typedef
+EFI_STATUS
+(EFIAPI *EFI_EDID_OVERRIDE_PROTOCOL_GET_EDID) (
+  IN      struct _EFI_EDID_OVERRIDE_PROTOCOL   *This,
+  IN      EFI_HANDLE                           *ChildHandle,
+  OUT     UINT32                               *Attributes,
+  IN OUT  UINTN                                *EdidSize,
+  IN OUT  UINT8                               **Edid);
+
+typedef struct _EFI_EDID_OVERRIDE_PROTOCOL {
+    EFI_EDID_OVERRIDE_PROTOCOL_GET_EDID  GetEdid;
+} EFI_EDID_OVERRIDE_PROTOCOL;
+
 #endif
 



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:25:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:25:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187621.336556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUv6-0007to-5a; Wed, 15 Sep 2021 13:25:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187621.336556; Wed, 15 Sep 2021 13:25:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUv6-0007th-2F; Wed, 15 Sep 2021 13:25:48 +0000
Received: by outflank-mailman (input) for mailman id 187621;
 Wed, 15 Sep 2021 13:25:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUv4-0007pg-OY
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:25:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6cafeef0-1628-11ec-b51b-12813bfff9fa;
 Wed, 15 Sep 2021 13:25:42 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-sU1tyMl4PJOKiOQMHaFUPA-1; Wed, 15 Sep 2021 15:25:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 13:25:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:25:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0033.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.17 via Frontend Transport; Wed, 15 Sep 2021 13:25:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6cafeef0-1628-11ec-b51b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712341;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+iOZotKkLcJY4CXebIky0sg+x0ejVE7dgfMqo24AelE=;
	b=J9a0+HBBNNPrmOxXSj9j3e38VTLqJouLFY04BeZ7O0+aGjG45v+LhnbCyz1AlelJMs0Fk5
	zAERE4A1D/rOv2KHj50RNOx40ivA6TviMPb9+4auAD/aHo7bSq1VNMiL82HWAugEYKtihf
	UadfENeMURHD1/QL9Dt7nfpJFbU/P98=
X-MC-Unique: sU1tyMl4PJOKiOQMHaFUPA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HjqLmfPJTlcT6Uq40v4YIkdeBsvCXYRr+SNip+0cff+Yj4E9YPoEuWcRjcyLTEEHHsE2l3vVO9Ryh8Nk+XZvyypp9WiJVvoPNyfdYQJDN821qic54FFrm2u+4JuEFMDrx9cLz4M5aIWe4eHNcpyPKotjYQlrV948BztXpv4Se4yGnfwL3YE80s+6AYd8UTPtfV6NLvu6l9R0itcwV/UvpMAUeLZSu2dmwqH8/z7Ru4b+34PpNFQkUG6XHba76THXbgwY6SA9qcYUvjJfeyVxH5NV9wXAFpDfaQvfZk01RCH1PA2U7ICDImb2muWAwfqE/cD/ZFVX+kCWhaAgJMkMSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+iOZotKkLcJY4CXebIky0sg+x0ejVE7dgfMqo24AelE=;
 b=mzST7ToNYe6I/VK1pRGOzGsiePTHMT03Q24GTE4axPv3GuXJK8ZCT6Jho2kqArpQMTjw2okltFH3DXg1zEXwjjk628Mvq0+b5L1ht6kYtW6fVb8RdN2NxkzSFUOLbp1uIJYk20RczXclysW5/m+XNSUfJb2fZbgWP3rJKR8XWZj6/6eXsx2qKGg+xyfx4eCfkbL417vgUn17lDLJamH06cDtRhLHLvOtZrOz7z845rcqMucQSP19h5pbMrQgCku/GgClZGEzkKKkh4yRLCvQDCADaMdbTWq90Fcmxf70Y+INRcTGQluG6SKMuxAQwMghnLBjr2pj75jA6uHl+4FXGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 4/7] x86/boot: simplify mode_table
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <855ba31d-45f1-7d98-68be-bb2f7f7d6a8a@suse.com>
Date: Wed, 15 Sep 2021 15:25:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0033.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 83b45e39-a3c6-44fb-f1a3-08d9784c4ecf
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43833143D8BE0967F5756460B3DB9@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OBwMWz9MYxasQAwJoxGRFgQgNkFWDlpWP/uqOGDeBWppJ56WFA/PPxjPDbfaZZL1Ot9WxU5Yfs/CbGcf1r60xJqLwsAuyfzBt95rCbzsHf0QGCtoSu7ELdQ0lBwcSXrbUvxFCzUvuRLdGTvRAIBwg+oIJboSPm2/+m62Xbgb5+oJat9FNnAM/JruoXYEkTy+GK7OHdJc3ol4ds2aseMr2WnpB/7c3aYAF0TNbLBycwkCr87bHOi60y5NKQMUeWDyaGvksz6SDYU2I8X6PdG2AZuIqdBCx0v3JCIbh/VqSeDBvvhmS4hwUcE6aYnUnBU+eKm9snLVQf+pWjdr8oJI+fHEd0yG/UBQ9t7EF8KNXqbPL/53/I3b0qeHP0oUoonc1/CudpG/3tkBqtYSWg6j05IZ3s/xuPjHab5dxqOlnEcKsBH3+sE3ApGahz/FfqktP+iy8HojppVPamq36s1W/bwA4J2ab7WaBdJLkYuvMUm11+/kuVkvBAg0c7xc/NJ8MqnbEZEmpTfh6Y8ara7UlQYoftWb5/7zSeLOP6pf1iGupFyOa21Bq45B3UN4A2hLiU5L4bJxpINGEeOi82C0ytXFc3onMLYMm2XQLoGgMcahS6z8WCCOkY4QnRJgAGPgOMgQtycvbrlS4VsATU+7sMn+sAJ0RqXaX7fAa1UNtczIpXkLr/dbudvgk+Y+EUnkpvSdm8klM6CDDMpVfG4GNQGv2yLdotkap8JCvikqZm8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(376002)(346002)(396003)(31686004)(36756003)(38100700002)(54906003)(956004)(16576012)(86362001)(4326008)(2616005)(31696002)(2906002)(478600001)(316002)(186003)(5660300002)(26005)(8936002)(66946007)(6486002)(6916009)(66556008)(8676002)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Skt5MHBYZUl2c3RsYmpKd1JXajdSNGtXNzNGZ0g0SUd1WTVqM1NucW5MNUFZ?=
 =?utf-8?B?NTVqSVRnS0thN3dGdU1Gc05IWXZHWmNKNjNMc3czbE1GMVZnQTVocWhWMHlo?=
 =?utf-8?B?c0tzV0lnNmh1anFOZVBmMWxsb2xBYWdNejlhbUw4L3pydkluMlJ3RFRRbUpC?=
 =?utf-8?B?bG94di9mTFUrK0N5V0RPR3FEb1E1NXVUZytXMTdPT1lEUlg3eVgrcmpSdExJ?=
 =?utf-8?B?RUEyd3RpdThkS3l6WDZJdzlPN3F1QjFONWpYVEJrY2NROGNIcG5PTnhRbVdN?=
 =?utf-8?B?S3RuQnVmWFBiTk9VbDlGaVY3eHlGU0RzVzA2WG5rYVp2SjI1ZmRQTG4wZEpw?=
 =?utf-8?B?NWV5WWFIWXRQZjVoZjB1NDVCNDk1OU95S3dzL3Q2VTFLQ2liWUpSajZFVlE3?=
 =?utf-8?B?c2F6dzBuMG5TWkJvb0RVT3FGeFBRNWl4UUtUQXpVd0ZULzlRcndBd1dIaWRQ?=
 =?utf-8?B?RDcvRDZ5bU8rakdIRWpLU2lxNmlCbmxxM0N4RGxmc0JOUUcxOTlNOURTTEU2?=
 =?utf-8?B?T1g5bXF3RmlGblU0MXpSNTd0R1RVSFp0VVcxZ0dqMFd5enE0SHNRU2tBbVRM?=
 =?utf-8?B?ak1JdTQ0OUxrZmpXWU84Q3VwWW42V0E4dHZiOXVzeVdJVVJoVDV1QzdsdHJD?=
 =?utf-8?B?MGZaOXRxMUpHeEJjVmhCVmtRM1IvZmhFU05VVjBxNG5uODF1eHpJSXphMmlJ?=
 =?utf-8?B?d1R4M1A4Y3JmbUN5ZDlnVFk4azRFWDN6Mzc5T3BRYlYxaW5TaEtLaXk2RGNY?=
 =?utf-8?B?TjY1dkszZTNiWEk0WUFVbmdtR2txZ1RhUlJKQUl6K002MVkwbmcxdnZMdFFO?=
 =?utf-8?B?bUdpWXFlWXZFVFI3MkxEazhaWXZVSTZBeWdVSTJhcDdNMUU0T21PdzlZTlRE?=
 =?utf-8?B?UXBqTTArV0VpMG4yVnFqWVkvV2FYcjlTaWNBa1JFVjJ2dkY0czJVbE9EOGxw?=
 =?utf-8?B?ZjlHNVl0bFJkeWJaZ1JObmhLWXkvUXBzR3FFUFFIRTk0SjZkbE5BOEkvZ3VX?=
 =?utf-8?B?by9xa29xYXBhd0ZaYlhvb3M5cHJhQkFaSFVmYmN6RVB3NjB6cmV0dzRmdzYy?=
 =?utf-8?B?eksvUnFnQ0V2TEtxRUNvUFZXd0NaUFhZQXd4d1BaSjVxK1l2R1ZYaTlYVzhF?=
 =?utf-8?B?cURkSFFnbzUzdjhYckYwZXVNRnY5djFwNElNQnQraGExamV6cW5BTjdTUjRq?=
 =?utf-8?B?V2R2UVBBcVlXbHFiVnVmR1QzT2pHUmF6cG5tRllIL2xmazM3SGVVYmYrQWNi?=
 =?utf-8?B?cUpiUktNQnRpaSt5VjBpUXltK244RlZWZExYU1ppRFFNRk5INndOUlJ3R2pm?=
 =?utf-8?B?V29WOWdIMzhaRDFXNWY5ZVBnQ0x1RUF2YS9XcUFWTGYrbTUwZXFLTHQ1bGhI?=
 =?utf-8?B?d0ExKytKazJIYzZiSTJVRFZEV2ZWVEhQY3lPZVpkbVMydEwxT3VPa2xleTVz?=
 =?utf-8?B?TXMwSHpaelJzVEVYSVJVem5SWDVoazRDK1ZCVTNSRmpLOXB2Z1owZ2J5NW9a?=
 =?utf-8?B?ZUgvYnk1UElIcG53OEVOTzhtL1l0ZERoeE5OYVBNaHBSSC9WUFlBTU5yNUV0?=
 =?utf-8?B?UFdXUk1pVDUyV2JpZHpmekFITEVFcE16Z2p1Wk5iNWNwTm5tYVRyNVhoaTlY?=
 =?utf-8?B?dnE1VXFPa1ZFVjVyTTFjV0Jhc1pmbElVdkZ0VlozR3ZCYk9Jc3l1TUFGYmdk?=
 =?utf-8?B?Y1NwQk05MWFxWFZybFRpczdIb2lSZ1M5NFBjczVjaUtRbHh4eWM0a2JnbHNz?=
 =?utf-8?Q?CRxk03tGnSfNkpu6K4HzrY+Pp/r1c5WiK4fEpgB?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 83b45e39-a3c6-44fb-f1a3-08d9784c4ecf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:25:38.5493
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PEz26FYzzEFc8n3R0otP0Ke9u+Kbeq20Wt+drrsEH+JOMCQZyv58KrGrN0Si6WltoG/qScmJdFPpyMrT6u8JCA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

There's no point in writing 80x25 text mode information via multiple
insns all storing immediate values. The data can simply be included
first thing in the vga_modes table, allowing the already present
REP MOVSB to take care of everything in one go.

While touching this also correct a related but stale comment.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -655,8 +655,9 @@ outidx: outb    %al, %dx
 # Build the table of video modes (stored after the setup.S code at the
 # `modelist' label. Each video mode record looks like:
 #        .word        MODE-ID             (our special mode ID (see above))
-#        .byte        rows                (number of rows)
-#        .byte        columns             (number of columns)
+#        .word        rows                (number of rows)
+#        .word        columns             (number of columns)
+#        .word        0                   (color depth; gfx modes only)
 # Returns address of the end of the table in DI, the end is marked
 # with a ASK_VGA ID.
 mode_table:
@@ -665,12 +666,6 @@ mode_table:
         jnz     mtab1
 
         leaw    modelist, %di           # Store standard modes:
-        movw    $VIDEO_80x25,(%di)      # The 80x25 mode (ALL)
-        movw    $0x50,2(%di)
-        movw    $0x19,4(%di)
-        movw    $0x00,6(%di)
-        addw    $8,%di
-
         leaw    bootsym(vga_modes), %si # All modes for std VGA
         movw    $vga_modes_end-vga_modes, %cx
         rep     movsb
@@ -684,6 +679,7 @@ ret0:   ret
 
 # Modes usable on all standard VGAs
 vga_modes:
+        .word   VIDEO_80x25, 0x50,0x19,0        # 80x25
         .word   VIDEO_80x50, 0x50,0x32,0        # 80x50
         .word   VIDEO_80x43, 0x50,0x2b,0        # 80x43
         .word   VIDEO_80x28, 0x50,0x1c,0        # 80x28



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:26:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:26:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187628.336567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUvQ-0008SK-EY; Wed, 15 Sep 2021 13:26:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187628.336567; Wed, 15 Sep 2021 13:26:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUvQ-0008SB-B8; Wed, 15 Sep 2021 13:26:08 +0000
Received: by outflank-mailman (input) for mailman id 187628;
 Wed, 15 Sep 2021 13:26:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUvP-0008Ln-Cd
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:26:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 79e5257c-1628-11ec-b51c-12813bfff9fa;
 Wed, 15 Sep 2021 13:26:04 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-lwp8FwXxNJSK1t9inH9vGw-1; Wed, 15 Sep 2021 15:26:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 13:26:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:26:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0119.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 13:26:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79e5257c-1628-11ec-b51c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712363;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GFSJGrEWRuZxL/lqFamJoez8zlHu39Sdg/lvj6ulEMc=;
	b=c7emvh79IgL9vkkDbGb3Aw9nWDSz/vo3UE1BiJ3CUa8IoQJh4SAP6ijMitWAhacyAKYEdD
	yUMkwuQp8sV/1iCryiS9TB7Ppd3HMFIFJmTc5Pr2z3F6NjIU9RHt8sq4qyZx9ge5mKxNlP
	WU+qFVWG6lrkB7x4v/xSLAq21CHmCLk=
X-MC-Unique: lwp8FwXxNJSK1t9inH9vGw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WBE2crGkAU+iP16h0N3S+/TEH8kR9ZIAXR35k9nAW2YXHk2028Fx2IhbP9TkNy041Gchl/NPc31fiIWVr7Qzp454+SSScgqRP9+B5jAD3WOn1dEVVJL5kHocl8bW8nqGcdKbIBnndKKA5Dy8+aB6vkbHYMnCA+tP/iavOwrcPaJrCFxxg2djvfHcs99iOyLIBD/aJY6jsUGfA20flh1PGDAEpn6b2OgC3gnES9wI3bbg9wdU8fmDwNHAoemhVhJvH5Sjtd9doHXNpmyVM6XwszvaIVkUy752XfmtX3wVZYBrab+idJcCm2EWhENZKUk/2bZfSguESDtAlfW+peJAGQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GFSJGrEWRuZxL/lqFamJoez8zlHu39Sdg/lvj6ulEMc=;
 b=gdsEEjka6fq55o2GToiY5wd11adRMDkXz6p10l+yuKWCvyv1rtDlfbbGi3IwJfPFsb24RpZD2uo2u4c+6CewjGRUq/HVCFkwV9ffLd9Vw1s3xsDYOPo9BObVzCw5Lk+tbTbnxxMc3blawTN5h6k4WP1QfNZfpWzjcqRzZ8J9j1VHWMrdYf94clamgygUHyMdyLycpq5JII0oh6n83rqDYKsK1ou5zlGo5mpf7hyUXNS/dxQBnNoBgGZkyZBuYPJilqP/515HPG/wiRyRt23ilqeT94KCf4UbIFtGFKjwWHg+N/7+Z6B0cyYfy/uZ3yIDF2odNSXN9iACbiFR+H5mEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 5/7] x86/boot: fold branches in video handling code
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <7a93f768-b416-1699-7db3-37e87f6d318b@suse.com>
Date: Wed, 15 Sep 2021 15:26:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0119.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a02da4d4-0116-4a6e-fd42-08d9784c5ca2
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383DB9B47472971EE4C9D10B3DB9@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eLSTnMvddxNo2NPOuDbjLV/xN/pN8BKMA/4Zm49LJAU3gEsekZrXlN7oDCgonsYkqhzN18sH+R1PiCpIV3QJMT0r8DSkV3u5THqjg+eq7bMMOruLVL4VgVI8iMpzntGegLhz2Vxf0bztG0vdPaceltyDMhwdazVw2js+8+XFdRDG8A2morzQ8FWfnpK1eLowt6vd3a4HUYD2pLkftTEw7JpTzGh8T3wh+Ar2p9t68+NWUwCW9kGgnco+ivE+Cggz/5vC1idEjA3ZGfUqGJLOLHuLz2cIpDhsiS9UADkKhpCEjYsVBGiVfZ6bEADPlCrPyDuD+5KOG0TkZbJ4yevzJLfUF55itfMbLruedoRFDUgryAdBKfrqC4Ie5+9VIR1lCpYFB9kTmOSO5/p/15tlafbFfqH2yTEGmeTu4/C3Mhpqe3GQ6irZ5vfhVwEd3nR3LpdVL0PLk/aTVN7p6dYWNUAhenOs1YLDbh1ovP0XWfJK7Kgd1R5xM+EVNfOf94s55qG5Xlc+mYTmz8Mbn9b/CgwJLLVHqrf+S8443kCmTNFOu8DQkA45f7hkC2h0FSG7fTfP58QDyDFQaWCIcNhEz1dGBu3zzLbBzgA59n5IPOxOZReXMK0wuSsgFQRCKHphPy03hr3jL1BKdD/omx04VTiLnww32mz5NJH5KG5XZVXpsxabf4395x1aSXsWyITA8TtVfaYdYNR00j7nH8VNOXjFWNE+m6vWQ1eyFpzn0k8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(39860400002)(376002)(346002)(396003)(31686004)(36756003)(38100700002)(54906003)(956004)(16576012)(86362001)(4326008)(2616005)(31696002)(2906002)(478600001)(316002)(186003)(5660300002)(26005)(8936002)(66946007)(6486002)(6916009)(83380400001)(66556008)(4744005)(8676002)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Zy9zOUI2ZEE0M3d4aWx6TG5JeW1nV0hKOTRjWmF1V2ZkemtNU0FoL1RFa3hD?=
 =?utf-8?B?azdPV3d6bi9naWFOR2lQT1NqMWJ2UGVzOFZuY1BIZlg2dkI4U0xUMVE2L0ZG?=
 =?utf-8?B?bVkxOHJpdGpyVDRFQVExTmcybkNSZ0txUzhVam9zVGRHOUJnZFRzYUpKOVdT?=
 =?utf-8?B?VDVtbUZJODNVdzZoYXkxZVI3RzJOVVowSmZWb3QvZ3p0ajdLNEUrSXJyOUx1?=
 =?utf-8?B?YWNQZFpPVXc2MitSVEpuUStwaUd1bmEvSWpFN0pWbjl4TEgwcXM5ai8wREVB?=
 =?utf-8?B?ZS9MRzB1S2Y2SkQvNStHZHlwUUgzRmd1WEZDSHBtVUlHenVOenpmaUIwanVx?=
 =?utf-8?B?cUdGNldoV1hsWHNNSnFtMVVCczZueUc1TnduaGdCNXJFVVdmSE5LYXRwNWV1?=
 =?utf-8?B?NFVpNXo5V1JLeTh1TzN0N3Vwb0FINVhOallvbnVpcVlhOFJRZ1Bwb3ZsVG9G?=
 =?utf-8?B?QzErd0pCQ2E2aWlEZXJ1S2ZhMCtvN3BwdWpSeUN3TU5SUnNvWXllUW1oNFp6?=
 =?utf-8?B?S0xnUmJSM0gxTTJYTm1pR0VuOFk1dnRyWmZzRVhmeHFQQm8xYU1nNGErS1dI?=
 =?utf-8?B?eER5TzJzQlBkUjRDdzlpeXQ4ZjZGYTFhTVFUNUVqb1VnTzNVRXlhRUhvdmRY?=
 =?utf-8?B?azhlMEw5VjZrTVV4R284TWwxSEcrbXo3ZlhqSmFrcmVQVllKRlNEYU1yVUFZ?=
 =?utf-8?B?blVsZUM3MmhyN0NraXM3NlkyT2U4M3JUd2R5VXZlQ3VUVW0rQjFzcVNSZEFa?=
 =?utf-8?B?eVBIWmpkbnlpTmx4Q3RJMFRNSCtZd3Z4bzYxMCtJVXVZRm1UalhGdGlGdmlF?=
 =?utf-8?B?Q2VDdks4cDVsaDZNTzJGWUFjSEUxeTd6VHloODJURGhocWxBOU0xVTdmYlRj?=
 =?utf-8?B?NFpVY0ZjM0VEU2RQNUM1bDZGZFJ2RnhJZjRLZFZiUnJZZkFuRXF0L25DaUlD?=
 =?utf-8?B?dlZxNlc5dVlUQUgrS3QzR0VQZTZIT0ttbWJWbERJYWFoYTBiZGNMK0ZSSlJT?=
 =?utf-8?B?RzhpR2t0SDlXdDZ0M3BhdTBLQkF2dzY2dXdndm5nVFIxdVhwbGREWWtiSE9W?=
 =?utf-8?B?Rkh5RkI3U2lCcVBCTUdUNEhtaHMrU09hdXAvY3lpVFJncm5DWU1ORjQwdnNH?=
 =?utf-8?B?MnpMTkZKOVNVaUsyaXlLaUYrSUZQcVJUQ3ExVmpvSVJvOTdNQjZrZWRqVnR5?=
 =?utf-8?B?NUg0QnZrQjMwRGJESDNzcGhyTEFDNklmZjhqcmc2dkJjVzREZnBXSTg2SEll?=
 =?utf-8?B?YjZITTRzNDY5TnVleE55MVRRcm0vYUdpKy8zNzVzdWRncm5ib0pUb2xwTm5Y?=
 =?utf-8?B?U1FCV0gvTVB3cFhoYmZzNmZCYS9WM2w1cUdOMm9HUmExb2RHcEtiL0ZIbExV?=
 =?utf-8?B?YXp5akFvK252OG12TWN4NVZaZTFkSG1ZRTJVbFMxV0kvanl6L05EZ1I3MW5t?=
 =?utf-8?B?RDg0b2hpOTl5aFlKMC9ZQmI0V3JRL0tzU0ttMmt4UTJEVDNzUUw5WDFCcmNB?=
 =?utf-8?B?Mlc4dkM2U1d5TzFRTk5TV2hVMFZWTUwyZTdTZHNCbFVEeEU1ZW1HQng5QS9D?=
 =?utf-8?B?YVZVNWVMVXJjam1aMkV4Y2txU0I3K0NmRFJkQUFiZTFXTGx1RlFlb1BHcm5Q?=
 =?utf-8?B?Uk5UNzMzZXhOcUdMS0VLUGMrYmNobjMyUWxrSjJPc2oyUmZsdko5UjA0dEZW?=
 =?utf-8?B?TlMxT0xuWm1BSEF4d1lGOUFkbkVVWWJBalB4Q1JQcTg3aXZQc1NETXJzbTdI?=
 =?utf-8?Q?bFcXPdWs6l1Q3qxSa40NC9BkIKVrF95iesTMkr+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a02da4d4-0116-4a6e-fd42-08d9784c5ca2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:26:01.7268
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oIc/iplh4qIPU0P89eMXxK4meNZP96sf2Z88Njds/aZtdpaOHcxQFA8OHb6RvRjQ4gpUy3wTqzonIbf5Cj+Wfw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

Using Jcc to branch around a JMP is necessary only in pre-386 code,
where Jcc is limited to disp8. Use the opposite Jcc directly in two
places. Since it's adjacent, also convert an ORB to TESTB.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -332,8 +332,7 @@ lment:  movb    $0, (%di)
         cmpw    $0x656d, (%si)          # 'me'
         jnz     lmhx
         cmpw    $0x756e, 2(%si)         # 'nu'
-        jnz     lmhx
-        jmp     listm
+        jz      listm
 
 lmhx:   xorw    %bx, %bx                # Else => mode ID in hex
 lmhex:  lodsb
@@ -401,10 +400,8 @@ mode_set:
         cmpb    $VIDEO_FIRST_VESA>>8, %ah
         jnc     check_vesa
 
-        orb     %ah, %ah
-        jnz     setbad
-
-        jmp     setmenu
+        testb   %ah, %ah
+        jz      setmenu
 
 setbad: clc
         ret



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:26:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:26:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187638.336578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUvp-0000kQ-RB; Wed, 15 Sep 2021 13:26:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187638.336578; Wed, 15 Sep 2021 13:26:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUvp-0000kH-NS; Wed, 15 Sep 2021 13:26:33 +0000
Received: by outflank-mailman (input) for mailman id 187638;
 Wed, 15 Sep 2021 13:26:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUvn-0000hM-Rr
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:26:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c64716c6-913d-4a17-9d25-b71c7f5df692;
 Wed, 15 Sep 2021 13:26:30 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-Te282JS8P-ipjAZzefVUmA-1; Wed, 15 Sep 2021 15:26:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5327.eurprd04.prod.outlook.com (2603:10a6:803:5c::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 13:26:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:26:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0062.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.9 via Frontend Transport; Wed, 15 Sep 2021 13:26:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c64716c6-913d-4a17-9d25-b71c7f5df692
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712389;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=juBJGq4ZapzX50bKGD151YensGcEVFSl2L4HTOV0500=;
	b=EoaUIVEMT1u5h5U6RUBEN0CRLlh/2tpTGZIiAtCpP2XfFOC7VDyPGXk4Cv+w/tJneER/4Z
	GO7SwA7N5JjMLxniwDpJpwugVbmOd0a2L0SJpfvqj2NQWaUDX7ZHMjJ+lzmmuz1YZFB8cn
	lqXqZeJEkHlBdHk+QmBwAWFdAets7Vk=
X-MC-Unique: Te282JS8P-ipjAZzefVUmA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RRf6l3iHjaqWdxP6tpBPUBUjDy1cAfswlfIgtIO+WAYxoRsUHmktuHfG6zO4bxb7W5WzcAiW4wR256JzD6bHoOrPv7/lUtTHWv/EzoKKoU8lhNkRqVAU8oZr9854R7IFRBTWyCI/BbXJ8zjksXn7NzmvRMiv2iFLQkyMD4lKChHathqBXezEpVaNEGg2uChGJiY3HvEnuGvlZFf+hvNcgM3nCQcORDIfLIKhpW2pTlceck7kezy3YasKHt0IMWlVSUcil8oPKOrEMUmCAev/MnV4gGUKnbEhNRE8pi95845SIMrAyjfq+LKdO1gGxjS0y1+qVYlE1jtUQyH/Oe2umg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=juBJGq4ZapzX50bKGD151YensGcEVFSl2L4HTOV0500=;
 b=dgQpvmqmrjMVrlsuiOlGbGA6Y7dYwHyGPqy+277Cq9hmHisjAiSfQhmOSEJyRDtzGX/eQDoONargnk7eY21Zi/XPVxAocUD7Hdebu1fZXLNP9D4PkzcXsOIhgf+zboyzdaB+GOBwMGqaNSBvVEfHA5SVNuFww41TBAE2AO10/0GzJu2YEvqDhiSAMgXfEpSAdraTBNbC9KazT1GeYp9hhcM8wL4ugj7lbo9L9Kd/DHIliukC0YrMlJ5He5Ecv7zHNbhiezglrecYwb5a7O6xKRWHjzWCoLG8XxEO9d0Ck0EK8p5/aI/pyjLylo3qAQAzyr8P4WgUv+gvOJwKwGd8uQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 6/7] x86/boot: fold/replace moves in video handling code
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <60ce3c64-53c2-9042-18d9-ff9e569fd4a2@suse.com>
Date: Wed, 15 Sep 2021 15:26:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0062.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 67564867-f239-4245-38d9-08d9784c68f0
X-MS-TrafficTypeDiagnostic: VI1PR04MB5327:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5327D0D6A4CCFF38741E8350B3DB9@VI1PR04MB5327.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	S5Ht2TRwz5P/eoFQfOSYAHcKk2ekUsJX5xFkUHbWoSPtYJHYSDSncBmQXpGhQDwR2jr9IZBbNhqSqyYEPahGMuulN7jeZXULDpM824XEK0pJkd85oTrCUFyE8TzKmZ/hAPCa8v7rW0FtDcekOO6pwHVntVZ4EHxTwaO8ghQ//vnfPCFYU2xMZo1SXCB9Crv9MG9yPyMrF/if2L7mmwg1xvdrZsGKeGJGplaVR7oR6L1It3d28sDgmohxBVJpiuVUrf2Dy69v2qZzyyEw6CXRJ0O6PX2GPxaEJELHMbmP5lHasC/E6Ld4H64o0ELabJ+QFve+XuooRAudPGE+i+4bPeeDdqEiUkbcMKlEg5PnU3+f74AmhWROIjk5pZvKozJt9wDIcC0GKYUrP4L3D88FWqXq7mZWyrZl6IqYYeJ1AIxfkbi3duk2cv/+UZpWsQGo8El52wqVK81qqpjm8cIWp4dCa2lXNSFu3rf2o2F0QeV7xjXlGI4kQ94jzJ7zjghd+W+9R+5RiaSM8V8ucb2dSqeqvEl1y9y8qmCmc4KDAWwCqVGGUv3ERE54LSd6/aEsx+JP7fpei0PD8zLy8yXpnLUoNAA7V7zKdTvs5SOHYKP8kMlXXNrxYc/l+tReSrMhawCsyRQghAJdRoaTBvH+eg+zS6MxE8tP2O6BuYm4JAP0E33KG1qH/AqJ6N8nhVL9v93bPlini/phF3zucvaKxGHUQlnHpyXU2V4VUWMujEY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(39860400002)(136003)(396003)(6486002)(316002)(66946007)(8676002)(2906002)(54906003)(478600001)(6916009)(8936002)(66476007)(66556008)(5660300002)(956004)(4326008)(86362001)(38100700002)(36756003)(31686004)(26005)(31696002)(186003)(16576012)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VW51cks4NE03bXlRbitwcmJ6QWtrUS9XYW9oeUZ3UytUZ212RjJzNkUyKzRL?=
 =?utf-8?B?eGVOMFE4bnpybUc4ZmlKNDJsWmpQa284RUJSNFcxSGl4dEpBR1FKYU5pYTRj?=
 =?utf-8?B?ZXlPWHVpcjY4RDR3eXV3QmcyZ3VqdGRFc0pUbGxhckNKQkR3L1NRT0Zqd0hj?=
 =?utf-8?B?Yys4WURuVXR0WjNBT3pjTkdhZVdkZlBWdlVLQU8rMStjWEFScnZZZmc3cHhZ?=
 =?utf-8?B?QXlJb1VwVzJOMUFpK0hsSmR6eUxGaVZJL0poOGxtSkVUUWVVeEtlOWlUWCtV?=
 =?utf-8?B?TGJyT0NGazlpd2loQXRaRTdYUVA4RkRSUVRiSDRtN2c3TlpvT3lPaTZaZGUy?=
 =?utf-8?B?OHA3NC9XaHVZSm1Kdld4MEpBeGI1RnRGRURCeU1xSXVtTGF1dnlRTG1FS01w?=
 =?utf-8?B?UUZ0T2taNFFHdlZSckVaYkh0U25aU1g0bFFMejk4aW5hMUdXbkM1eGJRRmFO?=
 =?utf-8?B?MzQ2aGR6bTg3U0N6NG5mU1oxQUF4THpMV3p2bWx2YnBPZ0E5VE1heXdBTXRi?=
 =?utf-8?B?TTVKKzMzYkpVZnpvY0RJbzZSazdOZ1JpOU15bDhWZlZvSGh6UUFMZWRhY3dG?=
 =?utf-8?B?OHdvQkZZOGhFaG84S3JrOGlkWjhJVTRHL2xOMDhjbG1qczJWbE5rMng1RnQ2?=
 =?utf-8?B?RWhQbTY1SGV4Vm9mNWxmdmhTL25rWW5PMFI2a0gvalFxQ2xVNmg4R2RiNGho?=
 =?utf-8?B?SExUNTNxdk8vTTdlUkx4SlJpK3J5VDk0cXRuKzlBU1RSMDNwaHNOQURhSnc2?=
 =?utf-8?B?Q1lxWDhoMkpXUjBxTVJXVW1NelVsRFR2K0VSZi9vMi9NTG1HUlg3RVZoeGRE?=
 =?utf-8?B?WTRtalJ6c0lsYkROQkZhQTF5dFY4d3QyNkZDem14aFd0a0dvVm4rbldaTmYw?=
 =?utf-8?B?ME9OaW44OFJCS2M5YnJYT3A5dzNoWUNuU05RSk5YQmZsREhZTzZUZU5mL2dj?=
 =?utf-8?B?ck81SHd4UGQxTlpRZ1hmemdJRkY1RndlYlFWMGVSeDZvbWJrT3o0YlhpYksr?=
 =?utf-8?B?eFVLZFVNaVpPeG8vOGh2bTY1L2JzZC80NWpOcXMrS0VKM0E0SDBCb0JQenZo?=
 =?utf-8?B?Ym9CNTVreGRZSy9lS01aWEZoNDFpZjlsZEo0NzJYdUljY1JsdGlIUjFIcmhO?=
 =?utf-8?B?Y1I0L1dIcUI1V3MwTXZYakQ1TVNFRDV6K0ZBNHBieVlGSEZLWXp2VVRCNU1H?=
 =?utf-8?B?VGdmK284UitObnd2MHdwaHNnbVN3SnZqYkpwMXg1REJ1Y1VlS3BZSTdXYUl0?=
 =?utf-8?B?STVCQlowUzh2V3hpM2dRSjBKbUE2enU1NU8wckk2RVVTbSsrM044T2RkZlZD?=
 =?utf-8?B?TGxELzBkdWJrSGNhNlZhck45bCtCSGdSV2hwZS9zYWY0SjFqTmVvRmdqYkU2?=
 =?utf-8?B?YW9DRC9zcm1nWTN1L3l2c1RTbStDWHlPdkU5Qll0ZlVBcFNoVWNKZWwwNlAy?=
 =?utf-8?B?dzlsdkZTTmMxNGg3aklQRHpVbHQxZjFsazkranFyOWIzNEN4Ymt1QnUvT0Qx?=
 =?utf-8?B?cUEwbVA1QU1UUUlKTjhLSXgyam51aVNVakQxMXp3UnBiRmwwVDBKcUV0VExx?=
 =?utf-8?B?L3dhSVRZZGtXaEY5UkhwbVZSbEUrWmpuV2xXMXI5MEVUTW4wOExKaEVNU0s0?=
 =?utf-8?B?dFVlSzFVaUNKZzJqaU5JeDFjMTBrM0VFYlFhNW8waVd2VkJ2ZXhHZXYvWTVi?=
 =?utf-8?B?ZmZ2S0dNc3NSSWVWZXhOMXFpYlRrZkhMNHZ4VUxMYVYxLzd6ZmlJRUNOcEpr?=
 =?utf-8?Q?XDw55ozUhL8DTJSrp4xRbn8R+ipg9s+wWaBp5ZJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 67564867-f239-4245-38d9-08d9784c68f0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:26:22.3706
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1SKC6uvzrGlXiwAwkpBGKsLb5EKv5Q2+lkOn9SInE94k4wAjqFikqs4rrrsRBR/RnCTuR21fC9/uUp9A8TNaTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5327

Replace (mainly) MOV forms with shorter insns (or sequences thereof).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Of course there's more room for improvement. For example there look to
be a number of LEAs which really could be MOVs.

--- a/xen/arch/x86/boot/video.S
+++ b/xen/arch/x86/boot/video.S
@@ -121,8 +121,7 @@ mopar_gr:
         movw    %ax, _param(PARAM_LFB_WIDTH)
         movw    20(%di), %ax
         movw    %ax, _param(PARAM_LFB_HEIGHT)
-        movb    25(%di), %al
-        movb    $0, %ah
+        movzbw  25(%di), %ax
         movw    %ax, _param(PARAM_LFB_DEPTH)
         movl    40(%di), %eax
         movl    %eax, _param(PARAM_LFB_BASE)
@@ -135,8 +134,7 @@ mopar_gr:
 
 # get video mem size
         leaw    vesa_glob_info, %di
-        xorl    %eax, %eax
-        movw    18(%di), %ax
+        movzwl  18(%di), %eax
         movl    %eax, _param(PARAM_LFB_SIZE)
 
 # store mode capabilities
@@ -144,14 +142,11 @@ mopar_gr:
         movl    %eax, _param(PARAM_CAPABILITIES)
 
 # switching the DAC to 8-bit is for <= 8 bpp only
-        movw    _param(PARAM_LFB_DEPTH), %ax
-        cmpw    $8, %ax
+        cmpw    $8, _param(PARAM_LFB_DEPTH)
         jg      dac_done
 
 # get DAC switching capability
-        xorl    %eax, %eax
-        movb    10(%di), %al
-        testb   $1, %al
+        testb   $1, 10(%di)
         jz      dac_set
 
 # attempt to switch DAC to 8-bit
@@ -164,17 +159,17 @@ mopar_gr:
 
 dac_set:
 # set color size to DAC size
-        movb    bootsym(dac_size), %al
+        movzbw  bootsym(dac_size), %ax
         movb    %al, _param(PARAM_LFB_COLORS+0)
         movb    %al, _param(PARAM_LFB_COLORS+2)
         movb    %al, _param(PARAM_LFB_COLORS+4)
         movb    %al, _param(PARAM_LFB_COLORS+6)
 
 # set color offsets to 0
-        movb    $0, _param(PARAM_LFB_COLORS+1)
-        movb    $0, _param(PARAM_LFB_COLORS+3)
-        movb    $0, _param(PARAM_LFB_COLORS+5)
-        movb    $0, _param(PARAM_LFB_COLORS+7)
+        movb    %ah, _param(PARAM_LFB_COLORS+1)
+        movb    %ah, _param(PARAM_LFB_COLORS+3)
+        movb    %ah, _param(PARAM_LFB_COLORS+5)
+        movb    %ah, _param(PARAM_LFB_COLORS+7)
 
 dac_done:
 # get protected mode interface information
@@ -504,7 +499,8 @@ setvesabysize:
         call    mode_table
         leaw    modelist,%si
 1:      add     $8,%si
-        cmpw    $ASK_VGA,-8(%si)        # End?
+        movw    -8(%si),%bx
+        cmpw    $ASK_VGA,%bx            # End?
         je      setbad
         movw    -6(%si),%ax
         cmpw    %ax,bootsym(vesa_size)+0
@@ -515,9 +511,7 @@ setvesabysize:
         movw    -2(%si),%ax
         cmpw    %ax,bootsym(vesa_size)+4
         jne     1b
-        movw    -8(%si),%ax
-        movw    %ax,%bx
-        movw    %ax,bootsym(boot_vid_mode)
+        movw    %bx,bootsym(boot_vid_mode)
         jmp     check_vesa
 
 # Table of routines for setting of the special modes.
@@ -773,8 +767,7 @@ vesa2:  pushw   %cx
         movw    %bx, 2(%di)
         movw    0x14(%di), %bx          # Height
         movw    %bx, 4(%di)
-        xorw    %bx, %bx
-        movb    0x19(%di), %bl          # Depth
+        movzbw  0x19(%di), %bx          # Depth
         movw    %bx, 6(%di)
 
         addw    $8, %di                 # The mode is valid. Store it.
@@ -901,8 +894,7 @@ gettime:
         movb    %dh, %al                # %dh contains the seconds
         andb    $0x0f, %al
         movb    %dh, %ah
-        movb    $0x04, %cl
-        shrb    %cl, %ah
+        shrb    $4, %ah
         aad
         popw    %cx
         ret
@@ -959,8 +951,8 @@ store_edid:
 .Lforce_edid:
         movw    $0x4f15, %ax            # do VBE/DDC
         movw    $0x01, %bx
-        movw    $0x00, %cx
-        movw    $0x00, %dx
+        xorw    %cx, %cx
+        xorw    %dx, %dx
         movw    $bootsym(boot_edid_info), %di
         int     $0x10
 



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:26:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:26:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187646.336589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUwB-0001Ha-3n; Wed, 15 Sep 2021 13:26:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187646.336589; Wed, 15 Sep 2021 13:26:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQUwB-0001HT-0i; Wed, 15 Sep 2021 13:26:55 +0000
Received: by outflank-mailman (input) for mailman id 187646;
 Wed, 15 Sep 2021 13:26:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQUw9-0001Ge-Ip
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:26:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 39e1bfec-14e6-450f-88d6-9d4ab6c63143;
 Wed, 15 Sep 2021 13:26:52 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-m0u-xFKfPUS8eSUh7SStsg-1; Wed, 15 Sep 2021 15:26:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5327.eurprd04.prod.outlook.com (2603:10a6:803:5c::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 13:26:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:26:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0043.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Wed, 15 Sep 2021 13:26:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39e1bfec-14e6-450f-88d6-9d4ab6c63143
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631712411;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KNU533Nas71KGaxkrjt8Y4qhfH0SyWRPA/lY3YgjMd0=;
	b=etd6icAyDgV0LsWvCl+WEd1BjTm3Qwv13LIdQUiFU6d4TBJQcn56hMZDQjJcKBcbH+ERAa
	bhpoH8jXVepYivBSVTERf2uFwhxTZHRM57A+s1F6K7tIs9jvwnO+8sRb/nTkMzr3c8qjLk
	Nc/aTSCbLFReFbKtj2MPOc/PPNZp5Wo=
X-MC-Unique: m0u-xFKfPUS8eSUh7SStsg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dXJe1CvuUrpYq6H9KGPvpt7ar1/bkWERk3uEVIjMxjBmHUVj6nvvzFtntGfT0Tp1PPvEcn0cmPgK6jSgt0JlB5el1Sf/OuCgxgudJrwSqjLJ23w7XM3dYwEuYQY9F8eFEyHVbmfs/1GDLX5VoRkMjfgnh37hyYcYanI9Z3BbZ3XPFliXjEylSY3CQdFmj1fB27yUNsHRqZbt208AXrc54mz//s5HipHUAbxsGlO+/z819P1w2Bj1HUkPpDCZ/vaFdaT93ZRq0WhFpYL9Rux+cUeZBtkSSOsuklB1TdoFVW5ztTQILQyWNMyiFr6ToIizA8r738qfyvD8VRO/184fhw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KNU533Nas71KGaxkrjt8Y4qhfH0SyWRPA/lY3YgjMd0=;
 b=VXKsUlbkagqgre7uehe3Q/anitst+U/HBD0UfQ4TQLj3h1YUYeVCzCUvbHsjsCm0cMtJ/1PuFMObcka++nmec9XVvFw/ORLleZdDIE1kNTD/vja6gu1zz8qY++st5q1AIpwlKkyfyjn/w2LYZyEm/+/ypYHut8FqyR2yvxQEw5A28fDPD+rbJf0HEyKartI7PqoWjuxSfHqISAch95u/3knU2snw//+byS0RQA3bw5YEhOIacylkkGrYD8mv0Yu65GDtBDhu913lkORih5Hu/kP1yeN96Mhz+LN2UzNTE+TYXmm85J+Mw9u5tsHHEi7ZWgP1+DHoZwadZRfKq07O3Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 7/7] x86/boot: fold two MOVs into an ADD
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Message-ID: <85e26647-caef-9f18-eadc-452c4c4ce19e@suse.com>
Date: Wed, 15 Sep 2021 15:26:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2dbcc7e4-58c5-5724-6434-51be8acfde35@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0043.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1c3a53a2-a77a-43df-c71c-08d9784c7965
X-MS-TrafficTypeDiagnostic: VI1PR04MB5327:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB53276C90210BA35DDB31CA1FB3DB9@VI1PR04MB5327.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GA2UriHqqpe/KQL/aOg1PCWco+ZVmlKQS7s1utumEl49r3d6SHgOo4fyonfRBChk1/ppN7+FupO40aBMDo1wawQJYu4BYA0RMjPcc5i3VlhDTgtDgM4x5BH+COOmjndL5ovAM8AY8NMTVrzzKfl/4S//D4gdY7WQMMXUirPcH7w+oj5gj0wR+76L908hFW1bBD07Iaxalv0Ags7N9VQrtI/Qsg4nmLNCHNleFfYKOahSliG7W0YuHX9Km/S7zaRphkhcbJ42Q55zxp9NzKYLcxWWSmO/zafRCHYicIyeNwu52C/DjZAC1gaEUIRZ3mNNnOdRF+YRMfrICECBN6JqQSupUDjI75JusFzHBKuIwlyXqqtfKXdnFmHXPgBx9TehfJbYA4XAgIWiE45Lh7ijOzEujyFexmmurMuL6PCFsnh937Se/MXDEXdqpGUloSlVXmuwnor+AkUr1ssA7lv670+h+lgVfQ210LUk37R3Q/lkCNu0V8KOXwRqkPp+oOx0j0sxgERoW/lNIKYAmHzXZYZmiK00KsVnwcyh2J266uW/HJWpnLC+4rzvU4mdsvpuX2BnllynR+y66LWh+NXb7k7MjvFD43be5C2nL976io3V8WiQKlWuff+0L0+zXLziCHXe1xeiPKGihT0JXvSZqTEoVN8siL0Aa+0mCe4DDzv96EouZKYItLv3TaD3q+v/SeN4ApeqBWs95pDTpk4nk0qIZxbGPMSBzIj8AxGQ+FQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(39860400002)(136003)(396003)(6486002)(316002)(66946007)(8676002)(2906002)(54906003)(478600001)(6916009)(8936002)(66476007)(66556008)(5660300002)(956004)(4326008)(86362001)(38100700002)(4744005)(36756003)(31686004)(26005)(31696002)(186003)(16576012)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TGpKZEZrQjFWUm13OG9EZmFBQVFta2lPSjB6TGsyMWhrWnpzd2lNZ2szWU5h?=
 =?utf-8?B?cFhYWDlCM25seHVjS0p1RFRGVWNHTTVNeUMrS3c1aWozVkJqZWhrM0xiWmVs?=
 =?utf-8?B?bGtVV2U0RzlOTG1xdEI5TzYzdXdRNXdaK1hLUHExV0RqVmJrVzBoMkdZYnVj?=
 =?utf-8?B?QTlHQm44c0lUZHorSWMyWEJ3RkZLaXViYi9wWFd6TmxIbXlFQVJjQWtacTE4?=
 =?utf-8?B?ZWpac2dqTkZ4N05ubE5uU3RHaXBKVzU5ajFncisyYWIyeEdLT1VCeDdLNTBt?=
 =?utf-8?B?M3lhVmowZDBIK0dLa09udFJUMjBCSmp1QzNBWitCenVBZUgxOVZrTmN5NVhW?=
 =?utf-8?B?TFM1RDd5TWlONmNobGRKT0RkNUMyRDMvT0JPSVFDaHJOSGlFUVFYMm5KSDEz?=
 =?utf-8?B?dnJFME9OM01sdkRtWWJzbXdqdlBXYnJKc2QwSTFncjcwcFdlTncyZ25nVXpB?=
 =?utf-8?B?RXBZYTNUb3NlR1duRDlUMENtZzlGeHRUS3BwS0UvWGdIamZPTjF3SjRCKzF5?=
 =?utf-8?B?RHdrTUIxQVRVWjlPNVEyNFp2Yy9UZTZWS1JvWE1NeUFxaHc0NEY2WExmRmpI?=
 =?utf-8?B?MmlUbW9hTElkUCtZNFhNcU4xNG5NKzM5M1IwdmJURUVld1dBNmRvcU1ITnB6?=
 =?utf-8?B?aExMS0N6bXBWYmY5eTlsOGhzdDc3REJ0WWNpNisyTktiL09OdHVKT2xKUDB4?=
 =?utf-8?B?cWZsZEJ2aTlrY2NqZTU1cFZhbFJBWi84WDNhWWxEbFNaMTFiRm9UQ0hYYUQ5?=
 =?utf-8?B?NmxWV3c5QnVUT3VpckQxQnQ4Y3Y0MHo0NlVYS0VtQTMrVjIzOGNRL0xYWmxi?=
 =?utf-8?B?RWwxOG9ORUVlZ1F6bkw3TFZjSDU5UmdPSzRlSWRnaTZJdzF5emN5OHVwV09a?=
 =?utf-8?B?S3UvUVZrcDdGMlVZL25taVdZWVJuMGIzRkc4aGZLbU9hTmdNR1NaV1VMMk54?=
 =?utf-8?B?K01nYlRHY2drQWcxdWNJZTFhZjhiNEE4MUcwbWM5b3B6dzlDYms0SFNJQ1p6?=
 =?utf-8?B?eFgvb2I2dlg4akZrcXkxT21aQkNaNytBTG9iSHY1a3Y3M00vQ0VFaHpiOEF5?=
 =?utf-8?B?TWcwRHAxSENaK2JBR2kyallPaW5adno2V2hoVC9RSnVRVmppcnhFUjZRRmFk?=
 =?utf-8?B?QlRSZndISVVKcjE0T2lKa0J0N1lRdmZPTXZaZi9LcXJEY1dKS3dTeFhyeVY5?=
 =?utf-8?B?RU9mWmRBWGcxMm8vRUFjOFVJeFRrd052SWFKQStYNkNxUzYycVVSSi8rQzhO?=
 =?utf-8?B?eDM2WXV2ZExEdEZwWDRweHhMV2plZVZ3NGJVeXJSMG8yaFMyK1M0MlVDVWJy?=
 =?utf-8?B?Z2FjNFgwTUh0YTNZRU1odXVCQXFLYVpSZjR1SzdFZWpiUFcrRGwvQnh5dGh0?=
 =?utf-8?B?bE91NEhvam9WOG4wTXBpdzl5RDlWNzRld2VaaGtXQnpROEV0bFVJOTZPZm1z?=
 =?utf-8?B?OVR2aWV6YWN3QzhKYTBLckdWL1dQT1E3LzhoOXEvL2c0bmZIL01UZnpJUVRz?=
 =?utf-8?B?ODh4RE9Db0YwcmsxZzBRNHdsaGt0TGtzSUpLQzZTSGhhMW1qMnFjaWZzVG5L?=
 =?utf-8?B?aGcySTkzaVZna0Exc3gzTkVTeVkwakl4ditIL2VYRldMVVI5NWVYTVVzSWxI?=
 =?utf-8?B?VURlc0lZaUxxSnBTNFV1dVBjLzFtV3Z6MkhGMDZPSjlyYWlOZnZuTFFUN09R?=
 =?utf-8?B?WExsOExxUGp1clNpNmRxbW9WMmhUcnpHU0x1LzJSMm9hNWFmWlEvdTZYL1Np?=
 =?utf-8?Q?jkNKXGE5fAENcyvai/g2CP77/ddXG0G/vmz2Tn9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1c3a53a2-a77a-43df-c71c-08d9784c7965
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:26:49.9786
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CvfVA7kYUob60b4OW2rkZk6C5mLHlr4hgDllW/ebIymETJ6FK/HBHpQqINK1Yl79tbh+grQlNZI+4ZkWUeWwYA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5327

There's no point going through %ax; the addition can be done directly in
%di.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/boot/mem.S
+++ b/xen/arch/x86/boot/mem.S
@@ -24,9 +24,7 @@ get_memory_map:
         cmpw    $E820_BIOS_MAX, bootsym(bios_e820nr) # up to this many entries
         jae     .Ldone
 
-        movw    %di,%ax
-        addw    $20,%ax
-        movw    %ax,%di
+        addw    $20,%di
         testl   %ebx,%ebx                       # check to see if
         jnz     1b                              # %ebx is set to EOF
 



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:38:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:38:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187659.336600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQV6z-0003E2-8Q; Wed, 15 Sep 2021 13:38:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187659.336600; Wed, 15 Sep 2021 13:38:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQV6z-0003Dv-4R; Wed, 15 Sep 2021 13:38:05 +0000
Received: by outflank-mailman (input) for mailman id 187659;
 Wed, 15 Sep 2021 13:38:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=h5Hw=OF=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mQV6y-0003Dp-5X
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:38:04 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7037a1e5-4023-4357-8c25-e3976b7ea55e;
 Wed, 15 Sep 2021 13:38:03 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id b18so4958555lfb.1
 for <xen-devel@lists.xenproject.org>; Wed, 15 Sep 2021 06:38:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7037a1e5-4023-4357-8c25-e3976b7ea55e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=90+qKy1YudQ2sg99JcqAjO4XWW79ok13oTy+0jzPIvI=;
        b=pWJY3kidUzGHksDFRIpgQnf4vUWN1vgPE+o3cbP4wXlnippo7ZiDFotxHaC8MSXaBM
         vNBrYetAgN2CPPdEzmkQctsPVj8A3iaBtpri3qDtPhHZimSdqoxIXBwHRJRDZ9vLa9+l
         6IOMYqcz7bpn5ONSY+QYxShXmQzDkPGD/T6/5/G7BXQucNCmtwZedFKEyok2o+3RtKMT
         VYwk4lsdzuny2NPeElvxKJrkxvufY7mcR/p4HOMPjET8Cod2htJk0E0BgXmrdajk70T7
         2Y4F/c8Epyv0cRuYA92V37bdauX1teip9YtlFtwxREkMZ3NCMvlGEQnEq7M+z4J3naFs
         roXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=90+qKy1YudQ2sg99JcqAjO4XWW79ok13oTy+0jzPIvI=;
        b=g19nSvOmfZ38m8sCNhx2sRMPGaFu76JP2V5AAssnxgnDinlPATn/WUYFaATvrAFMX+
         fK234wM2Qi83Jce01hhV9Or4l2HUPj8v5HpNAuqqBGqLESp5SM2/JN3hcQuiR6BE3K6b
         QyYsPnxXlP6vfHyLu9n0i1aNzfPZvVcrDakA7Rn+/ECFibWPPosblvcQbCvF7Myl1mlF
         1oZbidlJC/wp83do9Ocqcg/GbrYThDYdkJVRNeHZQMCMlX/QdVHT+6Rg+4whbT117Jeg
         CG53YSOyipLHdTlDKe9+8Fl07pl+q2u0iPe8wkNaV70BsTgpJWtviHIpbwEfweFVubRx
         P4Yg==
X-Gm-Message-State: AOAM530JrdwJVzlkDUiCJb9YCiauTzP1IwIOyrF0i6bsVZhuz/ntBSdq
	ySTV5lhH2pAu89wVQMrU31mKH3Mpq3BsJ5uzr20=
X-Google-Smtp-Source: ABdhPJzbP5i7b7U9v8l9tinl5uV6Cg0SkKCsv8bPOpIyik8V6mpJ9yfrA9+wOAMT6A3UvlVyAreBHNtPKbPaFKDmJo4=
X-Received: by 2002:a19:491a:: with SMTP id w26mr17767806lfa.588.1631713081917;
 Wed, 15 Sep 2021 06:38:01 -0700 (PDT)
MIME-Version: 1.0
References: <20210914151016.3174924-1-Roman_Skakun@epam.com> <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
In-Reply-To: <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
From: Roman Skakun <rm.skakun@gmail.com>
Date: Wed, 15 Sep 2021 16:37:50 +0300
Message-ID: <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrii Anisov <andrii_anisov@epam.com>, Roman Skakun <roman_skakun@epam.com>, 
	Jonathan Corbet <corbet@lwn.net>, Thomas Bogendoerfer <tsbogend@alpha.franken.de>, 
	Michael Ellerman <mpe@ellerman.id.au>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, 
	Paul Mackerras <paulus@samba.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Christoph Hellwig <hch@lst.de>, 
	Marek Szyprowski <m.szyprowski@samsung.com>, Robin Murphy <robin.murphy@arm.com>, 
	"Paul E. McKenney" <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>, 
	Andrew Morton <akpm@linux-foundation.org>, Thomas Gleixner <tglx@linutronix.de>, 
	"Maciej W. Rozycki" <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>, 
	Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra <peterz@infradead.org>, 
	Lu Baolu <baolu.lu@linux.intel.com>, Mike Rapoport <rppt@kernel.org>, 
	Will Deacon <will@kernel.org>, xen-devel@lists.xenproject.org, 
	linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, 
	linux-doc@vger.kernel.org, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>, 
	Roman Skakun <rm.skakun@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

Thanks for the answer.

>> From: Roman Skakun <roman_skakun@epam.com>
>>
>> It is possible when default IO TLB size is not
>> enough to fit a long buffers as described here [1].
>>
>> This patch makes a way to set this parameter
>> using cmdline instead of recompiling a kernel.
>>
>> [1] https://www.xilinx.com/support/answers/72694.html
>
>  I'm not convinced the swiotlb use describe there falls under "intended
>  use" - mapping a 1280x720 framebuffer in a single chunk?

I had the same issue while mapping DMA chuck ~4MB for gem fb when
using xen vdispl.
I got the next log:
[ 142.030421] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
3686400 bytes), total 32768 (slots), used 32 (slots)

It happened when I tried to map bounce buffer, which has a large size.
The default size if 128(IO_TLB_SEGSIZE) * 2048(IO_TLB_SHIFT) =3D 262144
bytes, but we requested 3686400 bytes.
When I change IO_TLB_SEGSIZE to 2048. (2048(IO_TLB_SEGSIZE)  *
2048(IO_TLB_SHIFT) =3D 4194304bytes).
It makes possible to retrieve a bounce buffer for requested size.
After changing this value, the problem is gone.

>  the bottom of this page is also confusing, as following "Then we can
>  confirm the modified swiotlb size in the boot log:" there is a log
>  fragment showing the same original size of 64Mb.

I suspect, this is a mistake in the article.
According to https://elixir.bootlin.com/linux/v5.14.4/source/kernel/dma/swi=
otlb.c#L214
and
https://elixir.bootlin.com/linux/v5.15-rc1/source/kernel/dma/swiotlb.c#L182
The IO_TLB_SEGSIZE is not used to calculate total size of swiotlb area.
This explains why we have the same total size before and after changing of
TLB segment size.

>  In order to be sure to catch all uses like this one (including ones
>  which make it upstream in parallel to yours), I think you will want
>  to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.

I don't understand your point. Can you clarify this?

>> +     /* get max IO TLB segment size */
>> +     if (isdigit(*str)) {
>> +             tmp =3D simple_strtoul(str, &str, 0);
>> +             if (tmp)
>> +                     io_tlb_seg_size =3D ALIGN(tmp, IO_TLB_SEGSIZE);
>
> From all I can tell io_tlb_seg_size wants to be a power of 2. Merely
> aligning to a multiple of IO_TLB_SEGSIZE isn't going to be enough.

Yes, right, thanks!

Cheers,
Roman.

=D0=B2=D1=82, 14 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 18:29, Jan B=
eulich <jbeulich@suse.com>:
>
> On 14.09.2021 17:10, Roman Skakun wrote:
> > From: Roman Skakun <roman_skakun@epam.com>
> >
> > It is possible when default IO TLB size is not
> > enough to fit a long buffers as described here [1].
> >
> > This patch makes a way to set this parameter
> > using cmdline instead of recompiling a kernel.
> >
> > [1] https://www.xilinx.com/support/answers/72694.html
>
> I'm not convinced the swiotlb use describe there falls under "intended
> use" - mapping a 1280x720 framebuffer in a single chunk? (As an aside,
> the bottom of this page is also confusing, as following "Then we can
> confirm the modified swiotlb size in the boot log:" there is a log
> fragment showing the same original size of 64Mb.
>
> > --- a/arch/mips/cavium-octeon/dma-octeon.c
> > +++ b/arch/mips/cavium-octeon/dma-octeon.c
> > @@ -237,7 +237,7 @@ void __init plat_swiotlb_setup(void)
> >               swiotlbsize =3D 64 * (1<<20);
> >  #endif
> >       swiotlb_nslabs =3D swiotlbsize >> IO_TLB_SHIFT;
> > -     swiotlb_nslabs =3D ALIGN(swiotlb_nslabs, IO_TLB_SEGSIZE);
> > +     swiotlb_nslabs =3D ALIGN(swiotlb_nslabs, swiotlb_io_seg_size());
>
> In order to be sure to catch all uses like this one (including ones
> which make it upstream in parallel to yours), I think you will want
> to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.
>
> > @@ -81,15 +86,30 @@ static unsigned int max_segment;
> >  static unsigned long default_nslabs =3D IO_TLB_DEFAULT_SIZE >> IO_TLB_=
SHIFT;
> >
> >  static int __init
> > -setup_io_tlb_npages(char *str)
> > +setup_io_tlb_params(char *str)
> >  {
> > +     unsigned long tmp;
> > +
> >       if (isdigit(*str)) {
> > -             /* avoid tail segment of size < IO_TLB_SEGSIZE */
> > -             default_nslabs =3D
> > -                     ALIGN(simple_strtoul(str, &str, 0), IO_TLB_SEGSIZ=
E);
> > +             default_nslabs =3D simple_strtoul(str, &str, 0);
> >       }
> >       if (*str =3D=3D ',')
> >               ++str;
> > +
> > +     /* get max IO TLB segment size */
> > +     if (isdigit(*str)) {
> > +             tmp =3D simple_strtoul(str, &str, 0);
> > +             if (tmp)
> > +                     io_tlb_seg_size =3D ALIGN(tmp, IO_TLB_SEGSIZE);
>
> From all I can tell io_tlb_seg_size wants to be a power of 2. Merely
> aligning to a multiple of IO_TLB_SEGSIZE isn't going to be enough.
>
> Jan
>


--=20
Best Regards, Roman.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:50:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:50:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187669.336611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVIb-0005TC-ET; Wed, 15 Sep 2021 13:50:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187669.336611; Wed, 15 Sep 2021 13:50:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVIb-0005SW-Ad; Wed, 15 Sep 2021 13:50:05 +0000
Received: by outflank-mailman (input) for mailman id 187669;
 Wed, 15 Sep 2021 13:50:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQVIZ-0005Bf-QJ
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:50:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d2eddce2-162b-11ec-b525-12813bfff9fa;
 Wed, 15 Sep 2021 13:50:02 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-WzxKjQ_FOyCp9ovxI31h5A-1; Wed, 15 Sep 2021 15:50:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5471.eurprd04.prod.outlook.com (2603:10a6:803:d0::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 13:49:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:49:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0117.eurprd04.prod.outlook.com (2603:10a6:208:55::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 13:49:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2eddce2-162b-11ec-b525-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631713801;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xRs3iwL2GN66k/6W48nYP2N7/4zO4rIJIIkNfYSSLuQ=;
	b=NW1bdbCVYhSjCIBgindKLrw8TGhsMXaLqJ0QkhLrug2lzli7KxSail3HXiYj42aoqFdmFo
	NXzeqtK7C8OmMZ+2APT0krvz8Sn2f4rFBlvwLGDzkKmO8XutDzXXrTFxN9HrCYha+oWw4m
	DPftBR+RF3LeH56BHGejaO3XqD0VWZs=
X-MC-Unique: WzxKjQ_FOyCp9ovxI31h5A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y8lt9p1Y+LIdUdIgr3fuIzarCZseiGSmYopU30G98+zDs5fp8kr+3twLEoIAcdiv+ka28ZTfxgcTHkGTMxrBqvYpYjx1MHs7jK3ZmZTos43BXkCSuPeRbHKh2hFmmSMFzZ4dcpkaQOJXH9jVzXT16OgriX01UKvToVcVfM15meWUvUDtVZwkNk+BV9q16Wx/BBKPGB8IuP6/phvRsLiyBAV+aXIoQhLnsDoKv/eFi76IuvpcBkb3T8nQsKZo3iDnpNgnHNpd7ZFkzLu5jAa/Mr5G8uzzgik7XKPtZTlxbfRAAq0ZAMUGs5fVSp/ccjgPN3tHo75an6DuYDu7IzkdhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xRs3iwL2GN66k/6W48nYP2N7/4zO4rIJIIkNfYSSLuQ=;
 b=hKb/3wEDzOgnV/lOG4ccZJk/VLhvDji9RO9J29oqaWtt7jrqqabQAy5xJwnxbFUpMIB/PpH8XcBDUc8c0sWqtAv4kWqbV3Fj8MIZn394lkPvohXzl9vJRq28Eukaa5/6KfH9EvPYWd1Ag2Vmu5LjYncxfwKiBmucvWmmPXry1a/fjVlvkt48AdY9yPMHJRmr3F4mYTroB/plskIYjsgYQzathTtAFRkL+pxT5MW5iCcgTbbTqpnHevAvkdMVf1y5N5uNZavsLIyv+xvY3h2y91LXfTi4D42g0TG9jVoVtPYM16l5Sn4cf7VA3GMQJeTVhR4cgVbX1U2pl33FT+WBfg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.linux-foundation.org; dkim=none (message not
 signed) header.d=none;lists.linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Roman Skakun <rm.skakun@gmail.com>
Cc: Andrii Anisov <andrii_anisov@epam.com>,
 Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>,
 Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
 Michael Ellerman <mpe@ellerman.id.au>,
 Benjamin Herrenschmidt <benh@kernel.crashing.org>,
 Paul Mackerras <paulus@samba.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Christoph Hellwig <hch@lst.de>, Marek Szyprowski <m.szyprowski@samsung.com>,
 Robin Murphy <robin.murphy@arm.com>, "Paul E. McKenney"
 <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>,
 Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, "Maciej W. Rozycki"
 <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>,
 Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra
 <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>,
 Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
 xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
 linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
 Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
 iommu <iommu@lists.linux-foundation.org>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
 <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
Date: Wed, 15 Sep 2021 15:49:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0117.eurprd04.prod.outlook.com
 (2603:10a6:208:55::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b0b5e282-5bbe-4cf8-0b58-08d9784fb384
X-MS-TrafficTypeDiagnostic: VI1PR04MB5471:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB547110D274360FD3C4EDBF3BB3DB9@VI1PR04MB5471.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JaXuFuBrcOfmQDdigdiyDseTXDqzxqZ7D+OcJ7e/NQ+4HjxSamTqC9C19d5ApqvcR/PDEdRLfdfhifzsV9l671gOEtQtTIEsAhDvnq/viB4PG6hhzTfB+Q9Ej1LW49ei01J9zaYHRpwtXByUqn5Pm3rgGgtNhMfAayaC2kHZeVbDz47zabRgCnp8KraclUERRxsfJ0RNkm/XfMij1ZmFTCMQeVAkGPokjTs0lVWfdcMVtfFTWDQg9Rj+fJRIeH81EhqfoxKN2euQJn7HgrCV4ZPWKWpRJuONkU2InfXd+PyiHJKEawjtzx7qOulUeqBaUkAosyES+N39BTZdJkySWJmHZD8r5rSMLbN9O1ouql+ZyfyAihRWE61PACdHUGXRbkmZMqzFyNx0QXHDiI1WrYm4bJ+Ex041VsHkg+ovDguBIlXLIBSfD29CIOCHnkZGPte7jq7n1tnxKTqSrJMcA5JGb3QpemVpHb+mLj+9vUbzFeZNZ/AnAXeqxh3j+arIy7QB6lOZIDkDgivSgoCJvPXHSgPSXWvSqmE6f5SQ66rx76a4pxTVxN+DPTcKCIkwSoU2tq1pQ2YsVABmQBBWk21/V2fr6Fv1IXcX1KE9ETjYPsFojGghKHVhuetvgUcO9pUMqX08hKTWYTrUle9V8/ZwV8i3Gac2fydBxqrZBYObxvXrfHiILcTc16+io0RhRZH5toDLEnwN5JT4di8ewmoKm5ALp+hhKR+33br6ZUrEn5chGDsUp8SUfEZ4Z+Rbj8qd1d8O3OvjapKWSyHry/1LrLPPQQ4sporNAztzyG3E5K2cCt5M1jL3qvKfIsz3
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(136003)(366004)(39860400002)(396003)(376002)(2616005)(53546011)(8936002)(38100700002)(956004)(31696002)(2906002)(5660300002)(7416002)(7406005)(16576012)(6916009)(6486002)(4326008)(31686004)(316002)(66476007)(966005)(66556008)(54906003)(478600001)(36756003)(26005)(66946007)(86362001)(8676002)(186003)(83380400001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TFRidnlrZWxiOFdDZ2lIcFMwTUR2cDVUSGVSbVJUdDlsYTN2elZ1UzRGaGFq?=
 =?utf-8?B?RFJYcGQ5NDFkMFcrSkdUd0J4K0IrVFVOZGZBVTlGYURMZitVbVoxOVR6Z3Vr?=
 =?utf-8?B?RlhNM0xZUWVESGt1WDRIK3QvSUl1Y2xKUEtuemo5bW1oZC9YQTNFVTF1bmpv?=
 =?utf-8?B?cDRGakh5clFrY05TYW5GV2ZNT3dXOGdrWjIyRXphQXY2bmtkZ3NzVitOT2ZC?=
 =?utf-8?B?ZDVtb05VQ2kwYjJJQ0VHcjk3dmY2Ymc4Z1dSVzk2K0lheDdPaGxubklneXRD?=
 =?utf-8?B?TUxXMkxEY3h2TmV6c2ZFS3RJNDhlb3hybWlhcmlkM25EaUdLQ1E3WmRMNmlE?=
 =?utf-8?B?ckFUSDdPdmtwRXkwZzJrazlQcnN5U0plWFRqMTh1NFMrS2h5MnZ6a0ZsOEw3?=
 =?utf-8?B?dHNuaml0dUxiaDA3M0JyNWQxTnhtWktNckhiekNuYTRkOHFMbGhLZTl1TzdJ?=
 =?utf-8?B?MzRNcko1WVpwWGhwRXVLbEZ0K2toL3dlLzcyTkZZbFMrNUZaSzJSMDRRU0Fj?=
 =?utf-8?B?R1ptWlJYZFZzY0JWbGZKUitQK2M4bUxlWDYxSk1pUkM2UitrQStlMUw0NUs5?=
 =?utf-8?B?dDBkNjViazlsbWIzOHAwMWhEVlp2OUE5TDlrU0lwdk9TblRZbDNqeFl0Zm43?=
 =?utf-8?B?TnVJSDh4ZWl1azVlTmswU1gyS0dUQkdQdFBZbjJ3bUZqMTF6bnh3SHU3QzNn?=
 =?utf-8?B?SnQrRC9aYjhvL0pKeEZnRVAzSlBsZGNvMXdoQjNVQW5KRDY5aWQrTllHOWhQ?=
 =?utf-8?B?Y2w2d216aTBsNnlqSFRXUFF2WWlEQXdJUmtBYmI3NnNnejhMT01RS2JXQ052?=
 =?utf-8?B?d1lZWGoxeXRxQVVOV3hqazd1MW5LZ2x4dExIQ2hDVW93RVZaeTJudDFBcFg4?=
 =?utf-8?B?UmNrMXZKaE9BaHgrZ3NPWEhjeVV2dFNkQmVUYUdsOC9BUUcxd2U2TVhORzJx?=
 =?utf-8?B?RE1tdWFmSkp5ZVJGejdBVXpMZlFiM2VnclN5MUY5SXd1a3E4eGRicWZ2dVRa?=
 =?utf-8?B?cmxtQUt4UVpOWlBRL25zSVdjWW5QUFR4cTgrNVkrS2pqK3dsN1QrODFyYi9t?=
 =?utf-8?B?NGZHd25KMjhkR0FscVlaaWNFMU5XaFhCSzlwbk5tV0pVVkFGRnpZK3VHTHIr?=
 =?utf-8?B?Tk54RjhyNHM4dkNDdEE4OWg3bUptNGozYlJXTVNoVWNLSnVKUWFCWXNQQzNr?=
 =?utf-8?B?REwxcXdDa2k4OFNsVHJXcmVkSVl5cmN5Vk9rQnQwYm9lVjg0elFiNFBUZjRl?=
 =?utf-8?B?Wk1CckNFQ2lNa0FNaUFHMkJFL2FuNjdKelpMVXkyTTA2OVlDamlrZjE4NEVK?=
 =?utf-8?B?WnE5Yms1ZE15VDhCZDU0cWR3MTBaaElKS2Y4SDV5UmJEMll6bzdKL0VZaG1Z?=
 =?utf-8?B?akVmWmlVUGhIYURZMmp3aW1Lb1dwRFVQRGV2c2FmT0UzWmN0RmxKcytWRmpl?=
 =?utf-8?B?NGpEQjVxVHFtT3FqVyt0bTFLTGthOHk2clBKMVRneUpwWm1ib2c2RWtTU3Vv?=
 =?utf-8?B?Q3FzWm9rd0tlL2s3Wm1oOGYrWFg2alJXWm9RSGFJRlpTSVljb2I0eVF3b1hQ?=
 =?utf-8?B?anVaSFBWQk5SREdYcUN6bnh2M3FqeVBsRWhNeGJrV3NaMElOeHZ6dlIyc25p?=
 =?utf-8?B?MjNlLzRaQno4cytSdFl6WmRVemtTNmpHODY3TG9DN3Y3cjhxRjBjbGtZeGtn?=
 =?utf-8?B?RGwrMUVOeTNHcmM5eUczNnhTZk9nSXR3MzNLekxYVkE0eG5RajdkWTVIcjNG?=
 =?utf-8?Q?ks2N/7BOJPGoHLXLwSSryflzFMuSIcoDBOJ1ug2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b0b5e282-5bbe-4cf8-0b58-08d9784fb384
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:49:56.1019
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Xsr9MvRkno86dox0iUTJAb/T9srjz1uVr6ApclDgSHv1xh0ZjtgtBgc5lzh5cLO4IQl05t2yiSrGZuxLrCHJgg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5471

On 15.09.2021 15:37, Roman Skakun wrote:
>>> From: Roman Skakun <roman_skakun@epam.com>
>>>
>>> It is possible when default IO TLB size is not
>>> enough to fit a long buffers as described here [1].
>>>
>>> This patch makes a way to set this parameter
>>> using cmdline instead of recompiling a kernel.
>>>
>>> [1] https://www.xilinx.com/support/answers/72694.html
>>
>>  I'm not convinced the swiotlb use describe there falls under "intended
>>  use" - mapping a 1280x720 framebuffer in a single chunk?
> 
> I had the same issue while mapping DMA chuck ~4MB for gem fb when
> using xen vdispl.
> I got the next log:
> [ 142.030421] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
> 3686400 bytes), total 32768 (slots), used 32 (slots)
> 
> It happened when I tried to map bounce buffer, which has a large size.
> The default size if 128(IO_TLB_SEGSIZE) * 2048(IO_TLB_SHIFT) = 262144
> bytes, but we requested 3686400 bytes.
> When I change IO_TLB_SEGSIZE to 2048. (2048(IO_TLB_SEGSIZE)  *
> 2048(IO_TLB_SHIFT) = 4194304bytes).
> It makes possible to retrieve a bounce buffer for requested size.
> After changing this value, the problem is gone.

But the question remains: Why does the framebuffer need to be mapped
in a single giant chunk?

>>  In order to be sure to catch all uses like this one (including ones
>>  which make it upstream in parallel to yours), I think you will want
>>  to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.
> 
> I don't understand your point. Can you clarify this?

There's a concrete present example: I have a patch pending adding
another use of IO_TLB_SEGSIZE. This use would need to be replaced
like you do here in several places. The need for the additional
replacement would be quite obvious (from a build failure) if you
renamed the manifest constant. Without renaming, it'll take
someone running into an issue on a live system, which I consider
far worse. This is because a simple re-basing of one of the
patches on top of the other will not point out the need for the
extra replacement, nor would a test build (with both patches in
place).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:53:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:53:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187677.336621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVLr-0006HV-Sf; Wed, 15 Sep 2021 13:53:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187677.336621; Wed, 15 Sep 2021 13:53:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVLr-0006HO-Pm; Wed, 15 Sep 2021 13:53:27 +0000
Received: by outflank-mailman (input) for mailman id 187677;
 Wed, 15 Sep 2021 13:53:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wsiD=OF=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mQVLq-0006HI-Ly
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:53:26 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 46135c73-c01a-49b5-a8d3-c099aa8769b2;
 Wed, 15 Sep 2021 13:53:24 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id AC35C67357; Wed, 15 Sep 2021 15:53:21 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46135c73-c01a-49b5-a8d3-c099aa8769b2
Date: Wed, 15 Sep 2021 15:53:21 +0200
From: Christoph Hellwig <hch@lst.de>
To: Jan Beulich <jbeulich@suse.com>
Cc: Roman Skakun <rm.skakun@gmail.com>,
	Andrii Anisov <andrii_anisov@epam.com>,
	Roman Skakun <roman_skakun@epam.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Christoph Hellwig <hch@lst.de>,
	Marek Szyprowski <m.szyprowski@samsung.com>,
	Robin Murphy <robin.murphy@arm.com>,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	"Maciej W. Rozycki" <macro@orcam.me.uk>,
	Muchun Song <songmuchun@bytedance.com>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
	xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	iommu <iommu@lists.linux-foundation.org>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
Message-ID: <20210915135321.GA15216@lst.de>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com> <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com> <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Wed, Sep 15, 2021 at 03:49:52PM +0200, Jan Beulich wrote:
> But the question remains: Why does the framebuffer need to be mapped
> in a single giant chunk?

More importantly: if you use dynamic dma mappings for your framebuffer
you're doing something wrong.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 13:56:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 13:56:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187685.336633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVOR-0006vH-Aw; Wed, 15 Sep 2021 13:56:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187685.336633; Wed, 15 Sep 2021 13:56:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVOR-0006vA-79; Wed, 15 Sep 2021 13:56:07 +0000
Received: by outflank-mailman (input) for mailman id 187685;
 Wed, 15 Sep 2021 13:56:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=aZgN=OF=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQVOQ-0006v4-44
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 13:56:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d6c37c10-42ae-4150-826b-f337866888b9;
 Wed, 15 Sep 2021 13:56:03 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-FjwYToJlO8q0yGYTmiC_qQ-1; Wed, 15 Sep 2021 15:56:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2336.eurprd04.prod.outlook.com (2603:10a6:800:27::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 13:55:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 13:55:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0043.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.16 via Frontend Transport; Wed, 15 Sep 2021 13:55:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6c37c10-42ae-4150-826b-f337866888b9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631714162;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oxt6RGP8crOhaUnAUT/V9CXgreRWKNIJv8oqL111RmU=;
	b=Y7H96rZF384FpzlN/PWgh401L/p9l+XdHeVKk54P6cesL4AP0YesMqXh+NoMN4wr0zlwLJ
	Z79WlHMsvlKxHXjhlVlB719GPHWw9N/5cqXx4/Nh4QjoYxx9+a9OAnqLA5RwlpY4BmjXWo
	wyBtQ7a5YYUHFL/eI+3T1d76tt7f1FA=
X-MC-Unique: FjwYToJlO8q0yGYTmiC_qQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=c7enu7DvG4AO1JXpuiF+6XqBM2+PFZTqGMD4kNQPCMszLi3wWRlSTY3FMhuGXIXskNYu+Gk25w3hQ7LTRCY+PViboN0DJ2zuUXSr4vDpLnqN8N5QQxtlZC+r7odBB0m4TKcmn2TzricbpQxGiW1S54Fw1h+5R3tAzIxSnWfGVeXkrBqpxmQjlTon69OO862rdAuoImvPq6ndfTXb/DISFyHHf8BnfTSojzT9p/jkIJ+9fhKONOLyjtTt/2kwuXi9PFFWhL/FFnfJQFTjaVTitjTbVNDse/YqLYSq10ZG7ixB0iLQVgAlgXFM82PDo9pXAQtWWU90nJxqFiLimu5rGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oxt6RGP8crOhaUnAUT/V9CXgreRWKNIJv8oqL111RmU=;
 b=WjO7/wH1PMQPtv2L+fu+AnDLCjDWb12P+qB1JK0IL7fyI5uyfZAe1lk8ZdnWt1qb3EpfN7Y743t+0FeNrI+MbTODIcT5r6923gWy9Z8towtm/yvukraIvlD+yXB/1q0RV/q3KGGwUSsS6uRRmHbo68s7sR67EjbYV7fX6yq8TOMvCMOZuUV48Fb2migh7lpCYgcvsBHpyJEzDuIVBZQtnF5aUxTdP+ioUuSd1zigv3ja9qZ76ALyCv9n7Mb+FeUwJldrCt2aCtDGsv267aL4tuZ7iPrcgJMBVC5pJ3C8Om7LqtdQvWQOE1zt77GwOrIQXUU+F7COWnZze7W3EpT1sQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v7 3/7] xen: introduce mark_page_free
To: Penny Zheng <penny.zheng@arm.com>
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com,
 xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org
References: <20210910025215.1671073-1-penny.zheng@arm.com>
 <20210910025215.1671073-4-penny.zheng@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <cd1ba586-0613-8e12-97b7-7b4cc2577bb9@suse.com>
Date: Wed, 15 Sep 2021 15:55:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910025215.1671073-4-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0043.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 492552e4-4474-400f-cce8-08d978508ba6
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2336:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2336EF6D53F0D7F3B7EEFDFBB3DB9@VI1PR0401MB2336.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uIxzLKN/UykCMKOCwyOxF+ugXif855lFBIGXwKiExEBXl94RLGm3MlzknjqQdXifib1tJqmcY34Scr4odxph3JhyakApZ4cT4hQW7tbdlKxiCAIFOcLtC1RFlB+kAOGw8CuwMnAK7khWYr0lWp1RyEr7VqpPdkT2ohyFd3cinmD4lkVu0Lj1KlT9J0es6hqDLDehG2/cWUaY76uAn2g3vAFw7HlCukQzI6nyKyi1//DkLufVX+kqML/dY/czudOO3RQrJaPxHblf6b28/fPJO+ZlH6CJupORqZzTEaw83WOZby8gAuRHWmIJrL/XaxgrDon6uxshuM2VGI4IFgyGTALu6MNHWymCEDmhCmN5mIQ5/N6HwBTnjjqUUQuEpKKvq91C4vbhPknQeh0KDasDz++Ljb+cYSRpwTvcj6PeC7ZoHUcMJvrYMPIIR5cNCpnFRm2+IjSBN6A8xG7v7z/ErknqsO57BbwjN4OgL61Wp35wSiWkTU6VW3aTL/avsK+8lTFjkscLWNEnTjzkVUAu1KaPJXVqz56w4BW97Vlt21U5Lny3I5sDyborJ2Yv3dBD+EfUU02acQuRycmPSwhM/WRfiJhEUSSFfAdwmXpp0X5zW87ihnQ4s63QzfYf/J+ZCFOIZqmPysHbGv/whUpx5Bqrd3k1c9Fz7D9I+1jnie3Q9v1rLlUh/mMTgKJOHQogb2WYurzVaBjp9Hl0a7EbyopZ9vO+tWMhv3gm8NgeipM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(346002)(376002)(366004)(39860400002)(4326008)(478600001)(86362001)(38100700002)(83380400001)(2906002)(2616005)(16576012)(31686004)(31696002)(316002)(36756003)(6486002)(6916009)(5660300002)(956004)(186003)(66556008)(66476007)(8936002)(66946007)(8676002)(53546011)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Qm5RTUlYNFU4WHBmaS9GSm1xQ2p2Z0FIbThLY2FtZUlKUEduNUMvYkpDcStG?=
 =?utf-8?B?SE1NVU5IM0VMYlZTQUlZcGl5bGU2VkZjZVZhRnJyb0krcXhLWEZvWGRvWkxh?=
 =?utf-8?B?WENGWnJ0anhxVDJQY3A3dkthdXVuVm1hVXkwV01sd0FpU3hEa2R5N3FHVjJq?=
 =?utf-8?B?REdkb0VvUy8xMGd5eW0wR0RkS2JsdGk2YWJUUXJyeVltYzlTVUZvNGlLY1hs?=
 =?utf-8?B?ZnVZYVNEaEpQT1pEK3B1WUw1azc1QnYyWTdoR2FwSnRXNFRBT09TTFRhWDVB?=
 =?utf-8?B?eG5HY20wN3NlUStPSDFIbm50c2F2RFZlcXY4VEwrb1VnNnp1TmdPdTVqNmE1?=
 =?utf-8?B?S2RUYzJwbjFZNzhlcFJkbDUwZFgwY1hJRytJQlRWUnVPbGhWbHQ0Vnd3YUtE?=
 =?utf-8?B?SGpmcU1YZURWdUszSDZCQ1FOU3JTSnZ0ejBDejdxblJOQVhMWHRNck11bUJn?=
 =?utf-8?B?TW5tQUhVejhXN2ptYXdPTStJdFRmNzRJN0dQRVJCWlhXTEZTa2tLQXNnV3N4?=
 =?utf-8?B?UjNqYjBTSXZkYXY3ZWpWQTYxcm9oZWtSY0hIRHU4Yk1DNFRYdkkrcGRlZVd4?=
 =?utf-8?B?QUFGU3ZQdlFWZ2pRMm1FR0FvMU04WXJraDBIQ3J6WDl4TEtqeVlNNVNQaHcw?=
 =?utf-8?B?aGs1RDU3bEtHdElSTXY5VkptQ01vYVhyb2hja2Yxa2lJYm13M2J2U0RmUDdZ?=
 =?utf-8?B?WWc3RW1FV1B6ZzdvWS9qVDljN09sczdBZ0FhVFh2aGV0ZDJxZFZjTC81aUg2?=
 =?utf-8?B?bjJ5NGR5V2taL2xGQVVxTy9heXJIUFNLR2h1WVZqempuLzZ4ZGYvTHowak1R?=
 =?utf-8?B?OXY5ODFtM2lOVExqQ21nS3FLWHkzSmE2Vm5XdnA2dmsyb1puK3pNUGFBZG0y?=
 =?utf-8?B?R3cxL3IxajF6a0dBbzByTW9vWTdnT2hhUldhTWpFZng1WlRYbnNKVGY4ZmU3?=
 =?utf-8?B?dWYwNTNBRTQwVU8zSU5EZ0RzQ3orT3dJL1NRUms2VVNEWmlNeGozRXV2Z2J0?=
 =?utf-8?B?SW9NQkM0Yk04UW5iME9seUVRbUZZOVp0aWd4TkRCUUxDVWtMNEFORUdkU2dW?=
 =?utf-8?B?VkQ0NHB5MFdDM0Jkc21ZSEJtUkhacHZ4YllMekFtWWNad1FXeDFuUlNkQjJo?=
 =?utf-8?B?VnIwVllXdk5jMXJLZU5OWHJUSm1pcEtURk82bFNFWGgzcmkxdWZWOFRkUWJn?=
 =?utf-8?B?NlpxeFRhdloxbFYxMVBLVHA5Z3MxU1pqL3dVK2NJMXlhR1cvWGsyQnRCTWR4?=
 =?utf-8?B?ZzJKeVQ0NFIxUFZTMllTdU9CTnBxZEdnMlBmSmVjbFhDYmhsRldJZDRNOXRt?=
 =?utf-8?B?a2FnVExqOU5ZRDdwWjRLUVF3M1FJM1R2S2RnQUJXWGR2UVFsdmJTd25wckVI?=
 =?utf-8?B?bS92WHFwbWtkWTVQbFJyc1hOaVVJQm0xcDdFL1hKdCttaVVyVlF5aWZqN3Jr?=
 =?utf-8?B?cWYvSldRM0FQNi9ZUDd1cm44TGlRdW9CU1IrVStIUElqWUdDR0U1SW9waVZt?=
 =?utf-8?B?Rm5lL1E4eitTb3RRL0VpVXZhdDJmSVRvU2tMN0cva3hVOEJnVk93d0gvM3Vy?=
 =?utf-8?B?OG5vaml1cmZwdXZpcDVURzU0eU9palcrSEhSdXJWQkhsMERkS0o2b1ZwVlpH?=
 =?utf-8?B?N2VnR0VLVUlvUmNvV2t5VFBsNnI3dmJCZ0YzMFkxSVZzcDZMcEdlSit1dyt3?=
 =?utf-8?B?S0pWTEgwWGJDWmUrN1JCQXg5N24vY2JEMG53bitsZTRGZmRxQzJ3NFRyZVNH?=
 =?utf-8?Q?uSJQj29/iXJaYColmHBhgtiIzv/jdP36ru+vAjs?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 492552e4-4474-400f-cce8-08d978508ba6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 13:55:58.6126
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Ki/LMNJEqlq9shejZV46H4BfvI/+/2Q7jF8CaY1BZgObbWdhsFoUfsJChYr3+Eoe0lfhb/oeN/OjqJCU139CIg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2336

On 10.09.2021 04:52, Penny Zheng wrote:
> This commit defines a new helper mark_page_free to extract common code,
> like following the same cache/TLB coherency policy, between free_heap_pages
> and the new function free_staticmem_pages, which will be introduced later.
> 
> The PDX compression makes that conversion between the MFN and the page can
> be potentially non-trivial. As the function is internal, pass the MFN and
> the page. They are both expected to match.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Reviewed-by: Julien Grall <jgrall@amazon.com>
> ---
>  xen/common/page_alloc.c | 89 ++++++++++++++++++++++-------------------
>  1 file changed, 48 insertions(+), 41 deletions(-)
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 958ba0cd92..a3ee5eca9e 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1376,6 +1376,53 @@ bool scrub_free_pages(void)
>      return node_to_scrub(false) != NUMA_NO_NODE;
>  }
>  
> +static void mark_page_free(struct page_info *pg, mfn_t mfn)
> +{
> +    ASSERT(mfn_x(mfn) == mfn_x(page_to_mfn(pg)));
> +
> +    /*
> +     * Cannot assume that count_info == 0, as there are some corner cases
> +     * where it isn't the case and yet it isn't a bug:
> +     *  1. page_get_owner() is NULL
> +     *  2. page_get_owner() is a domain that was never accessible by
> +     *     its domid (e.g., failed to fully construct the domain).
> +     *  3. page was never addressable by the guest (e.g., it's an
> +     *     auto-translate-physmap guest and the page was never included
> +     *     in its pseudophysical address space).
> +     * In all the above cases there can be no guest mappings of this page.
> +     */
> +    switch ( pg->count_info & PGC_state )
> +    {
> +    case PGC_state_inuse:
> +        BUG_ON(pg->count_info & PGC_broken);
> +        pg->count_info = PGC_state_free;
> +        break;
> +
> +    case PGC_state_offlining:
> +        pg->count_info = (pg->count_info & PGC_broken) |
> +                         PGC_state_offlined;
> +        tainted = 1;

You've broken two things at the same time here: You write to the global
variable of this name now, while ...

> @@ -1392,47 +1439,7 @@ static void free_heap_pages(
>  
>      for ( i = 0; i < (1 << order); i++ )
>      {
> -        /*
> -         * Cannot assume that count_info == 0, as there are some corner cases
> -         * where it isn't the case and yet it isn't a bug:
> -         *  1. page_get_owner() is NULL
> -         *  2. page_get_owner() is a domain that was never accessible by
> -         *     its domid (e.g., failed to fully construct the domain).
> -         *  3. page was never addressable by the guest (e.g., it's an
> -         *     auto-translate-physmap guest and the page was never included
> -         *     in its pseudophysical address space).
> -         * In all the above cases there can be no guest mappings of this page.
> -         */
> -        switch ( pg[i].count_info & PGC_state )
> -        {
> -        case PGC_state_inuse:
> -            BUG_ON(pg[i].count_info & PGC_broken);
> -            pg[i].count_info = PGC_state_free;
> -            break;
> -
> -        case PGC_state_offlining:
> -            pg[i].count_info = (pg[i].count_info & PGC_broken) |
> -                               PGC_state_offlined;
> -            tainted = 1;

... the local variable here doesn't get written anymore. Which Coverity
was kind enough to point out - please reference Coverity ID 1491872 in
the fix that I hope you will be able to provide quickly. (The easiest
change would seem to be to make mark_page_free() return bool, and set
"tainted" to 1 here accordingly.)

I understand that the two variables having the same name isn't very
helpful. I certainly wouldn't mind if you renamed the local one
suitably.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:08:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:08:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187694.336648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVaN-0000L4-Ez; Wed, 15 Sep 2021 14:08:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187694.336648; Wed, 15 Sep 2021 14:08:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVaN-0000Kx-B3; Wed, 15 Sep 2021 14:08:27 +0000
Received: by outflank-mailman (input) for mailman id 187694;
 Wed, 15 Sep 2021 14:08:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XwCH=OF=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQVaL-0000IR-Lh
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:08:25 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 99d7e694-6503-4ba8-85ca-645479a1cc2d;
 Wed, 15 Sep 2021 14:08:18 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 062BC2217F;
 Wed, 15 Sep 2021 14:08:17 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CC95213C37;
 Wed, 15 Sep 2021 14:08:16 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 3i++MFD+QWHsdgAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 15 Sep 2021 14:08:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99d7e694-6503-4ba8-85ca-645479a1cc2d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631714897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Xozn4dABy+P/hAZ8HNwNB0Im8TWq1cxoQG3rmcU2AYY=;
	b=d6rTs/CCIY0Ek/83+mMi5ojBDEp7c1YgI/Qh6fHn1xc6nJ8n7tJEfOozvFsgD0hNqV9RN5
	KSaAhzHzJHH/pq0I+0GtQ5av0qDqHf0ztMyym2xDDtBylUpyI92sx+7JFvvPWHZhRDF7Lj
	lUEMTf4esXwv3rYl3QR8yygg9P3FVJ8=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3] xenbus: support large messages
Date: Wed, 15 Sep 2021 16:08:15 +0200
Message-Id: <20210915140815.26736-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Today the implementation of the xenbus protocol in Mini-OS will only
allow to transfer the complete message to or from the ring page buffer.
This is limiting the maximum message size to lower values as the xenbus
protocol normally would allow.

Change that by allowing to transfer the xenbus message in chunks as
soon as they are available.

Avoid crashing Mini-OS in case of illegal data read from the ring
buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- drop redundant if (Samuel Thibault)
- move rmb() (Samuel Thibault)
V3:
- correct notification test (Samuel Thibault)
---
 xenbus/xenbus.c | 209 ++++++++++++++++++++++++++++--------------------
 1 file changed, 121 insertions(+), 88 deletions(-)

diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
index 23de61e..c8d358d 100644
--- a/xenbus/xenbus.c
+++ b/xenbus/xenbus.c
@@ -29,6 +29,7 @@
 #include <xen/hvm/params.h>
 #include <mini-os/spinlock.h>
 #include <mini-os/xmalloc.h>
+#include <mini-os/semaphore.h>
 
 #define min(x,y) ({                       \
         typeof(x) tmpx = (x);                 \
@@ -46,6 +47,7 @@
 static struct xenstore_domain_interface *xenstore_buf;
 static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
 DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
+static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
 
 xenbus_event_queue xenbus_events;
 static struct watch {
@@ -231,75 +233,102 @@ char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_
 }
 
 
+static void xenbus_read_data(char *buf, unsigned int len)
+{
+    unsigned int off = 0;
+    unsigned int prod, cons;
+    unsigned int size;
+
+    while (off != len)
+    {
+        wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons);
+
+        prod = xenstore_buf->rsp_prod;
+        cons = xenstore_buf->rsp_cons;
+        DEBUG("Rsp_cons %d, rsp_prod %d.\n", cons, prod);
+        size = min(len - off, prod - cons);
+
+        rmb();   /* Make sure data read from ring is ordered with rsp_prod. */
+        memcpy_from_ring(xenstore_buf->rsp, buf + off,
+                         MASK_XENSTORE_IDX(cons), size);
+        off += size;
+        xenstore_buf->rsp_cons += size;
+        wmb();
+        if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE)
+            notify_remote_via_evtchn(xenbus_evtchn);
+    }
+}
+
 static void xenbus_thread_func(void *ign)
 {
     struct xsd_sockmsg msg;
-    unsigned prod = xenstore_buf->rsp_prod;
+    char *data;
 
     for (;;) {
-        wait_event(xb_waitq, prod != xenstore_buf->rsp_prod);
-        while (1) {
-            prod = xenstore_buf->rsp_prod;
-            DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons,
-                  xenstore_buf->rsp_prod);
-            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < sizeof(msg))
-                break;
-            rmb();
-            memcpy_from_ring(xenstore_buf->rsp, &msg,
-                             MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                             sizeof(msg));
-            DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),
-                  xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id);
-
-            if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons <
-                sizeof(msg) + msg.len)
-                break;
-
-            DEBUG("Message is good.\n");
-
-            if (msg.type == XS_WATCH_EVENT) {
-                struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
-                xenbus_event_queue *events = NULL;
-                char *data = (char*)event + sizeof(*event);
-                struct watch *watch;
-
-                memcpy_from_ring(xenstore_buf->rsp, data,
-                    MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(msg)),
-                    msg.len);
-
-                event->path = data;
-                event->token = event->path + strlen(event->path) + 1;
-
-                mb();
-                xenstore_buf->rsp_cons += msg.len + sizeof(msg);
-
-                for (watch = watches; watch; watch = watch->next)
-                    if (!strcmp(watch->token, event->token)) {
-                        events = watch->events;
-                        break;
-                    }
-
-                if (events) {
-                    event->next = *events;
-                    *events = event;
-                    wake_up(&xenbus_watch_queue);
-                } else {
-                    printk("unexpected watch token %s\n", event->token);
-                    free(event);
+        xenbus_read_data((char *)&msg, sizeof(msg));
+        DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),
+              xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id);
+
+        if (msg.len > XENSTORE_PAYLOAD_MAX) {
+            printk("Xenstore violates protocol, message longer than allowed.\n");
+            return;
+        }
+
+        if (msg.type == XS_WATCH_EVENT) {
+            struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
+            xenbus_event_queue *events = NULL;
+            struct watch *watch;
+            char *c;
+            int zeroes = 0;
+
+            data = (char*)event + sizeof(*event);
+            xenbus_read_data(data, msg.len);
+
+            for (c = data; c < data + msg.len; c++)
+                if (!*c)
+                    zeroes++;
+            if (zeroes != 2) {
+                printk("Xenstore: illegal watch event data\n");
+                free(event);
+                continue;
+            }
+
+            event->path = data;
+            event->token = event->path + strlen(event->path) + 1;
+
+            for (watch = watches; watch; watch = watch->next)
+                if (!strcmp(watch->token, event->token)) {
+                    events = watch->events;
+                    break;
                 }
+
+            if (events) {
+                event->next = *events;
+                *events = event;
+                wake_up(&xenbus_watch_queue);
             } else {
-                req_info[msg.req_id].reply = malloc(sizeof(msg) + msg.len);
-                memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_id].reply,
-                                 MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                                 msg.len + sizeof(msg));
-                mb();
-                xenstore_buf->rsp_cons += msg.len + sizeof(msg);
-                wake_up(&req_info[msg.req_id].waitq);
+                printk("Xenstore: unexpected watch token %s\n", event->token);
+                free(event);
             }
 
-            wmb();
-            notify_remote_via_evtchn(xenbus_evtchn);
+            continue;
         }
+
+        data = malloc(sizeof(msg) + msg.len);
+        memcpy(data, &msg, sizeof(msg));
+        xenbus_read_data(data + sizeof(msg), msg.len);
+
+        if (msg.req_id >= NR_REQS || !req_info[msg.req_id].in_use) {
+            printk("Xenstore: illegal request id %d\n", msg.req_id);
+            free(data);
+            continue;
+        }
+
+        DEBUG("Message is good.\n");
+
+        req_info[msg.req_id].reply = data;
+
+        wake_up(&req_info[msg.req_id].waitq);
     }
 }
 
@@ -451,36 +480,40 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id,
 
     cur_req = &header_req;
 
-    BUG_ON(len > XENSTORE_RING_SIZE);
-    /* Wait for the ring to drain to the point where we can send the
-       message. */
-    prod = xenstore_buf->req_prod;
-    if (prod + len - xenstore_buf->req_cons > XENSTORE_RING_SIZE) 
-    {
-        /* Wait for there to be space on the ring */
-        DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n",
-                prod, len, xenstore_buf->req_cons, XENSTORE_RING_SIZE);
-        wait_event(xb_waitq,
-                xenstore_buf->req_prod + len - xenstore_buf->req_cons <=
-                XENSTORE_RING_SIZE);
-        DEBUG("Back from wait.\n");
-        prod = xenstore_buf->req_prod;
-    }
+    BUG_ON(len > XENSTORE_PAYLOAD_MAX);
+
+    /* Make sure we are the only thread trying to write. */
+    down(&xb_write_sem);
 
-    /* We're now guaranteed to be able to send the message without
-       overflowing the ring.  Do so. */
+    /* Send the message in chunks using free ring space when available. */
     total_off = 0;
     req_off = 0;
-    while (total_off < len) 
+    while (total_off < len)
     {
+        prod = xenstore_buf->req_prod;
+        if (prod - xenstore_buf->req_cons >= XENSTORE_RING_SIZE)
+        {
+            /* Send evtchn to notify remote */
+            notify_remote_via_evtchn(xenbus_evtchn);
+
+            /* Wait for there to be space on the ring */
+            DEBUG("prod %d, len %d, cons %d, size %d; waiting.\n", prod,
+                  len - total_off, xenstore_buf->req_cons, XENSTORE_RING_SIZE);
+            wait_event(xb_waitq,
+                       prod - xenstore_buf->req_cons < XENSTORE_RING_SIZE);
+            DEBUG("Back from wait.\n");
+        }
+
         this_chunk = min(cur_req->len - req_off,
-                XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod));
+                         XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(prod));
+        this_chunk = min(this_chunk,
+                         xenstore_buf->req_cons + XENSTORE_RING_SIZE - prod);
         memcpy((char *)xenstore_buf->req + MASK_XENSTORE_IDX(prod),
-                (char *)cur_req->data + req_off, this_chunk);
+               (char *)cur_req->data + req_off, this_chunk);
         prod += this_chunk;
         req_off += this_chunk;
         total_off += this_chunk;
-        if (req_off == cur_req->len) 
+        if (req_off == cur_req->len)
         {
             req_off = 0;
             if (cur_req == &header_req)
@@ -488,20 +521,20 @@ static void xb_write(int type, int req_id, xenbus_transaction_t trans_id,
             else
                 cur_req++;
         }
+
+        /* Remote must see entire message before updating indexes */
+        wmb();
+        xenstore_buf->req_prod = prod;
     }
 
+    /* Send evtchn to notify remote */
+    notify_remote_via_evtchn(xenbus_evtchn);
+
     DEBUG("Complete main loop of xb_write.\n");
     BUG_ON(req_off != 0);
     BUG_ON(total_off != len);
-    BUG_ON(prod > xenstore_buf->req_cons + XENSTORE_RING_SIZE);
 
-    /* Remote must see entire message before updating indexes */
-    wmb();
-
-    xenstore_buf->req_prod += len;
-
-    /* Send evtchn to notify remote */
-    notify_remote_via_evtchn(xenbus_evtchn);
+    up(&xb_write_sem);
 }
 
 /* Send a mesasge to xenbus, in the same fashion as xb_write, and
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:26:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:26:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187705.336681 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrk-0003MR-Jl; Wed, 15 Sep 2021 14:26:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187705.336681; Wed, 15 Sep 2021 14:26:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrk-0003MK-GV; Wed, 15 Sep 2021 14:26:24 +0000
Received: by outflank-mailman (input) for mailman id 187705;
 Wed, 15 Sep 2021 14:26:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7Uy1=OF=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mQVrj-0002oh-1A
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:26:23 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e433f3d8-1630-11ec-b535-12813bfff9fa;
 Wed, 15 Sep 2021 14:26:18 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 479AE6D;
 Wed, 15 Sep 2021 07:26:18 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B6D083F719;
 Wed, 15 Sep 2021 07:26:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e433f3d8-1630-11ec-b535-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI boot
Date: Wed, 15 Sep 2021 15:26:02 +0100
Message-Id: <20210915142602.42862-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210915142602.42862-1-luca.fancellu@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>

This patch introduces the support for dom0less configuration
when using UEFI boot on ARM, it permits the EFI boot to
continue if no dom0 kernel is specified but at least one domU
is found.

Introduce the new property "uefi,binary" for device tree boot
module nodes that are subnode of "xen,domain" compatible nodes.
The property holds a string containing the file name of the
binary that shall be loaded by the uefi loader from the filesystem.

Update efi documentation about how to start a dom0less
setup using UEFI

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
 docs/misc/efi.pandoc        |  37 ++++++
 xen/arch/arm/efi/efi-boot.h | 244 +++++++++++++++++++++++++++++++++++-
 xen/common/efi/boot.c       |  20 ++-
 3 files changed, 294 insertions(+), 7 deletions(-)

diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ac3cd58cae..db9b3273f8 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -165,3 +165,40 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+
+## UEFI boot and dom0less on ARM
+
+Dom0less feature is supported by ARM and it is possible to use it when Xen is
+started as an EFI application.
+The way to specify the domU domains is by Device Tree as specified in the
+[dom0less](dom0less.html) documentation page under the "Device Tree
+configuration" section, but instead of declaring the reg property in the boot
+module, the user must specify the "uefi,binary" property containing the name
+of the binary file that has to be loaded in memory.
+The UEFI stub will load the binary in memory and it will add the reg property
+accordingly.
+
+An example here:
+
+    domU1 {
+        #address-cells = <1>;
+        #size-cells = <1>;
+        compatible = "xen,domain";
+        memory = <0 0x20000>;
+        cpus = <1>;
+        vpl011;
+
+        module@1 {
+            compatible = "multiboot,kernel", "multiboot,module";
+            uefi,binary = "vmlinuz-3.0.31-0.4-xen";
+            bootargs = "console=ttyAMA0";
+        };
+        module@2 {
+            compatible = "multiboot,ramdisk", "multiboot,module";
+            uefi,binary = "initrd-3.0.31-0.4-xen";
+        };
+        module@3 {
+            compatible = "multiboot,ramdisk", "multiboot,module";
+            uefi,binary = "passthrough.dtb";
+        };
+    };
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 5ff626c6a0..8d7ced70f2 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -8,9 +8,39 @@
 #include <asm/setup.h>
 #include <asm/smp.h>
 
+typedef struct {
+    char* name;
+    int name_len;
+} dom0less_module_name;
+
+/*
+ * Binaries will be translated into bootmodules, the maximum number for them is
+ * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
+ */
+#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
+static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
+static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MODULES];
+static uint32_t __initdata dom0less_modules_available = MAX_DOM0LESS_MODULES;
+static uint32_t __initdata dom0less_modules_idx = 0;
+
+#define ERROR_DOM0LESS_FILE_NOT_FOUND -1
+
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
 
+static int __init get_dom0less_file_index(const char* name, int name_len);
+static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
+                                              const char* name, int name_len);
+static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
+                                               int module_node_offset,
+                                               int reg_addr_cells,
+                                               int reg_size_cells);
+static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                               int domain_node,
+                                               int addr_cells,
+                                               int size_cells);
+static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);
+
 #define DEVICE_TREE_GUID \
 {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
 
@@ -552,8 +582,209 @@ static void __init efi_arch_handle_module(const struct file *file,
                          kernel.size) < 0 )
             blexit(L"Unable to set reg property.");
     }
-    else
+    else if ( !((file >= &dom0less_files[0]) &&
+               (file <= &dom0less_files[MAX_DOM0LESS_MODULES-1])) )
+        /*
+         * If file is not a dom0 module file and it's not any domU modules,
+         * stop here.
+         */
         blexit(L"Unknown module type");
+
+    /*
+     * dom0less_modules_available is decremented here because for each dom0
+     * file added, there will be an additional bootmodule, so the number
+     * of dom0less module files will be decremented because there is
+     * a maximum amount of bootmodules that can be loaded.
+     */
+    dom0less_modules_available--;
+}
+
+/*
+ * This function checks for a binary previously loaded with a give name, it
+ * returns the index of the file in the dom0less_files array or a negative
+ * number if no file with that name is found.
+ */
+static int __init get_dom0less_file_index(const char* name, int name_len)
+{
+    int ret = ERROR_DOM0LESS_FILE_NOT_FOUND;
+
+    for (uint32_t i = 0; i < dom0less_modules_idx; i++)
+    {
+        dom0less_module_name* mod = &dom0less_bin_names[i];
+        if ( (mod->name_len == name_len) &&
+             (strncmp(mod->name, name, name_len) == 0) )
+        {
+            ret = i;
+            break;
+        }
+    }
+    return ret;
+}
+
+/*
+ * This function allocates a binary and keeps track of its name, it
+ * returns the index of the file in the dom0less_files array.
+ */
+static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
+                                              const char* name, int name_len)
+{
+    dom0less_module_name* file_name;
+    union string module_name;
+    struct file* file;
+    uint32_t ret_idx;
+
+    /*
+     * Check if there is any space left for a domU module, the variable
+     * dom0less_modules_available is updated each time we use read_file(...)
+     * successfully.
+     */
+    if ( !dom0less_modules_available )
+        blexit(L"No space left for domU modules");
+
+    module_name.s = (char*) name;
+    ret_idx = dom0less_modules_idx;
+    file = &dom0less_files[ret_idx];
+
+    /* Save at this index the name of this binary */
+    file_name = &dom0less_bin_names[ret_idx];
+
+    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
+                              (void**)&file_name->name) != EFI_SUCCESS )
+        blexit(L"Error allocating memory for dom0less binary name");
+
+    /* Save name and length of the binary in the data structure */
+    strlcpy(file_name->name, name, name_len);
+    file_name->name_len = name_len;
+
+    /* Load the binary in memory */
+    read_file(dir_handle, s2w(&module_name), file, NULL);
+
+    /* s2w(...) allocates some memory, free it */
+    efi_bs->FreePool(module_name.w);
+
+    dom0less_modules_idx++;
+
+    return ret_idx;
+}
+
+/*
+ * This function checks for the presence of the uefi,binary property in the
+ * module, if found it loads the binary as dom0less module and sets the right
+ * address for the reg property into the module DT node.
+ */
+static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
+                                          int module_node_offset,
+                                          int reg_addr_cells,
+                                          int reg_size_cells)
+{
+    const void* uefi_name_prop;
+    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
+    int uefi_name_len, file_idx;
+    struct file* file;
+
+    /* Read uefi,binary property to get the file name. */
+    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
+                                 &uefi_name_len);
+
+    if ( NULL == uefi_name_prop )
+        /* Property not found */
+        return;
+
+    file_idx = get_dom0less_file_index(uefi_name_prop, uefi_name_len);
+    if (file_idx < 0)
+        file_idx = allocate_dom0less_file(dir_handle, uefi_name_prop,
+                                          uefi_name_len);
+
+    file = &dom0less_files[file_idx];
+
+    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
+
+    /* Rename the module to be module@{address} */
+    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
+        blexit(L"Unable to add domU ramdisk FDT node.");
+
+    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
+                     file->addr, file->size) < 0 )
+        blexit(L"Unable to set reg property.");
+}
+
+/*
+ * This function checks for boot modules under the domU guest domain node
+ * in the DT.
+ */
+static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                               int domain_node,
+                                               int addr_cells,
+                                               int size_cells)
+{
+    /*
+     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
+     * inside this node
+     */
+    for ( int module_node = fdt_first_subnode(fdt, domain_node);
+          module_node > 0;
+          module_node = fdt_next_subnode(fdt, module_node) )
+    {
+        if ( (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,kernel") == 0) ||
+             (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,ramdisk") == 0) ||
+             (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,device-tree") == 0) )
+        {
+            /* The compatible is one of the strings above, check the module */
+            handle_dom0less_module_node(dir_handle, module_node, addr_cells,
+                                        size_cells);
+        }
+    }
+}
+
+/*
+ * This function checks for xen domain nodes under the /chosen node for possible
+ * domU guests to be loaded.
+ */
+static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle)
+{
+    int chosen;
+    int addr_len, size_len;
+
+    /* Check for the chosen node in the current DTB */
+    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
+    if ( chosen < 0 )
+        blexit(L"Unable to setup chosen node");
+
+    /* Check for nodes compatible with xen,domain under the chosen node */
+    for ( int node = fdt_first_subnode(fdt, chosen);
+          node > 0;
+          node = fdt_next_subnode(fdt, node) )
+    {
+        int addr_cells, size_cells, len;
+        const struct fdt_property *prop;
+
+        if ( fdt_node_check_compatible(fdt, node, "xen,domain") != 0 )
+            continue;
+
+        /* Get or set #address-cells and #size-cells */
+        prop = fdt_get_property(fdt, node, "#address-cells", &len);
+        if ( !prop )
+            blexit(L"#address-cells not found in domain node.");
+
+        addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+        prop = fdt_get_property(fdt, node, "#size-cells", &len);
+        if ( !prop )
+            blexit(L"#size-cells not found in domain node.");
+
+        size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+        /* Found a node with compatible xen,domain; handle this node. */
+        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_cells);
+    }
+
+    if ( dom0less_modules_idx > 0 )
+        return true;
+
+    return false;
 }
 
 static void __init efi_arch_cpu(void)
@@ -562,8 +793,19 @@ static void __init efi_arch_cpu(void)
 
 static void __init efi_arch_blexit(void)
 {
+    uint32_t i = 0;
     if ( dtbfile.need_to_free )
         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
+    /* Free dom0less files if any */
+    for ( ; i < dom0less_modules_idx; i++ )
+    {
+        /* Free dom0less binary names */
+        efi_bs->FreePool(dom0less_bin_names[i].name);
+        /* Free dom0less binaries */
+        if ( dom0less_files[i].need_to_free )
+            efi_bs->FreePages(dom0less_files[i].addr,
+                              PFN_UP(dom0less_files[i].size));
+    }
     if ( memmap )
         efi_bs->FreePool(memmap);
 }
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 758f9d74d2..65493c4b46 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
     union string section = { NULL }, name;
     bool base_video = false;
-    const char *option_str;
+    const char *option_str = NULL;
     bool use_cfg_file;
+    bool dom0less_found = false;
 
     __set_bit(EFI_BOOT, &efi_flags);
     __set_bit(EFI_LOADER, &efi_flags);
@@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
             efi_bs->FreePool(name.w);
         }
 
-        if ( !name.s )
-            blexit(L"No Dom0 kernel image specified.");
-
         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
 
-        option_str = split_string(name.s);
+#ifdef CONFIG_ARM
+        /* dom0less feature is supported only on ARM */
+        dom0less_found = check_dom0less_efi_boot(dir_handle);
+#endif
+
+        if ( !name.s && !dom0less_found )
+            blexit(L"No Dom0 kernel image specified.");
+
+        if ( name.s != NULL )
+            option_str = split_string(name.s);
 
-        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
+        if ( (!read_section(loaded_image, L"kernel", &kernel, option_str)) &&
+             (name.s != NULL) )
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:26:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:26:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187704.336663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrg-0002ri-A8; Wed, 15 Sep 2021 14:26:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187704.336663; Wed, 15 Sep 2021 14:26:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrg-0002r8-6w; Wed, 15 Sep 2021 14:26:20 +0000
Received: by outflank-mailman (input) for mailman id 187704;
 Wed, 15 Sep 2021 14:26:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7Uy1=OF=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mQVre-0002oh-5f
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:26:18 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e3319440-1630-11ec-b535-12813bfff9fa;
 Wed, 15 Sep 2021 14:26:16 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 839F71042;
 Wed, 15 Sep 2021 07:26:16 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9D5A43F719;
 Wed, 15 Sep 2021 07:26:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3319440-1630-11ec-b535-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH 1/2] xen/efi: Restrict check for DT boot modules on EFI boot
Date: Wed, 15 Sep 2021 15:26:01 +0100
Message-Id: <20210915142602.42862-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210915142602.42862-1-luca.fancellu@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>

When Xen is started as EFI application, it is checking
the presence of multiboot,module in the DT, if any is
found, the configuration file is skipped.
Restrict this check to just any multiboot,module that
is a direct child of the /chosen node.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
 xen/arch/arm/efi/efi-boot.h | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index cf9c37153f..5ff626c6a0 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -581,6 +581,8 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
 
 static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
 {
+    int node;
+    bool dom0_module_found = false;
     /*
      * For arm, we may get a device tree from GRUB (or other bootloader)
      * that contains modules that have already been loaded into memory.  In
@@ -592,11 +594,35 @@ static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
     fdt = lookup_fdt_config_table(SystemTable);
     dtbfile.ptr = fdt;
     dtbfile.need_to_free = false; /* Config table memory can't be freed. */
-    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
+
+    /* Locate chosen node */
+    node = fdt_subnode_offset(fdt, 0, "chosen");
+
+    /* Cycle through every node inside chosen having multiboot,module */
+    do {
+        int depth = 0;
+        node = fdt_node_offset_by_compatible(fdt, node, "multiboot,module");
+        /*
+         * If the multiboot,module just found is placed at depth less than 3,
+         * it means that it is here: /chosen/<module> so it is a module to
+         * start dom0. (root is counted as 0)
+         */
+        if ( node > 0 )
+        {
+            depth = fdt_node_depth(fdt, node);
+            if ( (depth >= 0) && (depth < 3) )
+            {
+                dom0_module_found = true;
+                break;
+            }
+        }
+    } while(node > 0);
+
+    if ( !fdt || !dom0_module_found )
     {
         /*
          * We either have no FDT, or one without modules, so we must have a
-         * Xen EFI configuration file to specify modules.  (dom0 required)
+         * Xen EFI configuration file to specify modules.
          */
         return true;
     }
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:26:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:26:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187703.336659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrg-0002oz-2A; Wed, 15 Sep 2021 14:26:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187703.336659; Wed, 15 Sep 2021 14:26:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVrf-0002os-VM; Wed, 15 Sep 2021 14:26:19 +0000
Received: by outflank-mailman (input) for mailman id 187703;
 Wed, 15 Sep 2021 14:26:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7Uy1=OF=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mQVre-0002og-0y
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:26:18 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 4f1d0d06-97b8-4803-9b99-515ab2e6830a;
 Wed, 15 Sep 2021 14:26:15 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B4DD6D;
 Wed, 15 Sep 2021 07:26:15 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EA4DC3F719;
 Wed, 15 Sep 2021 07:26:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f1d0d06-97b8-4803-9b99-515ab2e6830a
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 0/2] arm/efi: Add dom0less support to UEFI boot
Date: Wed, 15 Sep 2021 15:26:00 +0100
Message-Id: <20210915142602.42862-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1

This serie introduces a way to start a dom0less setup when Xen is booting as EFI
application.
Using the device tree it's now possible to fetch from the disk and load in
memory all the modules needed to start any domU defined in the DT.
Dom0less for now is supported only by the arm architecture.

Luca Fancellu (2):
  xen/efi: Restrict check for DT boot modules on EFI boot
  arm/efi: Use dom0less configuration when using EFI boot

 docs/misc/efi.pandoc        |  37 +++++
 xen/arch/arm/efi/efi-boot.h | 274 +++++++++++++++++++++++++++++++++++-
 xen/common/efi/boot.c       |  20 ++-
 3 files changed, 322 insertions(+), 9 deletions(-)

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:30:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:30:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187726.336692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVw5-0005Y5-5y; Wed, 15 Sep 2021 14:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187726.336692; Wed, 15 Sep 2021 14:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVw5-0005Xy-2k; Wed, 15 Sep 2021 14:30:53 +0000
Received: by outflank-mailman (input) for mailman id 187726;
 Wed, 15 Sep 2021 14:30:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WSgo=OF=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mQVw3-0005Xq-EF
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:30:51 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 36e509f7-e9ba-476d-96cd-f83c0f3eb406;
 Wed, 15 Sep 2021 14:30:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36e509f7-e9ba-476d-96cd-f83c0f3eb406
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631716249;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=K4mRVCnyoPHsjrfxdV+wp76sCfCx7m8LYcuTMrox51k=;
  b=clKqww9lnOzrd5DXENB+LEZaMHo9T8sysPecAECieKtV09igYecrrLRm
   yTW/FQZg84zaO2WrcmGZ63ihFrtYQQczlsWs+m8MMHmdxAcIjDqwdRZiJ
   Yf4mR7gLItFQB5lcM6rjKrqbCIC19Jpq9XhpZwPljYGB3ToghsMX1VYdJ
   A=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 5+oSUXDwJcrE8KlF9cUAt6yzz9DECW/JqoWR5oBP2F5WzwwNS/nqq40/MI7dGCCcz/uoC9m7d8
 umQjgvyz/1uZkhHL6L+D5CwIrKb7N5NoQVkzphOcj9ehwBWAHa2OK5X4VggUxIDkvEAp+wqjXN
 8CKXqULhpWICVnkicplorfLjE+IVzN4hk/i09MbbdiO/OB3b7KMdDveUWSJlQE1qPC3AhTjawW
 V+RWjVPo7yMHS2BISv/Jq6nH50pv9ZPhi0sVmPR3Uj4lvnAHlLLakGitB+ioZIafn242dJFmNf
 M0f4T7xlun0r+F9TGsi9fcAM
X-SBRS: 5.1
X-MesageID: 52806655
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:srAw5KMak16DBwbvrR29kMFynXyQoLVcMsEvi/4bfWQNrUpz1WABz
 2NKXmGHPa7YamT9f9B1O4+/8RkDupbdzNFlSQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Es+w7VRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYoy7Zvfpel
 PgWj7GhGBwgMPPdwLs+UyANRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsphmMUlavL3MY0WvHZ+5TrYEewnUdbIRKCiCdpwgW1g3p4QQKu2i
 8wxd3lvXC2dYTl1AnwVFc4snLmRpXvNfGgNwL6SjfVuuDWCpOBr65DsL9j9atGMXd9SnEuTu
 iTB5WuRKg4eHMySz3yC6H3Erv/Cm2b3VZwfEJW89+V2mxuDy2oLEhoUWFCn5/6jhSaWWdhSN
 kgV8SoGtrUp+QqgSdyVYvGjiCfa5FhGAYMWSrBkrlHWokbJ3+qHLnNUbBRuNPF9juInFCcj2
 AGYx4zKBQU65dV5VkmhGqeoQSKaYHZOdDJTOnBaFGPp8PG4/9pi1UunosJLVffv14yrQ2mYL
 yWi8XBm74j/m/LnwElSEbrvuDuqupGBZQo8/Ay/somNv14hOdLNi2BF7zHmARd8wGSxFQLpU
 JsswZH2AAUy4XalznflfQn1NOv1j8tpyRWF6bKVI3XEywlBBlb5JdwAiN2BGKuZGpldImK4C
 KMikShQ+IVSLBOXUEODWKroU55C5fG5TbzND6mIBvITMskZXFLWp0lGOB/Pt10BZWBxyMnTz
 7/AKp3yZZvbYIw6pAeLqxA1iuNynXFimjqIHPgWDX2PiNKjWZJccp9dWHPmUwzzxPrsTNz9/
 4kNOs2U5Q9YVeGiMCDb/ZRKdQIBLGQhBICwoMtSL7bRLg1jEWAnKvnQ3bJ+JNA1w/ULzr/Fr
 iOnR0tV6Fvjnnmbew+EXW9uNeH0VpFloHNlYSF1ZQS022IuaJqE5bsEc8dlZqEu8eFulKYmT
 /QMd8iaLO5ITzDLp2YUYZXn9dQwfxW3nwOeeSGiZWFnLZJnQgXI/P7ifxfuq3ZSXnbm65Nmr
 uT5hA3BQJcFSwBzN+rsaaqinwGroHwQuONuRE+UcNNdT1rhrdpxICvrg/5pf8xVcUffxiGX3
 hq9CAsDobWfuJc89dTEiPzWr4qtFOciTENWE3OCsOSzPCjeuGGi3ZVBQKCDejWEDDH4/6CrZ
 ON0yfDgMaJYwAYW4tQkS7s7n7gj49bPpqNBylU2FXrGWF2nF7d8LyTUxsJIrKBMmudUtAbet
 phjITWG1WFl4P/YLWM=
IronPort-HdrOrdr: A9a23:oqtJvaPv8958n8BcTjujsMiBIKoaSvp037BK7S1MoNJuEvBw9v
 re+MjzsCWftN9/Yh4dcLy7VpVoIkmskKKdg7NhXotKNTOO0AeVxelZhrcKqAeQeREWmNQ96U
 9hGZIOdeEZDzJB/LrHCN/TKade/DGFmprY+9s31x1WPGZXgzkL1XYDNu6ceHcGIjVuNN4CO7
 e3wNFInDakcWR/VLXAOpFUN9Kz3uEijfjdEGY7OyI=
X-IronPort-AV: E=Sophos;i="5.85,295,1624334400"; 
   d="scan'208";a="52806655"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>
Subject: [PATCH v3] tools/libxl: Correctly align the ACPI tables
Date: Wed, 15 Sep 2021 15:30:00 +0100
Message-ID: <20210915143000.36353-1-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The memory allocator currently calculates alignment in libxl's virtual
address space, rather than guest physical address space. This results
in the FACS being commonly misaligned.

Furthermore, the allocator has several other bugs.

The opencoded align-up calculation is currently susceptible to a bug
that occurs in the corner case that the buffer is already aligned to
begin with. In that case, an align-sized memory hole is introduced.

The while loop is dead logic because its effects are entirely and
unconditionally overwritten immediately after it.

Rework the memory allocator to align in guest physical address space
instead of libxl's virtual memory and improve the calculation, drop
errant extra page in allocated buffer for ACPI tables, and give some
of the variables better names/types.

Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>

v2: Rewrite completely, to align in guest physical address space

v3: Drop the now-unused local variable page_mask
---
 tools/libs/light/libxl_x86_acpi.c | 49 ++++++++++++-------------------
 1 file changed, 19 insertions(+), 30 deletions(-)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 3eca1c7a9f..57a6b63790 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -20,6 +20,7 @@
 
  /* Number of pages holding ACPI tables */
 #define NUM_ACPI_PAGES 16
+#define ALIGN(p, a) (((p) + ((a) - 1)) & ~((a) - 1))
 
 struct libxl_acpi_ctxt {
     struct acpi_ctxt c;
@@ -28,10 +29,10 @@ struct libxl_acpi_ctxt {
     unsigned int page_shift;
 
     /* Memory allocator */
-    unsigned long alloc_base_paddr;
-    unsigned long alloc_base_vaddr;
-    unsigned long alloc_currp;
-    unsigned long alloc_end;
+    unsigned long guest_start;
+    unsigned long guest_curr;
+    unsigned long guest_end;
+    void *buf;
 };
 
 extern const unsigned char dsdt_pvh[];
@@ -43,8 +44,7 @@ static unsigned long virt_to_phys(struct acpi_ctxt *ctxt, void *v)
     struct libxl_acpi_ctxt *libxl_ctxt =
         CONTAINER_OF(ctxt, struct libxl_acpi_ctxt, c);
 
-    return (((unsigned long)v - libxl_ctxt->alloc_base_vaddr) +
-            libxl_ctxt->alloc_base_paddr);
+    return libxl_ctxt->guest_start + (v - libxl_ctxt->buf);
 }
 
 static void *mem_alloc(struct acpi_ctxt *ctxt,
@@ -58,20 +58,16 @@ static void *mem_alloc(struct acpi_ctxt *ctxt,
     if (align < 16)
         align = 16;
 
-    s = (libxl_ctxt->alloc_currp + align) & ~((unsigned long)align - 1);
+    s = ALIGN(libxl_ctxt->guest_curr, align);
     e = s + size - 1;
 
     /* TODO: Reallocate memory */
-    if ((e < s) || (e >= libxl_ctxt->alloc_end))
+    if ((e < s) || (e >= libxl_ctxt->guest_end))
         return NULL;
 
-    while (libxl_ctxt->alloc_currp >> libxl_ctxt->page_shift != 
-           e >> libxl_ctxt->page_shift)
-        libxl_ctxt->alloc_currp += libxl_ctxt->page_size;
+    libxl_ctxt->guest_curr = e;
 
-    libxl_ctxt->alloc_currp = e;
-
-    return (void *)s;
+    return libxl_ctxt->buf + (s - libxl_ctxt->guest_start);
 }
 
 static void acpi_mem_free(struct acpi_ctxt *ctxt,
@@ -163,15 +159,12 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     struct acpi_config config = {0};
     struct libxl_acpi_ctxt libxl_ctxt;
     int rc = 0, acpi_pages_num;
-    void *acpi_pages;
-    unsigned long page_mask;
 
     if (b_info->type != LIBXL_DOMAIN_TYPE_PVH)
         goto out;
 
     libxl_ctxt.page_size = XC_DOM_PAGE_SIZE(dom);
     libxl_ctxt.page_shift =  XC_DOM_PAGE_SHIFT(dom);
-    page_mask = (1UL << libxl_ctxt.page_shift) - 1;
 
     libxl_ctxt.c.mem_ops.alloc = mem_alloc;
     libxl_ctxt.c.mem_ops.v2p = virt_to_phys;
@@ -186,19 +179,17 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
     config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
     /* Pages to hold ACPI tables */
-    acpi_pages =  libxl__malloc(gc, (NUM_ACPI_PAGES + 1) *
-                                libxl_ctxt.page_size);
+    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES *
+                                   libxl_ctxt.page_size);
 
     /*
      * Set up allocator memory.
      * Start next to acpi_info page to avoid fracturing e820.
      */
-    libxl_ctxt.alloc_base_paddr = ACPI_INFO_PHYSICAL_ADDRESS +
-        libxl_ctxt.page_size;
-    libxl_ctxt.alloc_base_vaddr = libxl_ctxt.alloc_currp =
-        (unsigned long)acpi_pages;
-    libxl_ctxt.alloc_end = (unsigned long)acpi_pages +
-        (NUM_ACPI_PAGES * libxl_ctxt.page_size);
+    libxl_ctxt.guest_start = libxl_ctxt.guest_curr = libxl_ctxt.guest_end =
+        ACPI_INFO_PHYSICAL_ADDRESS + libxl_ctxt.page_size;
+
+    libxl_ctxt.guest_end += NUM_ACPI_PAGES * libxl_ctxt.page_size;
 
     /* Build the tables. */
     rc = acpi_build_tables(&libxl_ctxt.c, &config);
@@ -208,10 +199,8 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     }
 
     /* Calculate how many pages are needed for the tables. */
-    acpi_pages_num =
-        ((libxl_ctxt.alloc_currp - (unsigned long)acpi_pages)
-         >> libxl_ctxt.page_shift) +
-        ((libxl_ctxt.alloc_currp & page_mask) ? 1 : 0);
+    acpi_pages_num = (ALIGN(libxl_ctxt.guest_curr, libxl_ctxt.page_size) -
+                      libxl_ctxt.guest_start) >> libxl_ctxt.page_shift;
 
     dom->acpi_modules[0].data = (void *)config.rsdp;
     dom->acpi_modules[0].length = 64;
@@ -231,7 +220,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     dom->acpi_modules[1].length = 4096;
     dom->acpi_modules[1].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS;
 
-    dom->acpi_modules[2].data = acpi_pages;
+    dom->acpi_modules[2].data = libxl_ctxt.buf;
     dom->acpi_modules[2].length = acpi_pages_num  << libxl_ctxt.page_shift;
     dom->acpi_modules[2].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS +
         libxl_ctxt.page_size;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:31:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:31:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187728.336707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVwC-0005uU-KM; Wed, 15 Sep 2021 14:31:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187728.336707; Wed, 15 Sep 2021 14:31:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVwC-0005uM-Gx; Wed, 15 Sep 2021 14:31:00 +0000
Received: by outflank-mailman (input) for mailman id 187728;
 Wed, 15 Sep 2021 14:30:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uklH=OF=ens-lyon.org=samuel.thibault@srs-us1.protection.inumbo.net>)
 id 1mQVwA-0005gO-IW
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:30:58 +0000
Received: from mail2-relais-roc.national.inria.fr (unknown [192.134.164.83])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 86930eac-1631-11ec-b536-12813bfff9fa;
 Wed, 15 Sep 2021 14:30:51 +0000 (UTC)
Received: from nat-eduroam-36-gw-01-bso.bordeaux.inria.fr (HELO begin)
 ([194.199.1.36])
 by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 15 Sep 2021 16:30:50 +0200
Received: from samy by begin with local (Exim 4.95-RC2)
 (envelope-from <samuel.thibault@ens-lyon.org>) id 1mQVw1-00FD1a-0C;
 Wed, 15 Sep 2021 16:30:49 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86930eac-1631-11ec-b536-12813bfff9fa
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A10s0O6PZyLiCZ8BcTqajsMiBIKoaSvp037BN?=
 =?us-ascii?q?7SFMoH1uHPBw+Pre/8jzuSWE6gr5O0tOpTn/Asm9qBrnnPYfi7X5Vo3PYOCJgg?=
 =?us-ascii?q?aVEL0=3D?=
X-IronPort-AV: E=Sophos;i="5.84,326,1620684000"; 
   d="scan'208";a="528623449"
Date: Wed, 15 Sep 2021 16:30:48 +0200
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH v3] xenbus: support large messages
Message-ID: <20210915143048.lluaekufonze6pic@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20210915140815.26736-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210915140815.26736-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Re,

Juergen Gross, le mer. 15 sept. 2021 16:08:15 +0200, a ecrit:
> +    while (off != len)
> +    {
> +        wait_event(xb_waitq, xenstore_buf->rsp_prod != xenstore_buf->rsp_cons);
> +
> +        prod = xenstore_buf->rsp_prod;
> +        cons = xenstore_buf->rsp_cons;
> +        DEBUG("Rsp_cons %d, rsp_prod %d.\n", cons, prod);
> +        size = min(len - off, prod - cons);
> +
> +        rmb();   /* Make sure data read from ring is ordered with rsp_prod. */
> +        memcpy_from_ring(xenstore_buf->rsp, buf + off,
> +                         MASK_XENSTORE_IDX(cons), size);
> +        off += size;
> +        xenstore_buf->rsp_cons += size;
> +        wmb();
> +        if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE)
> +            notify_remote_via_evtchn(xenbus_evtchn);
> +    }

Reading it again, I'm still not convinced we covered all cases. There is
at least one thing: Linux does

	virt_rmb();
	memcpy(data, src, avail);
	/* Other side must not see free space until we've copied out */
	virt_mb();
	intf->rsp_cons += avail;

which makes sense to me: we don't want the compiler or anything to
reorder the write to rsp_cons respectively to the memcpy. So I believe
we also need a full barrier before 

	xenstore_buf->rsp_cons += size;

in mini-os.


Then there is

	xenstore_buf->rsp_cons += size;
	wmb();
	if (xenstore_buf->rsp_prod - cons >= XENSTORE_RING_SIZE)
	    notify_remote_via_evtchn(xenbus_evtchn);

The Linux code does

	intf->rsp_cons += avail;
	
	/* Implies mb(): other side will see the updated consumer. */
	if (intf->rsp_prod - cons >= XENSTORE_RING_SIZE)
		notify_remote_via_evtchn(xen_store_evtchn);

(Note that the "Implies mb()" comment is about the notify_remote_via_evtchn call)

I believe the Linux code itself is missing a full barrier here: before
reading intf->rsp_prod, we want to make sure that our update of rsp_cons
is seen by the producer. Otherwise it may happen that the producer
doesn't see the rsp_cons and updates its rsp_prod (filling the ring
and going to sleep), and the consumer does not see the rsp_prod update
either, and thus we miss a notification and the producer stays stuck.

Extremely rare scenario etc. but I believe we do want a full barrier
between rsp_cons += and the if, both in mini-os and Linux.

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 14:31:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 14:31:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187744.336718 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVwu-0006p6-U5; Wed, 15 Sep 2021 14:31:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187744.336718; Wed, 15 Sep 2021 14:31:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQVwu-0006oz-QM; Wed, 15 Sep 2021 14:31:44 +0000
Received: by outflank-mailman (input) for mailman id 187744;
 Wed, 15 Sep 2021 14:31:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7Uy1=OF=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQVwu-0006or-61
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 14:31:44 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::61c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 53d206ba-56fa-43f0-917c-b3305290dd78;
 Wed, 15 Sep 2021 14:31:42 +0000 (UTC)
Received: from DU2PR04CA0343.eurprd04.prod.outlook.com (2603:10a6:10:2b4::29)
 by DB7PR08MB3914.eurprd08.prod.outlook.com (2603:10a6:10:32::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 15 Sep
 2021 14:31:40 +0000
Received: from DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b4:cafe::ef) by DU2PR04CA0343.outlook.office365.com
 (2603:10a6:10:2b4::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Wed, 15 Sep 2021 14:31:40 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT044.mail.protection.outlook.com (10.152.21.167) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 14:31:40 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Wed, 15 Sep 2021 14:31:40 +0000
Received: from 7768f521282e.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EF1A9EB9-5CAD-41B1-A47E-9E76909AF479.1; 
 Wed, 15 Sep 2021 14:31:33 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7768f521282e.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 Sep 2021 14:31:33 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6653.eurprd08.prod.outlook.com (2603:10a6:102:15f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Wed, 15 Sep
 2021 14:31:32 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 14:31:32 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0404.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:189::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 14:31:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53d206ba-56fa-43f0-917c-b3305290dd78
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=STGeZRz/sCVV+bj9yj0TBAhlmYRFaf/42Eo1Ncekw+o=;
 b=YOxcF+I4ObfBDmu6h68efHSgdoXJpWthEi59nuZZhJYTN5FC5L8eto8k9AJvX0M93+AH76LGraQYt9UJOemBlFgzHjlBtkYSDRvCJVJynATXBgoDs9SusHtpPIow8hQ5kyaE8R7dJXEJk9q70Mu+BPRCXVszHjVfuGkHvFTzox8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a6c22f69f37b4570
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gelrlvjbyX4Ad6oIp/bQehT+SzmgMqpUxbz+Ro4QWdwHZXRS5Voq/LDEpV7WO+h8ibD6UAmO/cbJFQ4qGTYNE19KlZAyi6DDjkz8y7paa9lc2+xAkFLE1wA8Yy6X8FIwVJ5G1TR7wcQWG2VOtMCAXPP682DR+nMkk8lB4obQQEWNcAbIu5JK54tcwxeIhTKmQDCFjAyoT9pgusJwZVLj2djZQATTCZpHuQWrWn9/y05kcfQ1LVuBX81TVlcRO20L1/LNi0Z/vVheJhrWElARA0bTEu5qoabf+YB//4bqx/fDttf3XtHjcpCsQYf3OO2HNL5BqiZ929xhvRkRcB+udQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=STGeZRz/sCVV+bj9yj0TBAhlmYRFaf/42Eo1Ncekw+o=;
 b=gyDHXRm5BPeyDM3Idi5RflS1NkRQE/4qBkqqXHRnd6LyGZw4YWcPA4GU6eh5uHvz57MLXelWKTbgT1TArHGUZCvA07xvnWUoloUoLTC+6Qe1dIngVtRHfu15h7Ziq9iif6n0e9QJSz9fVZu72M7OeOONI9kBNKad0KZ9C/QNH5mdx37PG9kBWuE/ss17RjXIaRv7RrBZZBkAS2Zh1At6z54dEhJN+wk98TZ6nkN8VSOGE5VtXndyREch+SFJ+LvaLJvc15TOrhsjUU5WGJlUPgD+DpHXuXt/19sU+axv8tj/rUPoOjzbvreCWDPtFFwDEd3jSuaCCnslrsEHp9f/4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=STGeZRz/sCVV+bj9yj0TBAhlmYRFaf/42Eo1Ncekw+o=;
 b=YOxcF+I4ObfBDmu6h68efHSgdoXJpWthEi59nuZZhJYTN5FC5L8eto8k9AJvX0M93+AH76LGraQYt9UJOemBlFgzHjlBtkYSDRvCJVJynATXBgoDs9SusHtpPIow8hQ5kyaE8R7dJXEJk9q70Mu+BPRCXVszHjVfuGkHvFTzox8=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v2] xen/sched: fix sched_move_domain() for domain without
 vcpus
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <20210908130440.24605-1-jgross@suse.com>
Date: Wed, 15 Sep 2021 15:31:25 +0100
Cc: xen-devel@lists.xenproject.org,
 George Dunlap <george.dunlap@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <A0542AFE-A372-4AC7-A0D0-5F997422DA4B@arm.com>
References: <20210908130440.24605-1-jgross@suse.com>
To: Juergen Gross <jgross@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0404.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:189::13) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 43b2f1f0-d62f-4939-8d4d-08d978558840
X-MS-TrafficTypeDiagnostic: PAXPR08MB6653:|DB7PR08MB3914:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB391462FE091AF62425A57DD5E4DB9@DB7PR08MB3914.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2276;OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 63dZx1mVpX1OthpuRugVb0S9d/ErTQEM5Iwd+kidR63B5MkFKagDtLOf8kyL7ofEb7ku7j8As/q5DllkTNg/W2uiRzG2BRiKkOul03+chFbuI41dRTE/h6qVUkRFcoFs400snCL8oAmv6RnvTEmTARGg/mn9rin1hhX9C2rtEuggrEIneLRyC3awG3SXFrbT6y2ViGTTAXgjqbVkohFv/y3Q4FebhZb6a15XzTQEXXIIYsA84EEUdwVG33dWEPxtYWclUiUd1SPtJ7dUWojRcNi2zcbNyVsJ2PMEyfdTFvihiQ18q4+Oj4pcHt5sHDFgvvDGQVOrWiK3hYEPoOSW7ZOQpsK51xP10my4X0vyLorJIhNmTSYDn5VwnAs4mTci7KdkhuB73TBdey+tT+ceM56DpsIyBODE1L0lvUcMdfOPQ5rCPo/7aBAQHbbY82jo3DWxaOPh8YP8Vt2JTHW1jk+6m63QkTCYi4AZJ9fIB+P0M4ioTm87qZPu7p1bAgAFRzMj8pjpoYRQLdAATkLkA9erhd1Qp3NSCZdkV97wIRCPweV5Et4faJ5Lyj4tgI3RtMRnqZHGO7b8wYimuV+eS+BFE3NyjTAgXUXBrLJkxwR5FJB2Ue8a+U+uVVjSyaThD1TCgiD0XrWq68S/CtfHHhDgqzAatvz91r/jhIjOliEYhmPfTgBTtxdlrZJHllwGXd2UoJEAHpQp9jsqw4taRGprsyjlwuMJ3ESyL3B1hM4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(396003)(39860400002)(366004)(376002)(186003)(8936002)(6666004)(26005)(66476007)(83380400001)(33656002)(52116002)(478600001)(44832011)(36756003)(2906002)(6506007)(38100700002)(54906003)(956004)(6512007)(6916009)(4326008)(66946007)(86362001)(316002)(66556008)(53546011)(6486002)(5660300002)(8676002)(2616005)(38350700002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6653
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	25561919-406b-4175-8232-08d978558326
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z30PJ1yP+kFt+irtv3unDErQzKjYHXdUMw7YXoxjrnBn56Aia/fVHoIBnXMTELYWrL2VhwUIu+gsDsh8trqq7AyJSWbdFh4EWNwcIJDcSQxekpLq72spnU177zp8M6h9eYC4DnnmPgx/AUusLiKrFZNrFzyERZ+d4qtXikIPdDlfvLL7r3rdsOkyCpCTvjSPia8SkoJWAnC1tvygVKDPt8DqQ7/NXsL4W5bg2ydDrikhjfYuaFvv55VCmJngeW7rtvyx08KO8efaAx1H6Neb5+bcW9V8Xc/ZMdeq0coxikINUm6M/iifDMYBS9pl9nohuak5ScTzeXOt4MKdHYo4duPP/z0QEyr5W4ay2HQlGN/mw4gwwpBiZt0K02p86NO8bX4gzipOjVIegpqCQfgoWJUw/WgWeCLfZfZi73e85n0N/z7oA34E3NaFna18BQxGTx5J4APe/fCIfAr6z4plkndzUF1jPGjrfOyrz/gkNgOFeprhlj0ZedQRG4+dgpcCuJkf2MJTVyXnw/Un+YR+kyOfnSwJQivS0xopSpNNvIXdeHR2Tr2PimE2cfptLyYxtxYqo3jHuRG++fRxr61x7WQPGCqsgxHe0IRuaN5ov2nmPYSTJ8xTP2H8uL82C3zrxKniri21hFwFtp3VdRRw4wKwijVFU+a3eH5KvqW3wJDjYnIigBPqR/7f58l9qM7GwibZlXbghptpkMXqNLa3aA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(396003)(346002)(376002)(39860400002)(36840700001)(46966006)(478600001)(86362001)(316002)(4326008)(36860700001)(47076005)(36756003)(82310400003)(54906003)(6862004)(44832011)(5660300002)(2616005)(956004)(336012)(8676002)(6486002)(53546011)(6512007)(83380400001)(8936002)(2906002)(6666004)(70586007)(70206006)(186003)(26005)(82740400003)(81166007)(356005)(33656002)(6506007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 14:31:40.3051
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 43b2f1f0-d62f-4939-8d4d-08d978558840
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3914



> On 8 Sep 2021, at 14:04, Juergen Gross <jgross@suse.com> wrote:
>=20
> In case a domain is created with a cpupool other than Pool-0 specified
> it will be moved to that cpupool before any vcpus are allocated.
>=20
> This will lead to a NULL pointer dereference in sched_move_domain().
>=20
> Fix that by tolerating vcpus not being allocated yet.
>=20
> Fixes: 70fadc41635b9b6 ("xen/cpupool: support moving domain between cpupo=
ols with different granularity")
> Reported-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Luca Fancellu <Luca.fancellu@arm.com>

> ---
> V2:
> - Only test for vcpu[0] not being present (Jan Beulich)
> ---
> xen/common/sched/core.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>=20
> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
> index 8d178baf3d..8f4b1ca10d 100644
> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -651,7 +651,7 @@ int sched_move_domain(struct domain *d, struct cpupoo=
l *c)
>     struct scheduler *old_ops =3D dom_scheduler(d);
>     void *old_domdata;
>     unsigned int gran =3D cpupool_get_granularity(c);
> -    unsigned int n_units =3D DIV_ROUND_UP(d->max_vcpus, gran);
> +    unsigned int n_units =3D d->vcpu[0] ? DIV_ROUND_UP(d->max_vcpus, gra=
n) : 0;
>     int ret =3D 0;
>=20
>     for_each_vcpu ( d, v )
> --=20
> 2.26.2
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:40:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:40:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187754.336729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX1O-0005u5-Ts; Wed, 15 Sep 2021 15:40:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187754.336729; Wed, 15 Sep 2021 15:40:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX1O-0005ty-Qa; Wed, 15 Sep 2021 15:40:26 +0000
Received: by outflank-mailman (input) for mailman id 187754;
 Wed, 15 Sep 2021 15:40:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX1N-0005ts-Tf
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:40:26 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.8])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3dad5288-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:40:23 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0783.namprd21.prod.outlook.com (2603:10b6:300:77::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.1; Wed, 15 Sep
 2021 15:40:18 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:40:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dad5288-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UQ+p/BEtE4Hu/1LS95NVH64lnNhmp5NJu60ibIfvHlW5NuaQR52CWKcGHBIBFa2Mpvync7D+X60eygG0TptF+kPFxGY00h/TfEOk53BqKQg4LaHTkzKkeM4feA1CfHmYHRBP+u+YdsdpWHZaTHzTCJ06rKAwhLAljzxHp9fR3j0j33eMCBbXW5if38jxJwhC1kwTUyoO4dQzrePDVcPPELNKhuQVwggxn8TXJvDURGGQC3/UAmJzMvltYG4qrrWv030OJTWEN5zz7K/tYRd0fTN7hLFGZVU3ZcmhMNqTnC45AFamv+p49KuIF45+irBmrtpexDyG7q68sNtuoUjLOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PuiDQ1qvHjVypg/77M68eUNojTmrE1TqpS61yWw/nTs=;
 b=jO6omh9/jiWeHLegYqyuZO4/cITO/cSc7G2PhFHI6CMf+CHiPMCfMdNWKXfrIH8GEFqeaQkwJyPDq3F7Wp8gNrIWJNOiGoHovtWsynKQp3jyaLBrBBOH3DjhxbVzPxso8Ulnt6qdM8SJ30/p36cI8MZcG9tqcELyMk787b9Vk2OafUhWWMy0F9N/BSEdRDGXnQdZINA7r6mtjTV0xGg2gdVs7FYxUg8eqDmg+QtcpiWdzSjw1+Sdp0n2wlO0oepLtCLzTfjfyy0WoTo6G2XBMO99bOYb47aLuMptsCfpbDJhZASW26lJhwBZLfU2yWrGaNWDMC/451KfDEQaF7i0Eg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PuiDQ1qvHjVypg/77M68eUNojTmrE1TqpS61yWw/nTs=;
 b=RZdWRmkpLxUjbs9Klonz0Q9MWBWeFWRMfsLJoErHzhqHNoLGBZ9PNgQ0VCzOrCnnxjW+OvjtY2AMp9WyjazHj0p+MdPn7YYHBmqio6kr6IuVUc/K3IrkslrRRIryzIEbHBG9W3+eqVbOZvIN4bBnHFMycUYaNtZm2XlwPWun66Y=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 04/12] Drivers: hv: vmbus: Mark vmbus ring buffer
 visible to host in Isolation VM
Thread-Topic: [PATCH V5 04/12] Drivers: hv: vmbus: Mark vmbus ring buffer
 visible to host in Isolation VM
Thread-Index: AQHXqW3+Ls5cj1t0FE+tX661cSIFmqujrk9Q
Date: Wed, 15 Sep 2021 15:40:17 +0000
Message-ID:
 <MWHPR21MB159336E9CB0BE98CB3F20348D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-5-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-5-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=39426b82-def3-4a21-8141-3f397c0df985;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T15:52:56Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 301ab89b-df54-4fde-3dc4-08d9785f1ec3
x-ms-traffictypediagnostic: MWHPR21MB0783:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB07838DAD01D32D38D645699DD7DB9@MWHPR21MB0783.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:48;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 z+w68+6cVGdajtk43BjOr0SrcUOhlmNoe6kMDvTlBbjLDGolIQ4YgSctfSkGfOa2Y3BsG1qdFxm70U1AGhstgO5IP4MacFz5/0SKPyoNTc//rkmLcelUhkylvriCyRFMR0cJ2J7TDSf2wQ0og3/EG6w4m/2Dbwq4ZrgKfvuqJ82CuG4AhG+YicYXspuTZIX0ySJY1ydbMrJOYeS4wLne2rhuDJqIKF8nVW0eBAFx21oqc14QRQQmxL9jm874NVcUzXFexi7tIE7dtgjiRHi1KUPGE1YPpJNiS/TS4L2Mis8sVi9uBr/qJ1DQ9PymKyXovpnQDlIQQHVdZA3nBdr1GncRY0+k/+fQwMq+tUL4pxdIa0PfWbIwDoeElnNuKUtrw6b4GO+fYLR3iyTFEQv+FDgouPsGX4NuG1W2MfjS2IulKDc5AbhEAyJ3eh5hBnVuwAol96Egcmy6R5TEPbf6gMEcMSgPjaPNTNkHrXJURBIvpEwMcW6xLBRkgEN2jwFK7v4APaABYdCnbwaRfzPdmubdBbcg4gNxwTmvQKZgcC/VrjoVQxMnzrnhW9+Pe8jKo15DJMqU5U2fm6R85dMZeVo4FCKzvFIDpQqsuqHRHOYHbds2FFdqE0jU3ijg6eTBwvUPPK/vUBI1jqCb/BAyfYY07gfQ8AGX1iFcbMg5Nw/+HiykDWkzEBZXIQv1NpxwMjtHScvORBtLlgi8uW/TSOLl3gOnyH7E0ZCkmWTcbVc=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(33656002)(7416002)(7406005)(66946007)(26005)(508600001)(9686003)(8936002)(2906002)(8676002)(30864003)(82960400001)(82950400001)(5660300002)(86362001)(921005)(186003)(122000001)(38070700005)(66556008)(10290500003)(66476007)(66446008)(64756008)(76116006)(38100700002)(316002)(54906003)(55016002)(83380400001)(6506007)(110136005)(71200400001)(52536014)(7696005)(8990500004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?5gpL0Yw3zFTsyL2aIf2DOBITn0FMvZDLaVRDDwtzb+k5e0vrmFfOJDuud08k?=
 =?us-ascii?Q?QCiiGVtkcJpoyb8+L9QpYV3Uyv5HsiuLrygXxlBKNNCVazx6rR0RLD1oAMwu?=
 =?us-ascii?Q?g3w9wSjzxerehacMJZ2Q3azCAB2thHYwb/oIDqe6wzO+CAu61dpFx4RbTB0R?=
 =?us-ascii?Q?CMZaV2Rf13RAdWkewiTgqy9sBHQ9FIq6snHBZb7ufwwMtAcMhyj881u6L0L1?=
 =?us-ascii?Q?AE6050MwVDVe6tHjnNSI/+cNsd+yYWnYquxKOQlcwtbNRJ+yCBHUvjI+tGMt?=
 =?us-ascii?Q?3Jb9gBCja+9nUE7yj9zvNhpMsC96YM1aZxHoK43TDVDd9+eEsKmPqSf26dNq?=
 =?us-ascii?Q?td3f8UM6noh3GHNsjJRy1akbrK9x6q9LxGy7o3o1dm9vzb9/VCuOn6AWu1oj?=
 =?us-ascii?Q?BxlR0cC4XkZ5S0apSNdTfH9CHlOq4qjNp1tEIc13hY2edLs/jY2V6lSQEyYW?=
 =?us-ascii?Q?roQjk2TXFr1UI5AclMYwHMHorN5q8MAV7g+U9oN5d+zrxZDsoHMxy+mV8rpi?=
 =?us-ascii?Q?b+kQsVaodzgfcrCjJOipPJg0fNjIhen3zTOJPoLJ+BFAms16sEFz/j5HKXDr?=
 =?us-ascii?Q?Hnq1Bq+fb/4hVjxFiOdI7+ax5PeIwwPAkZ2FtQYhH32DwyOntrZZw8xer4MA?=
 =?us-ascii?Q?ZjfnT23tnFrwF4jIyslRHgNSKFwxONm2Gb+3aps87sh6+ctq1kc5qYe814AJ?=
 =?us-ascii?Q?+fcF6lhz24hnnSEpGOiH9M571avCQhqwcLwcgxyd+eK82gveJ0cZy1/lIJMd?=
 =?us-ascii?Q?JPnIZmAua4aEyifHza5Tm5Eatv+sGJ/YbGx3Nq9QfarCdEZlm4Uq8amXqgNG?=
 =?us-ascii?Q?Zgy/nrdZxRmhjwiXWnwxGSl8Iee957l8HFHEWLADXAa9IBgRC3gq5WbNzCjO?=
 =?us-ascii?Q?CJlMn7blH+WfXdWKaefsrc6AvdnlTsEKwQsNgpnosD30AAcV2VdZAqLlALDK?=
 =?us-ascii?Q?TmlPtdtAdgx7Nko578Wnzx7h4Hmrv+lk8uvkqBqsZ/93WuozZEPLj1RL7Sni?=
 =?us-ascii?Q?Fr4Ghme1EUHBBs96f5IUBhE6t8WefV7pcqN9MG8noE5Jw9BtrLeSoaio74qu?=
 =?us-ascii?Q?BTRwa0KAkRUTVwcZp+i4LxSXBxQtxJHKjajzxNDo8+BGYIW6Rac8qiJvkPIi?=
 =?us-ascii?Q?YFanuSGc6qUkmaXNfnfuRMNc74Xs7wXIjmhMP4YKmLo3PRQT8h5CbMid2CFD?=
 =?us-ascii?Q?0qq1QUclKyxjNCD1G2xugjs8kS5xQKbQtPSlqG+nXc0fZhCsyI6/X+aGgeGQ?=
 =?us-ascii?Q?op+aID1i28hn5UFlyIS/FzAyTAhrodTv6g9o8aJnn7CkPvJogTJj4dbEHynz?=
 =?us-ascii?Q?uMJ92e7+NPntjSC7xkR3vXM4?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 301ab89b-df54-4fde-3dc4-08d9785f1ec3
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:40:17.9126
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sz49JHb3QuT41F68vM7pdl6fZq1l26Djcqb9/tU8Gq4wgbQbUaCSgI4PdTkzd7fWEf3YfFZH5dt+MDvK0UYEDokvc0E1iTLeC92zRCIasMc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0783

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> Mark vmbus ring buffer visible with set_memory_decrypted() when
> establish gpadl handle.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change sincv v4
> 	* Change gpadl handle in netvsc and uio driver from u32 to
> 	  struct vmbus_gpadl.
> 	* Change vmbus_establish_gpadl()'s gpadl_handle parameter
> 	  to vmbus_gpadl data structure.
>=20
> Change since v3:
> 	* Change vmbus_teardown_gpadl() parameter and put gpadl handle,
> 	  buffer and buffer size in the struct vmbus_gpadl.
> ---
>  drivers/hv/channel.c            | 54 ++++++++++++++++++++++++---------
>  drivers/net/hyperv/hyperv_net.h |  5 +--
>  drivers/net/hyperv/netvsc.c     | 17 ++++++-----
>  drivers/uio/uio_hv_generic.c    | 20 ++++++------
>  include/linux/hyperv.h          | 12 ++++++--
>  5 files changed, 71 insertions(+), 37 deletions(-)
>=20
> diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
> index f3761c73b074..cf419eb1de77 100644
> --- a/drivers/hv/channel.c
> +++ b/drivers/hv/channel.c
> @@ -17,6 +17,7 @@
>  #include <linux/hyperv.h>
>  #include <linux/uio.h>
>  #include <linux/interrupt.h>
> +#include <linux/set_memory.h>
>  #include <asm/page.h>
>  #include <asm/mshyperv.h>
>=20
> @@ -456,7 +457,7 @@ static int create_gpadl_header(enum hv_gpadl_type typ=
e, void *kbuffer,
>  static int __vmbus_establish_gpadl(struct vmbus_channel *channel,
>  				   enum hv_gpadl_type type, void *kbuffer,
>  				   u32 size, u32 send_offset,
> -				   u32 *gpadl_handle)
> +				   struct vmbus_gpadl *gpadl)
>  {
>  	struct vmbus_channel_gpadl_header *gpadlmsg;
>  	struct vmbus_channel_gpadl_body *gpadl_body;
> @@ -474,6 +475,15 @@ static int __vmbus_establish_gpadl(struct vmbus_chan=
nel *channel,
>  	if (ret)
>  		return ret;
>=20
> +	ret =3D set_memory_decrypted((unsigned long)kbuffer,
> +				   HVPFN_UP(size));

This should be PFN_UP, not HVPFN_UP.  The numpages parameter to
set_memory_decrypted() is in guest size pages, not Hyper-V size pages.

> +	if (ret) {
> +		dev_warn(&channel->device_obj->device,
> +			 "Failed to set host visibility for new GPADL %d.\n",
> +			 ret);
> +		return ret;
> +	}
> +
>  	init_completion(&msginfo->waitevent);
>  	msginfo->waiting_channel =3D channel;
>=20
> @@ -537,7 +547,10 @@ static int __vmbus_establish_gpadl(struct vmbus_chan=
nel *channel,
>  	}
>=20
>  	/* At this point, we received the gpadl created msg */
> -	*gpadl_handle =3D gpadlmsg->gpadl;
> +	gpadl->gpadl_handle =3D gpadlmsg->gpadl;
> +	gpadl->buffer =3D kbuffer;
> +	gpadl->size =3D size;
> +
>=20
>  cleanup:
>  	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
> @@ -549,6 +562,11 @@ static int __vmbus_establish_gpadl(struct vmbus_chan=
nel *channel,
>  	}
>=20
>  	kfree(msginfo);
> +
> +	if (ret)
> +		set_memory_encrypted((unsigned long)kbuffer,
> +				     HVPFN_UP(size));

Should be PFN_UP as noted on the previous call to set_memory_decrypted().

> +
>  	return ret;
>  }
>=20
> @@ -561,10 +579,10 @@ static int __vmbus_establish_gpadl(struct vmbus_cha=
nnel *channel,
>   * @gpadl_handle: some funky thing
>   */
>  int vmbus_establish_gpadl(struct vmbus_channel *channel, void *kbuffer,
> -			  u32 size, u32 *gpadl_handle)
> +			  u32 size, struct vmbus_gpadl *gpadl)
>  {
>  	return __vmbus_establish_gpadl(channel, HV_GPADL_BUFFER, kbuffer, size,
> -				       0U, gpadl_handle);
> +				       0U, gpadl);
>  }
>  EXPORT_SYMBOL_GPL(vmbus_establish_gpadl);
>=20
> @@ -639,6 +657,7 @@ static int __vmbus_open(struct vmbus_channel *newchan=
nel,
>  	struct vmbus_channel_open_channel *open_msg;
>  	struct vmbus_channel_msginfo *open_info =3D NULL;
>  	struct page *page =3D newchannel->ringbuffer_page;
> +	struct vmbus_gpadl gpadl;

I think this local variable was needed in a previous version of the patch, =
but
is now unused and should be deleted.

>  	u32 send_pages, recv_pages;
>  	unsigned long flags;
>  	int err;
> @@ -675,7 +694,7 @@ static int __vmbus_open(struct vmbus_channel *newchan=
nel,
>  		goto error_clean_ring;
>=20
>  	/* Establish the gpadl for the ring buffer */
> -	newchannel->ringbuffer_gpadlhandle =3D 0;
> +	newchannel->ringbuffer_gpadlhandle.gpadl_handle =3D 0;
>=20
>  	err =3D __vmbus_establish_gpadl(newchannel, HV_GPADL_RING,
>  				      page_address(newchannel->ringbuffer_page),
> @@ -701,7 +720,8 @@ static int __vmbus_open(struct vmbus_channel *newchan=
nel,
>  	open_msg->header.msgtype =3D CHANNELMSG_OPENCHANNEL;
>  	open_msg->openid =3D newchannel->offermsg.child_relid;
>  	open_msg->child_relid =3D newchannel->offermsg.child_relid;
> -	open_msg->ringbuffer_gpadlhandle =3D newchannel->ringbuffer_gpadlhandle=
;
> +	open_msg->ringbuffer_gpadlhandle
> +		=3D newchannel->ringbuffer_gpadlhandle.gpadl_handle;
>  	/*
>  	 * The unit of ->downstream_ringbuffer_pageoffset is HV_HYP_PAGE and
>  	 * the unit of ->ringbuffer_send_offset (i.e. send_pages) is PAGE, so
> @@ -759,8 +779,8 @@ static int __vmbus_open(struct vmbus_channel *newchan=
nel,
>  error_free_info:
>  	kfree(open_info);
>  error_free_gpadl:
> -	vmbus_teardown_gpadl(newchannel, newchannel->ringbuffer_gpadlhandle);
> -	newchannel->ringbuffer_gpadlhandle =3D 0;
> +	vmbus_teardown_gpadl(newchannel, &newchannel->ringbuffer_gpadlhandle);
> +	newchannel->ringbuffer_gpadlhandle.gpadl_handle =3D 0;

My previous comments had suggested letting vmbus_teardown_gpadl() set the
gpadl_handle to zero, avoiding the need for all the callers to set it to ze=
ro.
Did that not work for some reason?  Just curious ....

>  error_clean_ring:
>  	hv_ringbuffer_cleanup(&newchannel->outbound);
>  	hv_ringbuffer_cleanup(&newchannel->inbound);
> @@ -806,7 +826,7 @@ EXPORT_SYMBOL_GPL(vmbus_open);
>  /*
>   * vmbus_teardown_gpadl -Teardown the specified GPADL handle
>   */
> -int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle=
)
> +int vmbus_teardown_gpadl(struct vmbus_channel *channel, struct vmbus_gpa=
dl *gpadl)
>  {
>  	struct vmbus_channel_gpadl_teardown *msg;
>  	struct vmbus_channel_msginfo *info;
> @@ -825,7 +845,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channe=
l, u32 gpadl_handle)
>=20
>  	msg->header.msgtype =3D CHANNELMSG_GPADL_TEARDOWN;
>  	msg->child_relid =3D channel->offermsg.child_relid;
> -	msg->gpadl =3D gpadl_handle;
> +	msg->gpadl =3D gpadl->gpadl_handle;
>=20
>  	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
>  	list_add_tail(&info->msglistentry,
> @@ -859,6 +879,12 @@ int vmbus_teardown_gpadl(struct vmbus_channel *chann=
el, u32 gpadl_handle)
>  	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
>=20
>  	kfree(info);
> +
> +	ret =3D set_memory_encrypted((unsigned long)gpadl->buffer,
> +				   HVPFN_UP(gpadl->size));

PFN_UP here as well.

> +	if (ret)
> +		pr_warn("Fail to set mem host visibility in GPADL teardown %d.\n", ret=
);
> +
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vmbus_teardown_gpadl);
> @@ -896,6 +922,7 @@ void vmbus_reset_channel_cb(struct vmbus_channel *cha=
nnel)
>  static int vmbus_close_internal(struct vmbus_channel *channel)
>  {
>  	struct vmbus_channel_close_channel *msg;
> +	struct vmbus_gpadl gpadl;

I think this local variable was needed in a previous version of the patch, =
but
is now unused and should be deleted.

>  	int ret;
>=20
>  	vmbus_reset_channel_cb(channel);
> @@ -933,9 +960,8 @@ static int vmbus_close_internal(struct vmbus_channel =
*channel)
>  	}
>=20
>  	/* Tear down the gpadl for the channel's ring buffer */
> -	else if (channel->ringbuffer_gpadlhandle) {
> -		ret =3D vmbus_teardown_gpadl(channel,
> -					   channel->ringbuffer_gpadlhandle);
> +	else if (channel->ringbuffer_gpadlhandle.gpadl_handle) {
> +		ret =3D vmbus_teardown_gpadl(channel, &channel->ringbuffer_gpadlhandle=
);
>  		if (ret) {
>  			pr_err("Close failed: teardown gpadl return %d\n", ret);
>  			/*
> @@ -944,7 +970,7 @@ static int vmbus_close_internal(struct vmbus_channel =
*channel)
>  			 */
>  		}
>=20
> -		channel->ringbuffer_gpadlhandle =3D 0;
> +		channel->ringbuffer_gpadlhandle.gpadl_handle =3D 0;
>  	}
>=20
>  	if (!ret)
> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_=
net.h
> index bc48855dff10..315278a7cf88 100644
> --- a/drivers/net/hyperv/hyperv_net.h
> +++ b/drivers/net/hyperv/hyperv_net.h
> @@ -1075,14 +1075,15 @@ struct netvsc_device {
>  	/* Receive buffer allocated by us but manages by NetVSP */
>  	void *recv_buf;
>  	u32 recv_buf_size; /* allocated bytes */
> -	u32 recv_buf_gpadl_handle;
> +	struct vmbus_gpadl recv_buf_gpadl_handle;
>  	u32 recv_section_cnt;
>  	u32 recv_section_size;
>  	u32 recv_completion_cnt;
>=20
>  	/* Send buffer allocated by us */
>  	void *send_buf;
> -	u32 send_buf_gpadl_handle;
> +	u32 send_buf_size;
> +	struct vmbus_gpadl send_buf_gpadl_handle;
>  	u32 send_section_cnt;
>  	u32 send_section_size;
>  	unsigned long *send_section_map;
> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
> index 7bd935412853..1f87e570ed2b 100644
> --- a/drivers/net/hyperv/netvsc.c
> +++ b/drivers/net/hyperv/netvsc.c
> @@ -278,9 +278,9 @@ static void netvsc_teardown_recv_gpadl(struct hv_devi=
ce *device,
>  {
>  	int ret;
>=20
> -	if (net_device->recv_buf_gpadl_handle) {
> +	if (net_device->recv_buf_gpadl_handle.gpadl_handle) {
>  		ret =3D vmbus_teardown_gpadl(device->channel,
> -					   net_device->recv_buf_gpadl_handle);
> +					   &net_device->recv_buf_gpadl_handle);
>=20
>  		/* If we failed here, we might as well return and have a leak
>  		 * rather than continue and a bugchk
> @@ -290,7 +290,7 @@ static void netvsc_teardown_recv_gpadl(struct hv_devi=
ce *device,
>  				   "unable to teardown receive buffer's gpadl\n");
>  			return;
>  		}
> -		net_device->recv_buf_gpadl_handle =3D 0;
> +		net_device->recv_buf_gpadl_handle.gpadl_handle =3D 0;
>  	}
>  }
>=20
> @@ -300,9 +300,9 @@ static void netvsc_teardown_send_gpadl(struct hv_devi=
ce *device,
>  {
>  	int ret;
>=20
> -	if (net_device->send_buf_gpadl_handle) {
> +	if (net_device->send_buf_gpadl_handle.gpadl_handle) {
>  		ret =3D vmbus_teardown_gpadl(device->channel,
> -					   net_device->send_buf_gpadl_handle);
> +					   &net_device->send_buf_gpadl_handle);
>=20
>  		/* If we failed here, we might as well return and have a leak
>  		 * rather than continue and a bugchk
> @@ -312,7 +312,7 @@ static void netvsc_teardown_send_gpadl(struct hv_devi=
ce *device,
>  				   "unable to teardown send buffer's gpadl\n");
>  			return;
>  		}
> -		net_device->send_buf_gpadl_handle =3D 0;
> +		net_device->send_buf_gpadl_handle.gpadl_handle =3D 0;
>  	}
>  }
>=20
> @@ -380,7 +380,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
>  	init_packet->hdr.msg_type =3D NVSP_MSG1_TYPE_SEND_RECV_BUF;
>  	init_packet->msg.v1_msg.send_recv_buf.
> -		gpadl_handle =3D net_device->recv_buf_gpadl_handle;
> +		gpadl_handle =3D net_device->recv_buf_gpadl_handle.gpadl_handle;
>  	init_packet->msg.v1_msg.
>  		send_recv_buf.id =3D NETVSC_RECEIVE_BUFFER_ID;
>=20
> @@ -463,6 +463,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  		ret =3D -ENOMEM;
>  		goto cleanup;
>  	}
> +	net_device->send_buf_size =3D buf_size;
>=20
>  	/* Establish the gpadl handle for this buffer on this
>  	 * channel.  Note: This call uses the vmbus connection rather
> @@ -482,7 +483,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
>  	init_packet->hdr.msg_type =3D NVSP_MSG1_TYPE_SEND_SEND_BUF;
>  	init_packet->msg.v1_msg.send_send_buf.gpadl_handle =3D
> -		net_device->send_buf_gpadl_handle;
> +		net_device->send_buf_gpadl_handle.gpadl_handle;
>  	init_packet->msg.v1_msg.send_send_buf.id =3D NETVSC_SEND_BUFFER_ID;
>=20
>  	trace_nvsp_send(ndev, init_packet);
> diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c
> index 652fe2547587..548243dcd895 100644
> --- a/drivers/uio/uio_hv_generic.c
> +++ b/drivers/uio/uio_hv_generic.c
> @@ -58,11 +58,11 @@ struct hv_uio_private_data {
>  	atomic_t refcnt;
>=20
>  	void	*recv_buf;
> -	u32	recv_gpadl;
> +	struct vmbus_gpadl recv_gpadl;
>  	char	recv_name[32];	/* "recv_4294967295" */
>=20
>  	void	*send_buf;
> -	u32	send_gpadl;
> +	struct vmbus_gpadl send_gpadl;
>  	char	send_name[32];
>  };
>=20
> @@ -179,15 +179,15 @@ hv_uio_new_channel(struct vmbus_channel *new_sc)
>  static void
>  hv_uio_cleanup(struct hv_device *dev, struct hv_uio_private_data *pdata)
>  {
> -	if (pdata->send_gpadl) {
> -		vmbus_teardown_gpadl(dev->channel, pdata->send_gpadl);
> -		pdata->send_gpadl =3D 0;
> +	if (pdata->send_gpadl.gpadl_handle) {
> +		vmbus_teardown_gpadl(dev->channel, &pdata->send_gpadl);
> +		pdata->send_gpadl.gpadl_handle =3D 0;
>  		vfree(pdata->send_buf);
>  	}
>=20
> -	if (pdata->recv_gpadl) {
> -		vmbus_teardown_gpadl(dev->channel, pdata->recv_gpadl);
> -		pdata->recv_gpadl =3D 0;
> +	if (pdata->recv_gpadl.gpadl_handle) {
> +		vmbus_teardown_gpadl(dev->channel, &pdata->recv_gpadl);
> +		pdata->recv_gpadl.gpadl_handle =3D 0;
>  		vfree(pdata->recv_buf);
>  	}
>  }
> @@ -303,7 +303,7 @@ hv_uio_probe(struct hv_device *dev,
>=20
>  	/* put Global Physical Address Label in name */
>  	snprintf(pdata->recv_name, sizeof(pdata->recv_name),
> -		 "recv:%u", pdata->recv_gpadl);
> +		 "recv:%u", pdata->recv_gpadl.gpadl_handle);
>  	pdata->info.mem[RECV_BUF_MAP].name =3D pdata->recv_name;
>  	pdata->info.mem[RECV_BUF_MAP].addr
>  		=3D (uintptr_t)pdata->recv_buf;
> @@ -324,7 +324,7 @@ hv_uio_probe(struct hv_device *dev,
>  	}
>=20
>  	snprintf(pdata->send_name, sizeof(pdata->send_name),
> -		 "send:%u", pdata->send_gpadl);
> +		 "send:%u", pdata->send_gpadl.gpadl_handle);
>  	pdata->info.mem[SEND_BUF_MAP].name =3D pdata->send_name;
>  	pdata->info.mem[SEND_BUF_MAP].addr
>  		=3D (uintptr_t)pdata->send_buf;
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index ddc8713ce57b..a9e0bc3b1511 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -803,6 +803,12 @@ struct vmbus_device {
>=20
>  #define VMBUS_DEFAULT_MAX_PKT_SIZE 4096
>=20
> +struct vmbus_gpadl {
> +	u32 gpadl_handle;
> +	u32 size;
> +	void *buffer;
> +};
> +
>  struct vmbus_channel {
>  	struct list_head listentry;
>=20
> @@ -822,7 +828,7 @@ struct vmbus_channel {
>  	bool rescind_ref; /* got rescind msg, got channel reference */
>  	struct completion rescind_event;
>=20
> -	u32 ringbuffer_gpadlhandle;
> +	struct vmbus_gpadl ringbuffer_gpadlhandle;
>=20
>  	/* Allocated memory for ring buffer */
>  	struct page *ringbuffer_page;
> @@ -1192,10 +1198,10 @@ extern int vmbus_sendpacket_mpb_desc(struct vmbus=
_channel *channel,
>  extern int vmbus_establish_gpadl(struct vmbus_channel *channel,
>  				      void *kbuffer,
>  				      u32 size,
> -				      u32 *gpadl_handle);
> +				      struct vmbus_gpadl *gpadl);
>=20
>  extern int vmbus_teardown_gpadl(struct vmbus_channel *channel,
> -				     u32 gpadl_handle);
> +				     struct vmbus_gpadl *gpadl);
>=20
>  void vmbus_reset_channel_cb(struct vmbus_channel *channel);
>=20
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:41:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:41:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187763.336740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX2J-0006W6-BG; Wed, 15 Sep 2021 15:41:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187763.336740; Wed, 15 Sep 2021 15:41:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX2J-0006Vz-7y; Wed, 15 Sep 2021 15:41:23 +0000
Received: by outflank-mailman (input) for mailman id 187763;
 Wed, 15 Sep 2021 15:41:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX2H-0006Vt-Vm
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:41:22 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.15])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5f264686-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:41:19 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0783.namprd21.prod.outlook.com (2603:10b6:300:77::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.1; Wed, 15 Sep
 2021 15:41:14 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:41:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f264686-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W1oQ+BdmlwuKjbpzXOvUz6hvQSV0kOaslOZBkM2YVQWPy8C4L0XqqASX1R3rTjGBbK73IORsl85AdWR9IX564QecvhwfnGG30At7epj3GyE6BryYl55XIvQ1mIgnouK2oisqmPRtP3RRqwAzLbjCNBKoF9J/oh3QXmuPN3uAzt2fb0bwouBWk/YabqYFPHiqxct9yGUnIMDJGTS1qew2zHPWSJ5Fwwvpl6BD1dGT/5R04Ev60zILrs+5jtnNCv259BHTJYmHPeoZ2PPQYQsEr/XoJh5Dx8w1oSJAR7Tz/ucgIlPfa+HcoIeGj197tPlGk272/sarxRKhABAW44rwmw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cAjeTAevZElnIv9aFi1h/zCK5mbtaqIBDeFtbOXKBh8=;
 b=CxU2fkqyaPMKby26yxu7RpfUVxAXBR+KWXWEfhRRD+Mwh+Dwc1XS/5aazfPwuQHt43PG1OVnqlwOFatSoGPUrbLcKw/QXIi6lapnYLnLhkWU1VDJIyK3NhdGgHVGbIZvE5gW70ChpqnX3SQAru+vRfb97O7wdLO8ZX7TuPbACMY/iiiODShNuQDGDITencVOMXNAy0kcTrBcYAbj5z9K+4oGtO+tQkuxZ4KCzVzD+0u+C9XHODwwRiTKLtrQUKywa8HwLPn4w0DvUhZL77Zfz3cQ7bsEdQNLxyFeWrE/qkhg3QRLbpdqgDtJRun4y0wYzlnCZJa4bx3lIQ+x7Ovm2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cAjeTAevZElnIv9aFi1h/zCK5mbtaqIBDeFtbOXKBh8=;
 b=OMEhoceFD9kFQKJDsoYh94pBJwmSVdaCwfG8wVUtBI8j2urs80SiDPoRXfDzecQBBPLeUZvE/D/ipc4MG/YIIpDUGxYr/2yPyWF4pJkrxOG2aSTeaCLFo/yV3SiWsDu76hGuMNcjX4U0hc8Xd+zIky/CpbIYHQsQVrGF5IvzF3E=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 05/12] x86/hyperv: Add Write/Read MSR registers via
 ghcb page
Thread-Topic: [PATCH V5 05/12] x86/hyperv: Add Write/Read MSR registers via
 ghcb page
Thread-Index: AQHXqW39j2hpzRTyUUG/YzVB4+nRcqujwImA
Date: Wed, 15 Sep 2021 15:41:14 +0000
Message-ID:
 <MWHPR21MB15937575DE3050863C56F3D2D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-6-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-6-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=c7a4a87b-c9ee-4bd7-bbd3-c03c9a3f412f;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T16:58:10Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 52a3c87e-93a2-4615-7419-08d9785f405f
x-ms-traffictypediagnostic: MWHPR21MB0783:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB07832FA927AB6D27E88F3E00D7DB9@MWHPR21MB0783.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:127;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 O+ICcdiaZxhXo8oCd0aeWIPXBSrRl5YaIzPVeWbJBlh9XKWrv9mllGkXDLrcZ4LTc93rd7+zRu/j8xHwu/fQm4CB6xnfvs00lUyKdS+RsonWRICKcR3g6+UNkmfu3zwowVN0xFIlCMDqH4lsp6TlJrUdUI7+SGAzuNsgt7RBMVnMEHeF/55pj/C8LpIVok++JyCWoSMhxO+GYj4X1qOBam6xbCdA19MN/trmpPxBNE8nUkNDyWY9Ls5YfjuPlSKX5ijXlxr293Kkw5a/5LDXLxYnuTbnTisuzlANH/jCYbcg+BECW2JabIrSNP1/fBCtrouCwFOd5cx7mYnc8qVb6LQwipNGhDvN5BNGdDFjVrDKy4o2ACYNV7WkI21RqDGBX42D3gF4kKXIn9flw35E7/nIXLKP7V8JnFshtVhPCAGjzXVV+Fw1d5UGcfIuThfs5L2IrZek+dcIPqQRndz756eJx2ZlWXbf0vX1HgfXkwaCgO7TL6zxAm+pO2rGwikAcEovWcOtRSbc9crOH8MfXSnc/Xl280OyOr2WmIC2Pld5pXbxBsHrtxJOuUK+LUKhgGQMN01jfkYEV8WlQeaKtKG5m+INoFOpjSiUeOIts3lnhuYaKYFJC+u9CSyTOyq80Hl25ZIpMCWJ3s/DwJ+ElgtgGYeyD2nWAMMkVW0VjZPacbMvWoDB40XsIsyXmxKjK0KWTsbvXyHosGSchyhYqbKb/JWtHSaAHXkgueYt9Ag=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(33656002)(7416002)(7406005)(66946007)(26005)(508600001)(9686003)(8936002)(2906002)(8676002)(30864003)(82960400001)(82950400001)(5660300002)(86362001)(921005)(186003)(122000001)(38070700005)(66556008)(10290500003)(66476007)(66446008)(64756008)(76116006)(38100700002)(316002)(54906003)(55016002)(83380400001)(6506007)(110136005)(71200400001)(52536014)(7696005)(8990500004)(579004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?E5CCjcHCN7/4g0U3P52aMjrEANBXOOAtDwkNNNtAm6u1PUxcJUZMFgWkfQ2i?=
 =?us-ascii?Q?cg/HObkFLLM/CrnjDl8w/jynUD9FtiiyfbAVgRV+ooQEmR0OL0vdgJiQdGfp?=
 =?us-ascii?Q?V37m3wuxwEe6ulM8ptB03dysMXhlA8iJHk6qfc2e3oI2OZTXWvD0Qp7hf9uY?=
 =?us-ascii?Q?OtL+jQEZBynzNRztTtrsYPcWXDEVaqTAQhaqjiW2sHTDRG3Bx6VKyCMhYW2p?=
 =?us-ascii?Q?A8MctyD9eZP0A1oLnqaBuLRAmUliKlcZo2AKLQ8ZDXPzcnXM2bWfV59fX2tY?=
 =?us-ascii?Q?pSGJG1l8sn2lL+1/W4BkIreURTUbMFDvYQ3DdGCDgEs6iW5TTUU3d6Z/7HG8?=
 =?us-ascii?Q?dO/nYQ62O/A18yacZ7U47ufDCQypsWtwWQNULngEWTIPZXDy4KCSpxITW87E?=
 =?us-ascii?Q?OFvscc4yXb96j6PbH0jChQqKwSeDQKz8fSgO3OLGeevV/6bwx2duhSs5+J0R?=
 =?us-ascii?Q?fXhVWPAFhXLoF9yMlkIDl2Qofsv8ST/qBzs+TLzFnZWRekOUUp6XA/bChWUY?=
 =?us-ascii?Q?sJaFz2lvIByra0r4D+sZ9jgwH3wJ4Jt4H45SW/qsmccNqME/YI+q6sKrZhxA?=
 =?us-ascii?Q?kKMSnswnhhn6Pg46KkZx9kfIXeMphdh7MC+GClw0UftxsY5WgN4HVgcRDwbs?=
 =?us-ascii?Q?0orc1R7/JtLXfEdf5HF3CRLYMh1CiiiYljG60S2k3k6x2BQ6S/kbkdfVyzJs?=
 =?us-ascii?Q?jRMTA+Un+98z950DTkXpC99IJJEgQSZ7Ys7X/UQ1ZAuPal5xLBdujliXxct5?=
 =?us-ascii?Q?mV9ObgJY7Ba0bdV72gdBg0t/7l/ZT/lPDIohjG3DslMwR0odT9rrGNgs5khs?=
 =?us-ascii?Q?CSiTY+iJ4mrIXp5lju/Kl9uv7Pn8xEk8GHDra1C5iW/pNmUIaz1kObNQZ/el?=
 =?us-ascii?Q?QRgzfemTREnx9R2fbEo6lhv1RN5xu7PCLHt00KqHGXHIRfKivfvzlghtviLu?=
 =?us-ascii?Q?1rdq5vPNUlRouCXAhMhWlciy5+r8tijbOEbmtFm8PCXRY10EWIj40MbeSfwH?=
 =?us-ascii?Q?T5SmfAfBSU2TnCAXDud+u22n6PQND049wdMDq9BhnfszUIoUViGwqCqeCb1n?=
 =?us-ascii?Q?z0TMDzK6wnwBUYaZUDmOkEUG+gTg71g+Y2+/66Fd2/Iq2aCbhGSHyC8XO1gI?=
 =?us-ascii?Q?wClvICTDprvqTT65rD5oQ5duS6hVEKhGntJM2O5yVnydWm4/YcgP+u3SDhBJ?=
 =?us-ascii?Q?bqZ1qOg+67IHYmz6tMG+I9dggpHWeCTpLJLOS/NPfF7S2UYS5T36rGEAVC7O?=
 =?us-ascii?Q?KU0NFX8et8kip4mOlghQdujQATbO1VUh0YmImVLprhvejTmz7gYeRoxmPZzc?=
 =?us-ascii?Q?vyURd62N/4Riv0USYE+6gNTc?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 52a3c87e-93a2-4615-7419-08d9785f405f
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:41:14.4081
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 95l4bNYpqLPIt8DCGglG3iq9trViuNpC2WMxH6riGLSIXKUaPjqQbOJBipkzOxlmoU3LgS1J9IHdQM23RMbHwOwzFjFDg9s5xC2glEChu8Q=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0783

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> Hyperv provides GHCB protocol to write Synthetic Interrupt
> Controller MSR registers in Isolation VM with AMD SEV SNP
> and these registers are emulated by hypervisor directly.
> Hyperv requires to write SINTx MSR registers twice. First
> writes MSR via GHCB page to communicate with hypervisor
> and then writes wrmsr instruction to talk with paravisor
> which runs in VMPL0. Guest OS ID MSR also needs to be set
> via GHCB page.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	 * Remove hv_get_simp(), hv_get_siefp()  hv_get_synint_*()
> 	   helper function. Move the logic into hv_get/set_register().
>=20
> Change since v3:
>          * Pass old_msg_type to hv_signal_eom() as parameter.
> 	 * Use HV_REGISTER_* marcro instead of HV_X64_MSR_*
> 	 * Add hv_isolation_type_snp() weak function.
> 	 * Add maros to set syinc register in ARM code.
>=20
> Change since v1:
>          * Introduce sev_es_ghcb_hv_call_simple() and share code
> 	   between SEV and Hyper-V code.
>=20
> Fix for hyperv: Add Write/Read MSR registers via ghcb page
> ---
>  arch/x86/hyperv/hv_init.c       |  36 +++--------
>  arch/x86/hyperv/ivm.c           | 103 ++++++++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h |  56 ++++++++++++-----
>  arch/x86/include/asm/sev.h      |   6 ++
>  arch/x86/kernel/sev-shared.c    |  63 +++++++++++--------
>  drivers/hv/hv.c                 |  77 +++++++++++++++++++-----
>  drivers/hv/hv_common.c          |   6 ++
>  include/asm-generic/mshyperv.h  |   2 +
>  8 files changed, 266 insertions(+), 83 deletions(-)
>=20
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index d57df6825527..a16a83e46a30 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -37,7 +37,7 @@ EXPORT_SYMBOL_GPL(hv_current_partition_id);
>  void *hv_hypercall_pg;
>  EXPORT_SYMBOL_GPL(hv_hypercall_pg);
>=20
> -void __percpu **hv_ghcb_pg;
> +union hv_ghcb __percpu **hv_ghcb_pg;
>=20
>  /* Storage to save the hypercall page temporarily for hibernation */
>  static void *hv_hypercall_pg_saved;
> @@ -406,7 +406,7 @@ void __init hyperv_init(void)
>  	}
>=20
>  	if (hv_isolation_type_snp()) {
> -		hv_ghcb_pg =3D alloc_percpu(void *);
> +		hv_ghcb_pg =3D alloc_percpu(union hv_ghcb *);
>  		if (!hv_ghcb_pg)
>  			goto free_vp_assist_page;
>  	}
> @@ -424,6 +424,9 @@ void __init hyperv_init(void)
>  	guest_id =3D generate_guest_id(0, LINUX_VERSION_CODE, 0);
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, guest_id);
>=20
> +	/* Hyper-V requires to write guest os id via ghcb in SNP IVM. */
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, guest_id);
> +
>  	hv_hypercall_pg =3D __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START,
>  			VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX,
>  			VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
> @@ -501,6 +504,7 @@ void __init hyperv_init(void)
>=20
>  clean_guest_os_id:
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
>  	cpuhp_remove_state(cpuhp);
>  free_ghcb_page:
>  	free_percpu(hv_ghcb_pg);
> @@ -522,6 +526,7 @@ void hyperv_cleanup(void)
>=20
>  	/* Reset our OS id */
>  	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> +	hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0);
>=20
>  	/*
>  	 * Reset hypercall page reference before reset the page,
> @@ -592,30 +597,3 @@ bool hv_is_hyperv_initialized(void)
>  	return hypercall_msr.enable;
>  }
>  EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
> -
> -enum hv_isolation_type hv_get_isolation_type(void)
> -{
> -	if (!(ms_hyperv.priv_high & HV_ISOLATION))
> -		return HV_ISOLATION_TYPE_NONE;
> -	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
> -}
> -EXPORT_SYMBOL_GPL(hv_get_isolation_type);
> -
> -bool hv_is_isolation_supported(void)
> -{
> -	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
> -		return false;
> -
> -	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
> -		return false;
> -
> -	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
> -}
> -
> -DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
> -
> -bool hv_isolation_type_snp(void)
> -{
> -	return static_branch_unlikely(&isolation_type_snp);
> -}
> -EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
> diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c
> index 79e7fb83472a..5439723446c9 100644
> --- a/arch/x86/hyperv/ivm.c
> +++ b/arch/x86/hyperv/ivm.c
> @@ -6,12 +6,115 @@
>   *  Tianyu Lan <Tianyu.Lan@microsoft.com>
>   */
>=20
> +#include <linux/types.h>
> +#include <linux/bitfield.h>
>  #include <linux/hyperv.h>
>  #include <linux/types.h>
>  #include <linux/bitfield.h>
>  #include <linux/slab.h>
> +#include <asm/svm.h>
> +#include <asm/sev.h>
>  #include <asm/io.h>
>  #include <asm/mshyperv.h>
> +#include <asm/hypervisor.h>
> +
> +union hv_ghcb {
> +	struct ghcb ghcb;
> +} __packed __aligned(HV_HYP_PAGE_SIZE);
> +
> +void hv_ghcb_msr_write(u64 msr, u64 value)
> +{
> +	union hv_ghcb *hv_ghcb;
> +	void **ghcb_base;
> +	unsigned long flags;
> +
> +	if (!hv_ghcb_pg)
> +		return;
> +
> +	WARN_ON(in_nmi());
> +
> +	local_irq_save(flags);
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	hv_ghcb =3D (union hv_ghcb *)*ghcb_base;
> +	if (!hv_ghcb) {
> +		local_irq_restore(flags);
> +		return;
> +	}
> +
> +	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
> +	ghcb_set_rax(&hv_ghcb->ghcb, lower_32_bits(value));
> +	ghcb_set_rdx(&hv_ghcb->ghcb, upper_32_bits(value));
> +
> +	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 1, 0))
> +		pr_warn("Fail to write msr via ghcb %llx.\n", msr);
> +
> +	local_irq_restore(flags);
> +}
> +
> +void hv_ghcb_msr_read(u64 msr, u64 *value)
> +{
> +	union hv_ghcb *hv_ghcb;
> +	void **ghcb_base;
> +	unsigned long flags;
> +
> +	/* Check size of union hv_ghcb here. */
> +	BUILD_BUG_ON(sizeof(union hv_ghcb) !=3D HV_HYP_PAGE_SIZE);
> +
> +	if (!hv_ghcb_pg)
> +		return;
> +
> +	WARN_ON(in_nmi());
> +
> +	local_irq_save(flags);
> +	ghcb_base =3D (void **)this_cpu_ptr(hv_ghcb_pg);
> +	hv_ghcb =3D (union hv_ghcb *)*ghcb_base;
> +	if (!hv_ghcb) {
> +		local_irq_restore(flags);
> +		return;
> +	}
> +
> +	ghcb_set_rcx(&hv_ghcb->ghcb, msr);
> +	if (sev_es_ghcb_hv_call_simple(&hv_ghcb->ghcb, SVM_EXIT_MSR, 0, 0))
> +		pr_warn("Fail to read msr via ghcb %llx.\n", msr);
> +	else
> +		*value =3D (u64)lower_32_bits(hv_ghcb->ghcb.save.rax)
> +			| ((u64)lower_32_bits(hv_ghcb->ghcb.save.rdx) << 32);
> +	local_irq_restore(flags);
> +}
> +
> +enum hv_isolation_type hv_get_isolation_type(void)
> +{
> +	if (!(ms_hyperv.priv_high & HV_ISOLATION))
> +		return HV_ISOLATION_TYPE_NONE;
> +	return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b);
> +}
> +EXPORT_SYMBOL_GPL(hv_get_isolation_type);
> +
> +/*
> + * hv_is_isolation_supported - Check system runs in the Hyper-V
> + * isolation VM.
> + */
> +bool hv_is_isolation_supported(void)
> +{
> +	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
> +		return false;
> +
> +	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
> +		return false;
> +
> +	return hv_get_isolation_type() !=3D HV_ISOLATION_TYPE_NONE;
> +}
> +
> +DEFINE_STATIC_KEY_FALSE(isolation_type_snp);
> +
> +/*
> + * hv_isolation_type_snp - Check system runs in the AMD SEV-SNP based
> + * isolation VM.
> + */
> +bool hv_isolation_type_snp(void)
> +{
> +	return static_branch_unlikely(&isolation_type_snp);
> +}
>=20
>  /*
>   * hv_mark_gpa_visibility - Set pages visible to host via hvcall.
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index ede440f9a1e2..165423e8b67a 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -11,25 +11,14 @@
>  #include <asm/paravirt.h>
>  #include <asm/mshyperv.h>
>=20
> +union hv_ghcb;
> +
>  DECLARE_STATIC_KEY_FALSE(isolation_type_snp);
>=20
>  typedef int (*hyperv_fill_flush_list_func)(
>  		struct hv_guest_mapping_flush_list *flush,
>  		void *data);
>=20
> -static inline void hv_set_register(unsigned int reg, u64 value)
> -{
> -	wrmsrl(reg, value);
> -}
> -
> -static inline u64 hv_get_register(unsigned int reg)
> -{
> -	u64 value;
> -
> -	rdmsrl(reg, value);
> -	return value;
> -}
> -
>  #define hv_get_raw_timer() rdtsc_ordered()
>=20
>  void hyperv_vector_handler(struct pt_regs *regs);
> @@ -41,7 +30,7 @@ extern void *hv_hypercall_pg;
>=20
>  extern u64 hv_current_partition_id;
>=20
> -extern void __percpu **hv_ghcb_pg;
> +extern union hv_ghcb  __percpu **hv_ghcb_pg;
>=20
>  int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
>  int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
> @@ -193,6 +182,8 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level=
, int vcpu, int vector,
>  		struct hv_interrupt_entry *entry);
>  int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *=
entry);
>  int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible);
> +void hv_ghcb_msr_write(u64 msr, u64 value);
> +void hv_ghcb_msr_read(u64 msr, u64 *value);
>  #else /* CONFIG_HYPERV */
>  static inline void hyperv_init(void) {}
>  static inline void hyperv_setup_mmu_ops(void) {}
> @@ -209,9 +200,46 @@ static inline int hyperv_flush_guest_mapping_range(u=
64 as,
>  {
>  	return -1;
>  }
> +
> +static inline void hv_ghcb_msr_write(u64 msr, u64 value) {}
> +static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {}
>  #endif /* CONFIG_HYPERV */
>=20
> +static inline void hv_set_register(unsigned int reg, u64 value);

I'm not seeing why this declaration is needed.

>=20
>  #include <asm-generic/mshyperv.h>
>=20
> +static inline bool hv_is_synic_reg(unsigned int reg)
> +{
> +	if ((reg >=3D HV_REGISTER_SCONTROL) &&
> +	    (reg <=3D HV_REGISTER_SINT15))
> +		return true;
> +	return false;
> +}
> +
> +static inline u64 hv_get_register(unsigned int reg)
> +{
> +	u64 value;
> +
> +	if (hv_is_synic_reg(reg) && hv_isolation_type_snp())
> +		hv_ghcb_msr_read(reg, &value);
> +	else
> +		rdmsrl(reg, value);
> +	return value;
> +}
> +
> +static inline void hv_set_register(unsigned int reg, u64 value)
> +{
> +	if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) {
> +		hv_ghcb_msr_write(reg, value);
> +
> +		/* Write proxy bit via wrmsl instruction */
> +		if (reg >=3D HV_REGISTER_SINT0 &&
> +		    reg <=3D HV_REGISTER_SINT15)
> +			wrmsrl(reg, value | 1 << 20);
> +	} else {
> +		wrmsrl(reg, value);
> +	}
> +}
> +

This all looks OK to me, except that it would really be nice if the
#include of asm-generic/mshyperv.h stays last in the file.  I think the
problem is needing a declaration for hv_isolation_type_snp(), right?
And it is added into asm-generic/mshyperv.h at the very end of this
patch.

The alternative would be to put hv_get_register() and
hv_set_register() in a .c file rather than as static inline.  They get
called in quite a few places, and arguably are now fairly large for
being static inline, in my judgment.  But I guess I'm OK either way.

In previous versions, the EOM register was being handled
differently (GHCB vs. MSR write) for timer messages vs. other messages.
That distinction is no longer being made.  Did you learn something new
from the Hyper-V team about this?  Just want to make sure nothing
was inadvertently dropped.

>  #endif
> diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
> index fa5cd05d3b5b..60bfdbd141b1 100644
> --- a/arch/x86/include/asm/sev.h
> +++ b/arch/x86/include/asm/sev.h
> @@ -81,12 +81,18 @@ static __always_inline void sev_es_nmi_complete(void)
>  		__sev_es_nmi_complete();
>  }
>  extern int __init sev_es_efi_map_ghcbs(pgd_t *pgd);
> +extern enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2);
>  #else
>  static inline void sev_es_ist_enter(struct pt_regs *regs) { }
>  static inline void sev_es_ist_exit(void) { }
>  static inline int sev_es_setup_ap_jump_table(struct real_mode_header *rm=
h) { return 0; }
>  static inline void sev_es_nmi_complete(void) { }
>  static inline int sev_es_efi_map_ghcbs(pgd_t *pgd) { return 0; }
> +static inline enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghc=
b,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2) { return ES_VMM_ERROR; }
>  #endif
>=20
>  #endif
> diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
> index 9f90f460a28c..dd7f37de640b 100644
> --- a/arch/x86/kernel/sev-shared.c
> +++ b/arch/x86/kernel/sev-shared.c
> @@ -94,10 +94,9 @@ static void vc_finish_insn(struct es_em_ctxt *ctxt)
>  	ctxt->regs->ip +=3D ctxt->insn.length;
>  }
>=20
> -static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
> -					  struct es_em_ctxt *ctxt,
> -					  u64 exit_code, u64 exit_info_1,
> -					  u64 exit_info_2)
> +enum es_result sev_es_ghcb_hv_call_simple(struct ghcb *ghcb,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2)
>  {
>  	enum es_result ret;
>=20
> @@ -109,29 +108,45 @@ static enum es_result sev_es_ghcb_hv_call(struct gh=
cb *ghcb,
>  	ghcb_set_sw_exit_info_1(ghcb, exit_info_1);
>  	ghcb_set_sw_exit_info_2(ghcb, exit_info_2);
>=20
> -	sev_es_wr_ghcb_msr(__pa(ghcb));
>  	VMGEXIT();
>=20
> -	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) =3D=3D 1) {
> -		u64 info =3D ghcb->save.sw_exit_info_2;
> -		unsigned long v;
> -
> -		info =3D ghcb->save.sw_exit_info_2;
> -		v =3D info & SVM_EVTINJ_VEC_MASK;
> -
> -		/* Check if exception information from hypervisor is sane. */
> -		if ((info & SVM_EVTINJ_VALID) &&
> -		    ((v =3D=3D X86_TRAP_GP) || (v =3D=3D X86_TRAP_UD)) &&
> -		    ((info & SVM_EVTINJ_TYPE_MASK) =3D=3D SVM_EVTINJ_TYPE_EXEPT)) {
> -			ctxt->fi.vector =3D v;
> -			if (info & SVM_EVTINJ_VALID_ERR)
> -				ctxt->fi.error_code =3D info >> 32;
> -			ret =3D ES_EXCEPTION;
> -		} else {
> -			ret =3D ES_VMM_ERROR;
> -		}
> -	} else {
> +	if ((ghcb->save.sw_exit_info_1 & 0xffffffff) =3D=3D 1)
> +		ret =3D ES_VMM_ERROR;
> +	else
>  		ret =3D ES_OK;
> +
> +	return ret;
> +}
> +
> +static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
> +				   struct es_em_ctxt *ctxt,
> +				   u64 exit_code, u64 exit_info_1,
> +				   u64 exit_info_2)
> +{
> +	unsigned long v;
> +	enum es_result ret;
> +	u64 info;
> +
> +	sev_es_wr_ghcb_msr(__pa(ghcb));
> +
> +	ret =3D sev_es_ghcb_hv_call_simple(ghcb, exit_code, exit_info_1,
> +					 exit_info_2);
> +	if (ret =3D=3D ES_OK)
> +		return ret;
> +
> +	info =3D ghcb->save.sw_exit_info_2;
> +	v =3D info & SVM_EVTINJ_VEC_MASK;
> +
> +	/* Check if exception information from hypervisor is sane. */
> +	if ((info & SVM_EVTINJ_VALID) &&
> +	    ((v =3D=3D X86_TRAP_GP) || (v =3D=3D X86_TRAP_UD)) &&
> +	    ((info & SVM_EVTINJ_TYPE_MASK) =3D=3D SVM_EVTINJ_TYPE_EXEPT)) {
> +		ctxt->fi.vector =3D v;
> +		if (info & SVM_EVTINJ_VALID_ERR)
> +			ctxt->fi.error_code =3D info >> 32;
> +		ret =3D ES_EXCEPTION;
> +	} else {
> +		ret =3D ES_VMM_ERROR;
>  	}
>=20
>  	return ret;
> diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
> index e83507f49676..dee1a96bc535 100644
> --- a/drivers/hv/hv.c
> +++ b/drivers/hv/hv.c
> @@ -8,6 +8,7 @@
>   */
>  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>=20
> +#include <linux/io.h>
>  #include <linux/kernel.h>
>  #include <linux/mm.h>
>  #include <linux/slab.h>
> @@ -136,17 +137,24 @@ int hv_synic_alloc(void)
>  		tasklet_init(&hv_cpu->msg_dpc,
>  			     vmbus_on_msg_dpc, (unsigned long) hv_cpu);
>=20
> -		hv_cpu->synic_message_page =3D
> -			(void *)get_zeroed_page(GFP_ATOMIC);
> -		if (hv_cpu->synic_message_page =3D=3D NULL) {
> -			pr_err("Unable to allocate SYNIC message page\n");
> -			goto err;
> -		}
> +		/*
> +		 * Synic message and event pages are allocated by paravisor.
> +		 * Skip these pages allocation here.
> +		 */
> +		if (!hv_isolation_type_snp()) {
> +			hv_cpu->synic_message_page =3D
> +				(void *)get_zeroed_page(GFP_ATOMIC);
> +			if (hv_cpu->synic_message_page =3D=3D NULL) {
> +				pr_err("Unable to allocate SYNIC message page\n");
> +				goto err;
> +			}
>=20
> -		hv_cpu->synic_event_page =3D (void *)get_zeroed_page(GFP_ATOMIC);
> -		if (hv_cpu->synic_event_page =3D=3D NULL) {
> -			pr_err("Unable to allocate SYNIC event page\n");
> -			goto err;
> +			hv_cpu->synic_event_page =3D
> +				(void *)get_zeroed_page(GFP_ATOMIC);
> +			if (hv_cpu->synic_event_page =3D=3D NULL) {
> +				pr_err("Unable to allocate SYNIC event page\n");
> +				goto err;
> +			}
>  		}
>=20
>  		hv_cpu->post_msg_page =3D (void *)get_zeroed_page(GFP_ATOMIC);
> @@ -201,16 +209,35 @@ void hv_synic_enable_regs(unsigned int cpu)
>  	/* Setup the Synic's message page */
>  	simp.as_uint64 =3D hv_get_register(HV_REGISTER_SIMP);
>  	simp.simp_enabled =3D 1;
> -	simp.base_simp_gpa =3D virt_to_phys(hv_cpu->synic_message_page)
> -		>> HV_HYP_PAGE_SHIFT;
> +
> +	if (hv_isolation_type_snp()) {
> +		hv_cpu->synic_message_page
> +			=3D memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT,
> +				   HV_HYP_PAGE_SIZE, MEMREMAP_WB);
> +		if (!hv_cpu->synic_message_page)
> +			pr_err("Fail to map syinc message page.\n");
> +	} else {
> +		simp.base_simp_gpa =3D virt_to_phys(hv_cpu->synic_message_page)
> +			>> HV_HYP_PAGE_SHIFT;
> +	}
>=20
>  	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
>=20
>  	/* Setup the Synic's event page */
>  	siefp.as_uint64 =3D hv_get_register(HV_REGISTER_SIEFP);
>  	siefp.siefp_enabled =3D 1;
> -	siefp.base_siefp_gpa =3D virt_to_phys(hv_cpu->synic_event_page)
> -		>> HV_HYP_PAGE_SHIFT;
> +
> +	if (hv_isolation_type_snp()) {
> +		hv_cpu->synic_event_page =3D
> +			memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT,
> +				 HV_HYP_PAGE_SIZE, MEMREMAP_WB);
> +
> +		if (!hv_cpu->synic_event_page)
> +			pr_err("Fail to map syinc event page.\n");
> +	} else {
> +		siefp.base_siefp_gpa =3D virt_to_phys(hv_cpu->synic_event_page)
> +			>> HV_HYP_PAGE_SHIFT;
> +	}
>=20
>  	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
>=20
> @@ -257,30 +284,48 @@ int hv_synic_init(unsigned int cpu)
>   */
>  void hv_synic_disable_regs(unsigned int cpu)
>  {
> +	struct hv_per_cpu_context *hv_cpu
> +		=3D per_cpu_ptr(hv_context.cpu_context, cpu);
>  	union hv_synic_sint shared_sint;
>  	union hv_synic_simp simp;
>  	union hv_synic_siefp siefp;
>  	union hv_synic_scontrol sctrl;
>=20
> +

Spurious blank line?

>  	shared_sint.as_uint64 =3D hv_get_register(HV_REGISTER_SINT0 +
>  					VMBUS_MESSAGE_SINT);
>=20
>  	shared_sint.masked =3D 1;
>=20
> +
> +

Spurious blank lines?

>  	/* Need to correctly cleanup in the case of SMP!!! */
>  	/* Disable the interrupt */
>  	hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT,
>  				shared_sint.as_uint64);
>=20
>  	simp.as_uint64 =3D hv_get_register(HV_REGISTER_SIMP);
> +	/*
> +	 * In Isolation VM, sim and sief pages are allocated by
> +	 * paravisor. These pages also will be used by kdump
> +	 * kernel. So just reset enable bit here and keep page
> +	 * addresses.
> +	 */
>  	simp.simp_enabled =3D 0;
> -	simp.base_simp_gpa =3D 0;
> +	if (hv_isolation_type_snp())
> +		memunmap(hv_cpu->synic_message_page);
> +	else
> +		simp.base_simp_gpa =3D 0;
>=20
>  	hv_set_register(HV_REGISTER_SIMP, simp.as_uint64);
>=20
>  	siefp.as_uint64 =3D hv_get_register(HV_REGISTER_SIEFP);
>  	siefp.siefp_enabled =3D 0;
> -	siefp.base_siefp_gpa =3D 0;
> +
> +	if (hv_isolation_type_snp())
> +		memunmap(hv_cpu->synic_event_page);
> +	else
> +		siefp.base_siefp_gpa =3D 0;
>=20
>  	hv_set_register(HV_REGISTER_SIEFP, siefp.as_uint64);
>=20
> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> index c0d9048a4112..1fc82d237161 100644
> --- a/drivers/hv/hv_common.c
> +++ b/drivers/hv/hv_common.c
> @@ -249,6 +249,12 @@ bool __weak hv_is_isolation_supported(void)
>  }
>  EXPORT_SYMBOL_GPL(hv_is_isolation_supported);
>=20
> +bool __weak hv_isolation_type_snp(void)
> +{
> +	return false;
> +}
> +EXPORT_SYMBOL_GPL(hv_isolation_type_snp);
> +
>  void __weak hv_setup_vmbus_handler(void (*handler)(void))
>  {
>  }
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyper=
v.h
> index cb529c85c0ad..94750bafd4cc 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -24,6 +24,7 @@
>  #include <linux/cpumask.h>
>  #include <linux/nmi.h>
>  #include <asm/ptrace.h>
> +#include <asm/mshyperv.h>

This #include should not be done.  The architecture specific version
of mshyperv.h #includes the asm-generic version, not the other
way around.

In any case, I'm not seeing that this #include is needed.

>  #include <asm/hyperv-tlfs.h>
>=20
>  struct ms_hyperv_info {
> @@ -54,6 +55,7 @@ extern void  __percpu  **hyperv_pcpu_output_arg;
>=20
>  extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputadd=
r);
>  extern u64 hv_do_fast_hypercall8(u16 control, u64 input8);
> +extern bool hv_isolation_type_snp(void);
>=20
>  /* Helper functions that provide a consistent pattern for checking Hyper=
-V hypercall status. */
>  static inline int hv_result(u64 status)
> --
> 2.25.1


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:42:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:42:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187772.336751 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX2x-0007A2-Pp; Wed, 15 Sep 2021 15:42:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187772.336751; Wed, 15 Sep 2021 15:42:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX2x-00079v-MZ; Wed, 15 Sep 2021 15:42:03 +0000
Received: by outflank-mailman (input) for mailman id 187772;
 Wed, 15 Sep 2021 15:42:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX2w-00079j-RR
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:42:02 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7821708e-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:42:01 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MWHPR21MB0783.namprd21.prod.outlook.com (2603:10b6:300:77::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.1; Wed, 15 Sep
 2021 15:41:55 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:41:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7821708e-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Oj3Kbpw5bPy0Oq0iGwhWJKBOhtiUv2eSJ6ogKlkbyKjQWA4zbJqdI51ZU6QAEa0V8imOLv49A+PScsIE5ys5Xc4N0YVJ4rSIXnP3Zq7OxS9Oar1rUJ10ZG67NU1rl0RYGz0xO+ia2vhz5H5kdCHa2Pe1nJrxo1T+UIBE7ouU4j0nriO0hnp0RHpXkSrJHok7FqZAV79rOO+JC1kVSIH7BWo92DnqpxtZtwVgaWsIIsc+hG6Oxar42MFHgY89MugZcrd72oNGRzK4biAjm8b4WenI1A8zOdJznJGqdV9PJVh6TI5058WlPIvlePSwvBr81s4BY66T6OpidQnUWetGrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=F/918hg1YuX4z9H7s+A+X/uldHZBwBC2qP5cL5HAq7U=;
 b=c6anx5doZ7xjEd4dZAJj7N5uKX7KbvLbHFvoP37kiZOPb3WrgHAFWdDeUEuDhxd5xtTvOT8zXKmDu7+AnKe2QT44wL290hnI8t+M4ipLJOjCAC14vgByXxM0HI6478h9HuI3+3jvDsH3cCC7YlgORhtYb3Z5fWsQ9tcUVA3+8H8EIfd3CYMGmvZtfchySBHm3Nu4vZzEvxfaZALAVyM1mrza2XouJ3RYxD6+NNuttrCZGX2Kr259dnC0j3V1n0XCQeXOjkG+aQ/s8GXSLKc+JoL6Jsort7GPe1XmvsL/YfyxpbT3z2tbdQizW5G+PBbrLVbQtl3UUnu8fldHFWbixQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=F/918hg1YuX4z9H7s+A+X/uldHZBwBC2qP5cL5HAq7U=;
 b=H9j+UEgzzhMp7xaBJ3R+wx3E9ALA6nb/IH4dbAZcdHbT0Upfho15di3+0J4xQAKvXqNM8TCdPZAUZEZZa7c9r3BahRQVfSVdGJIa4kA+IiQVwpQcIBhgyFOsy8ZKYZmRDAXWlNsZVvq9JC4VAd9tezIbueqPpbYRC+6lHeNxrVE=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 07/12] Drivers: hv: vmbus: Add SNP support for VMbus
 channel initiate  message
Thread-Topic: [PATCH V5 07/12] Drivers: hv: vmbus: Add SNP support for VMbus
 channel initiate  message
Thread-Index: AQHXqW4CCd5VufaU3kyHpkfVIyVvMaulLo1w
Date: Wed, 15 Sep 2021 15:41:55 +0000
Message-ID:
 <MWHPR21MB1593D9CB27D41B128BF21DC9D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-8-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-8-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=08510df9-4063-4ea9-b787-b1133475bb7b;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-15T14:48:11Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9a283cbd-86ce-40f3-64ac-08d9785f58e9
x-ms-traffictypediagnostic: MWHPR21MB0783:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MWHPR21MB0783EE0DFCF6600D36BCD94BD7DB9@MWHPR21MB0783.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2803;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 EecLWhEpd9Yp8oSk4zhS0TGn/JGW3LxPvSYij5heKFZzKQ/lIkNfgacvuggazXz5vfNFe9it7HNRD+lnlWgQRvqfuOQg7nRwCHTKUH5954CoQfphLsYZoQI3AzYAx82hA1I3Dvfe5VSXn82DHAloXSW7aIvCmxTEmiTOf13hYVPsUN4WJQuLpahttNDcRL+bABTEuUQB2SFKNzyxfT8Y47+RMXX/QiAzsF/V245sZGYK0ibzbZH5/eajdSIfmud1SqmfJhNHrk7yJUEAt0CYTb/pTpo8PS1gafjHIDiAgIXb6Ml75LW2Jhf1Q6lRo+1XNiT4PNvGEdfnxesZMuylRAM72rvnuJJUisjzdHW+gjsPDRnrekz5U5JJzsk7bRCOXQMmQfryrd3OVqJZdCH4cFvdVtBO98d8spah8vyybSUUQMZp8A7mH2xLeDO0a69zrDywq+zCDOW1IsuPNYJfePHJQLVI/cqeHOPNp8mtSfibt3hUJfn7zuzUmXVaXap+ytTCvh+RPy2ESe0+VYt3eFezPNVJuQCINGOdx95riPS3moIUykSd/u+Pz4bkZ/3Wb4ZeUIJvB1gjzc+GHD3QpgMntu/js9n/Vl/xPrL92JoBUvYm2SA4+KK68zOFOvp0zQaJtkz+xvQRHfSpu0pzu1pVPghdP/jG0//KHvCwdtT0yJjSyrHS/EmISSv+c2S5e3jrJVfNjIE8QT/LSVFzx5mSCCsog/Q50TFTcOZUW/g=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(33656002)(7416002)(7406005)(66946007)(26005)(508600001)(9686003)(8936002)(2906002)(8676002)(82960400001)(82950400001)(5660300002)(86362001)(921005)(186003)(122000001)(38070700005)(66556008)(10290500003)(66476007)(66446008)(64756008)(76116006)(38100700002)(316002)(54906003)(55016002)(15650500001)(83380400001)(6506007)(110136005)(71200400001)(52536014)(7696005)(8990500004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?6ds3IGWen8r4I7FNzQCxiRe74jaiPJgyaH9dqCDRzYpdyyg96DjGIK+p5ML/?=
 =?us-ascii?Q?foo9Qn5cUqlkqciKeim3dHhpHsmkgDFE1msLck84xh5de/9/1NDCcamfelS9?=
 =?us-ascii?Q?AQFaNXwd3piYkHB3hnmwCfThZHbqSJIQ4mvRPdudiuYP4WRLMqYnJILjckgD?=
 =?us-ascii?Q?dT+SHtC3KPORp/vejl/g4KSM49Kqszljdpb0vlKg8pUzp0xHhype1ff8nnX4?=
 =?us-ascii?Q?zyfOQDjRwjrMBTiSLuNFgOCjppwF+a+dg2IFX4xWTus/byV25MWaqjSNm32l?=
 =?us-ascii?Q?vFXM2YRtz2KJhxX+acrvK4GVOMmjUbdN+n/TqhHl0C/VXsU3J5AkgQ5g66KQ?=
 =?us-ascii?Q?SRhyOMvUC7OOBvZGqza/eY6/UMyNeY99HV1i+BPn24rCjHpTNdxxQOd8tGDV?=
 =?us-ascii?Q?M34XUPZFIQ/HhYCZksMQqNIcny5oa3WeT99238V5QAYUY+9tNLgvBz61sZ5t?=
 =?us-ascii?Q?ydDmmAgOMNVk1LV7EPeyuNxAv6EC98DlywWiyW2fcZm6Dp9pAqSJmqA+D4FN?=
 =?us-ascii?Q?rq1+SQjHMMJ9JuetPbHpEMfCWBdnXNdkCn7q+kQWU/kqJs6vStAifo8c4kma?=
 =?us-ascii?Q?eR1kdAy0HNjHkSRWWTJHKeqZlw7MUPehtjV22fS3KveGucHYekCLGg0bWskG?=
 =?us-ascii?Q?JILMU1do4yt4e1sTSbmu4kntlazON6euj4Dr0bBrvhi2AH+fQs9jTF2VyxfK?=
 =?us-ascii?Q?nlwhiOfMGn95DWZWPAC0XpytuYc0MP2UPCLViBCMQ0i2W/Vyxnkx/XlFa1RM?=
 =?us-ascii?Q?SIN8Arw8jUDXFqeWKiN3A07y7H3LBUKZ8ggL6DU7/N56FXHyTYRp13LaElf/?=
 =?us-ascii?Q?zdMsR+E1ki0X84IsansA4qUK6y/5o8QuU4FZzf4SmsM2MzvRwn/VoVlmUScJ?=
 =?us-ascii?Q?yAKtg2LrsY9pvM6SKviBRpjXfn+oqq1VS/Q0XDHyF0jtFKsF2KVO0shA2WpL?=
 =?us-ascii?Q?ogzJAj4ODlLbE8ATnbAdo/BfhRq6sI4799PwdWXZ9Trd7wYxC76+HWFeyKyO?=
 =?us-ascii?Q?JK5eiNf71nkCKI8kBDFNG6PcIteDbsri+vtsmnwseCH98TxEeap9CzwN2DVu?=
 =?us-ascii?Q?nxc9fxqP5V42qKNh16WYENbVN4pOg3W6kc/OD8TMzzReZNhjEPFRcVYsqzN2?=
 =?us-ascii?Q?1DB4vklqVxHyJHytFCAOSfArsTmV2QPVU9pKPg1tfDcLSJmEgbD+eO4KmOZr?=
 =?us-ascii?Q?26FTlHcEpcfc9wQMlBPLmXBkS04qxxi8JqlTFhu/9GhSbr/73uyMk+0LNpvf?=
 =?us-ascii?Q?/q2FJV6rDRzcOZQdl2cffek7+3v2wniinf+ksnf6GDXXOCENgBY7Qkof7kmr?=
 =?us-ascii?Q?pgtme/KnhQ5yZnz3PmRzLSnW?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a283cbd-86ce-40f3-64ac-08d9785f58e9
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:41:55.5834
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sutlvI5gLKCFm8+kq4Fx9lAI3owHHagLGcVPeuWtzcY6JebzZmx+KV4pMzuRNIq2weYyEKIwS9D4wAVfDwZsBMR31J/PXq+Q1hJUfNL9bP4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0783

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> The monitor pages in the CHANNELMSG_INITIATE_CONTACT msg are shared
> with host in Isolation VM and so it's necessary to use hvcall to set
> them visible to host. In Isolation VM with AMD SEV SNP, the access
> address should be in the extra space which is above shared gpa
> boundary. So remap these pages into the extra address(pa +
> shared_gpa_boundary).
>=20
> Introduce monitor_pages_original[] in the struct vmbus_connection
> to store monitor page virtual address returned by hv_alloc_hyperv_
> zeroed_page() and free monitor page via monitor_pages_original in
> the vmbus_disconnect(). The monitor_pages[] is to used to access
> monitor page and it is initialized to be equal with monitor_pages_
> original. The monitor_pages[] will be overridden in the isolation VM
> with va of extra address. Introduce monitor_pages_pa[] to store
> monitor pages' physical address and use it to populate pa in the
> initiate msg.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	* Introduce monitor_pages_pa[] to store monitor pages' physical
> 	  address and use it to populate pa in the initiate msg.
> 	* Move code of mapping moniter pages in extra address into
> 	  vmbus_connect().
>=20
> Change since v3:
> 	* Rename monitor_pages_va with monitor_pages_original
> 	* free monitor page via monitor_pages_original and
> 	  monitor_pages is used to access monitor page.
>=20
> Change since v1:
>         * Not remap monitor pages in the non-SNP isolation VM.
> ---
>  drivers/hv/connection.c   | 90 ++++++++++++++++++++++++++++++++++++---
>  drivers/hv/hyperv_vmbus.h |  2 +
>  2 files changed, 86 insertions(+), 6 deletions(-)
>=20
> diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
> index 8820ae68f20f..edd8f7dd169f 100644
> --- a/drivers/hv/connection.c
> +++ b/drivers/hv/connection.c
> @@ -19,6 +19,8 @@
>  #include <linux/vmalloc.h>
>  #include <linux/hyperv.h>
>  #include <linux/export.h>
> +#include <linux/io.h>
> +#include <linux/set_memory.h>
>  #include <asm/mshyperv.h>
>=20
>  #include "hyperv_vmbus.h"
> @@ -102,8 +104,9 @@ int vmbus_negotiate_version(struct vmbus_channel_msgi=
nfo *msginfo, u32 version)
>  		vmbus_connection.msg_conn_id =3D VMBUS_MESSAGE_CONNECTION_ID;
>  	}
>=20
> -	msg->monitor_page1 =3D virt_to_phys(vmbus_connection.monitor_pages[0]);
> -	msg->monitor_page2 =3D virt_to_phys(vmbus_connection.monitor_pages[1]);
> +	msg->monitor_page1 =3D vmbus_connection.monitor_pages_pa[0];
> +	msg->monitor_page2 =3D vmbus_connection.monitor_pages_pa[1];
> +
>  	msg->target_vcpu =3D hv_cpu_number_to_vp_number(VMBUS_CONNECT_CPU);
>=20
>  	/*
> @@ -216,6 +219,65 @@ int vmbus_connect(void)
>  		goto cleanup;
>  	}
>=20
> +	vmbus_connection.monitor_pages_original[0]
> +		=3D vmbus_connection.monitor_pages[0];
> +	vmbus_connection.monitor_pages_original[1]
> +		=3D vmbus_connection.monitor_pages[1];
> +	vmbus_connection.monitor_pages_pa[0]
> +		=3D virt_to_phys(vmbus_connection.monitor_pages[0]);
> +	vmbus_connection.monitor_pages_pa[1]
> +		=3D virt_to_phys(vmbus_connection.monitor_pages[1]);
> +
> +	if (hv_is_isolation_supported()) {
> +		vmbus_connection.monitor_pages_pa[0] +=3D
> +			ms_hyperv.shared_gpa_boundary;
> +		vmbus_connection.monitor_pages_pa[1] +=3D
> +			ms_hyperv.shared_gpa_boundary;
> +
> +		ret =3D set_memory_decrypted((unsigned long)
> +					   vmbus_connection.monitor_pages[0],
> +					   1);
> +		ret |=3D set_memory_decrypted((unsigned long)
> +					    vmbus_connection.monitor_pages[1],
> +					    1);
> +		if (ret)
> +			goto cleanup;
> +
> +		/*
> +		 * Isolation VM with AMD SNP needs to access monitor page via
> +		 * address space above shared gpa boundary.
> +		 */
> +		if (hv_isolation_type_snp()) {
> +			vmbus_connection.monitor_pages[0]
> +				=3D memremap(vmbus_connection.monitor_pages_pa[0],
> +					   HV_HYP_PAGE_SIZE,
> +					   MEMREMAP_WB);
> +			if (!vmbus_connection.monitor_pages[0]) {
> +				ret =3D -ENOMEM;
> +				goto cleanup;
> +			}
> +
> +			vmbus_connection.monitor_pages[1]
> +				=3D memremap(vmbus_connection.monitor_pages_pa[1],
> +					   HV_HYP_PAGE_SIZE,
> +					   MEMREMAP_WB);
> +			if (!vmbus_connection.monitor_pages[1]) {
> +				ret =3D -ENOMEM;
> +				goto cleanup;
> +			}
> +		}
> +
> +		/*
> +		 * Set memory host visibility hvcall smears memory
> +		 * and so zero monitor pages here.
> +		 */
> +		memset(vmbus_connection.monitor_pages[0], 0x00,
> +		       HV_HYP_PAGE_SIZE);
> +		memset(vmbus_connection.monitor_pages[1], 0x00,
> +		       HV_HYP_PAGE_SIZE);
> +
> +	}
> +

This all looks good.  To me, this is a lot clearer to have all the mapping
and encryption/decryption handled in one place.

>  	msginfo =3D kzalloc(sizeof(*msginfo) +
>  			  sizeof(struct vmbus_channel_initiate_contact),
>  			  GFP_KERNEL);
> @@ -303,10 +365,26 @@ void vmbus_disconnect(void)
>  		vmbus_connection.int_page =3D NULL;
>  	}
>=20
> -	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[0]);
> -	hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[1]);
> -	vmbus_connection.monitor_pages[0] =3D NULL;
> -	vmbus_connection.monitor_pages[1] =3D NULL;
> +	if (hv_is_isolation_supported()) {
> +		memunmap(vmbus_connection.monitor_pages[0]);
> +		memunmap(vmbus_connection.monitor_pages[1]);
> +
> +		set_memory_encrypted((unsigned long)
> +			vmbus_connection.monitor_pages_original[0],
> +			1);
> +		set_memory_encrypted((unsigned long)
> +			vmbus_connection.monitor_pages_original[1],
> +			1);
> +	}
> +
> +	hv_free_hyperv_page((unsigned long)
> +		vmbus_connection.monitor_pages_original[0]);
> +	hv_free_hyperv_page((unsigned long)
> +		vmbus_connection.monitor_pages_original[1]);
> +	vmbus_connection.monitor_pages_original[0] =3D
> +		vmbus_connection.monitor_pages[0] =3D NULL;
> +	vmbus_connection.monitor_pages_original[1] =3D
> +		vmbus_connection.monitor_pages[1] =3D NULL;
>  }
>=20
>  /*
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 42f3d9d123a1..560cba916d1d 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -240,6 +240,8 @@ struct vmbus_connection {
>  	 * is child->parent notification
>  	 */
>  	struct hv_monitor_page *monitor_pages[2];
> +	void *monitor_pages_original[2];
> +	unsigned long monitor_pages_pa[2];

The type of this field really should be phys_addr_t.  In addition to
just making semantic sense, then it will match the return type from
virt_to_phys() and the input arg to memremap() since resource_size_t
is typedef'ed as phys_addr_t.

>  	struct list_head chn_msg_list;
>  	spinlock_t channelmsg_lock;
>=20
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:42:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:42:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187780.336762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX3b-0007ld-43; Wed, 15 Sep 2021 15:42:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187780.336762; Wed, 15 Sep 2021 15:42:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX3b-0007lW-0K; Wed, 15 Sep 2021 15:42:43 +0000
Received: by outflank-mailman (input) for mailman id 187780;
 Wed, 15 Sep 2021 15:42:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX3a-0007lI-06
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:42:42 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.15])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8f407738-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:42:40 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1819.namprd21.prod.outlook.com (2603:10b6:302:8::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.2; Wed, 15 Sep
 2021 15:42:29 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:42:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f407738-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RtKrp+lzji7XMd98vnSDmCDWfletLqhX00sGIiRD73cIb16nDoQkDGvE3pN2oe71HFHh2fae7NAh+o9A9cUJG1kbGMMR0FqXGeloiFfJxM0FwZ+od0+fMFISyMBI5xm785hYPNbO90c3+S5wopG7FoAl5Xzq5WMpHUvSexmExn585mcQBgCvPvR6vQKL1z7a9yQUFaNlUac+lG+j8OKyklDWcFaRdHxX7aZjnqxgjr0DOKJM62qKwrRHo116GW9T+9lylqesXiCK7ATh8+/loc5wNw8v3I1RqndDa/1H4t6DdQezIy+2b5eNr2GlUfQ3tXBGywAtOWuEAYlr5Gl92w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=BmgFBrbCC5NBlik2Cix7zD+7gpr/CsJhBYJ5DLtJP7Y=;
 b=eXZZn990OBqrqIvpN9oGye2FcerLP6uStqaDsTbq8TdVKy5c3vQRQB2iP7wnKOe5GZ7MUrjC+0vkhkQ90DoC/bWML8s1tYKRNBwcolvGwffFl2qlmdhflz3li/o15o6HUhkBuijrG4nSrfMzQz+q6tlrGfddmwHZJWOstlh9LBoZoNxA0Z1vK1FJiXvNnPasdfFD+ITiOaQFANhztPvHU6j80+b9YKQ6kihYi0dFx0kavy13WADN37JAAvIQ/q581zu6/PPxPpeJBJ56ZlFYki/3t8a42bt2UKlBJWIh5xGzQZyeZId6jfu6KVWniAYtq8p86O/LAIhhxYsKB57yRA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BmgFBrbCC5NBlik2Cix7zD+7gpr/CsJhBYJ5DLtJP7Y=;
 b=GXftKEyuhJh++1UnWeEAC0QWnYCFvPvGVS19qUJcwlUra/4kknm0QX3H3UbafklD0YUWdGxFEBCRqxQeo7TZUNOoXba6qQ65e2/nw6aw96gFRHxc7kzqzrUE6OjfGTaZYOYqqtnPk8ZkV1Mh+Dh7RVOHbUAYvTHHxtMPY92DWJQ=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 09/12] x86/Swiotlb: Add Swiotlb bounce buffer remap
 function for HV IVM
Thread-Topic: [PATCH V5 09/12] x86/Swiotlb: Add Swiotlb bounce buffer remap
 function for HV IVM
Thread-Index: AQHXqW4AcVkxLZ5eA0abh07NwOrvvKulNF0Q
Date: Wed, 15 Sep 2021 15:42:29 +0000
Message-ID:
 <MWHPR21MB159349234C15D0F04F87845CD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-10-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-10-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=58de890b-0740-424d-90fe-6625ce1167d3;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-15T15:09:00Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3711a4b1-0969-4987-2dd8-08d9785f6cec
x-ms-traffictypediagnostic: MW2PR2101MB1819:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB181994E36D2DD88A5BA50479D7DB9@MW2PR2101MB1819.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3276;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 i6IDirtIG/JY8dhPqpLI67X0jaa0NIi01ef16Iln5fgM1oQ5SqZIRiU0HLKRkeetbD2VsbqvQEWuNL8nq2MQN1XJ3Sx/yrgsWCno6fJoZBG8EkY5iXMUO+JXxz/jGPSi0ATVfi8w3wjf26UBFIyExewxg9VzB4do/FjwZj+CViixlLmjUUeHoujzTx/QYyB8tatm0DGC4aZMo844Mmd8JMMwar3pTqndadrldKSUAEDf6aO8U2cYmqzUFNjM8H6mVFEsGmb2UWms4q9ZWDsBHP8ykbIvLKpOW2dKtnNDo/JgxTorLrDmmjaVOPzPEVZ1JxfwQL431y9qepoi+MYF+RERb3yw/MoJCOpva5RH8NvOdv5OuXY5qtNrtHW4xS6d93VQrjyPBgWBggyMMRwTWm7dnf+WdYeV3k60uBrE481GXpSONyQCUWhdP0ee+1ElB+Lo5ZIcmMM+XpFp5e3HYKr2C1wzJLORdMTqRDG3oufDbppyhfLNohzJVKd7hozVdid4/ayHYi19uYAbVYPxctGpIeTvzvubSuN4KVgLYTN47/iLp0fRH6KbigdPJTKUq3jVsLt0MPdehbGlZXVCSrAYp5Fo3K1y6K6rvZCGGlZUiB0W1GDN1tk+2C1WhukGuqpIkEmb1JGlRXG4clCcLkFDvMP91qLX/fOYkhA4whmYTM0pSQTTqonPumXYpaBCgeZ16BH11TZYG0XZZnAVwR0sXmd69b9+UXfx1MhqY8U=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(921005)(71200400001)(110136005)(82950400001)(54906003)(5660300002)(10290500003)(7406005)(6506007)(9686003)(33656002)(8936002)(83380400001)(66946007)(38070700005)(38100700002)(316002)(122000001)(7416002)(55016002)(26005)(66556008)(66446008)(86362001)(76116006)(66476007)(64756008)(8676002)(508600001)(7696005)(2906002)(82960400001)(52536014)(4326008)(186003)(8990500004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?e63sg6qzwihPKEL2sQ0AiD7T/von3zNlILMkOCDOntRohQmYXi8yhQ62u2IC?=
 =?us-ascii?Q?zQVrVzzeqS28QexASAlBYGWVE8KIZZzOLI7e1dKJqd0KDpTJ1tHmdY3iA1Ep?=
 =?us-ascii?Q?4O7faisDjTP1du5xqGKm3kQqMi60TEXxot5LA/J8bSHta5WW2s/3T56VYswl?=
 =?us-ascii?Q?9hmIMRNAADOzAgL21BUHquxLe/H7htXPzad0WgsG6M6x5eIqIwvm6kKN2XBf?=
 =?us-ascii?Q?XaU/wBuwSR/bmGD8WcWSqsEYgyVcxlKdRpXvLzixuFQUQpMF8t0IKn1g4nvR?=
 =?us-ascii?Q?FxSUfp42g0EeWqfeFo+cBUq+2/foBSVtwJG5j9IysfPVtJZkb3UkNILZOmAc?=
 =?us-ascii?Q?WJB2m0l9u2O6Jz1QvQs0cQI7UgXWHuV6cXU4C4pWjAwSFTXy+P2BoJj3jcIJ?=
 =?us-ascii?Q?OGqSDNfZhvMsuMRb1wYqev8b6WRs+JZ/2ONjDo05Y9s8tlbOmtVl+s5vlpQu?=
 =?us-ascii?Q?FkZnkywD+JmEMhYAIY/5eG7u2gByWfQ/x+zvnFfxSEcqA98wzIbESm/7/bM8?=
 =?us-ascii?Q?cAabRqL89E/+v8g7rr/rE0lMnEZ2hSQIi8DSSVcdleZ5SRM/uehRTyEkLbkW?=
 =?us-ascii?Q?50H4h3VbPuNucjaqgVLE0EkYptAsCWCIvlPeRV1kK2UGbfAhAUFrDwnoDH7Z?=
 =?us-ascii?Q?/R6FpRVigwiBuHMecqA7+gJyKG2/fOnv6Zcm5x3sgaqWslaPjKbPlGhJE4e7?=
 =?us-ascii?Q?s+13149KLiP2Jke5Q0P6bALqwtfWKQJrLSOiQ+WDvAUlsR2yqZ1hJuiMEqNQ?=
 =?us-ascii?Q?9lNkKo4dl2bvAWu4QFfcdX+38Rlf33+naaa22lvsEE2xroRFoCqai3tg1r7x?=
 =?us-ascii?Q?j5ZVB4vhO3j+FVeeKXzkoEHAH4MTAY6QPF26rZ16F+8TpentmNOu3FL0JZJS?=
 =?us-ascii?Q?lfMYPAsKfrAqYo9AZRWu472Lhlmo/ZTLzuShXlQNGjQJGtKEdrbgsZgxAPU4?=
 =?us-ascii?Q?EUSQIekX5fy1Fxz6vaR0DPW57nZ5Z/+Unduro+LjdYy4zDE1A3BATkTdxnMw?=
 =?us-ascii?Q?LItU1ujLcD61wgcZk3E/BWwabOWonqAT3Kvw5kfaQGUBz/XrOQMhc3tC/E4N?=
 =?us-ascii?Q?+gp+iszOIOBcCQ2fKlNVPPBbmR3rPiUL7sjHxkP3YO9ojNliFqRLw/Rky7f+?=
 =?us-ascii?Q?6GQi5Lxxii/dBpZ9/1Pcl520UPU7JmKp54X864TWxI4mvd2N95V7bbC5cn3T?=
 =?us-ascii?Q?h57vuF5xSZK+AtEs/aYPM4An/4W1XDrlLspr5WY/tb2y7pQtmMJ+j2wXazfr?=
 =?us-ascii?Q?k9NvkQnvWmN7uspM8uT6ejfXbHLs2dDQWB5LpIO/pgoMo4PUbIOmR3YESXZu?=
 =?us-ascii?Q?Lq3utGvUcnJDcakCx6iaWr/U?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3711a4b1-0969-4987-2dd8-08d9785f6cec
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:42:29.1121
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: OfTbv+rW/eXSEDAIeHRYn6QW2NszlnXQS++cu8KUsMmq5B6uFZqOwTn+HhCrGrFIvF/UaX7eT2ijYBx0HsPtLg7TuAs/iDaGIARXtxHSfuo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1819

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> In Isolation VM with AMD SEV, bounce buffer needs to be accessed via
> extra address space which is above shared_gpa_boundary
> (E.G 39 bit address line) reported by Hyper-V CPUID ISOLATION_CONFIG.
> The access physical address will be original physical address +
> shared_gpa_boundary. The shared_gpa_boundary in the AMD SEV SNP
> spec is called virtual top of memory(vTOM). Memory addresses below
> vTOM are automatically treated as private while memory above
> vTOM is treated as shared.
>=20
> Expose swiotlb_unencrypted_base for platforms to set unencrypted
> memory base offset and call memremap() to map bounce buffer in the
> swiotlb code, store map address and use the address to copy data
> from/to swiotlb bounce buffer.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	* Expose swiotlb_unencrypted_base to set unencrypted memory
> 	  offset.
> 	* Use memremap() to map bounce buffer if swiotlb_unencrypted_
> 	  base is set.
>=20
> Change since v1:
> 	* Make swiotlb_init_io_tlb_mem() return error code and return
>           error when dma_map_decrypted() fails.
> ---
>  include/linux/swiotlb.h |  6 ++++++
>  kernel/dma/swiotlb.c    | 41 +++++++++++++++++++++++++++++++++++------
>  2 files changed, 41 insertions(+), 6 deletions(-)
>=20
> diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
> index b0cb2a9973f4..4998ed44ae3d 100644
> --- a/include/linux/swiotlb.h
> +++ b/include/linux/swiotlb.h
> @@ -72,6 +72,9 @@ extern enum swiotlb_force swiotlb_force;
>   * @end:	The end address of the swiotlb memory pool. Used to do a quick
>   *		range check to see if the memory was in fact allocated by this
>   *		API.
> + * @vaddr:	The vaddr of the swiotlb memory pool. The swiotlb
> + *		memory pool may be remapped in the memory encrypted case and store
> + *		virtual address for bounce buffer operation.
>   * @nslabs:	The number of IO TLB blocks (in groups of 64) between @start=
 and
>   *		@end. For default swiotlb, this is command line adjustable via
>   *		setup_io_tlb_npages.
> @@ -91,6 +94,7 @@ extern enum swiotlb_force swiotlb_force;
>  struct io_tlb_mem {
>  	phys_addr_t start;
>  	phys_addr_t end;
> +	void *vaddr;
>  	unsigned long nslabs;
>  	unsigned long used;
>  	unsigned int index;
> @@ -185,4 +189,6 @@ static inline bool is_swiotlb_for_alloc(struct device=
 *dev)
>  }
>  #endif /* CONFIG_DMA_RESTRICTED_POOL */
>=20
> +extern phys_addr_t swiotlb_unencrypted_base;
> +
>  #endif /* __LINUX_SWIOTLB_H */
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index 87c40517e822..9e30cc4bd872 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -50,6 +50,7 @@
>  #include <asm/io.h>
>  #include <asm/dma.h>
>=20
> +#include <linux/io.h>
>  #include <linux/init.h>
>  #include <linux/memblock.h>
>  #include <linux/iommu-helper.h>
> @@ -72,6 +73,8 @@ enum swiotlb_force swiotlb_force;
>=20
>  struct io_tlb_mem io_tlb_default_mem;
>=20
> +phys_addr_t swiotlb_unencrypted_base;
> +
>  /*
>   * Max segment that we can provide which (if pages are contingous) will
>   * not be bounced (unless SWIOTLB_FORCE is set).
> @@ -175,7 +178,7 @@ void __init swiotlb_update_mem_attributes(void)
>  	memset(vaddr, 0, bytes);
>  }
>=20
> -static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t =
start,
> +static int swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t s=
tart,
>  				    unsigned long nslabs, bool late_alloc)
>  {
>  	void *vaddr =3D phys_to_virt(start);
> @@ -196,13 +199,34 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_m=
em *mem, phys_addr_t start,
>  		mem->slots[i].orig_addr =3D INVALID_PHYS_ADDR;
>  		mem->slots[i].alloc_size =3D 0;
>  	}
> +
> +	if (set_memory_decrypted((unsigned long)vaddr, bytes >> PAGE_SHIFT))
> +		return -EFAULT;
> +
> +	/*
> +	 * Map memory in the unencrypted physical address space when requested
> +	 * (e.g. for Hyper-V AMD SEV-SNP Isolation VMs).
> +	 */
> +	if (swiotlb_unencrypted_base) {
> +		phys_addr_t paddr =3D __pa(vaddr) + swiotlb_unencrypted_base;

Nit:  Use "start" instead of "__pa(vaddr)" since "start" is already the nee=
ded
physical address.

> +
> +		vaddr =3D memremap(paddr, bytes, MEMREMAP_WB);
> +		if (!vaddr) {
> +			pr_err("Failed to map the unencrypted memory.\n");
> +			return -ENOMEM;
> +		}
> +	}
> +
>  	memset(vaddr, 0, bytes);
> +	mem->vaddr =3D vaddr;
> +	return 0;
>  }
>=20
>  int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int ve=
rbose)
>  {
>  	struct io_tlb_mem *mem =3D &io_tlb_default_mem;
>  	size_t alloc_size;
> +	int ret;
>=20
>  	if (swiotlb_force =3D=3D SWIOTLB_NO_FORCE)
>  		return 0;
> @@ -217,7 +241,11 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned=
 long nslabs, int verbose)
>  		panic("%s: Failed to allocate %zu bytes align=3D0x%lx\n",
>  		      __func__, alloc_size, PAGE_SIZE);
>=20
> -	swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
> +	ret =3D swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
> +	if (ret) {
> +		memblock_free(__pa(mem), alloc_size);
> +		return ret;
> +	}
>=20
>  	if (verbose)
>  		swiotlb_print_info();
> @@ -304,7 +332,7 @@ int
>  swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
>  {
>  	struct io_tlb_mem *mem =3D &io_tlb_default_mem;
> -	unsigned long bytes =3D nslabs << IO_TLB_SHIFT;
> +	int ret;
>=20
>  	if (swiotlb_force =3D=3D SWIOTLB_NO_FORCE)
>  		return 0;
> @@ -318,8 +346,9 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long n=
slabs)
>  	if (!mem->slots)
>  		return -ENOMEM;
>=20
> -	set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
> -	swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
> +	ret =3D swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
> +	if (ret)

Before returning the error, free the pages obtained from the earlier call
to __get_free_pages()?

> +		return ret;
>=20
>  	swiotlb_print_info();
>  	swiotlb_set_max_segment(mem->nslabs << IO_TLB_SHIFT);
> @@ -371,7 +400,7 @@ static void swiotlb_bounce(struct device *dev, phys_a=
ddr_t tlb_addr, size_t size
>  	phys_addr_t orig_addr =3D mem->slots[index].orig_addr;
>  	size_t alloc_size =3D mem->slots[index].alloc_size;
>  	unsigned long pfn =3D PFN_DOWN(orig_addr);
> -	unsigned char *vaddr =3D phys_to_virt(tlb_addr);
> +	unsigned char *vaddr =3D mem->vaddr + tlb_addr - mem->start;
>  	unsigned int tlb_offset, orig_addr_offset;
>=20
>  	if (orig_addr =3D=3D INVALID_PHYS_ADDR)
> --
> 2.25.1


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:43:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:43:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187787.336773 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX41-0008Ob-G9; Wed, 15 Sep 2021 15:43:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187787.336773; Wed, 15 Sep 2021 15:43:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX41-0008OS-CY; Wed, 15 Sep 2021 15:43:09 +0000
Received: by outflank-mailman (input) for mailman id 187787;
 Wed, 15 Sep 2021 15:43:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX3z-0008Ij-UE
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:43:07 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.8])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9eb4522a-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:43:06 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1819.namprd21.prod.outlook.com (2603:10b6:302:8::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.2; Wed, 15 Sep
 2021 15:43:02 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:43:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9eb4522a-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ar4Wvx31l2w/ihCAiuOKxLx6Pi6oBmOLIA49gWTySOAyMIlnh++hYmIdN3j1NCZ+7vplYREnOcBSiGbL+mw4ZZVzXJCdxVjutmfS/PW+kXZGB45i+cMgdwA0K/M4Fbm4Zri8e9Nzh18r85A+qSiCzFMwz9kFke442Y+Fl3ZPc/wO/Ufp56OYbNxieDucXSBIYoNnrPO/K+YvcJ0UQMAYgEmTmBTe4wBoE9t0gk/4r8iGV6XeL3k88fn9PZsW94UzknYovApuqSKcowRa48icGLyaCAUHPRPbL2ywEz34TeaZvJMQext3HpUXF89ArxuJvi6EYkuMK3HhHMsBN/fsjA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UdGf2dhGjdjrNSBr29knkOq/iN6qPOnXJ4Q3+w/IWiY=;
 b=ZblGtUWarLpPSp5BlrH1UE5OakPIg7ONf8X3klfJiOuAewiMFMn920W5G0coE7CDk0h4WBOqIr+T/lvDozL9N/DMNeigcaKkI1YFaxbdaYLJPybRc3WO+ut50+JJY2JqnOYDOfQXkshKUBEJK/mdXDJ/aoOMwI6+UTe3xk9oMO2t64LwEXZYWPvcriGJnI9sVmr1HLWgahfRvf8pqkMsmF9W2bCDb/FH3v6vUf4ICxJ7skjNdNmXHD9PXDN41sCWsE5AByY3zpz1cF5FCI2xhhgPYZHBJCbAhnY42mUHHpc8dy9RX6a1d0Vwf9NT2VqLiifY+IN4K2N53M0ntAK9Vw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UdGf2dhGjdjrNSBr29knkOq/iN6qPOnXJ4Q3+w/IWiY=;
 b=CM5IUcO6NalZWF3FIV3eQjZfLTH9jXovMWG6mhwmrMxTorQB1F/MbTvOOK/Bu0eIbVE0PnQmMV+Pyjk5l2V6URh1kWSSnZVn14iS+siUHIPUPRuXITiA2RQsfm+SJeW9yXcwdzNa4Ei86pQ8E2cZ64n0P/QizaCJqYvZGZ2hmKc=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 10/12] hyperv/IOMMU: Enable swiotlb bounce buffer for
 Isolation VM
Thread-Topic: [PATCH V5 10/12] hyperv/IOMMU: Enable swiotlb bounce buffer for
 Isolation VM
Thread-Index: AQHXqW4CnT0aDhDGb0m/WeqqHfER86ulOJww
Date: Wed, 15 Sep 2021 15:43:02 +0000
Message-ID:
 <MWHPR21MB1593A1ACC6B61B1296E2AF27D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-11-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-11-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=f1f7c7ec-7ac2-4096-b62b-ee94b06036b3;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-15T15:24:11Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3f433b1c-7082-4bca-01e7-08d9785f8093
x-ms-traffictypediagnostic: MW2PR2101MB1819:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB181981995E068CEBD4288FD7D7DB9@MW2PR2101MB1819.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:40;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 nypIRcpS7/9undBUoTXZ/SXlOP/49tVAp8/f5PC0tz/hSA0fEYl6GtrsdY8qP1O58WOjY25QLZCc4nXhVWzEWK2uPsZ5tj16M/R/M+azdKmW/pWgOjz8bf4/slVYDQZZ1lPbvB1yQUGgINUVLyqlvNTEzv7+OubJLj9CSan+iJqQVexI7mUaFaT1h2Ot0jkMyhZ7mxvWxWLwQuSxWoMLU029IPGCSgf8urf4k64cx22DjuRcDEEbLgjrBwfGE2V9Rt3LCdQi0juapLnVE7kqj2hgMLnSqpWps7+s0Vzunj7VB5EZ75Fg2DGalfC0koXJA15hILK97dhhMM7vSR1pnzXwn8kls4ysFNdvOt0gplGKm6g+6OfF29nNzmf1fPAGchCE8YfFYJaSrhDgqhdWKVPHeSBepiqbDTIjGm05vrFhcxtkUcrA32xHlExV4tExJnKG8fwYO5gcNqHBguw0Oh+5A95SS1VevvYH3AfJUr+GBm01jGVuCDWg3ZTc/++VmM6BBfaVTw19rHIwT/kde5fENQXZrBEsHy3CRRtUM964mRCQvv9MJ6rPkeEAnf4Vte3Nc6PfNd+wzQLsK5Kd9jWwvq024D5GDvoMfxQGjjoi+UTdtLF+AAZP0ijWfYf5njEoCDlBInMU+uTZ1HGFRGpoMuMgnw/3sraC/aRoQAq02RFeSQ7nHG82wbybpw37wiJGuUmXgdkfyevneWIIRXsOEqBf7vMS3AR5wcUB4A8=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(921005)(71200400001)(110136005)(82950400001)(54906003)(5660300002)(10290500003)(7406005)(6506007)(9686003)(33656002)(8936002)(83380400001)(66946007)(38070700005)(38100700002)(316002)(122000001)(7416002)(55016002)(26005)(66556008)(66446008)(86362001)(76116006)(66476007)(64756008)(8676002)(508600001)(7696005)(2906002)(82960400001)(52536014)(4326008)(186003)(8990500004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?pMI6xj/WxsOzorED9H7aWIdQzaKfcV9t39U93aywGTxcHi0eXBdW5w0tO5yI?=
 =?us-ascii?Q?E/2sCKPb+5mUl5FgKQHGejhH3p+kxwevpUorBfRR0VL1VWc+/uJjHl0dYcqJ?=
 =?us-ascii?Q?aHHnD0u3UDoKR3Vxsf122xoDpHUUprDmYJpKZxKE23JlL4L5ZXVPxdVY0lt8?=
 =?us-ascii?Q?rPy2ZbZfz3uQOo7sJHpfyqQxx8/c8Hu79d7gB5CjMfAlY4BQzn0sSJDjxTqC?=
 =?us-ascii?Q?ReRvwZKpdrKLiys/WGiB8/PrnDufG/KjBMb+G1PZU+W6qvt37gULBkrdSpbl?=
 =?us-ascii?Q?RROSkRjw5O/OIjO8TBsq6CaEoA75tnPhVQtTd1v5u8Pu/rT3IoKpNd77YYvT?=
 =?us-ascii?Q?U34XA8FTq5gvCtkzCIiLYzPep1Pz/n2+1aBbyiPkI+0CF3Wz5JfmixQmOOwU?=
 =?us-ascii?Q?7lXqvE9KNkJ107RWpUZbeM47Ar7P9jiUPL47aCrHVDGvgCmRxf/dNwrtHbav?=
 =?us-ascii?Q?a7+fUfn9eUORJknaZNRxHzzLcIx3cpLbB51wQc6KkLh0c2h2g1WldK6SOIhN?=
 =?us-ascii?Q?QGKN/XCpJ+Lrq7Y3f5HKOU3Aq3HalB8D3tPloB1xznyLeJdivev2E6TCaD40?=
 =?us-ascii?Q?OM5xYXniFdVz4Rclkde9avqHUS83MQVIsyMMUgLtsK10Rf8N48yWOZ5IcWDr?=
 =?us-ascii?Q?PLJhZZlsWFHdiw1anf+Epyrq+0M08BUKnGirSgDVgEBdBzAA2nDu9nDsoE4/?=
 =?us-ascii?Q?B9JDrD8IZFHZS2xJN6KtTYbYAhCumalehr+emJAWoCvTI8N+ScRQXG4CvCdB?=
 =?us-ascii?Q?HdVuVgGEaGVP1MIvkiySL5eRIOylJdN3YqHa7cdwDqpH7ZGrnoR4yvOGMX8O?=
 =?us-ascii?Q?FgZHQKyESb7m8h9kiKUNYkmghiZVl2Ju6pEoMbE+mFVBqOKUKka79VMpDVXo?=
 =?us-ascii?Q?4yTnROzy+1pvNBRaEXzwuz29chDMhG59Fmb1A+2+BmenJcrFOJ6uGGZN5Yw1?=
 =?us-ascii?Q?UcY39+tzD0wj82dJE4QtQJb2WJDlMeNInotsDXdY4tAIlwQR2Peu64CgZsbN?=
 =?us-ascii?Q?4axjCwgaYCwew+G5re6dhMrSvP1rafKcxOiLHKmG3nLzTzjsvRjQG7PLZQpp?=
 =?us-ascii?Q?Npttw9nDhQ5xB5oLZli1D5hlvSev7T+G//7vmOOnvafD4OnOzno7PIfYBexM?=
 =?us-ascii?Q?4chasNa+qPXHZbhamPShPDxEAHYxls0G+8OflW50vn4EXgHW0EPw5l1NDFZG?=
 =?us-ascii?Q?k6fJFPC7O1F2VWFWQIqn0vgOr/JTbAyWZHKLZXLWmaTgiLb8yYyQKIDn4K7r?=
 =?us-ascii?Q?+tMEphvemquakwc0tt3NTT0kQbcy51SnHFcZgBFQj2Dv30KCriYKvFs1Bfjl?=
 =?us-ascii?Q?HckH0qsQxDNGmIieUcdkEqrq?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f433b1c-7082-4bca-01e7-08d9785f8093
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:43:02.0931
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 78pkugS2AS6yDDws9kXGtcNRQGeOatlOsS2DkKE3GCbQmL24do8utGa8uCOUzNGlzfKggzgTRzScaXRA6S7jNTI5gCANjccJK2qrtUHxTGo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1819

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> hyperv Isolation VM requires bounce buffer support to copy
> data from/to encrypted memory and so enable swiotlb force
> mode to use swiotlb bounce buffer for DMA transaction.
>=20
> In Isolation VM with AMD SEV, the bounce buffer needs to be
> accessed via extra address space which is above shared_gpa_boundary
> (E.G 39 bit address line) reported by Hyper-V CPUID ISOLATION_CONFIG.
> The access physical address will be original physical address +
> shared_gpa_boundary. The shared_gpa_boundary in the AMD SEV SNP
> spec is called virtual top of memory(vTOM). Memory addresses below
> vTOM are automatically treated as private while memory above
> vTOM is treated as shared.
>=20
> Hyper-V initalizes swiotlb bounce buffer and default swiotlb
> needs to be disabled. pci_swiotlb_detect_override() and
> pci_swiotlb_detect_4gb() enable the default one. To override
> the setting, hyperv_swiotlb_detect() needs to run before
> these detect functions which depends on the pci_xen_swiotlb_
> init(). Make pci_xen_swiotlb_init() depends on the hyperv_swiotlb
> _detect() to keep the order.
>=20
> Swiotlb bounce buffer code calls set_memory_decrypted()
> to mark bounce buffer visible to host and map it in extra
> address space via memremap. Populate the shared_gpa_boundary
> (vTOM) via swiotlb_unencrypted_base variable.
>=20
> The map function memremap() can't work in the early place
> hyperv_iommu_swiotlb_init() and so initialize swiotlb bounce
> buffer in the hyperv_iommu_swiotlb_later_init().
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
>        * Use swiotlb_unencrypted_base variable to pass shared_gpa_
>          boundary and map bounce buffer inside swiotlb code.
>=20
> Change since v3:
>        * Get hyperv bounce bufffer size via default swiotlb
>        bounce buffer size function and keep default size as
>        same as the one in the AMD SEV VM.
> ---
>  arch/x86/include/asm/mshyperv.h |  2 ++
>  arch/x86/mm/mem_encrypt.c       |  3 +-
>  arch/x86/xen/pci-swiotlb-xen.c  |  3 +-
>  drivers/hv/vmbus_drv.c          |  3 ++
>  drivers/iommu/hyperv-iommu.c    | 60 +++++++++++++++++++++++++++++++++
>  include/linux/hyperv.h          |  1 +
>  6 files changed, 70 insertions(+), 2 deletions(-)
>=20
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index 165423e8b67a..2d22f29f90c9 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -182,6 +182,8 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level=
, int vcpu, int vector,
>  		struct hv_interrupt_entry *entry);
>  int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *=
entry);
>  int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool vi=
sible);
> +void *hv_map_memory(void *addr, unsigned long size);
> +void hv_unmap_memory(void *addr);

Aren't these two declarations now spurious?

>  void hv_ghcb_msr_write(u64 msr, u64 value);
>  void hv_ghcb_msr_read(u64 msr, u64 *value);
>  #else /* CONFIG_HYPERV */
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index ff08dc463634..e2db0b8ed938 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -30,6 +30,7 @@
>  #include <asm/processor-flags.h>
>  #include <asm/msr.h>
>  #include <asm/cmdline.h>
> +#include <asm/mshyperv.h>
>=20
>  #include "mm_internal.h"
>=20
> @@ -202,7 +203,7 @@ void __init sev_setup_arch(void)
>  	phys_addr_t total_mem =3D memblock_phys_mem_size();
>  	unsigned long size;
>=20
> -	if (!sev_active())
> +	if (!sev_active() && !hv_is_isolation_supported())
>  		return;
>=20
>  	/*
> diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xe=
n.c
> index 54f9aa7e8457..43bd031aa332 100644
> --- a/arch/x86/xen/pci-swiotlb-xen.c
> +++ b/arch/x86/xen/pci-swiotlb-xen.c
> @@ -4,6 +4,7 @@
>=20
>  #include <linux/dma-map-ops.h>
>  #include <linux/pci.h>
> +#include <linux/hyperv.h>
>  #include <xen/swiotlb-xen.h>
>=20
>  #include <asm/xen/hypervisor.h>
> @@ -91,6 +92,6 @@ int pci_xen_swiotlb_init_late(void)
>  EXPORT_SYMBOL_GPL(pci_xen_swiotlb_init_late);
>=20
>  IOMMU_INIT_FINISH(pci_xen_swiotlb_detect,
> -		  NULL,
> +		  hyperv_swiotlb_detect,
>  		  pci_xen_swiotlb_init,
>  		  NULL);
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 392c1ac4f819..b0be287e9a32 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -23,6 +23,7 @@
>  #include <linux/cpu.h>
>  #include <linux/sched/task_stack.h>
>=20
> +#include <linux/dma-map-ops.h>
>  #include <linux/delay.h>
>  #include <linux/notifier.h>
>  #include <linux/panic_notifier.h>
> @@ -2078,6 +2079,7 @@ struct hv_device *vmbus_device_create(const guid_t =
*type,
>  	return child_device_obj;
>  }
>=20
> +static u64 vmbus_dma_mask =3D DMA_BIT_MASK(64);
>  /*
>   * vmbus_device_register - Register the child device
>   */
> @@ -2118,6 +2120,7 @@ int vmbus_device_register(struct hv_device *child_d=
evice_obj)
>  	}
>  	hv_debug_add_dev_dir(child_device_obj);
>=20
> +	child_device_obj->device.dma_mask =3D &vmbus_dma_mask;
>  	return 0;
>=20
>  err_kset_unregister:
> diff --git a/drivers/iommu/hyperv-iommu.c b/drivers/iommu/hyperv-iommu.c
> index e285a220c913..a8ac2239de0f 100644
> --- a/drivers/iommu/hyperv-iommu.c
> +++ b/drivers/iommu/hyperv-iommu.c
> @@ -13,14 +13,22 @@
>  #include <linux/irq.h>
>  #include <linux/iommu.h>
>  #include <linux/module.h>
> +#include <linux/hyperv.h>
> +#include <linux/io.h>
>=20
>  #include <asm/apic.h>
>  #include <asm/cpu.h>
>  #include <asm/hw_irq.h>
>  #include <asm/io_apic.h>
> +#include <asm/iommu.h>
> +#include <asm/iommu_table.h>
>  #include <asm/irq_remapping.h>
>  #include <asm/hypervisor.h>
>  #include <asm/mshyperv.h>
> +#include <asm/swiotlb.h>
> +#include <linux/dma-map-ops.h>
> +#include <linux/dma-direct.h>
> +#include <linux/set_memory.h>
>=20
>  #include "irq_remapping.h"
>=20
> @@ -36,6 +44,9 @@
>  static cpumask_t ioapic_max_cpumask =3D { CPU_BITS_NONE };
>  static struct irq_domain *ioapic_ir_domain;
>=20
> +static unsigned long hyperv_io_tlb_size;
> +static void *hyperv_io_tlb_start;
> +
>  static int hyperv_ir_set_affinity(struct irq_data *data,
>  		const struct cpumask *mask, bool force)
>  {
> @@ -337,4 +348,53 @@ static const struct irq_domain_ops hyperv_root_ir_do=
main_ops =3D {
>  	.free =3D hyperv_root_irq_remapping_free,
>  };
>=20
> +static void __init hyperv_iommu_swiotlb_init(void)
> +{
> +	/*
> +	 * Allocate Hyper-V swiotlb bounce buffer at early place
> +	 * to reserve large contiguous memory.
> +	 */
> +	hyperv_io_tlb_size =3D swiotlb_size_or_default();
> +	hyperv_io_tlb_start =3D memblock_alloc(
> +		hyperv_io_tlb_size, PAGE_SIZE);
> +
> +	if (!hyperv_io_tlb_start) {
> +		pr_warn("Fail to allocate Hyper-V swiotlb buffer.\n");
> +		return;
> +	}
> +}
> +
> +int __init hyperv_swiotlb_detect(void)
> +{
> +	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV))
> +		return 0;
> +
> +	if (!hv_is_isolation_supported())
> +		return 0;
> +
> +	/*
> +	 * Enable swiotlb force mode in Isolation VM to
> +	 * use swiotlb bounce buffer for dma transaction.
> +	 */
> +	swiotlb_unencrypted_base =3D ms_hyperv.shared_gpa_boundary;
> +	swiotlb_force =3D SWIOTLB_FORCE;
> +	return 1;
> +}
> +
> +static void __init hyperv_iommu_swiotlb_later_init(void)
> +{
> +	/*
> +	 * Swiotlb bounce buffer needs to be mapped in extra address
> +	 * space. Map function doesn't work in the early place and so
> +	 * call swiotlb_late_init_with_tbl() here.
> +	 */
> +	if (swiotlb_late_init_with_tbl(hyperv_io_tlb_start,
> +				       hyperv_io_tlb_size >> IO_TLB_SHIFT))
> +		panic("Fail to initialize hyperv swiotlb.\n");
> +}
> +
> +IOMMU_INIT_FINISH(hyperv_swiotlb_detect,
> +		  NULL, hyperv_iommu_swiotlb_init,
> +		  hyperv_iommu_swiotlb_later_init);
> +
>  #endif
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index a9e0bc3b1511..bb1a1519b93a 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1739,6 +1739,7 @@ int hyperv_write_cfg_blk(struct pci_dev *dev, void =
*buf, unsigned int len,
>  int hyperv_reg_block_invalidate(struct pci_dev *dev, void *context,
>  				void (*block_invalidate)(void *context,
>  							 u64 block_mask));
> +int __init hyperv_swiotlb_detect(void);
>=20
>  struct hyperv_pci_block_ops {
>  	int (*read_block)(struct pci_dev *dev, void *buf, unsigned int buf_len,
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 15:43:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 15:43:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187792.336784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX4H-0000QH-Qe; Wed, 15 Sep 2021 15:43:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187792.336784; Wed, 15 Sep 2021 15:43:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQX4H-0000Q8-MC; Wed, 15 Sep 2021 15:43:25 +0000
Received: by outflank-mailman (input) for mailman id 187792;
 Wed, 15 Sep 2021 15:43:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQX4F-0000Mc-Rz
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 15:43:23 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a87fbd3a-163b-11ec-b54e-12813bfff9fa;
 Wed, 15 Sep 2021 15:43:22 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB1819.namprd21.prod.outlook.com (2603:10b6:302:8::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.2; Wed, 15 Sep
 2021 15:43:19 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 15:43:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a87fbd3a-163b-11ec-b54e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Pa7/lJ0isyv1RsJkef3bXdrVB2sNPu/Zywzz7J/m101ACUDYMwE95b0ueLrM5xmaWMVnnvIlFPmrm0yY/eni1t+j7kC+/EhXbnPzlV0hyMQTjjlpEpi/tvB2eP7tKewm53Nh0/laZV9kcgnZD2n4MVQD7/l++wfoasYOv6awx7VKQQ/NB+GzAAgIT3xzrW0KQBr3Jv3OT9ugE90H+McJWrglGTbZnb55y5efE0N/OyU/RhfB6BvaxveCNMiTWKJsLC0ANVgRBT/tb0FpFAXYlzNto5c/5USSCDR6doNSB/Y7g4vjBxWdQHyQr6MQWlKfNz8c6Y1Grc8HFbwEh+Y5TQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZTCgiElrCg7vhUsB24F98StewJFC6DipKoz7OVeqwGM=;
 b=n9AZkHSvTGdBV1Z+PEq/dZ7Oou/TcBnRr3gnBdQ4a8AevXhYxRo+NFLn81ZJlmLXCh8SMplObvYseDxVpqXXQ3DDLu7jmivZcUUUWXi3wR6uL1zuLvNwZxcgjsoLiT+f8MeEDlzS7whFaffkEI52r06Rfe0yHLFrTYJvOLFFirIzW0rlysiWv8CSYA1hPpVvJaxQfeUm80ZY7Rgbji9yY+oIZGoZYSIM0IghjFW9DqvssibwKjXv/ngsoqdiIVG0D+Qzhi01S6P6yc+T2thNMSlW1fa2b8YUSSyMk7Q6S/+oJw/yF/CTQQCVzxOCNR5fHa6IZZJ7W8o/jChfSTk5RQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZTCgiElrCg7vhUsB24F98StewJFC6DipKoz7OVeqwGM=;
 b=grVySTjNQGhIpHC4lbE6hSF7KJpTDTxLtbUHLF271YKz2i4FhEPPqFWNpiOe2ZuiUDvyH4HFqTw9p6jtaS0pCBhGoO1421ozmacRXF7OmYYur+YgbVLvy3rjPJLLWGfBrNEs788RGKOEm4JtH4gI+i1zx48p2eEDESo0xTapt+8=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 11/12] scsi: storvsc: Add Isolation VM support for
 storvsc driver
Thread-Topic: [PATCH V5 11/12] scsi: storvsc: Add Isolation VM support for
 storvsc driver
Thread-Index: AQHXqW4MrZFcBidRP0G2YSZHLzx10Kuj1Ldw
Date: Wed, 15 Sep 2021 15:43:18 +0000
Message-ID:
 <MWHPR21MB1593D62DF0B70278C57B20F1D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-12-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-12-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=7367a701-4304-4332-8506-b3f032e7ce4a;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T18:10:23Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0e5b4933-f8ca-4200-1ce2-08d9785f8a92
x-ms-traffictypediagnostic: MW2PR2101MB1819:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB1819AA6F52E2B59BAB51F514D7DB9@MW2PR2101MB1819.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:619;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 Q4Odije4pLqjrE+nh/8l2cew/C9G1KtuLAEd68EUdsJVMGlI1Z/Q03sbEe7iO1ctgBz59d1SiOEp6CdEkbnexF9+HNSsqo48Ux5rp9xkQxzpMPx7R4ZhukfR65xOxzVXdvwx/KEbm2XP/gksKundHqitbVhYi+gYDY6rUYId3ctYXTXUPhSPNu1DLfru9QCPeBDrIU48qkwrueZiakjRNFwaASqsFmUmLpr0T9KAsB6m28yxl3/dx95L51X6lqgoKVTXTxwILz9zoZ4rMl4QPkh4GllWBtzptGXhpHlkB59n/ySUvbR0Cth6LNGFWT965pflb1/l9O6AzAFbNJFo659DDXcu8nLbyKtVca8vWXxI5ZcDzxbkDmCoVsgRY62KbxVrPtXhQm60qhLQzZotOuWta5lnjZ2Q7wA8awS2zNYR1D4Ct6R7tBWmerGhOEzQwcb4FvhtO1T+y9w07enVMayQaMfvhbpwV1YFWuFniPCdMCOG62ecQ4QKcCtxhhuLUiKpCW6E4yTYQBkyTTo1G+FRxgCIx1NXN6o7tDo1vOhD4UDfmVTYR24D8w4meJ8xvaZds85GG/XDKlx2uVoU+cX8sJjCh2/8gj1Wg5Sb/4zAPg8j7PCbuTEigeynrSZ1V7rhTwpgHYicZOFkMtotNF8c0+XOOPJZEGQ/u8SsAIGdOd9Bg+Og0Wg0jMotMILGkcYHs2MhdZwqCgtRY8nR6uv3zc3tWczy62PjgcNCNXw=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(921005)(71200400001)(110136005)(82950400001)(54906003)(5660300002)(10290500003)(7406005)(6506007)(9686003)(33656002)(8936002)(83380400001)(66946007)(38070700005)(38100700002)(316002)(122000001)(7416002)(55016002)(26005)(66556008)(66446008)(86362001)(76116006)(66476007)(64756008)(8676002)(508600001)(7696005)(2906002)(82960400001)(52536014)(4326008)(186003)(8990500004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?Akx3w9h7DWQOdOqAMTijFtFz87wpWZTK5Rm8Nv8k+RsCBMvmbLyJ8IilP7BW?=
 =?us-ascii?Q?ieTOpzIzeTUIODhGUzt6la+DISoPi0C6BFLKbK322HN+iL981+pYUXlLWISF?=
 =?us-ascii?Q?/c36Qi40Fn1zYUOn8yqgdvPnN57TkimvcpaCMFaKQbOPNBNhMUdnO2bnwsgk?=
 =?us-ascii?Q?msDoUeIyeKRzEY+DfjShumrjGc9JXndvvCnPOSSDPqan804G6DuBSq2QLDuM?=
 =?us-ascii?Q?qpqBTaFZwx7inWZLGDJaSn/FhE2Tv3DefyIjuTK2aVSLNXQTmToYAmjHo6fv?=
 =?us-ascii?Q?P7MRWqqV1nk4b5Q9v9v4B9Q4MJEMzj4krJPlTa8ZwLDi7Le3QvGMyFQj7G6S?=
 =?us-ascii?Q?Dm73bm1AXFlHlHADJZ5mwgVz+FBKJ4sUmscvuL8Q23D1/EJmzEq2vWoQRrxz?=
 =?us-ascii?Q?EApmfxzlwwsz+bsJD7/Dm3vqhTLjwj6u4pLTRRfkBHhRuUAhj6fQsQYu1nGq?=
 =?us-ascii?Q?BoG8OcrOkqNJ+kiCvlLHhEdjqy5CXML6vOybgncjzHdsaorF2ep2Xkgfznxk?=
 =?us-ascii?Q?wK4hP78xcRv9XqaJocBuWd3Eh/2bvkNJsY2rx940gE21YLkdOf+GiSySX8Q3?=
 =?us-ascii?Q?qwmFkAkPGnSPsZF9gWi/HqV9U8LOljt4K+1N0HCJ77c4KU9WN6QDVrM6rpkI?=
 =?us-ascii?Q?ZMA9HYtC6eF2BuzLt82KxHT+Nyc1TMRD1hbtOPebm221Opql7cXSOaZj/JLX?=
 =?us-ascii?Q?/EQvIVkTcgqWSnlwusGcx6E66aePJErB4dvWOhj/8JMfbF/4Y02nkbb+wkvO?=
 =?us-ascii?Q?Z/wm8o0ZJe8WPKaJTCxtuI/EHFPxvywtryg4Lo9xDS8tiTJp+kS09YrxJldQ?=
 =?us-ascii?Q?otjblrrhWq/rGNbO3XkoB4BZP6410ZP9a9UFOZMh82fslrHYCGc3BIXWvH2y?=
 =?us-ascii?Q?FNZFbWyM66hg2VXJEK1mCz8J5kb+mgZ8XjrKSQxYmq8BgyriBE4FwLX1PaoN?=
 =?us-ascii?Q?2g3WihNlDjzPTrVdhLi4oE+aBcMP1Kxz24cIJVddVWHQwiEvagFegNwFe2py?=
 =?us-ascii?Q?MrXB8gXwuDTS1j3LpAp/T2ebpZaudpGgSZXpXdoScxZqPBTsWvdipPutrrMd?=
 =?us-ascii?Q?K92u4kLicqbitY60PQeGeWpl43yhUA8dfixtAYBy7oGig6bXecvfooTKWAr3?=
 =?us-ascii?Q?G68BgOYSxfn3nqdkSUoYUAS0hZ1JvNU7bkoIA+oyejDsxUJmz8nbAKzyXgbh?=
 =?us-ascii?Q?xFnzk/zBUaYszPX5aBLXYlBLed+nEJnjrqBDOlFr2FH87NGr8LXe4vYOzn09?=
 =?us-ascii?Q?7VKaWpUZvvswV0GDydsvYv9PTd2saXt+eCfZrHAWQh1TRrpmaxDxoDXP/Dnp?=
 =?us-ascii?Q?VFD6nO2VrqVTr2nrY7tG9yCT?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e5b4933-f8ca-4200-1ce2-08d9785f8a92
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 15:43:18.8355
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kp0xEe0j2pR4BSITs9K5GMBsLwALC2OIrPcOEdzxPZciX4M7fP0RFZuuHYpQrngsWUsW44NT+9VJFDi1AUVO8F/nBa0IliJ7P/hTuZHlezI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1819

From: Tianyu Lan <ltykernel@gmail.com> Sent: Tuesday, September 14, 2021 6:=
39 AM
>=20
> In Isolation VM, all shared memory with host needs to mark visible
> to host via hvcall. vmbus_establish_gpadl() has already done it for
> storvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
> mpb_desc() still needs to be handled. Use DMA API(scsi_dma_map/unmap)
> to map these memory during sending/receiving packet and return swiotlb
> bounce buffer dma address. In Isolation VM, swiotlb  bounce buffer is
> marked to be visible to host and the swiotlb force mode is enabled.
>=20
> Set device's dma min align mask to HV_HYP_PAGE_SIZE - 1 in order to
> keep the original data offset in the bounce buffer.
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	* use scsi_dma_map/unmap() instead of dma_map/unmap_sg()
> 	* Add deleted comments back.
> 	* Fix error calculation of  hvpnfs_to_add
>=20
> Change since v3:
> 	* Rplace dma_map_page with dma_map_sg()
> 	* Use for_each_sg() to populate payload->range.pfn_array.
> 	* Remove storvsc_dma_map macro
> ---
>  drivers/hv/vmbus_drv.c     |  1 +
>  drivers/scsi/storvsc_drv.c | 24 +++++++++++++++---------
>  include/linux/hyperv.h     |  1 +
>  3 files changed, 17 insertions(+), 9 deletions(-)
>=20
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index b0be287e9a32..9c53f823cde1 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -2121,6 +2121,7 @@ int vmbus_device_register(struct hv_device *child_d=
evice_obj)
>  	hv_debug_add_dev_dir(child_device_obj);
>=20
>  	child_device_obj->device.dma_mask =3D &vmbus_dma_mask;
> +	child_device_obj->device.dma_parms =3D &child_device_obj->dma_parms;
>  	return 0;
>=20
>  err_kset_unregister:
> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> index ebbbc1299c62..d10b450bcf0c 100644
> --- a/drivers/scsi/storvsc_drv.c
> +++ b/drivers/scsi/storvsc_drv.c
> @@ -21,6 +21,8 @@
>  #include <linux/device.h>
>  #include <linux/hyperv.h>
>  #include <linux/blkdev.h>
> +#include <linux/dma-mapping.h>
> +
>  #include <scsi/scsi.h>
>  #include <scsi/scsi_cmnd.h>
>  #include <scsi/scsi_host.h>
> @@ -1322,6 +1324,7 @@ static void storvsc_on_channel_callback(void *conte=
xt)
>  					continue;
>  				}
>  				request =3D (struct storvsc_cmd_request *)scsi_cmd_priv(scmnd);
> +				scsi_dma_unmap(scmnd);
>  			}
>=20
>  			storvsc_on_receive(stor_device, packet, request);
> @@ -1735,7 +1738,6 @@ static int storvsc_queuecommand(struct Scsi_Host *h=
ost, struct scsi_cmnd *scmnd)
>  	struct hv_host_device *host_dev =3D shost_priv(host);
>  	struct hv_device *dev =3D host_dev->dev;
>  	struct storvsc_cmd_request *cmd_request =3D scsi_cmd_priv(scmnd);
> -	int i;
>  	struct scatterlist *sgl;
>  	unsigned int sg_count;
>  	struct vmscsi_request *vm_srb;
> @@ -1817,10 +1819,11 @@ static int storvsc_queuecommand(struct Scsi_Host =
*host, struct scsi_cmnd *scmnd)
>  	payload_sz =3D sizeof(cmd_request->mpb);
>=20
>  	if (sg_count) {
> -		unsigned int hvpgoff, hvpfns_to_add;
>  		unsigned long offset_in_hvpg =3D offset_in_hvpage(sgl->offset);
>  		unsigned int hvpg_count =3D HVPFN_UP(offset_in_hvpg + length);
> -		u64 hvpfn;
> +		struct scatterlist *sg;
> +		unsigned long hvpfn, hvpfns_to_add;
> +		int j, i =3D 0;
>=20
>  		if (hvpg_count > MAX_PAGE_BUFFER_COUNT) {
>=20
> @@ -1834,8 +1837,11 @@ static int storvsc_queuecommand(struct Scsi_Host *=
host, struct scsi_cmnd *scmnd)
>  		payload->range.len =3D length;
>  		payload->range.offset =3D offset_in_hvpg;
>=20
> +		sg_count =3D scsi_dma_map(scmnd);
> +		if (sg_count < 0)
> +			return SCSI_MLQUEUE_DEVICE_BUSY;
>=20
> -		for (i =3D 0; sgl !=3D NULL; sgl =3D sg_next(sgl)) {
> +		for_each_sg(sgl, sg, sg_count, j) {
>  			/*
>  			 * Init values for the current sgl entry. hvpgoff
>  			 * and hvpfns_to_add are in units of Hyper-V size

Nit:  The above comment is now out-of-date because hvpgoff has
been removed.

> @@ -1845,10 +1851,9 @@ static int storvsc_queuecommand(struct Scsi_Host *=
host, struct scsi_cmnd *scmnd)
>  			 * even on other than the first sgl entry, provided
>  			 * they are a multiple of PAGE_SIZE.
>  			 */
> -			hvpgoff =3D HVPFN_DOWN(sgl->offset);
> -			hvpfn =3D page_to_hvpfn(sg_page(sgl)) + hvpgoff;
> -			hvpfns_to_add =3D	HVPFN_UP(sgl->offset + sgl->length) -
> -						hvpgoff;
> +			hvpfn =3D HVPFN_DOWN(sg_dma_address(sg));
> +			hvpfns_to_add =3D HVPFN_UP(sg_dma_address(sg) +
> +						 sg_dma_len(sg)) - hvpfn;

Good.  This looks correct now.

>=20
>  			/*
>  			 * Fill the next portion of the PFN array with
> @@ -1858,7 +1863,7 @@ static int storvsc_queuecommand(struct Scsi_Host *h=
ost, struct scsi_cmnd *scmnd)
>  			 * the PFN array is filled.
>  			 */
>  			while (hvpfns_to_add--)
> -				payload->range.pfn_array[i++] =3D	hvpfn++;
> +				payload->range.pfn_array[i++] =3D hvpfn++;
>  		}
>  	}
>=20
> @@ -2002,6 +2007,7 @@ static int storvsc_probe(struct hv_device *device,
>  	stor_device->vmscsi_size_delta =3D sizeof(struct vmscsi_win8_extension)=
;
>  	spin_lock_init(&stor_device->lock);
>  	hv_set_drvdata(device, stor_device);
> +	dma_set_min_align_mask(&device->device, HV_HYP_PAGE_SIZE - 1);
>=20
>  	stor_device->port_number =3D host->host_no;
>  	ret =3D storvsc_connect_to_vsp(device, storvsc_ringbuffer_size, is_fc);
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index bb1a1519b93a..c94c534a944e 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1274,6 +1274,7 @@ struct hv_device {
>=20
>  	struct vmbus_channel *channel;
>  	struct kset	     *channels_kset;
> +	struct device_dma_parameters dma_parms;
>=20
>  	/* place holder to keep track of the dir for hv device in debugfs */
>  	struct dentry *debug_dir;
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 16:22:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 16:22:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187807.336795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXfv-0005oh-VS; Wed, 15 Sep 2021 16:22:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187807.336795; Wed, 15 Sep 2021 16:22:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXfv-0005oa-SZ; Wed, 15 Sep 2021 16:22:19 +0000
Received: by outflank-mailman (input) for mailman id 187807;
 Wed, 15 Sep 2021 16:22:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xFo4=OF=microsoft.com=mikelley@srs-us1.protection.inumbo.net>)
 id 1mQXfv-0005oT-AL
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 16:22:19 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.10])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 16a14569-1641-11ec-b553-12813bfff9fa;
 Wed, 15 Sep 2021 16:22:15 +0000 (UTC)
Received: from MWHPR21MB1593.namprd21.prod.outlook.com (2603:10b6:301:7c::11)
 by MW2PR2101MB0908.namprd21.prod.outlook.com (2603:10b6:302:10::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.1; Wed, 15 Sep
 2021 16:21:58 +0000
Received: from MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3]) by MWHPR21MB1593.namprd21.prod.outlook.com
 ([fe80::9cb:4254:eba4:a4c3%7]) with mapi id 15.20.4544.005; Wed, 15 Sep 2021
 16:21:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16a14569-1641-11ec-b553-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=A3LX07e8BZhXgp9xN84lTvA+yllLu+JyJkg4ZXyI0VS479caRGhQS/YPj7QBboAUXtM3F1Uq0C3bZ9PEhVMbANqxiuZEdYsbzcyYzbylZ4A3OgBwwxgFyab6EAlEsOIuwsHltlnQIa5oy6SVICV/WKMqBc3wEtJDrjMgXJ6+erSYs+uzDko3ulobfNgbC1knSNrn6qaDYtwihOiKqGxwjjcf7ZZWrZsTPw3Vg2JDMgKpkGl0bDcFb0+Q/cEc9mxJ/0KWB99kCStTtyq8NOHB/qOH6HtYPM3AIgs71Jh85/n5ZZAidEn69+V/9jjKb8Y7rhCMLJkrUzm0R5mrwlfPNg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ja/LOgfHueyT+LFgGDjiCoTKEGIMWDcRcL8Q7H+Bn24=;
 b=kY5MNuQ/KtzeNWRkv82J8UlHPyyqr8AklH0MM+0kH/tr8TtY+hRbhmB5ZYnVQBHkdgpBKWsZdEI13PjY0oXXp81v/i1znIrRyOs+2mOKkj8fzGLFe3nKNDOe6xn4YGC4db6vVaMNmU2oLS5BN8qckGS1EDw1/O9snJ4jeK3xBPZI7vbVaYMnqQxMW2ENyc1qDH3l0DwsquS30/sX0kaZrxwL83kQWWUaTmAJwkc6wA9kFLvN5/6KoYvhgWdw0gga0aaoCFg6ECK68dJSVtWebwQzoQasvl4fEyA8J8DESvETOXFpyEYR05dOyAHx+TEQo+tvy+os6FRRgvo+7KLVzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ja/LOgfHueyT+LFgGDjiCoTKEGIMWDcRcL8Q7H+Bn24=;
 b=dMdSilmzqLRTmni51o47JOosuhnttehp+m7PVa7uz6V4v5pZHU06TL9hIHlYqylmfnmuFjLdydqaG/6ZlAV/V2oxTkeRca1srWW10MM9sR/Dvo6SSU53KxTItpL5hKMGW4XDX8ML4+40D5VZc1om8/UFrDwfBrSNOFbsKK9fTAU=
From: Michael Kelley <mikelley@microsoft.com>
To: Tianyu Lan <ltykernel@gmail.com>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
Thread-Topic: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
 netvsc driver
Thread-Index: AQHXqW4FepA+T2dkA0WfAWppSl9u0auj6xtA
Date: Wed, 15 Sep 2021 16:21:58 +0000
Message-ID:
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
In-Reply-To: <20210914133916.1440931-13-ltykernel@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=131cd4f8-e33d-45d8-94eb-3fa8bffaa59b;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T19:30:32Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3c69d908-c1e5-4c24-9cd4-08d97864f11f
x-ms-traffictypediagnostic: MW2PR2101MB0908:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MW2PR2101MB09085FD9B123206B8A3BDDA8D7DB9@MW2PR2101MB0908.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 uXxrmFQxEtGaDhzZ+dtP5PfrHN7w8XpRhnoh7Eez1AKpuQoKzsDutOeSrEcn9IBXkWrP082Kt0khZ+TByndJ/FD8hDHsl+SGYRpZlT0I6mDKS+9M/lar2VJAm3wT4O6vIqMAyoGBYkFQt/Xwn7vXuVTeYdZJrWgS5Foa+Gz5AJz60Hh9r7PAv8Y4UZxmzvRwYZMLcQeW5GPMSQupA+nfenmJBD1kRwvdSi/sWYKLGtZ+88OAknVfLmV2F/gNdebxWDlLfFJjtZTyNMi5irENRXs/hyfVifw9yyxcnTtfysLZXQymqgAVtY2ULRy8BzNNPpZCSufNTawUjWy/UQLgiiwADv47Abrg52CXpU7hrbL/9B5RV7WYNsD6OPBWPtldBfzK1uGEdWvCyG55WcVmm8rnNlSLmyATtVyw0OEgTd15vrLbqmyRX3+WBrwWlCoSRZSjEojLEuDG2hXRaf2d4egDI/2P8fZ3N0h+upaTaxgJ7X6O+LAVDfvsT1q/ORNsuAeDDY2lpfXTbKOUsmWnOGw+Sa10IDc4t0hdxVBYy0Ui8ZY2LPyzULK9I6iDUcXoIqEBnzwtfr5LhvAFKPC0FwFumqpLCEAupItwWNxjnK22kR5nplZyQ0sgaebKyvDi1let/YFeMeW2jWsfL10iLHoZlZh9gM/rVqDLFNl3GgSBhsQEkynGfT9KwK+QSoipYrtCB+SzhEIKpNQVnmDtrIsPRnV0/KGI+rIU2QZCHS4=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR21MB1593.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(83380400001)(66946007)(6506007)(55016002)(7406005)(66556008)(26005)(82950400001)(8936002)(64756008)(110136005)(7416002)(86362001)(30864003)(8990500004)(2906002)(66446008)(71200400001)(54906003)(4326008)(5660300002)(122000001)(76116006)(66476007)(9686003)(38100700002)(52536014)(316002)(508600001)(33656002)(921005)(7696005)(186003)(10290500003)(8676002)(82960400001)(579004);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?OY4wX8wRURLgunokENF2AQdGJDcGHf+y0zTurd21ejJnyIPzXiNCdOTV/3d4?=
 =?us-ascii?Q?wOR7WRhiOirGz9gE5xoOYNq9OD68xGmjkB71c09DlXef+gTe7kcwc3qQJ3NP?=
 =?us-ascii?Q?fAvxc+pLijFmkx1ioZrMHgtSq51GzUNW7qu1UHHkloC6B49rvj3CJqLU99BK?=
 =?us-ascii?Q?XD1NHKLcAh3Snu5nbCPPn1zJD/xknB+R53CCPXTcItYViNYEIQiOpGj7vsLZ?=
 =?us-ascii?Q?fELS+6cY4ZMKCKd+WR5UyRmLeWrrtmMFeNVj+f0R+HAMx/HTy4JL8Wsora5U?=
 =?us-ascii?Q?1Gx1HgmS572oCUi4GUIC1F1iLfU/zyfhEmZvHxlr3+G2Wt73UgxFNOpU8aDT?=
 =?us-ascii?Q?6K7fTiH+5SdLXC+eqqKu1KDS4NOoFA9QXk5TYP60zdAquFRE4gE9BFWSt6ac?=
 =?us-ascii?Q?w0SVr2VJX6VA+3vlhVx1Rj/89zcnloYBiZQP0hNPcwddWgMP6aaw4d5hJbrg?=
 =?us-ascii?Q?vRxK3nI4a10fWJnZRMQ2dtW4Cjs40DAMsUVHwxU9cGaAKGKXVVKL3d+FivBC?=
 =?us-ascii?Q?6MsO6ILN2bAjOGqFpFgiPaFJSrtQbJFQb81pNGG52WJfFKKgZGitQG671sdf?=
 =?us-ascii?Q?9G/cD7Eg4wUJDWVeeKVlZD8JHFHantiSQIttctmUnEABeL/E39xLJ1hb1U2Y?=
 =?us-ascii?Q?LfTNBCB0Uj+5mq6P5uN47v+c2IfM9Un3VyhD72upbOQzYsaIKc8xXxUN9dBI?=
 =?us-ascii?Q?/+yVD1H4LOpU/wt1ABkEdUiWxHktmsJGl+j/J1B5I/lFpCO703rrF7e7W/Cx?=
 =?us-ascii?Q?JKmKKH889DRo1HAwBiUOo+oHSBiTgbMNbw43q/MXqsdRd2fDkQ2s81p9H/gY?=
 =?us-ascii?Q?uCl9bGJxCW1edLizRXeb5X6RMxf0IjivzhJocGGR8WR+IoYWRJJmkEtH+AXM?=
 =?us-ascii?Q?M5xe40up5Y17A6WCHtgKUT2xe9g8bLVQVv6THGIZk+mCOkrtBV8jnvUrVpzD?=
 =?us-ascii?Q?hczyR31vlanNjBfA6auCd6g/Nrrtl5jkle5o0UyXvmKLmXefDVsaHuOjZFnb?=
 =?us-ascii?Q?RiCs5oEmaKAqG3y3c16Ahp61CiNG52eNyq9HxOqsKzpHl/wvQ15w1VcjseAQ?=
 =?us-ascii?Q?QLaA0tA5bs+ay7c/wcuJf7xm4lQcozxGqLRAvkKbNrdQ+pMJvBRq1Nhv2dMh?=
 =?us-ascii?Q?IoclcGSr/Xwv/wF0PI5igVXDK5MiM5Hg3uhb1S0c5AmyrezhlWu/01eYFdJ5?=
 =?us-ascii?Q?RaBVjAZOFTKnldJCxEWUKz3bc/5cr0UINvAJm172nM/G1E4ZOgDMX3TYC3jQ?=
 =?us-ascii?Q?qkzoMyqigeLYwtaTGyKb5p34E43jmuaVv8mfMpX6oMxahgeZiz4ScZCfsPsx?=
 =?us-ascii?Q?u3YroatrMyjJkhrF6UtgWwog?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB1593.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c69d908-c1e5-4c24-9cd4-08d97864f11f
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 16:21:58.3861
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: IV8G1vkOqp4uyUANOHSYIIwYA2mFz6PKfvBk+khBaJnZhgEEALbnxTbK4ioov5Nx9DLoK3IpPkkpR63hEiEKIMCXl1IsbWYuKa1P544pAVk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB0908

From: Tianyu Lan <ltykernel@gmail.com>  Sent: Tuesday, September 14, 2021 6=
:39 AM
>=20
> In Isolation VM, all shared memory with host needs to mark visible
> to host via hvcall. vmbus_establish_gpadl() has already done it for
> netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
> pagebuffer() stills need to be handled. Use DMA API to map/umap
> these memory during sending/receiving packet and Hyper-V swiotlb
> bounce buffer dma address will be returned. The swiotlb bounce buffer
> has been masked to be visible to host during boot up.
>=20
> Allocate rx/tx ring buffer via alloc_pages() in Isolation VM and map
> these pages via vmap(). After calling vmbus_establish_gpadl() which
> marks these pages visible to host, unmap these pages to release the
> virtual address mapped with physical address below shared_gpa_boundary
> and map them in the extra address space via vmap_pfn().
>=20
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> Change since v4:
> 	* Allocate rx/tx ring buffer via alloc_pages() in Isolation VM
> 	* Map pages after calling vmbus_establish_gpadl().
> 	* set dma_set_min_align_mask for netvsc driver.
>=20
> Change since v3:
> 	* Add comment to explain why not to use dma_map_sg()
> 	* Fix some error handle.
> ---
>  drivers/net/hyperv/hyperv_net.h   |   7 +
>  drivers/net/hyperv/netvsc.c       | 287 +++++++++++++++++++++++++++++-
>  drivers/net/hyperv/netvsc_drv.c   |   1 +
>  drivers/net/hyperv/rndis_filter.c |   2 +
>  include/linux/hyperv.h            |   5 +
>  5 files changed, 296 insertions(+), 6 deletions(-)
>=20
> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_=
net.h
> index 315278a7cf88..87e8c74398a5 100644
> --- a/drivers/net/hyperv/hyperv_net.h
> +++ b/drivers/net/hyperv/hyperv_net.h
> @@ -164,6 +164,7 @@ struct hv_netvsc_packet {
>  	u32 total_bytes;
>  	u32 send_buf_index;
>  	u32 total_data_buflen;
> +	struct hv_dma_range *dma_range;
>  };
>=20
>  #define NETVSC_HASH_KEYLEN 40
> @@ -1074,6 +1075,8 @@ struct netvsc_device {
>=20
>  	/* Receive buffer allocated by us but manages by NetVSP */
>  	void *recv_buf;
> +	struct page **recv_pages;
> +	u32 recv_page_count;
>  	u32 recv_buf_size; /* allocated bytes */
>  	struct vmbus_gpadl recv_buf_gpadl_handle;
>  	u32 recv_section_cnt;
> @@ -1082,6 +1085,8 @@ struct netvsc_device {
>=20
>  	/* Send buffer allocated by us */
>  	void *send_buf;
> +	struct page **send_pages;
> +	u32 send_page_count;
>  	u32 send_buf_size;
>  	struct vmbus_gpadl send_buf_gpadl_handle;
>  	u32 send_section_cnt;
> @@ -1731,4 +1736,6 @@ struct rndis_message {
>  #define RETRY_US_HI	10000
>  #define RETRY_MAX	2000	/* >10 sec */
>=20
> +void netvsc_dma_unmap(struct hv_device *hv_dev,
> +		      struct hv_netvsc_packet *packet);
>  #endif /* _HYPERV_NET_H */
> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
> index 1f87e570ed2b..7d5254bf043e 100644
> --- a/drivers/net/hyperv/netvsc.c
> +++ b/drivers/net/hyperv/netvsc.c
> @@ -20,6 +20,7 @@
>  #include <linux/vmalloc.h>
>  #include <linux/rtnetlink.h>
>  #include <linux/prefetch.h>
> +#include <linux/gfp.h>
>=20
>  #include <asm/sync_bitops.h>
>  #include <asm/mshyperv.h>
> @@ -150,11 +151,33 @@ static void free_netvsc_device(struct rcu_head *hea=
d)
>  {
>  	struct netvsc_device *nvdev
>  		=3D container_of(head, struct netvsc_device, rcu);
> +	unsigned int alloc_unit;
>  	int i;
>=20
>  	kfree(nvdev->extension);
> -	vfree(nvdev->recv_buf);
> -	vfree(nvdev->send_buf);
> +
> +	if (nvdev->recv_pages) {
> +		alloc_unit =3D (nvdev->recv_buf_size /
> +			nvdev->recv_page_count) >> PAGE_SHIFT;
> +
> +		vunmap(nvdev->recv_buf);
> +		for (i =3D 0; i < nvdev->recv_page_count; i++)
> +			__free_pages(nvdev->recv_pages[i], alloc_unit);
> +	} else {
> +		vfree(nvdev->recv_buf);
> +	}
> +
> +	if (nvdev->send_pages) {
> +		alloc_unit =3D (nvdev->send_buf_size /
> +			nvdev->send_page_count) >> PAGE_SHIFT;
> +
> +		vunmap(nvdev->send_buf);
> +		for (i =3D 0; i < nvdev->send_page_count; i++)
> +			__free_pages(nvdev->send_pages[i], alloc_unit);
> +	} else {
> +		vfree(nvdev->send_buf);
> +	}
> +
>  	kfree(nvdev->send_section_map);
>=20
>  	for (i =3D 0; i < VRSS_CHANNEL_MAX; i++) {
> @@ -330,6 +353,108 @@ int netvsc_alloc_recv_comp_ring(struct netvsc_devic=
e *net_device, u32 q_idx)
>  	return nvchan->mrc.slots ? 0 : -ENOMEM;
>  }
>=20
> +void *netvsc_alloc_pages(struct page ***pages_array, unsigned int *array=
_len,
> +			 unsigned long size)
> +{
> +	struct page *page, **pages, **vmap_pages;
> +	unsigned long pg_count =3D size >> PAGE_SHIFT;
> +	int alloc_unit =3D MAX_ORDER_NR_PAGES;
> +	int i, j, vmap_page_index =3D 0;
> +	void *vaddr;
> +
> +	if (pg_count < alloc_unit)
> +		alloc_unit =3D 1;
> +
> +	/* vmap() accepts page array with PAGE_SIZE as unit while try to
> +	 * allocate high order pages here in order to save page array space.
> +	 * vmap_pages[] is used as input parameter of vmap(). pages[] is to
> +	 * store allocated pages and map them later.
> +	 */
> +	vmap_pages =3D kmalloc_array(pg_count, sizeof(*vmap_pages), GFP_KERNEL)=
;
> +	if (!vmap_pages)
> +		return NULL;
> +
> +retry:
> +	*array_len =3D pg_count / alloc_unit;
> +	pages =3D kmalloc_array(*array_len, sizeof(*pages), GFP_KERNEL);
> +	if (!pages)
> +		goto cleanup;
> +
> +	for (i =3D 0; i < *array_len; i++) {
> +		page =3D alloc_pages(GFP_KERNEL | __GFP_ZERO,
> +				   get_order(alloc_unit << PAGE_SHIFT));
> +		if (!page) {
> +			/* Try allocating small pages if high order pages are not available. =
*/
> +			if (alloc_unit =3D=3D 1) {
> +				goto cleanup;
> +			} else {

The "else" clause isn't really needed because of the goto cleanup above.  T=
hen
the indentation of the code below could be reduced by one level.

> +				memset(vmap_pages, 0,
> +				       sizeof(*vmap_pages) * vmap_page_index);
> +				vmap_page_index =3D 0;
> +
> +				for (j =3D 0; j < i; j++)
> +					__free_pages(pages[j], alloc_unit);
> +
> +				kfree(pages);
> +				alloc_unit =3D 1;

This is the case where a large enough contiguous physical memory chunk coul=
d
not be found.  But rather than dropping all the way down to single pages,
would it make sense to try something smaller, but not 1?  For example,
cut the alloc_unit in half and try again.  But I'm not sure of all the impl=
ications.

> +				goto retry;
> +			}
> +		}
> +
> +		pages[i] =3D page;
> +		for (j =3D 0; j < alloc_unit; j++)
> +			vmap_pages[vmap_page_index++] =3D page++;
> +	}
> +
> +	vaddr =3D vmap(vmap_pages, vmap_page_index, VM_MAP, PAGE_KERNEL);
> +	kfree(vmap_pages);
> +
> +	*pages_array =3D pages;
> +	return vaddr;
> +
> +cleanup:
> +	for (j =3D 0; j < i; j++)
> +		__free_pages(pages[i], alloc_unit);
> +
> +	kfree(pages);
> +	kfree(vmap_pages);
> +	return NULL;
> +}
> +
> +static void *netvsc_map_pages(struct page **pages, int count, int alloc_=
unit)
> +{
> +	int pg_count =3D count * alloc_unit;
> +	struct page *page;
> +	unsigned long *pfns;
> +	int pfn_index =3D 0;
> +	void *vaddr;
> +	int i, j;
> +
> +	if (!pages)
> +		return NULL;
> +
> +	pfns =3D kcalloc(pg_count, sizeof(*pfns), GFP_KERNEL);
> +	if (!pfns)
> +		return NULL;
> +
> +	for (i =3D 0; i < count; i++) {
> +		page =3D pages[i];
> +		if (!page) {
> +			pr_warn("page is not available %d.\n", i);
> +			return NULL;
> +		}
> +
> +		for (j =3D 0; j < alloc_unit; j++) {
> +			pfns[pfn_index++] =3D page_to_pfn(page++) +
> +				(ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
> +		}
> +	}
> +
> +	vaddr =3D vmap_pfn(pfns, pg_count, PAGE_KERNEL_IO);
> +	kfree(pfns);
> +	return vaddr;
> +}
> +

I think you are proposing this approach to allocating memory for the send
and receive buffers so that you can avoid having two virtual mappings for
the memory, per comments from Christop Hellwig.  But overall, the approach
seems a bit complex and I wonder if it is worth it.  If allocating large co=
ntiguous
chunks of physical memory is successful, then there is some memory savings
in that the data structures needed to keep track of the physical pages is
smaller than the equivalent page tables might be.  But if you have to rever=
t
to allocating individual pages, then the memory savings is reduced.

Ultimately, the list of actual PFNs has to be kept somewhere.  Another appr=
oach
would be to do the reverse of what hv_map_memory() from the v4 patch
series does.  I.e., you could do virt_to_phys() on each virtual address tha=
t
maps above VTOM, and subtract out the shared_gpa_boundary to get the
list of actual PFNs that need to be freed.   This way you don't have two co=
pies
of the list of PFNs -- one with and one without the shared_gpa_boundary add=
ed.
But it comes at the cost of additional code so that may not be a great idea=
.

I think what you have here works, and I don't have a clearly better solutio=
n
at the moment except perhaps to revert to the v4 solution and just have two
virtual mappings.  I'll keep thinking about it.  Maybe Christop has other
thoughts.

>  static int netvsc_init_buf(struct hv_device *device,
>  			   struct netvsc_device *net_device,
>  			   const struct netvsc_device_info *device_info)
> @@ -337,7 +462,7 @@ static int netvsc_init_buf(struct hv_device *device,
>  	struct nvsp_1_message_send_receive_buffer_complete *resp;
>  	struct net_device *ndev =3D hv_get_drvdata(device);
>  	struct nvsp_message *init_packet;
> -	unsigned int buf_size;
> +	unsigned int buf_size, alloc_unit;
>  	size_t map_words;
>  	int i, ret =3D 0;
>=20
> @@ -350,7 +475,14 @@ static int netvsc_init_buf(struct hv_device *device,
>  		buf_size =3D min_t(unsigned int, buf_size,
>  				 NETVSC_RECEIVE_BUFFER_SIZE_LEGACY);
>=20
> -	net_device->recv_buf =3D vzalloc(buf_size);
> +	if (hv_isolation_type_snp())
> +		net_device->recv_buf =3D
> +			netvsc_alloc_pages(&net_device->recv_pages,
> +					   &net_device->recv_page_count,
> +					   buf_size);
> +	else
> +		net_device->recv_buf =3D vzalloc(buf_size);
> +

I wonder if it is necessary to have two different code paths here.  The
allocating and freeing of the send and receive buffers is not perf
sensitive, and it seems like netvsc_alloc_pages() could be used
regardless of whether SNP Isolation is in effect.  To my thinking,
one code path is better than two code paths unless there's a
compelling reason to have two.

>  	if (!net_device->recv_buf) {
>  		netdev_err(ndev,
>  			   "unable to allocate receive buffer of size %u\n",
> @@ -375,6 +507,27 @@ static int netvsc_init_buf(struct hv_device *device,
>  		goto cleanup;
>  	}
>=20
> +	if (hv_isolation_type_snp()) {
> +		alloc_unit =3D (buf_size / net_device->recv_page_count)
> +				>> PAGE_SHIFT;
> +
> +		/* Unmap previous virtual address and map pages in the extra
> +		 * address space(above shared gpa boundary) in Isolation VM.
> +		 */
> +		vunmap(net_device->recv_buf);
> +		net_device->recv_buf =3D
> +			netvsc_map_pages(net_device->recv_pages,
> +					 net_device->recv_page_count,
> +					 alloc_unit);
> +		if (!net_device->recv_buf) {
> +			netdev_err(ndev,
> +				   "unable to allocate receive buffer of size %u\n",
> +				   buf_size);
> +			ret =3D -ENOMEM;
> +			goto cleanup;
> +		}
> +	}
> +
>  	/* Notify the NetVsp of the gpadl handle */
>  	init_packet =3D &net_device->channel_init_pkt;
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
> @@ -456,13 +609,21 @@ static int netvsc_init_buf(struct hv_device *device=
,
>  	buf_size =3D device_info->send_sections * device_info->send_section_siz=
e;
>  	buf_size =3D round_up(buf_size, PAGE_SIZE);
>=20
> -	net_device->send_buf =3D vzalloc(buf_size);
> +	if (hv_isolation_type_snp())
> +		net_device->send_buf =3D
> +			netvsc_alloc_pages(&net_device->send_pages,
> +					   &net_device->send_page_count,
> +					   buf_size);
> +	else
> +		net_device->send_buf =3D vzalloc(buf_size);
> +
>  	if (!net_device->send_buf) {
>  		netdev_err(ndev, "unable to allocate send buffer of size %u\n",
>  			   buf_size);
>  		ret =3D -ENOMEM;
>  		goto cleanup;
>  	}
> +
>  	net_device->send_buf_size =3D buf_size;
>=20
>  	/* Establish the gpadl handle for this buffer on this
> @@ -478,6 +639,27 @@ static int netvsc_init_buf(struct hv_device *device,
>  		goto cleanup;
>  	}
>=20
> +	if (hv_isolation_type_snp()) {
> +		alloc_unit =3D (buf_size / net_device->send_page_count)
> +				>> PAGE_SHIFT;
> +
> +		/* Unmap previous virtual address and map pages in the extra
> +		 * address space(above shared gpa boundary) in Isolation VM.
> +		 */
> +		vunmap(net_device->send_buf);
> +		net_device->send_buf =3D
> +			netvsc_map_pages(net_device->send_pages,
> +					 net_device->send_page_count,
> +					 alloc_unit);
> +		if (!net_device->send_buf) {
> +			netdev_err(ndev,
> +				   "unable to allocate receive buffer of size %u\n",
> +				   buf_size);
> +			ret =3D -ENOMEM;
> +			goto cleanup;
> +		}
> +	}
> +
>  	/* Notify the NetVsp of the gpadl handle */
>  	init_packet =3D &net_device->channel_init_pkt;
>  	memset(init_packet, 0, sizeof(struct nvsp_message));
> @@ -768,7 +950,7 @@ static void netvsc_send_tx_complete(struct net_device=
 *ndev,
>=20
>  	/* Notify the layer above us */
>  	if (likely(skb)) {
> -		const struct hv_netvsc_packet *packet
> +		struct hv_netvsc_packet *packet
>  			=3D (struct hv_netvsc_packet *)skb->cb;
>  		u32 send_index =3D packet->send_buf_index;
>  		struct netvsc_stats *tx_stats;
> @@ -784,6 +966,7 @@ static void netvsc_send_tx_complete(struct net_device=
 *ndev,
>  		tx_stats->bytes +=3D packet->total_bytes;
>  		u64_stats_update_end(&tx_stats->syncp);
>=20
> +		netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>  		napi_consume_skb(skb, budget);
>  	}
>=20
> @@ -948,6 +1131,87 @@ static void netvsc_copy_to_send_buf(struct netvsc_d=
evice *net_device,
>  		memset(dest, 0, padding);
>  }
>=20
> +void netvsc_dma_unmap(struct hv_device *hv_dev,
> +		      struct hv_netvsc_packet *packet)
> +{
> +	u32 page_count =3D packet->cp_partial ?
> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
> +		packet->page_buf_cnt;
> +	int i;
> +
> +	if (!hv_is_isolation_supported())
> +		return;
> +
> +	if (!packet->dma_range)
> +		return;
> +
> +	for (i =3D 0; i < page_count; i++)
> +		dma_unmap_single(&hv_dev->device, packet->dma_range[i].dma,
> +				 packet->dma_range[i].mapping_size,
> +				 DMA_TO_DEVICE);
> +
> +	kfree(packet->dma_range);
> +}
> +
> +/* netvsc_dma_map - Map swiotlb bounce buffer with data page of
> + * packet sent by vmbus_sendpacket_pagebuffer() in the Isolation
> + * VM.
> + *
> + * In isolation VM, netvsc send buffer has been marked visible to
> + * host and so the data copied to send buffer doesn't need to use
> + * bounce buffer. The data pages handled by vmbus_sendpacket_pagebuffer(=
)
> + * may not be copied to send buffer and so these pages need to be
> + * mapped with swiotlb bounce buffer. netvsc_dma_map() is to do
> + * that. The pfns in the struct hv_page_buffer need to be converted
> + * to bounce buffer's pfn. The loop here is necessary because the
> + * entries in the page buffer array are not necessarily full
> + * pages of data.  Each entry in the array has a separate offset and
> + * len that may be non-zero, even for entries in the middle of the
> + * array.  And the entries are not physically contiguous.  So each
> + * entry must be individually mapped rather than as a contiguous unit.
> + * So not use dma_map_sg() here.
> + */
> +static int netvsc_dma_map(struct hv_device *hv_dev,
> +		   struct hv_netvsc_packet *packet,
> +		   struct hv_page_buffer *pb)
> +{
> +	u32 page_count =3D  packet->cp_partial ?
> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
> +		packet->page_buf_cnt;
> +	dma_addr_t dma;
> +	int i;
> +
> +	if (!hv_is_isolation_supported())
> +		return 0;
> +
> +	packet->dma_range =3D kcalloc(page_count,
> +				    sizeof(*packet->dma_range),
> +				    GFP_KERNEL);
> +	if (!packet->dma_range)
> +		return -ENOMEM;
> +
> +	for (i =3D 0; i < page_count; i++) {
> +		char *src =3D phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
> +					 + pb[i].offset);
> +		u32 len =3D pb[i].len;
> +
> +		dma =3D dma_map_single(&hv_dev->device, src, len,
> +				     DMA_TO_DEVICE);
> +		if (dma_mapping_error(&hv_dev->device, dma)) {
> +			kfree(packet->dma_range);
> +			return -ENOMEM;
> +		}
> +
> +		packet->dma_range[i].dma =3D dma;
> +		packet->dma_range[i].mapping_size =3D len;
> +		pb[i].pfn =3D dma >> HV_HYP_PAGE_SHIFT;
> +		pb[i].offset =3D offset_in_hvpage(dma);

With the DMA min align mask now being set, the offset within
the Hyper-V page won't be changed by dma_map_single().  So I
think the above statement can be removed.

> +		pb[i].len =3D len;

A few lines above, the value of "len" is set from pb[i].len.  Neither
"len" nor "i" is changed in the loop, so this statement can also be
removed.

> +	}
> +
> +	return 0;
> +}
> +
>  static inline int netvsc_send_pkt(
>  	struct hv_device *device,
>  	struct hv_netvsc_packet *packet,
> @@ -988,14 +1252,24 @@ static inline int netvsc_send_pkt(
>=20
>  	trace_nvsp_send_pkt(ndev, out_channel, rpkt);
>=20
> +	packet->dma_range =3D NULL;
>  	if (packet->page_buf_cnt) {
>  		if (packet->cp_partial)
>  			pb +=3D packet->rmsg_pgcnt;
>=20
> +		ret =3D netvsc_dma_map(ndev_ctx->device_ctx, packet, pb);
> +		if (ret) {
> +			ret =3D -EAGAIN;
> +			goto exit;
> +		}
> +
>  		ret =3D vmbus_sendpacket_pagebuffer(out_channel,
>  						  pb, packet->page_buf_cnt,
>  						  &nvmsg, sizeof(nvmsg),
>  						  req_id);
> +
> +		if (ret)
> +			netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>  	} else {
>  		ret =3D vmbus_sendpacket(out_channel,
>  				       &nvmsg, sizeof(nvmsg),
> @@ -1003,6 +1277,7 @@ static inline int netvsc_send_pkt(
>  				       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
>  	}
>=20
> +exit:
>  	if (ret =3D=3D 0) {
>  		atomic_inc_return(&nvchan->queue_sends);
>=20
> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_=
drv.c
> index 382bebc2420d..c3dc884b31e3 100644
> --- a/drivers/net/hyperv/netvsc_drv.c
> +++ b/drivers/net/hyperv/netvsc_drv.c
> @@ -2577,6 +2577,7 @@ static int netvsc_probe(struct hv_device *dev,
>  	list_add(&net_device_ctx->list, &netvsc_dev_list);
>  	rtnl_unlock();
>=20
> +	dma_set_min_align_mask(&dev->device, HV_HYP_PAGE_SIZE - 1);
>  	netvsc_devinfo_put(device_info);
>  	return 0;
>=20
> diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis=
_filter.c
> index f6c9c2a670f9..448fcc325ed7 100644
> --- a/drivers/net/hyperv/rndis_filter.c
> +++ b/drivers/net/hyperv/rndis_filter.c
> @@ -361,6 +361,8 @@ static void rndis_filter_receive_response(struct net_=
device *ndev,
>  			}
>  		}
>=20
> +		netvsc_dma_unmap(((struct net_device_context *)
> +			netdev_priv(ndev))->device_ctx, &request->pkt);
>  		complete(&request->wait_event);
>  	} else {
>  		netdev_err(ndev,
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index c94c534a944e..81e58dd582dc 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1597,6 +1597,11 @@ struct hyperv_service_callback {
>  	void (*callback)(void *context);
>  };
>=20
> +struct hv_dma_range {
> +	dma_addr_t dma;
> +	u32 mapping_size;
> +};
> +
>  #define MAX_SRV_VER	0x7ffffff
>  extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *icmsghdrp, u8 *b=
uf, u32 buflen,
>  				const int *fw_version, int fw_vercnt,
> --
> 2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 16:39:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 16:39:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187816.336806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXwW-0007hE-G6; Wed, 15 Sep 2021 16:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187816.336806; Wed, 15 Sep 2021 16:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXwW-0007h7-Cv; Wed, 15 Sep 2021 16:39:28 +0000
Received: by outflank-mailman (input) for mailman id 187816;
 Wed, 15 Sep 2021 16:39:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LStB=OF=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQXwV-0007gw-HX
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 16:39:27 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.71]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1ff7f0ab-942f-4f5f-a451-31279a3a3a92;
 Wed, 15 Sep 2021 16:39:24 +0000 (UTC)
Received: from AM6PR04CA0011.eurprd04.prod.outlook.com (2603:10a6:20b:92::24)
 by AM6PR08MB4660.eurprd08.prod.outlook.com (2603:10a6:20b:c1::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 16:39:12 +0000
Received: from AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:92:cafe::66) by AM6PR04CA0011.outlook.office365.com
 (2603:10a6:20b:92::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Wed, 15 Sep 2021 16:39:12 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT037.mail.protection.outlook.com (10.152.17.241) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 16:39:12 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Wed, 15 Sep 2021 16:39:10 +0000
Received: from 6f41d67f0089.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 93C8AB4B-236B-4568-9699-418FA6FC4820.1; 
 Wed, 15 Sep 2021 16:38:57 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6f41d67f0089.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 Sep 2021 16:38:57 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB5894.eurprd08.prod.outlook.com (2603:10a6:20b:23d::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 15 Sep
 2021 16:38:46 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 16:38:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ff7f0ab-942f-4f5f-a451-31279a3a3a92
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LXA62+xMX11sxy88lI6BLc2NogFQu9Gt+R69UZkk75M=;
 b=6Xf1KV5c4HcBPTy9gQi4Kq8BLRA4e5zWLvxHcorCvrMW0YLSU3r9HsMinHTzC2TccrHMOeQXQJctF09TbbPWx53Q5RfRO/yJGN/sJsz+YiVnXdZxacw7r66Na4Ar3VzQ3IO7imU5n2LwDD8BeO+hIK00Fwe+2QtkBIN0CgAzcO8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 202d5656dff7d719
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VK1nn+Nd3PfZoTvsbno56pPfOrTKQWBG2Ce1/JR78fiF2uvBrawCX8ZOnNuJOiIwczcAK0SSBpMr7SMLw3Ju5KA+LFYGISAjJV5l7/JvzJIq5Q25BQGTWrn6HH/zSZk9B2ly72yo83bs9oiAfugWh8qgof+jCtrPg/ihHXaNL2KfRhanc9fVw4VrncW/sILQxxxvTWMLrD9ZdIcpCzmP1/HjxBS1W0yrPAWfclU1LSsdAtvM/MBluEe3BbRdJr/c8PWPPXCW65v0am6+B+66DTmM0wAjI6yEQM+yOFL4+a5q1E/ExW2SkmAMFTWepcrUkTARWP21TVPGWliukUIEbA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LXA62+xMX11sxy88lI6BLc2NogFQu9Gt+R69UZkk75M=;
 b=RsqB1tMsIyPD58DrIDVXVEPDcMQJ1JlfGHyDn0skhaQuu9lEXkVcyKmFOGDhiltfC8uQtx13LpK7bfGZNn7wKhUnxuEWf/1oK4cAkEpxZumQikCBh0p1UMv7pYplALCDdEErjg0vV586MTjmEz6AY2ZC3sNsTRabprCE1+r6WqVZ1mcfIVq5DGsg4pYhEWGwtiOVP3LhqBa7skI4Ym/iQaloZgddf7Nkt2Oef82WO5nQHkOar0mzE1PY+IzhDQwnbGwb/1/4patHzwXqnt9/VlzBwTW7HUBNpYr/dv1Byygl702WeKTU+4MIrMUz/8ZdOzkyVi/6GJvpp5RrI+M51g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LXA62+xMX11sxy88lI6BLc2NogFQu9Gt+R69UZkk75M=;
 b=6Xf1KV5c4HcBPTy9gQi4Kq8BLRA4e5zWLvxHcorCvrMW0YLSU3r9HsMinHTzC2TccrHMOeQXQJctF09TbbPWx53Q5RfRO/yJGN/sJsz+YiVnXdZxacw7r66Na4Ar3VzQ3IO7imU5n2LwDD8BeO+hIK00Fwe+2QtkBIN0CgAzcO8=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Topic: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Index: AQHXlPK6hMBa2cV9wU+RB+r01dTauquceRAAgAcQPwCAAMcsAIABJesA
Date: Wed, 15 Sep 2021 16:38:45 +0000
Message-ID: <8B84E0D6-70A7-4C8F-ACBE-D1773F6C8FAD@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s>
 <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
 <alpine.DEB.2.21.2109141543090.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109141543090.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ca40c206-e276-4b30-6b19-08d978675936
x-ms-traffictypediagnostic: AS8PR08MB5894:|AM6PR08MB4660:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB46608D5C12F9F53547B675EEFCDB9@AM6PR08MB4660.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NJ24YsY2VSMvTVT9a4ig3Map/lkvxGHx2w7s2ekiYGXP2Y9kMo77J02X1+FhBxqOZk+M4hchQOB92OFPy6XN5hoGDnmRAyB6qqmuzuQcDTBbDDrDGLYbLCZHyQsSafCymrZ5Io6tvvdbCHrk1T2CaHDchqjRLpwdon3dm5gOyw/Zus8Mo9sfm1RPk2ITaQt3u5FumNSO7En0BUBNXZnBxJEyV4Dh1yUVyZItaTlNmNjgKnfMG6cg/cvxSeW530Uc17JkWEXIRQIGiNIVpNrSNLPXIl713II/4XkN1/up7YzzOXrUWG1waSoeDJkY6T4l3e47o+mHKJbqpkHjE+4kicTAEXEx2e7SpVfY3mp9qFNX5lZNU/jkUzAq7Her/ZxmHOAsPn6g/Rku2y2vgPjOPmPKJV2s+E6caBjEuD9vsSre54utb01zPtPGjoZ7ogoG/ovl5/FiYwm+coMAy9u9Zvv9zOedcFKbP0WpEos1/bvobvZ1vvldvbhnbKh77GDw7Vb3EDvkVWBIyCjZmwh+tinfMr2e1PE4vPE+fPxpMAnsjPPr5a9eVMKiDJTY3x1pASsAyMrl4U9ILz7p74hbC2svBFEguyQmgbj3w8lTBGlBQqrzAjG/QViz117FQNJXhLwugGjIGetBk4AbuHbxWEGaxdRr9W4Mxfb6B8SEHiMKhAFON1pRB0Y7evQHkIswFImSzxFG07vmd47p6TEdRMtLfwHQ/3x5JCVJ8cJaAl+n1cLMOmdRpCvyoPvM4mk0tkwlNIo0bvlyGFiziSrzs8FvfHp1OANMZYX47JlAC3MWAp0svcesU32vshTXhYPkOJkj0/YhOhp3T4U2BaEa56Xq1R8Jkd93BaK/Wb1Ue1ciHbAorGNjEEAlbnrJ9W8r
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(366004)(376002)(346002)(5660300002)(8936002)(64756008)(966005)(91956017)(6506007)(66556008)(66946007)(316002)(2616005)(76116006)(66476007)(478600001)(36756003)(38070700005)(122000001)(6916009)(2906002)(26005)(38100700002)(71200400001)(8676002)(53546011)(54906003)(86362001)(33656002)(6512007)(186003)(6486002)(4326008)(66446008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <BF22CEC672FBC44BA10AD4625CC917DF@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB5894
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	01cafcc7-fa5a-4f50-7b03-08d9786749a4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Jb2wEJzyXUP7DbOWVw8ZUDPXe9S8GoUnclD/g6rxfnvXLpuWXKZJeGj76HYIGNlzJvxGE+F4RgI9kq+LJTSRLCI3K1RpfJV9QtwYR7JdCQvCuMlUUbKhAb/nMunGOcqNouDmu8z6uSbsO20Sojw65EJQHo4SKxLzl7TUzVyswbiVI0TUQkN9yUi3B8XaxYJXT39oJn1nzYd2clWN2/6MUW9LJjSuq7mTG+omlyiQ9Bnnsm9zdkhcEYH+l1UhnT2JXD5jFurrwB/6Z6OM6TF0enicTAG1zT26chVQvTmHkCNTC9a4HkSZCftvSrGXegus7XRX9TUeMbwOCvzY7ctI4g7zElk1vLU5L6AqRJXgo68rD12gzmVoUxB8f1C3uYc5laYn4Jhb08eqWwBnYnubAwE+aNWLg1A8/9Q8gF88s1cpG7KL7sTGVJmB2Tbg82lvuqAzcDoSKoleAX1vChgUuJgXU60nbAlAdvOKNtjLQ1wp7hyKZNNcBYlHyk1PtzAkUQHWMre3GnlS2KgZMW2fHcKhKq42TIjyBq0Y0J+e80f0jVhYLXh6hQV170zNOBM6D0xQLlDLQ1VrYjgQqdl0SmCm2sQ32QeVrJfhYZD/LQHyU+mTNfh66/yG7VtaOAkJeHqc+w9p/aLxduBWcqJ7u/RLx9BPvhFe878IfD0tc9gs7LPcGkPcoUmhaPwQjrMj4qU7Gs/VK6e8q/7hb8Sqx/ce+4h4fr4IQ9pLMt1Ju3KQy1MF0IdDPpIPIAJ5Ufxbk+3F4c2VJr8W3Xy6mI9p2hlHQ3Y4l35uwae80825pcOJvjgfuiYj4MD3wJ8cgiLyCpS5mtxzjc4E28qGuF2hRg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(346002)(136003)(39860400002)(376002)(46966006)(36840700001)(356005)(81166007)(36756003)(2906002)(70206006)(8936002)(6862004)(966005)(8676002)(5660300002)(54906003)(4326008)(478600001)(33656002)(26005)(316002)(107886003)(53546011)(6506007)(36860700001)(70586007)(336012)(6486002)(82740400003)(47076005)(186003)(82310400003)(6512007)(86362001)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 16:39:12.2559
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ca40c206-e276-4b30-6b19-08d978675936
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4660

SGkgU3RlZmFubywNCg0KPiBPbiAxNSBTZXAgMjAyMSwgYXQgMTI6MDYgYW0sIFN0ZWZhbm8gU3Rh
YmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4gd3JvdGU6DQo+IA0KPiBPbiBUdWUsIDE0
IFNlcCAyMDIxLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+Pj4gKyAgICAgICAgcmV0dXJuIE5VTEw7
DQo+Pj4+ICsNCj4+Pj4gKyAgICBidXNuIC09IGNmZy0+YnVzbl9zdGFydDsNCj4+Pj4gKyAgICBi
YXNlID0gY2ZnLT53aW4gKyAoYnVzbiA8PCBjZmctPm9wcy0+YnVzX3NoaWZ0KTsNCj4+Pj4gKw0K
Pj4+PiArICAgIHJldHVybiBiYXNlICsgKFBDSV9ERVZGTihzYmRmX3QuZGV2LCBzYmRmX3QuZm4p
IDw8IGRldmZuX3NoaWZ0KSArIHdoZXJlOw0KPj4+PiArfQ0KPj4+IA0KPj4+IEkgdW5kZXJzdGFu
ZCB0aGF0IHRoZSBhcm0zMiBwYXJ0IGlzIG5vdCBpbXBsZW1lbnRlZCBhbmQgbm90IHBhcnQgb2Yg
dGhpcw0KPj4+IHNlcmllcywgdGhhdCdzIGZpbmUuIEhvd2V2ZXIgaWYgdGhlIHBsYW4gaXMgdGhh
dCBhcm0zMiB3aWxsIGR5bmFtaWNhbGx5DQo+Pj4gbWFwIGVhY2ggYnVzIGluZGl2aWR1YWxseSwg
dGhlbiBJIGltYWdpbmUgdGhpcyBmdW5jdGlvbiB3aWxsIGhhdmUgYW4NCj4+PiBpb3JlbWFwIGlu
IHRoZSBhcm0zMiB2ZXJzaW9uLiBXaGljaCBtZWFucyB0aGF0IHdlIGFsc28gbmVlZCBhbg0KPj4+
IHVubWFwX2J1cyBjYWxsIGluIHN0cnVjdCBwY2lfb3BzLiBJIHVuZGVyc3RhbmQgdGhhdCBwY2lf
ZWNhbV91bm1hcF9idXMNCj4+PiB3b3VsZCBiZSBhIE5PUCB0b2RheSBmb3IgYXJtNjQsIGJ1dCBJ
IHRoaW5rIGl0IG1ha2VzIHNlbnNlIHRvIGhhdmUgaXQgaWYNCj4+PiB3ZSB3YW50IHRoZSBBUEkg
dG8gYmUgZ2VuZXJpYy4NCj4+IA0KPj4gQXMgcGVyIG15IHVuZGVyc3RhbmRpbmcgd2UgZG9u4oCZ
dCBuZWVkIHBjaV9lY2FtX3VubWFwX2J1cyguLikgYXMgSSBkb27igJl0IHNlZSBhbnkgdXNlIGNh
c2UgdG8gdW5tYXAgdGhlIA0KPj4gYnVzIGR5bmFtaWNhbGx5LiBXZSBjYW4gYWRkIHRoZSBzdXBw
b3J0IGZvciBwZXJfYnVzX21hcHBpbmcgZm9yIEFSTTMyIG9uY2Ugd2UgaW1wbGVtZW50IGFybTMy
IHBhcnQuDQo+PiBMZXQgbWUga25vdyB5b3VyIHZpZXcgb24gdGhpcy4NCj4gDQo+IEluIHRoZSBw
YXRjaCB0aXRsZWQgInhlbi9hcm06IFBDSSBob3N0IGJyaWRnZSBkaXNjb3Zlcnkgd2l0aGluIFhF
TiBvbg0KPiBBUk0iIHRoZXJlIGlzIHRoZSBmb2xsb3dpbmcgaW4tY29kZSBjb21tZW50Og0KPiAN
Cj4gKiBPbiA2NC1iaXQgc3lzdGVtcywgd2UgZG8gYSBzaW5nbGUgaW9yZW1hcCBmb3IgdGhlIHdo
b2xlIGNvbmZpZyBzcGFjZQ0KPiAqIHNpbmNlIHdlIGhhdmUgZW5vdWdoIHZpcnR1YWwgYWRkcmVz
cyByYW5nZSBhdmFpbGFibGUuICBPbiAzMi1iaXQsIHdlDQo+ICogaW9yZW1hcCB0aGUgY29uZmln
IHNwYWNlIGZvciBlYWNoIGJ1cyBpbmRpdmlkdWFsbHkuDQo+ICoNCj4gKiBBcyBvZiBub3cgb25s
eSA2NC1iaXQgaXMgc3VwcG9ydGVkIDMyLWJpdCBpcyBub3Qgc3VwcG9ydGVkLg0KPiANCj4gDQo+
IFNvIEkgdGFrZSBpdCB0aGF0IG9uIGFybTMyIHdlIGRvbid0IGhhdmUgZW5vdWdoIHZpcnR1YWwg
YWRkcmVzcyByYW5nZQ0KPiBhdmFpbGFibGUsIHRoZXJlZm9yZSB3ZSBjYW5ub3QgaW9yZW1hcCB0
aGUgd2hvbGUgcmFuZ2UuIEluc3RlYWQsIHdlJ2xsDQo+IGhhdmUgdG8gaW9yZW1hcCB0aGUgY29u
ZmlnIHNwYWNlIG9mIGVhY2ggYnVzIGluZGl2aWR1YWxseS4NCg0KWWVzIHlvdSBhcmUgcmlnaHQg
bXkgdW5kZXJzdGFuZCBpcyBhbHNvIHNhbWUuDQo+IA0KPiBJIGFzc3VtZWQgdGhhdCB0aGUgaWRl
YSB3YXMgdG8gY2FsbCBpb3JlbWFwIGFuZCBpb3VubWFwIGR5bmFtaWNhbGx5LA0KPiBvdGhlcndp
c2UgdGhlIHRvdGFsIGFtb3VudCBvZiB2aXJ0dWFsIGFkZHJlc3MgcmFuZ2UgcmVxdWlyZWQgd291
bGQgc3RpbGwNCj4gYmUgdGhlIHNhbWUuDQoNCkFzIHBlciBteSB1bmRlcnN0YW5kaW5nIGZvciAz
Mi1iaXQgd2UgbmVlZCBwZXJfYnVzIG1hcHBpbmcgYXMgd2UgZG9u4oCZdCBoYXZlIGVub3VnaCB2
aXJ0dWFsIGFkZHJlc3Mgc3BhY2UgaW4gb25lIGNodW5rDQpidXQgd2UgY2FuIGhhdmUgdmlydHVh
bCBhZGRyZXNzIHNwYWNlIGluIGRpZmZlcmVudCBjaHVuay4NCg0KSSBhbSBub3Qgc3VyZSBpZiB3
ZSBuZWVkIHRvIG1hcC91bm1hcCB0aGUgdmlydHVhbCBhZGRyZXNzIHNwYWNlIGZvciBlYWNoIHJl
YWQvd3JpdGUgY2FsbC4gDQpJIGp1c3QgY2hlY2tlZCB0aGUgTGludXggY29kZVsxXSAgYW5kIHRo
ZXJlIGFsc28gbWFwcGluZyBpcyBkb25lIG9uY2Ugbm90IGZvciBlYWNoIHJlYWQvd3JpdGUgY2Fs
bC4NCg0KWzFdICBodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9zY20vbGludXgva2VybmVsL2dp
dC90b3J2YWxkcy9saW51eC5naXQvdHJlZS9kcml2ZXJzL3BjaS9lY2FtLmM/aWQ9ZWE0YWFlMDU5
NzQzMzRlOTgzN2Q4NmZmMWNiNzE2YmFkMzZiM2NhOCNuNzYNCg0KPiBJIGFsc28gdGhvdWdodCB0
aGF0IHRoZSBkeW5hbWljIG1hcHBpbmcgZnVuY3Rpb24sIHRoZSBvbmUNCj4gd2hpY2ggd291bGQg
ZW5kIHVwIGNhbGxpbmcgaW9yZW1hcCBvbiBhcm0zMiwgd291bGQgYmUgcGNpX2VjYW1fbWFwX2J1
cy4NCj4gSWYgc28sIHRoZW4gd2UgYXJlIG1pc3NpbmcgdGhlIGNvcnJlc3BvbmRpbmcgdW5tYXBw
aW5nIGZ1bmN0aW9uLCB0aGUgb25lDQo+IHRoYXQgd291bGQgY2FsbCBpb3VubWFwIG9uIGFybTMy
IGFuZCBkbyBub3RoaW5nIG9uIGFybTY0LCBjYWxsZWQgYmVmb3JlDQo+IHJldHVybmluZyBmcm9t
IHBjaV9nZW5lcmljX2NvbmZpZ19yZWFkIGFuZCBwY2lfZ2VuZXJpY19jb25maWdfd3JpdGUuDQo+
IA0KPiBBcyBhbHdheXMsIEkgYW0gbm90IGFza2luZyBmb3IgdGhlIGFybTMyIGltcGxlbWVudGF0
aW9uLCBidXQgaWYgd2UgYXJlDQo+IGludHJvZHVjaW5nIGludGVybmFsIEFQSXMsIGl0IHdvdWxk
IGJlIGdvb2QgdGhhdCB0aGV5IGFyZSBjb25zaXN0ZW50Lg0KPiANCj4gDQo+IA0KPj4+PiBAQCAt
NTAsMTAgKzUxLDQxIEBAIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2Ugew0KPj4+PiAgICB1OCBidXNf
c3RhcnQ7ICAgICAgICAgICAgICAgICAgICAvKiBCdXMgc3RhcnQgb2YgdGhpcyBicmlkZ2UuICov
DQo+Pj4+ICAgIHU4IGJ1c19lbmQ7ICAgICAgICAgICAgICAgICAgICAgIC8qIEJ1cyBlbmQgb2Yg
dGhpcyBicmlkZ2UuICovDQo+Pj4+ICAgIHZvaWQgKnN5c2RhdGE7ICAgICAgICAgICAgICAgICAg
IC8qIFBvaW50ZXIgdG8gdGhlIGNvbmZpZyBzcGFjZSB3aW5kb3cqLw0KPj4+PiArICAgIGNvbnN0
IHN0cnVjdCBwY2lfb3BzICpvcHM7DQo+Pj4+IH07DQo+Pj4+IA0KPj4+PiArc3RydWN0IHBjaV9v
cHMgew0KPj4+PiArICAgIHZvaWQgX19pb21lbSAqKCptYXBfYnVzKShzdHJ1Y3QgcGNpX2hvc3Rf
YnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVpbnQzMl90IG9mZnNldCk7DQo+Pj4+ICsgICAgaW50ICgqcmVhZCkoc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1aW50MzJfdCBzYmRmLA0KPj4+PiArICAgICAgICAg
ICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3QgbGVuLCB1aW50MzJfdCAqdmFsdWUpOw0KPj4+
PiArICAgIGludCAoKndyaXRlKShzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQz
Ml90IHNiZGYsDQo+Pj4+ICsgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3Qg
bGVuLCB1aW50MzJfdCB2YWx1ZSk7DQo+Pj4+ICt9Ow0KPj4+PiArDQo+Pj4+ICsvKg0KPj4+PiAr
ICogc3RydWN0IHRvIGhvbGQgcGNpIG9wcyBhbmQgYnVzIHNoaWZ0IG9mIHRoZSBjb25maWcgd2lu
ZG93DQo+Pj4+ICsgKiBmb3IgYSBQQ0kgY29udHJvbGxlci4NCj4+Pj4gKyAqLw0KPj4+PiArc3Ry
dWN0IHBjaV9lY2FtX29wcyB7DQo+Pj4+ICsgICAgdW5zaWduZWQgaW50ICAgICAgICAgICAgYnVz
X3NoaWZ0Ow0KPj4+PiArICAgIHN0cnVjdCBwY2lfb3BzICAgICAgICAgIHBjaV9vcHM7DQo+Pj4+
ICsgICAgaW50ICgqaW5pdCkoc3RydWN0IHBjaV9jb25maWdfd2luZG93ICopOw0KPj4+IA0KPj4+
IElzIHRoaXMgbGFzdCBtZW1iZXIgb2YgdGhlIHN0cnVjdCBuZWVkZWQvdXNlZD8NCj4+IA0KPj4g
WWVzIEl0IHdpbGwgYmUgdXNlZCBieSBzb21lIHZlbmRvciBzcGVjaWZpYyBib2FyZCggTjFTRFAg
KS4gUGxlYXNlIGNoZWNrIGJlbG93Lg0KPj4gaHR0cHM6Ly9naXQubGluYXJvLm9yZy9sYW5kaW5n
LXRlYW1zL3dvcmtpbmcvYXJtL24xc2RwLXBjaWUtcXVpcmsuZ2l0L2NvbW1pdC8/aWQ9MDRiN2U3
NmQwZmU2NDgxYTgwM2Y1OGU1NGUwMDhhMTQ4OWQ3MTNhNQ0KPiANCj4gT0suIEkgZG9uJ3QgZG91
YnQgdGhhdCB0aGVyZSBtaWdodCBiZSBicmlkZ2Utc3BlY2lmaWMgaW5pdGlhbGl6YXRpb25zLA0K
PiBidXQgd2UgYWxyZWFkeSBoYXZlIHRoaW5ncyBsaWtlOg0KPiANCj4gRFRfREVWSUNFX1NUQVJU
KHBjaV9nZW4sICJQQ0kgSE9TVCBaWU5RTVAiLCBERVZJQ0VfUENJKQ0KPiAuZHRfbWF0Y2ggPSBn
ZW5fcGNpX2R0X21hdGNoLA0KPiAuaW5pdCA9IGdlbl9wY2lfZHRfaW5pdCwNCj4gRFRfREVWSUNF
X0VORA0KPiANCj4gVGhlIHF1ZXN0aW9uIGlzIGRvIHdlIGFjdHVhbGx5IG5lZWQgdHdvIGRpZmZl
cmVudCB2ZW5kb3Itc3BlY2lmaWMgaW5pdA0KPiBmdW5jdGlvbnM/IE9uZSBpcyBkcml2ZW4gYnkg
ZGV2aWNlIHRyZWUsIGUuZy4gWnlucU1QIGlzIGNhbGxpbmcNCj4gZ2VuX3BjaV9kdF9pbml0LiBU
aGUgb3RoZXIgb25lIGlzIHBjaV9lY2FtX29wcy0+aW5pdCwgd2hpY2ggaXMgY2FsbGVkDQo+IGZy
b20gdGhlIGZvbGxvd2luZyBjaGFpbjoNCj4gDQo+IERUX0RFVklDRV9TVEFSVCAtPiBnZW5fcGNp
X2R0X2luaXQgLT4gcGNpX2hvc3RfY29tbW9uX3Byb2JlIC0+DQo+IGdlbl9wY2lfaW5pdCAtPiBw
Y2lfZ2VuZXJpY19lY2FtX29wcy5pbml0DQo+IA0KPiBXaGF0J3MgdGhlIGRpZmZlcmVuY2UgYmV0
d2VlbiBnZW5fcGNpX2R0X2luaXQgYW5kIHBjaV9lY2FtX29wcy5pbml0IGluDQo+IHRlcm1zIG9m
IHB1cnBvc2U/DQoNClllcyBJIGFsc28gYWdyZWUgd2l0aCB5b3Ugd2UgZG9u4oCZdCBuZWVkIHR3
byBpbml0IGZ1bmN0aW9uLiBJIHdpbGwgbW9kaWZ5IHRoZSBjb2RlIGxpa2UgYmVsb3cuDQpEVF9E
RVZJQ0VfU1RBUlQgLT5wY2lfaG9zdF9jb21tb25fcHJvYmUgLT4gZ2VuX3BjaV9pbml0IC0+IHBj
aV9nZW5lcmljX2VjYW1fb3BzLmluaXQNCg0KUmVnYXJkcywNClJhaHVsDQo+IA0KPiBJIGFtIGhh
cHB5IHRvIGhhdmUgcGNpX2VjYW1fb3BzLmluaXQgaWYgaXQgc2VydmVzIGEgZGlmZmVyZW50IHB1
cnBvc2UNCj4gZnJvbSBEVF9ERVZJQ0VfU1RBUlQuaW5pdC4gSW4gdGhhdCBjYXNlIHdlIG1pZ2h0
IHdhbnQgdG8gYWRkIGFuIGluLWNvZGUNCj4gY29tbWVudCBzbyB0aGF0IGFuIGVuZ2luZWVyIHdv
dWxkIGtub3cgd2hlcmUgdG8gYWRkIHZlbmRvci1zcGVjaWZpYyBjb2RlDQo+ICh3aGV0aGVyIHRv
IERUX0RFVklDRV9TVEFSVC5pbml0IG9yIHRvIHBjaV9lY2FtX29wcy5pbml0KS4NCg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 16:42:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 16:42:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187824.336817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXzb-0000bs-Us; Wed, 15 Sep 2021 16:42:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187824.336817; Wed, 15 Sep 2021 16:42:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQXzb-0000bl-Rs; Wed, 15 Sep 2021 16:42:39 +0000
Received: by outflank-mailman (input) for mailman id 187824;
 Wed, 15 Sep 2021 16:42:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LStB=OF=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQXza-0000bf-HB
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 16:42:38 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.51]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 839667b1-c68d-4c79-971b-80b0844fc646;
 Wed, 15 Sep 2021 16:42:35 +0000 (UTC)
Received: from DB6PR07CA0014.eurprd07.prod.outlook.com (2603:10a6:6:2d::24) by
 VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Wed, 15 Sep
 2021 16:42:25 +0000
Received: from DB5EUR03FT025.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2d:cafe::bc) by DB6PR07CA0014.outlook.office365.com
 (2603:10a6:6:2d::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.9 via Frontend
 Transport; Wed, 15 Sep 2021 16:42:25 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT025.mail.protection.outlook.com (10.152.20.104) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.14 via Frontend Transport; Wed, 15 Sep 2021 16:42:24 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Wed, 15 Sep 2021 16:42:24 +0000
Received: from 9b4b76d4de04.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 75EEBD51-898B-4586-86D4-15F922DD89C2.1; 
 Wed, 15 Sep 2021 16:42:14 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9b4b76d4de04.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 15 Sep 2021 16:42:14 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4181.eurprd08.prod.outlook.com (2603:10a6:20b:ad::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Wed, 15 Sep
 2021 16:42:03 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.014; Wed, 15 Sep 2021
 16:42:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 839667b1-c68d-4c79-971b-80b0844fc646
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZxhRmcR5JJUyjWh/8bUvn6442z4LPKazDKZWSrpxOCc=;
 b=Ice+wrHQEqMEiWn5V5I42GZGi157MsHuKFT2iT3iSDZaZTW/DYKa7F2fHUe6TU/+SY8uEc6+vaxg5MvLJEQwGKiIfn/T0qxmqZJIwCKd5MxxF4SvAPq5d0ohPXvJc6WJTNsYwiz6QMWxWrGUZifFNl0/i2CCWoclAy0KjKIDAY0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 60c57de651d18c16
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XooCQd6H5qSSy987/LRUULqipw7VgDKYiHdoFfnY22R2x6LHegAHYNlKgeWNpHRUa+c3K9mIpRo2qc3Nb1Y6dV/GptgpFL3IAriajTGYSfS1sHGjc5sskwXG7lgDg58IXaMl7kd4rAp35E4hhXdjSmOuSKrnvZ0c/QWq0KBxkRLOfmmsRiGZLGrPWiwuBBnoRXRT2oEyvGTF/uYhneVFcEP+zkMynbEH5pXAo1FKUbA1KKhf4CyiWsKmtmDxUfWbDRNxfZ7goP3VMDXGwZ06+LR/VtByYPYWsomggq9PlDrcpAWUtc6JfmYyh0HvIH8O2EgHwO+d9Tf+Dz0zyMyytw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZxhRmcR5JJUyjWh/8bUvn6442z4LPKazDKZWSrpxOCc=;
 b=RjK+RKlV7W2SVJ8Sj/uayJkK3APikfdGSYqtYJhghW7Fp9/kWSdPrezbQI3IuFofUizDIut8y7IhWRfPS1uREm2IOYN0rg3LU9xMg6MqbMgvPQq745RIUCZsHgKfAQIw7jnAtQY09FaRCvQHUAekTB0tJ60ZUDxGQGVvhD7S15UqwTTCTznVDfr9xzK4wfWVmQh5ZxAobbqcniDInrxb5gpb9QlU1/0+MooU+WeywhiVQFBDbv1Auh27QDYYQQQBWcHIwgyyLN0fMNVEKShCtTUtGthTv7u4DyMv3NsWJpDMD242uHMowYIlpO8SgtJcp8/J3/zc6PRuHCP4pYtInA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZxhRmcR5JJUyjWh/8bUvn6442z4LPKazDKZWSrpxOCc=;
 b=Ice+wrHQEqMEiWn5V5I42GZGi157MsHuKFT2iT3iSDZaZTW/DYKa7F2fHUe6TU/+SY8uEc6+vaxg5MvLJEQwGKiIfn/T0qxmqZJIwCKd5MxxF4SvAPq5d0ohPXvJc6WJTNsYwiz6QMWxWrGUZifFNl0/i2CCWoclAy0KjKIDAY0=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien.grall.oss@gmail.com>, Oleksandr Andrushchenko <andr2000@gmail.com>,
	xen-devel <xen-devel@lists.xenproject.org>, Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>,
	=?iso-8859-1?Q?Roger_Pau_Monn=E9?= <roger.pau@citrix.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index:
 AQHXoJ53JNtXRAJCBkuwB/fHai0QiKucB06AgAE4wgCAAAtQgIAADDiAgAARZwCAAFsJAIAAFA4AgAO3egCAAc7YgIAA860AgABTwQCAALojgA==
Date: Wed, 15 Sep 2021 16:42:02 +0000
Message-ID: <5AFD57E4-1C4B-4781-ADAC-BA3C4AC2E2D1@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
 <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
 <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
 <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s>
 <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
In-Reply-To: <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 49da10d1-ad6e-46ac-5fdd-08d97867cc03
x-ms-traffictypediagnostic: AM6PR08MB4181:|VE1PR08MB5215:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB521548CF32594DFD71512E68FCDB9@VE1PR08MB5215.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NlxyQFf6csXoAqJkProLpb9XxdJqhQikgf7AUja7PmGPp39Hh0qRzAqP+VKR1Qy1XPPSErfmJrdM3R+1ah9tpGtBcAeahSfZfHr4OIbpMTQeSn1raUTn6DkqrjuR4RTjZeX3SUjX5Jwt4O9S7Xot0H+LSgmmt7zeJAZw2nCxzpt93dhroqOO4MWo+BZgC+3AjtVRBvNKYpV3mEQPsnDEWPLnoz1o/fSsm3/CKgXxp4haLbLM3ZrQSmE1/+p8Eg5VqVo/KpYEyUtuqpYAzgiS05j+LkOL8pW9/r6t8vX5Lq5CPmscdG1PMgJreEryidfjudMH1R6zJt1gPwQdH+t7m6HHJLvSy2YAMDj4//52HfeTXS9jpBUoSxAZo1HiJ1fTMk6onaA8aATWtm0YSOrzx8SRMNNYn6n3GkFgT/yt9vY98f7pAKm91bQq7ux2I/WGoMHd0azsX1ENlA+qTV3g7y7+Z8WOF2qMR68v+9FUX8Y0dvBNTFH69Zti7cTa9rx4y3WlPWT47o3k4+/DPsVC7MhuCGqCY4CPsocae+BahQrsbEMKwkX5q821fbgKkh7MrrMiTyo+DPai98HcAvJmAHcxVHNFCBihXAFCvzrqJr7jw7GvHtKJgAWk+aNVGFL5O7WYGFLw9aSLqfRurowNI/lRa8a4rpszURjVdOv/zLPBX/XEyNQ1LncAFAo7wxJYD5pZHDYs5xFhN8PAEYy7AqmHOOPsuT+/14XFqc6Xzg9Mh71llVf2K+1EKwh3GcKo
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(64756008)(2616005)(508600001)(71200400001)(66446008)(66946007)(66556008)(66476007)(6916009)(122000001)(316002)(4326008)(2906002)(6486002)(36756003)(54906003)(6512007)(38100700002)(5660300002)(76116006)(33656002)(8936002)(91956017)(8676002)(186003)(26005)(6506007)(53546011)(38070700005)(86362001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <08953BF16169FF4CA68C28DC034218C6@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4181
Original-Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT025.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c09dd4e8-678a-4ba7-cfaa-08d97867bf02
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zICcguju0H4mr4nuZneEzQfv58J95Qc7tI5VrOzryinvKud/ZFkuD6uCyn8uELcvwmHhoh9VHwjPrM1AkxmsOkKBEex+jG8k1Rfjf2EGEfMfMQ0rJ1SogbDsrmRkdR92RkoCQH3HivNq7S+UpeGj0GuLkm86yTkzaa92G7S+8pBHZwQpHvZ4nMv7XSlocpWIrRiy75pXIo+Yzyy7oc3GoJuH9v4hnErIZ5p81EqV465a5Avg2rvJrm/VGM5lKXYXwPDKkTBocJEI/JXjSArrToVrPCPOuKNq6t9a39ZjZUMP34NtpTrOFPqAeBDxNsa19QH964S91YjNz+4Pvi6evuxr7JuWDrxZo14McnDaCSJm68Tvf4PPL/VTeFp2mdouxyVCW3ArOyKzN+NluRV893dg9QGW7mFGrbobi8rVKCuxAGScoLXxj4cm/c9CwbcgbnYdKQ/r4UtBFTa15S/nVhUPIXJJUOf6zej+vzYVUi4OjqI5VxD+wcUzOFl7iqUfKLAVCqfTu0Gnxmz7TDlEblGV3SpMRTlW1G6HCDNvUPhl2lTeOC4+NkiFFuDOwy3g0bcUpcnLtqo7rkI/3TX6LMZ3b8ElAQ7ODHDNELTDGFbowa0Lyg95XKg8qUrzWgH5nbqG3T+6JQmJ8tSsfOgwvmV8J3/sHKIjh6bz9BETV1Z/9qRAGr0zGjMKDuOOnGShiHo6+W8vHL7ckJj5jTsFlg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36756003)(53546011)(36860700001)(6506007)(70586007)(2906002)(6862004)(47076005)(86362001)(70206006)(82310400003)(83380400001)(81166007)(336012)(33656002)(2616005)(26005)(5660300002)(316002)(8936002)(54906003)(6512007)(6486002)(8676002)(4326008)(356005)(508600001)(186003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 16:42:24.9303
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 49da10d1-ad6e-46ac-5fdd-08d97867cc03
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT025.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5215

Hi Oleksandr,

> On 15 Sep 2021, at 6:35 am, Oleksandr Andrushchenko <Oleksandr_Andrushche=
nko@epam.com> wrote:
>=20
> Hi, Stefano, Rahul!
>=20
> On 15.09.21 03:36, Stefano Stabellini wrote:
>> On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
>>> With the patch above I have the following log in Domain-0:
>>>=20
>>> generic-armv8-xt-dom0 login: root
>>> root@generic-armv8-xt-dom0:~# (XEN) *** Serial input to Xen (type 'CTRL=
-a' three times to switch input)
>>> (XEN) =3D=3D=3D=3D PCI devices =3D=3D=3D=3D
>>> (XEN) =3D=3D=3D=3D segment 0000 =3D=3D=3D=3D
>>> (XEN) 0000:03:00.0 - d0 - node -1
>>> (XEN) 0000:02:02.0 - d0 - node -1
>>> (XEN) 0000:02:01.0 - d0 - node -1
>>> (XEN) 0000:02:00.0 - d0 - node -1
>>> (XEN) 0000:01:00.0 - d0 - node -1
>>> (XEN) 0000:00:00.0 - d0 - node -1
>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch inp=
ut)
>>>=20
>>> root@generic-armv8-xt-dom0:~# modprobe e1000e
>>> [   46.104729] e1000e: Intel(R) PRO/1000 Network Driver
>>> [   46.105479] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
>>> [   46.107297] e1000e 0000:03:00.0: enabling device (0000 -> 0002)
>>> (XEN) map [e0000, e001f] -> 0xe0000 for d0
>>> (XEN) map [e0020, e003f] -> 0xe0020 for d0
>>> [   46.178513] e1000e 0000:03:00.0: Interrupt Throttling Rate (ints/sec=
) set to dynamic conservative mode
>>> [   46.189668] pci_msi_setup_msi_irqs
>>> [   46.191016] nwl_compose_msi_msg msg at fe440000
>>> (XEN) traps.c:2014:d0v0 HSR=3D0x00000093810047 pc=3D0xffff8000104b4b00 =
gva=3D0xffff800010fa5000 gpa=3D0x000000e0040000
>>> [   46.200455] Unhandled fault at 0xffff800010fa5000
>>>=20
>>> [snip]
>>>=20
>>> [   46.233079] Call trace:
>>> [   46.233559]  __pci_write_msi_msg+0x70/0x180
>>> [   46.234149]  pci_msi_domain_write_msg+0x28/0x30
>>> [   46.234869]  msi_domain_activate+0x5c/0x88
>>>=20
>>>  From the above you can see that BARs are mapped for Domain-0 now
>>>=20
>>> only when an assigned PCI device gets enabled in Domain-0.
>>>=20
>>> Another thing to note is that we crash on MSI-X access as BARs are mapp=
ed
>>>=20
>>> to the domain while enabling memory decoding in the COMMAND register,
>>>=20
>>> but MSI-X are handled differently, e.g. we have MSI-X holes in the mapp=
ings.
>>>=20
>>> This is because before this change the whole PCI aperture was mapped in=
to
>>>=20
>>> Domain-0 and it is not. Thus, MSI-X holes are left unmapped now and the=
re
>>>=20
>>> was no need to do so, e.g. they were always mapped into Domain-0 and
>>>=20
>>> emulated for guests.
>>>=20
>>> Please note that one cannot use xl pci-attach in this case to attach th=
e PCI device
>>>=20
>>> in question to Domain-0 as (please see the log) that device is already =
attached.
>>>=20
>>> Neither it can be detached and re-attached. So, without mapping MSI-X h=
oles for
>>>=20
>>> Domain-0 the device becomes unusable in Domain-0. At the same time the =
device
>>>=20
>>> can be successfully passed to DomU.
>>>=20
>>>=20
>>> Julien, Stefano! Please let me know how can we proceed with this.
>> What was the plan for MSI-X in Dom0?
> It just worked because we mapped everything
>>=20
>> Given that Dom0 interacts with a virtual-ITS and virtual-GIC rather than
>> a physical-ITS and physical-GIC, I imagine that it wasn't correct for
>> Dom0 to write to the real MSI-X table directly?
>>=20
>> Shouldn't Dom0 get emulated MSI-X tables like any DomU?
>>=20
>> Otherwise, if Dom0 is expected to have the real MSI-X tables mapped, the=
n
>> I would suggest to map them at the same time as the BARs. But I am
>> thinking that Dom0 should get emulated MSI-X tables, not physical MSI-X
>> tables.
>=20
> Yes, it seems more than reasonable to enable emulation for Domain-0
>=20
> as well. Other than that, Stefano, do you think we are good to go with
>=20
> the changes I did in order to unmap everything for Domain-0?
>=20
> Rahul, it seems we will need a change to vPCI/MSI-X so we can also
>=20
> trap Domain-0 for MSI-X tables.

I agree that we need emulated MSI-X tables for Dom0 also. Let me check on t=
his and come back to you.

Regards,
Rahul
>=20
> Thank you,
>=20
> Oleksandr



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 16:47:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 16:47:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187834.336828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQY3y-0001TN-Jk; Wed, 15 Sep 2021 16:47:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187834.336828; Wed, 15 Sep 2021 16:47:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQY3y-0001TG-Ga; Wed, 15 Sep 2021 16:47:10 +0000
Received: by outflank-mailman (input) for mailman id 187834;
 Wed, 15 Sep 2021 16:47:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xUnt=OF=microsoft.com=haiyangz@srs-us1.protection.inumbo.net>)
 id 1mQY3x-0001TA-2Z
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 16:47:09 +0000
Received: from na01-obe.outbound.protection.outlook.com (unknown [40.93.3.13])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 90687a12-1644-11ec-b553-12813bfff9fa;
 Wed, 15 Sep 2021 16:47:07 +0000 (UTC)
Received: from MN2PR21MB1295.namprd21.prod.outlook.com (2603:10b6:208:3e::25)
 by MN2PR21MB1455.namprd21.prod.outlook.com (2603:10b6:208:204::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.0; Wed, 15 Sep
 2021 16:46:57 +0000
Received: from MN2PR21MB1295.namprd21.prod.outlook.com
 ([fe80::d804:7493:8e3d:68d3]) by MN2PR21MB1295.namprd21.prod.outlook.com
 ([fe80::d804:7493:8e3d:68d3%9]) with mapi id 15.20.4478.015; Wed, 15 Sep 2021
 16:46:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90687a12-1644-11ec-b553-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WpXMWDJWITqLlHVuVUlYkGNdC5zpr0MKrRTLiu7PxXfOHYPqUdmzQNOK+VDHRDdnaJz01Zl0c+562eTkkvq0R6wNXrhU1nLbY3xczKq01/daRhqO4LKNHcl3Bsni8GjTIeqrzRTvJtGqX6ORBg8haFA8oB/hTQ0wE/JxarZrzfOszCtf4F0G1QIrFHYU080KTRBbUh1tUBmMbd+DOFBCUvDvHlll56yl2QEbqm+fZdSZbnOXZS2iWHdCGxQII2YSFjvtXgost/8fLZn8oYTWfPNZLjcYTLFusppsgo3VvFgY1xwfmTDvAc4zRoApxCYrPwLG3b55W9aDb6Gyl++a3Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kjTAfamJN+Enx+YU2cWRaztjgL/WJp+2S1JsOQ69j48=;
 b=CHkdhF7/0Tq3At5Xe198lDtLqc28PZmOjFBy+Y1B5I2Zr1tu6wIsOs2ZDdpDoVcsogFaOoqDTCRYh74+nQAzGDmczg1jpXaYdrjqPWfWiGuGOUFQYUVesVHn3cjzINX53zYkrPdNCfjJoDNisCuASQhW8lVNgMLiUbtBDBMHiJhmMGLETSBf3XVBYleO0vQjFZJtSsA5ZOFk8VyBX5XaDGhcrmiBB+Tcm5nOnTH4mU972qRLol90SZGNWb0O888Aoq2cJz1AnQ90bJmqzrZQC4gFfkyjVheXfhdVbeVk31UmxscngwTzKvgqYEvtXWhezWPI+BQClyQe1yZa20i5kA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=microsoft.com; dmarc=pass action=none
 header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kjTAfamJN+Enx+YU2cWRaztjgL/WJp+2S1JsOQ69j48=;
 b=WGxDtOIHHtg9DTaJGq84oDMuvZDhrqCE52/DZ+mAONT93w8N/L9H46mTdPGe0plm7rfGfD4TojdeP4QMpwPE1TVCvuQIkNlAgp50N+MOz/phMdKULz9HE+b3IWL6DtEj1rRZ0uhFX+z0GudtP/ULc7qat8QnPJdYep7jpWxFZSE=
From: Haiyang Zhang <haiyangz@microsoft.com>
To: Michael Kelley <mikelley@microsoft.com>, Tianyu Lan <ltykernel@gmail.com>,
	KY Srinivasan <kys@microsoft.com>, Stephen Hemminger
	<sthemmin@microsoft.com>, "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan
 Cui <decui@microsoft.com>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, "jgross@suse.com"
	<jgross@suse.com>, "sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>, "kuba@kernel.org"
	<kuba@kernel.org>, "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "arnd@arndb.de"
	<arnd@arndb.de>, "hch@lst.de" <hch@lst.de>, "m.szyprowski@samsung.com"
	<m.szyprowski@samsung.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>, Tianyu Lan
	<Tianyu.Lan@microsoft.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>, "sfr@canb.auug.org.au"
	<sfr@canb.auug.org.au>, "aneesh.kumar@linux.ibm.com"
	<aneesh.kumar@linux.ibm.com>, "saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
CC: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>, vkuznets
	<vkuznets@redhat.com>, "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>
Subject: RE: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
Thread-Topic: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
 netvsc driver
Thread-Index: AQHXqW4FQ8MnKzPhAECYP0JKGb2C7qulSMAAgAACvIA=
Date: Wed, 15 Sep 2021 16:46:57 +0000
Message-ID:
 <MN2PR21MB12959F10240EC1BB2270B345CADB9@MN2PR21MB1295.namprd21.prod.outlook.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
In-Reply-To:
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=131cd4f8-e33d-45d8-94eb-3fa8bffaa59b;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-09-14T19:30:32Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=microsoft.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 65fcef49-2bfc-4abc-9a1a-08d978686e90
x-ms-traffictypediagnostic: MN2PR21MB1455:
x-ms-exchange-transport-forked: True
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs:
 <MN2PR21MB14556F18B555DA028D4C2EF9CADB9@MN2PR21MB1455.namprd21.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 xGl1C0Su/tD+WOaQChXs8rzYHOsiTb6SivahjGE5kjmdTc8hP9NgTlOaTCET+LqroRb7MIhxhzdyHAysilDMNGnbZuFV6mN6pBzlz87RadM4TPmTgdGX6IVVCZywiM6YHjW2WMVtH5PxXu+Oy/2AFit29qvXSvX9sBfTQcbTUD/6WnqZepiear0iBM81id7fDrx2g6W9TUSU6O4MirUdYdH4AOvjgQTsbIlhJwiOCRdTFwdssvyJRTP9mqcGUhhUsaHZ5dNDHXDNlt3ghjzFtkrKWNSR4H+oAAEnOkVP7HvV4MleuEtAEqbSR2ZRqy7e1VMUjJSnAtUA0aTPfD7kXsA6qDBqn8JoiNS43PAgJATnJHrY1xJGt63lQi8G6OyGykzlTRzPWqQYZxrtLHLE8/FS17dUYe8A6JdY/h+ZzG+kZqQX4xpr8lGfcxau8JnPUICI5eZO4HSD/MQDiifehrUO7L5RfK7ctMoXdThWwhWr0il60Ej94prC3IhBH3+ccPXirTXnyCqeAZgmiX+KuLnPQuwssw1Omw4mHiHVL5Xh7HJaZVSGwvu1L1odlbz3PIWRQWynV62IOulmsm8tf+dOuuIhi2OyyJuADtmRnpryk/vvmqLAUbSfjcw/oWgPsd8VUknskrdC9uKgHxwOq6NuRTvHbrARI1G3zQKJ64d7s6+BZAvMP7go+N9H1p7Tk2Bk9T6ULJKbKZf1X+PzP6b+scpn0llycirurm0mUSg=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR21MB1295.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52536014)(7696005)(82950400001)(8990500004)(508600001)(82960400001)(5660300002)(71200400001)(26005)(66946007)(54906003)(9686003)(38070700005)(66556008)(4326008)(186003)(10290500003)(2906002)(921005)(7416002)(110136005)(53546011)(83380400001)(6506007)(122000001)(38100700002)(8936002)(66446008)(64756008)(8676002)(66476007)(316002)(7406005)(76116006)(33656002)(86362001)(55016002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?0q2FR5kBUsiHGn8VbWhX7EDw1OB0uXP/oD1rM/XhCIvpmI8fc3j2b9++DdZR?=
 =?us-ascii?Q?/QGDa9anxw38L13zRnzafY3JdiV7PtD1SyyYiBt+qV/xif3hUTN+AEyZnytS?=
 =?us-ascii?Q?wVkd6cOeLSdsqwUtgY+OGKyqqMkvFFfmx1xPBbPQWjIEYnp/8gdpZtqwn8Js?=
 =?us-ascii?Q?UKyuFZjK1BxDkYflgvLuhl1tTQRIHZmSNTeJhOqOhcFrxSgWGbeRz1N9822x?=
 =?us-ascii?Q?Z5DRy+fVkAS64W4Ku3uqLcG7mtnk78COlFsTGvl7A+D0GJakK2UVqz136dtB?=
 =?us-ascii?Q?6hBELlgsY4oqvtjZ54CxhPEcy/ye03MBgUxHH9nA98mOw+KNzn+Qsb+CIt1S?=
 =?us-ascii?Q?H0ybkOMVXaRKwLzjosnyEefBLFx3h4g2DnvDeT7REBN01NpsrAtNEjMD9beT?=
 =?us-ascii?Q?bQncdkaDpS/hoHN4aKop7UqQxXO7MR22XajM8w5ix1zcYnOHaty60utyXXr1?=
 =?us-ascii?Q?syH+BiYc4sVnqOb6JYLE+b2XcZZSJgzfyTH9OANK2AMg2IRIWbte+54Qsmdm?=
 =?us-ascii?Q?X4RdWSSIhVereLBEeVhooh+79bAfO+fkXaQcM3dP1Ikj//4PqNlFkG2ADOYB?=
 =?us-ascii?Q?BU92zEee9pYGD5hag2fYtnpgTrVtVG/9UPsXjtF72eySFj6J3olhVHeMrBB+?=
 =?us-ascii?Q?yrDMy4mo/2cE9De6SK932A1FICdu0HQ0X96sb5ozkGsmO+MpnnekF6KiXMgg?=
 =?us-ascii?Q?qD5OyQea/TGfbMp49h71v3+zUxYxLSqZu/Yc7dTSUMc6YKz3DeMN852w7lSV?=
 =?us-ascii?Q?vfT/Nom/gDA8qYP1lPOZm3UYYTbYdjW15MnE7W357On9T+s3A/LQO7nOCKyz?=
 =?us-ascii?Q?WhI+i+00esKNAG7k4E2R7nQXNVwx4kVuA5xKicqxOQm9hBNpFSd93nVHedcc?=
 =?us-ascii?Q?C5iJ7MawTgv/Q5nEArqQQV+6vN12x2dXnAlOV1C/v7TuSPB5NH8cfPZU795v?=
 =?us-ascii?Q?LO9TtYIDRbuWOXT84eqf+N0r/JjYIp57XDuOgDKbTHYL4FugnJKCiyFPH4EG?=
 =?us-ascii?Q?UtV8MnmBxujLuLFLfoYlocN6+tWx6L1hZHGYE8eew0d2ZRH+cgQ5EjrAbovF?=
 =?us-ascii?Q?S8Yff10ScEbMctALgybHNUclMjXEnRkCL25pTdQzaGSnNmn95xPwTneIxoKb?=
 =?us-ascii?Q?JuyqWTxOnUWt+Pf9MnMUzhftDFWXZe0wIqcvssHF4T3O5eKSvLOTp5YdhMaS?=
 =?us-ascii?Q?ZQ1v8XR5eD1iRcEKnO/aKKAzhaPwS0UnbTdQKaiAc7RrbPq+hsqDksjeD6v0?=
 =?us-ascii?Q?v357YMl09zW91F1P+vwp9DVHsU1MVbofYOaV6wdkp07Q5sE3yjYX45SxbdaK?=
 =?us-ascii?Q?iUMuwk13GMcDKglnttym8iNx?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR21MB1295.namprd21.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 65fcef49-2bfc-4abc-9a1a-08d978686e90
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2021 16:46:57.5268
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4XHlmYxUxbmdLdPSH5H35ZgFxyje56csxvRzpdGEKZVJtlsm+u8WV25TtsJyZeKdAPgAsjdM0HUeAigA+Ad5SA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR21MB1455



> -----Original Message-----
> From: Michael Kelley <mikelley@microsoft.com>
> Sent: Wednesday, September 15, 2021 12:22 PM
> To: Tianyu Lan <ltykernel@gmail.com>; KY Srinivasan <kys@microsoft.com>;

> > +				memset(vmap_pages, 0,
> > +				       sizeof(*vmap_pages) * vmap_page_index);
> > +				vmap_page_index =3D 0;
> > +
> > +				for (j =3D 0; j < i; j++)
> > +					__free_pages(pages[j], alloc_unit);
> > +
> > +				kfree(pages);
> > +				alloc_unit =3D 1;
>=20
> This is the case where a large enough contiguous physical memory chunk
> could not be found.  But rather than dropping all the way down to single
> pages, would it make sense to try something smaller, but not 1?  For
> example, cut the alloc_unit in half and try again.  But I'm not sure of
> all the implications.

I had the same question. But probably gradually decrementing uses too much
time?

>=20
> > +				goto retry;
> > +			}
> > +		}
> > +
> > +		pages[i] =3D page;
> > +		for (j =3D 0; j < alloc_unit; j++)
> > +			vmap_pages[vmap_page_index++] =3D page++;
> > +	}
> > +
> > +	vaddr =3D vmap(vmap_pages, vmap_page_index, VM_MAP, PAGE_KERNEL);
> > +	kfree(vmap_pages);
> > +
> > +	*pages_array =3D pages;
> > +	return vaddr;
> > +
> > +cleanup:
> > +	for (j =3D 0; j < i; j++)
> > +		__free_pages(pages[i], alloc_unit);
> > +
> > +	kfree(pages);
> > +	kfree(vmap_pages);
> > +	return NULL;
> > +}
> > +
> > +static void *netvsc_map_pages(struct page **pages, int count, int
> > +alloc_unit) {
> > +	int pg_count =3D count * alloc_unit;
> > +	struct page *page;
> > +	unsigned long *pfns;
> > +	int pfn_index =3D 0;
> > +	void *vaddr;
> > +	int i, j;
> > +
> > +	if (!pages)
> > +		return NULL;
> > +
> > +	pfns =3D kcalloc(pg_count, sizeof(*pfns), GFP_KERNEL);
> > +	if (!pfns)
> > +		return NULL;
> > +
> > +	for (i =3D 0; i < count; i++) {
> > +		page =3D pages[i];
> > +		if (!page) {
> > +			pr_warn("page is not available %d.\n", i);
> > +			return NULL;
> > +		}
> > +
> > +		for (j =3D 0; j < alloc_unit; j++) {
> > +			pfns[pfn_index++] =3D page_to_pfn(page++) +
> > +				(ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
> > +		}
> > +	}
> > +
> > +	vaddr =3D vmap_pfn(pfns, pg_count, PAGE_KERNEL_IO);
> > +	kfree(pfns);
> > +	return vaddr;
> > +}
> > +
>=20
> I think you are proposing this approach to allocating memory for the
> send and receive buffers so that you can avoid having two virtual
> mappings for the memory, per comments from Christop Hellwig.  But
> overall, the approach seems a bit complex and I wonder if it is worth it.
> If allocating large contiguous chunks of physical memory is successful,
> then there is some memory savings in that the data structures needed to
> keep track of the physical pages is smaller than the equivalent page
> tables might be.  But if you have to revert to allocating individual
> pages, then the memory savings is reduced.
>=20
> Ultimately, the list of actual PFNs has to be kept somewhere.  Another
> approach would be to do the reverse of what hv_map_memory() from the v4
> patch series does.  I.e., you could do virt_to_phys() on each virtual
> address that maps above VTOM, and subtract out the shared_gpa_boundary
> to get the
> list of actual PFNs that need to be freed.   This way you don't have two
> copies
> of the list of PFNs -- one with and one without the shared_gpa_boundary
> added.
> But it comes at the cost of additional code so that may not be a great
> idea.
>=20
> I think what you have here works, and I don't have a clearly better
> solution at the moment except perhaps to revert to the v4 solution and
> just have two virtual mappings.  I'll keep thinking about it.  Maybe
> Christop has other thoughts.
>=20
> >  static int netvsc_init_buf(struct hv_device *device,
> >  			   struct netvsc_device *net_device,
> >  			   const struct netvsc_device_info *device_info) @@ -
> 337,7 +462,7
> > @@ static int netvsc_init_buf(struct hv_device *device,
> >  	struct nvsp_1_message_send_receive_buffer_complete *resp;
> >  	struct net_device *ndev =3D hv_get_drvdata(device);
> >  	struct nvsp_message *init_packet;
> > -	unsigned int buf_size;
> > +	unsigned int buf_size, alloc_unit;
> >  	size_t map_words;
> >  	int i, ret =3D 0;
> >
> > @@ -350,7 +475,14 @@ static int netvsc_init_buf(struct hv_device
> *device,
> >  		buf_size =3D min_t(unsigned int, buf_size,
> >  				 NETVSC_RECEIVE_BUFFER_SIZE_LEGACY);
> >
> > -	net_device->recv_buf =3D vzalloc(buf_size);
> > +	if (hv_isolation_type_snp())
> > +		net_device->recv_buf =3D
> > +			netvsc_alloc_pages(&net_device->recv_pages,
> > +					   &net_device->recv_page_count,
> > +					   buf_size);
> > +	else
> > +		net_device->recv_buf =3D vzalloc(buf_size);
> > +
>=20
> I wonder if it is necessary to have two different code paths here.  The
> allocating and freeing of the send and receive buffers is not perf
> sensitive, and it seems like netvsc_alloc_pages() could be used
> regardless of whether SNP Isolation is in effect.  To my thinking, one
> code path is better than two code paths unless there's a compelling
> reason to have two.

I still prefer keeping the simple vzalloc for the non isolated VMs, because
simple code path usually means more robust.=20
I don't know how much time difference between the two, but in some cases=20
we really care about boot time?=20
Also in the multi vPort case for MANA, we potentially support hundreds of=20
vPorts, and there will be the same number of synthetic NICs associated with=
=20
them. So even small time difference in the initialization time may add up.

Thanks,
- Haiyang



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 17:04:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 17:04:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187842.336839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQYKO-0003tN-3W; Wed, 15 Sep 2021 17:04:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187842.336839; Wed, 15 Sep 2021 17:04:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQYKN-0003tG-WE; Wed, 15 Sep 2021 17:04:08 +0000
Received: by outflank-mailman (input) for mailman id 187842;
 Wed, 15 Sep 2021 17:04:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQYKM-0003t6-MY; Wed, 15 Sep 2021 17:04:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQYKM-0002fY-Fc; Wed, 15 Sep 2021 17:04:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQYKM-0006hK-5U; Wed, 15 Sep 2021 17:04:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQYKM-0005FH-4d; Wed, 15 Sep 2021 17:04:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dy+fqMyqWYNf9JrYMfJcxGE6GNWbo+7tYGhxlBW8cKU=; b=XTzQK0KH5KAYq5U2KDInYFBf98
	eBMO5OAO9R0hqJs7Ys19nAVIOHLJ3fN5hYrYnVETiqSrNPZ3Tr+NkR6m+WWnKOygdMEeq8CfEU6rD
	mCcbBcYCSgNUnKsLJLedRxoWntoDbZ1ZVhGfzkWq6rQoi3NUOPn3hkIjB09FWXRwabfQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164998-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 164998: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3e910b648b99393561e7c523756c1ea49a6c1305
X-Osstest-Versions-That:
    xen=6254920587c33bcc7ab884e6c9a11cfc0d5867ab
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 17:04:06 +0000

flight 164998 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164998/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3e910b648b99393561e7c523756c1ea49a6c1305
baseline version:
 xen                  6254920587c33bcc7ab884e6c9a11cfc0d5867ab

Last test of basis   164995  2021-09-15 10:00:25 Z    0 days
Testing same since   164998  2021-09-15 14:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6254920587..3e910b648b  3e910b648b99393561e7c523756c1ea49a6c1305 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 18:12:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 18:12:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187851.336852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQZOb-0002zb-8R; Wed, 15 Sep 2021 18:12:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187851.336852; Wed, 15 Sep 2021 18:12:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQZOb-0002zU-4z; Wed, 15 Sep 2021 18:12:33 +0000
Received: by outflank-mailman (input) for mailman id 187851;
 Wed, 15 Sep 2021 18:12:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQZOZ-0002zO-Jz
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 18:12:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b077c241-ddb1-4fed-be5b-8bb6d17cebda;
 Wed, 15 Sep 2021 18:12:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A6BB6611C4;
 Wed, 15 Sep 2021 18:12:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b077c241-ddb1-4fed-be5b-8bb6d17cebda
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631729549;
	bh=KH8oYWjZ0RdqIvN/DV8JM1AQyoP/T4f7uNJ4aoqeHoU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=IeDbRpFGjbo55IOSZd6M6V2va3x/MZQihLl/C6lmGRqme6FN1DIm9gI+mbq+69rEO
	 j0bn8RE8tMI9l5NkCxBGgBiriWraG+sFT/D9Sr7H9kyo012WN2kjLUWsfqUE1gEkja
	 nS+elKHLgsSNK4MmOv15ldxMqz28BeLj6b+/81de94idRF6Higw3a0BLhv31SnUsA9
	 beqv99wpeFJMsewjecNWm4fDsOlX02vGID6ICx4Qgxchv9EG37p0v91RUC80t3zRJm
	 7ZPd+f/HpJlBT48gKUtGJmVMp+z2YOgNyhI1IDYc847+0hsNr5oFA9NVoVsmUEUfmY
	 3Td8Spiml0A/w==
Date: Wed, 15 Sep 2021 11:12:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Penny Zheng <penny.zheng@arm.com>, Bertrand.Marquis@arm.com, 
    Wei.Chen@arm.com, xen-devel@lists.xenproject.org, julien@xen.org
Subject: Re: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
In-Reply-To: <d2e87c7a-08e0-3cca-fe4a-6278d6939591@suse.com>
Message-ID: <alpine.DEB.2.21.2109151106430.21985@sstabellini-ThinkPad-T480s>
References: <20210910025215.1671073-1-penny.zheng@arm.com> <20210910025215.1671073-6-penny.zheng@arm.com> <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com> <alpine.DEB.2.21.2109100916270.10523@sstabellini-ThinkPad-T480s>
 <d2e87c7a-08e0-3cca-fe4a-6278d6939591@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 Sep 2021, Jan Beulich wrote:
> On 10.09.2021 18:23, Stefano Stabellini wrote:
> > On Fri, 10 Sep 2021, Jan Beulich wrote:
> >> On 10.09.2021 04:52, Penny Zheng wrote:
> >>> In order to deal with the trouble of count-to-order conversion when page number
> >>> is not in a power-of-two, this commit re-define assign_pages for nr pages and
> >>> assign_page for original page with a single order.
> >>>
> >>> Backporting confusion could be helped by altering the order of assign_pages
> >>> parameters, such that the compiler would point out that adjustments at call
> >>> sites are needed.
> >>
> >> Thanks, this now takes care of my primary concern. However, I (now?) have
> >> another (I thought I would have mentioned this before):
> >>
> >>> --- a/xen/common/page_alloc.c
> >>> +++ b/xen/common/page_alloc.c
> >>> @@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
> >>>  
> >>>  
> >>>  int assign_pages(
> >>> -    struct domain *d,
> >>>      struct page_info *pg,
> >>> -    unsigned int order,
> >>> +    unsigned long nr,
> >>
> >> If this really is to be "unsigned long" (and not "unsigned int"), then...
> > 
> > Thanks for spotting this. I think it makes sense to use "unsigned int
> > nr" here.
> 
> I see you've made the change while committing, but the subsequent patch
> then would have needed adjustment as well: It's now silently truncating
> an "unsigned long" value to "unsigned int". It was the splitting that's
> now needed there _anyway_ that made me wonder whether the patch here
> really is worthwhile to have. But of course acquire_domstatic_pages()
> could for now also simply reject too large values ...

Yes. Are you thinking of a check like the following at the beginning of
acquire_domstatic_pages?

    if ( nr_mfns > UINT_MAX )
        return -EINVAL;

An alternative would be to change acquire_domstatic_pages to take an
unsigned int as nr_mfns parameter, but then it would just push the
problem up one level to allocate_static_memory, which is reading the
value from device tree so it is out of our control. So I think it is a
good idea to have an explicit check and acquire_domstatic_pages would be
a good place for it.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 19:10:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 19:10:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187860.336864 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQaIg-00014K-JQ; Wed, 15 Sep 2021 19:10:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187860.336864; Wed, 15 Sep 2021 19:10:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQaIg-00014D-GJ; Wed, 15 Sep 2021 19:10:30 +0000
Received: by outflank-mailman (input) for mailman id 187860;
 Wed, 15 Sep 2021 19:10:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hLCT=OF=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQaIe-000147-Vs
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 19:10:29 +0000
Received: from mail-ed1-x535.google.com (unknown [2a00:1450:4864:20::535])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a95f2cb1-a4b7-4fb3-9659-1770f807852d;
 Wed, 15 Sep 2021 19:10:26 +0000 (UTC)
Received: by mail-ed1-x535.google.com with SMTP id g21so7419784edw.4
 for <xen-devel@lists.xenproject.org>; Wed, 15 Sep 2021 12:10:26 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id 6sm357505ejx.82.2021.09.15.12.10.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 Sep 2021 12:10:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a95f2cb1-a4b7-4fb3-9659-1770f807852d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=pn3+UGoyhb1u5WaznGKX/AfbsmQRh2tuWoPupZP9LlE=;
        b=QXqRcBpum5I5tcHEbYiRv8KiSyhTbeI97RRfZN7KTW5AOckOTEvULgm2t5xZ3yqdQF
         557n3EYhABv3qCymgPY82JfELibX3Y4WctQ7ZpH/5MXt9j/jNUAtKFwKqlhZXCXeQGo2
         X6QG2rKkzoSEB5JJ6altLxKgL66e5kkZcOlBw8yFpWyBBjAlsbcRhmZEywhwOpPaUun8
         hXIsNafDZ7omqVjuncXe+yiIPCNWRpTVKgsSE+cEtiFjojXGUFcNRzNgtiPKyBf8WPSO
         98ARtNN+GCWyI38MMQqgnbqCADKb4+pFZuBKdiu9tay7yDVQDj95sid3IymtXU5W7N9o
         O6tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=pn3+UGoyhb1u5WaznGKX/AfbsmQRh2tuWoPupZP9LlE=;
        b=p+Kg4MhgsYU007z404UJ/Y7c2zrzwR8wxg0N0dJI1p1nfAAi2ZvlEDHnAgb0zD4+gb
         PTe4kr/RYvqDBPcPPTimfegwWZ2OgMle9WtZBrrqHSBQZ+YrbxhSJo4twS/AfivHo6Nx
         KUFU8YkW3rL7JXyg8D5P+IfTiFP22jx/P473lpKgaEsOX1rhCgKpyYOIyx6A2wubh93R
         l+4xzOz6GbUhQXeAXdX2tQU1l88m/bgT1Luul+pm0G+6WOA2eAxL/oFVbTa9vi4cmBUh
         8nn4tb2yPbGV+qCUsg4LunI6ugD2Z1DBBrDTE6tzB0uGxtiARp+9g+yY1OOkTLdKhTdA
         7XoQ==
X-Gm-Message-State: AOAM531th6Z9nUtnbsh8dBIeeH/kMnYjK2BSgRDujlGJRy5ag2ePgxhj
	hkF8AFJQAd2IheGVsFRMaB0=
X-Google-Smtp-Source: ABdhPJwWnGJ2240DgmQR2Gsn0ebhDl12vCSqKN5ziU/A+sxj8OoWz0es4t85oiqW1g2SkL6XE2Rojg==
X-Received: by 2002:a17:906:4f96:: with SMTP id o22mr1650327eju.169.1631733025347;
        Wed, 15 Sep 2021 12:10:25 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
Date: Wed, 15 Sep 2021 22:10:24 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 14.09.21 03:55, Stefano Stabellini wrote:

Hi Stefano

> On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The extended region (safe range) is a region of guest physical
>> address space which is unused and could be safely used to create
>> grant/foreign mappings instead of wasting real RAM pages from
>> the domain memory for establishing these mappings.
>>
>> The extended regions are chosen at the domain creation time and
>> advertised to it via "reg" property under hypervisor node in
>> the guest device-tree. As region 0 is reserved for grant table
>> space (always present), the indexes for extended regions are 1...N.
>> If extended regions could not be allocated for some reason,
>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>
>> Please note the following limitations:
>> - The extended region feature is only supported for 64-bit domain.
>> - The ACPI case is not covered.
>>
>> ***
>>
>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>> the algorithm to choose extended regions for it is different
>> in comparison with the algorithm for non-direct mapped DomU.
>> What is more, that extended regions should be chosen differently
>> whether IOMMU is enabled or not.
>>
>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>> holes found in host device-tree if otherwise. Make sure that
>> extended regions are 2MB-aligned and located within maximum possible
>> addressable physical memory range. The maximum number of extended
>> regions is 128.
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> Changes since RFC:
>>     - update patch description
>>     - drop uneeded "extended-region" DT property
>> ---
>>
>>   xen/arch/arm/domain_build.c | 226 +++++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 224 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 206038d..070ec27 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domain *d,
>>       return res;
>>   }
>>   
>> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
>> +{
>> +    struct meminfo *ext_regions = data;
>> +    paddr_t start, size;
>> +
>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>> +        return 0;
>> +
>> +    /* Both start and size of the extended region should be 2MB aligned */
>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>> +    if ( start > e )
>> +        return 0;
>> +
>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>> +    if ( !size )
>> +        return 0;
> Can't you align size as well?
>
>    size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);

I am sorry, I don't entirely get what you really meant here. We get both 
start and size 2MB-aligned by the calculations above
(when calculating an alignment, we need to make sure that "start_passed 
<= start_aligned && size_aligned <= size_passed").
If I add the proposing string after, I will reduce the already aligned 
size by 2MB.
If I replace the size calculation with the following, I will get the 
reduced size even if the passed region is initially 2MB-aligned, so 
doesn't need to be adjusted.
size = e - s + 1;
size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);

>
>> +    ext_regions->bank[ext_regions->nr_banks].start = start;
>> +    ext_regions->bank[ext_regions->nr_banks].size = size;
>> +    ext_regions->nr_banks ++;
>                              ^ no space

ok


>
>> +    return 0;
>> +}
>> +
>> +/*
>> + * The extended regions will be prevalidated by the memory hotplug path
>> + * in Linux which requires for any added address range to be within maximum
>> + * possible addressable physical memory range for which the linear mapping
>> + * could be created.
>> + * For 48-bit VA space size the maximum addressable range are:
>> + * 0x40000000 - 0x80003fffffff
> Please don't make Linux-specific comments in Xen code for interfaces
> that are supposed to be OS-agnostic.

You are right. I just wanted to describe where these magic numbers come 
from.
Someone might question why, for example, "0 ... max_gpaddr" can't be 
used. I will move
that Linux-specific comments to the commit message to keep some 
justification of these numbers.


>> + */
>> +#define EXT_REGION_START   0x40000000ULL
>> +#define EXT_REGION_END     0x80003fffffffULL
>> +
>> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
>> +                                          struct meminfo *ext_regions)
>> +{
>> +    const struct meminfo *assign_mem = &kinfo->mem;
>> +    struct rangeset *unalloc_mem;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>> +
>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>> +
>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>> +    if ( !unalloc_mem )
>> +        return -ENOMEM;
>> +
>> +    /* Start with all available RAM */
>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.mem.bank[i].start;
>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
> Is the -1 needed? Isn't it going to screw up the size calculation later?
I thought, it was needed. The calculation seems correct.
For example, in my setup when IOMMU is disabled for Dom0 ("unallocated 
to Dom0 RAM"):

--- All available RAM and reserved memory found in DT:

(XEN) Initrd 0000000084000040-0000000085effc48
(XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
(XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
(XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
(XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3
(XEN)
(XEN) MODULE[0]: 0000000078080000 - 00000000781d74c8 Xen
(XEN) MODULE[1]: 0000000057fe7000 - 0000000057ffd080 Device Tree
(XEN) MODULE[2]: 0000000084000040 - 0000000085effc48 Ramdisk
(XEN) MODULE[3]: 000000008a000000 - 000000008c000000 Kernel
(XEN) MODULE[4]: 000000008c000000 - 000000008c010000 XSM
(XEN)  RESVD[0]: 0000000084000040 - 0000000085effc48
(XEN)  RESVD[1]: 0000000054000000 - 0000000056ffffff  <--- Reserved memory

--- Dom0 RAM:

(XEN) Allocating 1:1 mappings totalling 256MB for dom0:
(XEN) BANK[0] 0x00000060000000-0x00000070000000 (256MB)

--- Dom0 grant table range:

(XEN) Grant table range: 0x00000078080000-0x000000780c0000

--- Calculated extended regions printed in make_hypervisor_node():

printk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n", i, start, start + 
size);

(XEN) Extended region 0: 0x48000000->0x54000000
(XEN) Extended region 1: 0x57000000->0x60000000
(XEN) Extended region 2: 0x70000000->0x78000000
(XEN) Extended region 3: 0x78200000->0xc0000000
(XEN) Extended region 4: 0x500000000->0x580000000
(XEN) Extended region 5: 0x600000000->0x680000000
(XEN) Extended region 6: 0x700000000->0x780000000

--- Resulted hypervisor node in Dom0 DT:

hypervisor {
         interrupts = <0x01 0x00 0xf08>;
         interrupt-parent = <0x19>;
         compatible = "xen,xen-4.16\0xen,xen";
         reg = <0x00 0x78080000 0x00 0x40000 0x00 0x48000000 0x00 
0xc000000 0x00 0x57000000 0x00 0x9000000 0x00 0x70000000 0x00 0x8000000 
0x00 0x78200000 0x00 0x47e00000 0x05 0x00 0x00 0x80000000 0x06 0x00 0x00 
0x80000000 0x07 0x00 0x00 0x80000000>;
};

Where index 0 corresponds to the grant table region and indexes 1...N 
correspond to the extended regions.


>> +        res = rangeset_add_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove RAM assigned to Dom0 */
>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>> +    {
>> +        start = assign_mem->bank[i].start;
>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove reserved-memory regions */
>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.reserved_mem.bank[i].start;
>> +        end = bootinfo.reserved_mem.bank[i].start +
>> +            bootinfo.reserved_mem.bank[i].size - 1;
>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove grant table region */
>> +    start = kinfo->gnttab_start;
>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>> +                                 add_ext_regions, ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(unalloc_mem);
>> +
>> +    return res;
>> +}
>> +
>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>> +                                    struct meminfo *ext_regions)
>> +{
>> +    struct dt_device_node *np;
>> +    struct rangeset *mem_holes;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>> +
>> +    dt_dprintk("Find memory holes for extended regions\n");
>> +
>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>> +    if ( !mem_holes )
>> +        return -ENOMEM;
>> +
>> +    /* Start with maximum possible addressable physical memory range */
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_add_range(mem_holes, start, end);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
>> +    dt_for_each_device_node( dt_host, np )
> Don't you need something like device_tree_for_each_node ?
> dt_for_each_device_node won't go down any deeper in the tree?

Thank you for pointing this out, I will investigate and update the patch.


>
> Alternatively, maybe we could simply record the highest possible address
> of any memory/device/anything as we scan the device tree with
> handle_node. Then we can use that as the starting point here.
I also don't like the idea to scan the DT much, but I failed to find an 
effective solution how to avoid that.
Yes, we can record the highest possible address, but I am afraid, I 
didn't entirely get a suggestion. Is the suggestion to provide a single 
region
starting from highest possible address + 1 and up to the EXT_REGION_END 
suitably aligned? Could you please clarify?


> So that we
> don't need to scan the device tree twice, and also we don't need my
> suggestion below to remove 1GB-aligned 1GB-multiple regions.
I provided some thoughts regarding this below.


>
>
>> +    {
>> +        unsigned int naddr;
>> +        u64 addr, size;
>> +
>> +        naddr = dt_number_of_address(np);
>> +
>> +        for ( i = 0; i < naddr; i++ )
>> +        {
>> +            res = dt_device_get_address(np, i, &addr, &size);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
>> +                       i, dt_node_full_name(np));
> It might be possible for a device not to have a range if it doesn't have
> any MMIO regions, right? For instance, certain ARM timer nodes. I would
> not print any errors and continue.
I though, if device didn't have a range, then this loop wouldn't be 
executed at all as dt_number_of_address would return 0.
I will double check.


>
>
>> +                goto out;
>> +            }
>> +
>> +            start = addr & PAGE_MASK;
>> +            end = PAGE_ALIGN(addr + size) - 1;
>> +            res = rangeset_remove_range(mem_holes, start, end);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                       start, end);
>> +                goto out;
>> +            }
>> +        }
>> +    }
> As is, it will result in a myriad of small ranges which is unuseful and
> slow to parse. I suggest to simplify it by removing a larger region than
> strictly necessary. For instance, you could remove a 1GB-aligned and
> 1GB-multiple region for each range. That way, you are going to get fewer
> large free ranges instance of many small ones which we don't need.

I agree with you that a lot of small ranges increase the bookkeeping in 
Dom0 and there is also a theoretical (?) possibility
that small ranges occupy all space we provide for extended regions 
(NR_MEM_BANKS)...
But, let's consider my setup as an example again, but when the IOMMU is 
enabled for Dom0 ("holes found in DT").

--- The RAM configuration is the same:

(XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
(XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
(XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
(XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3

--- There are a lot of various platform devices with reg property 
described in DT, I will probably not post all IO ranges here, just say 
that mostly all of them to be mapped at 0xE0000000-0xFFFFFFFF.

--- As we only pick up ranges with size >= 2MB, the calculated extended 
regions are (based on 40-bit IPA):

(XEN) Extended region 0: 0x40000000->0x47e00000
(XEN) Extended region 1: 0xc0000000->0xe6000000
(XEN) Extended region 2: 0xe7000000->0xe7200000
(XEN) Extended region 3: 0xe7400000->0xe7600000
(XEN) Extended region 4: 0xe7800000->0xec000000
(XEN) Extended region 5: 0xec200000->0xec400000
(XEN) Extended region 6: 0xec800000->0xee000000
(XEN) Extended region 7: 0xee600000->0xee800000
(XEN) Extended region 8: 0xeea00000->0xf1000000
(XEN) Extended region 9: 0xf1200000->0xfd000000
(XEN) Extended region 10: 0xfd200000->0xfd800000
(XEN) Extended region 11: 0xfda00000->0xfe000000
(XEN) Extended region 12: 0xfe200000->0xfe600000
(XEN) Extended region 13: 0xfec00000->0xff800000
(XEN) Extended region 14: 0x100000000->0x500000000
(XEN) Extended region 15: 0x580000000->0x600000000
(XEN) Extended region 16: 0x680000000->0x700000000
(XEN) Extended region 17: 0x780000000->0x10000000000

So, if I *correctly* understood your idea about removing 1GB-aligned 
1GB-multiple region for each range we would get the following:

(XEN) Extended region 0: 0x100000000->0x500000000
(XEN) Extended region 1: 0x580000000->0x600000000
(XEN) Extended region 2: 0x680000000->0x700000000
(XEN) Extended region 3: 0x780000000->0x10000000000

As you can see there are no extended regions below 4GB at all. I assume, 
it would be good to provide them for 1:1 mapped Dom0 (for 32-bit DMA 
devices?)
What else worries me is that IPA size could be 36 or even 32. So, I am 
afraid, we might even fail to find extended regions above 4GB.


I think, if 2MB is considered small enough to bother with, probably we 
should go with something in between (16MB, 32MB, 64MB).
For example, we can take into the account ranges with size >= 16MB:

(XEN) Extended region 0: 0x40000000->0x47e00000
(XEN) Extended region 1: 0xc0000000->0xe6000000
(XEN) Extended region 2: 0xe7800000->0xec000000
(XEN) Extended region 3: 0xec800000->0xee000000
(XEN) Extended region 4: 0xeea00000->0xf1000000
(XEN) Extended region 5: 0xf1200000->0xfd000000
(XEN) Extended region 6: 0x100000000->0x500000000
(XEN) Extended region 7: 0x580000000->0x600000000
(XEN) Extended region 8: 0x680000000->0x700000000
(XEN) Extended region 9: 0x780000000->0x10000000000

Any thoughts?


>
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_report_ranges(mem_holes, start, end,
>> +                                 add_ext_regions,  ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(mem_holes);
>> +
>> +    return res;
>> +}
>> +
>>   static int __init make_hypervisor_node(struct domain *d,
>>                                          const struct kernel_info *kinfo,
>>                                          int addrcells, int sizecells)
>> @@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain *d,
>>       const char compat[] =
>>           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>>           "xen,xen";
>> -    __be32 reg[4];
>> +    __be32 reg[(NR_MEM_BANKS + 1) * 4];
> If you are xzalloc'ing struct meminfo then you might as well xzalloc reg
> too. Or keep both on the stack with a lower NR_MEM_BANKS.
sounds reasonable, I will probably xzalloc reg as well.

>
>
>>       gic_interrupt_t intr;
>>       __be32 *cells;
>>       int res;
>>       void *fdt = kinfo->fdt;
>> +    struct meminfo *ext_regions;
>> +    unsigned int i;
>>   
>>       dt_dprintk("Create hypervisor node\n");
>>   
>> @@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct domain *d,
>>       if ( res )
>>           return res;
>>   
>> +    ext_regions = xzalloc(struct meminfo);
>> +    if ( !ext_regions )
>> +        return -ENOMEM;
>> +
>> +    if ( is_32bit_domain(d) )
>> +        printk(XENLOG_WARNING "The extended region is only supported for 64-bit guest\n");
> This is going to add an unconditional warning to all 32bit boots. I
> would skip it entirely or only keep it as XENLOG_DEBUG.

agree, I will probably convert to XENLOG_DEBUG.


>
>
>> +    else
>> +    {
>> +        if ( !is_iommu_enabled(d) )
>> +            res = find_unallocated_memory(kinfo, ext_regions);
>> +        else
>> +            res = find_memory_holes(kinfo, ext_regions);
>> +
>> +        if ( res )
>> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
>> +    }
>> +
>>       /* reg 0 is grant table space */
>>       cells = &reg[0];
>>       dt_child_set_range(&cells, addrcells, sizecells,
>>                          kinfo->gnttab_start, kinfo->gnttab_size);
>> +    /* reg 1...N are extended regions */
>> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
>> +    {
>> +        u64 start = ext_regions->bank[i].start;
>> +        u64 size = ext_regions->bank[i].size;
>> +
>> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
>> +                   i, start, start + size);
>> +
>> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
>> +    }
>> +    xfree(ext_regions);
>> +
>>       res = fdt_property(fdt, "reg", reg,
>> -                       dt_cells_to_size(addrcells + sizecells));
>> +                       dt_cells_to_size(addrcells + sizecells) * (i + 1));
>>       if ( res )
>>           return res;
>>   
>> -- 
>> 2.7.4
>>

Thank you.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 20:09:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 20:09:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187869.336875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbDj-0006x6-8X; Wed, 15 Sep 2021 20:09:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187869.336875; Wed, 15 Sep 2021 20:09:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbDj-0006wz-5X; Wed, 15 Sep 2021 20:09:27 +0000
Received: by outflank-mailman (input) for mailman id 187869;
 Wed, 15 Sep 2021 20:09:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQbDi-0006wt-40
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 20:09:26 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d2624ce2-1660-11ec-b578-12813bfff9fa;
 Wed, 15 Sep 2021 20:09:24 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B58156105A;
 Wed, 15 Sep 2021 20:09:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2624ce2-1660-11ec-b578-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631736564;
	bh=yLQ2czOp5XpawjTjhFiCOrMTLRWxVYn/+crVhqG7g7c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=k45CSMPjdWR3hv8d2GRejhc8V3WuEBFbP/iL5GwGjBkeRQ2ASMkUmYyHTM6ZnSpaC
	 HB4ZWZ9M6UFsgifI0ISvpyTlKvdJ5KPY0qnpqQXYt0Bos92Si2AAfrAevxsh74Hu8O
	 wpYx/rTM1N1EZjvWTVSEOYFlqm3k6YdMiw6lVBhyRDLwCH9rYPi+C/M1iimBoTjysD
	 Vq31vhc5oDD/iN+BnkE753pITzVGJM6YZMzr//kY2n7aYTxcz/dvYQSspTTc58MCOv
	 WfMuV8K2+PegbeEun3wCOjA4To9+itdzPabJJq/sE86dcqYcRzYAPRvyhlEWEISaGE
	 wdBft+gDxNmsg==
Date: Wed, 15 Sep 2021 13:09:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Rahul Singh <rahul.singh@arm.com>, 
    Julien Grall <julien.grall.oss@gmail.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
In-Reply-To: <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
Message-ID: <alpine.DEB.2.21.2109151246450.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com> <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com> <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com> <684b3534-40eb-add7-f46e-c6258904757b@xen.org> <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s> <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com> <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com> <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s> <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-344476147-1631735844=:21985"
Content-ID: <alpine.DEB.2.21.2109151301490.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-344476147-1631735844=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109151301491.21985@sstabellini-ThinkPad-T480s>

On Wed, 15 Sep 2021, Oleksandr Andrushchenko wrote:
> On 15.09.21 03:36, Stefano Stabellini wrote:
> > On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
> >> With the patch above I have the following log in Domain-0:
> >>
> >> generic-armv8-xt-dom0 login: root
> >> root@generic-armv8-xt-dom0:~# (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input)
> >> (XEN) ==== PCI devices ====
> >> (XEN) ==== segment 0000 ====
> >> (XEN) 0000:03:00.0 - d0 - node -1
> >> (XEN) 0000:02:02.0 - d0 - node -1
> >> (XEN) 0000:02:01.0 - d0 - node -1
> >> (XEN) 0000:02:00.0 - d0 - node -1
> >> (XEN) 0000:01:00.0 - d0 - node -1
> >> (XEN) 0000:00:00.0 - d0 - node -1
> >> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> >>
> >> root@generic-armv8-xt-dom0:~# modprobe e1000e
> >> [   46.104729] e1000e: Intel(R) PRO/1000 Network Driver
> >> [   46.105479] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
> >> [   46.107297] e1000e 0000:03:00.0: enabling device (0000 -> 0002)
> >> (XEN) map [e0000, e001f] -> 0xe0000 for d0
> >> (XEN) map [e0020, e003f] -> 0xe0020 for d0
> >> [   46.178513] e1000e 0000:03:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
> >> [   46.189668] pci_msi_setup_msi_irqs
> >> [   46.191016] nwl_compose_msi_msg msg at fe440000
> >> (XEN) traps.c:2014:d0v0 HSR=0x00000093810047 pc=0xffff8000104b4b00 gva=0xffff800010fa5000 gpa=0x000000e0040000
> >> [   46.200455] Unhandled fault at 0xffff800010fa5000
> >>
> >> [snip]
> >>
> >> [   46.233079] Call trace:
> >> [   46.233559]  __pci_write_msi_msg+0x70/0x180
> >> [   46.234149]  pci_msi_domain_write_msg+0x28/0x30
> >> [   46.234869]  msi_domain_activate+0x5c/0x88
> >>
> >>   From the above you can see that BARs are mapped for Domain-0 now
> >>
> >> only when an assigned PCI device gets enabled in Domain-0.
> >>
> >> Another thing to note is that we crash on MSI-X access as BARs are mapped
> >>
> >> to the domain while enabling memory decoding in the COMMAND register,
> >>
> >> but MSI-X are handled differently, e.g. we have MSI-X holes in the mappings.
> >>
> >> This is because before this change the whole PCI aperture was mapped into
> >>
> >> Domain-0 and it is not. Thus, MSI-X holes are left unmapped now and there
> >>
> >> was no need to do so, e.g. they were always mapped into Domain-0 and
> >>
> >> emulated for guests.
> >>
> >> Please note that one cannot use xl pci-attach in this case to attach the PCI device
> >>
> >> in question to Domain-0 as (please see the log) that device is already attached.
> >>
> >> Neither it can be detached and re-attached. So, without mapping MSI-X holes for
> >>
> >> Domain-0 the device becomes unusable in Domain-0. At the same time the device
> >>
> >> can be successfully passed to DomU.
> >>
> >>
> >> Julien, Stefano! Please let me know how can we proceed with this.
> > What was the plan for MSI-X in Dom0?
> It just worked because we mapped everything
> >
> > Given that Dom0 interacts with a virtual-ITS and virtual-GIC rather than
> > a physical-ITS and physical-GIC, I imagine that it wasn't correct for
> > Dom0 to write to the real MSI-X table directly?
> >
> > Shouldn't Dom0 get emulated MSI-X tables like any DomU?
> >
> > Otherwise, if Dom0 is expected to have the real MSI-X tables mapped, then
> > I would suggest to map them at the same time as the BARs. But I am
> > thinking that Dom0 should get emulated MSI-X tables, not physical MSI-X
> > tables.
> 
> Yes, it seems more than reasonable to enable emulation for Domain-0
> 
> as well. Other than that, Stefano, do you think we are good to go with
> 
> the changes I did in order to unmap everything for Domain-0?


It might be better to resend the series with the patch in it, because it
is difficult to review the patch like this. Nonetheless I tried, but I
might have missed something.

Previously the whole PCIe bridge aperture was mapped to Dom0, and
it was done by map_range_to_domain, is that correct?

Now this patch, to avoid mapping the entire aperture to Dom0, is
skipping any operations for PCIe devices in map_range_to_domain by
setting need_mapping = false.

The idea is that instead, we'll only map things when needed and not the
whole aperture. However, looking at the changes to
pci_host_bridge_mappings (formerly known as
pci_host_bridge_need_p2m_mapping), it is still going through the full
list of address ranges of the PCIe bridge and map each range one by one
using map_range_to_domain. Also, pci_host_bridge_mappings is still
called unconditionally at boot for Dom0.

So I am missing the part where the aperture is actually *not* mapped to
Dom0. What's the difference between the loop in
pci_host_bridge_mappings:

  for ( i = 0; i < dt_number_of_address(dev); i++ )
    map_range_to_domain...

and the previous code in map_range_to_domain? I think I am missing
something but as mentioned it is difficult to review the patch like this
out of order.

Also, and this is minor, even if currently unused, it might be good to
keep a length parameter to pci_host_bridge_need_p2m_mapping.
--8323329-344476147-1631735844=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 20:19:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 20:19:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187878.336885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbNQ-000063-7L; Wed, 15 Sep 2021 20:19:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187878.336885; Wed, 15 Sep 2021 20:19:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbNQ-00005v-4I; Wed, 15 Sep 2021 20:19:28 +0000
Received: by outflank-mailman (input) for mailman id 187878;
 Wed, 15 Sep 2021 20:19:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQbNO-00005k-Oj
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 20:19:26 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a984686d-afd3-4eac-9718-63a830fc61cd;
 Wed, 15 Sep 2021 20:19:25 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 728D760E54;
 Wed, 15 Sep 2021 20:19:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a984686d-afd3-4eac-9718-63a830fc61cd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631737164;
	bh=iWfR+zpCXMOPRdW7D7sOhNO3mxAgwU5oc04Siu35BA4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Tgoa/QURVp+MAHy5GI/5U8Duszv1wB3UYpWR1chwDGRkZ0qWSkJQKk9lt29N5vXMv
	 JoEPhE7O4Klc3BZqtcT5dG/QGPpVO5rRnRN76aArJptjNkishPJm0MxRiE8wpdbjKl
	 hlM992GR1wXw7TK45gac1H7IdvEGPA6ASTLEQIu2uuz3N0aupHQoN4OoVL/O1e5KVs
	 DucxOqBKkx0HsqitH9M4lMqvWFdN0dFYzRj4zx4/+99NDwj8SUdh8Sfd6baF39BqXZ
	 ie04oQJwqd11OU3pvzqyckV/dEBvCqi9hPD2r/D6XjFv8UCEKpQwSn1g6R1HK5ycAu
	 JBFjjXIXgvHvA==
Date: Wed, 15 Sep 2021 13:19:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>, 
    Rahul Singh <rahul.singh@arm.com>, 
    Julien Grall <julien.grall.oss@gmail.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
In-Reply-To: <alpine.DEB.2.21.2109151246450.21985@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109151311480.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com> <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com> <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com> <684b3534-40eb-add7-f46e-c6258904757b@xen.org> <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s> <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com> <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com> <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s> <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com> <alpine.DEB.2.21.2109151246450.21985@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-269449723-1631737164=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-269449723-1631737164=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 15 Sep 2021, Stefano Stabellini wrote:
> On Wed, 15 Sep 2021, Oleksandr Andrushchenko wrote:
> > On 15.09.21 03:36, Stefano Stabellini wrote:
> > > On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
> > >> With the patch above I have the following log in Domain-0:
> > >>
> > >> generic-armv8-xt-dom0 login: root
> > >> root@generic-armv8-xt-dom0:~# (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input)
> > >> (XEN) ==== PCI devices ====
> > >> (XEN) ==== segment 0000 ====
> > >> (XEN) 0000:03:00.0 - d0 - node -1
> > >> (XEN) 0000:02:02.0 - d0 - node -1
> > >> (XEN) 0000:02:01.0 - d0 - node -1
> > >> (XEN) 0000:02:00.0 - d0 - node -1
> > >> (XEN) 0000:01:00.0 - d0 - node -1
> > >> (XEN) 0000:00:00.0 - d0 - node -1
> > >> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> > >>
> > >> root@generic-armv8-xt-dom0:~# modprobe e1000e
> > >> [   46.104729] e1000e: Intel(R) PRO/1000 Network Driver
> > >> [   46.105479] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
> > >> [   46.107297] e1000e 0000:03:00.0: enabling device (0000 -> 0002)
> > >> (XEN) map [e0000, e001f] -> 0xe0000 for d0
> > >> (XEN) map [e0020, e003f] -> 0xe0020 for d0
> > >> [   46.178513] e1000e 0000:03:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
> > >> [   46.189668] pci_msi_setup_msi_irqs
> > >> [   46.191016] nwl_compose_msi_msg msg at fe440000
> > >> (XEN) traps.c:2014:d0v0 HSR=0x00000093810047 pc=0xffff8000104b4b00 gva=0xffff800010fa5000 gpa=0x000000e0040000
> > >> [   46.200455] Unhandled fault at 0xffff800010fa5000
> > >>
> > >> [snip]
> > >>
> > >> [   46.233079] Call trace:
> > >> [   46.233559]  __pci_write_msi_msg+0x70/0x180
> > >> [   46.234149]  pci_msi_domain_write_msg+0x28/0x30
> > >> [   46.234869]  msi_domain_activate+0x5c/0x88
> > >>
> > >>   From the above you can see that BARs are mapped for Domain-0 now
> > >>
> > >> only when an assigned PCI device gets enabled in Domain-0.
> > >>
> > >> Another thing to note is that we crash on MSI-X access as BARs are mapped
> > >>
> > >> to the domain while enabling memory decoding in the COMMAND register,
> > >>
> > >> but MSI-X are handled differently, e.g. we have MSI-X holes in the mappings.
> > >>
> > >> This is because before this change the whole PCI aperture was mapped into
> > >>
> > >> Domain-0 and it is not. Thus, MSI-X holes are left unmapped now and there
> > >>
> > >> was no need to do so, e.g. they were always mapped into Domain-0 and
> > >>
> > >> emulated for guests.
> > >>
> > >> Please note that one cannot use xl pci-attach in this case to attach the PCI device
> > >>
> > >> in question to Domain-0 as (please see the log) that device is already attached.
> > >>
> > >> Neither it can be detached and re-attached. So, without mapping MSI-X holes for
> > >>
> > >> Domain-0 the device becomes unusable in Domain-0. At the same time the device
> > >>
> > >> can be successfully passed to DomU.
> > >>
> > >>
> > >> Julien, Stefano! Please let me know how can we proceed with this.
> > > What was the plan for MSI-X in Dom0?
> > It just worked because we mapped everything
> > >
> > > Given that Dom0 interacts with a virtual-ITS and virtual-GIC rather than
> > > a physical-ITS and physical-GIC, I imagine that it wasn't correct for
> > > Dom0 to write to the real MSI-X table directly?
> > >
> > > Shouldn't Dom0 get emulated MSI-X tables like any DomU?
> > >
> > > Otherwise, if Dom0 is expected to have the real MSI-X tables mapped, then
> > > I would suggest to map them at the same time as the BARs. But I am
> > > thinking that Dom0 should get emulated MSI-X tables, not physical MSI-X
> > > tables.
> > 
> > Yes, it seems more than reasonable to enable emulation for Domain-0
> > 
> > as well. Other than that, Stefano, do you think we are good to go with
> > 
> > the changes I did in order to unmap everything for Domain-0?
> 
> 
> It might be better to resend the series with the patch in it, because it
> is difficult to review the patch like this. Nonetheless I tried, but I
> might have missed something.
> 
> Previously the whole PCIe bridge aperture was mapped to Dom0, and
> it was done by map_range_to_domain, is that correct?
> 
> Now this patch, to avoid mapping the entire aperture to Dom0, is
> skipping any operations for PCIe devices in map_range_to_domain by
> setting need_mapping = false.
> 
> The idea is that instead, we'll only map things when needed and not the
> whole aperture. However, looking at the changes to
> pci_host_bridge_mappings (formerly known as
> pci_host_bridge_need_p2m_mapping), it is still going through the full
> list of address ranges of the PCIe bridge and map each range one by one
> using map_range_to_domain. Also, pci_host_bridge_mappings is still
> called unconditionally at boot for Dom0.
> 
> So I am missing the part where the aperture is actually *not* mapped to
> Dom0. What's the difference between the loop in
> pci_host_bridge_mappings:
> 
>   for ( i = 0; i < dt_number_of_address(dev); i++ )
>     map_range_to_domain...
> 
> and the previous code in map_range_to_domain? I think I am missing
> something but as mentioned it is difficult to review the patch like this
> out of order.
> 
> Also, and this is minor, even if currently unused, it might be good to
> keep a length parameter to pci_host_bridge_need_p2m_mapping.

It looks like the filtering is done based on:

return cfg->phys_addr != addr

in pci_ecam_need_p2m_mapping that is expected to filter out the address
ranges we don't want to map because it comes from
xen/arch/arm/pci/pci-host-common.c:gen_pci_init:

    /* Parse our PCI ecam register address*/
    err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size);
    if ( err )
        goto err_exit;

In pci_host_bridge_mappings, instead of parsing device tree again, can't
we just fetch the address and length we need to map straight from
bridge->sysdata->phys_addr/size ?

At the point when pci_host_bridge_mappings is called in your new patch,
we have already initialized all the PCIe-related data structures. It
seems a bit useless to have to go via device tree again.
--8323329-269449723-1631737164=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 20:31:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 20:31:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187887.336897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbZJ-0002QT-B0; Wed, 15 Sep 2021 20:31:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187887.336897; Wed, 15 Sep 2021 20:31:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbZJ-0002QM-7Y; Wed, 15 Sep 2021 20:31:45 +0000
Received: by outflank-mailman (input) for mailman id 187887;
 Wed, 15 Sep 2021 20:31:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQbZH-0002QC-Ie; Wed, 15 Sep 2021 20:31:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQbZH-0006HG-ER; Wed, 15 Sep 2021 20:31:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQbZH-0000hS-4y; Wed, 15 Sep 2021 20:31:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQbZH-0001HC-4U; Wed, 15 Sep 2021 20:31:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7aUWFM0B26awtCM1/QraNrZvbNxRnnsrlpQbThyU8sU=; b=twBZHM1dtt8ho4680Do3oes5YR
	SQZeNxh+OfWPnlFGULOoT6vj3gaKBlM4ogoSWsz+TzKu87QGoftQmalZtlMohEbexVHG3L2MwYLnu
	ngNXlEWUKSJ4EQhMD3ecO521bYG10Zj0wXnf+Dr34Xdk1fWa/mY7L5qKOGIwW42yjYd8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164992-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 164992: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=831aaf24967a49d7750090b9dcfd6bf356f16529
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 20:31:43 +0000

flight 164992 qemu-mainline real [real]
flight 164999 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164992/
http://logs.test-lab.xenproject.org/osstest/logs/164999/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                831aaf24967a49d7750090b9dcfd6bf356f16529
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    4 days
Failing since        164967  2021-09-13 13:06:52 Z    2 days    5 attempts
Testing same since   164992  2021-09-15 07:59:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Lara Lazier <laramglazier@gmail.com>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1178 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 20:33:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 20:33:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187897.336910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbb3-00037H-U9; Wed, 15 Sep 2021 20:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187897.336910; Wed, 15 Sep 2021 20:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbb3-00037A-RG; Wed, 15 Sep 2021 20:33:33 +0000
Received: by outflank-mailman (input) for mailman id 187897;
 Wed, 15 Sep 2021 20:33:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQbb2-00036F-DL
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 20:33:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 30875760-1664-11ec-b57e-12813bfff9fa;
 Wed, 15 Sep 2021 20:33:31 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2465961101;
 Wed, 15 Sep 2021 20:33:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30875760-1664-11ec-b57e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631738010;
	bh=DpsoCGwx9doLkCvTqKb9xgdJte7ZqWArXatW0OT7fMQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=P9v51plvOBKNC6AS+fA843HA/eCsoUqzDgMkb30BlNjOCnw7EvAmHOg+NF1eZYKEe
	 d8sSkGsgWPZSqp+JF6uCWB4o5Z5QNewylZ+WlYch2ZGCGKpSykPnswt0XN/mgfeheG
	 1J1EsItXZyoDHjWC+KirRbgPknQe8X+KqfEDQYPXkhui1sxBFYlBR/lfv8PvzR+f/Q
	 En9VB+/tvqURWbgiHOtDiu9fitMIh5bDx1TEZSw1cz+M2uyIa4zVXuMbqZtme+hTtd
	 W9NrmnYX0H5WiVBM0HqqfCYCaKTHebdDQFJ12Blxra8G/sy36heDW7x3FXS4q3M9DG
	 UNmz4w2fehb2Q==
Date: Wed, 15 Sep 2021 13:33:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for hardware
 domain
In-Reply-To: <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
Message-ID: <alpine.DEB.2.21.2109151321030.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-10-andr2000@gmail.com> <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s> <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com> <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
 <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-579779311-1631738010=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-579779311-1631738010=:21985
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 Sep 2021, Rahul Singh wrote:
> Hi Oleksandr, Stefano,
> 
> > On 15 Sep 2021, at 6:30 am, Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com> wrote:
> > 
> > Hi, Rahul!
> > 
> > On 14.09.21 17:24, Oleksandr Andrushchenko wrote:
> >> 
> >> }
> >>>>   +static int pci_ecam_register_mmio_handler(struct domain *d,
> >>>> +                                          struct pci_host_bridge *bridge,
> >>>> +                                          const struct mmio_handler_ops *ops)
> >>>> +{
> >>>> +    struct pci_config_window *cfg = bridge->sysdata;
> >>>> +
> >>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
> >>>> +    return 0;
> >>>> +}
> >>> Given that struct pci_config_window is generic (it is not specific to
> >>> one bridge), I wonder if we even need the .register_mmio_handler
> >>> callback here.
> >>> 
> >>> In fact,pci_host_bridge->sysdata doesn't even need to be a void*, it
> >>> could be a struct pci_config_window*, right?
> >> 
> >> Rahul, this actually may change your series.
> >> 
> >> Do you think we can do that?
> >> 
> > This is the only change requested that left unanswered by now.
> > 
> > Will it be possible that you change the API accordingly, so I can
> > 
> > implement as Stefano suggests?
> 
> We need pci_host_bridge->sysdata as void* in case we need to implement the new non-ecam PCI controller in XEN.
> Please have a look once in Linux code[1] how bridge->sysdata will be used. struct pci_config_window is used only for 
> ecam supported host controller. Different PCI host controller will have different PCI interface to access the PCI controller.
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-rcar-host.c#n309
> 
> I think we need bridge->sysdata in future to implement the new PCI controller.

In my opinion the pci_config_window is too important of an information
to be left inside an opaque pointer, especially when the info under
pci_config_window is both critical and vendor-neutral.

My preference would be something like this:


diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 9c28a4bdc4..c80d846da3 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -55,7 +55,6 @@ struct pci_config_window {
     uint8_t         busn_start;
     uint8_t         busn_end;
     void __iomem    *win;
-    const struct    pci_ecam_ops *ops;
 };
 
 /*
@@ -68,7 +67,8 @@ struct pci_host_bridge {
     uint16_t segment;                /* Segment number */
     u8 bus_start;                    /* Bus start of this bridge. */
     u8 bus_end;                      /* Bus end of this bridge. */
-    void *sysdata;                   /* Pointer to the config space window*/
+    struct pci_config_window* cfg;   /* Pointer to the bridge config window */
+    void *sysdata;                   /* Pointer to bridge private data */
     const struct pci_ops *ops;
 };


As a reference the attached patch builds. However, I had to remove const
where struct pci_ecam_ops *ops is used.
--8323329-579779311-1631738010=:21985
Content-Type: text/x-diff; name=cfg.patch
Content-Transfer-Encoding: BASE64
Content-ID: <alpine.DEB.2.21.2109151333290.21985@sstabellini-ThinkPad-T480s>
Content-Description: 
Content-Disposition: attachment; filename=cfg.patch

ZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9LY29uZmlnIGIveGVuL2FyY2gv
YXJtL0tjb25maWcNCmluZGV4IGVjZmE2ODIyZTQuLmY5ZDU3Y2EwZmEgMTAw
NjQ0DQotLS0gYS94ZW4vYXJjaC9hcm0vS2NvbmZpZw0KKysrIGIveGVuL2Fy
Y2gvYXJtL0tjb25maWcNCkBAIC03LDYgKzcsNyBAQCBjb25maWcgQVJNXzY0
DQogCWRlcGVuZHMgb24gIUFSTV8zMg0KIAlzZWxlY3QgNjRCSVQNCiAJc2Vs
ZWN0IEhBU19GQVNUX01VTFRJUExZDQorCXNlbGVjdCBIQVNfUENJDQogDQog
Y29uZmlnIEFSTQ0KIAlkZWZfYm9vbCB5DQpkaWZmIC0tZ2l0IGEveGVuL2Fy
Y2gvYXJtL3BjaS9lY2FtLmMgYi94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0K
aW5kZXggZDMyZWZiN2ZjYi4uZjZkMGQwMGMxYiAxMDA2NDQNCi0tLSBhL3hl
bi9hcmNoL2FybS9wY2kvZWNhbS5jDQorKysgYi94ZW4vYXJjaC9hcm0vcGNp
L2VjYW0uYw0KQEAgLTI2LDggKzI2LDkgQEANCiB2b2lkIF9faW9tZW0gKnBj
aV9lY2FtX21hcF9idXMoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdl
LA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWlu
dDMyX3Qgc2JkZiwgdWludDMyX3Qgd2hlcmUpDQogew0KLSAgICBjb25zdCBz
dHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+c3lzZGF0
YTsNCi0gICAgdW5zaWduZWQgaW50IGRldmZuX3NoaWZ0ID0gY2ZnLT5vcHMt
PmJ1c19zaGlmdCAtIDg7DQorICAgIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9v
cHMgKm9wcyA9IGJyaWRnZS0+c3lzZGF0YTsNCisgICAgY29uc3Qgc3RydWN0
IHBjaV9jb25maWdfd2luZG93ICpjZmcgPSBicmlkZ2UtPmNmZzsNCisgICAg
dW5zaWduZWQgaW50IGRldmZuX3NoaWZ0ID0gb3BzLT5idXNfc2hpZnQgLSA4
Ow0KICAgICB2b2lkIF9faW9tZW0gKmJhc2U7DQogDQogICAgIHBjaV9zYmRm
X3Qgc2JkZl90ID0gKHBjaV9zYmRmX3QpIHNiZGYgOw0KQEAgLTM3LDcgKzM4
LDcgQEAgdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVjdCBw
Y2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCiAgICAgICAgIHJldHVybiBOVUxM
Ow0KIA0KICAgICBidXNuIC09IGNmZy0+YnVzbl9zdGFydDsNCi0gICAgYmFz
ZSA9IGNmZy0+d2luICsgKGJ1c24gPDwgY2ZnLT5vcHMtPmJ1c19zaGlmdCk7
DQorICAgIGJhc2UgPSBjZmctPndpbiArIChidXNuIDw8IG9wcy0+YnVzX3No
aWZ0KTsNCiANCiAgICAgcmV0dXJuIGJhc2UgKyAoUENJX0RFVkZOKHNiZGZf
dC5kZXYsIHNiZGZfdC5mbikgPDwgZGV2Zm5fc2hpZnQpICsgd2hlcmU7DQog
fQ0KZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29t
bW9uLmMgYi94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQpp
bmRleCBjMDRiZTYzNjQ1Li40MWE1NDU3ZTgwIDEwMDY0NA0KLS0tIGEveGVu
L2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KKysrIGIveGVuL2Fy
Y2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KQEAgLTk3LDcgKzk3LDYg
QEAgc3RhdGljIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqZ2VuX3BjaV9p
bml0KHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KIA0KICAgICBjZmct
PnBoeXNfYWRkciA9IGFkZHI7DQogICAgIGNmZy0+c2l6ZSA9IHNpemU7DQot
ICAgIGNmZy0+b3BzID0gb3BzOw0KIA0KICAgICAvKg0KICAgICAgKiBPbiA2
NC1iaXQgc3lzdGVtcywgd2UgZG8gYSBzaW5nbGUgaW9yZW1hcCBmb3IgdGhl
IHdob2xlIGNvbmZpZyBzcGFjZQ0KQEAgLTIyNSw3ICsyMjQsNyBAQCBzdGF0
aWMgaW50IHBjaV9idXNfZmluZF9kb21haW5fbnIoc3RydWN0IGR0X2Rldmlj
ZV9ub2RlICpkZXYpDQogfQ0KIA0KIGludCBwY2lfaG9zdF9jb21tb25fcHJv
YmUoc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQotICAgICAgICAgICAg
ICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1fb3BzICpvcHMs
DQorICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcGNpX2VjYW1f
b3BzICpvcHMsDQogICAgICAgICAgICAgICAgICAgICAgICAgICBpbnQgZWNh
bV9yZWdfaWR4KQ0KIHsNCiAgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAq
YnJpZGdlOw0KQEAgLTI0NSw3ICsyNDQsOCBAQCBpbnQgcGNpX2hvc3RfY29t
bW9uX3Byb2JlKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KICAgICB9
DQogDQogICAgIGJyaWRnZS0+ZHRfbm9kZSA9IGRldjsNCi0gICAgYnJpZGdl
LT5zeXNkYXRhID0gY2ZnOw0KKyAgICBicmlkZ2UtPmNmZyA9IGNmZzsNCisg
ICAgYnJpZGdlLT5zeXNkYXRhID0gb3BzOw0KICAgICBicmlkZ2UtPm9wcyA9
ICZvcHMtPnBjaV9vcHM7DQogICAgIGJyaWRnZS0+YnVzX3N0YXJ0ID0gY2Zn
LT5idXNuX3N0YXJ0Ow0KICAgICBicmlkZ2UtPmJ1c19lbmQgPSBjZmctPmJ1
c25fZW5kOw0KZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtZ2VuZXJpYy5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5l
cmljLmMNCmluZGV4IDJkNjUyZTg5MTAuLjY2MTc2Zjk2NTggMTAwNjQ0DQot
LS0gYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMuYw0KKysr
IGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5lcmljLmMNCkBAIC0z
Miw3ICszMiw3IEBAIHN0YXRpYyBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21h
dGNoIGdlbl9wY2lfZHRfbWF0Y2hbXSA9IHsNCiBzdGF0aWMgaW50IGdlbl9w
Y2lfZHRfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwgY29uc3Qg
dm9pZCAqZGF0YSkNCiB7DQogICAgIGNvbnN0IHN0cnVjdCBkdF9kZXZpY2Vf
bWF0Y2ggKm9mX2lkOw0KLSAgICBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1fb3Bz
ICpvcHM7DQorICAgIHN0cnVjdCBwY2lfZWNhbV9vcHMgKm9wczsNCiANCiAg
ICAgb2ZfaWQgPSBkdF9tYXRjaF9ub2RlKGdlbl9wY2lfZHRfbWF0Y2gsIGRl
di0+ZGV2Lm9mX25vZGUpOw0KICAgICBvcHMgPSAoc3RydWN0IHBjaV9lY2Ft
X29wcyAqKSBvZl9pZC0+ZGF0YTsNCmRpZmYgLS1naXQgYS94ZW4vYXJjaC9h
cm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jIGIveGVuL2FyY2gvYXJtL3BjaS9w
Y2ktaG9zdC16eW5xbXAuYw0KaW5kZXggZmUxMDNlMzg1NS4uYjQxNzBjM2Jk
ZCAxMDA2NDQNCi0tLSBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3Qtenlu
cW1wLmMNCisrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtenlucW1w
LmMNCkBAIC0zMiw3ICszMiw3IEBAIHN0YXRpYyBjb25zdCBzdHJ1Y3QgZHRf
ZGV2aWNlX21hdGNoIGdlbl9wY2lfZHRfbWF0Y2hbXSA9IHsNCiBzdGF0aWMg
aW50IGdlbl9wY2lfZHRfaW5pdChzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRl
diwgY29uc3Qgdm9pZCAqZGF0YSkNCiB7DQogICAgIGNvbnN0IHN0cnVjdCBk
dF9kZXZpY2VfbWF0Y2ggKm9mX2lkOw0KLSAgICBjb25zdCBzdHJ1Y3QgcGNp
X2VjYW1fb3BzICpvcHM7DQorICAgIHN0cnVjdCBwY2lfZWNhbV9vcHMgKm9w
czsNCiANCiAgICAgb2ZfaWQgPSBkdF9tYXRjaF9ub2RlKGdlbl9wY2lfZHRf
bWF0Y2gsIGRldi0+ZGV2Lm9mX25vZGUpOw0KICAgICBvcHMgPSAoc3RydWN0
IHBjaV9lY2FtX29wcyAqKSBvZl9pZC0+ZGF0YTsNCmRpZmYgLS1naXQgYS94
ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oIGIveGVuL2luY2x1ZGUvYXNtLWFy
bS9wY2kuaA0KaW5kZXggOWMyOGE0YmRjNC4uYzgwZDg0NmRhMyAxMDA2NDQN
Ci0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCisrKyBiL3hlbi9p
bmNsdWRlL2FzbS1hcm0vcGNpLmgNCkBAIC01NSw3ICs1NSw2IEBAIHN0cnVj
dCBwY2lfY29uZmlnX3dpbmRvdyB7DQogICAgIHVpbnQ4X3QgICAgICAgICBi
dXNuX3N0YXJ0Ow0KICAgICB1aW50OF90ICAgICAgICAgYnVzbl9lbmQ7DQog
ICAgIHZvaWQgX19pb21lbSAgICAqd2luOw0KLSAgICBjb25zdCBzdHJ1Y3Qg
ICAgcGNpX2VjYW1fb3BzICpvcHM7DQogfTsNCiANCiAvKg0KQEAgLTY4LDcg
KzY3LDggQEAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSB7DQogICAgIHVpbnQx
Nl90IHNlZ21lbnQ7ICAgICAgICAgICAgICAgIC8qIFNlZ21lbnQgbnVtYmVy
ICovDQogICAgIHU4IGJ1c19zdGFydDsgICAgICAgICAgICAgICAgICAgIC8q
IEJ1cyBzdGFydCBvZiB0aGlzIGJyaWRnZS4gKi8NCiAgICAgdTggYnVzX2Vu
ZDsgICAgICAgICAgICAgICAgICAgICAgLyogQnVzIGVuZCBvZiB0aGlzIGJy
aWRnZS4gKi8NCi0gICAgdm9pZCAqc3lzZGF0YTsgICAgICAgICAgICAgICAg
ICAgLyogUG9pbnRlciB0byB0aGUgY29uZmlnIHNwYWNlIHdpbmRvdyovDQor
ICAgIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyogY2ZnOyAgIC8qIFBvaW50
ZXIgdG8gdGhlIGJyaWRnZSBjb25maWcgd2luZG93ICovDQorICAgIHZvaWQg
KnN5c2RhdGE7ICAgICAgICAgICAgICAgICAgIC8qIFBvaW50ZXIgdG8gYnJp
ZGdlIHByaXZhdGUgZGF0YSAqLw0KICAgICBjb25zdCBzdHJ1Y3QgcGNpX29w
cyAqb3BzOw0KIH07DQogDQpAQCAtMTAwLDcgKzEwMCw3IEBAIHN0cnVjdCBw
Y2lfZWNhbV9vcHMgew0KIGV4dGVybiBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1f
b3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzOw0KIA0KIGludCBwY2lfaG9zdF9j
b21tb25fcHJvYmUoc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQotICAg
ICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGNpX2VjYW1f
b3BzICpvcHMsDQorICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGNpX2VjYW1fb3BzICpvcHMsDQogICAgICAgICAgICAgICAgICAgICAgICAg
ICBpbnQgZWNhbV9yZWdfaWR4KTsNCiBpbnQgcGNpX2dlbmVyaWNfY29uZmln
X3JlYWQoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1aW50MzJf
dCBzYmRmLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50MzJf
dCByZWcsIHVpbnQzMl90IGxlbiwgdWludDMyX3QgKnZhbHVlKTsNCg==

--8323329-579779311-1631738010=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 20:45:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 20:45:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187905.336922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbmk-0004kN-1X; Wed, 15 Sep 2021 20:45:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187905.336922; Wed, 15 Sep 2021 20:45:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQbmj-0004kG-Uf; Wed, 15 Sep 2021 20:45:37 +0000
Received: by outflank-mailman (input) for mailman id 187905;
 Wed, 15 Sep 2021 20:45:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQbmi-0004kA-AL
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 20:45:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da10bb1a-5a90-4c3e-ad00-e784d3879975;
 Wed, 15 Sep 2021 20:45:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 61A3D6112E;
 Wed, 15 Sep 2021 20:45:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da10bb1a-5a90-4c3e-ad00-e784d3879975
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631738734;
	bh=HtJZPtHOZ0+0iv37c96zwKO2Gj2UEfo585JjsV72U4A=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=oT0jmbnn0SQ3cSQYFvRiGDtczD3vBSG6tjbyGuaV6c6QpoYckJzQwd9CUV8i5efOX
	 hLWfGugdyKQIZmQKpFifSXNLexvLLpxvI6rWwQ4Kb5+/hpiEqAEkJ5gN8P4QY6zaua
	 nvsXbHmiiFOKAkMKo646AgcO7g+DgBYWvNqqryO74cqrH/pwtsEwtK6aTQoc+fzYXk
	 8MvlAgQXHVbK3uLYTVpzhPezsKgk9zp0lQ96Xylfh7EcUU2tSc4FNwkhI8wCcD1lTY
	 jzR49Vq9SMNaOEE4nmnYyndTsqT6d3dkw3hEHxA4cHFxWu6CpN/YSsQi49DeDAAAHj
	 SOHRQPeX4AbeQ==
Date: Wed, 15 Sep 2021 13:45:33 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
In-Reply-To: <8B84E0D6-70A7-4C8F-ACBE-D1773F6C8FAD@arm.com>
Message-ID: <alpine.DEB.2.21.2109151340390.21985@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s> <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
 <alpine.DEB.2.21.2109141543090.21985@sstabellini-ThinkPad-T480s> <8B84E0D6-70A7-4C8F-ACBE-D1773F6C8FAD@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-35407010-1631738734=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-35407010-1631738734=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 15 Sep 2021, Rahul Singh wrote:
> > On 15 Sep 2021, at 12:06 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > On Tue, 14 Sep 2021, Rahul Singh wrote:
> >>>> +        return NULL;
> >>>> +
> >>>> +    busn -= cfg->busn_start;
> >>>> +    base = cfg->win + (busn << cfg->ops->bus_shift);
> >>>> +
> >>>> +    return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where;
> >>>> +}
> >>> 
> >>> I understand that the arm32 part is not implemented and not part of this
> >>> series, that's fine. However if the plan is that arm32 will dynamically
> >>> map each bus individually, then I imagine this function will have an
> >>> ioremap in the arm32 version. Which means that we also need an
> >>> unmap_bus call in struct pci_ops. I understand that pci_ecam_unmap_bus
> >>> would be a NOP today for arm64, but I think it makes sense to have it if
> >>> we want the API to be generic.
> >> 
> >> As per my understanding we don’t need pci_ecam_unmap_bus(..) as I don’t see any use case to unmap the 
> >> bus dynamically. We can add the support for per_bus_mapping for ARM32 once we implement arm32 part.
> >> Let me know your view on this.
> > 
> > In the patch titled "xen/arm: PCI host bridge discovery within XEN on
> > ARM" there is the following in-code comment:
> > 
> > * On 64-bit systems, we do a single ioremap for the whole config space
> > * since we have enough virtual address range available.  On 32-bit, we
> > * ioremap the config space for each bus individually.
> > *
> > * As of now only 64-bit is supported 32-bit is not supported.
> > 
> > 
> > So I take it that on arm32 we don't have enough virtual address range
> > available, therefore we cannot ioremap the whole range. Instead, we'll
> > have to ioremap the config space of each bus individually.
> 
> Yes you are right my understand is also same.
> > 
> > I assumed that the idea was to call ioremap and iounmap dynamically,
> > otherwise the total amount of virtual address range required would still
> > be the same.
> 
> As per my understanding for 32-bit we need per_bus mapping as we don’t have enough virtual address space in one chunk
> but we can have virtual address space in different chunk.

Interesting. I would have assumed that the sum of all the individual
smaller ioremaps would still be equal to one big ioremap. Maybe for
Linux is different, but I don't think that many smaller ioremaps would
buy us very much in Xen because it is the total ioremap virtual space
that is too small. Or am I missing something?


> I am not sure if we need to map/unmap the virtual address space for each read/write call. 
> I just checked the Linux code[1]  and there also mapping is done once not for each read/write call.

So my guess is that for arm32 we would have to resort to dynamic
map/unmap for each read/write call, unless there is a trick with the
individual smaller ioremaps that I haven't spotted (e.g. maybe something
doesn't get mapped that way?)

That said, given that we are uncertain about this and the arm32
implementation is nowhere close, I think that we are OK to continue like
this for this series. Maybe you could add a couple of sentences to the
in-code comment so that if somebody wants to jump in and implement
arm32 support they would know where to start.
--8323329-35407010-1631738734=:21985--


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 21:21:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 21:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187915.336944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcLL-00019V-6m; Wed, 15 Sep 2021 21:21:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187915.336944; Wed, 15 Sep 2021 21:21:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcLL-00019M-3W; Wed, 15 Sep 2021 21:21:23 +0000
Received: by outflank-mailman (input) for mailman id 187915;
 Wed, 15 Sep 2021 21:21:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQcLK-00018p-Ck; Wed, 15 Sep 2021 21:21:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQcLK-00077Y-3x; Wed, 15 Sep 2021 21:21:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQcLJ-0002Ty-Qe; Wed, 15 Sep 2021 21:21:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQcLJ-0007Nu-QA; Wed, 15 Sep 2021 21:21:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=s4X32ZLQIcDqiFmWtRGR5t6KSN4zmybOqKtZF7sj6dM=; b=bR8VjuGHcXjeBzSmIHdUMpieHb
	Mu6glLGOdAvD8uNphAmyIujxa6BzXpsDgpjPq8TQkAqYOArLx6XfinPkDLEMvD1abENBmZstwXGv2
	zMGQWYi3PpG7F9UAE45LFZeGU2IrpIvgt8/KUihC5qAh1Uud9gx6nVmRf/LntTqfcbEE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164993-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 164993: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=245f15a48cdc4d5a90902e140392dc151e528ab8
X-Osstest-Versions-That:
    linux=a0f68fb55ebc85e5ed3c6582ffc70379340c4a72
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 15 Sep 2021 21:21:21 +0000

flight 164993 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164993/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164960
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164960
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164960
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164960
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164960
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164960
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164960
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164960
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164960
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164960
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164960
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164960
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                245f15a48cdc4d5a90902e140392dc151e528ab8
baseline version:
 linux                a0f68fb55ebc85e5ed3c6582ffc70379340c4a72

Last test of basis   164960  2021-09-12 18:12:15 Z    3 days
Testing same since   164993  2021-09-15 08:12:34 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adriana Reus <adriana.reus@nxp.com>
  Ahmad Fatoum <a.fatoum@pengutronix.de>
  Alan Stern <stern@rowland.harvard.edu>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexei Starovoitov <ast@kernel.org>
  Amit Engel <amit.engel@dell.com>
  Anand Moon <linux.amoon@gmail.com>
  Andrey Ignatov <rdna@fb.com>
  Andrii Nakryiko <andrii@kernel.org>
  Andy Duan <fugang.duan@nxp.com>
  Andy Shevchenko <andy.shevchenko@gmail.com>
  Austin Kim <austin.kim@lge.com>
  Babu Moger <babu.moger@amd.com>
  Benedict Schlueter <benedict.schlueter@rub.de>
  Benjamin Coddington <bcodding@redhat.com>
  Bjorn Andersson <bjorn.andersson@linaro.org>
  Borislav Petkov <bp@suse.de>
  Cezary Rojewski <cezary.rojewski@intel.com>
  Chen-Yu Tsai <wenst@chromium.org>
  Chih-Kang Chang <gary.chang@realtek.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Chuck Lever <chuck.lever@oracle.com>
  Chunyan Zhang <chunyan.zhang@unisoc.com>
  Colin Ian King <colin.king@canonical.com>
  Coly Li <colyli@suse.de>
  Damien Le Moal <damien.lemoal@wdc.com>
  Dan Carpenter <dan.carpenter@oracle.com>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  Daniel Thompson <daniel.thompson@linaro.org>
  Daniel Vetter <daniel.vetter@ffwll.ch>
  Darrick J. Wong <djwong@kernel.org>
  David Ahern <dsahern@kernel.org>
  David S. Miller <davem@davemloft.net>
  Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dongliang Mu <mudongliangabcd@gmail.com>
  Dylan Hung <dylan_hung@aspeedtech.com>
  Edward Cree <ecree.xilinx@gmail.com>
  Eric Dumazet <edumazet@google.com>
  Evgeny Novikov <novikov@ispras.ru>
  Felipe Balbi <balbi@kernel.org>
  Florian Fainelli <f.fainelli@gmail.com>
  Frederic Weisbecker <frederic@kernel.org>
  Gang Deng <gavin.dg@linux.alibaba.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Geert Uytterhoeven <geert@linux-m68k.org>
  Giovanni Cabiddu <giovanni.cabiddu@intel.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guillaume Nault <gnault@redhat.com>
  Halil Pasic <pasic@linux.ibm.com>
  Hans de Goede <hdegoede@redhat.com>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  He Fengqing <hefengqing@huawei.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko@sntech.de>
  Herbert Xu <herbert@gondor.apana.org.au>
  Hongbo Li <herberthbli@tencent.com>
  Hsin-Yi Wang <hsinyi@chromium.org>
  Hulk Robot <hulkrobot@huawei.com>
  Håkon Bugge <haakon.bugge@oracle.com>
  Ingo Molnar <mingo@kernel.org>
  J. Bruce Fields <bfields@redhat.com>
  Jakub Kicinski <kuba@kernel.org>
  Jan Kara <jack@suse.cz>
  Jarkko Sakkinen <jarkko@kernel.org>
  Jeff Layton <jlayton@kernel.org>
  Jens Axboe <axboe@kernel.dk>
  Jeongtae Park <jeongtae.park@gmail.com>
  Joel Stanley <joel@jms.id.au>
  Johannes Berg <johannes.berg@intel.com>
  John Fastabend <john.fastabend@gmail.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Juri Lelli <juri.lelli@redhat.com>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Kalle Valo <kvalo@codeaurora.org>
  Kim Phillips <kim.phillips@amd.com>
  Kirill Shilimanov <kirill.shilimanov@huawei.com>
  Krzysztof Halasa <khalasa@piap.pl>
  Krzysztof Hałasa <khalasa@piap.pl>
  Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
  Kuniyuki Iwashima <kuniyu@amazon.co.jp>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee.jones@linaro.org>
  Len Baker <len.baker@gmx.com>
  Linus Walleij <linus.walleij@linaro.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Lorenz Bauer <lmb@cloudflare.com>
  Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
  Lukas Hannen <lukas.hannen@opensource.tttech-industrial.com>
  Lukasz Luba <lukasz.luba@arm.com>
  Lukasz Majczak <lma@semihalf.com>
  Marcel Holtmann <marcel@holtmann.org>
  Marco Chiappero <marco.chiappero@intel.com>
  Marek Vasut <marex@denx.de>
  Mark Brown <broonie@kernel.org>
  Martin Blumenstingl <martin.blumenstingl@googlemail.com>
  Martin KaFai Lau <kafai@fb.com>
  Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
  Maxim Mikityanskiy <maximmi@nvidia.com>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Miklos Szeredi <mszeredi@redhat.com>
  Mimi Zohar <zohar@linux.ibm.com>
  Nadezda Lutovinova <lutovinova@ispras.ru>
  Neil Armstrong <narmstrong@baylibre.com>
  Nguyen Dinh Phi <phind.uet@gmail.com>
  Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
  Ovidiu Panait <ovidiu.panait@windriver.com>
  Pali Rohár <pali@kernel.org>
  Paolo Bonzini <pbonzini@redhat.com>
  Paul Moore <paul@paul-moore.com>
  Pavel Machek <pavel@ucw.cz>
  Pavel Skripkin <paskripkin@gmail.com>
  Peter Oberparleiter <oberpar@linux.ibm.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Philipp Zabel <p.zabel@pengutronix.de>
  Phong Hoang <phong.hoang.wz@renesas.com>
  Ping-Ke Shih <pkshih@realtek.com>
  Piotr Krysiuk <piotras@gmail.com>
  Qiuxu Zhuo <qiuxu.zhuo@intel.com>
  Quanyang Wang <quanyang.wang@windriver.com>
  Quentin Perret <qperret@google.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rob Clark <robdclark@chromium.org>
  Ruozhu Li <liruozhu@huawei.com>
  Saeed Mahameed <saeedm@nvidia.com>
  Sasha Levin <sashal@kernel.org>
  Sean Anderson <sean.anderson@seco.com>
  Sean Christopherson <seanjc@google.com>
  Sean Young <sean@mess.org>
  Sebastian Krzyszkowiak <sebastian.krzyszkowiak@puri.sm>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Sergey Senozhatsky <senozhatsky@chromium.org>
  Sergey Shtylyov <s.shtylyov@omp.ru>
  Sergey Shtylyov <s.shtylyov@omprussia.ru>
  Sherry Sun <sherry.sun@nxp.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Sibi Sankar <sibis@codeaurora.org>
  Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
  Song Liu <songliubraving@fb.com>
  Stanimir Varbanov <stanimir.varbanov@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Wahren <stefan.wahren@i2se.com>
  Stephan Gerhold <stephan@gerhold.net>
  Stephen Boyd <sboyd@kernel.org>
  Steve French <stfrench@microsoft.com>
  Steven Price <steven.price@arm.com>
  Stian Skjelstad <stian.skjelstad@gmail.com>
  Subbaraya Sundeep <sbhatta@marvell.com>
  Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
  Sunil Goutham <sgoutham@marvell.com>
  Sven Eckelmann <sven@narfation.org>
  Syed Nayyar Waris <syednwaris@gmail.com>
  Tejun Heo <tj@kernel.org>
  Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
  THOBY Simon <Simon.THOBY@viveris.fr>
  Thomas Gleixner <tglx@linutronix.de>
  Tony Lindgren <tony@atomide.com>
  Tony Luck <tony.luck@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Vasily Gorbik <gor@linux.ibm.com>
  Vineeth Vijayan <vneethv@linux.ibm.com>
  Vladimir Oltean <vladimir.oltean@nxp.com>
  Waiman Long <longman@redhat.com>
  Wei Yongjun <weiyongjun1@huawei.com>
  Wen Jin <wen.jin@intel.com>
  William Breathitt Gray <vilhelm.gray@gmail.com>
  Wolfram Sang <wsa@kernel.org>
  Xiyu Yang <xiyuyang19@fudan.edu.cn>
  Xu Yu <xuyu@linux.alibaba.com>
  Yonghong Song <yhs@fb.com>
  Zelin Deng <zelin.deng@linux.alibaba.com>
  Zenghui Yu <yuzenghui@huawei.com>
  Zhen Lei <thunder.leizhen@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   a0f68fb55ebc..245f15a48cdc  245f15a48cdc4d5a90902e140392dc151e528ab8 -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 21:21:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 21:21:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187913.336933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcL8-0000ow-Oy; Wed, 15 Sep 2021 21:21:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187913.336933; Wed, 15 Sep 2021 21:21:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcL8-0000op-Lr; Wed, 15 Sep 2021 21:21:10 +0000
Received: by outflank-mailman (input) for mailman id 187913;
 Wed, 15 Sep 2021 21:21:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3+tO=OF=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQcL6-0000oj-NT
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 21:21:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d68f6a71-166a-11ec-b584-12813bfff9fa;
 Wed, 15 Sep 2021 21:21:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AB41961108;
 Wed, 15 Sep 2021 21:21:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d68f6a71-166a-11ec-b584-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631740866;
	bh=xnDWEdOh1AQwsewvkXeEULM+kziuFgr1Q7FC4n7uLGI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=o9uZci0nbSWll7GV0J+Gan4BX3kby2PHzsDtbRgaqnL5GMiyuO/ZJYHMDTUSPar+i
	 xKkIXCf36/W1Zo9uljdoo67rDd9peN1qI3zOongvXJ5SFW/hhb9nylp3PfiAsiYenr
	 bsKyHxMHwNrc5ucxKrkE0fF0v7rSQamUgl8bLmTRXNHADJSR0bEF5vjSs/0pYPWVDn
	 PTtBGGYdHoCrHDZ/TvBAWB64hQZeQgJ85xlkmM+mwpRuJcekRICE1oUBKxmE7ttVM0
	 MKkvRlanbUVUZzc57TIlR+h8SHVh0KcCaadDKSZZpczwn6q2oghcUXv5/WLmWAT5WZ
	 AfYixZl+hxwCw==
Date: Wed, 15 Sep 2021 14:21:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
Message-ID: <alpine.DEB.2.21.2109151352100.21985@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s> <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 Sep 2021, Oleksandr wrote:
> > On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
> > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > 
> > > The extended region (safe range) is a region of guest physical
> > > address space which is unused and could be safely used to create
> > > grant/foreign mappings instead of wasting real RAM pages from
> > > the domain memory for establishing these mappings.
> > > 
> > > The extended regions are chosen at the domain creation time and
> > > advertised to it via "reg" property under hypervisor node in
> > > the guest device-tree. As region 0 is reserved for grant table
> > > space (always present), the indexes for extended regions are 1...N.
> > > If extended regions could not be allocated for some reason,
> > > Xen doesn't fail and behaves as usual, so only inserts region 0.
> > > 
> > > Please note the following limitations:
> > > - The extended region feature is only supported for 64-bit domain.
> > > - The ACPI case is not covered.
> > > 
> > > ***
> > > 
> > > As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> > > the algorithm to choose extended regions for it is different
> > > in comparison with the algorithm for non-direct mapped DomU.
> > > What is more, that extended regions should be chosen differently
> > > whether IOMMU is enabled or not.
> > > 
> > > Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> > > holes found in host device-tree if otherwise. Make sure that
> > > extended regions are 2MB-aligned and located within maximum possible
> > > addressable physical memory range. The maximum number of extended
> > > regions is 128.
> > > 
> > > Suggested-by: Julien Grall <jgrall@amazon.com>
> > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > ---
> > > Changes since RFC:
> > >     - update patch description
> > >     - drop uneeded "extended-region" DT property
> > > ---
> > > 
> > >   xen/arch/arm/domain_build.c | 226
> > > +++++++++++++++++++++++++++++++++++++++++++-
> > >   1 file changed, 224 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > index 206038d..070ec27 100644
> > > --- a/xen/arch/arm/domain_build.c
> > > +++ b/xen/arch/arm/domain_build.c
> > > @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct
> > > domain *d,
> > >       return res;
> > >   }
> > >   +static int __init add_ext_regions(unsigned long s, unsigned long e,
> > > void *data)
> > > +{
> > > +    struct meminfo *ext_regions = data;
> > > +    paddr_t start, size;
> > > +
> > > +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> > > +        return 0;
> > > +
> > > +    /* Both start and size of the extended region should be 2MB aligned
> > > */
> > > +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> > > +    if ( start > e )
> > > +        return 0;
> > > +
> > > +    size = (e - start + 1) & ~(SZ_2M - 1);
> > > +    if ( !size )
> > > +        return 0;
> > Can't you align size as well?
> > 
> >    size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
> 
> I am sorry, I don't entirely get what you really meant here. We get both start
> and size 2MB-aligned by the calculations above
> (when calculating an alignment, we need to make sure that "start_passed <=
> start_aligned && size_aligned <= size_passed").
> If I add the proposing string after, I will reduce the already aligned size by
> 2MB.
> If I replace the size calculation with the following, I will get the reduced
> size even if the passed region is initially 2MB-aligned, so doesn't need to be
> adjusted.
> size = e - s + 1;
> size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);

Sorry I misread your original code, I think it was working as intended
except for the "+1". I think it should be:

  size = (e - start) & ~(SZ_2M - 1);



> > > + */
> > > +#define EXT_REGION_START   0x40000000ULL
> > > +#define EXT_REGION_END     0x80003fffffffULL
> > > +
> > > +static int __init find_unallocated_memory(const struct kernel_info
> > > *kinfo,
> > > +                                          struct meminfo *ext_regions)
> > > +{
> > > +    const struct meminfo *assign_mem = &kinfo->mem;
> > > +    struct rangeset *unalloc_mem;
> > > +    paddr_t start, end;
> > > +    unsigned int i;
> > > +    int res;
> > > +
> > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > +
> > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > +    if ( !unalloc_mem )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with all available RAM */
> > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.mem.bank[i].start;
> > > +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
> > Is the -1 needed? Isn't it going to screw up the size calculation later?
> I thought, it was needed. The calculation seems correct.

I think that normally for an example MMIO region:

start = 0x48000000
size  = 0x40000000
end   = 0x88000000

So end = start + size and points to the first address out of the range.
In other words, 0x88000000 doesn't actually belong to the MMIO region in
the example.

But here you are passing addresses to rangeset_add_range and other
rangeset functions and I think rangeset takes *inclusive* addresses as
input. So you need to pass start and end-1 because end-1 is the last
address of the MMIO region.

In fact you can see for instance in map_range_to_domain:

        res = iomem_permit_access(d, paddr_to_pfn(addr),
                paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));

Where iomem_permit_access is based on rangeset. So for clarity, I would
do:

start = assign_mem->bank[i].start;
end = assign_mem->bank[i].start + assign_mem->bank[i].size;
res = rangeset_remove_range(unalloc_mem, start, end - 1);

So that we don't get confused on the meaning of "end" which everywhere
else means the first address not in range.


> > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove RAM assigned to Dom0 */
> > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > +    {
> > > +        start = assign_mem->bank[i].start;
> > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove reserved-memory regions */
> > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove grant table region */
> > > +    start = kinfo->gnttab_start;
> > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    start = EXT_REGION_START;
> > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > +                                 add_ext_regions, ext_regions);
> > > +    if ( res )
> > > +        ext_regions->nr_banks = 0;
> > > +    else if ( !ext_regions->nr_banks )
> > > +        res = -ENOENT;
> > > +
> > > +out:
> > > +    rangeset_destroy(unalloc_mem);
> > > +
> > > +    return res;
> > > +}
> > > +
> > > +static int __init find_memory_holes(const struct kernel_info *kinfo,
> > > +                                    struct meminfo *ext_regions)
> > > +{
> > > +    struct dt_device_node *np;
> > > +    struct rangeset *mem_holes;
> > > +    paddr_t start, end;
> > > +    unsigned int i;
> > > +    int res;
> > > +
> > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > +
> > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > +    if ( !mem_holes )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with maximum possible addressable physical memory range */
> > > +    start = EXT_REGION_START;
> > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > +    res = rangeset_add_range(mem_holes, start, end);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
> > > +    dt_for_each_device_node( dt_host, np )
> > Don't you need something like device_tree_for_each_node ?
> > dt_for_each_device_node won't go down any deeper in the tree?
> 
> Thank you for pointing this out, I will investigate and update the patch.
> 
> 
> > 
> > Alternatively, maybe we could simply record the highest possible address
> > of any memory/device/anything as we scan the device tree with
> > handle_node. Then we can use that as the starting point here.
> I also don't like the idea to scan the DT much, but I failed to find an
> effective solution how to avoid that.
> Yes, we can record the highest possible address, but I am afraid, I didn't
> entirely get a suggestion. Is the suggestion to provide a single region
> starting from highest possible address + 1 and up to the EXT_REGION_END
> suitably aligned? Could you please clarify?

Yes, that is what I was suggesting as a possible alternative: start from
the highest possible address in DT + 1 and up to the EXT_REGION_END
suitably aligned. But that wouldn't solve the <4GB issue.

 
> > > +                goto out;
> > > +            }
> > > +
> > > +            start = addr & PAGE_MASK;
> > > +            end = PAGE_ALIGN(addr + size) - 1;
> > > +            res = rangeset_remove_range(mem_holes, start, end);
> > > +            if ( res )
> > > +            {
> > > +                printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                       start, end);
> > > +                goto out;
> > > +            }
> > > +        }
> > > +    }
> > As is, it will result in a myriad of small ranges which is unuseful and
> > slow to parse. I suggest to simplify it by removing a larger region than
> > strictly necessary. For instance, you could remove a 1GB-aligned and
> > 1GB-multiple region for each range. That way, you are going to get fewer
> > large free ranges instance of many small ones which we don't need.
> 
> I agree with you that a lot of small ranges increase the bookkeeping in Dom0
> and there is also a theoretical (?) possibility
> that small ranges occupy all space we provide for extended regions
> (NR_MEM_BANKS)...
> But, let's consider my setup as an example again, but when the IOMMU is
> enabled for Dom0 ("holes found in DT").
> 
> --- The RAM configuration is the same:
> 
> (XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
> (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
> (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
> (XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3
> 
> --- There are a lot of various platform devices with reg property described in
> DT, I will probably not post all IO ranges here, just say that mostly all of
> them to be mapped at 0xE0000000-0xFFFFFFFF.
> 
> --- As we only pick up ranges with size >= 2MB, the calculated extended
> regions are (based on 40-bit IPA):
> 
> (XEN) Extended region 0: 0x40000000->0x47e00000
> (XEN) Extended region 1: 0xc0000000->0xe6000000
> (XEN) Extended region 2: 0xe7000000->0xe7200000
> (XEN) Extended region 3: 0xe7400000->0xe7600000
> (XEN) Extended region 4: 0xe7800000->0xec000000
> (XEN) Extended region 5: 0xec200000->0xec400000
> (XEN) Extended region 6: 0xec800000->0xee000000
> (XEN) Extended region 7: 0xee600000->0xee800000
> (XEN) Extended region 8: 0xeea00000->0xf1000000
> (XEN) Extended region 9: 0xf1200000->0xfd000000
> (XEN) Extended region 10: 0xfd200000->0xfd800000
> (XEN) Extended region 11: 0xfda00000->0xfe000000
> (XEN) Extended region 12: 0xfe200000->0xfe600000
> (XEN) Extended region 13: 0xfec00000->0xff800000
> (XEN) Extended region 14: 0x100000000->0x500000000
> (XEN) Extended region 15: 0x580000000->0x600000000
> (XEN) Extended region 16: 0x680000000->0x700000000
> (XEN) Extended region 17: 0x780000000->0x10000000000
> 
> So, if I *correctly* understood your idea about removing 1GB-aligned
> 1GB-multiple region for each range we would get the following:
> 
> (XEN) Extended region 0: 0x100000000->0x500000000
> (XEN) Extended region 1: 0x580000000->0x600000000
> (XEN) Extended region 2: 0x680000000->0x700000000
> (XEN) Extended region 3: 0x780000000->0x10000000000
> 
> As you can see there are no extended regions below 4GB at all. I assume, it
> would be good to provide them for 1:1 mapped Dom0 (for 32-bit DMA devices?)
> What else worries me is that IPA size could be 36 or even 32. So, I am afraid,
> we might even fail to find extended regions above 4GB.
> 
> 
> I think, if 2MB is considered small enough to bother with, probably we should
> go with something in between (16MB, 32MB, 64MB).
> For example, we can take into the account ranges with size >= 16MB:
> 
> (XEN) Extended region 0: 0x40000000->0x47e00000
> (XEN) Extended region 1: 0xc0000000->0xe6000000
> (XEN) Extended region 2: 0xe7800000->0xec000000
> (XEN) Extended region 3: 0xec800000->0xee000000
> (XEN) Extended region 4: 0xeea00000->0xf1000000
> (XEN) Extended region 5: 0xf1200000->0xfd000000
> (XEN) Extended region 6: 0x100000000->0x500000000
> (XEN) Extended region 7: 0x580000000->0x600000000
> (XEN) Extended region 8: 0x680000000->0x700000000
> (XEN) Extended region 9: 0x780000000->0x10000000000
> 
> Any thoughts?

Yeah maybe an intermediate value would be best. I'd go with 64MB.


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 22:02:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 22:02:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187932.336958 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcz6-000647-Gb; Wed, 15 Sep 2021 22:02:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187932.336958; Wed, 15 Sep 2021 22:02:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQcz6-000640-DM; Wed, 15 Sep 2021 22:02:28 +0000
Received: by outflank-mailman (input) for mailman id 187932;
 Wed, 15 Sep 2021 22:02:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Vf4E=OF=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mQcz4-00063u-ND
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 22:02:27 +0000
Received: from forward500p.mail.yandex.net (unknown [77.88.28.110])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 185c1760-c4c5-402a-ae21-d4861e70dcb7;
 Wed, 15 Sep 2021 22:02:24 +0000 (UTC)
Received: from iva5-d37fe543de32.qloud-c.yandex.net
 (iva5-d37fe543de32.qloud-c.yandex.net
 [IPv6:2a02:6b8:c0c:6a6:0:640:d37f:e543])
 by forward500p.mail.yandex.net (Yandex) with ESMTP id 3ED3BF0207B;
 Thu, 16 Sep 2021 01:02:22 +0300 (MSK)
Received: from 2a02:6b8:c0c:128f:0:640:c3f7:29dd
 (2a02:6b8:c0c:128f:0:640:c3f7:29dd [2a02:6b8:c0c:128f:0:640:c3f7:29dd])
 by iva5-d37fe543de32.qloud-c.yandex.net (mxback/Yandex) with HTTP id
 H2ge3P1Df8c1-2LDmrj27; Thu, 16 Sep 2021 01:02:21 +0300
Received: by iva4-c3f729ddadf7.qloud-c.yandex.net with HTTP;
 Thu, 16 Sep 2021 01:02:20 +0300
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 185c1760-c4c5-402a-ae21-d4861e70dcb7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631743341;
	bh=8dOvvtT153yNC1427nLKREV1Y00TVP3qVUTQ6ci4o80=;
	h=References:Date:Message-Id:Cc:Subject:In-Reply-To:To:From;
	b=M21wcaZbiby4Ql+v51FCUqUNb8Gpet9iKHXZR0KAOfHBWISs/4fVMBbWzHwbFynHP
	 6RtaKe9HbiIMMKRGJpLFVOyPxVTqefJWWxja0gxTOCITWOwWudP+keaXV+nmX6FStn
	 2UehyeMR6SrpwjlXpCsS6j3tob3ggR9VEfzFT5rM=
Authentication-Results: iva5-d37fe543de32.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: =?utf-8?B?0JTQvNC40YLRgNC40Lkg0JjRgdCw0LnQutC40L0=?= <isaikin-dmitry@yandex.ru>
To: Ian Jackson <iwj@xenproject.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
In-Reply-To: <24896.33446.713051.453627@mariner.uk.xensource.com>
References: <cover.1631555561.git.isaikin-dmitry@yandex.ru>
	<9dc667d2017e6e69877a2b9859041c1f7d965899.1631555561.git.isaikin-dmitry@yandex.ru> <24896.33446.713051.453627@mariner.uk.xensource.com>
Subject: Re: [PATCH 1/1] tools/xl: Fix regular expression for autobaloon mode detection based on xen command line.
MIME-Version: 1.0
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Thu, 16 Sep 2021 01:02:20 +0300
Message-Id: <237131631740227@mail.yandex.ru>
Content-Transfer-Encoding: 8bit
Content-Type: text/html; charset=utf-8

<div><div>I checked only the cases "dom0_mem=101376M" and "dom0_mem=-24832M,max:-24832M".</div><div>In both cases, the regex worked correctly.</div></div><div> </div><div><div>Simultaneous restoration of several (1-15) virtual machines from snapshots also works stably in our production (on xen 4.14).</div></div><div> </div><div>14.09.2021, 14:08, "Ian Jackson" &lt;iwj@xenproject.org&gt;:</div><blockquote><p>Dmitry Isaikin writes ("[PATCH 1/1] tools/xl: Fix regular expression for autobaloon mode detection based on xen command line."):</p><blockquote> The case of specifying a negative size was handled incorrectly.</blockquote><p><br />Thank you. The change looks good to me. Did you test that after your<br />change, the functionality works ?<br /><br />Ian.</p></blockquote>


From xen-devel-bounces@lists.xenproject.org Wed Sep 15 22:13:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 22:13:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187942.336968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQdA9-0007l6-LW; Wed, 15 Sep 2021 22:13:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187942.336968; Wed, 15 Sep 2021 22:13:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQdA9-0007kz-IS; Wed, 15 Sep 2021 22:13:53 +0000
Received: by outflank-mailman (input) for mailman id 187942;
 Wed, 15 Sep 2021 22:13:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hLCT=OF=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQdA7-0007kt-SW
 for xen-devel@lists.xenproject.org; Wed, 15 Sep 2021 22:13:51 +0000
Received: from mail-ed1-x536.google.com (unknown [2a00:1450:4864:20::536])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c823cd46-a93b-4691-88ea-e924fb5a4d00;
 Wed, 15 Sep 2021 22:13:50 +0000 (UTC)
Received: by mail-ed1-x536.google.com with SMTP id v24so9105451eda.3
 for <xen-devel@lists.xenproject.org>; Wed, 15 Sep 2021 15:13:50 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id y5sm570302edt.21.2021.09.15.15.13.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 15 Sep 2021 15:13:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c823cd46-a93b-4691-88ea-e924fb5a4d00
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=z6s8HekUJoDb2a5ktczkdXRA12eitQ0ypYAE0sp77L0=;
        b=U4Z7xNlUBo3eb+qaiZt3wOT9cJbCBvKNfaqd+4N3h4n+7t2w7iPDOBO1645+7lbrsA
         6wtfmjay53SXDK7Efu4EoQNHwjt4OVu/iCDyZD9s8Dq2x9gbes+if6nbP3kRVmBplyZw
         E6gk74PMk5Q4xXiKmn2MIIn3HUW31U5+GTUkqGTfnD6SguA3ccR88EyXC3pN4myrSsPC
         NMXzywxRClj1di1b34XS0AUWBGV1d9F2no6Gf5RUVYIO6eYzAik3YqP+VVxyblXYbyUN
         PSzsu7Pw8/yy6ZAJ16so6b/NfCQMuXzQc/xCL7BhpvWkGCG5Xa05ljCDrJJYR4iK9p1W
         01QQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=z6s8HekUJoDb2a5ktczkdXRA12eitQ0ypYAE0sp77L0=;
        b=fcO/uv/8SRD9zSR450EjLe1p2a9G2ZNXk45VIH/oe+9KEPQTGlSYW0MTup8Htt1OSr
         h36fSz8nbTDIFfLVKrGOZFo07CxZIWasx0tyrVRxEX5ozUjMmXxV+S8Yzf+J6EYiCwUG
         8y/GZj4KkEZnOR6Pc3B9oSQSy82i7I0CLheuxeJY9LIz/bGVr35vWMxDJK/SI+08JRk1
         bj7l4n2TvWkMoUADvT6Y+/2dl4A6Jii5WHR2vVZyhblkB3XEzjnX1DjK8CEocDBz0NWg
         JJsIODydboUIMvLVvqpkTO1tsBrU7U6gy9Ip1Tzff7nQilwyu4bwCNYnto072EHuYJ99
         7Wbw==
X-Gm-Message-State: AOAM533lAH5byOu5GN5I5Yc/tbw7LmgxrWDnrpY3HBelzvn5YzyWrFH0
	De+vvWn5++r6oTsIHKWWsr46IImdjRT8agZy
X-Google-Smtp-Source: ABdhPJwsl8nXr+Q2TiUjOgek+P48TbmnNOA4dG6PDzIAOORzrpS+25CzoOCEfVOfrWuYfRSD4u2GOw==
X-Received: by 2002:a05:6402:1808:: with SMTP id g8mr2545894edy.188.1631744029320;
        Wed, 15 Sep 2021 15:13:49 -0700 (PDT)
Subject: Re: [RFC PATCH V2] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
 <b6744333-4d43-ef24-0f9b-b5cd54680660@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <86cdc577-7085-48cd-c417-85b20afc9bf1@gmail.com>
Date: Thu, 16 Sep 2021 01:13:48 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <b6744333-4d43-ef24-0f9b-b5cd54680660@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 15.09.21 13:06, Jan Beulich wrote:

Hi Jan

> On 14.09.2021 22:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/arch/arm/mm.c
>> +++ b/xen/arch/arm/mm.c
>> @@ -1376,14 +1376,18 @@ unsigned long domain_get_maximum_gpfn(struct domain *d)
>>   void share_xen_page_with_guest(struct page_info *page, struct domain *d,
>>                                  enum XENSHARE_flags flags)
>>   {
>> +    unsigned long type_info;
>> +
>>       if ( page_get_owner(page) == d )
>>           return;
>>   
>>       spin_lock(&d->page_alloc_lock);
>>   
>>       /* The incremented type count pins as writable or read-only. */
>> -    page->u.inuse.type_info =
>> -        (flags == SHARE_ro ? PGT_none : PGT_writable_page) | 1;
>> +    type_info = page->u.inuse.type_info & ~(PGT_type_mask | PGT_count_mask);
>> +    page->u.inuse.type_info = type_info |
>> +        (flags == SHARE_ro ? PGT_none : PGT_writable_page) |
>> +        (1UL << PGT_count_base);
> Just as a note: If this was x86 code, I'd request the redundant
> PGT_count_base to be dropped. Constructs like the above is what we
> have MASK_INSR() for.

I got it, I will look at it.


>
>> --- a/xen/common/page_alloc.c
>> +++ b/xen/common/page_alloc.c
>> @@ -2204,7 +2204,10 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
>>           return NULL;
>>   
>>       for ( i = 0; i < (1u << order); i++ )
>> +    {
>>           pg[i].count_info |= PGC_xen_heap;
>> +        arch_alloc_xenheap_page(&pg[i]);
>> +    }
>>   
>>       return page_to_virt(pg);
>>   }
>> @@ -2222,7 +2225,10 @@ void free_xenheap_pages(void *v, unsigned int order)
>>       pg = virt_to_page(v);
>>   
>>       for ( i = 0; i < (1u << order); i++ )
>> +    {
>>           pg[i].count_info &= ~PGC_xen_heap;
>> +        arch_free_xenheap_page(&pg[i]);
>> +    }
>>   
>>       free_heap_pages(pg, order, true);
>>   }
> You look to only be adjusting the !SEPARATE_XENHEAP instances of the
> functions. Isn't 32-bit Arm using SEPARATE_XENHEAP?

Hmm, looks like yes, thank you. At least config.h defines that on Arm32. 
I will update the instances.


>
>> --- a/xen/include/asm-arm/mm.h
>> +++ b/xen/include/asm-arm/mm.h
>> @@ -98,9 +98,18 @@ struct page_info
>>   #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>>   #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>>   
>> - /* Count of uses of this frame as its current type. */
>> -#define PGT_count_width   PG_shift(2)
>> -#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>> + /* 3-bit count of uses of this frame as its current type. */
>> +#define PGT_count_base    PG_shift(4)
>> +#define PGT_count_mask    PG_mask(7, 4)
>> +
>> +/*
>> + * Stored in bits [27:0] or [59:0] GFN if page is used for grant table frame.
> I don't know enough Arm details to tell whether this is properly
> one bit more than the maximum number of physical address bits.
> Without the extra bit you wouldn't be able to tell apart a
> guest specified GFN matching the value of PGT_INVALID_FRAME_GFN
> from an entry which was set from INVALID_GFN.
Really good point.

1. On Arm64 the p2m_ipa_bits could (theoretically) be 64-bit which, I 
assume, corresponds to the maximum guest physical address (1 << 64) - 1 
= 0xFFFFFFFFFFFFFFFF.
To store that GFN we need 52-bit. But, we provide 60-bit field which is 
more than enough, I think. Practically, the maximum supported 
p2m_ipa_bits is 48-bit, so the maximum supported GFN will occupy 36-bit 
only. Everything is ok here.
2. On Arm32 the p2m_ipa_bits is 40-bit which, I assume, corresponds to 
the maximum guest physical address (1 << 40) - 1 = 0xFFFFFFFFFF. To 
store that GFN we need 28-bit. If I did the calculation correctly, what 
we have on Arm32 is that PGT_INVALID_FRAME_GFN == maximum guest physical 
address and it looks like we need and extra bit on Arm32. Do you think 
we need to borrow one more bit from the count portion to stay on the 
safe side?


>
>> + * This only valid for the xenheap pages.
>> + */
>> +#define PGT_gfn_width     PG_shift(4)
>> +#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
> Any reason you don't use PG_mask() here? Any open-coding is prone
> to people later making mistakes.
I failed to come up with idea how to do that without #ifdef. As GFN 
starts at bit 0 different first parameter would be needed for PG_mask on 
32-bit and 64-bit systems.
I wonder whether PGC_count_mask/PGT_count_mask are open-coded on Arm/x86 
because of the same reason.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Wed Sep 15 23:51:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 15 Sep 2021 23:51:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187954.336980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQeg7-0001QO-Kc; Wed, 15 Sep 2021 23:50:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187954.336980; Wed, 15 Sep 2021 23:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQeg7-0001QH-HP; Wed, 15 Sep 2021 23:50:59 +0000
Received: by outflank-mailman (input) for mailman id 187954;
 Wed, 15 Sep 2021 23:50:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5PeA=OF=quicinc.com=quic_tsoni@srs-us1.protection.inumbo.net>)
 id 1mQeg5-0001QB-FV
 for xen-devel@lists.xen.org; Wed, 15 Sep 2021 23:50:57 +0000
Received: from alexa-out-sd-02.qualcomm.com (unknown [199.106.114.39])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c461b078-167f-11ec-b595-12813bfff9fa;
 Wed, 15 Sep 2021 23:50:56 +0000 (UTC)
Received: from unknown (HELO ironmsg02-sd.qualcomm.com) ([10.53.140.142])
 by alexa-out-sd-02.qualcomm.com with ESMTP; 15 Sep 2021 16:50:54 -0700
Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231])
 by ironmsg02-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 15 Sep 2021 16:50:54 -0700
Received: from [10.110.125.171] (10.80.80.8) by nasanex01a.na.qualcomm.com
 (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.922.7; Wed, 15 Sep 2021
 16:50:53 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c461b078-167f-11ec-b595-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim;
  t=1631749856; x=1663285856;
  h=subject:to:cc:references:from:message-id:date:
   mime-version:in-reply-to:content-transfer-encoding;
  bh=nrjlT73mofS7TuiPy02Gz2RmhHULOoSnTXJpfG/dMiU=;
  b=d+e8mea5pVmYRwo9neWo24MI/tadejENpXTBabQRT22QjxBd7X6RIRoS
   FSTdTiDq0H/ha3TCAq2idIR3BUcF/nu9NwY5bnNOcyNx8gVgiJnbw8yh8
   Dq+8gTdpvnVmf9PopkDmdIyynxLLnFRk/C7DpUsoqZQEw9KR76w9uu8Gx
   E=;
X-QCInternal: smtphost
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO backends
To: Stefano Stabellini <stefano.stabellini@xilinx.com>
CC: AKASHI Takahiro <takahiro.akashi@linaro.org>, <paul@xen.org>, "Stratos
 Mailing List" <stratos-dev@op-lists.linaro.org>,
	<virtio-dev@lists.oasis-open.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jan Kiszka <jan.kiszka@siemens.com>, Arnd Bergmann
	<arnd.bergmann@linaro.org>, <jgross@suse.com>, <julien@xen.org>, "Carl van
 Schaik" <cvanscha@qti.qualcomm.com>, <Bertrand.Marquis@arm.com>, "Stefan
 Hajnoczi" <stefanha@redhat.com>, <Artem_Mygaiev@epam.com>,
	<xen-devel@lists.xen.org>, <olekstysh@gmail.com>,
	<Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>
References: <87v94ldrqq.fsf@linaro.org>
 <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s>
 <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org>
 <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org>
 <20210906022356.GD40187@laputa>
 <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s>
 <e9df2014-b691-1ee4-7fcd-03912d0aed59@quicinc.com>
 <alpine.DEB.2.21.2109141127500.21985@sstabellini-ThinkPad-T480s>
From: Trilok Soni <quic_tsoni@quicinc.com>
Message-ID: <1fcd68fc-eace-2501-cc0d-4fa3ea96da75@quicinc.com>
Date: Wed, 15 Sep 2021 16:50:52 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109141127500.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.80.80.8]
X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To
 nasanex01a.na.qualcomm.com (10.52.223.231)

Hi Stefano,

On 9/14/2021 8:29 PM, Stefano Stabellini wrote:
> On Tue, 14 Sep 2021, Trilok Soni wrote:
>> On 9/13/2021 4:51 PM, Stefano Stabellini via Stratos-dev wrote:
>>> On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
>>>>> the second is how many context switches are involved in a transaction.
>>>>> Of course with all things there is a trade off. Things involving the
>>>>> very tightest latency would probably opt for a bare metal backend which
>>>>> I think would imply hypervisor knowledge in the backend binary.
>>>>
>>>> In configuration phase of virtio device, the latency won't be a big
>>>> matter.
>>>> In device operations (i.e. read/write to block devices), if we can
>>>> resolve 'mmap' issue, as Oleksandr is proposing right now, the only issue
>>>> is
>>>> how efficiently we can deliver notification to the opposite side. Right?
>>>> And this is a very common problem whatever approach we would take.
>>>>
>>>> Anyhow, if we do care the latency in my approach, most of virtio-proxy-
>>>> related code can be re-implemented just as a stub (or shim?) library
>>>> since the protocols are defined as RPCs.
>>>> In this case, however, we would lose the benefit of providing "single
>>>> binary"
>>>> BE.
>>>> (I know this is is an arguable requirement, though.)
>>>
>>> In my experience, latency, performance, and security are far more
>>> important than providing a single binary.
>>>
>>> In my opinion, we should optimize for the best performance and security,
>>> then be practical on the topic of hypervisor agnosticism. For instance,
>>> a shared source with a small hypervisor-specific component, with one
>>> implementation of the small component for each hypervisor, would provide
>>> a good enough hypervisor abstraction. It is good to be hypervisor
>>> agnostic, but I wouldn't go extra lengths to have a single binary. I
>>> cannot picture a case where a BE binary needs to be moved between
>>> different hypervisors and a recompilation is impossible (BE, not FE).
>>> Instead, I can definitely imagine detailed requirements on IRQ latency
>>> having to be lower than 10us or bandwidth higher than 500 MB/sec.
>>>
>>> Instead of virtio-proxy, my suggestion is to work together on a common
>>> project and common source with others interested in the same problem.
>>>
>>> I would pick something like kvmtool as a basis. It doesn't have to be
>>> kvmtools, and kvmtools specifically is GPL-licensed, which is
>>> unfortunate because it would help if the license was BSD-style for ease
>>> of integration with Zephyr and other RTOSes.
>>>
>>> As long as the project is open to working together on multiple
>>> hypervisors and deployment models then it is fine. For instance, the
>>> shared source could be based on OpenAMP kvmtool [1] (the original
>>> kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
>>> kvmtool was created to add support for hypervisor-less virtio but they
>>> are very open to hypervisors too. It could be a good place to add a Xen
>>> implementation, a KVM fatqueue implementation, a Jailhouse
>>> implementation, etc. -- work together toward the common goal of a single
>>> BE source (not binary) supporting multiple different deployment models.
>>
>> I have my reservations on using "kvmtool" to do any development here.
>> "kvmtool" can't be used on the products and it is just a tool for the
>> developers.
>>
>> The benefit of the solving problem w/ rust-vmm is that some of the crates from
>> this project can be utilized for the real product. Alex has mentioned that
>> "rust-vmm" today has some KVM specific bits but the rust-vmm community is
>> already discussing to remove or re-org them in such a way that other
>> Hypervisors can fit in.
>>
>> Microsoft has Hyper-V implementation w/ cloud-hypervisor which uses some of
>> the rust-vmm components as well and they had shown interest to add the Hyper-V
>> support in the "rust-vmm" project as well. I don't know the current progress
>> but they had proven it it "cloud-hypervisor" project.
>>
>> "rust-vmm" project's license will work as well for most of the project
>> developments and I see that "CrosVM" is shipping in the products as well.
> 
> Most things in open source start as a developers tool before they become
> part of a product :)

Agree, but I had an offline discussions with one the active developer of 
kvmtool and the confidence of using it in the product was no where near 
we expected during our evaluation. Same goes the QEMU and one of the 
biggest problem was no. of security issues against this huge codebase of 
QEMU.

> 
> I am concerned about how "embeddable" rust-vmm is going to be. Do you
> think it would be possible to run it against an RTOS together with other
> apps written in C?

I don't see any limitations of rust-vmm. For example, I am confident 
that we can port rust-vmm based backend into the QNX as host OS and same 
goes w/ Zephyr as well. Some work is needed but nothing fundamentally 
blocking it. We should be able to run it w/ Fuchsia as well with some 
effort.

---Trilok Soni


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 00:17:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 00:17:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187962.336990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQf5F-0004fK-6w; Thu, 16 Sep 2021 00:16:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187962.336990; Thu, 16 Sep 2021 00:16:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQf5F-0004fD-4A; Thu, 16 Sep 2021 00:16:57 +0000
Received: by outflank-mailman (input) for mailman id 187962;
 Thu, 16 Sep 2021 00:16:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQf5D-0004f7-LL
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 00:16:55 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3c9481d1-ad2c-441b-8808-02056be20422;
 Thu, 16 Sep 2021 00:16:54 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9C0D060EB2;
 Thu, 16 Sep 2021 00:16:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c9481d1-ad2c-441b-8808-02056be20422
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631751413;
	bh=onGWlFEk9oyIq3lK1eeV4qxYMy2d6NV2RY/n99pE2x8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VlxQgxuENrFMz6nl/au1A+RMahgcwbqcrwtg+XPbybcQIoP3EfCR5yEcpnPyB73uP
	 SYFYkZDBpqlvRJExytG5KDoI6p9D7kry+5HLLQ6u7wmkcG89biRmK1aMjYRh/gckXe
	 yIbgWVIA/IQ0rrAnV+KtE0hY3sJDi+pVvINKpjCo/a5GYHrL4AVSqXXotTO14NfGbI
	 t2QBCUwFHm2zBkbwgizYO2BfjxXw5grmbG5vV1KgpdBjVpyRDkmjQc9ss+fbxwOUJt
	 w/BdlxudpXg/2NRUorJw/IIRiisTHICfVBwzo9vvXYrnCEDQpx54iyq3wDFRY9IWp2
	 YrgVmihkscnrA==
Date: Wed, 15 Sep 2021 17:16:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, jbeulich@suse.com
Subject: Re: [PATCH 1/2] xen/efi: Restrict check for DT boot modules on EFI
 boot
In-Reply-To: <20210915142602.42862-2-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109151445080.21985@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-2-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Adding Jan for an opinion on the EFI common code changes. See below.


On Wed, 15 Sep 2021, Luca Fancellu wrote:
> When Xen is started as EFI application, it is checking
> the presence of multiboot,module in the DT, if any is
> found, the configuration file is skipped.
> Restrict this check to just any multiboot,module that
> is a direct child of the /chosen node.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  xen/arch/arm/efi/efi-boot.h | 30 ++++++++++++++++++++++++++++--
>  1 file changed, 28 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..5ff626c6a0 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -581,6 +581,8 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
>  
>  static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
>  {
> +    int node;
> +    bool dom0_module_found = false;
>      /*
>       * For arm, we may get a device tree from GRUB (or other bootloader)
>       * that contains modules that have already been loaded into memory.  In
> @@ -592,11 +594,35 @@ static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
>      fdt = lookup_fdt_config_table(SystemTable);
>      dtbfile.ptr = fdt;
>      dtbfile.need_to_free = false; /* Config table memory can't be freed. */
> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
> +
> +    /* Locate chosen node */
> +    node = fdt_subnode_offset(fdt, 0, "chosen");
> +
> +    /* Cycle through every node inside chosen having multiboot,module */
> +    do {
> +        int depth = 0;
> +        node = fdt_node_offset_by_compatible(fdt, node, "multiboot,module");
> +        /*
> +         * If the multiboot,module just found is placed at depth less than 3,
> +         * it means that it is here: /chosen/<module> so it is a module to
> +         * start dom0. (root is counted as 0)
> +         */
> +        if ( node > 0 )
> +        {
> +            depth = fdt_node_depth(fdt, node);
> +            if ( (depth >= 0) && (depth < 3) )
> +            {
> +                dom0_module_found = true;
> +                break;
> +            }
> +        }
> +    } while(node > 0);

It should be possible to enable the uefi,binary bootflow for Dom0 and
the Dom0 ramdisk too. It would be nice as we could have a 100% UEFI
boot, not dependent on U-Boot, both Dom0 and Dom0less, without the
xen.cfg file. It doesn't have to be done now by this series, but it
should be possible from a device tree bindings perspective.

With that in mind, is this check accurate? This patch is saying that if
Dom0 is not present in the device tree, then load xen.cfg. But what if
it is a true dom0less configuration? Then we would have no dom0, only
dom0less VMs, and we might still not want to load xen.cfg. True dom0less
is another one of those configurations that don't have to be enabled now
by this series but they should be possible from a device tree bindings
perspective.


I tried to think of ways to improve this check, for instance searching
for a module that doesn't have "uefi,binary" but has the regular "reg"
property. If there is one, then we could skip loading xen.cfg. But that
doesn't work if we have a UEFI-only true dom0less configuration.

So I am thinking that we have no choice but introducing a new property
to tell us whether we should or should not load xen.cfg when
multiboot,modules are present.

Taking inspiration from HyperLaunch, it could be a new node:

chosen {
    cfg {
        compatible = "xen,uefi-config", "multiboot,module";
        uefi,binary = "xen.cfg";
    };
};

In efi_arch_use_config_file we would check if there are any nodes
compatible with "multiboot,module". If there are none, it returns true.

If there are any, and one of them is also compatible "xen,uefi-config",
then efi_arch_use_config_file returns true and also the specified
configuration filename.

If there are nodes compatible to "multiboot,module" but none of them is
compatible to "module,uefi-config", then efi_arch_use_config_file
returns false. We use the device tree only.

I think that would be clearer and more consistent from a specification
perspective, but it requires one change in common code:
efi_arch_use_config_file would not just return bool but it would also
return a filename if found (it could be a pointer parameter to the
function).


Otherwise, we could add a simple property like the following, without a
specific value and without a filename:

chosen {
    xen,uefi-config;
};

The presence of xen,uefi-config could mean that Xen should go through
the usual guessing game to figure out the right cfg file to load. This
would not require any common code changes because
efi_arch_use_config_file could simply return bool as it does today.

My preference is the xen,uefi-config compatible node, but I think the
property would also work.


Jan, do you have an opinion on whether efi_arch_use_config_file has to
stay as it is today, or would you be open to the possibility of making
efi_arch_use_config_file return a filename too?


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 01:16:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 01:16:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187971.337002 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQg0X-0000yv-PK; Thu, 16 Sep 2021 01:16:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187971.337002; Thu, 16 Sep 2021 01:16:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQg0X-0000yo-M4; Thu, 16 Sep 2021 01:16:09 +0000
Received: by outflank-mailman (input) for mailman id 187971;
 Thu, 16 Sep 2021 01:16:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQg0W-0000yi-W8
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 01:16:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 22757346-043b-4e8c-845b-f65faaf7fbb6;
 Thu, 16 Sep 2021 01:16:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id DDFD360234;
 Thu, 16 Sep 2021 01:16:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22757346-043b-4e8c-845b-f65faaf7fbb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631754966;
	bh=iU5N8pj2WLSe0NTsK9lB/DhZIamg7KrxUhUpoX6quNw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=t8ld2PePCCSq3QkC6WdBOnLts4htGIOHonS2tRkRiHPsRy5Wg9/StUbz0LyQqOtTJ
	 XvTvyE1ISOp3IDX+8WOcgxboQtPdsAJuPvfV6fHtfylYmsUFPNssuXVHtYRIaRDo4P
	 ghC3yFMuMJcNL/iKPU2jjHRXbOfesoRYu5obfSfCs2hqwelyXQBqe3S/sLLfUnX7mH
	 EBtevkKU81H0C0i8m1YhRDmVT8y16D0C/DK7ooE73jusg77BM+NG5hcI2Ove3EYXfF
	 LBIUNgH9kSXnzigXWCGpEDY2fdn+gfFYlKHhEZ8fGrV+i9WIxhFz03+oRhxLBpJtH0
	 k4/gQ3yhl75rw==
Date: Wed, 15 Sep 2021 18:16:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
In-Reply-To: <20210915142602.42862-3-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-3-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 15 Sep 2021, Luca Fancellu wrote:
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "uefi,binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
>  docs/misc/efi.pandoc        |  37 ++++++
>  xen/arch/arm/efi/efi-boot.h | 244 +++++++++++++++++++++++++++++++++++-
>  xen/common/efi/boot.c       |  20 ++-
>  3 files changed, 294 insertions(+), 7 deletions(-)
> 
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ac3cd58cae..db9b3273f8 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -165,3 +165,40 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "uefi,binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +    domU1 {
> +        #address-cells = <1>;
> +        #size-cells = <1>;
> +        compatible = "xen,domain";
> +        memory = <0 0x20000>;
> +        cpus = <1>;
> +        vpl011;
> +
> +        module@1 {
> +            compatible = "multiboot,kernel", "multiboot,module";
> +            uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> +            bootargs = "console=ttyAMA0";
> +        };
> +        module@2 {
> +            compatible = "multiboot,ramdisk", "multiboot,module";
> +            uefi,binary = "initrd-3.0.31-0.4-xen";
> +        };
> +        module@3 {
> +            compatible = "multiboot,ramdisk", "multiboot,module";
> +            uefi,binary = "passthrough.dtb";
> +        };
> +    };

Can you please also update docs/misc/arm/device-tree/booting.txt ?
Either a link to docs/misc/efi.pandoc or a definition of the uefi,binary
property (mentioning that it is EFI-only.)


> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 5ff626c6a0..8d7ced70f2 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,39 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char* name;
> +    int name_len;
> +} dom0less_module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
> +static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MODULES];

I suggest a slightly different model where we don't call AllocatePool to
allocate dom0less_module_name.name and instead we just set the pointer
directly to the fdt string. There is no risk of the fdt going away at
this point so it should be safe to use.

Also, I don't think we need a global array of struct file, we only
really need 1 struct file which would be freed immediately after loading
to memory. We do need to remember the address and size in memory though.
So I would do something like:

typedef struct {
    const char* name;
    int name_len;
    EFI_PHYSICAL_ADDRESS addr;
    UINTN size;
} dom0less_module_name;

/*
 * Binaries will be translated into bootmodules, the maximum number for them is
 * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
 */
#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MODULES];


The purpose is to reduce memory allocations and memory consumption.


> +static uint32_t __initdata dom0less_modules_available = MAX_DOM0LESS_MODULES;
> +static uint32_t __initdata dom0less_modules_idx = 0;
> +
> +#define ERROR_DOM0LESS_FILE_NOT_FOUND -1
> +
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int __init get_dom0less_file_index(const char* name, int name_len);
> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                              const char* name, int name_len);
> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
> +                                               int module_node_offset,
> +                                               int reg_addr_cells,
> +                                               int reg_size_cells);
> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                               int domain_node,
> +                                               int addr_cells,
> +                                               int size_cells);
> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);
> +
>  #define DEVICE_TREE_GUID \
>  {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
>  
> @@ -552,8 +582,209 @@ static void __init efi_arch_handle_module(const struct file *file,
>                           kernel.size) < 0 )
>              blexit(L"Unable to set reg property.");
>      }
> -    else
> +    else if ( !((file >= &dom0less_files[0]) &&
> +               (file <= &dom0less_files[MAX_DOM0LESS_MODULES-1])) )
> +        /*
> +         * If file is not a dom0 module file and it's not any domU modules,
> +         * stop here.
> +         */
>          blexit(L"Unknown module type");

Without &dom0less_files we would have to do without this sanity check.


> +    /*
> +     * dom0less_modules_available is decremented here because for each dom0
> +     * file added, there will be an additional bootmodule, so the number
> +     * of dom0less module files will be decremented because there is
> +     * a maximum amount of bootmodules that can be loaded.
> +     */
> +    dom0less_modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the dom0less_files array or a negative
> + * number if no file with that name is found.
> + */
> +static int __init get_dom0less_file_index(const char* name, int name_len)
> +{
> +    int ret = ERROR_DOM0LESS_FILE_NOT_FOUND;
> +
> +    for (uint32_t i = 0; i < dom0less_modules_idx; i++)

uint32_t i;

for ( i = 0; i < dom0less_modules_idx; i++ )


> +    {
> +        dom0less_module_name* mod = &dom0less_bin_names[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it
> + * returns the index of the file in the dom0less_files array.
> + */
> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                              const char* name, int name_len)
> +{
> +    dom0less_module_name* file_name;
> +    union string module_name;
> +    struct file* file;
> +    uint32_t ret_idx;
> +
> +    /*
> +     * Check if there is any space left for a domU module, the variable
> +     * dom0less_modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !dom0less_modules_available )
> +        blexit(L"No space left for domU modules");
> +    module_name.s = (char*) name;
> +    ret_idx = dom0less_modules_idx;
> +    file = &dom0less_files[ret_idx];
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &dom0less_bin_names[ret_idx];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +        blexit(L"Error allocating memory for dom0less binary name");

As far as I can tell we could just set file_name = name;


> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len);
> +    file_name->name_len = name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), file, NULL);
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    dom0less_modules_idx++;
> +
> +    return ret_idx;
> +}
> +
> +/*
> + * This function checks for the presence of the uefi,binary property in the
> + * module, if found it loads the binary as dom0less module and sets the right
> + * address for the reg property into the module DT node.
> + */
> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
> +                                          int module_node_offset,
> +                                          int reg_addr_cells,
> +                                          int reg_size_cells)
> +{
> +    const void* uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx;
> +    struct file* file;
> +
> +    /* Read uefi,binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
> +                                 &uefi_name_len);
> +
> +    if ( NULL == uefi_name_prop )
> +        /* Property not found */
> +        return;
> +
> +    file_idx = get_dom0less_file_index(uefi_name_prop, uefi_name_len);
> +    if (file_idx < 0)
> +        file_idx = allocate_dom0less_file(dir_handle, uefi_name_prop,
> +                                          uefi_name_len);
> +
> +    file = &dom0less_files[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +        blexit(L"Unable to add domU ramdisk FDT node.");
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +        blexit(L"Unable to set reg property.");
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + */
> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                               int domain_node,
> +                                               int addr_cells,
> +                                               int size_cells)
> +{
> +    /*
> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
> +     * inside this node
> +     */
> +    for ( int module_node = fdt_first_subnode(fdt, domain_node);

int module_node;

for ( module_node = fdt_first_subnode(fdt, domain_node);


> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +    {
> +        if ( (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,kernel") == 0) ||
> +             (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,ramdisk") == 0) ||
> +             (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,device-tree") == 0) )
> +        {
> +            /* The compatible is one of the strings above, check the module */
> +            handle_dom0less_module_node(dir_handle, module_node, addr_cells,
> +                                        size_cells);
> +        }
> +    }
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + */
> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen;
> +    int addr_len, size_len;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +        blexit(L"Unable to setup chosen node");
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( int node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        int addr_cells, size_cells, len;
> +        const struct fdt_property *prop;
> +
> +        if ( fdt_node_check_compatible(fdt, node, "xen,domain") != 0 )
> +            continue;
> +
> +        /* Get or set #address-cells and #size-cells */
> +        prop = fdt_get_property(fdt, node, "#address-cells", &len);
> +        if ( !prop )
> +            blexit(L"#address-cells not found in domain node.");
> +
> +        addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +        prop = fdt_get_property(fdt, node, "#size-cells", &len);
> +        if ( !prop )
> +            blexit(L"#size-cells not found in domain node.");
> +
> +        size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +        /* Found a node with compatible xen,domain; handle this node. */
> +        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_cells);
> +    }
> +
> +    if ( dom0less_modules_idx > 0 )
> +        return true;
> +
> +    return false;
>  }
>  
>  static void __init efi_arch_cpu(void)
> @@ -562,8 +793,19 @@ static void __init efi_arch_cpu(void)
>  
>  static void __init efi_arch_blexit(void)
>  {
> +    uint32_t i = 0;
>      if ( dtbfile.need_to_free )
>          efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free dom0less files if any */
> +    for ( ; i < dom0less_modules_idx; i++ )
> +    {
> +        /* Free dom0less binary names */
> +        efi_bs->FreePool(dom0less_bin_names[i].name);
> +        /* Free dom0less binaries */
> +        if ( dom0less_files[i].need_to_free )
> +            efi_bs->FreePages(dom0less_files[i].addr,
> +                              PFN_UP(dom0less_files[i].size));
> +    }
>      if ( memmap )
>          efi_bs->FreePool(memmap);
>  }
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..65493c4b46 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    bool dom0less_found = false;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +#ifdef CONFIG_ARM
> +        /* dom0less feature is supported only on ARM */
> +        dom0less_found = check_dom0less_efi_boot(dir_handle);
> +#endif

Rather than an #ifdef here you can simply implement
check_dom0less_efi_boot on x86 as a static inline returning always
false.

Also, we are under the if ( use_cfg_file ) code path. So maybe it is
reasonable that dom0 is required if we are booting with use_cfg_file
= true. After all, it is specified as a required property today of
xen.cfg.

If you follow my suggestion with an explicit enabled/disabled of xen.cfg
from device tree, a true dom0less configuration could be fully specified
without xen.cfg.

If we do that, then here probable we don't need to change this code path.



> +        if ( !name.s && !dom0less_found )
> +            blexit(L"No Dom0 kernel image specified.");
> +
> +        if ( name.s != NULL )
> +            option_str = split_string(name.s);
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_str)) &&
> +             (name.s != NULL) )
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 02:11:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 02:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187980.337012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQgsL-0007t8-VI; Thu, 16 Sep 2021 02:11:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187980.337012; Thu, 16 Sep 2021 02:11:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQgsL-0007t1-Rk; Thu, 16 Sep 2021 02:11:45 +0000
Received: by outflank-mailman (input) for mailman id 187980;
 Thu, 16 Sep 2021 02:11:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=54Q8=OG=xilinx.com=stefanos@srs-us1.protection.inumbo.net>)
 id 1mQgsK-0007sv-8i
 for xen-devel@lists.xen.org; Thu, 16 Sep 2021 02:11:44 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (unknown
 [40.107.236.77]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed91bf96-5965-49c7-84be-7b64f29a52e3;
 Thu, 16 Sep 2021 02:11:42 +0000 (UTC)
Received: from BN0PR04CA0113.namprd04.prod.outlook.com (2603:10b6:408:ec::28)
 by BL3PR02MB7875.namprd02.prod.outlook.com (2603:10b6:208:33e::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 02:11:39 +0000
Received: from BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
 (2603:10b6:408:ec:cafe::23) by BN0PR04CA0113.outlook.office365.com
 (2603:10b6:408:ec::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 02:11:40 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by
 BN1NAM02FT024.mail.protection.outlook.com (10.13.2.138) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 02:11:38 +0000
Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2176.14; Wed, 15 Sep 2021 19:11:13 -0700
Received: from smtp.xilinx.com (172.19.127.95) by
 xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id
 15.1.2176.14 via Frontend Transport; Wed, 15 Sep 2021 19:11:13 -0700
Received: from [10.23.120.123] (port=58151 helo=localhost)
 by smtp.xilinx.com with esmtp (Exim 4.90)
 (envelope-from <stefano.stabellini@xilinx.com>)
 id 1mQgrp-000El2-Hq; Wed, 15 Sep 2021 19:11:13 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed91bf96-5965-49c7-84be-7b64f29a52e3
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BwIwyJEUx4bglYFyGgh4jAIDuTeqs3456XEUeenA0O6Fh6sDjxT/kiZ9Xd0nwX5JwQCYMCOZkPD0bIlu19CQGSbke3lo3kUZeSXVNuaG1M72jlrj83Wlouc8+hVTZCxLq0Dem/R0OmPxf6hddBJIJK62js3WSWQ4Sy1796w5NzwHcnxelb+A8MAb3MefxpWTqo5+TRr/JzX6T+aYtyxd3PS07wjYk851LRaG0ZUNin6FDOrGVB5ObpW47ClUqj6sCbRqb2OTDxRVAIpO1z/LEmnzU0gPPA8f/v0pekU9co0GRyDpR2xL410pKSe4G9urz+LEIq5S7jgczCtDCaR8QQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MU9RjuCzOWHaLkh+VZKJRtIQ/bdhrC8FhXuQTF7/8Uc=;
 b=eP8lzPeHDIJsTAY7t/2Z1v9XWqxFQzR3Ghs4Q32HJElR1rnAxmTZ77p7AT2r0lOfGfKrFe63rcaMuOEOWeRmG6NAdLqiYdAznq8DxAs16FuM1w25m/XUDMQhmvyz7rSxwj/5R0dZYT7djUp0tGrMqIcGG8Ji837D/s1RSAe69GXzwJWvm3hg5cBJWAJtKHr4k9wb8LuG98a1SklLd43cCPVsJaiFTwbvgBBGZVrfEEyhw1Vx8eHFcvozhCve7VpAOqWPOFL79+7J8Y+8JzMdIbh3qdnSE2WV3EJuLlyQYsTTphk29dPu7H1wtjoXgeRYBwn9K6fUVhFaahgn+TOLUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 149.199.62.198) smtp.rcpttodomain=quicinc.com smtp.mailfrom=xilinx.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MU9RjuCzOWHaLkh+VZKJRtIQ/bdhrC8FhXuQTF7/8Uc=;
 b=YuZo0KXYM7NaZtBQGJBq3nVZiaR6Y41SNI4c9rxcbAyTKHaODvPJr/oiAiGZcxKhGu7SnxQq/SWtYgpwoz2rFQQJ+JGTyVyYY5JXcW7ycZuzDmOg75G0EH1wyHJDQgyeQr6WyESscZ+aP3gHUQ5HCIt8VdeTz+uepwWY921mtrY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198)
 smtp.mailfrom=xilinx.com; quicinc.com; dkim=none (message not signed)
 header.d=none;quicinc.com; dmarc=pass action=none header.from=xilinx.com;
Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates
 149.199.62.198 as permitted sender) receiver=protection.outlook.com;
 client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com;
Date: Wed, 15 Sep 2021 19:11:12 -0700
From: Stefano Stabellini <stefano.stabellini@xilinx.com>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Trilok Soni <quic_tsoni@quicinc.com>
CC: Stefano Stabellini <stefano.stabellini@xilinx.com>, AKASHI Takahiro
	<takahiro.akashi@linaro.org>, <paul@xen.org>, Stratos Mailing List
	<stratos-dev@op-lists.linaro.org>, <virtio-dev@lists.oasis-open.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Jan Kiszka <jan.kiszka@siemens.com>,
	Arnd Bergmann <arnd.bergmann@linaro.org>, <jgross@suse.com>,
	<julien@xen.org>, Carl van Schaik <cvanscha@qti.qualcomm.com>,
	<Bertrand.Marquis@arm.com>, Stefan Hajnoczi <stefanha@redhat.com>,
	<Artem_Mygaiev@epam.com>, <xen-devel@lists.xen.org>, <olekstysh@gmail.com>,
	<Oleksandr_Tyshchenko@epam.com>, Elena Afanasova <eafanasova@gmail.com>
Subject: Re: [Stratos-dev] Enabling hypervisor agnosticism for VirtIO
 backends
In-Reply-To: <1fcd68fc-eace-2501-cc0d-4fa3ea96da75@quicinc.com>
Message-ID: <alpine.DEB.2.21.2109151849310.21985@sstabellini-ThinkPad-T480s>
References: <87v94ldrqq.fsf@linaro.org> <alpine.DEB.2.21.2108041055390.9768@sstabellini-ThinkPad-T480s> <YRuSPT9075NuWRYS@stefanha-x1.localdomain> <875yvkh1p1.fsf@linaro.org> <20210903080609.GD47953@laputa> <87czpqq9qu.fsf@linaro.org> <20210906022356.GD40187@laputa>
 <alpine.DEB.2.21.2109131615570.10523@sstabellini-ThinkPad-T480s> <e9df2014-b691-1ee4-7fcd-03912d0aed59@quicinc.com> <alpine.DEB.2.21.2109141127500.21985@sstabellini-ThinkPad-T480s> <1fcd68fc-eace-2501-cc0d-4fa3ea96da75@quicinc.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 220a1056-f972-49a8-5d55-08d978b750fe
X-MS-TrafficTypeDiagnostic: BL3PR02MB7875:
X-Microsoft-Antispam-PRVS:
	<BL3PR02MB787587DEE6CD092C89326734A0DC9@BL3PR02MB7875.namprd02.prod.outlook.com>
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KGdYC7oY0QMDdi5TXsgtlj67x2zt03M2EyJzFTZpCFjSvXLCS2Z/k7BWEzi0jAevVwqiI3qdIVDu04O9yG2RIXJziOKngOjfZfTobQVc2AwPZhsCBqP9cf9Le0DeVpZgovFztIn2cWuVrLCo2KhCtM2FuSq8wlAlYxuGF+pT2Cs9rS64s3fk4gjukFf2AX0BJwvRvYy3q/zz1YKTbcc5bacMm1q7snXMIkQ1Fj43hhQrU+9tqi5aF+f+MAAPaXD7kUw+q8VECkgG3zX/5+99v+GKM/RUlGY8iyqHanu5UjgzQ5N0uZpvVZmVW6SV0h7OK+pxaxcupb/sKPI2PqM5x5iaGfh2Q019xd7GQ9vO6BdGkxY2/ddhTfvHabF5jBpP7idp2r1NPvjf79qdaeDK4xOK48VRwS75TJHUBc3/29cZ64WJRAekEXrv/Lxn5AU87vhV9WeLl7vN4vth6mPMqszw+797KbB43Bw6o17cooXtCmpImNoKndMQouY/jq8I7okuSurFuGIdOInktQcNWocSFIrM0Tk7vYTyx6EtJQteKxsMZLX5GMZz0n5mOfuEHIDBZ8T7bBxXwWbqMZ0Zye2s+gcWu/FhOi4AqIxG45JqHGoiYRdv2T36yCltbbPYzkE9i7GhMVSGGU6l1fMhkK5cncc2c0RtSRb96BdtInw1rIHSyV99YvOs7E7AuznTa2AkLUyVqfY6QZNZnycp/Q==
X-Forefront-Antispam-Report:
	CIP:149.199.62.198;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:xsj-pvapexch02.xlnx.xilinx.com;PTR:unknown-62-198.xilinx.com;CAT:NONE;SFS:(7916004)(4636009)(39860400002)(346002)(136003)(396003)(376002)(46966006)(36840700001)(4326008)(33716001)(54906003)(45080400002)(47076005)(82740400003)(8676002)(7636003)(36860700001)(2906002)(356005)(316002)(7416002)(8936002)(9686003)(82310400003)(336012)(5660300002)(26005)(186003)(478600001)(426003)(36906005)(70206006)(70586007)(9786002)(53546011)(6916009)(83380400001)(44832011);DIR:OUT;SFP:1101;
X-OriginatorOrg: xilinx.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 02:11:38.1502
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 220a1056-f972-49a8-5d55-08d978b750fe
X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.62.198];Helo=[xsj-pvapexch02.xlnx.xilinx.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1NAM02FT024.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR02MB7875

On Wed, 15 Sep 2021, Trilok Soni wrote:
> On 9/14/2021 8:29 PM, Stefano Stabellini wrote:
> > On Tue, 14 Sep 2021, Trilok Soni wrote:
> > > On 9/13/2021 4:51 PM, Stefano Stabellini via Stratos-dev wrote:
> > > > On Mon, 6 Sep 2021, AKASHI Takahiro wrote:
> > > > > > the second is how many context switches are involved in a
> > > > > > transaction.
> > > > > > Of course with all things there is a trade off. Things involving the
> > > > > > very tightest latency would probably opt for a bare metal backend
> > > > > > which
> > > > > > I think would imply hypervisor knowledge in the backend binary.
> > > > > 
> > > > > In configuration phase of virtio device, the latency won't be a big
> > > > > matter.
> > > > > In device operations (i.e. read/write to block devices), if we can
> > > > > resolve 'mmap' issue, as Oleksandr is proposing right now, the only
> > > > > issue
> > > > > is
> > > > > how efficiently we can deliver notification to the opposite side.
> > > > > Right?
> > > > > And this is a very common problem whatever approach we would take.
> > > > > 
> > > > > Anyhow, if we do care the latency in my approach, most of
> > > > > virtio-proxy-
> > > > > related code can be re-implemented just as a stub (or shim?) library
> > > > > since the protocols are defined as RPCs.
> > > > > In this case, however, we would lose the benefit of providing "single
> > > > > binary"
> > > > > BE.
> > > > > (I know this is is an arguable requirement, though.)
> > > > 
> > > > In my experience, latency, performance, and security are far more
> > > > important than providing a single binary.
> > > > 
> > > > In my opinion, we should optimize for the best performance and security,
> > > > then be practical on the topic of hypervisor agnosticism. For instance,
> > > > a shared source with a small hypervisor-specific component, with one
> > > > implementation of the small component for each hypervisor, would provide
> > > > a good enough hypervisor abstraction. It is good to be hypervisor
> > > > agnostic, but I wouldn't go extra lengths to have a single binary. I
> > > > cannot picture a case where a BE binary needs to be moved between
> > > > different hypervisors and a recompilation is impossible (BE, not FE).
> > > > Instead, I can definitely imagine detailed requirements on IRQ latency
> > > > having to be lower than 10us or bandwidth higher than 500 MB/sec.
> > > > 
> > > > Instead of virtio-proxy, my suggestion is to work together on a common
> > > > project and common source with others interested in the same problem.
> > > > 
> > > > I would pick something like kvmtool as a basis. It doesn't have to be
> > > > kvmtools, and kvmtools specifically is GPL-licensed, which is
> > > > unfortunate because it would help if the license was BSD-style for ease
> > > > of integration with Zephyr and other RTOSes.
> > > > 
> > > > As long as the project is open to working together on multiple
> > > > hypervisors and deployment models then it is fine. For instance, the
> > > > shared source could be based on OpenAMP kvmtool [1] (the original
> > > > kvmtool likely prefers to stay small and narrow-focused on KVM). OpenAMP
> > > > kvmtool was created to add support for hypervisor-less virtio but they
> > > > are very open to hypervisors too. It could be a good place to add a Xen
> > > > implementation, a KVM fatqueue implementation, a Jailhouse
> > > > implementation, etc. -- work together toward the common goal of a single
> > > > BE source (not binary) supporting multiple different deployment models.
> > > 
> > > I have my reservations on using "kvmtool" to do any development here.
> > > "kvmtool" can't be used on the products and it is just a tool for the
> > > developers.
> > > 
> > > The benefit of the solving problem w/ rust-vmm is that some of the crates
> > > from
> > > this project can be utilized for the real product. Alex has mentioned that
> > > "rust-vmm" today has some KVM specific bits but the rust-vmm community is
> > > already discussing to remove or re-org them in such a way that other
> > > Hypervisors can fit in.
> > > 
> > > Microsoft has Hyper-V implementation w/ cloud-hypervisor which uses some
> > > of
> > > the rust-vmm components as well and they had shown interest to add the
> > > Hyper-V
> > > support in the "rust-vmm" project as well. I don't know the current
> > > progress
> > > but they had proven it it "cloud-hypervisor" project.
> > > 
> > > "rust-vmm" project's license will work as well for most of the project
> > > developments and I see that "CrosVM" is shipping in the products as well.
> > 
> > Most things in open source start as a developers tool before they become
> > part of a product :)
> 
> Agree, but I had an offline discussions with one the active developer of
> kvmtool and the confidence of using it in the product was no where near we
> expected during our evaluation. Same goes the QEMU and one of the biggest
> problem was no. of security issues against this huge codebase of QEMU.

That is fair, but it is important to recognize that these are *known*
security issues.

Does rust-vmm have a security process and a security response team? I
tried googling for it but couldn't find relevant info.

QEMU is a very widely used and very well inspected codebase. It has a
mailing list to report security issues and a security process. As a
consequence we know of many vulnerabilities affecting the code base.
As far as I am aware rust-vmm has not been inspected yet with the same
level of attention and the same amount of security researchers.

That said, of course it is undeniable that the larger size of QEMU
implies a higher amount of security issues. But for this project, we
wouldn't be using the whole of QEMU of course. We would be narrowing it
down to a build with only few revelant pieces. I imagine that the total
LOC count would still be higher but the number of relevant security
vulnerabilities would only be a small fraction of the QEMU total.

 
> > I am concerned about how "embeddable" rust-vmm is going to be. Do you
> > think it would be possible to run it against an RTOS together with other
> > apps written in C?
> 
> I don't see any limitations of rust-vmm. For example, I am confident that we
> can port rust-vmm based backend into the QNX as host OS and same goes w/
> Zephyr as well. Some work is needed but nothing fundamentally blocking it. We
> should be able to run it w/ Fuchsia as well with some effort.
 
That's good to hear.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 03:14:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 03:14:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.187990.337024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQhqb-00069C-Kx; Thu, 16 Sep 2021 03:14:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 187990.337024; Thu, 16 Sep 2021 03:14:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQhqb-000695-H9; Thu, 16 Sep 2021 03:14:01 +0000
Received: by outflank-mailman (input) for mailman id 187990;
 Thu, 16 Sep 2021 03:13:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=e8Ul=OG=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mQhqZ-00068x-BO
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 03:13:59 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 20f22874-169c-11ec-b59d-12813bfff9fa;
 Thu, 16 Sep 2021 03:13:57 +0000 (UTC)
Received: from AM3PR05CA0153.eurprd05.prod.outlook.com (2603:10a6:207:3::31)
 by AM9PR08MB6308.eurprd08.prod.outlook.com (2603:10a6:20b:287::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 03:13:51 +0000
Received: from VE1EUR03FT054.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:207:3:cafe::8a) by AM3PR05CA0153.outlook.office365.com
 (2603:10a6:207:3::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 03:13:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT054.mail.protection.outlook.com (10.152.19.64) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 03:13:50 +0000
Received: ("Tessian outbound d5def7722ff7:v103");
 Thu, 16 Sep 2021 03:13:50 +0000
Received: from a560d6b9d116.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BC920EB3-F034-4C3B-AC9F-CE266FC54BED.1; 
 Thu, 16 Sep 2021 03:13:43 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a560d6b9d116.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 03:13:43 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com (2603:10a6:803:10a::33)
 by VI1PR08MB3406.eurprd08.prod.outlook.com (2603:10a6:803:7b::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 03:13:40 +0000
Received: from VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c]) by VE1PR08MB5215.eurprd08.prod.outlook.com
 ([fe80::adf8:8d6f:2a30:b60c%5]) with mapi id 15.20.4500.019; Thu, 16 Sep 2021
 03:13:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20f22874-169c-11ec-b59d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1QavK+Rws1oseC5hjcMFJw17o0Ikj5T9DQ0fG5DM6Lk=;
 b=ARSK5q12OMEQPvfOJB6qY0JIqlG4SG9eZUMlpnmtGpzkM3GCNIV624DKF/S43OTaCJb4NC3yeKJxoVUhgzU0G3uf9JXD0g9AjBv2S9W1qNTrGZgbLvc5yrQpEeEEx1e5qyV4D80CwZm/h2cInr05poYQb+HUih4kYifykvy2ric=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hCH0o4JD9xCUpK/FuVtm4izNKwnyauvDZeH6sybgR0z8W9Y8s575Pt24j7R8GQEWpL5RzSySNxjH5um0ep/0w8QQBpsHOY2UkR7UCmo8rUTiFbMp16p5bp1AiTTwJmaxHagoFXIGO5FHt4wBRQyDcdcACZhRepzB6piB8qS3OhVSD8GZ9D+n4x3Oog5mrZrgua0+j6QxN6CORSsGx9GN2CBOYpVM+JOouTYgPnQPBb+WAsomU0D9SorZ96djukzWX3myoEsmMTpICj7v3OkgFvAG76u1lMVhfQgfiAIYd2Md+Jd8j0c6bG8kYvXLmkYIjAFrtWd7AkqMxHNxWGCIQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1QavK+Rws1oseC5hjcMFJw17o0Ikj5T9DQ0fG5DM6Lk=;
 b=ENH/cCDMxaafveVA3gYidBoUS59PrFbFyCwIIQJIMLf82+QB0b4aMc63GbTQRZWLXYhgJhPfQZatO7/lWeBk34oHSmzdjAJ4xrwqQeIxxT0ki8byWFrTzzUnbHR3eJCKP2EHeg0qUyT8IczX0UMkwreHdwuYET7OeiNzYiR8I1gBkMxxTXnpZlU3as3ggaPtD/vmcHgNDTcoTVbyJnsEgx/Pa0LtaL6Ke0ypU0kFlBDnyJI/U94iNRGVwflVfk9pjV0x5zoXXJJWBAwSzz0bOC03pD8Wq+UjmvE4mVHt0LDAclvf+LM9WWvctgNJz17mqcQQgZZijd1hQ5uYH2YaZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1QavK+Rws1oseC5hjcMFJw17o0Ikj5T9DQ0fG5DM6Lk=;
 b=ARSK5q12OMEQPvfOJB6qY0JIqlG4SG9eZUMlpnmtGpzkM3GCNIV624DKF/S43OTaCJb4NC3yeKJxoVUhgzU0G3uf9JXD0g9AjBv2S9W1qNTrGZgbLvc5yrQpEeEEx1e5qyV4D80CwZm/h2cInr05poYQb+HUih4kYifykvy2ric=
From: Penny Zheng <Penny.Zheng@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>, "julien@xen.org"
	<julien@xen.org>
Subject: RE: [PATCH v7 3/7] xen: introduce mark_page_free
Thread-Topic: [PATCH v7 3/7] xen: introduce mark_page_free
Thread-Index: AQHXqjl0kD+LuGbedUe7fBE0DC0ydaul+Iqg
Date: Thu, 16 Sep 2021 03:13:40 +0000
Message-ID:
 <VE1PR08MB5215CABEFC383EFF0DB372FAF7DC9@VE1PR08MB5215.eurprd08.prod.outlook.com>
References: <20210910025215.1671073-1-penny.zheng@arm.com>
 <20210910025215.1671073-4-penny.zheng@arm.com>
 <cd1ba586-0613-8e12-97b7-7b4cc2577bb9@suse.com>
In-Reply-To: <cd1ba586-0613-8e12-97b7-7b4cc2577bb9@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 61CEAD76E49D694BB20DA70866E1A65E.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 546cd9cb-63a9-4a86-bfcf-08d978c001b7
x-ms-traffictypediagnostic: VI1PR08MB3406:|AM9PR08MB6308:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB63089F0BCD5CA480E6B3A126F7DC9@AM9PR08MB6308.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 i6ZZqFLVUmAlQV46k6+5/hopwAIkcA/+vVET7CdXbT9OmNh3bThoiei3y/pt7x9K8hX8duBEp2uf4w/tLPiXJXa1S/fMlH9wFzXc2SPm91zEXyJAyifgZ1qqhP+r/ZAvD4S9NYDulJ+EafOTScnIvkv1WRxWy67l/au9nETHz653xM/0nzzqmnGulQ3Zm6now4LMSEnislUSJ0xznVRXyd27n5gYh2ib86b4DWMTS/EHiSwINqlS1QKflWp5UkWrw6f0f2zaZAZwIAmuHCbZyg2OtM4/q70zc0PNYsGyDGVnhtJpst4tLc15C1IbYmyUa/76TPdeUYWs4OGiAhr1m+vwKTXwKBLSe2H0L7YKXhWP8NfsH6TuJM3N2ozRQBfjKLSAkJt4biAhAXvZNDxzYrb/WmyXWkfvO1iuJly10c286aHk8qL+oCxu45mCQybDDzNaP/riXfyCTQUWONbW6Xz7PeEQKHyVm1PcTQEISmdFABFrJJs0JUViwnN9VhN85fOkbiqgud2O9Mj/lR8+6oBYQm8eM6g5iQsdRRBmhYR4yMgoTtbTkeiuEm0XVAw/g2lqbZV7R9sUQz5Yl1G4nUC1n6PQPgdKuEe6D3OUFBbYJo/6BH3hNINmscg3NhARwGzGUeeHDSaLswADYl7IJoBfySnDdlxkrX5oo4yfocd3WyEvSaqz6bTJbXhivbSlrvQfFMhwGBYxHBWYljxmlw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR08MB5215.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(39860400002)(396003)(366004)(346002)(6506007)(52536014)(5660300002)(186003)(9686003)(38100700002)(38070700005)(316002)(6916009)(71200400001)(122000001)(64756008)(7696005)(53546011)(8676002)(66556008)(66476007)(76116006)(4326008)(66946007)(478600001)(86362001)(33656002)(26005)(66446008)(55016002)(83380400001)(54906003)(8936002)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3406
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8bdbddf5-6da4-4809-fc74-08d978bffb97
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vjqwUfw+2HTc3CtNIIq7acv91/iUoZ2zIMTwtm2hpZguCAFgJIjIn+zfLUbkoLT/lVGcyg2Y8JEs5n8oV8Dpvs8T2ELxQteC3sPkvX4M0Nj7Ui3TPSEjvdSD9RP76/55W3z6a+pmpGJ37TbhuGmAh5ER24HB13jZTpxfeFBe1u8qTE2RwpSWM8AGAwkXtfIto/GGAARxLzv3XtdlHC5nn2o6tj/NyrUQttJLTDN6igQCTczoEHIlH6xLa+XG+j4aMnynwsalNFNkUuwB2HV+iAchp2agBjFQ+hGI7MWQkkLh4tdSPhG8sGKGi4Ysa/KFXTLDLKs2uApiM1qk4vT4mv9/bz3R7Df8kuddDWQJ4aEkAofP8o6wIaAxU6gBVJxPmrC8D6wmGajN3t/tW0SWh5+GTk6PmY/fyL2rh/izuVX4IeDfyHe5Q2ELnXeYqd1NTKwp0tXWa+qcrn8odcrsW9GD7J10npGXa6elX6y71mw2tqa9UnMss+4rB7RObX9kNLz1wUHxoW7GpQ4XvJjB8k4WheqbvvcbEvTbNFnQSaoNiDHfpBi/o8RkpjUN1kFzOZGFN517neGvt3YCWh6axis1/P8Qjkm0j+zE0CCfYmp/U0+VdWWeKm9Hen41eS8KYsl0cmYzymYH4V2sRbCvyOgsTLwuk1V5yqDdBIYfEypmjpewzIXk9Aimi6wyt6zUI0PAx/1SQkQOULPUi+k5BA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(376002)(396003)(136003)(39860400002)(346002)(36840700001)(46966006)(82740400003)(54906003)(70206006)(26005)(52536014)(86362001)(4326008)(70586007)(7696005)(316002)(5660300002)(336012)(8676002)(186003)(6506007)(53546011)(9686003)(83380400001)(47076005)(356005)(81166007)(82310400003)(36860700001)(478600001)(2906002)(8936002)(6862004)(55016002)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 03:13:50.6159
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 546cd9cb-63a9-4a86-bfcf-08d978c001b7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6308

SGkgSmFuDQoNClNvcnJ5IGZvciB0aGUgYnJva2VuLg0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn
ZS0tLS0tDQo+IEZyb206IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gU2VudDog
V2VkbmVzZGF5LCBTZXB0ZW1iZXIgMTUsIDIwMjEgOTo1NiBQTQ0KPiBUbzogUGVubnkgWmhlbmcg
PFBlbm55LlpoZW5nQGFybS5jb20+DQo+IENjOiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5N
YXJxdWlzQGFybS5jb20+OyBXZWkgQ2hlbg0KPiA8V2VpLkNoZW5AYXJtLmNvbT47IHhlbi1kZXZl
bEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsNCj4gc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVu
QHhlbi5vcmcNCj4gU3ViamVjdDogUmU6IFtQQVRDSCB2NyAzLzddIHhlbjogaW50cm9kdWNlIG1h
cmtfcGFnZV9mcmVlDQo+IA0KPiBPbiAxMC4wOS4yMDIxIDA0OjUyLCBQZW5ueSBaaGVuZyB3cm90
ZToNCj4gPiBUaGlzIGNvbW1pdCBkZWZpbmVzIGEgbmV3IGhlbHBlciBtYXJrX3BhZ2VfZnJlZSB0
byBleHRyYWN0IGNvbW1vbg0KPiA+IGNvZGUsIGxpa2UgZm9sbG93aW5nIHRoZSBzYW1lIGNhY2hl
L1RMQiBjb2hlcmVuY3kgcG9saWN5LCBiZXR3ZWVuDQo+ID4gZnJlZV9oZWFwX3BhZ2VzIGFuZCB0
aGUgbmV3IGZ1bmN0aW9uIGZyZWVfc3RhdGljbWVtX3BhZ2VzLCB3aGljaCB3aWxsIGJlDQo+IGlu
dHJvZHVjZWQgbGF0ZXIuDQo+ID4NCj4gPiBUaGUgUERYIGNvbXByZXNzaW9uIG1ha2VzIHRoYXQg
Y29udmVyc2lvbiBiZXR3ZWVuIHRoZSBNRk4gYW5kIHRoZSBwYWdlDQo+ID4gY2FuIGJlIHBvdGVu
dGlhbGx5IG5vbi10cml2aWFsLiBBcyB0aGUgZnVuY3Rpb24gaXMgaW50ZXJuYWwsIHBhc3MgdGhl
DQo+ID4gTUZOIGFuZCB0aGUgcGFnZS4gVGhleSBhcmUgYm90aCBleHBlY3RlZCB0byBtYXRjaC4N
Cj4gPg0KPiA+IFNpZ25lZC1vZmYtYnk6IFBlbm55IFpoZW5nIDxwZW5ueS56aGVuZ0Bhcm0uY29t
Pg0KPiA+IEFja2VkLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+ID4gUmV2
aWV3ZWQtYnk6IEp1bGllbiBHcmFsbCA8amdyYWxsQGFtYXpvbi5jb20+DQo+ID4gLS0tDQo+ID4g
IHhlbi9jb21tb24vcGFnZV9hbGxvYy5jIHwgODkNCj4gPiArKysrKysrKysrKysrKysrKysrKysr
LS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiA+ICAxIGZpbGUgY2hhbmdlZCwgNDggaW5zZXJ0aW9ucygr
KSwgNDEgZGVsZXRpb25zKC0pDQo+ID4NCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2NvbW1vbi9wYWdl
X2FsbG9jLmMgYi94ZW4vY29tbW9uL3BhZ2VfYWxsb2MuYyBpbmRleA0KPiA+IDk1OGJhMGNkOTIu
LmEzZWU1ZWNhOWUgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2NvbW1vbi9wYWdlX2FsbG9jLmMNCj4g
PiArKysgYi94ZW4vY29tbW9uL3BhZ2VfYWxsb2MuYw0KPiA+IEBAIC0xMzc2LDYgKzEzNzYsNTMg
QEAgYm9vbCBzY3J1Yl9mcmVlX3BhZ2VzKHZvaWQpDQo+ID4gICAgICByZXR1cm4gbm9kZV90b19z
Y3J1YihmYWxzZSkgIT0gTlVNQV9OT19OT0RFOyAgfQ0KPiA+DQo+ID4gK3N0YXRpYyB2b2lkIG1h
cmtfcGFnZV9mcmVlKHN0cnVjdCBwYWdlX2luZm8gKnBnLCBtZm5fdCBtZm4pIHsNCj4gPiArICAg
IEFTU0VSVChtZm5feChtZm4pID09IG1mbl94KHBhZ2VfdG9fbWZuKHBnKSkpOw0KPiA+ICsNCj4g
PiArICAgIC8qDQo+ID4gKyAgICAgKiBDYW5ub3QgYXNzdW1lIHRoYXQgY291bnRfaW5mbyA9PSAw
LCBhcyB0aGVyZSBhcmUgc29tZSBjb3JuZXIgY2FzZXMNCj4gPiArICAgICAqIHdoZXJlIGl0IGlz
bid0IHRoZSBjYXNlIGFuZCB5ZXQgaXQgaXNuJ3QgYSBidWc6DQo+ID4gKyAgICAgKiAgMS4gcGFn
ZV9nZXRfb3duZXIoKSBpcyBOVUxMDQo+ID4gKyAgICAgKiAgMi4gcGFnZV9nZXRfb3duZXIoKSBp
cyBhIGRvbWFpbiB0aGF0IHdhcyBuZXZlciBhY2Nlc3NpYmxlIGJ5DQo+ID4gKyAgICAgKiAgICAg
aXRzIGRvbWlkIChlLmcuLCBmYWlsZWQgdG8gZnVsbHkgY29uc3RydWN0IHRoZSBkb21haW4pLg0K
PiA+ICsgICAgICogIDMuIHBhZ2Ugd2FzIG5ldmVyIGFkZHJlc3NhYmxlIGJ5IHRoZSBndWVzdCAo
ZS5nLiwgaXQncyBhbg0KPiA+ICsgICAgICogICAgIGF1dG8tdHJhbnNsYXRlLXBoeXNtYXAgZ3Vl
c3QgYW5kIHRoZSBwYWdlIHdhcyBuZXZlciBpbmNsdWRlZA0KPiA+ICsgICAgICogICAgIGluIGl0
cyBwc2V1ZG9waHlzaWNhbCBhZGRyZXNzIHNwYWNlKS4NCj4gPiArICAgICAqIEluIGFsbCB0aGUg
YWJvdmUgY2FzZXMgdGhlcmUgY2FuIGJlIG5vIGd1ZXN0IG1hcHBpbmdzIG9mIHRoaXMgcGFnZS4N
Cj4gPiArICAgICAqLw0KPiA+ICsgICAgc3dpdGNoICggcGctPmNvdW50X2luZm8gJiBQR0Nfc3Rh
dGUgKQ0KPiA+ICsgICAgew0KPiA+ICsgICAgY2FzZSBQR0Nfc3RhdGVfaW51c2U6DQo+ID4gKyAg
ICAgICAgQlVHX09OKHBnLT5jb3VudF9pbmZvICYgUEdDX2Jyb2tlbik7DQo+ID4gKyAgICAgICAg
cGctPmNvdW50X2luZm8gPSBQR0Nfc3RhdGVfZnJlZTsNCj4gPiArICAgICAgICBicmVhazsNCj4g
PiArDQo+ID4gKyAgICBjYXNlIFBHQ19zdGF0ZV9vZmZsaW5pbmc6DQo+ID4gKyAgICAgICAgcGct
PmNvdW50X2luZm8gPSAocGctPmNvdW50X2luZm8gJiBQR0NfYnJva2VuKSB8DQo+ID4gKyAgICAg
ICAgICAgICAgICAgICAgICAgICBQR0Nfc3RhdGVfb2ZmbGluZWQ7DQo+ID4gKyAgICAgICAgdGFp
bnRlZCA9IDE7DQo+IA0KPiBZb3UndmUgYnJva2VuIHR3byB0aGluZ3MgYXQgdGhlIHNhbWUgdGlt
ZSBoZXJlOiBZb3Ugd3JpdGUgdG8gdGhlIGdsb2JhbA0KPiB2YXJpYWJsZSBvZiB0aGlzIG5hbWUg
bm93LCB3aGlsZSAuLi4NCj4gDQo+ID4gQEAgLTEzOTIsNDcgKzE0MzksNyBAQCBzdGF0aWMgdm9p
ZCBmcmVlX2hlYXBfcGFnZXMoDQo+ID4NCj4gPiAgICAgIGZvciAoIGkgPSAwOyBpIDwgKDEgPDwg
b3JkZXIpOyBpKysgKQ0KPiA+ICAgICAgew0KPiA+IC0gICAgICAgIC8qDQo+ID4gLSAgICAgICAg
ICogQ2Fubm90IGFzc3VtZSB0aGF0IGNvdW50X2luZm8gPT0gMCwgYXMgdGhlcmUgYXJlIHNvbWUg
Y29ybmVyIGNhc2VzDQo+ID4gLSAgICAgICAgICogd2hlcmUgaXQgaXNuJ3QgdGhlIGNhc2UgYW5k
IHlldCBpdCBpc24ndCBhIGJ1ZzoNCj4gPiAtICAgICAgICAgKiAgMS4gcGFnZV9nZXRfb3duZXIo
KSBpcyBOVUxMDQo+ID4gLSAgICAgICAgICogIDIuIHBhZ2VfZ2V0X293bmVyKCkgaXMgYSBkb21h
aW4gdGhhdCB3YXMgbmV2ZXIgYWNjZXNzaWJsZSBieQ0KPiA+IC0gICAgICAgICAqICAgICBpdHMg
ZG9taWQgKGUuZy4sIGZhaWxlZCB0byBmdWxseSBjb25zdHJ1Y3QgdGhlIGRvbWFpbikuDQo+ID4g
LSAgICAgICAgICogIDMuIHBhZ2Ugd2FzIG5ldmVyIGFkZHJlc3NhYmxlIGJ5IHRoZSBndWVzdCAo
ZS5nLiwgaXQncyBhbg0KPiA+IC0gICAgICAgICAqICAgICBhdXRvLXRyYW5zbGF0ZS1waHlzbWFw
IGd1ZXN0IGFuZCB0aGUgcGFnZSB3YXMgbmV2ZXIgaW5jbHVkZWQNCj4gPiAtICAgICAgICAgKiAg
ICAgaW4gaXRzIHBzZXVkb3BoeXNpY2FsIGFkZHJlc3Mgc3BhY2UpLg0KPiA+IC0gICAgICAgICAq
IEluIGFsbCB0aGUgYWJvdmUgY2FzZXMgdGhlcmUgY2FuIGJlIG5vIGd1ZXN0IG1hcHBpbmdzIG9m
IHRoaXMgcGFnZS4NCj4gPiAtICAgICAgICAgKi8NCj4gPiAtICAgICAgICBzd2l0Y2ggKCBwZ1tp
XS5jb3VudF9pbmZvICYgUEdDX3N0YXRlICkNCj4gPiAtICAgICAgICB7DQo+ID4gLSAgICAgICAg
Y2FzZSBQR0Nfc3RhdGVfaW51c2U6DQo+ID4gLSAgICAgICAgICAgIEJVR19PTihwZ1tpXS5jb3Vu
dF9pbmZvICYgUEdDX2Jyb2tlbik7DQo+ID4gLSAgICAgICAgICAgIHBnW2ldLmNvdW50X2luZm8g
PSBQR0Nfc3RhdGVfZnJlZTsNCj4gPiAtICAgICAgICAgICAgYnJlYWs7DQo+ID4gLQ0KPiA+IC0g
ICAgICAgIGNhc2UgUEdDX3N0YXRlX29mZmxpbmluZzoNCj4gPiAtICAgICAgICAgICAgcGdbaV0u
Y291bnRfaW5mbyA9IChwZ1tpXS5jb3VudF9pbmZvICYgUEdDX2Jyb2tlbikgfA0KPiA+IC0gICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgUEdDX3N0YXRlX29mZmxpbmVkOw0KPiA+IC0gICAg
ICAgICAgICB0YWludGVkID0gMTsNCj4gDQo+IC4uLiB0aGUgbG9jYWwgdmFyaWFibGUgaGVyZSBk
b2Vzbid0IGdldCB3cml0dGVuIGFueW1vcmUuIFdoaWNoIENvdmVyaXR5IHdhcw0KPiBraW5kIGVu
b3VnaCB0byBwb2ludCBvdXQgLSBwbGVhc2UgcmVmZXJlbmNlIENvdmVyaXR5IElEIDE0OTE4NzIg
aW4gdGhlIGZpeCB0aGF0DQo+IEkgaG9wZSB5b3Ugd2lsbCBiZSBhYmxlIHRvIHByb3ZpZGUgcXVp
Y2tseS4gKFRoZSBlYXNpZXN0IGNoYW5nZSB3b3VsZCBzZWVtIHRvDQo+IGJlIHRvIG1ha2UgbWFy
a19wYWdlX2ZyZWUoKSByZXR1cm4gYm9vbCwgYW5kIHNldCAidGFpbnRlZCIgdG8gMSBoZXJlDQo+
IGFjY29yZGluZ2x5LikNCj4gDQoNClN1cmUuIEknbGwgZml4IGl0IHRvZGF5IGFuZCBsZXQgbWFy
a19wYWdlX2ZyZWUoKSByZXR1cm4gdGhlIHN0YXR1cyBvZiAidGFpbnRlZCIuDQoNCj4gSSB1bmRl
cnN0YW5kIHRoYXQgdGhlIHR3byB2YXJpYWJsZXMgaGF2aW5nIHRoZSBzYW1lIG5hbWUgaXNuJ3Qg
dmVyeSBoZWxwZnVsLiBJDQo+IGNlcnRhaW5seSB3b3VsZG4ndCBtaW5kIGlmIHlvdSByZW5hbWVk
IHRoZSBsb2NhbCBvbmUgc3VpdGFibHkuDQo+IA0KDQpJJ2xsIHJlbmFtZSB0aGUgbG9jYWwgb25l
IHRvICJwZ190YWludGVkIiB0byB0ZWxsIHRoZSBkaWZmZXJlbmNlLg0KDQo+IEphbg0KDQpQZW5u
eQ0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 03:36:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 03:36:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188000.337035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQiCX-0000OX-MT; Thu, 16 Sep 2021 03:36:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188000.337035; Thu, 16 Sep 2021 03:36:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQiCX-0000OQ-IT; Thu, 16 Sep 2021 03:36:41 +0000
Received: by outflank-mailman (input) for mailman id 188000;
 Thu, 16 Sep 2021 03:36:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiCW-0000OG-8i; Thu, 16 Sep 2021 03:36:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiCW-00040Y-3b; Thu, 16 Sep 2021 03:36:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiCV-0004fF-R0; Thu, 16 Sep 2021 03:36:39 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiCV-00061a-QW; Thu, 16 Sep 2021 03:36:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0SfEuT91P7XFKQuwG5YPFwjCu5K/5LFgGFTfSuawe/4=; b=oLR5V6//nAVbW8GTymrrx4qHY4
	UAaXgJWgbDdLhcPbUgDAGFvghPBTL/IAUDVo5nj737mQa2MDi5GTiBFZ+Vo2m/hwsBjsc1Ww3lUs7
	TEVXCpU/V5JUH2PWc7pv1hKLuhTV6QhOuRnEake9hDebop7qDXr9UIwkaDxu7/Pc/Nlg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164997-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 164997: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=3ca706c189db861b2ca2019a0901b94050ca49d8
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 03:36:39 +0000

flight 164997 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/164997/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine    4 memdisk-try-append fail in 164988 pass in 164997
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 164988

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-freebsd11-amd64 21 guest-start/freebsd.repeat fail in 164988 baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                3ca706c189db861b2ca2019a0901b94050ca49d8
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  411 days
Failing since        152366  2020-08-01 20:49:34 Z  410 days  714 attempts
Testing same since   164988  2021-09-15 00:40:46 Z    1 days    2 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2255804 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 04:06:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 04:06:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188009.337049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQif0-0004Bv-RE; Thu, 16 Sep 2021 04:06:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188009.337049; Thu, 16 Sep 2021 04:06:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQif0-0004Bo-Mx; Thu, 16 Sep 2021 04:06:06 +0000
Received: by outflank-mailman (input) for mailman id 188009;
 Thu, 16 Sep 2021 04:06:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiez-0004Be-KQ; Thu, 16 Sep 2021 04:06:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiez-0004Zu-C5; Thu, 16 Sep 2021 04:06:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiey-0006Oa-Sp; Thu, 16 Sep 2021 04:06:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQiey-0000JB-S4; Thu, 16 Sep 2021 04:06:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CNA/JTwPP6pGmHGqcGs76ZfzuuREr3bWZv9m99T0Ffk=; b=e/Rm7dnRhhS1Ym1tO4UXaNFVT/
	h/IFOg00q5Zzh50mP0FQCQ+jXIhNV1mRih53C8r6OV5zvJFSQuLTjCse4EsIf2mkaPoLWyFksBMWT
	983ss/ZHKIbHdfjU2DAu8lq1Mqs0dA3Hw8whulF4+hMU09d008twg/69H8YXrSis3+nw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-164996-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 164996: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=487975df53b5298316b594550c79934d646701bd
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 04:06:04 +0000

flight 164996 xen-unstable real [real]
flight 165002 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/164996/
http://logs.test-lab.xenproject.org/osstest/logs/165002/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165002-retest
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165002-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  487975df53b5298316b594550c79934d646701bd
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    5 days
Failing since        164951  2021-09-12 00:14:36 Z    4 days    8 attempts
Testing same since   164996  2021-09-15 11:47:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 487975df53b5298316b594550c79934d646701bd
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:15 2021 +0000

    xen/arm: introduce allocate_static_memory
    
    This commit introduces a new function allocate_static_memory to allocate
    static memory as guest RAM for domains on Static Allocation.
    
    It uses acquire_domstatic_pages to acquire pre-configured static memory
    for the domain, and uses guest_physmap_add_pages to set up the P2M table.
    These pre-defined static memory banks shall be mapped to the usual guest
    memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
    xen/include/public/arch-arm.h.
    
    In order to deal with the trouble of count-to-order conversion when page number
    is not in a power-of-two, this commit exports p2m_insert_mapping and introduce
    a new function guest_physmap_add_pages to cope with adding guest RAM p2m
    mapping with nr_pages.
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit c7fe462c0d274ffa30c9448c0a80affa075d789d
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:14 2021 +0000

    xen/arm: introduce acquire_staticmem_pages and acquire_domstatic_pages
    
    New function acquire_staticmem_pages aims to acquire nr_mfns contiguous pages
    of static memory, starting at #smfn. And it is the equivalent of
    alloc_heap_pages for static memory.
    
    For each page, it shall check if the page is reserved(PGC_reserved)
    and free. It shall also do a set of necessary initialization, which are
    mostly the same ones in alloc_heap_pages, like, following the same
    cache-coherency policy and turning page status into PGC_state_inuse, etc.
    
    New function acquire_domstatic_pages is the equivalent of alloc_domheap_pages
    for static memory, and it is to acquire nr_mfns contiguous pages of
    static memory and assign them to one specific domain.
    
    It uses acquire_staticmem_pages to acquire nr_mfns pages of static memory.
    Then on success, it will use assign_pages to assign those pages to one
    specific domain.
    
    In order to differentiate pages of static memory from those allocated from
    heap, this patch introduces a new page flag PGC_reserved, then mark pages of
    static memory PGC_reserved when initializing them.
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit 5260e8fb93f0e1f094de4142b2abad45844ab89c
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:13 2021 +0000

    xen: re-define assign_pages and introduce a new function assign_page
    
    In order to deal with the trouble of count-to-order conversion when page number
    is not in a power-of-two, this commit re-define assign_pages for nr pages and
    assign_page for original page with a single order.
    
    Backporting confusion could be helped by altering the order of assign_pages
    parameters, such that the compiler would point out that adjustments at call
    sites are needed.
    
    [stefano: switch to unsigned int for nr]
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit 4a9e73e6e53e9d8bc005a08c3968ec36d793f140
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:12 2021 +0000

    xen/arm: static memory initialization
    
    This patch introduces static memory initialization, during system boot-up.
    
    The new function init_staticmem_pages is responsible for static memory
    initialization.
    
    Helper free_staticmem_pages is the equivalent of free_heap_pages, to free
    nr_mfns pages of static memory.
    
    This commit also introduces a new CONFIG_STATIC_MEMORY option to wrap all
    static-allocation-related code.
    
    Put asynchronously scrubbing pages of static memory in TODO list.
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit 540a637c3410780b519fc055f432afe271f642f8
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:11 2021 +0000

    xen: introduce mark_page_free
    
    This commit defines a new helper mark_page_free to extract common code,
    like following the same cache/TLB coherency policy, between free_heap_pages
    and the new function free_staticmem_pages, which will be introduced later.
    
    The PDX compression makes that conversion between the MFN and the page can
    be potentially non-trivial. As the function is internal, pass the MFN and
    the page. They are both expected to match.
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 41c031ff437b66cfac4b120bd7698ca039850690
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:10 2021 +0000

    xen/arm: introduce domain on Static Allocation
    
    Static Allocation refers to system or sub-system(domains) for which memory
    areas are pre-defined by configuration using physical address ranges.
    
    Those pre-defined memory, -- Static Memory, as parts of RAM reserved in the
    beginning, shall never go to heap allocator or boot allocator for any use.
    
    Memory can be statically allocated to a domain using the property "xen,static-
    mem" defined in the domain configuration. The number of cells for the address
    and the size must be defined using respectively the properties
    "#xen,static-mem-address-cells" and "#xen,static-mem-size-cells".
    
    The property 'memory' is still needed and should match the amount of memory
    given to the guest. Currently, it either comes from static memory or lets Xen
    allocate from heap. *Mixing* is not supported.
    
    The static memory will be mapped in the guest at the usual guest memory
    addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
    xen/include/public/arch-arm.h.
    
    This patch introduces this new `xen,static-mem` feature, and also documents
    and parses this new attribute at boot time.
    
    This patch also introduces a new field "bool xen_domain" in "struct membank"
    to tell whether the memory bank is reserved as the whole hardware resource,
    or bind to a xen domain node, through "xen,static-mem"
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit 904ba3ce2e46e59080dc09676cede5df63b59f20
Author: Penny Zheng <penny.zheng@arm.com>
Date:   Fri Sep 10 02:52:09 2021 +0000

    xen/arm: introduce new helper device_tree_get_meminfo
    
    This commit creates a new helper device_tree_get_meminfo to iterate over a
    device tree property to get memory info, like "reg".
    
    Signed-off-by: Penny Zheng <penny.zheng@arm.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit a89bcd9737757e4d671783588a6041a84a5e1754
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Wed Jul 7 09:15:31 2021 +0200

    tools/go: honor append build flags
    
    Make the go build use APPEND_{C/LD}FLAGS when necessary, just like
    other parts of the build.
    
    Reported-by: Ting-Wei Lan <lantw44@gmail.com>
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Acked-by: Nick Rosbrook <rosbrookn@ainfosec.com>
    Acked-by: Ian Jackson <iwj@xenproject.org>

commit 6d45368a0a89e01a3a01d156af61fea565db96cc
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:59 2021 -0400

    xsm: drop dubious xsm_op_t type
    
    The type xsm_op_t masks the use of void pointers. This commit drops the
    xsm_op_t type and replaces it and all its uses with an explicit void.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 2928c1d250b157fd4585ca47ba36ad4792723f1f
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:58 2021 -0400

    xsm: remove remnants of xsm_memtype hook
    
    In c/s fcb8baddf00e the xsm_memtype hook was removed but some remnants were
    left behind. This commit cleans up those remnants.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 4624912c0b5505387e53a12ef3417d001431a29d
Author: Daniel P. Smith <dpsmith@apertussolutions.com>
Date:   Fri Sep 10 16:12:57 2021 -0400

    xsm: remove the ability to disable flask
    
    On Linux when SELinux is put into permissive mode the discretionary access
    controls are still in place. Whereas for Xen when the enforcing state of flask
    is set to permissive, all operations for all domains would succeed, i.e. it
    does not fall back to the default access controls. To provide a means to mimic
    a similar but not equivalent behaviour, a flask op is present to allow a
    one-time switch back to the default access controls, aka the "dummy policy".
    
    While this may be desirable for an OS, Xen is a hypervisor and should not
    allow the switching of which security policy framework is being enforced after
    boot.  This patch removes the flask op to enforce the desired XSM usage model
    requiring a reboot of Xen to change the XSM policy module in use.
    
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit f26bb285949b8c233816c4c6a87237ee14a06ebc
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Fri Sep 10 16:12:56 2021 -0400

    xen: Implement xen/alternative-call.h for use in common code
    
    The alternative call infrastructure is x86-only for now, but the common iommu
    code has a variant and more common code wants to use the infrastructure.
    
    Introduce CONFIG_ALTERNATIVE_CALL and a conditional implementation so common
    code can use the optimisation when available, without requiring all
    architectures to implement no-op stubs.
    
    Write some documentation, which was thus far entirely absent, covering the
    requirements for an architecture to implement this optimisation, and how to
    use the infrastructure in general code.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:26:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:26:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188019.337062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkqY-0002Bs-4Y; Thu, 16 Sep 2021 06:26:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188019.337062; Thu, 16 Sep 2021 06:26:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkqY-0002Bl-1Y; Thu, 16 Sep 2021 06:26:10 +0000
Received: by outflank-mailman (input) for mailman id 188019;
 Thu, 16 Sep 2021 06:26:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkqW-0002Bf-7H
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:08 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id f99ac5ea-16b6-11ec-b5a6-12813bfff9fa;
 Thu, 16 Sep 2021 06:26:07 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9C05831B;
 Wed, 15 Sep 2021 23:26:06 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 634CC3F59C;
 Wed, 15 Sep 2021 23:26:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f99ac5ea-16b6-11ec-b5a6-12813bfff9fa
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 0/7] xen/arm: Sanitize cpuinfo
Date: Thu, 16 Sep 2021 07:25:33 +0100
Message-Id: <cover.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1

On arm architecture we might have heterogeneous platforms with different
types of cores. As a guest can potentialy run on any of those cores we
have to present them cpu features which are compatible with all cores
and discard the features which are only available on some cores.

As the features can be fairly complex, the way to deduce from 2
different features what should be the acceptable minimal feature can be
complex (and sometime impossible).

To reduce the implementation effort in Xen, this serie is importing the
structures and filtering system used by Linux in order to build a
cpuinfo containing the best values compatible with all cores on the
platform.

The serie start by importing the necessary code and structure from Linux
and then use it to sanitize the boot cpuinfo.
It is simplifying p2m code which was searching for the best support
value for PAR and VMID size.

On big.LITTLE platforms the current behaviour is kept and if hmp-unsafe
is provided in Xen command line then  Xen is using all cores. This serie
is tainting Xen with CPU_OUT_OF_SPEC when different MIDR are found.

It is also adding DCZID and CTR registers in cpuinfo in order to check
for incoherent values between cores for those 2 registers. Xen is
tainted if different DCZID registers are found and CTR register is
sanitized when possible. As CTR is not emulated, the sanitized value is
only available inside Xen.

Changes in v4:
- add and use arm64_ftr_set_value
- remove TID2 bit setting and CTR_EL0 emulation in case of different
  i-cache types. We now just sanitize ctr_el0 internally and taint Xen.
Changes in v3:
- restore hmp-unsafe need to use all core when different midr are found
- taint Xen if different MIDR are found
- sanitize system_cpuinfo only if core is not stopped by Xen on boot
Changes in v2:
- Sanitize DCZID register
- Sanitize CTR_EL0 and add emulation of registers catched by TID2
- rename cpu_boot_data to system_cpuinfo


Bertrand Marquis (7):
  xen/arm: Import ID registers definitions from Linux
  xen/arm: Import ID features sanitize from linux
  xen/arm: Rename cpu_boot_data to system_cpuinfo
  xen/arm: Sanitize cpuinfo ID registers fields
  xen/arm: Use sanitize values for p2m
  xen/arm: Taint Xen on incompatible DCZID values
  xen/arm: Sanitize CTR_EL0

 xen/arch/arm/arm64/Makefile            |   1 +
 xen/arch/arm/arm64/cpufeature.c        | 637 +++++++++++++++++++++++++
 xen/arch/arm/cpufeature.c              |  12 +-
 xen/arch/arm/p2m.c                     |  30 +-
 xen/arch/arm/setup.c                   |  36 +-
 xen/arch/arm/smpboot.c                 |  36 +-
 xen/common/kernel.c                    |   6 +-
 xen/include/asm-arm/arm64/cpufeature.h | 104 ++++
 xen/include/asm-arm/arm64/sysregs.h    | 312 ++++++++++++
 xen/include/asm-arm/cpufeature.h       |  32 +-
 xen/include/asm-arm/processor.h        |  18 +-
 xen/include/xen/lib.h                  |   1 +
 12 files changed, 1166 insertions(+), 59 deletions(-)
 create mode 100644 xen/arch/arm/arm64/cpufeature.c
 create mode 100644 xen/include/asm-arm/arm64/cpufeature.h

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:26:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:26:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188025.337074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkr8-0002if-EC; Thu, 16 Sep 2021 06:26:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188025.337074; Thu, 16 Sep 2021 06:26:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkr8-0002iY-AV; Thu, 16 Sep 2021 06:26:46 +0000
Received: by outflank-mailman (input) for mailman id 188025;
 Thu, 16 Sep 2021 06:26:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkr7-0002iQ-Qr
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:45 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 107f39f8-16b7-11ec-b5a6-12813bfff9fa;
 Thu, 16 Sep 2021 06:26:45 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 248CD31B;
 Wed, 15 Sep 2021 23:26:45 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 585953F59C;
 Wed, 15 Sep 2021 23:26:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 107f39f8-16b7-11ec-b5a6-12813bfff9fa
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 1/7] xen/arm: Import ID registers definitions from Linux
Date: Thu, 16 Sep 2021 07:25:34 +0100
Message-Id: <36798ba046b8d1862c144a0dca1db5d14a955abf.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Import some ID registers definitions from Linux sysreg header to have
required shift definitions for all ID registers fields.

Those are required to reuse the cpufeature sanitization system from
Linux kernel.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
Changes in v4: Add acked-by Julien
Changes in v3: none
Changes in v2: Rebase
---
 xen/include/asm-arm/arm64/sysregs.h | 312 ++++++++++++++++++++++++++++
 1 file changed, 312 insertions(+)

diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index 795901e1ba..d7e4772f21 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -85,6 +85,318 @@
 #define ID_DFR1_EL1                 S3_0_C0_C3_5
 #endif
 
+/* ID registers (imported from arm64/include/asm/sysreg.h in Linux) */
+
+/* id_aa64isar0 */
+#define ID_AA64ISAR0_RNDR_SHIFT      60
+#define ID_AA64ISAR0_TLB_SHIFT       56
+#define ID_AA64ISAR0_TS_SHIFT        52
+#define ID_AA64ISAR0_FHM_SHIFT       48
+#define ID_AA64ISAR0_DP_SHIFT        44
+#define ID_AA64ISAR0_SM4_SHIFT       40
+#define ID_AA64ISAR0_SM3_SHIFT       36
+#define ID_AA64ISAR0_SHA3_SHIFT      32
+#define ID_AA64ISAR0_RDM_SHIFT       28
+#define ID_AA64ISAR0_ATOMICS_SHIFT   20
+#define ID_AA64ISAR0_CRC32_SHIFT     16
+#define ID_AA64ISAR0_SHA2_SHIFT      12
+#define ID_AA64ISAR0_SHA1_SHIFT      8
+#define ID_AA64ISAR0_AES_SHIFT       4
+
+#define ID_AA64ISAR0_TLB_RANGE_NI    0x0
+#define ID_AA64ISAR0_TLB_RANGE       0x2
+
+/* id_aa64isar1 */
+#define ID_AA64ISAR1_I8MM_SHIFT      52
+#define ID_AA64ISAR1_DGH_SHIFT       48
+#define ID_AA64ISAR1_BF16_SHIFT      44
+#define ID_AA64ISAR1_SPECRES_SHIFT   40
+#define ID_AA64ISAR1_SB_SHIFT        36
+#define ID_AA64ISAR1_FRINTTS_SHIFT   32
+#define ID_AA64ISAR1_GPI_SHIFT       28
+#define ID_AA64ISAR1_GPA_SHIFT       24
+#define ID_AA64ISAR1_LRCPC_SHIFT     20
+#define ID_AA64ISAR1_FCMA_SHIFT      16
+#define ID_AA64ISAR1_JSCVT_SHIFT     12
+#define ID_AA64ISAR1_API_SHIFT       8
+#define ID_AA64ISAR1_APA_SHIFT       4
+#define ID_AA64ISAR1_DPB_SHIFT       0
+
+#define ID_AA64ISAR1_APA_NI                     0x0
+#define ID_AA64ISAR1_APA_ARCHITECTED            0x1
+#define ID_AA64ISAR1_APA_ARCH_EPAC              0x2
+#define ID_AA64ISAR1_APA_ARCH_EPAC2             0x3
+#define ID_AA64ISAR1_APA_ARCH_EPAC2_FPAC        0x4
+#define ID_AA64ISAR1_APA_ARCH_EPAC2_FPAC_CMB    0x5
+#define ID_AA64ISAR1_API_NI                     0x0
+#define ID_AA64ISAR1_API_IMP_DEF                0x1
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC           0x2
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2          0x3
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2_FPAC     0x4
+#define ID_AA64ISAR1_API_IMP_DEF_EPAC2_FPAC_CMB 0x5
+#define ID_AA64ISAR1_GPA_NI                     0x0
+#define ID_AA64ISAR1_GPA_ARCHITECTED            0x1
+#define ID_AA64ISAR1_GPI_NI                     0x0
+#define ID_AA64ISAR1_GPI_IMP_DEF                0x1
+
+/* id_aa64pfr0 */
+#define ID_AA64PFR0_CSV3_SHIFT       60
+#define ID_AA64PFR0_CSV2_SHIFT       56
+#define ID_AA64PFR0_DIT_SHIFT        48
+#define ID_AA64PFR0_AMU_SHIFT        44
+#define ID_AA64PFR0_MPAM_SHIFT       40
+#define ID_AA64PFR0_SEL2_SHIFT       36
+#define ID_AA64PFR0_SVE_SHIFT        32
+#define ID_AA64PFR0_RAS_SHIFT        28
+#define ID_AA64PFR0_GIC_SHIFT        24
+#define ID_AA64PFR0_ASIMD_SHIFT      20
+#define ID_AA64PFR0_FP_SHIFT         16
+#define ID_AA64PFR0_EL3_SHIFT        12
+#define ID_AA64PFR0_EL2_SHIFT        8
+#define ID_AA64PFR0_EL1_SHIFT        4
+#define ID_AA64PFR0_EL0_SHIFT        0
+
+#define ID_AA64PFR0_AMU              0x1
+#define ID_AA64PFR0_SVE              0x1
+#define ID_AA64PFR0_RAS_V1           0x1
+#define ID_AA64PFR0_FP_NI            0xf
+#define ID_AA64PFR0_FP_SUPPORTED     0x0
+#define ID_AA64PFR0_ASIMD_NI         0xf
+#define ID_AA64PFR0_ASIMD_SUPPORTED  0x0
+#define ID_AA64PFR0_EL1_64BIT_ONLY   0x1
+#define ID_AA64PFR0_EL1_32BIT_64BIT  0x2
+#define ID_AA64PFR0_EL0_64BIT_ONLY   0x1
+#define ID_AA64PFR0_EL0_32BIT_64BIT  0x2
+
+/* id_aa64pfr1 */
+#define ID_AA64PFR1_MPAMFRAC_SHIFT   16
+#define ID_AA64PFR1_RASFRAC_SHIFT    12
+#define ID_AA64PFR1_MTE_SHIFT        8
+#define ID_AA64PFR1_SSBS_SHIFT       4
+#define ID_AA64PFR1_BT_SHIFT         0
+
+#define ID_AA64PFR1_SSBS_PSTATE_NI    0
+#define ID_AA64PFR1_SSBS_PSTATE_ONLY  1
+#define ID_AA64PFR1_SSBS_PSTATE_INSNS 2
+#define ID_AA64PFR1_BT_BTI            0x1
+
+#define ID_AA64PFR1_MTE_NI           0x0
+#define ID_AA64PFR1_MTE_EL0          0x1
+#define ID_AA64PFR1_MTE              0x2
+
+/* id_aa64zfr0 */
+#define ID_AA64ZFR0_F64MM_SHIFT      56
+#define ID_AA64ZFR0_F32MM_SHIFT      52
+#define ID_AA64ZFR0_I8MM_SHIFT       44
+#define ID_AA64ZFR0_SM4_SHIFT        40
+#define ID_AA64ZFR0_SHA3_SHIFT       32
+#define ID_AA64ZFR0_BF16_SHIFT       20
+#define ID_AA64ZFR0_BITPERM_SHIFT    16
+#define ID_AA64ZFR0_AES_SHIFT        4
+#define ID_AA64ZFR0_SVEVER_SHIFT     0
+
+#define ID_AA64ZFR0_F64MM            0x1
+#define ID_AA64ZFR0_F32MM            0x1
+#define ID_AA64ZFR0_I8MM             0x1
+#define ID_AA64ZFR0_BF16             0x1
+#define ID_AA64ZFR0_SM4              0x1
+#define ID_AA64ZFR0_SHA3             0x1
+#define ID_AA64ZFR0_BITPERM          0x1
+#define ID_AA64ZFR0_AES              0x1
+#define ID_AA64ZFR0_AES_PMULL        0x2
+#define ID_AA64ZFR0_SVEVER_SVE2      0x1
+
+/* id_aa64mmfr0 */
+#define ID_AA64MMFR0_ECV_SHIFT       60
+#define ID_AA64MMFR0_FGT_SHIFT       56
+#define ID_AA64MMFR0_EXS_SHIFT       44
+#define ID_AA64MMFR0_TGRAN4_2_SHIFT  40
+#define ID_AA64MMFR0_TGRAN64_2_SHIFT 36
+#define ID_AA64MMFR0_TGRAN16_2_SHIFT 32
+#define ID_AA64MMFR0_TGRAN4_SHIFT    28
+#define ID_AA64MMFR0_TGRAN64_SHIFT   24
+#define ID_AA64MMFR0_TGRAN16_SHIFT   20
+#define ID_AA64MMFR0_BIGENDEL0_SHIFT 16
+#define ID_AA64MMFR0_SNSMEM_SHIFT    12
+#define ID_AA64MMFR0_BIGENDEL_SHIFT  8
+#define ID_AA64MMFR0_ASID_SHIFT      4
+#define ID_AA64MMFR0_PARANGE_SHIFT   0
+
+#define ID_AA64MMFR0_TGRAN4_NI         0xf
+#define ID_AA64MMFR0_TGRAN4_SUPPORTED  0x0
+#define ID_AA64MMFR0_TGRAN64_NI        0xf
+#define ID_AA64MMFR0_TGRAN64_SUPPORTED 0x0
+#define ID_AA64MMFR0_TGRAN16_NI        0x0
+#define ID_AA64MMFR0_TGRAN16_SUPPORTED 0x1
+#define ID_AA64MMFR0_PARANGE_48        0x5
+#define ID_AA64MMFR0_PARANGE_52        0x6
+
+/* id_aa64mmfr1 */
+#define ID_AA64MMFR1_ETS_SHIFT       36
+#define ID_AA64MMFR1_TWED_SHIFT      32
+#define ID_AA64MMFR1_XNX_SHIFT       28
+#define ID_AA64MMFR1_SPECSEI_SHIFT   24
+#define ID_AA64MMFR1_PAN_SHIFT       20
+#define ID_AA64MMFR1_LOR_SHIFT       16
+#define ID_AA64MMFR1_HPD_SHIFT       12
+#define ID_AA64MMFR1_VHE_SHIFT       8
+#define ID_AA64MMFR1_VMIDBITS_SHIFT  4
+#define ID_AA64MMFR1_HADBS_SHIFT     0
+
+#define ID_AA64MMFR1_VMIDBITS_8      0
+#define ID_AA64MMFR1_VMIDBITS_16     2
+
+/* id_aa64mmfr2 */
+#define ID_AA64MMFR2_E0PD_SHIFT      60
+#define ID_AA64MMFR2_EVT_SHIFT       56
+#define ID_AA64MMFR2_BBM_SHIFT       52
+#define ID_AA64MMFR2_TTL_SHIFT       48
+#define ID_AA64MMFR2_FWB_SHIFT       40
+#define ID_AA64MMFR2_IDS_SHIFT       36
+#define ID_AA64MMFR2_AT_SHIFT        32
+#define ID_AA64MMFR2_ST_SHIFT        28
+#define ID_AA64MMFR2_NV_SHIFT        24
+#define ID_AA64MMFR2_CCIDX_SHIFT     20
+#define ID_AA64MMFR2_LVA_SHIFT       16
+#define ID_AA64MMFR2_IESB_SHIFT      12
+#define ID_AA64MMFR2_LSM_SHIFT       8
+#define ID_AA64MMFR2_UAO_SHIFT       4
+#define ID_AA64MMFR2_CNP_SHIFT       0
+
+/* id_aa64dfr0 */
+#define ID_AA64DFR0_DOUBLELOCK_SHIFT 36
+#define ID_AA64DFR0_PMSVER_SHIFT     32
+#define ID_AA64DFR0_CTX_CMPS_SHIFT   28
+#define ID_AA64DFR0_WRPS_SHIFT       20
+#define ID_AA64DFR0_BRPS_SHIFT       12
+#define ID_AA64DFR0_PMUVER_SHIFT     8
+#define ID_AA64DFR0_TRACEVER_SHIFT   4
+#define ID_AA64DFR0_DEBUGVER_SHIFT   0
+
+#define ID_AA64DFR0_PMUVER_8_0       0x1
+#define ID_AA64DFR0_PMUVER_8_1       0x4
+#define ID_AA64DFR0_PMUVER_8_4       0x5
+#define ID_AA64DFR0_PMUVER_8_5       0x6
+#define ID_AA64DFR0_PMUVER_IMP_DEF   0xf
+
+#define ID_DFR0_PERFMON_SHIFT        24
+
+#define ID_DFR0_PERFMON_8_1          0x4
+
+#define ID_ISAR4_SWP_FRAC_SHIFT        28
+#define ID_ISAR4_PSR_M_SHIFT           24
+#define ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT 20
+#define ID_ISAR4_BARRIER_SHIFT         16
+#define ID_ISAR4_SMC_SHIFT             12
+#define ID_ISAR4_WRITEBACK_SHIFT       8
+#define ID_ISAR4_WITHSHIFTS_SHIFT      4
+#define ID_ISAR4_UNPRIV_SHIFT          0
+
+#define ID_DFR1_MTPMU_SHIFT          0
+
+#define ID_ISAR0_DIVIDE_SHIFT        24
+#define ID_ISAR0_DEBUG_SHIFT         20
+#define ID_ISAR0_COPROC_SHIFT        16
+#define ID_ISAR0_CMPBRANCH_SHIFT     12
+#define ID_ISAR0_BITFIELD_SHIFT      8
+#define ID_ISAR0_BITCOUNT_SHIFT      4
+#define ID_ISAR0_SWAP_SHIFT          0
+
+#define ID_ISAR5_RDM_SHIFT           24
+#define ID_ISAR5_CRC32_SHIFT         16
+#define ID_ISAR5_SHA2_SHIFT          12
+#define ID_ISAR5_SHA1_SHIFT          8
+#define ID_ISAR5_AES_SHIFT           4
+#define ID_ISAR5_SEVL_SHIFT          0
+
+#define ID_ISAR6_I8MM_SHIFT          24
+#define ID_ISAR6_BF16_SHIFT          20
+#define ID_ISAR6_SPECRES_SHIFT       16
+#define ID_ISAR6_SB_SHIFT            12
+#define ID_ISAR6_FHM_SHIFT           8
+#define ID_ISAR6_DP_SHIFT            4
+#define ID_ISAR6_JSCVT_SHIFT         0
+
+#define ID_MMFR0_INNERSHR_SHIFT      28
+#define ID_MMFR0_FCSE_SHIFT          24
+#define ID_MMFR0_AUXREG_SHIFT        20
+#define ID_MMFR0_TCM_SHIFT           16
+#define ID_MMFR0_SHARELVL_SHIFT      12
+#define ID_MMFR0_OUTERSHR_SHIFT      8
+#define ID_MMFR0_PMSA_SHIFT          4
+#define ID_MMFR0_VMSA_SHIFT          0
+
+#define ID_MMFR4_EVT_SHIFT           28
+#define ID_MMFR4_CCIDX_SHIFT         24
+#define ID_MMFR4_LSM_SHIFT           20
+#define ID_MMFR4_HPDS_SHIFT          16
+#define ID_MMFR4_CNP_SHIFT           12
+#define ID_MMFR4_XNX_SHIFT           8
+#define ID_MMFR4_AC2_SHIFT           4
+#define ID_MMFR4_SPECSEI_SHIFT       0
+
+#define ID_MMFR5_ETS_SHIFT           0
+
+#define ID_PFR0_DIT_SHIFT            24
+#define ID_PFR0_CSV2_SHIFT           16
+#define ID_PFR0_STATE3_SHIFT         12
+#define ID_PFR0_STATE2_SHIFT         8
+#define ID_PFR0_STATE1_SHIFT         4
+#define ID_PFR0_STATE0_SHIFT         0
+
+#define ID_DFR0_PERFMON_SHIFT        24
+#define ID_DFR0_MPROFDBG_SHIFT       20
+#define ID_DFR0_MMAPTRC_SHIFT        16
+#define ID_DFR0_COPTRC_SHIFT         12
+#define ID_DFR0_MMAPDBG_SHIFT        8
+#define ID_DFR0_COPSDBG_SHIFT        4
+#define ID_DFR0_COPDBG_SHIFT         0
+
+#define ID_PFR2_SSBS_SHIFT           4
+#define ID_PFR2_CSV3_SHIFT           0
+
+#define MVFR0_FPROUND_SHIFT          28
+#define MVFR0_FPSHVEC_SHIFT          24
+#define MVFR0_FPSQRT_SHIFT           20
+#define MVFR0_FPDIVIDE_SHIFT         16
+#define MVFR0_FPTRAP_SHIFT           12
+#define MVFR0_FPDP_SHIFT             8
+#define MVFR0_FPSP_SHIFT             4
+#define MVFR0_SIMD_SHIFT             0
+
+#define MVFR1_SIMDFMAC_SHIFT         28
+#define MVFR1_FPHP_SHIFT             24
+#define MVFR1_SIMDHP_SHIFT           20
+#define MVFR1_SIMDSP_SHIFT           16
+#define MVFR1_SIMDINT_SHIFT          12
+#define MVFR1_SIMDLS_SHIFT           8
+#define MVFR1_FPDNAN_SHIFT           4
+#define MVFR1_FPFTZ_SHIFT            0
+
+#define ID_PFR1_GIC_SHIFT            28
+#define ID_PFR1_VIRT_FRAC_SHIFT      24
+#define ID_PFR1_SEC_FRAC_SHIFT       20
+#define ID_PFR1_GENTIMER_SHIFT       16
+#define ID_PFR1_VIRTUALIZATION_SHIFT 12
+#define ID_PFR1_MPROGMOD_SHIFT       8
+#define ID_PFR1_SECURITY_SHIFT       4
+#define ID_PFR1_PROGMOD_SHIFT        0
+
+#define MVFR2_FPMISC_SHIFT           4
+#define MVFR2_SIMDMISC_SHIFT         0
+
+#define DCZID_DZP_SHIFT              4
+#define DCZID_BS_SHIFT               0
+
+/*
+ * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
+ * are reserved by the SVE architecture for future expansion of the LEN
+ * field, with compatible semantics.
+ */
+#define ZCR_ELx_LEN_SHIFT            0
+#define ZCR_ELx_LEN_SIZE             9
+#define ZCR_ELx_LEN_MASK             0x1ff
+
 /* Access to system registers */
 
 #define WRITE_SYSREG64(v, name) do {                    \
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:26:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:26:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188026.337084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrB-00031t-Lu; Thu, 16 Sep 2021 06:26:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188026.337084; Thu, 16 Sep 2021 06:26:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrB-00031k-Io; Thu, 16 Sep 2021 06:26:49 +0000
Received: by outflank-mailman (input) for mailman id 188026;
 Thu, 16 Sep 2021 06:26:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrA-0002iA-4a
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:48 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id c23a2476-3a36-4743-adbc-8bd0c054fb5e;
 Thu, 16 Sep 2021 06:26:46 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 245B031B;
 Wed, 15 Sep 2021 23:26:46 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 588D73F59C;
 Wed, 15 Sep 2021 23:26:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c23a2476-3a36-4743-adbc-8bd0c054fb5e
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 2/7] xen/arm: Import ID features sanitize from linux
Date: Thu, 16 Sep 2021 07:25:35 +0100
Message-Id: <c949658a9066a522a4a815e2051943d0a29fdc73.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Import structures declared in Linux file arch/arm64/kernel/cpufeature.c
and the required types from arch/arm64/include/asm/cpufeature.h.

Current code has been imported from Linux 5.13-rc5 (Commit ID
cd1245d75ce93b8fd206f4b34eb58bcfe156d5e9) and copied into cpufeature.c
in arm64 code and cpufeature.h in arm64 specific headers.

Those structure will be used to sanitize the cpu features available to
the ones availble on all cores of a system even if we are on an
heterogeneous platform (from example a big/LITTLE).

For each feature field of all ID registers, those structures define what
is the safest value and if we can allow to have different values in
different cores.

This patch is introducing Linux code without any changes to it.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v4: import arm64_ftr_set_value from Linux
Changes in v3: none
Changes in v2:
- Move add to Makefile to following patch to allow bisection
- Remove GPL text as SPDL is there
- Re-add introduction comment from Linux Kernel file
- Rename cpusanitize.c to cpufeature.c to keep Linux file name
- Move structures imported from linux headers into a new cpufeature.h
header in asm-arm/arm64.
- Move comment about imported code origin to the file header
- Remove not needed linux function declarations instead of removing them
in the following patch
- Add original arm64_ftr_safe_value from Linux
- include kernel.h to use max()
- remove unused ftr_single32 as we will not use it
- remove ctr associated structures that we cannot use (keep the one
defining sanitization bits)
---
 xen/arch/arm/arm64/cpufeature.c        | 514 +++++++++++++++++++++++++
 xen/include/asm-arm/arm64/cpufeature.h | 104 +++++
 2 files changed, 618 insertions(+)
 create mode 100644 xen/arch/arm/arm64/cpufeature.c
 create mode 100644 xen/include/asm-arm/arm64/cpufeature.h

diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
new file mode 100644
index 0000000000..152086dc9d
--- /dev/null
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -0,0 +1,514 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Contains CPU feature definitions
+ *
+ * The following structures have been imported directly from Linux kernel and
+ * should be kept in sync.
+ * The current version has been imported from arch/arm64/kernel/cpufeature.c
+ *  from kernel version 5.13-rc5 together with the required structures and
+ *  macros from arch/arm64/include/asm/cpufeature.h which are stored in
+ *  include/asm-arm/arm64/cpufeature.h
+ *
+ * Copyright (C) 2021 Arm Ltd.
+ * based on code from the Linux kernel, which is:
+ *  Copyright (C) 2015 ARM Ltd.
+ *
+ * A note for the weary kernel hacker: the code here is confusing and hard to
+ * follow! That's partly because it's solving a nasty problem, but also because
+ * there's a little bit of over-abstraction that tends to obscure what's going
+ * on behind a maze of helper functions and macros.
+ *
+ * The basic problem is that hardware folks have started gluing together CPUs
+ * with distinct architectural features; in some cases even creating SoCs where
+ * user-visible instructions are available only on a subset of the available
+ * cores. We try to address this by snapshotting the feature registers of the
+ * boot CPU and comparing these with the feature registers of each secondary
+ * CPU when bringing them up. If there is a mismatch, then we update the
+ * snapshot state to indicate the lowest-common denominator of the feature,
+ * known as the "safe" value. This snapshot state can be queried to view the
+ * "sanitised" value of a feature register.
+ *
+ * The sanitised register values are used to decide which capabilities we
+ * have in the system. These may be in the form of traditional "hwcaps"
+ * advertised to userspace or internal "cpucaps" which are used to configure
+ * things like alternative patching and static keys. While a feature mismatch
+ * may result in a TAINT_CPU_OUT_OF_SPEC kernel taint, a capability mismatch
+ * may prevent a CPU from being onlined at all.
+ *
+ * Some implementation details worth remembering:
+ *
+ * - Mismatched features are *always* sanitised to a "safe" value, which
+ *   usually indicates that the feature is not supported.
+ *
+ * - A mismatched feature marked with FTR_STRICT will cause a "SANITY CHECK"
+ *   warning when onlining an offending CPU and the kernel will be tainted
+ *   with TAINT_CPU_OUT_OF_SPEC.
+ *
+ * - Features marked as FTR_VISIBLE have their sanitised value visible to
+ *   userspace. FTR_VISIBLE features in registers that are only visible
+ *   to EL0 by trapping *must* have a corresponding HWCAP so that late
+ *   onlining of CPUs cannot lead to features disappearing at runtime.
+ *
+ * - A "feature" is typically a 4-bit register field. A "capability" is the
+ *   high-level description derived from the sanitised field value.
+ *
+ * - Read the Arm ARM (DDI 0487F.a) section D13.1.3 ("Principles of the ID
+ *   scheme for fields in ID registers") to understand when feature fields
+ *   may be signed or unsigned (FTR_SIGNED and FTR_UNSIGNED accordingly).
+ *
+ * - KVM exposes its own view of the feature registers to guest operating
+ *   systems regardless of FTR_VISIBLE. This is typically driven from the
+ *   sanitised register values to allow virtual CPUs to be migrated between
+ *   arbitrary physical CPUs, but some features not present on the host are
+ *   also advertised and emulated. Look at sys_reg_descs[] for the gory
+ *   details.
+ *
+ * - If the arm64_ftr_bits[] for a register has a missing field, then this
+ *   field is treated as STRICT RES0, including for read_sanitised_ftr_reg().
+ *   This is stronger than FTR_HIDDEN and can be used to hide features from
+ *   KVM guests.
+ */
+
+#include <xen/types.h>
+#include <xen/kernel.h>
+#include <asm/sysregs.h>
+#include <asm/cpufeature.h>
+#include <asm/arm64/cpufeature.h>
+
+#define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+	{						\
+		.sign = SIGNED,				\
+		.visible = VISIBLE,			\
+		.strict = STRICT,			\
+		.type = TYPE,				\
+		.shift = SHIFT,				\
+		.width = WIDTH,				\
+		.safe_val = SAFE_VAL,			\
+	}
+
+/* Define a feature with unsigned values */
+#define ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+	__ARM64_FTR_BITS(FTR_UNSIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
+
+/* Define a feature with a signed value */
+#define S_ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
+	__ARM64_FTR_BITS(FTR_SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
+
+#define ARM64_FTR_END					\
+	{						\
+		.width = 0,				\
+	}
+
+/*
+ * NOTE: Any changes to the visibility of features should be kept in
+ * sync with the documentation of the CPU feature register ABI.
+ */
+static const struct arm64_ftr_bits ftr_id_aa64isar0[] = {
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RNDR_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TLB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_FHM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_DP_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM4_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RDM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_ATOMICS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_CRC32_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA1_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_AES_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_I8MM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DGH_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_BF16_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SPECRES_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPA_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_API_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
+		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_APA_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_MPAM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SEL2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+				   FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
+	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
+	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_SHIFT, 4, ID_AA64PFR0_EL1_64BIT_ONLY),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL0_SHIFT, 4, ID_AA64PFR0_EL0_64BIT_ONLY),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MPAMFRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_RASFRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_BTI),
+				    FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_BT_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = {
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F64MM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F32MM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_I8MM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BF16_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
+		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
+	/*
+	 * Page size not being supported at Stage-2 is not fatal. You
+	 * just give up KVM if PAGE_SIZE isn't supported there. Go fix
+	 * your favourite nesting hypervisor.
+	 *
+	 * There is a small corner case where the hypervisor explicitly
+	 * advertises a given granule size at Stage-2 (value 2) on some
+	 * vCPUs, and uses the fallback to Stage-1 (value 0) for other
+	 * vCPUs. Although this is not forbidden by the architecture, it
+	 * indicates that the hypervisor is being silly (or buggy).
+	 *
+	 * We make no effort to cope with this and pretend that if these
+	 * fields are inconsistent across vCPUs, then it isn't worth
+	 * trying to bring KVM up.
+	 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN4_2_SHIFT, 4, 1),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN64_2_SHIFT, 4, 1),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN16_2_SHIFT, 4, 1),
+	/*
+	 * We already refuse to boot CPUs that don't support our configured
+	 * page size, so we can only detect mismatches for a page size other
+	 * than the one we're currently using. Unfortunately, SoCs like this
+	 * exist in the wild so, even though we don't like it, we'll have to go
+	 * along with it and treat them as non-strict.
+	 */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN16_SHIFT, 4, ID_AA64MMFR0_TGRAN16_NI),
+
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL0_SHIFT, 4, 0),
+	/* Linux shouldn't care about secure memory */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_SNSMEM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ASID_SHIFT, 4, 0),
+	/*
+	 * Differing PARange is fine as long as all peripherals and memory are mapped
+	 * within the minimum PARange of all CPUs
+	 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_PARANGE_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VHE_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VMIDBITS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HADBS_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_UAO_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CNP_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_ctr[] = {
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IDC_SHIFT, 1, 1),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_CWG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_ERG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DMINLINE_SHIFT, 4, 1),
+	/*
+	 * Linux can handle differing I-cache policies. Userspace JITs will
+	 * make use of *minLine.
+	 * If we have differing I-cache policies, report it as the weakest - VIPT.
+	 */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT),	/* L1Ip */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_BRPS_SHIFT, 4, 0),
+	/*
+	 * We can instantiate multiple PMU instances with different levels
+	 * of support.
+	 */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_mvfr2[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_dczid[] = {
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar0[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DEBUG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_COPROC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_CMPBRANCH_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITFIELD_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITCOUNT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_SWAP_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar5[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_RDM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_CRC32_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA1_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_AES_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SEVL_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr4[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_EVT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CCIDX_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_LSM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
+
+	/*
+	 * SpecSEI = 1 indicates that the PE might generate an SError on an
+	 * external abort on speculative read. It is safe to assume that an
+	 * SError might be generated than it will not be. Hence it has been
+	 * classified as FTR_HIGHER_SAFE.
+	 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_MMFR4_SPECSEI_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar4[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SWP_FRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_PSR_M_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_BARRIER_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SMC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WRITEBACK_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WITHSHIFTS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_UNPRIV_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_mmfr5[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR5_ETS_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_isar6[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_I8MM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_BF16_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SPECRES_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_FHM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_DP_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_JSCVT_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr0[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr1[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GIC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRT_FRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SEC_FRAC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GENTIMER_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRTUALIZATION_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_MPROGMOD_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SECURITY_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_PROGMOD_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_pfr2[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_SSBS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_CSV3_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_dfr0[] = {
+	/* [31:28] TraceFilt */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_id_dfr1[] = {
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_MTPMU_SHIFT, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_zcr[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
+		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
+	ARM64_FTR_END,
+};
+
+/*
+ * Common ftr bits for a 32bit register with all hidden, strict
+ * attributes, with 4bit feature fields and a default safe value of
+ * 0. Covers the following 32bit registers:
+ * id_isar[1-4], id_mmfr[1-3], id_pfr1, mvfr[0-1]
+ */
+static const struct arm64_ftr_bits ftr_generic_32bits[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
+	ARM64_FTR_END,
+};
+
+static const struct arm64_ftr_bits ftr_raz[] = {
+	ARM64_FTR_END,
+};
+
+static u64 arm64_ftr_set_value(const struct arm64_ftr_bits *ftrp, s64 reg,
+			       s64 ftr_val)
+{
+	u64 mask = arm64_ftr_mask(ftrp);
+
+	reg &= ~mask;
+	reg |= (ftr_val << ftrp->shift) & mask;
+	return reg;
+}
+
+static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
+				s64 cur)
+{
+	s64 ret = 0;
+
+	switch (ftrp->type) {
+	case FTR_EXACT:
+		ret = ftrp->safe_val;
+		break;
+	case FTR_LOWER_SAFE:
+		ret = min(new, cur);
+		break;
+	case FTR_HIGHER_OR_ZERO_SAFE:
+		if (!cur || !new)
+			break;
+		fallthrough;
+	case FTR_HIGHER_SAFE:
+		ret = max(new, cur);
+		break;
+	default:
+		BUG();
+	}
+
+	return ret;
+}
+
+/*
+ * End of imported linux structures and code
+ */
+
diff --git a/xen/include/asm-arm/arm64/cpufeature.h b/xen/include/asm-arm/arm64/cpufeature.h
new file mode 100644
index 0000000000..d9b9fa77cb
--- /dev/null
+++ b/xen/include/asm-arm/arm64/cpufeature.h
@@ -0,0 +1,104 @@
+#ifndef __ASM_ARM_ARM64_CPUFEATURES_H
+#define __ASM_ARM_ARM64_CPUFEATURES_H
+
+/*
+ * CPU feature register tracking
+ *
+ * The safe value of a CPUID feature field is dependent on the implications
+ * of the values assigned to it by the architecture. Based on the relationship
+ * between the values, the features are classified into 3 types - LOWER_SAFE,
+ * HIGHER_SAFE and EXACT.
+ *
+ * The lowest value of all the CPUs is chosen for LOWER_SAFE and highest
+ * for HIGHER_SAFE. It is expected that all CPUs have the same value for
+ * a field when EXACT is specified, failing which, the safe value specified
+ * in the table is chosen.
+ */
+
+enum ftr_type {
+	FTR_EXACT,			/* Use a predefined safe value */
+	FTR_LOWER_SAFE,			/* Smaller value is safe */
+	FTR_HIGHER_SAFE,		/* Bigger value is safe */
+	FTR_HIGHER_OR_ZERO_SAFE,	/* Bigger value is safe, but 0 is biggest */
+};
+
+#define FTR_STRICT	true	/* SANITY check strict matching required */
+#define FTR_NONSTRICT	false	/* SANITY check ignored */
+
+#define FTR_SIGNED	true	/* Value should be treated as signed */
+#define FTR_UNSIGNED	false	/* Value should be treated as unsigned */
+
+#define FTR_VISIBLE	true	/* Feature visible to the user space */
+#define FTR_HIDDEN	false	/* Feature is hidden from the user */
+
+#define FTR_VISIBLE_IF_IS_ENABLED(config)		\
+	(IS_ENABLED(config) ? FTR_VISIBLE : FTR_HIDDEN)
+
+struct arm64_ftr_bits {
+	bool		sign;	/* Value is signed ? */
+	bool		visible;
+	bool		strict;	/* CPU Sanity check: strict matching required ? */
+	enum ftr_type	type;
+	u8		shift;
+	u8		width;
+	s64		safe_val; /* safe value for FTR_EXACT features */
+};
+
+static inline int __attribute_const__
+cpuid_feature_extract_signed_field_width(u64 features, int field, int width)
+{
+	return (s64)(features << (64 - width - field)) >> (64 - width);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_signed_field(u64 features, int field)
+{
+	return cpuid_feature_extract_signed_field_width(features, field, 4);
+}
+
+static inline unsigned int __attribute_const__
+cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width)
+{
+	return (u64)(features << (64 - width - field)) >> (64 - width);
+}
+
+static inline unsigned int __attribute_const__
+cpuid_feature_extract_unsigned_field(u64 features, int field)
+{
+	return cpuid_feature_extract_unsigned_field_width(features, field, 4);
+}
+
+static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
+{
+	return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign)
+{
+	return (sign) ?
+		cpuid_feature_extract_signed_field_width(features, field, width) :
+		cpuid_feature_extract_unsigned_field_width(features, field, width);
+}
+
+static inline int __attribute_const__
+cpuid_feature_extract_field(u64 features, int field, bool sign)
+{
+	return cpuid_feature_extract_field_width(features, field, 4, sign);
+}
+
+static inline s64 arm64_ftr_value(const struct arm64_ftr_bits *ftrp, u64 val)
+{
+	return (s64)cpuid_feature_extract_field_width(val, ftrp->shift, ftrp->width, ftrp->sign);
+}
+
+#endif /* _ASM_ARM_ARM64_CPUFEATURES_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:26:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:26:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188028.337096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrE-0003KK-4O; Thu, 16 Sep 2021 06:26:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188028.337096; Thu, 16 Sep 2021 06:26:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrE-0003KB-04; Thu, 16 Sep 2021 06:26:52 +0000
Received: by outflank-mailman (input) for mailman id 188028;
 Thu, 16 Sep 2021 06:26:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrC-0002iQ-Pj
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:50 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 127c74c8-16b7-11ec-b5a6-12813bfff9fa;
 Thu, 16 Sep 2021 06:26:48 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7429C31B;
 Wed, 15 Sep 2021 23:26:48 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3CA053F59C;
 Wed, 15 Sep 2021 23:26:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 127c74c8-16b7-11ec-b5a6-12813bfff9fa
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 4/7] xen/arm: Sanitize cpuinfo ID registers fields
Date: Thu, 16 Sep 2021 07:25:37 +0100
Message-Id: <3f65e7f58a422d41d89b879409418b9c5f2f84b9.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Define a sanitize_cpu function to be called on secondary cores to
sanitize the system cpuinfo structure.

The safest value is taken when possible and the system is marked tainted
if we encounter values which are incompatible with each other.

Call the update_system_features function on all secondary cores that are
kept running and taint the system if different midr are found between
cores but hmp-unsafe=true was passed on Xen command line.

This is only supported on arm64 so update_system_features is an empty
static inline on arm32.

The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
Xen is running on a system with features differences between cores which
are not supported.

The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
as this patch is not handling sanitization of those registers.
CTR_EL0/DCZID will be handled in a future patch to properly handle
different cache attributes when possible.
ZCR should be sanitize once we add support for SVE in Xen.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v4:
- arm64_ftr_set_value
Changes in v3:
- in case of different midr but hmp-unsafe passed on the command line,
enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
- use current core info to sanitize cpu only if we keep it on
Changes in v2:
- add compilation of cpufeature.c in this patch instead of previous one
- remove functions reused from linux code and moved to header
- rename sanitize_cpu to update_system_features
- change to Linux coding style
- remove dev comments
- surround currently not used Linux structures with #if 0 and adapt the
commit message
- add missing aa64dfr1 register
- add TODO for CTR, DCZID and ZCR
- add CPU_OUT_OF_SPEC support to print_taint
- use system_cpuinfo instead of boot_cpu_data
---
 xen/arch/arm/arm64/Makefile      |   1 +
 xen/arch/arm/arm64/cpufeature.c  | 117 +++++++++++++++++++++++++++++++
 xen/arch/arm/smpboot.c           |  34 ++++++---
 xen/common/kernel.c              |   6 +-
 xen/include/asm-arm/cpufeature.h |   9 +++
 xen/include/xen/lib.h            |   1 +
 6 files changed, 158 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
index 40642ff574..701d66883d 100644
--- a/xen/arch/arm/arm64/Makefile
+++ b/xen/arch/arm/arm64/Makefile
@@ -1,6 +1,7 @@
 obj-y += lib/
 
 obj-y += cache.o
+obj-y += cpufeature.o
 obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
 obj-$(CONFIG_EARLY_PRINTK) += debug.o
 obj-y += domctl.o
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index 152086dc9d..58596495a8 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
 	ARM64_FTR_END,
 };
 
+#if 0
+/* TODO: use this to sanitize the cache line size among cores */
+
 static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
+#endif
 
 static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
 	ARM64_FTR_END,
 };
 
+#if 0
+/* TODO: handle this when sanitizing cache related registers */
 static const struct arm64_ftr_bits ftr_dczid[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
+#endif
 
 static const struct arm64_ftr_bits ftr_id_isar0[] = {
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
@@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
 	ARM64_FTR_END,
 };
 
+#if 0
+/* TODO: use this to sanitize SVE once we support it */
+
 static const struct arm64_ftr_bits ftr_zcr[] = {
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
 		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
 	ARM64_FTR_END,
 };
+#endif
 
 /*
  * Common ftr bits for a 32bit register with all hidden, strict
@@ -512,3 +523,109 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
  * End of imported linux structures and code
  */
 
+static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
+						const struct arm64_ftr_bits *ftrp)
+{
+	int taint = 0;
+	u64 old_reg = *cur_reg;
+
+	for (;ftrp->width != 0;ftrp++)
+	{
+		s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
+		s64 new_field = arm64_ftr_value(ftrp, new_reg);
+
+		if (cur_field == new_field)
+			continue;
+
+		if (ftrp->strict)
+			taint = 1;
+
+		*cur_reg = arm64_ftr_set_value(ftrp, *cur_reg,
+							arm64_ftr_safe_value(ftrp, new_field, cur_field));
+	}
+
+	if (old_reg != new_reg)
+		printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
+				reg_name, old_reg, new_reg);
+	if (old_reg != *cur_reg)
+		printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
+				reg_name, old_reg, *cur_reg);
+
+	if (taint)
+	{
+		printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
+				reg_name);
+		add_taint(TAINT_CPU_OUT_OF_SPEC);
+	}
+}
+
+
+/*
+ * This function should be called on secondary cores to sanitize the boot cpu
+ * cpuinfo.
+ */
+void update_system_features(const struct cpuinfo_arm *new)
+{
+
+#define SANITIZE_REG(field, num, reg)  \
+	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+				 #reg, ftr_##reg)
+
+#define SANITIZE_ID_REG(field, num, reg)  \
+	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+				#reg, ftr_id_##reg)
+
+#define SANITIZE_RAZ_REG(field, num, reg)  \
+	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+				#reg, ftr_raz)
+
+#define SANITIZE_GENERIC_REG(field, num, reg)  \
+	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
+				#reg, ftr_generic_32bits)
+
+	SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
+	SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
+
+	SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
+	SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
+
+	SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
+	SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
+	SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
+
+	SANITIZE_ID_REG(isa64, 0, aa64isar0);
+	SANITIZE_ID_REG(isa64, 1, aa64isar1);
+
+	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
+
+	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
+	{
+		SANITIZE_ID_REG(pfr32, 0, pfr0);
+		SANITIZE_ID_REG(pfr32, 1, pfr1);
+		SANITIZE_ID_REG(pfr32, 2, pfr2);
+
+		SANITIZE_ID_REG(dbg32, 0, dfr0);
+		SANITIZE_ID_REG(dbg32, 1, dfr1);
+
+		SANITIZE_ID_REG(mm32, 0, mmfr0);
+		SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
+		SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
+		SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
+		SANITIZE_ID_REG(mm32, 4, mmfr4);
+		SANITIZE_ID_REG(mm32, 5, mmfr5);
+
+		SANITIZE_ID_REG(isa32, 0, isar0);
+		SANITIZE_GENERIC_REG(isa32, 1, isar1);
+		SANITIZE_GENERIC_REG(isa32, 2, isar2);
+		SANITIZE_GENERIC_REG(isa32, 3, isar3);
+		SANITIZE_ID_REG(isa32, 4, isar4);
+		SANITIZE_ID_REG(isa32, 5, isar5);
+		SANITIZE_ID_REG(isa32, 6, isar6);
+
+		SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
+		SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
+#ifndef MVFR2_MAYBE_UNDEFINED
+		SANITIZE_REG(mvfr, 2, mvfr2);
+#endif
+	}
+}
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index c9f2827d56..60c0e82fc5 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -318,15 +318,26 @@ void start_secondary(void)
      * is manually specified for all domains). Better to park them for
      * now.
      */
-    if ( !opt_hmp_unsafe &&
-         current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
+    if ( current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
     {
-        printk(XENLOG_ERR
-               "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
-               XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
-               smp_processor_id(), current_cpu_data.midr.bits,
-               system_cpuinfo.midr.bits);
-        stop_cpu();
+        if ( !opt_hmp_unsafe )
+        {
+            printk(XENLOG_ERR
+                   "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
+                   XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
+                   smp_processor_id(), current_cpu_data.midr.bits,
+                   system_cpuinfo.midr.bits);
+            stop_cpu();
+        }
+        else
+        {
+            printk(XENLOG_ERR
+                   "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
+                   XENLOG_ERR "hmp-unsafe turned on so tainting Xen and keep core on!!\n",
+                   smp_processor_id(), current_cpu_data.midr.bits,
+                   system_cpuinfo.midr.bits);
+            add_taint(TAINT_CPU_OUT_OF_SPEC);
+         }
     }
 
     if ( dcache_line_bytes != read_dcache_line_bytes() )
@@ -337,6 +348,13 @@ void start_secondary(void)
         stop_cpu();
     }
 
+    /*
+     * system features must be updated only if we do not stop the core or
+     * we might disable features due to a non used core (for example when
+     * booting on big cores on a big.LITTLE system with hmp_unsafe)
+     */
+    update_system_features(&current_cpu_data);
+
     mmu_init_secondary_cpu();
 
     gic_init_secondary_cpu();
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index d77756a81e..e119e5401f 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -327,6 +327,7 @@ unsigned int tainted;
  *  'H' - HVM forced emulation prefix is permitted.
  *  'M' - Machine had a machine check experience.
  *  'U' - Platform is unsecure (usually due to an errata on the platform).
+ *  'S' - Out of spec CPU (One core has a feature incompatible with others).
  *
  *      The string is overwritten by the next call to print_taint().
  */
@@ -334,12 +335,13 @@ char *print_tainted(char *str)
 {
     if ( tainted )
     {
-        snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c",
+        snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c%c",
                  tainted & TAINT_MACHINE_UNSECURE ? 'U' : ' ',
                  tainted & TAINT_MACHINE_CHECK ? 'M' : ' ',
                  tainted & TAINT_SYNC_CONSOLE ? 'C' : ' ',
                  tainted & TAINT_ERROR_INJECT ? 'E' : ' ',
-                 tainted & TAINT_HVM_FEP ? 'H' : ' ');
+                 tainted & TAINT_HVM_FEP ? 'H' : ' ',
+                 tainted & TAINT_CPU_OUT_OF_SPEC ? 'S' : ' ');
     }
     else
     {
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 8f2b8e7830..52cb3133e0 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -330,6 +330,15 @@ extern struct cpuinfo_arm system_cpuinfo;
 
 extern void identify_cpu(struct cpuinfo_arm *);
 
+#ifdef CONFIG_ARM_64
+extern void update_system_features(const struct cpuinfo_arm *);
+#else
+static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
+{
+    /* Not supported on arm32 */
+}
+#endif
+
 extern struct cpuinfo_arm cpu_data[];
 #define current_cpu_data cpu_data[smp_processor_id()]
 
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 1198c7c0b2..c6987973bf 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -192,6 +192,7 @@ uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c);
 #define TAINT_ERROR_INJECT              (1u << 2)
 #define TAINT_HVM_FEP                   (1u << 3)
 #define TAINT_MACHINE_UNSECURE          (1u << 4)
+#define TAINT_CPU_OUT_OF_SPEC           (1u << 5)
 extern unsigned int tainted;
 #define TAINT_STRING_MAX_LEN            20
 extern char *print_tainted(char *str);
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:26:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:26:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188029.337107 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrG-0003dT-GD; Thu, 16 Sep 2021 06:26:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188029.337107; Thu, 16 Sep 2021 06:26:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrG-0003dD-BL; Thu, 16 Sep 2021 06:26:54 +0000
Received: by outflank-mailman (input) for mailman id 188029;
 Thu, 16 Sep 2021 06:26:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrF-0002iA-4z
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:53 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id d9c5e478-2216-464c-b853-947ae1c731da;
 Thu, 16 Sep 2021 06:26:47 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0A1771042;
 Wed, 15 Sep 2021 23:26:47 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 56FFA3F59C;
 Wed, 15 Sep 2021 23:26:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9c5e478-2216-464c-b853-947ae1c731da
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 3/7] xen/arm: Rename cpu_boot_data to system_cpuinfo
Date: Thu, 16 Sep 2021 07:25:36 +0100
Message-Id: <9ca3b3dd55662068517774b34d3ed32019383152.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

As we will sanitize the content of boot_cpu_data it will not really
contain the boot cpu information but the system sanitize information.
Rename the structure to system_cpuinfo so the user is informed that this
is the system wide available feature and not anymore the features of the
boot cpu.
The original boot cpu data is still available in cpu_data.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes in v4: none
Changes in v3: none
Changes in v2:
  - patch introduced in v2
---
 xen/arch/arm/cpufeature.c        |  8 ++------
 xen/arch/arm/setup.c             | 34 ++++++++++++++++++--------------
 xen/arch/arm/smpboot.c           |  6 +++---
 xen/include/asm-arm/cpufeature.h |  6 +++---
 4 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 1d88783809..f600a611bd 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -169,12 +169,8 @@ void identify_cpu(struct cpuinfo_arm *c)
  */
 static int __init create_guest_cpuinfo(void)
 {
-    /*
-     * TODO: The code is currently using only the features detected on the boot
-     * core. In the long term we should try to compute values containing only
-     * features supported by all cores.
-     */
-    guest_cpuinfo = boot_cpu_data;
+    /* Use the sanitized cpuinfo as initial guest cpuinfo */
+    guest_cpuinfo = system_cpuinfo;
 
 #ifdef CONFIG_ARM_64
     /* Hide MPAM support as xen does not support it */
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 5be7f2b0c2..4ab13d0fbe 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -56,7 +56,11 @@
 
 struct bootinfo __initdata bootinfo;
 
-struct cpuinfo_arm __read_mostly boot_cpu_data;
+/*
+ * Sanitized version of cpuinfo containing only features available on all
+ * cores (only on arm64 as there is no sanitization support on arm32).
+ */
+struct cpuinfo_arm __read_mostly system_cpuinfo;
 
 #ifdef CONFIG_ACPI
 bool __read_mostly acpi_disabled;
@@ -100,7 +104,7 @@ static const char * __initdata processor_implementers[] = {
 static void __init processor_id(void)
 {
     const char *implementer = "Unknown";
-    struct cpuinfo_arm *c = &boot_cpu_data;
+    struct cpuinfo_arm *c = &system_cpuinfo;
 
     identify_cpu(c);
     current_cpu_data = *c;
@@ -120,7 +124,7 @@ static void __init processor_id(void)
 #if defined(CONFIG_ARM_64)
     printk("64-bit Execution:\n");
     printk("  Processor Features: %016"PRIx64" %016"PRIx64"\n",
-           boot_cpu_data.pfr64.bits[0], boot_cpu_data.pfr64.bits[1]);
+           system_cpuinfo.pfr64.bits[0], system_cpuinfo.pfr64.bits[1]);
     printk("    Exception Levels: EL3:%s EL2:%s EL1:%s EL0:%s\n",
            cpu_has_el3_32 ? "64+32" : cpu_has_el3_64 ? "64" : "No",
            cpu_has_el2_32 ? "64+32" : cpu_has_el2_64 ? "64" : "No",
@@ -144,13 +148,13 @@ static void __init processor_id(void)
                boot_cpu_feature64(simd));
 
     printk("  Debug Features: %016"PRIx64" %016"PRIx64"\n",
-           boot_cpu_data.dbg64.bits[0], boot_cpu_data.dbg64.bits[1]);
+           system_cpuinfo.dbg64.bits[0], system_cpuinfo.dbg64.bits[1]);
     printk("  Auxiliary Features: %016"PRIx64" %016"PRIx64"\n",
-           boot_cpu_data.aux64.bits[0], boot_cpu_data.aux64.bits[1]);
+           system_cpuinfo.aux64.bits[0], system_cpuinfo.aux64.bits[1]);
     printk("  Memory Model Features: %016"PRIx64" %016"PRIx64"\n",
-           boot_cpu_data.mm64.bits[0], boot_cpu_data.mm64.bits[1]);
+           system_cpuinfo.mm64.bits[0], system_cpuinfo.mm64.bits[1]);
     printk("  ISA Features:  %016"PRIx64" %016"PRIx64"\n",
-           boot_cpu_data.isa64.bits[0], boot_cpu_data.isa64.bits[1]);
+           system_cpuinfo.isa64.bits[0], system_cpuinfo.isa64.bits[1]);
 #endif
 
     /*
@@ -161,7 +165,7 @@ static void __init processor_id(void)
     {
         printk("32-bit Execution:\n");
         printk("  Processor Features: %"PRIregister":%"PRIregister"\n",
-               boot_cpu_data.pfr32.bits[0], boot_cpu_data.pfr32.bits[1]);
+               system_cpuinfo.pfr32.bits[0], system_cpuinfo.pfr32.bits[1]);
         printk("    Instruction Sets:%s%s%s%s%s%s\n",
                cpu_has_aarch32 ? " AArch32" : "",
                cpu_has_arm ? " A32" : "",
@@ -174,18 +178,18 @@ static void __init processor_id(void)
                cpu_has_security ? " Security" : "");
 
         printk("  Debug Features: %"PRIregister"\n",
-               boot_cpu_data.dbg32.bits[0]);
+               system_cpuinfo.dbg32.bits[0]);
         printk("  Auxiliary Features: %"PRIregister"\n",
-               boot_cpu_data.aux32.bits[0]);
+               system_cpuinfo.aux32.bits[0]);
         printk("  Memory Model Features: %"PRIregister" %"PRIregister"\n"
                "                         %"PRIregister" %"PRIregister"\n",
-               boot_cpu_data.mm32.bits[0], boot_cpu_data.mm32.bits[1],
-               boot_cpu_data.mm32.bits[2], boot_cpu_data.mm32.bits[3]);
+               system_cpuinfo.mm32.bits[0], system_cpuinfo.mm32.bits[1],
+               system_cpuinfo.mm32.bits[2], system_cpuinfo.mm32.bits[3]);
         printk("  ISA Features: %"PRIregister" %"PRIregister" %"PRIregister"\n"
                "                %"PRIregister" %"PRIregister" %"PRIregister"\n",
-               boot_cpu_data.isa32.bits[0], boot_cpu_data.isa32.bits[1],
-               boot_cpu_data.isa32.bits[2], boot_cpu_data.isa32.bits[3],
-               boot_cpu_data.isa32.bits[4], boot_cpu_data.isa32.bits[5]);
+               system_cpuinfo.isa32.bits[0], system_cpuinfo.isa32.bits[1],
+               system_cpuinfo.isa32.bits[2], system_cpuinfo.isa32.bits[3],
+               system_cpuinfo.isa32.bits[4], system_cpuinfo.isa32.bits[5]);
     }
     else
     {
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a1ee3146ef..c9f2827d56 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -124,7 +124,7 @@ static void __init dt_smp_init_cpus(void)
     bool bootcpu_valid = false;
     int rc;
 
-    mpidr = boot_cpu_data.mpidr.bits & MPIDR_HWID_MASK;
+    mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
 
     if ( !cpus )
     {
@@ -319,13 +319,13 @@ void start_secondary(void)
      * now.
      */
     if ( !opt_hmp_unsafe &&
-         current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
+         current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
     {
         printk(XENLOG_ERR
                "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
                XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
                smp_processor_id(), current_cpu_data.midr.bits,
-               boot_cpu_data.midr.bits);
+               system_cpuinfo.midr.bits);
         stop_cpu();
     }
 
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index ba48db3eac..8f2b8e7830 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -3,7 +3,7 @@
 
 #ifdef CONFIG_ARM_64
 #define cpu_feature64(c, feat)         ((c)->pfr64.feat)
-#define boot_cpu_feature64(feat)       (boot_cpu_data.pfr64.feat)
+#define boot_cpu_feature64(feat)       (system_cpuinfo.pfr64.feat)
 
 #define cpu_feature64_has_el0_32(c)    (cpu_feature64(c, el0) == 2)
 
@@ -21,7 +21,7 @@
 #endif
 
 #define cpu_feature32(c, feat)         ((c)->pfr32.feat)
-#define boot_cpu_feature32(feat)       (boot_cpu_data.pfr32.feat)
+#define boot_cpu_feature32(feat)       (system_cpuinfo.pfr32.feat)
 
 #define cpu_has_arm       (boot_cpu_feature32(arm) == 1)
 #define cpu_has_thumb     (boot_cpu_feature32(thumb) >= 1)
@@ -326,7 +326,7 @@ struct cpuinfo_arm {
     } mvfr;
 };
 
-extern struct cpuinfo_arm boot_cpu_data;
+extern struct cpuinfo_arm system_cpuinfo;
 
 extern void identify_cpu(struct cpuinfo_arm *);
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:27:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:27:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188031.337118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrL-00045D-Qx; Thu, 16 Sep 2021 06:26:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188031.337118; Thu, 16 Sep 2021 06:26:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrL-000456-M7; Thu, 16 Sep 2021 06:26:59 +0000
Received: by outflank-mailman (input) for mailman id 188031;
 Thu, 16 Sep 2021 06:26:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrK-0002iA-50
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:26:58 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 8d67f46b-a11a-442c-ad85-6366e62b5814;
 Thu, 16 Sep 2021 06:26:49 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 596711042;
 Wed, 15 Sep 2021 23:26:49 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A72E23F59C;
 Wed, 15 Sep 2021 23:26:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d67f46b-a11a-442c-ad85-6366e62b5814
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 5/7] xen/arm: Use sanitize values for p2m
Date: Thu, 16 Sep 2021 07:25:38 +0100
Message-Id: <927ce40a9a5d3c56e207275e0b4147840017acc9.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Replace the code in p2m trying to find a sane value for the VMID size
supported and the PAR to use. We are now using the boot cpuinfo as the
values there are sanitized during boot and the value for those
parameters is now the safest possible value on the system.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes in v4: none
Changes in v3: drop arm32 mention in commmit message
Changes in v2:
 - use system_cpuinfo
---
 xen/arch/arm/p2m.c | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 6e01e83967..8b20b43077 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -2042,31 +2042,21 @@ void __init setup_virt_paging(void)
         [7] = { 0 }  /* Invalid */
     };
 
-    unsigned int i, cpu;
+    unsigned int i;
     unsigned int pa_range = 0x10; /* Larger than any possible value */
-    bool vmid_8_bit = false;
-
-    for_each_online_cpu ( cpu )
-    {
-        const struct cpuinfo_arm *info = &cpu_data[cpu];
 
-        /*
-         * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
-         * with IPA bits == PA bits, compare against "pabits".
-         */
-        if ( pa_range_info[info->mm64.pa_range].pabits < p2m_ipa_bits )
-            p2m_ipa_bits = pa_range_info[info->mm64.pa_range].pabits;
-
-        /* Set a flag if the current cpu does not support 16 bit VMIDs. */
-        if ( info->mm64.vmid_bits != MM64_VMID_16_BITS_SUPPORT )
-            vmid_8_bit = true;
-    }
+    /*
+     * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
+     * with IPA bits == PA bits, compare against "pabits".
+     */
+    if ( pa_range_info[system_cpuinfo.mm64.pa_range].pabits < p2m_ipa_bits )
+        p2m_ipa_bits = pa_range_info[system_cpuinfo.mm64.pa_range].pabits;
 
     /*
-     * If the flag is not set then it means all CPUs support 16-bit
-     * VMIDs.
+     * cpu info sanitization made sure we support 16bits VMID only if all
+     * cores are supporting it.
      */
-    if ( !vmid_8_bit )
+    if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
         max_vmid = MAX_VMID_16_BIT;
 
     /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:27:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:27:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188032.337129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrR-0004vs-6f; Thu, 16 Sep 2021 06:27:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188032.337129; Thu, 16 Sep 2021 06:27:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrR-0004vf-0f; Thu, 16 Sep 2021 06:27:05 +0000
Received: by outflank-mailman (input) for mailman id 188032;
 Thu, 16 Sep 2021 06:27:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrP-0002iA-5L
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:27:03 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 65db3c8c-0d03-4144-acff-4b6d04223061;
 Thu, 16 Sep 2021 06:26:50 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3D4521042;
 Wed, 15 Sep 2021 23:26:50 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8C99B3F59C;
 Wed, 15 Sep 2021 23:26:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65db3c8c-0d03-4144-acff-4b6d04223061
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 6/7] xen/arm: Taint Xen on incompatible DCZID values
Date: Thu, 16 Sep 2021 07:25:39 +0100
Message-Id: <fe9e2e803318ba2f1a8d16bae509c0c32843097f.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Use arm64 cpu feature sanitization to TAINT Xen if different DCZID values
are found (ftr_dczid is using only STRICT method).
In this case actual memory being cleaned by DC ZVA operations would be
different depending on the cores which could make a guest zeroing too
much or too little memory if it is merged between CPUs.

We could, on processors supporting it, trap access to DCZID_EL0 register
using HFGRTR_EL2 register but this would not solve the case where a
process is being migrated during a copy or if it cached the value of the
register.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Change in v4; Fix typos in commit message
Change in v3: none
Change in v2: Patch introduced in v2
---
 xen/arch/arm/arm64/cpufeature.c  | 14 +++++++++++---
 xen/arch/arm/cpufeature.c        |  2 ++
 xen/include/asm-arm/cpufeature.h |  8 ++++++++
 3 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index 58596495a8..d4679f5df3 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -329,14 +329,11 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
 	ARM64_FTR_END,
 };
 
-#if 0
-/* TODO: handle this when sanitizing cache related registers */
 static const struct arm64_ftr_bits ftr_dczid[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
-#endif
 
 static const struct arm64_ftr_bits ftr_id_isar0[] = {
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
@@ -598,6 +595,17 @@ void update_system_features(const struct cpuinfo_arm *new)
 
 	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
 
+	/*
+	 * Comment from Linux:
+	 * Userspace may perform DC ZVA instructions. Mismatched block sizes
+	 * could result in too much or too little memory being zeroed if a
+	 * process is preempted and migrated between CPUs.
+	 *
+	 * ftr_dczid is using STRICT comparison so we will taint Xen if different
+	 * values are found.
+	 */
+	SANITIZE_REG(dczid, 0, dczid);
+
 	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
 	{
 		SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index f600a611bd..113f20f601 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -125,6 +125,8 @@ void identify_cpu(struct cpuinfo_arm *c)
 
     c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
 
+    c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
+
     aarch32_el0 = cpu_feature64_has_el0_32(c);
 #endif
 
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 52cb3133e0..5219fd3bab 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -259,6 +259,14 @@ struct cpuinfo_arm {
         register_t bits[1];
     } zfr64;
 
+    /*
+     * DCZID is only used to check for incoherent values between cores
+     * and taint Xen in this case
+     */
+    struct {
+        register_t bits[1];
+    } dczid;
+
 #endif
 
     /*
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:27:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:27:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188038.337140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrV-0005W6-NA; Thu, 16 Sep 2021 06:27:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188038.337140; Thu, 16 Sep 2021 06:27:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkrV-0005Vy-J6; Thu, 16 Sep 2021 06:27:09 +0000
Received: by outflank-mailman (input) for mailman id 188038;
 Thu, 16 Sep 2021 06:27:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8py=OG=arm.com=bertrand.marquis@srs-us1.protection.inumbo.net>)
 id 1mQkrU-0002iA-5T
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:27:08 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id f46badef-cd2b-46aa-b34a-6e49c96c164d;
 Thu, 16 Sep 2021 06:26:51 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2654B31B;
 Wed, 15 Sep 2021 23:26:51 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 706043F59C;
 Wed, 15 Sep 2021 23:26:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f46badef-cd2b-46aa-b34a-6e49c96c164d
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v4 7/7] xen/arm: Sanitize CTR_EL0
Date: Thu, 16 Sep 2021 07:25:40 +0100
Message-Id: <3a6a63701df71c0a0ea743c6229266077da0563e.1631772970.git.bertrand.marquis@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
In-Reply-To: <cover.1631772970.git.bertrand.marquis@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>

Sanitize CTR_EL0 value between cores and taint Xen if incompatible
values are found.

In the case of different i-cache types, the sanitize ctr_el0 will have a
sanitize value but this is currently not used or exposed to guest which
are seeing the original ctr_el0 value.

Use the opportunity to rename CTR_L1Ip to use an upper case name like
Linux does.
The patch is also defining ICACHE_POLICY_xxx instead of only having
CTR_L1IP_xxx to sync the definitions with Linux and is updating the code
using those accordingly (arm32 setup).

On platforms with only the same type of cores, this patch should not
modify the current Xen behaviour.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes in v4: Remove TID2 support and handling of corresponding
register emulation in vcpreg/vsysreg.
Changes in v3: none
Change in v2: Patch introduced in v2
---
 xen/arch/arm/arm64/cpufeature.c  |  6 ++----
 xen/arch/arm/cpufeature.c        |  2 ++
 xen/arch/arm/setup.c             |  2 +-
 xen/include/asm-arm/cpufeature.h |  9 +++++++++
 xen/include/asm-arm/processor.h  | 18 +++++++++++++++---
 5 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index d4679f5df3..6e5d30dc7b 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
 	ARM64_FTR_END,
 };
 
-#if 0
-/* TODO: use this to sanitize the cache line size among cores */
-
 static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
@@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
-#endif
 
 static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
@@ -606,6 +602,8 @@ void update_system_features(const struct cpuinfo_arm *new)
 	 */
 	SANITIZE_REG(dczid, 0, dczid);
 
+	SANITIZE_REG(ctr, 0, ctr);
+
 	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
 	{
 		SANITIZE_ID_REG(pfr32, 0, pfr0);
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 113f20f601..6e51f530a8 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
 
     c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
 
+    c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
+
     aarch32_el0 = cpu_feature64_has_el0_32(c);
 #endif
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 4ab13d0fbe..49dc90d198 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -650,7 +650,7 @@ static void __init setup_mm(void)
         panic("No memory bank\n");
 
     /* We only supports instruction caches implementing the IVIPT extension. */
-    if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
+    if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
         panic("AIVIVT instruction cache not supported\n");
 
     init_pdx();
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 5219fd3bab..cab89ee142 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -267,6 +267,14 @@ struct cpuinfo_arm {
         register_t bits[1];
     } dczid;
 
+    /*
+     * CTR is only used to check for different cache types or policies and
+     * taint Xen in this case
+     */
+    struct {
+        register_t bits[1];
+    } ctr;
+
 #endif
 
     /*
@@ -339,6 +347,7 @@ extern struct cpuinfo_arm system_cpuinfo;
 extern void identify_cpu(struct cpuinfo_arm *);
 
 #ifdef CONFIG_ARM_64
+
 extern void update_system_features(const struct cpuinfo_arm *);
 #else
 static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 2058b69447..8ab2940f68 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -7,9 +7,21 @@
 #include <public/arch-arm.h>
 
 /* CTR Cache Type Register */
-#define CTR_L1Ip_MASK       0x3
-#define CTR_L1Ip_SHIFT      14
-#define CTR_L1Ip_AIVIVT     0x1
+#define CTR_L1IP_MASK       0x3
+#define CTR_L1IP_SHIFT      14
+#define CTR_DMINLINE_SHIFT  16
+#define CTR_IMINLINE_SHIFT  0
+#define CTR_IMINLINE_MASK   0xf
+#define CTR_ERG_SHIFT       20
+#define CTR_CWG_SHIFT       24
+#define CTR_CWG_MASK        15
+#define CTR_IDC_SHIFT       28
+#define CTR_DIC_SHIFT       29
+
+#define ICACHE_POLICY_VPIPT  0
+#define ICACHE_POLICY_AIVIVT 1
+#define ICACHE_POLICY_VIPT   2
+#define ICACHE_POLICY_PIPT   3
 
 /* MIDR Main ID Register */
 #define MIDR_REVISION_MASK      0xf
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:32:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:32:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188073.337151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkx4-00086D-Ei; Thu, 16 Sep 2021 06:32:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188073.337151; Thu, 16 Sep 2021 06:32:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQkx4-000866-Bl; Thu, 16 Sep 2021 06:32:54 +0000
Received: by outflank-mailman (input) for mailman id 188073;
 Thu, 16 Sep 2021 06:32:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQkx2-00085x-Mt
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:32:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ea4fc8e6-16b7-11ec-b5a8-12813bfff9fa;
 Thu, 16 Sep 2021 06:32:51 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-fAiQU2UxP4--whI59exFGg-1; Thu, 16 Sep 2021 08:32:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2336.eurprd04.prod.outlook.com (2603:10a6:800:27::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 06:32:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 06:32:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0004.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 06:32:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea4fc8e6-16b7-11ec-b5a8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631773970;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=k945y5QOEnDG8MEEiPbwh7dChsxCNiGHKDa1fcct2dk=;
	b=ZRFnbITd2YXeaW9VX/SB1m0mPF27DMapSAc9xjcjh4uqFqMAEsrNVnMRWkfN213rT6nKkr
	bcCU5AQlXNJKvXhicG4BRq5VHuNl/kPsvUKMvsk6IPZDATdDAUZhjAR3ssWCDGSWpLBuQe
	cfcxI7Q7/iOGpHalqKlkQ+Gx5WEcc2A=
X-MC-Unique: fAiQU2UxP4--whI59exFGg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QUe8hi+nQ6NVAxbmGMsjbTBuE6BIeB7HRoosTI/kHEV2K++D2/f8rUQ7cOnU81TF/Z59R/kh/3km7hHTBFqKSGs0n78+AVCfhZ456MMqOf1M2ppvJ/CSk+9WHBA1kAE771FrFLG/awWCxlTHjPv91k6ui3hO7i5WVxWlFsyxxsbsBeKzcN4tgLCkg4+Q/sLQrbgntjm6uv4ILR3WglF324Fq/32duAHhw9jyn2/qFuHiyax58cxxpblhB1pWSPzuzniEueu/7JZJPCwP1IY29Wle6GjUOnsR9sEERq4LFfPdiyFX7BaNTsYIioiuC41d1MEuHwExJIi713qas+YQ4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=k945y5QOEnDG8MEEiPbwh7dChsxCNiGHKDa1fcct2dk=;
 b=ZTqb99Ac4XTDCNvLoDOfufQ5c3M8YrsYvHTnY70t9TVc6UbFDaOK5jOUvlmz3ndAs4lv94wA+u2xTBCRz4kCgjePK0sl+n7jysqW5i0IBxdAXBDBRPqlPYu5tjfRfuhO2zi98l9Q2uM7o8UNDIvCI4dRjR/FzyiTFSz/psfbwdTElKDLShXgmF+fTi7sD8pWrFfmU1JwneY+JZptsKqL2BJiFtvzutKy+D64rnDihSLzwyLy9AZsNOQBMcRAWSFK06lqmHiovB3byhwlTQ/jMtU8xwebYWBQab7W8aIdnpUglhwC92SYxTIZX3k3wxCO1AmzZA+hzu4Q712zRc5S6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v7 5/7] xen: re-define assign_pages and introduce a new
 function assign_page
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Penny Zheng <penny.zheng@arm.com>, Bertrand.Marquis@arm.com,
 Wei.Chen@arm.com, xen-devel@lists.xenproject.org, julien@xen.org
References: <20210910025215.1671073-1-penny.zheng@arm.com>
 <20210910025215.1671073-6-penny.zheng@arm.com>
 <acf2e43e-5db1-1374-013e-b152093b41ba@suse.com>
 <alpine.DEB.2.21.2109100916270.10523@sstabellini-ThinkPad-T480s>
 <d2e87c7a-08e0-3cca-fe4a-6278d6939591@suse.com>
 <alpine.DEB.2.21.2109151106430.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <13c4bcf7-24bd-71cf-e7ab-2c6f1ec53998@suse.com>
Date: Thu, 16 Sep 2021 08:32:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109151106430.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0004.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9da854b2-c103-4fb4-8810-08d978dbcc31
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2336:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233600B0FAD4DB28FFD23788B3DC9@VI1PR0401MB2336.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WRvgfD/1mA+Wmay655qVsUgi9zd6jYuNw1sNPExmJDRvjUmER44xUEGs2DpAZ7Nyaxtc1aTK//m2bmUIHjVWqtqM/HLuHPg/BFJyXJurzrsGfgRKkNIU6xSFyvrZhFr7fedFSAuLpFCwT214O62r7UFgX+vnWdZMGURzQ0M1/29yNgEspuMfRynBEejwtYb33B0LTOVgaNrSYpGHZgXoeS++fpZi1IOG5HNsFY1TrxmHI3AeLBWVw1bKZyRfPi4lOFiDvzx7MxdLD7ZZYLTDeyLWWGhaERDZ3vRIGkaiTGLnRM/FEhimPOzpK5BeRYVr1mq6cuGkH/FD4Kb8P5i3j0kfzT+1SlOHaCiegj90CWjMZUDxjVg2b19MZ332VaAO2/LmOuO3u8SJxDhIcFNebLgu/hUyeYoLqzWqqhJqCSwdOAKneFowk2pjoHfdv+MZumf6w3o1gSatCzExLf4yQYPpBfh3fkImV6n/TzGavttgEAYUrC2HZK7E5QC/othrnpylc6CBJmKs7JfTz5pOoWGrauiJeN4VppYM4sLMnz1bmQI2Wm6Fgk5eMafVcdwVckmEj95N9wVKHYUaV0X28dZrzMRehCqjnPhGe1ITpzT9qYczxnXkdOSc2VR5WTdPI2NzoURjfivpZNxvV6sKnP0Xa6cXQjQHziwUQxBYQcuaNGJPD4SyxXUZHTkEUZrJfHTZaTna/4z4VVb6or2FQ5/YG6LvKZSIE25h6NJSGMY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(346002)(39860400002)(136003)(396003)(66476007)(956004)(5660300002)(186003)(66556008)(26005)(8936002)(66946007)(53546011)(8676002)(83380400001)(38100700002)(4326008)(86362001)(478600001)(31686004)(31696002)(36756003)(6916009)(316002)(6486002)(16576012)(2906002)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eGEvRU5lL3BoOTNGaENtLzF3cHoxY2o1YXRmRG1pUWxPaXNoR2J1blVNbC9y?=
 =?utf-8?B?VkVmV3ozZEJnVzZ1aFhCamV0bTNjOGVVM1RtUW1Ua2FPZnVOaUY4TW90UlRz?=
 =?utf-8?B?eGFWQWRJT2tyZTZBYmdKWW1VQkRBVkdyM1pubkRyQmhJaWI1UExMbFNHL1M3?=
 =?utf-8?B?M2MxN2JwK3A5VS9SaGpnWU1NODd6cHErTTJhTnF4RnpkUGN0WFR2VXdjc0JO?=
 =?utf-8?B?N2s3RURBcThIVVVtQkF1RmNCNFFBZWFhUjhPViswR3hUcHZMM2lpbGd2WTlN?=
 =?utf-8?B?L2NPZWg4cjljYUJoamozRXRBUVU0NVFtdDVQckFQTlZnQ2hVUUsycDRtdVVs?=
 =?utf-8?B?T3VMNHUwYjU1c1lrZC9ZYlQ5SHAyWnR2aWZKWDRZTCtjb1p5bjdBdElGVUFt?=
 =?utf-8?B?cmExLzczNEpIQUIzK0J1ZGRqNUVJSEV2ZDI5OUtnN0xXZUpXYVJmWDcrTlkw?=
 =?utf-8?B?ZXMvS08vRFg4c0F0ZjZFZUhVWUdtS1NwOUVUaVgyTUZjVmREeS9WWGJ5Y1Fy?=
 =?utf-8?B?NGpIT2w0NnR3RjI5bUNodHcvVTh6Qkw3SHVROC9FUUU5d3NudkhxTk14WGtl?=
 =?utf-8?B?dTNlcG9oRURLYlh0TzRXZEVGd3pvMlFaTjJuc1JhUkJ0UXc5eCswMW9kMXRy?=
 =?utf-8?B?VlMvdGpybFVTeHBDRXkxVnh1ekhObFUrT2lDOWY5VVJ0ZTFGb3RBbktPb2d2?=
 =?utf-8?B?bUpDZXRLN25WbTFFZzVOWVExa0lRL3c1cnpvNEtwV1hKTmxGR1RpSHlFaXR4?=
 =?utf-8?B?K2JIWmdER291WHVrYVhzMjRZV0ZYRmNaTlIwWFRRMXo0SnJ5eFE3RzFjWGc1?=
 =?utf-8?B?TENhanI3UEh3OUNtT1A2NUVTb2p6RmxWc0ExT3RLZy9kZlF5L2Zxb2E2Wkdk?=
 =?utf-8?B?QUEwczVwNjcxYngxMTZzSkp1TktHSkt4cmJKK2ZHOThQRnNYcWJvOWxhaEVq?=
 =?utf-8?B?ckdydFhUV0tpSDZHMXBPcW5MMzlGcFMzZmVSalZFa1k1cFhtSzR1dUJMcXNV?=
 =?utf-8?B?MVBwOGE3S2crU01QYUVHOUlxNTg0aDl6UmwxTVZEVTZleFRoVHdrandUUFBB?=
 =?utf-8?B?YmZqRU5QQnFEdDJGL0hnZE5Welh0MlBScDNUeDEyeGxiNEgwRjltaEc1TDRa?=
 =?utf-8?B?STZDdGNBTGgvejkrSWtCc0R5MGFWUi9jRWVuTS8rODUwMW5BTlJwZG85WlFI?=
 =?utf-8?B?aU80a2VVdjU5OGNOUElrZ244OHUvS2k4Q2xxS1IxRytLRzJjb0Z2REpxRnVP?=
 =?utf-8?B?YXZyQ0lEQ1dhU3drRU9Ta3BIbEZtaG5RNEd4MVB1N08rZmJHbU82S1YvaW9w?=
 =?utf-8?B?b2ZEd2xOZ3h1VXNuMGNRbXJYK296WFR0SFFJd3BiNWc0Q1pzbk50V0g0eUt3?=
 =?utf-8?B?d0FFTVptR1lEanB5N2NBelRMTFMyVXdEeTM0TVRRMFpUTUllVmRUUEdRd282?=
 =?utf-8?B?Q1lkM1cwOFgxa0wyNURmejFRc2FZTGRRbUhZbXNuVHM3SlRKcmg2dVoxWTl3?=
 =?utf-8?B?OHE5VkJva2MzLzh3dXNwZExJdFpMdlBYWVEybU5aUHR3VXVSSlMwUnpScmJQ?=
 =?utf-8?B?eS9VVGNLcDhzMHE4blRhUElrOFZRS0g4dUl4UUxuS2FweXNNdS9JK3JDRW5R?=
 =?utf-8?B?Tk1HK0kvb2gxUWtoZHhSWVUwSE5iK05lQ0ltTExDYUl2SnZRVEw0Q3J1NHBj?=
 =?utf-8?B?eXVWUkNLQmRYWWZxMS9kVHpDVEpkc2pRRWZsK0QxSEdTU3RmUVgwbTYrV1Zl?=
 =?utf-8?Q?n+6SfWrxQGnQoXrfL04R4QE9BT/oGALXIuD9N1i?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9da854b2-c103-4fb4-8810-08d978dbcc31
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 06:32:47.0573
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qApueVjA3f4TWgAYn2VerUEU/cYG/ufU7pdkRgZTyIuTORtDNpobJw8jiVkD13ZIOQWiuNXzmyCdJoX+SS0HUg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2336

On 15.09.2021 20:12, Stefano Stabellini wrote:
> On Wed, 15 Sep 2021, Jan Beulich wrote:
>> On 10.09.2021 18:23, Stefano Stabellini wrote:
>>> On Fri, 10 Sep 2021, Jan Beulich wrote:
>>>> On 10.09.2021 04:52, Penny Zheng wrote:
>>>>> In order to deal with the trouble of count-to-order conversion when page number
>>>>> is not in a power-of-two, this commit re-define assign_pages for nr pages and
>>>>> assign_page for original page with a single order.
>>>>>
>>>>> Backporting confusion could be helped by altering the order of assign_pages
>>>>> parameters, such that the compiler would point out that adjustments at call
>>>>> sites are needed.
>>>>
>>>> Thanks, this now takes care of my primary concern. However, I (now?) have
>>>> another (I thought I would have mentioned this before):
>>>>
>>>>> --- a/xen/common/page_alloc.c
>>>>> +++ b/xen/common/page_alloc.c
>>>>> @@ -2259,9 +2259,9 @@ void init_domheap_pages(paddr_t ps, paddr_t pe)
>>>>>  
>>>>>  
>>>>>  int assign_pages(
>>>>> -    struct domain *d,
>>>>>      struct page_info *pg,
>>>>> -    unsigned int order,
>>>>> +    unsigned long nr,
>>>>
>>>> If this really is to be "unsigned long" (and not "unsigned int"), then...
>>>
>>> Thanks for spotting this. I think it makes sense to use "unsigned int
>>> nr" here.
>>
>> I see you've made the change while committing, but the subsequent patch
>> then would have needed adjustment as well: It's now silently truncating
>> an "unsigned long" value to "unsigned int". It was the splitting that's
>> now needed there _anyway_ that made me wonder whether the patch here
>> really is worthwhile to have. But of course acquire_domstatic_pages()
>> could for now also simply reject too large values ...
> 
> Yes. Are you thinking of a check like the following at the beginning of
> acquire_domstatic_pages?
> 
>     if ( nr_mfns > UINT_MAX )
>         return -EINVAL;

Something like this might be the way to go.

> An alternative would be to change acquire_domstatic_pages to take an
> unsigned int as nr_mfns parameter, but then it would just push the
> problem up one level to allocate_static_memory, which is reading the
> value from device tree so it is out of our control. So I think it is a
> good idea to have an explicit check and acquire_domstatic_pages would be
> a good place for it.

If this was put closer to the DT parsing, maybe a (more) sensible error
message could be given?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:38:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188103.337161 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQl2f-0000ZK-5C; Thu, 16 Sep 2021 06:38:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188103.337161; Thu, 16 Sep 2021 06:38:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQl2f-0000ZD-1J; Thu, 16 Sep 2021 06:38:41 +0000
Received: by outflank-mailman (input) for mailman id 188103;
 Thu, 16 Sep 2021 06:38:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQl2d-0000Z7-NR
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:38:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b90cbb76-16b8-11ec-b5a9-12813bfff9fa;
 Thu, 16 Sep 2021 06:38:38 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2058.outbound.protection.outlook.com [104.47.4.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-ExB47IRgOD6NGSJPMzNQXg-1; Thu, 16 Sep 2021 08:38:36 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2829.eurprd04.prod.outlook.com (2603:10a6:800:bb::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 06:38:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 06:38:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0040.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Thu, 16 Sep 2021 06:38:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b90cbb76-16b8-11ec-b5a9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631774317;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AJv9z/VlZXMdQt4wLVBVf1OgyUJJ2fFb2mp4WuvxNrc=;
	b=XlJSETDkJBib7e1zJ/yP6+aSAiufZ/+6KNpmQ0uxFHnpHnyIphFedxSl3UuCIgIscbDNIx
	4B4aXhoPRtx9Y667p/kFQZKnoCpDcUd2EzIAZfe6D+Vnvl8Frqjbwfbq92Pysj8O7ZoszT
	4rlT0WDUE1hm1CQ42w1Nm1nZezFbybo=
X-MC-Unique: ExB47IRgOD6NGSJPMzNQXg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gd0SIERTX2UNQfzcvykVr1NRNb3sO0+wAh2XqHW0MjZkiza3a6ebM/LPRaqnVlAhGevHpJHLGWBqLhdivWKgWI6ZGYoJSmTQ+YET5B8GnZC9/NpD5lrHmSrTomGr1qopu3KqIjhbiijo5wqR5H8OXSDBH0QpRzSufvfbtGRKjHBba72wvmNhwcC5wCZIS6rYoBID9boOikgLa6AconRYdFB6CYYrTF5hFP8WHq1JrEPqqsLFBRjCg5eRjuv/ea5ZvHrSoZHap1EIUDE3Y7f7JpP1h6L7Ra96XPuGQJsg/7PUuRLQR6LqKdIxTdC4YzfL5VWGYQx1vPaKRino6BMQ/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AJv9z/VlZXMdQt4wLVBVf1OgyUJJ2fFb2mp4WuvxNrc=;
 b=Ud0+EzfGLKLym4JkLqMJnQQKtUXniDtWBt81RQkg0yPDTfBa3orSpiEe23IF3Dkb1a0iBJwEGyWSXoZuNLe73qAYQghkJPAOIz761tgNocf3bCpMhNOMGGEcN85EsU5+b2Omsao5xAgcg21p5EkcFmnwCzjmx7iZEz2oKq/yitQtkzGBTtLpuslMaLlGWDFRmIdoqWumQMTx9Lwd0zjpAkAb335VhhfJlB8T550ktoJiQ/eEaMNKQOZr391PT+NQyoRTMy0LfX3m6uCYSvGjlww0Z+ZlnGwKLUUgx92xa1UOoX8fZDO6O9+QNyEw77/YwLovNudawoI977HcjYmidQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [RFC PATCH V2] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Oleksandr <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
 <b6744333-4d43-ef24-0f9b-b5cd54680660@suse.com>
 <86cdc577-7085-48cd-c417-85b20afc9bf1@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <647bccf5-1d1e-1f21-fb7c-c338bc5119b6@suse.com>
Date: Thu, 16 Sep 2021 08:38:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <86cdc577-7085-48cd-c417-85b20afc9bf1@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0040.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 809f3c5a-f78d-4e9b-07fe-08d978dc9b41
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2829:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB28292039D56CDF06A0089A7CB3DC9@VI1PR0402MB2829.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CeCQJfFlHAZvnAstDxGdvSUU+FokA3PO6t+/GEGiFPkNlC3yWtViHHexejbrBAv20UJQzk5y3Eo/7/NanoTOn6Eb0+2i1qu/zRs+uMGY11Efb1wKutuBNNmRAB+YAIcCo/u7NenyeVuWbtoxf36/zGWr/mCRP2/A3Hzcepihw8yWL0opXgYHoA/OxLd3lyUawn6OYQAw75W8fpTwIzby2yqdiHOpm8YDc6A7zuLWnc5fQBfbIEyLuhD2o1X931q6jPF4hYWK72zeV8kDDjTrA6gfY4hGDFHZbU4f43WHYQOLgKIsB/0BfSoO1Pk92V6OVmY6OP7sHdX2D6SOnRzLDKjqGMVPRWv0VNP6oEYnbsIeayRoKTo2c0kiBMzIiIlL6Y/Qu/ZlIcMJ2b442gfo1+0OfEZT+EPFW9iveM6wogkoybAJRBh92McTSkTp2CLOyIP+sxQYdE1qhXCU137rsH4yMb12P35z1LdkFWFwwdKtMdB3WaRyf2XEw3JsxLD/L1p2sACmx0ywR7bHvZvy+SCJgl7TO4MLfJHVr8Zr8LzN9sOP5/0oJdpWK8XKeFmJ3Sr07sNdc8wiuQ2cXwCeYIKG/Bog3A2GxxC37b8v5e/y3V1Iq8o+4/itZGJjYs9Lr6UsZ9WvOqvVKgyhUiHr60a/0xlnOpgcnD3m+FLJDRugU736NS9mrOQglLbV1fD2UvRYrN6e7IJQuRs968XvpHWL1AIF83NL8kr3hblfBFqWMwKpoteTabEydsjG73LA
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(956004)(53546011)(6916009)(26005)(2616005)(66476007)(31686004)(66556008)(5660300002)(66946007)(86362001)(31696002)(186003)(36756003)(8676002)(83380400001)(4326008)(2906002)(8936002)(478600001)(7416002)(16576012)(316002)(54906003)(38100700002)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NEVIMEUxUGNyblA2dktoUVdmRjVvaDZSZHczdWt0My80ZVFqVDkwY2ppaHVh?=
 =?utf-8?B?NnRWRTNSR3dFY0xSNlpNMmE0eHNsUVpwMkxONm93VXd1eU5iTllNZG9aZ2NU?=
 =?utf-8?B?cXRncHRVaW9zNVUwbngvTkJEUnV1SUtLQ2RaNk81WGJyVXhzd2N2dDFWcFhK?=
 =?utf-8?B?MFpBM0lveHFXekc3TGFES0phdUFWSHJZSzErZlF0RmExVEN2ZlRpTnRhQXBD?=
 =?utf-8?B?c1RlbExtOUtoV21uTHpTTU84TEIrdzltaCt6WWZrOXpYcWFZeTdURWZiRnY4?=
 =?utf-8?B?MlprcWNtYkY0V0h1a09IOHRRQkpXRGhoY2ovT3hFQThUODZhcUJHaDJpTXYy?=
 =?utf-8?B?OTQwQVBMbEhWa0VBc1FvejlFWFBLU0xFV1hZMmpFN0hnUjJJSEZhVm1BY0VH?=
 =?utf-8?B?YjlOQjVudWxFM1F5d240cm9PVTBSUVYxbUhGMGtYc0labVpVOGFkcXVCSytp?=
 =?utf-8?B?WWpDNStrVGhhWldtZ3FSVTBCRmFzU0ZXYVEvWlRSWTNoSDM1NENHamp1bzlx?=
 =?utf-8?B?ZUtpZFRjRitmRkVZRFd6V0tlTlcyQXpyd0dpMG1QdWtVc2dtVmNHbldDa3la?=
 =?utf-8?B?WnRpZDBuZlVYYVgyNVkrRE5lZTkxRlVCK1ZLQWNDZGFYVjFDTXAzT0dkVCtu?=
 =?utf-8?B?cXM4aXNNRXRMUDJPSFlyamF4T1dtc3ZOb1gxVm9iZHhaeGs0eStneFJMWUZC?=
 =?utf-8?B?eW1yYXg5Z0xEaVplY3RkRnJVbW9UOFUyVFF4QzhHUTdVVUh2TVF0cGxFS2Rq?=
 =?utf-8?B?ZVZCZmhaMnlJOUp5L2dGSlNPMlQ3c1pZcHJ5d0d5WHkxUFJQTHdIaG5kV1Rp?=
 =?utf-8?B?ak5yY3IvaWF2Z1oraFFibSt4TEhhQ1Z0NlU3dTBFa2tQUGFGVUp5clpEWG5X?=
 =?utf-8?B?T29EdWVxZzc5eFAwUlg4b1JtTHl3dXZHc2tVS29rY1JvWkRPNmRHeGFQT21S?=
 =?utf-8?B?a0dHYTdnQTdETzA1TStUdUY5bmMxYXBkRzlNWS9VSko4T2hnWFFRS1d5VDZr?=
 =?utf-8?B?bEFVdmp0cDVFNS9TbXVFY1phb3BIQkx5OGRMeStKSXRoK2hTZHVmZEdqRnlC?=
 =?utf-8?B?YkRpM1pxcEl3QzAwMWVyVTd3SVZkUk1XQjAzTUtSTWpwR0tJaWJzeHlwbHpP?=
 =?utf-8?B?cG1EQkZxZ1U1MXlsNE45OWM1Y1I5d2Y5VUhBclVWbGFqRTVPeGZwakpEWm1k?=
 =?utf-8?B?MWUxV2FMRGszM0I2dkFBTENBeU0vQ3dabldCQzc0RmxMSGVlYXV5WGptbTIy?=
 =?utf-8?B?KzRVRDNhcnAxdGtUKzQ0b2JjbWJlS1Nsa2pxbmwxQ2M4MHVSbE9NYzVMTThG?=
 =?utf-8?B?d3hZTHFvZFUyM1RnbnF5aDN5Z0hVODZiaVh1a0pOaW1velVEa2VJbVpQU3Vx?=
 =?utf-8?B?WkpVZUkyT1Y2eVkrb2FvUktweHNYWWdEdVNzMzN5elVodTI3aWJqKzFZSFhu?=
 =?utf-8?B?eFNjTGVmMi9Ua2dqY1NvWVhaOURpbGJTRk9vSkZENW9LTXdpMVdIaW51VFJw?=
 =?utf-8?B?cXdkSStYa1BRS3EzcW1raGR6SDI0VHNCeUZPYzhxMEpaMTJYWkY3N3dlLzla?=
 =?utf-8?B?c3p0ZDZVY3JiUDVEMklIUnY4YVZkMXJCa3NSS1BzYnpsT3lQU0dhZXVyeG9o?=
 =?utf-8?B?QnlWNTNWeXZvUGc5WERyZXdGRnArbjBwZnJLeFNyVTB1YldZTTU3UldENFVY?=
 =?utf-8?B?MHZCMFJjM3pFejdUdjV0V0ttV0lNMExGTXRRelNYNmkyQS9jK21BTzBSVXVl?=
 =?utf-8?Q?b8b7504qo1wJxmX0dAlsfNy5BjyqxgmPyqIF2mC?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 809f3c5a-f78d-4e9b-07fe-08d978dc9b41
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 06:38:34.3933
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qpPn9OUjF/IxjlfPflknBERwcOR0b3SO0UyL2fLUFV/3vVRfKiAXLs+y/auZ7Ir8/u88MnDLSHglZBAE2VUX+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2829

On 16.09.2021 00:13, Oleksandr wrote:
> On 15.09.21 13:06, Jan Beulich wrote:
>> On 14.09.2021 22:44, Oleksandr Tyshchenko wrote:
>>> --- a/xen/include/asm-arm/mm.h
>>> +++ b/xen/include/asm-arm/mm.h
>>> @@ -98,9 +98,18 @@ struct page_info
>>>   #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>>>   #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>>>   
>>> - /* Count of uses of this frame as its current type. */
>>> -#define PGT_count_width   PG_shift(2)
>>> -#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>>> + /* 3-bit count of uses of this frame as its current type. */
>>> +#define PGT_count_base    PG_shift(4)
>>> +#define PGT_count_mask    PG_mask(7, 4)
>>> +
>>> +/*
>>> + * Stored in bits [27:0] or [59:0] GFN if page is used for grant table frame.
>> I don't know enough Arm details to tell whether this is properly
>> one bit more than the maximum number of physical address bits.
>> Without the extra bit you wouldn't be able to tell apart a
>> guest specified GFN matching the value of PGT_INVALID_FRAME_GFN
>> from an entry which was set from INVALID_GFN.
> Really good point.
> 
> 1. On Arm64 the p2m_ipa_bits could (theoretically) be 64-bit which, I 
> assume, corresponds to the maximum guest physical address (1 << 64) - 1 
> = 0xFFFFFFFFFFFFFFFF.
> To store that GFN we need 52-bit. But, we provide 60-bit field which is 
> more than enough, I think. Practically, the maximum supported 
> p2m_ipa_bits is 48-bit, so the maximum supported GFN will occupy 36-bit 
> only. Everything is ok here.
> 2. On Arm32 the p2m_ipa_bits is 40-bit which, I assume, corresponds to 
> the maximum guest physical address (1 << 40) - 1 = 0xFFFFFFFFFF. To 
> store that GFN we need 28-bit. If I did the calculation correctly, what 
> we have on Arm32 is that PGT_INVALID_FRAME_GFN == maximum guest physical 
> address and it looks like we need and extra bit on Arm32. Do you think 
> we need to borrow one more bit from the count portion to stay on the 
> safe side?

I think so, unless there are further restrictions on the GFN range
that I'm unaware of.

For 64-bit, if you only need 52 bits, why do you make the field 60
bits wide? I'd recommend against "wasting" bits. Better keep the
count field as wide as possible.

>>> + * This only valid for the xenheap pages.
>>> + */
>>> +#define PGT_gfn_width     PG_shift(4)
>>> +#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
>> Any reason you don't use PG_mask() here? Any open-coding is prone
>> to people later making mistakes.
> I failed to come up with idea how to do that without #ifdef. As GFN 
> starts at bit 0 different first parameter would be needed for PG_mask on 
> 32-bit and 64-bit systems.
> I wonder whether PGC_count_mask/PGT_count_mask are open-coded on Arm/x86 
> because of the same reason.

Hmm, that pre-existing open-coding isn't nice, but is perhaps a
good enough reason to keep what you have. (Personally I wouldn't
be afraid of adding an #ifdef here, but I realize views there may
differ.)

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:45:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:45:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188113.337173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQl9L-00021n-2q; Thu, 16 Sep 2021 06:45:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188113.337173; Thu, 16 Sep 2021 06:45:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQl9K-00021g-To; Thu, 16 Sep 2021 06:45:34 +0000
Received: by outflank-mailman (input) for mailman id 188113;
 Thu, 16 Sep 2021 06:45:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQl9K-00021a-23
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:45:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01efbaaa-a78d-4579-91d3-4231b25e27a5;
 Thu, 16 Sep 2021 06:45:32 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2057.outbound.protection.outlook.com [104.47.13.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-Tu625PwaPTiFSf8g0UE8Nw-1; Thu, 16 Sep 2021 08:45:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6477.eurprd04.prod.outlook.com (2603:10a6:803:11e::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 06:45:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 06:45:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0001.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Thu, 16 Sep 2021 06:45:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01efbaaa-a78d-4579-91d3-4231b25e27a5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631774731;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ljQFwkl0043Igxc8fvmVI209+Fmz0veBvqBLCjOpVCQ=;
	b=OAQpqsjXw63BjHptkB1TT6PMEQTS9xsn/IWKIfKodE6IuZwZpVrN5i+m0z/faSQpBWmnH/
	cEm4VTJ4bIM2tgbGmOJaH3o4Pky5MikGOy3tj7J/+wAjPcPh7Q7kRWyOcCQLIP4x+81kHh
	87vMth6yg9qECRHMXfobUNbwsb0Mzxs=
X-MC-Unique: Tu625PwaPTiFSf8g0UE8Nw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hzcFbFT0WOJa6GjVF+X3UomqVDsfO3tNTTMM1TdYdL1RyfHrzs6wdM1G47eWpmvzTuHdUxofRwp4KB8fQ59Fm7zzVeZ2HmYjsyS3HS/LMWJYuW1BcSVVaqfSD6H4BANx9BUe7VJq+YY/aaEefrwYG3RCCZNJH8lJlnoIZz+NMluR5Mq8cp8uY7Ozj7YP2oZgTQfXtm18928xgLpZ/5CTgEYw9KnD58bUahAOBU3QU8F3mNhj5ow2nL2TpILEI4O5t/zlnwdrKcNN35y4sJGtvTZPJd0PiHb9Vq1+EuntRxvtm9u2R44leBALfQCWQZ05VCcGcfpnD2t3+gVtA6FDHg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ljQFwkl0043Igxc8fvmVI209+Fmz0veBvqBLCjOpVCQ=;
 b=P3WQ5/nvGbwHDYZUPcCYib45heuocqLQON07BuoZzBlqlrxBOQL/+s6CdKXpPHKfpUpfv9FwHsz6C0LbDGskKfnAjfOKdes8olaEfb3bMpWK+9VQsM/pX0j2wA3JyJvcgN6QgXSAX8mZ7hNMz5VOLMdaAhKXbeVLlZaXQV3C3llP3dRl2ipWChOuXH7COC4n5qrdXh+rYKEdSF0mziSgW1KOL4gZeyf7+Gy9biABYeg4ddrCjk9PF39FCRK21YLl9H/dHlAS07OqpIoaNq+m7IL7BkvCmTqG8BOqoHv4YxuWHg73M5gqjy6xbgnNdCqknevjEWMVOOllEO2I16wGeQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] xen/efi: Restrict check for DT boot modules on EFI
 boot
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 wei.chen@arm.com, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Luca Fancellu <luca.fancellu@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-2-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109151445080.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d04b834a-5d5e-de44-5bfb-e6012660e295@suse.com>
Date: Thu, 16 Sep 2021 08:45:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109151445080.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0001.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 452d05bc-c1e1-4890-463e-08d978dd921d
X-MS-TrafficTypeDiagnostic: VE1PR04MB6477:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB647770C5652D81BE32A3E25FB3DC9@VE1PR04MB6477.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	baa3kYGgRx97DrPyYPiHwcfiPYUkl6yDRZ/0AM+8Hu26SKS4qf9CmWLz1xhP9tqAJo9x+/RevEDweGuHMLDzWKKeRA7bBBjJ9c5cLv0/Uv5bEAs9dOhHmbE1Lr3PohGGMwv48WRm76ZYSjL9CU/mIdxQWDdqVGa1EEWBeaYEcBTmncbptmBZedrde59jiGYzX+DkM8fmKZRmtxsq+Ow13LHawVS0jBjLoUm8S5E+nkAbIdawFop8TH/tZp3B+dR4DfTcHvsg7z1Cc9kVgNj6hwcwfdqB7wAC1k/7iOjw6STjU5ezRKYdzyUXQ/AZn0Qeb1wWyqzYrA95qgo5sMx5OEJoX4bD36BpZYdu+PrLQhoP6hjFC1+Vq8FLwnvygh/qpcefVOp7xrNvly4to+JKNNoMomLc3ExoSvSq91k7EMmRpb1YR+qDLGYu64Gnuqx++bPTFeV070xCYal00qKdyXNRlqZ/Wc/9ULW50cRcJnnWfxp5dtZ09pHVBRhUpdmSLJmqQNi/1DcaV/DuUyxNJb4pWvJRW+ePc2sfK/6OnjQOCUHBjNgZ4HGA37HOlRiNr/TBmCuJAELKml3zggZmmQwrh9AGq2lLjAyOnh8nyG7DMZp3vlQ3J7g8n6OmcDVfRtWEwegdUYoo6LC872xw+jLxHvBJXhY4XQ/NNQffXUufEHj9iIKSTIplLMb68BqFBdg+m6irdR5eQ719FLV34R4Ai3Xp6CmXO/lbCsw9Dsc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(366004)(396003)(136003)(376002)(86362001)(16576012)(8936002)(31686004)(8676002)(6916009)(478600001)(53546011)(5660300002)(316002)(83380400001)(186003)(2616005)(38100700002)(66476007)(2906002)(6486002)(31696002)(66946007)(66556008)(4326008)(956004)(54906003)(26005)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QUxOWjEydDc0dUNZSHZWUlRMQWkrU0E4S3pLOW9QdWpnOTBQNmZFZHZ3VDNN?=
 =?utf-8?B?b1lOQWpRWFRiUThOeWZpQVFtWFkvSEZjNWdOODdRMUF2cUFRbnlmZ04yV2px?=
 =?utf-8?B?TVArV0Q0andvU1NOTWQyN25hbG5jVnRpbWdoZDRVcEQwdGEyUHcwaFl1WnlK?=
 =?utf-8?B?NmhnNFNBcjVvYkhVWC9haGhNc1Q2TWk3SHY3MU1JaGo5bzJwd3laNzNZcFhQ?=
 =?utf-8?B?YmdIUElCdU90WTNzWjY3clJPTFhuUWpMN3dYc3l3cW9BWVlPYlBZYUI5UGky?=
 =?utf-8?B?OEsrUXdwVkJuUlZYK1BUM3QyQ2FtTmxnTllWeldnSUl0Y2w3NEpLSGNDVUhW?=
 =?utf-8?B?NkpGQnBaZVB6UWRjL2hZaFpwdU9wbEhaWDZnNy9BbGY2V0t6ZW9wQ2RmcnJy?=
 =?utf-8?B?U1EvRWRUWlZYMTk5N2JRLzNjZU1BcVl6Y2ZSeHAzbE04MW1WdzVmWEVkTGhU?=
 =?utf-8?B?S3dEUitrNzVhbFVXZW8xYnZCZ2l4Rncza2x5c2RSQ2tQWHBxdHg4OENuWFRa?=
 =?utf-8?B?cHhBbzBnWVB1bnZsRm5rTDNrS0tpS3JCaXFNdkRleUJtK1drWUVyNVRxZWlr?=
 =?utf-8?B?dTFjQXU3TktUcXdmNTNHbElHSlhIYjRPNHFJV3phV2JXYU1SY29WdGE5YVdE?=
 =?utf-8?B?NSs2OU94cnZEVGJiVlJRUTV0MjBnbFlLcFRYV2VMZFJPZWgyMXZYMGJOdEF2?=
 =?utf-8?B?OU9VRTk0ZURvc2E4V1d0TjRFTEJLSzBUWk5kOGptZEpXK0RscXBONUw3NGRW?=
 =?utf-8?B?Y3dyNDdGU0VLZVJMQmJJcVc1SXpQNUEyeUlWbjFzRnlEZm5NcXIxeXl2T29j?=
 =?utf-8?B?cFMxYkJUdk1ZTUwrOTRYOWlSMzhzSmk2NjR3M1JETEtpRDR0eFlBWmkxZDJp?=
 =?utf-8?B?dUZFYVVXRXQraUpzckhTaXdQcXFjOGZURDF6eVdiaVV0RkFveW1OUzkyd003?=
 =?utf-8?B?VXBqQXAzK0swcTdtOEd6dTAwdW1ldU9zZXBkYy8yYVYrdU5BdUUvQ3BCc2lT?=
 =?utf-8?B?MXViYzBYUVZsaW5xOGxTeTJUSkgxV2lCeWJ5QVlMSVhwUUMvNkVVNDhla2hK?=
 =?utf-8?B?NThlNmdlSkdoWFYrN2tyS2QrWHlhUG1yck1HdjdEa2NvOU9TUDIxcnIya2xE?=
 =?utf-8?B?bWMwdGI0dzFpRUM5US9UWFI5VlBjaS9IWm9CbHkvSjRmVTcxamRGalFoN2lh?=
 =?utf-8?B?d1krSDQ4UEV1QUM5V1BvUmcydEExbmd4T2dIcysxNnJLN0pwZmhZeEhTUDBZ?=
 =?utf-8?B?Ulh3aGdpZDdSTytWRjJCWDhSZzVmMlNzaGFXS3JiUklpaWEwRkJnZ3BveTFr?=
 =?utf-8?B?MDcvL1pwVEd2VzdmVHhwM29ybThTV0tGUE5qdVRBRFR5TlFIaHdueFZNaFA1?=
 =?utf-8?B?ZVI5eXVxWnlPSCtEbWYyNmVnZVdDYWNLWDVZakNGL25jYnFsaFRIZ0h5V1V5?=
 =?utf-8?B?R1RrVUZUQ2xiOEErWnJmekZOOXFvZVUwSnZkalRZUW5hSEh4Y3IzUVd0L05w?=
 =?utf-8?B?a3NCdUpnL1hqem52YVk0alc4RVE2cVhjRmRiU0FDdmxSbk8xbXUwZkttejVl?=
 =?utf-8?B?dFhFTXN6WEJydkIvNGMwVnlSSXRRdmJMV1ZRa1lCMnlvQkhHWmxkRHhKRTRZ?=
 =?utf-8?B?Ynl5U0dMbnJtUS96eUxONk9TM2Z1enZHQ3VLbVRVMURRbFpLQTB0VkJFaWtH?=
 =?utf-8?B?cjRGU296L29STWNrR2VFOVRiZjIvV2NPT0lBYUMvejFQekRmbFdINC9YL1kv?=
 =?utf-8?Q?yHILNmLnSNe3u6mmnAchaOz6Hnd0uxg/1MRgUZd?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 452d05bc-c1e1-4890-463e-08d978dd921d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 06:45:28.5282
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mWCNEeqkEMj3HSUxQFwVDVVWT0v/g7VrzlSpDQGij115mWv7jE3PgrTA18uUOVP1WQKhMNaGHWeGhyiEvriXzg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6477

On 16.09.2021 02:16, Stefano Stabellini wrote:
> So I am thinking that we have no choice but introducing a new property
> to tell us whether we should or should not load xen.cfg when
> multiboot,modules are present.
> 
> Taking inspiration from HyperLaunch, it could be a new node:
> 
> chosen {
>     cfg {
>         compatible = "xen,uefi-config", "multiboot,module";
>         uefi,binary = "xen.cfg";
>     };
> };
> 
> In efi_arch_use_config_file we would check if there are any nodes
> compatible with "multiboot,module". If there are none, it returns true.
> 
> If there are any, and one of them is also compatible "xen,uefi-config",
> then efi_arch_use_config_file returns true and also the specified
> configuration filename.
> 
> If there are nodes compatible to "multiboot,module" but none of them is
> compatible to "module,uefi-config", then efi_arch_use_config_file
> returns false. We use the device tree only.
> 
> I think that would be clearer and more consistent from a specification
> perspective, but it requires one change in common code:
> efi_arch_use_config_file would not just return bool but it would also
> return a filename if found (it could be a pointer parameter to the
> function).
> 
> 
> Otherwise, we could add a simple property like the following, without a
> specific value and without a filename:
> 
> chosen {
>     xen,uefi-config;
> };
> 
> The presence of xen,uefi-config could mean that Xen should go through
> the usual guessing game to figure out the right cfg file to load. This
> would not require any common code changes because
> efi_arch_use_config_file could simply return bool as it does today.
> 
> My preference is the xen,uefi-config compatible node, but I think the
> property would also work.
> 
> 
> Jan, do you have an opinion on whether efi_arch_use_config_file has to
> stay as it is today, or would you be open to the possibility of making
> efi_arch_use_config_file return a filename too?

I see no issue with making such a change; my preference would be an
altered return type, provided all present cases can be expressed
alongside the new one you need.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:46:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:46:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188121.337183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlAZ-0002cv-Cy; Thu, 16 Sep 2021 06:46:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188121.337183; Thu, 16 Sep 2021 06:46:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlAZ-0002co-9Q; Thu, 16 Sep 2021 06:46:51 +0000
Received: by outflank-mailman (input) for mailman id 188121;
 Thu, 16 Sep 2021 06:46:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQlAY-0002cc-Mt; Thu, 16 Sep 2021 06:46:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQlAY-0007mn-G5; Thu, 16 Sep 2021 06:46:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQlAY-0006S2-63; Thu, 16 Sep 2021 06:46:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQlAY-0006ot-5Z; Thu, 16 Sep 2021 06:46:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=O1gK2bQGxtTfIH2tNKXWWKwCElJyjZ7SL7OOynG+tLE=; b=3aHEV6YO7lhO4mBzEX49rwQdYI
	oRT8IlTvmWS9WyV7bHwo/3dLi+Fj1zZ0Q8/uyLarz+6BjoWBs2YKEpDrWFhS2soh05SY92bkzHnfa
	HjcayB+AvemKAEFJtmpjiENFLjmOtFbkYvQLTbl7y6JsUxfKGVLGUCT/yxBdyfs/kVWo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165001-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165001: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=c19d18136ef920e3e84f961e2a335a41147adcb8
X-Osstest-Versions-That:
    ovmf=3b69fcf5f849021aa3bc810f8100ea71c03019e5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 06:46:50 +0000

flight 165001 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165001/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 c19d18136ef920e3e84f961e2a335a41147adcb8
baseline version:
 ovmf                 3b69fcf5f849021aa3bc810f8100ea71c03019e5

Last test of basis   164971  2021-09-13 17:12:37 Z    2 days
Testing same since   165001  2021-09-16 01:55:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3b69fcf5f8..c19d18136e  c19d18136ef920e3e84f961e2a335a41147adcb8 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 06:50:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 06:50:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188131.337197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlDr-0004CZ-So; Thu, 16 Sep 2021 06:50:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188131.337197; Thu, 16 Sep 2021 06:50:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlDr-0004CS-Pw; Thu, 16 Sep 2021 06:50:15 +0000
Received: by outflank-mailman (input) for mailman id 188131;
 Thu, 16 Sep 2021 06:50:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQlDq-0004CM-T1
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 06:50:14 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23a98ab5-798f-4f7f-8af6-90433c5bddb4;
 Thu, 16 Sep 2021 06:50:14 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-xS9GE1LtPt2cEnDBHmEiJw-1; Thu, 16 Sep 2021 08:50:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5326.eurprd04.prod.outlook.com (2603:10a6:803:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Thu, 16 Sep
 2021 06:50:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 06:50:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0107.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 06:50:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23a98ab5-798f-4f7f-8af6-90433c5bddb4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631775013;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=k1gPeTyjFrHTLA41PiPhhSGUR96qFSIoFFCjAFprlOA=;
	b=SblS2KI9KGQvdPtGe9SkRjEMF1ZMkDnAVtngAB8z6ABKUuw+cOpAH/sZkHF1IuZt5VLqKO
	nCHiZylVJNJvLHGYgeeBkawyJwNXFEq/1zy3j+EFNsptO1ZJwXjJpiYpuincMht4/1oYzi
	0osgX3ZtzkhH+zTzZsnhqKdqFhOdWqg=
X-MC-Unique: xS9GE1LtPt2cEnDBHmEiJw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kRwsCDYZlLclh/ceRD/3oLIRE9PKJr9OZrmeE4ic4FONLtiVeRwMDIAlolLYV8DT4UUwwMT5g8vu4Vmyor9UGBs7OOgcj6dBxF/i9ajYdXMMbDPKy8nwWJpXyluYeSRdM57KLIzeiH4+gf79pL2wEbk+ZWo2euXZEuuFzjnqpuaA5kzrulj0bNoLsAFQkk/f0LS7WVQOIziHYLO1aeP2aD4M1l19svMy0bPD1WNCTAjFdURbSxmpIJ9BpOkc2UHB4xH+oN/qm2nBpZP6/clyWdyt+3Pzw8J5oNXLVIWwgu+rjznEWY2a7AMy+tiagImA5n79bcGPsV4Mi9P2IH0diQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=k1gPeTyjFrHTLA41PiPhhSGUR96qFSIoFFCjAFprlOA=;
 b=AKEpB8H7NcVq8Q4gNT8DrWQqnktufhD+YZUcbV+52MwR63yPdpj0+QYDx/dTjPDaNwNU0s/zhILGaz5I3pGIRknHq46KwHW39kJpEeA9UDJjKdWBNTVgDRYKpvH/hFD9C11SlC9GcGnJ30uqqX764Rz7phkRpNoKrwPFtCVHi5WmJv3LfB016iql0bgVgzlMXCY1WJj/0XtHUNTvhiExqP7eD/S9kYa8EPWcncSANP6K1nyMOYX1/yT/mZNsAf017OcakxfT8KFAp6gfkjji3UmfNH3neY6SRsRDHBRuroYzH1MfeCR5fGxpo0yVfjhbryEF8KFRgptKxKMPmcM52Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Stefano Stabellini <sstabellini@kernel.org>,
 Luca Fancellu <luca.fancellu@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 wei.chen@arm.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5e272a26-e1f7-a861-feb1-350293a9f809@suse.com>
Date: Thu, 16 Sep 2021 08:50:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0107.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 81bd7bb9-a1b4-45a5-e21b-08d978de372b
X-MS-TrafficTypeDiagnostic: VI1PR04MB5326:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5326579F63BE714B055CDD48B3DC9@VI1PR04MB5326.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1728;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hLVG4JzyOONaR91DT4laiaA+WBbyuoFB7b0tuzsK54ZxJiBZJGOZRRtUfUCV/3PY+95BROPJc4BlCRYqirON30KkXLKl86M+ZKoLZC7GM4SNWu/L1CJKGUKBvI5aG4Ep/xTjwTa3vFuFgqRoNl4cJePrB1kyXgCoS4lJTDX6fFvlkDCnWQjaQ7mKa8njLOXmQEV6cr2BNcerGujxk5Zbt+xeYW95+wiQeYnStjPmCgYKqypqSfeIQN/0FjNht3zGumTH3bmZ5Dkr5w5t+qfIpIEE2k/82GVePPGs0n0GAdn5fSodI5trBFuoiFDfOv+uQNL2gi8/BRuiUqhEdaj7FpcHO/Riat5wlrsocAbf4EAR36UvAUyLtKPEnYss+0gN+6UQcLt1buNMD6/z9sNIuimXf4ESix0UgN73qfJmHsJhz8oxwqZ8phEHeZtQsD8YMcIRxMrpbOadN5QAYwD7j3D3tlGePa6ZJK6L+0RnB/nFS0JWOXxzu5cKorDn9I8MbtyoA1wA2pR/21uaa2z1O2XOQodMqumig24JEeUtpci9AJrhtgMYpoybtRql4E4lBr7s3deBJluDb8n4SLWjd1x4YWysm1WUXQAV+LgH9uE5qRQytabwoVe8NIttQNXUkRE18QrBiVv3P8mzMu9C3yzAH7vfOOvHHImk9vTjxgComdRxP8nwBbj2uITtVe2A9zRX9/+PxGnTAE8n7bvr08f4XkRoVZRurY3kTOYIjrIUf+Fs5sMiYL4I2tld7V9fut/bR14dj/9ziVkfTfA40Q==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(346002)(136003)(366004)(376002)(16576012)(54906003)(956004)(2906002)(2616005)(110136005)(83380400001)(7416002)(8936002)(4326008)(8676002)(5660300002)(31686004)(186003)(26005)(478600001)(38100700002)(31696002)(53546011)(66946007)(36756003)(316002)(6486002)(86362001)(66476007)(66556008)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QUVHOU5GRVdCZEZ5Tmc2UDJ0cTNCRDhFeGNDUmpDUFpBV0ZmeGFOclpPdEJ5?=
 =?utf-8?B?N2tXYVJ1d0s4K3VRdk50U2M0ZGRMUGNHMXZLbXZmRGtDK081VktPRlI4bGVB?=
 =?utf-8?B?aHVLQlRSQjUvWmsvWmgxc1oyMUdYRlN2anQ2cERieHAvRVFvRjVheTlBV3Vh?=
 =?utf-8?B?M3hzdEhiQTZHVUc3dVMwalczQWdCdU1LdmpFSkVlaDVGcCtjMEFERFdSaTBh?=
 =?utf-8?B?S0tlajdYOFl6LzZucGVNRDljN1Q2MDdNMG1zNHBZNHhvcnZGVmUrKzFLWnU2?=
 =?utf-8?B?WmJLYnA5QXB0M0ZRQzBhUFJIdnRuVFRsaU1ydWUyMzZzRHBzdkVWaFlXSW5N?=
 =?utf-8?B?aW5oS2IvaHY0RVJhamQ5SVpHRUJWUi9iYmZuK3BzemxVeG5ITDB5YnN0Wjdo?=
 =?utf-8?B?R2x4UHUrZXBzOWF0U3Q3Q1pLQTFONjNPZC83T1FoUjRTZUtGaWk4OUVBZXEy?=
 =?utf-8?B?WXBiejl3d0FCMytiNFlQb2grMzVqdTdBSm5wTXpHZTZwZmNENVlYZXZObjhJ?=
 =?utf-8?B?bWlyOVdXWkhrUE5rYWhHQXFvSXNPVWJMdTBKczZITjVIVDQwZzBkVlg0UXFU?=
 =?utf-8?B?QjZGRlhmTG8zeThUeVNHQ04veE5NUkdJK2I3aThOelNHcUdwaERrMlljcVVJ?=
 =?utf-8?B?dWxWVnNITlBRZHFtQ0I1a3hMZXhNQ3pVaHF5THJEY2Ftbm0xdUlUSTdGd0ln?=
 =?utf-8?B?VkIwVUFRSWNOR2M1M1IyQi90RlI3Q1RGa2JkdlNhSmh5Nm9jZERxa0Q2VmlX?=
 =?utf-8?B?cXJXTFlBQ2JIVEVTTjgrY3pJODdhdjNUZW9Ob2VLUWh1SzVRMVBCcGsvOXk3?=
 =?utf-8?B?VUw5SkhJWFB2WDdqOVR0THRWbXlVQmp0cjZHZ1cxWFFYWWxEVVM0V0dCdWl6?=
 =?utf-8?B?TzRoaTRENllWeWNFaWNvY0VGSnJZMUw4NXlGaEU5UXZNWkRJdVBZQ2hXMkNq?=
 =?utf-8?B?THhyazg0dTNFamdHOExYSHlSakRkWmJ4dlpwR2ZUb0ZTVlY4cDZWR0FIVnVY?=
 =?utf-8?B?SGYydFQrSTVyM1RjRzhkNFhOZS9ZM1E5WjY5RVY2MjRvcEZkaE1YM0hjdTFB?=
 =?utf-8?B?Syswc3NYeHJ5Zzh6RTdnNVVZSUlXdVdrYXl2bVpoRFpoUFVHTDJMNlVGY3RG?=
 =?utf-8?B?cGRCbnkrRlkvcHRuR0g1b2FjME82OHlHR1ltRE1leGlTQkcva2FaaVBYa0U3?=
 =?utf-8?B?amI0QnpXaStQS0NMZGp1SENtN25xVW5BTVhZV3lQWmNPckkvQSs1bGFOQ3da?=
 =?utf-8?B?VmYwdGdJQWV4KzhLRWdhTVFTYlJOaEFRRllkZHVsOG1UY01SUHdxYlA5V1hI?=
 =?utf-8?B?Tjd2RkFPR2FucEZRNW1ZQnZkMnIxVFpTNFBDTnFSNlIzR2JXRUhQaHhyU0Nu?=
 =?utf-8?B?STBjcHhaN0k5SitMK2NRUENpN0dKaElnNkd5NnNOcGY2NjVuMmd4WmJVTVg1?=
 =?utf-8?B?TFRpeHp6VW5TWitEdHdaZkFiLzZ2Y09oc1B6ZXRGelltS0FIVW9nVTI0czYy?=
 =?utf-8?B?UnkrTXQ4aDNLNStxd1cxNzc2NERld0FBNGR0Q3VkMzg1SWh4L2dXdDNIY0ps?=
 =?utf-8?B?VFlxdkV1S25HSmorRmw0VWNSM25GeXc0bHZjQVlaNzIyMGJ0U1UxK2N3cUgx?=
 =?utf-8?B?MlYyTHRFV0ZZRWRNV0VZT2xWMlh6emNkTmNsNTh4SDVJSHFiUlhWNU1FMExo?=
 =?utf-8?B?azIwbTl6Zms2RXNmdG5USGNuL0hSK1gwcWtNd284TlBqMW14TzlPNHlydjFj?=
 =?utf-8?Q?rzix/AMA7Hw7Z8N53xLtZC3psSLME35VIJToEOk?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 81bd7bb9-a1b4-45a5-e21b-08d978de372b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 06:50:05.5200
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1NPrVlfxtDgo08eJHpYgcR5OUu8/RGyqQXXJvPFOhFXkslEuYLgAaVdWdd9f+1wfO+3YP/ZzvvxZxGh3ebiLTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5326

On 16.09.2021 03:16, Stefano Stabellini wrote:
> On Wed, 15 Sep 2021, Luca Fancellu wrote:
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells)
>> +{
>> +    /*
>> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
>> +     * inside this node
>> +     */
>> +    for ( int module_node = fdt_first_subnode(fdt, domain_node);
> 
> int module_node;
> 
> for ( module_node = fdt_first_subnode(fdt, domain_node);

Not just here iirc from briefly looking over the patch as a whole
yesterday: Use of plain "int" would better be limited to cases where
values may also be negative. I don't suppose that's possible here as
well as in a number of other cases.

>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>>              efi_bs->FreePool(name.w);
>>          }
>>  
>> -        if ( !name.s )
>> -            blexit(L"No Dom0 kernel image specified.");
>> -
>>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>  
>> -        option_str = split_string(name.s);
>> +#ifdef CONFIG_ARM
>> +        /* dom0less feature is supported only on ARM */
>> +        dom0less_found = check_dom0less_efi_boot(dir_handle);
>> +#endif
> 
> Rather than an #ifdef here you can simply implement
> check_dom0less_efi_boot on x86 as a static inline returning always
> false.

Indeed, and the properly named (efi_arch_...()).

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:16:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:16:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188140.337209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQldU-0006uN-3u; Thu, 16 Sep 2021 07:16:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188140.337209; Thu, 16 Sep 2021 07:16:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQldT-0006uG-Ul; Thu, 16 Sep 2021 07:16:43 +0000
Received: by outflank-mailman (input) for mailman id 188140;
 Thu, 16 Sep 2021 07:16:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F/Gy=OG=epam.com=prvs=9893242681=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mQldS-0006tu-3k
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:16:42 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3f562238-4fc3-441d-9c80-d73846a6c445;
 Thu, 16 Sep 2021 07:16:40 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18G6MCnr021917; 
 Thu, 16 Sep 2021 07:16:36 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b3yter9an-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 16 Sep 2021 07:16:36 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4308.eurprd03.prod.outlook.com (2603:10a6:208:ce::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 07:16:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:16:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f562238-4fc3-441d-9c80-d73846a6c445
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g1WEHDviZEbOXqq5p7vFFSOr2tfdc6Ef2BP6kC6/BOeXGWrk4L9/xnuHc+H1dpAgmilfz3W3GbzYkNbs+Xxx9ASKO8fdnbTW9V6cCYiWzxGabp2oX1YjvfihGDvskE4IRsi3ozicWLRd9ANip1f110KTrOQvP7YVV3GWSe81obwKfAB9MHHq3vl8LQ/NmVnftjZxO34A0aQs9no7Nx9TRhcP2h0x2stjLknG79DpnCm4Ff/cQ9MMWBLP/B+IpIod+On9HOWbqf2b8sKf77vBfYud3dWTatVYwhi7dcDSS2XLx2dSpNOx+mzFiWwR+TFLThN9ll2I+lvpSVyfIzbyYQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4kxncR3dmPW+A/TwUGeqPoNHqXq87uFi9iCENuBJwxc=;
 b=fTToUN3PFN6bbv8hWe6NK8PlMeLq+VputtoqNIAijFVekpm7k/JPx/CXqG3MkpMF+EhluX0szAbTzTZCJufKhepCcae0otWCQ42IeeDEFc+QfUedyiSC/mUNuFf8RY0i4WSWvbOUHSpK9FRaW9gyr3mZDzrPSUj6zp2brcwVK7vGmMZAvigXtrfghC8dnhNTh3jZjpD29qmQ9wScfne9e0/Oz2lA22Xq6+MZoWhXEuFzQoQSUCD2A7LivcNo8Iz7rdz3tUybQq3vd7IwF3rL7YugB3hhnX+9Tvqlun+oCCjPB66IYbOBqIaey1E1HluKQxxhpfBCCTVfL3CWYmeXVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4kxncR3dmPW+A/TwUGeqPoNHqXq87uFi9iCENuBJwxc=;
 b=g3VLpjR887KT+PRVohxuuyOZBIX5DfWo8Fvv6k62SYDA6A4Q1fQ0CmDndXbEQen96QT+weXgNCe9eI8f0i8Uf0eNuJJ7SU/x6KLoOz+VnfdJvWC8anVRpDmkIBlbYfieEAW3PS81yCnVp7bHD59clVVnE3lpAwzEc2RCxormIPbD10hxUuknLurqp/NHKejnTII8I1VpNh/lf/FOsKaJA2Lf+WGnmEAMdrD8x47D/Fh2ebqodr/YALokM41J3ZFW6gT+myQf58NX9q7KmWDJY6+9eE/jEu9bBWAL9wItx7HhY6r7oA25YqpI3llqqW2FTRDezhVeoGB+1T6Baq/DeQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Rahul Singh <rahul.singh@arm.com>,
        Julien Grall
	<julien.grall.oss@gmail.com>,
        Oleksandr Andrushchenko <andr2000@gmail.com>,
        xen-devel <xen-devel@lists.xenproject.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Topic: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
Thread-Index: 
 AQHXoJ50hf+zXQhc2ka7YfmmERKX+aucB06AgAE4wYCAAAtRgIAADDiAgAARZwCAAFsJAIAAFA4AgAO3eICAAc7agIAA860AgABTv4CAAPQVgIAAAsoAgAC3nAA=
Date: Thu, 16 Sep 2021 07:16:32 +0000
Message-ID: <4093d5fd-1d44-cc33-9fe2-2695f5d3f16c@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-11-andr2000@gmail.com>
 <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org>
 <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com>
 <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com>
 <684b3534-40eb-add7-f46e-c6258904757b@xen.org>
 <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com>
 <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com>
 <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s>
 <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com>
 <alpine.DEB.2.21.2109151246450.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109151311480.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109151311480.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f13f8b97-f22e-409d-03d5-08d978e1e986
x-ms-traffictypediagnostic: AM0PR03MB4308:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4308962D5BE2B68E210DB16BE7DC9@AM0PR03MB4308.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 NgbcG627dq2EbDMzVIwm6MnfvgXmUju05EXSm1ovUCDV4KAu7qPNtpE+ggXUuStojw9c8TBcZpgmZ2TbAgUcq68hKXRKPTwGKsV/nW4I1VsodWjigRwiCd3KsJeQMFNIoPLdf2sIzVfOvQt/ltLnyMAtuSqyJkGxXtvZmC7kXfVRMcQpi09gc9/jX3cHVLjmCBmmKel8n6PeTkKJs5Aq3tbdSQwfUWn8zdKB2b1elFQvC8Go7nIFOrhs98SN37vEZZ4qYYfClNcpVwa9xesb07UB1q+9+2HrpVM37LUvL32VbYubWJ3fxN7Qdq8ZtLo0rJULhzoMJ8AGY8HPH5VL5PmO3T9VKQTdpIYjSDSts2NaJbQ4g3R84+PBSc/zSlG8fC+uxbDT1WwGNJ4fi8J7ME97RWc1iUjJqRcxkNcLF8mlosqfPOs/HmrsNb587E236PKBDZkMLWzFXc5qgo0+DRmw1PCNbpRLfnknzJTecvBoLZkff2gJA5Iw62HUSXdowJcyat/SvhFIl0xrvf2DNZWpNoCfPhKkFzOgpUnRRbhQEfYeCXhyl8rRmbI62LLO2TbFM/0XbIyoWsrmM2j7KIrFYnbVgk1oXAoylRz/MvOp+idP3LdvK26aaY8hlg9eksHXwYaWzDffiQfAFiVHxn3JjGMrNoVl55v9r8jNs65wX1rK28zzbg1YsHa2Xzzu/Ociu52tIyK1LXx9juNe+kAbqusoX7nfAIN7/3Mi72tNs/pN+7G+cfG0q6TWM2GzmqqwgKZYv+dLoGQ+K1f2x9eKZW7dNz6gqrwUxVY/VthUJwbtETa+gTozdvr3P2yn4TV5MvqWme3z+J+tuJUH9A==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(366004)(376002)(346002)(64756008)(66946007)(66446008)(2906002)(83380400001)(76116006)(36756003)(8676002)(2616005)(316002)(38070700005)(6916009)(8936002)(54906003)(26005)(6486002)(31696002)(5660300002)(122000001)(38100700002)(71200400001)(6506007)(86362001)(186003)(91956017)(66476007)(4326008)(6512007)(966005)(478600001)(66556008)(31686004)(53546011)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NjBWeDQzUTRPN2FwVXYweHhxbWlOYmF4YkN5elRsWHB0Z2JuSi95R0NLWjQx?=
 =?utf-8?B?QjlMMHg4ck5NNGNseG9iYmF3TXRRK1RzNHlUVlpIVjQ1Z2hjTGNsb0NaMEND?=
 =?utf-8?B?RTFPMUxDbjdtQWRwOUgwVU9UZGNudTM2eTBSLzlXNDBEbUFzR3FsMVZPUWJv?=
 =?utf-8?B?VlJRVWVtZytMRitqT3dXamY0VHVOejBmaytaSUROaXV5RzgweWdIdm1rZmll?=
 =?utf-8?B?d0Y4dVNWL1ZBLzUwUWkwK0YzOUtPb3NMQXdiZm54L0xXUWRPOUx0QU93YWQ3?=
 =?utf-8?B?Y3hlZEhOTEsycWJQTjBndFRvdlZ3K2owc2Q4MDVtaytUY1M3eXVxQmNuZXVL?=
 =?utf-8?B?aUszaGdIT2JHZWFBVmEzcVZsaU9HV2FtYUxaUDBvd0RLNURCWHA2YnhUekd2?=
 =?utf-8?B?MnlzRXJjdC9iZVdMdkVUQXcxYnR1azdORjgya3NLemtwZXhkUmV1WUEwZjJH?=
 =?utf-8?B?aHNXdmhTVnRveW55NHNHaXE3aldueEtFT0pIbnZBSjdBbU8reVZCTlR0WGI1?=
 =?utf-8?B?ckJJYXhlMDVDazJVUHNBNmJUNEd6NkUwZ3NjUW5jeGhiSk1QM2xXMkY2RXVU?=
 =?utf-8?B?TmJIbEdjYlpYbkppR3Y2cHRCb2l1R0g1N3ZDcm14OEZ6YW1IaFdDZlRmdWY3?=
 =?utf-8?B?Z0lOVk8vN3dqWFZYTFZLT3ZPV3J5NUhxSEtXbWk0bG9yMTNmS0ZwOEt0WUpV?=
 =?utf-8?B?VEVYeCsySGcxb1FSbm4vc1RuMEgxVWg0b056UmRIV2dRb0lMSHZOYUYvaHZt?=
 =?utf-8?B?dzhFYllGVG1lb2pSNmNKUXk0SENwMjMvVnpkTnF1dmZ0UWtwWjRTOHZObTI0?=
 =?utf-8?B?VytZZ29jQ1c2eG12QmkvUHZPZU1ieFJFbndFTHRLUUVpb1pSZy9pRzJLVkoy?=
 =?utf-8?B?NWVSK2YwWDVHdkZEdGhBSytTc2xOMDFBQmp4MG8wclgybk9wZ0kvR1U3U1hE?=
 =?utf-8?B?M2dJMXdBK3Z0MzVmZDQyYWphVGV4b2MraE93ZzNjdDhHK1ZzTmk4ZWZ6cGVT?=
 =?utf-8?B?aGdGZklJS3JKODZ1QXFNcTlQTjFWTUYvelVlWk0vbHVOb0tESXd3OWtwTk4y?=
 =?utf-8?B?ZjVDZ0xvWDVZNW9sQTdsUHlXSnN2NkZHcUp4TkFqQkQ4eE9BK2ZMdk96ZnVx?=
 =?utf-8?B?SVNDZ3hiWWRTM1ZYZXErVHhpRitjOXJjZjQzWHVzYUxXVFJIS2JUMy9vclBN?=
 =?utf-8?B?QnorZ1BXZVBzSVc5ZlNWWFFpUzRDNmZzOUlONWkrOHZscFQwTFFTZEFZSDN3?=
 =?utf-8?B?UStYSFhPZldUUGpCTld0UFNmZFYya3RzZzlwS0p4eFBVMldNTktFeCsyajMy?=
 =?utf-8?B?WmhUTlEySklnUTRYMXJnWkNOMVRUTnQ4Z3l5VlZxZmlBNld0VVI3aUdsQXlr?=
 =?utf-8?B?UlV6dUFsR00zbi94ZXpWeldDSGVRRGo3MkJCczBhTHdRVXhZc0VMVGRFWDdU?=
 =?utf-8?B?UmkwTVMvcXdLR3ppbTNpcnM3dUgxTVFvQlZDQXZIUVlSZ200TEFLUEdqM3ZZ?=
 =?utf-8?B?cEhJUGpYczBkdEIyaE10amthc1BsYU5BcGtXN1l1b1dvOEw0aXZZVzc1U0VL?=
 =?utf-8?B?aE4wQ3duMmt6ajFWcERZdjVlZGRwdTlZKzE2V3Q2ZEhuQjVCYURMbXcwZzIw?=
 =?utf-8?B?R094ZjFESjVsOW1JR2NHQVZDTk5YamRaM0dETmdHenNZMnBjdTd0cXdubVlI?=
 =?utf-8?B?ZGZMVWZwcU9xNGo2dEdjWDRLS3NJeVBJaUp0clRxQ1pzbjUxdHNUbXFrVVRY?=
 =?utf-8?Q?4aD/h5wlzkWypLxr1izozZyrxmEs3qgTxVo/kah?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <DB6B9D0CB5A40E469DD08819EEB1627F@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f13f8b97-f22e-409d-03d5-08d978e1e986
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:16:32.9140
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +FODNYjQ5OYGMPRF2Hz2NfsB363A/VvNyGtZkOO1PfppZdDt/X0WKhPO1EEpEkySpnwdoJqngtPN9s3365w59yEqlrrumFK8MD7pYs+FgQZukao4+I+B2ufeqWr7/tGh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4308
X-Proofpoint-ORIG-GUID: 6OU7VgRqD9r65_CQJWNWrZx5I_D9G0y9
X-Proofpoint-GUID: 6OU7VgRqD9r65_CQJWNWrZx5I_D9G0y9
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-16_02,2021-09-15_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0
 impostorscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0
 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109160046

SGksIFN0ZWZhbm8hDQoNCk9uIDE1LjA5LjIxIDIzOjE5LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3Jv
dGU6DQo+IE9uIFdlZCwgMTUgU2VwIDIwMjEsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4+
IE9uIFdlZCwgMTUgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+
IE9uIDE1LjA5LjIxIDAzOjM2LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4+IE9uIFR1
ZSwgMTQgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+Pj4gV2l0
aCB0aGUgcGF0Y2ggYWJvdmUgSSBoYXZlIHRoZSBmb2xsb3dpbmcgbG9nIGluIERvbWFpbi0wOg0K
Pj4+Pj4NCj4+Pj4+IGdlbmVyaWMtYXJtdjgteHQtZG9tMCBsb2dpbjogcm9vdA0KPj4+Pj4gcm9v
dEBnZW5lcmljLWFybXY4LXh0LWRvbTA6fiMgKFhFTikgKioqIFNlcmlhbCBpbnB1dCB0byBYZW4g
KHR5cGUgJ0NUUkwtYScgdGhyZWUgdGltZXMgdG8gc3dpdGNoIGlucHV0KQ0KPj4+Pj4gKFhFTikg
PT09PSBQQ0kgZGV2aWNlcyA9PT09DQo+Pj4+PiAoWEVOKSA9PT09IHNlZ21lbnQgMDAwMCA9PT09
DQo+Pj4+PiAoWEVOKSAwMDAwOjAzOjAwLjAgLSBkMCAtIG5vZGUgLTENCj4+Pj4+IChYRU4pIDAw
MDA6MDI6MDIuMCAtIGQwIC0gbm9kZSAtMQ0KPj4+Pj4gKFhFTikgMDAwMDowMjowMS4wIC0gZDAg
LSBub2RlIC0xDQo+Pj4+PiAoWEVOKSAwMDAwOjAyOjAwLjAgLSBkMCAtIG5vZGUgLTENCj4+Pj4+
IChYRU4pIDAwMDA6MDE6MDAuMCAtIGQwIC0gbm9kZSAtMQ0KPj4+Pj4gKFhFTikgMDAwMDowMDow
MC4wIC0gZDAgLSBub2RlIC0xDQo+Pj4+PiAoWEVOKSAqKiogU2VyaWFsIGlucHV0IHRvIERPTTAg
KHR5cGUgJ0NUUkwtYScgdGhyZWUgdGltZXMgdG8gc3dpdGNoIGlucHV0KQ0KPj4+Pj4NCj4+Pj4+
IHJvb3RAZ2VuZXJpYy1hcm12OC14dC1kb20wOn4jIG1vZHByb2JlIGUxMDAwZQ0KPj4+Pj4gW8Kg
wqAgNDYuMTA0NzI5XSBlMTAwMGU6IEludGVsKFIpIFBSTy8xMDAwIE5ldHdvcmsgRHJpdmVyDQo+
Pj4+PiBbwqDCoCA0Ni4xMDU0NzldIGUxMDAwZTogQ29weXJpZ2h0KGMpIDE5OTkgLSAyMDE1IElu
dGVsIENvcnBvcmF0aW9uLg0KPj4+Pj4gW8KgwqAgNDYuMTA3Mjk3XSBlMTAwMGUgMDAwMDowMzow
MC4wOiBlbmFibGluZyBkZXZpY2UgKDAwMDAgLT4gMDAwMikNCj4+Pj4+IChYRU4pIG1hcCBbZTAw
MDAsIGUwMDFmXSAtPiAweGUwMDAwIGZvciBkMA0KPj4+Pj4gKFhFTikgbWFwIFtlMDAyMCwgZTAw
M2ZdIC0+IDB4ZTAwMjAgZm9yIGQwDQo+Pj4+PiBbwqDCoCA0Ni4xNzg1MTNdIGUxMDAwZSAwMDAw
OjAzOjAwLjA6IEludGVycnVwdCBUaHJvdHRsaW5nIFJhdGUgKGludHMvc2VjKSBzZXQgdG8gZHlu
YW1pYyBjb25zZXJ2YXRpdmUgbW9kZQ0KPj4+Pj4gW8KgwqAgNDYuMTg5NjY4XSBwY2lfbXNpX3Nl
dHVwX21zaV9pcnFzDQo+Pj4+PiBbwqDCoCA0Ni4xOTEwMTZdIG53bF9jb21wb3NlX21zaV9tc2cg
bXNnIGF0IGZlNDQwMDAwDQo+Pj4+PiAoWEVOKSB0cmFwcy5jOjIwMTQ6ZDB2MCBIU1I9MHgwMDAw
MDA5MzgxMDA0NyBwYz0weGZmZmY4MDAwMTA0YjRiMDAgZ3ZhPTB4ZmZmZjgwMDAxMGZhNTAwMCBn
cGE9MHgwMDAwMDBlMDA0MDAwMA0KPj4+Pj4gW8KgwqAgNDYuMjAwNDU1XSBVbmhhbmRsZWQgZmF1
bHQgYXQgMHhmZmZmODAwMDEwZmE1MDAwDQo+Pj4+Pg0KPj4+Pj4gW3NuaXBdDQo+Pj4+Pg0KPj4+
Pj4gW8KgwqAgNDYuMjMzMDc5XSBDYWxsIHRyYWNlOg0KPj4+Pj4gW8KgwqAgNDYuMjMzNTU5XcKg
IF9fcGNpX3dyaXRlX21zaV9tc2crMHg3MC8weDE4MA0KPj4+Pj4gW8KgwqAgNDYuMjM0MTQ5XcKg
IHBjaV9tc2lfZG9tYWluX3dyaXRlX21zZysweDI4LzB4MzANCj4+Pj4+IFvCoMKgIDQ2LjIzNDg2
OV3CoCBtc2lfZG9tYWluX2FjdGl2YXRlKzB4NWMvMHg4OA0KPj4+Pj4NCj4+Pj4+ICAgIEZyb20g
dGhlIGFib3ZlIHlvdSBjYW4gc2VlIHRoYXQgQkFScyBhcmUgbWFwcGVkIGZvciBEb21haW4tMCBu
b3cNCj4+Pj4+DQo+Pj4+PiBvbmx5IHdoZW4gYW4gYXNzaWduZWQgUENJIGRldmljZSBnZXRzIGVu
YWJsZWQgaW4gRG9tYWluLTAuDQo+Pj4+Pg0KPj4+Pj4gQW5vdGhlciB0aGluZyB0byBub3RlIGlz
IHRoYXQgd2UgY3Jhc2ggb24gTVNJLVggYWNjZXNzIGFzIEJBUnMgYXJlIG1hcHBlZA0KPj4+Pj4N
Cj4+Pj4+IHRvIHRoZSBkb21haW4gd2hpbGUgZW5hYmxpbmcgbWVtb3J5IGRlY29kaW5nIGluIHRo
ZSBDT01NQU5EIHJlZ2lzdGVyLA0KPj4+Pj4NCj4+Pj4+IGJ1dCBNU0ktWCBhcmUgaGFuZGxlZCBk
aWZmZXJlbnRseSwgZS5nLiB3ZSBoYXZlIE1TSS1YIGhvbGVzIGluIHRoZSBtYXBwaW5ncy4NCj4+
Pj4+DQo+Pj4+PiBUaGlzIGlzIGJlY2F1c2UgYmVmb3JlIHRoaXMgY2hhbmdlIHRoZSB3aG9sZSBQ
Q0kgYXBlcnR1cmUgd2FzIG1hcHBlZCBpbnRvDQo+Pj4+Pg0KPj4+Pj4gRG9tYWluLTAgYW5kIGl0
IGlzIG5vdC4gVGh1cywgTVNJLVggaG9sZXMgYXJlIGxlZnQgdW5tYXBwZWQgbm93IGFuZCB0aGVy
ZQ0KPj4+Pj4NCj4+Pj4+IHdhcyBubyBuZWVkIHRvIGRvIHNvLCBlLmcuIHRoZXkgd2VyZSBhbHdh
eXMgbWFwcGVkIGludG8gRG9tYWluLTAgYW5kDQo+Pj4+Pg0KPj4+Pj4gZW11bGF0ZWQgZm9yIGd1
ZXN0cy4NCj4+Pj4+DQo+Pj4+PiBQbGVhc2Ugbm90ZSB0aGF0IG9uZSBjYW5ub3QgdXNlIHhsIHBj
aS1hdHRhY2ggaW4gdGhpcyBjYXNlIHRvIGF0dGFjaCB0aGUgUENJIGRldmljZQ0KPj4+Pj4NCj4+
Pj4+IGluIHF1ZXN0aW9uIHRvIERvbWFpbi0wIGFzIChwbGVhc2Ugc2VlIHRoZSBsb2cpIHRoYXQg
ZGV2aWNlIGlzIGFscmVhZHkgYXR0YWNoZWQuDQo+Pj4+Pg0KPj4+Pj4gTmVpdGhlciBpdCBjYW4g
YmUgZGV0YWNoZWQgYW5kIHJlLWF0dGFjaGVkLiBTbywgd2l0aG91dCBtYXBwaW5nIE1TSS1YIGhv
bGVzIGZvcg0KPj4+Pj4NCj4+Pj4+IERvbWFpbi0wIHRoZSBkZXZpY2UgYmVjb21lcyB1bnVzYWJs
ZSBpbiBEb21haW4tMC4gQXQgdGhlIHNhbWUgdGltZSB0aGUgZGV2aWNlDQo+Pj4+Pg0KPj4+Pj4g
Y2FuIGJlIHN1Y2Nlc3NmdWxseSBwYXNzZWQgdG8gRG9tVS4NCj4+Pj4+DQo+Pj4+Pg0KPj4+Pj4g
SnVsaWVuLCBTdGVmYW5vISBQbGVhc2UgbGV0IG1lIGtub3cgaG93IGNhbiB3ZSBwcm9jZWVkIHdp
dGggdGhpcy4NCj4+Pj4gV2hhdCB3YXMgdGhlIHBsYW4gZm9yIE1TSS1YIGluIERvbTA/DQo+Pj4g
SXQganVzdCB3b3JrZWQgYmVjYXVzZSB3ZSBtYXBwZWQgZXZlcnl0aGluZw0KPj4+PiBHaXZlbiB0
aGF0IERvbTAgaW50ZXJhY3RzIHdpdGggYSB2aXJ0dWFsLUlUUyBhbmQgdmlydHVhbC1HSUMgcmF0
aGVyIHRoYW4NCj4+Pj4gYSBwaHlzaWNhbC1JVFMgYW5kIHBoeXNpY2FsLUdJQywgSSBpbWFnaW5l
IHRoYXQgaXQgd2Fzbid0IGNvcnJlY3QgZm9yDQo+Pj4+IERvbTAgdG8gd3JpdGUgdG8gdGhlIHJl
YWwgTVNJLVggdGFibGUgZGlyZWN0bHk/DQo+Pj4+DQo+Pj4+IFNob3VsZG4ndCBEb20wIGdldCBl
bXVsYXRlZCBNU0ktWCB0YWJsZXMgbGlrZSBhbnkgRG9tVT8NCj4+Pj4NCj4+Pj4gT3RoZXJ3aXNl
LCBpZiBEb20wIGlzIGV4cGVjdGVkIHRvIGhhdmUgdGhlIHJlYWwgTVNJLVggdGFibGVzIG1hcHBl
ZCwgdGhlbg0KPj4+PiBJIHdvdWxkIHN1Z2dlc3QgdG8gbWFwIHRoZW0gYXQgdGhlIHNhbWUgdGlt
ZSBhcyB0aGUgQkFScy4gQnV0IEkgYW0NCj4+Pj4gdGhpbmtpbmcgdGhhdCBEb20wIHNob3VsZCBn
ZXQgZW11bGF0ZWQgTVNJLVggdGFibGVzLCBub3QgcGh5c2ljYWwgTVNJLVgNCj4+Pj4gdGFibGVz
Lg0KPj4+IFllcywgaXQgc2VlbXMgbW9yZSB0aGFuIHJlYXNvbmFibGUgdG8gZW5hYmxlIGVtdWxh
dGlvbiBmb3IgRG9tYWluLTANCj4+Pg0KPj4+IGFzIHdlbGwuIE90aGVyIHRoYW4gdGhhdCwgU3Rl
ZmFubywgZG8geW91IHRoaW5rIHdlIGFyZSBnb29kIHRvIGdvIHdpdGgNCj4+Pg0KPj4+IHRoZSBj
aGFuZ2VzIEkgZGlkIGluIG9yZGVyIHRvIHVubWFwIGV2ZXJ5dGhpbmcgZm9yIERvbWFpbi0wPw0K
Pj4NCj4+IEl0IG1pZ2h0IGJlIGJldHRlciB0byByZXNlbmQgdGhlIHNlcmllcyB3aXRoIHRoZSBw
YXRjaCBpbiBpdCwgYmVjYXVzZSBpdA0KPj4gaXMgZGlmZmljdWx0IHRvIHJldmlldyB0aGUgcGF0
Y2ggbGlrZSB0aGlzLg0KDQpUaGlzIGlzIHRydWUuIFRha2luZyBpbnRvIGFjY291bnQgWGVuIHJl
bGVhc2UgcGxhbiBJIGFtIGp1c3QgdHJ5aW5nIHRvDQoNCm1pbmltaXplIHRoZSB0dXJuIGFyb3Vu
ZCBoZXJlLiBTb3JyeSBhYm91dCB0aGlzDQoNCj4+ICAgTm9uZXRoZWxlc3MgSSB0cmllZCwgYnV0
IEkNCj4+IG1pZ2h0IGhhdmUgbWlzc2VkIHNvbWV0aGluZy4NClRoYW5rIHlvdSBmb3IgeW91ciB0
aW1lISENCj4+DQo+PiBQcmV2aW91c2x5IHRoZSB3aG9sZSBQQ0llIGJyaWRnZSBhcGVydHVyZSB3
YXMgbWFwcGVkIHRvIERvbTAsIGFuZA0KPj4gaXQgd2FzIGRvbmUgYnkgbWFwX3JhbmdlX3RvX2Rv
bWFpbiwgaXMgdGhhdCBjb3JyZWN0Pw0KDQpZZXMsIGJ1dCBub3Qgb25seSB0aGUgYXBlcnR1cmU6
IHBsZWFzZSBzZWUgYmVsb3cuDQoNCg0KPj4NCj4+IE5vdyB0aGlzIHBhdGNoLCB0byBhdm9pZCBt
YXBwaW5nIHRoZSBlbnRpcmUgYXBlcnR1cmUgdG8gRG9tMCwgaXMNCj4+IHNraXBwaW5nIGFueSBv
cGVyYXRpb25zIGZvciBQQ0llIGRldmljZXMgaW4gbWFwX3JhbmdlX3RvX2RvbWFpbiBieQ0KPj4g
c2V0dGluZyBuZWVkX21hcHBpbmcgPSBmYWxzZS4NCj4+DQo+PiBUaGUgaWRlYSBpcyB0aGF0IGlu
c3RlYWQsIHdlJ2xsIG9ubHkgbWFwIHRoaW5ncyB3aGVuIG5lZWRlZCBhbmQgbm90IHRoZQ0KPj4g
d2hvbGUgYXBlcnR1cmUuIEhvd2V2ZXIsIGxvb2tpbmcgYXQgdGhlIGNoYW5nZXMgdG8NCj4+IHBj
aV9ob3N0X2JyaWRnZV9tYXBwaW5ncyAoZm9ybWVybHkga25vd24gYXMNCj4+IHBjaV9ob3N0X2Jy
aWRnZV9uZWVkX3AybV9tYXBwaW5nKSwgaXQgaXMgc3RpbGwgZ29pbmcgdGhyb3VnaCB0aGUgZnVs
bA0KPj4gbGlzdCBvZiBhZGRyZXNzIHJhbmdlcyBvZiB0aGUgUENJZSBicmlkZ2UgYW5kIG1hcCBl
YWNoIHJhbmdlIG9uZSBieSBvbmUNCj4+IHVzaW5nIG1hcF9yYW5nZV90b19kb21haW4uIEFsc28s
IHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5ncyBpcyBzdGlsbA0KPj4gY2FsbGVkIHVuY29uZGl0aW9u
YWxseSBhdCBib290IGZvciBEb20wLg0KPj4NCj4+IFNvIEkgYW0gbWlzc2luZyB0aGUgcGFydCB3
aGVyZSB0aGUgYXBlcnR1cmUgaXMgYWN0dWFsbHkgKm5vdCogbWFwcGVkIHRvDQo+PiBEb20wLg0K
V2l0aCBtYXBfcmFuZ2VfdG9fZG9tYWluIHdlIGFsc28gbWFwcGVkIGFsbCB0aGUgZW50cmllcw0K
DQpvZiB0aGUgInJlZyIgYW5kICJyYW5nZXMiIHByb3BlcnRpZXMuIExldCdzIGhhdmUgYSBsb29r
IGF0IFsxXToNCg0KLSByYW5nZXPCoMKgwqDCoMKgwqDCoMKgIDogQXMgZGVzY3JpYmVkIGluIElF
RUUgU3RkIDEyNzUtMTk5NCwgYnV0IG11c3QgcHJvdmlkZQ0KIMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCBhdCBsZWFzdCBhIGRlZmluaXRpb24gb2Ygbm9uLXByZWZldGNoYWJs
ZSBtZW1vcnkuIE9uZQ0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBvciBi
b3RoIG9mIHByZWZldGNoYWJsZSBNZW1vcnkgYW5kIElPIFNwYWNlIG1heSBhbHNvDQogwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGJlIHByb3ZpZGVkLg0KDQotIHJlZ8KgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgOiBUaGUgQ29uZmlndXJhdGlvbiBTcGFjZSBiYXNlIGFkZHJlc3Mg
YW5kIHNpemUsIGFzIGFjY2Vzc2VkDQogwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIGZyb20gdGhlIHBhcmVudCBidXMuwqAgVGhlIGJhc2UgYWRkcmVzcyBjb3JyZXNwb25kcyB0
bw0KIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB0aGUgZmlyc3QgYnVzIGlu
IHRoZSAiYnVzLXJhbmdlIiBwcm9wZXJ0eS7CoCBJZiBubw0KIMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCAiYnVzLXJhbmdlIiBpcyBzcGVjaWZpZWQsIHRoaXMgd2lsbCBiZSBi
dXMgMCAodGhlIGRlZmF1bHQpLg0KDQpUaGUgbW9zdCBpbnRlcmVzdGluZyBjb21lcyB3aGVuICJy
ZWciIGFsc28gaGFzIG90aGVyIHRoZW4gY29uZmlndXJhdGlvbg0KDQpzcGFjZSBhZGRyZXNzZXMs
IGZvciBleGFtcGxlIFsyXToNCg0KLSByZWc6IFNob3VsZCBjb250YWluIHJjX2RiaSwgY29uZmln
IHJlZ2lzdGVycyBsb2NhdGlvbiBhbmQgbGVuZ3RoLg0KLSByZWctbmFtZXM6IE11c3QgaW5jbHVk
ZSB0aGUgZm9sbG93aW5nIGVudHJpZXM6DQogwqAgInJjX2RiaSI6IGNvbnRyb2xsZXIgY29uZmln
dXJhdGlvbiByZWdpc3RlcnM7DQogwqAgImNvbmZpZyI6IFBDSWUgY29uZmlndXJhdGlvbiBzcGFj
ZSByZWdpc3RlcnMuDQoNClNvLCB3ZSBkb24ndCBuZWVkIHRvIG1hcCAicmFuZ2VzIiBhbmQgKmNv
bmZpZyogZnJvbSB0aGUgInJlZyIsIGJ1dCBhbGwNCg0KdGhlIHJlc3QgZnJvbSAicmVnIiBzdGls
bCBuZWVkcyB0byBiZSBtYXBwZWQgdG8gRG9tYWluLTAsIHNvIHRoZSBQQ0llDQoNCmJyaWRnZSBj
YW4gcmVtYWluIGZ1bmN0aW9uYWwgaW4gRG9tYWluLTAuDQoNCj4+ICAgV2hhdCdzIHRoZSBkaWZm
ZXJlbmNlIGJldHdlZW4gdGhlIGxvb3AgaW4NCj4+IHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5nczoN
Cj4+DQo+PiAgICBmb3IgKCBpID0gMDsgaSA8IGR0X251bWJlcl9vZl9hZGRyZXNzKGRldik7IGkr
KyApDQo+PiAgICAgIG1hcF9yYW5nZV90b19kb21haW4uLi4NCj4+DQo+PiBhbmQgdGhlIHByZXZp
b3VzIGNvZGUgaW4gbWFwX3JhbmdlX3RvX2RvbWFpbj8gSSB0aGluayBJIGFtIG1pc3NpbmcNCj4+
IHNvbWV0aGluZyBidXQgYXMgbWVudGlvbmVkIGl0IGlzIGRpZmZpY3VsdCB0byByZXZpZXcgdGhl
IHBhdGNoIGxpa2UgdGhpcw0KPj4gb3V0IG9mIG9yZGVyLg0KPj4NCj4+IEFsc28sIGFuZCB0aGlz
IGlzIG1pbm9yLCBldmVuIGlmIGN1cnJlbnRseSB1bnVzZWQsIGl0IG1pZ2h0IGJlIGdvb2QgdG8N
Cj4+IGtlZXAgYSBsZW5ndGggcGFyYW1ldGVyIHRvIHBjaV9ob3N0X2JyaWRnZV9uZWVkX3AybV9t
YXBwaW5nLg0KPiBJdCBsb29rcyBsaWtlIHRoZSBmaWx0ZXJpbmcgaXMgZG9uZSBiYXNlZCBvbjoN
Cj4NCj4gcmV0dXJuIGNmZy0+cGh5c19hZGRyICE9IGFkZHINCg0KQXMgSSBleHBsYWluZWQgYWJv
dmUgaXQgaXMgKm5vdyogdGhlIG9ubHkgcmFuZ2UgdGhhdCB3ZSAqZG8gbm90IHdhbnQqIHRvDQoN
CmJlIG1hcHBlZCB0byBEb21haW4tMC4gT3RoZXIgInJlZyIgZW50cmllcyBzdGlsbCBuZWVkIHRv
IGJlIG1hcHBlZC4NCg0KPg0KPiBpbiBwY2lfZWNhbV9uZWVkX3AybV9tYXBwaW5nIHRoYXQgaXMg
ZXhwZWN0ZWQgdG8gZmlsdGVyIG91dCB0aGUgYWRkcmVzcw0KPiByYW5nZXMgd2UgZG9uJ3Qgd2Fu
dCB0byBtYXAgYmVjYXVzZSBpdCBjb21lcyBmcm9tDQo+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtY29tbW9uLmM6Z2VuX3BjaV9pbml0Og0KPg0KPiAgICAgIC8qIFBhcnNlIG91ciBQQ0kgZWNh
bSByZWdpc3RlciBhZGRyZXNzKi8NCj4gICAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJlc3Mo
ZGV2LCBlY2FtX3JlZ19pZHgsICZhZGRyLCAmc2l6ZSk7DQo+ICAgICAgaWYgKCBlcnIgKQ0KPiAg
ICAgICAgICBnb3RvIGVycl9leGl0Ow0KPg0KPiBJbiBwY2lfaG9zdF9icmlkZ2VfbWFwcGluZ3Ms
IGluc3RlYWQgb2YgcGFyc2luZyBkZXZpY2UgdHJlZSBhZ2FpbiwgY2FuJ3QNCj4gd2UganVzdCBm
ZXRjaCB0aGUgYWRkcmVzcyBhbmQgbGVuZ3RoIHdlIG5lZWQgdG8gbWFwIHN0cmFpZ2h0IGZyb20N
Cj4gYnJpZGdlLT5zeXNkYXRhLT5waHlzX2FkZHIvc2l6ZSA/DQoNCldlIGNhbid0IGFzIHRoZSBh
ZGRyZXNzIGRlc2NyaWJlcyB0aGUgY29uZmlndXJhdGlvbiBzcGFjZSB3aGljaCB3ZQ0KDQoqZG8g
bm90KiB3YW50IHRvIGJlIG1hcHBlZCwgYnV0IHdoYXQgd2Ugd2FudCBpcyBvdGhlciB0aGFuIGNv
bmZpZ3VyYXRpb24NCg0KZW50cmllcyBpbiB0aGUgInJlZyIgcHJvcGVydHkuDQoNCj4NCj4gQXQg
dGhlIHBvaW50IHdoZW4gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzIGlzIGNhbGxlZCBpbiB5b3Vy
IG5ldyBwYXRjaCwNCj4gd2UgaGF2ZSBhbHJlYWR5IGluaXRpYWxpemVkIGFsbCB0aGUgUENJZS1y
ZWxhdGVkIGRhdGEgc3RydWN0dXJlcy4gSXQNCj4gc2VlbXMgYSBiaXQgdXNlbGVzcyB0byBoYXZl
IHRvIGdvIHZpYSBkZXZpY2UgdHJlZSBhZ2Fpbi4NCg0KQm90dG9tIGxpbmU6IHdlIGRvIG5lZWQg
dG8gZ28gb3ZlciB0aGUgInJlZyIgcHJvcGVydHkgYW5kIG1hcCB0aGUgcmVnaW9ucw0KDQpvZiB3
aGljaCBQQ0llIGJyaWRnZSBpcyBkZXBlbmRlbnQgYW5kIG9ubHkgc2tpcCAiY29uZmlnIiBwYXJ0
IG9mIGl0Lg0KDQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0KDQoNClsxXSBodHRwczovL3d3dy5r
ZXJuZWwub3JnL2RvYy9Eb2N1bWVudGF0aW9uL2RldmljZXRyZWUvYmluZGluZ3MvcGNpL2hvc3Qt
Z2VuZXJpYy1wY2kudHh0DQoNClsyXSBodHRwczovL3d3dy5rZXJuZWwub3JnL2RvYy9Eb2N1bWVu
dGF0aW9uL2RldmljZXRyZWUvYmluZGluZ3MvcGNpL2hpc2lsaWNvbi1wY2llLnR4dA0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:30:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:30:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188150.337220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlqL-0000L1-D6; Thu, 16 Sep 2021 07:30:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188150.337220; Thu, 16 Sep 2021 07:30:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlqL-0000Ku-9r; Thu, 16 Sep 2021 07:30:01 +0000
Received: by outflank-mailman (input) for mailman id 188150;
 Thu, 16 Sep 2021 07:30:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQlqK-0000Ko-11
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:30:00 +0000
Received: from mga04.intel.com (unknown [192.55.52.120])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bec81154-986e-45cc-b80f-d8ecf05fd8b3;
 Thu, 16 Sep 2021 07:29:57 +0000 (UTC)
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
 by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:29:12 -0700
Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83])
 by fmsmga002.fm.intel.com with ESMTP; 16 Sep 2021 00:29:12 -0700
Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by
 fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:29:11 -0700
Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by
 fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:29:11 -0700
Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by
 fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:29:11 -0700
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.42) by
 edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:29:09 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN8PR11MB3602.namprd11.prod.outlook.com (2603:10b6:408:8e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 07:29:05 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:29:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bec81154-986e-45cc-b80f-d8ecf05fd8b3
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="220622244"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="220622244"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="553730497"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IQB1uwW1sMv4/gxwoZ7VgTtSNv0/ExVwDFxrGt4u4a7O/e4C7GAjs5hNm4s5S27KKv2ixDzeCF2RKyQAc/eFnUJS1iYJbWTmq+/3ytql91MonPeo2azw5PmSrkzckRQld084soxWGZTpF0qrB8xKYmzEXnJMV0Y+j8lmefldglwt5EzhHMTPVNilFMqI9TWiD6+8AKFbND4+8wKnPkWUdY1JsWfj/rKbaQc3pKDWWjoatp/m6xtydYWE0xFCH77o4gfPMjkmrWZPf/1ttR/Anz53tdWhDKm0xxDIomIsJ/6v92KQ07QsscOS9rBcXdzOlt6bPgS49snUxIFfs3+JHA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Vid34ud0rFRa4Wkk82svfoB2ymr9OfvkJrroyZKvVtc=;
 b=TzL+xHTVRSBQ0Me6P9+OloVS33Qpa9FfCSv4Ny7HCa2QbsLcVlX4Q6kcqS1OdafzjQDvwf8yNpageA+MerODGtajMoZO+S+gt/JKyww0ObiFd4FcKMFNq468wu8U8x/6j0vTU5ao5SgFKu/6YhY6zfpvTNz5ZsTfaNq0oXeoEFci+fY9dondNTF31rYxh8r+rNhg7sIl2WQH/g5vgzPvUamSGgIQ6xGNDOh/hOIDXRJ7Ko25a/e+0Qt/MxpQvFxANYM+/P3i8PxLVZy/5j82kQxNHmg6VspX1SI9TddQdysTxpeO/YDmGEiGiC8cs7kOtxTxYXpaoiq0r5KOvJlKHw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Vid34ud0rFRa4Wkk82svfoB2ymr9OfvkJrroyZKvVtc=;
 b=Vvwoof6kZjK7AS1aF0QvAcd1EO8yrx8Ih2dmsryHDFef3/94iK5Dwvb8E5XhROrsBDakKCKuIstHomipyAjM/LPHIssnaZH2lEfN0eh2uX8izHLamnbIM9qAGK1VFGbBHXdX9cio4m9CEa8cZSi7mf5dlQe7TqGAS9Am6p7L8Qo=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
Thread-Topic: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
Thread-Index: AQHXmPQ6RAwx6Qn8tEyqmw4lz+vBDqumZRwQ
Date: Thu, 16 Sep 2021 07:29:05 +0000
Message-ID: <BN9PR11MB543311E87C32397FB82583A48CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
In-Reply-To: <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9ad4db60-c6ac-4132-643f-08d978e3aa46
x-ms-traffictypediagnostic: BN8PR11MB3602:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN8PR11MB3602F97C1E9E6296120ECCFB8CDC9@BN8PR11MB3602.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:240;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: j9/IVTlgij95ebhTKHdqdlt6SQ2PZiwQEM49E8itDxe4ztdIXkYQ4TYLcnluOTt6BDN8JetlmnBenU5J+QPGe0FVEIUpcfPS7+oWtVsDC5NEkvNXwVib5KRd6hOgK3mkLDv/s7NILYu5KyY0Ud/c9L7+SA9urhfyXyTP3gx8OMhYuQAN90CoK8NbpuA7t9fP4Ix7NeD6uryPgwc1WmeWiUlZW02DPKceLguqO1oouxvmcwtNjlArblGdQF9HyH9v3/w9gnZUlIXK4kFXJq30ddCehXoOCVQ0uvjfky9L9Vz+b6Rkr22UJpgQwT1s3s8bD9rIeXRV4tUUo31AF97ZFJQ2KdKpBiEA3KWveSButTSrX5bfgh5oyZylmVnrl+idiaREo2R257RJgqPwuidOH3JVBcCenVpANV7vVWh9PC1TAL7DTvcz7tk7BMWRFKkMNwbYdSd2PFHY2D8TjEIJ+/UDTAar3QqYxqbQ5+QOdmJuFciWUsM9sCzMF+J6uW5HCiCOQQLuwgcHC/JDXFdtD5J5F4pqLfRyBTgN4+aBowduBwVPj5/6wvC6rDjBoXasOC+33St6WHQYQQHydUTs6rUZinJCjBe/Zm3DpirtBPazba3OErie6FmtPH91eYmvb/HHwSYX9OlGZaUYKrG2FZn5KtOdauzsRwAo6sk4zgvz704qEP/pZOsvhxFq7YsANYzOFvimGiVwgds11wzJkQ==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(39860400002)(366004)(136003)(346002)(86362001)(9686003)(66446008)(66946007)(478600001)(71200400001)(54906003)(5660300002)(33656002)(76116006)(6506007)(186003)(52536014)(7696005)(66476007)(316002)(110136005)(38100700002)(55016002)(2906002)(8676002)(64756008)(26005)(66556008)(38070700005)(8936002)(122000001)(4326008);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?M1ZYWTVXTjdjeG1MaHZsdVdFSVhRdW5IWmJnZEE1Ymt2dkh5bmx1blRNLzVY?=
 =?utf-8?B?WVdJZGdEWFk5azVraTF2TTNqRlhTaWhobGtDRVhzdzlDSlFYalc4Q0UvZUR2?=
 =?utf-8?B?VVFNQ0JJaGdNemFRdjhUUk81ZnM4c08ybGx6RmxUdnVtTkw1MkY0UUhJalNK?=
 =?utf-8?B?MW1oY3hiMnQxMzJxTDJrK2VvVjlpMEhGeHVvMHJpQml4OEpCUThlcEhrNm9T?=
 =?utf-8?B?Q3lMSWRWdlJyRGU3TUJaa0laa3UvdytjY092RU5TQzNQek5Hd3p2RXBYblJZ?=
 =?utf-8?B?bDFwM1FzZ0FFV0dFa3RmV0EvWjVScHRwU1pYVFIrYzVMTDhxNERvK1ROZ0Yz?=
 =?utf-8?B?STdmSnBhaFNSU0lhUU1zMWRJZUVXUXk1T2pjcTZyaFRxVWRaY0VEdnB1RGg5?=
 =?utf-8?B?alR2Y241RGQxSndFK3MrZUdDbm5iVGpoYlRwUDhqcnAvRWdqUUFkcGdDVWg4?=
 =?utf-8?B?OG5WNi9hRnVTbjN2S1V6c2tXcThhQkhyZWFhYUdxMkk4YVJmbjdHL2hpKzdL?=
 =?utf-8?B?N1MrVHVjOUhrSkMvRmltRU9vYjZnQWFSMlZMWXcxKytkcFNycnpmbUVYWFJF?=
 =?utf-8?B?TFg2WDdSOFA1S1JFWVlBNVRhQVdwak5vR2gwd3JXM2l3d0Y5L0Q5dGdBejk0?=
 =?utf-8?B?RjZLbWphdWJlUFVSSkNqUzRtWDREZ2ExdmJTY2pkU0U5OFdUTzk0bXY1cWJ5?=
 =?utf-8?B?VCsxWjN6QnJwV1hZVkpSTkJZWXg2M25PRDRrV0Iyb3U2M0hpLzlFMmRwZ2Mr?=
 =?utf-8?B?d0lZRFRaNERiVXNKOTJBalFSY1lFbjFjWmlaOFdXbGcyTDg2dHNzQ0lpZkpL?=
 =?utf-8?B?d21DcU1zdU9FTmxKaUFJYkg4TmZBRUtZcmRtay92QWdEdFZ6ckplQThHT2Yz?=
 =?utf-8?B?RTc5dWUvM2c4d2d5dGl3QmlBTy9jOXFGZWZGVlRVaERoamJOQkNHSjE4ZVlB?=
 =?utf-8?B?Y1JYVnVSdVh2TFBxb0FjanowTitoU1huTGZRazFTZE5lOUN3VnlVUHNVZ3hJ?=
 =?utf-8?B?MHIwLzFyS2thbUU3QTZ4a3VRYmtubUROWkFoL1BCcjlWWm05Q050NkVqYXpn?=
 =?utf-8?B?V0xnYU9YMTNWb3hLbVIxZHArZVVzNG1GV3hmTHNaVWgwbDJQL014SXJSOGZn?=
 =?utf-8?B?VlZrQWJBWGxuMDhSS2Q4akxneTNzTGF0c0t5Z1BnV1NOdmpJVWJ0KzQ0OGs0?=
 =?utf-8?B?NXVlRDNLU0pmMzRJQ1NnM1FWQXB2Ulg3WVpYSXdlQzZsK2RheVNYZjZGc2ph?=
 =?utf-8?B?Q1ZIZnVwM0t5d0JXR2dDU3RWSDBlVkJvSTBRWXVOVEFCbXcyZUI1cG5sRVpF?=
 =?utf-8?B?ZXdHZzBIL0c5ZFFwWUl5alBxRHZPRkw3ZW1nRHhyN3htd2xmdk5za2pxa0xJ?=
 =?utf-8?B?OTNVUmdsYklLUzdjQ2JpcVFoSlgwQzdqckwxbTVTeTd3ZlhHVWx0VHZxSERH?=
 =?utf-8?B?R1IrTzJaSzAzRjc3aVJOcjI4b3NMMm5ZTGpaanRNVXg4YVcwaFBHZ0ZQcmlJ?=
 =?utf-8?B?RC9xZVBUUUlLamhoQ3ZHUncyVG1MVTdEK0kvMVM2SHpwMnVwcTY0RnRFcXhG?=
 =?utf-8?B?U2U3N3lLUHlnSUlXRktCK3FkNzBLWXJybXh5K3RxLytiTExZZWdxS1k0UGVt?=
 =?utf-8?B?WUlLajFBb0NvNGNkNHQwd3pETUdveEFDaU52c1pYWXJJWEZBc0dhLzAvWWxh?=
 =?utf-8?B?Y1hjZDZRVmYzWkJvcWdjYXRGakp4UmZ6VndxUGY3NEJoeHgvZnVBTVFwUU1R?=
 =?utf-8?Q?98+di8i2A2hJxTPBq0si8kpRIlDnq4T3ulZAtr5?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ad4db60-c6ac-4132-643f-08d978e3aa46
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:29:05.8687
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: soaraCOs9ie8UgZBn2Q00Ve1+nufbTFo2kyVI8DAJhVlzJd2AEVPp9Qyw3FNn0DH2dGlxixESd1rmb/+JM+nbw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR11MB3602
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFR1ZXNkYXks
IEF1Z3VzdCAyNCwgMjAyMSAxMDoyOCBQTQ0KPiANCj4gbWFwX2RvbWFpbl9wYWdlKCkgZXQgYWwg
bmV2ZXIgZmFpbDsgbm8gbmVlZCB0byBjaGVjayB0aGVpciByZXR1cm4gdmFsdWVzDQo+IGFnYWlu
c3QgTlVMTCwgYW5kIG5vIG5lZWQgdG8gY2FycnkgZGVhZCBwcmludGsoKXMuDQo+IA0KPiBTaWdu
ZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQoNClJldmlld2VkLWJ5
OiBLZXZpbiBUaWFuIDxrZXZpbi50aWFuQGludGVsLmNvbT4NCg0KPiANCj4gLS0tIGEveGVuL2Ry
aXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYw0KPiArKysgYi94ZW4vZHJpdmVy
cy9wYXNzdGhyb3VnaC9hbWQvcGNpX2FtZF9pb21tdS5jDQo+IEBAIC01MzAsMTIgKzUzMCw2IEBA
IHN0YXRpYyB2b2lkIGFtZF9kdW1wX3BhZ2VfdGFibGVfbGV2ZWwoc3QNCj4gICAgICAgICAgcmV0
dXJuOw0KPiANCj4gICAgICB0YWJsZV92YWRkciA9IF9fbWFwX2RvbWFpbl9wYWdlKHBnKTsNCj4g
LSAgICBpZiAoIHRhYmxlX3ZhZGRyID09IE5VTEwgKQ0KPiAtICAgIHsNCj4gLSAgICAgICAgcHJp
bnRrKCJBTUQgSU9NTVUgZmFpbGVkIHRvIG1hcCBkb21haW4gcGFnZSAlIlBSSXBhZGRyIlxuIiwN
Cj4gLSAgICAgICAgICAgICAgICBwYWdlX3RvX21hZGRyKHBnKSk7DQo+IC0gICAgICAgIHJldHVy
bjsNCj4gLSAgICB9DQo+IA0KPiAgICAgIGZvciAoIGluZGV4ID0gMDsgaW5kZXggPCBQVEVfUEVS
X1RBQkxFX1NJWkU7IGluZGV4KysgKQ0KPiAgICAgIHsNCj4gLS0tIGEveGVuL2RyaXZlcnMvcGFz
c3Rocm91Z2gvdnRkL2lvbW11LmMNCj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRk
L2lvbW11LmMNCj4gQEAgLTI4MDAsMTIgKzI4MDAsNiBAQCBzdGF0aWMgdm9pZCB2dGRfZHVtcF9w
YWdlX3RhYmxlX2xldmVsKHBhDQo+ICAgICAgICAgIHJldHVybjsNCj4gDQo+ICAgICAgcHRfdmFk
ZHIgPSBtYXBfdnRkX2RvbWFpbl9wYWdlKHB0X21hZGRyKTsNCj4gLSAgICBpZiAoIHB0X3ZhZGRy
ID09IE5VTEwgKQ0KPiAtICAgIHsNCj4gLSAgICAgICAgcHJpbnRrKFZURFBSRUZJWCAiIGZhaWxl
ZCB0byBtYXAgZG9tYWluIHBhZ2UgJSJQUklwYWRkciJcbiIsDQo+IC0gICAgICAgICAgICAgICBw
dF9tYWRkcik7DQo+IC0gICAgICAgIHJldHVybjsNCj4gLSAgICB9DQo+IA0KPiAgICAgIG5leHRf
bGV2ZWwgPSBsZXZlbCAtIDE7DQo+ICAgICAgZm9yICggaSA9IDA7IGkgPCBQVEVfTlVNOyBpKysg
KQ0KPiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvdXRpbHMuYw0KPiArKysgYi94
ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvdXRpbHMuYw0KPiBAQCAtMTA2LDExICsxMDYsNiBA
QCB2b2lkIHByaW50X3Z0ZF9lbnRyaWVzKHN0cnVjdCB2dGRfaW9tbXUNCj4gICAgICB9DQo+IA0K
PiAgICAgIHJvb3RfZW50cnkgPSAoc3RydWN0IHJvb3RfZW50cnkgKiltYXBfdnRkX2RvbWFpbl9w
YWdlKGlvbW11LQ0KPiA+cm9vdF9tYWRkcik7DQo+IC0gICAgaWYgKCByb290X2VudHJ5ID09IE5V
TEwgKQ0KPiAtICAgIHsNCj4gLSAgICAgICAgcHJpbnRrKCIgICAgcm9vdF9lbnRyeSA9PSBOVUxM
XG4iKTsNCj4gLSAgICAgICAgcmV0dXJuOw0KPiAtICAgIH0NCj4gDQo+ICAgICAgcHJpbnRrKCIg
ICAgcm9vdF9lbnRyeVslMDJ4XSA9ICUiUFJJeDY0IlxuIiwgYnVzLCByb290X2VudHJ5W2J1c10u
dmFsKTsNCj4gICAgICBpZiAoICFyb290X3ByZXNlbnQocm9vdF9lbnRyeVtidXNdKSApDQo+IEBA
IC0xMjMsMTEgKzExOCw2IEBAIHZvaWQgcHJpbnRfdnRkX2VudHJpZXMoc3RydWN0IHZ0ZF9pb21t
dQ0KPiAgICAgIHZhbCA9IHJvb3RfZW50cnlbYnVzXS52YWw7DQo+ICAgICAgdW5tYXBfdnRkX2Rv
bWFpbl9wYWdlKHJvb3RfZW50cnkpOw0KPiAgICAgIGN0eHRfZW50cnkgPSBtYXBfdnRkX2RvbWFp
bl9wYWdlKHZhbCk7DQo+IC0gICAgaWYgKCBjdHh0X2VudHJ5ID09IE5VTEwgKQ0KPiAtICAgIHsN
Cj4gLSAgICAgICAgcHJpbnRrKCIgICAgY3R4dF9lbnRyeSA9PSBOVUxMXG4iKTsNCj4gLSAgICAg
ICAgcmV0dXJuOw0KPiAtICAgIH0NCj4gDQo+ICAgICAgdmFsID0gY3R4dF9lbnRyeVtkZXZmbl0u
bG87DQo+ICAgICAgcHJpbnRrKCIgICAgY29udGV4dFslMDJ4XSA9ICUiUFJJeDY0Il8lIlBSSXg2
NCJcbiIsDQo+IEBAIC0xNTEsMTEgKzE0MSw2IEBAIHZvaWQgcHJpbnRfdnRkX2VudHJpZXMoc3Ry
dWN0IHZ0ZF9pb21tdQ0KPiAgICAgIGRvDQo+ICAgICAgew0KPiAgICAgICAgICBsID0gbWFwX3Z0
ZF9kb21haW5fcGFnZSh2YWwpOw0KPiAtICAgICAgICBpZiAoIGwgPT0gTlVMTCApDQo+IC0gICAg
ICAgIHsNCj4gLSAgICAgICAgICAgIHByaW50aygiICAgIGwldSA9PSBOVUxMXG4iLCBsZXZlbCk7
DQo+IC0gICAgICAgICAgICBicmVhazsNCj4gLSAgICAgICAgfQ0KPiAgICAgICAgICBsX2luZGV4
ID0gZ2V0X2xldmVsX2luZGV4KGdtZm4sIGxldmVsKTsNCj4gICAgICAgICAgcHRlLnZhbCA9IGxb
bF9pbmRleF07DQo+ICAgICAgICAgIHVubWFwX3Z0ZF9kb21haW5fcGFnZShsKTsNCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:34:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:34:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188158.337231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQluL-0001hg-VG; Thu, 16 Sep 2021 07:34:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188158.337231; Thu, 16 Sep 2021 07:34:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQluL-0001hZ-Rc; Thu, 16 Sep 2021 07:34:09 +0000
Received: by outflank-mailman (input) for mailman id 188158;
 Thu, 16 Sep 2021 07:34:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQluK-0001hT-F3
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:34:08 +0000
Received: from mga07.intel.com (unknown [134.134.136.100])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 790959e6-16c0-11ec-b5af-12813bfff9fa;
 Thu, 16 Sep 2021 07:34:06 +0000 (UTC)
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:34:05 -0700
Received: from orsmsx605.amr.corp.intel.com ([10.22.229.18])
 by fmsmga004.fm.intel.com with ESMTP; 16 Sep 2021 00:34:05 -0700
Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by
 ORSMSX605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:34:05 -0700
Received: from orsmsx606.amr.corp.intel.com (10.22.229.19) by
 ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:34:04 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
 orsmsx606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:34:04 -0700
Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.108)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:34:04 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR1101MB2145.namprd11.prod.outlook.com (2603:10b6:405:51::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 07:33:57 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:33:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 790959e6-16c0-11ec-b5af-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="286193492"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="286193492"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="530076911"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f+GR2XNOBi99kb5iXTS97gmA2V4/v35N+M6cSG35mDze6o6Cwgt4A6MIO80Z6D4RiBmS8NyiSv8stsmUTS+GR0lyxZ36H8X2BlAeyV1xS4PEH3H0Hit4K1WyzNanEoRDkUaCSllUDJGYPZ4vyobZWDUnQ2ga2XJLA+EsFiLyTy958ZXdTrbEtPiUU/ja7Ht8VQEZ6+mKlDqhrQO40ej1bOKug4J6vwAAbQrBhrXz3L18rBFkD77TrDvjpf4azlKsVWbGc+tJxY1WfFEdM2GWCH1pcltBDMCPbt0746h5pK/lT6szymosui0GQUl/KtitHrhHF3p/jadcrhwbU42MDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OJGXHW0s1Fyiyy7krPVWeIvnojJmgYgs09w5r3j8a8A=;
 b=JuaN8+iwyilosb1wbV9yI8dEPdvWlv0wbmacWHd+Lda+a+usIDAH9L707DQPBR3b+8BXEnbYHHN7yTWXOt3KEmVp3gefMVrv0ULQ5w2GRb3OrdD9rbUC1T8jZNvlR5Ejv+MqYGWSL/7uEphXFf3ShIuSRvlmyawVSIBzznDt9VCUHMi6GjJIDgsGAlqtBnWiKhpoMDa5HY3Z0xEmx1IsFeuNSx7SMlAslHVeYtwKoYLpBCNQhYqxJqJz703U+8VdUfYbGHdBr7FsnU4+QJ/YYkL9OixX13piEe61YaWjnKWCPT+NWMSPbV/4d4TeZPgMjuY2Jgd83OZ7pI/DCvKUhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OJGXHW0s1Fyiyy7krPVWeIvnojJmgYgs09w5r3j8a8A=;
 b=wGaMI4bLXOpo2owSE//76bx3fqNYI0cf99CTnNqKlekEQwnyHG9qvastvueJ5U6h2knxJecBuefB3s4NN8h5jQBfsLP9QptyukDpHPnXNsVYBg7ND4xWeK5YHgEo1k0wicaNi+TBcoDj9bZwgr8rt9if6FBUh6GhvyLgBM0Td7k=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 15/17] IOMMU: page table dumping adjustments
Thread-Topic: [PATCH 15/17] IOMMU: page table dumping adjustments
Thread-Index: AQHXmPQtySERcVTCKU+CLBLd1PeNTKuCttiAgCOxiVA=
Date: Thu, 16 Sep 2021 07:33:57 +0000
Message-ID: <BN9PR11MB5433F917721197AE7109C3798CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <d6b3d7e4-7ea0-387c-86d1-1fb3c59688fe@suse.com>
 <666ce0fd-5bc1-f12c-4a13-93bf441a4ce4@suse.com>
In-Reply-To: <666ce0fd-5bc1-f12c-4a13-93bf441a4ce4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 44f2779e-dfd9-401c-6784-08d978e45844
x-ms-traffictypediagnostic: BN6PR1101MB2145:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN6PR1101MB2145CF7C8B3165367441BAAE8CDC9@BN6PR1101MB2145.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: DFMbTl6eRMHA6yVsEz2BBc21Q1Cf9uGgJ64u7Xx5OclRmCGaJpTOg3DMzq5uociLxeKpe3vdrMpY/lGSRTRkONsYoIFrEJR4KOR1gBZIGg2R1sEj6UKPuSkBI3Yf3QoGFUrkRiIFv1INzteMBdTOemjrlln0Yx3NwNRffoOP/jm2JVu/MRn1x9L0cuKn/yi2br9o59tQdIUUcsZzkBZTizyYEBMjJkh/bPhbQs72CGF4SVwqTkaDHPc4boq9RhTOEukurCFO8lfW2SdyJb9NBtEaI6KvCciEQ/WvIOT5ZLf6YjTfS9thTb8c0bZO9dR9thtDkkHLCi9hN9jjhFXt8SzKo5zhieBW234k/UorswJ8w+EkA7ML7wRcpO0Q+/cIwuDug4elKZhIiWS45/U/uMX6n1QbrwDIuAgIvMhsJv7vIXQoju0jrmjAfrDnbZ0aHyZpzeRp49AbwnWGbAy10Zc0PH4WyAUnAUj1EI60Nqwpx0BdaJj8Ysv8exxf++YdKQEsEYQRs1dicFKZ0uBecN3G459I4TCxUhgZ/aiWdADK/htXWooMnjvc+zENC9UrDmDWM0mHw7lMk3/PL5F5tuxzxq3i5nE8WMVAL/HWVp9mRBsCP3hXZGEzuxtq35/RvIKqlswnP8jbuhpSME6n4ZmxoQwLk9UQStHyQMsaAsGpiFXLX/JOqdaS93r3jzUGb6GIpG73IrqI8h07lF1k7w==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39860400002)(366004)(346002)(396003)(376002)(8676002)(7696005)(8936002)(53546011)(4326008)(2906002)(38070700005)(83380400001)(71200400001)(6506007)(316002)(52536014)(478600001)(38100700002)(54906003)(110136005)(186003)(5660300002)(26005)(55016002)(9686003)(76116006)(66476007)(66946007)(66556008)(66446008)(64756008)(86362001)(33656002)(122000001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?Lyt3S0puUXFaOUFGWVUySnBWaGg0Yk90VGdzT1RQU2V4R2kwL1JKM0JBcDdq?=
 =?utf-8?B?bGJoRlFVT2VBQlhsRkJ4elJPTHcrRTUzcXhiNk5yWGVqMWwxQ3dTYVlnNjl0?=
 =?utf-8?B?dXR0bXdGV094OGhXUGUzeUtza0JFWkU4UkZoYmpPcUwrc0R5QUVIS2hxOFVB?=
 =?utf-8?B?Z2gya3FuU2RORGRLZlg5OVJ4SDR4QXZXcVFvdlNTSGtyOHBtYlo2c2ZsRHcr?=
 =?utf-8?B?MFdIQjhUcjBERS9rc2hab0swby9JdU1Nazd3Tjg5MFpPTDZKcHJ3amp0Z3NF?=
 =?utf-8?B?VXJjcUgyQ1Z0dlRIaGorSk9oQkRPZlJJUm1nRW1DMXcxL1Q5QXVoODhSRGIx?=
 =?utf-8?B?MjJ0dTJwNDVmaFdvdXA0NTROQUYvdXFpNzlIcjRtWk5PQlk2U0FEWWlGL3ZQ?=
 =?utf-8?B?aXFMU0RUT1Z0UEFNbndPUm1mZnNUbytwZVRvU3J5dHhJb3laSENJcjFKVWE0?=
 =?utf-8?B?Nlk1QlNuWUo3LzZrM01iSkMzRnVEMVlrRHVocG5OQ3NUQ1FGM28zS1dxY0pJ?=
 =?utf-8?B?VDAwYkdFYWowMkZudHF4ajEvYnZ2UkhKTVMva3dSVGp1djNSeUFJVnZnTnRF?=
 =?utf-8?B?bWttSjUwcUtmcitYQ0Q2VFpLZzkwOVFzK3dtcTNSUDZRQ1BEV00rY2xlYWRL?=
 =?utf-8?B?Tk1CcGt5SDV3VHlhRWNKUHhmZjBVbnZWMDlXcDZyS3lqc0JJblNHbWd5ekVC?=
 =?utf-8?B?Uks3SHZaS0wyZGxGRnh5Z1ZYTEs3cEo3SHRmNis4L29LeDV3czJBczZxZVZU?=
 =?utf-8?B?aC9FQnplYkdYMW1UUTJKdG10SXlpbEFiUzY5OWQ0czVMS2FPSTI0eXB1dGJS?=
 =?utf-8?B?ZDhtK1dra1pMeVd1alpoT2UrRUhXOFhoN3VLRjROWlhpMjcwTXU5bitFdndP?=
 =?utf-8?B?R3pQeEtVbUxINmtTenl4U3JNVnlzZjh1UDlGZEV5blNxcGlnUElGeFJFdW9T?=
 =?utf-8?B?dkhTRExha3MxR3plcGk1M1YzaUZvSjdYVlE0Q2FHZnROdUNubjVsWHgrWkpB?=
 =?utf-8?B?TERkU2NRcGIyVmpxR1M2N3lnVXBVa0JwUVZYYURLaDcrOWluMFN6YlR6anpy?=
 =?utf-8?B?TWJkZkpESlF5Nk5CZXJhYktFcTB4U3RWRUtQS3U3Znc0d3VjRTM3OU00SEo0?=
 =?utf-8?B?NGxvVHZGMzg5R3k3WkRDSFhXdDZjZENwM09HQm40S09wY2syUzdEUHpQL0RP?=
 =?utf-8?B?a05RZllUQ1ZIWnpmSFdUSS9GeWtEeVd2L3dmekcySExQd2VXL2RIM1ZvZUtF?=
 =?utf-8?B?WGtGSXc1RS9yU0pSV2xETzZ1UHpSV2tRQ21CSlEzYXJ5NzlQZUFFY2FoaTVY?=
 =?utf-8?B?VzQ3NjV6RnBmdmhWcTZ4ZmJ4ZUg3cmJvd0psYlRSa25HVEpsbk9FejBDUS83?=
 =?utf-8?B?V1BibTdoRERvSDAzN3JpOHp5WjlicDU1VkhaWWdBYTVPeE9IVHhmYmxrZ1RL?=
 =?utf-8?B?N1EzRUJiQXZUbWRuemRNQ3J0ZnpLZXFiN0ZubFE5RkZRSGMwS1NZYUpHUE4r?=
 =?utf-8?B?OEpTcEVXaTJldHQ5TTYyaGc3UmpKOXV0aTRWV0NNYmtVaC9Ec0VkQVZRQ1dn?=
 =?utf-8?B?VHNwSHptYU9GaDhWSXlnVExpSEtmb1Z6eXIxbjVQcEdOY1drTDZyWFpJeTdQ?=
 =?utf-8?B?WUZRRXd0ZnRPNzNJNzB1OGxEaFpCTG55RWZWR2p1RVVvQnZEcWxSeDJ5a2dT?=
 =?utf-8?B?b1FFZWR5anZNQ0xSNEdoeDFZSzRqVUg4RG1FbFR2d1AyV294UUhwYUpBSTVk?=
 =?utf-8?Q?bBzVfZAk1e4HuoiYolAcpToYLeNAsEQPjrzRRg/?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 44f2779e-dfd9-401c-6784-08d978e45844
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:33:57.7559
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: DTVq6RwcLIZYBenmYKdDzrz4F7xrOJBVx6VY45kGh4/2Njskv+JxWUF6NkqumsDOEn5x8I/ZlvuT9tYvR1BkuA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2145
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFR1ZXNkYXks
IEF1Z3VzdCAyNCwgMjAyMSAxMDoyOSBQTQ0KPiANCj4gT24gMjQuMDguMjAyMSAxNjoyNiwgSmFu
IEJldWxpY2ggd3JvdGU6DQo+ID4gRm9yIG9uZSBub25lIG9mIHRoZSB0aHJlZSBJT01NVSBpbXBs
ZW1lbnRhdGlvbnMgb24gQXJtIHNwZWNpZnkgYQ0KPiBkdW1waW5nDQo+ID4gaG9vay4gR2VuZXJh
bGl6ZSBWVC1kJ3MgImRvbid0IGR1bXAgc2hhcmVkIHBhZ2UgdGFibGVzIiB0byBjb3ZlciBmb3IN
Cj4gPiB0aGlzLg0KPiA+DQo+ID4gRnVydGhlciBpbiB0aGUgcGFzdCBJIHdhcyB0b2xkIHRoYXQg
b24gQXJtIGluIHByaW5jaXBsZSB0aGVyZSBjb3VsZCBiZQ0KPiA+IG11bHRpcGxlIGRpZmZlcmVu
dCBJT01NVXMsIGFuZCBoZW5jZSBkaWZmZXJlbnQgZG9tYWlucycgcGxhdGZvcm1fb3BzDQo+ID4g
cG9pbnRlcnMgY291bGQgZGlmZmVyLiBVc2UgZWFjaCBkb21haW4ncyBvcHMgZm9yIGNhbGxpbmcg
dGhlIGR1bXAgaG9vay4NCj4gPiAoSW4gdGhlIGxvbmcgcnVuIGFsbCB1c2VzIG9mIGlvbW11X2dl
dF9vcHMoKSB3b3VsZCBsaWtlbHkgbmVlZCB0bw0KPiA+IGRpc2FwcGVhciBmb3IgdGhpcyByZWFz
b24uKQ0KPiA+DQo+ID4gU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2Uu
Y29tPg0KPiANCj4gU2hvdWxkIGhhdmUgQ2MtZWQgS2V2aW4gb24gdGhpcyBvbmUgYXMgd2VsbDsg
bm93IGFkZGVkLg0KPiANCj4gSmFuDQoNClJldmlld2VkLWJ5OiBLZXZpbiBUaWFuIDxrZXZpbi50
aWFuQGludGVsLmNvbT4NCg0KPiANCj4gPiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9p
b21tdS5jDQo+ID4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvaW9tbXUuYw0KPiA+IEBA
IC02NDAsMTIgKzY0MCw5IEBAIGJvb2xfdCBpb21tdV9oYXNfZmVhdHVyZShzdHJ1Y3QgZG9tYWlu
ICoNCj4gPiAgc3RhdGljIHZvaWQgaW9tbXVfZHVtcF9wYWdlX3RhYmxlcyh1bnNpZ25lZCBjaGFy
IGtleSkNCj4gPiAgew0KPiA+ICAgICAgc3RydWN0IGRvbWFpbiAqZDsNCj4gPiAtICAgIGNvbnN0
IHN0cnVjdCBpb21tdV9vcHMgKm9wczsNCj4gPg0KPiA+ICAgICAgQVNTRVJUKGlvbW11X2VuYWJs
ZWQpOw0KPiA+DQo+ID4gLSAgICBvcHMgPSBpb21tdV9nZXRfb3BzKCk7DQo+ID4gLQ0KPiA+ICAg
ICAgcmN1X3JlYWRfbG9jaygmZG9tbGlzdF9yZWFkX2xvY2spOw0KPiA+DQo+ID4gICAgICBmb3Jf
ZWFjaF9kb21haW4oZCkNCj4gPiBAQCAtNjUzLDcgKzY1MCwxMyBAQCBzdGF0aWMgdm9pZCBpb21t
dV9kdW1wX3BhZ2VfdGFibGVzKHVuc2lnDQo+ID4gICAgICAgICAgaWYgKCBpc19oYXJkd2FyZV9k
b21haW4oZCkgfHwgIWlzX2lvbW11X2VuYWJsZWQoZCkgKQ0KPiA+ICAgICAgICAgICAgICBjb250
aW51ZTsNCj4gPg0KPiA+IC0gICAgICAgIG9wcy0+ZHVtcF9wYWdlX3RhYmxlcyhkKTsNCj4gPiAr
ICAgICAgICBpZiAoIGlvbW11X3VzZV9oYXBfcHQoZCkgKQ0KPiA+ICsgICAgICAgIHsNCj4gPiAr
ICAgICAgICAgICAgcHJpbnRrKCIlcGQgc2hhcmluZyBwYWdlIHRhYmxlc1xuIiwgZCk7DQo+ID4g
KyAgICAgICAgICAgIGNvbnRpbnVlOw0KPiA+ICsgICAgICAgIH0NCj4gPiArDQo+ID4gKyAgICAg
ICAgZG9tX2lvbW11KGQpLT5wbGF0Zm9ybV9vcHMtPmR1bXBfcGFnZV90YWJsZXMoZCk7DQo+ID4g
ICAgICB9DQo+ID4NCj4gPiAgICAgIHJjdV9yZWFkX3VubG9jaygmZG9tbGlzdF9yZWFkX2xvY2sp
Ow0KPiA+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jDQo+ID4gKysr
IGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRkL2lvbW11LmMNCj4gPiBAQCAtMjgzNSwxMiAr
MjgzNSw2IEBAIHN0YXRpYyB2b2lkIHZ0ZF9kdW1wX3BhZ2VfdGFibGVzKHN0cnVjdA0KPiA+ICB7
DQo+ID4gICAgICBjb25zdCBzdHJ1Y3QgZG9tYWluX2lvbW11ICpoZCA9IGRvbV9pb21tdShkKTsN
Cj4gPg0KPiA+IC0gICAgaWYgKCBpb21tdV91c2VfaGFwX3B0KGQpICkNCj4gPiAtICAgIHsNCj4g
PiAtICAgICAgICBwcmludGsoVlREUFJFRklYICIgJXBkIHNoYXJpbmcgRVBUIHRhYmxlXG4iLCBk
KTsNCj4gPiAtICAgICAgICByZXR1cm47DQo+ID4gLSAgICB9DQo+ID4gLQ0KPiA+ICAgICAgcHJp
bnRrKFZURFBSRUZJWCIgJXBkIHRhYmxlIGhhcyAlZCBsZXZlbHNcbiIsIGQsDQo+ID4gICAgICAg
ICAgICAgYWdhd190b19sZXZlbChoZC0+YXJjaC52dGQuYWdhdykpOw0KPiA+ICAgICAgdnRkX2R1
bXBfcGFnZV90YWJsZV9sZXZlbChoZC0+YXJjaC52dGQucGdkX21hZGRyLA0KPiA+DQo+ID4NCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:36:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:36:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188166.337241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlwG-0002KF-Am; Thu, 16 Sep 2021 07:36:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188166.337241; Thu, 16 Sep 2021 07:36:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlwG-0002K8-7o; Thu, 16 Sep 2021 07:36:08 +0000
Received: by outflank-mailman (input) for mailman id 188166;
 Thu, 16 Sep 2021 07:36:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQlwE-0002K1-IH
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:36:06 +0000
Received: from mga05.intel.com (unknown [192.55.52.43])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bfda514a-16c0-11ec-b5af-12813bfff9fa;
 Thu, 16 Sep 2021 07:36:05 +0000 (UTC)
Received: from orsmga006.jf.intel.com ([10.7.209.51])
 by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:36:04 -0700
Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85])
 by orsmga006.jf.intel.com with ESMTP; 16 Sep 2021 00:36:04 -0700
Received: from fmsmsx604.amr.corp.intel.com (10.18.126.84) by
 fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:36:03 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
 fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:36:03 -0700
Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.45) by
 edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:36:03 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR1101MB2338.namprd11.prod.outlook.com (2603:10b6:404:9c::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep
 2021 07:36:02 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:36:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfda514a-16c0-11ec-b5af-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="308051006"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="308051006"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="433702686"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eJHwkzcNfTgAidpnW5CmJQpxtUIGAzRfjbGg9GTPl2lmzfRvVeYAokJHg5Cy9r77TR+fDlHCypq3siWBfVII8X4lFlSFdGis7tw+QmiZlVTjEZVP6D2CMqMkMsFR/c34VdHpsB3prCCwiYNJZCrELwZleiFAeEAJc+/8ALUn+cJfqp2PRhTVn1XP3agofhBjjpcEVUJjgha1p05IrV325kFX+EHR6vtOxXDyBbR0Ye14l/o0IVa0rM3o8Zwh+N/tJQ5okAC9ngSa7jPf3WRrePb+CmLvbl1fWDxVEnEpQrwGV2F/GDJ3y5u+O4vJ32LlxOoTGObZww8nCaGIFlY4gQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YAMzS/7IF+dEZEaR1J3XvF85uPgjy1S6Ax1BxFc0brU=;
 b=RAgSDnIcOWHBmWkNAoKQN+ZUck4/Wjxzjau+WFN6cir6VRzhddohMDFGn4Cr0L26ENFHtaZR3oFrXmgI69EsRTQEBfGen6yTmz/R/gOGw68ik8o07xQQk81Dv2dkjKR2kYllzyrxh0/E4eYO/KvN4cOJXm4cGfF/ScizubW2fbSURDRzzoeXxn3VVYRj+8hxtQOrCGwVHBGU5tNbnsHmeoa0WQlV1WlrW+D8kcgqoWBvXsvFBeWRQvXdID5ojhUNier7LPfLqVLNM0J3i1rZAO5xTNj1Q5WJ9CjufUdI/TPcvy6eW7ZXj5765bm4h4lR3W6mpPQXtPvH/293H75QLQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YAMzS/7IF+dEZEaR1J3XvF85uPgjy1S6Ax1BxFc0brU=;
 b=vFyHZIsR3PFdl9Rnax6mDPaxITyXuGXoviLc9/9tFz4RwBAiKqyTVbV/AREtoxdS0k6I0DrGH6GKR90AHBbN2ZqlTcgbEj9zPKmi6RpEBYL7eGu0Eb621r6fIR9zSy8I6S6W11gux+6ImahClTZH79sm1x0P1srUc88RXfC/T2U=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 16/17] VT-d: show permissions during page table walks
Thread-Topic: [PATCH 16/17] VT-d: show permissions during page table walks
Thread-Index: AQHXmPQslV7KSBUtgUWpDvTxqV2qqqumaQDw
Date: Thu, 16 Sep 2021 07:36:02 +0000
Message-ID: <BN9PR11MB54333C720A1FAB6B1F3C07E88CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <d4619379-3557-05ac-236a-06a169479a4e@suse.com>
In-Reply-To: <d4619379-3557-05ac-236a-06a169479a4e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 934a5ca5-b9d9-4f09-91b7-08d978e4a265
x-ms-traffictypediagnostic: BN6PR1101MB2338:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN6PR1101MB233813AF4905F23A1C9F47898CDC9@BN6PR1101MB2338.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:913;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 0TYwVLEieFoFGJPjsQbLg0UJSowm/7xRmckhpJsgB4N34p7pZcKTcLoVvhQClubsjOUDAAN7HRXcdGHodRXjmKnJ1+BpsQFTsi5iPlkjGWPqAnmcVooozZTGtJ3jkzDyuYqZaxs5xznrmWvTgEPyQVtrDBRXrzZtdRwSJgOwCGZiPB9FQzVQ3BSbxIbTz42EPCms1bwMAucpJXjTzSLWyg2Vs4ue7AfUR4xWsDLcozdJc5kUyzlORzeJN5vTvmu6RI2OxSu5OG57/0WVSjwWWtPadwPn26iiepaL9PgfHFfcfat7V/OCQ3E3ndW/+bMpcwgLBYzz2GPStT6jpKk9dwlTcbARYWeKO0nfBc5vwgybDZbl1ZafyWLv87BHm2cW5wngmRb2R9K0AFwJxgytQLJ9P2AREgjZ3r2ObNWL2PUMKXtJEyo+pQ1wW2pwpC8M4XZ4mvyuBY4YDmb6JDDQ+c1ofp9aqvBoTmDRKG4lOBo59DfW0FaYE6zK2mQjrzMU2KnUMsYDUbP+CTFWL5FPYan59bzD2OJZYr2Rz57ERIYP3r9dwFbIdj142SOmbwV1Q7CGwegXClsw+YN5GU6ZxvDtLqNwNVb6EaD5UNLaJFSa3j4EhDUfGom4h3Ls853VgGrv3gRxRmnQb4XFmNrAFCNQtXPB45xOhFNEEDVXhIU9s1XbNHTtOeiLwCVIyhmNlImW4sRTFTSPPlSDHT2bGA==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(346002)(39860400002)(136003)(366004)(64756008)(7696005)(66556008)(66946007)(122000001)(71200400001)(478600001)(66476007)(52536014)(6506007)(8676002)(76116006)(110136005)(9686003)(38070700005)(5660300002)(38100700002)(66446008)(86362001)(4326008)(26005)(186003)(55016002)(33656002)(8936002)(316002)(2906002)(54906003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?bXVpUnVIdUlKOTVBNHg1Z1ZUMkNBWGx1cmwzQ1VibGFEYnJBVDdvU25MamV0?=
 =?utf-8?B?eFJ2dVBYbXlRUjVXdWd3bzg2OTJMaWZMUWRoWHd1Q0RjNjJ4eHM4dlNjV2Zo?=
 =?utf-8?B?VUp3ZHBzV3Zaam9JS1FKUU1vZWZYUXpPWVV3UGtQTWhUTTF3a2pGaVRNbnhr?=
 =?utf-8?B?dkpzdTYzVW4yVTl2S1dxS2QyMU5sa0JPeGM0YW5iQ0FhYTllTFZRczB1Y2ZT?=
 =?utf-8?B?d0pTMjArS2dXSHdSa2hVbGlabDNMN0NUbVNQUWdQYlQ5MUhSbE5GTTlaYWpV?=
 =?utf-8?B?TnVoZHdxQ2oweVpUenJSVWJDTE9PT0Znck82MzBPVnRUMDdKSnEva0htNFMz?=
 =?utf-8?B?WDZZNXFVUFAvbElmRklsenVVem5pSk82RzRKQ0xhdVhGUnZEdDNPV1NXaUZK?=
 =?utf-8?B?czlQMjlYc0RTamxTWDgwTW5zRkE3NWE1RkQ5SFRHVEpPMGkwZmJNVW1LTXdL?=
 =?utf-8?B?bnlTYmZwTE96eDFIWW1QWkhiYkIxNEE5YjBQMDQvUHhRUnNOWlVlblV2WFpG?=
 =?utf-8?B?cTZ2anhyUXlKK2hqaExBZitQNnhRRXhMcDJsYS9PY1hZbURnUTFhQSs0UW1p?=
 =?utf-8?B?YlRNTHVyR2JwdUhrcU5pQUErWmVhbDJ0NmhqS3R4Z2hVMzdSeXBQWGZObGNB?=
 =?utf-8?B?eGlrczFwUmd0TzZzdEdCT00rQUdUNmJKRS9SellWMloraDBYWm12R05EdzBj?=
 =?utf-8?B?amVnMTdyc3NiYVFzQm54QWtkUXRUd3BqMVkxaVJJUis2cTNQNDdLL21KUkN4?=
 =?utf-8?B?d01jMGxyOFJMNEVkRWhwZ1p2WEVNVEVaQjFiWnUxSGIvUmlhd25IOS9vd1pw?=
 =?utf-8?B?YW5MSUpDWkV3UEl6aUVtVlhYK3FldHZqWlBRQ3BvL01JeHpXd1NOZ0pnbkh6?=
 =?utf-8?B?ZncrQTFESzRTVE9ZN3FvSGZPelVESTJHS3EwSzd3dGVDa2plMFZjcEd5a0Zr?=
 =?utf-8?B?emtTQjREQUVBTHBlNlFJQzQ3NE4vMk1zbjdKQUhZQmZmc1BhbE1qVXFUdzBJ?=
 =?utf-8?B?dGgvVTJHOVhzK2ptYzVuSmJseXhlQmE4aWgweGorcEVxbDNGZDZiRjI5TFVr?=
 =?utf-8?B?WklXUHFpVmtyOU0yT3I5dVNpbU1IT2FnNTZyQ1ZFc2h6bjYzalZET0t4bysx?=
 =?utf-8?B?ZzZIaDJOS1hGUFQ5S0JWRitqeGpYVkV3elJmd0R1UFVkTHFXb1ZRcGV1YUl3?=
 =?utf-8?B?ajljdE5xbmZ2Z2hYNS9HUGwya05VeCtQQ2xYVHRsU0cyNGNsVXNmbUF6SG1J?=
 =?utf-8?B?dEtsMmVoRmR2am1ROGd1dUc3SzVONXBRb0hvYzZyTjdDZ1pYMDhZU2YwcDZj?=
 =?utf-8?B?YTFQQnBDdVZoWXJFcEx6a1NLTkJrbUJCS0xTN2ZyNVRNQlovNDN2cFhDSG9y?=
 =?utf-8?B?eHB6Y1ZKOVd1Q1RZTEF2blliM2pBV0d2T2JEc3NIZC9YWitVT0g1OXkxcnF1?=
 =?utf-8?B?VUwrSTN5ejFjRi9TUEVxR2tBQ29tMmZuSXJPZjFybm5ieUFjUVRhK0tOZFJM?=
 =?utf-8?B?Y1ZWU0FFNVJ0emhLb285WGhOUFg3YlVtZHRSY0dNdjF0aUZ1bjFwcWxhQXFY?=
 =?utf-8?B?dThSL0E4NFVLMGNsYkZqeVJHZ1M2b2lVY0xwbGxWeUNrM3RpUEFGbndYaGZC?=
 =?utf-8?B?UXJYRm8zemdPRkZOODNsUlNaUGRyaHppa0x2M0drUkgvVVBZSXk0aVJrTXM1?=
 =?utf-8?B?bS8yQXR4RklLY29yOUJsVVh6bWt6QzQ0N3pDaWFzSUxRVG51elFZOWsrN09j?=
 =?utf-8?Q?9FpxGKVZa91HeEzYSgN3PCUw2lpML5ViWNKy9gk?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 934a5ca5-b9d9-4f09-91b7-08d978e4a265
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:36:02.1599
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: S0g/xCHiLImBs3pRqGnnmKqROWatLXALiQsjzehAYd6w5eV82m+p3mpl5aeOz/VFwJM3bzwdRwFksPircFn5vw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2338
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFR1ZXNkYXks
IEF1Z3VzdCAyNCwgMjAyMSAxMDoyNyBQTQ0KPiANCj4gQmVzaWRlcyB0aGUgYWRkcmVzc2VzIHRo
aXMgaXMgdGhlIG5leHQgY3J1Y2lhbCBiaXQgb2YgaW5mb3JtYXRpb24gb25lDQo+IG1pZ2h0IGJl
IGFmdGVyLg0KPiANCj4gU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2Uu
Y29tPg0KDQpSZXZpZXdlZC1ieTogS2V2aW4gVGlhbiA8a2V2aW4udGlhbkBpbnRlbC5jb20+DQoN
Cj4gDQo+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jDQo+ICsrKyBi
L3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jDQo+IEBAIC0yODIyLDEwICsyODIy
LDEyIEBAIHN0YXRpYyB2b2lkIHZ0ZF9kdW1wX3BhZ2VfdGFibGVfbGV2ZWwocGENCj4gICAgICAg
ICAgICAgIHZ0ZF9kdW1wX3BhZ2VfdGFibGVfbGV2ZWwoZG1hX3B0ZV9hZGRyKCpwdGUpLCBuZXh0
X2xldmVsLA0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhZGRyZXNz
LCBpbmRlbnQgKyAxKTsNCj4gICAgICAgICAgZWxzZQ0KPiAtICAgICAgICAgICAgcHJpbnRrKCIl
KnNkZm46ICUwOGx4IG1mbjogJTA4bHhcbiIsDQo+ICsgICAgICAgICAgICBwcmludGsoIiUqc2Rm
bjogJTA4bHggbWZuOiAlMDhseCAlYyVjXG4iLA0KPiAgICAgICAgICAgICAgICAgICAgIGluZGVu
dCwgIiIsDQo+ICAgICAgICAgICAgICAgICAgICAgKHVuc2lnbmVkIGxvbmcpKGFkZHJlc3MgPj4g
UEFHRV9TSElGVF80SyksDQo+IC0gICAgICAgICAgICAgICAgICAgKHVuc2lnbmVkIGxvbmcpKGRt
YV9wdGVfYWRkcigqcHRlKSA+PiBQQUdFX1NISUZUXzRLKSk7DQo+ICsgICAgICAgICAgICAgICAg
ICAgKHVuc2lnbmVkIGxvbmcpKGRtYV9wdGVfYWRkcigqcHRlKSA+PiBQQUdFX1NISUZUXzRLKSwN
Cj4gKyAgICAgICAgICAgICAgICAgICBkbWFfcHRlX3JlYWQoKnB0ZSkgPyAncicgOiAnLScsDQo+
ICsgICAgICAgICAgICAgICAgICAgZG1hX3B0ZV93cml0ZSgqcHRlKSA/ICd3JyA6ICctJyk7DQo+
ICAgICAgfQ0KPiANCj4gICAgICB1bm1hcF92dGRfZG9tYWluX3BhZ2UocHRfdmFkZHIpOw0KPiAt
LS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvdXRpbHMuYw0KPiArKysgYi94ZW4vZHJp
dmVycy9wYXNzdGhyb3VnaC92dGQvdXRpbHMuYw0KPiBAQCAtMTU5LDE0ICsxNTksMTEgQEAgdm9p
ZCBwcmludF92dGRfZW50cmllcyhzdHJ1Y3QgdnRkX2lvbW11DQo+ICAgICAgICAgIGxfaW5kZXgg
PSBnZXRfbGV2ZWxfaW5kZXgoZ21mbiwgbGV2ZWwpOw0KPiAgICAgICAgICBwdGUudmFsID0gbFts
X2luZGV4XTsNCj4gICAgICAgICAgdW5tYXBfdnRkX2RvbWFpbl9wYWdlKGwpOw0KPiAtICAgICAg
ICBwcmludGsoIiAgICBsJXVbJTAzeF0gPSAlIlBSSXg2NCJcbiIsIGxldmVsLCBsX2luZGV4LCBw
dGUudmFsKTsNCj4gKyAgICAgICAgcHJpbnRrKCIgICAgbCV1WyUwM3hdID0gJSJQUkl4NjQiICVj
JWNcbiIsIGxldmVsLCBsX2luZGV4LCBwdGUudmFsLA0KPiArICAgICAgICAgICAgICAgZG1hX3B0
ZV9yZWFkKHB0ZSkgPyAncicgOiAnLScsDQo+ICsgICAgICAgICAgICAgICBkbWFfcHRlX3dyaXRl
KHB0ZSkgPyAndycgOiAnLScpOw0KPiANCj4gLSAgICAgICAgaWYgKCAhZG1hX3B0ZV9wcmVzZW50
KHB0ZSkgKQ0KPiAtICAgICAgICB7DQo+IC0gICAgICAgICAgICBwcmludGsoIiAgICBsJXVbJTAz
eF0gbm90IHByZXNlbnRcbiIsIGxldmVsLCBsX2luZGV4KTsNCj4gLSAgICAgICAgICAgIGJyZWFr
Ow0KPiAtICAgICAgICB9DQo+IC0gICAgICAgIGlmICggZG1hX3B0ZV9zdXBlcnBhZ2UocHRlKSAp
DQo+ICsgICAgICAgIGlmICggIWRtYV9wdGVfcHJlc2VudChwdGUpIHx8IGRtYV9wdGVfc3VwZXJw
YWdlKHB0ZSkgKQ0KPiAgICAgICAgICAgICAgYnJlYWs7DQo+ICAgICAgICAgIHZhbCA9IGRtYV9w
dGVfYWRkcihwdGUpOw0KPiAgICAgIH0gd2hpbGUgKCAtLWxldmVsICk7DQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:38:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:38:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188175.337253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlye-0003CT-TT; Thu, 16 Sep 2021 07:38:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188175.337253; Thu, 16 Sep 2021 07:38:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQlye-0003CM-Q3; Thu, 16 Sep 2021 07:38:36 +0000
Received: by outflank-mailman (input) for mailman id 188175;
 Thu, 16 Sep 2021 07:38:35 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQlyd-0003CG-7b
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:38:35 +0000
Received: from mga18.intel.com (unknown [134.134.136.126])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 17d4786c-16c1-11ec-b5af-12813bfff9fa;
 Thu, 16 Sep 2021 07:38:33 +0000 (UTC)
Received: from fmsmga008.fm.intel.com ([10.253.24.58])
 by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:38:32 -0700
Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84])
 by fmsmga008.fm.intel.com with ESMTP; 16 Sep 2021 00:38:31 -0700
Received: from fmsmsx605.amr.corp.intel.com (10.18.126.85) by
 fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:38:30 -0700
Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by
 fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:38:30 -0700
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168)
 by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:38:30 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR11MB1266.namprd11.prod.outlook.com (2603:10b6:404:49::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep
 2021 07:38:23 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:38:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17d4786c-16c1-11ec-b5af-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="209598006"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="209598006"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="509214307"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=grsD2ALQXPqTmpjouccUoQBSedObcShFH3LMRW9YV1PfiRrlGBLBOTutehCNctgb0izCtAFPU08MSQF9thm8eaRQWa/D8VBkP0de79demEBOANvcb6lH6wGcspsOPMb9X/q/UdReEGUxMtJR/xMZesduFXbYPGRatPzAJsHopW7v7g1Icm5uQt0Ed5VyYml0yNN8EpUs0EeBuBOARfBkHX7h3zRQUF09e77NPUtL81z6S3GVF7gykBrbml8vs7xIEmusWpa9jEeMAhgRaRmxA4SskyZFYfcFFRjE5O8u9zctipxQu2MG4H/S7Oml6Ts34hmejBr46gXj0ak2oSC5lA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+B+PeHhMAxaElaXnsp1gPzHe/Is6euh8kKAI3si2YAo=;
 b=GMg5OGgNA5ldoiPW0i0SmBIVsx0a2lrBOyJLrMw8y2qApwzTjd4QPLNR68Cftg3N5sS9PrsGWBANPjw7oySLZuq0M4jlYdQ3AsVQlZxZno97T33VLUCXR3ezBgbsaV0QlHGi5i7+fucntyCmbQyX/Gvbzuz+0Jlz84HPJxfSoSFiFspAcbcxTWdsNNrWe7YenSSGm3ut8poWKlDTvRyW9qZ58Mji1qF/JFqLhrHXJt1djAEoqFlMiKNhYDOKjj9+HMIa8U7jEMysXoa5y2+/1X/BCe6HmXnvghTRb+37lsqkXMOiQhfUUaoerz5kswtkZ4mogY7aWatEk1xAxG5lxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+B+PeHhMAxaElaXnsp1gPzHe/Is6euh8kKAI3si2YAo=;
 b=t906+oP9cOdU4VLMpxDA0+gQkL0/LBd3vALfbEEmaAL/KIGpf8bItLi9fHNwZVlmxkDRkrCM0l8DeF3EU3b24Dyrm7OtcZiOKj0SmezThmZM9WoJ7nfSdrziUFZe2lbHQbgoAvI7+R2bMhJRXryNCwpRm+ZLlN/zxeMlRN1tf68=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>
Subject: RE: [PATCH 04/17] IOMMU: have vendor code announce supported page
 sizes
Thread-Topic: [PATCH 04/17] IOMMU: have vendor code announce supported page
 sizes
Thread-Index: AQHXmPLSEQjLv6P2ME2fmo8TXLbyOqumaasQ
Date: Thu, 16 Sep 2021 07:38:23 +0000
Message-ID: <BN9PR11MB5433D5916CEEAC11E5EC31F18CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <53c5e1b5-b363-d2c0-edbc-f9fae1e6e2c1@suse.com>
In-Reply-To: <53c5e1b5-b363-d2c0-edbc-f9fae1e6e2c1@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 653ae5ef-f64c-40a1-d5b5-08d978e4f6bc
x-ms-traffictypediagnostic: BN6PR11MB1266:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN6PR11MB126693B5CAD41498B9DBB0888CDC9@BN6PR11MB1266.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6108;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: tl3Gtwelk64/b+9clDKhWCNQICH8aiIkdMxng5uJx4XL4eK13exuRvz7iyOKC/wDJx/vyidCqewedcg3+YHcsHt3GJB+PWW9ZHgkdq/L11IfB7eiuPQSUSwpiQ90BPhhZ3jCqOhqEK9+vMUxecNLUnY8cI7FTEnBpQ8ndTRg3Mq6TpobZMFBzRn1KOvYS6n400Kv0NpLwl7ltintaSYHtnAfm6Z/I6ZNw/sie7jG8THlXz/qnl2MM5sRf6R7SrjMjLm7Uq/65vFNP5BOBGfY+k/hL6M/7A/bcKwOqT2W+K8yy/F8URA+hnMjpykyyQjvmXLg5SEqj/YRcPToTMXG0lIa9VTFZpO3jJcvinpaS4gVVp3muv/CO1olAfkot3yiyCLmOjQzVx+GzPvGRFZ4RI/846V4WXbcWdBUyqDKEC0um8z9pNRx4BcC0x1SbeuaDpQ0ovZ1Oxng6OheWvt/XxzMDchzCniKNjzsuxezg8TaH0Pb/doxN3HC9R7iHVL2XQ+mC4lU0mPEUNb4QPLFcZzhv+fWvUKmm6ZriB6+BWDclAD6VneC5T7nyv6YruzB9uu7wF2ynTi4XXsQbplrSvP5oFgWYxOhY4i8KuBNbIt0RuMkpCSgk3SVZvW5LOub2S9pYB6V3+Ck5UHMXslPo+CaH1VhbZ/FhlWpCEAT1mjH3+qhngF8Rm9occcKNAbENaUOAFffw7pV84n5FNiE7A==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(376002)(39860400002)(346002)(136003)(86362001)(38070700005)(83380400001)(5660300002)(54906003)(33656002)(8676002)(110136005)(38100700002)(55016002)(9686003)(26005)(7696005)(8936002)(316002)(6506007)(2906002)(71200400001)(4326008)(66946007)(122000001)(478600001)(52536014)(66446008)(64756008)(66556008)(66476007)(76116006)(186003);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?OHhNV2lqMUpWSU9Pa0pvalRQMjhvQjR2M0lvem5BTmk0Y3JhTEVOUGUra1pm?=
 =?utf-8?B?dDFrRXAxTEtiZG94SmloV2xQb1YrdTBJZVVYZXY0WmxaQml0UnFJRUN6ZDVM?=
 =?utf-8?B?aTRKUVQ2dWJlMWVXSnpCSzdMRkxOQmdqb0h5ZW4yaHNMdE9kdG1NTVZIWEFy?=
 =?utf-8?B?d1JkMlJJdHg4S3J2NjVuSXRYSEJSWlIrWFczc3VMWEgxcUsxQmQ5bzdXaTlT?=
 =?utf-8?B?aVVsd2ZqL080UGRLdU80a0ZxRXhYYTVXYlA4NW9pd29hUlBXMVJlRGozVk9E?=
 =?utf-8?B?b01ja2JXQ0kwbUI2VnZOYmVUUThYRFRLb1E5RjRDWlh6cXNYVjM1UkZDaWta?=
 =?utf-8?B?OWY1cTU5TVpFeUtjdjVSS2hvUjRRdXkydVc4ck5VZDRIVjNPbnUzcERBeUZy?=
 =?utf-8?B?RnV5Q09YMU1FdjBVUFljbHRhUVY0QmRDWExKWVJ2NE4wczFWb1RxNWVLT3Rh?=
 =?utf-8?B?allVTW1GNE1rb1lQd09WZGVtOTBPTjJYcmIraUdvTmFrTnFDeVo5UlNTY3FI?=
 =?utf-8?B?dlRwZm9sSlQ2RGxDL0VHR25mZGNBTG45Zy8wVDQ1enVMV2xINXVNOGNwbExk?=
 =?utf-8?B?NUpSdjJjeEp4b3YxdThKd1l1L3JZRUlnODZvT2t2S21vVVJDQTJQY2ZrUGoz?=
 =?utf-8?B?NWVnU0FWbVY4YjFnWGZOZUZYeklxTjhMMzNKSG5RQXpkamdTclBxY1g4WVNO?=
 =?utf-8?B?ZXlVMHJFTXI5S015VjZDOE9kL3c1ZXhMRXlPdTIway8yMy80NU96K2QxUzdJ?=
 =?utf-8?B?Mk11MmFsWGU5bXRSeUNyUzVuYzhLa3dpMjBqVGtKVWVBQzZBODJ2L2tQLytT?=
 =?utf-8?B?K0tZZDRLQ1h2M1AweDlmQmVWZkM0N1gvVktMNzIxNEZ5Q1ZPcll1YjRiODAx?=
 =?utf-8?B?Y2VPSmdMS1JIcmNnTERWK2NqdUgyRFJLMEVGUVFUZHo1QTVQZWFKNDIyblR5?=
 =?utf-8?B?SkNtRmJ4WUZlY2ZxOG9TdXdSUWF0M05KZVREMjdLang0Z1dpaDFRaHJIY1M0?=
 =?utf-8?B?a1ZXdmRMcjYza2J6UENaYmhKNHlMbjBLSWRPUVVBQmN5d3ZoQ25xbjZXeXZJ?=
 =?utf-8?B?SFp3R2hxSHQwdHNXRGpac0Fla2NEcEl3cCtpSmdKSDUrUW5PdGFQNm9EVVpo?=
 =?utf-8?B?c0xQeGdQYXMxdGdOdlRzT0N5RlpHYVdyaGFxV0ZVdUZZaFc1ZGc2WVdQcTVX?=
 =?utf-8?B?SWhEUURzNEtvN1BkUE4veXlOQlZGdzBSdWRPZDNIMi8rVXQ4TlRwOUZmUW9l?=
 =?utf-8?B?bU95bi81U215QW9raGYzajdjSnYxVGFHWUh6UWNOVVBpZjZ0MXJ3eVNOQ01p?=
 =?utf-8?B?eFF1MVJaQWJnWSswNnBTR3oreE1XVm51UC8rRFFxRnlKMWgxT2dIb0F4VVlB?=
 =?utf-8?B?WXZrOXZvbmZXQmx3OWJibElGT040TDBaZDJFRE1pSVJXWTIvbHdqTzBTVjhm?=
 =?utf-8?B?MkFKdng5bi9Gam9teEphSFBIbEZGcVd5SkliNFRPblgyTWRxczJzSFppeUo5?=
 =?utf-8?B?RVNoWkRxaFFJUWcwRktUdXRiMGlzWjYwVng1QUFDOXNpUUt6NGVnUUJvVi9x?=
 =?utf-8?B?aFJhSHBSUVhLWk8vdVppODd2YVFMWnVaTTJrK0srcFdOMVp3TTBOaHRwOUJL?=
 =?utf-8?B?VURTRXRhakRiRkRVL3RHdmhRN1VzdnRHL1JReS85cU9VR3Q2ZWNiZXJnclk5?=
 =?utf-8?B?QmlBUGx6Rm1OUDVlalFNR0pPakw2cGpFZXBBVGZySkI2d1B4QUdiWHMrbWUz?=
 =?utf-8?Q?bg2z1wUAY69ZxMkTtYaVvSRx07iq3UjOr7Y9kCO?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 653ae5ef-f64c-40a1-d5b5-08d978e4f6bc
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:38:23.5858
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +6g7QLCEtSmRprkISlxyy/vL2u5TEcpOoHrwUY8vAfDVszM+/3B2RkuHJUlMFJMILMTZJf0QRmj1n2BvJxrVqA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1266
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFR1ZXNkYXks
IEF1Z3VzdCAyNCwgMjAyMSAxMDoxOCBQTQ0KPiANCj4gR2VuZXJpYyBjb2RlIHdpbGwgdXNlIHRo
aXMgaW5mb3JtYXRpb24gdG8gZGV0ZXJtaW5lIHdoYXQgb3JkZXIgdmFsdWVzDQo+IGNhbiBsZWdp
dGltYXRlbHkgYmUgcGFzc2VkIHRvIHRoZSAtPnssdW59bWFwX3BhZ2UoKSBob29rcy4gRm9yIG5v
dyBhbGwNCj4gb3BzIHN0cnVjdHVyZXMgc2ltcGx5IGdldCB0byBhbm5vdW5jZSA0ayBtYXBwaW5n
cyAoYXMgYmFzZSBwYWdlIHNpemUpLA0KPiBhbmQgdGhlcmUgaXMgKGFuZCBhbHdheXMgaGFzIGJl
ZW4pIGFuIGFzc3VtcHRpb24gdGhhdCB0aGlzIG1hdGNoZXMgdGhlDQo+IENQVSdzIE1NVSBiYXNl
IHBhZ2Ugc2l6ZSAoZXZlbnR1YWxseSB3ZSB3aWxsIHdhbnQgdG8gcGVybWl0IElPTU1VcyB3aXRo
DQo+IGEgYmFzZSBwYWdlIHNpemUgc21hbGxlciB0aGFuIHRoZSBDUFUgTU1VJ3MpLg0KPiANCj4g
U2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KDQpSZXZpZXdl
ZC1ieTogS2V2aW4gVGlhbiA8a2V2aW4udGlhbkBpbnRlbC5jb20+DQoNCj4gDQo+IC0tLSBhL3hl
bi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9wY2lfYW1kX2lvbW11LmMNCj4gKysrIGIveGVuL2Ry
aXZlcnMvcGFzc3Rocm91Z2gvYW1kL3BjaV9hbWRfaW9tbXUuYw0KPiBAQCAtNTgzLDYgKzU4Myw3
IEBAIHN0YXRpYyB2b2lkIGFtZF9kdW1wX3BhZ2VfdGFibGVzKHN0cnVjdA0KPiAgfQ0KPiANCj4g
IHN0YXRpYyBjb25zdCBzdHJ1Y3QgaW9tbXVfb3BzIF9faW5pdGNvbnN0cmVsIF9pb21tdV9vcHMg
PSB7DQo+ICsgICAgLnBhZ2Vfc2l6ZXMgPSBQQUdFX1NJWkVfNEssDQo+ICAgICAgLmluaXQgPSBh
bWRfaW9tbXVfZG9tYWluX2luaXQsDQo+ICAgICAgLmh3ZG9tX2luaXQgPSBhbWRfaW9tbXVfaHdk
b21faW5pdCwNCj4gICAgICAucXVhcmFudGluZV9pbml0ID0gYW1kX2lvbW11X3F1YXJhbnRpbmVf
aW5pdCwNCj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvYXJtL2lwbW11LXZtc2EuYw0K
PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaXBtbXUtdm1zYS5jDQo+IEBAIC0x
Mjk4LDYgKzEyOTgsNyBAQCBzdGF0aWMgdm9pZCBpcG1tdV9pb21tdV9kb21haW5fdGVhcmRvd24o
DQo+IA0KPiAgc3RhdGljIGNvbnN0IHN0cnVjdCBpb21tdV9vcHMgaXBtbXVfaW9tbXVfb3BzID0N
Cj4gIHsNCj4gKyAgICAucGFnZV9zaXplcyAgICAgID0gUEFHRV9TSVpFXzRLLA0KPiAgICAgIC5p
bml0ICAgICAgICAgICAgPSBpcG1tdV9pb21tdV9kb21haW5faW5pdCwNCj4gICAgICAuaHdkb21f
aW5pdCAgICAgID0gaXBtbXVfaW9tbXVfaHdkb21faW5pdCwNCj4gICAgICAudGVhcmRvd24gICAg
ICAgID0gaXBtbXVfaW9tbXVfZG9tYWluX3RlYXJkb3duLA0KPiAtLS0gYS94ZW4vZHJpdmVycy9w
YXNzdGhyb3VnaC9hcm0vc21tdS5jDQo+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2Fy
bS9zbW11LmMNCj4gQEAgLTI4NzMsNiArMjg3Myw3IEBAIHN0YXRpYyB2b2lkIGFybV9zbW11X2lv
bW11X2RvbWFpbl90ZWFyZG8NCj4gIH0NCj4gDQo+ICBzdGF0aWMgY29uc3Qgc3RydWN0IGlvbW11
X29wcyBhcm1fc21tdV9pb21tdV9vcHMgPSB7DQo+ICsgICAgLnBhZ2Vfc2l6ZXMgPSBQQUdFX1NJ
WkVfNEssDQo+ICAgICAgLmluaXQgPSBhcm1fc21tdV9pb21tdV9kb21haW5faW5pdCwNCj4gICAg
ICAuaHdkb21faW5pdCA9IGFybV9zbW11X2lvbW11X2h3ZG9tX2luaXQsDQo+ICAgICAgLmFkZF9k
ZXZpY2UgPSBhcm1fc21tdV9kdF9hZGRfZGV2aWNlX2dlbmVyaWMsDQo+IC0tLSBhL3hlbi9kcml2
ZXJzL3Bhc3N0aHJvdWdoL2FybS9zbW11LXYzLmMNCj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Ro
cm91Z2gvYXJtL3NtbXUtdjMuYw0KPiBAQCAtMzQyNiw3ICszNDI2LDggQEAgc3RhdGljIHZvaWQg
YXJtX3NtbXVfaW9tbXVfeGVuX2RvbWFpbl90ZQ0KPiAgfQ0KPiANCj4gIHN0YXRpYyBjb25zdCBz
dHJ1Y3QgaW9tbXVfb3BzIGFybV9zbW11X2lvbW11X29wcyA9IHsNCj4gLQkuaW5pdAkJPSBhcm1f
c21tdV9pb21tdV94ZW5fZG9tYWluX2luaXQsDQo+ICsJLnBhZ2Vfc2l6ZXMJCT0gUEFHRV9TSVpF
XzRLLA0KPiArCS5pbml0CQkJPSBhcm1fc21tdV9pb21tdV94ZW5fZG9tYWluX2luaXQsDQo+ICAJ
Lmh3ZG9tX2luaXQJCT0gYXJtX3NtbXVfaW9tbXVfaHdkb21faW5pdCwNCj4gIAkudGVhcmRvd24J
CT0NCj4gYXJtX3NtbXVfaW9tbXVfeGVuX2RvbWFpbl90ZWFyZG93biwNCj4gIAkuaW90bGJfZmx1
c2gJCT0gYXJtX3NtbXVfaW90bGJfZmx1c2gsDQo+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJv
dWdoL2lvbW11LmMNCj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvaW9tbXUuYw0KPiBA
QCAtNDcwLDcgKzQ3MCwxNyBAQCBpbnQgX19pbml0IGlvbW11X3NldHVwKHZvaWQpDQo+IA0KPiAg
ICAgIGlmICggaW9tbXVfZW5hYmxlICkNCj4gICAgICB7DQo+ICsgICAgICAgIGNvbnN0IHN0cnVj
dCBpb21tdV9vcHMgKm9wcyA9IE5VTEw7DQo+ICsNCj4gICAgICAgICAgcmMgPSBpb21tdV9oYXJk
d2FyZV9zZXR1cCgpOw0KPiArICAgICAgICBpZiAoICFyYyApDQo+ICsgICAgICAgICAgICBvcHMg
PSBpb21tdV9nZXRfb3BzKCk7DQo+ICsgICAgICAgIGlmICggb3BzICYmIChvcHMtPnBhZ2Vfc2l6
ZXMgJiAtb3BzLT5wYWdlX3NpemVzKSAhPSBQQUdFX1NJWkUgKQ0KPiArICAgICAgICB7DQo+ICsg
ICAgICAgICAgICBwcmludGsoWEVOTE9HX0VSUiAiSU9NTVU6IHBhZ2Ugc2l6ZSBtYXNrICVseCB1
bnN1cHBvcnRlZFxuIiwNCj4gKyAgICAgICAgICAgICAgICAgICBvcHMtPnBhZ2Vfc2l6ZXMpOw0K
PiArICAgICAgICAgICAgcmMgPSBvcHMtPnBhZ2Vfc2l6ZXMgPyAtRVBFUk0gOiAtRU5PREFUQTsN
Cj4gKyAgICAgICAgfQ0KPiAgICAgICAgICBpb21tdV9lbmFibGVkID0gKHJjID09IDApOw0KPiAg
ICAgIH0NCj4gDQo+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jDQo+
ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Z0ZC9pb21tdS5jDQo+IEBAIC0yODc1LDYg
KzI4NzUsNyBAQCBzdGF0aWMgaW50IF9faW5pdCBpbnRlbF9pb21tdV9xdWFyYW50aW5lDQo+ICB9
DQo+IA0KPiAgc3RhdGljIHN0cnVjdCBpb21tdV9vcHMgX19pbml0ZGF0YSB2dGRfb3BzID0gew0K
PiArICAgIC5wYWdlX3NpemVzID0gUEFHRV9TSVpFXzRLLA0KPiAgICAgIC5pbml0ID0gaW50ZWxf
aW9tbXVfZG9tYWluX2luaXQsDQo+ICAgICAgLmh3ZG9tX2luaXQgPSBpbnRlbF9pb21tdV9od2Rv
bV9pbml0LA0KPiAgICAgIC5xdWFyYW50aW5lX2luaXQgPSBpbnRlbF9pb21tdV9xdWFyYW50aW5l
X2luaXQsDQo+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9pb21tdS5oDQo+ICsrKyBiL3hlbi9pbmNs
dWRlL3hlbi9pb21tdS5oDQo+IEBAIC0yMzEsNiArMjMxLDcgQEAgc3RydWN0IHBhZ2VfaW5mbzsN
Cj4gIHR5cGVkZWYgaW50IGlvbW11X2dyZG1fdCh4ZW5fcGZuX3Qgc3RhcnQsIHhlbl91bG9uZ190
IG5yLCB1MzIgaWQsIHZvaWQNCj4gKmN0eHQpOw0KPiANCj4gIHN0cnVjdCBpb21tdV9vcHMgew0K
PiArICAgIHVuc2lnbmVkIGxvbmcgcGFnZV9zaXplczsNCj4gICAgICBpbnQgKCppbml0KShzdHJ1
Y3QgZG9tYWluICpkKTsNCj4gICAgICB2b2lkICgqaHdkb21faW5pdCkoc3RydWN0IGRvbWFpbiAq
ZCk7DQo+ICAgICAgaW50ICgqcXVhcmFudGluZV9pbml0KShzdHJ1Y3QgZG9tYWluICpkKTsNCg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:42:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:42:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188183.337264 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQm1t-0004aF-Ck; Thu, 16 Sep 2021 07:41:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188183.337264; Thu, 16 Sep 2021 07:41:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQm1t-0004a8-9Y; Thu, 16 Sep 2021 07:41:57 +0000
Received: by outflank-mailman (input) for mailman id 188183;
 Thu, 16 Sep 2021 07:41:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQm1s-0004a2-Cx
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:41:56 +0000
Received: from mga11.intel.com (unknown [192.55.52.93])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e09f6b9f-ade9-43cb-9c5a-5693cc740b39;
 Thu, 16 Sep 2021 07:41:53 +0000 (UTC)
Received: from orsmga005.jf.intel.com ([10.7.209.41])
 by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:41:32 -0700
Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83])
 by orsmga005.jf.intel.com with ESMTP; 16 Sep 2021 00:41:32 -0700
Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by
 fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:41:32 -0700
Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by
 fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:41:31 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
 fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:41:31 -0700
Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.175)
 by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:41:31 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN8PR11MB3761.namprd11.prod.outlook.com (2603:10b6:408:85::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 07:41:30 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:41:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e09f6b9f-ade9-43cb-9c5a-5693cc740b39
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="219322339"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="219322339"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="651471933"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dsUYmrZ51q0rq5yHv0cnBlqPOUChdB6ZQrpUVjsYjJK0PoLeCBM6sGOa+h8/zIztWmwDVJlvxR3bRLRGYMOcJtOQLR3td6pmESJrw4F9COC9Cwvjq1oul1GHNiQUfsvVby2h6/B4gurdlm76BYvZtRokRWwy/TZh8z3AbFFmiGhwIEQ2kUDsbvuP3LWhho/NuL4/NyHYkxet/Z3B4hNSwymxTmjFg9Rak1PC38NXWIvNTZSYQnef/iSSxktW7ysxWPCoPAjnomqzIY0aRBLzwFzzsqUC7n1P5lSsLrzSmvioQ01he7wtapeSFILuh0DnTfdIuE0q88/mai+ORJKTMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=c63MBIo3cLyHu1UTNeKivtlwAo2vTocRwbfVZ3sosrs=;
 b=e3P35Z3AuQfBWyw2N0y2I5ItOo2sU75Vrd9GPQDWUGCQReRclsyhEX8MWdfRiV/yFg19s6912WYjLV3HHBghbtxBZe66R6z33jvW1KAenTwIagglpukj8GDokkbumunkZCFtY5BfK/WKOyusEMo7A0pvn54eugs1Fx78Vjzi/SAQklJAKjDNoP2HTcmbHAdL0z30QeBHyQbbj4Yex/+z9eVwVgAwfQTehef7pp4jr8u0i01oAuiqouDPdDN6g95ByIWR3Uu4kWSBxre+PS4wLR40uLm2BbSGxHSE5rNhlxbvND2jjO0LjCzKItG/MYl5X2m6QkS32vMQIcogmqvbug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=c63MBIo3cLyHu1UTNeKivtlwAo2vTocRwbfVZ3sosrs=;
 b=uyxwGl5PjSpigJQxNTGAeRIsXVEmpBr4Fm2dOUZ9YgFdxsIFP1L13uMw5L3PP50bh+B5w65vg9LQF+XEdiwlRnvVKFX/av05GdKK8gXiVgCsDQOlEba7LI8rNhvJQmBpjEBjPRlEEHzoOCrpWQ6DcIijuXWFQSl+QAwZEOIqSPU=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Subject: RE: [PATCH 05/17] IOMMU: add order parameter to ->{,un}map_page()
 hooks
Thread-Topic: [PATCH 05/17] IOMMU: add order parameter to ->{,un}map_page()
 hooks
Thread-Index: AQHXmPMIXmb1MBaKjE+gIPEf1eDFcquman2A
Date: Thu, 16 Sep 2021 07:41:30 +0000
Message-ID: <BN9PR11MB54333609C5A10043B142A5C48CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <3141710b-2369-6e12-2ba2-4c01888f7ee1@suse.com>
In-Reply-To: <3141710b-2369-6e12-2ba2-4c01888f7ee1@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9c4eda42-5de3-4c6b-7e2b-08d978e56609
x-ms-traffictypediagnostic: BN8PR11MB3761:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN8PR11MB37617757C8A6CCFE705A1F308CDC9@BN8PR11MB3761.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 5TkfU3MgwLruqnoKHVgF8aFJYJX6J+GqGihhp3yiZcyuWk2+dco1OIye1H40KD8LcvGLYtqwX1AO3NuCglJBEyFznk+gluQDKrzywF/crW9S9q6bhz2NmQ5ZHjMRCVxXbxwZt8L7sK375bwCrUIGxXe83j7FJS5eBTXS/K4k0OU1/W+kwEN0VQC4CQ+VeywbSv0aK+jsN8ILxPmlf/+r2CVssFM5og6lN/2hcQlAwiVSt5iAr9DzrE1PuFB1RFm/o5yRMxDl2VUV+4pfizhw+Gdo6+my/YZLaeOphIDC3Ml9KaXTDzYawYzBgQCNaYlmRPWydtbkR+b98wYd9v7ICS+0lntio4iyg4AdnwVpbZsW0oa8nZyPDLpSLS+mPB+ysjaxqiagLFQDu0lCiPNZmmj2iNmQp+vMxXeYku/1miMjUolCBBNdBFe5GCupFFb3SOqKssLLDcrB5uD8pjy/UAmwEVjwTKY4DX6pYg7TH8CFrrUojH+tgWKdLVSTjPI1zlpN2nPsaqQsWkCd9NBDLWxW1dF9b/FYQEing72EW7YnTyn6Fq9d/ID1myEaIc3LukVikbPGGGErrjQ6otOGguy23BgGVYXYZlhzK4nRxtk4Dewa0y6REWhdiSx8WYha//DE6EcZGdij2cYnOKQh1N/btfGSlXW6P599L9SUmqtdeFyF2K8UrhkiR54xzqxPFpi0SIze6/K4HBsTxcrO2Q==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(346002)(136003)(366004)(376002)(39860400002)(26005)(64756008)(66446008)(66556008)(186003)(71200400001)(6506007)(7696005)(76116006)(66946007)(66476007)(86362001)(5660300002)(52536014)(122000001)(33656002)(9686003)(54906003)(55016002)(316002)(110136005)(38070700005)(8936002)(8676002)(4326008)(38100700002)(478600001)(83380400001)(2906002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?VlNYSXN6bW8ybUVOdGx5T3FER1I2cU1ZWHIzeWVoTjUyNk9rUnNabzFLVGpj?=
 =?utf-8?B?WkRxeHh4OE1MTGNMaFNDOGhnMzNZb0VKQ1dkeGpJYlpwMmh2bndkaWxaSWNN?=
 =?utf-8?B?Q2xNM0RjMWhEVFd5VWszU25rUDVYUDl4UXBUWE1wZ1VVaFoyYVd5SGNpdDdT?=
 =?utf-8?B?VXBQSzB5UnJMdy9LL0kvK0dHMTFqVjFobnE0TDFEY1g1NnZZdVJUYXQwc0R2?=
 =?utf-8?B?ZVUzbStGaWpJUE9SWEh0TnRoSTVVZlllMkVrUWV3M29YQ3FCdWRLdTBCZ3FZ?=
 =?utf-8?B?MXliSnRFOGNKQkZVN2haWk5ET2NGM3NncmhKMzRLRFhEeGdOU3cwb3hpRVZt?=
 =?utf-8?B?US9uall0dkU3cXFML3l1Z25Wc01zb25SZWNFM09icWhRbXBPZHM4Y3RVRFZY?=
 =?utf-8?B?cjZienF2ZWx5NUVwYk1TU3RrMUFwZEc4UG5ZQjdpcVhSLzIrSXNtekY1TnBq?=
 =?utf-8?B?VnN5TzNFMTdkU1BkQ3hzL1grbnQwVlpXb3hwYU1NVWxtbUpiY2FZUitsK1Vi?=
 =?utf-8?B?SlF6eXBJb3hqekdSdTdrZ25KQnJGTkxyeVRuWXVqUkFXOFQvU3R1RGswdXlz?=
 =?utf-8?B?aUhWNkVLYmdGSGRJQnN4V2RTcnJndGlQamtDdjNCODRROGFKOUM4QTd1cFdz?=
 =?utf-8?B?MnpMRHFDM3NTWkxlemxZM2VVSDh0WHFjMDlKeGJpYmZjQ1lta3pML2oxT3dB?=
 =?utf-8?B?SWRxWmFqZkNOYlRnaG9yRUVwbTZQd1VxTHpWMUFPVVlhZjdyYzd5eGRDRjZM?=
 =?utf-8?B?WlJWUXU3T29NZlZuMnJJc0FSK0c4VnZaT2NrQUh0Uko4KzhhSEE5ZUpDK1hM?=
 =?utf-8?B?anhOOXl5SHJJRlQ3U0hYc2V3M05WYlRUN295N0ROQ25lc1RCbnc5M2c3eks2?=
 =?utf-8?B?TEdZdGN0djJLNlY5Vm11N0t4RDVPMmpzc2NrU09renU0bzhDM3E0cXBCWHU2?=
 =?utf-8?B?bGx2ditMZ0h5TWIyeTh2UVRhVkpSYllxQ2dnSmxReHFkaThFUXd5dm5vT1g4?=
 =?utf-8?B?Yk9VTE1oRUluSEQ2QThqQU5KdzkwUkM5YTJES2pBcXAzTm1pQkFpc0VnVGVj?=
 =?utf-8?B?S3pGVDdSdWFoZjNFOUZjV29ZY1FaQ28vaXptZHE1OFpmdGRMRXM5TmIxSWhs?=
 =?utf-8?B?RERsbTZwUkwzeS9vaDlaam40aUt4MHR4ZkZzUXFrallsOEJoM1NEcmoxMnFs?=
 =?utf-8?B?Z3JxM1QvU2E3MXh1VStJeFp6QjRCbEd4VG56TWlWQ2trTnZOMTRKN2JCN21U?=
 =?utf-8?B?cUt5TmRlM3dqWW1ETER6M0NId000ODBxZ3pESEgycWdpclY5OTZmbmpHUUky?=
 =?utf-8?B?b09kVzFVM2lXVVFBejMwT1Q1ZzZEem5sRGcxM1ByVVRpWGd1QjFaY0NZVXEy?=
 =?utf-8?B?VDhxUzdDSE00SGV2ODVVdW9qNHRFUzJuM29rSVBaVFdUTlo5bk1DM2VLZzZ3?=
 =?utf-8?B?ZWJaMlk0SngzWTdxZnVxMTUzVzkrQ3YzT1cwemJpVlZ0bklUcTRNTjF6QVZN?=
 =?utf-8?B?SS96dkZCWFFRTVoyRUVWSTZRTnZoeGRjbFpubzdON2tVRk1GVzdPdkxXc1BD?=
 =?utf-8?B?K2hFMFZNc3RoSDlyTW5wRnZOTjlpdFM2VTY1TEQvdlZQTGpZdExVMGRMYnpv?=
 =?utf-8?B?TllsdnJqY01IZmw3bGN1cXhVL1NDVFhBTnZTYjh5NW1lb21iWnlnaUdwZyt2?=
 =?utf-8?B?OCszUXBhRkFOc1RLV3dKNHh1WGFZTGg4bXd1MUFnUlJ0UGdwUE5rWnlVS013?=
 =?utf-8?Q?uWvlqkJfcKWhdnvoZ1G2t82hBn7I8ZcLP5XFcaT?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c4eda42-5de3-4c6b-7e2b-08d978e56609
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:41:30.3611
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: YYZXcZ7IVrLrAmz4jeQH4zfaCUfvjkP+O0OMkG5jiMVgiOhD9v6Z030GK4+lbamswhM4sRqyQFwsohmqsVnHKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR11MB3761
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFR1ZXNkYXks
IEF1Z3VzdCAyNCwgMjAyMSAxMDoxOSBQTQ0KPiANCj4gT3IgcmVhbGx5LCBpbiB0aGUgY2FzZSBv
ZiAtPm1hcF9wYWdlKCksIGFjY29tbW9kYXRlIGl0IGluIHRoIGV4aXN0aW5nDQo+ICJmbGFncyIg
cGFyYW1ldGVyLiBBbGwgY2FsbCBzaXRlcyB3aWxsIHBhc3MgMCBmb3Igbm93Lg0KPiANCj4gU2ln
bmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KDQpSZXZpZXdlZC1i
eTogS2V2aW4gVGlhbiA8a2V2aW4udGlhbkBpbnRlbC5jb20+DQoNCj4gDQo+IC0tLSBhL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL2FtZC9pb21tdS5oDQo+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0
aHJvdWdoL2FtZC9pb21tdS5oDQo+IEBAIC0yMjUsNiArMjI1LDcgQEAgaW50IF9fbXVzdF9jaGVj
ayBhbWRfaW9tbXVfbWFwX3BhZ2Uoc3RydQ0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgbWZuX3QgbWZuLCB1bnNpZ25lZCBpbnQgZmxhZ3MsDQo+ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgKmZsdXNoX2ZsYWdzKTsNCj4gIGlu
dCBfX211c3RfY2hlY2sgYW1kX2lvbW11X3VubWFwX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwgZGZu
X3QgZGZuLA0KPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25l
ZCBpbnQgb3JkZXIsDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpOw0KPiAgaW50IF9fbXVzdF9jaGVjayBhbWRfaW9tbXVf
YWxsb2Nfcm9vdChzdHJ1Y3QgZG9tYWluICpkKTsNCj4gIGludCBhbWRfaW9tbXVfcmVzZXJ2ZV9k
b21haW5fdW5pdHlfbWFwKHN0cnVjdCBkb21haW4gKmRvbWFpbiwNCj4gLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvYW1kL2lvbW11X21hcC5jDQo+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0
aHJvdWdoL2FtZC9pb21tdV9tYXAuYw0KPiBAQCAtMzI4LDcgKzMyOCw3IEBAIGludCBhbWRfaW9t
bXVfbWFwX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwNCj4gICAgICByZXR1cm4gMDsNCj4gIH0NCj4g
DQo+IC1pbnQgYW1kX2lvbW11X3VubWFwX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwgZGZuX3QgZGZu
LA0KPiAraW50IGFtZF9pb21tdV91bm1hcF9wYWdlKHN0cnVjdCBkb21haW4gKmQsIGRmbl90IGRm
biwgdW5zaWduZWQgaW50DQo+IG9yZGVyLA0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpDQo+ICB7DQo+ICAgICAgdW5zaWduZWQgbG9uZyBwdF9t
Zm4gPSAwOw0KPiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9hcm0vaW9tbXVfaGVscGVy
cy5jDQo+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL2FybS9pb21tdV9oZWxwZXJzLmMN
Cj4gQEAgLTU3LDExICs1NywxMyBAQCBpbnQgX19tdXN0X2NoZWNrIGFybV9pb21tdV9tYXBfcGFn
ZShzdHJ1DQo+ICAgICAgICogVGhlIGZ1bmN0aW9uIGd1ZXN0X3BoeXNtYXBfYWRkX2VudHJ5IHJl
cGxhY2VzIHRoZSBjdXJyZW50IG1hcHBpbmcNCj4gICAgICAgKiBpZiB0aGVyZSBpcyBhbHJlYWR5
IG9uZS4uLg0KPiAgICAgICAqLw0KPiAtICAgIHJldHVybiBndWVzdF9waHlzbWFwX2FkZF9lbnRy
eShkLCBfZ2ZuKGRmbl94KGRmbikpLCBfbWZuKGRmbl94KGRmbikpLA0KPiAwLCB0KTsNCj4gKyAg
ICByZXR1cm4gZ3Vlc3RfcGh5c21hcF9hZGRfZW50cnkoZCwgX2dmbihkZm5feChkZm4pKSwgX21m
bihkZm5feChkZm4pKSwNCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSU9N
TVVGX29yZGVyKGZsYWdzKSwgdCk7DQo+ICB9DQo+IA0KPiAgLyogU2hvdWxkIG9ubHkgYmUgdXNl
ZCBpZiBQMk0gVGFibGUgaXMgc2hhcmVkIGJldHdlZW4gdGhlIENQVSBhbmQgdGhlDQo+IElPTU1V
LiAqLw0KPiAgaW50IF9fbXVzdF9jaGVjayBhcm1faW9tbXVfdW5tYXBfcGFnZShzdHJ1Y3QgZG9t
YWluICpkLCBkZm5fdCBkZm4sDQo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVuc2lnbmVkIGludCBvcmRlciwNCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgdW5zaWduZWQgaW50ICpmbHVzaF9mbGFncykNCj4gIHsNCj4gICAgICAvKg0KPiBA
QCAtNzEsNyArNzMsOCBAQCBpbnQgX19tdXN0X2NoZWNrIGFybV9pb21tdV91bm1hcF9wYWdlKHN0
DQo+ICAgICAgaWYgKCAhaXNfZG9tYWluX2RpcmVjdF9tYXBwZWQoZCkgKQ0KPiAgICAgICAgICBy
ZXR1cm4gLUVJTlZBTDsNCj4gDQo+IC0gICAgcmV0dXJuIGd1ZXN0X3BoeXNtYXBfcmVtb3ZlX3Bh
Z2UoZCwgX2dmbihkZm5feChkZm4pKSwNCj4gX21mbihkZm5feChkZm4pKSwgMCk7DQo+ICsgICAg
cmV0dXJuIGd1ZXN0X3BoeXNtYXBfcmVtb3ZlX3BhZ2UoZCwgX2dmbihkZm5feChkZm4pKSwNCj4g
X21mbihkZm5feChkZm4pKSwNCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBvcmRlcik7DQo+ICB9DQo+IA0KPiAgLyoNCj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91
Z2gvaW9tbXUuYw0KPiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9pb21tdS5jDQo+IEBA
IC0yNzEsNiArMjcxLDggQEAgaW50IGlvbW11X21hcChzdHJ1Y3QgZG9tYWluICpkLCBkZm5fdCBk
Zg0KPiAgICAgIGlmICggIWlzX2lvbW11X2VuYWJsZWQoZCkgKQ0KPiAgICAgICAgICByZXR1cm4g
MDsNCj4gDQo+ICsgICAgQVNTRVJUKCFJT01NVUZfb3JkZXIoZmxhZ3MpKTsNCj4gKw0KPiAgICAg
IGZvciAoIGkgPSAwOyBpIDwgcGFnZV9jb3VudDsgaSsrICkNCj4gICAgICB7DQo+ICAgICAgICAg
IHJjID0gaW9tbXVfY2FsbChoZC0+cGxhdGZvcm1fb3BzLCBtYXBfcGFnZSwgZCwgZGZuX2FkZChk
Zm4sIGkpLA0KPiBAQCAtMjg4LDcgKzI5MCw3IEBAIGludCBpb21tdV9tYXAoc3RydWN0IGRvbWFp
biAqZCwgZGZuX3QgZGYNCj4gICAgICAgICAgd2hpbGUgKCBpLS0gKQ0KPiAgICAgICAgICAgICAg
LyogaWYgc3RhdGVtZW50IHRvIHNhdGlzZnkgX19tdXN0X2NoZWNrICovDQo+ICAgICAgICAgICAg
ICBpZiAoIGlvbW11X2NhbGwoaGQtPnBsYXRmb3JtX29wcywgdW5tYXBfcGFnZSwgZCwgZGZuX2Fk
ZChkZm4sIGkpLA0KPiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZsdXNoX2ZsYWdzKSAp
DQo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgMCwgZmx1c2hfZmxhZ3MpICkNCj4gICAg
ICAgICAgICAgICAgICBjb250aW51ZTsNCj4gDQo+ICAgICAgICAgIGlmICggIWlzX2hhcmR3YXJl
X2RvbWFpbihkKSApDQo+IEBAIC0zMzMsNyArMzM1LDcgQEAgaW50IGlvbW11X3VubWFwKHN0cnVj
dCBkb21haW4gKmQsIGRmbl90DQo+ICAgICAgZm9yICggaSA9IDA7IGkgPCBwYWdlX2NvdW50OyBp
KysgKQ0KPiAgICAgIHsNCj4gICAgICAgICAgaW50IGVyciA9IGlvbW11X2NhbGwoaGQtPnBsYXRm
b3JtX29wcywgdW5tYXBfcGFnZSwgZCwgZGZuX2FkZChkZm4sIGkpLA0KPiAtICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICBmbHVzaF9mbGFncyk7DQo+ICsgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIDAsIGZsdXNoX2ZsYWdzKTsNCj4gDQo+ICAgICAgICAgIGlmICggbGlrZWx5KCFlcnIp
ICkNCj4gICAgICAgICAgICAgIGNvbnRpbnVlOw0KPiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhy
b3VnaC92dGQvaW9tbXUuYw0KPiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9t
bXUuYw0KPiBAQCAtMTkzMiw2ICsxOTMyLDcgQEAgc3RhdGljIGludCBfX211c3RfY2hlY2sgaW50
ZWxfaW9tbXVfbWFwXw0KPiAgfQ0KPiANCj4gIHN0YXRpYyBpbnQgX19tdXN0X2NoZWNrIGludGVs
X2lvbW11X3VubWFwX3BhZ2Uoc3RydWN0IGRvbWFpbiAqZCwgZGZuX3QNCj4gZGZuLA0KPiArICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQg
b3JkZXIsDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpDQo+ICB7DQo+ICAgICAgLyogRG8gbm90aGluZyBp
ZiBWVC1kIHNoYXJlcyBFUFQgcGFnZSB0YWJsZSAqLw0KPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20t
YXJtL2lvbW11LmgNCj4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9pb21tdS5oDQo+IEBAIC0z
MSw2ICszMSw3IEBAIGludCBfX211c3RfY2hlY2sgYXJtX2lvbW11X21hcF9wYWdlKHN0cnUNCj4g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBmbGFncywN
Cj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCAqZmx1
c2hfZmxhZ3MpOw0KPiAgaW50IF9fbXVzdF9jaGVjayBhcm1faW9tbXVfdW5tYXBfcGFnZShzdHJ1
Y3QgZG9tYWluICpkLCBkZm5fdCBkZm4sDQo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGludCBvcmRlciwNCj4gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50ICpmbHVzaF9mbGFncyk7DQo+IA0KPiAgI2VuZGlm
IC8qIF9fQVJDSF9BUk1fSU9NTVVfSF9fICovDQo+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9pb21t
dS5oDQo+ICsrKyBiL3hlbi9pbmNsdWRlL3hlbi9pb21tdS5oDQo+IEBAIC0xMjcsOSArMTI3LDEw
IEBAIHZvaWQgYXJjaF9pb21tdV9od2RvbV9pbml0KHN0cnVjdCBkb21haW4NCj4gICAqIFRoZSBm
b2xsb3dpbmcgZmxhZ3MgYXJlIHBhc3NlZCB0byBtYXAgb3BlcmF0aW9ucyBhbmQgcGFzc2VkIGJ5
IGxvb2t1cA0KPiAgICogb3BlcmF0aW9ucy4NCj4gICAqLw0KPiAtI2RlZmluZSBfSU9NTVVGX3Jl
YWRhYmxlIDANCj4gKyNkZWZpbmUgSU9NTVVGX29yZGVyKG4pICAoKG4pICYgMHgzZikNCj4gKyNk
ZWZpbmUgX0lPTU1VRl9yZWFkYWJsZSA2DQo+ICAjZGVmaW5lIElPTU1VRl9yZWFkYWJsZSAgKDF1
PDxfSU9NTVVGX3JlYWRhYmxlKQ0KPiAtI2RlZmluZSBfSU9NTVVGX3dyaXRhYmxlIDENCj4gKyNk
ZWZpbmUgX0lPTU1VRl93cml0YWJsZSA3DQo+ICAjZGVmaW5lIElPTU1VRl93cml0YWJsZSAgKDF1
PDxfSU9NTVVGX3dyaXRhYmxlKQ0KPiANCj4gIC8qDQo+IEBAIC0yNTUsNiArMjU2LDcgQEAgc3Ry
dWN0IGlvbW11X29wcyB7DQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgZmxhZ3MsDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNp
Z25lZCBpbnQgKmZsdXNoX2ZsYWdzKTsNCj4gICAgICBpbnQgX19tdXN0X2NoZWNrICgqdW5tYXBf
cGFnZSkoc3RydWN0IGRvbWFpbiAqZCwgZGZuX3QgZGZuLA0KPiArICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgb3JkZXIsDQo+ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCAqZmx1c2hfZmxhZ3MpOw0KPiAgICAgIGlu
dCBfX211c3RfY2hlY2sgKCpsb29rdXBfcGFnZSkoc3RydWN0IGRvbWFpbiAqZCwgZGZuX3QgZGZu
LCBtZm5fdA0KPiAqbWZuLA0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgaW50ICpmbGFncyk7DQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 07:47:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 07:47:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188192.337275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQm73-0005T8-4Z; Thu, 16 Sep 2021 07:47:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188192.337275; Thu, 16 Sep 2021 07:47:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQm73-0005T1-0p; Thu, 16 Sep 2021 07:47:17 +0000
Received: by outflank-mailman (input) for mailman id 188192;
 Thu, 16 Sep 2021 07:47:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQm71-0005Sv-NS
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 07:47:15 +0000
Received: from mga01.intel.com (unknown [192.55.52.88])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4e92a364-16c2-11ec-b5b0-12813bfff9fa;
 Thu, 16 Sep 2021 07:47:14 +0000 (UTC)
Received: from orsmga003.jf.intel.com ([10.7.209.27])
 by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 00:47:13 -0700
Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15])
 by orsmga003.jf.intel.com with ESMTP; 16 Sep 2021 00:47:13 -0700
Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by
 ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:47:10 -0700
Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by
 ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 00:47:10 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
 orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 00:47:10 -0700
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 00:47:09 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR11MB1940.namprd11.prod.outlook.com (2603:10b6:404:104::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 07:47:03 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 07:47:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e92a364-16c2-11ec-b5b0-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="244884110"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="244884110"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="434409150"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HHH27T6bs60giYHvD92wYMyXlhFvlmd7RxH/Bwvad8gLLBtMWPg91EqN1VVimb6nHyJiTV3/kjTQqkBr/oUgRRCHh1l8tUGpqWRhOeK5AffMJuHglUF1cM/D7mI/bsfxmMhC2VI0ZjOKwYjkiuDsVLUX2xLeM3toKNlDfTgB14zKnqtlxAXZUl7PS+1ysefqYzRq0kKcOLl89ZbsabGaMXgfGNzXqRXW6pPsnP/U1AQsm2fFLplEghLPocnjfmURdIH4TqaKCQ/wGEd+CDurjV1bOBvfwbj/uWaTD0Rw3bzmlu9Ddqdu/3/9Y32Hw4vvCohJdXx34kyvuU/qzp9o3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vgCAzWP8V6/yozaFZCPwrdB/RaDEFfOcI6TTLa2lfog=;
 b=iEyNhWmQOQpMQs16HTXI9WVOfijGF81w7062bQunWkLlDc2j6C+2XDMTncjGblg5+IVAdYJEg9Be5Em+/nzxafJL27AalLxbGnwBVSKfeGXzQymKiX7eE1d4a5oqY7L7IPFBfDsCk1ObtxtWwaiKNq7EWp9HjEraYXUePu0HdYlq+5la74rBYhZc9TWNRKQa2kf5I7xTtKU29tyKhw0Ocp0Tmqla7E6G7I2Hy2Ck4RV948Y3jmRvavgCAAKGMFLQZEEVHww+v06ntx1zvbFyKotNeh8JNQ8hFQ4904TEH9Zp0uO3wya/vPFs7rqPJ1aWEECtgaZIJfWJP7HjiCFR1w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vgCAzWP8V6/yozaFZCPwrdB/RaDEFfOcI6TTLa2lfog=;
 b=QmIK+4sGS8RyqPg0J2qo4tBFI+ajT6KTBg4yVanvUWGxyStyLWH0fhf5KFUa0SskRAxZvQkYrHRoQnPZk2KvE1RsoMtVTHDRjEPq4t8fkNVLNndDtdjPXHRgx+5xlm/XRzEzpoFxVR2H/FtwX0NXBnBZfJfju1fWGnn66tNqXnQ=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: RE: Ping: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
Thread-Topic: Ping: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
Thread-Index: AQHXmPQ6RAwx6Qn8tEyqmw4lz+vBDqulLEeAgAE+cwA=
Date: Thu, 16 Sep 2021 07:47:03 +0000
Message-ID: <BN9PR11MB5433D8A22A830BE9C2D715B58CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
 <9fb1d554-8fcc-fc0c-2b0c-52756001d36d@suse.com>
In-Reply-To: <9fb1d554-8fcc-fc0c-2b0c-52756001d36d@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3c2b2a74-9495-49c6-c8fe-08d978e62cb3
x-ms-traffictypediagnostic: BN6PR11MB1940:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-microsoft-antispam-prvs: <BN6PR11MB19409D444EF5BF4A5B7A31268CDC9@BN6PR11MB1940.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 00L6s7MVImJuL2oebegtMG45KK8BDZVt3JIdiyZJlIWGMyJp7nBlzM2e8Aa/mVNiSHAqpyaNawNMLdFmwLj2GRg2kEbtJ6dKDScvdTUHnTnTehsWQcCQihqRRC7BHI8Dg0waAhyWdg1crRyBK8giyu5gHBGFb6BsmsLnTRmmG4bLbXAf7VcwdklvU48fxFy8CPTGkO/qS02EK/QZFCQQNEoVmjSUHf27i+eRx/wdoG4NJ5GihtM7YzWO2SBHRZn31G50H7Q8LxAe7iOl910j5HpbL72CZ0nvIMU1mRI+ksFxiCth2wQvhhyEd48QRb5jb76KMu3acaeyIMrlwqm7pYHijIlfqqpWx7dmfCsWqn+bYudqfm5sqY1n37eCrWNPN/T/YGyOe50+bqMgnKnORv5SG9vaXmcmG4/IUazz5JeGz8ZA58AzEi2uov3CwfqYIF6GUsgFs2uNU5iIdYoMsyvLltX4EGiPcJSIXamev12ujjmVaQoV7RAqDe/hJIYAKrlOukUQxcnK2FqOr9NpmeTokCZx8/BG4K/WYWnWDElu94LMWNm7+r5qVWuxcxCa65MdRjYnnB7Rbx/8AJ22w6m1t+dpHmGkiUF6udP8zdgZfcdQcnS8fipfFQQjvnSC8w8GBKloxm2OUu4wm7vBp8fEKlwYJldsqm5NIsvW0VKrPqu255FEySWZlkCE+xLIDWYE2C3nXuR7MC3Y63U9fQ==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(376002)(346002)(136003)(366004)(4744005)(54906003)(38100700002)(316002)(122000001)(38070700005)(86362001)(6916009)(478600001)(9686003)(55016002)(8936002)(71200400001)(8676002)(26005)(52536014)(5660300002)(6506007)(53546011)(186003)(7696005)(2906002)(76116006)(4326008)(66556008)(66446008)(66476007)(64756008)(66946007)(33656002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?a3BsQWsvcWJkOTNOUDVUOElBaXFDcmt6UzdtdFViNlBGWUpQQkM0UExIRDhx?=
 =?utf-8?B?cmtuSVMvQk94L2ZQL3YzSUxubDJvRUp3bEljVGRmQU85cFJzVmpWK1lWTWg0?=
 =?utf-8?B?Zm1FM3VOQUI0aHdoQU81SEFCQUFrVnZ6aG5yQnd4UnpyS0F4ZDR5UGxqN3Jx?=
 =?utf-8?B?ZEptYXlNeFltK0JFMFl0M0RsK2xRajJOZk9DTTRCSUxpSTVlQVVZYmpvMjMz?=
 =?utf-8?B?dHd1b2dvY3FXMjJIeUxleUNKOHZhSC9XQ3ZQNzNwcnFGdTk3VGk5TkNNUjZ6?=
 =?utf-8?B?a3hxbkIwQUtoMHM5SHBHSEZYNi96dFFkZTZNMTQ4L3UvbTRPL3FxbmNHREZl?=
 =?utf-8?B?R2Z6Y2thTmJPQnV5UTNsY2ZERkpxVjkrN3BIWDRCbEl5Nk9QTTdaSjFZZEtE?=
 =?utf-8?B?c0pPWEttTzN6elEyNFRGVGI1bHlsemdjTmh2M3RiazVMYjB0dDRZcmFWYTV6?=
 =?utf-8?B?QldJZDNZZE5qSFNXbEwyUGV5S2NKdFBXT0s2bHRyVHpQbmdvcVV6aU95TzBR?=
 =?utf-8?B?YldQeU5ZRENPMWNrbUtKMFNVTXJOc1JJMEJjTzhiTlR2QnphNkZRK0U1RlpF?=
 =?utf-8?B?YXZmdzdwZ3V5RWV2am91LzdVVlorc0ZlV2xyNkNpNGZuTlRQNmgzMXZpbDRX?=
 =?utf-8?B?UCt1TlRvWHF5amZBRXJTTTBLeWJhODkyU3R4eUZTZkJWYWx6UjdOREh6NnN6?=
 =?utf-8?B?OW15NGszNWRWbTUwWEhTR0Z6MjdhUWxRZ1JEZHFCTEIyekwyMG9tanZkSVhn?=
 =?utf-8?B?RW9iSGl2OEZyajVNekdnNURpQ3BtZzR5NWM1ZElmZUd5d2crNzBwcVVIMC9l?=
 =?utf-8?B?Wnplc1JiVE1aTEZERDI5UVVyZEMwQ0daVXJXbUJmR0cyQzNHUzlxRkQ1MVZr?=
 =?utf-8?B?V2ZDUDNhcXRXWFVncllWNlVuS2VFY2Jkb1NoWHRzb3JBSS9pSCtORFFrUlNm?=
 =?utf-8?B?SVEySVcxcGZ2ajR0aHZFaTdiZld3cGUvOWhXTEtwZFNaL2JFUU05bUxuNnNn?=
 =?utf-8?B?eXFkaGdXNzIwVSsrZEpZZHhiZ2VvbUtRWGZlWnRwbmMzdTFITytGemt3eUJP?=
 =?utf-8?B?U0x2d0g3bm1PTEt2bno4Zlg1UzVuZkd4R1BzMTY0Y3BZSmhiSjJvMkQxNS82?=
 =?utf-8?B?QU9LMXVwMzExY1V4TGc2d2Mrd2MzNlR2OUdERmJmU0pwbjlHSEcwL0ExNFVW?=
 =?utf-8?B?NzdVbWI4YUxOWkxrWTZHdHJNR3V3MGcwYWFMMTZUTlJPUFRBbW5aSEN4Undj?=
 =?utf-8?B?UURVblBVOUZGUVNSS2UwTXMyV01BSmd1REk4Y3NsR1hrS0s5NFA4RUw3cmIz?=
 =?utf-8?B?QldSTkl2ekhxcVlENDZNdERPTVAxOW1SSFhLbzBaeDY0MWJwMVNCT2owRjla?=
 =?utf-8?B?QXlzaDlHaU9TQXBkUVhWWGgxamFoUEtlMTM2NEpZRlRDaEV0K2lQbFNZZ2dk?=
 =?utf-8?B?YlZMM0JLaUhBdzNHdTV4L0NKQVRNd2QrekFkMUJDR2k2a0V3d3ZibnpiV1ZK?=
 =?utf-8?B?d2dPNlpJclBkOWN6bzkzNXN3dGF5MUlLUWhKa1NkeFY0OWY3UnRjNjY2Q1N0?=
 =?utf-8?B?UEJVeHpXUXh1d3Rsakl3S1lTMUxtNTF6WGFGS3k0TjFONGowK25rUlZ4d3Ew?=
 =?utf-8?B?V1g3S05DV2N2Yk5zQzM1cE82VzJRS2FiVTczZVczVWhxOWo0QWlMcGpBSjU0?=
 =?utf-8?B?VENGVVRaZzI4WGdoMEVXaFIxMzNoUjJLcU02UXRRb0drUDhKcHZhRGhnNEN1?=
 =?utf-8?Q?IsQrCE7pitTeWyszwC6mQh962wE/pRvZmJ5e/Nk?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c2b2a74-9495-49c6-c8fe-08d978e62cb3
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 07:47:03.6212
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: GtfUTDfVkOFtW2Ol53zAUWWeC63ZHbty2UbrzUfhZmTi/DlNFnRMcf69nfwTZgbJf2M5yox8vdx7oniyj9Ds0A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1940
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFdlZG5lc2Rh
eSwgU2VwdGVtYmVyIDE1LCAyMDIxIDg6NDIgUE0NCj4gDQo+IEtldmluLA0KPiANCj4gT24gMjQu
MDguMjAyMSAxNjoyNywgSmFuIEJldWxpY2ggd3JvdGU6DQo+ID4gbWFwX2RvbWFpbl9wYWdlKCkg
ZXQgYWwgbmV2ZXIgZmFpbDsgbm8gbmVlZCB0byBjaGVjayB0aGVpciByZXR1cm4gdmFsdWVzDQo+
ID4gYWdhaW5zdCBOVUxMLCBhbmQgbm8gbmVlZCB0byBjYXJyeSBkZWFkIHByaW50aygpcy4NCj4g
Pg0KPiA+IFNpZ25lZC1vZmYtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4g
DQo+IG1heSBJIGdldCBhbiBhY2sgKG9yIG90aGVyd2lzZSkgb24gdGhpcyBwYXRjaCBwbGVhc2U/
IElkZWFsbHkgYWxzbw0KPiBmb3Igc29tZSBvdGhlciBWVC1kIHNwZWNpZmljIG9uZXMgaW4gdGhp
cyBzZXJpZXM/IEknZCByZWFsbHkgbGlrZQ0KPiB0byBnZXQgaW4gYXQgbGVhc3Qgc29tZSBwYXJ0
cyBvZiB0aGlzIHNlcmllcywgYmVmb3JlIHN1Ym1pdHRpbmcgYW4NCj4gZXZlbiBsYXJnZXIgdjIu
DQo+IA0KDQpJIGhhdmVuJ3QgZm91bmQgdGltZSB0byBjaGVjayB0aGUgZW50aXJlIHNlcmllcyBp
biBkZXRhaWwuIE9ubHkgYWNrZWQNCmEgZmV3IG9uZXMgd2hpY2ggbG9vayBvYnZpb3VzLg0KDQpU
aGFua3MNCktldmluDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:06:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:06:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188207.337286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmP5-0008SP-0I; Thu, 16 Sep 2021 08:05:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188207.337286; Thu, 16 Sep 2021 08:05:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmP4-0008SI-TO; Thu, 16 Sep 2021 08:05:54 +0000
Received: by outflank-mailman (input) for mailman id 188207;
 Thu, 16 Sep 2021 08:05:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQmP3-0008SC-4X
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:05:53 +0000
Received: from mga07.intel.com (unknown [134.134.136.100])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 391dfdb6-36fd-4bed-bbe6-0371b73f4da5;
 Thu, 16 Sep 2021 08:05:47 +0000 (UTC)
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
 by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 01:05:45 -0700
Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81])
 by fmsmga002.fm.intel.com with ESMTP; 16 Sep 2021 01:05:45 -0700
Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by
 fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 01:05:45 -0700
Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by
 fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 01:05:44 -0700
Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by
 fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 01:05:44 -0700
Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.101)
 by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 01:05:44 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN8PR11MB3699.namprd11.prod.outlook.com (2603:10b6:408:83::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 16 Sep
 2021 08:05:41 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:05:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 391dfdb6-36fd-4bed-bbe6-0371b73f4da5
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="286198014"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="286198014"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="553741054"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VYtOjjMTiuIp2Ro3qb/DxjmRrSArFGVKbpnH5ARkYucLwGkGSz0usf7AoyFmIGvjtWYVFRraEi3KKRfr/QhfWZGCYtiXzzVnXcIkvihpcMDuP2UDIckToZgzh65DdehLflw6UBfOYki4TET9Dla5MDshLfROkkE2iC+xEXgScjYCS7SYxIR9E/80bLv61UBT/wDsPImDJjDLNzHhT7X/glHC3+Oj+WLouuQnv+NvAAm12McRj/BvjxiG+uobLrPtjiWsV1kZ94aH41qWi+b0mZT9UiPwqFbEK8w4QHnGBzydcFsL/zFHIGFy8/VYsaSOeAfGespQ+jOzf4yHEDQTyA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=O8LK+FM6yRRrBERZwJtz3BrL+nVaynG4glbh+0FvoHo=;
 b=cN11K+v+y6UPUEIHYWOExRRfQIAUj+zRWLVWupCZMHe0WIkm6o750ufgoNsTLAz8MK7lmSYJc4aZ69bWJe/sbzeYmCPX4NEXt+xYYvj4AvO4ttl+yVApWaZ8EjJ2BCUACLGSYkN0hkDSx1xE0ZCq6VZYdhpNE2+sZk+t49qAo1ytVjuwqYf8TYRy85TrtnWhiQaRMaooIh0XSV0/+taQ45bgTaee8k0Fyfc/044pea2dc6nON9Mfaj0aoHLHkXq0xFQ2vjxebKK+hBb2fqNHUsuay2Wq1z51tkuYxf+87Fb399FyrVKzMvO/bCvBE/OzGxR9Ns3DS49jGFxMwtm+Og==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O8LK+FM6yRRrBERZwJtz3BrL+nVaynG4glbh+0FvoHo=;
 b=XKevgCXZbnUmPiCT+6gt0lThcLwCfesV7prgf+fwQHdyxV6a9NhMS4KsS9+sPKUpgJ+jg4oYEOQgpEYKWKJ3ge2zKvGtS3UZd34X1I36P6Ha1dTfRdlLYzn2YJzBcAAzZIMNTrZ5ijixAhF3XUCrFt/nOTh6LyLdtIPMKRx4HcA=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 1/4] VT-d: defer "no DRHD" check when (un)mapping devices
Thread-Topic: [PATCH 1/4] VT-d: defer "no DRHD" check when (un)mapping devices
Thread-Index: AQHXqhHWObtnwR6OkEiPWNnUbTtn96umTpdw
Date: Thu, 16 Sep 2021 08:05:41 +0000
Message-ID: <BN9PR11MB5433D08EC8FA6237C79832F88CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
 <afd4520f-7966-eec6-b1dd-9e0c12dc8836@suse.com>
In-Reply-To: <afd4520f-7966-eec6-b1dd-9e0c12dc8836@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 44a8e863-e549-4b7a-2574-08d978e8c6e9
x-ms-traffictypediagnostic: BN8PR11MB3699:
x-microsoft-antispam-prvs: <BN8PR11MB36991AA85A02EB0420A1F09D8CDC9@BN8PR11MB3699.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: a2PKcd5rE+1mwDbPvm3uT0ChlY66sGIJJmTqYCIY3FPsHNhxJYoxMpfaG8b3Zn/I2qt6/B9P2XOGk350wwLzoWk17HvovNKVGRqjJtd9bATBlr6584dRxvSGQxyc1EnnSDX3PkXyFpEedxjb5Dt5YzBuR7+X0oGYogwCun3jP5+PHg6ifo08s7Ncd/c/5QRR0cTW+bLAy40AM7+0K6nRZJKtYkXF4dHMlaqBFXkscp0z9Vdshelw/Gv2MCH4CLitRwioVEr0wY17QGUHEDUuIitPn4nFRy6jOdseBGdmyT49VJ6sMfMHzFvW9rjNoDA2Vh1SwhhAMLMqGxfG/XoewIdZ2CNOkQqO2hG/5hUjB9X3ruFyzO1JmocSPV/iEDaKqmZA3h2pHYbypsYrjDpJLrnGfx6uBi5/bRNIS0G0Yt/Y9EL7cC4BRF/7DRJkXzENwlyGQXNx05v1BqbFae1sDHOOB+OxNmyHd5kxpsGVeXX9mWRmpTY1DQLPuCYqk3jNJn8XC3dxeAgA3VwlHUipzw6/tHLw/GzafiVsIDuVP9ZAxVJAr8nvCeoSJVkGrT+gAzT5hlUrX0il7y3OyUQucGsZhqcg4XsQG0Cqx+DOad+I64DVorkgEMLqozO0FDYzUQOwQnFS3Ew3IuNZmyR8Pct4skzz99eTDFovHX8lQnR0qxXys0yuoEDNuTzv2euaaCMq8g4y5XBquBOPvrJB8w==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(136003)(366004)(346002)(39860400002)(86362001)(66446008)(71200400001)(110136005)(76116006)(4326008)(38100700002)(8676002)(7696005)(26005)(55016002)(122000001)(52536014)(8936002)(33656002)(9686003)(2906002)(478600001)(64756008)(6506007)(83380400001)(316002)(66946007)(38070700005)(186003)(66556008)(66476007)(5660300002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?RUMwTGhQVHVDd3hrM3BZMHNSWmVFc0x4T2pIdWdTQ05naU02TUdDRE4vUHY1?=
 =?utf-8?B?MXRMWGpGODZkODM5NVdtejhXWWRzd3RxbTA2dm5CYnU4c2hodURXNlRZTGxH?=
 =?utf-8?B?d3NjazVsbTl1MGNaVStNWWtwUlhGWHBmUUxUQ2gzVC84UGl6OVJic3pTazN6?=
 =?utf-8?B?QThDRlUwMHZDMDRjNmYyUTlRakEwanhyODNJZXR5b1hTSEgzNTM5alZnRGlE?=
 =?utf-8?B?UVN1QThBY2lORS9YOWJvc2tRVjRKZUpYOG5xWGRZOVVTVzc5SE9qMUViVitP?=
 =?utf-8?B?OExZdWUzaXUrTTUxM3Bic0xycGwyY2dzQ2xjSG5aelJqZkMvZitMR1BMdk5o?=
 =?utf-8?B?aW5uZTlCMkk1RjZsUldUQW5YdTNaaUs2b1FIKzJ0am1JTi9QR1JOSW1PVnJu?=
 =?utf-8?B?bWhWUGRTK1BnZHQwK0xEV2JsWmpxZmRBWEd6MXN6SjVsa2V4alFOc2huYnJJ?=
 =?utf-8?B?ZnJ2cGtUMHJ6MHhiaTlYalJDc2RWUFJHVHJmdUMzVVVwYThic1dEMTZTZGVB?=
 =?utf-8?B?YnNabnRVMlBNY1krTWJOQXJEa2J0c21uSzlBTXAxQkljaHNXcllEUEhITjhM?=
 =?utf-8?B?QklZT1JabU4rSzBHVzN5azNjNHRxcHhqYk9ueDhDN0kyeTFsTGxaTDZ4Z1Vq?=
 =?utf-8?B?L1Q1ZnVmVmdyRTQ3aS9VSEhTTTBuMmxPTHpyaWR2enBTMHBQVzlXVmUraWgz?=
 =?utf-8?B?VVdJc3lVR0xFWnJvVkdjUWp1ak11UUFlM2luUEZnR3pIb1FZZHp2T1BnMFh6?=
 =?utf-8?B?S25zeW1BSUhEWWNUa053Zm1ldk4wblBjU3BkOHFxV0ROUU1ER29Vbnh0aFpa?=
 =?utf-8?B?RmE1ME5FNm96bDd5cDV1ejJIclBUbkhWTHp2UkRCYW9mOVR0ZmVuNUZZLytl?=
 =?utf-8?B?ZE1SSHY4UXdqaGtnck8xZDZSWTIxaXdJR3VjWnRMRTR6UHdBOHNrNUVSWEhG?=
 =?utf-8?B?Uktrc0NpOGNGNkpwVklKQXhYd0hNdlFQcC9QM1VRZXM1K29YQXhwWkJ5YVBi?=
 =?utf-8?B?Y2J1cFltekcxOTIxSHYxWmVnMnJHbzNjcE91cFBGTzJ2a21wTkU3ZXFLSEpL?=
 =?utf-8?B?ZVlaZmpNRHlYVFJwdzJYMWx2d1E0OCtIMnJXdkZrb0ROamJETEFlMkJxazU1?=
 =?utf-8?B?eUxpSEF0QkRqMWI0NGM0cjl4Y2p1Q2RySXhRa2pjWWJFYXcrT2wwZGI0SExF?=
 =?utf-8?B?UWtDL1RBV0sxazdmdkNXV2ZBVDZZSnNCNDh0dk5mM0g0WkhaVXVuYndEbVQ1?=
 =?utf-8?B?eUJ5OEFXdHNQOWlMV1N4RUJiYnVqcmJ3V3RMaUk4bzhuM0RmdFhHZnByb2Ex?=
 =?utf-8?B?cXJIWHhCRWczTVlJMkZUQTF2TFZLMDN3ZmtveWI4dmQxTjJ6MW10cmx5dEpO?=
 =?utf-8?B?MXRvTHZELzlFL28vSm0xaUdDQkMzWFJCYVdmSkcyaVByQzI0cko5Zm1HVnRD?=
 =?utf-8?B?OFNjdlNJQWpJU1JlNWU5UWUxeHQvQ2crb2szdGcrRFV5M1oyM044RHpUdTl4?=
 =?utf-8?B?VFdOUlFXeTkvN3ZDY3FhVUdETVpUQldEZnE5UE95OEFHcVNTclIyZDQ0QUE2?=
 =?utf-8?B?S3o4U1VmUzBCVjRQQmhWOWZWN1R3TjVpd1JMTHNoL1NVaWM4bjVlZGU4dVJK?=
 =?utf-8?B?a0JPcUpuandBbFNteHQwL21QTUxpdmxLQ2phdGw3S1QwK1RCYkxCNlBwN2Rm?=
 =?utf-8?B?aDVMU2YxTlN1TjRUOW9rRHdkSWQ2a1IzdzhWdXA0SDFTbzlxR0F6WU5QU1ln?=
 =?utf-8?Q?D0JOUIQhk5U5tbymjFbNPA6dlwC1U9zveHs/vpv?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 44a8e863-e549-4b7a-2574-08d978e8c6e9
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 08:05:41.3957
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 5YWi9bMmfvlBE2Jg4jSF8aQu3/kZ3SQ5Hka1yL43gi8seSoNsC+PQgtHE6gi9F7ioug75VnbYZB/PWwxJ/nyxg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR11MB3699
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFdlZG5lc2Rh
eSwgU2VwdGVtYmVyIDE1LCAyMDIxIDU6MTIgUE0NCj4gDQo+IElmIGRldmljZXMgYXJlIHRvIGJl
IHNraXBwZWQgYW55d2F5ICh3aGljaCBpcyB0aGUgY2FzZSBpbiBwYXJ0aWN1bGFyIGZvcg0KPiBo
b3N0IGJyaWRnZXMpLCB0aGVyZSdzIG5vIHBvaW50IGNvbXBsYWluaW5nIGFib3V0IGEgbWlzc2lu
ZyBEUkhEIChhbmQNCj4gaGVuY2UgYSBtaXNzaW5nIGFzc29jaWF0aW9uIHdpdGggYW4gSU9NTVUp
Lg0KPiANCj4gV2hpbGUgdGhlcmUgY29udmVydCBhc3NpZ25tZW50cyB0byBpbml0aWFsaXplcnMg
YW5kIGNvbnN0aWZ5ICJkcmhkIg0KPiBsb2NhbCB2YXJpYWJsZXMuDQo+IA0KPiBTaWduZWQtb2Zm
LWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQoNClJldmlld2VkLWJ5OiBLZXZp
biBUaWFuIDxrZXZpbi50aWFuQGludGVsLmNvbT4NCg0KPiANCj4gLS0tIGEveGVuL2RyaXZlcnMv
cGFzc3Rocm91Z2gvdnRkL2lvbW11LmMNCj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gv
dnRkL2lvbW11LmMNCj4gQEAgLTE0NjAsMTQgKzE0NjAsMTAgQEAgc3RhdGljIGludCBkb21haW5f
Y29udGV4dF91bm1hcChzdHJ1Y3QgZA0KPiAgc3RhdGljIGludCBkb21haW5fY29udGV4dF9tYXBw
aW5nKHN0cnVjdCBkb21haW4gKmRvbWFpbiwgdTggZGV2Zm4sDQo+ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+ICB7DQo+IC0gICAgc3Ry
dWN0IGFjcGlfZHJoZF91bml0ICpkcmhkOw0KPiArICAgIGNvbnN0IHN0cnVjdCBhY3BpX2RyaGRf
dW5pdCAqZHJoZCA9IGFjcGlfZmluZF9tYXRjaGVkX2RyaGRfdW5pdChwZGV2KTsNCj4gICAgICBp
bnQgcmV0ID0gMDsNCj4gICAgICB1OCBzZWcgPSBwZGV2LT5zZWcsIGJ1cyA9IHBkZXYtPmJ1cywg
c2VjYnVzOw0KPiANCj4gLSAgICBkcmhkID0gYWNwaV9maW5kX21hdGNoZWRfZHJoZF91bml0KHBk
ZXYpOw0KPiAtICAgIGlmICggIWRyaGQgKQ0KPiAtICAgICAgICByZXR1cm4gLUVOT0RFVjsNCj4g
LQ0KPiAgICAgIC8qDQo+ICAgICAgICogR2VuZXJhbGx5IHdlIGFzc3VtZSBvbmx5IGRldmljZXMg
ZnJvbSBvbmUgbm9kZSB0byBnZXQgYXNzaWduZWQgdG8gYQ0KPiAgICAgICAqIGdpdmVuIGd1ZXN0
LiAgQnV0IGV2ZW4gaWYgbm90LCBieSByZXBsYWNpbmcgdGhlIHByaW9yIHZhbHVlIGhlcmUgd2UN
Cj4gQEAgLTE0NzYsNyArMTQ3Miw3IEBAIHN0YXRpYyBpbnQgZG9tYWluX2NvbnRleHRfbWFwcGlu
ZyhzdHJ1Y3QNCj4gICAgICAgKiB0aGlzIG9yIG90aGVyIGRldmljZXMgbWF5IGJlIHBlbmFsaXpl
ZCB0aGVuLCBidXQgc29tZSB3b3VsZCBhbHNvIGJlDQo+ICAgICAgICogaWYgd2UgbGVmdCBvdGhl
ciB0aGFuIE5VTUFfTk9fTk9ERSB1bnRvdWNoZWQgaGVyZS4NCj4gICAgICAgKi8NCj4gLSAgICBp
ZiAoIGRyaGQtPmlvbW11LT5ub2RlICE9IE5VTUFfTk9fTk9ERSApDQo+ICsgICAgaWYgKCBkcmhk
ICYmIGRyaGQtPmlvbW11LT5ub2RlICE9IE5VTUFfTk9fTk9ERSApDQo+ICAgICAgICAgIGRvbV9p
b21tdShkb21haW4pLT5ub2RlID0gZHJoZC0+aW9tbXUtPm5vZGU7DQo+IA0KPiAgICAgIEFTU0VS
VChwY2lkZXZzX2xvY2tlZCgpKTsNCj4gQEAgLTE0OTcsNiArMTQ5Myw5IEBAIHN0YXRpYyBpbnQg
ZG9tYWluX2NvbnRleHRfbWFwcGluZyhzdHJ1Y3QNCj4gICAgICAgICAgYnJlYWs7DQo+IA0KPiAg
ICAgIGNhc2UgREVWX1RZUEVfUENJZV9FTkRQT0lOVDoNCj4gKyAgICAgICAgaWYgKCAhZHJoZCAp
DQo+ICsgICAgICAgICAgICByZXR1cm4gLUVOT0RFVjsNCj4gKw0KPiAgICAgICAgICBpZiAoIGlv
bW11X2RlYnVnICkNCj4gICAgICAgICAgICAgIHByaW50ayhWVERQUkVGSVggIiVwZDpQQ0llOiBt
YXAgJXBwXG4iLA0KPiAgICAgICAgICAgICAgICAgICAgIGRvbWFpbiwgJlBDSV9TQkRGMyhzZWcs
IGJ1cywgZGV2Zm4pKTsNCj4gQEAgLTE1MDgsNiArMTUwNyw5IEBAIHN0YXRpYyBpbnQgZG9tYWlu
X2NvbnRleHRfbWFwcGluZyhzdHJ1Y3QNCj4gICAgICAgICAgYnJlYWs7DQo+IA0KPiAgICAgIGNh
c2UgREVWX1RZUEVfUENJOg0KPiArICAgICAgICBpZiAoICFkcmhkICkNCj4gKyAgICAgICAgICAg
IHJldHVybiAtRU5PREVWOw0KPiArDQo+ICAgICAgICAgIGlmICggaW9tbXVfZGVidWcgKQ0KPiAg
ICAgICAgICAgICAgcHJpbnRrKFZURFBSRUZJWCAiJXBkOlBDSTogbWFwICVwcFxuIiwNCj4gICAg
ICAgICAgICAgICAgICAgICBkb21haW4sICZQQ0lfU0JERjMoc2VnLCBidXMsIGRldmZuKSk7DQo+
IEBAIC0xNjUxLDE3ICsxNjUzLDEyIEBAIGludCBkb21haW5fY29udGV4dF91bm1hcF9vbmUoDQo+
ICBzdGF0aWMgaW50IGRvbWFpbl9jb250ZXh0X3VubWFwKHN0cnVjdCBkb21haW4gKmRvbWFpbiwg
dTggZGV2Zm4sDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KQ0KPiAgew0KPiAtICAgIHN0cnVjdCBhY3BpX2RyaGRfdW5pdCAqZHJoZDsNCj4g
LSAgICBzdHJ1Y3QgdnRkX2lvbW11ICppb21tdTsNCj4gKyAgICBjb25zdCBzdHJ1Y3QgYWNwaV9k
cmhkX3VuaXQgKmRyaGQgPSBhY3BpX2ZpbmRfbWF0Y2hlZF9kcmhkX3VuaXQocGRldik7DQo+ICsg
ICAgc3RydWN0IHZ0ZF9pb21tdSAqaW9tbXUgPSBkcmhkID8gZHJoZC0+aW9tbXUgOiBOVUxMOw0K
PiAgICAgIGludCByZXQ7DQo+ICAgICAgdTggc2VnID0gcGRldi0+c2VnLCBidXMgPSBwZGV2LT5i
dXMsIHRtcF9idXMsIHRtcF9kZXZmbiwgc2VjYnVzOw0KPiAgICAgIGludCBmb3VuZCA9IDA7DQo+
IA0KPiAtICAgIGRyaGQgPSBhY3BpX2ZpbmRfbWF0Y2hlZF9kcmhkX3VuaXQocGRldik7DQo+IC0g
ICAgaWYgKCAhZHJoZCApDQo+IC0gICAgICAgIHJldHVybiAtRU5PREVWOw0KPiAtICAgIGlvbW11
ID0gZHJoZC0+aW9tbXU7DQo+IC0NCj4gICAgICBzd2l0Y2ggKCBwZGV2LT50eXBlICkNCj4gICAg
ICB7DQo+ICAgICAgY2FzZSBERVZfVFlQRV9QQ0lfSE9TVF9CUklER0U6DQo+IEBAIC0xNjc2LDYg
KzE2NzMsOSBAQCBzdGF0aWMgaW50IGRvbWFpbl9jb250ZXh0X3VubWFwKHN0cnVjdCBkDQo+ICAg
ICAgICAgIHJldHVybiAwOw0KPiANCj4gICAgICBjYXNlIERFVl9UWVBFX1BDSWVfRU5EUE9JTlQ6
DQo+ICsgICAgICAgIGlmICggIWlvbW11ICkNCj4gKyAgICAgICAgICAgIHJldHVybiAtRU5PREVW
Ow0KPiArDQo+ICAgICAgICAgIGlmICggaW9tbXVfZGVidWcgKQ0KPiAgICAgICAgICAgICAgcHJp
bnRrKFZURFBSRUZJWCAiJXBkOlBDSWU6IHVubWFwICVwcFxuIiwNCj4gICAgICAgICAgICAgICAg
ICAgICBkb21haW4sICZQQ0lfU0JERjMoc2VnLCBidXMsIGRldmZuKSk7DQo+IEBAIC0xNjg2LDYg
KzE2ODYsOSBAQCBzdGF0aWMgaW50IGRvbWFpbl9jb250ZXh0X3VubWFwKHN0cnVjdCBkDQo+ICAg
ICAgICAgIGJyZWFrOw0KPiANCj4gICAgICBjYXNlIERFVl9UWVBFX1BDSToNCj4gKyAgICAgICAg
aWYgKCAhaW9tbXUgKQ0KPiArICAgICAgICAgICAgcmV0dXJuIC1FTk9ERVY7DQo+ICsNCj4gICAg
ICAgICAgaWYgKCBpb21tdV9kZWJ1ZyApDQo+ICAgICAgICAgICAgICBwcmludGsoVlREUFJFRklY
ICIlcGQ6UENJOiB1bm1hcCAlcHBcbiIsDQo+ICAgICAgICAgICAgICAgICAgICAgZG9tYWluLCAm
UENJX1NCREYzKHNlZywgYnVzLCBkZXZmbikpOw0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:18:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:18:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188216.337297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmbU-0001qK-8J; Thu, 16 Sep 2021 08:18:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188216.337297; Thu, 16 Sep 2021 08:18:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmbU-0001qD-55; Thu, 16 Sep 2021 08:18:44 +0000
Received: by outflank-mailman (input) for mailman id 188216;
 Thu, 16 Sep 2021 08:18:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQmbS-0001pr-RM
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:18:42 +0000
Received: from mga17.intel.com (unknown [192.55.52.151])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9bd00c04-7db4-4bfd-9247-0a586f7a2b9c;
 Thu, 16 Sep 2021 08:18:39 +0000 (UTC)
Received: from orsmga003.jf.intel.com ([10.7.209.27])
 by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 01:18:37 -0700
Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16])
 by orsmga003.jf.intel.com with ESMTP; 16 Sep 2021 01:18:37 -0700
Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by
 ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 01:18:37 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
 orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 01:18:37 -0700
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.170)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 01:18:37 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR11MB1458.namprd11.prod.outlook.com (2603:10b6:405:d::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 08:18:36 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:18:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9bd00c04-7db4-4bfd-9247-0a586f7a2b9c
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="202671594"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="202671594"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="434418387"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T7GYBkMeVM1XkZ813CXLbv9YQJWCEh3X2Z31aXgZ5efBllZoQX9MxaREuHjnxXKh30NtrvBBcUCmokNUvQA7mhGkiof0gQvSl9QrKeFTGVrb970PbxHy//9jpZR4ofPpqJTqR92KKVJz/An2lu2KDadsBP9xyFfhcr2dtHB4R3VUCwUmEaNxJ/6FYXC2yt1StTGFlyi0Im8uVeCtewIpRCcYy18tHxWhQvCUIyOAHV/zc0jE8PU3c0atfgbLt+1b1VETAX9tEVe5yeWLzUlaw13XGGjEvdbY8A/oogKol1YlYlOiBEDZrjvdC5pOLxv2xThnRCmlZcSzJW7eE8LKAA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kR67ruO5MPOO583+5PiBZ7zctYx+2qamf/UvRNENmYM=;
 b=kWtzGvbGU2yovDoOSCzdUGLc/M+1frO1JddjZgabdBfnbAHdXkDMXPfqdASN3m8t7raHnZh/iDsua4lOyvqqGEPgAn8dkpDK/6UNfiL6u2XgzFJtS+yizHwtApi+rmP87y3RXlqsY3P6/+LbnsSwcg2ghY5zmDJQ8J52aF9nAudWhiKTrLhIdCVWiiluVTjFJJhhTcV5KoODphZWU38TREDb3fwEVd9/kfNK4uTGVcyslYUSAp3R8rE8Pz5e16mb9eNf08WllCJYennBVibQAsZU4oI6DtqN7Vd/b/FfnmV34yq/mf7g/K/W6RDYoEBVTR3ZlnR0O1dXnSJrhcXnhg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kR67ruO5MPOO583+5PiBZ7zctYx+2qamf/UvRNENmYM=;
 b=kXvIs9dFLzdkWxTrmiBy1xaEEQr8EMGaP2IirciwToeUCBBwdVhaBpORU6LFFt+zPNg3cB4Qid+XZWN35h1DcyXRDfWVkl9N19ZnhvSmP2irsg1w/8nNB+PQGMO7ivqYdI1bxUg+mDuBs1NHyx9kUQJxZHdj54JaiH/2FdKxtic=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 2/4] VT-d: consider hidden devices when unmapping
Thread-Topic: [PATCH 2/4] VT-d: consider hidden devices when unmapping
Thread-Index: AQHXqhHtqP8x4M7szkGHc+SOfkqrHKumTyiA
Date: Thu, 16 Sep 2021 08:18:36 +0000
Message-ID: <BN9PR11MB5433706BE0A6C4B915D3707D8CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
 <5cf392cf-09a2-b23c-2a25-610c64f20707@suse.com>
In-Reply-To: <5cf392cf-09a2-b23c-2a25-610c64f20707@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d913a09d-8687-4367-c886-08d978ea94b0
x-ms-traffictypediagnostic: BN6PR11MB1458:
x-microsoft-antispam-prvs: <BN6PR11MB1458FE161BD3EB24C93D89B78CDC9@BN6PR11MB1458.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: htuuwxeLhVx67d9GyX0nENu581tlpo1TsIKEAcwkCiqOpKRDSQ6jeGqOIFxrsNYV+Eq7IzYueNrOrhuGnGag6lSxVU4N0FYq2RHGmGENBPUZdSEWtLoMh5d7WF7sN/0NAFe0U+nBOm/PV5JP7j2pk1F1fAgZx2qXeXS3Exrhdjd53WXKzlX9s1YyyZ3Xld5CRIrOBhVAGBUWeZKBGvFvl2dvQmAE4JjBK9A91MYyOvn3OYxrib3riGIm+V0EoDRI2OG2bqOVZqzpYcDAWX03d6waunG/kYcz2MTa9O9jllkG4kL2Itxi+m3gd3Xr87ibD3ehQNY9/teBYd5H9NjU6YEd/Geeyja/jCzuIlmINNKeLCJVvxF5WOQu/sQ7jTan+Mds9z/BFEBTZp/hJ5WcjuRck5Xn7cT8/2XvpeNn8J4ALn33ZeoLDbuvGyh7ciUUCbNXdf6BDKVDoQf6wDFz8qWpMJb+FTGs3Mr/iU+MLgyreEjt68Y4T9gS8Ccfccj5Sb/FIud07GOc7hmtAfdTtaZWkXdjAanePpJrF9vg2wywE7tj3vV/xS0cgBmy0lW+i6QvvEKoxDELYqRMTmO7PicAPBdJyuRQ7Qt4c3m+tixxOlBJ0c+eX0ySBG+PRXUwUbr63/Hay3sy/grSSqJFXOpc/qzU2A19BESfsCiEdzp2WNGB9bOUZpoFp8WHxdEzEl7r3OSm4W0TI3CQnsqd+w==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(316002)(110136005)(122000001)(86362001)(38070700005)(508600001)(55016002)(9686003)(8936002)(71200400001)(8676002)(26005)(52536014)(5660300002)(6506007)(186003)(7696005)(2906002)(76116006)(4326008)(66556008)(66446008)(66476007)(64756008)(66946007)(33656002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?aFZLZHRyUHZlRW93RDdkRGJRaHkzWFRRN0x4QmhiR2NGek9ha1pFRHNJenJD?=
 =?utf-8?B?bnpZc2dwWnJ2Rm1TUE1WU3NxalhDOXB3SGE1eUZzZG54b1BQcTFBVUlnOExv?=
 =?utf-8?B?anZZdjRJa2l0WkRKekZERFhsR1B6Ymo4UXRNUDRjOVp4N3A1dTJEMkdhZ3BW?=
 =?utf-8?B?WWc1bmt1UXJnNTdXZTlsOEIyTUxnSlA4dVJCS3lJamFEM1NXS0gwbHVZVWhm?=
 =?utf-8?B?RmdRdTdKbzM4SHBvM2J5SldmMnBaSTRBU0RqbWpNYnVMRmd6dklZbldMK3lr?=
 =?utf-8?B?eCtra2h1NGhaTGdiQk13bmpLZUZwcFBLRVduektIYVZYVnpwTCtyMVc5eWZx?=
 =?utf-8?B?dlVpeTQ3SVhlZENHTlpSb3Jnek5ab3NMSTFLMnlRR05nRWxrK0puakdtVlpK?=
 =?utf-8?B?akxMaUprR000Y1hEekVHdjBOK3Y3KzVGak1Idjhxemxib1ZLb2Z4d3drRlQw?=
 =?utf-8?B?U0xnVjZPeGpTSGZGQkllNDNqakozQXpoS011UUlHZDRJcjZ0TzRHWHR4QjBY?=
 =?utf-8?B?cjc4cVRkMlZlTFMzN3FQL002MUhsaHRuYVJXdnZkMVRMNDFVUFp3ZkdQNjJo?=
 =?utf-8?B?NVRjeFFGZlE4SFUvbW5QSnhuWlIyamxTbWd6S01XRUptMnlRVlhOS0tzRE9J?=
 =?utf-8?B?R1Y2N3ZCZFE2THlodWZHZklZMThkTkN2UWhSSm1lYjgxZWhDdTMwRlpBZ2g0?=
 =?utf-8?B?elU0MlhiRVY3T0lzNVhhZVhGanhtdmhwbHg4cjRtM2Nnb2c1THk5UEFqTFN5?=
 =?utf-8?B?WUx0UFgvblRsTUl6aVV1bGdpWVVZQ3BOQ3l2WHJUbkE0K3RTODNneUFlMmxm?=
 =?utf-8?B?ZU9vU1pQMUFQYzhKZ2FIeXplTnJnL1RVTUx3b3JVVVZ6dER3YkczUTZOSHBk?=
 =?utf-8?B?WE80c21MRmYvcXowbDhDRzIrb0diWU91VHBBUDQveGYvaVZRV0E2eVlNQ2R6?=
 =?utf-8?B?N21JdDJib0JWSEVhQ0JXNC8wOTJ4WkVZb3gveDF5NFZpMW5kamtuRHRiMkZS?=
 =?utf-8?B?WmVZSURtOGRkVTdya0dZTmxRSzArT1p2RW9OdW00NkVMUkhFZzhhSDNMTlJL?=
 =?utf-8?B?M2lLYnFCdWdBOGxSNnhmZGh1eGlmTFdaNTFzNFQybnRJSUhMeXFhSmpxRVZW?=
 =?utf-8?B?UWxnUlF1RC9VcGk0VDMvTWpQRnB2RUNiQ29kSjJjSFdvYUcvM2NFY2VwQlI4?=
 =?utf-8?B?cDhDTU5yZXl1cGFkVXoxSXMwa2hETjkrRUcrSkZJUE9ERmN2M2I0ZzJCbURo?=
 =?utf-8?B?dUYyQysyaytncW0yaUxudkROeXprSTFoSmFXS2lybnI0S0hMdjRvampxOEJi?=
 =?utf-8?B?SVZVS1ltcW1JSG51VUw0N0JJdGhPMDloMmY2emxjbS9Sem1TMDlYekFDTFFK?=
 =?utf-8?B?cjIxdGpCNitnanBqOVRMdGxoeUhmbXV3QURZT25uYnlUNGtjUUNsZkswUUcw?=
 =?utf-8?B?QkR3aThnUDlhL3VRelNiOElYSTZIcDFqWUxpRjB4QmFwOUJtU0R3MDhrWDVZ?=
 =?utf-8?B?cE1uanVUU2xIazBjWmZoNEpSK1B5S08xZXhaTlRmZGRJcWEyMEViMk5oL3hm?=
 =?utf-8?B?Wi9CcUNtTVhZS3ZRT1NxSmtNaXF2Qkh6SWl1U05rTUppZ0todlR5bmFFcHMx?=
 =?utf-8?B?RFZwSnZNR0tlWTcvTTJpTjhBV21tdkhEOUs4UlpESG1hQjJ2OGtvQUtWbjFR?=
 =?utf-8?B?c3pYR3dDZFFqV0R5ME5BdmY4WnFKdmljUHdYMnZOb0pVMW0yY0J2MzJxV2VC?=
 =?utf-8?Q?bB7hhH480korymM6R0VYBsIhDGK1s+Pn+9pav9Q?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d913a09d-8687-4367-c886-08d978ea94b0
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 08:18:36.1227
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: g4PidYB8JM+ntYMZtv9mhoIHwrUZbgREKHqlUgFLkKoarq33dx9gKPKhXrtVj8Ub70JvJsNU/50tmzX5t3QALA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1458
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaA0KPiBTZW50OiBXZWRuZXNkYXksIFNlcHRlbWJlciAxNSwgMjAy
MSA1OjEzIFBNDQo+IA0KPiBXaGV0aGVyIHRvIGNsZWFyIGFuIElPTU1VJ3MgYml0IGluIHRoZSBk
b21haW4ncyBiaXRtYXAgc2hvdWxkIGRlcGVuZCBvbg0KPiBhbGwgZGV2aWNlcyB0aGUgZG9tYWlu
IGNhbiBjb250cm9sLiBGb3IgdGhlIGhhcmR3YXJlIGRvbWFpbiB0aGlzDQo+IGluY2x1ZGVzIGhp
ZGRlbiBkZXZpY2VzLCB3aGljaCBhcmUgYXNzb2NpYXRlZCB3aXRoIERvbVhFTi4NCj4gDQo+IFdo
aWxlIHRvdWNoaW5nIHJlbGF0ZWQgbG9naWMsIGNvbnZlcnQgImZvdW5kIiB0byAiYm9vbCIuDQo+
IA0KPiBTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IA0K
PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUuYw0KPiArKysgYi94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUuYw0KPiBAQCAtMTY1Nyw3ICsxNjU3LDcgQEAg
c3RhdGljIGludCBkb21haW5fY29udGV4dF91bm1hcChzdHJ1Y3QgZA0KPiAgICAgIHN0cnVjdCB2
dGRfaW9tbXUgKmlvbW11ID0gZHJoZCA/IGRyaGQtPmlvbW11IDogTlVMTDsNCj4gICAgICBpbnQg
cmV0Ow0KPiAgICAgIHU4IHNlZyA9IHBkZXYtPnNlZywgYnVzID0gcGRldi0+YnVzLCB0bXBfYnVz
LCB0bXBfZGV2Zm4sIHNlY2J1czsNCj4gLSAgICBpbnQgZm91bmQgPSAwOw0KPiArICAgIGJvb2wg
Zm91bmQ7DQo+IA0KPiAgICAgIHN3aXRjaCAoIHBkZXYtPnR5cGUgKQ0KPiAgICAgIHsNCj4gQEAg
LTE3MzcsMjMgKzE3MzcsMzMgQEAgc3RhdGljIGludCBkb21haW5fY29udGV4dF91bm1hcChzdHJ1
Y3QgZA0KPiAgICAgICAgICByZXR1cm4gcmV0Ow0KPiANCj4gICAgICAvKg0KPiAtICAgICAqIGlm
IG5vIG90aGVyIGRldmljZXMgdW5kZXIgdGhlIHNhbWUgaW9tbXUgb3duZWQgYnkgdGhpcyBkb21h
aW4sDQo+IC0gICAgICogY2xlYXIgaW9tbXUgaW4gaW9tbXVfYml0bWFwIGFuZCBjbGVhciBkb21h
aW5faWQgaW4gZG9taWRfYml0bXANCj4gKyAgICAgKiBJZiBubyBvdGhlciBkZXZpY2VzIHVuZGVy
IHRoZSBzYW1lIGlvbW11IG93bmVkIGJ5IHRoaXMgZG9tYWluLA0KPiArICAgICAqIGNsZWFyIGlv
bW11IGluIGlvbW11X2JpdG1hcCBhbmQgY2xlYXIgZG9tYWluX2lkIGluIGRvbWlkX2JpdG1hcC4N
Cg0Kd2hhdCBpcyBjaGFuZ2VkIGFib3ZlPw0KDQo+ICAgICAgICovDQo+IC0gICAgZm9yX2VhY2hf
cGRldiAoIGRvbWFpbiwgcGRldiApDQo+ICsgICAgZm9yICggZm91bmQgPSBmYWxzZTsgOyBkb21h
aW4gPSBkb21feGVuICkNCg0KaG9uZXN0eSBzcGVha2luZyBJIGhhZCB0byByZWFkIHNldmVyYWwg
dGltZXMgdG8gdW5kZXJzdGFuZCB0aGUgYnJlYWsNCmNvbmRpdGlvbiBvZiB0aGlzIGxvb3AuIEl0
J2QgYmUgZWFzaWVyIHRvIHVuZGVyc3RhbmQgaWYgcHV0dGluZyBmb3JfZWFjaF8NCnBkZXYoKXt9
IGludG8gYSBmdW5jdGlvbiBhbmQgdGhlbjoNCg0KCWZvdW5kID0gZnVuYyhkb21haW4pOw0KCWlm
ICghZm91bmQgJiYgaXNfaGFyZHdhcmVfZG9tYWluKGRvbWFpbikpDQoJCWZvdW5kID0gZnVuYyhk
b21feGVuKTsNCg0KPiAgICAgIHsNCj4gLSAgICAgICAgaWYgKCBwZGV2LT5zZWcgPT0gc2VnICYm
IHBkZXYtPmJ1cyA9PSBidXMgJiYgcGRldi0+ZGV2Zm4gPT0gZGV2Zm4gKQ0KPiAtICAgICAgICAg
ICAgY29udGludWU7DQo+IC0NCj4gLSAgICAgICAgZHJoZCA9IGFjcGlfZmluZF9tYXRjaGVkX2Ry
aGRfdW5pdChwZGV2KTsNCj4gLSAgICAgICAgaWYgKCBkcmhkICYmIGRyaGQtPmlvbW11ID09IGlv
bW11ICkNCj4gKyAgICAgICAgZm9yX2VhY2hfcGRldiAoIGRvbWFpbiwgcGRldiApDQo+ICAgICAg
ICAgIHsNCj4gLSAgICAgICAgICAgIGZvdW5kID0gMTsNCj4gLSAgICAgICAgICAgIGJyZWFrOw0K
PiArICAgICAgICAgICAgaWYgKCBwZGV2LT5zZWcgPT0gc2VnICYmIHBkZXYtPmJ1cyA9PSBidXMg
JiYgcGRldi0+ZGV2Zm4gPT0gZGV2Zm4gKQ0KPiArICAgICAgICAgICAgICAgIGNvbnRpbnVlOw0K
PiArDQo+ICsgICAgICAgICAgICBkcmhkID0gYWNwaV9maW5kX21hdGNoZWRfZHJoZF91bml0KHBk
ZXYpOw0KPiArICAgICAgICAgICAgaWYgKCBkcmhkICYmIGRyaGQtPmlvbW11ID09IGlvbW11ICkN
Cj4gKyAgICAgICAgICAgIHsNCj4gKyAgICAgICAgICAgICAgICBmb3VuZCA9IHRydWU7DQo+ICsg
ICAgICAgICAgICAgICAgYnJlYWs7DQo+ICsgICAgICAgICAgICB9DQo+ICAgICAgICAgIH0NCj4g
Kw0KPiArICAgICAgICAvKg0KPiArICAgICAgICAgKiBIaWRkZW4gZGV2aWNlcyBhcmUgYXNzb2Np
YXRlZCB3aXRoIERvbVhFTiBidXQgdXNhYmxlIGJ5IHRoZQ0KPiArICAgICAgICAgKiBoYXJkd2Fy
ZSBkb21haW4uIEhlbmNlIHRoZXkgbmVlZCBjb25zaWRlcmluZyBoZXJlIGFzIHdlbGwuDQo+ICsg
ICAgICAgICAqLw0KPiArICAgICAgICBpZiAoIGZvdW5kIHx8ICFpc19oYXJkd2FyZV9kb21haW4o
ZG9tYWluKSApDQo+ICsgICAgICAgICAgICBicmVhazsNCj4gICAgICB9DQo+IA0KPiAtICAgIGlm
ICggZm91bmQgPT0gMCApDQo+ICsgICAgaWYgKCAhZm91bmQgKQ0KPiAgICAgIHsNCj4gICAgICAg
ICAgY2xlYXJfYml0KGlvbW11LT5pbmRleCwgJmRvbV9pb21tdShkb21haW4pLQ0KPiA+YXJjaC52
dGQuaW9tbXVfYml0bWFwKTsNCj4gICAgICAgICAgY2xlYW51cF9kb21pZF9tYXAoZG9tYWluLCBp
b21tdSk7DQo+IA0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:19:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188225.337308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmcG-0002So-Mw; Thu, 16 Sep 2021 08:19:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188225.337308; Thu, 16 Sep 2021 08:19:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmcG-0002Sh-Js; Thu, 16 Sep 2021 08:19:32 +0000
Received: by outflank-mailman (input) for mailman id 188225;
 Thu, 16 Sep 2021 08:19:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=URfn=OG=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mQmcF-0002SX-9i
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:19:31 +0000
Received: from mga03.intel.com (unknown [134.134.136.65])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d060a8ec-16c6-11ec-b5b3-12813bfff9fa;
 Thu, 16 Sep 2021 08:19:30 +0000 (UTC)
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
 by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 01:19:29 -0700
Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84])
 by FMSMGA003.fm.intel.com with ESMTP; 16 Sep 2021 01:19:29 -0700
Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by
 fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 01:19:28 -0700
Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by
 fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Thu, 16 Sep 2021 01:19:28 -0700
Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by
 fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Thu, 16 Sep 2021 01:19:28 -0700
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.42) by
 edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Thu, 16 Sep 2021 01:19:28 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR11MB1458.namprd11.prod.outlook.com (2603:10b6:405:d::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 08:19:27 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:19:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d060a8ec-16c6-11ec-b5b3-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="222567978"
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="222567978"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,297,1624345200"; 
   d="scan'208";a="545434477"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ErgSmgREHEjzNwR+R6oo5RtYb1FE1AcUPh5mUCh7/8ttl0f4l9eYQzssYgvbZub5TFDi4EG6T615CEvkWGTqGV8GwGKbeWH3jyys3CC/W0NQmNT4Pi0YxUjidZYAPcE+CH6zhOHrqvv1jOQSdF2ywj572bhNFRPr75hy2WruamFGf7VKsLDjGugRsXVgRHmlzi/VJEyNDdSdjEG14Ch43BO48vGzodvDA8pSJ1hw9ORORTEFlzFdGlPSEWCpLPxk0NBMWmMBLd/oG9H86gjMtp9r04+NA6O5Z1UEgmnf5yPWd3qA3oZ/OkctDrY2kRNvWtpmgxzgl1HCJprxPFjPuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vni7LZgLSjFb2OcXU8M1HmqnLKzm+Z+/DPBs9mlrFZ8=;
 b=HCJ/gsUxHPad3K/v/ChoCP6EeoVOAOex0b9JZE02A8wpQ6JBuSusYBMfbYyTf/AN3dv7E93J3Z1ZsoONwUDPV3663E4j2nSxgBf2j/EeJ0+oef1GCvWoL8/KkxTYoIrARbpWVrLLRQ7ATNwrAnQY9BWE220j8mALsntP/1+uyeRhugp85Uh/BKtIjpMqF+CvZ3OKVg0+LIVz900v38TEDZKvsr21klW0ePqzuXgkew6Y2my40z9AAhJbV0IvBGiieudTIl7lEtM8aZkUc/5/WHl408nIV67UuG/Lhu9ZnjPQBvZY/nMIPAZtPh+Bz4zBHqkT0VDUWxrQ2aUA+gdIhw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vni7LZgLSjFb2OcXU8M1HmqnLKzm+Z+/DPBs9mlrFZ8=;
 b=LCjrEGt8xQemBHASq43denHwDPjf4NHrdDKvaf96awRYSiVqSJIXVIMRaB5CIjR06ptCyzPBdh8gleUAsxrWyOE/uBRBUbwbuYL4v3K7rafO0XA3JXd1Cc1Z/OikX+lnfltTJjw7GUgLAvHrKnFl6yOLXttcqNedVR5Pn+oyP80=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Paul Durrant <paul@xen.org>
Subject: RE: [PATCH 3/4] VT-d: skip IOMMU bitmap cleanup for phantom devices
Thread-Topic: [PATCH 3/4] VT-d: skip IOMMU bitmap cleanup for phantom devices
Thread-Index: AQHXqhH+i+85PO0Ft0uMLVDjFxaknaumUujA
Date: Thu, 16 Sep 2021 08:19:27 +0000
Message-ID: <BN9PR11MB5433D6D1475F2E710A02D4D28CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
 <57c75e32-df31-c886-b2b2-86d0f82a57d0@suse.com>
In-Reply-To: <57c75e32-df31-c886-b2b2-86d0f82a57d0@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 5414abf0-3964-4ec7-6c10-08d978eab326
x-ms-traffictypediagnostic: BN6PR11MB1458:
x-microsoft-antispam-prvs: <BN6PR11MB1458E014A1E4CE95B286B53E8CDC9@BN6PR11MB1458.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: UPbJcGWrON81aNR8mDInjPDyOuwaEO8jNHgeDOTy02/6/yjGWZI9qZe30SSUYfdVFn9iKdCyZCMswIzar+HJoBdj5+Jo8+LVXxRHvWKfTo3JxkpzGSYd0w/TNIxUtmWh9/F5j8OC1yGIjXJAPgGi+VpFKJvPo0tuErb7arO9vaoa/+yOx+OsmoaMIqi2wsfzncHReyTuT5p/9vwACs8SGU2pRc30DDweJz+0dmKSTry5cZrXELF/fNHI0KcEJbCvNJofyXF9QK5M6gBSFZqpK6E7/t+Tzigec5fO7AxFP32Fv/Lxy/cqQsVlSbDyXjWHpBXwMfFJrpNgITJyuiUrXFLBNulI1/K/ztxDxo66IVRqp6i6uVMDUwDYvSFr21GszPu5FgSV0Hc3vSs9DfZ7sU32GYrOiQ1PoUI8kjGw0IAeijYDjlpUVLuaDuakqNvNhzxFhnrFUpyUq5zmVZkIKkV0j9BdLQ7ZNB3011buMGUn/PHM44HZe/d45wjnQCdK1FpLsnwbfg0g/RINvamLZmFJuV71iVWu581KKr/Se8tleqyQKhRml+y1q1vfon2nae8M6UkZg2MCOpBKRk3oiMtv5CLHPNpLJbVqQnAH9/39Fq4JOpIvJx/T2H+cUxWbeOrliQsotSY8L+BAEtX/h77aUX+OgmYWf7rgtjsx9IpPa900c6nDYHyRyP3WZ//OS84nDq0c3eJaxBfOKlEFng==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(71200400001)(8936002)(52536014)(26005)(8676002)(38070700005)(508600001)(55016002)(9686003)(4326008)(76116006)(2906002)(33656002)(66556008)(66446008)(66476007)(64756008)(66946007)(186003)(7696005)(5660300002)(6506007)(38100700002)(316002)(110136005)(4744005)(122000001)(86362001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?dTdpSnRiWmsyQXEwOXd3RURjOVgwUVQybURtempiWXo5SzRqU1Nqd0RySnRQ?=
 =?utf-8?B?a0pJTjJ2QWV0cWIvUWlaZ3RMN0lVZUE4Slg0NDZLZEZwQnY4MGR6SS9YTlJJ?=
 =?utf-8?B?bUhLTG5GV0NVNlczbWtCS1Q2UlJDcm9wM1FlOU1jL2FnRG9NWmFWNlhnYnlE?=
 =?utf-8?B?bG5SRTBCZmMxZFhrWlRkMjY3VGowWDBiWk9SYkpBU3Rnb0loMlFpR2NEL1RL?=
 =?utf-8?B?ek54NkJRWTdhbS8yZnZGV1l6ODlBbzcwK3FHL0pZOTJxeXRTZURaNWlPSCti?=
 =?utf-8?B?ZUo3cXM3bm03eUNXWmEyQStjMFVPaUFuN2RUWXpMNTI1blY4ZFBkY25DYll5?=
 =?utf-8?B?K3JZVmlVQXo5ZXRKdit1MnE4aERXYWpEbzRYWlB6MlJ3Y2RnWHdkNnRRaUFF?=
 =?utf-8?B?RU5NaGtjME5rd1U3aWg2ek5xSUNabWJoQlFFbVl4SVpOb1FqdklXMk8rcFg1?=
 =?utf-8?B?Sjd2dGcvY2R2bENHNWxCSHIrT3VvaFlsS3BMMm9QakFXV3o4UVRxNlgyQURP?=
 =?utf-8?B?ZG93MGVtK2c4aFc0Z2Q4SFhFU3BzeU5yM2JKVTJyRE9qWGdHOUt4WE9hc1cy?=
 =?utf-8?B?elFURGNYbmdhTjFSejRIYmZzcmpYTEtoOWdBb2ZQSElkaVFTNGUvYythLzNK?=
 =?utf-8?B?OEorNktodXhMWEpDdzZtY2ttd01lS3FWYjYzTzRvWE83ak5nQ0pacG9ROUw4?=
 =?utf-8?B?OUhiclNaeGN6VUhKMnUrUVdsdmpPYVBhZVBVaWpqR0pUUmNWVkd1dnpuVlE2?=
 =?utf-8?B?OFdPMHpYcjVtamYyOCtYWFZYeGpqenYyY2drNWNUL20vVWhpaHFBRzFobEM1?=
 =?utf-8?B?S3ZJQWZ6UzQ5VVlRR1VLWTA4Rm1odEdCb1ZuZE0zdEQrL3RwcUVjTEwza1NU?=
 =?utf-8?B?d2dOQ3lJQi9Uamd3SjdsS29ieldoK2ozbGduQk5lc3dJQUhSY3NXRU0vMDl0?=
 =?utf-8?B?U05GNjV2V3RUVmowbSt5V1FJNjVxRUc4MEhEK21ZS2dHbmhyQmpscTlMQXV2?=
 =?utf-8?B?MHZnOE5QTmdINllCVlZhRDlWWVV3T09QU0owZ0twSmJJWnN1amdvTDdqdVQz?=
 =?utf-8?B?cGtva1JsT0FGZ3JIZVAwM2FLbzI0YTNoYmRVcGhRVTRTZi95ZlppTlgwUERn?=
 =?utf-8?B?eWxzYzZoNmFTdTQwb0Zqdi9CcFNLS0hLdWRvengrSzY5T0YyaXdPNDJmaUND?=
 =?utf-8?B?UUxCVUk5ZUF5NUNXTW5RZFZ1cDYxeHFJRXpFNFNtcjNJaU94TVhZN1RteXRR?=
 =?utf-8?B?ZXovT2JjQ1VkTzJjNDhoZ1ZiOTNvNUNXSnlVRzhJZGUyUkZYZzJuRWtjNkdU?=
 =?utf-8?B?cS9KSllxb1libUdWYXd0VHdQdlBQemJ1M2IwWlp6cm1YV0JBbVRibXArM01m?=
 =?utf-8?B?a1puRnRYNys2dVdENFl1RjBKL09XWkp3aFRlQmQ4YXI1c0V2UGVzMUJjZjFV?=
 =?utf-8?B?SU9CNCt3bHY2TURKVjNmUk5DOXcyQWp6ZFU4bldtL1hNY3k3YUR6Mlk1SlFy?=
 =?utf-8?B?clFuTjZVczhDMXlaVENUVjIvTGV0UEprbVlENG9TN1NqQ05adkRVMnJMUS9M?=
 =?utf-8?B?ZG1ITG5aVm5OcmpHZk1vN21lNGlUS1ZxaGlhRDFTbWxrUkxaTGdqaXRXRW1P?=
 =?utf-8?B?c0RSWUNPOEdIOEc3clFGUDVnZ2k2MmRHSTNmSmZpc2thS3N5YTdnaEhpa05H?=
 =?utf-8?B?Zkk2dFBYN3ozUDZ2R1NCZEJpbDk1cHZGZHBseXBzTWdiRUY1UjBqQk1BSUJX?=
 =?utf-8?Q?rlUK1bO6KWyGr104uMlxCA3g45ETORjWJ1euSPs?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5414abf0-3964-4ec7-6c10-08d978eab326
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 08:19:27.1860
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FCmlBZSx6R3hecGpRyEiSjxHGrJx0Ws5oj3+5TCaBwDfTND1aj0TjXToAFsUWzRIH9ex7A7sScmA5CNqsQ+X+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1458
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IFdlZG5lc2Rh
eSwgU2VwdGVtYmVyIDE1LCAyMDIxIDU6MTMgUE0NCj4gDQo+IERvaW5nIHRoZSBjbGVhbnVwIGFs
c28gZm9yIHBoYW50b20gZGV2aWNlcyBpcyBhdCBiZXN0IHJlZHVuZGFudCB3aXRoDQo+IGRvaW5n
IGl0IGZvciB0aGUgY29ycmVzcG9uZGluZyByZWFsIGRldmljZS4gSSBjb3VsZG4ndCBmb3JjZSBt
eXNlbGYgaW50bw0KPiBjaGVja2luZyBhbGwgdGhlIGNvZGUgcGF0aHMgd2hldGhlciBpdCByZWFs
bHkgaXM6IEl0IHNlZW1zIGJldHRlciB0bw0KPiBleHBsaWNpdGx5IHNraXAgdGhpcyBzdGVwIGlu
IHN1Y2ggY2FzZXMuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hA
c3VzZS5jb20+DQoNClJldmlld2VkLWJ5OiBLZXZpbiBUaWFuIDxrZXZpbi50aWFuQGludGVsLmNv
bT4NCg0KPiANCj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRkL2lvbW11LmMNCj4g
KysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRkL2lvbW11LmMNCj4gQEAgLTE3MzMsNyAr
MTczMyw3IEBAIHN0YXRpYyBpbnQgZG9tYWluX2NvbnRleHRfdW5tYXAoc3RydWN0IGQNCj4gICAg
ICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ICAgICAgfQ0KPiANCj4gLSAgICBpZiAoIHJldCB8fCBR
VUFSQU5USU5FX1NLSVAoZG9tYWluKSApDQo+ICsgICAgaWYgKCByZXQgfHwgUVVBUkFOVElORV9T
S0lQKGRvbWFpbikgfHwgcGRldi0+ZGV2Zm4gIT0gZGV2Zm4gKQ0KPiAgICAgICAgICByZXR1cm4g
cmV0Ow0KPiANCj4gICAgICAvKg0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:24:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:24:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188233.337319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmhO-0003uo-Af; Thu, 16 Sep 2021 08:24:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188233.337319; Thu, 16 Sep 2021 08:24:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmhO-0003uh-7K; Thu, 16 Sep 2021 08:24:50 +0000
Received: by outflank-mailman (input) for mailman id 188233;
 Thu, 16 Sep 2021 08:24:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQmhM-0003ub-OC
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:24:48 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 62d70720-e747-44c5-8259-dd580893b04a;
 Thu, 16 Sep 2021 08:24:47 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2053.outbound.protection.outlook.com [104.47.2.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-pA3oyOJwOYWx89Xfs81eIA-1; Thu, 16 Sep 2021 10:24:45 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 16 Sep
 2021 08:24:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:24:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0125.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 08:24:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62d70720-e747-44c5-8259-dd580893b04a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631780686;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CuAQst5osSHV9vAPMHejBIJ2CncxLvl9tAt+Nr/ovEs=;
	b=Xo8BAWL+uFjfn/kczx3QWV5APfd/sUp51Snor6l6S7+phjbgr76hY9leieRLaDFg+ZeTOB
	AyLxENp3Uk4oYe+S+sb6fgL9XxSlcy5uXLCd1qquAkfplY7+N3hgi1nMiWX5jZeH0scf2A
	N0gtEuTJbVtTkWMyT1P4NHh2p0o6OFc=
X-MC-Unique: pA3oyOJwOYWx89Xfs81eIA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bFN8pbiKvMVel5LDbu0uIsfvxnVUEci7KNAgy48ouRa8WpF555ualDBvPlsCCScEMlfB+Rp7o2UkJ94dLMjjzCnSqfoD1s41FBJvnsmgl2CPQHHTGUpgdAGM1avQ3tMonXlg/KI2OFtTonrxdDJrAVKLR1m+do2LWFLObOFOvNJFW98PImVWrq+4TQueSA2gNLxg24HyOkOx3WCROKyOjOmXQcZPwJJin04D2iuwHi6xKpvhGmHpWxgVFLgvgWlhvcAZ9I83/2gF0LOvye4PP5SS3slP37cSHJ6kpi4zmJcrnd8EjVi9azOJpXYq0Zh5NH5Mpngc7oVLXgJTp6Ph8A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CuAQst5osSHV9vAPMHejBIJ2CncxLvl9tAt+Nr/ovEs=;
 b=EDJLnTlg0awGer++LMKA0YuJdoFjOgkmHjujkQAuqs0FkWLymYYQcnWjA+FLHFVCoMQ1Jba+02gok9QxXFw0STmq5QepZdSqjTwGiNesbZYDJeAj0MIeNT6YLH5nh/7328wgNpL+k4CRKKFoYeYEFQR/yJJdzn3h6yD6ibZY5FT6aUNUbSLh45S5t1a3WuUrI1ZcUNHVM8yfObY3dtCKp9bxyFW+AIonDsGEirij0VUHTPCaSKPow003Cw2y3apIMnSSdepaOjHz/9vDox6pS04W66Kr4dZ77tgG5RNKDmdlMrWHyMzM2BkQ2xOVFW+a0Nov4Jq8662bBDRDj3wkzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: Ping: [PATCH 17/17] IOMMU/x86: drop pointless NULL checks
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: "Cooper, Andrew" <andrew.cooper3@citrix.com>, Paul Durrant
 <paul@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d955d9a3-33f6-f228-d007-a844ed918168@suse.com>
 <25ad2429-df7d-95c6-ec70-4b23e6a023a2@suse.com>
 <9fb1d554-8fcc-fc0c-2b0c-52756001d36d@suse.com>
 <BN9PR11MB5433D8A22A830BE9C2D715B58CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e0bc1abd-baa2-7aa8-73cd-7bb36c548e2f@suse.com>
Date: Thu, 16 Sep 2021 10:24:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <BN9PR11MB5433D8A22A830BE9C2D715B58CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0125.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 164bf7d1-1ae9-42fa-8ed0-08d978eb6edc
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68612194C6A02DD5BDAC5ED7B3DC9@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JC98VSSIBlWlhFuZ/r1tBGMEUXIg81QisdDRhcIpCCe0xoKTXOPqxEioOLM0z7hTYOYc7ZbJzYazN3PZys9s0ybfei/R7Ws2rRCQWJP0vF5r5OAXoSBXIW1j6mlOFPdgxaoQKGYkVIZnb7NKEtOk6fLjswZgJFQJey2cpKiN6xbS/1gB7/TTkSduP76n0nx81URRtD8PGIjvmcq9nPqj7P8jHDjmntceiCCqtlaFYATUFPGE2wDElRzbbOeOJPcM2tEfaVeqymzFgEo7o9WRvMcqFUgKyt+Na0Mv2dyxgdoQXTeSkVIqxcbpK0WR7oY5lril4ZI+DW2veUdYmY4Y9ki8Z4IgzqSbLglceJGbxp8DvzsHfAx3y+pkzeTE5xIp5+upzsWtji8CZMdvc7T2v7GkVFzZqBr0OEvwl9tIIcBVl6GgDOB5jjkSni0Xn9V2dS9NfEQPwlMwYz2CYDrJGjpZpu1txOBxonGw/pssSQSMoyyb4ywBuZtn9IodTHBFCJXk0GHVYT5TZlR7X5eYILVtZdgj9bXeU4XEegz5vX0xn51oCjZ/FEDjvXdn4DVx6BAOh6c197XsKdUkx84uEeKFYdIswzKGbFVJHvFf0+4WpWSgP2Px8e7Pvd2WxjSUH9gekEAKMiBWsNgUnmOT9hMZdXfg3GNtoN+X1L4RQK4a50f//fzaJMS5R2CFZN0Ud4AJ/Jnd8TYC4rNpR1LR0J6hu8xEyEVXkv+XG8NYNNc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(366004)(346002)(376002)(136003)(54906003)(31686004)(6486002)(31696002)(2906002)(53546011)(956004)(316002)(66946007)(16576012)(6916009)(4326008)(5660300002)(66476007)(4744005)(66556008)(26005)(478600001)(36756003)(186003)(38100700002)(86362001)(2616005)(8676002)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NWxTNEM3VWx6dWlxUjFjeFp3V0NERXB3VzJvNC9PaHdRaEdkbms2RGFWMzRY?=
 =?utf-8?B?KytvY2R0UllzWFdneXRCdXRXVEtvVkIwYmdTKzhISzk1QTdWSnN3bnBBcHA1?=
 =?utf-8?B?ZFFEVEpXYnpSdXUyTmt0MUJLQWUxZXowMzhHNU5pakxHSGZ6azRScTRQbnlk?=
 =?utf-8?B?dFM2OXloTG9jZFF4T2phZTRsL2I0Zy9NSUd2WG9DbFpGZGlZZjFqaXU5Vm5G?=
 =?utf-8?B?UnV3T3dlWlBkOWtSK05lUjNiUytJcHdGazhURFRTL3dIM1FjdXp5Misvb3dG?=
 =?utf-8?B?aDFQWnRhQmNKQ2FUWVlDS1lNQTlnVVUxKzkvL1VnNmdQNHRLV0Z1NlJzcTN2?=
 =?utf-8?B?Vk9kK01tZ29vVU82YldqU1ZkZlJESGs3aDNVMHRBdmxWRU5jUDJJNVZxRzI1?=
 =?utf-8?B?cTA0bCttcE9lRUJUNnh5d0tuaG8vTTlONkljUjFnUlFFbnFyUVpaMFpjNmhh?=
 =?utf-8?B?cUtpemhnamNINlc2SGtMOWxGVld6cXFwckY3ZUR4ZU45b2NseWIyRXVUR0Nt?=
 =?utf-8?B?cVZaZG8rNDVrTFdtVnZ3VHJFVVhlaDcrNEZNTEtZQjFzdFgxbXZWTFQ0N3VH?=
 =?utf-8?B?d25Vd3VtbnYzbDAwNzBmZHNXTko2RUNjNkJXWElpY2k2aytKR2dMTUdwenRI?=
 =?utf-8?B?b0hPeDdvaE91VkZOeG1SSFdabWFVdS9zNUk1dTV2MC94b3B3a1pjaTdDeHd0?=
 =?utf-8?B?N1M0d1JXMkQxeXNBZWw1WFJzY2swRGQ5WXhOTkNtaHpDejk5eDNSTDAyRWk4?=
 =?utf-8?B?a2NjWHNaNXUyRVZCSnpoQlczSWtkbWRKMVh6SzI5LzFVQ2xYdU1OK1EzTXcv?=
 =?utf-8?B?TjM4dWJoZEZnZDNmK2JCdDRhQ0h3ZXBkWDhDcE9qWFVseTBwV253MVZqd2Ri?=
 =?utf-8?B?V3BPRVNxVGpIK0lZSVFNbGh1eFVua3NGdnBQbXV5Y1dPZnowVE9LbWZmeml1?=
 =?utf-8?B?ejVKVlU3ZDFRK3hISTF5aFNCM3o3anlkQkRqRTBiSndIWTNLcDZMVm1UMGNm?=
 =?utf-8?B?N1E3NnJPaUIxdys3R2JEamgyc0JCTXNDczU0RHZ3SWxvMXJLYlFnL0VCek5m?=
 =?utf-8?B?NEFoMGVJTStnT1BXSjY3dUs2TURFTHJ1cGNQaXRKbzRuaVcvUVBDQ1d1SWRC?=
 =?utf-8?B?b3c3WHd5UWIwdERXRjJwVmFZTGJ1NFpjVzBKZnJEcmJIK0VHNzhFZHpsOWNn?=
 =?utf-8?B?Q3poSGtiRWlaU3RvcXlTczl6dUlkSFl0K2hvOWQ0MmFMbzluZndxNCtDc2dv?=
 =?utf-8?B?Qk11bHNPRWdhZnNVaGkxK1Z3T2FLNFYreHUyMTRqLzBNa0pvTUdZdmdCWGNP?=
 =?utf-8?B?NzB2VzlURGdJOGlodjBxd29GdVZuclhJbWtJNTBGM2NVa3Z6ODh3VDVZRk00?=
 =?utf-8?B?cFFZUTZrWko1aEZyUEd5SlBuZTYrTHlhWU43YXVJWldWMGY3VldRMlVYbG5P?=
 =?utf-8?B?eFVEdFJOUmRrWDhFaWlDemk3ejhqRWdGd3RSNmxyZ0xBOVIwRUxkL1M2UGlZ?=
 =?utf-8?B?bXFnVks2WkxrNTBiSmdLMnN4aU90Rk51NEp4encyTUJKa01uOXlwUzAwQ3FC?=
 =?utf-8?B?YUNyTEJpTjVmdVJkckllRjRPVk0zbEEzc05UcDFCTzgxTElyM2dqekRnNE1i?=
 =?utf-8?B?T1pRVFlYd2lZUFFZNitrb1V1QkY0WitENVprODVMblVZc1JvN0ZaaldjeE5r?=
 =?utf-8?B?SlkwTWdlT1BuWGNVY2FOUDJoQi9vQW1hVTJndXY0NzRRdzRNd2pQQjNCVlRZ?=
 =?utf-8?Q?2aI+HX6gowW/E+0nNeCRY2+tSUtRXd3543/SQLk?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 164bf7d1-1ae9-42fa-8ed0-08d978eb6edc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 08:24:42.3281
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: p39/CKkwBrSyn8JBE+Sp8bGvKvRaY1S2hltXMifv+DNrirx47THbqtUvcMTvtgzkp6ORR1G7hBux7IBBxosdCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 16.09.2021 09:47, Tian, Kevin wrote:
>> From: Jan Beulich <jbeulich@suse.com>
>> Sent: Wednesday, September 15, 2021 8:42 PM
>>
>> Kevin,
>>
>> On 24.08.2021 16:27, Jan Beulich wrote:
>>> map_domain_page() et al never fail; no need to check their return values
>>> against NULL, and no need to carry dead printk()s.
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>
>> may I get an ack (or otherwise) on this patch please? Ideally also
>> for some other VT-d specific ones in this series? I'd really like
>> to get in at least some parts of this series, before submitting an
>> even larger v2.
> 
> I haven't found time to check the entire series in detail. Only acked
> a few ones which look obvious.

That's fine, thanks! It still allows me to move forward a little.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:31:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:31:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188241.337330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmnz-0005T5-2v; Thu, 16 Sep 2021 08:31:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188241.337330; Thu, 16 Sep 2021 08:31:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQmny-0005Sy-W7; Thu, 16 Sep 2021 08:31:38 +0000
Received: by outflank-mailman (input) for mailman id 188241;
 Thu, 16 Sep 2021 08:31:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQmnx-0005Ss-O2
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:31:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8124489a-16c8-11ec-b5b4-12813bfff9fa;
 Thu, 16 Sep 2021 08:31:36 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2057.outbound.protection.outlook.com [104.47.10.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-pNfbvh-zN7qMBvOEepz5kA-1; Thu, 16 Sep 2021 10:31:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6384.eurprd04.prod.outlook.com (2603:10a6:803:126::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 08:31:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:31:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0012.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 08:31:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8124489a-16c8-11ec-b5b4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631781095;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+KuwO8Ac+v8fcT7Ie21OEUa2zI/DBon1ywm4ELSnZqI=;
	b=kfKsJcfmmHfkXOdTRtlWC6mGZdcU6JNwMBLD2ko5t+JVtyX8EoZKiNWYhQisuJfBTq3RVC
	zskOksHDBRf1t3d9XBAooScXbXD8CJFycwhm37M2tMe31ZRdJURy5DFQDhcZLl/ozzfweu
	E/snD1k6sEUU4nAHkpz/vSJ9zOoEwNk=
X-MC-Unique: pNfbvh-zN7qMBvOEepz5kA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CA/8ZeOFQC4LIz4MvM4fMGL5Ad7MmwNfjOPLOF+gF3St7Ed41ewgyaHHfPbcjg52PeoL4OlZgKDEivLBs/cfsLKGbsBTrAsgi0/VXPWDCzFaU045vqSCqZuBCGHUOrStYXCwO0ygRgLiPifM0GKqVl1i7p03NJo6/IlO9a5nhxTw/xGKXlmH03x2CMv2DTdg6/9CaS5qs/BMlintydI5OgnDtcv6NDauLI3ZPQUchBW0P9xvrPyuoMNUhiI/o7mRKeNA54+SgR19Dv/IFwNFyrKVl9gvFbLJWLVDA4TKJYtPAJMKLYiINbMq72ONR/89EU0eR9N2R92DD/vbT7K3SQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+KuwO8Ac+v8fcT7Ie21OEUa2zI/DBon1ywm4ELSnZqI=;
 b=TlZ142/2CtffU87WlBKm01rDOhdXGiWVOMviIa7mvkuDz6TD4uoSXxlXTR4Nke3u7nqYeAA2wub8YHf9zlG4u9pPi9fngzhmJ+uuy112h3SCeGyvXkm2G98Ikd6czimhknP62YGaK79wgwMN5aJV4mkKZ/VeQKxxFRj8yxVFfEEkD1uZQHLbWugqN0Fght/MAPFOfwG6TGe7l68ms1HjmvIFJZ1Cnvp1cnp9W0c0KTFhnJcAf/iQetQwlpxEXWzcWHXUv7m54VeNpz39bosLV1xXRDzIOh0xl1a5QsLXdmk8pr/xPuMiDjpPPNG083NCRMHwFm1evbkP5MHbDZXUZQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/4] VT-d: consider hidden devices when unmapping
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Paul Durrant <paul@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <6b7fa64a-bd6b-353c-30cb-ec25b850cbed@suse.com>
 <5cf392cf-09a2-b23c-2a25-610c64f20707@suse.com>
 <BN9PR11MB5433706BE0A6C4B915D3707D8CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <030c4cf1-f361-1d36-c61f-28537a359c08@suse.com>
Date: Thu, 16 Sep 2021 10:31:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <BN9PR11MB5433706BE0A6C4B915D3707D8CDC9@BN9PR11MB5433.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0012.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0ef6a106-f59e-4e61-35dc-08d978ec630f
X-MS-TrafficTypeDiagnostic: VE1PR04MB6384:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB63842CE5F358EC33CE11ADCCB3DC9@VE1PR04MB6384.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Gwj85+MVCQWwo6/7oB1+Lm1W6kWPViFieWTKJ6DwJKAdOUVyzRrqHPOMOoplBTblJxETB7TwprAip/olHsx/thvmcOcEyHdsQgALrS9DdHfulFC7n456QSlH7aoA/rtyf2ALQAsEOFOQ1YBAXQXH7ybJVkYFZw88pM1bg9tIuj7aBMLtNFQtW1hs5VhiVDRUSnEWIDfx3YVUH1zS6Da57vwsIanVsrV/GnEsPz0pHe4zot2ZYpwObzvsPPHFhJr+wtiaQbzoziEw1KJ4L9Pdzj9xmgW8WW4FcSkyI6oR8xYdxtLgF1l72KkZ8s1c1QcaGY0TMy95VzFsSpEzO4SNykOFV8eO2obkXAaFY5USEwofHmuF4wtkmn5WPbVY0nWttukN57vX001TBZku+0oVQ4zZrwVSN9TtXNrgmtVIRnmKg3w2mnnkf1CzGpFEU15GdF96drv2wHQsOlndOA3i3aU47mMIHhkKztkju9cOM+w67UHkVmhpa/Foh/yOHtbp0KGNwCLNeZFh1BgdDIJGcD1ox99uBTYa7lvMhJOpHlhdptsUUSRZ1CiYZWoZMwzNQ786EOEekBWVmbeFNRb5Y3Ze2dF5uJIAFRgBvDQFvqQtaghkerMv0bmtj7S3h5+DlVoq1tk/Kfn3YVkrQdOvlIGV8t/2yFeYZkCSNrpGHEUR4m2dUO/RCIsRsf6IaBA8R4RNiYiah5gxfxzRn3gQWW5urUthipMcUmLd/IADnSOk9HOpVtc09A9Vivgck8uM
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(38100700002)(36756003)(54906003)(956004)(16576012)(86362001)(31686004)(8676002)(2906002)(31696002)(316002)(2616005)(5660300002)(8936002)(508600001)(26005)(186003)(66946007)(53546011)(6916009)(4744005)(66556008)(6486002)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eDdiWWlObk5NQ2FFT0V1VVgwOXB1RlI2RWZlNjArelpuZVJrWWU1ZkkrZi8x?=
 =?utf-8?B?UzYzb2NsR3I3NWo3UFhQT2xoT0ZVUytwL2FsTlB3anlqQStIODZ2MnhuSkpC?=
 =?utf-8?B?bGtmbnFTcWR2akV4RnBPK2VzRzVMUXlsWXljdXl6UG5QWUNCdUVUeURlRW9B?=
 =?utf-8?B?MjRtZVVldnZGUWRoc3RnQUZlbUlhY2tjb3IyVXZyOE9CTlFEMzZnWVUxWllZ?=
 =?utf-8?B?RExJdHRmODlPdlBkSHdkT0IxTUMram5SQ205ekMwMXZGZjRiZ01VU1NkcWRp?=
 =?utf-8?B?YmVnN0RMaG81d0JIS21ja2sra0ZNaTNDbVFRQllCTENIN1JEMjRYakNpZit1?=
 =?utf-8?B?R3NSLy8wNWpmSVdNK3ByVVVWY2kxajRDM2JIM08zZ3IxTWdkQktVNEw1THdO?=
 =?utf-8?B?Rkk1NERrWms2VTNPdEFSNTNsbnJmV1FKNVN6NER2dzlOeU5keDIyUGJBd01a?=
 =?utf-8?B?YjlXVm01a2hIbnZjSEVHeTMyNXRFRFRXMFdYenBTV0taUTVqQTZVR0NONU5n?=
 =?utf-8?B?ZnViZHZVWnhDazExRys3bzV6L2ZzdUNxLzlZQXJiZGxLUnVQZHhjQmNmd2dR?=
 =?utf-8?B?dWZpQ3IrcjQyb2JrQzRYM1hPYUU0ZjhCd055cVNheGZkVTNJYlRJZld6cEM0?=
 =?utf-8?B?eHRRT2ozY0VaeS9TZGp1STRmQitjY2FvOG0ybWxIQ3BCeU91Y3lEZXkvWFV5?=
 =?utf-8?B?OHJKYWpEbmxENmxTN29USjUrZE5LRDc2dElIRVRQbEM3MzArT25RVExDNDU5?=
 =?utf-8?B?V0hzUXFYVlM3Unp5aGhDZzdXVDZvMnZ6NmdpWEY5Z0NkWjBlaG54OUd2Y3Vp?=
 =?utf-8?B?NnpiUVNHeXBVM3FCN3lTL1cyb3FiWmN2ODJQcE5VTW56czF3Zk9TZXFKenkx?=
 =?utf-8?B?TGc5c0pHckpMbGZUTFhnMk13eE51b09oUXdRQUlQdElNMmdncG9qVVZxS0tF?=
 =?utf-8?B?N0QySlEzQmh0d1R3WHNzR1M5ZVRPa1dxUEhWQUFSTzFpN2loTkJidWhSQUFP?=
 =?utf-8?B?alRwbmViMXI5NlJmQ0xnTVVOMmsyMkppTmU4ZURNSU5pbHJJUTBTWUdaQmZY?=
 =?utf-8?B?S3h6dStjTlA1bFpTMWJmME1GUnV5ZzBJTkc5a1JGUWU4bTlmdGY0Uit1VGFD?=
 =?utf-8?B?bTNySW1HdWdRSE1TakdqK2tXVHJJbFpZbFJVZ1dvbnhVVlp5OE42V2hFR0cx?=
 =?utf-8?B?cEx4eENjYTRvMnZEK0lwMkFLWVlJOVliMUh3Z25pOC9lN00vY0xqUDFOV0xG?=
 =?utf-8?B?VExZeXpCdmJVRjVhSk9aS1FWc0I3WENMdnp5cHo2T2pGVzRRanR5TU1rRWZk?=
 =?utf-8?B?MmxQTE5VdGJ5U1VWQVFlMmp4emVoWWxpb1dzNXZValdHSjZ6S1QwUTBwTk9Q?=
 =?utf-8?B?eXBUVFQ0RXNRS0VyOWhVanpSbHE0TFFEbEsvUXlEK1JCa2dhSjJURU1QenpE?=
 =?utf-8?B?VHhudlEyY2gvNkRYMXhTZEUrYWIwUDZuVW5JZFFqSEl6SkN6Mnc5cjRqM2di?=
 =?utf-8?B?QmZkakhjUDZtaWJRVmxLZXF1VzVKamRPOTlCSkxjbjBKTU9HUjM3cmZpZFI1?=
 =?utf-8?B?V3lBOU9xSFcvWEZiSmRRY2phVzkwU1ZBUFZaZklMN2pwWHZFbFRDUGt3Qmxk?=
 =?utf-8?B?QzFDcHRJU1hidDlLd0puWm9uaFJVaE5DdnpGQWF5TVJsOGlrYVAreU84U3hp?=
 =?utf-8?B?MkRYUEF1RXBiMWFoSkFOdmZRWjVGOEV4UWFhWUZXWnMwL1Q2UGsxb1l3NElV?=
 =?utf-8?Q?v8Q9tBSAzlO4dhinn1ATaV5FHEPMfIR/rWDlQJ6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ef6a106-f59e-4e61-35dc-08d978ec630f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 08:31:32.0331
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: U0rvdgNCRx01AgzG7RhR0wYGb9bqM5eUUxs7TzwyMNhg7NVN/YVbYDmqDuCqEHDFXsLhsuqh716BcV9DS1vz0Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6384

On 16.09.2021 10:18, Tian, Kevin wrote:
>> @@ -1737,23 +1737,33 @@ static int domain_context_unmap(struct d
>>          return ret;
>>
>>      /*
>> -     * if no other devices under the same iommu owned by this domain,
>> -     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmp
>> +     * If no other devices under the same iommu owned by this domain,
>> +     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmap.
> 
> what is changed above?

A style and a spelling correction: Capital letter at the start and
a missing 'a' in domid_bitmap.

>>       */
>> -    for_each_pdev ( domain, pdev )
>> +    for ( found = false; ; domain = dom_xen )
> 
> honesty speaking I had to read several times to understand the break
> condition of this loop. It'd be easier to understand if putting for_each_
> pdev(){} into a function and then:
> 
> 	found = func(domain);
> 	if (!found && is_hardware_domain(domain))
> 		found = func(dom_xen);

Can do so, sure.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 08:46:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 08:46:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188250.337341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQn27-0007D1-Ft; Thu, 16 Sep 2021 08:46:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188250.337341; Thu, 16 Sep 2021 08:46:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQn27-0007Cu-CU; Thu, 16 Sep 2021 08:46:15 +0000
Received: by outflank-mailman (input) for mailman id 188250;
 Thu, 16 Sep 2021 08:46:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQn26-0007Ck-II
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 08:46:14 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8b5f3462-16ca-11ec-b5b5-12813bfff9fa;
 Thu, 16 Sep 2021 08:46:12 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2053.outbound.protection.outlook.com [104.47.4.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-aZ6v_N08NLGCKvICVYeHKQ-1; Thu, 16 Sep 2021 10:46:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5472.eurprd04.prod.outlook.com (2603:10a6:803:d3::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep
 2021 08:46:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 08:46:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0049.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Thu, 16 Sep 2021 08:46:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b5f3462-16ca-11ec-b5b5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631781971;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oB3hQsRN+X2NxD/nGE+17py16UYnIKt8HjInNCcduuY=;
	b=UnDTkb6qD+INOxVvhugB3aRgZqtNa9C91tY7j1JDgNX6QmRUBrxmwp1B0Y/nY7zIPxdmx+
	HeOA7P/Vv1MQVuwFA2X9RDpbqDByjQdLe4pA4YVSxBDlhcRWB49gHtv7RK0Q82qxJ87N0h
	IDlkQuMGM9E9F9OHe1xPMQyDr1IPOsY=
X-MC-Unique: aZ6v_N08NLGCKvICVYeHKQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eG9IkKy2y+RFQsttWQnVCTOqPk1xvrh6kROJIc8THYi6g0aMShYfUWq4tf55JQbQ48NsFzByzxm7C3nXHPZqcnyXGlTR88/xJ21fAgVH8DudJV4evw6bupmKWwJHLt+fYdUpe+cizlpLUU4KB56a814wi5bP2v8MPEAuf5tYShNXBoCee5EGoB0BiPTvyuLIo7LyKrvS4vOpIFvC9BY00i8K3+tEtzfcRT3GC48nO/KwKdHxbgFzRKW1lslQZtthP0M5oDhup1PveYLU+DNBaAiFx7UUe/dsVOFb4EjjmYihOBSFFZTaSFSsJuNikr7lg+WZcetC0B0IddtoWVxRig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oB3hQsRN+X2NxD/nGE+17py16UYnIKt8HjInNCcduuY=;
 b=LcTq4CDjLUmogOeYQiUmkUULtZEo5a4MOCeATzFNkh5fPeVmSpT1KQA36lJ0XK2bnuPmAGvdILpNr6byGWPBpkZfHv9i45iTLJEmod3IOWa3CsGU9blqcNSjfzh8gp7WQ+Z/hPi6xv0R//qRFirzK/qsQ/3nMhoPABxwSCTdFqbkv2wbShGsWAKfn3EyxDtkb4NHdB5uo/szpsRckCj/wwLlojE+Hm/52MUn8uTnL4wCaVPOz6toc37NtmkdLLXGevQgWHVxL6mxdUISrbWcw2QEXrUlzs6OGcddpUr3xINAtSJHnJnh1fXoCuD4SjM+Jejd0EI21P4D9Y2ne7I/EQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
Date: Thu, 16 Sep 2021 10:46:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210915142602.42862-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0049.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cb0a7f17-e919-4d84-f63c-08d978ee6de7
X-MS-TrafficTypeDiagnostic: VI1PR04MB5472:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB54726AEAE04EF4DDA0A2D361B3DC9@VI1PR04MB5472.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Pl+UIjvOEEUDkeT18lT01B+4yhrDaxU9OC6lDuilJYHSts/YEZd6udJ2r6Y0N+McCbBm27SxEqP1yRJTGDd99fuVxk/eR5dIkgIyG1UgQpgw3jmZpwp/o++AaOv7QY8ZhxpmOnRKxQSkwsQWJRVylRxCrjLqoesyoU1Quzn30IGm1EPlNntoqOaiGy1Vmxa3TWS3Cwkxs3pC0ccgyb8RI0m9kmq9Buw4CWTMkHjKLi7gopDN6a9/Txnf8OqDCiHQPCsHX72IlDxINCu3Q0eZM2kMbus+CB/u2SrrgNj6qHMUhNDKOKLRt3Aw/wu7lGcor2aJ6mIMdVVIiAj+0Xro+A7CBta136csGr2K1D8xlyIBGYVAbA5VcDxLnjAHO9ukhT8cWwOwJ4cRdp23c8WbQin+Bi+wckhEoi7jChJFCJJalGLfJfLlcYrwnEH1AXIJREh0WzesVxvENJlV7POizi0sY9cDKsw+TzeiW/vfz0M2yDhpqcH9Xa/oiG5GwHTZhWVVfgsjKE3jVi9xeRQdc8iggVsYOD4bU2lpP8x8/23XEhvs/Lq9QqS9i4I++yC5lVJQtd++ZZ6t/WDuohvrXDJn40gQoTGNqULxZKprF4r0BhLuNvRtAk7Ba81ejHJ2neS5eLcetJXYyCZvELMoWc+0jbxtNbtVsL/qDEhw2oJZxIZyzMltNFlYjSNP7wfC8ArdNd0M8igsiPJ+YMZE/dMqPjKyAU/zeHu8snozkuynNbLBjmfC7d8zWGjOqlZukVg5eLMezUtHoXqaOtRB/g==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(346002)(396003)(376002)(2616005)(956004)(86362001)(2906002)(186003)(8676002)(31696002)(66556008)(478600001)(5660300002)(31686004)(66476007)(54906003)(36756003)(16576012)(66946007)(26005)(6486002)(8936002)(7416002)(6916009)(53546011)(4326008)(38100700002)(316002)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Q1c2VmowNkJ0V3Y0N3FBUFpENm53NXVGVW1CRkpSU1JpWUFVMi9QMmFFM1VW?=
 =?utf-8?B?aHU3M3BuUE52Ui9UbElYOWMwK3c5b0hKcTdCSFJnM2tKT3FmOGptd0VBSFIz?=
 =?utf-8?B?V05NR0RJbVFrK0lTWmNBSEpROVhFamRXOXdWdENZOHFVbWZyZWpxUllPVHBE?=
 =?utf-8?B?ZGtTRWsxL3FsYVhDeFVQYUxFdUczVFNTQzAwODdabVJzRlhVNFllQjFhanNF?=
 =?utf-8?B?dHdxZldnek5MSllRUnY3dDV0TjRwUy9DVFg0YU5VOGtHRzdVUWZMUDdubVVD?=
 =?utf-8?B?ZXhKREhNRlplZlFQUGg2V3VBOXM0Z1F4K0ZxT0hsZ0lCNnZTSHZjQlkrTStm?=
 =?utf-8?B?R3lOQUVkWmNNL2w3UzNpRnR1bHpXWUlTL2gxak1DQU1lWml3NVhwVjNaTG5i?=
 =?utf-8?B?UmEzL05jMGdjYnhoa01yQnhYR1BDWEM2TjJPL21yRFZoQ1M0cER6MTJXMHVH?=
 =?utf-8?B?M2E4TVQvTU1HdW0vS05GUXE3VVoxWk9ZNzJKNUdCaG1Zc3JucVNKZXlTcHBm?=
 =?utf-8?B?SE1HSkxpOFRUZG9vbWRSaDE3NWY1ZldnWURMZytCS0hXZlAvaktxdUNLQ0V0?=
 =?utf-8?B?TzNFdUdwZlE1SVU0c0J2YWFuWm81TjBlRU5qdTM2dm40eUYya083Y1VRb1Nw?=
 =?utf-8?B?cUFRQ1NPWXFaa2FFTUhPYlk1MHo0d3FRaWtiL08xbTFFR2Q4VFZuOTZVcTA0?=
 =?utf-8?B?U1ZjZWVXRGZwV1NhMkEwOUtTZlNZN3lKeTUyODlsdEJzVFpTZzdjdXJQb0dY?=
 =?utf-8?B?TkVuVldaRzlGOHcvUU0zcnBaZEFkRWh4Z041UTR1MEFuelc0Vlhqd3Y2WnYy?=
 =?utf-8?B?OWZvUXhZRzFFVnNEZUE5TThmekpzTm5nN0VJT01oL1ZWTGZJeE5DekxUbzlL?=
 =?utf-8?B?YTBzb1Y3OE9weldPUFRUZkNCK2J3clppakt2MWRESHlIVkhpOHRwNXkvaDVW?=
 =?utf-8?B?S0JZMzYwUGRQeXhYT21ZT3lPSHNtaWdlOWlDdUJDK2liYktTYU5zN0N0YWpG?=
 =?utf-8?B?STUrcHpodnFpeC8zMGl3UkNwakEraWhITE9nTGZpZTF5WHl2SW5XOGlnSE1i?=
 =?utf-8?B?WmFSVDBCc0dBQ2hKKzZMNEpkclVhRTNhQmNiUnkvd3lvWkc3TVVCaVdGN0xJ?=
 =?utf-8?B?TEtZU0xSTGdndlhVaEo1OHhMZUdobnRKZjNTT0ZNSytBV1BkTmJrSEdrdzVG?=
 =?utf-8?B?cTJJQjVTeWM4RVJZUm1Td3dUNFNMbytDdmNYUGI2WUwwTVNUbXpzKzBQR0ln?=
 =?utf-8?B?UGNvVnVZMmg2RkNsR0xGTXhOUWUvbEdTMy94TTF2TEd1RGZwdmEvRXNNaEhk?=
 =?utf-8?B?c3hBV204N1kxNEp3ajN4dlhja2dQR1lVMjhqS0IwUlRWOVkyY3FLaERod3px?=
 =?utf-8?B?dFlMZjdZZExjelF6NmFVVnZXQnRYdnNvNUNXY0M1UUl3S3J5UDNkSk0zMmhy?=
 =?utf-8?B?TXovM082dm5vM2ptcEVWbERhMFZjTjdsY3V1anUyNUp1OHMzbldTOVBSRnBy?=
 =?utf-8?B?Z3l5OHo3RTRmWW4ySmVjdk9wMzlQU2R0aTFaRGxCTG9OQkJzZHRjQ1hvWGI5?=
 =?utf-8?B?Q2xhVFh1QlZuVjBYN3YxZzBPbFJBOE5GeG1ZOXNLS09xeUZ0T3FRY0pyRTU5?=
 =?utf-8?B?dG92L0xaVEw4MUVmc25pdjVrRG5YOUpFMVd2Qk1POENCZEFXN2FlWWo0Qy82?=
 =?utf-8?B?S3pCYmFFVkVqUEVZTUVPbGpjUWNHMUhxbUc0djNMSit3OS9naFI1Rlgvc0FT?=
 =?utf-8?Q?IosH3V2S6U/T63q1wiCz6dOm8QEbEtdMM5opRAb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cb0a7f17-e919-4d84-f63c-08d978ee6de7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 08:46:09.2083
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6VjkD3ITSS9RCMYw7H+T2HSeFTladV8GuZ/Ow1BS6YQAdFIbHvAEFjH0jnzkRDb8SrM4Kh7qKr2FkoRv6X2pqw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5472

A number of nits, sorry:

On 15.09.2021 16:26, Luca Fancellu wrote:
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,39 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char* name;

Misplaced *.

> +    int name_len;

Surely this can't go negative? (Same issue elsewhere.)

> +} dom0less_module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
> +static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MODULES];
> +static uint32_t __initdata dom0less_modules_available = MAX_DOM0LESS_MODULES;
> +static uint32_t __initdata dom0less_modules_idx = 0;

Please see ./CODING_STYLE for your (ab)use of uint32_t here and
elsewhere.

> +#define ERROR_DOM0LESS_FILE_NOT_FOUND -1

Macros expanding to more than a single token should be suitably
parenthesized at least when the expression can possibly be mistaken
precedence wise (i.e. array[n] is in principle fine without
parentheses, because the meaning won't change no matter how it's
used in an expression).

>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int __init get_dom0less_file_index(const char* name, int name_len);
> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                              const char* name, int name_len);
> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
> +                                               int module_node_offset,
> +                                               int reg_addr_cells,
> +                                               int reg_size_cells);
> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                               int domain_node,
> +                                               int addr_cells,
> +                                               int size_cells);
> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);

There are attributes (e.g. __must_check) which belong on the
declarations. __init, however, belongs on the definitions.

> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    bool dom0less_found = false;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +#ifdef CONFIG_ARM
> +        /* dom0less feature is supported only on ARM */
> +        dom0less_found = check_dom0less_efi_boot(dir_handle);
> +#endif
> +
> +        if ( !name.s && !dom0less_found )

Here you (properly ) use !name.s,

> +            blexit(L"No Dom0 kernel image specified.");
> +
> +        if ( name.s != NULL )

Here you then mean to omit the "!= NULL" for consistency and brevity.

> +            option_str = split_string(name.s);
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_str)) &&

Stray parentheses.

> +             (name.s != NULL) )

See above.

I also don't think this logic is quite right: If you're dom0less,
why would you want to look for an embedded Dom0 kernel image?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 09:11:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 09:11:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188266.337365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnQT-0002ew-Qf; Thu, 16 Sep 2021 09:11:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188266.337365; Thu, 16 Sep 2021 09:11:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnQT-0002ep-Mi; Thu, 16 Sep 2021 09:11:25 +0000
Received: by outflank-mailman (input) for mailman id 188266;
 Thu, 16 Sep 2021 09:09:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZTNT=OG=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mQnOh-0001q1-6d
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 09:09:35 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dc95ff71-0a91-46a1-986e-218db9df7375;
 Thu, 16 Sep 2021 09:09:34 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0125F60F93;
 Thu, 16 Sep 2021 09:09:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc95ff71-0a91-46a1-986e-218db9df7375
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631783373;
	bh=aVfsrAoPBPm3coDtX7qouVYko5lvAFm1LHfu7NPpcGA=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=MneUb0wyDDhB+jVjJDBgKLy9ijrzyLgrNSegUd6/ufKx2wmAO77YL8v1W7Ik0xdcu
	 4kcQiXhxthJmd4+/P0YKZRygro6d9sojkrXTR8Gaja+JSCcb3r14aRIlhK6xhphcfK
	 qhbs0IdIZmoR9gBA+I2tNMHb6bCZc6MqwwZeuqcolYjGaNZxL3uWnXOZagZWVh9sns
	 U3PWs9FQpVTaifKmtsAkRDOKLj7KOgFlYDvJqD3T+JUYBxhB+SZ/zBs7/5KbHCVe3I
	 ahEtu6Hs6WXeX5OzurOobc3oH1U32Gfu1KZxQ+MkTp5e3qXuV7eZ2mmeOeooyBfxIA
	 CyUVnQXowSVKA==
Date: Thu, 16 Sep 2021 12:09:27 +0300
From: Mike Rapoport <rppt@kernel.org>
To: Borislav Petkov <bp@alien8.de>
Cc: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
	Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>,
	xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUMJx2ckLlWKi3VF@kernel.org>
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
 <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>
 <YUHSRKubsGT2Jvur@zn.tnic>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <YUHSRKubsGT2Jvur@zn.tnic>

On Wed, Sep 15, 2021 at 01:00:20PM +0200, Borislav Petkov wrote:
> You forgot to Cc Mike, lemme add him.
> 
> And drop stable@ too.
> 
> On Tue, Sep 14, 2021 at 01:06:22PM +0200, Juergen Gross wrote:
> > On 14.09.21 12:03, Jan Beulich wrote:
> > > On 14.09.2021 11:41, Juergen Gross wrote:
> > > > Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> > > > reservations") introduced early_reserve_memory() to do all needed
> > > > initial memblock_reserve() calls in one function. Unfortunately the
> > > > call of early_reserve_memory() is done too late for Xen dom0, as in
> > > > some cases a Xen hook called by e820__memory_setup() will need those
> > > > memory reservations to have happened already.
> > > > 
> > > > Move the call of early_reserve_memory() to the beginning of
> > > > setup_arch() in order to avoid such problems.
> > > > 
> > > > Cc: stable@vger.kernel.org
> > > > Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
> > > > Reported-by: Marek Marczykowski-Grecki <marmarek@invisiblethingslab.com>
> > > > Signed-off-by: Juergen Gross <jgross@suse.com>
> > > > ---
> > > >   arch/x86/kernel/setup.c | 24 ++++++++++++------------
> > > >   1 file changed, 12 insertions(+), 12 deletions(-)
> > > > 
> > > > diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> > > > index 79f164141116..f369c51ec580 100644
> > > > --- a/arch/x86/kernel/setup.c
> > > > +++ b/arch/x86/kernel/setup.c
> > > > @@ -757,6 +757,18 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
> > > >   void __init setup_arch(char **cmdline_p)
> > > >   {
> > > > +	/*
> > > > +	 * Do some memory reservations *before* memory is added to
> > > > +	 * memblock, so memblock allocations won't overwrite it.
> > > > +	 * Do it after early param, so we could get (unlikely) panic from
> > > > +	 * serial.
> > > 
> > > Hmm, this part of the comment is not only stale now, but gets actively
> > > undermined. No idea how likely such a panic() would be, and hence how
> > > relevant it is to retain this particular property.
> > 
> > Ah, right.
> > 
> > The alternative would be to split it up again. Let's let the x86
> > maintainers decide which way is the better one.

I think the first sentence about reserving memory before memblock
allocations are possible is important and I think we should keep it.

With that

Acked-by: Mike Rapoport <rppt@linux.ibm.com>

> > 
> > 
> > Juergen
> > 
> > > 
> > > Jan
> > > 
> > > > +	 * After this point everything still needed from the boot loader or
> > > > +	 * firmware or kernel text should be early reserved or marked not
> > > > +	 * RAM in e820. All other memory is free game.
> > > > +	 */
> > > > +	early_reserve_memory();
> > > > +
> > > >   #ifdef CONFIG_X86_32
> > > >   	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
> > > > @@ -876,18 +888,6 @@ void __init setup_arch(char **cmdline_p)
> > > >   	parse_early_param();
> > > > -	/*
> > > > -	 * Do some memory reservations *before* memory is added to
> > > > -	 * memblock, so memblock allocations won't overwrite it.
> > > > -	 * Do it after early param, so we could get (unlikely) panic from
> > > > -	 * serial.
> > > > -	 *
> > > > -	 * After this point everything still needed from the boot loader or
> > > > -	 * firmware or kernel text should be early reserved or marked not
> > > > -	 * RAM in e820. All other memory is free game.
> > > > -	 */
> > > > -	early_reserve_memory();
> > > > -
> > > >   #ifdef CONFIG_MEMORY_HOTPLUG
> > > >   	/*
> > > >   	 * Memory used by the kernel cannot be hot-removed because Linux
> > > > 
> > > 
> > 
> 
> 
> 
> 
> 
> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 09:35:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 09:35:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188283.337400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnn7-0005d1-3y; Thu, 16 Sep 2021 09:34:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188283.337400; Thu, 16 Sep 2021 09:34:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnn7-0005cu-0v; Thu, 16 Sep 2021 09:34:49 +0000
Received: by outflank-mailman (input) for mailman id 188283;
 Thu, 16 Sep 2021 09:34:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=20iF=OG=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mQnn4-0005co-S4
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 09:34:47 +0000
Received: from forward102o.mail.yandex.net (unknown [37.140.190.182])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8cfc301c-fe41-4a0e-88ae-4e046f7aaba5;
 Thu, 16 Sep 2021 09:34:43 +0000 (UTC)
Received: from myt4-5dd3963c4bb1.qloud-c.yandex.net
 (myt4-5dd3963c4bb1.qloud-c.yandex.net
 [IPv6:2a02:6b8:c12:525:0:640:5dd3:963c])
 by forward102o.mail.yandex.net (Yandex) with ESMTP id 085ED6FFAE38;
 Thu, 16 Sep 2021 12:34:21 +0300 (MSK)
Received: from myt6-016ca1315a73.qloud-c.yandex.net
 (myt6-016ca1315a73.qloud-c.yandex.net [2a02:6b8:c12:4e0e:0:640:16c:a131])
 by myt4-5dd3963c4bb1.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 EBAyEc8Chw-YKE8rcCd; Thu, 16 Sep 2021 12:34:20 +0300
Received: by myt6-016ca1315a73.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id LLaIuIUSvU-YINeeR6k; Thu, 16 Sep 2021 12:34:18 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8cfc301c-fe41-4a0e-88ae-4e046f7aaba5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631784860;
	bh=C58Nyuu69+QXNydcYAmmqxNxFseKKbI9nuzRv/qU388=;
	h=Message-Id:Date:Subject:To:From:Cc;
	b=hosA1EL9TtCwvILAPgrnm0pzj3hRubo4yXDkY0oYCpJFT7x3pFvzQZug4p/IlnOxL
	 2tysvuFRPovk0BYu8R0YXdWwXbEl8Ft12ZHDhqjN5KJgr8wuyiVZwiBSN0DZmCcK+1
	 MSwF32FmkXYsiqiU10ybfBIiXcoRoqqgPkwbGBkM=
Authentication-Results: myt4-5dd3963c4bb1.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
To: xen-devel@lists.xenproject.org
Cc: Dmitry Isaykin <isaikin-dmitry@yandex.ru>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v2] tools/xl: fix autoballoon regex
Date: Thu, 16 Sep 2021 12:34:14 +0300
Message-Id: <6be4aba579bda042bef4cf0991a92444d0c3f05e.1631781599.git.isaikin-dmitry@yandex.ru>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>

This regex is used for auto-balloon mode detection based on Xen command line.

The case of specifying a negative size was handled incorrectly.
>From misc/xen-command-line documentation:

    dom0_mem (x86)
    = List of ( min:<sz> | max:<sz> | <sz> )

    If a size is positive, it represents an absolute value.
    If a size is negative, it is subtracted from the total available memory.

Signed-off-by: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
---
Changes in v2:
- add missing Signed-off-by tag
---
 tools/xl/xl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..a9f7e769fd 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -81,7 +81,7 @@ static int auto_autoballoon(void)
         return 1; /* default to on */
 
     ret = regcomp(&regex,
-                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
+                  "(^| )dom0_mem=((|min:|max:)-?[0-9]+[bBkKmMgG]?,?)+($| )",
                   REG_NOSUB | REG_EXTENDED);
     if (ret)
         return 1;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 09:40:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 09:40:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188293.337410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnso-0007Bk-Pq; Thu, 16 Sep 2021 09:40:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188293.337410; Thu, 16 Sep 2021 09:40:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnso-0007Bd-Mo; Thu, 16 Sep 2021 09:40:42 +0000
Received: by outflank-mailman (input) for mailman id 188293;
 Thu, 16 Sep 2021 09:40:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B7pY=OG=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mQnsn-0007BX-MT
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 09:40:41 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e4ddc122-e7c0-4946-9967-26855549a5ed;
 Thu, 16 Sep 2021 09:40:40 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id x27so16114123lfu.5
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 02:40:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4ddc122-e7c0-4946-9967-26855549a5ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=f4r5L82T8lVgmjDxktA/sisuupFmx7x30+6IlhMnYag=;
        b=jPcF0P+cHfyIuFllT9s+XQDbpTKL/DP/WZ12xBmDnC0LZJsbJeGpydQ+nj1jG3vyhQ
         MXkF+s9f9DZe82C3OUmuCqssXSW6Xy1D8PFT7XMOVkyU74aEXJL7HBQJL//WLAcHnHlT
         6JAWY0XBjZ7XwV9xpdiHpNmVxyHHj5cWcQPRN6QHBZx1dqJ5C5ttXeA+io6IcnyWEI1q
         dTKH6A8fDq2gQ2GNoUTTZGZkVAhST9ZUJ+pdKy05GPpnzXG3EkGwi93ZGAuCS4fkFkql
         PEeqVQZUULvnCU7rK1P4+9aDwbpW2w/MkOtNecCIBYtcqqkOy4RB/4LhCrlLfobSeeON
         wc2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=f4r5L82T8lVgmjDxktA/sisuupFmx7x30+6IlhMnYag=;
        b=XKBJwoUbQgctMShUBU/8BCVmJNcpQWq5kMQj2VItmGyJv/3SAhV5ZXEXWLtGfcVf6p
         Xf15CldBuZrmBhHQIVUA+FlleMF6pVbEmt2LYsmHZB8uNmES2tAexMYsxxFcsSycNShU
         V3W7PNz4Lo7PY7YVH74NGLhhanht2POUUqRCqcGIjTGUmsDv+eAAwTU9gFmABOTc/waA
         WnV3G5gP3X53YNArxWGvtIBidKSqsQKnzN+qEa2Hfu7tM755pYCxUftjnf6z9Pzvb41u
         7nSMGZwNTKsTFCzC2oFfWJFXf2+Id6RUgEDS10lAcDjDidiIVEBfQefyNJvx5cZBl+q+
         SjoQ==
X-Gm-Message-State: AOAM5334m7RIYrh4qFNzQb05mk235jSzeVwAHhy+5e+k58ef3Il1SgO3
	+cHa0lPQdAeyTvYIBlArPsVG1e87NUSm8KgpYu8=
X-Google-Smtp-Source: ABdhPJyIIyf4A7fhfDx7l6dTfpk1Ec/zzEqy4u9mdnznrZCiaqfyA9V5XQj08EEuePeq9U7P318zvMYzkOuwjxToNGM=
X-Received: by 2002:a2e:1508:: with SMTP id s8mr3587923ljd.240.1631785239506;
 Thu, 16 Sep 2021 02:40:39 -0700 (PDT)
MIME-Version: 1.0
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <20210914153046.GB815@lst.de> <alpine.DEB.2.21.2109141838290.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109141838290.21985@sstabellini-ThinkPad-T480s>
From: Roman Skakun <rm.skakun@gmail.com>
Date: Thu, 16 Sep 2021 12:40:28 +0300
Message-ID: <CADu_u-OjpYB0-B=tLKYsZH=auvQF-o7PJb-11W=1emY8jaA-mg@mail.gmail.com>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, Andrii Anisov <andrii_anisov@epam.com>, 
	Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>, 
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>, Michael Ellerman <mpe@ellerman.id.au>, 
	Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
	Juergen Gross <jgross@suse.com>, Marek Szyprowski <m.szyprowski@samsung.com>, 
	Robin Murphy <robin.murphy@arm.com>, "Paul E. McKenney" <paulmck@kernel.org>, 
	Randy Dunlap <rdunlap@infradead.org>, Andrew Morton <akpm@linux-foundation.org>, 
	Thomas Gleixner <tglx@linutronix.de>, "Maciej W. Rozycki" <macro@orcam.me.uk>, 
	Muchun Song <songmuchun@bytedance.com>, Viresh Kumar <viresh.kumar@linaro.org>, 
	Peter Zijlstra <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>, 
	Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>, xen-devel@lists.xenproject.org, 
	linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, 
	linux-doc@vger.kernel.org, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>, 
	Christoph Hellwig <hch@lst.de>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Stefano,

> Also, Option 1 listed in the webpage seems to be a lot better. Any
> reason you can't do that? Because that option both solves the problem
> and increases performance.

Yes, Option 1 is probably more efficient.
But I use another platform under Xen without DMA adjustment functionality.
I pinned this webpage only for example to describe the similar problem I ha=
d.

Cheers,
Roman

=D1=81=D1=80, 15 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 04:51, Stefa=
no Stabellini <sstabellini@kernel.org>:

>
> On Tue, 14 Sep 2021, Christoph Hellwig wrote:
> > On Tue, Sep 14, 2021 at 05:29:07PM +0200, Jan Beulich wrote:
> > > I'm not convinced the swiotlb use describe there falls under "intende=
d
> > > use" - mapping a 1280x720 framebuffer in a single chunk? (As an aside=
,
> > > the bottom of this page is also confusing, as following "Then we can
> > > confirm the modified swiotlb size in the boot log:" there is a log
> > > fragment showing the same original size of 64Mb.
> >
> > It doesn't.  We also do not add hacks to the kernel for whacky out
> > of tree modules.
>
> Also, Option 1 listed in the webpage seems to be a lot better. Any
> reason you can't do that? Because that option both solves the problem
> and increases performance.



--
Best Regards, Roman.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 09:45:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 09:45:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188302.337422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnx6-0007ug-Fr; Thu, 16 Sep 2021 09:45:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188302.337422; Thu, 16 Sep 2021 09:45:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQnx6-0007uZ-Cf; Thu, 16 Sep 2021 09:45:08 +0000
Received: by outflank-mailman (input) for mailman id 188302;
 Thu, 16 Sep 2021 09:45:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQnx4-0007uP-Ky; Thu, 16 Sep 2021 09:45:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQnx3-000357-Aw; Thu, 16 Sep 2021 09:45:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQnx3-00060m-2T; Thu, 16 Sep 2021 09:45:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQnx3-0002of-1x; Thu, 16 Sep 2021 09:45:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FXcA7Qsa+Rx6b0TV5westgTFQYYmW8QdnlTVX8+XnN4=; b=3RTtTflD2Ld34vU5aiJVFIFGFZ
	n2AlCjn0gCP+etGjqxpMVdIBqaB357FFv4L3woBM/V99Szh99vhqasLkVhZJoUvAZpC3jHDUxaHpv
	xjDLVAN+rN+RsVOQwL8FljAskGqpRvLIudu1vuqsL650mXjEmR2tafAMTiAqtO/p43sA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165000-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165000: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-stop:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=7b7ab2d6c99987e46aa53478798a05fcaf02226e
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 09:45:05 +0000

flight 165000 qemu-mainline real [real]
flight 165006 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165000/
http://logs.test-lab.xenproject.org/osstest/logs/165006/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     17 guest-stop               fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                7b7ab2d6c99987e46aa53478798a05fcaf02226e
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    4 days
Failing since        164967  2021-09-13 13:06:52 Z    2 days    6 attempts
Testing same since   165000  2021-09-15 20:38:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jose R. Ziviani <jziviani@suse.de>
  Lara Lazier <laramglazier@gmail.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1863 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:07:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:07:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188312.337436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQoIF-0002BI-AC; Thu, 16 Sep 2021 10:06:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188312.337436; Thu, 16 Sep 2021 10:06:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQoIF-0002BB-7F; Thu, 16 Sep 2021 10:06:59 +0000
Received: by outflank-mailman (input) for mailman id 188312;
 Thu, 16 Sep 2021 10:06:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQoIE-0002B1-BH; Thu, 16 Sep 2021 10:06:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQoIE-0003VW-5X; Thu, 16 Sep 2021 10:06:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQoID-0006m7-Pj; Thu, 16 Sep 2021 10:06:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQoID-0000CD-PF; Thu, 16 Sep 2021 10:06:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=b3Gq8QdDry0gpUiJlxPsRXuG8OQEvRl2YVYnI7hpYrQ=; b=bjg5M3HufKUUw4g6lDvFa/+h9g
	jKhUD3AToZKkQXNvKQLyK4Km6pgTm1ilONhapJlJeLK5eKTZDgNjkZA57jn6DKDUWSOlH09QSjujq
	8bVkJUgCJu8+a3WED/8E0kAK+2LFTv21yGg+6QuzM1P6COICnTrANdriOVpTQqVEWYjQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165003-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165003: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ff1ffd71d5f0612cf194f5705c671d6b64bf5f91
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 10:06:57 +0000

flight 165003 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165003/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                ff1ffd71d5f0612cf194f5705c671d6b64bf5f91
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  411 days
Failing since        152366  2020-08-01 20:49:34 Z  410 days  715 attempts
Testing same since   165003  2021-09-16 03:41:32 Z    0 days    1 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2256240 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:28:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:28:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188324.337449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQod1-00053A-7i; Thu, 16 Sep 2021 10:28:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188324.337449; Thu, 16 Sep 2021 10:28:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQod1-000533-4h; Thu, 16 Sep 2021 10:28:27 +0000
Received: by outflank-mailman (input) for mailman id 188324;
 Thu, 16 Sep 2021 10:28:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQocz-00052x-Qe
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:28:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d26569c2-16d8-11ec-b5c1-12813bfff9fa;
 Thu, 16 Sep 2021 10:28:24 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2059.outbound.protection.outlook.com [104.47.10.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-nAAPNTtrN1--AOH4Q7NWbA-1; Thu, 16 Sep 2021 12:28:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7375.eurprd04.prod.outlook.com (2603:10a6:800:1a8::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 16 Sep
 2021 10:28:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 10:28:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0225.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 10:28:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d26569c2-16d8-11ec-b5c1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631788103;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NMwn1Kd0VYtqHe8kwd7LAUWfk8cZ+xmjC6aX3kutlhw=;
	b=XzJ8PTBinUb3AczQvfq3A5hHAvQLK4m0YXK3nYrh7uD5Z2AvmZ3DMUZNcaA4kZqORZ+T3b
	G2KpCuz6r2dxWSHKA0gsw0AnRei0C8pyi3gBtRVtLSqIg8wg20tDn+nlYLz97OP0asKNen
	1rYt48q/oegfqGvUlvbg1DOdRBIXyLU=
X-MC-Unique: nAAPNTtrN1--AOH4Q7NWbA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Fg6qtAHj6KOqCUBcnKQMOjQPFHbv3PdPqWv1ha6naX8A0R9WR+/JKodOBI3JS2lVvPqHDObNYoFlJJQXv8IOMFYs5ANYaKHwl8iKvAvH0MePZocHBlBAwqH3IHZzkNyqDSSMod/YJ5kYUmMCBOebJ90obqF8h/4gMcwpng/RDf2fU9A2Xlri3FoF8/IfuU7Ztwc0WgeFX0TzCgx3FrvQHOEqH1uMGnbMMX2YhcxfegZDpWOY0saKly19utMXCYu4piCcvHBIDgrMSFlu9sjZjPbmvKIyGmocbR02STSm4PFz2aZk3chznR4OggYQF0TbHAZ1hCMB+X6oxHgv1DcGeg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NMwn1Kd0VYtqHe8kwd7LAUWfk8cZ+xmjC6aX3kutlhw=;
 b=n+fg5mkvQb8ZVWS3xULOFK5DqTBFavMuEHZw+kk0NmdB/9pkBhS6uJ3t2vLdjyre0rs+Wj9czHngLeBYRG+XmeYPbW8cZUSnv7sqKe+cOU+ZoRh9OgSR5N1nlpieaH/xgXmsC5Z+wk41rBTDDj6V90uN1Z2DlhQEpokrVrZnX1OugDik8S+wJXh2LAGCGy0d73qbxUpQ9YpBIr4msQ7hoyoS+v+MuKQNg/vGcsvFlUwZdpv4qcWJzUpkTGmP471JhqfA8VmoaWAsyxUDRamqg9IBFvYO8hmpRVeVLudVBaa8TEIdi1NtARkG/w5JbUBC1d18+E3Iu4xYRabFPid/9w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] tools/xl: fix autoballoon regex
To: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org
References: <6be4aba579bda042bef4cf0991a92444d0c3f05e.1631781599.git.isaikin-dmitry@yandex.ru>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8cfdc5f0-9139-c474-c60a-2495d4069cf7@suse.com>
Date: Thu, 16 Sep 2021 12:28:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6be4aba579bda042bef4cf0991a92444d0c3f05e.1631781599.git.isaikin-dmitry@yandex.ru>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0225.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ea049a5d-c4cd-4383-defc-08d978fcb4e3
X-MS-TrafficTypeDiagnostic: VE1PR04MB7375:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB7375DC62E7E0932437E640EDB3DC9@VE1PR04MB7375.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kjGvSW5JruD9QH6b0oS0kYwVxMwIdS4TL3NHSNuirUiBBrjO+ToneF9pFj04b3q3AmBTTTOOLrEF2ZApdqGbN/9hD6Cpgm0fk+ro+Ae/6SAT5CAucVu+9Xm3B5Wiy8QzEyGMuJi1NTkFhETCHJboH2+1UjLwzhOZ6hC6eh7blgmZ1Y92rPLgYleLi4nPuz5daALBbA20rO4185mpLzmZ+zwG1BcvDV/LiwwEhm/dBzs1V1luOGTcPw0nmU3ugTbC5dQigl8IoU8ek/CsGZJKv/dh6hilv5Qky2tPkI8huML9pU4FzxXqthkVMkIfsxY7SiZE8elgxahzTPYQrEuAeHxGr+fQ2FVDoAu2l11ucG5B8whzWDIonVfUQjDIz6be9exoB9tkUynvvvZ4P5Vlp/hSZSGZ86ttsJQpyGXFMReAeinsWiK0dSDMmVpDO7HX3AuKWDM0dXaRZjAKXvY+dTGEfSpU6N62jOa6Hek2OvY61BXRh5bmWU0qR9jNUPJ3ew4hAF3P+edeUKCvpFX+sZtPkg8Kshg4pdczrw6ogbfRKZuaBoLkQatpp9jyo6O7bvIIBu/o1rDcmgGfNT8s4tvN4hk+9xquhfErXpftpwrgBW9UWdCxeXJNZdtqCjyx7JM62QXxKBptTlN0RAgU+mLOyB1R1dFGK6mFFJ2ibJLIcLUl+1kmqSlK+VbTnwA79eI4JF2Z/I7q6F0f+3PzK4EwVdS5s7UdWmUtE5C36xs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66476007)(6486002)(66556008)(2616005)(86362001)(31696002)(2906002)(66946007)(4326008)(956004)(8676002)(5660300002)(186003)(36756003)(54906003)(26005)(38100700002)(53546011)(8936002)(316002)(6916009)(4744005)(31686004)(508600001)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SHNMRUxMRG5ESXFBRDNON1RvcWo3R0dLMDhyRGlnSEpwU2lJaEtxdU1JWUlt?=
 =?utf-8?B?V0xrQWQrQythaTBFSlVpRWtZWk5VY2J6Y3NpNmw5QUpCeXM5K0FkZXNTaytH?=
 =?utf-8?B?NFJmMytNRUduellGZEpBSWxETzRxd1dwdCtWcVlHN2ZzTWFzY0EwbHBqbmtZ?=
 =?utf-8?B?YVhPM1lGQ3ZQZjdIVHpRM3JlcFpNT3I3ZFBhbHIxQjJYaHd6M3RpeVdCdllD?=
 =?utf-8?B?cis4VE43SE1qeUxsV2VWQzNySmZyaldQWStWNE1hdHRvaUJUWW40REVaS3Bi?=
 =?utf-8?B?aWlWLy8rNTNJb0hTUk5iRnhyWkppd2pBb1hXeGVtckVsQ0I3T2pwcE8waFli?=
 =?utf-8?B?SkZlRm1TREtMYXhIZXNaUHdvSzVlQnJZNlJDZjB5cFdremNuODg3OEZUYlgr?=
 =?utf-8?B?V2tVbzVEQ0JjaHRNallEbmlJYmpLTnNMZnlmK0V6T01Qbk8vWFNNMDdvOXVt?=
 =?utf-8?B?SWxHZk92Q2FmWE16OTVLWXd6TSthZWc1Sm9Qck42QVJXbWw3NkVZYTlocDk0?=
 =?utf-8?B?d3BDdFlaeCtKRFJ6Y29tUE5EdllEM0pTTHFFOUFSL2FTTDZFanl5QWZTWGYz?=
 =?utf-8?B?R2MrVDI5QVYyb0NQaFowcjNyY2puaW5wQU9hYWdlQXQ0eUhoN0NtYXlYVGdi?=
 =?utf-8?B?T3dGQ1RJbDZZdmNrUW1HZmtWaDZNZ0xHbzAxendrei9vUzlQN2drZTdVQnNB?=
 =?utf-8?B?dWpzVktEMjdnTEtqa2dCeVFQMGtIR3dkZGpSUlA0TVJWemdZL3ZkMmhtMUpE?=
 =?utf-8?B?ZFZOcjI4K1FjcUc1V1VOSzV0ZzZuQm96NWJwSzU5TlQrQ0ZtMjVzVlJiUUpy?=
 =?utf-8?B?L1g0ZEkrUlB2RjE5NU5zcWlLR2l6N29mbXVQMnJyMHNhUldFNVBTMTA1clFl?=
 =?utf-8?B?NjMwaklpV3hPV3ZZL093YXdzMHh1MUw3dkFHVWwrRUhiRnBNblVPbSthUEcv?=
 =?utf-8?B?SFZJYnBkb0ZCVFJoMFNyWHcxY05TdGIwNWdiK21qVzluWWFta1IzUWxjTDRM?=
 =?utf-8?B?eDlWa0hNOUx3ZU1Qa1VVSDVRV0E5MTUxcStKUkF3MnhyN2RBWkRwdjV4Tnc2?=
 =?utf-8?B?eDNGc0ZWd0w4bE9VOWpnMm1OZjZEL1M3Tm5qVEd5b2ovV3VGUzNIYlZrSU9p?=
 =?utf-8?B?NUpmZGVUQzU3OEVLM3ptV2JhWEpXVzRrVlNzU0hDZ0VvZEp2UU1Bcm50UC9G?=
 =?utf-8?B?MWZhZjE4d3dxZXBOam1RWEk3RW5uMUlMUE1FM1pZTmdiMmkxcXU3WE50aFVs?=
 =?utf-8?B?VXFSWTcxWlhrWFBJWmdQVlMvemttcXp2VjA5Y01wZ0wzVjlzdHZGOFlqRXMv?=
 =?utf-8?B?Mm9zZ01KdXZOV3V5c0RZWUQ3T2h0QnZZMUQ2RGIvUDdpdkVRSFJIYWFjZkkx?=
 =?utf-8?B?dU5zV016R1JiYzhOc0VRTVkwMHVobWlaZy82UHJPbkozbnptSGh6U1pQcld4?=
 =?utf-8?B?TWVqcitKT3ZzbmpOTWhGQWwvKzk5OVZvWUo0Sk56YVFZa0hMemJqb0tqY0Jh?=
 =?utf-8?B?c1lJRzlvOXBvTjM1SFJuZHRvWXkzMW5nMHh1KzduVm1xRFlGRHpPRElYSnJi?=
 =?utf-8?B?ekc3bzhFUEZ6M0hVd04yOWRwZWEwQVZuU3pXemxlc1RVR3FaeVZ1R1V3NWxU?=
 =?utf-8?B?cG9BendLcThJOGtzenMrT3RwUi9mR0p6QzhXeXorSkpIdVNUMEgxRlRpMkNW?=
 =?utf-8?B?bU5hclVEWnE0cGZEUTczZzhaNnI3SjR2MUVxUmlrdGNFN0psNjlLWGUyemNF?=
 =?utf-8?Q?37y0FGaF20oNMQIr0LmCiCKElgXqm1JwfXqzvOK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ea049a5d-c4cd-4383-defc-08d978fcb4e3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 10:28:21.3209
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wSB986MJhQ9KchxLYBPZtoGuq+zcIlZQNPofFKh8rROe8iahPfx4rxb3K60Dbe/otBSiv11PMXH+UT3vgX1w7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7375

On 16.09.2021 11:34, Dmitry Isaikin wrote:
> --- a/tools/xl/xl.c
> +++ b/tools/xl/xl.c
> @@ -81,7 +81,7 @@ static int auto_autoballoon(void)
>          return 1; /* default to on */
>  
>      ret = regcomp(&regex,
> -                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
> +                  "(^| )dom0_mem=((|min:|max:)-?[0-9]+[bBkKmMgG]?,?)+($| )",

Wouldn't this be a good opportunity to also add [tT] to the list
of permitted granularity suffixes? The hypervisor's
parse_size_and_unit() has been supporting this for quite some
time ...

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:29:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:29:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188332.337460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQoeS-0005f5-J4; Thu, 16 Sep 2021 10:29:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188332.337460; Thu, 16 Sep 2021 10:29:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQoeS-0005ey-GB; Thu, 16 Sep 2021 10:29:56 +0000
Received: by outflank-mailman (input) for mailman id 188332;
 Thu, 16 Sep 2021 10:29:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SzYQ=OG=alien8.de=bp@srs-us1.protection.inumbo.net>)
 id 1mQoeR-0005el-5U
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:29:55 +0000
Received: from mail.skyhub.de (unknown [5.9.137.197])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d14273e-8dc6-4da1-a940-28d85bde6683;
 Thu, 16 Sep 2021 10:29:53 +0000 (UTC)
Received: from zn.tnic (p200300ec2f11c600b77f9e345fbfb487.dip0.t-ipconnect.de
 [IPv6:2003:ec:2f11:c600:b77f:9e34:5fbf:b487])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 1E3DB1EC01DF;
 Thu, 16 Sep 2021 12:29:48 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d14273e-8dc6-4da1-a940-28d85bde6683
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=dkim;
	t=1631788188;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:in-reply-to:in-reply-to:  references:references;
	bh=EXpt8wshiO7ImZN+Zx5zekJn4i6gD7QW4fNVEzzyMB0=;
	b=jdKCZeacNO/kKHUWytZdn5Fjo98CAZEQp+DgI7I37Kq2atS39x82DLxp+PzrfJ6w/OMpOx
	2kRvyfMEU9NGc0BdDMyQFTyKr41mebgtFFj1rEwSkta4odAJx3N6y0DSrQaVrO5hfk6wCD
	w9bRZIOCbxUdY5+wJ29B3EVPzm2K9/U=
Date: Thu, 16 Sep 2021 12:29:41 +0200
From: Borislav Petkov <bp@alien8.de>
To: Mike Rapoport <rppt@kernel.org>
Cc: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
	Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUMclVZ5uxKV/bRZ@zn.tnic>
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
 <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com>
 <YUHSRKubsGT2Jvur@zn.tnic>
 <YUMJx2ckLlWKi3VF@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <YUMJx2ckLlWKi3VF@kernel.org>

On Thu, Sep 16, 2021 at 12:09:27PM +0300, Mike Rapoport wrote:
> I think the first sentence about reserving memory before memblock
> allocations are possible is important and I think we should keep it.

I expanded that comment this way:

        /*
         * Do some memory reservations *before* memory is added to memblock, so
         * memblock allocations won't overwrite it.
         *
         * After this point, everything still needed from the boot loader or
         * firmware or kernel text should be early reserved or marked not RAM in
         * e820. All other memory is free game.
         *
         * This call needs to happen before e820__memory_setup() which calls the
         * xen_memory_setup() on Xen dom0 which relies on the fact that those
         * early reservations have happened already.
         */

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:31:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:31:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188340.337472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQofo-0006yS-Us; Thu, 16 Sep 2021 10:31:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188340.337472; Thu, 16 Sep 2021 10:31:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQofo-0006yL-RB; Thu, 16 Sep 2021 10:31:20 +0000
Received: by outflank-mailman (input) for mailman id 188340;
 Thu, 16 Sep 2021 10:31:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=mF6G=OG=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mQofm-0006y9-KN
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:31:18 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 56f74801-ceb7-4414-9046-921bd2b73362;
 Thu, 16 Sep 2021 10:31:17 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id D74D52006F;
 Thu, 16 Sep 2021 10:31:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 895BE13480;
 Thu, 16 Sep 2021 10:31:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 03RzH/McQ2GKdwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 16 Sep 2021 10:31:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56f74801-ceb7-4414-9046-921bd2b73362
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631788275; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=sgm0dAJp8BdvN0s6gKRbKGiaVg1v7MOgd6zX+l6mpH8=;
	b=glw2Dnz69PWFJ61XzOlJvfCspbTVjWBeVVaLPE+2u6BVz9afxkkakOFwGcgl8fKqligP4p
	TU90nDczwlcC64JWfeLKmiSrnzjW5F1TOkVUz4jwr5veKPY7VywYlLUwmnWrPlS/i0eYAl
	RE/9tVfdkEmp0RmazeBTVQxo3Py1Wtk=
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
To: Borislav Petkov <bp@alien8.de>, Mike Rapoport <rppt@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, Thomas Gleixner <tglx@linutronix.de>,
 Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel@lists.xenproject.org,
 x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
 <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com> <YUHSRKubsGT2Jvur@zn.tnic>
 <YUMJx2ckLlWKi3VF@kernel.org> <YUMclVZ5uxKV/bRZ@zn.tnic>
From: Juergen Gross <jgross@suse.com>
Message-ID: <780610f6-de4f-625d-e242-97063cea7cc5@suse.com>
Date: Thu, 16 Sep 2021 12:31:14 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YUMclVZ5uxKV/bRZ@zn.tnic>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ESaCxmyMRwfRXJ2Rqdj899fb35w3QKBrX"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ESaCxmyMRwfRXJ2Rqdj899fb35w3QKBrX
Content-Type: multipart/mixed; boundary="4cpB25eP0NjT0cdla0TpFh5C7UxGi85OM";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Borislav Petkov <bp@alien8.de>, Mike Rapoport <rppt@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, Thomas Gleixner <tglx@linutronix.de>,
 Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
 =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?=
 <marmarek@invisiblethingslab.com>, xen-devel@lists.xenproject.org,
 x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <780610f6-de4f-625d-e242-97063cea7cc5@suse.com>
Subject: Re: [PATCH] x86/setup: call early_reserve_memory() earlier
References: <20210914094108.22482-1-jgross@suse.com>
 <b15fa98e-f9a8-abac-2d16-83c29dafc517@suse.com>
 <6cdc71dc-c26d-5c59-b7dd-0eb47ab9c861@suse.com> <YUHSRKubsGT2Jvur@zn.tnic>
 <YUMJx2ckLlWKi3VF@kernel.org> <YUMclVZ5uxKV/bRZ@zn.tnic>
In-Reply-To: <YUMclVZ5uxKV/bRZ@zn.tnic>

--4cpB25eP0NjT0cdla0TpFh5C7UxGi85OM
Content-Type: multipart/mixed;
 boundary="------------02D53250C7CD0407D92266F9"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------02D53250C7CD0407D92266F9
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 16.09.21 12:29, Borislav Petkov wrote:
> On Thu, Sep 16, 2021 at 12:09:27PM +0300, Mike Rapoport wrote:
>> I think the first sentence about reserving memory before memblock
>> allocations are possible is important and I think we should keep it.
>=20
> I expanded that comment this way:
>=20
>          /*
>           * Do some memory reservations *before* memory is added to mem=
block, so
>           * memblock allocations won't overwrite it.
>           *
>           * After this point, everything still needed from the boot loa=
der or
>           * firmware or kernel text should be early reserved or marked =
not RAM in
>           * e820. All other memory is free game.
>           *
>           * This call needs to happen before e820__memory_setup() which=
 calls the
>           * xen_memory_setup() on Xen dom0 which relies on the fact tha=
t those
>           * early reservations have happened already.
>           */
>=20

Yes, this seems to be an accurate description.


Juergen

--------------02D53250C7CD0407D92266F9
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------02D53250C7CD0407D92266F9--

--4cpB25eP0NjT0cdla0TpFh5C7UxGi85OM--

--ESaCxmyMRwfRXJ2Rqdj899fb35w3QKBrX
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFDHPIFAwAAAAAACgkQsN6d1ii/Ey9K
mwf+Nks3rc+8ANWcRIQCcyFRuJFd+Ld6Alj9WlimXftXpX1zn+65HJ1nTYt2AitkDhJvWqyI9a/I
GYR4vjJmQS3MAv42oc/0dIVBSB6fphVQIGj7jyPesFAnETKIa4QLrBIvjWtzN51cemhcBpSLU4JY
RRH2SCwTah1b7RaO7O5cyNVDxiU4syvooTi3x7sKfDs2xuyqkQS3rNGj0paHR5P+fZvia3hfhnZ3
1PMuPo44AgwMlNkS/QKyauYeLrbQiJLUNYSJcEXw9sG7higdpv7ueEwIl3/Bbld2VJffEV41i18X
vg/cmDXgMOzVFht0Xww4ndtcO1Bk8f4INcD+uMdCBw==
=yMmO
-----END PGP SIGNATURE-----

--ESaCxmyMRwfRXJ2Rqdj899fb35w3QKBrX--


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:47:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:47:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188349.337483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQovC-0000QA-BM; Thu, 16 Sep 2021 10:47:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188349.337483; Thu, 16 Sep 2021 10:47:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQovC-0000Q3-7r; Thu, 16 Sep 2021 10:47:14 +0000
Received: by outflank-mailman (input) for mailman id 188349;
 Thu, 16 Sep 2021 10:47:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQovB-0000Px-1x
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:47:13 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.82]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 719d29ce-16db-11ec-b5c8-12813bfff9fa;
 Thu, 16 Sep 2021 10:47:10 +0000 (UTC)
Received: from PR2P264CA0034.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::22) by
 AM4PR0802MB2226.eurprd08.prod.outlook.com (2603:10a6:200:59::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 10:47:07 +0000
Received: from VE1EUR03FT042.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:101:1::4) by PR2P264CA0034.outlook.office365.com
 (2603:10a6:101:1::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 10:47:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT042.mail.protection.outlook.com (10.152.19.62) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 10:47:07 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Thu, 16 Sep 2021 10:47:06 +0000
Received: from 20e7183a0b91.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C48E8E27-25CB-45EC-9842-01D5DBF19A4E.1; 
 Thu, 16 Sep 2021 10:46:47 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 20e7183a0b91.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 10:46:47 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6182.eurprd08.prod.outlook.com (2603:10a6:20b:291::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 10:46:46 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 10:46:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 719d29ce-16db-11ec-b5c8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jBWW8BtV6F07wZ+bvK1ss0lYm9CIE0jZsuxfVVFRvB8=;
 b=baJOmxYcM09eyUZjd5EsgLDVeFr7EHlDcgrCHEs6aWnq6gJZTk6MHQ2UpTRZ2PQvQ6OmoXscJ76KrSqnOw7vMcy1Okmu0DdBZFyJMaRL86GD8dWJeR1sfpL5bFjq5mQY3oqdlwX1n9J0JMEErDZLfca+Jn3JB7/MNBsv6+u4rzo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 17a0c3848bc21e12
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=esBjBowRhHpXfHx8jUyYcOI8aCZyM6rUC40+JEiXkD7Z6MrkfnEAEyXdAp2xjmwzhdt1KIRH5iAY1BoxBZvNz6mN0TXC9C7k2FgR4DBxnBi5A26DiSXj9mnsHzZ5smSjVarWYF7X0Q/JTES+/Rmnm6ode6dT5qZ92VyOy6P7RouVsP3m/AH0thzU+Nb013LE5/VMe8rvRJ8h8sypUi3tyyAgnGSYSc8K3y5+NvbV0L5He6YbRxOivQBM3MAklpg7Vkpl+UMTnvHadJSWkfcTWLGUuMBjbrIPVZP5OjS7YdYU7a7I5z/PpwFM1S6yZ+X+tLw3k+rD5yswH3EkGZeEMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jBWW8BtV6F07wZ+bvK1ss0lYm9CIE0jZsuxfVVFRvB8=;
 b=VwSjdR6EoIoT4c7HdZ3YxhAnfSqSvJomZAANA4S5Zd0UNynxjjmyuC6w3ncp+tsVZ9NZDT+5EKU33X9hyTBLPUFTZOnf+yuLGocfA13DjQhtb3jXmgmL1CjZNcef5wnF6HT31GbfNA8L1UDgmhxf57+EhU3f+ZSs2BdiOGoiFdDCgNxctHKhzTYajix5Z1rxJq4P1mYFneZmjl4gnNjkC8vh9tuxmyjyVpFNKm633VU3h1kTqPRko1MYiMUblpXyeEapoZPHJngRe5z405AqI2fMccDk7i3UgECz3o8s8eeIOowikZmPl6yN8tobqwMhZaWj+o8JmNjMcYFV/BY/cg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jBWW8BtV6F07wZ+bvK1ss0lYm9CIE0jZsuxfVVFRvB8=;
 b=baJOmxYcM09eyUZjd5EsgLDVeFr7EHlDcgrCHEs6aWnq6gJZTk6MHQ2UpTRZ2PQvQ6OmoXscJ76KrSqnOw7vMcy1Okmu0DdBZFyJMaRL86GD8dWJeR1sfpL5bFjq5mQY3oqdlwX1n9J0JMEErDZLfca+Jn3JB7/MNBsv6+u4rzo=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Paul Durrant <paul@xen.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXlPL13zcSD0YvP0y1H1bgZa9bP6ub2WAAgArNAoA=
Date: Thu, 16 Sep 2021 10:46:46 +0000
Message-ID: <2083642E-943A-44E4-AF81-689164AC8F8D@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
 <a868e1e7-8400-45df-6eaa-69f1e2c99383@xen.org>
In-Reply-To: <a868e1e7-8400-45df-6eaa-69f1e2c99383@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 89d0255c-c57b-46e8-708f-08d978ff540c
x-ms-traffictypediagnostic: AS8PR08MB6182:|AM4PR0802MB2226:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM4PR0802MB222650B159C52B7ECA8F1EE1FCDC9@AM4PR0802MB2226.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4gvlplqIYBYjgMnO5yGGrQpCmM7AxMSfKq3H40E36hYHw9a33xhk1GF4kf7yqk+19/WkI3UaLKqjDScMbuajQANdPJm9sDsYLBNnPClR/aGfuGzBM8Gfk9qE83olxIGvxb2vfjo6yyXuU8D6BmSGtToBVO3ZcMyu+3+9SSp3+tw1g6FMdtAAYFDiIg72W1mJ03YjWa9dfFbp44ip/4GUr4SEO9Nu2Lmh6ftWytdgbMXsIbLlei5wQEJrwgtI9Hmlz8r+4IvMPi3w3ZhIowHqnZ/GHRr3QMY9W3O4FT/eSnCMul53ZOLswt8UCPdYiACjtgIm++pJbZaPFo9cDyRKaGyJnJCXN36CARWoXCbA6l1QxfyxnkiSUXwD6sdp6lUt3l/3D+ea4eTtJ05I8a7e1OkAI0P1e7KySi/+ry6dArbv8yvYNxDJ+uiAjngGlMY1NiDlEHQVHLrC0eKj2Itl8ntmGqif4u8LdJxdcgwXRPYfIFoOEPQGwc7049AblzhM5feADzgz/VT5BF5jjDLkFhWojqi95epiMBer2L+k3Vu+ijAVOCyqcKSB1XJ+Zp0Dg8rg5m+o5R08uYGnxADWHuEbm8nMUYw8O/skKmOMzv09dPZoWR3Hco1+Vg3zxfB5CWEcRLRbLzpWiTh5VkOUDqV1joq1wrcfJaV6/fApzMQDKbrOldnzTYXi4H7dOtuSwX2A5G/lp2y1tEeInflHsDpWNPmOMoz+r1cEEmu4z/VXB+HUOd3ugacOQkhvgZ3Np58vZdyxqI0C38NPd3uupQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(366004)(346002)(39860400002)(136003)(86362001)(8676002)(38070700005)(8936002)(66446008)(30864003)(64756008)(66476007)(66556008)(66946007)(316002)(71200400001)(26005)(122000001)(5660300002)(36756003)(76116006)(4326008)(91956017)(38100700002)(6512007)(6486002)(54906003)(53546011)(2616005)(33656002)(6506007)(2906002)(6916009)(186003)(478600001)(83380400001)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <F0739E9EBE799E45A704813DB8FA58E7@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6182
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ada5c56d-6cc0-44bc-a51e-08d978ff47a7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LPMmyVbd5hMVsgl97Q8A4Jok9NZ7hfffAE7k/dOxqAeBHFAOp/I3W6SwV1LPUus8yS4/zoHLnyLY9fFBR3aSXs3KROn22z25zZqTaanDeBFg/W3nPaid19dAv8VPGNM8SkkPf8//hCMGAp5Jr+jcGZdsoXNHrrCl+TrKp+ocV/+Vc1cjGwQ1gGoZRwhrmgwd3fBgy+HMIjvQnIwdJ/XHkpJzT/3U1veMU4g7pyX7rb9FguwmuqgkWwmzOOzG31Ccupj/4EuTpNfDDQYd/TQjOpkDhMb6d6iBizkMv/j48wW3/Vja94NVLW9Cvy7PYyh4ZtVuzL7CocqmWePrzseNE8u77Np0QOJeP25MGV94uVlTf4vh93vrFM8HMAfv1Ep9BnMqf9gai1YLgAT7oIdN5U1vR8ETLC5xwqPEHZuZDwyxKzUGCKEbHmGv+QKzSiJirQ5ay8waoshceXoqDuShe9CypUtLhWYSfI9Rnt2dlzKEDBhF9n/Hj2FDen5Cz3/tLqcsQlhIa/BxOTHL1o2a2nIQUOySG+ZeqrV4mFJiIVPTlUCOcEnCxm3OavXLdD+zhnaTnsMMylF+wXpDgPocfrYV3u/fXXBOKIxAo/HRgbeX8jMKsG0XaOLp3Ir5ALlEw4RFrYAYhxi6A+PtWxqRlHwOk6T4iIrPr3mbKshmH6q1We8P7cOUgBL5SotMIcbixVtT1enPpBRp1MlKQDOi+T3N805WgPRdAPb4CsQ64XI=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(39860400002)(376002)(346002)(396003)(36840700001)(46966006)(30864003)(70586007)(356005)(70206006)(2906002)(6512007)(36860700001)(6486002)(8936002)(6506007)(53546011)(82310400003)(2616005)(26005)(8676002)(186003)(336012)(33656002)(36756003)(81166007)(83380400001)(82740400003)(316002)(6862004)(54906003)(47076005)(5660300002)(86362001)(107886003)(478600001)(4326008)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 10:47:07.0311
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 89d0255c-c57b-46e8-708f-08d978ff540c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2226

SGkgSnVsaWVuLA0KDQo+IE9uIDkgU2VwIDIwMjEsIGF0IDI6NTAgcG0sIEp1bGllbiBHcmFsbCA8
anVsaWVuQHhlbi5vcmc+IHdyb3RlOg0KPiANCj4gSGkgUmFodWwsDQo+IA0KPiBPbiAxOS8wOC8y
MDIxIDEzOjAyLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+IFRoZSBleGlzdGluZyBWUENJIHN1cHBv
cnQgYXZhaWxhYmxlIGZvciBYODYgaXMgYWRhcHRlZCBmb3IgQXJtLg0KPj4gV2hlbiB0aGUgZGV2
aWNlIGlzIGFkZGVkIHRvIFhFTiB2aWEgdGhlIGh5cGVyIGNhbGwNCj4+IOKAnFBIWVNERVZPUF9w
Y2lfZGV2aWNlX2FkZOKAnSwgVlBDSSBoYW5kbGVyIGZvciB0aGUgY29uZmlnIHNwYWNlDQo+PiBh
Y2Nlc3MgaXMgYWRkZWQgdG8gdGhlIFhlbiB0byBlbXVsYXRlIHRoZSBQQ0kgZGV2aWNlcyBjb25m
aWcgc3BhY2UuDQo+PiBBIE1NSU8gdHJhcCBoYW5kbGVyIGZvciB0aGUgUENJIEVDQU0gc3BhY2Ug
aXMgcmVnaXN0ZXJlZCBpbiBYRU4NCj4+IHNvIHRoYXQgd2hlbiBndWVzdCBpcyB0cnlpbmcgdG8g
YWNjZXNzIHRoZSBQQ0kgY29uZmlnIHNwYWNlLFhFTg0KPj4gd2lsbCB0cmFwIHRoZSBhY2Nlc3Mg
YW5kIGVtdWxhdGUgcmVhZC93cml0ZSB1c2luZyB0aGUgVlBDSSBhbmQNCj4+IG5vdCB0aGUgcmVh
bCBQQ0kgaGFyZHdhcmUuDQo+PiBTaWduZWQtb2ZmLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2lu
Z2hAYXJtLmNvbT4NCj4+IC0tLQ0KPj4gIHhlbi9hcmNoL2FybS9NYWtlZmlsZSAgICAgICAgIHwg
IDEgKw0KPj4gIHhlbi9hcmNoL2FybS9kb21haW4uYyAgICAgICAgIHwgIDQgKysNCj4+ICB4ZW4v
YXJjaC9hcm0vdnBjaS5jICAgICAgICAgICB8IDk2ICsrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrDQo+PiAgeGVuL2FyY2gvYXJtL3ZwY2kuaCAgICAgICAgICAgfCAzNyArKysrKysr
KysrKysrKw0KPj4gIHhlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jIHwgIDcgKysrDQo+PiAg
eGVuL2RyaXZlcnMvdnBjaS9NYWtlZmlsZSAgICAgfCAgMyArLQ0KPj4gIHhlbi9kcml2ZXJzL3Zw
Y2kvaGVhZGVyLmMgICAgIHwgIDIgKw0KPj4gIHhlbi9pbmNsdWRlL2FzbS1hcm0vZG9tYWluLmgg
IHwgIDUgKy0NCj4+ICB4ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oICAgICB8ICA4ICsrKw0KPj4g
IHhlbi9pbmNsdWRlL3B1YmxpYy9hcmNoLWFybS5oIHwgIDQgKysNCj4+ICAxMCBmaWxlcyBjaGFu
Z2VkLCAxNjUgaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkNCj4+ICBjcmVhdGUgbW9kZSAx
MDA2NDQgeGVuL2FyY2gvYXJtL3ZwY2kuYw0KPj4gIGNyZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vYXJj
aC9hcm0vdnBjaS5oDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL01ha2VmaWxlIGIveGVu
L2FyY2gvYXJtL01ha2VmaWxlDQo+PiBpbmRleCAwZTE0YTVlNWM4Li43Y2RjZTY4NGE0IDEwMDY0
NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL01ha2VmaWxlDQo+PiArKysgYi94ZW4vYXJjaC9hcm0v
TWFrZWZpbGUNCj4+IEBAIC03LDYgKzcsNyBAQCBvYmoteSArPSBwbGF0Zm9ybXMvDQo+PiAgZW5k
aWYNCj4+ICBvYmotJChDT05GSUdfVEVFKSArPSB0ZWUvDQo+PiAgb2JqLSQoQ09ORklHX0hBU19Q
Q0kpICs9IHBjaS8NCj4+ICtvYmotJChDT05GSUdfSEFTX1ZQQ0kpICs9IHZwY2kubw0KPj4gICAg
b2JqLSQoQ09ORklHX0hBU19BTFRFUk5BVElWRSkgKz0gYWx0ZXJuYXRpdmUubw0KPj4gIG9iai15
ICs9IGJvb3RmZHQuaW5pdC5vDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL2RvbWFpbi5j
IGIveGVuL2FyY2gvYXJtL2RvbWFpbi5jDQo+PiBpbmRleCAxOWM3NTZhYzNkLi5kOTljNjUzNjI2
IDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL2RvbWFpbi5jDQo+PiArKysgYi94ZW4vYXJj
aC9hcm0vZG9tYWluLmMNCj4+IEBAIC00MCw2ICs0MCw3IEBADQo+PiAgI2luY2x1ZGUgPGFzbS92
dGltZXIuaD4NCj4+ICAgICNpbmNsdWRlICJ2dWFydC5oIg0KPj4gKyNpbmNsdWRlICJ2cGNpLmgi
DQo+IA0KPiBQbGVhc2Ugb3JkZXIgdGhlIGluY2x1ZGVzIGFscGhhYmV0aWNhbGx5LiBTbyB0aGlz
IG9uZSBzaG91bGQgZ28gYmVmb3JlICJ2dWFydC5oIi4NCj4gDQo+PiAgICBERUZJTkVfUEVSX0NQ
VShzdHJ1Y3QgdmNwdSAqLCBjdXJyX3ZjcHUpOw0KPj4gIEBAIC03NjcsNiArNzY4LDkgQEAgaW50
IGFyY2hfZG9tYWluX2NyZWF0ZShzdHJ1Y3QgZG9tYWluICpkLA0KPj4gICAgICBpZiAoIGlzX2hh
cmR3YXJlX2RvbWFpbihkKSAmJiAocmMgPSBkb21haW5fdnVhcnRfaW5pdChkKSkgKQ0KPj4gICAg
ICAgICAgZ290byBmYWlsOw0KPj4gICsgICAgaWYgKCAocmMgPSBkb21haW5fdnBjaV9pbml0KGQp
KSAhPSAwICkNCj4+ICsgICAgICAgIGdvdG8gZmFpbDsNCj4+ICsNCj4+ICAgICAgcmV0dXJuIDA7
DQo+PiAgICBmYWlsOg0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS92cGNpLmMgYi94ZW4v
YXJjaC9hcm0vdnBjaS5jDQo+PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPj4gaW5kZXggMDAwMDAw
MDAwMC4uZGE4YjFjYTEzYw0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIveGVuL2FyY2gvYXJt
L3ZwY2kuYw0KPj4gQEAgLTAsMCArMSw5NiBAQA0KPj4gKy8qDQo+PiArICogeGVuL2FyY2gvYXJt
L3ZwY2kuYw0KPj4gKyAqIENvcHlyaWdodCAoYykgMjAyMSBBcm0gTHRkLg0KPj4gKyAqDQo+PiAr
ICogVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0
IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5l
cmFsIFB1YmxpYyBMaWNlbnNlIGFzIHB1Ymxpc2hlZCBieQ0KPj4gKyAqIHRoZSBGcmVlIFNvZnR3
YXJlIEZvdW5kYXRpb247IGVpdGhlciB2ZXJzaW9uIDIgb2YgdGhlIExpY2Vuc2UsIG9yDQo+PiAr
ICogKGF0IHlvdXIgb3B0aW9uKSBhbnkgbGF0ZXIgdmVyc2lvbi4NCj4+ICsgKg0KPj4gKyAqIFRo
aXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0aGUgaG9wZSB0aGF0IGl0IHdpbGwgYmUgdXNl
ZnVsLA0KPj4gKyAqIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0aG91dCBldmVuIHRoZSBp
bXBsaWVkIHdhcnJhbnR5IG9mDQo+PiArICogTUVSQ0hBTlRBQklMSVRZIG9yIEZJVE5FU1MgRk9S
IEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRoZQ0KPj4gKyAqIEdOVSBHZW5lcmFsIFB1Ymxp
YyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuDQo+PiArICovDQo+PiArI2luY2x1ZGUgPHhlbi9z
Y2hlZC5oPg0KPiANCj4gTklUOiBQbGVhc2UgYWRkIGEgbmV3bGluZSBiZXR3ZWVuIGdlbmVyaWMg
YW5kIGFyY2ggc3BlY2lmaWMgaW5jbHVkZXMuDQpBY2suDQo+IA0KPj4gKyNpbmNsdWRlIDxhc20v
bW1pby5oPg0KPj4gKw0KPj4gKy8qIERvIHNvbWUgc2FuaXR5IGNoZWNrcy4gKi8NCj4+ICtzdGF0
aWMgYm9vbCB2cGNpX21taW9fYWNjZXNzX2FsbG93ZWQodW5zaWduZWQgaW50IHJlZywgdW5zaWdu
ZWQgaW50IGxlbikNCj4+ICt7DQo+PiArICAgIC8qIENoZWNrIGFjY2VzcyBzaXplLiAqLw0KPj4g
KyAgICBpZiAoIGxlbiAhPSAxICYmIGxlbiAhPSAyICYmIGxlbiAhPSA0ICYmIGxlbiAhPSA4ICkN
Cj4+ICsgICAgICAgIHJldHVybiBmYWxzZTsNCj4gWW91IHdpbGwgYWxsb3cgYWxsIHRoZSBwb3Nz
aWJsZSB2YWx1ZSBvZiBsZW4gKHRoaXMgaXMgY29taW5nIGZyb20gdGhlIEhXKS4gU28gSSBmZWVs
cyB0aGlzIGlzIGEgYml0IHRvbyBtdWNoIHRvIGNoZWNrIGZvciBldmVyeSBJL08gdG8gdGhlIHZQ
Q0kuDQo+IA0KPiBJZiB5b3UgcmVhbGx5IHdhbnQgdG8ga2VlcCB0aGUgY2hlY2sgdGhlbiB5b3Ug
Y2FuIHNpbXBseSBjaGVjayB0aGF0IGxlbiBpcyA8IDggYmVjYXVzZSB0aGUgdHdvIGNhbGxlcnMg
Y29tcHV0ZSBpdCB3aXRoICgxIDw8IFMpLiBTbyB0aGVyZSBpcyBubyB3YXkgdG8gc2V0IGl0IDMs
IDUsIDYgYW5kIDcuDQoNCkkgd2lsbCByZW1vdmUgdGhlIGNoZWNrIGZvciAxLDIsNCw4IGFuZCB3
aWxsIGhhdmUgY2hlY2sgKCBsZW4gPCA4KS4NCj4gDQo+PiArDQo+PiArICAgIC8qIENoZWNrIHRo
YXQgYWNjZXNzIGlzIHNpemUgYWxpZ25lZC4gKi8NCj4+ICsgICAgaWYgKCAocmVnICYgKGxlbiAt
IDEpKSApDQo+PiArICAgICAgICByZXR1cm4gZmFsc2U7DQo+PiArDQo+PiArICAgIHJldHVybiB0
cnVlOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW50IHZwY2lfbW1pb19yZWFkKHN0cnVjdCB2
Y3B1ICp2LCBtbWlvX2luZm9fdCAqaW5mbywNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAg
IHJlZ2lzdGVyX3QgKnIsIHZvaWQgKnApDQo+PiArew0KPj4gKyAgICB1bnNpZ25lZCBpbnQgcmVn
Ow0KPj4gKyAgICBwY2lfc2JkZl90IHNiZGY7DQo+PiArICAgIHVpbnQzMl90IGRhdGEgPSAwOw0K
Pj4gKyAgICB1bnNpZ25lZCBpbnQgc2l6ZSA9IDFVIDw8IGluZm8tPmRhYnQuc2l6ZTsNCj4+ICsN
Cj4+ICsgICAgc2JkZi5zYmRmID0gKCgoaW5mby0+Z3BhKSAmIDB4MGZmZmYwMDApID4+IDEyKTsN
Cj4+ICsgICAgcmVnID0gKCgoaW5mby0+Z3BhKSAmIDB4MDAwMDBmZmMpIHwgKGluZm8tPmdwYSAm
IDMpKTsNCj4gDQo+IFRoaXMgbG9naWMgaXMgdGhlIHNhbWUgYXMgYmVsb3cgaW4gdnBjaV9tbWlv
X3dyaXRlKCkuIFNvIEkgdGhpbmsgeW91IHdhbnQgdG8gcHJvdmlkZSBhbiBoZWxwZXIgYmVjYXVz
ZSB0aGlzIGlzIG5vdCB0cml2aWFsIHRvIHJlYWQuDQpPay4NCj4gDQo+IEFsc28sIGZvciB0aGUg
Zmlyc3QgbGluZSwgSSB0aGluayB5b3UgY2FuIHJlLXVzZSBNTUNGR19CREYoKSBmcm9tIHRoZSB4
ODYgY29kZS4gRm9yIHRoZSBzZWNvbmQgbGluZSwgSSB3b3VsZCBkZWZpbmUgdGhlIHZhbHVlIHNv
IGl0IGlzIGNsZWFyZXIgdG8gdW5kZXJzdGFuZCB0aGF0IHRoZXkgbWVhbiAoYWx0aG91Z2ggJiAz
IGlzIGZpbmUgdG8gbWUpIC4NCk9rLg0KPiANCj4+ICsNCj4+ICsgICAgaWYgKCAhdnBjaV9tbWlv
X2FjY2Vzc19hbGxvd2VkKHJlZywgc2l6ZSkgKQ0KPj4gKyAgICAgICAgcmV0dXJuIDE7DQo+IFNv
LCB5b3Ugd2lsbCBhIGd1ZXN0IHdpbGwgcmVhZCAwIGlmIHRoZSBhY2Nlc3MgaXMgdW5hbGlnbmVk
LiBUaGlzIHNlZW1zIGFuIG9kZCBiZWhhdmlvciBnaXZlbiB0aGlzIGlzIG5vdCBhbiBhbGxvd2Vk
IGFjY2Vzcy4gQUZBSVUsIHRoZSBIVyB3b3VsZCBsaWtlbHkgdHJvdyBhIGRhdGEgYWJvcnQgYmVj
YXVzZSB5b3UgY2FuJ3QgZG8gdW5hbGlnbiBhY2Nlc3Mgb24gdW5jYWNoYWJsZSBtZW1vcnkuIFNv
IEkgdGhpbmsgd2Ugc2hvdWxkIHJldHVybiAwIGhlcmUgdG8gbGV0IHRoZSBNTUlPIGhhbmRsZXIg
aW5qZWN0IGEgZGF0YSBhYm9ydC4NCg0KQWNrLiANCj4gDQo+PiArDQo+PiArICAgIGRhdGEgPSB2
cGNpX3JlYWQoc2JkZiwgcmVnLCBzaXplKTsNCj4gDQo+IFNvIGluIHZwY2lfbW1pb19hY2Nlc3Nf
YWxsb3dlZCgpLCB5b3Ugd2lsbCBhbGxvdyBhIGd1ZXN0IHRvIHJlYWQgYSA2NC1iaXQgdmFsdWUu
IEJ1dC4uLiB2cGNpX3JlYWQoKSB3aWxsIHJldHVybiBhIDMyLWJpdCB2YWx1ZS4NCj4gDQo+IExv
b2tpbmcgYXQgdGhlIHg4NiBjb2RlLCB0aGV5IGhhdmUgYSBzZWNvbmQgY2FsbCB0byB2cGNpX3Jl
YWQoKSB0byBoYW5kbGUgdGhlIHRvcCAzMi1iaXQuIEFueSByZWFzb24gd2h5IHRoaXMgd2FzIG5v
dCBpbXBsZW1lbnRlZCBvbiBBcm0/DQoNCkkgbWlzc2VkIHRoYXQgSSB3aWxsIGZpeCB0aGlzIGlu
IG5leHQgdmVyc2lvbi4NCg0KPiANCj4gSWYgd2UgbmVlZCB0byBpbXBsZW1lbnQgaXQgdGhlbiBJ
IHRoaW5rIHRoaXMgc2hvdWxkIGJlIGltcGxlbWVudCBpbiB2cGNpX3JlYWQoKSB0byBhdm9pZCBk
dXBsaWNhdGlvbiBiZXR3ZWVuIHg4NiBhbmQgYXJtLg0KSSB0aGluayBpdCB3aWxsIGJlIGdvb2Qg
dG8gbWFrZSB2cGNpX3JlYWQoKSAmIHZwY2lfd3JpdGUoKSBzaW1wbGUgbm90IHRvIGFkZCBleHRy
YSBsb2dpYyBiYXNlZCBvbiA2NCBiaXQgdmFsdWUuIA0KSSB3aWxsIGFkZCBzZWNvbmQgY2FsbCBm
b3IgdnBjaV9yZWFkKCkgYW5kIHZwY2lfd3JpdGUoKSBmb3IgQVJNLiANCj4gDQo+PiArDQo+PiAr
ICAgIG1lbWNweShyLCAmZGF0YSwgc2l6ZSk7DQo+IA0KPiBGcm9tIG15IHVuZGVyc3RhbmRpbmcs
IGFueSB1bnVzZWQgYml0IHNob3VsZCBiZSAwLiBTbyB0aGlzIHNob3VsZCBiZToNCj4gDQo+ICpy
ID0gZGF0YTsNCkFjay4NCj4gDQo+PiArDQo+PiArICAgIHJldHVybiAxOw0KPj4gK30NCj4+ICsN
Cj4+ICtzdGF0aWMgaW50IHZwY2lfbW1pb193cml0ZShzdHJ1Y3QgdmNwdSAqdiwgbW1pb19pbmZv
X3QgKmluZm8sDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVnaXN0ZXJfdCByLCB2
b2lkICpwKQ0KPiANCj4gTXkgcmVtYXJrcyBvbiB2cGNpX21taW9fcmVhZCgpIGFwcGxpZXMgaGVy
ZSB0b28uDQo+IA0KPj4gK3sNCj4+ICsgICAgdW5zaWduZWQgaW50IHJlZzsNCj4+ICsgICAgcGNp
X3NiZGZfdCBzYmRmOw0KPj4gKyAgICB1aW50MzJfdCBkYXRhID0gcjsNCj4+ICsgICAgdW5zaWdu
ZWQgaW50IHNpemUgPSAxVSA8PCBpbmZvLT5kYWJ0LnNpemU7DQo+PiArDQo+PiArICAgIHNiZGYu
c2JkZiA9ICgoKGluZm8tPmdwYSkgJiAweDBmZmZmMDAwKSA+PiAxMik7DQo+PiArICAgIHJlZyA9
ICgoKGluZm8tPmdwYSkgJiAweDAwMDAwZmZjKSB8IChpbmZvLT5ncGEgJiAzKSk7DQo+PiArDQo+
PiArICAgIGlmICggIXZwY2lfbW1pb19hY2Nlc3NfYWxsb3dlZChyZWcsIHNpemUpICkNCj4+ICsg
ICAgICAgIHJldHVybiAxOw0KPj4gKw0KPj4gKyAgICB2cGNpX3dyaXRlKHNiZGYsIHJlZywgc2l6
ZSwgZGF0YSk7DQo+PiArDQo+PiArICAgIHJldHVybiAxOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0
aWMgY29uc3Qgc3RydWN0IG1taW9faGFuZGxlcl9vcHMgdnBjaV9tbWlvX2hhbmRsZXIgPSB7DQo+
PiArICAgIC5yZWFkICA9IHZwY2lfbW1pb19yZWFkLA0KPj4gKyAgICAud3JpdGUgPSB2cGNpX21t
aW9fd3JpdGUsDQo+PiArfTsNCj4+ICsNCj4+ICtpbnQgZG9tYWluX3ZwY2lfaW5pdChzdHJ1Y3Qg
ZG9tYWluICpkKQ0KPj4gK3sNCj4+ICsgICAgaWYgKCAhaGFzX3ZwY2koZCkgKQ0KPj4gKyAgICAg
ICAgcmV0dXJuIDA7DQo+PiArDQo+PiArICAgIHJlZ2lzdGVyX21taW9faGFuZGxlcihkLCAmdnBj
aV9tbWlvX2hhbmRsZXIsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICBHVUVTVF9WUENJ
X0VDQU1fQkFTRSwgR1VFU1RfVlBDSV9FQ0FNX1NJWkUsIE5VTEwpOw0KPj4gKw0KPj4gKyAgICBy
ZXR1cm4gMDsNCj4+ICt9DQo+PiArDQo+PiArLyoNCj4+ICsgKiBMb2NhbCB2YXJpYWJsZXM6DQo+
PiArICogbW9kZTogQw0KPj4gKyAqIGMtZmlsZS1zdHlsZTogIkJTRCINCj4+ICsgKiBjLWJhc2lj
LW9mZnNldDogNA0KPj4gKyAqIGluZGVudC10YWJzLW1vZGU6IG5pbA0KPj4gKyAqIEVuZDoNCj4+
ICsgKi8NCj4+ICsNCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vdnBjaS5oIGIveGVuL2Fy
Y2gvYXJtL3ZwY2kuaA0KPj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4IDAwMDAwMDAw
MDAuLjhhMDkzYmI3MDUNCj4+IC0tLSAvZGV2L251bGwNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS92
cGNpLmgNCj4+IEBAIC0wLDAgKzEsMzcgQEANCj4+ICsvKg0KPj4gKyAqIHhlbi9hcmNoL2FybS92
cGNpLmgNCj4+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMjEgQXJtIEx0ZC4NCj4+ICsgKg0KPj4gKyAq
IFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0ZSBpdCBh
bmQvb3IgbW9kaWZ5DQo+PiArICogaXQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJh
bCBQdWJsaWMgTGljZW5zZSBhcyBwdWJsaXNoZWQgYnkNCj4+ICsgKiB0aGUgRnJlZSBTb2Z0d2Fy
ZSBGb3VuZGF0aW9uOyBlaXRoZXIgdmVyc2lvbiAyIG9mIHRoZSBMaWNlbnNlLCBvcg0KPj4gKyAq
IChhdCB5b3VyIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uDQo+PiArICoNCj4+ICsgKiBUaGlz
IHByb2dyYW0gaXMgZGlzdHJpYnV0ZWQgaW4gdGhlIGhvcGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1
bCwNCj4+ICsgKiBidXQgV0lUSE9VVCBBTlkgV0FSUkFOVFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1w
bGllZCB3YXJyYW50eSBvZg0KPj4gKyAqIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBB
IFBBUlRJQ1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUNCj4+ICsgKiBHTlUgR2VuZXJhbCBQdWJsaWMg
TGljZW5zZSBmb3IgbW9yZSBkZXRhaWxzLg0KPj4gKyAqLw0KPj4gKw0KPj4gKyNpZm5kZWYgX19B
UkNIX0FSTV9WUENJX0hfXw0KPj4gKyNkZWZpbmUgX19BUkNIX0FSTV9WUENJX0hfXw0KPj4gKw0K
Pj4gKyNpZmRlZiBDT05GSUdfSEFTX1ZQQ0kNCj4+ICtpbnQgZG9tYWluX3ZwY2lfaW5pdChzdHJ1
Y3QgZG9tYWluICpkKTsNCj4+ICsjZWxzZQ0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IGRvbWFpbl92
cGNpX2luaXQoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICt7DQo+PiArICAgIHJldHVybiAwOw0KPj4g
K30NCj4+ICsjZW5kaWYNCj4+ICsNCj4+ICsjZW5kaWYgLyogX19BUkNIX0FSTV9WUENJX0hfXyAq
Lw0KPj4gKw0KPj4gKy8qDQo+PiArICogTG9jYWwgdmFyaWFibGVzOg0KPj4gKyAqIG1vZGU6IEMN
Cj4+ICsgKiBjLWZpbGUtc3R5bGU6ICJCU0QiDQo+PiArICogYy1iYXNpYy1vZmZzZXQ6IDQNCj4+
ICsgKiBpbmRlbnQtdGFicy1tb2RlOiBuaWwNCj4+ICsgKiBFbmQ6DQo+PiArICovDQo+PiBkaWZm
IC0tZ2l0IGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMgYi94ZW4vZHJpdmVycy9wYXNz
dGhyb3VnaC9wY2kuYw0KPj4gaW5kZXggYzIzYzhjYjA2Yi4uNTZlMjYxZTliZCAxMDA2NDQNCj4+
IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+PiArKysgYi94ZW4vZHJpdmVy
cy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gQEAgLTc2Nyw2ICs3NjcsMTMgQEAgaW50IHBjaV9hZGRf
ZGV2aWNlKHUxNiBzZWcsIHU4IGJ1cywgdTggZGV2Zm4sDQo+PiAgICAgIGVsc2UNCj4+ICAgICAg
ICAgIGlvbW11X2VuYWJsZV9kZXZpY2UocGRldik7DQo+PiAgKyNpZmRlZiBDT05GSUdfQVJNDQo+
PiArICAgIHJldCA9IHZwY2lfYWRkX2hhbmRsZXJzKHBkZXYpOw0KPj4gKyAgICBpZiAoIHJldCAp
IHsNCj4+ICsgICAgICAgIHByaW50ayhYRU5MT0dfRVJSICJzZXR1cCBvZiB2UENJIGZvciBmYWls
ZWQ6ICVkXG4iLHJldCk7DQo+PiArICAgICAgICBnb3RvIG91dDsNCj4+ICsgICAgfQ0KPj4gKyNl
bmRpZg0KPj4gICAgICBwY2lfZW5hYmxlX2FjcyhwZGV2KTsNCj4+ICAgIG91dDoNCj4+IGRpZmYg
LS1naXQgYS94ZW4vZHJpdmVycy92cGNpL01ha2VmaWxlIGIveGVuL2RyaXZlcnMvdnBjaS9NYWtl
ZmlsZQ0KPj4gaW5kZXggNTVkMWJkZmRhMC4uMWExNDEzYjkzZSAxMDA2NDQNCj4+IC0tLSBhL3hl
bi9kcml2ZXJzL3ZwY2kvTWFrZWZpbGUNCj4+ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvTWFrZWZp
bGUNCj4+IEBAIC0xICsxLDIgQEANCj4+IC1vYmoteSArPSB2cGNpLm8gaGVhZGVyLm8gbXNpLm8g
bXNpeC5vDQo+PiArb2JqLXkgKz0gdnBjaS5vIGhlYWRlci5vDQo+PiArb2JqLSQoQ09ORklHX0hB
U19QQ0lfTVNJKSArPSBtc2kubyBtc2l4Lm8NCj4+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy92
cGNpL2hlYWRlci5jIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4gaW5kZXggYmE5YTAz
NjIwMi4uZjhjZDU1ZTdjMCAxMDA2NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVy
LmMNCj4+ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+IEBAIC05Niw4ICs5Niwx
MCBAQCBzdGF0aWMgdm9pZCBtb2RpZnlfZGVjb2RpbmcoY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBk
ZXYsIHVpbnQxNl90IGNtZCwNCj4+ICAgICAgICogRklYTUU6IHB1bmNoaW5nIGhvbGVzIGFmdGVy
IHRoZSBwMm0gaGFzIGJlZW4gc2V0IHVwIG1pZ2h0IGJlIHJhY3kgZm9yDQo+PiAgICAgICAqIERv
bVUgdXNhZ2UsIG5lZWRzIHRvIGJlIHJldmlzaXRlZC4NCj4+ICAgICAgICovDQo+PiArI2lmZGVm
IENPTkZJR19IQVNfUENJX01TSQ0KPj4gICAgICBpZiAoIG1hcCAmJiAhcm9tX29ubHkgJiYgdnBj
aV9tYWtlX21zaXhfaG9sZShwZGV2KSApDQo+PiAgICAgICAgICByZXR1cm47DQo+PiArI2VuZGlm
DQo+PiAgICAgICAgZm9yICggaSA9IDA7IGkgPCBBUlJBWV9TSVpFKGhlYWRlci0+YmFycyk7IGkr
KyApDQo+PiAgICAgIHsNCj4+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9hc20tYXJtL2RvbWFp
bi5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9kb21haW4uaA0KPj4gaW5kZXggYzkyNzdiNWM2ZC4u
ZDc0MmI5NGJkNiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vZG9tYWluLmgN
Cj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vZG9tYWluLmgNCj4+IEBAIC0yNjIsNyArMjYy
LDEwIEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBhcmNoX3ZjcHVfYmxvY2soc3RydWN0IHZjcHUgKnYp
IHt9DQo+PiAgICAjZGVmaW5lIGFyY2hfdm1fYXNzaXN0X3ZhbGlkX21hc2soZCkgKDFVTCA8PCBW
TUFTU1RfVFlQRV9ydW5zdGF0ZV91cGRhdGVfZmxhZykNCj4+ICAtI2RlZmluZSBoYXNfdnBjaShk
KSAgICAoeyAodm9pZCkoZCk7IGZhbHNlOyB9KQ0KPj4gKy8qIEZvciBYODYgVlBDSSBpcyBlbmFi
bGVkIGFuZCB0ZXN0ZWQgZm9yIFBWSCBET00wIG9ubHkgYnV0DQo+PiArICogZm9yIEFSTSB3ZSBl
bmFibGUgc3VwcG9ydCBWUENJIGZvciBndWVzdCBkb21haW4gYWxzby4NCj4+ICsgKi8NCj4+ICsj
ZGVmaW5lIGhhc192cGNpKGQpICh7ICh2b2lkKShkKTsgSVNfRU5BQkxFRChDT05GSUdfSEFTX1ZQ
Q0kpOyB9KQ0KPj4gICAgI2VuZGlmIC8qIF9fQVNNX0RPTUFJTl9IX18gKi8NCj4+ICBkaWZmIC0t
Z2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNp
LmgNCj4+IGluZGV4IDc1NmY4NjM3YWIuLmM1ODE1MmRlODAgMTAwNjQ0DQo+PiAtLS0gYS94ZW4v
aW5jbHVkZS9hc20tYXJtL3BjaS5oDQo+PiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5o
DQo+PiBAQCAtMjcsNiArMjcsMTQgQEAgc3RydWN0IGFyY2hfcGNpX2RldiB7DQo+PiAgICAgIHN0
cnVjdCBkZXZpY2UgZGV2Ow0KPj4gIH07DQo+PiAgKy8qIEFyY2gtc3BlY2lmaWMgTVNJIGRhdGEg
Zm9yIHZQQ0kuICovDQo+PiArc3RydWN0IHZwY2lfYXJjaF9tc2kgew0KPj4gK307DQo+PiArDQo+
PiArLyogQXJjaC1zcGVjaWZpYyBNU0ktWCBlbnRyeSBkYXRhIGZvciB2UENJLiAqLw0KPj4gK3N0
cnVjdCB2cGNpX2FyY2hfbXNpeF9lbnRyeSB7DQo+PiArfTsNCj4+ICsNCj4+ICAvKg0KPj4gICAq
IHN0cnVjdCB0byBob2xkIHRoZSBtYXBwaW5ncyBvZiBhIGNvbmZpZyBzcGFjZSB3aW5kb3cuIFRo
aXMNCj4+ICAgKiBpcyBleHBlY3RlZCB0byBiZSB1c2VkIGFzIHN5c2RhdGEgZm9yIFBDSSBjb250
cm9sbGVycyB0aGF0DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvcHVibGljL2FyY2gtYXJt
LmggYi94ZW4vaW5jbHVkZS9wdWJsaWMvYXJjaC1hcm0uaA0KPj4gaW5kZXggNjRhMmNhMzBkYS4u
MGE5NzQ5ZTc2OCAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL3B1YmxpYy9hcmNoLWFybS5o
DQo+PiArKysgYi94ZW4vaW5jbHVkZS9wdWJsaWMvYXJjaC1hcm0uaA0KPj4gQEAgLTQyMiw2ICs0
MjIsMTAgQEAgdHlwZWRlZiB1aW50NjRfdCB4ZW5fY2FsbGJhY2tfdDsNCj4+ICAjZGVmaW5lIEdV
RVNUX1BMMDExX0JBU0UgICAgeGVuX21rX3VsbG9uZygweDIyMDAwMDAwKQ0KPj4gICNkZWZpbmUg
R1VFU1RfUEwwMTFfU0laRSAgICB4ZW5fbWtfdWxsb25nKDB4MDAwMDEwMDApDQo+PiAgKy8qIFZQ
Q0kgRUNBTSBtYXBwaW5ncyAqLw0KPj4gKyNkZWZpbmUgR1VFU1RfVlBDSV9FQ0FNX0JBU0UgICAg
eGVuX21rX3VsbG9uZygweDEwMDAwMDAwKQ0KPiANCj4gQWxsIHRoZSB2YWx1ZXMgZm9yIHRoZSBt
ZW1vcnkgbGF5b3V0IGhhcyBiZWVuIGRlZmluZWQgaW4gYXNjZW5kaW5nIG9yZGVyLiBTbyBwbGVh
c2UgYWRkIHRoZSB2Q1BJIGF0IHRoZSBjb3JyZWN0IHBsYWNlLiBJZiBJIGFtIG5vdCBtaXN0YWtl
biwgdGhpcyBzaG91bGQgYmUgYmVmb3JlIHRoZSBHVUVTVF9BQ1BJXyouDQoNCkFjay4gDQo+IA0K
Pj4gKyNkZWZpbmUgR1VFU1RfVlBDSV9FQ0FNX1NJWkUgICAgeGVuX21rX3VsbG9uZygweDEwMDAw
MDAwKQ0KPiANCj4gUGxlYXNlIGRvY3VtZW50IGhvdyB0byBkZWNpZGUgdGhlIHNpemUuIFRoaXMg
aXMgaW1wb3J0YW50IGZvciB0aGUgZnV0dXJlIGlmIHdlIG5lZWQgdG8gY2hhbmdlIHRoZSBzaXpl
Lg0KDQpPayBsZXQgbWUgZG9jdW1lbnQgaW4gbmV4dCB2ZXJzaW9uLg0KDQovKiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIA0KICogMjU2IE1CIGlzIHJlc2VydmVkIGZvciBWUENJIGNvbmZpZ3VyYXRpb24g
c3BhY2UgYmFzZWQgb24gY2FsY3VsYXRpb24gICAgICAgICANCiAqIDI1NiBidXNlcyDDlyAzMiBk
ZXZpY2VzIMOXIDggZnVuY3Rpb25zIMOXIDQgS0IgPSAyNTYgTUIgICAgICAgICAgICAgICAgICAg
ICAgICAgDQogKi8gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIA0KI2RlZmluZSBHVUVTVF9WUENJX0VDQU1f
QkFTRSAgICB4ZW5fbWtfdWxsb25nKDB4MTAwMDAwMDApICAgICAgICAgICAgICAgICAgICAgICAN
CiNkZWZpbmUgR1VFU1RfVlBDSV9FQ0FNX1NJWkUgICAgeGVuX21rX3VsbG9uZygweDEwMDAwMDAw
KQ0KDQoNClJlZ3JhZHMsDQpSYWh1bA0KPiANCj4gQ2hlZXJzLA0KPiANCj4gLS0gDQo+IEp1bGll
biBHcmFsbA0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:52:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:52:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188358.337494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp0I-0001vx-1p; Thu, 16 Sep 2021 10:52:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188358.337494; Thu, 16 Sep 2021 10:52:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp0H-0001vq-Us; Thu, 16 Sep 2021 10:52:29 +0000
Received: by outflank-mailman (input) for mailman id 188358;
 Thu, 16 Sep 2021 10:52:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AVhn=OG=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQp0G-0001vk-Gd
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:52:28 +0000
Received: from mail-pg1-x531.google.com (unknown [2607:f8b0:4864:20::531])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3d460d58-1427-43ce-9436-276fe892b660;
 Thu, 16 Sep 2021 10:52:27 +0000 (UTC)
Received: by mail-pg1-x531.google.com with SMTP id h3so5760568pgb.7
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 03:52:27 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:18:efec::50b])
 by smtp.gmail.com with ESMTPSA id
 21sm2624212pfh.103.2021.09.16.03.52.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 03:52:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d460d58-1427-43ce-9436-276fe892b660
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=VNKbu9+xQ5ZpUtGvMTcqpK8Ho+xMSyVZ8+L+t3qGVwI=;
        b=XSZj6K6WXxfevgNkMQWTVWN/pRsGZ48Hpf1aapa7yFejBTCFdA6HUCPwkA38CUvHOb
         12EGRvxxebdCNuoE+ObAyvTNwWyxDdbNnDJYQNBCpkkMDkdT2sOdFGpKg9oYBfCBDwL0
         XflKHpxQXjZ521DE/0ZBRDz1uCKiQSaIrItzbkcc7HCVprwqAzWPmTfhfdTuxhcGIkae
         17ws4naiLQWag9g0raMGIBW/7G/5E7reQPZwNkqgzzUPtdDb0IjYRvViaMPYRdxPUde6
         Bl5njgBd1GAcMu72JAentZ78PNGD3qywdjPZPtQPg3O6M2RygtB2NwLk/e/0AhwfWnul
         yMPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=VNKbu9+xQ5ZpUtGvMTcqpK8Ho+xMSyVZ8+L+t3qGVwI=;
        b=Sv79o/8vV+WjuYfB0HCEl5wxMn5Hbh4Ac48XEU1ZLoXpG2tcz7aYsiB7Q18+ZuSwdh
         d9BuD88M0TYpsKdk0ttX3E7709O0DUetIskcwOMWN1BVlRzrghTAidRuGEodU1wNNXEj
         DfwRZIeH6CTLYdxosA/lGC/Xz0gilggsoZhu2PKfHGuq2IieTJIp/SV53N1InoS/83t3
         W16MCihpQciVbcK7uNq9bGSXkIFCVqZ7EVJZ9jmauMtPBG1yv6en6IJaeJc0uBgjl2Du
         wa6bflB5KnlRGKWvTdf+7hoNQ01uE4cVQQoH2zGeMpzqkm5InNPfTzb6HB01TB2Okq/A
         dE2g==
X-Gm-Message-State: AOAM530fUJdl/FIemQUtVXMfPsc5LLWeWVKS7VkwynZOA08Cf1XTLvHF
	VQiiP3MyytwjU+JSggoHvxs=
X-Google-Smtp-Source: ABdhPJyl+TWhw4qIPd+GNTqUwBCT0hUxX1fvNVQYkrBsEjf+FvFMSf2MpaDHDZtEUEbJwfeL0pUlhg==
X-Received: by 2002:a63:4614:: with SMTP id t20mr4441488pga.372.1631789546934;
        Thu, 16 Sep 2021 03:52:26 -0700 (PDT)
Subject: Re: [PATCH V5 07/12] Drivers: hv: vmbus: Add SNP support for VMbus
 channel initiate message
To: Michael Kelley <mikelley@microsoft.com>, KY Srinivasan
 <kys@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-8-ltykernel@gmail.com>
 <MWHPR21MB1593D9CB27D41B128BF21DC9D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <15d177f8-5e20-8630-2b2c-8a00a5309a61@gmail.com>
Date: Thu, 16 Sep 2021 18:52:12 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB1593D9CB27D41B128BF21DC9D7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/15/2021 11:41 PM, Michael Kelley wrote:
>> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
>> index 42f3d9d123a1..560cba916d1d 100644
>> --- a/drivers/hv/hyperv_vmbus.h
>> +++ b/drivers/hv/hyperv_vmbus.h
>> @@ -240,6 +240,8 @@ struct vmbus_connection {
>>   	 * is child->parent notification
>>   	 */
>>   	struct hv_monitor_page *monitor_pages[2];
>> +	void *monitor_pages_original[2];
>> +	unsigned long monitor_pages_pa[2];
> The type of this field really should be phys_addr_t.  In addition to
> just making semantic sense, then it will match the return type from
> virt_to_phys() and the input arg to memremap() since resource_size_t
> is typedef'ed as phys_addr_t.
> 

OK. Will update in the next version.

Thanks.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 10:57:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 10:57:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188366.337504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp5F-0002m0-Je; Thu, 16 Sep 2021 10:57:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188366.337504; Thu, 16 Sep 2021 10:57:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp5F-0002lt-Gm; Thu, 16 Sep 2021 10:57:37 +0000
Received: by outflank-mailman (input) for mailman id 188366;
 Thu, 16 Sep 2021 10:57:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AVhn=OG=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQp5E-0002ln-7N
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 10:57:36 +0000
Received: from mail-pf1-x431.google.com (unknown [2607:f8b0:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 08f3daf8-3b79-46e6-a20e-bd1c8b913bd2;
 Thu, 16 Sep 2021 10:57:35 +0000 (UTC)
Received: by mail-pf1-x431.google.com with SMTP id y8so5495284pfa.7
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 03:57:35 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 n141sm2856237pfd.90.2021.09.16.03.57.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 03:57:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08f3daf8-3b79-46e6-a20e-bd1c8b913bd2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=zetJ5G0LMyibhhBY6/35VWLw2vAGWh3kCNdD5BxkSZI=;
        b=Iq+3mvGj2jY0H1hK5L2+2LJkiT3nS6liFloV8Frq8+FJ3Y0LxB2A38VNCla7awjuhj
         Q3Cl5qmmvoV1YWOZQweI486bIgofq+t3jwjAQ3GGQ1BTx00wXTwDzZt3wO26vvMZXlEU
         Srw78VlxUG3ClRIenDB9DS9KklBz8Gy2Kt0AIAheRW83PteD9N403WMbpA39HXJuq8IF
         JiYUmg4zDGjeRBVLjEF25+aPcjKe8oOTUL3KuuMaaLo1G4aP3SLUOfZT+6b94AvHMe5Q
         tcf8/9xlWcz7WQzaFSNkMRaXjJszFIpx0K2ojEu+E4bzhBM/xSgQmOnlb5Oz/8Q8cN9d
         Qp1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=zetJ5G0LMyibhhBY6/35VWLw2vAGWh3kCNdD5BxkSZI=;
        b=WxfeuD+9X0GjDejHmTNoMGqGRZ703w+G0LNIRmNKO2ez6KSfZBtc4tybKZSwXcOyr1
         sov8Di1Ubk/WQk2+84uCkwFyFtGXHgZTENl3dC4SQc2/yn5/2KoXyN4g+bg3tQ56bCEd
         WLmTokYw+4mtCwwyA1bww8N8AHff3czR8gkF+Ku4Y7uHupz/g0e4Gzq7oDQL1i1+MwNl
         dxyUplPXTBoBba5ULW6dFTUNQX/eU5U0X1aCNAQoBW+sk0UgUar9T+Hewc+1QxZb5e66
         mUBAWZafjSEXUBovXsTsRVi9JxSn75WFYgfZrV/N2AzctiS9WJYY5OZrSBtlS8mNc6SZ
         3M4Q==
X-Gm-Message-State: AOAM533OVY4mSYDkft7HRiZMBmZ6sWezgN+itOeLas+xBwj/4vEN4qJw
	U8ftYbjrVdRu+1h0G0YELKU=
X-Google-Smtp-Source: ABdhPJyJnHqPNf+opjGA7LpPgHFY6nzsERWLwilu6XCHi7kaSPNw+MpELyJKztS12zzhPWoDpdap2Q==
X-Received: by 2002:aa7:959a:0:b0:43b:adeb:ef58 with SMTP id z26-20020aa7959a000000b0043badebef58mr4426857pfj.19.1631789854486;
        Thu, 16 Sep 2021 03:57:34 -0700 (PDT)
Subject: Re: [PATCH V5 09/12] x86/Swiotlb: Add Swiotlb bounce buffer remap
 function for HV IVM
To: Michael Kelley <mikelley@microsoft.com>, KY Srinivasan
 <kys@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-10-ltykernel@gmail.com>
 <MWHPR21MB159349234C15D0F04F87845CD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <3f6a6407-b8fd-803b-e4fe-ea9a873a5840@gmail.com>
Date: Thu, 16 Sep 2021 18:57:20 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB159349234C15D0F04F87845CD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/15/2021 11:42 PM, Michael Kelley wrote:
>> @@ -196,13 +199,34 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t start,
>>   		mem->slots[i].orig_addr = INVALID_PHYS_ADDR;
>>   		mem->slots[i].alloc_size = 0;
>>   	}
>> +
>> +	if (set_memory_decrypted((unsigned long)vaddr, bytes >> PAGE_SHIFT))
>> +		return -EFAULT;
>> +
>> +	/*
>> +	 * Map memory in the unencrypted physical address space when requested
>> +	 * (e.g. for Hyper-V AMD SEV-SNP Isolation VMs).
>> +	 */
>> +	if (swiotlb_unencrypted_base) {
>> +		phys_addr_t paddr = __pa(vaddr) + swiotlb_unencrypted_base;
> Nit:  Use "start" instead of "__pa(vaddr)" since "start" is already the needed
> physical address.


Yes, "start" should be used here.

> 
>> @@ -304,7 +332,7 @@ int
>>   swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
>>   {
>>   	struct io_tlb_mem *mem = &io_tlb_default_mem;
>> -	unsigned long bytes = nslabs << IO_TLB_SHIFT;
>> +	int ret;
>>
>>   	if (swiotlb_force == SWIOTLB_NO_FORCE)
>>   		return 0;
>> @@ -318,8 +346,9 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
>>   	if (!mem->slots)
>>   		return -ENOMEM;
>>
>> -	set_memory_decrypted((unsigned long)tlb, bytes >> PAGE_SHIFT);
>> -	swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
>> +	ret = swiotlb_init_io_tlb_mem(mem, virt_to_phys(tlb), nslabs, true);
>> +	if (ret)
> Before returning the error, free the pages obtained from the earlier call
> to __get_free_pages()?
> 

Yes, will fix.

Thanks.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 11:01:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 11:01:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188373.337516 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp9I-0004Ay-5k; Thu, 16 Sep 2021 11:01:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188373.337516; Thu, 16 Sep 2021 11:01:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQp9I-0004Ar-1o; Thu, 16 Sep 2021 11:01:48 +0000
Received: by outflank-mailman (input) for mailman id 188373;
 Thu, 16 Sep 2021 11:01:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQp9G-0004Al-Rt
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 11:01:46 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.56]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7ad617c4-16dd-11ec-b5cd-12813bfff9fa;
 Thu, 16 Sep 2021 11:01:45 +0000 (UTC)
Received: from AM6P192CA0071.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:82::48)
 by AM0PR08MB3059.eurprd08.prod.outlook.com (2603:10a6:208:5c::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 11:01:41 +0000
Received: from VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:82:cafe::bd) by AM6P192CA0071.outlook.office365.com
 (2603:10a6:209:82::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 16 Sep 2021 11:01:41 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT020.mail.protection.outlook.com (10.152.18.242) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:01:41 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Thu, 16 Sep 2021 11:01:41 +0000
Received: from 1455e65aae2e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8B25FB05-9BA9-4251-99C8-2BAED06518A3.1; 
 Thu, 16 Sep 2021 11:01:34 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1455e65aae2e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 11:01:34 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB5238.eurprd08.prod.outlook.com (2603:10a6:20b:e5::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 11:01:31 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 11:01:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ad617c4-16dd-11ec-b5cd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TF3bqlX9Ob/fzKijUBIxig+D2ujKb5+yLTxgtpcicX4=;
 b=vF2V4DPTIqPe4fIb6/8lwfIzKd72bzydw2k2BlK+oB3jOWTfNT54imGpog/Re2/FQWPExd5VY7hzxM2JU/78MxMDiGhuBdSewV6kjWQcfhygHUfDQisOEUWg9qWeEheoT0LmbmsvV9BedZ3qeohrD0bAIML+vOcHH6UTJxymIsw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: db2f20225d3206d8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WYXJuUmPml5e9GzELIpSzQ4rSQ8+d1HFu2bAlfd0hgeQK66tL12Bivm092PwuiiVxLmf5SXD0jLkXPBAOnNkv/FU3eqY5Tl5GsOFpfJpLoiMLs7d6t3PS0T/2ZtfPu/j0yghI/4fcoi4IM/s/FiznpLMacPI6c4UQy81pFyzwwdRLAgjBaoLY8wdy9e67A+m9M4R4G1XUMpuJ7AGpyhVLCxWiI7ZbxTNa6hhypy0+SfykpC9PVO3ThWDXb/kazPx92kqXwa+ZzftH8xpNGcvEC1UUERnOVutlX8cbIvqFnKRJE1Eo8RmeVxG8LY8nT/k850Up6IqEeu2iMJ3SsvXFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TF3bqlX9Ob/fzKijUBIxig+D2ujKb5+yLTxgtpcicX4=;
 b=bwemKVF8U/42xvA2MGUgCjHC5w1UF90Ximu9puo1dsBpCbMAUzJLNJODMsidbihplo1PE/73om/FgWwthcqyipnZZ72WC6jMQJuAMAuECm3pt/mT3YnM+waF+qAOma6c+cvmBPJkgzyocEopJ6em7LEIHR0ZjhQQeJCQCYkeKxqXIWir+i9woG9TeSTWdyXEGE0k1/STw0PVEnAWhHujV40853pH/SjIP2y0T9XnCpjESjBsbN0b+TUfw2yYeWiS51D8/wYDyK8F0A2oqo2kutcrghRfzz2Hcq+sOkafnkfHjHliBnMtuZuw4I0pfKY7CI8m7EvrYe17p7g7I4D4pw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TF3bqlX9Ob/fzKijUBIxig+D2ujKb5+yLTxgtpcicX4=;
 b=vF2V4DPTIqPe4fIb6/8lwfIzKd72bzydw2k2BlK+oB3jOWTfNT54imGpog/Re2/FQWPExd5VY7hzxM2JU/78MxMDiGhuBdSewV6kjWQcfhygHUfDQisOEUWg9qWeEheoT0LmbmsvV9BedZ3qeohrD0bAIML+vOcHH6UTJxymIsw=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Jan Beulich <jbeulich@suse.com>, Paul Durrant
	<paul@xen.org>, =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXlPL13zcSD0YvP0y1H1bgZa9bP6uciyMAgAofYIA=
Date: Thu, 16 Sep 2021 11:01:31 +0000
Message-ID: <A90145B0-DD30-4300-8248-73104C782172@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: d7d3129c-2f65-4a30-dabc-08d979015d38
x-ms-traffictypediagnostic: AM6PR08MB5238:|AM0PR08MB3059:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB305940C737EAB87750F389E7FCDC9@AM0PR08MB3059.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 RPG8/OEcMZxypEFQI5P0LWiJSVjeeDKk0yscKryfD+mtL7g8w2w0E2M77w7WHAJ4SvqW1U6hj73DzTlI3tnSlPSX/wBBQ5dCHvXlWyC9q6Fm5Ys49i/QywXEgHroRD2ILfHqaWLGELreEp21Ilw7hrx53/IW6T7cxFkgEln/b/CJOD1g89M7yYT8TThkWXUkLWpm4wBpUULlaVO1mCxjTGJHWPz+ecFkR/uUdEXZxet6DUanY0/2aZK4uG9MTQG/5QpCE47QEzh2+10Za0b6RfLvaa6AUcBiTAfTHMlfXgCnPUOOytAaAMxHe4oztm92tGZWrykGoLopfLB+LVAzVThZBbp/Lv6cPurbUTorRZEVpbA1C6d+4TEPSyaJ3tBYMYG0KjAmho+uOLSWt4yw1Pm4DA1Emf6I94IA6GzVu4LOvHu1gLpghwLNrwiibBYQopWXhBmCw8S4pMeE4Ba9ODmsjRmQPKvpdepJFxvZwwKFjsHkv4EQ2fx2UvtFXKgoLErGk0b93h4RKkWqpSdspiQyCq7KNqmb9xSVLsBwI8055Ed7PUwte/QJnA6q/p1I/EkRST2DlNnvuI4JcMP5zY/eW/VYLf9RuoNdAoEO2FJrGPn6JerNRNCcV8iNb4XPn6mou4FrFZaEk7e/A0GBLAIk4KIM/w4GBNSE57p298bjttp2WE40v6Dq2rMFvufEa26iaqsdVmTP6lzx/ZjJfKoDqXfWA5fXc9eKPJZ5/T3w/B1vYudZbFcuSVShbAG7PhPzopFIhaIkHsx73MphqQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(39860400002)(376002)(136003)(396003)(346002)(36756003)(33656002)(38100700002)(4326008)(54906003)(86362001)(71200400001)(122000001)(38070700005)(2906002)(8676002)(2616005)(316002)(5660300002)(6512007)(478600001)(26005)(8936002)(30864003)(186003)(66946007)(76116006)(91956017)(6916009)(83380400001)(53546011)(6506007)(6486002)(66446008)(66556008)(64756008)(66476007)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <CCC5FC31FBD88C49B466C575C56A8310@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5238
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f90d4a9c-ae3f-4425-1365-08d97901576d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	r2b4rifZc23bNQqrt30/yA5BG512XtCRfHQf+v8fZk8SDND8jD+T97SMdg5uR9XZ/a/35eVoSzDMH/+Gs5HO4WDOJ9dklZbAAHjjdy3LqZCS1ocMYrMDq5ygv8800NTSwnHPDE9rs5GTFm1XG7eOz5YvsxOcD50TtfaXW/taWdrrh9dyOz+BHP27nIWy1dBwkMvam1RDAGC5qggrjpOn9Daz1hx7Fcj6BJfngqhD8t51AWgSbNKqw+o28vISK/GCxB58ezP5b12tXClDOlu6sKhjJu/itx80UxC6Jf86A5JYWlq9KaNmar9sFxdgt/QWxo7O/14pJC+K34fWfhfyK1t9wmafY+/PHtEZPawp6V6e+aDtFXSCrSicOjCOPBA++d0JnFdJIaEI0nr1OXIYKYhLpwVTTlRfuUvz6nHmVB6wznZ+KTaEARvc6Teb/kAPx7YU01pAb28FsUHm/Rd75ro4wHRzTdiwvanuDf2L9DNOtzssOp6oXRATSRbeKPsh+d63Pup2A0WEjad0tJM2F4egroiCbwt4NcraudhlDzQNh1URy03jVubtt9y2bh7gtuoqHeSfL5dWeghHZbNdkMpZdo8aPeQ9T+qmPU0Sc9ox+JO4PZCeDLETroQmP1DS978lA6KThYI1fbhauXOK+R9FlXRkMupb5jv4aOKigO6crfMjJkpcdAlKfaJxhPOSW0h2ZQGb0PJRA+uuYejr3rpb/mycDfWWC1hvQrborIE=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(396003)(39860400002)(376002)(136003)(36840700001)(46966006)(8936002)(70206006)(186003)(26005)(70586007)(47076005)(478600001)(36860700001)(6512007)(4326008)(30864003)(81166007)(5660300002)(356005)(82740400003)(83380400001)(2616005)(316002)(33656002)(336012)(82310400003)(53546011)(6506007)(6486002)(36756003)(54906003)(6862004)(107886003)(8676002)(2906002)(86362001)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 11:01:41.4181
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d7d3129c-2f65-4a30-dabc-08d979015d38
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3059

SGkgU3RlZmFubywNCg0KPiBPbiAxMCBTZXAgMjAyMSwgYXQgMToyNiBhbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFRodSwgMTkg
QXVnIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4gVGhlIGV4aXN0aW5nIFZQQ0kgc3VwcG9y
dCBhdmFpbGFibGUgZm9yIFg4NiBpcyBhZGFwdGVkIGZvciBBcm0uDQo+PiBXaGVuIHRoZSBkZXZp
Y2UgaXMgYWRkZWQgdG8gWEVOIHZpYSB0aGUgaHlwZXIgY2FsbA0KPj4g4oCcUEhZU0RFVk9QX3Bj
aV9kZXZpY2VfYWRk4oCdLCBWUENJIGhhbmRsZXIgZm9yIHRoZSBjb25maWcgc3BhY2UNCj4+IGFj
Y2VzcyBpcyBhZGRlZCB0byB0aGUgWGVuIHRvIGVtdWxhdGUgdGhlIFBDSSBkZXZpY2VzIGNvbmZp
ZyBzcGFjZS4NCj4gDQo+IFRoaXMgaXMgZG9uZSBqdXN0IGZvciBkZXZpY2UgZGlzY292ZXJ5LCBy
aWdodD8NCj4gDQo+IEFsdGhvdWdoIGl0IGlzIGN1cnJlbnRseSBub3QgaW1wbGVtZW50ZWQgKGFu
ZCBJIGFtIG5vdCBhc2tpbmcgdG8NCj4gaW1wbGVtZW50IGl0IG5vdywgSSBhbSBvbmx5IHRyeWlu
ZyB0byB1bmRlcnN0YW5kIHRoZSBhcmNoaXRlY3R1cmUpLCBpdA0KPiB3b3VsZCBiZSBwb3NzaWJs
ZSB0byBkaXNjb3ZlciBhbGwgUENJIGRldmljZXMganVzdCBieSB3YWxraW5nIGRvd24gdGhlDQo+
IFBDSSBoaWVyYXJjaHkgYnkgb3Vyc2VsdmVzIGluIFhlbiAobm8gRG9tMCBpbnRlcmFjdGlvbnMp
IGdpdmVuIHRoYXQgd2UNCj4gaGF2ZSBhbiBFQ0FNIGRyaXZlci4NCj4gDQo+IEkgdGFrZSB0aGF0
IHdvdWxkIGJlIHRoZSB3YXkgdG8gaW1wbGVtZW50IFBDSSBzdXBwb3J0IGZvciBEb20wbGVzcz8N
Cg0KSXQgaXMgbm90IHBvc3NpYmxlIHRvIGRpc2NvdmVyIFBDSSBkZXZpY2VzIGluIFhFTiBpZiBl
bnVtZXJhdGlvbiBpcyBub3QgZG9uZSBiZWZvcmUgWEVOIGJvb3QuDQpJZiBib290IGZpcm13YXJl
IGRpZCB0aGUgZW51bWVyYXRpb24sICBYRU4gd2lsbCBkaXNjb3ZlciB0aGUgUENJIGRldmljZS4N
Cg0KPiANCj4gDQo+PiBBIE1NSU8gdHJhcCBoYW5kbGVyIGZvciB0aGUgUENJIEVDQU0gc3BhY2Ug
aXMgcmVnaXN0ZXJlZCBpbiBYRU4NCj4+IHNvIHRoYXQgd2hlbiBndWVzdCBpcyB0cnlpbmcgdG8g
YWNjZXNzIHRoZSBQQ0kgY29uZmlnIHNwYWNlLFhFTg0KPj4gd2lsbCB0cmFwIHRoZSBhY2Nlc3Mg
YW5kIGVtdWxhdGUgcmVhZC93cml0ZSB1c2luZyB0aGUgVlBDSSBhbmQNCj4+IG5vdCB0aGUgcmVh
bCBQQ0kgaGFyZHdhcmUuDQo+PiANCj4+IFNpZ25lZC1vZmYtYnk6IFJhaHVsIFNpbmdoIDxyYWh1
bC5zaW5naEBhcm0uY29tPg0KPj4gLS0tDQo+PiB4ZW4vYXJjaC9hcm0vTWFrZWZpbGUgICAgICAg
ICB8ICAxICsNCj4+IHhlbi9hcmNoL2FybS9kb21haW4uYyAgICAgICAgIHwgIDQgKysNCj4+IHhl
bi9hcmNoL2FybS92cGNpLmMgICAgICAgICAgIHwgOTYgKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysNCj4+IHhlbi9hcmNoL2FybS92cGNpLmggICAgICAgICAgIHwgMzcgKysrKysr
KysrKysrKysNCj4+IHhlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jIHwgIDcgKysrDQo+PiB4
ZW4vZHJpdmVycy92cGNpL01ha2VmaWxlICAgICB8ICAzICstDQo+PiB4ZW4vZHJpdmVycy92cGNp
L2hlYWRlci5jICAgICB8ICAyICsNCj4+IHhlbi9pbmNsdWRlL2FzbS1hcm0vZG9tYWluLmggIHwg
IDUgKy0NCj4+IHhlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmggICAgIHwgIDggKysrDQo+PiB4ZW4v
aW5jbHVkZS9wdWJsaWMvYXJjaC1hcm0uaCB8ICA0ICsrDQo+PiAxMCBmaWxlcyBjaGFuZ2VkLCAx
NjUgaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkNCj4+IGNyZWF0ZSBtb2RlIDEwMDY0NCB4
ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3Zw
Y2kuaA0KPj4gDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL01ha2VmaWxlIGIveGVuL2Fy
Y2gvYXJtL01ha2VmaWxlDQo+PiBpbmRleCAwZTE0YTVlNWM4Li43Y2RjZTY4NGE0IDEwMDY0NA0K
Pj4gLS0tIGEveGVuL2FyY2gvYXJtL01ha2VmaWxlDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vTWFr
ZWZpbGUNCj4+IEBAIC03LDYgKzcsNyBAQCBvYmoteSArPSBwbGF0Zm9ybXMvDQo+PiBlbmRpZg0K
Pj4gb2JqLSQoQ09ORklHX1RFRSkgKz0gdGVlLw0KPj4gb2JqLSQoQ09ORklHX0hBU19QQ0kpICs9
IHBjaS8NCj4+ICtvYmotJChDT05GSUdfSEFTX1ZQQ0kpICs9IHZwY2kubw0KPj4gDQo+PiBvYmot
JChDT05GSUdfSEFTX0FMVEVSTkFUSVZFKSArPSBhbHRlcm5hdGl2ZS5vDQo+PiBvYmoteSArPSBi
b290ZmR0LmluaXQubw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9kb21haW4uYyBiL3hl
bi9hcmNoL2FybS9kb21haW4uYw0KPj4gaW5kZXggMTljNzU2YWMzZC4uZDk5YzY1MzYyNiAxMDA2
NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9kb21haW4uYw0KPj4gKysrIGIveGVuL2FyY2gvYXJt
L2RvbWFpbi5jDQo+PiBAQCAtNDAsNiArNDAsNyBAQA0KPj4gI2luY2x1ZGUgPGFzbS92dGltZXIu
aD4NCj4+IA0KPj4gI2luY2x1ZGUgInZ1YXJ0LmgiDQo+PiArI2luY2x1ZGUgInZwY2kuaCINCj4+
IA0KPj4gREVGSU5FX1BFUl9DUFUoc3RydWN0IHZjcHUgKiwgY3Vycl92Y3B1KTsNCj4+IA0KPj4g
QEAgLTc2Nyw2ICs3NjgsOSBAQCBpbnQgYXJjaF9kb21haW5fY3JlYXRlKHN0cnVjdCBkb21haW4g
KmQsDQo+PiAgICAgaWYgKCBpc19oYXJkd2FyZV9kb21haW4oZCkgJiYgKHJjID0gZG9tYWluX3Z1
YXJ0X2luaXQoZCkpICkNCj4+ICAgICAgICAgZ290byBmYWlsOw0KPj4gDQo+PiArICAgIGlmICgg
KHJjID0gZG9tYWluX3ZwY2lfaW5pdChkKSkgIT0gMCApDQo+PiArICAgICAgICBnb3RvIGZhaWw7
DQo+PiArDQo+PiAgICAgcmV0dXJuIDA7DQo+PiANCj4+IGZhaWw6DQo+PiBkaWZmIC0tZ2l0IGEv
eGVuL2FyY2gvYXJtL3ZwY2kuYyBiL3hlbi9hcmNoL2FybS92cGNpLmMNCj4+IG5ldyBmaWxlIG1v
ZGUgMTAwNjQ0DQo+PiBpbmRleCAwMDAwMDAwMDAwLi5kYThiMWNhMTNjDQo+PiAtLS0gL2Rldi9u
dWxsDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiBAQCAtMCwwICsxLDk2IEBADQo+
PiArLyoNCj4+ICsgKiB4ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiArICogQ29weXJpZ2h0IChjKSAy
MDIxIEFybSBMdGQuDQo+PiArICoNCj4+ICsgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2Fy
ZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgaXQgYW5kL29yIG1vZGlmeQ0KPj4gKyAqIGl0IHVuZGVy
IHRoZSB0ZXJtcyBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVk
IGJ5DQo+PiArICogdGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbjsgZWl0aGVyIHZlcnNpb24g
MiBvZiB0aGUgTGljZW5zZSwgb3INCj4+ICsgKiAoYXQgeW91ciBvcHRpb24pIGFueSBsYXRlciB2
ZXJzaW9uLg0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRo
ZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdB
UlJBTlRZOyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJD
SEFOVEFCSUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhl
DQo+PiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+
ICsgKi8NCj4+ICsjaW5jbHVkZSA8eGVuL3NjaGVkLmg+DQo+PiArI2luY2x1ZGUgPGFzbS9tbWlv
Lmg+DQo+PiArDQo+PiArLyogRG8gc29tZSBzYW5pdHkgY2hlY2tzLiAqLw0KPj4gK3N0YXRpYyBi
b29sIHZwY2lfbW1pb19hY2Nlc3NfYWxsb3dlZCh1bnNpZ25lZCBpbnQgcmVnLCB1bnNpZ25lZCBp
bnQgbGVuKQ0KPj4gK3sNCj4+ICsgICAgLyogQ2hlY2sgYWNjZXNzIHNpemUuICovDQo+PiArICAg
IGlmICggbGVuICE9IDEgJiYgbGVuICE9IDIgJiYgbGVuICE9IDQgJiYgbGVuICE9IDggKQ0KPj4g
KyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKw0KPj4gKyAgICAvKiBDaGVjayB0aGF0IGFjY2Vz
cyBpcyBzaXplIGFsaWduZWQuICovDQo+PiArICAgIGlmICggKHJlZyAmIChsZW4gLSAxKSkgKQ0K
Pj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gdHJ1ZTsNCj4+
ICt9DQo+PiArDQo+PiArc3RhdGljIGludCB2cGNpX21taW9fcmVhZChzdHJ1Y3QgdmNwdSAqdiwg
bW1pb19pbmZvX3QgKmluZm8sDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICByZWdpc3Rl
cl90ICpyLCB2b2lkICpwKQ0KPj4gK3sNCj4+ICsgICAgdW5zaWduZWQgaW50IHJlZzsNCj4+ICsg
ICAgcGNpX3NiZGZfdCBzYmRmOw0KPj4gKyAgICB1aW50MzJfdCBkYXRhID0gMDsNCj4+ICsgICAg
dW5zaWduZWQgaW50IHNpemUgPSAxVSA8PCBpbmZvLT5kYWJ0LnNpemU7DQo+PiArDQo+PiArICAg
IHNiZGYuc2JkZiA9ICgoKGluZm8tPmdwYSkgJiAweDBmZmZmMDAwKSA+PiAxMik7DQo+PiArICAg
IHJlZyA9ICgoKGluZm8tPmdwYSkgJiAweDAwMDAwZmZjKSB8IChpbmZvLT5ncGEgJiAzKSk7DQo+
PiArDQo+PiArICAgIGlmICggIXZwY2lfbW1pb19hY2Nlc3NfYWxsb3dlZChyZWcsIHNpemUpICkN
Cj4+ICsgICAgICAgIHJldHVybiAxOw0KPj4gKw0KPj4gKyAgICBkYXRhID0gdnBjaV9yZWFkKHNi
ZGYsIHJlZywgc2l6ZSk7DQo+PiArDQo+PiArICAgIG1lbWNweShyLCAmZGF0YSwgc2l6ZSk7DQo+
PiArDQo+PiArICAgIHJldHVybiAxOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW50IHZwY2lf
bW1pb193cml0ZShzdHJ1Y3QgdmNwdSAqdiwgbW1pb19pbmZvX3QgKmluZm8sDQo+PiArICAgICAg
ICAgICAgICAgICAgICAgICAgICAgcmVnaXN0ZXJfdCByLCB2b2lkICpwKQ0KPj4gK3sNCj4+ICsg
ICAgdW5zaWduZWQgaW50IHJlZzsNCj4+ICsgICAgcGNpX3NiZGZfdCBzYmRmOw0KPj4gKyAgICB1
aW50MzJfdCBkYXRhID0gcjsNCj4+ICsgICAgdW5zaWduZWQgaW50IHNpemUgPSAxVSA8PCBpbmZv
LT5kYWJ0LnNpemU7DQo+PiArDQo+PiArICAgIHNiZGYuc2JkZiA9ICgoKGluZm8tPmdwYSkgJiAw
eDBmZmZmMDAwKSA+PiAxMik7DQo+PiArICAgIHJlZyA9ICgoKGluZm8tPmdwYSkgJiAweDAwMDAw
ZmZjKSB8IChpbmZvLT5ncGEgJiAzKSk7DQo+PiArDQo+PiArICAgIGlmICggIXZwY2lfbW1pb19h
Y2Nlc3NfYWxsb3dlZChyZWcsIHNpemUpICkNCj4+ICsgICAgICAgIHJldHVybiAxOw0KPj4gKw0K
Pj4gKyAgICB2cGNpX3dyaXRlKHNiZGYsIHJlZywgc2l6ZSwgZGF0YSk7DQo+PiArDQo+PiArICAg
IHJldHVybiAxOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgY29uc3Qgc3RydWN0IG1taW9faGFu
ZGxlcl9vcHMgdnBjaV9tbWlvX2hhbmRsZXIgPSB7DQo+PiArICAgIC5yZWFkICA9IHZwY2lfbW1p
b19yZWFkLA0KPj4gKyAgICAud3JpdGUgPSB2cGNpX21taW9fd3JpdGUsDQo+PiArfTsNCj4+ICsN
Cj4+ICtpbnQgZG9tYWluX3ZwY2lfaW5pdChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gK3sNCj4+ICsg
ICAgaWYgKCAhaGFzX3ZwY2koZCkgKQ0KPj4gKyAgICAgICAgcmV0dXJuIDA7DQo+PiArDQo+PiAr
ICAgIHJlZ2lzdGVyX21taW9faGFuZGxlcihkLCAmdnBjaV9tbWlvX2hhbmRsZXIsDQo+PiArICAg
ICAgICAgICAgICAgICAgICAgICAgICBHVUVTVF9WUENJX0VDQU1fQkFTRSwgR1VFU1RfVlBDSV9F
Q0FNX1NJWkUsIE5VTEwpOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gMDsNCj4+ICt9DQo+PiArDQo+
PiArLyoNCj4+ICsgKiBMb2NhbCB2YXJpYWJsZXM6DQo+PiArICogbW9kZTogQw0KPj4gKyAqIGMt
ZmlsZS1zdHlsZTogIkJTRCINCj4+ICsgKiBjLWJhc2ljLW9mZnNldDogNA0KPj4gKyAqIGluZGVu
dC10YWJzLW1vZGU6IG5pbA0KPj4gKyAqIEVuZDoNCj4+ICsgKi8NCj4+ICsNCj4+IGRpZmYgLS1n
aXQgYS94ZW4vYXJjaC9hcm0vdnBjaS5oIGIveGVuL2FyY2gvYXJtL3ZwY2kuaA0KPj4gbmV3IGZp
bGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4IDAwMDAwMDAwMDAuLjhhMDkzYmI3MDUNCj4+IC0tLSAv
ZGV2L251bGwNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS92cGNpLmgNCj4+IEBAIC0wLDAgKzEsMzcg
QEANCj4+ICsvKg0KPj4gKyAqIHhlbi9hcmNoL2FybS92cGNpLmgNCj4+ICsgKiBDb3B5cmlnaHQg
KGMpIDIwMjEgQXJtIEx0ZC4NCj4+ICsgKg0KPj4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNv
ZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9kaWZ5DQo+PiArICogaXQg
dW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBhcyBwdWJs
aXNoZWQgYnkNCj4+ICsgKiB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uOyBlaXRoZXIgdmVy
c2lvbiAyIG9mIHRoZSBMaWNlbnNlLCBvcg0KPj4gKyAqIChhdCB5b3VyIG9wdGlvbikgYW55IGxh
dGVyIHZlcnNpb24uDQo+PiArICoNCj4+ICsgKiBUaGlzIHByb2dyYW0gaXMgZGlzdHJpYnV0ZWQg
aW4gdGhlIGhvcGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1bCwNCj4+ICsgKiBidXQgV0lUSE9VVCBB
TlkgV0FSUkFOVFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGllZCB3YXJyYW50eSBvZg0KPj4gKyAq
IE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIgUFVSUE9TRS4gIFNl
ZSB0aGUNCj4+ICsgKiBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBmb3IgbW9yZSBkZXRhaWxz
Lg0KPj4gKyAqLw0KPj4gKw0KPj4gKyNpZm5kZWYgX19BUkNIX0FSTV9WUENJX0hfXw0KPj4gKyNk
ZWZpbmUgX19BUkNIX0FSTV9WUENJX0hfXw0KPj4gKw0KPj4gKyNpZmRlZiBDT05GSUdfSEFTX1ZQ
Q0kNCj4+ICtpbnQgZG9tYWluX3ZwY2lfaW5pdChzdHJ1Y3QgZG9tYWluICpkKTsNCj4+ICsjZWxz
ZQ0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IGRvbWFpbl92cGNpX2luaXQoc3RydWN0IGRvbWFpbiAq
ZCkNCj4+ICt7DQo+PiArICAgIHJldHVybiAwOw0KPj4gK30NCj4+ICsjZW5kaWYNCj4+ICsNCj4+
ICsjZW5kaWYgLyogX19BUkNIX0FSTV9WUENJX0hfXyAqLw0KPj4gKw0KPj4gKy8qDQo+PiArICog
TG9jYWwgdmFyaWFibGVzOg0KPj4gKyAqIG1vZGU6IEMNCj4+ICsgKiBjLWZpbGUtc3R5bGU6ICJC
U0QiDQo+PiArICogYy1iYXNpYy1vZmZzZXQ6IDQNCj4+ICsgKiBpbmRlbnQtdGFicy1tb2RlOiBu
aWwNCj4+ICsgKiBFbmQ6DQo+PiArICovDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2RyaXZlcnMvcGFz
c3Rocm91Z2gvcGNpLmMgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gaW5kZXgg
YzIzYzhjYjA2Yi4uNTZlMjYxZTliZCAxMDA2NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0
aHJvdWdoL3BjaS5jDQo+PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4g
QEAgLTc2Nyw2ICs3NjcsMTMgQEAgaW50IHBjaV9hZGRfZGV2aWNlKHUxNiBzZWcsIHU4IGJ1cywg
dTggZGV2Zm4sDQo+PiAgICAgZWxzZQ0KPj4gICAgICAgICBpb21tdV9lbmFibGVfZGV2aWNlKHBk
ZXYpOw0KPj4gDQo+PiArI2lmZGVmIENPTkZJR19BUk0NCj4+ICsgICAgcmV0ID0gdnBjaV9hZGRf
aGFuZGxlcnMocGRldik7DQo+PiArICAgIGlmICggcmV0ICkgew0KPj4gKyAgICAgICAgcHJpbnRr
KFhFTkxPR19FUlIgInNldHVwIG9mIHZQQ0kgZm9yIGZhaWxlZDogJWRcbiIscmV0KTsNCj4+ICsg
ICAgICAgIGdvdG8gb3V0Ow0KPj4gKyAgICB9DQo+PiArI2VuZGlmDQo+PiAgICAgcGNpX2VuYWJs
ZV9hY3MocGRldik7DQo+PiANCj4+IG91dDoNCj4+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy92
cGNpL01ha2VmaWxlIGIveGVuL2RyaXZlcnMvdnBjaS9NYWtlZmlsZQ0KPj4gaW5kZXggNTVkMWJk
ZmRhMC4uMWExNDEzYjkzZSAxMDA2NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvTWFrZWZp
bGUNCj4+ICsrKyBiL3hlbi9kcml2ZXJzL3ZwY2kvTWFrZWZpbGUNCj4+IEBAIC0xICsxLDIgQEAN
Cj4+IC1vYmoteSArPSB2cGNpLm8gaGVhZGVyLm8gbXNpLm8gbXNpeC5vDQo+PiArb2JqLXkgKz0g
dnBjaS5vIGhlYWRlci5vDQo+PiArb2JqLSQoQ09ORklHX0hBU19QQ0lfTVNJKSArPSBtc2kubyBt
c2l4Lm8NCj4+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy92cGNpL2hlYWRlci5jIGIveGVuL2Ry
aXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4gaW5kZXggYmE5YTAzNjIwMi4uZjhjZDU1ZTdjMCAxMDA2
NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+ICsrKyBiL3hlbi9kcml2
ZXJzL3ZwY2kvaGVhZGVyLmMNCj4+IEBAIC05Niw4ICs5NiwxMCBAQCBzdGF0aWMgdm9pZCBtb2Rp
ZnlfZGVjb2RpbmcoY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYsIHVpbnQxNl90IGNtZCwNCj4+
ICAgICAgKiBGSVhNRTogcHVuY2hpbmcgaG9sZXMgYWZ0ZXIgdGhlIHAybSBoYXMgYmVlbiBzZXQg
dXAgbWlnaHQgYmUgcmFjeSBmb3INCj4+ICAgICAgKiBEb21VIHVzYWdlLCBuZWVkcyB0byBiZSBy
ZXZpc2l0ZWQuDQo+PiAgICAgICovDQo+PiArI2lmZGVmIENPTkZJR19IQVNfUENJX01TSQ0KPj4g
ICAgIGlmICggbWFwICYmICFyb21fb25seSAmJiB2cGNpX21ha2VfbXNpeF9ob2xlKHBkZXYpICkN
Cj4+ICAgICAgICAgcmV0dXJuOw0KPj4gKyNlbmRpZg0KPj4gDQo+PiAgICAgZm9yICggaSA9IDA7
IGkgPCBBUlJBWV9TSVpFKGhlYWRlci0+YmFycyk7IGkrKyApDQo+PiAgICAgew0KPj4gZGlmZiAt
LWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vZG9tYWluLmggYi94ZW4vaW5jbHVkZS9hc20tYXJt
L2RvbWFpbi5oDQo+PiBpbmRleCBjOTI3N2I1YzZkLi5kNzQyYjk0YmQ2IDEwMDY0NA0KPj4gLS0t
IGEveGVuL2luY2x1ZGUvYXNtLWFybS9kb21haW4uaA0KPj4gKysrIGIveGVuL2luY2x1ZGUvYXNt
LWFybS9kb21haW4uaA0KPj4gQEAgLTI2Miw3ICsyNjIsMTAgQEAgc3RhdGljIGlubGluZSB2b2lk
IGFyY2hfdmNwdV9ibG9jayhzdHJ1Y3QgdmNwdSAqdikge30NCj4+IA0KPj4gI2RlZmluZSBhcmNo
X3ZtX2Fzc2lzdF92YWxpZF9tYXNrKGQpICgxVUwgPDwgVk1BU1NUX1RZUEVfcnVuc3RhdGVfdXBk
YXRlX2ZsYWcpDQo+PiANCj4+IC0jZGVmaW5lIGhhc192cGNpKGQpICAgICh7ICh2b2lkKShkKTsg
ZmFsc2U7IH0pDQo+PiArLyogRm9yIFg4NiBWUENJIGlzIGVuYWJsZWQgYW5kIHRlc3RlZCBmb3Ig
UFZIIERPTTAgb25seSBidXQNCj4+ICsgKiBmb3IgQVJNIHdlIGVuYWJsZSBzdXBwb3J0IFZQQ0kg
Zm9yIGd1ZXN0IGRvbWFpbiBhbHNvLg0KPj4gKyAqLw0KPj4gKyNkZWZpbmUgaGFzX3ZwY2koZCkg
KHsgKHZvaWQpKGQpOyBJU19FTkFCTEVEKENPTkZJR19IQVNfVlBDSSk7IH0pDQo+PiANCj4+ICNl
bmRpZiAvKiBfX0FTTV9ET01BSU5fSF9fICovDQo+PiANCj4+IGRpZmYgLS1naXQgYS94ZW4vaW5j
bHVkZS9hc20tYXJtL3BjaS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KPj4gaW5kZXgg
NzU2Zjg2MzdhYi4uYzU4MTUyZGU4MCAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1h
cm0vcGNpLmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+IEBAIC0yNyw2
ICsyNywxNCBAQCBzdHJ1Y3QgYXJjaF9wY2lfZGV2IHsNCj4+ICAgICBzdHJ1Y3QgZGV2aWNlIGRl
djsNCj4+IH07DQo+PiANCj4+ICsvKiBBcmNoLXNwZWNpZmljIE1TSSBkYXRhIGZvciB2UENJLiAq
Lw0KPj4gK3N0cnVjdCB2cGNpX2FyY2hfbXNpIHsNCj4+ICt9Ow0KPj4gKw0KPj4gKy8qIEFyY2gt
c3BlY2lmaWMgTVNJLVggZW50cnkgZGF0YSBmb3IgdlBDSS4gKi8NCj4+ICtzdHJ1Y3QgdnBjaV9h
cmNoX21zaXhfZW50cnkgew0KPj4gK307DQo+PiArDQo+PiAvKg0KPj4gICogc3RydWN0IHRvIGhv
bGQgdGhlIG1hcHBpbmdzIG9mIGEgY29uZmlnIHNwYWNlIHdpbmRvdy4gVGhpcw0KPj4gICogaXMg
ZXhwZWN0ZWQgdG8gYmUgdXNlZCBhcyBzeXNkYXRhIGZvciBQQ0kgY29udHJvbGxlcnMgdGhhdA0K
Pj4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL3B1YmxpYy9hcmNoLWFybS5oIGIveGVuL2luY2x1
ZGUvcHVibGljL2FyY2gtYXJtLmgNCj4+IGluZGV4IDY0YTJjYTMwZGEuLjBhOTc0OWU3NjggMTAw
NjQ0DQo+PiAtLS0gYS94ZW4vaW5jbHVkZS9wdWJsaWMvYXJjaC1hcm0uaA0KPj4gKysrIGIveGVu
L2luY2x1ZGUvcHVibGljL2FyY2gtYXJtLmgNCj4+IEBAIC00MjIsNiArNDIyLDEwIEBAIHR5cGVk
ZWYgdWludDY0X3QgeGVuX2NhbGxiYWNrX3Q7DQo+PiAjZGVmaW5lIEdVRVNUX1BMMDExX0JBU0Ug
ICAgeGVuX21rX3VsbG9uZygweDIyMDAwMDAwKQ0KPj4gI2RlZmluZSBHVUVTVF9QTDAxMV9TSVpF
ICAgIHhlbl9ta191bGxvbmcoMHgwMDAwMTAwMCkNCj4+IA0KPj4gKy8qIFZQQ0kgRUNBTSBtYXBw
aW5ncyAqLw0KPj4gKyNkZWZpbmUgR1VFU1RfVlBDSV9FQ0FNX0JBU0UgICAgeGVuX21rX3VsbG9u
ZygweDEwMDAwMDAwKQ0KPj4gKyNkZWZpbmUgR1VFU1RfVlBDSV9FQ0FNX1NJWkUgICAgeGVuX21r
X3VsbG9uZygweDEwMDAwMDAwKQ0KPiANCj4gSXNuJ3QgMjU2TUIgYSBiaXQgdG9vIHNtYWxsPyBV
c3VhbGx5IFBDSSBjb21lcyB3aXRoIHR3byByYW5nZXM6DQo+IA0KPiAtIGEgc21hbGwgcmFuZ2Ug
YmVsb3cgNEdCIGxpa2UgdGhpcyBvbmUNCj4gLSBhbiBhZGRpdGlvbmFsIGxhcmdlIHJhbmdlIGFi
b3ZlIDRHQiwgYXJvdW5kIDFHQiBvciBsYXJnZXINCj4gDQo+IFdlIGFyZSBtaXNzaW5nIHRoZSBz
ZWNvbmQgbGFyZ2VyIHJhbmdlPw0KDQpBcyBwZXIgUENJIHNwZWNpZmljYXRpb24gUENJIEV4cHJl
c3MgZXh0ZW5kcyB0aGUgQ29uZmlndXJhdGlvbiBTcGFjZSB0byA0MDk2IGJ5dGVzKCA0S0IpICBw
ZXIgRnVuY3Rpb24gYW5kIGJhc2VkIG9uIGNhbGN1bGF0aW9uIA0KIjI1NiBidXNlcyDDlyAzMiBk
ZXZpY2VzIMOXIDggZnVuY3Rpb25zIMOXIDQgS0IgPSAyNTYgTULigJ0sICAyNTYgTUIgaXMgc3Vm
ZmljaWVudCBhcyBwZXIgbXkgdW5kZXJzdGFuaW5nLg0KDQpSZWdhcmRzLA0KUmFodWwNCg0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 11:12:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 11:12:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188384.337527 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpJN-0005qS-8E; Thu, 16 Sep 2021 11:12:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188384.337527; Thu, 16 Sep 2021 11:12:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpJN-0005qL-50; Thu, 16 Sep 2021 11:12:13 +0000
Received: by outflank-mailman (input) for mailman id 188384;
 Thu, 16 Sep 2021 11:12:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=20iF=OG=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mQpJL-0005qF-Ix
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 11:12:11 +0000
Received: from forward106p.mail.yandex.net (unknown
 [2a02:6b8:0:1472:2741:0:8b7:109])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ffb8fef8-b56c-440d-bcee-389cc3e980f1;
 Thu, 16 Sep 2021 11:12:08 +0000 (UTC)
Received: from forward102q.mail.yandex.net (forward102q.mail.yandex.net
 [IPv6:2a02:6b8:c0e:1ba:0:640:516:4e7d])
 by forward106p.mail.yandex.net (Yandex) with ESMTP id 48A712FC32D3;
 Thu, 16 Sep 2021 14:10:38 +0300 (MSK)
Received: from vla1-8702476f8304.qloud-c.yandex.net
 (vla1-8702476f8304.qloud-c.yandex.net
 [IPv6:2a02:6b8:c0d:39a2:0:640:8702:476f])
 by forward102q.mail.yandex.net (Yandex) with ESMTP id 45A36BF0000A;
 Thu, 16 Sep 2021 14:10:38 +0300 (MSK)
Received: from vla5-445dc1c4c112.qloud-c.yandex.net
 (vla5-445dc1c4c112.qloud-c.yandex.net [2a02:6b8:c18:3609:0:640:445d:c1c4])
 by vla1-8702476f8304.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 shEebEo6nI-AbDGmBu5; Thu, 16 Sep 2021 14:10:38 +0300
Received: by vla5-445dc1c4c112.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id sXIilujlv4-AZWiLChk; Thu, 16 Sep 2021 14:10:36 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffb8fef8-b56c-440d-bcee-389cc3e980f1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631790638;
	bh=/ZV8AJQIrX7okcF2tddbAPz58I7WXDgYFEPO9SunjtQ=;
	h=Message-Id:Date:Subject:To:From:Cc;
	b=l9N/aDMdyJTI+rplM3AxPzMQ068qAgD8ghwfTr3fdQBsPe7jaGi2uwh8Sb+I+mkBl
	 G96e/gxvDvi1xBEe2+hruDIRQHAC8cpTxphHsGOML2lcWhAWzRCO7sUXjhfYpxQZJ1
	 iTgXEO2knWkt6OfDiQIhapbEHi1ZxWoFoA+hAHgE=
Authentication-Results: vla1-8702476f8304.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
To: xen-devel@lists.xenproject.org
Cc: Dmitry Isaykin <isaikin-dmitry@yandex.ru>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v1] domctl: hold domctl lock while domain is destroyed
Date: Thu, 16 Sep 2021 14:10:03 +0300
Message-Id: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>

This significantly speeds up concurrent destruction of multiple domains on x86.

I identify the place taking the most time:

    do_domctl(case XEN_DOMCTL_destroydomain)
      -> domain_kill()
           -> domain_relinquish_resources()
                -> relinquish_memory(d, &d->page_list, PGT_l4_page_table)

My reference setup: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Xen 4.14.

I use this command for test:

    for i in $(seq 1 5) ; do xl destroy test-vm-${i} & done

Without holding the lock all calls of `relinquish_memory(d, &d->page_list, PGT_l4_page_table)`
took on my setup (for HVM with 2GB of memory) about 3 seconds for each destroying domain.

With holding the lock it took only 100 ms.

Signed-off-by: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
---
 xen/common/domctl.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 12d6144d28..b9a50d3e5d 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -497,14 +497,13 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
         break;
 
     case XEN_DOMCTL_destroydomain:
-        domctl_lock_release();
         domain_lock(d);
         ret = domain_kill(d);
         domain_unlock(d);
         if ( ret == -ERESTART )
             ret = hypercall_create_continuation(
                 __HYPERVISOR_domctl, "h", u_domctl);
-        goto domctl_out_unlock_domonly;
+        break;
 
     case XEN_DOMCTL_setnodeaffinity:
     {
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 11:15:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 11:15:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188394.337538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpMz-0006V7-Oj; Thu, 16 Sep 2021 11:15:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188394.337538; Thu, 16 Sep 2021 11:15:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpMz-0006V0-Kz; Thu, 16 Sep 2021 11:15:57 +0000
Received: by outflank-mailman (input) for mailman id 188394;
 Thu, 16 Sep 2021 11:15:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U6Su=OG=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQpMy-0006Uu-6I
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 11:15:56 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.70]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 963249f7-c872-4988-ba42-d2f9df553403;
 Thu, 16 Sep 2021 11:15:54 +0000 (UTC)
Received: from DB6PR0501CA0017.eurprd05.prod.outlook.com (2603:10a6:4:8f::27)
 by DB8PR08MB5243.eurprd08.prod.outlook.com (2603:10a6:10:bf::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 11:15:50 +0000
Received: from DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:8f:cafe::1b) by DB6PR0501CA0017.outlook.office365.com
 (2603:10a6:4:8f::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 11:15:50 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT053.mail.protection.outlook.com (10.152.21.119) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:15:50 +0000
Received: ("Tessian outbound a8bfe25d7364:v103");
 Thu, 16 Sep 2021 11:15:50 +0000
Received: from 1d6c56a0c8ba.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7D90F02A-1F8F-41F7-9240-E53A85E92364.1; 
 Thu, 16 Sep 2021 11:15:30 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1d6c56a0c8ba.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 11:15:30 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6751.eurprd08.prod.outlook.com (2603:10a6:102:136::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 11:15:26 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Thu, 16 Sep 2021
 11:15:25 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0352.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:15:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 963249f7-c872-4988-ba42-d2f9df553403
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p9PCvFEzWeUMI44mSEn15pFntTzThpq9UYi4y4WWXWY=;
 b=k9zmlBFQStr7N2gW0GNQAR3Yc9QC7H1aHLaIxllfU63J8FeC6vvN+ypmgQtFzVLo0vLlIskRjmFhT0y4VB9iUFjyxCHjhgrsgSzNuMOSeohawfoHSQKnhEUQfwtk/rlTqS8/64uORw//xUpIzrXgSPJxjcmCltteUMTfa0xCQ04=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 75b474b7284c900c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=deBNPJu4aItrLgDgRDcWU3GaMaCHQ/TkoLN5nVXtI5VeKKL35vpJOaGSohvy8uUgLP2wR9Tvp5WnBWfFOESswJibkbMgejg1tp4vZCiUPLtazP3yzN6VmmJQJfg3zVkEeDP3o3kYgM8lSpPgeKrZ/5TLwN6XHrSwAVQl8I8h6mmTnUI1qVtuOclrx05Emw2CVioOnQxK7g00nxjdlm2LLWI/tHFoHNCXQS5P8iJKdSQFnx1OMSY1VpDc9jXi14CWF2zEWJd9flsyDnwhxGMbOhKMwsXv2IxjlLAjx0qzkNPgDXTAAdkP10FGv9r9N57j1G/uigZQmHDptpy1g5d1Kg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p9PCvFEzWeUMI44mSEn15pFntTzThpq9UYi4y4WWXWY=;
 b=Dv72jQgQZFc3wpRl26D3VfhCx2ywR11tJz7077nAQSOg/XShWK3ef6J9I8sXErkHZ2OlV539+Q7XP+YTGNDB7VpBhoPLjHXDxT9dKyd0puBlDpeXm78hRbJL5HI3waoKrdkwivytLKcsDKFmz4MRmeuoUeZh9H0soRt3poNnhM0ZclfNmaekkG96IlRojHwk2vVv8PiqUl1f0Cy+syz5XsjhO4T11wkW3QJv7WXgaaqDu6Qu5pM8dJCcv9ADCay73yFbTst0aV5cQWxSrNMSr21nfdWgiHQbhEHU6u7npeESSismzR9SE03eBZWVR0Dr8jxqu1ujUqC2XysIe62r7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p9PCvFEzWeUMI44mSEn15pFntTzThpq9UYi4y4WWXWY=;
 b=k9zmlBFQStr7N2gW0GNQAR3Yc9QC7H1aHLaIxllfU63J8FeC6vvN+ypmgQtFzVLo0vLlIskRjmFhT0y4VB9iUFjyxCHjhgrsgSzNuMOSeohawfoHSQKnhEUQfwtk/rlTqS8/64uORw//xUpIzrXgSPJxjcmCltteUMTfa0xCQ04=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <5e272a26-e1f7-a861-feb1-350293a9f809@suse.com>
Date: Thu, 16 Sep 2021 12:15:18 +0100
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <0E1E73EE-5F48-499A-89DA-EFE78642A2AD@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
 <5e272a26-e1f7-a861-feb1-350293a9f809@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0352.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18d::15) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5271f72c-dbb7-444b-0b4c-08d979035755
X-MS-TrafficTypeDiagnostic: PAXPR08MB6751:|DB8PR08MB5243:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB52431DCFD003B047D171B971E4DC9@DB8PR08MB5243.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +XUDdkDonzyLRHM3tgJSXAw9KK3DpwzjZH7BL/YYyzVGwCuyx6CCz9oyNMQAKWCR+7aRtL88RiChO+aqqpmqkWw8cXAW8t6aT4IAa5jl4S/+2BUp3RivX/E+/vhEmJohcYxMD1XSCMJWGKqhYIHYfHWNjowSZVqHHZJNsqMEnIvY5O8yG+kxph7l45jqYJZlKRxsmU7mQBVtUszgwcvZozWd5pok0w5HxqoqQvRqGWZNZ7jzpqrAmdv6y1IJrPpfHtH11hb5OarszR6tldFF/fskRkiaJwHEjzJI625JmEs15f4OEzdoLp68oZAtSkEMB/Xm8c2y4caUr7CXyQ6qhTcKQmJS7qjAnANhhtOFc6r2qGpUVSyQrDo/NGh73VVtxOQoWE0XWcXqGEnAm/1n5MNLm4Bz24ky2Pm1530XukqRfowIJkNoIQTocVd+bs5GUVmmI8XzJzPoYcGF7w+XOfU12dDy/jMdBRcDd1IHWDRwRoh91NB76pxE5uPdzzvsnqVMCut6TdPnsdpLKeA0S8Ultm9TAKbhG5eVn7ZP2RrVR2oATjG+vnV1WhJIVxLWcdjyM2v/0/7aecRAR2B3f3UpSlRBpIYu/ETEzkMRzWgPTbee08P5GcV/s8gqOARAj7MHi7FTF1AWJD+ydOEqT9abC2i/+MqwShuEWEqebaXUPN16269tXkzMQVTimSadCXp1amDzRarVGlq1y9bDMV4zkfN3+iRb9fH0U1IYJZy3TtKySPJtwkcN4FLQEfKszvol1dqIoy+W6Sj6QZDh9w==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(6486002)(6916009)(316002)(38350700002)(38100700002)(44832011)(83380400001)(33656002)(8936002)(2906002)(52116002)(6666004)(6512007)(4326008)(54906003)(2616005)(956004)(186003)(36756003)(53546011)(6506007)(26005)(508600001)(5660300002)(66946007)(66556008)(66476007)(86362001)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6751
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	40396318-0295-4180-7814-08d97903483f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sV1FsolW3AvGFsAJNU9O4NnU/qiI/xuwcfA2neYE/xYhUe7McGnqDVv+bWpX2bNDFBf3FV5JM0m20yiOmDOpwKf40jZhSQ4RpFGdx4KdDLwFCWOUvVjqyfezEacXKOWv5y1HF9xHAIug/cNcGGy7stxnxR4zesydhS6HeQ+GizugtcVu4OPLY5ofVFLEAOqvLis5O3Oc+iEZEx2LwHzUkVitm7/UeTpV9Z/CbuDsKSl6bi7lNP9ajxRa8dYt3FEl/7uQJFWlih40iioDPnIIvcZ6ITVq05EYZD0N+12cLjyQpRtpVevjDKrHkeYIDZ6JQULB8BY1Hjp/cwQIJN6J/9zi24npXEbEmUPnhkJd9h66YHOoUd4AZZfODnvm2IhJe5c3EH/rnnEu6DzighnN5b+jsoCJuUF/mgnGRFAM2Ar9Tmm619RHm/aGW1IxHX9AY7FlVnRSnxcO9XaE4arfQOQUWqAuvlhMLNJt4w6IP7iUdzTxPm0z7fp9uHBHiMjNa7VHWtufIdkvLS9m0VeURsyZxywA1SUlYmM/8wFS+Yy4b5M7xUmmy1INX8f03DePTSL0vSlBnUWsysmhf5ok4RsLD4/5hmWWBlKT/h3Qs5NZVrPPNtpItSkgeYHyH1sBEyXjpeZ+3VfoW6iiCbusYDYNkAnjzMLkkB4XlhJ3tpm/OINEkvJMRcpT6AZ1PUzSDwP12/QLMNjypUWTEhR9LIS9fsR7fXpbEvQRwFBXcjw=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(54906003)(86362001)(356005)(4326008)(70586007)(107886003)(81166007)(33656002)(508600001)(6506007)(70206006)(336012)(53546011)(6666004)(26005)(2906002)(82310400003)(186003)(47076005)(36860700001)(6486002)(44832011)(6512007)(5660300002)(956004)(2616005)(83380400001)(6862004)(8936002)(36756003)(316002)(8676002)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 11:15:50.6619
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5271f72c-dbb7-444b-0b4c-08d979035755
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5243



> On 16 Sep 2021, at 07:50, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 16.09.2021 03:16, Stefano Stabellini wrote:
>> On Wed, 15 Sep 2021, Luca Fancellu wrote:
>>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_han=
dle,
>>> +                                               int domain_node,
>>> +                                               int addr_cells,
>>> +                                               int size_cells)
>>> +{
>>> +    /*
>>> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,devic=
e-tree}
>>> +     * inside this node
>>> +     */
>>> +    for ( int module_node =3D fdt_first_subnode(fdt, domain_node);
>>=20
>> int module_node;
>>=20
>> for ( module_node =3D fdt_first_subnode(fdt, domain_node);
>=20
> Not just here iirc from briefly looking over the patch as a whole
> yesterday: Use of plain "int" would better be limited to cases where
> values may also be negative. I don't suppose that's possible here as
> well as in a number of other cases.

Hi Jan,

fdt_first_subnode(=E2=80=A6) can return -FDT_ERR_NOTFOUND.

>=20
>>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TA=
BLE *SystemTable)
>>>             efi_bs->FreePool(name.w);
>>>         }
>>>=20
>>> -        if ( !name.s )
>>> -            blexit(L"No Dom0 kernel image specified.");
>>> -
>>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>>=20
>>> -        option_str =3D split_string(name.s);
>>> +#ifdef CONFIG_ARM
>>> +        /* dom0less feature is supported only on ARM */
>>> +        dom0less_found =3D check_dom0less_efi_boot(dir_handle);
>>> +#endif
>>=20
>> Rather than an #ifdef here you can simply implement
>> check_dom0less_efi_boot on x86 as a static inline returning always
>> false.
>=20
> Indeed, and the properly named (efi_arch_...()).

Ok, I was unsure if a solution like that was going to be accepted, I will u=
pdate the code then.

Cheers,
Luca

>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 11:29:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 11:29:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188402.337548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpZm-0008K3-U8; Thu, 16 Sep 2021 11:29:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188402.337548; Thu, 16 Sep 2021 11:29:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpZm-0008Jw-RA; Thu, 16 Sep 2021 11:29:10 +0000
Received: by outflank-mailman (input) for mailman id 188402;
 Thu, 16 Sep 2021 11:29:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U6Su=OG=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQpZl-0008Jq-Qp
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 11:29:09 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0d::615])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1cb20ac6-eab3-451f-93bb-4c44cd159e9b;
 Thu, 16 Sep 2021 11:29:07 +0000 (UTC)
Received: from DB6P193CA0006.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::16) by
 VI1PR08MB3438.eurprd08.prod.outlook.com (2603:10a6:803:82::28) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4500.18; Thu, 16 Sep 2021 11:28:56 +0000
Received: from DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:29:cafe::d2) by DB6P193CA0006.outlook.office365.com
 (2603:10a6:6:29::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 11:28:56 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT031.mail.protection.outlook.com (10.152.20.142) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:28:56 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Thu, 16 Sep 2021 11:28:56 +0000
Received: from d00df6a44d20.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3B8A233E-596A-4325-AB5A-B400BDC41C9A.1; 
 Thu, 16 Sep 2021 11:28:45 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d00df6a44d20.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 11:28:45 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6559.eurprd08.prod.outlook.com (2603:10a6:102:df::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 11:28:43 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Thu, 16 Sep 2021
 11:28:43 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0080.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:28:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cb20ac6-eab3-451f-93bb-4c44cd159e9b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LcQXgh4Nw7cFXANxXv8ccHFbTdsliiRdzpD0HWYv7Bc=;
 b=s1ZGGAN+yeoJVQVSOXn+BP/j8AZbtC+vCkqHYNikqQwMknbQpO6EQYiM3Mne+pej0Sv73NRYt5YSWFuGgMR3DO+rhEjwSDfHNFeJas+4fWY4vWZRk4OT6whIxgz+5QUuha0OhARfewhc9M+cbQsFfqzwf+cpdqFL3kCjDdfyNCs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 24523d4d580f1da2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UxDl5IMVMapSW0M5WpCftA9ClGIDqUqr/0ch+nZipBqWOxNuJWWyd8lwuqmo9pyWLtJzB9yRtlyxd23MLkgT1BBR+7kAvY0LSP5IYqxWzM3tL7qsmKItHLdzrCSXRMj4w5P8A8fwdYcQipNyuh5w5729sZLu3oIG2nrZDjVzve3tZritPSPUs7v3zLbxyiEvzyvbL+4urHsElWSQVqzI2TrXIqh58/Tn22nanNGkgadOvhc8ySkBHTZatMvD7CY2nRYOFKKM9hSd6VFf6+4iXX+suDHUWhQjKoURmntmnjVvEYH2JApynYmd+oZkSgo5WwbunSc6RQYC/qMtgKHwZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LcQXgh4Nw7cFXANxXv8ccHFbTdsliiRdzpD0HWYv7Bc=;
 b=IbPj/C5mxOE2vK5MSNO5qd5XmBd0BMtz2emx1WthjTXg9CpuMb2WjlnrPx4T4EzdjJMDHNUK52pS5Mu9dGt2wZsmQ3pf7OaCAp6w1pI85AXmO+PJt5o64ZvSg6cqo1izuJFbXLBR+H8+qDg7vc8deBgOqvq/PpCBys+8PBcET9W0Dg/z3Sg5V0ep3S8eLwsuP/D/kRf7ptoLQCYpTjUnz4pKLYzqWM3RJcAnZYX5hD5PzHbiE2Ryu5VvpvUzAKIrjtlYBkLU75vW1DNwMQDQKTdHT8lzYbbLs4/RBDRA6Wo+RwbvrfUwaew7xGhK6Yuv96EPAPhWDtXIEWyTUVcroQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LcQXgh4Nw7cFXANxXv8ccHFbTdsliiRdzpD0HWYv7Bc=;
 b=s1ZGGAN+yeoJVQVSOXn+BP/j8AZbtC+vCkqHYNikqQwMknbQpO6EQYiM3Mne+pej0Sv73NRYt5YSWFuGgMR3DO+rhEjwSDfHNFeJas+4fWY4vWZRk4OT6whIxgz+5QUuha0OhARfewhc9M+cbQsFfqzwf+cpdqFL3kCjDdfyNCs=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
Date: Thu, 16 Sep 2021 12:28:36 +0100
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0080.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8::20) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a1009bf-762e-4725-1007-08d979052bbd
X-MS-TrafficTypeDiagnostic: PAXPR08MB6559:|VI1PR08MB3438:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3438CBE68987A952A46BCC83E4DC9@VI1PR08MB3438.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 uNBQx1fgO+qHxJHn8dGBIzp+vbPTlWQ3pf869d3jHZKlvh3awZMugDQEXLITkRw/yx2GRUbccrJzUwwIS0p8QbGPCeH0xewH2NbN49jXsUEy5H/vK3Mq8CZyBdknznufRohy7wL6E3SQZXJ0bQdt+6lOdXCx9k18AT7VhBaTvj6+iA3LWh6TWhpBu2ZP9GgFe8DkLwIlHSCsskeBVzt/myxUjSTclr0yw8GeLpn11dBtPw5e7JBsOysSJ8Zahn8tpg7tLGarnVNlyJg7cYmDsycWUXpSbXNAULh3lP1djlIGx7Uh9HoMcv5MhnP4OrENyNEr9vfVmZ4DROISUA5DISWAqmeTGJspNqUM6mcYOoomrdlzTvHv2TpsQ1ns8TArHct/xzkQOCIWHTnqznVTK+qmDM5+NIA0GvGTSHPRVFOgyNisWyKQGL2KNdRfcwe/mhAPJmVsDbhw9vnt/EgGaYefkoZMPsfl+XVhO4dTM6x39ULYcrhf4GHLwrs2Z9czqlbMaunUUNVb55pNItGc14ObNRDA9zDD0Lwy1RFlXNfUN0ZXpRMCLnSvRfyD8gu4NwZbn1cAb8L7qDzvpHKi+vmxSF9IeKqEa4gU6D/0y9W5gpwKlNDtcWsTQcUjGdQentMVbTokeSBJZwbCBBJ2d1buhZniGTK3C3dWxiA3QpVYjlkHK3e9ueN29WhA/LlUO3z5Cjn+IvrzCTycoGNxgXtuECwzB3BhiSLxp3nHMnCAqaUhbzKxQPWjvVM/gdqv3HS1moD30Sgfrlh7VT3B7A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(39860400002)(366004)(346002)(136003)(4326008)(2616005)(36756003)(956004)(66476007)(6486002)(66946007)(86362001)(6512007)(66556008)(8936002)(26005)(6666004)(186003)(44832011)(2906002)(8676002)(38100700002)(38350700002)(33656002)(5660300002)(52116002)(54906003)(316002)(6916009)(6506007)(53546011)(478600001)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6559
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6ce8f5ee-1f35-4515-8de3-08d9790523b8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zHrlij4EiK/4dEe/VUc5kcvegIgdRLUBRmwjUqfx7j0PrOhBsXkNzun8s8YKUzdSmQNQL3H6bRh4CUs8ZugoK9Fs9OW0k7nnn5Kq5RV1k+hs6for33PupiNy9NI/Fem0VL0yNXH3QxhRPX7JsN5hv6XSNb4uaAO7AhfogaCGXcwb8fj1lh47GiXBpLpa57bXVk/S8epAOdKfylKpR6BQfkiltLgvvxVbpDiy4SGDai84OdROTeTyNNkWfp6ptv7TWzrpZUrf/EO/DISxemX1HWp8hKsrmHdOt6O6+ylvvg/PUUHiIEVtt6QcFEdCbMiYOcfjdmehddld3+8OuTDFscEySUClPqGrGka32KC22X2JlCZ/VsjSPddrJheat2eaX7RIJhPgbMAhoAmB/wvkAzDxgXVlunCD99hmgrjqsBrlEZ4CvEdH58ntRUE7FuJfs6wnFfV7Jk0JbWqQsv4GeEcARmoON90fHHiN3L7KO18QPYK54lxbApFF3tcAStD512ohFJe3/Y7ZcWprzVgfs///yPP1eYSUyw9OAUF1JSAMWsDOdKvfu5s41Y0VW+w6bendnMNn8HpR2V1s8fBJMsxNbRr3yNncuWfzueVrF4bzJ1DwXjUgd61ULohm+BHlJ97FX1qbp0YcLKA7qOWmPYgOfjWyue1PqM1B8v++36GMwC49OKvrwj0Bk4/bFX5UejB3PHXK4herF2YyNahSYZenv9AXRFegrA/DDXd7nEY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(376002)(39860400002)(346002)(136003)(46966006)(36840700001)(4326008)(70586007)(70206006)(26005)(186003)(316002)(81166007)(44832011)(82310400003)(6512007)(82740400003)(6862004)(47076005)(8676002)(356005)(2616005)(33656002)(36860700001)(36756003)(6666004)(956004)(2906002)(5660300002)(53546011)(86362001)(6486002)(336012)(54906003)(478600001)(8936002)(6506007)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 11:28:56.5183
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a1009bf-762e-4725-1007-08d979052bbd
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3438



> On 16 Sep 2021, at 09:46, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> A number of nits, sorry:
>=20
> On 15.09.2021 16:26, Luca Fancellu wrote:
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -8,9 +8,39 @@
>> #include <asm/setup.h>
>> #include <asm/smp.h>
>>=20
>> +typedef struct {
>> +    char* name;
>=20
> Misplaced *.

I was looking in the CODING_STYLE and I didn=E2=80=99t found anything that =
mandates
char *name; instead of char* name; but if you prefer I can change it since =
I have
to do some modification to the patch.

>=20
>> +    int name_len;
>=20
> Surely this can't go negative? (Same issue elsewhere.)

I will change that to unsigned int.

>=20
>> +} dom0less_module_name;
>> +
>> +/*
>> + * Binaries will be translated into bootmodules, the maximum number for=
 them is
>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DT=
B
>> + */
>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
>> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
>> +static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_=
MODULES];
>> +static uint32_t __initdata dom0less_modules_available =3D MAX_DOM0LESS_=
MODULES;
>> +static uint32_t __initdata dom0less_modules_idx =3D 0;
>=20
> Please see ./CODING_STYLE for your (ab)use of uint32_t here and
> elsewhere.

Ok, I will change them to unsigned int

>=20
>> +#define ERROR_DOM0LESS_FILE_NOT_FOUND -1
>=20
> Macros expanding to more than a single token should be suitably
> parenthesized at least when the expression can possibly be mistaken
> precedence wise (i.e. array[n] is in principle fine without
> parentheses, because the meaning won't change no matter how it's
> used in an expression).

I will fix it to be (-1)

>=20
>> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>> void __flush_dcache_area(const void *vaddr, unsigned long size);
>>=20
>> +static int __init get_dom0less_file_index(const char* name, int name_le=
n);
>> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handl=
e,
>> +                                              const char* name, int nam=
e_len);
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int module_node_offset,
>> +                                               int reg_addr_cells,
>> +                                               int reg_size_cells);
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells);
>> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);
>=20
> There are attributes (e.g. __must_check) which belong on the
> declarations. __init, however, belongs on the definitions.

Ok, I will remove __init from declarations.

>=20
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop =3D NULL;
>>     union string section =3D { NULL }, name;
>>     bool base_video =3D false;
>> -    const char *option_str;
>> +    const char *option_str =3D NULL;
>>     bool use_cfg_file;
>> +    bool dom0less_found =3D false;
>>=20
>>     __set_bit(EFI_BOOT, &efi_flags);
>>     __set_bit(EFI_LOADER, &efi_flags);
>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>             efi_bs->FreePool(name.w);
>>         }
>>=20
>> -        if ( !name.s )
>> -            blexit(L"No Dom0 kernel image specified.");
>> -
>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>=20
>> -        option_str =3D split_string(name.s);
>> +#ifdef CONFIG_ARM
>> +        /* dom0less feature is supported only on ARM */
>> +        dom0less_found =3D check_dom0less_efi_boot(dir_handle);
>> +#endif
>> +
>> +        if ( !name.s && !dom0less_found )
>=20
> Here you (properly ) use !name.s,

This is not my code, I just added && !dom0less

>=20
>> +            blexit(L"No Dom0 kernel image specified.");
>> +
>> +        if ( name.s !=3D NULL )
>=20
> Here you then mean to omit the "!=3D NULL" for consistency and brevity.

I usually check explicitely pointers with NULL, is it something to be avoid=
ed in Xen?
There are some industrial coding standards that says to avoid the use of ! =
operator
with pointers. Is it important here to do !name.s instead of the solution a=
bove?

>=20
>> +            option_str =3D split_string(name.s);
>>=20
>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str=
) )
>> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_st=
r)) &&
>=20
> Stray parentheses.

Will fix

>=20
>> +             (name.s !=3D NULL) )
>=20
> See above.

Will fix

>=20
> I also don't think this logic is quite right: If you're dom0less,
> why would you want to look for an embedded Dom0 kernel image?

This is common code, that check is not from my patch.

Before this serie, EFI boot requires that a dom0 module was passed, otherwi=
se
the boot was stopped.

This serie instead removes this requirement, letting the boot continue if t=
here is no dom0
kernel.

However (as in the old code) if the user embed the dom0 kernel in the image=
, then it is
legit to use it and if there are also other domUs specified by DT, then the=
 system will
start the dom0 kernel and the domUs kernel as well.

Cheers,
Luca=20


>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 11:55:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 11:55:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188411.337559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpyp-0003Nh-42; Thu, 16 Sep 2021 11:55:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188411.337559; Thu, 16 Sep 2021 11:55:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQpyp-0003Na-12; Thu, 16 Sep 2021 11:55:03 +0000
Received: by outflank-mailman (input) for mailman id 188411;
 Thu, 16 Sep 2021 11:55:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U6Su=OG=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQpyn-0003NU-8n
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 11:55:01 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.72]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a6cb9572-571e-4d1a-8210-0b4933dacf1d;
 Thu, 16 Sep 2021 11:54:58 +0000 (UTC)
Received: from AS9PR06CA0143.eurprd06.prod.outlook.com (2603:10a6:20b:467::20)
 by AM6PR08MB4552.eurprd08.prod.outlook.com (2603:10a6:20b:74::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Thu, 16 Sep
 2021 11:54:56 +0000
Received: from AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:467:cafe::81) by AS9PR06CA0143.outlook.office365.com
 (2603:10a6:20b:467::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 16 Sep 2021 11:54:56 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT063.mail.protection.outlook.com (10.152.16.226) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:54:55 +0000
Received: ("Tessian outbound 16951d3c485e:v103");
 Thu, 16 Sep 2021 11:54:55 +0000
Received: from 79778860baef.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A42A6A28-D233-4654-AF6A-90915DFE4396.1; 
 Thu, 16 Sep 2021 11:54:49 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 79778860baef.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 11:54:49 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6351.eurprd08.prod.outlook.com (2603:10a6:102:15b::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 16 Sep
 2021 11:54:47 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Thu, 16 Sep 2021
 11:54:47 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0001.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:150::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 11:54:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6cb9572-571e-4d1a-8210-0b4933dacf1d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hg8PSPow3GWgGjEDcT49TQmyivCXN7jE+azps/yvcoU=;
 b=zojZ7aF2xG42AAp4gPfBIBpCfOyTDoSxZfcxwWXO3kemkjw0c5LJS0J37ye+HttiPIVcbS0LYQ3mBq1AxUaaH9xd2wPIwycsbC0NOy2NdLddtLGlrg8ovNQEnU8ubLN+r7PoqVcPqG1nxix5ApeZ3ulMlZpEV+eC31+PRpwEW3k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2586700339fbf282
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KR5MHA6nxCAAkvMQ4wj0AFcjtaGwcIgt+Bwj57gGQvF9SSz7F6m2OABFIqAp4bbqd6GecN2Bs+Zn8xK0sZnrXTd5ZLDXmYKGBg1KhebQgFwfMgGu+rHGe54UxT/PZQw5NGP1ogIXnfBya+m+c8llgJgZnfDfhkWsBa+rBFzTpup2WYeSt9HNC65iPSG7vSOn5qciJxjnFhD6sXiFS3EIcCRajEPQ4iXSBYTKL2jG+hYnhzHZZTWVbSGgbq06Agv4KCnHRuS/JXSFBzeHC2XpKQbwRawCmn1euS6Do6CUX6GkPYe66QedjJ2I+nyhBsFJRmy1jxVz4L5NHPBMs6C5FQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hg8PSPow3GWgGjEDcT49TQmyivCXN7jE+azps/yvcoU=;
 b=AOF4tJZ4HQOVdrtlGOExbH29Iyv1iRtb30fmw2eoZniW7uVGFO31MXJasHDwN3HTaovewKdTXpriEOX3zB9Nb+6PfJsDidVeYH8avOL9BKszWDT7rGZHR1EvjBhfDJPUAcKJ/c18wMwi0PYgM5rlrzv9Spe4r/nJZtRAL5X6L2FWSa3+2pN2GQlq6m7e9rLaNy9a9BoNPu2zo7SWxV9crZ1WQz7faMwm9OY7HJ0o932ga8yic5gdx/KdaB9nRVEEunnk8WlL5/9nkqY1j9qsJK6CMXcIiIrtlnwW/JVMJTCJx3J/jHXXwdABMFKaGUR6TiEYB3rH9097wjQQ1LtnrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hg8PSPow3GWgGjEDcT49TQmyivCXN7jE+azps/yvcoU=;
 b=zojZ7aF2xG42AAp4gPfBIBpCfOyTDoSxZfcxwWXO3kemkjw0c5LJS0J37ye+HttiPIVcbS0LYQ3mBq1AxUaaH9xd2wPIwycsbC0NOy2NdLddtLGlrg8ovNQEnU8ubLN+r7PoqVcPqG1nxix5ApeZ3ulMlZpEV+eC31+PRpwEW3k=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 1/2] xen/efi: Restrict check for DT boot modules on EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109151445080.21985@sstabellini-ThinkPad-T480s>
Date: Thu, 16 Sep 2021 12:54:39 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 jbeulich@suse.com
Content-Transfer-Encoding: quoted-printable
Message-Id: <3C00EB19-5513-4C1D-8F8D-B8CF5D4A302A@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-2-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109151445080.21985@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0001.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:150::6) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7b0fe7dc-6e08-4f72-2643-08d97908cd43
X-MS-TrafficTypeDiagnostic: PAXPR08MB6351:|AM6PR08MB4552:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB455288925C415964877F0ACEE4DC9@AM6PR08MB4552.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 H2FYtMdwaceOZi/sY7QxD3cbSnDBlaa0TH4sq7Y50mb2S417OHXkJPs3zXLgWSEDf5bK+lTEFkq/VGWsR9zRvxndIziVr+C9iePmNxWJFAn5aI4a9wiXQUB1+7weExfeIj9FFBNYJ471EAV9TJO1w2PUrgk1EuEp0OwKYsJzOUy5tWDbYl+8iTE5mNObuW7WWNTXNTWjF7a1X+qlFvlwxrhqtvF16aALUf9Llxof6aY6q5fPb61Qu4+jH62ZUksYX0aRXuZSk/aycj8DDKzYSkZKppsvEisk3jJOpWd73wQxtsV44ssA/lEh8KwitWD6lm/DUHLY4M942My9bclmXvZHH/jegTI9fh4ZGxxAzcDgekvHjNYBjpHvTmRObb/W9AGmCZpHMPYYGb1wULglhMd8vTh4aytdkKMB+FaGBBNiDa8Q8zZzeAodE1DKhJWRYicaOtfGFlgaeIitHhns9q6QR9CMnlS0dT88NPT2CU6Eo10nZkMuOL1cQIIwnupBFH5sIH+fApN+HenXk90OZWfl3n1Om9tumghMdo7d5aWiUX0z12+S0eLTsT4qOrq9Wjpk+3MFwNq9kxV6VLVBMu7KusXFlgwBdV2LowSpnVmnp0yXfkyRz31cda2aYupvg2XO61JS4Qvf0U00QLf39dRJ7q7lnrE0ZGIIEt3q4vJMRDD5IyfySKUelTdHH52VmhC/gQRgLQ+qOs4GejRhfC13d8atiQP88iSgRoISkWw=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(396003)(366004)(39860400002)(6512007)(4326008)(26005)(52116002)(6486002)(478600001)(186003)(38100700002)(86362001)(66556008)(66476007)(38350700002)(53546011)(6666004)(6506007)(33656002)(956004)(66946007)(2616005)(44832011)(8676002)(6916009)(8936002)(54906003)(36756003)(83380400001)(5660300002)(2906002)(316002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6351
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	db53daf6-55ca-4762-6866-08d97908c7ac
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9oECS5R5BjPl2P8CrjyZfrVVdoDFskeP4Z36GQDvyqx5OSpKbGL+5JMR7ChAodaGzTiI3THj0Gm9HkX1d0tUYdndMhFueCbOb9ugYoxWT+7J5Hbils2TS4HW+5ObJh5z6RPeu8v4DIA/9SwKPDzH+Ll7gzMehaie2h/rQFmuyUkIwknpiwF/U0meocY+zoOfHEQrzzDxBrQrmllRe/Jd2nEhu3wX1fRS+37KfYaTD4GRwQ/QH1uR7bAJIvQHe9LZT7Gdh3IlbU7lMc+8kQyluKtGQRVPnZHc2NK9oDWBfqV8BEue12KDCqj+8vCKGPm41/PJ2iM5uIzd7KmPdFMqo6xHUpStDcN8A4+t7m5oBq1UfM7PmcDtfJNYqJLJz/FwdpgZ+bMstLfUMGx76j6EI/8RTO78FrLwc5taPgxTGixnDt20p59fZXMWMBDfBYLio5jntg3n23Tnaj8x7qEDiuul7hSKZyd6arEWSqdWsThZ3+ue5OID8aUB5Dz/clsxN4MbL+nCaQQDlLb1pOqNjKhdvYOHvRls2bPcRhdgtkcnKa0+jpaEZKghl4PDSqysMwHLMRRzhu33HfObXQzI8Z6LQ564ytOwFKKw/Mp55IgIP835FFiFYlChNQ5OgyZHrWdo0UCfW9pf9kCnn8USUyZmU8b6BcC8Xg6f+zmQs8hJaGd0iHlDccMv3Q/xUQ/Zmq1W1wDxCeyZKahUvyV3tQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(396003)(136003)(376002)(36840700001)(46966006)(6862004)(33656002)(82310400003)(54906003)(336012)(83380400001)(36860700001)(2616005)(8936002)(44832011)(81166007)(2906002)(956004)(356005)(4326008)(5660300002)(8676002)(107886003)(26005)(186003)(6666004)(6512007)(6506007)(53546011)(478600001)(82740400003)(70586007)(70206006)(6486002)(316002)(36756003)(47076005)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 11:54:55.9255
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b0fe7dc-6e08-4f72-2643-08d97908cd43
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4552



> On 16 Sep 2021, at 01:16, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> Adding Jan for an opinion on the EFI common code changes. See below.
>=20
>=20
> On Wed, 15 Sep 2021, Luca Fancellu wrote:
>> When Xen is started as EFI application, it is checking
>> the presence of multiboot,module in the DT, if any is
>> found, the configuration file is skipped.
>> Restrict this check to just any multiboot,module that
>> is a direct child of the /chosen node.
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>> ---
>> xen/arch/arm/efi/efi-boot.h | 30 ++++++++++++++++++++++++++++--
>> 1 file changed, 28 insertions(+), 2 deletions(-)
>>=20
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index cf9c37153f..5ff626c6a0 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -581,6 +581,8 @@ static void __init efi_arch_load_addr_check(EFI_LOAD=
ED_IMAGE *loaded_image)
>>=20
>> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTabl=
e)
>> {
>> +    int node;
>> +    bool dom0_module_found =3D false;
>>     /*
>>      * For arm, we may get a device tree from GRUB (or other bootloader)
>>      * that contains modules that have already been loaded into memory. =
 In
>> @@ -592,11 +594,35 @@ static bool __init efi_arch_use_config_file(EFI_SY=
STEM_TABLE *SystemTable)
>>     fdt =3D lookup_fdt_config_table(SystemTable);
>>     dtbfile.ptr =3D fdt;
>>     dtbfile.need_to_free =3D false; /* Config table memory can't be free=
d. */
>> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,modul=
e") < 0 )
>> +
>> +    /* Locate chosen node */
>> +    node =3D fdt_subnode_offset(fdt, 0, "chosen");
>> +
>> +    /* Cycle through every node inside chosen having multiboot,module *=
/
>> +    do {
>> +        int depth =3D 0;
>> +        node =3D fdt_node_offset_by_compatible(fdt, node, "multiboot,mo=
dule");
>> +        /*
>> +         * If the multiboot,module just found is placed at depth less t=
han 3,
>> +         * it means that it is here: /chosen/<module> so it is a module=
 to
>> +         * start dom0. (root is counted as 0)
>> +         */
>> +        if ( node > 0 )
>> +        {
>> +            depth =3D fdt_node_depth(fdt, node);
>> +            if ( (depth >=3D 0) && (depth < 3) )
>> +            {
>> +                dom0_module_found =3D true;
>> +                break;
>> +            }
>> +        }
>> +    } while(node > 0);
>=20
> It should be possible to enable the uefi,binary bootflow for Dom0 and
> the Dom0 ramdisk too. It would be nice as we could have a 100% UEFI
> boot, not dependent on U-Boot, both Dom0 and Dom0less, without the
> xen.cfg file. It doesn't have to be done now by this series, but it
> should be possible from a device tree bindings perspective.
>=20
> With that in mind, is this check accurate? This patch is saying that if
> Dom0 is not present in the device tree, then load xen.cfg. But what if
> it is a true dom0less configuration? Then we would have no dom0, only
> dom0less VMs, and we might still not want to load xen.cfg. True dom0less
> is another one of those configurations that don't have to be enabled now
> by this series but they should be possible from a device tree bindings
> perspective.
>=20
>=20
> I tried to think of ways to improve this check, for instance searching
> for a module that doesn't have "uefi,binary" but has the regular "reg"
> property. If there is one, then we could skip loading xen.cfg. But that
> doesn't work if we have a UEFI-only true dom0less configuration.
>=20
> So I am thinking that we have no choice but introducing a new property
> to tell us whether we should or should not load xen.cfg when
> multiboot,modules are present.
>=20
> Taking inspiration from HyperLaunch, it could be a new node:
>=20
> chosen {
>    cfg {
>        compatible =3D "xen,uefi-config", "multiboot,module";
>        uefi,binary =3D "xen.cfg";
>    };
> };
>=20
> In efi_arch_use_config_file we would check if there are any nodes
> compatible with "multiboot,module". If there are none, it returns true.
>=20
> If there are any, and one of them is also compatible "xen,uefi-config",
> then efi_arch_use_config_file returns true and also the specified
> configuration filename.
>=20
> If there are nodes compatible to "multiboot,module" but none of them is
> compatible to "module,uefi-config", then efi_arch_use_config_file
> returns false. We use the device tree only.
>=20
> I think that would be clearer and more consistent from a specification
> perspective, but it requires one change in common code:
> efi_arch_use_config_file would not just return bool but it would also
> return a filename if found (it could be a pointer parameter to the
> function).
>=20
>=20
> Otherwise, we could add a simple property like the following, without a
> specific value and without a filename:
>=20
> chosen {
>    xen,uefi-config;
> };
>=20
> The presence of xen,uefi-config could mean that Xen should go through
> the usual guessing game to figure out the right cfg file to load. This
> would not require any common code changes because
> efi_arch_use_config_file could simply return bool as it does today.
>=20
> My preference is the xen,uefi-config compatible node, but I think the
> property would also work.
>=20
>=20
> Jan, do you have an opinion on whether efi_arch_use_config_file has to
> stay as it is today, or would you be open to the possibility of making
> efi_arch_use_config_file return a filename too?

Hi Stefano,

True dom0less is a configuration that this serie enables: if there is no do=
m0 kernel
specified in the xen.cfg then only the domUs will be loaded and started.

The following cases are valid:
1) start only dom0 [dom0 modules in xen.cfg or embedded in Xen image]
2) start only domUs, true dom0less [no dom0 modules in xen.cfg and inside X=
en image, domUs on DT]
3) start dom0 and domUs [(dom0 modules in xen.cfg or inside Xen image) and =
domUs on DT]

I don=E2=80=99t understand why we want to add new properties to avoid/not a=
void to read the xen.cfg, am I missing
something?

Cheers,
Luca



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 12:04:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 12:04:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188422.337570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQq7l-000569-71; Thu, 16 Sep 2021 12:04:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188422.337570; Thu, 16 Sep 2021 12:04:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQq7l-000562-3q; Thu, 16 Sep 2021 12:04:17 +0000
Received: by outflank-mailman (input) for mailman id 188422;
 Thu, 16 Sep 2021 12:04:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U6Su=OG=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQq7j-00055w-Gp
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 12:04:15 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe02::618])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae710410-0234-4481-8798-071c1c2d2332;
 Thu, 16 Sep 2021 12:04:12 +0000 (UTC)
Received: from AM6P191CA0042.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::19)
 by AM0PR08MB5138.eurprd08.prod.outlook.com (2603:10a6:208:15a::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 12:04:10 +0000
Received: from VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:7f:cafe::2a) by AM6P191CA0042.outlook.office365.com
 (2603:10a6:209:7f::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 12:04:10 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT031.mail.protection.outlook.com (10.152.18.69) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 12:04:10 +0000
Received: ("Tessian outbound 2e7020cc8961:v103");
 Thu, 16 Sep 2021 12:04:10 +0000
Received: from 1cbd12f9a1ba.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0B65E5BB-3C85-4397-B9CD-47A2958626D4.1; 
 Thu, 16 Sep 2021 12:04:03 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1cbd12f9a1ba.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 12:04:03 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR3PR08MB5836.eurprd08.prod.outlook.com (2603:10a6:102:8f::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 12:04:00 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Thu, 16 Sep 2021
 12:04:00 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0079.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 12:03:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae710410-0234-4481-8798-071c1c2d2332
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SDoawRWnyDQaw9PR09u3yHVtj17NII4KBcSYmjun2sA=;
 b=Ucqhr9frztnK6yJnY1h82XikomkLeVnRfe1UtawUAUxXiYrPth1pyTpF/B5hNBJUfYh8bPD4VQUwwrpCWi4IVRGCEg7D+R1Dzwnganb/vZyiozvk3nEg1mLE6xqyblG/xuH8q1+igbPZGrK9Hcim+TfDd/G0LRFsJ/VZY8giRmY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e240d2895d0d87ee
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eRobXTIn7yD6XEHX2b7O2zqfQeGOgsDGNEdL+TqBACALwQ6GEJpP4ptzenDoWETPGD+V/rdN6zgVN8rzMYz5TQ96lZrmMoCYTZM/u+UiUV9dsKe5LjeQJdvOvEQKRXt6aGvahUnvbZRrz+Cwu9I5HquX2wSoalvwWGD/rDW6UdJ1slN7aXt7TvgrSqkokCTvDeal2DTAyC+DroJdxSl+sL05obhwk3INdmHsCB8v8lY3vSSwfynpsQW9XkwZLcjs2jwLuBJ0B/Nm8HOTF2Ed+cKLRrDVGUoF3GBWNKyxKjrboT3fVDr7LmzNbckbsYZu5h1r88akEnnY8N8BL43CyQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=SDoawRWnyDQaw9PR09u3yHVtj17NII4KBcSYmjun2sA=;
 b=d6shWSm9PFoDg1bZIrLLdFj1TnClaPGWsZG5GO0VnTzpLCvQyfdbtxAKRZnhF8Szw5XQnP65Fm4C47qHjVlI19J13rNb7t+YGWJw5G2iLZQMeDVOiixaM6gKvLe6wUR3ZGPMlzNPUGnpli7UL1OO7atSNvsmxEj09kj1m9QCj0kgcQO74po+1vFgO/1BfM83QTo8pb6am6bM10MLpCx+mKtQkvOPBuswrGFDZwloYH4ewB+Abx8BsTWSlN8/vEmF2ycK/TxxEQZ7R0npghad++DK/bEwx/ojNF3X9t0+/OnfQWLWpwE4ehQ/++tg0yhe6hmGomCx3wBngt7KH1VW0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SDoawRWnyDQaw9PR09u3yHVtj17NII4KBcSYmjun2sA=;
 b=Ucqhr9frztnK6yJnY1h82XikomkLeVnRfe1UtawUAUxXiYrPth1pyTpF/B5hNBJUfYh8bPD4VQUwwrpCWi4IVRGCEg7D+R1Dzwnganb/vZyiozvk3nEg1mLE6xqyblG/xuH8q1+igbPZGrK9Hcim+TfDd/G0LRFsJ/VZY8giRmY=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
Date: Thu, 16 Sep 2021 13:03:53 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <CDC5AD61-D71B-46BC-952B-DE592879F9F3@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0079.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8::19) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ed5ab037-2cf0-4103-5cd9-08d9790a17d5
X-MS-TrafficTypeDiagnostic: PR3PR08MB5836:|AM0PR08MB5138:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB5138A9FAF311DC8F5AE76D42E4DC9@AM0PR08MB5138.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2399;OLM:2399;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 b5w8VrAkzwkOUa0B5Mx9FF9qVsQxPxhqZZgO+E2Jyc7K/vn7FojpcwCcVNR9z8NvH1RjF2o/7T0rm4I7OLq9PDC8UY7FN57zFNGGqXSmdZMWLiFB6OBKiWRY9BdSxz1TGVbXa6kIfCiThGGngIUEYPAG8AulWIiUPS/MQ9iKmTmIJf7k0wKwEUNxRkXDvJNqefAcggsTTh8eTfoLD9VPt56CyP4wfcpcGmP95BmYoYmgPYNE6eH+AEewkGjZxqJkK+E20/Ah1SGtpzuT56CfoQEUP1TGFDyhpfnPj1GhN1uqAj2EuTEq+haErJ5/bqv1D+ttfWm4voNwdkiUq/gGEWZ164s3ycFUssOTwchLeNg6pAiFk7n6sPRcHhjp4Sva5RfiZl2L0gtbPO7gZTpRT1nI2ikT8RY71wLWBVJjKNZMvjPYbyb3ZLmAaUjVq1lSxSj9iT6lWBaH9IO2zjv/pjpqrOWhsnlJpehTwigTzjpnupTvwhVBtbG7uzK5qrvvdZnApHQHnl9s7dxq9vNWwVUhVe/iPQ3PXL0SbNPdsQ+rS5xX9goMMXUH9LoVcAJ4dn5OZBrTe5gCj+zhrBKdFgLQDMt36tMKp8APjbRcQpi9I8LT5oHhwcYeF+Gj5eJpYSFxkjnPUlwcufq6nJt0HPaJZ4uilBgS0Vgs+0nXAPY8S1mEPvvyXJbIVF23iy7NraVdYBNf+qY9mMHhWzpSrO3V4KZI3N0+0qiYu4bEBiq+scxtVJ1Apg8PygfQz6bMr06h6aWpLdJlAU406mAnEw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(66556008)(66946007)(66476007)(53546011)(6506007)(86362001)(508600001)(44832011)(316002)(54906003)(52116002)(5660300002)(26005)(30864003)(2616005)(83380400001)(956004)(186003)(2906002)(36756003)(6486002)(8676002)(8936002)(33656002)(4326008)(6512007)(38350700002)(38100700002)(6666004)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5836
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1b062559-fb6f-46c6-4375-08d9790a1156
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	B2HMWyi1B9pyqP8Adp5IDho4WKG4e5LenDkmccerFG57oJ+akWKotLy8OVzU3u7DI7ZtJ8j7V3a88XCwTkoA1Ioh147nWmZD8F0kLKXUH8mT/SfUbqK8Hfn1lJubvH1zUrwR4ckPEVUibE8Y887lrtXw9J7BZht8GhAoanD+ij3l6mj2uw80NNQmDZI2KJDf0RFJ68l83CFtNBQH6W9Cz9lNtgLWaABQxv4SR2TTRTDMctvfA6tDCbNrRc4R8UifOKEmo25CtrROawYUb92vSKzup3KAFex6vw+xkCQkXe3QK0FMfZX90POnVycaxnPy1d+Yn06/VDopB1y9kjTWj3heJ1LlbXJ1eyGZ579flynBsZMhl+Xq5Tc/x5GGKenDZoE04alTT6Zk0WjSRkNQhKlyTRta+FSdv+e7/v4zmuQsCo6B8QWEUoaUhAelFC7cLnvW8YrDD2RUL0Z9miLgMs0sGlmAC2xOw8wVs9yCLp8slF3u/Kl3GbDrsQcz2QadoRFDA4ReqIOaTA0WS6w7CeJcVM7SQrzrTejACh8j/ARMyRlfyzmb0/XGf948+9ahbNQ9EQVgjB2AsPXOOCDh93EHvJNAc/m9mAolo72LQGk43x1NHdone5T/rlOVk2XvMmVw3gGV9zWyPLnkbtzosG4/mqPZBo91tm6mueLKne6ko3H5Gw1UMastZO5KbkGnHFO3ykleBU9dDKtCKVvszCY7HNDoGfpISIFluLZ3+/o=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(136003)(376002)(39860400002)(346002)(36840700001)(46966006)(4326008)(8936002)(82740400003)(26005)(6512007)(6666004)(70206006)(33656002)(70586007)(6862004)(36756003)(47076005)(30864003)(82310400003)(81166007)(356005)(5660300002)(316002)(44832011)(83380400001)(86362001)(6506007)(8676002)(478600001)(54906003)(2906002)(53546011)(6486002)(336012)(186003)(36860700001)(2616005)(107886003)(956004)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 12:04:10.4639
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ed5ab037-2cf0-4103-5cd9-08d9790a17d5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5138



> On 16 Sep 2021, at 02:16, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Wed, 15 Sep 2021, Luca Fancellu wrote:
>> This patch introduces the support for dom0less configuration
>> when using UEFI boot on ARM, it permits the EFI boot to
>> continue if no dom0 kernel is specified but at least one domU
>> is found.
>>=20
>> Introduce the new property "uefi,binary" for device tree boot
>> module nodes that are subnode of "xen,domain" compatible nodes.
>> The property holds a string containing the file name of the
>> binary that shall be loaded by the uefi loader from the filesystem.
>>=20
>> Update efi documentation about how to start a dom0less
>> setup using UEFI
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>> ---
>> docs/misc/efi.pandoc        |  37 ++++++
>> xen/arch/arm/efi/efi-boot.h | 244 +++++++++++++++++++++++++++++++++++-
>> xen/common/efi/boot.c       |  20 ++-
>> 3 files changed, 294 insertions(+), 7 deletions(-)
>>=20
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index ac3cd58cae..db9b3273f8 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -165,3 +165,40 @@ sbsign \
>> 	--output xen.signed.efi \
>> 	xen.unified.efi
>> ```
>> +
>> +## UEFI boot and dom0less on ARM
>> +
>> +Dom0less feature is supported by ARM and it is possible to use it when =
Xen is
>> +started as an EFI application.
>> +The way to specify the domU domains is by Device Tree as specified in t=
he
>> +[dom0less](dom0less.html) documentation page under the "Device Tree
>> +configuration" section, but instead of declaring the reg property in th=
e boot
>> +module, the user must specify the "uefi,binary" property containing the=
 name
>> +of the binary file that has to be loaded in memory.
>> +The UEFI stub will load the binary in memory and it will add the reg pr=
operty
>> +accordingly.
>> +
>> +An example here:
>> +
>> +    domU1 {
>> +        #address-cells =3D <1>;
>> +        #size-cells =3D <1>;
>> +        compatible =3D "xen,domain";
>> +        memory =3D <0 0x20000>;
>> +        cpus =3D <1>;
>> +        vpl011;
>> +
>> +        module@1 {
>> +            compatible =3D "multiboot,kernel", "multiboot,module";
>> +            uefi,binary =3D "vmlinuz-3.0.31-0.4-xen";
>> +            bootargs =3D "console=3DttyAMA0";
>> +        };
>> +        module@2 {
>> +            compatible =3D "multiboot,ramdisk", "multiboot,module";
>> +            uefi,binary =3D "initrd-3.0.31-0.4-xen";
>> +        };
>> +        module@3 {
>> +            compatible =3D "multiboot,ramdisk", "multiboot,module";
>> +            uefi,binary =3D "passthrough.dtb";
>> +        };
>> +    };
>=20
> Can you please also update docs/misc/arm/device-tree/booting.txt ?
> Either a link to docs/misc/efi.pandoc or a definition of the uefi,binary
> property (mentioning that it is EFI-only.)

Yes I will update it.

>=20
>=20
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index 5ff626c6a0..8d7ced70f2 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -8,9 +8,39 @@
>> #include <asm/setup.h>
>> #include <asm/smp.h>
>>=20
>> +typedef struct {
>> +    char* name;
>> +    int name_len;
>> +} dom0less_module_name;
>> +
>> +/*
>> + * Binaries will be translated into bootmodules, the maximum number for=
 them is
>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DT=
B
>> + */
>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
>> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
>> +static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_=
MODULES];
>=20
> I suggest a slightly different model where we don't call AllocatePool to
> allocate dom0less_module_name.name and instead we just set the pointer
> directly to the fdt string. There is no risk of the fdt going away at
> this point so it should be safe to use.

Yes I thought about this approach but since I was not sure how the DTB beha=
ves when we modify
It to add the reg property or to modify the module name, then I used this o=
ther approach.
Are you sure that the pointed memory will stay the same after we modify the=
 DTB? My main concern
was that the DTB structure was going to be modified and the string I was po=
inting in the DTB memory
can be relocated elsewhere.=20

>=20
> Also, I don't think we need a global array of struct file, we only
> really need 1 struct file which would be freed immediately after loading
> to memory. We do need to remember the address and size in memory though.
> So I would do something like:
>=20
> typedef struct {
>    const char* name;
>    int name_len;
>    EFI_PHYSICAL_ADDRESS addr;
>    UINTN size;
> } dom0less_module_name;
>=20
> /*
> * Binaries will be translated into bootmodules, the maximum number for th=
em is
> * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> */
> #define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MO=
DULES];
>=20
>=20
> The purpose is to reduce memory allocations and memory consumption.

Yes I can do that.

>=20
>=20
>> +static uint32_t __initdata dom0less_modules_available =3D MAX_DOM0LESS_=
MODULES;
>> +static uint32_t __initdata dom0less_modules_idx =3D 0;
>> +
>> +#define ERROR_DOM0LESS_FILE_NOT_FOUND -1
>> +
>> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>> void __flush_dcache_area(const void *vaddr, unsigned long size);
>>=20
>> +static int __init get_dom0less_file_index(const char* name, int name_le=
n);
>> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handl=
e,
>> +                                              const char* name, int nam=
e_len);
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int module_node_offset,
>> +                                               int reg_addr_cells,
>> +                                               int reg_size_cells);
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells);
>> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);
>> +
>> #define DEVICE_TREE_GUID \
>> {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, =
0xe0}}
>>=20
>> @@ -552,8 +582,209 @@ static void __init efi_arch_handle_module(const st=
ruct file *file,
>>                          kernel.size) < 0 )
>>             blexit(L"Unable to set reg property.");
>>     }
>> -    else
>> +    else if ( !((file >=3D &dom0less_files[0]) &&
>> +               (file <=3D &dom0less_files[MAX_DOM0LESS_MODULES-1])) )
>> +        /*
>> +         * If file is not a dom0 module file and it's not any domU modu=
les,
>> +         * stop here.
>> +         */
>>         blexit(L"Unknown module type");
>=20
> Without &dom0less_files we would have to do without this sanity check.

Sure, it will simplify to=20
+ else if ( file !=3D &dom0less_file )        =20

>=20
>=20
>> +    /*
>> +     * dom0less_modules_available is decremented here because for each =
dom0
>> +     * file added, there will be an additional bootmodule, so the numbe=
r
>> +     * of dom0less module files will be decremented because there is
>> +     * a maximum amount of bootmodules that can be loaded.
>> +     */
>> +    dom0less_modules_available--;
>> +}
>> +
>> +/*
>> + * This function checks for a binary previously loaded with a give name=
, it
>> + * returns the index of the file in the dom0less_files array or a negat=
ive
>> + * number if no file with that name is found.
>> + */
>> +static int __init get_dom0less_file_index(const char* name, int name_le=
n)
>> +{
>> +    int ret =3D ERROR_DOM0LESS_FILE_NOT_FOUND;
>> +
>> +    for (uint32_t i =3D 0; i < dom0less_modules_idx; i++)
>=20
> uint32_t i;
>=20
> for ( i =3D 0; i < dom0less_modules_idx; i++ )

Will fix that.

>=20
>=20
>> +    {
>> +        dom0less_module_name* mod =3D &dom0less_bin_names[i];
>> +        if ( (mod->name_len =3D=3D name_len) &&
>> +             (strncmp(mod->name, name, name_len) =3D=3D 0) )
>> +        {
>> +            ret =3D i;
>> +            break;
>> +        }
>> +    }
>> +    return ret;
>> +}
>> +
>> +/*
>> + * This function allocates a binary and keeps track of its name, it
>> + * returns the index of the file in the dom0less_files array.
>> + */
>> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handl=
e,
>> +                                              const char* name, int nam=
e_len)
>> +{
>> +    dom0less_module_name* file_name;
>> +    union string module_name;
>> +    struct file* file;
>> +    uint32_t ret_idx;
>> +
>> +    /*
>> +     * Check if there is any space left for a domU module, the variable
>> +     * dom0less_modules_available is updated each time we use read_file=
(...)
>> +     * successfully.
>> +     */
>> +    if ( !dom0less_modules_available )
>> +        blexit(L"No space left for domU modules");
>> +    module_name.s =3D (char*) name;
>> +    ret_idx =3D dom0less_modules_idx;
>> +    file =3D &dom0less_files[ret_idx];
>> +
>> +    /* Save at this index the name of this binary */
>> +    file_name =3D &dom0less_bin_names[ret_idx];
>> +
>> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(ch=
ar),
>> +                              (void**)&file_name->name) !=3D EFI_SUCCES=
S )
>> +        blexit(L"Error allocating memory for dom0less binary name");
>=20
> As far as I can tell we could just set file_name =3D name;

If you are sure I will modify that, I will wait your confirmation.

>=20
>=20
>> +    /* Save name and length of the binary in the data structure */
>> +    strlcpy(file_name->name, name, name_len);
>> +    file_name->name_len =3D name_len;
>> +
>> +    /* Load the binary in memory */
>> +    read_file(dir_handle, s2w(&module_name), file, NULL);
>> +
>> +    /* s2w(...) allocates some memory, free it */
>> +    efi_bs->FreePool(module_name.w);
>> +
>> +    dom0less_modules_idx++;
>> +
>> +    return ret_idx;
>> +}
>> +
>> +/*
>> + * This function checks for the presence of the uefi,binary property in=
 the
>> + * module, if found it loads the binary as dom0less module and sets the=
 right
>> + * address for the reg property into the module DT node.
>> + */
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                          int module_node_offset,
>> +                                          int reg_addr_cells,
>> +                                          int reg_size_cells)
>> +{
>> +    const void* uefi_name_prop;
>> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number +=
 \0 */
>> +    int uefi_name_len, file_idx;
>> +    struct file* file;
>> +
>> +    /* Read uefi,binary property to get the file name. */
>> +    uefi_name_prop =3D fdt_getprop(fdt, module_node_offset, "uefi,binar=
y",
>> +                                 &uefi_name_len);
>> +
>> +    if ( NULL =3D=3D uefi_name_prop )
>> +        /* Property not found */
>> +        return;
>> +
>> +    file_idx =3D get_dom0less_file_index(uefi_name_prop, uefi_name_len)=
;
>> +    if (file_idx < 0)
>> +        file_idx =3D allocate_dom0less_file(dir_handle, uefi_name_prop,
>> +                                          uefi_name_len);
>> +
>> +    file =3D &dom0less_files[file_idx];
>> +
>> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->ad=
dr);
>> +
>> +    /* Rename the module to be module@{address} */
>> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
>> +        blexit(L"Unable to add domU ramdisk FDT node.");
>> +
>> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_=
cells,
>> +                     file->addr, file->size) < 0 )
>> +        blexit(L"Unable to set reg property.");
>> +}
>> +
>> +/*
>> + * This function checks for boot modules under the domU guest domain no=
de
>> + * in the DT.
>> + */
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells)
>> +{
>> +    /*
>> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device=
-tree}
>> +     * inside this node
>> +     */
>> +    for ( int module_node =3D fdt_first_subnode(fdt, domain_node);
>=20
> int module_node;
>=20
> for ( module_node =3D fdt_first_subnode(fdt, domain_node);
>=20

Will fix that.

>=20
>> +          module_node > 0;
>> +          module_node =3D fdt_next_subnode(fdt, module_node) )
>> +    {
>> +        if ( (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,kernel") =3D=3D 0) |=
|
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,ramdisk") =3D=3D 0) =
||
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,device-tree") =3D=3D=
 0) )
>> +        {
>> +            /* The compatible is one of the strings above, check the mo=
dule */
>> +            handle_dom0less_module_node(dir_handle, module_node, addr_c=
ells,
>> +                                        size_cells);
>> +        }
>> +    }
>> +}
>> +
>> +/*
>> + * This function checks for xen domain nodes under the /chosen node for=
 possible
>> + * domU guests to be loaded.
>> + */
>> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle)
>> +{
>> +    int chosen;
>> +    int addr_len, size_len;
>> +
>> +    /* Check for the chosen node in the current DTB */
>> +    chosen =3D setup_chosen_node(fdt, &addr_len, &size_len);
>> +    if ( chosen < 0 )
>> +        blexit(L"Unable to setup chosen node");
>> +
>> +    /* Check for nodes compatible with xen,domain under the chosen node=
 */
>> +    for ( int node =3D fdt_first_subnode(fdt, chosen);
>> +          node > 0;
>> +          node =3D fdt_next_subnode(fdt, node) )
>> +    {
>> +        int addr_cells, size_cells, len;
>> +        const struct fdt_property *prop;
>> +
>> +        if ( fdt_node_check_compatible(fdt, node, "xen,domain") !=3D 0 =
)
>> +            continue;
>> +
>> +        /* Get or set #address-cells and #size-cells */
>> +        prop =3D fdt_get_property(fdt, node, "#address-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#address-cells not found in domain node.");
>> +
>> +        addr_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        prop =3D fdt_get_property(fdt, node, "#size-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#size-cells not found in domain node.");
>> +
>> +        size_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        /* Found a node with compatible xen,domain; handle this node. *=
/
>> +        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_=
cells);
>> +    }
>> +
>> +    if ( dom0less_modules_idx > 0 )
>> +        return true;
>> +
>> +    return false;
>> }
>>=20
>> static void __init efi_arch_cpu(void)
>> @@ -562,8 +793,19 @@ static void __init efi_arch_cpu(void)
>>=20
>> static void __init efi_arch_blexit(void)
>> {
>> +    uint32_t i =3D 0;
>>     if ( dtbfile.need_to_free )
>>         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
>> +    /* Free dom0less files if any */
>> +    for ( ; i < dom0less_modules_idx; i++ )
>> +    {
>> +        /* Free dom0less binary names */
>> +        efi_bs->FreePool(dom0less_bin_names[i].name);
>> +        /* Free dom0less binaries */
>> +        if ( dom0less_files[i].need_to_free )
>> +            efi_bs->FreePages(dom0less_files[i].addr,
>> +                              PFN_UP(dom0less_files[i].size));
>> +    }
>>     if ( memmap )
>>         efi_bs->FreePool(memmap);
>> }
>> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
>> index 758f9d74d2..65493c4b46 100644
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop =3D NULL;
>>     union string section =3D { NULL }, name;
>>     bool base_video =3D false;
>> -    const char *option_str;
>> +    const char *option_str =3D NULL;
>>     bool use_cfg_file;
>> +    bool dom0less_found =3D false;
>>=20
>>     __set_bit(EFI_BOOT, &efi_flags);
>>     __set_bit(EFI_LOADER, &efi_flags);
>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>             efi_bs->FreePool(name.w);
>>         }
>>=20
>> -        if ( !name.s )
>> -            blexit(L"No Dom0 kernel image specified.");
>> -
>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>=20
>> -        option_str =3D split_string(name.s);
>> +#ifdef CONFIG_ARM
>> +        /* dom0less feature is supported only on ARM */
>> +        dom0less_found =3D check_dom0less_efi_boot(dir_handle);
>> +#endif
>=20
> Rather than an #ifdef here you can simply implement
> check_dom0less_efi_boot on x86 as a static inline returning always
> false.

Sure I will create that on x86 code and I will update the code here.

>=20
> Also, we are under the if ( use_cfg_file ) code path. So maybe it is
> reasonable that dom0 is required if we are booting with use_cfg_file
> =3D true. After all, it is specified as a required property today of
> xen.cfg.
>=20
> If you follow my suggestion with an explicit enabled/disabled of xen.cfg
> from device tree, a true dom0less configuration could be fully specified
> without xen.cfg.
>=20
> If we do that, then here probable we don't need to change this code path.
>=20

Please check my reply on the previous patch.

Cheers,

Luca

>=20
>=20
>> +        if ( !name.s && !dom0less_found )
>> +            blexit(L"No Dom0 kernel image specified.");
>> +
>> +        if ( name.s !=3D NULL )
>> +            option_str =3D split_string(name.s);
>>=20
>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str=
) )
>> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_st=
r)) &&
>> +             (name.s !=3D NULL) )
>>         {
>>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>>             efi_bs->FreePool(name.w);



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 12:15:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 12:15:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188430.337582 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqIl-0006iN-AZ; Thu, 16 Sep 2021 12:15:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188430.337582; Thu, 16 Sep 2021 12:15:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqIl-0006iG-73; Thu, 16 Sep 2021 12:15:39 +0000
Received: by outflank-mailman (input) for mailman id 188430;
 Thu, 16 Sep 2021 12:15:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQqIj-0006iA-C8
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 12:15:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cbe5aea4-16e7-11ec-b5d7-12813bfff9fa;
 Thu, 16 Sep 2021 12:15:36 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-8iVdFdoIOA-mKxuFAPbM0A-1; Thu, 16 Sep 2021 14:15:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3119.eurprd04.prod.outlook.com (2603:10a6:802:10::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 16 Sep
 2021 12:15:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 12:15:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Thu, 16 Sep 2021 12:15:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cbe5aea4-16e7-11ec-b5d7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631794535;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KRmqhwUR41eofkPr8SzID7w4eBRv+gV/YQjDk7/DE2c=;
	b=efJ2LBD2Fn2+fopzMOtYGylLJYPZ0pmQDwWULl/YCn1gKA404Pkg7KhHeqtXpaIEHLAKfH
	lkgG/QfXfcdL6fwOTuTeklYOv2ycUYoJb6esDKeKdlFtQJ5SdqrqBuohtXBmJoZ+M4vZZ4
	KDc32YVhrloq0x+59bBlppKalRgazb4=
X-MC-Unique: 8iVdFdoIOA-mKxuFAPbM0A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XDVOzpsz4uLY2EZcPehdd6W+cPFD1oPWYB/Em4wEV48Gud0YNMD+QYQpLYTPxZWkJ+aWHIWYJeRoravoJz7HVTzJ8jQwzkpE5RE99CrSwb/PgjubAu9MbINlOAeyGzHMEqi4yuNglfLqUmfTNlvr8KLVlyTuJMNFEN/9DB0X1UO+xhnGFa46EUM+tVzOrjUfnzkh0+W6izFZ57VVBJR4G8x7nMYQ7CU2IvVB4DzP6ldvhtecpt1mKJtQ6aZE5FUpQJaN1IY+U//xpc2GOSL+z+6w1rYtq0C+Ud32flIr7IOKpOX2VL9JoS+o872TF3hupvx17i4rTmiiTUDaf5rzJg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Wwftvcb7mTw2LJsR9vHNEmfugrmfZACJVuXclaSOzqo=;
 b=Vxs7l50EJdoU7LyBZ3F0iNWZwsyV4bMEKuj47uAvnC3/LberLbyRXYb78WFkrUqiwdUx9qlVAYbO7wxzJYCoydPF5dc4ud6DePkRh7zVAI7oGtnkkNtdgZ5KYbnQ6RhvxzgYDo/Jbx1bQyLdSudtmjPgjhx62DpPCc+Z2o0VMK6s4Ieyx83RjwQ7SxWs++2detmsBzZaNmEo0XJfF1yFJEZfa7IT75JGOA+60CkNrlLCsToLcarvW1uYPvQZQiyaofqKUtedVM7ctWB+lmz5ZLEPWNAWugQ49Mst+K84lTiRpAsM6Xo5EezonzpUILQNDycUC5/4CTRpDIiDdf28gA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
Date: Thu, 16 Sep 2021 14:15:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0070.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 43906351-26dd-4654-6f82-08d9790badef
X-MS-TrafficTypeDiagnostic: VI1PR04MB3119:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB3119745041485673539202D9B3DC9@VI1PR04MB3119.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KeNXqxX2ugHif5tCK9mCyjnOHloLvz9PDGO+Po8XNa26KtWJ4BvGmNb6eUapujK+ZsujYocnuU2ysp2WA5MveLSrb2nnktmcZw9m4WYQR0Qj86UWU0yHMC8tlkEAiFO51Oqfo3/X+pOJIQ/HRX2hhIVKxkiBcwYWwWFvF4ESlw3M9t8jUmmJzmIuYa0v/F1fqClz7IoDQj6M1W/39yFg22fobyEbgeOaeKaMtr2drcXmtR4YRWX29kXtVw8ygGml+RbBEM38yIAPqwF4O2O0FU1NQFvFR3nYtq0q1kEYUTwN43528aEsv8onY5voM7WGGaIewY43f0NuFROclGhgKYXQJHUFA45SLPnopJRzrld87BNFSp5e/u5QcX3PSfi2rfdfqeNNgQFD0Qf2S/QF+hlwKVjGenSq/hUrrMP4qAXLkjxwGOE1SagFJen5FFVR1Pa+4x1HbfQovLneNF/cgNCl+nRKM2psKLe8dZ9BG4VkRJ5yhT5sNWan6yLlcGWWkFa/sohSK0vw5BktfCbj3jZrMiqjf4ovDs7aZbVf+SY05zZA4afaA8c3r6TxTtxOSWqy3sN+w+vzt18UDH3tDk1Xr+wx8si0Py6CW4lfpyqbJyznYhB/u6itv0Q5FaAV/fFQrN78DnuKf7LrV54P226a7eLBEpzlT/8/+RsCr+tDti6/QyRyhkiIE825gq8l5th9+DDUn/Gc/bZ89OwJ1TdlvWyM8Oe0Y2tGxuTapyUZZO8/9ynoYKcACjUR/z4Z
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(26005)(4326008)(86362001)(31696002)(8676002)(186003)(16576012)(316002)(38100700002)(7416002)(53546011)(54906003)(5660300002)(8936002)(6916009)(2616005)(956004)(66556008)(66946007)(2906002)(66476007)(508600001)(36756003)(31686004)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?e6BYLD6d0QdWOqxWrlStgh2iZLtuDboiDeK81uS/WWsgAn7GI6IPTX3McIHh?=
 =?us-ascii?Q?n8qwOx/TmSZwe79kzoHmnH+NUOS/yFoJCw0Bp0smGuifvyny+v22RzXhwU0Y?=
 =?us-ascii?Q?sP1ThlVm9Q4h9AesneHdpCEIyMo6syUF+9+xb7IA5fBrHfbE4PVGtU48htCU?=
 =?us-ascii?Q?9N3O7oi6difoamqwxOssojjkpORXCGd8yfX7xO9KZzlxV1mKQ9LJbrJGGCUg?=
 =?us-ascii?Q?Dz6WvwNQ11aU1SjClbwa52KoFz+lqhy4QCVZUwOT067piRbZrJQhiFI2SpPf?=
 =?us-ascii?Q?BuAHgmPqH2L8r25VhVz5dglOYLNJ5X3f+YTfCEQoLxxXf+nWxpzVE4S84CaH?=
 =?us-ascii?Q?LLJNI3ed7xTEI4dVcUuD2xHSKLYvZuadb+3+CtfzjO6mqqhwseeAYcYMXoPW?=
 =?us-ascii?Q?dBGHta1mjKIUl6OPVbEVzFyxLHiN6DhKe7T3diRhbjrLlJzC4RXQ4+K3EOzF?=
 =?us-ascii?Q?y5MzA5rDqNgka7QImgWVWbWu8fN0caq/EkomR2KWpZxk25OSjcn92t6YEvdY?=
 =?us-ascii?Q?qMDcwHXHuKHb2zT4LZ1FIEOPms20NkzV33KQ6EYzUr/wb7bV4hArMgxCHsfW?=
 =?us-ascii?Q?RpvzH/cpfmn8eEervls+yWPAEsXi04dZEGmUBvC/PpU6C8pRA3IysmeG2KXI?=
 =?us-ascii?Q?3p53lbBYpCVV92Gwe0ZJ6W4PODnbCP+VtGOADz4CSo9y5wcHTeC2AJqs7vi2?=
 =?us-ascii?Q?dziFXEKaqK0H9ZGOFd6etgOgJLt9GzPrO2Jctto7JtvdgB6m+D63EDBB5LtB?=
 =?us-ascii?Q?+Ljle83IZ9irpZ+QcaKvG0TH4hoMCRflBGQ9Whfcpg3G2CSVacv4+TOHN2pR?=
 =?us-ascii?Q?/5Z+QMOLRSNU6+EpL/US7EC1GT54enH8hx8VQAYCUuaclqu+lGcplSraLIuO?=
 =?us-ascii?Q?ajvWOg1j8pLAivEk3xhlkC2y+38S8Tx9RsEOtcYAvO+basb8BXv0lCfmLkb5?=
 =?us-ascii?Q?SUhIGVW3J//gDTJuJwcuCCGKbL7c+yFZ29S/+bLT2JHOWWgleXuNbFNBGhdw?=
 =?us-ascii?Q?bHcsxE8T819QjNkxmTFTZ0lge7yg8U9eBsBuMeNEVZfHPhplux9VzWlnNHYY?=
 =?us-ascii?Q?CNOo+/X5W3kC7aSxaJVyTsGqMjFHqeOw1gX8cbc9z7aa4RYJUSWXKQeuy8Gk?=
 =?us-ascii?Q?ltkTd5F5zRADGSI698CNSKJF6CVMLBpwcqsg97GRG++QtTnGvjFIDr+ALDyn?=
 =?us-ascii?Q?qSU+GZv/f8OqVJM7HKDU1LHEymE5OIoiPuMcKHL0wblxRSLz6bq+wIhRpwdi?=
 =?us-ascii?Q?AZUz/m0rk6rjQyg6HRJQaSoLJ8MiLfPV2JfAAaAtudDoP2nrlih9ZBOATSmJ?=
 =?us-ascii?Q?Ho/C7k7Ys9HVTwdglu8v7T+A?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43906351-26dd-4654-6f82-08d9790badef
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 12:15:32.0376
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Rh1cgEOoa68xeFWNiZ+rVJoYUIXaxjVi/2FoOhiTx/KaHy6SnpxmTGlkV1Df2iimGoNyL176px0dBPEWLaHqjQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3119

On 16.09.2021 13:28, Luca Fancellu wrote:
>> On 16 Sep 2021, at 09:46, Jan Beulich <jbeulich@suse.com> wrote:
>> On 15.09.2021 16:26, Luca Fancellu wrote:
>>> --- a/xen/arch/arm/efi/efi-boot.h
>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>> @@ -8,9 +8,39 @@
>>> #include <asm/setup.h>
>>> #include <asm/smp.h>
>>>
>>> +typedef struct {
>>> +    char* name;
>>
>> Misplaced *.
>=20
> I was looking in the CODING_STYLE and I didn=E2=80=99t found anything tha=
t mandates
> char *name; instead of char* name; but if you prefer I can change it sinc=
e I have
> to do some modification to the patch.

I don't think it's reasonable to spell out there every little detail.
You should also take adjacent code into consideration, making yours
match. Issues only arise when there's bad code that you happen to
look at.

>>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TA=
BLE *SystemTable)
>>>             efi_bs->FreePool(name.w);
>>>         }
>>>
>>> -        if ( !name.s )
>>> -            blexit(L"No Dom0 kernel image specified.");
>>> -
>>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>>
>>> -        option_str =3D split_string(name.s);
>>> +#ifdef CONFIG_ARM
>>> +        /* dom0less feature is supported only on ARM */
>>> +        dom0less_found =3D check_dom0less_efi_boot(dir_handle);
>>> +#endif
>>> +
>>> +        if ( !name.s && !dom0less_found )
>>
>> Here you (properly ) use !name.s,
>=20
> This is not my code, I just added && !dom0less

Correct, which is why this is fine.

>>> +            blexit(L"No Dom0 kernel image specified.");
>>> +
>>> +        if ( name.s !=3D NULL )
>>
>> Here you then mean to omit the "!=3D NULL" for consistency and brevity.
>=20
> I usually check explicitely pointers with NULL, is it something to be avo=
ided in Xen?
> There are some industrial coding standards that says to avoid the use of =
! operator
> with pointers. Is it important here to do !name.s instead of the solution=
 above?

As you can see from neighboring code, we prefer the shorter forms,
for being easier/shorter to read.

>>> +            option_str =3D split_string(name.s);
>>>
>>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_st=
r) )
>>> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_s=
tr)) &&
>>
>> Stray parentheses.
>=20
> Will fix
>=20
>>
>>> +             (name.s !=3D NULL) )
>>
>> See above.
>=20
> Will fix
>=20
>>
>> I also don't think this logic is quite right: If you're dom0less,
>> why would you want to look for an embedded Dom0 kernel image?
>=20
> This is common code, that check is not from my patch.

It is you who is adding the name.s !=3D NULL part, isn't it?

> Before this serie, EFI boot requires that a dom0 module was passed, other=
wise
> the boot was stopped.
>=20
> This serie instead removes this requirement, letting the boot continue if=
 there is no dom0
> kernel.
>=20
> However (as in the old code) if the user embed the dom0 kernel in the ima=
ge, then it is
> legit to use it and if there are also other domUs specified by DT, then t=
he system will
> start the dom0 kernel and the domUs kernel as well.

This doesn't match what I would expect - if configuration tells
to boot dom0less, why would an embedded Dom0 kernel matter? I can
see that views might differ here; you will want to write down
somewhere what the intended behavior in such a case is.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 12:16:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 12:16:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188437.337593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqJK-0007KT-NQ; Thu, 16 Sep 2021 12:16:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188437.337593; Thu, 16 Sep 2021 12:16:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqJK-0007KL-Ju; Thu, 16 Sep 2021 12:16:14 +0000
Received: by outflank-mailman (input) for mailman id 188437;
 Thu, 16 Sep 2021 12:16:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=20iF=OG=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mQqJI-00074J-Ek
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 12:16:12 +0000
Received: from forward105o.mail.yandex.net (unknown [2a02:6b8:0:1a2d::608])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 11bc9e2e-3565-47e4-bc63-6d940a048772;
 Thu, 16 Sep 2021 12:16:00 +0000 (UTC)
Received: from myt6-48cb7adf5983.qloud-c.yandex.net
 (myt6-48cb7adf5983.qloud-c.yandex.net
 [IPv6:2a02:6b8:c12:4c9c:0:640:48cb:7adf])
 by forward105o.mail.yandex.net (Yandex) with ESMTP id 731FF4C43F7;
 Thu, 16 Sep 2021 15:15:57 +0300 (MSK)
Received: from myt6-efff10c3476a.qloud-c.yandex.net
 (myt6-efff10c3476a.qloud-c.yandex.net [2a02:6b8:c12:13a3:0:640:efff:10c3])
 by myt6-48cb7adf5983.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 B2htycwpgs-FvDm3JOh; Thu, 16 Sep 2021 15:15:57 +0300
Received: by myt6-efff10c3476a.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA
 id wX5Y1UDLGI-FtDKBARt; Thu, 16 Sep 2021 15:15:56 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11bc9e2e-3565-47e4-bc63-6d940a048772
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631794557;
	bh=g3vf8XCHFE0NhfW12scKRmlBoFDcRBGQvYgVAD2JBkY=;
	h=Message-Id:Date:Subject:To:From:Cc;
	b=IL4XfRKHasKz2dw63v8RALFFZD2Lku0dUKzAxPqwr7QoZaUQvskd3iDVZ07eMBdhG
	 lZOcf9QFYF0ye91MbNNpKl1C6DswKhoKrX40/nA8nLqQZleAjw4m9ZfgX5sSi1CjxP
	 Jte673XsjSAHWNnEwBNR/tQ3E8QQMY9n7pLJxxco=
Authentication-Results: myt6-48cb7adf5983.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
To: xen-devel@lists.xenproject.org
Cc: Dmitry Isaykin <isaikin-dmitry@yandex.ru>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v3] tools/xl: fix autoballoon regex
Date: Thu, 16 Sep 2021 15:15:21 +0300
Message-Id: <dd3a1e1a7a3f8e7bca18dd4779efbc2af01decc7.1631793876.git.isaikin-dmitry@yandex.ru>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This regex is used for auto-balloon mode detection based on Xen command line.

The case of specifying a negative size was handled incorrectly.
>From misc/xen-command-line documentation:

    dom0_mem (x86)
    = List of ( min:<sz> | max:<sz> | <sz> )

    If a size is positive, it represents an absolute value.
    If a size is negative, it is subtracted from the total available memory.

Also add support for [tT] granularity suffix.
Also add support for memory fractions (i.e. '50%' or '1G+25%').

Signed-off-by: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
---
Changes in v3:
- add support for [tT] granularity suffix
- add support for memory fractions

Changes in v2:
- add missing Signed-off-by tag
---
 tools/xl/xl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..8500b3ac57 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -81,7 +81,7 @@ static int auto_autoballoon(void)
         return 1; /* default to on */
 
     ret = regcomp(&regex,
-                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
+                  "(^| )dom0_mem=((|min:|max:)(-?[0-9]+[bBkKmMgGtT]?)?(\+?[0-9]+%)?,?)+($| )",
                   REG_NOSUB | REG_EXTENDED);
     if (ret)
         return 1;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 12:30:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 12:30:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188448.337603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqXP-0001WG-0o; Thu, 16 Sep 2021 12:30:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188448.337603; Thu, 16 Sep 2021 12:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQqXO-0001W9-Tv; Thu, 16 Sep 2021 12:30:46 +0000
Received: by outflank-mailman (input) for mailman id 188448;
 Thu, 16 Sep 2021 12:30:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQqXN-0001Vn-Un
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 12:30:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e926aa84-16e9-11ec-b5db-12813bfff9fa;
 Thu, 16 Sep 2021 12:30:44 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-38-HRmPBIvxNIiTwsb10WjDrQ-1; Thu, 16 Sep 2021 14:30:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 12:30:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 12:30:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0133.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 12:30:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e926aa84-16e9-11ec-b5db-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631795443;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zlr36CY5+SAAvpoICBA0IsyuMWuU3ehiv1naH4j0+Ak=;
	b=ABVrukG3Chh2aWqca7VYFw3Pm5o0dwHLQs0kr6rXL38nv8wS9wIrWtLQR+VCzPICrQWus+
	BH6ZvOPKvEw7+U1nC71+XEWl2FMZ7DH00egSCCLLaxZLfxUvctPZIF4DUvYqIt4xY44TcS
	kOoKcViHFpqJBT8BrkaMIdeFXnapxaM=
X-MC-Unique: HRmPBIvxNIiTwsb10WjDrQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gppEqpwIFCNAJIQALLqX/vmWmQxdVusccMFXayUiO/xQrwlEmQ25IkZXauFb6F95PRYoyjTCNKif169bFG/I2W/ul8E/aIKnd3SS56oJxyE7lTAMcykhJnC9pGFNnUootwOex7L4B0dmw5CZS2sVjesA2Q9/iiFWNo+x39pOiJ8j2ejDHKGAR9pPY3fuFbBN52PZwPsxbQ7DwP4KXs4yEUaNpW9k0cLq2MbLR9QeMoD+QKnCu00+iMyZosCOf0MKhtXSM1ldusA1IKRKTzUMh3Hvp+FL+hVkL/rYto7Jv1QAr94z2bioqzQLwl4w1JqmDoYGket4XuHmzmqMmUOvdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zlr36CY5+SAAvpoICBA0IsyuMWuU3ehiv1naH4j0+Ak=;
 b=cKqWtm3L4aTI+KAYNG89wgsO2oKyQdtOLqN3UZDaMcLoKwOF/nYrCM1iMj2bDzS31RUGTF4rsYBJ8mCElij7825Nu76ZH18+jiijWJTbeMmYBDgVwbC/WVt2sH5fTQ4cEsnzqGYQT8IiNUGLTU4wRG4qsxJnY4+kBVLTVxmY0EQCzXKV5K+iKP9W+VsgUS5Q4YzuLc3o/T7bs4vSwVFfX47ST4irKdR4Qtr7Et5CCoQz9yIBMoycB0J5pSuwM7DRsd4XD6Zyx21d4TGj1AVkurjCky0nPvDfWUHxmymBSpynt76WD47vs2tp6WsrEcILNPuLeysV/aUYtzDqxwZJ0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
Date: Thu, 16 Sep 2021 14:30:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0133.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5721166f-a56d-444a-68dc-08d9790dcbd2
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7152722EFA832F1ABF6AF8B9B3DC9@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HNhMmFcySbATRL4sRpfr9nsTN25rMxQ91jsxbbdEMesIUWsr50KRHbA2JTZqO5GGkK43+ohHmVHPS1v26me9039o59l7jS+5dXqIfpa9Kb/1gshN0anWJozl0/YrrTfXfsegX5NN/hvn2Jhsw0iMRRowEEK4NP9EdNDj7GpquTZnSYWz3vTBfu7/TYZBjreOX6vKTG3TDI2N4TcWujPPQSy3GfRuEonwJ5yO478Sr2zcce9Z1esvHhQx+AJYMQc+cZUoBwAdkogxJ49NOAgKa+hdmtCwYo8lWtD41FhSK6btB3e3YJh5QMylmKtcr/2fsd3XAjs2qPoxUi1OSiWwd9q2G+BM7Yaq0yq7iif1Rtvhq4Tpuj/H7gkUWQssowCmW2srOTrhpZpBK3qdAd7vD1SAqM8XqwX+kct0ChVK7Q4yAGTvzGl4aCzt9q/LdSRcsWiAS7PcStk1HgLl2UgSJK/X8584Y6MB0hOMsXDjQi9k4TFJZjBeq8hLxk/YTu9Fx0b/kMarXtVbGbs0W0vCDTmDv1M1by1YOn2UflIioenG+pgXH5raeUdmhCLRrkRSyXdKTY5g9RJG5cOl38+zYnY6wpJRr8Mj88u7oZWtQGj/Qqexwm+eaheACJCXaIzEdrh/mbGtrNnfOf8HapeODY45Y5aq1YbdcdEkvhc5q82NxV6ImoL2ZjoNi5ZZcqOIQK2xkuFv4PwqyIaU1oTcxegJo3ILdOdMp973mUtJslr5jVIE5RwWTLDNUQ6G85waRXm+s5rwCtSfBM9XomzlbQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(396003)(136003)(366004)(39860400002)(346002)(31686004)(6916009)(31696002)(53546011)(38100700002)(6486002)(36756003)(186003)(26005)(2906002)(956004)(478600001)(4326008)(5660300002)(83380400001)(86362001)(2616005)(66946007)(66476007)(54906003)(66556008)(316002)(16576012)(8676002)(8936002)(23180200003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NzRXVFBGTFM5Q0U5WUdRMjFELys5d21qNW5PSzNFU2d2ekU3WmIwYUFxK2Rh?=
 =?utf-8?B?N0hab3IwNllwTEFhM3I5cDMyemVYOWs4MlJmeFRhTmM5YStxcjl0aEprbk9I?=
 =?utf-8?B?bGM5WThWeXNrSDZBUytOMUM1OWdxMHpEM0NvTkFIYVUyRGQ0RzJYRnRWaFFz?=
 =?utf-8?B?TVF4ZjFjMGlSUDd4WEt6Z2ZtSWp6WlIwWWozLzdqWEpXTkNkYzdFOUtNR0dS?=
 =?utf-8?B?VEs2Z0h4Z0wvSWNHUkRyTk0xQVBQOTM3Qm5vYTVYdVJMNjBjNGxHbFc1ZUxQ?=
 =?utf-8?B?L2JUUkJjeGpURXhVbDQxSWd4bjVTU3E4UXIzbWNRdHVZTGQ5MXByOWpjMHVs?=
 =?utf-8?B?LzVzQ1kwME9iNWpXZFlrcTZYZTYvU1JQd1phaURWOEhGNjdFUHpwTUgyZGJu?=
 =?utf-8?B?WU5GR2oycXpvSXVtMmpIcnhYY0czOWNHMHpMN0IwdTVwcVV4K21KeTlxRjU4?=
 =?utf-8?B?MmZ0akczQ1M5VXVHVTVaTXYxSkovNDVIbzczZ1ZlVitWLzY5Z0JVUVdSUTU1?=
 =?utf-8?B?Zjk4QnlwQlpYbWpsbXVXSlhTcldCNDdvNDRkY09Bdy9Kb3hBdGFPeEI5VVBl?=
 =?utf-8?B?eDkyOURhTXJ0Wm42dG54OU94NmFoaWp4VmZyMnVuQlZSMnAwVzBpU2F1eDUx?=
 =?utf-8?B?aHVKbDZBY0lROEZNL2ZYS3RZNVhYaHRaQlNPTFlOTm5oaXRrNkdPeDFVL0tR?=
 =?utf-8?B?cktjc0YreEtQWmwvNkxic0o0UWtxU1FoWHZqYzRFL3UzTkNCSCs3SjRUZ3Za?=
 =?utf-8?B?L2E2MGhYOVVlTHVUVW9HTGsxUjJqM3FwY09TcVpldHh6SFJubWZJL0lOcHA3?=
 =?utf-8?B?TVJFd3kyR1lsNlpIeEFseUlOUVg2azg2L3JHOFgxMjlDU29PNFZoU0I2OThB?=
 =?utf-8?B?YlRjQ0l4ODFQK0czMXJKUlFhYUR5VFVadDl3YjlZU3JUaEt2eXFXbXl5WWgr?=
 =?utf-8?B?UmFtR3UvSWFYWE5TNDhRR0lEQW8yeHg3ZzV4WFlFa1NvSFNZdnFNaHBiVnVt?=
 =?utf-8?B?d3ovYU5mVDlqbDBhblpPR0tWaWxyUDBqSTBIbFp5azQ5ZXJITDNia0RGZ2Vo?=
 =?utf-8?B?aWVaR21NYmhhZHUycUhpVTFyQmp0UzBKTVQweElpVW93OHFYdXNKM2JveXZt?=
 =?utf-8?B?R1BLYXZCdFI4eE1iUmw2ajBNZEZCRHQ2UHZpVnVlR0ZBeERhNGwzeVVTb2g1?=
 =?utf-8?B?MEppTTl2eFN3SHFMUnFhWkRTVHZaUC93UzFwbUYxc1QxWU1Kbkh6c3N6ZHZy?=
 =?utf-8?B?Y0Z0alZpZ0lCcmZ4QXc1TUhDV2VrNmFMSEZvMlJHK3hKVEJxbWE2QTBHREY4?=
 =?utf-8?B?NWZObktCNEQ1NDZSakwvd3E0Q0syUXY2UWxVMENiTTFRZWxIQVFqUG9GNTht?=
 =?utf-8?B?Nmg4UW93ZUphRmNwMHEvOTQvQ1hJMVBZWEhGRkVRMzhjT2lLTTY4M2pKdFgx?=
 =?utf-8?B?RXF6dC9xYVJncFE0Z2VMeExjU1RVYmhmT1NvbVFDOWRaa3VQT1lxRERWUUR6?=
 =?utf-8?B?OW1lM1FhbFFROEwvaGx2ZGc2M25KV2tjeGJOcUc5OTVmdTVhQ0xxdzZpTFAy?=
 =?utf-8?B?VEYwZERvWi9vQkJNdnFZUEViODFocEwrRVIxYmJYU3FiV241MUZkcThheHdR?=
 =?utf-8?B?VlA5MGE1Yk9ZdENnZlZUMFk5WXUxbVltZFEyNFBscHRmYmUvMkswQjU4NWhO?=
 =?utf-8?B?cGFHWUY1dU40cUx2d1VIZldWWVNoVkoyTVVsemR1cE16NXRjdlo3WUt6Qisy?=
 =?utf-8?Q?KVG+a0p9WGCVWvOe/wxZ+vdLJeZwzqa+n2IgvNh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5721166f-a56d-444a-68dc-08d9790dcbd2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 12:30:41.1685
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /fMgSAquxhxRnWEiRXswvWctSrn9j7tYkF3gbC3u9bmWaG9oFB5B+DFMbxubULmdgwIewwdEIxXlI+iWaVXOMA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

On 16.09.2021 13:10, Dmitry Isaikin wrote:
> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
> 
> This significantly speeds up concurrent destruction of multiple domains on x86.

This effectively is a simplistic revert of 228ab9992ffb ("domctl:
improve locking during domain destruction"). There it was found to
actually improve things; now you're claiming the opposite. It'll
take more justification, clearly identifying that you actually
revert an earlier change, and an explanation why then you don't
revert that change altogether. You will want to specifically also
consider the cleaning up of huge VMs, where use of the (global)
domctl lock may hamper progress of other (parallel) operations on
the system.

> I identify the place taking the most time:
> 
>     do_domctl(case XEN_DOMCTL_destroydomain)
>       -> domain_kill()
>            -> domain_relinquish_resources()
>                 -> relinquish_memory(d, &d->page_list, PGT_l4_page_table)
> 
> My reference setup: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Xen 4.14.
> 
> I use this command for test:
> 
>     for i in $(seq 1 5) ; do xl destroy test-vm-${i} & done
> 
> Without holding the lock all calls of `relinquish_memory(d, &d->page_list, PGT_l4_page_table)`
> took on my setup (for HVM with 2GB of memory) about 3 seconds for each destroying domain.
> 
> With holding the lock it took only 100 ms.

I'm further afraid I can't make the connection. Do you have an
explanation for why there would be such a massive difference?
What would prevent progress of relinquish_memory() with the
domctl lock not held?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 13:07:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 13:07:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188458.337615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQr6W-0005PW-UX; Thu, 16 Sep 2021 13:07:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188458.337615; Thu, 16 Sep 2021 13:07:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQr6W-0005PP-QS; Thu, 16 Sep 2021 13:07:04 +0000
Received: by outflank-mailman (input) for mailman id 188458;
 Thu, 16 Sep 2021 13:07:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Js+i=OG=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQr6V-0005NG-GK
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 13:07:03 +0000
Received: from mail-ed1-x52f.google.com (unknown [2a00:1450:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ab720410-eff5-440e-836c-510d35c83a5e;
 Thu, 16 Sep 2021 13:07:01 +0000 (UTC)
Received: by mail-ed1-x52f.google.com with SMTP id h17so16585621edj.6
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 06:07:01 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id u11sm1363059edo.65.2021.09.16.06.06.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 06:06:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab720410-eff5-440e-836c-510d35c83a5e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=1uhc8rR7lqyoe4rea7HKEmyIL/RfLEv22UTvPB6Y8VA=;
        b=SLgatQPrhcWyfL8lHASEpXJcNhRxuQTo7/VKZCMt1UYvBvseu4IO+5hYlpYZx0tDEl
         k/PgbXbHsD8LXJL9n4IKBNEIA/hHh4CGoampDyYorEraTbFAw3hdj5G7EKFA2liJMDf1
         7Q15hrSqKWUTyUO/ubo45ILD58vldfEFbX7QHEWGDLThQJrD4mT3Xp6YIH/TC3YCG/MB
         aWbITwu7CffDct+y6H75pGo5gvy/I8oytAM650rNx6wWizfvC+lyMveY8LVBDdj5EI/B
         h0RFUrU67FzCQIDYwnxN1qn6uWHruvwHbXnjIS6M3OjiZtpG9jy2fEOYHt88cqobtTV9
         hWFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=1uhc8rR7lqyoe4rea7HKEmyIL/RfLEv22UTvPB6Y8VA=;
        b=Vo1PTLFJNehPNwn4Ze+c2nYW3Y5D9KQXoLETqGCl8lGj8g9J/2dth3T2m+FJWHD9qn
         SyCd0zX6s7efgf4WOLvCSq90XJrT2hFvMPt/1i2kvLKU8V8pIb1hs+VMRFNRACx836nO
         SSS2jHv6c89SiaWU7YSeYAJu3JldQuH8OjRLXs1SSLVMBGBBa2/JQYi5IqJGRNjjd/Vp
         1psnGDObJQY2vr+/1nYNF3/TPbgnwphYrveaTDdmk8o3s4JBTTXTDB4TpPZgIHnVBdXq
         MT61IDNPsMeKa+zh+z2h1i9OQRVtl0lw7OP3Z4k26TY9QTrAwxL5u6IRdZZNq9tggamI
         Lsgw==
X-Gm-Message-State: AOAM530IGbuGoFVfyhdBuupw+EoHVcuxHZzmEcsUi9okRGHSxNbL+kyC
	1CJG2jceHlb5BI5YxrthfGGH8RE98/tjl9D7
X-Google-Smtp-Source: ABdhPJxbcn/9cMa5b76RPbPQ6iFb84YoyuR9DGNQkebXVFHklrZKHCXbvC4+z65RMXDLKGUHtdEYFA==
X-Received: by 2002:a05:6402:697:: with SMTP id f23mr6202839edy.253.1631797619996;
        Thu, 16 Sep 2021 06:06:59 -0700 (PDT)
Subject: Re: [RFC PATCH V2] xen/gnttab: Store frame GFN in struct page_info on
 Arm
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <1631652245-30746-1-git-send-email-olekstysh@gmail.com>
 <b6744333-4d43-ef24-0f9b-b5cd54680660@suse.com>
 <86cdc577-7085-48cd-c417-85b20afc9bf1@gmail.com>
 <647bccf5-1d1e-1f21-fb7c-c338bc5119b6@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <95071891-7b04-5dd7-2f39-9c7c4cefec43@gmail.com>
Date: Thu, 16 Sep 2021 16:06:58 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <647bccf5-1d1e-1f21-fb7c-c338bc5119b6@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 16.09.21 09:38, Jan Beulich wrote:

Hi Jan

> On 16.09.2021 00:13, Oleksandr wrote:
>> On 15.09.21 13:06, Jan Beulich wrote:
>>> On 14.09.2021 22:44, Oleksandr Tyshchenko wrote:
>>>> --- a/xen/include/asm-arm/mm.h
>>>> +++ b/xen/include/asm-arm/mm.h
>>>> @@ -98,9 +98,18 @@ struct page_info
>>>>    #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
>>>>    #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
>>>>    
>>>> - /* Count of uses of this frame as its current type. */
>>>> -#define PGT_count_width   PG_shift(2)
>>>> -#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
>>>> + /* 3-bit count of uses of this frame as its current type. */
>>>> +#define PGT_count_base    PG_shift(4)
>>>> +#define PGT_count_mask    PG_mask(7, 4)
>>>> +
>>>> +/*
>>>> + * Stored in bits [27:0] or [59:0] GFN if page is used for grant table frame.
>>> I don't know enough Arm details to tell whether this is properly
>>> one bit more than the maximum number of physical address bits.
>>> Without the extra bit you wouldn't be able to tell apart a
>>> guest specified GFN matching the value of PGT_INVALID_FRAME_GFN
>>> from an entry which was set from INVALID_GFN.
>> Really good point.
>>
>> 1. On Arm64 the p2m_ipa_bits could (theoretically) be 64-bit which, I
>> assume, corresponds to the maximum guest physical address (1 << 64) - 1
>> = 0xFFFFFFFFFFFFFFFF.
>> To store that GFN we need 52-bit. But, we provide 60-bit field which is
>> more than enough, I think. Practically, the maximum supported
>> p2m_ipa_bits is 48-bit, so the maximum supported GFN will occupy 36-bit
>> only. Everything is ok here.
>> 2. On Arm32 the p2m_ipa_bits is 40-bit which, I assume, corresponds to
>> the maximum guest physical address (1 << 40) - 1 = 0xFFFFFFFFFF. To
>> store that GFN we need 28-bit. If I did the calculation correctly, what
>> we have on Arm32 is that PGT_INVALID_FRAME_GFN == maximum guest physical
>> address and it looks like we need and extra bit on Arm32. Do you think
>> we need to borrow one more bit from the count portion to stay on the
>> safe side?
> I think so, unless there are further restrictions on the GFN range
> that I'm unaware of.

ok, thank you.


>
> For 64-bit, if you only need 52 bits, why do you make the field 60
> bits wide? I'd recommend against "wasting" bits. Better keep the
> count field as wide as possible.
Well, the reason almost the same as I already provided for why not using 
PG_mask for PGT_gfn_mask.
Although we waste some bits on Arm64, having the same amount of bits for 
count on Arm32 and Arm64 let us avoid introducing an extra #ifdef to 
that header
(if we go with maximum possible bits for count on each configuration we 
would need to have a set of PGT_count_*/PGT_gfn_*).
I was thinking that if there was indeed a lack of bits for count then an 
extra #ifdef wouldn't be an argument at all. If I am not mistaken, the 
code which deals with count (where only 1 bit is used) is one and the 
same on both Arm32 and Arm64, so what is the point of diverging here (I 
mean to provide more bits for count on Arm64 because simply there is a 
reserve).
But, if the reason I provided is weak I will update header to keep the 
count as wide as possible.


>
>>>> + * This only valid for the xenheap pages.
>>>> + */
>>>> +#define PGT_gfn_width     PG_shift(4)
>>>> +#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
>>> Any reason you don't use PG_mask() here? Any open-coding is prone
>>> to people later making mistakes.
>> I failed to come up with idea how to do that without #ifdef. As GFN
>> starts at bit 0 different first parameter would be needed for PG_mask on
>> 32-bit and 64-bit systems.
>> I wonder whether PGC_count_mask/PGT_count_mask are open-coded on Arm/x86
>> because of the same reason.
> Hmm, that pre-existing open-coding isn't nice, but is perhaps a
> good enough reason to keep what you have. (Personally I wouldn't
> be afraid of adding an #ifdef here, but I realize views there may
> differ.)
>
> Jan
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 13:14:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 13:14:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188468.337625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQrDV-0006pF-Lk; Thu, 16 Sep 2021 13:14:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188468.337625; Thu, 16 Sep 2021 13:14:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQrDV-0006p8-Iq; Thu, 16 Sep 2021 13:14:17 +0000
Received: by outflank-mailman (input) for mailman id 188468;
 Thu, 16 Sep 2021 13:14:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6X0d=OG=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mQrDT-0006p2-Se
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 13:14:16 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16a9b1af-94d8-4b65-b3f9-db6ebdd2991c;
 Thu, 16 Sep 2021 13:14:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16a9b1af-94d8-4b65-b3f9-db6ebdd2991c
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631798054;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=SOPXitM913DIQoRdH+dva/QNVVqNRaWpprWx7cSzCUg=;
  b=ibG9doECk3/TuY57HYUgh07XkjWJvirMslny+D4+yh/H3nEOb8dFsYyV
   IHNDa0t+vuIGYLfmWhmITD5Od8+WbcbDwvRcoV5Ab6HEW8DLdCAdhM8f6
   DeacB+8HQT5ok3QWeic1dfvBGcc+OyPH2HvwsYmFwnvhzQ6YHQDLtpUUk
   w=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: oW5rQi6Uoj8yvAyG/Z05o7vFxqbxhBDcciWdfMmU5vVLKy7CUM5To9io+pGAoCFHY7BDnF6LWS
 8jnjpZrDbXMrwvcgqu1OaBLsRbncO8aYwyjF61xwoqMHG9+9koKGIx1Ni/Cu3Ccy+7LyZtmIlK
 AJped7FAIK2zEuHnbmxiZrzdrMvD+n2oYhygZWY1UNzwxyBeNb8QTLjRJi0ldOcAdqutIErWgK
 K/vS0awPdwdLo1YrChnuoi3MDW2KtCd46JyPdROvQ4NutU2Da9CKtFJsiJQDJQjTc5RdTw5V1Q
 M6/h/NmaUjsa3i8zmkaPvvpN
X-SBRS: 5.1
X-MesageID: 52876676
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:wlqu0aC7Ce/7BhVW/7/kw5YqxClBgxIJ4kV8jS/XYbTApD53gmFWm
 GcWXD3TOf2PZmL2fN0kYYm3oU1X7MSGz4RmQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0Ew7wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/2naVoMt0j
 +d0tt+gFQIMZIzgtPQhTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHv+auoEGg2lYasZmLMjee
 c9ITwRTKyvLPU0UP0YeMo40tbL97pX4W2IB8w/EzUYt2EDI1xB42rXpNNvTe/SJSN9Tk0Leo
 XjJl0zmBjkKOdrZziCKmlq8i+mKkS7lVYY6ELyj6uUskFCV3nYUChAdSR28u/bRokyjX9NSL
 WQE9yxoqrI9nGSqVdD8XBy+5XKBvwIGSvJBGus25RHL0Le8ywSTC3UATzVBQMc7r8JwTjsvv
 neIgNHBFTFpqKeSS3+W6vGTtzzaBMQOBTZcP2leF1JDuoS95tFo5v7Scjp9OP/2kuLuHHbC+
 jPJvHQ7ou8yoeIb67ruqDgrnAmQSoj1oh8dv1uMBz/0sVwnPeZJdKTztgOKtq8owJKxCwDb5
 SlawZD2APUmUMnV/BFhVtnhC11ACxytCzTanVcnNJ0o7T3FF5WLLN0IvW0WyKuENK85ld7Vj
 K374lg5CHx7ZiLCgUpLj2SZUJxCIU/IT4+NaxwsRoASCqWdjSfelM2UWaJ144wLuBN0+U3YE
 czAGftA8F5AUfg3pNZIb75FjNfHORzSNUuMHMumnnxLIJK1ZWKPSKdtDbd9Rrlit8u5TPHu2
 48HbaOikkwHOMWnO3W/2dNDfDgicClgbbir+pM/SwJ2Clc/cI3XI6SKmu1Jlk0Mt/k9q9okC
 VnmBhYDmQag3CaeQehIA1g6AI7SsV9EhStTFQQnPEqy2mhlZoCq7awFcIAwc6Vh/+tmpcOYh
 dFbIJvo7i1nRmuV9jIDQ4P6qYA+Jh2niRjXZ3iuYSQlfo4mTAvMo4e2cgzq/SgILyy2qcph/
 OHwilKFGcIOF1Z4EcLbSPOz1Fft73ITr/1/AhnTKd5JdUSyrIUzc37tjuU6Kt0nIAnYwmfIz
 B6fBBoV/LGfo4I8/NTTq7qDqoOlT7l3EkZARjGJ5reqLyjKuGGkxNYYAuqPeDncUkLy+bmjO
 roJn62tbqVfkQ8T4YRmErttwaYv3PfVpudXnlZ+AXHGT1W3Ebc8cHOI6tZC6/9WzbhDtAroB
 k/WootGOa+EMd/OGUIKIFZ3dfyK0PwZl2WA7fkxJ0mmtiZ78KDeDBdXNhiIzidcMKF0IMUux
 uJ44Jwa7Am2ixwLNNeaj38LqzTQfyJYC6h35IsHBILLixYwzgAQaJPRPSb6/ZWTZogeKUItO
 DKV2PLPirk0KpAuqJbv+awhBdZguKk=
IronPort-HdrOrdr: A9a23:diBGiKD8UebtnerlHeg3sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHP9OkPIs1NKZMjUO11HYTr2KgbGSpgEIXheOi9K1tp
 0QDZSWaueAdGSS5PySiGLTc6dCsai6GeKT9J/jJh9WPH5XgspbnmFE42igYylLrF4sP+tEKH
 PQ3LsMmxOQPVAsKuirDHgMWObO4/XNiZLdeBYDQzoq8hOHgz+E4KPzV0Hw5GZTbxp/hZMZtU
 TVmQ3w4auu99m91x/nzmfWq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczOgBkF5MWUrHo6mt
 jFpBkte+x19nPqZ2mw5SDg3gHxuQxeokPK+Bu9uz/OsMb5TDU1B45qnoRCaCbU7EImoZVVzL
 9L93jxjesXMTrw2ADGo/TYXRBjkUS55VA4l/QIsnBZWYwCLJdMsI0k+l9PGptoJlO11GkeKp
 gsMCjg3ocVTbvDBEqp+1WHgebcEUjbJy32AnTr4aeuonxrdHMQ9Tpv+CVQpAZGyHqRI6M0kt
 gsCZ4Y542mePVmGp6VNN1xMfdfNVa9My4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,298,1624334400"; 
   d="scan'208";a="52876676"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=moSY1Dr4eGwyAHCgh6vfaRsX/x0mBD5S2K4HD4U0esPZotWhNopalpb1IKGMx09HSqClDBoZGVyNBU/U5FX16tpEQlhBDHDn7iJLbEt7xhX78HpaH7ajnrFMzvkwYCybLoc72HjoJ1bBT9Yp7kDawThvvKlVg8vpwXD+KVAuIrtHPCOdu+detXtU8Zuk0h2A7inZKgE6aJBjrHbFhwNnkgRmaEKnx3vmY3TyTdqKx7kzLtfnyebQgtSX53N8232hhb4yHOs1ZYPk+RuKEZz80ih6u4mdLKXyLici5kYtiQp4uNUZA8Xec9aUKk4o4+GbbJKD9fsGXbvAlYMZFBiNsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zKZBz67gWrmfGh5fqXsQOaiN41yOf90IY5B3x2Hnsos=;
 b=LBkRAmcSeOFphr324EuqoDhPlMN4ReZAGm4nu8PW5Etpm0PPITDyAKzKQo0y9SqBK/WpFFdn3BEQ4zlzwuW7itwwzWBKZmtVbP27r7OhaEeHDzviWlPPpGjNGNbFgE4r9y83cxL72O6S0mxRuyJxQb6NRVmWnvjtl0JcGLfnjbqTU4D8a2I5USS0XdXiiHYOKZG1I/mgNS9fb6spjImF4j4oei3Yv5Rx7JVxrBTaAMlPcJlXgGXIjV5bIbVbnWJQOEEIoCdPjV8ZX9xI7TmLfaiLyEcXGBk2J4mNSIo6KgQDuhlrZEMO1egcalqA3OMuKRRoM31RdlSHIQAuxu9dOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zKZBz67gWrmfGh5fqXsQOaiN41yOf90IY5B3x2Hnsos=;
 b=bn3iOYfRBS5ZEtSxoircAruWTFKtE+ln96ySgF8gRkYEiHodxZU2lKumaQA/1+XynsBcJaeTp8dBj953/b/t9G/soVCmCZGvUOgLJNwGIU3shB6MXY+ma6dr/O7ek3rP7faaqhiqB20sZCDlJjDSM+J3tAXQaVHKZf/ofuLQIj0=
Date: Thu, 16 Sep 2021 15:08:03 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
Message-ID: <YUNBswZ3rAFeDa6I@MacBook-Air-de-Roger.local>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
X-ClientProxiedBy: LO2P265CA0176.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3fecc8be-3fcb-42ff-86c1-08d97913071b
X-MS-TrafficTypeDiagnostic: DM6PR03MB5289:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB528930F7E169D914C690718C8FDC9@DM6PR03MB5289.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dMQyYjNy+uY3SEU6kiZQpmWCjJLn2diAVX19/pv/sSwmL9EI6Mk3iN6kulKBvA6tFKmaTrxBikiGz+xX47IQ2gGODYnEZmG1iB7s3HykPgTfjF2k0tclJuUDNBSvSNIzraaZc1JhMpbES1Ili46dmg5zsRcF/i9fgiA3sAyjeae0+0MSqwuXvHctPtYgbAiJBYodr1I0PLT+x4PtNogJwn/cRqyD8YhDiZpZ2vRY6wI9PIeCIaPCexzB6nXe3O6XF9I0TsBXgb0Pju/Ooo9gSeiwp7+nCpG2wjD8W4gfQI2aY3I+Mt1I4siOeXZ36/1axaKX/m5HFpQblssz4+5o/ZbRXFamvRzNA/PnAbbLaPgU9Vuh/lFSWeOVuLAneUCahV/ylIi0V5GI4M1R0SEOlZ3d7HtilswS23tkQ4IA97yL45Xq6Jb69p9qba/9Cmbga/XlIF0ZaQy2CEn7xO/AZa52t0wtI+qxAzPCE6ZJAxZkmm2KVg+WLe64iu6lyC4+86kU6R3ZQR0Ii35FjcM4MCJu1juMZJIPejMIlbah2YodjbK62HERUDywEbZxbBfHb5H3xP42E7LTpdtibJPFHWj8aMwOvVwVwFknGBxZIoEKZzHPlff5p1loLGD5pD3yWCOaLnN07Q5fcRE427Jq0ASW5XM06nYFNrYKMaq9SGNcWUHU+fu6EfkPaRjDB06Z7riOQbhDgswsPQBWzyivcA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(396003)(346002)(376002)(136003)(39860400002)(6496006)(66476007)(66556008)(85182001)(38100700002)(53546011)(54906003)(4326008)(6486002)(66946007)(2906002)(83380400001)(316002)(8936002)(956004)(86362001)(26005)(5660300002)(9686003)(6666004)(8676002)(186003)(6916009)(478600001)(23180200003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SmlJMDZZZ0lJMkZYV0FxaitCZjV1dVF4N29JUVpXNzFiaHJnSC82WXdKVTIr?=
 =?utf-8?B?NU11UHE0RTNZMFlyOWFkNGVFaXRvYmF1aFIvNVhDQnRqaDMyN01MUE9YbTJi?=
 =?utf-8?B?SGVhL2ZXSk9NaXlRaFRSSTE5MjloK2ZFcG9ROEhIbHkzcWtiWXA4L0V0Qmh0?=
 =?utf-8?B?SEUvb0pNandLR1MzeFhiQmdsMnVZaHB0ekdxMXRTUWFaUjdNRysrRlhUY010?=
 =?utf-8?B?K3VORS9JZUVBUmFCTnViYWdpaE9GRHpSdjJJanZXajUzekV6YjI3RDQxeEJM?=
 =?utf-8?B?QVgrTTAwRVllQTNZdWkzendzeWdRNWRtUTV6MFZrZ245MEloelByQXk0enM2?=
 =?utf-8?B?ek41QmRVNHF1Z1A5Z2FxRmdSbnJqU01JSEJDT3E5NURxUnVvbjhGWk82K3gv?=
 =?utf-8?B?d0x3cjZKMXNjS21vOVRFM09lWXdsSml0TS9pSi9SSTlEY2ZCSTR1UTV6N3RL?=
 =?utf-8?B?UjZIcHNPQmxYQjlWRU1LbjRwYXJzYmVBK21TNGUvbXBCcVlpRWw5L2loMVRV?=
 =?utf-8?B?V3hxYXlMbjRsemQyaGdLT21iV2hvbTZMR01oZVdMbm8wM2lmbVZkVWNEM0Z0?=
 =?utf-8?B?NHNkZDZ1VzlScUdWQ3dtT0R3L0h1MUluMUxqOUhTenlDdU15a3JDZzdZam9l?=
 =?utf-8?B?ZU4vSnEvUEtsVjZJVmZBd3RCdXc0eG9ITDFtQVBBcm5aeEE2WFRXencvNjlE?=
 =?utf-8?B?cEM5MGpyWmJkaDl0VDVKT2JkQlZ6dlprZ3BHS3pvV0t4R0w2M1k2QzI4R3pS?=
 =?utf-8?B?OWpNZzUzQ3FPV1dUamRCVjBFZ3gyMUI4aW9CWkx0TzkvSlpnemh5YlRSL3Vr?=
 =?utf-8?B?WSswbDNhRitiUm95b3pOY3U1eEZQZEthNVB4V2FCa0dWcDBRWEVVejM4QnBp?=
 =?utf-8?B?UGNtK0llMlR5QytERC9hdE4ydHlBRS9kUGZnSVVqTlhQRitOUURLb1UrK1Vz?=
 =?utf-8?B?UERhSUw5dkVaRCtUU2NvcnNpSzZKRlRYZlh4bER4bWdYYmVSRWNSM3ZPblgr?=
 =?utf-8?B?MzRESzY2WkxUY2hjYzhFYmdIQjFvS1VxazZCR1RmQXNQYVpmcXk0c0ErUVpo?=
 =?utf-8?B?V3dMelZUSXlYOC8rNGplQmV3U09mSHBmZEtMUHhoRGQxS3VyRUpKaUdOWjQz?=
 =?utf-8?B?UnNDSHVoc29YODJqbTNVTWJ4QzVqYkpCR0dTSVp2UTVZalMvZHM1SEQrWWFQ?=
 =?utf-8?B?cU9YK1VQblRvNU5pVEQyQkFvZ2hwaXhYenJNNlVqNTZMZDFRSERUcTRGbzcx?=
 =?utf-8?B?NmNWV3NQUGxoYlRrU21Bc1Z4S1R0Z3hSVWlqWHRDU2UwR1dQUjhpQnpwdk9E?=
 =?utf-8?B?K3dReWlOUmdMbkVwWEJnTjVZYU5yN0Q4MTllMjI3NUJReElaRDkycSt4SkdB?=
 =?utf-8?B?NWFReVdNempCOVIzSFRmdDRWSFlkaEI0TkV5Mkluek1LZUpOYzJCWlg5MS84?=
 =?utf-8?B?TW5ROTR1YldoMmY5aE1nUEQwOTh3VENQMFFGWDhoZkJvK3Z1WWVLRUlvaU1R?=
 =?utf-8?B?bnFmaW03YktLTmsrYmZYamlZQ0Y1ZDU2a0RRanRWYUp1dTRlbThOdCsyZktM?=
 =?utf-8?B?WTErS2lxcW82T1hxc2crdnBoWVQ1N1JzTURTTW83R0N5d2lValJqaDAyNDR6?=
 =?utf-8?B?NWlhcUxWdHVGZnlVTVFUSUhlNXM2TkgzaFNTenFOR1BFYWU3TUpBeXJuZ1BR?=
 =?utf-8?B?Zmh4eDViL2NYT2NIUVFHWXh1LzhMWUhibXhua0ZUU2ttMWZ1Y1hPdEhyOU1z?=
 =?utf-8?Q?wKPOPV9u5xPNefxJ1ITg4ehACtyxL5DS6JyKL6w?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3fecc8be-3fcb-42ff-86c1-08d97913071b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 13:08:08.2370
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7gJSgvcInAPwws0oYCJ+al1VIw5sz5OlLGaCh4m8Ny61bl6GfagTBhgdshexZP/VSbq8Ewvjyk53QOTqlSXvpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5289
X-OriginatorOrg: citrix.com

On Thu, Sep 16, 2021 at 02:30:39PM +0200, Jan Beulich wrote:
> On 16.09.2021 13:10, Dmitry Isaikin wrote:
> > From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
> > 
> > This significantly speeds up concurrent destruction of multiple domains on x86.
> 
> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
> improve locking during domain destruction"). There it was found to
> actually improve things; now you're claiming the opposite. It'll
> take more justification, clearly identifying that you actually
> revert an earlier change, and an explanation why then you don't
> revert that change altogether. You will want to specifically also
> consider the cleaning up of huge VMs, where use of the (global)
> domctl lock may hamper progress of other (parallel) operations on
> the system.
> 
> > I identify the place taking the most time:
> > 
> >     do_domctl(case XEN_DOMCTL_destroydomain)
> >       -> domain_kill()
> >            -> domain_relinquish_resources()
> >                 -> relinquish_memory(d, &d->page_list, PGT_l4_page_table)
> > 
> > My reference setup: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Xen 4.14.
> > 
> > I use this command for test:
> > 
> >     for i in $(seq 1 5) ; do xl destroy test-vm-${i} & done
> > 
> > Without holding the lock all calls of `relinquish_memory(d, &d->page_list, PGT_l4_page_table)`
> > took on my setup (for HVM with 2GB of memory) about 3 seconds for each destroying domain.
> > 
> > With holding the lock it took only 100 ms.
> 
> I'm further afraid I can't make the connection. Do you have an
> explanation for why there would be such a massive difference?
> What would prevent progress of relinquish_memory() with the
> domctl lock not held?

I would recommend to Dmitry to use lock profiling with and without
this change applied and try to spot which lock is causing the
contention as a starting point. That should be fairly easy and could
share some light.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 13:56:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 13:56:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188479.337636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQrsL-00037r-1x; Thu, 16 Sep 2021 13:56:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188479.337636; Thu, 16 Sep 2021 13:56:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQrsK-00037k-Ut; Thu, 16 Sep 2021 13:56:28 +0000
Received: by outflank-mailman (input) for mailman id 188479;
 Thu, 16 Sep 2021 13:56:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AVhn=OG=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQrsJ-00037e-Ho
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 13:56:27 +0000
Received: from mail-pf1-x42e.google.com (unknown [2607:f8b0:4864:20::42e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d752004f-0c63-46fd-ab43-69efa82726e5;
 Thu, 16 Sep 2021 13:56:26 +0000 (UTC)
Received: by mail-pf1-x42e.google.com with SMTP id y8so5938957pfa.7
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 06:56:26 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:18:efec::50b])
 by smtp.gmail.com with ESMTPSA id
 w11sm3376724pfj.65.2021.09.16.06.56.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 06:56:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d752004f-0c63-46fd-ab43-69efa82726e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=VQdB0cmtRyC2OUQYF1nmKrbKRoOhkmjS2WdWxiwvfV0=;
        b=CIfTuZFoNILRqkZ8DcW/GNbMyGgDnxBr4dVf4uWbZjU8pjj4lrHMRtQVPwcdxklAuQ
         JZziDYokWmjzVIErChST09t12g0u5Q0uRpqfgz1HLLBW1rjGMPZLz0kZ+aU/1VtdGCBf
         ScvmIZdC/KAH9/s5zwFCH/t+wJJgREk4jkxOEav0yNRvD2PeR9Mt0gXD43CEjS0rWa6X
         1ojan6I2m/RB70FhK0zBN0wx5wwjfaYXFG9iPhqmYChnfbWqL1/mqLAFucMtI4L4kMUT
         mDpT/XbuXdWRn1Ki+KD46f09IzCVvENPhyPuOqItQNeOtC2mXKgNu7d+59tp7LLGPGIj
         mQdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=VQdB0cmtRyC2OUQYF1nmKrbKRoOhkmjS2WdWxiwvfV0=;
        b=v6YvDFDXEphxdoGQPlXpzH+nTxdcmv551KDgBNPJEMqYPwQWhNRryMTz73UKdtmRJN
         jf4OGm9RlW2oD+oZxP6ewnwKmgsv45GDkVI0Ue0DfUcvymy+TVElyJ6j+rW7jpc1jbHg
         WeGELFyN5hOuaIEAOSLNO3G9FICxcgX7EkfCuZjInQ9n+IMuJ0F8CBcrl6o5QZU1dKYb
         EhuS9qGFK9czRGpskN2tLfdcQJ64zypkpBgG++56QWo6u0vhvu7Jp9zgVpRLokhbBbm8
         cYNVOPewV/UAJInqKypQ7Vs3p5RUNEmGZFZoMv/7k64zTv1G2kgJhlarBihPe6ofQfeV
         XOYg==
X-Gm-Message-State: AOAM530fE+iq+j/5wLS6wp3jln3G1BbDKUOOVICESdrXLNJartVlFVK1
	EEKd7v+BuTettBOpNRJGuug=
X-Google-Smtp-Source: ABdhPJwil566vQyG+FPIeqOpCAnK+VdYspH+w8JoanT8EwRLxY95XCPpJH9ls0yqVjdetF6RlrGZEg==
X-Received: by 2002:a65:44c4:: with SMTP id g4mr5187939pgs.254.1631800585774;
        Thu, 16 Sep 2021 06:56:25 -0700 (PDT)
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
To: Haiyang Zhang <haiyangz@microsoft.com>,
 Michael Kelley <mikelley@microsoft.com>, KY Srinivasan <kys@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <MN2PR21MB12959F10240EC1BB2270B345CADB9@MN2PR21MB1295.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <31da994f-a032-dd40-a90e-bfdaf313d45d@gmail.com>
Date: Thu, 16 Sep 2021 21:56:11 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MN2PR21MB12959F10240EC1BB2270B345CADB9@MN2PR21MB1295.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/16/2021 12:46 AM, Haiyang Zhang wrote:
>>> +				memset(vmap_pages, 0,
>>> +				       sizeof(*vmap_pages) * vmap_page_index);
>>> +				vmap_page_index = 0;
>>> +
>>> +				for (j = 0; j < i; j++)
>>> +					__free_pages(pages[j], alloc_unit);
>>> +
>>> +				kfree(pages);
>>> +				alloc_unit = 1;
>> This is the case where a large enough contiguous physical memory chunk
>> could not be found.  But rather than dropping all the way down to single
>> pages, would it make sense to try something smaller, but not 1?  For
>> example, cut the alloc_unit in half and try again.  But I'm not sure of
>> all the implications.
> I had the same question. But probably gradually decrementing uses too much
> time?
> 

This version is to propose the solution. We may optimize this to try
smaller size until to single page if this is right direction.




From xen-devel-bounces@lists.xenproject.org Thu Sep 16 14:10:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 14:10:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188487.337648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQs5O-00052B-8B; Thu, 16 Sep 2021 14:09:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188487.337648; Thu, 16 Sep 2021 14:09:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQs5O-000524-5A; Thu, 16 Sep 2021 14:09:58 +0000
Received: by outflank-mailman (input) for mailman id 188487;
 Thu, 16 Sep 2021 14:09:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQs5N-00051u-8m; Thu, 16 Sep 2021 14:09:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQs5M-0007nE-TB; Thu, 16 Sep 2021 14:09:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQs5M-000497-M9; Thu, 16 Sep 2021 14:09:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQs5M-0005Tc-JS; Thu, 16 Sep 2021 14:09:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8B9T8FJWSuhRTGwDTeltoayPil1DrB1tMY3uhpAl2S0=; b=v36mHqbDwOl+KNInRMqca9OnBF
	AlbeQNtCG8vC4QOa6CnK/9LW8Lw0M+D+CHPQTQnvDgKvxhLm207x7ms6QdyHhDkx0xSWHXyiN2VqG
	wG1XXk3z1BriHrR+rKNaDfTvlbbXEI786e4PWvBUiL54XHHC4u634vKQZG1SQqhBC+dU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165009-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165009: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9d92140d0255e14d8c3eaba7edc931327eb96fd0
X-Osstest-Versions-That:
    xen=3e910b648b99393561e7c523756c1ea49a6c1305
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 14:09:56 +0000

flight 165009 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165009/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9d92140d0255e14d8c3eaba7edc931327eb96fd0
baseline version:
 xen                  3e910b648b99393561e7c523756c1ea49a6c1305

Last test of basis   164998  2021-09-15 14:00:28 Z    1 days
Testing same since   165009  2021-09-16 10:01:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   3e910b648b..9d92140d02  9d92140d0255e14d8c3eaba7edc931327eb96fd0 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 14:42:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 14:42:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188497.337662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsab-0000yH-Pj; Thu, 16 Sep 2021 14:42:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188497.337662; Thu, 16 Sep 2021 14:42:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsab-0000yA-Ln; Thu, 16 Sep 2021 14:42:13 +0000
Received: by outflank-mailman (input) for mailman id 188497;
 Thu, 16 Sep 2021 14:42:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=20iF=OG=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mQsaa-0000y4-Co
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 14:42:12 +0000
Received: from forward501p.mail.yandex.net (unknown [77.88.28.111])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1b94f661-eeca-4f3e-b033-1e681013ad73;
 Thu, 16 Sep 2021 14:42:08 +0000 (UTC)
Received: from vla5-5db4dee6e8c2.qloud-c.yandex.net
 (vla5-5db4dee6e8c2.qloud-c.yandex.net
 [IPv6:2a02:6b8:c18:3482:0:640:5db4:dee6])
 by forward501p.mail.yandex.net (Yandex) with ESMTP id 7EA6562127A6;
 Thu, 16 Sep 2021 17:42:06 +0300 (MSK)
Received: from 2a02:6b8:c18:3607:0:640:4452:c58d
 (2a02:6b8:c18:3607:0:640:4452:c58d [2a02:6b8:c18:3607:0:640:4452:c58d])
 by vla5-5db4dee6e8c2.qloud-c.yandex.net (mxback/Yandex) with HTTP id
 4gZTQQ2E6Os1-g4E02cf5; Thu, 16 Sep 2021 17:42:05 +0300
Received: by vla5-4452c58d5c14.qloud-c.yandex.net with HTTP;
 Thu, 16 Sep 2021 17:42:04 +0300
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b94f661-eeca-4f3e-b033-1e681013ad73
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1631803325;
	bh=P7p9tucualVsAGee4lV6wL5I5LHJo8lvNYgnA4TEFAo=;
	h=References:Date:Message-Id:Cc:Subject:In-Reply-To:To:From;
	b=XWZNzVVNq4Ew2DF51gXNXcuDzm1poarpZuTlm+J2rtXY6YSoICrkRE+hthMqIJDOJ
	 yBRLIPdFbPJg6nFWqE4lo61rEm+lpKsM00bJtdVa7u9qUM9e6fmVkOqURxxGrh0GVs
	 OKPkgpGxVusEZa2GiLubkbyOoDomU28NbcYRyb9U=
Authentication-Results: vla5-5db4dee6e8c2.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
To: =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
In-Reply-To: <YUNBswZ3rAFeDa6I@MacBook-Air-de-Roger.local>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
	 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com> <YUNBswZ3rAFeDa6I@MacBook-Air-de-Roger.local>
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
MIME-Version: 1.0
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Thu, 16 Sep 2021 17:42:04 +0300
Message-Id: <1855101631803324@vla5-4452c58d5c14.qloud-c.yandex.net>
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=utf-8

PGJyIC8+PGJyIC8+MTY6MTQsIDE2INGB0LXQvdGC0Y/QsdGA0Y8gMjAyMSDQsy4sICJSb2dlciBQ
YXUgTW9ubsOpIiAmbHQ7cm9nZXIucGF1QGNpdHJpeC5jb20mZ3Q7OjxiciAvPjxibG9ja3F1b3Rl
IGNsYXNzPSIyMTBlN2E4NDhlOGZjYjQ1d21pLXF1b3RlIj48cD5PbiBUaHUsIFNlcCAxNiwgMjAy
MSBhdCAwMjozMDozOVBNICswMjAwLCBKYW4gQmV1bGljaCB3cm90ZTo8YnIgLz48L3A+PGJsb2Nr
cXVvdGUgY2xhc3M9IjIxMGU3YTg0OGU4ZmNiNDV3bWktcXVvdGUiPsKgT24gMTYuMDkuMjAyMSAx
MzoxMCwgRG1pdHJ5IElzYWlraW4gd3JvdGU6PGJyIC8+wqAmZ3Q7IEZyb206IERtaXRyeSBJc2F5
a2luICZsdDs8YSBocmVmPSJtYWlsdG86aXNhaWtpbi1kbWl0cnlAeWFuZGV4LnJ1Ij5pc2Fpa2lu
LWRtaXRyeUB5YW5kZXgucnU8L2E+Jmd0OzxiciAvPsKgJmd0OyA8YnIgLz7CoCZndDsgVGhpcyBz
aWduaWZpY2FudGx5IHNwZWVkcyB1cCBjb25jdXJyZW50IGRlc3RydWN0aW9uIG9mIG11bHRpcGxl
IGRvbWFpbnMgb24geDg2LjxiciAvPsKgPGJyIC8+wqBUaGlzIGVmZmVjdGl2ZWx5IGlzIGEgc2lt
cGxpc3RpYyByZXZlcnQgb2YgMjI4YWI5OTkyZmZiICgiZG9tY3RsOjxiciAvPsKgaW1wcm92ZSBs
b2NraW5nIGR1cmluZyBkb21haW4gZGVzdHJ1Y3Rpb24iKS4gVGhlcmUgaXQgd2FzIGZvdW5kIHRv
PGJyIC8+wqBhY3R1YWxseSBpbXByb3ZlIHRoaW5nczsgbm93IHlvdSdyZSBjbGFpbWluZyB0aGUg
b3Bwb3NpdGUuIEl0J2xsPGJyIC8+wqB0YWtlIG1vcmUganVzdGlmaWNhdGlvbiwgY2xlYXJseSBp
ZGVudGlmeWluZyB0aGF0IHlvdSBhY3R1YWxseTxiciAvPsKgcmV2ZXJ0IGFuIGVhcmxpZXIgY2hh
bmdlLCBhbmQgYW4gZXhwbGFuYXRpb24gd2h5IHRoZW4geW91IGRvbid0PGJyIC8+wqByZXZlcnQg
dGhhdCBjaGFuZ2UgYWx0b2dldGhlci4gWW91IHdpbGwgd2FudCB0byBzcGVjaWZpY2FsbHkgYWxz
bzxiciAvPsKgY29uc2lkZXIgdGhlIGNsZWFuaW5nIHVwIG9mIGh1Z2UgVk1zLCB3aGVyZSB1c2Ug
b2YgdGhlIChnbG9iYWwpPGJyIC8+wqBkb21jdGwgbG9jayBtYXkgaGFtcGVyIHByb2dyZXNzIG9m
IG90aGVyIChwYXJhbGxlbCkgb3BlcmF0aW9ucyBvbjxiciAvPsKgdGhlIHN5c3RlbS48YnIgLz7C
oDxiciAvPsKgJmd0OyBJIGlkZW50aWZ5IHRoZSBwbGFjZSB0YWtpbmcgdGhlIG1vc3QgdGltZTo8
YnIgLz7CoCZndDsgPGJyIC8+wqAmZ3Q7ICAgICBkb19kb21jdGwoY2FzZSBYRU5fRE9NQ1RMX2Rl
c3Ryb3lkb21haW4pPGJyIC8+wqAmZ3Q7ICAgICAgIC0mZ3Q7IGRvbWFpbl9raWxsKCk8YnIgLz7C
oCZndDsgICAgICAgICAgICAtJmd0OyBkb21haW5fcmVsaW5xdWlzaF9yZXNvdXJjZXMoKTxiciAv
PsKgJmd0OyAgICAgICAgICAgICAgICAgLSZndDsgcmVsaW5xdWlzaF9tZW1vcnkoZCwgJmFtcDtk
LSZndDtwYWdlX2xpc3QsIFBHVF9sNF9wYWdlX3RhYmxlKTxiciAvPsKgJmd0OyA8YnIgLz7CoCZn
dDsgTXkgcmVmZXJlbmNlIHNldHVwOiBJbnRlbChSKSBYZW9uKFIpIENQVSBFNS0yNjgwIHY0IEAg
Mi40MEdIeiwgWGVuIDQuMTQuPGJyIC8+wqAmZ3Q7IDxiciAvPsKgJmd0OyBJIHVzZSB0aGlzIGNv
bW1hbmQgZm9yIHRlc3Q6PGJyIC8+wqAmZ3Q7IDxiciAvPsKgJmd0OyAgICAgZm9yIGkgaW4gJChz
ZXEgMSA1KSA7IGRvIHhsIGRlc3Ryb3kgdGVzdC12bS0ke2l9ICZhbXA7IGRvbmU8YnIgLz7CoCZn
dDsgPGJyIC8+wqAmZ3Q7IFdpdGhvdXQgaG9sZGluZyB0aGUgbG9jayBhbGwgY2FsbHMgb2YgYHJl
bGlucXVpc2hfbWVtb3J5KGQsICZhbXA7ZC0mZ3Q7cGFnZV9saXN0LCBQR1RfbDRfcGFnZV90YWJs
ZSlgPGJyIC8+wqAmZ3Q7IHRvb2sgb24gbXkgc2V0dXAgKGZvciBIVk0gd2l0aCAyR0Igb2YgbWVt
b3J5KSBhYm91dCAzIHNlY29uZHMgZm9yIGVhY2ggZGVzdHJveWluZyBkb21haW4uPGJyIC8+wqAm
Z3Q7IDxiciAvPsKgJmd0OyBXaXRoIGhvbGRpbmcgdGhlIGxvY2sgaXQgdG9vayBvbmx5IDEwMCBt
cy48YnIgLz7CoDxiciAvPsKgSSdtIGZ1cnRoZXIgYWZyYWlkIEkgY2FuJ3QgbWFrZSB0aGUgY29u
bmVjdGlvbi4gRG8geW91IGhhdmUgYW48YnIgLz7CoGV4cGxhbmF0aW9uIGZvciB3aHkgdGhlcmUg
d291bGQgYmUgc3VjaCBhIG1hc3NpdmUgZGlmZmVyZW5jZT88YnIgLz7CoFdoYXQgd291bGQgcHJl
dmVudCBwcm9ncmVzcyBvZiByZWxpbnF1aXNoX21lbW9yeSgpIHdpdGggdGhlPGJyIC8+wqBkb21j
dGwgbG9jayBub3QgaGVsZD88YnIgLz48L2Jsb2NrcXVvdGU+PHA+PGJyIC8+SSB3b3VsZCByZWNv
bW1lbmQgdG8gRG1pdHJ5IHRvIHVzZSBsb2NrIHByb2ZpbGluZyB3aXRoIGFuZCB3aXRob3V0PGJy
IC8+dGhpcyBjaGFuZ2UgYXBwbGllZCBhbmQgdHJ5IHRvIHNwb3Qgd2hpY2ggbG9jayBpcyBjYXVz
aW5nIHRoZTxiciAvPmNvbnRlbnRpb24gYXMgYSBzdGFydGluZyBwb2ludC4gVGhhdCBzaG91bGQg
YmUgZmFpcmx5IGVhc3kgYW5kIGNvdWxkPGJyIC8+c2hhcmUgc29tZSBsaWdodC48YnIgLz48YnIg
Lz5SZWdhcmRzLCBSb2dlci48YnIgLz48L3A+PC9ibG9ja3F1b3RlPlRoYW5rcy4gSSB3aWxsIHRy
eS48YnIgLz48ZGl2PjxiciAvPjwvZGl2PjxkaXY+RG1pdHJ5LjwvZGl2Pg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 14:44:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 14:44:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188506.337673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQscY-0001bO-4y; Thu, 16 Sep 2021 14:44:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188506.337673; Thu, 16 Sep 2021 14:44:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQscY-0001bH-1H; Thu, 16 Sep 2021 14:44:14 +0000
Received: by outflank-mailman (input) for mailman id 188506;
 Thu, 16 Sep 2021 14:44:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AVhn=OG=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mQscW-0001b9-Bj
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 14:44:12 +0000
Received: from mail-pl1-x632.google.com (unknown [2607:f8b0:4864:20::632])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ec5314a0-9377-42e7-9063-bcc0573da18c;
 Thu, 16 Sep 2021 14:44:11 +0000 (UTC)
Received: by mail-pl1-x632.google.com with SMTP id o8so4028656pll.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 07:44:11 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 d3sm3759819pga.7.2021.09.16.07.43.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 07:44:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec5314a0-9377-42e7-9063-bcc0573da18c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=daOJCQ5ZGPvwc1951n97peFQn5CdNcYRqjhdeKz1vfU=;
        b=kFhwzpK+Y6fzZ8X7nPLZDzATwjammU+fVatUdP6GQmmOa4cwb7pps1Nw0JafcdN05c
         us+rB0fvubZS9oiEAFzEFlFf312AuI1o1bEmAQE4QmcO1zjNKu1RZh97KgxL0BNdbt63
         0Jr+VY4PzihMoQPiBbmbSubnZSyQYJcQQgB+obVuZmtoGKsNqbBtlR5Gbo8zv710wZ9R
         a8mRhJlIllHxWHu31340de111QSrO67EpQHGFhSYs/S/jlv0B8eWaFAGTKy4KKxxqu/i
         pxm1iG7a99zgqWZv5DChQlteGUPk7MFUk4oD2MTOCUtVe1ZR4X5ZKY5Ua3wnolLudPA4
         hmZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=daOJCQ5ZGPvwc1951n97peFQn5CdNcYRqjhdeKz1vfU=;
        b=fh6l41H/iaFznlq53s+o2IwThUPaTvUba5rKgQ6vIY+4Uc5XuZn6nw53BQYvCXGqP5
         80xYlThZU7IRkM5iLQLpvthXc9lKVPAbPVo+7GwhMz9TQyV6ClQdSb8JOrHItoQeeqhG
         G78PTFfjI6Z+GO5mcR4fiaMHxS1wzE6aOHENeNPbxekhemTjTq6mpSphwb8T1a0iQgWQ
         shMgS1uoB2sHzX73q2b9X3Lzjxt/dJEd706NLxlXruFfJb9VZOGoyy0rNqdnKJj1DTdM
         HThcUeCl0SQGBwruUw1z0zy+Fimvt7o57mbTH7l8T1hRfCZr0vzUSlwES4UViy8YvqVz
         /mWQ==
X-Gm-Message-State: AOAM531Lx1SSP2Tznm3hS9wf/59i/GRvP3Q8Cv2q6x6hAH/sb3aR/zy9
	MdsmEAb9QmFZULUErFUnqww=
X-Google-Smtp-Source: ABdhPJy1NmsnM3CBTxeh4uCd/At4jyjgdaLl8A9jDJfcHgoAtY82XgoTk9uMDIbNLVKuF0mho51VHA==
X-Received: by 2002:a17:903:248f:b029:128:d5ea:18a7 with SMTP id p15-20020a170903248fb0290128d5ea18a7mr5070119plw.83.1631803450601;
        Thu, 16 Sep 2021 07:44:10 -0700 (PDT)
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
To: Michael Kelley <mikelley@microsoft.com>, KY Srinivasan
 <kys@microsoft.com>, Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <850a0129-ef82-67e9-165f-8503da6889dd@gmail.com>
Date: Thu, 16 Sep 2021 22:43:56 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/16/2021 12:21 AM, Michael Kelley wrote:
> I think you are proposing this approach to allocating memory for the send
> and receive buffers so that you can avoid having two virtual mappings for
> the memory, per comments from Christop Hellwig.  But overall, the approach
> seems a bit complex and I wonder if it is worth it.  If allocating large contiguous
> chunks of physical memory is successful, then there is some memory savings
> in that the data structures needed to keep track of the physical pages is
> smaller than the equivalent page tables might be.  But if you have to revert
> to allocating individual pages, then the memory savings is reduced.
> 

Yes, this version follows idea from Christop in the previous 
discussion.(https://lkml.org/lkml/2021/9/2/112)
This patch shows the implementation and check whether this is a right 
direction.

> Ultimately, the list of actual PFNs has to be kept somewhere.  Another approach
> would be to do the reverse of what hv_map_memory() from the v4 patch
> series does.  I.e., you could do virt_to_phys() on each virtual address that
> maps above VTOM, and subtract out the shared_gpa_boundary to get the
> list of actual PFNs that need to be freed.

virt_to_phys() doesn't work for virtual address returned by 
vmap/vmap_pfn() (just like it doesn't work for va returned by 
vmalloc()). The pfn above vTom doesn't have struct page backing and
vmap_pfn() populates the pfn directly in the pte.(Please see the
vmap_pfn_apply()). So it's not easy to convert the va to pa.

>   This way you don't have two copies
> of the list of PFNs -- one with and one without the shared_gpa_boundary added.
> But it comes at the cost of additional code so that may not be a great idea.
> 
> I think what you have here works, and I don't have a clearly better solution
> at the moment except perhaps to revert to the v4 solution and just have two
> virtual mappings.  I'll keep thinking about it.  Maybe Christop has other
> thoughts.





From xen-devel-bounces@lists.xenproject.org Thu Sep 16 14:48:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 14:48:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188514.337683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsgT-0002QR-Kz; Thu, 16 Sep 2021 14:48:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188514.337683; Thu, 16 Sep 2021 14:48:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsgT-0002QK-I6; Thu, 16 Sep 2021 14:48:17 +0000
Received: by outflank-mailman (input) for mailman id 188514;
 Thu, 16 Sep 2021 14:48:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQsgS-0002QA-Cf; Thu, 16 Sep 2021 14:48:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQsgS-0008RT-7g; Thu, 16 Sep 2021 14:48:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQsgR-0005Xs-UN; Thu, 16 Sep 2021 14:48:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQsgR-000095-Tq; Thu, 16 Sep 2021 14:48:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fjhNLuC57vn5pIEiXXbd7xgnnQzukUJD7O9O+I1g+8o=; b=Dkkj71AJDxCN08/WrBu6BtQ8IP
	Rx2VhljyvQSOxbwJdeRtIvWTvsbcQIqUSM1TBWvXHtI0FoklWKdcBXEIX/hSX5huHUdCQp7gRZLk/
	BdAGhdvs/Xx9SGoeMF6AT3vPYgJvmjB9346L6WHAwFGK5YldzhPUOcbESJB+FrfY2kHk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165005-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165005: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=815056cef8dac827b07e76bf84c87eec0f9ac7ed
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 14:48:15 +0000

flight 165005 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165005/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              815056cef8dac827b07e76bf84c87eec0f9ac7ed
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  433 days
Failing since        151818  2020-07-11 04:18:52 Z  432 days  423 attempts
Testing same since   165005  2021-09-16 04:18:52 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73639 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 14:49:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 14:49:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188523.337698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQshL-000358-5X; Thu, 16 Sep 2021 14:49:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188523.337698; Thu, 16 Sep 2021 14:49:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQshL-000351-1O; Thu, 16 Sep 2021 14:49:11 +0000
Received: by outflank-mailman (input) for mailman id 188523;
 Thu, 16 Sep 2021 14:49:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQshJ-00034r-Rp
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 14:49:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3f139f3e-16fd-11ec-b60a-12813bfff9fa;
 Thu, 16 Sep 2021 14:49:09 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-7-2mv1bDHyMEu-Sba3Rzh9Cw-1; Thu, 16 Sep 2021 16:49:06 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 14:49:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 14:49:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0083.eurprd03.prod.outlook.com (2603:10a6:208:69::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Thu, 16 Sep 2021 14:49:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f139f3e-16fd-11ec-b60a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631803748;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=qsGwjLGjMd2sjdb1xjhyVcOKuQDtTDX1mMRtJDJN2l4=;
	b=Pi+61BtO5YQMukvvK1hk4HfV3buBwtHv2LV9Bao7N+k1RDsUr36ouBhmIYRv878FpARCh5
	PpDTvSP7EnqxOAvNwDOxyXhT9CjLACy6gYQyx+u4nxkA+MSPokOlRBijdhfHBKFLYMzmBM
	s4QWVmIRQoJ7yIuAkEFA/5qp9sD1hmU=
X-MC-Unique: 2mv1bDHyMEu-Sba3Rzh9Cw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eEIpGdr7sBcDl3WYy9KvNODReUDPDf57P3VkDAcVWtQaTIlzFXEVesg94xOlRqXPofS9SFpIoKuSNkdUBt2o0XbV5B7lJmsJuDMjaDJLvA5t5BPITXtygViwbNi0mnTlkGPiTqM+sJeY92Gho5SW4ieIG+DPASRlAb/DDC+iGCzkxaQHNBot7CpAnxNwZ3Myuw//o22CkvMMoshIm1MoFSP1gt5nmPyDg72QTnCphQYp/FEChdh3o+ksH9KUzWpHkF9qHzomQul04+oThaB8Htxg3ulNXvfA6W76R54G4RxPA/PWtEfqL4UoQPMsj0mVATpYTqtszDQw79f7ddDoVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qsGwjLGjMd2sjdb1xjhyVcOKuQDtTDX1mMRtJDJN2l4=;
 b=bHTZw4xJOTgkBodUJo7mrVjOpMMLn/A4EmiQz0l9LsjtCHqqX/ui9QS0Aey/8Ej0JOkFvhiNESVcv8nAttU9ZC/xU867L+9YSznwrUkONsQ3MOnH2QSXwFvejGTMU24ngizPKOxFROL+8W/3GV+c25jqr4mcyAUYhFOt/Ke13XUsFqUL2y1kqjQkNnQyQsKRB7RrouBcuA9CmM33vAAQba99In5ozxAX1AhgwE/Tq/H50BdRRbIqqxSjGthPJRg10QuRQLJKGtaGsqj5vxmW6yHkVPYP35i3fUhNvwHujf6rU1CJSiZDTbvkmoZdFXW1o6wk8t4paJ6TrHbVMzkLOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
 <jgross@suse.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 xen-devel@lists.xenproject.org
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <03b1a1bd-2630-0ee7-b995-114632f527aa@suse.com>
Date: Thu, 16 Sep 2021 16:49:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0083.eurprd03.prod.outlook.com
 (2603:10a6:208:69::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0ffd8d97-61d0-481d-bba0-08d97921208a
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB39331DDC764E18275CC390FAB3DC9@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Fab+ZPs/TGD2hmgqKOWvaEDXXljm/ekDblO9hKwGsVsXVA0SZ3MD5jiS1odGAsjPITMIkCI8acWiNvNvj8MnWaudFFHqZo8M3ExbwcWyGubha9DTH5WylaI/QVwWMDH6CsK7W93HTwF1dBZgAChLx0YhIsB3OAFSr3QFecXMKn9mVb2Se+DP4cmpC+ut9bcgzUblrdKyarzFdGvm62A9veo6dOZzcXMq524HJAcf4YR6mgvpVfAWrdpKMKDqRZ6Ex/Qw1Q0u+0Nejg9JAMUWrLeFv2jd0VrzVc2oiwKvXGwJEVABpO7J2SqiFbFOCU5p+6iWvBjY9xus4uJeKgrPBkHnuCe5VBiwsC2czDdEvjnKwrIfgATHTidu1vx4TSj0w65Dnqs1XdrZSB4fIh42kpvGdPVjDf3huOjUrkd5e6pRf98guHu/OIvpLbJYO9SwnJehcKUh+vTsSomXhUWZHMm/CIKJMJpJy/wNI9BO95CEElNhvXlbU13ZEPBl2loOnnEIAGiqsCEvSH0hs/K4F8Cji8bVDDQaX/6aY5WD3ZKkwftUu36GSTVj3YlM9AGHHdDn7IzaXVftGgZ5NPTUkMX5c1mrW8aACnerF7R2bKQ/ewc9s9b9Y3zN0BAA36yO6cqwdPW+A8t5joC0Ei9fYwBp8NlPdtJd9fpEMMbLED7R0Iq6hs/BS6NkHKtgjn3ULxc5ObgctwXu/+p1K6Oz2igDvCYpnJDBbxwki9P7578=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(39860400002)(346002)(376002)(136003)(38100700002)(186003)(36756003)(31686004)(478600001)(7416002)(53546011)(5660300002)(4744005)(86362001)(956004)(6916009)(26005)(31696002)(16576012)(8936002)(2906002)(316002)(54906003)(8676002)(66556008)(66476007)(6486002)(66946007)(2616005)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VHQ3SDdFNjF2S1lCR0JuVys0WXRlTWhsYUFhU2ZxUzJFc1Ird01DaEUrOTFU?=
 =?utf-8?B?TldUT0lQOW1jMEllMzdJd0FGM3BiNE45a29nK3J3OE8zYTZwMm1NVG5aV1dy?=
 =?utf-8?B?QXUxTkNTTDR2d2tVM0crUDIvWjRLQzlobi9NWFJaSENRSnVRNW5XN2JLbEI1?=
 =?utf-8?B?eExiMm51T0ROV09sT3c3REZyQ2Y1SkpTbElVUS9MVVM4VE9vYzErN1VuZU9x?=
 =?utf-8?B?SEVPZU83TUNFYXdWQkhZb2RrWCsvTi9zTVdCUzl5L3F3WUtJZUNHS1VKVllp?=
 =?utf-8?B?Z3ZFREpCa0ZZU0VVaWNUMHdiQWZuaUNuYk0rb3BDVnNLM0MxKy90K00xT2VD?=
 =?utf-8?B?VmJxSlcwWGs1dlVYSFVoSjZhQkdwQWdqMnoxdVRhT3ltTERBazJyNThZM0sx?=
 =?utf-8?B?Q0doQllEZndpTDRJbkI2Njc0LzhwbFoxc1RjVXZVWFdtS3Izd216b3FLR3JH?=
 =?utf-8?B?TGlXUTBlS0ZTUWxKZndzUzJrZWMrNEE0SzNKbTYzREs0RnZERlNxaU43MU0y?=
 =?utf-8?B?dFlkblEvaEp3Z1Fxenh1bnRUcXAyb1RkUy9OZFQybi9ZQzRHYTdvYU9GSWF1?=
 =?utf-8?B?cnJjbmxpWllucjdzY0xwMEtYVUVlYnNMOW5ncXhPZi9BWFdZaWl0dmhpWjAr?=
 =?utf-8?B?dTBjSnRsSkUrWWdueGV0dGhRcGRPWndjSVQreGd5WGtnNEt5eG82cytwWjQz?=
 =?utf-8?B?SVB6cFllZXM2eWN3NTVXWElqbHVCT2R2S3BhRzhtOGl5T3RQNDFOSE9kK3Y1?=
 =?utf-8?B?azFwV0JMd3g5WndSRGNwaURkQTRYbVUvajlsMnhsS2dYc3Bwc2ZQbnY0WmIv?=
 =?utf-8?B?MTZFYWxmYkVCTXhkZmFXWTNSWG9YSzFhSmxrb2xWSGcwRTRNdjMwUW1MUjlP?=
 =?utf-8?B?K0Y5Vm5oempQc1JkSm9oUHo5SGdxUHNDRGFQQ0U3eHJ4Qy9MYlUvMmpNTFBF?=
 =?utf-8?B?VmRGbmhGakc5QmJQa1E4UmlLS3hMYlVKRDlpaXlIWjBQa1JObWF5MnUyWFp1?=
 =?utf-8?B?dzNQZWQ2T1l0MzRTRmlNVUUrVFRpUlFONnhHNHlJZ3Z5bU01MzUvZktHTFZC?=
 =?utf-8?B?TGhwVG9hZDZJelZxUWgzZy9ibFlZTm42d3JQK0tHTU5QNXpFNEtXT1BEazBt?=
 =?utf-8?B?d0tpNm9yT09OVlczaDMwRmtDdnQvN2M3c09HRlZMNUw0Z0V0Uk0yZDF5aVJp?=
 =?utf-8?B?YUJRN2xpa2FNc3UzNzlTSjRYcXgrNHFOd285MzVkdTdMQTRsZkNCU0pqOFpy?=
 =?utf-8?B?M1JXeFBSZzhYNFhoUXZiMGFzT3c1VTVKNGo5bFYrVjc4a3hJbEsyYktWVzhH?=
 =?utf-8?B?ZHMydDl2Ty9ESGhTRDdOWWovc3pGbEdDYzBybUJiUWt3MktZZDNHeFdIcjcz?=
 =?utf-8?B?TUdJa0tBNktLUE12eGJrUVhFTmJRaXBFN25NemQ4NElIRzMrcE9NZCtqQWRt?=
 =?utf-8?B?UEYvV1IyYkFJQ21PRmQxRWEvZ2FJQ0FuMGFTWmZIVURFN3kzQWlUQkdpb3dq?=
 =?utf-8?B?NDFDTFpodHEzdlRYc1hxWHBJQ0hqNkN2RW1nSlhCWmY3cWlHRjhFMXdQYkpq?=
 =?utf-8?B?NHE4OHdSQmp2UnNIa0IvZVUxV1pHQmU2WitoNUFHN1F3Z250bHpOeG1tRCt4?=
 =?utf-8?B?VGJybEtkM1RYNUJrQnlzNEZScjNxTjdiWnZ3WWk4MXdXczVzTFZxVFVWSFAx?=
 =?utf-8?B?Y09STEZRZ0FaM3hsUzdkV1dKSGVESlhnRTE0aWJTUVFpUGRnL3M2ZHlYUU9R?=
 =?utf-8?Q?ZSX7kDlZf2XRk6xmpnL8ppPTSxYc6D+gJVvQ7J3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ffd8d97-61d0-481d-bba0-08d97921208a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 14:49:03.7579
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lLgdH2fJ9U/Jp2CeS2LlfRe1wTA1NSB5lP1FTB+ucWUa3oiFxfpeTlRYodTz3qFN9wb7HpwFNs8Cno8md8Ca7A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 10.09.2021 20:18, Oleksandr Tyshchenko wrote:
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -855,6 +855,13 @@ typedef struct libxl__ctx libxl_ctx;
>   */
>  #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
>  
> + /*
> +  * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
> +  *
> +  * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
> +  */
> + #define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1

Nit: I don't think you mean to have leading blanks here?

> @@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
>      uint64_aligned_t outstanding_pages;
>      uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
>      uint32_t hw_cap[8];
> +    uint32_t gpaddr_bits;
>  };

Please make trailing padding explicit. I wonder whether this needs
to be a 32-bit field: I expect we would need a full new ABI by the
time we might reach 256 address bits. Otoh e.g. threads_per_core is
pretty certainly oversized as well ...

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:04:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:04:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188531.337712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsvr-0005Yk-Ht; Thu, 16 Sep 2021 15:04:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188531.337712; Thu, 16 Sep 2021 15:04:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsvr-0005Yd-ER; Thu, 16 Sep 2021 15:04:11 +0000
Received: by outflank-mailman (input) for mailman id 188531;
 Thu, 16 Sep 2021 15:04:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQsvq-0005YX-Ex
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:04:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 57cd548c-16ff-11ec-b60a-12813bfff9fa;
 Thu, 16 Sep 2021 15:04:09 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2058.outbound.protection.outlook.com [104.47.14.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-53EVhdX0PkS0f6-hC2a2BA-2; Thu, 16 Sep 2021 17:04:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3776.eurprd04.prod.outlook.com (2603:10a6:803:18::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep
 2021 15:04:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:04:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Thu, 16 Sep 2021 15:04:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57cd548c-16ff-11ec-b60a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631804648;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=Q6WgI46VbGAw65/T84e6H5PFsEF1GgiBtZusySz5miE=;
	b=SaE70UTQBMtPGbooQ4kyKw2N/jMJPyPDolTfEowJQlL6kGRksAej7kBE/jXFRbAFVRZVHX
	8v08PYbSaW2LhqEbDv5Jx9cup7+RgqLk0JJN08rCj0MM7l7Zy4f3NvJ/IuGnaaPPa7UOP+
	CQPFcxvGA8gxc6k1YoND72nXPvpUdtM=
X-MC-Unique: 53EVhdX0PkS0f6-hC2a2BA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QNpbuhjPGqHbplYflYUAuqsphOBVrXRjWVcO59eVLqIx4q/xFFv7f9otMGiGsi5CHnKE97BHZcZlER5ICcm1XqDc/VnDOKpbG6t6yQgfWx6P4gvOcTU16sZfJ16oPybkbnxRfMo97YE0i1bmTMZFfVO07Ecp371ocFCFBnhxO/7b78NeBsLrvUu7stJY++h88oAtetboDZ+JEDhhw7v9nwh6VqIhtbevJ1HatDd1z+aNQhk2wwxryhFkCwaBPxPYajRxeLFXi/HoAUVp/nz0eaz8WhrXf0jQJZjB0VNhQz/hg96A++uip+Dg1xZXWFSm4JE9R85+MvcxEKwLOG1yNQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q6WgI46VbGAw65/T84e6H5PFsEF1GgiBtZusySz5miE=;
 b=n9dGAdavGklA1FWLKWcnyhZFvEcicaePh3hWIxRGkyrIaBgADiGid0L5XtpznpCceppC8HfetT3KmpNoiVxXh/pnbTkZaNME+MqGHgMScOEBhn18B0lmv8rXb54vOjblXG++/7GspUFuYKrWgOl/f6VgEyVoXRHnox76PVAqAOUywR1HXBoOiuRguZr0BWhIKDVgLyUiqKQYfdDEa5/ZdlKurMqnLtP0yCU4tWfThxyc5ozP1LxLE0/9ZDlG/jCsR2KxBeCLaXdr76Wfd7Lg77MlhS3za9b+WvTrcwi4CZa65uSg9IIFeTYN/URdDwYF/naXlfdyoJ8SV+VU5ngZ5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] xen/x86: fix PV trap handling on secondary processors
Message-ID: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
Date: Thu, 16 Sep 2021 17:04:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0017.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 27943df4-f1f7-4c9d-c43c-08d979233923
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3776:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB37768C2B682C46C921F067B4B3DC9@VI1PR0402MB3776.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	BAQeGepHg1/vtCcixej6RiVXYgnbDkMNfPb5LlT8RShXNSZWrC3l+DFvZRSTmWvGLIxgDEThvJT+vPBooAPvZnMYAplseKQXLNW6AWl4ey3hc06m6KmjPWojO0qDIKLB5KPeR+OrOW9VEUAPwWvu5bKX9NBDfRFOHBXNUiwRVNfG9mK/TUdMJK8Db4Do57Horf0ffED2CSADr5Fc2y49ehFpvmzyrpHSshBK/xdAs2PsvMjVd50gByzy1MwfL+dOFGMAd5fsODc1W3x/Ll9+WjIRl1LTqDGcbW6Lp+VioeeWBl/CJdPuIhHwXJN5Rbalb+D2hDIwM4ABR9yt+zePhynjmDN/z1SdMWDrRf7bDeVIh05LkBPFNztqUsLf9oA6BUEHPWyywZn8dFHSYazZwzjl9q5VvFPjEvURB3I+oFMOek+aokhuUS880/F3/vhU95SFD/Mt9vr6SQdCmOMGaK5oTGItGxAWc8dmaV+eP79dDVLIwPlNGVA8Lj42sqQsmuIgE/OHwkD4xR/e9iYJ2PHPITditbKPd2SFRGfy2rWgUD+8c4pFBSm9M6V4pNB5muEslx/qd1ybEAx9m/YvyOmqZWBaeRenwEnrlNGt0TAJ8JpRhj45w68DXo9rnjsZ5I0fiJ1ily1WxWNEMm4PB7XQB3BLHc1nAh7Oj6LgCajw2GgUEqAiidBjR4xV8uwlFm6VhSUoCPVXmdhYWb0UMB3ZKZ0/Q941A00bWikbXBo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(346002)(396003)(366004)(66946007)(8936002)(36756003)(66476007)(66556008)(4326008)(2616005)(26005)(38100700002)(110136005)(956004)(31686004)(186003)(8676002)(16576012)(316002)(86362001)(2906002)(31696002)(6486002)(54906003)(478600001)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TERKK2JrcnVIVXhyRUsyb1JVdVRYaW1mV3drN05xMjJVU3h0bmhzNFFES0JH?=
 =?utf-8?B?ZWc4dUZPWWNYK2xoNXR2R0NGNmNiK1ZZQStHN056cEpNQUpsL1NKVnY2SnVD?=
 =?utf-8?B?aTk4UFg3ODZsc2lVcGNYNEVGa2puYXlwMmpjQ3J6UEozMGUwNWJGZDUzdEhH?=
 =?utf-8?B?ZDBXSmFON2VUeDZaeTNHbFY3MU9JUWM3cGZ2R1o2WjBTTGxQcmIwM1p4UlNq?=
 =?utf-8?B?UVMxeGpFZlF6THdZY2g3Mm1nUmovbVVVQ0JTRlBRUlpjczJham5UVGVBYk1t?=
 =?utf-8?B?bFkxcmJmQ2x2WUlJODdEMElUbm1XOWUxeG9GTjhMcVBoVFdsRGtyS3pIRjIw?=
 =?utf-8?B?RHVIdE1OYkRzWG04TE5ReW9YNytyVGF0cVpHZVB4R3RiVG1BV28xOVUrMFZD?=
 =?utf-8?B?TVcwZkJlQ3ZDQlRaRi9rakRzVVB4a0JEcWNNcGlVYjMramNwRWQrUC93ZTFC?=
 =?utf-8?B?Mm9JR2N3R014YzFHaHA4YTA2VVpjTk5zb3NqUWFGQWlvM25FQlpkQUVUTHNk?=
 =?utf-8?B?cG42Z1JHaFFRZGpFaXZpS0dYUkFRcHVtN1VUV0NSZWUzdUdvaEtBWkd3TW9K?=
 =?utf-8?B?TnBYdnpjdnd6M0dtLzJ2WjEzU29UNmVLa1EraWIyeUdTYVNSZ3NLNkNVTmZv?=
 =?utf-8?B?TDNwcmNUL01UWmI4TGN0UERydHNLREZHUTVldm5QOTAxWk5oYm16VTZkaC9z?=
 =?utf-8?B?cG05QW10TVoxNHUrK2pJYm9UVnp4RDBDanBYaFBhc29nMFE4OGwwbXltRXow?=
 =?utf-8?B?S21tOWUvVlMwUWRyTmdtRnJJM1pkeVdOSHBNemZxaGZUNGRlTHZwYnJHK2NW?=
 =?utf-8?B?d1BWMG04UkFxczY3cUs2Qzh2Z0l0NFBhUDZSUFNVTW9pa3dWNGpqcEtWL1B5?=
 =?utf-8?B?dmJRWVVaY2UwQ0tUT0JRT1JCZ3FqaWV1M2lhSHZ5bExLbFcwMDVzRGMvRDNU?=
 =?utf-8?B?VEFZZlYvempNR2NtdjN2a3dIQnF2NE84TkJ0eUdiZnI1NmxXc05zdlBJcm1M?=
 =?utf-8?B?YlJpbVp4OTF3dkdxKzlqUGtqcHpFUlEySjVTemt5YlJEaVpweGRndDFBNmND?=
 =?utf-8?B?MllVOURIRC9SQXVqMXBpb0J1Ykx6NHlCRXFSa0FPcmZHa01xZnFtNDRkbFV2?=
 =?utf-8?B?RnVzUkpBak5zSE9Lam56U0IrUkIrQzFKT0k1OURoWlp2eXEwbmt4bS9HSGpJ?=
 =?utf-8?B?WGtjUjVqckgvN29JMXRVZnNSbzh0cHR6WHBuL05paWRSRnUwN2dDM3l4bklw?=
 =?utf-8?B?N2FqY25jVURTdXF6V1ZIZnMvRDVpZkdhcXBDT0VKd1A4RUtxb3lEZHhIYVJr?=
 =?utf-8?B?ZFdUK2hJMThjOUl3czFhOXhQN0pMQ1BpM0dwek94bThKZmt2bml2ZWx5Rk1P?=
 =?utf-8?B?UWVQV3NpUW1SQVYzOUtmbmNNbEFyV1pMd2dpdnFTdDlieWpHYU9yT1ZOTHJv?=
 =?utf-8?B?ZkwrTmJLeTFEb1hkSG5xTWlZTzJTMnhIWFNpUzN6ckJnYjduQ2M0MkZzK3px?=
 =?utf-8?B?cjZuMTAvZU1MOVppbzA1VjRkNHB4Vlg4VG5MWm84clFDNkk3dUhFVWZkWFBs?=
 =?utf-8?B?Mk44VkNtcmRCbFpwU0RjN3cvMkJQd1RaUldqeXFpajdZZDZHRm1EeCtTYy9y?=
 =?utf-8?B?bVBYSDNKRURHRFR2c2poeWt0ZHU1dFZQdjl0NUcrTW5MaEFCOFZzY0JJdGRm?=
 =?utf-8?B?bWdiYVY2UUJTS1RrMDhTNWJqRkFMMHlXMG4xU2ZGZzJOWnhZalFEZHBFR0Vr?=
 =?utf-8?Q?KrnCgsNdrVIy4jno558XTCXR25HZbyyHnMzdrT3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 27943df4-f1f7-4c9d-c43c-08d979233923
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:04:04.0076
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xBFEs8rEgWMPTukFLHsTNvSkxcujFMnG4RSnAG5Z0N6tvqA4Mi8XNVkaON8Y0dRRCUgq2Vf1Hdl8CIinT4kacg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3776

The initial observation was that in PV mode under Xen 32-bit user space
didn't work anymore. Attempts of system calls ended in #GP(0x402). All
of the sudden the vector 0x80 handler was not in place anymore. As it
turns out up to 5.13 redundant initialization did occur: Once from
cpu_initialize_context() (through its VCPUOP_initialise hypercall) and a
2nd time while each CPU was brought fully up. This 2nd initialization is
now gone, uncovering that the 1st one was flawed: Unlike for the
set_trap_table hypercall, a full virtual IDT needs to be specified here;
the "vector" fields of the individual entries are of no interest. With
many (kernel) IDT entries still(?) (i.e. at that point at least) empty,
the syscall vector 0x80 ended up in slot 0x20 of the virtual IDT, thus
becoming the domain's handler for vector 0x20.

Since xen_copy_trap_info() has just this single purpose, simply adjust
that function. xen_convert_trap_info() cannot be used here. Its use
would also have lead to a buffer overrun if all (kernel) IDT entries
were populated, due to the function setting a sentinel entry at the end.

(I didn't bother trying to identify the commit which uncovered the issue
in 5.14; the commit named below is the one which actually introduced the
bad code.)

Fixes: f87e4cac4f4e ("xen: SMP guest support")
Cc: stable@vger.kernel.org
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
In how far it is correct to use the current CPU's IDT is unclear to me.
Looks at least like another latent trap.

--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -775,8 +775,15 @@ static void xen_convert_trap_info(const
 void xen_copy_trap_info(struct trap_info *traps)
 {
 	const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
+	unsigned i, count = (desc->size + 1) / sizeof(gate_desc);
 
-	xen_convert_trap_info(desc, traps);
+	BUG_ON(count > 256);
+
+	for (i = 0; i < count; ++i) {
+		const gate_desc *entry = (gate_desc *)desc->address + i;
+
+		cvt_gate_to_trap(i, entry, &traps[i]);
+	}
 }
 
 /* Load a new IDT into Xen.  In principle this can be per-CPU, so we



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:05:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:05:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188538.337723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQswi-000665-Rv; Thu, 16 Sep 2021 15:05:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188538.337723; Thu, 16 Sep 2021 15:05:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQswi-00065y-O9; Thu, 16 Sep 2021 15:05:04 +0000
Received: by outflank-mailman (input) for mailman id 188538;
 Thu, 16 Sep 2021 15:05:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eAL9=OG=intel.com=tamas.lengyel@srs-us1.protection.inumbo.net>)
 id 1mQswg-00065o-OG
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:05:02 +0000
Received: from mga07.intel.com (unknown [134.134.136.100])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 76241d08-16ff-11ec-b60a-12813bfff9fa;
 Thu, 16 Sep 2021 15:05:00 +0000 (UTC)
Received: from fmsmga005.fm.intel.com ([10.253.24.32])
 by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 08:04:59 -0700
Received: from tlengyel-mobl.amr.corp.intel.com (HELO localhost.localdomain)
 ([10.212.12.147])
 by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 16 Sep 2021 08:04:57 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76241d08-16ff-11ec-b60a-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10109"; a="286269361"
X-IronPort-AV: E=Sophos;i="5.85,298,1624345200"; 
   d="scan'208";a="286269361"
X-IronPort-AV: E=Sophos;i="5.85,298,1624345200"; 
   d="scan'208";a="700650535"
From: Tamas K Lengyel <tamas.lengyel@intel.com>
To: xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas.lengyel@intel.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/mem_sharing: don't lock parent during fork reset
Date: Thu, 16 Sep 2021 11:04:42 -0400
Message-Id: <c21708c84c850ff8c976a3934099c54da044e7d9.1631802816.git.tamas.lengyel@intel.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

During fork reset operation the parent domain doesn't need to be gathered using
rcu_lock_live_remote_domain_by_id as the fork reset doesn't modify anything on
the parent. The parent is also guaranteed to be paused while forks are active.
This patch reduces lock contention when performing resets in parallel.

Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
---
 xen/arch/x86/mm/mem_sharing.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 8d5d44bdbc..b80b978ef3 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -1879,9 +1879,10 @@ static int fork(struct domain *cd, struct domain *d)
  * footprints the hypercall continuation should be implemented (or if this
  * feature needs to be become "stable").
  */
-static int mem_sharing_fork_reset(struct domain *d, struct domain *pd)
+static int mem_sharing_fork_reset(struct domain *d)
 {
     int rc;
+    struct domain *pd = d->parent;
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     struct page_info *page, *tmp;
 
@@ -2226,8 +2227,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)
 
     case XENMEM_sharing_op_fork_reset:
     {
-        struct domain *pd;
-
         rc = -EINVAL;
         if ( mso.u.fork.pad || mso.u.fork.flags )
             goto out;
@@ -2236,13 +2235,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)
         if ( !d->parent )
             goto out;
 
-        rc = rcu_lock_live_remote_domain_by_id(d->parent->domain_id, &pd);
-        if ( rc )
-            goto out;
-
-        rc = mem_sharing_fork_reset(d, pd);
-
-        rcu_unlock_domain(pd);
+        rc = mem_sharing_fork_reset(d);
         break;
     }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:05:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:05:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188542.337734 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsxH-0006fH-4a; Thu, 16 Sep 2021 15:05:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188542.337734; Thu, 16 Sep 2021 15:05:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQsxH-0006fA-1Q; Thu, 16 Sep 2021 15:05:39 +0000
Received: by outflank-mailman (input) for mailman id 188542;
 Thu, 16 Sep 2021 15:05:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQsxG-0006f0-Hb
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:05:38 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8c61f284-16ff-11ec-b60b-12813bfff9fa;
 Thu, 16 Sep 2021 15:05:37 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-sTps91zyMpWuwZIP37Keqw-2; Thu, 16 Sep 2021 17:05:35 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3391.eurprd04.prod.outlook.com (2603:10a6:803:3::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 15:05:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:05:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0028.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19f::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Thu, 16 Sep 2021 15:05:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c61f284-16ff-11ec-b60b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631804736;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=IP9Nv0cIrdlrBoAnJCxni7bp4ywqxfkmNM9UrWGzWYA=;
	b=WLxIgR2s+41eIX9orcyQ6LxBDz+yXQpe5B8FKKQqpVSb50cZFuB+92X0ADX2xpm4xJEkKG
	8ELVnKMhVrrGcT56GqQVunY7SEBuy4re/jjYJsfpdlC1NoHx+s+O9MDP/+JjowP0wZQacL
	fCTaX0nx+usa3P20iAsrfXNTFE9pLGk=
X-MC-Unique: sTps91zyMpWuwZIP37Keqw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ns8PIUH4HBPYN8nG/aNG+hoF+00HD2FvBYj04W+wLoQOLj3EPWr5pupwDtfkcdAavvCQVPOYb7jE2NX7iHDL6vbpVvhhOdHilnCQs+CtHQdckCmt345stSHVYCpjdaUhCayz2Ozkj+rRmd9PDVOdL6Iq0E9kKHNwFhuyQrKzFiiZxaJbU8FAgQ0HwydrJYlUUquEzU17i/bGehJlgN70eMEphMLsjlrsDqWFFIacDSFNZQl49m40vq+qVAT9QdBsvQSnEdd6yqqiPw3xyGopeAcj6SBFF5wi43WSb7FNcekqylm2obOJu74alV2Q6Ql24ud4mPBnoMLQkny1oZQHhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IP9Nv0cIrdlrBoAnJCxni7bp4ywqxfkmNM9UrWGzWYA=;
 b=C8flN417nCodQ2W3sFYV88/LHD78fPsNqZnE264MVif5zbV744T0LWwe+KsPa0b/WwmYiCsEF6iMfUOQFkoX4V7PSgMuYdToNrUGMZft2zM6jqBd84UNJkE1dQ/0Wt4MqFCOfFj7u8E2bniTRE8xs82JhSfdn5Iclk21IEcgUmyEx75jo2gQyUcbhZKLZ/4FaANJKZZpjooKMJinH5qn+8tnIGnsWit4S9s7wz16gY2i2gKqUkpDloTGKrKBT6T3Z16M6hrT3zpS1cnYPMGcKlVOBUp6au5+b9+RLGPZl/8vq72jbJiSivN8oYuioY/GlC57CIo+V4b1/12Pjt58CA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] xen/x86: drop redundant zeroing from cpu_initialize_context()
Message-ID: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
Date: Thu, 16 Sep 2021 17:05:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0028.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19f::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d05d157-bfbf-459e-c8bf-08d979236d7f
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3391:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB33912E40E85F83CA726EB613B3DC9@VI1PR0402MB3391.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	626dTTzEhrXcrterC0Mzfw5SNpDHveJqKy6BuH4I3wv9m0pqsqwQBHOkHizgtrVXqnvXy/C8iFJIPE1XCE1UG4VpFo/iEeO7w6eIkore8FXzPKrnOtom5QCyfYd6O7uLZ4KjKBUo3Ggs1rBJUZmDLwuG8smFl2QVacYXl3wErLYZkG3JgC9Nn7Uozf03BPF7yGYs7D7L5rcdQ208NMO2U5KRWyscVA6jm/lkZii5qIicSfslol+nkIJEM+DpmNsQJm75wEdhYdaqtbIPaocNThJyDGoJF+CD1uVq0spfPeURUyHZqt0/KJHygVagP/F04sXMLw+nOg61tLNZK8XnuriUfYEqhGcdNHe0PSNLv9aXdKIuwVLbQC0Zt8EQ2qoSk5A13Xjffz7NfL1JEEVcTjEc2XpexDLdjsagEo53p0+qTSvqdyP9vGbQfLDSd6FmlWYHxPaLVepYMBwIx6u/uGTU5GTSHTUH/ltxMyWwKJwAjP28uZIX6zL8pZ5H41/bTqpc7xDMm5bptzOEHeCItnowPORvCxRwPbZkGr29Badtl31Wr54uSAgYJ7V/REVp9iddGXt2fjLV4Qx1xnR6wHnFY4EH4mlXY9WLyXpsqQxR2fb91MTCIPVHd70EzTruitOhTrtO37qoAkpPe+jtDU3rZN9yHrf2sHrRuzr0L+8/15ZyUN+57u/RjYY3b/RxEBFrdUw55CPwzQwiBCoXmRmOYgCJbEIty92GxDdRzco=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(4326008)(36756003)(66556008)(26005)(54906003)(2906002)(66946007)(66476007)(2616005)(31686004)(8936002)(16576012)(508600001)(38100700002)(186003)(86362001)(4744005)(110136005)(316002)(5660300002)(31696002)(8676002)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NE9rdGRYNGdyWG96WTlqZHVacVB4N3I1cFNIaW5KRWVIQUdtaUhRMmlPT3Qw?=
 =?utf-8?B?c3l3ZTNKRlBWTGk2SFlFMWlqeFp0L2Z6QVFVZGpEM3d3NHcrWmNIYU9qR2hv?=
 =?utf-8?B?dnE2bDVoT0hUZ0pXNzhsbmVpZkJaZFg1TmVNUmdyWk1vMXM0VWg5MFZxcVFC?=
 =?utf-8?B?akozTURtNkg5aXM2bVYyVm05clg2UHB2NFBBUXdyS1NhZ0NrNEhOZ0lXMWNM?=
 =?utf-8?B?eUt2Z1JEUEJsdkZzek5LTVo0dVRpT2dFTUdJMkZVQVcrRHNQUGc5RnFHTitJ?=
 =?utf-8?B?MnZncWNOd3NXZTY5Q0hNaTJ4b1BOYVp5M1JPc2I4THM2ZEdBQWhLWGhDNUti?=
 =?utf-8?B?Q0FBdmRLa3RINVE3SG91TGlOY09GdXJSSzdGVmVZY2dpNzdvNG02VFJmaXVq?=
 =?utf-8?B?ZXEwSlpHdGZvMGk4UXVsU2FNK0NCNUVTbXJNdExCN0RRaEt2dVR1MFBnczB4?=
 =?utf-8?B?UHVMZkpXV0hJVDZEK2NPY25ndVNoS2tLclJPL01Tc3hFS0FiaFFuYzBKKzJT?=
 =?utf-8?B?UGt0Qm9mSm5FTjJHcWhoTWNLc1ZWK1ZyT2FyV1NmUnhPc2IrRytZa3dXZDJU?=
 =?utf-8?B?SVRqdHROaDRWY0thRWp6R1FOYXk0bUp6d3pzU25GQ1VSNzVxTXpHdE14K011?=
 =?utf-8?B?VzdkOTVVM1Jqb1p0dXhWMnErVzQxUE1RWlo0TXpxaWJBK3NkUnFvdCs5aHQy?=
 =?utf-8?B?aTVUT2tXNTQ3RnUzMmM2aGRNNlc4K3p2QVppSUFyV2N2Q3F4TzZiNFV1bTNj?=
 =?utf-8?B?TkZGRDZ3UWF5dWFzOFMzdzRtaGtaN0dIKzJOemE0blZRSWtDVE5SK1NVcFJz?=
 =?utf-8?B?NUprb09iTk02c0RueGx3RzB0azF2Um9raXZWYXZJZWlSb04yNG5YZ1BRdm5t?=
 =?utf-8?B?SGMxMGY2ZFZzbHpIcXdKNUIwTjNFU0FMRW9XeUJVVU41am1Odm9uNFJCRmNT?=
 =?utf-8?B?WkdEbmJFUktRc0U5UTRXemtXcnVjSEk5VjlPVHlCaVBqaUpEK3FhWDE2NTVm?=
 =?utf-8?B?Um5lOWhrS1RqMFpST3F4Wm9JVitiSDB1WDVxZHZFR3FhaG5JeC9abWJLcksw?=
 =?utf-8?B?aWltUytrRWwrRjNxbDZmUTI0ekFTRGowaEZjZXpaYmdDU2IyekZUWTdnN21R?=
 =?utf-8?B?UHM3a0NWN3BzVzFDK3owUTBxQkNOTFIzRnZseHBHYkVUcUsrUyt4dTdzQjhx?=
 =?utf-8?B?akk0d3N3OUh5TmVJOExIWVBsZ0JiUEQ2L01EcWpLS1hFOVhobVFETW5ISHRG?=
 =?utf-8?B?a1hIT2tUQXhaeGlGL0JBSTNaRHJqM0VhSDFwZVArS3Z2d0hveXdFczJoY3Vj?=
 =?utf-8?B?Zy9Fd2IwYktKWFBSOEQwRU9VU285TXNaQ21lYjFITDd3MDRYT1ZZSlJuQitC?=
 =?utf-8?B?MU5EUDBaMkovRCs1SUlZMlNZV3oram42RW5XaEJ6aFZYSjROQjdXV1JFUHFr?=
 =?utf-8?B?VHVhOFNBWVBpaW1vY1VzQ0NhanNoUUl5bjhiYm00RTRxak9OSkpqYndyRHFY?=
 =?utf-8?B?MkNsOUE5OTIvbmVSM1VsSytwUlVzK3o1S1JyTTJUR3Z5NmlZTjROYTlwZXgw?=
 =?utf-8?B?MEdGTnJEaURQSThLY1dMZTRld3Z1S2tMc0J1UUxQUE1CTzNYeGYwQjVRMzlu?=
 =?utf-8?B?V2E3bXBVZlZnMzJuUHV3aUJMNmg1TzdqalZwSG1odHlzZCt2VmNLUkRQQTk3?=
 =?utf-8?B?TGEyOHFCL1VCSUx2OUZnME1mdHV4WlFsaEtoMHNFdmNGdklSTW15L3FNYlJD?=
 =?utf-8?Q?xagxOGHF87mcWg/M0176/MiXG7YujzI0E0zrxAu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d05d157-bfbf-459e-c8bf-08d979236d7f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:05:31.8492
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Md2NeniJOpnnxW+2giAAEG2h9qgoEcLURlpj3saEUuEUFQn5fZGFYQcAYOphC7yMENC4hi/iwMBUz1PRQzjLVA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3391

Just after having obtained the pointer from kzalloc() there's no reason
at all to set part of the area to all zero yet another time. Similarly
there's no point explicitly clearing "ldt_ents".

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -290,8 +290,6 @@ cpu_initialize_context(unsigned int cpu,
 
 	gdt = get_cpu_gdt_rw(cpu);
 
-	memset(&ctxt->fpu_ctxt, 0, sizeof(ctxt->fpu_ctxt));
-
 	/*
 	 * Bring up the CPU in cpu_bringup_and_idle() with the stack
 	 * pointing just below where pt_regs would be if it were a normal
@@ -308,8 +306,6 @@ cpu_initialize_context(unsigned int cpu,
 
 	xen_copy_trap_info(ctxt->trap_ctxt);
 
-	ctxt->ldt_ents = 0;
-
 	BUG_ON((unsigned long)gdt & ~PAGE_MASK);
 
 	gdt_mfn = arbitrary_virt_to_mfn(gdt);



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:08:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:08:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188554.337745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQszk-0007dH-P3; Thu, 16 Sep 2021 15:08:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188554.337745; Thu, 16 Sep 2021 15:08:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQszk-0007dA-LV; Thu, 16 Sep 2021 15:08:12 +0000
Received: by outflank-mailman (input) for mailman id 188554;
 Thu, 16 Sep 2021 15:08:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U6Su=OG=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mQszj-0007d4-2J
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:08:11 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.88]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e69fbe20-16ff-11ec-b60b-12813bfff9fa;
 Thu, 16 Sep 2021 15:08:08 +0000 (UTC)
Received: from AM5PR1001CA0051.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::28) by PAXPR08MB7153.eurprd08.prod.outlook.com
 (2603:10a6:102:205::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 15:08:06 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:15:cafe::c1) by AM5PR1001CA0051.outlook.office365.com
 (2603:10a6:206:15::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 16 Sep 2021 15:08:06 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 15:08:05 +0000
Received: ("Tessian outbound 2e7020cc8961:v103");
 Thu, 16 Sep 2021 15:08:05 +0000
Received: from 64bdb17df644.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C358068C-8D72-4BBC-85D1-8C4DFFDE716A.1; 
 Thu, 16 Sep 2021 15:07:51 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 64bdb17df644.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 15:07:51 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6141.eurprd08.prod.outlook.com (2603:10a6:102:f2::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Thu, 16 Sep
 2021 15:07:48 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.014; Thu, 16 Sep 2021
 15:07:47 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0192.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a4::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 15:07:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e69fbe20-16ff-11ec-b60b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OuDY7kva6TB+DFTqavYUDWtKCiVsaC/k4SIMhEkr47k=;
 b=TfIQPoGVjjYzTLiZ7HrEOwUGgx76fFx4NHAgUnqC7bTTAgXzAy6SZwmLCo0WEEtMDG/5MvE5CH5a8V+BWFbs9byqlK2FyJW4dHAJr+b52mb1WnP+IZQMP7bk3ZsuOtkgVjbSbu94orTACzetM9Pj7nhICzu/CqnYDiHBVdeYjAk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 9447dfae84be5384
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vy7ojjr1QGzINLrWdtFc45jCxnNzbY8hpNzTzXH4ihyetVIcvwj0MCwgRPWjUj5fsI7QPskbcNy38KVUCEAGFx8vE0cfIFdQmoXSDDkxomaLwewJlo2HVnAJ5QNeIkybbqVX0hPuW3cYH8kfd7w3cGZFFz+KHvbD/Wj0jIsAq+9uCt4OzdwjfWC/AjMKlZ2LKYv1b42tOJMYHWXvw+jIyZElXe7FsBDd6AzdMsUOztNTYpemBP/YHXZT0eluw+BOqaA2qaYfiG1woNSnyRZk6qaGGHGPtWy1DYYRsnKztj4stBpoYyc696kOJbR0eMi7V161m3EzbBNxs0AOb8Bctw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OuDY7kva6TB+DFTqavYUDWtKCiVsaC/k4SIMhEkr47k=;
 b=Qf/eRcgLoBxvAkydXg+uKcxScqHaVkz16Bfrmwn93as0gQEDuJfY3exDKlmOJlHpWsttjNXLV3Z3Upep/G2kcVBcdN1jRy8f2FhLPaT6HOLxn2Wmic5ktYCvzaiS3yNgKmcHwXufhwAhgIyRBwdUvIUr/zykAxuAySyR2LhXiKT8UQ5RYBWPi4dEFxOuZQV/mfVYScEwveBk5qvUh+4SE+73JmxkLVd99nIHGniF0wvt8QqIN9MmnD72UFV1eLtZgAqu92JXiKhyEk/v06CxSYnA8N4ehIxR0URcn2fVE+U78e+hOPntABTmZFMzzaQDgNhD6tCCl9VWnLUciY99Sw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OuDY7kva6TB+DFTqavYUDWtKCiVsaC/k4SIMhEkr47k=;
 b=TfIQPoGVjjYzTLiZ7HrEOwUGgx76fFx4NHAgUnqC7bTTAgXzAy6SZwmLCo0WEEtMDG/5MvE5CH5a8V+BWFbs9byqlK2FyJW4dHAJr+b52mb1WnP+IZQMP7bk3ZsuOtkgVjbSbu94orTACzetM9Pj7nhICzu/CqnYDiHBVdeYjAk=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
Date: Thu, 16 Sep 2021 16:07:40 +0100
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0192.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a4::17) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 488a7de5-8727-4ee3-9b79-08d97923c941
X-MS-TrafficTypeDiagnostic: PA4PR08MB6141:|PAXPR08MB7153:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB7153A70A9F57C57AB7B2A001E4DC9@PAXPR08MB7153.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 EaEyweubzC1matEdu9ye2yaTz9YvBAvG49affzWTtmHZ0Hk2K8S5im9YUtmMz6YqbiG8LFitvIKpH4Ynd/rFXNh7NsR8zm3jqwZwk0W+kC5nXNc6VSHB+4R3RtbU+bThlL2U5oTlad3HtWRVkVGgGKOZ6bA6dlPi/kC7aRTcwyo4/dx65f5fuA2GI8KPaLdjSwrq4GDN+CDWBnWiUCuNeMK9Bu1Q8ZwB/a5ax+WfOpxHZl0Qb173Xzne/G0BmP1zMeH77UHqmHn7MZ5iS73Tnn/yiIR45ZCu/lBAj9ak1xCouLxHrAAIPYjQfdPLuccHtI4EJGtpCi6MZg413yPN9qjUDhCeUIFwCC+tcmrubaXSQ9NtxwOcEZQwnWMwhzxPPqMKUSWZa+pCQ+AR2mVXhDZ/j1k0gl8BCGKK0OI5C3I4dM6fZHA3CXyyXHIFFqXfL2kMxwpSxaXZdBwgVPhxbWKCkVBlcJGNheaAz8j5Oxnzl1aq3fp439BWaRDN0y4S1WkQhDSr2VVNw4nDBtAUS+PlJS+lwHMWFvq0jzlmmc3IQZSp3xJ03IM75wTk/xxCVV27n6qK7sKR2JJmE0isCD4N6l1hawAZ/c62FFwobVg05KP0iEMAAReNxf2EQtfexIzglzPoDZz6RUJRiZPRdiubx57APfgdHFLXCngknBClfYbpxFFOAuymJvmYwPjraesOnLCjOXtYzuslBEPHSLlRhSEWEMfIZyR5FHEBsvWDeZlYNsUGAONQ1d1hCvxi
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(54906003)(8676002)(38100700002)(38350700002)(8936002)(6486002)(6512007)(44832011)(83380400001)(2616005)(956004)(86362001)(6666004)(6506007)(6916009)(26005)(52116002)(33656002)(66476007)(36756003)(508600001)(2906002)(66556008)(186003)(53546011)(66946007)(5660300002)(316002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6141
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6c6a587a-c057-49a2-a4f4-08d97923be4f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	AFF481TkD4ZIYn/1ATyTbN8BiwmQG00/IGrTbowX6rgJWWUnrpjZi1GzjKeQ7D32o/fcgkF2mc0gWES6B3GX1x78zWuV1Un2nWsOmwvFKNd/zjkuA9eS90B6xFJTx/JlIEKK5S2qI/4jy4Pia3ecnb+nRulgIR4oGFbSBpypAwGxAooFVIFHSWD9zAVOZEfHxmBWSvzGhy1iriPbAPOi9+EQwUEMUDhf4v+dn2Em7Xi0dVo4FkT8SWAJwrJ1xV0QVvVS6ir81Uw6JVvVUXK6zLcIZ7bNTKo8Z9kUqixC7u3US067av7vJAaS9GFXmzGR2YaJZ+N6fCKe88d9ILpiQfIZmLPByHBsgoc3Pn80jKS3j47keaCTHhnta+mrlaWD9SZWd3Ou55NmfTxVaU3nN1zp4NUlr73fVRjc+QxcbqSByAlXlU/PjAT+lQ/89OiS5TVRqkMFH/I4KWR6SVqKXzfrYgozo11UFAVyeg4DElWS62IvDvilVKN6dWxq81oNNX3k1aBV9GQOX78T+f+n5NtFsMMm1J2e0rpOqpZOs9ypxNFY/fZr/mL7HoUaqjbiC2fFG6oo0WB+KgfzbcamT+cIXEZzkIrkbR8ZlgfW4nDILSeJbpLo0AJzFuToPG+hdXXR9e6q71F5s06deOy1HW0ep3qkHt0a5z9yHsaiZHPalNM229m2RoxpFtNCVs9HEyvS987MfnJ5IAkB47IcDuk5JbHSkcncj0FrrDtFVes=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(396003)(136003)(346002)(376002)(39860400002)(46966006)(36840700001)(2616005)(6862004)(54906003)(316002)(356005)(81166007)(36756003)(70206006)(86362001)(82740400003)(82310400003)(47076005)(36860700001)(478600001)(6486002)(6666004)(8936002)(6512007)(8676002)(26005)(6506007)(5660300002)(53546011)(186003)(83380400001)(4326008)(2906002)(336012)(70586007)(44832011)(33656002)(956004)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:08:05.6412
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 488a7de5-8727-4ee3-9b79-08d97923c941
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7153



> On 16 Sep 2021, at 13:15, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 16.09.2021 13:28, Luca Fancellu wrote:
>>> On 16 Sep 2021, at 09:46, Jan Beulich <jbeulich@suse.com> wrote:
>>> On 15.09.2021 16:26, Luca Fancellu wrote:
>>>> --- a/xen/arch/arm/efi/efi-boot.h
>>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>>> @@ -8,9 +8,39 @@
>>>> #include <asm/setup.h>
>>>> #include <asm/smp.h>
>>>>=20
>>>> +typedef struct {
>>>> +    char* name;
>>>=20
>>> Misplaced *.
>>=20
>> I was looking in the CODING_STYLE and I didn=E2=80=99t found anything th=
at mandates
>> char *name; instead of char* name; but if you prefer I can change it sin=
ce I have
>> to do some modification to the patch.
>=20
> I don't think it's reasonable to spell out there every little detail.
> You should also take adjacent code into consideration, making yours
> match. Issues only arise when there's bad code that you happen to
> look at.
>=20
>>>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_T=
ABLE *SystemTable)
>>>>            efi_bs->FreePool(name.w);
>>>>        }
>>>>=20
>>>> -        if ( !name.s )
>>>> -            blexit(L"No Dom0 kernel image specified.");
>>>> -
>>>>        efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>>>=20
>>>> -        option_str =3D split_string(name.s);
>>>> +#ifdef CONFIG_ARM
>>>> +        /* dom0less feature is supported only on ARM */
>>>> +        dom0less_found =3D check_dom0less_efi_boot(dir_handle);
>>>> +#endif
>>>> +
>>>> +        if ( !name.s && !dom0less_found )
>>>=20
>>> Here you (properly ) use !name.s,
>>=20
>> This is not my code, I just added && !dom0less
>=20
> Correct, which is why this is fine.
>=20
>>>> +            blexit(L"No Dom0 kernel image specified.");
>>>> +
>>>> +        if ( name.s !=3D NULL )
>>>=20
>>> Here you then mean to omit the "!=3D NULL" for consistency and brevity.
>>=20
>> I usually check explicitely pointers with NULL, is it something to be av=
oided in Xen?
>> There are some industrial coding standards that says to avoid the use of=
 ! operator
>> with pointers. Is it important here to do !name.s instead of the solutio=
n above?
>=20
> As you can see from neighboring code, we prefer the shorter forms,
> for being easier/shorter to read.

Ok

>=20
>>>> +            option_str =3D split_string(name.s);
>>>>=20
>>>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_s=
tr) )
>>>> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_=
str)) &&
>>>=20
>>> Stray parentheses.
>>=20
>> Will fix
>>=20
>>>=20
>>>> +             (name.s !=3D NULL) )
>>>=20
>>> See above.
>>=20
>> Will fix
>>=20
>>>=20
>>> I also don't think this logic is quite right: If you're dom0less,
>>> why would you want to look for an embedded Dom0 kernel image?
>>=20
>> This is common code, that check is not from my patch.
>=20
> It is you who is adding the name.s !=3D NULL part, isn't it?

Ok I think the logic needs to be explained because also the name dom0less
Is a little bit misleading, I will explain below.

Short answer for now: if there is no dom0 kernel embedded in Xen and name.s=
 is NULL,
do not try to load something or check dom0 command line because there is no=
 dom0.

>=20
>> Before this serie, EFI boot requires that a dom0 module was passed, othe=
rwise
>> the boot was stopped.
>>=20
>> This serie instead removes this requirement, letting the boot continue i=
f there is no dom0
>> kernel.
>>=20
>> However (as in the old code) if the user embed the dom0 kernel in the im=
age, then it is
>> legit to use it and if there are also other domUs specified by DT, then =
the system will
>> start the dom0 kernel and the domUs kernel as well.
>=20
> This doesn't match what I would expect - if configuration tells
> to boot dom0less, why would an embedded Dom0 kernel matter? I can
> see that views might differ here; you will want to write down
> somewhere what the intended behavior in such a case is.

I explain here my understanding on dom0less, this feature is used to start =
domUs at
Xen boot in parallel, the name is misleading but it doesn=E2=80=99t require=
 dom0 to be absent.

So if you have a dom0 kernel embed in the image, it's completely fine to st=
art it and it=20
doesn=E2=80=99t have to be skipped.

Here the possible user cases:
1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded =
in Xen image, domUs on DT]
3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image)=
 and domUs on DT]

Let me know your thoughts about that.

Cheers,

Luca


>=20
> Jan
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:08:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188556.337756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQszv-0007y3-1b; Thu, 16 Sep 2021 15:08:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188556.337756; Thu, 16 Sep 2021 15:08:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQszu-0007xt-UR; Thu, 16 Sep 2021 15:08:22 +0000
Received: by outflank-mailman (input) for mailman id 188556;
 Thu, 16 Sep 2021 15:08:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQszt-0007wf-UF; Thu, 16 Sep 2021 15:08:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQszt-0000PL-PN; Thu, 16 Sep 2021 15:08:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQszt-0006nb-Fg; Thu, 16 Sep 2021 15:08:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQszt-0002S8-F5; Thu, 16 Sep 2021 15:08:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NIXLT3EwRTE7NYyzfb34E9On+CB9t2D6ic8YaDRkcIY=; b=fVsKv6MHIjunUpRu2cNgdXFakY
	hmaUo+2RjYLNmpluIx6Y9FyBW62hhYAGky8vlbVMm16GghCcK+OUCfsM3XDEnkLC17VAXQCpSUl2G
	EDO+jYaEkglOdHYi1oBGBxquw49VK0DUnCwDfhJYM7XLmYg9swzNPFL9eO3MSPvreHdQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165007-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165007: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=f4e72cf9d665a8f1a54170b0b62739a628823c8b
X-Osstest-Versions-That:
    ovmf=c19d18136ef920e3e84f961e2a335a41147adcb8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 15:08:21 +0000

flight 165007 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165007/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 f4e72cf9d665a8f1a54170b0b62739a628823c8b
baseline version:
 ovmf                 c19d18136ef920e3e84f961e2a335a41147adcb8

Last test of basis   165001  2021-09-16 01:55:08 Z    0 days
Testing same since   165007  2021-09-16 09:41:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dun Tan <dun.tan@intel.com>
  Tan, Dun <dun.tan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   c19d18136e..f4e72cf9d6  f4e72cf9d665a8f1a54170b0b62739a628823c8b -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:10:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:10:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188570.337770 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQt1y-0001Br-FI; Thu, 16 Sep 2021 15:10:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188570.337770; Thu, 16 Sep 2021 15:10:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQt1y-0001Bk-BX; Thu, 16 Sep 2021 15:10:30 +0000
Received: by outflank-mailman (input) for mailman id 188570;
 Thu, 16 Sep 2021 15:10:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQt1w-0001Be-LR
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:10:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 39213cb4-1700-11ec-b60b-12813bfff9fa;
 Thu, 16 Sep 2021 15:10:27 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-ryzTmbsDN9ubuo782gcXiQ-1;
 Thu, 16 Sep 2021 17:10:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5743.eurprd04.prod.outlook.com (2603:10a6:803:e0::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Thu, 16 Sep
 2021 15:10:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:10:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0100.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 15:10:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39213cb4-1700-11ec-b60b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631805026;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=c8pPSKM5Q0L3yIcVNn8XWc4jzv3bY9pqdqLn7+Or50s=;
	b=fFcQHuQH3uGA0OVv0x7fkgdYBt8UZsS8h1Z0mPGCio3EY+/S+cpYotyqy4LYPcNbySdlgm
	+Eu5c5BveNbChcoEXG1hqOE9yk7MNnSUX36diZddbkkZUQPjc+DDhFeaV5iWHVa9JSsUT5
	sK8B0lqKXseipDTsruP6NPiGSZJ3vkc=
X-MC-Unique: ryzTmbsDN9ubuo782gcXiQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jg4bMAvZ0W6D7L0s2t00QUk2hLmF4e+fE7STz6bq/W+ZairtWDcPzV8hrfokN9EAukY/9qiI1+4LrGYVHRtY6t2NbvJ96cUpJetmXhq7Y2K8NsbnaeZVPDTZNn95i0c8w2w6qvn7uOD7nMqknMAGnckVeBjTgho+tLhZA7esG2ErGQCh/luXcFy4nHlSjtuhv5BsmLAnwdjQCm71yD7T3KRKjiJ62RNgLeYEE6AH4fNoMuj/iqpF9WHQJR7l93uNtj4wLbFL3A18EABSzPLNnRSCAJKCRmChgi+RaqupBCpKGWNxE7Qtf5PmWo2H+NcfrAuX3GYzrBvnEWuRVnMYIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=a97Ig6zm8T1jnMaVipdEfnwGiSbwMa8VsaSPBnHCBfA=;
 b=ACVBLmrnjBIEDztw1cCTUwNwasJBOJxB+CHzKtMdNVwP5BaqWUrzrHDLSZ/bIdY5VefEwF4SUQ0hv+RbYOqQTmF5vF06hTtlQHQ8IywfTwQ87D1C/kAOhD+CD9rfi0OB6IIdRJPmE8+2H1tCfBZlzxigYdnYMbqzhaf0TRzhsgqOZihfCl7y1kzbCdzyVjcbjbEwmn2SDbQKVWohafFrmibfnZ+JTclLp5Omk+80AtqcmWmK8pV0j4IKHZjKbunsI8k6Zwj3XkO14so9iDiHP81dLKaYHjDQoKCrMEuerviZrE218wqmwlVXpRHy4r7pxLk9TG5V0y1qENNHBPvDwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>
CC: Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
Date: Thu, 16 Sep 2021 17:10:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0100.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f54e0552-8518-46f8-e09c-08d979241b64
X-MS-TrafficTypeDiagnostic: VI1PR04MB5743:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5743B16181251A496CBCACF7B3DC9@VI1PR04MB5743.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xhNnTrLW5gEfk3iblEStHa/N/DbRru7//8R31f0kdTdJLKSxSPwZa+hDDdlywPqc00HG+0/Wa+J4AhBkhBsAL1jtF5N7/olakA7AwcLZHxG4vWgm7qRTA+kEhiCrFAkAqN/lAC2oxFXfVnpbeiSGOc/vrEX8ACNeEsQwYinrdJpg3hu+qGAGLSbRMtYeedbtxMRTTEMwdgyoojIVp25Rvu786PZOvSSDqBbyXsgoAvxuaalHllxJ7dvtqaxrCA4I0U/ZcyxGHtl3sVzcGPcNlm2p4UJKvN70fev4C+oc1fb9G/fyMu+4hHIT52Yt9NHGS99tuBE0rBAseIvzdI+8epokVgtoaES5i5l5SKjxvJzWc/9XeYROkY8XNOdvdruxLXMNIQQ+yXhWgq20ZwRr9mrm7dEt2LIQh+Szg7P12Ivp8HRfzzLiJxO12cnRR76YkcTsu9QfimqPpQr5HWghGfvTu8nUPKjG4O8NbL26Uk2XVHoHCsZzfSbwq9REmgSbTloQZir1QuqocqKiVcs08K9Rv/q+DxR/27LRxB0tCGI3PMxf686CCbEZbnpBQPlvQB3QzYmLjVS8JTE2L61s8q9Jwm0bxRZw1xyM1Z/Z42BFMphJxaSbC5ngecT9UYYAb+JGMPM9HBXEY/bKwDIpPIglPd9S0jPToFPcBI3yeoBSVB+4iRZQ3HcVgw9xipAcg3PZyarX72JAqS+9Mc7SdP7f+wvdqiyExFQ59xujnY+mgrqyW3c3hZZhzUvJlke+
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(136003)(39860400002)(366004)(376002)(186003)(38100700002)(6486002)(7416002)(478600001)(2616005)(956004)(66476007)(31686004)(66946007)(316002)(8676002)(5660300002)(16576012)(8936002)(26005)(86362001)(110136005)(53546011)(54906003)(4326008)(83380400001)(4744005)(2906002)(66556008)(36756003)(31696002)(32563001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?zlgbt9XGKlYqKr4DYXF70yzxNT4MBTdLdeYqgrhuMP/56jfE/YqvJYaUeQU4?=
 =?us-ascii?Q?5/SzRxp7K6u9wY9ivcjLuKGDGRBtM0V0WrH/y64Yi54fm5u/ErR75UrUllQv?=
 =?us-ascii?Q?N6fNxVUR/fowgbZ2cRSOp4chTiMsA0qHQiR4C3mdNkiY4O5vUrX2Vl9DbI14?=
 =?us-ascii?Q?oy7Fc+F5p2MCRkjcajc240SgU4E/NvNLmxCehSP66Cc0q9Q3xRsQzkpUJlcb?=
 =?us-ascii?Q?83NrGMj7S8CHKSH/9KMCy5rDT03pJOyX+9sapaemzgi6AxYjO4Qf3DGZlrSd?=
 =?us-ascii?Q?IV9apCP0tU6AylFJxYTascjoPwaXrEPKBcvDbyzgldUIxLEZghQyA4BbY8I+?=
 =?us-ascii?Q?jI/S7APmNt1r6DVs6/fDZQCCHj3ZN5TT8lmUYXfxRG3gSGHTwhml+GINE6pC?=
 =?us-ascii?Q?3H1UmhO7R10t8XBBFIMPDjIs/dNzz9uciqSuVLV/PxMB6c3s2vk1hJLYJdIg?=
 =?us-ascii?Q?x81WGnFWICO/q2/c7oePk1UcoqU9ErftHOg2Tx4nNdZql6zMJyHItG0/CzBV?=
 =?us-ascii?Q?io1LspRjkaNZRHy2JLPIzN53B5ai00bzF6e7yQMIcoKZGLXcJFtwWNPUkKSI?=
 =?us-ascii?Q?DR+FmTLRUk60v0mYDAAmmPxmwgm/QbigRrpiOFcxg2RpHWH3zzrJyCxfbiWA?=
 =?us-ascii?Q?6SofEpwoBSDctEZ7IEADxRVXPU0VEiAFT+wr5MPeFmzrNn3wWAQvlv4qidji?=
 =?us-ascii?Q?t1rvjkkZvVuEh9AeZGowI8Kk54PxxSq8+qDmYQJ4l/ZcyVIiS7Y0y/TjPdVi?=
 =?us-ascii?Q?pJF+vSqR4EbnrNQfCvAT14Kc9UuwO7GS6WGIt5G0sJbohj7mCjx+pbSeCneg?=
 =?us-ascii?Q?tJtymc8fFdHC6Ml8tTvg4Ldlj78M/m8ylOoh+C+4pKEzyQcEjQXI6l/Yy9xT?=
 =?us-ascii?Q?YZ50/qVNif49NoOO7TsMVwXwI5qwbblyL8UjVJcB9dYKdmOsGvGE2xPBC5DS?=
 =?us-ascii?Q?zy/hxEyMb/bf9IcyiYKeiqa4bYg6RqCh2158wzRdmaHZMMqvXTyKaN3Z+FvL?=
 =?us-ascii?Q?lawmS3UjDSoVz3CHMKvGPAKEfk/UUdqG++RnVLaRGa75jMxsFGiTz6PpUzLr?=
 =?us-ascii?Q?L6aMekRmDKMmOPlCjElsbV8K3OtUhPYdN1TT4WhetS8+GMA84GH8PkiEFCJH?=
 =?us-ascii?Q?IYGUxIf/Du4G+f8VAHuJEpRjBvLot9uV4C/I6IxWGP69omavhlT8CFQ7VMNO?=
 =?us-ascii?Q?tnpOc3J67kvMwAlb7XRsEI/OKvSpDGIa6iR0HMn4lpVEkiGMC0MiUXOrkxyS?=
 =?us-ascii?Q?uu4fUnLQ24Tzraf8cCMCxYx163e+ownLW7YOvS4zsAfEytyKphFg+r9SsSlx?=
 =?us-ascii?Q?kedtHQkNEnr7FTlf22k8henE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f54e0552-8518-46f8-e09c-08d979241b64
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:10:23.6294
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AUrDVi2hnoVYiDGP1fNQ8u4cj+P3DwpUWw0ZBLFPyeaenboUimK7uCCv/M6Nhem9uY1h9i+MDERQd4sPmOvGkg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5743

On 16.09.2021 17:07, Luca Fancellu wrote:
> I explain here my understanding on dom0less, this feature is used to star=
t domUs at
> Xen boot in parallel, the name is misleading but it doesn=E2=80=99t requi=
re dom0 to be absent.
>=20
> So if you have a dom0 kernel embed in the image, it's completely fine to =
start it and it=20
> doesn=E2=80=99t have to be skipped.
>=20
> Here the possible user cases:
> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedde=
d in Xen image, domUs on DT]
> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen imag=
e) and domUs on DT]

If that's the intention - fine. Stefano?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:34:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:34:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188579.337784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtOr-0003wS-GK; Thu, 16 Sep 2021 15:34:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188579.337784; Thu, 16 Sep 2021 15:34:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtOr-0003wL-Ck; Thu, 16 Sep 2021 15:34:09 +0000
Received: by outflank-mailman (input) for mailman id 188579;
 Thu, 16 Sep 2021 15:34:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQtOq-0003wF-Mg
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:34:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f74405e-1f04-4757-a459-ccc6a51b427f;
 Thu, 16 Sep 2021 15:34:07 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2052.outbound.protection.outlook.com [104.47.10.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-JjXlT15sMv2b-unXNYDlyw-1; Thu, 16 Sep 2021 17:34:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 16 Sep
 2021 15:34:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:34:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P250CA0010.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 15:34:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f74405e-1f04-4757-a459-ccc6a51b427f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631806446;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=QAU95lKgjP591NS7TknXuAXLyWzCj94rCuuPDnhasOM=;
	b=aKLP21yRZENbaVqL35IUGu/hfM201kRJ34JtUAtBDtySxFmK07Skpp31OhMXvOBvIyUAVk
	qRIreo4IFCpuaJgUanwiFgIW5uvOoSOGpSA4uTvT6TPKj9a6Vpz8S1L7Jrh9LwAL/RlUUX
	SIZ3IWTSQaVAsSOTLo5PlDFx2ovCPAQ=
X-MC-Unique: JjXlT15sMv2b-unXNYDlyw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KTe26F61co4YGXM2ZSzPvOPbW9gT5UPj1rAU7PcO9PxHLnCRVQ1aZkQGqjWHvRIhzzcqH2UPDgS4abQc5Dnp/yJwhCmIQChRJBAIu5cwdF7Ehv0GSA/HJo+i/aEyjI9oaYpaP4V893ZeBluwdjvIg/mgI7tubouvHczWhcR2GsqEoXzP3S6ErqIYezEEPzp6IlQqTJNJHnJ3aj4eRFYoSD1vfUIGCEIVpXLgbnZmd0db4ulq5r0WT6cWGreKOY9xb+Kz8Mle7BdWDiw1W5N9wOnKBLJfYi0wkrebqJU/kaXuAIS8M2bG1fv5I5HinP7fYvQ7/4XYeYvUBmU3veI62g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QAU95lKgjP591NS7TknXuAXLyWzCj94rCuuPDnhasOM=;
 b=VjpfrvsZd9eyyfhHbf4UHgg1d1IPIqIwQET14uG/As0RgmY7NZqaKktZBFCPg+WcMOxGzlGcCk3LAIwUWNiQnn+tSl4dywstX8VkwmkJ3eI0kxUK3BWDHxpzvXv+0Y5OMln+zM8FH53ChHdq6Abp7ASW0jGDJeH9Y2uusRDgDAISgiIYPZibFO+u0E8EWFEa7mA3sgiG21oP+64HWOQCO3iTmwgjG+dPGVAa99aKvyLFIeAQuV5JVn2bk80k+c5oD1uFf2O34ECOyhzh7yBO7VmPkCjKqpNhblabRgX5FXtNh/pjHab1Zi+otFtNX8WvjaCg2EA3ZNzPwdysqxxSwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v3] xen: rework `checkpolicy` detection when using
 "randconfig"
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210908111727.440265-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <bb40484f-fbfb-a206-fa98-0e927f5398fa@suse.com>
Date: Thu, 16 Sep 2021 17:34:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210908111727.440265-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P250CA0010.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:102:57::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bd46e001-b027-45c7-f5ae-08d979276a0c
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70399208AAAD76C7587574D5B3DC9@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/d+qxQ4/i39IzBwFAYbO8Y/6iXcLn5vzbBzjMuy31hksrkPFyvWzlaDOyPckQ6MF2xsvpx9seVcRVjqlS8vOpTJ6kWvYOWjOju+jXqvprHU31e/vzoynpQTsNtJTOmreHl+5WdYfqLrBZ373wSIL+9fAJsta0fG0kzzb4Sr5xaCfmC/XYk2P5CpCU/652M31DdwVeW+eec0EH+LCOVVPypURZ9AvPcJBKQXnoRsHKZgNQE1vFPzCeUPGAkAiuQeDobTqXP7tsx6IZsarJHddNL9y2l9RryfiB1UOOsOXHKcxyizOS3eHDzrmn+7uYTUDjshhPZiKYzflQ07tlanwN5cOPQrZZU7FQ/+uoiRagENx7VHG3u5njazCijLo9YJqOf5SM1zNdIhc2IW1jRZIag2AFi1augEVdJ/wdR6PRBBBRhLfQi7SNPUS//1kyMDGQQz8ztFyGbvWmRNGCaznpPl617VihMISZVpGks+n0K1ceO6ZeNglb7gi6AyGLWKYS2GAuqfh9Dbdzvql/7coAJENmOWeW0kiiKsIMGZg/CE5PnoIiCOakpRZS8j5gHclnT+DZEy3UTK8W2d9/+Y3TT45g1cEEIpVy4GD0Ss4oj6sDmhmFhfqh5XfopYEg7uteGL1DSRKcRK0u6mMIOOHPwyv1fAjVHh4C/dIMZHXedqUZs11MP2CPGxWuXtsrBcIfDbnC2/ud/8/GzHwqU26KTqxmCBtVCRlldvYERpjSq2y5rEGTS/1q/tvwXwMfg4H
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(396003)(366004)(39860400002)(136003)(316002)(38100700002)(26005)(66556008)(66476007)(6916009)(2906002)(478600001)(8936002)(66946007)(31696002)(83380400001)(54906003)(2616005)(8676002)(6486002)(36756003)(5660300002)(186003)(86362001)(53546011)(4326008)(16576012)(31686004)(956004)(81973001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?amM0bUh0MUxMUHIwU0MySS81TWN5MTVNVzh0cU1Ocm55NkhrODBnVTRDNmcr?=
 =?utf-8?B?ZmZqSWJMUld5akN5Ym10THFTQVkrSVFMSmJqRzVxMkVBRjZoOG1qc1JidDZP?=
 =?utf-8?B?ZVkwdWE2OGVBOXdwZU5DQTIrYlVJcnczUkcvUVRPeHN6eWlMT09zRjRWVHlE?=
 =?utf-8?B?Z043bzFQOEZoWmprTWswZU5zbUlmUjlFNXc3alNHejBhMC9IYTRGekQvcjJQ?=
 =?utf-8?B?ZTcyZnFJeGdWWFNqeis1akkzNnpWQUVWRmhDQjNrMXZwdE9GT1dXRG9HNUl3?=
 =?utf-8?B?bnJadjlXUm5IQkZDS0N5MHE3VFZoNHRPbjhNdVRJM3ZnaVVhNTVXOWs2L0dM?=
 =?utf-8?B?UTlXYkJUV0JjZXN5N05XWisweU1uWkxJMTc1WGxIZ0hpNW1wUmdHY1FFT0xI?=
 =?utf-8?B?RTRyUDk4RXlubmJQaG9FdEpGd0x6NzFNUFIyTzI5MnU4dTlZTXR1OUd5eW1W?=
 =?utf-8?B?KzFaZ3dQWTNJQmFLemJDTFJwNGJmdVp4TzIvS2xZK0FkK1pOaTROV1Azeno2?=
 =?utf-8?B?SlZheTU4T2dBQUJrQmJzaFozc1ZnenM4bkgweEs5MVhTcUZhZWhTbEhxZnd5?=
 =?utf-8?B?MVRNa2YvaWw5bDFIOVE1dDNkelJWRzlGVWZ1Q3BUOWNyckZBR28xVlR6VGZu?=
 =?utf-8?B?MUxNYWZ2S1hVV2F0Sy9pSWJUSUh0emU5c2RpY3hOVDh3d2hQbEFtTVVIMWR1?=
 =?utf-8?B?bnpKTlRvQXVzK1NXck1BNXJUY1JlUFE3VTFKSE5CNG5pZkxWUUIvM2gwOXp5?=
 =?utf-8?B?OUYvUThyS01temNpa1NKc3o3anJjeVBOa0k2eEtJbDBnV0kzWEtzeGplcFhh?=
 =?utf-8?B?QzlSTStrMEU5Z3c4aXdRU0tJQlcyNXJEQnliZ3RRWG5DTUpRUjZ1QTBkeHdB?=
 =?utf-8?B?QXVpUXhJbnpiK3ZlOVdCYnJlbFY5WGZFRGg3bTlDSjZLSUMxOXF6MFRPT2Fw?=
 =?utf-8?B?cGM4a1VtbVhmQ1pyWGNsNm9xYU5jbDlqREtmNnNsZjl1MXRlY00rQzRmUVlm?=
 =?utf-8?B?V1huUlhUbjhwV0NEa3lLNm1mV1o0T2EydUpNMS83bXFJeFVWTTlsbW5JNWtX?=
 =?utf-8?B?Q3JRQVZOY0xmN0hWQ0diRWxlQjhtZk5tMjFXeERVMnFIZXNycmppTDBGREpC?=
 =?utf-8?B?OUxVR1JpUEZoRHZ5QkhaNENrMjhkYzA2K1U4dFRvdG5yckNLWWFCTWI4dmNm?=
 =?utf-8?B?bjFIcUdWYVZnL2pvQnJ2bVJGUlUySGxieVhtcTl2c2hsTUtBTFVTMkRzUzFQ?=
 =?utf-8?B?TVJNL3BhK3lrQVFrMXVYelM3a2FvUVRTTlBWWGdxNzl0MkxWbHlXWlpoNmFz?=
 =?utf-8?B?NjNMQzR2THh1SEsrSCtHMG5laEVrQmtUQkFneFRpZTFWWXNSQm5OWVpxUFlu?=
 =?utf-8?B?Vy82QkpmY2dEeVdpZm8yK2ZuS0VUUTFhdk5oMmN2cE9yTWlQU1dUVVltUjhp?=
 =?utf-8?B?QUFROXkwNHVuVmNXcTNFK3BMYSs1RkRiWXNlWEJwYnVkblozR29jeklZS2tR?=
 =?utf-8?B?K0xTN2VqVkZGejBMZHRzb2RtQmZUc2U0czBkOVp0TG1sd2pMRzF0N2N1ekNm?=
 =?utf-8?B?c2pONFpmNjIyRTllRktWYW1CT3RIQ1lJc3A1TDd0ZkdOQTlSTE1rdzhiRTk2?=
 =?utf-8?B?OGlUVkNoQm5NdWJlTGlzeklKN2Rwa0s4cUhiMllHN0hpODhOK1NDdXBOK0pu?=
 =?utf-8?B?YXp6Z3dqaDhhUXRaSVFOVDlBcGFaNGxSTDNsb3VSUzJzc0xoY2ZOWEhVUzJk?=
 =?utf-8?Q?9jcQQ6FwJZFoD3a/L3dk+iLU3Fs+il/GBDAyIaK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bd46e001-b027-45c7-f5ae-08d979276a0c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:34:04.0584
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2jIOJQolnMW5Bqjd+Cnlbz4gxIKrzBtfZVXCRrH5x4FSF8c2yiSH23+0wSdi2mojW4j/U0Bg6nfLBsVjlZ/WaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 08.09.2021 13:17, Anthony PERARD wrote:
> --- a/Config.mk
> +++ b/Config.mk
> @@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=y
>  build_id_linker := --build-id=sha1
>  endif
>  
> -ifndef XEN_HAS_CHECKPOLICY
> -    CHECKPOLICY ?= checkpolicy
> -    XEN_HAS_CHECKPOLICY := $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xen && echo y || echo n)
> -    export XEN_HAS_CHECKPOLICY
> -endif

Is there a particular reason to go from XEN_HAS_CHECKPOLICY to ...

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?= $(shell hostname)
>  PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
>  export PYTHON		?= $(PYTHON_INTERPRETER)
>  
> +export CHECKPOLICY	?= checkpolicy
> +
>  export BASEDIR := $(CURDIR)
>  export XEN_ROOT := $(BASEDIR)/..
>  
> @@ -178,6 +180,8 @@ CFLAGS += $(CLANG_FLAGS)
>  export CLANG_FLAGS
>  endif
>  
> +export HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)

... HAS_CHECKPOLICY? As soon as things get put in the environment,
I'm always suspecting possible name collisions ...

> @@ -189,14 +193,24 @@ ifeq ($(config-build),y)
>  # *config targets only - make sure prerequisites are updated, and descend
>  # in tools/kconfig to make the *config target
>  
> +# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
> +# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
> +filechk_kconfig_allconfig = \
> +    $(if $(findstring n,$(HAS_CHECKPOLICY)),echo 'CONFIG_XSM_FLASK_POLICY=n';) \
> +    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG), :)

Nit: It would be nice if you were consistent with the blanks after
commas in $(if ...). Personally I'm also considering $(if ...)s the
more difficult to follow the longer they are. Hence for the 2nd one
I wonder whether

    $(if $(KCONFIG_ALLCONFIG),cat,:) $(KCONFIG_ALLCONFIG)

wouldn't be easier to read.

> +
> +

Nit: Please avoid double blank lines.

> +.allconfig.tmp: FORCE
> +	set -e; { $(call filechk_kconfig_allconfig); } > $@

Is there a particular reason for the .tmp suffix?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:44:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:44:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188586.337795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtYP-0005YT-G9; Thu, 16 Sep 2021 15:44:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188586.337795; Thu, 16 Sep 2021 15:44:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtYP-0005YM-Cw; Thu, 16 Sep 2021 15:44:01 +0000
Received: by outflank-mailman (input) for mailman id 188586;
 Thu, 16 Sep 2021 15:44:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Js+i=OG=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQtYO-0005YG-9B
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:44:00 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d9c1c681-16b6-49bb-b6f4-a4497e299896;
 Thu, 16 Sep 2021 15:43:59 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id h16so19944872lfk.10
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 08:43:59 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id bt38sm288648lfb.269.2021.09.16.08.43.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 08:43:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9c1c681-16b6-49bb-b6f4-a4497e299896
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=1HZG8pKuU4LzdX28wFwGbG7vSzzx3Nkf7FVz0NUnbzI=;
        b=i31nNwlE+/pBQHd4grJUqb1KYMA2e5jpfRtlXmUd4i+vasnAtRSI9acC4hWecitz9B
         bREmI20Mt8QGD6DSw3SjrtH6wkM0sxlbZZJVD9jvYaM4ROunN0+3vuMQqvXDcfelWb2+
         n06pmOmVmHgQwhYZu0gl/N8OjOBhZzn1hvXf332vU9h8WbtoXMC+7CNURfc+8udP9l+u
         jSWKSNMi28yMwex9yq9lJ333I0NVYMCAQsOQcwq++Tn/z4Ie9ZFiYxHz/tshBUP0R1+D
         MtevEXXNx+gUpTkp/5KU8lk1FyzY8cJcWG+op2G8MmbSHkDh/TxiEQxXAi24YuIFrsjT
         r/qg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=1HZG8pKuU4LzdX28wFwGbG7vSzzx3Nkf7FVz0NUnbzI=;
        b=JYdkPfa8hk2rrt5FKEd8njReXmmBfljyU5QG2Kx5CCmxiJys9PGYH8MGTnmi8m6l9h
         4HLcDUhXSOVrtf58KqwPhZislbha+OdNSFj/7ePaAwVky5Dyz8JlmLlFH4usNj7tsknX
         NdcVVwWGFVWpuPOwcqVFO/pS+uwkOXGwS9kNBDPybBPDkouzKw66RQuSCOJpAHNPXgtN
         +W+di+M8CGt6R2MUMS2PEMNGzgEK+B4aJy5J+Z1sXfMOHxEvYfR43zxCe3VdCFiqdH4F
         NmMyAochIWYS0k6Q7bi5UCuWgq2K/PPi2cAY54FWPXSPBJi6TZJQHel0SwKO8h6UiylC
         KIag==
X-Gm-Message-State: AOAM531IEQOzjC9NT0mCOXiIL5mcPX1VvMedZDTj9oXmFzYMgsvFkDXS
	Y+JW9Rk3tmfpmPs60nINg8sn13m8Mnc6qXtb
X-Google-Smtp-Source: ABdhPJyO5pUDjuV3+yyLfujA1izuJPFWJfhJxVD+8NQNhPnSkN7u9nBkKHqTUKyhKzDrsmfpnIgFfg==
X-Received: by 2002:a05:6512:228f:: with SMTP id f15mr2606198lfu.253.1631807033285;
        Thu, 16 Sep 2021 08:43:53 -0700 (PDT)
Subject: Re: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
 <jgross@suse.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 xen-devel@lists.xenproject.org
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
 <03b1a1bd-2630-0ee7-b995-114632f527aa@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <6b1c715f-e71e-718e-3cde-b298a5a38807@gmail.com>
Date: Thu, 16 Sep 2021 18:43:51 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <03b1a1bd-2630-0ee7-b995-114632f527aa@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 16.09.21 17:49, Jan Beulich wrote:

Hi Jan

> On 10.09.2021 20:18, Oleksandr Tyshchenko wrote:
>> --- a/tools/include/libxl.h
>> +++ b/tools/include/libxl.h
>> @@ -855,6 +855,13 @@ typedef struct libxl__ctx libxl_ctx;
>>    */
>>   #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
>>   
>> + /*
>> +  * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
>> +  *
>> +  * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
>> +  */
>> + #define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
> Nit: I don't think you mean to have leading blanks here?

Yes, will remove.


>
>> @@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
>>       uint64_aligned_t outstanding_pages;
>>       uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
>>       uint32_t hw_cap[8];
>> +    uint32_t gpaddr_bits;
>>   };
> Please make trailing padding explicit. I wonder whether this needs
> to be a 32-bit field: I expect we would need a full new ABI by the
> time we might reach 256 address bits. Otoh e.g. threads_per_core is
> pretty certainly oversized as well ...

I take it, this is a suggestion to make the field uint8_t and add 
uint8_t pad[7] after?


Please note, these are still unaddressed review comments for the last 
version [1], with a suggestion to use domctl (new?).
Also it is not entirely clear to me regarding whether this field will 
even remain gpaddr_bits or became maxphysaddr for example.

[1] 
https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/


>
> Jan
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:46:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:46:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188593.337806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtaO-0006AZ-T4; Thu, 16 Sep 2021 15:46:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188593.337806; Thu, 16 Sep 2021 15:46:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtaO-0006AS-PZ; Thu, 16 Sep 2021 15:46:04 +0000
Received: by outflank-mailman (input) for mailman id 188593;
 Thu, 16 Sep 2021 15:46:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQtaN-0006AM-I5
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:46:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2740aa18-32a8-49fd-88c7-7de7b9e13caa;
 Thu, 16 Sep 2021 15:46:02 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-8-57pnKs7GOdC6qvlBTufvuQ-1; Thu, 16 Sep 2021 17:46:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3934.eurprd04.prod.outlook.com (2603:10a6:803:1b::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 15:45:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:45:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0035.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 16 Sep 2021 15:45:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2740aa18-32a8-49fd-88c7-7de7b9e13caa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631807161;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0yDVeRNlDDrwVTmm+uLJzYWe4+HRUO5u12ZGOtIo0O0=;
	b=MFXTi36bETzJrndYXclggUrSb/Ko+VvAZ6LsAxaWBVE240BvT0NUzhMyVl42YbUm8FB73C
	YExH3L7pS0Fb6cTJGCxN/J55nBc/MloetdqNIEJj/jihIWrh2BINVduyqb69Me9eegofUg
	nsjZDqrSPSRuPalc8KngOeHklDwNqcc=
X-MC-Unique: 57pnKs7GOdC6qvlBTufvuQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jpiQ8Vd4nU83PtalOUyyEHBeQTMw1MbVGJS5QrMHL3Y7nkVl3IMVpAQgm13wq3D64FuxmMdctFzRjXkV9PmLtzuBf17zcrdWHnuzUGz1hUvFn9M/OAgmLufDu9Rznv9Q++PO8NBNKviaxDhIa/RpG3kSXB7mHKJCzaVjamlYSLtXEUqHGNK6Scw1Yfbzbj40rTkjPyQ/isRiooEEDKWHxUOmCDEMObjvWIW0SLTcjekqIikDXi8dJstu2aGLZbCD2iFkLFSD9obDwC0cFUGY75+aVs6FbOV4taRdWizE0jqpl1v5JLdQ7vEYnBvY9ji/477a1qc4wSYtQDZjiH+9lA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0yDVeRNlDDrwVTmm+uLJzYWe4+HRUO5u12ZGOtIo0O0=;
 b=STkNGivzQfVM535X4cUbeQBbvMP4Ip3tJb9UrMyLdkD9X7CgHU/3/VK9ggKySdRROumP79+U+LRZYyEu+FOMAta82sRUhlcsKgq8IaPguQPBJTLOteJ6dEGd3D3pB2fPKZbwN4SrqVazRIwCDkO8xWzKQnDguOcWj2Qlw8s6ABLnR8xGGJifJ4UfgTGgoIM9oImfpQ94H7gN9a8CtaqUS6LU5PAJBSxYBZNKcjgu7X9kTtrSbipGAGZr2vp3x1+Lp0YZH0hLbZJvIw9XKKPXbSLfellxG7MLqkWc9xxlcIlHmfHDPmt024N3cmQSnzR5/dog0HHH/cNP7n/AJA8W6g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: =?UTF-8?Q?Ping=c2=b2=3a_=5bPATCH=5d_xen-netback=3a_correct_success/?=
 =?UTF-8?Q?error_reporting_for_the_SKB-with-fraglist_case?=
From: Jan Beulich <jbeulich@suse.com>
To: paul@xen.org, Wei Liu <wl@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>
References: <4dd5b8ec-a255-7ab1-6dbf-52705acd6d62@suse.com>
 <67bc0728-761b-c3dd-bdd5-1a850ff79fbb@xen.org>
 <76c94541-21a8-7ae5-c4c4-48552f16c3fd@suse.com>
 <17e50fb5-31f7-60a5-1eec-10d18a40ad9a@xen.org>
 <57580966-3880-9e59-5d82-e1de9006aa0c@suse.com>
 <a26c1ecd-e303-3138-eb7e-96f0203ca888@xen.org>
 <1a522244-4be8-5e33-77c7-4ea5cf130335@suse.com>
 <9d27a3eb-1d50-64bb-8785-81de1eef3102@suse.com>
Message-ID: <d4f381e9-6698-3339-1d17-15e3abc71d06@suse.com>
Date: Thu, 16 Sep 2021 17:45:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9d27a3eb-1d50-64bb-8785-81de1eef3102@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0035.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:20b:150::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a4d0b0b9-5f10-4f63-1b80-08d9792913d3
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3934:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB39349771094D2911C128FFCFB3DC9@VI1PR0402MB3934.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8+YwVhZxphC0duk/+RGgfSD/sqCUsnRQf329P7WCoXMj4/o9/9Hlf7BeCdFQp7wn9N2/LsvsxaqL8A82JR0pVDDiEQlBBULCW3wAeRv7sjdf8Ni7NdV/2xKCjTCwBHSrM62jj3th+KFH108ftH5x+/8kcKeWAAzKvgkkl8CILufgm6HsLgYezaIaEc8w8y5tA/teR5MFtk8Y+dCwhit8e5DbIqtDkf4XLfPlG3QS/V7Tu+wxOzXXYCRHlYKdrv2Imv01YTet6pGaRDi2mTupdh+bLBezH1cM6pbDFWjMm8oms488VDOPg9WyuFdJNZuMfN8m4D91FSnk7zsX82tMQylU9NDW8HTF72s+FIs51hl9F7gAnmxcAqkYLYXAq9Dwa2rZzBB0mkypTxXhycpm7pvOe/YZycQlbusPAosyFWHe5jJHRCGUMhMw9Jp9l+MxKiHgvIJb5qW6QTS5RTK+uR0zq2TImUdfz6Pagup+gHZBZYz+BuTkZiqho536iKZireEdZh6W9dE+pNGLirGDAOYNKU3kgOXPOlSMA3ZVgYx0VaLsN4+eHSldu7vIK4idY7XguANawErLYQIaLmVoTDAfcUrw13aZIkwsuqVCYrYzlg62o/7OTe2+zLwojJrex8bSoTCRyhkH8gmNhJ56nosawWdm0PVD6qKva1BZweidmVDkczWGIzt1KXtExA9BMoWqguTEnvAMmq+sLLRwoOR+9uKCrbzmb9W9r8lYF4w=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(376002)(39860400002)(346002)(86362001)(66946007)(53546011)(66556008)(31696002)(66476007)(31686004)(316002)(16576012)(2616005)(4326008)(478600001)(186003)(5660300002)(26005)(38100700002)(956004)(36756003)(8936002)(2906002)(6916009)(54906003)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z3k1emhoVTBSTDdPOFRCclUrRUNpa2tneS9GSUVnYzBlMjF1cTl1L2lNTWVU?=
 =?utf-8?B?aU4rMURwR0pIL1ZLeGE0NGRuZk5LK3poWUJuV2I1OEVXaXJIbFJaV21aT1Z6?=
 =?utf-8?B?OFk3bkhZZjdYeklOaDBqWGRvV0dkK1VLdTFKaEs3TkRUQVFnVjRzdUxxQndB?=
 =?utf-8?B?WXRSV3ZwQXhIT3liQnJjUXJXN0xsNWg5MXBDSlNhVEdzc1NCSG8yaU5IRCtY?=
 =?utf-8?B?SkdtMFVBdlk1N1dhUzIwbFcwVENuY2I3MWIza0d2TVdTa0ovU3JrekFPUjRr?=
 =?utf-8?B?YlNXNXk0akNLRmY4d3lEU0x1ZHlSZU1zQld0VUJKU2h1NzFZMTFkWWFqWGww?=
 =?utf-8?B?RHF5YVIwOVNWSW0yK1lQQ1V3LzhMd3JPWCtwdmQ4SmsxZ2N3cGtXTi9oNHEx?=
 =?utf-8?B?S1B1REwxZVdzZHRjQ283WXlLSXBMbm43TmRrNFpHWjZOZkI3NjBHYlQzeVB5?=
 =?utf-8?B?bVhjdVVMZWFGSUpxRCtOTTFXbWhqMEdQYTMrVGZNRW9TRGl0ZjNTcDBlVm1G?=
 =?utf-8?B?Qjc4VkhLOVpobmk4d01BK0hmdE5JYlh5d2ZyMWFZOFN6dk1VTHhlZUVlUk9G?=
 =?utf-8?B?MEdEQWhQUFB0NG9GZmRheUVnMGJCZjh2azAzcUFlbHJ5MTF1YjFCUmdTV0l3?=
 =?utf-8?B?c1RIc29HMDZubUxDZTc4VjdDbU93U0lQTDgweTRWbjdONXlHaTAzVmdoNlV6?=
 =?utf-8?B?ZEJEanhmQ1kreFUyVmVaU0w1TmFmL1N4K3l6Zm9yM3krRnlURkxFVnVFdStj?=
 =?utf-8?B?TFB6OU9zSHpCTStqbXpLSkxNTnNUQ0NvSWpFeFIxdmNXUGZEekZoQmtoUzk1?=
 =?utf-8?B?QW9KVEc0TnZ6N3pDWGR5ZGJpbjFlckRPQVJkMmNnMklCYTJJN3ZJaGF2Tmhp?=
 =?utf-8?B?bkp3WXF5M3IxTjRGemF3Z3gzcFp2TUl0SzNlRFRzRlJCUjlZT0o0TUdZV3A5?=
 =?utf-8?B?M25JdXRJanBOa1UvQ0hQWFRqRS9OcDRna3VOOU0ycTRTMlJQL2NpZUhZcDVt?=
 =?utf-8?B?TmVHWS9mMlF5K1lob1d6UExDQlE3WVVuMkRmTE1iOVZFcndpTlJESWlLV1lM?=
 =?utf-8?B?QVM0YWNQK3R1NzF5Mk91UVZyOTZ3dmZocHc5bGFVZVBvUENQNnJDT2VWYXov?=
 =?utf-8?B?L2hRcklkT2NseDZNVm0xVGV6WlAwMW90YVl3TWNrUldsRDIrd1VTeFBNaG84?=
 =?utf-8?B?NXJzeDN5Ry9LRXkrR1ErZFpESjZzZWlHZG1idnVBRGRCOGM3OUFSZ1MxYnc3?=
 =?utf-8?B?N3ZGM0JWd1lBOWxjSVl1aFFlQU5uYnJQdHg2WERlMS9jYmEvZGZnQlFGM2dX?=
 =?utf-8?B?NnhCMExvaVBnZnEvV2wxRmpmLzZZMGl6MmRmTG5jcklwRVRVa2M0dXF4eW1S?=
 =?utf-8?B?TkNFM29JeWNHaTd2NGQ3STR5bUxnK0VNMW02bUpxUXFBcDE2aUJZakFyK21p?=
 =?utf-8?B?NVdBMHQyUHIvU0R2alpGNmFyMko2VE9za3hxNHloV2hxYTg3cGlFTXM2Mi9S?=
 =?utf-8?B?aUJRYStCeXN0bjMzUXlldGpReFBvNTQ4KythVllWUmVsdUVscmRaaFJnREVz?=
 =?utf-8?B?dmQyVnFrR0xJUU1Zbzc5RXQ5aEEvQS9TWk5ocE8waUdKZ05ZUWhia2ZkVWJD?=
 =?utf-8?B?cWtqdnNuY3dNbmdabkJHbzF2SXNMUUdVWkJZRWlzTTR5MWUyd2Z5K2cvMFZ0?=
 =?utf-8?B?THFKdW1ZeHNuT2Z1d0FwWkNQOWhaOFptWTBhTGtjOHlwTDhoUzBsWWxMbVlh?=
 =?utf-8?Q?xaCJirdazdFrPhWAeeYatG702CahhD/CYdFGjds?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a4d0b0b9-5f10-4f63-1b80-08d9792913d3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:45:58.4087
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Nc6agaP1m7mYXw++eJsHXhY0r54exClQIvAdsMSGacO5FA70+DRixO8Bw1hcFlfyAET+kQvfxLcI3kABDKi6bw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3934

On 15.07.2021 10:58, Jan Beulich wrote:
> On 20.05.2021 13:46, Jan Beulich wrote:
>> On 25.02.2021 17:23, Paul Durrant wrote:
>>> On 25/02/2021 14:00, Jan Beulich wrote:
>>>> On 25.02.2021 13:11, Paul Durrant wrote:
>>>>> On 25/02/2021 07:33, Jan Beulich wrote:
>>>>>> On 24.02.2021 17:39, Paul Durrant wrote:
>>>>>>> On 23/02/2021 16:29, Jan Beulich wrote:
>>>>>>>> When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
>>>>>>>> special considerations for the head of the SKB no longer apply. Don't
>>>>>>>> mistakenly report ERROR to the frontend for the first entry in the list,
>>>>>>>> even if - from all I can tell - this shouldn't matter much as the overall
>>>>>>>> transmit will need to be considered failed anyway.
>>>>>>>>
>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>
>>>>>>>> --- a/drivers/net/xen-netback/netback.c
>>>>>>>> +++ b/drivers/net/xen-netback/netback.c
>>>>>>>> @@ -499,7 +499,7 @@ check_frags:
>>>>>>>>     				 * the header's copy failed, and they are
>>>>>>>>     				 * sharing a slot, send an error
>>>>>>>>     				 */
>>>>>>>> -				if (i == 0 && sharedslot)
>>>>>>>> +				if (i == 0 && !first_shinfo && sharedslot)
>>>>>>>>     					xenvif_idx_release(queue, pending_idx,
>>>>>>>>     							   XEN_NETIF_RSP_ERROR);
>>>>>>>>     				else
>>>>>>>>
>>>>>>>
>>>>>>> I think this will DTRT, but to my mind it would make more sense to clear
>>>>>>> 'sharedslot' before the 'goto check_frags' at the bottom of the function.
>>>>>>
>>>>>> That was my initial idea as well, but
>>>>>> - I think it is for a reason that the variable is "const".
>>>>>> - There is another use of it which would then instead need further
>>>>>>     amending (and which I believe is at least part of the reason for
>>>>>>     the variable to be "const").
>>>>>>
>>>>>
>>>>> Oh, yes. But now that I look again, don't you want:
>>>>>
>>>>> if (i == 0 && first_shinfo && sharedslot)
>>>>>
>>>>> ? (i.e no '!')
>>>>>
>>>>> The comment states that the error should be indicated when the first
>>>>> frag contains the header in the case that the map succeeded but the
>>>>> prior copy from the same ref failed. This can only possibly be the case
>>>>> if this is the 'first_shinfo'
>>>>
>>>> I don't think so, no - there's a difference between "first frag"
>>>> (at which point first_shinfo is NULL) and first frag list entry
>>>> (at which point first_shinfo is non-NULL).
>>>
>>> Yes, I realise I got it backwards. Confusing name but the comment above 
>>> its declaration does explain.
>>>
>>>>
>>>>> (which is why I still think it is safe to unconst 'sharedslot' and
>>>>> clear it).
>>>>
>>>> And "no" here as well - this piece of code
>>>>
>>>> 		/* First error: if the header haven't shared a slot with the
>>>> 		 * first frag, release it as well.
>>>> 		 */
>>>> 		if (!sharedslot)
>>>> 			xenvif_idx_release(queue,
>>>> 					   XENVIF_TX_CB(skb)->pending_idx,
>>>> 					   XEN_NETIF_RSP_OKAY);
>>>>
>>>> specifically requires sharedslot to have the value that was
>>>> assigned to it at the start of the function (this property
>>>> doesn't go away when switching from fragments to frag list).
>>>> Note also how it uses XENVIF_TX_CB(skb)->pending_idx, i.e. the
>>>> value the local variable pending_idx was set from at the start
>>>> of the function.
>>>>
>>>
>>> True, we do have to deal with freeing up the header if the first map 
>>> error comes on the frag list.
>>>
>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>
>> Since I've not seen this go into 5.13-rc, may I ask what the disposition
>> of this is?
> 
> I can't seem to spot this in 5.14-rc either. I have to admit I'm
> increasingly puzzled ...

Another two months (and another release) later and still nothing. Am
I doing something wrong? Am I wrongly assuming that maintainers would
push such changes up the chain?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 15:47:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 15:47:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188602.337817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtbs-00070s-BK; Thu, 16 Sep 2021 15:47:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188602.337817; Thu, 16 Sep 2021 15:47:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQtbs-00070j-8N; Thu, 16 Sep 2021 15:47:36 +0000
Received: by outflank-mailman (input) for mailman id 188602;
 Thu, 16 Sep 2021 15:47:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQtbr-00070b-JZ
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 15:47:35 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 74df0400-818b-4010-a7aa-886f30a48d57;
 Thu, 16 Sep 2021 15:47:34 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-3-3LExGeY_PzaXhva-uZ9fJw-1; Thu, 16 Sep 2021 17:47:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3934.eurprd04.prod.outlook.com (2603:10a6:803:1b::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 15:47:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 15:47:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P250CA0029.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 15:47:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74df0400-818b-4010-a7aa-886f30a48d57
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631807253;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+ODLk6buZ5bRYNHDO5mkilEfQa+D2sDqx44UesSdg94=;
	b=QkC7+i+OQcniulVjZ8PI0ZSiGUrEqikur/7oCFRIIX7uF6xYuV0mJE+J8+RrQ6Ws23JgP+
	ootzaQ+II068V3y9/Iq+Q6kTBEdkxgYDsTvbTFz4cznhbHeFf/hy1XL2pwBk+Z3tWI1PFW
	MRCwVyQUjFXE1QheDf4MDClcWqX1DXk=
X-MC-Unique: 3LExGeY_PzaXhva-uZ9fJw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IhBdAfDgxlYeQczniHLpSAKw1o9LTK5KPZTgl7r3Nfy8CGv/w6qCrdGvS0WOtP4ufnhbFoBkuVYRTL8yo3uzOH1OGBYHzYrxxeg7UeIzfURm3hOP5ZUm35SFFGD3trPqJtQ82RWaMWqTPvetS63Z02WzLzejPhYH+WkTnMhXhndNl5fSE03/kmwQlgvfB8YwwrEUSDFKB3PAVuEV9EE8b52fWCL0rKmBicHyJ1bhN1BIqfR/LaIgrJC5vGh+wpkFLCFwinnPolKVgq2UHiqmaBWJRdsuGy5NQTtl5SQWPLbWzLgdJedJbHaOrrBpg7gQaf4jRdIs1kwDP1WWmXW1lg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+ODLk6buZ5bRYNHDO5mkilEfQa+D2sDqx44UesSdg94=;
 b=nn/6/qAEQ271143Xq51MqZRxtT9Py2SZtSJipc216OjnMop26oo2+9XNlKXLA/nFqzK0Py3Tq2UPKJ9OhVUCSUrYHcmXU7x9sQU8vA3Zn6HZdaNVUCxkV1xzUxWmn2l+1UolxjeNF4HXpjATW3lcrnNYPZ1Rzu3DgKJLQrsq7Bv85N/Gn5T1ltVjuiD8MLco3kts94GfDu1yNmETp8q10y6b7kIwqQcfsf92qqqwFEa7FvRCYReB+8ZCte841wFNvI/C4D5wLdaQGTkdrkIGVWNXfjka8FuJg5YnulH9g+l5jwi7YpEk4OukgNw/fyKPYwRk98EjHOXp2q2AwpK0Xg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Oleksandr <olekstysh@gmail.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
 <jgross@suse.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 xen-devel@lists.xenproject.org
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
 <03b1a1bd-2630-0ee7-b995-114632f527aa@suse.com>
 <6b1c715f-e71e-718e-3cde-b298a5a38807@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ebb05297-b9ae-c475-b2f2-4c06e986f99e@suse.com>
Date: Thu, 16 Sep 2021 17:47:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6b1c715f-e71e-718e-3cde-b298a5a38807@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P250CA0029.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:102:57::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 96b8fcb9-5045-425f-9966-08d979294af5
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3934:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB393480201CECF66BA87F6E37B3DC9@VI1PR0402MB3934.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	D7IsiVTi03NSm1sKEs0fWo4rD6N7hMjzS6xq4qnjehckww0XGOk2aOGaJUg6TG6manNcDWaOogcNBguihzWL+TdQF3v87yyO2QRFJWeJc455Oy7cf1Pe3Lr/sca2mYFRsO5Y1FzsBLNX/4PlNJR705c3Q8xlsaTyJdy2ibndefJBlsva8CCFyPkC/690xpe4FoSOY8yIQCXkNx3U5jnqDOD05iJkvPFjrLBiG/fm6hXVTcyeSuKIMRkeiglGJBtRn4BuODwaDfTEf4Ynuk2/FhutoqVWMhgkwoHpRdP9xKa/dorPc4SgpGp/QHZt+x4TC2BxzrTyAnbosvcz6eEvl633DsvXBNG2EaArc4w8c/iX55R3sb+HPTAl/CjQncp2SOHqKQSoe2IAOi+XWMjpDVRPhgHZokD+QnOjQne1mLzzXQGQm2+1SdETox0CONH7/6b9OajjmI6kmNMeOBrjo7cSkXYOJoEjaVe3jGu54uBlNOc9CJS/CXqKCJ7oAy9ZVZ4HMGaCc2bS5JIF3pL8Q6mu1No0nCYCOwA0BNiHe6dMH3TpPrZdMDMR3Uv5B927zxgEUYzWkVy6tCflBzaGMiV0cYhoC89DxCVwzJmSgr9iEZS09MQ6ELZIPtqrwB91ctUfNPcCSyW/6IwZ+ZdpTiqD8KLcDFnIP0zzA5PvADWpc0piZeehBdDvld+A69KUVpJF3I4qXy+cs5wj4MZ8RTH4MOuFflTPv4jC3//xjZs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(136003)(376002)(39860400002)(346002)(86362001)(66946007)(7416002)(53546011)(66556008)(31696002)(66476007)(31686004)(316002)(16576012)(2616005)(8676002)(4326008)(478600001)(4744005)(186003)(5660300002)(26005)(38100700002)(956004)(36756003)(8936002)(2906002)(6916009)(54906003)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WHVGR2Foa0JQVzl0M1U3Y1FkL1pRK1pRZWM2dnBDM3M3VHFwQU5JOHF4cFhz?=
 =?utf-8?B?TnluUldZNmRGK2h5ZkRuT291TEJVamsrd2xjTGJVMU9jckZHaTRHVDZ5djFq?=
 =?utf-8?B?eDM0RTBDTm5Fb2tZbDZzUkZYeHdBdGFVQnhZa2xQR3p2TFNNODNmeFEzZkR6?=
 =?utf-8?B?WnYwVmxIL290TzdaN0d6dCs4ZnJGQ0lxc1lYT1FUUFpJams2b2MrZGhKdjNP?=
 =?utf-8?B?NnAzbllKaGtPY0E0VTVuVnVnZW1RWG9ITWVzUUlkYzJmcFBGd09iS3Rkd0Fw?=
 =?utf-8?B?K2MyYkxuUmIydTVyWFJFcDNObmVzVGp3UCttVXJPZ1l0RmZydFZ3YVRodXZR?=
 =?utf-8?B?bUMrUnh2L3NHWFo5bDgxbnpuSFY2eThqeERGYkQ4eWsyODJvZTJ6NUR6OUpj?=
 =?utf-8?B?dXplTmpxU3ZPUDVZeVhWd2xFdmNzZUlXYkV5dVppczl6VTRlaTIzcXg4d0to?=
 =?utf-8?B?aWJoNVFTZVo4OWcwczVsV1FFVC9JY2xmalhKUjh0SFhCTDdhMnl5L2FGbENo?=
 =?utf-8?B?WVVNT3lhTXIrbTJpQ2VTVDM1dldtT29KZk1zRmZTYklLM21rc3dnMUQ1RUEw?=
 =?utf-8?B?RTdUQmFBVm1PQTBEWWtJOXB3QjkydGFQVkc0eDJyeXQ0bWliaTZxWWN6Rlhl?=
 =?utf-8?B?cTVSTFErTmpDY2VSY0MwVTJtK2FVbUZxRk5UcEcxMnFxQzFhTVRnbVhmT0RF?=
 =?utf-8?B?Y05kaDFNS3NSTjZlcllXUm1SUG1LaVFhdkRKRjNXZHJFWGVXT2JtczJoVFVX?=
 =?utf-8?B?WVZsR3NLUmdFUG0xY0F6RWphUEdnMnpPZVE0TFZzb3ArMVk4VDBsR2h2SkdK?=
 =?utf-8?B?eUIxb1lLVHlvSXVON0ZrMFVHNnI0cTlGOFk0d1FhV3dtRGNFZnBJbkF0Zlpl?=
 =?utf-8?B?Z00xeVM3WGoza1NCUWc3OXFUd0hiOTNLMW1FZDE3OVhRZXZ2cURlSUV6dkIz?=
 =?utf-8?B?QVNncXhwZDRkcDllTFlRVU5uRjNKYStPekRCNXNYenlaK3NocW9lNFRpVW84?=
 =?utf-8?B?bCtsd2ZyUStLK0lWQXJpem9UTEg2OFl0b0xzK0RuaUtzd1pKbEJnT0ZBK2lG?=
 =?utf-8?B?b1I5S3h6YURnZTdhRHBXNWtobE15enRrcmtwdEwvZDBSeEpiZTJIRHVHcU8r?=
 =?utf-8?B?RDczQWo0WlNBV1l2QWE2L01NT2grZmJWc2krdGFVRWx4VnpFWVlvUjhINHZC?=
 =?utf-8?B?VWNkSm9xQzNZMUxjSThXT3NqRjVBTzh4RmQrclpuT1JIdTBPc3RXRlFDUW93?=
 =?utf-8?B?dzJ3UW5taDA0UFN1L2pleWlORU9XRm43YlZHNzN3TnE1SGFUTXZNVGhPM1h3?=
 =?utf-8?B?YVBaUmhrWUVpNExXUzRWQ0xqVnE0RHlpWFVEWDJzUCtPRkRSZGpURFdycldZ?=
 =?utf-8?B?ZHRaR0cyUCtKVXNXSElqeTRoZmtsa1BsekZNNDRUZDBGWE91dnRwUWoyRFQy?=
 =?utf-8?B?aXdyQzRhYWJCR1hBY1lCUTFsUkpmeFVXQngxRENITEFKUklqTDJBUHBVd3Yz?=
 =?utf-8?B?akpwcWVlNHVUZmtyNkJONHJTWEh6Z0gxdXZ4QjgvNnQraFN5Qld0YkN5YUZQ?=
 =?utf-8?B?SW1rbDBCbG4xMmxFRFMwRHIrVUdiQUF1bHJXSHhrbXA0Ri9pWUYrbEEwTGdl?=
 =?utf-8?B?dnd3V01LNkJPamZ1MnBFWGZJaEthWjMxdGVHL3pUMm4rM2JXWGd0OGJ3UVBR?=
 =?utf-8?B?d091OUZINmxGSkhYRlo1MDR4eGU4bEg1NW9Hdk5YazF5anVHc2Mvc1NRMFRR?=
 =?utf-8?Q?NvaKDJ+IQ4WRm831HMM2WbSTdN6uvIWi1aYyq0w?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 96b8fcb9-5045-425f-9966-08d979294af5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 15:47:30.9044
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SUEATkh5UDUC6xTba0rTxQmeSn4Xind+Nf3nWh9+AP0+jF+8FaOz6dH1NdxByUQKZtBlEh+qADfMEgLSM/QPQg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3934

On 16.09.2021 17:43, Oleksandr wrote:
> On 16.09.21 17:49, Jan Beulich wrote:
>> On 10.09.2021 20:18, Oleksandr Tyshchenko wrote:
>>> @@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
>>>       uint64_aligned_t outstanding_pages;
>>>       uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
>>>       uint32_t hw_cap[8];
>>> +    uint32_t gpaddr_bits;
>>>   };
>> Please make trailing padding explicit. I wonder whether this needs
>> to be a 32-bit field: I expect we would need a full new ABI by the
>> time we might reach 256 address bits. Otoh e.g. threads_per_core is
>> pretty certainly oversized as well ...
> 
> I take it, this is a suggestion to make the field uint8_t and add 
> uint8_t pad[7] after?

I view this as a viable option at least.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:05:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:05:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188609.337828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQttV-0001ZP-SC; Thu, 16 Sep 2021 16:05:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188609.337828; Thu, 16 Sep 2021 16:05:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQttV-0001ZI-PA; Thu, 16 Sep 2021 16:05:49 +0000
Received: by outflank-mailman (input) for mailman id 188609;
 Thu, 16 Sep 2021 16:05:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Js+i=OG=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQttU-0001ZB-Hu
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:05:48 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id af57f8b5-cff0-40a6-9a27-7610c4913a39;
 Thu, 16 Sep 2021 16:05:47 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id i4so20481166lfv.4
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 09:05:47 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id f15sm392299ljn.120.2021.09.16.09.05.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 09:05:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af57f8b5-cff0-40a6-9a27-7610c4913a39
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=xBjKlDPRl5dWJIkl+qH7qFtKoxaKH7EY5tDc1AdHm4c=;
        b=LAUKRoNvFJCZ72IK2MKIoEMZAUasws5WqxOBsjKp3Mpr6DKPiE+Eizw00ONZpw9TQF
         wYhjvwDQ0JHB2x2dXY1b5KdMG0EFD77GxH6GnkZcMTkJiX/ykQebfDMG7t+7wvq4bZux
         RIUIDhDqJoni5nWK9T+Jny7QcV864J9fpZ2FUSZ1cXZ8IVJH+jBoVZ6I4na2N2IjX4hi
         6AuzGhvUBD1KG4PEKzkUQ2kOpvD2/nf4FuRlMg19rZLUtIk1L4KaOs13Uv691R+NFO0p
         P3FmwmmOttEWV91eIgJ+4tXGxzfg/rEIU30v0eUXA9qMA0sSp6XOtOgIsQcW68EUxDAx
         rWww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=xBjKlDPRl5dWJIkl+qH7qFtKoxaKH7EY5tDc1AdHm4c=;
        b=XjqiyRcx8CIvwpIyMl/2JgMLv6tS7f0kaJGrh5MhVPzj3ezzIsWIgg1GOsJcIJTZU4
         UICCt2FhgHAsi5tLdmqejtFmZNn+JNVtyNdD7XSeu+cEwJ0aSCi0JnCLX6vnNxKFcbGJ
         u3ldC7Wj5RLIyoDdodk2ZkincQ66a7GB7C3vboBHcLjGtQ/xoUx2kQgaXhwa3lWYkwBw
         Hdss+RGXJY1TGzhhq+d1cl3OL6L7TUz2tNmrX6NFA9wR1+eGRV/FuoaIx36vhlMc/soC
         B/DR7FyJ7FFISyHN/ASh0awAYXi+/bzRGR/YPoXIVrg863LnRt6H1uJ9/oImbbnKashY
         OJrw==
X-Gm-Message-State: AOAM531DT5tcKBZWn39yy+J5LopOnnNVfE5iZzt5Y9hCpLThskKucHvL
	YRZAkW9d6ptU2l2/Jl+yg6+nFiYZg4yBklIu
X-Google-Smtp-Source: ABdhPJycwtJPpL5SzvY/Gftt9tiQyXvMjich9Cr8mp1cHyDN1G0xBEoW651qjPS1mqZaYakWji9EaA==
X-Received: by 2002:a05:651c:1b3:: with SMTP id c19mr5672590ljn.16.1631808346058;
        Thu, 16 Sep 2021 09:05:46 -0700 (PDT)
Subject: Re: [PATCH V2 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Jan Beulich <jbeulich@suse.com>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
 <jgross@suse.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 xen-devel@lists.xenproject.org
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-2-git-send-email-olekstysh@gmail.com>
 <03b1a1bd-2630-0ee7-b995-114632f527aa@suse.com>
 <6b1c715f-e71e-718e-3cde-b298a5a38807@gmail.com>
 <ebb05297-b9ae-c475-b2f2-4c06e986f99e@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <5fa4ab4a-682b-cd2f-9388-8f99a9c4d57d@gmail.com>
Date: Thu, 16 Sep 2021 19:05:44 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <ebb05297-b9ae-c475-b2f2-4c06e986f99e@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 16.09.21 18:47, Jan Beulich wrote:

Hi Jan

> On 16.09.2021 17:43, Oleksandr wrote:
>> On 16.09.21 17:49, Jan Beulich wrote:
>>> On 10.09.2021 20:18, Oleksandr Tyshchenko wrote:
>>>> @@ -120,6 +120,7 @@ struct xen_sysctl_physinfo {
>>>>        uint64_aligned_t outstanding_pages;
>>>>        uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
>>>>        uint32_t hw_cap[8];
>>>> +    uint32_t gpaddr_bits;
>>>>    };
>>> Please make trailing padding explicit. I wonder whether this needs
>>> to be a 32-bit field: I expect we would need a full new ABI by the
>>> time we might reach 256 address bits. Otoh e.g. threads_per_core is
>>> pretty certainly oversized as well ...
>> I take it, this is a suggestion to make the field uint8_t and add
>> uint8_t pad[7] after?
> I view this as a viable option at least.

I got it, sounds reasonable.


>
> Jan
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:17:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188616.337839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQu4g-0003OA-Vf; Thu, 16 Sep 2021 16:17:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188616.337839; Thu, 16 Sep 2021 16:17:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQu4g-0003O3-RS; Thu, 16 Sep 2021 16:17:22 +0000
Received: by outflank-mailman (input) for mailman id 188616;
 Thu, 16 Sep 2021 16:17:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQu4f-0003Nx-7H
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:17:21 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9016ee02-1709-11ec-b619-12813bfff9fa;
 Thu, 16 Sep 2021 16:17:18 +0000 (UTC)
Received: from AM6P192CA0093.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::34)
 by AM6PR08MB4199.eurprd08.prod.outlook.com (2603:10a6:20b:b0::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 16:17:16 +0000
Received: from VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8d:cafe::e2) by AM6P192CA0093.outlook.office365.com
 (2603:10a6:209:8d::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 16:17:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT052.mail.protection.outlook.com (10.152.19.173) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 16:17:15 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Thu, 16 Sep 2021 16:17:15 +0000
Received: from 6226cb8cd231.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 006E3BBB-F8E3-4918-874A-2B011E902145.1; 
 Thu, 16 Sep 2021 16:17:08 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6226cb8cd231.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 16:17:08 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6711.eurprd08.prod.outlook.com (2603:10a6:20b:351::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 16 Sep
 2021 16:16:56 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 16:16:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9016ee02-1709-11ec-b619-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4q0yQQlbylH6/PLL1la6FC82QDE2edNdgXeRP5R4iVM=;
 b=DXM5MDf1PnJRcKdmeRHNT4naO2tPtoV96MpqydNktPIxS/Jn45OoVFPq3xxmiJcRq7uFNsfBm/yRpRvsyWucK1ooWwx+sBI/P5ArvAlgiycuPppaiamP8uz0J0ZALebNd1PBeLPUaUimCVwTa4Mr794OIxFz6obEt2KwrAtt1M0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6dffc4d5269c652f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aYNDIlEHF4HKjUz2ggv9EY25bP9UfFuiibj0N996F88ayS+IF+0iO14JV34HLb7Em9r/zj6KSYhTpZRTo1AUR2oKDtfdRJOnaEQKOtJIhahihAyuDTlUgt/VuqkJb10JZrA4idp6Mxz+SzX3uf1e6QS0W6gmOOD2//E5wxMmZtz3H5ynnwcOertW13TXbv9KuI0hd/J/bVfN4rMKakaPzACSLP8VtYsV+Gysbv0+NR13QZyDOdAwFNM2xaoneZogkZ7nnDutuqzEBy3kwPuevjbIsVbvRcA4l/uxJVYg5UVhPi16DltoQj0/QhxS1BYpHCDqb5lVSvmXJknuTGKdQA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4q0yQQlbylH6/PLL1la6FC82QDE2edNdgXeRP5R4iVM=;
 b=jQ7FICMLkdygPI2KTJG66dHFvnIUo40cv7MT3g9J0jQxGIii2SmDGrEDOrBqtLs6EaMqxPTyw+dOUcH6kfwel8BA0Y1v6UxXc1tqnJqvgmRtllQbVRdHp45305oiM8TU0quonZhJ7yy/M0F6dpV1WjUkmhqxch+ckJsn22SaKuahXOM1H/c2NLWBgHEt+oxi+JKH8sHzEbdW6VtgPrQFUpdO3XYkVMch/YEtLEq6waaDfk3rCiWwiPlx8rbwEXk+hdVceMklrYbf2A5/FQKZJj3i6uFfbfXMWIdpst0O22sNjjbAkRXvZtYCCftvjTMpr29n424/lKsfNbJbcuDTOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4q0yQQlbylH6/PLL1la6FC82QDE2edNdgXeRP5R4iVM=;
 b=DXM5MDf1PnJRcKdmeRHNT4naO2tPtoV96MpqydNktPIxS/Jn45OoVFPq3xxmiJcRq7uFNsfBm/yRpRvsyWucK1ooWwx+sBI/P5ArvAlgiycuPppaiamP8uz0J0ZALebNd1PBeLPUaUimCVwTa4Mr794OIxFz6obEt2KwrAtt1M0=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Topic: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Index: AQHXlPL+C+mRsI0CR06NagiBtFP5pat6ypyAgAHFXICAH0vvgIALJruA
Date: Thu, 16 Sep 2021 16:16:56 +0000
Message-ID: <BAD51906-B4EB-42E4-AC8E-DCABA6220861@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com>
 <c0ea2f05-bf0a-a8b0-a811-d899516c26ba@xen.org>
 <664B0578-B440-4229-8D6B-7B98857E75BF@arm.com>
 <3e192581-4167-c9aa-2c32-ff0338e9b800@xen.org>
In-Reply-To: <3e192581-4167-c9aa-2c32-ff0338e9b800@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 2afb5802-2d4a-4541-bcd2-08d9792d72ed
x-ms-traffictypediagnostic: AS8PR08MB6711:|AM6PR08MB4199:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4199611D857210969A12F1CAFCDC9@AM6PR08MB4199.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 oFz2LxSbB1lrXH9nMBh/VVv8oXQqarw983LnKUOlxEPyYGh9cweyMtE6OFK6Yh58ePoUXh4mYO3jx80y/NF7pritR2xcTphuw57HyljarBGUUyEBikgFhR3y5mHs3zwKDXsszy1legRDqITar23iIpGzc7cjb/Kp/6CHcEX7RPMFP5qEkvSNH++hLE6qNhWBxDnAH23ONjjMY9oMDMVhuwiwV1KIn8579FYRA/PFEG2DUTgu+Vz8harldsPSwcuUuJ8tsAlyEXNy3XxLDCIBeDpzVAfTVmhp/nlRbLUX88rrhbPs6LRX0JLQymO1h8FkU6MJV6u6WRKsFzPDqwag5Q+xjXF7F2HwdBEL97gJIb1BUtQAdCigrSeqqzh6PeyzshstufOAHHqtVBOMuKxDr8+0G+ozbMscMlLZeNZwbpZNF9QbKrtk9LAjue7M7AyPLBzdkkr+QZadMxwzdcGc255Lpg80tIDU95OyTwd1sqKrnNTpcDR8O7yOTVX2+9+8RVeu7soZqUVkXMY6MuPF4dnNJnbHO08YgWzT+vjj2dJ2srnkXskbMIwejmZ/gh/RJyK3ZaLJkbaNwmHHIMivSeerjwhSbaDN/8ONa4o+e5CtF8D9gIp3h2tMC1vZs/PAp3QYTKrml/RvrsBr33HVOd4VqSrd1CCTkKA+KYT3S7r1YwodnMEQ+gC9UI9SapIApWpqTn9f3gT9zI3lz96VrQtiJDxp9fOq5/0HY7jljl40B3P90C9BSYm/7nSSWkmPlwHvFemndmAu3lnh5gUZUiuIe0QQGZcag1/IuzzKdv6LncuC4NoYofBklUlFpdd4n+sRZO2+DtcPn82uEYLr1g==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(39860400002)(136003)(346002)(396003)(366004)(66476007)(66556008)(7416002)(36756003)(66446008)(76116006)(6512007)(66946007)(64756008)(8676002)(316002)(71200400001)(2906002)(2616005)(8936002)(186003)(86362001)(26005)(6486002)(53546011)(33656002)(38100700002)(6506007)(478600001)(5660300002)(38070700005)(6916009)(91956017)(4326008)(83380400001)(122000001)(966005)(54906003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <ABC34A4BF96D1846AFC721267DB7B68E@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6711
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7744177e-38ff-4ad2-3fd3-08d9792d6789
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QzFIvmffbsGWAokPXazTW040VTtZ1cCTJ2QJBkydyh0YV46YXmIA26MVfhT++sCG4yQEsLvnSapwDgNwD08eBHx4662vWBwQJUt4tP/T1B97iYoq8660ZuZIXChBYHsiU4tmwx/LuBwQSzsD06PJvWJsYZKYI+6C52TmdnSNuw8oXyNcM8dXVMq4yk4Jx9LjUfAJj+oUdUWe8rEoNCVzTP8/vzELU8YfMDuGrXGAxia1duNygdppilJv0uPqG2kvusqOhgxiWyXLBrtyWu06LxYKaCXGxFMo+yFaYOZMkaqSEqP+yl3rZ4aBxc5OUOg9hwQKjp2pY5huiamuS8tHaKkVsxCJ4cJlaKWBAfR8UbibYsODNLXoEDR5ghr9A2Jt4B666KfIMa0x4eZC+AbpJuEnNvEhu1lKZPYeyFyo+6PhUtOggp7LFdv5pc25/KDXgENNyeZodDpRHf4W1lLhqQdRGRAxGIXmKOyL8HNxUdq86N8WzCdqwF3E5aIVDp6sAPUL6iSMJc9hFeRt5xKqB9lBtOKBQQekwtCxBeMDZbVfmoCH17xZ/Omq0yWrvBYiDaMvyZC85Vfa2Ue13kr2jdukxM5Z+Qd5esSn0lYDr+3Qn/VI8o1BfZafuUSGtRO2cp83y5838tfI7Halwm3d6gzcr3V1TVKFVKGsa4X/J13Bf5xF6oDpzWcCvHW0qo6ypue5HrzPJNaX2UXtZl231BxN2DLqUd49/IEajRpDXnmV9V8bDEM5yTjixtOlHFgGe6yqZXqASiFEmiUoYLGmAFCPw33vewZ3VNukx8LyIW8=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(26005)(966005)(36756003)(6506007)(47076005)(53546011)(508600001)(336012)(2906002)(107886003)(33656002)(82310400003)(54906003)(2616005)(316002)(70586007)(186003)(6512007)(83380400001)(5660300002)(356005)(81166007)(8676002)(6486002)(70206006)(4326008)(8936002)(6862004)(36860700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 16:17:15.6838
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2afb5802-2d4a-4541-bcd2-08d9792d72ed
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4199

Hi Julien,

> On 9 Sep 2021, at 2:59 pm, Julien Grall <julien@xen.org> wrote:
>=20
>=20
>=20
> On 20/08/2021 17:03, Rahul Singh wrote:
>> Hi Julien,
>=20
> Hi Rahul,
>=20
>>> On 19 Aug 2021, at 2:00 pm, Julien Grall <julien@xen.org> wrote:
>>>=20
>>> Hi Rahul,
>>>=20
>>> On 19/08/2021 13:02, Rahul Singh wrote:
>>>> libxl will create an emulated PCI device tree node in the device tree =
to
>>>> enable the guest OS to discover the virtual PCI during guest boot.
>>>> Emulated PCI device tree node will only be created when there is any
>>>> device assigned to guest.
>>>> A new area has been reserved in the arm guest physical map at
>>>> which the VPCI bus is declared in the device tree (reg and ranges
>>>> parameters of the node).
>>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>>> ---
>>>>  tools/libs/light/libxl_arm.c          | 109 +++++++++++++++++++++++++=
+
>>>>  tools/libs/light/libxl_types.idl      |   1 +
>>>>  tools/xl/xl_parse.c                   |   2 +
>>>>  xen/include/public/arch-arm.h         |  11 +++
>>>>  xen/include/public/device_tree_defs.h |   1 +
>>>>  5 files changed, 124 insertions(+)
>>>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm=
.c
>>>> index e3140a6e00..a091e97e76 100644
>>>> --- a/tools/libs/light/libxl_arm.c
>>>> +++ b/tools/libs/light/libxl_arm.c
>>>> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void =
*fdt,
>>>>      return fdt_property(fdt, "reg", regs, sizeof(regs));
>>>>  }
>>>>  +static int fdt_property_values(libxl__gc *gc, void *fdt,
>>>> +        const char *name, unsigned num_cells, ...)
>>>> +{
>>>> +    uint32_t prop[num_cells];
>>>> +    be32 *cells =3D &prop[0];
>>>> +    int i;
>>>> +    va_list ap;
>>>> +    uint32_t arg;
>>>> +
>>>> +    va_start(ap, num_cells);
>>>> +    for (i =3D 0 ; i < num_cells; i++) {
>>>> +        arg =3D va_arg(ap, uint32_t);
>>>> +        set_cell(&cells, 1, arg);
>>>> +    }
>>>> +    va_end(ap);
>>>> +
>>>> +    return fdt_property(fdt, name, prop, sizeof(prop));
>>>> +}
>>>> +
>>>> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
>>>> +                                    unsigned addr_cells,
>>>> +                                    unsigned size_cells,
>>>> +                                    unsigned num_regs, ...)
>>>> +{
>>>> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
>>>> +    be32 *cells =3D &regs[0];
>>>> +    int i;
>>>> +    va_list ap;
>>>> +    uint64_t arg;
>>>> +
>>>> +    va_start(ap, num_regs);
>>>> +    for (i =3D 0 ; i < num_regs; i++) {
>>>> +        /* Set the memory bit field */
>>>> +        arg =3D va_arg(ap, uint64_t);
>>>> +        set_cell(&cells, 1, arg);
>>>> +
>>>> +        /* Set the vpci bus address */
>>>> +        arg =3D addr_cells ? va_arg(ap, uint64_t) : 0;
>>>> +        set_cell(&cells, addr_cells , arg);
>>>> +
>>>> +        /* Set the cpu bus address where vpci address is mapped */
>>>> +        set_cell(&cells, addr_cells, arg);
>>>> +
>>>> +        /* Set the vpci size requested */
>>>> +        arg =3D size_cells ? va_arg(ap, uint64_t) : 0;
>>>> +        set_cell(&cells, size_cells,arg);
>>>> +    }
>>>> +    va_end(ap);
>>>> +
>>>> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
>>>> +}
>>>> +
>>>>  static int make_root_properties(libxl__gc *gc,
>>>>                                  const libxl_version_info *vers,
>>>>                                  void *fdt)
>>>> @@ -668,6 +720,57 @@ static int make_vpl011_uart_node(libxl__gc *gc, v=
oid *fdt,
>>>>      return 0;
>>>>  }
>>>>  +static int make_vpci_node(libxl__gc *gc, void *fdt,
>>>> +        const struct arch_info *ainfo,
>>>> +        struct xc_dom_image *dom)
>>>> +{
>>>> +    int res;
>>>> +    const uint64_t vpci_ecam_base =3D GUEST_VPCI_ECAM_BASE;
>>>> +    const uint64_t vpci_ecam_size =3D GUEST_VPCI_ECAM_SIZE;
>>>> +    const char *name =3D GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
>>>> +
>>>> +    res =3D fdt_begin_node(fdt, name);
>>>> +    if (res) return res;
>>>> +
>>>> +    res =3D fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
>>>> +    if (res) return res;
>>>> +
>>>> +    res =3D fdt_property_string(fdt, "device_type", "pci");
>>>> +    if (res) return res;
>>>> +
>>>> +    res =3D fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
>>>> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size)=
;
>>>> +    if (res) return res;
>>>> +
>>>> +    res =3D fdt_property_values(gc, fdt, "bus-range", 2, 0,17);
>>>=20
>>> AFAICT, the "bus-range" is optional. Can you explain why we need it?
>> We need it to implement the function pci_ecam_map_bus().
>=20
> Ok. Then why next question is what does the 17 mean? Is it tie to how we
> implement the vPCI in Xen or the region we reserved?
>=20

Ah. I forgot to change it to 0xff. It should be 0xff.=20

> [...]
>=20
>>>=20
>>>> +
>>>>      if (b_info->type !=3D LIBXL_DOMAIN_TYPE_PV)
>>>>          return;
>>>>  diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libx=
l_types.idl
>>>> index 3f9fff653a..78b1ddf0b8 100644
>>>> --- a/tools/libs/light/libxl_types.idl
>>>> +++ b/tools/libs/light/libxl_types.idl
>>>> @@ -644,6 +644,7 @@ libxl_domain_build_info =3D Struct("domain_build_i=
nfo",[
>>>>        ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>>>>                                 ("vuart", libxl_vuart_type),
>>>> +                               ("vpci", libxl_defbool),
>>>=20
>>> Any new addition in the structure should be accompanied with a LIBXL_HA=
VE_* in the libxl.h header.
>> OK.
>>>=20
>>>>                                ])),
>>>>      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>>>>                                ])),
>>>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>>>> index 17dddb4cd5..ffafbeffb4 100644
>>>> --- a/tools/xl/xl_parse.c
>>>> +++ b/tools/xl/xl_parse.c
>>>> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source=
,
>>>>          }
>>>>          if (d_config->num_pcidevs && c_info->type =3D=3D LIBXL_DOMAIN=
_TYPE_PV)
>>>>              libxl_defbool_set(&b_info->u.pv.e820_host, true);
>>>> +        if (d_config->num_pcidevs)
>>>> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>>>>      }
>>>>        if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
>>>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-a=
rm.h
>>>> index 0a9749e768..01d13e669e 100644
>>>> --- a/xen/include/public/arch-arm.h
>>>> +++ b/xen/include/public/arch-arm.h
>>>> @@ -426,6 +426,17 @@ typedef uint64_t xen_callback_t;
>>>>  #define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
>>>>  #define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
>>>>  +/* PCI-PCIe memory space types */
>>>> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000)
>>>> +#define GUEST_VPCI_ADDR_TYPE_MEM          xen_mk_ullong(0x02000000)
>>>=20
>>> What the size of those regions?
>> Non Prefetch Memory: Size 64 MB start at 512 MB
>> Prefetch Memory: Size 128 GB start at 36 GB
>>>=20
>>>> +
>>>> +/* Guest PCI-PCIe memory space where config space and BAR will be ava=
ilable.*/
>>>> +#define GUEST_VPCI_PREFETCH_MEM_ADDR  xen_mk_ullong(0x900000000)
>>>> +#define GUEST_VPCI_MEM_ADDR           xen_mk_ullong(0x20000000)
>>> So far the memory layout defines the address in ascending order. So ple=
ase add that after GUEST_RAM_BANK_BASES_*.
>> Ok.
>>>=20
>>> However, if I am not mistaken that base address you provide will clash =
with RAM bank 1. It also seem to be pretty high which means that this will =
not work for 32-bit domain or on CPUs that don't allow offer large IPA bits=
.
>> Yes I also checked that now that it is having clash with RAM bank 1.
>> There is unused space is guest memory that we can use for Non Prefetch M=
emory as per below guest memory map.
>> https://gitlab.com/xen-project/fusa/fusa-docs/-/blob/master/high-level/g=
uest-memory-layout-arm.rst
>> Proposed value:
>> Non Prefetch Memory: Size 64 MB start at 0x22001000
>> Prefetch Memory: Size 4 GB start at 4 GB.
>=20
> The base address looks fine to me. However, the sizes are much smaller to=
 what you initially suggested. Would you be able to clarify why the smaller=
 sizes are fine?

As per current design we can have 32 device in guest therefore I thought 64=
 MB is sufficient ( 2 MB per device ).
>=20
>>>=20
>>> I think we need to start making the guest layout more dynamic. The VPCI=
 memory space would have to go right after the end of the RAM allocated for=
 a given guest.
>>>=20
>>>> +
>>>> +#define GUEST_VPCI_PREFETCH_MEM_SIZE      xen_mk_ullong(0x2000000000)
>>>> +#define GUEST_VPCI_MEM_SIZE               xen_mk_ullong(0x04000000)
>>>=20
>>> It would be better if the size for each region is defined right after e=
ach base.
>> OK.
>>>=20
>>> Also, how did you decide the size of each region?
>> I thought 64 MB will be sufficient. I think it should be based on number=
 of devices we can assign to the guest.
>=20
> We don't have to get the size right now. What I am more interested is to =
have a trace about how those values were decided (even if it just saying ra=
ndom). This will help to make any decision if in the future we need to resi=
ze (in particular downsize) the regions.

As per Stefano suggestion in another mail he suggested 256 MB. I also think=
 256 MB will be sufficient ( 8 MB per device) . I will comment this in code=
.

Regards,
Rahul
>=20
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:21:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:21:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188624.337849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQu8q-0004p4-Ka; Thu, 16 Sep 2021 16:21:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188624.337849; Thu, 16 Sep 2021 16:21:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQu8q-0004ox-HZ; Thu, 16 Sep 2021 16:21:40 +0000
Received: by outflank-mailman (input) for mailman id 188624;
 Thu, 16 Sep 2021 16:21:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=2MkR=OG=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mQu8p-0004or-5E
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:21:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2a4a559b-170a-11ec-b619-12813bfff9fa;
 Thu, 16 Sep 2021 16:21:38 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2057.outbound.protection.outlook.com [104.47.6.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-43BqqwukOHe_IaFL9g1n1Q-1; Thu, 16 Sep 2021 18:21:35 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6303.eurprd04.prod.outlook.com (2603:10a6:803:f2::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 16:21:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 16:21:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0058.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Thu, 16 Sep 2021 16:21:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a4a559b-170a-11ec-b619-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631809297;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wY7TnsrZbWO1VPz9llXFhPuXTujUHRVNln7a9qarJVQ=;
	b=fD5oqxzxMQyMJqKsNsKlymNpEQiLiuUuejuMYKIb2Vmb7YF3NWcBXD1BkMOSPQ4TK5HoeG
	TjP+y1EA9q61OyLzfK5E4F9a33iune1k6goCSmH1idhUcnEXqZsZLGgSTIVcvOMAG6Pc3I
	Q3n79JgeFRPHEDmEzJcNViNKS51i9Sc=
X-MC-Unique: 43BqqwukOHe_IaFL9g1n1Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jXLxaUrZc04u09im7vYN3iEtnKB8q580ZyRMOJdQnlf9Z34W+VsRONJ/z6VMY1Ow+776uJqXRfrXyvOXt7DUq1Kho3fXAjD2219kctPp9jziB6Yw8mU4Fu6L9K1dZQqp1rAwY+Nb8PnudE3pA7/PMnS9T8hUbrfOPne88MMXHCq7zodZbjq7ML0D21mmCSG/aMaaMhh5Dd/rKhMepSL6sCnzABjwxj6IzLVpDyC5TQ47Bfr6j5xHbMEiTAxRlNJHGRge9ez3lzw7rCmDknYsXNJZOY1FIMIk+7FZK/qwscfquBKDhsX2FD5foHdIZa1CFOIO4xGg16PwZ1cFPUApBQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wY7TnsrZbWO1VPz9llXFhPuXTujUHRVNln7a9qarJVQ=;
 b=muY+lrfhERL5+/KRuAxT++KhOHUp2T/zTPKvkwPgTspu//IwdMhqgiRgV/ACBd5ECyrOu2dG99M8TMU/9HDbOopS64Mn0YgtGTTVq7EuS31z7ND2JiY8mqC26bgEoouv2h0B0R8Ag9DPjqdEhNgaVniTUMnzyVVZC33neufN16ufDNwdBJhrLh4jzKBc+0JzF/7MiBPDAoAzi3yO+FzpNM1OQdmD4PvDv75DFdoTPkFyKO68QFN3Z3x60GRSrx3WzEQy4Xnl+58J00fzhTZYx1S/Y4SCr0jItLC2H10z2gUwXbUud2wVUK18B+aXLavcZSiv//1gYgMiUu9N0pXMQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xenproject.org; dkim=none (message not signed)
 header.d=none;xenproject.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: xen-devel@lists.xenproject.org
References: <osstest-164996-mainreport@xen.org>
Cc: osstest service owner <osstest-admin@xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
Date: Thu, 16 Sep 2021 18:21:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <osstest-164996-mainreport@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0058.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d92193a0-a1b1-4aec-d89a-08d9792e0cd3
X-MS-TrafficTypeDiagnostic: VI1PR04MB6303:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6303A406665D9253EA1F073AB3DC9@VI1PR04MB6303.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tH8h31edKHSy/nvCFZHULRYCKNbm4i+8gOMTNNZ5Qn96ueSOomXa9kxDYslN7brEHhJIIVntFJ2GDR9sq+hJ3YgPvDuZ3n6yRlOh9xD2DsDijx1hJMTFqNI0mLx2TPFOgMUbvLqWLthBFt9WbGEQnXl7pE/fSYi46r/xT8IXEFvduSfsYMkSE+++hrQg71hwyGmghm6rBLYb65Nq1GqkdrDi15dziGLuSG4Zb4592bBsAXM0tsBpV9JqIKJZNwFatBkcwcZpkkmtFrR8/buE6ruuuNSRQO3AfyDcMRR8Mc+goeuhb8uqOmlD/IN4Yg2Cz5i+HPTBFGvlWnVSqiNGEt+HQPfjiIpydoJLiSFM/s9livEa+QnWuMc7ExKNrsz4nCPukqfY3DPQ0NM+fgXZKqAyuB9xirMy0Aoi0bSrME374WjkQdB7cmRoAmTLsecAx5AW1w7pJFsUgE67ynjqpqRee3P9Tk3lEqCx/rlPWG2r+/OR3oiyY/XNwo5eGQtkTSAmf2LEygt0ZotY/LPRpK0r99z8LOoAjfUw4w2rPgZPfgMTIKMUnflySz65PJ/LUQEld87q4vlOx65sdy4h+n7+xAV2RGH7qI9cqDcoBf3oX03VQZYjOu5lS9SVyapySoZadOAekptvI9QkrRIizIOxEvN2poYbcz/ViTfIS7GEqhtjphoECC6GUgWrThlz7FkNpbOJhk/VayfTJp5ChDmrgmnZOJpHGnZi1I20pCXAxPQ2adn6nviYo5Iw5hD7L/++4gIAKQTiN7Cf491Kn6x1KkhyDhR2TLZhNWZhTlgGWuMCqW22tDI7nsAuIG3rcXL3FbFPiJ9ENZt6c+DPMA1Kr01XBPK/hY7zW0J4n6U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(966005)(66556008)(66946007)(2906002)(6486002)(66476007)(186003)(8936002)(38100700002)(83380400001)(6916009)(2616005)(4326008)(956004)(8676002)(26005)(36756003)(316002)(16576012)(31686004)(5660300002)(86362001)(31696002)(508600001)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VHcyb0FYeGRpWWV6d0h6WWIrR1JYZjBLUGJXMUUvYjFRYjYxWlAzblluM1R2?=
 =?utf-8?B?SEZZYTRHeFV4RGE4cGhzclVJVE5BMVNDZTdsUHdqa1ZOUGRGcmNoQWJxazAy?=
 =?utf-8?B?dlVBVUNXcGd4MWY1Mjl1NE83YlFnYUJiWkVPUXp4QTlOWGc0SXI3TzdhK0N4?=
 =?utf-8?B?N3l3anNFRTRkaEo2SmFpY2dYSThkR2wwS0NnMlFZQVdXTXRCSVFPbFpCT1ZL?=
 =?utf-8?B?by9VS2x4RTFobHUvNXM3MmJwcFo0N00wbHpablV1bXU0RkpLbExrUTFvZ3Q4?=
 =?utf-8?B?OWV1bEFiVzVQNjRJa1V5S09odHp6VmtsK2ZRTHN6V2UrQjlpVWlYNnJ5Uk1u?=
 =?utf-8?B?VDlPWFB4RHRpNjBJK2RGNlhicnUwbkJqWUhvV3g4eXVVcGRCdEgrMmFXWWhu?=
 =?utf-8?B?d0c2cDIwNjg1UmVrY0xDZ2MxK1JJVVhzSEFESUFtVVczOG5pS09wYjZZb2ZD?=
 =?utf-8?B?eWEwa3pocXdyam5pZUJyNXVXUXRvdDF5YW92dEZlbGNTamNaMnl3ZGk4L0wz?=
 =?utf-8?B?RFlHN3p4aXgzR2ZodURHeGRYOXQ5aDNiSkcyZGtXL0s3ZG5sYXBkb2ZtTm45?=
 =?utf-8?B?WjFGRURPUHFOb0ZFOFhsN2ViclBXTmF0V29KQk12dEY5c2p0OU5HQkNxaU5R?=
 =?utf-8?B?bUlkMGJFR3dkT3lQMDR6VUwyUmpneTRKUGxqdm9qR2ROR2VVenpFbWxHdTdo?=
 =?utf-8?B?bnhvMTRQSGRaZ1JvdExkY09LRnIwSnlFQXBSVUVoZ3F3K1lWZU9YV21oWEhC?=
 =?utf-8?B?LzB3WnNaQXo4SUlWamkrS1MzSTZNeEsweWdZL1YzR2xKMTl1QWVGSmxLZEpw?=
 =?utf-8?B?WWY0bVQvWmpDYi9WRkxBSUF4Y0V0cWFTY3ZORG96VGhMWG4xMG1zbFEvK25M?=
 =?utf-8?B?bW9iYkUybytENUFVbXdtWFBnNEpWcWNjbTdYTVliU1FYOXU5c3JuYzZmc05L?=
 =?utf-8?B?MW1vMURldnN6eko5bDBEM1F5cW5tV0srMjF4aW1jQXhMell5ZE5Bd3ZJSmZM?=
 =?utf-8?B?ODdsM2gvL0JTZVZLZmpHQVlhOHdkT25OZ2VpOXh4TFAzeGZGWEMxbXlEUytn?=
 =?utf-8?B?bW12eWhCTVk5ajR0TUdUNks2dVJiQ0JLMkI5TG1yYVl1ZXFZVDhVYThjaHg2?=
 =?utf-8?B?VHJoR3ArMmJjSXVIL2VlNGZ3bWRBbXo0eE53T2NTdWpvUmt6RVNBMFZWTS9q?=
 =?utf-8?B?SjlwMHJOZzAwa05paTZHRVFDc0pJVDAyMGovN0NXekVDN1BneEFQTWkxdGll?=
 =?utf-8?B?MjFCTXJGbnluN0gzdTNkK293dWtjZTJmQjJXby9hYko1dmU4UmtnZHJFaitG?=
 =?utf-8?B?RXE5UnlmMHF6SzZpR29DQ0o4eUVaN3VCSzhQSGhjT05LeEJtQk5LSFdtYkEr?=
 =?utf-8?B?ZnFTYzhDbjNleDNNQytxVXl0YU85SFU4WDF1Y0VheHFSVkMwRG9jTFJGRlBK?=
 =?utf-8?B?QzU1eDlyTmxBMmk3Zk81SjIvdHdCWVlMTFNFb0ZYY3M5cTNiMmg0bU1FL3RC?=
 =?utf-8?B?UkIyRWhHdnovVENuTmVIT29Xdk1tNXdESTJndUNubDBLQld4SlZ6WFh6RUJG?=
 =?utf-8?B?NW9BZG1FWDF6UmtZRkFvZ3FSUFFOMSs2Unk5Z1dNb0h5bGNkektmUEtEZkQ1?=
 =?utf-8?B?bnJFNldOZks0dHlhZzV3TVZXVmpiZWV4N3d5YVJqTFQwd3BsZldXZnRJNk5M?=
 =?utf-8?B?TFdDM0hpOGt0bHd1QlFXQWNUejRhT3lWRW9ZM0FhOVg0Smh6MWhmN2ZmNG5S?=
 =?utf-8?Q?7acQxRsKgFwmA4VBgJXk950rO7EytKVXlKK7CZ/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d92193a0-a1b1-4aec-d89a-08d9792e0cd3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 16:21:34.1351
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7uDfg/kDN62BOjCUCFkoFx61hICQCNhkhJftZZYiHhprhznP7fS9qcIscCfZCBUryRey3f/auro4iysyYmK12Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6303

On 16.09.2021 06:06, osstest service owner wrote:
> flight 164996 xen-unstable real [real]
> flight 165002 xen-unstable real-retest [real]
> http://logs.test-lab.xenproject.org/osstest/logs/164996/
> http://logs.test-lab.xenproject.org/osstest/logs/165002/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Since no-one gave a sign so far of looking into this failure, I took
a look despite having little hope to actually figure something. I'm
pretty sure the randomness of the "when" of this failure correlates
with

Sep 15 14:44:48.518439 [ 1613.227909] rpc-worker: page allocation failure: order:4, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0
Sep 15 14:44:55.418534 [ 1613.240888] CPU: 48 PID: 2029 Comm: rpc-worker Not tainted 5.4.17+ #1
Sep 15 14:44:55.430511 [ 1613.247370] Hardware name: Cavium ThunderX CN88XX board (DT)
Sep 15 14:44:55.430576 [ 1613.253099] Call trace:
Sep 15 14:44:55.442497 [ 1613.255620]  dump_backtrace+0x0/0x140
Sep 15 14:44:55.442558 [ 1613.259348]  show_stack+0x14/0x20
Sep 15 14:44:55.442606 [ 1613.262734]  dump_stack+0xbc/0x100
Sep 15 14:44:55.442651 [ 1613.266206]  warn_alloc+0xf8/0x160
Sep 15 14:44:55.454512 [ 1613.269677]  __alloc_pages_slowpath+0x9c4/0x9f0
Sep 15 14:44:55.454574 [ 1613.274277]  __alloc_pages_nodemask+0x1cc/0x248
Sep 15 14:44:55.466498 [ 1613.278878]  kmalloc_order+0x24/0xa8
Sep 15 14:44:55.466559 [ 1613.282523]  __kmalloc+0x244/0x270
Sep 15 14:44:55.466607 [ 1613.285995]  alloc_empty_pages.isra.17+0x34/0xb0
Sep 15 14:44:55.478495 [ 1613.290681]  privcmd_ioctl_mmap_batch.isra.20+0x414/0x428
Sep 15 14:44:55.478560 [ 1613.296149]  privcmd_ioctl+0xbc/0xb7c
Sep 15 14:44:55.478608 [ 1613.299883]  do_vfs_ioctl+0xb8/0xae0
Sep 15 14:44:55.490475 [ 1613.303527]  ksys_ioctl+0x78/0xa8
Sep 15 14:44:55.490536 [ 1613.306911]  __arm64_sys_ioctl+0x1c/0x28
Sep 15 14:44:55.490584 [ 1613.310906]  el0_svc_common.constprop.2+0x88/0x150
Sep 15 14:44:55.502489 [ 1613.315765]  el0_svc_handler+0x20/0x80
Sep 15 14:44:55.502551 [ 1613.319583]  el0_svc+0x8/0xc

As per

Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650

the system doesn't look to really be out of memory; as per

Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB

there even look to be a number of higher order pages available (albeit
without digging I can't tell what "(C)" means). Nevertheless order-4
allocations aren't really nice.

What I can't see is why this may have started triggering recently. Was
the kernel updated in osstest? Is 512Mb of memory perhaps a bit too
small for a Dom0 on this system (with 96 CPUs)? Going through the log
I haven't been able to find crucial information like how much memory
the host has or what the hypervisor command line was.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:26:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:26:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188631.337860 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuDs-0005UE-8G; Thu, 16 Sep 2021 16:26:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188631.337860; Thu, 16 Sep 2021 16:26:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuDs-0005U7-4w; Thu, 16 Sep 2021 16:26:52 +0000
Received: by outflank-mailman (input) for mailman id 188631;
 Thu, 16 Sep 2021 16:26:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQuDr-0005Tx-8V; Thu, 16 Sep 2021 16:26:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQuDr-0002GQ-3P; Thu, 16 Sep 2021 16:26:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQuDq-0001dK-QX; Thu, 16 Sep 2021 16:26:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQuDq-0005L5-Pt; Thu, 16 Sep 2021 16:26:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NeG8TOuXw61APYVyhimGzCwHtmTR7VE/t0b2J57xTE4=; b=gsElJ9cCLYT4oFlrlqLnMPpRUQ
	p/UR3Tp2YAvcZTZ7ghS24Qu9ixoE28hvITML7OB360zyJh8KYph4K+ulqMHCa+VmjqidVbdCXKo1V
	XM9hkj54pJIEQsvvb2Nn/oZsjHlQlGRZRddiMlpWJ96Fb1eMP3Q00/t2JkWaTSc18SaI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165004-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165004: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:<job status>:broken:regression
    xen-unstable:test-armhf-armhf-libvirt:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3e910b648b99393561e7c523756c1ea49a6c1305
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 16:26:50 +0000

flight 165004 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165004/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt        <job status>                 broken
 test-armhf-armhf-libvirt      5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3e910b648b99393561e7c523756c1ea49a6c1305
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    5 days
Failing since        164951  2021-09-12 00:14:36 Z    4 days    9 attempts
Testing same since   165004  2021-09-16 04:09:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     broken  
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-libvirt broken
broken-step test-armhf-armhf-libvirt host-install(5)

Not pushing.

(No revision log; it would be 332 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:35:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:35:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188641.337875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuM6-0007AX-9i; Thu, 16 Sep 2021 16:35:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188641.337875; Thu, 16 Sep 2021 16:35:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuM6-0007AQ-6J; Thu, 16 Sep 2021 16:35:22 +0000
Received: by outflank-mailman (input) for mailman id 188641;
 Thu, 16 Sep 2021 16:35:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQuM4-0007AK-LR
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:35:20 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 13d872ae-170c-11ec-b61f-12813bfff9fa;
 Thu, 16 Sep 2021 16:35:18 +0000 (UTC)
Received: from DU2P251CA0023.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:230::35)
 by DBBPR08MB6012.eurprd08.prod.outlook.com (2603:10a6:10:205::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 16:35:14 +0000
Received: from DB5EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:230:cafe::9d) by DU2P251CA0023.outlook.office365.com
 (2603:10a6:10:230::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 16 Sep 2021 16:35:14 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT020.mail.protection.outlook.com (10.152.20.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 16:35:14 +0000
Received: ("Tessian outbound 16951d3c485e:v103");
 Thu, 16 Sep 2021 16:35:14 +0000
Received: from 53b85a6a011b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A0E24062-5434-4A8C-8C72-F0892A3C3F17.1; 
 Thu, 16 Sep 2021 16:35:07 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 53b85a6a011b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 16:35:07 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1620.eurprd08.prod.outlook.com (2603:10a6:203:3a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 16 Sep
 2021 16:35:05 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 16:35:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13d872ae-170c-11ec-b61f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OA9fprGsLIHwdgvV+o6Ibk9ZfgK3uN7JblgkMGYQA3k=;
 b=aO1d0oZdPc9p0xYglspRz5o5dq1zOFNC5QJCk0tHHlyyPXHd4uAqDGFk08x4UcN937AFJjykUVeMpKObiQv+HTfjo1RqIrqWIDoo3g9UQ5xCKySqtrxOUhT7eCtj+YwFV/MskQwFs1OTTljxHYC5yzhOjqEUPYj/j0Iou7uwPjg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2e0352f647ed9c7a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X9p/9gQ5TTQNU+M5YfaGZW4tfY63veH0Bi53TrpsqrYcLxzIuAoGxVTnR6IB9mrdUdiOEUpA0shrnHHXfPRu0sqBniswyrpnz0EW63o3gvNV8dmOfB9o4cQwBDXjInUhqCjHeUrJ1krRwpmN/CHeA5rtVjWD+xYC1471I6LSuWDfSj7YkHVJoQ7KRYx5NTxQaP4HkmUp8LcKHNzVq6qrFqfaKlQdCEsoKeyZ5/V59CoZkm6+EXvlX17diGczGEK3ilb1edSXzCFJ1dDTPHFI7zc0OwYzCZRPXBidRqRNIy9DxnZ4js1zHR45rmDMCa0k6LruitALVEfqlSlNluHB0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OA9fprGsLIHwdgvV+o6Ibk9ZfgK3uN7JblgkMGYQA3k=;
 b=YUosHlHMvMgQ+aDEgw9VClGo0gCEBky4YxlP91EgHA5rGag0ZXdaCfnljaVgmrZ0V1BivfnFSDeLQNgiAY2VkQ7Hp2y2mMd/pWtnBF8XjcBv7FjCbfZzknnKW3As5+E0CopbbmA9KtWk+xLgsfW5zXdxNpCmCVZJNYBxEUgU0O4gVsOSo9pR9oMimn/DleMFA0jdOkOt3rQ0vKKjqR1lY5T3TOMw9QW4pDNf7iz1VeAxEYNqNqYMbPLQnK4POa6o5/YSKaWz/w1YvSs8x60Yfw0Yb+IihgsAjqFudgkX2p9s80F7/fWYgs7xGZnFZtIXbKB/7SkhFTeLqrzVYnSwnQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OA9fprGsLIHwdgvV+o6Ibk9ZfgK3uN7JblgkMGYQA3k=;
 b=aO1d0oZdPc9p0xYglspRz5o5dq1zOFNC5QJCk0tHHlyyPXHd4uAqDGFk08x4UcN937AFJjykUVeMpKObiQv+HTfjo1RqIrqWIDoo3g9UQ5xCKySqtrxOUhT7eCtj+YwFV/MskQwFs1OTTljxHYC5yzhOjqEUPYj/j0Iou7uwPjg=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Topic: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Index: AQHXlPL+C+mRsI0CR06NagiBtFP5pauckjcAgAp1fQA=
Date: Thu, 16 Sep 2021 16:35:05 +0000
Message-ID: <38E94844-FB48-4245-9D73-48ED897CAF13@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091742210.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091742210.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: f0075bea-c6f7-4202-4836-08d9792ff5d4
x-ms-traffictypediagnostic: AM5PR0801MB1620:|DBBPR08MB6012:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB60126F8756BC4CC923D66DEDFCDC9@DBBPR08MB6012.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 v52AEfNM0gXxOOXrAHbidek0PnGxFN07PNlJX8t7D9XKcPaG7CcLHertD+rISd+6CJxeuRpmcDYKGeSv+JDLdrg1G7F5RZXGjtY5knWyu3ql2PW444WaQ7+8XdGnJ1RA7Y8aNRbkDokfzfOVdRVk20WPZy5RoV8vlZ2xK8c9J11Irfn2QT4krOT5qZZcCHetsy92iUoUsO/trGJ1NpE6mhNPjaqZwGg1ZLCDjF27vtI/TlEaEHmENDs7Nam33M9+91gdOigcuzBLi682TkOAe93xhjBgf64ypzxKUafEHnGzaEUuwI7JihJFX4RSYpItCAu/xaL26ztr4osUuOT9Z5zm/a/+KkMgkXhCldqpBKjjtASY064rpzPqc5SfOpVd25ksR/H7J3lfhd79g2+/JYB3V/mQUK23lpO6iRsWVbgRqJmW8CV5LZ+OSxcMvqcl2THUG0IWJWq5RQ6bO+GzbMIhHefYzvLsrncwO7rXHUdtBcvICuSUE2JwNy8A+9N1nOF+clpwKdI4gJ+D+ZQ7Q0763Vk0TZbQRlor8KGtoqI+o3+DNqou7PPe3dconlXot00zG0UJOswM/p7Hrzsqh1tPgMrNeM85e3UNIYOW/xDDKzyPPvfb97slagbUyAChBNXXQBgGeDuqT3Cz8KcBL+vB4ZLdjQoBdQTL/u4JzRo2LUFFaWUlIfMlzqOo5cjH5tT2tCTSafXtzVjkznbPfjVeKRofKZdOjeF8jBvAfP6m+Un/J9txrN/SI+bocT07
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(346002)(366004)(396003)(39860400002)(376002)(86362001)(8936002)(4326008)(6916009)(316002)(83380400001)(26005)(36756003)(33656002)(53546011)(6486002)(122000001)(478600001)(38070700005)(7416002)(91956017)(186003)(66446008)(66556008)(66476007)(66946007)(2906002)(71200400001)(8676002)(38100700002)(76116006)(5660300002)(2616005)(6512007)(64756008)(54906003)(6506007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <5B8CE46B74D305488FD66E50E7424A09@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1620
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c32697e2-4f7b-4b05-463d-08d9792ff0a1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qux8u5qNmPCAjCoxx3hIuajZDp+hOHgZNSIVUG5NaWh38rQs+O8Uy/EQ4hSpms0W3+63LsOTE1tYGgByOjX5x369z1VnTQWAX0Gl3CoqwNWvoQJLab1B1oeJ9rchq85tC1VkM38y71lXbyTUxBjCkumIGDjD54D4C71Df6xo8sBjquz5WS8bhJdyeMl5la99eoyWSZkiM3tX26BQDLZTpdHfhk3YWPy7B//Lsg6UE8wdDPH0DG5C3y5kutpDxIexJ9yz8CPpR5q1rjwRfsmJfOStflmtvD3L2NSeqGNuR/AnPpqTC2/4IzEQnrQeBl5khEeZwJHRR5q/CS5XdAez5974au2/LkJz272WYAZUXTIJtlnZ4anCRjS7z/yxSeAZ7akMJtpXV9YfvJ1iDHHdRdr1la+4K18VbWxxVnOObJdqwqR+0jy6vpK3Xg6uNOb5HcRZW3SXxmJmHCW5TGDvV7onePE9XWfwPPFDX66pMq4O70zde9IvS/FDlKnNANSso6HwhZOGMjo5s3HUAFdCsK10uKUz3/FZWKG1vkIk4ZDtpwUWUJKMrHDPiWx9Bu74aPDeGlLFwgYDTq8ShkA3wEkhZf+/ZO4v+Qbi+6VSqKnBgM58gZ01THqIEAQMQm/6xwV7g+9Ox+NJO2Q6lk4BUHG/pP19Q3yMUrN8qa8aoQz04zuLc7CycVQfvAyBfjq2AR8DlaCKFXLxg4MtO4ioAg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(6862004)(54906003)(316002)(356005)(81166007)(36756003)(70206006)(82310400003)(86362001)(47076005)(36860700001)(508600001)(6486002)(6512007)(107886003)(8936002)(8676002)(26005)(6506007)(5660300002)(53546011)(186003)(83380400001)(4326008)(2906002)(336012)(70586007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 16:35:14.4309
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f0075bea-c6f7-4202-4836-08d9792ff5d4
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6012

Hi Stefano,

> On 10 Sep 2021, at 1:51 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Thu, 19 Aug 2021, Rahul Singh wrote:
>> libxl will create an emulated PCI device tree node in the device tree to
>> enable the guest OS to discover the virtual PCI during guest boot.
>> Emulated PCI device tree node will only be created when there is any
>> device assigned to guest.
>>=20
>> A new area has been reserved in the arm guest physical map at
>> which the VPCI bus is declared in the device tree (reg and ranges
>> parameters of the node).
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> tools/libs/light/libxl_arm.c          | 109 ++++++++++++++++++++++++++
>> tools/libs/light/libxl_types.idl      |   1 +
>> tools/xl/xl_parse.c                   |   2 +
>> xen/include/public/arch-arm.h         |  11 +++
>> xen/include/public/device_tree_defs.h |   1 +
>> 5 files changed, 124 insertions(+)
>>=20
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index e3140a6e00..a091e97e76 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *f=
dt,
>>     return fdt_property(fdt, "reg", regs, sizeof(regs));
>> }
>>=20
>> +static int fdt_property_values(libxl__gc *gc, void *fdt,
>> +        const char *name, unsigned num_cells, ...)
>> +{
>> +    uint32_t prop[num_cells];
>> +    be32 *cells =3D &prop[0];
>> +    int i;
>> +    va_list ap;
>> +    uint32_t arg;
>> +
>> +    va_start(ap, num_cells);
>> +    for (i =3D 0 ; i < num_cells; i++) {
>> +        arg =3D va_arg(ap, uint32_t);
>> +        set_cell(&cells, 1, arg);
>> +    }
>> +    va_end(ap);
>> +
>> +    return fdt_property(fdt, name, prop, sizeof(prop));
>> +}
>> +
>> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
>> +                                    unsigned addr_cells,
>> +                                    unsigned size_cells,
>> +                                    unsigned num_regs, ...)
>> +{
>> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
>> +    be32 *cells =3D &regs[0];
>> +    int i;
>> +    va_list ap;
>> +    uint64_t arg;
>> +
>> +    va_start(ap, num_regs);
>> +    for (i =3D 0 ; i < num_regs; i++) {
>> +        /* Set the memory bit field */
>> +        arg =3D va_arg(ap, uint64_t);
>> +        set_cell(&cells, 1, arg);
>=20
> Shouldn't this be uint32_t given that it is 1 cell exactly?

Yes I will change it to uint32_t.
>=20
>=20
>> +        /* Set the vpci bus address */
>> +        arg =3D addr_cells ? va_arg(ap, uint64_t) : 0;
>> +        set_cell(&cells, addr_cells , arg);
>> +
>> +        /* Set the cpu bus address where vpci address is mapped */
>> +        set_cell(&cells, addr_cells, arg);
>> +
>> +        /* Set the vpci size requested */
>> +        arg =3D size_cells ? va_arg(ap, uint64_t) : 0;
>> +        set_cell(&cells, size_cells,arg);
>                                       ^ space

Ack.
>=20
>=20
>> +    }
>> +    va_end(ap);
>> +
>> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
>> +}
>> +
>> static int make_root_properties(libxl__gc *gc,
>>                                 const libxl_version_info *vers,
>>                                 void *fdt)
>> @@ -668,6 +720,57 @@ static int make_vpl011_uart_node(libxl__gc *gc, voi=
d *fdt,
>>     return 0;
>> }
>>=20
>> +static int make_vpci_node(libxl__gc *gc, void *fdt,
>> +        const struct arch_info *ainfo,
>> +        struct xc_dom_image *dom)
>> +{
>> +    int res;
>> +    const uint64_t vpci_ecam_base =3D GUEST_VPCI_ECAM_BASE;
>> +    const uint64_t vpci_ecam_size =3D GUEST_VPCI_ECAM_SIZE;
>> +    const char *name =3D GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
>> +
>> +    res =3D fdt_begin_node(fdt, name);
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_string(fdt, "device_type", "pci");
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
>> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_values(gc, fdt, "bus-range", 2, 0,17);
>                                                            ^ space
>=20
Ack.=20
>=20
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_cell(fdt, "#address-cells", 3);
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_cell(fdt, "#size-cells", 2);
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_string(fdt, "status", "okay");
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
>> +        GUEST_ROOT_SIZE_CELLS, 2,
>> +        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_S=
IZE,
>> +        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR=
,
>> +        GUEST_VPCI_PREFETCH_MEM_SIZE);
>> +    if (res) return res;
>> +
>> +    res =3D fdt_property_values(gc, fdt, "msi-map", 4, 0, GUEST_PHANDLE=
_ITS,
>> +                              0, 0x10000);
>> +    if (res) return res;
>=20
> I agree with Julien that we shouldn't add it now if it is not working.
Ack.
>=20
> One question: what about legacy interrupts? If they are supported,
> shouldn't we have interrupts and interrupt-parent properties?

As per current design legacy interrupt will not be supported.
>=20
>=20
>> +    res =3D fdt_end_node(fdt);
>> +    if (res) return res;
>> +
>> +    return 0;
>> +}
>> +
>> static const struct arch_info *get_arch_info(libxl__gc *gc,
>>                                              const struct xc_dom_image *=
dom)
>> {
>> @@ -971,6 +1074,9 @@ next_resize:
>>         if (info->tee =3D=3D LIBXL_TEE_TYPE_OPTEE)
>>             FDT( make_optee_node(gc, fdt) );
>>=20
>> +        if (libxl_defbool_val(info->arch_arm.vpci))
>> +            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
>> +
>>         if (pfdt)
>>             FDT( copy_partial_fdt(gc, fdt, pfdt) );
>>=20
>> @@ -1189,6 +1295,9 @@ void libxl__arch_domain_build_info_setdefault(libx=
l__gc *gc,
>>     /* ACPI is disabled by default */
>>     libxl_defbool_setdefault(&b_info->acpi, false);
>>=20
>> +    /* VPCI is disabled by default */
>> +    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
>> +
>>     if (b_info->type !=3D LIBXL_DOMAIN_TYPE_PV)
>>         return;
>>=20
>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_t=
ypes.idl
>> index 3f9fff653a..78b1ddf0b8 100644
>> --- a/tools/libs/light/libxl_types.idl
>> +++ b/tools/libs/light/libxl_types.idl
>> @@ -644,6 +644,7 @@ libxl_domain_build_info =3D Struct("domain_build_inf=
o",[
>>=20
>>     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>>                                ("vuart", libxl_vuart_type),
>> +                               ("vpci", libxl_defbool),
>>                               ])),
>>     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>>                               ])),
>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>> index 17dddb4cd5..ffafbeffb4 100644
>> --- a/tools/xl/xl_parse.c
>> +++ b/tools/xl/xl_parse.c
>> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>>         }
>>         if (d_config->num_pcidevs && c_info->type =3D=3D LIBXL_DOMAIN_TY=
PE_PV)
>>             libxl_defbool_set(&b_info->u.pv.e820_host, true);
>> +        if (d_config->num_pcidevs)
>> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>>     }
>>=20
>>     if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm=
.h
>> index 0a9749e768..01d13e669e 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -426,6 +426,17 @@ typedef uint64_t xen_callback_t;
>> #define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
>> #define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
>>=20
>> +/* PCI-PCIe memory space types */
>> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000)
>> +#define GUEST_VPCI_ADDR_TYPE_MEM          xen_mk_ullong(0x02000000)
>> +
>> +/* Guest PCI-PCIe memory space where config space and BAR will be avail=
able.*/
>> +#define GUEST_VPCI_PREFETCH_MEM_ADDR  xen_mk_ullong(0x900000000)
>> +#define GUEST_VPCI_MEM_ADDR           xen_mk_ullong(0x20000000)
>> +
>> +#define GUEST_VPCI_PREFETCH_MEM_SIZE      xen_mk_ullong(0x2000000000)
>> +#define GUEST_VPCI_MEM_SIZE               xen_mk_ullong(0x04000000)
>=20
> Now I get it: GUEST_VPCI_ECAM_BASE was just for the config space, not
> the aperture. The apertures are starting at 0x20000000 and 0x900000000.
Yes right.
>=20
> Speaking of which, isn't GUEST_VPCI_MEM_SIZE a bit too small? There
> could be PCI devices with a <4GB requirement. Maybe 256MB?

Yes I will allocate the 256 MB.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:36:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:36:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188649.337886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuNU-0007lg-Kx; Thu, 16 Sep 2021 16:36:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188649.337886; Thu, 16 Sep 2021 16:36:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuNU-0007lZ-Hn; Thu, 16 Sep 2021 16:36:48 +0000
Received: by outflank-mailman (input) for mailman id 188649;
 Thu, 16 Sep 2021 16:36:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQuNT-0007lT-C8
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:36:47 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.43]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 71a7c576-2367-4dc0-8510-543361e13219;
 Thu, 16 Sep 2021 16:36:45 +0000 (UTC)
Received: from AM5PR0701CA0014.eurprd07.prod.outlook.com
 (2603:10a6:203:51::24) by DB9PR08MB7195.eurprd08.prod.outlook.com
 (2603:10a6:10:2ce::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 16:36:43 +0000
Received: from AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:51:cafe::1e) by AM5PR0701CA0014.outlook.office365.com
 (2603:10a6:203:51::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.5 via Frontend
 Transport; Thu, 16 Sep 2021 16:36:43 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT021.mail.protection.outlook.com (10.152.16.105) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 16:36:42 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Thu, 16 Sep 2021 16:36:40 +0000
Received: from ba851a647237.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 65041264-D4B2-4FC6-A3E5-88A493900A93.1; 
 Thu, 16 Sep 2021 16:36:23 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ba851a647237.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 16:36:23 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB3303.eurprd08.prod.outlook.com (2603:10a6:209:40::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep
 2021 16:36:19 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 16:36:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71a7c576-2367-4dc0-8510-543361e13219
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rh5U5jBs7lh719Lm+U+c+VrgWPm2L8w30+OVH8bS7NQ=;
 b=zh1acYIj8fhfHwtz80NdaRl9H/tEK5qb+uGks7e28yGDgubo9Be9uNPY12I0bfysAe9fNjmdKwxpBlGWPHh8pmwmWEaXqponHkiIjTYnSRpGrrGqx61ZALcvZbwmEPCyiS0cpQ+CrSiVuywQJFyFT8PSXG9ey4eCd2mVqHN/Hm8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: b01904e5e115c3d8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Qq8u6PTZUrKX4I3c0QI0owmnhAXTNg//YP4fGSuFkoOGVZd70Hui7cJAW1AQ3gB6NHF7bNM+sXLihpm42eU6jjiMu9HtXLtEwW3/pQumy8n30J/FcQOAE0Tbkhz6Ag4clBnj7EKVKYX2qG+JEB7V+gYFUiYRrcGO5wkYSHgtL/CdNgGz31akOmXruQ0xl8lyv9eamwRj5gqVIIMdmpgmTTUtcVwhkuG+/bDnOhEdd89xlTzUqeBHd18LRY6xpQuRiDzrEESUXracRv/c4Nul2rErfddO9guy8y1Qfaf0yUxs3gjZ5icboC2DUz9lXe4SxoxHELXidPmgJXBK4ukInw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Rh5U5jBs7lh719Lm+U+c+VrgWPm2L8w30+OVH8bS7NQ=;
 b=MxEGdEqz8MTu1CA4iT3hYxLjUszzXlwvPgEJbGovA+ln67po4jg5VdMdSFUqeroZf4tajKv29J+WVjPJCNICCUCgHmCgtAi/f04sxR1TG/a3sTULa4MnXsRWeDXHPRSOG/H6REIGzFa24C9yHK7QblftqXlHYq7tr+idEDAXYqfbWr3wr16q19wC0zZFVAQKA6Tfmyvxt/zhlKDr4aTnnQpzs2v65/KSsqpEO9WLS+YxjK9kyG+UTdyx0WoKD6lZthJFBzguEI/4+ZzlaZlGaIIC2QvT31Mm49eP9PvZW7GFvc9lzhdMZxy/vkl2HJkh/ZAprL+m4jFSQUChU5AODA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rh5U5jBs7lh719Lm+U+c+VrgWPm2L8w30+OVH8bS7NQ=;
 b=zh1acYIj8fhfHwtz80NdaRl9H/tEK5qb+uGks7e28yGDgubo9Be9uNPY12I0bfysAe9fNjmdKwxpBlGWPHh8pmwmWEaXqponHkiIjTYnSRpGrrGqx61ZALcvZbwmEPCyiS0cpQ+CrSiVuywQJFyFT8PSXG9ey4eCd2mVqHN/Hm8=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 14/14] xen/arm: Add linux,pci-domain property for hwdom
 if not available.
Thread-Topic: [PATCH v1 14/14] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
Thread-Index: AQHXlPMDjN8YIk1EsES2o1HUm/aDequclJ+AgApzbQA=
Date: Thu, 16 Sep 2021 16:36:18 +0000
Message-ID: <1A906E81-C6B4-4631-8A40-2B06E2D139EB@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <0b979ccab6a4c9b2070748709f737f41fa108a42.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091758290.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109091758290.10523@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: dd8ad2c4-28ba-4b90-11d7-08d979302a74
x-ms-traffictypediagnostic: AM6PR08MB3303:|DB9PR08MB7195:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB71950BBE9D2E5BCE2A67C7F4FCDC9@DB9PR08MB7195.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ZomSBdQBOQRqInV9pep68ErloF6AaxTepghhuyf16G23v1ZufFwnrmCxYCzLwyT/K7PcJpZ0sxDuZH1YKQevRLzC3qHA8uyGoKUA/APQ5ZnDtV6kHkzzY9dPzrh/oVG6xe1zuLiE6HGlXcRbmdw+gHZWt7uN+7idX998/Go2J6QeSy25OiI8z6au5lmDh4GmoQDF7LpZYAC4cPhGeK9HbhWLhe9vSO2lAZhPNnKftPQzPdJs3KeRQrzeEtOhfjrbLxUKvMBp3QirBQlkNA8gMJ6srNtgT3TbtaQhhbYyT3TL3/Rv68zswJdFgu8HWue5Vwbo3AnGlwnf99skewbBXbYyC3LATCDBUW0JQkAq6fYBn+kZ0JKxLcUYtCrM3XpwaRYVvSZ1zg45dm/TBSCBYbR5h+6F+HxLB56OeKoqPP5FCRDYbGd3CBGThpMl3dAmfgnvLoWgt+xF8ArAWBNR4XLsyGG62f79K5hk++HODmFKrRuw/Y/EPaiZC6Df3AHxmVD7jReFhuHah3FP3Lme50mAL/jAbps33gGsDc/SLWpOlifUt6oybIiuq0NUwCkJKuRF4d68PDmevlEib19K43aoqcUKWyR1pAJdCgQwAi7ApsYBH5q3Jt+5GoyBYtEIbtbd/BwKEG7CBb++JBoOHMzULtRtQq2EgE61BTOwrlAYYNGV2vXyy18tiZ2cnwkaJQ2OM4GPar/Arfroe/b+elvcd6Mikl8qLFbDK+i1x4g1KBwWMPl2QqsUHzg22tY8
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(366004)(396003)(376002)(39860400002)(346002)(54906003)(316002)(4326008)(6486002)(8676002)(8936002)(76116006)(6916009)(86362001)(91956017)(478600001)(38070700005)(2906002)(83380400001)(66946007)(5660300002)(2616005)(64756008)(38100700002)(53546011)(66446008)(26005)(66556008)(122000001)(66476007)(6506007)(6512007)(71200400001)(36756003)(33656002)(186003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2B2B4F3B0F74B14EA12DBD4AB9417D01@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3303
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ee391fce-5aee-4eda-1485-08d979301c70
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8wrGjyISXqTEDLFN4rLaeQ67gjLrmtFhkkdZkeeL/ZMVrZwTa0g+1gkUaMp14Je/v8uBJd+UlStnVDeN+6W7aD1QfrsRWX7NKuU4H+ILVk1UAfWeNzTQpPoYj3AMUHbjKg2xau3sUR0rX98c/Yoy+JiFfI+Psjp7AIoSuDBmoGpxJmWw3koK8b7obwr0p4u04tSPgMk9m+IaG/boQza3lBNBvFM3UDF6lr0g6/IyyeAjRJAKppxHFUM6slxfuq5UWDu5/l5s5/KI/3YXpMzK88u5I89xri0VcPbJZ4V4z2W+ZtZbYeVZNcibHtQTIXhFXmRoNC8OOz5fDDLoAIT90T1b2+OvDwJpnQ/Mpbo82jeB9LhLmKyCBM2Pl3TPE0Zbk1K6KEzdC/VifzwoNdFodqvI46AYmXWeDJajMWQ/74Ya8czMN3LQQNjnMcQ7wPt7jU5pD/pPxnJ+giMxa0w0R2MUi4r+c2DFr0QxfTzxse/M3r8u5K5Y86t/nrWNTTuo21pO4Aav6GzzTPjfB6jcjRZs47YL2VSQWaaEDWwnHCHMhVSl8guBofE1lsLw3X1rEVVeY3mZDkoWv9FC2Tq4qmG+nOUNnAb1lFu2rZKyCoEvXy68jL2Rr9JLSeWRoPRSAwKkjfyek8+NEuW/TX5cOd2QmEJaDb6vs1z4M83IqzrXvXkmk5/zUEqDgivdq/rX8tPsvsYEsRaMmVq199UUQQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(346002)(396003)(136003)(376002)(46966006)(36840700001)(26005)(107886003)(53546011)(6506007)(2616005)(336012)(36756003)(70206006)(70586007)(5660300002)(86362001)(82310400003)(36860700001)(186003)(82740400003)(33656002)(356005)(81166007)(83380400001)(4326008)(2906002)(8936002)(478600001)(6862004)(6512007)(316002)(47076005)(54906003)(8676002)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 16:36:42.4747
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dd8ad2c4-28ba-4b90-11d7-08d979302a74
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7195

Hi Stefano,

> On 10 Sep 2021, at 2:00 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Thu, 19 Aug 2021, Rahul Singh wrote:
>> If the property is not present in the device tree node for host bridge,
>> XEN while creating the dtb for hwdom will create this property and
>> assigns the already allocated segment to the host bridge
>> so that XEN and linux will have the same segment for the host bridges.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> xen/arch/arm/domain_build.c        | 18 ++++++++++++++++++
>> xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
>> xen/include/asm-arm/pci.h          |  3 +++
>> 3 files changed, 42 insertions(+)
>>=20
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 6c86d52781..e0cf2ff19d 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -581,6 +581,24 @@ static int __init write_properties(struct domain *d=
, struct kernel_info *kinfo,
>>             return res;
>>     }
>>=20
>> +#ifdef CONFIG_HAS_PCI
>> +    if ( dt_device_type_is_equal(node, "pci") )
>> +    {
>> +        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
>> +        {
>> +            uint16_t segment;
>> +
>> +            res =3D pci_get_host_bridge_segment(node, &segment);
>> +            if ( res < 0 )
>> +                return res;
>> +
>> +            res =3D fdt_property_cell(kinfo->fdt, "linux,pci-domain", s=
egment);
>> +            if ( res )
>> +                return res;
>> +        }
>> +    }
>> +#endif
>=20
> If param_pci_enable is false it might be possible that Xen didn't
> allocate a segment. In that case, we should just let Linux do whatever
> it wants in terms of segment allocation. So I think the code here should
> not return error if param_pci_enable is false.
> returning an error instead.

Ack. I will modify the code as per your request in next version.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 16:51:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 16:51:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188657.337897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQuby-000203-2B; Thu, 16 Sep 2021 16:51:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188657.337897; Thu, 16 Sep 2021 16:51:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQubx-0001zw-VO; Thu, 16 Sep 2021 16:51:45 +0000
Received: by outflank-mailman (input) for mailman id 188657;
 Thu, 16 Sep 2021 16:51:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Lm3Y=OG=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mQubw-0001zq-Nk
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 16:51:44 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.47]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5e1e6330-170e-11ec-b620-12813bfff9fa;
 Thu, 16 Sep 2021 16:51:42 +0000 (UTC)
Received: from AS9PR06CA0117.eurprd06.prod.outlook.com (2603:10a6:20b:465::32)
 by VI1PR0801MB1904.eurprd08.prod.outlook.com (2603:10a6:800:81::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 16 Sep
 2021 16:51:36 +0000
Received: from VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::2f) by AS9PR06CA0117.outlook.office365.com
 (2603:10a6:20b:465::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 16 Sep 2021 16:51:36 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT015.mail.protection.outlook.com (10.152.18.176) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Thu, 16 Sep 2021 16:51:36 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Thu, 16 Sep 2021 16:51:34 +0000
Received: from 2e85e9f66db2.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 11DD48BF-9CE6-46E9-BE69-AF77150B1842.1; 
 Thu, 16 Sep 2021 16:51:27 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2e85e9f66db2.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 16 Sep 2021 16:51:27 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4439.eurprd08.prod.outlook.com (2603:10a6:20b:be::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 16 Sep
 2021 16:51:17 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.016; Thu, 16 Sep 2021
 16:51:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e1e6330-170e-11ec-b620-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KXAwJSeiZbEQxNNV56SDXxraOP0Fs0R+3mN2KqTATQ8=;
 b=u9rphU3zAIKPKf+RQn8RACg1BmLRsASc/JcYjcPW39fo17XicnfgbBEGhO+ik3O0PjYGAkHjYhhUSojAOmAkAu8zm/Na0YxFe+AZg+9AbykDXL63C6OF5F3f76aj535MC2srUpL+jgYCV4h1fWVtk0yp7sqqn6JwOXRlikT49iw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2b125cd73e4fd786
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=i8o9D0EdBec1BKSSpSsyPM0pLz8iMYQ5P26b8JvOcZG7NNvFCycCi+43hVLnJFT+KvXmSYBMBVU/RPezxACMfoVe9CJPZDalYjb2GCyi4cRhZiILOf+YILNAF57FNnK9Bn6nNYlefUT56Mzasp7VtrdeL6njTEYubkqsJvvUlo5G4zPWzD8vpq6ozDwmf98Je1X27SgLEHbk8lTq/ko8CybApoSKvAFRg8o0ZTAa1u/ORq5+JH8kR1M6wuzpI6pYdryLODegjfkBqS4BGTrUJ0auhaaXEE7R2PKwojPYYlR3TnhOEgiiDcyAUrobxHxkZ0nWeqVmd0y3JW9BsIVFpw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KXAwJSeiZbEQxNNV56SDXxraOP0Fs0R+3mN2KqTATQ8=;
 b=KCgdOMKR9kukzMu5IIgp7gKbqj9bzbG/4XSRBYSOMgzHdx+WuiaiBGR1XtnoVfIw0dMNEadhMol1XKVa1dp1WjLvv046sX4DSSI7fUDGJHeBNgYy7/SYXxsKVMYP81gLt8vyU3wYfhsHXRbeB5JasIpExuWI8VQ8OsOw5yaX6dAdTg+IDzAyRYGuxdpyz8oGg7x3eAeFZYEQdrI5uM1JbkcyJehZQU8SXtWMvmRCDOirRixGtwpETS8RTPrtlmm4pcKWSVNSket/w4+06TsYxfhjYWLYQhigZMpMIvph70EUz4ua0y9cHWN2m92vNql4KLd5jNc9BMLomDuQC2+Ofg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KXAwJSeiZbEQxNNV56SDXxraOP0Fs0R+3mN2KqTATQ8=;
 b=u9rphU3zAIKPKf+RQn8RACg1BmLRsASc/JcYjcPW39fo17XicnfgbBEGhO+ik3O0PjYGAkHjYhhUSojAOmAkAu8zm/Na0YxFe+AZg+9AbykDXL63C6OF5F3f76aj535MC2srUpL+jgYCV4h1fWVtk0yp7sqqn6JwOXRlikT49iw=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Topic: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations
Thread-Index:
 AQHXlPK6hMBa2cV9wU+RB+r01dTauquceRAAgAcQPwCAAMcsAIABJesAgABE9YCAAVDgAA==
Date: Thu, 16 Sep 2021 16:51:16 +0000
Message-ID: <FE13F3A1-C6EF-452F-BB5F-AD80B6FD97C6@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091607070.10523@sstabellini-ThinkPad-T480s>
 <B0EBC722-A74A-4742-9D93-904398FDDF1B@arm.com>
 <alpine.DEB.2.21.2109141543090.21985@sstabellini-ThinkPad-T480s>
 <8B84E0D6-70A7-4C8F-ACBE-D1773F6C8FAD@arm.com>
 <alpine.DEB.2.21.2109151340390.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109151340390.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 45183dc7-1abd-42aa-d45c-08d979323f10
x-ms-traffictypediagnostic: AM6PR08MB4439:|VI1PR0801MB1904:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0801MB1904A4FB8CAD55B466BFD25BFCDC9@VI1PR0801MB1904.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4O+RzwGX7mN6YK/eD63wwV0s15V6si76G0Wffwr0i+U67MSGSsCe3t7B4EtXmJmjod4ZLUzHAuZKUAunPtrTQYhkJJLNoudxjYpso9TjNgmkLeLNoG6v5Mx5kFNr2w7wPljhnAEn71++RZuTPY7DzYNa7JTLDF/cwHBy/U6qAcWJgwQAJNeAJ8Mh2Ngwnxx11txPw4kTiujG25C8ehxwLHMarfAjzNouyzlR7iyEox8rlnAHN8r0O8btT/uyNawH2NOhYOkWrDcjpHpE4nZzEC96GsArpOR9caXWFpEzFNelseEznPLtWCkau9jbzi/ZXbmQlM2mCSeqilLlr/ovgYPHanThL8sO9xouS7aOvnZcL9Om5bnsAwpQ+wOpCO5G16cIPwM53g9FIeLS60zahPvDJRxtr2kxp0SFrAQLNcnkGNc4Rhc0aNIVwmF3tIGSxT8zORKvI59reGMhhKk5/TdfJPyJjk+VUu+tI5C2uTfB/1R8TOvx8C8DjwBk5M3nvOmcsDsVZl3PE6r9/6pQzHB+QbEuSDV+OtpDQcpUA4DASViY5cgUS43EmHELPq5RPb1xjSjyVyY64hkHgSJZMq6CMjxqkDdUGTougb4Hu0yAfACoTxBXHpGvdHs7XXywtmrSIKDHIx9Y9mwRiHM6qlSGJ6j+5xGZBeYFAF5idMmP1VhIn3ibTAjh0wmZql25dpcc0qBN39kSezpgqpFpf4GzZnZHSbKBXnU+77JmrVRLtjzS9aKKmYV4kTzMNwXgJMpAMgY0SV3NrcN7I1D6RC7x2j4WsflUTbSPLyO0QrPax/QEp9xXHmLFB84/x4VICdLBXAPLhGyPeUDiMKQVPQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(346002)(366004)(136003)(396003)(39860400002)(122000001)(316002)(2906002)(8676002)(53546011)(86362001)(8936002)(6506007)(478600001)(38100700002)(4326008)(54906003)(36756003)(6486002)(71200400001)(66556008)(64756008)(66446008)(66946007)(66476007)(2616005)(5660300002)(966005)(91956017)(6512007)(33656002)(38070700005)(26005)(6916009)(186003)(83380400001)(76116006)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <E9278EB5E4CF304EB976CC7E5AA9B41C@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4439
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	39daf147-c23e-4f7f-08b6-08d97932338e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	D4u7Xw7mb9RyNRLgsYWsR1JoKZcbCHWewmAWXPX2GV3WVbpwBGXKsfO6XY36p8HJmMxxfMrVqlbtFVFf2ynBOHVw5pB9Y/GZBUdonNis8SSFOkoRrhAtLm8OLIdO5CXgt7Lr1KyEETPmnLTW7qf274aPfu8+l6K3oobN+LbbwXuQ+605GdkGjG8hPVUxzdIPu7z7tMrQAnu0xVStDoSyy538a7j3sM0QPLTYsUezqHzhEqbxGifXZUs8/tuStRhTrpJQ/nb25++3zO5jbnKn5PTu/yLJ8/7ZVMWxPeJrlG1OunuW0xpYtFcLg7FWogDNi4awGtfx0NSw5KUR32/Z7MCU6zeXSID7KehlrpiGwO6v7Kl5t0D4+KNCVk9LBVaK4nclFHHa1ik8ydPcj/Fyd6b9VhogjGN1gT9mfbe54BNF2nKlD+kr0aVUr2Bj7ZK5GZQSfXozG2sTeABvYopLngDvbY10c/IEIu8d8jN6cEQ4NFbxB78Zf4bb5FBOXip+QjNeWp2fzPxnkfVatKP+Dw3VNvB3KxbBaUSoRiJOoNapUkDQvnZkaMhMhSjaHq5K/ZviY1Xx7RRs3PTr/vW5ipqwUgRf/PvWJIXkXibDNEbTNCLvGFuj3AjEi6xa+5F60YbJL39wz8o5XpWsAZp6WpjYroE/bpzM2aoLywhASPdD8OtBykCZ1Zx2ovGm+Bn99N7mR7wMgSTtZmmj5j4eQZQbSaceglTWNR/dBhPT70hpveFqLHsg+9yEHoFQFLa0hEauLgYgmZZ/80fNZyJ5sJ23p8HvxrR7mI2EVsXFaJM=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(136003)(396003)(346002)(39860400002)(376002)(36840700001)(46966006)(6506007)(82310400003)(478600001)(6486002)(6512007)(8676002)(47076005)(54906003)(356005)(316002)(36860700001)(107886003)(82740400003)(336012)(81166007)(6862004)(8936002)(36756003)(86362001)(2616005)(83380400001)(4326008)(70586007)(33656002)(26005)(5660300002)(186003)(70206006)(2906002)(53546011)(966005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 16:51:36.1929
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 45183dc7-1abd-42aa-d45c-08d979323f10
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1904

SGkgU3RlZmFubywNCg0KPiBPbiAxNSBTZXAgMjAyMSwgYXQgOTo0NSBwbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFdlZCwgMTUg
U2VwIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4+IE9uIDE1IFNlcCAyMDIxLCBhdCAxMjow
NiBhbSwgU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToN
Cj4+PiBPbiBUdWUsIDE0IFNlcCAyMDIxLCBSYWh1bCBTaW5naCB3cm90ZToNCj4+Pj4+PiArICAg
ICAgICByZXR1cm4gTlVMTDsNCj4+Pj4+PiArDQo+Pj4+Pj4gKyAgICBidXNuIC09IGNmZy0+YnVz
bl9zdGFydDsNCj4+Pj4+PiArICAgIGJhc2UgPSBjZmctPndpbiArIChidXNuIDw8IGNmZy0+b3Bz
LT5idXNfc2hpZnQpOw0KPj4+Pj4+ICsNCj4+Pj4+PiArICAgIHJldHVybiBiYXNlICsgKFBDSV9E
RVZGTihzYmRmX3QuZGV2LCBzYmRmX3QuZm4pIDw8IGRldmZuX3NoaWZ0KSArIHdoZXJlOw0KPj4+
Pj4+ICt9DQo+Pj4+PiANCj4+Pj4+IEkgdW5kZXJzdGFuZCB0aGF0IHRoZSBhcm0zMiBwYXJ0IGlz
IG5vdCBpbXBsZW1lbnRlZCBhbmQgbm90IHBhcnQgb2YgdGhpcw0KPj4+Pj4gc2VyaWVzLCB0aGF0
J3MgZmluZS4gSG93ZXZlciBpZiB0aGUgcGxhbiBpcyB0aGF0IGFybTMyIHdpbGwgZHluYW1pY2Fs
bHkNCj4+Pj4+IG1hcCBlYWNoIGJ1cyBpbmRpdmlkdWFsbHksIHRoZW4gSSBpbWFnaW5lIHRoaXMg
ZnVuY3Rpb24gd2lsbCBoYXZlIGFuDQo+Pj4+PiBpb3JlbWFwIGluIHRoZSBhcm0zMiB2ZXJzaW9u
LiBXaGljaCBtZWFucyB0aGF0IHdlIGFsc28gbmVlZCBhbg0KPj4+Pj4gdW5tYXBfYnVzIGNhbGwg
aW4gc3RydWN0IHBjaV9vcHMuIEkgdW5kZXJzdGFuZCB0aGF0IHBjaV9lY2FtX3VubWFwX2J1cw0K
Pj4+Pj4gd291bGQgYmUgYSBOT1AgdG9kYXkgZm9yIGFybTY0LCBidXQgSSB0aGluayBpdCBtYWtl
cyBzZW5zZSB0byBoYXZlIGl0IGlmDQo+Pj4+PiB3ZSB3YW50IHRoZSBBUEkgdG8gYmUgZ2VuZXJp
Yy4NCj4+Pj4gDQo+Pj4+IEFzIHBlciBteSB1bmRlcnN0YW5kaW5nIHdlIGRvbuKAmXQgbmVlZCBw
Y2lfZWNhbV91bm1hcF9idXMoLi4pIGFzIEkgZG9u4oCZdCBzZWUgYW55IHVzZSBjYXNlIHRvIHVu
bWFwIHRoZSANCj4+Pj4gYnVzIGR5bmFtaWNhbGx5LiBXZSBjYW4gYWRkIHRoZSBzdXBwb3J0IGZv
ciBwZXJfYnVzX21hcHBpbmcgZm9yIEFSTTMyIG9uY2Ugd2UgaW1wbGVtZW50IGFybTMyIHBhcnQu
DQo+Pj4+IExldCBtZSBrbm93IHlvdXIgdmlldyBvbiB0aGlzLg0KPj4+IA0KPj4+IEluIHRoZSBw
YXRjaCB0aXRsZWQgInhlbi9hcm06IFBDSSBob3N0IGJyaWRnZSBkaXNjb3Zlcnkgd2l0aGluIFhF
TiBvbg0KPj4+IEFSTSIgdGhlcmUgaXMgdGhlIGZvbGxvd2luZyBpbi1jb2RlIGNvbW1lbnQ6DQo+
Pj4gDQo+Pj4gKiBPbiA2NC1iaXQgc3lzdGVtcywgd2UgZG8gYSBzaW5nbGUgaW9yZW1hcCBmb3Ig
dGhlIHdob2xlIGNvbmZpZyBzcGFjZQ0KPj4+ICogc2luY2Ugd2UgaGF2ZSBlbm91Z2ggdmlydHVh
bCBhZGRyZXNzIHJhbmdlIGF2YWlsYWJsZS4gIE9uIDMyLWJpdCwgd2UNCj4+PiAqIGlvcmVtYXAg
dGhlIGNvbmZpZyBzcGFjZSBmb3IgZWFjaCBidXMgaW5kaXZpZHVhbGx5Lg0KPj4+ICoNCj4+PiAq
IEFzIG9mIG5vdyBvbmx5IDY0LWJpdCBpcyBzdXBwb3J0ZWQgMzItYml0IGlzIG5vdCBzdXBwb3J0
ZWQuDQo+Pj4gDQo+Pj4gDQo+Pj4gU28gSSB0YWtlIGl0IHRoYXQgb24gYXJtMzIgd2UgZG9uJ3Qg
aGF2ZSBlbm91Z2ggdmlydHVhbCBhZGRyZXNzIHJhbmdlDQo+Pj4gYXZhaWxhYmxlLCB0aGVyZWZv
cmUgd2UgY2Fubm90IGlvcmVtYXAgdGhlIHdob2xlIHJhbmdlLiBJbnN0ZWFkLCB3ZSdsbA0KPj4+
IGhhdmUgdG8gaW9yZW1hcCB0aGUgY29uZmlnIHNwYWNlIG9mIGVhY2ggYnVzIGluZGl2aWR1YWxs
eS4NCj4+IA0KPj4gWWVzIHlvdSBhcmUgcmlnaHQgbXkgdW5kZXJzdGFuZCBpcyBhbHNvIHNhbWUu
DQo+Pj4gDQo+Pj4gSSBhc3N1bWVkIHRoYXQgdGhlIGlkZWEgd2FzIHRvIGNhbGwgaW9yZW1hcCBh
bmQgaW91bm1hcCBkeW5hbWljYWxseSwNCj4+PiBvdGhlcndpc2UgdGhlIHRvdGFsIGFtb3VudCBv
ZiB2aXJ0dWFsIGFkZHJlc3MgcmFuZ2UgcmVxdWlyZWQgd291bGQgc3RpbGwNCj4+PiBiZSB0aGUg
c2FtZS4NCj4+IA0KPj4gQXMgcGVyIG15IHVuZGVyc3RhbmRpbmcgZm9yIDMyLWJpdCB3ZSBuZWVk
IHBlcl9idXMgbWFwcGluZyBhcyB3ZSBkb27igJl0IGhhdmUgZW5vdWdoIHZpcnR1YWwgYWRkcmVz
cyBzcGFjZSBpbiBvbmUgY2h1bmsNCj4+IGJ1dCB3ZSBjYW4gaGF2ZSB2aXJ0dWFsIGFkZHJlc3Mg
c3BhY2UgaW4gZGlmZmVyZW50IGNodW5rLg0KPiANCj4gSW50ZXJlc3RpbmcuIEkgd291bGQgaGF2
ZSBhc3N1bWVkIHRoYXQgdGhlIHN1bSBvZiBhbGwgdGhlIGluZGl2aWR1YWwNCj4gc21hbGxlciBp
b3JlbWFwcyB3b3VsZCBzdGlsbCBiZSBlcXVhbCB0byBvbmUgYmlnIGlvcmVtYXAuIE1heWJlIGZv
cg0KPiBMaW51eCBpcyBkaWZmZXJlbnQsIGJ1dCBJIGRvbid0IHRoaW5rIHRoYXQgbWFueSBzbWFs
bGVyIGlvcmVtYXBzIHdvdWxkDQo+IGJ1eSB1cyB2ZXJ5IG11Y2ggaW4gWGVuIGJlY2F1c2UgaXQg
aXMgdGhlIHRvdGFsIGlvcmVtYXAgdmlydHVhbCBzcGFjZQ0KPiB0aGF0IGlzIHRvbyBzbWFsbC4g
T3IgYW0gSSBtaXNzaW5nIHNvbWV0aGluZz8NCj4gDQo+IA0KPj4gSSBhbSBub3Qgc3VyZSBpZiB3
ZSBuZWVkIHRvIG1hcC91bm1hcCB0aGUgdmlydHVhbCBhZGRyZXNzIHNwYWNlIGZvciBlYWNoIHJl
YWQvd3JpdGUgY2FsbC4gDQo+PiBJIGp1c3QgY2hlY2tlZCB0aGUgTGludXggY29kZVsxXSAgYW5k
IHRoZXJlIGFsc28gbWFwcGluZyBpcyBkb25lIG9uY2Ugbm90IGZvciBlYWNoIHJlYWQvd3JpdGUg
Y2FsbC4NCj4gDQo+IFNvIG15IGd1ZXNzIGlzIHRoYXQgZm9yIGFybTMyIHdlIHdvdWxkIGhhdmUg
dG8gcmVzb3J0IHRvIGR5bmFtaWMNCj4gbWFwL3VubWFwIGZvciBlYWNoIHJlYWQvd3JpdGUgY2Fs
bCwgdW5sZXNzIHRoZXJlIGlzIGEgdHJpY2sgd2l0aCB0aGUNCj4gaW5kaXZpZHVhbCBzbWFsbGVy
IGlvcmVtYXBzIHRoYXQgSSBoYXZlbid0IHNwb3R0ZWQgKGUuZy4gbWF5YmUgc29tZXRoaW5nDQo+
IGRvZXNuJ3QgZ2V0IG1hcHBlZCB0aGF0IHdheT8pDQo+IA0KPiBUaGF0IHNhaWQsIGdpdmVuIHRo
YXQgd2UgYXJlIHVuY2VydGFpbiBhYm91dCB0aGlzIGFuZCB0aGUgYXJtMzINCj4gaW1wbGVtZW50
YXRpb24gaXMgbm93aGVyZSBjbG9zZSwgSSB0aGluayB0aGF0IHdlIGFyZSBPSyB0byBjb250aW51
ZSBsaWtlDQo+IHRoaXMgZm9yIHRoaXMgc2VyaWVzLiBNYXliZSB5b3UgY291bGQgYWRkIGEgY291
cGxlIG9mIHNlbnRlbmNlcyB0byB0aGUNCj4gaW4tY29kZSBjb21tZW50IHNvIHRoYXQgaWYgc29t
ZWJvZHkgd2FudHMgdG8ganVtcCBpbiBhbmQgaW1wbGVtZW50DQo+IGFybTMyIHN1cHBvcnQgdGhl
eSB3b3VsZCBrbm93IHdoZXJlIHRvIHN0YXJ0Lg0KDQpJIGFtIG9rIHdpdGggYm90aCB3YXlzIGFk
ZGluZyBjb21tZW50IGluIGNvZGUgdG8gZXhwbGFpbiBvciBpbXBsZW1lbnQgdGhlIHBjaV9lY2Ft
X2FkZF9idXMoLi4pIGFuZCBwY2lfZWNhbV9yZW1vdmVfYnVzKCkgbGlrZSBMaW51eCBbMV0gYW5k
IA0Kd2UgY2FuIGNhbGwgdGhvc2UgZnVuY3Rpb24gaW4gcGNpX3JlYWQoKS9wY2lfd3JpdGUuDQoN
Cmh0dHBzOi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0L3RvcnZhbGRz
L2xpbnV4LmdpdC90cmVlL2RyaXZlcnMvcGNpL2VjYW0uYyNuMTI2DQoNCg0KUmVnYXJkcywNClJh
aHVsDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 17:53:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 17:53:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188664.337908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQvZ3-0000Iy-HZ; Thu, 16 Sep 2021 17:52:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188664.337908; Thu, 16 Sep 2021 17:52:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQvZ3-0000Ir-EP; Thu, 16 Sep 2021 17:52:49 +0000
Received: by outflank-mailman (input) for mailman id 188664;
 Thu, 16 Sep 2021 17:52:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DjOx=OG=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mQvZ2-0000Il-55
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 17:52:48 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5fa3ee20-62a6-44e9-b943-8a02da2ab043;
 Thu, 16 Sep 2021 17:52:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5fa3ee20-62a6-44e9-b943-8a02da2ab043
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631814766;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=3GDBpvebfB/L2nkRQc5xS7kV2QZ9dw0uo8UFwVeRI6M=;
  b=D3ArgnPKrICvfUUqnn6jGZC3TFIWVyQAfqGQ+VvCnYpLEV9UvY/G6SSr
   EPVd4xSqJ5hC550tvGLyToh3ZvH6qhu+bZdXjO3bi9sfmmBtVErOLgRF0
   HfApM1AU1U+oJQAG42QS5NywpCZ+ahFqQ+LWNU2LNdMUKHRmchm+VBLdX
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: iK7ikIm5m0LJIiMKKLwYYTD+K3Yhx9qmM3h9PfHqkFq3ZYgjKtx8K1mCv7PTzMiHbKQ0fulxCs
 IdX2aHiqbF+REYZoem5FZ2VBkj2CLdwfbjLY/F13tIFmY8hfGvK0TPr1iLHhOCwY8vBxzpeodt
 kZjkxySgiwhC2CA/pi2jGw8eYaEn50sInR0Bh57UW/yCiEKVPU0Gr9zcJuXOKFSbreuqu0PWjv
 16jhgyzoCHd6euZJdm1SL4pU4dHEhXm+ZsdZtxzFOjXoZ64CEIz111NyJbkmZObNulm8Gpn8fd
 ab9noJtJz9LBqdzwlu3EbWmF
X-SBRS: 5.1
X-MesageID: 53329632
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kl9Os6sATwBqdn7FMcr4pR/05OfnVMVZMUV32f8akzHdYApBsoF/q
 tZmKTyOMqyCa2GgKookOYW/phxUuJPXn9FqGVRo/Cg8FXkW+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y52IbhWGthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NplpYOCSCgsOLXwv6dGSh9ZKwxdGot55+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DP
 JFDOGQ0NnwsZTVvM3wWJpwChd6axXXGTxl7sw6JuaMotj27IAtZj+G2bYu9lsaxbcRKnG6Iq
 2Te5WP7DxoGctuFxlKt82mlh+zOm2b3XYMODqyQ5/dsjFCDgHYOYDUUX1ampfiyimalRslSb
 UcT/0IGr7U29UGtZsnwWVu/unHslg4RXZ9cHvM37CmJy7HI+ECJC24cVDlDZdc68sgsSlQCy
 Vuhj97vQzt1v9W9UmmB/72ZqTezPyk9LmIYYyIACwwf7LHeTJob10yVCIw5Sejs04OzSWqYL
 y22QDYWl6ohjOEO0b2H2hOcmjStnrXqaSgK+VCCNo661T+VdLJJdqTxtwOCsKcfc9jAJrWSl
 CNbwJnFtYjiGbnIzXbUEbtXRNlF8t7YaGW0vLJ5I3U2G91BEVaYdIZM6XlVIE5zO67okhe4P
 ReO5Wu9CHJVVUZGjJObgarqUKzGLoC6TLwJs8w4iPIVOvCdkyfdp0lTiba4hTyFraTVufhX1
 W2nnSOQ4ZAyUvwPIN2eHLx17FPW7npmmTO7qW7Tlkz6uVZhWJJlYehcawbfBgzIxIiFvB/U4
 75i2ziikk4EONASlhL/qNZJRXhTdCBTLcmv96R/K77SSiI7STpJI6KAntscl3lNwv09ehHgp
 SrmBCe1CTPX2BX6FOl9Qis/Mei+Ackm9itT0O5FFQ/A5kXPqL2Htc83X5A2YaMm5Kpky/t1R
 OMCYMKOHrJETTGvxtjXRcaVQFVKeEv5iASQETCiZTRjLZdsSxaQoo3vfxf19TlIBS2y7JNsr
 7ql3wLdYJwCWwU9U5qGNKPxlwu87SoHheZ/f0rUOd0PKk/ix5dndn7qhfgtLsBSdRianmmG1
 xybCAszrPXWp9Nn68HAgK2J9t/7E+Z3EkdAMXPc6LK6aXvT8ma5mNcSW+eUZzHNEmjz/fz6N
 +lSyvj9NtwBnUpL7NUgQ+o6k/pm6oK29bFAzwliEHHaVHiRC+ttciucwM1ClqxR3bsF6wG4b
 V2Cp4tBMrKTNcK7TFNIfFg5bv6O3O0/kyXJ6ahnO13z4SJ68ebVUUhWOBXQ2iVRIKEsbdEgy
 OYl/sUX9xa+mlwhNdPf1nJY8GGFL3ohVaQ7t85FXN+321RzklwSM4bBDiLW4Y2Ub4QeO0YnF
 TaYma7eiukO3UHFaXcySSDA0Oc1aU7iY/yWIIvu/2i0p+c=
IronPort-HdrOrdr: A9a23:SiqXiKOhQoo59cBcT1b155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/OxoS5PwPk80kqQFnbX5XI3SITUO3VHHEGgM1/qb/9SNIVyYygcZ79
 YbT0EcMqyBMbEZt7eC3ODQKb9Jq7PmgcPY9ts2jU0dKT2CA5sQnjuRYTzrdHGeKjM2Z6bRWK
 Dsnfau8FGbCAoqh4mAdzU4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kLEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 bxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72zeBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlJXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbZrmGuhHjXkV1RUsZiRtixZJGbAfqFCgL3V79FupgE686NCr/Zv2Evp9/oGOtF5Dq
 r/Q/1VfBwndL5gUUtHPpZ1fSKAMB2Fffv9ChPhHb3ZLtByB5vske+83Fxn3pDmRHQ3pKFC7q
 gpFmko7VIPRw==
X-IronPort-AV: E=Sophos;i="5.85,299,1624334400"; 
   d="scan'208";a="53329632"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P86p+9ImKYpdANxLTf+4uQ6lt7UqK1bDiEqPoelIllJo+JjOM864aqe9VRMhBwyPulPEoJKRQWGT9eFNG00ScNCW9MtgFNaDhLyl1TecIiybXakxvKNEu8Z7aBW69T15U/k19fWawyGazY4si6TQwUuxObmUWah2CoBj1PGWAON9vbBKC5X1ZP8+iIxhygBI1hdb7SLT4Tn4MufePxOm08wCaoRySBiE+W+VEWVgc630Kw45tx9/GmE/egptI+XEK0VZItnnDlqs1GJSe0MunuGTLHoqN5ei2cM6Hxk66nuJVoiEWAbGieyvXe3MHWp64+k0dbrLnxhUHPfozSxSxA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3GDBpvebfB/L2nkRQc5xS7kV2QZ9dw0uo8UFwVeRI6M=;
 b=ZWohzHZng1EwXnLd8UzV220F23b32X4KwDzy6qb4YhY0/S9iNSXkDFD7NuNVnHJS2Psypv1oakqmDu1SBt1zG+1+owzsg8Sbvh928d4yO02EkxnDb3M2fxhnSENHeEqSK6AMA+ToVWrQSww6d9KcNK2hK2vavV51XkiDpQsJQjM8nSYWb+4BCXXZRLxwGIO9C/rh8C2hvAr3PVLz4Bo8KK/D/9UnKAj4imDoT9He4x0HN8/JQ/r4IWqWsLog6ba2Q6kgDa8ciK5tVpnNB2Nx0ssF5NbftFbOzPxu8PTBG5oTvcgirDxIUvIhIhOljr9pJdfYHDINEOXdI3fMOhwhdQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3GDBpvebfB/L2nkRQc5xS7kV2QZ9dw0uo8UFwVeRI6M=;
 b=T5q8RvWXBtg8/4fzSMYSt/H7WpdJQxj47BgTQ7+7sKseLwbcqAU9Ja34cpzXjT/k81JGpqVKkAF+jTpVETq0SrmVPLa3NIGGx3vYzwHiSD1fCmEGqZcGRJlrry+Y1jznitiW5PTq2qCmunqsMyL4pZJI0mdkFd9VjXnpt/tN6QE=
To: Jan Beulich <jbeulich@suse.com>, Dmitry Isaikin <isaikin-dmitry@yandex.ru>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	<xen-devel@lists.xenproject.org>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
Message-ID: <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
Date: Thu, 16 Sep 2021 18:52:20 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO3P265CA0009.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:bb::14) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e4d28485-575a-4849-5f43-08d9793abe68
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5775:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5775C531557EBF75F153FE50BADC9@SJ0PR03MB5775.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: rUGLJttSgHomJyVqAKoz3ja+t04piXVpafC0SZqFM/5soQ0j0/dIGqD3jdZeFLweyEkDbpLUqMumNAeCTaIrO4hBgJRhwnRm+9cX4NqNoS6jxDlxbCvlLNzjHQ3hyzpWn1SMkG9FVHXXGCqQVDOQAPWr/DIuO55j0tkx/PVNkIh2C0t+YMHEiwQJ80IYQpoJyo2DVMQ0zaZTCaHy0vCyPQ4jJB9iDQMAQ3ol19mwOgfVFfr+xEyWOe/ofA+reRnvvfy0/ETPXhauJhyqD0klGBNPd1Qq0fUmHvNwH0/WeklX+UUrHQRHLGgd1oC/TTUwrwpP7f0E82mhUQvCdgleqQveWwWXpOPorY21VcM0v/9Zm1DoPLGjJ4pyuiJrgRwQcHTzXapvkxyXiVmzVY4N+bWHgeyJ+tke4nrggkEezGEdi5BF89xbsU/Zrjpvqw+RDNfqORIBp/kVbm1DPj0sIjqShaTHQHXfEiZw0ambdohlBs4e6IiZcZ4CyoX1YQoD3N/SH1TjmTMZ+bTRlNIk3qmKFtCcJvZxHD7KuPYwBG55mAJTwOv8vVz9mvO2s0yHtiY95Gw7uDcTRlx2IXmtY0qkObQyGPQwYKXwWFWDW3/wyThecsN8yNwtN2ntZEHN0TphUlXt68Fv3lC4qde7fxYGAURgB2fwMCRIqY7qrYQEw2ElWQPf0P5i80TMKXMaLC1bFcPczBlF7yzFZRjX6pMElvQe++ckg0E8JCtIUjlGozhVPU0+Ymxi0hoS7aAhqm81KEDg3RtIr25Jpx0XwA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(2906002)(186003)(316002)(36756003)(110136005)(54906003)(66946007)(66476007)(66556008)(16576012)(31686004)(2616005)(956004)(4744005)(8676002)(38100700002)(8936002)(83380400001)(55236004)(6486002)(6666004)(86362001)(53546011)(5660300002)(31696002)(508600001)(26005)(23180200003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OXdHY1ZtcXF5bjlQdllmQUorRnBxaDlpM3hKS0JmMmlBZFR1MGM5RGVjeHY3?=
 =?utf-8?B?VVdTUGllNndNcVBqeFRTS0k1czMvMWF3ZUx0cTRWN0R5WlZTQVZtdFB5WS9u?=
 =?utf-8?B?VG92MzZZclk2bFVJd3E1ZERURXpOWVArSEI3OEVQaXAyWVNNSTRJaGc1L2ll?=
 =?utf-8?B?V3IvNDhSK0VVVU5DcmtZcW90MnFnMWxxQWYxcVJVYWxrU0h3bTl5YzN6bmFJ?=
 =?utf-8?B?TytsSmJJa3J6NjlyMHRHK3VjRHo0dUhINWRjNmt6bXczR2RJWTVQM3JwaEEx?=
 =?utf-8?B?NHVIeGZ4SjhYN2VFQ3IwNEtsMlVKZE1yMm9leFhNVVl3RjMzSHBDdUdtaEN2?=
 =?utf-8?B?UUZnL0ZUVlBSaG9laHRIWGNEbWlqb1FsamcvM1g3SE1HdnhjSmNreW5vRnBq?=
 =?utf-8?B?QjVwZjJLQ3VaYzNUZmc1SlVlR2NCUFBHSVhQVi9rODVtZVBIK205R3IvaE5O?=
 =?utf-8?B?aTdlN3dUeWsvUUtaVWRlMmY0cXpVMXlTaFZFSlJIZkZ3MlBuUzMrNmZUQXlB?=
 =?utf-8?B?YVQ3cUF2MEhBd0QvUFM2RGdmVEh5TTkxYWpxZ0ZPSzRkWG9IQ2JTWVRTcm5V?=
 =?utf-8?B?TnBPWmtpd1NocDFxQ2J6UmVCaThxeWhIRWdVUEV1eWFOK2tRSWJyeXNnaWZ5?=
 =?utf-8?B?VkJqYThEUTg5cHE4MVc1MHhiczR6NjB5RFdycklBQXd5MStuS0NEWXlXL00v?=
 =?utf-8?B?cHAydXRxZ2VFUXFBRVpQYnEybis4ZzkvY2RTSW56UmZJWEFqdUNUdFNzTkFy?=
 =?utf-8?B?R1luc1BzQWRrYzZCTi9FVWp6OWI2Tmdwa3dESy9UdXk3Q2F3ZkVzLyttOVB1?=
 =?utf-8?B?OURySHpRTm5PcDI3Z3VWcnZ5eld2TVhPeWRuYmN6dldsZ0tLQ1JudFoyRTJU?=
 =?utf-8?B?aFpqU0ZYQnVnanNVYWdmbGYzU080VmkxaEdrWUx5MkJxRWRVclFJTGkxQ2M0?=
 =?utf-8?B?WXVBdnd0MVhzbEg1dFhOQUtpVVJBWE9JdGhqZGhSbjM4L0YybGd5cWp3WnJL?=
 =?utf-8?B?L3gzb0hqeExZVENBb2ZnRUJGZWtOV2FWencrcmNITFczYWJSNy81OXJ4ZnpF?=
 =?utf-8?B?WFQ1RzZNYzhUSW5OV1pxckNxMUhKMUZUcmhRcjY0eGZEWmo4NWI3UzlzWlM3?=
 =?utf-8?B?SkYrV1dQRlNsdU85ZGd0dU13K00rZ0lmY1pmdlY4a0tZZ0JHN0x5R2RQOGNr?=
 =?utf-8?B?c3JQQTlwZDdLNUJsTVE0bzJOemlLcGZUZ2NqeE1adDdvWmhYclh5cWlBL0d2?=
 =?utf-8?B?SVZIUlRMMytPZGJhZjNqWVI0aVpBZTFQOUFPQ2RJdDQ2dVIyZDJFY2ZxelB1?=
 =?utf-8?B?ZGovQ0p3aG9nd1JMWllHYndldHdyd0MveFozalo5dXFyckVCdkdOV2hMSlA1?=
 =?utf-8?B?SVdZTzFUejE1eGZtaVY3Y3gxaVdadG5EeGQ0TzgwdFV3UWlZL0E0ZEdvQUlP?=
 =?utf-8?B?OVNkM202Uno5dHJ3bzZvdnl5SVRSVXRYOHJZRHhTdzQvOWo1Q0FnSGFBancw?=
 =?utf-8?B?VHFtK1pVTXJCSUVRbmRTSkJnN0QxVmwxNlNXMTVyS1hWaUM5cGxrTkhjMEpT?=
 =?utf-8?B?RmJSMDhYZ3JKa2YvWis2R0l3cldGVHVBeUludExEQURMOTF6SU9LQ2YxVkYv?=
 =?utf-8?B?RUdDaVFDREdtaXlSd2lhQURQaTlyUEJzbWRIOEdpNXd6dFFFZHZpQXM2RFMz?=
 =?utf-8?B?dVFyRHBQY0FlZUpqL0xJNXduNENDL1V2MDBiMGJ2b0g4Y1FweVlyckltc1g3?=
 =?utf-8?Q?ErffMO3Ignp3HX85tZnVJTrc0Lskx4IfgYisYZe?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e4d28485-575a-4849-5f43-08d9793abe68
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2021 17:52:26.1070
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: emXLVIiZ6jpxpTWMzXtYopWEqSuk71poXRctJFdDVIZJKKu2PB9GSqI4rL3wIrgeO9Me9aYjhkEhsRn5J3B92MnZ2/wS3rab2B8OeNQwOqo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5775
X-OriginatorOrg: citrix.com

On 16/09/2021 13:30, Jan Beulich wrote:
> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>
>> This significantly speeds up concurrent destruction of multiple domains =
on x86.
> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
> improve locking during domain destruction"). There it was found to
> actually improve things;

Was it?=C2=A0 I recall that it was simply an expectation that performance
would be better...

Amazon previously identified 228ab9992ffb as a massive perf hit, too.

Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
incomplete, and it appears as if it wasn't necessarily a wise move in
hindsight.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 18:34:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 18:34:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188673.337919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQwDH-00056W-QK; Thu, 16 Sep 2021 18:34:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188673.337919; Thu, 16 Sep 2021 18:34:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQwDH-00056P-Mb; Thu, 16 Sep 2021 18:34:23 +0000
Received: by outflank-mailman (input) for mailman id 188673;
 Thu, 16 Sep 2021 18:34:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8L3r=OG=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mQwDG-00056J-Fr
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 18:34:22 +0000
Received: from mail-wr1-x436.google.com (unknown [2a00:1450:4864:20::436])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d43359eb-4a97-40cb-9fd7-db64c17e9ed9;
 Thu, 16 Sep 2021 18:34:21 +0000 (UTC)
Received: by mail-wr1-x436.google.com with SMTP id t8so4006134wri.1
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 11:34:21 -0700 (PDT)
Received: from ?IPv6:2a00:23c5:5785:9a01:6d33:e960:5a2a:9aae?
 ([2a00:23c5:5785:9a01:6d33:e960:5a2a:9aae])
 by smtp.gmail.com with ESMTPSA id k6sm6947250wmo.37.2021.09.16.11.34.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 11:34:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d43359eb-4a97-40cb-9fd7-db64c17e9ed9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:reply-to:subject:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=w4Ekv6N9l8FCIqGomJ6KrcmtU6Dai3vMtJDZaK7FIpM=;
        b=aQJNdADA4sjWyHEyiZB8v/YOimfup3q/yreXaUIsDRkCIWXuEkGURS6rDJLMDBGo8j
         /q9oVBu4VyMzqzJmkCtZZO4Wk+KhZnXKB6K0nK8DfpQaKq3edXaU30WCMQIH2JRok3vr
         0bIbbWxeglawI62Ne9ttrxhN+YsQkmIJiE+KibjbbAAEqy81UR0j+y4oJ/h44DOepE4C
         RILQh364ipYOnZFaRHBJF/GywrJwkXBgd5fiO2zt6kOLwc3jK2NUQtmYYzdxNlAb75fk
         nNnmtmHBit70CIhpc7eU7xfS5gy87sdOHSSlvQxZKGOfDtSthBD58mDltOMv0ZnTIViK
         MMjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:reply-to:subject:to:cc:references
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=w4Ekv6N9l8FCIqGomJ6KrcmtU6Dai3vMtJDZaK7FIpM=;
        b=N8jvcWZbwDoam4AW0QeOKzxzEQJzAjnuTExKCNmlejn9XK1aNZ/AvJhG26LXHQUN70
         TA+fboTyB8XNU6VB4IpjOGxFiTiW2Ip/GplsfFjG5Dm9WVOu5iZel6vVQUubWrDMRpQ5
         y6p8ahYhtJ1nGVT2T0v/0fGbvugAIPdOwFtpSHLigEsd3W8hrofHGUJxRJgI5gkxvb/e
         cayyE0/xUOLHzjrYvI9gXSC/UwiqfCO9VrEGTCZTpnGAcn/reR/GJhomEshdnH+keq9e
         Pg/bK6LYU4GHvvE8KdNalNmw+Yxb3RNtNq8ckHwdHyYk8Ls0v+1ZsxpiQ9WI7MnuzEUc
         3bCA==
X-Gm-Message-State: AOAM531oCJPvj3oiqIiAG8EZxacj7W5/0TXPERLboLDiwm7UH16x7mnk
	En1JxvcCtwqjUbYfhFPAPhg=
X-Google-Smtp-Source: ABdhPJy0sR3FbB/pw8T5ItPYEMv6Lo96GQc/NTvLmcBSZhuqqvbxHEwECve262i1QyUWdJo2iIVcxA==
X-Received: by 2002:a5d:65d0:: with SMTP id e16mr7789848wrw.182.1631817260434;
        Thu, 16 Sep 2021 11:34:20 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Reply-To: paul@xen.org
Subject: =?UTF-8?Q?Re=3a_Ping=c2=b2=3a_=5bPATCH=5d_xen-netback=3a_correct_su?=
 =?UTF-8?Q?ccess/error_reporting_for_the_SKB-with-fraglist_case?=
To: Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>
References: <4dd5b8ec-a255-7ab1-6dbf-52705acd6d62@suse.com>
 <67bc0728-761b-c3dd-bdd5-1a850ff79fbb@xen.org>
 <76c94541-21a8-7ae5-c4c4-48552f16c3fd@suse.com>
 <17e50fb5-31f7-60a5-1eec-10d18a40ad9a@xen.org>
 <57580966-3880-9e59-5d82-e1de9006aa0c@suse.com>
 <a26c1ecd-e303-3138-eb7e-96f0203ca888@xen.org>
 <1a522244-4be8-5e33-77c7-4ea5cf130335@suse.com>
 <9d27a3eb-1d50-64bb-8785-81de1eef3102@suse.com>
 <d4f381e9-6698-3339-1d17-15e3abc71d06@suse.com>
Message-ID: <0dff83ff-629a-7179-9fef-77bd1fbf3d09@xen.org>
Date: Thu, 16 Sep 2021 19:34:19 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <d4f381e9-6698-3339-1d17-15e3abc71d06@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 16/09/2021 16:45, Jan Beulich wrote:
> On 15.07.2021 10:58, Jan Beulich wrote:
>> On 20.05.2021 13:46, Jan Beulich wrote:
>>> On 25.02.2021 17:23, Paul Durrant wrote:
>>>> On 25/02/2021 14:00, Jan Beulich wrote:
>>>>> On 25.02.2021 13:11, Paul Durrant wrote:
>>>>>> On 25/02/2021 07:33, Jan Beulich wrote:
>>>>>>> On 24.02.2021 17:39, Paul Durrant wrote:
>>>>>>>> On 23/02/2021 16:29, Jan Beulich wrote:
>>>>>>>>> When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
>>>>>>>>> special considerations for the head of the SKB no longer apply. Don't
>>>>>>>>> mistakenly report ERROR to the frontend for the first entry in the list,
>>>>>>>>> even if - from all I can tell - this shouldn't matter much as the overall
>>>>>>>>> transmit will need to be considered failed anyway.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>>
>>>>>>>>> --- a/drivers/net/xen-netback/netback.c
>>>>>>>>> +++ b/drivers/net/xen-netback/netback.c
>>>>>>>>> @@ -499,7 +499,7 @@ check_frags:
>>>>>>>>>      				 * the header's copy failed, and they are
>>>>>>>>>      				 * sharing a slot, send an error
>>>>>>>>>      				 */
>>>>>>>>> -				if (i == 0 && sharedslot)
>>>>>>>>> +				if (i == 0 && !first_shinfo && sharedslot)
>>>>>>>>>      					xenvif_idx_release(queue, pending_idx,
>>>>>>>>>      							   XEN_NETIF_RSP_ERROR);
>>>>>>>>>      				else
>>>>>>>>>
>>>>>>>>
>>>>>>>> I think this will DTRT, but to my mind it would make more sense to clear
>>>>>>>> 'sharedslot' before the 'goto check_frags' at the bottom of the function.
>>>>>>>
>>>>>>> That was my initial idea as well, but
>>>>>>> - I think it is for a reason that the variable is "const".
>>>>>>> - There is another use of it which would then instead need further
>>>>>>>      amending (and which I believe is at least part of the reason for
>>>>>>>      the variable to be "const").
>>>>>>>
>>>>>>
>>>>>> Oh, yes. But now that I look again, don't you want:
>>>>>>
>>>>>> if (i == 0 && first_shinfo && sharedslot)
>>>>>>
>>>>>> ? (i.e no '!')
>>>>>>
>>>>>> The comment states that the error should be indicated when the first
>>>>>> frag contains the header in the case that the map succeeded but the
>>>>>> prior copy from the same ref failed. This can only possibly be the case
>>>>>> if this is the 'first_shinfo'
>>>>>
>>>>> I don't think so, no - there's a difference between "first frag"
>>>>> (at which point first_shinfo is NULL) and first frag list entry
>>>>> (at which point first_shinfo is non-NULL).
>>>>
>>>> Yes, I realise I got it backwards. Confusing name but the comment above
>>>> its declaration does explain.
>>>>
>>>>>
>>>>>> (which is why I still think it is safe to unconst 'sharedslot' and
>>>>>> clear it).
>>>>>
>>>>> And "no" here as well - this piece of code
>>>>>
>>>>> 		/* First error: if the header haven't shared a slot with the
>>>>> 		 * first frag, release it as well.
>>>>> 		 */
>>>>> 		if (!sharedslot)
>>>>> 			xenvif_idx_release(queue,
>>>>> 					   XENVIF_TX_CB(skb)->pending_idx,
>>>>> 					   XEN_NETIF_RSP_OKAY);
>>>>>
>>>>> specifically requires sharedslot to have the value that was
>>>>> assigned to it at the start of the function (this property
>>>>> doesn't go away when switching from fragments to frag list).
>>>>> Note also how it uses XENVIF_TX_CB(skb)->pending_idx, i.e. the
>>>>> value the local variable pending_idx was set from at the start
>>>>> of the function.
>>>>>
>>>>
>>>> True, we do have to deal with freeing up the header if the first map
>>>> error comes on the frag list.
>>>>
>>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>>
>>> Since I've not seen this go into 5.13-rc, may I ask what the disposition
>>> of this is?
>>
>> I can't seem to spot this in 5.14-rc either. I have to admit I'm
>> increasingly puzzled ...
> 
> Another two months (and another release) later and still nothing. Am
> I doing something wrong? Am I wrongly assuming that maintainers would
> push such changes up the chain?
> 

It has my R-b so it ought to go in via netdev AFAICT.

   Paul



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 20:16:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 20:16:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188682.337929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxo2-0007l2-D5; Thu, 16 Sep 2021 20:16:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188682.337929; Thu, 16 Sep 2021 20:16:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxo2-0007kv-A0; Thu, 16 Sep 2021 20:16:26 +0000
Received: by outflank-mailman (input) for mailman id 188682;
 Thu, 16 Sep 2021 20:16:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQxo1-0007ko-Q7
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 20:16:25 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8f134838-720e-4091-adf4-13c5a1d206b2;
 Thu, 16 Sep 2021 20:16:24 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 99CF9611EE;
 Thu, 16 Sep 2021 20:16:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f134838-720e-4091-adf4-13c5a1d206b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631823384;
	bh=ZW2rGbgXDtL5/e9pwiE2HXgUIAYjFeyzNO2p0Uc8wkE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JYBPuoCcOqDSVUbz3gyZ+wNoBaMxEmZ7X58C86Z2azh3bVU6X3TBggweCDw+KZydU
	 aSqBb6VPfw7yKavujo/B/iSYVstYU8uAt1T3fhk5hkWsaCrY3sXv4xPulvoywpDn3R
	 xINQGi4+dZANRoIlCq8iawOSEBhR7lMIL4zPE/XLfclE9EnHOz8LjACYBaAEEj4lpA
	 TiD9qBI8hH8KaArjea4usePyawNxiUX7Ahzw8/AsRTbrmfet9g5NHu4Sgag4lGr8Qq
	 WWhlkaF8WgDOOJR3xMBIX15V+mw01mJ8+s4PuaGEKVzD8ZUWjayN2nMRwz1KjcN1U0
	 0v424SB+0nyfQ==
Date: Thu, 16 Sep 2021 13:16:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Luca Fancellu <luca.fancellu@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
In-Reply-To: <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
Message-ID: <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-3-luca.fancellu@arm.com> <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com> <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com> <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com> <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-810505497-1631822111=:21985"
Content-ID: <alpine.DEB.2.21.2109161255180.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-810505497-1631822111=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109161255181.21985@sstabellini-ThinkPad-T480s>

On Thu, 16 Sep 2021, Jan Beulich wrote:
> On 16.09.2021 17:07, Luca Fancellu wrote:
> > I explain here my understanding on dom0less, this feature is used to start domUs at
> > Xen boot in parallel, the name is misleading but it doesn’t require dom0 to be absent.
> > 
> > So if you have a dom0 kernel embed in the image, it's completely fine to start it and it 
> > doesn’t have to be skipped.
> > 
> > Here the possible user cases:
> > 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
> > 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded in Xen image, domUs on DT]
> > 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image) and domUs on DT]
> 
> If that's the intention - fine. Stefano?

What do you mean by dom0 modules embedded in the Xen image? I am not
familiar with it, but I imagine it doesn't involve any multiboot,module
nodes in device tree, right?

Putting aside "dom0 modules embedded in Xen image" that I didn't fully
understand, there are three ways to load Dom0:

- dom0 kernel (and ramdisk, optional) on xen.cfg
- dom0 kernel (and ramdisk, optional) on device tree using the "reg" property
- dom0 kernel (and ramdisk, optional) on device tree using the "uefi,binary" property

Then, the other use cases are:

- true dom0less, domUs on device tree using the "reg" property
- true dom0less, domUs on device tree using the "uefi,binary" property

And of course all the possible combinations between Dom0 and DomU
loading.


Currently, patch #1 checks for the presence of a Dom0 kernel node and, if
present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
node is *not* present, efi_arch_use_config_file returns true.

However, this could be a true dom0less configuration without any Dom0
kernel. If so, you might not want to load xen.cfg at all because it is
not needed. In a true dom0less configuration, we probably want
efi_arch_use_config_file to return false.


In general, loading xen.cfg or not loading xen.cfg doesn't seem like it
should be dependent on whether there is or there is not a dom0 kernel
node in device tree. I think there are too many possible combinations to
guess correctly based on the presence of the dom0 kernel node. Instead,
I think it would be better to have an explicit flag.

Today, if a "multiboot,module" node is present, efi_arch_use_config_file
returns false. So I suggested to extend it so that if a
"multiboot,module" node is present we look into a specific
xen.cfg-loading property and if present return true otherwise false.
This way, we are backward compatible. 
--8323329-810505497-1631822111=:21985--


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 20:22:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 20:22:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188690.337941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxuF-0000vQ-37; Thu, 16 Sep 2021 20:22:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188690.337941; Thu, 16 Sep 2021 20:22:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxuF-0000vJ-05; Thu, 16 Sep 2021 20:22:51 +0000
Received: by outflank-mailman (input) for mailman id 188690;
 Thu, 16 Sep 2021 20:22:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQxuD-0000vC-NE
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 20:22:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3bd534b-9929-4bfe-9b11-d7af97296f2f;
 Thu, 16 Sep 2021 20:22:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 364956105A;
 Thu, 16 Sep 2021 20:22:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3bd534b-9929-4bfe-9b11-d7af97296f2f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631823767;
	bh=qRheOX3jLja53pFFz1L0wW4MRNpUmJ9GnvMcw4YU2f8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=c+djUHbIiz8AisN4w45HiCkuK67+8XsQt0Fshu6yaT4dVkcKZFvPbfMrZxLOuBDr0
	 qIBJelS7t3PgxpbXk87N/gcAkkHvfgC31z+eKMf5N2CPrEc24ye6oYvFyzIjJECFRZ
	 el1unYlemX9fZ+ZnEtPDxvBUCLOmykosz2TJt3atXlynWiRmv+AITXJzIKxsqUmabx
	 a50CWBzD0YS0rTOlNmd9WWq7wG60544QArF12AOJXy1SXE5k88PZhUEDThnlPWYV54
	 2VObdCTvYQmI67AaFb6P6gKGTqwlsO+q6aEEs+ra+DvbZtFUgsf/n3ZO5MJeN9Xfkb
	 5reSygJtQ4bKQ==
Date: Thu, 16 Sep 2021 13:22:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Rahul Singh <rahul.singh@arm.com>, 
    Julien Grall <julien.grall.oss@gmail.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>
Subject: Re: [PATCH 10/11] xen/arm: Do not map PCI ECAM space to Domain-0's
 p2m
In-Reply-To: <4093d5fd-1d44-cc33-9fe2-2695f5d3f16c@epam.com>
Message-ID: <alpine.DEB.2.21.2109161322160.21985@sstabellini-ThinkPad-T480s>
References: <20210903083347.131786-1-andr2000@gmail.com> <20210903083347.131786-11-andr2000@gmail.com> <35f7faf6-db90-f279-8ed1-fa4ba96812fb@xen.org> <468a868c-1183-e05f-0c92-3cba172cecb3@epam.com> <9ec2c22c-b834-1c87-71af-236e13538c4a@xen.org>
 <15a930ff-77d5-b962-b346-c586a2769009@epam.com> <684b3534-40eb-add7-f46e-c6258904757b@xen.org> <alpine.DEB.2.21.2109101325110.10523@sstabellini-ThinkPad-T480s> <CAJ=z9a1pSoLpesjqNTiG3-t4+pvju3EetYzcFpuNzMdRWi1GYg@mail.gmail.com> <3ecfc742-b720-0381-dbd8-7147615494c8@epam.com>
 <1e71ebba-b2d3-1201-05ac-e035f182226f@epam.com> <alpine.DEB.2.21.2109141731410.21985@sstabellini-ThinkPad-T480s> <ee38bc53-c590-24d2-9061-3d4fd33f58bd@epam.com> <alpine.DEB.2.21.2109151246450.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109151311480.21985@sstabellini-ThinkPad-T480s> <4093d5fd-1d44-cc33-9fe2-2695f5d3f16c@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-6714969-1631823767=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-6714969-1631823767=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 Sep 2021, Oleksandr Andrushchenko wrote:
> On 15.09.21 23:19, Stefano Stabellini wrote:
> > On Wed, 15 Sep 2021, Stefano Stabellini wrote:
> >> On Wed, 15 Sep 2021, Oleksandr Andrushchenko wrote:
> >>> On 15.09.21 03:36, Stefano Stabellini wrote:
> >>>> On Tue, 14 Sep 2021, Oleksandr Andrushchenko wrote:
> >>>>> With the patch above I have the following log in Domain-0:
> >>>>>
> >>>>> generic-armv8-xt-dom0 login: root
> >>>>> root@generic-armv8-xt-dom0:~# (XEN) *** Serial input to Xen (type 'CTRL-a' three times to switch input)
> >>>>> (XEN) ==== PCI devices ====
> >>>>> (XEN) ==== segment 0000 ====
> >>>>> (XEN) 0000:03:00.0 - d0 - node -1
> >>>>> (XEN) 0000:02:02.0 - d0 - node -1
> >>>>> (XEN) 0000:02:01.0 - d0 - node -1
> >>>>> (XEN) 0000:02:00.0 - d0 - node -1
> >>>>> (XEN) 0000:01:00.0 - d0 - node -1
> >>>>> (XEN) 0000:00:00.0 - d0 - node -1
> >>>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> >>>>>
> >>>>> root@generic-armv8-xt-dom0:~# modprobe e1000e
> >>>>> [   46.104729] e1000e: Intel(R) PRO/1000 Network Driver
> >>>>> [   46.105479] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
> >>>>> [   46.107297] e1000e 0000:03:00.0: enabling device (0000 -> 0002)
> >>>>> (XEN) map [e0000, e001f] -> 0xe0000 for d0
> >>>>> (XEN) map [e0020, e003f] -> 0xe0020 for d0
> >>>>> [   46.178513] e1000e 0000:03:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
> >>>>> [   46.189668] pci_msi_setup_msi_irqs
> >>>>> [   46.191016] nwl_compose_msi_msg msg at fe440000
> >>>>> (XEN) traps.c:2014:d0v0 HSR=0x00000093810047 pc=0xffff8000104b4b00 gva=0xffff800010fa5000 gpa=0x000000e0040000
> >>>>> [   46.200455] Unhandled fault at 0xffff800010fa5000
> >>>>>
> >>>>> [snip]
> >>>>>
> >>>>> [   46.233079] Call trace:
> >>>>> [   46.233559]  __pci_write_msi_msg+0x70/0x180
> >>>>> [   46.234149]  pci_msi_domain_write_msg+0x28/0x30
> >>>>> [   46.234869]  msi_domain_activate+0x5c/0x88
> >>>>>
> >>>>>    From the above you can see that BARs are mapped for Domain-0 now
> >>>>>
> >>>>> only when an assigned PCI device gets enabled in Domain-0.
> >>>>>
> >>>>> Another thing to note is that we crash on MSI-X access as BARs are mapped
> >>>>>
> >>>>> to the domain while enabling memory decoding in the COMMAND register,
> >>>>>
> >>>>> but MSI-X are handled differently, e.g. we have MSI-X holes in the mappings.
> >>>>>
> >>>>> This is because before this change the whole PCI aperture was mapped into
> >>>>>
> >>>>> Domain-0 and it is not. Thus, MSI-X holes are left unmapped now and there
> >>>>>
> >>>>> was no need to do so, e.g. they were always mapped into Domain-0 and
> >>>>>
> >>>>> emulated for guests.
> >>>>>
> >>>>> Please note that one cannot use xl pci-attach in this case to attach the PCI device
> >>>>>
> >>>>> in question to Domain-0 as (please see the log) that device is already attached.
> >>>>>
> >>>>> Neither it can be detached and re-attached. So, without mapping MSI-X holes for
> >>>>>
> >>>>> Domain-0 the device becomes unusable in Domain-0. At the same time the device
> >>>>>
> >>>>> can be successfully passed to DomU.
> >>>>>
> >>>>>
> >>>>> Julien, Stefano! Please let me know how can we proceed with this.
> >>>> What was the plan for MSI-X in Dom0?
> >>> It just worked because we mapped everything
> >>>> Given that Dom0 interacts with a virtual-ITS and virtual-GIC rather than
> >>>> a physical-ITS and physical-GIC, I imagine that it wasn't correct for
> >>>> Dom0 to write to the real MSI-X table directly?
> >>>>
> >>>> Shouldn't Dom0 get emulated MSI-X tables like any DomU?
> >>>>
> >>>> Otherwise, if Dom0 is expected to have the real MSI-X tables mapped, then
> >>>> I would suggest to map them at the same time as the BARs. But I am
> >>>> thinking that Dom0 should get emulated MSI-X tables, not physical MSI-X
> >>>> tables.
> >>> Yes, it seems more than reasonable to enable emulation for Domain-0
> >>>
> >>> as well. Other than that, Stefano, do you think we are good to go with
> >>>
> >>> the changes I did in order to unmap everything for Domain-0?
> >>
> >> It might be better to resend the series with the patch in it, because it
> >> is difficult to review the patch like this.
> 
> This is true. Taking into account Xen release plan I am just trying to
> 
> minimize the turn around here. Sorry about this
> 
> >>   Nonetheless I tried, but I
> >> might have missed something.
> Thank you for your time!!
> >>
> >> Previously the whole PCIe bridge aperture was mapped to Dom0, and
> >> it was done by map_range_to_domain, is that correct?
> 
> Yes, but not only the aperture: please see below.
> 
> 
> >>
> >> Now this patch, to avoid mapping the entire aperture to Dom0, is
> >> skipping any operations for PCIe devices in map_range_to_domain by
> >> setting need_mapping = false.
> >>
> >> The idea is that instead, we'll only map things when needed and not the
> >> whole aperture. However, looking at the changes to
> >> pci_host_bridge_mappings (formerly known as
> >> pci_host_bridge_need_p2m_mapping), it is still going through the full
> >> list of address ranges of the PCIe bridge and map each range one by one
> >> using map_range_to_domain. Also, pci_host_bridge_mappings is still
> >> called unconditionally at boot for Dom0.
> >>
> >> So I am missing the part where the aperture is actually *not* mapped to
> >> Dom0.
> With map_range_to_domain we also mapped all the entries
> 
> of the "reg" and "ranges" properties. Let's have a look at [1]:
> 
> - ranges         : As described in IEEE Std 1275-1994, but must provide
>                     at least a definition of non-prefetchable memory. One
>                     or both of prefetchable Memory and IO Space may also
>                     be provided.
> 
> - reg            : The Configuration Space base address and size, as accessed
>                     from the parent bus.  The base address corresponds to
>                     the first bus in the "bus-range" property.  If no
>                     "bus-range" is specified, this will be bus 0 (the default).
> 
> The most interesting comes when "reg" also has other then configuration
> 
> space addresses, for example [2]:
> 
> - reg: Should contain rc_dbi, config registers location and length.
> - reg-names: Must include the following entries:
>    "rc_dbi": controller configuration registers;
>    "config": PCIe configuration space registers.
> 
> So, we don't need to map "ranges" and *config* from the "reg", but all
> 
> the rest from "reg" still needs to be mapped to Domain-0, so the PCIe
> 
> bridge can remain functional in Domain-0.
> 
> >>   What's the difference between the loop in
> >> pci_host_bridge_mappings:
> >>
> >>    for ( i = 0; i < dt_number_of_address(dev); i++ )
> >>      map_range_to_domain...
> >>
> >> and the previous code in map_range_to_domain? I think I am missing
> >> something but as mentioned it is difficult to review the patch like this
> >> out of order.
> >>
> >> Also, and this is minor, even if currently unused, it might be good to
> >> keep a length parameter to pci_host_bridge_need_p2m_mapping.
> > It looks like the filtering is done based on:
> >
> > return cfg->phys_addr != addr
> 
> As I explained above it is *now* the only range that we *do not want* to
> 
> be mapped to Domain-0. Other "reg" entries still need to be mapped.
> 
> >
> > in pci_ecam_need_p2m_mapping that is expected to filter out the address
> > ranges we don't want to map because it comes from
> > xen/arch/arm/pci/pci-host-common.c:gen_pci_init:
> >
> >      /* Parse our PCI ecam register address*/
> >      err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size);
> >      if ( err )
> >          goto err_exit;
> >
> > In pci_host_bridge_mappings, instead of parsing device tree again, can't
> > we just fetch the address and length we need to map straight from
> > bridge->sysdata->phys_addr/size ?
> 
> We can't as the address describes the configuration space which we
> 
> *do not* want to be mapped, but what we want is other than configuration
> 
> entries in the "reg" property.
> 
> >
> > At the point when pci_host_bridge_mappings is called in your new patch,
> > we have already initialized all the PCIe-related data structures. It
> > seems a bit useless to have to go via device tree again.
> 
> Bottom line: we do need to go over the "reg" property and map the regions
> 
> of which PCIe bridge is dependent and only skip "config" part of it.

OK, thanks for the explanation, please add it to the commit message
and/or as an in-code comment when you resend the patch
--8323329-6714969-1631823767=:21985--


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 20:26:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 20:26:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188698.337951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxyC-0001b6-KD; Thu, 16 Sep 2021 20:26:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188698.337951; Thu, 16 Sep 2021 20:26:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQxyC-0001az-HF; Thu, 16 Sep 2021 20:26:56 +0000
Received: by outflank-mailman (input) for mailman id 188698;
 Thu, 16 Sep 2021 20:26:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQxyB-0001at-6t
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 20:26:55 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6e6c1e26-172c-11ec-b644-12813bfff9fa;
 Thu, 16 Sep 2021 20:26:54 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6ED8C60F26;
 Thu, 16 Sep 2021 20:26:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e6c1e26-172c-11ec-b644-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631824013;
	bh=5NdsIIYXCWwDK+C4KP7XV0yFDXJzb/4hB75/tZKrnDc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Nhb2md3DtLetR4FeTZFEV8aQX3zkDtoaNcgJIqOszLc5LroXdYvoGThPG5O/idXJ9
	 04EGj9kPCbiwwy3GH+Q/s2JG4iC7SfR1OepzoYulZegtuYbBnv3du8xuCjRHuqgppv
	 CpeQWN9xN7c+Hgpo+MU0v9/Uy8Bo/feXWJVBUvULz/Lia0LIRO9a+Xxh/CqNgdzPnK
	 ozPgoEp2HBAzIwNIQUDDGO7L3MGUC2tqC68HWn1b9MrZSmCjzsauxp6GHNx8zh7rD4
	 KwAw6W8ZCTzc82BZYSkPTv4TVHD7G6ivbSIRwUK82hVHnDzJn6UpSQFXXCI4uzFk4x
	 5yjKrZRFT0msQ==
Date: Thu, 16 Sep 2021 13:26:52 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Jan Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
In-Reply-To: <A90145B0-DD30-4300-8248-73104C782172@arm.com>
Message-ID: <alpine.DEB.2.21.2109161324200.21985@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s> <A90145B0-DD30-4300-8248-73104C782172@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1341339614-1631824013=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1341339614-1631824013=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 16 Sep 2021, Rahul Singh wrote:
> Hi Stefano,
> 
> > On 10 Sep 2021, at 1:26 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 19 Aug 2021, Rahul Singh wrote:
> >> The existing VPCI support available for X86 is adapted for Arm.
> >> When the device is added to XEN via the hyper call
> >> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> >> access is added to the Xen to emulate the PCI devices config space.
> > 
> > This is done just for device discovery, right?
> > 
> > Although it is currently not implemented (and I am not asking to
> > implement it now, I am only trying to understand the architecture), it
> > would be possible to discover all PCI devices just by walking down the
> > PCI hierarchy by ourselves in Xen (no Dom0 interactions) given that we
> > have an ECAM driver.
> > 
> > I take that would be the way to implement PCI support for Dom0less?
> 
> It is not possible to discover PCI devices in XEN if enumeration is not done before XEN boot.
> If boot firmware did the enumeration,  XEN will discover the PCI device.

OK, but if the boot firmware does the enumeration, how will Xen discover
the PCI device exactly? Will Xen discover it because the PCI device will
be present in device tree explicitly (will have its own device tree
node)? Or will Xen discover it by walking the PCI bus?
--8323329-1341339614-1631824013=:21985--


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 20:47:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 20:47:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188707.337963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyHr-0004Vv-C4; Thu, 16 Sep 2021 20:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188707.337963; Thu, 16 Sep 2021 20:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyHr-0004Vo-9F; Thu, 16 Sep 2021 20:47:15 +0000
Received: by outflank-mailman (input) for mailman id 188707;
 Thu, 16 Sep 2021 20:47:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQyHp-0004Vi-HP
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 20:47:13 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 56da84a9-0238-4d8d-a0c1-a090dff96290;
 Thu, 16 Sep 2021 20:47:12 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A24A26105A;
 Thu, 16 Sep 2021 20:47:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56da84a9-0238-4d8d-a0c1-a090dff96290
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631825231;
	bh=hUUxl6qnTkEwYaeIJkEF9g5jwBeOS7mM8Haq/htk6Z0=;
	h=Date:From:To:cc:Subject:From;
	b=ICYUStp/U0HV9KnebF+TKWAF8X+IQSBN2WH3in68bQImOlpk/KvikyKqpeQXIei3s
	 HpTimv0ANfWRK5z8NNqZblLs2Wiq1UhwWjQrt4ylKUofZseGmTrNVIf1D8j/bzr6aE
	 d9EXT0PEKbakf1BtblJCAGecKzNxGHfnCZiYrO/eVAK41draXBETr6Fx6H6DdhbNQr
	 cnxhI8z54ihJVQopl0eIzqJYQvpdwY7gFP1J4OHDWeB5IU2wm8kHG+DUzqUrM9BA/A
	 tbdFdvTdItqNjwjFMejefr269c7FfM8eyZ4t2WFyqUEhO19lc1t9op3Aq3ICXnezMD
	 LkYePmE74tAoQ==
Date: Thu, 16 Sep 2021 13:47:11 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: jbeulich@suse.com
cc: sstabellini@kernel.org, penny.zheng@arm.com, Bertrand.Marquis@arm.com, 
    Wei.Chen@arm.com, xen-devel@lists.xenproject.org, julien@xen.org
Subject: [PATCH] modify acquire_domstatic_pages to take an unsigned int size
 parameter
Message-ID: <alpine.DEB.2.21.2109161341270.21985@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

acquire_domstatic_pages currently takes an unsigned long nr_mfns
parameter, but actually it cannot handle anything larger than an
unsigned int nr_mfns. That's because acquire_domstatic_pages is based on
assign_pages which also takes an unsigned int nr parameter.

So modify the nr_mfns parameter of acquire_domstatic_pages to be
unsigned int.

There is only one caller in
xen/arch/arm/domain_build.c:allocate_static_memory. Check that the value
to be passed to acquire_domstatic_pages is no larger than UINT_MAX. If
it is, print an error and goto fail.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---

Jan, I took your suggestion of moving the check closer to where the
value is read from DT. At that point I also took the opportunity to
change acquire_domstatic_pages to take an unsigned int parameter
instead of unsigned long.


diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 62ab7d0ead..d233d634c1 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -554,6 +554,12 @@ static void __init allocate_static_memory(struct domain *d,
         device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize);
         ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE));
 
+        if ( PFN_DOWN(psize) > UINT_MAX )
+        {
+            printk(XENLOG_ERR "%pd: static memory size too large: %#"PRIpaddr,
+                   d, psize);
+            goto fail;
+        }
         smfn = maddr_to_mfn(pbase);
         res = acquire_domstatic_pages(d, smfn, PFN_DOWN(psize), 0);
         if ( res )
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index b9441cb06f..b64c07ae92 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2714,7 +2714,7 @@ static struct page_info * __init acquire_staticmem_pages(mfn_t smfn,
  * then assign them to one specific domain #d.
  */
 int __init acquire_domstatic_pages(struct domain *d, mfn_t smfn,
-                                   unsigned long nr_mfns, unsigned int memflags)
+                                   unsigned int nr_mfns, unsigned int memflags)
 {
     struct page_info *pg;
 
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index dd49237e86..5db26ed477 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -89,7 +89,7 @@ bool scrub_free_pages(void);
 /* These functions are for static memory */
 void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
                           bool need_scrub);
-int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned long nr_mfns,
+int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned int nr_mfns,
                             unsigned int memflags);
 #endif
 



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 20:57:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 20:57:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188714.337974 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQySB-0006BA-CH; Thu, 16 Sep 2021 20:57:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188714.337974; Thu, 16 Sep 2021 20:57:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQySB-0006B3-98; Thu, 16 Sep 2021 20:57:55 +0000
Received: by outflank-mailman (input) for mailman id 188714;
 Thu, 16 Sep 2021 20:57:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Js+i=OG=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mQyS9-0006Au-FJ
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 20:57:53 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 25f49dfd-9386-4e37-90d9-c459a5214bdb;
 Thu, 16 Sep 2021 20:57:51 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id p29so23653911lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 13:57:51 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id x1sm465924ljh.113.2021.09.16.13.57.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 16 Sep 2021 13:57:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25f49dfd-9386-4e37-90d9-c459a5214bdb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=PBwMpJ8RXnLuJPR1IIalhXLuam6FXctzmKajI1mHnsU=;
        b=QbnOjHJrwCrpx9QWhcszQ5L5LfLqNa5A80YISlWpKQDBf/Ybdtqwz8KiUT6eUxhxpe
         ScZP5ogAS/jEvRCpotl76lLUgIrSMKnxKajE1wU37dg5+5LZfVXxX2VHsy+KXVldK8z0
         s1yqxu2+4Mi+ohXIW3ZT4IVZNTMVlFlaZGVaiyBQI+PmUre0zO+dWga5OPAarVrIVLGD
         BPUVvdVvThbE0RC1UIigVdeUQR3jdHPhId33RcgUvZUBwBWR1pkCgkMYkVAmsMEK3iVx
         hvYr6RJkNsQ0sEJJEnWpSaLlYONl/gmFz2TVKutVKG9fUL4EqiajQ9B3ELwtyon1/drV
         Nrdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=PBwMpJ8RXnLuJPR1IIalhXLuam6FXctzmKajI1mHnsU=;
        b=6FwUEC1Rs3Rs7CHRyyaxXOxe7gi+6237XaebN/LTv81s63qmZ9GrpJOIPEcdhEArqY
         iP8yAk6YPntutAosyr4CXArUsMo+t2EVsqC50RNxpt8HAGX6Yf8BC806CSRqDfJdAUHK
         YN0DvZgHk/0dukyTbi3DBe/usZ7E71pHzNK7RmpknAg4BjG4/4BWltTpnDEBmULJotxT
         UgfmHaUkGBzk1D800JZSw+oO7xRDWanhxelM5vGoS/aof0K8hCp4oYvQ8K03EyKChxZE
         3kjmu6wIl+RGqkJk9eoJqLbdt1o88Bg5Tds3VRAA7/ptLDJjHbCWpxHgeWETgptmNyaX
         hQ/Q==
X-Gm-Message-State: AOAM532ZzuAYKUslMNWGf1cwRYRcF0pSxgleS0HwbDnr6LrPCbe/MObL
	E0k4bLk7pLv1onAbqwb+FtM=
X-Google-Smtp-Source: ABdhPJwoiNdjd1zh5Hsgj5SQiMoYabTLGMqud3Na3vBKBo1TyEUE6B3WxcUerWOG35Y41hpPwrQfsg==
X-Received: by 2002:a2e:9296:: with SMTP id d22mr6498843ljh.413.1631825869866;
        Thu, 16 Sep 2021 13:57:49 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
 <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
 <alpine.DEB.2.21.2109151352100.21985@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <b349e43a-91d9-16ba-57c6-34e790b8b31c@gmail.com>
Date: Thu, 16 Sep 2021 23:57:48 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109151352100.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 16.09.21 00:21, Stefano Stabellini wrote:

Hi Stefano

> On Wed, 15 Sep 2021, Oleksandr wrote:
>>> On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>
>>>> The extended region (safe range) is a region of guest physical
>>>> address space which is unused and could be safely used to create
>>>> grant/foreign mappings instead of wasting real RAM pages from
>>>> the domain memory for establishing these mappings.
>>>>
>>>> The extended regions are chosen at the domain creation time and
>>>> advertised to it via "reg" property under hypervisor node in
>>>> the guest device-tree. As region 0 is reserved for grant table
>>>> space (always present), the indexes for extended regions are 1...N.
>>>> If extended regions could not be allocated for some reason,
>>>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>>>
>>>> Please note the following limitations:
>>>> - The extended region feature is only supported for 64-bit domain.
>>>> - The ACPI case is not covered.
>>>>
>>>> ***
>>>>
>>>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>>>> the algorithm to choose extended regions for it is different
>>>> in comparison with the algorithm for non-direct mapped DomU.
>>>> What is more, that extended regions should be chosen differently
>>>> whether IOMMU is enabled or not.
>>>>
>>>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>>>> holes found in host device-tree if otherwise. Make sure that
>>>> extended regions are 2MB-aligned and located within maximum possible
>>>> addressable physical memory range. The maximum number of extended
>>>> regions is 128.
>>>>
>>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>> ---
>>>> Changes since RFC:
>>>>      - update patch description
>>>>      - drop uneeded "extended-region" DT property
>>>> ---
>>>>
>>>>    xen/arch/arm/domain_build.c | 226
>>>> +++++++++++++++++++++++++++++++++++++++++++-
>>>>    1 file changed, 224 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>> index 206038d..070ec27 100644
>>>> --- a/xen/arch/arm/domain_build.c
>>>> +++ b/xen/arch/arm/domain_build.c
>>>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct
>>>> domain *d,
>>>>        return res;
>>>>    }
>>>>    +static int __init add_ext_regions(unsigned long s, unsigned long e,
>>>> void *data)
>>>> +{
>>>> +    struct meminfo *ext_regions = data;
>>>> +    paddr_t start, size;
>>>> +
>>>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>>>> +        return 0;
>>>> +
>>>> +    /* Both start and size of the extended region should be 2MB aligned
>>>> */
>>>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>>>> +    if ( start > e )
>>>> +        return 0;
>>>> +
>>>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>>>> +    if ( !size )
>>>> +        return 0;
>>> Can't you align size as well?
>>>
>>>     size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
>> I am sorry, I don't entirely get what you really meant here. We get both start
>> and size 2MB-aligned by the calculations above
>> (when calculating an alignment, we need to make sure that "start_passed <=
>> start_aligned && size_aligned <= size_passed").
>> If I add the proposing string after, I will reduce the already aligned size by
>> 2MB.
>> If I replace the size calculation with the following, I will get the reduced
>> size even if the passed region is initially 2MB-aligned, so doesn't need to be
>> adjusted.
>> size = e - s + 1;
>> size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
> Sorry I misread your original code, I think it was working as intended
> except for the "+1". I think it should be:
>
>    size = (e - start) & ~(SZ_2M - 1);
But why without "+1"? Isn't "e" here the *last address* of passed range?
Without "+1" I get non entirely correct calculations, last valid 2MB is 
missed.

[snip]
(XEN) Extended region 14: 0x580000000->0x5ffe00000
(XEN) Extended region 15: 0x680000000->0x6ffe00000
(XEN) Extended region 16: 0x780000000->0xffffe00000

But should get:

[snip]
(XEN) Extended region 15: 0x580000000->0x600000000
(XEN) Extended region 16: 0x680000000->0x700000000
(XEN) Extended region 17: 0x780000000->0x10000000000

Let's consider how a hole between (for example) RAM bank 1 and bank 2 is 
calculated:
(XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1 with size 
0x80000000
(XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2 with size 
0x80000000
So the hole size should also be 0x80000000.
If we pass these RAM banks to rangeset_remove_range() one by one:
1: s = 0x500000000 e = 0x57FFFFFFF
2. s = 0x600000000 e = 0x67FFFFFFF
we get s = 0x580000000 e = 0x5FFFFFFFF in add_ext_regions(), where "e" 
is the last address of the hole (not the first address out of the hole), 
so I think, that for proper size calculation we need to add 1 to "e - 
s". Or I really missed something?


>
>>>> + */
>>>> +#define EXT_REGION_START   0x40000000ULL
>>>> +#define EXT_REGION_END     0x80003fffffffULL
>>>> +
>>>> +static int __init find_unallocated_memory(const struct kernel_info
>>>> *kinfo,
>>>> +                                          struct meminfo *ext_regions)
>>>> +{
>>>> +    const struct meminfo *assign_mem = &kinfo->mem;
>>>> +    struct rangeset *unalloc_mem;
>>>> +    paddr_t start, end;
>>>> +    unsigned int i;
>>>> +    int res;
>>>> +
>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>> +
>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>> +    if ( !unalloc_mem )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    /* Start with all available RAM */
>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>> +    {
>>>> +        start = bootinfo.mem.bank[i].start;
>>>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
>>> Is the -1 needed? Isn't it going to screw up the size calculation later?
>> I thought, it was needed. The calculation seems correct.
> I think that normally for an example MMIO region:
>
> start = 0x48000000
> size  = 0x40000000
> end   = 0x88000000
>
> So end = start + size and points to the first address out of the range.
> In other words, 0x88000000 doesn't actually belong to the MMIO region in
> the example.
>
> But here you are passing addresses to rangeset_add_range and other
> rangeset functions and I think rangeset takes *inclusive* addresses as
> input. So you need to pass start and end-1 because end-1 is the last
> address of the MMIO region.
>
> In fact you can see for instance in map_range_to_domain:
>
>          res = iomem_permit_access(d, paddr_to_pfn(addr),
>                  paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
>
> Where iomem_permit_access is based on rangeset. So for clarity, I would
> do:
>
> start = assign_mem->bank[i].start;
> end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> res = rangeset_remove_range(unalloc_mem, start, end - 1);
>
> So that we don't get confused on the meaning of "end" which everywhere
> else means the first address not in range.

I got your point, I will update the code if it much cleaner.


>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove RAM assigned to Dom0 */
>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>> +    {
>>>> +        start = assign_mem->bank[i].start;
>>>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove reserved-memory regions */
>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>> +    {
>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove grant table region */
>>>> +    start = kinfo->gnttab_start;
>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +    if ( res )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>>> +               start, end);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    start = EXT_REGION_START;
>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>> +                                 add_ext_regions, ext_regions);
>>>> +    if ( res )
>>>> +        ext_regions->nr_banks = 0;
>>>> +    else if ( !ext_regions->nr_banks )
>>>> +        res = -ENOENT;
>>>> +
>>>> +out:
>>>> +    rangeset_destroy(unalloc_mem);
>>>> +
>>>> +    return res;
>>>> +}
>>>> +
>>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>>> +                                    struct meminfo *ext_regions)
>>>> +{
>>>> +    struct dt_device_node *np;
>>>> +    struct rangeset *mem_holes;
>>>> +    paddr_t start, end;
>>>> +    unsigned int i;
>>>> +    int res;
>>>> +
>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>> +
>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>> +    if ( !mem_holes )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    /* Start with maximum possible addressable physical memory range */
>>>> +    start = EXT_REGION_START;
>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>> +    if ( res )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>> +               start, end);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
>>>> +    dt_for_each_device_node( dt_host, np )
>>> Don't you need something like device_tree_for_each_node ?
>>> dt_for_each_device_node won't go down any deeper in the tree?
>> Thank you for pointing this out, I will investigate and update the patch.
>>
>>
>>> Alternatively, maybe we could simply record the highest possible address
>>> of any memory/device/anything as we scan the device tree with
>>> handle_node. Then we can use that as the starting point here.
>> I also don't like the idea to scan the DT much, but I failed to find an
>> effective solution how to avoid that.
>> Yes, we can record the highest possible address, but I am afraid, I didn't
>> entirely get a suggestion. Is the suggestion to provide a single region
>> starting from highest possible address + 1 and up to the EXT_REGION_END
>> suitably aligned? Could you please clarify?
> Yes, that is what I was suggesting as a possible alternative: start from
> the highest possible address in DT + 1 and up to the EXT_REGION_END
> suitably aligned. But that wouldn't solve the <4GB issue.
>
>>>> +                goto out;
>>>> +            }
>>>> +
>>>> +            start = addr & PAGE_MASK;
>>>> +            end = PAGE_ALIGN(addr + size) - 1;
>>>> +            res = rangeset_remove_range(mem_holes, start, end);
>>>> +            if ( res )
>>>> +            {
>>>> +                printk(XENLOG_ERR "Failed to remove:
>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>> +                       start, end);
>>>> +                goto out;
>>>> +            }
>>>> +        }
>>>> +    }
>>> As is, it will result in a myriad of small ranges which is unuseful and
>>> slow to parse. I suggest to simplify it by removing a larger region than
>>> strictly necessary. For instance, you could remove a 1GB-aligned and
>>> 1GB-multiple region for each range. That way, you are going to get fewer
>>> large free ranges instance of many small ones which we don't need.
>> I agree with you that a lot of small ranges increase the bookkeeping in Dom0
>> and there is also a theoretical (?) possibility
>> that small ranges occupy all space we provide for extended regions
>> (NR_MEM_BANKS)...
>> But, let's consider my setup as an example again, but when the IOMMU is
>> enabled for Dom0 ("holes found in DT").
>>
>> --- The RAM configuration is the same:
>>
>> (XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
>> (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
>> (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
>> (XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3
>>
>> --- There are a lot of various platform devices with reg property described in
>> DT, I will probably not post all IO ranges here, just say that mostly all of
>> them to be mapped at 0xE0000000-0xFFFFFFFF.
>>
>> --- As we only pick up ranges with size >= 2MB, the calculated extended
>> regions are (based on 40-bit IPA):
>>
>> (XEN) Extended region 0: 0x40000000->0x47e00000
>> (XEN) Extended region 1: 0xc0000000->0xe6000000
>> (XEN) Extended region 2: 0xe7000000->0xe7200000
>> (XEN) Extended region 3: 0xe7400000->0xe7600000
>> (XEN) Extended region 4: 0xe7800000->0xec000000
>> (XEN) Extended region 5: 0xec200000->0xec400000
>> (XEN) Extended region 6: 0xec800000->0xee000000
>> (XEN) Extended region 7: 0xee600000->0xee800000
>> (XEN) Extended region 8: 0xeea00000->0xf1000000
>> (XEN) Extended region 9: 0xf1200000->0xfd000000
>> (XEN) Extended region 10: 0xfd200000->0xfd800000
>> (XEN) Extended region 11: 0xfda00000->0xfe000000
>> (XEN) Extended region 12: 0xfe200000->0xfe600000
>> (XEN) Extended region 13: 0xfec00000->0xff800000
>> (XEN) Extended region 14: 0x100000000->0x500000000
>> (XEN) Extended region 15: 0x580000000->0x600000000
>> (XEN) Extended region 16: 0x680000000->0x700000000
>> (XEN) Extended region 17: 0x780000000->0x10000000000
>>
>> So, if I *correctly* understood your idea about removing 1GB-aligned
>> 1GB-multiple region for each range we would get the following:
>>
>> (XEN) Extended region 0: 0x100000000->0x500000000
>> (XEN) Extended region 1: 0x580000000->0x600000000
>> (XEN) Extended region 2: 0x680000000->0x700000000
>> (XEN) Extended region 3: 0x780000000->0x10000000000
>>
>> As you can see there are no extended regions below 4GB at all. I assume, it
>> would be good to provide them for 1:1 mapped Dom0 (for 32-bit DMA devices?)
>> What else worries me is that IPA size could be 36 or even 32. So, I am afraid,
>> we might even fail to find extended regions above 4GB.
>>
>>
>> I think, if 2MB is considered small enough to bother with, probably we should
>> go with something in between (16MB, 32MB, 64MB).
>> For example, we can take into the account ranges with size >= 16MB:
>>
>> (XEN) Extended region 0: 0x40000000->0x47e00000
>> (XEN) Extended region 1: 0xc0000000->0xe6000000
>> (XEN) Extended region 2: 0xe7800000->0xec000000
>> (XEN) Extended region 3: 0xec800000->0xee000000
>> (XEN) Extended region 4: 0xeea00000->0xf1000000
>> (XEN) Extended region 5: 0xf1200000->0xfd000000
>> (XEN) Extended region 6: 0x100000000->0x500000000
>> (XEN) Extended region 7: 0x580000000->0x600000000
>> (XEN) Extended region 8: 0x680000000->0x700000000
>> (XEN) Extended region 9: 0x780000000->0x10000000000
>>
>> Any thoughts?
> Yeah maybe an intermediate value would be best. I'd go with 64MB.

I completely agree.

So what I got on my setup with that value.

1. IOMMU is enabled for Dom0:

(XEN) Extended region 0: 0x40000000->0x47e00000
(XEN) Extended region 1: 0xc0000000->0xe6000000
(XEN) Extended region 2: 0xe7800000->0xec000000
(XEN) Extended region 3: 0xf1200000->0xfd000000
(XEN) Extended region 4: 0x100000000->0x500000000
(XEN) Extended region 5: 0x580000000->0x600000000
(XEN) Extended region 6: 0x680000000->0x700000000
(XEN) Extended region 7: 0x780000000->0x10000000000

2. IOMMU is disabled for Dom0:

(XEN) Extended region 0: 0x48000000->0x54000000
(XEN) Extended region 1: 0x57000000->0x60000000
(XEN) Extended region 2: 0x70000000->0x78000000
(XEN) Extended region 3: 0x78200000->0xc0000000
(XEN) Extended region 4: 0x500000000->0x580000000
(XEN) Extended region 5: 0x600000000->0x680000000
(XEN) Extended region 6: 0x700000000->0x780000000

Which is not bad.

Thank you.

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:11:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:11:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188722.337984 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyfU-0000F0-LN; Thu, 16 Sep 2021 21:11:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188722.337984; Thu, 16 Sep 2021 21:11:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyfU-0000Et-IQ; Thu, 16 Sep 2021 21:11:40 +0000
Received: by outflank-mailman (input) for mailman id 188722;
 Thu, 16 Sep 2021 21:11:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQyfT-0000Ei-6s; Thu, 16 Sep 2021 21:11:39 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQyfT-0007HJ-1M; Thu, 16 Sep 2021 21:11:39 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQyfS-0001Cg-Lv; Thu, 16 Sep 2021 21:11:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQyfS-0006RQ-LO; Thu, 16 Sep 2021 21:11:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+c0V8jg5F6gLu00aBtPbQIjcah0lTIcB8oSMJhFgIfM=; b=WmPnViSJr1DMdhb0KZ0TmOj2sf
	xN4UZJYk+rMyWPGhbR43ym4dUZFr9r89kL3rzDYsb8e2oQ4WBZsbBKeFkR2d1XLcEglt8gpvZ+VkR
	0J0H4INB66Yo6EIwuf9wIKtptXJ6/UU+INe9kNe1+sSFu7Vd9zOL4VMH/yQHMdwQWR8o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165008-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165008: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=7b7ab2d6c99987e46aa53478798a05fcaf02226e
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 21:11:38 +0000

flight 165008 qemu-mainline real [real]
flight 165016 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165008/
http://logs.test-lab.xenproject.org/osstest/logs/165016/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                7b7ab2d6c99987e46aa53478798a05fcaf02226e
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    5 days
Failing since        164967  2021-09-13 13:06:52 Z    3 days    7 attempts
Testing same since   165000  2021-09-15 20:38:14 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jose R. Ziviani <jziviani@suse.de>
  Lara Lazier <laramglazier@gmail.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Thomas Huth <thuth@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1863 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:20:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:20:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188730.337999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQynY-00019K-FS; Thu, 16 Sep 2021 21:20:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188730.337999; Thu, 16 Sep 2021 21:20:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQynY-00019D-Br; Thu, 16 Sep 2021 21:20:00 +0000
Received: by outflank-mailman (input) for mailman id 188730;
 Thu, 16 Sep 2021 21:19:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQynX-000197-3C
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:19:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d14e8d4e-0213-4dec-902b-0f8f977ed240;
 Thu, 16 Sep 2021 21:19:56 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1E01D60F8F;
 Thu, 16 Sep 2021 21:19:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d14e8d4e-0213-4dec-902b-0f8f977ed240
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631827195;
	bh=ZzUiwk9KFvzm8UKF/Jsr6lF8dLnyGWhO3er1PzA/7VM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=e5lv1ELWDWbLJR9sKc5q0RzcMwBKe/ai2ISWLAPwrdXQeqLWT1Bq0muw4703n7dpx
	 i42e8MkQj5g4Lihnxj/2ys/oeWTLPq88FHPa6DxRTCXuoQnYmsBbAC9rePysKFktM3
	 SSxFdFJXMSnYViImM/7TKDLy5G3RHauhQIOcz5eoBV4hs0iGYSgkaJkjBrxEAT77k6
	 YlucQVmmYEALqbmNPJwclCBF26a/MMbQUrGISdGqj6mQmWgFMPBZVmH5DehBIeEwIf
	 vO/XNKPP6aXLNI+YiwrHOB7f4oMwq7hBZFhue3cF5v99hCyq/p5SwdHY0PCdVNcJN9
	 hA6pckZu52BbQ==
Date: Thu, 16 Sep 2021 14:19:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v4 2/7] xen/arm: Import ID features sanitize from linux
In-Reply-To: <c949658a9066a522a4a815e2051943d0a29fdc73.1631772970.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109161419460.21985@sstabellini-ThinkPad-T480s>
References: <cover.1631772970.git.bertrand.marquis@arm.com> <c949658a9066a522a4a815e2051943d0a29fdc73.1631772970.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 Sep 2021, Bertrand Marquis wrote:
> Import structures declared in Linux file arch/arm64/kernel/cpufeature.c
> and the required types from arch/arm64/include/asm/cpufeature.h.
> 
> Current code has been imported from Linux 5.13-rc5 (Commit ID
> cd1245d75ce93b8fd206f4b34eb58bcfe156d5e9) and copied into cpufeature.c
> in arm64 code and cpufeature.h in arm64 specific headers.
> 
> Those structure will be used to sanitize the cpu features available to
> the ones availble on all cores of a system even if we are on an
> heterogeneous platform (from example a big/LITTLE).
> 
> For each feature field of all ID registers, those structures define what
> is the safest value and if we can allow to have different values in
> different cores.
> 
> This patch is introducing Linux code without any changes to it.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v4: import arm64_ftr_set_value from Linux
> Changes in v3: none
> Changes in v2:
> - Move add to Makefile to following patch to allow bisection
> - Remove GPL text as SPDL is there
> - Re-add introduction comment from Linux Kernel file
> - Rename cpusanitize.c to cpufeature.c to keep Linux file name
> - Move structures imported from linux headers into a new cpufeature.h
> header in asm-arm/arm64.
> - Move comment about imported code origin to the file header
> - Remove not needed linux function declarations instead of removing them
> in the following patch
> - Add original arm64_ftr_safe_value from Linux
> - include kernel.h to use max()
> - remove unused ftr_single32 as we will not use it
> - remove ctr associated structures that we cannot use (keep the one
> defining sanitization bits)
> ---
>  xen/arch/arm/arm64/cpufeature.c        | 514 +++++++++++++++++++++++++
>  xen/include/asm-arm/arm64/cpufeature.h | 104 +++++
>  2 files changed, 618 insertions(+)
>  create mode 100644 xen/arch/arm/arm64/cpufeature.c
>  create mode 100644 xen/include/asm-arm/arm64/cpufeature.h
> 
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> new file mode 100644
> index 0000000000..152086dc9d
> --- /dev/null
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -0,0 +1,514 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Contains CPU feature definitions
> + *
> + * The following structures have been imported directly from Linux kernel and
> + * should be kept in sync.
> + * The current version has been imported from arch/arm64/kernel/cpufeature.c
> + *  from kernel version 5.13-rc5 together with the required structures and
> + *  macros from arch/arm64/include/asm/cpufeature.h which are stored in
> + *  include/asm-arm/arm64/cpufeature.h
> + *
> + * Copyright (C) 2021 Arm Ltd.
> + * based on code from the Linux kernel, which is:
> + *  Copyright (C) 2015 ARM Ltd.
> + *
> + * A note for the weary kernel hacker: the code here is confusing and hard to
> + * follow! That's partly because it's solving a nasty problem, but also because
> + * there's a little bit of over-abstraction that tends to obscure what's going
> + * on behind a maze of helper functions and macros.
> + *
> + * The basic problem is that hardware folks have started gluing together CPUs
> + * with distinct architectural features; in some cases even creating SoCs where
> + * user-visible instructions are available only on a subset of the available
> + * cores. We try to address this by snapshotting the feature registers of the
> + * boot CPU and comparing these with the feature registers of each secondary
> + * CPU when bringing them up. If there is a mismatch, then we update the
> + * snapshot state to indicate the lowest-common denominator of the feature,
> + * known as the "safe" value. This snapshot state can be queried to view the
> + * "sanitised" value of a feature register.
> + *
> + * The sanitised register values are used to decide which capabilities we
> + * have in the system. These may be in the form of traditional "hwcaps"
> + * advertised to userspace or internal "cpucaps" which are used to configure
> + * things like alternative patching and static keys. While a feature mismatch
> + * may result in a TAINT_CPU_OUT_OF_SPEC kernel taint, a capability mismatch
> + * may prevent a CPU from being onlined at all.
> + *
> + * Some implementation details worth remembering:
> + *
> + * - Mismatched features are *always* sanitised to a "safe" value, which
> + *   usually indicates that the feature is not supported.
> + *
> + * - A mismatched feature marked with FTR_STRICT will cause a "SANITY CHECK"
> + *   warning when onlining an offending CPU and the kernel will be tainted
> + *   with TAINT_CPU_OUT_OF_SPEC.
> + *
> + * - Features marked as FTR_VISIBLE have their sanitised value visible to
> + *   userspace. FTR_VISIBLE features in registers that are only visible
> + *   to EL0 by trapping *must* have a corresponding HWCAP so that late
> + *   onlining of CPUs cannot lead to features disappearing at runtime.
> + *
> + * - A "feature" is typically a 4-bit register field. A "capability" is the
> + *   high-level description derived from the sanitised field value.
> + *
> + * - Read the Arm ARM (DDI 0487F.a) section D13.1.3 ("Principles of the ID
> + *   scheme for fields in ID registers") to understand when feature fields
> + *   may be signed or unsigned (FTR_SIGNED and FTR_UNSIGNED accordingly).
> + *
> + * - KVM exposes its own view of the feature registers to guest operating
> + *   systems regardless of FTR_VISIBLE. This is typically driven from the
> + *   sanitised register values to allow virtual CPUs to be migrated between
> + *   arbitrary physical CPUs, but some features not present on the host are
> + *   also advertised and emulated. Look at sys_reg_descs[] for the gory
> + *   details.
> + *
> + * - If the arm64_ftr_bits[] for a register has a missing field, then this
> + *   field is treated as STRICT RES0, including for read_sanitised_ftr_reg().
> + *   This is stronger than FTR_HIDDEN and can be used to hide features from
> + *   KVM guests.
> + */
> +
> +#include <xen/types.h>
> +#include <xen/kernel.h>
> +#include <asm/sysregs.h>
> +#include <asm/cpufeature.h>
> +#include <asm/arm64/cpufeature.h>
> +
> +#define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	{						\
> +		.sign = SIGNED,				\
> +		.visible = VISIBLE,			\
> +		.strict = STRICT,			\
> +		.type = TYPE,				\
> +		.shift = SHIFT,				\
> +		.width = WIDTH,				\
> +		.safe_val = SAFE_VAL,			\
> +	}
> +
> +/* Define a feature with unsigned values */
> +#define ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	__ARM64_FTR_BITS(FTR_UNSIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +/* Define a feature with a signed value */
> +#define S_ARM64_FTR_BITS(VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
> +	__ARM64_FTR_BITS(FTR_SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL)
> +
> +#define ARM64_FTR_END					\
> +	{						\
> +		.width = 0,				\
> +	}
> +
> +/*
> + * NOTE: Any changes to the visibility of features should be kept in
> + * sync with the documentation of the CPU feature register ABI.
> + */
> +static const struct arm64_ftr_bits ftr_id_aa64isar0[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RNDR_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TLB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_TS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_FHM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_DP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM4_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SM3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_RDM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_ATOMICS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_CRC32_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_SHA1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR0_AES_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DGH_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SPECRES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_API_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH),
> +		       FTR_STRICT, FTR_EXACT, ID_AA64ISAR1_APA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_MPAM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SEL2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +				   FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_GIC_SHIFT, 4, 0),
> +	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
> +	S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_SHIFT, 4, ID_AA64PFR0_EL1_64BIT_ONLY),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL0_SHIFT, 4, ID_AA64PFR0_EL0_64BIT_ONLY),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MPAMFRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_RASFRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_BTI),
> +				    FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_BT_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F64MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_F32MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
> +		       FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
> +	/*
> +	 * Page size not being supported at Stage-2 is not fatal. You
> +	 * just give up KVM if PAGE_SIZE isn't supported there. Go fix
> +	 * your favourite nesting hypervisor.
> +	 *
> +	 * There is a small corner case where the hypervisor explicitly
> +	 * advertises a given granule size at Stage-2 (value 2) on some
> +	 * vCPUs, and uses the fallback to Stage-1 (value 0) for other
> +	 * vCPUs. Although this is not forbidden by the architecture, it
> +	 * indicates that the hypervisor is being silly (or buggy).
> +	 *
> +	 * We make no effort to cope with this and pretend that if these
> +	 * fields are inconsistent across vCPUs, then it isn't worth
> +	 * trying to bring KVM up.
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN4_2_SHIFT, 4, 1),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN64_2_SHIFT, 4, 1),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64MMFR0_TGRAN16_2_SHIFT, 4, 1),
> +	/*
> +	 * We already refuse to boot CPUs that don't support our configured
> +	 * page size, so we can only detect mismatches for a page size other
> +	 * than the one we're currently using. Unfortunately, SoCs like this
> +	 * exist in the wild so, even though we don't like it, we'll have to go
> +	 * along with it and treat them as non-strict.
> +	 */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN16_SHIFT, 4, ID_AA64MMFR0_TGRAN16_NI),
> +
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL0_SHIFT, 4, 0),
> +	/* Linux shouldn't care about secure memory */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_SNSMEM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_BIGENDEL_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ASID_SHIFT, 4, 0),
> +	/*
> +	 * Differing PARange is fine as long as all peripherals and memory are mapped
> +	 * within the minimum PARange of all CPUs
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_PARANGE_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VHE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_VMIDBITS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HADBS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_UAO_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CNP_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_ctr[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IDC_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_CWG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_OR_ZERO_SAFE, CTR_ERG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DMINLINE_SHIFT, 4, 1),
> +	/*
> +	 * Linux can handle differing I-cache policies. Userspace JITs will
> +	 * make use of *minLine.
> +	 * If we have differing I-cache policies, report it as the weakest - VIPT.
> +	 */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT),	/* L1Ip */
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_BRPS_SHIFT, 4, 0),
> +	/*
> +	 * We can instantiate multiple PMU instances with different levels
> +	 * of support.
> +	 */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_mvfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_dczid[] = {
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
> +	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DEBUG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_COPROC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_CMPBRANCH_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITFIELD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_BITCOUNT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_SWAP_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar5[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_RDM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_CRC32_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SHA1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_AES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR5_SEVL_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr4[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_EVT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CCIDX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_LSM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
> +
> +	/*
> +	 * SpecSEI = 1 indicates that the PE might generate an SError on an
> +	 * external abort on speculative read. It is safe to assume that an
> +	 * SError might be generated than it will not be. Hence it has been
> +	 * classified as FTR_HIGHER_SAFE.
> +	 */
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_MMFR4_SPECSEI_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar4[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SWP_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_PSR_M_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SYNCH_PRIM_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_BARRIER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_SMC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WRITEBACK_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_WITHSHIFTS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR4_UNPRIV_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_mmfr5[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR5_ETS_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_isar6[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_I8MM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_BF16_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SPECRES_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_SB_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_FHM_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_DP_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR6_JSCVT_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr0[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr1[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GIC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRT_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SEC_FRAC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_GENTIMER_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_VIRTUALIZATION_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_MPROGMOD_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_SECURITY_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR1_PROGMOD_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_pfr2[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_SSBS_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR2_CSV3_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> +	/* [31:28] TraceFilt */
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_id_dfr1[] = {
> +	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR1_MTPMU_SHIFT, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> +		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
> +	ARM64_FTR_END,
> +};
> +
> +/*
> + * Common ftr bits for a 32bit register with all hidden, strict
> + * attributes, with 4bit feature fields and a default safe value of
> + * 0. Covers the following 32bit registers:
> + * id_isar[1-4], id_mmfr[1-3], id_pfr1, mvfr[0-1]
> + */
> +static const struct arm64_ftr_bits ftr_generic_32bits[] = {
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
> +	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
> +	ARM64_FTR_END,
> +};
> +
> +static const struct arm64_ftr_bits ftr_raz[] = {
> +	ARM64_FTR_END,
> +};
> +
> +static u64 arm64_ftr_set_value(const struct arm64_ftr_bits *ftrp, s64 reg,
> +			       s64 ftr_val)
> +{
> +	u64 mask = arm64_ftr_mask(ftrp);
> +
> +	reg &= ~mask;
> +	reg |= (ftr_val << ftrp->shift) & mask;
> +	return reg;
> +}
> +
> +static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
> +				s64 cur)
> +{
> +	s64 ret = 0;
> +
> +	switch (ftrp->type) {
> +	case FTR_EXACT:
> +		ret = ftrp->safe_val;
> +		break;
> +	case FTR_LOWER_SAFE:
> +		ret = min(new, cur);
> +		break;
> +	case FTR_HIGHER_OR_ZERO_SAFE:
> +		if (!cur || !new)
> +			break;
> +		fallthrough;
> +	case FTR_HIGHER_SAFE:
> +		ret = max(new, cur);
> +		break;
> +	default:
> +		BUG();
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * End of imported linux structures and code
> + */
> +
> diff --git a/xen/include/asm-arm/arm64/cpufeature.h b/xen/include/asm-arm/arm64/cpufeature.h
> new file mode 100644
> index 0000000000..d9b9fa77cb
> --- /dev/null
> +++ b/xen/include/asm-arm/arm64/cpufeature.h
> @@ -0,0 +1,104 @@
> +#ifndef __ASM_ARM_ARM64_CPUFEATURES_H
> +#define __ASM_ARM_ARM64_CPUFEATURES_H
> +
> +/*
> + * CPU feature register tracking
> + *
> + * The safe value of a CPUID feature field is dependent on the implications
> + * of the values assigned to it by the architecture. Based on the relationship
> + * between the values, the features are classified into 3 types - LOWER_SAFE,
> + * HIGHER_SAFE and EXACT.
> + *
> + * The lowest value of all the CPUs is chosen for LOWER_SAFE and highest
> + * for HIGHER_SAFE. It is expected that all CPUs have the same value for
> + * a field when EXACT is specified, failing which, the safe value specified
> + * in the table is chosen.
> + */
> +
> +enum ftr_type {
> +	FTR_EXACT,			/* Use a predefined safe value */
> +	FTR_LOWER_SAFE,			/* Smaller value is safe */
> +	FTR_HIGHER_SAFE,		/* Bigger value is safe */
> +	FTR_HIGHER_OR_ZERO_SAFE,	/* Bigger value is safe, but 0 is biggest */
> +};
> +
> +#define FTR_STRICT	true	/* SANITY check strict matching required */
> +#define FTR_NONSTRICT	false	/* SANITY check ignored */
> +
> +#define FTR_SIGNED	true	/* Value should be treated as signed */
> +#define FTR_UNSIGNED	false	/* Value should be treated as unsigned */
> +
> +#define FTR_VISIBLE	true	/* Feature visible to the user space */
> +#define FTR_HIDDEN	false	/* Feature is hidden from the user */
> +
> +#define FTR_VISIBLE_IF_IS_ENABLED(config)		\
> +	(IS_ENABLED(config) ? FTR_VISIBLE : FTR_HIDDEN)
> +
> +struct arm64_ftr_bits {
> +	bool		sign;	/* Value is signed ? */
> +	bool		visible;
> +	bool		strict;	/* CPU Sanity check: strict matching required ? */
> +	enum ftr_type	type;
> +	u8		shift;
> +	u8		width;
> +	s64		safe_val; /* safe value for FTR_EXACT features */
> +};
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field_width(u64 features, int field, int width)
> +{
> +	return (s64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_signed_field(u64 features, int field)
> +{
> +	return cpuid_feature_extract_signed_field_width(features, field, 4);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width)
> +{
> +	return (u64)(features << (64 - width - field)) >> (64 - width);
> +}
> +
> +static inline unsigned int __attribute_const__
> +cpuid_feature_extract_unsigned_field(u64 features, int field)
> +{
> +	return cpuid_feature_extract_unsigned_field_width(features, field, 4);
> +}
> +
> +static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
> +{
> +	return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign)
> +{
> +	return (sign) ?
> +		cpuid_feature_extract_signed_field_width(features, field, width) :
> +		cpuid_feature_extract_unsigned_field_width(features, field, width);
> +}
> +
> +static inline int __attribute_const__
> +cpuid_feature_extract_field(u64 features, int field, bool sign)
> +{
> +	return cpuid_feature_extract_field_width(features, field, 4, sign);
> +}
> +
> +static inline s64 arm64_ftr_value(const struct arm64_ftr_bits *ftrp, u64 val)
> +{
> +	return (s64)cpuid_feature_extract_field_width(val, ftrp->shift, ftrp->width, ftrp->sign);
> +}
> +
> +#endif /* _ASM_ARM_ARM64_CPUFEATURES_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:20:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:20:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188732.338010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQynn-0002DN-SS; Thu, 16 Sep 2021 21:20:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188732.338010; Thu, 16 Sep 2021 21:20:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQynn-0002DE-OI; Thu, 16 Sep 2021 21:20:15 +0000
Received: by outflank-mailman (input) for mailman id 188732;
 Thu, 16 Sep 2021 21:20:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQynl-0002AX-Th
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:20:13 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 352c84b0-76ad-472a-980c-880651cbb2f0;
 Thu, 16 Sep 2021 21:20:12 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 716BE60FC0;
 Thu, 16 Sep 2021 21:20:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 352c84b0-76ad-472a-980c-880651cbb2f0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631827211;
	bh=GndBUpJR8Wbm6WGh9Zpmuv+robmPcZ5OpkNN1kUTkOg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=QWNFFshQeyQe9PImhprgPUIImqw+mFvXeHK3FcBFXGMCj3tHQwH5zYep/aPUyARyr
	 5UMK61KXn1pRIBbkAe50VFoQdMS5UIdAFuCK/VY9mdgKsjutQ+I8ctW15NUBLpkpbs
	 f5jUVo2L76wwbzvJ/5Q81hliAxrsXW7qnKWu6iVQWEPHJuRJyEMEwPE2JCJAHkOljv
	 LTZ03ZeJlYe2KXQx3YQr9emxzFaArVn1t1G276R7849+yuuQjoNdnVJsVbGgqCl1GE
	 d2Mz+LSYYUrDTffRa6ewt3yNIEeJF2cAXitHiR2wYVTtHYVA9r8/U9PPkfBfW6DV0F
	 j5NOkHUAfCBeQ==
Date: Thu, 16 Sep 2021 14:20:10 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v4 4/7] xen/arm: Sanitize cpuinfo ID registers fields
In-Reply-To: <3f65e7f58a422d41d89b879409418b9c5f2f84b9.1631772970.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109161420040.21985@sstabellini-ThinkPad-T480s>
References: <cover.1631772970.git.bertrand.marquis@arm.com> <3f65e7f58a422d41d89b879409418b9c5f2f84b9.1631772970.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 Sep 2021, Bertrand Marquis wrote:
> Define a sanitize_cpu function to be called on secondary cores to
> sanitize the system cpuinfo structure.
> 
> The safest value is taken when possible and the system is marked tainted
> if we encounter values which are incompatible with each other.
> 
> Call the update_system_features function on all secondary cores that are
> kept running and taint the system if different midr are found between
> cores but hmp-unsafe=true was passed on Xen command line.
> 
> This is only supported on arm64 so update_system_features is an empty
> static inline on arm32.
> 
> The patch is adding a new TAINT_CPU_OUT_OF_SPEC to warn the user if
> Xen is running on a system with features differences between cores which
> are not supported.
> 
> The patch is disabling CTR_EL0, DCZID_EL0 and ZCRusing #if 0 with a TODO
> as this patch is not handling sanitization of those registers.
> CTR_EL0/DCZID will be handled in a future patch to properly handle
> different cache attributes when possible.
> ZCR should be sanitize once we add support for SVE in Xen.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> Changes in v4:
> - arm64_ftr_set_value
> Changes in v3:
> - in case of different midr but hmp-unsafe passed on the command line,
> enable all cores anyway but taint Xen with CPU_OUT_OF_SPEC.
> - use current core info to sanitize cpu only if we keep it on
> Changes in v2:
> - add compilation of cpufeature.c in this patch instead of previous one
> - remove functions reused from linux code and moved to header
> - rename sanitize_cpu to update_system_features
> - change to Linux coding style
> - remove dev comments
> - surround currently not used Linux structures with #if 0 and adapt the
> commit message
> - add missing aa64dfr1 register
> - add TODO for CTR, DCZID and ZCR
> - add CPU_OUT_OF_SPEC support to print_taint
> - use system_cpuinfo instead of boot_cpu_data
> ---
>  xen/arch/arm/arm64/Makefile      |   1 +
>  xen/arch/arm/arm64/cpufeature.c  | 117 +++++++++++++++++++++++++++++++
>  xen/arch/arm/smpboot.c           |  34 ++++++---
>  xen/common/kernel.c              |   6 +-
>  xen/include/asm-arm/cpufeature.h |   9 +++
>  xen/include/xen/lib.h            |   1 +
>  6 files changed, 158 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
> index 40642ff574..701d66883d 100644
> --- a/xen/arch/arm/arm64/Makefile
> +++ b/xen/arch/arm/arm64/Makefile
> @@ -1,6 +1,7 @@
>  obj-y += lib/
>  
>  obj-y += cache.o
> +obj-y += cpufeature.o
>  obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
>  obj-$(CONFIG_EARLY_PRINTK) += debug.o
>  obj-y += domctl.o
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index 152086dc9d..58596495a8 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,6 +275,9 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: use this to sanitize the cache line size among cores */
> +
>  static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -291,6 +294,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
>  	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -325,11 +329,14 @@ static const struct arm64_ftr_bits ftr_mvfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: handle this when sanitizing cache related registers */
>  static const struct arm64_ftr_bits ftr_dczid[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  static const struct arm64_ftr_bits ftr_id_isar0[] = {
>  	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_ISAR0_DIVIDE_SHIFT, 4, 0),
> @@ -444,11 +451,15 @@ static const struct arm64_ftr_bits ftr_id_dfr1[] = {
>  	ARM64_FTR_END,
>  };
>  
> +#if 0
> +/* TODO: use this to sanitize SVE once we support it */
> +
>  static const struct arm64_ftr_bits ftr_zcr[] = {
>  	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>  		ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0),	/* LEN */
>  	ARM64_FTR_END,
>  };
> +#endif
>  
>  /*
>   * Common ftr bits for a 32bit register with all hidden, strict
> @@ -512,3 +523,109 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new,
>   * End of imported linux structures and code
>   */
>  
> +static void sanitize_reg(u64 *cur_reg, u64 new_reg, const char *reg_name,
> +						const struct arm64_ftr_bits *ftrp)
> +{
> +	int taint = 0;
> +	u64 old_reg = *cur_reg;
> +
> +	for (;ftrp->width != 0;ftrp++)
> +	{
> +		s64 cur_field = arm64_ftr_value(ftrp, *cur_reg);
> +		s64 new_field = arm64_ftr_value(ftrp, new_reg);
> +
> +		if (cur_field == new_field)
> +			continue;
> +
> +		if (ftrp->strict)
> +			taint = 1;
> +
> +		*cur_reg = arm64_ftr_set_value(ftrp, *cur_reg,
> +							arm64_ftr_safe_value(ftrp, new_field, cur_field));
> +	}
> +
> +	if (old_reg != new_reg)
> +		printk(XENLOG_DEBUG "SANITY DIF: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> +				reg_name, old_reg, new_reg);
> +	if (old_reg != *cur_reg)
> +		printk(XENLOG_DEBUG "SANITY FIX: %s 0x%"PRIx64" -> 0x%"PRIx64"\n",
> +				reg_name, old_reg, *cur_reg);
> +
> +	if (taint)
> +	{
> +		printk(XENLOG_WARNING "SANITY CHECK: Unexpected variation in %s.\n",
> +				reg_name);
> +		add_taint(TAINT_CPU_OUT_OF_SPEC);
> +	}
> +}
> +
> +
> +/*
> + * This function should be called on secondary cores to sanitize the boot cpu
> + * cpuinfo.
> + */
> +void update_system_features(const struct cpuinfo_arm *new)
> +{
> +
> +#define SANITIZE_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				 #reg, ftr_##reg)
> +
> +#define SANITIZE_ID_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_id_##reg)
> +
> +#define SANITIZE_RAZ_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_raz)
> +
> +#define SANITIZE_GENERIC_REG(field, num, reg)  \
> +	sanitize_reg(&system_cpuinfo.field.bits[num], new->field.bits[num], \
> +				#reg, ftr_generic_32bits)
> +
> +	SANITIZE_ID_REG(pfr64, 0, aa64pfr0);
> +	SANITIZE_ID_REG(pfr64, 1, aa64pfr1);
> +
> +	SANITIZE_ID_REG(dbg64, 0, aa64dfr0);
> +	SANITIZE_RAZ_REG(dbg64, 1, aa64dfr1);
> +
> +	SANITIZE_ID_REG(mm64, 0, aa64mmfr0);
> +	SANITIZE_ID_REG(mm64, 1, aa64mmfr1);
> +	SANITIZE_ID_REG(mm64, 2, aa64mmfr2);
> +
> +	SANITIZE_ID_REG(isa64, 0, aa64isar0);
> +	SANITIZE_ID_REG(isa64, 1, aa64isar1);
> +
> +	SANITIZE_ID_REG(zfr64, 0, aa64zfr0);
> +
> +	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
> +	{
> +		SANITIZE_ID_REG(pfr32, 0, pfr0);
> +		SANITIZE_ID_REG(pfr32, 1, pfr1);
> +		SANITIZE_ID_REG(pfr32, 2, pfr2);
> +
> +		SANITIZE_ID_REG(dbg32, 0, dfr0);
> +		SANITIZE_ID_REG(dbg32, 1, dfr1);
> +
> +		SANITIZE_ID_REG(mm32, 0, mmfr0);
> +		SANITIZE_GENERIC_REG(mm32, 1, mmfr1);
> +		SANITIZE_GENERIC_REG(mm32, 2, mmfr2);
> +		SANITIZE_GENERIC_REG(mm32, 3, mmfr3);
> +		SANITIZE_ID_REG(mm32, 4, mmfr4);
> +		SANITIZE_ID_REG(mm32, 5, mmfr5);
> +
> +		SANITIZE_ID_REG(isa32, 0, isar0);
> +		SANITIZE_GENERIC_REG(isa32, 1, isar1);
> +		SANITIZE_GENERIC_REG(isa32, 2, isar2);
> +		SANITIZE_GENERIC_REG(isa32, 3, isar3);
> +		SANITIZE_ID_REG(isa32, 4, isar4);
> +		SANITIZE_ID_REG(isa32, 5, isar5);
> +		SANITIZE_ID_REG(isa32, 6, isar6);
> +
> +		SANITIZE_GENERIC_REG(mvfr, 0, mvfr0);
> +		SANITIZE_GENERIC_REG(mvfr, 1, mvfr1);
> +#ifndef MVFR2_MAYBE_UNDEFINED
> +		SANITIZE_REG(mvfr, 2, mvfr2);
> +#endif
> +	}
> +}
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index c9f2827d56..60c0e82fc5 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -318,15 +318,26 @@ void start_secondary(void)
>       * is manually specified for all domains). Better to park them for
>       * now.
>       */
> -    if ( !opt_hmp_unsafe &&
> -         current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
> +    if ( current_cpu_data.midr.bits != system_cpuinfo.midr.bits )
>      {
> -        printk(XENLOG_ERR
> -               "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
> -               XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
> -               smp_processor_id(), current_cpu_data.midr.bits,
> -               system_cpuinfo.midr.bits);
> -        stop_cpu();
> +        if ( !opt_hmp_unsafe )
> +        {
> +            printk(XENLOG_ERR
> +                   "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
> +                   XENLOG_ERR "disable cpu (see big.LITTLE.txt under docs/).\n",
> +                   smp_processor_id(), current_cpu_data.midr.bits,
> +                   system_cpuinfo.midr.bits);
> +            stop_cpu();
> +        }
> +        else
> +        {
> +            printk(XENLOG_ERR
> +                   "CPU%u MIDR (0x%"PRIregister") does not match boot CPU MIDR (0x%"PRIregister"),\n"
> +                   XENLOG_ERR "hmp-unsafe turned on so tainting Xen and keep core on!!\n",
> +                   smp_processor_id(), current_cpu_data.midr.bits,
> +                   system_cpuinfo.midr.bits);
> +            add_taint(TAINT_CPU_OUT_OF_SPEC);
> +         }
>      }
>  
>      if ( dcache_line_bytes != read_dcache_line_bytes() )
> @@ -337,6 +348,13 @@ void start_secondary(void)
>          stop_cpu();
>      }
>  
> +    /*
> +     * system features must be updated only if we do not stop the core or
> +     * we might disable features due to a non used core (for example when
> +     * booting on big cores on a big.LITTLE system with hmp_unsafe)
> +     */
> +    update_system_features(&current_cpu_data);
> +
>      mmu_init_secondary_cpu();
>  
>      gic_init_secondary_cpu();
> diff --git a/xen/common/kernel.c b/xen/common/kernel.c
> index d77756a81e..e119e5401f 100644
> --- a/xen/common/kernel.c
> +++ b/xen/common/kernel.c
> @@ -327,6 +327,7 @@ unsigned int tainted;
>   *  'H' - HVM forced emulation prefix is permitted.
>   *  'M' - Machine had a machine check experience.
>   *  'U' - Platform is unsecure (usually due to an errata on the platform).
> + *  'S' - Out of spec CPU (One core has a feature incompatible with others).
>   *
>   *      The string is overwritten by the next call to print_taint().
>   */
> @@ -334,12 +335,13 @@ char *print_tainted(char *str)
>  {
>      if ( tainted )
>      {
> -        snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c",
> +        snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c%c",
>                   tainted & TAINT_MACHINE_UNSECURE ? 'U' : ' ',
>                   tainted & TAINT_MACHINE_CHECK ? 'M' : ' ',
>                   tainted & TAINT_SYNC_CONSOLE ? 'C' : ' ',
>                   tainted & TAINT_ERROR_INJECT ? 'E' : ' ',
> -                 tainted & TAINT_HVM_FEP ? 'H' : ' ');
> +                 tainted & TAINT_HVM_FEP ? 'H' : ' ',
> +                 tainted & TAINT_CPU_OUT_OF_SPEC ? 'S' : ' ');
>      }
>      else
>      {
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 8f2b8e7830..52cb3133e0 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -330,6 +330,15 @@ extern struct cpuinfo_arm system_cpuinfo;
>  
>  extern void identify_cpu(struct cpuinfo_arm *);
>  
> +#ifdef CONFIG_ARM_64
> +extern void update_system_features(const struct cpuinfo_arm *);
> +#else
> +static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)
> +{
> +    /* Not supported on arm32 */
> +}
> +#endif
> +
>  extern struct cpuinfo_arm cpu_data[];
>  #define current_cpu_data cpu_data[smp_processor_id()]
>  
> diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
> index 1198c7c0b2..c6987973bf 100644
> --- a/xen/include/xen/lib.h
> +++ b/xen/include/xen/lib.h
> @@ -192,6 +192,7 @@ uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c);
>  #define TAINT_ERROR_INJECT              (1u << 2)
>  #define TAINT_HVM_FEP                   (1u << 3)
>  #define TAINT_MACHINE_UNSECURE          (1u << 4)
> +#define TAINT_CPU_OUT_OF_SPEC           (1u << 5)
>  extern unsigned int tainted;
>  #define TAINT_STRING_MAX_LEN            20
>  extern char *print_tainted(char *str);
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:20:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:20:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188745.338020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyoN-00030r-6A; Thu, 16 Sep 2021 21:20:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188745.338020; Thu, 16 Sep 2021 21:20:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyoN-00030k-2t; Thu, 16 Sep 2021 21:20:51 +0000
Received: by outflank-mailman (input) for mailman id 188745;
 Thu, 16 Sep 2021 21:20:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQyoM-00030X-5d
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:20:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23c1e5de-cf17-4120-ab0b-ac28d331571e;
 Thu, 16 Sep 2021 21:20:49 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6421460FC0;
 Thu, 16 Sep 2021 21:20:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23c1e5de-cf17-4120-ab0b-ac28d331571e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631827248;
	bh=HWZT8ndIb8TWmTrUBrNqxhi3WlzxH1yvxu+3R3piHXE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OYUSFaz2ifdR3LUF8ght7wG0HwdADG8odifMf+QPq1mGB/jqGVVyfeoW1b9Djmvj3
	 EQnv9CRvpYaB+nEa0PPvYxdS0EnskohAFlENsEqnNWYspICUoH8WwUOE060tcCxnEX
	 zIrOa57zhPBberwDzLJ9Fsd6jbVT5cy+sBNbRPmeMx6F0mr07WYeWK2Uc1TgtdzJsz
	 k440LJ4EQ2fkU7krUvd/6eFKqwmBK86PHb/FTzxKnjWHZtrZN4MatBdiNE6QgK7kkd
	 eKgg5scJgsJlHPfgqy9vLcL4l9syB0DCJCq6R6v8m7MsUy1SgYijXHKJb3I7bcpbbu
	 BejP+xSgS83yQ==
Date: Thu, 16 Sep 2021 14:20:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Bertrand Marquis <bertrand.marquis@arm.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v4 7/7] xen/arm: Sanitize CTR_EL0
In-Reply-To: <3a6a63701df71c0a0ea743c6229266077da0563e.1631772970.git.bertrand.marquis@arm.com>
Message-ID: <alpine.DEB.2.21.2109161420210.21985@sstabellini-ThinkPad-T480s>
References: <cover.1631772970.git.bertrand.marquis@arm.com> <3a6a63701df71c0a0ea743c6229266077da0563e.1631772970.git.bertrand.marquis@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 Sep 2021, Bertrand Marquis wrote:
> Sanitize CTR_EL0 value between cores and taint Xen if incompatible
> values are found.
> 
> In the case of different i-cache types, the sanitize ctr_el0 will have a
> sanitize value but this is currently not used or exposed to guest which
> are seeing the original ctr_el0 value.
> 
> Use the opportunity to rename CTR_L1Ip to use an upper case name like
> Linux does.
> The patch is also defining ICACHE_POLICY_xxx instead of only having
> CTR_L1IP_xxx to sync the definitions with Linux and is updating the code
> using those accordingly (arm32 setup).
> 
> On platforms with only the same type of cores, this patch should not
> modify the current Xen behaviour.
> 
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Changes in v4: Remove TID2 support and handling of corresponding
> register emulation in vcpreg/vsysreg.
> Changes in v3: none
> Change in v2: Patch introduced in v2
> ---
>  xen/arch/arm/arm64/cpufeature.c  |  6 ++----
>  xen/arch/arm/cpufeature.c        |  2 ++
>  xen/arch/arm/setup.c             |  2 +-
>  xen/include/asm-arm/cpufeature.h |  9 +++++++++
>  xen/include/asm-arm/processor.h  | 18 +++++++++++++++---
>  5 files changed, 29 insertions(+), 8 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
> index d4679f5df3..6e5d30dc7b 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
>  	ARM64_FTR_END,
>  };
>  
> -#if 0
> -/* TODO: use this to sanitize the cache line size among cores */
> -
>  static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 */
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, 1, 1),
> @@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] = {
>  	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
>  	ARM64_FTR_END,
>  };
> -#endif
>  
>  static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
>  	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
> @@ -606,6 +602,8 @@ void update_system_features(const struct cpuinfo_arm *new)
>  	 */
>  	SANITIZE_REG(dczid, 0, dczid);
>  
> +	SANITIZE_REG(ctr, 0, ctr);
> +
>  	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>  	{
>  		SANITIZE_ID_REG(pfr32, 0, pfr0);
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index 113f20f601..6e51f530a8 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>  
>      c->dczid.bits[0] = READ_SYSREG(DCZID_EL0);
>  
> +    c->ctr.bits[0] = READ_SYSREG(CTR_EL0);
> +
>      aarch32_el0 = cpu_feature64_has_el0_32(c);
>  #endif
>  
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 4ab13d0fbe..49dc90d198 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -650,7 +650,7 @@ static void __init setup_mm(void)
>          panic("No memory bank\n");
>  
>      /* We only supports instruction caches implementing the IVIPT extension. */
> -    if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT )
> +    if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) == ICACHE_POLICY_AIVIVT )
>          panic("AIVIVT instruction cache not supported\n");
>  
>      init_pdx();
> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
> index 5219fd3bab..cab89ee142 100644
> --- a/xen/include/asm-arm/cpufeature.h
> +++ b/xen/include/asm-arm/cpufeature.h
> @@ -267,6 +267,14 @@ struct cpuinfo_arm {
>          register_t bits[1];
>      } dczid;
>  
> +    /*
> +     * CTR is only used to check for different cache types or policies and
> +     * taint Xen in this case
> +     */
> +    struct {
> +        register_t bits[1];
> +    } ctr;
> +
>  #endif
>  
>      /*
> @@ -339,6 +347,7 @@ extern struct cpuinfo_arm system_cpuinfo;
>  extern void identify_cpu(struct cpuinfo_arm *);
>  
>  #ifdef CONFIG_ARM_64
> +
>  extern void update_system_features(const struct cpuinfo_arm *);
>  #else
>  static inline void update_system_features(const struct cpuinfo_arm *cpuinfo)

Spurious change. I fixed it on commit.

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:30:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188760.338060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyxw-00055v-Ir; Thu, 16 Sep 2021 21:30:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188760.338060; Thu, 16 Sep 2021 21:30:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQyxw-00055o-Fl; Thu, 16 Sep 2021 21:30:44 +0000
Received: by outflank-mailman (input) for mailman id 188760;
 Thu, 16 Sep 2021 21:30:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQyxv-00055i-0L
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:30:43 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4c7982ca-daf3-415b-ba97-a41d2ed9f523;
 Thu, 16 Sep 2021 21:30:40 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id CFDD66120C;
 Thu, 16 Sep 2021 21:30:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c7982ca-daf3-415b-ba97-a41d2ed9f523
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631827840;
	bh=eZZ35RcCd6+mRhvokKdnetE2wKYE1suc/2Tx47bnKnQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uLENYyf5gaiXC2WtoMStbV0VKkO6hCUt+y77f7Xa7h431+6UF8ll3MrjCI+tAkdNL
	 hwJvh0F7Yll63zel6FUb3QdPwYhFU3PidLrWp9Js5vUunGtRmqN7UtQ59QNVi7rphv
	 XBut7w1DVlmKCjiapzrSM6OIn53SRv6W7jtVu9w2QVllZrRGD0NJgOdiP08pUQfY+4
	 0wTQEBE8PP35vhL+r1JRJ0w1ti75X6uWdIpm0j4HW8jdFyjsIigXmb5djAcvZj26kJ
	 XlsaVH803uJXc4Dkqri+U1YrYcuUac0qlSdzxBsD2FkQh1KbVaOYdTKto9KSTB3y8Z
	 5X/JQq7cVg3og==
Date: Thu, 16 Sep 2021 14:30:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <b349e43a-91d9-16ba-57c6-34e790b8b31c@gmail.com>
Message-ID: <alpine.DEB.2.21.2109161423020.21985@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s> <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
 <alpine.DEB.2.21.2109151352100.21985@sstabellini-ThinkPad-T480s> <b349e43a-91d9-16ba-57c6-34e790b8b31c@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 Sep 2021, Oleksandr wrote:
> > On Wed, 15 Sep 2021, Oleksandr wrote:
> > > > On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
> > > > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > > > 
> > > > > The extended region (safe range) is a region of guest physical
> > > > > address space which is unused and could be safely used to create
> > > > > grant/foreign mappings instead of wasting real RAM pages from
> > > > > the domain memory for establishing these mappings.
> > > > > 
> > > > > The extended regions are chosen at the domain creation time and
> > > > > advertised to it via "reg" property under hypervisor node in
> > > > > the guest device-tree. As region 0 is reserved for grant table
> > > > > space (always present), the indexes for extended regions are 1...N.
> > > > > If extended regions could not be allocated for some reason,
> > > > > Xen doesn't fail and behaves as usual, so only inserts region 0.
> > > > > 
> > > > > Please note the following limitations:
> > > > > - The extended region feature is only supported for 64-bit domain.
> > > > > - The ACPI case is not covered.
> > > > > 
> > > > > ***
> > > > > 
> > > > > As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> > > > > the algorithm to choose extended regions for it is different
> > > > > in comparison with the algorithm for non-direct mapped DomU.
> > > > > What is more, that extended regions should be chosen differently
> > > > > whether IOMMU is enabled or not.
> > > > > 
> > > > > Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> > > > > holes found in host device-tree if otherwise. Make sure that
> > > > > extended regions are 2MB-aligned and located within maximum possible
> > > > > addressable physical memory range. The maximum number of extended
> > > > > regions is 128.
> > > > > 
> > > > > Suggested-by: Julien Grall <jgrall@amazon.com>
> > > > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > > > ---
> > > > > Changes since RFC:
> > > > >      - update patch description
> > > > >      - drop uneeded "extended-region" DT property
> > > > > ---
> > > > > 
> > > > >    xen/arch/arm/domain_build.c | 226
> > > > > +++++++++++++++++++++++++++++++++++++++++++-
> > > > >    1 file changed, 224 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > > > index 206038d..070ec27 100644
> > > > > --- a/xen/arch/arm/domain_build.c
> > > > > +++ b/xen/arch/arm/domain_build.c
> > > > > @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct
> > > > > domain *d,
> > > > >        return res;
> > > > >    }
> > > > >    +static int __init add_ext_regions(unsigned long s, unsigned long
> > > > > e,
> > > > > void *data)
> > > > > +{
> > > > > +    struct meminfo *ext_regions = data;
> > > > > +    paddr_t start, size;
> > > > > +
> > > > > +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> > > > > +        return 0;
> > > > > +
> > > > > +    /* Both start and size of the extended region should be 2MB
> > > > > aligned
> > > > > */
> > > > > +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> > > > > +    if ( start > e )
> > > > > +        return 0;
> > > > > +
> > > > > +    size = (e - start + 1) & ~(SZ_2M - 1);
> > > > > +    if ( !size )
> > > > > +        return 0;
> > > > Can't you align size as well?
> > > > 
> > > >     size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
> > > I am sorry, I don't entirely get what you really meant here. We get both
> > > start
> > > and size 2MB-aligned by the calculations above
> > > (when calculating an alignment, we need to make sure that "start_passed <=
> > > start_aligned && size_aligned <= size_passed").
> > > If I add the proposing string after, I will reduce the already aligned
> > > size by
> > > 2MB.
> > > If I replace the size calculation with the following, I will get the
> > > reduced
> > > size even if the passed region is initially 2MB-aligned, so doesn't need
> > > to be
> > > adjusted.
> > > size = e - s + 1;
> > > size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
> > Sorry I misread your original code, I think it was working as intended
> > except for the "+1". I think it should be:
> > 
> >    size = (e - start) & ~(SZ_2M - 1);
> But why without "+1"? Isn't "e" here the *last address* of passed range?
> Without "+1" I get non entirely correct calculations, last valid 2MB is
> missed.

You are right: the "+1" should not be needed if this was "end",
following the normal definition of end. However, add_ext_regions is
called by rangeset_report_ranges, so end here is not actually "end", it
is "end-1".

For clarity, I would ask you to rewrite it like this:

/* 
 * e is actually "end-1" because it is called by rangeset functions
 * which are inclusive of the last address.
 */
e += 1;
size = (e - start) & ~(SZ_2M - 1);


> [snip]
> (XEN) Extended region 14: 0x580000000->0x5ffe00000
> (XEN) Extended region 15: 0x680000000->0x6ffe00000
> (XEN) Extended region 16: 0x780000000->0xffffe00000
> 
> But should get:
> 
> [snip]
> (XEN) Extended region 15: 0x580000000->0x600000000
> (XEN) Extended region 16: 0x680000000->0x700000000
> (XEN) Extended region 17: 0x780000000->0x10000000000
> 
> Let's consider how a hole between (for example) RAM bank 1 and bank 2 is
> calculated:
> (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1 with size
> 0x80000000
> (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2 with size
> 0x80000000
> So the hole size should also be 0x80000000.
> If we pass these RAM banks to rangeset_remove_range() one by one:
> 1: s = 0x500000000 e = 0x57FFFFFFF
> 2. s = 0x600000000 e = 0x67FFFFFFF
> we get s = 0x580000000 e = 0x5FFFFFFFF in add_ext_regions(), where "e" is the
> last address of the hole (not the first address out of the hole), so I think,
> that for proper size calculation we need to add 1 to "e - s". Or I really
> missed something?
> 
> 
> > 
> > > > > + */
> > > > > +#define EXT_REGION_START   0x40000000ULL
> > > > > +#define EXT_REGION_END     0x80003fffffffULL
> > > > > +
> > > > > +static int __init find_unallocated_memory(const struct kernel_info
> > > > > *kinfo,
> > > > > +                                          struct meminfo
> > > > > *ext_regions)
> > > > > +{
> > > > > +    const struct meminfo *assign_mem = &kinfo->mem;
> > > > > +    struct rangeset *unalloc_mem;
> > > > > +    paddr_t start, end;
> > > > > +    unsigned int i;
> > > > > +    int res;
> > > > > +
> > > > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > > > +
> > > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > > +    if ( !unalloc_mem )
> > > > > +        return -ENOMEM;
> > > > > +
> > > > > +    /* Start with all available RAM */
> > > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > > +    {
> > > > > +        start = bootinfo.mem.bank[i].start;
> > > > > +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size
> > > > > - 1;
> > > > Is the -1 needed? Isn't it going to screw up the size calculation later?
> > > I thought, it was needed. The calculation seems correct.
> > I think that normally for an example MMIO region:
> > 
> > start = 0x48000000
> > size  = 0x40000000
> > end   = 0x88000000
> > 
> > So end = start + size and points to the first address out of the range.
> > In other words, 0x88000000 doesn't actually belong to the MMIO region in
> > the example.
> > 
> > But here you are passing addresses to rangeset_add_range and other
> > rangeset functions and I think rangeset takes *inclusive* addresses as
> > input. So you need to pass start and end-1 because end-1 is the last
> > address of the MMIO region.
> > 
> > In fact you can see for instance in map_range_to_domain:
> > 
> >          res = iomem_permit_access(d, paddr_to_pfn(addr),
> >                  paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
> > 
> > Where iomem_permit_access is based on rangeset. So for clarity, I would
> > do:
> > 
> > start = assign_mem->bank[i].start;
> > end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> > res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > 
> > So that we don't get confused on the meaning of "end" which everywhere
> > else means the first address not in range.
> 
> I got your point, I will update the code if it much cleaner.
> 
> 
> > > > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > > > +        if ( res )
> > > > > +        {
> > > > > +            printk(XENLOG_ERR "Failed to add:
> > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > +                   start, end);
> > > > > +            goto out;
> > > > > +        }
> > > > > +    }
> > > > > +
> > > > > +    /* Remove RAM assigned to Dom0 */
> > > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > > +    {
> > > > > +        start = assign_mem->bank[i].start;
> > > > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size -
> > > > > 1;
> > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > +        if ( res )
> > > > > +        {
> > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > +                   start, end);
> > > > > +            goto out;
> > > > > +        }
> > > > > +    }
> > > > > +
> > > > > +    /* Remove reserved-memory regions */
> > > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > > > +    {
> > > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > +        if ( res )
> > > > > +        {
> > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > +                   start, end);
> > > > > +            goto out;
> > > > > +        }
> > > > > +    }
> > > > > +
> > > > > +    /* Remove grant table region */
> > > > > +    start = kinfo->gnttab_start;
> > > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > +    if ( res )
> > > > > +    {
> > > > > +        printk(XENLOG_ERR "Failed to remove:
> > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > +               start, end);
> > > > > +        goto out;
> > > > > +    }
> > > > > +
> > > > > +    start = EXT_REGION_START;
> > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > > +                                 add_ext_regions, ext_regions);
> > > > > +    if ( res )
> > > > > +        ext_regions->nr_banks = 0;
> > > > > +    else if ( !ext_regions->nr_banks )
> > > > > +        res = -ENOENT;
> > > > > +
> > > > > +out:
> > > > > +    rangeset_destroy(unalloc_mem);
> > > > > +
> > > > > +    return res;
> > > > > +}
> > > > > +
> > > > > +static int __init find_memory_holes(const struct kernel_info *kinfo,
> > > > > +                                    struct meminfo *ext_regions)
> > > > > +{
> > > > > +    struct dt_device_node *np;
> > > > > +    struct rangeset *mem_holes;
> > > > > +    paddr_t start, end;
> > > > > +    unsigned int i;
> > > > > +    int res;
> > > > > +
> > > > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > > > +
> > > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > > +    if ( !mem_holes )
> > > > > +        return -ENOMEM;
> > > > > +
> > > > > +    /* Start with maximum possible addressable physical memory range
> > > > > */
> > > > > +    start = EXT_REGION_START;
> > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > > +    if ( res )
> > > > > +    {
> > > > > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > > > +               start, end);
> > > > > +        goto out;
> > > > > +    }
> > > > > +
> > > > > +    /* Remove all regions described by "reg" property (MMIO, RAM,
> > > > > etc) */
> > > > > +    dt_for_each_device_node( dt_host, np )
> > > > Don't you need something like device_tree_for_each_node ?
> > > > dt_for_each_device_node won't go down any deeper in the tree?
> > > Thank you for pointing this out, I will investigate and update the patch.
> > > 
> > > 
> > > > Alternatively, maybe we could simply record the highest possible address
> > > > of any memory/device/anything as we scan the device tree with
> > > > handle_node. Then we can use that as the starting point here.
> > > I also don't like the idea to scan the DT much, but I failed to find an
> > > effective solution how to avoid that.
> > > Yes, we can record the highest possible address, but I am afraid, I didn't
> > > entirely get a suggestion. Is the suggestion to provide a single region
> > > starting from highest possible address + 1 and up to the EXT_REGION_END
> > > suitably aligned? Could you please clarify?
> > Yes, that is what I was suggesting as a possible alternative: start from
> > the highest possible address in DT + 1 and up to the EXT_REGION_END
> > suitably aligned. But that wouldn't solve the <4GB issue.
> > 
> > > > > +                goto out;
> > > > > +            }
> > > > > +
> > > > > +            start = addr & PAGE_MASK;
> > > > > +            end = PAGE_ALIGN(addr + size) - 1;
> > > > > +            res = rangeset_remove_range(mem_holes, start, end);
> > > > > +            if ( res )
> > > > > +            {
> > > > > +                printk(XENLOG_ERR "Failed to remove:
> > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > +                       start, end);
> > > > > +                goto out;
> > > > > +            }
> > > > > +        }
> > > > > +    }
> > > > As is, it will result in a myriad of small ranges which is unuseful and
> > > > slow to parse. I suggest to simplify it by removing a larger region than
> > > > strictly necessary. For instance, you could remove a 1GB-aligned and
> > > > 1GB-multiple region for each range. That way, you are going to get fewer
> > > > large free ranges instance of many small ones which we don't need.
> > > I agree with you that a lot of small ranges increase the bookkeeping in
> > > Dom0
> > > and there is also a theoretical (?) possibility
> > > that small ranges occupy all space we provide for extended regions
> > > (NR_MEM_BANKS)...
> > > But, let's consider my setup as an example again, but when the IOMMU is
> > > enabled for Dom0 ("holes found in DT").
> > > 
> > > --- The RAM configuration is the same:
> > > 
> > > (XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
> > > (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
> > > (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
> > > (XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3
> > > 
> > > --- There are a lot of various platform devices with reg property
> > > described in
> > > DT, I will probably not post all IO ranges here, just say that mostly all
> > > of
> > > them to be mapped at 0xE0000000-0xFFFFFFFF.
> > > 
> > > --- As we only pick up ranges with size >= 2MB, the calculated extended
> > > regions are (based on 40-bit IPA):
> > > 
> > > (XEN) Extended region 0: 0x40000000->0x47e00000
> > > (XEN) Extended region 1: 0xc0000000->0xe6000000
> > > (XEN) Extended region 2: 0xe7000000->0xe7200000
> > > (XEN) Extended region 3: 0xe7400000->0xe7600000
> > > (XEN) Extended region 4: 0xe7800000->0xec000000
> > > (XEN) Extended region 5: 0xec200000->0xec400000
> > > (XEN) Extended region 6: 0xec800000->0xee000000
> > > (XEN) Extended region 7: 0xee600000->0xee800000
> > > (XEN) Extended region 8: 0xeea00000->0xf1000000
> > > (XEN) Extended region 9: 0xf1200000->0xfd000000
> > > (XEN) Extended region 10: 0xfd200000->0xfd800000
> > > (XEN) Extended region 11: 0xfda00000->0xfe000000
> > > (XEN) Extended region 12: 0xfe200000->0xfe600000
> > > (XEN) Extended region 13: 0xfec00000->0xff800000
> > > (XEN) Extended region 14: 0x100000000->0x500000000
> > > (XEN) Extended region 15: 0x580000000->0x600000000
> > > (XEN) Extended region 16: 0x680000000->0x700000000
> > > (XEN) Extended region 17: 0x780000000->0x10000000000
> > > 
> > > So, if I *correctly* understood your idea about removing 1GB-aligned
> > > 1GB-multiple region for each range we would get the following:
> > > 
> > > (XEN) Extended region 0: 0x100000000->0x500000000
> > > (XEN) Extended region 1: 0x580000000->0x600000000
> > > (XEN) Extended region 2: 0x680000000->0x700000000
> > > (XEN) Extended region 3: 0x780000000->0x10000000000
> > > 
> > > As you can see there are no extended regions below 4GB at all. I assume,
> > > it
> > > would be good to provide them for 1:1 mapped Dom0 (for 32-bit DMA
> > > devices?)
> > > What else worries me is that IPA size could be 36 or even 32. So, I am
> > > afraid,
> > > we might even fail to find extended regions above 4GB.
> > > 
> > > 
> > > I think, if 2MB is considered small enough to bother with, probably we
> > > should
> > > go with something in between (16MB, 32MB, 64MB).
> > > For example, we can take into the account ranges with size >= 16MB:
> > > 
> > > (XEN) Extended region 0: 0x40000000->0x47e00000
> > > (XEN) Extended region 1: 0xc0000000->0xe6000000
> > > (XEN) Extended region 2: 0xe7800000->0xec000000
> > > (XEN) Extended region 3: 0xec800000->0xee000000
> > > (XEN) Extended region 4: 0xeea00000->0xf1000000
> > > (XEN) Extended region 5: 0xf1200000->0xfd000000
> > > (XEN) Extended region 6: 0x100000000->0x500000000
> > > (XEN) Extended region 7: 0x580000000->0x600000000
> > > (XEN) Extended region 8: 0x680000000->0x700000000
> > > (XEN) Extended region 9: 0x780000000->0x10000000000
> > > 
> > > Any thoughts?
> > Yeah maybe an intermediate value would be best. I'd go with 64MB.
> 
> I completely agree.
> 
> So what I got on my setup with that value.
> 
> 1. IOMMU is enabled for Dom0:
> 
> (XEN) Extended region 0: 0x40000000->0x47e00000
> (XEN) Extended region 1: 0xc0000000->0xe6000000
> (XEN) Extended region 2: 0xe7800000->0xec000000
> (XEN) Extended region 3: 0xf1200000->0xfd000000
> (XEN) Extended region 4: 0x100000000->0x500000000
> (XEN) Extended region 5: 0x580000000->0x600000000
> (XEN) Extended region 6: 0x680000000->0x700000000
> (XEN) Extended region 7: 0x780000000->0x10000000000
> 
> 2. IOMMU is disabled for Dom0:
> 
> (XEN) Extended region 0: 0x48000000->0x54000000
> (XEN) Extended region 1: 0x57000000->0x60000000
> (XEN) Extended region 2: 0x70000000->0x78000000
> (XEN) Extended region 3: 0x78200000->0xc0000000
> (XEN) Extended region 4: 0x500000000->0x580000000
> (XEN) Extended region 5: 0x600000000->0x680000000
> (XEN) Extended region 6: 0x700000000->0x780000000
> 
> Which is not bad.

Yeah I think that's good.


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:37:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:37:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188769.338071 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQz4q-00061Q-G1; Thu, 16 Sep 2021 21:37:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188769.338071; Thu, 16 Sep 2021 21:37:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQz4q-00061J-Bm; Thu, 16 Sep 2021 21:37:52 +0000
Received: by outflank-mailman (input) for mailman id 188769;
 Thu, 16 Sep 2021 21:37:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=BmaE=OG=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mQz4o-00060u-9N
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:37:50 +0000
Received: from mail-pj1-x102b.google.com (unknown [2607:f8b0:4864:20::102b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 088a6a5e-c2ac-448f-ac8f-fd5c79fec54f;
 Thu, 16 Sep 2021 21:37:49 +0000 (UTC)
Received: by mail-pj1-x102b.google.com with SMTP id k23so5541415pji.0
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 14:37:49 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id t15sm4013977pgk.13.2021.09.16.14.37.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 Sep 2021 14:37:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 088a6a5e-c2ac-448f-ac8f-fd5c79fec54f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=HRmclIUd+IXQYZmIP/8HeB41v2sSRX/lMU3hqPZHzVc=;
        b=Y5uK5usJpzBqYTtETIZzqzrlpEcCGhV1j72nBokzqPxRHfzfqcv9p8/VHqtByMLMa0
         7qnKk2mXRWZ/QVGLWlLUF5fmAtrANk6SVsEq1yvp8Onbj0l+EW2VdhbQ4ssaqq3XvXG+
         UIKM1VSQcdm2YELLCqHTNz0YRbJwYR+FmuDGCbuEaAumq4dsS2jCwrqpahV4jYDPFBXn
         +EFOprbkB9HNvtLDp/UkAPEYfLhp0RIxcvNz5Sev+mrGTECnwFfzaEXBv/uyDuKWSXl9
         MKEl69zQIthFjeiLPFYtD50iSHjTDfTtVEJ8PfVuNwd6e3VXHqiBdyMOjirUU+xrvFcM
         TwGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=HRmclIUd+IXQYZmIP/8HeB41v2sSRX/lMU3hqPZHzVc=;
        b=tMquEV7csHpPHpZqP9tYArW9smEKMOOvLZOtvYEVwuaalIUnbVb7hBnujox2uhE6BN
         vMTnqmjoDYktQ51USVN0u6xE5+LEHVu7LW79FawaAmGgvKe9rFT3DPXYWjGw1P6XMtg1
         uKTrQEaCiZ2EUQJR+bb0u1WcUTnBBCSjG+j4mMdDyp27IdOANQlUHe+x/7whCL5gYH/t
         pTyKCrxLzyl+rZjTiIZ0+AhWECrtS9DWBqCKD9MgjsHR2EAxP4Czxxduz3SquQ7Pknln
         EGj41t97F5CNb1Be6CWPzou8NKrUB686BB9hR0PpL9G6K6zhKO4hbdz2xI9HeYnga5ZJ
         7LUA==
X-Gm-Message-State: AOAM531ipQKbatYZUxciHf+s5j1SAMpwn9Nb6EuPErHZmkUHS8u+cioN
	1X6ucgcyAapNueQTF744sDwNiA==
X-Google-Smtp-Source: ABdhPJxrNs7jIZy34jmkRTyON4Y+9pNnsbkx7GECTLo7e1rXLwAWohi0Xxo5Jb11PEVRFoL9wO4HSQ==
X-Received: by 2002:a17:90b:1291:: with SMTP id fw17mr6609537pjb.135.1631828268138;
        Thu, 16 Sep 2021 14:37:48 -0700 (PDT)
Date: Thu, 16 Sep 2021 21:37:43 +0000
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
Message-ID: <YUO5J/jTMa2KGbsq@google.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210828201336.GD4353@worktop.programming.kicks-ass.net>

On Sat, Aug 28, 2021, Peter Zijlstra wrote:
> On Fri, Aug 27, 2021 at 05:35:45PM -0700, Sean Christopherson wrote:
> > Like Xu (2):
> >   perf/core: Rework guest callbacks to prepare for static_call support
> >   perf/core: Use static_call to optimize perf_guest_info_callbacks
> > 
> > Sean Christopherson (11):
> >   perf: Ensure perf_guest_cbs aren't reloaded between !NULL check and
> >     deref
> >   KVM: x86: Register perf callbacks after calling vendor's
> >     hardware_setup()
> >   KVM: x86: Register Processor Trace interrupt hook iff PT enabled in
> >     guest
> >   perf: Stop pretending that perf can handle multiple guest callbacks
> >   perf: Force architectures to opt-in to guest callbacks
> >   KVM: x86: Drop current_vcpu for kvm_running_vcpu + kvm_arch_vcpu
> >     variable
> >   KVM: x86: More precisely identify NMI from guest when handling PMI
> >   KVM: Move x86's perf guest info callbacks to generic KVM
> >   KVM: x86: Move Intel Processor Trace interrupt handler to vmx.c
> >   KVM: arm64: Convert to the generic perf callbacks
> >   KVM: arm64: Drop perf.c and fold its tiny bits of code into arm.c /
> >     pmu.c

Argh, sorry, I somehow managed to miss all of your replies.  I'll get back to
this series next week.  Thanks for the quick response!

> Lets keep the whole intel_pt crud inside x86...

In theory, I like the idea of burying intel_pt inside x86 (and even in Intel+VMX code
for the most part), but the actual implementation is a bit gross.  Because of the
whole "KVM can be a module" thing, either the static call and __static_call_return0
would need to be exported, or a new register/unregister pair would have to be exported.

The unregister path would also need its own synchronize_rcu().  In general, I
don't love duplicating the logic, but it's not the end of the world.

Either way works for me.  Paolo or Peter, do either of you have a preference?

> ---
> Index: linux-2.6/arch/x86/events/core.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/events/core.c
> +++ linux-2.6/arch/x86/events/core.c
> @@ -92,7 +92,7 @@ DEFINE_STATIC_CALL_RET0(x86_pmu_guest_ge
>  
>  DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
>  DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> -DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, *(perf_guest_cbs->handle_intel_pt_intr));
> +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, unsigned int (*)(void));

FWIW, the param needs to be a raw function, not a function pointer. 


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:41:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:41:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188776.338082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQz7z-0007Nr-VN; Thu, 16 Sep 2021 21:41:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188776.338082; Thu, 16 Sep 2021 21:41:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQz7z-0007Nk-Rm; Thu, 16 Sep 2021 21:41:07 +0000
Received: by outflank-mailman (input) for mailman id 188776;
 Thu, 16 Sep 2021 21:41:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=BmaE=OG=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mQz7z-0007Ne-AW
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:41:07 +0000
Received: from mail-pl1-x631.google.com (unknown [2607:f8b0:4864:20::631])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 133a1bce-1998-46de-ae64-6ab81342fef7;
 Thu, 16 Sep 2021 21:41:06 +0000 (UTC)
Received: by mail-pl1-x631.google.com with SMTP id f21so4789521plb.4
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 14:41:06 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id z11sm4163558pff.144.2021.09.16.14.41.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 Sep 2021 14:41:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 133a1bce-1998-46de-ae64-6ab81342fef7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=H6MB2ODxA8ncTb3aji91NU1uUQ9q1AebxOl6ENeKyck=;
        b=MXQC+ZfLxlKAl3MhEJW1BaTu7x3eZBr25vDIOP4P1K/hKKNsBxR9bYAbQrcXeqx0IT
         UwrZDJawrFH60RbkF94BswFhIwo0cou0aN+gLacubbeU6IcTTMJrkdfHEYWn3Q3CVBfY
         FWOlqrCFMP2DA28EATBAk/sg6Cxl8pCwP9qJHgGrwzQTOnAwwW0mxDrD/8RBAtDi8Sen
         7O4AbiMouCC1KnoWzOLrohcS9Vtb2Dd0sBl4eKlMlTi8+NrzRnGujJOfnGMsmwaTYUXB
         tHZDCMU2HvcsbXJF8DOiqaPVnBEOB3kiFk+1tNf5dSr4/Tt9/mMCbKmuD1xBAf0iTv/g
         rERQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=H6MB2ODxA8ncTb3aji91NU1uUQ9q1AebxOl6ENeKyck=;
        b=yjoE+1br+58vp8BuDViQInopId4wYFIS3uOEx3fcwAjOYYI2ELUiVeZGhRMGa8bpTg
         WJrAbbVogpLLng/XmxO0fRvklcw8ll6B+bDu9SiR+PlAHiByYrCl6oGUrM2Jb9cl4Iht
         +SpL0MMhVIScM0vgBZabG/ZheIg96INb9e+JXjSt/BA1bdpwmMJ97WpNqhWkypP86PHu
         mdtU+y30ankARF7w2WTq4cFhH9dkiQ1NhfCGZaXv8QQNdth4udnjIiAJpkaA3x26iNsh
         jlf9fXV6W+yz3cRXiCU4lxLJw9EfcYnIPRNDzcWfJr9PjabVA3HE3U3GQyJDTpBt43aj
         TIYg==
X-Gm-Message-State: AOAM533gSA4jgKutF8c7Yr0+9ia1x5+lXthavIZw4C5AUPmkTKQQe8BW
	XDE75az6A8TtRZBOke6ooVongw==
X-Google-Smtp-Source: ABdhPJyEB8CLbdwiUNdfZdvrZtOKbQbkduopskaqSrtZd3BkA7cXNgyeY28844ULJzTSWyVMUXnICw==
X-Received: by 2002:a17:903:244e:b0:13c:802d:92c with SMTP id l14-20020a170903244e00b0013c802d092cmr6486996pls.78.1631828465371;
        Thu, 16 Sep 2021 14:41:05 -0700 (PDT)
Date: Thu, 16 Sep 2021 21:41:01 +0000
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 01/13] perf: Ensure perf_guest_cbs aren't reloaded
 between !NULL check and deref
Message-ID: <YUO57TlEGlUk2Q03@google.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828003558.713983-2-seanjc@google.com>
 <20210828194421.GB4353@worktop.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210828194421.GB4353@worktop.programming.kicks-ass.net>

On Sat, Aug 28, 2021, Peter Zijlstra wrote:
> On Fri, Aug 27, 2021 at 05:35:46PM -0700, Sean Christopherson wrote:
> > diff --git a/kernel/events/core.c b/kernel/events/core.c
> > index 464917096e73..2126f6327321 100644
> > --- a/kernel/events/core.c
> > +++ b/kernel/events/core.c
> > @@ -6491,14 +6491,19 @@ struct perf_guest_info_callbacks *perf_guest_cbs;
> >  
> >  int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
> >  {
> > -	perf_guest_cbs = cbs;
> > +	if (WARN_ON_ONCE(perf_guest_cbs))
> > +		return -EBUSY;
> > +
> > +	WRITE_ONCE(perf_guest_cbs, cbs);
> > +	synchronize_rcu();
> 
> You're waiting for all NULL users to go away? :-) IOW, we can do without
> this synchronize_rcu() call.

Doh, right.  I was thinking KVM needed to wait for in-progress NMI to exit to
ensure guest PT interrupts are handled correctly, but obviously the NMI handler
needs to exit for that CPU to get into a guest...

> >  	return 0;
> >  }
> >  EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
> >  
> >  int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
> >  {
> > -	perf_guest_cbs = NULL;
> 
> 	if (WARN_ON_ONCE(perf_guest_cbs != cbs))
> 		return -EBUSY;
> 
> ?

Works for me.  I guess I'm more optimistic about people not being morons :-)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 21:49:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 21:49:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188784.338093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzFX-0008HY-P9; Thu, 16 Sep 2021 21:48:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188784.338093; Thu, 16 Sep 2021 21:48:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzFX-0008HR-Lm; Thu, 16 Sep 2021 21:48:55 +0000
Received: by outflank-mailman (input) for mailman id 188784;
 Thu, 16 Sep 2021 21:48:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aD9s=OG=eikelenboom.it=sander@srs-us1.protection.inumbo.net>)
 id 1mQzFV-0008HL-Pe
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 21:48:54 +0000
Received: from server.eikelenboom.it (unknown [91.121.65.215])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e19669e6-1737-11ec-b64c-12813bfff9fa;
 Thu, 16 Sep 2021 21:48:52 +0000 (UTC)
Received: from 76-24-144-85.ftth.glasoperator.nl ([85.144.24.76]:55038
 helo=[172.16.1.50]) by server.eikelenboom.it with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <sander@eikelenboom.it>)
 id 1mQzFN-0005CU-QZ; Thu, 16 Sep 2021 23:48:45 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e19669e6-1737-11ec-b64c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=eikelenboom.it; s=20180706; h=Content-Transfer-Encoding:Content-Type:
	In-Reply-To:MIME-Version:Date:Message-ID:Subject:From:References:Cc:To:Sender
	:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:
	Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:
	List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=+kmogzI7kwhnNwR3x3lgp5G8pQBmKvDxMAVeBzIgThY=; b=HzjMEh1Y5BxK04vXaYCv4o0Ryw
	AdEqAi5XN3Zw1SQnlpjMHhQawsYbhR4RISsinSFbSgJ789gyo0C0b9fZP9utaDIGaMp4KqojWkoIl
	RsBM5x1YMbDNG+QkRZ5ctr0ufXOPGTMK42vjt8W5MAL/Ux1fbt/wMhflQwqziHIFr6/k=;
To: paul@xen.org, Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>
References: <4dd5b8ec-a255-7ab1-6dbf-52705acd6d62@suse.com>
 <67bc0728-761b-c3dd-bdd5-1a850ff79fbb@xen.org>
 <76c94541-21a8-7ae5-c4c4-48552f16c3fd@suse.com>
 <17e50fb5-31f7-60a5-1eec-10d18a40ad9a@xen.org>
 <57580966-3880-9e59-5d82-e1de9006aa0c@suse.com>
 <a26c1ecd-e303-3138-eb7e-96f0203ca888@xen.org>
 <1a522244-4be8-5e33-77c7-4ea5cf130335@suse.com>
 <9d27a3eb-1d50-64bb-8785-81de1eef3102@suse.com>
 <d4f381e9-6698-3339-1d17-15e3abc71d06@suse.com>
 <0dff83ff-629a-7179-9fef-77bd1fbf3d09@xen.org>
From: Sander Eikelenboom <sander@eikelenboom.it>
Subject: =?UTF-8?Q?Re=3a_Ping=c2=b2=3a_=5bPATCH=5d_xen-netback=3a_correct_su?=
 =?UTF-8?Q?ccess/error_reporting_for_the_SKB-with-fraglist_case?=
Message-ID: <5a8c1f28-eefb-87e2-998b-7cbfb0f0b8dd@eikelenboom.it>
Date: Thu, 16 Sep 2021 23:48:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <0dff83ff-629a-7179-9fef-77bd1fbf3d09@xen.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 16/09/2021 20:34, Paul Durrant wrote:
> On 16/09/2021 16:45, Jan Beulich wrote:
>> On 15.07.2021 10:58, Jan Beulich wrote:
>>> On 20.05.2021 13:46, Jan Beulich wrote:
>>>> On 25.02.2021 17:23, Paul Durrant wrote:
>>>>> On 25/02/2021 14:00, Jan Beulich wrote:
>>>>>> On 25.02.2021 13:11, Paul Durrant wrote:
>>>>>>> On 25/02/2021 07:33, Jan Beulich wrote:
>>>>>>>> On 24.02.2021 17:39, Paul Durrant wrote:
>>>>>>>>> On 23/02/2021 16:29, Jan Beulich wrote:
>>>>>>>>>> When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
>>>>>>>>>> special considerations for the head of the SKB no longer apply. Don't
>>>>>>>>>> mistakenly report ERROR to the frontend for the first entry in the list,
>>>>>>>>>> even if - from all I can tell - this shouldn't matter much as the overall
>>>>>>>>>> transmit will need to be considered failed anyway.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>>>
>>>>>>>>>> --- a/drivers/net/xen-netback/netback.c
>>>>>>>>>> +++ b/drivers/net/xen-netback/netback.c
>>>>>>>>>> @@ -499,7 +499,7 @@ check_frags:
>>>>>>>>>>       				 * the header's copy failed, and they are
>>>>>>>>>>       				 * sharing a slot, send an error
>>>>>>>>>>       				 */
>>>>>>>>>> -				if (i == 0 && sharedslot)
>>>>>>>>>> +				if (i == 0 && !first_shinfo && sharedslot)
>>>>>>>>>>       					xenvif_idx_release(queue, pending_idx,
>>>>>>>>>>       							   XEN_NETIF_RSP_ERROR);
>>>>>>>>>>       				else
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I think this will DTRT, but to my mind it would make more sense to clear
>>>>>>>>> 'sharedslot' before the 'goto check_frags' at the bottom of the function.
>>>>>>>>
>>>>>>>> That was my initial idea as well, but
>>>>>>>> - I think it is for a reason that the variable is "const".
>>>>>>>> - There is another use of it which would then instead need further
>>>>>>>>       amending (and which I believe is at least part of the reason for
>>>>>>>>       the variable to be "const").
>>>>>>>>
>>>>>>>
>>>>>>> Oh, yes. But now that I look again, don't you want:
>>>>>>>
>>>>>>> if (i == 0 && first_shinfo && sharedslot)
>>>>>>>
>>>>>>> ? (i.e no '!')
>>>>>>>
>>>>>>> The comment states that the error should be indicated when the first
>>>>>>> frag contains the header in the case that the map succeeded but the
>>>>>>> prior copy from the same ref failed. This can only possibly be the case
>>>>>>> if this is the 'first_shinfo'
>>>>>>
>>>>>> I don't think so, no - there's a difference between "first frag"
>>>>>> (at which point first_shinfo is NULL) and first frag list entry
>>>>>> (at which point first_shinfo is non-NULL).
>>>>>
>>>>> Yes, I realise I got it backwards. Confusing name but the comment above
>>>>> its declaration does explain.
>>>>>
>>>>>>
>>>>>>> (which is why I still think it is safe to unconst 'sharedslot' and
>>>>>>> clear it).
>>>>>>
>>>>>> And "no" here as well - this piece of code
>>>>>>
>>>>>> 		/* First error: if the header haven't shared a slot with the
>>>>>> 		 * first frag, release it as well.
>>>>>> 		 */
>>>>>> 		if (!sharedslot)
>>>>>> 			xenvif_idx_release(queue,
>>>>>> 					   XENVIF_TX_CB(skb)->pending_idx,
>>>>>> 					   XEN_NETIF_RSP_OKAY);
>>>>>>
>>>>>> specifically requires sharedslot to have the value that was
>>>>>> assigned to it at the start of the function (this property
>>>>>> doesn't go away when switching from fragments to frag list).
>>>>>> Note also how it uses XENVIF_TX_CB(skb)->pending_idx, i.e. the
>>>>>> value the local variable pending_idx was set from at the start
>>>>>> of the function.
>>>>>>
>>>>>
>>>>> True, we do have to deal with freeing up the header if the first map
>>>>> error comes on the frag list.
>>>>>
>>>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>>>
>>>> Since I've not seen this go into 5.13-rc, may I ask what the disposition
>>>> of this is?
>>>
>>> I can't seem to spot this in 5.14-rc either. I have to admit I'm
>>> increasingly puzzled ...
>>
>> Another two months (and another release) later and still nothing. Am
>> I doing something wrong? Am I wrongly assuming that maintainers would
>> push such changes up the chain?
>>
> 
> It has my R-b so it ought to go in via netdev AFAICT.
> 
>     Paul

Could it be the missing "net" or "net-next" designation in the subject 
[1] which seems to be used and important within their patchwork 
semi-automated workflow ?

--
Sander

[1] 
https://www.kernel.org/doc/html/latest/networking/netdev-FAQ.html#how-do-i-indicate-which-tree-net-vs-net-next-my-patch-should-be-in


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 22:11:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 22:11:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188792.338104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzbf-0003HA-LL; Thu, 16 Sep 2021 22:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188792.338104; Thu, 16 Sep 2021 22:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzbf-0003H3-I8; Thu, 16 Sep 2021 22:11:47 +0000
Received: by outflank-mailman (input) for mailman id 188792;
 Thu, 16 Sep 2021 22:11:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQzbd-0003Gt-Nu; Thu, 16 Sep 2021 22:11:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQzbd-0008KH-Ii; Thu, 16 Sep 2021 22:11:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mQzbd-0003gS-7w; Thu, 16 Sep 2021 22:11:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mQzbd-0008Pq-7Q; Thu, 16 Sep 2021 22:11:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zNGTZ1PDKW/W9BE/I3Th9VYsAWZ6rbTu9z3+aqHHcb4=; b=DBhj8GU+fEQNC0VhEuF3MPPxlp
	BzSEa+7Gl5pwkLFgFZOmya0neNz/U45ddXSGhmzZ+UxH8CUGk1IZpkXDGcv4+kvkSe/U55xjcwRtl
	/bs6FLT70tiwBTuxckRD/5kjaWEJnOP++GjjjbxyeMthb6Qhu01Kp6P5SLQaIaqH/oCQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165010-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165010: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ff1ffd71d5f0612cf194f5705c671d6b64bf5f91
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 16 Sep 2021 22:11:45 +0000

flight 165010 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165010/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165003
 test-armhf-armhf-xl-vhd      17 guest-start/debian.repeat  fail pass in 165003

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165003 like 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                ff1ffd71d5f0612cf194f5705c671d6b64bf5f91
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  412 days
Failing since        152366  2020-08-01 20:49:34 Z  411 days  716 attempts
Testing same since   165003  2021-09-16 03:41:32 Z    0 days    2 attempts

------------------------------------------------------------
7299 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2256240 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 22:31:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 22:31:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188801.338118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzu9-0005zF-Fj; Thu, 16 Sep 2021 22:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188801.338118; Thu, 16 Sep 2021 22:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzu9-0005z8-Br; Thu, 16 Sep 2021 22:30:53 +0000
Received: by outflank-mailman (input) for mailman id 188801;
 Thu, 16 Sep 2021 22:30:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=BmaE=OG=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mQzu7-0005yk-QR
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 22:30:51 +0000
Received: from mail-pf1-x42a.google.com (unknown [2607:f8b0:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8c867e3d-7d09-4fd7-894e-c9a69cc21db6;
 Thu, 16 Sep 2021 22:30:50 +0000 (UTC)
Received: by mail-pf1-x42a.google.com with SMTP id x7so7239345pfa.8
 for <xen-devel@lists.xenproject.org>; Thu, 16 Sep 2021 15:30:50 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id b1sm3951150pjl.4.2021.09.16.15.30.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 16 Sep 2021 15:30:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c867e3d-7d09-4fd7-894e-c9a69cc21db6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=SbpJPJ4GLHhtaxBB1xHIdRhJVAuKGVRHlo9Wp9BtXvg=;
        b=igafNvMA1/3Bo/Hn8R9+USa5il0ulFxjx+nPIh2XyZCbl93vTOyT82CSDzdcKRJJgO
         FkJ5tfOZgSqPwIB2uy13Vsv6TyxDZo5SSSg6NO0tCl0m6iggfHrPIgqTtDvi9KZvPNYG
         8S05G6PCVB/DKu6Qps+LsPTZQ05wevCQFNZ+BICkpGZZdeZrOwvM4eNTBKGGZV/zaEby
         /Uo1noNrpUpRnME+TSkjRKmfn4qrb0maTPZTyYR72JBGteB/DiewqWAa7W2K5Npk1E/o
         h4Ealz2zUj5Dq5lsSJIpFakB2CCLO0hEG1zQ+M9xbiD1cysQ1HBn46apv1+McHJBcrYb
         Amkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=SbpJPJ4GLHhtaxBB1xHIdRhJVAuKGVRHlo9Wp9BtXvg=;
        b=59nd1ceaspCwzX7w3Mv86OQVxNXb0FVHc5ra7AN93doYd9zhQ3d5VN++eTLYyvoZIE
         VEh+LnotO2Y6zGxG8Js6vm3LPxjLGmX2Y7ZuQTrt4ZbeHCp6Fn5i9QUkEbVgBD/nAKV9
         RnGeRBmQjmxc2CnTUhGKySnC0HdL8xLTJAr0cVWtrBkcka3zR/Ewe8HLwPOZ6atht9ub
         iA/E1wZFHy75MWInajoml5GcqYTLAVoySXzpprre8hR84HRGL+OHhf0dALbLzs3zg4Kg
         G88TN+MBL2OzoBZ1+ojR5PE4bWKUlRFVFRZxZY/PMZieaht3HQlbPMJ+foqn1ac0pdX8
         q0LQ==
X-Gm-Message-State: AOAM533lbYKwd1g/Q3ZfVH+6FNT8ZpFhoQ4mxFuZ9VE6nZzg2llLfiBY
	hMAq9POTwkEF06zX27PCNwqt5w==
X-Google-Smtp-Source: ABdhPJx9Zno3oAszEpQSd/n4PnO2QNAuNkUGb12hjpp8HsLen9cxO4C/uNjsvLGvYRAHddpH9w+INQ==
X-Received: by 2002:a63:6ec9:: with SMTP id j192mr7022031pgc.115.1631831449675;
        Thu, 16 Sep 2021 15:30:49 -0700 (PDT)
Date: Thu, 16 Sep 2021 22:30:45 +0000
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 05/13] perf: Force architectures to opt-in to guest
 callbacks
Message-ID: <YUPFlb6r1udRKcBH@google.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828003558.713983-6-seanjc@google.com>
 <20210828194752.GC4353@worktop.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210828194752.GC4353@worktop.programming.kicks-ass.net>

On Sat, Aug 28, 2021, Peter Zijlstra wrote:
> On Fri, Aug 27, 2021 at 05:35:50PM -0700, Sean Christopherson wrote:
> > diff --git a/init/Kconfig b/init/Kconfig
> > index 55f9f7738ebb..9ef51ae53977 100644
> > --- a/init/Kconfig
> > +++ b/init/Kconfig
> > @@ -1776,6 +1776,9 @@ config HAVE_PERF_EVENTS
> >  	help
> >  	  See tools/perf/design.txt for details.
> >  
> > +config HAVE_GUEST_PERF_EVENTS
> > +	bool
> 	depends on HAVE_KVM
> 
> ?

Ah, nice!  We can go even further to:

	depends on HAVE_PERF_EVENTS && HAVE_KVM

though I'm pretty sure all architectures that select HAVE_KVM also select
HAVE_PERF_EVENTS.

Huh.  arm64 doesn't select HAVE_KVM even though it selects almost literally every
other HAVE_KVM_* config.  arm64 has some other weirdness with CONFIG_KVM, I'll add
a patch or two to fix that stuff and amend this patch as above.

Thanks again!


From xen-devel-bounces@lists.xenproject.org Thu Sep 16 22:35:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 16 Sep 2021 22:35:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188808.338128 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzyY-0006dg-16; Thu, 16 Sep 2021 22:35:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188808.338128; Thu, 16 Sep 2021 22:35:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mQzyX-0006dZ-UH; Thu, 16 Sep 2021 22:35:25 +0000
Received: by outflank-mailman (input) for mailman id 188808;
 Thu, 16 Sep 2021 22:35:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ef7n=OG=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mQzyW-0006dT-MN
 for xen-devel@lists.xenproject.org; Thu, 16 Sep 2021 22:35:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fe660cc7-c5c6-4ba4-9163-bf04931612c1;
 Thu, 16 Sep 2021 22:35:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7A8426113E;
 Thu, 16 Sep 2021 22:35:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe660cc7-c5c6-4ba4-9163-bf04931612c1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631831722;
	bh=9wf8RrJc7qaYHtjJvC7AdPnrubDpeoy0Zr2TU7L4zko=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mL2Wp6LfDlOUBgd23tUSsIcPBbkgqvjc38m+SCA6c+Y7qvoITjdMEN5IqsdoCqiIr
	 rI/uSf/+QmxnoQwUc1t45xwe3QANgYumBew/iPdG1ZeClzaxbbYMSPmwqpSPi6DDq6
	 qjkqmqyWgx9OR4Mr0A+R+Ctp+K4z1BAbRiDMq08XFGnWaAWmBlGQJoVaQZbIDSxNMB
	 sQkPYqup3vCoMgw+/4JKr0evBgmx9oZr87OkRKyaUn50SdWbXeE+9tgpAwOv4pSYqN
	 5yT3pO7S3D3gl1nzpG+U4Of93zutDTmWi5cEx8urzgWbPn8tG9wNhD0vJywrLymuyv
	 2Ty8BvY2CPXRg==
Date: Thu, 16 Sep 2021 15:35:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH V2 3/3] libxl/arm: Add handling of extended regions for
 DomU
In-Reply-To: <1631297924-8658-4-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109161516350.21985@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-4-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain.
> - The ACPI case is not covered.
> 
> ***
> 
> The algorithm to choose extended regions for non-direct mapped
> DomU is simpler in comparison with the algorithm for direct mapped
> Dom0. As we have a lot of unused space above 4GB, provide single
> 1GB-aligned region from the second RAM bank taking into the account
> the maximum supported guest address space size and the amount of
> memory assigned to the guest. The maximum size of the region is 128GB.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> Changes since RFC:
>    - update patch description
>    - drop uneeded "extended-region" DT property
>    - clear reg array in finalise_ext_region() and add a TODO
> ---
>  tools/libs/light/libxl_arm.c | 89 +++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 87 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6..8c1d9d7 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -615,9 +615,12 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
>                                "xen,xen");
>      if (res) return res;
>  
> -    /* reg 0 is grant table space */
> +    /*
> +     * reg 0 is a placeholder for grant table space, reg 1 is a placeholder
> +     * for the extended region.
> +     */
>      res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
> +                            2, 0, 0, 0, 0);
>      if (res) return res;
>  
>      /*
> @@ -1069,6 +1072,86 @@ static void finalise_one_node(libxl__gc *gc, void *fdt, const char *uname,
>      }
>  }
>  
> +#define ALIGN_UP_TO_GB(x)   (((x) + GB(1) - 1) & (~(GB(1) - 1)))

Why do we need to align it to 1GB when for Dom0 is aligned to 2MB? I
think it makes sense to have the same alignment requirement.


> +
> +#define EXT_REGION_SIZE   GB(128)

The region needs to be described in xen/include/public/arch-arm.h like
GUEST_RAM1_BASE/SIZE.


> +static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image *dom)
> +{
> +    void *fdt = dom->devicetree_blob;
> +    uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
> +    be32 *cells = &regs[0];
> +    uint64_t region_size = 0, region_base, bank1end_align, bank1end_max;
> +    uint32_t gpaddr_bits;
> +    libxl_physinfo info;
> +    int offset, rc;
> +
> +    offset = fdt_path_offset(fdt, "/hypervisor");
> +    assert(offset > 0);
> +
> +    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
> +        LOG(WARN, "The extended region is only supported for 64-bit guest");
> +        goto out;
> +    }
> +
> +    rc = libxl_get_physinfo(CTX, &info);
> +    assert(!rc);
> +
> +    gpaddr_bits = info.gpaddr_bits;
> +    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
> +
> +    /*
> +     * Try to allocate single 1GB-aligned extended region from the second RAM
> +     * bank (above 4GB) taking into the account the maximum supported guest
> +     * address space size and the amount of memory assigned to the guest.
> +     * The maximum size of the region is 128GB.
> +     */
> +    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
> +    bank1end_align = GUEST_RAM1_BASE +
> +        ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << XC_PAGE_SHIFT);
> +
> +    if (bank1end_max <= bank1end_align) {
> +        LOG(WARN, "The extended region cannot be allocated, not enough space");
> +        goto out;
> +    }
> +
> +    if (bank1end_max - bank1end_align > EXT_REGION_SIZE) {
> +        region_base = bank1end_max - EXT_REGION_SIZE;
> +        region_size = EXT_REGION_SIZE;
> +    } else {
> +        region_base = bank1end_align;
> +        region_size = bank1end_max - bank1end_align;
> +    }
> +
> +out:
> +    /*
> +     * The first region for grant table space must be always present.
> +     * If we managed to allocate the extended region then insert it as
> +     * a second region.
> +     * TODO If we failed to allocate the region, we end up inserting
> +     * zero-sized region. This is because we don't know in advance when
> +     * creating hypervisor node whether it would be possible to allocate
> +     * a region, but we have to create a placeholder anyway. The Linux driver
> +     * is able to deal with by checking the region size. We cannot choose
> +     * a region when creating hypervisor node because the guest memory layout
> +     * is not know at that moment (and dom->rambank_size[1] is empty).
> +     * We need to find a way not to expose invalid regions.
> +     */

This is not great -- it would be barely spec compliant. 

When make_hypervisor_node is called we know the max memory of the guest
as build_info.max_memkb should be populate, right?

If so, we could at least detect whether we can have an extended region
(if not caculate the exact start address) from make_hypervisor_node.

total_guest_memory = build_info.max_memkb * 1024;
rambank1_approx = total_guest_memory - GUEST_RAM0_SIZE;
extended_region_size = GUEST_RAM1_SIZE - rambank1_approx;

if (extended_region_size >= MIN_EXT_REGION_SIZE)
    allocate_ext_region


> +    memset(regs, 0, sizeof(regs));
> +    set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> +              GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
> +    if (region_size > 0) {

I think we want to check against a minimum amount rather than 0. Maybe 64MB?


> +        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
> +            region_base, region_base + region_size);
> +
> +        set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> +                  region_base, region_size);
> +    }
> +
> +    rc = fdt_setprop_inplace(fdt, offset, "reg", regs, sizeof(regs));
> +    assert(!rc);
> +}
> +
>  int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
>                                                 uint32_t domid,
>                                                 libxl_domain_config *d_config,
> @@ -1109,6 +1192,8 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
>  
>      }
>  
> +    finalise_ext_region(gc, dom);
> +
>      for (i = 0; i < GUEST_RAM_BANKS; i++) {
>          const uint64_t size = (uint64_t)dom->rambank_size[i] << XC_PAGE_SHIFT;
>  
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 01:35:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 01:35:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188818.338142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR2mJ-0007lG-6k; Fri, 17 Sep 2021 01:34:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188818.338142; Fri, 17 Sep 2021 01:34:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR2mJ-0007l9-3R; Fri, 17 Sep 2021 01:34:59 +0000
Received: by outflank-mailman (input) for mailman id 188818;
 Fri, 17 Sep 2021 01:34:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ysil=OH=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mR2mH-0007l3-QE
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 01:34:57 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 95e6cf11-41ff-43d4-a7d1-739c6f4b18df;
 Fri, 17 Sep 2021 01:34:55 +0000 (UTC)
Received: from pps.filterd (m0246632.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18GNVHmQ009148; 
 Fri, 17 Sep 2021 01:34:54 GMT
Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b3s74mq93-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 01:34:53 +0000
Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1])
 by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18H1Ue6u153268;
 Fri, 17 Sep 2021 01:34:53 GMT
Received: from nam10-bn7-obe.outbound.protection.outlook.com
 (mail-bn7nam10lp2108.outbound.protection.outlook.com [104.47.70.108])
 by aserp3030.oracle.com with ESMTP id 3b0jggy9e2-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 01:34:53 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4192.namprd10.prod.outlook.com (2603:10b6:208:1d9::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 01:34:50 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 01:34:50 +0000
Received: from [10.74.113.247] (138.3.201.55) by
 SN4PR0501CA0049.namprd05.prod.outlook.com (2603:10b6:803:41::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Fri, 17 Sep 2021 01:34:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95e6cf11-41ff-43d4-a7d1-739c6f4b18df
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=1RNWjIRij83DQK3/NgNDTxM5HwLq/MaxDMSD14Y6B0Y=;
 b=rCo1SX7DPP5a1KjkkjcN1nOFkxdMdSuUZRaHYDgncGJbMD6q54UF6GaTqFg/6YkaZS/s
 a6Hhq+8mhCsSMwg95oRC3gQH2Cqy6tem450y9nFG2KC1m/XOnknAVPQXUF3C7DfdOwfr
 OzF4ABxGHqFOQRgPl2aB9kwnR5XrxiG/BTk8EPsRUJY5uG/9vxp0AWWY4vc6P8dXB/tc
 yLzrscbvzKKVfwfed5uz0XDHbqNTHZ85qSmbqk1XfcfJv9A/HE3G4h78FbqG3bn5u8px
 3rEUFufGVmK8Zg3U+IH6Vm2hy1LbmEMtgU77GA/x62W3DsHxaqXzSe/5jFnWxB9D2RxS vw== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=1RNWjIRij83DQK3/NgNDTxM5HwLq/MaxDMSD14Y6B0Y=;
 b=Ym6e3ClpPUoFvcHZjUBasKuzb/6ATxF3eX+ZgGQmwYKmK5MXFfjegHOPRmJjGCagbfvM
 PymQrUIX6bIMSR7lSOhqUsLL40sKhrC9JS4/LLjmBO5gcKaYvHGSL6ox5uJSAPy22BYx
 bdSiCf2voxUcfEH4CjGDSlpoqoF/GpppBWPBGUOiSXz/dJbPy60LejLzZIZOCsj06dSQ
 P4KoZiOEvbhJRTg0/9iWkjU81BX4htCKn9aE+Uxl5ho/jnbq7Fk5oOoK6kFiMv4QxwsX
 gI/JZJgKvYuIx2vYgYe/XYdfQenedyzciZT63phTyNWaZRhRDUj/uwO644sbAyPM5np9 FQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lZoue0kQ9hS+QxdWfOuNj9YXy2VooSz35ZP76e5DnPV+r9SP1BSq8NCalygQyM7e8URmCxmzyC2kM6oUturdxe9NtmwzyAFPZRCNdkCSChvfmlohpvPYlx7yyXDiNTwAUd/eHEJCnZZbBkvNA5prot28+eEN4s96FQYNPPK8K/0bzUjt8+2W0sOSmW1dxEfh9j4cyScSLCVrU4ai85N1dTTRLjHHf2p0N8xhCjaGxT2jNimHV3J1USZDLBTDVq3b0HDlVzgRwtM7mw5AMTuJ88UBB/FykaQhPDxs9Xocg7X8E4r/8QIA0GUC6rzG3j9uc6I6n71WVaCF3QfkZfiTJA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1RNWjIRij83DQK3/NgNDTxM5HwLq/MaxDMSD14Y6B0Y=;
 b=NVfHi3hRMhIFVLp0heb7wQyeTkNItJuMPomI8o5SW6EFOkVBWdqQqug4GvoiamlWLzmPF6qTbcgVA+CYXlXHGqionhCmYv7UdukEP+zxWR+WKqUju7ERZtZOP28Tnt0IZSAEOEK1aXxXVvl1x2S1OA/CHYlSaPfdfSz7G7gY/v6oxHYOl8/0yVZGVx9VV01nL4lK1os5UKc10SDvbQ2U1HzF8TP5Lu/KpXJaVS3loUrvO3ixC4IJeYu6bt0BMUUE+GyUn7te9bMAlWnTAJ9lumunXd8x2Unz8Nl3nCbeKra7BQkrAGS/ivwKw3izfSiMFp2g6V4mrTva7beg5PJEOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1RNWjIRij83DQK3/NgNDTxM5HwLq/MaxDMSD14Y6B0Y=;
 b=SVHppOt29vmUHeL7CGZOlTSnbX2MIA1RnfAKToP986nrrZK75bV6n+0cP1NHD1339D+3pMf7ptjGCAn1/03jhYOWuw1jsmxKm3sThTqVZJouHorbat1EPL42V2kLTDGTIBp1FC6QpZaAU+HgvNWEhR62ivq2WBQfC7FCq5epc2I=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=oracle.com;
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
Date: Thu, 16 Sep 2021 21:34:46 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN4PR0501CA0049.namprd05.prod.outlook.com
 (2603:10b6:803:41::26) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d4f6588c-cc38-4330-8580-08d9797b5737
X-MS-TrafficTypeDiagnostic: MN2PR10MB4192:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB4192514D1C8CEA483F06AB968ADD9@MN2PR10MB4192.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1332;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	bLZSPoPbp144LQEoKNV3zMBSWjcIbuSpuGbNgq3AWEJoAm5fbEgvK4O3zO0qFUajYOCM+3WpbkoS/ly2JvTNxv+SUOK9BIz6L6TJKM5hB+enG+h9jTviFmLFKg0lCZRAOjnMdsATVpi2iNngWmFbr84MEMEBU3LUzUD36N9rPrF6qMIqEocFc57Nm4es+hYhzcpVBvMft1Xxht409GAdmC+Ba67oNcVqRduJ3WfwPY9YvgRdfNyuwuQRCQsncjWcCg1MisDcHu5ir2mCUBIOQ1B0MdsbvqnV0WRrcfAO7I4K1L3qh//jxMwWOPGGtDRg9yx78XRtf1v7Y5l3P59iU4Ng99/8gaRPggXmhv9aAJU2N8rkEJ+Jj2mWSvpp7HveOFlXCstU+eZBjbRJh5MDKell3IW7X5G4v0ramdk3w56AWhiKcYlaJSYhwNOIpeV8ri/2wjmgN/PtneL79hLyr6q9fBJ9TwaAwhZC/+ZU/KJm66OhYInkjhZoV+xWYAyYZkw3SpSWlcIjBa7QHPILz0L0mWE2vnQRlITnhH0bPRuGp1oqHKF+c1Ty0L+MckJlWAaHOhOMP/uV0LoTb71gueWCoYg823cveGZtwqPHKXIiV6N0jrU9Ps3k1Li7sRboaOlRyEjN7VerxYfXdN5KnoNelzjmHpK/qOBK2e9qotI5+bT5+IJqZS87XFl+H+iURfVXIXlRlkJa97Hh1MvOgMLONeRUlm99feBg07BRKprvu2JAloWKhxqK7hz3gQya
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(136003)(376002)(366004)(39860400002)(110136005)(38100700002)(4744005)(5660300002)(6666004)(31696002)(8676002)(478600001)(66946007)(66556008)(66476007)(316002)(44832011)(4326008)(16576012)(2616005)(8936002)(31686004)(2906002)(956004)(86362001)(54906003)(36756003)(6486002)(53546011)(186003)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?cXI0ZGZGbUZNRWtXQlhkNnVSNldRVGY1NFVFZUppZmdlU0RFbmE5bnpLM3Zv?=
 =?utf-8?B?ODFWS3dNZThDRzdraldpekVOZk4rUWVRdjRsQlp2RkNtQVcxYVhoUm5JeFBB?=
 =?utf-8?B?RkFnbndwVHdMbjNmbm5hZGlFS0FhV3pXRVFKbVNHbDg2ZEdIU2dCOGdJM2xN?=
 =?utf-8?B?TGRsWDFyQjh3RVErS3VMZ1RlbmVFdEFPeEJramFXN0NjTjVCRGFObVErQnhK?=
 =?utf-8?B?blhMYVd5eG9hNS9tazRXMEpKcnAyTnpuRDIxMWo2Mmp1dE5QWXdicnZUUEtJ?=
 =?utf-8?B?RjZra2poQkdaYXdweXZTSU00NXNORXkwOWlESXRjSURhcGJ6N1BHdTJrZU8r?=
 =?utf-8?B?QllRRGpEUTNnNWc5YTZpNE9kUE1QOEtCUEtxVVRXWEVNUkcyNml5UE1LUndQ?=
 =?utf-8?B?aExGRXFlNk4yY0xIVkxSdkc3ZUxjUis0WkhoNFpVRDVsWjNFSFNWaVIrOXVO?=
 =?utf-8?B?d3JmN2IrSk9BNUNDakgyaE8rNERnc1Q3dmphUWpkTTI4VzhkTkVXMTdrSmFl?=
 =?utf-8?B?cG9yRkdKcVdRU3lpSzJYaDQvM3o5anorL2ExSVYweDhtRlE1TUVFZnhJWmhQ?=
 =?utf-8?B?UGVjaXJxRnM3Z2xJcWUwZC9pWWQxK2RPaE1CRUZYYldnSjFuN3VVYWl3emlL?=
 =?utf-8?B?SVNjREpPNHNoNS9zVnF6V0RSTXVVbklRWERRcjRSLzlRZk9jUFl5Zll2b2lC?=
 =?utf-8?B?emdkbWdXZWdVMlRZYXJ0cmN1MjdrL3NkaWRlSno2emtjTERGYWIvdTZvY0NH?=
 =?utf-8?B?TVZwUHhpaklKMEpoVGRMRWhxYUlWRitSU1dtUzVaMnF2aWN5bU5mcEN5eWJG?=
 =?utf-8?B?NExSVjNBeFBSbzVQVlh6ODFHZWhLMEh0Sk5qcU1qdFAra1pVd0NTcG4va3Vp?=
 =?utf-8?B?SnF0SGtWa21YWm5PZUUrUzlyTVYyM1QreGVMeVc0YkNjbUhNb2x0dUw3Q29N?=
 =?utf-8?B?R3V4UG5RSUNMSUtUeTFuT2o4UWxMU0FEdWF3MVZjNjZqeWNXODNnbForNXlQ?=
 =?utf-8?B?alh5MUdlMGZGRTZUaDc4cWNqa1cyUWlMOE1uNnRjMy84M3FCY0RMSmtJZjhy?=
 =?utf-8?B?N1I2WFZCR3BaQkFPZXRiT3BDRU93L21xRzEwRUpsT0JpVUZOZkJxenpWTnVF?=
 =?utf-8?B?UUZ2bi85ak9JVlowc3ZiU1NFRUc0eVJqZURBUGYwM3BXRHdhRVorOSsvS0VG?=
 =?utf-8?B?V2lYWTJoY01FNGM4RXZpM0NkdmcvV0lZeFJ2SnlveHNlRENqb2haY1lhRDFw?=
 =?utf-8?B?QjAyWkRpYWFUejcrOFBBMXUvTmxYMEczZ1l5eUlZeXZ1U0tXNXFJTUY5ajVj?=
 =?utf-8?B?QTdKMkZKRjIybHhWMjloNUlVbDU1K01LQ2ZQWHJVbEs5bVJ1WDZlWElDWmZS?=
 =?utf-8?B?VFVQV3p2ZVFSS0FWWk9HY05jWEpDcktIMWZtSWhsMmxKVGd0S1dlckVwZ012?=
 =?utf-8?B?cE1YVTNJZzRJMVExZlpyUHRRMU5xcFhTMTRoOVhCV1VNLzNHaXNGcWRxbTN5?=
 =?utf-8?B?SS9nV01RR0tqTnhkd0hKeDF4OFB4Y2hJemFFckJScUJvdXU0TnNRWUlkRVN4?=
 =?utf-8?B?c0g4bys0MU84S0VCWnJtN1RVa08rVTZwSkJHNGRJK2pnV3REM3gvQyt4L1R5?=
 =?utf-8?B?WThOOENBZmdieG9pRzF1Q0o0cit4aFZTc2tkQW9PUUVnbjNTOG9NN09jclha?=
 =?utf-8?B?Q3pMbVZPOHc5VUdrNU5IbjRPQXlCTlgwM0thTmdrOG5SQ0doajhRL2UyRWhN?=
 =?utf-8?Q?b3cqGdbMAu0JOJpuXym5mf7FUAnPpc89a35/gZm?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d4f6588c-cc38-4330-8580-08d9797b5737
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 01:34:50.2971
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5VDwvB/jq5c6PXezWaF6Ad72HqmYdRu9wBn5PEW7AStnjADhhnuOxwlvy0Bkf1szHS8lQAeNkEX+/ZnyGFb1Mj6AM36sl0VfepUZltOi0P0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4192
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10109 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxscore=0
 mlxlogscore=999 adultscore=0 bulkscore=0 spamscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109170007
X-Proofpoint-ORIG-GUID: GhWPyo4Tx_KQtWfNAsjXhmxVIymFwbxN
X-Proofpoint-GUID: GhWPyo4Tx_KQtWfNAsjXhmxVIymFwbxN


On 9/16/21 11:04 AM, Jan Beulich wrote:
>  {
>  	const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
> +	unsigned i, count = (desc->size + 1) / sizeof(gate_desc);
>  
> -	xen_convert_trap_info(desc, traps);


Can you instead add a boolean parameter to xen_convert_trap_info() to indicate whether to skip empty entries? That will avoid (almost) duplicating the code.


-boris


> +	BUG_ON(count > 256);
> +
> +	for (i = 0; i < count; ++i) {
> +		const gate_desc *entry = (gate_desc *)desc->address + i;
> +
> +		cvt_gate_to_trap(i, entry, &traps[i]);
> +	}
>  }
>  
>  /* Load a new IDT into Xen.  In principle this can be per-CPU, so we
>


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 01:35:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 01:35:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188825.338154 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR2nC-0008JI-Hz; Fri, 17 Sep 2021 01:35:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188825.338154; Fri, 17 Sep 2021 01:35:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR2nC-0008JB-DN; Fri, 17 Sep 2021 01:35:54 +0000
Received: by outflank-mailman (input) for mailman id 188825;
 Fri, 17 Sep 2021 01:35:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ysil=OH=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mR2nB-0008J3-Sp
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 01:35:53 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9812f936-1757-11ec-b65e-12813bfff9fa;
 Fri, 17 Sep 2021 01:35:52 +0000 (UTC)
Received: from pps.filterd (m0246630.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18H0aVTU026758; 
 Fri, 17 Sep 2021 01:35:51 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b3jy7nryy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 01:35:50 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18H1Ti6J139848;
 Fri, 17 Sep 2021 01:35:49 GMT
Received: from nam12-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam12lp2173.outbound.protection.outlook.com [104.47.59.173])
 by userp3020.oracle.com with ESMTP id 3b167w2xn5-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 01:35:49 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4192.namprd10.prod.outlook.com (2603:10b6:208:1d9::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 01:35:48 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 01:35:48 +0000
Received: from [10.74.113.247] (138.3.201.55) by
 SN4PR0501CA0045.namprd05.prod.outlook.com (2603:10b6:803:41::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.5 via Frontend
 Transport; Fri, 17 Sep 2021 01:35:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9812f936-1757-11ec-b65e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=V1kCTDQ+5740MBTc/UN748rPeIbBl3mcCLheJp4qkdc=;
 b=CfUC+ICBBekfLy+NJbH5hbOOAxFbWGIzKPqE+/pkz5fraBFMa8RKlOGhE2QJc4O4Wt9V
 ynAihZxO1bDr7TaQg+NNEhfpb2Fdz7dwLNRNAUmlleIZSSmpYWKll+Phg2XpE912fDKT
 Qf05CZFfp/lPAmpH+F+wgHxUpkhGNhhLg2LRtbsEewK4oBuLeP2smw9iS5ns4Lag+fQF
 hka0rATWNTSh4UPTMoBHS20tEeJD4DhnSvSLH4dd3GCR0oO6Lt5Ey01ztLLBl1P5glFo
 PD5DcHms8EjV8aWeA22WW6OK1HkaJ+s/Z03qTdfE08cwm7atB5OOXE0IRSqa5px9c2pt Mw== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=V1kCTDQ+5740MBTc/UN748rPeIbBl3mcCLheJp4qkdc=;
 b=iIZtFHT9cZMV7rqXiErG80tjOyen7fZSxADIzeWZZQk5D79gxO2HLY/U9C3n4puXmrZ3
 awKil23HfMVY0JmgGymr1g1+vKPYUIY5xZwTZ3Xfu5wW7/rxnqCkFjfqDWbe32GFHtBW
 BH3CrdgBp+94iCumKEAzaxMsj30aNFTtDbYzCVIhGnZeCvf60GNqsvksgI+Kcr7W5Gj/
 lAuurHAPhwg98tpBCPZnW/NAmmp0gCv6Gee2d3+/ftR3wGxXy6slhJVFMxGGWFVJgmdw
 JVD3rS4GhQz99u6wQbNsDGJx0EGYmdpcalrSpWMPUVg4fC1wOWlss/KKlvwSOU3HcLfN BA== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mnreMX8uyV/4zleVxMfRdwGL9TvEYs++UPZZ3SjEhjOVEPBLWEFP4QD8gTOySZhfYwe5mV8rn48hhtK8FvBp/fwHOsYhOcnmPVlcni/ZJtINrTy4DDQGxGodsq1Ez8GMGIK5z4s1RZ2znotJXwSOlg5c0YzxyV5wkxOkcGCk+3Dou3aoGCU59dvfTM7P/eeu6o5pn10ayxHSIkpwm2QFW4A88ycAFFIWUPZkZM4njXzQz+Z1OogufOFsef7xKV1cgwHpa8LA4HHkSGc4J4l9EzUqHA6+CRX09pDIlj8Fp5hlG5dkuUFiWR3cb0ByYcnEB7N4Qtt5z3jd4ZviAL+45g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V1kCTDQ+5740MBTc/UN748rPeIbBl3mcCLheJp4qkdc=;
 b=TkXX55ePThMzjSZImgQJYwyXCULXN3EjV1CgY2OzLB1A4UHPvF3o+daPCrRjqaymQD5KZCktqOpf4iO9P3TEBIri6TTcsR9mm6c/werCYkM8cBivH1sANccvRw1ZtaWuP2jnmeTGQG777xSiP0Ke3MXG5++ftUz5P7RXh34758RW6P4fK452VWOF6FrQtSTxaUpbIot7dnBsHQVb6lpzR6sngVi5Ti4+JNsebiO1bRMOH65xyYwEHFaAe6oeMow9ntq2cQWJx1snf87h5yV2sqAow6vEqWCgGXnIQ3inbxcbfo1twJF2U+qKIa0XRmerIO5/Z6iyjnVdz1q//AQy0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=V1kCTDQ+5740MBTc/UN748rPeIbBl3mcCLheJp4qkdc=;
 b=dg6QRNmwmop19mO3vUIEC1Ae7vSPX8+4HavVo1fR4Lw7vW8xZYfdjoZfqSJnsDTHzkAEzw89xPi03wnlQhg3A7RRdMhnO1OPgnTIUvEDhq1EMuNLMpYbzO1544zM01+2WwKeDxfv3YY4O8aPLmU6M+UoXDbJb5qHpZ/haJ0AkU0=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=oracle.com;
Subject: Re: [PATCH] xen/x86: drop redundant zeroing from
 cpu_initialize_context()
To: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <3ea71751-1de9-2da8-9a90-b4d22277c6c9@oracle.com>
Date: Thu, 16 Sep 2021 21:35:44 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN4PR0501CA0045.namprd05.prod.outlook.com
 (2603:10b6:803:41::22) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 18620748-a5b3-4779-1eb7-08d9797b7949
X-MS-TrafficTypeDiagnostic: MN2PR10MB4192:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB41928F31D6EC7D87AB36FEAE8ADD9@MN2PR10MB4192.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1002;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	lBWcWQzGtf+fRbqZk42Iyfiw0KKVfDJr3sdnhYSBgcKV3Rxzg/pp3STQ6X7/TDez2a3B/dnIpw1i8H5NHLfND52LL1ghR7pwTFBTFKDPY/R2wuLEe0GJmB/Ar4LCifHzkom4enMbyJHwvofgu1KTbyZyCDkFTeSX8/x7DCMCZbrPm36ajfFgpWS4zCIN0frvwx94bsDlNUW8Hhf4OFtMp9S7MTfZzTPw/9FrNAefuzF587li0FMnhuwWiHWLz7i06SPjQPzBvHgYiLMU7sn1gBuNK4wHsAJo1EYQjyyAg1mHa7KXQ/bWvv86GY2AEuoTjHNO+MSbfkuC+QHsolur+W7XBhT9kd2WUw2BH8MlVso3lVf4JITbprM8DGC2W8THSmUzEipDh7VYp2ppDFo2LE+cv/uBPqFd+F7z2d712p/trJUGQMm+RHX7If48H1eRE9cGJH+GmtQLR6OZBAyf6nin+4+VizLORTwG5yrDXXj9H/d748foL/wDNd+gvWpnVT2cXvY0iwzilbx1tPglFMB8Hd0GUhlvg4Govo7OVwRfGmZcQfGEFt+qWV4Nt0P7ged9+qiRKCLt+YkXxZLwq/pecdaW9zFaAvhvC14FENm6qZwLErofc823l/DVjFS5IH3rkKbJuW/Sq3hhFFcgSijXAC2bdDCFUzdYu/P1PqKtTUjvy3Wa/UGzCYuy0ivIOKNd3ZDVruIMBzBDkmPQB5v7icXBaMVJly9q+kWjYEBMUarir1PPlp/2cw4sLlDU
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(346002)(136003)(376002)(366004)(39860400002)(110136005)(38100700002)(4744005)(5660300002)(31696002)(8676002)(478600001)(66946007)(66556008)(66476007)(316002)(44832011)(4326008)(16576012)(2616005)(8936002)(31686004)(2906002)(956004)(86362001)(54906003)(36756003)(6486002)(53546011)(186003)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?T2JKVHNtRlAxcS9IczYyWFVPeit4eXRyMFREdmllT3VMcXlNaGtHcmlWTW9p?=
 =?utf-8?B?TmtiYjJJYXFGMVVzdGxPckhoRm5raHpNWjVWbnNvTDBheXdoZklTaDNtVHZp?=
 =?utf-8?B?c2l2QkpXaS9NZE03bmpKN1AyTDI1UzdsYkx1bUZvZzJBK1o5T3drZVJBUnpC?=
 =?utf-8?B?eEdZVk9Fa3NERmFwNXdRY2RaTUsxQmVMbU44TGlXanFwTFptVHNjL2JLcDlp?=
 =?utf-8?B?WDVZYllVcERxTmFFZk9sUkhFMEk0WXRwU3p6OWlQTlVqeThST09uZU9VaFpQ?=
 =?utf-8?B?cEhYV3llUHBlVTZjMmhRSk42YWVBRmRENFc0UTdiM2k2cWI0Q21PWUtGdUx0?=
 =?utf-8?B?SUF1UzVTdC9CM3plMXdSWVZacG1ONlR0STU5ZlZ0TW5iT3pGa2dNeUhkcVVC?=
 =?utf-8?B?b3NCSzFWZmpXUmVwdHJiRDhzR1pTSXRmNkpUeS9iZTFNbnE2V3k1M3VrTzkv?=
 =?utf-8?B?YnZmZzk0M2VaYjFrTWtBdUdhWlBma05DZ0tWSFR1d3EyTllyOW9HS1BRL2xo?=
 =?utf-8?B?aS9nYUc5Nk4vQmVZQ29EN0kzenM2b09zdnF4dGs4SUZNSmJ6VlNIY25lbVRn?=
 =?utf-8?B?ZllyaUZDRmwvVCs0SzZoNDJna1lZQWgwdDNQUzVqYk5FOTY3VGY4RTRxbXJz?=
 =?utf-8?B?b1c0U2REdG8xZ1lDRXR3UEY1ZnVHSWNBOVVpSW9Oa1JiQWhMUC9GQmp3S1VY?=
 =?utf-8?B?Tks2VkJNczROTythUDdxWVQ2cTAzRFZTSHl5dHVqOGRkaTh0SUlXK2JWL01h?=
 =?utf-8?B?YzZRZGM1OXgxb2tKRUkweHRZeVBQckZIOEt0YW1WQjQ0Zi82R2EwMGFxNnMz?=
 =?utf-8?B?WDl3c0cxV0RkWE5KMkx4cGpnV1RuVWN1Zlp1OGU1bUc0dW1mNkxyOWpyL294?=
 =?utf-8?B?cjgxNWIxSitZY2hVY3gxRVcvdys0UTRiRUIwOEplOTVCUDhkQmhsb0Y1Vi9K?=
 =?utf-8?B?OWFUcFZTbDhaK1V2QkkrZ0dQR2VqdnFvOWRLcXdoN280NzFnSDN4dFhoUzY1?=
 =?utf-8?B?dHhpWDFJVTV4TUhCdzdTYi9BNUtqM2Nyb29Pb3N3UzhEMm4vMDBQWmZvaVk5?=
 =?utf-8?B?NDlrNHFHNTBqMWlhSDFmZ3ozNTVVbHp6dDlnUXk4MGVvVDRGQmpIWkVOMHJN?=
 =?utf-8?B?Y0tIa1Z3YVFBajJsRWVVUnNyUTB1VXE1d09Kb2pKczZjd1Y4UHIyTU1reUlI?=
 =?utf-8?B?b0NKbW52STkrOWhnSHpGcTBMRzZmMm56ODZGNkErSVRmSlNKMVRrV2UxM3FV?=
 =?utf-8?B?VURUOU01M1ZmUVFuaDVtb0ZMaHNkZnBOVzlWOW0wamhxQXBVa1g4dDNpajYy?=
 =?utf-8?B?ZXFSY2huVTZhUHp0WnJZcWl0RTVLYlJXb0dISmFFbU5kUGladHAxOWRSeHBh?=
 =?utf-8?B?U2dYVFJSV3dVUmJsaG9SZkYySkxaOFBWRlR5YWZlLzVZOC8xcWhoVWI3cW5Q?=
 =?utf-8?B?VUljVForSisyd2FkK2s0Q2NlL1FUMnowclU0d0tDNkRkbGtRSzQrSlFXVkJh?=
 =?utf-8?B?ZmNQNlFxQ0I0b1BsR2R6VStEZTlEbEhWMlNFalZlby8rNElBSGhkaVZuY2NG?=
 =?utf-8?B?UXVUS3hlUjdqaUg0SE1PRUc0VlBYU1FkNy9sVi9haDN1TVRVejRtTm4zOEVa?=
 =?utf-8?B?azRsOTRwQmpvQXowU2JVWkZGNFVOYXIvR09KVkIxNGtNZkl1VU9IYm9uUTY2?=
 =?utf-8?B?UzRYU0gydGpQVVFROGRibEl4ZVhTNTZCVTIxWE1VQ25QaXN3aC9rbzNMeVc3?=
 =?utf-8?Q?TaxMLWYtwtdJHNK73Hc2LBJUYLJFaLAsLi+1h6w?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 18620748-a5b3-4779-1eb7-08d9797b7949
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 01:35:48.0424
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: z9rxFLvIJx5ESFJBd0WdGwb8laajDXY5wrxrHvmtyL7MyegTCBC3PeNiG93L+hK/ODZFzv1iRgZF0ycPK8Ho61tzVTbZMcYcB8yOrLG7ZiA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4192
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10109 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 suspectscore=0
 phishscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109170007
X-Proofpoint-GUID: 8Lvy2NR6RkG2NNG8CW1GBnXtIdCjCt7d
X-Proofpoint-ORIG-GUID: 8Lvy2NR6RkG2NNG8CW1GBnXtIdCjCt7d


On 9/16/21 11:05 AM, Jan Beulich wrote:
> Just after having obtained the pointer from kzalloc() there's no reason
> at all to set part of the area to all zero yet another time. Similarly
> there's no point explicitly clearing "ldt_ents".
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>


Reviewed-by: Boris Ostrovsky <boris.ostrvsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Fri Sep 17 02:31:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 02:31:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188833.338165 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3fF-00075P-0D; Fri, 17 Sep 2021 02:31:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188833.338165; Fri, 17 Sep 2021 02:31:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3fE-00075I-TS; Fri, 17 Sep 2021 02:31:44 +0000
Received: by outflank-mailman (input) for mailman id 188833;
 Fri, 17 Sep 2021 02:31:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3fC-000758-WE; Fri, 17 Sep 2021 02:31:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3fC-00034S-GM; Fri, 17 Sep 2021 02:31:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3fC-0007O7-4U; Fri, 17 Sep 2021 02:31:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3fC-0007EA-2O; Fri, 17 Sep 2021 02:31:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gc8EbGWbEuVj5eRX/GmXJJOAj5clp2UQ2lhPuytDN00=; b=TVCyQLJWCcCH/r4f3IndgoapTT
	UV0kkT9BdHTdZ/5UMZ9QBSLgQJgxQx85wB88x4g0aBxcQrpm402f+GYzGgb+CwpJvuTucb6xns4DS
	vRG6VgRpMxshuwlYe5kDO/Ua4v/Ucf/e9gCnxAFNAGaX+Oug4MdSxpU46Yt477YcCTVU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165013-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165013: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=ac6388add4ade3ae5c4036ea6c2ce9c8d301d057
X-Osstest-Versions-That:
    ovmf=f4e72cf9d665a8f1a54170b0b62739a628823c8b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 02:31:42 +0000

flight 165013 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165013/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 ac6388add4ade3ae5c4036ea6c2ce9c8d301d057
baseline version:
 ovmf                 f4e72cf9d665a8f1a54170b0b62739a628823c8b

Last test of basis   165007  2021-09-16 09:41:13 Z    0 days
Testing same since   165013  2021-09-16 15:12:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ashraf Ali S <ashraf.ali.s@intel.com>
  Dun Tan <dun.tan@intel.com>
  Jason Lou <yun.lou@intel.com>
  Leif Lindholm <leif@nuviainc.com>
  Lou, Yun <Yun.Lou@intel.com>
  Nhi Pham <nhi@os.amperecomputing.com>
  Tan, Dun <dun.tan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   f4e72cf9d6..ac6388add4  ac6388add4ade3ae5c4036ea6c2ce9c8d301d057 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 02:47:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 02:47:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188841.338178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3ud-0000Yk-CS; Fri, 17 Sep 2021 02:47:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188841.338178; Fri, 17 Sep 2021 02:47:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3ud-0000Yd-9H; Fri, 17 Sep 2021 02:47:39 +0000
Received: by outflank-mailman (input) for mailman id 188841;
 Fri, 17 Sep 2021 02:47:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3ub-0000YT-OW; Fri, 17 Sep 2021 02:47:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3ub-0003Kg-Hs; Fri, 17 Sep 2021 02:47:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3ub-0007yH-5w; Fri, 17 Sep 2021 02:47:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mR3ub-0004kC-30; Fri, 17 Sep 2021 02:47:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YRWlP7KnazxMWU0BEZ4FD2MGKJ0McTfBZSnt9C49/hk=; b=lsEjUGslbVQVn5PEQ3M3CBuuJd
	y0Nf/cxG3Ue2N3n1wB/lSnEEcZKxub/19eZQ/Ft2d1wwBywq864aOh6Du+IZLe0m2/0tED8ZU/+ax
	WzslVH6+ZgMdjlZYKTwGH1FBPjtJZMEj3+Su6Tzo8QH34aDb6gHgOJeEC9t1QGAXkdes=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165011-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 165011: regressions - FAIL
X-Osstest-Failures:
    linux-5.4:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    linux-5.4:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=48a24510c328b3b3d7775377494b4ad4f58d189a
X-Osstest-Versions-That:
    linux=245f15a48cdc4d5a90902e140392dc151e528ab8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 02:47:37 +0000

flight 165011 linux-5.4 real [real]
flight 165020 linux-5.4 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165011/
http://logs.test-lab.xenproject.org/osstest/logs/165020/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 164993

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164993

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164993
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164993
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164993
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164993
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164993
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164993
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164993
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164993
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164993
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164993
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164993
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164993
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                48a24510c328b3b3d7775377494b4ad4f58d189a
baseline version:
 linux                245f15a48cdc4d5a90902e140392dc151e528ab8

Last test of basis   164993  2021-09-15 08:12:34 Z    1 days
Testing same since   165011  2021-09-16 11:11:20 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Sasha Levin <sashal@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 48a24510c328b3b3d7775377494b4ad4f58d189a
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Thu Sep 16 12:56:14 2021 +0200

    Linux 5.4.147
    
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 1f8ee024498d00d4f20a34278c23a6d77dd37616
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Thu Sep 16 11:12:30 2021 +0200

    Revert "time: Handle negative seconds correctly in timespec64_to_ns()"
    
    This reverts commit 7a25a0a94c8b49759582ac6141c06af4f3e8ae8f which is
    commit 39ff83f2f6cc5cc1458dfcea9697f96338210beb upstream.
    
    Arnd reports that this needs more review before being merged into all of
    the trees.
    
    Link: https://lore.kernel.org/r/CAK8P3a0z5jE=Z3Ps5bFTCFT7CHZR1JQ8VhdntDJAfsUxSPCcEw@mail.gmail.com
    Reported-by: Arnd Bergmann <arnd@kernel.org>
    Cc: Lukas Hannen <lukas.hannen@opensource.tttech-industrial.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit dc15f641c6ccf59b157c39a938b74298e6392b98
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Thu Sep 16 10:51:42 2021 +0200

    Revert "posix-cpu-timers: Force next expiration recalc after itimer reset"
    
    This reverts commit c322a963d522e9a4273e18c9d7bd6fd40a25160f which is
    commit 406dd42bd1ba0c01babf9cde169bb319e52f6147 upstream.
    
    It is reported to cause regressions.  A proposed fix has been posted,
    but it is not in a released kernel yet.  So just revert this from the
    stable release so that the bug is fixed.  If it's really needed we can
    add it back in in a future release.
    
    Link: https://lore.kernel.org/r/87ilz1pwaq.fsf@wylie.me.uk
    Reported-by: "Alan J. Wylie" <alan@wylie.me.uk>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Frederic Weisbecker <frederic@kernel.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Sasha Levin <sashal@kernel.org>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 541e757944aa63af6409a3bb7d483fd5cf9fcab3
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Thu Sep 16 10:17:11 2021 +0200

    Revert "block: nbd: add sanity check for first_minor"
    
    This reverts commit b3fa499d72a0db612f12645265a36751955c0037 which is
    commit b1a811633f7321cf1ae2bb76a66805b7720e44c9 upstream.
    
    The backport of this is reported to be causing some problems, so revert
    this for now until they are worked out.
    
    Link: https://lore.kernel.org/r/CACPK8XfUWoOHr-0RwRoYoskia4fbAbZ7DYf5wWBnv6qUnGq18w@mail.gmail.com
    Reported-by: Joel Stanley <joel@jms.id.au>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Pavel Skripkin <paskripkin@gmail.com>
    Cc: Jens Axboe <axboe@kernel.dk>
    Cc: Sasha Levin <sashal@kernel.org>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 5f3ecbf4d586b0e78c6399c4f6bbd9dcd4fd6fa0
Author: Sasha Levin <sashal@kernel.org>
Date:   Wed Sep 15 21:16:27 2021 -0400

    Revert "Bluetooth: Move shutdown callback before flushing tx and rx queue"
    
    This reverts commit abbcd61d091f69ec98013dc0ae9c992e152fc303.
    
    Botched backport, dropping to reword for next release.
    
    Reported-by: Guenter Roeck <linux@roeck-us.net>
    Signed-off-by: Sasha Levin <sashal@kernel.org>


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 02:49:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 02:49:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188850.338193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3wl-0001C7-RW; Fri, 17 Sep 2021 02:49:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188850.338193; Fri, 17 Sep 2021 02:49:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR3wl-0001C0-Nz; Fri, 17 Sep 2021 02:49:51 +0000
Received: by outflank-mailman (input) for mailman id 188850;
 Fri, 17 Sep 2021 02:49:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4o4g=OH=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mR3wk-0001Bu-VG
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 02:49:51 +0000
Received: from FRA01-PR2-obe.outbound.protection.outlook.com (unknown
 [40.107.12.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ec202d28-1761-11ec-b663-12813bfff9fa;
 Fri, 17 Sep 2021 02:49:48 +0000 (UTC)
Received: from DB3PR06CA0025.eurprd06.prod.outlook.com (2603:10a6:8:1::38) by
 PR2PR08MB5226.eurprd08.prod.outlook.com (2603:10a6:101:24::21) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.19; Fri, 17 Sep 2021 02:49:46 +0000
Received: from DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8:1:cafe::4f) by DB3PR06CA0025.outlook.office365.com
 (2603:10a6:8:1::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 02:49:46 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT062.mail.protection.outlook.com (10.152.20.197) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 02:49:46 +0000
Received: ("Tessian outbound e27daf245730:v103");
 Fri, 17 Sep 2021 02:49:46 +0000
Received: from bb74ca36c858.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B7142E55-3151-4BC2-B567-366281D58F74.1; 
 Fri, 17 Sep 2021 02:49:40 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bb74ca36c858.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 02:49:40 +0000
Received: from AS8PR04CA0207.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::32)
 by VE1PR08MB4928.eurprd08.prod.outlook.com (2603:10a6:802:aa::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 17 Sep
 2021 02:49:28 +0000
Received: from AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2f3:cafe::53) by AS8PR04CA0207.outlook.office365.com
 (2603:10a6:20b:2f3::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 02:49:28 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT038.mail.protection.outlook.com (10.152.17.118) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 02:49:28 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Fri, 17 Sep
 2021 02:49:35 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Fri, 17 Sep 2021 02:49:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec202d28-1761-11ec-b663-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Hy7+zIJZBtT6U8kppnGcIZcA+xQgczSoUdZnpK5RE5E=;
 b=DBuCp6AWIDbOmdHIIFWGSrx7vS6STeIie4vmX7YPH0sO/FMWGV28w1No8FEwdEmNfkFiHXRLMCux5J8xuOaTfUhcg0rdG5ZN4C6siZBUR+zwnnsjPyfB4eXB2yZaPBo55PLVezkkfFXHe+bYuH/W8+1pk2ynMz4GbV7Ikf+HF78=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: dc8ad1cd1d130b88
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MTOVuDGpuVMxJQchmT8PoHR1TpjAb3COK66Zqtdybsptzyv0Rm1tTUv5cE1a5WJdLmIK3Og98gnSM1AkhAXXgtcQD5lVq6B6JZ1vu/FfTCQ3fVF4utSyQESAwgetD0WjGyC5BSHoM27GznV498Z+ZAXXUWwwo2eQsYhCFY2KKeS3xTKQVumwvFCGBOic9t5Dc0MjIjeL+ap7uZG91YbxhYDCfTLlRVkOLuV++xlUevRs8bsQgx7WwawcbG0aqz/1hwtuawG0htmVeDyzMPNAo2KDGimL2ybqmDRgd1tqjJEwhRv1eNaqzUsxjo3FP6Cj+xgdAnrnEJ6ctd4upN48Sw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Hy7+zIJZBtT6U8kppnGcIZcA+xQgczSoUdZnpK5RE5E=;
 b=KCa3A+zyRWOKifHEdSk2t6cte3L9AN2fxZnV1pIVDAScRROiHsPNOMnFWwDg2lrGEP87DSED9e36mKTGGyL0VoY/jz2l36FthTBZLfnCWbLHVApnIEjv+Qigie5BKPKvYUa+pOS3G75eAsthWvDI9cXYBML4xcawcaVV2hkD8zk57cnBI5w0LnNcplgtieT9oaUDIRP1equrOxPYPTzoBih4UAXBkDCZ/sVZIMlVP/CNjATNhjGZC9eml02Z3l448W/QhQWH0lzkAJrtEKR+RW7by61OCnHBy5xW33Cv8SW5+yRNYZJkgiKy6tgLU3mf4cQ2xcfJyL4sp3jO+aRkMw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Hy7+zIJZBtT6U8kppnGcIZcA+xQgczSoUdZnpK5RE5E=;
 b=DBuCp6AWIDbOmdHIIFWGSrx7vS6STeIie4vmX7YPH0sO/FMWGV28w1No8FEwdEmNfkFiHXRLMCux5J8xuOaTfUhcg0rdG5ZN4C6siZBUR+zwnnsjPyfB4eXB2yZaPBo55PLVezkkfFXHe+bYuH/W8+1pk2ynMz4GbV7Ikf+HF78=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <jbeulich@suse.com>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH] xen: fix broken tainted value in mark_page_free
Date: Fri, 17 Sep 2021 02:48:55 +0000
Message-ID: <20210917024855.605551-1-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d9ebe49d-9ea1-40c2-31ef-08d97985cf64
X-MS-TrafficTypeDiagnostic: VE1PR08MB4928:|PR2PR08MB5226:
X-Microsoft-Antispam-PRVS:
	<PR2PR08MB522651EF8D6B814E0DBC4432F7DD9@PR2PR08MB5226.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2089;OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 MRtkX/UL+tAReUShFDYnckk0Zp8qRbzhbFF23rAkYLSMBrmtvuLbGkn0gIxprP3qJRv6MjPXHYM1Q4oFNr5oPxwVRG0dNa4M8dDdM3J9pZp2ZkVorTGy7j+1fUyjZqPcDAy14e4UHKch9u1YlouVJEl8KGQxC+Zmbz19jaDtr4DAN2vf9ZSuEpdB8UXsYdL8P7fuPK08kplASDtycPp7UXy/HgWpUQIHlHxnAQb2zGrlmJD0Qnadnv1sP4ThDWbomzWpOxYWMcClH8NAjisVr+ePX+SVSBIw9HDqyn+fY+ceciq0JkcXSQ9AC8BJaU9Ggsf1GP4DgzaYuSkRk463u6MKGKtOdrWgWi/JPAfUL8+82QiSNNEHQHiViel1wGa0cDQ99IAnkasqi9KGkj89nwCpHyL+A8ZLwwJ8f/dXNb4GEFAgFpPnYPnQ3jrK4D5gBz0Ui+U4TdV6UucVxAmrTBtgwFKKK7BPckKyezSEe5ddqlNp8jrqAhlr5IJVBdF4jIjyJLhau7dMGfPuyqbny72dC747PTzLrHYjqXrBY32LkNWCtwHsuKVUJIEQ+Nddwh1FNpd8RaaRypBQqi2dbrAz5eukzDLPwYoBORVOMzQuHeNmj4vTFTCbVmpooVJnlQZ11Uyv6xbZLImr7KUMzmUjFf7i8MSqAcP3fdqw4xRwp7ggcxZJkn2SUtqqZNAbno7m1tTHphnTpA1r1epXa/bLmpr94pl4qCIXjlSXmkU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(81166007)(186003)(26005)(336012)(2616005)(2906002)(86362001)(70586007)(4326008)(316002)(426003)(47076005)(82310400003)(508600001)(36860700001)(8936002)(70206006)(6666004)(7696005)(54906003)(5660300002)(110136005)(44832011)(356005)(1076003)(8676002)(83380400001)(36756003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4928
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	887364c2-504b-4a4c-04f8-08d97985c45a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cuH1guZvHFLSb9m93lcLdaxtSK4d+M/ZDkhcKd3pRUdthKCXbYk//SEq29HP3x8plwtxxjxkNY5kpPEp3cc3jFuUgPTlqHzv8XmaSMPE2qfteSGoLGlq5yfG23UUpNJ9zRX3vdMadRgG1uNcWuw1/Sl3zR3VTrzP6Ms9bOqkVMCVZAkctZeQIoy+Z9NivNa1yNRl+HlfvtEEBIJ+GCe6dCe9S+OIIJ58JOCf30xdztUh6pKJhRkF2vB8MbXO6SvdwSx/FIC6Gp25PWystgp5MOJS+TcPRVUxKjyCgCgCuFUgbVCqDpNWu74rTzdTPHazgkwLA+mhiC0npE+s3PXJADyA9+1WDJ4Bnky5Hkeb1OWsimL9htvFTf535IVEHWw3RL5VLq9ZSLSYTUAO9H9N3Yi0MOs6JcILl3M9Dnl024w3ZlG8aM4XBi9Cj1Ksq5oT8RbAtm0fRqNjWgjuuqxf1XCUuHUfmptidSnC1SIl9zOR+tYJ75rORjU3M1Pjf7LQTVCn3WMGHgb6FDFkDQGNed3eQI7eWtt+mDl8YqxEwdoOcjwNUK4I8jIV03mneGC7LmgIt6Ahy4cnX+zhVkXOou1l2qIbrxDcWdOvammdZmRXFEV4K+Yd8NXFcJ2Hkx338CZlN+oes1SFM/be5uw8cWN2Jf96rzXE/VLZkDP0ijaW8RW5/WLGVjwrnuppqtDF2YNlchhPGxNSRdJ5aHS2YA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(376002)(396003)(136003)(39860400002)(46966006)(36840700001)(36860700001)(82310400003)(8936002)(336012)(70206006)(54906003)(70586007)(86362001)(1076003)(110136005)(478600001)(316002)(4326008)(26005)(5660300002)(36756003)(44832011)(47076005)(81166007)(7696005)(6666004)(186003)(2616005)(82740400003)(426003)(2906002)(8676002)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 02:49:46.6627
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d9ebe49d-9ea1-40c2-31ef-08d97985cf64
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB5226

Commit 540a637c3410780b519fc055f432afe271f642f8 defines a new
helper mark_page_free to extract common codes, but it broke the
local variable "tainted", revealed by Coverity ID 1491872.

This patch let mark_page_free() return boolean value of variable
"tainted" and rename local variable "tainted" to "pg_tainted"
to tell the difference from the global variable of the same name.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/common/page_alloc.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index b9441cb06f..c6f48f7a97 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1380,8 +1380,10 @@ bool scrub_free_pages(void)
     return node_to_scrub(false) != NUMA_NO_NODE;
 }
 
-static void mark_page_free(struct page_info *pg, mfn_t mfn)
+static bool mark_page_free(struct page_info *pg, mfn_t mfn)
 {
+    unsigned int pg_tainted = 0;
+
     ASSERT(mfn_x(mfn) == mfn_x(page_to_mfn(pg)));
 
     /*
@@ -1405,7 +1407,7 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
     case PGC_state_offlining:
         pg->count_info = (pg->count_info & PGC_broken) |
                          PGC_state_offlined;
-        tainted = 1;
+        pg_tainted = 1;
         break;
 
     default:
@@ -1425,6 +1427,8 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
     /* This page is not a guest frame any more. */
     page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
     set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
+
+    return pg_tainted;
 }
 
 /* Free 2^@order set of pages. */
@@ -1433,7 +1437,7 @@ static void free_heap_pages(
 {
     unsigned long mask;
     mfn_t mfn = page_to_mfn(pg);
-    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), tainted = 0;
+    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), pg_tainted = 0;
     unsigned int zone = page_to_zone(pg);
 
     ASSERT(order <= MAX_ORDER);
@@ -1443,7 +1447,8 @@ static void free_heap_pages(
 
     for ( i = 0; i < (1 << order); i++ )
     {
-        mark_page_free(&pg[i], mfn_add(mfn, i));
+        if ( mark_page_free(&pg[i], mfn_add(mfn, i)) )
+            pg_tainted = 1;
 
         if ( need_scrub )
         {
@@ -1517,7 +1522,7 @@ static void free_heap_pages(
 
     page_list_add_scrub(pg, node, zone, order, pg->u.free.first_dirty);
 
-    if ( tainted )
+    if ( pg_tainted )
         reserve_offlined_page(pg);
 
     spin_unlock(&heap_lock);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 03:05:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 03:05:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188861.338207 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR4Bj-0003it-DI; Fri, 17 Sep 2021 03:05:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188861.338207; Fri, 17 Sep 2021 03:05:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR4Bj-0003im-AC; Fri, 17 Sep 2021 03:05:19 +0000
Received: by outflank-mailman (input) for mailman id 188861;
 Fri, 17 Sep 2021 03:05:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR4Bi-0003ic-4d; Fri, 17 Sep 2021 03:05:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR4Bi-0003eY-1L; Fri, 17 Sep 2021 03:05:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mR4Bh-0000Ai-Ps; Fri, 17 Sep 2021 03:05:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mR4Bh-0001cw-PO; Fri, 17 Sep 2021 03:05:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=f50pcCM0wor/woavixJfK4MkaTJqTJZgCPhYXFCj/fc=; b=UH/j4Ac7pAPRx9DpJLWwaIDIfO
	h5w8WB3gSxsgNmyXzUnB5Rz6SUl12FqRX8nCBkHsNLulQEZLqlmsDOGc9mm1biTyVrNsgeSewid0y
	OFK0mZjJvQ+yQnn+sU8fr7t6BRhxrIUN3IX4ZSj5ZcZ07jteYH6lXUHt4LXAOQK3E75U=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165018-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165018: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=98f176b62b6136f696da88032f9c12a244ac9aed
X-Osstest-Versions-That:
    xen=9d92140d0255e14d8c3eaba7edc931327eb96fd0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 03:05:17 +0000

flight 165018 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165018/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  98f176b62b6136f696da88032f9c12a244ac9aed
baseline version:
 xen                  9d92140d0255e14d8c3eaba7edc931327eb96fd0

Last test of basis   165009  2021-09-16 10:01:36 Z    0 days
Testing same since   165018  2021-09-16 22:01:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Bertrand Marquis <bertrand.marquis@arm.com>
  Julien Grall <jgrall@amazon.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9d92140d02..98f176b62b  98f176b62b6136f696da88032f9c12a244ac9aed -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:13:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:13:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188900.338310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR77v-0008FM-7u; Fri, 17 Sep 2021 06:13:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188900.338310; Fri, 17 Sep 2021 06:13:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR77v-0008FF-3Y; Fri, 17 Sep 2021 06:13:35 +0000
Received: by outflank-mailman (input) for mailman id 188900;
 Fri, 17 Sep 2021 06:13:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KSMa=OH=epam.com=prvs=9894170f70=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mR77t-0008Ef-Sg
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:13:34 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 61aa9293-177e-11ec-b66d-12813bfff9fa;
 Fri, 17 Sep 2021 06:13:32 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18H5xRrr024299; 
 Fri, 17 Sep 2021 06:13:28 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2055.outbound.protection.outlook.com [104.47.8.55])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b4eyhgkhw-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 17 Sep 2021 06:13:28 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7298.eurprd03.prod.outlook.com (2603:10a6:20b:26c::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 06:13:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 06:13:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61aa9293-177e-11ec-b66d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mfW8aEVabUB9BOQe0+rLofZBWk3gfegJbqs89l72WkixPQCdGptuo7MGYMV65M/8EUZgRZ1CooX3pbLsrdJA0+EAi8pGBiDbSmal0SuXQwcoNTcfhC9fBYDhw5MeWpvFTIXD3n/dyKRVJmpcT4NtNuUMb7IxHl7/azl2hGOgXMxGshJwNYiJMMSINndoG0BjUap34M4tEzO98B9ITQnln3hYHOldJtG4vw77QBXBdSXac/AKJ2ZJ9GUOeo12m0AT5xiBVP8SVnMMf6KTiDGE3iRWxUPz12m9XTe2tJFDufuDzo35SgS6+HtbTWkrkXbwrq9Nzzk5bWL1gHNmFpMj3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=o8wErlSaar3+Ph3dkvw6ArHP5WXhVZsLJbe9lTTMHLw=;
 b=eSztGXaLNov/m9SxtXsWgtrScwSaHEDeam4k0uSU0kFKZVwurKvadShx29nH0eODpK6xI2alQR5n+SyxD51/+oASJIkKzFhvll/vIWmZGsXJmsJgbseeGGVa5SgeLLwDXCOgkDiHMU/wTWl8eVVIFNfMY+9ju3R8B0laFj5Q64fAjh3Q3xNvzcDQMvDv4IMeHK2H89z6bsvq5Pq8UKepsv8MONOUeEGRgs0r9nEaZOSPLGZnuMh9dhNKe/PxZ8LnlSJp8ekMcs9cPolYsf2mukS1Bnxden8VN7Lu4OEc8mlXf0mxKZoVOZ0vb+zHHrTrTvSIllYSE2zM2sL7zfAM6A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=o8wErlSaar3+Ph3dkvw6ArHP5WXhVZsLJbe9lTTMHLw=;
 b=fY3ZjVxZaFHW+sMbugcyuddajZFuG7jOGJaZtVPiFDoGIU5cj7hGHWrCSxDdTN9h5AAnbC57kOMpbNQpkT4R7TXtxYmTWrpe2S1QmvE87Hyhc3HkAP18iow4SNm5mfrNFxdcdy17mC4hBMZ05UyI+nfAohLUJWoVZTVXe2AeyycEs/Pw/2Gwer42KVgg6OV6HXNEj2OKgA9wFWyWa+t/PVHoWLcm78TzLw97ZmrgUUraA9GK/C/o8xL//ZcbxDKmSRdY1dBB6A0MlD9kZJXzIEvRc3lhevpMj3huKm5lBiWPPFO60UuziWc2qQmOSaJHsXPTLFjbvsP9lzQ/hwolmA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: 
 AQHXoJ6eIuPEzuAZLUmx1GA90MGEBKudvyQAgAXn/ACAAP1GAIAAWBmAgACkJoCAAjRYgA==
Date: Fri, 17 Sep 2021 06:13:22 +0000
Message-ID: <e413eea9-b6ae-9cf8-66a6-f02fdcb645bd@epam.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
 <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
 <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
 <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
 <alpine.DEB.2.21.2109151321030.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109151321030.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4f148901-3a18-48a1-d99c-08d979a240b0
x-ms-traffictypediagnostic: AM9PR03MB7298:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB729805D9315E859BEC66E530E7DD9@AM9PR03MB7298.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 EK87Ldy5OguzKIazwVLqLGgeQt81OYM9Kq+BBmxk6lGTkCy1ffvlQSmgioi9Lx9FoVL2dr8hqdmVebgxWhLURnW/wp+F25px79PoFgPW/loFApBwfG60hXqb6WKIBA/jLHsNw9fcF4NV1SX6QURv0wnTP9weJCQsBec9VVS4HAKzcajFrPQFzqaWUYaQbq7rriwc6+bDXBb7jty4cD+tILwOlYe7q2n+mvNAZ+QVXXMEBFHwStSJdalLDXBX+vt11qs2S6RqyM2Uy/DzeHFV4UVdrWguuuGmJkMq6VvPJs4zb3rrbsEAaHa3oWAQ9tLyIDOGxe3qtNscT3NdByuGqBFR5ddlLOE/xjsZY0tZwFDxY5rgxPYc8sV6gtZuQbiR8KfZhzCikBqdCZ/XWmwqaUWDpJ3AQG+XdsVi4wTkJ61O7NPRgFyzlamNNKGH+T/I6aszvipWnYpCuNBgo+0AiNU5d9xu8e0st3LeRebwUaVLteizrY5wZpY+yOlxH9Y+bydnAVPMaCkTo+aLaZYiCfNM82r5uEPbVWffVWrTEa9htpD+2Js0PKYcFNv85MHkuATSfW6zJY5oZcVjkS2h5IRiEoFpr0CZOyIaK6WMuR8NE9xPfS7Jvz0ek0RoZQ+14V0Nxq7CrC6gcLhgXEkkg6SDDnVbvkSVlt3L7Vc6ugkyHVezGY1GeaEvPWp18pE6wzlPqAEIjvHTayOecstk+YycIADThvASMEGajX2iT3JMs258flEr6ArnyMdTx3louyJifABBIivldNhd5yBrF6lLjWMydmUTz0X5YdNj/36WcAOb5hrW9pvrwUuSeexh/XjH6GFNytnlDc9XOwgKsg==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(5660300002)(86362001)(186003)(31696002)(91956017)(76116006)(36756003)(26005)(6916009)(8936002)(316002)(6512007)(6486002)(64756008)(83380400001)(66446008)(53546011)(31686004)(2616005)(54906003)(2906002)(71200400001)(38100700002)(966005)(38070700005)(8676002)(508600001)(122000001)(4326008)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?Windows-1252?Q?TSwWIxHEbUz3F6eRYjA7vVVf83sxLMIVCsVwGJqnPsCTPbZmu6FyFb68?=
 =?Windows-1252?Q?qY85xRQHwJ4xbncnYFJPpwhqeQdoG/ySK44CeDztKKMOIshiEysje633?=
 =?Windows-1252?Q?vtu6rRwwU7y3iXLJZdOygQD77Q08Q26t5VApSL+FMkiUAHSVq2LXDara?=
 =?Windows-1252?Q?YqF6/ivYPPkxmAiTArC/++/PzA4OavQ5EKyQr/QbsW6HMMKg/DTeQs9I?=
 =?Windows-1252?Q?Ci7XLrjSUd0vd4XPuKFoY+Q1DfgoS7sMC2jeWNQFRNdc7v+RadXrOnBi?=
 =?Windows-1252?Q?Pfo4kN5RUFmK1SIZFXVCAWXshHWukQOfR1fKQpK0AOyTsn0IybVyjsMI?=
 =?Windows-1252?Q?xlyg62ufuTPQUgcoUSqhAqXZpKJaJQXhHFpmHyAzymhR8dN5P7NOun0n?=
 =?Windows-1252?Q?I4MIIw5PVFLrnEtNyMYa5/4JUyTGdB7iQbubYw6Y5pclpJ1mbEwcbXT4?=
 =?Windows-1252?Q?WcO4H63hsKtpbnKvr9Q21WoQuJ/Wxmm/CbRNpXimBY0KPo4jB332Xh+x?=
 =?Windows-1252?Q?jNU79MMdmcVzT5H2etaP/D35zigLDjkDsvnBzV7oBcz8mSlQ64QUP1aK?=
 =?Windows-1252?Q?S1Izsg2tztVAthNMhHzcgWWoRib/leTvwxlacAIRFRi5ZSLBdNZYcuLS?=
 =?Windows-1252?Q?of8r+JzGZWQ1m3US5CGtW5j4xtNOK1IGcHzpjOijudB8VzL3MS0o05nq?=
 =?Windows-1252?Q?1f/25M9oibjqboj4uCxerxx1/FwbxnO4JPLLCv5yQFSoyE/adjV0Xrtp?=
 =?Windows-1252?Q?9MlnMpy/nhGG3/3FKYVzvQ5o5rabdYEE6aqhitIOZ2fwRjqHK+/0DV7G?=
 =?Windows-1252?Q?gaOWeQ6lyBDMgQNFZTIZ75iGlWOAOq/nB3VaFBD4uQqL2UxAfp1EWWNq?=
 =?Windows-1252?Q?Y2uBDfyt3poUQTXSQ6y8z4XdYkdCEqPwx7qZ9MSFyemWRNj++WEykAoi?=
 =?Windows-1252?Q?jnufIYW4XrHxqjuDQTTdzmOiTjA+0FFLtn11o60/bSO7cWhi8FGAn/L+?=
 =?Windows-1252?Q?4gq7G+537x2qDnf1coQfGtKKUiZU3Zs1QE0qihWx6Srk8LcG5XoJsTZ+?=
 =?Windows-1252?Q?4Q0Qu6UqFVwK8goqraIZIX7bhYd96ZCl/IU8vldApbvsf1kbk2Mg2vvs?=
 =?Windows-1252?Q?pjt9Vskt73Tn8EgGnSThRZm9d14wql/qMJIeelt9Ml0v7Ll/2TExyvlU?=
 =?Windows-1252?Q?bR7eNLqwHkwp6GJScLEJ8TBVkLm8IQxVKuC3y6St48E9p8Iz5j5cnn1a?=
 =?Windows-1252?Q?3NsEVESbl2LkoeSuQEO1jUaPbHWQOSr79zd/jNRR77IgeX4p/dMrQDjG?=
 =?Windows-1252?Q?R8qf/t6A7Yl4DIIpZUAM9WzRDTXV2OomSWkAPKX0a8ttOrg/i5z4M3l2?=
 =?Windows-1252?Q?7WV4R/D99yBkTZjtkiNOXZ4Ejbqr+uglB0HQxR61QrZkGuLpLFEEu1pU?=
Content-Type: text/plain; charset="Windows-1252"
Content-ID: <2B44B73081C7394D82DBF62321EC10BB@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f148901-3a18-48a1-d99c-08d979a240b0
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Sep 2021 06:13:22.4947
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yiHqLErbKiboUH8Pm7uI3/ZaNWEhffttHJQ1p09tWYXMqKQeKG8W5gz4hXIm5dzRAdkFnk6bH/Xrp7baqpw6+A1XaZB79rGumHNlTZoI0IVBh8DKoUgwgQoLl7MXmHv7
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7298
X-Proofpoint-ORIG-GUID: Sj2uQbxknBPftXChShWHbC65H4gwLHd2
X-Proofpoint-GUID: Sj2uQbxknBPftXChShWHbC65H4gwLHd2
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-17_03,2021-09-16_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 mlxlogscore=999 mlxscore=0 phishscore=0 lowpriorityscore=0 bulkscore=0
 clxscore=1015 spamscore=0 suspectscore=0 impostorscore=0 adultscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109170038

Hi, Rahul!

On 15.09.21 23:33, Stefano Stabellini wrote:
> On Wed, 15 Sep 2021, Rahul Singh wrote:
>> Hi Oleksandr, Stefano,
>>
>>> On 15 Sep 2021, at 6:30 am, Oleksandr Andrushchenko <Oleksandr_Andrushc=
henko@epam.com> wrote:
>>>
>>> Hi, Rahul!
>>>
>>> On 14.09.21 17:24, Oleksandr Andrushchenko wrote:
>>>> }
>>>>>>    +static int pci_ecam_register_mmio_handler(struct domain *d,
>>>>>> +                                          struct pci_host_bridge *b=
ridge,
>>>>>> +                                          const struct mmio_handler=
_ops *ops)
>>>>>> +{
>>>>>> +    struct pci_config_window *cfg =3D bridge->sysdata;
>>>>>> +
>>>>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
>>>>>> +    return 0;
>>>>>> +}
>>>>> Given that struct pci_config_window is generic (it is not specific to
>>>>> one bridge), I wonder if we even need the .register_mmio_handler
>>>>> callback here.
>>>>>
>>>>> In fact,pci_host_bridge->sysdata doesn't even need to be a void*, it
>>>>> could be a struct pci_config_window*, right?
>>>> Rahul, this actually may change your series.
>>>>
>>>> Do you think we can do that?
>>>>
>>> This is the only change requested that left unanswered by now.
>>>
>>> Will it be possible that you change the API accordingly, so I can
>>>
>>> implement as Stefano suggests?
>> We need pci_host_bridge->sysdata as void* in case we need to implement t=
he new non-ecam PCI controller in XEN.
>> Please have a look once in Linux code[1] how bridge->sysdata will be use=
d. struct pci_config_window is used only for
>> ecam supported host controller. Different PCI host controller will have =
different PCI interface to access the PCI controller.
>>
>> [1] https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/ker=
nel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-rcar-host.c*n30=
9__;Iw!!GF_29dbcQIUBPA!kjkv6KIlvXOEgVaS6BNPRo0gyABihhO0XmNHRPFJaFAGhhTEuK1m=
IsWqPs0cXEipzkT_MmA-Eg$ [git[.]kernel[.]org]
>>
>> I think we need bridge->sysdata in future to implement the new PCI contr=
oller.
> In my opinion the pci_config_window is too important of an information
> to be left inside an opaque pointer, especially when the info under
> pci_config_window is both critical and vendor-neutral.
>
> My preference would be something like this:
>
>
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 9c28a4bdc4..c80d846da3 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -55,7 +55,6 @@ struct pci_config_window {
>       uint8_t         busn_start;
>       uint8_t         busn_end;
>       void __iomem    *win;
> -    const struct    pci_ecam_ops *ops;
>   };
>  =20
>   /*
> @@ -68,7 +67,8 @@ struct pci_host_bridge {
>       uint16_t segment;                /* Segment number */
>       u8 bus_start;                    /* Bus start of this bridge. */
>       u8 bus_end;                      /* Bus end of this bridge. */
> -    void *sysdata;                   /* Pointer to the config space wind=
ow*/
> +    struct pci_config_window* cfg;   /* Pointer to the bridge config win=
dow */
> +    void *sysdata;                   /* Pointer to bridge private data *=
/
>       const struct pci_ops *ops;
>   };
>
>
> As a reference the attached patch builds. However, I had to remove const
> where struct pci_ecam_ops *ops is used.

I'd like to know which route we go with this as this is now the last

thing which stops me from sending v2 of this series.

Thank you,

Oleksandr


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:13:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:13:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188898.338299 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR77f-0007tq-Uw; Fri, 17 Sep 2021 06:13:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188898.338299; Fri, 17 Sep 2021 06:13:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR77f-0007tj-Rs; Fri, 17 Sep 2021 06:13:19 +0000
Received: by outflank-mailman (input) for mailman id 188898;
 Fri, 17 Sep 2021 06:13:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR77e-0007td-O1
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:13:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 585ab23a-177e-11ec-b66d-12813bfff9fa;
 Fri, 17 Sep 2021 06:13:16 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2057.outbound.protection.outlook.com [104.47.0.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-DphBANeBOFqbfeylBos-Ig-1; Fri, 17 Sep 2021 08:13:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 06:13:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:13:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0038.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 06:13:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 585ab23a-177e-11ec-b66d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631859195;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=E0C70BnHkf+Xjjy8puyk2DxDQEN3l79sxqdB6zmJekE=;
	b=JmJC1B19LS4GKHjFNuyGeRG8pEtGKCuuzIcThxRszeZlpllSOTxUUe4RzpiKK3cHLxXuq/
	gAWcG010ZWCP/iL9cHzSjUGSwm1ihOHr0lyQa1QieBysC7v5QV4XKUQ3yTGL6+3O9M5YtP
	8OnMlTWVd92xRkhsPSgIWewl5F1ss4g=
X-MC-Unique: DphBANeBOFqbfeylBos-Ig-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fXhkYtSdvVa4wdvtljJxcynR9dYQupXOYSXT0Bm0PPrOssm3I2WzkX0vJXUrR6Es0cDBP4xIa1IQ+Ag1C5eoOLv68MbJKnd0SRLl21W9XjryTYKxUKJOCDhnEJdv5n73UaXjHLZx6XH0wWgAMv51tOqZm5si9SzmqTMjQEgtLHa3F3OL56Zo+XXsfAYVjGSvrOTHo0RGJSSNOP9G5Yi3GpqD9aaPnuVEiVirc8wCHlHqwwAyMx+tcGIUfhEV6DJaX/ssqIOhBggjiFZxa3w2f66XmWYqcvGLcwqPFaLO3FL/bVCKmkzmG9Ml8D5hPjP5lLFlKrB8gTl1Zj5CujU0Qw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=E0C70BnHkf+Xjjy8puyk2DxDQEN3l79sxqdB6zmJekE=;
 b=M4SPYxDp6Zv9q4EjccpFIm1vr6twNv2RfCnmptwY3uzI1mBlJBl8OjMR7a+Eg113PjmtrCl9TcmyqK3RahDBbyzEp77CeXwe9d9UzhNnHYaMcT6iEhQuI4QgKQIquYIuAF0JtW9dpGPhS0Uly/q+Apg5ymzURSq8bSkjl/wC2Y4KSlLmzkIw4dIzOfeFJDEHwKyGGITbuJ5KrMyc7BdktzYcNTimWBtZKm7j9RgogjxiTB0VVjuj0TNnbHI4WKCMCh0GdWMN9CkmitqPqHkLnN5weI6y73ut4cIOP8IIpwoB9GfbFMfGUrnaNZGkeNfwP4BRyKAPqrHHXdBMgSKszA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] Xen/gntdev: don't ignore kernel unmapping error
Message-ID: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
Date: Fri, 17 Sep 2021 08:13:08 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0038.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 714bbcbd-c616-4fa3-49e1-08d979a2396f
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383156B10C9FAF1D21EDCB0B3DD9@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7x/6Tbo6yHVA/UJL7sQ/DuSLRGvDWU4w6O0UPchq73AiHCJpGTUPNzxBNcSh5bBzbP3nwGV5NRx95QpJDYtKm/EoEz54XtfeJGTiL+ZNJHsFUi60do0gQzCqWT8/rVI1gCqPCZzZ1yXG/AHI/C9Ey8FA/RIXajBjfWULZZ2HETqRemw2z9wdF/3GoZWrGqbkPEmS1NF43N4xOdmLBsV2M6hKKCPvam432y1rupa6QseviGyLrMxtbcjHYyqGO2TqmHoskVgeFdMhJ78zzYBlhJ4KN8+VH/Sr9uIaBeaOOMqqO5i2vpPh6s/UQTvMI0RBMJ6qx6lYenGWDM+5suJ73LJiVhITWMgwmNc3zoC0kc2Z0KKK0xCu0AZ9u79lfVpdyKfmzHd9yzI19zuAzWTH2TrPeisK53WN9tCqZuY5f535I81WB0P7O+ueTOoCHoxlZqjDCGyyY3qQaECXudAKYHk+WzVtQZZCkgucFQ4pk7goKnEIZ6/0mx4/iBRATSAMU1KQlj6zub4l4aWMFu3TeowJpGMnDcepspWzDTb8V4ZS39FDiDLRgvC7yCgmpU94j1R8rZoS4B8r7qJOsft5RN0xJpA1pVc1q2t+X+C+S31kzaImYGccDQFvi9WfazP2n71NscM7S8LTTV6ZPx/UtBbXvI1OSIKxShcw7BrLDodumrujHyxXDkATq8gWtuOkpknMhuizRlxGrZjmJSMaeDvaQvidnulSafxbBu0YldI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(366004)(39860400002)(136003)(396003)(38100700002)(16576012)(6486002)(54906003)(8676002)(31696002)(66476007)(66556008)(66946007)(2906002)(5660300002)(478600001)(8936002)(26005)(186003)(83380400001)(2616005)(4326008)(316002)(110136005)(956004)(36756003)(86362001)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ck1XY05kbmtIUzZsc2xrZHdBMlVBV0hRNEdPQkhoVW05R25kN0R2Qm5WdG1t?=
 =?utf-8?B?TU0vY2JPRldXRTZPWTVvWlZtNFBlcWZ5bXpjWVNzZGpGTXEwazh6MG0yWGZa?=
 =?utf-8?B?eEIxdGlNcWxkd3ovYTZreVkrVFBiR1BCa1c5ckNVYThVcUViRHprTjh1WFQv?=
 =?utf-8?B?MVRDQ2hhd3JiTlp5dTFUenB5b0FsZTdjY3NXUEdFeGtaZEF3WGVHNlBaNHoy?=
 =?utf-8?B?L1FRVXpLUzM5YnRTdzVlVzV1dnpkSXI5Q0JwSlpLdWI1N3Z2TXJwdHh3Mnl2?=
 =?utf-8?B?TGxIOWV2MnloY0NSWVBsVDBBd2VTMnNVMFRGQ2IrTkZDNzhMMHNRUVVOYlVt?=
 =?utf-8?B?ZlI5S1FGdnByM1VjS3ZWREtmZW5ES05kUUdjWVZ1eE1VR005SFV6ckNISlpa?=
 =?utf-8?B?VFk5M1lvRzFWblV5cktHbjdSOFVhbzVvd0M2ZElTSDkwUENMTS9waHZBSnc4?=
 =?utf-8?B?MENSaGtvK0dRUTAyMWNsTHlvcjlYMU9RRFhJbWNpcUdUekNRc05mdzFaUHBU?=
 =?utf-8?B?a2xndTlxNXZhOWdjOHBWL3RDRng4ek5XVmZyb2w1OXR0RHcxSlBRNGlDZGlh?=
 =?utf-8?B?MUZLSG40Z2o3KzFyMTVJWnZPNjdRMCtJS1BWTlVzUmNZdlgydndRb3hCSWEv?=
 =?utf-8?B?bjFCekE3MmtLL1R0SzB0UVdXZXlaRElQbGk3dnFNdmhvdHBZaUdKZC9nOUZB?=
 =?utf-8?B?b1Q2UlY3Q3pUN1RaUGlycjVPUVZ0Q0hXSDVad0JULzBldGZOSWhMUjhBL2V3?=
 =?utf-8?B?SE84WkRiWU1mdlV3MFVKMlRiN0pGQ0dzNVVBcHFwTWtNUWQrVzV6d3hIUkNP?=
 =?utf-8?B?Tmd2eHA1ODV0R1orMERuUUcwNmg3bmRCalp2alVRMGZrWS8yTGVHWk5kL014?=
 =?utf-8?B?UUNIZjd2dGh1dWpwZ2plc1ZTZWh6NkVXTXhLZGFGMXNTK0pjNEFja0R6ZDF5?=
 =?utf-8?B?UkV3Sk92OFZoWnFZTzVHRzFkaVFDQVdJblRXcXhCVi9VbFU0dmkzSy9tRUQ0?=
 =?utf-8?B?ZlhwQlVjSWlTRGdxS2JUMkRqNHR5WHdsdmxESUhLQTJTN0gzVEQ3V2VhQWFJ?=
 =?utf-8?B?TTFyV1NySXc1RDhXcTFSbEZWazg3Q3ZhQ3FTVHJ3T1N0bithVDQzY2dqOXlR?=
 =?utf-8?B?M2EvWitBc3RUR3BGc3VUazVCR28yNkhyaW5wazNlQTkvTEExcmtWWFhOUk1w?=
 =?utf-8?B?TzJYQmJvTDZ4M2NEcVNoUHFXVm1ERnJDZ2JhSmRqdWtKMUFQU25IKzNNcXg3?=
 =?utf-8?B?cFNsUjdYTUNZNElJQkxmK3BtckNFWDhucjlaRWRZUHBMUGdBUnREeHBKd0tR?=
 =?utf-8?B?Q1RQYlpnUlRTcXpDMFBoYjNoaFFQVG5LOWIvcmE0QmdoLzVvdkhOZDR2d3FQ?=
 =?utf-8?B?dGYyWC92WGl5aDFDYnB3c2lhT1h6d0MzK0EyMlVTa0JIZ3pJSTRYcVZPZlI2?=
 =?utf-8?B?L28yR0F0Q3lzcUFpK1BEc2NBTTRGdE9CK2p1bkU0bXZOY2pZUzl3ZHlTYnhw?=
 =?utf-8?B?Sll1akp6bzZpTmZyY0hXYWptazlXTmFIRmpBUzZYcm4veElDd3VxMEJESGgv?=
 =?utf-8?B?b1lOTEQzRjR0MDljdjEybGk0a2RQN0x6c3dFbjFlelZYRDk3bXhuYWlIYUdS?=
 =?utf-8?B?UVJONnorYmpNeTIxVFFJRi9NUHpSbERhR1BMTStMenAxV01IbklPdFJmVUNl?=
 =?utf-8?B?RGc3RVpadjFKbmE4TUlOZTJMNGhtbUNqQ0tCa3lUQ3RSWDJ1R3JLOWxGWk5V?=
 =?utf-8?Q?I9bIqo8XJ+Yny+LHjPUrdt9ukpIjf6bh+Iomj4v?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 714bbcbd-c616-4fa3-49e1-08d979a2396f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:13:10.6572
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: G+bVECsj+i/F5j9JH5o2MtArGakmdG/IdvAG0m/gHqbZbYZPIIann4GG9yNWFWPUwjuYljFlMvg2wgYJ5n/tcw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

While working on XSA-361 and its follow-ups, I failed to spot another
place where the kernel mapping part of an operation was not treated the
same as the user space part. Detect and propagate errors and add a 2nd
pr_debug().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
It is of course questionable whether zapping handles even upon failure
is the best course of action. Otoh unmapping shouldn't normally fail
unless there's a bug elsewhere (which is how I came to notice this
discrepancy).

--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -396,6 +396,14 @@ static int __unmap_grant_pages(struct gn
 			map->unmap_ops[offset+i].handle,
 			map->unmap_ops[offset+i].status);
 		map->unmap_ops[offset+i].handle = INVALID_GRANT_HANDLE;
+		if (use_ptemod) {
+			if (map->kunmap_ops[offset+i].status)
+				err = -EINVAL;
+			pr_debug("kunmap handle=%u st=%d\n",
+				 map->kunmap_ops[offset+i].handle,
+				 map->kunmap_ops[offset+i].status);
+			map->kunmap_ops[offset+i].handle = INVALID_GRANT_HANDLE;
+		}
 	}
 	return err;
 }



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:17:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:17:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188914.338324 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7BQ-0000zc-1C; Fri, 17 Sep 2021 06:17:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188914.338324; Fri, 17 Sep 2021 06:17:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7BP-0000zV-Tj; Fri, 17 Sep 2021 06:17:11 +0000
Received: by outflank-mailman (input) for mailman id 188914;
 Fri, 17 Sep 2021 06:17:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7BN-0000zP-QZ
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:17:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e2dffef6-177e-11ec-b66d-12813bfff9fa;
 Fri, 17 Sep 2021 06:17:08 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2050.outbound.protection.outlook.com [104.47.13.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-TziojznkO-uiertQfHGRlg-1; Fri, 17 Sep 2021 08:17:06 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4944.eurprd04.prod.outlook.com (2603:10a6:803:60::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 17 Sep
 2021 06:17:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:17:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0036.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Fri, 17 Sep 2021 06:17:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2dffef6-177e-11ec-b66d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631859427;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2jfRulNEr3zGjCFpmvySCoH63JC7fzl7FjmUjzdpIQ8=;
	b=MqaeA98Q4rhzR6VB+Ex86yBEmYOVrKBD3ASetPwRIamTfreQ+UonuzRmSjiFoe1gMSIyos
	RFdWcfDBKQlbxESntH5jFY1XimvYes/S8YvGK+St9nI5x3QFFjqE4c4HkOSJeFmK6UfMuH
	80vnLIdcAZonxXVGxKTBDO00fngdJkU=
X-MC-Unique: TziojznkO-uiertQfHGRlg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZsjoerqQX7xjsY7AZmXcZwLnkHcunE9VKx9J4DZzwwagW/4WJ/E5rq/7U/Znj9g8+Sqw2PdImwTNDQ4su9YxUQ/zh6x4wBnYgjUy9VoSxk9RmJ3usR20GocOzr18UmYovqxogaWc8etRGS2uZhtTIg8ND948pNEqHIHgqdseHhxdHoo6LYY5A3xJs0ll4Un4kQaRylypkOmGP8n1nBuJwm1LwDQ+33OsjD4ceDz9TtL4hdxrnqVE/f9oS77DTDiEnEa9BGmOQXdQHGY3EsToLrz0yfalKV0Dv70HsPj2CpGT7cmbS7SI8kz6peosOv6EFf/2QXAtDACli/9Jn1Ptkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rh/EC3CAU/E8XbmlJ9mJqxNlOX4p+TIrptso6vc3xdI=;
 b=WD93xzn2iL6TbrNzz6YWS4x9VcAoDu5tA2hmfh4JWzUohCQAmAU5GxdsVk0+VPoLKHaJwFZsWfeYiGRBX36I2B/xWFlnrfn6eBbqJJzo/OEvITiSs/7585jbYFcSMmei/nqdg7Z75QoBcKO6H0O41Hupaw3rwySEqEXKoscyT1CON7MG7mo+6qU/eIrqtvP3njos+Snckmsv0xWmimXAEVZFVQ8HNZ58GBasVDRQHITsXd9FXtD0xaSPYaJTL6hOU8l/yI1OuaTdCdOlli6yWclHHRm/t/bzHsh/cEIcwgCRM3APgrW2TiIJY2M36RDA/vvMznBXsna6I8EuglshZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: yandex.ru; dkim=none (message not signed)
 header.d=none;yandex.ru; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Dmitry Isaikin <isaikin-dmitry@yandex.ru>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
Date: Fri, 17 Sep 2021 08:17:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0036.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a33c854-b125-47bf-7c5a-08d979a2c51c
X-MS-TrafficTypeDiagnostic: VI1PR04MB4944:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB494423C8C8E7A358E2555249B3DD9@VI1PR04MB4944.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LiJlaa0EtDRgGiHBoPZVs4Lzy9cXU/hooOnOqsmri0gqJ0p2lXxjLFTyZNh5zyk2izWx12ZdfbCpAbjFySszmB6e9XzT/fv1VJr4uLV4RHgNFoCjGhHEd9UeAu08QCsoZYt4DS+ySEHzsHx1chYY6aM2fNKGdfUFqAnVn26caPPDbVmf8fYcJYf0RC3IuSa9BwHWcjxqxcsjiEdeVUXHr81XYIAYP0s4rxOaJl/ImZGyxw4vHhAozeQkLZxr2v2ix2e+XO+6QZkOJ/6jgr18sCUaHG9rYzrbsBsOWp81mDu2YFTyXDMdVtU4p0mejb2KzSXVoR9rVt85iGKmGgr5zCqBm0f0NGeJ1tE38anCwCC8neyBtsu1vxDlNpCg83uwM8BOYu07YeBhdrFAhIUFlCtCmorZygH9udWVrLvd36lYmK0hOwPsyNlEf7ItS24FxxGVdXDqHqOV3aYOvhAECtwoiHXFXyPdjEayDMCcz/7hF3VKXJicv8auP5+j4DutQsrQuUzTmgy2x8uXXj1DxBzEvID/82H9KG148MFECEpwWjZLtJJ15QAwec0h1mn+9qls5UdWb4o/wbOxZySCTptGbF2mAZRnCGF9c4AXwNsOKpOmao4zQ7mnlA0YkeAm+UvdghrRLl5AIoTGx+ijreyLZpfDdf+GyAuFcmoHsjA8gq3ATcNkw57GyOed6xBKZelNVwQzMd0Tv2b8rSmvEzX/QF84F7Wy/h+ef/O7Sb9R3pq5QUThTpkWxlL4EWwsrl3qyEBbTw+JW3o3ayj1oQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(6916009)(8936002)(956004)(2616005)(66556008)(53546011)(66946007)(2906002)(54906003)(6486002)(66476007)(16576012)(86362001)(8676002)(26005)(38100700002)(186003)(31696002)(316002)(5660300002)(31686004)(83380400001)(4326008)(36756003)(23180200003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?pMesRSQHt8kuPDi8Ac1lzkeQo+0dSa8c5seO8ySoOG0IRPPOsqnD8fhkPSt+?=
 =?us-ascii?Q?Wgr/chZbK2XCott7NZQdjxMwC1euTfWYkc0UA6H5FU585KffJKW/Fk8ePBcP?=
 =?us-ascii?Q?RfV5H7aglvuPjmveSGvHw676zBoFPD45qyPGf02PGCegzZBgptRd578bz6bU?=
 =?us-ascii?Q?hmluINxcsSYg3Kgkkt+jYSWbD1dJ7h0Zp7IqWhBL4f7jtxvmaWXr55SgbJPh?=
 =?us-ascii?Q?nWEXsDdAsAKshSOhdDGlu+O6cIvIRtRGr4FbVPCTNNLKG/eyDGTSRKj8lrNY?=
 =?us-ascii?Q?JYwob2I4pNb+vWxL7DJSZSjctZCbSb6BsrqKuggD+DqTR0C7oLBalf99f2yi?=
 =?us-ascii?Q?zFPh/OEen/DUPLdhYZ8GvzQeqivBoLjSg119Kc7BzXkVlOujNMDnNWhoxPg4?=
 =?us-ascii?Q?LTgNfHmtyrndo0gvXgcUQ/4OawJQ3xvNeznQ2FWnfWZg+/fdH/q4mTLdO+fE?=
 =?us-ascii?Q?GE7g4CGvuJYPR4htYYTB6lgJldRFn01KjL6BaAPGQFy0/uL2VNns2mwlsTxv?=
 =?us-ascii?Q?FScNP+HzGCkO7f5CO8iJZOUGeDA7RWquEnSOJFDa2ZTqCO3PaizVmo3FhYt9?=
 =?us-ascii?Q?IBHjIox33F/+pk4IRi4x29y7VEfv1qxjYG88LYs91de0i+MyBMVzaR6pbGPs?=
 =?us-ascii?Q?OZcFMJc/KK796hVVA6qMHgaRIzDyeIEdW1+yf3isHVuJX52lRSVv8Js0MWJ4?=
 =?us-ascii?Q?ofzy4xZuAVL5yT/nf0lupT606CxWsLhpBROaKGg0M1inOY6MTMulE1UIXRXo?=
 =?us-ascii?Q?tAD2YAO/qkA1TCMDiMxZkwshLabKmv97sY/FZXjleUVaFJVQqD2He/Hq93hL?=
 =?us-ascii?Q?36a/ovvm52ea1lmWsytqMZ0EnHcpRLoqu1gnlRXf2LoCgFNorl6F2/LnbO6U?=
 =?us-ascii?Q?czXU8wifvGmQOFdpprDc3JQU4VgqQDnglxgH2XVkuafnMGVjmeXyyz5rtEEJ?=
 =?us-ascii?Q?9rF5u/wyde3evtEKQ1+3ICsgp5T+Z4kBWisekVQUMSLl3Ehc4j3V61EXZSHt?=
 =?us-ascii?Q?JqI8KLOLN7LIfxMR64IML5tXS16PrM7ZI33lrfhtep46cN/NHwuKTQw65bwb?=
 =?us-ascii?Q?Xpzv88I/lYluQ883cmY6OkdIXZrT8v22bCpWNnhK3M8uOUiJ1nhBxR9l5xCJ?=
 =?us-ascii?Q?+Xb1H7T/pO1cugn8a83dMipgUfYRimNOrHIKlnhxW4R2Rn0fhFAHGyPS9jaa?=
 =?us-ascii?Q?wssRYbMrHdJ18lb8CnIVKEA2UEGU6P93dyzhPvTQEQswmNRzw7f9NIuqTp52?=
 =?us-ascii?Q?alSitj3u/bbJy2NyZHrp29WySIyFEnA/MJ/rZ75cT+WQE0dKhf05wSkNh1JP?=
 =?us-ascii?Q?LMHo4aPsZIxmM1GTdxoh79Py?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a33c854-b125-47bf-7c5a-08d979a2c51c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:17:04.9856
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 09Wou8mrFhgA2z7b4UCdUGrGRTWqBj6B/jUsaqJ98N/XjFBwKIHquepfbvferTBpr6oQNx+wK265j528o92TmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4944

On 16.09.2021 19:52, Andrew Cooper wrote:
> On 16/09/2021 13:30, Jan Beulich wrote:
>> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>>
>>> This significantly speeds up concurrent destruction of multiple domains=
 on x86.
>> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
>> improve locking during domain destruction"). There it was found to
>> actually improve things;
>=20
> Was it?=C2=A0 I recall that it was simply an expectation that performance
> would be better...

My recollection is that it was, for one of our customers.

> Amazon previously identified 228ab9992ffb as a massive perf hit, too.

Interesting. I don't recall any mail to that effect.

> Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
> incomplete, and it appears as if it wasn't necessarily a wise move in
> hindsight.

Possible; I continue to think though that the present observation wants
properly understanding instead of more or less blindly undoing that
change.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:22:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:22:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188921.338335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7G4-0002PH-Jn; Fri, 17 Sep 2021 06:22:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188921.338335; Fri, 17 Sep 2021 06:22:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7G4-0002PA-Gd; Fri, 17 Sep 2021 06:22:00 +0000
Received: by outflank-mailman (input) for mailman id 188921;
 Fri, 17 Sep 2021 06:21:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7G3-0002P4-9Y
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:21:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8eb8855e-177f-11ec-b66d-12813bfff9fa;
 Fri, 17 Sep 2021 06:21:57 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2057.outbound.protection.outlook.com [104.47.9.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-2XHNJDdAPzOW9fkG3Hlk-g-1;
 Fri, 17 Sep 2021 08:21:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2701.eurprd04.prod.outlook.com (2603:10a6:800:af::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 06:21:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:21:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0083.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Fri, 17 Sep 2021 06:21:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8eb8855e-177f-11ec-b66d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631859716;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CMM/97FhbTpeL+vc4edx0Gfl6KNqYqL9R5Sqa0QJDBQ=;
	b=KgrO33uY8xZ3Aez/5RobpMyaNKOEBm6HTnAuUEjhYWmtpBLvtWxzpTb0mLUeAOf7V+LWH3
	fF3eTX6ASHFaKIwQ2TCCwGGnewr9j28KBPj+5XapMNmESwrbS8R9n35xH48W/Ud/yrNJw9
	qMMHEacAv3IUcZBunIyANtyOWHB6MoA=
X-MC-Unique: 2XHNJDdAPzOW9fkG3Hlk-g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LBuDm9U8ePbj6QZONi6zz7a+671/mr88cytxzCnbdnj27mNGUQe9RK8iHGD5io2sR879wht96NfvtZGwkMKSVX+H2EXxmyPvr9KnzlT2Dk57v/IuClk5f433goKkaGJawA3WE1ucj7FqjEDscIXqcsSal8F5sdYbB+GWwpfVK95fJh5EeFuvy3UzpXBy3IwRtBQPtjfdIVnjFtCNCSkh91HbkwkZssQGncqao/ZMEYk+Bx3g6PWgBG/qxlmZCNWFFgCAfyMajIsKrl641uexdRlyZx8LmhtayUe49URNiLAUDVhm6TTjrjPZ1QHZsAxEPqmF7X39SCW9XaWgx7mZkA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CMM/97FhbTpeL+vc4edx0Gfl6KNqYqL9R5Sqa0QJDBQ=;
 b=e07MXve257lbbSQsT3TzrGGxt5aPcRVWE0GB3sf6hbB2i7EllH31r7LBV3z0Q3gg2r0UPEod5BRDlevJap2B+rX5MgtbgdsOJRoXS8Pm7dj1IQ/bTx15SfGaTuIpg4I9oEMXdGHrizG3JU7hARPu4HSP95XI9iBz7tgUqIjpb0XgO0TNS50p7THS8hccRwW+fq1Slk3Okd0TNMHNfR9rLdQLqheDq0pUgbK4kmWoyxlA1wNC5q0tDkp5LbcCBzUJGOGijPS5CJfwwvN22M1kjQvCKPLIhxpqXOwlnODI1yclMEL/lj/F8O8hmF2c/Ja+vJUlGtYgnL91cbHh4fZi7g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: =?UTF-8?Q?Re=3a_Ping=c2=b2=3a_=5bPATCH=5d_xen-netback=3a_correct_su?=
 =?UTF-8?Q?ccess/error_reporting_for_the_SKB-with-fraglist_case?=
To: Sander Eikelenboom <sander@eikelenboom.it>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 paul@xen.org, "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 Wei Liu <wl@xen.org>
References: <4dd5b8ec-a255-7ab1-6dbf-52705acd6d62@suse.com>
 <67bc0728-761b-c3dd-bdd5-1a850ff79fbb@xen.org>
 <76c94541-21a8-7ae5-c4c4-48552f16c3fd@suse.com>
 <17e50fb5-31f7-60a5-1eec-10d18a40ad9a@xen.org>
 <57580966-3880-9e59-5d82-e1de9006aa0c@suse.com>
 <a26c1ecd-e303-3138-eb7e-96f0203ca888@xen.org>
 <1a522244-4be8-5e33-77c7-4ea5cf130335@suse.com>
 <9d27a3eb-1d50-64bb-8785-81de1eef3102@suse.com>
 <d4f381e9-6698-3339-1d17-15e3abc71d06@suse.com>
 <0dff83ff-629a-7179-9fef-77bd1fbf3d09@xen.org>
 <5a8c1f28-eefb-87e2-998b-7cbfb0f0b8dd@eikelenboom.it>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <289a190d-0591-a5a1-669e-28c48a3168c5@suse.com>
Date: Fri, 17 Sep 2021 08:21:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <5a8c1f28-eefb-87e2-998b-7cbfb0f0b8dd@eikelenboom.it>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0083.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4a88c9d7-cb18-44ec-ae07-08d979a3713b
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2701:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2701F07D73E6D4E22DC1CC93B3DD9@VI1PR0402MB2701.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6vpsJN/MyORCsMmJvfm0AP1FXRikDwMyI0x47mBUdCYeo2SXOv1V9LOCrGm7UEh/XyAy7HDTAs4zb5RFc74xDknu0LocTty8AAb4iKlEFe20E/abY+4EyiVVPzCv/UGOhUUnVr3mrjor0uh66rN15o/sLvlIMbTdhEyWCvrYOeAfKJRXQot0L/DlzGpD4HiRl//1oWAEnl+qEZiI5OvGs2YD6aZLzRzdcP/+iejbhTtV+QwqKTQemwdGRe3BRT7/lGkbVOMLd0M8lQ0e6BqmfFvW3jv3IqmItmMwHQrV4A5CGC61SkMaJKRLQy05Oo1e77OFBbwODCZCgj9ChOmgRl+85Yp1TTkT8sX9Lj3ZrEEtdlND6DewYIm1r8dp0JdMC2xaQC+oIQFaZpZUcJHzQl54pKaNqKrbnYMz2QDiRmGmpflhc3QAWCIcssfjV7YJoczPnLyIO23WOTOcQYnexvm3Yy9kdZ2rVMvmzwxrFVjLD3oVb8kZrlEH+zdQ9hW7HxGLuvSfrtMP4JWXLXj0T+JKvwfObfLXqmok6tGWIxUZRI78z5VBXoxuBUG0fZJ1MGmwOr0Vf4Ytc70i9jpsh5diwSgywJKS1u41waWpaE8g/GBsQ9x9ZFyVXUJecnQux+pqeBPXGHAuPKZ8o0RBU7HRVQILZ7Y5lE6oTpXzaIBc8jrCXOvkyh2bk7qjB8483Zmx8k9esESfpzo3nus73vqaq2bm/Wqpa48AVRH7No4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(66946007)(31696002)(2906002)(38100700002)(26005)(186003)(66476007)(316002)(16576012)(6916009)(36756003)(31686004)(4326008)(508600001)(86362001)(2616005)(956004)(5660300002)(54906003)(66556008)(6486002)(53546011)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?d093aThuRTlEdWV5TWpYOVNrVzBvZGJOT1BMVElpRVZONCtVZGJvcUQvQkhK?=
 =?utf-8?B?QkNCRlR5cHhpcndaNVp6SGRsbGt1TjZMV1cyMWNpcEpmU1F1ditCWHZtbkZK?=
 =?utf-8?B?THczdHBzLzg5NS9IMXhoT1BYcm56QzdWYnVPSFdnWWluMnB5ZCtFSHNwRVRm?=
 =?utf-8?B?MEN3Zjc2WlpRQlB3MGJwWHRnc3E0TXdDVitlVkRMOTZNdzlqL01hamM3SWdv?=
 =?utf-8?B?QVpRSWN5Mk5mSmRIUW10Q3cyRVlTTmh2cXpIWllMZWZ0YUo3WExTNk4weG5x?=
 =?utf-8?B?MmpVVW9XN0VVdXM2Z2NvMmlXd2ZIZnEydWtrZ0tyTGF4WEliaW05aUJVUFhs?=
 =?utf-8?B?aTZ0Smo1WERNdGZ2cDUrZ3p1OTA0S1NjU0dmTzRXK1ZINEFLTlRPL2FXczEr?=
 =?utf-8?B?ZVdEOXo4cTdQVTE2aW5QdE5QZklIYjM3aGJhQldqdTYzVWx0TGhRZ3VEYWpn?=
 =?utf-8?B?eTl1eW1RbzlaRDl1eGQ1QUJLQyt0MkhhMXBaVEJHeG03TVIwcW1ZM2J6eVBP?=
 =?utf-8?B?cVR3aFhuOHNuSTdBNU9xTFVmclBrRTNIQVk0ejNETTRGNmFReW1WajB3dzlO?=
 =?utf-8?B?M0VRZDdKTElDUzhUQ1hzaXg2ZTVSMDNFaGxucGlLcUJlbHZQS0Z3NUpsYTFO?=
 =?utf-8?B?YXJpS2EzUnFOdng0K0tzT2tPcFdpd2o2M3BDakpwaEVZdHJ3SDZNNFd1dHEr?=
 =?utf-8?B?MFFPTUZKOWYyaXl5YzJndFdKaHpuWVN5Yzk4bm8vd0w3cURKV2ZrTUp5Ti9Y?=
 =?utf-8?B?cytJdld2QzdOVGtFQUorVjd0dnFjNnBra1VSdC9HQUwvcFlsNkhSbW9odStT?=
 =?utf-8?B?UzlEL0UwcksvT2x4a0RSeE9UajEvN0pBY2F5dmVGM1A5cDNIZFVUMzlDM25y?=
 =?utf-8?B?T0JDNCsranAya01VRHZoV0w5cDFMVUM1WEUyT1k2Rkk1S2MzblNGSHcrREJF?=
 =?utf-8?B?eURlTjdDdjBqYmtGcWFLL0kvWjlzbkd2SG5LWUlWR1YxU25jQVJqRGljMzFE?=
 =?utf-8?B?UjZKb2VsYm1qTHY3WWZRQ1JWTmJFSzZIbE1BRWs1MGZ6NnNBcjF3YXhLWDhE?=
 =?utf-8?B?emhmQTM3czBuL3MrUVdFdVpNb09VeUpIRUJjNko3N1JXQXMzMlNzSWpsK1lt?=
 =?utf-8?B?OFgrNnNqZGcyTlQ2YmtyMmVsWTJIZVdKSnJJTUkvc21UeElUM25jdUx1R3J2?=
 =?utf-8?B?ZVltYnM5YWtRbVNGSXVFbWdEcVRka0RpUFBBQ09vNXNzOWRIMzJxTXFjR0J2?=
 =?utf-8?B?QmJoYzE0NzIvRHZaR2pXU1ZBcCt1OXBtdVR1Y3QrcmJDMFljdUpWdEpjcGdu?=
 =?utf-8?B?SWZJbjQ1VGhrbWxaanZqRmpVTlpqYzliVTBQanFXalcwaTE0SG9mU2lnVUdX?=
 =?utf-8?B?Z2RTQU5iQXlzWTZwZ2RtYi96SFZ0MEFsNzl6eVRONlpHRTVOWTR5KzhPWlVE?=
 =?utf-8?B?VzBCRDNIdG9uZmtTZVZGZy8wV0MvOW1XREs4SHZEYzlvUVJZOERaclZGbi96?=
 =?utf-8?B?VXhKd3p2WFhsSnZMclFUZHlLVDVjVDlMK3BNTnVsMko5NnZMUytkb1VQMysw?=
 =?utf-8?B?UFdSdGYwangyM1dobWNOVUNPVE04R2YzNmVtOVNBb3VCMDhoRDl5N2pPd216?=
 =?utf-8?B?UXlYOVRETTBUV1BWeFpoWHFJdi9OTzBFd2tDNDhnWGtRUHRwN251UHlsWFlz?=
 =?utf-8?B?M0dDcXZaQkRBUUR3eDl6T0VQOXVkRzcwN0VQTGpuaktpR2ZLVGRrWjFpYXdz?=
 =?utf-8?Q?EHc4U/Yy2rS9cL2ibN+0HDR0PNRm9k/XnEkGHHy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a88c9d7-cb18-44ec-ae07-08d979a3713b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:21:53.8211
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CRcddLvMtkbFHGPvD48Fwgf6SI1c4IN7aEqN5tuVL+LQMyBTiPSGtipMh4cSHN+zwmmALb4B03htMNJ26eFSQg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2701

On 16.09.2021 23:48, Sander Eikelenboom wrote:
> On 16/09/2021 20:34, Paul Durrant wrote:
>> On 16/09/2021 16:45, Jan Beulich wrote:
>>> On 15.07.2021 10:58, Jan Beulich wrote:
>>>> On 20.05.2021 13:46, Jan Beulich wrote:
>>>>> On 25.02.2021 17:23, Paul Durrant wrote:
>>>>>> On 25/02/2021 14:00, Jan Beulich wrote:
>>>>>>> On 25.02.2021 13:11, Paul Durrant wrote:
>>>>>>>> On 25/02/2021 07:33, Jan Beulich wrote:
>>>>>>>>> On 24.02.2021 17:39, Paul Durrant wrote:
>>>>>>>>>> On 23/02/2021 16:29, Jan Beulich wrote:
>>>>>>>>>>> When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
>>>>>>>>>>> special considerations for the head of the SKB no longer apply. Don't
>>>>>>>>>>> mistakenly report ERROR to the frontend for the first entry in the list,
>>>>>>>>>>> even if - from all I can tell - this shouldn't matter much as the overall
>>>>>>>>>>> transmit will need to be considered failed anyway.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>>>>
>>>>>>>>>>> --- a/drivers/net/xen-netback/netback.c
>>>>>>>>>>> +++ b/drivers/net/xen-netback/netback.c
>>>>>>>>>>> @@ -499,7 +499,7 @@ check_frags:
>>>>>>>>>>>       				 * the header's copy failed, and they are
>>>>>>>>>>>       				 * sharing a slot, send an error
>>>>>>>>>>>       				 */
>>>>>>>>>>> -				if (i == 0 && sharedslot)
>>>>>>>>>>> +				if (i == 0 && !first_shinfo && sharedslot)
>>>>>>>>>>>       					xenvif_idx_release(queue, pending_idx,
>>>>>>>>>>>       							   XEN_NETIF_RSP_ERROR);
>>>>>>>>>>>       				else
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I think this will DTRT, but to my mind it would make more sense to clear
>>>>>>>>>> 'sharedslot' before the 'goto check_frags' at the bottom of the function.
>>>>>>>>>
>>>>>>>>> That was my initial idea as well, but
>>>>>>>>> - I think it is for a reason that the variable is "const".
>>>>>>>>> - There is another use of it which would then instead need further
>>>>>>>>>       amending (and which I believe is at least part of the reason for
>>>>>>>>>       the variable to be "const").
>>>>>>>>>
>>>>>>>>
>>>>>>>> Oh, yes. But now that I look again, don't you want:
>>>>>>>>
>>>>>>>> if (i == 0 && first_shinfo && sharedslot)
>>>>>>>>
>>>>>>>> ? (i.e no '!')
>>>>>>>>
>>>>>>>> The comment states that the error should be indicated when the first
>>>>>>>> frag contains the header in the case that the map succeeded but the
>>>>>>>> prior copy from the same ref failed. This can only possibly be the case
>>>>>>>> if this is the 'first_shinfo'
>>>>>>>
>>>>>>> I don't think so, no - there's a difference between "first frag"
>>>>>>> (at which point first_shinfo is NULL) and first frag list entry
>>>>>>> (at which point first_shinfo is non-NULL).
>>>>>>
>>>>>> Yes, I realise I got it backwards. Confusing name but the comment above
>>>>>> its declaration does explain.
>>>>>>
>>>>>>>
>>>>>>>> (which is why I still think it is safe to unconst 'sharedslot' and
>>>>>>>> clear it).
>>>>>>>
>>>>>>> And "no" here as well - this piece of code
>>>>>>>
>>>>>>> 		/* First error: if the header haven't shared a slot with the
>>>>>>> 		 * first frag, release it as well.
>>>>>>> 		 */
>>>>>>> 		if (!sharedslot)
>>>>>>> 			xenvif_idx_release(queue,
>>>>>>> 					   XENVIF_TX_CB(skb)->pending_idx,
>>>>>>> 					   XEN_NETIF_RSP_OKAY);
>>>>>>>
>>>>>>> specifically requires sharedslot to have the value that was
>>>>>>> assigned to it at the start of the function (this property
>>>>>>> doesn't go away when switching from fragments to frag list).
>>>>>>> Note also how it uses XENVIF_TX_CB(skb)->pending_idx, i.e. the
>>>>>>> value the local variable pending_idx was set from at the start
>>>>>>> of the function.
>>>>>>>
>>>>>>
>>>>>> True, we do have to deal with freeing up the header if the first map
>>>>>> error comes on the frag list.
>>>>>>
>>>>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>>>>
>>>>> Since I've not seen this go into 5.13-rc, may I ask what the disposition
>>>>> of this is?
>>>>
>>>> I can't seem to spot this in 5.14-rc either. I have to admit I'm
>>>> increasingly puzzled ...
>>>
>>> Another two months (and another release) later and still nothing. Am
>>> I doing something wrong? Am I wrongly assuming that maintainers would
>>> push such changes up the chain?
>>>
>>
>> It has my R-b so it ought to go in via netdev AFAICT.
>>
>>     Paul
> 
> Could it be the missing "net" or "net-next" designation in the subject 
> [1] which seems to be used and important within their patchwork 
> semi-automated workflow ?

I wouldn't exclude this, but having to play special games there means
I'll try to refrain from fixing any bugs under net/ in the future. I'll
resend following their apparently required pattern.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:27:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:27:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188930.338346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7LC-0003GV-8m; Fri, 17 Sep 2021 06:27:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188930.338346; Fri, 17 Sep 2021 06:27:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7LC-0003GO-56; Fri, 17 Sep 2021 06:27:18 +0000
Received: by outflank-mailman (input) for mailman id 188930;
 Fri, 17 Sep 2021 06:27:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7LB-0003GI-Gh
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:27:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4d12c2ef-1780-11ec-b66d-12813bfff9fa;
 Fri, 17 Sep 2021 06:27:16 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-s9X0kTPrPVa5rO-rDPYSaw-1; Fri, 17 Sep 2021 08:27:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5904.eurprd04.prod.outlook.com (2603:10a6:803:e6::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 06:27:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:27:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0108.eurprd02.prod.outlook.com (2603:10a6:208:154::49) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Fri, 17 Sep 2021 06:27:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d12c2ef-1780-11ec-b66d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631860035;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=HWKHW7ZoiNVaIy0mfo3LS1NtMKPljl8Txy7BqM8aytQ=;
	b=IMMEwipMnSpV5qPVcVNpI56p6qKegI16gbBPl5XOBItKJmHNJxKZ+hr4QR9GV9u5BwphqX
	enUCNypDP8eqWDghfdS4cmWDTgmXxlTrswj9sy0lJFa7+Fb1Pwn4BZGOiQvi5JJIbc5Gla
	71wUOzLu30v4wTFPhyD/kBaJ6Cn9H60=
X-MC-Unique: s9X0kTPrPVa5rO-rDPYSaw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Q25i2ebGgB79+F5oGE336FliMymy9jndvpF6sCPFMpktHzv9fCJBD6V6/RDhbTOb/8v+/+UTwoWnDgX2PGmf6EWUU/Vewo44f+sVf0543zaZlqQ7Zy0Iv0S5r10aSgjfyShMbf6JidR3YQOMwF2IR3TVP8XebuZNgXLFMKxBGR5Fv9oHff0rrmu2THRs8k0sTQlbtzhclbn8CBQsEHkGKzLJjTYQQipsJwzYUJ1TMYT0hWkk9LDatGGg8M7tvQ5L0FzlBBjXfSUzWg87jy4fduWxsWH5bPXCfC3QFRHwCToW8MUCPm2X3d3BN3dxi01MwPnbC/0hEElEeQHtcUhxOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HWKHW7ZoiNVaIy0mfo3LS1NtMKPljl8Txy7BqM8aytQ=;
 b=iFP8GZQRiNLvgYFHFfd/cZRlx8c/xh71RocLtmSEFJu2Abx7keyP9LNBpMsYGYff/5D1h35pA01hG8x7RnsBhpCEOhxxFQ68z0RTE4qJfJgCvCJ0HLFQH2UR0JnMmAObOC8q8+1dR0qSzW6ZD6IsxzreDn1OWuSIhxZ+LLvcVCb5B2pgKkWZxMMACkRNvvtoX2bgHV9n/rPPg7PvRtP16pu07bOv6xyq+brhLPgpZG/2HLsKs1MFyJ6/abf289bCZHTUWwBeOijnQ30I/rANzj+x8/fN5U449y2yRGjJyMTeLPP/p4fhnjeMthEK+d98gpfTmn8etW2orxZyrraV4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Cc: Paul Durrant <paul@xen.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH net][RESEND] xen-netback: correct success/error reporting for
 the SKB-with-fraglist case
Message-ID: <ef9e1ab6-17b9-c2d7-ef6c-99ef6726a765@suse.com>
Date: Fri, 17 Sep 2021 08:27:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0108.eurprd02.prod.outlook.com
 (2603:10a6:208:154::49) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b47cc365-0393-44f9-1ca5-08d979a42f49
X-MS-TrafficTypeDiagnostic: VI1PR04MB5904:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB59041FE61A77E0DA91B850D4B3DD9@VI1PR04MB5904.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6rZLycOOX7gljECy/boTmyNvbOBUQfqn0Jjmn8YQ50XeNytYYLyVDdJLwkWzdonrZ650Jp+BjDknK/Ui8GZtgIY6xMP5FPMiw/l3iVlq8yh9ZRI4K/6akHdFxrK+4P0WKnienl0MHtuUbXQjpHVf1lUBxOL9DYKUhFN8q+B1kqDKwu3ERrx5KtXzK/dlJCuzFVS/QYglp9TxVvnglX6nAXHHui+rax1LpXhT9hMepTSZgijQa5OpN///GAHZQKiBpjM+af3wbTu4BZYKazBEH2OB3AHE3vwFfsoznuwWJ0HBZarRCRyA/Wm2/Ipozx4IYxUD8zfa6RdKDnatAUoWEfP42C/KTsZ896jM4Jnw+/4xqQNgo9Hc78VU08I8QyFaEJZUGJ8QVUuikYyb8JO13fmIQgF8KvPxKgR658WAW+emP4xiYpfwOcfW91GWlYipLy7IPUXvluVguAM0DArrhx/WeXA9GJebUtWReVWsblQW0B949Vgw/Rs58Tbc/ELo2hYaB4xT2Q0d072ccNlOigrTrUgtqoXZyX+rMuSry9Jn2c7/Ktz7pME80pkTI198jX5NwI7HHayfh5TkpouLbE+A1gFlltkEWdl+o1MF1vR3LhJoeed2dlEYnM68ZiVNpxwysOMcvGT6XfLDNp/2tI6jtk7JeROkshUXJ+/axz91zcf1FqHtUNjY3Cv8C2jnTHdb1sVUhjFoAS2elx/jvVcyQPjpFDBbVpCvLVZsJkU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(4326008)(8936002)(66556008)(38100700002)(956004)(4744005)(2616005)(6916009)(66946007)(54906003)(508600001)(186003)(6486002)(16576012)(8676002)(31696002)(31686004)(316002)(36756003)(5660300002)(26005)(86362001)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UkdhWmVFTTRNUUZZWmxiVUxyazBBMWcyUURsTTBoQ293Nk0xa3lCOEdKeHlH?=
 =?utf-8?B?OThJQm5oc1kvRmVreEZVWC9JNXVvNlUyRVEvd2tER083ZVFrSTZoZzlZeUc2?=
 =?utf-8?B?TjNTc2FYTFlTK3JPR2kzMU9rcjRrdXVsWXFZb1BFUFIzRFRGMldWQXpZclRM?=
 =?utf-8?B?bkIwd3ZNWTF6S0ZOOWQwZ1puMUZYcWJaM1M2bTVLVHBrRXdqcW55Qnd1K2Vv?=
 =?utf-8?B?OGNRaFh6ZUwzMk9POTRXSzhjVnJFU3B5WXU5OGxrdHZCZGFXYlVMa29wc3ZM?=
 =?utf-8?B?Nk1DL0k4WTVWSWRrT0Q0N2pESWNoWXNveWdMUkgvYnNweHNWckNJRFRuMndX?=
 =?utf-8?B?dzM3QlV4aDFqa1BpYkFXeWI0Y3RwaDllWXFiMlU2U0FOK0FETWk2UTdsVGFx?=
 =?utf-8?B?V2VBNUNUaW04dWhqMTZoSmtaaDZ0OWRJaWRnbURzQzhwZTZVR1NyN3YxZktB?=
 =?utf-8?B?UjRXdWpYNDBiRzlyLzBWY1REeit2QzNLVDBpeVJoK0lMZGlmZEtrVE5QYXFR?=
 =?utf-8?B?VDU3bFRDS2o2ZmhjclhoeXhtVUtSSURFT1BYMk54dXRYOE9KdWxnVEdwN1Ni?=
 =?utf-8?B?KzhTR3kyTythVjNqaTlrZ2h0SnJBL2xRVWlJamRLNEJuYkdNc2xPMnhOc3Mr?=
 =?utf-8?B?cVdKZERhM1R5SkYveVZRdnQ0MUd3WDg1cktHTlpqZDdHM0o2b0s3anh4dGtp?=
 =?utf-8?B?aWFIall0TTdZWFZ2SFhIaEIzRUpUZ0dYOTFRbU02bWhLcjc3TUFVRXRHWko0?=
 =?utf-8?B?M2VXVjNZZ0JtU05EYjBWbC9abjhWUWlrbjhJSk16Y05VNEtRUURldm9aenhU?=
 =?utf-8?B?NEpUOEhTeHAzVUx4Qm5OTGhKNktVVUpNMmVpK3pabFV1amk1d2JRS3FPMWds?=
 =?utf-8?B?a3Vza2IyblNZVEFUQkl3ZWxJby92SThQMVQwYW52R0JsWndCWUs2d2RndlF2?=
 =?utf-8?B?TDVIRzVEdnpZL3hkMnNRTjJkUGZ3aEtpRWNZTU9JMnJyQ3BXcVdrOWZBU2Jq?=
 =?utf-8?B?WjRaZUFzaUh5WHFYaUtVMHVERm56bTZsdHBhOEc5eXVEMmR3ckw3bEFuSHZi?=
 =?utf-8?B?SG5kcVZKdGdTTTFobTNadE52Qk1wMUxvdEVyQnBNOTVucTBKUmVETlZoMHFs?=
 =?utf-8?B?MStlajFlc1RiWlRpa1IzVmFmd0d4MlJGQ2kwSk9iWFFBaEc0SzRnbE1IR1RM?=
 =?utf-8?B?ZkdwRmdqaWFTZTRDYjl4bFJyd05pR2g5OUNLMzZIMHRad3JCaUVuQmlva3RF?=
 =?utf-8?B?YWR1cXdQZzErbkkxQmRZQS92Z2haM2tmb2xzK0JrbzJlM3E1Y0d1QU44K012?=
 =?utf-8?B?eVcwejNmcE95a05nWE1hSW5xcHRzS3FkQmhKTFhKYkc2UVVSaUtldFJ1NitQ?=
 =?utf-8?B?bUs4WkhIdFJzY3AwRmxXVWgwNzlXdFJyMTBYeHg1V3Ixd2xwR1hScVp1eVdT?=
 =?utf-8?B?WmhjL0JlRWNMUXp2RnN6MVNiejRMc2pwVnlnWFRsSzhEMEtRcUovM21nVWUz?=
 =?utf-8?B?aFFveWtwT1VDUHFPMEFkSU1lUytuYklPL2t1R1lMbElrMHBkaENFUnk0a3JY?=
 =?utf-8?B?Nk1EY2xadHNObGdvdHVRYjlCK2ZvcEg1TmpwUFNxTWZ3T2FIOUNIb1JpZUlH?=
 =?utf-8?B?WUdCSDBKUTZZK1BVWVB1RUxNeHMrdG95dDBmTTN2SnlodzlEeVgzMmN3WHdH?=
 =?utf-8?B?UnZmSkNlSko1Y095S2RhbkRJSG9EVUIrQ0FuZk9NWHNBWmQzVy9ZYW1HRU0v?=
 =?utf-8?Q?TiFPAtX2OQLpS5QyEpdzeuUWnrWhI+HbqySlpkO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b47cc365-0393-44f9-1ca5-08d979a42f49
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:27:12.6642
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2TG1/VZESJloobIdrSjVMOIwvM8w5GQv39oMIE/jTFYNbdmyGELDJ1l0J6iPzWm1cnRyOXZUNi/WJ9L8zsfl0w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5904

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>

--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -499,7 +499,7 @@ check_frags:
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:40:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:40:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188941.338363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7Y9-0005lb-M5; Fri, 17 Sep 2021 06:40:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188941.338363; Fri, 17 Sep 2021 06:40:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7Y9-0005lU-Hj; Fri, 17 Sep 2021 06:40:41 +0000
Received: by outflank-mailman (input) for mailman id 188941;
 Fri, 17 Sep 2021 06:40:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7Y8-0005lO-56
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:40:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2b4cda76-1782-11ec-b66e-12813bfff9fa;
 Fri, 17 Sep 2021 06:40:38 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-iHAqES0YPuGfJ-JzOAbWJw-1; Fri, 17 Sep 2021 08:40:36 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 17 Sep
 2021 06:40:35 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:40:35 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0213.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 06:40:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b4cda76-1782-11ec-b66e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631860837;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=s7rb3jCSHExyz8eZmcspOvf3jieEjAd49nmXFnNZWiI=;
	b=FFAsL6jNdKXmAR4I69OWeyE8gGtasq/sHhkMIPYdv+ak1BRv2nw7s9p534XBOZ+zUMa6I6
	bdhFuLkwAJIozXO1IaePXEVFpoSTQjiwTNXjy5Tdu3auvc44aAumZnATWmwuvgHhNylsQ7
	wwrllOdK/tlrStA04bLVGgIM1nZpupM=
X-MC-Unique: iHAqES0YPuGfJ-JzOAbWJw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f9MEn6BXNPg0dHOJiSHIFGZ6Zp/95X7OwOOWzub2Y1AASax+HACJYw1UpnoJNfVzqjNaPXy2zYrCZ5iwR+bwYDWo3N2hsaYWKRghF8waUoOVPa8LFfNeQTOy05CBtjYveiVBe6rUJ/BdENF7gwXmVQoo/AW183h/yiMQOUxrEbtUoH74NcbFsmFMxOYUF0Ww+I3G8otBQLyUShyaBfGkLTHkWdzw+HN7ANymctMnbErx/hzCz8t2266H1FDa3KxXOMwqzXp4eaCmKjoIJxClJeNaNpC0u0n/nCGplkw+ph7NETbVlt08D6zgEz+RUCxUwNL+WzmR0uwM5Xj/W5zmLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=s7rb3jCSHExyz8eZmcspOvf3jieEjAd49nmXFnNZWiI=;
 b=QEX9HbF03oh+HOukV4kISAbliJUKu/hTkxOhT9LIf8CTPWnreDjMtC9rJ/Z81RQuntfEX6e7GIYjJGw7PKBD8NBdDcaQzMPT6Iwq/fzjdtCitgIVZpxRhMi1EhW7W+35JM6EwRkMDxKFFYwyBaH7L9Gj35Mz5+BxJEJRkEIiEfvzCox6pUX/BdI2mJqrh6P4VUgePj7+ABf5FdhJ6uh2JoW81HsYum7cSFrEi4POemgVPBSvdVEmxIVsJ3rkzwyJbpgLtrPqvoRurhwr6bd2dmzP5fTXQc2j4Cc8KUoowNW6rIBTM0kYEbdgMS/GdFlaPBWuPMwIVGnnE4hs+F3WPg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Juergen Gross <jgross@suse.com>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
Date: Fri, 17 Sep 2021 08:40:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0213.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cf53f549-a884-41dc-6935-08d979a60d8f
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70379CEFFA1488D2CEB13742B3DD9@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fP073QF1ahvxF7ZiQOsWKUPfjxawEZFTb+2/2fJ6PuAIErBDyF2Fb16rNVhS7qZB+cP5mARfgjcL9RKcQMOP587IfhAy1NG0/X8FfVKC8yWBUHfYgJM0jgAjAMiw54UWEIRhms07/YlTlfYrYm4872xPvYZfPCikYT88OOKHEGBD/x4En+qjNbww+587PnbLk5urDNHyJBUOZ0GPlOdcfqg5a20GLKd9NGSsMWRlLJMf9bGNYd4VmZqtA3zKrGxiqSVVD0rCOyJeFpA6vsXI5TI6OtLpoHGBw8Jxw4KcxbTVj9pVUagMwWQz+YHURs8lsgvMYnmqtkoLP9AvlJTyEzCBXFeHtPexZzBxzNgVtz7GQFJVibMUW503HvAiuoFDQ21TlQoGvonNMDXeGca4GV5dXw8FlSuk4mNnmGNnCzM73vdfDAK/B4mfh2r2EYvRqDnMewOBUhcBwcDF7OijbYd0CMSGaQTa7udLchaT1m85BhlgSIVELIYFrt0qm3fW+bz1m17A3egSbF9FJ+RzTyKDoyy/dJuT3S9XcB6J4Gw+uJq6wYZVXFyj2sFsO2CCYBpf+3E+M94wp0zgmrDmvRYcivOgiqp2PtVQwvDcOUIyc/iG4fXtl0285M44kSEWmPHGZDAtKNAXQUGKCSbk6bdSm+dJnlQza9rU5vG/CKf+wefvzjkOHcRvN+IEQLYZ9v2LLlUTDNuatXCDyd997lDdLW7pifM63KQFT909NX0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(31686004)(8676002)(4326008)(86362001)(956004)(26005)(5660300002)(66946007)(2616005)(2906002)(38100700002)(107886003)(36756003)(16576012)(54906003)(316002)(66476007)(4744005)(6916009)(53546011)(66556008)(6486002)(8936002)(186003)(31696002)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZGh4RWREaHhQTldSMUZKZnVSQkpvM2hsSWkwMm5sN3ZjU0NtS2xCWjd1a2xn?=
 =?utf-8?B?Mk5QczlRQnJuMXh5RkQxSHBqV2ZNOVlDdGdPQXBWeDF2RXYvYUE3aWFnMHlJ?=
 =?utf-8?B?ZEpiNFlyRmNLQU0za09Ja2tnejdBaEtRWG0zUTh6aXJUU1FXTDdnYXJHVzJJ?=
 =?utf-8?B?VE5vZFJ4L0sxWGh1alc2MEQyTVhBdXlrNjRNRFNlT3ZIV1ZaVE9RZjJTYTZL?=
 =?utf-8?B?bDg3WjhrdW5nRnNkTlR0TkM2K1pJY1MybWVBSDRUajBDYk1GL3IrWEtGNWp5?=
 =?utf-8?B?R3lwdmZDbTNJSEE2SmR4S0dGRnlLVmJ3NkY3QURMaEswZHQwMy9Jb1JzRnJV?=
 =?utf-8?B?Y2JYd3R1MURUbmxYZXF6U0ZoOG5xNXR6YlpQMlZnZ0RTeVh6TnA3UTVUY1M2?=
 =?utf-8?B?ZTVtUkxJcGFSRTdHaG9hT3NLY3ZVc2w0WTJCYU5vaWNLUS9xL0RUS3ZTaDFi?=
 =?utf-8?B?T0VvTlAxMTFyL3ZKamxwUHlJL0FrQ1JuQU02SWxIYVdtQmZKRlBZUjRyTmd1?=
 =?utf-8?B?WFprTHlBdXRzNFBnK2FBTW1PckpNMW5Qa2pRN29OZTEvV2MzYndQUGhodzZT?=
 =?utf-8?B?RDBLS0NYVTJjMHpxUU5QNUdTUy95aTdqazhhTXdQTzJ4cjFIL0piQWxqMEVD?=
 =?utf-8?B?Szk1dFBHRDFmWUcyRFl0aUtzVkRKRmowK1hNVGlOY29EbHBxOFdjbUNKOStk?=
 =?utf-8?B?cHE2bFhpWHBuenh3SFUrSHhKKzQxbStZbWgzN2NSSnQ2eUVoeFU5WXh4dUJp?=
 =?utf-8?B?RjVPanhSd0pxd0M2U1BLNnV5K1RxYzlCZHNZVGtObWV6cDdjend1dmtpa0Jw?=
 =?utf-8?B?SHFjcUdldlRSMlEzZ2d5ZUNMTUV5V0c0Z1Vrb2Jzd3JJbWY4alAvckEvNmdM?=
 =?utf-8?B?MXJ4K0F3RWJJWDRXQ0t4ak92b00wZVh3QWU2S0h2ZHJMVUZHRWhiazgwVVpy?=
 =?utf-8?B?RUhuTVNNakVUQ2dtTTlsbndmRWM4dU55aXN0MHpqY01YZ1NZTDVhcTlKVHFD?=
 =?utf-8?B?bjlCbElnOWpVb21QNnRlUmh0N21lMWI1MHdqSUlvS0FIUHUrSm45RklxRVR5?=
 =?utf-8?B?QnMwUVhmUW5uVXF6bzNQdDdtaFMvTC94ck80WmRyOGFkcHhiQ3pSeWFFNUNX?=
 =?utf-8?B?bVlvNWtnY1VLL290Z2dULzh1bVBiZVM4THhmbWYxZjEySWJxMzA4ckxITkFG?=
 =?utf-8?B?R1NQV2hPSWhvZmNhVUdGV1UvcHVPQzg2NzlRVldJRWJvSWMyYmo4SjYwYlds?=
 =?utf-8?B?SEoxWi9zR3VxZXZlTnowbzFkNERxNXVsY3RFL1JjZ1IySmNtTnpTeVppMzI5?=
 =?utf-8?B?UmxzYXNtaWluT2xlTWErK2gwVi9YZXh6NkplVWdjYTRNQkVTNGVraU0yVDhJ?=
 =?utf-8?B?SXFCUEFWM1IzM3IxWjk1M3dsWEV2d0JKM2hRbWtIanlYNUl6akVKTkNyTEdn?=
 =?utf-8?B?Tm8xakRnTXF0cm9keXphdFZQajkwYm50RmFTcHR1NTE5eGJKYUpKMnpUNEJY?=
 =?utf-8?B?Q0kySzBKVVZISkU3Q0F5UUltS1ZCb0ovZEg5NVV1KzVsOTd4Z280NmgrdkJG?=
 =?utf-8?B?bHpjNHdRRUlKMWFxUkNDVjFxMGhlN09NNXpmWVZqZ1VNWnJYL1d4T0htUTl6?=
 =?utf-8?B?ck1rdkZVa0M4cjVzMk8vSHpWaGV2alJEUVl4clI3dHBTTG4xUS92S1dhdWVW?=
 =?utf-8?B?NzUwa3lFTUhzVlBQTExjZWlmNkdjY2ZPU210MjJTSFEvdVBMOEhCekQ4Y3k0?=
 =?utf-8?Q?G2kGFfvgYshGebxlstpfes6f6dx2P3E/+CKoVNb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cf53f549-a884-41dc-6935-08d979a60d8f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:40:35.0219
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GsSTe3j0pMUo6G5YDbnQ5ZS/J1KjhrZ7CearnoLOgFW8F7RwmbgVPz8aeEde63dOx32YV36gOIdkzUtsbpRe6g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

On 17.09.2021 03:34, Boris Ostrovsky wrote:
> 
> On 9/16/21 11:04 AM, Jan Beulich wrote:
>>  {
>>  	const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
>> +	unsigned i, count = (desc->size + 1) / sizeof(gate_desc);
>>  
>> -	xen_convert_trap_info(desc, traps);
> 
> 
> Can you instead add a boolean parameter to xen_convert_trap_info() to indicate whether to skip empty entries? That will avoid (almost) duplicating the code.

I can, sure, but I specifically didn't, as the result is going to be less
readable imo. Instead I was considering to fold xen_convert_trap_info()
into its only remaining caller. Yet if you're convinced adding the
parameter is the way to do, I will go that route. But please confirm.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:44:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:44:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188949.338374 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7bg-0006Q4-55; Fri, 17 Sep 2021 06:44:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188949.338374; Fri, 17 Sep 2021 06:44:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7bg-0006Px-1j; Fri, 17 Sep 2021 06:44:20 +0000
Received: by outflank-mailman (input) for mailman id 188949;
 Fri, 17 Sep 2021 06:44:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7bf-0006Pr-MH
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:44:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f4158203-3cd0-4ae8-bf35-2d1c21107a9e;
 Fri, 17 Sep 2021 06:44:18 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-OjuI2k-cPfOPcw8Mo5VUdA-1;
 Fri, 17 Sep 2021 08:44:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4848.eurprd04.prod.outlook.com (2603:10a6:803:55::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 06:44:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:44:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0274.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 06:44:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4158203-3cd0-4ae8-bf35-2d1c21107a9e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631861057;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FTK2kMlqhZDtS1pH3nyeuf37Zb8K+rVr4lP+7nA1TqE=;
	b=E+i/XTSRLx4lKpcVdaUEug3S7p7T4mBWw8YJLznghpW7tfACOTHzFCl1nlZ8UX1Hh9GZ/D
	qajLOKhxCaBER/6L1U988YZdIdUEeURYsLcZuACsdlSvJ/EyiQBuwHC/9Jk2Cg+c/YpqnJ
	x/j33460RrxAZfXQM6cSPPJ26ASDa7Y=
X-MC-Unique: OjuI2k-cPfOPcw8Mo5VUdA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EYvylNVfEkYIDWQtKwtDSWTnG0C5OAknOGoxtatZPJ7uzen48rCGomCkLK4p6pTZzqwM28sGnXG5VoJuNz7dzyBnhJy0UOktWAwlaQRVNZiUS7YaJr+JqkVjIXesoxnrSJsx/zz5NxuYZrrOmk45bWTxxYThSMgZRoieTYsmdOtgDRHA42U5AooAU1Yv8jbevwtwJyuCnQP2TG36gslWGHRyuqYrpxkSNxz+KAKaennvxOoMariofx95WhoyiKiBRixFMj8rhyEkhRrt54ipaMRWW541xRK9fsfKXmbxHQVlsg2o2j4zeq+pRr4BOhnSNH/NWHDQ0s3dqnlDd7ZL0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6yWu6BQ5ivwsWsLLZFhrPujMJy9qa1cwJD3Vnzeo0Kc=;
 b=cAsJkJePV3+MuVKGCMDyhSTlHooR+Dh9FpBj/2gQOJD0fZaP2fG2ocIB0kSXFJLmFXnxZ1LpnBUQRnFLjQTuI+Xcv9PZkdWwtHZGEgZhRd+oxDscvLCykALV0tFsMa+6BlVDyHziNW+k7Oyez2F41+sKpxuFzsQaDC0wqa/P0u3ixcbw65A1grHLf/1Npu8IhjVw4Hp6DudxqZWjEyJAwqSn+OYZ0iAB8aeAaSm3nD8mncCWDn671h2mJfHcuKp31a2ayhePaSwZpMmzKujhCSVt5DrSf3PN40JRi0lDsQPGvEoyk+s3VNLkdOJ8ezch8V5JvqXXyDbZWYXhnqog9g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Luca Fancellu <luca.fancellu@arm.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
 <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
 <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e3cf065c-213a-b4be-1f2f-fcfd096883c1@suse.com>
Date: Fri, 17 Sep 2021 08:44:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0274.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d575d683-bf97-4287-8921-08d979a69034
X-MS-TrafficTypeDiagnostic: VI1PR04MB4848:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4848D5A98120DDB31D07EEA5B3DD9@VI1PR04MB4848.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+hgWeDM0gJryPwvgEaCCHByJv/BIUWw01yPYakL4D/hkU7sDUixTq9Irr4cVKzTdl2O2gl6IvQKk7/GT9f35EqpbAhYVeXZHfxjrTfzdfG1r+XaIodzkfpVctKA5B6sRmvO8z2H0FJ63YI/CPYw9kgU1+dTUGj00gEgCyzEbk9gnwBOfbZm7wZ4patjt2PPmQR+Q4QhWb86H0KSlJYkc78Y2gjYrOe554YOl7RXa2r/9KBz1P7VyNNY3Mk2/hThyKH61PHOQv8W7eCbXqjt8eKE8YTRFKW0Us/R6wzJkfxEYySmtmqmxSfVLZT3FlsuoXAZinxOxmirtHk8uXyD+angUABxwj8RgOOM53fGGhkWToBm5Gyzaxp7DIp3Ew8aHDXy7LBBkuKlL0na4l7o+A/6RusImtVwoJWN3tBEBjzg0LVOezeWAq0M1za4FUB28T6yUzj/W5qQnB4J2s0n4Q40wx7UzLCKjHd1UxxdYCIpaNrNoSwEoCCcv6KP5FeMcqH7Jj08HJzRFO35YbyUG8lvBG7wpRtvqKOOqiDD4nQDNZCsp8hs0D8U4kKRUT8mc3XKu7Y63PJ+SN6oCe7hKHRPbHx9a5Eea7wN8ixhquwsN0x4N0gf+tK0j/5AbIjkg7syw22BQMQMljay68YeGT0sprYzIj58IjWSJyzjsLUXjXNZqlo8VHMVxGe9i+TO2aQHM4UVqKmKXvcr6RtabuXEngeZdoPW1DCxlrda1KXZCzuDms7kj0mTFLXW8nBTo
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(26005)(956004)(2616005)(508600001)(186003)(8676002)(8936002)(31696002)(86362001)(6486002)(5660300002)(36756003)(31686004)(6916009)(7416002)(4326008)(53546011)(16576012)(54906003)(38100700002)(66556008)(66476007)(66946007)(2906002)(316002)(32563001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?xpM67I6jhU6+l54X1swJpGwuXIbw2iH+d842JRNrlFJWbATyFn7YS1gf3BD9?=
 =?us-ascii?Q?5JsgQMb4f2uQz7rtZNfZMiHYmACft5JpyVVDrafwc6OeO+7KGIQzC3PWROUb?=
 =?us-ascii?Q?eO7AyOmAg50YElMPTKkuZOX0FOHwhZjaniIw2uBkuKeHE03yGGr9iuKGUsz8?=
 =?us-ascii?Q?Po6U8/Tq1j46PZQ74KXccqb1Up3y/DncqdeKGQpr2MeBMmgPg/WVZCdLgaeP?=
 =?us-ascii?Q?SF2PrPwuha2Nxo6J3dpVszcDGwIhXN9Wh5azt8NWeDlxfI5nIZKqG2adyBsQ?=
 =?us-ascii?Q?23GABTFjVHpfbrMhyOsjjAwYqz74EVUpLXWduLinSfj1xD31f24HvvlekxmT?=
 =?us-ascii?Q?jN7A8FsRFnMLoR1xPppw0mqE88KshaDURrn72qg332xQOV+exVd3PKqPE1dX?=
 =?us-ascii?Q?ayQ0EvC4meyQnr4F0Ivf+e35D5wbtUlj/pAgjPShw/ulcCkhet8Y70qY3nID?=
 =?us-ascii?Q?gABknQqQtqrle3vXwf0vrcWR7y/xf/i2jYU5HbtgvaGjVNC1OFrjiMFO5aTK?=
 =?us-ascii?Q?CqO+JivRsa3Fb6rgtTJsGBBgBtPdXakEnmdwYVukuibO2UEAsw0LkFyO6xVV?=
 =?us-ascii?Q?NIiw7nMjdTOSlKnjqw7E7rcseigJ3EC3osAod7bLhI3B+9InF125BidilpZP?=
 =?us-ascii?Q?VQgN0EF9bfKHSxhCvhQxDPYr1sdmW7jNodJd5irFNX555jTh4WnIxA8bSmZk?=
 =?us-ascii?Q?3qnv+K0dxp956TsRKUobyZbTdqx+cIWC6GUN06P93fRUrj5xix/5E7BuowZI?=
 =?us-ascii?Q?Sl3NMGas78rEA20MVPGffIV3YsydAEcPVNuxHxMP0UfpK8cGgS5qiMF7RipH?=
 =?us-ascii?Q?IDz1MRgA9uqIUKPZ1PgBQf/Lktz9ElBrFysRtUSMd+BtCjRLC6QaIt+sXgrt?=
 =?us-ascii?Q?ZWpQ0EpdTXPXcYJIdabDiV+Gg1ZQvf8dT96N3UgBGGD5RG7p9nMIdLyy/286?=
 =?us-ascii?Q?pmZFUAacgjLcBJzDuRkJzOzWsaRFwvKjhx0v9DbZrbinlbWWz38u1e8CMlnN?=
 =?us-ascii?Q?aQ7mCQwPC3cw1kaL1VmKS5PkD9tkkiL1swzv9uwf/m6Zr98ds1YUqt8m1KmO?=
 =?us-ascii?Q?nftNDxM+6DhjJJb1SE2MFmz54dGkB1oG3zxbQEDknXkl2m+RGgdxLscpUSZw?=
 =?us-ascii?Q?PYutFORfixhA9Ii/A/fdg3a9OTalrGLB6altpHn1FSqorAb9H7udmWBmeKAa?=
 =?us-ascii?Q?kVIIn2SgMGqM4DAZctMcJAjBnj4GzqeySageV3D/6om88zMEzkG+bDTcgOTp?=
 =?us-ascii?Q?LfFFYmwdgr+9fOKgNcdcZ8gLXNbICVoqV70a34q6N7XnZytVHvDxtgRfoHXz?=
 =?us-ascii?Q?aXyGE/OaxJAO7xT7DoM6B6vO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d575d683-bf97-4287-8921-08d979a69034
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:44:14.1859
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jExuV+Tc9aQuuKJAy2PQTmIuQEkIIf8PQ+GzJh8wSxGZl4RlIBNEbhHlUfHOw7TSeiEOK0TdvRxSvqwkN7GURw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4848

On 16.09.2021 22:16, Stefano Stabellini wrote:
> On Thu, 16 Sep 2021, Jan Beulich wrote:
>> On 16.09.2021 17:07, Luca Fancellu wrote:
>>> I explain here my understanding on dom0less, this feature is used to st=
art domUs at
>>> Xen boot in parallel, the name is misleading but it doesn=E2=80=99t req=
uire dom0 to be absent.
>>>
>>> So if you have a dom0 kernel embed in the image, it's completely fine t=
o start it and it=20
>>> doesn=E2=80=99t have to be skipped.
>>>
>>> Here the possible user cases:
>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embed=
ded in Xen image, domUs on DT]
>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen im=
age) and domUs on DT]
>>
>> If that's the intention - fine. Stefano?
>=20
> What do you mean by dom0 modules embedded in the Xen image? I am not
> familiar with it, but I imagine it doesn't involve any multiboot,module
> nodes in device tree, right?

There's no DT interaction there afaict. See commit 8a71d50ed40b
("efi: Enable booting unified hypervisor/kernel/initrd images").

> Putting aside "dom0 modules embedded in Xen image" that I didn't fully
> understand, there are three ways to load Dom0:
>=20
> - dom0 kernel (and ramdisk, optional) on xen.cfg
> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" prop=
erty
> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,bina=
ry" property
>=20
> Then, the other use cases are:
>=20
> - true dom0less, domUs on device tree using the "reg" property
> - true dom0less, domUs on device tree using the "uefi,binary" property
>=20
> And of course all the possible combinations between Dom0 and DomU
> loading.

Okay, so as Luca says "dom0less" is a misnomer really. I wasn't
aware of this.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:47:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:47:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188960.338384 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7eu-0007Ew-L3; Fri, 17 Sep 2021 06:47:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188960.338384; Fri, 17 Sep 2021 06:47:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7eu-0007Ep-Hg; Fri, 17 Sep 2021 06:47:40 +0000
Received: by outflank-mailman (input) for mailman id 188960;
 Fri, 17 Sep 2021 06:47:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+lGq=OH=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mR7et-0007Ej-1X
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:47:39 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 24da1c70-1783-11ec-b66e-12813bfff9fa;
 Fri, 17 Sep 2021 06:47:37 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 831D222308;
 Fri, 17 Sep 2021 06:47:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5F7EB13A79;
 Fri, 17 Sep 2021 06:47:36 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 9lIYFgg6RGFIAwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 17 Sep 2021 06:47:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24da1c70-1783-11ec-b66e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631861256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=T7qVzWrfbaR+Cg64eoCijjDHD1M0KvPoMaiopVbnFuU=;
	b=Vhqe6a8nWSD3IgvToxpXIU6pRO+rtloeF1LMWpZ60SJy45ZMhxdCVPiY/WRZbQ3ihEtvFt
	IB2UgDb8NGGq9bfqhDhoZ/WCLQ6d7b0J230Iz/AG+xD6OF18Pf5fNlnd0KKkuJnFN+nibV
	M6sPXFC4jtt/GV2Z9NgsDD70vUdmRXM=
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
Date: Fri, 17 Sep 2021 08:47:35 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="TfFI4OElXqzn9AyCgez8IR0kpl6ts1NgM"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--TfFI4OElXqzn9AyCgez8IR0kpl6ts1NgM
Content-Type: multipart/mixed; boundary="VyKTfuTUEuumpyk746zj4W8NilR2uKD3Y";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
In-Reply-To: <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>

--VyKTfuTUEuumpyk746zj4W8NilR2uKD3Y
Content-Type: multipart/mixed;
 boundary="------------279F817B8478DF0B10F430EA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------279F817B8478DF0B10F430EA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 08:40, Jan Beulich wrote:
> On 17.09.2021 03:34, Boris Ostrovsky wrote:
>>
>> On 9/16/21 11:04 AM, Jan Beulich wrote:
>>>   {
>>>   	const struct desc_ptr *desc =3D this_cpu_ptr(&idt_desc);
>>> +	unsigned i, count =3D (desc->size + 1) / sizeof(gate_desc);
>>>  =20
>>> -	xen_convert_trap_info(desc, traps);
>>
>>
>> Can you instead add a boolean parameter to xen_convert_trap_info() to =
indicate whether to skip empty entries? That will avoid (almost) duplicat=
ing the code.
>=20
> I can, sure, but I specifically didn't, as the result is going to be le=
ss
> readable imo. Instead I was considering to fold xen_convert_trap_info()=

> into its only remaining caller. Yet if you're convinced adding the
> parameter is the way to do, I will go that route. But please confirm.

I don't think the result will be very hard to read. All you need is the
new parameter and extending the if statement in xen_convert_trap_info()
to increment out always if no entry is to be skipped.


Juergen


--------------279F817B8478DF0B10F430EA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------279F817B8478DF0B10F430EA--

--VyKTfuTUEuumpyk746zj4W8NilR2uKD3Y--

--TfFI4OElXqzn9AyCgez8IR0kpl6ts1NgM
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFEOgcFAwAAAAAACgkQsN6d1ii/Ey9s
Swf/Wk16JCMb8ogQrim92gEAKYI2AW6QlgvCrEGd3sApOLn4HTxXnwYxrMx7wRG7O3HTRAiVIAQ7
jY3wklehZDwGJdH8B2h5MzJHs8Y3M+Hnts1ik9QhOu0UGw01nIXSsm9gaMxmfBegNFBKhIlZE6zU
eySBIglvguNlavrPPvWGY/xFTnBy8E0l71bjX8ScVYjwfakcAzkaUePf6JksR+/xXd/BcIa1bJHK
dMBwvSW6wQ6h1N0YO4hDXXzn7iad0/mTpVIzc7rSRZIP3ROM0yaFL3UHI1cJvzapD4O1e/ewWMNZ
IOltcX1UAhUOo43cTstKqo+mhxUEvQcA11hRbx0OMg==
=zSeB
-----END PGP SIGNATURE-----

--TfFI4OElXqzn9AyCgez8IR0kpl6ts1NgM--


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:50:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:50:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188968.338396 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7hL-0000FK-6V; Fri, 17 Sep 2021 06:50:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188968.338396; Fri, 17 Sep 2021 06:50:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7hL-0000FD-2m; Fri, 17 Sep 2021 06:50:11 +0000
Received: by outflank-mailman (input) for mailman id 188968;
 Fri, 17 Sep 2021 06:50:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7hJ-0000F4-JS
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:50:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7f07dbc4-1783-11ec-b66e-12813bfff9fa;
 Fri, 17 Sep 2021 06:50:08 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-AbPAybkkP8yYkEdNhk1H8A-1;
 Fri, 17 Sep 2021 08:50:06 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5901.eurprd04.prod.outlook.com (2603:10a6:803:e9::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 06:50:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:50:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0034.eurprd02.prod.outlook.com (2603:10a6:208:3e::47) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 06:50:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f07dbc4-1783-11ec-b66e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631861407;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DVe4Aso3E2y3NYywvH6035yhOAAjc8CdQzTeVqfYczE=;
	b=jly3Sb+QjgGB/dGUtrOrxMR5jjPKyzTCIn48vExOTgTu6d9MmZpyx3JEjrSc7nXd8dVAbM
	62c7/VYZ5Nej9ahp60hQzm73WLbyTXvDh4hFG3gaCjwLI1THo9XAkQPp+JCvYbpUKp68u9
	Qe32xic1IcFVMzHYTHqMWht+60lNi5k=
X-MC-Unique: AbPAybkkP8yYkEdNhk1H8A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NwCpruoNzv/WEK9yseZCrB6UZY3YV9kZfiKLicIi4f1lgGNYzmQx/MLiY0Ibk+dYimGWyXxamEHgZ3NEczshGjxR7VdGe3dFSlFFPkJQ/jOCOagqpOikBA59ivB+K7gX7RBBAzYecuKO9uh69/azNzE8Hp3YCiBpHLTMwOGyMVsU0/Iiss9Kx2sv9zysBZ2E9Dv+Gs4amqHblvoo2S76sVp76/RapVRIKEpJ72DlJBbbUxEVCh+E5lQ9Hl/lRURjdRTDvI0BT1Bv97BHxAumYNoGgk6nf1KH13QRt8y6Kdk/cSXS0wOve6DDMACUl5ABu+tDEB6Zb5W4hWNTsZ9Jdg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DVe4Aso3E2y3NYywvH6035yhOAAjc8CdQzTeVqfYczE=;
 b=mH7ta+NWX3g4WacVQBODUyEGB+NztoYrTooiAOlEYpg9VmYZPWmcQeOr1w/Q/whM99DIHEmWWluWxRZYcFXyFFKdZlzscT/qawSSzRwe06XkbJUcy3Lf+4KQx4q8I9wIiUogXJmwBasxje1U11wHdtJi1Tq66NoW1nZPo29EfeTgwlay+W6HfsGGiIiFEYWIBh9ZOL6/mNEzP8fBE6fmN0Pelcvn+46gF51AjhQlCy0FzmLZy8OmCWzejtolhaFSY62Hd3d/WV4V1pVvpzkh+fS267ryAW5DUFnNiFbpLB0BaltLyqZkEidDPOP8XPigsQQAVqAtrHeFM6K8eCaZ6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
 <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>
Date: Fri, 17 Sep 2021 08:50:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0034.eurprd02.prod.outlook.com
 (2603:10a6:208:3e::47) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6dfae42b-c3f8-45e5-eb65-08d979a760e1
X-MS-TrafficTypeDiagnostic: VI1PR04MB5901:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5901255B0F3701CA32472049B3DD9@VI1PR04MB5901.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SOxWRvGAEh/iopF9c1ArzN3KuYfuLhWHAXAH1lDtehRAu09Vv6/CzmsdNEv5y5Joc12feTqGDMboDvMXtIsnUIXA/IGuT49wi1tdbsO/K50NF+6UD+uZwCefX36UccQZVUzV4U1P9kRK15mAWTH/yaBTQPUUZLw2sJYqFPLSC9U/bGYXrn5l+PFydq3bo3nqu4UBVfrVIpTtYuDKj2cQBmM+AG/6zjZ9EjU55alHGAORmWH+VOtSwfe1Rt9Jx5sBASWjgzXCkjzgqFunlR5AefjqnXnv70CvPYZsnNoUbnjhvbsDQrs28kcciUpZpPu7fg8v+7ubQK/KdxFCZRtvQ4mPuoiKGUkdoOIIeZ43KIpMPKpaV3wOv0O4FUJUcc5U3KGAvJ9nMIYTzqzASe5eThbXIG3pK8IsNaA5X9EwSqe7/ynow2HBCIlUV+Q2eJ9jQrorDPjfpW9sxC8iKVvKPN5LrJ8hj/PUH0aTXdSldETfv9R1H/x7fe521RyKSEDPxaoDn67EtbecsFey4A3t05lJSvofwxe40B23qyeqSnjy6oxlhCV9my4NE4XnAZtVj+CpVHDyQmSaW5LRt7MRMndzdXMS9PUbEECvlUGbA3jmrOdnYYchx48Chp7MTqsWKHgb8gaEyf7QeizCHJqNcY6AyV8i9EA0QNuyrJPu/mq4uQEtXqDy8SckuTwjLdHvWWP4jYncFVGNYYAVBIqb4t8QXyDYKtocTLJKpqG9a50=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(136003)(366004)(346002)(376002)(396003)(66476007)(66556008)(31696002)(5660300002)(316002)(186003)(2616005)(956004)(36756003)(6486002)(38100700002)(83380400001)(6636002)(478600001)(8676002)(8936002)(53546011)(26005)(31686004)(37006003)(54906003)(16576012)(66946007)(2906002)(86362001)(4326008)(6862004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?b3Q5ejNYdU8xSjB4N1dTeGM2cnRlbk1pNnNGZ3J5NGJ6SW1XUHNkNDdDeUY2?=
 =?utf-8?B?QVFQdUVtSldXSlk1ekJrblp1M09mL3hJYjdUd21hUTNQYlc4cHZaR0RXZzdk?=
 =?utf-8?B?Q3dGeHU4Z1BiMzJGVFVjQWVqdGwzMFVVL1JuOXM0ZDBqMUNPN21XOVJtRW93?=
 =?utf-8?B?WEU5YXora0g3ckt0OExCaXB1TEs3M2RUNnhKeUNRQWREam1MZjc0L2xRNjFq?=
 =?utf-8?B?R0RkVGlXS1p5QTFNbHcvWmhsdi90OTBRbXlEM1R4OFBCZTR1L3Q4QmQ0cnBY?=
 =?utf-8?B?ajVMa3NGaGFMY2MyV3kyYmZWb2tuR3VHN2tQTzFCai8zcDVJaTNkeTdPKzJn?=
 =?utf-8?B?RkpvdXNxelRDUWJvcTI5Ni9kVUk4VExTVFQyTWg1c2lmRC95ZDY0Y2lHZmVD?=
 =?utf-8?B?eG5PMFJ5RTllck5xUFUrT3dZQ3BSajNsNHN4S3FjNW01NW80djFtNzhRb09R?=
 =?utf-8?B?SVY1ckdRMmdWNnlTelZ0MXNMdWEvUUxOOWI4QUc3ZmVxVC81UDdLQnBvcDRh?=
 =?utf-8?B?YnRBc2Uyd0NUY1pkdkw5eFhGcjF6MXFaUW44dyt2a3VROFI5Z1NNemxkZ3Nt?=
 =?utf-8?B?YnhZbkJOeW15VDRQWXJOVmlQTkppNkViYVJzT20zcGtGWWxhMXlqbENGSlIx?=
 =?utf-8?B?UVYrODR4UGVNTkNwVUYwL0QyNW1Xd2YvK041YytXeDMyWTBhdFVSV2MxUTB2?=
 =?utf-8?B?VkRxVFJMQmEyWWFCZU9iWkVkTUFTb0pML2N6dDN0NWdpTWdod1lMT0Vhcnk2?=
 =?utf-8?B?SXdyampHaWdsY25UZXg3NzQ1T1RxL3B5RndlUnZhRnNuRjJwOW16WGRmSDB0?=
 =?utf-8?B?REU2KzNBeHp2RW5XcnhFbm1abW9WbTdkUDFLbys3KzlEWG1HYlhwaFViMUdY?=
 =?utf-8?B?WDJ5Y2c1Y21JeWQ2WWFsZEswUWZRQ0NxZ1Bab1VvY1NvRkhhMGtzdkk3cDJl?=
 =?utf-8?B?azQrbFlGWFFpMzhZYmx3NTE3QVp5bm9XRWJJMnNmeUZ4U04ybFhCL0pTWDhB?=
 =?utf-8?B?Zlp1Y0ZyRGhEN3lYa3d3Q05VaUV1Z1VWWW8zdHZyeUhIOEFCakZOMk9NUU41?=
 =?utf-8?B?YTZwVkxESElwSkhianloSGkrR1ZaNDVOT3FZSGJuTVNKVkVhZzRzemNrTlFi?=
 =?utf-8?B?ejg0T0tvSHZmaU1xeFVPTU1SZGpRbjQrQWdPRHZJR3A0bk11VDRidGlhelM4?=
 =?utf-8?B?WGRQSWVPOFk4aHZZc1dlUXdHbjJaMUJBSzI4bEx4YjVpM1hmTHQyTWhpb3lC?=
 =?utf-8?B?cTdsWlZlRkFid2RSYUJpdXVaT0NmbWtGSXFaeUk1S3FTaDJNRzcvYzJJbDFO?=
 =?utf-8?B?YVBIenp3RGlITkZTaVoreVR3YlVmVk9kQTJpbkZrNW1FOW0wRXovRDNQZXRr?=
 =?utf-8?B?UkRucVRQbjZ3emE0MFM2U1MzUUdyZllJNjNhdm9tV3VOSE8xbDB1OUU3eXdG?=
 =?utf-8?B?NHI5Y3dYSkNRelhFanRvQzUrQ0I3RWFZTjZuRXVscms5WjRtekwrRkx0bjVL?=
 =?utf-8?B?VThYYXFRUzVlZG1JZ1BYS25DYmxKUVBYbklCV2UvQ2Q4eVNDbEt2RFBTMjdy?=
 =?utf-8?B?RWdObHNUWEtGTzBZMUpqRWp0Nk52YzRMTHdtaVhzai94bitkWEtrUlQwa2lL?=
 =?utf-8?B?WGFxUXBQbFdkRVZEY1RxMlVGMkl1OUlmdVFhajVMUmUrK1Nib3VsME1KV1pk?=
 =?utf-8?B?ek5HSnRxL0gvWUduVGdZOW1vWHJXeG1tbUhlMEkyT2MzNzFIQUpRSVRqRmlE?=
 =?utf-8?Q?Zv7Z4Q1ImVfrr9pPTtNm6Z3T0m3XYbAD5oIB1Vy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6dfae42b-c3f8-45e5-eb65-08d979a760e1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:50:04.2773
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /vVJ8VpcVhbjlIebMAuHKpmyQGFkit66vdEWkMjl8//7S0wQ+pfmGR8wRNxxT0b+J3YXCFsGUSxQRozPaJeqvw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5901

On 17.09.2021 08:47, Juergen Gross wrote:
> On 17.09.21 08:40, Jan Beulich wrote:
>> On 17.09.2021 03:34, Boris Ostrovsky wrote:
>>>
>>> On 9/16/21 11:04 AM, Jan Beulich wrote:
>>>>   {
>>>>   	const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
>>>> +	unsigned i, count = (desc->size + 1) / sizeof(gate_desc);
>>>>   
>>>> -	xen_convert_trap_info(desc, traps);
>>>
>>>
>>> Can you instead add a boolean parameter to xen_convert_trap_info() to indicate whether to skip empty entries? That will avoid (almost) duplicating the code.
>>
>> I can, sure, but I specifically didn't, as the result is going to be less
>> readable imo. Instead I was considering to fold xen_convert_trap_info()
>> into its only remaining caller. Yet if you're convinced adding the
>> parameter is the way to do, I will go that route. But please confirm.
> 
> I don't think the result will be very hard to read. All you need is the
> new parameter and extending the if statement in xen_convert_trap_info()
> to increment out always if no entry is to be skipped.

And skip writing the sentinel.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:51:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:51:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188975.338407 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7ih-0000qv-Hl; Fri, 17 Sep 2021 06:51:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188975.338407; Fri, 17 Sep 2021 06:51:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7ih-0000qo-E3; Fri, 17 Sep 2021 06:51:35 +0000
Received: by outflank-mailman (input) for mailman id 188975;
 Fri, 17 Sep 2021 06:51:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7ig-0000qg-A2
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:51:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3e872609-466c-4c5f-ad97-ef5ad58d35aa;
 Fri, 17 Sep 2021 06:51:33 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-tIcJ4vFCPY6MsjZQW7XD-Q-1; Fri, 17 Sep 2021 08:51:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 17 Sep
 2021 06:51:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:51:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0061.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Fri, 17 Sep 2021 06:51:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e872609-466c-4c5f-ad97-ef5ad58d35aa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631861492;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=K9l25Mm1Xmd8Y5fKa3uYtWfNkiU7OCS2lWznkeIB720=;
	b=WHGLtgy7prTa3gabvUS5/Oyt62D8CYKTZKUFnxb9PhXsn2n72gmaNMnuEs9ALbadKWoBN6
	sV+heCov18kNce9t6wP4+9zlRQtYqiHXi+6FtzFX0jT4+DpDJaENgx12vZZ4ms12RIeCAt
	Es+gp0wcoHLFUE1enAD0Yls4y9qdNBY=
X-MC-Unique: tIcJ4vFCPY6MsjZQW7XD-Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZRMVErH8R/MHI9FykEhHGlAOwfJbpT8u/+qepnZ3q8hjgasuuK7pfxsz46DlVG/ezLBIz5KQZWhdOH3PAvmBh0SbJbNsyYOregH5MVrMeYepyySB9jjd2w83lkgFsvb4A56Zc9rZwANnI8tBH/9XZqFtWOSl1whAUCwnBhKwjQVLBM6HVnHP5t6XpcqyWNQmyigcKk9iGqZENkM+2ofysOQhmujz+sgJPs8Cj5r1wJrXeKFkuc0Y2Wg2+AhS5ziUXg/FSeb10JSGrqL4O0SjKCPwcnJAQZ4p3dvOSKfoPcL78KaYdItqPzzZ6IbCHc1bEAUgZHLvr1bz4GWNGI+X7g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=K9l25Mm1Xmd8Y5fKa3uYtWfNkiU7OCS2lWznkeIB720=;
 b=JElDhx5mDBM7tHytj4PJnqkHyMHkgVow3fPboaE0XQnGa9k/Na+ohxtFqw2OiHTBhcvhZZii4f/U5imsZMQ0QEQwG2IkYpiz9nkhboTYFnhsOdi2UVeKQwMB6xotObGOhapkR+DHKKHJqs7sRDJa7LPBfoN/etsse47jB+QilmtYyi6Lo6FSIM7sboU/1nGWZqNO5H+VIPNxcU2PCvs3iSuRAx9YSjTdX9Nl2EbCULoWp5RTBFmTiKH8h/H7R82WFRUUfvSxg+OP5LWhccbebfBbFThUtq7JhwMzUr5K40ZdPWcxKeIrE855V+L3heXKGB3zbPC/qvpRp3gqr5DwRw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] modify acquire_domstatic_pages to take an unsigned int
 size parameter
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: penny.zheng@arm.com, Bertrand.Marquis@arm.com, Wei.Chen@arm.com,
 xen-devel@lists.xenproject.org, julien@xen.org
References: <alpine.DEB.2.21.2109161341270.21985@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b17a3349-fda6-8f1b-e6d3-3db2658f531d@suse.com>
Date: Fri, 17 Sep 2021 08:51:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109161341270.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0061.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2b4da87b-73d8-4fc1-7e7f-08d979a793fd
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260699F54D3CB662353657AAB3DD9@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CbVmbrtbxGEFZMKOFLggRaPiUmat8vRL9eGOxYpQ5c9LRlBeQlL8EiKi6hmrtshENFROTLwaIPrb+J1wnqsR/IFOXUZiuYZ5HvQW3Zt+wSttU8bFBIq5ku0OT5yDGwCbC4LDl7XS8cm4bsvoN9PpSVsYBXbxeroEL1fSBc5TAir/Zz6XcaZInZrosuSJXdwmF95CWKoTfy09lis6qnFZ9mCJYYSAn8nz/+6sehjz85XbUzkf9buMSGzFUcL63pOmNyADR4uNIGOBI3Jaq5f3g9vh3YESvZ7lPWEgjQSUONbU+qxxfwfqk5HnlGW+9sgKqGQJrXIQtx2fuP37Rzb0fR6Ox0z5+uiiHkC1tbtf5RcTM5ticfwpmMOlxkbmyhDgXQEalQhdPL1OHRQLYcCNiINtJzhxsBJHpbRd+YXCPh2jSu1Ir+Dp8eE4bnv0UYqebOZOs03VdbPVFYf43SKM3YwpAnc/JXx7PkdRqYVZF9rxPrDemXWHdeHWszgID8iZuHkhirkjK/KuvOH2BnB/PjmTs/dY01raSat4qiXfRQmUN0JjP1/bdp/5vBmGuq1RtxbsBjEmd3iaUgQEn+sAbViCsY2o0dXBpa7HUHif/2Fnsp3j21k0Lzfo37Moam4qmDVmma+wOKwcKq7T9wTx6B+ChK6MG85LjUiGeE2sqawLZokrlpGn9w2eHSHzHqZyMbPk04sBcyaDF5se/Ark5wRtWdVZGu9lhbudaGHvJng=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(38100700002)(508600001)(2906002)(5660300002)(956004)(83380400001)(53546011)(2616005)(66556008)(66946007)(66476007)(31696002)(31686004)(86362001)(186003)(26005)(8936002)(36756003)(6916009)(8676002)(16576012)(316002)(4744005)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K01WS0FZYUVwWDRld3NhWkNUQ0Q3S1JvaEdTYXlWNHJ4b0F6NVgrcFFzU2xy?=
 =?utf-8?B?QkxuUjAzVW05WjhJeThFdW1LMjNvYTBweWdoQVFQYjdsSGtVclFaOHZ0eGZF?=
 =?utf-8?B?TGJBZGdjbWg3TS9SRnZVd0pqZk5LTmtvYnJvZTA5Lzk5c3FkS0FlQ0FnSjZ5?=
 =?utf-8?B?czRTUXlBYm5EWWMrNXBUdGFRc21CVWNuUVB0TmxSTUlVbFEvWHQvb3FZVmFi?=
 =?utf-8?B?cjZIMUNhOGMwNlBTRzh4YU15Q0E0VWg1VytOWjdTUzN3bVl0SUQxSzR6STBP?=
 =?utf-8?B?VnQwNHNQbjVpU2djOGt4Y3hReUM2cVA4ZWRzcGNWOUxDN1F2VGUyNjRGWWMr?=
 =?utf-8?B?VEU3dFVIQjZ0QXlWSHE0enNyVEFWaXhZV2tsbU1uZWRIc3hhdkJqU3NRZHhB?=
 =?utf-8?B?c3ZSUlgvdzQ4RXpGUWNiQlE1Y04vMDFISms2aHcwWG80SXVOSDQwdWd0S1Uz?=
 =?utf-8?B?QXU4RTRpQzdOVTlKRmRGZ1Z6eXdOUDh4MERrNDhTU0RzOXlYVzB0ckJQNGhm?=
 =?utf-8?B?Q2t3OXVpMFNQaW1LbHM1Y0FucWl6cCsxeThxekNCZHlFTkQ0T0dXelJPOVZ2?=
 =?utf-8?B?S0RpeVI2dTQ4cWhrNE9nQUV5WVBGREtKcTFtYXNwTUQ3WnlScWo0czNyR2RU?=
 =?utf-8?B?VWZsQVZ1cEFSY1UvVDN0dkpXbHRRbVEyN3IvQXcxL2JkRGhoMzhndG9hdklC?=
 =?utf-8?B?M0t4Zjc4QUxnTVdKMG9rd3ZNYXk3cnkzczBaMjdaMDdLWEZqcmY5cGhPT29S?=
 =?utf-8?B?TXhRbzlNNkh1ZEl1RDI5RExndXM0SndBbjVBQ29yb3EyMmR0M3F4N2d0RzlO?=
 =?utf-8?B?S052K3ZtdWJwQWJQcDZQV2VQa1dIMmtVdnBVckw0N2Qvci9oVHV4T2VqZmdM?=
 =?utf-8?B?L0NiL0xET2oyNXgrQmxLdHJWVDRVSm9QMXF4Ri9zRjBrM3hjS2Z3OVFiSGJ4?=
 =?utf-8?B?SEc0RTAwQUJvdjdBNlNhUnVRK2J0ZmlyOGQwR0psK0FTQmpSZVprMHdQZGZ2?=
 =?utf-8?B?Y05NRlRXL3FBcnRGODVFQlBTZDM2OUJuamNSbVZoL256VHh2QXA4M2JyWG45?=
 =?utf-8?B?WWtrblB5U2RBSU1ibkhtaUZ6NzBRQy9vM0JaQkpHOVdFY2RIZmwzV2QyaWxP?=
 =?utf-8?B?YlpsM01kT1pVM0pRQndOVER3cGQwKzd5a1VBdDJnV1daRGI3Mm1BakNmN0x2?=
 =?utf-8?B?ZTRueENiMUxPUTY3ZHN1eE9VejVHM1JGWWhPTWlQQW9yYzhwbldJOEg2SDFt?=
 =?utf-8?B?d0lTNGZJeG9oZVdIVHRsSVp5dVZMRlI2b3Nyb3hkbXNLbk1nS0dMcmRCTmY0?=
 =?utf-8?B?Y3NxaUZZUlk1ZXpJYlI5dzUyRFRaTmVJY3hFbzFpczVDRk80QUgzazJqaTBP?=
 =?utf-8?B?L0g5NTF2QTFxVTN2cGoyb1hCNnpnRXlmQXB3UjJzRGxwK0o4S0ErR2o0MGRt?=
 =?utf-8?B?Z0QxVU9mSTc0eUxGdkNPeDVpcmFRR0IxaVNKL2pZOWtDWlZLUnd5Q2hPQlN2?=
 =?utf-8?B?Q1RyNDJUNlIrV0VhbTBUdC80dEtMb0pHMHBCY3docmRKYnZqK1lqWE1lcXJO?=
 =?utf-8?B?VGwycENJMW01OGRqb1JCMDlxenZxVXE1MWR0akNUUS9LMjE3UFlXb1laUkJW?=
 =?utf-8?B?SXdTeXBMaFBOL0pnMEFmeVFNUGhyZllsUnhISGVneTFFTzM5Z1cwYmN4NXRO?=
 =?utf-8?B?eFFPRFBTcElVQlBvSmkycW1neWNOeVRmUUw2dFZNM1F5bG8zbUttOTF5eENT?=
 =?utf-8?Q?zCbNF1nanpd1i723NOPa3GbHSFSsI2YRrG9QI9d?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b4da87b-73d8-4fc1-7e7f-08d979a793fd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:51:30.0138
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: voit5ewuEgKlLxQu4Vp+BMkHwYBeUraUkMY/OsVGtwU15xYCbkEFB+GPWGPcEO+bNN/t4lgXfEDwiQ8hnQ4lIQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

On 16.09.2021 22:47, Stefano Stabellini wrote:
> acquire_domstatic_pages currently takes an unsigned long nr_mfns
> parameter, but actually it cannot handle anything larger than an
> unsigned int nr_mfns. That's because acquire_domstatic_pages is based on
> assign_pages which also takes an unsigned int nr parameter.
> 
> So modify the nr_mfns parameter of acquire_domstatic_pages to be
> unsigned int.
> 
> There is only one caller in
> xen/arch/arm/domain_build.c:allocate_static_memory. Check that the value
> to be passed to acquire_domstatic_pages is no larger than UINT_MAX. If
> it is, print an error and goto fail.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> ---
> 
> Jan, I took your suggestion of moving the check closer to where the
> value is read from DT. At that point I also took the opportunity to
> change acquire_domstatic_pages to take an unsigned int parameter
> instead of unsigned long.

Fine by me; fwiw
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 06:57:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 06:57:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188984.338421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7oh-0001jc-B5; Fri, 17 Sep 2021 06:57:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188984.338421; Fri, 17 Sep 2021 06:57:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR7oh-0001jV-7H; Fri, 17 Sep 2021 06:57:47 +0000
Received: by outflank-mailman (input) for mailman id 188984;
 Fri, 17 Sep 2021 06:57:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR7og-0001jP-CO
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 06:57:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 06bf7bb6-4b34-4dc1-804a-f6223362d829;
 Fri, 17 Sep 2021 06:57:45 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-CzZJouMqN9yPKeNlYR9BGQ-1;
 Fri, 17 Sep 2021 08:57:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2607.eurprd04.prod.outlook.com (2603:10a6:800:58::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 06:57:40 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 06:57:40 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR05CA0003.eurprd05.prod.outlook.com (2603:10a6:205::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 06:57:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06bf7bb6-4b34-4dc1-804a-f6223362d829
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631861864;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R3u7vFGXazRIkg5mIgOTXNgjjxSjdm51TrgOQkoaQHM=;
	b=bn4bZvl3mBukAIdqdWId+16eYc2cq7/RsoLGNhuV05WAGzhEKevm0wqJHQ5ff6FM9lXbB9
	wdyyA5ENpmKRQKt62oAL5EfB0I7cutUBDSQc1YHc2G5uezDJRFZv/dU/S6L7WYgLRSsbGu
	JROOu7pH5X1sR3Ll2DX2IGEiQhzwX00=
X-MC-Unique: CzZJouMqN9yPKeNlYR9BGQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SEzdsP/dS+w8Crlba0fkmhX9/KCCLKZZ8IxuIbSvXWzHtFumv6ZlHRJwjGc0LGbYZb9L9T2rerVSX3tIsfwh6hWc1trBQ0ThQcfOt/S9UwvvlZqwkVUDOVqQOQ8T6gMEWE8LE3hA+jlU9SzOWSR4u7ZqpQnaR68JO2PhG4sMyrkyhI4zrB5XBw4hQTCa9b7w/xiwve/MxCtaYkvxfTNvyFnBYrtBq165T14qlyquQm4aagaelPnI1slJFszbxHaLY1j3Kg2cdM50fDWV4WFY37qpAufrkdK5IypeyixH26axKXNrW+BynqyhRCRDI1JPYbxlov2s97u0JVyR9MC9TQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R3u7vFGXazRIkg5mIgOTXNgjjxSjdm51TrgOQkoaQHM=;
 b=mJEmpKsnZhdkqXmuRMOr3BqU0+1bRRceipOjFTHeI3jpMUl47WAl2L54OaVsy0M+f1c0ptLObXV2W/hGax+qOXzD4gtpd+qb6gNHOmIqIn6BcSCQiEqEVPI+PbPU/siQ8gp9dGH8G4aQGRG0OUAXT103YkdMKP6P1vgO6PG9fb5issUw5bOkV+WbnjtQdJWrzxpKGxuuaIw++9Tr0QLgJ6QVJxRh92wn/cVuKM25cs8OO42a9kXWU4E7bQRbWuaqii0VBQZGMWtc/EPv+Ao4Dbz/Bpkel3ZQrjTuNRUdAz9A/21386w7lpFhftmnZIwtK/ODSXsvXNLg1K3S9HUr9Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] xen: fix broken tainted value in mark_page_free
To: Penny Zheng <penny.zheng@arm.com>
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com, xen-devel@lists.xenproject.org
References: <20210917024855.605551-1-penny.zheng@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6019d137-5298-d01d-eeff-51c228c133db@suse.com>
Date: Fri, 17 Sep 2021 08:57:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917024855.605551-1-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR05CA0003.eurprd05.prod.outlook.com (2603:10a6:205::16)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6d540d25-ae41-46e4-3b0c-08d979a870c3
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2607:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26072FBD792DA761F8991462B3DD9@VI1PR0401MB2607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2887;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Yj1i1sDdHBXT40iObTUXeyrErmpzj4J2ce98QIJP64QhXCKSKyTiNuuzR/7k3LEt1Aia2Ycf883fOvg43nVC1pFxfXW4PO7HJbPNVtWgFf0cHpCmqsDvZhVwHnkZFWvIwvmmwCJybgmvUw24CVFF0aqGuSaCBU+w72/CMAka+D4GKDVgKkNnJW7xRUW4Bzf/tF89lm9zsDBWnNm7LJSJa767Q0E3VQQsaQvQDjZZocoNUpHeP3j60GZZ36abuzHhFD1habnDmkAsEBvFkGw63suf0nsdFJjeZqHZub3GIro6EtRXFRAI6L3X5ujjIzK7NKhBG9FLYmue24FQhaj1QnTmhTn5+gnxZOQg2FcX3H9BYHe9JNl1TKqG9rf5/DP/oWHZt3HrbVAKiysp7z64z/A1Al31KhDX95MDhkSeg9e0Gi27RGUhhswWGRoNaBgBKCoV5cqFBqhUP+bpXlirGRcFgrkRAUc/Dke/j7Xp9225BnJMP7T2pLsgxXgMY2lvi8Xd0EX8RHoIU79U+7l57Jg0sqv7mjd0IpTOzGTIltpu8rFrzfiPRJvJz/IkhBaqqucO7ln3z9ZIC1r9be3ytSuSf6Uak5JftCxuqGEFeCD9+67Et9Xjk8Wy+y2PZOHc1vljjMFiEXfIgBHBBs9oj9iz3Ch+13FjivfaSSYxozc2z0MSUuceaOHyNxVAmIcoBJp7UqRyC61XMbFBQ2M/U8zEpp+17pml+SVFw8Jx5kw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(396003)(136003)(346002)(366004)(86362001)(478600001)(6916009)(83380400001)(53546011)(2906002)(8936002)(2616005)(38100700002)(956004)(66476007)(26005)(66556008)(66946007)(6666004)(5660300002)(6486002)(31686004)(36756003)(31696002)(4326008)(186003)(16576012)(316002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WmxKd1U5NjZtZTFzczNoL1NIb1pDTm9rbjAvb3RiS0hFcnRiZGltdmtFYlJT?=
 =?utf-8?B?YzdBRDE2YUN6YTB6d0YzZHMvbUxMK2NiMnBrVFRPTHVEQ1NNQmNOdTBwL0dH?=
 =?utf-8?B?UnlwSHVFWWNiaGVCTy9YOWpqQXNsY3l1MXFXclVucVNGL2o3NjZUZ1ljazEz?=
 =?utf-8?B?blErYXFzeWhWNjhkK3ZDQUF0dWI5N2ZjTDJlWGFCek1xNGFoZ3N1Nlp4RFBH?=
 =?utf-8?B?RmNxbVIwZ05qRU9EeVozSjJYWVhOeHJyQW5yZHdFb3I5dTYyanJqa1prYUhQ?=
 =?utf-8?B?UFhLUHpNSVBlRmpsSU9XU3VQZlUxU3ljbVBCWTBuUURsYmJ5L0RINEliVldR?=
 =?utf-8?B?byt0VmVJVUxmeVNPWk1mSjJUYWJPLzZ4Z2lTcXdlQ1ZJb1IxcHVXdVJqcGNT?=
 =?utf-8?B?SllRVlNZbjRGVkY0Zyt5R2F3dURudkxtem0zeGVsbXdOdXlMNC85OTJqQTNC?=
 =?utf-8?B?WWRqUTIrS3p3bXMrQjRScmRyWDVsRUo1clNUQTdaaWJvSmpPWkpkdFBhc0kr?=
 =?utf-8?B?c2pGNDNyYm8wS1MwNmVwc2czeDBYUkR4aGx5ZE91V0l0cHUvY3Z2d2ZDYnY2?=
 =?utf-8?B?UFJVemRacXZMbHN3WU9nRnRtUHBmWEpwMTJ4QndwSWNmQU45OXhGZitBd3gx?=
 =?utf-8?B?QnQ3WVN0Qi9kdU12TXlzRXlHdVRmRS9JVmYwUzhORmlqaEdpUDI1UzRLajFU?=
 =?utf-8?B?eTRnbGF5cFJ4QVJpdUdPZTRpdzRIRDBaS2NiYzlUeTZIc3N4eXhrWnZ2aERx?=
 =?utf-8?B?TGNnWGM4c29JUEZ1NXlnSktrWktIeEVsdFpWQU5yZUpxMkwxNzQzU2h5KzhC?=
 =?utf-8?B?MDdXTVpnZFBpRG1NbjJqRnpwZWpOQ0cyMWxweXlBUVpTemdtUTM4UG4xSlph?=
 =?utf-8?B?cksweGFreWFObE4vNzdCcFdLRVhSbGl3VVRPMWl6L3hQRHZPTnZCRTJoYU1n?=
 =?utf-8?B?Rm1hMVVwdnB3SDJnNW5ZVEx4TFZQSGQ0d29qUndaRWpnZmZJNW9YRCtnNVU1?=
 =?utf-8?B?NW9HMWlKeWhhU255VlEydHF6Wlc1NG9LNlNudkp6TnNMM2V3cGUrVTFlM3NF?=
 =?utf-8?B?RnQwa1ZZWFVadUdFR1AvQnd5YVdTOVVwamZnTERBSDREcWNBWTJwcGNKczN3?=
 =?utf-8?B?bmtaMjN3akg1YW1USnFBK1RMNm1mT3hNZ2U2dGRTYk5wTGZJVUpRelE2bVJE?=
 =?utf-8?B?NERHNmtIcHRUS204aVM4ZzJDOHdoUmJJNnhlR1A5ZUduQm9yaW93U3lqcGhU?=
 =?utf-8?B?UWlqckUxcWgxTGZkSFhDcGhOTVEvc1hHY2U1TkU4OFNveXY3ZTdnZDVuUU5k?=
 =?utf-8?B?UGFPNVFHY3JjaGdjME9Kc2EyRGxIdnJQMXNFL3BLWW1ETWJQSnloR294VGV4?=
 =?utf-8?B?K1dWdHNOdm5LWXRxV245RkZzZE84YVpYajUxbitEbG5uU2RGWnVWcnF4MS90?=
 =?utf-8?B?c2x3VHY4Y1pZVk16cEtFK0xYTFRWck05SmlaemhrTDNwZW5GNlVLZVBSQW00?=
 =?utf-8?B?RDh4Uk51YlBVMG03dUlzTnhmYXZVQmFTZkZIQXhwSUtlU1MrME8rUzFNVWNU?=
 =?utf-8?B?VHIyay9pT3FTWnBzWFZMTFR0TnJOSldKN21rOUx1ZGxkV082ZDRRa3oxcjhv?=
 =?utf-8?B?ZDkrcjlncnh3WW5ybiszOTh3R3N3blpDQzhnbERpZ2FEZno1RkUyWEVCb285?=
 =?utf-8?B?THN1ZEJFRldoRTdXWk1LT1lYZHhwU3VaRHlXYjRIM1lwOFpHUmhFNG8xVkgv?=
 =?utf-8?Q?B/ec46D0tzCNwNTG+mL6LoJGtj0ImZFLbL5g9x2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6d540d25-ae41-46e4-3b0c-08d979a870c3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 06:57:40.3984
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iyjDbvuUeFe+zmsxDY6Bjmx9U4Fxr1mm6FX1geU0AXDPEsLXdEyuLzukXYkNl0lj1y3i8Znz1aHpzZk9pw8VmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2607

On 17.09.2021 04:48, Penny Zheng wrote:
> Commit 540a637c3410780b519fc055f432afe271f642f8 defines a new
> helper mark_page_free to extract common codes, but it broke the
> local variable "tainted", revealed by Coverity ID 1491872.

Instead of mentioning the ID here, please ...

> This patch let mark_page_free() return boolean value of variable
> "tainted" and rename local variable "tainted" to "pg_tainted"
> to tell the difference from the global variable of the same name.

add a tag-like instance here, as we do elsewhere:

Coverity ID: 1491872

> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>  xen/common/page_alloc.c | 15 ++++++++++-----
>  1 file changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index b9441cb06f..c6f48f7a97 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1380,8 +1380,10 @@ bool scrub_free_pages(void)
>      return node_to_scrub(false) != NUMA_NO_NODE;
>  }
>  
> -static void mark_page_free(struct page_info *pg, mfn_t mfn)
> +static bool mark_page_free(struct page_info *pg, mfn_t mfn)
>  {
> +    unsigned int pg_tainted = 0;

bool please, considering ...

> @@ -1405,7 +1407,7 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
>      case PGC_state_offlining:
>          pg->count_info = (pg->count_info & PGC_broken) |
>                           PGC_state_offlined;
> -        tainted = 1;
> +        pg_tainted = 1;
>          break;
>  
>      default:
> @@ -1425,6 +1427,8 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
>      /* This page is not a guest frame any more. */
>      page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
>      set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
> +
> +    return pg_tainted;
>  }

... this. Also both here and ...

> @@ -1433,7 +1437,7 @@ static void free_heap_pages(
>  {
>      unsigned long mask;
>      mfn_t mfn = page_to_mfn(pg);
> -    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), tainted = 0;
> +    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), pg_tainted = 0;

... here I don't see why you stick to using "tainted" in the name
when the goal is to ...

> @@ -1517,7 +1522,7 @@ static void free_heap_pages(
>  
>      page_list_add_scrub(pg, node, zone, order, pg->u.free.first_dirty);
>  
> -    if ( tainted )
> +    if ( pg_tainted )
>          reserve_offlined_page(pg);

... control the call to this function: "offline" or "offlined"
would seem quite a bit more to the point, getting us further away
from the global meaning of "tainted".

Also please follow patch submission rules: To the list, with (all)
maintainers Cc-ed.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:24:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:24:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.188999.338447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Eg-0005OM-Us; Fri, 17 Sep 2021 07:24:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 188999.338447; Fri, 17 Sep 2021 07:24:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Eg-0005OF-RN; Fri, 17 Sep 2021 07:24:38 +0000
Received: by outflank-mailman (input) for mailman id 188999;
 Fri, 17 Sep 2021 07:24:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+lGq=OH=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mR8Ef-0005O9-2w
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:24:37 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d2abf78-4fc0-4e2f-ad3c-4b7df5273902;
 Fri, 17 Sep 2021 07:24:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1A63B2229C;
 Fri, 17 Sep 2021 07:24:35 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DC7B113A7D;
 Fri, 17 Sep 2021 07:24:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id sHSCM7JCRGG8FAAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 17 Sep 2021 07:24:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d2abf78-4fc0-4e2f-ad3c-4b7df5273902
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631863475; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=bE5NUshZqGoJZC75jbTVmSF6zC94Ejk6ktxVeD+/QPI=;
	b=t+OPqhofgpuu6n7IulctcNx7szquc6WpryAnO3B5vZgJ/d+Aosci9UoDvozF9Ssr4IaM4l
	f0n1bOn3TsIv8uM1gPpIko2518MrAjpqngC1gHiGhqKP7HwBFDsw+LSWnQ42VFsBVTZueX
	ics88Z9Dd292erTQPnBJafYqgQC1ESo=
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
 <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
 <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <35835650-bbbb-7dbd-061b-693f39f9453e@suse.com>
Date: Fri, 17 Sep 2021 09:24:34 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="M2XcnizTokna6hopAL0dWC5lqtlHkp8ZI"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--M2XcnizTokna6hopAL0dWC5lqtlHkp8ZI
Content-Type: multipart/mixed; boundary="5rydOrmp7qY4iIJ4s8W5Quk59Tk3u8ZFr";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <35835650-bbbb-7dbd-061b-693f39f9453e@suse.com>
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
 <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
 <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>
In-Reply-To: <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>

--5rydOrmp7qY4iIJ4s8W5Quk59Tk3u8ZFr
Content-Type: multipart/mixed;
 boundary="------------BA95EA9CD276B930AE413EB4"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------BA95EA9CD276B930AE413EB4
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 08:50, Jan Beulich wrote:
> On 17.09.2021 08:47, Juergen Gross wrote:
>> On 17.09.21 08:40, Jan Beulich wrote:
>>> On 17.09.2021 03:34, Boris Ostrovsky wrote:
>>>>
>>>> On 9/16/21 11:04 AM, Jan Beulich wrote:
>>>>>    {
>>>>>    	const struct desc_ptr *desc =3D this_cpu_ptr(&idt_desc);
>>>>> +	unsigned i, count =3D (desc->size + 1) / sizeof(gate_desc);
>>>>>   =20
>>>>> -	xen_convert_trap_info(desc, traps);
>>>>
>>>>
>>>> Can you instead add a boolean parameter to xen_convert_trap_info() t=
o indicate whether to skip empty entries? That will avoid (almost) duplic=
ating the code.
>>>
>>> I can, sure, but I specifically didn't, as the result is going to be =
less
>>> readable imo. Instead I was considering to fold xen_convert_trap_info=
()
>>> into its only remaining caller. Yet if you're convinced adding the
>>> parameter is the way to do, I will go that route. But please confirm.=

>>
>> I don't think the result will be very hard to read. All you need is th=
e
>> new parameter and extending the if statement in xen_convert_trap_info(=
)
>> to increment out always if no entry is to be skipped.
>=20
> And skip writing the sentinel.

Maybe it would be even better then to let xen_convert_trap_info() return
the number of entries written and to write the sentinel in
xen_load_idt() instead, as this is the only place where it is needed.


Juergen

--------------BA95EA9CD276B930AE413EB4
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------BA95EA9CD276B930AE413EB4--

--5rydOrmp7qY4iIJ4s8W5Quk59Tk3u8ZFr--

--M2XcnizTokna6hopAL0dWC5lqtlHkp8ZI
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFEQrIFAwAAAAAACgkQsN6d1ii/Ey+F
VwgAhC2MWq8hSJIV5LTI9ri7tMI+XMDRGUllT1Z2+ymA5o5GtMO5meoDH5HsfwE4tZPMxNh79S9M
IKf3EydS546e7TC1ZF5RB3u0PFlbZyNL0H0pM2OpuF7jSGOagUt7hMbzF+uG6OYNsNJvgQSc+/Bw
8L31sExVIhUmNJ6yl5Nxelm1ZYMtSBEQfiIy3t9RVop2kmHLxgzfU0xexIQPc/eXDhTvrXLqwj+V
dd/c0yXCOnLD7lqWTXmHFbp9a6/SFoyWqxwvMmzTeqCv6pQX1QtSQwh27Q28SQdIdBwVMy7A3INS
F3dedrJD1Cxz9+p9ET8oJoanBRZGDQwqVCzbO+WZ0g==
=caFE
-----END PGP SIGNATURE-----

--M2XcnizTokna6hopAL0dWC5lqtlHkp8ZI--


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:26:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:26:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189006.338458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8GK-0005yX-9q; Fri, 17 Sep 2021 07:26:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189006.338458; Fri, 17 Sep 2021 07:26:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8GK-0005yQ-6R; Fri, 17 Sep 2021 07:26:20 +0000
Received: by outflank-mailman (input) for mailman id 189006;
 Fri, 17 Sep 2021 07:26:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mR8GI-0005yG-I4
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:26:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8bc0ac10-1788-11ec-b66f-12813bfff9fa;
 Fri, 17 Sep 2021 07:26:17 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-rovLXMCbM8GrtK30tCfadw-1; Fri, 17 Sep 2021 09:26:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2608.eurprd04.prod.outlook.com (2603:10a6:800:4f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 07:26:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 07:26:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0050.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Fri, 17 Sep 2021 07:26:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bc0ac10-1788-11ec-b66f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631863576;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YFBhz6nwdzlrstTFpNbiXuqgmlHYUueogQifd2M0ax4=;
	b=nqR5tI+Bd/oP8WeKpY3VF0CLaj4HqOJkSAPnUDWjw5w7ks5u6w6fIO/65OEbZNGfS+HREh
	l78IlYrM75m9WjGvIsCxj8Dzgsd4xGhpeeytJo0xxGboW1d8lhmEwAMN9DYf0iUMsgn5IJ
	Kg+NUQbqxCVTBwFcHJg7zxmCUeTpebk=
X-MC-Unique: rovLXMCbM8GrtK30tCfadw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EekxNJxJ9SQX8GMNt9/fa/1NHAioyG88UI2hbI4Gxeq8dZlIJvaCj4ifxzKjUTkLTFl09itZyODrABngWdFlzyY5pGOoBXiK5CmEfHnPJObrT0Zeg6eF33H6QU7VA/5zEU0sVF6fblBwERxa9oupz2lPNhK0j9nKEz89zYQsoTrZII0IEkjd6fnV2yLpxTjHNR7MX51uKt4XKWr4TrM1s4xdf24RYXgLya9y8wQEQbfHV4giPBlIYbJpM9aAvu4NIjuBcT1qZ/2WjOUwVTQN7y3eGDjpGKpEog9y/Xj9bT1DxmIn+vMW9/QKFGxlsmclnngChYjSvDDrvW9YhbMzhA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YFBhz6nwdzlrstTFpNbiXuqgmlHYUueogQifd2M0ax4=;
 b=N84SdCnduDCJMc3jNqm1JNxgQMjxMJgA+rsSY0ftDPn9u5aKccicrf2XcZHk6oi7inyXGdsGln2pnVqHXfkX6GUvi/sykMxKU83bkHXCt/SiLu3uKNx62qK+sBwCkRxZZ6PcMFiaPADAjWoKG7dC+tHORPxQCZtZ3giXY+Suv4Bul0gZNA/Y477hm3ww939srAT/j2ZugPXmY3o71+QElsQkNmh1vUMA+N/mM34XYJOBA/Cvr/cWirelIgSiUbNaeiib1ut4OY7j5Ubf0zIMfOhLQUT670uWe9lBN2Zt64zQJZ1LapLmHqVeUpsQolks0x9S7aazoRTMy6swBy8Jig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/mem_sharing: don't lock parent during fork reset
To: Tamas K Lengyel <tamas.lengyel@intel.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <c21708c84c850ff8c976a3934099c54da044e7d9.1631802816.git.tamas.lengyel@intel.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <52b28c4d-1cf1-6c98-e1dc-1e0f7b2f768c@suse.com>
Date: Fri, 17 Sep 2021 09:26:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c21708c84c850ff8c976a3934099c54da044e7d9.1631802816.git.tamas.lengyel@intel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0050.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7b150297-fbec-40c9-0402-08d979ac6da2
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2608:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260853BC50ADF76A78E78C1AB3DD9@VI1PR0401MB2608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lhoNbeBRHbqWdI9IUM3+jlv58pzvAVqwkGJ8s7aruERNV/80dDTxWIdSmx3PGNcUZbXRhNg225ryLA1YvaqOcwmdfVcP4ttKcuJGw42jAspnF8TpziKfczaa5inoN0hujwM5wA08Qs8OA0sKLC/MBUHslWTAOMqSS+Nra9n04v8H0ZXsfKCpv14d8QOkGxO4MkQq9IYpGC3gjlfWSJYLrKj0u7CEEoI2ibdTALWX9dpZu7FYcvFAwupp6Lrb0QHFA/aTF922W6nfrTrhXLQUMLIPPkI0zWj0z+mpi0wQI6NSn53Wx/fCBVepkxsncFfQQ2dcrUxtelA7DnBroCsBjWHFHnUADc6H6IjkGWYApKUp2FoCtffu+DyIbKrhn+WJmMHWlp6euEsVSTWktGKcmNk2UtBgDBO6SCBxnmYDmRYGziUl/4L8+zfqWKo4yx4gH9vEA3DeNsdmPyN9N5N1h7Lcluc+oLZs5kqOdzyflmtCaLAyp86cuJEQecVoqjezSQQrEG6JS2j2q9MZkTCAwseBDWrAkfxbfwIbe17z8pEXH0MtMpHLH5pasOVYdKu4Re0+T4OeFc+amLp6YpovyGtIWlX0rrNL43eZNT6gKsgOjmC8zmrtTj9Hv/+y3OJNHDFt+ppRJ1eMZDFh1VpUaVlXBO79rhAeZjjVEuDLYD96/fB1y52UqwbPxsvoygRbVG91eft3oepjxbEFoe/yCJJ79fJS6c6Hpzu62GRdtiA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(2616005)(83380400001)(31696002)(38100700002)(6916009)(86362001)(31686004)(16576012)(8676002)(66556008)(66476007)(66946007)(316002)(8936002)(5660300002)(54906003)(26005)(186003)(2906002)(508600001)(6486002)(4326008)(53546011)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?V1Z6MmlpTUJQcTZoU3JMajhsN2hoazM1QnJ1WHdvK2duMkxubU5ORFE5YTA5?=
 =?utf-8?B?RGhLQlNmYk1RVERBdmpRQU9YUHM4ZDlaazYxNTFyVEhCc3Bha2EzbEpVTG5u?=
 =?utf-8?B?bTZKUFhqMit6d2xLcTdaRUEvVzh0VTBBZktZNVRFQ2tGU0FINlI2cldnOEZV?=
 =?utf-8?B?bEYra0lIZkJXeXZKbGRDc3UvVUhLQy9xdVRacTJRUHlISFZIMlJHOUV6clNM?=
 =?utf-8?B?M08xNUdPbWE0ajZQVDRlc2tJM2ZGSWZwZVM3QzlscDRKUVZXVGJOQXZwS1N5?=
 =?utf-8?B?SU53dzd5OE9WeHBVQWRaRFpaSVVaY2kzWFFXOVZGdzY4Q0tRUWxORnE5clZ6?=
 =?utf-8?B?UFRGWXFlQkJ0TW1nZFpWVmRXaURic2dhMms4ODc4VTRKSDhDSmVEdFJHUGlN?=
 =?utf-8?B?YWY3aHVuS0YrdE9qRCtvUDRnMVo4ODlDWmlwWXhIVkVwZ0RYbzBSOWRSYkNx?=
 =?utf-8?B?bC8xeHppVFVPbHJPKzB4WXRzbDYzOUhnSERFS09uak5VOVAzOTZrQTlTNlJs?=
 =?utf-8?B?Qmh2V1FyWWxjczZhQlcvZ3I3YVc2OUd1RjFlRlIvV0pLNXJzTWJaa01MekNv?=
 =?utf-8?B?b28ySitwenRmQ1BMOGNmcUg0ODhURTJucGduem03Y2RISm5Wa0psZ3RZWDNx?=
 =?utf-8?B?TU05aThWNlBucWM1Zk1LS0tnMHZIUERxdUo5MHJ6VXpBVTd1b1JZeGNUUXZk?=
 =?utf-8?B?UXhVeXhSdXIvMWdLU3hSaGJodnFuS3N4SFR6K1E3N0hVMEF4cmVTdlB1ZG9w?=
 =?utf-8?B?U25xYWd1RFQrZzIzWVJwcHA4V0NOeFFnaU1NSTAxbkU1ZkNGbE04RWtLZWZl?=
 =?utf-8?B?WTNCNlNEcGFNMmhiTFlZMEhDYlFJa0wxejZybTVxLzhYd25OTnkxb3dSMUQ0?=
 =?utf-8?B?TVlqbnVDUVlFbGM1ZmQxdVFyakpWNFFGbkhORHUzaXBLTkMvQ3ZjWGd1MmdB?=
 =?utf-8?B?bzNVKzlZYmlxNVVFdFRzZkI2V0w4Z3R4eW9OaGpVZ245KzdDaTZYeU11ekVC?=
 =?utf-8?B?QjJVWFlvOHE1SmxWNmhYWmFZOEZMZ3Q1UUNrbENqUnFvbHRNMjZpWVFoY1lM?=
 =?utf-8?B?VThzT1BwMkxOYzVqOGRVYWhRQ0NUb2g2a0FJTjlQTi9ybGIvSkRKWkhNanFt?=
 =?utf-8?B?aEFOeXpsK29lbXpYUkF6MWFlaWxtODlBRHlwUW5yOTh6RGovQWg4aTFQRTRV?=
 =?utf-8?B?Uzk3Mi9WTWI5TCswbGpFRzVUWFNzeHJuQmxjTEJ4elZiVjQ5YWZPSjl6Y0RB?=
 =?utf-8?B?ZVlhVkUzYktWRFFhN0JUWTV0MGo1d3N6endHcFJZSWp2T3B5OEdSQWNHS3lw?=
 =?utf-8?B?QWJ1ZHpmWGlWWjY0bldIM29UbmovMUMycm9YQWZGNmhoeCtuOWdMeVh0ZWZZ?=
 =?utf-8?B?MTlkdDJsaE9mcFp6RjVrRHMvb0YxOVZ0M3RMLzNNM1hpOE5XWFlSZ1FzWW1N?=
 =?utf-8?B?dHArQTdDcHZIYlROQi93ME95ems1b1ZxN2NvN3gvY1BQcjZsQXg5MW1jUjQx?=
 =?utf-8?B?TGJBaFhzbGJveUZyRGdZQ2taNktPZVJBUzB1Y2I4T0NMM2xYVDJPNmpUUjRT?=
 =?utf-8?B?Rks1Sm5ZVG93TWlNRmZodkFmQ1d3RDRNZ0xKeDhGa202Mmo2SkVlbVNlUkhs?=
 =?utf-8?B?T295Mm42Y0dTN0RWSVVwcTZSRE5jcitWamF5aEYxbitRbFkzZEV0Y3FDQWhr?=
 =?utf-8?B?b205bUxjL1BEQWdIR3lyUWFmd1lRSkJsWTAwWXlNVnJQcUw3cUkxditsNDVt?=
 =?utf-8?Q?MtW9e9Rt5ZW43M9kqVZGZIQQ9MQtjDKRUIWGB7h?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b150297-fbec-40c9-0402-08d979ac6da2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 07:26:13.1375
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KoCoWMBA/jKtjtDDuKZ1uXUa9LE8dCMxvU3IAU1tUwgFyOnbm6zwlkdI0V1ABAxXe8ivKYWQOlhk4MRgbrzMTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2608

On 16.09.2021 17:04, Tamas K Lengyel wrote:
> During fork reset operation the parent domain doesn't need to be gathered using
> rcu_lock_live_remote_domain_by_id as the fork reset doesn't modify anything on
> the parent. The parent is also guaranteed to be paused while forks are active.
> This patch reduces lock contention when performing resets in parallel.

I'm a little in trouble following you here: RCU locks aren't really
locks in that sense, so "lock contention" seems misleading to me. I
can see that rcu_lock_domain_by_id()'s loop is extra overhead.

Furthermore - does the parent being paused really mean the parent
can't go away behind the back of the fork reset? In fork() I see

    if ( rc && rc != -ERESTART )
    {
        domain_unpause(d);
        put_domain(d);
        cd->parent = NULL;
    }

i.e. the ref gets dropped before the parent pointer gets cleared. If
the parent having a reference kept was indeed properly guaranteed, I
agree the code change itself is fine.

(The sequence looks correct at the other put_domain() site [dealing
with the success case of fork(), when the reference gets retained]
in domain_relinquish_resources().)

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:28:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:28:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189016.338472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Ik-0006qz-RJ; Fri, 17 Sep 2021 07:28:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189016.338472; Fri, 17 Sep 2021 07:28:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Ik-0006qs-MH; Fri, 17 Sep 2021 07:28:50 +0000
Received: by outflank-mailman (input) for mailman id 189016;
 Fri, 17 Sep 2021 07:28:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jmcU=OH=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mR8Ij-0006qm-Ut
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:28:50 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 13df9831-2454-42ab-ad77-d0e9ebddb375;
 Fri, 17 Sep 2021 07:28:47 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id c8so29373086lfi.3
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 00:28:47 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id x15sm459360lfn.108.2021.09.17.00.28.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 Sep 2021 00:28:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13df9831-2454-42ab-ad77-d0e9ebddb375
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=qgbI4RF4QtWHUfQV5/L0znctien73LjWZZ67XaNeI5w=;
        b=E7VObHj9RQ7Qn+awOIdOrYGCihJnVKu8gfFutbDmfsFEgotq9R9Mfd/4JnfTWJ+gdv
         disBkphpHhXzCFyXyp6FFWVVsjyu397TbkrWT39q/u6pUjWxZx2jpN0MT2ZiSu1AWjmN
         jmW+b+gmPOeXyK26EJb9eGl0P04pa0Q5bWqi4seMF8+iTop2t8BL+xb0uGT6OO863VF7
         tPPEvEtOYRk5NcE8kBtHaJnVbj25A1Ozo0/Yw2LgELLQ48IRU4nd3Smtpgj/7QA3ensn
         JCDh0MCDpZrecT2rveX0n4nQU6QplARo6clNJwXKkTii3IifWCWZr7PGFIDYxeN1ECit
         +30A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=qgbI4RF4QtWHUfQV5/L0znctien73LjWZZ67XaNeI5w=;
        b=xqL/aLUC7M+0334hXHzC7xmPbg0W+8pqnaJQtIMEMK+gwCmJf9KaY2lDcq9QIMTS9X
         WtC4jVjO+xM5WmR7h0WUVEJSu30Gz+cWpFIpLeKivdAUk1JJNBuBHZj5wFiYjVNtRtx+
         uI+RiNj6FgynvAkRG22Us1qNc8I5NmiLdeU/5MzAZW/5l+eV2oV7KPggOMwCmdeVN69b
         wd8qRJKcXGlCb8m9Rs+qzVVIRtYmXFbHc9tbI+J6dVWIZ+4Nnx9m0enYEvwHI9VIx7Ur
         SFfz7pPLreZVYoIZOaRQ/pdezYXr2Oy1j30TESGRzqvMyADt/glvRSjol8l8p+QaEG9S
         fwOA==
X-Gm-Message-State: AOAM533hi/i6FLrJ3Ug0KC7OJyngBi91/JV02AaMgVmq9Q4WM1YKgVx0
	0R6oYlzr/n0QIp6+ntX6trA=
X-Google-Smtp-Source: ABdhPJzpSDNnHkqmEYGG+hBHZbvMQT3mSwCXoZDsE1gMe5DyB0xA6daB9AYGDVGUT62c5wZuxrSZ8Q==
X-Received: by 2002:a05:6512:114a:: with SMTP id m10mr3195414lfg.382.1631863726431;
        Fri, 17 Sep 2021 00:28:46 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
 <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
 <alpine.DEB.2.21.2109151352100.21985@sstabellini-ThinkPad-T480s>
 <b349e43a-91d9-16ba-57c6-34e790b8b31c@gmail.com>
 <alpine.DEB.2.21.2109161423020.21985@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <9579f367-d7f8-b4e7-c18f-af871ee0102d@gmail.com>
Date: Fri, 17 Sep 2021 10:28:45 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109161423020.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 17.09.21 00:30, Stefano Stabellini wrote:

Hi Stefano

> On Thu, 16 Sep 2021, Oleksandr wrote:
>>> On Wed, 15 Sep 2021, Oleksandr wrote:
>>>>> On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
>>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>>
>>>>>> The extended region (safe range) is a region of guest physical
>>>>>> address space which is unused and could be safely used to create
>>>>>> grant/foreign mappings instead of wasting real RAM pages from
>>>>>> the domain memory for establishing these mappings.
>>>>>>
>>>>>> The extended regions are chosen at the domain creation time and
>>>>>> advertised to it via "reg" property under hypervisor node in
>>>>>> the guest device-tree. As region 0 is reserved for grant table
>>>>>> space (always present), the indexes for extended regions are 1...N.
>>>>>> If extended regions could not be allocated for some reason,
>>>>>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>>>>>
>>>>>> Please note the following limitations:
>>>>>> - The extended region feature is only supported for 64-bit domain.
>>>>>> - The ACPI case is not covered.
>>>>>>
>>>>>> ***
>>>>>>
>>>>>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>>>>>> the algorithm to choose extended regions for it is different
>>>>>> in comparison with the algorithm for non-direct mapped DomU.
>>>>>> What is more, that extended regions should be chosen differently
>>>>>> whether IOMMU is enabled or not.
>>>>>>
>>>>>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>>>>>> holes found in host device-tree if otherwise. Make sure that
>>>>>> extended regions are 2MB-aligned and located within maximum possible
>>>>>> addressable physical memory range. The maximum number of extended
>>>>>> regions is 128.
>>>>>>
>>>>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>> ---
>>>>>> Changes since RFC:
>>>>>>       - update patch description
>>>>>>       - drop uneeded "extended-region" DT property
>>>>>> ---
>>>>>>
>>>>>>     xen/arch/arm/domain_build.c | 226
>>>>>> +++++++++++++++++++++++++++++++++++++++++++-
>>>>>>     1 file changed, 224 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>>>> index 206038d..070ec27 100644
>>>>>> --- a/xen/arch/arm/domain_build.c
>>>>>> +++ b/xen/arch/arm/domain_build.c
>>>>>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct
>>>>>> domain *d,
>>>>>>         return res;
>>>>>>     }
>>>>>>     +static int __init add_ext_regions(unsigned long s, unsigned long
>>>>>> e,
>>>>>> void *data)
>>>>>> +{
>>>>>> +    struct meminfo *ext_regions = data;
>>>>>> +    paddr_t start, size;
>>>>>> +
>>>>>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>>>>>> +        return 0;
>>>>>> +
>>>>>> +    /* Both start and size of the extended region should be 2MB
>>>>>> aligned
>>>>>> */
>>>>>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>>>>>> +    if ( start > e )
>>>>>> +        return 0;
>>>>>> +
>>>>>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>>>>>> +    if ( !size )
>>>>>> +        return 0;
>>>>> Can't you align size as well?
>>>>>
>>>>>      size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
>>>> I am sorry, I don't entirely get what you really meant here. We get both
>>>> start
>>>> and size 2MB-aligned by the calculations above
>>>> (when calculating an alignment, we need to make sure that "start_passed <=
>>>> start_aligned && size_aligned <= size_passed").
>>>> If I add the proposing string after, I will reduce the already aligned
>>>> size by
>>>> 2MB.
>>>> If I replace the size calculation with the following, I will get the
>>>> reduced
>>>> size even if the passed region is initially 2MB-aligned, so doesn't need
>>>> to be
>>>> adjusted.
>>>> size = e - s + 1;
>>>> size = (size - (SZ_2M - 1)) & ~(SZ_2M - 1);
>>> Sorry I misread your original code, I think it was working as intended
>>> except for the "+1". I think it should be:
>>>
>>>     size = (e - start) & ~(SZ_2M - 1);
>> But why without "+1"? Isn't "e" here the *last address* of passed range?
>> Without "+1" I get non entirely correct calculations, last valid 2MB is
>> missed.
> You are right: the "+1" should not be needed if this was "end",
> following the normal definition of end. However, add_ext_regions is
> called by rangeset_report_ranges, so end here is not actually "end", it
> is "end-1".

Yes.


>
> For clarity, I would ask you to rewrite it like this:
>
> /*
>   * e is actually "end-1" because it is called by rangeset functions
>   * which are inclusive of the last address.
>   */
> e += 1;
> size = (e - start) & ~(SZ_2M - 1);

Ack, will do.


>
>
>> [snip]
>> (XEN) Extended region 14: 0x580000000->0x5ffe00000
>> (XEN) Extended region 15: 0x680000000->0x6ffe00000
>> (XEN) Extended region 16: 0x780000000->0xffffe00000
>>
>> But should get:
>>
>> [snip]
>> (XEN) Extended region 15: 0x580000000->0x600000000
>> (XEN) Extended region 16: 0x680000000->0x700000000
>> (XEN) Extended region 17: 0x780000000->0x10000000000
>>
>> Let's consider how a hole between (for example) RAM bank 1 and bank 2 is
>> calculated:
>> (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1 with size
>> 0x80000000
>> (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2 with size
>> 0x80000000
>> So the hole size should also be 0x80000000.
>> If we pass these RAM banks to rangeset_remove_range() one by one:
>> 1: s = 0x500000000 e = 0x57FFFFFFF
>> 2. s = 0x600000000 e = 0x67FFFFFFF
>> we get s = 0x580000000 e = 0x5FFFFFFFF in add_ext_regions(), where "e" is the
>> last address of the hole (not the first address out of the hole), so I think,
>> that for proper size calculation we need to add 1 to "e - s". Or I really
>> missed something?
>>
>>
>>>>>> + */
>>>>>> +#define EXT_REGION_START   0x40000000ULL
>>>>>> +#define EXT_REGION_END     0x80003fffffffULL
>>>>>> +
>>>>>> +static int __init find_unallocated_memory(const struct kernel_info
>>>>>> *kinfo,
>>>>>> +                                          struct meminfo
>>>>>> *ext_regions)
>>>>>> +{
>>>>>> +    const struct meminfo *assign_mem = &kinfo->mem;
>>>>>> +    struct rangeset *unalloc_mem;
>>>>>> +    paddr_t start, end;
>>>>>> +    unsigned int i;
>>>>>> +    int res;
>>>>>> +
>>>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>>>> +
>>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>>> +    if ( !unalloc_mem )
>>>>>> +        return -ENOMEM;
>>>>>> +
>>>>>> +    /* Start with all available RAM */
>>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size
>>>>>> - 1;
>>>>> Is the -1 needed? Isn't it going to screw up the size calculation later?
>>>> I thought, it was needed. The calculation seems correct.
>>> I think that normally for an example MMIO region:
>>>
>>> start = 0x48000000
>>> size  = 0x40000000
>>> end   = 0x88000000
>>>
>>> So end = start + size and points to the first address out of the range.
>>> In other words, 0x88000000 doesn't actually belong to the MMIO region in
>>> the example.
>>>
>>> But here you are passing addresses to rangeset_add_range and other
>>> rangeset functions and I think rangeset takes *inclusive* addresses as
>>> input. So you need to pass start and end-1 because end-1 is the last
>>> address of the MMIO region.
>>>
>>> In fact you can see for instance in map_range_to_domain:
>>>
>>>           res = iomem_permit_access(d, paddr_to_pfn(addr),
>>>                   paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
>>>
>>> Where iomem_permit_access is based on rangeset. So for clarity, I would
>>> do:
>>>
>>> start = assign_mem->bank[i].start;
>>> end = assign_mem->bank[i].start + assign_mem->bank[i].size;
>>> res = rangeset_remove_range(unalloc_mem, start, end - 1);
>>>
>>> So that we don't get confused on the meaning of "end" which everywhere
>>> else means the first address not in range.
>> I got your point, I will update the code if it much cleaner.
>>
>>
>>>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to add:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = assign_mem->bank[i].start;
>>>>>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size -
>>>>>> 1;
>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove reserved-memory regions */
>>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove grant table region */
>>>>>> +    start = kinfo->gnttab_start;
>>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +    if ( res )
>>>>>> +    {
>>>>>> +        printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +               start, end);
>>>>>> +        goto out;
>>>>>> +    }
>>>>>> +
>>>>>> +    start = EXT_REGION_START;
>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>>> +                                 add_ext_regions, ext_regions);
>>>>>> +    if ( res )
>>>>>> +        ext_regions->nr_banks = 0;
>>>>>> +    else if ( !ext_regions->nr_banks )
>>>>>> +        res = -ENOENT;
>>>>>> +
>>>>>> +out:
>>>>>> +    rangeset_destroy(unalloc_mem);
>>>>>> +
>>>>>> +    return res;
>>>>>> +}
>>>>>> +
>>>>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>>>>> +                                    struct meminfo *ext_regions)
>>>>>> +{
>>>>>> +    struct dt_device_node *np;
>>>>>> +    struct rangeset *mem_holes;
>>>>>> +    paddr_t start, end;
>>>>>> +    unsigned int i;
>>>>>> +    int res;
>>>>>> +
>>>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>>>> +
>>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>>> +    if ( !mem_holes )
>>>>>> +        return -ENOMEM;
>>>>>> +
>>>>>> +    /* Start with maximum possible addressable physical memory range
>>>>>> */
>>>>>> +    start = EXT_REGION_START;
>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>>> +    if ( res )
>>>>>> +    {
>>>>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +               start, end);
>>>>>> +        goto out;
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM,
>>>>>> etc) */
>>>>>> +    dt_for_each_device_node( dt_host, np )
>>>>> Don't you need something like device_tree_for_each_node ?
>>>>> dt_for_each_device_node won't go down any deeper in the tree?
>>>> Thank you for pointing this out, I will investigate and update the patch.
>>>>
>>>>
>>>>> Alternatively, maybe we could simply record the highest possible address
>>>>> of any memory/device/anything as we scan the device tree with
>>>>> handle_node. Then we can use that as the starting point here.
>>>> I also don't like the idea to scan the DT much, but I failed to find an
>>>> effective solution how to avoid that.
>>>> Yes, we can record the highest possible address, but I am afraid, I didn't
>>>> entirely get a suggestion. Is the suggestion to provide a single region
>>>> starting from highest possible address + 1 and up to the EXT_REGION_END
>>>> suitably aligned? Could you please clarify?
>>> Yes, that is what I was suggesting as a possible alternative: start from
>>> the highest possible address in DT + 1 and up to the EXT_REGION_END
>>> suitably aligned. But that wouldn't solve the <4GB issue.
>>>
>>>>>> +                goto out;
>>>>>> +            }
>>>>>> +
>>>>>> +            start = addr & PAGE_MASK;
>>>>>> +            end = PAGE_ALIGN(addr + size) - 1;
>>>>>> +            res = rangeset_remove_range(mem_holes, start, end);
>>>>>> +            if ( res )
>>>>>> +            {
>>>>>> +                printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                       start, end);
>>>>>> +                goto out;
>>>>>> +            }
>>>>>> +        }
>>>>>> +    }
>>>>> As is, it will result in a myriad of small ranges which is unuseful and
>>>>> slow to parse. I suggest to simplify it by removing a larger region than
>>>>> strictly necessary. For instance, you could remove a 1GB-aligned and
>>>>> 1GB-multiple region for each range. That way, you are going to get fewer
>>>>> large free ranges instance of many small ones which we don't need.
>>>> I agree with you that a lot of small ranges increase the bookkeeping in
>>>> Dom0
>>>> and there is also a theoretical (?) possibility
>>>> that small ranges occupy all space we provide for extended regions
>>>> (NR_MEM_BANKS)...
>>>> But, let's consider my setup as an example again, but when the IOMMU is
>>>> enabled for Dom0 ("holes found in DT").
>>>>
>>>> --- The RAM configuration is the same:
>>>>
>>>> (XEN) RAM: 0000000048000000 - 00000000bfffffff <--- RAM bank 0
>>>> (XEN) RAM: 0000000500000000 - 000000057fffffff <--- RAM bank 1
>>>> (XEN) RAM: 0000000600000000 - 000000067fffffff <--- RAM bank 2
>>>> (XEN) RAM: 0000000700000000 - 000000077fffffff <--- RAM bank 3
>>>>
>>>> --- There are a lot of various platform devices with reg property
>>>> described in
>>>> DT, I will probably not post all IO ranges here, just say that mostly all
>>>> of
>>>> them to be mapped at 0xE0000000-0xFFFFFFFF.
>>>>
>>>> --- As we only pick up ranges with size >= 2MB, the calculated extended
>>>> regions are (based on 40-bit IPA):
>>>>
>>>> (XEN) Extended region 0: 0x40000000->0x47e00000
>>>> (XEN) Extended region 1: 0xc0000000->0xe6000000
>>>> (XEN) Extended region 2: 0xe7000000->0xe7200000
>>>> (XEN) Extended region 3: 0xe7400000->0xe7600000
>>>> (XEN) Extended region 4: 0xe7800000->0xec000000
>>>> (XEN) Extended region 5: 0xec200000->0xec400000
>>>> (XEN) Extended region 6: 0xec800000->0xee000000
>>>> (XEN) Extended region 7: 0xee600000->0xee800000
>>>> (XEN) Extended region 8: 0xeea00000->0xf1000000
>>>> (XEN) Extended region 9: 0xf1200000->0xfd000000
>>>> (XEN) Extended region 10: 0xfd200000->0xfd800000
>>>> (XEN) Extended region 11: 0xfda00000->0xfe000000
>>>> (XEN) Extended region 12: 0xfe200000->0xfe600000
>>>> (XEN) Extended region 13: 0xfec00000->0xff800000
>>>> (XEN) Extended region 14: 0x100000000->0x500000000
>>>> (XEN) Extended region 15: 0x580000000->0x600000000
>>>> (XEN) Extended region 16: 0x680000000->0x700000000
>>>> (XEN) Extended region 17: 0x780000000->0x10000000000
>>>>
>>>> So, if I *correctly* understood your idea about removing 1GB-aligned
>>>> 1GB-multiple region for each range we would get the following:
>>>>
>>>> (XEN) Extended region 0: 0x100000000->0x500000000
>>>> (XEN) Extended region 1: 0x580000000->0x600000000
>>>> (XEN) Extended region 2: 0x680000000->0x700000000
>>>> (XEN) Extended region 3: 0x780000000->0x10000000000
>>>>
>>>> As you can see there are no extended regions below 4GB at all. I assume,
>>>> it
>>>> would be good to provide them for 1:1 mapped Dom0 (for 32-bit DMA
>>>> devices?)
>>>> What else worries me is that IPA size could be 36 or even 32. So, I am
>>>> afraid,
>>>> we might even fail to find extended regions above 4GB.
>>>>
>>>>
>>>> I think, if 2MB is considered small enough to bother with, probably we
>>>> should
>>>> go with something in between (16MB, 32MB, 64MB).
>>>> For example, we can take into the account ranges with size >= 16MB:
>>>>
>>>> (XEN) Extended region 0: 0x40000000->0x47e00000
>>>> (XEN) Extended region 1: 0xc0000000->0xe6000000
>>>> (XEN) Extended region 2: 0xe7800000->0xec000000
>>>> (XEN) Extended region 3: 0xec800000->0xee000000
>>>> (XEN) Extended region 4: 0xeea00000->0xf1000000
>>>> (XEN) Extended region 5: 0xf1200000->0xfd000000
>>>> (XEN) Extended region 6: 0x100000000->0x500000000
>>>> (XEN) Extended region 7: 0x580000000->0x600000000
>>>> (XEN) Extended region 8: 0x680000000->0x700000000
>>>> (XEN) Extended region 9: 0x780000000->0x10000000000
>>>>
>>>> Any thoughts?
>>> Yeah maybe an intermediate value would be best. I'd go with 64MB.
>> I completely agree.
>>
>> So what I got on my setup with that value.
>>
>> 1. IOMMU is enabled for Dom0:
>>
>> (XEN) Extended region 0: 0x40000000->0x47e00000
>> (XEN) Extended region 1: 0xc0000000->0xe6000000
>> (XEN) Extended region 2: 0xe7800000->0xec000000
>> (XEN) Extended region 3: 0xf1200000->0xfd000000
>> (XEN) Extended region 4: 0x100000000->0x500000000
>> (XEN) Extended region 5: 0x580000000->0x600000000
>> (XEN) Extended region 6: 0x680000000->0x700000000
>> (XEN) Extended region 7: 0x780000000->0x10000000000
>>
>> 2. IOMMU is disabled for Dom0:
>>
>> (XEN) Extended region 0: 0x48000000->0x54000000
>> (XEN) Extended region 1: 0x57000000->0x60000000
>> (XEN) Extended region 2: 0x70000000->0x78000000
>> (XEN) Extended region 3: 0x78200000->0xc0000000
>> (XEN) Extended region 4: 0x500000000->0x580000000
>> (XEN) Extended region 5: 0x600000000->0x680000000
>> (XEN) Extended region 6: 0x700000000->0x780000000
>>
>> Which is not bad.
> Yeah I think that's good.

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:29:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:29:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189024.338483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Jd-0007UV-9n; Fri, 17 Sep 2021 07:29:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189024.338483; Fri, 17 Sep 2021 07:29:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Jd-0007UO-5Z; Fri, 17 Sep 2021 07:29:45 +0000
Received: by outflank-mailman (input) for mailman id 189024;
 Fri, 17 Sep 2021 07:29:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=82tu=OH=infradead.org=peterz@srs-us1.protection.inumbo.net>)
 id 1mR8Jb-0007UC-9F
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:29:43 +0000
Received: from desiato.infradead.org (unknown
 [2001:8b0:10b:1:d65d:64ff:fe57:4e05])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 73375631-c64d-4f40-a014-5d4236266216;
 Fri, 17 Sep 2021 07:29:40 +0000 (UTC)
Received: from j217100.upc-j.chello.nl ([24.132.217.100]
 helo=noisy.programming.kicks-ass.net)
 by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux))
 id 1mR8Ig-003p42-0g; Fri, 17 Sep 2021 07:28:46 +0000
Received: from hirez.programming.kicks-ass.net
 (hirez.programming.kicks-ass.net [192.168.1.225])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (Client did not present a certificate)
 by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 64E5430031A;
 Fri, 17 Sep 2021 09:28:42 +0200 (CEST)
Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000)
 id EBC3F2B27F98D; Fri, 17 Sep 2021 09:28:41 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73375631-c64d-4f40-a014-5d4236266216
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=17mkqfuKdfnPg9YSAeGWHc/O4GLOFEnsaFTJ4hUb1Qc=; b=R175PzEhjMPDTuiig9T+/8FvBN
	wCwITbQjI86keC2UZcKL3oHklcXqScPxlhfsprjzgtQHXNUwlN8OYqdLQ7ztxGmPqLPGGL1HvK5TW
	8un1CwadAtDZMXP8kHm7KJ60CKQlzq6kBsXmSOCidcFaqUG49FGQ/Q91WQSCSzrudnHUpN1n/dtAK
	3oCiHTPea1eYl35kiZcqB3afHwTSqwkSRYpje8CyTQKsMtTXOUjDfhQXOeCpbGDLw3jXUuiLQwU+o
	fpmbPjE5lGD64IJe7KimQfKhdgCV7/TWkKj798HvtN0fT0soP/DzPI91f4RdvEreoDEx02Cp2qq0O
	NM1LgiIg==;
Date: Fri, 17 Sep 2021 09:28:41 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Sean Christopherson <seanjc@google.com>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
Message-ID: <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
 <YUO5J/jTMa2KGbsq@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YUO5J/jTMa2KGbsq@google.com>

On Thu, Sep 16, 2021 at 09:37:43PM +0000, Sean Christopherson wrote:
> On Sat, Aug 28, 2021, Peter Zijlstra wrote:

> Argh, sorry, I somehow managed to miss all of your replies.  I'll get back to
> this series next week.  Thanks for the quick response!
> 
> > Lets keep the whole intel_pt crud inside x86...
> 
> In theory, I like the idea of burying intel_pt inside x86 (and even in
> Intel+VMX code for the most part), but the actual implementation is a
> bit gross.  Because of the whole "KVM can be a module" thing,

ARGH!! we should really fix that. I've heard other archs have made much
better choices here.

> either
> the static call and __static_call_return0 would need to be exported,
> or a new register/unregister pair would have to be exported.

So I don't mind exporting __static_call_return0, but exporting a raw
static_call is much like exporting a function pointer :/

> The unregister path would also need its own synchronize_rcu().  In general, I
> don't love duplicating the logic, but it's not the end of the world.
> 
> Either way works for me.  Paolo or Peter, do either of you have a preference?

Can we de-feature kvm as a module and only have this PT functionality
when built-in? :-)


> > ---
> > Index: linux-2.6/arch/x86/events/core.c
> > ===================================================================
> > --- linux-2.6.orig/arch/x86/events/core.c
> > +++ linux-2.6/arch/x86/events/core.c
> > @@ -92,7 +92,7 @@ DEFINE_STATIC_CALL_RET0(x86_pmu_guest_ge
> >  
> >  DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
> >  DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> > -DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, *(perf_guest_cbs->handle_intel_pt_intr));
> > +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, unsigned int (*)(void));
> 
> FWIW, the param needs to be a raw function, not a function pointer. 

Yeah, I keep making that mistake.. and I wrote the bloody thing :/

I have a 'fix' for that, but I need to finish that and also flag-day
change :-(

  https://lkml.kernel.org/r/YS+0eIeAJsRRArk4@hirez.programming.kicks-ass.net


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:29:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:29:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189025.338494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Jh-0007mK-Ip; Fri, 17 Sep 2021 07:29:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189025.338494; Fri, 17 Sep 2021 07:29:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Jh-0007mB-E3; Fri, 17 Sep 2021 07:29:49 +0000
Received: by outflank-mailman (input) for mailman id 189025;
 Fri, 17 Sep 2021 07:29:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eQru=OH=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mR8Jg-0007UC-1o
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:29:48 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.78]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 569c687d-b668-41af-a710-7b088ddc5ace;
 Fri, 17 Sep 2021 07:29:40 +0000 (UTC)
Received: from PR3P191CA0025.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::30)
 by HE1PR0801MB1929.eurprd08.prod.outlook.com (2603:10a6:3:50::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 07:29:38 +0000
Received: from VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:102:54:cafe::b0) by PR3P191CA0025.outlook.office365.com
 (2603:10a6:102:54::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 07:29:38 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT026.mail.protection.outlook.com (10.152.18.148) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 07:29:38 +0000
Received: ("Tessian outbound 16951d3c485e:v103");
 Fri, 17 Sep 2021 07:29:37 +0000
Received: from 8e2ff51be636.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A760724D-C089-4E6E-A93C-0966CF724A45.1; 
 Fri, 17 Sep 2021 07:29:26 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8e2ff51be636.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 07:29:26 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4359.eurprd08.prod.outlook.com (2603:10a6:20b:b9::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Fri, 17 Sep
 2021 07:29:23 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 07:29:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 569c687d-b668-41af-a710-7b088ddc5ace
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=32hs9lkEXp8u8+aXUbXvjic0i9Ga9cqihqwtBjJ6HRo=;
 b=QhLSgxB9viWgu6mzaW4fpiZBIwmcU8D8Sy4FTzkLF8R7cEMUEdul8OwIZ2UyI0BPGlyKqh+wS2vuSF5dPAZ1LE1ttpuGgh3lS/ug7nobQga3gRLiHuhDMf4qofZvjUwwOOnsIw6NV7KgJWkuJ+ipJAOaerNRmUv2lfbuGo2bTxY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cd03f33da22e8fff
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b/JH8L1Z5jlkvAqEwGQo1K2upYa92R1OGqhst8BLevhp95Lw4tWEaNnmw5p7Ef23at0aZpcBDmjF3xVYiRltqOUFphEf5m77BdWp/bl+/WC8TR8ppcoaSjVLZ5ha4F+w3LhCuoWRntuofXD2KCEmwkeMJqxdUoT4aol/SeLZNbzPugMYpwwQC/JGO3cxjAluyR3nFmeOaZcnRCkuHzMb0LY3+JZopNe+gkb+qTrMABRhVa93R6qr9Nbjt+1L0AoJrHV1i2Yosq0StX6zKAVetkJKgVSI5vP3CVbx5EpZxIgbCj3KRMyKUD0keoYHuXILH89WNB+eH3ywx1HmgTxRmw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=32hs9lkEXp8u8+aXUbXvjic0i9Ga9cqihqwtBjJ6HRo=;
 b=UUZK3M0BXZnheq639Y8zNmZj2ZjwsW+dgSFeplemiLOSyLuOPpClw+ml0pFKyi737L6DWFN1yHMC3Ibr2ENlIRfaWQNuC1zrcXD0clXLE0PIlcP01Crd1Mx0RCorEDqr8QOseVlKrlhFCzYJBuCQxYEKs8oOzm92jRJ5xKrhLHikZDtXF3J0LEwULiScwSwqrx/btTlanM8fyCWfGBB61FoG66a02lmWBO28DB2VVrxkBvh+fGgqNSMy0+4OBCGfyh5ZqZyVA/wil8+P8Lery/9vKbGHNlKI7Mu0V0x+4FmLek+svppU9V1xLxoERLzgmuXob9gmOBPcabP08d2PLw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=32hs9lkEXp8u8+aXUbXvjic0i9Ga9cqihqwtBjJ6HRo=;
 b=QhLSgxB9viWgu6mzaW4fpiZBIwmcU8D8Sy4FTzkLF8R7cEMUEdul8OwIZ2UyI0BPGlyKqh+wS2vuSF5dPAZ1LE1ttpuGgh3lS/ug7nobQga3gRLiHuhDMf4qofZvjUwwOOnsIw6NV7KgJWkuJ+ipJAOaerNRmUv2lfbuGo2bTxY=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Artem Mygaiev <Artem_Mygaiev@epam.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index:
 AQHXoJ57xrO/H5MzpEqKB3+2EFX1LqudvyUAgAXn+wCAAP1GAIAAWBqAgACkJYCAAjRZAIAAFToA
Date: Fri, 17 Sep 2021 07:29:23 +0000
Message-ID: <DA121AFC-E89C-43CB-A7F2-160D579AEC2C@arm.com>
References: <20210903083347.131786-1-andr2000@gmail.com>
 <20210903083347.131786-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109101308490.10523@sstabellini-ThinkPad-T480s>
 <6a4e6a1d-12c9-09bb-442d-ce7f6586ef89@epam.com>
 <20b0a8c0-10dc-7473-3c56-a9baeaa43e62@epam.com>
 <6EA9A8EE-8101-4679-832C-A912819891BC@arm.com>
 <alpine.DEB.2.21.2109151321030.21985@sstabellini-ThinkPad-T480s>
 <e413eea9-b6ae-9cf8-66a6-f02fdcb645bd@epam.com>
In-Reply-To: <e413eea9-b6ae-9cf8-66a6-f02fdcb645bd@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 6122d151-4095-4289-7f4f-08d979ace7f0
x-ms-traffictypediagnostic: AM6PR08MB4359:|HE1PR0801MB1929:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR0801MB1929734195B3D01E1EFCD1D9FCDD9@HE1PR0801MB1929.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 5Yt+B9UpJK46HE73clWGR+L1BPWIBAua3jfXy95ItJrG+QotKJG8M8/+qo5BLypTCLBLFPZLwwrOvVE++b012FhQoWNaccg3XKoQ2QJra7auUZClXOq1XBnbHqvzNjso8B0jjxI2y6ntrgNQjgJ9FKy+2zadDghOoge4uvkCuh5yrtXZmyoRUZ4vX486iKuOYhsIFzoJdXB5O0ReJCrBSuviK4XsLMP5Q4kx1cWB4gjOJL+0dAlDpM6ZsKkLUUJcFa5nDpU6yDOyvAn5UCGIOD01fP+/rw8IoFOcCFzy5VvWNKhs0PZ1MadtG0Q/IFPj9Km8MCIaUC/La0YCAPDMarzmDdTGcQdPvTDreuQN39eS1INf0xa3itLN8rEm/DJyUO3yod91ifgrhbu2oVfSoRvUFGYYptDDhzLF+GeUjBlWcMKWMkZT1g98hBv4ceGw269aMrTiBQlOxZcL4l+YmBTVd1z9QweT5Trc44YbnL4ItgMNSDImapmjkKBlJUqDV+BYLY1v+wzsicu0J4d5b3usf15DnsSnLA4VbgXKewvIzaDehTOONlvhRN9YhCzyx1GFJXoLNBVPSxcZvfMlXZB9lSq86blngfpxqAvvWpM9PDta7z3Nj1uObvyf06ekKm3QbPdMMCD/pDM0H9oGOOOQHjcThtmFRnAq0SIm6DEyuxK2eCniwKK4VryaxjZQweTMmDiRzc6m8aovB9098Ck5lYxCi3XXiKEw0vM6gM0ttArXCXfGiTEyakelLPhI28Pmtfts95PJsX5Paykyk25cU0p3m3lsujyjqwjKf5pnx/8unEgXySe3iJtoqghSQVb0wxRJoOckB3NWvTT2SQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39850400004)(346002)(396003)(376002)(366004)(136003)(33656002)(66446008)(71200400001)(26005)(6916009)(2616005)(478600001)(2906002)(66556008)(83380400001)(966005)(316002)(64756008)(66946007)(91956017)(66476007)(36756003)(76116006)(186003)(8676002)(5660300002)(4326008)(122000001)(38100700002)(38070700005)(86362001)(6506007)(6512007)(8936002)(6486002)(54906003)(53546011)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <A64F14B0DF37A94FB47121445280DEA2@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4359
Original-Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b91a1034-4e33-4e29-65ed-08d979acdf31
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4B60ywczJsrj6mCSycmBDAq+2uBNEb36uQ9aES7G7cTqpWwxzM7LUJe4uiaUIwLYg5VOnseS8XGUOaD7IgqKDdGEOWuyGGz/izTHUmBL0s4qVbShq9rCXIBw1HO+njOfb7imQcwbPr6YZ9e072tJCRgRxPaAO/68nN1taHkn+RszMdPLG7GYECvq7BQqKoYfxrw4qlNU91YLpqhY/rEFLQ68uEe6a+e5L0cflJ8Aeq9pzzSOXl6lqwDIpYAMa4wUoKNngJvDNl9Wztog5cVRUTny9/Y6JNLg7Ej0mZB14OOQ6a/RyXVuNxrqzY890oL/kyl40uzok3M55vdbiu97uPf02D2dzQQoILSoK41INfL8LBcv/dhSb0tsGfEsgJfpwAdbUG5hgN42YJqHP8QEXurY04Ue2N1vFD2Ok/K+1rq0n0B9bUxnOLkfCMvNi/h8OqcIOX8S7johtsb3JVWvERUfBl4Rc2yrXfyIgnY/GoWUR4Gqsk/SXEv98w0VkpGuwARzJASFOxcGkCDrNPWNfL/ORezazd9fj6wka8wg8m4pQP3DYjCMoYh9Yn3ihdkVkmBnlemKkFj5cuh4Nahjbpba2Kvfn72nBqs/eddwzYfbaBqQA6g06YJcNOH5RCbBEzBo8FLYxnWwlFgKB1rynWtw4uVPCj78Jry7Rufu4BeRWKozlM4bO4SJwnzi15aM+XUujLmOAt90pEW2fCjnkE5msCQkOWWACxueGpKwfTg96HF4caNogOCaR7BADWZLOrdPCIfvqvUJs6OInOm792gj+73XVufXVEzue89XiOc=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2906002)(316002)(53546011)(6506007)(70586007)(4326008)(6862004)(82310400003)(356005)(86362001)(26005)(81166007)(6486002)(36756003)(8676002)(36860700001)(54906003)(508600001)(5660300002)(2616005)(83380400001)(8936002)(966005)(186003)(33656002)(107886003)(47076005)(336012)(6512007)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 07:29:38.1220
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6122d151-4095-4289-7f4f-08d979ace7f0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB1929

Hi Oleksandr,

> On 17 Sep 2021, at 7:13 am, Oleksandr Andrushchenko <Oleksandr_Andrushche=
nko@epam.com> wrote:
>=20
> Hi, Rahul!
>=20
> On 15.09.21 23:33, Stefano Stabellini wrote:
>> On Wed, 15 Sep 2021, Rahul Singh wrote:
>>> Hi Oleksandr, Stefano,
>>>=20
>>>> On 15 Sep 2021, at 6:30 am, Oleksandr Andrushchenko <Oleksandr_Andrush=
chenko@epam.com> wrote:
>>>>=20
>>>> Hi, Rahul!
>>>>=20
>>>> On 14.09.21 17:24, Oleksandr Andrushchenko wrote:
>>>>> }
>>>>>>>   +static int pci_ecam_register_mmio_handler(struct domain *d,
>>>>>>> +                                          struct pci_host_bridge *=
bridge,
>>>>>>> +                                          const struct mmio_handle=
r_ops *ops)
>>>>>>> +{
>>>>>>> +    struct pci_config_window *cfg =3D bridge->sysdata;
>>>>>>> +
>>>>>>> +    register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL)=
;
>>>>>>> +    return 0;
>>>>>>> +}
>>>>>> Given that struct pci_config_window is generic (it is not specific t=
o
>>>>>> one bridge), I wonder if we even need the .register_mmio_handler
>>>>>> callback here.
>>>>>>=20
>>>>>> In fact,pci_host_bridge->sysdata doesn't even need to be a void*, it
>>>>>> could be a struct pci_config_window*, right?
>>>>> Rahul, this actually may change your series.
>>>>>=20
>>>>> Do you think we can do that?
>>>>>=20
>>>> This is the only change requested that left unanswered by now.
>>>>=20
>>>> Will it be possible that you change the API accordingly, so I can
>>>>=20
>>>> implement as Stefano suggests?
>>> We need pci_host_bridge->sysdata as void* in case we need to implement =
the new non-ecam PCI controller in XEN.
>>> Please have a look once in Linux code[1] how bridge->sysdata will be us=
ed. struct pci_config_window is used only for
>>> ecam supported host controller. Different PCI host controller will have=
 different PCI interface to access the PCI controller.
>>>=20
>>> [1] https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/ke=
rnel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-rcar-host.c*n3=
09__;Iw!!GF_29dbcQIUBPA!kjkv6KIlvXOEgVaS6BNPRo0gyABihhO0XmNHRPFJaFAGhhTEuK1=
mIsWqPs0cXEipzkT_MmA-Eg$ [git[.]kernel[.]org]
>>>=20
>>> I think we need bridge->sysdata in future to implement the new PCI cont=
roller.
>> In my opinion the pci_config_window is too important of an information
>> to be left inside an opaque pointer, especially when the info under
>> pci_config_window is both critical and vendor-neutral.
>>=20
>> My preference would be something like this:
>>=20
>>=20
>> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
>> index 9c28a4bdc4..c80d846da3 100644
>> --- a/xen/include/asm-arm/pci.h
>> +++ b/xen/include/asm-arm/pci.h
>> @@ -55,7 +55,6 @@ struct pci_config_window {
>>      uint8_t         busn_start;
>>      uint8_t         busn_end;
>>      void __iomem    *win;
>> -    const struct    pci_ecam_ops *ops;
>>  };
>>=20
>>  /*
>> @@ -68,7 +67,8 @@ struct pci_host_bridge {
>>      uint16_t segment;                /* Segment number */
>>      u8 bus_start;                    /* Bus start of this bridge. */
>>      u8 bus_end;                      /* Bus end of this bridge. */
>> -    void *sysdata;                   /* Pointer to the config space win=
dow*/
>> +    struct pci_config_window* cfg;   /* Pointer to the bridge config wi=
ndow */
>> +    void *sysdata;                   /* Pointer to bridge private data =
*/
>>      const struct pci_ops *ops;
>>  };
>>=20
>>=20
>> As a reference the attached patch builds. However, I had to remove const
>> where struct pci_ecam_ops *ops is used.
>=20
> I'd like to know which route we go with this as this is now the last
>=20
> thing which stops me from sending v2 of this series.

I will modify the code as per Stefano request and will send the next versio=
n.

Regards,
Rahul
>=20
> Thank you,
>=20
> Oleksandr



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:35:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:35:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189039.338508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8PY-00016p-B2; Fri, 17 Sep 2021 07:35:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189039.338508; Fri, 17 Sep 2021 07:35:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8PY-00016i-7a; Fri, 17 Sep 2021 07:35:52 +0000
Received: by outflank-mailman (input) for mailman id 189039;
 Fri, 17 Sep 2021 07:35:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+lGq=OH=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mR8PX-00016c-DM
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:35:51 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 08df92e2-165a-4a42-9978-e5efe0b36805;
 Fri, 17 Sep 2021 07:35:50 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id ADB7D222B6;
 Fri, 17 Sep 2021 07:35:49 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 877BF13A78;
 Fri, 17 Sep 2021 07:35:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id dEy6H1VFRGGBGgAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 17 Sep 2021 07:35:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08df92e2-165a-4a42-9978-e5efe0b36805
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631864149; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=ZKHhczD4ZPOnkjIr1F9uI5astUnlALC1U4GQ2QogJ4A=;
	b=Hduyuvy/zqiHGyOIMu8VzzY/CaPM184syRBOs72yGbDPM1sUL/f7/NqXtk8f1LCSdG2VMC
	X54UG3PvLZHpHA5vdPbDHSq41Tn3SxeQLsA5lv9+RimiNVcWBW0CWXBZUoCYgvIxhAmOgp
	7gl1wpkoktmjOkauvsCGjHIfTR5F+b0=
Subject: Re: [PATCH] Xen/gntdev: don't ignore kernel unmapping error
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <6d5cc143-0436-a2da-c3f4-ee7a2cec840c@suse.com>
Date: Fri, 17 Sep 2021 09:35:48 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="L2IepqGuXH9lYRXao45ZswOeYQCpW1c6r"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--L2IepqGuXH9lYRXao45ZswOeYQCpW1c6r
Content-Type: multipart/mixed; boundary="HSlCyNwh3btw6jZ5G6b21xIga5DV9TQlY";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <6d5cc143-0436-a2da-c3f4-ee7a2cec840c@suse.com>
Subject: Re: [PATCH] Xen/gntdev: don't ignore kernel unmapping error
References: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
In-Reply-To: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>

--HSlCyNwh3btw6jZ5G6b21xIga5DV9TQlY
Content-Type: multipart/mixed;
 boundary="------------4F6AC4DCD1F519DE870AA7BC"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------4F6AC4DCD1F519DE870AA7BC
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 08:13, Jan Beulich wrote:
> While working on XSA-361 and its follow-ups, I failed to spot another
> place where the kernel mapping part of an operation was not treated the=

> same as the user space part. Detect and propagate errors and add a 2nd
> pr_debug().
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen


--------------4F6AC4DCD1F519DE870AA7BC
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------4F6AC4DCD1F519DE870AA7BC--

--HSlCyNwh3btw6jZ5G6b21xIga5DV9TQlY--

--L2IepqGuXH9lYRXao45ZswOeYQCpW1c6r
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFERVQFAwAAAAAACgkQsN6d1ii/Ey8H
Fwf+KrAQ9n9MPSoUb2ScP9E4CRU4amw9nl4bj1ssmF60xOsQOzr0rAwpkCDw3T2MToD4jkG8f798
UPYPCV3p3GGOkh+xfDNT2d/zU/YTKRAM5Rq4oV1GM06p+pA6hrGqoAoCod4+eNh0GBxsjbP9Y9ak
Rb60PTugVjx1DKhxNLj5kKA95phc61OswV65iD73tBb9KgBBrXGj3S+mcePxWqc4z4LNEV+LNzVj
lCUt72zEcRvGIq2Sl6qDV3wwUcEb5GXyx36M2ka75wHpr+Vv0iwZVvLU1uMq9Mdgd2crzhgwyQca
t3KwSQwFx4fJNiOYORov75GGLrxxjGbbWRmWCWQrYA==
=0RKF
-----END PGP SIGNATURE-----

--L2IepqGuXH9lYRXao45ZswOeYQCpW1c6r--


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:40:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189048.338519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Tj-0002kQ-2x; Fri, 17 Sep 2021 07:40:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189048.338519; Fri, 17 Sep 2021 07:40:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8Ti-0002kJ-Vr; Fri, 17 Sep 2021 07:40:10 +0000
Received: by outflank-mailman (input) for mailman id 189048;
 Fri, 17 Sep 2021 07:40:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eQru=OH=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mR8Th-0002kC-VI
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:40:10 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe06::61a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ac878083-cc1e-4427-a44c-b32241627c98;
 Fri, 17 Sep 2021 07:40:07 +0000 (UTC)
Received: from AM6PR01CA0058.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::35) by DB9PR08MB7099.eurprd08.prod.outlook.com
 (2603:10a6:10:2c4::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Fri, 17 Sep
 2021 07:40:05 +0000
Received: from AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:e0:cafe::b4) by AM6PR01CA0058.outlook.office365.com
 (2603:10a6:20b:e0::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 07:40:05 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT052.mail.protection.outlook.com (10.152.17.161) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 07:40:05 +0000
Received: ("Tessian outbound f1898412aff1:v103");
 Fri, 17 Sep 2021 07:40:04 +0000
Received: from 11b69b236501.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6B05E100-9103-4605-88CD-D55B5DF1920F.1; 
 Fri, 17 Sep 2021 07:39:58 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 11b69b236501.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 07:39:58 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6519.eurprd08.prod.outlook.com (2603:10a6:20b:31c::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 07:39:55 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 07:39:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac878083-cc1e-4427-a44c-b32241627c98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zv5OmffAvWkQG7PkKxEBmNF/9eeTfmf/uHmr6bM019A=;
 b=oiHo+TW634A13CGCMQAUxH5Erup3Ay1iXAtabMmhjwfB8+ARFjchZwfgW1T//dUfFuB1+ZjmG1SGeno3EU95vH7CDwOei8joVanf+TyxQRhGqbjlDkGIlMrKqPdid56YxanfYOTN83AJmIkzrFBi8nfv3/CMx6tfu9yeFJ9J/zw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5ec47c9712987c3d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=At9ULBezPCwSTubMekcQgOfeHAqlsuGDLBCYlMSve/GEF7+dhwk1ipQ4lQZd3aKFKdLGHvaXkyH17XWfTU07sUo7X9TzBDfsDKwiXlYzuy5XvaGJ8Xp8uqBQAwhcomnSwmfk9nXg5tWPpEpVJS+5ewmSUhcAjPsQ41sLfhJ5z1F+lI8dRj6+Okr4HdgqrUkt+yC/cncQsG8vLNl72+zvl8JNr2I9q1bvSQBeKtUm3F8R/fVzwcvkzYvN1mkrcjRScGymnxAe3LUPMIDjnnCI5uCkqzRu+Z911Hqwnt8qYzqMPNqKp+EhX+LTNFpclWF6qP78SjhiiuQgWH6ycfBX/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zv5OmffAvWkQG7PkKxEBmNF/9eeTfmf/uHmr6bM019A=;
 b=DQGIvuJWGT1VUxzmjuR25iN8C86JO6fHBAgf9STO6bpXhn6uE2buchLlx7iBiVooy9NnaA2OSYFBvM873ADaS/Ifyp9RCdTCDtkuXW/+cWxcn8/VvGKTpJNTwtZAIFU6ksFqZefL+/mdP6n+66yYBiVnyjD1AKV6Qr5A8AXzZSRRYCGsOeXKvubktrSD+SZQ2kExvSgDjccTvB/aZYJ6j7xYHDRSiLh3UHbyAehZUpmLybkzfGtoRF7ZifWMOevttSfGbJS+wstvtAfZry34rYyBzcPP/4LJqkDL2mt4td3rn+lWW0as7sf8tSLsBpx68lCZ7AdNi9gyTyfS5tWl5g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zv5OmffAvWkQG7PkKxEBmNF/9eeTfmf/uHmr6bM019A=;
 b=oiHo+TW634A13CGCMQAUxH5Erup3Ay1iXAtabMmhjwfB8+ARFjchZwfgW1T//dUfFuB1+ZjmG1SGeno3EU95vH7CDwOei8joVanf+TyxQRhGqbjlDkGIlMrKqPdid56YxanfYOTN83AJmIkzrFBi8nfv3/CMx6tfu9yeFJ9J/zw=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index:
 AQHXlPLBYGJaUEBt502oB439VJRFraucfJAAgADQqYCABOUXAIAAaPoAgAB9foCABOu6gA==
Date: Fri, 17 Sep 2021 07:39:55 +0000
Message-ID: <C100D81C-16F6-48F3-A069-EEF83D19DEBF@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <a3318d9459ace64224a14e4424eef657e2ed5b69.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091629020.10523@sstabellini-ThinkPad-T480s>
 <1DB601D4-C446-4102-811C-63EDDE3D2BC5@arm.com>
 <aa251354-3e5f-e1ae-2647-3a112ad5d12e@epam.com>
 <alpine.DEB.2.21.2109131356090.10523@sstabellini-ThinkPad-T480s>
 <f580b6fa-24b4-f50f-7745-a7d4c1420ac0@epam.com>
In-Reply-To: <f580b6fa-24b4-f50f-7745-a7d4c1420ac0@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a1d493f7-c2c4-4706-936a-08d979ae5d9e
x-ms-traffictypediagnostic: AS8PR08MB6519:|DB9PR08MB7099:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB709939741A11109425F606DAFCDD9@DB9PR08MB7099.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 dRGbAzRXB3j7/AJ/Jap+wHF9pF5tmd0VPJpEVfv07FDbGjoDlO/NukVB63pz6tXd/KlQ9T6zerlICvwdv7v1o5cjaFewWvGYLuCd7MLjmddCERwrtDg0MOwu8hKXv9Xdo6vwR8EOXmsGdm0m0ebmdQ9UEeeHxm1GQRBkL+IP5EOae5KVCuKRhl/0D83o0uQfQKifcuIWLOT7XPFizBBha1uNWM8kO6gfoYX/QDdH0xkooIL26lOEW6hYVZheOOxR1N0lNhWM4ZCz3sBCn8T9G26FIeeiW5I3ZE79Tlys6YduIPeHaSuyw0swM5KzEeHXqTt8szHH0xFmZphZ2YgIZgR3alssHOVPbsXrZUYgBsbmTUGsPeYgmdMGKxnNgBrBQBSkRWe3YpAdjgBQU/qWnQWzcjONJ+ZIMNmmQeuadAMA3YckZFkl4juc0qQHpbiry9VRiI8cSoybr0j1HJZQsSyyONSQ2U7OaIhnZHSLRusYAXYg/KJApD+K4gBMQHkwU3oouukosbQNifCHvWE0mNcYiuInRXDoc8/clWDR99jVHILibYXMJ8hRVyZOOfN1ZUmGcWQP3UPb4pN+3gqnAPscoy30puNj+7YDro3HNaoMplRM/ExcbKeO1hkpVGZR3cSQd4QcPKI4TmJuYXDB7+Gn86ENLoMD6Jxj5QwtHPi46s8nJxw9n2KdEDjet3pWg16v6xtEIHOe6TpavJ6J4lPI9t0LXHnVieQvfxQ+rm59zk2ww/X/g3DiLYwjR0rD98/eCifC0wZ+zMW2LAEQTVHj9Gjsfh2OhZZO+Xte+JN/fl6pA1G9p9yilRlPQIyEpjV3YESBeLr7liS0Dt8FJsizF72j8YNz2piWPgAE0bFvbXszXAO2DaaWic87hv7T
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(26005)(508600001)(53546011)(6506007)(966005)(36756003)(86362001)(6512007)(122000001)(66556008)(66476007)(64756008)(66946007)(91956017)(76116006)(66446008)(6486002)(2616005)(5660300002)(83380400001)(71200400001)(316002)(38100700002)(6916009)(8936002)(38070700005)(8676002)(4326008)(54906003)(2906002)(33656002)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <FC7928E326470742A77008A2285C5668@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6519
Original-Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9dc443a4-36f5-4d66-b841-08d979ae57ef
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	AlWBi7f5VMDfGi1LKK7fq899Vm1XlyLnrgFouPA8lM7/Tcsm3aPFi2pux8Vhj7U2aL0oFmSTWBDubngdWX5A7L3SDcaYhwcdJr0SDi0e+xVsC3YgSlIG8/AlTpuFG76zevc78nljaap6bZ54lMGrP12dbscSBeksv1wcuk3TU+rJtD/A7nbCQxay6ONE6owKZt26oKmIsg0xJLo3ic/4NhFbzZgUh7ffjqOkt/J0H2sClcijC93mrACg2BMACsdaYTyucwAml45G/mBAHFP6a9fF3cMlFfzwBwaZZyC6FAoF2stVh5PvHFocBPKCQ4vH735Ri5XfyBSh6sif/l612Dt1F4BZkBYUlSEe1XfnZRIqhluZOb7f1Puw7dweFU4a0UQfB+sFXM+wcDieQff4u+axETEl1AIFydJn3gC+q7rLx/lfrM1Ut3ehsEF2Sujw3fSRFJ+/UOHUdUnTHQJ7rSRzHFs8iB4r3wC+NJpRB/oUJUUSFz8CbYh+2a+hTTarm0ealBUyle9mNULztMDScvHrBHZ35W6R2koPHPUdXGCDlneN8sIeKodhi6bO6xlQRF+zeTR/DNPtf28lSSEVnWkjrWKSrKFHKSZ7FWX/qgK7bwzuew/tkbu3ywxyea36GS7ySQ3l5ijx6snMr/0ZOGlmJYZ8+3ieqwbJnC0VpjzslZMXAWgvuEAeViOHv9v9FAhh72WGSfhDrBzaY35eFWgG8K2KY3GqwTbugYw4WYPMH3cKDjVfp7toz2OiDNNk3MwgmT4dZ4JjSE/GJrpRI8zsqusTO7OvD7LvPuDW+9hyWfrsGX5BcIee+9YRLDf67BnSs/Bvr5qo8cPAQqqQUUyJ0neeVmwO/h0bPmFdTB4=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(346002)(376002)(46966006)(36840700001)(6506007)(33656002)(36860700001)(316002)(186003)(6862004)(82310400003)(478600001)(70206006)(36756003)(107886003)(6486002)(82740400003)(4326008)(26005)(70586007)(81166007)(53546011)(86362001)(2616005)(966005)(8936002)(5660300002)(54906003)(2906002)(8676002)(83380400001)(47076005)(336012)(356005)(6512007)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 07:40:05.0811
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a1d493f7-c2c4-4706-936a-08d979ae5d9e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7099

Hi Oleksandr, Stefano,

> On 14 Sep 2021, at 5:31 am, Oleksandr Andrushchenko <Oleksandr_Andrushche=
nko@epam.com> wrote:
>=20
>=20
> On 14.09.21 00:02, Stefano Stabellini wrote:
>> On Mon, 13 Sep 2021, Oleksandr Andrushchenko wrote:
>>> On 10.09.21 15:01, Rahul Singh wrote:
>>>> Hi Stefano,
>>>>=20
>>>>> On 10 Sep 2021, at 12:34 am, Stefano Stabellini <sstabellini@kernel.o=
rg> wrote:
>>>>>=20
>>>>> On Thu, 19 Aug 2021, Rahul Singh wrote:
>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>>=20
>>>>>> Add support for Xilinx ZynqMP PCI host controller to map the PCI con=
fig
>>>>>> space to the XEN memory.
>>>>>>=20
>>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam=
.com>
>>>>>> ---
>>>>>> xen/arch/arm/pci/Makefile          |  1 +
>>>>>> xen/arch/arm/pci/pci-host-zynqmp.c | 59 ++++++++++++++++++++++++++++=
++
>>>>>> 2 files changed, 60 insertions(+)
>>>>>> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>>>>>>=20
>>>>>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
>>>>>> index 6f32fbbe67..1d045ade01 100644
>>>>>> --- a/xen/arch/arm/pci/Makefile
>>>>>> +++ b/xen/arch/arm/pci/Makefile
>>>>>> @@ -3,3 +3,4 @@ obj-y +=3D pci-access.o
>>>>>> obj-y +=3D pci-host-generic.o
>>>>>> obj-y +=3D pci-host-common.o
>>>>>> obj-y +=3D ecam.o
>>>>>> +obj-y +=3D pci-host-zynqmp.o
>>>>>> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/p=
ci-host-zynqmp.c
>>>>>> new file mode 100644
>>>>>> index 0000000000..fe103e3855
>>>>>> --- /dev/null
>>>>>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
>>>>>> @@ -0,0 +1,59 @@
>>>>>> +/*
>>>>>> + * Copyright (C) 2020-2021 EPAM Systems
>>>>>> + *
>>>>>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>>>>>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>>>>>> + * Based on xen/arch/arm/pci/pci-host-generic.c
>>>>>> + * Copyright (C) 2014 ARM Limited Will Deacon <will.deacon@arm.com>
>>>>> Only one Copyright line per file is enough :-)
>>>>>=20
>>>>> But actually all the Copyright lines with a name or a company name ar=
e
>>>>> not really required or useful, as the copyright is noted in full deta=
ils
>>>>> in the commit messages (author and signed-off-by lines). I would remo=
ve
>>>>> them all from the new files added by this series.
>>>> Ok. Let me remove in next version.
>>>>>> + * This program is free software; you can redistribute it and/or mo=
dify
>>>>>> + * it under the terms of the GNU General Public License version 2 a=
s
>>>>>> + * published by the Free Software Foundation.
>>>>>> + *
>>>>>> + * This program is distributed in the hope that it will be useful,
>>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>>>> + * GNU General Public License for more details.
>>>>>> + *
>>>>>> + * You should have received a copy of the GNU General Public Licens=
e
>>>>>> + * along with this program.  If not, see <https://urldefense.com/v3=
/__http://www.gnu.org/licenses/__;!!GF_29dbcQIUBPA!lAdL_CvsuMuuX9ai6cwzm3NY=
iT1vwIIlxGU7nezSqq_nqJk40Zz-kT44LOsemcghJ_3j2CfflQ$ [gnu[.]org]>.
>>>>>> + */
>>>>>> +
>>>>>> +#include <asm/device.h>
>>>>>> +#include <xen/pci.h>
>>>>>> +#include <asm/pci.h>
>>>>>> +
>>>>>> +static const struct dt_device_match gen_pci_dt_match[] =3D {
>>>>>> +    { .compatible =3D "xlnx,nwl-pcie-2.11",
>>>>>> +      .data =3D       &pci_generic_ecam_ops },
>>>>>> +    { },
>>>>>> +};
>>>>>> +
>>>>>> +static int gen_pci_dt_init(struct dt_device_node *dev, const void *=
data)
>>>>>> +{
>>>>>> +    const struct dt_device_match *of_id;
>>>>>> +    const struct pci_ecam_ops *ops;
>>>>>> +
>>>>>> +    of_id =3D dt_match_node(gen_pci_dt_match, dev->dev.of_node);
>>>>> This should be superfluous
>>>> Ack. I will remove the dt_match_node(..) in next version.
>>> I am not entirely sure we need this patch at all as the main reason for=
 its existence
>>>=20
>>> was that we can run Xilinx QEMU for ZCU102. But, the final setup is not=
 going
>>>=20
>>> to be functional as legacy IRQs are not supported and ITS is not a part=
 of ZynqMP.
>>>=20
>>> So, QEMU allows to do a lot with PCI passthrough, but at the end of the=
 day one
>>>=20
>>> won't have it working...
>>>=20
>>> Please consider
>>>=20
>>> If we decide to remove it then
>>>=20
>>> int pci_host_common_probe(struct dt_device_node *dev,
>>>                            const struct pci_ecam_ops *ops,
>>>                            int ecam_reg_idx)
>>>=20
>>> doesn't need the last parameter.
>> With my open source maintainer hat on, I don't see this patch as very
>> important; from that point of view I'd be happy for it to be dropped.
>> However, it would be good to have at least one non-default host bridge
>> (doesn't have to be the Xilinx bridge), otherwise it becomes difficult
>> to understand how the generic infrastructure introduced by this series
>> could be useful.
>>=20
>> Moreover, your recent comment [1] made it even more evident that it
>> would be good to have at least two different drivers to spot
>> compatibility issues between them more easily.
>=20
> Don't take me wrong here ;) I still do use Xilinx QEMU for most of
>=20
> the tests, so it is good for me to have this patch in the tree. But,
>=20
> to be fair, Xilinx QEMU won't give you a possibility to see the fully
>=20
> functional system. This is why I say the patch can be dropped.
>=20
> If we add some words in the commit message about this and have the
>=20
> patch in the tree I'll be more than happy

I will have this patch in my series and I will add more comments about the =
patch.

Regards,
Rahul
>=20
>>=20
>> [1] https://urldefense.com/v3/__https://marc.info/?l=3Dxen-devel&m=3D163=
154474008598__;!!GF_29dbcQIUBPA!lAdL_CvsuMuuX9ai6cwzm3NYiT1vwIIlxGU7nezSqq_=
nqJk40Zz-kT44LOsemcghJ_0bKs6zpA$ [marc[.]info]



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:44:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:44:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189057.338532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8YC-0003Pw-Ni; Fri, 17 Sep 2021 07:44:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189057.338532; Fri, 17 Sep 2021 07:44:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8YC-0003Pp-Km; Fri, 17 Sep 2021 07:44:48 +0000
Received: by outflank-mailman (input) for mailman id 189057;
 Fri, 17 Sep 2021 07:44:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6HF0=OH=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mR8YB-0003Pj-0Q
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:44:47 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.48]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 20b7be7e-178b-11ec-b671-12813bfff9fa;
 Fri, 17 Sep 2021 07:44:46 +0000 (UTC)
Received: from DB6PR0301CA0076.eurprd03.prod.outlook.com (2603:10a6:6:30::23)
 by AM0PR08MB4051.eurprd08.prod.outlook.com (2603:10a6:208:125::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 07:44:44 +0000
Received: from DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:30:cafe::c7) by DB6PR0301CA0076.outlook.office365.com
 (2603:10a6:6:30::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Fri, 17 Sep 2021 07:44:44 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT031.mail.protection.outlook.com (10.152.20.142) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 07:44:44 +0000
Received: ("Tessian outbound b9598e0ead92:v103");
 Fri, 17 Sep 2021 07:44:44 +0000
Received: from 090e408896b1.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7A7BCED8-1C5E-4726-8D0D-CE3F0E40D62F.1; 
 Fri, 17 Sep 2021 07:44:38 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 090e408896b1.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 07:44:38 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6925.eurprd08.prod.outlook.com (2603:10a6:102:13a::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 07:44:35 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Fri, 17 Sep 2021
 07:44:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20b7be7e-178b-11ec-b671-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oObuuc468GUHjWzYe1lyj/n7xQ24z9nhbjoLJfHBUIM=;
 b=7JC9/tYDn3+qUaWew57qTKmxuGsSQQpxdg+LF7TfOW5cVo+RTqTxMuz34xKXpk/4qQHRk2Joruw1RGFPRaUTJYcHizULeFpRmR6wkIkFlJv7qkXmOz8EE+ArjDVr1OOcwTn32NF/XBYSjjkIK39qB87KSnHXTjxgS4SD7oDY+Uw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5727b28eada941d2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QwOyFrZbr4pEAW8qAfZE/QIxWtvQhWyA8sqxfo5635q5BtiRXHlAGuHGuyhGieAjRZa2+YwitDROwGRahfQ1sqPE5NDDw1KUWBR+ADK7N4CiF/zlgi+cQE4rx6vWz0Y8wrwOCcUt0cq75dLoQjJvSICuJVHJOCKoz7Pz8wLoB3oObNO9mMB9QMFbUuDhJZ9Vt5alXOqeIBjdqNYc1zuNd0kTxgVqa8X6W36c5WTznf7O6fReX6sy7j5wf93Wtb9rPU0a5sxyJiroXAlhE9hmyr3w45WLaJUYChX9/dp1HwX+CdMnry+ie564cwEyNJGzpI2cSOGr/qKC9bPb2P4aYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oObuuc468GUHjWzYe1lyj/n7xQ24z9nhbjoLJfHBUIM=;
 b=agiWVinrAdcDPGbL/acZ4IJ5yQ3SrQpJ7n7pAMmSIhMYGaFRW2ZfXp+gFYy9J7oKvkG6Hou1sOpogmojvFbK0gwLXz7qAObX51iVZSPbU6WIN8HHazCRkSJLS/qE/5U7r32i62xQQDrpEXvWsfpmFoHWFTS/7txdLst8fGsWABDZ/M93Ioq7i78uPeHm9Kx0KX9GfWXEYnMmGrpf11n0fsEVQNlk0SmLkTj1B3l3QIr7FrfsHaZ6IZqHyAtoA51fhXChtyFef+mm3Aaoi26rqotbsSi3d0OTsI9tDD0aciYosHTHRXLty0KvmERlcvseWkcSk0bEdNDBhHEldWrQAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oObuuc468GUHjWzYe1lyj/n7xQ24z9nhbjoLJfHBUIM=;
 b=7JC9/tYDn3+qUaWew57qTKmxuGsSQQpxdg+LF7TfOW5cVo+RTqTxMuz34xKXpk/4qQHRk2Joruw1RGFPRaUTJYcHizULeFpRmR6wkIkFlJv7qkXmOz8EE+ArjDVr1OOcwTn32NF/XBYSjjkIK39qB87KSnHXTjxgS4SD7oDY+Uw=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>, Penny Zheng <Penny.Zheng@arm.com>, Wei
 Chen <Wei.Chen@arm.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "julien@xen.org" <julien@xen.org>
Subject: Re: [PATCH] modify acquire_domstatic_pages to take an unsigned int
 size parameter
Thread-Topic: [PATCH] modify acquire_domstatic_pages to take an unsigned int
 size parameter
Thread-Index: AQHXqzwMeY29ReTQ9Eqwx8Co+TX8raun2T4A
Date: Fri, 17 Sep 2021 07:44:35 +0000
Message-ID: <6B295C63-A536-4D14-8818-30DBB77B7FF8@arm.com>
References: <alpine.DEB.2.21.2109161341270.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109161341270.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 879759f0-e0a7-4740-3045-08d979af03e8
x-ms-traffictypediagnostic: PAXPR08MB6925:|AM0PR08MB4051:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB40517D1D7D80929730C14EEE9DDD9@AM0PR08MB4051.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:5236;OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 /LQdROCaph6QeGfJ9QMm3LBTsx+RH6AWUb3rabTZJjlc+WfrQlmUlH5vt4bws7dX/52ItXClnLyBuc8UTCVR5+RUcaIXDno/OqNwVkfVkwJhv+9e3J0IqE00cpF6Lj/W7QfMTF4VxEqpOk5xvhrotkt0NqZSrQ0vuNqYkXIeuNHP63jIzsF9NVivKQl0zH9ZDeZTH3doRzTYp6yT6xvNJULy2K1QSgS0ClHLwVM0RgFsBKA6zafxBwUK1U4MVn0ZByHLhzLJIemPLjUq2CCN/Kf1MGF6FKL9yqIX+4kfBXkwrK0Rt1XukgmbTiJgsXrmt2Tc/Pmz2KkiOo/JKU1wE8DSwCRmWXjxkULaSxBAzYHRKzLWU34j0TH/YpLaK8EvCvkBXoN5ueLXwnVjSS7fI584IFyLCllJdZLCVu+yDvnAU3YG4kXX6gN6j3jFx06l2itMR9oiWDH4Zv6N7vr1Ojjchflx7DW+j4E9GlmE8PpfAu+k2NvhtrSzkS3rF5pYO82JV5B0rl/yLf67IuwALIsrzg49EIZ/OuProCPSwRKP+IrGRjYa3xbJnq1aTXWKMQhNB2mOp8xubAAFI2aYIpC5/7HXXyrzheJs8neCoPQ6A6MY3SARznwkwX5zmALnMftlhgCaEt2i8P1eArirZ1oeJWRfjP+Oe1nEqfd00iLIFr3PcJbskblO5AQIqx350pcr6n3uz9n2DyxUQLJH4pAfWPkGK1rNKX2pvSuYCIU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(66446008)(53546011)(6506007)(316002)(33656002)(8676002)(4326008)(66556008)(66476007)(66946007)(6916009)(91956017)(38100700002)(54906003)(6512007)(186003)(2906002)(2616005)(83380400001)(8936002)(6486002)(26005)(38070700005)(71200400001)(508600001)(122000001)(76116006)(36756003)(5660300002)(86362001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <861FDAEEE1B1134C9F2AB0316421D331@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6925
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1472d90d-5864-42b9-d1e5-08d979aeff01
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uX+Y7aXc8ZTekdGOGFMJtUiXHeWud/lD6vge0lfWDyllZFFF7gl13OfoyPREnP0n0LVy5MWBAID15nJvAx7z3BA0LgAjCyVdupy3M1PKzATZAgHAzjXOnmiit4nHjc5h0R3D9oiEP6olgpOESGw2HCWcQdYaBISNiobVvNUdJazpXTXOvt4eorMqPWxFkbhZ80uIKq+h1BRUdS3Y8kaLCpeTZq26ZWM97ulszv+igyv48fdAjdRZckF/m/FI3JTMh4IMwI+iSk1eClN+Sq/Ksj/mqnou4S2XOhYcd/aFgTzggJGZiPqxUngE+/ByCaS2ewre0ipkbuXkKS8bCEN3QBVVieEZSI2t/OigSGy7JYDsILXrKm2FLOoICk7ZpsI02dUdJ6Spwop/M7CaHTR56EV3Dbjj+6QdoXxDJcE/E9Ek4sqywFswt1rAxRnXYeJK0TttbbAvM1s9Z8f21riWItrp4tSgBI4SZHrfLPZz1f7KOLItcDV6VQWOpDMIVflwX4Yp4mgZnEgGoM27gK5jdQkzgNo3lMp1zeP2rkjZ4tzGcDZ9Z7Xy0FLzpqiG10/ojPdU5ciKI8ERiK7xSRHo+Py7evWbyM6Bm2XAvjEha9awSc+yMflbeXbkfsROjRwwxY5ltYofUiHZgeF+zD/YDcc1Ca2SQ6J6P7xN1mFLMRWKa5mFG+D0zebidxohq/TVdx1EhtKBpGNvfuBlcsLL0Q==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(54906003)(316002)(356005)(81166007)(36756003)(2616005)(86362001)(70206006)(6862004)(47076005)(36860700001)(508600001)(82310400003)(6512007)(6486002)(8936002)(8676002)(26005)(5660300002)(6506007)(53546011)(186003)(83380400001)(4326008)(2906002)(336012)(70586007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 07:44:44.1351
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 879759f0-e0a7-4740-3045-08d979af03e8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4051

Hi Stefano,

> On 16 Sep 2021, at 21:47, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> acquire_domstatic_pages currently takes an unsigned long nr_mfns
> parameter, but actually it cannot handle anything larger than an
> unsigned int nr_mfns. That's because acquire_domstatic_pages is based on
> assign_pages which also takes an unsigned int nr parameter.
>=20
> So modify the nr_mfns parameter of acquire_domstatic_pages to be
> unsigned int.
>=20
> There is only one caller in
> xen/arch/arm/domain_build.c:allocate_static_memory. Check that the value
> to be passed to acquire_domstatic_pages is no larger than UINT_MAX. If
> it is, print an error and goto fail.
>=20
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
>=20
> Jan, I took your suggestion of moving the check closer to where the
> value is read from DT. At that point I also took the opportunity to
> change acquire_domstatic_pages to take an unsigned int parameter
> instead of unsigned long.
>=20
>=20
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 62ab7d0ead..d233d634c1 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -554,6 +554,12 @@ static void __init allocate_static_memory(struct dom=
ain *d,
>         device_tree_get_reg(&cell, addr_cells, size_cells, &pbase, &psize=
);
>         ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZ=
E));
>=20
> +        if ( PFN_DOWN(psize) > UINT_MAX )
> +        {
> +            printk(XENLOG_ERR "%pd: static memory size too large: %#"PRI=
paddr,
> +                   d, psize);
> +            goto fail;
> +        }
>         smfn =3D maddr_to_mfn(pbase);
>         res =3D acquire_domstatic_pages(d, smfn, PFN_DOWN(psize), 0);
>         if ( res )
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index b9441cb06f..b64c07ae92 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2714,7 +2714,7 @@ static struct page_info * __init acquire_staticmem_=
pages(mfn_t smfn,
>  * then assign them to one specific domain #d.
>  */
> int __init acquire_domstatic_pages(struct domain *d, mfn_t smfn,
> -                                   unsigned long nr_mfns, unsigned int m=
emflags)
> +                                   unsigned int nr_mfns, unsigned int me=
mflags)
> {
>     struct page_info *pg;
>=20
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index dd49237e86..5db26ed477 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -89,7 +89,7 @@ bool scrub_free_pages(void);
> /* These functions are for static memory */
> void free_staticmem_pages(struct page_info *pg, unsigned long nr_mfns,
>                           bool need_scrub);
> -int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned long =
nr_mfns,
> +int acquire_domstatic_pages(struct domain *d, mfn_t smfn, unsigned int n=
r_mfns,
>                             unsigned int memflags);
> #endif
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 07:47:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 07:47:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189068.338547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8an-0004Jc-Cn; Fri, 17 Sep 2021 07:47:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189068.338547; Fri, 17 Sep 2021 07:47:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR8an-0004JV-9a; Fri, 17 Sep 2021 07:47:29 +0000
Received: by outflank-mailman (input) for mailman id 189068;
 Fri, 17 Sep 2021 07:47:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6HF0=OH=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mR8am-0004JP-Nc
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 07:47:28 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.43]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81164218-178b-11ec-b671-12813bfff9fa;
 Fri, 17 Sep 2021 07:47:28 +0000 (UTC)
Received: from AM5PR1001CA0072.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::49) by VI1PR08MB3248.eurprd08.prod.outlook.com
 (2603:10a6:803:4e::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 07:47:21 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:15:cafe::63) by AM5PR1001CA0072.outlook.office365.com
 (2603:10a6:206:15::49) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 07:47:21 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 07:47:21 +0000
Received: ("Tessian outbound 8e26f7114b75:v103");
 Fri, 17 Sep 2021 07:47:20 +0000
Received: from e37737cd780c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6543172F-1B1F-495F-81A6-4B89BDD3F46A.1; 
 Fri, 17 Sep 2021 07:47:14 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e37737cd780c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 07:47:14 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB7033.eurprd08.prod.outlook.com (2603:10a6:102:205::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 07:47:13 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Fri, 17 Sep 2021
 07:47:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81164218-178b-11ec-b671-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=05eRu5Hk5gKvbYa4WlKCEx0v3OIBRFb4f0ekRTzeEQI=;
 b=B7aoeHIJ0/9IOvDSlSqLsDS0gSa95aC9Fm4zxABcgW6D26Dcp/3RA5KtBZr+KJnH7kHFxXVzRprrwJiQIK4Xwl99UkMknw+kZ66qAZudrMuTXWG5chiCeyb+G8bix8XMHjpL+NCblK8AAwFFKHkmVP+gpwY5wY/mXnDSsGhsTg0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6bec12e7dca10541
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aImnDFNV0q5w06Tn5NZe5BSq7Rq/nc6GqoQFfBu4geLTSDxl+rcIwRvpv6n2KWHd8i9NDcyiSGSV038gELki0gS9G7dCSRXUcDYZOJBOqDR+yimNHVkNfbqf51l5p8nbR7JftHPWNp9ZluDxea9HcBgtOv/8eKuHmE9Rbd5wa4+yWn3uD+hMktiDNmBYY5bDWUOrNdJvY7J6xXHzMy41KkMrBFVL5DwNkEM4i+jGDk3M5hRtx+VmseIQovz/7ChIhHXSFHtJbh98TfvCOwYxLtngeXeeOuaDSgQQaAh15qjyY6EkdE8bQOfBSp5m1OZOFQ0Xa+jywCF/WsjobBPEeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=05eRu5Hk5gKvbYa4WlKCEx0v3OIBRFb4f0ekRTzeEQI=;
 b=O1kZllEXVKdvsMGZcddk7CdPt94Gwx8OvFS6ls0v0WOhhLb/UZpEEM5j/lSQUPPzCkum+5fVcWXfeC/Pa988ZYWYg5cu7cEOwLZhhLIVJpsmZsUlSBFBB0CIPZtbBZ8M9bs5MN9vi51GC4pXxGdUKIyzX+m43pLCq/fYMu2gvCqC2eV+4yGgVj4GJu4HwiiuI6OOBFZtBBnSGh9llSDN8SLXVY1wzKHAT7Das41AVvKCkNOubdMwaLb/Xnro7E5bKUzkBepTNuMYGKx1RZ4q/bjwJllsVb9eqpfKqA6THhoS2aFGyp8PjcIlMQwqWn8S/YOj7X3EqObP711E5wjjnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=05eRu5Hk5gKvbYa4WlKCEx0v3OIBRFb4f0ekRTzeEQI=;
 b=B7aoeHIJ0/9IOvDSlSqLsDS0gSa95aC9Fm4zxABcgW6D26Dcp/3RA5KtBZr+KJnH7kHFxXVzRprrwJiQIK4Xwl99UkMknw+kZ66qAZudrMuTXWG5chiCeyb+G8bix8XMHjpL+NCblK8AAwFFKHkmVP+gpwY5wY/mXnDSsGhsTg0=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v4 7/7] xen/arm: Sanitize CTR_EL0
Thread-Topic: [PATCH v4 7/7] xen/arm: Sanitize CTR_EL0
Thread-Index: AQHXqsPus1tQg1NxCEKJXP/MCMUVQ6unK+aAgACvBQA=
Date: Fri, 17 Sep 2021 07:47:13 +0000
Message-ID: <2C65F110-1DC6-485F-AA4F-B4D5B40DAF8E@arm.com>
References: <cover.1631772970.git.bertrand.marquis@arm.com>
 <3a6a63701df71c0a0ea743c6229266077da0563e.1631772970.git.bertrand.marquis@arm.com>
 <alpine.DEB.2.21.2109161420210.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109161420210.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: fe1de7d7-5469-4aa3-833b-08d979af6191
x-ms-traffictypediagnostic: PAXPR08MB7033:|VI1PR08MB3248:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3248189BAB1D53DD4599F25C9DDD9@VI1PR08MB3248.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:1751;OLM:1751;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 DIzMKwegguPdlIdlatykvuGNk8XDVILASUrt4M32dw4a5BfHxO+1X/Omaqe+33fv4aBv7t3rLZ3xdTw/2LEisJkeGhxu/D7yhkj1ixdD6THDvs+nFbkVLVxAQ6CCT0Qh86zlEPJp13zgupw5GzcEv/4marJnVKkTpNjOmHyz2scrluNoMhE7iYjZc+A3kNSVxyOIaEDgLMOZjbPY/uNUVjiEefhls5MlSPBjW+lqFNNy6E83+GeyvR4p4p2BJ/Bl9N38gfg6fyfQ9Xat4yMEKQZLvCIlYOo9Ara9x1whE3i0RHI7p3WTyf223kCDXY3HpdTMRokLYF7tchxPaW+uxmSz0MgNxemqUzGK+oWxK9xJbce5xigQL5/4MuLuOFJKY9p9Z1EjCDqzFTMEYljqX/oKDZnbWu2l4xDhGNnBvKg1cqsEJwATsJFaxHgKkeQSBnSbfnFl1ypfbXkztjfRamthbttSkskxgbWcKNkqpDq5axDj/TMN2pJ8mSIO6HE3ch1XjS4mYdhJ0P6o9sDmfnpxQWKfPNWuXNqAa6PcQ/FlM2zPo8Mx51ZHdrss3pYvO7OPUGJ7atxxTuRd5AGoXbMQDZnsLvQONms6cPv9Z5Yg3uhKA4HJkGFRbrTuGaFB3BfsLC0lgTQNpnxSLb9fnoRCl9Pgbun4PyFvxSE2FoCG6D4O+/+zP7uVWhwILSV4TdMJ10rjqHfeT7XOplDQICTN9D9djHb6vOS7XGz7R28tRyB60ilQJvzsKKrmjjmwfgX4f/jI+x4i/poKXpPYVw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(396003)(39860400002)(366004)(346002)(376002)(6916009)(86362001)(316002)(38100700002)(71200400001)(122000001)(8676002)(33656002)(4326008)(6486002)(6512007)(36756003)(2616005)(5660300002)(186003)(478600001)(54906003)(6506007)(26005)(83380400001)(53546011)(2906002)(8936002)(91956017)(66446008)(66946007)(64756008)(38070700005)(66556008)(66476007)(76116006)(21314003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <BA7710CB056E7D4385DC7F570AD8081A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7033
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	964fd50b-fd07-4c0d-2bd6-08d979af5d00
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/gIYBcW/y5rgepWxmjjzdcnm5iTOxgfObRcTsfclB2tnEPMMa3U3x/qFMPmgSIgNgsvfA6pElvi0SG9IewLiGK+mwoqLI9m43fDyxS+MZ2fVaS9QIVHfJqat3qNgIrK5GSi7vs9YwvixB4X4QAM3Tembs/6kz502T0ku+bHa4IQRT/j95Po9/2KHPACfTd1ZkkWyVgH4uUBXvCLAwF0752wSWCVzOMXzo2a+1VJfr0VfA6hyczdEQShZ9uUviORTHqtc4vZgQwCQA8YBYeLXp3qECRN8tw2UEcFWPx3B2HrFxkbY20Z0EuEiE7Mzaq9b7z7A2LY54WlPciy5TpmUysZxFBckdm1oQPP7RTKb+3R72hMpAl3USKgy5IwWBkD85Al9SNXpYr0ACHdJfzSN/hmHWbvQujQHzGMaJ6j9D1FFmFaXk+q61zmVsJK16HcbPN0iaI3M7owvX4TugMTXJkNssdrOPiK0L+aFXAs8acNEaYIzSNQilxfxqfrZFV6bPhcASUBBRN2kS2p7hhem0cyHc0D/lN1X8RKsqrhVVgPbpmnpL4L5v0+115LOY8y/yn5dJ9g96QNTvjYvoRNkJ8gExmTvt6y9YbzJcnGRFcAbVObAg73aLHfPfz6Vgs7gcDgtObBo5Zr2b43SX5/yHCAZhV7rh44UZmpCbjIkOYCLZ0p9pAZSgPekNiC5dOQpv1U1haeNicVqHnETj0PEeiFDb85R+YXHpKAb+2qaHjA=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(396003)(136003)(376002)(39850400004)(36840700001)(46966006)(26005)(6506007)(53546011)(107886003)(8676002)(2616005)(336012)(70206006)(5660300002)(86362001)(70586007)(82740400003)(356005)(33656002)(36756003)(81166007)(36860700001)(83380400001)(82310400003)(8936002)(2906002)(478600001)(6862004)(4326008)(186003)(6512007)(316002)(47076005)(54906003)(6486002)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 07:47:21.1000
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fe1de7d7-5469-4aa3-833b-08d979af6191
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3248

Hi Stefano,

> On 16 Sep 2021, at 22:20, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Thu, 16 Sep 2021, Bertrand Marquis wrote:
>> Sanitize CTR_EL0 value between cores and taint Xen if incompatible
>> values are found.
>>=20
>> In the case of different i-cache types, the sanitize ctr_el0 will have a
>> sanitize value but this is currently not used or exposed to guest which
>> are seeing the original ctr_el0 value.
>>=20
>> Use the opportunity to rename CTR_L1Ip to use an upper case name like
>> Linux does.
>> The patch is also defining ICACHE_POLICY_xxx instead of only having
>> CTR_L1IP_xxx to sync the definitions with Linux and is updating the code
>> using those accordingly (arm32 setup).
>>=20
>> On platforms with only the same type of cores, this patch should not
>> modify the current Xen behaviour.
>>=20
>> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
>> ---
>> Changes in v4: Remove TID2 support and handling of corresponding
>> register emulation in vcpreg/vsysreg.
>> Changes in v3: none
>> Change in v2: Patch introduced in v2
>> ---
>> xen/arch/arm/arm64/cpufeature.c  |  6 ++----
>> xen/arch/arm/cpufeature.c        |  2 ++
>> xen/arch/arm/setup.c             |  2 +-
>> xen/include/asm-arm/cpufeature.h |  9 +++++++++
>> xen/include/asm-arm/processor.h  | 18 +++++++++++++++---
>> 5 files changed, 29 insertions(+), 8 deletions(-)
>>=20
>> diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufea=
ture.c
>> index d4679f5df3..6e5d30dc7b 100644
>> --- a/xen/arch/arm/arm64/cpufeature.c
>> +++ b/xen/arch/arm/arm64/cpufeature.c
>> @@ -275,9 +275,6 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr2[=
] =3D {
>> 	ARM64_FTR_END,
>> };
>>=20
>> -#if 0
>> -/* TODO: use this to sanitize the cache line size among cores */
>> -
>> static const struct arm64_ftr_bits ftr_ctr[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 31, 1, 1), /* RES1 *=
/
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_DIC_SHIFT, =
1, 1),
>> @@ -294,7 +291,6 @@ static const struct arm64_ftr_bits ftr_ctr[] =3D {
>> 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SH=
IFT, 4, 0),
>> 	ARM64_FTR_END,
>> };
>> -#endif
>>=20
>> static const struct arm64_ftr_bits ftr_id_mmfr0[] =3D {
>> 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNER=
SHR_SHIFT, 4, 0xf),
>> @@ -606,6 +602,8 @@ void update_system_features(const struct cpuinfo_arm=
 *new)
>> 	 */
>> 	SANITIZE_REG(dczid, 0, dczid);
>>=20
>> +	SANITIZE_REG(ctr, 0, ctr);
>> +
>> 	if ( cpu_feature64_has_el0_32(&system_cpuinfo) )
>> 	{
>> 		SANITIZE_ID_REG(pfr32, 0, pfr0);
>> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
>> index 113f20f601..6e51f530a8 100644
>> --- a/xen/arch/arm/cpufeature.c
>> +++ b/xen/arch/arm/cpufeature.c
>> @@ -127,6 +127,8 @@ void identify_cpu(struct cpuinfo_arm *c)
>>=20
>>     c->dczid.bits[0] =3D READ_SYSREG(DCZID_EL0);
>>=20
>> +    c->ctr.bits[0] =3D READ_SYSREG(CTR_EL0);
>> +
>>     aarch32_el0 =3D cpu_feature64_has_el0_32(c);
>> #endif
>>=20
>> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>> index 4ab13d0fbe..49dc90d198 100644
>> --- a/xen/arch/arm/setup.c
>> +++ b/xen/arch/arm/setup.c
>> @@ -650,7 +650,7 @@ static void __init setup_mm(void)
>>         panic("No memory bank\n");
>>=20
>>     /* We only supports instruction caches implementing the IVIPT extens=
ion. */
>> -    if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) =3D=3D CTR_L1Ip_AIVI=
VT )
>> +    if ( ((ctr >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) =3D=3D ICACHE_POLICY=
_AIVIVT )
>>         panic("AIVIVT instruction cache not supported\n");
>>=20
>>     init_pdx();
>> diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpuf=
eature.h
>> index 5219fd3bab..cab89ee142 100644
>> --- a/xen/include/asm-arm/cpufeature.h
>> +++ b/xen/include/asm-arm/cpufeature.h
>> @@ -267,6 +267,14 @@ struct cpuinfo_arm {
>>         register_t bits[1];
>>     } dczid;
>>=20
>> +    /*
>> +     * CTR is only used to check for different cache types or policies =
and
>> +     * taint Xen in this case
>> +     */
>> +    struct {
>> +        register_t bits[1];
>> +    } ctr;
>> +
>> #endif
>>=20
>>     /*
>> @@ -339,6 +347,7 @@ extern struct cpuinfo_arm system_cpuinfo;
>> extern void identify_cpu(struct cpuinfo_arm *);
>>=20
>> #ifdef CONFIG_ARM_64
>> +
>> extern void update_system_features(const struct cpuinfo_arm *);
>> #else
>> static inline void update_system_features(const struct cpuinfo_arm *cpui=
nfo)
>=20
> Spurious change. I fixed it on commit.

Sorry for that and thanks.

Bertrand

>=20
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189091.338581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vd-0003Ac-7H; Fri, 17 Sep 2021 08:46:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189091.338581; Fri, 17 Sep 2021 08:46:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vd-0003AV-4E; Fri, 17 Sep 2021 08:46:13 +0000
Received: by outflank-mailman (input) for mailman id 189091;
 Fri, 17 Sep 2021 08:46:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vb-0003AJ-Bv
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:11 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb5656a5-9797-4987-ab65-05c08dbf14b4;
 Fri, 17 Sep 2021 08:46:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb5656a5-9797-4987-ab65-05c08dbf14b4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868369;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=ZQl96R3uvW4xzr4o62S9ScZWcGCrnlnt7bSHNhrnNaU=;
  b=V09bcFVy7+SVBcd2ifwZ9uUI2NgUFkIDiv8RWdrtnTHdsI72C71Zuv6U
   8hiuP1JyMU69fOBqdoVykwrFgVXYgOA7mtS9J/JsNkY/G7I3CrOBlAiwl
   pbR9NukIr4KDwzPRXiq125RSue9+26K5NF6LTF++8Fq/FfmuccjHeXglO
   c=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 2HwSl0Wy37iCWqK1V2Atl+jEb9D23bzvNaTNjZfsfml/Qw0LbobFvO+2H51Am8NEvA6N9q8F3U
 u54mHzrzOLsXTfuzDuba9XHO80bbx50yLrOKpjJlie+FkcH9SSI4iGEvZFN/2jIBQkK4hSFCrN
 ESafWK9rZpDjulP0SBHVmGpclH4boiTG5a38st7j+ffVkR2f9kwwh2zVl3pappV6OUpt08Rzjb
 qm1niSeRnGMQNgqCA5iucmJl/mIrXSmfsQ0AHQKoA0UsIE+Fe9NJJL4/A4aT4PVYSb2tMjDUVZ
 ZJsjGLJZTw8khpJpO+cCA3ce
X-SBRS: 5.1
X-MesageID: 54750985
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fbU+dqkyrGYGByzO8oRKKlTo5gx9IURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIWCG6Pbq2LazDxKooga9+/pE5VsZ/cztFmT1Rqq3swHiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09c32YHR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 I8dtICxQjUzBKr3ydUiWCYANCBRBYQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQKqBO
 pJIOWcHgBLoSCZJJls3B4wFrb2ku0HmY21WrXWxuv9ii4TU5FMoi+W8WDbPQfSIWMFUk0Cwt
 m/AuWPjDXkyFvaS1D6E+XKEnfLUkGXwX4d6PLG2//FlgVqXx0QIFQYbE1C8pJGRlUqWS99Zb
 UsO9UIGvaU0sUCmUNT5dxm5u2Kf+A4RXcJKFO834x3LzbDbiy6YGWosXjNHcMYhtsI9WXotz
 FDht8jyGTVlvbmRSHSc3rSZtzW/PW4SN2BqWMMfZVJbuZ+5+th110+RCIY4eEKosjHrMWugw
 iyLpxokvbIo3P803Jrj2lHouxv58/AlUTUJChXrsnONt10iPdP0ONP2sDA3/t4bc93IFQDpU
 Gws3pHEtbFQV8nleDmlHb1VdIxF8cppJ9E1bbRHJJAn6z3l0HqqZ4k4DNpWdRoxb5psldMEZ
 ibuVeJtCH17ZyDCgUxfOdvZ5yEWIU7IToyNuhf8NIEmX3SJXFXblByCnGbJt4wXrKTJrU3ZE
 czDGSpLJS1BYZmLMRLsH7tNuVPV7nlmmAs/uqwXPzz4iOHDNRZ5uJ8uMUeUb/BR0U93iFyOq
 L5i2z+x40wHCoXWO3CPmaZKdAxiBSVrVPje9p0MHsbec1UOJY3UI6KIqV/XU9c+xPo9eyah1
 izVZ3K0P3Kk2SSceFnVMys6AF4tNL4mxU8G0eUXFQ7A8xAejUyHtc/zrrM7Iusq8vJN1/lxQ
 6VXcsmMGK0XGD/G5y4cfd/2q4k7LEanggeHPiyEZjkjfsE/G1yVq4G8Jga/pjMTCieXtNclp
 +Hy3A3sXpdeFR9pC9zbaazzwgrp72Qdgu97Q2DBPsJXJBf36IFvJiGo1q03LsgAJA/t3Dyf0
 wrKUx4UqfOU+90+8cXThLDCpICsSrMsEk1fFmjdzLC3KSiFoTbznd4eCL6FJGmPWnn19aOuY
 fRu48v9aPBXzkxXt4dcEqpwyf5s7dXYuLIHnB9vG2/Gbgr3B+o4cGWGx8RGqoZE2qRd5VmtQ
 kuK99RXZeeJNcfiHAJDLQYpdL3eh/Qdmz2U5vUpOkTqoiRw+ePfA0lVOhCNjg1bLad0b9x5k
 btw5pZO5lztkAcuP/aHkjtQpjaFIXE3Wqk6so0XXd3wgQ0xx1AeOZHRB0caOn1Uhwmg5qXyH
 gKpuQ==
IronPort-HdrOrdr: A9a23:TYb2g61ZJvQfTtENkAZnJQqjBLAkLtp133Aq2lEZdPRUGvb3qy
 nOpoVj6faaslYssR0b9exofZPwJE80lqQFh7X5X43SPzUO0VHAROoJgLcKgQeQfxEWntQtsp
 uIGJIeNDSfNzdHZL7BkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="54750985"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 1/6] xen/trace: Don't over-read trace objects
Date: Fri, 17 Sep 2021 09:45:54 +0100
Message-ID: <20210917084559.22673-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

In the case that 'extra' isn't a multiple of uint32_t, the calculation rounds
the number of bytes up, causing later logic to read unrelated bytes beyond the
end of the object.

Also, asserting that the object is within TRACE_EXTRA_MAX, but truncating it
in release builds is rude.

Instead, reject any out-of-spec records, leaving enough of a message to
identify the faulty caller.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>

I've eyeballed the code and can't spot any problematic callers, but I came
very close to accidentally introducing some when trying to fix the stack
rubble leaks in subsequent patches.
---
 xen/common/trace.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

diff --git a/xen/common/trace.c b/xen/common/trace.c
index a2a389a1c7c3..25af6e1bd25e 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     unsigned long flags;
     u32 bytes_to_tail, bytes_to_wrap;
     unsigned int rec_size, total_size;
-    unsigned int extra_word;
     bool_t started_below_highwater;
 
     if( !tb_init_done )
         return;
 
-    /* Convert byte count into word count, rounding up */
-    extra_word = (extra / sizeof(u32));
-    if ( (extra % sizeof(u32)) != 0 )
-        extra_word++;
-    
-    ASSERT(extra_word <= TRACE_EXTRA_MAX);
-    extra_word = min_t(int, extra_word, TRACE_EXTRA_MAX);
-
-    /* Round size up to nearest word */
-    extra = extra_word * sizeof(u32);
+    /*
+     * Trace records require extra data which is an exact multiple of
+     * uint32_t.  Reject out-of-spec records.  Any failure here is an error in
+     * the caller.
+     */
+    if ( extra % sizeof(uint32_t) ||
+         extra / sizeof(uint32_t) > TRACE_EXTRA_MAX )
+        return printk_once(XENLOG_WARNING
+                           "Trace event %#x bad size %u, discarding\n",
+                           event, extra);
 
     if ( (tb_event_mask & event) == 0 )
         return;
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189096.338632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vn-0004Tf-7w; Fri, 17 Sep 2021 08:46:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189096.338632; Fri, 17 Sep 2021 08:46:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vm-0004Qd-WE; Fri, 17 Sep 2021 08:46:23 +0000
Received: by outflank-mailman (input) for mailman id 189096;
 Fri, 17 Sep 2021 08:46:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vl-0003AK-FI
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:21 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5c2337a-1793-11ec-b674-12813bfff9fa;
 Fri, 17 Sep 2021 08:46:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5c2337a-1793-11ec-b674-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868372;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=ygfYeKRHUam3n76JUrPhA5cc2MCBioaXf/z5nQljajU=;
  b=PXdInP1qkoK0HX/luY4x9tM+D5uac7N8fnWx5mP1TvUmuykJaT7S9zha
   68IMer5LyAPlKHN01f1Nrmw8LJ0uhwunoLiSesmBg9GaBcSgcBJfMtRcI
   hI9o6lvxCLZNDSiqLOD+RRsa7kapzFOBbSLQBRxxX8h02e5jrdA1Ruuiy
   U=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: CXyVjty1K4B7IfW+acmd4wWK5wt94QebHnEe10gHKYcgtacvTTk7UXAaa1xDk3c5vqe0JbxrlS
 4CwMXp0sJ/z4BNeMoIR2FigOHuNhGTC7aQ8JqzScEDNQm8jB7pCgzxKSIAMqVsU+HXuskv9L7x
 bNUoGpCuP7v6Dw9pChzUDknPp/dJ+EI4QkreVtn0xCmrA+A7DwvQ1FU5b8r3NMWMquaZT16R7F
 uWLadfRkFqotYV86M8wQc26GU58bOFrsq7XoBfFNvib9S3iWmZDudMPV3GQG2dE7/8MtsLhwys
 EM9LpMrY8ctZyDGni6srMhqZ
X-SBRS: 5.1
X-MesageID: 52955352
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:WaLL86IJ9GqZ9ZnnFE+RLZIlxSXFcZb7ZxGr2PjKsXjdYENS1mFSn
 2ZNWjjQaf/ZamL1fot/bYux8EhQ7cSAmoNrTwZlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM6xLZg6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2AgelP5
 tNtrKWBbhsFMovCmr5DXShXRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gpq150TRquED
 yYfQSBTKzvxXgdvAFwwGoozgfialEX0dwQN/Tp5ooJoujOOnWSdyoPFMsfRe9GMbdVYmACfv
 G2u10bTDwweNdef4SGY6X/qjejK9Qv3VosdG7y/8v9Cm0CIyyoYDxh+fUS/iem0jAi5Qd03A
 1wZ/G8ioLY/8GSvT8LhRFuorXicpBkeVtFMVeog52mly7XWyxaUAHAeSTxMY8Bgs9U5LRQxz
 UOAld7tAT1psZWWRGib+7PSqim9URX5NkdbO3VCF1FcpYC+/sdj1XojU+qPDoa8kOTLNzfQ2
 gm7izQSi5Mdj5RV2ZS0qAWvby2XmnTZcuIkzlyJBTv4v1wmOtfNi5+AsgeAvK0ZRGqNZhzY5
 iFVxZLGhAwbJczVzESwrPMx8KZFDhpvGAbVh0JmV7Il/i6kk5JIVdENuGwiTKuF38BtRNMIX
 KMxkVgKjHOwFCHzBUOSX25XI59ypUQHPY6+Ps04lvIUPvBMmPavpUmCn3J8OlwBdmB3ycnT3
 r/AKq6R4YsyU/w7nFJauc9EieJDKt8CKZP7GsmgkkXPPUu2T3+JU7YVWGZinchgt/jsnekhy
 P4Gb5Hi40wGCIXWO3CLmaZOfQFiBSVqXvje9p0IHtNv1yI7QQnN/deKmuh/E2Gk9owI/tr1E
 oaVARMBkwuj2iSccG1nqBlLMdvSYHq2llpjVQREALpi8yJLjV+H4PhNep0pU6Mg8eA/n/d4Q
 +NcI5eLA+hVSySB8DMYNMGvoItnfRWtpASPIyv6P2RvI88+H1TEqo3+Yw/i1CgSFS7r58Ewl
 KKti1HASp0ZSgU8UMuPMKCzz0m8tGQ2kf5pWxeaOcFafUjhqdA4KyH4gvItDdsLLBHPmmmT2
 wqMWE9KrujRuY4lttLOgPnc/YuuFuJ/GGtcHnXasunqZXWLoDL7zNYZAuiSfD3bWGflw4mYZ
 L1Yn6PmLfkKvFdWqI4gQbxl+r0zuon0rLhAwwU6QHiSNwa3CqltK2Wt1NVUsvEf3adQvAa7V
 x7d+tRePrnVasrpHERIeVggZ+WHk/oVhiPT/bI+J0CjvH17+7+OUENzORiQiXMCcOspYd19m
 epx6tQL7wGfiwYxNofUhy9ZwG2AM3gcXvh1rZodGoLq1lImx1wqjUYw0cMqDEVjs+lxD3Q=
IronPort-HdrOrdr: A9a23:ToqY0qw7bTpHHS80/MlFKrPwLr1zdoMgy1knxilNoRw8SKKlfq
 GV7Y0mPHDP6Ar5NEtNpTnEAtjkfZq+z+8S3WByB8bAYOCOggLBR+sO0WKh+UyFJ8SXzJ876U
 4KSclD4bPLYmSS9fyKgjWFLw==
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="52955352"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 2/6] xen/memory: Remove tail padding from TRC_MEM_* records
Date: Fri, 17 Sep 2021 09:45:55 +0100
Message-ID: <20210917084559.22673-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

Four TRC_MEM_* records supply custom structures with tail padding, leaking
stack rubble into the trace buffer.  Three of the records were fine in 32-bit
builds of Xen, due to the relaxed alignment of 64-bit integers, but
POD_SUPERPAGE_SPLITER was broken right from the outset.

We could pack the datastructures to remove the padding, but xentrace_format
has no way of rendering the upper half of a 16-bit field.  Instead, expand all
16-bit fields to 32-bit.

For POD_SUPERPAGE_SPLINTER, introduce an order field as it is relevant
information, and to matche DECREASE_RESERVATION, and so it doesn't require a
__packed attribute to drop tail padding.

Update xenalyze's structures to match, and introduce xentrace_format rendering
which was absent previously.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>

The xentrace_format script isn't remotely Py3 compatible, and was another
script missed by our previous efforts.
---
 tools/xentrace/formats    |  4 ++++
 tools/xentrace/xenalyze.c | 12 ++++++------
 xen/arch/x86/mm/p2m-pod.c | 17 +++++++++--------
 xen/common/memory.c       |  4 ++--
 4 files changed, 21 insertions(+), 16 deletions(-)

diff --git a/tools/xentrace/formats b/tools/xentrace/formats
index deac4d8598b0..0fcc327a4078 100644
--- a/tools/xentrace/formats
+++ b/tools/xentrace/formats
@@ -136,6 +136,10 @@
 0x0010f001  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_map      [ domid = %(1)d ]
 0x0010f002  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_unmap    [ domid = %(1)d ]
 0x0010f003  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_transfer [ domid = %(1)d ]
+0x0010f005  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  decrease_reservation   [ d%(3)d gfn 0x%(2)08x%(1)08x, order %(4)u ]
+0x0010f010  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_populate           [ d%(5)d gfn 0x%(2)08x%(1)08x => mfn 0x%(4)08x%(3)08x, order %(6)u ]
+0x0010f011  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_zero_reclaim       [ d%(5)d gfn 0x%(2)08x%(1)08x => mfn 0x%(4)08x%(3)08x, order %(6)u ]
+0x0010f012  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_superpage_splinter [ d%(3)d gfn 0x%(2)08x%(1)08x, order %(4)u ]
 
 0x00201001  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  hypercall  [ eip = 0x%(1)08x, eax = 0x%(2)08x ]
 0x00201101  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  hypercall  [ rip = 0x%(2)08x%(1)08x, eax = 0x%(3)08x ]
diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 5de167031e01..12dcca964645 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -8121,7 +8121,7 @@ void mem_pod_zero_reclaim_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn, mfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( v && v->hvm.vmexit_valid )
@@ -8171,7 +8171,7 @@ void mem_pod_populate_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn, mfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
@@ -8204,14 +8204,14 @@ void mem_pod_superpage_splinter_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn;
-        int d:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
     {
-        printf(" %s pod_spage_splinter d%d g %llx\n",
+        printf(" %s pod_spage_splinter d%d o%d g %"PRIx64"\n",
                ri->dump_header,
-               r->d, (unsigned long long)r->gfn);
+               r->d, r->order, r->gfn);
     }
 }
 
@@ -8255,7 +8255,7 @@ void mem_decrease_reservation_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 8abc57265c10..90f02ae765f6 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -819,8 +819,8 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, gfn_t gfn)
     if ( tb_init_done )
     {
         struct {
-            u64 gfn, mfn;
-            int d:16,order:16;
+            uint64_t gfn, mfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
@@ -987,8 +987,8 @@ p2m_pod_zero_check(struct p2m_domain *p2m, const gfn_t *gfns, unsigned int count
             if ( tb_init_done )
             {
                 struct {
-                    u64 gfn, mfn;
-                    int d:16,order:16;
+                    uint64_t gfn, mfn;
+                    uint32_t d, order;
                 } t;
 
                 t.gfn = gfn_x(gfns[i]);
@@ -1217,8 +1217,8 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
     if ( tb_init_done )
     {
         struct {
-            u64 gfn, mfn;
-            int d:16,order:16;
+            uint64_t gfn, mfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
@@ -1260,12 +1260,13 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
     if ( tb_init_done )
     {
         struct {
-            u64 gfn;
-            int d:16;
+            uint64_t gfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
         t.d = d->domain_id;
+        t.order = order;
 
         __trace_var(TRC_MEM_POD_SUPERPAGE_SPLINTER, 0, sizeof(t), &t);
     }
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 63642278fda9..8fd88ccb70bf 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -450,8 +450,8 @@ static void decrease_reservation(struct memop_args *a)
         if ( tb_init_done )
         {
             struct {
-                u64 gfn;
-                int d:16,order:16;
+                uint64_t gfn;
+                uint32_t d, order;
             } t;
 
             t.gfn = gmfn;
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189094.338616 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vi-0003z3-8n; Fri, 17 Sep 2021 08:46:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189094.338616; Fri, 17 Sep 2021 08:46:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vi-0003yu-31; Fri, 17 Sep 2021 08:46:18 +0000
Received: by outflank-mailman (input) for mailman id 189094;
 Fri, 17 Sep 2021 08:46:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vg-0003AK-FD
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:16 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b3cf480a-1793-11ec-b674-12813bfff9fa;
 Fri, 17 Sep 2021 08:46:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3cf480a-1793-11ec-b674-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868369;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=15+6z5EM4y397ipt3dNnQGW78oEoP00mzOswuO2CQJw=;
  b=VPdj1W4SV5GLu5vRqT0h05f4Bosx5qsSsO6WrK3h7pqXNoCsr/XTAgcl
   xn2V/2kGYGJVVEfZIbEuHf8D5o+YZNCMgKfb7GmfF4z3A3a/GcpIqQUHx
   5TfHJwvqKIz1REmNDkkuUnXcKnyT4vak/nLVbBOhAale1/H6zGbKA3xKB
   4=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: Aedc5C6amQnhqJGYqM0ObEOzLMD1itbaJL378IHVOSyhA/iAtFYUWo4wrkVc2bDkX6XdJgwwoE
 kMz/adWbfUBKyT1ebqyDC1TutMOzALTu0jlfTjKGbe196er4juLOpvIEIep7Pf9XiUAQl5eE0E
 uok2bAWOAn8r30LzNHugR/VtoL4L/XgjudRNT3SDPIWZk2MjqzBU1tWUYmDc+PxX51OhlWMhWP
 jLALNaRd3mUb7iZ/nZbt+DUfcGLJqlR7a0SvLOw8t5fOO3nLb0H6UVljrsDQLzlJZCTy109bRb
 YSgMW66agaqdKnRsLjq1g8Z3
X-SBRS: 5.1
X-MesageID: 52955350
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:5XvjIKP4PmOGH+7vrR1ZkMFynXyQoLVcMsEvi/4bfWQNrUohgTQOm
 2YWCz2BbqqIY2LzeIwlYdy38hgEvsPSndJkQQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Es8w7dRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYozyXpu5fz
 vVUjK2XVSslGKPOwroTYhYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgW1h250VQK62i
 8wxVRNASzPkMht0GHgKVbcYpdeCjF3wWmgNwL6SjfVuuDWCpOBr65DvLd7Ud9qiVchT2EGCq
 Qru3U70HxUbP9y30iee/zSngeqntS/1UY0dFbq899ZxnUaegGcUDXU+Sl+TsfS/zEmkVLp3O
 0ESvyYjs6U23EiqVcXmGQ21pmaeuRwRUMYWFPc1gCmM1aj88wufHnIDTDNKdJohrsBebScxy
 laDktftBDpumL6YU3SQ8vGTtzzaBMQOBTZcP2leF1JDuoS95tFo5v7Scjp9OKert+PbHxHp+
 gGtsAQyobsR15AF5qruqDgrnAmQSoj1oh8dv1uMBz/0sVwnPeZJdKTztgOKtq8owJKxCwDb5
 SlawZD2APUmUMnV/BFhVtnhC11ACxytCzTanVcnNJ0o7T3FF5WLLN0IvW0WyKuENK85ld7Vj
 K374lg5CHx7ZiLCgUpLj2SZUJxCIU/IT4+NaxwsRoASCqWdjSfelM2UWaJ144wLuBN0+U3YE
 czAGftA8F5AUfg3pNZIb75FjNfHORzSNUuMHMumnnxLIJK1ZWKPSKdtDbd9Rrlit8u5TPHu2
 48HbaOikkwHOMWnO3W/2dNDfDgicClgbbir+pM/SwJ2Clc/cI3XI6SKmu1Jlk0Mt/k9q9okC
 VnmBhYDmQag3CaeQehIA1g6AI7SsV9EhSpTFUQR0ZyAghDPuK6js/UScYUZZ74i+LAxxPJ4V
 aBdKc6BHu5OWnLM/DFENcvxq4lrdRKKgwOSPnX6PGhjLsA4HwGZqMX5egbP9TUVCnblv8UJv
 LD9hBjQRoAORlo+AZ+OOu6v1V64oVMUhPl2AxnTOtBWdUi1qNpqJiX9g+UZOcYJLRmflDKW2
 xzPWUUTpPXXop9z+97M3PjWo4CsGup4P0xbA2iEsurmaXiEpjKumNYSXvyJcDbRUHLP1J+jP
 egFnevhNPAnnUpRt9YuGbhc0q9jtcDkoKVXz1o4ESyTPUirEL5pPlKPwdJL6v9W3rZctAa7B
 hCP991dNenbMc/pCgdMdg8sb+DF3vAIgDjCq/8yJRyitiNw+bOGV2RUPgWN13MBfOckbtt9z
 LdzotMS5iy+lgEuY4SPgS1j/miRKmANDvc8vZYADY630gcmxzmuu3AH5vMaNH1XV+hxDw==
IronPort-HdrOrdr: A9a23:FCOFEqtGdC5Gp4shE4Ost2bi7skDctV00zEX/kB9WHVpm6uj5q
 eTdZUgpHvJYVMqM03I9urtBEDtexzhHP1OgbX5X43NYOCOggLBRuxfBODZogHIKmnT8fNcyL
 clU4UWMqyUMbGit7eY3OBvKadD/OW6
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="52955350"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 6/6] xen/trace: Minor code cleanup
Date: Fri, 17 Sep 2021 09:45:59 +0100
Message-ID: <20210917084559.22673-7-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

 * Delete trailing whitespace
 * Replace an opencoded DIV_ROUND_UP()
 * Drop bogus smp_rmb() - spin_lock_irqsave() has full smp_mb() semantics.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/common/trace.c              | 37 +++++++++++++++----------------------
 xen/include/asm-x86/hvm/trace.h |  2 +-
 2 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/xen/common/trace.c b/xen/common/trace.c
index 25af6e1bd25e..18008df69e10 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -75,10 +75,6 @@ static cpumask_t tb_cpu_mask;
 /* which tracing events are enabled */
 static u32 tb_event_mask = TRC_ALL;
 
-/* Return the number of elements _type necessary to store at least _x bytes of data
- * i.e., sizeof(_type) * ans >= _x. */
-#define fit_to_type(_type, _x) (((_x)+sizeof(_type)-1) / sizeof(_type))
-
 static int cpu_callback(
     struct notifier_block *nfb, unsigned long action, void *hcpu)
 {
@@ -96,8 +92,8 @@ static struct notifier_block cpu_nfb = {
 
 static uint32_t calc_tinfo_first_offset(void)
 {
-    int offset_in_bytes = offsetof(struct t_info, mfn_offset[NR_CPUS]);
-    return fit_to_type(uint32_t, offset_in_bytes);
+    return DIV_ROUND_UP(offsetof(struct t_info, mfn_offset[NR_CPUS]),
+                        sizeof(uint32_t));
 }
 
 /**
@@ -148,7 +144,7 @@ static int calculate_tbuf_size(unsigned int pages, uint16_t t_info_first_offset)
         pages = max_pages;
     }
 
-    /* 
+    /*
      * NB this calculation is correct, because t_info_first_offset is
      * in words, not bytes
      */
@@ -167,7 +163,7 @@ static int calculate_tbuf_size(unsigned int pages, uint16_t t_info_first_offset)
  * trace buffers.  The trace buffers are then available for debugging use, via
  * the %TRACE_xD macros exported in <xen/trace.h>.
  *
- * This function may also be called later when enabling trace buffers 
+ * This function may also be called later when enabling trace buffers
  * via the SET_SIZE hypercall.
  */
 static int alloc_trace_bufs(unsigned int pages)
@@ -401,7 +397,7 @@ int tb_control(struct xen_sysctl_tbuf_op *tbc)
         break;
     case XEN_SYSCTL_TBUFOP_enable:
         /* Enable trace buffers. Check buffers are already allocated. */
-        if ( opt_tbuf_size == 0 ) 
+        if ( opt_tbuf_size == 0 )
             rc = -EINVAL;
         else
             tb_init_done = 1;
@@ -438,7 +434,7 @@ int tb_control(struct xen_sysctl_tbuf_op *tbc)
     return rc;
 }
 
-static inline unsigned int calc_rec_size(bool_t cycles, unsigned int extra) 
+static inline unsigned int calc_rec_size(bool_t cycles, unsigned int extra)
 {
     unsigned int rec_size = 4;
 
@@ -597,7 +593,7 @@ static inline void __insert_record(struct t_buf *buf,
         rec->u.cycles.cycles_lo = (uint32_t)tsc;
         rec->u.cycles.cycles_hi = (uint32_t)(tsc >> 32);
         dst = rec->u.cycles.extra_u32;
-    } 
+    }
 
     if ( extra_data && extra )
         memcpy(dst, extra_data, extra);
@@ -717,9 +713,6 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     if ( !cpumask_test_cpu(smp_processor_id(), &tb_cpu_mask) )
         return;
 
-    /* Read tb_init_done /before/ t_bufs. */
-    smp_rmb();
-
     spin_lock_irqsave(&this_cpu(t_lock), flags);
 
     buf = this_cpu(t_bufs);
@@ -735,14 +728,14 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
 
     /* Calculate the record size */
     rec_size = calc_rec_size(cycles, extra);
- 
+
     /* How many bytes are available in the buffer? */
     bytes_to_tail = calc_bytes_avail(buf);
-    
+
     /* How many bytes until the next wrap-around? */
     bytes_to_wrap = calc_bytes_to_wrap(buf);
-    
-    /* 
+
+    /*
      * Calculate expected total size to commit this record by
      * doing a dry-run.
      */
@@ -756,7 +749,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
         {
             total_size += bytes_to_wrap;
             bytes_to_wrap = data_size;
-        } 
+        }
         total_size += LOST_REC_SIZE;
         bytes_to_wrap -= LOST_REC_SIZE;
 
@@ -768,7 +761,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     if ( rec_size > bytes_to_wrap )
     {
         total_size += bytes_to_wrap;
-    } 
+    }
     total_size += rec_size;
 
     /* Do we have enough space for everything? */
@@ -781,7 +774,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     }
 
     /*
-     * Now, actually write information 
+     * Now, actually write information
      */
     bytes_to_wrap = calc_bytes_to_wrap(buf);
 
@@ -791,7 +784,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
         {
             insert_wrap_record(buf, LOST_REC_SIZE);
             bytes_to_wrap = data_size;
-        } 
+        }
         insert_lost_records(buf);
         bytes_to_wrap -= LOST_REC_SIZE;
 
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
index 2bbac45044ce..fbf834d10aff 100644
--- a/xen/include/asm-x86/hvm/trace.h
+++ b/xen/include/asm-x86/hvm/trace.h
@@ -52,7 +52,7 @@
 #define DO_TRC_HVM_CLTS        DEFAULT_HVM_MISC
 #define DO_TRC_HVM_LMSW        DEFAULT_HVM_MISC
 #define DO_TRC_HVM_LMSW64      DEFAULT_HVM_MISC
-#define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC 
+#define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC
 #define DO_TRC_HVM_TRAP             DEFAULT_HVM_MISC
 #define DO_TRC_HVM_TRAP_DEBUG       DEFAULT_HVM_MISC
 #define DO_TRC_HVM_VLAPIC           DEFAULT_HVM_MISC
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189093.338603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vg-0003hT-OX; Fri, 17 Sep 2021 08:46:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189093.338603; Fri, 17 Sep 2021 08:46:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vg-0003hL-Ka; Fri, 17 Sep 2021 08:46:16 +0000
Received: by outflank-mailman (input) for mailman id 189093;
 Fri, 17 Sep 2021 08:46:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vg-0003AJ-45
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:16 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d547ed4a-333f-44c9-9f2c-1ddc9a887db3;
 Fri, 17 Sep 2021 08:46:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d547ed4a-333f-44c9-9f2c-1ddc9a887db3
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868371;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=r4SmFOIEP9O9irUeiBnJE4TyoFxvXpO9Kd5PyOYMvEY=;
  b=VdEryaI+n2oP8rggt8mhuolRClVpUxk3JcmvrzkvHqAhVGxULyN2Du3l
   LVSA08yOljQtbzURU1rby7k+yjWHjlyJZO3/7kJsvt1x5bjfsJ/fMGL4q
   EoCW2icpl2G1FrbYfv4SnkgKfozUuSuTqP9mVHRm0BDlqO604BovW/F0f
   4=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: JSm8SffdyxHUd9GVLPoEWHdUZetYvTIPnsvJw++KHnM9g7TtIDSLADXykFj5d7TLYoGcH2iEVY
 WjjDFN0dKbyGZkmP1n30c94zl5/niS2MpbxpVhETM5l1iWfoCYpAzSfkg03znMt+HQRrcUxVLO
 S+EBalbPRG1YJmf7lZ6ASho8Mwhc9iiT+KCvCU/2SQDXQkQTEM76Yx/k1yIoiwBfQoy4ki0ezK
 qF4aH0SMs6FT4NwSN8yIiI4OJ1vROdfUTGJkJ+lNCVp327z8beqmgU+y7XFXN1oRx2EhMO3Vac
 net3eBX44u6KPgYql4hj69hI
X-SBRS: 5.1
X-MesageID: 54750987
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:d1SstK8cXCf95j2PDyx5DrUDfHmTJUtcMsCJ2f8bNWPcYEJGY0x3z
 jFOWWGHOP2OZjb3e9AjbIvg/UtSupTUx9YwGVdupC88E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79g3dYx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPgrm
 NRRtJeCUz0uBfz9p+8iCz5ZPBlXaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguwKKsXxMZxZkXZn1TzDVt4tQIzZQrWM7thdtNs1rp0SQquGN
 ppGAdZpRB/CbTwRAwwZMbsZluaMqWPeVDcJo2vA8MLb5ECMlVcsgdABKuH9eMGORMhTtlaVo
 CTB5WuRKjMwOcGbyDGF2mmxneKJliT+MKoeGrS3+f9thFy73XEIBVsdUl7Tiea9ol6zXZRYM
 UN80jojq+0++VKmSvH5XgakuziUsxgEQd1SHuYmrgaXxcL84hmbLngJSCZbb94rv9NwQiYlv
 neShM/gDzFrtLyTSFqe+62SoDf0PjIaRVLufgddE1FDuYO65thu0FSfFb6PDZJZkPWvSTDsw
 ju4thI5vOQissU0j4K60mju1mfESofycuIl2unGdjv7tVokNdH6OdXABUvztqkbfd3AJrWVl
 D1dwZHPsrpWZX2YvHHVGI0w8KeVC+Fp2dE2qWVmGYUovx+p8mSqFWy7yGAjfBo1WirolDmAX
 aMyhe+zzMQIVJdJRfUuC25UNyjM5fO4fTgCfqqPBueimrArKGe6ENhGPCZ8JVwBdXTAd4lkY
 f93lu73Vh4n5VlPlmLqF4/xL5dynn1WKZzvqWDTkE38jOv2iI+9YrYZKlqeBt3VH4vd+169z
 jqrDOPTk083eLSnOkH/qNdPRXhXfSlTLc2n8KR/K7/cSjeK7Ul8Upc9N5t6INc790mU/8+Vl
 kyAtrhwkwam3iybdlzRMxiOqtrHBP5CkJ7yBgR0VX7A5pTpSd/HAH43e8RlcL852vZkyPIoH
 fAJd9/ZWqZESyjd+iRbZp749dQweBOujAOIHiykfDlgIMIwG12XoofpLln16S0DLiurrs9i8
 beu4RzWHMgYTAN4AceINP/2lwGtvWIQkf5ZVlfTJoUBY13l9YVncnSjjvI+L8wWBw/Ewz+Wi
 1SfDRsC/LGfqI4p6tjZw6uDqt7xQed5G0NbGUjd7Kq3anaGrjbyn9cYXb/RLz7HVW7y9KGzX
 slvzqnxYK8dgVJHk4tgCLI3n6gw0MTi+u1BxQN+EXSVM1nyUuF8ImOL1NVkv7FWwuMLohO/X
 0+C94UIObiNP8+5QlcdKBB8M7aG3PAQ3DLT8e40MAPx4youpOiLVkBbPh+tji1BLeQqbNN5k
 Ll54MNGuRaijhcKM8qdinEG/muBGXUMTqE7u8xIG4TskAcqlglPbJG05vUaO31ThwGg6nUXH
 wI=
IronPort-HdrOrdr: A9a23:Dqtkna1wdC6jgNq1E38FPAqjBLAkLtp133Aq2lEZdPRUGvb3qy
 nOpoVj6faaslYssR0b9exofZPwJE80lqQFh7X5X43SPzUO0VHAROoJgLcKgQeQfxEWntQtsp
 uIGJIeNDSfNzdHZL7BkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="54750987"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 4/6] x86/trace: Reduce stack usage from HVMTRACE_ND()
Date: Fri, 17 Sep 2021 09:45:57 +0100
Message-ID: <20210917084559.22673-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

It is pointless to write all 6 entries and only consume the useful subset.
bloat-o-meter shows quite how obscene the overhead is in vmx_vmexit_handler(),
weighing in at 11% of the function arranging unread zeroes on the stack, and
8% for svm_vmexit_handler().

  add/remove: 0/0 grow/shrink: 0/20 up/down: 0/-1867 (-1867)
  Function                                     old     new   delta
  hvm_msr_write_intercept                     1049    1033     -16
  vmx_enable_intr_window                       238     214     -24
  svm_enable_intr_window                       337     313     -24
  hvmemul_write_xcr                            115      91     -24
  hvmemul_write_cr                             350     326     -24
  hvmemul_read_xcr                             115      91     -24
  hvmemul_read_cr                              146     122     -24
  hvm_mov_to_cr                                438     414     -24
  hvm_mov_from_cr                              253     229     -24
  vmx_intr_assist                             1150    1118     -32
  svm_intr_assist                              459     427     -32
  hvm_rdtsc_intercept                          138     106     -32
  hvm_msr_read_intercept                       898     866     -32
  vmx_vmenter_helper                          1142    1094     -48
  vmx_inject_event                             813     765     -48
  svm_vmenter_helper                           238     190     -48
  hvm_hlt                                      197     146     -51
  svm_inject_event                            1678    1614     -64
  svm_vmexit_handler                          5880    5416    -464
  vmx_vmexit_handler                          7281    6473    -808
  Total: Before=3644184, After=3642317, chg -0.05%

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>

Normally I wouldn't recommend patches like this for backport, but
{vmx,svm}_vmexit_handler() are fastpaths and this is a *lot* of I-cache lines
dropped...
---
 xen/arch/x86/hvm/svm/svm.c      |  8 +++-----
 xen/arch/x86/hvm/vmx/vmx.c      |  9 ++++-----
 xen/include/asm-x86/hvm/trace.h | 28 ++++++++++------------------
 3 files changed, 17 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index afb1ccb342c2..f0e10dec046e 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1052,7 +1052,7 @@ void svm_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(tb_init_done) )
         HVMTRACE_ND(VMENTRY,
                     nestedhvm_vcpu_in_guestmode(curr) ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 0, 0, 0, 0, 0, 0, 0);
+                    1/*cycles*/);
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
@@ -2565,12 +2565,10 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
 
     if ( hvm_long_mode_active(v) )
         HVMTRACE_ND(VMEXIT64, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 3, exit_reason,
-                    regs->eip, regs->rip >> 32, 0, 0, 0);
+                    1/*cycles*/, exit_reason, TRC_PAR_LONG(regs->rip));
     else
         HVMTRACE_ND(VMEXIT, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 2, exit_reason,
-                    regs->eip, 0, 0, 0, 0);
+                    1/*cycles*/, exit_reason, regs->eip);
 
     if ( vcpu_guestmode )
     {
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index b0a42d05f86a..d403e2d8060a 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3864,11 +3864,10 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
     __vmread(VM_EXIT_REASON, &exit_reason);
 
     if ( hvm_long_mode_active(v) )
-        HVMTRACE_ND(VMEXIT64, 0, 1/*cycles*/, 3, exit_reason,
-                    regs->eip, regs->rip >> 32, 0, 0, 0);
+        HVMTRACE_ND(VMEXIT64, 0, 1/*cycles*/, exit_reason,
+                    TRC_PAR_LONG(regs->rip));
     else
-        HVMTRACE_ND(VMEXIT, 0, 1/*cycles*/, 2, exit_reason,
-                    regs->eip, 0, 0, 0, 0);
+        HVMTRACE_ND(VMEXIT, 0, 1/*cycles*/, exit_reason, regs->eip);
 
     perfc_incra(vmexits, exit_reason);
 
@@ -4645,7 +4644,7 @@ bool vmx_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(curr->arch.hvm.vmx.lbr_flags & LBR_FIXUP_MASK) )
         lbr_fixup();
 
-    HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/, 0, 0, 0, 0, 0, 0, 0);
+    HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/);
 
     __vmwrite(GUEST_RIP,    regs->rip);
     __vmwrite(GUEST_RSP,    regs->rsp);
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
index 5cd459b855b7..2bbac45044ce 100644
--- a/xen/include/asm-x86/hvm/trace.h
+++ b/xen/include/asm-x86/hvm/trace.h
@@ -67,38 +67,30 @@
 #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
     TRACE_6D(_e, d1, d2, d3, d4)
 
-#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d6) \
+#define HVMTRACE_ND(evt, modifier, cycles, ...)                           \
     do {                                                                  \
         if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )               \
         {                                                                 \
-            struct {                                                      \
-                u32 d[6];                                                 \
-            } _d;                                                         \
-            _d.d[0]=(d1);                                                 \
-            _d.d[1]=(d2);                                                 \
-            _d.d[2]=(d3);                                                 \
-            _d.d[3]=(d4);                                                 \
-            _d.d[4]=(d5);                                                 \
-            _d.d[5]=(d6);                                                 \
+            uint32_t _d[] = { __VA_ARGS__ };                              \
             __trace_var(TRC_HVM_ ## evt | (modifier), cycles,             \
-                        sizeof(*_d.d) * count, &_d);                      \
+                        sizeof(_d), _d);                                  \
         }                                                                 \
     } while(0)
 
 #define HVMTRACE_6D(evt, d1, d2, d3, d4, d5, d6)    \
-    HVMTRACE_ND(evt, 0, 0, 6, d1, d2, d3, d4, d5, d6)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5, d6)
 #define HVMTRACE_5D(evt, d1, d2, d3, d4, d5)        \
-    HVMTRACE_ND(evt, 0, 0, 5, d1, d2, d3, d4, d5,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5)
 #define HVMTRACE_4D(evt, d1, d2, d3, d4)            \
-    HVMTRACE_ND(evt, 0, 0, 4, d1, d2, d3, d4,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4)
 #define HVMTRACE_3D(evt, d1, d2, d3)                \
-    HVMTRACE_ND(evt, 0, 0, 3, d1, d2, d3,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3)
 #define HVMTRACE_2D(evt, d1, d2)                    \
-    HVMTRACE_ND(evt, 0, 0, 2, d1, d2,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2)
 #define HVMTRACE_1D(evt, d1)                        \
-    HVMTRACE_ND(evt, 0, 0, 1, d1,  0,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1)
 #define HVMTRACE_0D(evt)                            \
-    HVMTRACE_ND(evt, 0, 0, 0,  0,  0,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0)
 
 #define HVMTRACE_LONG_1D(evt, d1)                  \
                    HVMTRACE_2D(evt ## 64, (d1) & 0xFFFFFFFF, (d1) >> 32)
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189097.338648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vr-00056J-I3; Fri, 17 Sep 2021 08:46:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189097.338648; Fri, 17 Sep 2021 08:46:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vr-00055z-E5; Fri, 17 Sep 2021 08:46:27 +0000
Received: by outflank-mailman (input) for mailman id 189097;
 Fri, 17 Sep 2021 08:46:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vq-0003AJ-4H
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:26 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 96d0679e-c637-4476-adba-b04b33a520e1;
 Fri, 17 Sep 2021 08:46:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96d0679e-c637-4476-adba-b04b33a520e1
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868372;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=eIm9X8tHd1Tqh1vwMRrxy3l4K0nxBP+YaEDaO0VDgfI=;
  b=KftQD36peUPJ75JYLY91bHFCiJ+gQx7lNAjmct/xfe7LFvbeRDKg6l9S
   8SGpUsws5uTrvO0hqYBi6Kd6KHuc5w6RCzLycp6vHQmVyRrVCD4PzV89j
   QULSfk0QnY+CraBjSc8XEchZg5p5L/yK9ChHTdioR20pOkum301/ITw0c
   Q=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: AJGC5ytMX6wDLwBH5aceLiWn1K3VQH9POy2hpb/QiQBV4/hUkdAF0vqbjdjPhmWoGCGRIVpE20
 9mHYMWtG4BpnsAvWjRuTb83hR3GuIURHYjY3Y31lMaucVzU31sIGV6Ug3xViXkzJnOtex/LkrO
 E3oXF+BAbd3EPfq2sI5quzo700VPjqn/C9/x7EhxuJypMc/WfcLkbkVvfCl+pYBK5kElPeQRht
 AKpdCgVwhJi2+L8mqYnbo3tnGMXWB9n4KCus+MSWLQ412asFds6+9YhQF5yxTI/JK2Z98MebNy
 Il9BX7LVvjCTVmu+aUhgZJgP
X-SBRS: 5.1
X-MesageID: 52955354
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:mKrjQ6A8Zb4okBVW/9Xkw5YqxClBgxIJ4kV8jS/XYbTApDwmgTRTy
 2oWDG2POf2LZ2amLox1boy08ExSu8TQx9NrQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0E07wobVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/1Gy4ptpP2
 PV05IWTViERZpeLp98WTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFhmZt25sTQZ4yY
 eISaDZPRTDmYyFQEWkVA64iwKCOqGvgJmgwRFW9+vNsvjm7IBZK+LrwNNvYfPSaSMMTmVyXz
 krk1WnkBhARNPSE1CGItHmrg4fnkS3yV40TE7m53uV3m1DVzWsWYDUJUXOrrP//jVSxM/pPJ
 kpR9icwoKwa8E2wUsK7TxC+uGSDvBMXR5xXCeJSwACSyILE7gCBHG8GQzVdLts8u6ceXic23
 1WEm9foAz1Hs7CPT3+ZsLCOoluP1TM9dDFYI3VeFE1cvoel8NpbYg/zoshLT/eEt9fLCRHJk
 impsw4G349UsvUAyPDulbzYuA5AtqQlXyZsuF6OAzP6sV0gDGK2T9f3sgmAtJ6sOK7cFwPY7
 SZew6By+chTVcnlqcCbfAka8FhFDd6+OTvAiBZEG5A7/lxBEFbyINgNvFmSyKpvW/vomAMFg
 meI4mu9B7cJZRNGiJObharqU6zGKoC6TrzYugj8NIYmX3SIXFbvENtSiausM4bFyxBEfUYXY
 szzTCpRJSxCVfQPIMSeHr9AuVPU+szO7TyKHs2qp/hW+ZGfeGSUWd843KimN7tihJ5oVD79q
 o4FX+PTkk03eLSnPkH/rN5CRXhXfCNTLc2n9KRqmhurf1MO9JcJUKSKn9vMuuVNwsxoqws/1
 irhAxQJlAWm3iSvxMfjQikLVY4DlK1X9RoTVRHA937xs5T6SYrwvqoZabUterwrqL5qwfJuF
 qFXcMScGPVfDD/A/m1FP5X6qYVjcjWthB6PYHX5MGRuIcY4Slyb4MLgcyvu6DIKUni9u/whr
 uDyzQjcW5cCGVhvVZ6EdPK1wlqtlnEBg+YuDVDQK9xedRy0oohnIiD8lNEtJMQIJUmRzzeWz
 V/OUxwZufPMs8k+99yQ3fKIqIKgEu1fGEtGHjaEsebqZHeCpmf6mN1OSueFezzZRVjYwqT6a
 LUH1ez4Pd0GgE1O79h2HYF0wP9s/NDovbJbkFhpRS2Zc1SxB7p8CXCaxs0T5LZVz7pUtAbqC
 EKC/t5WZeeANM//SQNDIQMkaqKI1O0OmymU5vMweR2o6Chy9buBcENTIxjT13ANcOoraNsok
 bU7pcobyw2jkR57YN+Jgxdd+3mIMnFdAb4ssYsXAdOzhwcmor2YjUcw1sMiDEmzVuhx
IronPort-HdrOrdr: A9a23:YxKC4KCa6fkXarflHemo55DYdb4zR+YMi2TC1yhKKCC9E/bo7/
 xG885rtiMc5Ax/ZJhko6HlBEDiewKkyXcW2/hyAV7KZmCP0wHEQL2Kr7GSpAEIcxeOkNK1vp
 0BT0ERMrPN5CBB/KPH3DU=
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="52955354"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 3/6] xen/credit2: Remove tail padding from TRC_CSCHED2_* records
Date: Fri, 17 Sep 2021 09:45:56 +0100
Message-ID: <20210917084559.22673-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

All three of these records have tail padding, leaking stack rubble into the
trace buffer.  Introduce an explicit _pad field and have the compiler zero the
padding automatically.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/common/sched/credit2.c | 46 +++++++++++++++++++++++++++-------------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index 6396b38e044c..d5f41bc3d603 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -1106,12 +1106,14 @@ _runq_assign(struct csched2_unit *svc, struct csched2_runqueue_data *rqd)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned rqi:16;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.rqi=rqd->id;
+            uint16_t unit, dom;
+            uint16_t rqi, _pad;
+        } d = {
+            .unit = svc->unit->unit_id,
+            .dom  = svc->unit->domain->domain_id,
+            .rqi  = rqd->id,
+        };
+
         __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1,
                     sizeof(d),
                     (unsigned char *)&d);
@@ -1336,13 +1338,16 @@ update_runq_load(const struct scheduler *ops,
     {
         struct {
             uint64_t rq_avgload, b_avgload;
-            unsigned rq_load:16, rq_id:8, shift:8;
-        } d;
-        d.rq_id = rqd->id;
-        d.rq_load = rqd->load;
-        d.rq_avgload = rqd->avgload;
-        d.b_avgload = rqd->b_avgload;
-        d.shift = P;
+            uint16_t rq_load; uint8_t rq_id, shift;
+            uint32_t _pad;
+        } d = {
+            .rq_avgload  = rqd->avgload,
+            .b_avgload   = rqd->b_avgload,
+            .rq_load     = rqd->load,
+            .rq_id       = rqd->id,
+            .shift       = P,
+        };
+
         __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1,
                     sizeof(d),
                     (unsigned char *)&d);
@@ -2799,12 +2804,15 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
     {
         struct {
             uint64_t lb_avgload, ob_avgload;
-            unsigned lrq_id:16, orq_id:16;
-        } d;
-        d.lrq_id = st.lrqd->id;
-        d.lb_avgload = st.lrqd->b_avgload;
-        d.orq_id = st.orqd->id;
-        d.ob_avgload = st.orqd->b_avgload;
+            uint16_t lrq_id, orq_id;
+            uint32_t _pad;
+        } d = {
+            .lb_avgload  = st.lrqd->b_avgload,
+            .ob_avgload  = st.orqd->b_avgload,
+            .lrq_id      = st.lrqd->id,
+            .orq_id      = st.orqd->id,
+        };
+
         __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1,
                     sizeof(d),
                     (unsigned char *)&d);
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189095.338626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vm-0004Lx-GT; Fri, 17 Sep 2021 08:46:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189095.338626; Fri, 17 Sep 2021 08:46:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vm-0004Lo-Cd; Fri, 17 Sep 2021 08:46:22 +0000
Received: by outflank-mailman (input) for mailman id 189095;
 Fri, 17 Sep 2021 08:46:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vl-0003AJ-49
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:21 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7a06d743-cc27-45d6-9aab-e8e2e547bf81;
 Fri, 17 Sep 2021 08:46:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a06d743-cc27-45d6-9aab-e8e2e547bf81
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868369;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=zVahpMkV6GyvrVUFEu34Z/ndYjYizS9n7x26jqVkpFU=;
  b=aBM2VIopfqekS36WShbxBBpgWMCicwrynxcssFi5F+hhLNGefrH+yYaQ
   IxOsvheEHIIcd0CYz9/WBbRZetchMjSLAHZhmD0GSg7Eo2QrXvPf8Ezwz
   kr0cR77TJlmDTgkYAg1OK976L0E8j1aHJnQ0mv4G6EkQ1Ur68z4smaVDZ
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: CDt0mO0rf3EoaTbkta+U/lvgO+b4WKpUXYKghOLVGWuQqhX33jqtb9tgonMSHf1c+uf2JpJ2kx
 xI3t1OKSdI1684wL8W47oRlCRkbY1DMVqaQVqng9Ko8ysxAR2ItdDd6Wcp1H6b+UpDDTb2SGUW
 oW2E/XvwzakWEiLDF6dA782DLuoG6bfSGm1/hUkyYMwK0lNlVhYIsPzE0d4yEoUF/Tw/N14Lzd
 g9cunEejVLp0/yK0dFiEU3Edj5MShEXmLhP3/NEvGyQSkgM1sRvQSAGmgAhK6bV0tZGPihcb/N
 iP3Ry+j2elBtNw0uDlLAke4E
X-SBRS: 5.1
X-MesageID: 52983294
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:JLr7La3mmy4cUVnHhfbD5S92kn2cJEfYwER7XKvMYLTBsI5bp2NRz
 GRNWDiDOfiCY2qjf4gka9+1p0JXvpSDy4JhHAQ6pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrFh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhn/Vp9
 tBpjseMVgICZIzuwbkMcQhpHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t3JEeRqyPO
 6L1bxIscRqeQQNIPGxUVpYdvuuGg1T2Uzdh/Qf9Sa0fvDGIkV0ZPKLWGNjIft2HQ+1Fk0Deo
 XjJl0zbKBwHMN2UyRKe72mhwOTImEvTVIwbG7K58fprqEaO3WFVAxoTPXOkpdGph0j4XMhQQ
 2QE9yxroaUs+UiDStjmQwb+sHOCpgQbWddbD6s98g7l4qvL4S6JC24cVDlDZdc68sgsSlQCz
 USVltnkAThutry9Sn+H8LqQ6zSoNkAowXQqPHFeC1Ffup+6/d913kmnostf/LCdjvjtJzTU+
 Gu2sy0V3ZQx1JMa9bSK8gWS696znaQlXjLZ9y2OADn8s1ImO9D8D2C7wQOEtqcbde51WnHE5
 SJdypbEtIjiGLnQzHTlfQkbIF2+Cx9p2hXniFhzFtEK8z238hZPlqgBvWkjeC+F3isCEAIFg
 XM/WysKv/e/31PwNMebhr5d7exxlcAM8vy/DZjpgiJmOMQZSeN+1HgGibSsM4XRfK4EyvlX1
 XCzKp3EMJrnIf4/kGreqxk1iOd2rszB+Y8jbc+ilEn2uVZvTFWUVa0EIDOzghMRtfjfyDg5B
 +13bpPQoz0GCbWWSnCOreY7cABbRVBmVMueg5EGKYa+zv9ORThJ5wn5muh6JeSIXs19y4/1w
 51KchQEkQGu2SKeclzih7IKQOqHYKuTZEkTZUQEVWtEEVB5CWp2xKtAJZYxY5c98+lvkax9Q
 /UfIp3SCfVTUDXXvT8aaMCl/oBlcR2qgyOIPjakP2djL8IxGVSR94+2ZBbr+QkPEjGz6Zk0r
 Yq/216JWpEEXQljUprbMar901OrsHEBs+tuRE+UcMJLcUDh/dEyeSz8h/M6Oe8WLhDHymfI3
 gqaG05A9+LMv5U04J/CgqXd99WlFO53H0x7GWjH7OnpaXmGrzT7mYIZCbSGZzHQUm/w6Z6OX
 +QNwqGuKuADkXZLr5F4T+Rhw5Uh6oa9vLRd1AllQinGNgz5FrN6L3Ca9sBTrakRlKRBsA67V
 0/TqNlXPbKFZJHsHFILfVd3a+2C0bcfmyXI7ORzK0L/vXcl8L2CWERUHh+NlC0Cc+clbNJ7m
 b8s6JwM9giyqhs2KdLX3Clb+lOFImEET6h65IoRB5Xmi1Zzx1xPCXAG5vQaPH1bhw1wD3QX
IronPort-HdrOrdr: A9a23:JOTANqoHt9lG92aj+wcptLgaV5opeYIsimQD101hICG8cqSj+f
 xG/c5rrCMc5wxwZJhNo7y90ey7MBbhHP1OkO8s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpM
 BdmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="52983294"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 5/6] xen/credit2: Clean up trace handling
Date: Fri, 17 Sep 2021 09:45:58 +0100
Message-ID: <20210917084559.22673-6-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210917084559.22673-1-andrew.cooper3@citrix.com>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

There is no need for bitfields anywhere - use more sensible types.  There is
also no need to cast 'd' to (unsigned char *) before passing it to a function
taking void *.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/common/sched/credit2.c | 297 ++++++++++++++++++++++-----------------------
 1 file changed, 144 insertions(+), 153 deletions(-)

diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index d5f41bc3d603..339b9fd75926 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -1080,13 +1080,13 @@ static void update_max_weight(struct csched2_runqueue_data *rqd, int new_weight,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned rqi:16, max_weight:16;
-        } d;
-        d.rqi = rqd->id;
-        d.max_weight = rqd->max_weight;
-        __trace_var(TRC_CSCHED2_RUNQ_MAX_WEIGHT, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t rqi, max_weight;
+        } d = {
+            .rqi         = rqd->id,
+            .max_weight  = rqd->max_weight,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_MAX_WEIGHT, 1, sizeof(d), &d);
     }
 }
 
@@ -1114,9 +1114,7 @@ _runq_assign(struct csched2_unit *svc, struct csched2_runqueue_data *rqd)
             .rqi  = rqd->id,
         };
 
-        __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1, sizeof(d), &d);
     }
 
 }
@@ -1348,9 +1346,7 @@ update_runq_load(const struct scheduler *ops,
             .shift       = P,
         };
 
-        __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1, sizeof(d), &d);
     }
 }
 
@@ -1400,16 +1396,16 @@ update_svc_load(const struct scheduler *ops,
     {
         struct {
             uint64_t v_avgload;
-            unsigned unit:16, dom:16;
-            unsigned shift;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.v_avgload = svc->avgload;
-        d.shift = P;
-        __trace_var(TRC_CSCHED2_UPDATE_UNIT_LOAD, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t shift;
+        } d = {
+            .v_avgload  = svc->avgload,
+            .unit       = svc->unit->unit_id,
+            .dom        = svc->unit->domain->domain_id,
+            .shift      = P,
+        };
+
+        __trace_var(TRC_CSCHED2_UPDATE_UNIT_LOAD, 1, sizeof(d), &d);
     }
 }
 
@@ -1456,15 +1452,15 @@ static void runq_insert(struct csched2_unit *svc)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned pos;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.pos = pos;
-        __trace_var(TRC_CSCHED2_RUNQ_POS, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t pos;
+        } d = {
+            .unit  = svc->unit->unit_id,
+            .dom   = svc->unit->domain->domain_id,
+            .pos   = pos,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_POS, 1, sizeof(d), &d);
     }
 }
 
@@ -1556,16 +1552,16 @@ static s_time_t tickle_score(const struct scheduler *ops, s_time_t now,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            int credit, score;
-        } d;
-        d.dom = cur->unit->domain->domain_id;
-        d.unit = cur->unit->unit_id;
-        d.credit = cur->credit;
-        d.score = score;
-        __trace_var(TRC_CSCHED2_TICKLE_CHECK, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t credit, score;
+        } d = {
+            .unit    = cur->unit->unit_id,
+            .dom     = cur->unit->domain->domain_id,
+            .credit  = cur->credit,
+            .score   = score,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE_CHECK, 1, sizeof(d), &d);
     }
 
     return score;
@@ -1603,17 +1599,16 @@ runq_tickle(const struct scheduler *ops, struct csched2_unit *new, s_time_t now)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned processor;
-            int credit;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.processor = cpu;
-        d.credit = new->credit;
-        __trace_var(TRC_CSCHED2_TICKLE_NEW, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t processor, credit;
+        } d = {
+            .dom        = unit->domain->domain_id,
+            .unit       = unit->unit_id,
+            .processor  = cpu,
+            .credit     = new->credit,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE_NEW, 1, sizeof(d), &d);
     }
 
     /*
@@ -1752,12 +1747,12 @@ runq_tickle(const struct scheduler *ops, struct csched2_unit *new, s_time_t now)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, pad:16;
-        } d;
-        d.cpu = ipid; d.pad = 0;
-        __trace_var(TRC_CSCHED2_TICKLE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t cpu, _pad;
+        } d = {
+            .cpu = ipid,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE, 1, sizeof(d), &d);
     }
 
     tickle_cpu(ipid, rqd);
@@ -1833,16 +1828,16 @@ static void reset_credit(int cpu, s_time_t now, struct csched2_unit *snext)
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-                int credit_start, credit_end;
-            } d;
-            d.dom = svc->unit->domain->domain_id;
-            d.unit = svc->unit->unit_id;
-            d.credit_start = start_credit;
-            d.credit_end = svc->credit;
-            __trace_var(TRC_CSCHED2_CREDIT_RESET, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+                uint32_t credit_start, credit_end;
+            } d = {
+                .unit          = svc->unit->unit_id,
+                .dom           = svc->unit->domain->domain_id,
+                .credit_start  = start_credit,
+                .credit_end    = svc->credit,
+            };
+
+            __trace_var(TRC_CSCHED2_CREDIT_RESET, 1, sizeof(d), &d);
         }
     }
 
@@ -1888,18 +1883,17 @@ void burn_credits(struct csched2_runqueue_data *rqd,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            int credit, budget;
-            int delta;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.credit = svc->credit;
-        d.budget = has_cap(svc) ?  svc->budget : INT_MIN;
-        d.delta = delta;
-        __trace_var(TRC_CSCHED2_CREDIT_BURN, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t credit, budget, delta;
+        } d = {
+            .unit    = svc->unit->unit_id,
+            .dom     = svc->unit->domain->domain_id,
+            .credit  = svc->credit,
+            .budget  = has_cap(svc) ?  svc->budget : INT_MIN,
+            .delta   = delta,
+        };
+
+        __trace_var(TRC_CSCHED2_CREDIT_BURN, 1, sizeof(d), &d);
     }
 }
 
@@ -2544,17 +2538,17 @@ csched2_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
     {
         struct {
             uint64_t b_avgload;
-            unsigned unit:16, dom:16;
-            unsigned rq_id:16, new_cpu:16;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.rq_id = min_rqd ? min_rqd->id : -1;
-        d.b_avgload = min_avgload;
-        d.new_cpu = new_cpu;
-        __trace_var(TRC_CSCHED2_PICKED_CPU, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint16_t rq_id, new_cpu;
+        } d = {
+            .b_avgload  = min_avgload,
+            .unit       = unit->unit_id,
+            .dom        = unit->domain->domain_id,
+            .rq_id      = min_rqd ? min_rqd->id : -1,
+            .new_cpu    = new_cpu,
+        };
+
+        __trace_var(TRC_CSCHED2_PICKED_CPU, 1, sizeof(d), &d);
     }
 
     return get_sched_res(new_cpu);
@@ -2615,16 +2609,16 @@ static void migrate(const struct scheduler *ops,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned rqi:16, trqi:16;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.rqi = svc->rqd->id;
-        d.trqi = trqd->id;
-        __trace_var(TRC_CSCHED2_MIGRATE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint16_t rqi, trqi;
+        } d = {
+            .unit  = unit->unit_id,
+            .dom   = unit->domain->domain_id,
+            .rqi   = svc->rqd->id,
+            .trqi  = trqd->id,
+        };
+
+        __trace_var(TRC_CSCHED2_MIGRATE, 1, sizeof(d), &d);
     }
 
     if ( svc->flags & CSFLAG_scheduled )
@@ -2761,15 +2755,15 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned lrq_id:16, orq_id:16;
-                unsigned load_delta;
-            } d;
-            d.lrq_id = st.lrqd->id;
-            d.orq_id = st.orqd->id;
-            d.load_delta = st.load_delta;
-            __trace_var(TRC_CSCHED2_LOAD_CHECK, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t lrq_id, orq_id;
+                uint32_t load_delta;
+            } d = {
+                .lrq_id      = st.lrqd->id,
+                .orq_id      = st.orqd->id,
+                .load_delta  = st.load_delta,
+            };
+
+            __trace_var(TRC_CSCHED2_LOAD_CHECK, 1, sizeof(d), &d);
         }
 
         /*
@@ -2813,9 +2807,7 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
             .orq_id      = st.orqd->id,
         };
 
-        __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1, sizeof(d), &d);
     }
 
     SCHED_STAT_CRANK(acct_load_balance);
@@ -3401,15 +3393,15 @@ runq_candidate(struct csched2_runqueue_data *rqd,
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-                unsigned runtime;
-            } d;
-            d.dom = scurr->unit->domain->domain_id;
-            d.unit = scurr->unit->unit_id;
-            d.runtime = now - scurr->unit->state_entry_time;
-            __trace_var(TRC_CSCHED2_RATELIMIT, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+                uint32_t runtime;
+            } d = {
+                .unit     = scurr->unit->unit_id,
+                .dom      = scurr->unit->domain->domain_id,
+                .runtime  = now - scurr->unit->state_entry_time,
+            };
+
+            __trace_var(TRC_CSCHED2_RATELIMIT, 1, sizeof(d), &d);
         }
         return scurr;
     }
@@ -3462,13 +3454,13 @@ runq_candidate(struct csched2_runqueue_data *rqd,
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-            } d;
-            d.dom = svc->unit->domain->domain_id;
-            d.unit = svc->unit->unit_id;
-            __trace_var(TRC_CSCHED2_RUNQ_CAND_CHECK, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+            } d = {
+                .unit  = svc->unit->unit_id,
+                .dom   = svc->unit->domain->domain_id,
+            };
+
+            __trace_var(TRC_CSCHED2_RUNQ_CAND_CHECK, 1, sizeof(d), &d);
         }
 
         /*
@@ -3536,17 +3528,16 @@ runq_candidate(struct csched2_runqueue_data *rqd,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned tickled_cpu;
-            int credit;
-        } d;
-        d.dom = snext->unit->domain->domain_id;
-        d.unit = snext->unit->unit_id;
-        d.credit = snext->credit;
-        d.tickled_cpu = snext->tickled_cpu;
-        __trace_var(TRC_CSCHED2_RUNQ_CANDIDATE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t tickled_cpu, credit;
+        } d = {
+            .unit         = snext->unit->unit_id,
+            .dom          = snext->unit->domain->domain_id,
+            .tickled_cpu  = snext->tickled_cpu,
+            .credit       = snext->credit,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_CANDIDATE, 1, sizeof(d), &d);
     }
 
     if ( unlikely(snext->tickled_cpu != -1 && snext->tickled_cpu != cpu) )
@@ -3602,18 +3593,18 @@ static void csched2_schedule(
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, rq_id:16;
-            unsigned tasklet:8, idle:8, smt_idle:8, tickled:8;
-        } d;
-        d.cpu = cur_cpu;
-        d.rq_id = c2r(sched_cpu);
-        d.tasklet = tasklet_work_scheduled;
-        d.idle = is_idle_unit(currunit);
-        d.smt_idle = cpumask_test_cpu(sched_cpu, &rqd->smt_idle);
-        d.tickled = tickled;
-        __trace_var(TRC_CSCHED2_SCHEDULE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t cpu, rq_id;
+            uint8_t tasklet, idle, smt_idle, tickled;
+        } d = {
+            .cpu       = cur_cpu,
+            .rq_id     = c2r(sched_cpu),
+            .tasklet   = tasklet_work_scheduled,
+            .idle      = is_idle_unit(currunit),
+            .smt_idle  = cpumask_test_cpu(sched_cpu, &rqd->smt_idle),
+            .tickled   = tickled,
+        };
+
+        __trace_var(TRC_CSCHED2_SCHEDULE, 1, sizeof(d), &d);
     }
 
     /* Update credits (and budget, if necessary). */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 08:46:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 08:46:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189092.338587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vd-0003Cq-Gs; Fri, 17 Sep 2021 08:46:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189092.338587; Fri, 17 Sep 2021 08:46:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mR9Vd-0003CG-BV; Fri, 17 Sep 2021 08:46:13 +0000
Received: by outflank-mailman (input) for mailman id 189092;
 Fri, 17 Sep 2021 08:46:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mR9Vb-0003AK-LB
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 08:46:11 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b3d53b2b-1793-11ec-b674-12813bfff9fa;
 Fri, 17 Sep 2021 08:46:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3d53b2b-1793-11ec-b674-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631868369;
  h=from:to:cc:subject:date:message-id:mime-version;
  bh=VrbXBQo8s3hHOV5Jry2hT2yLKMoeTSgjl76QM+cDmXQ=;
  b=bPdHE0RjGqBewxIa+Pf8G8g1g847MYsApHca9R468l+Bpu+r72zUoq1f
   o1Qm7yHRVMyD2THH3S7v4YuDxKMeOrqnhN+3wyNTCkMtp6l+YKS34BDmf
   2VjyqiD/jOFK61g1rqCQ07sRQbPcbPCZZfvYCir3Ze9zmPewb93QVC4Q0
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: QUyhN+jlEdxEXNrvPpd1KLdwoLYz96c6X3mIsq8p/QyRPzrbUWk1j3asbC7SgUZJz2p1umetlx
 Mnn5DWcpmKyiEl2p2XS/yzwkFbQqyu8HwaqcwdvuRNvT4i9Z4qYQ4x/yBm1FZLOxuHAAhw5xtV
 XCTZzw+5QT/ysqv57cx5Rf27ZRFA1wvk2w/G78z6Wcup0/uy6+l1yHudsy1Joo8/022+G3YTF3
 rde0ys0UAMoa+h0bB1O2WyqachXn4WcqBocU2+zna1NATqweWyuKBrhXZUXsY90FYK46O4zAlm
 qGBWvu+bhlaYuK0wLnP2LtUm
X-SBRS: 5.1
X-MesageID: 53378055
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:V1A1zK0Mu9L6p2VqufbD5S92kn2cJEfYwER7XKvMYLTBsI5bpz0Bn
 WUZWmHQbPeDYWekLdx0OYy0pktS68DWnYUyHVM5pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrFh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhzvJXk
 oxLn66JbzwgOoztiOQaYRN2OnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t3ZsRTa6DO
 qL1bxI3NBWdWw8IG2wLCas8m8X2nSbmMAJh/Qf9Sa0fvDGIkV0ZPKLWGNjIft2HQ+1Fk0Deo
 XjJl0zbKBwHMN2UyRKe72mhwOTImEvTVIwbG7K58fprqEaO3WFVAxoTPXOkpdGph0j4XMhQQ
 2QE9yxroaUs+UiDStjmQwb+sHOCpgQbWddbD6s98g7l4qvL4S6JC24cVDlDZdc68sgsSlQCz
 USVltnkAThutry9Sn+H8LqQ6zSoNkAowXQqPHFeC1Ffup+6/d913kmnostf/LCd3//TQ2j26
 QKzihM3g5Qync4M5a+V1AWS696znaQlXjLZ9y2OADn8s1ImO9D8D2C7wQOEtqcbde51WnHE5
 SJdypbEtIjiGLnQzHTlfQkbIF2+Cx9p2hXniFhzFtEK8z238hZPlqgBvWkjeC+F3isCEAIFg
 XM/WysKv/e/31PwNMebhr5d7exxlcAM8vy/DZjpgiJmOMQZSeN+1HgGibSsM4XRfK4EyvlX1
 XCzKp3EMJrnIf4/kGreqxk1iOd2rszB+Y8jbc+ilEn2uVZvTFWUVa0EIDOzghMRtfjfyDg5B
 +13bpPQoz0GCbWWSnCOreY7cABbRVBmVMueg5EGKYa+zv9ORThJ5wn5muh6JeSIXs19y4/1w
 51KchQEkQGu2SKeclzih7IKQOqHYKuTZEkTZUQEVWtEEVB6CWp2xKtAJZYxY5c98+lvkax9Q
 /UfIp3SCfVTUDXXvT8aaMCl/oBlcR2qgyOIPjakP2djL8IxGVSR94+2ZBbr+QkPEjGz6Zk0r
 Yq/216JWpEEXQljUprbMar901OrsHEBs+tuRE+UcMJLcUDh/dEyeSz8h/M6Oe8WLhDHymfI3
 gqaG05A9+LMv5U04J/CgqXd99WlFO53H0x7GWjH7OnpaXmGrzT7mYIZCbSGZzHQUm/w6Z6OX
 +QNwqGuKuADkXZLr5F4T+Rhw5Uh6oa9vLRd1AllQinGNgz5FrN6L3Ca9sBTrakRlKRBsA67V
 0/TqNlXPbKFZJHsHFILfVd3a+2C0bcfmyXI7ORzK0L/vXcl8L2CWERUHh+NlC0Cc+clbNJ7m
 b8s6JwM9giyqhs2KdLX3Clb+lOFImEET6h65IoRB5Xmi1Zzx1xPCXAG5vQaPH1bhw1wD3QX
IronPort-HdrOrdr: A9a23:PL0HYqArr5CIKgDlHem655DYdb4zR+YMi2TC1yhKJyC9E/bo8P
 xG88566faZslossRIb6LS90cu7MBDhHPdOiOF7V9qftWHdyQ6VxepZjLcKrQeOJ8SHzJ8+6Z
 td
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="53378055"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH 0/6] xen/trace: Fix leakage of uninitialised stack into the tracebuffer
Date: Fri, 17 Sep 2021 09:45:53 +0100
Message-ID: <20210917084559.22673-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain

Patches 1-3 fix actual or latent bugs causing uninitialised stack to leak into
the trace buffers.  Xentrace is a developer/debugging activity restricted to
fully privileged entities, so the leaking of uninitialised stack contents is
not a security concern here.

Patches 4-6 are various pieces of cleanup.  This entire subsystem is a mess,
but the practical gains in patch 4 speak for themselves.

Andrew Cooper (6):
  xen/trace: Don't over-read trace objects
  xen/memory: Remove tail padding from TRC_MEM_* records
  xen/credit2: Remove tail padding from TRC_CSCHED2_* records
  x86/trace: Reduce stack usage from HVMTRACE_ND()
  xen/credit2: Clean up trace handling
  xen/trace: Minor code cleanup

 tools/xentrace/formats          |   4 +
 tools/xentrace/xenalyze.c       |  12 +-
 xen/arch/x86/hvm/svm/svm.c      |   8 +-
 xen/arch/x86/hvm/vmx/vmx.c      |   9 +-
 xen/arch/x86/mm/p2m-pod.c       |  17 +-
 xen/common/memory.c             |   4 +-
 xen/common/sched/credit2.c      | 343 ++++++++++++++++++++--------------------
 xen/common/trace.c              |  58 +++----
 xen/include/asm-x86/hvm/trace.h |  30 ++--
 9 files changed, 235 insertions(+), 250 deletions(-)

-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:27:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:27:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189154.338674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRA9f-0003Yh-73; Fri, 17 Sep 2021 09:27:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189154.338674; Fri, 17 Sep 2021 09:27:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRA9f-0003Ya-3H; Fri, 17 Sep 2021 09:27:35 +0000
Received: by outflank-mailman (input) for mailman id 189154;
 Fri, 17 Sep 2021 09:27:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRA9e-0003YU-B1
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:27:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRA9b-0002x7-V2; Fri, 17 Sep 2021 09:27:31 +0000
Received: from 54-240-199-97.amazon.com ([54.240.199.97]
 helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRA9b-0000vv-BX; Fri, 17 Sep 2021 09:27:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=OVPtBbA96+972EE33f2c7pWRmvVZUFsGppBQap9lvn8=; b=P8yNgPVrzC0w16HGGWaYeLtqbT
	CJI6MUyDfaJGUwHWxgqfLaJ/iZMaW63opY3eSG4gcSxbxyYM560OWqb/u4+ehWgHAZJ5kq0w/mYDy
	cESh6Rofzt830uVvPPBePnxPSTeI+dX3gmf/XMaNAeUbnp49Tfyk5w63o5ZcSFuHcZJY=;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org,
 Dmitry Isaikin <isaikin-dmitry@yandex.ru>, rjstone@amazon.co.uk,
 raphning@amazon.co.uk, Paul Durrant <paul@xen.org>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
Date: Fri, 17 Sep 2021 14:27:20 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

(+ some AWS folks)

On 17/09/2021 11:17, Jan Beulich wrote:
> On 16.09.2021 19:52, Andrew Cooper wrote:
>> On 16/09/2021 13:30, Jan Beulich wrote:
>>> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>>>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>>>
>>>> This significantly speeds up concurrent destruction of multiple domains on x86.
>>> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
>>> improve locking during domain destruction"). There it was found to
>>> actually improve things;
>>
>> Was it?  I recall that it was simply an expectation that performance
>> would be better...
> 
> My recollection is that it was, for one of our customers.
> 
>> Amazon previously identified 228ab9992ffb as a massive perf hit, too.
> 
> Interesting. I don't recall any mail to that effect.

Here we go:

https://lore.kernel.org/xen-devel/de46590ad566d9be55b26eaca0bc4dc7fbbada59.1585063311.git.hongyxia@amazon.com/

We have been using the revert for quite a while in production and didn't 
notice any regression.

> 
>> Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
>> incomplete, and it appears as if it wasn't necessarily a wise move in
>> hindsight.
> 
> Possible; I continue to think though that the present observation wants
> properly understanding instead of more or less blindly undoing that
> change.

To be honest, I think this is the other way around. You wrote and merged 
a patch with the following justification:

"
     There is no need to hold the global domctl lock across domain_kill() -
     the domain lock is fully sufficient here, and parallel cleanup after
     multiple domains performs quite a bit better this way.
"

Clearly, the original commit message is lacking details on the exact 
setups and numbers. But we now have two stakeholders with proof that 
your patch is harmful to the setup you claim perform better with your patch.

To me this is enough justification to revert the original patch. Anyone 
against the revert, should provide clear details of why the patch should 
not be reverted.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:36:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:36:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189163.338688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAI7-00052s-4w; Fri, 17 Sep 2021 09:36:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189163.338688; Fri, 17 Sep 2021 09:36:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAI7-00052l-17; Fri, 17 Sep 2021 09:36:19 +0000
Received: by outflank-mailman (input) for mailman id 189163;
 Fri, 17 Sep 2021 09:36:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wMc=OH=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mRAI5-00052f-Fy
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:36:17 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 57aa80e4-7d11-4adf-9589-a52d5f2a78d6;
 Fri, 17 Sep 2021 09:36:16 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id i25so30741757lfg.6
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 02:36:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57aa80e4-7d11-4adf-9589-a52d5f2a78d6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=iK4mpVKVVYi9/BGqZbaVX4RSahiZlKiCXLvFBZ3uV8Y=;
        b=MN1/jjZbppwReGdyLZpyF5ymRtRGFu/Up778Ys6rsxRU6QdZsKzVvIV1Q5LMSVHi6Z
         LMWOqEixSS5S1UiNP6owwflUknFLs1ErQafLBTm1cdf5WCbo14UWSsE6/KV+DIGfOqjV
         AyKGWKq6Yh+vZ9xqARQDEgXI+FhmqY29Z33fH6+Ub3CcWbHN7pydnuuP4botigVFYLMy
         33nx/CbANxIL6zJDEYOpA2yEgz1OMqNEISlQrFUuBE6towtd9HN1UqjyGBakn7j61ZaJ
         GLIWXE28d5uJ2toCULBsue8DI3bB0s7qDKIuYv8KsoZ9U2hiDf3PWxFEWtOdeGEfcBVA
         03Sg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=iK4mpVKVVYi9/BGqZbaVX4RSahiZlKiCXLvFBZ3uV8Y=;
        b=Dbg4hTILN4uM8d4J/XPVuQCbzlXZ2dG58LBxgCpepED9IdIjY72jLPdrRBLk3nTrmV
         gK3Xc6cQxivzJhjD3wUzgzLw7oH91gQ4cA3ML8v/4IK3VrX+go6TMPViC3/gP6CW8KYk
         j0Nlb2qR+3T5n1u/40MGNHTRyuDCRi+j9xM0JyYt0V1GcLu+5hKg5thN+n6h0IpjoQnp
         CISkCezV4w6hDYjPQZEb5Uo/eojmPim7vZLjEIGlI0RPi/UCO8YJRszb8mv8b2uav05w
         WWoHVoEOKsnsb+pQoqr11BXv7DuawLPPHLUaQMPmgGmtWad3j3YznPX25Q+xzTkF2BCI
         kXrQ==
X-Gm-Message-State: AOAM533y/SB+hmTtYLuZfDJIoFNcOb0PTEwtuln0nEnpt6k/F063w7u5
	DpTHoLbrI5vw3TD35WqcdpvSvwzF1xOSCdBOVfI=
X-Google-Smtp-Source: ABdhPJzQUNFm8ZGVYUpqQ4Z42IbxHswXEbwr86F8d6HCMiXlqvVN8bj/2/tgPLfPn2KT6aH0vcM4FCwUBAENJfLsLbE=
X-Received: by 2002:a2e:586:: with SMTP id 128mr8467207ljf.310.1631871375448;
 Fri, 17 Sep 2021 02:36:15 -0700 (PDT)
MIME-Version: 1.0
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
 <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com> <20210915135321.GA15216@lst.de>
In-Reply-To: <20210915135321.GA15216@lst.de>
From: Roman Skakun <rm.skakun@gmail.com>
Date: Fri, 17 Sep 2021 12:36:04 +0300
Message-ID: <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Christoph Hellwig <hch@lst.de>
Cc: Jan Beulich <jbeulich@suse.com>, Andrii Anisov <andrii_anisov@epam.com>, 
	Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>, 
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>, Michael Ellerman <mpe@ellerman.id.au>, 
	Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
	Juergen Gross <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Marek Szyprowski <m.szyprowski@samsung.com>, Robin Murphy <robin.murphy@arm.com>, 
	"Paul E. McKenney" <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>, 
	Andrew Morton <akpm@linux-foundation.org>, Thomas Gleixner <tglx@linutronix.de>, 
	"Maciej W. Rozycki" <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>, 
	Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra <peterz@infradead.org>, 
	Lu Baolu <baolu.lu@linux.intel.com>, Mike Rapoport <rppt@kernel.org>, 
	Will Deacon <will@kernel.org>, xen-devel@lists.xenproject.org, 
	linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, 
	linux-doc@vger.kernel.org, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>, 
	Roman Skakun <rm.skakun@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi, Christoph

I use Xen PV display. In my case, PV display backend(Dom0) allocates
contiguous buffer via DMA-API to
to implement zero-copy between Dom0 and DomU.

When I start Weston under DomU, I got the next log in Dom0:
```
[ 112.554471] CPU: 0 PID: 367 Comm: weston Tainted: G O
5.10.0-yocto-standard+ #312
[ 112.575149] Call trace:
[ 112.577666] dump_backtrace+0x0/0x1b0
[ 112.581373] show_stack+0x18/0x70
[ 112.584746] dump_stack+0xd0/0x12c
[ 112.588200] swiotlb_tbl_map_single+0x234/0x360
[ 112.592781] xen_swiotlb_map_page+0xe4/0x4c0
[ 112.597095] xen_swiotlb_map_sg+0x84/0x12c
[ 112.601249] dma_map_sg_attrs+0x54/0x60
[ 112.605138] vsp1_du_map_sg+0x30/0x60
[ 112.608851] rcar_du_vsp_map_fb+0x134/0x170
[ 112.613082] rcar_du_vsp_plane_prepare_fb+0x44/0x64
[ 112.618007] drm_atomic_helper_prepare_planes+0xac/0x160
[ 112.623362] drm_atomic_helper_commit+0x88/0x390
[ 112.628029] drm_atomic_nonblocking_commit+0x4c/0x60
[ 112.633043] drm_mode_atomic_ioctl+0x9a8/0xb0c
[ 112.637532] drm_ioctl_kernel+0xc4/0x11c
[ 112.641506] drm_ioctl+0x21c/0x460
[ 112.644967] __arm64_sys_ioctl+0xa8/0xf0
[ 112.648939] el0_svc_common.constprop.0+0x78/0x1a0
[ 112.653775] do_el0_svc+0x24/0x90
[ 112.657148] el0_svc+0x14/0x20
[ 112.660254] el0_sync_handler+0x1a4/0x1b0
[ 112.664315] el0_sync+0x174/0x180
[ 112.668145] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
3686400 bytes), total 65536 (slots), used 112 (slots)
```
The problem is happened here:
https://elixir.bootlin.com/linux/v5.14.4/source/drivers/gpu/drm/rcar-du/rca=
r_du_vsp.c#L202

Sgt was created in dma_get_sgtable() by dma_common_get_sgtable() and
includes a single page chunk
as shown here:
https://elixir.bootlin.com/linux/v5.14.5/source/kernel/dma/ops_helpers.c#L1=
8

After creating a new sgt, we tried to map this sgt through vsp1_du_map_sg()=
.
Internally, vsp1_du_map_sg() using ops->map_sg (e.g
xen_swiotlb_map_sg) to perform
mapping.

I realized that required segment is too big to be fitted to default
swiotlb segment and condition
https://elixir.bootlin.com/linux/latest/source/kernel/dma/swiotlb.c#L474
is always false.

I know that I use a large buffer, but why can't I map this buffer in one ch=
unk?

Thanks!

=D1=81=D1=80, 15 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 16:53, Chris=
toph Hellwig <hch@lst.de>:
>
> On Wed, Sep 15, 2021 at 03:49:52PM +0200, Jan Beulich wrote:
> > But the question remains: Why does the framebuffer need to be mapped
> > in a single giant chunk?
>
> More importantly: if you use dynamic dma mappings for your framebuffer
> you're doing something wrong.



--=20
Best Regards, Roman.


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:41:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:41:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189173.338702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRANN-0006hT-U9; Fri, 17 Sep 2021 09:41:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189173.338702; Fri, 17 Sep 2021 09:41:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRANN-0006hM-Qz; Fri, 17 Sep 2021 09:41:45 +0000
Received: by outflank-mailman (input) for mailman id 189173;
 Fri, 17 Sep 2021 09:41:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mRANM-0006hG-9Y
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:41:44 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ec57acde-5335-427d-8304-76c45b1d8392;
 Fri, 17 Sep 2021 09:41:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec57acde-5335-427d-8304-76c45b1d8392
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631871702;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=goznNx9biLP39gTT9ltJqgHGv2ns4jazfM+InMjg3iM=;
  b=MexDGKBJy+XDGvw/uvynEHLz8awdSnz1GBCdoYD+fZOsjeiJ4rr4NARt
   HvrCgRDwk5tsyFDeg5aYKcvSHfhyNS1HJM7JBdD7oWCYN3li3ALCy10kQ
   FxIDK3PEJB1F6B++AQzQS0BNzR23Qrx+d1vPEnueURuQ+1h0/jIFhaRuf
   I=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: gHxsnH2r3isRuinxo6JZV91/7czAZLK2BwTds1M6iRd+oUrzAZyE41iEw7kmS0VQs2BmB2H+xA
 IRUKNuHvF2cSl2tlE3GlFRtqizZlLhAqjRnVjF5sK1rOMlfbzaP/4xAPCE5Ur4/2napEjtVVlK
 MT4sQK/khulBbcbr3bKdiv2LQF+arnoms6cjCcJJJT6U4LMQoKqK8v4xbq3wzw6IZmiw65BAeO
 Rs76vL8cH82a6Cw31LZak9+mZwPpTRA5NahjFhxwmC0HUgboqH+RCytc50XC9pLBvdhF+k7vjI
 2cEIFPO6POK75iNOy++7V+1O
X-SBRS: 5.1
X-MesageID: 52958604
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:vD8hkq7qtSsdgwCbgd67TgxRtGzBchMFZxGqfqrLsTDasY5as4F+v
 mFMWG6Gb66JZTT3fNp1a42zoRhQu5HSn4NjGwBuri4xHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo29Ew2LBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Zm
 f9jpI6LTx8VP4qPoc4XYyRbTDxlIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALBc/nJo4A/FpnyinUF60OSpHfWaTao9Rf2V/cg+gTR6eDN
 5RBNVKDajz4MjsVEVUrGq5hv/uNv0HEYQJ1inia8P9fD2/7k1UqjemF3MDuUseRWcxfk0Kcp
 2TH12f0GBcXMJqY0zXt2m2orv/Cm2X8Qo16PLyn9NZ6jVuL3GsRBRYKE1yhrpGRiFO6Wt9ZA
 1wZ/Gwpt6da3E6hQ8T5Xha4iGWZpRNaUN1Ve8Ug4RmNzKvS7C6QA2EWSTgHY9tgqcxebSQjy
 1uhj97vQzt1v9W9U3CG6rCSoBu7PCEHKmlEbihCUAhty8nupsQ/gwzCSv5nEbWplZvlFDfo2
 TeIoSMiwbIJgqYj1r6y/FPGhXego5nVVB8d9wzRUm+1qBlkDLNJfKTxtwKdt6wZat/EEB/R5
 xDohvRy8shfJ7+chheLWNkMO/KQzfy1DT6DnEVwSsxJGyuWx5KzQWxByGggfx40Y5lbJ2aBj
 Fz74lwKtcQKVJe+ReouOdvgVZ5ypUT1PYm9Dpjpgsxyjo+dneNt1BpnY1KZl0vpmVIl+U3UE
 cbGKZvwZZr25KIO8dZXewv/+eRwrszd7TmKLXwe8/hB+eHGDJJyYe1eWGZilshjsMu5TPz9q
 r6zzfdmLimztsWlOkE7FqZIdjg3wYUTX8iq+6S7iMbaelcO9J4d5w/5nup6Jt0Nc1V9vebU5
 HCtMnK0O3Km3iavFOl+UVg6MOmHdc8m9RoTZHVwVX71iylLSdv+t883KspoFYTLAcQ+lJaYu
 dFeIJ7eahmOIxyakwkggW7V99A7L0X32lvVZkJIolEXJvZdeuAAwfe9FiPH/ygSFCun88w4p
 ryrzATARpQfAQ9lCa7rhDiHlDtdZFARx7B/WVXmON5WdBm++YRmMXWp3PQ2P9sNOVPIwT7Dj
 1SaBhIRpO/spY4p8YaW2fDY/tnxS+YuTFBHG2T77KqtMXWI9GSU3oIdAv2DeirQVT2o9fz6N
 /lV1fz1LNYOgE1O79hnC79uwK9nv4nvqrZWwx5KBnLOa1j3WLpsLmPfhZtEt7FXx68fsgyzA
 xrd9t5fMLSPGcXkDF9Oe1Z1MrXdjakZw2CA4+40LUP24D5M0ICGCUgCbQORjCF9LaduNN93y
 +kWp8NLuRe0jQAnM4jag3kMpXiMNHEJT44uqooeXN3wkgMux1xPPc7cByvx7M3dYtlAKBB3c
 Dqdha6EjLVA3EvSNXE0ECGVj+ZagJ0PvjFMzUMDeAvVyoaU2Kdv0U0D6ykzQyRU0g5DgrB6N
 WVcPkFoIbmDomVzj89ZUmHwQwxMCXV1IKAqJ4flQIEBc3SVaw==
IronPort-HdrOrdr: A9a23:WizkAqGBrJUdZQTmpLqFYZHXdLJyesId70hD6qkvc3Jom52j+P
 xGws526faVslYssHFJo6HmBEDyewKiyXcT2/huAV7CZnibhILMFuBfBOTZskbd8kHFh5dgPO
 JbAtVD4b7LfCtHZKTBkXGF+r8bqbHtms3Y5pa9vgVQpENRGsddBm9Ce3am+yZNNWx77PQCZf
 6hD4Z81kCdkSN9VLXLOpBJZZmNm/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTjj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZvA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQfZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv3nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLYU5nghBgq/DWQZAV1Iv/fKXJy/PB9kgIm3EyR9nFogfD2xRw7hcsAo5ot3Z
 WODk0nrsAXcie6BZgNctvpevHHflAldyi8eF56EW6XYZ3vBEi93KIfwI9Fqd1CK6Z4gKfbpv
 z6IRplXCgJChnTNfE=
X-IronPort-AV: E=Sophos;i="5.85,300,1624334400"; 
   d="scan'208";a="52958604"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eZycynE5HMNSuVG7PbYtlsAzlMtYJIsQAbG14TwMfl4dJW+4ozhoSnj5sKxFD1UzkAFaDo3b0Pw+/ySDtK+k3lruPArRf13/DaFwGfgCV9Gf9rOn3m4l5GYJkfuzii/k9/pJ+irWptdmNnQlX22AXI9SPkmqXLsfT3dQdPlrtiYQydBiXVHXx2OozfM9e+F+HRzcfCMBNh5GiMxp6OkfbyH0WUE0bxf8BTaLn2ETTf6E56w6TGDDDHlvSzInArf22qqRBhYhJ4onbffhKotacmUkE1WzqCV82gIBlP5YGTQt1FtN46Wrxe354mGIBXzmtNWXuQVWrogmnzMlOdoQ8A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=coWda/k9IouttF7c2IlLgWtAZ4QLMG93DOX9yngDXj0=;
 b=YNn1Lfb1AVY/e73gR2IRW1k3XzZjV/ZZungGN03VzaNp8c74LCGHHI97yVG1dRI/ZMHgrvE1Bg9stoyPSrcyT1Y2+IVxJPdDxcwGKYBjzWDJsuzsX9jpgzbwwePhE5xPCXx+JiG7EnEttBbWb1zSRFBQgMnT6DyHaSqN3cL2tGfDOHqDXaremS5lMO8WGKeFP4AL8bWqapxUWgKTd4s8MIjHEQrH5n3Fl9rmd04ZzWHoz/ynKaWEuzkGqFG2ftwq6s94ABEaV+Xk3Bc52ZIwhgqdpKDB922NlsBDrkbC7ovovtDcrQCC1oUbEsrt3JfbQmVE2TQBlkhmMuvTVtiTmg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=coWda/k9IouttF7c2IlLgWtAZ4QLMG93DOX9yngDXj0=;
 b=qhI4gHXWxCUhJllbLpJ1WvI0RaiPpfHWpmgCmMemEUxSurSyMHi73DvlprJ76WCi5J4PYzhOv2AokVnbI5sELJqNaSMubQCx10Rye5FUUyzEtO/KdllpCG9ARhuGywAGty2OSH4sh4U+TTZ8udXe7wlgefceZWF29q8rA3eSr9Y=
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, <xen-devel@lists.xenproject.org>, Dmitry Isaikin
	<isaikin-dmitry@yandex.ru>, <rjstone@amazon.co.uk>, <raphning@amazon.co.uk>,
	Paul Durrant <paul@xen.org>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
Date: Fri, 17 Sep 2021 10:41:30 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0087.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:190::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 020039dc-925d-45f8-e1d0-08d979bf5865
X-MS-TrafficTypeDiagnostic: BY5PR03MB5045:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB5045A7C4222FB7C24B95C805BADD9@BY5PR03MB5045.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Zkb+VtVTMUyETf+eyuoIvLM8/UpzimhfwXQAzpU0VYMdyC3sTHlf2Buy6ivU2h6a5tJgnQwW4xDmt+XWPOGl0Z8RNOM+6/qonbSHOsdrFYjiVjDEsQPtWuPRvbEmzyWxl1ZSAeMmdd5B329XEYldUoIMmsUchYjHRlITOz+3m7Hi22BYm3QLJ2KLNpqlvlEAITxntCJzdIOvE8ezHiht1795MKQZGLtaDRIZ52dErxtIE9qsiEVlY8FY6iBY0yxTCaEECqwnGBCG6XlgrnADPC2OTEMtk04OcnRUmGyzfOnWZ2sq1aRWyb5zw4Bg698Szu/vYfzoIxsOkUkvcyWVTfMIfMkcn1eMZCuJmgFOVIrZ98oQm7JFkkXDVTAIdr6AikHNkLzCwTSBGwJ+fSNSRimnGm253u+ync0At89Sck28UKocgBPib70PNir+sCPWPJFLvVB7pk7LnxIx82HSZRmkiYvBxYvVIvaJmF8uMi2p/fFNt6ggamimmkNPrnQB6g/XlCGmC/y4Je3ZlosKCOH7hP6NpogC6t2ywxZBtjRXWp839KVPPSHU6+vy94kAXJO+Zj/oce4RoDp4b2JMiJLzx4T2bh/7u8oHwm/BFvAZXFF6Re4BsDKjU/kUNIJFgNii/2kunxQLO6XEXL9VNVQVzdNByzIrjkbA6VIXNUsot0V8/5/CdMEZZMhGzt+GYKRueRXubdGtI/R084WUr2IKFKqXmoYd3lzCVdxn3GYwCK0nidLcN2M0Mg4cgzKjwymhR64cGIyyy/gekiGlRvmObFztyFz0DccmYFvEnhGn8LhkFixpGkWpSVEs/UPkzzfY6ADorMNmWh1CNapaw2JL0iPik+5yGf3TbhW1Uk4=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55236004)(36756003)(86362001)(2616005)(2906002)(4326008)(6486002)(26005)(7416002)(83380400001)(38100700002)(8936002)(508600001)(8676002)(966005)(66556008)(31696002)(110136005)(45080400002)(186003)(956004)(316002)(54906003)(31686004)(16576012)(66476007)(53546011)(6666004)(66946007)(5660300002)(23180200003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RnBEbXlLK0tZZDhLWE1ncU14a0JNOERUejV1UjZGOWd1d3l1MW9kZmdlWjhs?=
 =?utf-8?B?OVpOUWs0YWFybS9PYURDU2xuUUR3Y3luSVBHdUpZQW0raElUTjI4MW9nZ0hJ?=
 =?utf-8?B?bE9MUW5wMEt3OVFDMG02bjFKVG4waXpMOGEzWXpRVkljNEVNNDN2WCtiRnEw?=
 =?utf-8?B?VmVmSjBwcHNvL2NjT21oV0EyZWovZEdydXp0aXJDNVlBUWJWUVR5WXdCT01Q?=
 =?utf-8?B?cmx4b3dLb2kyNUpna1ZDaS9oTnVWMy9pNlhTZEJ5a0VnZkRmcTBEWDROTTJ1?=
 =?utf-8?B?SEpleHBKS2orWE54RC9ySEEwSzNkNWN5TGprQUpaMmVycDFnckFXeGhRTjlB?=
 =?utf-8?B?cWcrMzNPZGJDbGJDbnpIS0JiTFNWMERwdGp1REdxN3Q4L0lUbUQ1ZTJrdUNp?=
 =?utf-8?B?NEtMZ2tNeTA4SGI2NnlveEdmODYyQlBkRFc3S3J1d0tjNGtnb3FKc2pRTlRF?=
 =?utf-8?B?cVFRK1o3eWExVGs0T05ROXYzS0N5bFN5NjhMaHl5cGZvdHVxN0pPYzJaUXE5?=
 =?utf-8?B?ampDY1Zjd0V0dTYzL2pIZUlIR25jQzJURlR1TC9ROG9MSGxOOFR1bm9YSHFl?=
 =?utf-8?B?c2tSTXFxeEdsdTgzK21JMTh3Mnd5aWs2S3RUNUxtNCtrcEZXbVJNdFg3T3NP?=
 =?utf-8?B?ZVYyS0RUVE5zdHpPM3dTbjdwT0xxemRQcXVhQWJ5M2ViSmhFQ1hROXRFeVFj?=
 =?utf-8?B?M29zMU9mQ21rbXk1eDdHMDBnYi92bDZDMUVLc2ZxZEViVk5sK3diSk1mcURX?=
 =?utf-8?B?K25iM0pzMDFJODFQUDBUWWhFSm1MamxnS2NTem9jS1ZUSDJ4bVRkMjFxY216?=
 =?utf-8?B?YTkvbmEycHRUc2JaVjJuV2VUZ1RuUWR3L3F5V29YbFZsdzg4ZWNTZ0dVb1pj?=
 =?utf-8?B?V1BBQjlsR0tLRUk1NkU3ZFJxdEtHd1JWSnlZSEtFSXBZQlpFN3JNRXp6M0hF?=
 =?utf-8?B?VDdVeWxPTTIzMTdNY1pmakl6YmlEQjI2aFl4RzZUNzB4L0c4VmNBenVDK0FE?=
 =?utf-8?B?YW5GMnB2UG90UGtTaDIxTWQySnRWenNhL0ZQVEU0Y29qcExTTXdpVW81M083?=
 =?utf-8?B?VTZyREpVM3RKK2lzMjRMZ3kwVjlSVno4UDFzZ29JWHZTS3JCK0hydFJSVEw3?=
 =?utf-8?B?YitBNHhJSXJZWFhuNUsvRThkdmdzNXA3Y2dlWFg5QVZ4RkpRNHNoaVQwbVBa?=
 =?utf-8?B?bEE2K3VQZmxqY211cVpMZjBNbExIdHh6STdOVDByR29TTzFRRW5xSU8yRlJa?=
 =?utf-8?B?eDl3bGxVVEVNNWJXUUc4V3VlMVBsbXhrNUZVOWdQZ2R1QXNyN0tINXdPNHVS?=
 =?utf-8?B?dVQ5Ti9pbm1kTXBTNGJQQkhTZ1c3aHAyM0Q5OGtkT1hwYktkWmkrdUs0TWM2?=
 =?utf-8?B?VER1K1FmWlRLMzdBTXIzUW54NjRhcWE4YWdhR3cxbTJkbDNwOWY3Rno0cmRq?=
 =?utf-8?B?Mnp6b0x2QTRhSVk2clU2YU8rc3JTUjlUUFB3VldHN1BiZ1NpZmtWSVg0M1p4?=
 =?utf-8?B?cHg3V2NlcStEREZyVW1FWE56Tlc4V3M3TmMwR2hzVmJvdE9HRFNHQ3BRMHZG?=
 =?utf-8?B?LzFUMmd0bTA1ZU5abEl6WWpCQWhVOHpMVDdKeDd4V0pZU2hTNnkxcnRXK0tL?=
 =?utf-8?B?ZmxjeStzSWpXa0I1U0dLUngwcnpYQXVTZ09VUWFIU1FNcGZ5TlE1bDExOS95?=
 =?utf-8?B?SytURk9ZTi9mUVZweDVRYUo2ZFFESS85anJiRkI0V0g0a1RPYm51alR1djhi?=
 =?utf-8?Q?INWZVtmYs0EToh5HI8sdZn9ifLxNe0ktKpHdNsG?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 020039dc-925d-45f8-e1d0-08d979bf5865
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 09:41:38.1120
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fzBb2cPpCTP98bKSgqo5yAKJW4QCM0sXZllFf87gatzk8JLZaYBA2PH8cprbHg9KGt+tH6aAPInrZpqpreD/lIeOx+jCIlf+bMprdZkD5e4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5045
X-OriginatorOrg: citrix.com

On 17/09/2021 10:27, Julien Grall wrote:
> Hi,
> 
> (+ some AWS folks)
> 
> On 17/09/2021 11:17, Jan Beulich wrote:
>> On 16.09.2021 19:52, Andrew Cooper wrote:
>>> On 16/09/2021 13:30, Jan Beulich wrote:
>>>> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>>>>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>>>>
>>>>> This significantly speeds up concurrent destruction of multiple
>>>>> domains on x86.
>>>> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
>>>> improve locking during domain destruction"). There it was found to
>>>> actually improve things;
>>>
>>> Was it?  I recall that it was simply an expectation that performance
>>> would be better...
>>
>> My recollection is that it was, for one of our customers.
>>
>>> Amazon previously identified 228ab9992ffb as a massive perf hit, too.
>>
>> Interesting. I don't recall any mail to that effect.
> 
> Here we go:
> 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fxen-devel%2Fde46590ad566d9be55b26eaca0bc4dc7fbbada59.1585063311.git.hongyxia%40amazon.com%2F&amp;data=04%7C01%7CAndrew.Cooper3%40citrix.com%7C8cf65b3fb3324abe7cf108d979bd7171%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637674676843910175%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=si7eYIxSqsJY77sWuwsad5MzJDMzGF%2F8L0JxGrWTmtI%3D&amp;reserved=0
> 
> 
> We have been using the revert for quite a while in production and didn't
> notice any regression.
> 
>>
>>> Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
>>> incomplete, and it appears as if it wasn't necessarily a wise move in
>>> hindsight.
>>
>> Possible; I continue to think though that the present observation wants
>> properly understanding instead of more or less blindly undoing that
>> change.
> 
> To be honest, I think this is the other way around. You wrote and merged
> a patch with the following justification:
> 
> "
>     There is no need to hold the global domctl lock across domain_kill() -
>     the domain lock is fully sufficient here, and parallel cleanup after
>     multiple domains performs quite a bit better this way.
> "
> 
> Clearly, the original commit message is lacking details on the exact
> setups and numbers. But we now have two stakeholders with proof that
> your patch is harmful to the setup you claim perform better with your
> patch.
> 
> To me this is enough justification to revert the original patch. Anyone
> against the revert, should provide clear details of why the patch should
> not be reverted.

I second a revert.

I was concerned at the time that the claim was unsubstantiated, and now
there is plenty of evidence to counter the claim.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:44:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:44:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189181.338712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAQF-0007La-Cf; Fri, 17 Sep 2021 09:44:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189181.338712; Fri, 17 Sep 2021 09:44:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAQF-0007LT-9d; Fri, 17 Sep 2021 09:44:43 +0000
Received: by outflank-mailman (input) for mailman id 189181;
 Fri, 17 Sep 2021 09:44:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YY/J=OH=arm.com=robin.murphy@srs-us1.protection.inumbo.net>)
 id 1mRAQE-0007LN-S2
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:44:42 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e12a4d24-179b-11ec-b67a-12813bfff9fa;
 Fri, 17 Sep 2021 09:44:40 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4EFAC101E;
 Fri, 17 Sep 2021 02:44:40 -0700 (PDT)
Received: from [10.57.24.25] (unknown [10.57.24.25])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D29AC3F59C;
 Fri, 17 Sep 2021 02:44:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e12a4d24-179b-11ec-b67a-12813bfff9fa
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Roman Skakun <rm.skakun@gmail.com>, Christoph Hellwig <hch@lst.de>
Cc: Jan Beulich <jbeulich@suse.com>, Andrii Anisov <andrii_anisov@epam.com>,
 Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>,
 Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
 Michael Ellerman <mpe@ellerman.id.au>,
 Benjamin Herrenschmidt <benh@kernel.crashing.org>,
 Paul Mackerras <paulus@samba.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Marek Szyprowski <m.szyprowski@samsung.com>,
 "Paul E. McKenney" <paulmck@kernel.org>, Randy Dunlap
 <rdunlap@infradead.org>, Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, "Maciej W. Rozycki"
 <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>,
 Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra
 <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>,
 Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
 xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
 linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
 Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
 iommu <iommu@lists.linux-foundation.org>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
 <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
 <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
 <20210915135321.GA15216@lst.de>
 <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com>
From: Robin Murphy <robin.murphy@arm.com>
Message-ID: <eb98aeac-af61-0dd6-2052-5b55921746c1@arm.com>
Date: Fri, 17 Sep 2021 10:44:30 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

On 2021-09-17 10:36, Roman Skakun wrote:
> Hi, Christoph
> 
> I use Xen PV display. In my case, PV display backend(Dom0) allocates
> contiguous buffer via DMA-API to
> to implement zero-copy between Dom0 and DomU.

Well, something's gone badly wrong there - if you have to shadow the 
entire thing in a bounce buffer to import it then it's hardly zero-copy, 
is it? If you want to do buffer sharing the buffer really needs to be 
allocated appropriately to begin with, such that all relevant devices 
can access it directly. That might be something which needs fixing in Xen.

Robin.

> When I start Weston under DomU, I got the next log in Dom0:
> ```
> [ 112.554471] CPU: 0 PID: 367 Comm: weston Tainted: G O
> 5.10.0-yocto-standard+ #312
> [ 112.575149] Call trace:
> [ 112.577666] dump_backtrace+0x0/0x1b0
> [ 112.581373] show_stack+0x18/0x70
> [ 112.584746] dump_stack+0xd0/0x12c
> [ 112.588200] swiotlb_tbl_map_single+0x234/0x360
> [ 112.592781] xen_swiotlb_map_page+0xe4/0x4c0
> [ 112.597095] xen_swiotlb_map_sg+0x84/0x12c
> [ 112.601249] dma_map_sg_attrs+0x54/0x60
> [ 112.605138] vsp1_du_map_sg+0x30/0x60
> [ 112.608851] rcar_du_vsp_map_fb+0x134/0x170
> [ 112.613082] rcar_du_vsp_plane_prepare_fb+0x44/0x64
> [ 112.618007] drm_atomic_helper_prepare_planes+0xac/0x160
> [ 112.623362] drm_atomic_helper_commit+0x88/0x390
> [ 112.628029] drm_atomic_nonblocking_commit+0x4c/0x60
> [ 112.633043] drm_mode_atomic_ioctl+0x9a8/0xb0c
> [ 112.637532] drm_ioctl_kernel+0xc4/0x11c
> [ 112.641506] drm_ioctl+0x21c/0x460
> [ 112.644967] __arm64_sys_ioctl+0xa8/0xf0
> [ 112.648939] el0_svc_common.constprop.0+0x78/0x1a0
> [ 112.653775] do_el0_svc+0x24/0x90
> [ 112.657148] el0_svc+0x14/0x20
> [ 112.660254] el0_sync_handler+0x1a4/0x1b0
> [ 112.664315] el0_sync+0x174/0x180
> [ 112.668145] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
> 3686400 bytes), total 65536 (slots), used 112 (slots)
> ```
> The problem is happened here:
> https://elixir.bootlin.com/linux/v5.14.4/source/drivers/gpu/drm/rcar-du/rcar_du_vsp.c#L202
> 
> Sgt was created in dma_get_sgtable() by dma_common_get_sgtable() and
> includes a single page chunk
> as shown here:
> https://elixir.bootlin.com/linux/v5.14.5/source/kernel/dma/ops_helpers.c#L18
> 
> After creating a new sgt, we tried to map this sgt through vsp1_du_map_sg().
> Internally, vsp1_du_map_sg() using ops->map_sg (e.g
> xen_swiotlb_map_sg) to perform
> mapping.
> 
> I realized that required segment is too big to be fitted to default
> swiotlb segment and condition
> https://elixir.bootlin.com/linux/latest/source/kernel/dma/swiotlb.c#L474
> is always false.
> 
> I know that I use a large buffer, but why can't I map this buffer in one chunk?
> 
> Thanks!
> 
> ср, 15 сент. 2021 г. в 16:53, Christoph Hellwig <hch@lst.de>:
>>
>> On Wed, Sep 15, 2021 at 03:49:52PM +0200, Jan Beulich wrote:
>>> But the question remains: Why does the framebuffer need to be mapped
>>> in a single giant chunk?
>>
>> More importantly: if you use dynamic dma mappings for your framebuffer
>> you're doing something wrong.
> 
> 
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:45:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:45:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189187.338724 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAQl-0007tZ-Mh; Fri, 17 Sep 2021 09:45:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189187.338724; Fri, 17 Sep 2021 09:45:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAQl-0007tS-ID; Fri, 17 Sep 2021 09:45:15 +0000
Received: by outflank-mailman (input) for mailman id 189187;
 Fri, 17 Sep 2021 09:45:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRAQk-0007sI-Bm; Fri, 17 Sep 2021 09:45:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRAQk-0003HF-7p; Fri, 17 Sep 2021 09:45:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRAQj-0005cL-VZ; Fri, 17 Sep 2021 09:45:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRAQj-0004ia-V8; Fri, 17 Sep 2021 09:45:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=YZvpbgA7lCRt3NXjLk2CVmCTtznEJ0+KUgPZ3JHmDh4=; b=tQ8H+H9lrf65/vtbWt0PVC/zhp
	oM5yPaGAQ67L1ShEtKqvXanHoSogfYCBIE5HPFOcCmzx2fDeePx2omGvWhOPm1NLNCCaS0XMxW+Oq
	851rFcsPBXZrMfvVLDutREmQdEfPU4ZQuna8+tbN6Pja7kA1l9PCbrbO7YjBJ6KMfuWQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165015-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165015: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start.2:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9d92140d0255e14d8c3eaba7edc931327eb96fd0
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 09:45:13 +0000

flight 165015 xen-unstable real [real]
flight 165046 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165015/
http://logs.test-lab.xenproject.org/osstest/logs/165046/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 18 guest-start.2            fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9d92140d0255e14d8c3eaba7edc931327eb96fd0
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    6 days
Failing since        164951  2021-09-12 00:14:36 Z    5 days   10 attempts
Testing same since   165015  2021-09-16 16:38:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 452 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:48:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:48:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189199.338741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRATQ-0000Rv-Bp; Fri, 17 Sep 2021 09:48:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189199.338741; Fri, 17 Sep 2021 09:48:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRATQ-0000Ro-8E; Fri, 17 Sep 2021 09:48:00 +0000
Received: by outflank-mailman (input) for mailman id 189199;
 Fri, 17 Sep 2021 09:47:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRATO-0000RP-I9
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:47:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5553fcff-179c-11ec-b67a-12813bfff9fa;
 Fri, 17 Sep 2021 09:47:56 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-3KGv8gYBMPqaujWFuMCgnQ-1; Fri, 17 Sep 2021 11:47:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4607.eurprd04.prod.outlook.com (2603:10a6:803:71::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Fri, 17 Sep
 2021 09:47:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 09:47:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0045.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Fri, 17 Sep 2021 09:47:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5553fcff-179c-11ec-b67a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631872075;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=idwOnHYM370iMoPA0YAP6F7uMR14s3cegSi1bn1KsHw=;
	b=PHTnJyI/QUATb0asPXcFm7QGiarvqJ6fMfO7fbChb4ol+XCimZPvYOyED369dwIKl9Zyjp
	BOHSyX96lP9eh4zwjXzmId/0rLYDVMGSAzAIfMqPe+yFmIy3wOx1rTbFyaqNHwOYZcX8pb
	Q6TTZZmZChOUNaZ0wfsWItzl9FtRlio=
X-MC-Unique: 3KGv8gYBMPqaujWFuMCgnQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L4ZgNOmH6jGjjN5cVTWNdfsGt3LoyAwYAQJXbCf5aKmDT6JnX6heOQA0v2uFGJ21mPFmnXSqczl76ZIFiKcpnQuVRGku+ZiQYsgqLiHdgViceFnpNschPiSzOdpWLFV3J496sbPjjVpECpl/jsbyHFOSocKAYukt7xNh2gj0YFdZfrOjkxBzSjvXoSEFn5gjECMRZ4KHtxsyrOh2rpZsvNFwCwxdQzc0f3Lwqinmeww53krM626/n+8jLe/ndH1d26+C3EVWwacAQBn/Gst360ci1enjaxYTQ+/4T9EHKc3K6swi2VGDLnkWYDk7bR6aZmaY8BO74LIVaZmMeVJOdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ax/VI5edREgAIO4of/Cl/ZPvcu5LUESwbiEQY1Fd8fk=;
 b=VHuF1eGfZBJbah6NlYMCDbAqFNFhy7LQtH9uEPznNkwklEeutoneXSglQX0UaHvLrWCK8dwK2dg160/q1cQ8DC2grCSs0o6FvUdbc5t7NYO2VZ5yl54Fsz5SwDPbp78RbRlEMcwYuYZd/sAeER4eVG14pxWUxxqox7Qa7iTI7BPfz3qhiJMZj4AtA+AFMBAlU9BbQ6Xs/hOWovn8441hIVfi6loaEvqfLiodDo/xi6pXu4js4pvk2pmLIx92QdZAVNmtHeTCc13RXL6ZtuRMPn74bCt+xHUa/aX8C1XrcgEeT72Et2yhVDbeipYzOXqK+rA7RwU05giasB+MjEudlQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall <julien@xen.org>
CC: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org,
 Dmitry Isaikin <isaikin-dmitry@yandex.ru>, rjstone@amazon.co.uk,
 raphning@amazon.co.uk, Paul Durrant <paul@xen.org>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
 <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
Date: Fri, 17 Sep 2021 11:47:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR10CA0045.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:20b:150::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7bb84c91-b3a1-4e6e-814d-08d979c037bb
X-MS-TrafficTypeDiagnostic: VI1PR04MB4607:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB46071331A5B1C448389B4AF0B3DD9@VI1PR04MB4607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FPYBm9TIUkFebc4GmyqDrvshCCc38JmHsDBthA+i6Bpkr3tjp0a/q/5vICv5RaUWWBKlOX0r7W796xgfDIpguI4vysper9t2+3e+/nRgwSCZa0wqfoGFSFVeBbQxX68l6Dt21w5PDwVGBq7Iwvy4i9SvF4z3B8jsgr8SKHzJqfjeQFiLNQIY3BIzhZfCM3eghZLxtMKpOzG7skTahBziI4QEAKJRTZF0mOp59xYZ66t7/U3F6JqSwKSlrBHuxYXPQmjGlQ2JhSDdVwuVojn7s9Iyl+INXXTaw79F1UnaM7srsKBTgbSXVfK2nO9HSmVuLTB+NxkVYdiKdvJOfNB/NMiol8pfOiUBij3ToVDiIrxxUX/RxQoBSZTun9ehDOF3+HFs0TAX6KIqpse02yIAPI+g2Fa2j/VUhMJEX4y9ncp2ED5Ma6+coJIR/YxbPl7W8NvvGpJSD2G/hISBvBNA5Y2UcCBJWxlJQ8BpvLAlHcWLd3pDIKhIea+/nqTG2WtAwvx9B94wtfjcYkyGWrAnlJpI34eyV82/LwdvA1unsfndL84zs7ZkjFE48p1qNXVQAb7YDcKdKtW32ci8kJYiQGcdswiQ0EfoHjHdpsLALhjmX3uKv6t3wW6ZShRN4IQ7R68kSVk+rOVeJDvasT4ZkTTcHIRcDou/Ow9/q848PFccjMw6xbxn4tynvtXmOpt3fVwxDYhMd3HH+14WFZGSVl8xj3HnzYGZrrOmHkf7Xpnc39ym/H3F0FOWVAkHDunIJ+AM7+AR0EhLP+ddEfKrorA6OcR+QlH35Dx9k9dYF/LbBX9J3k9m8pzctJkBdCB9dqzOZFnPCpaJFTepNygEh/m1U8bPIB/IMNClyiyTJmI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(39860400002)(396003)(346002)(376002)(366004)(53546011)(66946007)(4326008)(66476007)(66556008)(6486002)(54906003)(31696002)(5660300002)(16576012)(956004)(86362001)(186003)(2616005)(26005)(966005)(36756003)(110136005)(8936002)(478600001)(316002)(31686004)(45080400002)(8676002)(2906002)(38100700002)(83380400001)(7416002)(23180200003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?rHrrBXUCpDTVvwVROi2R9OgWoJCTEsKuprR4NZ2hZnJjYHt5EGDIcFmvnyO2?=
 =?us-ascii?Q?bpzNEryPO5mb8AlH1HPlipltOgaby7pN44rwW/dFGlVY35i7GIaKAdDAt2hD?=
 =?us-ascii?Q?Jy5tU+fvgKFjzHTtOy/YLGf6z9n8UBYucWF6nH8unXh+RS1rdaD1nvlGqkzn?=
 =?us-ascii?Q?p6wyZP6SgYpHLTkO3MBfSfUIWaYEJribP5aSogwlzsoiH3H4rrDHcSJtgkTs?=
 =?us-ascii?Q?ZNcw4Kx6bUzHD2Qu16AssCHT/U4Ssg1WPLg6Q3ZZkSIJIdc6OeYtUMsJVpSu?=
 =?us-ascii?Q?9k15IJh98/pnHMyKiwyCYZ01DoqXGPALA3dsDCGfiEjVMw+2xtjlBSMXgzHG?=
 =?us-ascii?Q?qQXof86YejFS+RbCOkxiUs1YWikhWhVdx50SwIpDPIDchSZNa8oYARBc3x3B?=
 =?us-ascii?Q?Av8ynOs1fKr7ORaMtUoxCm7Xi8HYsk/45QdOFxUtnu/DOWvqFX4vRBEdxaQf?=
 =?us-ascii?Q?C2tMJsFgEr5Ffh5unA+0TXtKhLbvlsX7ndBhQV42U68kTIL6JNA6W0CJj51D?=
 =?us-ascii?Q?mlpCCIqM9g9eP+aQyrl4cz1GB8hRUo8HNq7rkewtxJAXMlvDnJMc1/xney18?=
 =?us-ascii?Q?ITnE5UmlyTsWf8sUTX//Kx2dAx8pPArdg/vMOlnSyb11C87IQZmVYnhExA6M?=
 =?us-ascii?Q?kwYS9OoR4JathY78r5tAlATzWWuldzipCt4LoFRAFarixLkwyXANEzZYU8kt?=
 =?us-ascii?Q?F6JQUHRsVo4MHO4LI3d5AEE3v6fHkoABUoANYeMgGsVkTjwuoURYLZhmJpFM?=
 =?us-ascii?Q?oJ7L4py5aUlrwpXoxZGp6g7BQ3ugKQReZOweqcZkbTqvL0A4newQadoCKqE0?=
 =?us-ascii?Q?okBheaAyEJL8QSwMwKAMSX1fpnnb0VsCQSdGVUm6GXEdKzlkB37p3vN5jzDN?=
 =?us-ascii?Q?yNUfFZ8rP4AmH/I8KOUam7PJQ1H25fmvKbeFK8UpLQYRnGcNRj2Ua2wJGf5H?=
 =?us-ascii?Q?dkKNkGrbEizp2KgVP3ODQ3I8ham77Vf5jquG/EQ3TpqBlZAXu4I0HJFDk9ST?=
 =?us-ascii?Q?vJeRQ1qlDFEnbQ5v9xlbHGXsWWKwfsYJ++Ol/Gn52BMilmoHZEXUviaFGS3B?=
 =?us-ascii?Q?2l7HlOP979ytsKdLJVhqcCOJnGyFitEJ1e+X/POtrOgalUtmWJ1DaLZcriff?=
 =?us-ascii?Q?JIJ6BtvfC5lEuQW+Ykvxz0IUTyWVHBsKb9r1AWAABnAKRrhszWX1w5xzxanK?=
 =?us-ascii?Q?SOlVXReDxJK532PEotyPpV9KQhGyEWLakuLusZEPf5XNbniQU8I1AqQfmIFI?=
 =?us-ascii?Q?5Z1kE/4QTrv8ThmrhiCle10+gTx+zjUTsM+6u9y8a/aX6ToTMMM5b+aeR/Bg?=
 =?us-ascii?Q?gb3VdG21BoRCTDszMqKL6PWZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7bb84c91-b3a1-4e6e-814d-08d979c037bb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 09:47:52.7001
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rQfFpTrH+nzFzCY4ca3jabMPBH9mnZ0TfOVde0XiIN3tWlyfRFQjbu6DKVRK+VCbrfpe9Ofo3iijS20mSneLIw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4607

On 17.09.2021 11:41, Andrew Cooper wrote:
> On 17/09/2021 10:27, Julien Grall wrote:
>> Hi,
>>
>> (+ some AWS folks)
>>
>> On 17/09/2021 11:17, Jan Beulich wrote:
>>> On 16.09.2021 19:52, Andrew Cooper wrote:
>>>> On 16/09/2021 13:30, Jan Beulich wrote:
>>>>> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>>>>>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>>>>>
>>>>>> This significantly speeds up concurrent destruction of multiple
>>>>>> domains on x86.
>>>>> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
>>>>> improve locking during domain destruction"). There it was found to
>>>>> actually improve things;
>>>>
>>>> Was it?=C2=A0 I recall that it was simply an expectation that performa=
nce
>>>> would be better...
>>>
>>> My recollection is that it was, for one of our customers.
>>>
>>>> Amazon previously identified 228ab9992ffb as a massive perf hit, too.
>>>
>>> Interesting. I don't recall any mail to that effect.
>>
>> Here we go:
>>
>> https://nam04.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flore=
.kernel.org%2Fxen-devel%2Fde46590ad566d9be55b26eaca0bc4dc7fbbada59.15850633=
11.git.hongyxia%40amazon.com%2F&amp;data=3D04%7C01%7CAndrew.Cooper3%40citri=
x.com%7C8cf65b3fb3324abe7cf108d979bd7171%7C335836de42ef43a2b145348c2ee9ca5b=
%7C0%7C0%7C637674676843910175%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL=
CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3Dsi7eYIxSqs=
JY77sWuwsad5MzJDMzGF%2F8L0JxGrWTmtI%3D&amp;reserved=3D0
>>
>>
>> We have been using the revert for quite a while in production and didn't
>> notice any regression.
>>
>>>
>>>> Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
>>>> incomplete, and it appears as if it wasn't necessarily a wise move in
>>>> hindsight.
>>>
>>> Possible; I continue to think though that the present observation wants
>>> properly understanding instead of more or less blindly undoing that
>>> change.
>>
>> To be honest, I think this is the other way around. You wrote and merged
>> a patch with the following justification:
>>
>> "
>> =C2=A0=C2=A0=C2=A0 There is no need to hold the global domctl lock acros=
s domain_kill() -
>> =C2=A0=C2=A0=C2=A0 the domain lock is fully sufficient here, and paralle=
l cleanup after
>> =C2=A0=C2=A0=C2=A0 multiple domains performs quite a bit better this way=
.
>> "
>>
>> Clearly, the original commit message is lacking details on the exact
>> setups and numbers. But we now have two stakeholders with proof that
>> your patch is harmful to the setup you claim perform better with your
>> patch.
>>
>> To me this is enough justification to revert the original patch. Anyone
>> against the revert, should provide clear details of why the patch should
>> not be reverted.
>=20
> I second a revert.
>=20
> I was concerned at the time that the claim was unsubstantiated, and now
> there is plenty of evidence to counter the claim.

Well, I won't object to a proper revert. I still think we'd better get to
the bottom of this, not the least because I thought there was agreement
that mid to long term we should get rid of global locking wherever
possible. Or are both of you saying that using a global lock here is
obviously fine? And does either of you have at least a theory to explain
the observation? I can only say that I find it puzzling.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:52:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:52:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189206.338752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAXe-0001ok-U2; Fri, 17 Sep 2021 09:52:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189206.338752; Fri, 17 Sep 2021 09:52:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAXe-0001od-Q6; Fri, 17 Sep 2021 09:52:22 +0000
Received: by outflank-mailman (input) for mailman id 189206;
 Fri, 17 Sep 2021 09:52:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRAXd-0001oX-33
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:52:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c81f2f3a-ec5c-4b59-8cce-6df4a7469d92;
 Fri, 17 Sep 2021 09:52:20 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2052.outbound.protection.outlook.com [104.47.4.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-NQXKte9aM7SPAoFTTsTJOg-2; Fri, 17 Sep 2021 11:52:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3936.eurprd04.prod.outlook.com (2603:10a6:803:23::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 09:52:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 09:52:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0178.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 09:52:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c81f2f3a-ec5c-4b59-8cce-6df4a7469d92
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631872339;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GXHtNvXg31UQSdkLZPhgMtWUpYK/lfDRKw9DfFedgMc=;
	b=OF54Xgu3bDNP+32BLf7zECCaYqPHoopDxPAbbHu/fg9DmGRWny3C39Tk+Jr5nFvy/cB4nH
	7TcIdjKFdiMfkbcQbW7aJj+7cMLjpony6f22x4akX9exyCd456MkNaeDAfEI3vJ4NEkyQ/
	QfNjpezBmiojI27JTuafz/vhirdnU9E=
X-MC-Unique: NQXKte9aM7SPAoFTTsTJOg-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KUQsuEEnqhFH6XUniREkRWHldNtvBNSFqKWRJyjQ47DLH6REuqr55XETps2U7VE0eJa6iaSLrU5eR0JWtDh9ngNhOMmjsMpgF/714vi33IOFe1f4w7lnDzIznJa0HYICMcCCNTAGZ+74D4xNSiwSe/al1QkCgXgScJZFWUEKkyjXYrCXOb0Cs0xl7SPH2nYmJqfWvz7f/mis2/CyJ6rHWhDSIfNCnrV7z4wJqhRqnICc50eH9bq1+HL26hL+IIfeZCih3W2AhIFExaBKLTqq+1njywA1hACQ7Epf7Cc8HDrAe65zkhuTFy998jBhQiJrb9DVimV08kpn8tAJfiUAsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GXHtNvXg31UQSdkLZPhgMtWUpYK/lfDRKw9DfFedgMc=;
 b=OLltJ3gskAm7qdPVBo7pvXAtBW/wT09MdgkntEQTZlOjl26TqZHoVL8MkCPx7/SivPold0Kdwi/2n1In4B2aJE5QDkGywwClTqpTgQPBMbhL86iTgkei6LOMrKIr5xEzPL1Y4V8zt3YsFePbCHP3E+dluRbDmS7Ildru0U6ghPUfhDpbj0e4e6HYxjdh12i2zTH5zHFpG6heZigZVgcHITqGU1mYgmsGTXmagDjiboEz90g5csC3AJMvyT/4qQJ3FA8XPciH9aCCA8ObLe3yTR7YtXoEoCFIGT6GMV86rxiYFqm9OLgj0su3jbQAEk6+7+koQ8uPPgqGIxOQdM2e7Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lst.de; dkim=none (message not signed)
 header.d=none;lst.de; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Roman Skakun <rm.skakun@gmail.com>
Cc: Andrii Anisov <andrii_anisov@epam.com>,
 Roman Skakun <roman_skakun@epam.com>, Jonathan Corbet <corbet@lwn.net>,
 Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
 Michael Ellerman <mpe@ellerman.id.au>,
 Benjamin Herrenschmidt <benh@kernel.crashing.org>,
 Paul Mackerras <paulus@samba.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Marek Szyprowski <m.szyprowski@samsung.com>,
 Robin Murphy <robin.murphy@arm.com>, "Paul E. McKenney"
 <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>,
 Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, "Maciej W. Rozycki"
 <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>,
 Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra
 <peterz@infradead.org>, Lu Baolu <baolu.lu@linux.intel.com>,
 Mike Rapoport <rppt@kernel.org>, Will Deacon <will@kernel.org>,
 xen-devel@lists.xenproject.org, linux-mips@vger.kernel.org,
 linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org,
 Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
 iommu <iommu@lists.linux-foundation.org>, Christoph Hellwig <hch@lst.de>
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com>
 <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
 <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
 <20210915135321.GA15216@lst.de>
 <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ece0bbcf-2be8-e655-b164-b39b5d535c63@suse.com>
Date: Fri, 17 Sep 2021 11:52:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0178.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1c::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e7dda98c-565b-4ff6-e6d7-08d979c0d3bd
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3936:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB393646432D8345BD571CFCD1B3DD9@VI1PR0402MB3936.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VSBJD+Hpucunccno3ObKklm855Ig/SKMAaqRud98vyrGFLRBH+JYmoy/Hy7BzM1qlbD3Hw82yH38E+61gAQGQkTu3F6VBgOutaClbVpzKCSZT7y6lQp7q7B6CwECHBJo+XR4nWT0mxOjpDMXBIqyvkejLaeOLI3TETFOWy6F3goaEqIhYtE778B2kCIH00lndBwLcPzUgKajEsJDnwal0NGWAE1KFPVMg+5dNUZUCo+0Vcr9/UD8AkBN7S6sN74XoJF8PwAO2yFbIvQ0BZ4th5axLvdidJEAyRZ5c0GhOE3L1VznzmUJ0RS9ZBePAX/aPAePS70l9j+c8nmasK9eRkRpHpkdtBU6FCSbfX3wJmWaVG2vRG3FGknSs0iZM4IKkoTsaL+4g9k0MY+93hrFqGx3OkqPBy8WNxo4rSuaYj3O40wXvZ9oYfrmKukwIhNewJxnPlFi08lUXxPFEsvbmFAmgnwEUYqlJdSCsPdIidp9ccPzxVEJMz/KToZFdJZhI4IGT67JZ8rDuGSWvmxKhu0JIBWCDSDs1Q4TjWzRjKoRh/bqyPfqPzUI6N1Zxx+mDJbnyVhqhD6+BrwABQONjTBTHUTYQz7JK2fQmWoOlw1K11eh0RfGoWxrjSJGLif3XP800x6XbphXriKg+BTZwD1I1lZz5d5X4WmViz0NtAXGunYhWnrdPe8+pQb4DGg1vLxFT4C8SCOXez+T5yaQASbH6TzfiDtYiOqYfhG4tRs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(26005)(4326008)(53546011)(54906003)(7406005)(316002)(66946007)(36756003)(8936002)(5660300002)(2616005)(31686004)(956004)(66556008)(16576012)(83380400001)(66476007)(508600001)(7416002)(6486002)(86362001)(2906002)(4744005)(31696002)(186003)(8676002)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aWh1Qk45cENNVHNnTGcrUnRkYXpQRllDQ0NJZmZvR3dzbGIzTmUzRkdkTmc1?=
 =?utf-8?B?Y2lBck1PdjVNY3BGU24rVlJLSlEzNnNYd1Z1WHA3MUJ0RmoycTE2bVdPckZ2?=
 =?utf-8?B?ZWFFZTBZL2pTdEJpRlNmZHluMEhURVB4dzA5UFVRTU14UURHeUgzWHBxUlRh?=
 =?utf-8?B?b3R5cUgrM3NvWTVtcm5oQkpFelBKdUl0bkFQZldHNE1OczN4eEdLd0NNRS9K?=
 =?utf-8?B?Vmx1WXJrelVQVHlDWndHbER3aWZhdmlkTlByN0I3anRaMFFHVTk5alJiTWU1?=
 =?utf-8?B?SjhOUE9jN3owOVhQQTJUREtFQzdubENkRFpNYmxSa0Z0d3N4R0pWcy8rQW1C?=
 =?utf-8?B?a1p5N3NDdXNRdXZ6OGVoSVZEcitFclJWUHIweU1JclVpOFdUbk12Wm1UMXd3?=
 =?utf-8?B?cWtMZkpFcExIN0JNU0tEMHFlOHVTUTBHZ2lISEhEOTVHb3p5cmpzWm13OVhy?=
 =?utf-8?B?Q3d3WGdnV0hhd2pualBSRm1pR0tzOGVWdnBmOHdsUldHaml0UHY3OHphbTU2?=
 =?utf-8?B?Yk1jb29PQnpFRGFwS3VLTWM4c0VTR0o0Rnd3NEQveTluakpLTGovWkl5M3JN?=
 =?utf-8?B?aWQwY0d2SXZGenFLL0F1c3ZndHJtZ2FFYkt3VjdJNmsxODJCTGxuTGQzWTY5?=
 =?utf-8?B?RFE2ZWJEY0VyY1J0Tk9rV0pwd2JEbkp0ZzJJODJzNCtiRDJkV00rd2k4NnhH?=
 =?utf-8?B?cjRGWWU5RXRsU3M2dkxualBqK09pNXlhUXFleGs3Uk54T2U3aFR5eldMR3Av?=
 =?utf-8?B?ZUJkZ3dVTVZtTk5xZWNxSFQveVloZEFmUVFOemxwc3owSGxQVVhoRG9NM0I0?=
 =?utf-8?B?Q1lXQlpTMk1hN1RUWW5JUnBMUDRjeDV5bWw2UmxZenJWT0R6SW5qN1pTWEs2?=
 =?utf-8?B?TzhnUXIxMlU4MnVCNGxaNkdYcWQxN01TQ29hQUhuSFNxOGEzTm9ESGN3aWtk?=
 =?utf-8?B?UFFtbEd5UWlmVExKcStvTytuOHpyY3RYTzJ3QkkzdVYvTHFwYjBLenZUc09x?=
 =?utf-8?B?Unp1WXdJdTdiZEovdHlSdWs4QXo3VFM4UlRqWjMvWmZYRnNQUTBaUEd1Vm1Q?=
 =?utf-8?B?aEorMjlGbFJZNWpmYlhmNEJhQ2VTbmR5ZXU5eENzV1JMcmVhNFlhb0Z3SUpp?=
 =?utf-8?B?WnhraEcveVhKb2VFR0RDWHNwQXlJWFRIMDB1UFNuNjBMaEhiTGdURWNDTEVw?=
 =?utf-8?B?MUpyajE5dkI4MmthcFRidmo0SmxpQUJVL3NXUFJQbTRpTGQza1NhRVZnNjBk?=
 =?utf-8?B?VEhVblNUTnpDZ2pOaFlLdzNxQkFFbVpWZ0JvZWhtNGp0a1UvWHJLT3ZRWnhq?=
 =?utf-8?B?cUYzU2pTQVlaUjhnWGo0a1dGaTh4SklTbS81WVAxVkZobkdqc1JmeGpQbVlh?=
 =?utf-8?B?UTFxaXdjSTFwRnJCQ000N2RrTm83VGx6clczYkhFcWI3R3BEaVBJK3ZVQW50?=
 =?utf-8?B?Zy9uNjNJNU9wNEFJbjZXcW5JK3RrQ3h6ZnhEY1F6VHl1MC9MVGpWN3I2WlVw?=
 =?utf-8?B?UzBHVmlFaWt2UTlBRk1SV2pNVllBQmR3cTZvRUN6MzhDODFuUnJTVTQwSUZy?=
 =?utf-8?B?QXluT1NTR1poU2hZMXJ6eDQ5UHVaOC93V2lqcDFzOU5HVk5oN1hxNXgyaVRi?=
 =?utf-8?B?aGZkSkVkdHBsSXlvVDg5T2dBd25mWTV3bCtONTNaZnAxNzJNVFVtN0Qrd09M?=
 =?utf-8?B?WWt0UjJoN2RmeW9ZOFZYckdiRWN4N1krb3BwbXROU3g1ZjZLYkpoY21qYkZl?=
 =?utf-8?Q?tXdvUegyoB1lBi98nVCMJcsPYInAOshnLPJ5MAu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e7dda98c-565b-4ff6-e6d7-08d979c0d3bd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 09:52:14.4984
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qdlybJsY7Wdjq7223mSN8+JLrLBqmHTX8mLHVQUV8SHHaEKtzY/812DucDUvDwsb/d20AautVG5LUT35fL2u1Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3936

On 17.09.2021 11:36, Roman Skakun wrote:
> I use Xen PV display. In my case, PV display backend(Dom0) allocates
> contiguous buffer via DMA-API to
> to implement zero-copy between Dom0 and DomU.

Why does the buffer need to be allocated by Dom0? If it was allocated
by DomU, it could use grants to give the backend direct (zero-copy)
access.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 09:59:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 09:59:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189214.338765 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAe6-0002j2-LM; Fri, 17 Sep 2021 09:59:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189214.338765; Fri, 17 Sep 2021 09:59:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAe6-0002iv-ID; Fri, 17 Sep 2021 09:59:02 +0000
Received: by outflank-mailman (input) for mailman id 189214;
 Fri, 17 Sep 2021 09:59:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRAe5-0002ip-Tq
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 09:59:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16bf2d24-037a-4f12-985e-605c55691ebb;
 Fri, 17 Sep 2021 09:59:00 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-1Vt9hYSjNEmrknK-JtHlTg-1; Fri, 17 Sep 2021 11:58:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 09:58:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 09:58:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM4PR07CA0036.eurprd07.prod.outlook.com (2603:10a6:205:1::49) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Fri, 17 Sep 2021 09:58:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16bf2d24-037a-4f12-985e-605c55691ebb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631872740;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=xdPDbcvCiZUeDEAkhwr7XaH1aVMs9KTNxlzH84DbT0s=;
	b=ErciHtPFLuvaUGr7oUXcha3jUxd10j4vRwwd43gKVguEb3JsYr/DTslXgz7e1+Loq4pxXa
	WPyd4m7o3jy3FqTm5QZLNL/XXiw/g7y3rq6TUICHML1bPMfHdOdQVVD/+nHDZLlSvggTfy
	jK3GJtMyXidaI/JiXWc9JuTUrf7s7F8=
X-MC-Unique: 1Vt9hYSjNEmrknK-JtHlTg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OyHf+8Lg0XMBumu9uRY7r/t9M2l+7QCfb8WP2r+frCe4xEYpeZ7NLxjXZLv2/KQaoITZbsN9n/GRHG/q88Eaqxy+anlo/IeHhp7+UimfybT2gB4fvB5Q1ooVFxF/eNCHEtNmfcmJkndHJdeuosW0csekvrm7C+k0IqtR9nuPCPfcUHDkAXuhC1+gfbUSgQYjINcurLmxYu5An5flACJNwnl7d/hj0+8KNviEiXydYpcX5NfJDfNUroFaCUeS8CJ1RXpzkbq06ilLAWoQxCCyQG3cSYicWNjwP1VoW+Pv5A/ntxVkw1eB0D/PfDHrGCqABDZ5VtdH4i4yZALpJQ9rKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xdPDbcvCiZUeDEAkhwr7XaH1aVMs9KTNxlzH84DbT0s=;
 b=Yh6UT5U2b2WVMYb4tio3G1SL6Ndxd7d/maa7Q0VUn/yWzVYTrOXfHA7c0WbHQER0VJfXPhgdaZPr7MpTqqCtr+2kWEV21i2FJzxU/oEN+kiqqPFhUfeVIoLG2Zt2tj5I889aSyDJ2O7OD1cBKWmQ+fNgqEnz5s9SO3J+NS8zUqn0aSB8+y2+EKNZ/Iee9/zRxgCTdv8mw8+9qmTDIZfzHnTN6vx7c29IB2Ml5WoKR2RnEto8VANwID1Hlr/yfFln/nFgoW4ZnoF3w130RwJKx69LLoiQwNZWlD/DBQtxmEc++r8jN51bMRcxihjEgZ062MNVmzPlD/PpGWremr6KwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] page-alloc: further adjust assign_page{,s}()
Message-ID: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
Date: Fri, 17 Sep 2021 11:58:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM4PR07CA0036.eurprd07.prod.outlook.com
 (2603:10a6:205:1::49) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 49c1c2cc-92b8-48e2-6e44-08d979c1c340
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6861C17F198AB398C55830ADB3DD9@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:240;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ui4dj/cgScpV3hx7CxLV2F2bimV3tY4RiAjUiLT9OgHaf9nZ0iblBgJYOeuoMMicf26XXEzOZeT/2AXMyINuHPwzUz+931Xj2oK6ySHd6aBYmikqzQANsOnmYENITF/2NlCsQJ6bd4CUhrDfTbJqOFpefXocr5LfvNizZPx1E7nQOfaxWJyKrgIbDMZiUGmRsTTpMl79Wkvvj/DV23StMeNChAtHbNG5KxYBkEVKS3HURLc5wZH4NzmH2NCi++bDPp7DNhs+MCVdQ9Jq4t3IEB7ZMQN6D7LerCM20UuIwf5lDeMaD7+bvOaU/5EtrayuuUDfs2Btpk5YZJNvEiyXE/S1y/UaBVIg60o4FM2kveNnKFRo9g9Lun0Zsb1Iss3iQDD2rI4LC5kPIQ4NhgSRJ2hJL41fNp0yMpRuBM5A9PfMwkz1dVFV5d1UE59sil2z++NkD7cqJy/U9HvgxTbNoQjRCleBmbU6WbYE3jG9z8Xi1HqqUWGGeKBtDXdp0YCT6ZCcqWB/R01IDuJdUDBxwoY56RFDzI3ZJvJwpdy1otbKm8VknHpI6gfi3K1pfTyRPbNMVGdLQV/QQi8vmtkouETNpjJznNIQJllcvjqEWVL5trRBE2GM8KIcDuIJwfCLrfIoAp5Np5TF5mVxgpS0NQjjmfWS6ky/p9Wemd3gUo8PD/TwzWa+xSosU/+MDTKUREDOZUagMpWl2ap9wdIL78z0uukPHM/SQUhWjLxc/dw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4744005)(66556008)(26005)(31696002)(6916009)(8936002)(66476007)(2906002)(66946007)(8676002)(5660300002)(508600001)(186003)(83380400001)(4326008)(38100700002)(36756003)(6486002)(316002)(16576012)(54906003)(2616005)(31686004)(956004)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OGdQWWNZQUNBT0JGTXBOZVduZGxFWTBDeXdUSi93aTdsK21SR1RISFNFY3F6?=
 =?utf-8?B?WWtrczR4azU4UnFoQU5ueDJISHl1aWVtOGdTN01JT3pQeFdrai91SCthWlZ0?=
 =?utf-8?B?Z3ZmNE9JVTVvT2lrVUdwc2w4eHA0NGFxekZoSTFsbjVjRzNYOGd6L3RsSUlM?=
 =?utf-8?B?VE5xQVMwclF6MVQ1NzdRTHgrYzdJeC9pUmtMZWR3VVl4SXZCK0dxSWQvVzVr?=
 =?utf-8?B?bk5RaHRmckxGWTVPdnJFbG9pemtUZXRua05lWmFJSUxRaWs5TnNHamJnTW1p?=
 =?utf-8?B?c2xwR1NmN3VVd0x6N09nZTljNm9Kd0FEcVBMK2pBcEhYVnJMcXRsL05sZHl3?=
 =?utf-8?B?bFl3YTRINWw3NHcvM3I3QzEyR1VaVHNXbGkyOXNTSHJOaUkwQzhZNG5kM002?=
 =?utf-8?B?cHJGOEt2VTk1OHRTMUpqTitOWkh0bEtaaXNRRWRqRnRwMzhiK2FBMHN6NWlo?=
 =?utf-8?B?eGhQSGhlL2ErUWt0cDBnc0wrcUpoQjFpNjFscWN5VHJNK2hOQkxDSWdKak8r?=
 =?utf-8?B?WS9XTmFJbWdJNENqWlRvZEgrbWZoSDZ3WU9MMUtoZHVHQkxiNWU0WEIxN2F2?=
 =?utf-8?B?cFhvUU41YjJKT1RyL0ZjV0RSZkY0TVMxNkdQejV5RVZUb21MMlMxZ3VsMHlD?=
 =?utf-8?B?SDZQdzJSTlVpRm1XajZwWExyQW5LWldYNWlQdnZQL0Rwb0RiY3pMODI4ZDdD?=
 =?utf-8?B?SmR6RktTNzFHZnh1eW1Dc3BxRUhMc05jQ1Z1a2R2NWNrTkZGN3dPcjQ4dTdm?=
 =?utf-8?B?ajluZldDWFhxT2M1Yi91VDVVV0lSUG53UDMyZ2pzdXIxMkFobDNJK3M0M3dI?=
 =?utf-8?B?cW5YTHVnemJ3ejNGakVXaGNDaVRwTzUyYTgybGd6ajV5bjNYREFDWFJXb2Mx?=
 =?utf-8?B?MnFlcnovaDhOTXcrUW1FaTcySkl6OUVsd2Q2ZnptMzAxdHJsT3A1WG8wYlR6?=
 =?utf-8?B?VWtVSEkwSlYyNlZHZXlpSFZnTkJKcTkybWMvVDJiNmlYZTN4Q3RrSHpRUWFV?=
 =?utf-8?B?YW1MTTVlend4NmxUT2tLN0c5ekVsbVcrNUJUNEJKQnorR3JTUGswQUNqSkxW?=
 =?utf-8?B?VnozYVJoYytMOXNEa3V5akNvbXZkZ3FTMThGVTQ3L0VtTjJmaXV6SjErSnRs?=
 =?utf-8?B?ZTEvOW9yRDY5WlFVaXlkUzlPTnkyQm5BVUpBck1WWWQydFYzL0ZURjBJRERZ?=
 =?utf-8?B?L29NTml4WnAvWm1wQUc3cjlKUWlLbXhHVkE5Wk4wd0c5MVQ4QkhBOTh6TTRr?=
 =?utf-8?B?MU94aEFOMjV3aG1DSFBXQi9vOUVzMUVCQ0QyeTJWMmJZcWtKVEw5NURtYkxH?=
 =?utf-8?B?ZTJ3ZGNOMGE3M2dzM0xZT2lzTEM1bDFwRkpuczdSYk96VDA4SThJV2RFUWpi?=
 =?utf-8?B?TE1wa2YyQjVzSTdpU2hhK0daSlFtWFdhNXpCVWwyMklMdCs0NGUzK1dCRHU5?=
 =?utf-8?B?azFrMC95TlJaNlpMT0xaOVVIOTZoNnV2THRic0R0V3VNTEVyVUh6aEJjeURi?=
 =?utf-8?B?MEc5d2lsQmRITG94OG0rNjQ0MldhNmRzR0xyT0IzbEtpT21FcVJXZ2l1U2I5?=
 =?utf-8?B?WnZhbSsvc0EzTEtlSzVRMi9jV01FZ3dOSWhCbk1ydHJBd0RFSDZvTldiWkVl?=
 =?utf-8?B?cFdQcVBzRE1jUUl5dnZjWXplMW13OFdnQWhqRDQ4UUdsVUpXN3pGeFU0N2tr?=
 =?utf-8?B?U08vb2VLbGRhMVA4N3BacEJLcU1rVkF1bldxOGgwK2ZIL1FzTU5QdlpOMTU1?=
 =?utf-8?Q?GDIAKDPhDI/uJQ1DeJ2x0mjhLIvfP5X+1ysCsNt?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 49c1c2cc-92b8-48e2-6e44-08d979c1c340
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 09:58:56.2383
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AE1ofaFkaOiM/qabVDJmDLV/S3wUTwbXv9pg6q6qn0b1pc0778GeOHX8M5iJ8pa6SizYbU8jO5JN9dAwEGHR1g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

The on-commit editing of 5260e8fb93f0 ("xen: re-define assign_pages and
introduce a new function assign_page") didn't go quite far enough: A
local variable and a function argument also would have wanted adjusting.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2269,7 +2269,7 @@ int assign_pages(
     unsigned int memflags)
 {
     int rc = 0;
-    unsigned long i;
+    unsigned int i;
 
     spin_lock(&d->page_alloc_lock);
 
@@ -2339,7 +2339,7 @@ int assign_pages(
 int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
                 unsigned int memflags)
 {
-    return assign_pages(pg, 1UL << order, d, memflags);
+    return assign_pages(pg, 1U << order, d, memflags);
 }
 
 struct page_info *alloc_domheap_pages(



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:07:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:07:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189224.338783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAmU-0004Q3-Ri; Fri, 17 Sep 2021 10:07:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189224.338783; Fri, 17 Sep 2021 10:07:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRAmU-0004Pw-NA; Fri, 17 Sep 2021 10:07:42 +0000
Received: by outflank-mailman (input) for mailman id 189224;
 Fri, 17 Sep 2021 10:07:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6HF0=OH=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mRAmT-0004Pq-F9
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:07:41 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.81]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6c939480-0d8a-4b5e-861d-6d702d42010e;
 Fri, 17 Sep 2021 10:07:39 +0000 (UTC)
Received: from AM6PR0202CA0067.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::44) by VI1PR08MB3101.eurprd08.prod.outlook.com
 (2603:10a6:803:45::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:06:35 +0000
Received: from AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:3a:cafe::6) by AM6PR0202CA0067.outlook.office365.com
 (2603:10a6:20b:3a::44) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Fri, 17 Sep 2021 10:06:31 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT016.mail.protection.outlook.com (10.152.16.142) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 10:06:30 +0000
Received: ("Tessian outbound a8bfe25d7364:v103");
 Fri, 17 Sep 2021 10:06:30 +0000
Received: from 7b96f042fc11.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 48D2558C-2D16-40D9-95A5-0C830A8DFB52.1; 
 Fri, 17 Sep 2021 10:06:24 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7b96f042fc11.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 10:06:24 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5561.eurprd08.prod.outlook.com (2603:10a6:102:83::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:06:22 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Fri, 17 Sep 2021
 10:06:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c939480-0d8a-4b5e-861d-6d702d42010e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IZgxB81np4TnDRupKJrs1tz6pX5AZNgDbrypJd0tpdI=;
 b=k+jRcEvtwvIMpV/0halT1BHR9teOQ9+RWwSufpItSv7F5x3fMcW54ujqF+vHcuOL676xNsPUnW6tqcd82cT2QK+UTkHe6JzkASTYZb6xSG3fHWWTa1gbhC9MGDwL31BF0InMbfaBIYRS+PFfbdxKUY4V95CLWUj0K4DNZRwzeLw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0c21c397768a0fde
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BnLO3iWIt2iBNCmR90L9f7xCk7xKzBPSMk4+cTplwtKKjUY2v/CtSqtuUYVPT1Kn8m6vpGtOYH3bzTdIW1No1Djrv30DUkfRrGBYWS0c1Tcg9TA/I33fzOn0lnbAkS/I8jCFgnEdcDWauluKmW6Z1zmg5fa8iYFuKd+CFnGYJcFA+RkwxAxcKI/+hoWuF64rGTCCY+chZVOY9QJoRaxRBi2ir77r+n+dJ6EJjgYwh5k5PRDxIBcTIYEvXBQP2kVpspOc5VL04aoD3fGJmL1hY8e6L0y+gLFQ1B6wibhN4vrw5i3bCIKFdSLIm2oZV8EgP79+VAunySKH4NhfGKqVqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IZgxB81np4TnDRupKJrs1tz6pX5AZNgDbrypJd0tpdI=;
 b=WquEaa5v/++1/G0PzAe1ugq2uSDHWRa8D5FSv3yqUCJ+6n94ehC8g5iFeE5L5c3aw0aIhjVZwUj9qE3ipFocwoPHBNMC5IHpQ45RirF5tSrwDC2b3crL3wQUe8QotfN8dLW5qxFYE7+AipdEZoVYFYJXdxGB4mec3NvabFWddf3ljL4D2eSwZ9ZsbueZy251ppueDnw6LuQFv075IDpLoSh2aiE649I3zxjEEXLGfLv+r7EJZ9QfOEW+L5c3RCtwFMV6cER0oPqrsVI7DS+GPgCmpwpl/xUB2ktZTiksDcHMcraHimApcHuKXzywf7dHcAF215zgypYAoNjLK+oUWg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IZgxB81np4TnDRupKJrs1tz6pX5AZNgDbrypJd0tpdI=;
 b=k+jRcEvtwvIMpV/0halT1BHR9teOQ9+RWwSufpItSv7F5x3fMcW54ujqF+vHcuOL676xNsPUnW6tqcd82cT2QK+UTkHe6JzkASTYZb6xSG3fHWWTa1gbhC9MGDwL31BF0InMbfaBIYRS+PFfbdxKUY4V95CLWUj0K4DNZRwzeLw=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] page-alloc: further adjust assign_page{,s}()
Thread-Topic: [PATCH] page-alloc: further adjust assign_page{,s}()
Thread-Index: AQHXq6qu/1IFuhFez06uzu67oLNGcaun//6A
Date: Fri, 17 Sep 2021 10:06:21 +0000
Message-ID: <3164A827-E883-44C5-9E47-6D3FC3EAC42F@arm.com>
References: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
In-Reply-To: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c0f4ce43-13be-47cc-8c2d-08d979c2d252
x-ms-traffictypediagnostic: PR3PR08MB5561:|VI1PR08MB3101:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3101F18C6B8225CBE076CD059DDD9@VI1PR08MB3101.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:291;OLM:291;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +SO0adpP4NfNdUtKPHxhZmJOXmxFxxcdyAQgG47geNkhYKDvMjVHSRjaIQo0s/c2+9RwjNMTGtM+N3q9MLhU2vmnmgKYdfqngsuGfyL45c15UH8XWiKkh6ouz94PfCF0bqZThBljjjcU1TxcViLGYE92J2j5uzX/1qWYzMxcaNbpzvH1h1rGM5aG/Y9S8BnKWYvLVSoyZfsxuI1dHb24YFodQrayxlfZyXBjcYN4Smu/cc8mKSFyhjUHuHG3rw+tSh5xBo7rVYtR3JAucedxLkO34Hcy1a8l3uoDm9ktgbRschYS47ra82GXQ+ixq8ulrTp1yGPx7rHllLzf1lcmz//Ev/B5VQDcJrby18xSt/wfklHY20NGwFiH7lZmeVbocRSymM5foqmPy8FKovdEXbmilziP1XE5T3mlc1Rxst57H9x3nQk9jMpiG/h4rjG1gf3Pn3Lxb79x4v6iLGlBld01vj36lW4PyoM2QnZkt98GtF3r2D2+L0w4pFXWRIsyJsSbkfchN35HS/oI67sSWzoQWqxB1v5aVQ6rbfqcWR5kgPHeKiEgaWdu6kFb9wUvgg7zZJusQQhjD24ywuRHHx8gPPsFVTO67QA8J4VHcMHQNkMWSvTXwQovEPjAsCVI1XnDiOuIECa0U3MR+p/p6jxR536dY2joepttmh80YMVr+260IJQzqtYlIyyshYyEojaYaRKngmrJOVrREXNEziDgnn7IDBNcGnWYcKf1U3o=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(33656002)(71200400001)(36756003)(38070700005)(6506007)(53546011)(316002)(86362001)(2616005)(6916009)(66946007)(66476007)(66556008)(2906002)(64756008)(66446008)(38100700002)(54906003)(76116006)(6486002)(122000001)(8676002)(83380400001)(26005)(186003)(6512007)(4744005)(5660300002)(508600001)(8936002)(91956017)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1D322D72FC88BE45B7DE1C9EA2CF0242@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5561
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6053ca45-dffd-4170-1094-08d979c2ccd4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KYSN0PRFF831Oiqw1gWeXselAT+DlicMF9uRigxNDGZRC8bh1RuXVNxyLctuKnSHA6Ezg5q0464O/epVpvDjys8+1Ndue5RUVHQVxwdLtMkT3WXstuVrmVTPq/xXPaYfHSrCUeDp/gHh8TkPQhaBRdibATGlotUxDwDTfBV4GnN5TUT4hUabzof6VoyDWzu8RHwt+g+TtK2xfSD1NCL6q187SgruDpMNn/xK90nCqHI3EhtfLa3KBc25DAN4/SshBcG6en/Iy4TxjtUnnIbxHoR4XZGUVjk+t1UhFajj+94x1zHeZ53Qa4pptmQO0Rg7XQsBCbjhU91TGSVCKuMHbZS40PpGjQKfLOnAqumIm0pLe5WKo/wHl/9Wcrl0gVE0hy3EpASGEINrZujBSwX0D+kOMLfFXTNFGVaMVUnwvCZNN9yVNSP5XqJFesrWVMXzqpHo099i3wJAqSHCW7btVVsJI1YxgximEqlshBxov9NkT4UYppX/NLfZEKU3+ffSad5tYRNlij7V5cef1JTaVtoZBb9bw7lU+G0odNaJ2mQ0iXLfrUMe6/GRtDIXAi2ExjueDC3iUzbhWnhXQ7bxpFErQQ0FOmwiGiC0t22t3N1/aUAmsevh8KQu1bL4CTaDkFngf3h+lX5fm+yymc1WZnbdJLJ1w2HnrXEr7f6x0kdcNUGHZYJB+nRhifzKeFzEY/wPupaP21Jfx/9RgKMzlg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6862004)(336012)(2906002)(186003)(4326008)(6512007)(36756003)(36860700001)(54906003)(33656002)(70206006)(70586007)(8676002)(47076005)(4744005)(83380400001)(81166007)(8936002)(6486002)(6506007)(53546011)(356005)(86362001)(82310400003)(5660300002)(498600001)(26005)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:06:30.8337
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c0f4ce43-13be-47cc-8c2d-08d979c2d252
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3101

Hi Jan,

> On 17 Sep 2021, at 10:58, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> The on-commit editing of 5260e8fb93f0 ("xen: re-define assign_pages and
> introduce a new function assign_page") didn't go quite far enough: A
> local variable and a function argument also would have wanted adjusting.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

>=20
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2269,7 +2269,7 @@ int assign_pages(
>     unsigned int memflags)
> {
>     int rc =3D 0;
> -    unsigned long i;
> +    unsigned int i;
>=20
>     spin_lock(&d->page_alloc_lock);
>=20
> @@ -2339,7 +2339,7 @@ int assign_pages(
> int assign_page(struct page_info *pg, unsigned int order, struct domain *=
d,
>                 unsigned int memflags)
> {
> -    return assign_pages(pg, 1UL << order, d, memflags);
> +    return assign_pages(pg, 1U << order, d, memflags);
> }
>=20
> struct page_info *alloc_domheap_pages(
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:44:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:44:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189235.338803 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBLg-0000du-Tl; Fri, 17 Sep 2021 10:44:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189235.338803; Fri, 17 Sep 2021 10:44:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBLg-0000dn-QQ; Fri, 17 Sep 2021 10:44:04 +0000
Received: by outflank-mailman (input) for mailman id 189235;
 Fri, 17 Sep 2021 10:44:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBLg-0000dh-9z
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:44:04 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1a8b80e2-19de-4136-adb8-b06f414aaf38;
 Fri, 17 Sep 2021 10:44:03 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-DiK1aySnOY6fT0fXkH_g7Q-2; Fri, 17 Sep 2021 12:44:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7087.eurprd04.prod.outlook.com (2603:10a6:800:12a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 17 Sep
 2021 10:43:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:43:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0041.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 10:43:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a8b80e2-19de-4136-adb8-b06f414aaf38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631875442;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=3fkICs2NUVZw+X2zzeXCMA7XH+sCYDVqsCZWlrdy5Ss=;
	b=C7yDkLc/0T8/5YgOp/fYUFJIZuf5CWVTNek44KiBVMFUmLfrl08fI8SnL2s5IZ9J6UQ+Am
	vYHkn9M77BL/9w+H5UESSbntP9H+MS7ItmXHMXuZTy7vsgosRrzaNsWQuKGwexwxSjKgwT
	UfdophElqd/njBExs4tppXsByXh+r+w=
X-MC-Unique: DiK1aySnOY6fT0fXkH_g7Q-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EG3bnqdiIIvcL4mJmgRBBqpzMwrrZJ4QeuWPEaA+caDYfYCdtDEhuHxExVBE/Fl6GbSvg6NxXDTn0SZO17eOvHwezoOv1gGUXJTxgzn38oVTYJnuZ59sDG8ofgIF5zTJpxTkXtEE1pzfNzGHr/mPyWAoSXbvYKkNhPo8RnDwjiS3Ye9bUacKCnRChtnqsfvIJUp8vnzA6pg6RfBddbXUT2hXsIHAXw2Dok5Z+Vg76vHv6sVz6fY9aPGshJjtJEGxHgNiy744oXA/Ty4GVC/uk43danmx53MVJgBxJnkldQQxWBPv3vsTASewOx6IUHyq4ouTVFI42itBZC+wUFk8Nw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3fkICs2NUVZw+X2zzeXCMA7XH+sCYDVqsCZWlrdy5Ss=;
 b=eoiwDI3QqZn5G6TkbbWM2bMI0/XyH5iDwUBYmqSnjKtzkpc+ROkSyDsRGSi+i0HGtL6zLJNh4+KXCB8gdrQbcEROB0SuMC3M382YJHyJg7uK3H4V3zwt9R5IJ42b2TFvqgbKAoxL235gYZ0TgSm7CSxfs5SwRwdkfqbvrgMyZsFFcygSW3TcpyHE9LVEOjpbPgFRH9x2x0mbmRsUNOkgiIvOyC0IRSy6LNaNUubajbu8oWo/1nD+0yV9J15w1fcL0KSFI8tjBArGrxoGQ7AoG6c4LqlJSYYE7ACX+k+VwOFEnhoBj4EixwrGQFS/ZsaT6svZr8rm0mHmu1EMk7XgCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.linux-foundation.org; dkim=none (message not
 signed) header.d=none;lists.linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/4] swiotlb-xen: remaining fixes and adjustments
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Message-ID: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Date: Fri, 17 Sep 2021 12:43:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0041.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:20b:150::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1ad8c43a-c84c-47b7-8258-08d979c80d0b
X-MS-TrafficTypeDiagnostic: VI1PR04MB7087:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70875114D12052171888F5CDB3DD9@VI1PR04MB7087.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IRQwFcetYoGR3tRBeWr+YtIloZxjnKjyVhV/sOKN+zMr8anVIoZeLGCp/cHQmBmdMhow06JNz11oZrbni4bOKpcS0v2f6LoZ2ULKFf6ThNujsrkxl9oOWo+yDPLEqvAEKn4HvI+Z82bSSpMPmbpyUfUojjIqBb8ffkH+IbEPkjQc7Vy3yCBeTt4NLibAr4KvxCA8DCJIZvxOYdW+wP31rupjWH2LEv9rsIhIxfynB5nb6J5K804TQZTQ/ZyjCwmbRemDKgIx0IxvdCrlauDfie+xvuJufSqOM0lI/WraeI5Alf/3s58x546K5CqUxCBO1w7fqHhhTdV2QYoDu5QpxqhRD5DjivRMfnRls2KQLouMzL8ueKsZfVeWsZbU7GzlK0KKD1N2I4eYdmIDL6vJPSmqbAX07toj3G1n3kc7yPENglowPSvwJFdAtW2vbrN4vJTg7xC64Pl3MyY+NrEfAqUJ3NZkSZfBCPFruvDA19kJCK/xySZBPSGt0uczpS1fsfK+DPLL0mxNeLNu+NV6yzHRPdVH9IzxeP0oBGMbQCYpP+scnpxvAgRIiuEoYtKHfazQ9McWn/U3VGBYGWZpO3rB4f2ToMgQM3hQ+pQXYb3cquKnh7p3B4b6ekBtWzMUwu/QxeoNCcbgwS+dD8vRb7G/R/oBEjqjHcCkfUT8CiQs3rrt9wH9Xz1io/Qp/uO/bwnP1xBh+zFJrOeCj1cUZAxuDW4Pa67uWgV1tC3zmfY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(346002)(136003)(376002)(39860400002)(6486002)(36756003)(186003)(5660300002)(8676002)(956004)(26005)(86362001)(2616005)(110136005)(4326008)(16576012)(4744005)(66556008)(2906002)(38100700002)(316002)(66476007)(54906003)(31696002)(83380400001)(8936002)(478600001)(31686004)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TnV4VXp6SE9mSFNldkJoZ05xcmhaYnJvMEJXUWRxY0kyOWU4Y2d2RkF0cVZQ?=
 =?utf-8?B?Y3FleisvYXd1ZGJpdXBTUjZsRG9lSXlyaHl6TnJWL1lSbFhCVXRvOEdMandm?=
 =?utf-8?B?cHdkclAxZi9hTzF5WnFCUmdPakIzK1Z4OU91MUJYVlYwVGMxQkRTbW1YV0hQ?=
 =?utf-8?B?eDJGUkxXVUwzS1lYRnhwbnFFRU1HQXJOemkyQVRxM25oNDFvL1lXZFNXMGVH?=
 =?utf-8?B?V2piMTdiaGFpcTNQbnA4L0wyRjF1a0paV21SUERVM0c0LzBKblk1QXBhMmFR?=
 =?utf-8?B?MGRxMTgzS3g5Qy80MUtPNjdzRGxLYWpROTQybHBDWFI2M3Jla0xhU2FXejFR?=
 =?utf-8?B?U2luZmJXalhsSkJDbGlUZTJNOHZTckdQMWJLeExBRGJpUHNjUWowOWkwU1g0?=
 =?utf-8?B?aFVFSU9LNS9zbS9paVVocnMvY2xUUDk5b29yWHlYd21VOUlUTURKejZ0SEEv?=
 =?utf-8?B?UkxveGR3ZmdGeDgxbE85aGN0L25KNmlmQkxMUS9NRTEzQUVaVXNYNlh2eHpT?=
 =?utf-8?B?U0hFN0dCNmVRQ2NiaFlYR1c1Qmg2dTc3NnpvQVlvTTlRWFNUVjhmSjF0STBm?=
 =?utf-8?B?Uzdla21sN3FxaUk1K3BSenNGVGRGSDgzb0MyM29KZmtnck1IUFh0K2x6dm1N?=
 =?utf-8?B?Nlc0MTgvZjZDNDVNNE15aDluYTR0OFFnb25hUExiWnR3bXM1cWZRTUx1VkNy?=
 =?utf-8?B?S2F0eUtZM1lDVDZqcWpsblViaDhoR3J6WHZlMUU0eGNrMEF0TFRXK2dRZXJ1?=
 =?utf-8?B?MlV5K042dm5tU2prVm44VGZwNGxUTGt5MDd3b25GWjRaeWVPWU4yai9XTndw?=
 =?utf-8?B?NnNBeFdvUTBsZmFzNGtka3VYVEtuaEJvazh0Skl5SFJpWFBtdHR3UWF3aitQ?=
 =?utf-8?B?Q1hkMUxFVTI1ekJPRlRVRk16VnlkdHprb0JyNGdSTHlGSVAzYlI0bHU2VVMr?=
 =?utf-8?B?bEo1NytBUzRSczJSbVM2V08wS2NhMGZZVlRlT2RjbE9qSEdHMkU3ZGIwZGtx?=
 =?utf-8?B?cXEwclBhK0FzVkIvY05BanJaMTh4akdxK1N4Vjc4Rm9PRFY2ckRuZVdsQUdo?=
 =?utf-8?B?QWhoOSszaGZ4QWluZnZVczM2QlhjaHJNSzdqL28zWGJDUkROcmJ6NTJlSjhp?=
 =?utf-8?B?TUM1Z1lETG50Z3NxbzRUSlJJZm5yckV6cWdaWUJPSVVUeG5BVDN0UWlKcE5a?=
 =?utf-8?B?SlEyWHhOeFVCU1FwUjN2SmoyM2VYNXBHcXdVelJxV2RGNldqeWtpbzF0R1Nv?=
 =?utf-8?B?QVZwMy8yMVgxNDczN2pwMy83R09KdlZiTEI0Z3paWHFPNm9TNGZzUFRrZlRz?=
 =?utf-8?B?NU44WHJBc3BJS0cxNjNjRHdjUUN2S3R2Y2l5THg0OEdIYmUwbmhwZFhnejkz?=
 =?utf-8?B?akdyZDlFKzB5RXlLSTNWc1FndVlxVmRVQU8vd1dyUXRHSjRubHRyWkhmVW5U?=
 =?utf-8?B?NXVEb1BTTkZjSDFwdVFQN2RzcmJpSmNpRkVOSlMxeUFmZDZJZWl3UW9kdTVV?=
 =?utf-8?B?THJPUUViL08xa1oxeHhydmwvZGpwdzFGc01UOWdGL2NBNzhocEhqVGxYY1Mw?=
 =?utf-8?B?WVZRY1lpOVBnOGUxZUZ3eldxQSttYkJFNFZrOHhvRllreWlGUXRCQU9wOGdq?=
 =?utf-8?B?dlh5ZHRtSHc4b3BMK0NRM3EvNW1TcGRsd0dodHdralJOK1dqTFkxQVA2OTc5?=
 =?utf-8?B?QUY3WEY3ZFR2bjRxYnM4c2JPUzUwOElGZlB3T0hYRWNPQU1QZkZZU1pjcG9L?=
 =?utf-8?Q?21TaPGaJ6xAhpr62qCt52smv5IZm/AEGvAJOCra?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ad8c43a-c84c-47b7-8258-08d979c80d0b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:43:56.9593
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AWGIGYVkloBgfHZ6GNeFSicjF0b+mu48RUrOma40TZQqqdWTYF/JyRCi/2mtc4ez8CKMO+5valWTjm1qLcoYPQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7087

The primary intention really was the last patch, there you go (on top
of what is already in xen/tip.git for-linus-5.15) ...

1: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
2: PCI: only build xen-pcifront in PV-enabled environments
3: xen/pci-swiotlb: reduce visibility of symbols
4: swiotlb-xen: this is PV-only on x86

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:45:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:45:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189243.338817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBNW-0001FW-CK; Fri, 17 Sep 2021 10:45:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189243.338817; Fri, 17 Sep 2021 10:45:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBNW-0001FP-8b; Fri, 17 Sep 2021 10:45:58 +0000
Received: by outflank-mailman (input) for mailman id 189243;
 Fri, 17 Sep 2021 10:45:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBNV-0001FH-Bz
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:45:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e2c63ba3-9a54-4ac7-bd9d-d83c977a2db1;
 Fri, 17 Sep 2021 10:45:56 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-GrDtbzgpNkWMdQW2zCUVqg-1; Fri, 17 Sep 2021 12:45:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 17 Sep
 2021 10:45:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:45:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0085.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 10:45:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2c63ba3-9a54-4ac7-bd9d-d83c977a2db1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631875555;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xygT969cDBZ/j2pA5jAQZPS3mU7sdWk4emBIUX/9lVE=;
	b=W9TAmaYNRJX6PYHTNKiCWHjq0BNDf8SV3TKCRCC5PYoinxmk2e3IUZOgaRRU0k8dRZ1fYI
	Wy7d/uPBxu1US9dBO9scyjvHwVAnpMifEq/xnqM6itJwwWyeg1ohLQE/nz6LlIzPDnwfiz
	J3EjnL50pBwNKnhHqPlwvPWmjVGqjAE=
X-MC-Unique: GrDtbzgpNkWMdQW2zCUVqg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QipcfJjC3Xsc6Dmx/BuF/G9EHVfVtUX1uqsYY8SDfToaGCDxpA+2TmZPbBhITA6nHnVHJ0l21rS0x8aBxdOBbnjkNDRDsxNVKNhxx7J4zNT13fY2A+/4sdTb3NycPl77+nW6vC+EVBDYQEvZ6f154TyZnr2zizkCOrlwjId6oA/pUeLqRFBsLFdU57DAf1dDOrcKaBhOXkJskTDB+LuCONqvq0EYecKduk3pn/1cvOHdL/LS7R+DDLgYhWpSz55dj8V0LWCMJT8wLcp08N9IggF14TtNqbb56mVz+ufn/PmBc9d7THdv+K4CLzmiLZ2nyEtTA37RmfQWHvyuQrpT2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xygT969cDBZ/j2pA5jAQZPS3mU7sdWk4emBIUX/9lVE=;
 b=H1GMQi1hiWB9rb6AhRrlSKlWcuoeaNGgzGBuZYAqHnGuSrUz5GLZlU+kIdLT5t54O1a9LRYRAoFF2prEy2yqmnWhmnh7cB9NSTPzdmrzQqSviyFyMMYcF35DJVslYFfyIOZzMUQwZBClVljmx6qObMnesHdp9Nxaofp4bL9/ZJBNe7GFoRHSh7pthHFls6Rah1ZyPfnExcnfp2WIv5taFJlwnjhxWZ7RwYP8CR8BzflpJ8PFb2KiBFscCPFuqg+XVWnskRIKoJeAeuC1L5lqHmx3bOqjhyXtVSs6ZWLNx78yXohowi0mJNnKS4mrQODRX2X2j7roI8HMjE46Opbx0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.linux-foundation.org; dkim=none (message not
 signed) header.d=none;lists.linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 1/4] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Message-ID: <7b3998e3-1233-4e5a-89ec-d740e77eb166@suse.com>
Date: Fri, 17 Sep 2021 12:45:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0085.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 51a8b658-b3cb-4ac7-1729-08d979c8519b
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038133B5FD6F34E9FEB12C8B3DD9@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1201;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jJvfl4TOCMxZk5Hkw6X1UGtuuJekKSpg5sbARmQDJzbR4PwzObuEuRLB4ElgjWxJ4DozlfkadWHD37Y3J/e9Fy86fiIg0OS/iWfK2TcROwvraU1+z69n8vFDvx9yzQ3DYLlwnF9Q5eIr0tMWUJ8Njc0xR+mXmdbdhauFYRziA9WjaY+LuldbvUY6HrMJX0Nb0ZBYtwLv3XtX34tBqlZk6YjVTjlgR5boCKsL06fnI0MKVnNdsV4RuV6B5dmqlHC7RoUo/9gSFu6+VuNgPweWUYooq8ABV53bwlXHVK4mBG6ghApj17HTUihckeXoOTwQZDrPgKE26eAikPz8/T+iCmCQ9NMGzU0OAdGRZBa7/9VoqowHDX7gEHT8TlSiu5JphBiuMeT1f8Uvc0841/9OVTLPWlMOwXhSlab7mdkMFk6S/29MvxEKSCJcNcuMECJLNJb93r44NzT7hf3sfBGLqAGMPZ2VpT7d13Rsc7BSpkZh47TtnLqyqejmZKtbKtCLGSZjqmN557F4nM24XgB6BKdZp6tSfhxL4fMKm/C15vYCTu75zp3gZcB3H6dOWT76KaIQ1R4lhlvlv6m4ZCoPAsbcu0yjVOV2OWYRN3UIATu2NHcDddDyz3zvHd5Tzcr41ZdZwEIKbbLRVNWjmJEFIZ3OFMC2AGFbpwBgKPRYiDl7bMq9WzZjFiS0LIv5Z59YeXX0NL9F2vBAkUVLHcYLwY20uDlEHR6JCBLUfdReuFg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(376002)(39860400002)(346002)(366004)(26005)(38100700002)(83380400001)(8676002)(36756003)(2616005)(66476007)(66556008)(6486002)(8936002)(2906002)(86362001)(186003)(31686004)(66946007)(956004)(5660300002)(316002)(4326008)(54906003)(110136005)(478600001)(31696002)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Nzh0eE1QbWhKemFEN1F3N0V0anEwT0Q0TTNSb1BLZlRjOVA2cEtjQ1gwRVJT?=
 =?utf-8?B?eTVEQitvRzlKOUlXd2Q1MTVQK2tRSzQxZnVJZm1qOXFyNmlQVENrSVVTWW0v?=
 =?utf-8?B?Y2NRb1o0V29vY2c0elJlVGE4NjNFQ0ZnQkVpQlRVb2dScHJ5Sm11dlBBalQ5?=
 =?utf-8?B?L0ZsaDV1YWpDMG9kTnl4RUFJS2tsTldtUlo3TFNZSW10d013Y3JtZzdzTGRs?=
 =?utf-8?B?OVZBYzJvZCtadmlYMFF3VGphNUtZZXdhck5vNWE5RzZxOWtjaXNONTRqT3JE?=
 =?utf-8?B?eFNsUi9aMkFGN2dyRFFJTHpkRGR3MWdWeENtYlBUcUs4SDJubXdTSjRQWHF3?=
 =?utf-8?B?bkl5VWRpTW1nUXFnSmhubDkzYWpyMktTNzZ6R2JWTkk3aGlBbE5oNE45Z09R?=
 =?utf-8?B?c3JkbzZNRS9kYkIzU0JNbTFZYzV1aGNDaFp6MjdxdDJ4RkVmRHgxOGc4SWJy?=
 =?utf-8?B?TTdSL0VBYzVwVVM3THBBd05zeGYzbFF4TnpNSnpQSENNeUJwUmY3WkZaVWJm?=
 =?utf-8?B?eGoxSnMvejZTRUtmaWVuQkNPS3VWck5objdwUC81d0JWSGVDVnh6NWJCK2Jr?=
 =?utf-8?B?ZUZudTEreTlpVTVEdExCSUNtU0pmWTJvN1lwejlNdzBXeS9WVFQ1ajN4N2tn?=
 =?utf-8?B?a25zM0hvRXBnZ0xXN09mbGg2VWM0T3hQc1c2VERyNHl1cW1uMVZYRDZ0Mi8v?=
 =?utf-8?B?SytzT3p1VHlCTGNpWEd2eElZcUlBaVg0NTY2ZkJhZ0dBNUtlSzk4TU13SWVu?=
 =?utf-8?B?NGdLNkI0WEIxbjcyNEpoRUlmdnkvVmpnN25TUDdHVkpMUVUyeGUrN2I1eUFx?=
 =?utf-8?B?UEFIeGlEZWliZjE2RWpDc292SVdjR0tOTXU0Z1preDdiOTU1eWpaT3Q4Qkd6?=
 =?utf-8?B?RXB4VitOUVJMNHBEVHVvU1dNdU5RRkxocTlzSVRQaHJuSzZ5bmI1LzF1U0xj?=
 =?utf-8?B?MTZIUUNTRDU2TzY3Z1M3QmZwcnhGaTFsNjBEUm91ekRmbVpYd3pScGw4bFYw?=
 =?utf-8?B?dy9yVklFejV4cFplb2lXSkd4aUpQNVplYTR1NHVyZkdxeWNTK0d2bHZuYTlV?=
 =?utf-8?B?TmcxMG5rTzJxSThzU2xWTzRRUzhteVlJSld1RWllcjBOWENrMVBubmhxcVVB?=
 =?utf-8?B?Zy9PMG1Uc1VtZVViUHRBQU9GV1FGK003QlZPVnhkTTQ0aG1qRVVORkdJb21X?=
 =?utf-8?B?UVRGTEIzOWFMNENUZ0FVaEFBc3lPV2N1bnBzY3Z5eVR1THV4NVBFcVV0ZE5M?=
 =?utf-8?B?dzY3MFQ3R25WVlV5K1A4QTRBWTBRMk5ZcE1hRng4R2dMUFZBMDBSdE9RRjBu?=
 =?utf-8?B?VWZxMWtSaG9pU3Z2OVhDV3ZGWjMzTXNzakVQVElJb1lEa3hHK0Q5aWxBRjNo?=
 =?utf-8?B?SkdNNzh5R3hlQ3pqeHBIRlgrWVkvOWl6cnNSdWhWSGhUT2w4SmE5amlzNERv?=
 =?utf-8?B?d2F3K1p0dFVOYk9hbDFGbTlPSSs1YVV6b3lrT242cm5XK3ptVERMeDR3djFD?=
 =?utf-8?B?aWoybDRNR0FtM1lJZ2xGK2NOZmVMZHRieGVIbDcrYUNBVE1zQnVKYjFheDla?=
 =?utf-8?B?Ykp6ckhXZnpOMUlmcWVuOWtLVWVGVjZIM0JzMTVYeGxxWXUxNVp2Qmhmajlm?=
 =?utf-8?B?eUNCamdqSWd2WUhJeVl5ZkpJQUtRaERtdnhHeWFPUFJyeW02MXo1YnpkV0xZ?=
 =?utf-8?B?SThhSlhCQTBpdHprZTJFbGUrK3FiS1ZoNFBiUUdySjU4RTZnNHZnZERqck5M?=
 =?utf-8?Q?9BNYdZqpM/QexeFOt3WvwlNcoxKVn3MxN/rO72D?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 51a8b658-b3cb-4ac7-1729-08d979c8519b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:45:51.9854
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gg4diNqLAQ1fx64FdDFPcPfx1T/o/UzWchvG1YWd1pA9kTjucQnGvrZeUel1lzk+QfIzb/1NrKAowws/Hhq85A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

While the hypervisor hasn't been enforcing this, we would still better
avoid issuing requests with GFNs not aligned to the requested order.
Instead of altering the value also in the call to panic(), drop it
there for being static and hence easy to determine without being part
of the panic message.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I question how useful it is to wrap "bytes" in PAGE_ALIGN(), when it is
a multiple of a segment's size anyway (or at least was supposed to be,
prior to "swiotlb-xen: maintain slab count properly"). But that's
perhaps yet another separate patch.
---
v2: Drop logging of alignment. Wrap lines.

--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -230,10 +230,11 @@ retry:
 	/*
 	 * Get IO TLB memory from any location.
 	 */
-	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
+	start = memblock_alloc(PAGE_ALIGN(bytes),
+			       IO_TLB_SEGSIZE << IO_TLB_SHIFT);
 	if (!start)
-		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
-		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
+		panic("%s: Failed to allocate %lu bytes\n",
+		      __func__, PAGE_ALIGN(bytes));
 
 	/*
 	 * And replace that memory with pages under 4GB.



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:48:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189251.338828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBPg-00029N-Ue; Fri, 17 Sep 2021 10:48:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189251.338828; Fri, 17 Sep 2021 10:48:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBPg-00029G-R3; Fri, 17 Sep 2021 10:48:12 +0000
Received: by outflank-mailman (input) for mailman id 189251;
 Fri, 17 Sep 2021 10:48:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBPf-00029A-Cj
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:48:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 678291d5-51f9-4473-b3e7-2065d00adb81;
 Fri, 17 Sep 2021 10:48:10 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-yXw_6uaLN6SmmYHNTNyg4Q-1; Fri, 17 Sep 2021 12:48:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6672.eurprd04.prod.outlook.com (2603:10a6:803:127::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:48:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:48:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P251CA0018.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:21b::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 10:48:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 678291d5-51f9-4473-b3e7-2065d00adb81
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631875689;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jXZiYU8mKeQlBechJ1DkPrS64flo7XmAmn4tzLrzntA=;
	b=XpYcTTHXhkhmQOzrrSIglMm8Psb5R1+L5s2vC7NH8Mb6aS5zuVtDjl66GvYKd8Mw8Xquvo
	bFtuBgQdkzyTQ0/A2hGUou+WgVDc6g601BIm+KQ+tI/pgjUTOcXfKolqMJBW0detT1Cq0L
	0Zl332wserL0y/jXwQ0Qzie/KmyxuDc=
X-MC-Unique: yXw_6uaLN6SmmYHNTNyg4Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=adtmQkJ1ULl8weYlXLfqUczdzyaPdyGdqyc3AA6q8JQynzvHLdATv0VrQCDvvdSE/b4++5KCtMwrPApM5odmvu9iGGOXKEoiWMunbhAMqZZVXZov2anky5GgFgYPrswuiH8/ON/VQUvWzh6QvPYzit41GScDYvql2IZ+wf8ZhwR/5PLVh+u35wwCWJwxZU8JXyr/mN4tSsRHhobOsaja+a9g2d8RGohew2O8WqabehqDkZ9aimQVsV/Nvgfcz5XmhtKjqGQJ0xmL7QkEhIDJJsrwhkHgD8Ob+6CGMXEoJUoshIJIfrLiBNKZg+zLYGQYQ/trN0ZD00mRjs7qXjOOXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jXZiYU8mKeQlBechJ1DkPrS64flo7XmAmn4tzLrzntA=;
 b=j/nqEUCa7GJUd+0T1JKPTVMyal9G/iXU+pugQMt96dxm0SoemHhwFG5AGbnd/M5s5DKyee3+hD7YkFg2G06bC55E2q5awNYg+tz2fcMbDnXhzUH0BjB9D+4MjsVrNyGyld6Du/7SwMcTgbJBLa9D36X+iNQiMyDbtuU6WRlqS5PoENKS7NWMTyXJmqkXqhAmBrMI2iMCj7nU4hX7qmXcFVZFigXE1zD/Ppot0oJkVGy2ZwrBpcf+qTlLlF90cLni59PfnDsff8P1cpKmREtce8m3YLfP5NcLo+8NpUQ8Uvg6BKYxwrEdQn+Pb2b42oSyp3Yip0RTiouLk8DeHfv+ng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 2/4] PCI: only build xen-pcifront in PV-enabled
 environments
From: Jan Beulich <jbeulich@suse.com>
To: "bhelgaas@google.com" <bhelgaas@google.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Message-ID: <3a7f6c9b-215d-b593-8056-b5fe605dafd7@suse.com>
Date: Fri, 17 Sep 2021 12:48:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P251CA0018.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:20b:21b::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 04d4e28e-4bc9-4a09-c08e-08d979c8a182
X-MS-TrafficTypeDiagnostic: VE1PR04MB6672:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66723E1C4BD9E731501F775AB3DD9@VE1PR04MB6672.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vHhBhsWw8iSpZCCjgEQ2YpqZ/HxIdaG8FMKZH1wTEz2kM8WSZLAr2HmIU6FzFA004Vw2/xNXc8dN5vvGj3ioWcjaTiqqqxHkKki7Tq3dpdXAhn9hDa99z96dCBHW44ppdd8Eo6a9Wu06Q0Vaphb+FfNtKAluA3W+s+2TPata/OEfZO8yOHtCWidwuDNRS+02taRvCcVdGYHW1VtXq3ntXyIqYTxTlAzyZ6xmn9ZuYQXLlrZTsGStqMJm8kIyTHN39l2j1zukvZDMlqIHiD+2FjPqmJ/T4SIV2h4mfpH3SGXUgjLw2hqAXK0LQ2DDPsnS/mEofLF65drQxVYU0IkA57IY96NUX1Fa87x8fhhUgcHJGQfE6JbhBLCz0h2DsMoqoKTA0K/U1ndj7SkezAs39nLjJzqSpSYtJs7WsISHQgNZknDDlSyl6s1vooA96W4Ht89b7dj7WbHQnNvdt77wckuTfA40zm95lxB/LQ0PkMwPVXbZY4E4TAOBC/n7jpNa4+uLzlzZRK8f8PiX2F1CXW/bgbC93IlHAauhkbr8CcDuB9W96ZOvXxoPUop8bCDCHUnj4JKSpDxyE4lXaL+cDd2pvEmGkCnDzW5+JVPyOXzW9qRTqP4ovD6uD3h62SsQzWKf8Bt4XaoGdhiEOc1fESeGj9TBO0oH+NEj5bhnyo5UcbpLlMqRN7BH9qtHN1WCVo2hdOFGnKr9NtArMFr7+nZZ2VFSevPub6CDxB9yYLE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(136003)(346002)(376002)(366004)(478600001)(2906002)(956004)(26005)(8676002)(38100700002)(86362001)(31686004)(5660300002)(6916009)(31696002)(186003)(66946007)(54906003)(4744005)(2616005)(66476007)(16576012)(66556008)(316002)(6486002)(8936002)(36756003)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SzdKbmVlMUFDYmtaRDRyQWVYcG5WOVo2R0JaeDZiRXVZUzU3bUk1V2VBU2JI?=
 =?utf-8?B?R2lpK2NpRkh0L3k2dno4V0wwbU1TY2F4RnQzWG44eitraWxXWjloN0VCTkF5?=
 =?utf-8?B?QklHL21vSXRPTENsT01Qb2U3YlFTc1l4ZmIwcjgwTkRCclhLbzgzRmlGVXRJ?=
 =?utf-8?B?Sksxc2c2SFp5NVQrQ3ZIaWVCenV2Myt3dEROWk5hR3lnT1FWTldWN1g3TXpJ?=
 =?utf-8?B?YmNpRENaa3BWQklvSzJYZXZTUWdkNlZCWi9tVUNWNXUxdFRUU0hsYlM5akdl?=
 =?utf-8?B?SERDQ2w1NFZZMTRucEJ5ek5kdndRenAwU3lHbzN0ZXZDQStzcVgrQ29nQm15?=
 =?utf-8?B?a1Z6RnY5WjlIMFZOVXBaNUNYcUE2a2tEdnpJd21HbXhuMWlWeHBnOFZGY1Ax?=
 =?utf-8?B?c3JRUC8yNFBsQW9nVU1TN2hHNlRwK2JSZUVEME5XWmJjT1MzdWc0V3NhTVZw?=
 =?utf-8?B?Njc0anl2RDRkb0xhS3VPeXRVSVpadFJyR2ZtTWE3b3BDeE1wZzd2OGVMelBh?=
 =?utf-8?B?cklSbEpBWSsrdWhpR1hCZ0Q2UGd4ZVk3TnNkRTZ2WFQzaG1EWGRPUEpRZTRq?=
 =?utf-8?B?ZE9GWFVEdEQxQjRkSG0rVTl5NnM2TmdGbSt1SGVNTjVaNmJXaVQzbjhSZVcv?=
 =?utf-8?B?K3lMcVJ6OEN0eGtBOGFhOUJaZDFZZmJtdHp5MTJKaTBTc2xYREpmbjJ5Q082?=
 =?utf-8?B?MWlYcjA0UFBFb2l2emlmdWxNQm9oNE13U1I4NWxLZy9qenpQWTd0bkVocHdp?=
 =?utf-8?B?Q2RvamtoOWVwWkFXVzVRWVNjcUYzNFM0c0JNTFFuWVc3b1QwRHlxcCtaa0Ji?=
 =?utf-8?B?UnFiQ3NRNUpReldFTC9mcUNHdUhrRHhFUXpEbXJ5VFpBZG1JUjR5VmgzcDBn?=
 =?utf-8?B?azZIN0pmakUwQkhrbXFSbTdsVkZKdTA0bjdNM2lSVnVseUVXNEkxM0xaUDR0?=
 =?utf-8?B?SUkyNkVaZWZ0TUxCMTU5aCtqV0NieXFiaU5LMEFGUDBlU00wU3lQRnYyVFdQ?=
 =?utf-8?B?YjkzSyt6ck1GZVdrTWhNMWIzeFNtWXpFTDNGc2xOT252b2NZK2xBeTg3MG5j?=
 =?utf-8?B?VmFwMzNRNThpdGQ3SndncWI4WWxVbm5iZU5PSXJURDVLUXdzS1JiM2JVd0Qr?=
 =?utf-8?B?Y0g2Nkwrc1AwUjdDOVdGbjR5RTJCSWpRcmU2UEJLSnBWaU04aXo0UEVCaWdM?=
 =?utf-8?B?OVBSay9KKzlKK01jT1F1TXFtRk5wU1c3VWFXU3lQSjZLcU5vN08rNU5ta2xr?=
 =?utf-8?B?bytnTHkrWFkxOWdUOW43VDJxM21iczBPb1JlRTl4amQ3MWY2eExheVdHQkln?=
 =?utf-8?B?cmZJQVZBWE84L2hwMXk0Y1ZEcnBHVlJURUJYNjk5VjJabzhCaDJhN0ZXYXNt?=
 =?utf-8?B?WmhUaXJUWExDZ3NIM3JOK0NxVDBpWEQ1RXFYcmM3WmN4ZnkzRWZYeHNSb3VB?=
 =?utf-8?B?SEEzWDltQmtxd3dqR0RsaEdOb0hzTVVyZEZBQWhyM0F4VnFzUlQzTWJJMUln?=
 =?utf-8?B?d2FQbHI4ZEJ0dlFnVGd3YUlDN09qdlM5OTBYOXlJSzJ0a0MxamZXS21teEw3?=
 =?utf-8?B?VFhCTFZnL3pOZW9uK0pqbzB3akl5T29vUGErUkExeEtweUpvMk45N1RXRTJk?=
 =?utf-8?B?Nkw2NXRXTVd3RE4rRk5CVFdhZzY0cUEvVlhCMFp3eEpwNk9YQTFySFdlMXhB?=
 =?utf-8?B?c1g1aVBJSUVGUCtqUVluNDhnS3pCUG1WckNSTjhMZjNzMXU4elp3c05uUC9h?=
 =?utf-8?Q?y3xYC90l34sRKq92sh6lihAQYpTsyzjakHfQaFa?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 04d4e28e-4bc9-4a09-c08e-08d979c8a182
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:48:06.0546
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vOpc827aY4Y5uJnuMYKPwLaLMXRTrj1B2ZCaA9k5HR/UraVJh/wDUz8abVGpCg0ur5bn94OaTahuHMeNuhFB6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6672

The driver's module init function, pcifront_init(), invokes
xen_pv_domain() first thing. That construct produces constant "false"
when !CONFIG_XEN_PV. Hence there's no point building the driver in
non-PV configurations.

Drop the (now implicit and generally wrong) X86 dependency: At present,
XEN_PV con only be set when X86 is also enabled. In general an
architecture supporting Xen PV (and PCI) would want to have this driver
built.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
v2: Title and description redone.

--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -110,7 +110,7 @@ config PCI_PF_STUB
 
 config XEN_PCIDEV_FRONTEND
 	tristate "Xen PCI Frontend"
-	depends on X86 && XEN
+	depends on XEN_PV
 	select PCI_XEN
 	select XEN_XENBUS_FRONTEND
 	default y



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:49:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:49:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189258.338839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBQe-0002je-9o; Fri, 17 Sep 2021 10:49:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189258.338839; Fri, 17 Sep 2021 10:49:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBQe-0002jX-5Z; Fri, 17 Sep 2021 10:49:12 +0000
Received: by outflank-mailman (input) for mailman id 189258;
 Fri, 17 Sep 2021 10:49:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBQd-0002jR-H8
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:49:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4c18fcb4-61ae-4bb3-b232-d5537363191d;
 Fri, 17 Sep 2021 10:49:10 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-PkQaqTS1OHiajKqKIZMyHQ-1;
 Fri, 17 Sep 2021 12:49:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6672.eurprd04.prod.outlook.com (2603:10a6:803:127::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:49:07 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:49:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0022.eurprd09.prod.outlook.com (2603:10a6:101:16::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 10:49:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c18fcb4-61ae-4bb3-b232-d5537363191d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631875749;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kndVqjY9sCpaHVWNklWOKb8oFcdnKBVKLygRhlHka6U=;
	b=SEGytB2/zcGY2RWw1wKxosQ0Vu9FVjvGQok87OTnvNNvaErtDAoTZreF6fWmtlmXrtExpO
	ufTbFeGK0NZas1imsc/M3fiJAy41So9ZePQHdifBL5d+Ei2kYWl/bJXp/vykyaRNiCArlu
	Vpj0D9ktDcQRb4b5nI4V0PozMVRf7Qo=
X-MC-Unique: PkQaqTS1OHiajKqKIZMyHQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KmFMBGQFtMAd5FDk1XbObktPuQyY9PE29Vh1aztnY4BW2K8F3rL3Qt7cpX3iN8hKygHqsKIzGnRBP+QrxMLyEivpMih4CTbVQaj3oTzcSQ/+/9RArJPzdHz+35PspHpCEJOEn8v106F4WVG3l3b/64uW4Tn/gbReK++FEINH6P0aMCxqhx206x2BCBN+qONLjN9eCVFJ0A0qO/Wu5cl9MsvoUF5c73ao2IPIsFiC+vz4v6o80E0t2Vcd94k5VYJ0IUMAAGS/n9uFzaWLCBsP4c6cDSicJmi0xLNmXKujz58wz8ygb2r3f3RCzwaOnadNHUjXWsHgR5al5/zn6VDSdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kndVqjY9sCpaHVWNklWOKb8oFcdnKBVKLygRhlHka6U=;
 b=QoLU57JC/x81rEFxjlfeX6kCbuxpTWzm8rUaFZ/IWYGStvQlfH2VnwEh0Q+oXre8WysIiQN9pIpiUSiPcT0lE1xyY5+EqRF7Ux1eU9hV1yVQ86NAIYGlDw2OfpqIg4hdZ4xv6JFx8qrm9oswts//eKLJ1cMUoltg+/gxiNiDbQJbEAHFnbi/oqXfxi9GzEghmLU8umfsu2GqBLZ1xSetSdyBU4EC9v7FpR95tOfwzR5CRhqRCVPcA1fFbAYZs6Pe8eZZED4UhoD4As7FrF+i9vvKQanCOfOSB3EmJJI/v1GEc4rbwx5JnU/EKbvJx8FzikvOVgDFKwzfDaN9F/mblA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.linux-foundation.org; dkim=none (message not
 signed) header.d=none;lists.linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 3/4] xen/pci-swiotlb: reduce visibility of symbols
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Message-ID: <aef5fc33-9c02-4df0-906a-5c813142e13c@suse.com>
Date: Fri, 17 Sep 2021 12:49:04 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0022.eurprd09.prod.outlook.com
 (2603:10a6:101:16::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a54bc54f-3d61-4688-7218-08d979c8c5fa
X-MS-TrafficTypeDiagnostic: VE1PR04MB6672:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB667250B1123561ABEC648420B3DD9@VE1PR04MB6672.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FWbW2r4tXDfuI1LT+hNd8i8PNpnTorkZ1xGGNh/073M1Ypigs4NJuPrFiyj9axEXr+RccClClbveXpPVAEBSSC9wuKFZvZQUn0gE3urO947gXoUVKr7CWlt72yRoqhWisqSiSk0wxAI6/N3AKgj+JuX/wMmdTGmNrUrhlT3kiiY75TbPq3kJy8CImjIW7JfAW7z2kQ/q6/Ix7lh6xAODw0WUfMdQF6QCiSLPW51jFBAdi161yv6nQ8gENfIXaeHbKDLcuoGn9SVfQBk/mQkDsCO8zdJ4OxmfCfEgt6IePAbN3o+Orl3etS6bkSi5iP6gDqOqTO6T1heCT5Wz7BAWIuCWxPsOt7ueRS7GdDBtk8LzAs/EzbUHNR/aFbpxHo9d13f0LJPzKMQVvBmGUSZx9PBOK+dMf4Apiee9f0FOQw6L4axcdJdRjEkr4ex7sfLlZbbL8kx2i/Xh117Itk4x34+6FCKgbSAO4U4kBp33HqklGFaC3MEQL//uThTCwVDSge96aq0Go+ld52nOZ4cCRC/WN4QPCDUXx7m/4SkM5ZONL4Pitf5Ib9bqHbsI/cmQG32b8Giq6ezBvk2wwnBF6WIHdtZKv7HG4NZx2luwb3juQLpf1piGin5AL3R8pFZbxtbhXmJwyZ1BlttujrghYjc1imrv2rDv05XNuv6+MtqEofMKjEPatyf6thMhS4n6vyEdfaBRUSe9J8VYBMdUm6Ehk7abZw6Gq5M5b8e+RYg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(136003)(346002)(376002)(366004)(478600001)(2906002)(956004)(26005)(8676002)(38100700002)(86362001)(31686004)(5660300002)(31696002)(186003)(66946007)(54906003)(2616005)(66476007)(16576012)(66556008)(110136005)(316002)(6486002)(83380400001)(8936002)(36756003)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VmplNWRWQWVlUEZqNDdpQ0xYR2l6bk1YbElkRmRFSzZTVUZGT05MbVhxK29D?=
 =?utf-8?B?NjZHN3RKUkpuK3p4QURYZnZUTnhGdVkrKzdFZkIwRzRFVE05QXpqOEx6a0Jy?=
 =?utf-8?B?WWg1UDB1NlExckhlOExTK3c3ZjNncklsUFA1Q1c1eHFGOUVpMldCOHkxWnpO?=
 =?utf-8?B?OERYTHpESGg5bWxBZkxBaTIvVFdTclVwV0FVTTQ3K0M4UUozVHpOazNRV0hz?=
 =?utf-8?B?RnpjVXR2OTFDcGlvaWpybUd6enZySnJ5R0RoUTh6SnUwQ2JKR1lRMU1KaXZk?=
 =?utf-8?B?ckxOWUR5NVhROE5qQ1YxWkNWZXo0WThrN1crY29rOXVGVEdsUnBvOHNOQmhi?=
 =?utf-8?B?TDB5bkh4aXkzOVhKR01tWSt1VjN5MjA1Q1BzTEEyMHh3V0VOVElYeCtWeTRv?=
 =?utf-8?B?UlhkMWgwZTYvQmlnZE40VVp3K1Z2QVVzQXpIeGZjTW44RjZNbjhDdjc1bDdC?=
 =?utf-8?B?SmNlc1p4ay9lOFF0MGdSeWFCRi91UlFTTGZaem8xcDkrM2ljYSttQVVRNU5i?=
 =?utf-8?B?azJOdFBrNFc1d2NQSk0vcnNyVXRraU9kdFhNMTQzLzBQMDN6QTJxMjEwVjV0?=
 =?utf-8?B?QktYYitBUksweWNzUTlMbHFpUVFmVVVUME9mVFE4UEZRWjRGL0gyVVJtL3Ny?=
 =?utf-8?B?R3REVElvUERvOHlWNU9pamVQMldtM2w3d093WnVyRWJleWVweXhXSi9KeXJh?=
 =?utf-8?B?Y2kxWTJxMU9talJKSmoyNG12ZkgxT2dRTU41bE5SZ2lIQStxVjJNazFTeUk5?=
 =?utf-8?B?ZEE1RzhrNXROWHk4RVJxNjlEMWVRQjJJYjFFSWpyRkpaSzJUWVd3OXpvYURi?=
 =?utf-8?B?VEhqWng4b05IYS9uRFQvS2M1bGRZTjF1R1hzb2tvajlrZ09Qakw5eGY4MHBl?=
 =?utf-8?B?RERzdkZxd3Z2YlJFSjJBUnJDMXNPVGtRdTNZZEcwdnJuVEFBRnpSZzJnYXVJ?=
 =?utf-8?B?NjgrRkd4SjFaWEJqTlFaQkwvdUNuUy9kdzRNdmtDa0dXbit3TWhxYmdtMUdQ?=
 =?utf-8?B?MkFuMmw1MC9NZ2dHSU11RVpKbmFYWmc2M24vZ3JMZnBlbDVBRjNNNWt0RG96?=
 =?utf-8?B?ZmRnbnNjZWd0dkNKL3Q1UDVpQitOL3RYNURRd2o3bTczYlBXVldkcEhnOWtW?=
 =?utf-8?B?WTFtUVgvaEtkSGJhaG56TnZJZi8rNU9MWWxyVGNtdldFakpZZjJoZnN2cDFL?=
 =?utf-8?B?azhzeDJiRDZnY2pRemFEQUtZUWhnYmR6MTludGhBcWwrb2ExMC9FY2VmSzds?=
 =?utf-8?B?eXBBN1l6L001K1oxaVhHdFdhVHZPTmkvaHdSenZSVEJkWTNYTVVMSHlpRG1W?=
 =?utf-8?B?YkpKZ0I3aFJaZkRqRnQwUnZNMjhwRFdra0REM3lRY0kwUWhKajR1WHlhME1I?=
 =?utf-8?B?eXBwdGpDV05Qc3VHV1ROWnhjbkZwcGtlNVFtc2M1VHhQYksrZVVscnhNaXo4?=
 =?utf-8?B?YURLVitDa0RGV015RkVOMDlYbUtOcVBtemxtbkRJUkJkNVlYWUl5QVNyMlRM?=
 =?utf-8?B?NlpEeUxIS01IWU1BVktneVdvS0gySnY4WVN6bklqRU1GZXZFRFJLSVRtbEd6?=
 =?utf-8?B?dCtuT1RabEpFZ3lVNFA2TUJMUy9HN2t3byttcE9sWjF4MkFqYm5aNDRXWmNx?=
 =?utf-8?B?ZEVEMlVsZG9aREo2cWJWc2h5SnhUQ2s5QlU3VUtjS2hrY1lVZ3NSMThKd1pm?=
 =?utf-8?B?cTR3NkZhaWVraVVLcFJBRDZET2ZFelpnbHZBYXR4alRFNHY3Mi9xekR2OEs5?=
 =?utf-8?Q?Xriafnac+vppW8qKREz4bYAu1OGrQclhfDkfTDW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a54bc54f-3d61-4688-7218-08d979c8c5fa
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:49:07.2300
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HcSZ6RLEwzc3lSy0RMwhB+2RbPH6c97eWXvHMHfpiw80A3tdK/At6WCsCRoK6BnEU6oza9I0Ryy8ZoqKtJZqyA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6672

xen_swiotlb and pci_xen_swiotlb_init() are only used within the file
defining them, so make them static and remove the stubs. Otoh
pci_xen_swiotlb_detect() has a use (as function pointer) from the main
pci-swiotlb.c file - convert its stub to a #define to NULL.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,14 +3,10 @@
 #define _ASM_X86_SWIOTLB_XEN_H
 
 #ifdef CONFIG_SWIOTLB_XEN
-extern int xen_swiotlb;
 extern int __init pci_xen_swiotlb_detect(void);
-extern void __init pci_xen_swiotlb_init(void);
 extern int pci_xen_swiotlb_init_late(void);
 #else
-#define xen_swiotlb (0)
-static inline int __init pci_xen_swiotlb_detect(void) { return 0; }
-static inline void __init pci_xen_swiotlb_init(void) { }
+#define pci_xen_swiotlb_detect NULL
 static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
 #endif
 
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -18,7 +18,7 @@
 #endif
 #include <linux/export.h>
 
-int xen_swiotlb __read_mostly;
+static int xen_swiotlb __read_mostly;
 
 /*
  * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
@@ -56,7 +56,7 @@ int __init pci_xen_swiotlb_detect(void)
 	return xen_swiotlb;
 }
 
-void __init pci_xen_swiotlb_init(void)
+static void __init pci_xen_swiotlb_init(void)
 {
 	if (xen_swiotlb) {
 		xen_swiotlb_init_early();



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:50:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:50:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189266.338853 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBSC-00047H-Nt; Fri, 17 Sep 2021 10:50:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189266.338853; Fri, 17 Sep 2021 10:50:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBSC-00047A-KL; Fri, 17 Sep 2021 10:50:48 +0000
Received: by outflank-mailman (input) for mailman id 189266;
 Fri, 17 Sep 2021 10:50:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBSB-000474-Cw
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:50:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3cd70413-8da4-4b02-83dd-19a213e7d817;
 Fri, 17 Sep 2021 10:50:46 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2056.outbound.protection.outlook.com [104.47.8.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-P8aNfK2EPouioQHcyxwjtQ-1; Fri, 17 Sep 2021 12:50:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:50:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:50:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P189CA0028.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:218::33) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 10:50:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cd70413-8da4-4b02-83dd-19a213e7d817
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631875845;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9m0yUEYOIwtXoQEw1AT8DnQj0nd8vXqc0yecdszzCso=;
	b=UMomkR0eS59cs9RW5DgvnzQlB34/90gkUTChOq6Kikr5l80T9PaudPUPBARt8Y5QgXTZbQ
	tum4Pq74QpDueHd1LvQxjUEbJTBOQZ4MSMq72z46hieqeU5OoFS1emSQHTp5Mx+8yFDx8X
	X0dtg720M2d04ETA5ZJ/jhPJuWLzT5E=
X-MC-Unique: P8aNfK2EPouioQHcyxwjtQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V1tlc4SNiDViK5kqmMb1qtbfmRiQGG/VxWMn1zrAe5ppX7usWcjm2xeW0VwrHsQtTrs3nY9GSTaev4NR7/Jgu1AO9NPqu2XwOmL1KEXRjNO7qh5KOeBUm12ihfMJE7tN1zuCMzn/BHepXG7cAGz6uL4FHrYRv2agrCSXP6INazxjklfOtp/Piu18H913bDv4kHiKsRsMNl1eroePs4bwiMQPCDp6Qwbi4NzgARNgYEzq2caujnZgKtVvnlksnqevKiiOt5K+PpCkpQQmIDCPvAK9qL3AL2l3tpMN+rTZjAyDAcmR2oNtCOqxmmidjgdf1HYXzvGlFLyyvBZHViwDgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9m0yUEYOIwtXoQEw1AT8DnQj0nd8vXqc0yecdszzCso=;
 b=lIZLqxXq61JykHKxx4DkqsySe4RfHQEVjwaktnQnXvfytG0WQ7rLRqSbA78mWjR7XgcylHnMgMvA7eMNT1l6lev7GU5J6N5R7flqrQDa27UbOtkj+vSGUatOKVMoA0fA38BnptJfZvTybbPgugwcc8CmMIXPh2fPJd22P6LhiZ1Yuj1DYgBKqT1iMIhfd9ROP44gH4cE9xYMbAYCMUdWmNU3dg3oFy4Q4bx481bm/2nodw5bTxxWnO7+pdrff3USFquPlOz+rDHdzmzTHiHgrc91tVaE2G7968BlevzLbDonae8Os1k4pAB1oC/za/mdlKvnwSZMTs2jVifztHM0DA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: alien8.de; dkim=none (message not signed)
 header.d=none;alien8.de; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 4/4] swiotlb-xen: this is PV-only on x86
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 the arch/x86 maintainers <x86@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Message-ID: <5947b8ae-fdc7-225c-4838-84712265fc1e@suse.com>
Date: Fri, 17 Sep 2021 12:50:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P189CA0028.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:20b:218::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3666c03f-0fcd-42f9-0b1c-08d979c8fdea
X-MS-TrafficTypeDiagnostic: VI1PR04MB5600:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5600A6886B6CF4AB6A86FCC5B3DD9@VI1PR04MB5600.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UUvxXgay/gSI7NGlMWXcXdc4negUrzmJaVrSy9BcskwMNgUqRDBesK1uOcsBCdwFY7UYnnROSbdZyxR1D0Q9AC4HYGz+BQPeyaJGehZJpAbsfolYwC/l47cyOV8eH8czaU/eTkN5WH+yItVDkbpRVxdB8eoDcZOjbkyABHbDTP5DLaKA94iKFJvpUwzuiTRMQGmLii2HWgpcdpRuNSqMe55FHLQqfZfxbTKqIu677toceZbhz9UxuVil8gF3F5zJe7Mwsijzu1SZzqO1oU6ZnWoUZRiuUGu2si8ObeQ8A0eExH0l2eFYgtSs7EJ7NYEFnEOz5qgNNEP0CcUpQi1GURZgeAr3a5zvdY6dft9QORZAQfdQvfe+agjPi2PHDUff/40BUGnVyZMxowL1/Fe6wSWXewVtjrykoqnx9vssgd4u5L4Jbtw1zn/lXHX8A0Koe/51ETm4/r6RmeTUCtgpW7AqOe1Rl3XCQoCdmr48RB+FFrWN6R5oV0UeZoLnZ97wmlss+Lr3fkDDFdqDf3iTX8FSCsBllsiEmPYJka+IOSYq3y9fQ7xf3ZaNZmSvoHMoLfkSXO3KderUtjx5Uh513I2/fRwuKUYmGOZmC07862xtkD0Ru/DeDqfCjGG4ftUWeV/IuVs78N842FbIjuqwb/EvR8hQW62y/R0c2Exk5ybR32kB5M7fy0vVe9dcrkNn3ZDKi5sWBcBqZoB2wirGBXYdfEHsfp8bfuByjyvv8tE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(31696002)(7416002)(66946007)(508600001)(66476007)(66556008)(86362001)(31686004)(4326008)(186003)(26005)(38100700002)(956004)(83380400001)(36756003)(2616005)(2906002)(316002)(16576012)(8676002)(110136005)(54906003)(8936002)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RUhEZHdvMDF1b251WnpZOTJJMkFXRzk2YVo2M2E2RC9wWjZtdlQyckRrNzFt?=
 =?utf-8?B?ODFDZmhMSmZhQ3M0NUwxNUZRcW1LdytIT0l3cnA4VFNyaGxDbE40Uld5L1Ax?=
 =?utf-8?B?ay84QU1xYjhFWDEzTGFqcXVLdmhaL2hkK0F5SThoWXlSWnFuTWkwQjA1dlNy?=
 =?utf-8?B?NVc3eUNQUlVSeXhFbjFEWkk2aDVTNlVsUzRFeW16ZmErMWR5MzFaZDgvdkM3?=
 =?utf-8?B?VEQ1QjQ5L3dwTDhLdEdNaDR3Wm5LV2JIVzZFbnpRaCtha0Z1TlVib2RXVE9Q?=
 =?utf-8?B?Q3d3SkNqa1I2cEljYlFIa1dkMWpqTENFc2d3VEEzTEgrWFNUSXUwcFJ6U210?=
 =?utf-8?B?eitjcVFlaGRtZFo2WVNWUnEzeUQ1WlIvQ2h6eWh2UkpUdG8wZXF2Z3daWWNT?=
 =?utf-8?B?bUFSWlVaYjYvdlc1NktjaUM5YkZxUTZCU1U0Q1QwTlZIUHNMZUpZeW5SVUdK?=
 =?utf-8?B?RHk0dUNhYUxSbGFWTGZrOHVoQTJlcS9TWW4rd0E2S2tRMmdIZGRsempiMkNi?=
 =?utf-8?B?cGkzOHFQZSs3TlRlTEVwMStSOXhWS1FjcFMyMFR3UHZJSkxpeU1JbDRyd0dy?=
 =?utf-8?B?WllhZjkzTEpVZ043bmQyMGNsTGJyNU9TR2poajcwclYyUTNSRnc1aUw5QUk1?=
 =?utf-8?B?Y3diNTJsRmJwV3JaZ0hRajdYay9hMGU5VnVWTVNQZ0pVSkM5L3RMaXdKMU5F?=
 =?utf-8?B?Q3cyTk96QVpQWnhVZURyRDVYWjJoWlRsU3k5YXdwa2U5Uzhxb1MreWFYbTR3?=
 =?utf-8?B?OFZ5NnRsNW0rRUdISjZScUZoaFJYZGs2eXNqcDBtODJrSnh1Y2kwVFNSRlRF?=
 =?utf-8?B?WG9hT2JzdlVBQ2lEOUhxbytBbW9mTXJXdVVmallDb0FCOWJRclY5aWNpS2Qv?=
 =?utf-8?B?QTF4VTdxQzNDSFE5bldsY2tubGw0MDc5Y3ZwUjBUUXdoRG5oTm54cE03RHJv?=
 =?utf-8?B?cHNkUjFtM0ZNbDlndmZ2OGpYeVhRSU84a1lmS0dJb2lJZDFVbnBab2Raayto?=
 =?utf-8?B?QURacUt2SDZ2QmJSZU9KMUJSZWYrTXVkck9CeFIySzlHNm1wU1Z5Yk1TUnRS?=
 =?utf-8?B?TlBJTEExOGtScWJneHdpMndZZzJkMWI1NEVERjN4WGxmQ1VlUno4K3daN1NR?=
 =?utf-8?B?bklPYjBPb3dKNmc1cUhoaDVHN2luZ29qdUNNa1pwUk0yM2tLWWcyK2ZCWWp1?=
 =?utf-8?B?djZPVEEwN2NsaGhVNGpCTzN1c3hZTUtQOUEwWGluUXlFRjBIUFcvVjYzSlhW?=
 =?utf-8?B?MXBjSlgwd0pETjQzTkVaT2x5OUJic0NoZHc4dFVNbk5IUUYyaVZ6YzlFL21T?=
 =?utf-8?B?bEoySkFINERlNGNSSnIvZnJmRWVTYUNxSzlqTlBzWm81cXdnL1NxeWErNkxH?=
 =?utf-8?B?R3g4R2lQU1NWSUZibmp1SjR3RjlKSmQySkxSUnp3dlV0VG1VOWx4SmZ5Q01J?=
 =?utf-8?B?THpnQ0dYem1CTzJUczhhdU04VTdMdUJ1T3c1dTVvOHo0SmxWTEE5Ni9acnFT?=
 =?utf-8?B?UEFvV084c3FoYWYwYU5ldUlkWTBJdXdwTG9jNjd1dHpLblVYNkFGbUNWeVF6?=
 =?utf-8?B?a2M5L1dTUUo1Y2Rta1V1TC92OGRiWlAzSEVMc0FnUkkrK0tnRDl3aHZDTE96?=
 =?utf-8?B?RFE4YWdicmZUREdYVm41bmZFdVpzWExPajFBMUl1N0orL0dnUjF1Si9sbDg3?=
 =?utf-8?B?VzErTzVOWUJRaytIZ2Q3cW1zVmh2TWc1Sll4ZmxTRnV1RW41eHVpZk9ZM01L?=
 =?utf-8?Q?559Wla61/o03FiuOirM2RehDXGFXy/dFb3P91Vz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3666c03f-0fcd-42f9-0b1c-08d979c8fdea
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:50:41.0719
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fZ2RrmKjN84ov5BMOyBpAoEtGhqGW9BbnmFL3zdwmvG+tZ12+ibQYAmMbwB4z1f6V9bNY78pHT/TMDdkloOt6w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5600

The code is unreachable for HVM or PVH, and it also makes little sense
in auto-translated environments. On Arm, with
xen_{create,destroy}_contiguous_region() both being stubs, I have a hard
time seeing what good the Xen specific variant does - the generic one
ought to be fine for all purposes there. Still Arm code explicitly
references symbols here, so the code will continue to be included there.

Instead of making PCI_XEN's "select" conditional, simply drop it -
SWIOTLB_XEN will be available unconditionally in the PV case anyway, and
is - as explained above - dead code in non-PV environments.

This in turn allows dropping the stubs for
xen_{create,destroy}_contiguous_region(), the former of which was broken
anyway - it failed to set the DMA handle output.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2605,7 +2605,6 @@ config PCI_OLPC
 config PCI_XEN
 	def_bool y
 	depends on PCI && XEN
-	select SWIOTLB_XEN
 
 config MMCONF_FAM10H
 	def_bool y
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -177,6 +177,7 @@ config XEN_GRANT_DMA_ALLOC
 
 config SWIOTLB_XEN
 	def_bool y
+	depends on XEN_PV || ARM || ARM64
 	select DMA_OPS
 	select SWIOTLB
 
--- a/include/xen/xen-ops.h
+++ b/include/xen/xen-ops.h
@@ -46,19 +46,7 @@ extern unsigned long *xen_contiguous_bit
 int xen_create_contiguous_region(phys_addr_t pstart, unsigned int order,
 				unsigned int address_bits,
 				dma_addr_t *dma_handle);
-
 void xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order);
-#else
-static inline int xen_create_contiguous_region(phys_addr_t pstart,
-					       unsigned int order,
-					       unsigned int address_bits,
-					       dma_addr_t *dma_handle)
-{
-	return 0;
-}
-
-static inline void xen_destroy_contiguous_region(phys_addr_t pstart,
-						 unsigned int order) { }
 #endif
 
 #if defined(CONFIG_XEN_PV)



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:53:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:53:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189274.338864 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBUe-0004oD-9U; Fri, 17 Sep 2021 10:53:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189274.338864; Fri, 17 Sep 2021 10:53:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBUe-0004o6-5v; Fri, 17 Sep 2021 10:53:20 +0000
Received: by outflank-mailman (input) for mailman id 189274;
 Fri, 17 Sep 2021 10:53:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1wMc=OH=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mRBUc-0004o0-KS
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:53:18 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21d9aeec-7c54-4f41-9d2c-6e967361a567;
 Fri, 17 Sep 2021 10:53:17 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id g1so31304813lfj.12
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 03:53:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21d9aeec-7c54-4f41-9d2c-6e967361a567
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=GwPKvHPa7SWPdAxFNaWPHxr/qhyv9zoje0np7fedQKE=;
        b=WPqhGp7YFJx9AUCevICXrAodjCpr6nIpm2Ih1DUPRgTO2JIO4D88GB1S962YfHO9vm
         DRezWJVTxjj9rs0AZRWxJOWrvy9r+qKMeskCPE927IzPuRXFDgRa2iadYLcTMOd/64+4
         nzSruXNpaJKUZ/QJMboU8PnvfhM3HxH1/+n2Hxdlkx7TK9deyd8BenrLblqAQ+bMgzVD
         B6WKlUg0EoDvkkaNdaR+KyTdmSq6OG0BlpLoAPZbqIl9QPO8MWAsO5TjFbpFLhkwDdr4
         XYe66QAT2Ou7DTn+qqH8GubuMGhG8e16UInR2xW2MYta03lHYWn05LHNPgYs5vhLU3q9
         bpYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=GwPKvHPa7SWPdAxFNaWPHxr/qhyv9zoje0np7fedQKE=;
        b=hOZv71PoOYyz5sP2WIWASNJOxLZl+eVxivCEDXlLAR0meS7g5ARqLbUyRbm5zJCRg+
         BOuztJBUuuSULXdW+67xdrHK3CYuTNt6YUC7uxg3KAexJR9VAw1jMHuCNLr5LGJNFWi7
         9lMHsCqegVnbgoGTrLqP19L4gbNA58FcHtNUqJCOMj3HzaEReuGQgShizr056mugZC6k
         36TFqNyCzoIb1oYvuIh+5hNHqfercewm0rIghGF7y4KC234C/xEM6BJqzo4fSxhr00Wx
         y1ua8m15mF83g5gF74gjrVw7eWdOFcuY+LTpJr2B5YIcTEKwstaaVTgCPF+Yi4zDhBTR
         PAig==
X-Gm-Message-State: AOAM530JEPpE+YaptC9TRv+BQT8C/5HTstCxWSUllW3o49hadg+rIn6y
	y4uQhU/pLAANMTAX4r2YDW6kviOhtxb1vfGnrU0=
X-Google-Smtp-Source: ABdhPJxECw0V3jWNF/BA8gW1zDMxBYDkGOJs8qFL3/XqUc3j9ICvKltI4RP4oRxJGpZWee5/Y+mW8GOAZO8FS/BZnHE=
X-Received: by 2002:ac2:5310:: with SMTP id c16mr7346042lfh.393.1631875996413;
 Fri, 17 Sep 2021 03:53:16 -0700 (PDT)
MIME-Version: 1.0
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
 <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
In-Reply-To: <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com>
From: Roman Skakun <rm.skakun@gmail.com>
Date: Fri, 17 Sep 2021 13:53:05 +0300
Message-ID: <CADu_u-NvDrKkYJXoPSH8BL2GzKmCUNtsSz=6mn1yA+pPcUMpLA@mail.gmail.com>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrii Anisov <andrii_anisov@epam.com>, Roman Skakun <roman_skakun@epam.com>, 
	Jonathan Corbet <corbet@lwn.net>, Thomas Bogendoerfer <tsbogend@alpha.franken.de>, 
	Michael Ellerman <mpe@ellerman.id.au>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, 
	Paul Mackerras <paulus@samba.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Christoph Hellwig <hch@lst.de>, 
	Marek Szyprowski <m.szyprowski@samsung.com>, Robin Murphy <robin.murphy@arm.com>, 
	"Paul E. McKenney" <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>, 
	Andrew Morton <akpm@linux-foundation.org>, Thomas Gleixner <tglx@linutronix.de>, 
	"Maciej W. Rozycki" <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>, 
	Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra <peterz@infradead.org>, 
	Lu Baolu <baolu.lu@linux.intel.com>, Mike Rapoport <rppt@kernel.org>, 
	Will Deacon <will@kernel.org>, xen-devel@lists.xenproject.org, 
	linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, 
	linux-doc@vger.kernel.org, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

>>>  In order to be sure to catch all uses like this one (including ones
>>>  which make it upstream in parallel to yours), I think you will want
>>>  to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.
>>
>> I don't understand your point. Can you clarify this?
>
> There's a concrete present example: I have a patch pending adding
> another use of IO_TLB_SEGSIZE. This use would need to be replaced
> like you do here in several places. The need for the additional
> replacement would be quite obvious (from a build failure) if you
> renamed the manifest constant. Without renaming, it'll take
> someone running into an issue on a live system, which I consider
> far worse. This is because a simple re-basing of one of the
> patches on top of the other will not point out the need for the
> extra replacement, nor would a test build (with both patches in
> place).

It's reasonable.
I will change the original IO_TLB_SEGSIZE to IO_TLB_DEFAULT_SEGSIZE in the
next patch series.

Thanks.

=D1=81=D1=80, 15 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 16:50, Jan B=
eulich <jbeulich@suse.com>:
>
> On 15.09.2021 15:37, Roman Skakun wrote:
> >>> From: Roman Skakun <roman_skakun@epam.com>
> >>>
> >>> It is possible when default IO TLB size is not
> >>> enough to fit a long buffers as described here [1].
> >>>
> >>> This patch makes a way to set this parameter
> >>> using cmdline instead of recompiling a kernel.
> >>>
> >>> [1] https://www.xilinx.com/support/answers/72694.html
> >>
> >>  I'm not convinced the swiotlb use describe there falls under "intende=
d
> >>  use" - mapping a 1280x720 framebuffer in a single chunk?
> >
> > I had the same issue while mapping DMA chuck ~4MB for gem fb when
> > using xen vdispl.
> > I got the next log:
> > [ 142.030421] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
> > 3686400 bytes), total 32768 (slots), used 32 (slots)
> >
> > It happened when I tried to map bounce buffer, which has a large size.
> > The default size if 128(IO_TLB_SEGSIZE) * 2048(IO_TLB_SHIFT) =3D 262144
> > bytes, but we requested 3686400 bytes.
> > When I change IO_TLB_SEGSIZE to 2048. (2048(IO_TLB_SEGSIZE)  *
> > 2048(IO_TLB_SHIFT) =3D 4194304bytes).
> > It makes possible to retrieve a bounce buffer for requested size.
> > After changing this value, the problem is gone.
>
> But the question remains: Why does the framebuffer need to be mapped
> in a single giant chunk?
>
> >>  In order to be sure to catch all uses like this one (including ones
> >>  which make it upstream in parallel to yours), I think you will want
> >>  to rename the original IO_TLB_SEGSIZE to e.g. IO_TLB_DEFAULT_SEGSIZE.
> >
> > I don't understand your point. Can you clarify this?
>
> There's a concrete present example: I have a patch pending adding
> another use of IO_TLB_SEGSIZE. This use would need to be replaced
> like you do here in several places. The need for the additional
> replacement would be quite obvious (from a build failure) if you
> renamed the manifest constant. Without renaming, it'll take
> someone running into an issue on a live system, which I consider
> far worse. This is because a simple re-basing of one of the
> patches on top of the other will not point out the need for the
> extra replacement, nor would a test build (with both patches in
> place).
>
> Jan
>


--=20
Best Regards, Roman.


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 10:59:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 10:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189282.338874 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBa9-0005hd-U1; Fri, 17 Sep 2021 10:59:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189282.338874; Fri, 17 Sep 2021 10:59:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBa9-0005hW-Qu; Fri, 17 Sep 2021 10:59:01 +0000
Received: by outflank-mailman (input) for mailman id 189282;
 Fri, 17 Sep 2021 10:59:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBa8-0005hA-Jn
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 10:59:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4288b3da-17a6-11ec-b681-12813bfff9fa;
 Fri, 17 Sep 2021 10:58:59 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2050.outbound.protection.outlook.com [104.47.13.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-wjH2GhXON1qsyv1BXMb0cA-1; Fri, 17 Sep 2021 12:58:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 10:58:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 10:58:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0026.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19f::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 10:58:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4288b3da-17a6-11ec-b681-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631876338;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=91FfNUcKzgqj/Jsl8Hee2KoGJn6BnknKpEFuuxiLugs=;
	b=VYdITvkpyOqBMmGknv5ljfBmc7ZTAKr3zH863+O8ORoO+zri34CC+rZJPqF7/y2yOFu/H4
	TXo4VPAwxpPtrgkAC4XkImwAOuBMTHA+mehwLaGkO5rUKmL+BwxsI60bfNLtozRNsIPnY5
	EpVu2tuS/32lUuXbzZULFa417UEA5fA=
X-MC-Unique: wjH2GhXON1qsyv1BXMb0cA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kFvLwyJn/QgUJTCF35Ll0Sgx1skte8PUiu+ZAMtVk8XRSaeno8dhW5HEIF7c+2XOhaDpPfWMs7Tdh5Wp3DvkzaP7GG+eTdeFYqXPrNetTbiYujGcX/1v45hAbcgtOgwuiVoZgawNmcxmyCuyRrweK433gcOHC37IEEUwli7xKwQqiegsglo5Wpe7ctmsbQK/1yoeXF0ejQ2p9H/avawei9tHMvobNbUW6/FEqeqt4he+76kYQhAwBBS4xQ9C9qd6h3ew2al+tRJIyscnANXXJgeKYjbKd0spMlQKlWGk8w7sXwxXWx/6htwx1XAlE1VIJZbCXUDeeJLvcoRrBnSaUw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=91FfNUcKzgqj/Jsl8Hee2KoGJn6BnknKpEFuuxiLugs=;
 b=SGCY1art8DgoyF7FNhYRM4rrPGP6Z9Vdk0+o2XX7w/bj9c0Ufk4EfvA5z1ZkMPSCFbMrV9FveP7CQVWyzaSAWv09IouAi8ed0Zw+tlEbEpk3AXvygnf8GguWAMBppfjYsDfyJdxOqv3kLBQOqahihaG6dQsGbPxMY7+9Cu33U400U+fRzRZgdvQl5It+nNRK9tBSsQGEkLpaZBGNhhnSx1qZt6Txxc6f+TqSLnAh5anQeyrejtNIsfaTc92dDfLJYR+Fc0EHXM8byr2Tf6rfuKyps4oZwW90R+5hZfuRACSQZBWKJ1tm4IYz8K0V4dxKztDEuDNu4zsjerQ16z3VGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/3] IOMMU/PCI: respect device specifics
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Date: Fri, 17 Sep 2021 12:58:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1P264CA0026.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19f::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d392e66-c0fb-4eab-25e1-08d979ca2489
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB270345D62A753F9212F9ED17B3DD9@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0qyeurjT4P8/pDYal3z7NoYobwcayWdnl/YbZhzx11TBV5JYaX4n9OOTItsOGrvsGFk1ZP+efHaCnFqxRBGRqebIc2pxJXmQOZ/b48pUyhzNMJSdovEeITe5uc35Y/BxCKq4zZaX66PF5Z4vivQTxe9Wklkk5kZ2dKm9oGoDt64JmX/kTkXXYR8O0be1+dKXA9PETeK2Dm814tgLTeU6dwtkFSziMJ0w27+02GXQy2CYrCvJJ1Ny+8XQWimRxmZLwYOyeCeLatLb+aXnGwTPD3mupzAWqGUxto5loaNCE8tKZpsE8/ADL6DOIg9dwqAFbBjQzadjVTpDnDKeYi9OI1hxGEA6kECzRp3Pz29W++a/A5s5/siOCYZfXpTeW+qpTyBi3TmoWOFUR6dM76HyYefc/AwhBSIyc/C4f0g37fQ38JwhCgT0KLCAqQRMl3uP5C2JYFPlFfGRrA1+ccjehqHEdzUmjq+nb9nql/eVDq30c3fRQ8/XPV6kpLwom5mXxjjg/BkMXbzCLNCqdZWJlbrmfxEmnIC8Z1SuH8qX9PxMAEOkSInU98YuNA2Ly+6d4pxKNnpPEc8UNgEvv3p/qhTVFmq/AXXc8D+UZr7WDOm1ysEpfPH8VSA1YuuSWkO1UtAfxLRikuV+pECqlvg2dvTb3/CGGe22hoKYwljBOLToFnUbwiDTSKXhkp1/Ty4oKydG+zMfkztOLIlDCGU1JrIgn8FQEhwZH+gJ3QQL9gw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(39860400002)(346002)(136003)(2906002)(186003)(316002)(4326008)(16576012)(31686004)(956004)(2616005)(6666004)(26005)(8676002)(8936002)(31696002)(6916009)(83380400001)(6486002)(38100700002)(66476007)(86362001)(66946007)(66556008)(36756003)(5660300002)(54906003)(4744005)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZVZHRURua2oxbmt3WGozSUY4Q085a1E2UzlRR3FwdlNQRUl2VjM0TkNacVRK?=
 =?utf-8?B?TDJnTzRmTXdWTFRFTlNVNVMxZlgxMmtuOGlJeDg0S1laTEsrUVZ4NkVqdUhs?=
 =?utf-8?B?VHp1Vy9sSGgrQWFGOWMraTdIcGpKVjh1ekEvSmJMN3BMUEdjV2FQampIM0Q1?=
 =?utf-8?B?ZTEwSitMZE5yUVVxVFh0d1FWaHdYRHF2eklUVDZXdDFiWUE0NldnbWIveGVR?=
 =?utf-8?B?YXA5VDJJTy9OSURkTGhVZ1NqUVV6R1NCN1g5Q3ptR0hjbkhtTjNVdnNsazNY?=
 =?utf-8?B?NmdFT1JRZktCT2hKYjd5eXVzSlhiVUJLOWl5cklhMXNGUzRhWWVMZndqYitB?=
 =?utf-8?B?bFlSZ245bjNzVFYydk1sM0JVZWM4VEZ2YTJZd2hTN3FDN2JDZG8xNVJyRG1U?=
 =?utf-8?B?Q0RjTTYycy8rU0ZrV2RWV0pXUEltOGlqT2tzdUpjMklLa2xHYTRMVFlVa3Bp?=
 =?utf-8?B?NTBCT05rZk8wa3RsekNXa3pScXlRVEloWk14a051TmpTRjAxVVdvNVI3dDk0?=
 =?utf-8?B?NFFrVVZKMm9aZUFEaC9LdXd6ak1XeG02a2hUbkpCRlc1WnZ2N3JwMTNJZlNU?=
 =?utf-8?B?VkRQSCtmZWxJWitOckxkSWFEZkVDSW5aMWVLREppYys1c2JXbmYxTWdFYzFH?=
 =?utf-8?B?SXhxTTY4MU1PMHRFMWxHSHk3K2RxYUJPZmZEenZqYzRSN25KQXZBNzBiVHFq?=
 =?utf-8?B?MTl0WUdrTS9TaG5pbWo5a2tQWjNpNFVsaEQxd3BZZWdFOXViUytvaUdSQWNS?=
 =?utf-8?B?VUdkanVRUVpsTjR1eVRjSjBid1M2RGk2bVV5S3cwRCtOTGJpcEVuODg3UVB4?=
 =?utf-8?B?ZGtFWUc1ckpLaGNOR2VIMXpma3JmWUFFaGFNa3VqSE9STDRscVkzTVFCM20v?=
 =?utf-8?B?emcrU0c3eEVGSTlDUG1PMHcyb0J2WTlSdnEzeG9QdFo1VVpreFYvOE1XdmhW?=
 =?utf-8?B?NUZIcjR0Z2xtdUIyZUVCRCswQzA4TTlNdVN4SFNQY3RTYXFZL3M5MlRuUlhO?=
 =?utf-8?B?Zk5qbkV1RXlUcEYyd3NEemZkTmY2ZjdvU00xeFg3MUl1TGR4akVJVkt0MWg2?=
 =?utf-8?B?MFA4c3FOMXZjU3d6SmRWRFBkdEZkZDRacm9RZUZFRE1MTU1OVUtyL0l2aHkz?=
 =?utf-8?B?dWJNQ1hRY1lNclN2dlVqTGN3NWRqb3VBTUZxbVNiQnh1S2dvMnNCaytrenlF?=
 =?utf-8?B?NmhwSzdva0lpOEplemtmL28vM1VaU2NUb1VXYnZrN3lDY2RMeFllMi9iczFQ?=
 =?utf-8?B?MmszK2RRVHdtNFE1VUJUZWF6WlBBWGEreWJpQkMyWkREQlJCNEVhWEtEVm9I?=
 =?utf-8?B?T1RiWGNNWnhRa3lFR283MVY1WWZzNmhJd1RoK1hXZjI0bkxjYUtXNS9CRjNR?=
 =?utf-8?B?VFBQNHpZNjY5V0U1MS9QcVdCQTczY1RKTXJQdERKQjZiWi9RbEZHdXpXSHQ3?=
 =?utf-8?B?ZlFFTkgyZVhnRFA3RXlHN1NROXZmUEo2VjJHOFFoMFViZmRJM0QwdmpGVVZW?=
 =?utf-8?B?YjlIbHBrV1NtSXYwMy9DR09haHhQSEtsa3I3OHgwUGlRYkVuS3BXY0xiL3Va?=
 =?utf-8?B?S3VKdFRDZ20zUlk4dmF0MVc2TlNjaHJ4bE5WQU4rM3lMUmpDSFR0OXV2M0R4?=
 =?utf-8?B?RTBMeGNpWENEakttYTJLZ2FCaE51eTlVdVJHNlNrUWErVWNaMks1SS9iczBO?=
 =?utf-8?B?YWZtVTRPRnVQZEg1VHBnMTc4Q0RsQWJERFNuTXp4dERkTEpkRWNjaWYxeEhN?=
 =?utf-8?Q?8GMY+tBj3tZS1N/XUXSuJAi/FA0hKvF0zWSpu4J?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d392e66-c0fb-4eab-25e1-08d979ca2489
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 10:58:55.3797
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wP0IOP4RwPgVHDnEPOn5VZncpHQkV+EjM0DSIz1kOO19anIDAHsQ5uEbl9/4q/erJAKCFfIFUuQnnCL2xxePuA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

It has occurred to me that the recent vPCI-related discussion about
hidden devices has some relevance also elsewhere. In the course of
addressing review comments of what is now patch 1 I then came to
notice yet another quirk.

1: VT-d: consider hidden devices when unmapping
2: VT-d: PCI segments are up to 16 bits wide
3: AMD/IOMMU: consider hidden devices when flushing device I/O TLBs

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 11:00:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 11:00:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189290.338886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBbE-00071i-Al; Fri, 17 Sep 2021 11:00:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189290.338886; Fri, 17 Sep 2021 11:00:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBbE-00071X-4n; Fri, 17 Sep 2021 11:00:08 +0000
Received: by outflank-mailman (input) for mailman id 189290;
 Fri, 17 Sep 2021 11:00:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBbD-0006xR-E3
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 11:00:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6a450cfc-17a6-11ec-b682-12813bfff9fa;
 Fri, 17 Sep 2021 11:00:06 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2050.outbound.protection.outlook.com [104.47.13.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-tYwZSGEaO0W7H4KmVvNh_w-1; Fri, 17 Sep 2021 13:00:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 11:00:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 11:00:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0011.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 11:00:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a450cfc-17a6-11ec-b682-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631876405;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UppQocqB70WkQjVr0azZfPR+2KVYqhGJJEBclpY3yOc=;
	b=mBjrbQ4qRu+WvdMsZtVBDbT8YdJPrNCffticJRp3KvriMxiVuMtzgLoYzPV07AxSt+5RJY
	gI6MNEXVldzHvS5U7NiXRF1zTKhJAvxBP0m9hjwve1xsxK/802gAtdDCsNMwLWZTWqs90K
	AeVKoECF3JKqdmvyS1H8qw8hB/4v7FE=
X-MC-Unique: tYwZSGEaO0W7H4KmVvNh_w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=esBbik5m8aj4/bF7Qq79cwRcycdRp8/IY9JuTaL+h2HZk3EcfOnDm/L3lM+htqnPrx+fBLBEjGj6eY/xNiIz1+Gby59NZWnW4XCfjS/2Fx79FSjnksru0xGcqUrHTby6IgiMiYN3NOu3ou/dPkuBoDePTqQFdKPI8SqazBj82M55UgJ3oUcjqTZJnfK5Gn6RnMO4eK3yCkQ1Cm3UEZtRaanR5lZcDUoch2bMdzyA9Qr6VMBwAN10Cgezs70+pe4UgzsXsuAoOTPL3S+DYay7dU1YIjdhJq1ud2dEliqUaDFYsh6m9cX74/z7TAojaPgLxiCR4cckkQIMGPOS8+HvVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UppQocqB70WkQjVr0azZfPR+2KVYqhGJJEBclpY3yOc=;
 b=hH/snVUFNV+ZU+56I7eTrijwfQ7Sp0wOsdkrBWl8OUeFyDsU9gX80Sp2Lcvc/UpKiql2m6E0Iiy5+XW0W390CNC8tbSr2uedAMPmeSKrZZi7VymbG/gV3SJk542WAt0ggtwwbDum/lBw/STqGJwUmQdzpZnqAasdbQmGZ/+GjgYpO/bJyAy44yU65dUlJgmZN/+BVrYzGoIYAbxx8aWQerrHbh/UQwO/zSlJV3d24AQCaqneNcZWPZYav78c+XUFGUgssEovnywk5iGd0Gu5/4gBgi2ZMY9uEnMrZz0Ya3b6dmykyJx9WGanzTuglY6XtJH2qBiL7aF5RXsVqehANQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 1/3] VT-d: consider hidden devices when unmapping
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>
References: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Message-ID: <64c6b3e8-045b-9308-b77f-721c3d57c08b@suse.com>
Date: Fri, 17 Sep 2021 13:00:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P191CA0011.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7ec7cb6e-93ee-4b4d-9d0c-08d979ca4ce1
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2703794C093615299B78C6F3B3DD9@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dKyHHbmrcPEuo/NCRuctTFLIFhU/QU4k74zor3ELysknluJmr16aa/lw4vh0/8jbq8dd7d9Zoc2qAWhWMxbtwMePaIxcm1TXgNFOgi5VJAS3HE13iDNgfrFJVlZ+xW9iBmKlXe6XUZIiIZvmfuTVbf7CcXhFXWxAyp4eHbeZdtgwJDF8nbGx4gSyXRQcVIeD5/Z4rVacsxqODbLjxWHdsTd+20gej8vQ0hIXEfjOlvYvUO7bH8HzdsONGVAazOTWZa6Ag5ADBePge60rL8yZGG2lZPyUYhUvwdBKevgrdn9ak6J9MDIC/k73pwq9wRi6QYL+SSAdCWvYsVDRFiLmTZ0/6LUiYY6wUWsfQQ+3u8zruE+ArUAPOk5FC5qD3lSzSIpJ+0vTk1V2fCOrkiwq71xQrt+e5RrENvr9kA2GmtEn0d0l5UeTltxd4XCcuceYR9fbNKor53iau2uGboVvek3LBjR7GugkPllLCrdhRRfS9CLelMBU3f8zOOBXnhjT8rUvYQp1av7bRsh9uUQUVq3cJgzgRI0fRjlGw7rSxk5iTHMIXEw5X1s2mqAF4ennsacwh/sk1+cisyp4kAGMeINfYYwOB5agPx6zk6Qia1yA8FjDgCgyOkoepFg2ir1s4ER1808jtBFG7eD+/iJAVm6BeuhR4NyvqLgLk+8+o/LwOEX16QpLfQUNfO3JodzFHd3eLQ642o2eRcG9dl39aoDV30bGs6IRrphp7P+6upc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(39860400002)(346002)(136003)(2906002)(186003)(316002)(4326008)(16576012)(31686004)(956004)(2616005)(26005)(8676002)(8936002)(31696002)(6916009)(6486002)(38100700002)(66476007)(86362001)(66946007)(66556008)(36756003)(5660300002)(54906003)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cW8rWGM3NWJpWE1LOEZXQ0k4MGJkWmY0eENGcm1EUG1YNEhEclBRT1pjL0VL?=
 =?utf-8?B?cmNiQzI3SlRnMnI5WWJiMmtqSlJoem1wUUpZTFZaUHVqMkZFY01UZm92N0Yr?=
 =?utf-8?B?d0JyYlNKNG9NV0N1QkJyR0RyTVB5VlArRWRWYk53ZDFSTm9PWFZWT0pqMWNa?=
 =?utf-8?B?Ykk5QWtjK1N4TWltMUo5SExFYmxmT0p0MFJFN2ptK0ViOFFiWG0xRmtpTlVv?=
 =?utf-8?B?MmlHSndrWHFNUk9OdS8rTUtqOFFGN2owb0lPV3ZUeUYzL2pFdTBNVkQ1UDNs?=
 =?utf-8?B?NHJtREFRdFVMZC9LbkoxcTh1Y3VaNEEyYlk2d3IxMy9oaUdiUXVYeTJMTTlp?=
 =?utf-8?B?eUFoNldpcUxyNk1pbWhHZVlSTktZOWpRVFl0YkhLWmpkUjA3VkFQSDE5R0U4?=
 =?utf-8?B?Mlp3SW53NDNZb1pzOHJWMytNcFdGVnBkTnBCM0dhS2VoQXBRUkxYZEU4U3c2?=
 =?utf-8?B?ZHd6ZzhsUG9rT3l4TWN1Q1hGNlBUZ1dCMDJzbEhLZVBTeUsvck0xalA0cGNX?=
 =?utf-8?B?b3ZBMW9aM0VtemhzWWl6SWpLWFVCbStjWkQ1L25FNDc1b3hZOVNYWkd1ZitZ?=
 =?utf-8?B?T3RBRXBWY2VHT0dkUm9zTnoyOVE4ZGVldEk1SFZ5QWlnL1hnblNJVmVodWZR?=
 =?utf-8?B?WlpqOEhkUi9VZ3BtRWU1VjZGZllBMHNjT3pmeFIzcC82VUxEOWUxaDhIN2Fr?=
 =?utf-8?B?Q0NIZGhIdElZTy81eUgzZ2d3YTh4OHMvK0psR2JrcFFUcjhpSW53V21ldGlq?=
 =?utf-8?B?UXNQaVM0YkZHK2lrUm5TcTN1dS9xeU01ZFI3UTRMMENkdEx6V01KR3I2eEdt?=
 =?utf-8?B?azNCUXI0OVRXdEtrQlIxcmNQN2l2V0F5dGVHYXZjOE9wK0lVeHZBUDA3SUZV?=
 =?utf-8?B?WXJxOW1ubndaRW8wemF2YVE2TmVpTUIrMTZka280R0p4T2I1OHp2UnRPT0w3?=
 =?utf-8?B?MWFuVXc2dFpnK3JhaWR0ZWlSQ0Q1R3J0TFZiYWpyVFRRMWdrM3hzS1hBSWNS?=
 =?utf-8?B?K3ErZ0VkNzJJRzE2OXB6QkMxK1VObjFpTEE2YkM0WTZZc0V5ZGFyOUFQM1lV?=
 =?utf-8?B?eGJpbldxcHFNQ0M2QXh4K3R4ODNLcUdidEZlNXNWaXhYdDJiSTF1VlZrZW56?=
 =?utf-8?B?SGZQUDg5ckxzVVlPeG53aDJQbkJvS3VKL1NleXhWT2lpL3ZLYlFnWGs5Nk03?=
 =?utf-8?B?dDV5RDVKb2YrRjllaFVhVnEzSDNwai9kZ1dwdXRiclk2WGFEdGhNcXZVb2hw?=
 =?utf-8?B?d1h5ZFlyWGV4UTJVaGw5N0dNT2hWc0J2VXVQZys2QUNMMktFczRrWE5rQStO?=
 =?utf-8?B?ckx1VGJrWUliakRXSk42c2tBWXhLM3cySG1yMlhhV1ZhUGlHcW1jUmRBMDZX?=
 =?utf-8?B?SGVVd0Fwc29BeTRvQnYxSW45VUNVRS9UcUVabXBxVEpBcTdMMU5NR1Z2VCsr?=
 =?utf-8?B?amxyd1BpZ1hCd005bnd3WDJMNWM3VTRIZXBZTnZxUm5EZ1FjNEcvekNoWUM5?=
 =?utf-8?B?SkNYOEROdUZmUGs1TGc3UmtMd1hBeFJzbnUvR0JxWTNER0JmdUVYaDRraUZQ?=
 =?utf-8?B?WmpHTEduL2k4OEFkWXJ5aGJoZDZnM2N0aklnSkZtTFlJL3dFOGNqck1nYlE1?=
 =?utf-8?B?Zlk5Qk80K0VvQWZMRmxZaXdlNVYyUkZlSk83QzA0RWo0SnVFYXVYU2prcnBo?=
 =?utf-8?B?ek9KWkxBNUlJTnEwbkxOMDBiT3pFcSt1YVZoQjUxR0JleFJ2WkIvMUUvNkk5?=
 =?utf-8?Q?RNrl5n4n4mPsscVjP66XAMgYrc98U5xwjcaUxwv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ec7cb6e-93ee-4b4d-9d0c-08d979ca4ce1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 11:00:03.0704
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BL0M5Gv8SywsN8nJ3IGt1Hm5zkrlziDRlouI/vHLYjQvGuy+4eIfYW/yDAs4lW5delnivzbhZNVIQvGfuuax/g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

Whether to clear an IOMMU's bit in the domain's bitmap should depend on
all devices the domain can control. For the hardware domain this
includes hidden devices, which are associated with DomXEN.

While touching related logic
- convert the "current device" exclusion check to a simple pointer
  comparison,
- convert "found" to "bool",
- adjust style and correct a typo in an existing comment.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Introduce helper function.

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1650,6 +1650,27 @@ int domain_context_unmap_one(
     return rc;
 }
 
+static bool any_pdev_behind_iommu(const struct domain *d,
+                                  const struct pci_dev *exclude,
+                                  const struct vtd_iommu *iommu)
+{
+    const struct pci_dev *pdev;
+
+    for_each_pdev ( d, pdev )
+    {
+        const struct acpi_drhd_unit *drhd;
+
+        if ( pdev == exclude )
+            continue;
+
+        drhd = acpi_find_matched_drhd_unit(pdev);
+        if ( drhd && drhd->iommu == iommu )
+            return true;
+    }
+
+    return false;
+}
+
 static int domain_context_unmap(struct domain *domain, u8 devfn,
                                 struct pci_dev *pdev)
 {
@@ -1657,7 +1678,7 @@ static int domain_context_unmap(struct d
     struct vtd_iommu *iommu = drhd ? drhd->iommu : NULL;
     int ret;
     u8 seg = pdev->seg, bus = pdev->bus, tmp_bus, tmp_devfn, secbus;
-    int found = 0;
+    bool found;
 
     switch ( pdev->type )
     {
@@ -1737,23 +1758,18 @@ static int domain_context_unmap(struct d
         return ret;
 
     /*
-     * if no other devices under the same iommu owned by this domain,
-     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmp
+     * If no other devices under the same iommu owned by this domain,
+     * clear iommu in iommu_bitmap and clear domain_id in domid_bitmap.
      */
-    for_each_pdev ( domain, pdev )
-    {
-        if ( pdev->seg == seg && pdev->bus == bus && pdev->devfn == devfn )
-            continue;
-
-        drhd = acpi_find_matched_drhd_unit(pdev);
-        if ( drhd && drhd->iommu == iommu )
-        {
-            found = 1;
-            break;
-        }
-    }
+    found = any_pdev_behind_iommu(domain, pdev, iommu);
+    /*
+     * Hidden devices are associated with DomXEN but usable by the hardware
+     * domain. Hence they need considering here as well.
+     */
+    if ( !found && is_hardware_domain(domain) )
+        found = any_pdev_behind_iommu(dom_xen, pdev, iommu);
 
-    if ( found == 0 )
+    if ( !found )
     {
         clear_bit(iommu->index, &dom_iommu(domain)->arch.vtd.iommu_bitmap);
         cleanup_domid_map(domain, iommu);



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 11:00:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 11:00:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189294.338897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBbf-0007aw-Le; Fri, 17 Sep 2021 11:00:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189294.338897; Fri, 17 Sep 2021 11:00:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBbf-0007ap-Id; Fri, 17 Sep 2021 11:00:35 +0000
Received: by outflank-mailman (input) for mailman id 189294;
 Fri, 17 Sep 2021 11:00:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBbe-0007aX-K6
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 11:00:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dba2181f-69e1-4b5f-b8a3-f1100c411361;
 Fri, 17 Sep 2021 11:00:33 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-b6cIs-thN3izIHwwUkJOkw-1; Fri, 17 Sep 2021 13:00:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7040.eurprd04.prod.outlook.com (2603:10a6:800:121::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Fri, 17 Sep
 2021 11:00:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 11:00:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0001.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 11:00:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dba2181f-69e1-4b5f-b8a3-f1100c411361
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631876432;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=BuyVlwQZ1n9cLvqfMjg3BbUkdLGOi0HDaN5Q34aqRMQ=;
	b=K57bYeG3KBdY8QhHH9Vp3dz2R9Aq/MRvrpSqZClyLMNJVSU7t6D/Ts4kZXWEskkKwA95od
	F9uaNjg4QHR+U86DDVXeec3lRHubvj+rLwgPD27p0gv5a94c3lMdK1Jv40nmhtg/zHo26h
	xFaWnHLgZXAcRzpUBoCuSZ23bl/MY70=
X-MC-Unique: b6cIs-thN3izIHwwUkJOkw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BYJZ7k/9X7JNJ6/GnKJzg9BIvtA3j4USwP1Ex1TNefCsbHoP0NPKNR3ExXpLS1JVlbtxyISc0EpQ0PMctg1YiZLO+2SBpjTzf72u72Uzfcklu0hm6OZhtS3lMhb57gpkfRlMRoToofI8BClumvvSq4i/1DCUXoJ18BdZQ+5JOy3xDXEwIBv8dAZnrK7YV9Y4T7thzoUUVR0zzWQ7iZrM69Hmu7n6uQh6URtFxc4b00AqzuZ4YzwO8FSF0dF+IFftX5tT7we84/bLHwdJQozds2E8EwBZzP1OwtwNbe28FI8xH+BCO2TsiJ6mM9lSDdrY6jmZ/qw8/Qm7gAUFqwMm/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=BuyVlwQZ1n9cLvqfMjg3BbUkdLGOi0HDaN5Q34aqRMQ=;
 b=i5Me0uXErtVNAnAKKchHQHNN/ffI2xdc0wS+m3YraATuQCCiSpiefsr8/QwRDzkc/HEvMfghYBBSpXf6yrvlz28BJnLgfVO5oYAiNNL7UB8JAmU/iaqu/pRqAb38z8/6LnOm1jyrU66mHO7RS0O2xvCxntl/n1qxKAXHRXDb2m4K9IUABVMnE039y4Nh7ElcTCA+zRHK0ey5lD9Yz6kr0QVwyGc4Oz/irjGIZMI+iFQFASzZNRGxD6UwWIIjdKjDkb6LXWDnreMMDh8f+uWZWXoIh/y6dJLWgP/yNzBu5VUQQzkkgzsyjnkYKhENJQPjpxM1X/VK8BF1juXStIiPFA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 2/3] VT-d: PCI segments are up to 16 bits wide
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Kevin Tian <kevin.tian@intel.com>
References: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Message-ID: <ee538608-4be8-b277-099a-fa82e0b59a9c@suse.com>
Date: Fri, 17 Sep 2021 13:00:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P191CA0001.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c89d7dad-b80d-4b95-4a6c-08d979ca5cf0
X-MS-TrafficTypeDiagnostic: VI1PR04MB7040:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70408C7624923F179BFEB71EB3DD9@VI1PR04MB7040.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1850;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xtbHt0eMcwhs/HIEjft8nVeEPQDIuasyXFXMg7NDZAc9rxQ3c7W8YhK6qB3Gb2eUmxltkI4I8WI14MHl/MT4XMFb7fC11iluNXL3r0ivMP/XWs6XcMIMGMLC4UKOr5ctL7SoVrKTCDiUCnFU9x1dyvkVS8mF7+VGZcsvJFumj1CEnVoZW9p9PFdGARq15m7TVyMvyX9K2Lgl/muhTtPD4/jsAnAc9ihFmCaYcgGCHk0lt+qTo9kXSZ1XrW+iBKCErlosef51SllNBtr3ga20xzwf1WQsQglpqjlQzyAob2LO4HmuaJVtZxIyQJ/DX9eNh+jzESSAxvN5SAN0TwJWkxe1kxZLhgcpZ9Gte1Tlvj8juL4sLjfFW+stY1mpwI9RWbmXFKnU4a5Afmo2ahTkKfV+xONhjN05tW5ZAFa8RPppP7pp//0taF5kU/EB84mXia0fmPusFTfQh2/vjZY/Jd6qoGAMUV3c0XCC+5semxblCjZr6Ai7SgHapLaxmOJY8Z16HPS4qhxGSAq2cpAL7OnbmCa2o0r0zH/FQ4HuGCimlMrCLw9W2G36xg1udnlUltZWlUEDdezsnz4KGRlYYErxv4gGERGtHeAfwQVxp0jlR28a1D2zY7dxo1xv9GewQD4YQDxZJPYBTKVjPT7Df6h6RgtzJEVbiTiWNr1BZCzfVphCUxAEUrlJeRb8Gm5SZdFhmZCBoYiC9vNSrl/Cos/SOukEhGPd5rSE/5PcTIU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(186003)(54906003)(4326008)(31696002)(5660300002)(956004)(6916009)(36756003)(31686004)(66946007)(2616005)(8676002)(508600001)(8936002)(66476007)(38100700002)(2906002)(316002)(6486002)(26005)(86362001)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MGtJTTZxQytYOHBwdlJuTGVhUlBYRnVxNlAzaEd5b2ZiM2RwTjNDcTdaYjJa?=
 =?utf-8?B?NEdXTjl0dlVIdTZJZHFqRVRuQURrcXZKd0Zua2dMM1NvVGdSVW5qRzQ4RXZm?=
 =?utf-8?B?T0tlWE4vR2FCRURqOXgwbTVsNDJlL2tzRGx4dHU2dHZhR3BPUzlFOWhUdmdT?=
 =?utf-8?B?UDVENWxzMEZ2WEoyMmlRcjNLNTluT2YxY2hyUWI4YVd2SzRGQ0dTOUZIYVZv?=
 =?utf-8?B?NmErcDRyVkRGQlpZdkdLRFNiSkhHTlptZDFUdUcwSmxIRnFMS21HcTFxaGpX?=
 =?utf-8?B?bmRMRWJJcEUxNXZMQllQdHNPNzQ4UjBnWjNNaG1zY2kyUjNONE1xSkMzNW5p?=
 =?utf-8?B?aXprQ1pSMTJmd2xjYkdnanBrR1dLMnVuSE15VisydlVpNjZVZkdYa1NsdEVW?=
 =?utf-8?B?cGh4NTBrZEJtbGdqeGhZM3QwTHNuTUJ3NEI3UWw5ZkJPQmJYa1grUXNMMWlw?=
 =?utf-8?B?WVhFRU85bkc2MjNNUUt0V2cxMys2YXdGNllHT3p6ckQ5VEwvR202MGR4Q1Rp?=
 =?utf-8?B?TTJ1NVNtSlBTYml6ZUxBZ2puUjZjT2NUdVMyKzlUbU85YVpGNGRDVmE4Zmg0?=
 =?utf-8?B?RXF1UHNLbDI2TVlUeU1oL0pjcG5zYzFIM2lLSjR3b1cvV1E1cThBZ1RsWHlw?=
 =?utf-8?B?M0dGMVNCdEJxL3AvamFxOFN0VERGV1FSQmRrVzkvUmVwWWVWdmNaVkN1N0Jq?=
 =?utf-8?B?dGgvVHJVK0dTTXZKd3hzNzJjcnpFY1NQekhPdXFMOWcxdDF5M0UzbXRnWlJD?=
 =?utf-8?B?LytWQXdPNTlLeTh3OWovUE56czF1SUNiV2hqTXpBRWttQTFqOUhCNFlPcUZC?=
 =?utf-8?B?MVBQNGtTRGxnaEFXbzVLeHpzdXhDVUhRbW52K014dUExcWlmbmd3NlkxOC9K?=
 =?utf-8?B?emJkdnlUL1VKSks0TlhkSkZXS1lnVFZVOUltZDhrNHphWGhwL1djQVQ0eEUy?=
 =?utf-8?B?eThwbWM4ajRnckMrSjd2cTRTTU1UVStOL25VeTZ3NFpPc3pNWUg2Uzc4bjhN?=
 =?utf-8?B?Q1Y5WUVLeUJvMFNRY0dqbUIyTjhvK3hvcGRubXc0Nm5nM3dpbTRrWk5LV1lD?=
 =?utf-8?B?UnM3TGdmNjkwaVZsdnluNWpFVU5wbm9FRCsrTkhTczN2aFhmV0N4VG0yMmdi?=
 =?utf-8?B?NTB3VmtwVFlQK0MrQjBoSXMrVUtIRFFmZjVBZExKcXBXakF1RzVnL25jYVoy?=
 =?utf-8?B?RTh0aTZoWllDSHlBTmxKMDN2YzFhd2ZvdlU4NHNnaFpkelBQWDRBclRkSE9T?=
 =?utf-8?B?RmhjK3dFNUNsU0lSVzhIa0tMd2FsbzhnWmVpRUZwZ1liT25ETXhYaWVIaUwx?=
 =?utf-8?B?anc4aEhhcll3OUxIWDVqS0txSzdCSkZpRUdzU2xpMzF0RGJGL29DU0haMU9T?=
 =?utf-8?B?SmU0NGx4Qk1aU2JHR0hSVEJGSG5aRzhWNm5XcEovOE5WQzFlN2ZUeGxZRjEy?=
 =?utf-8?B?YThSbmVqbWtENTVGVFFXd2VrZ25oMU1xV2dpRHNFMFZ1bkV0Q2dwdHR5Sm1m?=
 =?utf-8?B?cWJXVFMyY3psSjVzYU9zclpEL1RDSFhldDRmeENvaVhrejNFVmF2UkNFd29Z?=
 =?utf-8?B?aU9qY2dWKzM4dElEL3VGaDNUZGpqRTg1MGUwNlhSaDVNYkt3MWtUQnBYZWRu?=
 =?utf-8?B?a0d5V21Rd2RKcndna0VCSHR0ajM3N21GVm9iakVqd01Za3hTK01qUHdkMWJ1?=
 =?utf-8?B?OEZqUDYrdTgzZ2dCZEoxRSs5OHJQNXpwSThOSTVJaW84bmJ4ZHRDangvNnA2?=
 =?utf-8?Q?8OoWNk7nYOijk6dPXqmX5ASyyUsur1oLKt5lddF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c89d7dad-b80d-4b95-4a6c-08d979ca5cf0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 11:00:30.0851
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hj01Lu5gQS6Ot+0CwKoGTFOL5UQK4MhZCIgrDQ9K82wYZxGX014HfOp8vcWCA2DUk8H1WEatH5pbNQ2uY1sJnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7040

We shouldn't silently truncate respective values.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Strictly speaking we shouldn't use uint<N>_t here at all. I wasn't sure
though whether going straight to unsigned int wouldn't be viewed as
introducing inconsistencies.
---
v2: New.

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1462,7 +1462,8 @@ static int domain_context_mapping(struct
 {
     const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
     int ret = 0;
-    u8 seg = pdev->seg, bus = pdev->bus, secbus;
+    uint16_t seg = pdev->seg;
+    uint8_t bus = pdev->bus, secbus;
 
     /*
      * Generally we assume only devices from one node to get assigned to a
@@ -1677,7 +1678,8 @@ static int domain_context_unmap(struct d
     const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
     struct vtd_iommu *iommu = drhd ? drhd->iommu : NULL;
     int ret;
-    u8 seg = pdev->seg, bus = pdev->bus, tmp_bus, tmp_devfn, secbus;
+    uint16_t seg = pdev->seg;
+    uint8_t bus = pdev->bus, tmp_bus, tmp_devfn, secbus;
     bool found;
 
     switch ( pdev->type )



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 11:01:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 11:01:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189299.338911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBc7-0008CT-1v; Fri, 17 Sep 2021 11:01:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189299.338911; Fri, 17 Sep 2021 11:01:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBc6-0008CM-VA; Fri, 17 Sep 2021 11:01:02 +0000
Received: by outflank-mailman (input) for mailman id 189299;
 Fri, 17 Sep 2021 11:01:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRBc5-0008Bk-BC
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 11:01:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 15ba86b3-c4a3-4e7b-93e9-93551948eb10;
 Fri, 17 Sep 2021 11:01:00 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-85KPOHlRNvOtDdsEpc1B1A-1; Fri, 17 Sep 2021 13:00:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 11:00:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 11:00:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0P190CA0018.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 11:00:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15ba86b3-c4a3-4e7b-93e9-93551948eb10
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631876459;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TPv3uK8bqSIgwLCsg7oLnpJv0fedDK5iubeaeJBmDWs=;
	b=JAXznEpDxU/T0/JiYq5pz2hICcn/M/uavPpBjrJjiNpCrrotgPxWUT0MDu+jvW36DFvDk4
	3O5QAzoUucEdh8mPDrh08erO1HZBhBESQLyXpZZkPGb432AgvbQbnrGgfTehN9sT78So68
	ovL3ihjI/Un0/Q5Wt7KlJ5W8Vj/+bKI=
X-MC-Unique: 85KPOHlRNvOtDdsEpc1B1A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fUvvTFdXKYnQSW4Na6p4WYAg6rkEuje5wR8MNiExTSXDxdWPr8r6W/d+gk8D+wk8m34o2hS1m7aMcUrGlGLMdwhBnDqM2VpYDIIfSNMFJh/a8hrizPWQlNlykTMd6h7NGrry6hntOst1rYsPmrq0MZDXh/Ai9aW3XpD5QBP/j1V4uvHVdG11Dco9NGhLuL11xBlT1yO25/UHpWslUlpqb4dAH3lWO7P8EnRvjZyf+UKibpQR2cDU62avo5bczCg4JkjZDjJ89tcZ3azPTgFVLpVreQivUvGEYw7G69j6pu9CXhSj+FNdMf+vHkXvyTu3mfdOCUFvcmAl+m8pPa8O7w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TPv3uK8bqSIgwLCsg7oLnpJv0fedDK5iubeaeJBmDWs=;
 b=JSbiT1d6FmCTnu1o+VNb0Zlx8dUtna4t4MbB1lCzCHBlwI6FQUAmIFPK85EIv8XL/onRm14aqpM/pd/fZifjRP82zIxs+pAfOOY/7zNXaio2+3hxMpJgQ5o7DLTaMhiBBtLZEbM+FPRmGlG2BE5h/0WTlimZp/scntxyab1RlLEboo/9JD+KXHk598ex7MT3WAEcHfWaWakgqVMapj5A+19J8Kcve5DhM7O2Lp0fDuqBWHLJ6trj023GHfgjNyQD1qfyTBaETyrpDz39wBhQTV9RvjPFBornYgEYWkjgeERdDXyfQVTl4+VPB/lDfVSmMs4KwVWTdHK3MyLZnj04YQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 3/3] AMD/IOMMU: consider hidden devices when flushing
 device I/O TLBs
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Message-ID: <09b6d94d-0116-c8ee-5daf-ddcc19d37029@suse.com>
Date: Fri, 17 Sep 2021 13:00:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0P190CA0018.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:208:190::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 36cf4f44-f01b-4a7c-8621-08d979ca6d4b
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB270328D1C14B1EE4A5272B83B3DD9@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KGTGPCCMm0q1JRK10lrwyZx1NeJytN8Qpzh4Ejmx8egLCQ85cv1fou4gEnKnRodjrOHhMsh7fVt2cuIW1zd6/7YSD04PUS5363B8Ixlq7CF9UIOVPt/y4xJwpJGFSdSmMbgwqHnV1ewEOgcYfhMPqYGKsr+kFsQL8OORDngmp1DrcbT1cWdpZEkvBl94hX77tbatzvyVrbg90GzT85eF62RaPTkOuJWWkM7LdLssAOiVJh7TphoFajCYW13F7s/RjzzEaxFZOdJzeIB9/H9fn32G8s2aANb3sRVdQjwulw92h/uYitDEf/8yanFzhU5l9DSBmX1aVHoLE6Jz/uH1/LVi+qs+nGfA3O2Q66sAEwCSZ+a5r/zkP71tMpPeCaX8061ov0eHgui84PkGxZ4gSUE5J5aa9AnQKEuuS61kGMCXtgbA78RDvetDRH5oed8tt8+WF/wRdAJOoJDSpS2glYpQ0BGebO5teInyGts7g3EdOXH5PU856qDmf/VuuKFchLwgg+cD39LeEhKUJIHTOnPGZdKJljfvKj/UHL7NOOOVOf7I99K/KdUdOWoWJTue+4GuBAwolZNe6dTZ70vnn7Qfhzd2DDBpGPa26NHfmoa0igN9tjUDxLMnW6zcXn0nDFZwSj8ICBML5VnSEpJ+WiNoKnTYDcMC7nSAwx3344fNS6+gcbi9TKFF476kOMu7hNw407mfnDN8YSXiKsPJHAdFK61KkrlSRWVa9IU8yvM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(39860400002)(346002)(136003)(2906002)(186003)(316002)(4326008)(16576012)(31686004)(956004)(2616005)(26005)(8676002)(8936002)(31696002)(6916009)(83380400001)(6486002)(38100700002)(66476007)(86362001)(66946007)(66556008)(36756003)(5660300002)(54906003)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bkN2dGZ0NnpmRGtNV1lKTE50eVdnU2VRMC9LQ0g0dnJVSnRWSXFVSWRNUVNz?=
 =?utf-8?B?RDRtNHAxT296UHJoajBjN2FpSE1VSGtWYWM0MTdqSm1LeENud1ZsUnZsdjdr?=
 =?utf-8?B?aERDUWl0Q0F6dVBrQkIzV2ZpNjBvb3M5R0IwQVZRV1pzNzRwQU94WVNMMHFy?=
 =?utf-8?B?QklSaTludVNSbzJpNGNQQ3diZkt2bURMU1hCVEo2cy9EWnkrZ2FjUHdLZ0dz?=
 =?utf-8?B?WmVUWmw3bThNTWdkWmJtZ0UwbGx0TGFBZWlZTVdjRmVmQVpBY3RqNDA0TlBC?=
 =?utf-8?B?bDFOL1B6dTRtZ0VHdHFFdFpaekV5cUduci9TZU1NUjAzQm9wTnJCMnE3MlBu?=
 =?utf-8?B?QU9jeTI5V2JWZ2xreWFVM0MxMHdrT21xWDZEUzdWQkRTZ040eHZadEpVaHY4?=
 =?utf-8?B?R3R0RDFad2w0bUNJMkFDUlBXMURDTEtKU3lleDk4YnJmOE1ubDRMN29nQWRW?=
 =?utf-8?B?WFY3V3gwWU1VcHBDbjFSMFVrZmFmMmJyVWZwaXJJcjFZbWhnYmZNTUtvUUxL?=
 =?utf-8?B?c3Uwbk9pUW93Nkc2VzZqaGh4dExBNDBqYnlwUU1KL3VtNWk5VkYzU2hQOTVo?=
 =?utf-8?B?ZDBta3B4eVEveWRneDJTOTdqVFVRaW1BTVBGR3JMQWhuOHloN2ZXcTFLZTZB?=
 =?utf-8?B?ajFpUm5RRkVtZ0JVOUVFR05BSEk5OGlGSXVUSGVwMFFZYnJ5R01MakFvSmxj?=
 =?utf-8?B?REhJU2JkTHdzRGlNS3ZZTnA4bTEwcVhDSVd3OTkzSHk4bExXbEVTbE1pREY0?=
 =?utf-8?B?WUw4eFlrdVJhV1dyNEJTSG16VERNNmpWNlVtSFEvN29DSEZiVVdUWXdWYTVy?=
 =?utf-8?B?TVJFM3llZmI3U0VhU3ZYZkxBaWdPcEg3WHNOSjJySDhtNllZYnMvNGV4MzNu?=
 =?utf-8?B?NUlyWk52UXlHWWlTcGo0UXg2RVZ6V25YU0JaRlZiMy9wbVErRGY0cHUwdDY5?=
 =?utf-8?B?WmsyWnI3bHVVaDcwOGFSbHE5aEdPSWEvdjlTVTVQZkZsbURvaXdIOVhXMWNX?=
 =?utf-8?B?MHdjVnRmY1c4c0JvS0xEOVp3Z1ZJRk5YeUV6OEIwYjB6VThPQ1pFMzdjL0lT?=
 =?utf-8?B?VVl0UGI1OGpZakM1bTZ6TzQzaUJnSE4rcldrS1FNc2tKS00vLzhzZHN6UEtH?=
 =?utf-8?B?K2R0d0RjWGFrVGFsZkpOWHFQNlVNV3dFZHB5NlFMYmVuU3ltWVpXdDI4RjF1?=
 =?utf-8?B?WUFvRDRWMjI3cTJmNzJ5MHlES3Jtc1NieWFtVDI5ZFZhZXFrVE9VM3pBdFpE?=
 =?utf-8?B?VUhUS1dlcXFOY092QmJHditsa0J6VzlJOTF6bEIwZkFnZlZUek9sTXNHbjh5?=
 =?utf-8?B?NWhPZ3BoakIwdXdQalljVWgzS043WUdBYTRBYWw5WVQwVEMrSHY5b0lJb1ps?=
 =?utf-8?B?dThlZTc1ZVJ5RUcwU0hjUC9LZXVGb2IvcXVEUHVYcGlaaHBtTDBpYWwzU3Zn?=
 =?utf-8?B?UHBuL3BzRDhzMldaRm41Tk1LVWo3NjVPZXNmdXMyUjZmazk4UjBxZlNSWEds?=
 =?utf-8?B?cjZMbXdyRy9DbmFYWmxGUTVWSDA2dHJLWXpyaytveTJTblJselhvcU44emNP?=
 =?utf-8?B?SUwrZkxuSnUvL2N0OXFuTVB0anQ2c3pOSXpvQXU1Z2EvNGFlMFkvdjRCT0pJ?=
 =?utf-8?B?WnlGYThIU0dNME1BblNBTzNqQWsvTEYzUDB5MlhDVlFOdkJTTTNWdU1lYTNq?=
 =?utf-8?B?OVY3OWFneTUyYXFWTU5XWDhmd0MyWWFTNXpCNHNHOXZzeHVTNTRUNnYwUm13?=
 =?utf-8?Q?0q2bFKecNqmiTZNcyPdFfd/YQ5P/lNULqFXD95s?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 36cf4f44-f01b-4a7c-8621-08d979ca6d4b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 11:00:57.4965
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YUD4ROF98DrhOs/ELWeR88UJWCn6VOx8NZyK0TGaGfogV5msEMo+hSbD2+Y9esviinfECq2G95hDZAneVRihZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

Hidden devices are associated with DomXEN but usable by the
hardware domain. Hence they need flushing as well when all devices are
to have flushes invoked.

While there drop a redundant ATS-enabled check and constify the first
parameter of the involved function.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu_cmd.c
+++ b/xen/drivers/passthrough/amd/iommu_cmd.c
@@ -308,14 +308,11 @@ void amd_iommu_flush_iotlb(u8 devfn, con
     flush_command_buffer(iommu, iommu_dev_iotlb_timeout);
 }
 
-static void amd_iommu_flush_all_iotlbs(struct domain *d, daddr_t daddr,
+static void amd_iommu_flush_all_iotlbs(const struct domain *d, daddr_t daddr,
                                        unsigned int order)
 {
     struct pci_dev *pdev;
 
-    if ( !ats_enabled )
-        return;
-
     for_each_pdev( d, pdev )
     {
         u8 devfn = pdev->devfn;
@@ -343,7 +340,16 @@ static void _amd_iommu_flush_pages(struc
     }
 
     if ( ats_enabled )
+    {
         amd_iommu_flush_all_iotlbs(d, daddr, order);
+
+        /*
+         * Hidden devices are associated with DomXEN but usable by the
+         * hardware domain. Hence they need dealing with here as well.
+         */
+        if ( is_hardware_domain(d) )
+            amd_iommu_flush_all_iotlbs(dom_xen, daddr, order);
+    }
 }
 
 void amd_iommu_flush_all_pages(struct domain *d)



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 11:11:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 11:11:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189317.338931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBmD-0001cE-89; Fri, 17 Sep 2021 11:11:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189317.338931; Fri, 17 Sep 2021 11:11:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRBmD-0001c7-4j; Fri, 17 Sep 2021 11:11:29 +0000
Received: by outflank-mailman (input) for mailman id 189317;
 Fri, 17 Sep 2021 11:11:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C8mj=OH=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mRBmB-0001c1-UC
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 11:11:28 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe1f::616])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 27469ca6-0152-4019-a368-cc98f6524a41;
 Fri, 17 Sep 2021 11:11:24 +0000 (UTC)
Received: from DB9PR02CA0010.eurprd02.prod.outlook.com (2603:10a6:10:1d9::15)
 by AM8PR08MB6404.eurprd08.prod.outlook.com (2603:10a6:20b:362::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 11:11:21 +0000
Received: from DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:1d9:cafe::7c) by DB9PR02CA0010.outlook.office365.com
 (2603:10a6:10:1d9::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 11:11:21 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT016.mail.protection.outlook.com (10.152.20.141) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 11:11:21 +0000
Received: ("Tessian outbound a8bfe25d7364:v103");
 Fri, 17 Sep 2021 11:11:21 +0000
Received: from 81b846e8a460.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EA41F501-88C0-4637-B1AB-EB41EE5FD578.1; 
 Fri, 17 Sep 2021 11:11:14 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 81b846e8a460.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 17 Sep 2021 11:11:14 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR3PR08MB5707.eurprd08.prod.outlook.com (2603:10a6:102:8b::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 17 Sep
 2021 11:11:12 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.017; Fri, 17 Sep 2021
 11:11:12 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0135.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9f::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 11:11:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27469ca6-0152-4019-a368-cc98f6524a41
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=80l0B2gDxDQdj0K3hw5wQGZw9mIT5whM0YFZB9lnsTM=;
 b=mFTs8lWxWJApdH1skk0FXCErRhvsc00kLTcWwIYso2nIQgnkSiLuAnoVwGEy/pqCMjH38QUldNVapR4FJEIDsg4eDaqKK2yJp5Lj1e4mU+cDVFHgPeVD5vIN75FrZqTfoXdZvL5HWk0MKK1L3pJWc3b9UFwh0Z8EwKgzyomDOuM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f82c96ddd71e3f37
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VNyXxLGfpw4nH0Wr/4RzqSmKgHAg8GRittnWcUSrWz4HOntsWIPoa8iPUyATRqONTaeAE+xiKv+KiPmApD9YpUqH4ekBsVyDfibJiZTblvcrWDciYEqYoqjPVVr0aodsTulsQ6iqh9FJQOyyb9YYtgx3zB1AEA+ShOMVxaIMhInU8qrYB5yiTEXsQL3KehRlXV1/Zy6A22pbcCuMRx9170W5Mmzu+XbCDzL1+mRN9U5bowdvcsxcHsXueD2KG4XeNgBSwSkTgezqGA0AeNXmDZl5Vy6gjC13F/0g1O9kD8jYRCfVYYgnXiGF5LqwZrCrTv/ZBmuWMtiQ/L5xCZpwTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=80l0B2gDxDQdj0K3hw5wQGZw9mIT5whM0YFZB9lnsTM=;
 b=PcQzmgwkJfXzg+KAPw7g+BduN6sPbWtHTLTJOkhWr98J6G6r1SlThS5NJYuXqjPLYqHOEyEwl/Mm/Z3gpvkeG87T/ADIzihrebDJwjG+fAqSvhrAvvGWqrbp35UYuQreShqP6l0njvuOkUwuYLB0OLsCYtSgiOJomA2CkOuhdDE+6hdfkaLwupdyulb+UIUq4gk8AGRJZVxYc0A8bzHJPH/CLon6/xrDlUz1xHdGfSrt5yV3snev1SUg/Re8M56dksIJKGWV/5LdAJ54hfmklbln39qkOyspmeG2h3bH7bYstiT2IItMQExIAL90jxmxpAf+KgkY1s5Bi/6cw2cxVg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=80l0B2gDxDQdj0K3hw5wQGZw9mIT5whM0YFZB9lnsTM=;
 b=mFTs8lWxWJApdH1skk0FXCErRhvsc00kLTcWwIYso2nIQgnkSiLuAnoVwGEy/pqCMjH38QUldNVapR4FJEIDsg4eDaqKK2yJp5Lj1e4mU+cDVFHgPeVD5vIN75FrZqTfoXdZvL5HWk0MKK1L3pJWc3b9UFwh0Z8EwKgzyomDOuM=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
Date: Fri, 17 Sep 2021 12:11:05 +0100
Cc: Jan Beulich <jbeulich@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
 <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
 <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0135.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9f::27) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5d91c96e-c267-4b5f-7799-08d979cbe153
X-MS-TrafficTypeDiagnostic: PR3PR08MB5707:|AM8PR08MB6404:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM8PR08MB6404C725DDF7373C54133389E4DD9@AM8PR08MB6404.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 lD8h8HIajlFrZsW0dFycRW0aCkGVst/f6LnzgSDGTr0RkailRsBVvmctjtHbE+Lp2dmf65iglKNlGAbgw1061QaWDkiU8KHsQd5GrfkSUC9PkvjX+hZHSKGaRqtDk+zEWC+Cz1+PFv3GvxZt6FInjWiOqfqx4DDjk+J2JVCwA7hcBVusEuRWMLizCd7+fU2CQZDqWs9HgEOvUznT03b2YhX1vZZaHAb+tei5KQw/e5M5EkXiaFImuUhI79dJrrCMcrsvnbwZP7HG9k/sJgcjQE+XIHC4js55S2kbbbgzwTmvbwIg91MdD3U9PhNf5ZfnohKWXMb9Mf4zrtfAK9tVQqEYhIf3fbYIWgJs2hcWQDCAip0mhjrQAi1Ppe/6Jj2wwyUQq+Vcus2gyy0cuV2AokGRde/9mAdd2G3PeVKATSJwMn+qqblKEodmeH+9xmR99mBsI6x1A04dXYTNV7pd+cu0g332koF2VFMFAZ7/t9/A5JxkEAPOcW3QVAaz+1hgBVhoFpZd+dYrxgYu8yPf6u7ms/5+V4VRl/88gWuTY3ch5QgFW9gR+HETorgNlEir1jjZwXKSQJetiOS1BcDllMUZ8o6vCbGjNRF3vySO/yC1o/FSM3ceGzYBM7UpQSc4bQBz0RlcZxzxL42xPDf426l1WGXfFQHPRlIvMY05zcM1IVT7ZNDYwT7NW1+A67MbUCg2xWXhOqMekbrvyiskca1A2xrVQj5nL5IkdHtC56UxNrqMZfVyhz8l3A4OyPfw
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(508600001)(83380400001)(6666004)(66556008)(66476007)(44832011)(6916009)(956004)(38350700002)(5660300002)(186003)(36756003)(6512007)(316002)(38100700002)(8936002)(2616005)(26005)(86362001)(52116002)(53546011)(8676002)(33656002)(6486002)(54906003)(6506007)(2906002)(4326008)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5707
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	87400aed-8072-4559-7e76-08d979cbdbca
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nrhWL4U4cOxfFa7+GEdIVo3fZA/SUNQFct3SB4UZjeUuv0g81uTTpO6PFyo17x93+fuoitDvPU+LXd7CxITzctp1Scg5hxE5sHEoMwW7mF/vDwBdF6B2sa10nKjt6dpz2oiVIomGhjBnaZtSHfmJ3B+A+BMnh3/QmtpsYKw6rcciRGp9I4n1It6a316b+gw4IavUL8tjjWk8qwfblqwSF/iixqNK+NbOopo1foJcjXlzgiT2OE3SKGcCmfI6Yr0QnkylJw67XzZsjUueAngUcmcmRAxlyKbUAKtKl9FMirTjIoVw3kS+4u9kxuQ7s2V5LkkwRD4N2zVfGYtVRpJHo+lhkxDS71zloyTu2oFEwpdwRWxmrTYAkaW/FG5Nb4hCUwl5rEkpkb+GnoOo4lq7w1E659UXGzoC9wzqHFSIkThUnT+RW5x06axoUxTRVJ6jA3mEorPtaOwqvJgc40ls/TZ7MdOy1tHPBAS6Rqv8eCksq2wEyNrTVXI+WqVhkGKucstQedtRZU9C+9m5Y/G8gRS8DFuzhgNUSc03T+c4BFFk7h5r/Eg2z4vsWNSfvTHOgUGV9LPQyqxJBEDBZqXM/O/3TMhl016B1k3HsdPH0a6p9/1tlqqXvnZLyfnIzsC4gR+fgJlNkukiy4B+lM6QKTbwfmk9E2UCaWylRBkIMcl3AgjSAr82ph7MxDjYbdzp7y9hHM2JNm90Xva301sk+WbKDl2izdVHyCP+k3vhNwY=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(396003)(376002)(136003)(46966006)(36840700001)(478600001)(2906002)(44832011)(956004)(8676002)(70206006)(26005)(82740400003)(336012)(70586007)(186003)(54906003)(356005)(4326008)(36860700001)(2616005)(53546011)(6506007)(82310400003)(6666004)(316002)(6486002)(83380400001)(8936002)(47076005)(36756003)(6862004)(33656002)(5660300002)(6512007)(81166007)(86362001)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 11:11:21.5191
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5d91c96e-c267-4b5f-7799-08d979cbe153
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6404



> On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Thu, 16 Sep 2021, Jan Beulich wrote:
>> On 16.09.2021 17:07, Luca Fancellu wrote:
>>> I explain here my understanding on dom0less, this feature is used to st=
art domUs at
>>> Xen boot in parallel, the name is misleading but it doesn=E2=80=99t req=
uire dom0 to be absent.
>>>=20
>>> So if you have a dom0 kernel embed in the image, it's completely fine t=
o start it and it=20
>>> doesn=E2=80=99t have to be skipped.
>>>=20
>>> Here the possible user cases:
>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embed=
ded in Xen image, domUs on DT]
>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen im=
age) and domUs on DT]
>>=20
>> If that's the intention - fine. Stefano?
>=20

Hi Stefano,

> What do you mean by dom0 modules embedded in the Xen image? I am not
> familiar with it, but I imagine it doesn't involve any multiboot,module
> nodes in device tree, right?
>=20
> Putting aside "dom0 modules embedded in Xen image" that I didn't fully
> understand, there are three ways to load Dom0:
>=20
> - dom0 kernel (and ramdisk, optional) on xen.cfg
> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" prop=
erty
> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,bina=
ry" property

True for the #1 and #2, the last one is not implemented. The uefi,binary pr=
operty
for now is only used to load domU modules.

>=20
> Then, the other use cases are:
>=20
> - true dom0less, domUs on device tree using the "reg" property
> - true dom0less, domUs on device tree using the "uefi,binary" property
>=20
> And of course all the possible combinations between Dom0 and DomU
> loading.
>=20
>=20
> Currently, patch #1 checks for the presence of a Dom0 kernel node and, if
> present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
> node is *not* present, efi_arch_use_config_file returns true.
>=20
> However, this could be a true dom0less configuration without any Dom0
> kernel. If so, you might not want to load xen.cfg at all because it is
> not needed. In a true dom0less configuration, we probably want
> efi_arch_use_config_file to return false.

In a true dom0less configuration we might need to read xen.cfg to retrieve =
the
Xen command line, but following the actual implementation of the common cod=
e
there is more. I=E2=80=99m going to explain.

What efi_arch_use_config_file really does is not only choosing to read xen.=
cfg
or not. Following the common code (xen/common/efi/boot.c) and what its resu=
lt activate
along the path, it basically decides if the UEFI stub is used as a loader f=
rom filesystem
or not. We need the UEFI stub as a loader to be 100% UEFI and load our modu=
les.

The original check basically says =E2=80=9Cif there are multiboot,module in=
 the DT, then some
bootloader has loaded in memory the required modules so I=E2=80=99m not gon=
na load anything
from the filesystem because I assume it puts everything in place for me to =
boot.=E2=80=9D

>From misc/efi.txt:
When booted as an EFI application, Xen requires a configuration file as des=
cribed below unless a bootloader,
such as GRUB, has loaded the modules and describes them in the device tree =
provided to Xen. If a bootloader
provides a device tree containing modules then any configuration files are =
ignored, and the bootloader is
responsible for populating all relevant device tree nodes.

What I=E2=80=99m doing in patch #1 is restricting that check to just the mu=
ltiboot,module that are
Dom0 module, why? Because with the introduction of dom0less we need to spec=
ify
multiboot,modules for domUs, but the presence or not of dom0 modules is the=
 only
Information we need to understand if the user decided to start Xen with eve=
rything
in places (modules in memory, xen command line, dtb) or if the job is deman=
ded to the
UEFI stub and its configuration file.

By the configuration file you can also load in memory the Xen dtb, so Xen c=
an
be started as an EFI application without the DTB and then load it using the=
 EFI stub.

I=E2=80=99m not against this new property =E2=80=9Cxen,cfg-loading=E2=80=9D=
, I just think it is not needed because
we have all the information we need without it and in any case we need to r=
ead the
configuration file because otherwise we won=E2=80=99t have access to the Xe=
n command line.

Now I=E2=80=99m going to show you examples of all use cases that are valid =
with the introduction
of this serie:

1) Start Xen as EFI application and load only Dom0

    Xen.cfg:
    [global]
    default=3Dxen_dom0

    [xen_dom0]
    options=3D<Xen command line>
    kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
    ramdisk=3Dinitrd-3.0.31-0.4-xen
    dtb=3D<xen DTB>

    DT:
    {no modification}

2) Start Xen as EFI application to load a true dom0less setup

    Xen.cfg:
    [global]
    default=3Dxen_true_dom0less

    [xen_true_dom0less]
    options=3D<Xen command line>
    dtb=3D<xen DTB>

    DT:
    chosen {
        #size-cells =3D <0x1>;
	#address-cells =3D <0x1>;

	domU1 {
            #size-cells =3D <0x1>;=20
            #address-cells =3D <0x1>;
            compatible =3D "xen,domain=E2=80=9D;
            cpus =3D <1>;
            memory =3D <0 0xC0000>;
          =20
            module@1 {
                compatible =3D "multiboot,kernel", "multiboot,module=E2=80=
=9D;
                bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=
=9D;
                uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
            };

            module@2 {
                compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,modu=
le=E2=80=9D;
                uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
            };

            module@3 {
                compatible =3D "multiboot,device-tree", "multiboot,module=
=E2=80=9D;
                uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
            };=20
        };
       =20
        domU2 { <as above> };
    }

3) Start Xen as EFI application to load Dom0 and DomUs

    Xen.cfg:
    [global]
    default=3Dxen_dom0_domUs

    [xen_dom0_domUs]
    options=3D<Xen command line>
    kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
    ramdisk=3Dinitrd-3.0.31-0.4-xen
    dtb=3D<xen DTB>

    DT:
    chosen {
        #size-cells =3D <0x1>;
	#address-cells =3D <0x1>;

	domU1 {
            #size-cells =3D <0x1>;=20
            #address-cells =3D <0x1>;
            compatible =3D "xen,domain=E2=80=9D;
            cpus =3D <1>;
            memory =3D <0 0xC0000>;
          =20
            module@1 {
                compatible =3D "multiboot,kernel", "multiboot,module=E2=80=
=9D;
                bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=
=9D;
                uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
            };

            module@2 {
                compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,modu=
le=E2=80=9D;
                uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
            };

            module@3 {
                compatible =3D "multiboot,device-tree", "multiboot,module=
=E2=80=9D;
                uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
            };=20
        };
       =20
        domU2 { <as above> };
    }

So as you see every case is covered without the introduction of the
property.

Please let me know what do you think.

Cheers,
Luca

>=20
>=20
> In general, loading xen.cfg or not loading xen.cfg doesn't seem like it
> should be dependent on whether there is or there is not a dom0 kernel
> node in device tree. I think there are too many possible combinations to
> guess correctly based on the presence of the dom0 kernel node. Instead,
> I think it would be better to have an explicit flag.
>=20
> Today, if a "multiboot,module" node is present, efi_arch_use_config_file
> returns false. So I suggested to extend it so that if a
> "multiboot,module" node is present we look into a specific
> xen.cfg-loading property and if present return true otherwise false.
> This way, we are backward compatible.=20



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 11:35:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 11:35:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189333.338960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRC9S-0004VI-Kf; Fri, 17 Sep 2021 11:35:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189333.338960; Fri, 17 Sep 2021 11:35:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRC9S-0004VB-H7; Fri, 17 Sep 2021 11:35:30 +0000
Received: by outflank-mailman (input) for mailman id 189333;
 Fri, 17 Sep 2021 11:35:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+lGq=OH=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mRC9Q-0004V5-Mz
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 11:35:28 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0a3addd1-23b8-4e6f-8e03-f1eeb8d4b1bc;
 Fri, 17 Sep 2021 11:35:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 1AFDF1FDC7;
 Fri, 17 Sep 2021 11:35:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E730014102;
 Fri, 17 Sep 2021 11:35:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id +CBmN359RGFAKQAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 17 Sep 2021 11:35:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a3addd1-23b8-4e6f-8e03-f1eeb8d4b1bc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1631878527; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=HcGZS+FDX8DdT1W8lmf9U7WzYN/MAtRfhAF9s0ohGYM=;
	b=WD3lw50Gt+6ZMpkmVs40GJTztbcc911Io34NZGq4RfGTN7onx9wdnRg5XiuqTpO3kHYcQ2
	UWLLLN+bzcioig35fjD2mkEpEW54WB+D42oTlQS2zJGkbi4RK7O9pRa7MiPP96kd8Zm35X
	LTIF7U12YiZ+shwCAA1egJwg0XOM75M=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	boris.ostrovsky@oracle.com
Subject: [GIT PULL] xen: branch for v5.15-rc2
Date: Fri, 17 Sep 2021 13:35:26 +0200
Message-Id: <20210917113526.7963-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc2-tag

xen: branch for v5.15-rc2

It contains:
- The first hunk of a Xen swiotlb fixup series fixing multiple minor
  issues and doing some small cleanups
- Some further Xen related fixes avoiding WARN() splats when running
  as Xen guests or dom0
- A Kconfig fix allowing the pvcalls frontend to be built as a module

Thanks.

Juergen

 arch/x86/xen/enlighten_pv.c |  7 +++++
 arch/x86/xen/mmu_pv.c       |  7 +++--
 drivers/base/power/trace.c  | 10 ++++++++
 drivers/xen/Kconfig         |  2 +-
 drivers/xen/balloon.c       | 62 ++++++++++++++++++++++++++++++++-------------
 drivers/xen/swiotlb-xen.c   | 37 +++++++++++++--------------
 6 files changed, 85 insertions(+), 40 deletions(-)

Jan Beulich (9):
      xen/pvcalls: backend can be a module
      swiotlb-xen: avoid double free
      swiotlb-xen: fix late init retry
      swiotlb-xen: maintain slab count properly
      swiotlb-xen: suppress certain init retries
      swiotlb-xen: limit init retries
      swiotlb-xen: drop leftover __ref
      swiotlb-xen: arrange to have buffer info logged
      swiotlb-xen: drop DEFAULT_NSLABS

Juergen Gross (4):
      xen/balloon: use a kernel thread instead a workqueue
      PM: base: power: don't try to use non-existing RTC for storing data
      xen: reset legacy rtc flag for PV domU
      xen: fix usage of pmd_populate in mremap for pv guests


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 12:09:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 12:09:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189345.338983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRCgY-0000DP-Mr; Fri, 17 Sep 2021 12:09:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189345.338983; Fri, 17 Sep 2021 12:09:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRCgY-0000DI-Jf; Fri, 17 Sep 2021 12:09:42 +0000
Received: by outflank-mailman (input) for mailman id 189345;
 Fri, 17 Sep 2021 12:09:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRCgX-0000DC-5v
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 12:09:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2187597a-17b0-11ec-b689-12813bfff9fa;
 Fri, 17 Sep 2021 12:09:39 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-2-fEf65TicOAqmXWqh0E-pGQ-1; Fri, 17 Sep 2021 14:09:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4608.eurprd04.prod.outlook.com (2603:10a6:803:72::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 12:09:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 12:09:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR05CA0091.eurprd05.prod.outlook.com (2603:10a6:208:136::31) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Fri, 17 Sep 2021 12:09:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2187597a-17b0-11ec-b689-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631880578;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mgKqa1a2alCeEDcpSkSE04vyMAsol14/hENrc2dKKfQ=;
	b=YjvpnnEaGGgLGvwWkaEt8DfZxHdFkcuS/20JyzwjgsNHPNuxDUji/M1qt5i+HEBTeZR/gf
	tyLZyu26hr5HAmmT/SyeZH+4zoLPKt/NkUj7tIhJb+YkmibS9TQHLtvYhqIFrhTUauheyB
	FGqvN+Jn2OKbGY+OIVnD08hAKKjEmXY=
X-MC-Unique: fEf65TicOAqmXWqh0E-pGQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WvuHhOIOXfGqJP49vGk2rmdXQUCQHhzOGiVru4g6pgswX0/ioGJVcDfSe0NtS8h2R32L1wPI6UOKGnVzxGJdHESDzbyhU0sCOL6MpJI3E18wsVsbYBLg3QglCeEmXW2ogTaAT4Wsr+9Kbfv5QCv3L6ZzLl7Y0Ab4zjLt5KZpPA5BIg/+xEoyYoejvdNxQV6Ls0f8FXPzw2auoxVuFQ/Nu7c1ijGKiYVgrBQw0zocI3Fyls+Pw2ga4WlxEavIyFzGZOqR8BO3vJzLT/m8AYwd4wvMNMaIQUR00FxU9T1IdJ+He32akbB+2+ok+Gx980mUIEr6y3fCv6Fl+4Dc4QaDfA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mgKqa1a2alCeEDcpSkSE04vyMAsol14/hENrc2dKKfQ=;
 b=UVWnfE8R9p7s37vjS6/ZKavpzPc04WJKxXMuEb9P+8zmYfLEImk7LhNumBEH1ob30E55e6qLfV2k/UnVpa4LH8pt3ecqMm9c/A4Aj1DRgTmnDMVr3yVrjUn8UKhNj5/oYmeeUM4YmbcyCGElerCpAYT7JwmjbFNdpha2IoxGFOFBPQuaA3LLrVj8uPHve9DdPvoxmbd1eTPjCtiKoJF5dQMrvlloNUeh226ynb/q9KdVKOEb4S2cF84fklJgYTLe41zaTe7I9iBwmAC773wg67ndg7g8usyhC1bcJa/Bk0kgYFnvA/iC0xN87r3K7l0OD7jz5bKDKO6cAyJq7emqRw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v6 09/10] kconfig: update xsm config to reflect reality
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 xen-devel@lists.xenproject.org
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
 <20210910201305.32526-10-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <33d87667-4be7-0ec8-0c1e-f773d77f2d72@suse.com>
Date: Fri, 17 Sep 2021 14:09:33 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910201305.32526-10-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR05CA0091.eurprd05.prod.outlook.com
 (2603:10a6:208:136::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cef4b084-c852-4ecc-bcf8-08d979d40425
X-MS-TrafficTypeDiagnostic: VI1PR04MB4608:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB46081F63970E1039EC1FBD3FB3DD9@VI1PR04MB4608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1ixsxzN2C3uhfvoyFhhRS9prEQcEeson01LF+EctWRFRjyBwL48HXk0yrq35S/mZqAj6bJVf7djW4ky0bCXl1SmXEcNfrObuXseunMIGa5s5eVfcwanFLgG8i1p/DhydiE0pNsHdHTNbpWb1vKxrLj5/lRZiqGnqo8Plh+cvtE1m8vMffQYfxsDgXaSKIKsUOIQNDsuYyaCTUMeFpAZ2H8MH+LZMfwISEFVMAg6TUT0VIMHT7R54iCnG8RLaqo3hd2/m0n4PtYGMzgAS9NeuAlsFittdl+ltxeuqONrVKuzct1LEjWmPKfSwv47sYsR7+saoAdXA1/xKaxWS7BTDopJNfqkCbxxjpLdJmMuvKoifRrqppSunGLCvIjXQBq/jOFdK0Ol+c8Ip5DEsc8OozqYslGthTBzQLvXsZqYRTb2Hngv4dwfPpO7RWofPBw88hsMZv9mAARBmmyMdouPxhNFb4KLm97AoIxOz5s6NZRzcrZukW2V30gFsRIIngB9qJvfcAMZpeVCJ0qOrkwe55miiBHuqo2IWceScGuUr/20miuXZ2Ieg7GTorrbURoL6cPsINjD3aP8lSAgL6ws9sU7yQCMZbNBvN69j2xe2gNcVMxHW8fK5NkwZACH8XKO9cKKsNsLyl3cptiMAqHBoNm/dRlRNtnRpeO9rUnstKDvn6M7GBLFI/nPKXwqFYfWG1vHNC3g80OuxKH7D0W6KepAo77jiMWTu/GY2ZyyCmUk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(366004)(376002)(346002)(5660300002)(38100700002)(478600001)(956004)(26005)(36756003)(2616005)(316002)(15650500001)(54906003)(16576012)(4326008)(31696002)(6916009)(86362001)(31686004)(6486002)(83380400001)(186003)(66556008)(66946007)(8676002)(8936002)(2906002)(66476007)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZUdDcFJDMDRZOGIyOFFNUEV2SHA5amdlZzhDaWgycmorcVQ5dlFKZHJhd0w2?=
 =?utf-8?B?cWdMYVB0VVB3bVFlSGltREhzUE1FM3hOTHoweFhMV3ZXQ0FqbG9ya01kemRz?=
 =?utf-8?B?cmVCMWxFc20rTjBVc1B2aDh6NUYyZDJXV2d1RlJZSGhNdnd0Wm1ENWhFN3Ey?=
 =?utf-8?B?Nyt0b2duTERMWU1JcWd6c0Uzd1I2dkN1OFhNdGhxOXJxN21iOC9STVkyWmll?=
 =?utf-8?B?RTVBWHJoQTJOR09YR3BaelJibG5CMThoL2U3dVBSUUhKbkVIV2VhSlRVbXUz?=
 =?utf-8?B?OEJvS0tGZC9Fd05xOE9ZdnNWc0xDUjlJMGJCMmIzZVZ5RWZyVHBCb3lTSnEx?=
 =?utf-8?B?RmNlWGRRaXpQSFBSSVFhcElQUzFKd2Q1OHhabm0rY3hMS0dsZmJDTm45a3ZW?=
 =?utf-8?B?dU5GL1F1MzkzRzRQbG5mVnBiL0c5RTZqYTJUbWw0NDR3VWhNUE8xZWlFamVN?=
 =?utf-8?B?bWNqL3pQaUk1cy8zSTlReHk4QW1iUlFWenhvM0h3RlQyZnhxRW9zakNjdDJB?=
 =?utf-8?B?NXVrLzZpdUt5b3NjTGJiNXFDSW5BcXZCVnN6M3hPWExvSWpZNnpySTU3YjE4?=
 =?utf-8?B?V3hRMkxjcHlBZGp5VCtLbFNuVmkxdVE4NWdtT0ovWFJPdGZpM2V2NzBvblVk?=
 =?utf-8?B?MVNEMUo0bHZ5Z2llWVNUYWZTVmhlQWh4WktUa212WktxRjJ0WGhyYlcxYTNq?=
 =?utf-8?B?YmppZXFRc2JkYXJFU0ltaFljcjgxSTZOQ09YanNDU2pjYTVsbjRKRm51eXUv?=
 =?utf-8?B?d1dVaEZiUXUwQmI0N2s5cXN2eUpKOFlTVElBWFZBMTc3RFE4MkFtbitaV1dU?=
 =?utf-8?B?cjB1cThTY0tQak11WHdRbzhBL3RrMDNpTEd0bGlQR0hPV05Ld2FwS25VNDls?=
 =?utf-8?B?ekZFTWxHVGVpU3NTNmtTbFhZY0pXWW9qQTVDeUtKTEJVKzNiYWh0RnRjYjlW?=
 =?utf-8?B?d3FVNUVTSEljTHUwYWQ5R01Bemo4eDcreW8vWEI0YUtFcTVsTDB0b0JlVEpi?=
 =?utf-8?B?NU9SdVgyN3lST2NQRGNwUG1ERTVqd3JOV2VWZ2ZYT29ndjVUT3N2YzRObHJB?=
 =?utf-8?B?NG1pa0x4WlpnNlIyL0Q1Y3ZpaytRQXMwVkY3WG04ajdUbCtQME9VeVpKOXBE?=
 =?utf-8?B?UnovYnIrN3d6dGswRHNid0NQRWZaeWo3Y2pUeWNWVmNucVl1b0s3d2ZMWGor?=
 =?utf-8?B?NG53ZmZwN20zbTM4dTZsbEI1ZmdMa2h1aXJXQjZJU1ZQNk1JMmJMN1hUVFVM?=
 =?utf-8?B?QXhNQWtKUERYSU43cUk1UjNUZWJGQjcybVFEOUh1SW5nZEdibjBicU94R1Vx?=
 =?utf-8?B?UXBPdWRyM01ZblVBcEF3QkpENG5JbVY2eUNtVXFLZGIyWTdwdVRWeGdFOWk2?=
 =?utf-8?B?T3NyU2hXSVo3UFhMT2ZzOFM3YnptK0JyWFNvc0JBQThId3YzVXRkbEx2d0Jr?=
 =?utf-8?B?UDVFU1dQbnI2b2Z1bXBEZ3lNNFIyQUxVYXhlY2VoanNKNXZTY0Z5Mml0ZkxZ?=
 =?utf-8?B?UFZVQ3MxbEV0Z3QrTC9mazZYc0JRSzRzak9TbVJKM0hQOGRlNmVmYUgzUEpC?=
 =?utf-8?B?U3k3c1VGaUZOaW1WM2M5TlZSY09XN0ljUWxOdHhHRXAzT2Y4V1MwRUVNMkRl?=
 =?utf-8?B?M2tRK3VSczh5WUNPdndtd3JlRExzRFVacmU3TUVKUzFRMXV5NWNPeFpUWnZF?=
 =?utf-8?B?YjliNUFsd01wa1RNaFc2eTlkUzZtaGh6OWZ4ZDU3VEhwUGNDS0g5TElIRzFq?=
 =?utf-8?Q?yFiXvqOhD8wpzCKk4J5Iows14fcgyMnSbkVc37e?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cef4b084-c852-4ecc-bcf8-08d979d40425
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 12:09:36.0344
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RAx1+ea9poGruR9lNhm4vE1rBOg6r1Mj2Yfn/FA1mLiiyAPJ25zZ8Q4SLK9HqIYSbQstfyX7L/+ldmJArH/9kw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4608

On 10.09.2021 22:13, Daniel P. Smith wrote:
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -200,23 +200,20 @@ config XENOPROF
>  
>  	  If unsure, say Y.
>  
> -config XSM
> -	bool "Xen Security Modules support"
> +config XSM_CONFIGURABLE
> +	bool "Configure Xen Security Modules"
>  	default ARM
> -	---help---
> -	  Enables the security framework known as Xen Security Modules which
> -	  allows administrators fine-grained control over a Xen domain and
> -	  its capabilities by defining permissible interactions between domains,
> -	  the hypervisor itself, and related resources such as memory and
> -	  devices.
> +	help
> +	  Allows for configuring the Xen Security Modules (XSM) policy or policies
> +	  modules that will be availble and which will be the default.
>  
>  	  If unsure, say N.
>  
>  config XSM_FLASK
> -	def_bool y
> -	prompt "FLux Advanced Security Kernel support"
> -	depends on XSM
> -	---help---
> +	bool "FLux Advanced Security Kernel support"
> +	depends on XSM_CONFIGURABLE
> +	select XSM_EVTCHN_LABELING
> +	help
>  	  Enables FLASK (FLux Advanced Security Kernel) as the access control
>  	  mechanism used by the XSM framework.  This provides a mandatory access
>  	  control framework by which security enforcement, isolation, and

I continue to think that the default here and ...

> @@ -253,10 +250,10 @@ config XSM_FLASK_POLICY
>  	  If unsure, say Y.
>  
>  config XSM_SILO
> -	def_bool y
> -	prompt "SILO support"
> -	depends on XSM
> -	---help---
> +	bool "SILO support"
> +	default y if ARM
> +	depends on XSM_CONFIGURABLE
> +	help
>  	  Enables SILO as the access control mechanism used by the XSM framework.
>  	  This is not the default module, add boot parameter xsm=silo to choose
>  	  it. This will deny any unmediated communication channels (grant tables

... here should not change. If it changes, the change needs justifying
in the description.

> @@ -282,15 +279,15 @@ endchoice
>  config LATE_HWDOM
>  	bool "Dedicated hardware domain"
>  	default n
> -	depends on XSM && X86
> -	---help---
> +	depends on XSM_FLASK && X86
> +	help
>  	  Allows the creation of a dedicated hardware domain distinct from
>  	  domain 0 that manages devices without needing access to other
>  	  privileged functionality such as the ability to manage domains.
>  	  This requires that the actual domain 0 be a stub domain that
>  	  constructs the actual hardware domain instead of initializing the
>  	  hardware itself.  Because the hardware domain needs access to
> -	  hypercalls not available to unprivileged guests, an XSM policy
> +	  hypercalls not available to unprivileged guests, an XSM Flask policy
>  	  is required to properly define the privilege of these domains.

I also continue to think that this would better be a separate change.
Or if not, the seemingly unrelated change needs mentioning in the
description (to make clear it's not a stray change).

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 12:13:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 12:13:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189353.338997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRCkX-0001bP-A2; Fri, 17 Sep 2021 12:13:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189353.338997; Fri, 17 Sep 2021 12:13:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRCkX-0001bI-6k; Fri, 17 Sep 2021 12:13:49 +0000
Received: by outflank-mailman (input) for mailman id 189353;
 Fri, 17 Sep 2021 12:13:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRCkV-0001bC-Di
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 12:13:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1ac81270-c439-4e05-bd01-7a23dab0d2bd;
 Fri, 17 Sep 2021 12:13:46 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2054.outbound.protection.outlook.com [104.47.5.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-S3h9_AvENdKWIa2K-qSsPQ-1; Fri, 17 Sep 2021 14:13:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3392.eurprd04.prod.outlook.com (2603:10a6:803:7::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 17 Sep
 2021 12:13:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 12:13:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0007.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 12:13:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ac81270-c439-4e05-bd01-7a23dab0d2bd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631880825;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8F/k79EhMHT3GBRKH4JctgK/8Uqk+wst81sqDUTHM9I=;
	b=UK+K/Y77HuQKddygaHNmsFcX3cNHaIkrm3+OHltNYMXi7BeWQijrf/MUt0b1DvyM0U/gvO
	tmryKkae2coL/tFyuWIjeBaFiB5loMK/ZLiGi9JI37SBUE7TFoYxgufYj+QZr32+QHvZTy
	H5nQqmeutsvvqCpufB5R3ibbxrmkJuo=
X-MC-Unique: S3h9_AvENdKWIa2K-qSsPQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BJnBINjIc//vEDQnFihqQY/8i9/C0wr6xH6W44N8kFmXc8aRUZ+MqU1tbSpy3U9SGo2MjhMgknEmxSWz/sJnnoHCkJySyWk6NRpKZPqjjsQUShBEQGTm6o5ie0l7j0QpxLtST9A0DnxhE/GSZ3RI0EBKS3k0fbxsPkqHO4lSuSlyn4DYowV3FTHLpVrPpQMz6MdB9+923F72hOi7Kvo3CxYHmqc55dCezpwPkM82zxr5piUwWQhKE4HHBWrVyxPLvz7eAZA5EaRQht/qBh4CDjgC0guTiqbmjmYBsBP1v3G16w5pDnq1gt+HOC2s1guAulqLwaSgfNnPeowE6LaiRA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8F/k79EhMHT3GBRKH4JctgK/8Uqk+wst81sqDUTHM9I=;
 b=a32RJeqHP3F/ItJ+wd796j+ORuUXLOVUVkEqiy92/5l4Mji2fOezKwskcOp8dlpnqODBF2TheR7mSmxTUAefaAmUtGHPQre+lfkdYxPB7thSWnr6r/otaleJ1mqLymtyqw++ZytyDhsEBjEev3Oij7Bg9DK/ThJKuIaZnW38mFsfSB/cX4qsmz2Foqa51GeYEbM+eEuUD5XiXEz8Ez2YiV88qh8Sawei8lw4sjku5Es/3W1O+LoP2nRploldDE6+3uR6BSwer3nMwF4f4LVbCnjrnkItstQywuAAp1wPLFVP424gXg/CDtkfMS8pY6tLlANVJyQAxVru+zVXaa367w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v6 10/10] xsm: remove alternate xsm hook interface
To: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>, xen-devel@lists.xenproject.org
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
 <20210910201305.32526-11-dpsmith@apertussolutions.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f438520a-4692-5487-5bf7-86b71a723e49@suse.com>
Date: Fri, 17 Sep 2021 14:13:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210910201305.32526-11-dpsmith@apertussolutions.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P195CA0007.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f68737fd-5835-47d5-2266-08d979d49712
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3392:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB33923782CC492AA5B293797CB3DD9@VI1PR0402MB3392.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tRQJ4l8Ow39znwr3VL1ARbxlYzv7FdZh6Zr0ZTHCkLCLKwimTSEtorHOysFy5jzd/lZxZA4Zn5W8O2N+J3JpXeVNjgE/ezshQ2m/UdvFqxaVFY/MYbATLqzE1ELkcwJBVvfMek7uyOAj9uaNpLbVyHR7Hjn35ojpNHzrNTHXhlGmolY6qlVU+gLjy2yC0hffL6hpnJwzemX2TNHxuLPmrdrPEw1LSF0HgZEhsYPoPAbIR0q0ePsZWFYbWTVRQDCdRMrIMYqC64/8H86yypdkOjYjOnsT2BvthosRKkMUX62k38gq8lXl2Y2oWfxrMyV/ELix6zhPf634PWuC9McmVrKM8v+x2SQNGMef/Fu2dETfGzT4cTm7tMgahnwdE73zIJXytmfqgogxM5Sz+sOqz6EAAIJH+KkCLC/Myi8QPJpRmjvwM/uGv8DB+p686u9tlPmCeO57FL/DPdCy9TFgTDvLtxgqmqTrHYjlF9/Ui6XEBpQzg86QN5WsJCwiYQBkshLT0iDa7f00BBaOW9ulp4c9Rb/X8f6R96wKixhiU17Puei0hCGFFWg1NPHBG5V7guhzykeDTBVmR4uFRvjWyse3u0tJ4cAPqGbR2cGsCSSasA3lcs3sFM+rfwPEVpUCIccWm0zYo3EnehJ/bUvln/NtPfqmhQSLS6PnApGHSkWtsRb9GwQce/N9gNtum84VsjSQXOt93ZoHDsEyG5caWDm7JDJiMChzMd/2DYTlvCg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(6916009)(16576012)(86362001)(316002)(66556008)(66476007)(38100700002)(8676002)(956004)(6486002)(2616005)(36756003)(4326008)(5660300002)(186003)(508600001)(31686004)(53546011)(83380400001)(2906002)(8936002)(66946007)(4744005)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R3VVWm9GQkM2NzRFaFQ2bGdmTDNBMlFGNk1vZkpwQlp6a1NHZWw5T3BJcjhv?=
 =?utf-8?B?cm40S2JlemZvUnd4bGU3VndYMno4RnpXdk1NK2I4ZUpmSVZEbzZtMjRjaU1I?=
 =?utf-8?B?WnF4aWM4L3A4Qk9EdUt5ejFlS3lINW1YN3BQL0YwTG9GQ0V5djl5MDJSeFNB?=
 =?utf-8?B?MW1ZWnZ1UzNqbXhwcDRYa0dOb3pJU2NUVklrTTFBSGFNNm11bnN2OEd2NDh5?=
 =?utf-8?B?QUNWQ3BHZ1hHYndqZlpYeWNkTWFFM0hWUzlieU1EVnJrZDZ0YUoranF1R2xE?=
 =?utf-8?B?eUxsUDlocDFwR01vNU9sZ1pudmtMVEN3NTVjOVVTZlNWU1BDT3drbHVVR2xZ?=
 =?utf-8?B?WjQ5c2k1eEUydTE2N0FqSVU2MnhEMnloQ0tHRTVydVJKbFR4WTA5WUI1eUZJ?=
 =?utf-8?B?S1dqTU9GTUVpNmc3QlNSc3ZxUHo0ZTVXR3U1SEltK2NLeDZoY0k1T3dIQm56?=
 =?utf-8?B?anhSd0dpVkF6dVFvR3FaS3ZYb3JDZ09ZaG8vc0pGNExNM0JTRjhsMUxobW1r?=
 =?utf-8?B?U2VKdXhqWmRZZG5vY0lkM3hpV1NTbnJWb0xLK3pRYkIzUmJFMTJnR0E1Vmpm?=
 =?utf-8?B?ODVMZ0tqR1hqWCtpT0d1UzgvaVltUmtadTlDclJjNkROMVZYT3k5YkxVSE9s?=
 =?utf-8?B?KzJ2UkI5aVFQN2NoVDQ0TXJIUXRQSG9OZUF6NTlvZGs4UTZsYk4xT1lMM1lu?=
 =?utf-8?B?cWF2SCtNeitHWUhTRGo3dTFRK002RUJ3K1hrbUVpVVg1SUc3TmpSVjVDbEwv?=
 =?utf-8?B?dFBFY2FCMzFsTjc2YXN4NGE1OGNjck0reHA5Ym9pWExHdDg1OCtyOHNyckgr?=
 =?utf-8?B?cnczSDc0Tmg4K1dUR0lTVW52R1VOVGQ0SmlBWmVRdXEwT3huN2RqNVNLOUND?=
 =?utf-8?B?S2FWZGx0T0MrZUR3MTlQY2FTVG96RVVTVGFVRWVlUy8wVmY4VUpxZHd5cTAz?=
 =?utf-8?B?MWVCYjEyNERPb1lTYWZSam94ZzRDUWliT09ldVZVY2w0WDVwbWxSYWlDaFF2?=
 =?utf-8?B?UXZsUDU2NkpBQ2cwenNobHBIWUxMbWtHSjB1RTR0ellKczlzTWJ2Vk4ybW9E?=
 =?utf-8?B?M2RYUTgvK3NBd0xsK296MjhaaDV5RTFGY25UQkZFTnhsWTZOQ3VWSi9sY2RH?=
 =?utf-8?B?d2dYdkVIWXZyc1B6ako2K21xL1g4b0V2aFdHcnNlR0g3RGlBdkVOclRwdGpV?=
 =?utf-8?B?eXN3ejJUYVhGNW1GYzhwNVJKc3VGNFBlRFF0VVZJYVpTV2VvcW5OejRvOTVz?=
 =?utf-8?B?RHFCNUphTzQ2ajRQZHRaMGtjVk9tdTlHOWcvNFNvS0RRc1FSTU5IRjY2Tytp?=
 =?utf-8?B?cDdSM1pwb25IUTNvY2hhbG5KMmVTRyszM0dUaTM5eWJRQU5jdDE4enROQkx3?=
 =?utf-8?B?Wjg0Zm80eEo1K0NsSFUxU0dXTk9Kc0tJeHEzYVBWOXpkN1FZUXV1Q0dDc1JD?=
 =?utf-8?B?WEdHaU4xUlJyK3BHNCt0ZWh0MS8rMVpxajhEUEJNN2lsekx6S256UkdMcGt5?=
 =?utf-8?B?RU5TWWRsdU4ranllSGtBMXBRUCtTUDN2QW53YkpFelNBMWZ2ZlVBWkJ3dkRN?=
 =?utf-8?B?b1Rwam1qenllUzNkcGQyNURKL0pnaHQyNlpLcXg4aHpUYmdCSWJRQzVEQ2sy?=
 =?utf-8?B?MWtNK0lFa0FoemFuZ08xQ1NVVWxTczVsQ29GRkVkRHpaRDJmOFNTU2R3R0Yy?=
 =?utf-8?B?R3ptaUlwZ0srK1UrZTZIV1c4TTczY05KS015T0FwQWdTMlhndXRBdmdGb2xR?=
 =?utf-8?Q?fEGc+zNHTy+4e+J5cvdBnBIvnuxHBq5CNudutHy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f68737fd-5835-47d5-2266-08d979d49712
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 12:13:42.5800
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: P+7WAxKrA4797zcxk4x+UZtuYo2rluw3h6aBNLTwMpXQkUluO5X7fMMQmHCcvIycexJHST5PMtGrzOAnsWxHwQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3392

On 10.09.2021 22:13, Daniel P. Smith wrote:
> Hidden behind macro magic is an alternative xsm hook interface dedicated for
> use when the dummy/default policy is the only one built. This alternative
> interface increases code complexity and code size in the core security
> framework of Xen.

What does "code size" refer to here? I would have expected the binaries to
be smaller with inline functions in use. And ...

> ---
>  xen/include/xsm/xsm-core.h |  51 ++-
>  xen/include/xsm/xsm.h      | 275 ++++++++++------
>  xen/xsm/Makefile           |   2 +-
>  xen/xsm/dummy.c            |   2 +-
>  xen/xsm/dummy.h            | 659 +++++++++++++++++++++++++++++++++++++
>  xen/xsm/silo.c             |   2 +-
>  xen/xsm/xsm_core.c         |   4 -
>  7 files changed, 868 insertions(+), 127 deletions(-)
>  create mode 100644 xen/xsm/dummy.h

... the statistics here suggest an increase in source code size.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 12:58:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 12:58:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189368.339025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDS3-0006mQ-3r; Fri, 17 Sep 2021 12:58:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189368.339025; Fri, 17 Sep 2021 12:58:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDS3-0006mJ-0I; Fri, 17 Sep 2021 12:58:47 +0000
Received: by outflank-mailman (input) for mailman id 189368;
 Fri, 17 Sep 2021 12:58:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRDS2-0006mD-CY
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 12:58:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fce17194-17b6-11ec-b697-12813bfff9fa;
 Fri, 17 Sep 2021 12:58:44 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-oqh7d6_nMy6YG1wRPXOCbQ-2; Fri, 17 Sep 2021 14:58:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2701.eurprd04.prod.outlook.com (2603:10a6:800:af::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 12:58:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 12:58:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0017.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Fri, 17 Sep 2021 12:58:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fce17194-17b6-11ec-b697-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631883523;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=RONU+ejaKqP7oeNza1QQPRqupv8noL3rm/46uCuwdSk=;
	b=MKDo8d1oRjK6WaQ0DOKgqksK5cJHMny4uooFgNXETFW9PWK9d6OKdUEL+NyKZjy4smBSEo
	h4OpRDjjuxF1JoimWmQi+R6JMLAZ6jz1hvdRm3Cr0c/8MAJa5FGV2NimfWJfPgQqaT5GdE
	ZPZWPaofrtT3/zaRPzFtjIjzXxNYEa0=
X-MC-Unique: oqh7d6_nMy6YG1wRPXOCbQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XFEHw5XkJFcmpvX+NCQwd/9y1/kIrZbVzbNy3D7Woj9wQ3QYAdPHJsGjdkJ0sKg6QI5Jge7/m0J28vBMKfO7O8Cmn5TwWzy+GS4psUtalAKvvViDXxGSnkDHFxoagnBgGEHTKSqHDXr888qGNjdKaNmphpGV8Qm3FVbkZGLZBjcMhvVs1Oah6yJdloTMCK4RQnqW4eKfCymW3npndfDCgtOS6wAe7a17NCVdoddAN+EcRZmHdPiYKGEllxyFRskS+VBY+MYKybMbfWYrQ0dxeAjN/jo9kIKGpRzAkg0bMTAWUs1yC51tiMPHETRoOxSPT9tovtYocD+RDlP2d+gcqw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=RONU+ejaKqP7oeNza1QQPRqupv8noL3rm/46uCuwdSk=;
 b=FY2uJA4cYtf9cXamrfAHu3QMOYpT746e86WlxSDPA2AjpjjWrBSIg2TAsvaLFnKZredyinfAtUUnuDvKCgFPAVnY8YfLH7h2es8SS8tAJaxKkv/MP3iY7/PCS4E8yVioRWgROS+80+tmrjy3g/2699O2gojQlVvuMu1SK4tWedIVt5PV4HW95hQDL2R91q8QmxqDzSc3VxB0Mx24bs7A8sbwPbRlRDfHsR23WzG+vLXv+X+A6bJxCSkHfejyurtfbf9OxM1ZVhM51bp6+DIqSHJgmC7BNuwuDrhuW+i/1V147BGMA5gKZhdT7VCJu+RuDxNbq1Sc1qpLo62wfgSYaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/6] xen/trace: Don't over-read trace objects
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-2-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ed07d2de-9354-6c6d-b9f6-4b325ca328ce@suse.com>
Date: Fri, 17 Sep 2021 14:58:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0017.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 36c7304f-63bf-45d1-aef6-08d979daddc4
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2701:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2701E861CD7C93499139375CB3DD9@VI1PR0402MB2701.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XM5JxTuwDEsL5himI1xL428sYXFotQpMjib8xLaKV6SddYxsspVsxWEgmYfvaetwZ9A3e+tji78c5NYCr9opjuuYZn5gSDk7DDeKE4HgqeqEXtimcpgZqtEF/+bbbItc3QNumfFEiMDQbgEHct4K1AnofhWu9BBVTiKTLR8pD4A1OpF6qbYnOuX/QTlRT0S/HZR96Zi1eQHJF3IHzNws9bhv1mIBl07tnc+Dl0Z3YNcagJprQbYX+pGMP+B8wAm2dHPIDUSyjcdn9Jo43sjuDlnHIqrxmnZh55Ng0WWjPFEfdZ2tWwv+ysViVpn01ObCaivpvcr3rbUAsV/PcrOAb2ChoO3SrvPEqxxKHBa58VSsuohhr65AKU0txSnoy0G131GPHKyazoWQYuSJGIlcvSSUgRwd5YlJs3CuJgT5jTmmsVg5GJy9UsQnkzYwqQ9p9xuhjBCoRFokRgWPKI1AyfCFpwD13+xkEVIdl2mpeoKRwmGqQdiEMDOMZTZun2Wd7OdAl/pc2sYbOVk55xsqC9XeiWH7Fy7g6CqQdxUfIKIIE9aWyjP32383zf1Z86GbqgudNQBCildq0w8Xa2dXf6pmU2E5GhGZugZPb6ZSDLt8PFCAQqs4aXq4d8YecHqs74FGtRKO3rJjQARtQKOvavON53RtRWHhfZaZeeX06CNmQLEw7GkjN0vEAA6gfkP0ZqY/PccCF81xyJwJroXJhrTHqB/ZDP+1CSfNcyFMPw0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(16576012)(36756003)(31686004)(508600001)(4326008)(86362001)(54906003)(66476007)(6916009)(53546011)(6486002)(956004)(8676002)(5660300002)(2616005)(83380400001)(66556008)(66946007)(8936002)(2906002)(31696002)(26005)(186003)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Vytna3dUdWp4QWVsalR6SVg4R2pnMHluTUtzcVVKdzVvTTZnZlQ2R1RFTjY1?=
 =?utf-8?B?WTE1T0lEa3lvdVBlc0lDZm5KUE5TZG11SjI2MFNsWG9Pai9oandOTXVQaExS?=
 =?utf-8?B?TVZhZFpDR3pxM29Qb2RubGpKN0huL2FkUFlUVXFpaDIxL0JkaFVnV3VEWnV4?=
 =?utf-8?B?SnRDc3VxS3F4djJUcFBEQ2dWSkFmNHdYVmZCOHZKWkNxT0FwRmpHSjJaMXJa?=
 =?utf-8?B?NXNXLzc2aHZXM2hLSFF4N3FJQmdUL2dkaWRRNXBZenR6U2RRa2F2akNRVVRk?=
 =?utf-8?B?MGx1cC9GdHNrVFBtd2NvdVpFQWhTUWZYK2JTajF3cHlTWWJBd1BWSUlpOVl5?=
 =?utf-8?B?UEQxMWJlSWVtSXorTlNBRGxMUkFYWVlBVzJwZlI3bENxVlF2cEtpVjUrZGFm?=
 =?utf-8?B?RlBKUDNCWjlnTURTTXBXQUcrWWxDdWliL2Y2UHBRWjBmK012UzBzY09jcW1X?=
 =?utf-8?B?akR0TXFKY0dDS1dKVlNmaXVKYzJJOGtEajMzRmpJTkFaR3hXV01ydEdPbURW?=
 =?utf-8?B?TkVyZmxicy92aW9QMHBGSUFoOWoxdzV6Y1Q1WWRzSjdiaGN1RnJmaDdNNG54?=
 =?utf-8?B?ZmpNS0hQNGlLRFpmdmJ1SUxjT1p2REtsSWhaaC9Td291eXZkaldIc2pzaGhi?=
 =?utf-8?B?a2NLZGR6V05sQ1dUZndKaFJpaWRiS2huOWgrZExRWDc2UHVSL3pyZnk4UHlE?=
 =?utf-8?B?dGRYbW8xQXdxYXo4amtLMm5md0kvVXVVUDB1ZzM3SW9LaUJPTm9ySHNoLy9j?=
 =?utf-8?B?emZMbnlRdHNvdWliRGwxSjdMeEZSYU5kWmcybjhkcTZFdG5pMk9kakF0NWF5?=
 =?utf-8?B?TGRGT3dlZzdQNUgrUFhWUitIcWhEclNzNUJxY1NoYTBDZlRVeHlVVDQ1Y2pK?=
 =?utf-8?B?ZDZ2dUdlUFdVWnVlQVhCREgzeE9BZnVQU1JicjlNRDl0L1FCNXNUS0ZpM1Vv?=
 =?utf-8?B?aFhHc0xYdXdNRDl4ajBpZ25pcnRRTGZzMVBLdHhmNVZUakFMQkplZUJZbkQz?=
 =?utf-8?B?aUtBQjc3ZlNid3RWUDYvcVhTN0tMakdMc3d2VUNPa2J5UVVHTzhWNE1yNFJa?=
 =?utf-8?B?L1hGMGtrRE15ZnpNdXVmL05VbXlrUEQxVUVhQWJrVzhUT1BRMS9yRzlnU2dY?=
 =?utf-8?B?emxQcHI2TGJhUjd6VC9IcUZFNURibWpROStFVENSN3JYdGRPVnI0cTIzVTlD?=
 =?utf-8?B?cis0TWJmbUxGcFloUmhQMkNWdnRxSUd0UHlHTHUvVGxXL25GNlQ3Z3VVbTh3?=
 =?utf-8?B?YnQ0bHFieDNGTTR6U0JkSzQ2c0ZxcGQwanNNVEJ5V0tjbkl3WkVuK1pUS0do?=
 =?utf-8?B?WC9YL2Y3SmdpdTdMVEQxVHVnM2k1K0NsQlJHRjJLUk81S0hzeTVTamNWZndn?=
 =?utf-8?B?MVQrWXZIL3k1ZnMyMzUrR1lscUdlSmxHU1VLRmswSmJaSkVjTHYzek9ZcW5R?=
 =?utf-8?B?OVh1RGlJTVkxVGFSTFhXZzJYMXZBTkg0cW91bHpFYk56YXNWWTRYM1ZSYUc5?=
 =?utf-8?B?Si9VRzcrMm1kZXlZNm5RL2pmaTFDV2VnM2tWaFFseXdMMnp3QkNEdWFwdDZI?=
 =?utf-8?B?UytLMll4eDBSNDhzdi9PbjkxcGNLQk9pTHVQc1haWjRVcTlHWTh1MXc0MWll?=
 =?utf-8?B?MHRHMG5PY1owVGRVbjlNWVBPQTBNTnlVUzJjSTdLYTBqVEVQSHd3UkZBbUgz?=
 =?utf-8?B?YTBTRzgrZmNZSFRnaU9VNVJDQTY0c2xkSkNmRURWcm8vRVRDMGVydm15c2VY?=
 =?utf-8?Q?Ovp5865mM57BFSS6CRSKG6Ircpxd3JiEIcp3ZiD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 36c7304f-63bf-45d1-aef6-08d979daddc4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 12:58:38.1391
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dMMptEYsm/4qK2uJMERSPawaE1qODn72iJ2l6xtXEeWT3xkP/FuRqA3VTIOXY6oWUS7+dFxDfJq+kLstH1BStg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2701

On 17.09.2021 10:45, Andrew Cooper wrote:
> --- a/xen/common/trace.c
> +++ b/xen/common/trace.c
> @@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
>      unsigned long flags;
>      u32 bytes_to_tail, bytes_to_wrap;
>      unsigned int rec_size, total_size;
> -    unsigned int extra_word;
>      bool_t started_below_highwater;
>  
>      if( !tb_init_done )
>          return;
>  
> -    /* Convert byte count into word count, rounding up */
> -    extra_word = (extra / sizeof(u32));
> -    if ( (extra % sizeof(u32)) != 0 )
> -        extra_word++;
> -    
> -    ASSERT(extra_word <= TRACE_EXTRA_MAX);
> -    extra_word = min_t(int, extra_word, TRACE_EXTRA_MAX);
> -
> -    /* Round size up to nearest word */
> -    extra = extra_word * sizeof(u32);
> +    /*
> +     * Trace records require extra data which is an exact multiple of
> +     * uint32_t.  Reject out-of-spec records.  Any failure here is an error in
> +     * the caller.
> +     */

Hmm, is "require" accurate? They may very well come without extra data
afaics.

> +    if ( extra % sizeof(uint32_t) ||
> +         extra / sizeof(uint32_t) > TRACE_EXTRA_MAX )
> +        return printk_once(XENLOG_WARNING
> +                           "Trace event %#x bad size %u, discarding\n",
> +                           event, extra);

Any HVM guest looks to be able to trivially trigger this log message
(via HVMOP_xentrace), thus pointing out an issue in a guest and hiding
any other Xen related output. I'd like to suggest to adjust that call
site in prereq patch (I'm not overly fussed which of the two relatively
obvious ways).

Further sched/rt.c:burn_budget() has a bool field last in a packed
struct, yielding a sizeof() that's not a multiple of 4. All the uses of
__packed there look at best suspicious anyway.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:01:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:01:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189378.339044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDUl-0008HW-Py; Fri, 17 Sep 2021 13:01:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189378.339044; Fri, 17 Sep 2021 13:01:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDUl-0008HP-N1; Fri, 17 Sep 2021 13:01:35 +0000
Received: by outflank-mailman (input) for mailman id 189378;
 Fri, 17 Sep 2021 13:01:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wkJT=OH=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mRDUk-0008HJ-LM
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:01:34 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1b639384-70fb-4b3f-ba95-c3536fe5a9e2;
 Fri, 17 Sep 2021 13:01:33 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id b15so14135591lfe.7
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 06:01:33 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id y14sm695484ljm.39.2021.09.17.06.01.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 Sep 2021 06:01:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b639384-70fb-4b3f-ba95-c3536fe5a9e2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=nOUOG75GOz571OX8ITuOEpvSRHg2zXkMyJh97BY3RUY=;
        b=DVleVrCvRAdqZp/yPatEk2JqDQdaCKPV9hMsJSSPCFvPJISePzzWa1fFVmSDK1kOSN
         HFnXNvK5yZkZKMBnLx7wvAnaenTIDGX+D5BBcT6f3Tkvt94WxReAAvpyt+33Xk1wbWcZ
         +9UMsH9pH9VW2JbRgJ0yOUq76NKVQbWOrDwizNp3L7vBUKg946XYIhg6q3vkKUuRWOV9
         a4Wcm6Wcw2a331LlEt2w22YGNIVJEhzato59jlml/xrk0txWGTigockGiwIpb0FL9STt
         t2rk0+HI11YBMYieGy2HZKw8yoadOCM9rSbc/N8GIeh5mjT+SgN2KTinZiTPniMBHthj
         cw1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=nOUOG75GOz571OX8ITuOEpvSRHg2zXkMyJh97BY3RUY=;
        b=NAdOt/EnYA+H0DWJJqIcMdDHySiQxiGDi9C7ubQ+KvDR4nNluXpn60PuOOEF7iqtBn
         B4fVH+OF3+Hbv/OOkQIpT9CGqF/HMhrrl+bTv1iB35aDfqm7nsHx5g4BUV6/rwYR88xG
         hXc4IExmcYRs2M+JdFtg/HSViG2uEi/OGS992xa5Xdvd4Xwe7Op5svQYoC/CI5zTgqlf
         TZDdiJv2TRmdugYJsZ3k7U9KAf8pa8VdyybOKypolR8anBVEKlnj4ug2hfTxXhG9+w3l
         zYV6ogPZV6CIs/R9SSIZm6BvDY/fZnRxrcDACGnwmeilemAgpu4uVPMMuCPpYliBTE+M
         8dTQ==
X-Gm-Message-State: AOAM531Epfu3leHcNBcC2j4SFIdSZDEPC4gUIzmP3r4vf7DrniE6Upn6
	4xvBXeqJNhA8qk1ntwYbDOY3rZlgXQouDA==
X-Google-Smtp-Source: ABdhPJxvPqjCvYrNlgvVOuQYRuzxkAEBoERlUud1NqfPmVJFcqleK6VZh409vbqQY0ZNiqLipkcD8Q==
X-Received: by 2002:a2e:a37a:: with SMTP id i26mr9180044ljn.466.1631883686553;
        Fri, 17 Sep 2021 06:01:26 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: [PATCH] xen-pciback: allow compiling on other archs than x86
Date: Fri, 17 Sep 2021 16:01:23 +0300
Message-Id: <20210917130123.1764493-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen-pciback driver was designed to be built for x86 only. But it
can also be used by other architectures, e.g. Arm.
Re-structure the driver in a way that it can be built for other
platforms as well.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

---
Tested on Arm and x86.
---
 arch/x86/include/asm/xen/pci.h     | 24 ----------
 arch/x86/pci/xen.c                 | 74 +----------------------------
 drivers/xen/Kconfig                |  2 +-
 drivers/xen/events/events_base.c   |  1 +
 drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c |  3 +-
 drivers/xen/xen-pciback/xenbus.c   |  2 +-
 include/xen/pci.h                  | 34 ++++++++++++++
 8 files changed, 115 insertions(+), 100 deletions(-)
 create mode 100644 include/xen/pci.h

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 3506d8c598c1..9ff7b49bca08 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -14,30 +14,6 @@ static inline int pci_xen_hvm_init(void)
 	return -1;
 }
 #endif
-#if defined(CONFIG_XEN_DOM0)
-int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
-#else
-static inline int __init pci_xen_initial_domain(void)
-{
-	return -1;
-}
-static inline int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-static inline int xen_register_device_domain_owner(struct pci_dev *dev,
-						   uint16_t domain)
-{
-	return -1;
-}
-static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-#endif
 
 #if defined(CONFIG_PCI_MSI)
 #if defined(CONFIG_PCI_XEN)
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 3d41a09c2c14..4a45b0bf9ae4 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -23,6 +23,7 @@
 
 #include <xen/features.h>
 #include <xen/events.h>
+#include <xen/pci.h>
 #include <asm/xen/pci.h>
 #include <asm/xen/cpuid.h>
 #include <asm/apic.h>
@@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
-
-struct xen_device_domain_owner {
-	domid_t domain;
-	struct pci_dev *dev;
-	struct list_head list;
-};
-
-static DEFINE_SPINLOCK(dev_domain_list_spinlock);
-static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
-
-static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	list_for_each_entry(owner, &dev_domain_list, list) {
-		if (owner->dev == dev)
-			return owner;
-	}
-	return NULL;
-}
-
-int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-	int domain = -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (owner)
-		domain = owner->domain;
-	spin_unlock(&dev_domain_list_spinlock);
-	return domain;
-}
-EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
-
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
-{
-	struct xen_device_domain_owner *owner;
-
-	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
-	if (!owner)
-		return -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	if (find_device(dev)) {
-		spin_unlock(&dev_domain_list_spinlock);
-		kfree(owner);
-		return -EEXIST;
-	}
-	owner->domain = domain;
-	owner->dev = dev;
-	list_add_tail(&owner->list, &dev_domain_list);
-	spin_unlock(&dev_domain_list_spinlock);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
-
-int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (!owner) {
-		spin_unlock(&dev_domain_list_spinlock);
-		return -ENODEV;
-	}
-	list_del(&owner->list);
-	spin_unlock(&dev_domain_list_spinlock);
-	kfree(owner);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
 #endif
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index a37eb52fb401..057ddf61ef61 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -182,7 +182,7 @@ config SWIOTLB_XEN
 
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
-	depends on PCI && X86 && XEN
+	depends on PCI && XEN
 	depends on XEN_BACKEND
 	default m
 	help
diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index a78704ae3618..35493ff0d146 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -65,6 +65,7 @@
 #include <xen/interface/vcpu.h>
 #include <xen/xenbus.h>
 #include <asm/hw_irq.h>
+#include <xen/pci.h>
 
 #include "events_internal.h"
 
diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
index 224df03ce42e..fc8c1249d49f 100644
--- a/drivers/xen/pci.c
+++ b/drivers/xen/pci.c
@@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_XEN_DOM0
+struct xen_device_domain_owner {
+	domid_t domain;
+	struct pci_dev *dev;
+	struct list_head list;
+};
+
+static DEFINE_SPINLOCK(dev_domain_list_spinlock);
+static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
+
+static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	list_for_each_entry(owner, &dev_domain_list, list) {
+		if (owner->dev == dev)
+			return owner;
+	}
+	return NULL;
+}
+
+int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+	int domain = -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (owner)
+		domain = owner->domain;
+	spin_unlock(&dev_domain_list_spinlock);
+	return domain;
+}
+EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
+
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
+{
+	struct xen_device_domain_owner *owner;
+
+	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
+	if (!owner)
+		return -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	if (find_device(dev)) {
+		spin_unlock(&dev_domain_list_spinlock);
+		kfree(owner);
+		return -EEXIST;
+	}
+	owner->domain = domain;
+	owner->dev = dev;
+	list_add_tail(&owner->list, &dev_domain_list);
+	spin_unlock(&dev_domain_list_spinlock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
+
+int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (!owner) {
+		spin_unlock(&dev_domain_list_spinlock);
+		return -ENODEV;
+	}
+	list_del(&owner->list);
+	spin_unlock(&dev_domain_list_spinlock);
+	kfree(owner);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
+#endif
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index f8e4faa96ad6..bba527620507 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -19,7 +19,8 @@
 #include <linux/sched.h>
 #include <linux/atomic.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
+#include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index c09c7ebd6968..da34ce85dc88 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -14,7 +14,7 @@
 #include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
diff --git a/include/xen/pci.h b/include/xen/pci.h
new file mode 100644
index 000000000000..5c988d5ff38f
--- /dev/null
+++ b/include/xen/pci.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __XEN_PCI_H__
+#define __XEN_PCI_H__
+
+#if defined(CONFIG_XEN_DOM0)
+int __init pci_xen_initial_domain(void);
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
+static inline int __init pci_xen_initial_domain(void)
+{
+	return -1;
+}
+
+static inline int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+
+static inline int xen_register_device_domain_owner(struct pci_dev *dev,
+						   uint16_t domain)
+{
+	return -1;
+}
+
+static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+#endif
+
+#endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:04:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:04:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189385.339055 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDXp-0000VC-9m; Fri, 17 Sep 2021 13:04:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189385.339055; Fri, 17 Sep 2021 13:04:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDXp-0000V5-67; Fri, 17 Sep 2021 13:04:45 +0000
Received: by outflank-mailman (input) for mailman id 189385;
 Fri, 17 Sep 2021 13:04:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRDXn-0000Uz-Pi
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:04:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d2713420-17b7-11ec-b697-12813bfff9fa;
 Fri, 17 Sep 2021 13:04:42 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-k3N5bST_Ocivc0jqqI1RFQ-1; Fri, 17 Sep 2021 15:04:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 17 Sep
 2021 13:04:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 13:04:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0111.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 17 Sep 2021 13:04:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2713420-17b7-11ec-b697-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631883881;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7fLgZmeBSwiV+NTbSMwgYaCPjruwQAJUnYxmLKIu9ug=;
	b=dSrXto/wpEXuoRIefclEHsqoWdxlOd1A9LVik0NrdqA6ppPiu9XBPHxAlvHSLJjSks+Xg+
	RdxWb01ENP97y/eRg3Zn3+lJ4grh5ob6QijmvMvioTlF5gGGBcy8leUhEYrfoLobRqmJeM
	8zqHZ1lhj7tNlNG+D+BaIYZvKSLnxJw=
X-MC-Unique: k3N5bST_Ocivc0jqqI1RFQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nqN4ryCrainoNuzHqrMWFVqtHazMpMvMP0Od5Mir7XoVqqDnExWILcnoiWWqKuWFhjtPgyITyVY1pS8IoxvwF5/4iPoEK8IR2dpANPt7dtMQVFSJpAEy0IULNnoK+YgP6JeA4/AOjkQhOc8dZbNYdhVH6bBkLwnYFLfDB2Y+rhiTJdHqb8mrkcEv7nudq+HhtON0Zg2p5PVdIF5jq5WQOZu+r4YVdl1VcvrOi4DYm6RLgqetIiekgX7FKdKKWFuqEicroo6UiJfgTIXplpeE4w8ojOO4dEVV2fsDVCf05quqoDUiDZfPa5Jijv4DJYzUctdVxKPmeR9cpHiiNs2inA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7fLgZmeBSwiV+NTbSMwgYaCPjruwQAJUnYxmLKIu9ug=;
 b=A0Cvv3UU3w/fP1uzNYApHH4LtvG3Fj+Y4y9K4BtXNZOnlM4tOKWXOa9Kj7bHRHY8WqBl6m17FwiLCdpd8E6tBsm49wwbKWlmao3AVJvtj14uOsPKJL1IhthLNjyqlpLogaxGVQ3yEUxj/3qec8EdF5ppoQAjuUAhHkYudHeiF9N2M+tmMFuGmgO8U4QRMCEDNfoAxamRNYBHwGZbBotLYYjBgr+GExFeMJqFw+UHJu/qtSStPy6xMsFjHQMeaUNjIf+jkQ16amoKgtGtULs2geQcueAEkwD1/kcyc4Qt18qnv3q4GMOGh/tF7pAthNghU3SSp/Gx452whfoj5g0A0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 2/6] xen/memory: Remove tail padding from TRC_MEM_*
 records
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <24fa2834-95e5-11fb-e522-a37e3d8356b3@suse.com>
Date: Fri, 17 Sep 2021 15:04:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0111.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 807b2927-8aaf-4f5d-563d-08d979dbb4bd
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB295914C68D39B43D4A08D270B3DD9@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pnNg1ddcfII4WAu3PO7D7ysjsCh06hhhtPfTCNszwPI6Gqg0NZqgKDIMTn9K+a9Ul5zdZhLgtD83avPNHNzTvfwi7Ay3xMGvRsxsKjzLWtgE/409OEA6az68sahba/xqDJQXNoYe6s2mTCa6JqebEpCI/gEYfQBvIqd661LyAsbkHdhAMLKAwix7vQTcCMDpHsHk0Ad1Uh6eg+FrP1WpzyqjHZ7jWVi1iOILtXmeDpC1H7AY9cB992Prt+gP2lMI4HwNU9CFuG7Q3A3FcaWI51U5kQIXuxtoHCWoKYzYwVy73dsh5ns3FjdRH5py7oyyTJPVSZbzNOyoikXqiocCdsudU/mFjT+fMY5ptHGUpfVW//s5x7M4+tx/WMnkbgggDsfLSPO0toul/7Armd/GlrdXMSdDNBM4EfJmo1fl1MvSJUybWUv6JjfAguFVbAP6OUyKb4onXXELba8UdFqacPgSfg3/W23VAjsudIBhjPNxP3qbrvEdvfeB7VXL0fyYt+IWqt8s7wvO50rP050r26yVEA/wtpIVBOYOe4yPsJm/BPSIcUoBV2hNJDOiTtSz8Xs9oDfwztcHN4mHvpQ1AU+2x9zMbdA/wILSmuNp+RiIy8Q4XTTsMO/visroMsAjFMj2NwE7FacR6Y/L/U80vGNUz15qX2bt1kg8tKKaLFBxk3T6/tTzGt0FfclfwwhA+tZaX2Kpzp9Bcl0WHYiNRbv3sAHVgTKvttb02lSX4d8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(66556008)(2906002)(66476007)(6486002)(956004)(31686004)(186003)(66946007)(8676002)(31696002)(508600001)(4326008)(16576012)(2616005)(4744005)(83380400001)(6916009)(38100700002)(36756003)(26005)(53546011)(5660300002)(86362001)(316002)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cUZpa2hBdzV2Sm5CSHFMMGkwTk9xUFNsTmZUOEduWDBOTnFQbCtHSFFvQy9z?=
 =?utf-8?B?KyszcUhWN241RlhkaSt4KzBNZnpBWTN0SGNWZ1g4YlZKTkEvN3NKTlZQRHlZ?=
 =?utf-8?B?MGZMb29uamdKV0wvMUtDcnhLWDIxemVwQ3JGbHpYVG9qOGF5b1pPQ1p2ZHNi?=
 =?utf-8?B?ZGlFYm4zMk1hOHpidjE3YmlaVGM0cnMxZENwTFhXZURJd3ZkS1RGMWd2S2xS?=
 =?utf-8?B?WGNaeVJWT21XbXNWbnF6WUtKK01RWUdXSDFQb1MzcWlhU1l5SEpIMVpJTitJ?=
 =?utf-8?B?a2thKzdpOVhJQjVzWFd6bFpCWStpaVZBTEpXSEhySU5ZYzlQZldWak5VMG9U?=
 =?utf-8?B?bFhIMmlXRWc2dGlYTy9LbExKQkk5OHRzUDhsOVNWb2pGZmFIb0tJeU9qUU9a?=
 =?utf-8?B?dnhkQ2hrSC91MVhQVmo4Z29OTUZ3V1dUbXplemx2THMrc3VoM2NtZlMvZGRE?=
 =?utf-8?B?MTgzKzV6THd5b2YrUGwxWUlOVElycXlFM2J4VFRVUDdmQjNBY0RnY2ZmeE52?=
 =?utf-8?B?UE5XUEF3bW1yaGZoa2xQQnY3VTFoUGp5ZVZrTXpETjM5UG1VdC9PLzZJeVBl?=
 =?utf-8?B?a2s3OW5ZWXV0NjhhcUdyZnp0Y0JWaVBYVWlxZXBMbFZkYlFQZzZsc25pS0wr?=
 =?utf-8?B?cEdXQ24wVU1ZZ29qVDlLT2kzazRROUtZNXRlbmJvMGJMQVVQRElFSlF1aENt?=
 =?utf-8?B?ZHBQazRwRERoNk5aUXNVS1JoZG1zbXNzbjF0SmowOXFBaytIRE5TZ01yQk8v?=
 =?utf-8?B?SEdjTTh0T1p4ZWhuOUtiK0hvRHFEVDNKZ0F0ZEV1dndIRmJJN1FFSFpsUkwy?=
 =?utf-8?B?UFRFeUs1Sm9UZXhDOGZOSW9ZdHRNcExlb3RNRWxtSjFKZ3lOa2pvYnE3c1U4?=
 =?utf-8?B?enJNZkdMV0JZdmJZcFp4ME4ydHl4NHFQQnNHbGw0a1h2UTE5V2ZHdU9kOVRi?=
 =?utf-8?B?T2N3M1dpdjEwNmxnYUhScEtIMGVmZy9UUVdPbXcrRVR4dStUOXNIVVNaZTVz?=
 =?utf-8?B?eWh3K25XZDVxeHpyWDJ3TURFSlYzZW5ZU29sYk1hU0VqblE5NitVb25zWVpJ?=
 =?utf-8?B?T25KQ1NRNzlyUlU1bG02ZVlDa3dLdFBrUzdQcTFXVk9NUkEwRW5aNS9BSlhr?=
 =?utf-8?B?M0NhZVdLTTkvQWUzZExTcXFpNDBwK1g4TWJNaG9yRFBiZHIwb2NVV08zTGNB?=
 =?utf-8?B?QW92UnAvTVFWREdBTmVCZFVFQ1g2dmhEZ1BVUmMxS0xxemN1c2t5YnVBcUNo?=
 =?utf-8?B?WFRiTFNRR2JJZ3Eya1BRUVgxb2t3ZURONDNXVWs1TW9lVjRKVnhjWldSTDNp?=
 =?utf-8?B?NlB0NkprNFdQenIwM2p2M0RyREhhZWNvYThoNlhEenN1cE9QdlhDa0ZPcUpI?=
 =?utf-8?B?Mi96d0cvenk0VEMxUS8zajlyTStyb216bittMEF6ck9KQTU4Myt3VGdMRjBv?=
 =?utf-8?B?c21vMFBCaTh4djRTYWRoMVVNZGdsTTUvUGwyT1E1eHhQZjBQcld3akx4YnM5?=
 =?utf-8?B?aGVwTEs1VEJaQ1RpdWFxbU5zTG5YVmU3ek5XY2xXd3hvY2tLNk53eWxobHZN?=
 =?utf-8?B?eEE3eVh0ZjZtQVA0V0dLaVJSN1NJUHhvS05xTmFrS0k0WDZxSDlURWJTTDVY?=
 =?utf-8?B?SDBVVk04N0duSVdjK2pxSEtXQ1RVZnVZK2tib3VPNkI1V1F6dk85MTlhTzFF?=
 =?utf-8?B?T2JueHBRdFdiSWsxWHlZWUtUWmdMZWVYTkhScDY3cVFYbWRWdEUyRDNQMnpF?=
 =?utf-8?Q?HgGeu0J7B5ho5i6ftEEzXoWOXKKWJBUznihTWOo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 807b2927-8aaf-4f5d-563d-08d979dbb4bd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 13:04:38.8041
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: c0vAFjcMUBnQDN6EKbHI3ICF+kK00iCWPI07Ripu8wQC6tdVzjRPfAZgjZcx9wHwOtjlbQlrz0AsRHL5E+fowg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

On 17.09.2021 10:45, Andrew Cooper wrote:
> Four TRC_MEM_* records supply custom structures with tail padding, leaking
> stack rubble into the trace buffer.  Three of the records were fine in 32-bit
> builds of Xen, due to the relaxed alignment of 64-bit integers, but
> POD_SUPERPAGE_SPLITER was broken right from the outset.
> 
> We could pack the datastructures to remove the padding, but xentrace_format
> has no way of rendering the upper half of a 16-bit field.  Instead, expand all
> 16-bit fields to 32-bit.
> 
> For POD_SUPERPAGE_SPLINTER, introduce an order field as it is relevant
> information, and to matche DECREASE_RESERVATION, and so it doesn't require a
> __packed attribute to drop tail padding.
> 
> Update xenalyze's structures to match, and introduce xentrace_format rendering
> which was absent previously.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Hypervisor part
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:11:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:11:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189392.339066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDe0-00025u-0V; Fri, 17 Sep 2021 13:11:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189392.339066; Fri, 17 Sep 2021 13:11:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDdz-00025n-TS; Fri, 17 Sep 2021 13:11:07 +0000
Received: by outflank-mailman (input) for mailman id 189392;
 Fri, 17 Sep 2021 13:11:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fBbp=OH=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mRDdy-00025h-7c
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:11:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc783ba0-7ac3-4c65-8898-406d379d9fe5;
 Fri, 17 Sep 2021 13:11:05 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-RnlRYklbPTiK5mn8xVy6CQ-1; Fri, 17 Sep 2021 15:11:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5904.eurprd04.prod.outlook.com (2603:10a6:803:e6::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 13:11:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.016; Fri, 17 Sep 2021
 13:11:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0008.eurprd01.prod.exchangelabs.com (2603:10a6:102::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 17 Sep 2021 13:11:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc783ba0-7ac3-4c65-8898-406d379d9fe5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1631884264;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LghKfE4iKbQSz30plE0IC+HIWMwQiUkQ5z32wjJKUwk=;
	b=XexzhsAfZRbHEDa5gtIg7q+tApXXlFOGhDHD2IqMiAziYKp1yr4Rl0sc1SBY51I5p1PUPf
	JX/N3zkPHPdLzCZM09DGZY1nr65RU+d3/uKyXm1SrYn7ox0GQoOgPuflSUWxhRj7ByY7mm
	ZVHvapuIXRWm9UKfO9mK30rcpRwIj+c=
X-MC-Unique: RnlRYklbPTiK5mn8xVy6CQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fOSvcZzm1ydMu/yr3p0KgpwKAXNKJe6vPWFTGvSLGBIooflk3rS9gUg/EMxOT8f6F5KK/LOmMXQHefNHOuK6sySvAr37VNDY46epdvhGCLkRyH+FNjomtoS3Y2FwlLZTUM6sJEeX4GcoeDwwQj8ocMYnRTSG0kJGlRpLdkkwh1nAb06qCz8eY0Wfb/4HFKYY+9nm1/CLQrjRLuC+7wO9b1aXP02VsEDt9oMiZpzMkYr3pe6bDlcXTCHEpscKctPkrzPokrr8+uCwmtTs5oj8cN2HrALI5/R1BtIAgSNNfzyg7+VYCVxsGLkS2BlpZdynxAmGOmA2yt+WntQN7/sWzw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LghKfE4iKbQSz30plE0IC+HIWMwQiUkQ5z32wjJKUwk=;
 b=TBfS5CetIYaks+9NjvKN7lfBiwAM51u2bcL1XErNIel5k8n7sNJra6yVpheO9y9sJFZBNBwxVM3WlGS3/SbFsttjOxUziSFzmCZn5ZLxx+wttgD2oIsAXsZC0ywI290UNSR93YomweTNyyd0A91WLrb0eDG+aElcCW0bdWrYaBWfeHmPt8DZnRBiRCUDUYGx2wFXwLsuRqNPhuuHN3JV7Wj8r9olDj4NzmtEx4dgiYxM6Aov7VnVv8Nyqrad3SCa4PwsLiuVYt1goWmL+5eXrk5qhNsCuZ7XHwLx5Z9UQLlZr/2tRYbzzFkuXp0WFkSec01K6AzFoSuJ1f0BPWWksA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 3/6] xen/credit2: Remove tail padding from TRC_CSCHED2_*
 records
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-4-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <66d0cd95-a9db-05ca-3a4c-b728a2f5cdbd@suse.com>
Date: Fri, 17 Sep 2021 15:10:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0008.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2ca824bb-762a-480d-546e-08d979dc98ee
X-MS-TrafficTypeDiagnostic: VI1PR04MB5904:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5904BDD45405A7B3C6AD9825B3DD9@VI1PR04MB5904.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Qkx58pfvYwNcaGkjKMQN64Dw40pHERASKNKXQ7eejmlS8F++IJY3i38wjXzl1mDPpVRvHV73PvB6uklDfnPCFd+WzM8odRDlPtJNikZRWjqe3nrwZOXYkosGZ9i4llj7twyQ0po3wy7lSh4sYdJkU2H4fdVLlwe9EEyn1CExpff0c0wmEhZjRfzRWU1oWhv8Sr4OAJfS6JzsuM7HSdzefaUr256FYxleorPs6x2NQI+OWqPiS97zn8/vXPWRfe1Cer/tiuEhkiXge7kjo8E4VWlm5f0yrHy/5K9qUXJH3eilbmmZjKc2wvDZUF/Jy5FqIK7eXiZAK4yCGl9C4Z/SK5QXum/DQrKMcO+n/oFEHC99OuPB7PtZm4pNBg+ezuquUoc+yaPFywd9bnEvI7QrugKPheOCjVhnQnxtFoCU9jCbPrUNbfZo3bQorgjCx38oG/Iu22PRVLM/UvCpvd3VUeoAEhTP6P21DWsSP6ZXjxvU5DWzp1gu/qK+A/VxuL6ZzyQWDbCR8VSd9bvmCND0W3Q7n7RD7oZc4uflq8e+cWA4x8bnvvw02wjrRUvwkkjHJQmb7Aph8oF8TNw6kbB9xWZSNm6TpPSB2rOTeSGbXMWnlYUnNKmW6A85tCLajpz+LGBy9sZ+YE3p5esJq/bIA25KoIOVNchC9e0c9fHnzQenNkehFR6hrKGfrLcxET6N1iUfLVUSVri7mrQNFmUNNA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(6486002)(66946007)(38100700002)(66556008)(54906003)(508600001)(6916009)(5660300002)(26005)(86362001)(66476007)(31696002)(16576012)(53546011)(8676002)(31686004)(36756003)(316002)(4326008)(8936002)(2906002)(4744005)(2616005)(956004)(83380400001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UUVKRkJ6YU1NRkUxUitUQnk4ekxDbmFNM0dDN2xNRDV6bW0xd1pjbFc0QWNV?=
 =?utf-8?B?T1hwK3JTcVgrUHV6L0FOeUsyNEUwSGR4aEVWSVRXK1dYNVNLMkdhSEQ3Y0NE?=
 =?utf-8?B?OUtjSmRDY0lWbUU2RWZOVXEvRDV5OTF5YXpPUVFjYkhOYVZuVTVmNlNwUjg5?=
 =?utf-8?B?UUVNN1FIbXlvYWZDM3RxNUhacjJqbTI5aDFPNmN1TFgvT1hVUS9HSzlVSEo4?=
 =?utf-8?B?R3lGdlpTV2EvV3pHUHB6MDJvTERMUVphMnBWZ2d4ak04TEpoWVloK2k2V1hw?=
 =?utf-8?B?d0RocGhXSTFQaG9pRTBKU1RIaGJ4cklYK1dNcEtMZ0wyVTZqV1N1RU94aWJs?=
 =?utf-8?B?ZWpUSXRPTEN4M0ZCRGNHVS83T05tNEFCVDU4aGZGemFZNFNXVitEK0FVTmFY?=
 =?utf-8?B?dEdncWdBeDd5eEIzTHEyOGw1NXpuUnViQ3Jra0FwMnpDcVY2M29SRHNrUWY1?=
 =?utf-8?B?eTl1dGtyZnhTZ0EvYlljR3pGTVcxZGVhQ3NhU1lqelBrMVcrYm9KMU9XNG14?=
 =?utf-8?B?MzlIVVBNcG1oTmp1Q3A4eGFQbXVyVjJTNHNBcG54UTBKZ0VlVWpqRHRNOTE3?=
 =?utf-8?B?bm9UN0FHUWNRdEFaRkQ3K21lQU10dngwTnpjNG1SN1Zuc0R4alZUcGNkNmhI?=
 =?utf-8?B?WkFSSldWQldGTm1ORW1vWW9HSXRVSFVjTStpNUZJbURkTVpwZkxnQUVQUTlG?=
 =?utf-8?B?THppa0pRdWlma3ZxL3pZMmUwcnAvNGlFakY3VzNad3ZNbE8zVzNjVGtnNk5Q?=
 =?utf-8?B?ckRhNXo0NHpXZm5NQTVXK0puNmRsS3ZXTzYwaEJsdWlKcnIvdW9LamNscVJq?=
 =?utf-8?B?OFlSVHljcytxQW04cklhenF3RXFlOVU1d2d3dmgydjRKMXA0VlNmQitkWEVu?=
 =?utf-8?B?YXQyWEJnNElIeVFlY29USUxGaFkwS3FmY3NaOTI5U1BIY21KZVd4L2U1dWtm?=
 =?utf-8?B?TUY0TC9hM1BwWDdBdEFtc1V1VFd1WDBscTVIckxmTXlVTEpWUFhlRlgrbm44?=
 =?utf-8?B?UHVqd3hyZ013eUZWdUM0eTVGTWNGUGJ6bDNkbjNWbDBuVnI3NVBiNVdKcEl1?=
 =?utf-8?B?S0c4OG0yQTV5YUlsM3NMSjMwTmpFK3BEbUZGcDE3a3JmWHQrakpUOUNWdDhm?=
 =?utf-8?B?NW5BUERjQ0JCUG80endjc1pheW0xNm9mMUVtRVhLdzM2Tm1ndFRTby9zN0lH?=
 =?utf-8?B?RFo5cXVtcHp1a1RqUk14d3NjR2lkQkRzTi9mQlhvNjB6cFVVQzJnTUNHa0tQ?=
 =?utf-8?B?bkF3QTNFd0FlRjc1MWpMKzB0ZnZQL3F5YnhpaStIaFVLb3EzQ1h1TDRzYVNR?=
 =?utf-8?B?VE4xTXY2Y0M2ek9TK1pzeUlHc1lNQ0Z1b2d3RFpDclhoSW13QXpOc2hXejla?=
 =?utf-8?B?bnJZdDhEaTRwclA5ZXJxVllPZ1pEM3NpV25hc004V2M2RS9VVmNmYXdGVDk0?=
 =?utf-8?B?QzN1VDhHdStucS9FQjlhTjJnNFVPSzF3ZzRYZG1qQzZtems3QUJMcG8xM043?=
 =?utf-8?B?N1gvQ0kwVVphNG1tSTZ6aUNzQXhQOENoWHdlSG1FQkw3V1cwT2ZNWHk3Q1NG?=
 =?utf-8?B?dEN1ZDNVdVdpbGw2ekRONlEvaHdOc3lvS01UMUttNzhkdlpPTU8yR0RWc1pL?=
 =?utf-8?B?T2U3TWdrVU5zbmswRGxvc1pubG52NXJ2YmszbUIyZTlBdzZ2bnYyZ1gyTVRB?=
 =?utf-8?B?MkxuYmpCaEVNNXZLc0RZOU5XMjhJNE9aMm52YmE1WHF3bHp5SnFHVGo0dEdy?=
 =?utf-8?Q?hcPPy7Op/5LMYdCcAU/SVRUqmeVea3EG8x1hX+y?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2ca824bb-762a-480d-546e-08d979dc98ee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 13:11:01.6377
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KV80HJv51nLnVJaDrz/GuB3FrTcDavZKi8Zd7J4XR/k6EcbOOScRC9OqmaYrCgsF/ruxE2oMVinljy+F3dDrCQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5904

On 17.09.2021 10:45, Andrew Cooper wrote:
> @@ -1336,13 +1338,16 @@ update_runq_load(const struct scheduler *ops,
>      {
>          struct {
>              uint64_t rq_avgload, b_avgload;
> -            unsigned rq_load:16, rq_id:8, shift:8;
> -        } d;
> -        d.rq_id = rqd->id;
> -        d.rq_load = rqd->load;
> -        d.rq_avgload = rqd->avgload;
> -        d.b_avgload = rqd->b_avgload;
> -        d.shift = P;
> +            uint16_t rq_load; uint8_t rq_id, shift;

Split into two lines? Preferably with this adjustment
Reviewed-by: Jan Beulich <jbeulich@suse.com>

I'd like to note that the remaining uses of "unsigned int" or plain
"int" in some of the instances you don't touch assume
sizeof(int) == 32, while generally we assume only sizeof(int) >= 32.
This is one of the cases where fixed width types are imo mandatory
to use.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:20:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:20:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189401.339080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDmd-0003IV-31; Fri, 17 Sep 2021 13:20:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189401.339080; Fri, 17 Sep 2021 13:20:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDmc-0003IO-Ve; Fri, 17 Sep 2021 13:20:02 +0000
Received: by outflank-mailman (input) for mailman id 189401;
 Fri, 17 Sep 2021 13:20:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y96v=OH=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mRDmb-00036j-SA
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:20:02 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f5b01a94-17b9-11ec-b6a0-12813bfff9fa;
 Fri, 17 Sep 2021 13:20:00 +0000 (UTC)
Received: from [10.10.1.24] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1631884791627845.5371428471088;
 Fri, 17 Sep 2021 06:19:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5b01a94-17b9-11ec-b6a0-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1631884797; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=YpJjkWR2KTeCl/tPoVDK4LqnKEAIAvj/+YPvDIeCeQqsgbEIXnufveLZUqHHfamyEoHP+vIK2eOG7/nPLQ0MEV4/kPicFTG97nwCC24kKveskpB0SQEZm01N7tRIR+37zGD2yLau3cKnxtSKwpfhjbOwnHo+aBl430mfXqhbOgo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1631884797; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=HhssghJGx7/IL6SOq974BeYYZj6SlieVg2iEnlJF4po=; 
	b=XMGUN8IV0QJRRPRm7A0osGc9/dzvtPllTASogOsA6IfDkY+RL4t8warNZD2w48KljrgTBmvJyWaMIyAO6CHfgKNTs/73ts9wtuHH5cXfXdaP65c9sYK8Qje6syh9mLl+LerFE+MVqTpWRILgzAxx2nOTHPVXemV5MbRIak9ZMuA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1631884797;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=To:Cc:References:From:Subject:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
	bh=HhssghJGx7/IL6SOq974BeYYZj6SlieVg2iEnlJF4po=;
	b=CnINWBt1FD+9eazMkK8mRprHOW/s0VqV1inbL2elvY52RBMGhCELXAwcOsr53qOz
	Ok9741p8gUMnJAx8opfyAblujVmxxsHge/pTDV5Dyxh2AVCR1w6o3MZIbuX4Ftirqao
	Z9exR3reqZeJcYUJFqthkHY5e6jIGQbak+HAjeU8=
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 xen-devel@lists.xenproject.org
References: <20210910201305.32526-1-dpsmith@apertussolutions.com>
 <20210910201305.32526-10-dpsmith@apertussolutions.com>
 <33d87667-4be7-0ec8-0c1e-f773d77f2d72@suse.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH v6 09/10] kconfig: update xsm config to reflect reality
Message-ID: <7b6c830f-c4b3-3fef-a1ed-0d557eeacf52@apertussolutions.com>
Date: Fri, 17 Sep 2021 09:19:50 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <33d87667-4be7-0ec8-0c1e-f773d77f2d72@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-ZohoMailClient: External

On 9/17/21 8:09 AM, Jan Beulich wrote:
> On 10.09.2021 22:13, Daniel P. Smith wrote:
>> --- a/xen/common/Kconfig
>> +++ b/xen/common/Kconfig
>> @@ -200,23 +200,20 @@ config XENOPROF
>>  
>>  	  If unsure, say Y.
>>  
>> -config XSM
>> -	bool "Xen Security Modules support"
>> +config XSM_CONFIGURABLE
>> +	bool "Configure Xen Security Modules"
>>  	default ARM
>> -	---help---
>> -	  Enables the security framework known as Xen Security Modules which
>> -	  allows administrators fine-grained control over a Xen domain and
>> -	  its capabilities by defining permissible interactions between domains,
>> -	  the hypervisor itself, and related resources such as memory and
>> -	  devices.
>> +	help
>> +	  Allows for configuring the Xen Security Modules (XSM) policy or policies
>> +	  modules that will be availble and which will be the default.
>>  
>>  	  If unsure, say N.
>>  
>>  config XSM_FLASK
>> -	def_bool y
>> -	prompt "FLux Advanced Security Kernel support"
>> -	depends on XSM
>> -	---help---
>> +	bool "FLux Advanced Security Kernel support"
>> +	depends on XSM_CONFIGURABLE
>> +	select XSM_EVTCHN_LABELING
>> +	help
>>  	  Enables FLASK (FLux Advanced Security Kernel) as the access control
>>  	  mechanism used by the XSM framework.  This provides a mandatory access
>>  	  control framework by which security enforcement, isolation, and
> 
> I continue to think that the default here and ...
> 
>> @@ -253,10 +250,10 @@ config XSM_FLASK_POLICY
>>  	  If unsure, say Y.
>>  
>>  config XSM_SILO
>> -	def_bool y
>> -	prompt "SILO support"
>> -	depends on XSM
>> -	---help---
>> +	bool "SILO support"
>> +	default y if ARM
>> +	depends on XSM_CONFIGURABLE
>> +	help
>>  	  Enables SILO as the access control mechanism used by the XSM framework.
>>  	  This is not the default module, add boot parameter xsm=silo to choose
>>  	  it. This will deny any unmediated communication channels (grant tables
> 
> ... here should not change. If it changes, the change needs justifying
> in the description.

IMHO the configure system should not presumptuously assume that if a
user selects XSM_CONFIGURABLE that the Flask module, which is not
currently security supported, should be enabled. I would agree that we
could turn on the SILO module since it is security supported. Later when
I am able to get Flask into a security supported state, I would be all
for enabling it as well. A more roadmap-ish idea is to see SILO subsumed
as a select-able policy under Flask, but that is a bit of a digression.

I will add to the commit message to clarify this position that is being
encoded.

>> @@ -282,15 +279,15 @@ endchoice
>>  config LATE_HWDOM
>>  	bool "Dedicated hardware domain"
>>  	default n
>> -	depends on XSM && X86
>> -	---help---
>> +	depends on XSM_FLASK && X86
>> +	help
>>  	  Allows the creation of a dedicated hardware domain distinct from
>>  	  domain 0 that manages devices without needing access to other
>>  	  privileged functionality such as the ability to manage domains.
>>  	  This requires that the actual domain 0 be a stub domain that
>>  	  constructs the actual hardware domain instead of initializing the
>>  	  hardware itself.  Because the hardware domain needs access to
>> -	  hypercalls not available to unprivileged guests, an XSM policy
>> +	  hypercalls not available to unprivileged guests, an XSM Flask policy
>>  	  is required to properly define the privilege of these domains.
> 
> I also continue to think that this would better be a separate change.
> Or if not, the seemingly unrelated change needs mentioning in the
> description (to make clear it's not a stray change).

Apologies that I missed the suggestion to break this out. Since this
really is more of a general clean-up over being part of the intended
improvement for this patch set, I will break it out and move it to the
front of the patch set.

v/r,
dps



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:26:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:26:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189415.339105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDtG-0004ZL-TU; Fri, 17 Sep 2021 13:26:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189415.339105; Fri, 17 Sep 2021 13:26:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDtG-0004ZE-QS; Fri, 17 Sep 2021 13:26:54 +0000
Received: by outflank-mailman (input) for mailman id 189415;
 Fri, 17 Sep 2021 13:26:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mRDtF-0004Z8-9H
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:26:53 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eadff8a4-17ba-11ec-b6a4-12813bfff9fa;
 Fri, 17 Sep 2021 13:26:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eadff8a4-17ba-11ec-b6a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631885211;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=LdVokYFUq6jaaLc4yGIoZkkBwGC7+cLM+1HXYT59Izc=;
  b=Y2pcQGaJXlNd4Te5DE7wKgKsMQRx8glKEjWCoh64pVS7gNrjjikL+fVH
   PLsMnaUbhWQtwFHBrTi7JZw7UaDW6jRP9zVGUnTljAtwH1xM1kQ9mPBEh
   WRXvtv53wcmZBsSdZUCnhlXE4Ioq6GnI8CK2z6PHTLOOuEyy/0m23f48E
   A=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: YG5Mi/Q92JDBauIHSxUBmJ/9rE47i0uAK1qTmwdFC8CDb2gpdcunmfLg64rTQrFHoo2j7e4Unw
 g5vWh6Hz6KrgSFehheVFFvP3Fup47JEu0F38Nno1bfuPOLoZ2M2N30A+QMGlyDPXNHU9VpO5Ea
 IF7VMxTJ2K7l6yax8pwR78AoTZYUNBVzOO2XWiRF7fYjkHCnt6P/h5cVXcXq57r/ky/2XZEvu8
 xnXA878jf+Nm0B+6xOtkv0ObhEg+d8aAsuTr4MgpzVASaMbtAJ0SFIjFBG1j+L2GHOegMoWkz4
 QkC1Rs9znEegOA7zj9Gn+Mbs
X-SBRS: 5.1
X-MesageID: 52973140
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:J4c7gKjjr3PMqOZsPCzDJqjFX1614RcKZh0ujC45NGQN5FlHY01je
 htvUWCGP/2MYGrzfdp2at629h8FvMeGnd5qHFE//ng2Fi4b9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8x3oDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /0Op6GcFjoMEpaQt84xAid+IgJBfo1ZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHCOo8Ft24m5jbeFfs8GrjIQrnQ5M8e1zA17ixLNauEN
 pVCMGY3BPjGSwFhCk4wE6Ngp+6h2X3aWR129U2sopNitgA/yyQuieOwYbI5YOeiXt5Jl0yVo
 mbH+WXRARwAMtGbjz2f/RqEh/DNtTP2XpoIE7+1/eIsh0ecrkQRAhALUVqwodGil1WzHdlYL
 iQ85S4GvaU0skuxQbHVTxC+5XKJoBMYc95RCPEhrhGAzLLO5ASUDXRCSSROAPQqusY5Sj0t0
 l6hhM7yCHpkt7j9YXCA8raZqxuiNC5TKnUNDRLoViNcvYOl+ttqyEuSEJAzS8ZZk+EZBxmz0
 Q2m8xUbq45PouQnhpudzHP4wBKz882hohEO2unHYo60xlonP9f1PN35sQKzAeVod9nCHwLY1
 JQQs43HtrlfU8vV/MCYaLhVRNmUC+C53CowaLKFN6Io8Siks1WndJpZiN2VDBY0aptYEdMFj
 Um6hO+w2HOxFCDxBUOUS9joYyjP8UQHPY6+Ps04lvIUPvBMmPavpUmCn3J8OlwBd2B3yskC1
 WqzK57wXR7294w+lGfeqxghPU8DmXllmDK7qWHT5BW7y7uODEN5up9cawDmUwzN14vd+F+92
 48Gb6OikkwDOMWjMni/2dNCdjgicClkba0aXuQKL4Zv1CI9Qzp/YxIQqJt8E7FYc1N9zbuRp
 SDhBRYDkTISRxTvcG23V5yqU5u2Nb5XpnMnJy08e1Gu3nkoe4G066kDMZAweNEaGCZLl5aYl
 tEJJJeNBOphUDPC92hPZJXxttU6Jh+qmRiPL2yuZz1mJ8xsQAnA+9nFeArz9XZRUnrr5JVm+
 7DwhBnGRZcjRhh5CJqEYvyY0F7s72MWn/h/XhWUL4ALKlns6oVjNwf4kuQzf5MXMRzGyzbDj
 1SWDB4UqPPjuYgw9NWV16mIo53wS7l1H1ZAHnmd5rGzbHGI8m2myI5GceCJYTGCCz+kpPT8P
 b1YlqiuPucGkVBGt5tHP4xqla9utcHyo7J6zxh/GCmZZVqcFb49cGKN2tNCt/MRy+YB6xe2Q
 E+G5vJTJa6NZJH+CFcUKQcoMraD2PUTlmWA5Pg5Ohyntip+/b7BWkROJRiczidaKeItYo8ix
 O4gvu8Q6hC+1UV2YorX0HgM+jTeNGEEXoUmqooeUd3ihQccw11fZYDRV33t65aVZtQQakQnL
 1d4XkYZa2iwEqYaT0cOKA==
IronPort-HdrOrdr: A9a23:5TPCFqmoZJ/fgjPGUeWZJ5H1QNrpDfIo3DAbv31ZSRFFG/Fw8P
 re+8jztCWE7Ar5PUtKpTnuAsW9qB/nmqKdgrNwAV7BZmfbUQKTRekJgLcKqAeAJwTOssJbyK
 d8Y+xfJbTLfD1HZB/BkWqF+gAbsbu6zJw=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="52973140"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wej+XIEDC+j8pfX95/w6gfHQsdKGpOO1ol1WeA5x6MIWve1oJzDM1ef3d7RAJVghK8iv5b1tkIpwuekV/gJph+QOmwmMqhhuRmn87qfT1GfyHefwYA2ybWRaQJTUxnSluCjhu5hOZA7XIFFrJG72qmeBaJHztU/Canudv/z4PgNFOEFZVz+60M90Pg8PM8mYADF2YzHODobChWiV8cXUxWf6v1tsaWiOH4rF8stiBvHM65myWX/ASpPx/cPH5EHOeNLlVtUtQ48KvpjbZ9Nk7cM7j5vDmkH5ATbXuUDpH1Ek8NffOEZT9KQN6msPZTtA1/VZXpFQazPu2ChBGgwhaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HZAdBgS4/6chgWS5jvU7DTQYRfcpq5y/k6AEYNbo5OQ=;
 b=T2kiObiJCtbwxy1pE0zsXvHV7nyLyls1vyzvRsx4SUA3UfKm3w2hLa/EqCnb0/+64mEZvRsUix6jFac6JtZ5aZhe0jW7e2lYB6nv3aD0hH15pT4PeiQZpqsbJ61nRRMK4TxFttny1SZju423BRG0B1cFWfgekpQfpKNgVuKvTZgQOyck+gcyvYfQo4/gvZSr652wq8wWnKVhWU7hT+kGSFUF3ZDDJpMl86sZ60DtMyGkPfNSS5Wficug/RbLHovmz5v2x+rjLMqfw8XEDp5zIM+VBzs6j3WyG1Y8JjnT4uL7TN7EYpZ2gkyznHsAqxg7PaJ8Z4K2i7DROtDgDmSgvQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HZAdBgS4/6chgWS5jvU7DTQYRfcpq5y/k6AEYNbo5OQ=;
 b=bQSVpsDYHxNci7p+NTARw7C4+VI6a1gjSw7knio3JiNlsHYyAEQMD95dDwHsnb98NMTmJoqZTTCqNXgdzb6AE6ilxMwBo07ns5cRbcNIPluWw7ylIAtwwpdnzNpZf97c8NrzYl7X9Fcb275R271q71/4taOe+7gGBgf8jQ1zTGY=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-2-andrew.cooper3@citrix.com>
 <ed07d2de-9354-6c6d-b9f6-4b325ca328ce@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/6] xen/trace: Don't over-read trace objects
Message-ID: <0db4dc68-ccf0-0de6-61ce-7f155431a977@citrix.com>
Date: Fri, 17 Sep 2021 14:26:35 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ed07d2de-9354-6c6d-b9f6-4b325ca328ce@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LNXP265CA0080.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:76::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9700348b-18aa-40e4-91d4-08d979dec9eb
X-MS-TrafficTypeDiagnostic: BYAPR03MB3799:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3799B07F71A40E7DF256D7A2BADD9@BYAPR03MB3799.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dNKkkSYXKKOD/AT0/Nc7COfVWGn9knM0zf3LJe1NvqoIw0zvnOeNqaSmcb7BFoSMR6Uq9bcOCYCebiqBGy6yN5MLeLrgD2mabpCDEL4MNbgwoKStkcTMdgQP0cO3MpqTNK7OsW/Tf5PK73tHWG4Qjrr2ufhigVKOAGaRdq2X5Ccicvp4VRGPmXL+mSbG6gaOyvlx1w12Le4Rl3WebC6ggu3rEONdB60yt9G6wOY8BWHxMHwUphobprA2oTk5WAFLKf7mawm/PklKnO7UayPXpAQa/A14GVGSubXrciy/qHAh+46mbOp9YR/S+GxSdtEZPj7pcDiT0ov3TaWpLhgAbLAeB1EAeTXKM5D/tCuMsOIxMUZxJXJZlgDvSgKUlmPDhB6802ijuoEfLOfZtaUMNbNdKrtdGx3zWiD1kVt/c5APgwM91l45/YJGR+BxlPpgg+Ma7Q5GRsXjgvTcoxM9qv/KmiQ8eqzH4LVFgbxY7QzXKucsH3FUJZRGg5jfY/M/z/8RFUdtyth/RZFFTFCOMRogygtIwRchRBXqwukWOUYxR3PN+tMoCHliXoyp3WwvGV37k/WOQ+F7zJS9Dl2g3xa+DxzkxnatD9T2wLyLpdecigElm++K7u9ywHhEENVKM1Q1RRsPif3b9+9YOyInG7rskBWEXOiExBjIQ8/S3bSYnVcIfZq9718xeHzoHb/I95fAYzFWAkoslElLtWep8aZcyPKeLc9E9PDkBNsWhZE=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(186003)(31686004)(2616005)(508600001)(956004)(53546011)(66556008)(4326008)(54906003)(66476007)(36756003)(8676002)(8936002)(6666004)(83380400001)(2906002)(38100700002)(6486002)(31696002)(6916009)(55236004)(16576012)(5660300002)(316002)(26005)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bmkxbVo1cDZzWFZMTnp1VlN2Uk93SWI4MDZSYW9QaWlMUFAwSTZoS0l6SHFu?=
 =?utf-8?B?YUNxMDZNakUxY3FXbEtGQlJBYzVkMEtKbkF0NzBXV0RrQnZTY3pyOUwxMHVD?=
 =?utf-8?B?T1g0WVUrNnAxMTA5bzVxMVZsdGlGS0Z4dmk3cjI3SGljWm5LdmJ3c3drYXha?=
 =?utf-8?B?SXdyRjZEamgxbW4xUnFzQlVaUlplSVJyUEI2T1QwaWV6TldlNnVSTmtLUlcz?=
 =?utf-8?B?VnVDTXBhQUpKZjdwTlZPd2QyZlA2TGpiWERLODI1SWViMitMVVkveUVUbXVw?=
 =?utf-8?B?Z2JzTDc4eFUxR2wrTHNlTHR1d1VqNmpCTzg1TUx6L1I0bHBhSTlhdlFBYk5p?=
 =?utf-8?B?emM3Wm16cllna0o4NkxaV2N0b0N2WVp2NlU3RnorOFdNNHJNTXlzTVkwYmxK?=
 =?utf-8?B?OGlYdlNyNVpOTFVmRmdmY0ZaMkUxcTRZMzREc1ZhM3JMUHVKTnNrQndUWWN3?=
 =?utf-8?B?RjJTTzg2OUJqQ2g4ekFETjlSM2x0SFNkckdkYUFvbzd0azQ2V0o1VVUwTzNB?=
 =?utf-8?B?cGNmQTdmOVhmRkhjbVpJT3lWV3B0VnYrRFFWUDRwMlFvWVBZRVJiRlpEcm1n?=
 =?utf-8?B?U2ZvTEFESmVIRTI0VmVFMHVaekNVU2RSNkhyZU13SXk0cXJ5R3dqOFBNc1JW?=
 =?utf-8?B?QXhQN1UzSW1FRE5BVm1Da1FMcWhSQTA5eGY3WEx2UHRCT0hiY0VLTFhpSlY4?=
 =?utf-8?B?UzNIZHgzaitJeFFUWXBRK2o5MnVjelN5TVU2aHFlZlQzSVA0TnZXakpXbDkr?=
 =?utf-8?B?M3VyNUxsbVphZG9vUU8zTXkyL3pxMVRia1FIM1ZlRlNUMFpoNTg5WjlBcnRv?=
 =?utf-8?B?cm5DdllCNTI3bTBrV0FrMmRYVEZ3N1o0bFFXenBKaUpCWERwN3cvZHFQdDFi?=
 =?utf-8?B?TG0rRHBHSmxCNXcvaHNHbDJqb1Z4T3F5Y01tWTZPaVcxUEw5dGhVRlBxdWlJ?=
 =?utf-8?B?WjBCQ1dnclA1dkxtd1I1MWMvQnB6WGk0Ti9qd3c4dkJsOUJxSEl3dFhZeWZV?=
 =?utf-8?B?Q092MG9LdXoxdXM2ZmlINVV1NTNrelFOR1ZVYzh4S3J3dkw3OGdkMHMrTG9w?=
 =?utf-8?B?R3psSEkvMHdQQjY3UEhFSUZSWnB1YXgyOXcwTGEwUi9JSjhlUHFyOXBZSHND?=
 =?utf-8?B?SzMxZVE5R0RCYnhwU2xVZjlzMjlyWE8rRVdFT0pJWjF0RXRKRmp4bmJaNC9K?=
 =?utf-8?B?OG03dmgzeU85UzFod2FNTGkrc3JPcG4rSEwyVXJ5NW8rZ08yRXFuRVRZUEdW?=
 =?utf-8?B?V2x2THZjMXRBbGRNeUNLdlpYWkxMT0pZZU1Sc0hodDF6UjJnQVcxMkxsbzRy?=
 =?utf-8?B?TXdFRm5DaW1FM05tbHdsRFNOSmFqZXlnN0JCN0VwNkZZeUZSTUEvM2gzc3Q2?=
 =?utf-8?B?ZFJ6MFZQNGkwL29yT0JydkZPR3JJVzlDbWQ0OStsbjJEL2F3V2JPNUxIK1Mr?=
 =?utf-8?B?V0QzbGtvQ0ZHY1hGcGVxWHBHclJtbzFXV0IvZGlhWTNGQ3o5bHdHUlFLR2J0?=
 =?utf-8?B?TC9zeVRDSU84UHowZ2taYWJOVm1pYXdpb1AyelQ2aC81WitkZUw5TnJSTkJj?=
 =?utf-8?B?UHF4V3hpMEdUL0VkSUdwVkUvRkdhTndwc0xCZjlGM2xpOXl3UEd4OGh4TklI?=
 =?utf-8?B?RVR0WHBQOGlmZGRLck9XQXdJRWhkUlJweHZnd0VRUXg3V0lNWC95SWtQQ2JJ?=
 =?utf-8?B?OURiQ3FIeElXcGlIMHhjMEx6ckgvbC9rNlIyVXdoOU1kQzUxOXZyNlIxakxV?=
 =?utf-8?Q?x+N+QIi9DMeUITMflxKyPJjI5USvnnfL8cQorsn?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9700348b-18aa-40e4-91d4-08d979dec9eb
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 13:26:43.0133
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wiW4EGTRfDi68n41ne99Dpwxn7fAHoIAOP/ZzyyodP/58nP0j5wVTbPm3vMhPwvTLQpe1aB5KrH+J1ceyAmF5AxQ9yuZHHKQBRwOdKikQiA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3799
X-OriginatorOrg: citrix.com

On 17/09/2021 13:58, Jan Beulich wrote:
> On 17.09.2021 10:45, Andrew Cooper wrote:
>> --- a/xen/common/trace.c
>> +++ b/xen/common/trace.c
>> @@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsigne=
d int extra,
>>      unsigned long flags;
>>      u32 bytes_to_tail, bytes_to_wrap;
>>      unsigned int rec_size, total_size;
>> -    unsigned int extra_word;
>>      bool_t started_below_highwater;
>> =20
>>      if( !tb_init_done )
>>          return;
>> =20
>> -    /* Convert byte count into word count, rounding up */
>> -    extra_word =3D (extra / sizeof(u32));
>> -    if ( (extra % sizeof(u32)) !=3D 0 )
>> -        extra_word++;
>> -   =20
>> -    ASSERT(extra_word <=3D TRACE_EXTRA_MAX);
>> -    extra_word =3D min_t(int, extra_word, TRACE_EXTRA_MAX);
>> -
>> -    /* Round size up to nearest word */
>> -    extra =3D extra_word * sizeof(u32);
>> +    /*
>> +     * Trace records require extra data which is an exact multiple of
>> +     * uint32_t.  Reject out-of-spec records.  Any failure here is an e=
rror in
>> +     * the caller.
>> +     */
> Hmm, is "require" accurate?

In terms of "what will go wrong if this condition is violated", yes.

>  They may very well come without extra data
> afaics.

0 is fine, and used by plenty of records, and also permitted by the
filtering logic.

>
>> +    if ( extra % sizeof(uint32_t) ||
>> +         extra / sizeof(uint32_t) > TRACE_EXTRA_MAX )
>> +        return printk_once(XENLOG_WARNING
>> +                           "Trace event %#x bad size %u, discarding\n",
>> +                           event, extra);
> Any HVM guest looks to be able to trivially trigger this log message
> (via HVMOP_xentrace), thus pointing out an issue in a guest and hiding
> any other Xen related output. I'd like to suggest to adjust that call
> site in prereq patch (I'm not overly fussed which of the two relatively
> obvious ways).
>
> Further sched/rt.c:burn_budget() has a bool field last in a packed
> struct, yielding a sizeof() that's not a multiple of 4. All the uses of
> __packed there look at best suspicious anyway.

Ugh - I checked the __trace_var() users, but not trace_var().=C2=A0 Luckily=
,
there are far fewer of the latter.

HVMOP_xentrace has no business being a hypercall in the first place.=C2=A0
That can be fixed by also enforcing the multiple-of-4 requirement.

But yes - burn_budget() needs fixing in this patch too, taking it from a
theoretical to real problem.

~Andrew



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 13:29:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 13:29:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189424.339120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDvZ-0005RW-E3; Fri, 17 Sep 2021 13:29:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189424.339120; Fri, 17 Sep 2021 13:29:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRDvZ-0005RP-9g; Fri, 17 Sep 2021 13:29:17 +0000
Received: by outflank-mailman (input) for mailman id 189424;
 Fri, 17 Sep 2021 13:29:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2OQJ=OH=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mRDvY-0005RJ-1x
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 13:29:16 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 402a7cd0-17bb-11ec-b6a4-12813bfff9fa;
 Fri, 17 Sep 2021 13:29:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 402a7cd0-17bb-11ec-b6a4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631885354;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=q5It2uaMZ9gTW/mN1H3bafbIZWfxVecyJARSdZ/IT/g=;
  b=YgKP5xWiJoz2b/4sXdfnt3l5CwTkXGHuDvzg+N9FkXBTFoYLTJYVjEn6
   Gz88DOOmSIo+Lwg6ykOB1yuXrWRLnYaoBQ4MBB00OVw7jBF7hlmLXJZl6
   VOJSfREMQoVDVZR5ho60rp5O9DLLurHeb+059jclcEg1WvwaCNxZUJ6oK
   4=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: XsUgqEzkGOvXZ+bbzB/UuN6vkjGHb6NjvdKS1EHlwfoGekNVJEx/5SuxZhRQXnft+MhJ0z9scE
 7nN5NJsVplM0jyDaI9ZUOiPPqmL0gPUG+euxrIowUdI1zPHd+gBed41fDarYUkgQ+x6SaSTTKQ
 wCpY9iYfHvRvpU8Tplocg8dsxgZQqMSqMDY+IsMunAAQ/jxRFYt9/MyQronndEl0JvlWeLdS1d
 ssIxZP0vw90FOsDj7LdM+BocS02t5eYhP1L5CmXjJTeB7qMMaa7Nzup8JZLPb9Zj7ht2A60oiL
 xgnMZECkasu+Sd8r94ZmfmWr
X-SBRS: 5.1
X-MesageID: 52973320
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:RsRbiqwKMhoPTGhKZGp6t+e2wSrEfRIJ4+MujC+fZmUNrF6WrkVVm
 mEYCDvTMvaCZWb3KIokbISx/U8G65CEyIM1TlA6/yAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrdm2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt9lXl
 O5thZrrcwEGJfPqldgWWUV9MD4raMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25ofQq2FP
 ZJxhTxHSR3GYT9UZ3QsOZMQrMu5gCHNegZFtwfAzUYwyzeKl1EguFT3C/LOYcCDT8hRmkeep
 0rF8n7/DxVcM8aQoRKE7XCEluLJhTn8Wo8ZCPu/7PECqF+Zy3EXCRYWfUCmuvT/gUm7M/pAL
 2QE9yxoqrI9nGS7Q9+4UxCmrXqsuh8HR8EWA+A88BuKyKff/0CeHGdsZjJPYdMptcgyRBQxy
 0SE2djuAFRHsqCRSH+b3qeZq3W1Iyd9EIMZTXZaF01fuYCl+dxtyEKUJjp+LEKrptTTRQrQ8
 zbRli4RnOs1gPIA3fme3k+S1lpAuaP1oh4JChT/Bzz+t1MiNd/9PeRE+nCAsq0Rd9/xok2p+
 SFewpnAtrhm4YSlyXTVKNjhCo1F8Bps3Nf0uldpA5Bp3DCk4XfLkWt4sWwmeRsB3irpf1bUj
 K7vVeF5v8Q70JiCN/Yfj2eN5yMCl/OIKDgdfqqIBueim7AoHON9wM2LWaJ39zu2+HXAbIllY
 cvLGSpSJS9CVMyLMwZat89CiOR2l0jSNEv4RIzhzgTP7FZtTCfOEt843K+1Rrlhtsus+VyNm
 /4Gbpfi40gPAYXWP3iMmaZOfA9iEJTOLc2vwyChXrXYeVQO9aBII6K5/I7NjKQ+w/0Ox7uXp
 yrkMqKaoXKm7UD6xcyxQikLQJvkXIplrGJ9OiopPF2y3GMkb5rp56AaH6bbt5F+nAC65fIrH
 fQDZeuaBfFDFmbO9zgHNMGvp41+bhW7wwmJOnP9MjQ4epdhQS3P+8PlIVSzpHVfUHLvuJtsu
 aCk2yPaXYEHG1ZoAvHJZa/91Fi2p3Ucxr5/BhOaPtlJdUzw24F2MCit3OQvKsQBJEyblDuX3
 gqbGzkCouzJr9Nn+dXFn/nc/YyoD/F/DgxRGGyCteS6MizT/2yCx45cUbnXIWCBBT2soKj7P
 Ldb1fDxNvEDjW1miYskHuY517866vvuu6ReklZuEkLUYgn5EbhnOHSHg5VC7/Uf2r9DtAKqc
 UuT4d0Ga66RMcboHVNNdgooauOPiaMdljXItKlnJUz74Gl8/aadUFUUNB6J0XQPILxwOYIj4
 OEgpM9JtFDv1kt0ao6L3nJO6mCBDn0cSKF25JgVDbjihhcv1lwfM4fXDTX74c3XZthBWqXwz
 uR4WEYWa2xg+3f/
IronPort-HdrOrdr: A9a23:lEaMYKsRijKSzOPv7NhMkjTq7skCJIAji2hC6mlwRA09TyXGra
 GTdaUguyMc1gx/ZJh5o6H+BEGBKUmskqKdkrNhQ4tKPTOW+FdASbsD0WKM+UyaJ8STzJ856U
 4kSdkDNDSSNyk4sS+Z2njDLz9I+rDum8rE6Za8vhVQpENRGtxdBmxCe2Cm+zhNNXF77O0CZe
 OhD6R81l6dUEVSSv7+KmgOXuDFqdGOvonhewQ6Cxku7xTLpS+06ZbheiLonCs2Yndq+/MP4G
 LFmwv26uGIqPeg0CLR0GfV8tB/hMbh8N1eH8aB4/JlbwkEyzzYILiJaYfy+gzdk9vfsWrCV+
 O8+yvICv4DrE85uFvF+icFlTOQigrGoEWSuGNwyUGT0fARAghKVvaoQeliA0TkA41KhqAh7E
 sD5RPri7NHSRzHhyjz/N7OSlVjkVe1u2MrlaoJg2VYSpZ2Us4fkWUzxjIZLH47JlO31GnnKp
 gZMOjMoPJNNV+KZXHQuWdihNSqQ3QoBx+DBkwPoNac3TRalG1wixJw/r1fol4QsJYmD5VU7e
 XNNapl0LlIU88NdKp4QOMMW9G+BGDBSQ/FdGiSPVPkHqcaPG+lke+43JwloOWxPJAYxpo7n5
 rMFFteqG4pYkrrTdaD2ZVamyq9CllVnQ6dgf22wqIJzYEUaICbRBFreWpe2/dI+c9vcPEzc8
 zDTK5+Erv/JWGrHY5I2mTFKtBvFUU=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="52973320"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EvQvhb1d/qPMzoBzjMQ+9+lrSY6E+hAVf0ezOVBk9n+GUcxs3LCuCTltzEpRFT/oETiT4aFWUljej1E4TAYTz5SdxpVwhk0G7eaXiqmxbuBk/6HcneRYsI15Z6Gv7UWXuLf4C7uphit6W1mVd7zEBOPQQlMXDnjAPwSm8dI4BLU0+RyZUjeRciF81TU1rUMiRw4e3yEjyTnEbx3GeQZ8nBltKvzh2PXN/kMtcqdUfB4XZGM5JxzHPyRIC3zXgKrqILJ0MEqVW1AbaWA3JZDtFwPMJAiAeo9DLVsSBheRAFLelQX/SEbq/Hxd8LTSY6wMuIPNAGEJ5XvTokOTSJGhRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NKQPTZs0jxTjQ9A1ySBjY71pxV11rHjXty003ceqxio=;
 b=cJvEVU7Sisbef9lFhXzObRXsLTmrmfYmZ+GSp5/wl9ZqWf+LgO7whD3roOydJiXpNNBGeru3AFYpzPfssBsem445s3MdeNecZTnKDBSnuo56zE4hb7XfDZAQoVPQh2yUBGvDLjZP0jfBsLgBhR4elc/ev/W+NxdaED8LoGtNuc47daYcUPCuSYFVScIYoPKBlE3tZQQevbRgaYu3GBvI7E20nfBknoUGbUW2J5GaTSPVrAxZiR8QhoXLWuZQFDe4BjGjncXBfz5ALfMd4i9dPGKmN4L9/fZ7jOfl1Xz/8mxMUSCdPkBurelOoCbQFyYm7Q5An5bkfwHFKM//8KgGsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NKQPTZs0jxTjQ9A1ySBjY71pxV11rHjXty003ceqxio=;
 b=uvZFmeCHdoazqRIlYQLKlaGsDgcHM1j6xvKoFSngiwZ6ev22HDb1hGj0t4VXymzhIXqjzoYpHp3/Du9LtpHuzNSPF3gif4fKfYZKuwm8NpNRkHygG6WDCsIifdl1UC5nWQqv/6E0fj4Bc8dMZon5TLK4gK5Muu53HLYTW8mfJRA=
Subject: Re: [PATCH 3/6] xen/credit2: Remove tail padding from TRC_CSCHED2_*
 records
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-4-andrew.cooper3@citrix.com>
 <66d0cd95-a9db-05ca-3a4c-b728a2f5cdbd@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <ffb38e63-e18b-c143-9b30-46d6cb2cc8a9@citrix.com>
Date: Fri, 17 Sep 2021 14:28:58 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <66d0cd95-a9db-05ca-3a4c-b728a2f5cdbd@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0299.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:196::16) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ade5775-0eb5-4411-5e71-08d979df1e8a
X-MS-TrafficTypeDiagnostic: BYAPR03MB3557:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB355702B6B4A59A18DAC2CA00BADD9@BYAPR03MB3557.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: IjaSaEinXo7XU6zkgAZSkw3CiHtbLysO8W11AR6hfoEuEPH8G/F9XERP9A+W0uWk1pD52NklJyplF0S/96iRs66ijlg/rrR11j+BLClr7hMTR3fPE6vaBVS7g0j7kZwTOc3/XRocqnk/aXFhujhRlqmcdsPWrGqpFIlhay7vkJYLMQIDKqGKinBlYASR9UBCQqnmlFFtdF+DG4VIQw/FjaqUItJ33dNxMs8PgMmiHwzwg8lp8eJn3+xQkVdVAg9MZwEkwQvZSWtKxm0/y82zU6boEmHGQcN2ksgtZ0OO3993uB7JxTs4kb8AV16Gi3TbAdlxJwe5uumK6ZDcTtneKFNsqyaOrliQvXBjPAxswC8TuIZV+b62M6LqYBGjU4dVbsVOUtQiEyLXEZnV0zfHML+J2aZ9LI/mMRlCCpbzQ3U3ZxubppWkvqn5M+SeyHoQxHk4E+TRzyMujuEOpN249RirMj9d4gM5z3X3B5xcmiozxpPriuxPtO1HGFG8VWukQf0vdm16mm2Kg1dZFP3dDbYcxUyZDoKXU3v/UTGzWB0ZfSZ78bXXB92vbBfA+mpv0g7ak8mLY+H2YFuYh8CjWNP2yzs08YX2S26cWfBslFeF4gV1DOmUibYBZObbcm34M7Sx5Iw18DfHBJD1to+/Fh80drH+4IRp0PgRfvEbdtRwuAeLhimFe9cnRVcRLKio5bESEmdudEyeKY202I8UsDiUKfI4Y/35hZ/+1S2e244=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(508600001)(83380400001)(4326008)(2906002)(86362001)(31696002)(6486002)(4744005)(2616005)(36756003)(54906003)(316002)(66946007)(31686004)(66476007)(66556008)(26005)(8676002)(186003)(53546011)(16576012)(38100700002)(5660300002)(6916009)(55236004)(6666004)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGFOM01zUUZEV3BYd0hqNllQVytvbU1lTjdEWGFDL1BVRCtyS3VQK0F6TCtu?=
 =?utf-8?B?REtVSTlmaU1ndURHVXYyc3p2azRKNUk2TklXZVM2aUVDVFAzMDREbVlza1JE?=
 =?utf-8?B?alQrY1lHdlc4dVVtd2dyaEllQndtUjZ1V3RFR1JSZ2J6YW5sdlY2L1NqazZz?=
 =?utf-8?B?K1lwOUxpdHI3NGcwK2ZLcGowemtHMmxGOGhpbEVZY1lkYWdnZFZ4Z0NlUkJP?=
 =?utf-8?B?MktteFREcHBSU0tleHo2SHdGY0ZPZks0UGUzU2J4SkhzRXpEd3ZXeDVVRWp1?=
 =?utf-8?B?UGpGVHNqSUtXN2c3aDFXa004SUtaNUNBQ3NPSkRzb3ZEaFAxOXQ2SWtIZkQ2?=
 =?utf-8?B?N2JvcHJ2TittQ3lmeHNMZEZYTnQ1L2RDQ3lDSVg1SzVpdXVzeVVacmtvUTM1?=
 =?utf-8?B?d3dvdGxyZVVBT0J0Y1Q4RkJiVjB4dlhTQUp5aGdjZkZoWjNHZWFxRERQdmZX?=
 =?utf-8?B?VkxiL1J4eWVZaTRkNmNTSlYvVmtpOWhvWG1yK2FLLzRmS2l1L0VqYTZQRHFa?=
 =?utf-8?B?OHV2cFgvaldHQ0VoaVEwVGV6U0VzT045aTcvWHRJeDVLS3ByMnI5WGlsVito?=
 =?utf-8?B?UnFFbXJmVFdIdWxVVXIzZlhJelNSTGNhRjlXWFlZSWNHL2o5VDhJM2hId2p4?=
 =?utf-8?B?SGpKWndkVkVWYjJKeENwVDhXRzVOUGhMd2VJWEduQnVoVkJtSGdVcHMrMVNC?=
 =?utf-8?B?R1VrekF6LzJtNDh5c2xtdE4rcVpxZnlQVXBEUXNGa2twdURYRjdtemtabHpx?=
 =?utf-8?B?S3VVVks3TWJDSThwUFF4dVBNTHBuclpjRnNPNGVJbGYwbzhZaE9uT0JTWXo2?=
 =?utf-8?B?Tk9ZRXc3WGNaR0RVbFVuQ1pPc1YybzJvK2RndE9nbjZIRmllSEtZNlJaNi9j?=
 =?utf-8?B?QWdITlpub2JoSnR1dzU0R2VGLzNaUmR1UHM4VE9yMDFrRzJCbmxreVdERkZl?=
 =?utf-8?B?bkRDbkhoeXJjUktvUXJGZWlvOTZaOG1IZHVlK0cxWXdWcjY5WkhWRmVYUUJv?=
 =?utf-8?B?cWN6dGpKTWJyZlU4aGxuRkd4ei90a25nRUJrbmhSY0VIeWpmcXdZMEJOMEVs?=
 =?utf-8?B?RnFMMXBHWnpLVlczc2NSbTJXVVJkVUZjOVlCZERkRnlUSkJOOVYzdVFMYWEy?=
 =?utf-8?B?V1JZRUNsUXBmOTdEWm1HQ3JRZmErOHVZYXpxOWIzMGxSRlZUcHF2T25hUmEr?=
 =?utf-8?B?THhMYlk0ZHliZWpSWndVc0FQQ1hJVEQ1M0s5c3FWd28yM0w4NXl6QUVka0Iv?=
 =?utf-8?B?cUduNG44b0I0VldGMmlwaUFHcUNpUzlOODlXVW5zckUwL1ZvSVlpazM3UnBp?=
 =?utf-8?B?MHFWSkQvVmxDc0JKRXlZS0R6WGhHbStLaFlFMVpOaGV4NWZiNjlFSHVUK0cr?=
 =?utf-8?B?anFRdE1aclgyTlBmbUwzNDREVnZxRVdqT3pvQUJNd01ON2crc1dWdlZVeEZk?=
 =?utf-8?B?R2kwYk81clVHUmhUbS9XRXZYbWJNUy9DVXZESTFGcUdmMFdDZkNJbTBVcXU1?=
 =?utf-8?B?WG9mT1grelVHTW52Sy8vbGh3QkdtM1IwRC9FOXBRWmhDSmtzSFFwUXdFZlNn?=
 =?utf-8?B?OU5RMlJneFpWU1Z2b0VKTnh2UHhYR2RVSTkyUmYzcmdtV084VGdIcFVnc0tC?=
 =?utf-8?B?S3BidmxFZkdaTWZiTUUxVWFoSHFrTFJ2UFk1RVhKZGRKYUFIcU92WnhSbXN3?=
 =?utf-8?B?SUlXRnMxM0VKZitTZis1a0IrR1hndzk4LzJHV1ZjWUJXdHBnMjI4Rmp3d09P?=
 =?utf-8?Q?BncJH/p2DsXA7IMlL9DPouWhB3Lzohhlut/CtFU?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ade5775-0eb5-4411-5e71-08d979df1e8a
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 13:29:04.8957
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gZewf/Qr+/40JqBiM5gM/JshoLrbMOiuwd4e76SZoNEIKqQmknTIEjBYTuQBTKoj0GNrEe647+0GOEiF2KIXzRO7iZ1DpAG0Eb0XYsZkoXI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3557
X-OriginatorOrg: citrix.com

On 17/09/2021 14:10, Jan Beulich wrote:
> On 17.09.2021 10:45, Andrew Cooper wrote:
>> @@ -1336,13 +1338,16 @@ update_runq_load(const struct scheduler *ops,
>>      {
>>          struct {
>>              uint64_t rq_avgload, b_avgload;
>> -            unsigned rq_load:16, rq_id:8, shift:8;
>> -        } d;
>> -        d.rq_id = rqd->id;
>> -        d.rq_load = rqd->load;
>> -        d.rq_avgload = rqd->avgload;
>> -        d.b_avgload = rqd->b_avgload;
>> -        d.shift = P;
>> +            uint16_t rq_load; uint8_t rq_id, shift;
> Split into two lines? Preferably with this adjustment
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> I'd like to note that the remaining uses of "unsigned int" or plain
> "int" in some of the instances you don't touch assume
> sizeof(int) == 32, while generally we assume only sizeof(int) >= 32.
> This is one of the cases where fixed width types are imo mandatory
> to use.

See patch 5.  There was far too much cleanup to merge with this patch.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 14:09:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 14:09:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189442.339146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREY8-0001zu-TZ; Fri, 17 Sep 2021 14:09:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189442.339146; Fri, 17 Sep 2021 14:09:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREY8-0001zn-QE; Fri, 17 Sep 2021 14:09:08 +0000
Received: by outflank-mailman (input) for mailman id 189442;
 Fri, 17 Sep 2021 14:09:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jmcU=OH=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mREY7-0001zh-Ve
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 14:09:08 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fb8ec25e-8fbe-4602-b83e-833a312ac6b2;
 Fri, 17 Sep 2021 14:09:06 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id i4so33588863lfv.4
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 07:09:06 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id z3sm587071lfe.50.2021.09.17.07.08.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 Sep 2021 07:08:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb8ec25e-8fbe-4602-b83e-833a312ac6b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=876uV0C2rcWmwrVuv5sB0QCkT2DUPnwIqbxAymkbhGY=;
        b=I0354vm+ZXAn9OXlZSPSjslFLeojEIM6bJ3IUjMQyBurtJMsGtgHJuvTEOeN5qq+2y
         da6kMnMRke9I1+S96RODEc9Km9BC73WDrrPkTISxuE15KISsVgLf2GuR9jDZxXNZqhTb
         mkXwJeYt1ZEx2p7mruFv2L450X55LFz19VMcRHnhJTA96KjQGfQEMwcisrx4fTvEBCAV
         Ir8mRxORZNRQuYX7A1EuqEKr3bmQdZQjliZVQA3nQ/4c7vAtiVXYNyaZaZJ6TNyn55S8
         1OLJgCLLZSQTK9751uayYzZIawLJ4K++P10oDA8Q1r9/tcnwi+mBdM4059xouio8Yvbz
         41sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=876uV0C2rcWmwrVuv5sB0QCkT2DUPnwIqbxAymkbhGY=;
        b=LBM9BRkt/7ftb+jT77PBq5DGqNc3HU5R8oVMTgp/XMWs04sXsZsaPYmRpa87HBw2RI
         L+RnoCNAf3lBcnPRONypERYbJKfwId9PzIVXSBEbca1BZ+uki3TfxeqhaGGwlrOnvvg9
         KUwFY3GFz2zEEbsjDDz+w2GSFzqJjyEXCM0GCXCovsKXgU3EDXkzlVK/UP1JmQbvhwPO
         1emyY3ntaDKo/NmduwZgCfcPGUHlSdxMvj55MNMEiWU86XJeD4A7GPZ52c1hk5/KHulg
         5uxFZvhQxVCYML5qTTTvOotKCWssG+zsh1j5bdoY5UaxF/sGwg9ZjN5UZtc8SuuCn9Xa
         CRzA==
X-Gm-Message-State: AOAM531vFXVPgF+NKCTqg2tw/C/SVBA8XurRAsXYyMeXqvIcrZ/J6zcC
	EHHIoCSs37Mrf2oqGhFiOl4=
X-Google-Smtp-Source: ABdhPJxFLHTY84Sh7J4RvIVU6Xge5i6CGjqon6egl6nUELnCNTRJI6nu0pAppWEHR43MnVdblg5qQQ==
X-Received: by 2002:a2e:2417:: with SMTP id k23mr9534031ljk.256.1631887713242;
        Fri, 17 Sep 2021 07:08:33 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
From: Oleksandr <olekstysh@gmail.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
 <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
Message-ID: <27da6f7a-5bb5-daa3-4012-60fc598e0f6b@gmail.com>
Date: Fri, 17 Sep 2021 17:08:30 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 15.09.21 22:10, Oleksandr wrote:

Hi Stefano.

[snip]
>
>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>> +                                    struct meminfo *ext_regions)
>>> +{
>>> +    struct dt_device_node *np;
>>> +    struct rangeset *mem_holes;
>>> +    paddr_t start, end;
>>> +    unsigned int i;
>>> +    int res;
>>> +
>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>> +
>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>> +    if ( !mem_holes )
>>> +        return -ENOMEM;
>>> +
>>> +    /* Start with maximum possible addressable physical memory 
>>> range */
>>> +    start = EXT_REGION_START;
>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>> +    res = rangeset_add_range(mem_holes, start, end);
>>> +    if ( res )
>>> +    {
>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>> +               start, end);
>>> +        goto out;
>>> +    }
>>> +
>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, 
>>> etc) */
>>> +    dt_for_each_device_node( dt_host, np )
>> Don't you need something like device_tree_for_each_node ?
>> dt_for_each_device_node won't go down any deeper in the tree?
>
> Thank you for pointing this out, I will investigate and update the patch.

I have checked, dt_for_each_device_node( dt_host, np ) iterates all 
nodes, so nothing will be skipped.

As an example for this node:

         hdmi@fead0000 {
             compatible = "renesas,r8a7795-hdmi", "renesas,rcar-gen3-hdmi";
             reg = <0x0 0xfead0000 0x0 0x10000>;
             interrupts = <0x0 0x185 0x4>;
             clocks = <0xc 0x1 0x2d9 0xc 0x0 0x28>;
             clock-names = "iahb", "isfr";
             power-domains = <0x9 0x20>;
             resets = <0xc 0x2d9>;
             status = "okay";
             iommus = <0x50 0xc>;
             xen,passthrough;

             ports {
                 #address-cells = <0x1>;
                 #size-cells = <0x0>;

                 port@0 {
                     reg = <0x0>;

                     endpoint {
                         remote-endpoint = <0xb1>;
                         phandle = <0xc1>;
                     };
                 };

                 port@1 {
                     reg = <0x1>;

                     endpoint {
                         remote-endpoint = <0xb2>;
                         phandle = <0xd1>;
                     };
                 };

                 port@2 {
                     reg = <0x2>;

                     endpoint {
                         remote-endpoint = <0x6f>;
                         phandle = <0x6e>;
                     };
                 };
             };
         };


(XEN) process /soc/hdmi@fead0000
(XEN) ---number_of_address = 1
(XEN) -------0: 0xfead0000->0xfeae0000
(XEN) process /soc/hdmi@fead0000/ports
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@0
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@0/endpoint
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@1
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@1/endpoint
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@2
(XEN) ---number_of_address = 0
(XEN) process /soc/hdmi@fead0000/ports/port@2/endpoint
(XEN) ---number_of_address = 0




[snip]

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 14:22:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 14:22:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189452.339160 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREkw-0004Oo-6O; Fri, 17 Sep 2021 14:22:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189452.339160; Fri, 17 Sep 2021 14:22:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREkw-0004Oh-34; Fri, 17 Sep 2021 14:22:22 +0000
Received: by outflank-mailman (input) for mailman id 189452;
 Fri, 17 Sep 2021 14:22:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ph2s=OH=tklengyel.com=tamas@srs-us1.protection.inumbo.net>)
 id 1mREku-0004Ob-VD
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 14:22:20 +0000
Received: from MTA-06-4.privateemail.com (unknown [198.54.122.56])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id aaa55fce-17c2-11ec-b6ba-12813bfff9fa;
 Fri, 17 Sep 2021 14:22:20 +0000 (UTC)
Received: from mta-06.privateemail.com (localhost [127.0.0.1])
 by mta-06.privateemail.com (Postfix) with ESMTP id 3E1401800209
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 10:22:19 -0400 (EDT)
Received: from mail-wr1-f51.google.com (unknown [10.20.151.230])
 by mta-06.privateemail.com (Postfix) with ESMTPA id 254A41800200
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 10:22:19 -0400 (EDT)
Received: by mail-wr1-f51.google.com with SMTP id t18so15549490wrb.0
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 07:22:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aaa55fce-17c2-11ec-b6ba-12813bfff9fa
X-Gm-Message-State: AOAM53041YfH5HfKwI3+5hbkl5jUPFODplhy5jQeqyYLEWDqnNshk9sz
	z+ZAM0mZ7qKJSlRlifja4WkOxrYpfQRybI157r0=
X-Google-Smtp-Source: ABdhPJwyY6jEs4fX6nDZzhwypKgH6lJhUFFf92sguZ1vRcBsb8Y/SI0mAz2NX0eMVkH5dpPS9rrXHpMQRS1GzNeocbc=
X-Received: by 2002:adf:e6c9:: with SMTP id y9mr12616102wrm.429.1631888537841;
 Fri, 17 Sep 2021 07:22:17 -0700 (PDT)
MIME-Version: 1.0
References: <c21708c84c850ff8c976a3934099c54da044e7d9.1631802816.git.tamas.lengyel@intel.com>
 <52b28c4d-1cf1-6c98-e1dc-1e0f7b2f768c@suse.com>
In-Reply-To: <52b28c4d-1cf1-6c98-e1dc-1e0f7b2f768c@suse.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 17 Sep 2021 10:21:41 -0400
X-Gmail-Original-Message-ID: <CABfawh=B3zwfroiu=pPY5BTCMBkapYEGXixkRg66aXLQWxwU4A@mail.gmail.com>
Message-ID: <CABfawh=B3zwfroiu=pPY5BTCMBkapYEGXixkRg66aXLQWxwU4A@mail.gmail.com>
Subject: Re: [PATCH] x86/mem_sharing: don't lock parent during fork reset
To: Jan Beulich <jbeulich@suse.com>
Cc: Tamas K Lengyel <tamas.lengyel@intel.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
X-Virus-Scanned: ClamAV using ClamSMTP

On Fri, Sep 17, 2021 at 3:26 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 16.09.2021 17:04, Tamas K Lengyel wrote:
> > During fork reset operation the parent domain doesn't need to be gathered using
> > rcu_lock_live_remote_domain_by_id as the fork reset doesn't modify anything on
> > the parent. The parent is also guaranteed to be paused while forks are active.
> > This patch reduces lock contention when performing resets in parallel.
>
> I'm a little in trouble following you here: RCU locks aren't really
> locks in that sense, so "lock contention" seems misleading to me. I
> can see that rcu_lock_domain_by_id()'s loop is extra overhead.
>
> Furthermore - does the parent being paused really mean the parent
> can't go away behind the back of the fork reset? In fork() I see
>
>     if ( rc && rc != -ERESTART )
>     {
>         domain_unpause(d);
>         put_domain(d);
>         cd->parent = NULL;
>     }
>
> i.e. the ref gets dropped before the parent pointer gets cleared. If
> the parent having a reference kept was indeed properly guaranteed, I
> agree the code change itself is fine.
>
> (The sequence looks correct at the other put_domain() site [dealing
> with the success case of fork(), when the reference gets retained]
> in domain_relinquish_resources().)

This code above you copied is when the fork() fails. Calling
fork_reset() before fork() successfully returns is not a sane sequence
and it is not "supported" by any means. If someone would try to do
that it would be racy as-is already with or without this patch.
Clearing the cd->parent pointer first here on the error path wouldn't
guarantee that sequence to be safe or sane either. Adding an extra
field to struct domain that signifies that "fork is complete" would be
a way to make that safe. But since the toolstack using this interface
is already sane (ie. never calls fork_reset before a successful fork)
I really don't think that's necessary. It would just grow struct
domain for very little benefit.

Tamas


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 14:24:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 14:24:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189460.339171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREmc-00051L-Hk; Fri, 17 Sep 2021 14:24:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189460.339171; Fri, 17 Sep 2021 14:24:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mREmc-00051E-EF; Fri, 17 Sep 2021 14:24:06 +0000
Received: by outflank-mailman (input) for mailman id 189460;
 Fri, 17 Sep 2021 14:24:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mREmb-000512-8Z; Fri, 17 Sep 2021 14:24:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mREmb-0008M7-0c; Fri, 17 Sep 2021 14:24:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mREma-0001oS-Mg; Fri, 17 Sep 2021 14:24:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mREma-0004i0-MB; Fri, 17 Sep 2021 14:24:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pGBAyKy0JmrJy99pMZgV2YZVINovOol7PfU0w9xXI+0=; b=axFqDSiLnxMsLlwuZTGUWffP4C
	5MeG3j6Ipy/UqETflRiiNlEuntBG6ipgEPrvGPMG2gxOOonl4U5KAOlFCELZptpd+ntgvTLMDzI0B
	pT99sVF+ThsYs6q+hNR+20xkkmCCH8RYtL+xPON0XFw2cjeBvads2w2fvzAqski+NLQ4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165017-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165017: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-stop:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=d1fe59377bbbf91dfded1f08ffe3c636e9db8dc0
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 14:24:04 +0000

flight 165017 qemu-mainline real [real]
flight 165067 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165017/
http://logs.test-lab.xenproject.org/osstest/logs/165067/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     17 guest-stop               fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                d1fe59377bbbf91dfded1f08ffe3c636e9db8dc0
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    5 days
Failing since        164967  2021-09-13 13:06:52 Z    4 days    8 attempts
Testing same since   165017  2021-09-16 21:38:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2898 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:46:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:46:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189492.339230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4a-0005qo-Gj; Fri, 17 Sep 2021 15:46:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189492.339230; Fri, 17 Sep 2021 15:46:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4a-0005qh-Bn; Fri, 17 Sep 2021 15:46:44 +0000
Received: by outflank-mailman (input) for mailman id 189492;
 Fri, 17 Sep 2021 15:46:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG4Y-0005qb-FB
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:46:42 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d4d07b8-3706-44ed-9b18-3ac40607897d;
 Fri, 17 Sep 2021 15:46:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d4d07b8-3706-44ed-9b18-3ac40607897d
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893601;
  h=from:to:cc:subject:date:message-id:
   content-transfer-encoding:mime-version;
  bh=PGUmvzpBb5O/wQ2yH12dj3ZbbsJPWrk2hWzsrNzemoI=;
  b=BRfOnopOZe76wLRBTyR9CN69WVOS9bWY0eoTtgii1jfvralWT7ZR8EcM
   Wmk8l1BM2ruIM2BEgo07+T0iKE70NXgBrR8EYgmzS8QPlpwSIIRt80xI/
   hUccGfJJTTsIuSQcl/qM6Yedidw/1hZZEVOBYiKNN4Q7l35whSqjaWLGE
   8=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: yR0fiyfSD5z6IqPd7DEjuanhF4Md+MKP4hQkHjTfPSCkf6ARIx70CN6o3YtDMdJzznqPAw2/cU
 b/qEMdZZXwUckdWLKL4dttPSZfTFlRRyo0lpUqsaxCTRApJCfxewlvoPaKxHf4WW9lmB3hGluP
 96AM9Vwb2YqHX6W6FE/wAfjTbm1dxR+Do9wTBMPlvUcM34/mu2safN6eJdpeqyWVrSrf7HXG0M
 JRJyY+baKy56ZvDFQe3/VxrG1tF0Rke8Dpdl0yfbrGPzjAp/vKbhFh4zpQpai3v90KvI+e/d0E
 JixZff17f/7FX7YBO3GQeeNw
X-SBRS: 5.1
X-MesageID: 52991220
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QJyoN60IRbn+jHXE5vbD5X92kn2cJEfYwER7XKvMYLTBsI5bpzQGx
 mZMWzuPM6zcN2T3e41zYIu18k0Pu8DSxtJgHVNspC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrFh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhkOJ/5
 pZglZKJFA42GIjimesMAisIHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIUJjW5q3Jgm8fD2O
 tESMApqQT/8SjJKYXQ6DowBmv6aryyqG9FfgA3M/vdmi4TJ9yRz36LqK8H9YcGRSINemUPwj
 nnd423zDxUeNdqe4TmI6HShgqnIhyyTcLwVELq05/t7mmq5z2YYCAAVfVajqPz/gUm7M/pTI
 lIZ0jAjpq8z8AqsVNaVdx+lpH+JuDYMVtwWFPc1gCmM1aj88wufHnIDTDNKdJohrsBebSwn0
 BqFks3kARRrsaaJUjSN+7GMtzSwNCMJa2gYakcsTwEI/t3iq4EblQ/UQ5BoF6vdpsLxMSH9x
 XaNtidWulkIpZdVjePhpwmB2m/y4MiSJuIo2unJdm2X4C06Sa6rXtGh9n7+w/dPPderHlbU6
 RDohPOiAPAy4YClzXLWGbVWTeDxvZ5pIxWH3gU+RMBJGyCFvif5JNEOumkWyFJBb55cEQIFd
 nM/ru+4CHV7B3KscaY/SIa4Ec1CIUPIRIm9C6y8gjajZPFMmO67EMNGPhX4M4PFyhFEfUQD1
 XCzK53EMJriIf47pAdavs9EuVPR+szb+Y80bcugpylLLJLEPCLFIVv7GALWM4jVE59oUC2Kq
 o0CZqNmOj10UfHkYzm/zGLgBQlRdhAG6WTNg5UPLIare1M+cEl4UqO56e5xKuRNwvUO/s+Vr
 y7VZ6Ot4Aem7ZExAV7RMS4LhXKGdcsXkE/XygR2bA70hCB8MdjwhErdHrNuFYQaGCVY5accZ
 9EOetmaA+QJTTLC+j8HaoL6opAkfxOu7T9i9QL8CNTmV5I/FQHP5PH+eQ7jqHsHAiat7JNsq
 Ly8zALLB5EEQl06XsrRbfuuyXK3vGQcx70uDxeZfIELdRW+6pVuJgzwkuQzf5MGJyLcy2bIz
 A2RGxoZ+7XA+tdn7NnTiKmYhI61CO8iTFFCFmzW4O/uZynX92au26FaV+OMcWyPXW/44vz6N
 +5U0+v9ILsMm1MT69hwFLNizKQf4drzpuAFklQ4TSuTN1nyU+FuOHiL28VLp5Zh/L4BtFvkQ
 F+L9/lbJa6NZJHvHmkOKVd3de+Ez/wVxGXftKxnPEXg6SZr17ObSkEObQKUgSlQIbYpYoMox
 eAt5Jwf5wCl00d4N9+HimZf9niWL2xGWKIi78lIDIjugwst61dDfZ2DVXOmvMDRM41BYhswP
 zuZpKvenLANlEPNfk06GWXJwecA148FvwpHzQNaKlmE8jYfaiTbAPGFHewLczlo
IronPort-HdrOrdr: A9a23:ds0jWaOKZt6OiMBcTyT155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/exoX5PwO080lKQFmrX5WI3NYOCIghrPEGgP1/qB/9SCIVyAygc+79
 YYT0EWMrSZZjIb/KWKnjVQe+xQvOVvm5rY4ts2oU0dKD2DPMpbnnpE40ugYwZLbTgDIaB8OI
 uX58JBqTblUXMLbv6jDn1Ae+TYvdXEmL/vfBZDXnccmUSzpALtzIS/PwmT3x8YXT8K6bA+8V
 Ldmwi8wqm4qfm0xjLVymeWxZVLn9nKzMdFGaW3+4IoAwSprjztSJVqWrWEsjxwiOaz6GwymN
 2JmBskN9Qb0QKmQkiF5T/WnyXw2jcn7HHvjXWCh2H4nMD/TDUmT+JcmINwaHLimgQdleA59J
 gO83OStpJRAx+Ftj/6/cL0WxZjkVfxiWY+kNQUk2dUXeIlGfBsRLQkjQNo+ao7bWHHANhNKp
 gtMCic3ocbTbqiVQGZgoE1q+bcG0jaHX+9Mz8/U4KuonxrdN0Q9Tpr+CUlpAZxyHsKcegx2w
 31CNUYqFhwdL5kUUsEPpZ4fSKWMB2FffueChPbHbzYfJt3Tk4l7aSHp4kI2A==
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="52991220"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZUh5kXr6VKYOU4qhsdIlySm8bKO0XklUPBM5WEcxzSKa1mjeHCgA9DSMvBBipNfr0lfkHX5CDqd9BblbVSGKsI2IC6n/zTEjbLPxpNkxI562cGLOoZf9IYTfoaH1/l/COuQYGG9lpel/xxNZwxrl2fOILyh61Vrj+elqJfUD8US6IGUT/P/bhJgS49HUE2/2HWg8XrXqbjrcRwZL0Ma8jA8NxqBZ+OI0XxIsZ3PVVAvpbaCGWuZTnOP7FHoG320VO3R+fWDo0cP6dpZgilKFTCpjA6tV0SxvFv0NpMLWHB4wdfLJwuTo41cezKD0rUxjt1+vZyMFDq5vHbJj2qVvlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p1G4HGjwj7EyEDiHSa/jjXUT15nvGsX+uxA+Os5xktY=;
 b=eVUQoFpgBFqoKUbKanmiyiTyMnUSYbsVjrE3Q8jg3Pu1Wefmmp6a28PG4ygNNvl0AnuZ5EQLYcNZ3e3r6mN1vLmVcXtllNCgHNQPRB8vomUBioyR9Bq1OdtRk07IYdE534PG+5HvlIq9ncQXIbkvzkRCZfXxzhfKwYlxiFN9vNlPV1AEOuqxB9nxe0K9ZYN8OI7FRLkUC1v3/5jvTxljVAHTAJwGsJMg6ce+CTyBu3YsQrnIw2PQs/Z+iZaT+Octyidv9ZxjKmhrhZr0lX2OHLsB69wginlQcsd4HnpLbjn8v2XvY9KDocqjdtSXCeuUwCUkp4J0MrtlRjDKWmEqbw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p1G4HGjwj7EyEDiHSa/jjXUT15nvGsX+uxA+Os5xktY=;
 b=gDQ5gmxCUiCjT1G1KpcNSTKi/ETS++PtGFVicyNrz25doQpD8NEQ5t6xXIWmv/5D28uaG5a5lj79b+rdPIYinXWtdycSSolW8bU5HbPaxC5lUyqLvmvYxojh7KP6W30wJQDHVhZFBsyueQrGzua1DqWef1DfT+gCLQoqK54XanM=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH 0/6] gnttab: add per-domain controls
Date: Fri, 17 Sep 2021 17:46:19 +0200
Message-ID: <20210917154625.89315-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P123CA0090.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::23) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7ffa6118-e16d-4ee2-ef1b-08d979f25616
X-MS-TrafficTypeDiagnostic: DM4PR03MB6000:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM4PR03MB60005B5B16CF079CBC038AB08FDD9@DM4PR03MB6000.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: z9Ng+5tqNzIMrni9IA2JnlfEYKmhrfIa0v+dFIFqXCdBf0ol4753SzbRCntFwmarqgZhFBBQ9hQxHvMEMDKPYSbMjJRtcCz3iY1gelVZALPoHtJuhepTrC+80BlrK3opjGFmb/3qDt60gmvu7cALIVYD2bc4sCfEeKWaj8cJis+HOJdLMJCTw4ZMyiMwXXNwMRGLT1qn3Px1CyEzBNr6yNQo+3AQK/qgHcLrICKq6KdrxzKM6ZDkAaU6R9CuDdno150SfKf9ONBv5VklakSfDbVPij+Nrndp5+AOFjp3iU7eoCwCLNWZgYyzbtNegdKq0pjznVc+1Q+D03DZ07LxrNtwPzaP9exmumAJTrznoJE6JZNTsbefXt8EnbhYdK/DLkkwStYZ81f0kZ8iZAtIKVU+MB869TnTuI82lVhkBrRT3wXZPw1GNqsdoljzlwnrZ13v1kEhgtwpldiLkm366x5/XGrdPsPaxnd6s93WE5bsFnAeSfZBe0yzaS3pHod4hRKUamm2i1x1yv1q73P93dWVN04zTAlBv1RktPN1LOQvT9KmW5dJsU7/3S9GBjMqXlni4hUPGDdUai4wyRqgVH4Rijql4D6+gCdD5VHJ8wrSD/QVbxRW7YYYMY/M94FVRtXrJs1HRMlkkF7wxOWtfQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(396003)(366004)(136003)(4326008)(2906002)(186003)(478600001)(6486002)(2616005)(6666004)(956004)(1076003)(66556008)(36756003)(66476007)(8676002)(54906003)(86362001)(66946007)(316002)(6496006)(8936002)(38100700002)(5660300002)(6916009)(83380400001)(26005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?REF6VlBXMEl3emdUQmVzMjIvOEdGZlZYSFJMVThZSnBhZCtGMS81VER0dDdC?=
 =?utf-8?B?V3c3S29QK1A5MmN2TWMrRUx2S2tPaG1zaUtDV2VpT2V0dVJNbjc2d1FSejVl?=
 =?utf-8?B?aGdML3RzekdXWnJ0Y0ZLejI1aE9qUTVRb0pTRGNJL2d3WUZ4WGNlN3d4YmFQ?=
 =?utf-8?B?a21JNDk2VVM1OWJaNS93a3pBL1AxR0JscTdiams4RE1PRm1UQ2xpYmNDUGc4?=
 =?utf-8?B?MnpMUjdjSjdOTERlSDNJNmRWOVRyVHljQ29ybldmcGxJZXZ6UFUwVVdNN0d1?=
 =?utf-8?B?TzBzcEkzVUZBdERPZEVLeVlndGtFUFl3MVhTUkNqTld3NDJEYytxOEpGV2xH?=
 =?utf-8?B?UmNESzJ0cDQvVW9rVGYyZlB4MjRTSkZQVkJuVldTVHRJYlF1VVJzN3dvc2lM?=
 =?utf-8?B?cDBQcENzSGRMcXRqTmlhWFRNRGhNanB6ekcvUVp5bHRiYzNSVXllQitHODdi?=
 =?utf-8?B?ZzFQQVRTZTlmZGZmVWRLVWN1REl3ajdVazBhUC9aUTAzaXorRDVEc0pLSCtD?=
 =?utf-8?B?SUZvb29KbUdBcVlPWUJ3dmRxazB2cjkrOHBTYmVlWnJOTE9SNHI3Nk5XT3pS?=
 =?utf-8?B?S2JtSXlkSG9HcnlLOFc3REh0UE40QStoQ2pWcFlkZWV2YXNXTjJSaUVPUi84?=
 =?utf-8?B?NUlma3FJN2podk4xQm11MytBWjdOUG9lOTh4bCtiVnZtQjVqOGQwY2c1WW5o?=
 =?utf-8?B?WGFZRWdlaVMwUzlVbExEMGVhbStWeUNMTVZNU0VHRVB0ZVVyL3VHTUpyUkpG?=
 =?utf-8?B?eUliZE5VN2RySlFraUhXblJGOW52ZXV2U0t3YjJQaUY5L1B6ZktNTDlMT0tD?=
 =?utf-8?B?QzAxNGVoOUhSbHdNZERpVjc1SHg5d1Vack0zamtrWXpHSUgrQzJJa2c3SitT?=
 =?utf-8?B?NDM5ZFVEZFl6bXdKWEllZkZVOWUzUWpTL0ZzeWtiYjN3SHhpZ3FQVEF3UHJ5?=
 =?utf-8?B?ZGlJTEFYdW03RzNBVGE0dlFYNW1MNUdHdUs3eGgxWXVSSnJWWjQ1OVdFdEw2?=
 =?utf-8?B?S3EvK3hGbjAxSFNHajZnVDEvN0M3cmhraGV4UWkxNFZRa0tDM0JkNlpTWVR5?=
 =?utf-8?B?KzZEd1VIZXV6QUpTRlU0eTgzMDNFa1ZzZ09neHVLT051cDZseUwwRGVlQTc3?=
 =?utf-8?B?OHM3M3hSa2xER0VTWndtNTgwMmF6bEVmVERIMlJHRk4vVGcyZUNOYWNlbUZr?=
 =?utf-8?B?bnZUcW54K3RMczNXYjlkRDR5dWEza2VGL3pMdDFJV2tTdVFuK3o3Y1p0M3RU?=
 =?utf-8?B?WXBWY3FBc29XNzNyNUY1SjZRL0dQRXJ1NXRicmlPSzRobkM1cE84N3F5MUJI?=
 =?utf-8?B?TUQvNGpHZnpqUU03Y3QzcmRFQmw3Y2xYeFpyd3dnZkFHaEhFOWZxajBGczE0?=
 =?utf-8?B?a2FINmhLa0YvSXVpMkZZSXVCeEhVa3ZJVWpCKzI2UTk3dmxxYVFTN0pBVUti?=
 =?utf-8?B?K2h2TmNPdTlkeGNYb1BZRVpobkkrRkhFNWNxZmdFTy9KRGFhWHlFM0U4ejRk?=
 =?utf-8?B?dmJaNDBUY3V4WTh6WnBTUCtnaHlsRm5mNGJHMjV3NEQ2Z015NDlvSVVqU1kv?=
 =?utf-8?B?dnBoVkZsUkxIOWdxcGdnWjNvbm5OU1dXc01HRTk2WTBvTU1YY3AxdENMU2Ex?=
 =?utf-8?B?M2V4eFd0dlBxL2Exbis0RWhYeS9TMnhkYm1JNDcwOGVuSUszdUFvTFlSQTNt?=
 =?utf-8?B?ek5oYmQ0bXZXQzU0S1F0dXdJQ2pRU3pjY2dJKzhGdUlrYSt6YTVrdUxRalhU?=
 =?utf-8?Q?/NnpfOHhqdtTD8LCCEXfweKga3ePodSam2vwQY5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ffa6118-e16d-4ee2-ef1b-08d979f25616
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:46:38.4608
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EQ/w264YLgtY3j8NhzZkDvTypZ5sEmE3ieTfBhHPTxzuENuzOxHVjzCTpr5n9vF/r6ORDrJADADEbYFN7Mq22w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6000
X-OriginatorOrg: citrix.com

Hello,

The first two patches of this series allows setting the preisoutly host
wide command line `gnttab` option on a per domain basis. That means
selecting the max allowed grant table version and whether transitive
grants are allowed.

The last 4 patches attempt to implement support for creating guests
without grant table support at all. This requires some changes to
xenstore in order to map shared ring using foreign memory instead of
grant table.

Note that patch 5 will break the save format for xenstore records, and
should not be applied.

Thanks, Roger.

Roger Pau Monne (6):
  gnttab: allow setting max version per-domain
  grant: allow per-domain control over transitive grants
  tools/console: use xenforeigmemory to map console ring
  tools/xenstored: use atexit to close interfaces
  tools/xenstored: restore support for mapping ring as foreign memory
  gnttab: allow disabling grant table per-domain

 docs/man/xl.cfg.5.pod.in                |  12 +++
 docs/man/xl.conf.5.pod.in               |  14 +++
 tools/console/Makefile                  |   4 +-
 tools/console/daemon/io.c               |  25 ++++-
 tools/helpers/init-xenstore-domain.c    |   1 +
 tools/include/libxl.h                   |  14 +++
 tools/libs/light/libxl_create.c         |   4 +
 tools/libs/light/libxl_dm.c             |   2 +
 tools/libs/light/libxl_dom.c            |   2 +-
 tools/libs/light/libxl_types.idl        |   2 +
 tools/ocaml/libs/xc/xenctrl.ml          |   5 +
 tools/ocaml/libs/xc/xenctrl.mli         |   5 +
 tools/ocaml/libs/xc/xenctrl_stubs.c     |  12 ++-
 tools/xenstore/Makefile                 |   4 +-
 tools/xenstore/include/xenstore_state.h |   1 +
 tools/xenstore/xenstored_core.h         |   2 +-
 tools/xenstore/xenstored_domain.c       | 120 +++++++++++++++---------
 tools/xl/xl.c                           |  15 +++
 tools/xl/xl.h                           |   2 +
 tools/xl/xl_parse.c                     |  13 +++
 xen/arch/arm/domain_build.c             |   4 +
 xen/arch/x86/setup.c                    |   2 +
 xen/common/domain.c                     |   4 +-
 xen/common/grant_table.c                | 119 ++++++++++++++++++++++-
 xen/include/public/domctl.h             |  13 ++-
 xen/include/xen/grant_table.h           |   7 +-
 26 files changed, 341 insertions(+), 67 deletions(-)

-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:46:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:46:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189493.339241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4k-000692-OO; Fri, 17 Sep 2021 15:46:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189493.339241; Fri, 17 Sep 2021 15:46:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4k-00068t-Kn; Fri, 17 Sep 2021 15:46:54 +0000
Received: by outflank-mailman (input) for mailman id 189493;
 Fri, 17 Sep 2021 15:46:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG4j-00068c-Vs
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:46:54 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6b0d47b6-8517-4a4f-8811-acbd96c09264;
 Fri, 17 Sep 2021 15:46:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b0d47b6-8517-4a4f-8811-acbd96c09264
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893611;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=mOE0yL5slFfYU8+bdX/AoH1hs+X1pgLMgHslV6f/8Xw=;
  b=W4CBNh7lJrOB3u1ogW3QWLwsc3djUiWjt/r8deIlSn5fXRdRDCiihkjK
   DH9PwAtL8j5caYbmtlEgl0r/DH9dV5z9MF5SaBE4SLivtCRyQ7SL0wlgn
   UH/pA86xCZJYNwknz4yHdOl7Cam1Y4SnSMExQnZBtJ5onbcP+EsObkGWz
   I=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: ww4bUqDkPIdjUfB0EqodsCtqGwj5JsHL672fvGGXY1Gr6GH01PqO9yF8DEzFqtXZ77S8y1oKQZ
 9UzI5Fs4x8mENwM8jF9z7sOdRQ6w2nzMn6npXkgXBR7OSzvcp2mse9t6A/KcGWPLNapeXTaFDu
 HOC7k29itaYwSfFovaVyVPaLTFFbXx/SeUWCClfu17LXtGdeCJ1yM8t+x23GKfGY53YO5rYWfb
 EgMTWgMfSp8NwJlnceDkouwxltZAnTHuwgzkuEc2nLtmZ2BvYjr3EFTlK3GRxPwGfHIIjPTkyr
 nPzsGJEZYFzZobUmaMnmHDKn
X-SBRS: 5.1
X-MesageID: 53016148
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:tu6asaywVcQXNIwMucV6t+ebwSrEfRIJ4+MujC+fZmUNrF6WrkVRm
 2AdCm+EbveJa2Khe4snO4S080oH657Xz4diTQU6pCAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrdm2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt8w25
 M5zub+3dR8SMp/gxaM6awVFTwgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVort4kI8TxepsWp1lrzC3DDOZgSpfGK0nPzYIDhG5u35wedRrYT
 +s0dTtxMSiRWhpCHVQeKbVuobiwxUCqJlW0r3rK/PFqsgA/1jdZ0rf3N8HOUseXXsgTlUGdz
 krd5HjwCBweMN2ZyBKG/2iqi+uJmjn0MKosE7m/+u9vkUek7GUZAx0LVnO2ufC8zEW5Xrp3J
 0UK9wI0oKM19UjtScPyNzW6vXqFsxg0S9dWVeog52mly7XWyxaUAHAeSTxMY8Bgs9U5LRQ62
 1nMk973CDhHtLyOVWnb5rqStSm1OyUeMSkFfyBsZQkP7sTnoYozpgnSVdslG6mw5vXqHRngz
 jbMqzIx74j/luZSif/9pwqexWvx+N6ZFWbZ+zk7QEq5qS9iWoKGYLCw0lTpwasHC76FUwWo6
 S1sd9el0AweMX2cvHXTG7xdR+D4v6zt3C702gE0TsJ4n9i50zv6J9kBvmsmTKt8GptcIVfUj
 FnvVRS9DXO5FECja7N+K6m1At4jpUQLPYW4Dq2IBjaij54YSeNmwM2MTRXLt4wOuBJ1+U3aB
 Xt8WZz3ZUv28Yw9kFKLqx41iNfHPBzSIF8/origlHyaPUe2PibJGd/pznPXNr1RAFy4TPX9r
 I8EapriJ+R3e+zieCjHmbP/3nhTdiNTOHwCkOQOLrTrClM/QAkJUqaNqZt8K90Nt/kEzY/go
 yDiMnK0PXKi3BUr3y3RMSs9AF4uNL4ixU8G0dsEZwfwhyd7Ptbxs8/ytfIfJNEayQCq9tYtJ
 9EtcMScGPVfDDPB/jUWd57mq4J+Mh+sgGqz0+CNOVDTprZsGF7E/MHKZAzq+HVcBya7r5Jm8
 bahyhnaUdwIQAE7VJTab/emzlWQu3kBmb0tAxuUc4cLIEi8opJ3LyHRj+MsJ51eIxv02TbHh
 R2dBg0VpLeRrtZtosXJn62Ns6ygD/B6QhhBB2De4LvvbXva82OvzJVuSuGNeTyBBmr49L/7P
 bdezu3mMe1Bl1FP6tIuH7FuxKM4xt3uu74FkVg0QCSVNwymU+oyLGOH0M9Dsrx26oVY4QbmC
 FiS/tR6OKmSPJ+3GlAmOwd4PP+I0usZm2eO4K1tcln6/iJ+4JGOTV5WY0uXkCVYIbZ4bNElz
 OMmtJJE4gCzkEN3YNOPjyQS/GWQNH0QFa4gs8hCUoPsjwMqzHBEYIDdVXCqsM3eNY0UPxl4O
 CKQiYrDm69YlxjLfHcEHHTQ2fZQ2MYVsxdQwV5ef1mEl7IpXBPsMMG9Jdjvcjlo8w==
IronPort-HdrOrdr: A9a23:vw842K92RLk/tJ5CrYZuk+FJdb1zdoMgy1knxilNoENuHfBwxv
 rDoB1E73LJYVYqOU3Jmbi7Sc29qBTnhOJICOgqTMqftWzd1ldAQ7sSi7cKrweQeREWs9Qtrp
 uIEJIOeeEYb2IK9PoSiTPQe71LoKjlgdCVbKXlvg1QpGlRGt9dBmxCe2Gm+yNNNWx77c1TLu
 vi2iMLnUvsRV0nKuCAQlUVVenKoNPG0LrgfB49HhYirC2Dlymh5rLWGwWRmk52aUIB/Z4StU
 z+1yDp7KSqtP+2jjfaym/o9pxT3P/s0MFKCsCggtUcbh/slgGrToJ8XKDqhkF4nMifrHIR1P
 XcqRYpOMp+r1vXY2GOuBPonzLt1T4/gkWSgWOwsD/Gm4jUVTg6A81OicZyaR3C8Xctu9l6ze
 Ziw3+Zn4A/N2KBoA3No/zzEz16nEu9pnQv1cQJiWZEbIcYYLhN6aQC4UJuFosaFi6S0vFnLA
 BXNrCf2B9qSyLeU5iA1VMfhOBEH05DUCtue3Jy+fB8iFNt7TdEJ0hx/r1Xop5PzuN6d3Hoj9
 60RpiAr4s+PPP+W5gNctvpcfHHeVAlfii8RV56WW6XbJ3vG0i94KIfs49Frt1DRvQzvewPcd
 L6IQpliVI=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="53016148"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iMzcNMe9DMlYNt62q7HrlD5R66M7A9G1wYbYUJBSrqqKDQ7tQAzzV5j68B7FDCaD1pnmeFqOCeYsw7uHNmeMfa00oRlbi2m9ezUvQYwC3FSjEcrnMHpBf4dBIU6tFhCo4+vzQBsIL85UPNkSQBGhJuN7qT7yoK0gdizSe/H66ezxPyRGiDG4MABELN8v17uLVUowM84LfoZ1rcRv2AB3rlyWexyho8TamxvblxA68ZaeTrh4/VBgD6CnPM2j3bj7IAWbVp2TEKIT6ODXSy1TM8PIhb4vXjz8IHKSESlHYP+eLzWSe7wBYSo1zbv0iT7A5hgHx3+WT/Z3D8eZyoY5tg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oFmL7GXpSQBvcx3T6o/NI73rUPRPzWCG2d3RTcwysco=;
 b=k+g/fVRac8eC8q7441bW4DXLl59yNar+fYEUgSbqJ77Ah9KaHzlTBxo4rr5TTl3K7CSvxZUosvxS504p427btXbkicho+fFRGHwuSNANDpc4l74Eje+zKD4thlbuFpf7rNk+qmD+RyW7bWCmmNCU36wn2JYpSAdKmxkwxImkiWcuHMr85AXdhD3eeDUoXCn3d+HQ1wl7ShIfnFwhUwF5yEeFIRX6QgmqY146nSw3JcKqdGX46jP0BJ1H+E4PUxWXvjeqzPD7l8FIbDv4W63T4DN4NwEYr3LswP1Ey1Y2uOumppWOKErs4aCL4UuCx8751u+fGR9BOr/6fi42wwtpdA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oFmL7GXpSQBvcx3T6o/NI73rUPRPzWCG2d3RTcwysco=;
 b=E2xo1Px0dG9WOFt5myTACXHQxTDXJmMoQQVOjJ+6AC3dbgN5A349GT606AeRNqAOSNY4J8Pww6pB4JBFztcHfyGE4tEAQ6GKyRNtAinzm1ksI/ct9UwaQqt/WSpSGjAJA4koKDvXh64Qdc97TGLojQmRaoB/d5IVFpYuG5f7hQI=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH 2/6] gnttab: allow per-domain control over transitive grants
Date: Fri, 17 Sep 2021 17:46:21 +0200
Message-ID: <20210917154625.89315-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0279.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:195::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 124d820b-a3a1-40c6-9706-08d979f25bc3
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB405733C673C3AC8D53C762948FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: VSvtcpToyVj2OZyE4sDXy5RqI+toq05GBFmH0P/HhvfgfCuNAtxNU1uQv9oNWMGByuUwxXsLTVwR317t6JgUB6i4qUUuBOa8DCnyZTTuh8IXWALuAHLqFEZ4DQrSDRIMBv4VBurkoqHvo/85cZKhxC8dX6Iy2+k/cTpLAWY4OzvFTx3Do+ovjS72FR21jHvurP0utRHJJ6BY793XhjkwFXOBMLZPNa2q2gsWVxtuWUJc2eLTH8nZdc26P8qhAGBEHq10fRFJMLThXIsRL5bO9MtADwvWcktNbwVonSsZwErqySVI0Svn/yHxelMjLVMzCkD9HjoYo3Od1EQfWciq3qygKZs+HxDi4jOJ3mAeugT4bnouFX1+1NG7eknY+dTDG0bnmjsVsRBO7D4lpXC+uIefc+qY3CICwQla+I+a6lhUQFiNBJSIItR7itG05nCeKmyYP8HAtPjhy2zw8m4hXKULRGmXYZ9lU8j2ddB6EutCeQwfoFOhq2POJKqrzhKWZuPCy6y+RHlGWzaVCxqriVusiYkAv+ftuBx3o5ewuRCWijahgL4+3V73SkknBAF9n9XB7DgjCPf5amkLuvSddGwd1ZheN1I5ZNSgAJ+Lc0Ng1g8/tnyecHe5BF+gyedOgEg1wwwA085EAnCbu958ng==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(30864003)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZDcwOGpkMUhjczlVTkVtYWM4SVF4Kzc0RUJ0TGxjL0s4eGJYSWI5dUhkM0Jw?=
 =?utf-8?B?V2VzTDBZQVdLelh3ZkQ4SWtCV3M1ZzljTWpNZ2d4ZnNleHNLWnZ5eFplTWVW?=
 =?utf-8?B?Z2FXVnh3c3ZaZXNOMFBOb0hOR2dSK2NyWXhEbXFWNU03ZE52NGpPb2dldlFm?=
 =?utf-8?B?bzVWVDFpRk5PaWhua0Qva3FCYVUwNW5ldU9nWG5OVUg1ZEUvcDhkMEJMZDVh?=
 =?utf-8?B?TC9Ic0lmN2xjUXpmbnQ5WHovbW84eXFDSURoSkdCVmJCWWNBQTB5L0hTNlJL?=
 =?utf-8?B?U2o1L2hPVlZrWWRkQ0YrWkJUSWoxL0xHeFlVYVdGckNyM0xNbUhBRkE5eHpl?=
 =?utf-8?B?ZUQvWEh0d3F2RnNJRkxNblA4N0Q1N2NCM29yOEduNjN5NExxMzg2WVlFbGd5?=
 =?utf-8?B?Q3NMTy8xUlJINjdEKzQ0RVFkSllJNVZCbEtuN3hXek9zYVpxMVhlTU1mczA4?=
 =?utf-8?B?ZzdPeW1tcmdCY2pacVc5OFRROVZncm91VzVJVGxVbUF5bEQ5a3hVdWNVY002?=
 =?utf-8?B?b1ZKSk5zSk9TTml1Q2JHZmpQeXdHUDNtalJucGpMeU1ZNGVJMTQwbFpaV1dt?=
 =?utf-8?B?cXpacFdwVmJnVXNUVytOZnpaMFhxK2ZBSFhVeWE5eE5VUklXRysxNnB3bGgy?=
 =?utf-8?B?cnAzRTdZR0t0OVRVbFV2M3p1Umw0Y1hsME5ET2FObTBJazE5LzRIYVVjamdQ?=
 =?utf-8?B?dzhTVEVKRDFEVm1kTFNnT29PYXJIbTF0aWpLazIwNlE5VEdhRlBNMW5Sc3pG?=
 =?utf-8?B?ZUZqMFBhQXU2TFUyZEc2NWgzMFJDMTMrNzdVZlZCWCtUTVp4a2xxQ0tGaWNu?=
 =?utf-8?B?SFpxcTdMYVpCRjFtaHBhRmwwbVk3ckd0N1FNTzcrMEwvQTVpQlJZTEtSQTlj?=
 =?utf-8?B?RXUwWE1FbjhobWhXRFFmSklBbUUxb3RXc0ZRNVdadWNLbU1GR2FRbkdaaHhu?=
 =?utf-8?B?b3pUSFNHRzdsdHdEZ09LVkcvQU83ZzFKUGVEYzJJM0tCbUYrSjFXQkF5VVBM?=
 =?utf-8?B?YkNOdGxCZU43aUxSd3MyNFNER2J4UnFZTmQ0R3VrajZpdmZkQU5NYU90WVBP?=
 =?utf-8?B?RzNqU2ZiMTVGRjhiYWQwWDR4bnRkeXNzUEtmMTJTbjUrbWVYNkFaQmFaeS9v?=
 =?utf-8?B?N3RtOHB5c2poQTJjc09oTktDeEZ5WVVDV0Z2YmpSQnBFRUdJbVRTS2dEYmRs?=
 =?utf-8?B?MEhkcU1KWjBBZ3poT3NWNU13TDBQcjRLd09oRU1kci9FbFQ0UC9kK0Vwdmti?=
 =?utf-8?B?RDFZY0M2ZTgwcW1rUjgvREp4Mm4wUUgrSGZ4d29hZVA4eHMzRWFLaGF4NWtI?=
 =?utf-8?B?MHFscmV4d09MWklnQTBOMGE1SDZvZVYzWVhlckxIaUExc2FWb0E1TGdBcmVO?=
 =?utf-8?B?REQxN2hoODRpUThzR0w0ZkVWK3hQVlJXZUErK0MxTHlQODg5MVdRSENCZG5k?=
 =?utf-8?B?STZlY0JDQWRnVFRTU0IwN2NIZlRTNDgwWUdQSkdhMWlFWWZ1d2F2V3U1Sm4r?=
 =?utf-8?B?MDFGTnZkK3gyVGlqbDVxREVwUko0KzNBSjRMTXZVOFdmTnhNUjBFU3lyQzU5?=
 =?utf-8?B?MWxGL1hBY0lTSndTOUdvZlkvenRtcnpvY3MyMEpTYlppWk1oZXFFNzNKeTh2?=
 =?utf-8?B?allBalExc2VVUXR4bkRobHc4eCtjZlFFRDdNMk5QbUlyZWhyM2JQb28xdm9q?=
 =?utf-8?B?eDkxaDNZMVpROHVZYmtjRkc0SXM3UFEyNDF6eWNtZkI3ZmN6eUJYZkFqOEdq?=
 =?utf-8?Q?Oy3XytBW5qFP28fRPYL/2Ia+E4+GVHkdS9xLGhp?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 124d820b-a3a1-40c6-9706-08d979f25bc3
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:46:48.2010
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZO8CfCP9tEuLOREF4JL7gvprEA0FNkpzyWBMoPnoTJ6oo7gqNd9rI6NmrMO8TW4FqIwG6BETEld/wLL1O82wUA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

Introduce a new grant options flags field in domain create and use it
to signal whether transitive grants are allowed on the domain. This is
settable from xl using the transitive_grants option.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 docs/man/xl.cfg.5.pod.in            |  6 ++++++
 docs/man/xl.conf.5.pod.in           |  7 +++++++
 tools/include/libxl.h               |  7 +++++++
 tools/libs/light/libxl_create.c     |  3 +++
 tools/libs/light/libxl_dm.c         |  1 +
 tools/libs/light/libxl_types.idl    |  1 +
 tools/ocaml/libs/xc/xenctrl.ml      |  4 ++++
 tools/ocaml/libs/xc/xenctrl.mli     |  4 ++++
 tools/ocaml/libs/xc/xenctrl_stubs.c |  9 ++++++++-
 tools/xl/xl.c                       |  7 +++++++
 tools/xl/xl.h                       |  1 +
 tools/xl/xl_parse.c                 |  4 ++++
 xen/arch/arm/domain_build.c         |  2 ++
 xen/arch/x86/setup.c                |  1 +
 xen/common/domain.c                 |  3 ++-
 xen/common/grant_table.c            | 11 +++++++++--
 xen/include/public/domctl.h         |  8 ++++++++
 xen/include/xen/grant_table.h       |  6 ++++--
 18 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index a4bf2caafa..c5a447dfcd 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -586,6 +586,12 @@ Specify the maximum grant table version the domain is allowed to use. Current
 supported versions are 1 and 2. The default value is settable via
 L<xl.conf(5)>.
 
+=item B<transitive_grants=BOOLEAN>
+
+Specify whether transitive grants should be available to the domain. Note such
+functionality only applies to grant table version 2. The default value is
+settable via L<xl.conf(5)>.
+
 =item B<nomigrate=BOOLEAN>
 
 Disable migration of this domain.  This enables certain other features
diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in
index 0a70698a7c..88efbee5fe 100644
--- a/docs/man/xl.conf.5.pod.in
+++ b/docs/man/xl.conf.5.pod.in
@@ -108,6 +108,13 @@ Sets the default value for the C<max_grant_version> domain config value.
 Default: value of Xen command line B<gnttab> parameter (or its default value if
 unspecified).
 
+=item B<transitive_grants=BOOLEAN>
+
+Sets the default value for the C<transitive_grants> domain config value.
+
+Default: value of Xen command line B<gnttab> parameter (or its default value if
+unspecified).
+
 =item B<vif.default.script="PATH">
 
 Configures the default hotplug script used by virtual network devices.
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 7a35833312..d23586f2cc 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -509,6 +509,13 @@
  */
 #define LIBXL_HAVE_MAX_GRANT_VERSION 1
 
+/*
+ * LIBXL_HAVE_TRANSITIVE_GRANTS indicates libxl_domain_build_info has a
+ * transitive_grants field for setting whether such functionality should be
+ * available to the domain.
+ */
+#define LIBXL_HAVE_TRANSITIVE_GRANTS 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 69b5419416..61d65d1342 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -633,6 +633,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
             create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
 
+        if (libxl_defbool_val(b_info->transitive_grants))
+            create.grant_opts |= XEN_DOMCTL_GRANT_transitive;
+
         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 9a8ddbe188..4ade437257 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -2321,6 +2321,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
     dm_config->b_info.max_grant_frames = guest_config->b_info.max_grant_frames;
     dm_config->b_info.max_maptrack_frames = guest_config->b_info.max_maptrack_frames;
     dm_config->b_info.max_grant_version = guest_config->b_info.max_grant_version;
+    dm_config->b_info.transitive_grants = guest_config->b_info.transitive_grants;
 
     dm_config->b_info.u.pv.features = "";
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 37789a568c..d05b5d2abc 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -519,6 +519,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("max_grant_frames",    uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_maptrack_frames", uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_grant_version",   integer, {'init_val': '-1'}),
+    ("transitive_grants",   libxl_defbool),
     
     ("device_model_version", libxl_device_model_version),
     ("device_model_stubdomain", libxl_defbool),
diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index 6a8658bfec..ec7a296776 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -73,6 +73,9 @@ type domain_create_flag =
 type domain_create_iommu_opts =
 	| IOMMU_NO_SHAREPT
 
+type domain_create_grant_opts =
+	| GRANT_TRANSITIVE
+
 type domctl_create_config =
 {
 	ssidref: int32;
@@ -84,6 +87,7 @@ type domctl_create_config =
 	max_grant_frames: int;
 	max_maptrack_frames: int;
 	max_grant_version: int;
+	grant_opts: domain_create_grant_opts list;
 	arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 5933d32c70..e47fd1947f 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -66,6 +66,9 @@ type domain_create_flag =
 type domain_create_iommu_opts =
   | IOMMU_NO_SHAREPT
 
+type domain_create_grant_opts =
+  | GRANT_TRANSITIVE
+
 type domctl_create_config = {
   ssidref: int32;
   handle: string;
@@ -76,6 +79,7 @@ type domctl_create_config = {
   max_grant_frames: int;
   max_maptrack_frames: int;
   max_grant_version: int;
+  grant_opts: domain_create_grant_opts list;
   arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index aad3c6a726..772704759d 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -189,7 +189,8 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
 #define VAL_MAX_GRANT_VERSION   Field(config, 8)
-#define VAL_ARCH                Field(config, 9)
+#define VAL_GRANT_OPTS          Field(config, 9)
+#define VAL_ARCH                Field(config, 10)
 
 	uint32_t domid = Int_val(wanted_domid);
 	int result;
@@ -214,6 +215,11 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 		/* ! XEN_DOMCTL_IOMMU_ XEN_DOMCTL_IOMMU_MAX max */
 		(VAL_IOMMU_OPTS);
 
+	cfg.grant_opts = ocaml_list_to_c_bitmap
+		/* ! domain_create_grant_opts GRANT_ lc */
+		/* ! XEN_DOMCTL_GRANT_ XEN_DOMCTL_GRANT_MAX max */
+		(VAL_GRANT_OPTS);
+
 	arch_domconfig = Field(VAL_ARCH, 0);
 	switch ( Tag_val(VAL_ARCH) )
 	{
@@ -253,6 +259,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 	}
 
 #undef VAL_ARCH
+#undef VAL_GRANT_OPTS
 #undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 0fde879cf4..9bd398f8c9 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -56,6 +56,7 @@ bool timestamps = 0;
 int max_grant_frames = -1;
 int max_maptrack_frames = -1;
 int max_grant_version = -1;
+bool transitive_grants = true;
 libxl_domid domid_policy = INVALID_DOMID;
 
 xentoollog_level minmsglevel = minmsglevel_default;
@@ -221,6 +222,12 @@ static void parse_global_config(const char *configfile,
     else if (e != ESRCH)
         exit(1);
 
+    e = xlu_cfg_get_long (config, "transitive_grants", &l, 0);
+    if (!e)
+        transitive_grants = l;
+    else if (e != ESRCH)
+        exit(1);
+
     libxl_cpu_bitmap_alloc(ctx, &global_vm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_hvm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_pv_affinity_mask, 0);
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index cf12c79a9b..d7f83c9abd 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -283,6 +283,7 @@ extern char *blkdev_start;
 extern int max_grant_frames;
 extern int max_maptrack_frames;
 extern int max_grant_version;
+extern bool transitive_grants;
 extern libxl_bitmap global_vm_affinity_mask;
 extern libxl_bitmap global_hvm_affinity_mask;
 extern libxl_bitmap global_pv_affinity_mask;
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 1206d7ea51..8e4e561df8 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1440,6 +1440,10 @@ void parse_config_data(const char *config_source,
     else
         exit(1);
 
+    xlu_cfg_get_defbool(config, "transitive_grants", &b_info->transitive_grants,
+                        0);
+    libxl_defbool_setdefault(&b_info->transitive_grants, transitive_grants);
+
     libxl_defbool_set(&b_info->claim_mode, claim_mode);
 
     if (xlu_cfg_get_string (config, "on_poweroff", &buf, 0))
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index e9a34f2f8d..b4763b5ec6 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2485,6 +2485,7 @@ void __init create_domUs(void)
             .max_grant_frames = -1,
             .max_maptrack_frames = -1,
             .max_grant_version = -1,
+            .grant_opts = XEN_DOMCTL_GRANT_transitive,
         };
 
         if ( !dt_device_is_compatible(node, "xen,domain") )
@@ -2593,6 +2594,7 @@ void __init create_dom0(void)
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
         .max_grant_version = -1,
+        .grant_opts = XEN_DOMCTL_GRANT_transitive,
     };
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index af69e20029..c743a88592 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -751,6 +751,7 @@ static struct domain *__init create_dom0(const module_t *image,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
         .max_grant_version = -1,
+        .grant_opts = XEN_DOMCTL_GRANT_transitive,
         .max_vcpus = dom0_max_vcpus(),
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 0c85d89419..ab16c422f7 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -670,7 +670,8 @@ struct domain *domain_create(domid_t domid,
 
         if ( (err = grant_table_init(d, config->max_grant_frames,
                                      config->max_maptrack_frames,
-                                     config->max_grant_version)) != 0 )
+                                     config->max_grant_version,
+                                     config->grant_opts)) != 0 )
             goto fail;
         init_status |= INIT_gnttab;
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index d01c6813d1..280dbc850a 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -74,6 +74,10 @@ struct grant_table {
      * progress.
      */
     unsigned int          maptrack_limit;
+
+    /* Allow transitive grants. Only applies to grant v2. */
+    bool                  allow_transitive:1;
+
     /* Shared grant table (see include/public/grant_table.h). */
     union {
         void **shared_raw;
@@ -1918,7 +1922,8 @@ active_alloc_failed:
 }
 
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames, int max_grant_version)
+                     int max_maptrack_frames, int max_grant_version,
+                     unsigned int options)
 {
     struct grant_table *gt;
     int ret = -ENOMEM;
@@ -1964,6 +1969,8 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     gt->max_grant_frames = max_grant_frames;
     gt->max_maptrack_frames = max_maptrack_frames;
     gt->max_grant_version = max_grant_version;
+    gt->allow_transitive = !!(options & XEN_DOMCTL_GRANT_transitive) &&
+                           opt_transitive_grants;
 
     /* Install the structure early to simplify the error path. */
     gt->domain = d;
@@ -2886,7 +2893,7 @@ static int gnttab_copy_claim_buf(const struct gnttab_copy *op,
                                     buf->read_only,
                                     &buf->mfn, &buf->page,
                                     &buf->ptr.offset, &buf->len,
-                                    opt_transitive_grants);
+                                    buf->domain->grant_table->allow_transitive);
         if ( rc != GNTST_okay )
             goto out;
         buf->ptr.u.ref = ptr->u.ref;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 7f8456c50e..fe2201fca1 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -96,6 +96,14 @@ struct xen_domctl_createdomain {
     int32_t max_maptrack_frames;
     int32_t max_grant_version;
 
+/* Allow transitive grants. */
+#define _XEN_DOMCTL_GRANT_transitive  0
+#define XEN_DOMCTL_GRANT_transitive   (1U << _XEN_DOMCTL_GRANT_transitive)
+
+#define XEN_DOMCTL_GRANT_MAX XEN_DOMCTL_GRANT_transitive
+
+    uint32_t grant_opts;
+
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;
 
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index d6da067fc1..f264b1c3fc 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ -36,7 +36,8 @@ extern unsigned int opt_max_grant_frames;
 
 /* Create/destroy per-domain grant table context. */
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames, int max_grant_version);
+                     int max_maptrack_frames, int max_grant_version,
+                     unsigned int options);
 void grant_table_destroy(
     struct domain *d);
 void grant_table_init_vcpu(struct vcpu *v);
@@ -68,7 +69,8 @@ int gnttab_acquire_resource(
 static inline int grant_table_init(struct domain *d,
                                    int max_grant_frames,
                                    int max_maptrack_frames,
-                                   int max_grant_version)
+                                   int max_grant_version,
+                                   unsigned int options)
 {
     return 0;
 }
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:47:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:47:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189495.339252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4q-0006U7-4v; Fri, 17 Sep 2021 15:47:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189495.339252; Fri, 17 Sep 2021 15:47:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4q-0006U0-1O; Fri, 17 Sep 2021 15:47:00 +0000
Received: by outflank-mailman (input) for mailman id 189495;
 Fri, 17 Sep 2021 15:46:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG4o-00068c-UY
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:46:58 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 11889cf3-44ea-448b-97de-562d11ded3d2;
 Fri, 17 Sep 2021 15:46:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11889cf3-44ea-448b-97de-562d11ded3d2
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893617;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=xc5XifEQVsXnyl+ZbEPS5l0E4aBxZPLm3FQyJKAUnc0=;
  b=b0EmUgeoZhOOSWUxpQrluSmqoUoRA+lCRXNGOcQ1CIyMzjsv9cZndKJ3
   FZQnHkQ6rXGoos5LKatC/7ZcJbjQmD3LaEmd+Ld7ct/oDmDV7TH0F7SXF
   3xFousb9J+YZpJkxIiq+Kk+uf0DtSi9SG9Va8Q8WIRkAdrORh8v/rz41S
   o=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: pOxK3cjNsiCcsVk2qd0VoLNGC3qVs7nlM9GbjU0ETwP5mjDbCqV4JndzjiUQ9sCu1dJf8NRvfS
 bmB81hJyPB3xxKTJyfjU5PbYQbPfr1X210I0Ri+ExKC6P0c57RtfjXgwcT7j1TjJI4RFJYv89Y
 54j6Wgzbmzgf1p11pR3c6SmAAo6rt7ti2eXq7Klo7QPwaKEtXNLOR1mBC1sOObscD0R2QlQxxq
 5izpWvDR8FZNK/x+9xz3ZJoo3AQvauD8UZZ9bZrJlowkLg4LtUCYyWaXj+lu6fWNu+62GlsHQ3
 c1/dATSiksu3SMK7tcCb78VC
X-SBRS: 5.1
X-MesageID: 53410938
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kVrkTq0Wkh0LjcACBvbD5eZ2kn2cJEfYwER7XKvMYLTBsI5bpzJRy
 2IeDW/Ta/+PMGP1Kd8gbIzjoxlX7ZDVmoBrSAJvpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywrFh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhso0yx
 oVrn7WLSDgUb+7Rut4cQwEFKnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQEhW9h3ZAm8fD2R
 Jc8bQt3cx36bTpqfQxOEIIat+KHvyyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1UgUSwt
 m/A537+ABwRKJqY0zXt2mKhgKrDkD32XKoWFaak7bh6jVuL3GsRBRYKE1yhrpGEZlWWAowFb
 RZOo2x38PZ0pBfDosTBswOQjSPHsE46cfNrEtIHuUKQ6JDV8x7EPz1RJtJeU+DKpPPaVBRzi
 ATQz4i4VGQ12FGGYSnCrebP9FteLQBQdDVbNHFeFWPp9vG++Nlbs/7Zcjp0/EdZZPXOEDfsy
 nihqCEkjt3/ZuZaiv3moTgrb9+qz6UlrzLZBC2MBQpJDSsjPeZJgrBEDnCBtp59wH6xFAXpg
 ZT9s5H2ABoy4XSxeMulG75lIV1Uz6zdbG20baBHRsF8n9hSx5JTVd8Ju2wvTKuYGu0FZSXoc
 CfuVfB5vcQIVEZGmZRfOtrrY+xzlPCIPY28Cpj8M4ofCrAsJVTv1Hw/OiatM5XFzRFEfVcXY
 szAL65BzB8yVMxa8dZBb71MieN0nn9umzy7qFKS503P7IdyrUW9EN8tGFCPcvo4/OWDpgDU+
 MxYLMyE11NUV+iWX8Ud2dR7wYkiISdpCJbohdZQc+LfcANqFHt4U63awK87epwjlKNQz7+a8
 na4U05e6Vz+mXyYdlnaNiE9MOviDcRlsHY2HS0wJlL0iXItVpmisfUEfJwtcLh5qOE6laxoT
 +MIctmrC+hUTmiV4CwUaJTw9dQwdBmiiQ+UETCiZTwzI8xpSwDTo4e2dQrz7igeSCGwsJJm8
 bGn0wraR7sFRhhjU5mKOK7+kQvpsCFEyuxoXkbOLt1CQ2nW8dBne37rk/s6A8AQMhGflDGU4
 BmbXEUDru7Xro5rrNSQ3fKYr52kGvdVF1ZBGzWJ9q6/MCTX8zbxwYJEV+rULznRWHmtpfenb
 ORRifr9LOcGjBBBtI8lS+Rnyqc35t3Oob5Gz1s7QCWXPgrzUr4wcGOb2cRvt7FWwu4LsAS7b
 UuD591GNOjbI8jiClMQeFIoY+nrOSv4QdUOASDZ+HnH2RI=
IronPort-HdrOrdr: A9a23:SBrV2azKQmuU5hfoH25pKrPxv+skLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjmfZq6z+8L3WBxB8bfYOCCggqVxe5ZnPLfKlHbakjDH6tmpN
 1dmstFea3N5DpB/L7HCWCDer5KqrT3k9HL9JTjJjVWPHxXgslbnnZE422gYytLrWd9dP4E/M
 323Ls4m9PsQwVaUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZtzU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDh1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoykfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW/2/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 gvMCjl3ocSTbqmVQGcgoE2q+bcGUjbXy32A3Tqg/blkQS/xxtCvg0lLM92pAZJyHtycegc2w
 xoWp4Y442mdfVmGJ6VMt1xCPdfOla9My4kD1jiVWgPbJt3QE4li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="53410938"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YrfTnXCmYWfM+R0x/MdNBensOUWNPuBdEmzcz4b3LAgSH1udHnWKDMrCu1BG5WSSSs9rgA7unc+BFtgRpsF1OhsmtDIzm45G3L21O1k5nf0aVx9PN9n/zyRPYCp2Xz7l7sQhnxXtTxpRLWUZjReoXfM2fiiDq0FY+8Qv1JpSAOkH4y0s5pv0P2NFWkj4BdQTPoUAsBVH7OebrRAeNNsfIstxPOdQxMJ/s7J7Fsx/eC95cziM5kDXhd8BR2kSkqtqinoVYi2g+Spj1GoELzWF6FOPL5H9X6Tq5QZOj2R1MYJqNRD41+0vPZZ1UJNCo34Lev+5KzegKhXKomrW7UszOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=P6No9nF8vaLlLWVxNdu1JhmKZDYGo8kkft5wXq/DnGc=;
 b=QZOm5UlHuVOLtEtJyTtlGa6rhHTP+M3FYYIJ92lj1DaXjxKWYIefz6e6MAmfH70Ng3nxu9968dKiyFlDkAgXD3jNw1I3n74YgKn3eQs1fBqQCSZaA5PaZfNup5fAG26ehJtdoaOFstdP23nxBXluzi+5lrtxe6PfiAcZwVTT2lDxanGgwWGFprFqM8vDn/XnwmFqCeHkwR9p9ii3PdufJ9efzQs6o5O2Ig/jDjvbsQMA8H/NkHB8XDLhW88mN+PZznToBDgujmf2QYC9sLPWe8rfC4kLghribmMW1tlaLZKTsxREm402PQaR9+55aBK/uQyAxNLw7PrShY5/EZsW7Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=P6No9nF8vaLlLWVxNdu1JhmKZDYGo8kkft5wXq/DnGc=;
 b=Vle/hxLrbcpyANodfXK1dFKMtMO5qk+6W+9hh/de/Ser8O0gCr6yv3lunPI72aIIOg2MKNa+R+rs9weQlNw149vMOFzZsgKZNoRJ2wgitJt7ZW2GGoVD+ZMfsQ2BTDF8cBKueNbvZTi70h7r8zaFaEN/ShFPfAcmQeTCy44FRlk=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 3/6] tools/console: use xenforeigmemory to map console ring
Date: Fri, 17 Sep 2021 17:46:22 +0200
Message-ID: <20210917154625.89315-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0243.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8a::15) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 51663f29-f51c-47d4-db21-08d979f25eff
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4057574B77A8171B40C800E18FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +IP6b9WqtYY0BCvD0zGkG70xxa0sfnFZSFJt2lqxtZOE+q+bR7+nrm4J8TwNmhnmGvVqYoOGE9OkxiJ2reg2QiHv4x8oPZca/8prw4iAnoyDkLLnGRkNTCYD7Whgja1mZqBqETRUOrOtCfkNRf5mtWiw5/RG6ssBX8PSrZCDymixcPruAlUO1UUpOk8W6NTs+4ovb20EkHKxwviYXpaLFHVfeSwqS5GiKu9bYTH7hKlYYA02aEP8zFjb+2f04gpvAvENJmq0CHI2XoEbfwY8zTsE/ZqF/0oZDkyIiSUhEW+9cxOrGFdeE1xJwR5M+qkClw0xbAjm54laaV52PP40Zd3tivHGv05RRGB7bHu+K8E8xEJ8kuOv8Rf00IjuiKx1DbtWopfpwu6E369cz354nbOVZGFqgLGmj8IFn5HTjyvCjx/aKvIqkNfC3DxZgTnoCz7o2Vnq5oi0KCJL//RDBHO9S3cZzPcDFpXNJjj/vof3k1J7f5WEi+GBYGLsbOFh1DJ4uqKeojxSYuun3XOc6cI/SW5rF9xJE2Y4utQgr54BOXmo/vPibTU2j1MyRTMLa0nQmv1292rAQllqR56KMo3ZgBBE+l6+XUvcjfyD7hf+wfjEvAFErSA15ErvrK19zSzNWjavPbv6bGpIILrhvg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aHRaa0dZd0F5eTVVQlN3WkVHY1dxeVVib2RzK1hRVmZ2MHFVcVBod0ZEMjJr?=
 =?utf-8?B?eHBrN3VESTArblhLZEJQMmsxejkzQUZaU3FWNVpPL0VWUzI3SGhBZGlRaTh2?=
 =?utf-8?B?M0gwcWoySW1Ucnh0bmNqZXQraFM1cEt3amFMam5pR0llRTdWSmRudkRKcCt6?=
 =?utf-8?B?MjF3eWo1ekliUDdYNWZOampYbERmV0hxZTlxeTQ3QzJWSUJxYU9hdTBOMTha?=
 =?utf-8?B?enZkcDFqaWNCRkdTWld2dGxZbHczRkhHam5RSFVFenh2VDd3SzRjdm1FWmo4?=
 =?utf-8?B?clN5MGFaR2txWDA2cWZlQmxUUkNvYnRuZWVWZGh5VlgxOUtjYTFIUE02WDdH?=
 =?utf-8?B?NStFNTU3L0JGRUhET2tFSExVcStNN2ZFUCtiUVQ5bkNZL3NBbFB3TzZrOTNN?=
 =?utf-8?B?T3E5UEh1Snh0TDJaY2pVL0I1U0M4TmkxanpURTBTY3d5MVVZaVg1bTRHRUsx?=
 =?utf-8?B?VVJ3L3hsdzV3ZWkyb2d1S0VFN2dDRDBTOVZpSUI0dXE5djBtTHEwTm5iNXdO?=
 =?utf-8?B?eS9hZ2dOUU1rWjlYRmpzWXJzYlJ2ejdFZzB2VUJqbnlaUVI2NlZqTDNiOWFa?=
 =?utf-8?B?RFZReFYyNS82ajhGS2JidU5NdDFFY1U2Nys0dzJLVlVPOUN6QTVRVzU2N3Ji?=
 =?utf-8?B?TmZWdWd2WmZTZCtueXNlWHQ5eFFrUUFzVkZubkRoUEVMZXdUK2NJcXB1WEU2?=
 =?utf-8?B?QkNCeDdJSEs0RkNZRmwwc0ZqZFRJbGpIS2l2VFJ5ZHNTSUJXZ2ZOK1FzZ1gr?=
 =?utf-8?B?OXg5V3JmTWxhNTZTSjNwQ3IvaU5tUGR2c2RZZmNoUE5XVVM0WkRVaWEybVhJ?=
 =?utf-8?B?U1dESUpLKys1WHlFZmhTK2VFTW1aRzkzNGJ4NkpKQmNHV2NTSjhhTUdxbHVZ?=
 =?utf-8?B?RERzYi9ObGhKNllVcXZmemt4cUt5N2VPeTMvckZUUzl0VktrVUxXMG0wanZr?=
 =?utf-8?B?S0tCZEtnTFpNaUhUN0I4eERlREVNSk5DVXZRa1dFc01QZEJHQjV1YjY5NlRn?=
 =?utf-8?B?MkV1NTdGMDZBeFBPTHhKSkNDTlY1ZE43WmVOU1BqWTdqSS9lcDl0Mm9sQ2JL?=
 =?utf-8?B?TXpWSkNvV1hTaUdCVGh5WXY5K1VXc1l2UDFyK3JrV1pOd3dpczRxMk0rdEYr?=
 =?utf-8?B?S3dlbmQ1ZmRqRWIwRW52Y3dxVk8xZDdJQXBnaWpwemRaVmJhMHovYmx2VWtO?=
 =?utf-8?B?TlR1djd1QWJQa2JDUEdBRXlXTU1VdENvSURTd1dEY0VBb0xGKzZRalIxUzJG?=
 =?utf-8?B?WXJ4eC91Q1BKcmNHNzQ1dHFmbzBCNDViMDA0Y0tGWTloMUlTNGZ3MmF3dUo5?=
 =?utf-8?B?NkpiVXptdlpjWTJBZHFOU20wVjR5cE1IaVlWb1BsdGw2Y0lYN3VjdUorM1pN?=
 =?utf-8?B?M0FKYllPSFZOMVpHbGhoSGJEYzNUNHZyUm1XRlQwcVYwekYrRHp3bnlmaGtG?=
 =?utf-8?B?bS92aENPMTdLa1JESWNKZ0xCWFBrVW5YVkE0SW1EVEZUUjJXMXJIbE1iNkEx?=
 =?utf-8?B?MHdIaE92NkdNMFRMQkJ4NDI3SE96eHNPd3dmNEdLK1hnS3dXZUczL3l1ZWNP?=
 =?utf-8?B?aE5iak1ZSHFpZHpiMFJ3VitQNmlVTmJlTFl6WUNJL0h4U2NJekFHRkU0djRC?=
 =?utf-8?B?UFVEeFNJLzhzeVlmblhOTk9jRU4xNTdwSUtZRzFwVWlLQnhvZXpTVEpBMjI4?=
 =?utf-8?B?SHJ4UWhvL3NyL1ZLQmRXWjlvNHRiaEJma1B0eGM2MWJYNC9Md3R4N3FFelI5?=
 =?utf-8?Q?LLmaRa3nvc8yhR9Qky5RguCZBFPiPsW1P1E3zMk?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 51663f29-f51c-47d4-db21-08d979f25eff
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:46:53.4280
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: F0woerLFdqsuRCe72mwIkSNYeJvsgP0s56UMCmMCybsjtimS/Ee49skWezQBYyo7JTXsY1HPOjUmxwpCPLiqWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

This patch replaces the usage of xc_map_foreign_range with
xenforeignmemory_map from the stable xenforeignmemory library. Note
there are still other uses of libxc functions which prevents removing
the dependency.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 tools/console/Makefile    |  4 ++--
 tools/console/daemon/io.c | 25 ++++++++++++++++++++-----
 2 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/tools/console/Makefile b/tools/console/Makefile
index 84796eac8f..3f4cddab03 100644
--- a/tools/console/Makefile
+++ b/tools/console/Makefile
@@ -29,9 +29,9 @@ clean:
 distclean: clean
 
 daemon/main.o: daemon/_paths.h
-daemon/io.o: CFLAGS += $(CFLAGS_libxenevtchn) $(CFLAGS_libxengnttab) $(CONSOLE_CFLAGS-y)
+daemon/io.o: CFLAGS += $(CFLAGS_libxenevtchn) $(CFLAGS_libxengnttab) $(CFLAGS_libxenforeignmemory) $(CONSOLE_CFLAGS-y)
 xenconsoled: $(patsubst %.c,%.o,$(wildcard daemon/*.c))
-	$(CC) $(LDFLAGS) $^ -o $@ $(LDLIBS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_xenconsoled) $(APPEND_LDFLAGS)
+	$(CC) $(LDFLAGS) $^ -o $@ $(LDLIBS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxenforeignmemory) $(LDLIBS_xenconsoled) $(APPEND_LDFLAGS)
 
 client/main.o: client/_paths.h
 xenconsole: $(patsubst %.c,%.o,$(wildcard client/*.c))
diff --git a/tools/console/daemon/io.c b/tools/console/daemon/io.c
index 200b575d76..682c1f4008 100644
--- a/tools/console/daemon/io.c
+++ b/tools/console/daemon/io.c
@@ -22,6 +22,7 @@
 #include "utils.h"
 #include "io.h"
 #include <xenevtchn.h>
+#include <xenforeignmemory.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 #include <xen/io/console.h>
@@ -73,6 +74,7 @@ static int log_time_guest_needts = 1;
 static int log_hv_fd = -1;
 
 static xengnttab_handle *xgt_handle = NULL;
+static xenforeignmemory_handle *xfm_handle;
 
 static struct pollfd  *fds;
 static unsigned int current_array_size;
@@ -675,7 +677,7 @@ static void console_unmap_interface(struct console *con)
 	if (xgt_handle && con->ring_ref == -1)
 		xengnttab_unmap(xgt_handle, con->interface, 1);
 	else
-		munmap(con->interface, XC_PAGE_SIZE);
+		xenforeignmemory_unmap(xfm_handle, con->interface, 1);
 	con->interface = NULL;
 	con->ring_ref = -1;
 }
@@ -722,11 +724,12 @@ static int console_create_ring(struct console *con)
 		con->ring_ref = -1;
 	}
 	if (!con->interface) {
+		xen_pfn_t pfn = ring_ref;
+
 		/* Fall back to xc_map_foreign_range */
-		con->interface = xc_map_foreign_range(
-			xc, dom->domid, XC_PAGE_SIZE,
-			PROT_READ|PROT_WRITE,
-			(unsigned long)ring_ref);
+		con->interface = xenforeignmemory_map(
+			xfm_handle, dom->domid,	PROT_READ|PROT_WRITE, 1,
+			&pfn, NULL);
 		if (con->interface == NULL) {
 			err = EINVAL;
 			goto out;
@@ -1341,6 +1344,14 @@ void handle_io(void)
 		      errno, strerror(errno));
 	}
 
+	xfm_handle = xenforeignmemory_open(NULL, 0);
+	if (xfm_handle == NULL) {
+		dolog(LOG_ERR,
+		      "Failed to open xen foreign memory handle: %d (%s)",
+		      errno, strerror(errno));
+		goto out;
+	}
+
 	enum_domains();
 
 	for (;;) {
@@ -1462,6 +1473,10 @@ void handle_io(void)
 		xengnttab_close(xgt_handle);
 		xgt_handle = NULL;
 	}
+	if (xfm_handle != NULL) {
+		xenforeignmemory_close(xfm_handle);
+		xfm_handle = NULL;
+	}
 	log_hv_evtchn = -1;
 }
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:47:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:47:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189499.339263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4w-000766-Fe; Fri, 17 Sep 2021 15:47:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189499.339263; Fri, 17 Sep 2021 15:47:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG4w-00075s-B0; Fri, 17 Sep 2021 15:47:06 +0000
Received: by outflank-mailman (input) for mailman id 189499;
 Fri, 17 Sep 2021 15:47:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG4v-0006W6-BF
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:47:05 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7ddbc655-17ce-11ec-b6db-12813bfff9fa;
 Fri, 17 Sep 2021 15:47:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ddbc655-17ce-11ec-b6db-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893620;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=1etlgP7mFsflfylfK61PxYqfdVOAtjgUkmrFPWOAfDA=;
  b=B/DH57/Bv7Qn/CdtFSMJ6FZiX6yYcLJR5Ef7TJWw4DZvYCi6ovG5JB48
   faSnNcWHuQTyc6S/iLUynCEMaEDNcY6ej9xy8SNt0ziRN9nDUkgMn0KZa
   HRZbv60mYWfhprsVSf7/jQXNvCfGSSKby+4dNJ1MxPfShh88wvVG0UE0f
   8=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 5f+G22Fki+OfZN26Pv7RKruV6RxMPHJj8Zb815KCJHFJKaoMw8qwbyHEi0mucEcrY657uV3RAp
 s/3EWfkSbLEqLBnMiuuqpIexW37AZrgrePdUBalU/HWbWtZ37nrp2hs3BkmzA5ZwwSg3vPh98z
 4ROX338qrJguUkkUs1ojw1WAidVyh5nQ9I2fuoqwH9u1X5Y8LN6GLCyvsPFh5zagX7MSdtgTzH
 FeyJi9b00jyARX9L2Dizy7lMxm/4Q9vOok8D7PgCwclpdEk5Wq7VDRpGngKDHy7JXyoVb1bn/C
 /ZmisBWXuDKzKHXBo8Vh2phl
X-SBRS: 5.1
X-MesageID: 52589406
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:pUYhq6nklf5B8/E/WBcZVdTo5gwuIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIaDGrTP/ffNGCjKtpyOorg909X7J7dn95jQARrpSowHyMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09c32YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 Ml9rp6tZhYSBa3vlNw7eRMfHAhvJZQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpq25AfQa6AD
 yYfQWcsQzTCTBwTAHZJArYktej0j1fvKyIN/Tp5ooJoujOOnWSdyoPFMtXPfceRbd5IhUver
 WXDl0ziGQ0TPtGbzTuD81qvi/XJkCe9X5gdfJWG8fptjEyW13YkIhQcXlumotG0kke7HdlYL
 iQ89iMwoIAo+UesT927WAe3yFaGoxodVtx4A+A8rgaXxcL84hmbLngJSCZbb94rv9NwQiYlv
 neZktWsCTFxvbm9TXOG6qzSvT60ITISL2IJeWkDVwRty9vprZw3jxnPZs1+C6PzhdrwcRnvx
 xiaoS54gK8c5fPnzI3iowqB2Wj14MGUEEhlvW07Q15J8CtlVJWneqO50mPbzvp4CqyTCVaor
 WA9zp32AP81MX2dqMCcaLxTR+j5vKfUaGe0bU1HRMZ6pm/0k5K3VcUJumgveh0xWioRUWKxO
 CfuVRVtCIi/1ZdARZR+ZZ64Q+8uxLLpfTgOfqGJNocSCnSdmQnuwc2PWaJy9zu2+KTPuftmU
 Xt+TSpLJSxBYZmLNBLsG48gPUYDn0jSP18/oKwXKDz8itKjiIO9E+9ZYDNikMhgtP7sTPrpH
 yZ3aJLRlkQ3vBzWSSjL648DRW3m3lBiXsueliCjTcbaelAOMDh4U5f5mOp9E6Q4z/U9vrqZp
 RmVBx4HoGcTcFWacG1mnFg4M+ixNXu+xFpmVRER0aGAgSl7Pdn3s/dBKPPav9APrYRe8BK9d
 NFcE+2oCfVTUDXXvTMbaJj2tot5cxq3wwmJOkKYjPIXJMcIq9Xh9oC2cw3x2jMJCybr58Iyr
 6f5jlHQQIYZRhQkB8HTMar9w1S0tHkbueRzQ0qXfYUDJBSyqNBne37rk/s6A8AQMhGflDGU4
 BmbXEUDru7Xro5rrNSQ3fKYr52kGvdVF1ZBGzWJ9q6/MCTXpzLxwYJJXOuSUyraUWf4pPera
 elPlqmuO/wbhlda9YF7Fu8zn6454tLuoZ5czxhlQyqXPwj6VOs4LyDfj8dVt6BLyrtIgieMW
 xqCqotAJLGEGML5C1pNdgArWfuOiKMPkT7I4PVrfEijvH1r/KCKWFl5NgWXjHAPN6N8NY4oz
 LtzuMMS7ADj2BMmPszf03JR/mWIaHcBT78mptcRB4qy0lgnzVRLYJr9DC7q4c7QN4UQYxdye
 jLE1rDfg7l8x1bZdythHHfA6uNRmJAStU0Y11QFPVmIxoLIi/JfMMe9KtjrotC5Fil67t8=
IronPort-HdrOrdr: A9a23:eVanpKD4DMFqoVHlHeg9sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LS90c67MA/hHP9OkPQs1NKZMjUO11HYSr2KgbGSoQEIXheOjdK1tp
 0QApSWaueAdGSS5PySiGLTc6dCsam6GeKT9J3jJh9WPH9XgspbnmBE42igYyhLrF4sP+tHKH
 PQ3LsLmxOQPVAsKuirDHgMWObO4/XNiZLdeBYDQzoq8hOHgz+E4KPzV0Hw5GZQbxp/hZMZtU
 TVmQ3w4auu99m91x/nzmfWq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczPgBkF5MWUrHo6mt
 jFpBkte+x19nPqZ2mw5SDg3gHxuQxeo0PK+Bu9uz/OsMb5TDU1B45qnoRCaCbU7EImoZVVzL
 9L93jxjesWMTrw2ADGo/TYXRBjkUS55VA4l/QIsnBZWYwCLJdMsI0k+l9PGptoJlO61GkeKp
 gtMCjg3ocSTbvDBEqp+lWHgebcEEjbJy32AXTr4aeuon9rdHMQ9TpQ+CVQpAZHyHqRI6M0k9
 gsCZ4Y5o2mePVmG56VNN1xMvdfNVa9NC4kSFjiWWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="52589406"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NAuMLAZWBc1iFweknCy4RbD2zLvawFtieXRQYv1GlvDlki1U+lvTHV91yR+K1PKvZZCn4WYSWyaNa0cDTdWyQqMBYhxqa2e8xMGkMGi7f4Yy3XMHzRL4Z4rLYGfRkvKlMtMsNsVG5o1rn91JkEBcwn1sIfwxFz3A4PnRntME3FClP0CjUmQsYvFU96cHSV9hetYn6yZmTw0w/7NJtB/XD8sN/26MH225ecX2B4tvKP2Qojaww8GkihQSUFgMPjI/8zGViQmo7oW7g8K3auPSDrFYSyXk/8fJSSYM4VVKRgCjl8oSVfz3Ic5rc8hF5zM4HIBd23LvBc1wO1epT8zH0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Bu0TMxjWbgiww8scS6K16pgJHFVz6vxT3LdI3IYuik0=;
 b=kUMWcBVpt1t2b4klqTFfwLJ8Pv2kMS7GU13BB314Cm7Gs4Q4HrkMaZAgX7lWOV3EU0vQpIgldAkMJPq1h0kPr8DyVSkAyg5YUEFNrVpWUaJ+rVofqrVtCf29Lzwjg0CA69rhTR7W71pXgl/9Z0PvEzC4ZdTE2TOnV6VK0XHNRJa09KY6W6nDxe/0k6TzvtP+hXEYRG/hhnHDi1UvhcXJoDX1u41DZGkfSBx9hsN9bMnVkto04x3uYlT3lU2FPot3ZyfTAKrWb8iYfmxB3wkc01wSqpBIW+oGKRXxtUHG+oRG/aDrF+X/2IEJrhj7cEvnsi494+1j0rMJi7/jDS5uGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bu0TMxjWbgiww8scS6K16pgJHFVz6vxT3LdI3IYuik0=;
 b=MKeAI1VklbvZUZHIGTO50eMJxE8o2GFHE2+tb4/+UJ4EBMgfDsoDPpT3Pi5YdYvsCKPB0XAdCUI1b4SruBTB7OBYvTNfs7Z1kGKvBmzs3Sv4hANJ8QVBrZaKs3U0lIQqvbneG5vZu4u66d0B1RYNLePHZgARnH+9lhgdTBlE4J0=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH 1/6] gnttab: allow setting max version per-domain
Date: Fri, 17 Sep 2021 17:46:20 +0200
Message-ID: <20210917154625.89315-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0174.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: de553a2c-7165-4c43-b22d-08d979f25907
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4057A414C8E9A6701EDA358E8FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: pI+Geo3HKY3FeSPmOcwH8O5Kbs7Ff39+UtD1jiHhx+OwoARP66WdbJ4mI9dHju3oRn81R36yc0icepko06kimHlQ/4VQlRk5ju56hqTNfp5tFXae3gny9od9tJCwqkb8gvn8F4VRc9zqDjhKGBjibHoBm3awp7GZOPQEoRcuyb9nv9bRq1c4iqtw0M+bb3+Pe00IfVZrO59JR/TJlDF+1N+/ZjLQWFApe05Rd/vpF565mfd4DC3wxae6vt0OItJD2n85D5VmdeIkQFG3Bsn8TWbX5okov+1FDEe2engchjOLABZHL9Xly7hjrn1TFIhzA7Dev10NoQrzLvm9bCTZMzBtAJ16fZd7Ycc/4ICO6zAyWAcQHsKcI42IAzzqR4yDOaP3SS8Lg7xrlqTZB1NIZnFHMspVWnyCC16+rKskz0y3ojNZpYvmeG7QFCezk3jK9dURvbWe4p85gE+XjtovT6Pg4I4WtUS79RHDn52R6g3f1yYR/9Q8u6nTm2H2iYtfy+ecKMhP9O6LS8PyT9GOQiPlgVkc8hOcpCT1v0tt12qifHi0LT/9KlcfXT7tYayw5FeuvsgW4IHnWECwu2cGu+yFjSAKKFAm5Iaz7yFe4g0ICpoMm/YgOoA4yJMqVuRNkw2pAojUCmvylLs/pYGqFw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(30864003)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eFFVanIrNGE2aktmSXd5UTlMMXBZQWRIb3M5NmFPcHBBSk9TQkx1ZEUvNXFh?=
 =?utf-8?B?VlRacVorRXI5ODVzejNvZVFxbldFVnpRTXYwQzYvMkNBSEFqbUdQaHBCb2ty?=
 =?utf-8?B?R3g2bWpOWVFHWjNRbDJtalBEdEc5M0pZaWhqWGtiRnluOXhmK0tvM1VteGlC?=
 =?utf-8?B?UnA0d3c5ZUgvTzAzSlROVjBNNm9mWnJJc1d4VFBuTUUwZ1hURGN0NHZCQ3Nu?=
 =?utf-8?B?NGI0WGFwVUZ3a01OeTRHaW94RTd5OEpja2FIcVNGd3c2UjI0UzNsUFVhblRW?=
 =?utf-8?B?UHNySU05SEMwQXE1VUVzaWdVTlh5ZGNmN2twTU12RmdYY2RMZUQwVlVFd281?=
 =?utf-8?B?WlNpaE5Kbnl5VEJnTlNqbGhUNEV0aG14WFZ4YTRvUHgrQ3ljY0h2YnFOUWJX?=
 =?utf-8?B?N1JmSHdmeG04azVwUnpoaW44dnVRVEMvalFNU1ZCWTZYdmpnMDJZYjFzdGZq?=
 =?utf-8?B?ZGdBdDdCVWR2UGsrMDZJMHB6NVZScFB0bkRsdlEyWnlFRmRmU2ZQeVFHY3VO?=
 =?utf-8?B?MUV1TFlQSmpBTTQrQU0rYzg1bnVjWUZvOEd4ZlRyWld1TTFVNXZwRGJUM0E5?=
 =?utf-8?B?UDQyUUxZTlEyYzVNWlVyZFZ6eSszNG1rVm9zSnh2elJCeUtKcnhYa1pVczdY?=
 =?utf-8?B?cUJyc1E3d2pzTmtXZUxMRlBweHp5NmI2S1JRTEdMTXRKUFJUeUcyWjRNWGQw?=
 =?utf-8?B?RlFnMFpZZllUK3lHR3NLSGwrYk1MS21lOGcxbHdVdlpqTGxtZStlTnFna3Rr?=
 =?utf-8?B?SFFrckMwM2k0MExVamNXNTN5ZHdHa1BPVVNiME5PWFdHaHBXNm5ieU5jM2lv?=
 =?utf-8?B?aGdUT05DODZaUEE1Y2gwTnI5M2FabUppYW0zY0hQeVdyNVhuSzYybE95TStZ?=
 =?utf-8?B?RWI1b1lDbHhGVkxMYmN0SmY2UEJ3eDRyT2IvR0dEL1dFRzBLazIrSUdyMTdI?=
 =?utf-8?B?RW5NQXVrZDJYZDRSay9pTnFVWnQxMzlKeld4b21sT0Vubk1oM2RiemlFVmtl?=
 =?utf-8?B?aHdUUmdUV0JmZDlkOGlxQjBpQmxKRitkYks3VXVkU0JSS0lUdkp2NzJUaW9v?=
 =?utf-8?B?VjhjSXV5eTZ6Z2RDbTZneFVUTjhZOTB1Q1hCQjcvNnpiTG9FOXVOcGxzU3cr?=
 =?utf-8?B?YTRBcldkaGxzdGxwOHNpL1FZTEtLR0pFdUNndHpjMlRmRjYxd0RNVmFoN3ZX?=
 =?utf-8?B?SU16UkdtbFZzKzFKZnk3bDhxY2huWWEwRkx3bGEwOEY3ckZJNVlrTEgzY1F0?=
 =?utf-8?B?UWFPc3F6MGdQYi8xQ3daWWZJd2hhVWM5blk5aHQrbHlkeXNqWE9mVGJURGI3?=
 =?utf-8?B?ZmhnYXhTN0xQN3lzaXhrZUZ1dW15Ymt2N2p1eTVwc2YrdGhTWnlKdUxCQjdT?=
 =?utf-8?B?NjROcjgrTGpOTjFtTFdvS29mZlBKK2kvMXhDV3JyRTJ5TEN6VVZZd01odkdG?=
 =?utf-8?B?MzNRcy9JZ25acENxejNVQlFiQ3I3VGNzKzFrM1drQk9XR0JFTHhGSUxualVB?=
 =?utf-8?B?MjBQY1NyL1IzaTEvRWlsTEowd0JJbTVrUUt6UnpsRDhrckR3aUlUbGJCQ25r?=
 =?utf-8?B?RnQ1OGZDM2plWXlkcVc0YkIrb3JCTlFkYkszTEREZkZ5Y0trbmsxL3ZKNE1E?=
 =?utf-8?B?dG1ucko4TXd3aVBmeU1OcjIwVU1MSTVmSWQ2SzZkUi9VUWVRd3RRVXZ5K0lX?=
 =?utf-8?B?Tlo1QUxOQ3dha1Z2UVB4MU5zQXBpZXlxZTFSY0NLcDJVb1QweGlWNmVZVnh5?=
 =?utf-8?Q?WSYdp+lw9Wwo+TxkXcbBTqRKOsKy1ZBB2nIIeRk?=
X-MS-Exchange-CrossTenant-Network-Message-Id: de553a2c-7165-4c43-b22d-08d979f25907
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:46:43.3742
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: D6NtZZjRXxfc3ualrD4KL6juyTi9ZMvENLUXel/TlD6WNn/+uaXWIdaPLSijWnymNdS1WYs9/526DBpO+0gynw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

Introduce a new domain create field so that toolstack can specify the
maximum grant table version usable by the domain. This is plumbed into
xl and settable by the user as max_grant_version.

Previously this was only settable on a per host basis using the
gnttab command line option.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
NB: the stubdom max grant version is cloned from the domain one. Not
sure whether long term we might want to use different options for the
stubdom and the domain. In any case the attack surface will always be
max(stubdom, domain), so maybe it's just pointless to allow more fine
setting.
---
 docs/man/xl.cfg.5.pod.in             |  6 ++++++
 docs/man/xl.conf.5.pod.in            |  7 +++++++
 tools/helpers/init-xenstore-domain.c |  1 +
 tools/include/libxl.h                |  7 +++++++
 tools/libs/light/libxl_create.c      |  1 +
 tools/libs/light/libxl_dm.c          |  1 +
 tools/libs/light/libxl_types.idl     |  1 +
 tools/ocaml/libs/xc/xenctrl.ml       |  1 +
 tools/ocaml/libs/xc/xenctrl.mli      |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c  |  5 ++++-
 tools/xl/xl.c                        |  8 ++++++++
 tools/xl/xl.h                        |  1 +
 tools/xl/xl_parse.c                  |  9 +++++++++
 xen/arch/arm/domain_build.c          |  2 ++
 xen/arch/x86/setup.c                 |  1 +
 xen/common/domain.c                  |  3 ++-
 xen/common/grant_table.c             | 21 +++++++++++++++++++--
 xen/include/public/domctl.h          |  5 +++--
 xen/include/xen/grant_table.h        |  5 +++--
 19 files changed, 78 insertions(+), 8 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 4b1e3028d2..a4bf2caafa 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -580,6 +580,12 @@ to have. This value controls how many pages of foreign domains can be accessed
 via the grant mechanism by this domain. The default value is settable via
 L<xl.conf(5)>.
 
+=item B<max_grant_version=NUMBER>
+
+Specify the maximum grant table version the domain is allowed to use. Current
+supported versions are 1 and 2. The default value is settable via
+L<xl.conf(5)>.
+
 =item B<nomigrate=BOOLEAN>
 
 Disable migration of this domain.  This enables certain other features
diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in
index b48e99131a..0a70698a7c 100644
--- a/docs/man/xl.conf.5.pod.in
+++ b/docs/man/xl.conf.5.pod.in
@@ -101,6 +101,13 @@ Sets the default value for the C<max_maptrack_frames> domain config value.
 Default: value of Xen command line B<gnttab_max_maptrack_frames>
 parameter (or its default value if unspecified).
 
+=item B<max_grant_version=NUMBER>
+
+Sets the default value for the C<max_grant_version> domain config value.
+
+Default: value of Xen command line B<gnttab> parameter (or its default value if
+unspecified).
+
 =item B<vif.default.script="PATH">
 
 Configures the default hotplug script used by virtual network devices.
diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 6836002f0b..dae2b68f27 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -88,6 +88,7 @@ static int build(xc_interface *xch)
          */
         .max_grant_frames = 4,
         .max_maptrack_frames = 128,
+        .max_grant_version = 1,
     };
 
     xs_fd = open("/dev/xen/xenbus_backend", O_RDWR);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d698..7a35833312 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -502,6 +502,13 @@
  */
 #define LIBXL_HAVE_X86_MSR_RELAXED 1
 
+/*
+ * LIBXL_HAVE_MAX_GRANT_VERSION indicates libxl_domain_build_info has a
+ * max_grant_version field for setting the max grant table version per
+ * domain.
+ */
+#define LIBXL_HAVE_MAX_GRANT_VERSION 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index e356b2106d..69b5419416 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -606,6 +606,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
             .max_evtchn_port = b_info->event_channels,
             .max_grant_frames = b_info->max_grant_frames,
             .max_maptrack_frames = b_info->max_maptrack_frames,
+            .max_grant_version = b_info->max_grant_version,
             .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SHIFT),
         };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 9d93056b5c..9a8ddbe188 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -2320,6 +2320,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
 
     dm_config->b_info.max_grant_frames = guest_config->b_info.max_grant_frames;
     dm_config->b_info.max_maptrack_frames = guest_config->b_info.max_maptrack_frames;
+    dm_config->b_info.max_grant_version = guest_config->b_info.max_grant_version;
 
     dm_config->b_info.u.pv.features = "";
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff653a..37789a568c 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -518,6 +518,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 
     ("max_grant_frames",    uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_maptrack_frames", uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
+    ("max_grant_version",   integer, {'init_val': '-1'}),
     
     ("device_model_version", libxl_device_model_version),
     ("device_model_stubdomain", libxl_defbool),
diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index a5588c643f..6a8658bfec 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -83,6 +83,7 @@ type domctl_create_config =
 	max_evtchn_port: int;
 	max_grant_frames: int;
 	max_maptrack_frames: int;
+	max_grant_version: int;
 	arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 6e94940a8a..5933d32c70 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -75,6 +75,7 @@ type domctl_create_config = {
   max_evtchn_port: int;
   max_grant_frames: int;
   max_maptrack_frames: int;
+  max_grant_version: int;
   arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index ad953d36bd..aad3c6a726 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -188,7 +188,8 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 #define VAL_MAX_EVTCHN_PORT     Field(config, 5)
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
-#define VAL_ARCH                Field(config, 8)
+#define VAL_MAX_GRANT_VERSION   Field(config, 8)
+#define VAL_ARCH                Field(config, 9)
 
 	uint32_t domid = Int_val(wanted_domid);
 	int result;
@@ -198,6 +199,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 		.max_evtchn_port = Int_val(VAL_MAX_EVTCHN_PORT),
 		.max_grant_frames = Int_val(VAL_MAX_GRANT_FRAMES),
 		.max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES),
+		.max_grant_version = Int_val(VAL_MAX_GRANT_VERSION),
 	};
 
 	domain_handle_of_uuid_string(cfg.handle, String_val(VAL_HANDLE));
@@ -251,6 +253,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 	}
 
 #undef VAL_ARCH
+#undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
 #undef VAL_MAX_EVTCHN_PORT
diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..0fde879cf4 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -55,6 +55,7 @@ bool progress_use_cr = 0;
 bool timestamps = 0;
 int max_grant_frames = -1;
 int max_maptrack_frames = -1;
+int max_grant_version = -1;
 libxl_domid domid_policy = INVALID_DOMID;
 
 xentoollog_level minmsglevel = minmsglevel_default;
@@ -213,6 +214,13 @@ static void parse_global_config(const char *configfile,
     else if (e != ESRCH)
         exit(1);
 
+    e = xlu_cfg_get_bounded_long (config, "max_grant_version", 0,
+                                  INT_MAX, &l, 1);
+    if (!e)
+        max_grant_version = l;
+    else if (e != ESRCH)
+        exit(1);
+
     libxl_cpu_bitmap_alloc(ctx, &global_vm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_hvm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_pv_affinity_mask, 0);
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 7e23f30192..cf12c79a9b 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -282,6 +282,7 @@ extern char *default_colo_proxy_script;
 extern char *blkdev_start;
 extern int max_grant_frames;
 extern int max_maptrack_frames;
+extern int max_grant_version;
 extern libxl_bitmap global_vm_affinity_mask;
 extern libxl_bitmap global_hvm_affinity_mask;
 extern libxl_bitmap global_pv_affinity_mask;
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd5..1206d7ea51 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1431,6 +1431,15 @@ void parse_config_data(const char *config_source,
     else
         exit(1);
 
+    e = xlu_cfg_get_bounded_long (config, "max_grant_version", 0,
+                                  INT_MAX, &l, 1);
+    if (e == ESRCH) /* not specified */
+        b_info->max_grant_version = max_grant_version;
+    else if (!e)
+        b_info->max_grant_version = l;
+    else
+        exit(1);
+
     libxl_defbool_set(&b_info->claim_mode, claim_mode);
 
     if (xlu_cfg_get_string (config, "on_poweroff", &buf, 0))
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 206038d1c0..e9a34f2f8d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2484,6 +2484,7 @@ void __init create_domUs(void)
             .max_evtchn_port = -1,
             .max_grant_frames = -1,
             .max_maptrack_frames = -1,
+            .max_grant_version = -1,
         };
 
         if ( !dt_device_is_compatible(node, "xen,domain") )
@@ -2591,6 +2592,7 @@ void __init create_dom0(void)
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
+        .max_grant_version = -1,
     };
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 8105dc36bb..af69e20029 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -750,6 +750,7 @@ static struct domain *__init create_dom0(const module_t *image,
         .max_evtchn_port = -1,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
+        .max_grant_version = -1,
         .max_vcpus = dom0_max_vcpus(),
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..0c85d89419 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -669,7 +669,8 @@ struct domain *domain_create(domid_t domid,
         init_status |= INIT_evtchn;
 
         if ( (err = grant_table_init(d, config->max_grant_frames,
-                                     config->max_maptrack_frames)) != 0 )
+                                     config->max_maptrack_frames,
+                                     config->max_grant_version)) != 0 )
             goto fail;
         init_status |= INIT_gnttab;
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index e80f8d044d..d01c6813d1 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -53,6 +53,7 @@ struct grant_table {
     percpu_rwlock_t       lock;
     /* Lock protecting the maptrack limit */
     spinlock_t            maptrack_lock;
+    unsigned int          max_grant_version;
     /*
      * Defaults to v1.  May be changed with GNTTABOP_set_version.  All other
      * values are invalid.
@@ -1917,11 +1918,26 @@ active_alloc_failed:
 }
 
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames)
+                     int max_maptrack_frames, int max_grant_version)
 {
     struct grant_table *gt;
     int ret = -ENOMEM;
 
+    if ( max_grant_version < 0 )
+        max_grant_version = opt_gnttab_max_version;
+    if ( !max_grant_version )
+    {
+        dprintk(XENLOG_INFO, "Invalid grant table version 0 requested\n");
+        return -EINVAL;
+    }
+    if ( max_grant_version > opt_gnttab_max_version )
+    {
+        dprintk(XENLOG_INFO,
+                "Requested grant version (%u) greater than supported (%u)\n",
+                max_grant_version, opt_gnttab_max_version);
+        return -EINVAL;
+    }
+
     /* Default to maximum value if no value was specified */
     if ( max_grant_frames < 0 )
         max_grant_frames = opt_max_grant_frames;
@@ -1947,6 +1963,7 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     gt->gt_version = 1;
     gt->max_grant_frames = max_grant_frames;
     gt->max_maptrack_frames = max_maptrack_frames;
+    gt->max_grant_version = max_grant_version;
 
     /* Install the structure early to simplify the error path. */
     gt->domain = d;
@@ -3076,7 +3093,7 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop)
         goto out;
 
     res = -ENOSYS;
-    if ( op.version == 2 && opt_gnttab_max_version == 1 )
+    if ( op.version == 2 && gt->max_grant_version == 1 )
         goto out; /* Behave as before set_version was introduced. */
 
     res = 0;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..7f8456c50e 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -87,13 +87,14 @@ struct xen_domctl_createdomain {
     /*
      * Various domain limits, which impact the quantity of resources
      * (global mapping space, xenheap, etc) a guest may consume.  For
-     * max_grant_frames and max_maptrack_frames, < 0 means "use the
-     * default maximum value in the hypervisor".
+     * max_grant_frames, max_maptrack_frames and max_gnttab_version < 0
+     * means "use the default maximum value in the hypervisor".
      */
     uint32_t max_vcpus;
     uint32_t max_evtchn_port;
     int32_t max_grant_frames;
     int32_t max_maptrack_frames;
+    int32_t max_grant_version;
 
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index 41713e2726..d6da067fc1 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ -36,7 +36,7 @@ extern unsigned int opt_max_grant_frames;
 
 /* Create/destroy per-domain grant table context. */
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames);
+                     int max_maptrack_frames, int max_grant_version);
 void grant_table_destroy(
     struct domain *d);
 void grant_table_init_vcpu(struct vcpu *v);
@@ -67,7 +67,8 @@ int gnttab_acquire_resource(
 
 static inline int grant_table_init(struct domain *d,
                                    int max_grant_frames,
-                                   int max_maptrack_frames)
+                                   int max_maptrack_frames,
+                                   int max_grant_version)
 {
     return 0;
 }
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:47:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:47:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189502.339274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG51-0007Yt-48; Fri, 17 Sep 2021 15:47:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189502.339274; Fri, 17 Sep 2021 15:47:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG50-0007Yc-Uv; Fri, 17 Sep 2021 15:47:10 +0000
Received: by outflank-mailman (input) for mailman id 189502;
 Fri, 17 Sep 2021 15:47:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG50-0006W6-BE
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:47:10 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 826b16fa-17ce-11ec-b6db-12813bfff9fa;
 Fri, 17 Sep 2021 15:47:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 826b16fa-17ce-11ec-b6db-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893626;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=cW68oEJxhg7ruw/Wz/7ZZTJZ5IG09WPYChe/mZQXPdQ=;
  b=EDWsobZ1a1aq4AKTefkRmALj1Skr9u60fVd0fRP/kgadxVyBY8p7TjPk
   oeRw8WOUSeUlqEDcC+h1/FXlQSl20PnCJijsmQBmTn8M1X5KXMVCtEaZw
   3qqTY6sXxN//YN4W5qgDAVsoIpgJeILKUgg1nhI0UQVRER70dCEmakIyn
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: xItb0CioMwH5Zii2Prkp7bX2rdwRmY3bKzif0LdSjIujch0rlR42CfVRLl8apsOmfX3W2HJVQ0
 vhcXVy8D1P5bQ51b8bApDdqO6k3NftypaBp3PuyHDkiTWeGs8wknPbFtv3EJ6sgUFCUZdHJNH6
 QJy02e6xSpL1/8PfLGek9NQ5vwQ0quA5OQeycHeb8NMr55d9FMuw+KLokEp2D6X45P/hBOl3E7
 P7WxaFBKIY3AzBPZQuAVCT1S2mi1sOxlJyUMolXJXaKArS6YyfW6+7+esMR1mJinvdc2ZViiXJ
 VJzLnk77rFSpBLAqn4JhewXO
X-SBRS: 5.1
X-MesageID: 54784990
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:1VvuP6tZXAqzZ8MeXI9zlWjPc+fnVIJZMUV32f8akzHdYApBsoF/q
 tZmKWuAb67fMGuket1ybtvl90oD7ZaDxtQ2TAtl/CBjEXkU+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y52IfhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npl6p2xQzoMJKT1scskSkRcSnklE79G5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25wRTaqPN
 pJxhTxHbB/wSTdXAgYrCZcPuuCUtFnCYxxIgQfAzUYwyzeKl1EguFT3C/LXdcaNXt59hVuDq
 yTN+GGRKhMHMN2SzxKV/3TqgfXA9QvkXKoCGbv+8eRl6HWDy2pWBBAIWF+TpfiillX4S99ZM
 1YT+Cclse417kPDZtjiXTWorXicpBkeVtFMVeog52mwJrH8uljDQDJeF3gYNYJg5JReqSEWO
 kGhwNLoDwFKqrSsYF2k7+qQ9GvpFzoVMjpXDcMbdjfp8+UPsalq0EmWHo0yQfbv5jHmMWqvm
 GHR9UDSk51W1JRShvvhpTgrlhrx/sChc+Ij2unAsotJBCtCbYi5bsSD7VHB5J6sx67IEwHc4
 BDodyWYhd3i7K1hdgTWG43h/5nzvp5p1QEwZnY1RPEcG8yFoSLLQGypyGgWyL1V3iM4ldjBO
 xS7hO+szMULYCvCgVFfOtrsYyjV8UQQPYu8Da2FBja/SrNwaBWG7ElTib24hju2+HXAZZoXY
 M/BGe71VC5yIf0+kFKeGrdMuZd2l3tW7T6CGvjGI+GPjOP2iIi9EuxebjNjr4kRscu5neki2
 4wOb5fVk0oPDrKWj+u+2dd7EG3m5EMTXPjeg8dWavSCMkxhHmQgAOXW2rQvZ8pumKE9qwsC1
 ijVtpZwxAWtiHvZBx+Nb3w/OrrjUYwm9SAwPDA2PEbu0H8mON794KAafpoxXL8m6O08kqIkE
 6hbI52NUqZVVzDK2zUBdp2h/oZsQwum2FCVNC2/bTlhI5M5H17V+sXpdxfE/TUVCnblrtM3p
 rCtj1uJQZcKSwl4ItzRbfajkwG4sXQHwbogVErUONhDPk7r9dEyeSD2i/Y2JeAKKAnCmWTGh
 1rHX09AqLCU8YEv8dTPiaSVlKuTErNzThhAAm3WzbeqLi2GrGCt9pBNDbSTdjfHWWKqpKj7P
 bdJz+vxOeEslUpRt9YuCK5iyK8z6oe9p7JeyQg4TnzHY07yV+FlK3iCm8JOqrdM1vlSvg7vA
 hCD/dxTOLOoPsL5EQFOeFp5P7rbjfxEyCPP6fkVIVnh4H4l9bWKZkxeIh2QhXEPN7ByKo4kn
 b8stcN+B9ZTUfb23gJqVhxpylk=
IronPort-HdrOrdr: A9a23:E7TCM6zPfimGcrC7Ey+MKrPxv+skLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjmfZq6z+8L3WBxB8bfYOCCggqVxe5ZnPLfKlHbakjDH6tmpN
 1dmstFea3N5DpB/L7HCWCDer5KqrT3k9HL9JTjJjVWPHxXgslbnnZE422gYytLrWd9dP4E/M
 323Ls4m9PsQwVaUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZtzU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDh1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoykfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW/2/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 gvMCjl3ocSTbqmVQGcgoE2q+bcGUjbXy32A3Tqg/blkQS/xxtCvg0lLM92pAZJyHtycegc2w
 xoWp4Y442mdfVmGJ6VMt1xCPdfOla9My4kD1jiVWgPbJt3QE4li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="54784990"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fPrljNJuUzlaiE3uB7BySX4ZgGxchSW3zB/SUI87DnmM4V2i/ve1V5LJUNGL7YgQDM6MN5wSOmk3e6RlmPS9QwoFjcr7TYvegKJWro4H0kqMHhzE+Y8PXuYZmFTVm8d4GVIOeVSQtHoBBzpuX1QpgjkC8RwNonP1hw8RKtqvScUtQedFYu4bjzs957A0463RgUJaVPIktlbKi6sDIhh8e3NKsJfSM/ukZx9vRkNnJFRIPtW7mj6FjmAiRP9RXJYcmNxAcZ5M5s/1KQv5LSAKabtyosn5r6pqIR1uiTD48Dn3nSP6lPZ3P06cRsrXJjr/gYtwvS1wNUhJcaDRVq7rbA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PD/IfqxTWeG2APbkwJ0yiHnyMtoPw+kS5Zi2MNtycSo=;
 b=h4vL46bJUIRvTbg8xpKGn7bRQo64FPPVSslmKMhEMiAF8NH9Mb42S2v/0rqzYGXQr5AtqgO5pQnoSs23fRjpxGXz4NVU8hrMpvY0BG+A9QPqe8cabewtXOBEaJEdWioK7YcpjXZMkN3zumz6RgEkhJjzzSlHRjFVWUpRn777IsfgZnCBqxSuZ86mK8gGsqlArnpOicfYvTFIRA1KlqZ22P9O2Vm6b6T+oZs1kz6slSaakkoKcJZcSNZPr19ze456wud7S+Q5WAA5bdxD1vEoUH6wgFyzGPh7gFTe5jLqGdJJB0mTttinuP18fH/J9JLVEk5WZbxWZgi9LrG37CnyUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PD/IfqxTWeG2APbkwJ0yiHnyMtoPw+kS5Zi2MNtycSo=;
 b=m3kTSLDzfO7UgZuI6VMI+XhP63NJQovDXRrvdJEhI2BWuDvdBjgw7oIY/yM2DS3+cIP0UprQHfPo+SVqpexnNmeDrEznHzfVKvWZ7slcKmumhecgSwJYlGggtw6rBKugAvsiZV7UAccdduA4mwzG4kSfpqQEEFfQksPPXckNv+s=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>, Julien Grall
	<julien@xen.org>
Subject: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
Date: Fri, 17 Sep 2021 17:46:23 +0200
Message-ID: <20210917154625.89315-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0415.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18b::6) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d2de9c44-070a-4d21-4c1c-08d979f2622e
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB40577209041B6852C18BBCE48FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: OTZ/QHx/pBtUWN3R32f+OFxoi37zvv5PVbCqVR0ZLI/M9FcJKBfXz1BWV+46XSMz8njvmk7J9iwdk2jcz7ftLgd2eXzCuMowWzqEYrYw26scAF5R68IbX2q9/k5w2VzxQg/3fVeiC9sRctdcnu5+E93LhyGOa91syHL7m9njtZzOP32G4hu9564ZHchC+kGmXeI6kQFgkcNu8BXm64LJRvXgoevjE0+kBaC870YOO9Y9eHmcKnPTkditQ6dZ1y1Czeb8KI+uECqxXBC5K8ditWsRfrCGukejv1XMPkNqluzHwTp1WabATkKyt9KosKf0oUed63SHon/1aR/pIWBDq+TEhKDiT+Kec7+rDpX3myxfZE4dVgI4/XU+QGX/jJLteC3FtzQW8cD+jMX61n+qilPprtH/oRh6unBPz9XeeL5Kkx5fHOZaqZgB3sBgGexHDWKt7wyUwwhgk+AmmCV43Kn0XxS295CADF8i4mETA/CAvklfXusdolPTPEnun2w6v8nesQ30ecJBpK4JUMyp2kjowCw5YfjJrXmIdHG2cUrorKDZaQC+fMDw/8yuPVIIwGaB3EtNz9XVcu+BnMrSxCRWsIjsrcIIOgILJUcLuLZ5BPSPh8fu1pJyAGbn7yXqknScxLmkyjb2tTacOWoORw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cjhYUkRGU0MyZjRUOFE2RTJac1NLL1VGMVI0cW5PelpZTzdZby84ZG4wY283?=
 =?utf-8?B?VDF3NDVJNDBHMFZPbjBJUHhkenFlZXlkbklPQUl2YWF6WVdlNmhIalRiWjhD?=
 =?utf-8?B?QnhzbGw5OXg5VTJrYjZRWm1IZ2JzZHJkM0dsWXJZZzUxMWM4WWFqbU80akph?=
 =?utf-8?B?aU5sRm1rU21JUjMzeFlwcXRUL0gvME1VMEU3KzhiRlFxUFBYY2VsN2RvSnZx?=
 =?utf-8?B?ejZ2MXg0VXpQbmJ3dU5LS2lJL3lIOVRZYXNIVXQ2TGV0V1BVSFRTd3JlVzNj?=
 =?utf-8?B?RGttemtCRE12VTlUd21qdGtxcWpIZWJPMXBXRWIrZFdTSmNkdEp4a01wam1u?=
 =?utf-8?B?dG1pOHdNT1BWaXl1T0RrRU9ZQlVsZEZKNWRiYlVqZnlKa2RML00rcjRSYVRN?=
 =?utf-8?B?QXQvNVdJU3dZbWltaVJSbGJaZjRsdEdwYVVNRklYVy84ZTJxVzRkMmRqb25z?=
 =?utf-8?B?L1B1Slp0WElrZ0U2ZjlFNzBZMG82dytvb3NSWjJwUDBablVYR2VXSzgxalNs?=
 =?utf-8?B?N2ZlVjlSSDFNdHVMSDczQmNWbmwzaEZsZTdxUUR5ZkhZZXBHYWJWQS9hUDZv?=
 =?utf-8?B?Tm9yS3BPbnQ0QVNRSmFqUVNWMEhBU2VXc1hGc3FtanNMamZ0NTFGYnFkTXkw?=
 =?utf-8?B?Y3NIa29VWTdoM3ovS1B5RWgzeTdYTWl2aEcyalVjdDEvNXM2azJZRmZ2Rnpo?=
 =?utf-8?B?N05Pb05FeVQwcjJvSFVIekNGWXVRZnpQVkY0OVQrV21RYVVZSlFhK3U0aHJK?=
 =?utf-8?B?YW9jMFExaVViVVdyWHZ4ZGVVWER2aXpTMjgrWG9laUVoYUlNdE02ZXpJNk9Y?=
 =?utf-8?B?WEV5WkRCM3ZnSm82a2lMQk5yNWpZeGFBUHF6aHhKNnBmcGhsU2x1ejNOS0Ju?=
 =?utf-8?B?Y2V1RUp3VVo4RXpKMTV6Ukc3VDcxVkJiRk44L1Nscnk4NDhudXdubEpZMWE0?=
 =?utf-8?B?ZStxSTB0M2thR1VkUEpRaVFvNkdVTUVtcGpjYVQwcjFMYVM2TXBCckdWbWl3?=
 =?utf-8?B?TmMvbklxZGhTQURES2ZsNjM5UjRISHJ2UUFaTzhYL3FDZ29iRUxWRmFoQ05y?=
 =?utf-8?B?dGpPTkZaajI5K1BhSkozait0QVE3RmpaVXRIOWE1R00wOXZCRGVxUHYzTEkw?=
 =?utf-8?B?V2ZUQWlZbnZ1czh3WEN6enpJcWhoeWRWZXVCaUtFbzM0alFVelZIc2MzMnJu?=
 =?utf-8?B?dmlMZ3NHQzViNGFaV205endnZ2tGdlB0eUxvVU1QU3FQbStjTXllR29rSlRT?=
 =?utf-8?B?RE13MmZYc2czZktjNlFZYXloYUhRdlNUcWhNMnVhSkwzUjVDVXNYS0JNM040?=
 =?utf-8?B?TE1WcWY1RXdNTjZlb2wrVUhuamhBaEVxTTdORVB3YmNNOEx4dVVDZzIxb0xV?=
 =?utf-8?B?Q3kveVZEN3BleWZ4bWxKOXZ5RDE3bEsxNDNvZjROcTRFc1I5VkQ4Tk1XZEhy?=
 =?utf-8?B?RGFmWUlZL0R5eHNodTV0OFhDTWRGZk5GZHhpZDlGa0l2R1NwSUt2MHIxUm1w?=
 =?utf-8?B?Y01tWW1iMGR2NXljZ25NRVl1K1VQZHh5SU5tWTFhMVZBbGp4T2lhSXgyaFlM?=
 =?utf-8?B?dFdFUGh1d2xXeXB5ck8yZ1dSelFDWE5Jb2hkVDA4V2kwcVFFL0NsNENjVjh1?=
 =?utf-8?B?elFBT0txL2ltRVo2dGZLaWtlN2t2MzlVSzNrTFZRSUJsaC8zdndDaktnZENi?=
 =?utf-8?B?bktzd2dKdjJKakxTalpwYzBucjdtY3I0NHFIaGg4Y1ZYK1N3ekFZYXFEaXRz?=
 =?utf-8?Q?Xu2Z635D5HLPdFXC1D0O/ZjNQIjrua9ey3hB479?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d2de9c44-070a-4d21-4c1c-08d979f2622e
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:46:58.6929
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DmsC/IgnDRv+SzLndDcxq1N6dIs2mgYBIwBPWehRmFqaS0vbg/ld5qFtVLd8s5chucux5+C864q6EUTE3hgEDw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

Exploiting the talloc clean up routines to close the Xen interfaces
is cumbersome, specially when atexit can be used to the same effect.

Convert xc and gnttab to use atexit which allows to drop one
indirection from the storing variables.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 tools/xenstore/xenstored_core.h   |  2 +-
 tools/xenstore/xenstored_domain.c | 57 +++++++++++++------------------
 2 files changed, 25 insertions(+), 34 deletions(-)

diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h
index 258f6ff382..a813bc5a8c 100644
--- a/tools/xenstore/xenstored_core.h
+++ b/tools/xenstore/xenstored_core.h
@@ -266,7 +266,7 @@ void init_pipe(int reopen_log_pipe[2]);
 #ifndef NO_SOCKETS
 extern const struct interface_funcs socket_funcs;
 #endif
-extern xengnttab_handle **xgt_handle;
+extern xengnttab_handle *xgt_handle;
 
 int remember_string(struct hashtable *hash, const char *str);
 
diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
index 47e9107c14..8930303773 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -35,8 +35,8 @@
 #include <xenctrl.h>
 #include <xen/grant_table.h>
 
-static xc_interface **xc_handle;
-xengnttab_handle **xgt_handle;
+static xc_interface *xc_handle;
+xengnttab_handle *xgt_handle;
 static evtchn_port_t virq_port;
 
 xenevtchn_handle *xce_handle = NULL;
@@ -198,14 +198,14 @@ static const struct interface_funcs domain_funcs = {
 
 static void *map_interface(domid_t domid)
 {
-	return xengnttab_map_grant_ref(*xgt_handle, domid,
+	return xengnttab_map_grant_ref(xgt_handle, domid,
 				       GNTTAB_RESERVED_XENSTORE,
 				       PROT_READ|PROT_WRITE);
 }
 
 static void unmap_interface(void *interface)
 {
-	xengnttab_unmap(*xgt_handle, interface, 1);
+	xengnttab_unmap(xgt_handle, interface, 1);
 }
 
 static int destroy_domain(void *_domain)
@@ -240,7 +240,7 @@ static int destroy_domain(void *_domain)
 
 static bool get_domain_info(unsigned int domid, xc_dominfo_t *dominfo)
 {
-	return xc_domain_getinfo(*xc_handle, domid, 1, dominfo) == 1 &&
+	return xc_domain_getinfo(xc_handle, domid, 1, dominfo) == 1 &&
 	       dominfo->domid == domid;
 }
 
@@ -648,18 +648,6 @@ int do_reset_watches(struct connection *conn, struct buffered_data *in)
 	return 0;
 }
 
-static int close_xc_handle(void *_handle)
-{
-	xc_interface_close(*(xc_interface**)_handle);
-	return 0;
-}
-
-static int close_xgt_handle(void *_handle)
-{
-	xengnttab_close(*(xengnttab_handle **)_handle);
-	return 0;
-}
-
 /* Returns the implicit path of a connection (only domains have this) */
 const char *get_implicit_path(const struct connection *conn)
 {
@@ -737,35 +725,38 @@ void dom0_init(void)
 	xenevtchn_notify(xce_handle, dom0->port);
 }
 
+void interface_cleanup(void)
+{
+	if (xc_handle) {
+		xc_interface_close(xc_handle);
+		xc_handle = NULL;
+	}
+	if (xgt_handle) {
+		xengnttab_close(xgt_handle);
+		xgt_handle = NULL;
+	}
+}
+
 void domain_init(int evtfd)
 {
-	int rc;
+	int rc = atexit(interface_cleanup);
 
-	xc_handle = talloc(talloc_autofree_context(), xc_interface*);
-	if (!xc_handle)
-		barf_perror("Failed to allocate domain handle");
+	if (rc)
+		barf_perror("Unable to register cleanup handler");
 
-	*xc_handle = xc_interface_open(0,0,0);
-	if (!*xc_handle)
+	xc_handle = xc_interface_open(0,0,0);
+	if (!xc_handle)
 		barf_perror("Failed to open connection to hypervisor");
 
-	talloc_set_destructor(xc_handle, close_xc_handle);
-
-	xgt_handle = talloc(talloc_autofree_context(), xengnttab_handle*);
+	xgt_handle = xengnttab_open(NULL, 0);
 	if (!xgt_handle)
-		barf_perror("Failed to allocate domain gnttab handle");
-
-	*xgt_handle = xengnttab_open(NULL, 0);
-	if (*xgt_handle == NULL)
 		barf_perror("Failed to open connection to gnttab");
 
 	/*
 	 * Allow max number of domains for mappings. We allow one grant per
 	 * domain so the theoretical maximum is DOMID_FIRST_RESERVED.
 	 */
-	xengnttab_set_max_grants(*xgt_handle, DOMID_FIRST_RESERVED);
-
-	talloc_set_destructor(xgt_handle, close_xgt_handle);
+	xengnttab_set_max_grants(xgt_handle, DOMID_FIRST_RESERVED);
 
 	if (evtfd < 0)
 		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189504.339284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG55-00082j-Ed; Fri, 17 Sep 2021 15:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189504.339284; Fri, 17 Sep 2021 15:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG55-00082R-AS; Fri, 17 Sep 2021 15:47:15 +0000
Received: by outflank-mailman (input) for mailman id 189504;
 Fri, 17 Sep 2021 15:47:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG53-00068c-V6
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:47:14 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a040cada-593e-4371-9c63-1d06627651ef;
 Fri, 17 Sep 2021 15:47:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a040cada-593e-4371-9c63-1d06627651ef
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893627;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=8yaSBJfS4uBB9CmxQZt73y0W3kku5w3/WtdaE+LvS9c=;
  b=gv+pTvZeWfWw29hMFNISOvI6h8ZTuZHGF2cn2g7SQBcYJ90MRrqavERQ
   nlliqexqskgULnKIlvDrzQ2v9CV0N8rkPXpyq7Hf4ZbKboqSYzUeNvLBv
   xtr9eU8S4E1F4RUFZY/Irl/VZThPyLMw/SoTJE9SRUXitMHeItB6cb9tT
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: EmJ3mxN3nx6zzStBk3fTRuwk7pnO7KwmLP0B2u8QxQXnQ51V592DGCTH2sRhkYhlmxJhZR1KBn
 omw1YMRBKrFaM3R8PUeLPqCM8uJNZOV6a02ZOffodFAisS1GHCq8ns6Jo6HGvKMUX3HHgTPr9d
 PZHuDzcsl7MvIH3qOIgiIQhdT4ie2FTKOtMHEO1XHn7cyFQhpARIL90EI6mEOCG05XTLvqNZ7z
 ve8bumARwld8WapcgTsCzhmeLinps8Uo9F4BoXdzUpHEsKQMHwm91Mpot3vaWJzaKnlYENLTce
 W8QWfSe0NT14qHLrw1j34oLc
X-SBRS: 5.1
X-MesageID: 53410957
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:3Pnbz65wYTCpQmsHDM4yFAxRtOzAchMFZxGqfqrLsTDasY5as4F+v
 jFMX22PP//fNjOkc4tyaovi8khT7JPTzN43HldvrHs9Hi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo29Ew27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z6
 /Ngn5CcRzcQGKzpibRMaUJZAi1YIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWxs35gfQKm2i
 8wxTwNUXRTiPj12ABQ4EckVxcqToUKkSmgNwL6SjfVuuDWCpOBr65DvP8DSYceiXthOkwCTo
 Weu12bkBhAXMvSPxDzD9Wij7sfQmQvrVYRUE6e3ntZ6jVvWymENBRk+UVqgveL/mkO4Q8hYK
 UEf5mwpt6da3E20TPHtUhugunmGsxUAHd1KHIUHBBqlk/SOpVzDXy5dE2AHOIdOWNIKqSIC9
 Xmktd/FCwxWlrywWCLMzOyVvD+UJn1ARYMdXhPoXTfp8vG6/tpq3kmeH445eEKmpoaqQmCrm
 lhmuAB73u9K3JBRj81X6Hia22rEm3TfcuIiCuw7tEqe5wVlbcaOY4Wy4DA3Bt4Rcd7EEjFtU
 JUC8vVyDdzi77nWz0Rho81XRdlFAspp1xWG2jZS82EJrWjFxpJaVdk4DMtCyKJV3iEsImSBX
 aMukVkJuM870IWCNPcqC25ONyja5fe5Tom0PhwlRvFPfoJwZGe6ENJGPBXLt10BZHMEyPllU
 b/CKJ7EJS9DVcxPkWrnL89AgORD7n1vmgvuqWXTkk3PPUy2PyXOF9/o8TKmM4gE0U9ziF+Eq
 4oAb5TalUo3vS+XSnC/zLP/5GsidBATLZv3t9ZWZqiEJA9nE3smEPjf3fUqfIkNokifvr6gE
 qiVVhAKxVzhq2fALAnWOHlvZKm2BcR0rG4hPDxqNlGtgiBxbYGq5aYZVp02Ybh4q7Azka8qF
 6EIK5eaH/BCajXb4DBBP5Pzm5NvKUawjgWUMiv7PDVmJ8x8RxbE88PPdxf08HVcFTK+sMYz+
 uXy1g7STZcZaR5lCcLaNKCmw1+r5CBPk+NuRUrYZNJUfRy0ooRtLiXwiN4xIt0NdkqflmfLi
 V7ODE5B9+fXooIz/N3Yvoy+rt+kQ7lkA05XP2jH9rLqZyPUyXWunN1bW+GScDGDCG6toPe+Z
 f9Yxu3XOeEcmAoYqJJ1FrtmwP5s59broLMGnA1oEG+SMgauA7JkZHKHwdNOputGwboA4Vm6X
 UeG+997P7SVOZy6TA5NdVR9NunTh+sJnjTy7OguJBSo7SB6y7OLTEFOMkTekydaNrZ0bNsoz
 OpJVBT6MOBjZs7G6uq7sx0=
IronPort-HdrOrdr: A9a23:GHULGqq7icN6S65es1HxuPEaV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBfhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSpQEIdxeOlNK1kJ
 0QCJSWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHtXgslbnnlE422gYypLrWd9dP8E/M
 323Ls5m9PsQwVbUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZszU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDi1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyUfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW82/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 guMCjl3ocVTbqmVQGdgoE2q+bcGkjbXy32DHTqg/blkAS/xxtCvgwlLM92pAZIyHtycegD2w
 xoWp4Y4I2mdfVmH56VMt1xN/dfOla9Mi4kD1jiVGgPbJt3Q04li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="53410957"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gJP/gv9BZcBvmv3vrYTwxXY4zUImWbh5MbXJz1n0j9jdXM2hcCbVHCdl+0laV8EieWioyw0RCSKuA+Q0H5e32NLVqN0TlRnt8VYj7p+zaoCaK9OUexC2kGa62aMXtQ2miPnBk2fpRdEUCI6qThc/kfjfyjuwadzQMwYpm5yMmSqOZe4vrRY3ghvL5fcm/L4hr2QHnWNo3uCxigj18eP9nTFuDMwMoYLqk938avRkNPO5ki17EbGmSpyH1XzcS9o2g8AJzVqqvRO52X5AD0agSy15IC3gFCLwYODAabOomk+odtXDj3FpQGaygxHTwKgr6EIHiHsuQYWkrmbdzLgM7w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ODViCmJYnKpIZT6ta/bzfRMYz5jZaA8JqNXfGiUOkig=;
 b=jPzNrGxDY1wy9FTQdtSiC1MqrQMqWyrrvdYuY3/xsQOvXuQU/b07v8I3RTq92Is6m4am0eAPl3br7JVrpCisBvJ5U7ycqmChW6CdQAFDC0l8FKY528N5il3T1wyaJ/3vR2UF2h7cpIDvjrEBOKyaXXI4lcp5OU9ufEIsp6TT1uUlSGNGrGQz22Bvn2S11DVpeoaNgoig6zFx9N3Ebmu3wwE9v//jadcGrjmZDowzuwI5F6qdzKngOCKFhrEdndV1I9ZP7oIw6nG5QzkBD/R6aIf98BF2Mg9Dexjf8slLvnXiXkIYAHHEPBRS8Yf1D68ShRFqOw0zVkNlgZXKRFnCMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ODViCmJYnKpIZT6ta/bzfRMYz5jZaA8JqNXfGiUOkig=;
 b=gFHgp9xe/umT5196u6fsHKnMDL0VZEayKmXoRPEEfD5tKTV0yVuU1pRjL8RmDQ0jiDNuNzZBV9xmlLHZgVDkyOS6G6EjbOWWGEuU+UKzZyz9AvqFk8H2My3oI31G/UNHxI7SywmnYmxYfXK+vGq1l3/8POBYAl+SCZ/Rt1W/sVc=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>, Julien Grall
	<julien@xen.org>
Subject: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring as foreign memory
Date: Fri, 17 Sep 2021 17:46:24 +0200
Message-ID: <20210917154625.89315-6-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0148.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9::16) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f94f4209-069c-46ed-38ae-08d979f26567
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4057C187EA79259F7A27939F8FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 9NR5mG7bGcEpi2nyflGUVC9ZRtjpY5LvZbSOvBHOeOlj2eGwZmQSqf/G+48ZRrWvuRkAJl4YBFpuCPsPxRdkMgqco3JaC4M4kGzwJw8saTgpmH3PfEge6zk9BdxEXtm/OVLhj0yUeZ4ttl0XIi4+1dwtnFOhh22Kljhdw3LZnCZVFXkgq+5S2goGFDpTCVkQB27Pzt+CsSNYzPPSm+k6Ttg6NFeTkS2h3tiW70oFeKZM2JEwHmVzqgCBveKARwxObfSerb+cVbz+dfOiJAiXEi9E9MO9uzajdELiGSpkV0UACk2ZvFc2zEiKUPuPmbiILvZ/YXgl5oVVhltD/hSofgA1OiXaVBcM06t8GcpZc6Ar8wWrF1blqIZVXrmW5TxJO8LPITahTCNZeWOB59yLZiLlHWeWFGakdUS8jq4lh85GLPUwh7pryDs+jaxyko+8PpVXNZ15b681eEkOJ0eR5IIph7R77vMZM5/A3K23Izq69JspH5M9z5PpYTfUPxvMJXcHjGWxXoUco/4HO0X5Aoj+khJht7eRWf6gKZ7TN4kmZV8dY6woFWLUObRnnVZ31uZ/EdnGxj+K40lBUG3nIoCBv2RVOvPP80E0Ddg/tFw7Gr41Z2CZqciQNP7velw7QeBwTrUy3oMPeXWXGGD5NBeLG22bUHISH4dhm87NbsBUrXlTdu3A9jTGY9RUbJop4QB3ftwQ3EC9pWfcvJI7Vw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002)(309714004);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZmNGN2VOd0FncGRYdUhxZ1ExWUR0VDBYMWRJSVBMUGJXSi9XcC94VWhaczRv?=
 =?utf-8?B?RWNkNHF2TkZTb0x4U243OHpNZ08wdnVBbXZPMFJxN08yK1RVbnVOWkN5SzJT?=
 =?utf-8?B?L3l0WGUvdUZid0hVMW5XZ0Jtb2lDL2kxMHBsSE5Cb05KdWFxM2lGN3V1ZmZh?=
 =?utf-8?B?enF6dWJsbGNkcVppaUFUeDNRQVU2VWIrNlhOMzFYYlVha2s5STF4aksrZnEx?=
 =?utf-8?B?YjRlakcweVNUMVZtL0VxSTM3UFcwelMzUWd3YTN0cGxNZWJQL1ZSaHREYWhG?=
 =?utf-8?B?bFY3YlJwaW94RmRFcnhMY1BPd25tZm1mTUZ0SzdrVFFaT0dIUEhVaU13TG1T?=
 =?utf-8?B?d1F5WWFMbTQvS1Bsa3hxcHhXSWVzcTlXaGIxT2ZGcitleS9UZEo3LzFtTEhZ?=
 =?utf-8?B?S0lQTGp5UGxVYXYzaDJ6dkRRdzBEMFZwYkZxL0t3ZjFzNjhhaXU1cnY0ZUxq?=
 =?utf-8?B?OHNqTzZpKzB6T0hEankvbmllWHpYVFZackI5WldjVnhOOCtBT3JObU5qVXhI?=
 =?utf-8?B?c2dXWSsxZjFxUDdnL3pjbnNlNDU1TnczQm9wOFAzUkxxQ3BpS205cmhkUFpp?=
 =?utf-8?B?Y3JSL1ZqZjUwcWxLTXlSK1VlcksxK3hJZlJkWk1ZUHptZzhta3NzaE5vMVU3?=
 =?utf-8?B?TnE2SDhpRW10MEdNaU90MVdITVh0U2lMZmhLTWlpNW1QTWZCMllVVi9ra3Z3?=
 =?utf-8?B?TEJabDlRUUJyT05QSVQwb2tVSHdpN3FBRHdJd080YloycHVCOUJWY3VqTHhS?=
 =?utf-8?B?Y0czUGk2NS9hcWpWRU03QVNSZVkrUmlESTRIWEhhcmRaSU5rNGVCaWRlelZO?=
 =?utf-8?B?RXpGMTFwRXNDcFNxZHQ5Tzd4U3lqRVR5YU50ZGQwVWpHaXBUQ2RXZTN6dUhK?=
 =?utf-8?B?QzFmejdMSHNjaWRtRDcxbGoxZXhFWjIxWlJDbzZsRGFLSlNOaWhTRFd1VEJW?=
 =?utf-8?B?eHZJVTRSdHJ0bWFibFZKYUpYOUpFVUJvbjZLYXhUWVRPbnhmNUl4TUhvNmtr?=
 =?utf-8?B?RkxsR0xDSWJCYmg3NDJxTU0xSXNlbXA0V1VGemlkZEJObktQQ3JZQUlDV2JK?=
 =?utf-8?B?MWZPNEhTL1R0WUlNczBGK0hlUncvUWUyYkMyTGt5WWNWbVZtd3Nab1hjaFEr?=
 =?utf-8?B?VFExdDdPS1daNExPbFhTMVFIcXRKcG0yTWdpdy9rNWU4SVJ4dzNXVlhtRUx1?=
 =?utf-8?B?em5KcXphazVMMTNGWFNLQlpONkdwN29QdWVsaHlFRW81ZEtwamUxZ2p0RHJt?=
 =?utf-8?B?UnFRZENRclpIUmVMQ3JvNjAyTmd6RitRV1dSQ29OUllvWHdmZzgyU0dybG1h?=
 =?utf-8?B?ek1Pc05YNzU0L3MvS05VNWFxL2VqS3JZQmRrbmxiL3pkOFRsVjFTOUpSZEl1?=
 =?utf-8?B?VDkvVFlyOEwrRmU1UUpqbG9mOExGUmo3bjgvSmVuaVBVb1JMN051dnZsUE1O?=
 =?utf-8?B?UGkxZWdmY2F5Q3hDak90ZVRvTnJ6eEhCa1ZFSmJLU29qb1J0TVFkTnp4WXQv?=
 =?utf-8?B?bHpLV09kSGs5d0hFa3F4OGJ3YmJYM0hrUituNGtwYmNDaWp6aklxMm9za0h3?=
 =?utf-8?B?SnpPL2NYLzU2cEJ2cEV4MGtCNEMxTmNZN21DYlRqdjNUN3BhK2MyMzRaS0l4?=
 =?utf-8?B?WnNhNE1TMk1YOG5SY1hWRUJDcndiMTZHSVcvMzdmLzM4dGtHVWhvWktxN0h0?=
 =?utf-8?B?dHhUNXZrOU9acGRnbHgvVWJzQ1dsaFhZaG9nTDdSVU1tQmlLSUQ1L25iRlRF?=
 =?utf-8?Q?XPT9cdnrStLfuGIE5EowHg03ZMRk9pCXGQIhsAQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f94f4209-069c-46ed-38ae-08d979f26567
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:47:04.1160
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5MZKf3+IwtwmBdpfSJTqDvL035q0mNj6+6N34pm0VNYMawg0AZZdZJenSB5SigB/Fmh/1K7B6RLnSSJpdniqfQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

Restore the previous way of mapping the xenstore ring using foreign
memory. Use xenforeignmemory instead of libxc in order to avoid adding
another dependency on a unstable interface.

This in turn requires storing the gfn into xs_state_connection for
resume purposes, which breaks the current format.

Note this is a preparatory patch in order to support the usage of
xenstore on domains without grant table. This not only allows xenstore
usage, but also makes things like "@introduceDomain" events work when
using such guests, otherwise the mapping done in introduce_domain
fails and the "@introduceDomain" event won't be signaled.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 tools/xenstore/Makefile                 |  4 +-
 tools/xenstore/include/xenstore_state.h |  1 +
 tools/xenstore/xenstored_domain.c       | 69 +++++++++++++++++++------
 3 files changed, 56 insertions(+), 18 deletions(-)

diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile
index 292b478fa1..9a9f7be5cb 100644
--- a/tools/xenstore/Makefile
+++ b/tools/xenstore/Makefile
@@ -67,10 +67,10 @@ $(XENSTORED_OBJS): CFLAGS += $(SYSTEMD_CFLAGS)
 xenstored: LDFLAGS += $(SYSTEMD_LIBS)
 endif
 
-$(XENSTORED_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+$(XENSTORED_OBJS): CFLAGS += $(CFLAGS_libxengnttab) $(CFLAGS_libxenforeignmemory)
 
 xenstored: $(XENSTORED_OBJS)
-	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
+	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxenforeignmemory) $(LDLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
 
 xenstored.a: $(XENSTORED_OBJS)
 	$(AR) cr $@ $^
diff --git a/tools/xenstore/include/xenstore_state.h b/tools/xenstore/include/xenstore_state.h
index ae0d053c8f..8dcc8d9d8b 100644
--- a/tools/xenstore/include/xenstore_state.h
+++ b/tools/xenstore/include/xenstore_state.h
@@ -80,6 +80,7 @@ struct xs_state_connection {
             uint16_t domid;  /* Domain-Id. */
             uint16_t tdomid; /* Id of target domain or DOMID_INVALID. */
             uint32_t evtchn; /* Event channel port. */
+            uint64_t gfn;    /* Store GFN. */
         } ring;
         int32_t socket_fd;   /* File descriptor for socket connections. */
     } spec;
diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
index 8930303773..f3563e47aa 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -33,10 +33,12 @@
 
 #include <xenevtchn.h>
 #include <xenctrl.h>
+#include <xenforeignmemory.h>
 #include <xen/grant_table.h>
 
 static xc_interface *xc_handle;
 xengnttab_handle *xgt_handle;
+static xenforeignmemory_handle *xfm_handle;
 static evtchn_port_t virq_port;
 
 xenevtchn_handle *xce_handle = NULL;
@@ -66,12 +68,18 @@ struct domain
 	/* Generation count at domain introduction time. */
 	uint64_t generation;
 
+	/* Store GFN (if using a ring connection). */
+	xen_pfn_t gfn;
+
 	/* Have we noticed that this domain is shutdown? */
 	bool shutdown;
 
 	/* Has domain been officially introduced? */
 	bool introduced;
 
+	/* Is the ring memory foreign mapped? */
+	bool foreign_mapped;
+
 	/* number of entry from this domain in the store */
 	int nbentry;
 
@@ -196,16 +204,29 @@ static const struct interface_funcs domain_funcs = {
 	.can_read = domain_can_read,
 };
 
-static void *map_interface(domid_t domid)
+static void *map_interface(domid_t domid, xen_pfn_t gfn, bool *foreign_mapped)
 {
-	return xengnttab_map_grant_ref(xgt_handle, domid,
-				       GNTTAB_RESERVED_XENSTORE,
-				       PROT_READ|PROT_WRITE);
+	void *map = xengnttab_map_grant_ref(xgt_handle, domid,
+					    GNTTAB_RESERVED_XENSTORE,
+					    PROT_READ|PROT_WRITE);
+
+	if (!map)
+	{
+		map = xenforeignmemory_map(xfm_handle, domid,
+					   PROT_READ|PROT_WRITE, 1,
+					   &gfn, NULL);
+		*foreign_mapped = !!map;
+	}
+
+	return map;
 }
 
-static void unmap_interface(void *interface)
+static void unmap_interface(void *interface, bool foreign_mapped)
 {
-	xengnttab_unmap(xgt_handle, interface, 1);
+	if (foreign_mapped)
+		xenforeignmemory_unmap(xfm_handle, interface, 1);
+	else
+		xengnttab_unmap(xgt_handle, interface, 1);
 }
 
 static int destroy_domain(void *_domain)
@@ -228,7 +249,8 @@ static int destroy_domain(void *_domain)
 		if (domain->domid == 0)
 			unmap_xenbus(domain->interface);
 		else
-			unmap_interface(domain->interface);
+			unmap_interface(domain->interface,
+					domain->foreign_mapped);
 	}
 
 	fire_watches(NULL, domain, "@releaseDomain", NULL, false, NULL);
@@ -363,12 +385,15 @@ static struct domain *find_or_alloc_domain(const void *ctx, unsigned int domid)
 	return domain ? : alloc_domain(ctx, domid);
 }
 
-static int new_domain(struct domain *domain, int port, bool restore)
+static int new_domain(struct domain *domain, int port, xen_pfn_t gfn,
+		      bool foreign_mapped, bool restore)
 {
 	int rc;
 
 	domain->port = 0;
 	domain->shutdown = false;
+	domain->gfn = gfn;
+	domain->foreign_mapped = foreign_mapped;
 	domain->path = talloc_domain_path(domain, domain->domid);
 	if (!domain->path) {
 		errno = ENOMEM;
@@ -436,7 +461,8 @@ static void domain_conn_reset(struct domain *domain)
 
 static struct domain *introduce_domain(const void *ctx,
 				       unsigned int domid,
-				       evtchn_port_t port, bool restore)
+				       evtchn_port_t port, xen_pfn_t gfn,
+				       bool restore)
 {
 	struct domain *domain;
 	int rc;
@@ -448,17 +474,21 @@ static struct domain *introduce_domain(const void *ctx,
 		return NULL;
 
 	if (!domain->introduced) {
+		bool foreign_mapped = false;
+
 		interface = is_master_domain ? xenbus_map()
-					     : map_interface(domid);
+					     : map_interface(domid, gfn,
+							     &foreign_mapped);
 		if (!interface && !restore)
 			return NULL;
-		if (new_domain(domain, port, restore)) {
+		if (new_domain(domain, port, gfn, foreign_mapped, restore)) {
 			rc = errno;
 			if (interface) {
 				if (is_master_domain)
 					unmap_xenbus(interface);
 				else
-					unmap_interface(interface);
+					unmap_interface(interface,
+							foreign_mapped);
 			}
 			errno = rc;
 			return NULL;
@@ -489,19 +519,20 @@ int do_introduce(struct connection *conn, struct buffered_data *in)
 	char *vec[3];
 	unsigned int domid;
 	evtchn_port_t port;
+	xen_pfn_t gfn;
 
 	if (get_strings(in, vec, ARRAY_SIZE(vec)) < ARRAY_SIZE(vec))
 		return EINVAL;
 
 	domid = atoi(vec[0]);
-	/* Ignore the gfn, we don't need it. */
+	gfn = atol(vec[1]);
 	port = atoi(vec[2]);
 
 	/* Sanity check args. */
 	if (port <= 0)
 		return EINVAL;
 
-	domain = introduce_domain(in, domid, port, false);
+	domain = introduce_domain(in, domid, port, gfn, false);
 	if (!domain)
 		return errno;
 
@@ -718,7 +749,7 @@ void dom0_init(void)
 	if (port == -1)
 		barf_perror("Failed to initialize dom0 port");
 
-	dom0 = introduce_domain(NULL, xenbus_master_domid(), port, false);
+	dom0 = introduce_domain(NULL, xenbus_master_domid(), port, 0, false);
 	if (!dom0)
 		barf_perror("Failed to initialize dom0");
 
@@ -758,6 +789,10 @@ void domain_init(int evtfd)
 	 */
 	xengnttab_set_max_grants(xgt_handle, DOMID_FIRST_RESERVED);
 
+	xfm_handle = xenforeignmemory_open(NULL, 0);
+	if (!xfm_handle)
+		barf_perror("Failed to create handle for foreign memory");
+
 	if (evtfd < 0)
 		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
 	else
@@ -1189,6 +1224,7 @@ const char *dump_state_connections(FILE *fp)
 			sc.spec.ring.tdomid = c->target ? c->target->id
 						: DOMID_INVALID;
 			sc.spec.ring.evtchn = c->domain->port;
+			sc.spec.ring.gfn = c->domain->gfn;
 		} else {
 			sc.conn_type = XS_STATE_CONN_TYPE_SOCKET;
 			sc.spec.socket_fd = c->fd;
@@ -1290,7 +1326,8 @@ void read_state_connection(const void *ctx, const void *state)
 #endif
 	} else {
 		domain = introduce_domain(ctx, sc->spec.ring.domid,
-					  sc->spec.ring.evtchn, true);
+					  sc->spec.ring.evtchn,
+					  sc->spec.ring.gfn, true);
 		if (!domain)
 			barf("domain allocation error");
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:47:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:47:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189511.339296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG5B-0000At-4n; Fri, 17 Sep 2021 15:47:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189511.339296; Fri, 17 Sep 2021 15:47:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG5A-0000Ai-Uo; Fri, 17 Sep 2021 15:47:20 +0000
Received: by outflank-mailman (input) for mailman id 189511;
 Fri, 17 Sep 2021 15:47:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=50Eq=OH=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mRG5A-0006W6-BW
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:47:20 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 855571e5-17ce-11ec-b6db-12813bfff9fa;
 Fri, 17 Sep 2021 15:47:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 855571e5-17ce-11ec-b6db-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631893632;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=DW6Fveyl5rzZoyiIlEPBEWmEFUr6L5PhG/wZX+BGMtQ=;
  b=ANhHhdEboSm26i1YXwUMFiTz7Un3u2E2Ne2ft4bZ9KTu22jhz+GvEtBJ
   4mMGxLuFRxVLdv3Lsj07rQB03hjDRaI67tKRrq98mL97SbOklkQsAx9G9
   nkySXdDgDX6NGzSiYbV337aB+d0iH+Nsmcgjjxc+5fV+maxeF3c2lB0ob
   M=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 84oIj8BE9p55PxfC9ZERa/u8i0MpK7xcXN4/KoTqWPgZmsTzH2mBW1BxVQeOrHqI87ndTyuI9j
 d3k3J3c6sLy9EpXO+VGmF931mzXWup7/HKx6vVPGcb+kA3fBA3hG3zQJMUPAolESXKNWvy3uxW
 iWQX2uu+F0tSukQ+HZO+OVyghhIs/eUsNKGVOE6gAeL9+zczzttgyQD1mpQkZjlUd2J0QrERjL
 dpcqI0GdOH/UTiBF4kmF92imwzUFG/C6qCyBaYiOp4lAoz4yL6N0xNEmn9yE3QTu1KCLJo6sjb
 QggMRAVApZ/KeBGaFuvKHHcA
X-SBRS: 5.1
X-MesageID: 52589463
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:AyHeYq96yGgmKx7KigCjDrUDeXmTJUtcMsCJ2f8bNWPcYEJGY0x3y
 WoXWD+DPP7ZZTOgeo0jPYzioUIF65HTxoQyHVBoryE8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79g3dYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPggz
 991mdu8TjxqI6eWndYXYgBeA3tXaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFhmpg15wQRp4yY
 eIrMBNLcRLKXSRQGWcYKKxhkeeRnSXWJmgwRFW9+vNsvjm7IBZK+LrwNNvYfPSaSMMTmVyXz
 krd5HjwCBweMN2ZyBKG/2iqi+uJmjn0MKoQG6e/7eVCm0CIyyoYDxh+fVmxrOS9i0W+c8lCM
 EFS8S0rxYAi+UruQtTjUhmQpH+fogVaS9dWC/c96gyG1uzT+QnxLmkbTBZRZdo+rsg0SDc2k
 FiTkLvBGjhHoLCTD3WH+d+pQSiaYHZPazVYPGldEFVDs4KLTJwPYgznH9FpKILygtrOWiz6m
 TW4phkngaVJpJtev0mkxmwrkw5At7CQEFVstl6GDzr/hu9qTNX6PN3zsDA3+d4FddzAHwfb5
 BDojuDDtLhmMH2bqMCarAzh9pmS7vCZOXX3hVd1FvHNHBz8pib+Iei8DNx4TXqF0/romxeyO
 yc/WisLvfe/2UdGioctOOqM5zwCl/SIKDgcfqm8giByjn1NmOivp3oGWKJt9zq1zBhEfV8XY
 M/GGSpTMZrqIfs+l2fnLwvs+ZQq2jo/1QvuqWPTlk/8uYdykEW9EO9fWHPXN7hRxPrd/G39r
 oYOX+PXmk43eLCvPUHqHXs7cAliwY4TXsut9aS6t4erf2JbJY3WI6WAme9+I9M1wfs9eyWh1
 ijVZ3K0AWHX3BXvAQ6LdmpiePXoW5N+pmg8JisiIRCj3H1LXGplxP53m0IfceZ1+ep94+RzS
 vVZKcyMDu4WEmbM+igHbIm7p4tnLUz5iQWLNiujQT4+Y58/GFCZpo66JlPipHsUEy66lcoiu
 Ln8hAnVdoUOGlZ5B8HMZfPxk17o5SoBmPh/VlfjK8VIfBm+65BjLiH816dlI8wFJRjZ6CGd0
 gKaXUURqeXX+tdn+9jVn6GU6YyuFrImTEZdGmDa65ewNDXboTX/kdMRDr7QcGmEBm3u+aika
 eFE9N3GMaUKzARQro5xM7d31qZitdHhkKBXk1Z/F3LRYlX1Vr45eiua3dNCv7Fmz6NCvVfkQ
 VqG/9RXNOnbOM7hF1JNdgMpYv7aiKMRkzjWq/80PF/79Gl8+7/eCRdeOByFiSp8KrppMdx6n
 bd96ZBOswHv2AA3NtumjzxP8zXeJ3MNZKwrq5UGDdK5kQEs0FxDPcTRByKeDEtjsDmQ3p3G+
 gOpuZc=
IronPort-HdrOrdr: A9a23:tIL5C6NoCf+r8sBcTyT155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/exoX5PwO080lKQFmrX5WI3NYOCIghrPEGgP1/qB/9SCIVyAygc+79
 YYT0EWMrSZZjIb/KWKnjVQe+xQvOVvm5rY4ts2oU0dKD2DPMpbnnpE40ugYwZLbTgDIaB8OI
 uX58JBqTblUXMLbv6jDn1Ae+TYvdXEmL/vfBZDXnccmUSzpALtzIS/PwmT3x8YXT8K6bA+8V
 Ldmwi8wqm4qfm0xjLVymeWxZVLn9nKzMdFGaW3+4IoAwSprjztSJVqWrWEsjxwiOaz6GwymN
 2JmBskN9Qb0QKmQkiF5T/WnyXw2jcn7HHvjXWCh2H4nMD/TDUmT+JcmINwaHLimgQdleA59J
 gO83OStpJRAx+Ftj/6/cL0WxZjkVfxiWY+kNQUk2dUXeIlGfBsRLQkjQNo+ao7bWHHANhNKp
 gtMCic3ocbTbqiVQGZgoE1q+bcG0jaHX+9Mz8/U4KuonxrdN0Q9Tpr+CUlpAZxyHsKcegx2w
 31CNUYqFhwdL5kUUsEPpZ4fSKWMB2FffueChPbHbzYfJt3Tk4l7aSHp4kI2A==
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="52589463"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cbHMNXuD9bdFr9bQpgF7ljI13L5DFsV6xYz4ohgBe6w4G3/3RshK9kzes0T4dbU7TOOR81Wh+M+o03eCHq8V4LOD6BMVthurBeFVrDLJro822LAGPEzPa7biNvBmpicF1pGDCLdA8OYM90iqZzAQWPTzDRyFnJw0LETd2pgg1mqrDentP/lhjW3+2b1JzXBwOQzUGE8d7KtzGP5MFyoz32bJvTXWYFm95h1X3/deXVBmF0Xxov36aoP0XK5dArhDVlOi8o6jnxFpyY+ezShDdOPcEhYdfeD1gQM3g5nkLSu1fRl1c6gyl62GN0MdwosPLTpsi/Mj80io0gc4+DGo9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LWJyzQpn1vGwUvHb8ce1Tl9dvZuuKifQq5imQ14Lgos=;
 b=Zy3LF5zBDxaQQKlurz2UurqVaVunP24Mwsgwr8QyvPRjrWJ8eQdC/lvzYhpEYHM2T/fxPVNrT8HLt3eWeAJdCKFybX2MNrZ3mmKKo7VHDIdKRR4rJ2I/vYdy8KtzYWwDoZuyl7Oxd9t8x2QM9DxKTQrUb4NiO/BahK0yjz8fTckRhqCrXJBZkQZT8LnT/uSbNVzNljGz1j2Rjy5YlZ3Yft+HMsCKld4EM11QcmP0FX2L8XAUVO370F1ocYsHwJGpRDuFp8w5/OHHV1N0OfmVZYOY7oquE5fWeyiV1cno4mi/oHcxnQpg90Hjz1XiKuAyb9YvB0/I68BfoEG8oxQybg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LWJyzQpn1vGwUvHb8ce1Tl9dvZuuKifQq5imQ14Lgos=;
 b=u96UQu2ojvkNqdr9ZPaQ6UnHPImQGlluenLbNvcO7UqPRZhbO9s3CQAUYBae9U1J9jWJ8/7VaReJLy6zXX00WFlj+0x/dBCffGHODOl33n02IphswhmTNgoZoar7c/1Wz1S0wORe9PJIO56bd6VxA4ka13TdP56E+l8F8kiu/0w=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Julien
 Grall" <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
	"Anthony PERARD" <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>
Subject: [PATCH 6/6] gnttab: allow disabling grant table per-domain
Date: Fri, 17 Sep 2021 17:46:25 +0200
Message-ID: <20210917154625.89315-7-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0008.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:62::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e177f618-8a9f-4544-8694-08d979f2684b
X-MS-TrafficTypeDiagnostic: DM6PR03MB4057:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4057BFDACCF05B24A4157D748FDD9@DM6PR03MB4057.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: JQV6etO/I6MOy5isAT3FfV8BhrT0HdyqHid9PcU39CeiraJ6s+8Tw/WgUJJxgmgEMVRYTu3qJaxOzrWLdopA0VvrYdgapOrfKFVG/BRe1z2T0ZqD75/b8PhgsCJqbCtnTyuuUWDz/794/XeENaF+ie+lxgiTlh5j4+OgzuHa1OK6LmPB/Jbi6mt9qrDNLyjxdDu28qKs0znk5IQaLdY4Qb/YW435Wxa9ZSi9XqOcoDb79ZHXELOF+cKupHTrUNqcxFhw0B03J7WfIqzy1skdqsP5Z4CP6OQgTIeFDyKwbnHrzsqJFovImJXfemVNX8x9Qmn6/4FIW+4xAAT0vP5gjMwgjzuuVERkjHELbswm+wisVBZlaf9jo0kOihgfa5aep6pysnyahhBj6hC7DiCfu9GA1Ww7UASIt4POI+YQcIWEz621rDs6e4KsChqg/AzBXtq9ViNURW32NCD52TgGFAOOTALqj9f4E2WPictqDg07jXU9eOJQ0PwISFruY60V8XrP6elJXm2uTlmwrWAoybvC3uNM86EygvewrzDKt8Pf0OAQ6AdQbxqL4hjzbZKNf47LU2L1wcZDg25vaRYe5VAAi+SWyHCL2KMFsXMO1PftGhScy2KlRNRyX9yOp3QJtZ+o2hl+zFxkwwoFuA+Whw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(1076003)(2616005)(956004)(66946007)(66556008)(66476007)(316002)(6666004)(8936002)(8676002)(2906002)(4326008)(186003)(26005)(6916009)(54906003)(6496006)(508600001)(6486002)(38100700002)(36756003)(83380400001)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?alAzd0hhNzZhcGt6VVA0SkFXbndJT2JMUUZLS2JZcnMrZEI1VHl5dVQrZjRh?=
 =?utf-8?B?cWllSGNXbGUxdU9Wa25qTldDTU9VNXpxV1I2Q0xLL2p4cVNwWWwrOU1xWDJP?=
 =?utf-8?B?TzIxQUxWUUl3ZVBZT0NUQkNZVXhaNG1zekVvemNtZEpnWi85V0E2ZzlaczUy?=
 =?utf-8?B?enpEWitBdXpkZ1ZHaXovREVocWI0b2hjNFprclZXejRrVnU3R00wUDg2dXJ0?=
 =?utf-8?B?ME83NWljWGEveHFFbWdFRVIyNVlnalF1OCtXNk5NNmo5ZEdaTjRDZVU5UXlY?=
 =?utf-8?B?MFdDUDBhZGwwWXV0Q0lBR09mUSsxNlB2dGdsQ1BpSmcvOGhCZ2VoeENaTS85?=
 =?utf-8?B?ZEhnbzNTL3VFZHZlT1dZcVZBT1NqOExvUUNZdGlnYmJPQ09ueGZJOFlVSlgx?=
 =?utf-8?B?YTUxdWpDd0tRaUc0Yk8vK2h1M1FBdVFrTWlPdCtVUlc2VERnTHZHWE9IYTUx?=
 =?utf-8?B?UWZETGFLaWlSWjcvVUZ4QjBsQmFqdWU5c0tFbEZzaHR4WDRVL1E3YkhQTUQw?=
 =?utf-8?B?ZGRkQUxCdE5vaGxGa0RxNk12cWI3d1VyVjQ1SEZ6TGpLNWJUNndOTkswN1Zk?=
 =?utf-8?B?YWRoS1NoMnJOV3hBM1k4RUVCbERxOGtBZHZoV3pRU3FvOXBiZGxrQm9YMlVK?=
 =?utf-8?B?MWlkUm1nVmhSVUxUUVJzTzVWS1BoMFJXTkRIZUM5OWE4ZmUrWTNBTDUyQXF1?=
 =?utf-8?B?S2wwQlRsY3ArcTFIak52ajFJTmxVR0Q0RmRvdDJiQTAvbTRheTB3ckpYcDR6?=
 =?utf-8?B?TVpmUFh3WnQvSmgwUmNRcm9VcGZIbXVDRXF6WlM5NWxvRHFkMi96KzJDWFhH?=
 =?utf-8?B?bnlFRUxoWTV0WnZGVkxDZUhBWEJoeXg5YTRsWGptbDJRd1R4bHBFbnlrNkEy?=
 =?utf-8?B?ZEdzTDRWZk1QRE8yQ1pIVDRVYTF6c1pja0FyaTJ1Z0xWMlhDRzZncmJoeGMz?=
 =?utf-8?B?aDA0UXN3VkJsTVoyWHVveGpnempOSkZNODRsMkhqWlZhM1NtNy8xOUdtYUJz?=
 =?utf-8?B?M0U4eWtwOERPRFVaUEJPcjA2bkc1YkdyWVRZRDB3TWdqeU5uZzRFOFhoTHVP?=
 =?utf-8?B?c1o2am8vamdudEJRNFdMNmNhbzRaQ2hBN1BjWUtHK3pCWjBteWs5NHE0V3U0?=
 =?utf-8?B?RVBOTzBja2pXZDlhZXgxbk5INjBJS2MrSzZmNTVQblJKV3pWYURpbHllYUtV?=
 =?utf-8?B?dkdTRGlXOHRJYVdmd1dSQXRkVFl0T21QVUFaWkdLTlB3OEM1Q3p3dnVud0NX?=
 =?utf-8?B?WDFYRDBtdkNONGx3YmtwamtlTmpSUjRQWjFHcXUwMnRVUFJTNk1ydlV4M2x2?=
 =?utf-8?B?M1hHWXIvVUo5UERBYm92eWhUYnRsdUd2RnJaYTJhS1pWb1gwYlNjMzB2Vzl4?=
 =?utf-8?B?dGJ1WVZkWi9xVnhTMEUvTnBFQ3FxRS8vUXduQ0RIYjlMcDkwZlI4Nmd0TDFi?=
 =?utf-8?B?Z3NNTDFqelVzelFsVk5hSVZ4d0tGTjMvTTVUYSt1K2NWTVN6RnZsOU90NDNO?=
 =?utf-8?B?TXdTOEtmcHoxdmZJNU5rZTVseWJmQkN5N2Q3c0w3MzRqSkJ5bXFLM2EvbHBQ?=
 =?utf-8?B?WTlKZitTSStpdlBmTkV4ZFBjVE9VYkxxM2JwYllvYjNtaGo2VmxCUWN0ZWh4?=
 =?utf-8?B?ZVRWbmhNMFR4YjBwaWdIeE5oQzc1OVVxRGsvZHprN3JHb3BiM0lieEE1OHdQ?=
 =?utf-8?B?cFJUeUliOW9Na3BsQUZlbFdNaW9SODZtY25QSmNQRzZOOXJRTHNJS3U2TVND?=
 =?utf-8?Q?0c1xtUdZcgXdYLe5P6YoJFV8AFmREIPjqY1SqrW?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e177f618-8a9f-4544-8694-08d979f2684b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 15:47:08.9487
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 37Vqbtn6OtKyj2TqE8qTmxmQ50r6KUOSkz9fs0wejEyNOEKr4u/V6IjF3SewjM5s7HU1gOsAy52AyvyIDLKiBA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4057
X-OriginatorOrg: citrix.com

Allow setting max_grant_version to 0 in order to disable grant table
usage by a domain. This prevents allocating the grant-table structure
inside of Xen and requires guards to be added in several functions in
order to prevent dereferencing the structure.

Note that a domain without a grant table could still use some of the
grant related hypercalls, it could for example issue a GNTTABOP_copy
of a grant reference from a remote domain into a local frame.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 docs/man/xl.cfg.5.pod.in     |  4 +-
 tools/libs/light/libxl_dom.c |  2 +-
 xen/common/grant_table.c     | 97 ++++++++++++++++++++++++++++++++++--
 3 files changed, 95 insertions(+), 8 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index c5a447dfcd..d507540c2c 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -583,8 +583,8 @@ L<xl.conf(5)>.
 =item B<max_grant_version=NUMBER>
 
 Specify the maximum grant table version the domain is allowed to use. Current
-supported versions are 1 and 2. The default value is settable via
-L<xl.conf(5)>.
+supported versions are 1 and 2. Setting to 0 disables the grant table for the
+domain. The default value is settable via L<xl.conf(5)>.
 
 =item B<transitive_grants=BOOLEAN>
 
diff --git a/tools/libs/light/libxl_dom.c b/tools/libs/light/libxl_dom.c
index e9f58ee4b2..afc8b88497 100644
--- a/tools/libs/light/libxl_dom.c
+++ b/tools/libs/light/libxl_dom.c
@@ -598,7 +598,7 @@ static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "xc_dom_boot_image failed");
         goto out;
     }
-    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
+    if ( info->max_grant_version && (ret = xc_dom_gnttab_init(dom)) != 0 ) {
         LOGE(ERROR, "xc_dom_gnttab_init failed");
         goto out;
     }
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 280dbc850a..68ea742498 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1027,6 +1027,12 @@ map_grant_ref(
     }
 
     lgt = ld->grant_table;
+    if ( !lgt )
+    {
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", ld);
+        op->status = GNTST_bad_domain;
+        return;
+    }
     handle = get_maptrack_handle(lgt);
     if ( unlikely(handle == INVALID_MAPTRACK_HANDLE) )
     {
@@ -1037,6 +1043,14 @@ map_grant_ref(
     }
 
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        put_maptrack_handle(lgt, handle);
+        rcu_unlock_domain(rd);
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
     grant_read_lock(rgt);
 
     /* Bounds check on the grant ref */
@@ -1367,6 +1381,13 @@ unmap_common(
     ld = current->domain;
     lgt = ld->grant_table;
 
+    if ( !lgt )
+    {
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", ld);
+        op->status = GNTST_bad_domain;
+        return;
+    }
+
     if ( unlikely(op->handle >= lgt->maptrack_limit) )
     {
         gdprintk(XENLOG_INFO, "Bad d%d handle %#x\n",
@@ -1406,6 +1427,13 @@ unmap_common(
     TRACE_1D(TRC_MEM_PAGE_GRANT_UNMAP, dom);
 
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        rcu_unlock_domain(rd);
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
 
     grant_read_lock(rgt);
 
@@ -1556,6 +1584,12 @@ unmap_common_complete(struct gnttab_unmap_common *op)
 
     rcu_lock_domain(rd);
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        rcu_unlock_domain(rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
 
     grant_read_lock(rgt);
 
@@ -1931,10 +1965,7 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     if ( max_grant_version < 0 )
         max_grant_version = opt_gnttab_max_version;
     if ( !max_grant_version )
-    {
-        dprintk(XENLOG_INFO, "Invalid grant table version 0 requested\n");
-        return -EINVAL;
-    }
+        return 0;
     if ( max_grant_version > opt_gnttab_max_version )
     {
         dprintk(XENLOG_INFO,
@@ -2056,6 +2087,11 @@ gnttab_setup_table(
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out;
+    }
     grant_write_lock(gt);
 
     if ( unlikely(op.nr_frames > gt->max_grant_frames) )
@@ -2138,6 +2174,11 @@ gnttab_query_size(
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out;
+    }
 
     grant_read_lock(gt);
 
@@ -2302,6 +2343,13 @@ gnttab_transfer(
             goto put_gfn_and_copyback;
         }
 
+        if ( unlikely(!e->grant_table) )
+        {
+            gdprintk(XENLOG_INFO, "%pd has no grant table\n", e);
+            gop.status = GNTST_bad_domain;
+            goto unlock_and_copyback;
+        }
+
         if ( xsm_grant_transfer(XSM_HOOK, d, e) )
         {
             gop.status = GNTST_permission_denied;
@@ -2888,6 +2936,12 @@ static int gnttab_copy_claim_buf(const struct gnttab_copy *op,
 
     if ( op->flags & gref_flag )
     {
+        if ( !buf->domain->grant_table )
+        {
+            rc = GNTST_bad_domain;
+            goto out;
+        }
+
         rc = acquire_grant_for_copy(buf->domain, ptr->u.ref,
                                     current->domain->domain_id,
                                     buf->read_only,
@@ -3092,6 +3146,9 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop)
     int res;
     unsigned int i, nr_ents;
 
+    if ( !gt )
+        return -ENODEV;
+
     if ( copy_from_guest(&op, uop, 1) )
         return -EFAULT;
 
@@ -3270,6 +3327,11 @@ gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop,
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out2;
+    }
 
     op.status = GNTST_okay;
 
@@ -3332,7 +3394,11 @@ gnttab_get_version(XEN_GUEST_HANDLE_PARAM(gnttab_get_version_t) uop)
         return rc;
     }
 
-    op.version = d->grant_table->gt_version;
+    if ( d->grant_table )
+        op.version = d->grant_table->gt_version;
+    else
+        /* Use 0 to signal no grant table. */
+        op.version = 0;
 
     rcu_unlock_domain(d);
 
@@ -3351,6 +3417,12 @@ swap_grant_ref(grant_ref_t ref_a, grant_ref_t ref_b)
     struct active_grant_entry *act_b = NULL;
     s16 rc = GNTST_okay;
 
+    if ( !gt )
+    {
+        rcu_unlock_domain(d);
+        return GNTST_bad_domain;
+    }
+
     grant_write_lock(gt);
 
     /* Bounds check on the grant refs */
@@ -3872,6 +3944,9 @@ void grant_table_warn_active_grants(struct domain *d)
 
 #define WARN_GRANT_MAX 10
 
+    if ( !gt )
+        return;
+
     grant_read_lock(gt);
 
     nr_ents = nr_grant_entries(gt);
@@ -3953,6 +4028,9 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,
     int rc = 0;
     uint16_t flags = 0;
 
+    if ( !gt )
+        return -ENODEV;
+
     grant_read_lock(gt);
 
     if ( gt->gt_version < 1 )
@@ -4069,6 +4147,9 @@ unsigned int gnttab_resource_max_frames(const struct domain *d, unsigned int id)
     const struct grant_table *gt = d->grant_table;
     unsigned int nr = 0;
 
+    if ( !gt )
+        return 0;
+
     /* Don't need the grant lock.  This limit is fixed at domain create time. */
     switch ( id )
     {
@@ -4100,6 +4181,9 @@ int gnttab_acquire_resource(
     if ( !nr_frames )
         return rc;
 
+    if ( !gt )
+        return -ENODEV;
+
     final_frame = frame + nr_frames - 1;
 
     /* Grow table if necessary. */
@@ -4156,6 +4240,9 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn)
     struct grant_table *gt = d->grant_table;
     bool status = false;
 
+    if ( !gt )
+        return -ENODEV;
+
     grant_write_lock(gt);
 
     if ( evaluate_nospec(gt->gt_version == 2) && (idx & XENMAPIDX_grant_table_status) )
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:48:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:48:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189533.339307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG6N-0001tC-Fa; Fri, 17 Sep 2021 15:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189533.339307; Fri, 17 Sep 2021 15:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRG6N-0001t5-Bs; Fri, 17 Sep 2021 15:48:35 +0000
Received: by outflank-mailman (input) for mailman id 189533;
 Fri, 17 Sep 2021 15:48:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRG6M-0001sp-GV
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:48:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRG6K-0001MC-GG; Fri, 17 Sep 2021 15:48:32 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRG6K-00038b-5D; Fri, 17 Sep 2021 15:48:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=LkFviuD6UU0xBDGYDHkI6su/g59hRYCUtjNw8n/rYI4=; b=NK4DoEUFFlrcbrnovahbjJMx+W
	+Vvas9Bua0zXLOk26zou8YkMBImncvVcwGfwl9soUiUv9gZgHhKz5q/UBUuHq4kw+0hr66fgScEBH
	HWIU4y8ykUTKeiKApqenMjg2fkKyFMLrGlYjsIDwso4AnzLlXqYiA8AAkliv5WvuMLFc=;
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Oleksandr Tyshchenko <olekstysh@gmail.com>, xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
Date: Fri, 17 Sep 2021 20:48:23 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 10/09/2021 23:18, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain.
> - The ACPI case is not covered.

I understand the ACPI is not covered because we would need to create a 
new binding. But I am not sure to understand why 32-bit domain is not 
supported. Can you explain it?

> 
> ***
> 
> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> the algorithm to choose extended regions for it is different
> in comparison with the algorithm for non-direct mapped DomU.
> What is more, that extended regions should be chosen differently
> whether IOMMU is enabled or not.
> 
> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> holes found in host device-tree if otherwise. 

For the case when the IOMMU is disabled, this will only work if dom0 
cannot allocate memory outside of the original range. This is currently 
the case... but I think this should be spelled out in at least the 
commit message.

> Make sure that
> extended regions are 2MB-aligned and located within maximum possible
> addressable physical memory range. The maximum number of extended
> regions is 128.

Please explain how this limit was chosen.

> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> Changes since RFC:
>     - update patch description
>     - drop uneeded "extended-region" DT property
> ---
> 
>   xen/arch/arm/domain_build.c | 226 +++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 224 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 206038d..070ec27 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct domain *d,
>       return res;
>   }
>   
> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
> +{
> +    struct meminfo *ext_regions = data;
> +    paddr_t start, size;
> +
> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> +        return 0;
> +
> +    /* Both start and size of the extended region should be 2MB aligned */
> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> +    if ( start > e )
> +        return 0;
> +
> +    size = (e - start + 1) & ~(SZ_2M - 1);
> +    if ( !size )
> +        return 0;
> +
> +    ext_regions->bank[ext_regions->nr_banks].start = start;
> +    ext_regions->bank[ext_regions->nr_banks].size = size;
> +    ext_regions->nr_banks ++;
> +
> +    return 0;
> +}
> +
> +/*
> + * The extended regions will be prevalidated by the memory hotplug path
> + * in Linux which requires for any added address range to be within maximum
> + * possible addressable physical memory range for which the linear mapping
> + * could be created.
> + * For 48-bit VA space size the maximum addressable range are:

When I read "maximum", I understand an upper limit. But below, you are 
providing a range. So should you drop "maximum"?

Also, this is tailored to Linux using 48-bit VA. How about other limits?

> + * 0x40000000 - 0x80003fffffff
> + */
> +#define EXT_REGION_START   0x40000000ULL

I am probably missing something here.... There are platform out there 
with memory starting at 0 (IIRC ZynqMP is one example). So wouldn't this 
potentially rule out the extended region on such platform?

> +#define EXT_REGION_END     0x80003fffffffULL
> +
> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
> +                                          struct meminfo *ext_regions)
> +{
> +    const struct meminfo *assign_mem = &kinfo->mem;
> +    struct rangeset *unalloc_mem;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;

We technically already know which range of memory is unused. This is 
pretty much any region in the freelist of the page allocator. So how 
about walking the freelist instead?

The advantage is we don't need to worry about modifying the function 
when adding new memory type.

One disavantage is this will not cover *all* the unused memory as this 
is doing. But I think this is an acceptable downside.

> +
> +    dt_dprintk("Find unallocated memory for extended regions\n");
> +
> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> +    if ( !unalloc_mem )
> +        return -ENOMEM;
> +
> +    /* Start with all available RAM */
> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.mem.bank[i].start;
> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
> +        res = rangeset_add_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove RAM assigned to Dom0 */
> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> +    {
> +        start = assign_mem->bank[i].start;
> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
> +        res = rangeset_remove_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove reserved-memory regions */
> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.reserved_mem.bank[i].start;
> +        end = bootinfo.reserved_mem.bank[i].start +
> +            bootinfo.reserved_mem.bank[i].size - 1;
> +        res = rangeset_remove_range(unalloc_mem, start, end);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove grant table region */
> +    start = kinfo->gnttab_start;
> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> +    res = rangeset_remove_range(unalloc_mem, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_report_ranges(unalloc_mem, start, end,
> +                                 add_ext_regions, ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(unalloc_mem);
> +
> +    return res;
> +}
> +
> +static int __init find_memory_holes(const struct kernel_info *kinfo,
> +                                    struct meminfo *ext_regions)
> +{
> +    struct dt_device_node *np;
> +    struct rangeset *mem_holes;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find memory holes for extended regions\n");
> +
> +    mem_holes = rangeset_new(NULL, NULL, 0);
> +    if ( !mem_holes )
> +        return -ENOMEM;
> +
> +    /* Start with maximum possible addressable physical memory range */
> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_add_range(mem_holes, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */

Well... The loop below is not going to handle all the regions described 
in the property "reg". Instead, it will cover a subset of "reg" where 
the memory is addressable.

You will also need to cover "ranges" that will describe the BARs for the 
PCI devices.

> +    dt_for_each_device_node( dt_host, np )
> +    {
> +        unsigned int naddr;
> +        u64 addr, size;
> +
> +        naddr = dt_number_of_address(np);
> +
> +        for ( i = 0; i < naddr; i++ )
> +        {
> +            res = dt_device_get_address(np, i, &addr, &size);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> +                       i, dt_node_full_name(np));
> +                goto out;
> +            }
> +
> +            start = addr & PAGE_MASK;
> +            end = PAGE_ALIGN(addr + size) - 1;
> +            res = rangeset_remove_range(mem_holes, start, end);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                       start, end);
> +                goto out;
> +            }
> +        }
> +    }
> +
> +    start = EXT_REGION_START;
> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> +    res = rangeset_report_ranges(mem_holes, start, end,
> +                                 add_ext_regions,  ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(mem_holes);
> +
> +    return res;
> +}
> +
>   static int __init make_hypervisor_node(struct domain *d,
>                                          const struct kernel_info *kinfo,
>                                          int addrcells, int sizecells)
> @@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct domain *d,
>       const char compat[] =
>           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>           "xen,xen";
> -    __be32 reg[4];
> +    __be32 reg[(NR_MEM_BANKS + 1) * 4];

This is a fairly large allocation on the stack. Could we move to a 
dynamic allocation?

>       gic_interrupt_t intr;
>       __be32 *cells;
>       int res;
>       void *fdt = kinfo->fdt;
> +    struct meminfo *ext_regions;
> +    unsigned int i;
>   
>       dt_dprintk("Create hypervisor node\n");
>   
> @@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct domain *d,
>       if ( res )
>           return res;
>   
> +    ext_regions = xzalloc(struct meminfo);
> +    if ( !ext_regions )
> +        return -ENOMEM;
> +
> +    if ( is_32bit_domain(d) )
> +        printk(XENLOG_WARNING "The extended region is only supported for 64-bit guest\n");
> +    else
> +    {
> +        if ( !is_iommu_enabled(d) )
> +            res = find_unallocated_memory(kinfo, ext_regions);
> +        else
> +            res = find_memory_holes(kinfo, ext_regions);
> +
> +        if ( res )
> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> +    }
> +
>       /* reg 0 is grant table space */
>       cells = &reg[0];
>       dt_child_set_range(&cells, addrcells, sizecells,
>                          kinfo->gnttab_start, kinfo->gnttab_size);
> +    /* reg 1...N are extended regions */
> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
> +    {
> +        u64 start = ext_regions->bank[i].start;
> +        u64 size = ext_regions->bank[i].size;
> +
> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
> +                   i, start, start + size);
> +
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> +    }
> +    xfree(ext_regions);
> +
>       res = fdt_property(fdt, "reg", reg,
> -                       dt_cells_to_size(addrcells + sizecells));
> +                       dt_cells_to_size(addrcells + sizecells) * (i + 1));
>       if ( res )
>           return res;
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 15:53:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 15:53:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189541.339318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGAh-0003Th-5h; Fri, 17 Sep 2021 15:53:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189541.339318; Fri, 17 Sep 2021 15:53:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGAh-0003Ta-1f; Fri, 17 Sep 2021 15:53:03 +0000
Received: by outflank-mailman (input) for mailman id 189541;
 Fri, 17 Sep 2021 15:53:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRGAf-0003TU-KF
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 15:53:01 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRGAd-0001Qu-L7; Fri, 17 Sep 2021 15:52:59 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRGAd-0003Sx-Ah; Fri, 17 Sep 2021 15:52:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=0xA5ARbzT7zqy34RdeBkN5rAk75b7CzxrS79b8ftDgo=; b=PRNtL20ZUluTHgwrsQQ11JAvGa
	nGzq1hfgmbYmsGmcFZk4uZbYFOkMx3Wm2T30Wl8cznV19oSiYnALRCyCnNbAxo/KUt/hnzIcbuZkC
	oL6yTyu538eiVTZIkyjmsAX51Pj45aObMfT0y78g+VaCzsEPf3dSJ34mBJPXs+ybB9io=;
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Oleksandr <olekstysh@gmail.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
 <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <e832477f-4ed7-4f12-90aa-ebc1843cdc36@xen.org>
Date: Fri, 17 Sep 2021 20:52:52 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 16/09/2021 00:10, Oleksandr wrote:
>>> + * The extended regions will be prevalidated by the memory hotplug path
>>> + * in Linux which requires for any added address range to be within 
>>> maximum
>>> + * possible addressable physical memory range for which the linear 
>>> mapping
>>> + * could be created.
>>> + * For 48-bit VA space size the maximum addressable range are:
>>> + * 0x40000000 - 0x80003fffffff
>> Please don't make Linux-specific comments in Xen code for interfaces
>> that are supposed to be OS-agnostic.
> 
> You are right. I just wanted to describe where these magic numbers come 
> from.
> Someone might question why, for example, "0 ... max_gpaddr" can't be 
> used. I will move
> that Linux-specific comments to the commit message to keep some 
> justification of these numbers.

Please keep some rationale in the code. This is a lot easier to 
understand the code without having to play the git blame game.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:01:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:01:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189567.339335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGJ3-0005ki-6K; Fri, 17 Sep 2021 16:01:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189567.339335; Fri, 17 Sep 2021 16:01:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGJ3-0005kb-1L; Fri, 17 Sep 2021 16:01:41 +0000
Received: by outflank-mailman (input) for mailman id 189567;
 Fri, 17 Sep 2021 16:01:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRGJ1-0005kV-Q8
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 16:01:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRGIy-00029O-IQ; Fri, 17 Sep 2021 16:01:36 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRGIy-0004VB-6L; Fri, 17 Sep 2021 16:01:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=4EIFlC7bSP228SxUsP6EbSZoZ3lV7hhvf99Td8CaC3w=; b=qCIM68wzeRUtZz9X+rSitI7QTJ
	5E2qc6BISOOm3QsKy9YynIG8kXqu/4AaBFTR/YpgFh1nPgTJWCiYni567Wcu2BuBP6pzeaFMx93q3
	k/u5KgWVmyJRSq4ojOHRbrJewhTaamdF/8B8SF8TS96ayGEZYvQrxZu5HaoYkRRfzuWw=;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org,
 Dmitry Isaikin <isaikin-dmitry@yandex.ru>, rjstone@amazon.co.uk,
 raphning@amazon.co.uk, Paul Durrant <paul@xen.org>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
 <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
 <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <f6225dc6-0590-3456-8c48-7ab29aa00200@xen.org>
Date: Fri, 17 Sep 2021 21:01:26 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Jan,

On 17/09/2021 14:47, Jan Beulich wrote:
> On 17.09.2021 11:41, Andrew Cooper wrote:
>> On 17/09/2021 10:27, Julien Grall wrote:
>>> Hi,
>>>
>>> (+ some AWS folks)
>>>
>>> On 17/09/2021 11:17, Jan Beulich wrote:
>>>> On 16.09.2021 19:52, Andrew Cooper wrote:
>>>>> On 16/09/2021 13:30, Jan Beulich wrote:
>>>>>> On 16.09.2021 13:10, Dmitry Isaikin wrote:
>>>>>>> From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
>>>>>>>
>>>>>>> This significantly speeds up concurrent destruction of multiple
>>>>>>> domains on x86.
>>>>>> This effectively is a simplistic revert of 228ab9992ffb ("domctl:
>>>>>> improve locking during domain destruction"). There it was found to
>>>>>> actually improve things;
>>>>>
>>>>> Was it?  I recall that it was simply an expectation that performance
>>>>> would be better...
>>>>
>>>> My recollection is that it was, for one of our customers.
>>>>
>>>>> Amazon previously identified 228ab9992ffb as a massive perf hit, too.
>>>>
>>>> Interesting. I don't recall any mail to that effect.
>>>
>>> Here we go:
>>>
>>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fxen-devel%2Fde46590ad566d9be55b26eaca0bc4dc7fbbada59.1585063311.git.hongyxia%40amazon.com%2F&amp;data=04%7C01%7CAndrew.Cooper3%40citrix.com%7C8cf65b3fb3324abe7cf108d979bd7171%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637674676843910175%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=si7eYIxSqsJY77sWuwsad5MzJDMzGF%2F8L0JxGrWTmtI%3D&amp;reserved=0
>>>
>>>
>>> We have been using the revert for quite a while in production and didn't
>>> notice any regression.
>>>
>>>>
>>>>> Clearly some of the reasoning behind 228ab9992ffb was flawed and/or
>>>>> incomplete, and it appears as if it wasn't necessarily a wise move in
>>>>> hindsight.
>>>>
>>>> Possible; I continue to think though that the present observation wants
>>>> properly understanding instead of more or less blindly undoing that
>>>> change.
>>>
>>> To be honest, I think this is the other way around. You wrote and merged
>>> a patch with the following justification:
>>>
>>> "
>>>      There is no need to hold the global domctl lock across domain_kill() -
>>>      the domain lock is fully sufficient here, and parallel cleanup after
>>>      multiple domains performs quite a bit better this way.
>>> "
>>>
>>> Clearly, the original commit message is lacking details on the exact
>>> setups and numbers. But we now have two stakeholders with proof that
>>> your patch is harmful to the setup you claim perform better with your
>>> patch.
>>>
>>> To me this is enough justification to revert the original patch. Anyone
>>> against the revert, should provide clear details of why the patch should
>>> not be reverted.
>>
>> I second a revert.
>>
>> I was concerned at the time that the claim was unsubstantiated, and now
>> there is plenty of evidence to counter the claim.
> 
> Well, I won't object to a proper revert. I still think we'd better get to
> the bottom of this, not the least because I thought there was agreement
> that mid to long term we should get rid of global locking wherever
> possible. Or are both of you saying that using a global lock here is
> obviously fine? And does either of you have at least a theory to explain
> the observation? I can only say that I find it puzzling.

I will quote what Hongyan wrote back on the first report:

"
The best solution is to make the heap scalable instead of a global
lock, but that is not going to be trivial.

Of course, another solution is to keep the domctl lock dropped in
domain_kill() but have another domain_kill lock so that competing
domain_kill()s will try to take that lock and back off with hypercall
continuation. But this is kind of hacky (we introduce a lock to reduce
spinlock contention elsewhere), which is probably not a solution but a
workaround.
"

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:06:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:06:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189575.339346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGO0-0006Po-PK; Fri, 17 Sep 2021 16:06:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189575.339346; Fri, 17 Sep 2021 16:06:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGO0-0006Ph-MF; Fri, 17 Sep 2021 16:06:48 +0000
Received: by outflank-mailman (input) for mailman id 189575;
 Fri, 17 Sep 2021 16:06:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hVMF=OH=citrix.com=christian.lindig@srs-us1.protection.inumbo.net>)
 id 1mRGNz-0006PZ-7j
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 16:06:47 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 67bbcb14-6e30-46bb-901e-eaff7323f5de;
 Fri, 17 Sep 2021 16:06:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67bbcb14-6e30-46bb-901e-eaff7323f5de
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1631894805;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-id:content-transfer-encoding:
   mime-version;
  bh=fWXZivuGbdRiQGIsQOsFfvQX9cFEApvTxe0sr16//bk=;
  b=RvbUWyHmnI3ldN9aMHC/Cic4DA63RytuXuAbB24wUIDgDW1Epn18kWXh
   OkWRjBRt2FlpI1EXG1XIJxvTkuPQvLy9pIDYjIyY5HSgNvmvDRIi1m6Vk
   uP0Mbc6PzxTsNsQB1Lqrs909rJWLJPgzB4iCk11oEwuBqOgh4jtoakC57
   Q=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 8Aru0YOHM1sgvWspVA6KcXb8k4nxprKy+bLVQqNHZ7OhVXkV4n1akSSGM5IeMAfoquJzJCPdJi
 OeWPJKIO4AJ8xafI6oENVKuFZMTQlpMSOFxCMgPNddrU0ujWenv/N/S57rH+8TCFuT7UrS3Cav
 S8yGXtp0D32CpjtfWGrRWxWuvzAb4tcdYureaEdLirW2vlLtQHNctxSslaoV1cP5cdAb6JZvZV
 kQObcr/cMOqQp5d8Ls32dz3kR7HO/Xqx81lx0xOc4HzgAe7LM2ptEsf3OsFOEqznglwusoJhMW
 L76A/XLWFYr0kkV9Fo+FnT3Y
X-SBRS: 5.1
X-MesageID: 53412760
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:PihOh6/IKyyJ+fBuEQMvDrUDNXmTJUtcMsCJ2f8bNWPcYEJGY0x3n
 2ofXmiDPfyPNjH3Kdojb9u/8kMH6sXQnNUxQVM5rik8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79g3dYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhVl
 +x1kK2ZTTwFYIHFtu8XeEF5GjlhaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguwoLNPqJ58es3cmyTDfAfcpaZvCX7/L9ZlT2zJYasVmQK2CP
 pBGN2IHgBLocg9DOHgrI6gHpsSP3VLyQhxEh26unP9ii4TU5FMoi+W8WDbPQfSISt9ShV2wv
 X/d8iLyBRRyHMeE1TOP/3aoh+nOtSD2QoQfEPu/7PECqGOUwmsfGRgHT2yRqPOyilO9c99HI
 klS8S0rxYA+/VKmZsPwVBq5pDiDpBF0c9hNF+w37imdx6yS5ByWbkAFUTxpeNEgrNUxRzEhy
 hmOhdyBLSxitviZRGyQ8p+QrCiuIm4FIGkafygGQAAZpd75r+kbjBjCU9JiG66dlcDuFHf7x
 DXihDc6r6Uei4gMzarTwLzcq2vy/N6TFFdzv1iJGDL+hu9kWGK7T6j30GD+3N9EF6rHYlCvt
 3Ivw+mX9P9bWPlhixexrPUx8KCBvqjeaW2H0A83RfHN5Bz2pCXyJtk4DCVWYR4za5daI2eBj
 Fr75FsJjKK/KkdGekOej2iZMM0s0aGoPtDsTPm8gjFmM8UpKVPvEM2DYyetM4HRfKoEyvpX1
 XSzK5/E4ZMm5UNPlmHeegvl+eV3rh3SPEuKLXwB8/hC7VZ5TCXOIYrpzXPUNrxphE96iFyNr
 r6zyPdmOz0ACbajM0E7AKY4LEwQLGhTOHwFg5UMLYa+zv5dMDh5UZf5mOp5E6Q8xvg9vrqYr
 xmVBx4DoHKi1CKvFOl/Qi06AF8Zdc0k9ixT0O1FFQvA5kXPlq71vf9DKMNoLed4nAGhpNYtJ
 8Q4lwy7Kq0nYhzM+igHbIm7q4pndR+xghmJMTbjaz86F6OMjSSQkjM9VgewpiQIEAStss4y/
 ++p2g/BGMJRTAV+FsfGLvmoygrp73QanetzWWrOI8VSJxqwoNQ7dXSpg69lOdwIJDXC2iCei
 1ScDyAHqLSfuIQy6tTI2/yJ9t/7D+tkE0NGNGDH9rLqZzLC92+uzNYYAuaFdDzQTk3u/6Cma
 bkHxv3wKqRfzl1Lr5B9A/Bgyqdnv4njoLpTzwJFGnTXbgv0VuM8cyfehcQW7/9D3L5UvweyS
 3mjwNgCNOXbIt7hHX4QOBEhMraJ28YLl2SA9v8yOkj7unN6peLVTUVIMhCQoyVBN78pYpg9y
 OIstcNKuQyyjh0mboSPgixOrjneK3UBV+Mst40AAZ+tgQ0ukwkQbZvZAy7wwZeOd9QTbRV6f
 m7K3PLP1+ZG207PU3svDnycj+NSiKMHtA1O0FJfdU+CncDIh6Nv0RBcmdjtot+5EvmTPzpPB
 1VW
IronPort-HdrOrdr: A9a23:m8rsgK39kY/JK9d69SGBwgqjBehxeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5AEtQ5OxpOMG7MBbhHQYc2/hRAV7QZnibhILOFvAj0WKC+UyvJ8SazIBgPM
 hbAtFD4bHLfDtHZIPBkXOF+rUbsZq6GcKT9J/jJh5WJGkAAcAB0+46MHfhLqQffngaOXNTLu
 v52iMznUvHRZ1hVLXdOpBqZZmgm/T70LbdJTIWDR8u7weDyRmy7qThLhSe1hACFxtS3LYL6w
 H+4k7Ez5Tml8v+5g7X1mfV4ZgTssDm0MF/CMuFjdVQAinwizyveJ9qV9S5zXUISaCUmRIXee
 v30lEd1vdImirsl6aO0EPQMjzboXETArnZuASlaDXY0JbErXkBerV8bMpiA2XkAgwbzY1BOe
 twrhKknosSAhXakCvn4d/UExlsi0qvuHIn1fUelnpFTOIlGfJsRRx2xjIlLH4sJlOw1GkcKp
 glMCgc3ocgTXqKK3TC+mV/yt2lWXo+Wh+AX0gZo8SQlzxbhmpwwUcUzNEW2i5ozuNzd7BUo+
 Dfdqh4nrBHScEbKap7GecaWMOyTmjAWwjFPm6eKUnuUKsHJ3XOoZjq56hd3pDnRLUYiJ8p3J
 jRWlJRsmA/P0roFM2VxZVOtgvARW2sNA6dgv22J6IJ84EUYYCbRRFrZGpe4fdIks9vdPEzAc
 zDS66+K8WTWFfTJQ==
X-IronPort-AV: E=Sophos;i="5.85,301,1624334400"; 
   d="scan'208";a="53412760"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W54Oar4enRhG4uHsMecR6bTiz8BZFTfMt1BXeecozNQHQZrBFPZ/9mcmLpr0vAb+6vdDj2Y3pYSnoXZTjG+hvmhHW+E4FB42yogE+4X0jeSBoDBk11KhaEXPGvOK/7zASUk1+8Cm0jHW4L0Qk0bXYWTL8MdJ8dqUfSnnNk9ERp7q7jMdH3WfvYjGgBsP4NgBxr75r3jZPHOfP3p83CU70usOl4LSaVSSviTxLmtu59smaWH4eJctz8yk8FL38yWyNgx9c48oXQ1bapsC52R3u6g3n19ykn+7viOC06Bx8TsKZVjKjb8ruPwdHfe8B5pSk5VT1JUAREgu0xQl43lrtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=v3F8QEn72i58rKNTvmlNs6XOnoIn95bdmjUqwCN7gms=;
 b=a4jo42jBxy9YTwEYtPowyv2zUmNDW0/zOIBVV50a+Bi0uvyqqlmNuSswkozkfAkb2KnbIKulr1YzA4wHuTZb+4H0JJPdPRatn0OO4vQN+48w5hEoQZz+P8f5iGSslxAOU3S46vdsdJGK+veqwQBB3qmUBHmL/VMO9zB5HSTE6P8AlH1FtLTJCN4n3oE16+U28smxo9l3Qrz4VOpVqGAWUd6r0edjJHoIcYchPI3B5hSvIBQ764N/29rkmwFOSY4aSHVCj22TeUNRbR5DFEKoBY/jCpj3oNQpi/Ki2Ju5pDlrAwRiOJwcxcGEpi2hZH5K65xXqQA/QAL/ZQX7ADQQ5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v3F8QEn72i58rKNTvmlNs6XOnoIn95bdmjUqwCN7gms=;
 b=ZbeLEwUsS6fWuD3mcqYnKBkyYQJuwIDEyw8c+DspizKVZUBvWqc5rxLuO8WA9/ngT2/0h5Awd6xnKQJtDuIZ8qQBwJfzoU5sz8tLTQaJsxgsJyUb7kFtgaf+UvDbH/0+ACH10/XeyfnClGpiN2VVn+wXm/TqL3WQJjw2FZOmrYM=
From: Christian Lindig <christian.lindig@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>, Edwin Torok
	<edvin.torok@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Ian
 Jackson" <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<Andrew.Cooper3@citrix.com>, George Dunlap <George.Dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony Perard
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 0/6] gnttab: add per-domain controls
Thread-Topic: [PATCH 0/6] gnttab: add per-domain controls
Thread-Index: AQHXq9s0IM3OAQqHo06+ALWIm01YaauoZEqA
Date: Fri, 17 Sep 2021 16:06:42 +0000
Message-ID: <0D7E40CC-EA8F-4CE6-B4C7-A69C403BD4DF@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
In-Reply-To: <20210917154625.89315-1-roger.pau@citrix.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3608.120.23.2.4)
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 32b609c0-0da3-4ca7-7eab-08d979f523f1
x-ms-traffictypediagnostic: MWHPR03MB2574:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MWHPR03MB257438D663C07ED293091F23F6DD9@MWHPR03MB2574.namprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: NziO5PCjM3LXBCqzUVUZsVYShEy7xek4VL1YviRFIUfr4vxVOoeWyYd8lvxUnEsNTP/+YO9T9NPBD21tHBL3sDmNGKTrHBHcLSiKXDhh+vCHoG9BRWRmYvs1Aj4mbUcoGzO5/J6Q2zY2dGsdOO9a1GU9YGkOSbRim12vt0nnrrxxhwJOoQaXqZR7i48sy0ErYk9GZuwBIEcpjbjX0Gz+8WvY1rI7P58uHW10H6HeVldPM/LERSquIrfoB5Qqnn4gTHvqMAOpOzffxwNlc+0J6/znlwam8m60yKFbYzpzONkrYias02+U/9icUe6RSNIqzGRLuc0J9EjNGlsic5pGXfkynmuU8BZdbJxp9FuBesBHIGU5754+w1i1qGRX13dbTCpPctVLs3XJSC1oq2z+LM2aNjQq1UE4mYSEzG//doTg2hwxFotKhR6wnut/V/K55QEe8x/Iamdf9Z8Q6h0YJawGbfB+QJ0MHIlq+q4Pw/C5m4sHGX6no8HjQ+xWr30FzYDhLzdFrxck0c3M6ilYZwKYugUbQwTxSlELY96NL9NRctcUn5m4SWPvZisTreJTzzJzw3EMfUFWg/SR6h4CV5fLXfSdKGu31gTMCFgKncgPRpKfYqiui7eksBmHIHfuT8D3xpdfssMN0FoxQrO/CyiNeb4hYnPMdhTx/5XVg5QQRry4InaQiCSFTT9bQie4olQnUJrcnb9Z8DJhkQR11dPEcSVgqMJAPlwfbRhyPDw=
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6380.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(136003)(366004)(39850400004)(376002)(346002)(91956017)(8936002)(8676002)(4326008)(26005)(316002)(64756008)(2616005)(66446008)(66946007)(66476007)(110136005)(66556008)(6636002)(6512007)(186003)(86362001)(6486002)(83380400001)(55236004)(53546011)(36756003)(44832011)(6506007)(38070700005)(38100700002)(122000001)(71200400001)(478600001)(33656002)(5660300002)(2906002)(76116006)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?qnFeU4jdgMHIkBz9J7Kd7NFhKv4sFn3wCfudPPq8ayzV6gqJzh5k6P/+ZsqF?=
 =?us-ascii?Q?Uohdo9fquUzcUVEAJ75BHwXeGd4YkpDkzUA26dHYl0sOMW6HxqM3eH8Vqfls?=
 =?us-ascii?Q?llxLT3JPjzmAK8XQYJYSeoz9NIPoqZnnHxl9OZruji8W7W6cRfpOTN5fawwh?=
 =?us-ascii?Q?giM7hEIzOibr/wzYTVsk21JouTzp0d8g4Qs4M0XF6io9S8SGqHOdUolICHP/?=
 =?us-ascii?Q?3YQ2yISBurkU2k7enoWasj2KU7o64rI04flqnHGg2DXsiOQH57vXl6VuVBGu?=
 =?us-ascii?Q?MevCo3hQxYieaRyzi6Gpu7ZYrq7BGiJ7MmjsNgFBX1z9vsbehwjvwKNYowgu?=
 =?us-ascii?Q?Vd40vEMlCCxMnFcuaEg1KI7EZjJEIkwX//rv5Uy1SF7oJ7bvZUruw2UQ9M9q?=
 =?us-ascii?Q?nqhVgo/97E09e2UDu/5ecRX+s9DeCvgBMZ8MGogY0cQwCFq9LduBfUKCVcyD?=
 =?us-ascii?Q?XFUy+HLsYlubFX+u6yyWATIg2ZpWeLNs+FCe6OHVTj+Qayv+0X1kqricYe9A?=
 =?us-ascii?Q?cUCKqAUEZzb4FZ370ZfJLV+B1jDs+3UfNjzG9rOgpmOebL+X8lYKRH3E4g8P?=
 =?us-ascii?Q?dC64/kEI8YygeChjySrC4PLkfPyO1w7zaVLxjDqZj8rIlkj7C/Qu3TVEtc8j?=
 =?us-ascii?Q?bG9THeEgAEjYl7IJGUqGwfjwRdoi481yZTQKM+MGu/Asep+SVkOJ8+da58oi?=
 =?us-ascii?Q?fpmNEoL1km815ECMJ4rDchjd12loyir4svECdaXdxb2HF9tO6Sz1VDG/Goou?=
 =?us-ascii?Q?RfOMc17edmDyLySNlls87NyKI5J0bmmwMUvPnof3N4ZwUfQWwHjNlD4HeacA?=
 =?us-ascii?Q?0xXnAxqqDJM5V+VH9bhmsRCnBxI95o7K5dgfemP+jhWqq+6hWYoLcqJPh7nV?=
 =?us-ascii?Q?5+Gr1iS0OopvqeVnYVbqkVerH8ULji3m5UQ+GRiC67QfNnsKrfrFQdNz3Kku?=
 =?us-ascii?Q?egfzJWe8uP+Gmg1zSApX0QqkKnt9ls9wDDkIOA8d9HTV1HIanaL4Gj3+i8C4?=
 =?us-ascii?Q?MJmTJCnBdEL8C99a9bxgTkFzoB58m0scLU9ePcYXc3a5VhvM8UvolI5KvSft?=
 =?us-ascii?Q?K/jpKYkDikLZ/n/wzgGYUomRFHQojXqW/7S+Jg8+sh3HQ4XWB+P2YTdGQ3OW?=
 =?us-ascii?Q?HJhMucfpBbpNWtF2W4rJvE1EQnTc15I4OLZkFdNiZcuNQ7TsQS5RcV4bOVj1?=
 =?us-ascii?Q?g1hnlMGMNWT1McmR58zVNtH+uhjglLUVmEmpsy52eNhqJdmOtevtxSJ4l+B7?=
 =?us-ascii?Q?gMivo0V+4Zte0DtoBtidbZzo/J5Ig/nGlHox3ZpNZedbqyEYtGcF2PlZKtci?=
 =?us-ascii?Q?hhF/iZtiwvWJPJK/ygQm91V1?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <C38193C9C5E72A4FBB45470C58E9C072@namprd03.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6380.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 32b609c0-0da3-4ca7-7eab-08d979f523f1
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Sep 2021 16:06:42.4612
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 5UjoXQZiF+KJBSGRptqeZGMlhqdvH810oAPMbtV5dnpgUrxkihpuhLBT0Nnlp4W7g4FhyKqjVdVIWhdy8KqddsjfhaaV+YQH8isg0lmOK+0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2574
X-OriginatorOrg: citrix.com



> On 17 Sep 2021, at 16:46, Roger Pau Monne <roger.pau@citrix.com> wrote:
>=20
> Hello,
>=20
> The first two patches of this series allows setting the preisoutly host
> wide command line `gnttab` option on a per domain basis. That means
> selecting the max allowed grant table version and whether transitive
> grants are allowed.
>=20
> The last 4 patches attempt to implement support for creating guests
> without grant table support at all. This requires some changes to
> xenstore in order to map shared ring using foreign memory instead of
> grant table.
>=20
> Note that patch 5 will break the save format for xenstore records, and
> should not be applied.

Has this relevance for the format used by oxenstored?

>=20
> Thanks, Roger.
>=20
> Roger Pau Monne (6):
>  gnttab: allow setting max version per-domain
>  grant: allow per-domain control over transitive grants
>  tools/console: use xenforeigmemory to map console ring
>  tools/xenstored: use atexit to close interfaces
>  tools/xenstored: restore support for mapping ring as foreign memory
>  gnttab: allow disabling grant table per-domain
>=20
> docs/man/xl.cfg.5.pod.in                |  12 +++
> docs/man/xl.conf.5.pod.in               |  14 +++
> tools/console/Makefile                  |   4 +-
> tools/console/daemon/io.c               |  25 ++++-
> tools/helpers/init-xenstore-domain.c    |   1 +
> tools/include/libxl.h                   |  14 +++
> tools/libs/light/libxl_create.c         |   4 +
> tools/libs/light/libxl_dm.c             |   2 +
> tools/libs/light/libxl_dom.c            |   2 +-
> tools/libs/light/libxl_types.idl        |   2 +
> tools/ocaml/libs/xc/xenctrl.ml          |   5 +
> tools/ocaml/libs/xc/xenctrl.mli         |   5 +
> tools/ocaml/libs/xc/xenctrl_stubs.c     |  12 ++-
> tools/xenstore/Makefile                 |   4 +-
> tools/xenstore/include/xenstore_state.h |   1 +
> tools/xenstore/xenstored_core.h         |   2 +-
> tools/xenstore/xenstored_domain.c       | 120 +++++++++++++++---------
> tools/xl/xl.c                           |  15 +++
> tools/xl/xl.h                           |   2 +
> tools/xl/xl_parse.c                     |  13 +++
> xen/arch/arm/domain_build.c             |   4 +
> xen/arch/x86/setup.c                    |   2 +
> xen/common/domain.c                     |   4 +-
> xen/common/grant_table.c                | 119 ++++++++++++++++++++++-
> xen/include/public/domctl.h             |  13 ++-
> xen/include/xen/grant_table.h           |   7 +-
> 26 files changed, 341 insertions(+), 67 deletions(-)
>=20
> --=20
> 2.33.0
>=20



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:08:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:08:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189583.339356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGPX-0007FQ-5x; Fri, 17 Sep 2021 16:08:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189583.339356; Fri, 17 Sep 2021 16:08:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGPX-0007FJ-2n; Fri, 17 Sep 2021 16:08:23 +0000
Received: by outflank-mailman (input) for mailman id 189583;
 Fri, 17 Sep 2021 16:08:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRGPW-0007F9-7r; Fri, 17 Sep 2021 16:08:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRGPW-0002J2-3D; Fri, 17 Sep 2021 16:08:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRGPV-0006rE-Lt; Fri, 17 Sep 2021 16:08:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRGPV-0003D2-LR; Fri, 17 Sep 2021 16:08:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2/THn79/S6pUWnBMAmvj4tGAlerbQC9W4Get2c+AhjQ=; b=STtIN6PZ50cKxfRd3PVhqZgmav
	DHdVSKG3zO6vPIwbq9M4ePJ+fiGjqQpRSfzy/UQ9myJ1W3XHjB0/fUjVfTSVNEgXYXCkp/USkrpgn
	OeeUq2mCdhzsq+giQDsRK2ZXy86LWixQRaJwQjhsOnPyxCTRHnjwWOty1GZqewnuKAY4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165019-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165019: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=bdb575f872175ed0ecf2638369da1cb7a6e86a14
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 16:08:21 +0000

flight 165019 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165019/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                bdb575f872175ed0ecf2638369da1cb7a6e86a14
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  412 days
Failing since        152366  2020-08-01 20:49:34 Z  411 days  717 attempts
Testing same since   165019  2021-09-16 22:41:26 Z    0 days    1 attempts

------------------------------------------------------------
7312 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2259637 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:28:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:28:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189595.339380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGjJ-0001f2-9c; Fri, 17 Sep 2021 16:28:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189595.339380; Fri, 17 Sep 2021 16:28:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGjJ-0001ev-6t; Fri, 17 Sep 2021 16:28:49 +0000
Received: by outflank-mailman (input) for mailman id 189595;
 Fri, 17 Sep 2021 16:28:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/QZl=OH=kernel.org=helgaas@srs-us1.protection.inumbo.net>)
 id 1mRGjI-0001ep-AP
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 16:28:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 54c99d42-17d4-11ec-b6e7-12813bfff9fa;
 Fri, 17 Sep 2021 16:28:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D155160555;
 Fri, 17 Sep 2021 16:28:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54c99d42-17d4-11ec-b6e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631896126;
	bh=RRE6NMda31/dIqqeP4g2Zg1MqUaKnLsM3Ro/I9LVt3w=;
	h=Date:From:To:Cc:Subject:In-Reply-To:From;
	b=IBbNMhquNK9liwy6lIWmdPIXzOZVsmIh3XmUdeu0OCTs9jMQRFZj4wNKUIerdiVKo
	 mBMDO993ITGHINd9RLo1DY66iD3VOGkNh0ua2l7uUucIPa9VgX2bJ3+P0Wr0K7/+HG
	 og0S6Mz5MOJn7XReD5EeUMdc7TA2gihk25XztwGh86GImWJqlCB0TDSX67KV04NaMv
	 p0osTnLc/lTsOfpedLQ+EjMmJ/gAqIX/e1CLxFaP52jp3kxgAwF0cddgUxqB22Gld3
	 JCAGcSeUs7Bd1NpgzoHU2PijOf49moRGDvKEylzevEZ2zsFtcDoaQ8cwiS5GKTUyN4
	 Jjblai00CbORg==
Date: Fri, 17 Sep 2021 11:28:44 -0500
From: Bjorn Helgaas <helgaas@kernel.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: "bhelgaas@google.com" <bhelgaas@google.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
	"iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH v2 2/4] PCI: only build xen-pcifront in PV-enabled
 environments
Message-ID: <20210917162844.GA1722208@bjorn-Precision-5520>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3a7f6c9b-215d-b593-8056-b5fe605dafd7@suse.com>

s/only/Only/ in subject

On Fri, Sep 17, 2021 at 12:48:03PM +0200, Jan Beulich wrote:
> The driver's module init function, pcifront_init(), invokes
> xen_pv_domain() first thing. That construct produces constant "false"
> when !CONFIG_XEN_PV. Hence there's no point building the driver in
> non-PV configurations.

Thanks for these bread crumbs.  xen_domain_type is set to
XEN_PV_DOMAIN only by xen_start_kernel() in enlighten_pv.c, which is
only built when CONFIG_XEN_PV=y, so even I can verify this :)

> Drop the (now implicit and generally wrong) X86 dependency: At present,
> XEN_PV con only be set when X86 is also enabled. In general an
> architecture supporting Xen PV (and PCI) would want to have this driver
> built.

s/con only/can only/

> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

> ---
> v2: Title and description redone.
> 
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -110,7 +110,7 @@ config PCI_PF_STUB
>  
>  config XEN_PCIDEV_FRONTEND
>  	tristate "Xen PCI Frontend"
> -	depends on X86 && XEN
> +	depends on XEN_PV
>  	select PCI_XEN
>  	select XEN_XENBUS_FRONTEND
>  	default y
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:29:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:29:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189603.339391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGkA-0002Dn-K1; Fri, 17 Sep 2021 16:29:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189603.339391; Fri, 17 Sep 2021 16:29:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRGkA-0002Dg-Gl; Fri, 17 Sep 2021 16:29:42 +0000
Received: by outflank-mailman (input) for mailman id 189603;
 Fri, 17 Sep 2021 16:29:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j5MY=OH=kernel.org=pr-tracker-bot@srs-us1.protection.inumbo.net>)
 id 1mRGk9-0002DW-90
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 16:29:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 74ddd350-17d4-11ec-b6e7-12813bfff9fa;
 Fri, 17 Sep 2021 16:29:40 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPS id F100861019;
 Fri, 17 Sep 2021 16:29:39 +0000 (UTC)
Received: from pdx-korg-docbuild-2.ci.codeaurora.org (localhost.localdomain
 [127.0.0.1])
 by pdx-korg-docbuild-2.ci.codeaurora.org (Postfix) with ESMTP id EAEF960726;
 Fri, 17 Sep 2021 16:29:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74ddd350-17d4-11ec-b6e7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631896180;
	bh=3hNpJBVLgBNsFzSfX3k7E2bkIa1y6yO1fC6HDXbuBH4=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=s7iULun5C/kR5/GiKW9wJjaZvu1Ky75KLgmsEIpWYz1fgmENm9MueTTmg6cyiSLoS
	 28vX2EpQJu9xTvwDFr7N9uOJTOer5VWP053CERoEh/21FVgE02CKoi6jg2KlciTWnu
	 BG/vBVvbd88xEPgcitFE0X1sb4E+nGyzFPRe2S8P+EiANbH0scraDGLAj1tu0ijYun
	 ASev0Dr1LY63NrTinykSLRhKRyhhKBf4m/W7vJA3U4TJOXAL/Se7Zvg6n/ENNFnINv
	 EA57I0RLpPsH/vlER9KYif/sbQRj6AqfPB0poitYPwSrUOELX2RaVA4jwdskxNXr/n
	 +Fn6NCjiqOMog==
Subject: Re: [GIT PULL] xen: branch for v5.15-rc2
From: pr-tracker-bot@kernel.org
In-Reply-To: <20210917113526.7963-1-jgross@suse.com>
References: <20210917113526.7963-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20210917113526.7963-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc2-tag
X-PR-Tracked-Commit-Id: d859ed25b24289c87a97889653596f8088367e16
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: c6460daea23dcd160f2dc497c64b4c882ea1de69
Message-Id: <163189617995.30150.3872477863690937936.pr-tracker-bot@kernel.org>
Date: Fri, 17 Sep 2021 16:29:39 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, boris.ostrovsky@oracle.com

The pull request you sent on Fri, 17 Sep 2021 13:35:26 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc2-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/c6460daea23dcd160f2dc497c64b4c882ea1de69

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:53:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:53:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189619.339411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRH7D-0005kx-PK; Fri, 17 Sep 2021 16:53:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189619.339411; Fri, 17 Sep 2021 16:53:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRH7D-0005kq-LO; Fri, 17 Sep 2021 16:53:31 +0000
Received: by outflank-mailman (input) for mailman id 189619;
 Fri, 17 Sep 2021 16:53:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRH7C-0005kg-FK; Fri, 17 Sep 2021 16:53:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRH7C-00034d-7T; Fri, 17 Sep 2021 16:53:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRH7B-00018Y-Uf; Fri, 17 Sep 2021 16:53:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRH7B-0004aA-U5; Fri, 17 Sep 2021 16:53:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KXtdGFPgRZIYDEYzSZif777nyNpl6CVx/duukOgMN2c=; b=68wichmpUZEVc1EtvEd0dOtu0L
	FrzZOcVDAnCvbjJH4VgYIh/qPnOTh08LkHNbS63diHEuAzJ3Fg5rt9Ys0oriYq9qfCTHBvO5jNjOG
	Sz/siz+6YXVl6ZfEjEfnj1FDzVFU1A+Ebol4yAzDZ3H1rWz5aaNsa0xoS7UFHywf0EhQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165028-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165028: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=eba98a1db93474678d7358886fcd12a823e158f4
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 16:53:29 +0000

flight 165028 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165028/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              eba98a1db93474678d7358886fcd12a823e158f4
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  434 days
Failing since        151818  2020-07-11 04:18:52 Z  433 days  424 attempts
Testing same since   165028  2021-09-17 04:18:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73782 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 16:54:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 16:54:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189625.339425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRH7h-0006IX-2j; Fri, 17 Sep 2021 16:54:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189625.339425; Fri, 17 Sep 2021 16:54:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRH7g-0006IQ-Uk; Fri, 17 Sep 2021 16:54:00 +0000
Received: by outflank-mailman (input) for mailman id 189625;
 Fri, 17 Sep 2021 16:53:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LSos=OH=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mRH7e-0006IA-Pw
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 16:53:58 +0000
Received: from mail-pg1-x52e.google.com (unknown [2607:f8b0:4864:20::52e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 465bc9bf-2a29-4fac-a646-cdc1bac30418;
 Fri, 17 Sep 2021 16:53:58 +0000 (UTC)
Received: by mail-pg1-x52e.google.com with SMTP id q68so10146157pga.9
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 09:53:58 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id h21sm6580708pfc.118.2021.09.17.09.53.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 17 Sep 2021 09:53:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 465bc9bf-2a29-4fac-a646-cdc1bac30418
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=fRMxd8azh7E1BFxsMIqGYJmHYephRNLnlBlA+X7n5iE=;
        b=OCd2JnsFm+PC0WRg9Dy8nwrv0xb6S01W9Op4r/ZIu2wiEWDdIoF9l2WbVBKByhSPPF
         pgrkna7a3wLYVWEKuAik8asYNbpDEgJtdjZ+5dk4g6kfSL2XrdSoKcoPwqesfcoOwhOu
         WWbUTglAWniqlHYXmzaYzrF0F49uRRMs/V59Ek2JYLZBnMp92N934h45zumxZj/0PpGD
         /zvBIntmv3EDlaS6KVxhx1MfK4sOMy/cLwcByoT+tkw90D/Cj3jlQCjxNda2rrqk2GSV
         L6ETbBTdUnfohyRSi9IUyYAHcI9Z9pYHZQ5tjcYdmN/t6vpNHZYY3PO0hm+uRnpvMlkD
         xKpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=fRMxd8azh7E1BFxsMIqGYJmHYephRNLnlBlA+X7n5iE=;
        b=ePKZMYN+A4Jr1Xt1WTD+sTanhLsq6GNFVOIsyvuZQ6ZVEDupL5NgrGMFVPaBR7LT3b
         Ugd0tAtzFzNjWwu+rXYUDny9U5NKNuJ3IdZyaUPgK/WSHc8FDtA04+ebfVasGymTkmCo
         XnTGQTFo8XbUJBGpjHmR5nqtUVa4LgybnvykN7jForO9xALhdegJXmsQpm+GFIELgEJr
         6NfFDx67mEbrpBaD3CURqHSDJtMH36uj3SsaFULu8uyyNVxqPUqk46KIXzqgtNUmqtrO
         g2S0ebnm5D8GVg7SdBBuA/vaOb1diDQSTnTjS1rjvuPoKwRgK2Q6tnbgmX+KBfvKU2iv
         jL9g==
X-Gm-Message-State: AOAM532E+ppTudcm8xWAE3Q8SGofva6gxUsOuHQx9QSOvx4QrOJYrtqy
	Q6g36fVyXXZ+prNAYhvL8EqWNg==
X-Google-Smtp-Source: ABdhPJyVPh/MP+xok4eNnL6IICz9yFWugy2q7G+GDgXi8Z/wRNwWTua9Dj2q+LfRii59WPG0Vad6Iw==
X-Received: by 2002:a05:6a00:174a:b0:433:9589:bdb5 with SMTP id j10-20020a056a00174a00b004339589bdb5mr11713777pfc.5.1631897636965;
        Fri, 17 Sep 2021 09:53:56 -0700 (PDT)
Date: Fri, 17 Sep 2021 16:53:52 +0000
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
Message-ID: <YUTIIG4ZCKMbqrFi@google.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
 <YUO5J/jTMa2KGbsq@google.com>
 <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>

On Fri, Sep 17, 2021, Peter Zijlstra wrote:
> On Thu, Sep 16, 2021 at 09:37:43PM +0000, Sean Christopherson wrote:
> So I don't mind exporting __static_call_return0, but exporting a raw
> static_call is much like exporting a function pointer :/

Ya, that part is quite gross.

> > The unregister path would also need its own synchronize_rcu().  In general, I
> > don't love duplicating the logic, but it's not the end of the world.
> > 
> > Either way works for me.  Paolo or Peter, do either of you have a preference?
> 
> Can we de-feature kvm as a module and only have this PT functionality
> when built-in? :-)

I agree that many of the for-KVM exports are ugly, especially several of the
perf exports, but I will fight tooth and nail to keep KVM-as-a-module.  It is
invaluable for development and testing, and in the not-too-distant future there
is KVM-maintenance related functionality that we'd like to implement that relies
on KVM being a module.

I would be more than happy to help explore approaches that reduce the for-KVM
exports, but I am strongly opposed to defeaturing KVM-as-a-module.  I have a few
nascent ideas for eliminating a handful of a random exports, but no clever ideas
for eliminating perf's for-KVM exports.


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 17:45:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 17:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189645.339460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRHvh-0003uu-EZ; Fri, 17 Sep 2021 17:45:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189645.339460; Fri, 17 Sep 2021 17:45:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRHvh-0003un-A0; Fri, 17 Sep 2021 17:45:41 +0000
Received: by outflank-mailman (input) for mailman id 189645;
 Fri, 17 Sep 2021 17:45:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRHvg-0003ud-EK; Fri, 17 Sep 2021 17:45:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRHvg-0003xd-92; Fri, 17 Sep 2021 17:45:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRHvg-0003wy-3l; Fri, 17 Sep 2021 17:45:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRHvg-0001lr-3H; Fri, 17 Sep 2021 17:45:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nxXGX4P6Mx64kHJpNdanKfMin7YyCLEPvCcTnRbTQ4k=; b=BPRjgm7uuaadclq//q0PfMmykT
	lPno8tgd7OSxluh6FaOLhFWAM6CoumJD5M8ogf0mUuJq06iA/gCcZQwpJ17esXjGilisQ6BVBBzWN
	rbNOE93/tiNIgzLan3fnANlOnKZZ33e/5zEh5zG6h/6Wt9EKldmR8qhNQHgV6nKXnr/Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165069-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165069: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=48a160f46251f9752bd39a854ca74d80fbbbe90c
X-Osstest-Versions-That:
    xen=98f176b62b6136f696da88032f9c12a244ac9aed
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 17:45:40 +0000

flight 165069 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165069/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  48a160f46251f9752bd39a854ca74d80fbbbe90c
baseline version:
 xen                  98f176b62b6136f696da88032f9c12a244ac9aed

Last test of basis   165018  2021-09-16 22:01:36 Z    0 days
Testing same since   165069  2021-09-17 13:00:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   98f176b62b..48a160f462  48a160f46251f9752bd39a854ca74d80fbbbe90c -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 17:51:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 17:51:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189655.339477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRI0u-0005am-7g; Fri, 17 Sep 2021 17:51:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189655.339477; Fri, 17 Sep 2021 17:51:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRI0u-0005af-3h; Fri, 17 Sep 2021 17:51:04 +0000
Received: by outflank-mailman (input) for mailman id 189655;
 Fri, 17 Sep 2021 17:51:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ysil=OH=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mRI0s-0005aZ-PW
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 17:51:02 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1855a3c4-e705-4135-bf73-d2c82e1bab8c;
 Fri, 17 Sep 2021 17:51:00 +0000 (UTC)
Received: from pps.filterd (m0246631.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18HFwp4f015158; 
 Fri, 17 Sep 2021 17:50:59 GMT
Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b4vvy0qxk-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 17:50:59 +0000
Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1])
 by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18HHUaZO049488;
 Fri, 17 Sep 2021 17:50:58 GMT
Received: from nam11-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam11lp2173.outbound.protection.outlook.com [104.47.57.173])
 by aserp3020.oracle.com with ESMTP id 3b0m9b5sex-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 17 Sep 2021 17:50:58 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4318.namprd10.prod.outlook.com (2603:10b6:208:1d8::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 17 Sep
 2021 17:50:56 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4523.018; Fri, 17 Sep 2021
 17:50:56 +0000
Received: from [10.74.115.10] (138.3.201.10) by
 BL1PR13CA0201.namprd13.prod.outlook.com (2603:10b6:208:2be::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Fri, 17 Sep 2021 17:50:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1855a3c4-e705-4135-bf73-d2c82e1bab8c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=8RzBI9rQ7gL3OCEC1uwYXpUxnhWvDlTVcndSDMLtFQo=;
 b=IlUXXzcYnvSia0ltzINuloUhOTzXcpbnRjvo3tEqZaD2xAadSUxglIw3R5tw+O6TzuuJ
 IQ9Utm6zgMvCmBrXRfrL9bQ2TSQDqDUO9uMn/jTypB5lYElsTRSBk735Q/ql0QfqUEby
 5vCmT+B84bi1ukj2rLHBKSqYCflXKSqVktul7AlQXT3YmCnrPAPNTUACutNtLWpP3HQr
 TrvIB6VOwW4B1MxZvr5SDCSejRIcqz3Ux/pWrNduKVsUmv13NMO9C4kFXllCChne06pf
 2rLXwivxcex1w0iPi3VP05DrLbmltiyUE8G7KJswYs2X3S07RP2NMP/So4kzzl8jjTNH iQ== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2020-01-29;
 bh=8RzBI9rQ7gL3OCEC1uwYXpUxnhWvDlTVcndSDMLtFQo=;
 b=jo48fvV20lVq/Kd9VbWSX7CfPFOjNt+A6h96687kIu8JNgG/lh5QWHgxjOFVjzvCAsuL
 pX6bWdweK65ccmJdLEfXWapvuTkhPp0s3UIYslkBmKgJlutTcTdvHmLOUwYvpRchB0eX
 8NZ76ckRMsJp9hPMfB2hAD1gOBsreOXdAO+tV/FGUw0l/iUVc+A1dt58BnXmWCiYCTeS
 sG8k7VGcg355PCYdqzOMkU0mncNWbvWlaN1GleiKqtuUNgQ2VmE2lLqwrnJJYt5fZAXx
 qAph1RILkn5dMZ4YS8jgJufd+xZHNNwRM1IQfhaNxohbojpiMfR+TWEz6ffgYOIahZJi xg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=A0qFHVYgg0pVljjE4GeYun+vqQW6/uIFrecfatxk2fKczPSad21cqo1EvP2tzWny7D2W6pMagRG/UW5tJZKVVxfaaLIjmrXU/E3vM9i6QTOWCjmUfdrYKTBn6BAUUqHkdfYXlDuZrrQLtE9suwCbQEwPaShpWEL5nnmApXbxS17ehYJOkIVGCpR3d6jI5pcINldPp4uO54R/zH9h6c0mDs0DxFDJDqlVLUGG4FoL6WRjHs7BUB+LLEF6dyZ33cjEAanlA9GqaImAdBGoFbV73akhwa4bX7BPFwHK4gl1xM4faW8/oRRSZa3FYbpmarhG51EbB6e/+JDz3L6nv+IrKA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8RzBI9rQ7gL3OCEC1uwYXpUxnhWvDlTVcndSDMLtFQo=;
 b=kCSbPAQHPN8LwOik1/b9WV4ZDn5o2m4m2Ahb7b9coiwD4z8vTsNfyZOzjja21RGsJ8svjleIu5hkvlHfMmBHpOQe0CVLpGITqJ5SeZ8MbfYZr0r3xP4kjTU/wSOR1zRglaWpWPXt0SfZqssnGqUWB+3e1ABxzTpOZua9E6OecxaG97tg6y914h9JPQHWo3Os7rxrH0fnwA/vofPIn5jWp/A2O8mwkfyCCSQqELDUsxdb0j0644xNJe9nzmS2ywPWLAB/Ikggp3nXTNMSVqALYm8ZsvFKQwXdpFqD03Qc1x3ZsozOvOqe0hrBOYHVIoVx00XlCmN1bGb1H+wTz1EMjw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8RzBI9rQ7gL3OCEC1uwYXpUxnhWvDlTVcndSDMLtFQo=;
 b=AJl38nw7E5m04fncPbBD2dHedShEHcSJ9oGxuCIWw3jYTtAzsF2UOhB2c56BG+yzh2oyW+Dh9UpL+jL3Pdqz91JxOjRQGGjrnmLDB+Gr6CDm1FGcJF66v6MiFWCX779cU4rczSVvMVoP4pwn36eJbRjKqrFZRco7/Zfp4pAGtTA=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=oracle.com;
Subject: Re: [PATCH] xen/x86: fix PV trap handling on secondary processors
To: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <34898e9c-5883-a978-98ee-b81b22d8caed@suse.com>
 <823f4ef4-f9e5-68cb-d6e9-d079483c1e21@oracle.com>
 <0afce6e8-3c8a-e5ae-cd54-0fd598276506@suse.com>
 <62ccf5b7-b903-e604-d113-67c7633278cd@suse.com>
 <2c4549c8-bdeb-3584-95c4-7076b7cf79bb@suse.com>
 <35835650-bbbb-7dbd-061b-693f39f9453e@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <1bdda8a5-4748-6791-58a9-b1e342281b01@oracle.com>
Date: Fri, 17 Sep 2021 13:50:54 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <35835650-bbbb-7dbd-061b-693f39f9453e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-ClientProxiedBy: BL1PR13CA0201.namprd13.prod.outlook.com
 (2603:10b6:208:2be::26) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 20811149-4c5f-4209-bb56-08d97a03b395
X-MS-TrafficTypeDiagnostic: MN2PR10MB4318:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB431811C69CD925E62A7FA54F8ADD9@MN2PR10MB4318.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	CmHu0Td6lzWtcHJmxK4ZS6uXRwJyMP5RAabkbn4bDmb/IrJtPeq4Z6A7bReI7iFQqeKDMxntTUZh99sHy+/nOHqfZPXOq0c8wcyJ4xUVyNINqRNviDvdPLxN5lzJNsIFnmHmD//7oNxFPVoJMfnGCQKmNfOq4AFTqJXQzkzK9Y4Vh0Inlw8MUfhKpcdH0KOYhCQC+MwkL8oK2poBBlgn5aOsBPPSovH37e7BWZMmc5hh7f0pJ84ECAeKoTMAS9DRAjDE0cncgt2cN5EN0WSIpoNuplow7cIaYpVERb+zf0OmcEs1+VLlf8NR6IWLJS71ZXW0AzZZ9kHO/0UIl/GjouVqt8QtGq7Q07LmG3nSmp5n3XGWV+WCH2sBpgXErP1jB0cxNbccCGpEuwf946IgR890xsmlnlAR8TjsjM5gPaKV0G44wx/GdqK551QlUYhGLb3UItZLxJjr1m1SWrtfhZNAtB/E8FxbNKh4++Z4udEoJITC5QVAWaroCCc8fvlHjFIg1jMX11PYqXjmxvtPFbwWFb6jARhm6oJleq8sWJCtMgDW51uFMGN1QZyDy9PxsiazaWU6QVGqi43HfMfvoCJpKo9guJ5TMJ4QW42gO3+6um4PsW7uZMY1P7OXzUoyZpAQQeHHI3i1H+9IRMfYktB1xqmGak+B1w1unFD7dS9+KRCjjwfGrBPrh52HDaof7RuVtXphdct8DSJyRp9BKgsnawQezE8GminhNf9x+v+b83OFOKCJA9g7ajWbN9Vl
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(396003)(346002)(136003)(39860400002)(31696002)(478600001)(4326008)(2906002)(186003)(86362001)(6486002)(44832011)(38100700002)(316002)(66476007)(54906003)(53546011)(83380400001)(16576012)(36756003)(66556008)(26005)(5660300002)(2616005)(8936002)(956004)(8676002)(31686004)(66946007)(110136005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?dGtEbXZDanhHUTlFWHBwYktiN3hRUkZ1a1VHWUVnM2djVjF3ZEZoRVMyNDlV?=
 =?utf-8?B?a1JpcTYrd2wwajRDMEUzY2xZeXN5N011R2NoL3JBRXNVMkNGYnBmN2c4YXIx?=
 =?utf-8?B?WitmSGFVK0pLZ2NkN1p2Z2JNWVQzbzg0ZGpVYVd1MlA1TmhoWVN5L1B2U1J2?=
 =?utf-8?B?eUVPSUl0S3NRbmNwckhTcmRjd0VxVVdEVXprZnVHaUxEUDc4RnJkeUkwaGlt?=
 =?utf-8?B?ZmJFQVJlK2JEMlpvSkxiM2Fzd2dHemt0VHRTMzFTRU4rTU5JcHQrNTdvMzA4?=
 =?utf-8?B?NENidjN5eFBZVWxtenBkWk9Rc1B3RU1DTm1ocDhUSGNQUWhWSWpVNFZybkVv?=
 =?utf-8?B?aS80aW5zMVNkRDBGT2FtLzk3L0Zsa2RHSnVEQXNxWnZDWkVmUEtReUFMd3RU?=
 =?utf-8?B?ZWVkbXZHTFgvY05xT254M2NPQlBVSmNYa0czcnBiSEJPNlNjY2w3R1JlOVd6?=
 =?utf-8?B?N2lyVVBPQzIxS1pwUGdNTGpGc2R2VFZVOUlxUko4SkF4N1I0eDRPdkdOYWtG?=
 =?utf-8?B?bG9vSmlyc1Z1MFJucmV6ZnViWEcvb0NoUGxVT0k1MEMxcTE1UVd4MWQyQ0Nt?=
 =?utf-8?B?clFtdkdhMXN4TGtJUVJjZlFodjhCa05USVl5Q2N5ZjF1c2prSlJtc1FPSFY0?=
 =?utf-8?B?bUJYZ2x0bG5ZdVVES2dVbFRwVTN0NmRKMFBaTEZTTHp3NFZHVnBlVjlPS1Zr?=
 =?utf-8?B?WStzVWszRHFVMVRJdXl4bGxZOFVSblVhREtMUkxOckJ0SGl0TjlpN0cwaXpV?=
 =?utf-8?B?SU45eld0aXA1L3lRYmRMcjNLdDRldEpUT2o1ZGhXMitZOUVsK0lMMFAvNVBl?=
 =?utf-8?B?U3l4SytzaDNxaDQwaHFQejFpMHVXMkRSSGlJenk3TjFBODFiSVhvVWFwUXhS?=
 =?utf-8?B?QUM4NTIzUHIrenNuZWcvQW5MZHoyYWFGTm11MDUyZXh4V1lJdVppRGFrajVT?=
 =?utf-8?B?MFBFL21GYzFwOHQ3RUo3SHIxRGRUd01VVUtFdFB0Sk9USWUyVUpsZnh5dFp0?=
 =?utf-8?B?S0tYQmh2Qm5qakE5V1lpS0FGSGIxa2JTc2FhNytINWZ2ZkhnallGVHVtUkta?=
 =?utf-8?B?M0k3VnBZWTd4M0p2dk9KVGM5anVqSW1KdTUzaTgzSUFxbjhxVGxWZVJablAz?=
 =?utf-8?B?MzV3aEFqcElJM3RHVE5EY1BUaGlxSkU0YkhSdC95Y0cybkpoSXp0RW1OZnZu?=
 =?utf-8?B?SnU1SXA0bnpnZ3k3YldGNHYzY2tWK0tRVkpjWnNnQmZlU0ZLVGZzZytHS1dn?=
 =?utf-8?B?bUdnb0d3L1pFTlBrRHZTdFVRY0J3M2d6ejVDaU9QMWU4WktsdlZoa1FsNTdQ?=
 =?utf-8?B?WG5zMU94WWxqeGFHbENKdU4wU1Z5TGtwVmZjYmxIdGhUa1NNejFrRDd3R1hS?=
 =?utf-8?B?VVRHR1VZS3gxZ1p4bVllSkdnK3U1cGlLbllVVHJYS3VNOW5OUXlwMjFjbGRo?=
 =?utf-8?B?b0dVT29ZeFc4ajg5ZkUvczFwcFB4b2k5R3FiRENCejB3NDJUd2N0SVhOdWwz?=
 =?utf-8?B?ZEJXdW1SdDNvaWM5dU1IWWkvSlZibnFoc3M3WERnNkNJSmFQQUZDYzFGbkhW?=
 =?utf-8?B?ZUhpK0YvYnpVbzJ0aGQ5cm1NYk96SFZubzBsQlhwTHBhcEtDbnJjSHExcDh6?=
 =?utf-8?B?MXRMK2ZndUh5cnBvb01WeGx5WmZyZjRTeVYxZVdHRVZVNFpOTjNSR3Nnb1ZO?=
 =?utf-8?B?UE1EdzBaZHFQQk50YzlXcDV0NHJWbk1hcXErZ2l1Q0RqcDVFc2NQTXE3MVFC?=
 =?utf-8?Q?HzJGQwmsTUN02aqc/gM/dHEKTotHCYgrQF/HDr2?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 20811149-4c5f-4209-bb56-08d97a03b395
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2021 17:50:56.7584
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1M7lCaYKdya8lcRjxAFU/MwWD+sIZw9IBG3tBSWqycq9JoSh4gknGQZ86xLw3IUBuHM8VEnqnBSNJ/5oOyUCkz7/KhFjeaTVpZIh1juUk6A=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4318
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10110 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 malwarescore=0
 adultscore=0 mlxscore=0 mlxlogscore=999 suspectscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109170108
X-Proofpoint-GUID: qz4rghgpVjQRIrD0YimMuXXzC1OH82TJ
X-Proofpoint-ORIG-GUID: qz4rghgpVjQRIrD0YimMuXXzC1OH82TJ


On 9/17/21 3:24 AM, Juergen Gross wrote:
> On 17.09.21 08:50, Jan Beulich wrote:
>> On 17.09.2021 08:47, Juergen Gross wrote:
>>> On 17.09.21 08:40, Jan Beulich wrote:
>>>> On 17.09.2021 03:34, Boris Ostrovsky wrote:
>>>>>
>>>>> On 9/16/21 11:04 AM, Jan Beulich wrote:
>>>>>>    {
>>>>>>        const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
>>>>>> +    unsigned i, count = (desc->size + 1) / sizeof(gate_desc);
>>>>>>    -    xen_convert_trap_info(desc, traps);
>>>>>
>>>>>
>>>>> Can you instead add a boolean parameter to xen_convert_trap_info() to indicate whether to skip empty entries? That will avoid (almost) duplicating the code.
>>>>
>>>> I can, sure, but I specifically didn't, as the result is going to be less
>>>> readable imo. Instead I was considering to fold xen_convert_trap_info()
>>>> into its only remaining caller. Yet if you're convinced adding the
>>>> parameter is the way to do, I will go that route. But please confirm.


Yes, that would be my preference. No preference on where to set the sentinel.


Thanks.

-boris


>>>
>>> I don't think the result will be very hard to read. All you need is the
>>> new parameter and extending the if statement in xen_convert_trap_info()
>>> to increment out always if no entry is to be skipped.
>>
>> And skip writing the sentinel.
>
> Maybe it would be even better then to let xen_convert_trap_info() return
> the number of entries written and to write the sentinel in
> xen_load_idt() instead, as this is the only place where it is needed.
>
>
> Juergen


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 18:27:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 18:27:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189669.339506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRIaH-0001Jj-Da; Fri, 17 Sep 2021 18:27:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189669.339506; Fri, 17 Sep 2021 18:27:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRIaH-0001Jc-AL; Fri, 17 Sep 2021 18:27:37 +0000
Received: by outflank-mailman (input) for mailman id 189669;
 Fri, 17 Sep 2021 18:27:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRIaG-0001JS-1o; Fri, 17 Sep 2021 18:27:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRIaF-0004iX-H8; Fri, 17 Sep 2021 18:27:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRIaF-0005W5-7H; Fri, 17 Sep 2021 18:27:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRIaF-0001ps-6o; Fri, 17 Sep 2021 18:27:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EDtWCLJeN+EzxOS+tEKeoIUIEFU4OkPyo+Lh3voNWX8=; b=xoISNW108+1B5RKWLT6Q1Icpyc
	6nqELYKO8F2tzYnArsjCQFVU4Q8iijhULWRAdVogbUtCAwoEwH5aTrnCK3NaAVp7moGVv5wiD+WkJ
	mczwjqwmJpMmxPGlBIN4/BPQWyGHlHNa6lRGUXP5m9B6M+2vkwW0uIHN8iBYR3trim9k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165021-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 165021: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=48a24510c328b3b3d7775377494b4ad4f58d189a
X-Osstest-Versions-That:
    linux=245f15a48cdc4d5a90902e140392dc151e528ab8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 17 Sep 2021 18:27:35 +0000

flight 165021 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165021/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164993
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164993
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164993
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164993
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164993
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164993
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164993
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164993
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164993
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164993
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164993
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164993
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                48a24510c328b3b3d7775377494b4ad4f58d189a
baseline version:
 linux                245f15a48cdc4d5a90902e140392dc151e528ab8

Last test of basis   164993  2021-09-15 08:12:34 Z    2 days
Testing same since   165011  2021-09-16 11:11:20 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Sasha Levin <sashal@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   245f15a48cdc..48a24510c328  48a24510c328b3b3d7775377494b4ad4f58d189a -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 19:51:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 19:51:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189680.339524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRJtO-0002jb-N4; Fri, 17 Sep 2021 19:51:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189680.339524; Fri, 17 Sep 2021 19:51:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRJtO-0002jU-Jy; Fri, 17 Sep 2021 19:51:26 +0000
Received: by outflank-mailman (input) for mailman id 189680;
 Fri, 17 Sep 2021 19:51:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jmcU=OH=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mRJtN-0002jO-DD
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 19:51:25 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2d116eca-84fe-481c-876e-fd5a4aeaa2bd;
 Fri, 17 Sep 2021 19:51:22 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id x27so37406443lfu.5
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 12:51:22 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id z5sm797124ljc.123.2021.09.17.12.51.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 Sep 2021 12:51:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d116eca-84fe-481c-876e-fd5a4aeaa2bd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=WYr4zzNq1cI/srAikGZcpR00UKFOVEEZrIQbhI86kbo=;
        b=AI0Q45z32pmw6+nANgi0B+P18bxWzZragWOa8Ufoys/erHxMFnupZ/5lduGRaCIKgb
         hzS9tu6IMa9E/1P4kjYs4pZXD+X9omYu9Oq8XciV5wlarHwoPKUSh384PzxMGVMKeP9l
         WTdnikgnAi/R6ScmPlrEu700wzn2BI3H8ft2NT6NlM3zGMtuIqKALM4bVMZ0xdEl6AaA
         /MooHNpSXT9SHdxcOhvEqQmGrgk/jbeUdtL1lKNcQla90Qto5FUdDSd4ynnlZSlp4vQe
         gW7aji1svI93ltelv7svDEKjEHru6wGM/p0qdVr/yz4ymHtLs0Q/ur+oDRUGkwo7kss0
         Yj4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=WYr4zzNq1cI/srAikGZcpR00UKFOVEEZrIQbhI86kbo=;
        b=ExnzU00j+04mTZe0YldQC4vNMpukAuG5dk3bjyaktu4blm9k8k5QEj7lv5VlVEUr+A
         ArqcpjOI3bzMWGiruyQrtieM3HlumGN0k5ig51bsGTRK6qQI884SZBljtrVQBErNQGga
         /zZfyFJ7mjiBvPzkBt4KhYNZVlgU8NZcTECBVsVcJowfi4hXVdKP3CE32HUjbyZSwp13
         fNqaqT5RCrpgFSdSVxXWYwoqxGPKq7LSpX4vYXUimg/ZUf3YilJngkltQpVkovau62sH
         5iGDLuHwmbO+e6uDeORkRcRYYtlHHvWHquO4+1j4/LSnee8O8aVH7pLH0hVSd+bxaFMh
         qxHA==
X-Gm-Message-State: AOAM531GAR4U1PwBRw/gQXhXCE4U9NTJFJmExqCFxDeb14v52p699XNW
	tmMyFYm6xPB4OHXpM+7ybto=
X-Google-Smtp-Source: ABdhPJyitVwl2ACKH3Mpth0/Vzijvys8A5Obs/WtYg6EIQgFHgdV3haNS3KwWD/v5hI65KkoTfYaVg==
X-Received: by 2002:a19:f24d:: with SMTP id d13mr9308532lfk.3.1631908281566;
        Fri, 17 Sep 2021 12:51:21 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
Date: Fri, 17 Sep 2021 22:51:20 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 17.09.21 18:48, Julien Grall wrote:
> Hi Oleksandr,

Hi Julien


>
> On 10/09/2021 23:18, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The extended region (safe range) is a region of guest physical
>> address space which is unused and could be safely used to create
>> grant/foreign mappings instead of wasting real RAM pages from
>> the domain memory for establishing these mappings.
>>
>> The extended regions are chosen at the domain creation time and
>> advertised to it via "reg" property under hypervisor node in
>> the guest device-tree. As region 0 is reserved for grant table
>> space (always present), the indexes for extended regions are 1...N.
>> If extended regions could not be allocated for some reason,
>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>
>> Please note the following limitations:
>> - The extended region feature is only supported for 64-bit domain.
>> - The ACPI case is not covered.
>
> I understand the ACPI is not covered because we would need to create a 
> new binding. But I am not sure to understand why 32-bit domain is not 
> supported. Can you explain it?

The 32-bit domain is not supported for simplifying things from the 
beginning. It is a little bit difficult to get everything working at 
start. As I understand from discussion at [1] we can afford that 
simplification. However, I should have mentioned that 32-bit domain is 
not supported "for now".

>
>>
>> ***
>>
>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>> the algorithm to choose extended regions for it is different
>> in comparison with the algorithm for non-direct mapped DomU.
>> What is more, that extended regions should be chosen differently
>> whether IOMMU is enabled or not.
>>
>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>> holes found in host device-tree if otherwise. 
>
> For the case when the IOMMU is disabled, this will only work if dom0 
> cannot allocate memory outside of the original range. This is 
> currently the case... but I think this should be spelled out in at 
> least the commit message.

Agree, will update commit description.


>
>
>> Make sure that
>> extended regions are 2MB-aligned and located within maximum possible
>> addressable physical memory range. The maximum number of extended
>> regions is 128.
>
> Please explain how this limit was chosen.
Well, I decided to not introduce new data struct and etc to represent 
extended regions but reuse existing struct meminfo
used for memory/reserved-memory and, as I though, perfectly fitted. So, 
that limit come from NR_MEM_BANKS which is 128.


>
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> Changes since RFC:
>>     - update patch description
>>     - drop uneeded "extended-region" DT property
>> ---
>>
>>   xen/arch/arm/domain_build.c | 226 
>> +++++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 224 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 206038d..070ec27 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct 
>> domain *d,
>>       return res;
>>   }
>>   +static int __init add_ext_regions(unsigned long s, unsigned long 
>> e, void *data)
>> +{
>> +    struct meminfo *ext_regions = data;
>> +    paddr_t start, size;
>> +
>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>> +        return 0;
>> +
>> +    /* Both start and size of the extended region should be 2MB 
>> aligned */
>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>> +    if ( start > e )
>> +        return 0;
>> +
>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>> +    if ( !size )
>> +        return 0;
>> +
>> +    ext_regions->bank[ext_regions->nr_banks].start = start;
>> +    ext_regions->bank[ext_regions->nr_banks].size = size;
>> +    ext_regions->nr_banks ++;
>> +
>> +    return 0;
>> +}
>> +
>> +/*
>> + * The extended regions will be prevalidated by the memory hotplug path
>> + * in Linux which requires for any added address range to be within 
>> maximum
>> + * possible addressable physical memory range for which the linear 
>> mapping
>> + * could be created.
>> + * For 48-bit VA space size the maximum addressable range are:
>
> When I read "maximum", I understand an upper limit. But below, you are 
> providing a range. So should you drop "maximum"?

yes, it is a little bit confusing.


>
>
> Also, this is tailored to Linux using 48-bit VA. How about other limits?
These limits are calculated at [2]. Sorry, I didn't investigate yet what 
values would be for other CONFIG_ARM64_VA_BITS_XXX. Also looks like some 
configs depend on 16K/64K pages...
I will try to investigate and provide limits later on.


>
>
>> + * 0x40000000 - 0x80003fffffff
>> + */
>> +#define EXT_REGION_START   0x40000000ULL
>
> I am probably missing something here.... There are platform out there 
> with memory starting at 0 (IIRC ZynqMP is one example). So wouldn't 
> this potentially rule out the extended region on such platform?

 From my understanding the extended region cannot be in 0...0x40000000 
range. If these platforms have memory above first GB, I believe the 
extended region(s) can be allocated for them.


>
>
>> +#define EXT_REGION_END 0x80003fffffffULL
>> +
>> +static int __init find_unallocated_memory(const struct kernel_info 
>> *kinfo,
>> +                                          struct meminfo *ext_regions)
>> +{
>> +    const struct meminfo *assign_mem = &kinfo->mem;
>> +    struct rangeset *unalloc_mem;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>
> We technically already know which range of memory is unused. This is 
> pretty much any region in the freelist of the page allocator. So how 
> about walking the freelist instead?

ok, I will investigate the page allocator code (right now I have no 
understanding of how to do that). BTW, I have just grepped "freelist" 
through the code and all page context related appearances are in x86 
code only.

>
> The advantage is we don't need to worry about modifying the function 
> when adding new memory type.
>
> One disavantage is this will not cover *all* the unused memory as this 
> is doing. But I think this is an acceptable downside.
>
>> +
>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>> +
>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>> +    if ( !unalloc_mem )
>> +        return -ENOMEM;
>> +
>> +    /* Start with all available RAM */
>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.mem.bank[i].start;
>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size 
>> - 1;
>> +        res = rangeset_add_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to add: 
>> %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove RAM assigned to Dom0 */
>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>> +    {
>> +        start = assign_mem->bank[i].start;
>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: 
>> %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove reserved-memory regions */
>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.reserved_mem.bank[i].start;
>> +        end = bootinfo.reserved_mem.bank[i].start +
>> +            bootinfo.reserved_mem.bank[i].size - 1;
>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: 
>> %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove grant table region */
>> +    start = kinfo->gnttab_start;
>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>> +                                 add_ext_regions, ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(unalloc_mem);
>> +
>> +    return res;
>> +}
>> +
>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>> +                                    struct meminfo *ext_regions)
>> +{
>> +    struct dt_device_node *np;
>> +    struct rangeset *mem_holes;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>> +
>> +    dt_dprintk("Find memory holes for extended regions\n");
>> +
>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>> +    if ( !mem_holes )
>> +        return -ENOMEM;
>> +
>> +    /* Start with maximum possible addressable physical memory range */
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_add_range(mem_holes, start, end);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    /* Remove all regions described by "reg" property (MMIO, RAM, 
>> etc) */
>
> Well... The loop below is not going to handle all the regions 
> described in the property "reg". Instead, it will cover a subset of 
> "reg" where the memory is addressable.

As I understand, we are only interested in subset of "reg" where the 
memory is addressable.


>
>
> You will also need to cover "ranges" that will describe the BARs for 
> the PCI devices.
Good point. Could you please clarify how to recognize whether it is a 
PCI device as long as PCI support is not merged? Or just to find any 
device nodes with non-empty "ranges" property
and retrieve addresses?


>
>
>> +    dt_for_each_device_node( dt_host, np )
>> +    {
>> +        unsigned int naddr;
>> +        u64 addr, size;
>> +
>> +        naddr = dt_number_of_address(np);
>> +
>> +        for ( i = 0; i < naddr; i++ )
>> +        {
>> +            res = dt_device_get_address(np, i, &addr, &size);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Unable to retrieve address %u for 
>> %s\n",
>> +                       i, dt_node_full_name(np));
>> +                goto out;
>> +            }
>> +
>> +            start = addr & PAGE_MASK;
>> +            end = PAGE_ALIGN(addr + size) - 1;
>> +            res = rangeset_remove_range(mem_holes, start, end);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Failed to remove: 
>> %#"PRIx64"->%#"PRIx64"\n",
>> +                       start, end);
>> +                goto out;
>> +            }
>> +        }
>> +    }
>> +
>> +    start = EXT_REGION_START;
>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>> +    res = rangeset_report_ranges(mem_holes, start, end,
>> +                                 add_ext_regions, ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(mem_holes);
>> +
>> +    return res;
>> +}
>> +
>>   static int __init make_hypervisor_node(struct domain *d,
>>                                          const struct kernel_info 
>> *kinfo,
>>                                          int addrcells, int sizecells)
>> @@ -731,11 +921,13 @@ static int __init make_hypervisor_node(struct 
>> domain *d,
>>       const char compat[] =
>> "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>>           "xen,xen";
>> -    __be32 reg[4];
>> +    __be32 reg[(NR_MEM_BANKS + 1) * 4];
>
> This is a fairly large allocation on the stack. Could we move to a 
> dynamic allocation?

Of course, will do.


>
>
>>       gic_interrupt_t intr;
>>       __be32 *cells;
>>       int res;
>>       void *fdt = kinfo->fdt;
>> +    struct meminfo *ext_regions;
>> +    unsigned int i;
>>         dt_dprintk("Create hypervisor node\n");
>>   @@ -757,12 +949,42 @@ static int __init make_hypervisor_node(struct 
>> domain *d,
>>       if ( res )
>>           return res;
>>   +    ext_regions = xzalloc(struct meminfo);
>> +    if ( !ext_regions )
>> +        return -ENOMEM;
>> +
>> +    if ( is_32bit_domain(d) )
>> +        printk(XENLOG_WARNING "The extended region is only supported 
>> for 64-bit guest\n");
>> +    else
>> +    {
>> +        if ( !is_iommu_enabled(d) )
>> +            res = find_unallocated_memory(kinfo, ext_regions);
>> +        else
>> +            res = find_memory_holes(kinfo, ext_regions);
>> +
>> +        if ( res )
>> +            printk(XENLOG_WARNING "Failed to allocate extended 
>> regions\n");
>> +    }
>> +
>>       /* reg 0 is grant table space */
>>       cells = &reg[0];
>>       dt_child_set_range(&cells, addrcells, sizecells,
>>                          kinfo->gnttab_start, kinfo->gnttab_size);
>> +    /* reg 1...N are extended regions */
>> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
>> +    {
>> +        u64 start = ext_regions->bank[i].start;
>> +        u64 size = ext_regions->bank[i].size;
>> +
>> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
>> +                   i, start, start + size);
>> +
>> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
>> +    }
>> +    xfree(ext_regions);
>> +
>>       res = fdt_property(fdt, "reg", reg,
>> -                       dt_cells_to_size(addrcells + sizecells));
>> +                       dt_cells_to_size(addrcells + sizecells) * (i 
>> + 1));
>>       if ( res )
>>           return res;
>>
>
> Cheers,

[1] 
https://lore.kernel.org/xen-devel/cb1c8fd4-a4c5-c18e-c8db-f8e317d95526@xen.org/

[2] 
https://elixir.bootlin.com/linux/v5.15-rc1/source/arch/arm64/mm/mmu.c#L1448


Thank you.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 20:04:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 20:04:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189688.339535 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRK6P-0004XV-UY; Fri, 17 Sep 2021 20:04:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189688.339535; Fri, 17 Sep 2021 20:04:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRK6P-0004XO-QE; Fri, 17 Sep 2021 20:04:53 +0000
Received: by outflank-mailman (input) for mailman id 189688;
 Fri, 17 Sep 2021 20:04:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRK6O-0004XI-QF
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 20:04:52 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b27a69a9-28b1-4f14-bfe1-52bd1bf787c7;
 Fri, 17 Sep 2021 20:04:52 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 18D11611C3;
 Fri, 17 Sep 2021 20:04:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b27a69a9-28b1-4f14-bfe1-52bd1bf787c7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631909091;
	bh=mEyjE0OtAQmdOBy6JF1uWMiaz+LsaQ2TOO9N0GzD4q8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TwffbNS4RL5k7jgYAk9bPvqGar9gPSXlDk7tGrFRJS6vG73q/79FwFr4xCCJNw66u
	 faPcb3eKXbUXcWv89rMvuYi/3EqZIZqPynZ0WV/pN974Hj6SuS9OvfoEUzia0kGSgw
	 mzsjR/DoMpf7tw4UA3fGaUygmCF//mPQMFR0RSwEkTVgEGfu0fJtHyKA2mJa1p15cL
	 7iBAO8W5stpj5jk+KUlDxzg1MU6JehWVMawg6sskx2DvAncMAwMtsqXL9ioGWwOMYY
	 VslTVrm1j8TiDQ36VTupNVotaWGVYguVOhI/f4EI09U3JMvFkddeor/0PdlCclJ7XV
	 T6eyo4p4Y6/pA==
Date: Fri, 17 Sep 2021 13:04:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    lkml <linux-kernel@vger.kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>, 
    "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Subject: Re: [PATCH v2 1/4] swiotlb-xen: ensure to issue well-formed
 XENMEM_exchange requests
In-Reply-To: <7b3998e3-1233-4e5a-89ec-d740e77eb166@suse.com>
Message-ID: <alpine.DEB.2.21.2109171304380.21985@sstabellini-ThinkPad-T480s>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com> <7b3998e3-1233-4e5a-89ec-d740e77eb166@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 Sep 2021, Jan Beulich wrote:
> While the hypervisor hasn't been enforcing this, we would still better
> avoid issuing requests with GFNs not aligned to the requested order.
> Instead of altering the value also in the call to panic(), drop it
> there for being static and hence easy to determine without being part
> of the panic message.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> I question how useful it is to wrap "bytes" in PAGE_ALIGN(), when it is
> a multiple of a segment's size anyway (or at least was supposed to be,
> prior to "swiotlb-xen: maintain slab count properly"). But that's
> perhaps yet another separate patch.
> ---
> v2: Drop logging of alignment. Wrap lines.
> 
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -230,10 +230,11 @@ retry:
>  	/*
>  	 * Get IO TLB memory from any location.
>  	 */
> -	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
> +	start = memblock_alloc(PAGE_ALIGN(bytes),
> +			       IO_TLB_SEGSIZE << IO_TLB_SHIFT);
>  	if (!start)
> -		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
> -		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
> +		panic("%s: Failed to allocate %lu bytes\n",
> +		      __func__, PAGE_ALIGN(bytes));
>  
>  	/*
>  	 * And replace that memory with pages under 4GB.
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 20:13:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 20:13:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189696.339545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRKEQ-00069c-Mu; Fri, 17 Sep 2021 20:13:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189696.339545; Fri, 17 Sep 2021 20:13:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRKEQ-00069V-K0; Fri, 17 Sep 2021 20:13:10 +0000
Received: by outflank-mailman (input) for mailman id 189696;
 Fri, 17 Sep 2021 20:13:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jmcU=OH=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mRKEO-00069P-Vx
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 20:13:09 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0597f28-8f6b-4610-8999-3754b3e3c759;
 Fri, 17 Sep 2021 20:13:08 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id g1so38073941lfj.12
 for <xen-devel@lists.xenproject.org>; Fri, 17 Sep 2021 13:13:08 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a8sm601181lfr.256.2021.09.17.13.13.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 17 Sep 2021 13:13:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0597f28-8f6b-4610-8999-3754b3e3c759
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=019AdyqpxG1TlQueuJaYYUhEsPxQ+fUHO79eW3US8lw=;
        b=Cgy7/q8/wG1mnhu6ObUXArrEf2g2bgxD9mk/YANBhnVCnfQjKREmyAU0PwSWwx3ApO
         KWwYUD4296/oOc7eCOflRMzou9l4bfBeJ7usgeLBkyP2Hz67TAF50F0C5FhKsVton5/w
         hF2LVI2cj3apQTtvlLUJ+qb2zVKck3ap/+d0xuMxJ+ENyhk+6QT8m0Vz7pqq4UKxjLBP
         iMH+nUi4P3GWMRKvMjgpFjcuxMi6OmRQfTbTXA+oxgWB6DeS///3DtHvBEMNjUI5L+/Z
         9mA65irQAErxtptt4nNf5oSGnYV4A7vd24Q3qaEva53s/i4m+8Kfym1umoDv9b9XJ9WJ
         0Bsw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=019AdyqpxG1TlQueuJaYYUhEsPxQ+fUHO79eW3US8lw=;
        b=qC3j2Z2RmXyIGcRrExAxt25Vy8vbBe4vU6E24shBoVnumoh+R71DO37xYwaAzLk7pn
         AWkgf4GhyV+55LTcjS4uxXESR1Kgcsra64dZYvsjpiLKHtPBIgMhQ2bvp1Dt2hce+b6f
         TZl7p2vUOgMTwWlaDFl1yqQbggRzl//swo52XiX4s41C3dFsI3mVtk0ly84Xk7KsGUl6
         YH0MR59wQs1oII3TxHrvgQIdT/DWdqCkkv5ZNOhAjLXj4xh7Rf+NIPAtWqXwZEE4Tkjk
         zpGGe8TWvfF0RAFW6vTVa1NV6y9Ed3pHLsmj0Y7UUYL/SigeOOUMTqUpfd4inIFWjqky
         W+wg==
X-Gm-Message-State: AOAM530F/ryj9CKGbiAaoIKAbxqVG+xIp8oIS3W7OlHT44W+G6v10W+n
	/LXis78SHMJudsPLM8BJGidmQsWfpYE=
X-Google-Smtp-Source: ABdhPJzpFfah1LH0Y642rq6UCLNivCQkBImFjhBnlsK/CNcAkjyp7Z/bm/hWKF3iJodpVM/YBHhXaA==
X-Received: by 2002:a2e:4e01:: with SMTP id c1mr10691530ljb.460.1631909587018;
        Fri, 17 Sep 2021 13:13:07 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109131726420.10523@sstabellini-ThinkPad-T480s>
 <c3fd1198-15bf-6c05-fe5d-ea29255f1dfe@gmail.com>
 <e832477f-4ed7-4f12-90aa-ebc1843cdc36@xen.org>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <01c8b151-b035-16de-a26a-0fa19efb9c04@gmail.com>
Date: Fri, 17 Sep 2021 23:13:05 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <e832477f-4ed7-4f12-90aa-ebc1843cdc36@xen.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 17.09.21 18:52, Julien Grall wrote:

Hi Julien

>
>
> On 16/09/2021 00:10, Oleksandr wrote:
>>>> + * The extended regions will be prevalidated by the memory hotplug 
>>>> path
>>>> + * in Linux which requires for any added address range to be 
>>>> within maximum
>>>> + * possible addressable physical memory range for which the linear 
>>>> mapping
>>>> + * could be created.
>>>> + * For 48-bit VA space size the maximum addressable range are:
>>>> + * 0x40000000 - 0x80003fffffff
>>> Please don't make Linux-specific comments in Xen code for interfaces
>>> that are supposed to be OS-agnostic.
>>
>> You are right. I just wanted to describe where these magic numbers 
>> come from.
>> Someone might question why, for example, "0 ... max_gpaddr" can't be 
>> used. I will move
>> that Linux-specific comments to the commit message to keep some 
>> justification of these numbers.
>
> Please keep some rationale in the code. This is a lot easier to 
> understand the code without having to play the git blame game.

ok, to be honest I failed to find how to express OS-depended constraints 
in a OS-agnostic way.


>
>
> Cheers,
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Fri Sep 17 21:02:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 21:02:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189710.339563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRKzd-0003ZE-82; Fri, 17 Sep 2021 21:01:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189710.339563; Fri, 17 Sep 2021 21:01:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRKzd-0003Z7-4t; Fri, 17 Sep 2021 21:01:57 +0000
Received: by outflank-mailman (input) for mailman id 189710;
 Fri, 17 Sep 2021 21:01:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRKzc-0003Z1-Ah
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 21:01:56 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7cf1d6f7-17fa-11ec-b72f-12813bfff9fa;
 Fri, 17 Sep 2021 21:01:55 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2158761041;
 Fri, 17 Sep 2021 21:01:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7cf1d6f7-17fa-11ec-b72f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631912514;
	bh=Ndurqa6Butxfy9Dwbs+bYXWInRArT/CnntG/OJC22e4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=YNQsVuLdqg20Wv3H41tUq4bScWLDV31+2M+AHwEOG5TuJ1VKAtFmst8LhW4YZwCSO
	 TLn7iThn/F0Dhunm92S+DbWGxiFalSGpbdSwpb0RP5DkK3jGik30OWaDVG5M2VTbWR
	 XxzgmryWm4Sl8Y+cTX0DruFOFTbiV3P0zXoagpe1W9t95zMPCcx4/zURTO7QNNnVqb
	 5j7vzgan1shzEvnnCnUYKfN3TnmoUszeZBS4eeTknS+J5+cj11lKyjs9oiTN0mucaZ
	 BK6bH8ShdBpGAy5TFcxzFzJ4/I1T6tU+nXSiL+bhRtK4yqxjgi4qgFpjnn7rHsZ9ME
	 41/o4jmBk5FTA==
Date: Fri, 17 Sep 2021 14:01:53 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] page-alloc: further adjust assign_page{,s}()
In-Reply-To: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
Message-ID: <alpine.DEB.2.21.2109171401450.21985@sstabellini-ThinkPad-T480s>
References: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 17 Sep 2021, Jan Beulich wrote:
> The on-commit editing of 5260e8fb93f0 ("xen: re-define assign_pages and
> introduce a new function assign_page") didn't go quite far enough: A
> local variable and a function argument also would have wanted adjusting.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2269,7 +2269,7 @@ int assign_pages(
>      unsigned int memflags)
>  {
>      int rc = 0;
> -    unsigned long i;
> +    unsigned int i;
>  
>      spin_lock(&d->page_alloc_lock);
>  
> @@ -2339,7 +2339,7 @@ int assign_pages(
>  int assign_page(struct page_info *pg, unsigned int order, struct domain *d,
>                  unsigned int memflags)
>  {
> -    return assign_pages(pg, 1UL << order, d, memflags);
> +    return assign_pages(pg, 1U << order, d, memflags);
>  }
>  
>  struct page_info *alloc_domheap_pages(
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 21:46:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 21:46:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189722.339576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRLgI-0008UA-RD; Fri, 17 Sep 2021 21:46:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189722.339576; Fri, 17 Sep 2021 21:46:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRLgI-0008U3-OL; Fri, 17 Sep 2021 21:46:02 +0000
Received: by outflank-mailman (input) for mailman id 189722;
 Fri, 17 Sep 2021 21:46:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRLgH-0008Tx-D4
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 21:46:01 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 26aef2be-12a2-4668-b355-21e255aa3902;
 Fri, 17 Sep 2021 21:45:59 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F1A40611C8;
 Fri, 17 Sep 2021 21:45:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26aef2be-12a2-4668-b355-21e255aa3902
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631915159;
	bh=7zTdpYwxjmexKAmPlX9fm8wT3BGL0kUjqkGEgiCkz2A=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JYBPmjC1/cm/1ZRV7m+XQZIoo05Pb3QqclDN0TyOw9e2fNEV6IjDlBvOe404d0X8f
	 zM/coSleUt5aSXgI+e2Z98BRDCk1qL6hErbequ4v5gbrYr0ESLz4VjjWFeoZZVyduH
	 7whR6SCh0Ayp27B3YueWVUJEsyhf3kvV3bYyPOXvlVaQZEqTJOd/TS7dpPOOYkA9lB
	 Do+pOaPWQFvmL8tJdsU/f44VoAiTz0GBY4ZRk17EKVqwAe+jPFi7HZ1JvGYg0XliU2
	 lOlbVvNpV9nPsPkO8c3rY3YjeLtIHlDhdLEvW8uCHPjTwFiGpAxVbF8+u6PHxTCN5R
	 8/9+jDzQ9GUIw==
Date: Fri, 17 Sep 2021 14:45:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org, 
    sstabellini@kernel.org, jbeulich@suse.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
In-Reply-To: <20210917130123.1764493-1-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
References: <20210917130123.1764493-1-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Hi Oleksandr,

Why do you want to enable pciback on ARM? Is it only to "disable" a PCI
device in Dom0 so that it can be safely assigned to a DomU?

I am asking because actually I don't think we want to enable the PV PCI
backend feature of pciback on ARM, right? That would clash with the PCI
assignment work you have been doing in Xen. They couldn't both work at
the same time.

If we only need pciback to "park" a device in Dom0, wouldn't it be
possible and better to use pci-stub instead?


On Fri, 17 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Xen-pciback driver was designed to be built for x86 only. But it
> can also be used by other architectures, e.g. Arm.
> Re-structure the driver in a way that it can be built for other
> platforms as well.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
> 
> ---
> Tested on Arm and x86.
> ---
>  arch/x86/include/asm/xen/pci.h     | 24 ----------
>  arch/x86/pci/xen.c                 | 74 +----------------------------
>  drivers/xen/Kconfig                |  2 +-
>  drivers/xen/events/events_base.c   |  1 +
>  drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
>  drivers/xen/xen-pciback/pci_stub.c |  3 +-
>  drivers/xen/xen-pciback/xenbus.c   |  2 +-
>  include/xen/pci.h                  | 34 ++++++++++++++
>  8 files changed, 115 insertions(+), 100 deletions(-)
>  create mode 100644 include/xen/pci.h
> 
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
> index 3506d8c598c1..9ff7b49bca08 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -14,30 +14,6 @@ static inline int pci_xen_hvm_init(void)
>  	return -1;
>  }
>  #endif
> -#if defined(CONFIG_XEN_DOM0)
> -int __init pci_xen_initial_domain(void);
> -int xen_find_device_domain_owner(struct pci_dev *dev);
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> -int xen_unregister_device_domain_owner(struct pci_dev *dev);
> -#else
> -static inline int __init pci_xen_initial_domain(void)
> -{
> -	return -1;
> -}
> -static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> -						   uint16_t domain)
> -{
> -	return -1;
> -}
> -static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -#endif
>  
>  #if defined(CONFIG_PCI_MSI)
>  #if defined(CONFIG_PCI_XEN)
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 3d41a09c2c14..4a45b0bf9ae4 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -23,6 +23,7 @@
>  
>  #include <xen/features.h>
>  #include <xen/events.h>
> +#include <xen/pci.h>
>  #include <asm/xen/pci.h>
>  #include <asm/xen/cpuid.h>
>  #include <asm/apic.h>
> @@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
>  	}
>  	return 0;
>  }
> -
> -struct xen_device_domain_owner {
> -	domid_t domain;
> -	struct pci_dev *dev;
> -	struct list_head list;
> -};
> -
> -static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> -static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> -
> -static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	list_for_each_entry(owner, &dev_domain_list, list) {
> -		if (owner->dev == dev)
> -			return owner;
> -	}
> -	return NULL;
> -}
> -
> -int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -	int domain = -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (owner)
> -		domain = owner->domain;
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return domain;
> -}
> -EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> -
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> -	if (!owner)
> -		return -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	if (find_device(dev)) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		kfree(owner);
> -		return -EEXIST;
> -	}
> -	owner->domain = domain;
> -	owner->dev = dev;
> -	list_add_tail(&owner->list, &dev_domain_list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> -
> -int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (!owner) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		return -ENODEV;
> -	}
> -	list_del(&owner->list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	kfree(owner);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
>  #endif
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index a37eb52fb401..057ddf61ef61 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -182,7 +182,7 @@ config SWIOTLB_XEN
>  
>  config XEN_PCIDEV_BACKEND
>  	tristate "Xen PCI-device backend driver"
> -	depends on PCI && X86 && XEN
> +	depends on PCI && XEN
>  	depends on XEN_BACKEND
>  	default m
>  	help
> diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
> index a78704ae3618..35493ff0d146 100644
> --- a/drivers/xen/events/events_base.c
> +++ b/drivers/xen/events/events_base.c
> @@ -65,6 +65,7 @@
>  #include <xen/interface/vcpu.h>
>  #include <xen/xenbus.h>
>  #include <asm/hw_irq.h>
> +#include <xen/pci.h>
>  
>  #include "events_internal.h"
>  
> diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
> index 224df03ce42e..fc8c1249d49f 100644
> --- a/drivers/xen/pci.c
> +++ b/drivers/xen/pci.c
> @@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
>  	return 0;
>  }
>  #endif
> +
> +#ifdef CONFIG_XEN_DOM0
> +struct xen_device_domain_owner {
> +	domid_t domain;
> +	struct pci_dev *dev;
> +	struct list_head list;
> +};
> +
> +static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> +static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> +
> +static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	list_for_each_entry(owner, &dev_domain_list, list) {
> +		if (owner->dev == dev)
> +			return owner;
> +	}
> +	return NULL;
> +}
> +
> +int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +	int domain = -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (owner)
> +		domain = owner->domain;
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return domain;
> +}
> +EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> +
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> +	if (!owner)
> +		return -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	if (find_device(dev)) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		kfree(owner);
> +		return -EEXIST;
> +	}
> +	owner->domain = domain;
> +	owner->dev = dev;
> +	list_add_tail(&owner->list, &dev_domain_list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> +
> +int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (!owner) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		return -ENODEV;
> +	}
> +	list_del(&owner->list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	kfree(owner);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
> +#endif
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index f8e4faa96ad6..bba527620507 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -19,7 +19,8 @@
>  #include <linux/sched.h>
>  #include <linux/atomic.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
> +#include <xen/xen.h>
>  #include <asm/xen/hypervisor.h>
>  #include <xen/interface/physdev.h>
>  #include "pciback.h"
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index c09c7ebd6968..da34ce85dc88 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -14,7 +14,7 @@
>  #include <linux/workqueue.h>
>  #include <xen/xenbus.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
>  #include "pciback.h"
>  
>  #define INVALID_EVTCHN_IRQ  (-1)
> diff --git a/include/xen/pci.h b/include/xen/pci.h
> new file mode 100644
> index 000000000000..5c988d5ff38f
> --- /dev/null
> +++ b/include/xen/pci.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __XEN_PCI_H__
> +#define __XEN_PCI_H__
> +
> +#if defined(CONFIG_XEN_DOM0)
> +int __init pci_xen_initial_domain(void);
> +int xen_find_device_domain_owner(struct pci_dev *dev);
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> +int xen_unregister_device_domain_owner(struct pci_dev *dev);
> +#else
> +static inline int __init pci_xen_initial_domain(void)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> +						   uint16_t domain)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +#endif
> +
> +#endif
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 21:56:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 21:56:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189732.339594 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRLqK-0001j1-TM; Fri, 17 Sep 2021 21:56:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189732.339594; Fri, 17 Sep 2021 21:56:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRLqK-0001iu-PQ; Fri, 17 Sep 2021 21:56:24 +0000
Received: by outflank-mailman (input) for mailman id 189732;
 Fri, 17 Sep 2021 21:56:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRLqK-0001io-2S
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 21:56:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 079dac70-fbc7-4fc8-9b70-80c28e69822c;
 Fri, 17 Sep 2021 21:56:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id DE41861212;
 Fri, 17 Sep 2021 21:56:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 079dac70-fbc7-4fc8-9b70-80c28e69822c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631915782;
	bh=DXFEsf7Ah+ri2OUA8b05bEelNaJwBed7q8ydY0juMac=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rrtw+guji0Ese/PQlHh+TPTc0aMxDi9Q9NLWgwL6/SIHcmXRvZx6Oc2qsfSNWPBjv
	 cA1tDMfZ8riW7fANve1OPIjMprVaxVrZO1o58jQzpo/fg55M76R4cd3/hI2f2rGivN
	 id0DyVdFGD8qSmBmkT7FlpzlcYjo5918HeHBjFeGJzyVIItKMEazakLvrmbVrTSmBB
	 RMQGWcAFXN5D0KjwOo+hYFRyYLpkOPxpYE56Go4fMvea804TXnrZP2IJwNEpnsfcBi
	 Zl0kJoWZM2OLFG1FvgcHcz1RYtxPSKNoxQ9X1YQE0VSr4V9aj0afYBs0fr/f/5QOLT
	 75yCcPqZN3zFQ==
Date: Fri, 17 Sep 2021 14:56:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
Message-ID: <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-47340259-1631915598=:21985"
Content-ID: <alpine.DEB.2.21.2109171453231.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-47340259-1631915598=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109171453232.21985@sstabellini-ThinkPad-T480s>

On Fri, 17 Sep 2021, Oleksandr wrote:
> > > +
> > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > +
> > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > +    if ( !unalloc_mem )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with all available RAM */
> > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.mem.bank[i].start;
> > > +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
> > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove RAM assigned to Dom0 */
> > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > +    {
> > > +        start = assign_mem->bank[i].start;
> > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove reserved-memory regions */
> > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove grant table region */
> > > +    start = kinfo->gnttab_start;
> > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    start = EXT_REGION_START;
> > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > +                                 add_ext_regions, ext_regions);
> > > +    if ( res )
> > > +        ext_regions->nr_banks = 0;
> > > +    else if ( !ext_regions->nr_banks )
> > > +        res = -ENOENT;
> > > +
> > > +out:
> > > +    rangeset_destroy(unalloc_mem);
> > > +
> > > +    return res;
> > > +}
> > > +
> > > +static int __init find_memory_holes(const struct kernel_info *kinfo,
> > > +                                    struct meminfo *ext_regions)
> > > +{
> > > +    struct dt_device_node *np;
> > > +    struct rangeset *mem_holes;
> > > +    paddr_t start, end;
> > > +    unsigned int i;
> > > +    int res;
> > > +
> > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > +
> > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > +    if ( !mem_holes )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with maximum possible addressable physical memory range */
> > > +    start = EXT_REGION_START;
> > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > +    res = rangeset_add_range(mem_holes, start, end);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
> > 
> > Well... The loop below is not going to handle all the regions described in
> > the property "reg". Instead, it will cover a subset of "reg" where the
> > memory is addressable.
> 
> As I understand, we are only interested in subset of "reg" where the memory is
> addressable.
> 
> 
> > 
> > 
> > You will also need to cover "ranges" that will describe the BARs for the PCI
> > devices.
> Good point.

Yes, very good point!


> Could you please clarify how to recognize whether it is a PCI
> device as long as PCI support is not merged? Or just to find any device nodes
> with non-empty "ranges" property
> and retrieve addresses?

Normally any bus can have a ranges property with the aperture and
possible address translations, including /amba (compatible =
"simple-bus"). However, in these cases dt_device_get_address already
takes care of it, see xen/common/device_tree.c:dt_device_get_address.

The PCI bus is special for 2 reasons:
- the ranges property has a different format
- the bus is hot-pluggable

So I think the only one that we need to treat specially is PCI.

As far as I am aware PCI is the only bus (or maybe just the only bus
that we support?) where ranges means the aperture.
--8323329-47340259-1631915598=:21985--


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 22:33:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 22:33:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189739.339604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRMQ6-0006TS-PV; Fri, 17 Sep 2021 22:33:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189739.339604; Fri, 17 Sep 2021 22:33:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRMQ6-0006TL-MJ; Fri, 17 Sep 2021 22:33:22 +0000
Received: by outflank-mailman (input) for mailman id 189739;
 Fri, 17 Sep 2021 22:33:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRMQ5-0006TD-DU
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 22:33:21 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da4fecf4-eb70-4bb1-9217-4ed0e6ee9673;
 Fri, 17 Sep 2021 22:33:19 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id ED8186112E;
 Fri, 17 Sep 2021 22:33:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da4fecf4-eb70-4bb1-9217-4ed0e6ee9673
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631917997;
	bh=hwjaTeuJO6RsTxtk9kcnXZVC5SKxYYbdPQCB8BSkpoc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=LPvhv4qXBlnp4x1lLUb4FQfEfIDBVGCYN5sZtOODcv6yy1yugBS1WRZbQ2q10s0ca
	 O80D1Do68zr69RlOY3jln9OHMIRNcDJZc88o4Chy7SNv/kJ6rrQu6r1xboK6dGWw/u
	 +k3Mt9Um5fXgwbeucbyR1ENeJ2B9lqpIdbK1vGpm6mengr9rcXnNpzSnv2xvyEoYgI
	 3cFGR+ShQ7xq+pXyed4mTpMdcoTdcnirfx4Rwb6k6xWML4N33X4iW/0n+UqnyzT15L
	 uKrD/DBMVWWKgDZ24qIZkFwoMiEshVObqq2j+OdoXsYdeAom4GrAadmywSeqqjcPyU
	 40Qy5s1/iuGQQ==
Date: Fri, 17 Sep 2021 15:33:15 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
In-Reply-To: <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com>
Message-ID: <alpine.DEB.2.21.2109171516270.21985@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-3-luca.fancellu@arm.com> <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com> <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com> <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com> <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com> <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s> <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-264986810-1631917997=:21985"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-264986810-1631917997=:21985
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 17 Sep 2021, Luca Fancellu wrote:
> > On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 16 Sep 2021, Jan Beulich wrote:
> >> On 16.09.2021 17:07, Luca Fancellu wrote:
> >>> I explain here my understanding on dom0less, this feature is used to start domUs at
> >>> Xen boot in parallel, the name is misleading but it doesn’t require dom0 to be absent.
> >>> 
> >>> So if you have a dom0 kernel embed in the image, it's completely fine to start it and it 
> >>> doesn’t have to be skipped.
> >>> 
> >>> Here the possible user cases:
> >>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
> >>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded in Xen image, domUs on DT]
> >>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image) and domUs on DT]
> >> 
> >> If that's the intention - fine. Stefano?
> > 
> 
> Hi Stefano,
> 
> > What do you mean by dom0 modules embedded in the Xen image? I am not
> > familiar with it, but I imagine it doesn't involve any multiboot,module
> > nodes in device tree, right?
> > 
> > Putting aside "dom0 modules embedded in Xen image" that I didn't fully
> > understand, there are three ways to load Dom0:
> > 
> > - dom0 kernel (and ramdisk, optional) on xen.cfg
> > - dom0 kernel (and ramdisk, optional) on device tree using the "reg" property
> > - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,binary" property
> 
> True for the #1 and #2, the last one is not implemented. The uefi,binary property
> for now is only used to load domU modules.

Yeah, it is no problem that is not currently implemented, but from a
device tree binding / efi interface perspective it should be possible.

 
> > Then, the other use cases are:
> > 
> > - true dom0less, domUs on device tree using the "reg" property
> > - true dom0less, domUs on device tree using the "uefi,binary" property
> > 
> > And of course all the possible combinations between Dom0 and DomU
> > loading.
> > 
> > 
> > Currently, patch #1 checks for the presence of a Dom0 kernel node and, if
> > present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
> > node is *not* present, efi_arch_use_config_file returns true.
> > 
> > However, this could be a true dom0less configuration without any Dom0
> > kernel. If so, you might not want to load xen.cfg at all because it is
> > not needed. In a true dom0less configuration, we probably want
> > efi_arch_use_config_file to return false.
> 
> In a true dom0less configuration we might need to read xen.cfg to retrieve the
> Xen command line, 

The Xen command line could also be on device tree
(/chosen/xen,xen-bootargs).


> but following the actual implementation of the common code
> there is more. I’m going to explain.
> 
> What efi_arch_use_config_file really does is not only choosing to read xen.cfg
> or not. Following the common code (xen/common/efi/boot.c) and what its result activate
> along the path, it basically decides if the UEFI stub is used as a loader from filesystem
> or not. We need the UEFI stub as a loader to be 100% UEFI and load our modules.
>
> The original check basically says “if there are multiboot,module in the DT, then some
> bootloader has loaded in memory the required modules so I’m not gonna load anything
> from the filesystem because I assume it puts everything in place for me to boot.”

OK, I am following. It looks like this is the source of the issue.

 
> >From misc/efi.txt:
> When booted as an EFI application, Xen requires a configuration file as described below unless a bootloader,
> such as GRUB, has loaded the modules and describes them in the device tree provided to Xen. If a bootloader
> provides a device tree containing modules then any configuration files are ignored, and the bootloader is
> responsible for populating all relevant device tree nodes.
> 
> What I’m doing in patch #1 is restricting that check to just the multiboot,module that are
> Dom0 module, why? Because with the introduction of dom0less we need to specify
> multiboot,modules for domUs, but the presence or not of dom0 modules is the only
> Information we need to understand if the user decided to start Xen with everything
> in places (modules in memory, xen command line, dtb) or if the job is demanded to the
> UEFI stub and its configuration file.

I don't think so. Imagine a case where the user has everything in device
tree, doesn't need xen.cfg, but dom0 and domUs are specified as
uefi,binary.

We don't want xen.cfg but we do want to be able to load files from the
filesystem. This might not be currently implemented but from an bindings
perspective it should be possible.


> By the configuration file you can also load in memory the Xen dtb, so Xen can
> be started as an EFI application without the DTB and then load it using the EFI stub.

This can be very useful but it would follow the !fdt check and return
true from efi_arch_use_config_file. So it doesn't really conflict with
anything we would around multiboot,module and xen,cfg-loading.


> I’m not against this new property “xen,cfg-loading”, I just think it is not needed because
> we have all the information we need without it and in any case we need to read the
> configuration file because otherwise we won’t have access to the Xen command line.

We don't necessarely need to read the Xen command line from xen.cfg :-)


> Now I’m going to show you examples of all use cases that are valid with the introduction
> of this serie:
> 
> 1) Start Xen as EFI application and load only Dom0
> 
>     Xen.cfg:
>     [global]
>     default=xen_dom0
> 
>     [xen_dom0]
>     options=<Xen command line>
>     kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>     ramdisk=initrd-3.0.31-0.4-xen
>     dtb=<xen DTB>
> 
>     DT:
>     {no modification}
> 
> 2) Start Xen as EFI application to load a true dom0less setup
> 
>     Xen.cfg:
>     [global]
>     default=xen_true_dom0less
> 
>     [xen_true_dom0less]
>     options=<Xen command line>
>     dtb=<xen DTB>
> 
>     DT:
>     chosen {
>         #size-cells = <0x1>;
> 	#address-cells = <0x1>;
> 
> 	domU1 {
>             #size-cells = <0x1>; 
>             #address-cells = <0x1>;
>             compatible = "xen,domain”;
>             cpus = <1>;
>             memory = <0 0xC0000>;
>            
>             module@1 {
>                 compatible = "multiboot,kernel", "multiboot,module”;
>                 bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
>                 uefi,binary = “domU_kernel1.bin"
>             };
> 
>             module@2 {
>                 compatible = “multiboot,ramdisk", "multiboot,module”;
>                 uefi,binary = “domU_ramdisk1.bin"
>             };
> 
>             module@3 {
>                 compatible = "multiboot,device-tree", "multiboot,module”;
>                 uefi,binary = “domU_passthrough1.dtb"
>             }; 
>         };
>         
>         domU2 { <as above> };
>     }
> 
> 3) Start Xen as EFI application to load Dom0 and DomUs
> 
>     Xen.cfg:
>     [global]
>     default=xen_dom0_domUs
> 
>     [xen_dom0_domUs]
>     options=<Xen command line>
>     kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>     ramdisk=initrd-3.0.31-0.4-xen
>     dtb=<xen DTB>
> 
>     DT:
>     chosen {
>         #size-cells = <0x1>;
> 	#address-cells = <0x1>;
> 
> 	domU1 {
>             #size-cells = <0x1>; 
>             #address-cells = <0x1>;
>             compatible = "xen,domain”;
>             cpus = <1>;
>             memory = <0 0xC0000>;
>            
>             module@1 {
>                 compatible = "multiboot,kernel", "multiboot,module”;
>                 bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
>                 uefi,binary = “domU_kernel1.bin"
>             };
> 
>             module@2 {
>                 compatible = “multiboot,ramdisk", "multiboot,module”;
>                 uefi,binary = “domU_ramdisk1.bin"
>             };
> 
>             module@3 {
>                 compatible = "multiboot,device-tree", "multiboot,module”;
>                 uefi,binary = “domU_passthrough1.dtb"
>             }; 
>         };
>         
>         domU2 { <as above> };
>     }
> 
> So as you see every case is covered without the introduction of the
> property.
> 
> Please let me know what do you think.

I think that from an interface perspective (not a code perspective) we
need to be able to account for cases like:

4) Start Xen as EFI application and load only Dom0
no Xen.cfg
DT:
  xen,xen-bootargs
  dom0/uefi,binary
  domUs/uefi,binary


But in any case, even disregarding this case, past experience has taught
me that it is always better to have an explicit flag to trigger a new
behavior, rather than relying on "guesswork". If we introduce
"xen,cfg-loading", we are going to be a lot more future-proof that if we
don't introduce it in terms of backward and forward compatibility in
case we need to change anything.
--8323329-264986810-1631917997=:21985--


From xen-devel-bounces@lists.xenproject.org Fri Sep 17 22:37:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 17 Sep 2021 22:37:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189747.339615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRMUC-0007Ol-EH; Fri, 17 Sep 2021 22:37:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189747.339615; Fri, 17 Sep 2021 22:37:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRMUC-0007Oe-BO; Fri, 17 Sep 2021 22:37:36 +0000
Received: by outflank-mailman (input) for mailman id 189747;
 Fri, 17 Sep 2021 22:37:35 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+WA1=OH=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRMUB-0007OY-Cp
 for xen-devel@lists.xenproject.org; Fri, 17 Sep 2021 22:37:35 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d984e266-1807-11ec-b73c-12813bfff9fa;
 Fri, 17 Sep 2021 22:37:33 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E0C5E6113A;
 Fri, 17 Sep 2021 22:37:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d984e266-1807-11ec-b73c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631918253;
	bh=CUjCHVm6pyMaK2hZ3SXWr+2ugHPouhVVfUKVTVrvP5U=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=q7yJwAwKUf2+L13Pr3Vkl2hCyGavmc2nMDs9Am/eMhZdIf7g+FhlPwMao9PNnagWh
	 jWYLNc9F/0jMJfnVzj5KrlR812Wf5YBr/9b3CO9mwjsY2U3QZL/NVGJHudC8VgIRmY
	 JQHNaDZQnyvuYVgR/UpfJshJvUORRkvTiQgi5g1QQKQoW9udKdXI8YaS6OtiKbSYnT
	 AKtP45ZsdwX2vImqZu17P6qYCz/pN71YMHNueBHf4iayTafC9omFoFderbkCBhvWjD
	 o5OLBgTr2OnUmePn2baXjJJtlO3X2kS+MqwRryUcIvGJ2ZRDPIJD2xkIO1Pu/ndFQs
	 aTQpdI89dJLwA==
Date: Fri, 17 Sep 2021 15:37:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Oleksandr <olekstysh@gmail.com>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1207203985-1631916030=:21985"
Content-ID: <alpine.DEB.2.21.2109171500460.21985@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1207203985-1631916030=:21985
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109171500461.21985@sstabellini-ThinkPad-T480s>

On Fri, 17 Sep 2021, Stefano Stabellini wrote:
> On Fri, 17 Sep 2021, Oleksandr wrote:
> > > > +
> > > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > > +
> > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > +    if ( !unalloc_mem )
> > > > +        return -ENOMEM;
> > > > +
> > > > +    /* Start with all available RAM */
> > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > +    {
> > > > +        start = bootinfo.mem.bank[i].start;
> > > > +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
> > > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > > +        if ( res )
> > > > +        {
> > > > +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > > +                   start, end);
> > > > +            goto out;
> > > > +        }
> > > > +    }
> > > > +
> > > > +    /* Remove RAM assigned to Dom0 */
> > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > +    {
> > > > +        start = assign_mem->bank[i].start;
> > > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
> > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > +        if ( res )
> > > > +        {
> > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > +                   start, end);
> > > > +            goto out;
> > > > +        }
> > > > +    }
> > > > +
> > > > +    /* Remove reserved-memory regions */
> > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > > +    {
> > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > +        if ( res )
> > > > +        {
> > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > +                   start, end);
> > > > +            goto out;
> > > > +        }
> > > > +    }
> > > > +
> > > > +    /* Remove grant table region */
> > > > +    start = kinfo->gnttab_start;
> > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > +    if ( res )
> > > > +    {
> > > > +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > > > +               start, end);
> > > > +        goto out;
> > > > +    }
> > > > +
> > > > +    start = EXT_REGION_START;
> > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > +                                 add_ext_regions, ext_regions);
> > > > +    if ( res )
> > > > +        ext_regions->nr_banks = 0;
> > > > +    else if ( !ext_regions->nr_banks )
> > > > +        res = -ENOENT;
> > > > +
> > > > +out:
> > > > +    rangeset_destroy(unalloc_mem);
> > > > +
> > > > +    return res;
> > > > +}
> > > > +
> > > > +static int __init find_memory_holes(const struct kernel_info *kinfo,
> > > > +                                    struct meminfo *ext_regions)
> > > > +{
> > > > +    struct dt_device_node *np;
> > > > +    struct rangeset *mem_holes;
> > > > +    paddr_t start, end;
> > > > +    unsigned int i;
> > > > +    int res;
> > > > +
> > > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > > +
> > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > +    if ( !mem_holes )
> > > > +        return -ENOMEM;
> > > > +
> > > > +    /* Start with maximum possible addressable physical memory range */
> > > > +    start = EXT_REGION_START;
> > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > +    if ( res )
> > > > +    {
> > > > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > > +               start, end);
> > > > +        goto out;
> > > > +    }
> > > > +
> > > > +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
> > > 
> > > Well... The loop below is not going to handle all the regions described in
> > > the property "reg". Instead, it will cover a subset of "reg" where the
> > > memory is addressable.
> > 
> > As I understand, we are only interested in subset of "reg" where the memory is
> > addressable.
> > 
> > 
> > > 
> > > 
> > > You will also need to cover "ranges" that will describe the BARs for the PCI
> > > devices.
> > Good point.
> 
> Yes, very good point!
> 
> 
> > Could you please clarify how to recognize whether it is a PCI
> > device as long as PCI support is not merged? Or just to find any device nodes
> > with non-empty "ranges" property
> > and retrieve addresses?
> 
> Normally any bus can have a ranges property with the aperture and
> possible address translations, including /amba (compatible =
> "simple-bus"). However, in these cases dt_device_get_address already
> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
> 
> The PCI bus is special for 2 reasons:
> - the ranges property has a different format
> - the bus is hot-pluggable
> 
> So I think the only one that we need to treat specially is PCI.
> 
> As far as I am aware PCI is the only bus (or maybe just the only bus
> that we support?) where ranges means the aperture.

Now that I think about this, there is another "hotpluggable" scenario we
need to think about:

[1] https://marc.info/?l=xen-devel&m=163056546214978

Xilinx devices have FPGA regions with apertures currently not described
in device tree, where things can programmed in PL at runtime making new
devices appear with new MMIO regions out of thin air.

Now let me start by saying that yes, the entire programmable region
aperture could probably be described in device tree, however, in
reality it is not currently done in any of the device trees we use
(including the upstream device trees in linux.git).

So, we have a problem :-(


I can work toward getting the right info on device tree, but in reality
that is going to take time and for now the device tree doesn't have the
FPGA aperture in it. So if we accept this series as is, it is going to
stop features like [1] from working.

If we cannot come up with any better plans, I think it would be better
to drop find_memory_holes, only rely on find_unallocated_memory even
when the IOMMU is on. One idea is that we could add on top of the
regions found by find_unallocated_memory any MMIO regions marked as
xen,passthrough: they are safe because they are not going to dom0 anyway.

The only alternative I can think of is to have a per-board
enable/disable toggle for the extend region but it would be very ugly.
--8323329-1207203985-1631916030=:21985--


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 00:06:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 00:06:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189759.339633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRNrw-0000xA-PK; Sat, 18 Sep 2021 00:06:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189759.339633; Sat, 18 Sep 2021 00:06:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRNrw-0000x3-M3; Sat, 18 Sep 2021 00:06:12 +0000
Received: by outflank-mailman (input) for mailman id 189759;
 Sat, 18 Sep 2021 00:06:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EgpN=OI=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mRNrv-0000wx-5A
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 00:06:11 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 39fec4b6-1814-11ec-b742-12813bfff9fa;
 Sat, 18 Sep 2021 00:06:09 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 92C1161164;
 Sat, 18 Sep 2021 00:06:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39fec4b6-1814-11ec-b742-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1631923569;
	bh=NPLabLNTVNuhRxRzuKO9eoANfe1U0WU4IO68gQjGKoc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ENzvoWYvZVw7btIl8cwgL86MPVhZJTMgHFsQdkm0JRZ4c368Ts33vuLbaM6okoszg
	 whwMfy+wSpCMXL3ZZwnWGuuPqsprYZZ3WtgL8siDrz7gpURnEkq9lLWIK5U5bwvHbI
	 gBHefva0sbQ5cZpO5Abqhc+EcmUrIdJcjbO/UDjBS48OnBP4cLSc0EZcPom8KxMVm8
	 snYh3snzyGyfhj7fCNF0ZWRbzdZnXIoWIInHtqGfOjdkEzjZWxPyDFp+TSzj3y51AS
	 T8GieKeJ5FtXf0OpD1JIECu++GOPHBkWv/FCaEghI1UvhJMgTjQFy/o2/Z7voxhua5
	 WF1sC/7wUR55Q==
Date: Fri, 17 Sep 2021 17:06:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
In-Reply-To: <CDC5AD61-D71B-46BC-952B-DE592879F9F3@arm.com>
Message-ID: <alpine.DEB.2.21.2109171630460.21985@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-3-luca.fancellu@arm.com> <alpine.DEB.2.21.2109151721210.21985@sstabellini-ThinkPad-T480s> <CDC5AD61-D71B-46BC-952B-DE592879F9F3@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 16 Sep 2021, Luca Fancellu wrote:
> > On 16 Sep 2021, at 02:16, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Wed, 15 Sep 2021, Luca Fancellu wrote:
> >> This patch introduces the support for dom0less configuration
> >> when using UEFI boot on ARM, it permits the EFI boot to
> >> continue if no dom0 kernel is specified but at least one domU
> >> is found.
> >> 
> >> Introduce the new property "uefi,binary" for device tree boot
> >> module nodes that are subnode of "xen,domain" compatible nodes.
> >> The property holds a string containing the file name of the
> >> binary that shall be loaded by the uefi loader from the filesystem.
> >> 
> >> Update efi documentation about how to start a dom0less
> >> setup using UEFI
> >> 
> >> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> >> ---
> >> docs/misc/efi.pandoc        |  37 ++++++
> >> xen/arch/arm/efi/efi-boot.h | 244 +++++++++++++++++++++++++++++++++++-
> >> xen/common/efi/boot.c       |  20 ++-
> >> 3 files changed, 294 insertions(+), 7 deletions(-)
> >> 
> >> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> >> index ac3cd58cae..db9b3273f8 100644
> >> --- a/docs/misc/efi.pandoc
> >> +++ b/docs/misc/efi.pandoc
> >> @@ -165,3 +165,40 @@ sbsign \
> >> 	--output xen.signed.efi \
> >> 	xen.unified.efi
> >> ```
> >> +
> >> +## UEFI boot and dom0less on ARM
> >> +
> >> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> >> +started as an EFI application.
> >> +The way to specify the domU domains is by Device Tree as specified in the
> >> +[dom0less](dom0less.html) documentation page under the "Device Tree
> >> +configuration" section, but instead of declaring the reg property in the boot
> >> +module, the user must specify the "uefi,binary" property containing the name
> >> +of the binary file that has to be loaded in memory.
> >> +The UEFI stub will load the binary in memory and it will add the reg property
> >> +accordingly.
> >> +
> >> +An example here:
> >> +
> >> +    domU1 {
> >> +        #address-cells = <1>;
> >> +        #size-cells = <1>;
> >> +        compatible = "xen,domain";
> >> +        memory = <0 0x20000>;
> >> +        cpus = <1>;
> >> +        vpl011;
> >> +
> >> +        module@1 {
> >> +            compatible = "multiboot,kernel", "multiboot,module";
> >> +            uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> >> +            bootargs = "console=ttyAMA0";
> >> +        };
> >> +        module@2 {
> >> +            compatible = "multiboot,ramdisk", "multiboot,module";
> >> +            uefi,binary = "initrd-3.0.31-0.4-xen";
> >> +        };
> >> +        module@3 {
> >> +            compatible = "multiboot,ramdisk", "multiboot,module";
> >> +            uefi,binary = "passthrough.dtb";
> >> +        };
> >> +    };
> > 
> > Can you please also update docs/misc/arm/device-tree/booting.txt ?
> > Either a link to docs/misc/efi.pandoc or a definition of the uefi,binary
> > property (mentioning that it is EFI-only.)
> 
> Yes I will update it.
> 
> > 
> > 
> >> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> >> index 5ff626c6a0..8d7ced70f2 100644
> >> --- a/xen/arch/arm/efi/efi-boot.h
> >> +++ b/xen/arch/arm/efi/efi-boot.h
> >> @@ -8,9 +8,39 @@
> >> #include <asm/setup.h>
> >> #include <asm/smp.h>
> >> 
> >> +typedef struct {
> >> +    char* name;
> >> +    int name_len;
> >> +} dom0less_module_name;
> >> +
> >> +/*
> >> + * Binaries will be translated into bootmodules, the maximum number for them is
> >> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> >> + */
> >> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> >> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
> >> +static dom0less_module_name __initdata dom0less_bin_names[MAX_DOM0LESS_MODULES];
> > 
> > I suggest a slightly different model where we don't call AllocatePool to
> > allocate dom0less_module_name.name and instead we just set the pointer
> > directly to the fdt string. There is no risk of the fdt going away at
> > this point so it should be safe to use.
> 
> Yes I thought about this approach but since I was not sure how the DTB behaves when we modify
> It to add the reg property or to modify the module name, then I used this other approach.
> Are you sure that the pointed memory will stay the same after we modify the DTB? My main concern
> was that the DTB structure was going to be modified and the string I was pointing in the DTB memory
> can be relocated elsewhere. 

You are right: fdt_set_name and fdt_set_reg can cause a memmove to be
called, which might change the pointers. Which means we cannot simply
set the char* pointer to the device tree string as it might change.
That's unfortunate. For the lack of a better suggestion, go ahead and
keep AllocatePool/FreePool for the next version.


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 00:48:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 00:48:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189765.339643 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROWp-0005up-3V; Sat, 18 Sep 2021 00:48:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189765.339643; Sat, 18 Sep 2021 00:48:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROWp-0005ui-0K; Sat, 18 Sep 2021 00:48:27 +0000
Received: by outflank-mailman (input) for mailman id 189765;
 Sat, 18 Sep 2021 00:48:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Orqi=OI=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mROWn-0005uc-IA
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 00:48:25 +0000
Received: from mga03.intel.com (unknown [134.134.136.65])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2003e429-181a-11ec-b745-12813bfff9fa;
 Sat, 18 Sep 2021 00:48:23 +0000 (UTC)
Received: from orsmga007.jf.intel.com ([10.7.209.58])
 by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 17 Sep 2021 17:48:22 -0700
Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19])
 by orsmga007.jf.intel.com with ESMTP; 17 Sep 2021 17:48:22 -0700
Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by
 ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Fri, 17 Sep 2021 17:48:21 -0700
Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by
 ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Fri, 17 Sep 2021 17:48:21 -0700
Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by
 orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Fri, 17 Sep 2021 17:48:21 -0700
Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.107)
 by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Fri, 17 Sep 2021 17:48:21 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR11MB3891.namprd11.prod.outlook.com (2603:10b6:405:77::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Sat, 18 Sep
 2021 00:48:20 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.018; Sat, 18 Sep 2021
 00:48:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2003e429-181a-11ec-b745-12813bfff9fa
X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222948526"
X-IronPort-AV: E=Sophos;i="5.85,302,1624345200"; 
   d="scan'208";a="222948526"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,302,1624345200"; 
   d="scan'208";a="473283567"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iErusLYViDWrV5jNPUQBhEyt/eXwhaqR3SBsP9FXj9ISFsYfr5SrRJDljPDzIoHZprimXvrPwYYr/lUc4nmEBNeJs6baN1VLKbbnkTbJyr5HnFR0fA9YUiyey3G6jluJGDWRbTbeginNmwh3xXiTFskHjrUs8kzTpfepkICgOI/cdu3WCsCsUuN6QXukeIiMgoJMbjkYWXuqO0ZrTi1ZFp0qCUhJvwSvyWZEjPg7X0N3YbX1zmrKGiuLkkbvh7gCkj74piXDnml9rwYhsW4lswwBwNYKPnxoAanaGeKKjtiFdLo2JdrqIEnsH/KtxW+FfZ9kblmHz4LzFIPhwjae+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=q4izqWtMOYHvwE5erlAuG4n3VcCpUQNhbN5FnLO8ILk=;
 b=QM2PSjgNYuw0mJ4+SVuHv4Ssk1xdQhnzZjgvCQU0CltN5x95eOmHEDa+iSktua/nO+p+gTHTymNa4clX29moAM0dm9X3cvqyjZxv3+k3tJQ37nm+ehEGAN/6oEPPhrhh56NYfzi/gYKZROsPTBD2GQbbEfzoBasd65G/p9YAKOLhikoPnAQqEwcs1jBZRsdd+sOll1vM0j6rtsSo32d6rGoho4m+vSYSsmyi3Efs0Hp8699IZBaSHKHyruO9DHJk8WTh7a35K22W9LTT3S79JDP2vsHA6HEYabVHeoEf8WUARzvVYQLqodDZM8nwrnuyFOZ0AQblRQCPy50iS1QE7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=q4izqWtMOYHvwE5erlAuG4n3VcCpUQNhbN5FnLO8ILk=;
 b=iBICIqIkhMCmCdT1uk9gQngfv88t31RTc+PhfD32xmQWvfhk9ljkfpEbx8vqS+5T1fv4xZH+cfhXNURSWwk1UmGLn/xfTqN1+mXPf5Fikz/57ay1Csv7HXu5gjsZ/gPBdkCKoufmRxXdNeK4ExA8oKCHyWu09YYXQV0T6i7bszw=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Paul Durrant <paul@xen.org>
Subject: RE: [PATCH v2 1/3] VT-d: consider hidden devices when unmapping
Thread-Topic: [PATCH v2 1/3] VT-d: consider hidden devices when unmapping
Thread-Index: AQHXq7NAuWJs+9yJ2ECO4xKe2sffuauo9kZw
Date: Sat, 18 Sep 2021 00:48:20 +0000
Message-ID: <BN9PR11MB54331945BAE8A87353DED1AE8CDE9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
 <64c6b3e8-045b-9308-b77f-721c3d57c08b@suse.com>
In-Reply-To: <64c6b3e8-045b-9308-b77f-721c3d57c08b@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 84228d54-0ac6-48c9-91e3-08d97a3e02d4
x-ms-traffictypediagnostic: BN6PR11MB3891:
x-microsoft-antispam-prvs: <BN6PR11MB3891CBAAF729D4D4CAA55BEC8CDE9@BN6PR11MB3891.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: Kr4AMrPq1quZq5VV+nB3DJfJXIVyWfc9Vr10omG9t02srgcw5dhwoN+k/Cloj7OYA8orCCYY4VwvpHm24X2xhEuDQym4YREHk87vuBlkNZi1ntm10hyAtyhNUfWr+2e+WFbe/r/2wI6VwwZAw34KN8N8x22A+xQ1hCme4f1bJzXjpJIurdClKSQGjxirTyQA7mZAh0DkcNb7POOMLwyO11RxTacEkDZ++TmiLdriSMG9+djxsmwjg8lRrkgIb+pJPfqH1ecZn5c01ao65IuQ/czabgRjW04cBwsq2h2VV5fVF+2jKjBmwaypc8KwJ2eVNm7RvrrTBA9hJLS1i9Yp0gjMfzrO2/TmL9x7ZxYm7at28YzUERNHw3q7I66JptrCYDUidc5cXOzLc31cKLvXND3cYGtQFgxil/F8AOf5HYq86wBVbKl40CBIROBv54eKdbtgGo/YVH44hbimA1APWUfn/+DQV2I5q4PI+3W2iULKkqwfjAggpwmJZWNMIt3t/EA2SVyQKkyiGbe9YHroybKU+LreGnf1KciZLSXVqRxx/8rj0pn+SZq1w7+Ka6h5f0N1RHpXQIrGUfhwURPhOzZgKqAPECQT9+U1zCRy6YfwR6fSJ/ib/oQkmSPD0msivT4y8qg2V0ZgX3aKxPD+dsUy7v64XnhVzGneHHsSIQGd+GR/7gB3rGePsbSvE14Z0vqj/I0cSCZ0r4zf6/hJ5Q==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(136003)(376002)(346002)(396003)(366004)(5660300002)(71200400001)(6506007)(4326008)(9686003)(7696005)(52536014)(38070700005)(33656002)(64756008)(66446008)(66476007)(76116006)(66556008)(186003)(26005)(66946007)(8936002)(8676002)(478600001)(122000001)(316002)(55016002)(2906002)(110136005)(38100700002)(86362001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?MmpEaW5GNVQwaXo0RHhpL1NGRk1iZUljTWVwck1BaFJMbkZkL1VYamNYb080?=
 =?utf-8?B?QS9HVUkrRytOZ3MyazN3RWxZbE9FV3crRS9BaFY3cXdRbFNqeDdsNXY3UjRw?=
 =?utf-8?B?dUdxeDBsamc2MTVxbEI1ZlJpVy9wUE1nWGUzWlVURDlTTmNkNm15ZUJsK2pH?=
 =?utf-8?B?bjUwcWV3QXpBSXBJQUpTaSthU1JUOHA5OVN6SDBIRWoxNjdxOVRraHB0STdM?=
 =?utf-8?B?S2F2NTRzRml2aGNhNFltU1Jvb1RDNHFrb1FTdTc1Y09TRWJqVTFkdUpTRHZL?=
 =?utf-8?B?U1ZtSUZpQzZ0emZHbHVaVVpJUk1XNFZkUlVieHMrR0w0NzR0anZ6VzJRNFJm?=
 =?utf-8?B?YVR5eURLaU1xWDcwWHVCNndaYlkwOG5hZTRENENFanVpbG5GbXZzOUU2WmVR?=
 =?utf-8?B?bEJzNGVWSFpNcmt2NGxzUjNmK3plWkFTNkFvcUF2MkVob3ZZTkhSOG40WXkz?=
 =?utf-8?B?ZG03dHBvOXBHaG9ZTEg4ODNoRWpXeSt3N2M0U1U4RDQ4Zms0eStlQVBjWGJx?=
 =?utf-8?B?bktZZ3BudkpEUmxyOHYyNHF5M0JVSFhMV1NmdUdYeDQ4Z0w2SEZCVlJMc2xR?=
 =?utf-8?B?b09kcnBsQVFpcHhrSzkzT0tTejc4WUFDT285Y0puMVpzSFRwYzhEQjJYR1FP?=
 =?utf-8?B?eUo4bmhjT3U1ck9lWlJKMk5DNjdrODdLbzRoUDgvdzlDNFFYRCs2NFZDSVdS?=
 =?utf-8?B?ZEpEUlArcHJoaVJWM0lNOXl6Y3E3SGd3RzFsVFhwNHZHalNnVEpPM1I1djNr?=
 =?utf-8?B?akpUOXFIV3BERU54QXBnMFRIU0ZaTHoySy9ZMm5OcEhKNGZGcFViMW5NbmdR?=
 =?utf-8?B?NzNSbjVDK2VKVkhXTXhNRDlrM0VEKzl5T3pYWmxLaVdTUEt1bXgrN29ORFdV?=
 =?utf-8?B?cER3QWtpR0pEN0tNNE9BOTNqZlRYZTlWb3RTSWR0R1BXRTVYUy9JYVUxQlNl?=
 =?utf-8?B?aHlQUjRuYVpkQTVUOHhWWmlvaExiczlJSElTM2hCcWtuSlh5b3U2V0dRN1Mw?=
 =?utf-8?B?QnlSN01yeXZmVjhRcWp6UjRyWDRrbFBHajRwUVdZL1JHL3Z2VGVxNFE5aDZo?=
 =?utf-8?B?ZFhDT2JkUE5XT3dVSUZGYjArajVXRWdoMHA0WUZzbVRJWHZHcmFRTU50Wkl3?=
 =?utf-8?B?YWVKaHhOWEh0V1U0cllJTDMrME83aVA0NWRkemV3clcwbHEwNGJhVFJsUmdM?=
 =?utf-8?B?NUpxUHVkL2xqYlB3aEFVSGR4cXV1KzBBK0ZnazZrQU5xbENCdmJ6b3dsWEdM?=
 =?utf-8?B?cTU0S2FIQWJ5d1BsbzdVb045R09LVEs0TmgwRU9QcXFMVVV5MkF2NDNoQ1Zn?=
 =?utf-8?B?VTNXekUrOUVlT0FwRm04TEEvcUVwSkxBTGRMa1o5SUJlVGRzWEYyNExOTmp0?=
 =?utf-8?B?KzVEZ1lzMWVHd3ZKVHFEcFB5TkEyVTdvN3hhV0swL1dIbXk1dVNMVXgycXRY?=
 =?utf-8?B?a1F5Q2liWW9RTFV6YnRDQjJ0akc5KzY0bitHTFBFY1VCYkVkeHc2SGFZQjJO?=
 =?utf-8?B?Nk1QODhySFlkWE9aSWNzL21iSkd2dTI0bFpzV3lrNnpqRDVoVm9neUVrejZx?=
 =?utf-8?B?SDAyNHRhM255d0NrMTBBTnp5bVF0aG9zYzV1Z2hlVXo0QTFCYWVPb2MzSUp1?=
 =?utf-8?B?bysrVXY2SkdMNGFIUVJSOFVJczBFR3lNem0xemNlc05GbmlEUmhQTWZlMFlB?=
 =?utf-8?B?TEFEOWU0VVZsNGRIbW1NYU9KMEpYSzE0MzRTUTU1SGVwc2ZnaFNITVF6VkVh?=
 =?utf-8?Q?/qhXAHUR2g7rtNexLNGb3jXXZM9ywNY+QuYLEON?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 84228d54-0ac6-48c9-91e3-08d97a3e02d4
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Sep 2021 00:48:20.2646
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: POYGggIJ/rO1GPnqK/KA5ldcnBCfh7lPhU52nWpFjEH0JYNkjIRO92skVdFnu5bU0fENgFc+QmMsfqFN9KaVyg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB3891
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IEZyaWRheSwg
U2VwdGVtYmVyIDE3LCAyMDIxIDc6MDAgUE0NCj4gDQo+IFdoZXRoZXIgdG8gY2xlYXIgYW4gSU9N
TVUncyBiaXQgaW4gdGhlIGRvbWFpbidzIGJpdG1hcCBzaG91bGQgZGVwZW5kIG9uDQo+IGFsbCBk
ZXZpY2VzIHRoZSBkb21haW4gY2FuIGNvbnRyb2wuIEZvciB0aGUgaGFyZHdhcmUgZG9tYWluIHRo
aXMNCj4gaW5jbHVkZXMgaGlkZGVuIGRldmljZXMsIHdoaWNoIGFyZSBhc3NvY2lhdGVkIHdpdGgg
RG9tWEVOLg0KPiANCj4gV2hpbGUgdG91Y2hpbmcgcmVsYXRlZCBsb2dpYw0KPiAtIGNvbnZlcnQg
dGhlICJjdXJyZW50IGRldmljZSIgZXhjbHVzaW9uIGNoZWNrIHRvIGEgc2ltcGxlIHBvaW50ZXIN
Cj4gICBjb21wYXJpc29uLA0KPiAtIGNvbnZlcnQgImZvdW5kIiB0byAiYm9vbCIsDQo+IC0gYWRq
dXN0IHN0eWxlIGFuZCBjb3JyZWN0IGEgdHlwbyBpbiBhbiBleGlzdGluZyBjb21tZW50Lg0KPiAN
Cj4gU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KDQpSZXZp
ZXdlZC1ieTogS2V2aW4gVGlhbiA8a2V2aW4udGlhbkBpbnRlbC5jb20+DQoNCj4gLS0tDQo+IHYy
OiBJbnRyb2R1Y2UgaGVscGVyIGZ1bmN0aW9uLg0KPiANCj4gLS0tIGEveGVuL2RyaXZlcnMvcGFz
c3Rocm91Z2gvdnRkL2lvbW11LmMNCj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvdnRk
L2lvbW11LmMNCj4gQEAgLTE2NTAsNiArMTY1MCwyNyBAQCBpbnQgZG9tYWluX2NvbnRleHRfdW5t
YXBfb25lKA0KPiAgICAgIHJldHVybiByYzsNCj4gIH0NCj4gDQo+ICtzdGF0aWMgYm9vbCBhbnlf
cGRldl9iZWhpbmRfaW9tbXUoY29uc3Qgc3RydWN0IGRvbWFpbiAqZCwNCj4gKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqZXhjbHVkZSwNCj4g
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1Y3QgdnRkX2lvbW11
ICppb21tdSkNCj4gK3sNCj4gKyAgICBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldjsNCj4gKw0K
PiArICAgIGZvcl9lYWNoX3BkZXYgKCBkLCBwZGV2ICkNCj4gKyAgICB7DQo+ICsgICAgICAgIGNv
bnN0IHN0cnVjdCBhY3BpX2RyaGRfdW5pdCAqZHJoZDsNCj4gKw0KPiArICAgICAgICBpZiAoIHBk
ZXYgPT0gZXhjbHVkZSApDQo+ICsgICAgICAgICAgICBjb250aW51ZTsNCj4gKw0KPiArICAgICAg
ICBkcmhkID0gYWNwaV9maW5kX21hdGNoZWRfZHJoZF91bml0KHBkZXYpOw0KPiArICAgICAgICBp
ZiAoIGRyaGQgJiYgZHJoZC0+aW9tbXUgPT0gaW9tbXUgKQ0KPiArICAgICAgICAgICAgcmV0dXJu
IHRydWU7DQo+ICsgICAgfQ0KPiArDQo+ICsgICAgcmV0dXJuIGZhbHNlOw0KPiArfQ0KPiArDQo+
ICBzdGF0aWMgaW50IGRvbWFpbl9jb250ZXh0X3VubWFwKHN0cnVjdCBkb21haW4gKmRvbWFpbiwg
dTggZGV2Zm4sDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KQ0KPiAgew0KPiBAQCAtMTY1Nyw3ICsxNjc4LDcgQEAgc3RhdGljIGludCBkb21h
aW5fY29udGV4dF91bm1hcChzdHJ1Y3QgZA0KPiAgICAgIHN0cnVjdCB2dGRfaW9tbXUgKmlvbW11
ID0gZHJoZCA/IGRyaGQtPmlvbW11IDogTlVMTDsNCj4gICAgICBpbnQgcmV0Ow0KPiAgICAgIHU4
IHNlZyA9IHBkZXYtPnNlZywgYnVzID0gcGRldi0+YnVzLCB0bXBfYnVzLCB0bXBfZGV2Zm4sIHNl
Y2J1czsNCj4gLSAgICBpbnQgZm91bmQgPSAwOw0KPiArICAgIGJvb2wgZm91bmQ7DQo+IA0KPiAg
ICAgIHN3aXRjaCAoIHBkZXYtPnR5cGUgKQ0KPiAgICAgIHsNCj4gQEAgLTE3MzcsMjMgKzE3NTgs
MTggQEAgc3RhdGljIGludCBkb21haW5fY29udGV4dF91bm1hcChzdHJ1Y3QgZA0KPiAgICAgICAg
ICByZXR1cm4gcmV0Ow0KPiANCj4gICAgICAvKg0KPiAtICAgICAqIGlmIG5vIG90aGVyIGRldmlj
ZXMgdW5kZXIgdGhlIHNhbWUgaW9tbXUgb3duZWQgYnkgdGhpcyBkb21haW4sDQo+IC0gICAgICog
Y2xlYXIgaW9tbXUgaW4gaW9tbXVfYml0bWFwIGFuZCBjbGVhciBkb21haW5faWQgaW4gZG9taWRf
Yml0bXANCj4gKyAgICAgKiBJZiBubyBvdGhlciBkZXZpY2VzIHVuZGVyIHRoZSBzYW1lIGlvbW11
IG93bmVkIGJ5IHRoaXMgZG9tYWluLA0KPiArICAgICAqIGNsZWFyIGlvbW11IGluIGlvbW11X2Jp
dG1hcCBhbmQgY2xlYXIgZG9tYWluX2lkIGluIGRvbWlkX2JpdG1hcC4NCj4gICAgICAgKi8NCj4g
LSAgICBmb3JfZWFjaF9wZGV2ICggZG9tYWluLCBwZGV2ICkNCj4gLSAgICB7DQo+IC0gICAgICAg
IGlmICggcGRldi0+c2VnID09IHNlZyAmJiBwZGV2LT5idXMgPT0gYnVzICYmIHBkZXYtPmRldmZu
ID09IGRldmZuICkNCj4gLSAgICAgICAgICAgIGNvbnRpbnVlOw0KPiAtDQo+IC0gICAgICAgIGRy
aGQgPSBhY3BpX2ZpbmRfbWF0Y2hlZF9kcmhkX3VuaXQocGRldik7DQo+IC0gICAgICAgIGlmICgg
ZHJoZCAmJiBkcmhkLT5pb21tdSA9PSBpb21tdSApDQo+IC0gICAgICAgIHsNCj4gLSAgICAgICAg
ICAgIGZvdW5kID0gMTsNCj4gLSAgICAgICAgICAgIGJyZWFrOw0KPiAtICAgICAgICB9DQo+IC0g
ICAgfQ0KPiArICAgIGZvdW5kID0gYW55X3BkZXZfYmVoaW5kX2lvbW11KGRvbWFpbiwgcGRldiwg
aW9tbXUpOw0KPiArICAgIC8qDQo+ICsgICAgICogSGlkZGVuIGRldmljZXMgYXJlIGFzc29jaWF0
ZWQgd2l0aCBEb21YRU4gYnV0IHVzYWJsZSBieSB0aGUNCj4gaGFyZHdhcmUNCj4gKyAgICAgKiBk
b21haW4uIEhlbmNlIHRoZXkgbmVlZCBjb25zaWRlcmluZyBoZXJlIGFzIHdlbGwuDQo+ICsgICAg
ICovDQo+ICsgICAgaWYgKCAhZm91bmQgJiYgaXNfaGFyZHdhcmVfZG9tYWluKGRvbWFpbikgKQ0K
PiArICAgICAgICBmb3VuZCA9IGFueV9wZGV2X2JlaGluZF9pb21tdShkb21feGVuLCBwZGV2LCBp
b21tdSk7DQo+IA0KPiAtICAgIGlmICggZm91bmQgPT0gMCApDQo+ICsgICAgaWYgKCAhZm91bmQg
KQ0KPiAgICAgIHsNCj4gICAgICAgICAgY2xlYXJfYml0KGlvbW11LT5pbmRleCwgJmRvbV9pb21t
dShkb21haW4pLQ0KPiA+YXJjaC52dGQuaW9tbXVfYml0bWFwKTsNCj4gICAgICAgICAgY2xlYW51
cF9kb21pZF9tYXAoZG9tYWluLCBpb21tdSk7DQoNCg==


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 00:49:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 00:49:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189773.339655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROYI-0006a4-Js; Sat, 18 Sep 2021 00:49:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189773.339655; Sat, 18 Sep 2021 00:49:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROYI-0006Zx-Fu; Sat, 18 Sep 2021 00:49:58 +0000
Received: by outflank-mailman (input) for mailman id 189773;
 Sat, 18 Sep 2021 00:49:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Orqi=OI=intel.com=kevin.tian@srs-us1.protection.inumbo.net>)
 id 1mROYH-0006Zr-Fc
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 00:49:57 +0000
Received: from mga18.intel.com (unknown [134.134.136.126])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0f9c8bf-cdf2-4560-9f31-a54945cfcf27;
 Sat, 18 Sep 2021 00:49:53 +0000 (UTC)
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
 by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 17 Sep 2021 17:49:52 -0700
Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83])
 by fmsmga002.fm.intel.com with ESMTP; 17 Sep 2021 17:49:52 -0700
Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by
 fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12; Fri, 17 Sep 2021 17:49:52 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
 fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.12 via Frontend Transport; Fri, 17 Sep 2021 17:49:52 -0700
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168)
 by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.12; Fri, 17 Sep 2021 17:49:51 -0700
Received: from BN9PR11MB5433.namprd11.prod.outlook.com (2603:10b6:408:11e::13)
 by BN6PR1101MB2148.namprd11.prod.outlook.com (2603:10b6:405:52::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Sat, 18 Sep
 2021 00:49:49 +0000
Received: from BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df]) by BN9PR11MB5433.namprd11.prod.outlook.com
 ([fe80::ddb7:fa7f:2cc:45df%8]) with mapi id 15.20.4523.018; Sat, 18 Sep 2021
 00:49:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0f9c8bf-cdf2-4560-9f31-a54945cfcf27
X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="209986833"
X-IronPort-AV: E=Sophos;i="5.85,302,1624345200"; 
   d="scan'208";a="209986833"
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,302,1624345200"; 
   d="scan'208";a="554909859"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NTPUwn85ORayBI/GP58oeb3h095DX9aTn+IvDZCNacSIZQEyht61qvLarL8B9xt1VeTPR3h9pmM5rh+uDGXE5G443bSJ6MuWLO6yOWKIAAW/eftF6vYkRTr2TaKXiX1JANt7RbzprySclQ92SDh0t4ee924JK2mdT1wcY4GdZsUsGce8T74LC/JUvyQAusMwo7HioKYcSI9kz/j7hkEHssO3JFe9xTB0UvapDIGiGHsXKNRXpjNgyWDYBAC50ujilukYCzk/fLifCzuV90o2kZeoFkiIyyLTOrcYckSWDVImv9DsWCgH9bQOLqm5eViNRNWxRM15Ytcvt+GTEpEgZQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5dTTKHcpujEOYDG34UvjZmvHsr0DQdB8zXpj7ac5aQk=;
 b=C1uVo0kikXJ44Me+xQh3ajgSZyO69JKQWWnejLNdSdFb2/wHl7eitn37+8qtqC2LUvr7uXm7HGC2xUlGbYKr48BeEVvaZgCcMa/bx1HTDWjbOY3Cddykj2RX1DHRtIwPUPL6gju0PFM35I+hbXiq7MM4JS9nDYaV+D1zkdw8SolppRZPMFd85TkIgK9j3amEu3AfD+lWgAf/M3cSHQGrYxRH4McEYRl8YkW4j3Qul8NaHIphMcuI+VwGLC1/GVoFYzl6AjxPTnUIKBOHuKLtMe98aLycV4ROxI3E1oprhA4fia4phn2CW+N2M/ozPMH++cZteR1OCT1iefTJp1uGVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5dTTKHcpujEOYDG34UvjZmvHsr0DQdB8zXpj7ac5aQk=;
 b=OS6iOM/hEXb5rcQyXAESv4Xuk+QFwLMWGICtmkLvm1LRa5/2bcJtGEn8MLN4oQlhfTelO9piLE+nQzLVvo0nbe45rjHJ6c/hLuAsbeMRHq7z6zjVzj/FYgiGBKIWrLhJADwnqcTGA6PHGhJ31MVRoF8ujCfRCVFCj4qLbKnxc6A=
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Paul Durrant <paul@xen.org>
Subject: RE: [PATCH v2 2/3] VT-d: PCI segments are up to 16 bits wide
Thread-Topic: [PATCH v2 2/3] VT-d: PCI segments are up to 16 bits wide
Thread-Index: AQHXq7NLI4GDIompwEO0/aktEApWj6uo9l8Q
Date: Sat, 18 Sep 2021 00:49:49 +0000
Message-ID: <BN9PR11MB5433696E7D9D586C413773368CDE9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <c0a460c0-9d7e-c015-37cf-4775bdef33a8@suse.com>
 <ee538608-4be8-b277-099a-fa82e0b59a9c@suse.com>
In-Reply-To: <ee538608-4be8-b277-099a-fa82e0b59a9c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9b8682f4-834d-4dc2-0320-08d97a3e381a
x-ms-traffictypediagnostic: BN6PR1101MB2148:
x-microsoft-antispam-prvs: <BN6PR1101MB21488C6BDF5D945BE53D00828CDE9@BN6PR1101MB2148.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:1850;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 3zLAeOY5EI5tv+2Pa+303DXToDKkOfwNFG/FTbmlZKcnl2SJO8NC/9gjRI/O7T9SRoqULIlrxIC9MKdsuD9HEV4nTzh5uQjM+ivsIhlGADmV6n3FHUA8IIz+h2/FDdQbJgsdxNq54v+ARmhWR/0vPI+0FJCiJHFhFVsM0n64kRXsXa+ttcm3UytiT9TU8TNHJGxYuej9yTFTTdt4LZdZiOOw5YMtOCGsbu3SZFRUu56sqHRI42Y1ae9T1xVFsfU9jmwX/qg2E/pKmIvpEu0w3YeDebDI40NENS+qqklPCaW+7/fzsQsdj0qCrCVf5GLHwJBToLKsddD0i0VzRKnE1ZPcR6GyiZGdSAEy9pDL0KDQ9zeIoL1+4Jy3v25wJk/hUUmCGbS7tSjHjHobyWSpiOjInAChKCFSPWlFJPhoDI6NLcjtrJ9iBQHhNaGICzivJZ7cTdjseMYICjLTepKijk/DcxXi4Y1Gcops+WM07JF12uu5LhHACHS21OGBLO8JVNS2UJr+T+acONDtnOCMan0ptQQr+Q35m5zO4wJcus/jDZr1d266zjUpbVFOCQz1vMKxLVZBUk9b77rsymS96tH3k+NvUh7d2h8ujZ8UvUDgEa42YSaEDokvATp/qN9GVObvufpm5svPQwAz8IX2fP5tsi0R6impmwqIeLSSMRRawbBKr15mcUfpi3OVLGMCUvPP1lNNjwS/8cbwfJ9F8g==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5433.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(64756008)(9686003)(55016002)(7696005)(186003)(110136005)(508600001)(71200400001)(26005)(86362001)(6506007)(66476007)(66446008)(66556008)(8936002)(76116006)(122000001)(52536014)(38100700002)(4326008)(33656002)(316002)(5660300002)(2906002)(8676002)(66946007);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?Yk1NNVR0RDU2YTFsUHl1OHVmWWhGd1FCMDVwM2dFWVVFQWtkSjJ6dDJXVStp?=
 =?utf-8?B?Nzk0TGk2anptdGx6YmFtYnF5M0xhZzlDVzNkT1VnYi9kWjJGdTdFZEVyN2t3?=
 =?utf-8?B?c084VmFoNWRlOUpuMmthbVlhY2Y4Si9HN1dzVmY4UFpWRSszRzlCWFRCQWNv?=
 =?utf-8?B?Zm1aeExOQWI0Mm9vYlFnejEzbGZGTE9ERHJwOTR0TWJreUI5RTE3WGRNeEM2?=
 =?utf-8?B?R1VkL0FGVXI4WXVQYVNvQ1k4N1pMT2czYXRrRm8zTWkreG1MdDByaEFLUWEv?=
 =?utf-8?B?anEvaTMvck1QWEVGQkcrZVZWMHdKNzhQVGFjK3ZCS01XSi9oNXliQnEyL3JL?=
 =?utf-8?B?VkEvZzF2MjVoQmk5STR4RFpUQ0NNekx3WEJLYU9VVkNLbFlNZTBNMkF3L254?=
 =?utf-8?B?bFdvUy9vNXdvSmgzWnJEcUNpMmJNZnIySDl5VHppNVNvVFcvTVg1b1BKdHcy?=
 =?utf-8?B?MlFxVGdTYXdLUnYxaUs0MkhnVXh5RlkxTElHU2dFWXpsWVVKUWJmL0JKajJ3?=
 =?utf-8?B?cnBYakVNemZOUSt0SkN3OEF5SG95VmxVaTFlKzk4L2V4V21BS2RDWUFOVC8y?=
 =?utf-8?B?ZVNqZnVsb3dzMm9DM3VjK1BMbmFMWWJ4U1FvSURXeXFxODA5Uys0cEN6ek9O?=
 =?utf-8?B?MkM2WWtWRU9yVE1FQk1Pa1RyVGNPVmFFU2VlV3MyOU5KNmxHajRNQjZNZXlJ?=
 =?utf-8?B?aVNsdEdvbGRPQUpWbVJxSlhaVGtmWW9IS0dMbjBBUWFpNWRhRWt2NGhqZGVL?=
 =?utf-8?B?RFd0QncxUTRhaE9zSVdMbGoyVVpyQlVVaXUwSVk2QVNUWUdWT2pEanlQai92?=
 =?utf-8?B?a09aMWJNb0U1Z0thRTI1OHpLVVlGTDF5YW5DYnNBbmdQakp1Q2dRV2FBTmNG?=
 =?utf-8?B?Znc2Z1F4RTVyeTVvYjJGTXhiTHBHalc5cklGWWtoMUp1VXNBMi9OVTBHNFpy?=
 =?utf-8?B?QmdwNUNuM1I0ZDFJbEdkMHQ1OGZPVDRNT05wK2xRRTRsWUJNZmpPdi9yeEE2?=
 =?utf-8?B?aCsvUDhuTU44NnNQVWcxNHpvOUEvVVRGN0QrNjN2bCs3N09qcVB6NE9QcldH?=
 =?utf-8?B?SWthSnBodnFpTyt2M00ra1N5d2dVaVE5YUJRZEJqU3ByYmROTlMzdkMxbFQx?=
 =?utf-8?B?QkU2elcvenQwajR2TjlDZitza1Q5a1A1KzN4Z3ZTWVBydnU2OW5TWW9Lbmpu?=
 =?utf-8?B?czZGVUIzZHdZdzRiTkdTTEZhbUY1WlBJZHFiZDVnZ25RUFllaUFJTnM2MzNZ?=
 =?utf-8?B?QVhwenBTWWNXc1NpcXduTHVXTGRzNDMvKys1eE1vM0xSUDVmNzZaV0dkWDVW?=
 =?utf-8?B?eG5JUmNFc0NSdlh1d0lUeWZXNktxcmg0TTBtU0xrbFc0ZHlObkx0ZmU4RmNJ?=
 =?utf-8?B?Q3Z2aGFDYk9KN01Dd0grUTZuQW0vbytlcFNmWllWVmg5TktFWkpqRkRUQkNh?=
 =?utf-8?B?SlMxR2dZWXJUTWRmMWovdGJqc3crWExhd1p0bzVxQTlJWVI3VmwvbExvZFU1?=
 =?utf-8?B?V2Y1MVRUL2ZFNUxPTGNKMGpNbWFkK2MrRnZSdFQvc25KVXhvWEVvSVIyNkNC?=
 =?utf-8?B?cmF1TGNBN2pXY3owazZDMGZKaFdJdXQyQ0pRekNwdElPTDJBWFJkTkptdFhU?=
 =?utf-8?B?Z3MwR0dRc1liT1NNSElqb1UxaW96WHc0eXNhWFovdU42T2lSRXFQb3M1ZEFL?=
 =?utf-8?B?YUJoaFhLRU93bGJXMVdHVytXZTJpZGc5dmtRYXpPTFBhYzJwKzBIZUNuTUE1?=
 =?utf-8?Q?MklQwZRcQvC4Dd9usW6tY280gpXiIEmOqSsRojw?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9b8682f4-834d-4dc2-0320-08d97a3e381a
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Sep 2021 00:49:49.3731
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: aFPyl1huvfmH2MadhB2gAGE0nm3KC7K2e2PLNC1P7aSubj3q3lhIiYUwIQ76F4f2Z5GGk3jGyK3+oAvBPLN1zg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2148
X-OriginatorOrg: intel.com

PiBGcm9tOiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IFNlbnQ6IEZyaWRheSwg
U2VwdGVtYmVyIDE3LCAyMDIxIDc6MDAgUE0NCj4gDQo+IFdlIHNob3VsZG4ndCBzaWxlbnRseSB0
cnVuY2F0ZSByZXNwZWN0aXZlIHZhbHVlcy4NCj4gDQo+IFNpZ25lZC1vZmYtYnk6IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCg0KUmV2aWV3ZWQtYnk6IEtldmluIFRpYW4gPGtldmlu
LnRpYW5AaW50ZWwuY29tPg0KDQo+IC0tLQ0KPiBTdHJpY3RseSBzcGVha2luZyB3ZSBzaG91bGRu
J3QgdXNlIHVpbnQ8Tj5fdCBoZXJlIGF0IGFsbC4gSSB3YXNuJ3Qgc3VyZQ0KPiB0aG91Z2ggd2hl
dGhlciBnb2luZyBzdHJhaWdodCB0byB1bnNpZ25lZCBpbnQgd291bGRuJ3QgYmUgdmlld2VkIGFz
DQo+IGludHJvZHVjaW5nIGluY29uc2lzdGVuY2llcy4NCj4gLS0tDQo+IHYyOiBOZXcuDQo+IA0K
PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUuYw0KPiArKysgYi94ZW4v
ZHJpdmVycy9wYXNzdGhyb3VnaC92dGQvaW9tbXUuYw0KPiBAQCAtMTQ2Miw3ICsxNDYyLDggQEAg
c3RhdGljIGludCBkb21haW5fY29udGV4dF9tYXBwaW5nKHN0cnVjdA0KPiAgew0KPiAgICAgIGNv
bnN0IHN0cnVjdCBhY3BpX2RyaGRfdW5pdCAqZHJoZCA9IGFjcGlfZmluZF9tYXRjaGVkX2RyaGRf
dW5pdChwZGV2KTsNCj4gICAgICBpbnQgcmV0ID0gMDsNCj4gLSAgICB1OCBzZWcgPSBwZGV2LT5z
ZWcsIGJ1cyA9IHBkZXYtPmJ1cywgc2VjYnVzOw0KPiArICAgIHVpbnQxNl90IHNlZyA9IHBkZXYt
PnNlZzsNCj4gKyAgICB1aW50OF90IGJ1cyA9IHBkZXYtPmJ1cywgc2VjYnVzOw0KPiANCj4gICAg
ICAvKg0KPiAgICAgICAqIEdlbmVyYWxseSB3ZSBhc3N1bWUgb25seSBkZXZpY2VzIGZyb20gb25l
IG5vZGUgdG8gZ2V0IGFzc2lnbmVkIHRvIGENCj4gQEAgLTE2NzcsNyArMTY3OCw4IEBAIHN0YXRp
YyBpbnQgZG9tYWluX2NvbnRleHRfdW5tYXAoc3RydWN0IGQNCj4gICAgICBjb25zdCBzdHJ1Y3Qg
YWNwaV9kcmhkX3VuaXQgKmRyaGQgPSBhY3BpX2ZpbmRfbWF0Y2hlZF9kcmhkX3VuaXQocGRldik7
DQo+ICAgICAgc3RydWN0IHZ0ZF9pb21tdSAqaW9tbXUgPSBkcmhkID8gZHJoZC0+aW9tbXUgOiBO
VUxMOw0KPiAgICAgIGludCByZXQ7DQo+IC0gICAgdTggc2VnID0gcGRldi0+c2VnLCBidXMgPSBw
ZGV2LT5idXMsIHRtcF9idXMsIHRtcF9kZXZmbiwgc2VjYnVzOw0KPiArICAgIHVpbnQxNl90IHNl
ZyA9IHBkZXYtPnNlZzsNCj4gKyAgICB1aW50OF90IGJ1cyA9IHBkZXYtPmJ1cywgdG1wX2J1cywg
dG1wX2RldmZuLCBzZWNidXM7DQo+ICAgICAgYm9vbCBmb3VuZDsNCj4gDQo+ICAgICAgc3dpdGNo
ICggcGRldi0+dHlwZSApDQoNCg==


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 01:10:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 01:10:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189784.339672 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROs5-0008Iu-AT; Sat, 18 Sep 2021 01:10:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189784.339672; Sat, 18 Sep 2021 01:10:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mROs5-0008In-6J; Sat, 18 Sep 2021 01:10:25 +0000
Received: by outflank-mailman (input) for mailman id 189784;
 Sat, 18 Sep 2021 01:10:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mROs3-0008Id-LD; Sat, 18 Sep 2021 01:10:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mROs3-0001dp-74; Sat, 18 Sep 2021 01:10:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mROs2-0000Op-Ur; Sat, 18 Sep 2021 01:10:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mROs2-0000oL-UJ; Sat, 18 Sep 2021 01:10:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=g40SMcYHhdOB0IEUpZUsE4k4cEnEBqyiLfciSQyeMxs=; b=bruIuz/Mk45q0FArAEvSC4ulf3
	pgURKzhMJMH0ssrvfK9Mk6C4t/DU/1Xm8OmsUwjoM74FEK8DBQnNKn5zjpWnC3M0kQpzDOhbdmvkh
	nj7qMETvyUz/MYrlrWo5LhB0lSpOdHtLpVz9vkAfZ3DenBf4nQGozpYCFzkJp1GCNoMw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165097-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165097: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=48a160f46251f9752bd39a854ca74d80fbbbe90c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 01:10:22 +0000

flight 165097 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165097/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  48a160f46251f9752bd39a854ca74d80fbbbe90c

Last test of basis   165069  2021-09-17 13:00:29 Z    0 days
Testing same since   165097  2021-09-17 20:01:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   48a160f462..437e88417b  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8 -> smoke


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 01:54:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 01:54:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189793.339686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRPYA-0004hJ-O3; Sat, 18 Sep 2021 01:53:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189793.339686; Sat, 18 Sep 2021 01:53:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRPYA-0004hC-KT; Sat, 18 Sep 2021 01:53:54 +0000
Received: by outflank-mailman (input) for mailman id 189793;
 Sat, 18 Sep 2021 01:53:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRPY9-0004h2-Op; Sat, 18 Sep 2021 01:53:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRPY9-0002Li-8H; Sat, 18 Sep 2021 01:53:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRPY8-00035G-TY; Sat, 18 Sep 2021 01:53:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRPY8-0003jb-St; Sat, 18 Sep 2021 01:53:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Oy1mpM4vbJ4avzviUANKYRdi5wLqqpHhc3a9BszEHHg=; b=RcCYr0p1Olms6Fcs0SsVY1ob+S
	LZYXMJ1vVIBGbF7MZtk9tHRe60GV0guwBqR6YaVW6eknaAzum3meZeIAlfo7Q2CvlbgQ28krFrx5e
	1vJaSFm3KEP9jx/mAFwEWroUdceeaEQeCtR+shE6gy0qQFbbIuDXLS5aodlrj+n6buO8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165052-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165052: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=98f176b62b6136f696da88032f9c12a244ac9aed
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 01:53:52 +0000

flight 165052 xen-unstable real [real]
flight 165101 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165052/
http://logs.test-lab.xenproject.org/osstest/logs/165101/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  98f176b62b6136f696da88032f9c12a244ac9aed
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    7 days
Failing since        164951  2021-09-12 00:14:36 Z    6 days   11 attempts
Testing same since   165052  2021-09-17 09:47:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 601 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 06:15:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 06:15:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189809.339702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRTco-0008By-Cf; Sat, 18 Sep 2021 06:14:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189809.339702; Sat, 18 Sep 2021 06:14:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRTco-0008Br-9U; Sat, 18 Sep 2021 06:14:58 +0000
Received: by outflank-mailman (input) for mailman id 189809;
 Sat, 18 Sep 2021 06:14:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRTcm-0008Bf-Uw; Sat, 18 Sep 2021 06:14:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRTcm-0007ca-Nq; Sat, 18 Sep 2021 06:14:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRTcm-0006Lg-DC; Sat, 18 Sep 2021 06:14:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRTcm-0003aJ-Cf; Sat, 18 Sep 2021 06:14:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NnNFLRqfbTo4TMnhGXLBal+Cldox2aOSBwvdCMge3wg=; b=RSFyxY77a/vRgReC63FoJd5jiy
	79qJNqO0pglo2r6YsnpkRTm4RnE2bKvNVvR6Rxz8tYBE4/7G/VOWvlZyHg9tNnyHAstTvKo2MF+ca
	jVtBST8fLnJokCq0mwN/kpcx6MtvzySCXZSljIy4x+00qiDbJT6mqHnR+klR5GreEUUE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165076-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165076: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-debianhvm-amd64:guest-start/debianhvm.repeat:fail:heisenbug
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 06:14:56 +0000

flight 165076 qemu-mainline real [real]
flight 165105 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165076/
http://logs.test-lab.xenproject.org/osstest/logs/165105/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 20 guest-start/debianhvm.repeat fail pass in 165105-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    6 days
Failing since        164967  2021-09-13 13:06:52 Z    4 days    9 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 07:12:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 07:12:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189821.339716 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRUWS-0006mK-Oy; Sat, 18 Sep 2021 07:12:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189821.339716; Sat, 18 Sep 2021 07:12:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRUWS-0006mD-Le; Sat, 18 Sep 2021 07:12:28 +0000
Received: by outflank-mailman (input) for mailman id 189821;
 Sat, 18 Sep 2021 07:12:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRUWR-0006m3-IL; Sat, 18 Sep 2021 07:12:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRUWR-00008E-1g; Sat, 18 Sep 2021 07:12:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRUWQ-0000yK-NL; Sat, 18 Sep 2021 07:12:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRUWQ-0007j2-Mw; Sat, 18 Sep 2021 07:12:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=AhuCfY/1ogbbRmUd994brlwtsCz9rBQLygG+GMzWj/g=; b=OVIogta5D39hR5ksiOF46zhZs0
	+6AbwXcE0V8CCnCeShtOZojHyafFEMUwxo7B7CZ8oruqKTrNOwqWkEMeCpfseUafmiRDcfL7J38qD
	gTyWwBrGr/likwJA8ZLVIxRqMvEiRUXTDfLbmIGkNGAc5RgaWHopUtNT7KixE6bLvrd4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165085-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165085: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:guest-start/debianhvm.repeat:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-vhd:xen-boot:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=bdb575f872175ed0ecf2638369da1cb7a6e86a14
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 07:12:26 +0000

flight 165085 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165085/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-xl          14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl          13 debian-fixup     fail in 165019 pass in 165085
 test-arm64-arm64-xl-credit2  13 debian-fixup     fail in 165019 pass in 165085
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 165019 pass in 165085
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 18 guest-start/debianhvm.repeat fail pass in 165019

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       8 xen-boot                fail baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                bdb575f872175ed0ecf2638369da1cb7a6e86a14
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  413 days
Failing since        152366  2020-08-01 20:49:34 Z  412 days  718 attempts
Testing same since   165019  2021-09-16 22:41:26 Z    1 days    2 attempts

------------------------------------------------------------
7312 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2259637 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 08:06:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 08:06:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189838.339731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRVMC-0004oE-Cw; Sat, 18 Sep 2021 08:05:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189838.339731; Sat, 18 Sep 2021 08:05:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRVMC-0004o7-9y; Sat, 18 Sep 2021 08:05:56 +0000
Received: by outflank-mailman (input) for mailman id 189838;
 Sat, 18 Sep 2021 08:05:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRVMB-0004nx-K8; Sat, 18 Sep 2021 08:05:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRVMB-0001XZ-EQ; Sat, 18 Sep 2021 08:05:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRVMB-0004st-1Y; Sat, 18 Sep 2021 08:05:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRVMB-0006hp-0o; Sat, 18 Sep 2021 08:05:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zZP4HCYRIEtU6q8uLIES5l2y6fLE9Q93G3IIScxN23k=; b=tAI34wH9127pwFtMZ4Oy1BVZN+
	6pVH1Q2Y+6S93S11lz/54IIhx08Fd263gl/eVcgGFzB4EMtYvEBKZehiRDGVmiV+8eqh9oHJIsFfD
	rlED88ffwWKBLyZyzccv/5u7jS4z93i/Q5qsZIBGxRqkwE4of/7ZsxDt8xvXnGgU/unA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165104-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165104: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=785a11cec8693de7df024aae68975dd1799b646a
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 08:05:55 +0000

flight 165104 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165104/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              785a11cec8693de7df024aae68975dd1799b646a
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  435 days
Failing since        151818  2020-07-11 04:18:52 Z  434 days  425 attempts
Testing same since   165104  2021-09-18 04:20:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73994 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 13:37:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 13:37:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189886.339745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRaWv-0006HR-9o; Sat, 18 Sep 2021 13:37:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189886.339745; Sat, 18 Sep 2021 13:37:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRaWv-0006HK-5r; Sat, 18 Sep 2021 13:37:21 +0000
Received: by outflank-mailman (input) for mailman id 189886;
 Sat, 18 Sep 2021 13:37:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRaWt-0006H9-GT; Sat, 18 Sep 2021 13:37:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRaWt-00078G-8L; Sat, 18 Sep 2021 13:37:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRaWs-0004Zq-Sb; Sat, 18 Sep 2021 13:37:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRaWs-0006DK-S8; Sat, 18 Sep 2021 13:37:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7tTYLYKtlB+eMaTvsLfk7PUQJ8wVqO5K6LN3llYcxfU=; b=lRokfBTvB4+anCXjAAakMLilqG
	7HWWpEVdyVGPDY9YCGqhvbtkX2L90CiTCog3u0fG/+ycTEPd/sipfy3LyzIO0I0briFZIZZkczbfk
	ipA9UcqtbjOQAw+YfkkvXKsuocao39dNr+DYv0eb3E57kltuPtriExju9Anvi2InxfNc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165103-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165103: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 13:37:18 +0000

flight 165103 xen-unstable real [real]
flight 165109 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165103/
http://logs.test-lab.xenproject.org/osstest/logs/165109/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    7 days
Failing since        164951  2021-09-12 00:14:36 Z    6 days   12 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 15:30:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 15:30:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189917.339759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRcI0-0001tQ-Ke; Sat, 18 Sep 2021 15:30:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189917.339759; Sat, 18 Sep 2021 15:30:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRcI0-0001sm-HL; Sat, 18 Sep 2021 15:30:04 +0000
Received: by outflank-mailman (input) for mailman id 189917;
 Sat, 18 Sep 2021 15:30:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcHz-0001ee-4Q; Sat, 18 Sep 2021 15:30:03 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcHy-0000fj-Oy; Sat, 18 Sep 2021 15:30:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcHy-0008Kz-DS; Sat, 18 Sep 2021 15:30:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcHy-0004H8-Cv; Sat, 18 Sep 2021 15:30:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yIulY5YsR//ptRds0456DnEV4TBJRIuQiiJTGfl6piQ=; b=6RyKzIMIwKE+k3/Ijx6BDBd3sZ
	8FeQFnV7wOaaAxgvG1nBYG+gk0Gk652l/RW09wt3yNw3WnyLZ7RhfIwP8w8CdNbqsV0XqxSnK7IL3
	KNrGYttf5AGEgVTRZF94QsjIm/v2nAXTswt7dQ+EhkdRaUOjG2A6BDZB3WsFMK2VrQfk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165106-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165106: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 15:30:02 +0000

flight 165106 qemu-mainline real [real]
flight 165111 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165106/
http://logs.test-lab.xenproject.org/osstest/logs/165111/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    6 days
Failing since        164967  2021-09-13 13:06:52 Z    5 days   10 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 15:48:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 15:48:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189928.339772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRcZf-00043s-8M; Sat, 18 Sep 2021 15:48:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189928.339772; Sat, 18 Sep 2021 15:48:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRcZf-00043l-5V; Sat, 18 Sep 2021 15:48:19 +0000
Received: by outflank-mailman (input) for mailman id 189928;
 Sat, 18 Sep 2021 15:48:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcZe-00043b-G5; Sat, 18 Sep 2021 15:48:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcZe-0000zV-AT; Sat, 18 Sep 2021 15:48:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcZe-0000Zr-1W; Sat, 18 Sep 2021 15:48:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRcZd-0002Uh-3U; Sat, 18 Sep 2021 15:48:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rEJlhPf+9VaxWbPU8JOkFIrH976vMXwMncJrEl/Gs9k=; b=GaPiokbdqqYTAJ+xq5R/gKhNLs
	ERkmRkEuTMzblwVU2ijed6HSi0ZbHEN7Zm3LIqgG4CxzrzgNpJwNDYdQ5VO9xa4tzkkazrcMKA+bg
	e+5nnse532kS4EUTF86voth6Zpe8dpIqNwdzNH/bYYcq3ifM81fMGt9cu4jycLuxVGFE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165108-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165108: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=e3e47d7963985f669f5826eac41b3ac4f19fc444
X-Osstest-Versions-That:
    ovmf=ac6388add4ade3ae5c4036ea6c2ce9c8d301d057
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 15:48:17 +0000

flight 165108 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165108/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 e3e47d7963985f669f5826eac41b3ac4f19fc444
baseline version:
 ovmf                 ac6388add4ade3ae5c4036ea6c2ce9c8d301d057

Last test of basis   165013  2021-09-16 15:12:46 Z    2 days
Testing same since   165108  2021-09-18 08:42:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ashraf Ali S <ashraf.ali.s@intel.com>
  S, Ashraf Ali <ashraf.ali.s@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   ac6388add4..e3e47d7963  e3e47d7963985f669f5826eac41b3ac4f19fc444 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 16:59:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 16:59:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189944.339787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRdge-0003rP-EK; Sat, 18 Sep 2021 16:59:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189944.339787; Sat, 18 Sep 2021 16:59:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRdge-0003rI-Ay; Sat, 18 Sep 2021 16:59:36 +0000
Received: by outflank-mailman (input) for mailman id 189944;
 Sat, 18 Sep 2021 16:59:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fwkB=OI=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mRdgd-0003rC-Dt
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 16:59:35 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 06e9363e-5b5e-403e-811c-96118525d999;
 Sat, 18 Sep 2021 16:59:33 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id i4so47124195lfv.4
 for <xen-devel@lists.xenproject.org>; Sat, 18 Sep 2021 09:59:33 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id a7sm1083765ljd.85.2021.09.18.09.59.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sat, 18 Sep 2021 09:59:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06e9363e-5b5e-403e-811c-96118525d999
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=oW8YxKEe7diMMMV71t8u/w/9n13dsQhHhMJA3zl9ZgA=;
        b=qnNyTK//nUZKtUZcbSlWFZIQq9HNxKM8119U0Wls4are5tp/GPivXGoADYUqMCUGfR
         ySB8seF7bG5/ON/gcuqnngi58HgGIQAUVNJ2vmUkEPp1m+9UaMYfc7ei4lhDMCeCpBE1
         wyqjiEivVnK597oNcoNACEgOLe3g5FkT5M6NmaCvTwSXfaMDO9q23eoV00+vqi14Us05
         c9/j5umRlp12i2astzIFmOCxqWld6zMIqnWtu/U8Hv2VTyUrb1GedrTICYDCg6uOFip1
         VdcP8+bZdmqJtYWUT9+l5OvNVLz8hD96U4BVs97aCO4VG1AMsfUWELp629fMHmbNY/hG
         HdQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=oW8YxKEe7diMMMV71t8u/w/9n13dsQhHhMJA3zl9ZgA=;
        b=AU/P9T/eljxTDzDszrmd7VhqSJeocxcqzzDk8Ksjn5Xtr+P6a3FwFi8EAhi3J5gGuC
         tnwtdFeTQzJlsc8ZvRsQtQ+9a+j3dUVANmxWDgR5WGCOGsJEM0kFSNS/fYhC1HnkRNqf
         rr4Dj4feU2zzsUpHYaSX4jDaDAVER8dgF3Bx8+EEvxepabcjr2k7ZNjxJJxKOl06nEZu
         FGtYtGP4pkj7XB0Z4KW3Wa9d2m7gisMYeKJdOpb0GccdkAD20UlC1F9tImULqt8ohrn7
         CZiUw3QtIQblqmudLchJ+30o1oJ4HGvSOUkDQVMS2hquLzPAfAzglNlcEFi4FGApHjja
         yhvg==
X-Gm-Message-State: AOAM531R4ofX6Pcj1RIQD1sPmdM5j5FniihdCXyyeRajMe/ja8Jexz/a
	e7xCN5yMtmN25uNMT01iFVY=
X-Google-Smtp-Source: ABdhPJzyFsQv56UPUmc5ll/AfkD2laPQ25ywOfpsSucVb2uXutrlu9tKa8P89TOwQSh9FZUrCpoFyA==
X-Received: by 2002:ac2:5a48:: with SMTP id r8mr4518269lfn.317.1631984372087;
        Sat, 18 Sep 2021 09:59:32 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
From: Oleksandr <olekstysh@gmail.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
Message-ID: <cc5ee8cc-84ac-a27f-af99-ac0ba3ab8d68@gmail.com>
Date: Sat, 18 Sep 2021 19:59:30 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


Hi Julien.


[snip]


>>
>>
>>> +#define EXT_REGION_END 0x80003fffffffULL
>>> +
>>> +static int __init find_unallocated_memory(const struct kernel_info 
>>> *kinfo,
>>> +                                          struct meminfo *ext_regions)
>>> +{
>>> +    const struct meminfo *assign_mem = &kinfo->mem;
>>> +    struct rangeset *unalloc_mem;
>>> +    paddr_t start, end;
>>> +    unsigned int i;
>>> +    int res;
>>
>> We technically already know which range of memory is unused. This is 
>> pretty much any region in the freelist of the page allocator. So how 
>> about walking the freelist instead?
>
> ok, I will investigate the page allocator code (right now I have no 
> understanding of how to do that). BTW, I have just grepped "freelist" 
> through the code and all page context related appearances are in x86 
> code only.
>
>>
>> The advantage is we don't need to worry about modifying the function 
>> when adding new memory type.
>>
>> One disavantage is this will not cover *all* the unused memory as 
>> this is doing. But I think this is an acceptable downside.

I did some investigations and create test patch. Although, I am not 100% 
sure this is exactly what you meant, but I will provide results anyway.

1. Below the extended regions (unallocated memory, regions >=64MB ) 
calculated by my initial method (bootinfo.mem - kinfo->mem - 
bootinfo.reserved_mem - kinfo->gnttab):

(XEN) Extended region 0: 0x48000000->0x54000000
(XEN) Extended region 1: 0x57000000->0x60000000
(XEN) Extended region 2: 0x70000000->0x78000000
(XEN) Extended region 3: 0x78200000->0xc0000000
(XEN) Extended region 4: 0x500000000->0x580000000
(XEN) Extended region 5: 0x600000000->0x680000000
(XEN) Extended region 6: 0x700000000->0x780000000

2. Below the extended regions (unallocated memory, regions >=64MB) 
calculated by new method (free memory in page allocator):

(XEN) Extended region 0: 0x48000000->0x54000000
(XEN) Extended region 1: 0x58000000->0x60000000
(XEN) Extended region 2: 0x70000000->0x78000000
(XEN) Extended region 3: 0x78200000->0x84000000
(XEN) Extended region 4: 0x86000000->0x8a000000
(XEN) Extended region 5: 0x8c200000->0xc0000000
(XEN) Extended region 6: 0x500000000->0x580000000
(XEN) Extended region 7: 0x600000000->0x680000000
(XEN) Extended region 8: 0x700000000->0x765e00000

Some thoughts regarding that.

1. A few ranges below 4GB are absent in resulting extended regions. I 
assume, this is because of the modules:

(XEN) Checking for initrd in /chosen
(XEN) Initrd 0000000084000040-0000000085effc48
(XEN) RAM: 0000000048000000 - 00000000bfffffff
(XEN) RAM: 0000000500000000 - 000000057fffffff
(XEN) RAM: 0000000600000000 - 000000067fffffff
(XEN) RAM: 0000000700000000 - 000000077fffffff
(XEN)
(XEN) MODULE[0]: 0000000078080000 - 00000000781d74c8 Xen
(XEN) MODULE[1]: 0000000057fe7000 - 0000000057ffd080 Device Tree
(XEN) MODULE[2]: 0000000084000040 - 0000000085effc48 Ramdisk
(XEN) MODULE[3]: 000000008a000000 - 000000008c000000 Kernel
(XEN) MODULE[4]: 000000008c000000 - 000000008c010000 XSM
(XEN)  RESVD[0]: 0000000084000040 - 0000000085effc48
(XEN)  RESVD[1]: 0000000054000000 - 0000000056ffffff

2. Also, it worth mentioning that relatively large chunk (~417MB) of 
memory above 4GB is absent (to be precise, at the end of last RAM bank), 
which I assume, used for Xen internals.
We could really use it for extended regions.
Below free regions in the heap (for last RAM bank) just in case:

(XEN) heap[node=0][zone=23][order=5] 0x00000765ec0000-0x00000765ee0000
(XEN) heap[node=0][zone=23][order=6] 0x00000765e80000-0x00000765ec0000
(XEN) heap[node=0][zone=23][order=7] 0x00000765e00000-0x00000765e80000
(XEN) heap[node=0][zone=23][order=9] 0x00000765c00000-0x00000765e00000
(XEN) heap[node=0][zone=23][order=10] 0x00000765800000-0x00000765c00000
(XEN) heap[node=0][zone=23][order=11] 0x00000765000000-0x00000765800000
(XEN) heap[node=0][zone=23][order=12] 0x00000764000000-0x00000765000000
(XEN) heap[node=0][zone=23][order=14] 0x00000760000000-0x00000764000000
(XEN) heap[node=0][zone=23][order=17] 0x00000740000000-0x00000760000000
(XEN) heap[node=0][zone=23][order=18] 0x00000540000000-0x00000580000000
(XEN) heap[node=0][zone=23][order=18] 0x00000500000000-0x00000540000000
(XEN) heap[node=0][zone=23][order=18] 0x00000640000000-0x00000680000000
(XEN) heap[node=0][zone=23][order=18] 0x00000600000000-0x00000640000000
(XEN) heap[node=0][zone=23][order=18] 0x00000700000000-0x00000740000000

Yes, you already pointed out this disadvantage, so if it is an 
acceptable downside, I am absolutely OK.


3. Common code updates. There is a question how to properly make a 
connection between common allocator internals and Arm's code for 
creating DT. I didn’t come up with anything better
than creating for_each_avail_page() for invoking a callback with page 
and its order.

**********

Below the proposed changes on top of the initial patch, would this be 
acceptable in general?

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 523eb19..1e58fc5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -753,16 +753,33 @@ static int __init add_ext_regions(unsigned long s, 
unsigned long e, void *data)
      return 0;
  }

+static int __init add_unalloc_mem(struct page_info *page, unsigned int 
order,
+                                  void *data)
+{
+    struct rangeset *unalloc_mem = data;
+    paddr_t start, end;
+    int res;
+
+    start = page_to_maddr(page);
+    end = start + pfn_to_paddr(1UL << order);
+    res = rangeset_add_range(unalloc_mem, start, end - 1);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        return res;
+    }
+
+    return 0;
+}
+
  #define EXT_REGION_START   0x40000000ULL
  #define EXT_REGION_END     0x80003fffffffULL

-static int __init find_unallocated_memory(const struct kernel_info *kinfo,
-                                          struct meminfo *ext_regions)
+static int __init find_unallocated_memory(struct meminfo *ext_regions)
  {
-    const struct meminfo *assign_mem = &kinfo->mem;
      struct rangeset *unalloc_mem;
      paddr_t start, end;
-    unsigned int i;
      int res;

      dt_dprintk("Find unallocated memory for extended regions\n");
@@ -771,59 +788,9 @@ static int __init find_unallocated_memory(const 
struct kernel_info *kinfo,
      if ( !unalloc_mem )
          return -ENOMEM;

-    /* Start with all available RAM */
-    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
-    {
-        start = bootinfo.mem.bank[i].start;
-        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
-        res = rangeset_add_range(unalloc_mem, start, end - 1);
-        if ( res )
-        {
-            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
-                   start, end);
-            goto out;
-        }
-    }
-
-    /* Remove RAM assigned to Dom0 */
-    for ( i = 0; i < assign_mem->nr_banks; i++ )
-    {
-        start = assign_mem->bank[i].start;
-        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
-        res = rangeset_remove_range(unalloc_mem, start, end - 1);
-        if ( res )
-        {
-            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
-                   start, end);
-            goto out;
-        }
-    }
-
-    /* Remove reserved-memory regions */
-    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
-    {
-        start = bootinfo.reserved_mem.bank[i].start;
-        end = bootinfo.reserved_mem.bank[i].start +
-            bootinfo.reserved_mem.bank[i].size;
-        res = rangeset_remove_range(unalloc_mem, start, end - 1);
-        if ( res )
-        {
-            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
-                   start, end);
-            goto out;
-        }
-    }
-
-    /* Remove grant table region */
-    start = kinfo->gnttab_start;
-    end = kinfo->gnttab_start + kinfo->gnttab_size;
-    res = rangeset_remove_range(unalloc_mem, start, end - 1);
+    res = for_each_avail_page(add_unalloc_mem, unalloc_mem);
      if ( res )
-    {
-        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
-               start, end);
          goto out;
-    }

      start = EXT_REGION_START;
      end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
@@ -840,8 +807,7 @@ out:
      return res;
  }

-static int __init find_memory_holes(const struct kernel_info *kinfo,
-                                    struct meminfo *ext_regions)
+static int __init find_memory_holes(struct meminfo *ext_regions)
  {
      struct dt_device_node *np;
      struct rangeset *mem_holes;
@@ -961,9 +927,9 @@ static int __init make_hypervisor_node(struct domain *d,
      else
      {
          if ( !is_iommu_enabled(d) )
-            res = find_unallocated_memory(kinfo, ext_regions);
+            res = find_unallocated_memory(ext_regions);
          else
-            res = find_memory_holes(kinfo, ext_regions);
+            res = find_memory_holes(ext_regions);

          if ( res )
              printk(XENLOG_WARNING "Failed to allocate extended 
regions\n");
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 8fad139..7cd1020 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1572,6 +1572,40 @@ static int reserve_heap_page(struct page_info *pg)

  }

+/* TODO heap_lock? */
+int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, 
void *),
+                        void *data)
+{
+    unsigned int node, zone, order;
+    int ret;
+
+    for ( node = 0; node < MAX_NUMNODES; node++ )
+    {
+        if ( !avail[node] )
+            continue;
+
+        for ( zone = 0; zone < NR_ZONES; zone++ )
+        {
+            for ( order = 0; order <= MAX_ORDER; order++ )
+            {
+                struct page_info *head, *tmp;
+
+                if ( page_list_empty(&heap(node, zone, order)) )
+                    continue;
+
+                page_list_for_each_safe ( head, tmp, &heap(node, zone, 
order) )
+                {
+                    ret = cb(head, order, data);
+                    if ( ret )
+                        return ret;
+                }
+            }
+        }
+    }
+
+    return 0;
+}
+
  int offline_page(mfn_t mfn, int broken, uint32_t *status)
  {
      unsigned long old_info = 0;
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 667f9da..64dd3e2 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -123,6 +123,9 @@ unsigned int online_page(mfn_t mfn, uint32_t *status);
  int offline_page(mfn_t mfn, int broken, uint32_t *status);
  int query_page_offline(mfn_t mfn, uint32_t *status);

+int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, 
void *),
+                        void *data);
+
  void heap_init_late(void);

  int assign_pages(


[snip]

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Sat Sep 18 17:00:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 17:00:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189948.339799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRdh9-00050B-S6; Sat, 18 Sep 2021 17:00:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189948.339799; Sat, 18 Sep 2021 17:00:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRdh9-000500-JR; Sat, 18 Sep 2021 17:00:07 +0000
Received: by outflank-mailman (input) for mailman id 189948;
 Sat, 18 Sep 2021 17:00:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRdh8-0004yc-LC; Sat, 18 Sep 2021 17:00:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRdh8-0002m5-Gv; Sat, 18 Sep 2021 17:00:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRdh8-0003nC-6g; Sat, 18 Sep 2021 17:00:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRdh8-0007Rd-6H; Sat, 18 Sep 2021 17:00:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9YDLe3zSOz/wqsq8MxIiyr45rLHlLjVukd6OBbg1yjA=; b=ulYmzxy5H4ca2a5kqOpo9m6uRR
	ewc+q3KkzizuudTBmgNIjJnvvKbNR/Y9ohS2OVF5tEp5g1f+hERkvQH3AmvZGHVsC82IDJRCH3tEe
	1TRMqkO3WlZa4ADK5/CX5VF9wYXUDbmk6+O2StSrnRYlTnD90JDaN7qqqyV9KIkwE17g=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165107-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165107: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-vhd:<job status>:broken:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-seattle:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-arm64-arm64-xl-vhd:host-install(5):broken:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:xen-boot:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4357f03d6611753936e4d52fc251b54a6afb1b54
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 17:00:06 +0000

flight 165107 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165107/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-vhd         <job status>                 broken
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot                 fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd       5 host-install(5)       broken baseline untested
 test-arm64-arm64-libvirt-raw  8 xen-boot                fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4357f03d6611753936e4d52fc251b54a6afb1b54
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  413 days
Failing since        152366  2020-08-01 20:49:34 Z  412 days  719 attempts
Testing same since   165107  2021-09-18 07:14:05 Z    0 days    1 attempts

------------------------------------------------------------
7317 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      broken  
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-arm64-arm64-xl-vhd broken
broken-step test-arm64-arm64-xl-vhd host-install(5)

Not pushing.

(No revision log; it would be 2261040 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 17:43:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 17:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189938.339812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mReN4-0001Ut-5X; Sat, 18 Sep 2021 17:43:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189938.339812; Sat, 18 Sep 2021 17:43:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mReN4-0001Um-0y; Sat, 18 Sep 2021 17:43:26 +0000
Received: by outflank-mailman (input) for mailman id 189938;
 Sat, 18 Sep 2021 16:05:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZxcA=OI=oracle.com=alec.r.brown@srs-us1.protection.inumbo.net>)
 id 1mRcpr-00071J-KH
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 16:05:03 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2df376b8-189a-11ec-b7af-12813bfff9fa;
 Sat, 18 Sep 2021 16:05:01 +0000 (UTC)
Received: from pps.filterd (m0246631.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18IFV1CH008193; 
 Sat, 18 Sep 2021 16:04:18 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b56w295na-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Sat, 18 Sep 2021 16:04:17 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18IG01oF100752;
 Sat, 18 Sep 2021 16:04:16 GMT
Received: from nam12-bn8-obe.outbound.protection.outlook.com
 (mail-bn8nam12lp2171.outbound.protection.outlook.com [104.47.55.171])
 by userp3020.oracle.com with ESMTP id 3b57ek8p4u-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Sat, 18 Sep 2021 16:04:16 +0000
Received: from DM6PR10MB2986.namprd10.prod.outlook.com (2603:10b6:5:6b::26) by
 DM6PR10MB4299.namprd10.prod.outlook.com (2603:10b6:5:216::11) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14; Sat, 18 Sep 2021 16:04:13 +0000
Received: from DM6PR10MB2986.namprd10.prod.outlook.com
 ([fe80::8974:894e:1c4b:35d0]) by DM6PR10MB2986.namprd10.prod.outlook.com
 ([fe80::8974:894e:1c4b:35d0%7]) with mapi id 15.20.4500.014; Sat, 18 Sep 2021
 16:04:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2df376b8-189a-11ec-b7af-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc :
 subject : date : message-id : content-type : content-transfer-encoding :
 mime-version; s=corp-2021-07-09;
 bh=xbrB7To9cV7w66tQJVgfX7XDn/uoZN2YFAi+ilQyqMY=;
 b=MgiwZs0z2KHWYNekBa6uzgbRCb0+lDcbNCxELCxvelEVV2CNo8cV0wCZd6KS2lVcyq4g
 TDY4kShlSiSUzzngJQ9jYWxghnlUM9axaiUO6Oh7PjSoNuT+VnEJcwu0BSDQJEBk/wwg
 mbOUOwgh9Ks7uniqKLBcR0ExDarwTV1TK8YYnGJ8KVEKBYt9bqTMKPircWXKTpqr6bQs
 6n+sDWu7/gzDI2QCPbY5wenFlAF23i+5tzL+zBSFA0Y0zPMO4dvsy10tkD5CwQUaL2Ma
 R/Nnpx/l0ph/pEIi5/6qPuF5fTBswYC73Vm58aQbggKJ/OLsyIC/Y5MF6/zNUX6l02Zw ig== 
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc :
 subject : date : message-id : content-type : content-transfer-encoding :
 mime-version; s=corp-2020-01-29;
 bh=xbrB7To9cV7w66tQJVgfX7XDn/uoZN2YFAi+ilQyqMY=;
 b=xXa4pr1tsCr9NEuRLBKi/kKEovBLb6yyl0nNUPT2dkLY4nCI1kGDxYDy01zlQtOKl2B1
 JVDLX/t1AS3gccF8gbu3f++be3Cs+zSSQCFck7QvwJED+b8PyXJ3tuXkSNSX2Ev7htge
 QrFKCTOWDXZ5zAr5kPq4lJWHws3SWWw0k637rjhIcnkQ7CLsZsMRgVrAIWzBmTzONOdE
 kZDRNWIhh565TdJZmCYgX5MltSnSOT6tlaMVspbcaKX4ygjysnJNtNIDnyMQtw04Nrjv
 L++/c2ngqDbKYtULngtLgz+d9ixuvRXJKSKaIiGpsz3A7E+mMsdhK/hrALgloufBsAqA vQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kFoP/wqpMqTCqdObRP9HZtgbfW4mjeujv+l3lEZzKqfJWbj1LhpF0Zt1wrU3GvhlYsHunm9feUJd/tlggjvEJi8Tw5xn3D+z/hy3QPRT4V63PGSAx1HfwwPIvkatGAbUkjIPsRQUNqHJMuRggQqIOO4xV+CLcQnLV2jiD5Tt7YgxOIufYRWTj3E1xFyGSySK2sx0t5/jTxZj0pVlQ+2nJlYy5B0mB9o2AV1mWJMuats72T7z1lQHDeDmQmDP8i7rd3NWy7cCzlBn56mnXio8MuSogTXaY0UTmwlTACDp7tpeooUJeVUna0Aa4/+AsxA2weUV5j/CWIueBPn3vBKTOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xbrB7To9cV7w66tQJVgfX7XDn/uoZN2YFAi+ilQyqMY=;
 b=KmCTcgaERDqyqfsjiEOeKkmnXj4J8LQMIAELN0Mq2nGKTrd5EzIhrqz9ai4AcRk4B0ufpDG2xvNXJo3wqSCMsIR4QL6ZwLik26DlXHwVY1w7TMNd9pjt3rReqvihX+cx0Z4QWutAK7jAA7p5kmiGoc0FKQFCPpHR6NzQwenc7cfoEnYMibBlHxOhWr5LwjZVlSw4yHQMPX9yUmkpTRmhDbFzEJ8Rz6on7cV0ZeZge8TGV6DEFrUvBSzKz2fwh587caenNSZXnxOOKRpJIN/m3j+B3NFs35hXU8+ELUgK6Me09+oST0kPzWLLwWlQkgzqWt8k2dtZZPBoWaOOkjAEJA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xbrB7To9cV7w66tQJVgfX7XDn/uoZN2YFAi+ilQyqMY=;
 b=J00NYJ9PjDv86A6oRcsPXkbDO2la+ss8rFddUIqy/ZKDFTo3f2NUVUhqJIRKRUHNco3X1dghm7SsC6ZlIYsoj18dgQuRG6J7XEKea3l9xv2u+7Dc3rd3DkkO7zY/P4AkBVeFLxX6166tZ36HhgYWtllbW2+H4YxETTsWQh7rcH8=
From: Alec Brown <alec.r.brown@oracle.com>
To: "coreboot@coreboot.org" <coreboot@coreboot.org>,
        "grub-devel@gnu.org"
	<grub-devel@gnu.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        "systemd-devel@lists.freedesktop.org"
	<systemd-devel@lists.freedesktop.org>,
        "trenchboot-devel@googlegroups.com"
	<trenchboot-devel@googlegroups.com>,
        "u-boot@lists.denx.de"
	<u-boot@lists.denx.de>,
        "x86@kernel.org" <x86@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Alec Brown <alec.r.brown@oracle.com>,
        Aleksandr Burmashev
	<alexander.burmashev@oracle.com>,
        "allen.cryptic@gmail.com"
	<allen.cryptic@gmail.com>,
        "andrew.cooper3@citrix.com"
	<andrew.cooper3@citrix.com>,
        "andy.shevchenko@gmail.com"
	<andy.shevchenko@gmail.com>,
        "ardb@kernel.org" <ardb@kernel.org>,
        "btrotter@gmail.com" <btrotter@gmail.com>,
        Daniel Kiper
	<daniel.kiper@oracle.com>,
        "dpsmith@apertussolutions.com"
	<dpsmith@apertussolutions.com>,
        Eric DeVolder <eric.devolder@oracle.com>,
        Eric Snowberg <eric.snowberg@oracle.com>,
        "frowand.list@gmail.com"
	<frowand.list@gmail.com>,
        "hpa@zytor.com" <hpa@zytor.com>,
        "hun@n-dimensional.de" <hun@n-dimensional.de>,
        "james.dutton@gmail.com"
	<james.dutton@gmail.com>,
        "javierm@redhat.com" <javierm@redhat.com>,
        Joao
 Martins <joao.m.martins@oracle.com>,
        "jwerner@chromium.org"
	<jwerner@chromium.org>,
        Kanth Ghatraju <kanth.ghatraju@oracle.com>,
        Konrad
 Wilk <konrad.wilk@oracle.com>,
        "krystian.hebel@3mdeb.com"
	<krystian.hebel@3mdeb.com>,
        "leif@nuviainc.com" <leif@nuviainc.com>,
        "lukasz.hawrylko@intel.com" <lukasz.hawrylko@intel.com>,
        "luto@amacapital.net" <luto@amacapital.net>,
        "michal.zygowski@3mdeb.com"
	<michal.zygowski@3mdeb.com>,
        "mjg59@google.com" <mjg59@google.com>,
        "mtottenh@akamai.com" <mtottenh@akamai.com>,
        "nico.h@gmx.de" <nico.h@gmx.de>,
        "phcoder@gmail.com" <phcoder@gmail.com>,
        "piotr.krol@3mdeb.com"
	<piotr.krol@3mdeb.com>,
        "pjones@redhat.com" <pjones@redhat.com>,
        "pmenzel@molgen.mpg.de" <pmenzel@molgen.mpg.de>,
        "rasmus.villemoes@prevas.dk"
	<rasmus.villemoes@prevas.dk>,
        "rdunlap@infradead.org"
	<rdunlap@infradead.org>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Ross
 Philipson <ross.philipson@oracle.com>,
        "sjg@chromium.org" <sjg@chromium.org>,
        "trini@konsulko.com" <trini@konsulko.com>,
        "tyhicks@linux.microsoft.com"
	<tyhicks@linux.microsoft.com>,
        "ulrich.windl@rz.uni-regensburg.de"
	<ulrich.windl@rz.uni-regensburg.de>,
        "wvervoorn@eltan.com"
	<wvervoorn@eltan.com>,
        "xypron.glpk@gmx.de" <xypron.glpk@gmx.de>,
        "rharwood@redhat.com" <rharwood@redhat.com>
Subject: [SPECIFICATION RFC v3] The firmware and bootloader log specification
Thread-Topic: [SPECIFICATION RFC v3] The firmware and bootloader log
 specification
Thread-Index: Adeso0zdR7rpvUcgTOKp4wEL6mxamA==
Date: Sat, 18 Sep 2021 16:04:13 +0000
Message-ID: 
 <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: coreboot.org; dkim=none (message not signed)
 header.d=none;coreboot.org; dmarc=none action=none header.from=oracle.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 92a6c0a8-fbd8-45b6-3a09-08d97abdf59a
x-ms-traffictypediagnostic: DM6PR10MB4299:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <DM6PR10MB42998D247186AA97716DB268BCDE9@DM6PR10MB4299.namprd10.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 +XeejkzqCerWKQ4kFG7sw+qGeW6H7YE4gqehBjPAbztFx9u74MMmmqwQGmyC818XJSHq40Zqaau/BzPmOOlLX9QQxBwFkO8jRVfcvQRZPcULuk3UpwbhP4zQJRVe0VwGvxoYHocuLlxtwz9R4uGT0Q1Le7Vr9gErz2n3X1eECVbAwh0jUTL9MOo1BmIsN6JJbSaDyZO6fFuBkseJ8QOvoAL5mn6ZQDXDgjTD1m8qlgSEiqcwSIRIjRZyURUPWP2UW1f+azuc4h2tazdOsOmf3lNH7KoCf2iH6xiuxhqgsNNzRsl4kXktnjtrphC9J7oxChDLdOHSXT4OKaFdRNx6ugocAfAt7ttJzFcLcR59v3+ySR/JRwIAjnwcX71PZ1MDpF9W+Lh8HCDY7Gu1MlBgfuLOGdkASQgsrU9Yw/5l9kjpcc8r7OGYDiQp0nsuaIE+acw9uz7EFqXzqNmmtK4mAdM9YlDH0Ww5l5zQALhOFBi3KEjYUTAn5V8N6Ra8O+dXWtdyrf3e5zdDwuPA5hGIJDg10F4Tg/wK8eYi213Bmu6UuHpAR6g16UJGD80Rk1wxOxYgWCsk5+ZtKlNqMxf+juI94a48mKh9+TpzEo1q8AjNHLOh7yIM8BFh/hW5VIsmqxGFtWxGmuhCRzFQCFGaYTy7degb0YaO7xSGylFDk1IyTRZlPSpf5RfBUZG+Z7i86n60CWkIhO0pOEk+KAqdtIa3GJMVENNVbXuCHOS+lSOeNA+xSp8Url/pe1b1zzlNxM6scvAV1r68izlRVF/7ewh7xVUpOujLAD/xgPYLPh63GJSxz5meoeCxZ1XL4tYag57xnZFkyE/fgBlpqYD3Cg==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR10MB2986.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7416002)(8676002)(54906003)(122000001)(186003)(64756008)(76116006)(66556008)(66476007)(83380400001)(508600001)(66946007)(7696005)(6506007)(316002)(71200400001)(38100700002)(26005)(66446008)(38070700005)(7406005)(2906002)(5660300002)(110136005)(86362001)(966005)(9686003)(33656002)(52536014)(55016002)(8936002)(4326008);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?us-ascii?Q?IBEYD651bZBh5kYu6KQAC3eBK5Ec5wXHi2aWQgoRNC6gPRPk7/I9whbMWa9g?=
 =?us-ascii?Q?rojXotSQxAmmk277+XwBpOh9jJ5VlJ820R1EPzd+SNODtFtWdwHV0QsEiA7/?=
 =?us-ascii?Q?CvOyCgrT86WBHCS3gdlfzQp1PYknUOSP02AjJlfxOZhpAO+AccYwje7Jni4u?=
 =?us-ascii?Q?ZMt01gqrEjVjamo8Sg4y75MIK9w6ts9aUvN0mGvSWOXKkuIJvUI20d7JXHKg?=
 =?us-ascii?Q?AjdT9idJrsL21R0aMWh2arpIyieNXjAdKp55suljiPUHlZbqI4A2jtdU7Ikw?=
 =?us-ascii?Q?nyuzHU2oqXT5QRgi4XTLJY+XVUVjlgX8ZImtT479VDdrVvA1qezOdTnlNRz5?=
 =?us-ascii?Q?M9eCxZz6+4hE0vB0477W6ZzuQjSilVF5Wbfz/4wBjZmrTCs9HnvrLQAUQoeJ?=
 =?us-ascii?Q?NgiYY/oLz9D4iV62YmJKkszUZa7Q/rQ+697BCqGadfAqspymlQ4D9HTIJ4qM?=
 =?us-ascii?Q?Qlyd+emaHMZYrHiHL8gG6XK6NuTIBRJboDe9pOIjmp5BY5WLiYMDxpAuv56C?=
 =?us-ascii?Q?YF99JGFVpxXZD9aft4l/cJw6mM79cE2iOl4u0qLLUhHpCeICWMi5dTEw8+hc?=
 =?us-ascii?Q?oMmPNCODwYEvcYZd1ffxnhnz9UpzRLfkuKZG4k5SvH8J0NaPDg9NSUsQf02P?=
 =?us-ascii?Q?vNpmx6ECRDdnTojSb8tcuMa0/zaF0+1s8ZfphVFkSykaTBd5DCybd2PeXCli?=
 =?us-ascii?Q?YNrxFM1+2wuYZKOvrU+kkcdYbqVby0S5LL8OaMlliT3rqDf6br8hv+HHyE6f?=
 =?us-ascii?Q?r73YWRQcmIW3QdWC5aSgIPw2ZzynWyEQ7vSYca2Epj26GWqR4hrYZwzzmu0x?=
 =?us-ascii?Q?OjZDUsPXswjZpkMSVdeXfzGYbu4VCr8D9O7+0DOs8xCeFWK7KSbN5VanQABu?=
 =?us-ascii?Q?g1i6x6DAlXFCkmRPgOwPMwLeMjs/rij/yyT2shHobugGoovGN/IY9ZtsR12w?=
 =?us-ascii?Q?iQd6v7HZQKcCSnrGpGJ6sV2dbv4drtThMbKX5P+nSsL38l1fr0wbeNDwxoPT?=
 =?us-ascii?Q?CqMFv/ZPTy4ejdZGdMjcwSsMoOa8y4rcYhtMRq9Tlx9v4XOzAWfJ+6rx2rF7?=
 =?us-ascii?Q?fftA6zAmVojq8JS5h1AVoQeU2tpgWTWy3hMSW+wJic2rjm8oYQdnywVJjqQj?=
 =?us-ascii?Q?QBuEnoN18B+UFa82aKOA+q7oecP+gm9vp+M216IDHWHGWe1lqbCDl4NPZS+F?=
 =?us-ascii?Q?R2UrjqQSWO89PtvEt7/cJoQYA9Uu7BPZA4JIqFIqcqssQRB0FTbAuOQ3CCzD?=
 =?us-ascii?Q?6lpjRLP/UiZefG6xK5aAfLLMXPDb2lY1W7fDhR1r/8Rt059KOjnU/k+u/uKa?=
 =?us-ascii?Q?fnE=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB2986.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 92a6c0a8-fbd8-45b6-3a09-08d97abdf59a
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Sep 2021 16:04:13.5360
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: XgjUFwIwcbFqVjxgocOeNHvJXRb3scmDXxPPxaAQ6OPOrHO7LrA8AtEGh3ogqrn22xAorrG81pVCKqd5DLRDQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR10MB4299
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10111 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 malwarescore=0
 mlxlogscore=999 adultscore=0 bulkscore=0 phishscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109180114
X-Proofpoint-ORIG-GUID: BzC4zYK0BxX4Eu679X5ckd50UaCuzhOC
X-Proofpoint-GUID: BzC4zYK0BxX4Eu679X5ckd50UaCuzhOC

Hi everyone,

I've been working on improving the specification for the firmware and bootl=
oader
log that Daniel Kiper has proposed and take into account most of the sugges=
tions
that were made in these threads [1], [2].

The goal is to allow various boot components to pass logs to the running OS=
 and
then to the user space for processing and analysis. These logs should be ge=
neric
enough so that it can work in multiple environments and be human readable.

It has yet to be decided where to put the final version of this specificati=
on.
It should be merged into an existing specification, e.g. UEFI, ACPI, Multib=
oot2,
or be standalone, such as a part of OASIS Standards.

Below is how the layout of these logs would store their data.

bf_log_header:
               +-------------------+
u32            | version           |
u32            | size              |
u8[64]         | producer          |
u8[64]         | log_format        |
u64            | flags             |
u64            | next_bflh_addr    |
u64            | log_addr          |
u32            | log_size          |
               +-------------------+

bf_log_buffer:
               +-------------------+
u32            | version           |
u32            | size              |
u8[64]         | producer          |
u32            | next_msg_off      |
bf_log_msg[l]  | msgs              |
               +-------------------+

bf_log_msg:
               +-------------------+
u32            | size              |
u64            | ts_nsec           |
u32            | level             |
u32            | facility          |
u32            | msg_off           |
u8[n]          | type              |
u8[m]          | msg               |
               +-------------------+

Where l is the number of msgs, n is the size of type, and m is the size of =
the
msg.

The bf_log_header structure forms a linked list. Each bf_log_header element=
 in
the linked list points to the individual log buffer and the next bf_log_hea=
der
element in the linked list. The first element in the linked list points to =
the
last boot component in the boot chain. The last element points to the start=
ing
boot component in the boot chain. The log buffers which contain the log
messages are produced by the various boot components, typically from the
firmware to the bootloader. The log message is stored in a log format that =
is
compatible with the boot component that produced it.

The fields in bf_log_header structure:
  - version: the firmware and bootloader log header version number, 1 for n=
ow,
  - size: the size of the bf_log_header to allow for backward compatibility=
 if=20
    other fields are added,
  - producer: the producer/firmware/bootloader/... entity, NUL terminated
    string, e.g. GRUB, Coreboot; the length allows for ASCII UUID storage,
  - log_format: the format used to record the log messages, NUL terminated
    string, e.g. bf_log_msg, cbmem_cons, etc.; various producers may genera=
te
    logs in various formats if needed,
  - flags: bit field used to store information about the log state, if bit =
0 has
    been set it means the log was truncated,
  - next_bflh_addr: the physical address of the next bf_log_header structur=
e,
    none if zero,
  - log_addr: the physical address of where the log buffer is stored,
  - log_size: the total size of the log buffer.

The bf_log_buffer is used to store log messages from the firmware and
bootloader. This format for storing messages is called the bf log format. T=
he
bf_log_buffer contains the header information of the bf log format with the=
 log
messages being stored in an array of bf_log_msg messages.

The fields in bf_log_buffer structure:
  - version: the firmware and bootloader log version number, 1 for now,
  - size: the total allocated space for the bf_log_buffer including the log
    messages stored in msgs,
  - producer: the producer/firmware/bootloader/... entity, NUL terminated
    string, e.g. GRUB, Coreboot; the length allows for ASCII UUID storage; =
same
    as the field in bf_log_header,
  - next_msg_off: the byte offset from the beginning of the allocated space=
 for
    bf_log_buffer to the next byte after the last bf_log_msg in msgs,
  - msgs: the array of log messages stored in the bf_log_msg structures.

The fields in bf_log_msg structure:
  - size: the total size of the bf_log_msg entry,
  - ts_nsec: the timestamp in nanoseconds starting from 0 (zero); the produ=
cer
    using this log format defines the meaning of 0,
  - level: similar to the syslog meaning; used to differentiate normal log
    messages from debug log messages, but the exact interpretation depends =
on
    the producer,
  - facility: similar to the syslog meaning; used to differentiate the sour=
ces
    of the log messages, but the exact interpretation depends on the produc=
er,
  - msg_off: the byte offset which the msg field starts in bf_log_msg,
  - type: the log message type; similar to facility but NUL terminated stri=
ng
    instead of integer, but the exact interpretation depends on the produce=
r,
  - msg: the log message, NUL terminated string.

In bf_log_msg, the producers are free to use or ignore any of the level,
facility, and type fields. If level or facility are ignored, they should be=
 set
to 0. If type is ignored, it should be set to an empty NUL terminated strin=
g.

Since it doesn't seem possible to have each boot component using the same l=
og
format, we added a log_format and log_phys_addr fields to give flexibility =
in
how logs are stored. An example of a different log format that can be used =
is
the cbmem_console log format used by coreboot:

cbmem_console:
               +-------------------+
u32            | size              |
u32            | cursor            |
u8[m]          | body              |
               +-------------------+

There is still the outstanding issue of how the logs will be sent to the OS=
. If
UEFI is used, we can use config tables. If ACPI or Device Tree is used, we =
can
use bf_log_header.next_bflh_addr to present the logs. If none of these plat=
forms
are used, it becomes a lot trickier to solve this issue.

Any suggestions are much appreciated and will be taken into consideration.

I will be presenting this work at the LPC System Boot and Security
Micro-conference on the 22nd of September at 7:50 AM PDT (14:50 UTC). Come =
and
join if you want to discuss the design. The schedule for the System Boot an=
d
Security Micro-conference can be found here [3].

Thanks!
Alec Brown

[1] https://lists.gnu.org/archive/html/grub-devel/2020-11/msg00100.html
[2] https://lists.gnu.org/archive/html/grub-devel/2020-12/msg00021.html
[3] https://linuxplumbersconf.org/event/11/sessions/116/#20210922


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 22:54:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 22:54:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189974.339823 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRjDf-0000aa-ST; Sat, 18 Sep 2021 22:54:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189974.339823; Sat, 18 Sep 2021 22:54:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRjDf-0000aT-P4; Sat, 18 Sep 2021 22:54:03 +0000
Received: by outflank-mailman (input) for mailman id 189974;
 Sat, 18 Sep 2021 22:54:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/AP9=OI=gmx.de=xypron.glpk@srs-us1.protection.inumbo.net>)
 id 1mRjDe-0000aN-VK
 for xen-devel@lists.xenproject.org; Sat, 18 Sep 2021 22:54:03 +0000
Received: from mout.gmx.net (unknown [212.227.17.21])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 50143a0a-18d3-11ec-b7e2-12813bfff9fa;
 Sat, 18 Sep 2021 22:54:01 +0000 (UTC)
Received: from [127.0.0.1] ([89.15.236.83]) by mail.gmx.net (mrgmx104
 [212.227.17.168]) with ESMTPSA (Nemesis) id 1Mgeo8-1n6ZCm3Vc4-00h86f; Sun, 19
 Sep 2021 00:53:45 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50143a0a-18d3-11ec-b7e2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net;
	s=badeba3b8450; t=1632005625;
	bh=Pwh/KmGzeiwrqQm3C9sI0E5AfSnePPH3AAuooUuBTY4=;
	h=X-UI-Sender-Class:Date:From:To:CC:Subject:In-Reply-To:References;
	b=AfrHCn31S00A+m3IQ6CvlxYlpRX53Lkbs/Ru5BY+abRvw9V27s/cMSqh8HlJehfUN
	 JaNmhUhyb83dCO1Hnvs6OzcqXhrmMoW/TE0jmkTExd53CesKdb9Rv4BIOY/gHD+kcV
	 GvwM/PUhuEEwFp8yIzyfDrAEzDQaSohI5b2gFwzc=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Date: Sun, 19 Sep 2021 00:53:35 +0200
From: Heinrich  Schuchardt <xypron.glpk@gmx.de>
To: Alec Brown <alec.r.brown@oracle.com>,
 "coreboot@coreboot.org" <coreboot@coreboot.org>,
 "grub-devel@gnu.org" <grub-devel@gnu.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "systemd-devel@lists.freedesktop.org" <systemd-devel@lists.freedesktop.org>,
 "trenchboot-devel@googlegroups.com" <trenchboot-devel@googlegroups.com>,
 "u-boot@lists.denx.de" <u-boot@lists.denx.de>,
 "x86@kernel.org" <x86@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Aleksandr Burmashev <alexander.burmashev@oracle.com>,
 "allen.cryptic@gmail.com" <allen.cryptic@gmail.com>,
 "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
 "andy.shevchenko@gmail.com" <andy.shevchenko@gmail.com>,
 "ardb@kernel.org" <ardb@kernel.org>,
 "btrotter@gmail.com" <btrotter@gmail.com>,
 Daniel Kiper <daniel.kiper@oracle.com>,
 "dpsmith@apertussolutions.com" <dpsmith@apertussolutions.com>,
 Eric DeVolder <eric.devolder@oracle.com>,
 Eric Snowberg <eric.snowberg@oracle.com>,
 "frowand.list@gmail.com" <frowand.list@gmail.com>,
 "hpa@zytor.com" <hpa@zytor.com>,
 "hun@n-dimensional.de" <hun@n-dimensional.de>,
 "james.dutton@gmail.com" <james.dutton@gmail.com>,
 "javierm@redhat.com" <javierm@redhat.com>,
 Joao Martins <joao.m.martins@oracle.com>,
 "jwerner@chromium.org" <jwerner@chromium.org>,
 Kanth Ghatraju <kanth.ghatraju@oracle.com>,
 Konrad Wilk <konrad.wilk@oracle.com>,
 "krystian.hebel@3mdeb.com" <krystian.hebel@3mdeb.com>,
 "leif@nuviainc.com" <leif@nuviainc.com>,
 "lukasz.hawrylko@intel.com" <lukasz.hawrylko@intel.com>,
 "luto@amacapital.net" <luto@amacapital.net>,
 "michal.zygowski@3mdeb.com" <michal.zygowski@3mdeb.com>,
 "mjg59@google.com" <mjg59@google.com>,
 "mtottenh@akamai.com" <mtottenh@akamai.com>, "nico.h@gmx.de" <nico.h@gmx.de>,
 "phcoder@gmail.com" <phcoder@gmail.com>,
 "piotr.krol@3mdeb.com" <piotr.krol@3mdeb.com>,
 "pjones@redhat.com" <pjones@redhat.com>,
 "pmenzel@molgen.mpg.de" <pmenzel@molgen.mpg.de>,
 "rasmus.villemoes@prevas.dk" <rasmus.villemoes@prevas.dk>,
 "rdunlap@infradead.org" <rdunlap@infradead.org>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Ross Philipson <ross.philipson@oracle.com>,
 "sjg@chromium.org" <sjg@chromium.org>,
 "trini@konsulko.com" <trini@konsulko.com>,
 "tyhicks@linux.microsoft.com" <tyhicks@linux.microsoft.com>,
 "ulrich.windl@rz.uni-regensburg.de" <ulrich.windl@rz.uni-regensburg.de>,
 "wvervoorn@eltan.com" <wvervoorn@eltan.com>,
 "rharwood@redhat.com" <rharwood@redhat.com>
Subject: =?US-ASCII?Q?Re=3A_=5BSPECIFICATION_RFC_v3=5D_The_firmw?= =?US-ASCII?Q?are_and_bootloader_log_specification?=
User-Agent: K-9 Mail for Android
In-Reply-To: <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>
References: <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>
Message-ID: <A7F710D3-5148-4E92-9E3D-5D850AD0245F@gmx.de>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:p+pgeoo6oBsAP4kAPcHthdyoXpZ4AZYVoI5rLrwKC/Udvo8IvyC
 9B0KDNX92DA77n0Iq/jNAwdnMSZ4ZBx9tN8lOSZfSACpGaqq5KS7YKa0+1xIPBL02PF4Xui
 yl/ffVYfKSD5IF5H/4RJMjiFO8h3Z0d73xYMJiRgqEWPf/GH4VkK9yuHTfh7ibu6tQ2g7ET
 OCUlrIKDVm9LiXYsCD48Q==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:1uRDmIY6xbk=:xsJ4Qt3Clc/AmnfVmjW15V
 mnfp54ZfezrJamNK/0LSMX2ZRC0GVYPIUt0YTHrdS4GRGbTzOhXHfSrB3srhlBcDHi3IKjpQM
 ZYPAFeDLGtBZsH2J1o1jC+C916gcziGLaDmYW0Q7Hk3YtEhcorqg2JqfmA7NsCZJF9yyev0ks
 PaSvUW2LsEiL/nxF27y8kWIWsqqMZ3Vpnp3ZhaBU/hLrcj4dn+EKkcImtVmz7AgOj3zEjg0ia
 eYU86/Ud6AUbUFIB8l74tZ3v6MHFXRmALSAUONJPe3sGykkaY95WwDTt/b7OlRJlQVIcavw2O
 WtjdjHojv4Xfwdfe/Hmv27YwHxs5N7eYlKk/J84J/XV0QK5zZQ9BQqWhV1TP3DeZLNWTn5bnL
 XtDHpRNwYuKZB9Jdoj5o6eVT8gxPDPW2PoBzW5DGreXhHiYzvacXI1BDZy36r4xj8hu8aYPkI
 rLXraOitFx7/8Yh+BUHA/+xBbrbhO+9Pw5WjavxWAN2ojK8UcB2NE4qwAX6CK+0OoaRnqtUjG
 XYWlX3T0mbFi7nPmpPolWbItPTGbpRIeOPwxs0exR16IfxBpmYvXm0kAoYfcl+niZVQybx3Ki
 Z/Z53jxyIcTDQAxH/lHEYL32U6PNFA6gIg7x/Mhfsb3RNgOsCX/4SFsoB7p5hM8kqdLNu2JHn
 inYsrnfpNFprGhfYFsP54/gJayFFV7CYeSCaDi09oo07JWt0jYlcBHq4ZAPPuflAOI0HeXQLo
 r1jpX/YtxWHzW4CyqPSpXMn1PNrLJ+Cl5PU1OjlJ2KD5lzwdqWfEt+Q2+r+xzfpTKz/hBiY7T
 OT0c3DfNf/DWBEv/zOH0RheBwlIselptrIybSc03c3jThRGibdINSSAaH/vHpGfnxwPYp4K/6
 UcvOx8GGN9kzQOjHg3Xn33SQOtCRsm/QXTNZ7PsOZarnUlvsh6rVJt6JzTQmdPUyBQKWJU4NS
 +dplF3ZPX88ILFf/pIcWPxtDXTauS2TDHUwtYYd72EENou8XI+/7lmGl8AU9qF1uDhBZ7t5qQ
 t2KOCJ0Jc5RZtWhDHljvtWR6wtzvOaD8EvowmWgikvTIrkA7h2POZtZZuKt1b81/ZkDoodZMR
 YDx8rKGb1NOqA0=



Am 18=2E September 2021 18:04:13 MESZ schrieb Alec Brown <alec=2Er=2Ebrown=
@oracle=2Ecom>:
>Hi everyone,
>
>I've been working on improving the specification for the firmware and boo=
tloader
>log that Daniel Kiper has proposed and take into account most of the sugg=
estions
>that were made in these threads [1], [2]=2E
>
>The goal is to allow various boot components to pass logs to the running =
OS and
>then to the user space for processing and analysis=2E These logs should b=
e generic
>enough so that it can work in multiple environments and be human readable=
=2E

Hello Alec,

in your mail it remains unclear which information you want to put into the=
 log and why it is needed=2E I would prefer the motivation and content to b=
e clarified before defining any interface structures=2E

We already the EFI_TCG2_PROTOCOL and RFC 5424 (The syslog protocol)=2E Why=
 do we need to start from scratch?

Best regards

Heinrich=20

>
>It has yet to be decided where to put the final version of this specifica=
tion=2E
>It should be merged into an existing specification, e=2Eg=2E UEFI, ACPI, =
Multiboot2,
>or be standalone, such as a part of OASIS Standards=2E
>
>Below is how the layout of these logs would store their data=2E
>
>bf_log_header:
>               +-------------------+
>u32            | version           |
>u32            | size              |
>u8[64]         | producer          |
>u8[64]         | log_format        |
>u64            | flags             |
>u64            | next_bflh_addr    |
>u64            | log_addr          |
>u32            | log_size          |
>               +-------------------+
>
>bf_log_buffer:
>               +-------------------+
>u32            | version           |
>u32            | size              |
>u8[64]         | producer          |
>u32            | next_msg_off      |
>bf_log_msg[l]  | msgs              |
>               +-------------------+
>
>bf_log_msg:
>               +-------------------+
>u32            | size              |
>u64            | ts_nsec           |
>u32            | level             |
>u32            | facility          |
>u32            | msg_off           |
>u8[n]          | type              |
>u8[m]          | msg               |
>               +-------------------+
>
>Where l is the number of msgs, n is the size of type, and m is the size o=
f the
>msg=2E
>
>The bf_log_header structure forms a linked list=2E Each bf_log_header ele=
ment in
>the linked list points to the individual log buffer and the next bf_log_h=
eader
>element in the linked list=2E The first element in the linked list points=
 to the
>last boot component in the boot chain=2E The last element points to the s=
tarting
>boot component in the boot chain=2E The log buffers which contain the log
>messages are produced by the various boot components, typically from the
>firmware to the bootloader=2E The log message is stored in a log format t=
hat is
>compatible with the boot component that produced it=2E
>
>The fields in bf_log_header structure:
>  - version: the firmware and bootloader log header version number, 1 for=
 now,
>  - size: the size of the bf_log_header to allow for backward compatibili=
ty if=20
>    other fields are added,
>  - producer: the producer/firmware/bootloader/=2E=2E=2E entity, NUL term=
inated
>    string, e=2Eg=2E GRUB, Coreboot; the length allows for ASCII UUID sto=
rage,
>  - log_format: the format used to record the log messages, NUL terminate=
d
>    string, e=2Eg=2E bf_log_msg, cbmem_cons, etc=2E; various producers ma=
y generate
>    logs in various formats if needed,
>  - flags: bit field used to store information about the log state, if bi=
t 0 has
>    been set it means the log was truncated,
>  - next_bflh_addr: the physical address of the next bf_log_header struct=
ure,
>    none if zero,
>  - log_addr: the physical address of where the log buffer is stored,
>  - log_size: the total size of the log buffer=2E
>
>The bf_log_buffer is used to store log messages from the firmware and
>bootloader=2E This format for storing messages is called the bf log forma=
t=2E The
>bf_log_buffer contains the header information of the bf log format with t=
he log
>messages being stored in an array of bf_log_msg messages=2E
>
>The fields in bf_log_buffer structure:
>  - version: the firmware and bootloader log version number, 1 for now,
>  - size: the total allocated space for the bf_log_buffer including the l=
og
>    messages stored in msgs,
>  - producer: the producer/firmware/bootloader/=2E=2E=2E entity, NUL term=
inated
>    string, e=2Eg=2E GRUB, Coreboot; the length allows for ASCII UUID sto=
rage; same
>    as the field in bf_log_header,
>  - next_msg_off: the byte offset from the beginning of the allocated spa=
ce for
>    bf_log_buffer to the next byte after the last bf_log_msg in msgs,
>  - msgs: the array of log messages stored in the bf_log_msg structures=
=2E
>
>The fields in bf_log_msg structure:
>  - size: the total size of the bf_log_msg entry,
>  - ts_nsec: the timestamp in nanoseconds starting from 0 (zero); the pro=
ducer
>    using this log format defines the meaning of 0,
>  - level: similar to the syslog meaning; used to differentiate normal lo=
g
>    messages from debug log messages, but the exact interpretation depend=
s on
>    the producer,
>  - facility: similar to the syslog meaning; used to differentiate the so=
urces
>    of the log messages, but the exact interpretation depends on the prod=
ucer,
>  - msg_off: the byte offset which the msg field starts in bf_log_msg,
>  - type: the log message type; similar to facility but NUL terminated st=
ring
>    instead of integer, but the exact interpretation depends on the produ=
cer,
>  - msg: the log message, NUL terminated string=2E
>
>In bf_log_msg, the producers are free to use or ignore any of the level,
>facility, and type fields=2E If level or facility are ignored, they shoul=
d be set
>to 0=2E If type is ignored, it should be set to an empty NUL terminated s=
tring=2E
>
>Since it doesn't seem possible to have each boot component using the same=
 log
>format, we added a log_format and log_phys_addr fields to give flexibilit=
y in
>how logs are stored=2E An example of a different log format that can be u=
sed is
>the cbmem_console log format used by coreboot:
>
>cbmem_console:
>               +-------------------+
>u32            | size              |
>u32            | cursor            |
>u8[m]          | body              |
>               +-------------------+
>
>There is still the outstanding issue of how the logs will be sent to the =
OS=2E If
>UEFI is used, we can use config tables=2E If ACPI or Device Tree is used,=
 we can
>use bf_log_header=2Enext_bflh_addr to present the logs=2E If none of thes=
e platforms
>are used, it becomes a lot trickier to solve this issue=2E
>
>Any suggestions are much appreciated and will be taken into consideration=
=2E
>
>I will be presenting this work at the LPC System Boot and Security
>Micro-conference on the 22nd of September at 7:50 AM PDT (14:50 UTC)=2E C=
ome and
>join if you want to discuss the design=2E The schedule for the System Boo=
t and
>Security Micro-conference can be found here [3]=2E
>
>Thanks!
>Alec Brown
>
>[1] https://lists=2Egnu=2Eorg/archive/html/grub-devel/2020-11/msg00100=2E=
html
>[2] https://lists=2Egnu=2Eorg/archive/html/grub-devel/2020-12/msg00021=2E=
html
>[3] https://linuxplumbersconf=2Eorg/event/11/sessions/116/#20210922


From xen-devel-bounces@lists.xenproject.org Sat Sep 18 23:27:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 18 Sep 2021 23:27:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.189985.339833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRjk8-0004T3-Aa; Sat, 18 Sep 2021 23:27:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 189985.339833; Sat, 18 Sep 2021 23:27:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRjk8-0004Sw-7i; Sat, 18 Sep 2021 23:27:36 +0000
Received: by outflank-mailman (input) for mailman id 189985;
 Sat, 18 Sep 2021 23:27:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRjk7-0004Sm-Au; Sat, 18 Sep 2021 23:27:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRjk7-0000pb-3K; Sat, 18 Sep 2021 23:27:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRjk6-0007IY-Ok; Sat, 18 Sep 2021 23:27:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRjk6-00086G-O6; Sat, 18 Sep 2021 23:27:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=N9KzWENrSkatit1bjnEep55Zr9/o4ejd5eWSrcLXJVs=; b=gRuib73rI5uttSTZZASnW6nrYF
	CckA5ZMyb7Yc8Uy3xQ0iFDTwIZZJTIcdzTLZOWeDDtF3e/udQ7UiR8iFJTWcbx5WeJ6dVBa3SzSQu
	Lh6TkmZfxI3ymQlfOYe0mV1ZDSnCVGa2vHTR+rHs8uHMRFtAwpQBXoDJCPTdCIufp86o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165110-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165110: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-xsm:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-xsm:host-install(5):broken:heisenbug
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 18 Sep 2021 23:27:34 +0000

flight 165110 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165110/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm    <job status>                 broken
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-xsm  5 host-install(5)          broken pass in 165103
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 165103
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install    fail pass in 165103

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 165103 like 164945
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check fail in 165103 never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check fail in 165103 never pass
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164945
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    8 days
Failing since        164951  2021-09-12 00:14:36 Z    6 days   13 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 broken  
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-arm64-arm64-libvirt-xsm broken
broken-step test-arm64-arm64-libvirt-xsm host-install(5)

Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 01:44:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 01:44:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190001.339848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRlsV-0000ue-Ej; Sun, 19 Sep 2021 01:44:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190001.339848; Sun, 19 Sep 2021 01:44:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRlsV-0000uX-Au; Sun, 19 Sep 2021 01:44:23 +0000
Received: by outflank-mailman (input) for mailman id 190001;
 Sun, 19 Sep 2021 01:44:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRlsU-0000uN-2t; Sun, 19 Sep 2021 01:44:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRlsT-0001UV-OX; Sun, 19 Sep 2021 01:44:21 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRlsT-0006jz-E0; Sun, 19 Sep 2021 01:44:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRlsT-0005ov-DW; Sun, 19 Sep 2021 01:44:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xqRN6Ndgt+CGcZzR1JNr1oCn5u1GELUJLjbKdvIPyNI=; b=EYLYdy3Fuzh/QssC7zHI5eoz1Z
	9yHIZ2JEUBwhQNv+DsmrdbHJmeXcZxWfXp2cNCQJTZEVG7D9N8+Y7SrfaIe0Twtok4M28lfHA+xrQ
	Tt69/gOCRSSN3/hjQbhb0n/64azbj0MmstggOh6tIRFLVVfkXAGwP7zO4TPjHsCkpCb0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165113-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165113: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=542cba73d21b7bcc1e3852a9d9843d5fffc2d173
X-Osstest-Versions-That:
    ovmf=e3e47d7963985f669f5826eac41b3ac4f19fc444
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 01:44:21 +0000

flight 165113 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165113/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 542cba73d21b7bcc1e3852a9d9843d5fffc2d173
baseline version:
 ovmf                 e3e47d7963985f669f5826eac41b3ac4f19fc444

Last test of basis   165108  2021-09-18 08:42:25 Z    0 days
Testing same since   165113  2021-09-18 16:11:19 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Longlong Yang <longlong.yang@intel.com>
  Yang, Longlong <longlong.yang@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   e3e47d7963..542cba73d2  542cba73d21b7bcc1e3852a9d9843d5fffc2d173 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 02:22:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 02:22:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190016.339862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRmTM-0005s7-Iy; Sun, 19 Sep 2021 02:22:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190016.339862; Sun, 19 Sep 2021 02:22:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRmTM-0005s0-Et; Sun, 19 Sep 2021 02:22:28 +0000
Received: by outflank-mailman (input) for mailman id 190016;
 Sun, 19 Sep 2021 02:22:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRmTL-0005rq-PA; Sun, 19 Sep 2021 02:22:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRmTL-0002X6-Hc; Sun, 19 Sep 2021 02:22:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRmTL-0008AT-5t; Sun, 19 Sep 2021 02:22:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRmTL-0001rp-5S; Sun, 19 Sep 2021 02:22:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=W1jT2/cvE4Ty3HSCO/pfQgXa4weaLtHMYIE/9zQf4LQ=; b=SErmuYrcsocpGO9tdQaTcya8Cs
	/bBN8ts8cN5pqp5ZjPs3Qafrg/VRASAXYnmfleHmkEOh/ncYg2fpMyQYdrBbLnhg8TUtzm8gKoack
	KOaUTDL7sws9vmpxeQlYLCuKPgG0TxggQooDKkR9oIotKMfuqORhbZWxhGbvQj9MGmCY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165112-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165112: trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    qemu-mainline:build-arm64:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:<job status>:broken:regression
    qemu-mainline:build-arm64-xsm:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:host-install(4):broken:regression
    qemu-mainline:build-arm64-xsm:host-install(4):broken:regression
    qemu-mainline:build-arm64:host-install(4):broken:regression
    qemu-mainline:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:build-arm64-libvirt:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 02:22:27 +0000

flight 165112 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165112/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 164950
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164950
 build-arm64                   4 host-install(4)        broken REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    7 days
Failing since        164967  2021-09-13 13:06:52 Z    5 days   11 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64-xsm host-install(4)
broken-step build-arm64 host-install(4)

Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 03:04:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 03:04:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190025.339875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRn85-00029n-2b; Sun, 19 Sep 2021 03:04:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190025.339875; Sun, 19 Sep 2021 03:04:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRn84-00029g-Vw; Sun, 19 Sep 2021 03:04:32 +0000
Received: by outflank-mailman (input) for mailman id 190025;
 Sun, 19 Sep 2021 03:04:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRn83-00029W-Az; Sun, 19 Sep 2021 03:04:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRn83-0003EZ-09; Sun, 19 Sep 2021 03:04:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRn82-0001I2-L7; Sun, 19 Sep 2021 03:04:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRn82-00063C-Ka; Sun, 19 Sep 2021 03:04:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Sl25WCQXN7r6YFHq6Mh2fNG30TOIzn5OTa9AF5fGkRI=; b=2Gg7aK4oN6lWZb0WyRw5xT9bOu
	EwdbMbAmkO5hpkD2xOtMYDjJ0qeeu4EEdz2rypRiEzfJhnSjoO9gQOelrRxJsi+oe1wXZfK8119eK
	6V0XQSwIyvrli2LXpIYDVx50punPz3Cedy3E96wymruLcH9TO+TzzprhgICm0kko9PAY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165114-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165114: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    linux-linus:build-arm64:<job status>:broken:regression
    linux-linus:build-arm64-pvops:<job status>:broken:regression
    linux-linus:build-arm64-xsm:<job status>:broken:regression
    linux-linus:build-arm64-pvops:host-install(4):broken:regression
    linux-linus:build-arm64:host-install(4):broken:regression
    linux-linus:build-arm64-xsm:host-install(4):broken:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    linux-linus:build-arm64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4357f03d6611753936e4d52fc251b54a6afb1b54
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 03:04:30 +0000

flight 165114 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165114/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 152332
 build-arm64                   4 host-install(4)        broken REGR. vs. 152332
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4357f03d6611753936e4d52fc251b54a6afb1b54
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  414 days
Failing since        152366  2020-08-01 20:49:34 Z  413 days  720 attempts
Testing same since   165107  2021-09-18 07:14:05 Z    0 days    2 attempts

------------------------------------------------------------
7317 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64 host-install(4)
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 2261040 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 08:51:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 08:51:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190054.339890 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRsXs-0005RJ-Jw; Sun, 19 Sep 2021 08:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190054.339890; Sun, 19 Sep 2021 08:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRsXs-0005RC-GO; Sun, 19 Sep 2021 08:51:32 +0000
Received: by outflank-mailman (input) for mailman id 190054;
 Sun, 19 Sep 2021 08:51:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRsXr-0005Qm-1s; Sun, 19 Sep 2021 08:51:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRsXq-0001Un-RE; Sun, 19 Sep 2021 08:51:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRsXq-00027c-IL; Sun, 19 Sep 2021 08:51:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRsXq-0006Ci-Hv; Sun, 19 Sep 2021 08:51:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UcsJHfPcgK0EpPmrw9/VlryFag2mf8n9JAXxPNCWLMw=; b=kYOLU4XtvQJrZeVeb6IeyH5HMR
	pQmN6bSpl3BxQj6Llj/yQJ1HOI9XWTtQOPfZL/BiQDhzpr/0rvupZVhnJxHkzO9eMqzOPoJj9biMV
	6CoP0R3G0IIIo1ZfNDSEGy8BF/juiM2Wii4Zxp3dVqgHYL8YNLb9nXeGVz3XejO2cDWQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165115-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165115: trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    xen-unstable:build-arm64:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:<job status>:broken:regression
    xen-unstable:build-arm64-xsm:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:host-install(4):broken:regression
    xen-unstable:build-arm64:host-install(4):broken:regression
    xen-unstable:build-arm64-xsm:host-install(4):broken:regression
    xen-unstable:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable:build-arm64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 08:51:30 +0000

flight 165115 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165115/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 164945
 build-arm64                   4 host-install(4)        broken REGR. vs. 164945
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164945
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    8 days
Failing since        164951  2021-09-12 00:14:36 Z    7 days   14 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64 host-install(4)
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 10:54:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 10:54:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190076.339903 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRuSp-0001Fj-M1; Sun, 19 Sep 2021 10:54:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190076.339903; Sun, 19 Sep 2021 10:54:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRuSp-0001Fc-Ii; Sun, 19 Sep 2021 10:54:27 +0000
Received: by outflank-mailman (input) for mailman id 190076;
 Sun, 19 Sep 2021 10:54:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRuSo-0001FS-Ef; Sun, 19 Sep 2021 10:54:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRuSo-0003YV-8P; Sun, 19 Sep 2021 10:54:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRuSn-0000Ot-WE; Sun, 19 Sep 2021 10:54:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRuSn-0005aJ-Vg; Sun, 19 Sep 2021 10:54:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=j7d2HLogYSDzsaK1i1QDkwcJSqh69YdDQHdrwd9ippM=; b=h/Vvbhex3csC1CiwdIG2TWAOT8
	gS1HXjn0y4ldFj5fYnL8ZssXaZt6wN5clBs1cHJuwzX4wtcxVIPkwr8GCJHHLZHbRk6mdujWmRSFs
	bWXGIFZuPaq1JonYtdaGDvrB2Q21QMwullVemEfjwMCLTH14GdLnOYV6w4LFszfVMDM0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165120-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 165120: all pass - PUSHED
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=487975df53b5298316b594550c79934d646701bd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 10:54:25 +0000

flight 165120 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165120/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  487975df53b5298316b594550c79934d646701bd

Last test of basis   164994  2021-09-15 09:19:43 Z    4 days
Testing same since   165120  2021-09-19 09:19:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   487975df53..437e88417b  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 11:20:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 11:20:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190090.339917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRurj-0004lj-OJ; Sun, 19 Sep 2021 11:20:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190090.339917; Sun, 19 Sep 2021 11:20:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRurj-0004lc-LK; Sun, 19 Sep 2021 11:20:11 +0000
Received: by outflank-mailman (input) for mailman id 190090;
 Sun, 19 Sep 2021 11:20:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tZQJ=OJ=kernel.org=patchwork-bot+netdevbpf@srs-us1.protection.inumbo.net>)
 id 1mRuri-0004lV-FC
 for xen-devel@lists.xenproject.org; Sun, 19 Sep 2021 11:20:10 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c7637f31-53f1-418d-85ff-94fc32433042;
 Sun, 19 Sep 2021 11:20:09 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPS id EE7FA61242;
 Sun, 19 Sep 2021 11:20:08 +0000 (UTC)
Received: from pdx-korg-docbuild-2.ci.codeaurora.org (localhost.localdomain
 [127.0.0.1])
 by pdx-korg-docbuild-2.ci.codeaurora.org (Postfix) with ESMTP id E19AC60A37;
 Sun, 19 Sep 2021 11:20:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7637f31-53f1-418d-85ff-94fc32433042
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632050409;
	bh=JgOlWT0pbtf7IVNWGZ6Y18loUrVLEywZaq+kJszZk28=;
	h=Subject:From:Date:References:In-Reply-To:To:Cc:From;
	b=McYbWhBfv9t4VvwDDYNBAFiHD5PCAfs8VVqsgoYw+7amWtY1FVzDJQMZdARAMaRQn
	 hazuosxKMmRlMk0usDx33Hr4ru7oaQ7Eu2ss3lGTe1ZN4DQaSSNmYS8wGQOS2rBI2E
	 xuHGod568wdnTud8oD6FGIZWQcY82cc9McjeggXmX6hTd0MjHZ8oy0dp6gX8RyRMkf
	 xxHDsllhbtBBvdVrmvOBu6qVTu47B1+YjZcG8ZB9bwMYVRW8ovX3Pjen8jLg9Hmu4b
	 bQIKEvC1FsPbDIOUfImizS/Z3q0twtxXmCJlP4VMMP3bMnA/ZTc2IoLSbwEpl8sQr4
	 Vq6Wlj0OSvCow==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Subject: Re: [PATCH net][RESEND] xen-netback: correct success/error reporting for
 the SKB-with-fraglist case
From: patchwork-bot+netdevbpf@kernel.org
Message-Id: 
 <163205040891.14261.10527748339776839114.git-patchwork-notify@kernel.org>
Date: Sun, 19 Sep 2021 11:20:08 +0000
References: <ef9e1ab6-17b9-c2d7-ef6c-99ef6726a765@suse.com>
In-Reply-To: <ef9e1ab6-17b9-c2d7-ef6c-99ef6726a765@suse.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: netdev@vger.kernel.org, paul@xen.org, wl@xen.org,
 xen-devel@lists.xenproject.org

Hello:

This patch was applied to netdev/net.git (refs/heads/master):

On Fri, 17 Sep 2021 08:27:10 +0200 you wrote:
> When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
> special considerations for the head of the SKB no longer apply. Don't
> mistakenly report ERROR to the frontend for the first entry in the list,
> even if - from all I can tell - this shouldn't matter much as the overall
> transmit will need to be considered failed anyway.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Reviewed-by: Paul Durrant <paul@xen.org>

Here is the summary with links:
  - [net,RESEND] xen-netback: correct success/error reporting for the SKB-with-fraglist case
    https://git.kernel.org/netdev/net/c/3ede7f84c7c2

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html




From xen-devel-bounces@lists.xenproject.org Sun Sep 19 11:45:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 11:45:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190098.339929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRvG9-0007NJ-Qy; Sun, 19 Sep 2021 11:45:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190098.339929; Sun, 19 Sep 2021 11:45:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRvG9-0007NC-NB; Sun, 19 Sep 2021 11:45:25 +0000
Received: by outflank-mailman (input) for mailman id 190098;
 Sun, 19 Sep 2021 11:45:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRvG8-0007N2-Ky; Sun, 19 Sep 2021 11:45:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRvG8-0004Q9-Bq; Sun, 19 Sep 2021 11:45:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRvG8-0003GC-3D; Sun, 19 Sep 2021 11:45:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRvG8-0008So-2g; Sun, 19 Sep 2021 11:45:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XR2M1LnIs44ZvGOHvLQC7dWLpLx1vk8K9UYVbyVeQOY=; b=JZg9XshKlTYDa186HJZVwpf1xS
	li0dctGlYaJBfwskUCvyZ1j6l8eu4qkFzpB+fR6C9pVIqKNbzCfQawNG7f4U8WeNmXk25rPYAHslU
	aWNTPQhDyP1O+/ujjBUDmvh0+IzQX+g8yjK6rjOgIHjQuMkLBHynOo/InoKuaxruEa2c=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165118-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165118: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    libvirt:build-arm64:<job status>:broken:regression
    libvirt:build-arm64-pvops:<job status>:broken:regression
    libvirt:build-arm64-xsm:<job status>:broken:regression
    libvirt:build-arm64-xsm:host-install(4):broken:regression
    libvirt:build-arm64:host-install(4):broken:regression
    libvirt:build-arm64-pvops:host-install(4):broken:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=785a11cec8693de7df024aae68975dd1799b646a
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 11:45:24 +0000

flight 165118 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165118/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 151777
 build-arm64                   4 host-install(4)        broken REGR. vs. 151777
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              785a11cec8693de7df024aae68975dd1799b646a
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  436 days
Failing since        151818  2020-07-11 04:18:52 Z  435 days  426 attempts
Testing same since   165104  2021-09-18 04:20:03 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)
broken-step build-arm64 host-install(4)
broken-step build-arm64-pvops host-install(4)

Not pushing.

(No revision log; it would be 73994 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 13:31:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 13:31:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190123.339943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRwuD-0001vi-F2; Sun, 19 Sep 2021 13:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190123.339943; Sun, 19 Sep 2021 13:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRwuD-0001vb-Bn; Sun, 19 Sep 2021 13:30:53 +0000
Received: by outflank-mailman (input) for mailman id 190123;
 Sun, 19 Sep 2021 13:30:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRwuB-0001vP-PU; Sun, 19 Sep 2021 13:30:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRwuB-0006MZ-0p; Sun, 19 Sep 2021 13:30:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRwuA-0006uS-Mq; Sun, 19 Sep 2021 13:30:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRwuA-0001Vz-MQ; Sun, 19 Sep 2021 13:30:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zauIkVtpLwbb36lVOJsw2VEi6BjgPnmTH/OWtTMRbwI=; b=uQmI9M3VpDrK59WjhE68L7oNFR
	QtUMVrJFqaqg4oKEvcj6iVLIwx5AQXN1Z4byVSz142vKvXrZwGMrMlvjzmOaPr/cBctWnGjvJTjqs
	5KX4xWz2GDyht1kIxS2uylFKP8RVrVkL+hw0JmlOk/JXBKjYdeL4oUgqrMB26o6VUKME=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165116-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165116: trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    qemu-mainline:build-arm64:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:<job status>:broken:regression
    qemu-mainline:build-arm64-xsm:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:host-install(4):broken:regression
    qemu-mainline:build-arm64-xsm:host-install(4):broken:regression
    qemu-mainline:build-arm64:host-install(4):broken:regression
    qemu-mainline:test-armhf-armhf-libvirt:debian-install:fail:heisenbug
    qemu-mainline:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:build-arm64-libvirt:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 13:30:50 +0000

flight 165116 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165116/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 164950
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164950
 build-arm64                   4 host-install(4)        broken REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt     12 debian-install             fail pass in 165112

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 165112 like 164950
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 165112 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    7 days
Failing since        164967  2021-09-13 13:06:52 Z    6 days   12 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64-xsm host-install(4)
broken-step build-arm64 host-install(4)
broken-job build-arm64-xsm broken
broken-job build-arm64-pvops broken
broken-job build-arm64 broken

Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 14:01:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 14:01:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190134.339957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRxND-0005XL-R8; Sun, 19 Sep 2021 14:00:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190134.339957; Sun, 19 Sep 2021 14:00:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRxND-0005XE-O9; Sun, 19 Sep 2021 14:00:51 +0000
Received: by outflank-mailman (input) for mailman id 190134;
 Sun, 19 Sep 2021 14:00:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRxNB-0005X8-Rp
 for xen-devel@lists.xenproject.org; Sun, 19 Sep 2021 14:00:49 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRxN9-0006ze-HN; Sun, 19 Sep 2021 14:00:47 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRxN8-0001AN-MI; Sun, 19 Sep 2021 14:00:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=IT4zDZcVajd+PWlWmr9l0aIfazKhe573k0IXfZTGzGk=; b=PPp53nyOReb9ZeyTwSLa1zypmy
	rULocry9ryeKt9EUoTTbx4HMq56bOcUyTRdu6mbXdFAFZgdjNSzhaP8pkLfLiQcgyKV8LtBxcDHPj
	ZX6nlqNMYCsInZHvKe9RdIg5XS/iW7Cp2j727kLISgLoyMH8QGqHb+aAyDtWX+H1Xks4=;
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Oleksandr <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
From: Julien Grall <julien@xen.org>
Message-ID: <8b8a7055-9983-3e83-a308-21dc61b27d4d@xen.org>
Date: Sun, 19 Sep 2021 19:00:39 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

On 18/09/2021 00:51, Oleksandr wrote:
> On 17.09.21 18:48, Julien Grall wrote:
>> On 10/09/2021 23:18, Oleksandr Tyshchenko wrote:
>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>
>>> The extended region (safe range) is a region of guest physical
>>> address space which is unused and could be safely used to create
>>> grant/foreign mappings instead of wasting real RAM pages from
>>> the domain memory for establishing these mappings.
>>>
>>> The extended regions are chosen at the domain creation time and
>>> advertised to it via "reg" property under hypervisor node in
>>> the guest device-tree. As region 0 is reserved for grant table
>>> space (always present), the indexes for extended regions are 1...N.
>>> If extended regions could not be allocated for some reason,
>>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>>
>>> Please note the following limitations:
>>> - The extended region feature is only supported for 64-bit domain.
>>> - The ACPI case is not covered.
>>
>> I understand the ACPI is not covered because we would need to create a 
>> new binding. But I am not sure to understand why 32-bit domain is not 
>> supported. Can you explain it?
> 
> The 32-bit domain is not supported for simplifying things from the 
> beginning. It is a little bit difficult to get everything working at 
> start. As I understand from discussion at [1] we can afford that 
> simplification. However, I should have mentioned that 32-bit domain is 
> not supported "for now".

Right, I forgot that. This is where it is useful to write down the 
decision in the commit message.

> 
>>
>>>
>>> ***
>>>
>>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>>> the algorithm to choose extended regions for it is different
>>> in comparison with the algorithm for non-direct mapped DomU.
>>> What is more, that extended regions should be chosen differently
>>> whether IOMMU is enabled or not.
>>>
>>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>>> holes found in host device-tree if otherwise. 
>>
>> For the case when the IOMMU is disabled, this will only work if dom0 
>> cannot allocate memory outside of the original range. This is 
>> currently the case... but I think this should be spelled out in at 
>> least the commit message.
> 
> Agree, will update commit description.
> 
> 
>>
>>
>>> Make sure that
>>> extended regions are 2MB-aligned and located within maximum possible
>>> addressable physical memory range. The maximum number of extended
>>> regions is 128.
>>
>> Please explain how this limit was chosen.
> Well, I decided to not introduce new data struct and etc to represent 
> extended regions but reuse existing struct meminfo
> used for memory/reserved-memory and, as I though, perfectly fitted. So, 
> that limit come from NR_MEM_BANKS which is 128.

Ok. So this is an artificial limit. Please make it clear in the commit 
message.

> 
>>
>>>
>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>> ---
>>> Changes since RFC:
>>>     - update patch description
>>>     - drop uneeded "extended-region" DT property
>>> ---
>>>
>>>   xen/arch/arm/domain_build.c | 226 
>>> +++++++++++++++++++++++++++++++++++++++++++-
>>>   1 file changed, 224 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 206038d..070ec27 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const struct 
>>> domain *d,
>>>       return res;
>>>   }
>>>   +static int __init add_ext_regions(unsigned long s, unsigned long 
>>> e, void *data)
>>> +{
>>> +    struct meminfo *ext_regions = data;
>>> +    paddr_t start, size;
>>> +
>>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>>> +        return 0;
>>> +
>>> +    /* Both start and size of the extended region should be 2MB 
>>> aligned */
>>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>>> +    if ( start > e )
>>> +        return 0;
>>> +
>>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>>> +    if ( !size )
>>> +        return 0;
>>> +
>>> +    ext_regions->bank[ext_regions->nr_banks].start = start;
>>> +    ext_regions->bank[ext_regions->nr_banks].size = size;
>>> +    ext_regions->nr_banks ++;
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +/*
>>> + * The extended regions will be prevalidated by the memory hotplug path
>>> + * in Linux which requires for any added address range to be within 
>>> maximum
>>> + * possible addressable physical memory range for which the linear 
>>> mapping
>>> + * could be created.
>>> + * For 48-bit VA space size the maximum addressable range are:
>>
>> When I read "maximum", I understand an upper limit. But below, you are 
>> providing a range. So should you drop "maximum"?
> 
> yes, it is a little bit confusing.
> 
> 
>>
>>
>> Also, this is tailored to Linux using 48-bit VA. How about other limits?
> These limits are calculated at [2]. Sorry, I didn't investigate yet what 
> values would be for other CONFIG_ARM64_VA_BITS_XXX. Also looks like some 
> configs depend on 16K/64K pages...
> I will try to investigate and provide limits later on.

I have thought a bit more about it. At the moment, you are relying on 
Xen to find a range that is addressable by the OS. This can be quite 
complex as different OS may have different requirement. So how about 
letting the OS to filter the ranges based on its limitations?

> 
> 
>>
>>
>>> + * 0x40000000 - 0x80003fffffff
>>> + */
>>> +#define EXT_REGION_START   0x40000000ULL
>>
>> I am probably missing something here.... There are platform out there 
>> with memory starting at 0 (IIRC ZynqMP is one example). So wouldn't 
>> this potentially rule out the extended region on such platform?
> 
>  From my understanding the extended region cannot be in 0...0x40000000 
> range. If these platforms have memory above first GB, I believe the 
> extended region(s) can be allocated for them.

Do you mean "cannot"? Technically this is a limitation of the current 
version of Linux. Tomorrow, someone may be able to remove that 
limitations. So, as mentionned above, maybe Xen should not do the filtering.

>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>> +                                    struct meminfo *ext_regions)
>>> +{
>>> +    struct dt_device_node *np;
>>> +    struct rangeset *mem_holes;
>>> +    paddr_t start, end;
>>> +    unsigned int i;
>>> +    int res;
>>> +
>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>> +
>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>> +    if ( !mem_holes )
>>> +        return -ENOMEM;
>>> +
>>> +    /* Start with maximum possible addressable physical memory range */
>>> +    start = EXT_REGION_START;
>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>> +    res = rangeset_add_range(mem_holes, start, end);
>>> +    if ( res )
>>> +    {
>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>> +               start, end);
>>> +        goto out;
>>> +    }
>>> +
>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, 
>>> etc) */
>>
>> Well... The loop below is not going to handle all the regions 
>> described in the property "reg". Instead, it will cover a subset of 
>> "reg" where the memory is addressable.
> 
> As I understand, we are only interested in subset of "reg" where the 
> memory is addressable.

Right... That's not what your comment is saying.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 14:34:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 14:34:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190146.339968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRxtp-0000ky-Hk; Sun, 19 Sep 2021 14:34:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190146.339968; Sun, 19 Sep 2021 14:34:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRxtp-0000kr-EC; Sun, 19 Sep 2021 14:34:33 +0000
Received: by outflank-mailman (input) for mailman id 190146;
 Sun, 19 Sep 2021 14:34:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mRxto-0000kl-3K
 for xen-devel@lists.xenproject.org; Sun, 19 Sep 2021 14:34:32 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRxtm-0007dz-6t; Sun, 19 Sep 2021 14:34:30 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mRxtl-0004OE-QA; Sun, 19 Sep 2021 14:34:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=NPqax0K85Rn1tgWNrKuhFMIiw0yhsK7RuuSCpIkX8ps=; b=p3kSlH9ib9c8OTLnkgH9errBHl
	9m19AFSol0ubA9WqOghaB3IURuY44+cxN7sBU0ZRXCLp8QxemHk6S45yj//BQkfqUyoUy3P9zW+4f
	HnQnutgwiAFOUI2S5q7ZX+j+o704u5BtAdMwjQh6KIzF49Nfej/ri68NyAkc83iN527E=;
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Oleksandr <olekstysh@gmail.com>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
Date: Sun, 19 Sep 2021 19:34:23 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Stefano,

On 18/09/2021 03:37, Stefano Stabellini wrote:
> On Fri, 17 Sep 2021, Stefano Stabellini wrote:
>> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>>> +
>>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>>> +
>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>> +    if ( !unalloc_mem )
>>>>> +        return -ENOMEM;
>>>>> +
>>>>> +    /* Start with all available RAM */
>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>> +    {
>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
>>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>>> +        if ( res )
>>>>> +        {
>>>>> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>>> +                   start, end);
>>>>> +            goto out;
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>> +    {
>>>>> +        start = assign_mem->bank[i].start;
>>>>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>> +        if ( res )
>>>>> +        {
>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>> +                   start, end);
>>>>> +            goto out;
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    /* Remove reserved-memory regions */
>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>>> +    {
>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>> +        if ( res )
>>>>> +        {
>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>> +                   start, end);
>>>>> +            goto out;
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    /* Remove grant table region */
>>>>> +    start = kinfo->gnttab_start;
>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>> +    if ( res )
>>>>> +    {
>>>>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>>>> +               start, end);
>>>>> +        goto out;
>>>>> +    }
>>>>> +
>>>>> +    start = EXT_REGION_START;
>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>> +                                 add_ext_regions, ext_regions);
>>>>> +    if ( res )
>>>>> +        ext_regions->nr_banks = 0;
>>>>> +    else if ( !ext_regions->nr_banks )
>>>>> +        res = -ENOENT;
>>>>> +
>>>>> +out:
>>>>> +    rangeset_destroy(unalloc_mem);
>>>>> +
>>>>> +    return res;
>>>>> +}
>>>>> +
>>>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>>>> +                                    struct meminfo *ext_regions)
>>>>> +{
>>>>> +    struct dt_device_node *np;
>>>>> +    struct rangeset *mem_holes;
>>>>> +    paddr_t start, end;
>>>>> +    unsigned int i;
>>>>> +    int res;
>>>>> +
>>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>>> +
>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>> +    if ( !mem_holes )
>>>>> +        return -ENOMEM;
>>>>> +
>>>>> +    /* Start with maximum possible addressable physical memory range */
>>>>> +    start = EXT_REGION_START;
>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>> +    if ( res )
>>>>> +    {
>>>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>>> +               start, end);
>>>>> +        goto out;
>>>>> +    }
>>>>> +
>>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
>>>>
>>>> Well... The loop below is not going to handle all the regions described in
>>>> the property "reg". Instead, it will cover a subset of "reg" where the
>>>> memory is addressable.
>>>
>>> As I understand, we are only interested in subset of "reg" where the memory is
>>> addressable.
>>>
>>>
>>>>
>>>>
>>>> You will also need to cover "ranges" that will describe the BARs for the PCI
>>>> devices.
>>> Good point.
>>
>> Yes, very good point!
>>
>>
>>> Could you please clarify how to recognize whether it is a PCI
>>> device as long as PCI support is not merged? Or just to find any device nodes
>>> with non-empty "ranges" property
>>> and retrieve addresses?
>>
>> Normally any bus can have a ranges property with the aperture and
>> possible address translations, including /amba (compatible =
>> "simple-bus"). However, in these cases dt_device_get_address already
>> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>>
>> The PCI bus is special for 2 reasons:
>> - the ranges property has a different format
>> - the bus is hot-pluggable
>>
>> So I think the only one that we need to treat specially is PCI.
>>
>> As far as I am aware PCI is the only bus (or maybe just the only bus
>> that we support?) where ranges means the aperture.
> 
> Now that I think about this, there is another "hotpluggable" scenario we
> need to think about:
> 
> [1] https://marc.info/?l=xen-devel&m=163056546214978
> 
> Xilinx devices have FPGA regions with apertures currently not described
> in device tree, where things can programmed in PL at runtime making new
> devices appear with new MMIO regions out of thin air.
> 
> Now let me start by saying that yes, the entire programmable region
> aperture could probably be described in device tree, however, in
> reality it is not currently done in any of the device trees we use
> (including the upstream device trees in linux.git).

This is rather annoying, but not unheard. There are a couple of 
platforms where the MMIOs are not fully described in the DT.

In fact, we have a callback 'specific_mappings' which create additional 
mappings (e.g. on the omap5) for dom0.

> 
> So, we have a problem :-(
> 
> 
> I can work toward getting the right info on device tree, but in reality
> that is going to take time and for now the device tree doesn't have the
> FPGA aperture in it. So if we accept this series as is, it is going to
> stop features like [1] from working. >
> If we cannot come up with any better plans, I think it would be better
> to drop find_memory_holes, only rely on find_unallocated_memory even
> when the IOMMU is on. One idea is that we could add on top of the
> regions found by find_unallocated_memory any MMIO regions marked as
> xen,passthrough: they are safe because they are not going to dom0 anyway.

(Oleksandr, it looks like some rationale about the different approach is 
missing in the commit message. Can you add it?)

When the IOMMU is on, Xen will do an extra mapping with GFN == MFN for 
every grant mapping in dom0. This is because Linux will always program 
the device with the MFN as it doesn't know whether the device has been 
protected by the hypervisor.

Therefore we can't use find_unallocated_memory() with the IOMMU on as it 
stands.

> 
> The only alternative I can think of is to have a per-board
> enable/disable toggle for the extend region but it would be very ugly.
At least, for your board, you seem to know the list of regions that are 
reserved for future use. So how about adding a per-board list of regions 
that should not be allocated?

This will also include anything mentioned in 'specific_mappings'.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 14:52:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 14:52:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190156.339979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRyB2-0003F3-3U; Sun, 19 Sep 2021 14:52:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190156.339979; Sun, 19 Sep 2021 14:52:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mRyB1-0003Ew-W5; Sun, 19 Sep 2021 14:52:19 +0000
Received: by outflank-mailman (input) for mailman id 190156;
 Sun, 19 Sep 2021 14:52:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRyB0-0003El-Vx; Sun, 19 Sep 2021 14:52:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRyB0-0007xE-Ph; Sun, 19 Sep 2021 14:52:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mRyB0-0002Lk-Bv; Sun, 19 Sep 2021 14:52:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mRyB0-000715-BP; Sun, 19 Sep 2021 14:52:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EksbBo4SfWlGoK32ubwyqhWTy7T4zIk4kC1TrxDhNDs=; b=39P4hEXHoAPLbc1v3BWvAp8LnH
	54O/WRYbh4f46kG7Mf2Fe0R8SRdmpdzR724RacIlvUQ2sOLeNbPNpfD/WC+BKVLgUiYINNnQng+KA
	eb+dDh3xmvK3DSwoylFOgtgrKMHoGO9SB/xf1e/VzqbS9O7vihnY7OJhJqKTjUo1iUc8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165117-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165117: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    linux-linus:build-arm64:<job status>:broken:regression
    linux-linus:build-arm64-pvops:<job status>:broken:regression
    linux-linus:build-arm64-xsm:<job status>:broken:regression
    linux-linus:build-arm64-pvops:host-install(4):broken:regression
    linux-linus:build-arm64:host-install(4):broken:regression
    linux-linus:build-arm64-xsm:host-install(4):broken:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    linux-linus:build-arm64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-localmigrate:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d4d016caa4b85b9aa98d7ec8c84e928621a614bc
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 14:52:18 +0000

flight 165117 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165117/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 152332
 build-arm64                   4 host-install(4)        broken REGR. vs. 152332
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-freebsd11-amd64 17 guest-localmigrate  fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                d4d016caa4b85b9aa98d7ec8c84e928621a614bc
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  414 days
Failing since        152366  2020-08-01 20:49:34 Z  413 days  721 attempts
Testing same since   165117  2021-09-19 03:07:42 Z    0 days    1 attempts

------------------------------------------------------------
7317 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64 host-install(4)
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 2261430 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 17:59:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 17:59:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190175.339993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS15p-0005NB-4a; Sun, 19 Sep 2021 17:59:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190175.339993; Sun, 19 Sep 2021 17:59:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS15p-0005N4-1V; Sun, 19 Sep 2021 17:59:09 +0000
Received: by outflank-mailman (input) for mailman id 190175;
 Sun, 19 Sep 2021 17:59:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iuPp=OJ=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mS15n-0005My-Dp
 for xen-devel@lists.xenproject.org; Sun, 19 Sep 2021 17:59:07 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78a1c39f-a660-498e-b547-32df6cce442f;
 Sun, 19 Sep 2021 17:59:05 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id g41so23945165lfv.1
 for <xen-devel@lists.xenproject.org>; Sun, 19 Sep 2021 10:59:05 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id m12sm1068943ljh.21.2021.09.19.10.59.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 19 Sep 2021 10:59:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78a1c39f-a660-498e-b547-32df6cce442f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=f1QseqEMTj6gyZJlqR485ScF1zaUGJm0X/256UAJv7U=;
        b=lxCKPAu153HpaBvh4sEDRlyYx2jUHAMfsqv5LntwlanZkV2AATUK36M+onspNt3u7p
         9DPGWSF+Hu7ODQMtj0vsdcSPts2Q+6GHyK+hkanuEM4UIdW5oql0HvtXH2njmJMIZbIr
         519KWU7W6+O8OeElXI40w2L7kbUMHru62LpZmX8ujRKVvMCfAIos0c1ctmgUD0iXVXdx
         7s07Cu3fAu6/zEQM9Ljk7EFMRL0oCHsZBHuIngJrjUhCv15I21bTcapyHxmBK6Gzotru
         cdZafRSzPoMJ12YbwJnTX9w8ngg2PbSzXHURD/QGeADO5qp+G/D/XYQSdcbWmNZBygz5
         Rv0w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=f1QseqEMTj6gyZJlqR485ScF1zaUGJm0X/256UAJv7U=;
        b=MaQA3+6kNnkzcOTczdhyjc4xJsaS2jek96wYbhufuRlOlFyuTJoQXNiAQuzUASM6y8
         UuOQwZWA1unXaxZIlZ1TtDMOoco5yanSfYVH7S3bDqYuJQ0dLRwDigN2VtnbxpWFJF4t
         b/Rf5MBOX6FWAcCbz7QHkJ1KLPPHjP3le05fMHnH8M1/WpIPIOFeebzYGEaoNACVnJuF
         3dxgI2acCzlOlzLDwr/O3j3gQ0Z/Qt0/BqrmS+h7d2OXU71nTDOyQQh17IQzpxyVRYWF
         B7DYSVqKuF1kpUp4PvoSFQAbl5O6K/f6phSeGeQvvleMRi9Jw+B5RNFh4wNyG/NGlYId
         ZmPQ==
X-Gm-Message-State: AOAM532QzQficLvzmNX0EYOvtaCYtS77lXm8w1sInxNyeYoTMGMIxEit
	ksw0X8UX4Sy5ppLR9BKUnZo=
X-Google-Smtp-Source: ABdhPJy0l4ae4OX0COjvzHnO799N3ACEZicSb3Lbwm6lln/iAdiFWTJMDkMKy4sYwRHzNnKS9sOgfw==
X-Received: by 2002:a2e:a555:: with SMTP id e21mr19881477ljn.490.1632074344174;
        Sun, 19 Sep 2021 10:59:04 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <8b8a7055-9983-3e83-a308-21dc61b27d4d@xen.org>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <239fe26b-1d64-ca54-aa0d-c45fcf3d44b6@gmail.com>
Date: Sun, 19 Sep 2021 20:59:03 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <8b8a7055-9983-3e83-a308-21dc61b27d4d@xen.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 19.09.21 17:00, Julien Grall wrote:
> Hi,

Hi Julien


>
> On 18/09/2021 00:51, Oleksandr wrote:
>> On 17.09.21 18:48, Julien Grall wrote:
>>> On 10/09/2021 23:18, Oleksandr Tyshchenko wrote:
>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>
>>>> The extended region (safe range) is a region of guest physical
>>>> address space which is unused and could be safely used to create
>>>> grant/foreign mappings instead of wasting real RAM pages from
>>>> the domain memory for establishing these mappings.
>>>>
>>>> The extended regions are chosen at the domain creation time and
>>>> advertised to it via "reg" property under hypervisor node in
>>>> the guest device-tree. As region 0 is reserved for grant table
>>>> space (always present), the indexes for extended regions are 1...N.
>>>> If extended regions could not be allocated for some reason,
>>>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>>>
>>>> Please note the following limitations:
>>>> - The extended region feature is only supported for 64-bit domain.
>>>> - The ACPI case is not covered.
>>>
>>> I understand the ACPI is not covered because we would need to create 
>>> a new binding. But I am not sure to understand why 32-bit domain is 
>>> not supported. Can you explain it?
>>
>> The 32-bit domain is not supported for simplifying things from the 
>> beginning. It is a little bit difficult to get everything working at 
>> start. As I understand from discussion at [1] we can afford that 
>> simplification. However, I should have mentioned that 32-bit domain 
>> is not supported "for now".
>
> Right, I forgot that. This is where it is useful to write down the 
> decision in the commit message.

ok, will do.


>
>
>>
>>>
>>>>
>>>> ***
>>>>
>>>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>>>> the algorithm to choose extended regions for it is different
>>>> in comparison with the algorithm for non-direct mapped DomU.
>>>> What is more, that extended regions should be chosen differently
>>>> whether IOMMU is enabled or not.
>>>>
>>>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>>>> holes found in host device-tree if otherwise. 
>>>
>>> For the case when the IOMMU is disabled, this will only work if dom0 
>>> cannot allocate memory outside of the original range. This is 
>>> currently the case... but I think this should be spelled out in at 
>>> least the commit message.
>>
>> Agree, will update commit description.
>>
>>
>>>
>>>
>>>> Make sure that
>>>> extended regions are 2MB-aligned and located within maximum possible
>>>> addressable physical memory range. The maximum number of extended
>>>> regions is 128.
>>>
>>> Please explain how this limit was chosen.
>> Well, I decided to not introduce new data struct and etc to represent 
>> extended regions but reuse existing struct meminfo
>> used for memory/reserved-memory and, as I though, perfectly fitted. 
>> So, that limit come from NR_MEM_BANKS which is 128.
>
> Ok. So this is an artificial limit. Please make it clear in the commit 
> message.

ok, will do


>
>
>>
>>>
>>>>
>>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>> ---
>>>> Changes since RFC:
>>>>     - update patch description
>>>>     - drop uneeded "extended-region" DT property
>>>> ---
>>>>
>>>>   xen/arch/arm/domain_build.c | 226 
>>>> +++++++++++++++++++++++++++++++++++++++++++-
>>>>   1 file changed, 224 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>> index 206038d..070ec27 100644
>>>> --- a/xen/arch/arm/domain_build.c
>>>> +++ b/xen/arch/arm/domain_build.c
>>>> @@ -724,6 +724,196 @@ static int __init make_memory_node(const 
>>>> struct domain *d,
>>>>       return res;
>>>>   }
>>>>   +static int __init add_ext_regions(unsigned long s, unsigned long 
>>>> e, void *data)
>>>> +{
>>>> +    struct meminfo *ext_regions = data;
>>>> +    paddr_t start, size;
>>>> +
>>>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>>>> +        return 0;
>>>> +
>>>> +    /* Both start and size of the extended region should be 2MB 
>>>> aligned */
>>>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>>>> +    if ( start > e )
>>>> +        return 0;
>>>> +
>>>> +    size = (e - start + 1) & ~(SZ_2M - 1);
>>>> +    if ( !size )
>>>> +        return 0;
>>>> +
>>>> +    ext_regions->bank[ext_regions->nr_banks].start = start;
>>>> +    ext_regions->bank[ext_regions->nr_banks].size = size;
>>>> +    ext_regions->nr_banks ++;
>>>> +
>>>> +    return 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * The extended regions will be prevalidated by the memory hotplug 
>>>> path
>>>> + * in Linux which requires for any added address range to be 
>>>> within maximum
>>>> + * possible addressable physical memory range for which the linear 
>>>> mapping
>>>> + * could be created.
>>>> + * For 48-bit VA space size the maximum addressable range are:
>>>
>>> When I read "maximum", I understand an upper limit. But below, you 
>>> are providing a range. So should you drop "maximum"?
>>
>> yes, it is a little bit confusing.
>>
>>
>>>
>>>
>>> Also, this is tailored to Linux using 48-bit VA. How about other 
>>> limits?
>> These limits are calculated at [2]. Sorry, I didn't investigate yet 
>> what values would be for other CONFIG_ARM64_VA_BITS_XXX. Also looks 
>> like some configs depend on 16K/64K pages...
>> I will try to investigate and provide limits later on.

I have rebuilt Linux with CONFIG_ARM64_VA_BITS_39=y and printed the limits.

These are: 0x40000000 - 0x403fffffff


>>
>
> I have thought a bit more about it. At the moment, you are relying on 
> Xen to find a range that is addressable by the OS. This can be quite 
> complex as different OS may have different requirement. So how about 
> letting the OS to filter the ranges based on its limitations?

I think, it is a nice idea, thank you. So, I will drop OS specific 
limits (EXT_REGION_*) from the patch.

>
>
>>
>>
>>>
>>>
>>>> + * 0x40000000 - 0x80003fffffff
>>>> + */
>>>> +#define EXT_REGION_START   0x40000000ULL
>>>
>>> I am probably missing something here.... There are platform out 
>>> there with memory starting at 0 (IIRC ZynqMP is one example). So 
>>> wouldn't this potentially rule out the extended region on such 
>>> platform?
>>
>>  From my understanding the extended region cannot be in 
>> 0...0x40000000 range. If these platforms have memory above first GB, 
>> I believe the extended region(s) can be allocated for them.
>
> Do you mean "cannot"? 
No. I think, there was some misunderstanding from my size. Initially, I 
got this as "extended region feature cannot be used on such platform in 
general", so I tried to say that if these platforms
also had some RAM *above* 0x40000000 then extended region could be 
allocated for them in principle, we only won't be able to take advantage 
of 0...0x40000000...


> Technically this is a limitation of the current version of Linux. 
> Tomorrow, someone may be able to remove that limitations. So, as 
> mentionned above, maybe Xen should not do the filtering.

I got it, sounds reasonable.


>
>
>>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>>> +                                    struct meminfo *ext_regions)
>>>> +{
>>>> +    struct dt_device_node *np;
>>>> +    struct rangeset *mem_holes;
>>>> +    paddr_t start, end;
>>>> +    unsigned int i;
>>>> +    int res;
>>>> +
>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>> +
>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>> +    if ( !mem_holes )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    /* Start with maximum possible addressable physical memory 
>>>> range */
>>>> +    start = EXT_REGION_START;
>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>> +    if ( res )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>> +               start, end);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, 
>>>> etc) */
>>>
>>> Well... The loop below is not going to handle all the regions 
>>> described in the property "reg". Instead, it will cover a subset of 
>>> "reg" where the memory is addressable.
>>
>> As I understand, we are only interested in subset of "reg" where the 
>> memory is addressable.
>
> Right... That's not what your comment is saying.

ok, will update.


>
>
> Cheers,
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Sun Sep 19 19:04:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 19:04:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190182.340004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS276-0004BI-77; Sun, 19 Sep 2021 19:04:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190182.340004; Sun, 19 Sep 2021 19:04:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS276-0004BB-2x; Sun, 19 Sep 2021 19:04:32 +0000
Received: by outflank-mailman (input) for mailman id 190182;
 Sun, 19 Sep 2021 19:04:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS275-0004B1-AL; Sun, 19 Sep 2021 19:04:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS275-0004mb-41; Sun, 19 Sep 2021 19:04:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS274-0007XA-PB; Sun, 19 Sep 2021 19:04:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mS274-0000gb-OW; Sun, 19 Sep 2021 19:04:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6kbDBen+lMzrK87xjk0/X1jfQj/eAGsHLwhNOkPVsQg=; b=4dG5rPSSxPJqVMjUXADl17+9Rw
	7Lm7zAJvB1VgwjctbFHbOVCa1eXDb3k1C8EiqU6BjcxM7Ca1yvHVZKPtLr+1S5yeVkAe0jzMX854J
	oLW9pWQ6qEM6EVUbZDQAynX2mjhx0j8lPFXBZVdiT1Cmv363kuAkUHsZI/7iDEWKsDP8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165119-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165119: trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    xen-unstable:build-arm64:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:<job status>:broken:regression
    xen-unstable:build-arm64-xsm:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:host-install(4):broken:regression
    xen-unstable:build-arm64:host-install(4):broken:regression
    xen-unstable:build-arm64-xsm:host-install(4):broken:regression
    xen-unstable:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable:build-arm64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 19 Sep 2021 19:04:30 +0000

flight 165119 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165119/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken
 build-arm64-pvops               <job status>                 broken
 build-arm64-xsm                 <job status>                 broken
 build-arm64-pvops             4 host-install(4)        broken REGR. vs. 164945
 build-arm64                   4 host-install(4)        broken REGR. vs. 164945
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    8 days
Failing since        164951  2021-09-12 00:14:36 Z    7 days   15 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  broken  
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            broken  
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken
broken-step build-arm64-pvops host-install(4)
broken-step build-arm64 host-install(4)
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 19 20:19:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 19 Sep 2021 20:19:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190192.340018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS3Gw-0003OD-JE; Sun, 19 Sep 2021 20:18:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190192.340018; Sun, 19 Sep 2021 20:18:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS3Gw-0003O6-Fj; Sun, 19 Sep 2021 20:18:46 +0000
Received: by outflank-mailman (input) for mailman id 190192;
 Sun, 19 Sep 2021 20:18:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iuPp=OJ=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mS3Gv-0003O0-SI
 for xen-devel@lists.xenproject.org; Sun, 19 Sep 2021 20:18:45 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df83c6be-9939-491e-af7e-4e8b353c693e;
 Sun, 19 Sep 2021 20:18:44 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id b15so39817220lfe.7
 for <xen-devel@lists.xenproject.org>; Sun, 19 Sep 2021 13:18:43 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id z21sm1483076ljh.114.2021.09.19.13.18.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 19 Sep 2021 13:18:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df83c6be-9939-491e-af7e-4e8b353c693e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=pvIxQwiFj2vWD+5A5po5dfBhxIeOOD0+fMUphus6ujM=;
        b=PoliQTllbWACCNEqHkl9HQHngnLIC8OhFXPcLcJiQa8vTz+j0i1yDxdvwcwP0BBJmh
         vy9HbTi+QJ7i35Lni6CH3JmtoKN2wQgBXbO3OTdMInayhOwDE2uFR1pAJMvf3vecqqsY
         sio1Ul+e7eWPRY5jBZY5X/M/G1gdlewKPgxjBtldc4Sff0bAhFaEdadMxjr0QPOMvNwN
         nPYWo0pa4e1GvUjU4Hs7uE8gi9tyjiJBbzhrtr6B8c0OwK5JDJkgxlYg9n3Ai4nLCO69
         5/SYjb9gSE/+aNsXmL+K6uG74paxHWf+AFWjmAtbH/yT0KJGGpTwD31Ymk820K8VwgbO
         HUiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=pvIxQwiFj2vWD+5A5po5dfBhxIeOOD0+fMUphus6ujM=;
        b=X1lHrsgQNXjcGfUKU/qMw746bmbEYX17KGsFDfOT3ZE9ai4zhyFR/eQho9knUuPOOK
         PFafy1H3BOEVrjQ1gzm7OYmu33r7iTJ4MBuR2Mbn+bBhzOYnd/0rrZU3N1v1QILBpN/t
         0Uop0MF21Mkp0n7yT2XgRFfOC1XAfRN1sU95OoNmi0i3e9zMh8sKNS0jiK4474WoKOaD
         r9BSIfbZFPQKQ/4B1BKpM7xb4N5tn66ZuQyjRJDtxf+HLrXuPlbzNpS+JwCdgH0t+bzv
         AQ0eWwnphsxmWDEUiLbSl2eWpwW3sYxrks3aA/g85OIYDTIF1GDNcR75B8ukvXc765cH
         sRng==
X-Gm-Message-State: AOAM532N1QEHGAip5kR9fRbH3ftQskkxbs2Jcxv3gIE7RzGDsWJu3agc
	qsFQpu8Mv2TTQspOsfCHzyQ=
X-Google-Smtp-Source: ABdhPJziqN3oySqSk0sELbHySfDZIh5FEm0a+3K6x6Rl+vJMZh2uXYrSJLUumt0/BG0c5SByUOfk5g==
X-Received: by 2002:ac2:5451:: with SMTP id d17mr15835549lfn.367.1632082722891;
        Sun, 19 Sep 2021 13:18:42 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
 <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
Date: Sun, 19 Sep 2021 23:18:41 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 19.09.21 17:34, Julien Grall wrote:



> Hi Stefano,

Hi Julien


>
> On 18/09/2021 03:37, Stefano Stabellini wrote:
>> On Fri, 17 Sep 2021, Stefano Stabellini wrote:
>>> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>>>> +
>>>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>>>> +
>>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>>> +    if ( !unalloc_mem )
>>>>>> +        return -ENOMEM;
>>>>>> +
>>>>>> +    /* Start with all available RAM */
>>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>>> +        end = bootinfo.mem.bank[i].start + 
>>>>>> bootinfo.mem.bank[i].size - 1;
>>>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to add: 
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = assign_mem->bank[i].start;
>>>>>> +        end = assign_mem->bank[i].start + 
>>>>>> assign_mem->bank[i].size - 1;
>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove reserved-memory regions */
>>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>>>> +    {
>>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +        if ( res )
>>>>>> +        {
>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +                   start, end);
>>>>>> +            goto out;
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove grant table region */
>>>>>> +    start = kinfo->gnttab_start;
>>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>> +    if ( res )
>>>>>> +    {
>>>>>> +        printk(XENLOG_ERR "Failed to remove: 
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +               start, end);
>>>>>> +        goto out;
>>>>>> +    }
>>>>>> +
>>>>>> +    start = EXT_REGION_START;
>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>>> +                                 add_ext_regions, ext_regions);
>>>>>> +    if ( res )
>>>>>> +        ext_regions->nr_banks = 0;
>>>>>> +    else if ( !ext_regions->nr_banks )
>>>>>> +        res = -ENOENT;
>>>>>> +
>>>>>> +out:
>>>>>> +    rangeset_destroy(unalloc_mem);
>>>>>> +
>>>>>> +    return res;
>>>>>> +}
>>>>>> +
>>>>>> +static int __init find_memory_holes(const struct kernel_info 
>>>>>> *kinfo,
>>>>>> +                                    struct meminfo *ext_regions)
>>>>>> +{
>>>>>> +    struct dt_device_node *np;
>>>>>> +    struct rangeset *mem_holes;
>>>>>> +    paddr_t start, end;
>>>>>> +    unsigned int i;
>>>>>> +    int res;
>>>>>> +
>>>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>>>> +
>>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>>> +    if ( !mem_holes )
>>>>>> +        return -ENOMEM;
>>>>>> +
>>>>>> +    /* Start with maximum possible addressable physical memory 
>>>>>> range */
>>>>>> +    start = EXT_REGION_START;
>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>>> +    if ( res )
>>>>>> +    {
>>>>>> +        printk(XENLOG_ERR "Failed to add: 
>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>> +               start, end);
>>>>>> +        goto out;
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Remove all regions described by "reg" property (MMIO, 
>>>>>> RAM, etc) */
>>>>>
>>>>> Well... The loop below is not going to handle all the regions 
>>>>> described in
>>>>> the property "reg". Instead, it will cover a subset of "reg" where 
>>>>> the
>>>>> memory is addressable.
>>>>
>>>> As I understand, we are only interested in subset of "reg" where 
>>>> the memory is
>>>> addressable.
>>>>
>>>>
>>>>>
>>>>>
>>>>> You will also need to cover "ranges" that will describe the BARs 
>>>>> for the PCI
>>>>> devices.
>>>> Good point.
>>>
>>> Yes, very good point!
>>>
>>>
>>>> Could you please clarify how to recognize whether it is a PCI
>>>> device as long as PCI support is not merged? Or just to find any 
>>>> device nodes
>>>> with non-empty "ranges" property
>>>> and retrieve addresses?
>>>
>>> Normally any bus can have a ranges property with the aperture and
>>> possible address translations, including /amba (compatible =
>>> "simple-bus"). However, in these cases dt_device_get_address already
>>> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>>>
>>> The PCI bus is special for 2 reasons:
>>> - the ranges property has a different format
>>> - the bus is hot-pluggable
>>>
>>> So I think the only one that we need to treat specially is PCI.
>>>
>>> As far as I am aware PCI is the only bus (or maybe just the only bus
>>> that we support?) where ranges means the aperture.
>>
>> Now that I think about this, there is another "hotpluggable" scenario we
>> need to think about:
>>
>> [1] https://marc.info/?l=xen-devel&m=163056546214978
>>
>> Xilinx devices have FPGA regions with apertures currently not described
>> in device tree, where things can programmed in PL at runtime making new
>> devices appear with new MMIO regions out of thin air.
>>
>> Now let me start by saying that yes, the entire programmable region
>> aperture could probably be described in device tree, however, in
>> reality it is not currently done in any of the device trees we use
>> (including the upstream device trees in linux.git).
>
> This is rather annoying, but not unheard. There are a couple of 
> platforms where the MMIOs are not fully described in the DT.
>
> In fact, we have a callback 'specific_mappings' which create 
> additional mappings (e.g. on the omap5) for dom0.
>
>>
>> So, we have a problem :-(
>>
>>
>> I can work toward getting the right info on device tree, but in reality
>> that is going to take time and for now the device tree doesn't have the
>> FPGA aperture in it. So if we accept this series as is, it is going to
>> stop features like [1] from working. >
>> If we cannot come up with any better plans, I think it would be better
>> to drop find_memory_holes, only rely on find_unallocated_memory even
>> when the IOMMU is on. One idea is that we could add on top of the
>> regions found by find_unallocated_memory any MMIO regions marked as
>> xen,passthrough: they are safe because they are not going to dom0 
>> anyway.
>
> (Oleksandr, it looks like some rationale about the different approach 
> is missing in the commit message. Can you add it?)

Yes sure, but let me please clarify what is different approach in this 
context. Is it to *also* take into the account MMIO regions of the 
devices for passthrough for case when IOMMU is off (in addition to 
unallocated memory)? If yes, I wonder whether we will gain much with 
that according to that device's MMIO regions are usually not big enough 
and we stick to allocate extended regions with bigger size (> 64MB).


>
>
> When the IOMMU is on, Xen will do an extra mapping with GFN == MFN for 
> every grant mapping in dom0. This is because Linux will always program 
> the device with the MFN as it doesn't know whether the device has been 
> protected by the hypervisor.
>
> Therefore we can't use find_unallocated_memory() with the IOMMU on as 
> it stands.
>
>>
>> The only alternative I can think of is to have a per-board
>> enable/disable toggle for the extend region but it would be very ugly.
> At least, for your board, you seem to know the list of regions that 
> are reserved for future use. So how about adding a per-board list of 
> regions that should not be allocated?
>
> This will also include anything mentioned in 'specific_mappings'.
>
> Cheers,
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 00:04:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 00:04:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190207.340029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS6mp-0001Ai-9I; Mon, 20 Sep 2021 00:03:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190207.340029; Mon, 20 Sep 2021 00:03:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS6mp-0001Ab-5f; Mon, 20 Sep 2021 00:03:55 +0000
Received: by outflank-mailman (input) for mailman id 190207;
 Mon, 20 Sep 2021 00:03:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS6mn-0001AR-9P; Mon, 20 Sep 2021 00:03:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS6mn-0002hb-1b; Mon, 20 Sep 2021 00:03:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS6mm-0005Pf-L2; Mon, 20 Sep 2021 00:03:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mS6mm-0000Kj-Kb; Mon, 20 Sep 2021 00:03:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xaqKSQd7GPsdwAddkrM1BbtKS36s6kg2k5GZeR2gGmk=; b=Ga0x5fHTwu8xOequJpATXqRHVq
	ENPB7Ak6s2jgxy5NQlO3Nmbybby+3WGTPaMtOpbB55o78gEOLKx3LPqalaMI2mzqXv4r6ugRg3+gn
	fiBfFmqBYBntg+UBjJz4X4+2hJNfnv1g+1krMfffEldyJT39vxjl6VvF3nG94Ti14Uzg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165121-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165121: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    qemu-mainline:build-arm64-xsm:<job status>:broken:regression
    qemu-mainline:build-arm64-xsm:host-install(4):broken:regression
    qemu-mainline:build-arm64:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:<job status>:broken:regression
    qemu-mainline:build-arm64-pvops:host-install(4):broken:regression
    qemu-mainline:build-arm64:host-install(4):broken:regression
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-libvirt:debian-install:fail:heisenbug
    qemu-mainline:test-amd64-amd64-libvirt-vhd:guest-saverestore:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:heisenbug
    qemu-mainline:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    qemu-mainline:build-arm64-libvirt:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 00:03:52 +0000

flight 165121 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165121/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm                 <job status>                 broken
 build-arm64-xsm               4 host-install(4)        broken REGR. vs. 164950
 build-arm64                     <job status>                 broken  in 165116
 build-arm64-pvops               <job status>                 broken  in 165116
 build-arm64-pvops          4 host-install(4) broken in 165116 REGR. vs. 164950
 build-arm64                4 host-install(4) broken in 165116 REGR. vs. 164950
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt     12 debian-install   fail in 165116 pass in 165121
 test-amd64-amd64-libvirt-vhd 16 guest-saverestore          fail pass in 165116
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat  fail pass in 165116

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-seattle   1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)           blocked in 165116 n/a
 build-arm64-libvirt           1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl           1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)           blocked in 165116 n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    8 days
Failing since        164967  2021-09-13 13:06:52 Z    6 days   13 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    2 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              broken  
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 fail    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)
broken-job build-arm64-xsm broken
broken-job build-arm64 broken
broken-job build-arm64-pvops broken

Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 01:27:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 01:27:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190217.340043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS851-0007jw-RI; Mon, 20 Sep 2021 01:26:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190217.340043; Mon, 20 Sep 2021 01:26:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mS851-0007jp-NV; Mon, 20 Sep 2021 01:26:47 +0000
Received: by outflank-mailman (input) for mailman id 190217;
 Mon, 20 Sep 2021 01:26:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS84z-0007jf-VR; Mon, 20 Sep 2021 01:26:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS84z-0001yJ-RB; Mon, 20 Sep 2021 01:26:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mS84z-0001jK-FQ; Mon, 20 Sep 2021 01:26:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mS84z-0007kh-Es; Mon, 20 Sep 2021 01:26:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=S4y+tDKdtcXzDALDqCLQ8HpNi4Z4ePNEPsSkZl1Yi4c=; b=LFHl7lGjSwcPa9iO10e/D+KDD5
	J85QE03XstWxkCy6oNewBqUS68WDrLa6UygqkUbkd9nyiPKnGWwVBOgbx0yoELWKIeDUByPsunbps
	R5m3dFE+javXgiHfazK5waHup1hsPMTvfPZ15Qfzx7KjbCey1wz+uQVW3wuP/g47vQxg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165122-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165122: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d4d016caa4b85b9aa98d7ec8c84e928621a614bc
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 01:26:45 +0000

flight 165122 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165122/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                d4d016caa4b85b9aa98d7ec8c84e928621a614bc
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  415 days
Failing since        152366  2020-08-01 20:49:34 Z  414 days  722 attempts
Testing same since   165117  2021-09-19 03:07:42 Z    0 days    2 attempts

------------------------------------------------------------
7317 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2261430 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 05:01:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 05:01:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190230.340057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSBQE-0005Mw-S2; Mon, 20 Sep 2021 05:00:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190230.340057; Mon, 20 Sep 2021 05:00:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSBQE-0005Mp-Oh; Mon, 20 Sep 2021 05:00:54 +0000
Received: by outflank-mailman (input) for mailman id 190230;
 Mon, 20 Sep 2021 05:00:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSBQC-0005Mh-Om
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 05:00:52 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c38f531d-3ba5-4e8b-bb69-3fdc0f73afe7;
 Mon, 20 Sep 2021 05:00:51 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6883D21F96;
 Mon, 20 Sep 2021 05:00:50 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5209F13AC9;
 Mon, 20 Sep 2021 05:00:50 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id P727EoIVSGFxRQAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 05:00:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c38f531d-3ba5-4e8b-bb69-3fdc0f73afe7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632114050; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=DU3s4YmsIh5ZGnyJm3rw1HbPXvF4HLEnafqBXU5a/+E=;
	b=IT2HVrYqYLxNoaDTX4CPKHmV151ePx3B1TdwN+CktujabPEEFPhBJS9lHqiHVifWQMTnV7
	fFRt3DKc0iABLdov909RLxKvlhL3pJWfvWunX2DsKRNsAHih/crMrIdF78nzuIxDUmKNHs
	DoFUznCioIXRfA9Xx1aeNsFBm4XEcZI=
Subject: Re: [linux-linus test] 165122: regressions - FAIL
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org
References: <osstest-165122-mainreport@xen.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <7f310a51-471c-8b2f-9d0e-5f5a5ed1e765@suse.com>
Date: Mon, 20 Sep 2021 07:00:49 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <osstest-165122-mainreport@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="AljxGCrPcO425AYhuasZsrvn96ME2Jfv3"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--AljxGCrPcO425AYhuasZsrvn96ME2Jfv3
Content-Type: multipart/mixed; boundary="0v7ZkDnGmK2plBidXbJntrf8OMKh5SvGP";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org
Message-ID: <7f310a51-471c-8b2f-9d0e-5f5a5ed1e765@suse.com>
Subject: Re: [linux-linus test] 165122: regressions - FAIL
References: <osstest-165122-mainreport@xen.org>
In-Reply-To: <osstest-165122-mainreport@xen.org>

--0v7ZkDnGmK2plBidXbJntrf8OMKh5SvGP
Content-Type: multipart/mixed;
 boundary="------------1B1F2A7E7A3B138CD10E16A5"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------1B1F2A7E7A3B138CD10E16A5
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 03:26, osstest service owner wrote:
> flight 165122 linux-linus real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/165122/
>=20
> Regressions :-(
>=20
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>   test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. v=
s. 152332
>   test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-xl          17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. v=
s. 152332
>   test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. v=
s. 152332
>   test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail=
 REGR. vs. 152332
>   test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. v=
s. 152332
>   test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. v=
s. 152332
>   test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. v=
s. 152332
>   test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. v=
s. 152332
>   test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. v=
s. 152332
>   test-arm64-arm64-xl          13 debian-fixup             fail REGR. v=
s. 152332
>   test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. v=
s. 152332

Nice to see the linux-linus tests being usable again. :-)

They already found a new bug in the kernel, introduced by one of my
recent patches. Writing a fix now.


Juergen

--------------1B1F2A7E7A3B138CD10E16A5
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------1B1F2A7E7A3B138CD10E16A5--

--0v7ZkDnGmK2plBidXbJntrf8OMKh5SvGP--

--AljxGCrPcO425AYhuasZsrvn96ME2Jfv3
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIFYEFAwAAAAAACgkQsN6d1ii/Ey/x
fAgAmduzzRlGFMlbuGsVhLiCrb8VhpJpHAzI2q9CZaPtgYogNFgM5QGXnW91BQ1iFhICwlbwMkRT
26ibQWdKVjX7jERr2ejdFhQ3PS9Pgqq62vS6xa95ok1HnAa95TAvZQw1nWFd3wqYXIxVc1nklXOO
XoFnnvbylz4rUkGmvkzEEFUx90zyeVNOABqXIynEGbS94RnL0l8UIQkbciMwmwlzGES76bqshe/l
B7o+VKDSfjKyHVPXuKt5na5dJqoL0tPL5Op1J7sVgNOd4efhJ1177KsJuNCpLUj5jxDE9zbf+YhU
O/jlPEsRrU+qRvhVBTV7ajXdSUk+WpInnOW2FFgoVA==
=b/+A
-----END PGP SIGNATURE-----

--AljxGCrPcO425AYhuasZsrvn96ME2Jfv3--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 05:23:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 05:23:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190236.340067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSBmI-0007ti-P8; Mon, 20 Sep 2021 05:23:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190236.340067; Mon, 20 Sep 2021 05:23:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSBmI-0007tb-LY; Mon, 20 Sep 2021 05:23:42 +0000
Received: by outflank-mailman (input) for mailman id 190236;
 Mon, 20 Sep 2021 05:23:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OU0u=OK=epam.com=prvs=9897642576=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSBmH-0007tV-4s
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 05:23:41 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3a83a80e-f47d-4895-a5d4-2d6ef7ecb151;
 Mon, 20 Sep 2021 05:23:39 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18K4Rops002978;
 Mon, 20 Sep 2021 05:23:37 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2058.outbound.protection.outlook.com [104.47.14.58])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b6faf0r1h-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 20 Sep 2021 05:23:37 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3376.eurprd03.prod.outlook.com (2603:10a6:803:1d::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 05:23:34 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 05:23:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a83a80e-f47d-4895-a5d4-2d6ef7ecb151
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R0WLSj+DdoG/YX3vylgdZV8yCHfrHg6kt5lswywdO5iWSGf2RLrPV3B1IeumOczwtNV7JsFm+P3lmsb//r6IHEEuQiKETdFspQrHegOdSmHRUorulBTpa8+YzOIvk5qUpEHCM+EE9qVLkCJOswjEcQO9eSMrciZrWpTTxf6zZXwHNwd/e2dZCKgsAkU2zoYG6uQ8fi3rWNzMAL3Fxw9YyawEoKyWY/sfLfJU+5ydW3ZaxJaTXa5TQ/2Wp12K4SQyjQYtPNaIh2CRhQo51kRStFFfkAuWSWg6GHM8VYSLPTLZxQ+zn+qzUG8jzH6nHq4hkCm5GN9nf3Q3TTXmpLGprg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9oYxHWWTaB+OfLkBIGIw9qw7VCs2YrGkcF6M7Gcj0u4=;
 b=iqevxfb9kbedC4IYur/EKAdSpo5rqNWVDUeerYkfbUC4Uo4ubRKTZ5XLWHLVo4qoN+AQCcUaYV7KgMJWgBCjiF1AvDewGdMm5QvHP5oQSYnq4upugGH5fMtDMgd3b5bgCUac0xHHZvbnW5WacYMw2jdqKEQ9SUc9k7GHVzPFCNYlAobKbAp70P7WCUuky7cI9M7aRl9awF7lSK1f/tm+lyoirJbKcmPtsZnF/FlmytCzvovYeOT65yMJBbSLmYgticSB0pi7Ip2Rjk4nea4nMIPnYvpPkAxVrPD2ZR6MjSSCXLWp7Lk8fBhpTbVlWpOr6GPrVkhOIXVkCfzut0FD6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9oYxHWWTaB+OfLkBIGIw9qw7VCs2YrGkcF6M7Gcj0u4=;
 b=vZL1EV/QHs9RSaAcku1buP66ZAhfOII3BSs1c84JosM2UR+aT0FJd7ua50ol+NeCOA1Amh5Mq7gEVupHcXe59RwPHZ8N/5wWEtwHuIkC24R1G6ptzul82L+GJwDixud8EGH69HdvbzHWzNbttjrVCbnQHfsdaPVt6/chTPocNW2QrrRk/FXwLgzKtdRsmhDAJ3jpnjHOV1E39qf+KeO//vjVnRinJeSL+qQ7PfUq9CMrrF/L5KqQ3cSW0z5/Qu0CDV2g+0ZnTcs4kdy8sPDCg5AQXKr3GvdyVABlodUcr/BVQi6k9OYkTwnybEYZo8SkhTcDT9fAzMoOai6Kjv3aPg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com"
	<jgross@suse.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>,
        Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhIA=
Date: Mon, 20 Sep 2021 05:23:34 +0000
Message-ID: <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: bec98b93-7de1-4b3d-c558-08d97bf6caa7
x-ms-traffictypediagnostic: VI1PR0302MB3376:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <VI1PR0302MB33760442E4E0C8554807E855E7A09@VI1PR0302MB3376.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 o0cG3/WxHOEbjGdGAI48KzextUd8URO3v36EBCM+3Xi7Xn4ROiSz56PqtusXO5NO6+4R5d8ENH1ubl9/1LeUNC6BQyPuoffNV+7oSMjHHxoiH5gW5jRQ++aE2huQTQQYJSBWc6HA36OeparWL94d7Bz2VT7otVM0NdEERhARLMubXU57jV9K4ZCkTrWqdBKFKjoUu/vxefTQ05xUigsM8cWbBmGHlqigDtlqYoHA0I97GJyPKqyk5VY2czHgHJ8rWVcfGiVXJAbckvBHVDpjebPdzfGUvk2LQmSgC0XMuxh2+3s1cDBdGY/2GiTxLYvePsY4fMXCiSmSp3fAg5MQqPpktN4I9EnHeKPy0Me+l5xw9vgHwZDzp+6PrgN15UteGcln9VZhVjKz8ahfrDXTl4MKS5Va/t17RXWQrFI+tH+3WKeryWZ8gmR9OhoZdjlbKrZr5dpR7jb3xSN8JDLFIKyVnow6vvtONTy9MBlyLDMx3vs65+2p6rhi6btJ6U3zGsxMPV/Eh+JW0l+ylKJ87T4JxEO59u/vXVSsbvt092DgB9d6e0K8zHyJ+5pCMtu+wH0NhNSUTRl5DRRwiIhuFw2RtFM+bhb7lkO93qgNMQ7FtWDMlmxJK3XJwAL7k/y3sXbD5vc56JipuFU9nS6EKNpteFlZRkNnEVRWv4uRDv6Fncz2npL0ItNPS5L90+opzq4gZvLNwVvRozQitpvrjOuzBE5Q1ySqgUMJp2HuXr4geVuXTP56EbcGoeXBrmLgSd9M5ZAGrywAyE//7oQNYQmu09jq3x/9F/1SuXQJWi+xGWPSil3gfFhHlRYh31h8
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(107886003)(91956017)(86362001)(31696002)(83380400001)(38070700005)(6486002)(76116006)(8676002)(31686004)(6512007)(6506007)(508600001)(71200400001)(66946007)(26005)(66556008)(64756008)(66476007)(38100700002)(66446008)(53546011)(110136005)(2906002)(30864003)(316002)(5660300002)(54906003)(36756003)(966005)(122000001)(2616005)(186003)(4326008)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RTI3MGRoUVFqLzZXR0V1TkduUEJXb1FHVVlLc1pxWURqekdwcVV0cWtHTmtP?=
 =?utf-8?B?eTQ0ZThYWDM5VzhMd2tsUFpMUWFXTEdUTW5oVUhySHhoMllBTEpvTUJCVkYx?=
 =?utf-8?B?VEU5L0pKMTFVQnFiUnRkRzZ2V2dwbzVqZWJlbG8xSTg3eWRWOEJXT1JJMnIz?=
 =?utf-8?B?NlFIMkZWTmwzWm9EWkQ4TnZZMmpmdjZVZmxwbXRPVG41Vk1CSlljekUxMXZ5?=
 =?utf-8?B?UDZvVHVVbTVRUVNtNUl1Q1pjN2xjTDZQM2g2bnoxaHhramJFbjg0aGxQeDVv?=
 =?utf-8?B?U0VxWEIwQTk1SVpMT3VJK2VPM0pzUlFrWEdjZmxTRlNxM2dNcE9PbUFudzBr?=
 =?utf-8?B?LzlzV2Y1a0pmcFRKOHJ4b0hYT3BaWStwZy9uZFZzWFVMNDNsTG85OFcySm54?=
 =?utf-8?B?dE55S1gxV0crZ1JjR0F0bGFjYnBuQm05QWJGSVBuNHFMZzNGK2lCVVJOcVFM?=
 =?utf-8?B?UGp5UGJZeDFoUTh4TitEcHV6SnAwaXkvNnY4OUZMQ2dWbjNHNFVHTS9pQWc2?=
 =?utf-8?B?YWc1K1A0dlJCMFlsWnJRTUhFTmJDMDZJL3Z5bHdsZGJUNlYvaVVQTFJoK0J0?=
 =?utf-8?B?QURlbUFlQlNKYm5wVDRmOWRHaHJLbTJqdXR1MnZYWkxVTlJhR2JtZE0rREYw?=
 =?utf-8?B?VkJmeEthcDJzdzhNUDY0cXlibjZNTldJYjZWUzBJamgzajhLSjNOVHE5NVRk?=
 =?utf-8?B?NXA4YnBMUnAyY0RhbkVrNU01Q1dFZDRuV0NGRldES1R2ZHQ3S0RUaGlPejlr?=
 =?utf-8?B?cTlLVGsva0xNMWJVM2ovNURQZm9hNnE3Qk1Db1lKUGdFS0tRM3g4eTg3eUNn?=
 =?utf-8?B?Z3lQZmVBd1dKSTg4M3d1UExxOFBGT3ptaXEyUGJnSktTVElUdXpsQldpSXg1?=
 =?utf-8?B?T1pVQjJiQTI3UVluVjRtWjdaQzFXc01BUk9PSmEvYm1tYUpsUmVmUFI1UnpU?=
 =?utf-8?B?cWdoNlh1ZXpjYUl1VG54RHJHS1N4Q0hMQytpQXcyM3JCT1YyWHJNcjdsZ0U1?=
 =?utf-8?B?UHV4Y3d0M04xMDd1a3lVQjkxeFYxZ3VRc2ZZdk5DSU9CYmV1eEVoLzV0b2lC?=
 =?utf-8?B?eEo5WEt4NTZIaVdwV21DYW5UcVdhVVcvRHByRWVPNFVjN3VBcmlDZ1NBK3hY?=
 =?utf-8?B?cHhkT0RPRzRLcUQrZHFnZmxFc2xyTWZjd3B1VlJDcUk4RlhqU1UzclZHckds?=
 =?utf-8?B?QnI3WjJKL25ueVpmK0VNZ3JwdHBKajlYUFFUNmR1OGl6U0RQUWVxZ0hoclNS?=
 =?utf-8?B?WFAyeGN5bTVpRkYrRTNPM09WNWZKR0NyQTMxMDB6SlFGWDMzeG5LTVl0eTl6?=
 =?utf-8?B?S2pPOHBRcjFRZEFkSUxVUE9ONXA4NDY2UjJWRmNHNjBNci9jWmFrVFcrbVlH?=
 =?utf-8?B?Tk80Ly9QUzg1cVZDcmZQM21uU0dJUEdjbTZVQnRWOTExbERZV3BvQ0FNSzZY?=
 =?utf-8?B?MGdvb0NXTXhHL0hUdEhRRHpYU0pFWU45cFVONDlIUXlGSGQzL0RGSktKRlVH?=
 =?utf-8?B?cUs5Sk5LWEhVdjRORGlJMUVVTERkbXRZdEtxQVhGRXI2T1M0QTJmaXl4RVdq?=
 =?utf-8?B?UGRKdnIwc0J0QURqcmVPbnNocmd2SDN3bHRMcnN0TWF2VWhYLy9jbmtIcEV1?=
 =?utf-8?B?K0JzdU5oSUlLOCtUYzg0Rm56SVgyckYra1YxK2c3TjJiZFJsancvMTEydVN4?=
 =?utf-8?B?NGNwNE5DUktJVGR4YnBJcktja0NLQzI3Q0xiSk11d0UvNVZxT1l4MlVpVWx5?=
 =?utf-8?Q?ZadZTOdh9rLco86QcwPyVktV2lq3xKGKgjs/lw/?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <8099C3C312984943A7E0A43B76ABF187@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bec98b93-7de1-4b3d-c558-08d97bf6caa7
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2021 05:23:34.0953
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jPn93yeHadBNfmBlMtY3JUrjleQiggI1a+EEvCRXAJw+u6BTqN0k4g4kWdfqlPeAGSm+DHWDkLIg4Gj6THFG6q14i7hs1neWUqAl1z/LtXho+zOsxqQACdXRMDdHAjIj
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3376
X-Proofpoint-GUID: FvBWiBq5sKVxRYRveEy9I3zIhjwJQJOw
X-Proofpoint-ORIG-GUID: FvBWiBq5sKVxRYRveEy9I3zIhjwJQJOw
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-20_01,2021-09-17_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 clxscore=1011 bulkscore=0 mlxscore=0 malwarescore=0 suspectscore=0
 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 spamscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109200031

SGVsbG8sIFN0ZWZhbm8hDQoNCk9uIDE4LjA5LjIxIDAwOjQ1LCBTdGVmYW5vIFN0YWJlbGxpbmkg
d3JvdGU6DQo+IEhpIE9sZWtzYW5kciwNCj4NCj4gV2h5IGRvIHlvdSB3YW50IHRvIGVuYWJsZSBw
Y2liYWNrIG9uIEFSTT8gSXMgaXQgb25seSB0byAiZGlzYWJsZSIgYSBQQ0kNCj4gZGV2aWNlIGlu
IERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRvIGEgRG9tVT8NCk5vdCBv
bmx5IHRoYXQNCj4NCj4gSSBhbSBhc2tpbmcgYmVjYXVzZSBhY3R1YWxseSBJIGRvbid0IHRoaW5r
IHdlIHdhbnQgdG8gZW5hYmxlIHRoZSBQViBQQ0kNCj4gYmFja2VuZCBmZWF0dXJlIG9mIHBjaWJh
Y2sgb24gQVJNLCByaWdodD8gVGhhdCB3b3VsZCBjbGFzaCB3aXRoIHRoZSBQQ0kNCj4gYXNzaWdu
bWVudCB3b3JrIHlvdSBoYXZlIGJlZW4gZG9pbmcgaW4gWGVuLiBUaGV5IGNvdWxkbid0IGJvdGgg
d29yayBhdA0KPiB0aGUgc2FtZSB0aW1lLg0KQ29ycmVjdCwgaXQgaXMgbm90IHVzZWQNCj4NCj4g
SWYgd2Ugb25seSBuZWVkIHBjaWJhY2sgdG8gInBhcmsiIGEgZGV2aWNlIGluIERvbTAsIHdvdWxk
bid0IGl0IGJlDQo+IHBvc3NpYmxlIGFuZCBiZXR0ZXIgdG8gdXNlIHBjaS1zdHViIGluc3RlYWQ/
DQoNCk5vdCBvbmx5IHRoYXQsIHNvIHBjaS1zdHViIGlzIG5vdCBlbm91Z2gNCg0KVGhlIGZ1bmN0
aW9uYWxpdHkgd2hpY2ggaXMgaW1wbGVtZW50ZWQgYnkgdGhlIHBjaWJhY2sgYW5kIHRoZSB0b29s
c3RhY2sNCmFuZCB3aGljaCBpcyByZWxldmFudC9taXNzaW5nL25lZWRlZCBmb3IgQVJNOg0KDQox
LiBwY2liYWNrIGlzIHVzZWQgYXMgYSBkYXRhYmFzZSBmb3IgYXNzaWduYWJsZSBQQ0kgZGV2aWNl
cywgZS5nLiB4bA0KIMKgwqAgcGNpLWFzc2lnbmFibGUte2FkZHxyZW1vdmV8bGlzdH0gbWFuaXB1
bGF0ZXMgdGhhdCBsaXN0LiBTbywgd2hlbmV2ZXIgdGhlDQogwqDCoCB0b29sc3RhY2sgbmVlZHMg
dG8ga25vdyB3aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2ggaXQgcmVhZHMN
CiDCoMKgIHRoYXQgZnJvbSB0aGUgcmVsZXZhbnQgc3lzZnMgZW50cmllcyBvZiB0aGUgcGNpYmFj
ay4NCg0KMi4gcGNpYmFjayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVuYm91bmQgUENJIGRldmljZXMs
IGUuZy4gd2hlbiBwYXNzaW5nIHRocm91Z2gNCiDCoMKgIGEgUENJIGRldmljZSBpdCBuZWVkcyB0
byBiZSB1bmJvdW5kIGZyb20gdGhlIHJlbGV2YW50IGRldmljZSBkcml2ZXIgYW5kIGJvdW5kDQog
wqDCoCB0byBwY2liYWNrIChzdHJpY3RseSBzcGVha2luZyBpdCBpcyBub3QgcmVxdWlyZWQgdGhh
dCB0aGUgZGV2aWNlIGlzIGJvdW5kIHRvDQogwqDCoCBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBh
Z2FpbiB1c2VkIGFzIGEgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSQ0KIMKgwqAg
ZGV2aWNlcywgc28gd2UgY2FuIHJlLWJpbmQgdGhlIGRldmljZXMgYmFjayB0byB0aGVpciBvcmln
aW5hbCBkcml2ZXJzIHdoZW4NCiDCoMKgIGd1ZXN0IGRvbWFpbiBzaHV0cyBkb3duKQ0KDQozLiBE
ZXZpY2UgcmVzZXQNCg0KV2UgaGF2ZSBwcmV2aW91c2x5IGRpc2N1c3NlZCBvbiB4ZW4tZGV2ZWwg
TUwgcG9zc2libGUgc29sdXRpb25zIHRvIHRoYXQgYXMgZnJvbSB0aGUNCg0KYWJvdmUgd2Ugc2Vl
IHRoYXQgcGNpYmFjayBmdW5jdGlvbmFsaXR5IGlzIGdvaW5nIHRvIGJlIG9ubHkgcGFydGlhbGx5
IHVzZWQgb24gQXJtLg0KDQpQbGVhc2Ugc2VlIFsxXSBhbmQgWzJdOg0KDQoxLiBJdCBpcyBub3Qg
YWNjZXB0YWJsZSB0byBtYW5hZ2UgdGhlIGFzc2lnbmFibGUgbGlzdCBpbiBYZW4gaXRzZWxmDQoN
CjIuIHBjaWJhY2sgY2FuIGJlIHNwbGl0IGludG8gdHdvIHBhcnRzOiBQQ0kgYXNzaWduYWJsZS9i
aW5kL3Jlc2V0IGhhbmRsaW5nIGFuZA0KDQp0aGUgcmVzdCBsaWtlIHZQQ0kgZXRjLg0KDQozLiBw
Y2lmcm9udCBpcyBub3QgdXNlZCBvbiBBcm0NCg0KU28sIGxpbWl0ZWQgdXNlIG9mIHRoZSBwY2li
YWNrIGlzIG9uZSBvZiB0aGUgYnJpY2tzIHVzZWQgdG8gZW5hYmxlIFBDSSBwYXNzdGhyb3VnaA0K
DQpvbiBBcm0uIEl0IHdhcyBlbm91Z2ggdG8ganVzdCByZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBh
bmQgaGF2ZSBpdCBydW4gb24gQXJtIHRvIGFjaGlldmUNCg0KYWxsIHRoZSBnb2FscyBhYm92ZS4N
Cg0KSWYgd2Ugc3RpbGwgdGhpbmsgaXQgaXMgZGVzaXJhYmxlIHRvIGJyZWFrIHRoZSBwY2liYWNr
IGRyaXZlciBpbnRvICJjb21tb24iIGFuZCAicGNpZnJvbnQgc3BlY2lmaWMiDQoNCnBhcnRzIHRo
ZW4gaXQgY2FuIGJlIGRvbmUsIHlldCB0aGUgcGF0Y2ggaXMgZ29pbmcgdG8gYmUgdGhlIHZlcnkg
Zmlyc3QgYnJpY2sgaW4gdGhhdCBidWlsZGluZy4NCg0KU28sIEkgdGhpbmsgdGhpcyBwYXRjaCBp
cyBzdGlsbCBnb2luZyB0byBiZSBuZWVkZWQgYmVzaWRlcyB3aGljaCBkaXJlY3Rpb24gd2UgdGFr
ZS4NCg0KSG9wZSB0aGlzIHNoZWRzIHNvbWUgbGlnaHQsDQoNCk9sZWtzYW5kcg0KDQo+DQo+DQo+
IE9uIEZyaSwgMTcgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4g
RnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVw
YW0uY29tPg0KPj4NCj4+IFhlbi1wY2liYWNrIGRyaXZlciB3YXMgZGVzaWduZWQgdG8gYmUgYnVp
bHQgZm9yIHg4NiBvbmx5LiBCdXQgaXQNCj4+IGNhbiBhbHNvIGJlIHVzZWQgYnkgb3RoZXIgYXJj
aGl0ZWN0dXJlcywgZS5nLiBBcm0uDQo+PiBSZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBpbiBhIHdh
eSB0aGF0IGl0IGNhbiBiZSBidWlsdCBmb3Igb3RoZXINCj4+IHBsYXRmb3JtcyBhcyB3ZWxsLg0K
Pj4NCj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJf
YW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEFuYXN0YXNpaWEgTHVr
aWFuZW5rbyA8YW5hc3Rhc2lpYV9sdWtpYW5lbmtvQGVwYW0uY29tPg0KPj4NCj4+IC0tLQ0KPj4g
VGVzdGVkIG9uIEFybSBhbmQgeDg2Lg0KPj4gLS0tDQo+PiAgIGFyY2gveDg2L2luY2x1ZGUvYXNt
L3hlbi9wY2kuaCAgICAgfCAyNCAtLS0tLS0tLS0tDQo+PiAgIGFyY2gveDg2L3BjaS94ZW4uYyAg
ICAgICAgICAgICAgICAgfCA3NCArLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPj4gICBk
cml2ZXJzL3hlbi9LY29uZmlnICAgICAgICAgICAgICAgIHwgIDIgKy0NCj4+ICAgZHJpdmVycy94
ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMgICB8ICAxICsNCj4+ICAgZHJpdmVycy94ZW4vcGNpLmMg
ICAgICAgICAgICAgICAgICB8IDc1ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4g
ICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5jIHwgIDMgKy0NCj4+ICAgZHJpdmVy
cy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMgICB8ICAyICstDQo+PiAgIGluY2x1ZGUveGVuL3Bj
aS5oICAgICAgICAgICAgICAgICAgfCAzNCArKysrKysrKysrKysrKw0KPj4gICA4IGZpbGVzIGNo
YW5nZWQsIDExNSBpbnNlcnRpb25zKCspLCAxMDAgZGVsZXRpb25zKC0pDQo+PiAgIGNyZWF0ZSBt
b2RlIDEwMDY0NCBpbmNsdWRlL3hlbi9wY2kuaA0KPj4NCj4+IGRpZmYgLS1naXQgYS9hcmNoL3g4
Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgN
Cj4+IGluZGV4IDM1MDZkOGM1OThjMS4uOWZmN2I0OWJjYTA4IDEwMDY0NA0KPj4gLS0tIGEvYXJj
aC94ODYvaW5jbHVkZS9hc20veGVuL3BjaS5oDQo+PiArKysgYi9hcmNoL3g4Ni9pbmNsdWRlL2Fz
bS94ZW4vcGNpLmgNCj4+IEBAIC0xNCwzMCArMTQsNiBAQCBzdGF0aWMgaW5saW5lIGludCBwY2lf
eGVuX2h2bV9pbml0KHZvaWQpDQo+PiAgIAlyZXR1cm4gLTE7DQo+PiAgIH0NCj4+ICAgI2VuZGlm
DQo+PiAtI2lmIGRlZmluZWQoQ09ORklHX1hFTl9ET00wKQ0KPj4gLWludCBfX2luaXQgcGNpX3hl
bl9pbml0aWFsX2RvbWFpbih2b2lkKTsNCj4+IC1pbnQgeGVuX2ZpbmRfZGV2aWNlX2RvbWFpbl9v
d25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2KTsNCj4+IC1pbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9k
b21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwgdWludDE2X3QgZG9tYWluKTsNCj4+IC1p
bnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2
KTsNCj4+IC0jZWxzZQ0KPj4gLXN0YXRpYyBpbmxpbmUgaW50IF9faW5pdCBwY2lfeGVuX2luaXRp
YWxfZG9tYWluKHZvaWQpDQo+PiAtew0KPj4gLQlyZXR1cm4gLTE7DQo+PiAtfQ0KPj4gLXN0YXRp
YyBpbmxpbmUgaW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYg
KmRldikNCj4+IC17DQo+PiAtCXJldHVybiAtMTsNCj4+IC19DQo+PiAtc3RhdGljIGlubGluZSBp
bnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwN
Cj4+IC0JCQkJCQkgICB1aW50MTZfdCBkb21haW4pDQo+PiAtew0KPj4gLQlyZXR1cm4gLTE7DQo+
PiAtfQ0KPj4gLXN0YXRpYyBpbmxpbmUgaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5f
b3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXJldHVybiAtMTsNCj4+IC19
DQo+PiAtI2VuZGlmDQo+PiAgIA0KPj4gICAjaWYgZGVmaW5lZChDT05GSUdfUENJX01TSSkNCj4+
ICAgI2lmIGRlZmluZWQoQ09ORklHX1BDSV9YRU4pDQo+PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYv
cGNpL3hlbi5jIGIvYXJjaC94ODYvcGNpL3hlbi5jDQo+PiBpbmRleCAzZDQxYTA5YzJjMTQuLjRh
NDViMGJmOWFlNCAxMDA2NDQNCj4+IC0tLSBhL2FyY2gveDg2L3BjaS94ZW4uYw0KPj4gKysrIGIv
YXJjaC94ODYvcGNpL3hlbi5jDQo+PiBAQCAtMjMsNiArMjMsNyBAQA0KPj4gICANCj4+ICAgI2lu
Y2x1ZGUgPHhlbi9mZWF0dXJlcy5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4g
KyNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiAgICNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4g
ICAjaW5jbHVkZSA8YXNtL3hlbi9jcHVpZC5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL2FwaWMuaD4N
Cj4+IEBAIC01ODMsNzcgKzU4NCw0IEBAIGludCBfX2luaXQgcGNpX3hlbl9pbml0aWFsX2RvbWFp
bih2b2lkKQ0KPj4gICAJfQ0KPj4gICAJcmV0dXJuIDA7DQo+PiAgIH0NCj4+IC0NCj4+IC1zdHJ1
Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgew0KPj4gLQlkb21pZF90IGRvbWFpbjsNCj4+IC0J
c3RydWN0IHBjaV9kZXYgKmRldjsNCj4+IC0Jc3RydWN0IGxpc3RfaGVhZCBsaXN0Ow0KPj4gLX07
DQo+PiAtDQo+PiAtc3RhdGljIERFRklORV9TUElOTE9DSyhkZXZfZG9tYWluX2xpc3Rfc3Bpbmxv
Y2spOw0KPj4gLXN0YXRpYyBzdHJ1Y3QgbGlzdF9oZWFkIGRldl9kb21haW5fbGlzdCA9IExJU1Rf
SEVBRF9JTklUKGRldl9kb21haW5fbGlzdCk7DQo+PiAtDQo+PiAtc3RhdGljIHN0cnVjdCB4ZW5f
ZGV2aWNlX2RvbWFpbl9vd25lciAqZmluZF9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+
IC17DQo+PiAtCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiAtDQo+
PiAtCWxpc3RfZm9yX2VhY2hfZW50cnkob3duZXIsICZkZXZfZG9tYWluX2xpc3QsIGxpc3QpIHsN
Cj4+IC0JCWlmIChvd25lci0+ZGV2ID09IGRldikNCj4+IC0JCQlyZXR1cm4gb3duZXI7DQo+PiAt
CX0NCj4+IC0JcmV0dXJuIE5VTEw7DQo+PiAtfQ0KPj4gLQ0KPj4gLWludCB4ZW5fZmluZF9kZXZp
Y2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLQlzdHJ1Y3Qg
eGVuX2RldmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPj4gLQlpbnQgZG9tYWluID0gLUVOT0RF
VjsNCj4+IC0NCj4+IC0Jc3Bpbl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4g
LQlvd25lciA9IGZpbmRfZGV2aWNlKGRldik7DQo+PiAtCWlmIChvd25lcikNCj4+IC0JCWRvbWFp
biA9IG93bmVyLT5kb21haW47DQo+PiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3Bp
bmxvY2spOw0KPj4gLQlyZXR1cm4gZG9tYWluOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQ
TCh4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4+IC0NCj4+IC1pbnQgeGVuX3JlZ2lz
dGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwgdWludDE2X3QgZG9t
YWluKQ0KPj4gLXsNCj4+IC0Jc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsN
Cj4+IC0NCj4+IC0Jb3duZXIgPSBremFsbG9jKHNpemVvZihzdHJ1Y3QgeGVuX2RldmljZV9kb21h
aW5fb3duZXIpLCBHRlBfS0VSTkVMKTsNCj4+IC0JaWYgKCFvd25lcikNCj4+IC0JCXJldHVybiAt
RU5PREVWOw0KPj4gLQ0KPj4gLQlzcGluX2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7
DQo+PiAtCWlmIChmaW5kX2RldmljZShkZXYpKSB7DQo+PiAtCQlzcGluX3VubG9jaygmZGV2X2Rv
bWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC0JCWtmcmVlKG93bmVyKTsNCj4+IC0JCXJldHVybiAt
RUVYSVNUOw0KPj4gLQl9DQo+PiAtCW93bmVyLT5kb21haW4gPSBkb21haW47DQo+PiAtCW93bmVy
LT5kZXYgPSBkZXY7DQo+PiAtCWxpc3RfYWRkX3RhaWwoJm93bmVyLT5saXN0LCAmZGV2X2RvbWFp
bl9saXN0KTsNCj4+IC0Jc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+
PiAtCXJldHVybiAwOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fcmVnaXN0ZXJf
ZGV2aWNlX2RvbWFpbl9vd25lcik7DQo+PiAtDQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2Rldmlj
ZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXN0cnVjdCB4
ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiAtDQo+PiAtCXNwaW5fbG9jaygmZGV2
X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC0Jb3duZXIgPSBmaW5kX2RldmljZShkZXYpOw0K
Pj4gLQlpZiAoIW93bmVyKSB7DQo+PiAtCQlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3Nw
aW5sb2NrKTsNCj4+IC0JCXJldHVybiAtRU5PREVWOw0KPj4gLQl9DQo+PiAtCWxpc3RfZGVsKCZv
d25lci0+bGlzdCk7DQo+PiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2sp
Ow0KPj4gLQlrZnJlZShvd25lcik7DQo+PiAtCXJldHVybiAwOw0KPj4gLX0NCj4+IC1FWFBPUlRf
U1lNQk9MX0dQTCh4ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4+ICAgI2Vu
ZGlmDQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4vS2NvbmZpZyBiL2RyaXZlcnMveGVuL0tj
b25maWcNCj4+IGluZGV4IGEzN2ViNTJmYjQwMS4uMDU3ZGRmNjFlZjYxIDEwMDY0NA0KPj4gLS0t
IGEvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4gKysrIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4g
QEAgLTE4Miw3ICsxODIsNyBAQCBjb25maWcgU1dJT1RMQl9YRU4NCj4+ICAgDQo+PiAgIGNvbmZp
ZyBYRU5fUENJREVWX0JBQ0tFTkQNCj4+ICAgCXRyaXN0YXRlICJYZW4gUENJLWRldmljZSBiYWNr
ZW5kIGRyaXZlciINCj4+IC0JZGVwZW5kcyBvbiBQQ0kgJiYgWDg2ICYmIFhFTg0KPj4gKwlkZXBl
bmRzIG9uIFBDSSAmJiBYRU4NCj4+ICAgCWRlcGVuZHMgb24gWEVOX0JBQ0tFTkQNCj4+ICAgCWRl
ZmF1bHQgbQ0KPj4gICAJaGVscA0KPj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL2V2ZW50cy9l
dmVudHNfYmFzZS5jIGIvZHJpdmVycy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMNCj4+IGluZGV4
IGE3ODcwNGFlMzYxOC4uMzU0OTNmZjBkMTQ2IDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy94ZW4v
ZXZlbnRzL2V2ZW50c19iYXNlLmMNCj4+ICsrKyBiL2RyaXZlcnMveGVuL2V2ZW50cy9ldmVudHNf
YmFzZS5jDQo+PiBAQCAtNjUsNiArNjUsNyBAQA0KPj4gICAjaW5jbHVkZSA8eGVuL2ludGVyZmFj
ZS92Y3B1Lmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4veGVuYnVzLmg+DQo+PiAgICNpbmNsdWRlIDxh
c20vaHdfaXJxLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICAgDQo+PiAgICNpbmNs
dWRlICJldmVudHNfaW50ZXJuYWwuaCINCj4+ICAgDQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94
ZW4vcGNpLmMgYi9kcml2ZXJzL3hlbi9wY2kuYw0KPj4gaW5kZXggMjI0ZGYwM2NlNDJlLi5mYzhj
MTI0OWQ0OWYgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi9wY2kuYw0KPj4gKysrIGIvZHJp
dmVycy94ZW4vcGNpLmMNCj4+IEBAIC0yNTQsMyArMjU0LDc4IEBAIHN0YXRpYyBpbnQgeGVuX21j
ZmdfbGF0ZSh2b2lkKQ0KPj4gICAJcmV0dXJuIDA7DQo+PiAgIH0NCj4+ICAgI2VuZGlmDQo+PiAr
DQo+PiArI2lmZGVmIENPTkZJR19YRU5fRE9NMA0KPj4gK3N0cnVjdCB4ZW5fZGV2aWNlX2RvbWFp
bl9vd25lciB7DQo+PiArCWRvbWlkX3QgZG9tYWluOw0KPj4gKwlzdHJ1Y3QgcGNpX2RldiAqZGV2
Ow0KPj4gKwlzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7DQo+PiArfTsNCj4+ICsNCj4+ICtzdGF0aWMg
REVGSU5FX1NQSU5MT0NLKGRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArc3RhdGljIHN0
cnVjdCBsaXN0X2hlYWQgZGV2X2RvbWFpbl9saXN0ID0gTElTVF9IRUFEX0lOSVQoZGV2X2RvbWFp
bl9saXN0KTsNCj4+ICsNCj4+ICtzdGF0aWMgc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVy
ICpmaW5kX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gK3sNCj4+ICsJc3RydWN0IHhl
bl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4+ICsNCj4+ICsJbGlzdF9mb3JfZWFjaF9l
bnRyeShvd25lciwgJmRldl9kb21haW5fbGlzdCwgbGlzdCkgew0KPj4gKwkJaWYgKG93bmVyLT5k
ZXYgPT0gZGV2KQ0KPj4gKwkJCXJldHVybiBvd25lcjsNCj4+ICsJfQ0KPj4gKwlyZXR1cm4gTlVM
TDsNCj4+ICt9DQo+PiArDQo+PiAraW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIoc3Ry
dWN0IHBjaV9kZXYgKmRldikNCj4+ICt7DQo+PiArCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9v
d25lciAqb3duZXI7DQo+PiArCWludCBkb21haW4gPSAtRU5PREVWOw0KPj4gKw0KPj4gKwlzcGlu
X2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArCW93bmVyID0gZmluZF9kZXZp
Y2UoZGV2KTsNCj4+ICsJaWYgKG93bmVyKQ0KPj4gKwkJZG9tYWluID0gb3duZXItPmRvbWFpbjsN
Cj4+ICsJc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArCXJldHVy
biBkb21haW47DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl9maW5kX2RldmljZV9k
b21haW5fb3duZXIpOw0KPj4gKw0KPj4gK2ludCB4ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9v
d25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2LCB1aW50MTZfdCBkb21haW4pDQo+PiArew0KPj4gKwlz
dHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPj4gKw0KPj4gKwlvd25lciA9
IGt6YWxsb2Moc2l6ZW9mKHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciksIEdGUF9LRVJO
RUwpOw0KPj4gKwlpZiAoIW93bmVyKQ0KPj4gKwkJcmV0dXJuIC1FTk9ERVY7DQo+PiArDQo+PiAr
CXNwaW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJaWYgKGZpbmRfZGV2
aWNlKGRldikpIHsNCj4+ICsJCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2sp
Ow0KPj4gKwkJa2ZyZWUob3duZXIpOw0KPj4gKwkJcmV0dXJuIC1FRVhJU1Q7DQo+PiArCX0NCj4+
ICsJb3duZXItPmRvbWFpbiA9IGRvbWFpbjsNCj4+ICsJb3duZXItPmRldiA9IGRldjsNCj4+ICsJ
bGlzdF9hZGRfdGFpbCgmb3duZXItPmxpc3QsICZkZXZfZG9tYWluX2xpc3QpOw0KPj4gKwlzcGlu
X3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJcmV0dXJuIDA7DQo+PiAr
fQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVy
KTsNCj4+ICsNCj4+ICtpbnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1
Y3QgcGNpX2RldiAqZGV2KQ0KPj4gK3sNCj4+ICsJc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293
bmVyICpvd25lcjsNCj4+ICsNCj4+ICsJc3Bpbl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3Bpbmxv
Y2spOw0KPj4gKwlvd25lciA9IGZpbmRfZGV2aWNlKGRldik7DQo+PiArCWlmICghb3duZXIpIHsN
Cj4+ICsJCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gKwkJcmV0
dXJuIC1FTk9ERVY7DQo+PiArCX0NCj4+ICsJbGlzdF9kZWwoJm93bmVyLT5saXN0KTsNCj4+ICsJ
c3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArCWtmcmVlKG93bmVy
KTsNCj4+ICsJcmV0dXJuIDA7DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl91bnJl
Z2lzdGVyX2RldmljZV9kb21haW5fb3duZXIpOw0KPj4gKyNlbmRpZg0KPj4gZGlmZiAtLWdpdCBh
L2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMgYi9kcml2ZXJzL3hlbi94ZW4tcGNp
YmFjay9wY2lfc3R1Yi5jDQo+PiBpbmRleCBmOGU0ZmFhOTZhZDYuLmJiYTUyNzYyMDUwNyAxMDA2
NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMNCj4+ICsrKyBi
L2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMNCj4+IEBAIC0xOSw3ICsxOSw4IEBA
DQo+PiAgICNpbmNsdWRlIDxsaW51eC9zY2hlZC5oPg0KPj4gICAjaW5jbHVkZSA8bGludXgvYXRv
bWljLmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4vZXZlbnRzLmg+DQo+PiAtI2luY2x1ZGUgPGFzbS94
ZW4vcGNpLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5jbHVkZSA8eGVuL3hl
bi5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL3hlbi9oeXBlcnZpc29yLmg+DQo+PiAgICNpbmNsdWRl
IDx4ZW4vaW50ZXJmYWNlL3BoeXNkZXYuaD4NCj4+ICAgI2luY2x1ZGUgInBjaWJhY2suaCINCj4+
IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYyBiL2RyaXZlcnMv
eGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+PiBpbmRleCBjMDljN2ViZDY5NjguLmRhMzRjZTg1
ZGM4OCAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+
PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gQEAgLTE0LDcgKzE0
LDcgQEANCj4+ICAgI2luY2x1ZGUgPGxpbnV4L3dvcmtxdWV1ZS5oPg0KPj4gICAjaW5jbHVkZSA8
eGVuL3hlbmJ1cy5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4gLSNpbmNsdWRl
IDxhc20veGVuL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiAgICNpbmNsdWRl
ICJwY2liYWNrLmgiDQo+PiAgIA0KPj4gICAjZGVmaW5lIElOVkFMSURfRVZUQ0hOX0lSUSAgKC0x
KQ0KPj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUveGVuL3BjaS5oIGIvaW5jbHVkZS94ZW4vcGNpLmgN
Cj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+PiBpbmRleCAwMDAwMDAwMDAwMDAuLjVjOTg4ZDVm
ZjM4Zg0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIvaW5jbHVkZS94ZW4vcGNpLmgNCj4+IEBA
IC0wLDAgKzEsMzQgQEANCj4+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAq
Lw0KPj4gKw0KPj4gKyNpZm5kZWYgX19YRU5fUENJX0hfXw0KPj4gKyNkZWZpbmUgX19YRU5fUENJ
X0hfXw0KPj4gKw0KPj4gKyNpZiBkZWZpbmVkKENPTkZJR19YRU5fRE9NMCkNCj4+ICtpbnQgX19p
bml0IHBjaV94ZW5faW5pdGlhbF9kb21haW4odm9pZCk7DQo+PiAraW50IHhlbl9maW5kX2Rldmlj
ZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAraW50IHhlbl9yZWdpc3Rl
cl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsIHVpbnQxNl90IGRvbWFp
bik7DQo+PiAraW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBj
aV9kZXYgKmRldik7DQo+PiArI2Vsc2UNCj4+ICtzdGF0aWMgaW5saW5lIGludCBfX2luaXQgcGNp
X3hlbl9pbml0aWFsX2RvbWFpbih2b2lkKQ0KPj4gK3sNCj4+ICsJcmV0dXJuIC0xOw0KPj4gK30N
Cj4+ICsNCj4+ICtzdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVy
KHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+PiArfQ0KPj4g
Kw0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVy
KHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiArCQkJCQkJICAgdWludDE2X3QgZG9tYWluKQ0KPj4g
K3sNCj4+ICsJcmV0dXJuIC0xOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW5saW5lIGludCB4
ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+
PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+PiArfQ0KPj4gKyNlbmRpZg0KPj4gKw0KPj4gKyNlbmRp
Zg0KPj4gLS0gDQo+PiAyLjI1LjENCj4+DQpbMV0gaHR0cHM6Ly93d3cubWFpbC1hcmNoaXZlLmNv
bS94ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmcvbXNnODQ0NTIuaHRtbA0KDQpbMl0gaHR0
cHM6Ly9wYXRjaHdvcmsua2VybmVsLm9yZy9wcm9qZWN0L3hlbi1kZXZlbC9wYXRjaC8yMDIwMTEw
OTEyNTAzMS4yNjQwOS0zLWFuZHIyMDAwQGdtYWlsLmNvbS8NCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 05:41:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 05:41:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190244.340078 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSC3O-00020K-Ct; Mon, 20 Sep 2021 05:41:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190244.340078; Mon, 20 Sep 2021 05:41:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSC3O-00020D-9u; Mon, 20 Sep 2021 05:41:22 +0000
Received: by outflank-mailman (input) for mailman id 190244;
 Mon, 20 Sep 2021 05:41:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSC3N-000203-G4; Mon, 20 Sep 2021 05:41:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSC3M-0007WK-P5; Mon, 20 Sep 2021 05:41:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSC3M-0006Gg-DN; Mon, 20 Sep 2021 05:41:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSC3M-0003n7-Bi; Mon, 20 Sep 2021 05:41:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gRTYo5C4QrJF6JLSXH/KEN9cM1rydDZkNlXubsEQAS4=; b=XY8CcrLlv7d13rd9mcGjijTHBy
	PPE+WtVK5XpZ6JW/f3Rk/4mTl34dTRfVcLk+phajmex+3mP7brH6EgwTKxopIhNwcxgJbP4spg3Ee
	z4ci1I5oQFIlZ+9dMt332w8ZNuk5Tj0Yp+aQS9sta0HFt2gh4hcDckiT9+El0oYoV/IA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165123-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165123: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-xsm:<job status>:broken:regression
    xen-unstable:build-arm64:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:<job status>:broken:regression
    xen-unstable:build-arm64-xsm:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:host-install(4):broken:regression
    xen-unstable:build-arm64:host-install(4):broken:regression
    xen-unstable:build-arm64-xsm:host-install(4):broken:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-xsm:host-install(5):broken:heisenbug
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:build-arm64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 05:41:20 +0000

flight 165123 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165123/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm    <job status>                 broken  in 165110
 build-arm64                     <job status>                 broken  in 165119
 build-arm64-pvops               <job status>                 broken  in 165119
 build-arm64-xsm                 <job status>                 broken  in 165119
 build-arm64-pvops          4 host-install(4) broken in 165119 REGR. vs. 164945
 build-arm64                4 host-install(4) broken in 165119 REGR. vs. 164945
 build-arm64-xsm            4 host-install(4) broken in 165119 REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail in 165110 REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-xsm 5 host-install(5) broken in 165110 pass in 165123
 test-amd64-amd64-examine    4 memdisk-try-append fail in 165110 pass in 165123
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail in 165110 pass in 165123
 test-arm64-arm64-libvirt-raw 13 guest-start                fail pass in 165110
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165119

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt           1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-examine      1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl           1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)           blocked in 165119 n/a
 test-amd64-amd64-xl-rtds  20 guest-localmigrate/x10 fail in 165110 like 164945
 test-arm64-arm64-libvirt-raw 14 migrate-support-check fail in 165110 never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check fail in 165110 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    9 days
Failing since        164951  2021-09-12 00:14:36 Z    8 days   16 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    2 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-arm64-arm64-libvirt-xsm broken
broken-job build-arm64 broken
broken-job build-arm64-pvops broken
broken-job build-arm64-xsm broken

Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 07:17:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 07:17:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190265.340092 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSDYb-00034b-A7; Mon, 20 Sep 2021 07:17:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190265.340092; Mon, 20 Sep 2021 07:17:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSDYb-00034U-75; Mon, 20 Sep 2021 07:17:41 +0000
Received: by outflank-mailman (input) for mailman id 190265;
 Mon, 20 Sep 2021 07:17:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSDYZ-00034O-Cc
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 07:17:39 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d52d99a6-19e2-11ec-b85f-12813bfff9fa;
 Mon, 20 Sep 2021 07:17:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d52d99a6-19e2-11ec-b85f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632122257;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=QtqKnSxyfqDkCSVslXETfVc1jFd0cYlLbRSaQcaNm3k=;
  b=iJzXLa/SPm6Npc/2720tui8Fq2ZQAxziGSmcvy5sX+ps/3jXPXKWKjYN
   iZ6c9qQpRI6JwteYOoZp4/mEWbmwOCnY7iuCUMF2h7EQrRO8/ibyjxloj
   vmEy5TNCGUL/gAMfgSmzf3M0YE6zYAPxVw8jAuTC0br7nQn4ag7DlIbzq
   U=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: oPNN69iHYSAK2c8QO6B0HPuKrKvLnpdp3HKkwJhvrUiIRh2srOUSdSUolLlcKotmg1YFwmjNUF
 IobPR59vae2KT64RDMbObSa3IGL9Jw0ji4UlS6LLY+mta3nLeTiD7hPSVqsoJcTl/mvdlhBkLh
 2zvP6+8/Ic6Zj0G5WmRADSgrnywCWGtAjVWdxvH8Xu4nOAv06DJghAnEp5ucmJByshc9qjDM1L
 uYpdf44F9d2DsfchPBV1MkxYYYH7Ea46Wf8Rq1zaO9venMWPsk99NXnistMr2Qy6ioNvC8Dexg
 E8BoK2K9gznXyWHCjCyierj2
X-SBRS: 5.1
X-MesageID: 53510126
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:EMdIwKvn5nmCx0l2oxb+pFsBSOfnVIJZMUV32f8akzHdYApBsoF/q
 tZmKW7QOfyPYGCnLdgia4uw8RlX6MXVyt5jGQRk+StmFXsS+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524DhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpluYKREQYVAIb3vM8QDCtZAmJQY7cb5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sTRK6HP
 ZRxhTxHNyTGRgYUa3QtBcw1sMG0viTBehlEkQfAzUYwyzeKl1EguFT3C/LXdcaNXt59hVuDq
 yTN+GGRKhMHMN2SzxKV/3TqgfXA9QvkXKoCGbv+8eRl6HWDy2pWBBAIWF+TpfiillX4S99ZM
 1YT+Cclse417kPDZtjiXTWorXicpBkeVtFMVeog52mwJrH8uljDQDJeF3gYNYJg5JReqSEWO
 kGhk83IXT1TjpaubVmQqYWVvHCcZzJIMjpXDcMbdjfp8+UPsalq0EmWHo0yQfbv5jHmMWqvm
 GHR9UDSk51W1JRShvvhpTgrlhrx/sChc+Ij2unAsotJBCtCbYi5bsSD7VHB5J6sx67IEwHc4
 BDodyWYhd3i7K1hdgTWG43h/5nzvp5p1QEwZnY1RPEcG8yFoSLLQGypyGgWyL1V3iM4ldjBO
 xW7hO+szMULYCvCgVFfOtrsYyjV8UQQPYu8Da2FBja/SrNwaBWG7ElTib24hju2+HXAZZoXY
 M/BGe71VC5yIf0+kFKeGrdMuZd2l3tW7T6CGvjGI+GPjOP2iIi9EuxebjNjr4kRscu5neki2
 4wOb5fVk0oPDrKWj+u+2dd7EG3m5EMTXPjeg8dWavSCMkxhHmQgAOXW2rQvZ8pumKE9qwsC1
 ijVtpZwxAWtiHvZBx+Nb3w/OrrjUYwm9SAwPDA2PEbu0H8mON794KAafpoxXL8m6O08kqIkE
 6hbI52NUqZVVzDK2zUBdp2h/oZsQwum2FCVNC2/bTlhI5M5H17V+sXpdxfE/TUVCnblrtM3p
 rCtj1uJQZcKSwl4ItzRbfajkwG4sXQHwbogVErUONhDPk7r9dEyeSD2i/Y2JeAKKAnCmWTGh
 1rHX09AqLCU8YEv8dTPiaSVlKuTErNzThhAAm3WzbeqLi2GrGCt9pBNDbSTdjfHWWKqpKj7P
 bdJz+vxOeEslUpRt9YuCK5iyK8z6oe9p7JeyQg4TnzHY07yV+FlK3iCm8JOqrdM1vlSvg7vA
 hCD/dxTOLOoPsL5EQFOeFp5P7rbjfxEyCPP6fkVIVnh4H4l9bWKZkxeIh2QhXEPN7ByKo4kn
 b8stcN+B9ZTUfb23gJqVhxpylk=
IronPort-HdrOrdr: A9a23:PN1HBavUoR+r5FEcA9hFz3nj7skC7IMji2hC6mlwRA09TyXGra
 +TdaUguSMc1gx9ZJhBo7G90KnpewK6yXdQ2/hqAV7EZniahILIFvAY0WKG+VPd8kLFh4xgPM
 tbAs1D4ZjLfCRHZKXBkXiF+rQbsaC6GcmT7I+0pRcdLnAbV0gj1XYANu/yKDwJeOAsP+teKH
 Pz3Lsim9L2Ek5nEfhTS0N1FdTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqpmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87isIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXsHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa13ackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmP29yV0qp/VWH/ebcHEjaRny9Mw0/U42uondrdUlCvgslLJd1pAZFyHo/I6M0kd
 gsfJ4Y042mdfVmH56VMt1xNvdfOla9Mi4kD1jiVGgPNJt3c04l+KSHq4nc2omRCeg1Jd0J6d
 L8bG8=
X-IronPort-AV: E=Sophos;i="5.85,307,1624334400"; 
   d="scan'208";a="53510126"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kqd3DSpvHa1LkMVVAEp4kW8L2haJk1rgi58pdr8W4FOA/c4ksaYVMlCIepb9Gt57V/1D/etJRgQIgABXTABsxeqAcT7D5YpZk7j75b1A0f2EMhztKIxm3u9P7BpMNzdmG7U0+40sCxwSnSnGGDvbO3IYqNZa6op9LxSDYHYEklPARfr9xrJdiVgOrUXwKuQr2M+S7zAQTqS0MWwsPda/gQfM4pdpjSf+UNiIpQhSLj1FwBLn1t0ZQoq3/rheW27xL6kpOY57v/DoV3dH6Hflq5xB7VU31xMCuQdQJ0I+26dp3gyPf3MXYf3sMrJNsLr8V8yT0yFyimNlp6XNPRNMzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tqyd/H6DW3LiP/1acj21isINZYh0GarHjLSg8u38J38=;
 b=Q137pt7mtrgaYgJeuZY4nWZi88ljXDHq9hghSZRb5hJuJQtQK0h2X/sMcGVS06oB6lszskOUMI4bg0j5relDdZuE9A54JlGB9QashoBBQ/dtam0LVOnAyqa/SOd/Ir9a/kFXbOokgRQNdkra84GZVFNVKl1b1Bn2pqgyDOHlnqaZ0xcJ+0jgl9SuezTYo2RTgrWxWH3aQgQ36LxAhj1E9hlYexDMkoOvVCsbmxc6izIke0KKUzeD9UmwffZhMfaP5QL8TB1tjQqacJ1gr37QGRf4V9WY9JRQdZP0IkRXSh6x4lebQ/COdW0OPLE0z16QfewsgPv1PKRU3YIvt6Zg2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tqyd/H6DW3LiP/1acj21isINZYh0GarHjLSg8u38J38=;
 b=v0aGO32z72/ng5Fe+QSQkaRYIP4SApTZtGqVKV5JXGZXyM2tDfjCtSyNi+JLBGf4pNL6ui7570ViIzGBQ+8nkPMqpPJWDSFUDDprLn2zTBygeyTWu1A9iEUm7B/6HPVZt7GjbsVSka8KHrczYvBIfx9e8mEXs8NAoUb/eHbCdCA=
Date: Mon, 20 Sep 2021 09:17:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Juergen Gross
	<jgross@suse.com>, Julien Grall <julien@xen.org>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
Message-ID: <YUg1idb2DBtWSDTE@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210917154625.89315-5-roger.pau@citrix.com>
X-ClientProxiedBy: LO2P265CA0281.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a1::29) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d200175-e413-4a6b-6613-08d97c06b76f
X-MS-TrafficTypeDiagnostic: DM4PR03MB5965:
X-Microsoft-Antispam-PRVS: <DM4PR03MB59656B97D8112950244A461F8FA09@DM4PR03MB5965.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 8dKf2txODrx5LIJT/eMNoe8wXfYG0aLZ5kFr5pCTHFlpBtRUP3WxpH9pf2/KGbVUbCHhIhYI6TYTDzqBhNxliny4vdJQAKg5Nyh4J28vxxE0M5HuCfEMMRxuiuYxNNsF9tJqoqKhNT4kpMrfg4iKq3r7/S4bqSK4r1VmiL/yBEEBVSP96+kXy2nK9/7hhFO3eAf2fi5nMpFOkDupCmiwyGcI/9STb2Mme2ymiDaQoFTEFGBvsbNor9s0DDGKb3NWR+NIy6s5bqfNGQDduCpKili63Kp8R3h2Pbjb3dTxwbPl3ESV5y4IBuhG9774mHf9mkj9adXo7arvmgBm9dKkVgn0+cQSYhbyt5hRf3WWgYpQLPX/9FeIlchiSjG725qzkXg8THtEpccDql4CTWnA3jI62r6JKrjp17/3dQjn0aHsGG2jZHhx7+Kngn24CYhadjfzB6AWNKzVOA/19dpNotemMXFGWpxiu8PQ79OYuwL4Fgi8HplKJZFLN8hz+8xdJdqfMAp+8RiRnTgEoGbpDx81ioubYDrBbANMvF2Q1/gl2t+bvs45Ttx9ZCpsib2hKlgSwsQyFq+vr+q36WKgrsV6YMkQbthifucLmDOgYfmi9/7DRWeiKSTCrr/EJfC4qkTIfyLQ65hvMkHyusJTEA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(4326008)(6916009)(316002)(66556008)(54906003)(508600001)(2906002)(6666004)(86362001)(85182001)(83380400001)(9686003)(6496006)(38100700002)(956004)(5660300002)(8936002)(26005)(66946007)(186003)(8676002)(66476007);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SytEVnpwdFQyRy9lTGIydkIxZXVpNUdQWXRqTjJUNllPNGJuT2VpaE9SaW9F?=
 =?utf-8?B?bVpFRGlWcXlwQlZpZUFPVGtpZGZGeHN0dU8ybWF1UUxhZ0dVSENIREZENWYy?=
 =?utf-8?B?QzBFb1JwbldUcG1vMFBTSXdHQnJVTGNsbDk1amd2Mk10TmQ4bkdFbDdmeDB3?=
 =?utf-8?B?SUZGeGtJdHRkK3M2bU1jdkRaUnV4V1J3QVkvelRtZDltSXhGMjFFdmJTSHJy?=
 =?utf-8?B?SEZpb2c2b3pWM0pxb01uS3JOZ2V6L28waGdJQkJyVXFMbjBJbnZJRjltVE5u?=
 =?utf-8?B?Z0Q0dnZjVWpTc0loTHp3cGhoZyt3blFUR1dLYWRmZ0ZzazlTb2JnVVZvZFpy?=
 =?utf-8?B?NkNrYVlIei93cDduWmpPVjM3S0xxVjJDYk40Q1lQaEpsb1dsYjJUN1FqT2VQ?=
 =?utf-8?B?K21QMmxrQjJYL1JoZTExeWJLWk1BbmU3bzFZWXVWSkhxc3RyTEh2UWxvL3Av?=
 =?utf-8?B?SmJlNk5JQUM5a2ErYkFxMlE5RSt4ZnhOWXBHbUhTUUF6K3BCdmVnVTY0WHB6?=
 =?utf-8?B?UURWQ09NZEVaaVZhMEs5NThjWkR2ZEtzY3BVK3FtL3c4OCtZRjJnaUdiaVEy?=
 =?utf-8?B?QVJoOFQxUHF1ZFR2RTE2S2VhSmQ2ak8yenh4Mi9hamZzNkttL0JhUUs4aVlx?=
 =?utf-8?B?NXBZZmlSRXMxdHJrdzhQSVUyV0FIaEtjNmJORTBLdHRsTDdiT1ZmNzlMRytN?=
 =?utf-8?B?MzU5dWt4QjVaNGdWQXRjc2l3M0ViZlp5dWJyRlBJc3c4R0p2MjY3UkNaZ0lj?=
 =?utf-8?B?YmpBZWRXck5URXlxRlZMb2gyZXZKdjJxQldpODlUZHFpekt5RXgrYjhZZ3gz?=
 =?utf-8?B?cGtyQjM1MytVSkh0WTFLTWJoTXN0a3JrYStHUUVja1pVU1NTR25LTlZmSjNz?=
 =?utf-8?B?LzBobHh5Y042RjhNNHJlcFdTWUlzZ3ppTTN0eUhTaURiRzdqK3ZLT2lta0Z6?=
 =?utf-8?B?TGNsQlJGc2tWa01oOFFLN1d6cG5uUXhYMzhVL0JCUjM5UjRlMi8yZXpOeEh5?=
 =?utf-8?B?UkhhMEZSVGp1dzhnMVhQdE1ib2k5dVU4Rlk3U1JXVHAyT3RXdkdaOGkzR1BD?=
 =?utf-8?B?dzYvY0Z3bGZqUVRhM0pOOERIaTNYYzNnVTF4K29mYnQ4SitRWm1CUXg3UEdF?=
 =?utf-8?B?bW9qZ1l1QnQxd2MxTzVzeDhaeXdSbVd2TURDczRhYkVxWGRERUs5YUlibndS?=
 =?utf-8?B?ZnNWZVB5dVZDQXhBOEw3YjljdXgwNjRQdHR5MEtQZE1sUlZEUDlHQzdWVnFH?=
 =?utf-8?B?Nm5UL2tPOWZQMEZJaythNHB5MnN6MFpEUlg0SDhEN0dDeFNtMERJYUlUWDlU?=
 =?utf-8?B?UG5UdEY0VmszV241Rm44YjlNUzBVQzJRVTc4VUdCK1hTYjNiUHFzSlhwdWRS?=
 =?utf-8?B?RjRsTmhGRnI3dHlEWE80NlZEMmFkek9CQ1hxQ0tJSXhPTkh4R0I5U2hlbGdi?=
 =?utf-8?B?NFdaT1l5OFN5R21oY0Y2MlVwMC93YmxLLzRKZ3R3azk0eEhDWEhxT1UzNTJD?=
 =?utf-8?B?ZmhvU3UxSkh6NFBpYmdSSTRSUlFWN3FOMmZpRXIydFBmT0NnM3lKMWlhbDhq?=
 =?utf-8?B?MEFBVDZwY1hiWnNodFpmbnNoN1BEUEZDdk54VEFEaExVemdxRzVobFdHS2py?=
 =?utf-8?B?aXdhSmhRQXhRMGNlS01PNWZmdUlwTjYwUldZVXIrNnFXWWZPTG83SG5QazVy?=
 =?utf-8?B?ZS9jWjdiSTBoSEl1UkRmdW5uN052L1BsR1JRMVlvaWNMNUxCYVRCSDRGK3dY?=
 =?utf-8?Q?k8eTrxHjnPnsLkwWfECr21Vh4oNJgkKUAppGiHs?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d200175-e413-4a6b-6613-08d97c06b76f
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 07:17:34.1254
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KEIf1aOCsTmU/e41gsk/l7+FjXdm8h5Mdyka3aja8ELdpkQzXynCfTk49Hss5Pk9NsmUr0EEa37h1qLvrB848Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB5965
X-OriginatorOrg: citrix.com

On Fri, Sep 17, 2021 at 05:46:23PM +0200, Roger Pau Monne wrote:
> Exploiting the talloc clean up routines to close the Xen interfaces
> is cumbersome, specially when atexit can be used to the same effect.
> 
> Convert xc and gnttab to use atexit which allows to drop one
> indirection from the storing variables.
> 
> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

This patch is missing the following chunk:

diff --git a/tools/xenstore/xenstored_minios.c b/tools/xenstore/xenstored_minios.c
index c94493e52a..9b050c7e02 100644
--- a/tools/xenstore/xenstored_minios.c
+++ b/tools/xenstore/xenstored_minios.c
@@ -49,12 +49,12 @@ evtchn_port_t xenbus_evtchn(void)
 
 void *xenbus_map(void)
 {
-	return xengnttab_map_grant_ref(*xgt_handle, xenbus_master_domid(),
+	return xengnttab_map_grant_ref(xgt_handle, xenbus_master_domid(),
 			GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE);
 }
 
 void unmap_xenbus(void *interface)
 {
-	xengnttab_unmap(*xgt_handle, interface, 1);
+	xengnttab_unmap(xgt_handle, interface, 1);
 }
 


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 07:26:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 07:26:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190273.340103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSDhN-0004ZA-BZ; Mon, 20 Sep 2021 07:26:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190273.340103; Mon, 20 Sep 2021 07:26:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSDhN-0004Z3-82; Mon, 20 Sep 2021 07:26:45 +0000
Received: by outflank-mailman (input) for mailman id 190273;
 Mon, 20 Sep 2021 07:26:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSDhM-0004Yx-Mr
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 07:26:44 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1a98369e-19e4-11ec-b860-12813bfff9fa;
 Mon, 20 Sep 2021 07:26:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a98369e-19e4-11ec-b860-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632122803;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=CWnSXaAMaEaSR9p3TCiV9WJmxGN1l/ggPdqewB+1sGg=;
  b=JCDMAoHQCa2PDbQ2MhcM3YI5yqCmiag5W1czqF3EKrv8a3+VKzxOtrp4
   jUL2UUjsbSV0TWDpb2KMKRviKb6WfiklrPXDZeP6C5Q6Dt9Ccr54CuRiS
   a6kKjpEup2wsXtgQqctB2pLiWqDk//weA4/8IKE8H45PHpu4o+FW7JZay
   A=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: GXjahnF3T9WBtzeSCrQx+Diqc2y3GNYGyU1AZyeAawL8urkUqqaIiB45Us0Y9VmRaXJY6vDXOX
 MCXOSGtK5tcqSsbQvZ6QJCu+EKnvnQ4jI6PoOP8n2Ua3PYvEv/jFKQseE20xvw/6TLglOM5qpv
 VMBmEBSH03knXMqTgIHphXZC0DLkHDMOzwSoR8YMdGYEUhhfsn4B02vZsQPelMIhLeowLHJl+D
 oegD/x0+hcAn9u6qNyGWcc5c9RTOseHKc875cud1K18Vtdsq2tSU7OEDtTjo1XKItF0I8/KL/T
 ais=
X-SBRS: 5.1
X-MesageID: 53510795
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:YzHc+atrfsdB2zN5H/bES/ezcOfnVLZeMUV32f8akzHdYApBsoF/q
 tZmKTzVPvuLamemL9Fzb9u2phkFscDVzt9iHAJppS5nQn5E+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524DhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NplpLqWQDh2AY32ktsAUSFhFCZhD59Y5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sTRK6BN
 pdxhTxHMT7gY0VKJX4uNJcZkOmUtFTnThEfgQfAzUYwyzeKl1EguFT3C/LXdcaNXt59hVuDq
 yTN+GGRKgoBKNWVxD6B83StruzChyX2XMQVDrLQ3sBthFqf12kCEio8XFGwoeS6okOmUtcZI
 EsRkgIipLY/3FamRd78W1u/unHslhwBX9tdFcUq5QfLzbDbiy6YGWosXjNHcMYhtsI9WXotz
 FDht8PkA3ljvaOYTVqZ96yItnWiNC4NN2gAaCQYCwwf7LHLoos+kxbORdZLC7Oug5v+HjSY6
 y+OhDgzgfMUl8Fj6kmg1QmZ2XT2/MGPF1NroFWMNo640u9nTJ+DQYWXtmf71/x/MYShd16Go
 0o2hdfLuYjiEqqxeDyxrPQlRe/yvqbeYWyE2zaDDLF6qG/8oCfLkZR4pWgkfRY3aJ5slSrBP
 ReL0T698qO/K5dDgUVfWIu3F81i5qzpD92Nuhv8P4cWP8QZmONq+khTia+sM4LFyxNEfUIXY
 87znSOQ4ZEyU/8P8dZOb71BuYLHPwhnrY8peXwe8/hA+eHEDEN5tJ9faAfeBgzHxPrc/W05D
 Oqzx+PVkk4CAYUSkwH88JIJLEBiEJTILcmt8KRqmhq4ClM+QgkJUqaJqZt4ItANt/kFx4/go
 yDmMmcFmQWXuJEyAVjTApyVQOi0BsgXQLNSFXFEAGtELFB5MN7zs/9AJ8VoFVTlncQ6pcNJo
 zA+U5zoKtxESyjd+iRbap/4rYd4cw+six7INC2gCAXTtbY5L+Bk0o+8IGMDKAFm4vKLiPYD
IronPort-HdrOrdr: A9a23:CDEmp6ECMhyCzZ1OpLqE7MeALOsnbusQ8zAXPidKOHtom62j5q
 STdZEgviMc5wx8ZJhNo7+90cq7IU80l6Qa3WB5B97LNmTbUQCTTb1K3M/PxCDhBj271sM179
 YET0GmMqySMbGtt7eZ3DWF
X-IronPort-AV: E=Sophos;i="5.85,307,1624334400"; 
   d="scan'208";a="53510795"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j5EyWTuEqIVtN/xtO3eknKXSVUQcTQQk0yku0qp2Bzl57HbYhSrL1yqV+lFddeBeUYVhDSit576YAjHPDH4/vktmTf8Di++/UXeXX0nq700SaZA94wSKVYmRGx1nK59pRmzFaFe+4hc0qz8pkT1wAhWQ9fsN0qeHERApNDL0ZSWyh0IPEadDHf8IEhLHMBFAXdgDzJbllIZNv6/GTVDc6SKQrm70zghUq0KI9A63ldPxgeso6d97C02ZPVCbTa7AgxP9+raaa7Zop/+cFXUBXM4RWDHJPz0o240EZ4eHzLekPZnlzVOKhAtW3wU4hSnm7byjBOb+SJjkd/a1xyu1iA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=h++yva7142k2qdDypKToq7jAJmeMCxs7Et6brqngGSo=;
 b=e/I9Xhy5Yxyk2GDs+I4HGR291YQv3mCCCDkgnNBBr0UPBlLH32DikGtSRKXgiVRVZbB7W6IipnGkBi48Xy8bW67aKUAbT3go+Pp+pb2+6hZoCx+/b2tEviNSkMmsOhB0B43Z0L++9/6jR65A5ryz2PRNv2Qex4ElREIJCq5lcGQvHoL0LxwR20l1zxQfFj4y2DPAr3fPyX5iQb5eC6gx6JKAvsu2gEQOm9oKttokPsECxCMU5+rAjMRWF92GopO5Jz0+GXgc4rX+P/Ukzm+FL1ZbUlOAgU6T9zHIVz9452vJYI3SieE59a47AqgTeizCZfvrmcjAoafihpMOk2CSLw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=h++yva7142k2qdDypKToq7jAJmeMCxs7Et6brqngGSo=;
 b=rV/gNHT/VQjo9k8iB+qSQ1Fz9h+zi58e5cMUzdLw/JFi3AWdQv8iGQdfu4YOQXxLWRjbUnj44enfumxQXety9o5Lo6CMpdcLT/BOhyrfgfqvaI+Z9cOml/mrSLJEMmbQqsPqNg3YFWcfRFvAVK89khpIMlxdf1EGCq7xrGZDFww=
Date: Mon, 20 Sep 2021 09:26:29 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Torok <edvin.torok@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Andrew Cooper <Andrew.Cooper3@citrix.com>, George Dunlap
	<George.Dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony Perard
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 0/6] gnttab: add per-domain controls
Message-ID: <YUg3pc9A5C4GTTeR@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <0D7E40CC-EA8F-4CE6-B4C7-A69C403BD4DF@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <0D7E40CC-EA8F-4CE6-B4C7-A69C403BD4DF@citrix.com>
X-ClientProxiedBy: LO2P265CA0036.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:61::24) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b71939ff-3c38-4923-8557-08d97c07f9b3
X-MS-TrafficTypeDiagnostic: DM5PR03MB2924:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2924CEA8E373E80A17DD1ED98FA09@DM5PR03MB2924.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PHPB4YIbWe+5zWOsuODf+WHNEScLvgf7+4S1osmte4MZ17lQKF6Q1bJ+mNyXd/7if608qMgHM9Vz6BmN8EwGPmF6W1sBd61sLleZHUptx6eFLPthvcw1Wey8iEiETzn7uPyaJSejXZGiYcs8Zqus8qAP+7tgC1Wvd8aJqRN8WCXI5VokSc+B93U6bmSEX9Og0S1u+oY5NYQ0PjrBpUnHoTVz0wlchwpc27cRkyn05E08qh8ziWW0vPaO6volqUQe2Vr/8sutlXi7u9GV+UzVnaTALXG0CXWxaZhqGM14DZQM3tJoLYws0fgJxf5ERGW9NwvwuGcXuIxcSduky4yXBQZnHSIu3yK5jb8xc96EDMsWieTmX7quUibX4bwlgYds6ogqWxIa8dcT6fgKql9fChrFSIpgDSL432bVDayR6hYU0K6RURcO6ytyJcMdEiolxbwZjYHY1ye1h33RXqE742vzxO4El64FOPrZrM0bbNA1GQEFf8CGAw/YFB2VaXJa+LBpeszEUseMu64PrOkVegcNP38QPzRk0BmoM/cxYiSuaYXIcvgN4DaXDKfP8UORI1V+1aDgprKQJboP9ayhlEReuTyLwi/9Sgqjfx7MFWOmkC7yCtO0CFP4ynbwQZ/BkP64YmKFTMsreC8Ee8fhSQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(83380400001)(85182001)(53546011)(54906003)(508600001)(66946007)(9686003)(86362001)(8936002)(6862004)(66476007)(66556008)(6486002)(6636002)(2906002)(956004)(186003)(5660300002)(38100700002)(6496006)(8676002)(26005)(6666004)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cm45VkpKVUh6N2dnZXNPUmo1VWhiNXgwZnJHVldaN0JCRnkvaVlZM1p5Y2Ns?=
 =?utf-8?B?bW93MW9KWGRaUmZJS1Z3SHVCSm8yU2IvWW42QWdrS2E3VW9FcDhUVHVxRU93?=
 =?utf-8?B?Qnk0dFQxbGlEOFpLRS9QYWtTZDVNNUZaTjlQMU5JR1EwYnB1Zk9JZEwxZEZB?=
 =?utf-8?B?MDVvUTQrZFd0aEV3T1hySnd2aHVabmV0MFRWa05mZndUSWhqZGFHKzdyYW4y?=
 =?utf-8?B?b3RFWms3WlQrMXMxMG1vMUZOYStYNy90LzJJelllSEZSdnZQRCtnZG14VVd2?=
 =?utf-8?B?aS8wOWkxNzlXam1xQndSNjJiQWo2L1ZGaVZCMjZaQVQ4bGdJNDQvaDBoMXFl?=
 =?utf-8?B?WDRMVmFZcFFsZlJBNnVQQVpiK3ZyZEJ6alJaVmxHeG9IY2ppVHZiK1UxYVFG?=
 =?utf-8?B?MXRFV2ZxTUJUVGNraWlMckwvbW9kYUMyZzEyeVlYUzVyVjh1OWxWYTFreHNa?=
 =?utf-8?B?UFRHVEtzQWJMUCtnaThSWEVtcjFHMnZDMEhPdzdRTm52VmVWWml4ZUZsZk5o?=
 =?utf-8?B?T0dFc28yY21icm40VlJmVmZlR01LSVhYVHhuZm5zUnhHR0dHQVNNWWJzb1px?=
 =?utf-8?B?M2V5SC9xWC94Q21CODNJT0ljOUdteXIvY0FFa284M0tXMnJMS1oxdEx6SmtW?=
 =?utf-8?B?eWc0bzAxaVpEeFlaUmUxVDFnVzdqTjdlekQ4MnZNVzNPdjlrU3lxc1FxWVVZ?=
 =?utf-8?B?Y2FVTFIzKzV0NGY2S0wxcVJSSXlrTDcyS3RjMStwYmpoR0V5WjNsQXZYbytU?=
 =?utf-8?B?RnRvWFJYVThIdmYzUU84eVpsZmVyYUpKOUpkeWxOa0taTmptcSt5eWdKWkF3?=
 =?utf-8?B?NmNKcWVUZVkwV1NGU0orcTdEdWJTMDFyMHJibU1KUXdka2V4S1RraUIwdERs?=
 =?utf-8?B?MjFTL3lNTk1wTm5WTmFwRW9HS0c5M2ZHT05kcDNTcU1ZcnB2UUJHZVFuRTFp?=
 =?utf-8?B?SGF2VTZhRVRENFNqc3REK09rSmF6US9WU2JPMlBmOVJyYmRsbTU0dEZoMmgw?=
 =?utf-8?B?N1VwOXZEelZ0QU9KcDJaYVo4UFFZazFCSGdNd3BySWt5ejBJUnpwVkUrcEtx?=
 =?utf-8?B?V2hMWjFFSHMrRjZJL3R1UFpCL2k2T2s4RUtlbC9lNTNkT3ZsUWNNcFIrcDg0?=
 =?utf-8?B?eDN2OUF3ZmpjcXRqS3V2c1pjQXROSDBhUVdxNWUyT0NmZVZuaUNsVFZCcmtT?=
 =?utf-8?B?cTBpcW1JVUkzQzdvTVV1YWxDbzdmWkY3V0hySjVtK2NZc3NZbUxRbDROWXQw?=
 =?utf-8?B?TmNReGpzVklrY3ozNjFzQzc0YWU5K2F4bC8wNTEwU21lTmg3UXFKTlRqVVhj?=
 =?utf-8?B?UUxLKzZrY0h6MUMwUklsWUF5UUFjT1FhdXBBVithbnlWbDAzbisvWkQ0Q284?=
 =?utf-8?B?dkN0T0NOQ3FRdHBzYTV4eExSRFJmT0NJN2ZlOGtiTUROM2RMdEE5K2RNLytI?=
 =?utf-8?B?UXJHYWJrOVJRejZERUFTR3d3UUZLN2trMHUrZS9YaHR3RHl6ZituWVJlMUx2?=
 =?utf-8?B?QmxYTkhqUk1tczk0aWxDQ2d3NnJMLzdtWW0vSFE2UFgvbHdMNjhUYzIwQ21I?=
 =?utf-8?B?SHU1eGJ1TmxTaUE4OEk1VEhIMGJ5SU43emdleEhzdDhtYjYybjVHUzZsRm9x?=
 =?utf-8?B?Ukl6eGluRUJrQWE1UmhqR2tWOC9Rb1JGZFNrTEtRcnZYMDBNaXhzYzhWUWwv?=
 =?utf-8?B?Tm01aEU5YjE2QjdmRzNLWXdUZ2dlb0ZydTFGM1FIWDBxajdZb20zam5oU3Ro?=
 =?utf-8?Q?DMmx0JgGkldU9p1X9OBN5O3FnudA3HVsbdTM59i?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b71939ff-3c38-4923-8557-08d97c07f9b3
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 07:26:34.7564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NFkWFQ/IlJyS4YuPtJdgiHZig200IB9sWE9o5EFIXBEMG5VWMfm59WV2Z5TaFEBNI43gYazvrvQqRQunVWCY/A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2924
X-OriginatorOrg: citrix.com

On Fri, Sep 17, 2021 at 06:06:42PM +0200, Christian Lindig wrote:
> 
> 
> > On 17 Sep 2021, at 16:46, Roger Pau Monne <roger.pau@citrix.com> wrote:
> > 
> > Hello,
> > 
> > The first two patches of this series allows setting the preisoutly host
> > wide command line `gnttab` option on a per domain basis. That means
> > selecting the max allowed grant table version and whether transitive
> > grants are allowed.
> > 
> > The last 4 patches attempt to implement support for creating guests
> > without grant table support at all. This requires some changes to
> > xenstore in order to map shared ring using foreign memory instead of
> > grant table.
> > 
> > Note that patch 5 will break the save format for xenstore records, and
> > should not be applied.
> 
> Has this relevance for the format used by oxenstored?

I'm no expert on oxenstored, but I think it has always mapped the
shared ring as foreign memory, and hence no changes are needed there.
AFAICT it also stores the mfn on the save format, so I think this is
all fine.

Should have mentioned it on the cover letter.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:01:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:01:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190289.340115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEEf-00011H-91; Mon, 20 Sep 2021 08:01:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190289.340115; Mon, 20 Sep 2021 08:01:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEEf-00011A-50; Mon, 20 Sep 2021 08:01:09 +0000
Received: by outflank-mailman (input) for mailman id 190289;
 Mon, 20 Sep 2021 08:01:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSEEe-000114-2k
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:01:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e7a7cfc6-19e8-11ec-b863-12813bfff9fa;
 Mon, 20 Sep 2021 08:01:05 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-qOJb3w6APdipx3UhY0As-w-1; Mon, 20 Sep 2021 10:01:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 08:01:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 08:01:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0125.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::30) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Mon, 20 Sep 2021 08:01:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7a7cfc6-19e8-11ec-b863-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632124865;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=dyevKjBK76O5ncYs/3yqDWIl6YXimggBjpRUnzmX/UU=;
	b=DuP3mNmqBz4xrcOlPFu4rznScW4BLeGH4TCUdW+IVGglzugxIzDtZeDzamR9Ota0ogcpud
	LQEJUHeUWCVF3NhNhid27sCPEXB4mNCuV1gzNU6tgpIs4VL1M7BQFhHFtUCX1oO3ywbpUk
	E3hS+6G3IrYiLSctOhDRoIUA45iA4WE=
X-MC-Unique: qOJb3w6APdipx3UhY0As-w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UC7rOPcUN5aCkojKf6l1VAWgaGb71zMZ6pu6RDBisMY6+RhF7i4OMFf9cvB9HqlPbyxWLxhFRgmI91r3teTd8vFK7vB+g2kjEVWJQ3+Vwb/He3D1x06+gd9e8rdl6nhtTwsoaA1dsTqgLMZhOqvNrWp4eAwOLBIxh9MADrDL877A4BCXEKkpYvVlri+Cp8g3Vsp93xxG5hjJrSfzDNwLPwwdFFyPIFGiv28JNhgCazUx7h+Nov44FFWSZxEZ97fSz2X0K3h5EvTFQmfS3+bqawXr2xMufBOE37vTivhizpnqo3oxc1UG+zZT4Ybl/bW3Syw8lyPHTrpgtJxF5MXkVg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dyevKjBK76O5ncYs/3yqDWIl6YXimggBjpRUnzmX/UU=;
 b=hitddAUXiJRCzk8iyt9NjIUPoqk8//6TQMn8TzKRDg9ZTHHs1Mz39usVpjjt9BFG7Ehm4FOa7+4bWQsCs6IQi8HKBLj+tmbAwv0eUpYrK3cl0H3yqiC2Ke8o5scUOyg+igl3dqJOqKhxQeRXa3ve4PuP/hNyQf6BW0J54NpH4Aq+eR1O38jTdAiwS6Q/Y6LwmXjjbWIuONBSwKykcRTFJFKYT2EftJx4TncrGeddjnnKMf1d5evG9ch8zV0xYOJmLKcS/mh53FafM/1wq/8FotJD6O9hEHRUrXzpOUOa5/81Fmvpa2mP9DgqjcJIjYr17TsK0ku0Feo+zS19gBkFbw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/6] xen/trace: Don't over-read trace objects
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-2-andrew.cooper3@citrix.com>
 <ed07d2de-9354-6c6d-b9f6-4b325ca328ce@suse.com>
 <0db4dc68-ccf0-0de6-61ce-7f155431a977@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8b2a548f-a955-302b-82ce-59c061537e27@suse.com>
Date: Mon, 20 Sep 2021 10:00:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0db4dc68-ccf0-0de6-61ce-7f155431a977@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0125.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bad3ad26-677a-4f79-eb6f-08d97c0cc96f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2447A92DE92690D1925EC3ECB3A09@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	i0ISI0XqPxsBCpX5INHJbuVIgfFZDXd9+jw8BC6h/N9Kl4XGf22fpCAUnADIHC2D2cT2Iww/C2S3G+o/9cq/I0Ez+B+Skcy7r+b2xviIe+SmBnbzuk+CLSvARznHGplEL450e5IQ7NFNSNYAyEaoV2R43TvJ15XEUa6zpGFtk28SziFljsazgpJWO5eLA6M4Rs8wPRroOHLdypOV1iRKXJFWo1GdL7G46yssPsHaC46fjqKIy/sslQGu623EyQCY06wBETgBuM2J8+kQIFgLGNwgCdWV20Bwi3NWaHmSkbQxZxBe/2cWW9wUiqUiqrKQmlcdqRJuhAKc39Ydh9SwY6dWN+h+AkqkW3eWZjKftFXkLncUvcduMpdHhleJbqJdiLkZ4x1nCabDCHwBPMaqq2zPhA7Qb5FuLJs4Bf16Um9cyxwDzPesX8REs96fBQmRkw3YE4yJquCNalsjN4GAW6qxD/bqc4luYnqcc9bbtbZQTt7+kiqkeLqfq+f/eA9o2rJ1IPU9n4ggaIRRvVSDPqx0dM1dGhF7T4bCWik3ym8nztvx+TOecmF9rNYXVnpPUpiT27rlQz9zEHGG7rr+2cta1frLUVBgq6pg/QB9xXF2wm5Al9kKGNy7tpsXVbE2J7SR5qa1vllK9EjujpG/VjzZKeH7OVWfZ24KgTIBaACohRLsaZHEh39++drSrrLxXNEtulkIHISricKKCGddPw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(366004)(346002)(376002)(396003)(136003)(186003)(2616005)(956004)(38100700002)(66946007)(66476007)(66556008)(83380400001)(16576012)(6916009)(53546011)(86362001)(6486002)(31696002)(54906003)(26005)(36756003)(4326008)(8936002)(8676002)(31686004)(2906002)(5660300002)(478600001)(316002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OUt0cHBWdlk3TzZPenZScXJlOUloRVV1SDNmVE1GS25qL1YyTmtzK0YxT3pV?=
 =?utf-8?B?SDNlblVMczZPM0QvU0ZhZHFGbFBxWUNlNXVLV0kycjVRdEJtUFFnT2xtVlFt?=
 =?utf-8?B?elB1ZXA2Y0dJZGxGZitOeUs2YzFBd0lWcURCbm5lazFpaEdZSUQzajJNdWZr?=
 =?utf-8?B?Wmp5ajk5Y0ZLWjUvSllESVByUHpuR2txK2dFQkx1dWdva3Azbk10Ujk1MFBC?=
 =?utf-8?B?cllEUDNEdWV1VXFwa0Q1Zi9ENTJVVUZrQTlyZ1JQN0VSWEFvOGhhWWtLUndF?=
 =?utf-8?B?RGhPVmQyS3NocWpPRFdzZFN0cVZiemdVTWtwaWNKVjhaTVJWWE9PTFN3TUNk?=
 =?utf-8?B?Zm9vT0FGZU5hWkM5NHpNNVNLOXR5M3RqcUVkKzAxSlFkTyswbVZKM0NjSllI?=
 =?utf-8?B?Z3lkYjhNblZacFU5bnFQVTdqM0traTBUT2lvbzU4T0VraDhGVVcwSzJya2ZI?=
 =?utf-8?B?OXp2czcwYitUcklVZDZpNW14T0gyd0s2cnJLUkdndm9Ub0cvMUtKaW00SFdB?=
 =?utf-8?B?RzhqV0l1MEVWYjQ4Q01ZcFV3cGp4czhnZ21ldTRDVVVHaEozUEpNaDVRcElD?=
 =?utf-8?B?YkdMWGFHcHZERTF6NitHNHZCd3gzcWJ1ZTZieXg0NkVDNndBcjE5c1ZQeGFY?=
 =?utf-8?B?c2pvNjYyOU5SczVadnR5eUNFRTdjbDd3WnZ2czcxUzJKeW9SUjVpdWY5RjlQ?=
 =?utf-8?B?TldCSkZpMFRlTFNsZGROWEdmOC8xd2RzNHpWamQ2UmZSTmpRY0ZmUGdCb0tG?=
 =?utf-8?B?L3NSMnVBZTBOZHFuMitFZUc2U000alBaVk1wU0Y0TkJmT0lFMzF6MmlMVms3?=
 =?utf-8?B?THo2UzBLWVNDdmVRRFRzR0puMG92UG1pY2tFY2cvWXhIQmN1MklkWVZzcFFJ?=
 =?utf-8?B?ZkxpVnBkcE9XNFlTS2NyZkMrSEZDaTVhQWFIc3VwL1A5TUcxZ3NLcXQvZzNP?=
 =?utf-8?B?YUhmUGxPZ1k2YWlDWWNlS1JmLzdpOFpkQjRSZDg5YjhpN2R0TXZCOTN6eURx?=
 =?utf-8?B?SjE0aFVRTUhBU1VmdE14TGt1eWJha0gvZng5TGk1eTZxT3B1Zk1BbHo2enN6?=
 =?utf-8?B?Q1VQSFVpVk5HVVBIMmRwWk1OcjIrcm0vR3NFMEE3aE9JTWI4Q1VCUWE5cVc0?=
 =?utf-8?B?dC9iOGxDODhicllrNEhFWkkvZEsrQThJeFAybCt4ZFpEN0Nxa0xJbVVQZXVp?=
 =?utf-8?B?RFBJL2hCQVlxdWNMV3liRkgrU3RxQXFUZFJuUFVXZTh3MkFCMnVpRGpBZitt?=
 =?utf-8?B?RjFDZ3hnZzRmZlhRQkMvNjBzMTgrN213YUNZMGt5MHc2dWw0ZjZYdW9IYm9B?=
 =?utf-8?B?WHlxVWx2ZTNQQzNyYWJJYXAwejc0TCtQVDQxOFV0cEFXTi91STlWVjBCM0tR?=
 =?utf-8?B?U3BiZVR0cWFYNU41OTAvcDR5Q2huY1hVZ29JeTZFa0xleUowSC92Z2NwcmQv?=
 =?utf-8?B?MnJHSEVwUkFVMDVONUVJWE1FcnptWnBNdFlvMkxVc0RyZVozK2c1RDR1Smhx?=
 =?utf-8?B?MVBXQ2t6YldkOHQzRzQyTzZqdjJlcTZWSVJxd0xYYWJ0ckE4OGxic1ZQZXRS?=
 =?utf-8?B?S2Y3V3JZRDYzcWpMV0R2NEU2VEpLcUJxa2ZTUDBiQ2tkNERpOENIYzhONVZ2?=
 =?utf-8?B?M2grRi9oYkFMMHNjQ1JnaTM1VFdLck8xaklWdWU4clFtR25nMDJwOFVGell5?=
 =?utf-8?B?enltUVlxTWxtTm82eUN4V2JKYW9mcWdCTDBlNzNITlVoWDkyc1RMNUdtVzlY?=
 =?utf-8?Q?eGtPfgs977Z98HpcMe4Uh2cVgKplzJC4elbNFRr?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bad3ad26-677a-4f79-eb6f-08d97c0cc96f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 08:01:01.1732
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2Vq4E97X1TySph1jKL4TZbGpGQnTneGWfETsc3VuGpRRQ8UG8av70cnD+9H3BYrNgdEl5rBlaJ47fEkYFNVUcQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

On 17.09.2021 15:26, Andrew Cooper wrote:
> On 17/09/2021 13:58, Jan Beulich wrote:
>> On 17.09.2021 10:45, Andrew Cooper wrote:
>>> --- a/xen/common/trace.c
>>> +++ b/xen/common/trace.c
>>> @@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
>>>      unsigned long flags;
>>>      u32 bytes_to_tail, bytes_to_wrap;
>>>      unsigned int rec_size, total_size;
>>> -    unsigned int extra_word;
>>>      bool_t started_below_highwater;
>>>  
>>>      if( !tb_init_done )
>>>          return;
>>>  
>>> -    /* Convert byte count into word count, rounding up */
>>> -    extra_word = (extra / sizeof(u32));
>>> -    if ( (extra % sizeof(u32)) != 0 )
>>> -        extra_word++;
>>> -    
>>> -    ASSERT(extra_word <= TRACE_EXTRA_MAX);
>>> -    extra_word = min_t(int, extra_word, TRACE_EXTRA_MAX);
>>> -
>>> -    /* Round size up to nearest word */
>>> -    extra = extra_word * sizeof(u32);
>>> +    /*
>>> +     * Trace records require extra data which is an exact multiple of
>>> +     * uint32_t.  Reject out-of-spec records.  Any failure here is an error in
>>> +     * the caller.
>>> +     */
>> Hmm, is "require" accurate?
> 
> In terms of "what will go wrong if this condition is violated", yes.
> 
>>  They may very well come without extra data
>> afaics.
> 
> 0 is fine, and used by plenty of records, and also permitted by the
> filtering logic.

I was about to say that the two parts of your reply contradict one
another, when I finally realized that it looks like the first sentence
in the comment can be read two ways: "Trace records require extra data"
then going on to describe properties, or "Trace records require extra
data to be an exact multiple of uint32_t." Obviously this is to me as a
non-native speaker. But maybe you could still reword this to be
unambiguous? (I'm not going to exclude that the lack of a comma, which
I did silently add while reading, makes a difference here: Does "Trace
records require extra data, which is an exact multiple of uint32_t" end
up altering the meaning?)

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:14:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:14:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190296.340125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSERc-0002bT-Dg; Mon, 20 Sep 2021 08:14:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190296.340125; Mon, 20 Sep 2021 08:14:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSERc-0002bM-Af; Mon, 20 Sep 2021 08:14:32 +0000
Received: by outflank-mailman (input) for mailman id 190296;
 Mon, 20 Sep 2021 08:14:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSERa-0002bG-BU
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:14:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d743f36b-c10f-47a7-a785-b20200901f7a;
 Mon, 20 Sep 2021 08:14:29 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2052.outbound.protection.outlook.com [104.47.0.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-wSrTziYuMt2trocxUB_AVg-1; Mon, 20 Sep 2021 10:14:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 08:14:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 08:14:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0053.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Mon, 20 Sep 2021 08:14:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d743f36b-c10f-47a7-a785-b20200901f7a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632125668;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TWErE/ky9SiU6UZ0auj0vn+oHlKF51mWPkEyZNXviko=;
	b=jgOahPfayqm4uPnCNzo4nd35OrYMfw7RiMwCe4ozpSETliRri/m0TrfgZucgWdxp3EHoYd
	Q/GX4VzpqYFWOioVJwnyUA4rhpC0467K7Rr3ZV1YwvfdNQmG2dH6gMHj+6/MRHkihlUO7c
	mO1A2CQ/dW8kH1AYITozKwzNjVxf7kQ=
X-MC-Unique: wSrTziYuMt2trocxUB_AVg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CI6FqHoutxeCVuMjMGgQ8qco3tviAV/3VW7tTkcdzWGde6dEg7HbkqeuV5Mch4wmitjFnWiTcVGf7e9v21k69IErMbEbArepdOSIOwC5gNwzVQrDxlIaWS/zPyjc7g6yHTHHzlvm5iFnbbkva96pgM4yzvF0Y14JFaeBj6AAN9CsNEQ1n5kZfW+LAdPM/BdE5n2o9+PQSLypcNSfptlJw3Bmwi7pmpq5BK1QvHTfgOKflk2jWgQW3bL24dn8k7l5lfadRuLATY+dyKVmq14HzCqRuE+YzpGheg3jCrusaxezLobYQYxtWYQctqkt+urQ2srpmzfl2gIUklDA2Hb7RA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TWErE/ky9SiU6UZ0auj0vn+oHlKF51mWPkEyZNXviko=;
 b=J0j53cAfjT6bkgBN+x72uTePCZCHIdxtNTkKdYrdroBpehdkN+D3yS1HA2JN42+WEHgrW55/JRnV0D1ZSTKXSCsCwGm9MFk8biVxrfdKxtubH6IyH/0bEhi3uUsJgplHYuEXVsMnxYwk1d1rJKdBnovhyuk910RBrKo/ePeo+M/269idToDZl3uvvXidLrnyo3wzL+uBe206aMcnFe885jtS/OgQF7CEPGOF5cIwnN7DzIz6fHs+l46IyRaMaKmpM3X94NyBb3wjARrJbvJCLBnmL+H6J75r2JCxoI+IXpF4Bj6c4iqEN9m76Oqs2ccw2MoPlKCZDCZmm9nnOU8IsQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/mem_sharing: don't lock parent during fork reset
To: Tamas K Lengyel <tamas@tklengyel.com>
Cc: Tamas K Lengyel <tamas.lengyel@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <c21708c84c850ff8c976a3934099c54da044e7d9.1631802816.git.tamas.lengyel@intel.com>
 <52b28c4d-1cf1-6c98-e1dc-1e0f7b2f768c@suse.com>
 <CABfawh=B3zwfroiu=pPY5BTCMBkapYEGXixkRg66aXLQWxwU4A@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <bd4b342f-8a90-7643-9232-02c08cf14f93@suse.com>
Date: Mon, 20 Sep 2021 10:14:23 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CABfawh=B3zwfroiu=pPY5BTCMBkapYEGXixkRg66aXLQWxwU4A@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0053.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2507cce1-885b-4590-c2e0-08d97c0ea7e1
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2703E878404948B6A964CC92B3A09@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aRhU/bAny9uwymExqWCiVKp1DG/UTyLD0D4h8w3zTxVk512lv96cPHXXGBJb72cXzB08NTOkL7Zjvci7+jsQ9UHW8xdRTg+BwCQOZ00AN5364FhULloWSC3HSAqSq0qdYyieJke/+GLTtYFmlp7kvh+z05yvRGhuky78k9FUdcbF1VDC7tPsJ80kIHl33NmT1X7/YXVnOSmnq1Q0S7bES0ldtc3X+Fe0ctveZ9gaMos6ucGa3CtSJXWjxnog6O+ywnh7z1kkf7WBD9DC0MNggl3cUY0OliG/Wkfh0yUVD1HLe5tLjanfFpUXgKISw1AYSM9GbyloN2YKwHTVxtv3UVwsBxpHbZFa2MGDG83Is20zmjWDN1onBLxiieVKX3jTkfUa+C8NZXs1UbKpC6fS+OxcaAqaNjwXbDtjyT44DjcdXKmTpxenLQ3vg8QRsIUkTiw4LJPxa9B1j1bmI+ZIikIiKZCgMakZEPR7KThKlu8kHtvGKQPBGcWavPFAEYqQF/3M22A1M1rF0MzO68qE2DkJIf1RQBR027n0iXZMs9/3oXzemnUQyO2SCOz70a0Ioii+ej9SPVOm6ujS5igOoF8Qsed8gElIwQoA7EqCcML6l0IrcIbzVVhMJ/HbcaovZqyGIFh/RoDxkMPIqZZo6kbQyo+4dZK0ii303ad4370h9esiwqt7Iip149r8G4I6kFGt7tca0tAjXUjcTVmd39UH1e7BZ/7PCpnY9EpiqWY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(376002)(136003)(346002)(39860400002)(366004)(36756003)(956004)(31686004)(2616005)(5660300002)(4326008)(16576012)(31696002)(66556008)(316002)(8936002)(186003)(54906003)(38100700002)(66476007)(6916009)(53546011)(8676002)(478600001)(86362001)(83380400001)(6486002)(26005)(2906002)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L01ySXFNTFc4UE9oTGZaQkhvTWhaaVMrQ0l2aVhTVTV1eFpSdFRBWmhZNDdR?=
 =?utf-8?B?Q3JQRGZwVzZ4VlhhYVFrOFB6TTFkYVh2L3o4cW0vbjhaR1Arc3drcWFXelps?=
 =?utf-8?B?Z2NORGw3VWxPWlV3SCtHaWd1WVdtY21PSUdISE9IUm9DYmdnSFNRUTdUazhY?=
 =?utf-8?B?WDhUcTVlVzVUeTRXdUJSK3haZFROR0JSWlc3VDUvU1I0UTZoSW9VR3I2V3Ey?=
 =?utf-8?B?ZUgvZ0granJBc085emFiaFFlUno0R0t3bEtmdU1xcmpaSUU1bU9ORHhDakJ3?=
 =?utf-8?B?N2hGSWVZanZtUzg4b1VCL1p5Y1FhckJ3NU8yL2RJVVZjQ2tUNWFrVDVlWDFJ?=
 =?utf-8?B?TTVEM2NmanRHMWhLSzhXcmtBTTU1ZTh3UisrRWJsTzZrbGlGMFBjckltalFW?=
 =?utf-8?B?elVUSFgwMHZFcXN0L1RLNjk1Zjk1djJMYkV4QUM3MVNRYW9naDk0MWx1SFNF?=
 =?utf-8?B?MkVWTTVMc3hyS0Z6c1JEYWtaejBqVU9hR2FsVHlwMENlRWxLQnZBbTVHZGVx?=
 =?utf-8?B?ZHdZakMvcVA5NWRXNDZKcjZzR0l1ZThxdUs2YjVnKzUxUUF0ZHlUQnZMWG1O?=
 =?utf-8?B?M01NMlRPUUtxMFU2RDZCV0ExMkVpcERsa29CRThlTHFuY2JGb0dyNlRDbVR5?=
 =?utf-8?B?TXVrRThJZ2ZSZExSdlNYdFRaMU8yK3lBN01TeFk4T3VyV0d4azBta2Zlcmtw?=
 =?utf-8?B?OCswME9FdlhxRGlHMk1oSkdubHlYRFoyS3krdVJpNGZZVzgwV1dUQ1NyV3My?=
 =?utf-8?B?TTdwdFFqZU14WTRldjZsZURFOTF4ejdzK0UxQVk4MWFUcnp1WmJVeHFwdjBr?=
 =?utf-8?B?MFNoL2hwQjc3aGNaQU1jTEI2Yyt0ZTBCZEl5bTNYSlpGRWZ2Z3hLOW1DUXdq?=
 =?utf-8?B?VnZsVXRFVWxiZExTcFlzQzVoSk45dzJoT1Q5QVIwWUprMTZyTHJHTC9lTmw5?=
 =?utf-8?B?VzRUR1NmUWJRMVBSTW5EaVozQXpFOUY5ZGlicDVlMG5NZHIwZmZxSGhBaThq?=
 =?utf-8?B?blUwdkgrKzNuaUZPbTZ3L0QwRGRrRW00MEdoQkYzdGN0MWwvbzFlZDZLSmRF?=
 =?utf-8?B?cjUrK1pKbzMyL2VrcHQvZW9FVzBaREpRdGwxUWR4aDFvUnBUZFhZeldvMTlF?=
 =?utf-8?B?U3gvQ2ZDZTEwZ1IvUjFWQTdlWTJDSjhFUStMUnAwT3g1amFnTGxmMnlFZVVs?=
 =?utf-8?B?TzZDZGxNV2dyM3dEcnZ1dG05dzFWdE15L2V0TXJWbzFIdEdLY2h6OENJREhi?=
 =?utf-8?B?WU9CODRwK0ZuVVE1VlJKeWh2ZXlFa2I2VXpnaWMybmRjcUtDY2RtMGFuR0JX?=
 =?utf-8?B?aFJTYmZDZTdVeW14cFpOTW5hSU1xUUJlVkg5UFM0TzBhTnM1cDJnL3ljbnJ3?=
 =?utf-8?B?NUdMRGZ4Nm16Uzcwcjc2NlBKT0oyNGcvby95SjBCSVlaZ0l3bVI1Nk1DK2Rw?=
 =?utf-8?B?aEFnUHhDbW5WT2JsSkJNb0hoblg3T0ZkTEJwV3RRVFZLelAybm9hanlteFhB?=
 =?utf-8?B?dDU5T2hsS0JubUlCWVhlTE5VcU1UaldHalhoS1RONU5Mc0NoS3JQdEZMTWFv?=
 =?utf-8?B?SFRkK0JWcHlZV3hjT1ZxWTdhMVFFRWZFVnhOQ1R6SEc0QkhHU1RZb3Z1RUNv?=
 =?utf-8?B?Y0hQbE45NjlHb2FmSWs4Mzd2K0Q3N3hTRzIwN2RBR29iSllXYlhZOFpyanZh?=
 =?utf-8?B?YnJyREd0OEpCRngydjVnQUxFMkVWbFdjN2hWT1RFbzZ6RnVrTUJZeUVpVlRj?=
 =?utf-8?Q?WTOCw6AUJTc7RH/VvmSU7SySy/FhvwWCAB3i91T?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2507cce1-885b-4590-c2e0-08d97c0ea7e1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 08:14:23.8622
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: D8e1xbRb8S9yikHoi7PmTSiHfbgMlTDkdgZRHeN4q/K/kvvAGN9M8G0SsPjQmgEy+Zpjhs2qzKIBO1b8pfcJgw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

On 17.09.2021 16:21, Tamas K Lengyel wrote:
> On Fri, Sep 17, 2021 at 3:26 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 16.09.2021 17:04, Tamas K Lengyel wrote:
>>> During fork reset operation the parent domain doesn't need to be gathered using
>>> rcu_lock_live_remote_domain_by_id as the fork reset doesn't modify anything on
>>> the parent. The parent is also guaranteed to be paused while forks are active.
>>> This patch reduces lock contention when performing resets in parallel.
>>
>> I'm a little in trouble following you here: RCU locks aren't really
>> locks in that sense, so "lock contention" seems misleading to me. I
>> can see that rcu_lock_domain_by_id()'s loop is extra overhead.
>>
>> Furthermore - does the parent being paused really mean the parent
>> can't go away behind the back of the fork reset? In fork() I see
>>
>>     if ( rc && rc != -ERESTART )
>>     {
>>         domain_unpause(d);
>>         put_domain(d);
>>         cd->parent = NULL;
>>     }
>>
>> i.e. the ref gets dropped before the parent pointer gets cleared. If
>> the parent having a reference kept was indeed properly guaranteed, I
>> agree the code change itself is fine.
>>
>> (The sequence looks correct at the other put_domain() site [dealing
>> with the success case of fork(), when the reference gets retained]
>> in domain_relinquish_resources().)
> 
> This code above you copied is when the fork() fails. Calling
> fork_reset() before fork() successfully returns is not a sane sequence
> and it is not "supported" by any means. If someone would try to do
> that it would be racy as-is already with or without this patch.
> Clearing the cd->parent pointer first here on the error path wouldn't
> guarantee that sequence to be safe or sane either. Adding an extra
> field to struct domain that signifies that "fork is complete" would be
> a way to make that safe. But since the toolstack using this interface
> is already sane (ie. never calls fork_reset before a successful fork)
> I really don't think that's necessary. It would just grow struct
> domain for very little benefit.

The point of this latter part of my comments wasn't to suggest that
fork-reset ought to work before fork completed. That's fine to not be
'"supported" by any means'. What your change here does, though, is to
add a dependency (maybe not the first one) on there being a ref held
as long as ->parent is non-NULL. That requirement is violated by the
error path I've quoted. IOW my request isn't really fork or even
mem-sharing specific, but it instead is asking that the code in
question please follow a common, safe model (as soon as at least one
such dependency exists).

If there are pre-existing cases where the wrong order of operations
is an issue, then adjusting that sequence in a separate prereq patch
might be better than folding the fix in here. Whereas if there isn't
any other such case or it's simply unknown (without extended audit)
whether there is, then I see no issue folding that adjustment in here.

Of course - you're the maintainer of this code, so if you think the
adjustment isn't needed, so be it. It's just that then I can't give
you an R-b, so you'd need someone else's for your change to actually
go in. (Of course you could also convince me of your pov, but for now
I can't see this happening.)

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:19:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:19:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190304.340137 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEWW-0003Qz-6r; Mon, 20 Sep 2021 08:19:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190304.340137; Mon, 20 Sep 2021 08:19:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEWW-0003Qs-3W; Mon, 20 Sep 2021 08:19:36 +0000
Received: by outflank-mailman (input) for mailman id 190304;
 Mon, 20 Sep 2021 08:19:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSEWV-0003Qm-Bc
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:19:35 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fadc63c1-a583-431d-9695-7d17f8d0acbe;
 Mon, 20 Sep 2021 08:19:34 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-2-nioiX3AEM1CtRthzbdCQBw-1; Mon, 20 Sep 2021 10:19:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4942.eurprd04.prod.outlook.com (2603:10a6:803:59::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 20 Sep
 2021 08:19:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 08:19:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0265.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 08:19:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fadc63c1-a583-431d-9695-7d17f8d0acbe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632125973;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XHapG8vGOL6k8MmAWvtdgtXa/ewF3aDJIfEjIaAOB6k=;
	b=FwJX5PZgRaSprrhLldhZgVDml0fcgzEEuj0hh5QBmAszrD/yjImFqSavGmpx3agTaKD/Sc
	sYtylUZIVBrPT4XyBsMs1eirc8MvTTt+LAjoD3SgtnHv0fBcpPo7XGe2zcjLUvPbM2O2Y9
	WOD53WnMBB3ixCcTse0rJdtjoTrTd9o=
X-MC-Unique: nioiX3AEM1CtRthzbdCQBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LEtieiN88TuWYQhzutF/GZiVq7N+zHXmC1Iz8ofaA9mipkaAUUZ3NGGU6vTj8VCSaLV+lyPgkapZ+KmCWWLfyvLX0LSD9DksgzIYtjjM6LYo68IeByzyjBRu6cG6pcqGHH/NuTutlZV67eir1vho6Iykmr2BqB3gHDaEmRaN/UABHd4FXU3lVeSPJTpnvYisOAXcM08KP7OoDtyv6yX4xar7VAf1ChYOvEFIUGyou+fVEiQ7ouIY7GhlQpHCAsJnlLB5A4lcncYmlDQBlkGpElLaxEfZKjLMvBWGKJdq7o54J1rFJs0gH/8ooIsCUEYqk1ZE8Ueagha1ssiHKGXXJQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XHapG8vGOL6k8MmAWvtdgtXa/ewF3aDJIfEjIaAOB6k=;
 b=DXwpXIJ2Aw6cVHz0zosNQG5ky/iJ93fmFJFNCLUmQdhIKVu7Sluvvqq3Vy4zr8biXq9VC7q4Dicv/UYzQ8fxea9CgF2Hr2MdPY9f4OMsOC63S8Ml3vNjTcDQUrkEGGQIjWOlWqk2Mmjfby1uwrFV0ZGbovVFoZre/GlDgHsfdN304wa/O+TFvEzRp8IqtpFwqiilLkaV19+tmML+faydhiZx78rSMQcj9Eyd0maRC6HQGDf/+MSUKCXe10GLHUw7nX8jmzx48WTp4FsqnoUrneCgdSUKLe/Xcv8xQemTOT7tWUwUkFjwdEluWt6PSMBXicSRgwMcxEb2JzAC01Y70w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
To: Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Ian Jackson
 <iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org,
 Dmitry Isaikin <isaikin-dmitry@yandex.ru>, rjstone@amazon.co.uk,
 raphning@amazon.co.uk, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
 <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
 <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
 <f6225dc6-0590-3456-8c48-7ab29aa00200@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c242e3cc-d4ed-73b7-8e38-f94a8c479d20@suse.com>
Date: Mon, 20 Sep 2021 10:19:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f6225dc6-0590-3456-8c48-7ab29aa00200@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0265.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 85a2d57f-b30d-48f2-1c37-08d97c0f5e81
X-MS-TrafficTypeDiagnostic: VI1PR04MB4942:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB494245E9BEC2A8A9D298224FB3A09@VI1PR04MB4942.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wCg3nREFKT8aFarEE8RfArIqMTggtj3HBrhMp8PIqSiB21710EDUMDSA+sbEkKgDmjqUasKCXMaBcUH2Lhe/PlK4QL1tiwjQRPEvQo3EBPKElgf9a4AT4ZaitD6cG+C8CfvAY6i+LXiMo3KZ1lmantZSdf5qflkXRjbBsL5MJUHbx/CPfojbk5pcCUf8BNuka4OWb25z0gCKF0zxOeUIHAuxxNiTMhjj5kM9uwn38tnx5uYe3K2X/7UzJkyOAO6Mg3Y1SsAbF8ebxy4uiC6QJaOqk/XtaZTEV2JyRv5HmPue+sZ7ef/L0k2OD3QgtWfChHDN82Q281CRYcewduOtR7qBTFPm1kdsAZj3kM/LtAXRxEYT9grttFH9M01HMLk51NUsyDkiMBQjluEvcV2Q/1nZxzGCpQVc0LBisyqeEJ9LxMxtcRziOs3B/QdUc88mgbnXe/Bna+QswDTgIAp62T3Be4HFO1Q8udhnoK0GCf9W9l1Czxs+FMA41GZyecQjhkRj0HNqz5JnbIOfI/g8tJb+XdYvNdp4j+9ZzLMVoTGhxsBlaNCxLb6qdK/EZBCDI3NKHgXF+dZJPyBEc+owGoXd0B2fihCXklJlPnrLsNZOr+QbeNz6zJU38WI5m19c77FilVROAW/Qu7sQ1MqULA49XR5YfFBTBbnJjOhcB60Re1rELSJklPOi9PNuyemMgybZ2P9ROQiNliwSK1ZQg3WKoWGkxf1lUwq8ri2iJTM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(376002)(39860400002)(396003)(136003)(6486002)(2616005)(38100700002)(6916009)(7416002)(66946007)(956004)(186003)(26005)(83380400001)(36756003)(31696002)(53546011)(86362001)(2906002)(4744005)(54906003)(316002)(5660300002)(16576012)(31686004)(4326008)(66556008)(8676002)(478600001)(8936002)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QXBMNGlWa2NTUVBPTGxMUjRvUmRVODFEVTIxZjdTdlM1Q29yenVDbkM4UGlR?=
 =?utf-8?B?b0JmM29YQ2NvOWJLOHBXVWc2WTFQcytydUtveWUyOTI0eE84eUV6RU1QcjJm?=
 =?utf-8?B?aVVnNlBaUklMOWxDNmV6eGcyNituNytLNlhESXJzaFlvbDEyd0VrdlhuNWFz?=
 =?utf-8?B?Q04ycmZtS2hMU1VobHFnMzl6RnJydG9tbmpiaHJ0MXFSa25Pbjl0bjV4S0wy?=
 =?utf-8?B?NmxURUN4bHdFUyt1b2hGODkwOXRhSUc3ZjVnSlNPOE5iR3VJN0M5TldOdkh0?=
 =?utf-8?B?LzFQTCttOFFDNUxGSitpV1JhYi9JcWpPQ0JLMnZXM1IwOFV2azlHUERWRXBQ?=
 =?utf-8?B?TXpjM0JMUm9BWDF6UDZKOXZqdGE4dmwrQVFoUm8rclNPWlhZV213VzV6cDlQ?=
 =?utf-8?B?UEc0ZzFCK25sczAyakwyS0o0aUdMVXkwLzJYckNBWDc0R0RiU2djc3RJcFZG?=
 =?utf-8?B?SUJsRm1pY1U0aHVQWWNzVXoyZ251TTIyVWYxb2I3RG16ZytUcmo4ODR5Y0Yr?=
 =?utf-8?B?dzhpNmVySFZRN1lYSVlQalplZC8xT21hWnhJN2NmZ3orVWg3eXh0MldObFBy?=
 =?utf-8?B?dzlXSnU5aTJUdDNaanVIam5rZmRDcy9LZyt4d1hHWnBpYnMzRXN1YUJDT3lv?=
 =?utf-8?B?cUp4UFAzbngrZ3gxalIwN3JuZzJ4ZHRJZG9tUXM3cEdaV2FSbVl0MGZLT3NC?=
 =?utf-8?B?dlZWQ0VsZGdMb2U4YUxSdENLd2phb2xqTjNCVUkzbW1PckNWMDlKRWRPcDlM?=
 =?utf-8?B?ZXVjMzlSRjUreDZJV1BRQlRydEkxVGQxY000ZHdmVElqbTN2dXR5MDdsUWJF?=
 =?utf-8?B?TUJldkliK2E4R1dJUFRnY2JEV01kbnJNODVsVXZxcVF2MGV4dGc5R0JicWFO?=
 =?utf-8?B?eWVRalhsbzk1UVBvSWdaRUZKMGdRQTlMaSs4QS9FRkY2bE5TMEpSc0Z6YTdl?=
 =?utf-8?B?R0h4c2czdTBTMHNEaVpGSTVoYkUxR29VVTF6VkJYcEtjbWJBUWJlWjVMWk5o?=
 =?utf-8?B?YlhFZ0pNNkliSlkyZEl1dlVUTTNhbXcwUmFEUmk3ZFF3REplVDR3blpscjRY?=
 =?utf-8?B?ZTFIeDZBaEhhZmFHTjQ3KzYwRXhhU2RhUGlkQXArOU9rNjlQK3ppa09UMGNo?=
 =?utf-8?B?aDJQMktOYmZ2SE5yVk5NSWRaRlNiTThqU3EvYnJTZG92RTZiK01WbUQxdUph?=
 =?utf-8?B?eElOWmtvZlY4L2VtMFdjekM0cVdLZmw1UFl0R0t6Q2NNREw0YzBSMTJ1YjVB?=
 =?utf-8?B?bDZEd1JiL09FN1J0VTFvTm1tOGF5cVJMS1JPakNBMlBMejlNNFNGR3FNVWx6?=
 =?utf-8?B?YTdKSEUyaTRjaXMxY0M1VDN1M2N1d0Uxa040a2FaRnJDdnU0alRCRVhiTmVG?=
 =?utf-8?B?b000TWUwRzFWOXA1c3o1M0ZNRjd4eGdETXpKbGNDT0ZSQXF3MnBXWEpvYlVB?=
 =?utf-8?B?MlVqUEsrckZ1TjJuUVJPdzZIU25mRWJ5OHV2SGplcTRNUkxpUnJwUFBIMml2?=
 =?utf-8?B?bEJRczdnbm1ZSGdjUVZWQytGQzNEbnJmN2NiTzN2N05hY0JnZTJJb1p4N2NO?=
 =?utf-8?B?ZjVpYUhraE14cUlqRFdJdldtUXpqekFORW9wei9MQzNjak1EeXpVeFBzZ2dT?=
 =?utf-8?B?ZGtzYkVHdjc5Sy9LODU2U3JhYVNWdDg1cWZnKzQ5R1crNWhNQ0VNbGVqK3RX?=
 =?utf-8?B?TTE4T1ZhSXIzc2c2YWpxWUJMYlBlc2o0a3hHTnZoTHZyMThFZWlXZVYrZkVF?=
 =?utf-8?Q?N3itjP5/hiWZmDK+yBt9kC4hQkZmoYLa+wErEY/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 85a2d57f-b30d-48f2-1c37-08d97c0f5e81
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 08:19:30.2648
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gE7+JpUld+MYA9ixvqLZfAlU1NMdB/FK2RHLRxtJqIINKnRCv+O8R3tLsGK/wPo6m9vWGSkIr7p5vku/htBGew==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4942

On 17.09.2021 18:01, Julien Grall wrote:
> I will quote what Hongyan wrote back on the first report:
> 
> "
> The best solution is to make the heap scalable instead of a global
> lock, but that is not going to be trivial.
> 
> Of course, another solution is to keep the domctl lock dropped in
> domain_kill() but have another domain_kill lock so that competing
> domain_kill()s will try to take that lock and back off with hypercall
> continuation. But this is kind of hacky (we introduce a lock to reduce
> spinlock contention elsewhere), which is probably not a solution but a
> workaround.
> "

Interesting. Is contention on the heap lock merely suspected or
was that lock proven to be the problem one?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:24:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:24:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190310.340148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEbZ-0004oT-Rq; Mon, 20 Sep 2021 08:24:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190310.340148; Mon, 20 Sep 2021 08:24:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEbZ-0004oM-OA; Mon, 20 Sep 2021 08:24:49 +0000
Received: by outflank-mailman (input) for mailman id 190310;
 Mon, 20 Sep 2021 08:24:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSEbZ-0004oG-3o
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:24:49 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2e571aaa-350e-4913-8e85-f981df22da30;
 Mon, 20 Sep 2021 08:24:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id ABD9620041;
 Mon, 20 Sep 2021 08:24:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7C6F913ACC;
 Mon, 20 Sep 2021 08:24:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id ysJuHE5FSGGNGgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 08:24:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e571aaa-350e-4913-8e85-f981df22da30
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632126286; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=KG9nN2IOsT4a5w0uHs+Gi11848zjWgSPv3lXhfAD5uo=;
	b=eGzBTSp2wL66vo61MJMX+mLk4xEawq2JwFxzC1cSNuDkx40cirTpwRDMPJNeV0wb+wyaB2
	YxYVEv0H6U8/+YV2RfsI5T1t8VKBPi8sQfboNLgjM4z76G2vqQvGJQ9NTBJTraiw0B7cPK
	ZjmQoNzpgbxyeI73fsu6/D7+OYmXp6s=
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-6-roger.pau@citrix.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring
 as foreign memory
Message-ID: <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
Date: Mon, 20 Sep 2021 10:24:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210917154625.89315-6-roger.pau@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Hsp4u7qZ65kAJHwz1ETznUGJ4lJeCDzwu"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Hsp4u7qZ65kAJHwz1ETznUGJ4lJeCDzwu
Content-Type: multipart/mixed; boundary="M7VJ1cDQq1MuKlTopkoGEWM0fgh7UKna2";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
Message-ID: <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring
 as foreign memory
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-6-roger.pau@citrix.com>
In-Reply-To: <20210917154625.89315-6-roger.pau@citrix.com>

--M7VJ1cDQq1MuKlTopkoGEWM0fgh7UKna2
Content-Type: multipart/mixed;
 boundary="------------A99C9F8B3E6D7D0E4FB0E828"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A99C9F8B3E6D7D0E4FB0E828
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 17:46, Roger Pau Monne wrote:
> Restore the previous way of mapping the xenstore ring using foreign
> memory. Use xenforeignmemory instead of libxc in order to avoid adding
> another dependency on a unstable interface.

Mapping a guest page via xenforeignmemory is no good move IMO. A guest
not supporting a grant table for security reasons is a rather strange
idea, as it needs to trade that for a general memory access by any
backend without a way to restrict such accesses. This contradicts one
of the main concepts of the Xen security architecture.

At the same time this will either kill the use of xenstore-stubdom, or
it will require an extended XSM configuration allowing xenstore-stubdom
to establish arbitrary page mappings of those guests.


Juergen

>=20
> This in turn requires storing the gfn into xs_state_connection for
> resume purposes, which breaks the current format.
>=20
> Note this is a preparatory patch in order to support the usage of
> xenstore on domains without grant table. This not only allows xenstore
> usage, but also makes things like "@introduceDomain" events work when
> using such guests, otherwise the mapping done in introduce_domain
> fails and the "@introduceDomain" event won't be signaled.
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>   tools/xenstore/Makefile                 |  4 +-
>   tools/xenstore/include/xenstore_state.h |  1 +
>   tools/xenstore/xenstored_domain.c       | 69 +++++++++++++++++++-----=
-
>   3 files changed, 56 insertions(+), 18 deletions(-)
>=20
> diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile
> index 292b478fa1..9a9f7be5cb 100644
> --- a/tools/xenstore/Makefile
> +++ b/tools/xenstore/Makefile
> @@ -67,10 +67,10 @@ $(XENSTORED_OBJS): CFLAGS +=3D $(SYSTEMD_CFLAGS)
>   xenstored: LDFLAGS +=3D $(SYSTEMD_LIBS)
>   endif
>  =20
> -$(XENSTORED_OBJS): CFLAGS +=3D $(CFLAGS_libxengnttab)
> +$(XENSTORED_OBJS): CFLAGS +=3D $(CFLAGS_libxengnttab) $(CFLAGS_libxenf=
oreignmemory)
>  =20
>   xenstored: $(XENSTORED_OBJS)
> -	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(L=
DLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLA=
GS)
> +	$(CC) $^ $(LDFLAGS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(L=
DLIBS_libxenforeignmemory) $(LDLIBS_libxenctrl) $(LDLIBS_xenstored) $(SOC=
KET_LIBS) -o $@ $(APPEND_LDFLAGS)
>  =20
>   xenstored.a: $(XENSTORED_OBJS)
>   	$(AR) cr $@ $^
> diff --git a/tools/xenstore/include/xenstore_state.h b/tools/xenstore/i=
nclude/xenstore_state.h
> index ae0d053c8f..8dcc8d9d8b 100644
> --- a/tools/xenstore/include/xenstore_state.h
> +++ b/tools/xenstore/include/xenstore_state.h
> @@ -80,6 +80,7 @@ struct xs_state_connection {
>               uint16_t domid;  /* Domain-Id. */
>               uint16_t tdomid; /* Id of target domain or DOMID_INVALID.=
 */
>               uint32_t evtchn; /* Event channel port. */
> +            uint64_t gfn;    /* Store GFN. */
>           } ring;
>           int32_t socket_fd;   /* File descriptor for socket connection=
s. */
>       } spec;
> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstor=
ed_domain.c
> index 8930303773..f3563e47aa 100644
> --- a/tools/xenstore/xenstored_domain.c
> +++ b/tools/xenstore/xenstored_domain.c
> @@ -33,10 +33,12 @@
>  =20
>   #include <xenevtchn.h>
>   #include <xenctrl.h>
> +#include <xenforeignmemory.h>
>   #include <xen/grant_table.h>
>  =20
>   static xc_interface *xc_handle;
>   xengnttab_handle *xgt_handle;
> +static xenforeignmemory_handle *xfm_handle;
>   static evtchn_port_t virq_port;
>  =20
>   xenevtchn_handle *xce_handle =3D NULL;
> @@ -66,12 +68,18 @@ struct domain
>   	/* Generation count at domain introduction time. */
>   	uint64_t generation;
>  =20
> +	/* Store GFN (if using a ring connection). */
> +	xen_pfn_t gfn;
> +
>   	/* Have we noticed that this domain is shutdown? */
>   	bool shutdown;
>  =20
>   	/* Has domain been officially introduced? */
>   	bool introduced;
>  =20
> +	/* Is the ring memory foreign mapped? */
> +	bool foreign_mapped;
> +
>   	/* number of entry from this domain in the store */
>   	int nbentry;
>  =20
> @@ -196,16 +204,29 @@ static const struct interface_funcs domain_funcs =
=3D {
>   	.can_read =3D domain_can_read,
>   };
>  =20
> -static void *map_interface(domid_t domid)
> +static void *map_interface(domid_t domid, xen_pfn_t gfn, bool *foreign=
_mapped)
>   {
> -	return xengnttab_map_grant_ref(xgt_handle, domid,
> -				       GNTTAB_RESERVED_XENSTORE,
> -				       PROT_READ|PROT_WRITE);
> +	void *map =3D xengnttab_map_grant_ref(xgt_handle, domid,
> +					    GNTTAB_RESERVED_XENSTORE,
> +					    PROT_READ|PROT_WRITE);
> +
> +	if (!map)
> +	{
> +		map =3D xenforeignmemory_map(xfm_handle, domid,
> +					   PROT_READ|PROT_WRITE, 1,
> +					   &gfn, NULL);
> +		*foreign_mapped =3D !!map;
> +	}
> +
> +	return map;
>   }
>  =20
> -static void unmap_interface(void *interface)
> +static void unmap_interface(void *interface, bool foreign_mapped)
>   {
> -	xengnttab_unmap(xgt_handle, interface, 1);
> +	if (foreign_mapped)
> +		xenforeignmemory_unmap(xfm_handle, interface, 1);
> +	else
> +		xengnttab_unmap(xgt_handle, interface, 1);
>   }
>  =20
>   static int destroy_domain(void *_domain)
> @@ -228,7 +249,8 @@ static int destroy_domain(void *_domain)
>   		if (domain->domid =3D=3D 0)
>   			unmap_xenbus(domain->interface);
>   		else
> -			unmap_interface(domain->interface);
> +			unmap_interface(domain->interface,
> +					domain->foreign_mapped);
>   	}
>  =20
>   	fire_watches(NULL, domain, "@releaseDomain", NULL, false, NULL);
> @@ -363,12 +385,15 @@ static struct domain *find_or_alloc_domain(const =
void *ctx, unsigned int domid)
>   	return domain ? : alloc_domain(ctx, domid);
>   }
>  =20
> -static int new_domain(struct domain *domain, int port, bool restore)
> +static int new_domain(struct domain *domain, int port, xen_pfn_t gfn,
> +		      bool foreign_mapped, bool restore)
>   {
>   	int rc;
>  =20
>   	domain->port =3D 0;
>   	domain->shutdown =3D false;
> +	domain->gfn =3D gfn;
> +	domain->foreign_mapped =3D foreign_mapped;
>   	domain->path =3D talloc_domain_path(domain, domain->domid);
>   	if (!domain->path) {
>   		errno =3D ENOMEM;
> @@ -436,7 +461,8 @@ static void domain_conn_reset(struct domain *domain=
)
>  =20
>   static struct domain *introduce_domain(const void *ctx,
>   				       unsigned int domid,
> -				       evtchn_port_t port, bool restore)
> +				       evtchn_port_t port, xen_pfn_t gfn,
> +				       bool restore)
>   {
>   	struct domain *domain;
>   	int rc;
> @@ -448,17 +474,21 @@ static struct domain *introduce_domain(const void=
 *ctx,
>   		return NULL;
>  =20
>   	if (!domain->introduced) {
> +		bool foreign_mapped =3D false;
> +
>   		interface =3D is_master_domain ? xenbus_map()
> -					     : map_interface(domid);
> +					     : map_interface(domid, gfn,
> +							     &foreign_mapped);
>   		if (!interface && !restore)
>   			return NULL;
> -		if (new_domain(domain, port, restore)) {
> +		if (new_domain(domain, port, gfn, foreign_mapped, restore)) {
>   			rc =3D errno;
>   			if (interface) {
>   				if (is_master_domain)
>   					unmap_xenbus(interface);
>   				else
> -					unmap_interface(interface);
> +					unmap_interface(interface,
> +							foreign_mapped);
>   			}
>   			errno =3D rc;
>   			return NULL;
> @@ -489,19 +519,20 @@ int do_introduce(struct connection *conn, struct =
buffered_data *in)
>   	char *vec[3];
>   	unsigned int domid;
>   	evtchn_port_t port;
> +	xen_pfn_t gfn;
>  =20
>   	if (get_strings(in, vec, ARRAY_SIZE(vec)) < ARRAY_SIZE(vec))
>   		return EINVAL;
>  =20
>   	domid =3D atoi(vec[0]);
> -	/* Ignore the gfn, we don't need it. */
> +	gfn =3D atol(vec[1]);
>   	port =3D atoi(vec[2]);
>  =20
>   	/* Sanity check args. */
>   	if (port <=3D 0)
>   		return EINVAL;
>  =20
> -	domain =3D introduce_domain(in, domid, port, false);
> +	domain =3D introduce_domain(in, domid, port, gfn, false);
>   	if (!domain)
>   		return errno;
>  =20
> @@ -718,7 +749,7 @@ void dom0_init(void)
>   	if (port =3D=3D -1)
>   		barf_perror("Failed to initialize dom0 port");
>  =20
> -	dom0 =3D introduce_domain(NULL, xenbus_master_domid(), port, false);
> +	dom0 =3D introduce_domain(NULL, xenbus_master_domid(), port, 0, false=
);
>   	if (!dom0)
>   		barf_perror("Failed to initialize dom0");
>  =20
> @@ -758,6 +789,10 @@ void domain_init(int evtfd)
>   	 */
>   	xengnttab_set_max_grants(xgt_handle, DOMID_FIRST_RESERVED);
>  =20
> +	xfm_handle =3D xenforeignmemory_open(NULL, 0);
> +	if (!xfm_handle)
> +		barf_perror("Failed to create handle for foreign memory");
> +
>   	if (evtfd < 0)
>   		xce_handle =3D xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
>   	else
> @@ -1189,6 +1224,7 @@ const char *dump_state_connections(FILE *fp)
>   			sc.spec.ring.tdomid =3D c->target ? c->target->id
>   						: DOMID_INVALID;
>   			sc.spec.ring.evtchn =3D c->domain->port;
> +			sc.spec.ring.gfn =3D c->domain->gfn;
>   		} else {
>   			sc.conn_type =3D XS_STATE_CONN_TYPE_SOCKET;
>   			sc.spec.socket_fd =3D c->fd;
> @@ -1290,7 +1326,8 @@ void read_state_connection(const void *ctx, const=
 void *state)
>   #endif
>   	} else {
>   		domain =3D introduce_domain(ctx, sc->spec.ring.domid,
> -					  sc->spec.ring.evtchn, true);
> +					  sc->spec.ring.evtchn,
> +					  sc->spec.ring.gfn, true);
>   		if (!domain)
>   			barf("domain allocation error");
>  =20
>=20


--------------A99C9F8B3E6D7D0E4FB0E828
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A99C9F8B3E6D7D0E4FB0E828--

--M7VJ1cDQq1MuKlTopkoGEWM0fgh7UKna2--

--Hsp4u7qZ65kAJHwz1ETznUGJ4lJeCDzwu
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIRU0FAwAAAAAACgkQsN6d1ii/Ey9b
SQf/Z1afyFTCfUjM2Xw7BBOxu/XLkEl0EJnYY6wwmlqsrqzDftcWj6me2kOBWA4e97WvPhSvdzbL
pCibiBMAbQjtfkgP9CWmhYLDgL39fs4SIZ+hGYr2dAusxMHRxSD133+/L1z9Gfiw6H3W5hW3JCZV
srjGYR179+GAv6bP2EDCDh0nloovppbC4cufZLq37gdRRW8zzZEno9LX9mAD95CzL8fSmR3+1Y01
Z7w7Oasqt5IR+jpt4sQlSk+3a/aQIVAl6S6LdMdvJodvAiT3D/OtKYt66ziqAZjqZCMbRmb/VVoS
j2FV8F4ssJb7Qvz6XCt/kfPAf+WKh+/kcRMgRNSnlA==
=TKD+
-----END PGP SIGNATURE-----

--Hsp4u7qZ65kAJHwz1ETznUGJ4lJeCDzwu--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:24:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:24:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190311.340159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEbf-00055n-3y; Mon, 20 Sep 2021 08:24:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190311.340159; Mon, 20 Sep 2021 08:24:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEbf-00055e-07; Mon, 20 Sep 2021 08:24:55 +0000
Received: by outflank-mailman (input) for mailman id 190311;
 Mon, 20 Sep 2021 08:24:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I+Th=OK=citrix.com=edvin.torok@srs-us1.protection.inumbo.net>)
 id 1mSEbd-0004oG-Vw
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:24:54 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b9d9065e-d9a4-4c6e-94d6-83607cc27844;
 Mon, 20 Sep 2021 08:24:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9d9065e-d9a4-4c6e-94d6-83607cc27844
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632126290;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-id:content-transfer-encoding:
   mime-version;
  bh=fx1zSfA1R220O/aaGUHyRpJ2+0Ro7oxjPjSBnJ41RiE=;
  b=YfjQmFbtAOmqBEdVhnl7HYYoGjo5O9BaIFJHOjL4Qu9dgoDsgHMW8j18
   WYDJxky874ltdkCc51X9kXj0Zf8poHDbsLJKont7/izvWqvoQ+8maeAEI
   r8lD53HMThKBjOh2T17pl9S+nyRxT9vkhWo7Wt9kTmF8XXVxTw0J2EDU3
   E=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 9LOx6gyPRIdLbue+VS7SBouuWKLAVP7GXXwr0y3/FRxvi+lQdoNPPHpxycfyNAnjaBZiEztiRg
 I8p95plcfsM6eb3oI+Q4xW7JA8FMCtbh8M3pibb42Gc8OxrNrtxDV25NQyh32A17r3UL7nsemE
 7lm43I88Bi90FF7plx9Ov8my4FHcYwb8NncjQhYwo5ItVGAHHyI2sF8dcdwkXUpjS4zZXDujXN
 iNMkWsV8Y1u6b8A9371DUxUxthyfdveTm8WfQciXAsSyj9zw+JUsvrynaCry6UndAbV/eBmpE9
 xLFp1dJnC3LUcOmTOXj5PNCO
X-SBRS: 5.1
X-MesageID: 53118422
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:jHKsZq53RdxtHyaDSARvKgxRtJbAchMFZxGqfqrLsTDasY5as4F+v
 jRKXWqFafmNZGOnKt1+Povn/RtSsJGGz9cwGgFt+S1gHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NYw2LBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Zz
 tdhh7m2VB8SJJLPisI+az4bAR5uIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALIcX1PYVZpnZ6yjLxBvc6W5HTBa7N4Le02R9t3ZkXTaqEP
 aL1bxJPTS/dekQICGwHAYA4jOyQqlr1dhNx/Qf9Sa0fvDGIkV0ZPKLWGNjaYNuRXu1Og12V4
 GnB+gzRGQoGPdaSzT6E9HOEheLVmy7/HoUIG9WQyPluh1GCw30JPzcfX1C7vPqRh1a3XpRUL
 El80iYns6Ua7kGgSdj5GRqirxasvAMYWtdWO/037keK0KW8yweGBEAUQzhZctskucQqAzsw2
 Tehj97vQDBirrCRYXac7auP6yO/PzAPKm0PbjNCShEKi/HhqowuihPETv54DbW4yNbyHFnY3
 DSivCU4wbIJgqY2O76TpA6dxWj2/96QE1Bztl6/sn+ZAh1RYoyJVamu9EDg98lDPqC6dn7Cr
 kcUhJ3LhAwRNq1hhBBhUc1UQurwt63UYWWG6bJ8N8J+rGX2ohZPaagVuWsndRkzaq7obBe0O
 Be7hO9H2HNE0JJGh4dMaoStAowByaH6HLwJvdiFM4ITPvCdmOKBlRyChHJ8PUi2yyDAcollY
 P93lPpA6l5AUsxaIMKeHbt17FPS7nlWKZnvqXXHI/OPiuD2WZJoYe1dbAvmgh4RtfvZyOkqz
 zqvH5TTkEgOOAEPSgLW7ZQSPTg3wYsTXMutw/G7gtWre1I8cEl4Uqe56ep4J+RNwvQE/s+Vr
 yrVchIJlzLCaYjvdFzihoZLM+i0A/6SbBsTYEQRALpf8yJyOdrwtPhPKcRfkHtO3LUL8MOYh
 sItIq2oKv9OVi7G63Iaa5z8p5ZlbxOlmUSFOC/NXdT1V8cIq9Xh9oC2cw3x2jMJCybr58Iyr
 6f5jlHQQIYZRhQkB8HTMar9w1S0tHkbueRzQ0qXfYUDJBSyqNBne37rk/s6A8AQMhGflDGU4
 BmbXEUDru7Xro5rrNSQ3fKYr52kGvdVF1ZBGzWJ9q6/MCTXpzLxwYJJXOuSUyraUWf4pPera
 elPlqmuO/wbhlda9YF7Fu8zn6454tLuoZ5czxhlQyqXPwj6VOs4LyDfj8dVt6BLyrtIgieMW
 xqCqotAJLGEGML5C1pNdgArWfuOiKMPkT7I4PVrfEijvH1r/KCKWFl5NgWXjHAPN6N8NY4oz
 LtzuMMS7ADj2BMmPszf03JR/mWIaHcBT78mptcRB4qy0lgnzVRLYJr9DC7q4c7QN4UQYxdye
 jLE1rDfg7l8x1bZdythHHfA6uNRmJAStU0Y11QFPVmIxoLIi/JfMMe9KtjrotC5Fil67t8=
IronPort-HdrOrdr: A9a23:6lvD/6rbQAQqatjWlNMJSWsaV5txLNV00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssSkb6K290KnpewK4yXbsibNhfItKLzOWxFdAS7sSrbcKogeQVREWk9Qy6U
 4OSdkGNDSdNykYsS++2njDLz9C+qjFzEnLv5an854Fd2gDAMsAjzuRSDzraXGeLDM2WKbRf6
 Dsgvav0gDQH0j/Gf7LYUXtMdKzxeHjpdbDW1orFhQn4A6BgXeD87jhCSWV2R8YTndm3aoi2X
 KtqX262oyT99WAjjPM3W7a6Jpb3PH7zMFYOcCKgs8Jbh3xlweTYph7UbHqhkFxnAjv0idvrD
 D/mWZnAy1B0QKJQohzm2q05+DU6kdo15Yl8y7CvZKsm72ieNtwMbs/uWsQSGqm16NnhqAh7E
 sD5RPoi7NHSRzHhyjz/N7OSlVjkVe1u2MrlaoJg2VYSpZ2Us4dkWSOlHklYavoMRiKo7zPKt
 MeRv00JcwmBm+yfjTcpC1i0dasVnM8ElOPRVUDoNWc13xTkGpix0UVycQDljNYnahNBKVs9q
 DBKOBlhbtORsgZYeZ0A/oAW9K+DijITQjXOGyfLFz7HOUMOm7LqZTw/LIpjdvaNKAg3d83gt
 DMQVlYvWk9dwbnDtCPxoRC9lTXTGC0TV3Wu4ljDlhCy/TBrZ/QQFm+oXwV4rmdSsQkc7vmsq
 yISeFr6tfYXB7TJbo=
X-IronPort-AV: E=Sophos;i="5.85,307,1624334400"; 
   d="scan'208";a="53118422"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JCSPu50Gd8fJzEvK3P4ebEVZcl5nUUp/nRAWOZh09WDjLxb8q3qJzx01/C2Tar3gvoxSR+YwWM3q8d1Il6+eRsemVtp8T4nRnoQkayjjtPHYmBGu9hqqwC0XrANMO8aSpX0LL4hwN+tB6lsM6V80Y1uX4NyihlRRTTo3kOTFV2TONxkXxNGAo9WSBoloRJZY/JxWBZeON1v/xxzTVc/72GxVCeUjvFQKdWC7SKaDZqtqCNk8hk0NhhehCCaPaIJE1r4iAJbR5v3E1wV8sR6oRrpBKh5TvRil+GcWRnHh6mW0PiAl9F9MUYAsrlv70Qxn4g0JAciq51cJqHy90OOSZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fx1zSfA1R220O/aaGUHyRpJ2+0Ro7oxjPjSBnJ41RiE=;
 b=B7A0TQ7AW/3ybR6xSJe6oao2dPkMAQSS3pGXk8PsUAjz+b6Qf0d3PD2YbORB84mWnv5sMlmQiZQ2nYzUPs8J89k8m9HJ99XFZQSiGw1LiUAZItmFHxZTqo/wNFwd5iu2s4If3sjfrsCHTup3zJEeRu1kzNQUQ9Z1PpzLLMR6frBc67/95th4c3oEbHYpW/7QpgyBNYLrG2fHrxTmeE+BnbF14MZFQQ/A4obNwWf49GtsKTbFGOPW+TvC3vaXmATuJOeHXszdFR5Q4iCuagCl4iD1PumlUKOABfvKBQ1ZeH0DEstOpSSFfWc7P08KfrP5LXS/HD4hU+1+4RlxrnGP1w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fx1zSfA1R220O/aaGUHyRpJ2+0Ro7oxjPjSBnJ41RiE=;
 b=j6AFwBfPT+/YhImyUXv/AARtQSaCAgMlZocOXNvn0Of5OvjSeclYPkf/N2biatd2dTIEsSJs1k697NRl9VfSBifXiu1JLPKTD3NtT3qk/fSr4iY/W/vGeObIB20v2BwOG2OfQ81bzb72hLn/K3q3wAJf9QVr7VZ+3PulTBuJ98s=
From: Edwin Torok <edvin.torok@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>
CC: Christian Lindig <christian.lindig@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Ian
 Jackson" <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<Andrew.Cooper3@citrix.com>, George Dunlap <George.Dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony Perard
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 0/6] gnttab: add per-domain controls
Thread-Topic: [PATCH 0/6] gnttab: add per-domain controls
Thread-Index: AQHXq9s0IM3OAQqHo06+ALWIm01YaauoZEqAgAQlp4CAABBIgA==
Date: Mon, 20 Sep 2021 08:24:45 +0000
Message-ID: <BA298B54-07C6-45AE-AFE7-A9BF89C06D01@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <0D7E40CC-EA8F-4CE6-B4C7-A69C403BD4DF@citrix.com>
 <YUg3pc9A5C4GTTeR@MacBook-Air-de-Roger.local>
In-Reply-To: <YUg3pc9A5C4GTTeR@MacBook-Air-de-Roger.local>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d443d22e-b013-4891-7a9a-08d97c101ad6
x-ms-traffictypediagnostic: SJ0PR03MB5791:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <SJ0PR03MB5791A7E813E07706749A7DF39BA09@SJ0PR03MB5791.namprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 2IUZ1Jxd1x0c++rG0hqvYTGIP7Dzz8B7s1l/E+uam61k/I5ynqqKS31q4Cw0bo6Qq2U257V0BmVZ+DRcofgASbQUqx4qbmxglE3lJwlz/Qfbebytg7Czh9zSgciP0jkkgJ4av6CGLn4rp/oidopxAnepdFtUNoyGZM3e/a4n/hDJKE4t5cMhaT66dh67f03GdxdbcspGSaFb+OS3OIRcsHzQxsiWoTOW9pVgoEKM75Ixo6pjmw60UhkyFNWmK57Kz3t1HKO8vUYqt8HnswyorigDvjdaalxafb52xUw2MffqiVb5FlOOIYPLTQZJCQlTPMmU3m8tEE6rYj4IfKH9VOntB5P3oZiFM+Wwpbs3aqinO2eyPS1K95E0q1XXJDxzvGZVNCu94s5TYu2P8/A/yUVTFtCt+jeOUxbup9Qyj2Odbl9WNvnViqSxttXiyOjiMOwwA4IxSd1uyU6bVC03cKhu4AMHh1++iUsgXRFcC52trsAt+aJI2YE1fntyT7GOK6K3ifEWP0stDGpqTSIQ6l+USVy+Wqzp2nfOHoHmmSa3HTii5H7s3UCOjrrkwm3cO+yMs0GLbSqlnINGYZ2fAt9V0WH6XRKkh3KiNumiMrLdSd3uwB5X8CBrRBaL5/16HBD2/gFkCydBofaR2OjyYrRa/Hwpj80QhibWrPA9VsMvWy3fwB6pRkqk1BALk07fCMmE3eR3lO8vZbhBpSf/6WzabnYaQRxNxlv/wdAVJ0elIyV2d9W1ckkaD4TICBNVJSZ9ZI4DmZO+5p5vZoQiY/4IHiUNaB1hvtGEieFNhshmTFHWGSnSnNVPc1lARHPY
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR03MB5888.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(366004)(346002)(39850400004)(136003)(396003)(38070700005)(4326008)(6862004)(122000001)(91956017)(2616005)(36756003)(76116006)(6636002)(5660300002)(6512007)(66446008)(316002)(33656002)(71200400001)(66556008)(8936002)(186003)(54906003)(38100700002)(37006003)(66476007)(6506007)(966005)(26005)(8676002)(478600001)(53546011)(86362001)(2906002)(6486002)(83380400001)(66946007)(64756008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?bnP2gKlUQiSDhxbSrriJiNthMqKMBJZuhs/Ic83OAe0sLS2maNve3KyTlVoi?=
 =?us-ascii?Q?rgzkO4DXb9xcOyI8ugT5N7nc6hIg9EbZEfnLQt3m5PXQWuCPgi4IA9ApjqaZ?=
 =?us-ascii?Q?saVPcuGp9bor2ubIRlXLE3xSElww6dTw6te4ORdOMbEy6V64SbKc5+8p0fX4?=
 =?us-ascii?Q?fvchNmOIKRA0ob+gUqCDQS0lwYIs/pTGNtSV/hOlffM5OuY1L77Z1tJWM83U?=
 =?us-ascii?Q?C40S7Q6Uvgp/2hSP8MMfTcHXcSGcSxc7/rFX8WlmKM7bqRviOaZcbw6UwILc?=
 =?us-ascii?Q?s8Yh2sEzhPL7CNZBeLbALqGAUSG/RgGDMkzxcFyb/7TezFPGx8Fw39QAHP4z?=
 =?us-ascii?Q?9ppYUSq8iC2Q+9btN2lbnksB/YQqlI/2FnHrWDMPGNV9Cr4odniuUT+OMu3c?=
 =?us-ascii?Q?Jgl6Cq0xJYaGV/99UAtfXomQE/s6XLZb24zNPakhzYyZ7pDtlsRnwIfuodIp?=
 =?us-ascii?Q?RAoEjpqxWYeB1zLIc8sx7ciwnqt8g8+BoIWp2xDaBdltJMfMdCdy++qL/mU0?=
 =?us-ascii?Q?EliAvwcTT5S5jUzbQT54ISQ88hNzVZhQOr2v0RmX4jX9wgLCYmsVYRNlBX3t?=
 =?us-ascii?Q?OW1+Df5aQcNvz2WoX6ZXPQ0YRTd3EDd0pxNDBTw8vAO9Vc1BJnVtB9FdPJ/y?=
 =?us-ascii?Q?kOpjSh4Ma/MrN4zIF6eq6iPbqY/MU/Im0OgZBJLaUZeimp8MwDv3y2m4XCIz?=
 =?us-ascii?Q?QwjDG/FFAlgU+od5QfHsYIp34RaX/TtF+ATy9xzOdOS8OtS4Dx3l+zvoQtXw?=
 =?us-ascii?Q?9idViiu/YQJUp/RRo6IqtFiCzBSy/uk0nZ0XzS1JSDKCsS8P1LpqBzaDEXS3?=
 =?us-ascii?Q?8hYLgII0is2Q2lfMNuhYn3JqN3GI1hfYR5OQ0YyM1kZdScEKm/iMXeLMv0gR?=
 =?us-ascii?Q?rsbdyG+Jhv/WSURCXjzCmjkXWgvykeL0MekCGkRjXZMNdeIDzWYzXK+6ma8E?=
 =?us-ascii?Q?94zvpEUVAwE/FzFaKVw/JAp+An77asywAH1LwzVk8FoWyzZQUsdpqUT+qR8F?=
 =?us-ascii?Q?RvlBvlmnuS9oFJD8KKP/KcHcuLtHuHOtLTfbyErwgS/xXX3OMQntn57d8oE0?=
 =?us-ascii?Q?PaovuSd+bO9DcSA8x9swXoWMq9Hifaw5RuIjEorNgNRmgQuR25tm03gI0Z9Y?=
 =?us-ascii?Q?v1kam1DFzwE05rRgq2w3OMMWAdSEy5EihRdrpyX+Ap+CgTx7PRGgoisP4Z/I?=
 =?us-ascii?Q?4UL3ULNqyneE9/Jp5s/ZtbvVIhEw8+LLU2MkafoCjhYUt/eFKSRBYgxV7g1W?=
 =?us-ascii?Q?NlURCKPPA6YJItoKwvEtIV4UremlpBK/lfE7DD1BH3hi+8kw6XH2YOWYnHcw?=
 =?us-ascii?Q?4QktV359p6GynK7obX3FHr51?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D0E0A553EBDB3F409B551E57A9A9D767@namprd03.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR03MB5888.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d443d22e-b013-4891-7a9a-08d97c101ad6
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2021 08:24:45.9279
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: EbS2D4tku1afMP2vLSOuOSuC1O+8uBIlDc+GukccHavPJ0LG4uNJ1u07uCPJetIJR9/X6oolxTFagm5GCky0Jw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5791
X-OriginatorOrg: citrix.com



> On 20 Sep 2021, at 08:26, Roger Pau Monne <roger.pau@citrix.com> wrote:
>=20
> On Fri, Sep 17, 2021 at 06:06:42PM +0200, Christian Lindig wrote:
>>=20
>>=20
>>> On 17 Sep 2021, at 16:46, Roger Pau Monne <roger.pau@citrix.com> wrote:
>>>=20
>>> Hello,
>>>=20
>>> The first two patches of this series allows setting the preisoutly host
>>> wide command line `gnttab` option on a per domain basis. That means
>>> selecting the max allowed grant table version and whether transitive
>>> grants are allowed.
>>>=20
>>> The last 4 patches attempt to implement support for creating guests
>>> without grant table support at all. This requires some changes to
>>> xenstore in order to map shared ring using foreign memory instead of
>>> grant table.
>>>=20
>>> Note that patch 5 will break the save format for xenstore records, and
>>> should not be applied.
>>=20
>> Has this relevance for the format used by oxenstored?
>=20
> I'm no expert on oxenstored, but I think it has always mapped the
> shared ring as foreign memory, and hence no changes are needed there.
> AFAICT it also stores the mfn on the save format, so I think this is
> all fine.
>=20
> Should have mentioned it on the cover letter.
> =20


There is a patch series from last year to make oxenstored use gnttab instea=
d of map_foreign_range.
https://patchwork.kernel.org/project/xen-devel/cover/cover.1598548832.git.e=
dvin.torok@citrix.com/
This got lost/forgotten amid all the oxenstored XSA work.

Later on I discovered and fixed some bugs in it, and is part of this refres=
hed patch series (part of which got committed, part of which didn't):
https://patchwork.kernel.org/project/xen-devel/list/?series=3D480623
https://github.com/edwintorok/xen/pull/2

I think the current status is:
* there was an objection that the commit vendoring the external dependencie=
s for the unit tests was too big, and should be replaced by just an opam an=
d lockfile telling 'opam' or 'opam monorepo' where to download it from
* I've discovered some bugs while testing this code together with other cod=
e, and need to retest with just this code alone to check that the bug was n=
ot in this code


As for the save format, that is part of this patch series too, and we don't=
 store the mfn anymore. Do we need to go back to storing the mfn?

What do I need to change here? The reason to move away from foreign memory =
was that we could avoid relying on xenctrl for that function (and thus havi=
ng one less unstable interface to link to). If we need to conditionally use=
 foreign memory mapping then we're back to using unstable interfaces, unles=
s there is a stable interface equivalent to mapping foreign pages?
I see there is a libs/foreignmemory (it has no OCaml bindings though). If w=
e wrote OCaml bindings would the API/ABI of libs/foreignmemory be stable?
In which case we should probably replace the commit introducing the use of =
gnttab with the one using foreignmemory and always use foreignmemory instea=
d of gnttab libs.

What do you think?

Best regards,
--Edwin





From xen-devel-bounces@lists.xenproject.org Mon Sep 20 08:33:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 08:33:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190335.340174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEjX-00073H-6D; Mon, 20 Sep 2021 08:33:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190335.340174; Mon, 20 Sep 2021 08:33:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSEjX-00073A-36; Mon, 20 Sep 2021 08:33:03 +0000
Received: by outflank-mailman (input) for mailman id 190335;
 Mon, 20 Sep 2021 08:33:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2sKa=OK=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mSEjV-000734-9P
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 08:33:01 +0000
Received: from forward501p.mail.yandex.net (unknown
 [2a02:6b8:0:1472:2741:0:8b7:120])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1029640b-b823-403e-a326-7f5d54bfb2d1;
 Mon, 20 Sep 2021 08:32:58 +0000 (UTC)
Received: from myt6-9503d8936a58.qloud-c.yandex.net
 (myt6-9503d8936a58.qloud-c.yandex.net
 [IPv6:2a02:6b8:c12:4684:0:640:9503:d893])
 by forward501p.mail.yandex.net (Yandex) with ESMTP id 1C17C6211E86;
 Mon, 20 Sep 2021 11:32:56 +0300 (MSK)
Received: from 2a02:6b8:c12:379f:0:640:4204:cefb
 (2a02:6b8:c12:379f:0:640:4204:cefb [2a02:6b8:c12:379f:0:640:4204:cefb])
 by myt6-9503d8936a58.qloud-c.yandex.net (mxback/Yandex) with HTTP id
 sWXJU75E2Os1-WsEaVVJn; Mon, 20 Sep 2021 11:32:55 +0300
Received: by myt6-4204cefb5b39.qloud-c.yandex.net with HTTP;
 Mon, 20 Sep 2021 11:32:54 +0300
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1029640b-b823-403e-a326-7f5d54bfb2d1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1632126775;
	bh=NPhA/tCoKEgQZpIfT+8QCx4jzUUVo2p/wBjLlJqAFYc=;
	h=References:Date:Message-Id:Cc:Subject:In-Reply-To:To:From;
	b=RiLguzLPgcHdqJnooj0QUZC+aoWTpLlSrNw33p/SoTDKxiwxAMkU7ZRaE9WibzZ9G
	 f1f99MzcVPQvmpBHkuCwOU+k2aVFDlow1qApeVhNFl5HfvwVrHMotJHRjzOffeWyVn
	 YtCnFqajEflRklCuw86cE9BG/54h0WwyPhJeX7eU=
Authentication-Results: myt6-9503d8936a58.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaikin <isaikin-dmitry@yandex.ru>
To: Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"rjstone@amazon.co.uk" <rjstone@amazon.co.uk>,
	"raphning@amazon.co.uk" <raphning@amazon.co.uk>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
In-Reply-To: <c242e3cc-d4ed-73b7-8e38-f94a8c479d20@suse.com>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
	 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
	 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
	 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
	 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
	 <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
	 <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
	 <f6225dc6-0590-3456-8c48-7ab29aa00200@xen.org> <c242e3cc-d4ed-73b7-8e38-f94a8c479d20@suse.com>
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
MIME-Version: 1.0
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Mon, 20 Sep 2021 11:32:54 +0300
Message-Id: <12214401632126774@myt6-4204cefb5b39.qloud-c.yandex.net>
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=utf-8

SSB0aGluayBpdCBjb250ZW5kIG9uIHBhZ2UgaW5zZXJ0aW9uIGludG8gZnJlZSBwYWdlcyBsaXN0
LiBCdXQgSSBkb24ndCBjb25maXJtIG9yIGRlbmllZCB0aGlzIHlldC4gSSdtIG5ldyBpbiBoeXBl
cnZpc29ycy4uLjxiciAvPjxiciAvPjExOjE5LCAyMCDRgdC10L3RgtGP0LHRgNGPIDIwMjEg0LMu
LCBKYW4gQmV1bGljaCAmbHQ7amJldWxpY2hAc3VzZS5jb20mZ3Q7OjxiciAvPjxibG9ja3F1b3Rl
IGNsYXNzPSIyMTBlN2E4NDhlOGZjYjQ1d21pLXF1b3RlIj48cD5PbiAxNy4wOS4yMDIxIDE4OjAx
LCBKdWxpZW4gR3JhbGwgd3JvdGU6PGJyIC8+PC9wPjxibG9ja3F1b3RlIGNsYXNzPSIyMTBlN2E4
NDhlOGZjYjQ1d21pLXF1b3RlIj7CoEkgd2lsbCBxdW90ZSB3aGF0IEhvbmd5YW4gd3JvdGUgYmFj
ayBvbiB0aGUgZmlyc3QgcmVwb3J0OjxiciAvPsKgPGJyIC8+wqAiPGJyIC8+wqBUaGUgYmVzdCBz
b2x1dGlvbiBpcyB0byBtYWtlIHRoZSBoZWFwIHNjYWxhYmxlIGluc3RlYWQgb2YgYSBnbG9iYWw8
YnIgLz7CoGxvY2ssIGJ1dCB0aGF0IGlzIG5vdCBnb2luZyB0byBiZSB0cml2aWFsLjxiciAvPsKg
PGJyIC8+wqBPZiBjb3Vyc2UsIGFub3RoZXIgc29sdXRpb24gaXMgdG8ga2VlcCB0aGUgZG9tY3Rs
IGxvY2sgZHJvcHBlZCBpbjxiciAvPsKgZG9tYWluX2tpbGwoKSBidXQgaGF2ZSBhbm90aGVyIGRv
bWFpbl9raWxsIGxvY2sgc28gdGhhdCBjb21wZXRpbmc8YnIgLz7CoGRvbWFpbl9raWxsKClzIHdp
bGwgdHJ5IHRvIHRha2UgdGhhdCBsb2NrIGFuZCBiYWNrIG9mZiB3aXRoIGh5cGVyY2FsbDxiciAv
PsKgY29udGludWF0aW9uLiBCdXQgdGhpcyBpcyBraW5kIG9mIGhhY2t5ICh3ZSBpbnRyb2R1Y2Ug
YSBsb2NrIHRvIHJlZHVjZTxiciAvPsKgc3BpbmxvY2sgY29udGVudGlvbiBlbHNld2hlcmUpLCB3
aGljaCBpcyBwcm9iYWJseSBub3QgYSBzb2x1dGlvbiBidXQgYTxiciAvPsKgd29ya2Fyb3VuZC48
YnIgLz7CoCI8YnIgLz48L2Jsb2NrcXVvdGU+PHA+PGJyIC8+SW50ZXJlc3RpbmcuIElzIGNvbnRl
bnRpb24gb24gdGhlIGhlYXAgbG9jayBtZXJlbHkgc3VzcGVjdGVkIG9yPGJyIC8+d2FzIHRoYXQg
bG9jayBwcm92ZW4gdG8gYmUgdGhlIHByb2JsZW0gb25lPzxiciAvPjxiciAvPkphbjxiciAvPjxi
ciAvPjwvcD48L2Jsb2NrcXVvdGU+PGJyIC8+PGJyIC8+LS0gPGJyIC8+0J7RgtC/0YDQsNCy0LvQ
tdC90L4g0LjQtyDQvNC+0LHQuNC70YzQvdC+0LPQviDQv9GA0LjQu9C+0LbQtdC90LjRjyDQr9C9
0LTQtdC60YEu0J/QvtGH0YLRiw==


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:05:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:05:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190375.340209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFF6-0002WU-W4; Mon, 20 Sep 2021 09:05:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190375.340209; Mon, 20 Sep 2021 09:05:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFF6-0002WN-Sa; Mon, 20 Sep 2021 09:05:40 +0000
Received: by outflank-mailman (input) for mailman id 190375;
 Mon, 20 Sep 2021 09:05:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSFF5-0002WH-Sz
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:05:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ebb118e3-19f1-11ec-b866-12813bfff9fa;
 Mon, 20 Sep 2021 09:05:38 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-qpL20eRVMDuAkqAgV67LvQ-2;
 Mon, 20 Sep 2021 11:05:36 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Mon, 20 Sep
 2021 09:05:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 09:05:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0024.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Mon, 20 Sep 2021 09:05:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebb118e3-19f1-11ec-b866-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632128737;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=x1gRd4rVTbYh/if/ZLwT2eaARXO4ZWVJhsrxlMf4eyw=;
	b=U7bxCq3xI9yHVacqC8OQY0fAJGxvbNWbgjF+/otWxVJ9SZsIICPBThGgJt9PDNHE+Ozcp2
	5ciaF5MYk7oK3I+tXBS6SZdiYq5WlsLJ4Cqam6HDh1cdPN7x3qzyu+1oKCZ+uACnlRdxQo
	tcdpZcL5F4eJ7+XVBc8+Hat0iE6g5S0=
X-MC-Unique: qpL20eRVMDuAkqAgV67LvQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dA+cnksg6+iorDGNfEEK9px7MAF+imw3CZIw+T8eYyQfqbTL27RvXmnmwuA3iPRMA4UniELGZuMIm2EXpc7LQlK9zRyXjphdiBrDqjQoB6JV0uCui9Oio93UiBIKR7iHiUoCyZZgqJC6/o2c7mqmUdEG0D6UMa1K5DMXelVBq0fmrXI+ZJybeMs9ChvolIxTHvxkK1TGTNtjBXZjGRlkPFM12rO08n1i0BJUKlr3faAytQKCFzQynxMbyTEMBJSuvMa9lJFQuMEl+P3+s6E4rMSfIlk70l/rK8lNko6Y35uQAtO1KJA6U1CJv5zPHoY9PWmnQ9jF/SJjDHaCBJHqPw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=x1gRd4rVTbYh/if/ZLwT2eaARXO4ZWVJhsrxlMf4eyw=;
 b=fysGRk6tFU5OkwCJ2wZVQfTnFeBG60CX+or0i71JmObAB4TLT5If4OUuKOOhR0uMAcagolP2bEfTdEQRKgKE0+eovpalgU2eAnOAhpgJjrwJNWCWHLKVUI4FPXYx7U8e7Bvx8xNnxitCjeYr8MPW2jDRqlFG8UcgbGMVikoDm1gwa+2NMnVw8TvU/8ZH9YdW5ARGPUfO96OJMfscAjqblLnA2/2ubvWdueAPGo6EEUdwmArFrmd+jgoaCV03eqDi+1B98hi+hnyRTZAEL9xrYFuSb3k5IU69AwlSjJoFX2ZAOwgrnJHGTvsF/Sg/0XT6b/n61DIAuxcuJUtErifHFg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/6] x86/trace: Reduce stack usage from HVMTRACE_ND()
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ef2df9df-df4a-8f8a-3f69-88a027ba66eb@suse.com>
Date: Mon, 20 Sep 2021 11:05:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0024.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6ffd54a4-4ece-4c8e-fe61-08d97c15cd6f
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41905DD3E3F22435672C96BFB3A09@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MEc6eWKwyLPsdldVOfMde3Ht+4a+kRcMxELbhsEW02XBRIR073UI6P1CAkvmy5IJ3uaj5+CgCjBd5g/YY/7lirDMmxzhTkEQfIbq3ojIsEItnLTsqunnyUUuvFpuBbQfHVQW4UYLsuChFv+kVna3wppSe/1WG31OdwMr6nyM7WxxKhg+q9RiFPKOnwZWqmmkNKx/ibRb6adZJGsjK4IH4pmvAjF8JOzhSsBbhPlRl3aKofKwsGENC2+cFNGMClnVha21q5ezLEfuHoZC3QEqv7TR+mD/edb3S4of+9PrH/KP/PNwq+EUNltL3q4Gqhw3kDyr1JV6gcbpRZAOFhIKTlbTeJ1LN6QCIqIVUDqd+7vtz/dNGdBOmi3VZqz9lIB+U+mXZBjG0B7/SgfNBob0CYwjUrYRYTQBt32czVJeknKH1GNXV3zuoiuvZcFaruQmoCGnLjbxynxvHB7vm9cLjnvm1eAZORpZOxAS+8lS2Eye8ZCfSTp/la6ZrI/Pop4bBkqEpLa9PxntsVhQ2zI4Wahqbh3GXnQlXw6X1NQYw0ymZoT0jrwFCX5M0CPRQslZ2HStw/UayA/G/DX6XfacklTGeNKmKsYSt2n1fWgwVGHI6YL38CXd8NKGLDVd930H4T169aRv28vs+owWNupI6ncp71ccmTbjb6sVzYz9chKgljGaSIZzNvdVO8ZZvaCP/DOrlV71LoXDLbFA0smFqbu22NTxPJuXcTuLSXsA57g=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(366004)(39860400002)(136003)(376002)(396003)(6916009)(38100700002)(16576012)(5660300002)(36756003)(2616005)(956004)(478600001)(6486002)(2906002)(26005)(66476007)(83380400001)(66556008)(8936002)(186003)(66946007)(4326008)(54906003)(86362001)(316002)(31686004)(53546011)(31696002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UlUzZHNsa0VwRDBVVXhCeU5ZN0wwRVNZRzZIYmhnMlp4S0tYUFNudUVtSHpD?=
 =?utf-8?B?eEp5ekt1SGZWVEJ5T3l2WkZERXlzWE5vTlducE02NTJNMTU2RTRGMjQ5VWg0?=
 =?utf-8?B?ZjlpTDFMS09WMEZuSGc1SFVrRUFTRmVBUmFqNlpJQXdJR0FZK0duS0lkb2sv?=
 =?utf-8?B?RitDSGZRSmUwTkpCdjUrT1VOMW9uZDRxVWZQZUc2K0FkZ1B3emw3OGpCejhq?=
 =?utf-8?B?cGE0YzZ5L2toUmE5c3ZQZkZnaGlwNHVHUERKVDBRdlI4bXk0WG9teDgrRGpo?=
 =?utf-8?B?bm9xdWFKOHBCeHVkQWFYWGdMcys2WEZaMFBOV3R6NWlZOW9XN1Y4RDZ4TWpy?=
 =?utf-8?B?cGIzanYydjUvc0ZmSU1RbzZJVWJuSS9IeWxiNHdkbDlSMUhBbUlCL0xta1dI?=
 =?utf-8?B?WXdtRXdaRk1PbkFKcUJreVl0MkZqZFh3VnM3L1R0UWp1YTdwM0lFU0tnQ0Rz?=
 =?utf-8?B?OUY5anRJSFJ3M29XOXhhUVVTdjVXRWZYenNNRGdVTXltTzc4U3Joa2Z6Tmdq?=
 =?utf-8?B?ajZPbk1MdDB0WU9jVTBXMUVXR1ZzdXJrMm9UZDUrYmJXTERSYXdFdUpzSWxC?=
 =?utf-8?B?aFhHVjlYbnpoTlBjdVhTdjFrUDF1UVJSMk4yV1Y4cFJtZlRYTXR0N2dQREtn?=
 =?utf-8?B?K0h5SFBBcnFuTXVSeEJiNkpvcHVvbjRncVRiVTNaQkVIaDltVWNQWUNhdjdw?=
 =?utf-8?B?WXV1NHpCSXZqc3FmTktqeWNpZmhpWkxteitzcXYyanZYNHRqeExIR1hvRVFC?=
 =?utf-8?B?dHViaWpDTzIyanlZUkljL3VNR3F2aitEemVXc1lxSThURFJsRWVhUnJ6SGhi?=
 =?utf-8?B?S2dPYjJnSUdNa1A5NEw2WVdWOEI4Tk1oeHdlN2NVVFliSkhaNXcwMnE4anNa?=
 =?utf-8?B?UHdJZkEvbCtlZWxxdzVhQ2lueTVVVERYKzBEcmVtWmtaUkNFTXgzWHZNTVlP?=
 =?utf-8?B?cGFzMCtoK1AzcFFvNmw5NTM2dGd5dDIrWkMzMElaUERFZ3d3T1VJenZMc1Yv?=
 =?utf-8?B?VFhpMVVQMHREd3NZd09QbmwveUFJV05hd2lJL1NwbDZmcVhkRkNQQ3RreEMy?=
 =?utf-8?B?MTVQc0VqV2lDUTJEN0ROYTVjcS81MHQ1bGhHL0I1cDVibk9FcEJoU1lSL3RX?=
 =?utf-8?B?NGY4OHdFN1FhaS85QU5XQ3JWMzZBa3B0MlZ4aUtDdGxRTTc5YkpRdTl0S3gz?=
 =?utf-8?B?N2hkMjY5WWRUQWJ3NVRBa2dTTm9nelpmMGtLZGY0YzRyTGE0Q1dsTGFXM2pB?=
 =?utf-8?B?ZVVwMGw2OVFRaXluNGUvSVY1UFBjVFExdnArRTloNFZ0OGxXaFVySzhFZ3h6?=
 =?utf-8?B?cWJ2UU5yQStPM1BZamdPallUVmtkaisyWDhPYXhBZjQ3TzBlZm52RTZrV0Jm?=
 =?utf-8?B?VlJKNE0xZXFvT3R2YWtBVzNLK1NaK3JYSStaVHAwRUZCeWZpZGNYZDIxdlBZ?=
 =?utf-8?B?S1F1M3E1MGY1MHMrdVNubkdXalRhdmNCcThIYWFQL2VLTzFJSHFlMHVhL1Za?=
 =?utf-8?B?dTR4L2Y3aE5aeUdlbGo5UTNlTVVhaHJjUzE4dTdRUDlqQVJVc1QvNktDM0Uw?=
 =?utf-8?B?dXZsUi9yS0RhNi9kNTVRam0wSGhRTWtQYVAwczk1eXZ2MUFMU3QzTzVIMy9m?=
 =?utf-8?B?MHh5LzRkR0Q2N3VIbFE5MEZkZ0daeUZsMG9uR0RJYlgxYUlDRDk2R1FLblFS?=
 =?utf-8?B?V24xbmdpdjhEMnQyaWpPdCtIOEhFSllXcm1za1FEaFIvd1N1TGRUTGM3ZVV1?=
 =?utf-8?Q?dIFP8b2dvVhIyh8uuPSfaHuB5AlTfPnLYNys6M7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ffd54a4-4ece-4c8e-fe61-08d97c15cd6f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 09:05:33.3581
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NBGPNkjBck0ULsnk4Ep71TJjXNTssemEGcgJQGYlrTHj8SaoLvk1/Qt4INgFBMvrkaLZDwDrGXDgTHiJ6HPvzQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

On 17.09.2021 10:45, Andrew Cooper wrote:
> It is pointless to write all 6 entries and only consume the useful subset.
> bloat-o-meter shows quite how obscene the overhead is in vmx_vmexit_handler(),
> weighing in at 11% of the function arranging unread zeroes on the stack, and
> 8% for svm_vmexit_handler().
> 
>   add/remove: 0/0 grow/shrink: 0/20 up/down: 0/-1867 (-1867)
>   Function                                     old     new   delta
>   hvm_msr_write_intercept                     1049    1033     -16
>   vmx_enable_intr_window                       238     214     -24
>   svm_enable_intr_window                       337     313     -24
>   hvmemul_write_xcr                            115      91     -24
>   hvmemul_write_cr                             350     326     -24
>   hvmemul_read_xcr                             115      91     -24
>   hvmemul_read_cr                              146     122     -24
>   hvm_mov_to_cr                                438     414     -24
>   hvm_mov_from_cr                              253     229     -24
>   vmx_intr_assist                             1150    1118     -32
>   svm_intr_assist                              459     427     -32
>   hvm_rdtsc_intercept                          138     106     -32
>   hvm_msr_read_intercept                       898     866     -32
>   vmx_vmenter_helper                          1142    1094     -48
>   vmx_inject_event                             813     765     -48
>   svm_vmenter_helper                           238     190     -48
>   hvm_hlt                                      197     146     -51
>   svm_inject_event                            1678    1614     -64
>   svm_vmexit_handler                          5880    5416    -464
>   vmx_vmexit_handler                          7281    6473    -808
>   Total: Before=3644184, After=3642317, chg -0.05%
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> Normally I wouldn't recommend patches like this for backport, but
> {vmx,svm}_vmexit_handler() are fastpaths and this is a *lot* of I-cache lines
> dropped...

The change in size is indeed unexpectedly large for these two functions.
However, what I find puzzling is that TRACEBUFFER is enabled by default
(i.e. also in release builds) in the first place, and that it can only
be disabled when EXPERT. More gains could be had towards dropped code if
the option wasn't on by default in at least release builds. "Debugging
or performance analysis" (as its help text says) after all isn't a
primary target of release builds.

IOW what I'd prefer to consider a backport candidate would be a patch
changing the option's default. Thoughts?

> --- a/xen/include/asm-x86/hvm/trace.h
> +++ b/xen/include/asm-x86/hvm/trace.h
> @@ -67,38 +67,30 @@
>  #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
>      TRACE_6D(_e, d1, d2, d3, d4)
>  
> -#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d6) \
> +#define HVMTRACE_ND(evt, modifier, cycles, ...)                           \
>      do {                                                                  \
>          if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )               \
>          {                                                                 \
> -            struct {                                                      \
> -                u32 d[6];                                                 \
> -            } _d;                                                         \
> -            _d.d[0]=(d1);                                                 \
> -            _d.d[1]=(d2);                                                 \
> -            _d.d[2]=(d3);                                                 \
> -            _d.d[3]=(d4);                                                 \
> -            _d.d[4]=(d5);                                                 \
> -            _d.d[5]=(d6);                                                 \
> +            uint32_t _d[] = { __VA_ARGS__ };                              \
>              __trace_var(TRC_HVM_ ## evt | (modifier), cycles,             \
> -                        sizeof(*_d.d) * count, &_d);                      \
> +                        sizeof(_d), _d);                                  \
>          }                                                                 \
>      } while(0)
>  
>  #define HVMTRACE_6D(evt, d1, d2, d3, d4, d5, d6)    \
> -    HVMTRACE_ND(evt, 0, 0, 6, d1, d2, d3, d4, d5, d6)
> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5, d6)
>  #define HVMTRACE_5D(evt, d1, d2, d3, d4, d5)        \
> -    HVMTRACE_ND(evt, 0, 0, 5, d1, d2, d3, d4, d5,  0)
> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5)
>  #define HVMTRACE_4D(evt, d1, d2, d3, d4)            \
> -    HVMTRACE_ND(evt, 0, 0, 4, d1, d2, d3, d4,  0,  0)
> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4)
>  #define HVMTRACE_3D(evt, d1, d2, d3)                \
> -    HVMTRACE_ND(evt, 0, 0, 3, d1, d2, d3,  0,  0,  0)
> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3)
>  #define HVMTRACE_2D(evt, d1, d2)                    \
> -    HVMTRACE_ND(evt, 0, 0, 2, d1, d2,  0,  0,  0,  0)
> +    HVMTRACE_ND(evt, 0, 0, d1, d2)
>  #define HVMTRACE_1D(evt, d1)                        \
> -    HVMTRACE_ND(evt, 0, 0, 1, d1,  0,  0,  0,  0,  0)
> +    HVMTRACE_ND(evt, 0, 0, d1)
>  #define HVMTRACE_0D(evt)                            \
> -    HVMTRACE_ND(evt, 0, 0, 0,  0,  0,  0,  0,  0,  0)
> +    HVMTRACE_ND(evt, 0, 0)

These HVMTRACE_<n>D() aren't this much of a gain anymore; perhaps down
the road we will want to have just a single wrapper macro adding the
modifier and cycles arguments, otherwise making use of variable
arguments as well?

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:11:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:11:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190412.340231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFKc-0004Ev-QV; Mon, 20 Sep 2021 09:11:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190412.340231; Mon, 20 Sep 2021 09:11:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFKc-0004Eo-NX; Mon, 20 Sep 2021 09:11:22 +0000
Received: by outflank-mailman (input) for mailman id 190412;
 Mon, 20 Sep 2021 09:11:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSFKb-0004EN-Bo
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:11:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b46554ed-19f2-11ec-b866-12813bfff9fa;
 Mon, 20 Sep 2021 09:11:14 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-19Ol8OHHNF6txHZRVlkRYg-1; Mon, 20 Sep 2021 11:11:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2608.eurprd04.prod.outlook.com (2603:10a6:800:4f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Mon, 20 Sep
 2021 09:11:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 09:11:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Mon, 20 Sep 2021 09:11:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b46554ed-19f2-11ec-b866-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632129073;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZQ14YNKsbXfYrrub9l0XRKId0d3U7KU+ivHIckNGKJ4=;
	b=HhC3LNP3lwl9js6Fuz9T0bQF/fhbjDuqFZFOGCZgfqoPVqr7TDMx1yrWdW2e6/xnrp/Yo1
	ua2Hc5Cqs8ROhysyNqvpQpNp6htVXCrAiIbRhP75qTzt2GS9FxIJymdBuIRUAOr1Ey/SV0
	m6xvpdyJwmb57B06zvCM0fCpeAJrk80=
X-MC-Unique: 19Ol8OHHNF6txHZRVlkRYg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WktPLMPgIBogDq6XhyIlKH7Gp3XIkpDAVIoYLI21aGQcCuwPUqbRzkDgML/Onx2qpiEwHsB2VRyhdFThJJL3SJTmY0pU8qdy5AMTaSo7RSytiWoVlHRjNrFt+spxHQi8Kuv6MvWN5GqxxMxC4YmOnmkOdFTznbheKLcPUomIg502ncw0dNQAXaBTHf6Evm2bYxbtdykssDtff5gwU095XyGZTBa7LzX6H2qLKlenU16lDsEsJMxPyTrhI5Na2FnQep0S3QOx8sBlTsUGoYnR8ZAxJbxUAO23capv45ij238ydAlpJZTx+lPobuFtQhz9JXyHCTXoKSNo9L4chJsAuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZQ14YNKsbXfYrrub9l0XRKId0d3U7KU+ivHIckNGKJ4=;
 b=kwm51En0PbTJpdBQ3cfSeZU31wCYn7XJguTQAZ2sToCesFE3RaedbGcmp6mIx93hKWt/dE6yn0pXLmUG9oWVdp8dV0PlLVkvtqJOIb+XWcBQcF7IzyJQhXhg2PmJ21jWbtt3shOLkLEm1ejWYUmig2X6bAJkJRHq3ifRF1LimWIHMJ1hC/OeMk5Q55II1lHSj4q02XCdwZbywaAAKZjYPSu0vxnVi1lFw8JlNEYl6xS8VvZGc7iwJ2VqLJaFayeoHZFY+8cvKZfqVEend+pnrG03JuP02SeYMlOhJGoUiDpk2wSqtcUsblnfPaobUMr5HulyU1eWExVvpI/qEGizGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 5/6] xen/credit2: Clean up trace handling
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-6-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e93db022-169d-e58d-84fb-c5d7edd7da06@suse.com>
Date: Mon, 20 Sep 2021 11:11:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-6-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bef3ee10-87d2-46b4-e196-08d97c1696e0
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2608:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26080740B3EAD426EE3A9C58B3A09@VI1PR0401MB2608.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PrdjB8MZ+7e/CMUTolZFLohrq4ZmWLnK09ZCF1SxJ7eBVBxLrf9jTdbI95s2KCb1qaNuaK3BtXM5f8y5ddfIbTvVCO2udxF/YokUvSrx/DtkHI76VChlq65KLiHcxfsfu+Z6hxdy0fYiJXQ/69cqV7YJSKAq4HEVPiaE+X8PLb3ox3MnqdlUzmIMFbB5mtfECl4uHM+WXFcIWCAvHtf4TgWsjMPVB1B62fLgPWA5Uw4nZPUt8HAt/m+rbaHDfSJTJRuohdwdDNwGPC+qDyzUDKeJI7Lts+2bp72SWZe9W2Lu9US6NsIZpQAhP0YM9gfa726ZLRbKoRzRlgSnB3EQJCoetZ0+8AiP4fwRXOIYLAtsoCJSOyMBVZiDCP1eoOMzklUggkdvaRrbiHUN0QM89Dd3N6MTC1MqSDKSQHivx12H+GBf1++uRzosXVNxXW/19Xnyk2/LgsqfTEx2EYO4ELfl9sa8jlgjky87qXoKwjKc1xgtUxwLfzJvSwHX/poN57qgHLDUzA140i25yRQkT3V3mDJm98uYCh8Q7SzRBBQRiOqpNq/HbnLkzlp6a8KCbWYsT9uYc5/GYYpaprSdvdWxzcr/nDx6UimmgWmitYDN2JKrU5FuzTSe2bG8NaGO2HFlAamx3rmV6oEXHrvpvFxG/OebLQXnPYjzGwxcfC7oLryqVNHDjQxpnxSNhO/MBiIpWWPLlfjsPT58r8PYzFDmynDu1ayf5DanEHZ8VCA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(53546011)(2616005)(54906003)(66476007)(26005)(186003)(956004)(66946007)(38100700002)(66556008)(8676002)(316002)(8936002)(6916009)(5660300002)(16576012)(2906002)(86362001)(4326008)(31696002)(6486002)(36756003)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OVRsQm5CSjl6VFJTUEVCRG1mYlFOSTdQOTRrVktDYW1xL3VVd21xRURBSVNE?=
 =?utf-8?B?a2xodlplKzhaUmlheWdxa0tNcS9LeHNOUThGVFJEemtPNkw0Q3pHSTVUaW5s?=
 =?utf-8?B?Y0NpMnh3UFkzaW95RDJKNVBZbDdqK2lWSDZScEZjVGxIWEFDYmNkdHhsY2ha?=
 =?utf-8?B?VWg5eUw4cVhVRUUrdXp1UENmSmI3Umhka01oSmUxaDlCRmkzaUxqdlBKc2dv?=
 =?utf-8?B?aDBYa1FUVUZTazVoZkthQXNWZ2JBZ2JPVXZZRTQ3SlU4eFl1TFZQeXV1VkY2?=
 =?utf-8?B?TnBYeDhISVVEVW5hWnp5UVY0QytHK25nTHZtalpxQlcwWnpmRFdCMGJ0TERN?=
 =?utf-8?B?UjJRVHBOdU9BSWxnWGF5eHNYc1dSNzI5ZU9XcmVWZk1PYUpBUEZmd0dYeDJG?=
 =?utf-8?B?SDhZNlo5WGFWWlZEM0dmbG9abHBvRjBBcHk0MFg2V0RSbUVERzBVZGxRYWFq?=
 =?utf-8?B?WEt4R2YyWWhCYmQ1bUdUZjkwc2dIM0I3MGF4T0JxR0tRUWpiYVVQeWZVdHpS?=
 =?utf-8?B?L1VOQXd0alhrdjNZbFFVdXgva3Zsd2F0cUJSWlVTV05HSVZRa0tCanN0Skgz?=
 =?utf-8?B?bHN6MmxPVk9aWEd5Um9UYUlYZ0t6VVkxaWhqSzhpMXNkanFFQm13dHlURWV3?=
 =?utf-8?B?UjUzT1E1bjd6Y1h2emJvRXJqNldobE5DQ05teXFVQUxNVHZjRkVwbzBZeGNo?=
 =?utf-8?B?RlhqUktNWmxBS2V1S2N2SHFVRE95NkVwTWxuMWhIdkZrM05aR0tNclB5UlNH?=
 =?utf-8?B?NkxaYmJ5cnFnamFJa1lkRHJvc3lDMC9PNU5iZU92UGZ6cEZ2aVFFZWJOeExD?=
 =?utf-8?B?ZnBhM01GUUkzeFpOdDZYa1hITnZWbWJjNG5yRWxwL3ZoTlhONVJ0aFVPb1gy?=
 =?utf-8?B?Mll1dHRMbkJod1VBVWhENGx5eXZINEVveFppc2hmb3ZrZjd6VjFKVlRsZ3do?=
 =?utf-8?B?N3VoRlA5a09OM0lCUGRUYzBXY2pqcS9MekVLWmZhb3VCbFNyaUVBOE9scHg3?=
 =?utf-8?B?TXJmTHkvWldmN2R0ckZjOUhuY2NSR2Z1VlJyM3FZcWJNS2UwRDRBZ1h5Wm1v?=
 =?utf-8?B?dzlqREcrNm1hNXZ5VHMza2J4Mkh6UGw3OWdjSi8vcFY3RUhnVFY2SkZFbDNK?=
 =?utf-8?B?UHBFZm5zbWNWNTJ3MG9PMU9qc0lMN21INEl5TG1FaFpaazZSZE93M3NRSkdE?=
 =?utf-8?B?Y2k4K2w4T1ZhMEFrUnlLRndPUHNQMFdraXQrdVJjR1hJbFJRNFVEVHkydkdp?=
 =?utf-8?B?OGtKNWNxQVlCSHdZdmlvUzBzcHVmSHA3amhGWEZrRXdRMGZTUnpNdnNoeEFv?=
 =?utf-8?B?RDBHa1J5SXdWL2pxMU9WK1RYSmZvTW1KcUhDWWx2TnJwV2NDS3REa0FOVTlz?=
 =?utf-8?B?R3N6cHQzQVp0enVvWUZsNy9HcXRlR2M1c2cwalpSalU1RGJDaDVPa2Nrcmdn?=
 =?utf-8?B?Si82TFpGSVNYemtrc29ZZ05IN3NpL0NiWE5XZ25rQnJ0UzQxOGNrTC96YWYz?=
 =?utf-8?B?L1pNUnd5czNTMjFNS3U1VkJ6ZnI4Q1VRTnIrbHhxYStSeUhtWnJjVk5seUtZ?=
 =?utf-8?B?RFdjbGFydGtLYWZpblRUb1BPWENCeFIvakNSdVBSeitpQk5OWjJ1K1RDd2RB?=
 =?utf-8?B?ZXEzWHRkRHZNZkx5L3dEYWljOW5MQzJTNlE0blJjbytLS3lHeXFkTW8wTG9a?=
 =?utf-8?B?TVRpT2Q1blJzYURiY2tOMmUxWTJ2QUhRVG1NbGhiUUNmdFJSWlh2Skd1eE9L?=
 =?utf-8?Q?f9ktxkYUec6194ofxLHtzZpS/13S6jb8e8Qo20Q?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bef3ee10-87d2-46b4-e196-08d97c1696e0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 09:11:11.3180
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iRNEe7DJITAgZ0jdGMzpK/9F0fWxHT9e1gmOkD+aa3v9u+XfAbC5vHwJzgMtHxhJ96qDOocdYR3f0q+zTl7vdg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2608

On 17.09.2021 10:45, Andrew Cooper wrote:
> There is no need for bitfields anywhere - use more sensible types.  There is
> also no need to cast 'd' to (unsigned char *) before passing it to a function
> taking void *.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with (nit) ...

> @@ -1888,18 +1883,17 @@ void burn_credits(struct csched2_runqueue_data *rqd,
>      if ( unlikely(tb_init_done) )
>      {
>          struct {
> -            unsigned unit:16, dom:16;
> -            int credit, budget;
> -            int delta;
> -        } d;
> -        d.dom = svc->unit->domain->domain_id;
> -        d.unit = svc->unit->unit_id;
> -        d.credit = svc->credit;
> -        d.budget = has_cap(svc) ?  svc->budget : INT_MIN;
> -        d.delta = delta;
> -        __trace_var(TRC_CSCHED2_CREDIT_BURN, 1,
> -                    sizeof(d),
> -                    (unsigned char *)&d);
> +            uint16_t unit, dom;
> +            uint32_t credit, budget, delta;
> +        } d = {
> +            .unit    = svc->unit->unit_id,
> +            .dom     = svc->unit->domain->domain_id,
> +            .credit  = svc->credit,
> +            .budget  = has_cap(svc) ?  svc->budget : INT_MIN,

... a stray blank removed here.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:15:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:15:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190434.340251 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFOr-000568-L3; Mon, 20 Sep 2021 09:15:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190434.340251; Mon, 20 Sep 2021 09:15:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFOr-000561-HC; Mon, 20 Sep 2021 09:15:45 +0000
Received: by outflank-mailman (input) for mailman id 190434;
 Mon, 20 Sep 2021 09:15:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSFOq-00055v-GS
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:15:44 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a86fd60d-74d2-4988-9074-2bb758e988fb;
 Mon, 20 Sep 2021 09:15:43 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2059.outbound.protection.outlook.com [104.47.8.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-FPKw79hJN7yRL4Ww8yZr4g-2; Mon, 20 Sep 2021 11:15:41 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 20 Sep
 2021 09:15:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 09:15:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0064.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Mon, 20 Sep 2021 09:15:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a86fd60d-74d2-4988-9074-2bb758e988fb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632129342;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VxRie8cOTZnfkAjp5xIlxvn2twoJsUwQCtRW9MmLH7I=;
	b=CxYbY1BI+LdzuuuEc67Ao1Xj1AAKUjpTn04ftBOMYOEg+iMNQvKDTQrA9lX4Puel8LtJHX
	nRRFdndedebQbaKecLaxHPALvx6U2BWzwSNlREQtIVio1kijbhxgqeiCi8PQIJsXWxMjyG
	ja5NRXatNpmNQX1k9/4Lv0J/QWFUXio=
X-MC-Unique: FPKw79hJN7yRL4Ww8yZr4g-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y/+utKpuaDOaB9Mw+McKaI5p2gltbtk68AV2/9w1HPHSSn4iHqWRpDmnGGw7wMXs2UAmMj500pwI5QkF8VRGJazMXb+rej7LkOyqntWb/5J/cfb6/RCk21NBNnUwu8KPZmoTJuAR6no3Vr96GvxMeiJgf94KnxFxF3s0YLHTz792RjHvBhRu7GAnq5djUb3MclSbSPCAkxWDp5ZoGsWe/3teEU5q9tSgZgcfWMvGKVD91XYj7L2hSMdJAcbxVD1dVeHo3N0lH/3YZ8fhVTF7HUgc1NtMwYuWJzKk9hKc/v7tKJwAawTz3r2CJ830l2UqYk0L3HJCHolmrHq1sBLScQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VxRie8cOTZnfkAjp5xIlxvn2twoJsUwQCtRW9MmLH7I=;
 b=E0bojPRTyMEZnaN9VePzip4dJM1Toq3tb4CuFf4aXkqC6LdEIgKG+eyec50zSVJtR3rbKaJM7oB/xcnPrUIfjDPh8e4AsJK95eTLKLn7ncwAvrUVVnFAi9I8Ylbl00oCg1ogDvDq4Ry13PFkTDNfL/vH87W7hK6+GaGkkEFaLd7W8dx4FPRgO3/25cCIxfcvac+nWQ3366vg9OL96fqdl20ZDySFxPcopKzq2+XD9MLgguEUqcu4aA+Lrqnr73sUIVEenmj0NE8jQF2wZ5b+6dNl7lTTX6O8oqm0a3PXyIqNJmngiGnvIyORBnTsezzw/2jomkxwZvwSP/woBTKlug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 6/6] xen/trace: Minor code cleanup
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-7-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9b3bc06a-43a3-06b2-e109-bcbbb46ef087@suse.com>
Date: Mon, 20 Sep 2021 11:15:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917084559.22673-7-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0064.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e73940f-b776-42ca-c7f3-08d97c173641
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2606D44BE435F26CE080EB24B3A09@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DsKOZn0gZhtOB/V89Nr/e/jgLaZIiuN0vYqgN78PG27jv3+vUf71rxvLOLmL7xFBw0BhNPTemU5LtvGmKgZw7GAott6Y6lSTOq++jgHf3QIDCbJYWQT1smFowKeIERQrk0f38fSEf1RimbLwjcIU0xzXMZgpcjf+l7H8lc6yfaAcyd+ySRM+T7xXa/m7PS4D5QMr2o7Q2DHkZceRfpxWJIfMOES8M6qyBV8uIXndlafI77mNiJYRZwvjfryYtxC3O7oxeN5aaJxY8KTEubTekVAfmuI/8+gWoYpxTBOp5N21VVQT8K3NWF4HDsaoYeyzVj9J1aGi1OK9oQJnwx24EZmvnPP5zKep5xrAyaxUIqiq4Wx7TD87ArH+79cx2dWhCm2zhQiWeGtqc7tefTIlQ8Q7JPQYjc80U8tmdtwys6nuU66g5g7IFYXaDYLFI1IqbtP14T2BMVnWg8QgyXQdSXpuvV6zwv1FaHWbV7zo4h3zcOn43fetyv20//xD4lYIRrYlzdJ4MAAjPv4lFsCw7YWkgMJO8JhURBKMw6mMgQbkZL9sh9lRU5arLUptGVsjt4kD19b+ZYzPS7YIQsBp0QYq27/VDZ77/bEla45r1G6vqwWJOq2pDBnnSLualpdXciyYAHAMLCh1Ffdu6XbzAgamZW8vOJnIiPAsP6t2n5jHoJMK7RYX/GaeS5kXegm/xB+7BJKLs9hye7jsc4AM16QEauRrFg1wsQJjT+oF5Ak=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(31696002)(4744005)(2616005)(4326008)(66476007)(83380400001)(316002)(66556008)(508600001)(53546011)(186003)(26005)(956004)(86362001)(54906003)(8676002)(5660300002)(16576012)(8936002)(31686004)(66946007)(6916009)(6486002)(38100700002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eHdoTVVTaUZieFF3Z1hWRWpLbFNQRlc4RzZmZGlhbWpidWRHOFVTK0RiK3Jv?=
 =?utf-8?B?SVhpZnpER2hUd2JQU3E0RkhOZWhmeFFva1lRcVhXOE1sd3J0Z2lNN3NCU1lh?=
 =?utf-8?B?Wis0Nk5PRzVSV3U5bTlPSmJGQ0hsNTVpanRSYTUxb1hSdWdoejhKL1dNZ3cz?=
 =?utf-8?B?R1hrQzZTQ0dxT2srNDMvNnA0Rkt2VmhQcy9mQ0U5ZUF2WU9vWlpnU1hGTE9v?=
 =?utf-8?B?NnA4cmdXdU5NQm1uOHprQllNMklNanp3ckdsV01uVHIyZFlJcDlMemJ0QzhR?=
 =?utf-8?B?MTVPdkRUQXNUU3A5UTFJS0xSRFZaS1lMVWV3UGVRV1didnk5OVVSbUlDZUho?=
 =?utf-8?B?MEVueEJySWo3OFBRd0hJK1dmQmhWczQ0b0pnMWNSMDhMM3AwcFRiYW4yNTIz?=
 =?utf-8?B?dUJjbW41R2pVc3pJSkRQbGtpNmE0TTZscHdWVGhiMmJDQTJQMk5XVER3d1dj?=
 =?utf-8?B?eFdHU0JuNnBmd2RlcWtsaStRS2dEOWlIYUh2U3JPNjJXeC9nNDNQN2w3VDFp?=
 =?utf-8?B?OEh0QXR3cC9tUUtOQXM1WjEwd1ZiRkFvQm8vU3dJLzRoVDBsMGNmTDMxRTVj?=
 =?utf-8?B?NGRJOTc3K21HOHpKcG1nTmR1RFZBbzFXWjA5UCtiQ3RWK2FldFFMRmlGc2dN?=
 =?utf-8?B?SVhSeHE1aktpWDdYclV0L2x4eHloTWt1eXQ0NXdUNW5aUDNMK2NoZ2NsUUkw?=
 =?utf-8?B?RkJBWXRUVEp3ZVgvUmlTcEx1OWhUcktHbGhpWTNISEV1bWhudWZlZ1NhZzRK?=
 =?utf-8?B?OU9JalV5dVdweExvMmFxYXBob2huamxDMytzbDJqUDNBTk1GaUp2b3dwVElv?=
 =?utf-8?B?Zy8vaHEzSlBYRXRqMlNFRjRiL0gxUkpoUGlqVTZwYmtrSkg2Z3dlZXFqczkz?=
 =?utf-8?B?YjZVaHlWQ2Voc0cvZGpzYzYwMWEvc3NDVUVSZ2JpV0FFRXdjeDlpemxEYTlV?=
 =?utf-8?B?OEhiUy9IVnpGbU0xaWVUMHd6a1E4ZGNPRlN2eUEzZmdCT3Uvbytvbk9uRUZm?=
 =?utf-8?B?VFJUeis1UmhyNzIvVTl4azljVXhiVC9ScGtqOHRMMXJrbjkwWVBCVGtjVk9S?=
 =?utf-8?B?SlppU2FQbHdJV0g3ZUNPNjYxWURLWlkxQVd3RktlRUxqbkJUTGhHNXhJNWpP?=
 =?utf-8?B?L0JreEwzNW5LRmRNN0FycTNCNUVZY3pGZm9sZFI3b04rbXhlRk5zT3BNa2VD?=
 =?utf-8?B?cFJNekhrdC9Sekc4aGk4UzIrTDRwRTlzL1JWV1YxdzdTVmhIblZNaDkrZW1r?=
 =?utf-8?B?Mm5oZGhWL01TOC93OWkzWm9OYjQ0ZE9qYXgyeENmdnN5L0pBV1FYQlVvTTVs?=
 =?utf-8?B?Z0JpM3lIaCtRdklDbURFUnNNeVJiYXQ5cHhjaHJNa3pJd0VRRTRyc2NRSzhD?=
 =?utf-8?B?elRia1B0b3hBdTZYVG42b2pFa0NoSFhuSXFiSHVUWVFYT05SeE01dm04SnFy?=
 =?utf-8?B?eXkyM3FGR0I2enMwWSs0UHhLN2xXRVdpN2k1WmR4T0NadWd1T1ZqNVp6QUFH?=
 =?utf-8?B?c3lDdklNTjg0MnVqek12cHA0NWVGci9IamxhSHJFaHltMWpYU2hBdHU4UTBp?=
 =?utf-8?B?RDZ1b0xnZ1dQWEsvL0JZV2ViMWh1SlJ0TTlFVUtTbG1RVlJOKzg5WUhkbGRZ?=
 =?utf-8?B?ZUlwUWFRRnU3dUt1clhkVlpTSXlOUlNveW5QdTVwcXRoT1dhaTMxTkpvTXlL?=
 =?utf-8?B?Q0RaanlPeTdLbXBsNFJRamlyQXErOUdJYmd4YUl5SFRkbXc1TGlYdlNoT3Rr?=
 =?utf-8?Q?OX6uHrJsLcRTxw8k5k1RIfhur4+s4ucQwZXJFE2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e73940f-b776-42ca-c7f3-08d97c173641
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 09:15:38.7067
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RzVqpYaKB2zyrWZRoD1jKTplZSp+sbE9WnEWMO7ShClDe2MGBgD4qcVdvqsFIbuZYA0ZdAQo0eHnm7fn8+d/sw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

On 17.09.2021 10:45, Andrew Cooper wrote:
>  * Delete trailing whitespace
>  * Replace an opencoded DIV_ROUND_UP()
>  * Drop bogus smp_rmb() - spin_lock_irqsave() has full smp_mb() semantics.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Largely
Reviewed-by: Jan Beulich <jbeulich@suse.com>
One remark:

> @@ -717,9 +713,6 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
>      if ( !cpumask_test_cpu(smp_processor_id(), &tb_cpu_mask) )
>          return;
>  
> -    /* Read tb_init_done /before/ t_bufs. */
> -    smp_rmb();
> -
>      spin_lock_irqsave(&this_cpu(t_lock), flags);
>  
>      buf = this_cpu(t_bufs);

I wonder whether the comment wouldn't be helpful to move down here,
in of course a slightly edited form (going from /before/ to /after/).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:22:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:22:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190443.340262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFVE-0006an-8q; Mon, 20 Sep 2021 09:22:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190443.340262; Mon, 20 Sep 2021 09:22:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFVE-0006ag-5R; Mon, 20 Sep 2021 09:22:20 +0000
Received: by outflank-mailman (input) for mailman id 190443;
 Mon, 20 Sep 2021 09:22:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSFVD-0006aZ-ER
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:22:19 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3f8ea34c-19f4-11ec-b867-12813bfff9fa;
 Mon, 20 Sep 2021 09:22:17 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C9A5422075;
 Mon, 20 Sep 2021 09:22:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9E1B413A81;
 Mon, 20 Sep 2021 09:22:16 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id p7nYJMhSSGHWPQAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 09:22:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f8ea34c-19f4-11ec-b867-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632129736; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=oS03HDtKLKfHZjyM+5QbPlj+nBWvnzYfMe8HQgMTFdA=;
	b=GqYVrzoKEv8ooxuzEvmoSWZXLxYtDeuG8zjaevG1+I4DBR0boFFgwjaU90efz8JFitTuCG
	kSLbV23aulQ/bmSvisyTVB5sEnVv2jSKuUVMHN4DEU5bTwYm6qUrkYD8fpF5ZevAB4QK+/
	Wo7MXud9i6SvjSWIJ6O1kO8IpAZKfgM=
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
Message-ID: <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
Date: Mon, 20 Sep 2021 11:22:15 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210917154625.89315-5-roger.pau@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="CqVaxyiOTWfZrucNpfGOhilOMvonvIArM"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--CqVaxyiOTWfZrucNpfGOhilOMvonvIArM
Content-Type: multipart/mixed; boundary="4zhg7M398UstSw5PtCmq64k9YCY8ST5nY";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
Message-ID: <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
In-Reply-To: <20210917154625.89315-5-roger.pau@citrix.com>

--4zhg7M398UstSw5PtCmq64k9YCY8ST5nY
Content-Type: multipart/mixed;
 boundary="------------2A7B7DE445A6071263267B5F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2A7B7DE445A6071263267B5F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 17:46, Roger Pau Monne wrote:
> Exploiting the talloc clean up routines to close the Xen interfaces
> is cumbersome, specially when atexit can be used to the same effect.
>=20
> Convert xc and gnttab to use atexit which allows to drop one
> indirection from the storing variables.
>=20
> No functional change intended.
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>   tools/xenstore/xenstored_core.h   |  2 +-
>   tools/xenstore/xenstored_domain.c | 57 +++++++++++++-----------------=
-
>   2 files changed, 25 insertions(+), 34 deletions(-)
>=20
> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstor=
ed_domain.c
> index 47e9107c14..8930303773 100644
> --- a/tools/xenstore/xenstored_domain.c
> +++ b/tools/xenstore/xenstored_domain.c
> @@ -737,35 +725,38 @@ void dom0_init(void)
>   	xenevtchn_notify(xce_handle, dom0->port);
>   }
>  =20
> +void interface_cleanup(void)
> +{
> +	if (xc_handle) {
> +		xc_interface_close(xc_handle);
> +		xc_handle =3D NULL;
> +	}
> +	if (xgt_handle) {
> +		xengnttab_close(xgt_handle);
> +		xgt_handle =3D NULL;
> +	}

Could you please add closing of xce_handle(), too?


Juergen

--------------2A7B7DE445A6071263267B5F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2A7B7DE445A6071263267B5F--

--4zhg7M398UstSw5PtCmq64k9YCY8ST5nY--

--CqVaxyiOTWfZrucNpfGOhilOMvonvIArM
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIUscFAwAAAAAACgkQsN6d1ii/Ey80
wQf/QeSaaesxhFHKUvVz4H+PTKY8l08BJ5TUEAfqwsVJpfxWm7BACP6o3Tdn6pAkcjSdzUtFIOAm
DWokeeTh5c0CXpr5zT0E//C1ukx7kb6SDEf13XzaaJ7XXlINpidXd3GbqJAoadxVapeFv83/FQUL
VhG6PUPHEmThR5EpNhr+9MaudxKhWjXdICk8gQFcdTHw5MQIdfANJ9QBeOmVU7aUqso3xEpGVeom
kLBKOYg0obCcHNF2CBjR315fCU8n2bApxKlUOzYmxjGpyptSQXIgTUIk+ulcUkz9oqqQdIaW/Af2
hObPUeE89rtOMq5UTBb5aTydV5Wkz3ujJ104weJyEA==
=FiVS
-----END PGP SIGNATURE-----

--CqVaxyiOTWfZrucNpfGOhilOMvonvIArM--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:26:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:26:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190449.340273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFZ9-0007FO-Pw; Mon, 20 Sep 2021 09:26:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190449.340273; Mon, 20 Sep 2021 09:26:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFZ9-0007FH-Mj; Mon, 20 Sep 2021 09:26:23 +0000
Received: by outflank-mailman (input) for mailman id 190449;
 Mon, 20 Sep 2021 09:26:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSFZ8-0007F7-6j; Mon, 20 Sep 2021 09:26:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSFZ7-0003U0-V7; Mon, 20 Sep 2021 09:26:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSFZ7-0001OE-NF; Mon, 20 Sep 2021 09:26:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSFZ7-0007k8-Ki; Mon, 20 Sep 2021 09:26:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wMVstKarXFtYkU4vdFwK9HWM4Qs8eg/fXbQkp8/RW88=; b=l3p9sKK8VjiW6GC5c8DGFIO2UZ
	OWOXlxdBms8pRz9Or2iKZVwuNPIcXY9266ZuywgPUGByTSyVxZitS+exkVMFi0f+JZ27iu8MwPF4l
	xruN5mchw5zhn9tkaHCthxgoKuOFp7/9XfcJcEDUT2cfbBXRdDWxRGRGeXdq83mffUNY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165124-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165124: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:build-arm64-xsm:<job status>:broken:regression
    qemu-mainline:build-arm64-xsm:host-install(4):broken:regression
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-libvirt-vhd:guest-saverestore:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:heisenbug
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:host-ping-check-native:fail:heisenbug
    qemu-mainline:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=c99e34e537f13a431a80e3e414e5904e9dd0a116
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 09:26:21 +0000

flight 165124 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165124/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm                 <job status>                 broken  in 165121
 build-arm64-xsm            4 host-install(4) broken in 165121 REGR. vs. 164950
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-libvirt-vhd 16 guest-saverestore fail in 165121 pass in 165124
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165121 pass in 165124
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 6 host-ping-check-native fail pass in 165121

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm       1 build-check(1)           blocked in 165121 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)           blocked in 165121 n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail in 165121 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                c99e34e537f13a431a80e3e414e5904e9dd0a116
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    8 days
Failing since        164967  2021-09-13 13:06:52 Z    6 days   14 attempts
Testing same since   165076  2021-09-17 14:25:05 Z    2 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken

Not pushing.

(No revision log; it would be 3112 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:32:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:32:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190460.340287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFfL-0000OG-L5; Mon, 20 Sep 2021 09:32:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190460.340287; Mon, 20 Sep 2021 09:32:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFfL-0000O9-I9; Mon, 20 Sep 2021 09:32:47 +0000
Received: by outflank-mailman (input) for mailman id 190460;
 Mon, 20 Sep 2021 09:32:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSFfK-0000O3-6z
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:32:46 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5b010a0-19f5-11ec-b867-12813bfff9fa;
 Mon, 20 Sep 2021 09:32:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5b010a0-19f5-11ec-b867-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632130365;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=v4YEtbFzgT3qP8LwEqrQaBDA+cTqaF3A5gedDkVpVRI=;
  b=Q+ohbT93ELD9OCVI3j3UXGQ3RneWo/+xkb+2pZ5R/1j2RXs8mNzfC6IP
   ML0/VVEDGhb3qY5PPgHHQkscZEe/wFKZggEfH5jAfcLf+T+T8hU6hLj5/
   6YkaYBI1BqicSwJO/hCSh/hyijLpNT6fCT06XUeLNqdK99CoOJf9RAAMa
   Y=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: Fjy7jjLh3jV4I7sGs7Hhz76U7R+P4pq3GobmjGUGQ5KgUIOcwjqBUaxvI7eKkrbfVOpHbkWG63
 XQTGTj/DL9Gmf9rn9mHp1ad8SxRpspMyqh0V5nW1xT/DtthE5CY2x4r4dK3SocU+nWwjJYwLaI
 BK83Jk2MCSpJXOeJZkVbcsHSIHOiu6bbnhQgUGd9ezv6nO4JpIsWj6AdDIJ+f5Q/OWj/4GZ7w7
 DYRq+duuo/AF9Y6BmxvvaQBVoEZMRnFAqhF1qihc3XJ3aZj2ic/WU45zMuhdS57KusPvRNzJOR
 pvn+r+L6WyeNFMzbkLMiAMn3
X-SBRS: 5.1
X-MesageID: 54894368
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:92j7XqnN9l3/x0hd/uSg+Dfo5gwqIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIeUWqOOfncYjTyeN4iPYiz9koO7JaByoc3QFFl/yxnRiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NFqupHvRB5wArbvh8EaVh9DSC45PIQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQKqON
 pZCN2oHgBLoMyxuCgcYKaoCgL2Lq37AcgEFqm+ouv9ii4TU5FMoi+W8WDbPQfSVQe1Fk0Deo
 XjJl0z7CBwHMN2UyRKe72mhwOTImEvTSI8UUbG16PNuqFmS3XAITg0bU0Ohpvu0gVL4XMhQQ
 2QV5SgGvaU07FasTNT2Q1u/unHslhwBX9tdFcUq5QfLzbDbiy6VD3YJZiRMY9snsIkxXzNC/
 mGOm9TlFDl+qoq/QHiW9qqXhT6qMC1TJmgHDQcfVhcM6dTnpIA1jzrMQ8xlHarzicf6cRnyz
 CqNtzMWnKgIgIgA0KDTwLzcq2vy/N6TFFdzv1iJGDL+hu9kWGK7T4iP5lWE4O8ZEJ26UXyGu
 X4fwMfZ7clbWPlhixexrPUx8KCBvqjeaW2H0A83RfHN5Bz2pCXyJtk4DCVWYR4za5daI2eBj
 Fr75FsJjKK/KkdGekOej2iZMM0s0aGoPtDsTPm8gjFmM8UpKVPvEM2DYyetM4HRfKoEyvpX1
 XSzK5/E4ZMm5UNPlmHeegvl+eV3rh3SPEuKLXwB8/hC7VZ5TCXPIYrpzXPUNrxphE96iFyNr
 r6zyPdmOz0ACbajM0E7AKY4LEwQLGhTOHwFg5UMLYa+zv5dMDh5UZf5mOp5E6Q8xvg9vrqYr
 xmVBx4DoHKi1CKvFOl/Qi06AF8Zdc0k9ixT0O1FFQvA5kXPlq71vf9DKMNoLed4nAGhpNYtJ
 8Q4lwy7Kq0nYhzM+igHbIm7q4pndR+xghmJMTbjaz86F6OMjSSSkjM9VgewpiQIEAStss4y/
 ++p2g/BGMJRTAV+FsfGLvmoygrp73QanetzWWrOI8VSJxqwoNQ7dXSpg69lOdwIJDXC2iCei
 1ScDyAHqLSfuIQy6tTI2/yJ9t/7D+tkE0NGNGDH9rLqZzLC92+uzNYYAuaFdDzQTk3u/6Cma
 bkHxv3wKqRfzl1Lr5B9A/Bgyqdnv4njoLpTzwJFGnTXbgv0VuM8cyfehcQW7/9D3L5UvweyS
 3mjwNgCNOXbIt7hHX4QOBEhMraJ28YLl2SA9v8yOkj7unN6peLVTUVIMhCQoyVBN78pYpg9y
 OIstcNKuQyyjh0mboSPgixOrjneK3UBV+Mst40AAZ+tgQ0ukwkQbZvZAy7wwZeOd9QTbRV6f
 m7K3PLP1+ZG207PU3svDnycj+NSiKMHtA1O0FJfdU+CncDIh6Nv0RBcmdjtot+5EvmTPzpPB
 1VW
IronPort-HdrOrdr: A9a23:x4MkiKnm7XGQCgDmFgZSe+IZeyvpDfO7imdD5ihNYBxZY6Wkfp
 +V88jzhCWZtN9OYhwdcLC7WZVpQRvnhPtICPoqTMiftW7dyReVxeBZnPbfKljbdREWmdQtrZ
 uIH5IOb+EYSGIK9/oSgzPIY+rIouP3iZxA7N22pxwGLXAIGtJdBkVCe2Km+yVNNXh77PECZf
 yhD6R81lidkDgsH7+G7i5vZZm8mzSHruOrXTc2QzocrCWehzKh77D3VzCewxclSjtKhZMv63
 LMnQDV7riq96jT8G6T60bjq7Bt3PfxwNpKA8KBzuATNzXXkw6tIKBsQaeLsjwZqPymrHwqjN
 7PiRE9ONkb0QKfQkiF5T/WnyXw2jcn7HHvjXeenHvYuMT8AAk3DsJQ7LgpOSfx2g4FhpVRwa
 hL12WWu958FhXbhhnw4NDOSlVDile0iWBKq59Rs1VvFa8lLJNBp40W+01YVL0aGjjh1YwhGO
 5ySOnB+fdtd0+AZXyxhBgu/DWVZAV3Iv66eDlHhiTMuAIm20yRjnFohfD3p01wtq7UEPJ/lq
 L52qcBrsAHciZZV9MlOA+tKfHHfVAleii8RV56F26XXZ3vC0i92aIf349FkN1CWKZ4hqfay6
 6xHW+xiwYJCjTT4Iu1rcV2ziw=
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54894368"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=m4Was3vY7KLasnAynXbzxpffCvstLd+voixBI2/55FZONnGQ5SQq0GOKWShoA2rjwvmJSZSdoMQn3xtEatHHU5uhiysJZvKGhrZeWLQdFreEky3W/Rs3ndi4YHL4xw/3UuK8mXFu1iIKzUdLWQHv3/HGHLaSdMp3UQMvX2p20GbbY2lZzgxq2vZmqbik3pkUs/OEE7+UBdc80iexpp921lXwAMFYzNhfYhToSOX7xMYhf7hzJARGQ4UrQZh+7yIvRofCnutj6H0hexXTo+GENZ5S0O1qoAUGlu9Lm8izgANfqsJO7umH+i4/EHYX7Jq35ZKZYXSk2Tcc5v0KmbyWFw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nHjxeA3dzMdrSXROFM0Iq38LOnMVMcK3+TJ654bNhG0=;
 b=A/fq6vpPyJC/i8cBDpJRvtdtQvT1fitHyI9uWj47OJ6zuWXHcDXxRPDOzGvgPmSrR41JfEKd6A/WvsiVsL9lr2umlnGQ0Oxj/S4mxKawnwd+IDmRP0Vse3QPgEOfudmH8nYGUAIJZ3WgCjrC2oD96K8uJmeq8orOcWaU5pd1aAFu0Yl0mAAgHmp+7WSMhPNvcjMtP/7+m1tfHIimlCB2QUtxkC1CLWAOkoNqxFRUshN7kNTtg4k6kPdnyFHUVVueUKqes1gVnM5tN2iX5tIncB4Rlddq0AIggkmGa86ho7eVPUcjC/WFiGkZrKq551aMZG63eb9QhB7Cf+HyAzqmYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nHjxeA3dzMdrSXROFM0Iq38LOnMVMcK3+TJ654bNhG0=;
 b=BpbKF3SdqmWdR7PapGP1nCUOCY0LMifVm77J35Sv0PhsncU0kcXuK3D6fMU7fD5jPrBoz18Y4deaYfS8hHvFF9rC9frhCNsSy5xjoYYFpQqAJeNA7Q2tfddSfUFvnjjcaW2DnehEr/GBRh9mUR3DLM6Ro7g41sUQWAlLLXmkzl4=
To: Roger Pau Monne <roger.pau@citrix.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, Christian
 Lindig <christian.lindig@citrix.com>, David Scott <dave@recoil.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-3-roger.pau@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/6] gnttab: allow per-domain control over transitive
 grants
Message-ID: <60295250-d6b2-1a0e-be30-d83b4f3c76bc@citrix.com>
Date: Mon, 20 Sep 2021 10:32:24 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210917154625.89315-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0042.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::30)
 To BYAPR03MB3623.namprd03.prod.outlook.com (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4de0eb42-7488-49aa-dc61-08d97c199280
X-MS-TrafficTypeDiagnostic: BYAPR03MB3797:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB379763A585513D6861C24D2DBAA09@BYAPR03MB3797.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: SgCWk85TnL8hZfVIHvbrTA00tLXZjmhu/NNBbTYMTA7fYOlZWiVEQuHc0Hzv//jcvR4tj18Nz9WcyIg+ldwvPgtbugsRr85KzFq/QiL4zgF9QfsdcGgayFQdjgnttnLBRHcdWxBTLHDNjyG8hnLfo7OMhjhMt+8m+pydOnsiB61H1OkrSeKysHSGvwLUa19zsyWxMzdr7C2+yTTnEfWBumCLEBcuZRNrilWgl7m/DEsv7VfVM86UMScH39/RKUKxmWJJV8XqkBHedmg7wS4H98hlIst+z0yUytZfvUF9IVDWWSZ3n6ACDFdSeKMEg6PoIqPltVSTBK9U5agpILl74Jtbn0s65SyFDbCQ8hxyp6huom8tDZtsO34iSn4nBlkq0OHDiN3+kh8WnYUuvomyb60fd2WeI7eRBd7gmtyx9RSefqt8a9JAkfP/mXo/fmE3xnIKUAv1ptdEOYgFHPNUb7ax8VTWZq6rcuAFeQXeXIcL1mYOOSlj+pn+QPQJ34/KPt4JXGxbt7GZxCzjjeq1/mnnS5RNXp2Ekecbn+OkQezVQI3VGXIklA8hBlrN9T/aw7u70ALZTpt1SXe7v3NTvk5+a6RXnREgV05VBDtJs43ejtw6hc41j0BmCLvAjmWqrylkoKD23r4zgQgdAuzUnrjqBZl8XkkdfWL41mVeis6AmUeITm3Qf52oHHmRqAlidndeOLI4DFzOsh4l0cBAvc6GEyqYJpfzKWpDyrsXyVY=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(396003)(39860400002)(376002)(136003)(366004)(2906002)(54906003)(5660300002)(83380400001)(38100700002)(53546011)(31696002)(31686004)(86362001)(316002)(16576012)(8676002)(8936002)(55236004)(36756003)(478600001)(66946007)(26005)(6666004)(66476007)(4326008)(2616005)(66556008)(186003)(956004)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QWZKdVc3VzkyQ0pCRzBGM3FlandRR2N4QlpXYVhFeGJ0T2Q4UTYrT1dWVFNq?=
 =?utf-8?B?cHptcENLWXlZc1NqTHJrUXpwY1NjSnY2d0hXK0F1MWVUL1hUN2kydG9LZGg2?=
 =?utf-8?B?Wm5oZDk4ZkdjcDg3WnE2RVp6NEliK1J0eTlad3pWWWlHZVRNNmFuQ01yWGR1?=
 =?utf-8?B?cTNrT09uZ0t2bEJFTkk2eEJ2T0tBMnVLM0dGUTI4UzdPZHZqZkUxOERneDFw?=
 =?utf-8?B?NEp2TWdWS0ZrYzdYaldON3h0QldzSzhFVEViTVNhLzBCR0k3OFNnTVV3amQ0?=
 =?utf-8?B?ckJyRHJtZUxjT29ZOWNyZS9pRGR3enlZbU1hb0RyYThlcDhPbWFCZFd6NzFw?=
 =?utf-8?B?R29ZeGZsbGRpVW56Y0c2dDh3bDNsczRGOUM2WlZ2TzZTZmhDUVJ5dWl6bE9L?=
 =?utf-8?B?U2cvdWcyUUw5NmU3UFFaMnFBbGtLUDNkOTdvRWJQaXRpRnVGam5ZWUkwNGVr?=
 =?utf-8?B?TmdQQ2lPcVF4anhGNlY5UWZsc2pLZ0c2R29sanFlVGpXU3dwTmV0R1RLRFZV?=
 =?utf-8?B?WWJlK0tjUzc2d3dFZ0pFai9pSTVBQnh3eHZiM1BwcjdQbmw3cWRaaW5kaERz?=
 =?utf-8?B?SzBCU3RHZmMrd0ttR2hDZ05BKzNSUHVUT0M4cHBLWUlleW16Zy95ZEgrZjVv?=
 =?utf-8?B?VHFuaEtmWi9wODREdTFCaXhsQ2ZvUlR6dStEVlphcHlrclU0L1hwbk0yZVV5?=
 =?utf-8?B?QStRendKMlNoMmtRWDh4S1JmdURudGxuUnRRUFhGbEx1V3UxMU5qMmRDYTJ1?=
 =?utf-8?B?cHNBQ05sUWhiZEdNYXM3UUVOekFhektsWDZrVXAwdUxuaXBBRHUyVUdDeFN4?=
 =?utf-8?B?ZThqUDVEcnJUak0ya1hXWjVCSGhGWXRPOHB5aUVYZXd3cC9DVkhzZHprN29S?=
 =?utf-8?B?UzNzTnUyTURoRjdNREpZMThEa2l0WXpCTEZxWEVyYlZieVFkak1PZUI1d0JO?=
 =?utf-8?B?NC9nZHdmVUphU3VkMGkvbDIrMnhkSHA2TnZGN0ZjMEJSLzFJNkxaSnJGUGFV?=
 =?utf-8?B?djhIWnJNY2RVdjZpelZVQ2t3ZVVPTGhGbDVTL3hudXd5V0MrS08zV09iK0Ns?=
 =?utf-8?B?QjRpRTlyMzFsc3ZaTERZejBYdVFseXpUWGg5eW1lQ0tBQWttVWZtNjR6NC9K?=
 =?utf-8?B?cWZjV0FJd2JsTXlPcEdmV1IwYVJiU2hDVmNKMG54MTdPeE14eG12QUZNOG1J?=
 =?utf-8?B?TXVOYUZnQVgwU0s0R01tQWpObkJlbkd3S25FREpLakpjMXFxMjNQcGRBdU1n?=
 =?utf-8?B?aG0xZzJzcVJDWWdpaU9kN25vdngyTEZ0QXdJbWxGL3NqV3lJN3YweHdhLy9S?=
 =?utf-8?B?VEYzMWZTdUN0emJ5MHNVZlY5K2RraER0VXViOGI2MU1PbzZEU25lUUpyT0FO?=
 =?utf-8?B?bXRYZE9KRzIxbVJDbXdsckVqYkFlYlRpeDlHUmxTWC9BT1Z3SE5RSFBSTFdN?=
 =?utf-8?B?cFZYK0hKc3VMZitKcExXdzNORUhSQStaQjVRTTF2MVp6R3JVQkQ4Mzg3WVZZ?=
 =?utf-8?B?aEZPcEgxODM5LzNTRnNxT1dwZUdBVUJpTUNTdHkzanRHY3ZFNkFlK2hSdUFv?=
 =?utf-8?B?L1BxallENFhHc1NpWWFyQmd1dmVTQzNEeTRJNmRVUUxDRHZVY0J0SkI3SVBP?=
 =?utf-8?B?M3Z6U2hXcGdXUnNUY3dOYmNueXg1VDEyejVpYXJyM2dUR3FmRUhMWnpueGo5?=
 =?utf-8?B?OUNtUE0yYnFEWGcvN016QzFscEZ4eEpmbGVmbjNCWG5tQnI2dmUxdGQ1SHNL?=
 =?utf-8?Q?aINaKnJKZqQTHTnmrY2gcRvHYLQf361wXEh0iAi?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4de0eb42-7488-49aa-dc61-08d97c199280
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 09:32:32.6599
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EHNReRaLwYuBOOwH1Iqzlp9RV+0U8/afqHoCRD+yjOufB36vTH8sq4CeVDNGtLEAJTgxB2R/SfxtH5i+/Sh6TCzZl720ITeJprco+AERWNU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3797
X-OriginatorOrg: citrix.com

On 17/09/2021 16:46, Roger Pau Monne wrote:
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 7f8456c50e..fe2201fca1 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -96,6 +96,14 @@ struct xen_domctl_createdomain {
>      int32_t max_maptrack_frames;
>      int32_t max_grant_version;
> =20
> +/* Allow transitive grants. */
> +#define _XEN_DOMCTL_GRANT_transitive  0
> +#define XEN_DOMCTL_GRANT_transitive   (1U << _XEN_DOMCTL_GRANT_transitiv=
e)

There's no need for bit position variables.

> +
> +#define XEN_DOMCTL_GRANT_MAX XEN_DOMCTL_GRANT_transitive
> +
> +    uint32_t grant_opts;

So far, we've got 3 bits of information, v1, v2 and transitive, and
we're tight on space in the structure with loads more to fit in.

I was thinking grant_flags or equiv to contain these 3 settings, and any
further which might appear.


One thing which is missing however is the enumeration of which settings
are available, and rejection of bad settings.=C2=A0 If v2 is disabled
globally, trying to create a VM with v2 needs to fail.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 09:45:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 09:45:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190494.340310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFrZ-0002Dx-NP; Mon, 20 Sep 2021 09:45:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190494.340310; Mon, 20 Sep 2021 09:45:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSFrZ-0002Dq-Jx; Mon, 20 Sep 2021 09:45:25 +0000
Received: by outflank-mailman (input) for mailman id 190494;
 Mon, 20 Sep 2021 09:45:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSFrY-0002Dk-Cu
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 09:45:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bfb67704-a133-457f-9a29-3a3eff9e6deb;
 Mon, 20 Sep 2021 09:45:23 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-kkjp7ZiANfmSWetVR3QDsQ-1; Mon, 20 Sep 2021 11:45:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2333.eurprd04.prod.outlook.com (2603:10a6:800:28::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 20 Sep
 2021 09:45:20 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 09:45:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0014.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 09:45:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfb67704-a133-457f-9a29-3a3eff9e6deb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632131122;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=D6JHYbDQM88nV390LQ//+KbJwGL2McjgWmKGEXAilAA=;
	b=OJNnqb/jSeQtylqS+rVnRZ++ptewQFrRIPWS5TF0CG8iakqKq+DjRTB5w4kZEElJHZyfuY
	L6uwvi/WYqameIa94DSR78C7LZrDEQ3ScPByofn/7WZ7utLkPPuNLIV/0LQ4DrnVB4dHMo
	ix0HFTKV48jh0z6Gb+N3zBqK0G/74Ms=
X-MC-Unique: kkjp7ZiANfmSWetVR3QDsQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NA8Z8cSHf1gv189oOMzLSDDBwkW6wygaaVTn25qh0T3Lo3gNiZtR2FI5tQuDjxbBAPcqQ0wcA+wpTZMktME0bq9g3fNVGy/BeET/ixO78z4liCFbZDh3SpZEcq2D+ggtxSXjhJ63mC8a+z6+o+GK0QX4OKxCvyo3KHCV+hQ5JFLDqlv9RFU7Gbrdvv1A/huJaKBZqDkKnXf/c7HtbcOkGtyx2QnU3PvKLyZTuZ/0IB+WnTD0ep8sSetlN77Ka9BrpTwzsfO4/hBJW45Q0JRsFPsSUDpO7VNEAgkawd962tAgxOYUjNNGFydUoUYSRyxZySsUf/fe+9BBlKsmYxVAjQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=D6JHYbDQM88nV390LQ//+KbJwGL2McjgWmKGEXAilAA=;
 b=YBJPzjdng1S75UsGLiapP0hIaAaPpmOxN/zixxZ9FW6syLNTrGRlIYpWDtlJoVR0pIOAzCWti8+6AT1L5hsFxm8NlzY4Z2j9OzA0RKFZYuiag5YcLNsI7ISrQzFDmTyb628uXS5UT78GSuWSWNp2KyBHO/qYpXc/uN+EU/NdkJUcNAeHJ86xzcyWCsOzePBDhNmLCJMZCd41btnAdPA/PfHjWFo1VoW3G9dVWLZ+9Vnobg4dQiD+lVstD5cy/uPVsAwpZD5Pgycnlre8GnahUBmhzIqGpFzhKDR9E+7z7c/2mTLPlb8vKK8V3jc/P2jz2CN694zJfQRLEXb1BIBIHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: xc_dom_alloc_segment() zero-filling its output
Message-ID: <e02357ce-54d7-e7ff-109f-681f46944732@suse.com>
Date: Mon, 20 Sep 2021 11:45:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P195CA0014.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4704727c-023f-47c4-094c-08d97c1b5c12
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23331BD6F637812A1BDD772BB3A09@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JLXgc02mP9IDSzSns0I7r0LA72FmvT87Q/FgjWps6vmtssmsgR5DLTwXeYZPhMHqL9XdIjClYOfiVlOVCoP2rZfgnY8wvUv8ma6Hgfi+iruvH7ux62rexiWxQ9QjL01G3S59dRlndDKRqFLeng9IqFCBaHDkd5Ug/GqotHODKLYDsJiihgSgn944KCWsgO2QT5lP26o/YqDswcBv2UOk5Pa5/gKZ16MOw3/17AmnQFottdChFUsdJxD43z+l/4PK4+tI6/ugpBBhCbtJU7qPerxSJKuPwNwUkvk1MWdleO+twVu8SOQic1dV/OUxhfASkPnoKKSYsK7y/P2mj9S7dYdna2hOYf3yOSKbHoAw2AgpE6eaqe2K6a5AFsojoBeKoOzNDTYr6vj0lF7uHXvIvwndEdJ7EsTGOz05Jgk90F4GnbYETG/Kh1PeE4BA+gqzZQk+ystjUQMIWA1d5o779dvjxTmbdwi7UIpMPPzaM9nub+TJygSu+lKrO29Smu1X3736WXJ5I3dboZqlT0hKn9uaAYGB07/X1hdOS1NYH6P75cgRBD4vHH3Adh9ccPG8YiTqLN/VnbYZTInr4kNxqZbYbcX7+TG5kt3ZXcDab4mh70avT8VP3WC0oR+KxHEe/TU5WOxchOtP4OT1MEPyIIoCBcZBcVdHRAe8xCLv56JSjKSv1n7WsMRWm/yGruqgM/eMCYORY3mNSVMEszoeysCabZGcLbRWfUSeUoJvwzM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(346002)(39860400002)(136003)(396003)(366004)(31696002)(26005)(4326008)(316002)(4744005)(956004)(2616005)(66946007)(66476007)(16576012)(38100700002)(8936002)(6486002)(31686004)(478600001)(5660300002)(2906002)(86362001)(8676002)(36756003)(66556008)(6916009)(54906003)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WjZ0SHV4eFFUVXhrc0ErUXh2VnpSdVEySUVHcVQ4S0ZQTzVJSzBMV1ZOMnJE?=
 =?utf-8?B?OVFaNkJHZm00ZS9objZlQTRtWEtSU3VCKzg5Y1pLZkZzVmM3ZVFNWXRJRUsw?=
 =?utf-8?B?c3YzUm4zZUk5YUErdkNST014dXpTbytISmY1eFBBWWtQVnYzczJPdEM5TjdF?=
 =?utf-8?B?eGVKWFFoODRPSFNrKzBLZVBWNnlMZGFSUXRWUWcxY1J2YnBRWFlML1NCblp3?=
 =?utf-8?B?SVFjV1kzN0tOR0o1MmI2aGFzUFBJSjJQU0pTVFZDZFowRCtvbTlKc3ltMjNl?=
 =?utf-8?B?WjhienUrMU1yN1grS2VJUVlnbndWVC9oL3dDcWdnSlAzNnJUUGtjek9ybkQ2?=
 =?utf-8?B?dzFIT1F2NHR4aTZMZFZrRXQ3UE02TlNCMUVEVDgxaC9ldzM2VnY0cTd5VHIx?=
 =?utf-8?B?bVA0c2J0Ukc1bmg5ZjJIdjE0SjhLMXBhbUNVWFRjbW9Bb3VFWDBDdnlRUGtE?=
 =?utf-8?B?eDJJdUlPOE5jQkFPQytFeUNvRXBqditsUjErZ0g5WkFOU2VXcmQ2eHJtb3No?=
 =?utf-8?B?VEJxcmlTVzM5bHdjVTMxRDF2SXphVTQxNWN5YmdSZHE0N0ozaXkxRTZObGpz?=
 =?utf-8?B?SzMxT2dCcWtSUVJtRUh1TGlJQ2F6Rmd5cmdUaWRTZUMwVERUWjQvMUlZUHBV?=
 =?utf-8?B?ajJTRk5tNkhPWHZvTWdQWHErTXBVR2hxVTNiWFZhU2xUZnJoUmk2YnlwKzdo?=
 =?utf-8?B?eTJBQXhQZmhTUjRxbTZwOHU2UXhoMi9idkhzRUZJTC8vaHJ1cW1tNkRVQ0pn?=
 =?utf-8?B?bElGUnJOUFRTTmMrU3VNbXdCcm9DN2l2S0tPZzA2RkZ6eUhmZlM1Q1M1aHVx?=
 =?utf-8?B?dTlWakxPeXA1L2dPL1VGSHllRFRsZ0taYW5CdGFsZ2tjT1oxMENkcUY1Vjcy?=
 =?utf-8?B?STN1NE9aL1hVSytzU2g3eW1SQkFpaDFqbzV2bzFkeGJucElsZVJibStFV1BD?=
 =?utf-8?B?V1FBVmFYUVJCWDZoby8xYkxpR3UzS09WWGZ4bGx1K2NQK01YNkpqSEZGUkZC?=
 =?utf-8?B?bzR4YlRySVhoa1hNVWxGRFc3ZDI3Q2pBY2tCRVBkMW5PdElYSnNNV1RFbkxm?=
 =?utf-8?B?dlFOM1N1TzE5emlHaFVORjh4VklvSHBCd0tQWEFtdkgyd1ZrN09TOFBGdXBa?=
 =?utf-8?B?cy8vVUFVNGJvMkhpTlVtK2ZWQUxTZ3A0NEdvLzlSL0YzZ25hQ0pVa2g2VERR?=
 =?utf-8?B?VHdCYjJISnJueC80VnJoLzA3Yk1jZXdPNzJ3Y1RwMXhXNisxbDVCVjhENnJt?=
 =?utf-8?B?MHREVW8zMzIrK2lyT2NFM1lKOUFPVDhoMko5RkMvZDd1NTk2WWxLcWNuNmp5?=
 =?utf-8?B?WGN5RnlMeVB3Z1l4T1lXYkQrcUVpNWVMWjN6YlhIU3ZYSloxNW4ydXBtRFZ2?=
 =?utf-8?B?WFV2NTgxa2R0NVpXQytTenJsa0JJcDBMMUdzay9XVks4YWVCK081S3pzbXRq?=
 =?utf-8?B?dFBWWDZSdGRMY2tOQ2FiNi9TNzBnZU9USG9mK2RNb20ySm5JYlZDbFgvVnVI?=
 =?utf-8?B?bWhMMk1sZjhIMDRXTFVKbWZxZ2prcXFDa2ZXNkwwcGw2RTF0WDNxaTZUMWJz?=
 =?utf-8?B?d1J3b3ppbnhVeUNTLzB6bE1XZWFpcHR0UytiVHhNUkRXRGJUT00wL2FQaVpp?=
 =?utf-8?B?Kzc1QnI3QjNxVEFBWkZLZDRURmt1Y3YxTVBneFRyTFd0VVExeEY4K1lmTjR2?=
 =?utf-8?B?cHEvZElGZ2VMayt1N3lXaUI1WGVwcUo0d3R1all6YkNrVE15ZUhjU0h2Z1dZ?=
 =?utf-8?Q?Y9H1fQuYtME6hP8nMZN7prs1bV3Yiz3/VDOgYae?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4704727c-023f-47c4-094c-08d97c1b5c12
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 09:45:20.1262
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iSn+9FumHWYYoR2Y59mILeg864EJOyRkoctmCL+YfY6OwU3ib8EnwaD5pTGrtkEQ9WHGTgA4lTIbxHD8AvaR9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

Hello,

originally I was puzzled to find that libxenguest's setting up of
struct hvm_start_info didn't have any explicit clearing of reserved
fields. Turns out the memory gets zeroed right after allocation.
Isn't this a waste of bandwidth particularly when considering the
kernel, initrd, or P2M segments, which may be large and which are
going to get filled with actual (non-zero) data anyway?

Thanks, Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:03:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:03:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190531.340333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSG9N-000506-EK; Mon, 20 Sep 2021 10:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190531.340333; Mon, 20 Sep 2021 10:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSG9N-0004zz-Ax; Mon, 20 Sep 2021 10:03:49 +0000
Received: by outflank-mailman (input) for mailman id 190531;
 Mon, 20 Sep 2021 10:03:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSG9M-0004zt-Cd
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:03:48 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0b9a13ae-19fa-11ec-b869-12813bfff9fa;
 Mon, 20 Sep 2021 10:03:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A0AF72004C;
 Mon, 20 Sep 2021 10:03:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 76661139F0;
 Mon, 20 Sep 2021 10:03:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id QViJG4JcSGFrVgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 10:03:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b9a13ae-19fa-11ec-b869-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632132226; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ivJziB2O5eFrbTydHKD9ChXEfjN0ENtyaE/rDNMKk0Q=;
	b=cZgMUi8gadqqqojok7RDjUHPHmU55H/Sv/j8a8W/FgBoU9GGROExHxlTI3pGQIV16Y8bAZ
	LMRAfIvGkJfhKjd8jFSogY2ylM0LvNLvjC9rWM7JgAJQp1ChOuN4H4PnIVsNncShIP782F
	3W9Ec46o2+0FgAgHYg1BRZkiNlZAuCE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH] xen/balloon: fix balloon kthread freezing
Date: Mon, 20 Sep 2021 12:03:45 +0200
Message-Id: <20210920100345.21939-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Commit 8480ed9c2bbd56 ("xen/balloon: use a kernel thread instead a
workqueue") switched the Xen balloon driver to use a kernel thread.
Unfortunately the patch omitted to call try_to_freeze() or to use
wait_event_freezable_timeout(), causing a system suspend to fail.

Fixes: 8480ed9c2bbd56 ("xen/balloon: use a kernel thread instead a workqueue")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/xen/balloon.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 2d2803883306..43ebfe36ac27 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -522,8 +522,8 @@ static int balloon_thread(void *unused)
 			timeout = 3600 * HZ;
 		credit = current_credit();
 
-		wait_event_interruptible_timeout(balloon_thread_wq,
-				 balloon_thread_cond(state, credit), timeout);
+		wait_event_freezable_timeout(balloon_thread_wq,
+			balloon_thread_cond(state, credit), timeout);
 
 		if (kthread_should_stop())
 			return 0;
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:20:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:20:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190545.340348 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGPn-0007VG-V0; Mon, 20 Sep 2021 10:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190545.340348; Mon, 20 Sep 2021 10:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGPn-0007V9-Ra; Mon, 20 Sep 2021 10:20:47 +0000
Received: by outflank-mailman (input) for mailman id 190545;
 Mon, 20 Sep 2021 10:20:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSGPm-0007V3-9l
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:20:46 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 69f176de-19fc-11ec-b869-12813bfff9fa;
 Mon, 20 Sep 2021 10:20:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69f176de-19fc-11ec-b869-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632133244;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=NE1NDT0O1Vox6buW2I/yZnEzDajwQBpWzQjV2RO5tAw=;
  b=eGpj4zHMBE/bU7sEWnGpZ8uDGnNdQRpblNaQ8bwxAiXIz559uWW3Yik5
   gISqfdi6QM27/9945Z80xBaeOTyMTCigvv6SEguz/xyoO9WoQv4Pa1udY
   VzIBVwQCXJ4vEqh3s9cTV392BOyLEAIawd3MBe0BTOBJAoJPvXF7UzO6g
   0=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: zWt2k0TqsCvW4d7392peLDWF7Rlagt08/Id8JBGFKnWWhSTCujneDlwjFcd4VgZhBdoMx83DbD
 0BXRAlBUVjaMLqoHlw2X98FCwe3HKEk3vrZF0WrGR2N7yoSktUSI9i1c+T9ncC3Rw3NL14sBbw
 C64zNtGo/XaunasqZjGuylnWWc/3VXqTBGIgtEjYbHOQp1VdGmKUVDBR0X/fwQF17Ha/xgSi+g
 RzSLNAhwFwdIj6zl3RUn040/U5Bgh4opgshhCahnd1LJMMlitQQyg5oNNB9j043sYbMjfWF2kZ
 gXdLe5zAmcq3rrzIAPA2DDkW
X-SBRS: 5.1
X-MesageID: 54896941
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fVk8jK8V3Il3ht8Ew7jFDrUDbnmTJUtcMsCJ2f8bNWPcYEJGY0x3x
 zEYX22GO/rfYmanKNt/Pdi/ph9X6sTdm9RiQQJt+Xw8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j2tYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPh3k
 /ths4WaTT0MEa7Lvro7Qj1cSjFHaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFgGdh2JESRJ4yY
 eJGVH1kNi74USdeBXQYNMI4x/6UiEjGJmgwRFW9+vNsvjm7IBZK+LrwNNvYfPSaSMMTmVyXz
 krd5HjwCBweMN2ZyBKG/2iqi+uJmjn0MKoNEJWo+/gsh0ecrkQRAhALUVqwodGil1WzHdlYL
 iQ86ico6KQ/6kGvZt38RAGj5m6JuAYGXNhdGPF87xuCooLV/ASxFmUCViRGatEtqIkxXzNC6
 7OSt4q3X3o16uTTEC/DsOfPxd+vBcQLBWlZSwMCVCEB2fzMo4YV0DTeatI6AJfg27UZBgrML
 yC2QDkW3utI1J5QhvTjpzgrkBr3+cOYFVddChH/Gzv/t1InPtbNi5mAtACDhcusOrp1WbVoU
 JIsoMGY8OlGJpWEjiXlrA4lTezxuqrt3NExhzdS83gdG9aFoCXLkWN4umgWyKJV3iEsI2SBX
 aMrkVkNjKK/xVPzBUONX25UNyjN5fO6fekJq9iONoYeCnSPXFbfoUmCmnJ8L0iyyRNxwMnTy
 L+wcNq2DGZyNEiU5GPtHI8gPUsQ7nlmnwv7HMmjpzz+iOb2TCPFGN8tbQrVBshkvfzsnekg2
 4sGXyd8404EC7OWj+i+2dN7EG3m2lBhVMiq9JQIJrDcSuekcUl4Y8LsLXoaU9UNt4xel/vS/
 2H7XUldyVHlgmbAJxnMYXdmAI4Dl74lxZ7iFSBzb1uuxVY5ZoOjsPUWe5ctJOF1/+1/1/9kC
 fICfpzYUPhITz3G/RUbbIX889M+JEj621rWMnr3eiU7cr5hWxfNpo3ucDzw+XRcFSGwr8Y//
 eGtj1uJXZoZSg1+J8/Kc/bznUiptH0QlbsqDUvFK9VeYmv2941uJ3Cjh/M7OZhUex7C2iGbx
 0CdBhJB/bvBpIo88d/og6GYrtj2T7siTxQCR2SCtOS4LyjX+Gan0LRsaufQcGCPTn7w9YWje
 f5Rk6P2PsoYkQsYqIF7Cbtqk/4zvoO9u79Aww14N3zXdFD3WKh4K3yL0MQT5K1AwrhV5Vm/V
 k6Vo4QIPLyIPIXuEUILJRpjZeOGjKlGlj7X5PUzAUP7+C4oo+bXDRQMZ0GB2H5HMb94EII52
 uNw6scZ5ju2hgcuLtvb3Dtf8H6BLyBYXqgq3n3A7FQHVub/Jol+XKHh
IronPort-HdrOrdr: A9a23:Zk8AlaqPOluCcVG/poK17EYaV5u2L9V00zEX/kB9WHVpm5Oj+f
 xGzc516farslossREb+expOMG7MBXhHLpOkPQs1NaZLXPbUQ6TTb2KgrGSpgEIdxeOktK1kJ
 0QD5SWa+eAfGSS7/yKmDVQeuxIqLLsndHK9IWuv0uFDzsaEJ2Ihz0JdDpzeXcGPTWua6BJc6
 Z1saF81kWdkDksH4mGL0hAe9KGi8zAlZrgbxJDLxk76DOWhTftzLLhCRCX0joXTjsKmN4ZgC
 f4uj28wp/mn+Cwyxfa2WOWx5NKmOH5wt8GIMCXkMAaJhjllw7tToV8XL+puiwzvYiUmRoXue
 iJhy1lE9V46nvXcG3wiRzx2zP42DJr0HPmwU/wuwqqneXJABYBT+ZRj4NQdRXUr2A6ustn7a
 5N12WF87JKEBLphk3Glpj1fiAvsnDxjWspkOYVgXAae5AZcqVtoYsW+14QOIscHRj99JssHI
 BVfYDhDc5tABGnhk3izyxSKITGZAV2Iv7GeDlNhiWt6UkUoJgjpHFog/D2nR87hdsAotd/lq
 L52gkBrsA7ciYsV9MOOA42e7rANoX8e2O+DIusGyWTKEgmAQOHl3el2sR+2AmVEKZ4u6fa3q
 6xCW9liQ==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54896941"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W+l/ff5Raoeomm5E1Q8LInZBD7tpPYZmHZH1XWqcGZJPUPYC05JX2j1LVoC5WjrwjLZwcAvduIlixmfQ0wZhvJx+gh9cx/eesU3R6ETYkNrGFc732WIGIY45BlTpEqO6IkDNFgyjpjqQVGWByXArHhL+k0e2Xid96Xn8maT29xjpIy+EYWpndWvcm9dVsrYCDh8nPm+5dxTr5qPQVzt1kkiDZPL3s65sqNpag9T3Vle8LegBDQF074y4ZahQ9czDKrCFlVJdtNbLtSN3y5QXE75+ptNkdozHwpHuJrX6h/glBi/uvYods64Ojl+61uJAUzDbdW3P5R5M1ItinLuotQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QFjOZkC/+Er8C/QZmW2cSOtkDavmbprnYBIx9h1t/pY=;
 b=j7bp+WKv+w2TjBgXiQfPbDhrkhIBtNgV1fzb6lVo9mHhyR2LP7fq7hwz0gO/r3BBdL/jRSBwr9BcXq6fhOfbl/6nSff73wcpv653nd+DWFAx+6LO/pTJLSEYfzGK2dkNz0x5FUxl543r5u3YKYC219AENrH4Ij57bwELQWc3pcFFyHclNr39p/u9h4lfr912ltlAL/oKbMlXuOloi4aoeAJvdRfWBZ5EDj1Yngjf2kkcPRemXeZvopYxo82F2KBD9mpfqXlnR/mZgJX4F9bjbv9/WWv6tPeoLJTbKDu5JV5l0zrOVdW6HgQdbEQ4tdhxiqxGY5AHzOv+uvwqCb5Eyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QFjOZkC/+Er8C/QZmW2cSOtkDavmbprnYBIx9h1t/pY=;
 b=Oh9oYRr7WVZ/M+PDCXEpjP42uBecac229rxetY7IDq/MUudcauVYzZL/VVIUbkdYt6Nv55UpaHnI+EGYzeuISuVe9Rrsi+qrkPXi6ru27Kks/s71s+mtrI26V+TEQ1J0wDCu4W7kRvh6JpOpHIpbysfXuOD1QCE/Wp+V8PykAVQ=
Date: Mon, 20 Sep 2021 12:20:36 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, "Stefano Stabellini" <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
Message-ID: <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
X-ClientProxiedBy: LO4P123CA0308.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::7) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f1086f08-235c-4852-d090-08d97c204c23
X-MS-TrafficTypeDiagnostic: DM5PR03MB2554:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2554B7E20A82A554F704337D8FA09@DM5PR03MB2554.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: BY1xk59L0io8ikbMHRQcQsF4yQoR4iW0W67/Z/4U+Ru+cuquXBXiLNTpddTZsW5/7xwrQiRnNc0K68cJokJAyA5w1xExsvGsXDuCUiOJ3QPOB4kCVZ1yQpSd/k7f6T7VMF7jB3QcLYJwucPLAdabW60eq8m908w20BOmF7kFJJUPLYw4ljMjvV66oPNNy2Wu1Qg1eQrll5tmXzuXM3FGDmYgQOkE7/xHn42YTNsyU6NBx2nVu+iTODmbHbEhnKaUj++5X2hlm/CCJEGCnk5MjGNWGDa6R+yGK+gdoKVWVUuS8u2O6w9mLmiJM7CQyvuzrx3d/bywlxVhYxI6wkKcLfI83vilk+jJ9g7uWAfPE/WviUn5wPJFPEblZIdernNzBok0lj0f0OcdabS0l1IVmL0iRxAbeUw4JJCIOclqc4hKtsVZ1MbO4zmQr1AYVr/LO0Wu9Wqs7o78PrOGfxHbadIGlnpcBOQ5wCpPq6Vfufdr0jnKsBjycjWK892zZqsTU794GNl3bKrAyuJqi+gO6XNL7nfWfweXUrByQvoMS1IBfuD1QnMv15XhIzPMvVdy0bzwsG1S8vLTlwrasOXOgIJrFZLfZ6wzBT2PUXP/k6B3+uKsYOzYD+M4dFCYHN8tRY39/yvyXhc7tM9Bbzrtdg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(6916009)(54906003)(38100700002)(83380400001)(5660300002)(86362001)(8676002)(8936002)(9686003)(316002)(6496006)(6666004)(508600001)(66946007)(26005)(85182001)(66556008)(4326008)(66476007)(956004)(6486002)(186003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aE9WdnA5RGhRQUw4ck1EZFBaSU1qRFp5MS93a21IY2k4TCtGMnV6Q2FmWnhB?=
 =?utf-8?B?M2lUT0JZb3h4YUtPa2kzZW9lc29mYlJWbitvTWVGeXlSUS9QaExEdXFnL2N4?=
 =?utf-8?B?akV4YjB4c1dWSmZjSkNYSzQ2S00weU16aFZDTUJqYnVSSUlpQUF0aFJTYTk3?=
 =?utf-8?B?cERjbkdpWndlWkRhSUdRR2NYd0t1N3pZbktpaGxrRVpMbVNaVWgwUmprKzBR?=
 =?utf-8?B?WXZqb2toc25QUmJhR1J0bmg5NTJQK2k5cmdLM09XZFdIYnNvVHR3V2hiUUhE?=
 =?utf-8?B?VWM0R0ppWlpCd2szYmx2ajV5UFRnekVQNTF6aHBkZjNKVXh0ZVlqLzI5cDhF?=
 =?utf-8?B?MmVmMjJzOUVGMFk4S0pscHM2TDhSVjNFRllkenpwMjV2R3lJckJySnkwalF6?=
 =?utf-8?B?NFdsRWUwVm9rSTRlRyt4MWRvdzVMbHJVOG5IUWlHVWw5bjIrcFZvbnYxa1ZE?=
 =?utf-8?B?Z0JrVkdLWnU0Tzl0WFNiUDZxQTRXczhUU2ZOZGJ6N2lZSzBmSWpxVmR6eUJs?=
 =?utf-8?B?cDlvVnAraUJSSlRTL3lsM3NEbVBnUFBjbHNaQWZsZkxaZ08vNEMvSXJ6Tmkr?=
 =?utf-8?B?MFNjejlnZk1MUEV5S2k5UDNTSXVMb3g3NDJvQkI4dzBERzdMdWc5QUxPSFcv?=
 =?utf-8?B?L0phWHdpMHpkdzhxNUlxU2pJVkpobkZLV1o0clhVeUJwVXZncWhXTE9BL1VC?=
 =?utf-8?B?WkF2VW14VmxwNjhURWdFVERkV2JFMXdKVU1HZjhtMUFYU1lEUWRIYkx3aVVT?=
 =?utf-8?B?UGhieGdqVmRlZXMwUWNBYkUrbEZJSzVRMTQrSFlqa0xrcXBCTExWVG1LQjNC?=
 =?utf-8?B?QUJLbDgxMnlBLzd0ZUs4QWlNUnBvbGtJTGlTODZablhtSkF1bjdFRCtLcjFU?=
 =?utf-8?B?alJZb1hWMXhnZmtxMldOTjVodlVPYjJnVEVveUpJWlpXMWUvODFjWS9OZkFC?=
 =?utf-8?B?S2pTcTRnVFdUUW0wRVZ0UDdtYmVKdXJQUlIxZE9EV0MvYW50ZGZYb2tVVVQ1?=
 =?utf-8?B?RC85bWZxSUZPUEpTa04rdE81cmlrN0N4Y0VkdWRqNHpaZGJvWlJoblBJaWxM?=
 =?utf-8?B?VlVaY1crdi9rcE53THNmMFhmTDhNN1lQdGJJTEVnVWFUbFphVm9pRWNsbThT?=
 =?utf-8?B?eGwzWFd4czhKNWdBTlVDVGlqZXp0YWxQQkZRZ2FpUjJUb0pCUHNaeVFpM2Yv?=
 =?utf-8?B?cTdFdHNVajJ1K2FRb1hXYUI0RmF6akl5ekJkbU81bHpkdFA0cmpjamRaMUd5?=
 =?utf-8?B?aTczMDM0WngzaFg5Yll0c1k3ZWpQUEd3S2FWRzV6Y1VCaUpqb2ErbVkrd2Ra?=
 =?utf-8?B?UWlQTTdTWllaWXBRRGpwUU1HcDFyNmsvWUtZY3NKdDF3ZFFjVDY2R2pzN2JG?=
 =?utf-8?B?am55RThTY1k5elBlODV1MFJnd1pGaE1iSm42cTBnMkN4REo4T2ZDVVgxdVFX?=
 =?utf-8?B?L24rUExRd1BISmR6Z203L3JrNGdEMll1R0hVbUxJL3F3djZjQlk5ejdEcFpS?=
 =?utf-8?B?WHdZTUkzNFVBZldLa20xLzQvaVd4aHNSc3g5MUhCdkVjUnIzRHR1ZHRyUWtR?=
 =?utf-8?B?amNOb3k0b3NCdHF4UVIzU3dMejFCMk9RdHV2ZkpqV3Ria054ZEN0NXhMNlV3?=
 =?utf-8?B?czZGUlNZaEJjV2QwMlBPU3lEa1dYZGZYc0tteHg5NTU3VHFvVE5oWGRYdUk1?=
 =?utf-8?B?UUlUWDlvOXVlTVZRb0I5d24rQVQzc0JNZ2xpNWJZam00OHAyQ3pJWkhoM0J5?=
 =?utf-8?Q?CMDRGUTviK2e8deOJ4sXUCfgZgok9pVnJE6MhTS?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f1086f08-235c-4852-d090-08d97c204c23
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 10:20:40.9601
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rjeYjm9J8l76UG+mnFZ9j0q+dku1B4kWz8ej/i15cxsLUR1XmNzhYx03koAG4W1MyJB6css/LIZgCaZvRvrzHw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2554
X-OriginatorOrg: citrix.com

On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
> Without holding appropriate locks, attempting to remove a prior mapping
> of the underlying page is pointless, as the same (or another) mapping
> could be re-established by a parallel request on another vCPU. Move the
> code to Arm's gnttab_set_frame_gfn(). Of course this new placement
> doesn't improve things in any way as far as the security of grant status
> frame mappings goes (see XSA-379). Proper locking would be needed here
> to allow status frames to be mapped securely.
> 
> In turn this then requires replacing the other use in
> gnttab_unpopulate_status_frames(), which yet in turn requires adjusting
> x86's gnttab_set_frame_gfn(). Note that with proper locking inside
> guest_physmap_remove_page() combined with checking the GFN's mapping
> there against the passed in MFN, there then is no issue with the
> involved multiple gnttab_set_frame_gfn()-s potentially returning varying
> values (due to a racing XENMAPSPACE_grant_table request).
> 
> This, as a side effect, does away with gnttab_map_frame() having a local
> variable "gfn" which shadows a function parameter of the same name.
> 
> Together with XSA-379 this points out that XSA-255's addition to
> gnttab_map_frame() was really useless.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -92,7 +92,7 @@ struct grant_table {
>      struct radix_tree_root maptrack_tree;
>  
>      /* Domain to which this struct grant_table belongs. */
> -    const struct domain *domain;
> +    struct domain *domain;
>  
>      struct grant_table_arch arch;
>  };
> @@ -1795,8 +1795,8 @@ gnttab_unpopulate_status_frames(struct d
>          {
>              int rc = gfn_eq(gfn, INVALID_GFN)
>                       ? 0
> -                     : guest_physmap_remove_page(d, gfn,
> -                                                 page_to_mfn(pg), 0);
> +                     : gnttab_set_frame_gfn(gt, true, i, INVALID_GFN,
> +                                            page_to_mfn(pg));
>  
>              if ( rc )
>              {
> @@ -1806,7 +1806,6 @@ gnttab_unpopulate_status_frames(struct d
>                  domain_crash(d);
>                  return rc;
>              }
> -            gnttab_set_frame_gfn(gt, true, i, INVALID_GFN);
>          }
>  
>          BUG_ON(page_get_owner(pg) != d);
> @@ -4132,6 +4131,9 @@ int gnttab_map_frame(struct domain *d, u
>      struct grant_table *gt = d->grant_table;
>      bool status = false;
>  
> +    if ( gfn_eq(gfn, INVALID_GFN) )
> +        return -EINVAL;
> +
>      grant_write_lock(gt);
>  
>      if ( evaluate_nospec(gt->gt_version == 2) && (idx & XENMAPIDX_grant_table_status) )
> @@ -4144,24 +4146,18 @@ int gnttab_map_frame(struct domain *d, u
>      else
>          rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
>  
> -    if ( !rc && paging_mode_translate(d) )
> -    {
> -        gfn_t gfn = gnttab_get_frame_gfn(gt, status, idx);
> -
> -        if ( !gfn_eq(gfn, INVALID_GFN) )
> -            rc = guest_physmap_remove_page(d, gfn, *mfn, 0);
> -    }
> -
>      if ( !rc )
>      {
> +        struct page_info *pg = mfn_to_page(*mfn);
> +
>          /*
>           * Make sure gnttab_unpopulate_status_frames() won't (successfully)
>           * free the page until our caller has completed its operation.
>           */
> -        if ( get_page(mfn_to_page(*mfn), d) )
> -            gnttab_set_frame_gfn(gt, status, idx, gfn);
> -        else
> +        if ( !get_page(pg, d) )
>              rc = -EBUSY;
> +        else if ( (rc = gnttab_set_frame_gfn(gt, status, idx, gfn, *mfn)) )
> +            put_page(pg);
>      }
>  
>      grant_write_unlock(gt);
> --- a/xen/include/asm-arm/grant_table.h
> +++ b/xen/include/asm-arm/grant_table.h
> @@ -71,11 +71,17 @@ int replace_grant_host_mapping(unsigned
>          XFREE((gt)->arch.status_gfn);                                    \
>      } while ( 0 )
>  
> -#define gnttab_set_frame_gfn(gt, st, idx, gfn)                           \
> -    do {                                                                 \
> -        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =    \
> -            (gfn);                                                       \
> -    } while ( 0 )
> +#define gnttab_set_frame_gfn(gt, st, idx, gfn, mfn)                      \
> +    ({                                                                   \
> +        int rc_ = 0;                                                     \
> +        gfn_t ogfn = gnttab_get_frame_gfn(gt, st, idx);                  \

Newline maybe? Not sure whether we decided that macros should also
follow coding style regarding variable definition followed by newline.

> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \

I'm slightly confused by this checks, don't you need to check for
gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
guest_physmap_remove_page?

Or assuming that ogfn is not invalid can be used to imply a removal?

Also the check for ogfn == gfn is only used on Arm, while I would
assume a similar one would be needed on x86 to guarantee the same
behavior?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:25:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:25:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190574.340429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGU4-0002nW-5K; Mon, 20 Sep 2021 10:25:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190574.340429; Mon, 20 Sep 2021 10:25:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGU4-0002nN-1E; Mon, 20 Sep 2021 10:25:12 +0000
Received: by outflank-mailman (input) for mailman id 190574;
 Mon, 20 Sep 2021 10:25:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSGU1-0002Ba-Tc
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:25:10 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff975f15-19fc-11ec-b86b-12813bfff9fa;
 Mon, 20 Sep 2021 10:24:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff975f15-19fc-11ec-b86b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632133497;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=YLpdQ302sJlbyW1N6F4FWflrQ/BIaUleO+XxIMSS7cQ=;
  b=iP4K9D/yZ1v5C1ipoYD+rz9kU6P4b4l83zzN0JdMhkcX/3d9fuuUYp5V
   MIojY28TXOdHVmVl43FHQqZm6DULPEgljm3JIevKVUXNJXFs3lmPOrk6A
   ST4xEBS1YLm9BBX7Jw/6SN60MFB5Uo8QKYwDpL7Byj/lPtpfhTVT4uLe1
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: TCGJyqO8gytrFeIpPRGs7im+weToOjOzYFqtEZjrWOCUlthe8ZegqGGKKxC8bkpv7DSibcETZx
 UT4iebUJf0zm6Jvv/EeanHyBoUxi/PNTLt5inSHX6rkZ1m+CwphFnpPNxBkdDIE6eCLpL2UQRd
 n++XLe2NficoawfseuZJfL0EXfo72k/vo0UMTrCAftW9JLGyb8VS/+iPGXt/SX7gT9KJaCJqwT
 4nqZX8rl8VgD0/ak9DdK0lxsW9hchg/JxeJ7SEPF8qN9rQp9RDWsizQ/CPUahrCug6wIl5vsxv
 es2WQ/vaJpM+yHKEqSYVGGDw
X-SBRS: 5.1
X-MesageID: 53102433
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ujfZ1amOH3mRRXh5ArhTUUPo5gwAIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJNDWuPO/uCMTD1eNgkYY/n/U5U6pSAx4RiSQI6pCxjESMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 ItBr625TRUOB5XvtOJab0MHAR1AYKITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQK2HP
 5BFMmEHgBLoUgQRNRAZJosFxuaXlHTVaRxThkvEqv9ii4TU5FMoi+W8WDbPQfSRXtlclEuco
 mPA/kz6DwscOdjZziCKmlqtme3njS79QJgVFrCz6rhtmlL77m4ZBQASVFC7ieKkkUP4UNVaQ
 2QL/gI+oK5081akJvHtUhv9rHOasxo0X9tLD/Z8+AyL0rDT4QuSGi4DVDEpQNAovcA5RDsu1
 3eSgsjkQzdotdW9S2+Z97qShSO/P24SN2BqWMMfZVJbuZ+5+th110+RCIY4eEKosjHrMW/82
 QGDqDoZvo0wochM6//k72z53j358/AlUTUJChXrsnONt10iPd7+Nt3wszA3/t4bc93IFQDpU
 Gws3pHEtbFQV8nleDmlHb1VdIxF8cppJ9E1bbRHJJAn6z3l0HqqZ4k4DNpWdRoxb5psldMEZ
 ibuVeJtCH17ZyDCgUxfOdvZ5yEWIU/IT4+NaxwsRoASCqWdjSfelM2UWaJ144wKuBN2+ZzTx
 L/BKZr8ZZrkIf0/kVJauNvxIZd0n3tjlAs/tLjQzgi90Kr2WZJmYe5eazOzghQCxPrc+m39q
 o8HX+PTkkk3eLCuM0H/rN9IRXhXfCdTOHwDg5EOHgJ1ClE9Qz9J5j646e5JRrGJaIwPx7+Uo
 SniChEHoLc97FWeQTi3hrlYQOqHdb50rG4hPDxqOlCt2nM5Zp2o4rtZfJwyFYTLPsQ5pRKtZ
 /VaKciGHNpVTTHLp2YUYZXn9dQwfxW3nwOeeSGiZWFnLZJnQgXI/P7ifxfuq3ZSXnbm65Nmr
 u3yzB7fTLoCWx9mUJTcZsWwwg7jpnMagu9zARfFe4EBZEX2/YF2ACXtlftrcdoUIBDOy2LCh
 QabCBsVv8fXpIox/IWbjKyItd7xQeB/AlBbDy/Q6rPvbXvW+W+qwIlhVueUfG+CCDOoqfv6P
 egMlqPyKvwKmlpOorFQKbczwPJs/cbrqp9b0h9gQCfBYWO0B+4yOXKBx8RO6PFAn+cLpQusV
 0uT0dBGIrHVatj9GVscKQd5POSO0fYYxmvb4fgveRio4SZ2+PyMUFlIPgnKgytYdeMnPIQgy
 OYnmcgX9w3g1UZ6bofY1nhZpzaWM3gNc6Q7rZVLUobkhz0ixkxGfZGBWDT954uCaokUP0QnS
 tNOaHEuW1iIKpL+TkcO
IronPort-HdrOrdr: A9a23:aodGc6jGj+yp7F2U26teC0MBkHBQXuAji2hC6mlwRA09TyX+rb
 HJoB17726StN91YhodcL+7VJVoLUmyyXcX2+ks1NWZMjUO0VHAROsO0WKI+Vzd8kPFh4tg/J
 YlX69iCMDhSXhW5PyKhDVQyuxQo+VvJprY59vj8w==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53102433"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JKZ6015f7lOjirwy8KMnLBtd1N3P4UG78gKgPE5JxyHRi4r1G2mw4uGdgU4bU7WG5cGyWWLly+oTMpywImENlKtUSaPqrCkHnak3bSn73En0uM0qnFFqpBTgOSrRdHBe+xIVZ2AxxhYXZx/jOGkG1miwxdJUu45mGFPejjrmgo82DOV/77NlANWCBGsUc6/dmaIONeOy2GtMC6CE3VabvA3fLBVzMHMh1o8Lu7LftGMK4uuloCjOo6XVOSCGY1YATT+8padLnA1+z3khUmp97bBnCRR2W8suHqvpLnAJ1YxHDFwwICKzfUMgk8bWfJz5o8Nam+a3iLJeBA8/dwI9NQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=52fKv83QhqJGhWASPxnT8whS4RezB2CT5RddIuOzg1A=;
 b=e3gbLLpDxLpG0JclEe2BTH22inzokOuSZ8w00nx2KxKBzdO1VKqe/zfrF/g881M5bjUnH8qUqWULtczASbsGlAlPeFzmddxYywZe0PfpqzifMtnzcUAO5627ajxeH+dGDwS0gHPOQBodMYfULCSWpOkdLP4OfSC2zKL+u1gWUuw+qca5VdIzksJsZk7sBagQrKqvkMH34mTFmHnus/S3Sr8CK9t5zndjtiwbQxOATrbyvaPIQTdvD6uXGQW8u6UrIHQ9G/KuynJipJLna2TKpUHo+LwvASW52MPCC8X2oE0OBDncl9CncdAKv1exEA/RAs3hRHLSK4fpE1RJHdz5qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=52fKv83QhqJGhWASPxnT8whS4RezB2CT5RddIuOzg1A=;
 b=QbAVcXKnWJjVTroa58AWR+SqFi8TbsDXUPkBaC8Sy4M82ONmmxjSDWbUx0Q9kSGRBlBdEhi3xM4bfZ0Di9njWp8CfBt0qjF5P3WLHEniHVtMmuWcHMFfObRGy+4gw1HoLeF3hxXqFGpXbkijPc+xO09pTCx3FSFNDRxHZPRjXDo=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-2-andrew.cooper3@citrix.com>
 <ed07d2de-9354-6c6d-b9f6-4b325ca328ce@suse.com>
 <0db4dc68-ccf0-0de6-61ce-7f155431a977@citrix.com>
 <8b2a548f-a955-302b-82ce-59c061537e27@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/6] xen/trace: Don't over-read trace objects
Message-ID: <b5c877ee-368e-efc1-da24-2e4ddc821e8b@citrix.com>
Date: Mon, 20 Sep 2021 11:24:45 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <8b2a548f-a955-302b-82ce-59c061537e27@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0438.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a9::11) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6d897989-7d72-464f-adb0-08d97c20e1f9
X-MS-TrafficTypeDiagnostic: BYAPR03MB3671:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB36717AA4882E39F833A4D2B4BAA09@BYAPR03MB3671.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 62o3kKDIXhXI203EFl2UGznbuKGvrx8VSW7fkah5536TosGJjpaxKS6GXta2Qzp8kkRx1EQz5z8jRWzuiDRg5875VszGYHLUoZNOXIrw/zcENftZBm5CBB1VCYIPyvD6QCsd89DXWPo+aF8e7XqF3SG8OfZPTNeXUhmTxsPO2oI0/0og5GKStKCxCIrzQHQTbMkYeq+FJDw331cAP8MeoaWT9FsiF8GXIDG22s3NGaOvY+5ZvKwiGUNGrTwod7juFdHkpp+cHPu5ixu+h+CfAKkX+thb8N386vuDnrZWpkY0SkODT/wobb8r0bZ7MQ0UJ4H0cTtuubQ+mESJ4t7YfZWWwV8Ri/39STvP0dvgohUuzSPcfLLfvz/TaS/7Tx3MqxoDW/BEw+UTp7vT2t69SS77A/1P6e116wFPchiwIm9r2gM7Bfqt7UPX7STyMsmkQ+nFTvHoTjOD2qdh+f/t2ksabAjmYT1X8hrilH+b5O5VQRs/iSof/CGO9MuDHY8fKdDRPlh2WyqQB6srhxFZUnjS+hyd6Pa8o5tdvphrvFSjx301q/i56JBg3dklnQ8hfO5Bax3ho2AQgWlmuENdD6kFoqTPdZTE3bT9RTR+ETylMzzibhGD9wndHF28p0HNjYMmx7Q5r3Mmn7NFb+GZi4/zJ54prY+f7kJVJYUZg49aczVaLRPWKO2TGOiA+2C5UBNonLJ8EGTtYIIwZF2EWX0YFpNEdYkfpm4ibq3uBDs=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(956004)(186003)(508600001)(86362001)(31696002)(316002)(53546011)(83380400001)(5660300002)(38100700002)(54906003)(66946007)(26005)(6486002)(2616005)(16576012)(6916009)(55236004)(66556008)(4326008)(66476007)(36756003)(31686004)(8936002)(8676002)(6666004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZFAwS1dKVHRFUHBaeDVsbkx6TkJ1YncvRUdkQnZTNjRIdEs2L2prRmVyY0xC?=
 =?utf-8?B?L0lYZWs1ZjVkazc4Tzd5aTFPQkhyK3Z6VTdQZ3RKQkR0WXFvL25uMVJNSk95?=
 =?utf-8?B?eXNLbWxvblVDK3RZUVlRTVFtWkdnWHhTSGZORlJDWXRJa2FZSU5zTXhjem9W?=
 =?utf-8?B?Nk9ud1VOekxoODJCVkRqMEwrUXhnaUk3ZzhIdzRDQjljVDZhZ3dqalNOQzZu?=
 =?utf-8?B?amlpWVFFTjc5NTF4UTFiMGdXdU1YY00vVjZVZEtRZUlWUGl1aGZxVjZZR0Ji?=
 =?utf-8?B?Q3l0a2RRRWpLcXB0M3RPcXZHY0xzRzFVMjFzRnM1alJhT0xPNTRlMkhyeVp5?=
 =?utf-8?B?Vk1yektVbjJmK3dvZzVDL1dIc2FDVGo2b2t1Z3BIaldtd2FZekJuOEFnc2Ru?=
 =?utf-8?B?aHdCU3BVMFV4V1BKejJkREJ5S3h3aUFMbU9KUXErVVhlVmtWWEtiZWR4VWoy?=
 =?utf-8?B?cGE2TVpSc2VkWUZVbitHd0lSNkVpU2JJa3loQW9LYnZ2ZkJ6THlHUVdMY3Zp?=
 =?utf-8?B?NnY3Nm8rNkwwUzRCSkJjYng5OHJ5bG4rRGoxejdPS0pUOXBvdXViYVREb2pz?=
 =?utf-8?B?WkJidU52V1NSWTZKS3NGb3pVSGx3SUlRT0ZQQWFWTHBnSTR5QjlqaTcxUklp?=
 =?utf-8?B?Nkova1U4dm5abTIzdnh4VEJvRlZGck9yanNUTzZMUDh6S0YvOUFWSXg1U1pI?=
 =?utf-8?B?UkJxQXZhdXJXSy9oSm9zTkVyZTFHSk0wNStaaTJScnNac0ltVUtEbmVtcG1O?=
 =?utf-8?B?cWg0dHBhZStVWEIrUUpEbm1pRUEzYzJkTXFGWFg5TVp5YWNySGhiQ2U1MTFN?=
 =?utf-8?B?dXZOVDVTQ1FlNFFwREVRNW5DbVNZZ0orZFVEdE1BVENZWDhzRUdRQjdlUEJE?=
 =?utf-8?B?K1l6ZHdTTExRYUhXcDFEZmcxY0RrRG5oWjN5QjErNktOY0ZxcytBZk5YUmFv?=
 =?utf-8?B?MkNQWFVFaTNrdlVxaVFGNXpnOTVIZzhTMjJqanRUbUFETHNtNWY2dU5zVHoy?=
 =?utf-8?B?Vmxrdlh1UnR2cTAzVWJjdnZ5eVhOK2FRZyszdlo4UXplVG8zWTBaelZWcWMr?=
 =?utf-8?B?Zzl1SDlDSXdjNFl2OVcrY0Fkb0orMlY0WURSV2FQcEtabFJnblhWa2hocEtJ?=
 =?utf-8?B?b2JzMHkxZWdEM3owTGI5czU0dXN4eHlvWVNKK3oyb0xYSFZ6MDV6RktJa2Jt?=
 =?utf-8?B?cGFabDFadTQ5dmNpZ3VQT3dETXVUdjFXckNPRmtVcVNadlZPOFVqUUlJVlV6?=
 =?utf-8?B?WTRveGRnWjdaM1pWYTkzZDR2V3lBUE5PME1VSitlK3Bqd3NvazlnNnRqQU5W?=
 =?utf-8?B?b3dxaEZnMEFmYjZzbHo1aDdCNjZiWG92WHpDaVBlSlNIMEgvdHRrWFRuMDNy?=
 =?utf-8?B?WTJzUTUxaStaVUpPMmlpSzd3ekNjNVFFMC82b1RRN1pYSEQybmZjbzY4QTNa?=
 =?utf-8?B?ZU5oNnV6N1YyWEJCOHdzRGhEdXVrTC9ZaFZxWnBjNkRZQVUwTGI2bzNubXE0?=
 =?utf-8?B?cjdSQUtJYVY0aVZPa1JydWxjQ0dRM0N4OExRSVVqdFNETFNkenluVWNZSDc4?=
 =?utf-8?B?WDhaUFNvb29LV0xudEFuOGU1NUlTbUF2UWJyVndiQUFsb2s3b05kMmNOclZk?=
 =?utf-8?B?Mkg4YmcwL2ZYWHRMWWxSenA5b2F3UWNaUGhvdFdaMi93cHozQ2pqa3paL2Nk?=
 =?utf-8?B?YXA3VldYTmUrNDl6SDJvYndnT3pkZmVsRGQ4cGIxbFF4amV0ck5XZGRzSDRx?=
 =?utf-8?Q?5teGLnrJ21XNCeBySlayjTlF1jBZCKI9J8jjdLK?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6d897989-7d72-464f-adb0-08d97c20e1f9
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 10:24:52.4668
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: l3rIPJLnM2G6pKhAhPi0naNQDU72uM4hW9kZbVv5bsIDgPSWKSBP6Ql/ZVNGsVi/p2dz3NWkeKmQFR+a0u0ZVGzd/ZfxLOF6PJ5gu4wpj/c=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3671
X-OriginatorOrg: citrix.com

On 20/09/2021 09:00, Jan Beulich wrote:
> On 17.09.2021 15:26, Andrew Cooper wrote:
>> On 17/09/2021 13:58, Jan Beulich wrote:
>>> On 17.09.2021 10:45, Andrew Cooper wrote:
>>>> --- a/xen/common/trace.c
>>>> +++ b/xen/common/trace.c
>>>> @@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsig=
ned int extra,
>>>>      unsigned long flags;
>>>>      u32 bytes_to_tail, bytes_to_wrap;
>>>>      unsigned int rec_size, total_size;
>>>> -    unsigned int extra_word;
>>>>      bool_t started_below_highwater;
>>>> =20
>>>>      if( !tb_init_done )
>>>>          return;
>>>> =20
>>>> -    /* Convert byte count into word count, rounding up */
>>>> -    extra_word =3D (extra / sizeof(u32));
>>>> -    if ( (extra % sizeof(u32)) !=3D 0 )
>>>> -        extra_word++;
>>>> -   =20
>>>> -    ASSERT(extra_word <=3D TRACE_EXTRA_MAX);
>>>> -    extra_word =3D min_t(int, extra_word, TRACE_EXTRA_MAX);
>>>> -
>>>> -    /* Round size up to nearest word */
>>>> -    extra =3D extra_word * sizeof(u32);
>>>> +    /*
>>>> +     * Trace records require extra data which is an exact multiple of
>>>> +     * uint32_t.  Reject out-of-spec records.  Any failure here is an=
 error in
>>>> +     * the caller.
>>>> +     */
>>> Hmm, is "require" accurate?
>> In terms of "what will go wrong if this condition is violated", yes.
>>
>>>  They may very well come without extra data
>>> afaics.
>> 0 is fine, and used by plenty of records, and also permitted by the
>> filtering logic.
> I was about to say that the two parts of your reply contradict one
> another, when I finally realized that it looks like the first sentence
> in the comment can be read two ways: "Trace records require extra data"
> then going on to describe properties, or "Trace records require extra
> data to be an exact multiple of uint32_t." Obviously this is to me as a
> non-native speaker. But maybe you could still reword this to be
> unambiguous? (I'm not going to exclude that the lack of a comma, which
> I did silently add while reading, makes a difference here: Does "Trace
> records require extra data, which is an exact multiple of uint32_t" end
> up altering the meaning?)

Yes.=C2=A0 The requirement is for "extra data which is an exact multiple of
uint32_t", not "extra data".

The comma massively changes the meaning.

I'll see about tweaking the wording.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:25:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:25:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190598.340440 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGUV-0003rB-Dk; Mon, 20 Sep 2021 10:25:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190598.340440; Mon, 20 Sep 2021 10:25:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGUV-0003r4-AT; Mon, 20 Sep 2021 10:25:39 +0000
Received: by outflank-mailman (input) for mailman id 190598;
 Mon, 20 Sep 2021 10:25:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGUU-0003qI-5u
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:25:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGUU-0004Yy-4F
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:25:38 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGUU-0005d5-2A
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:25:38 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGUQ-0005RN-NE; Mon, 20 Sep 2021 11:25:34 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=OBBSdiHztldZB/EBdjqdaA/K7bZRaJl+x5djZHDPwrw=; b=lAPoRXKrvPFXtOzKsJbaOgeqcG
	kmIQKF0PPNvdrQXZTvNM9OLwzXXNhF7XG+SW5By7C0sf976rA61mqBV/2PgO6RA3Gm6k5rywuGnf6
	rcPr4kCnJ3YrTnSx9gulyO+urgLqw7Y97uObIPEFgFtG6HUPclICBDk/UR76T3kL8nu4=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.24990.422751.855186@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:25:34 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] page-alloc: further adjust assign_page{,s}()
In-Reply-To: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
References: <90df9041-4f1f-9911-d64e-f8d7b2a99534@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("[PATCH] page-alloc: further adjust assign_page{,s}()"):
> The on-commit editing of 5260e8fb93f0 ("xen: re-define assign_pages and
> introduce a new function assign_page") didn't go quite far enough: A
> local variable and a function argument also would have wanted adjusting.

This kind of thing is why I really dislike our practice of on-commit
editing.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:32:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:32:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190669.340451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGbR-0006f7-84; Mon, 20 Sep 2021 10:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190669.340451; Mon, 20 Sep 2021 10:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGbR-0006f0-4Y; Mon, 20 Sep 2021 10:32:49 +0000
Received: by outflank-mailman (input) for mailman id 190669;
 Mon, 20 Sep 2021 10:32:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGbP-0006eb-V6
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:32:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGbP-0004gO-U7
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:32:47 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGbP-0006J9-T6
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:32:47 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGbM-0005TT-Jm; Mon, 20 Sep 2021 11:32:44 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=E4B127heE16WiQzfnSIV4THNFKniVsefWPf1gQnMdRw=; b=dS39rMhlqkKfO1MDDsH7JwgMlP
	jMqy5yDGgHsvvIJNF9leLR7xJO1IKMF+E7OFUx5oOjcGftibMph/H3eOpliR1NHJiDmX2BxqccbVL
	9guEg/IoS/ix6K+4VQUvTbQDQO5vXiXltxjLf+xtZym2hIWvMGlGOClYKPJFLsdKExtM=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.25420.346051.604198@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:32:44 +0100
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: <xen-devel@lists.xenproject.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH 3/6] tools/console: use xenforeigmemory to map console ring
In-Reply-To: <20210917154625.89315-4-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
	<20210917154625.89315-4-roger.pau@citrix.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monne writes ("[PATCH 3/6] tools/console: use xenforeigmemory to map console ring"):
> This patch replaces the usage of xc_map_foreign_range with
> xenforeignmemory_map from the stable xenforeignmemory library. Note
> there are still other uses of libxc functions which prevents removing
> the dependency.
> 
> No functional change intended.

Reviewed-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:34:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:34:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190676.340462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGcy-0007HG-Lh; Mon, 20 Sep 2021 10:34:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190676.340462; Mon, 20 Sep 2021 10:34:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGcy-0007H9-I2; Mon, 20 Sep 2021 10:34:24 +0000
Received: by outflank-mailman (input) for mailman id 190676;
 Mon, 20 Sep 2021 10:34:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGcx-0007H3-9r
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:34:23 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGcx-0004jd-95
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:34:23 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGcx-0006d7-8I
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:34:23 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGcu-0005Tu-33; Mon, 20 Sep 2021 11:34:20 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=+tKE82eIy0f699InwlfCUc6vRoD1PpPgEdrGJLTDL3Q=; b=srY2peH1G4gWpRNhrcbl3TxKwr
	XqCJ1IyJoUhsxqtscvX2dhVZswsK14I+U6PQP7fp2qV4oBTjCITS5AfAj16U4daPwI8uUwoOt8+Mx
	hP0pCXSHlIq1P4AUjh5KJFjAMSOJc0W5Wj+RgcxSUSJlFm1t0lXw828ZgSYP/LdhX8PA=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.25515.842754.402625@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:34:19 +0100
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: <xen-devel@lists.xenproject.org>,
    Ian Jackson <iwj@xenproject.org>,
    Wei Liu <wl@xen.org>,
    Juergen Gross <jgross@suse.com>,
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
In-Reply-To: <20210917154625.89315-5-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
	<20210917154625.89315-5-roger.pau@citrix.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monne writes ("[PATCH 4/6] tools/xenstored: use atexit to close interfaces"):
> Exploiting the talloc clean up routines to close the Xen interfaces
> is cumbersome, specially when atexit can be used to the same effect.

Wait, what ?  Why do we need to do this at all ?  xenstored can't be
restarted so if it exits we might as well leave whatever wreckage.

AIUI the live update doesn't end up calling atexit ?

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:35:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:35:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190682.340473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGe5-0007sB-0L; Mon, 20 Sep 2021 10:35:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190682.340473; Mon, 20 Sep 2021 10:35:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGe4-0007s4-Sr; Mon, 20 Sep 2021 10:35:32 +0000
Received: by outflank-mailman (input) for mailman id 190682;
 Mon, 20 Sep 2021 10:35:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGe3-0007ry-DL
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:35:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGe3-0004lN-CV
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:35:31 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGe3-0006kq-Bg
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:35:31 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGe0-0005Ui-5a; Mon, 20 Sep 2021 11:35:28 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=AywCxshs2A46tEZsdTT1hl1DjftwdGO+aYQ8DibDXdI=; b=bXaf4/yN3G9o26i2bIG5DkY3RJ
	+fK/490fjy71nYXMwkjL9ba1J0dXnjtkJU56Ja9PN9k8rT3bc11szLf/rXN6wDR4dL0R3uPIKOL7a
	yT1DXr/bT7fxRrWE4MFQUvA6n+HnPTEK0jOO6KTgNX5Kd/UQG78JXsGXxMvhznAJpMlM=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.25583.920124.88776@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:35:27 +0100
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: <xen-devel@lists.xenproject.org>,
    Ian Jackson <iwj@xenproject.org>,
    Wei Liu <wl@xen.org>,
    Juergen Gross <jgross@suse.com>,
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring as foreign memory
In-Reply-To: <20210917154625.89315-6-roger.pau@citrix.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
	<20210917154625.89315-6-roger.pau@citrix.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monne writes ("[PATCH DNA 5/6] tools/xenstored: restore support for mapping ring as foreign memory"):
> Restore the previous way of mapping the xenstore ring using foreign
> memory. Use xenforeignmemory instead of libxc in order to avoid adding
> another dependency on a unstable interface.

AIUI this therefore partially reverts something - can you point us to
the specific commits ?  In the commit message, ideally.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:39:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:39:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190715.340484 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGhi-0000KH-Ex; Mon, 20 Sep 2021 10:39:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190715.340484; Mon, 20 Sep 2021 10:39:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGhi-0000KA-Bf; Mon, 20 Sep 2021 10:39:18 +0000
Received: by outflank-mailman (input) for mailman id 190715;
 Mon, 20 Sep 2021 10:39:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSGhg-0000Jy-Ff
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:39:16 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 20cabdb1-1335-4cb2-8513-1ce2d0c1791c;
 Mon, 20 Sep 2021 10:39:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 64B2D2207C;
 Mon, 20 Sep 2021 10:39:14 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3028A139F0;
 Mon, 20 Sep 2021 10:39:14 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id Wli1CdJkSGHVawAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 10:39:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20cabdb1-1335-4cb2-8513-1ce2d0c1791c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632134354; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=grRSFnoZ9kXDdJqZQUyQDd60De6qFR68Iwa8zZOd7Rc=;
	b=mJCjv5z0RC3Iz89U14eunb/FDPoH+LYFFfOR9NW0/e5u86+Ls7LNEGhcQ7Uu6YX5oXBdfI
	6c+saq9iZRYtlSkqp7gsvRyTDIGkLD+YM/bCsjlp2YRdTj2XuuQ4+1+cg5V22n34ya0LUI
	tNcQZIz7chTIcE+1r8ncRqfmS9eSSVM=
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
To: Ian Jackson <iwj@xenproject.org>, Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
 <24904.25515.842754.402625@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <d1cf1ebf-f8c3-0f0e-8579-2173936792bd@suse.com>
Date: Mon, 20 Sep 2021 12:39:13 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24904.25515.842754.402625@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="LIHnmXpW7x3pCY2l126jMjgZ2EWVIbk1E"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--LIHnmXpW7x3pCY2l126jMjgZ2EWVIbk1E
Content-Type: multipart/mixed; boundary="4vMcJlHLLXnnGnuE7jRqhar20lWgCjwZH";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>, Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
Message-ID: <d1cf1ebf-f8c3-0f0e-8579-2173936792bd@suse.com>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
 <24904.25515.842754.402625@mariner.uk.xensource.com>
In-Reply-To: <24904.25515.842754.402625@mariner.uk.xensource.com>

--4vMcJlHLLXnnGnuE7jRqhar20lWgCjwZH
Content-Type: multipart/mixed;
 boundary="------------D40CD807B7DD0130EC126140"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------D40CD807B7DD0130EC126140
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 12:34, Ian Jackson wrote:
> Roger Pau Monne writes ("[PATCH 4/6] tools/xenstored: use atexit to clo=
se interfaces"):
>> Exploiting the talloc clean up routines to close the Xen interfaces
>> is cumbersome, specially when atexit can be used to the same effect.
>=20
> Wait, what ?  Why do we need to do this at all ?  xenstored can't be
> restarted so if it exits we might as well leave whatever wreckage.

In practice, yes. But having those cleanups might make it easier to
find memory leaks. Why do we have cleanups in normal Xen tools before
exiting? The system is doing cleanup anyway... ;-)

> AIUI the live update doesn't end up calling atexit ?

That's correct.


Juergen


--------------D40CD807B7DD0130EC126140
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------D40CD807B7DD0130EC126140--

--4vMcJlHLLXnnGnuE7jRqhar20lWgCjwZH--

--LIHnmXpW7x3pCY2l126jMjgZ2EWVIbk1E
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIZNEFAwAAAAAACgkQsN6d1ii/Ey+2
aAf9HfGLZRam369kMS/cce73PfDqlbAkXShaI/EXEtdasT1++e2LBRT7HXnnXZQhCALEFYQ+HqDW
IWAMc7k65hcEdjuB+65mT/rHkBW10crtel/7TW3P96X3zyHRznrJHk7eCTxW0HsToSUnJNbr81lJ
YrXIGlOuBeXrvgiqSZAdTEgq+kKRMwWKV2PBXSsRDYZV29AC3bUWFRO7XkOYIGnBYkc6zazCm6YT
pz99Cmkwa0zRs0nCVPYpj+RWrFt9Sc45nl6cWWGdYVFihqtkEcXX5zYbDTvmdMnixC2oJnJDoPfV
zhntdLyxay8p02A8+bew5+Mts0/os4pwcKnYLxRm8w==
=0/3W
-----END PGP SIGNATURE-----

--LIHnmXpW7x3pCY2l126jMjgZ2EWVIbk1E--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:42:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:42:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190723.340494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGkh-0001jz-11; Mon, 20 Sep 2021 10:42:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190723.340494; Mon, 20 Sep 2021 10:42:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGkg-0001js-UM; Mon, 20 Sep 2021 10:42:22 +0000
Received: by outflank-mailman (input) for mailman id 190723;
 Mon, 20 Sep 2021 10:42:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSGkg-0001jm-EQ
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:42:22 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6edee32c-19ff-11ec-b86e-12813bfff9fa;
 Mon, 20 Sep 2021 10:42:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6edee32c-19ff-11ec-b86e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632134541;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=RcS4Ias/qh+USI/Q0L0mrQZsXpX+mUfq8asXM59iWmA=;
  b=eUsJk/pfyrXdREIvalsKK2YM6zzqVQRRmMOw4LpytIqEKEkLlYceZpkO
   Jic3PPbkfWpAyo2XWy67TSjnzQOJborG3KqbPTFhBNJ70f7fopwsm43tO
   zkutXZe3Dqu9V+lmZd8sqDaC1d40lBtuwRVaB/xJNgM4mtyW6utiwHzjG
   c=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 4ey2PRfT1v/83B8s0szqxgn4fA+KHflC2asy9EYXjHgCxZpgQzXRsXWi3W+jFnZsVR5KIXM9rf
 LY/V+CFKO5yAlaoxDnHJo4Hro+/B3YnvIRup5v3Lh5GaMzb2I03i/MHBUOk8bND+36lztV2usV
 SEk1ZeI62em9lQk4PC1bXGJTsT1ZtGGrLLwC/P5d9RlihJM69yTbSQ0lnfQzvMmtDAZmtccjid
 XloV3DyUMa9V/gp3tnmdxeIwWuPR3fNQtCj31cI7KgYFOKTLNQlQxhxX9c27A6IH27Oof65exX
 F9YOOwVvEwXRcFsnPFQZyu9n
X-SBRS: 5.1
X-MesageID: 53096156
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:h8T7nKnOH15zjQxOZycp2Wno5gxVIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIfCzqEPf/bYGKnKdEnbom/pBwGu8LUzYA1SwJp+y82ECMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 PlfuMaIYDxwB4/rx94mVR8HAgZQZIQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpr3pEQRKuAD
 yYfQWBkZkz6fTdvBgZULJAis8OsnVn6ciIN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wp
 H/C/mn/KgEXMpqY0zXt2nOzhMffkCXjQoUQGbaksPlwjzW7xGMJDwYNfUCmuvT/gUm7M/pPJ
 kpR9icwoKwa8E2wUsK7TxC+uGSDvBMXR5xXCeJS1e2W4vOKuUDDXDFCF2MfLox93CMredA0/
 nObkonRLAc2iZ+yFkim8LHLvBScYAFAeAfuehQ4oRs5D8jL+d9o10KUH4c7SsZZnfWuRmqhm
 GniQDwWwuxJ1J9Vjf3TEUXv3mr0zqUlWDLZ8ek+soiN1Qp/eIftTIih81GzAR1ofdvBEwXpU
 JTpnaGjAAEy4XOlz3flrAYlRujBCxO53Nr02wUHInXZ327xk0NPhKgJiN2EGKuMDirjUWSzC
 HI/RCsLvMMDVJdURfYvPupd9PjGPYC/TI+4B5g4n/JlY4RrdR/vwc2dTRfLhAjQfLwXufhnY
 /+zKJ/0ZV5DUPgP5GfmFo81jO5wrghjlDy7eHwO50n+uVZoTCXOEult3ZrnRr1R0Z5oVy2Po
 o4DaJPVm0sAOAA8CwGOmbMuwZkxBSFTLbj9qtBNd/7FJQxjGWo7DOTWz69ncItg95m5XM+Rl
 p1kckMHmlf5m1PdLgCGNiJqZL/1BM4tpnMnJy08e12v3iF7M4qo6a4ecboxfKUmq7M/naIlE
 aFddpXSGOlLRxTG5y8ZMcv3ort9eUn5ngmJJSekPmQyJsYyWwzT99b4VQLz7y1SXDGvvM4zr
 uT4hAPWSJYOXSp4C8PSZK79xl+9pyFFyulzQ1HJMp9Yf0C1qNpmLCn4j/kWJcAQKEqcmmvGh
 ljOWRpB/LvDuY449tXNlJuolYbxHrssBFdeEkna8a2yaXvQ8F28zNISS+2PZz3cCj/5of3we
 eVPwvjgG/Qbh1IW4ZFkGrNmwK9itdvio7hWklZtEHnRNgn5D7phJj+N3NVVt70Lzbhc4FPkV
 kWK89hcGLOIJMK6TwJBeFt7NryOhaMOhz3fzfUpO0GrtiZ48Y2OXVhWIxTR2jdWK6F4Md99z
 Oos0CLMB9dTVvb+3g66sx1p
IronPort-HdrOrdr: A9a23:3YPw1quPZsaslin1oN4xD2Ib7skCkoMji2hC6mlwRA09TyXGra
 +TdaUguSMc1gx9ZJhBo7G90KnpewK6yXdQ2/hqAV7CZnichILMFu9fBOTZsl/d8kHFh4tgPO
 JbAtVD4b7LfCZHZKTBkXCF+r8bqbHtmsDY5pau854ud3ATV0gJ1XYHNu/xKDwReOApP+tcKH
 LKjfA32wZINE5nJfiTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1SvF
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfomoCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8A3eiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6NqeTgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeQ/003MwmMW9yUkqp/VWGmLeXLzYO91a9MwQ/U/WuonlrdCsT9Tpc+CQd9k1wg67VBaM0o9
 gsCZ4Y542mePVmGZ6VNN1xMfdfNVa9My4kEFjiaGgPR5t3c04klfbMkcAIDaeRCds18Kc=
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53096156"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Pb0NNroX+5SGXbUyspNLTbnhOUeVScw39LwqvtEGtLmdtU5bdU8c3nCEulVYM+QVEeHy/DBi7QVY31R0Y2OWXOvy8+YOVa33su3yxmL+OfiK3PcLPAW/ei/n+ey57u6zhUxia7KQ6ciq05wC3MPLFRBxrpOFs/JnDTi5OLgDv39//nrvm+WXzTa/Dg1MXBTZnUyGUeIIJZFBZikl3moOevVklBt1UGJNRtNgltd+8zfgpGrkBCnu9A/GrRXGD/3tnFjuhuCTCeBQvz5u0hNhHzqtpCfT5GHYvIzL1tcu7Ni+KUrz+P+YZxymqqA0NEdqojI1RwaN0x4CFG7s4XAIog==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HREBB8T+nUmy+x6pP6ZXUpeHUyTGeO6nLNdZ6ilPr0w=;
 b=OBNF/HaCghDtYdwqTH/h3i/0yXQ6NtU6zi0U3J3AoHi6ODN2JR80rIZD1vE23qAXaCC5IkCbYfJcNjb2W2Q+Lu9jdfSut09BNoD+BYSYA228L6Ax57eEr/5VuB3wKauWtxkqsVNOHd6DX46eX2POcFYZTqsvCTDeLjphyKqybGPTT+zoWMhhiWO7S77XNQo8iXcJ4lw8elJzFoi0v4S3zrGXIgHujJB7Pspn+E9SN01yIvA2I1ncHoF9iAS+0EnBxzA0XrwTQfikPRvrK5swAa79mgPMwIGxLuJgstIo49AbisVajX6Hp+CHVNjcQ7ikaUWAaH46g0wg6qwtRXBd/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HREBB8T+nUmy+x6pP6ZXUpeHUyTGeO6nLNdZ6ilPr0w=;
 b=SeF74RnD7eCc1xHbqhqLU84N8PLDvefFst0hfD3epG7NkVzN/Mi8cdTYbpNZdK4MxGpvJaFW++RhqoMIALeOx59XVfqGXJiC8SI/oArcDoS57gsOWkofjtkBUnA5KcnGs1rqaIS1fauCc2VVMr7vsA8ESDc5zJm9VvuirGBPJqs=
Date: Mon, 20 Sep 2021 12:42:06 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Juergen Gross <jgross@suse.com>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, Wei
 Liu <wl@xen.org>, Julien Grall <julien@xen.org>
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping
 ring as foreign memory
Message-ID: <YUhlfuPWEqnT0/42@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-6-roger.pau@citrix.com>
 <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
X-ClientProxiedBy: LNXP123CA0023.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:d2::35) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4f0a4a3d-9886-42c7-bb3d-08d97c234db0
X-MS-TrafficTypeDiagnostic: DM5PR03MB2553:
X-Microsoft-Antispam-PRVS: <DM5PR03MB255364E4B73D20DC4F7A07098FA09@DM5PR03MB2553.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: on+mgyz2DBjpkItOe/xtk0stwjV6wBF+rk0sG/NMtsREZPVxKRE4ndPxduIC3E9xNVp8ogqcbE3Qt1eioM+pIv4wLM62N+MkDVUIqSu/2hCUapelSwnBHm+foW4n49kXi8Wpi3YCnM61mmrF1owR9IKmkN3jgLnEiGDtwfGdGU8rJRbef9Dr4lJ+FjjZPu+INJ8szOCQ0BNLNRo6B6jiYnGCNMpt63m9tCw/2BwZrO7zZYpY92sZItRwZ4Kansd1LL8+Ua2uMYTwQQIoGyOMiSSgyU79Y8IpWXuBQ+caIgY4v8uF2KHlIMuDd5SiOAgAj1re4+RLktANia7Ens/8/PcSqE6bPVKxvqdkSslUm1QSjxJpjkw82efpWtorL7vb8W1vee1CIEz5sek7ghsT5V6auhpDysFVNjzC+2xt77Z+t26B0O1Pn6+XqdB3zZ8QLfbtgDkeDU9gJfO6xemtOt2BQgKBDTEDhCHRqjZVMNOmDpkDIlr0t6ynfjjjZIZ2EBibrcDmBNZboSVVdf06wgtl5xTZBnjfXMXWuc46S+Dc+PuR7Mxc44I6YIXT6SHpajeO9twTWUbICWpubyJokJemS75kAJULPGWVEolakV1lYPsCLoTBGHi55xrTNBKTUwuCf6tgDxcvwdnV3JGa8g==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(39860400002)(346002)(136003)(366004)(956004)(66476007)(54906003)(66556008)(5660300002)(4326008)(6916009)(66946007)(8676002)(8936002)(53546011)(186003)(26005)(6666004)(2906002)(478600001)(85182001)(86362001)(316002)(38100700002)(83380400001)(6486002)(9686003)(6496006);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TzBVRVBvQTFtNW9KekdEMnpubXphL091ZGhRVzFDTm1tbGZXUXFBZFlxRHdp?=
 =?utf-8?B?WXJpZTFxU1NMdCtvcEhYRnhYZVl1ZXE4Q3M0bnNTeGxHcmRBelV5MmFNaDVB?=
 =?utf-8?B?d0U0cmNDamx4NVg3VGFlK3lobHlNcks1bEhIVXMwTnUzVWYxNjF3QWx6ZGN4?=
 =?utf-8?B?ZmlFaFdCNFZNYW41U3g1Ni9BUVRadWt2UE4vUnpBbVl5WlJyaTRXcEsrUFBh?=
 =?utf-8?B?dWo4UGNZZ2FMZ0NQcVM5R2d5aE1jTFdSTkVWWURHU3Z0bFVZRHh4d3FpbjhT?=
 =?utf-8?B?OEJ1dUpzN1Fxbjg4MGxPYkxNSEYvdkRxNyt4YnNDYVdmTTIvWjRxWVFwZkZa?=
 =?utf-8?B?VTlsRjVqdjZVcXJ3ZFZ1MlplSzFzNTYxRUJHNHNxckk2elVrTDBMZTJreFhF?=
 =?utf-8?B?TFM5UHFmSFU3WHE0L0IyN3hIUm1YRGpFZjZIeWxWUVJpR2JMVUpHQ1hrRjB2?=
 =?utf-8?B?c2xjejZyZjZBb0xJN3dKM0tCcmdQTkhQb0NSRWVNQnhkYVdLdlgyUlBNM1dI?=
 =?utf-8?B?MTNLcFpWSFQzRzZqMmdJMHR5L3BmdnVUSUc4d3R2TEhQMHJTYzRHUXE4MFBT?=
 =?utf-8?B?Y3ZtM2U0QWRJZ1A0MXZGeEpTTjY0enQrWk1oK08xb1VPNDNUVzVCOVcwTXZ2?=
 =?utf-8?B?T3l5M011M1JVMHNRSG5MbW1GVzJoZ3VsNmMzZ1Z1Wjc5ZjVBeVZ0Q0FjVXRj?=
 =?utf-8?B?R0cyUytmcEsyKzZRWWIrUCtjTDRNMDVaeFRtZFVETFc1SlJFR0NCWnhxZWNi?=
 =?utf-8?B?NGRGak93bWhtWWx5eHRhY2VOV3pDZWR4ZEFBdStmMzlpRkg0bXNFa3MwQUtz?=
 =?utf-8?B?Y0RwVzBqdGFabXBrOGpTcng4SXNPTm9VSUU2eFRVMEFzVmlFdGs2NUxqKzl4?=
 =?utf-8?B?cmx3WldYVWJoSnNlNTJkOVRFRnBJdEI0TjgwY2dZSkQxSXBTSFMxelRXZFN2?=
 =?utf-8?B?Tjl6RElTNWF3RU5QVXhvNzNabjJSUUVtblZNNzFzY2w2Y2IvS2FtK3FVbnk3?=
 =?utf-8?B?cWhPMERjQ1R3OUhKMlhEdHVsVXorY3hXT2IzNURQU0hubVZONUJsRzgvanZn?=
 =?utf-8?B?bWhtV1VBS0ZCQ3FtN3F3MTlaQ2xFWTE3akl4M1dkLzd4R1E2Yms1ek5Uc1RZ?=
 =?utf-8?B?OFhJd3lOM2xLUWY4b1ZnK0Q4b3hQYThJYTk0d0xUUzVFeHIzYitMWmI3amlL?=
 =?utf-8?B?dFFHQlFaVkZqcmVSZHlHQ3pjQWdVTFh1YmZlK0pCQnEzUFR3Zk5RWVUwRlJX?=
 =?utf-8?B?U3BvMlBGOTBnUy9wejZveHE0RGtFMjE3V25CRVdvdjMyUnhMdzZ4bXZzRE1t?=
 =?utf-8?B?UjV4RkQvcm5XSVBXMmErZkRCZlNBMjdNUi9kQUM5dTlJOHRHTjc2OHVKOXVM?=
 =?utf-8?B?VFBmSGE4TXZoWk0wdE84YkJhb2NpdWxOWE5aY1dxN1FqakVMU1IvUnVSNThP?=
 =?utf-8?B?a2RIeExDTTdHWCt0dGNEdUViM3VYUVBrak9UbGFpWjJlbXVOUHRjQXNnWjBE?=
 =?utf-8?B?OU5GSmNCaE8zN2lWM29VR1d6dFdUQ3NXYzRGUzlka0ZiUE1qWGdBUUc3d1k2?=
 =?utf-8?B?NWJOenY2TGNNWVJ6a2NROExnMmFFSDZVeW5ma0tzN1A3WFJmYWRQdDh6TSsv?=
 =?utf-8?B?Mi9DNE9odW44aXYvMHBlU3JXbHdDdThhM2FJR3BXYzV4a2FDM0daM3JTdW9r?=
 =?utf-8?B?QkdUc3RuYUdpY0NodFV2V3oweUlydCtXUGQwZy9NUmhvdDBOU250bG9ucTRj?=
 =?utf-8?Q?8v1nhlPWTE50QybzPF2SD/GkDEYN7oRq0PEEcj6?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f0a4a3d-9886-42c7-bb3d-08d97c234db0
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 10:42:11.9729
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Hz4K8YU00hBP9pBrJWu2Xh/8u+XlFSIc47eSIocrf6AzIAVyt16L3ldTF+pjhDftKzaGhzxMiFpg3VURa2JFnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2553
X-OriginatorOrg: citrix.com

On Mon, Sep 20, 2021 at 10:24:45AM +0200, Juergen Gross wrote:
> On 17.09.21 17:46, Roger Pau Monne wrote:
> > Restore the previous way of mapping the xenstore ring using foreign
> > memory. Use xenforeignmemory instead of libxc in order to avoid adding
> > another dependency on a unstable interface.
> 
> Mapping a guest page via xenforeignmemory is no good move IMO. A guest
> not supporting a grant table for security reasons is a rather strange
> idea, as it needs to trade that for a general memory access by any
> backend without a way to restrict such accesses. This contradicts one
> of the main concepts of the Xen security architecture.

I've done this in order to be able to assert that the switch to
disable grant tables was working correctly, I don't intended this
specific mode to be something that is desirable or that should be used
in production, but I do think it's useful to be able to create such
guests in order to assert that the option is taking effect.

The main problem of xenstore not being correctly initialized on a
domain is that the "@introduceDomain" watch doesn't fire, and thus
other components don't get notified of the newly created domain.

This seems to be a limitation of the current design, where the only
way to get notifications of new domain creation is using
"@introduceDomain", even when the caller doesn't care of whether the
created domain as a working xenstore connection.

Maybe I can workaround this differently in xenstore, so that the watch
fires even when the shared xenstore ring cannot be initialized.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:52:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:52:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190739.340509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGu0-0003He-0q; Mon, 20 Sep 2021 10:52:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190739.340509; Mon, 20 Sep 2021 10:51:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGtz-0003HX-U8; Mon, 20 Sep 2021 10:51:59 +0000
Received: by outflank-mailman (input) for mailman id 190739;
 Mon, 20 Sep 2021 10:51:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSGty-0003HR-Ei
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:51:58 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c63350c6-1a00-11ec-b86e-12813bfff9fa;
 Mon, 20 Sep 2021 10:51:57 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A640C22040;
 Mon, 20 Sep 2021 10:51:56 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7A32E139F0;
 Mon, 20 Sep 2021 10:51:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 40cEHMxnSGHucwAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 10:51:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c63350c6-1a00-11ec-b86e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632135116; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=ovsJwCgE2oOX88DPBvaAJ9HrXG+ZgjILiELUqCGyC90=;
	b=GKqF7oCw2HTl3CB5jWEpVQHcgvg7SCmDD0d9cdS2IShPdBxvvzvCKEXNpJzdQH8Q6AeTVH
	HAO2GGovm4bJSWnf7U+Vf0OJCPmVoOlNVTJZHea+IeOna6oYjoT5NPPfN0KRjSUOhm2Bsp
	JWaQRCdWKadPQCPuXFlKq39GqS+irCQ=
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring
 as foreign memory
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-6-roger.pau@citrix.com>
 <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
 <YUhlfuPWEqnT0/42@MacBook-Air-de-Roger.local>
From: Juergen Gross <jgross@suse.com>
Message-ID: <1cff7e8e-613f-629e-99b9-0407767900ae@suse.com>
Date: Mon, 20 Sep 2021 12:51:55 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YUhlfuPWEqnT0/42@MacBook-Air-de-Roger.local>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="GX7ZRe8wthL4fhbkAXwyuXPehx8Kjc4L5"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--GX7ZRe8wthL4fhbkAXwyuXPehx8Kjc4L5
Content-Type: multipart/mixed; boundary="katGje8aBZw1iHOan691wrfy64l9KqL8g";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>
Message-ID: <1cff7e8e-613f-629e-99b9-0407767900ae@suse.com>
Subject: Re: [PATCH DNA 5/6] tools/xenstored: restore support for mapping ring
 as foreign memory
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-6-roger.pau@citrix.com>
 <b1fb5e04-29a8-c60b-c754-5a4275a0601e@suse.com>
 <YUhlfuPWEqnT0/42@MacBook-Air-de-Roger.local>
In-Reply-To: <YUhlfuPWEqnT0/42@MacBook-Air-de-Roger.local>

--katGje8aBZw1iHOan691wrfy64l9KqL8g
Content-Type: multipart/mixed;
 boundary="------------2F459516E5B601A825AE7E33"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2F459516E5B601A825AE7E33
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 12:42, Roger Pau Monn=C3=A9 wrote:
> On Mon, Sep 20, 2021 at 10:24:45AM +0200, Juergen Gross wrote:
>> On 17.09.21 17:46, Roger Pau Monne wrote:
>>> Restore the previous way of mapping the xenstore ring using foreign
>>> memory. Use xenforeignmemory instead of libxc in order to avoid addin=
g
>>> another dependency on a unstable interface.
>>
>> Mapping a guest page via xenforeignmemory is no good move IMO. A guest=

>> not supporting a grant table for security reasons is a rather strange
>> idea, as it needs to trade that for a general memory access by any
>> backend without a way to restrict such accesses. This contradicts one
>> of the main concepts of the Xen security architecture.
>=20
> I've done this in order to be able to assert that the switch to
> disable grant tables was working correctly, I don't intended this
> specific mode to be something that is desirable or that should be used
> in production, but I do think it's useful to be able to create such
> guests in order to assert that the option is taking effect.
>=20
> The main problem of xenstore not being correctly initialized on a
> domain is that the "@introduceDomain" watch doesn't fire, and thus
> other components don't get notified of the newly created domain.
>=20
> This seems to be a limitation of the current design, where the only
> way to get notifications of new domain creation is using
> "@introduceDomain", even when the caller doesn't care of whether the
> created domain as a working xenstore connection.
>=20
> Maybe I can workaround this differently in xenstore, so that the watch
> fires even when the shared xenstore ring cannot be initialized.

Yes, I think this would be the way to go.


Juergen

--------------2F459516E5B601A825AE7E33
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2F459516E5B601A825AE7E33--

--katGje8aBZw1iHOan691wrfy64l9KqL8g--

--GX7ZRe8wthL4fhbkAXwyuXPehx8Kjc4L5
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIZ8sFAwAAAAAACgkQsN6d1ii/Ey+L
XAf/aTbYYQAEuF8lCaunhhAf2CHcLul3xiEMJSA6Ahbefpp+MSbKSBwpE7xOxRAmwIW8nzUiaW7x
vIpuoOW7z73q6XD/yFfTjwypripPbwu+Q8RKBoV23amiysj7OgxxlNJG3nHGxw0jmTjUmI9G1Yer
4tHYqTGcMkihm94neLrRarE1sEs2RjI1imXLCtXFpu9Pk/Ok+CTI8Zq0u2LB14PkOLx5TfYm/KR+
DMt6PKYa3tkYULhFGB6Rryo0UIbkQO5SkRiVyH4Xl2KNvDw/Y6NzeQWAX7i7pW+ty/IzSoCDG3G9
4NzIEmzoLMZx7qT0XrSesof/EbM6rlCMY0XfxS/k2A==
=dJFI
-----END PGP SIGNATURE-----

--GX7ZRe8wthL4fhbkAXwyuXPehx8Kjc4L5--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:53:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:53:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190746.340520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGvk-0003uZ-CL; Mon, 20 Sep 2021 10:53:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190746.340520; Mon, 20 Sep 2021 10:53:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGvk-0003uS-9O; Mon, 20 Sep 2021 10:53:48 +0000
Received: by outflank-mailman (input) for mailman id 190746;
 Mon, 20 Sep 2021 10:53:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSGvi-0003uK-GI
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:53:46 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0652b07a-1a01-11ec-b86e-12813bfff9fa;
 Mon, 20 Sep 2021 10:53:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0652b07a-1a01-11ec-b86e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632135224;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=eIpQVoAWPDK76f3PRLpFpmQxugrY0f9lvwH9M+M46rg=;
  b=Z/kEi6KSds1RDPEnTGA4eh/1mbmPSYjue0idsDm/faTKRmsCWLK+vM3E
   +TBJ7XbnOInUTCuAAtZz2lu6DVGJCfzCPAwVdBcL0RyeccoVjbm1nM4lv
   B3sQUTNJ+SW3W44QX1WYktQAY5ycyMFYX+Q1hHliI+Ks2i6Ak3SJJ2zdn
   U=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: RAVCgpKBBnsDhrI9+xBzeb7pLzk4gcyjl0IPITZMFit6sK6PPCRlEN6eSxTEhryA0CzZeYlxsQ
 UC1R+9q+T5NgEMPLDQHNzEvHc0dGY7JU/U+XJkJzRyKqRqsfN/4IDNx+Sl2qkI/YbxTBW+E3B+
 iW2qIJIfcKPEyRCGTpiUwWmi6rF5Zx92cr98BwyM37llVf+Aq+3LDXdwbXEyVYAG2hPXOkrCoV
 7JuqXMRR45b0jrq7dg3FEOsnWEgATjg5WXltMPGwFRM0iZeR6zUVtP5vms+bed9dYXaUWIGzed
 CPgUYhvbFJvUZ+vz5ES0Z6a4
X-SBRS: 5.1
X-MesageID: 53522888
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:IGMRiKMYJY9chnjvrR1xkMFynXyQoLVcMsEvi/4bfWQNrUol1TMFn
 GEbDGjQP6mLYGrzL9kkYN7k/R4G7cWGztAwTAto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Eg7w7RRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYozOInNMuk
 YhJia7zYjYRN5aSlecReRYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YvNrick5atHiIasUu216zCGfBvEjKXzGa/iRuIUCjGdg7ixINd3VW
 ek0exFPVxXBSBgeBQk+N6M4pt790xETdBUH8QnI9MLb+VP70whZwLXrdt3PdbSiR9hRn0uej
 nLL+SL+GB5yHN2AyxKV/3S0nOjNkCjnHoUIG9WQ9PFwh0aI7ncOExBQXly+ydGph0j7V99BJ
 kg8/is1sbN05EGtVsP6XRCzvDiDpBF0ZjZLO7RkskfXkPOSulvHQDhfJtJcVDA4nMk3Rj91j
 wKNo+GqLg1XuYbLRHGR3LjB+FteJhMpwX8+iT4sFFVeuom48dBr1HojXf44T/Xk1YSd9SXYh
 mnQ9Xlg3e17Ydsjiv3jlW0rlQ5AsXQgouQd3QzRQmvtxQdwfofNi2eAuAWDsKoowGp0SDC8U
 Jk4dyq2t7tm4XKlznXlrAAx8FaBva3tDdEkqQQzd6TNDhz0k5JZQWy13N2ZDBwzWvvohBezO
 BOD0e+vzMYLYRNGkpObk6ruUp93nMAM5PzOV+zOb8omX3SCXFbcp0lTibqr9zm1yiAEyPhnU
 b/CKJrEJStKWMxPkWvtL89AgOBD+8zL7T6KLXwN50/8iuT2ib/8YeptDWZimchjvfvY/12Ko
 4gAXyZIoj0GONDDjuDs2dd7BXgBLGQhBICwrMpSd+WZJRFhFn1nAPjUqY7NsaQ890iMvuuXr
 Hy7RGFCz1/z2S/OJQmQMygxY7LzR5dv63k8OHV0b1qv3nEiZ6ep7bseKMRrLeV2qrQ7wK4mV
 eQBduWBHu9LFmbN9QMCYMSvt4dlbhmq216DZnL3fDglcpd8bAXV4du4LBD3/SwDA3Pv58szq
 rGtzC3BRp8HS1gwBcracqv3nViwoWIciKR5WE6Reotff0Dl8Y5LLS3tj6Bof5FQeEubnjbDj
 lSYGxYVo+XJsrQZytiRiPDWtZqtHst/AlFeQzvR44GpOHSI5WGk24JBDrqFJGiPSGPu9ay+T
 uxJ1PWgYuYflVNHvocgQbZmyaUyu4nmq7NAl1k2GXzKaxKgC696I2nA1s5K7/UfyrhcsAqwe
 0SO5tgFZunZZJK7SAYcdFg/c+CO9fAIgT2Dv/06LXLz6DJz4LfaA15ZOAOBiXAFIbZ4WG//L
 TzNZCLCB9SDtycX
IronPort-HdrOrdr: A9a23:C0v8+62OgWQISiqZX5wk4gqjBS5yeYIsimQD101hICG9Lfb3qy
 n+ppsmPEHP5Ar5OEtBpTiBUJPwJ0800aQFnLX5Wo3SIDUO2VHYVr2KiLGC/9SOIVyaygcw79
 YFT0E6MqyOMbEYt7eL3ODbKadZ/DDvysnB7o2yvhQdL3AYV0gj1XYDNu/yKDwGeOAsP+tBKH
 Pz3Lshm9L2Ek5nEPhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2mdfVmGJ6VMt1xN/dfOla9My4kD1jiVWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53522888"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fyBVWgihQPpJFdpahJn6SA7LSBaugD+k2Kc5rCRgJBU0flEV3MGHC9HpcZ75RGuxnTxOjFoYqYC9TA4cRrlaYceFEAXh1GQmcNe9MNqgk1jxWPGLwH7St3Wa5KMFPZMUPm+9goZk0hXPFmLMFpCCR6Pt+9gQWrcCY22ky217O6cKKngJgcvJgBDclfk747cFusviYlX33V2PtBVcPG9qHSH37oZj+/fLJW5x7lWr68CXbikvk4hJoaD7GtqkJ+A9a+416KAkAtFoNrtLU6ErUut3PI4sjkHkhdLK80nT2qazAPVDzGwTrlmqDw9BKXsTEhIElXOkbJjpONSOWFv9Pg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ea/QQyMu7ExwpRYNFalsAY59NvqeBjZYHdIQjejtLwU=;
 b=Fl/UzympctbNGmBKK2sjBk5pCgAQMAZ4XpZuZETMposdsMsQepgHp1hj9h6hM+8OySB2mvbwhalwAW5CqdWKyEwv+ga4yKCofrTurSYT1VWvooz7fZODT/6RBy5imRy3qCy7jzQjQZymFWahfdd+wGNAPKk0bYye4cZtQPVPVRr2u5B7OIrdoGyuMwlnKOOo5dnp9tygZYoNUE5ZwRUBZgbhyyYfGCJlKK83ZdMYbBxhJY23p1/Ry9sZSaM2Dy5Ss7OXglcj7um68lUu7wyc0c5hCfuIG26D1vL1al7wG1/t7uKRMo1Yp8NZoBoE/ROrWKaU0z9RoznyGCN3Zm+jhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ea/QQyMu7ExwpRYNFalsAY59NvqeBjZYHdIQjejtLwU=;
 b=Cpn7oC7aGTn3Dps/yKFDR4tE6bype8ZBtzTyMcNEz6CiP00wNuYMx9M4gKr5gm9bFLkWXYqIPddYZBHzEM6I52XzkHVA6RhxcX6AFOf4lj62AD7bgAwyaG69c2QcGgeO8gFfDTU+5CMpBsOARtLQ+Quaygh75UFvXEjMLZ+lLfI=
Date: Mon, 20 Sep 2021 12:53:34 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Juergen Gross <jgross@suse.com>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, "Wei
 Liu" <wl@xen.org>, Julien Grall <julien@xen.org>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
Message-ID: <YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
 <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
X-ClientProxiedBy: LO4P123CA0167.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18a::10) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 82f95343-9efa-48bc-5b4f-08d97c24e749
X-MS-TrafficTypeDiagnostic: DM6PR03MB4603:
X-Microsoft-Antispam-PRVS: <DM6PR03MB4603B9C76620434485C174BD8FA09@DM6PR03MB4603.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: dya1pNuQkT6bLBjXnQPrExlRSpgkN/mCNHNlXkDB+0ELzR7kynANuaHM0Sa7t7Rrjko86znJ06WP/s4z/t7MK1vLnGqTvtkz0SWyGFbJwdDM7UbrKHjCNi3BbOad67XAFBxZ1gc9eedUuYa8sMp9gQcjGaNmIAqHnSAwA10+Xu/zvaMqj0iet/v3TCGtDxTXJSxpbmefta3jE4Meh3V6dZNkQKmdQ8/IjqSjouSAleAkSVb/65GkVlO6jf1dAfpvxx4qcv1k2gPW06q7FVswuNz8VKwd7LDhSLtSzUyDQZeaQfGwZwdVfoQI4ZSUtTixEsiYvdBLZw6rTMf8Peu1Qovs850eBqUXAJZDoAUxWaTHgqdZVyjoXoZr2MJwr24kyPVddKfP+muLfs0Mw1sto8CgBAsPHDDHFdjB7K/lzGuI5jrZDj1x4gLjFr0deRQkUC4JJC2ehv90Psuf1t+PJjSLW71Y1QDPnyqJ2uR1yjAp3EpO+26Cra5ZDdnYVjM/Uo95on5KabQxE+glikZdMb/xn0w3g7XFGRijZQ1ka0iWhK9oKZYuzSAAs4cXzEcTCOXPNHKiyhqHaf63ovGdy9wXNcYM47g1nReVdATSFaJ6MnNbwrMmzbWPwAqxlKKPwD2j7bX2yDBmfhVE891evg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(5660300002)(8676002)(8936002)(6916009)(508600001)(66556008)(6486002)(53546011)(38100700002)(4326008)(54906003)(66476007)(66946007)(6496006)(316002)(186003)(86362001)(85182001)(26005)(956004)(9686003)(6666004)(83380400001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dG1QZVhYbXMzQkZ5dUZtVno3cUd1QmtnTDRIMVRHaDNXY2dQUnkvdTI0d3NM?=
 =?utf-8?B?L3pycVF5bzFYT2tweWpGVUl5b2hzL0szSUZidS9UbWN0SU5mc3c0ZEFJMzhJ?=
 =?utf-8?B?M2N4eWFtWFZLTGhkSGVYL1JJamZvbk44MEEveWtlby9CT1QrbE4wV1hUcVBo?=
 =?utf-8?B?YmtkK0dyWWtNSi83L2RiWmJ1UnNCd3g2ZGZjYnFWenJMV3V6WnBOc2xQU2E3?=
 =?utf-8?B?OWtmUkRMLy95czdHdkM2NDFuK2RHUXlCNERNQ1dtSWw5dzJiVFY0K1FaTERY?=
 =?utf-8?B?bHkzMUhqaUlmTnZla01XazNtOUtwU1p1YzZLQmNTb0tlRTY0aUNnRk56WmRX?=
 =?utf-8?B?KzY4RzNXVWpvYnZjb2lyRUpTVmwvSDU2UlBkNDJWR2hPOXgrZ2pZWjlzRnR6?=
 =?utf-8?B?a2RwdzZSQ3FDTUdvVVlNajB4c01TK1FwZ1BkaVFGWkNwYk5BRXNoeTBnR2ZS?=
 =?utf-8?B?UzVUb3p4V2JvWGkzM294OWd4bUNCSG12SlZFeGxYVXZzanZiQUNXeFRMaVhU?=
 =?utf-8?B?Z2hvYUc0N2NYUDFsdGlQSGRBTTEreHB5YmxGRDl0cSsyUHhCSlIwRVhGaTNZ?=
 =?utf-8?B?U1NERTRGRlNhby82TmpBdERZTG1ZNzNrT3FFcnBiVmJTSzkzbXIrL2xrcjhy?=
 =?utf-8?B?TUs5L1FOWkRVeC9QZ09QVnNBWTBHZ2t0MDVNZHAyZjdJRW9FTUpxN3FQWEVI?=
 =?utf-8?B?U0lIdkhCVVlNNnlETUdDMXNhYUladWNVMTRsS09zSjZ5RTE4dzQ2TjlMVEly?=
 =?utf-8?B?ZDk3ZFU4Ukp5b3ZLRzUvd3p5TU5DNUlGSkhDbGpCMm9mQ3UrbGUzNUlLdHEz?=
 =?utf-8?B?cGxuUHJvVVV0Q29HQmhIOC9uaWNMTzBHb01oU0wvNkV5TEt2VTRHczJ3SGxZ?=
 =?utf-8?B?OUthRzNSR21vMGFnck1Qc081LzNQSFM4cG15UTQ1anNlcFRWNmZvclBXbjFz?=
 =?utf-8?B?K1pUZ3hFZHVJRE1pbTFBanVETG80QkNlaHNhL1VnRVhKcm95NU9CaXNOcnBz?=
 =?utf-8?B?Nmo4Nm5XaEVxWUZneGJiMGN4VWYvcE9BazdZdVU0QnpEOGFGOCs2bHAxdnJJ?=
 =?utf-8?B?aitrOFdTQU90QUh5YkU3OFo3MzJ2bThTZjZXVlhxRlRWQ004MFRrTGo2RlNL?=
 =?utf-8?B?bDJ3WldjMkNuSjFjMVk5aG5Ib1hteTc1R0VBQkVBSWQza0s2RWJhUjZISlZC?=
 =?utf-8?B?OEt6TXR4aGUvWS9tOEtWeVJwS1ozOTNkVWlZVUJ5WkM4bTVoY2pEby9xYVZ1?=
 =?utf-8?B?L1dSNjFmS2FVd21wa0pIZEVGbk5OcUVSZWtsSWVHQUxwZ244SWhkZCtQQzRm?=
 =?utf-8?B?WTRXZlp4UWVoeklhOFlWbVVWTWdQTlZMRFFGQllKWEhPZzR2M0JOV1ZvWGh0?=
 =?utf-8?B?NmpoOXFyWUNnSlV5Z0ZKNFpFSTFqbVNhdHdQb0JFaTJEVXk5VktiK3ZpQm0y?=
 =?utf-8?B?cEFXRUdtdnNKcE5FYW9lUW5VZVNERWw3dmRnT0wwaXd5ZmtRcDQ5ejlwRDZm?=
 =?utf-8?B?aTFteVNvZVBIelVpOStpZ2FCNjlmdnNYbk9hN0ZaU1lzSTYrRFo2NkpzN2lV?=
 =?utf-8?B?SGUvTlNXUWgvZWExc2p3L29tTy9lWHFkNlhielo1LzBSNUx5YjFmbGtkcWdv?=
 =?utf-8?B?cTR5bjlORGk5UFBlTTQ0S3dZekxvR2dtR2NuaGQ0M1lKcm9ZYjYzcnRxS2tu?=
 =?utf-8?B?QUMwWXNyOTgySDY3L1d2VEdILzFXMG4zUWk4SS9oWWpIVWpUTllOdEtjb3pw?=
 =?utf-8?Q?LMCFWbphCX6qMOzGpfiZd9xvHuYsMdpGZny73fc?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 82f95343-9efa-48bc-5b4f-08d97c24e749
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 10:53:39.2618
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fmfdmPNR5Tg0+Q0YAL04qRXeDHNhtfiTeY0viNYrfVnIGg04CNVY3C+2fztCM2G1BFq5aaDF9P+3rgfbKAySUw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4603
X-OriginatorOrg: citrix.com

On Mon, Sep 20, 2021 at 11:22:15AM +0200, Juergen Gross wrote:
> On 17.09.21 17:46, Roger Pau Monne wrote:
> > Exploiting the talloc clean up routines to close the Xen interfaces
> > is cumbersome, specially when atexit can be used to the same effect.
> > 
> > Convert xc and gnttab to use atexit which allows to drop one
> > indirection from the storing variables.
> > 
> > No functional change intended.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >   tools/xenstore/xenstored_core.h   |  2 +-
> >   tools/xenstore/xenstored_domain.c | 57 +++++++++++++------------------
> >   2 files changed, 25 insertions(+), 34 deletions(-)
> > 
> > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
> > index 47e9107c14..8930303773 100644
> > --- a/tools/xenstore/xenstored_domain.c
> > +++ b/tools/xenstore/xenstored_domain.c
> > @@ -737,35 +725,38 @@ void dom0_init(void)
> >   	xenevtchn_notify(xce_handle, dom0->port);
> >   }
> > +void interface_cleanup(void)
> > +{
> > +	if (xc_handle) {
> > +		xc_interface_close(xc_handle);
> > +		xc_handle = NULL;
> > +	}
> > +	if (xgt_handle) {
> > +		xengnttab_close(xgt_handle);
> > +		xgt_handle = NULL;
> > +	}
> 
> Could you please add closing of xce_handle(), too?

Sure, I somehow assumed there was a reason for not closing it related
to live update, but I see that's not the case as you use exec to
launch the new image and atexit handlers are not called in that case.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:56:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:56:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190754.340532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGy7-0004bV-T6; Mon, 20 Sep 2021 10:56:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190754.340532; Mon, 20 Sep 2021 10:56:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGy7-0004bO-Pn; Mon, 20 Sep 2021 10:56:15 +0000
Received: by outflank-mailman (input) for mailman id 190754;
 Mon, 20 Sep 2021 10:56:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGy6-0004bE-5G
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:56:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGy6-00058P-2l
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:56:14 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGy6-0000JE-1d
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:56:14 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGy2-0005aE-J8; Mon, 20 Sep 2021 11:56:10 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:To:Date:
	Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=yOoSHejLBLZyTtXudnhRfgQ504IHPIpwIB7k6A/wacQ=; b=Tq0sF7BQSKLBKF27sYoougjp1V
	VF0LD/kVE/ZjYXHyMNy2LYI5FZZfFnHXG8R/HUmBlBa+DQq1l29iMctKAr/Fc9oo5ZFvXpkSxdZ25
	Bw8p8oi85Igbu/Gq/gMVpJ70TwqbRf4diKkD4RVyNEhdXJZevyYgqT87Y6LeLRe6GC54=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.26826.333349.562985@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:56:10 +0100
To: committers@xenproject.org,
    xen-devel@lists.xenproject.org,
    George Dunlap <George.Dunlap@citrix.com>
Subject: Xen 4.16 - Last Posting Date - 24th September
In-Reply-To: <24873.5576.45086.392600@mariner.uk.xensource.com>
References: <24873.5576.45086.392600@mariner.uk.xensource.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

As a reminder:

All new features and substantial refactorings, intended for Xen 4.16,
must be posted by xen-devel by the end of this coming Friday, the 24th
of Septebmer.

Patch series first posted after that date will be eligible for
subsequent Xen releases, but not Xen 4.16.


Full release schedule for Xen 4.16 (recap):

  Friday 24th September                 Last posting date

    Patches adding new features should be posted to the mailing list
    by this cate, although perhaps not in their final version.
    (3 weeks)

  Friday 15th October                   Feature freeze
 
    Patches adding new features should be committed by this date.
    Straightforward bugfixes may continue to be accepted by
    maintainers.
    (3 weeks)

  Friday 5th November **tentatve**      Code freeze

    Bugfixes only, all changes to be approved by the Release Manager,
    on the basis of a (progressively stricter[*]) risk assessment.
    (2 weeks)

  Friday 19th November **tentative**    Hard code freeze [*]

    Bugfixes for serious bugs (including regressions), and low-risk
    fixes only.
    (0.5 weeks)

  Tuesday 23rd November **tentative**   Branch off staging-4.16
                                        
    xen-unstable open again - with caveats to avoid release disruption.
    (1.5 weeks)

  Friday 3rd December **tentative**     Final commits (docs/prep only)
  Week of 6th December **tentative**    Release
    (probably Tuesday or Wednesday)

Any patches containing substantial refactoring are to treated as
new features, even if they intent is to fix bugs.

Freeze exceptions will not be routine, but may be granted in
exceptional cases for small changes (on the basis of risk assessment,
like any release-ack).  Large series will not get exceptions.
Contributors *must not* rely on, or expect, a freeze exception, or
release schedule slip.

If as a feature proponent you feel your feature is at risk and there
is something the Xen Project could do to help, please consult me or
ask here on xen-devel.  In such situations please reach out earlier
rather than later.  I will try to put you in touch with people who may
be able to help.

[*] The distinction between Code Freeze and Hard Code Freeze is a
matter of degree, not kind; the Hard Code Freeze data and associated
tighter policy text is indicative rather than normative.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 10:57:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 10:57:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190760.340542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGzR-0005FK-6p; Mon, 20 Sep 2021 10:57:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190760.340542; Mon, 20 Sep 2021 10:57:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSGzR-0005FD-3o; Mon, 20 Sep 2021 10:57:37 +0000
Received: by outflank-mailman (input) for mailman id 190760;
 Mon, 20 Sep 2021 10:57:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGzP-0005Eq-QE
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:57:35 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGzP-00059v-PT
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:57:35 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSGzP-0000Qo-Oh
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 10:57:35 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSGzL-0005ao-0w; Mon, 20 Sep 2021 11:57:31 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=00zV4tK66LUBzcj89z3S80Eid7EFag8wE6U1xr8Zs5g=; b=UDBpHhqk4IkGG8Tyw4PJU0iY3Q
	A/U+Y+R3sNyUKqDZVXQH5yQg/OEPX76K6f7qhgZ3KXeItlPZTu5VQdM2lqSkG+jQE3vrLnG3ttx79
	C40UciiYdAfhVottGv2FWZqgG3abWhh2Gu1SSGjkW2oMwp/FbIyz4Vm8qT//lgf842X4=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24904.26906.715083.732081@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 11:57:30 +0100
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Juergen Gross <jgross@suse.com>,
    <xen-devel@lists.xenproject.org>,
    "Wei  Liu" <wl@xen.org>,
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
In-Reply-To: <YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
	<20210917154625.89315-5-roger.pau@citrix.com>
	<90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
	<YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monn writes ("Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces"):
> On Mon, Sep 20, 2021 at 11:22:15AM +0200, Juergen Gross wrote:
> > Could you please add closing of xce_handle(), too?
> 
> Sure, I somehow assumed there was a reason for not closing it related
> to live update, but I see that's not the case as you use exec to
> launch the new image and atexit handlers are not called in that case.

Are these fds marked CLOEXEC ?  I don't think they are BICBW.
It would probably be good idea to make them so.  An fd leak might
result in one fd being consumed per live update operation.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 11:02:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 11:02:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190769.340554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSH4R-0006pt-TS; Mon, 20 Sep 2021 11:02:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190769.340554; Mon, 20 Sep 2021 11:02:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSH4R-0006pm-P9; Mon, 20 Sep 2021 11:02:47 +0000
Received: by outflank-mailman (input) for mailman id 190769;
 Mon, 20 Sep 2021 11:02:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSH4R-0006pg-09
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 11:02:47 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3a8a7855-97f9-4cf9-9887-b976f10b5291;
 Mon, 20 Sep 2021 11:02:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a8a7855-97f9-4cf9-9887-b976f10b5291
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632135764;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=uftEc9rMz6wnZWOUh6vz8YwXDilVvtcYd0ZzyCFcMig=;
  b=UxaQhyOWj6hOYSHe5foHkblwgDeN5/+ZsGXRsyglkXVMvrmLjeki4iQ2
   W0jBKkEN2y8DDVxafNWW58CJtz3JHeMlfJXvH659KQtyiovOyq+JA9t5o
   0dF98107fXHtVD+H/XZBhsl8B3q5JL0AvVuhpAD9sloBe7Rips+0n0HXi
   I=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: qX5kjBJrjxrLDa3GCfAWyqc6seaOQor1M+lF8PfbbOM2CNDypjRcP0UQbRCW1L9AyYptZRLked
 RubN3sr4Ijez4yRouJvq7kQyRi/oRIoQNbxnjt92j9zpqf1QxLGI8prxQx7TXdZ+wJN/FJZbaA
 DOSkUC/2prsjW5E/2PRnWXm5ezXW+gNnqmbgnwcQiAhijbzP2Xpss4huyINlLXJS5Kufyq0jfu
 4Gtq6maZmr75ivWVYHOhfLyNGy83Ln7JmrTArWiS5OyFAP2DIfag4jWuLUx5Zfwhfq/0ZFV651
 122b/RfLfH5sabAd28y1ZfkS
X-SBRS: 5.1
X-MesageID: 53523512
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Ks3aeKAQl8oQWxVW/6jkw5YqxClBgxIJ4kV8jS/XYbTApD0g12EOn
 zFJX2yBMv+MM2uhettyPoji9hhX7ZXTmodqQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00o7wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/zAmO3MFek
 9x0rLvqWF0DPIfBks00TEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFh2pq3Z0XR54yY
 eIweDVjQEnnPSRNPw4PI9Umt8qzvyXwJmgwRFW9+vNsvjm7IBZK+KP2LNPfd9iORMNUtkWVv
 GTL+yL+GB5yHN2AyxKV/3S0nOjNkCjnHoUIG9WQ9PRnnVmSzWw7EwANWB2wpvzRolG6c8JSL
 QoT4CVGhbg/8gmnQ8fwWzW8oWWYpVgMVtxICeo45QqRjK3O7G6xCmYNQTFAb9gnnNQrXjFs3
 ViM9/vrGDhuvbu9WX+bsLCOoluP1TM9dDFYI3VeFE1cvoel8NpbYg/zoshLV5KyzczaQjjJ2
 hfQvXIUhZ8eh+ciyPDulbzYuA5AtqQlXyZsuF6ODjn0v1sjDGK2T9f3sgmAtJ6sOK7cFwPY7
 SZew6By+chTVcnlqcCbfAka8FhFDd6+OTvAiBZEG5A7/lxBEFbyINgNvFmSyKpvW/vomAMFg
 meI4mu9B7cJZRNGiJObhKrrUKwXIVDIT4iNaxwtRoMmjmJNmOq7EMZGPhT44owQuBJ0zfFX1
 WmzKJ7xZZrlNUiX5GXvHLpMuVPa7gs/2XnSVfjGI+ePiOHFDEN5vYwtaQPUBshgtfvsiFyMr
 753apvboz0CAbaWSnSGruYuwaUicCFT6Wbe8JcMKIZu42NORQkcNhMm6elwI9A6x/sFyLigE
 7PUchYw9WcTTEbvcG2iQntidKnuTdB4q3c6NjYrJlGmxz4oZoPH0UvVX8JfkWAP+LMxwPhqY
 eMCfsncUP1DRi6eo2YWbIXnrZwkfxOu3FrcMy2gaTk5XphhWw2WpYO0IlqxrHEDXnitqM8zg
 7y8zQeHE5ANcBtvUZTNY/W1wlLv4XVEwLBuX1HFK8V4cVn39NQ4MDT4i/I6epleKRjKyjaA+
 RyRBBMU+bvEr4MvqYGbjqGYtYa5VeB5GxMCTWXc6L+3Mwjc/3aintAcALrZI2iFWTqtqqu4Z
 OhTw/XtC9E9nQ5H49hmDrJm7aMi/N+z9bVU+RtpQSfQZFOxB7I+fnTfhZtTtrdAz6NysBetX
 h7d4cFTPLiENZ+3EFMVIwZ5PO2P2etNx2vX5PUxZk77+DV27PyMVkALZ0uAjylULb1UNoI5w
 Lh+5J5KulLn0hd6YMybii109niXKi1SWqoqgZgWHYv3h1d50VpFe5HdVnf77Zznhw+g6aX2z
 ut4XJb/uok=
IronPort-HdrOrdr: A9a23:KLvlt651e17sb31w1QPXwXOBI+orL9Y04lQ7vn2ZFiY6TiXIra
 +TdaoguSMc0AxhI03I6urwQpVoIEmsuaKdhLNxAV7MZniehILFFvAB0WKA+UyuJ8SdzJ8k6U
 4IScEXY7ecbSkYsS+T2njgLz9K+qjizEncv5a5854bd3AMV0gP1XYdNi+rVmlNACVWD5swE5
 SRouBdoSC7RHgRZsOnQlEYQunqvbTw5dzbSC9DIyRixBiFjDuu5rK/OQOfxA0iXzRGxqpn2X
 TZkjb++r6ov5iAu17hPi7ontRrcenau5l+7f+3+40ow/LX+0KVjbFaKv6/VfYO0aaSARgR4Z
 /xSlwbTrlOAjvqDx2ISF3WqlHdOX8VmgDf4E7djn35rcPjQjUmT8JHmIJCaxPcr1Etpddmzc
 twriqkXjVsfGD9dQnGlq71vitR5wOJSLsZ4Jwupm0aVZFbZK5arIQZ8k8QGJAcHDji4IRiFO
 V1FsnT6PtfbFvfNhnizyNS6c3pWm52EgaNQ0AEtMDQ2z9KnGphx09dwMAEhH8P+J80VpEB7e
 XZNaZjkq1IU6YtHOlALfZERdHyBn3GQBrKPm7XKVP7FLsfM3aIsJLz6KVd3pDcRHXJ9upHpH
 3laiIUiYcfQTOeNSS+5uw/zvmWehTCYd3E8LAv26RE
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53523512"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G2yFAtAIAhN8CjLgLgdH6Fb+6cI5Lc/XPvidAMUMwLZnfhbp15JhrH79GCmJcXlCQNHVuSLD24DscxIKUkp6wuduzqgXZQQx4ETiTrz+HGaIjui5hYpXbCF1U367LgvtqnPtenk6qkkvlcZ2kBJhMc9B2pbZWfFqhF4iDVyvHHJPP5woeFjVOegGAHKorp0cPHCbV/m7z61oOqsvpPVnCXNC0tlrOQ0OBoDAnYVkNXnoJWe0I/+cWS4TVTYMqu9eR2hhW4pIoeRVU01hHWL5fdKDhUjbgs2Zq7K3jFxl5ZwYQMUipatzRg9qLjUaGbBFCQqBGKqmuAYCfS6C35cYRg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0x5ChsYwve31t/TPkJ37BAV51m9YgYbZ4FHOk28I/QU=;
 b=Ig9IBDpUaGohQ+f2mUIBOS17mNMTkhEnYz5XsUYy51s/Hd8bGRaS+wN50ohrmg2r4IQAdONwmmDiocm5+9IM5kpNZll1WRtOQVvXGuRvc4yKe4pRrhvJJfOX7/eVnjEvjG5+1EM2P9AYIELdFmTz/4co5EcQCMnizEatyscQ3z/YGkDsgpNdknPcpzMbI/VxE34DXP5SHkoSpgYF4OQ+JdUHrDsh3cPsmOJY4X0KKdKjz/npcwTsY48BPgsMKl5mr7Sc6kBu+Bk4SDzp4RIORhiW7ea/4oRT1AZSqD9eIkYl3S3FpSPl089+qa/idE41xELQyakurUh6csXe434DOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0x5ChsYwve31t/TPkJ37BAV51m9YgYbZ4FHOk28I/QU=;
 b=JA6gckcHBsPsOYK9YR+BXwPma45f5g2MtL6cfhYa+e5Ko5mixZm/em9pb5fnoBV4WCh5vZ44TVsvrmaNlVfrYaS3r2uSfdgstUpiqKPil2gzasdSFlUBPJ6co33vKXiiClcqXEcdrySzAWRF+tHxuyk/8sW+mUJwtszUgj8SKuI=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-5-andrew.cooper3@citrix.com>
 <ef2df9df-df4a-8f8a-3f69-88a027ba66eb@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 4/6] x86/trace: Reduce stack usage from HVMTRACE_ND()
Message-ID: <e116bff4-9080-1629-221d-f2ba3255d03e@citrix.com>
Date: Mon, 20 Sep 2021 12:02:33 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ef2df9df-df4a-8f8a-3f69-88a027ba66eb@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0199.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a5::6) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 730590a8-05e3-40ce-5858-08d97c2629c3
X-MS-TrafficTypeDiagnostic: BY5PR03MB5284:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB52845B9DF7BE4D9C9C8C5D05BAA09@BY5PR03MB5284.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: hSU2NKuLzKq6aRFtCBq4gX8mzaB3Gr8XmILIBCKinrhyffi4QjZboAE1BTpR3w/UzL6kW1GblVAyH2fbJwSbuef3IEOTWrQUcpovuIiYWHBamYtNUcOtUYwZBvF3gMx/09jVpkyvi1CiLPNtyHgR8igczyui6awf//FzZoclY5xppBoD/KVkMxzIsff3pDUkJZEY3jfVDrwL7QerqAqiQSFIS5ZVGtVc782SwzKKep94scshEe8E0IkbZJrO4QGnOf3bTnfWIxikEppN/sRyyA8raM+URQQTY+s+jss0omLrahhW6LGMzELRFTklVQKMHhZWG33D+IlFz6Zlx8t2rVjUbol4ozAyE1di/FBHCh3JeI3Jf+v6/yTmqv1T77jWIcVq6vk06GTBOLHWUnZvD+rwqRU8/Kmpn5PoGRKuqpehkK5jRTmZ4K8bvEvHPaTJtVsd4oeXeqCKcUZ+QQcb2USntzIIt8DyJm9CLcP1cOxs1xg5XCXsGQbAzPgiL3H85n7iKeZHoellhV5xs4C9lQ+3T/FStjXeGAQ7nBaCBJW/BrAzjfhdc2NBC7j6IxnoU5EpJFXMSl7DuJF8dJTbcm8w2JyiDGT5LKkbRDGaRteGeh8l5HIobHTZHOuB2m2DKhMCk0ui0AsoZaxHlR/NlYMkTYb7e1HQlgX+QN1UiTQIRrwzGRdswo7Svznsvt48epVDw9hW+0JBqFcq60MbZSrXSZeOnrgc+ABYbNg/A/w=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(136003)(396003)(39860400002)(376002)(366004)(4326008)(55236004)(38100700002)(186003)(31686004)(5660300002)(83380400001)(54906003)(66556008)(66946007)(53546011)(6486002)(66476007)(26005)(31696002)(478600001)(6916009)(8936002)(36756003)(86362001)(16576012)(2906002)(2616005)(956004)(316002)(6666004)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UnNIbmwwbkhpNDQzRlUvNXAvUFdvNDAvOG03cldyMXAwQnFQZzJJb0J6Z3ha?=
 =?utf-8?B?WUdXbGhoeWUreHRkeFRIRUtISVEvUml1RHByMVlHdGF2aWYvakhZcXNJZkVB?=
 =?utf-8?B?RmNiWFBsbmJHUHkyd0cyeVNKejlmczlpVHU0YjIwaDNxRytUenlGN21abUNr?=
 =?utf-8?B?czJSa2FnVG1JT010QWtiODMxNFdkUGdtdFlwVkxLS2VhUkJqQUYwRE42WTRN?=
 =?utf-8?B?d3RndFFQQzV1WDZ5ODJoaFJTOEZ6OHVIb3hqUDMwL084ZmNveWZqOXZGMXMz?=
 =?utf-8?B?OWErY0hGa3NWa01XWE9qNXdIdW04Y296V1M3TG5jYit3MDRQQ1FIdVlPaVpa?=
 =?utf-8?B?WjY1VFNQNi9GRmltOE5uOEZ6Q3JPdHcyeDZSNng4eGtQU3UwQ2J2V0VYZEpp?=
 =?utf-8?B?ek9Ga1pSOVZobGZOUU9FU1ZlcXNBaGVMcFlnT2ZNdHZ6VnlwK3VmOUVTRUQ1?=
 =?utf-8?B?NUU5SnVabk9lZ1hyNFBOYzFOQjFheHRlS0hMcFpSTHRJOUF2S1VnamRpckYz?=
 =?utf-8?B?NHdldG5YbmpPeFM5QXlkb0p6emxMUXVWL2tvVmx4TnVYaGNZb1VwakRzVnJL?=
 =?utf-8?B?VUkzODZSQXhwcFBKK3p6NGRBdUYxRmtBeHZJWW50eExta0ljei9oc3pHdCtl?=
 =?utf-8?B?dnBzVWpTVS9oQURwTVMxTkRZVU5WUTUvcU9aQUVUdWFmaUI2eG40VXBFL1ND?=
 =?utf-8?B?bmxyRGI5aWxacHVMSzJSRFlERDM4U2EybkV6Y0Qwak9KYmtpV0hNM0J5Szhm?=
 =?utf-8?B?cVp0VTAwRHRzZC9GSWFsTzJhd0RmdHdNUGNsd2ZqY00zcXFkNUF3S28zS2F5?=
 =?utf-8?B?Skl6NS9UR0VRdjZtL3hiNmY5RVZFMzIvUkhKYWxvK1RXanc2ZFBUcUFPOGlQ?=
 =?utf-8?B?cElQelBhNXVseThrMjZuelpucDRnU2hGeUhTZ3VlODU4czU0cGFRQWpodDl2?=
 =?utf-8?B?RWZyMHlWR0VhazY4YWt1RitpUExIMW9NQ0RTZkJVcmc3WHE4YjR1ZTFFcUQx?=
 =?utf-8?B?THdiNitKVVNvM0NZcGd6ZUlvbXVDbjYrY3lIU0ZNZG9nNmdJYjlFb3hQUWxU?=
 =?utf-8?B?b2FYTGw5N3BrdFd0eXJJVlQwSmxYbmkycEgwVkJBRnJxTmdzeWRqRzZibVha?=
 =?utf-8?B?Yi8zMGpMaXQ5Rjc5amEwM2pvbFQ5bC9zcXVpeGg0WXVCb0tXNm5udStwL0d2?=
 =?utf-8?B?amtoMUVLSm5FY1U4MkRnaEFTK2cwbmFNcnJPSGN6V3B2TUxPVEZ4T1ZGbW9M?=
 =?utf-8?B?ME4ySlNJRE5DU09Gc1FsUVY0ZXlpSzFtd0ZVdzhkZEdvbTFPRXVhcEc3N2FW?=
 =?utf-8?B?Sytoa1JubmNuUDVITlRvcEcxeTRidWFCaWRGRWt3dkNVZW5kOXVrUUVhOUlJ?=
 =?utf-8?B?d1NoN0hybzJQYXkvdUVVS0tVVm9YcmdDUGJWU2VYZ3RZY0V1RERrQVBIT2hY?=
 =?utf-8?B?T1hXVTZIVHg0Sm9BdlN5VVJWUmtaQUcrcVZRZjlEL0NVY29KNHdYcU1LeTNq?=
 =?utf-8?B?eUE5OTUvbHd6czdHdzlBcGp2bGRCUTdaZjFkd3k4WlM2dVRETWkzTHk1ZnR3?=
 =?utf-8?B?WkZwTS9xT2Y1V3RXYzhQdUM1ck5MMGhjTDQ3ZWNZbVVHSWdvN29hdHpTNTh3?=
 =?utf-8?B?eWV0RXFlZGhiRlJFczFwRjBhTXZabElKMEllQndBUWZ6aEZzM3N1Vy9LZERG?=
 =?utf-8?B?VkFyK2VKUnhnTVFkZDN4KzcwYnVabm00RnlkTWh4dmhrUXo1em9sZ0RhOXNw?=
 =?utf-8?Q?H/o658vEwAfUH6ryz7b9zEUI+4umFA1X+/D2Tjd?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 730590a8-05e3-40ce-5858-08d97c2629c3
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 11:02:40.2564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IaJgyNLTHVjpSmzAyy2mzmJAm9JAEvWE5q+qWcZR/JxGlz5H4LBY+DDx+pfZ1s2LMurxSMkv9uB8+pzID7EAOQIXwENbFeuhYPdrErFP7Yc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5284
X-OriginatorOrg: citrix.com

On 20/09/2021 10:05, Jan Beulich wrote:
> On 17.09.2021 10:45, Andrew Cooper wrote:
>> It is pointless to write all 6 entries and only consume the useful subse=
t.
>> bloat-o-meter shows quite how obscene the overhead is in vmx_vmexit_hand=
ler(),
>> weighing in at 11% of the function arranging unread zeroes on the stack,=
 and
>> 8% for svm_vmexit_handler().
>>
>>   add/remove: 0/0 grow/shrink: 0/20 up/down: 0/-1867 (-1867)
>>   Function                                     old     new   delta
>>   hvm_msr_write_intercept                     1049    1033     -16
>>   vmx_enable_intr_window                       238     214     -24
>>   svm_enable_intr_window                       337     313     -24
>>   hvmemul_write_xcr                            115      91     -24
>>   hvmemul_write_cr                             350     326     -24
>>   hvmemul_read_xcr                             115      91     -24
>>   hvmemul_read_cr                              146     122     -24
>>   hvm_mov_to_cr                                438     414     -24
>>   hvm_mov_from_cr                              253     229     -24
>>   vmx_intr_assist                             1150    1118     -32
>>   svm_intr_assist                              459     427     -32
>>   hvm_rdtsc_intercept                          138     106     -32
>>   hvm_msr_read_intercept                       898     866     -32
>>   vmx_vmenter_helper                          1142    1094     -48
>>   vmx_inject_event                             813     765     -48
>>   svm_vmenter_helper                           238     190     -48
>>   hvm_hlt                                      197     146     -51
>>   svm_inject_event                            1678    1614     -64
>>   svm_vmexit_handler                          5880    5416    -464
>>   vmx_vmexit_handler                          7281    6473    -808
>>   Total: Before=3D3644184, After=3D3642317, chg -0.05%
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks, but this is buggy.=C2=A0 There are direct callers of HVMTRACE_ND()
which need adjustments too.

There is also a further optimisation for the 0 case which drops even more.

>
>> Normally I wouldn't recommend patches like this for backport, but
>> {vmx,svm}_vmexit_handler() are fastpaths and this is a *lot* of I-cache =
lines
>> dropped...
> The change in size is indeed unexpectedly large for these two functions.
> However, what I find puzzling is that TRACEBUFFER is enabled by default
> (i.e. also in release builds) in the first place, and that it can only
> be disabled when EXPERT.

Its not surprising in the slightest.=C2=A0 TRACEBUFFER long predates Kconfi=
g.

>  More gains could be had towards dropped code if
> the option wasn't on by default in at least release builds. "Debugging
> or performance analysis" (as its help text says) after all isn't a
> primary target of release builds.

All performance analysis needs doing on release builds.=C2=A0

> IOW what I'd prefer to consider a backport candidate would be a patch
> changing the option's default. Thoughts?

I very much doubt that XenServer are the only people who use xentrace in
customer environments.

I'm -1 to changing the default in staging, and firmly against doing so
in older releases.

>> --- a/xen/include/asm-x86/hvm/trace.h
>> +++ b/xen/include/asm-x86/hvm/trace.h
>> @@ -67,38 +67,30 @@
>>  #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
>>      TRACE_6D(_e, d1, d2, d3, d4)
>> =20
>> -#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d=
6) \
>> +#define HVMTRACE_ND(evt, modifier, cycles, ...)                        =
   \
>>      do {                                                               =
   \
>>          if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )            =
   \
>>          {                                                              =
   \
>> -            struct {                                                   =
   \
>> -                u32 d[6];                                              =
   \
>> -            } _d;                                                      =
   \
>> -            _d.d[0]=3D(d1);                                            =
     \
>> -            _d.d[1]=3D(d2);                                            =
     \
>> -            _d.d[2]=3D(d3);                                            =
     \
>> -            _d.d[3]=3D(d4);                                            =
     \
>> -            _d.d[4]=3D(d5);                                            =
     \
>> -            _d.d[5]=3D(d6);                                            =
     \
>> +            uint32_t _d[] =3D { __VA_ARGS__ };                         =
     \
>>              __trace_var(TRC_HVM_ ## evt | (modifier), cycles,          =
   \
>> -                        sizeof(*_d.d) * count, &_d);                   =
   \
>> +                        sizeof(_d), _d);                               =
   \
>>          }                                                              =
   \
>>      } while(0)
>> =20
>>  #define HVMTRACE_6D(evt, d1, d2, d3, d4, d5, d6)    \
>> -    HVMTRACE_ND(evt, 0, 0, 6, d1, d2, d3, d4, d5, d6)
>> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5, d6)
>>  #define HVMTRACE_5D(evt, d1, d2, d3, d4, d5)        \
>> -    HVMTRACE_ND(evt, 0, 0, 5, d1, d2, d3, d4, d5,  0)
>> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5)
>>  #define HVMTRACE_4D(evt, d1, d2, d3, d4)            \
>> -    HVMTRACE_ND(evt, 0, 0, 4, d1, d2, d3, d4,  0,  0)
>> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4)
>>  #define HVMTRACE_3D(evt, d1, d2, d3)                \
>> -    HVMTRACE_ND(evt, 0, 0, 3, d1, d2, d3,  0,  0,  0)
>> +    HVMTRACE_ND(evt, 0, 0, d1, d2, d3)
>>  #define HVMTRACE_2D(evt, d1, d2)                    \
>> -    HVMTRACE_ND(evt, 0, 0, 2, d1, d2,  0,  0,  0,  0)
>> +    HVMTRACE_ND(evt, 0, 0, d1, d2)
>>  #define HVMTRACE_1D(evt, d1)                        \
>> -    HVMTRACE_ND(evt, 0, 0, 1, d1,  0,  0,  0,  0,  0)
>> +    HVMTRACE_ND(evt, 0, 0, d1)
>>  #define HVMTRACE_0D(evt)                            \
>> -    HVMTRACE_ND(evt, 0, 0, 0,  0,  0,  0,  0,  0,  0)
>> +    HVMTRACE_ND(evt, 0, 0)
> These HVMTRACE_<n>D() aren't this much of a gain anymore; perhaps down
> the road we will want to have just a single wrapper macro adding the
> modifier and cycles arguments, otherwise making use of variable
> arguments as well?

Same on the plain TRACE() side.=C2=A0 There is an awful lot of cleanup to d=
o
here.

Other findings include HVM records using the non-HVM helpers (to have
cycles included), and examples such as vpic_ack_pending_irq() which
duplicate calls vlapic_accept_pic_intr(), causing 3 trace records to be
written out.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 11:02:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 11:02:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190770.340565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSH4a-000791-5S; Mon, 20 Sep 2021 11:02:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190770.340565; Mon, 20 Sep 2021 11:02:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSH4a-00078u-21; Mon, 20 Sep 2021 11:02:56 +0000
Received: by outflank-mailman (input) for mailman id 190770;
 Mon, 20 Sep 2021 11:02:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSH4Y-00078E-JJ
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 11:02:54 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4d5a4e8c-1a02-11ec-b86e-12813bfff9fa;
 Mon, 20 Sep 2021 11:02:53 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id CB99521B6C;
 Mon, 20 Sep 2021 11:02:52 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A5F5413A81;
 Mon, 20 Sep 2021 11:02:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id R0xOJ1xqSGGWegAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 11:02:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d5a4e8c-1a02-11ec-b86e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632135772; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=+TVS2SqHFrmFSbMMppf0wvv3pK+xtBrNM0FRyOccvbE=;
	b=mXjaX6ZuwfyMitE6fZanTm6/xjs6/l86y+ja93vy8RQ0RJmD8z0OIpeyBLopECRT00zzIC
	9CD3GdFcY9IUNVhQpo36DOXYDMTj6cUSi5qbMzj21ghzJrsGLXDlcGhkGL6eV9F6C/Ln/3
	91lOZVoQ8PcARJ8RGtjBZ7/P6ZckL5s=
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
To: Ian Jackson <iwj@xenproject.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
 <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
 <YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
 <24904.26906.715083.732081@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <25710b42-b2e9-54a2-a510-d8e09dbb1c25@suse.com>
Date: Mon, 20 Sep 2021 13:02:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24904.26906.715083.732081@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="NByWTPrdaKC00GvXJ0H6WWw1BBxPhQA99"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--NByWTPrdaKC00GvXJ0H6WWw1BBxPhQA99
Content-Type: multipart/mixed; boundary="jyB29GWEhZhoI8seEmpN67tbzqdkNOsDl";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>
Message-ID: <25710b42-b2e9-54a2-a510-d8e09dbb1c25@suse.com>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-5-roger.pau@citrix.com>
 <90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
 <YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
 <24904.26906.715083.732081@mariner.uk.xensource.com>
In-Reply-To: <24904.26906.715083.732081@mariner.uk.xensource.com>

--jyB29GWEhZhoI8seEmpN67tbzqdkNOsDl
Content-Type: multipart/mixed;
 boundary="------------79ED079A120A1A2A6D87821C"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------79ED079A120A1A2A6D87821C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 12:57, Ian Jackson wrote:
> Roger Pau Monn=C3=A9 writes ("Re: [PATCH 4/6] tools/xenstored: use atex=
it to close interfaces"):
>> On Mon, Sep 20, 2021 at 11:22:15AM +0200, Juergen Gross wrote:
>>> Could you please add closing of xce_handle(), too?
>>
>> Sure, I somehow assumed there was a reason for not closing it related
>> to live update, but I see that's not the case as you use exec to
>> launch the new image and atexit handlers are not called in that case.
>=20
> Are these fds marked CLOEXEC ?  I don't think they are BICBW.

They are. I checked when implementing LU and just rechecked.

The event-channel fd is opened explicitly without CLOEXEC in order
to support LU (the new xenstored won't open it again, but gets its
fd via the migration stream).


Juergen

--------------79ED079A120A1A2A6D87821C
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------79ED079A120A1A2A6D87821C--

--jyB29GWEhZhoI8seEmpN67tbzqdkNOsDl--

--NByWTPrdaKC00GvXJ0H6WWw1BBxPhQA99
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIalwFAwAAAAAACgkQsN6d1ii/Ey/b
BQf+Ptu0E3SDZzKJ+yYurEIAUMzaa8B0rSvVisHdgliHCT1FTcUY6KwQVc5DuJlMN686fB2vDrKH
YFmInn0Z3Gasi2xt/UulzvOp0BUM4QwYtgMfdEcgVdQ3ubgvb5VQFBnSgPPIzmtqjRzqwt642hvo
iGhe9t7tzjatEwq9AtXaA8PtPwjsWDPhbJ0EhheZ2NSShq0TyapesgYcIO4mF8wzU8imAojVaCja
bHVTMiLp6D7jwNsqpuUTGnT85B6EpfPUbitBkIFaU3fCyRVZV63OPFLm24SfDdwjt15SpIAHcYu9
FY+4G/qt5SguvVnAfKEN/kCHk9/i0daggTXPQ93VmQ==
=l/5l
-----END PGP SIGNATURE-----

--NByWTPrdaKC00GvXJ0H6WWw1BBxPhQA99--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 11:30:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 11:30:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190786.340576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHVD-0002bp-FQ; Mon, 20 Sep 2021 11:30:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190786.340576; Mon, 20 Sep 2021 11:30:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHVD-0002bi-CC; Mon, 20 Sep 2021 11:30:27 +0000
Received: by outflank-mailman (input) for mailman id 190786;
 Mon, 20 Sep 2021 11:30:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSHVB-0002bc-HR
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 11:30:25 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2511c88e-1a06-11ec-b86f-12813bfff9fa;
 Mon, 20 Sep 2021 11:30:24 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 399572203B;
 Mon, 20 Sep 2021 11:30:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EF48913483;
 Mon, 20 Sep 2021 11:30:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id u9HUOM5wSGERDAAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 11:30:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2511c88e-1a06-11ec-b86f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632137423; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=XdH3uK3BHg3l2ywsZCNDi1CtvO2ZsydRlDR/fOYFywk=;
	b=fdKg0foE439xs4CJHdAKBDgj5zxakX+wtk5gvgxrqCwDra7QezF6VZqgqTntVTC+rq8kdh
	TEXs/EFAxTUmLPSLjFy0ynwL9Z9gSHlzKD3lybjJss4BMKtkedJO+GbC3rJk/vqhSi1+28
	gQxS2hecHkm/6Ghhxi6HB9vhp/ngYvw=
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Message-ID: <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
Date: Mon, 20 Sep 2021 13:30:22 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ZdyfyBR7MOUm3RfxfGJsHaZA5G1amdYmX"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ZdyfyBR7MOUm3RfxfGJsHaZA5G1amdYmX
Content-Type: multipart/mixed; boundary="glacxsSpWxu549JB1FOTUbpsSfoGxhXZc";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>
Message-ID: <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
In-Reply-To: <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>

--glacxsSpWxu549JB1FOTUbpsSfoGxhXZc
Content-Type: multipart/mixed;
 boundary="------------AADA71A647DD592FAA99BF4E"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------AADA71A647DD592FAA99BF4E
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
> Hello, Stefano!
>=20
> On 18.09.21 00:45, Stefano Stabellini wrote:
>> Hi Oleksandr,
>>
>> Why do you want to enable pciback on ARM? Is it only to "disable" a PC=
I
>> device in Dom0 so that it can be safely assigned to a DomU?
> Not only that
>>
>> I am asking because actually I don't think we want to enable the PV PC=
I
>> backend feature of pciback on ARM, right? That would clash with the PC=
I
>> assignment work you have been doing in Xen. They couldn't both work at=

>> the same time.
> Correct, it is not used
>>
>> If we only need pciback to "park" a device in Dom0, wouldn't it be
>> possible and better to use pci-stub instead?
>=20
> Not only that, so pci-stub is not enough
>=20
> The functionality which is implemented by the pciback and the toolstack=

> and which is relevant/missing/needed for ARM:
>=20
> 1. pciback is used as a database for assignable PCI devices, e.g. xl
>   =C2=A0=C2=A0 pci-assignable-{add|remove|list} manipulates that list. =
So, whenever the
>   =C2=A0=C2=A0 toolstack needs to know which PCI devices can be passed =
through it reads
>   =C2=A0=C2=A0 that from the relevant sysfs entries of the pciback.
>=20
> 2. pciback is used to hold the unbound PCI devices, e.g. when passing t=
hrough
>   =C2=A0=C2=A0 a PCI device it needs to be unbound from the relevant de=
vice driver and bound
>   =C2=A0=C2=A0 to pciback (strictly speaking it is not required that th=
e device is bound to
>   =C2=A0=C2=A0 pciback, but pciback is again used as a database of the =
passed through PCI
>   =C2=A0=C2=A0 devices, so we can re-bind the devices back to their ori=
ginal drivers when
>   =C2=A0=C2=A0 guest domain shuts down)
>=20
> 3. Device reset
>=20
> We have previously discussed on xen-devel ML possible solutions to that=
 as from the
> above we see that pciback functionality is going to be only partially u=
sed on Arm.
>=20
> Please see [1] and [2]:
>=20
> 1. It is not acceptable to manage the assignable list in Xen itself
>=20
> 2. pciback can be split into two parts: PCI assignable/bind/reset handl=
ing and
> the rest like vPCI etc.
>=20
> 3. pcifront is not used on Arm

It is neither in x86 PVH/HVM guests.

> So, limited use of the pciback is one of the bricks used to enable PCI =
passthrough
> on Arm. It was enough to just re-structure the driver and have it run o=
n Arm to achieve
> all the goals above.
>=20
> If we still think it is desirable to break the pciback driver into "com=
mon" and "pcifront specific"
> parts then it can be done, yet the patch is going to be the very first =
brick in that building.

Doing this split should be done, as the pcifront specific part could be
omitted on x86, too, in case no PV guests using PCI passthrough have to
be supported.

> So, I think this patch is still going to be needed besides which direct=
ion we take.

Some kind of this patch, yes. It might look different in case the split
is done first.

I don't mind doing it in either sequence.


Juergen

--------------AADA71A647DD592FAA99BF4E
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------AADA71A647DD592FAA99BF4E--

--glacxsSpWxu549JB1FOTUbpsSfoGxhXZc--

--ZdyfyBR7MOUm3RfxfGJsHaZA5G1amdYmX
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIcM4FAwAAAAAACgkQsN6d1ii/Ey/e
mwf+Inhfr1fSRVZZvt2yJsyusaHTDmPCOwkUiIckLA1PW8CGiCkYRH0zlWEb0frVjQy0WaJKqYwa
yJ8cUmGrja3VOHMpxXsIrWoo2ZZoN+ZT68ZWexDF/wuYHBq1bI7xlLzB01+bzoarUcyhKffyiprE
cqTHyAlgqeh7eornHYTDgzm/zyJ9ltaVL/h+wLTwQs8Hxpp/4w7DvgyznySizQan9JWM+WMg98Ej
ScUVm6+1Z6u8nod0OR3miDVhVgkCGYl3gAM0TNdCn2OKOAo/uRCHN9Uldi1mBejI/5RGMHg0v0Kh
+HJYqpkkAXoHDeeaRLExP4Da6lmwdjT5xb6PFHaQiw==
=NFIO
-----END PGP SIGNATURE-----

--ZdyfyBR7MOUm3RfxfGJsHaZA5G1amdYmX--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 11:39:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 11:39:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190802.340591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHdZ-0003Xq-CL; Mon, 20 Sep 2021 11:39:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190802.340591; Mon, 20 Sep 2021 11:39:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHdZ-0003Xj-9F; Mon, 20 Sep 2021 11:39:05 +0000
Received: by outflank-mailman (input) for mailman id 190802;
 Mon, 20 Sep 2021 11:39:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OU0u=OK=epam.com=prvs=9897642576=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSHdY-0003Xd-8Q
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 11:39:04 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0fe7d193-f0c5-4e21-baf0-afa5a6afb683;
 Mon, 20 Sep 2021 11:39:03 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18K9vsJd027588; 
 Mon, 20 Sep 2021 11:39:01 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b6nf58vwn-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 20 Sep 2021 11:39:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3425.eurprd03.prod.outlook.com (2603:10a6:208:3::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Mon, 20 Sep
 2021 11:38:56 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 11:38:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fe7d193-f0c5-4e21-baf0-afa5a6afb683
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kNchorLNEEEy5J9hDMuYsJcCLjjdS13dsjWYlq0fhubdi/SACFFxuHr2SLSzE6QeSb8HxjuX/TbP0aCJ6T/aszDG7n0tBejTOYJic+TblhT8vnTNiIKjuU0jO+TQ1HITHqZcGQd+KvV7e4ofxp8uo0yNFUjQQns0KuSw69kfQj2hGLehFrtH34QtJDx0fcBBGx+zSSdz5o4I9kyi1opBk7uwwSyjihUdg9INcRxpRTBAEUTf4mRyHOJAJ7jEsEIVxwmDffra9taPIhO/IPMYamit5pmLYnaTdjJXLpnHqL0plioH6VfhDiyg83tVP7fL7ZpaXvwceipnsPM8gFraJw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gLPEPmNqvodFxSNvrsQXjNALp/bhHYk2JHzMRRn8SMQ=;
 b=irsNS4Mzm5dXMc8WDlapYbOgAL0MxWd5AygZm/+4+8M9tFBVaQ5rko59HcJUvgNqQYp3cUYmFX2wngZ0ZXg4tTVfynmvZPG35XajZaZabr3rTfR8EpC7izWpKBHj1jshHO5dJvxG97a1KqDcB2ljn5RtAxwlQaeqNiAswPNyytEIH7/CIe8TTk8qmRlsVQCexxRk5XVZMAuX3yhDyj7YdQnSvMUZhDmExmNXlwr3ThormF3xiBss2eXgtcTNhwIzzG/w/7LshMsu1klgiwEtZBYJEYD4+dSIQzqFIUo7Et6EpM9TCBz11mddLis60+ewPyKxCnHdsA1kmjnHXv1VBQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gLPEPmNqvodFxSNvrsQXjNALp/bhHYk2JHzMRRn8SMQ=;
 b=eotOO9wzB38SfNj2ZPlbUDVleaWQk2HnZYKXQe08vGkcuXWtO+tyPZtVQajtqQNUaGeAvl4vn/abqFI9ETrwo7Rgz/47YOS33SaeyhU+25nPiRNFopJwAOAMkI5GW58zbo2TNMN3M6lK9S3SYwrHaaq6d1wybx1s94YdRhhj8xzHCJ3QMre5x/3+67CBQlhc3Ip0RMynjZb7XB9PE/E/kOQkbtnKJ4BQujavmeVXhPq72MgerNmPwW/9x00MfwHv9AsP/z/tS0GIaeDVIaIp2WZ/05T8MD+o+zf3dXGRyiU4MOlOpH+bT9KyggjXRE4/IxciA3TzUgtVLnD+DIRCZQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOA
Date: Mon, 20 Sep 2021 11:38:55 +0000
Message-ID: <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
In-Reply-To: <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: fe436960-046c-4877-1678-08d97c2b3abe
x-ms-traffictypediagnostic: AM0PR0302MB3425:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR0302MB3425621378B5E9C4EECC8B77E7A09@AM0PR0302MB3425.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 rPG4IowOL6wV33kmd8FXejJiL6xEcTJkC6FU8d7yPgTHOl7sTrqehyn8GUlKNmMEdA/JwSd5g/ceR0xn26UZi7UsyFGmQJdRA4NXkuN6RSsN61L8sMEfI7FYsJGSCt865kD/w9z1he3mMJFfXhU+ruyZvJHUScaDwS/vfkkzvfnyNulOdEntEjqxo6pegfAKaQUrOjkb901Csq62+bylQy48xFfObLT1cCM+hzf2Q+MmyYq0MnXYpNdNWyipYD1H+kU9gZfOq/CmklP/0oLdCHjLW8NoS00wixUFFGxJ9SjpQ7YSVGuP1hoBX3C9yGFh/wKCckw7Pmulk238zu2L8IN1VnVGfZ+xNSxJSPXcU9frO/a5CgPh9p/OlAEQnU76u5eT///c6B7LqES+Nk6tCjMC1c4QcSdfLaz7xFMoQt1MB4XtQB9hnVDfVjchNThEeKpQTFVWwcsuTPq/9ly/M196ANEqEL/nCP3Ax+hn4c6vTEA0PYBchJF5Oquowvhp9+meg4peZulqZ7RowU11WMMFXzJIc4YN5WudgUNeKo5dZ1Xx2orkUMBbLHa1ZhRYssDast0Myn5mHZL5aENm4O41Q2GzhvODhjBPjVeGv8usYoP7kVzSU7hwJMUzHJLJebl4wubU0RqgAWwxaXr5OvjuFWirp18xQD5SRK23O/svBJoYUsN1gqlvU7dYoHLPQs0kZRPnaVCrghk67X67cDtx0jWrugmDtZHtTk+1PBAWWCY9scE4Mm3llAc+PcY+
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(36756003)(6512007)(26005)(6506007)(83380400001)(2616005)(8676002)(38100700002)(31686004)(8936002)(122000001)(186003)(53546011)(508600001)(5660300002)(6486002)(66946007)(316002)(54906003)(86362001)(66446008)(38070700005)(66556008)(71200400001)(66476007)(64756008)(76116006)(31696002)(110136005)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?THVWQjZKbmU1QzdHMFFqTVc4MEErY2dBSjZkVlg4MldjRndkNzZwUjlqMFY2?=
 =?utf-8?B?aUd3Vm0vTDhtVVpqWmZhZmxhRTFzSUJzaXBuVC9JZzZXRVNrRlpyV2t0OHox?=
 =?utf-8?B?eTRSYjNFRkF6RUVqTFVBSzg4UFRpNDUwSXRVOW1QdjdPMFk5Z1hIODNlK0ZE?=
 =?utf-8?B?Y1hIWEZlNms2bFdqN25sVTZKZ3BMV21nazIrTm9QT3ZXVDgvdmQwN1U1YUt6?=
 =?utf-8?B?VUp1V3dJcWV4blFsRkxOZE15aUM5eUZ1WlB6M0toOVpGdURaRnZlU2F0OFlr?=
 =?utf-8?B?NVd1WGNIQTFwNlg4dXZsWWxXcDVvQW5WUy9oTlExWTlJNFo1RWx4NWgvWEJ2?=
 =?utf-8?B?WmY4OEpjNjQ2STNoSTYyTm9KM1R5WnRqdlNFUE1SWExYQVVLZGxwUzBjR2cx?=
 =?utf-8?B?cWJ6YU13QUd1S1c4R0lWUTZYVXNjT2VjSFdwSUJ4dStxRzI5c3BvNHhxaDk1?=
 =?utf-8?B?L1g2M1lIOWRBUm8yaTVlUDNUdmFoM2NVendtL2hmK21mSmk1bUllblNmR3po?=
 =?utf-8?B?SEpRbzU5R0IxWGhwQ28wQ25CRWhVSW5XaEgveGpja2NZRE9mTGhHdW04UmlT?=
 =?utf-8?B?cVlBVlR0NlBsMmNnbkdkQzRqSGFPZFdlTS9mSC9EaXdmZEtpS1psZmRocENj?=
 =?utf-8?B?WjEvU0s3bnJURzBFTklpdmZLNHhXTkZjZ2JYY21zSUNHS1pCMW9oNkVRVFB4?=
 =?utf-8?B?V2lTaDZCc1dLaWY0YjlxZEM2czFnbzZ1TzI5Y3RldGtrUnV5WDMvdUo2WFRC?=
 =?utf-8?B?ZG5KbWpOUUxJdEF5aE9abEVweTFVQjlVMmJZeUdLb0I5ZG1aYjBUMVROdFJT?=
 =?utf-8?B?SGVmNUVFbi9yNG15Y2pFNWxPTU81U1c5M2FtZ0NDN3pFSUxCS1EvYkdVMEdP?=
 =?utf-8?B?UlpBTERWVnFzUm9UM0EvS1VaYmhrL0tWcm1JcW1kVDZkY3Y0SUtacXArUFB3?=
 =?utf-8?B?NDU4QVhNR25uVHIxMkxCMDdEVXU1L2hOSXJHR1hzM0UxUEI5SzIrVHQrWnRM?=
 =?utf-8?B?ZzFZcnFqS2JsUmtORkZ4SHV0ZFlFQTZ6Z2p5akNsTHVEcitJNnpJQ3pxZzdx?=
 =?utf-8?B?NmJXS2NVSHorcUhZTFRPY1h4cWZFWVBRd2QxUnRqNlM5ZWo5ekJBamZvOUhQ?=
 =?utf-8?B?bTEwd0JkSTJ4S3FsaXJFSlJJUDQrVEd0aXk0MThVY29rWGRkVmUwVzcyV0pC?=
 =?utf-8?B?UWh1VFFiT1JGVHBYVXhkTG9qUk9hdDJnekk5VFZpaW5DajB5bFlmUnFWNlUy?=
 =?utf-8?B?NXk3OXVIdFp6bFB0OFRpTzk0dE4rNGR2L0dWZThBbm5vKzZ2V2JLWE5LZ2Zl?=
 =?utf-8?B?bEZ5R3FhQTBZcUlmVjg4TGhzcVlQL1U4azVXN054UnVOdUU1TlJFNjVsM2xU?=
 =?utf-8?B?WGtKT3lwSjNuVGYwSC95M2ROelpMeEdBRTNLcGo2Mlp1UjdKZWxUMnpCaEF0?=
 =?utf-8?B?YzlUaDJ5Z0tvbFJPRDREVk9DejZGeEFqMjA1d3NQY1UvQWk0cVlDTlpLVFEz?=
 =?utf-8?B?RDdQOUJmVzFtV2VuSkgxYi9oRWlTeHlOYUNvV0dHbjdqK3ZMUXRZVVdZeFJp?=
 =?utf-8?B?aTVvQTFRdVczYjlqbVZ0czcrT2wzVmkzQVo3TG9yUFlJUDM1MFZGNEhFRVNH?=
 =?utf-8?B?VjhaSzJWR1g0aGpFK0haRmdkVXp5NEZ1aW4rN2FpOUpZTTVSdDNzSWdidjN6?=
 =?utf-8?B?L1lxSG5YS0pzVWtKSEZFTXNoT0IyNitRUTd3UGRKaWh6M2p0aVpzRGNPMjBu?=
 =?utf-8?Q?wZ54eA/QuHlRGKIJSuAd5bmVQB68vlrKQQ8DlfT?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C9385FCB10ECAD4DA369973DD1032D68@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fe436960-046c-4877-1678-08d97c2b3abe
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2021 11:38:55.9693
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FHqEuwk4/397eSOjdrUmf/ceNyBsmpvlNAwVB4mX0RNa43nr8SB2Fh3zp8TazhAcyGzP7DRIyDSQq0fqWX4aesk07HUYPOz/zRqD8gVGZ1k/jU20t7MYLSoqs3k2/SK8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3425
X-Proofpoint-ORIG-GUID: yubpBfw5fDnaiQ5sWxWSjt1GkKUtA1-D
X-Proofpoint-GUID: yubpBfw5fDnaiQ5sWxWSjt1GkKUtA1-D
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-20_06,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 impostorscore=0
 lowpriorityscore=0 suspectscore=0 mlxlogscore=985 phishscore=0 bulkscore=0
 adultscore=0 priorityscore=1501 spamscore=0 malwarescore=0 clxscore=1015
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109200073

SGVsbG8sIEp1ZXJnZW4hDQoNCk9uIDIwLjA5LjIxIDE0OjMwLCBKdWVyZ2VuIEdyb3NzIHdyb3Rl
Og0KPiBPbiAyMC4wOS4yMSAwNzoyMywgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+
PiBIZWxsbywgU3RlZmFubyENCj4+DQo+PiBPbiAxOC4wOS4yMSAwMDo0NSwgU3RlZmFubyBTdGFi
ZWxsaW5pIHdyb3RlOg0KPj4+IEhpIE9sZWtzYW5kciwNCj4+Pg0KPj4+IFdoeSBkbyB5b3Ugd2Fu
dCB0byBlbmFibGUgcGNpYmFjayBvbiBBUk0/IElzIGl0IG9ubHkgdG8gImRpc2FibGUiIGEgUENJ
DQo+Pj4gZGV2aWNlIGluIERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRv
IGEgRG9tVT8NCj4+IE5vdCBvbmx5IHRoYXQNCj4+Pg0KPj4+IEkgYW0gYXNraW5nIGJlY2F1c2Ug
YWN0dWFsbHkgSSBkb24ndCB0aGluayB3ZSB3YW50IHRvIGVuYWJsZSB0aGUgUFYgUENJDQo+Pj4g
YmFja2VuZCBmZWF0dXJlIG9mIHBjaWJhY2sgb24gQVJNLCByaWdodD8gVGhhdCB3b3VsZCBjbGFz
aCB3aXRoIHRoZSBQQ0kNCj4+PiBhc3NpZ25tZW50IHdvcmsgeW91IGhhdmUgYmVlbiBkb2luZyBp
biBYZW4uIFRoZXkgY291bGRuJ3QgYm90aCB3b3JrIGF0DQo+Pj4gdGhlIHNhbWUgdGltZS4NCj4+
IENvcnJlY3QsIGl0IGlzIG5vdCB1c2VkDQo+Pj4NCj4+PiBJZiB3ZSBvbmx5IG5lZWQgcGNpYmFj
ayB0byAicGFyayIgYSBkZXZpY2UgaW4gRG9tMCwgd291bGRuJ3QgaXQgYmUNCj4+PiBwb3NzaWJs
ZSBhbmQgYmV0dGVyIHRvIHVzZSBwY2ktc3R1YiBpbnN0ZWFkPw0KPj4NCj4+IE5vdCBvbmx5IHRo
YXQsIHNvIHBjaS1zdHViIGlzIG5vdCBlbm91Z2gNCj4+DQo+PiBUaGUgZnVuY3Rpb25hbGl0eSB3
aGljaCBpcyBpbXBsZW1lbnRlZCBieSB0aGUgcGNpYmFjayBhbmQgdGhlIHRvb2xzdGFjaw0KPj4g
YW5kIHdoaWNoIGlzIHJlbGV2YW50L21pc3NpbmcvbmVlZGVkIGZvciBBUk06DQo+Pg0KPj4gMS4g
cGNpYmFjayBpcyB1c2VkIGFzIGEgZGF0YWJhc2UgZm9yIGFzc2lnbmFibGUgUENJIGRldmljZXMs
IGUuZy4geGwNCj4+IMKgIMKgwqAgcGNpLWFzc2lnbmFibGUte2FkZHxyZW1vdmV8bGlzdH0gbWFu
aXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywgd2hlbmV2ZXIgdGhlDQo+PiDCoCDCoMKgIHRvb2xzdGFj
ayBuZWVkcyB0byBrbm93IHdoaWNoIFBDSSBkZXZpY2VzIGNhbiBiZSBwYXNzZWQgdGhyb3VnaCBp
dCByZWFkcw0KPj4gwqAgwqDCoCB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVudHJpZXMg
b2YgdGhlIHBjaWJhY2suDQo+Pg0KPj4gMi4gcGNpYmFjayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVu
Ym91bmQgUENJIGRldmljZXMsIGUuZy4gd2hlbiBwYXNzaW5nIHRocm91Z2gNCj4+IMKgIMKgwqAg
YSBQQ0kgZGV2aWNlIGl0IG5lZWRzIHRvIGJlIHVuYm91bmQgZnJvbSB0aGUgcmVsZXZhbnQgZGV2
aWNlIGRyaXZlciBhbmQgYm91bmQNCj4+IMKgIMKgwqAgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3Bl
YWtpbmcgaXQgaXMgbm90IHJlcXVpcmVkIHRoYXQgdGhlIGRldmljZSBpcyBib3VuZCB0bw0KPj4g
wqAgwqDCoCBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1c2VkIGFzIGEgZGF0YWJhc2Ug
b2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSQ0KPj4gwqAgwqDCoCBkZXZpY2VzLCBzbyB3ZSBjYW4g
cmUtYmluZCB0aGUgZGV2aWNlcyBiYWNrIHRvIHRoZWlyIG9yaWdpbmFsIGRyaXZlcnMgd2hlbg0K
Pj4gwqAgwqDCoCBndWVzdCBkb21haW4gc2h1dHMgZG93bikNCj4+DQo+PiAzLiBEZXZpY2UgcmVz
ZXQNCj4+DQo+PiBXZSBoYXZlIHByZXZpb3VzbHkgZGlzY3Vzc2VkIG9uIHhlbi1kZXZlbCBNTCBw
b3NzaWJsZSBzb2x1dGlvbnMgdG8gdGhhdCBhcyBmcm9tIHRoZQ0KPj4gYWJvdmUgd2Ugc2VlIHRo
YXQgcGNpYmFjayBmdW5jdGlvbmFsaXR5IGlzIGdvaW5nIHRvIGJlIG9ubHkgcGFydGlhbGx5IHVz
ZWQgb24gQXJtLg0KPj4NCj4+IFBsZWFzZSBzZWUgWzFdIGFuZCBbMl06DQo+Pg0KPj4gMS4gSXQg
aXMgbm90IGFjY2VwdGFibGUgdG8gbWFuYWdlIHRoZSBhc3NpZ25hYmxlIGxpc3QgaW4gWGVuIGl0
c2VsZg0KPj4NCj4+IDIuIHBjaWJhY2sgY2FuIGJlIHNwbGl0IGludG8gdHdvIHBhcnRzOiBQQ0kg
YXNzaWduYWJsZS9iaW5kL3Jlc2V0IGhhbmRsaW5nIGFuZA0KPj4gdGhlIHJlc3QgbGlrZSB2UENJ
IGV0Yy4NCj4+DQo+PiAzLiBwY2lmcm9udCBpcyBub3QgdXNlZCBvbiBBcm0NCj4NCj4gSXQgaXMg
bmVpdGhlciBpbiB4ODYgUFZIL0hWTSBndWVzdHMuDQpEaWRuJ3Qga25vdyB0aGF0LCB0aGFuayB5
b3UgZm9yIHBvaW50aW5nDQo+DQo+PiBTbywgbGltaXRlZCB1c2Ugb2YgdGhlIHBjaWJhY2sgaXMg
b25lIG9mIHRoZSBicmlja3MgdXNlZCB0byBlbmFibGUgUENJIHBhc3N0aHJvdWdoDQo+PiBvbiBB
cm0uIEl0IHdhcyBlbm91Z2ggdG8ganVzdCByZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBhbmQgaGF2
ZSBpdCBydW4gb24gQXJtIHRvIGFjaGlldmUNCj4+IGFsbCB0aGUgZ29hbHMgYWJvdmUuDQo+Pg0K
Pj4gSWYgd2Ugc3RpbGwgdGhpbmsgaXQgaXMgZGVzaXJhYmxlIHRvIGJyZWFrIHRoZSBwY2liYWNr
IGRyaXZlciBpbnRvICJjb21tb24iIGFuZCAicGNpZnJvbnQgc3BlY2lmaWMiDQo+PiBwYXJ0cyB0
aGVuIGl0IGNhbiBiZSBkb25lLCB5ZXQgdGhlIHBhdGNoIGlzIGdvaW5nIHRvIGJlIHRoZSB2ZXJ5
IGZpcnN0IGJyaWNrIGluIHRoYXQgYnVpbGRpbmcuDQo+DQo+IERvaW5nIHRoaXMgc3BsaXQgc2hv
dWxkIGJlIGRvbmUsIGFzIHRoZSBwY2lmcm9udCBzcGVjaWZpYyBwYXJ0IGNvdWxkIGJlDQo+IG9t
aXR0ZWQgb24geDg2LCB0b28sIGluIGNhc2Ugbm8gUFYgZ3Vlc3RzIHVzaW5nIFBDSSBwYXNzdGhy
b3VnaCBoYXZlIHRvDQo+IGJlIHN1cHBvcnRlZC4NCkFncmVlLCB0aGF0IHRoZSBmaW5hbCBzb2x1
dGlvbiBzaG91bGQgaGF2ZSB0aGUgZHJpdmVyIHNwbGl0DQo+DQo+PiBTbywgSSB0aGluayB0aGlz
IHBhdGNoIGlzIHN0aWxsIGdvaW5nIHRvIGJlIG5lZWRlZCBiZXNpZGVzIHdoaWNoIGRpcmVjdGlv
biB3ZSB0YWtlLg0KPg0KPiBTb21lIGtpbmQgb2YgdGhpcyBwYXRjaCwgeWVzLiBJdCBtaWdodCBs
b29rIGRpZmZlcmVudCBpbiBjYXNlIHRoZSBzcGxpdA0KPiBpcyBkb25lIGZpcnN0Lg0KPg0KPiBJ
IGRvbid0IG1pbmQgZG9pbmcgaXQgaW4gZWl0aGVyIHNlcXVlbmNlLg0KPg0KV2l0aCB0aGlzIHBh
dGNoIHdlIGhhdmUgQXJtIG9uIHRoZSBzYW1lIHBhZ2UgYXMgdGhlIGFib3ZlIG1lbnRpb25lZCB4
ODYgZ3Vlc3RzLA0KDQplLmcuIHRoZSBkcml2ZXIgaGFzIHVudXNlZCBjb2RlLCBidXQgeWV0IGFs
bG93cyBBcm0gdG8gZnVuY3Rpb24gbm93Lg0KDQpBdCB0aGlzIHN0YWdlIG9mIFBDSSBwYXNzdGhy
b3VnaCBvbiBBcm0gaXQgaXMgeWV0IGVub3VnaC4gTG9uZyB0ZXJtLCB3aGVuDQoNCnRoZSBkcml2
ZXIgZ2V0cyBzcGxpdCwgQXJtIHdpbGwgYmVuZWZpdCBmcm9tIHRoYXQgc3BsaXQgdG9vLCBidXQg
dW5mb3J0dW5hdGVseSBJIGRvIG5vdA0KDQpoYXZlIGVub3VnaCBiYW5kd2lkdGggZm9yIHRoYXQg
cGllY2Ugb2Ygd29yayBhdCB0aGUgbW9tZW50Lg0KDQo+DQo+IEp1ZXJnZW4NCg0KVGhhbmsgeW91
LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 11:45:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 11:45:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190810.340602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHji-0004vg-2G; Mon, 20 Sep 2021 11:45:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190810.340602; Mon, 20 Sep 2021 11:45:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSHjh-0004vZ-VM; Mon, 20 Sep 2021 11:45:25 +0000
Received: by outflank-mailman (input) for mailman id 190810;
 Mon, 20 Sep 2021 11:45:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rxQs=OK=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSHjg-0004vT-R6
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 11:45:24 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3d3a3a48-1a08-11ec-b86f-12813bfff9fa;
 Mon, 20 Sep 2021 11:45:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d3a3a48-1a08-11ec-b86f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632138323;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=rLs+vo2xnfIZAr9Y4eBC8TltHw05a57daAB/3pHGxF0=;
  b=SrYwDoW6qixYAY8pRtVeeFZg08xzZcFK6rQ5v9iz50nvY5y222qX+rg0
   gWyoFMrjvnRPynnJv658jXyjMVQ8EuAVFbrFV/6oYOuaI52BwnYv1BhKK
   qmSb4Jtr5BNKW41z8/xCsyAqz0tJlZRFanVa5p6RL08HttPpOICkafJZB
   U=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: RKXA7ldO6ghHwCrdyVWkDTTIfEeHloXEYiZ716xwPakUfSjpm6D6rZkVMaHOk4UyqFr00ECvw6
 5/odAfEqxQ4O8D/Vt2UVax/9WfSfXSrO0SAr333foG6V3rRhAseG9QoA0XsCZKB/+EnbfmqBLI
 X0/UNhKVMoQEN4aEiJW027xW1C+P6Yej5pYP1EogkFQJH4Tcd+gO/Yc+w5FbKFt4mQGCv9LfF3
 iyZ2k+RXQbUkAqxO/qQNtW2oR6dSl5Ieysi7iANt0mfVU1A1fxziMvUDGZ1u6+SFBMGnEOwCWX
 85HEeof0UtPpL9IcCpuMHBR9
X-SBRS: 5.1
X-MesageID: 53526107
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:s1GGAqngoz1scXHWS//olEjo5gwtIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJMX22CbP3ZZGD8KtxzOo62o0wBsJbdydFkSwtrrnxkFCMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 OkdjIDhEVcXApXVgPoWcBMCNAFOMoQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpr25oQRK6BD
 yYfQQh3d0reSAFCA1oOFa0ew+zx2WXkbDIN/Tp5ooJoujOOnWSdyoPFMtXPfceRbd5IhUver
 WXDl0ziGQ0TPtGbzTuD81qvi/XJkCe9X5gdfJWG8fptjEyW13YkIhQcXlumotG0kke7HdlYL
 iQ89iMwoIAo+UesT927WAe3yFaGoxodVtx4A+A8rgaXxcL84hmbLngJSCZbb94rv9NwQiYlv
 neZktWsCTFxvbm9TXOG6qzSvT60ITISL2IJeWkDVwRty9vprZw3jxnPZs1+C6PzhdrwcRnvx
 xiaoS54gK8c5fPnzI3iowqB2Wj14MGUEEhlvW07Q15J8CtFNLe5NtT412Tj9O5eJ6WTFH6hj
 Hctzp32AP81MX2dqMCcaLxTR+j5vKfUaGe0bU1HRMZ6pm/0k5K3VcUJumgveh0xWioRUWKxO
 CfuVRVtCIi/1ZdARZR+ZZ64Q+8uxLLpfTgOfqGJNocSCnSdmQnuwc2PWaJy9zu2+KTPuftmU
 Xt+TSpLJSxBYZmLNBLsG48gPUYDn0jSP18/oKwXKDz8i9KjiIO9E+9ZYDNikMhgtP7sTPrpH
 yZ3aJLRlkQ3vBzWSSjL648DRW3m3lBiXsueliCjTcbaelAOMDh4U5f5mOp9E6Q4z/U9vrqZp
 RmVBx4HoGcTcFWacG1mnFg4M+ixNXu+xFpmVRER0aGAgSl7Pdn3s/dBKPPav9APrYRe8BK9d
 NFcE+2oCfVTUDXXvTMbaJj2tot5cxq3wwmJOkKYjPIXJfaMniTFpY3peBXB7i4LAnblvMcyu
 eT4hAjaXYACV0JpC8OPMKCjyFa4vH48nuNuXhSXfokPKRu0qIU6eTbsivIXIt0XLUmRzDWty
 AvLUwwTovPAotFp/YCR17yEtYqgD8B3AlFeQzvA9b+zOCSDpjijzIZMXfymZzfYUG+oqqyua
 f8Ml6P3MeEdnUYMuI15Su45waU77trphrlb0gU7QymbMwX1UutteyDU09NOu6tBwq5ilTG3A
 k/fqMNHPbipOd/+FAJDLgQSceneh+ofnSPf7KppLRyitjN35reOTW5bIwKI1H5GNLJwPY4on
 bUhtcoR51DtgxYmKI/b3CVd9mDKJX0cSaQ38JodBdaz2AYsz1hDZ73aCzP3v87TO4kdbBFyL
 2/GnrfGipRd2lHGIig6GnX61OZAgYgD5UJRx1gYKlXVwtfIi5fbBvGKHejbmuiN8ih67g==
IronPort-HdrOrdr: A9a23:K4edt6oQIOPRPa/RzAMgCVgaV5oweYIsimQD101hICG9E/bo8f
 xG88536faZskd1ZJhCo6HmBED/ex/hHPdOiOF7V4tKNzOW3FdAQrsMg+qDsl6QeRHDyg==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53526107"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=m4iKRU3MIGvuFiRBUs4ifhWTcNEUlrSq7EsfBurVybEEaNp0x4NNGbS6uleKJ47Jg3P5O/OyTBzDg2iS0IFTmylV/9OP9eiEJpp8HkAPyaF4bjETgI7JFmbIDC0dNbRW1KIlD4Be5Y/0jHNbZ8q+3DWfC2k3MLkPWpk6XrlbBBT3arp9jKHJu967VzhMUS806MOLjz0O9+aeQNdJrPdc56BBAym/SCtua+HoUCOCIrwmjeJQiL8CbBMZAOwnFf8+GB6xtuG/f429Fk0h0TuGTdntf1jLfEQmxYJ5fX6HUEqr1rLrWjbGdld688lTBZdt2p8Xi2gThyrRa2Vooa6YYQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5Z/zlOkhAe34r08JIFK07u2iOSIf4wkWJ5+6+6nXOdo=;
 b=QKPuA2DBGzad3yKxXLoQprPJMFBJsIuMP253D+G4Sl2cfT23MkvLw9MGGFarh6pykyLZjxasTF1pCN7ABSSwNG1kRUIZCeP+oBuvIjXvq1/2R5qBnx+8DzQ5IK/frr6G6QCPuPcLZoX9ZnoGU1LN0pwn49DBv1E4uSkp5gO/S4PqJQ0GLfa/Y4bIvN6FfgADBN0MW5bqDCicSeXjNbOafAqlyqM5+CDL4xRlKIvW8ltrZCTlkrozcsexw5+OHQ6eC9KV7v24EKGxURoB5wMmu1z/LRIctdZB7aWBItJoZeWy/useCPrqNyj6ZfKvxwKFxJAv/UrkdEJ5W107hG+thg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Z/zlOkhAe34r08JIFK07u2iOSIf4wkWJ5+6+6nXOdo=;
 b=FAVGJS2Y+SD0MrJdLNoF3T/aHCfLxd9Gjfmx2cZ5qZW17WN9QbS2Y3TkvGcOve+UkOs4XTEITzrrM4tLyid3YycjBOyJGEn89g5yU5ShiV6/MLIaAI1PIM0vIfvUei0w/rl4FAZhpYdj4FYwEreQR5ycqKmSww1HQMAny4v1Pak=
Date: Mon, 20 Sep 2021 13:45:10 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, "Wei
 Liu" <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Jan Beulich
	<jbeulich@suse.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony.perard@citrix.com>,
	"Juergen Gross" <jgross@suse.com>, Christian Lindig
	<christian.lindig@citrix.com>, David Scott <dave@recoil.org>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH 2/6] gnttab: allow per-domain control over transitive
 grants
Message-ID: <YUh0RtozVdav0iSQ@MacBook-Air-de-Roger.local>
References: <20210917154625.89315-1-roger.pau@citrix.com>
 <20210917154625.89315-3-roger.pau@citrix.com>
 <60295250-d6b2-1a0e-be30-d83b4f3c76bc@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <60295250-d6b2-1a0e-be30-d83b4f3c76bc@citrix.com>
X-ClientProxiedBy: LO2P265CA0391.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:f::19) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d0e0140a-bc03-4da5-f2bd-08d97c2c1d37
X-MS-TrafficTypeDiagnostic: DM6PR03MB4059:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4059382C2A86E6A889615ABE8FA09@DM6PR03MB4059.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: eA4A1UhFV7U88lEwvW2RTo457U4SzVmM0GRMFeoaBrZ72mDdjx/BSAHHdwtWvZ+XVSrpEl0toqiNioU34V7+RRbOmBXYeBGIo+fIWIE/LSc8aa0O+ZOsrrKX20yrcCKOAdSfi4BdM+rFXkRnDjfPwZaRWlT2wAiXTbDvP15jNv7AD98SwWtMmopoKnDDm5BkC36UM3LTi5vHHasGkL3/QvJX613eogqGyfuZ2gjF0NIUif61TXI3hGO8E7Kh8rV+jMt5MXx1Wr6jKhyZ4Kn4setAHq2VMSFXW4fpz3Vqdm6IxsPAkMSDZn0Vv22wcKWudopSBbTJtAc6xYIH/sWcxc7vtKbEfekZZl3NJBlsvXR285Rsm3G5P8wMK8Tms0Bta2v1/wy8Q43mkdJzjAI4XpORE6DS/XZ4gBQw0AF1xPiKtrIMzco4vBfvDzRXXMRcwnDdKGsUid170bC1KX4wjQl5euHRFOeJs+ZB0J6nBRfDuMtB8OOQ6KjPJf4j5QvE43tYHti+CpbcHjk8AhhnXl35bvlN4MSR+dLLnFSAuOilGyjrA29FaiQq1kW8gsym038Sn5v/8j6CVeoF6beJhkOUNDY1BI4nEFdlZYUB/zuN32NUjmqc9yMFNCTJBZlDgdF0+pBLIvAg2mkgktnZ/Q==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(396003)(136003)(366004)(478600001)(6636002)(54906003)(4326008)(6862004)(2906002)(66946007)(86362001)(8676002)(85182001)(5660300002)(316002)(6496006)(66476007)(66556008)(6666004)(6486002)(956004)(53546011)(83380400001)(9686003)(26005)(186003)(38100700002)(8936002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OXhTVnNHclJ5b2lQT09FSXpyanF2T0FPNGxvbXBQb2JUVU1zOHoxWDF5M3Bz?=
 =?utf-8?B?eXJVYWljbGFkQUoxWE1PU1ZFeDduQWQ0Zk9NMGVJMmJ0aUJQSFlkekdvcEdn?=
 =?utf-8?B?ZEZBamNBQ3BXd2E3bHQ1SDE1WUFaUGJoc2pFVTBockY5UUZHSmpLU0lPSnlX?=
 =?utf-8?B?TEdCbDhpVzBiNTVxSzVod2swcWEwOUF1VGF4djNDYXhGNUhvZThIYlZucGhE?=
 =?utf-8?B?ZHJ3U1hON0x1MlBTbTdyNXRVa0RKdmgyaUU3eTRranplSHF0Y1dUaVoxRlpL?=
 =?utf-8?B?L25IYVo2VWlVSm5OaloydGhWYUl5ak8rS09JeUQrZGVvNjBNcXNuWmlHUmNE?=
 =?utf-8?B?NTlVTXVCMTBOSjV0U1M4VmhLVW02bEZwNXJiVjlwbXozeWtpMFlCZDd5Wi9u?=
 =?utf-8?B?YUU3TExtcHBNUDJMNEt5eG85TTFsd3pFd0VmbEFFWE1mSFlZVnNqajBCdWVs?=
 =?utf-8?B?Vi9sdUZVdW9SV0hOcXV3N1FYaVBwTjRkdHlYV3FxM3E3TlJtd0c4dlpFaCtL?=
 =?utf-8?B?Z2VZNUY4Y0dkZGFpeVM2QW9ZK2ZZMmxYYm5KUEVJSzRUZC85ZXRSYS9SWWUy?=
 =?utf-8?B?dXlEU3VxREZWTWhDYUU3TUN0b2ttSjdRaHFXQWwyY3FUSHJxaFUzK1FkOFpS?=
 =?utf-8?B?L0haSFpUQzdpQnN5Uy8zWnJpdTE5NWVTMWM4TFF4bzhNZHdMdElCcjBaOVlB?=
 =?utf-8?B?YlhGc0RBRUMyUjRaOUtQZmJFa2p2V3hudHhBM1BBNDZjZUtiS2FQb0c2eGJu?=
 =?utf-8?B?aC9QTzF5MmFKYnBOQ1pubkNVeWpFV1AwdnhGQnBHbDZITnZENEZvaElOZ25D?=
 =?utf-8?B?N2hSMTk0STZPd3FnREZKbCtkSWpINGVpaFV2clBLTFc2STJVcEJMWklWVXRu?=
 =?utf-8?B?Z3hOaGFyd2hyR01pcStWTmZoVjJxcVhTbTliME1sWTZIS0h4MDlCaEl3bzhU?=
 =?utf-8?B?d2tBL1NjODA4RTlZSEVWTzUzc2JkcFF3MTU5KzBWYm1ManR6eGc0R2kwRElD?=
 =?utf-8?B?ZFZDZmNCNXM4Wmd4ZG0xMmpwTUgxbEEweVdMcmlRTmRUMnFjOGlySnNkMldK?=
 =?utf-8?B?ajl2NDFpZFBIR3NodFZTdjNQYTBML0xDMXBubVcxVjBnSnMvN2FNS3pTMGJM?=
 =?utf-8?B?R1l4QXVGek1Ja2FaOUJLeVhzcCs5NzFtMTNRUGpDNEJFbUdJMkV2NnBRVGIz?=
 =?utf-8?B?VzI5VmJ0QUQxRm55bm04R3YzRk8wRnJvNmRKMFNtSnpYaVpEenFrRzVpa0gx?=
 =?utf-8?B?OHc0VFc3VGpDd0ZmMXhvOUt2RDVBeXFONldnbm0xek56dWM0QlpKMzBiNlZk?=
 =?utf-8?B?TlVsTmZxR0lXdTh4Ym83S25WZ004SW5ZL1g0NVlJMkdBMWlDTGt5WkJiRGo2?=
 =?utf-8?B?NHhzSXZWYkVqRXB0YkxDSjZORmJZN1dhY2pGTUswdUdMVG9tTS9xc3VFa2M3?=
 =?utf-8?B?cTdIZzJhMytyM3lSWktGdHZCY0lDMm1QeWUwTWJ6YVlFTEpRVFVlSFBiZ3JH?=
 =?utf-8?B?bW5ERlVRMCtYTzJLYkF1OCtEOThvdWdBdzEvQzhaeHF2ZmtvOS9hNHBxMHpZ?=
 =?utf-8?B?OGVla2lZcnVnOTM3d3d1MVdzVGcyQ0R2TXV5WWJBVTBrMXozaUMvQ09NTGVp?=
 =?utf-8?B?TG1RZ1JZbW11cnNqMFB3Z0pZV0N4VnJENDZqSVV6YzlYS1RKWStZdE9OS1Ey?=
 =?utf-8?B?QjZiNzBRSGY5L3FhN3VhV2xVRVNJZmhlSGlFQzJYK0F4elFBRUVUZjREcVZh?=
 =?utf-8?Q?URvk5gA7M2/Vj2qndaXT0lajJLbj4Sqm7Ev49SG?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d0e0140a-bc03-4da5-f2bd-08d97c2c1d37
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 11:45:16.1849
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TO/WjPsRkmmxt/ZeBsSc7TMvG323J4EZfZHUnLiod0DLyXHV/1BH4GgH6Ltbr6wgC1c8221jb1PldPvPuuzXNw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4059
X-OriginatorOrg: citrix.com

On Mon, Sep 20, 2021 at 10:32:24AM +0100, Andrew Cooper wrote:
> On 17/09/2021 16:46, Roger Pau Monne wrote:
> > diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> > index 7f8456c50e..fe2201fca1 100644
> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> > @@ -96,6 +96,14 @@ struct xen_domctl_createdomain {
> >      int32_t max_maptrack_frames;
> >      int32_t max_grant_version;
> >  
> > +/* Allow transitive grants. */
> > +#define _XEN_DOMCTL_GRANT_transitive  0
> > +#define XEN_DOMCTL_GRANT_transitive   (1U << _XEN_DOMCTL_GRANT_transitive)
> 
> There's no need for bit position variables.
> 
> > +
> > +#define XEN_DOMCTL_GRANT_MAX XEN_DOMCTL_GRANT_transitive
> > +
> > +    uint32_t grant_opts;
> 
> So far, we've got 3 bits of information, v1, v2 and transitive, and
> we're tight on space in the structure with loads more to fit in.
> 
> I was thinking grant_flags or equiv to contain these 3 settings, and any
> further which might appear.

What about using something like the below?

We also need to consider selecting the default version (whatever is
set on the hypervisor) and no grant table at all.

/* Grant version, use low 4 bits. */
#define XEN_DOMCTL_GRANT_disable         0
#define XEN_DOMCTL_GRANT_version_v1      1
#define XEN_DOMCTL_GRANT_version_v2      2
#define XEN_DOMCTL_GRANT_version_default 0xf
/* Allow transitive grants. */
#define _XEN_DOMCTL_GRANT_transitive  4
#define XEN_DOMCTL_GRANT_transitive   (1U << _XEN_DOMCTL_GRANT_transitive)

#define XEN_DOMCTLGRANT_MAX XEN_DOMCTL_GRANT_transitive

    uint32_t grant_opts;

> 
> 
> One thing which is missing however is the enumeration of which settings
> are available, and rejection of bad settings.  If v2 is disabled
> globally, trying to create a VM with v2 needs to fail.

Right, I think this is already the case with the current
implementation. This doesn't happen however with the transitive
option, as I implemented it and'ing the hypervisor selection to the
tools provided one, partially due to the lack of a 'use hypervisor
default' option.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:04:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:04:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190828.340616 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI26-0007cY-TC; Mon, 20 Sep 2021 12:04:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190828.340616; Mon, 20 Sep 2021 12:04:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI26-0007cR-QD; Mon, 20 Sep 2021 12:04:26 +0000
Received: by outflank-mailman (input) for mailman id 190828;
 Mon, 20 Sep 2021 12:04:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSI25-0007cL-89
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:04:25 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e4ef18d8-1a0a-11ec-b86f-12813bfff9fa;
 Mon, 20 Sep 2021 12:04:24 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 350D92004D;
 Mon, 20 Sep 2021 12:04:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DD4E513483;
 Mon, 20 Sep 2021 12:04:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id qQejNMZ4SGFcIAAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 12:04:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4ef18d8-1a0a-11ec-b86f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632139463; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=ZubaKTTsXgUFZeDqZBtaX0Nm6q+Zb0+QJDfKEe2ZOB0=;
	b=TRbibNaBw9G4ffLO9IpRWu3FotMcA752oAO1A9dRbotpmFyTBVpvcurZUpUiTsjus4WhAX
	p5bgN1t3WNH9uKH9AtdSTbqZYYGBh0IilObJrSuGDgMRJ+586GNI7L3a+B0HartVxBcRI0
	2TvIh5lA5Lr5tp6fC757ZhZnP4jql9E=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: efault@gmx.de,
	Juergen Gross <jgross@suse.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	stable@vger.kernel.org,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
Subject: [PATCH v2] x86/setup: call early_reserve_memory() earlier
Date: Mon, 20 Sep 2021 14:04:21 +0200
Message-Id: <20210920120421.29276-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
reservations") introduced early_reserve_memory() to do all needed
initial memblock_reserve() calls in one function. Unfortunately the
call of early_reserve_memory() is done too late for Xen dom0, as in
some cases a Xen hook called by e820__memory_setup() will need those
memory reservations to have happened already.

Move the call of early_reserve_memory() before the call of
e820__memory_setup() in order to avoid such problems.

Cc: stable@vger.kernel.org
Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)
---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f164141116..40ed44ead063 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:05:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:05:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190834.340628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI3L-0008C9-7s; Mon, 20 Sep 2021 12:05:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190834.340628; Mon, 20 Sep 2021 12:05:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI3L-0008C2-4e; Mon, 20 Sep 2021 12:05:43 +0000
Received: by outflank-mailman (input) for mailman id 190834;
 Mon, 20 Sep 2021 12:05:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rsqB=OK=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSI3K-0008Bs-9y
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:05:42 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 0756f935-369a-40e6-9824-92dd59c4f9e1;
 Mon, 20 Sep 2021 12:05:41 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-461-Q2Q90twzMEeA8yWJ2-I_-Q-1; Mon, 20 Sep 2021 08:05:40 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 x7-20020a5d6507000000b0015dada209b1so5864535wru.15
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 05:05:39 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id l15sm19128605wme.42.2021.09.20.05.05.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 Sep 2021 05:05:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0756f935-369a-40e6-9824-92dd59c4f9e1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632139541;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=q8wAH/IsmVnI2ydhQ1BFoaOBGT1C6T31sGkkhWrMszg=;
	b=E5O5IYb3E/Ra/xlxED0QOc8QrPeCDxCsIi9ztHhZjYBOh6ejFUGPudbBXxiS/wHIdenNep
	qIFlUvG5zJwA+fqK3RXonYrssqDunAdP9q4HaYuW0GkV6Qxp86eISEvTcccX4DFIPFEpFG
	Kuo6tHq2RiDnvU1gHyMflYXobeoveWY=
X-MC-Unique: Q2Q90twzMEeA8yWJ2-I_-Q-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:to:cc:references:from:subject:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=q8wAH/IsmVnI2ydhQ1BFoaOBGT1C6T31sGkkhWrMszg=;
        b=mGpYWvipljWjJ9Ypsi19pyCoxY/K/wfHktHHIXNRH8ixVA6NN5BIjWLV72JOz+Tyni
         r9OcMWxYYfP0cnKthjOSCrSHE8qq1zk+WMm+ZTAyEYc5zqTCccEDT+RJEOs9PvIaHvoG
         jqLQ3V/9b70WNyxa8Su6DVXgl1Rp75rBSCuPo9ap21njrJiz42hGtnZd4iLFWiTNRP31
         jXDftar4AGcFe2HqMadPUxPShpWbC44feYqFw+Lo6IoJlYenl+mU3+lp0eYZxlGVQXis
         gBJOGxLpC5mvEGhsyyBPj/cD38Wg/XCTeUNUVVcnpLenJ4+Af1fYlIZnj/RtcBCL5r/r
         D+8w==
X-Gm-Message-State: AOAM530KTrPuY5VX4/G1Z4JCXg08i2f7ANsRI36gXHFvJkNuVsgnuxVj
	GMYy1M6NA+j+DnU+9dLowZC4s+arhBdxOAaN4mm/+po5vduNhG7hTaCYJafbapp1MWVlgMcIXq3
	7r2Y7ZfjdGuDItaNeXWehKuFP8Wc=
X-Received: by 2002:a5d:4608:: with SMTP id t8mr28038945wrq.136.1632139538950;
        Mon, 20 Sep 2021 05:05:38 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJz64XIPtY2fM12srCDcTMCwDqENNrjhBUdJzYmop3FKDPBAPZx+uh9ah7Isb2beeBqNbwRMJw==
X-Received: by 2002:a5d:4608:: with SMTP id t8mr28038880wrq.136.1632139538580;
        Mon, 20 Sep 2021 05:05:38 -0700 (PDT)
To: Peter Zijlstra <peterz@infradead.org>,
 Sean Christopherson <seanjc@google.com>
Cc: Ingo Molnar <mingo@redhat.com>, Arnaldo Carvalho de Melo
 <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>,
 Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Thomas Gleixner <tglx@linutronix.de>, Borislav Petkov <bp@alien8.de>,
 x86@kernel.org, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Juergen Gross <jgross@suse.com>,
 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>, "H. Peter Anvin" <hpa@zytor.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
 <YUO5J/jTMa2KGbsq@google.com>
 <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
From: Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
Message-ID: <662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
Date: Mon, 20 Sep 2021 14:05:25 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 17/09/21 09:28, Peter Zijlstra wrote:
>> In theory, I like the idea of burying intel_pt inside x86 (and even in
>> Intel+VMX code for the most part), but the actual implementation is a
>> bit gross.  Because of the whole "KVM can be a module" thing,
> 
> ARGH!! we should really fix that. I've heard other archs have made much
> better choices here.

I think that's only ARM, and even then it is only because of limitations 
of the hardware which mostly apply only if VHE is not in use.

If anything, it's ARM that should support module build in VHE mode 
(Linux would still need to know whether it will be running at EL1 or 
EL2, but KVM's functionality is as self-contained as on x86 in the VHE 
case).

Paolo



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:11:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:11:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190842.340639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI8l-0001KY-SE; Mon, 20 Sep 2021 12:11:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190842.340639; Mon, 20 Sep 2021 12:11:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSI8l-0001KR-PA; Mon, 20 Sep 2021 12:11:19 +0000
Received: by outflank-mailman (input) for mailman id 190842;
 Mon, 20 Sep 2021 12:11:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSI8l-0001KH-78; Mon, 20 Sep 2021 12:11:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSI8k-0006Uo-Vg; Mon, 20 Sep 2021 12:11:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSI8k-00016K-Jf; Mon, 20 Sep 2021 12:11:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSI8k-0002sr-J7; Mon, 20 Sep 2021 12:11:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mOERV2ln5BVL+gTFhZB9CdCKi5sNzbU67wsG39c6rl4=; b=UouTI/9Ncs6muB90ei+G9Hwq9S
	c/VElWEF/ts2VxQxH+sKTWoIyg7V9YSuwDPRwfuSKiIoKAtkCntFsr5YPGHoCrdHv46cjzZEVIEpm
	J/0aqq7cJj2HD8GyGF3Ba+UkfCulW0G8LMWCAuoFkX6jk/Q4ao/ThejmgIrKjPmwHxHc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165125-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165125: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-armhf-armhf-xl-vhd:guest-start.2:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e4e737bb5c170df6135a127739a9e6148ee3da82
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 12:11:18 +0000

flight 165125 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165125/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd      18 guest-start.2            fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                e4e737bb5c170df6135a127739a9e6148ee3da82
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  415 days
Failing since        152366  2020-08-01 20:49:34 Z  414 days  723 attempts
Testing same since   165125  2021-09-20 01:42:52 Z    0 days    1 attempts

------------------------------------------------------------
7321 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2262655 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:21:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:21:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190851.340652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIIf-0002rH-0N; Mon, 20 Sep 2021 12:21:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190851.340652; Mon, 20 Sep 2021 12:21:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIIe-0002rA-Tn; Mon, 20 Sep 2021 12:21:32 +0000
Received: by outflank-mailman (input) for mailman id 190851;
 Mon, 20 Sep 2021 12:21:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSIId-0002r4-Oi
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:21:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSIId-0006fA-Lu
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:21:31 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSIId-0000Le-Kq
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:21:31 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSIIY-0005ln-UB; Mon, 20 Sep 2021 13:21:26 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=vKFgzzjUPzhgJWQ7Q6xL+L25e4KgSe6Xatv9RvE5nr0=; b=I0+SL5rWGWMateXaSRELtPL2NS
	gmuCS03Tu6L1zstUEDcjxp5cXBL7TjlCEAR78Md0IIELH9j6bF6NDzebpfceuwsUvfgnjsQ0X8BCd
	VBJb/Jb0o5ITAhccOHvj1p7lJCWM09+DQYfsUZnC9czXcvI2xL+esjKdSIMB17N+1f6s=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.31942.699227.516440@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 13:21:26 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Roger Pau =?iso-8859-1?Q?Monn=E9?=  <roger.pau@citrix.com>,
    xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>,
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces
In-Reply-To: <25710b42-b2e9-54a2-a510-d8e09dbb1c25@suse.com>
References: <20210917154625.89315-1-roger.pau@citrix.com>
	<20210917154625.89315-5-roger.pau@citrix.com>
	<90f64a21-d5fc-fab3-2f3c-73de00421cb3@suse.com>
	<YUhoLnpv+Dq/dFBR@MacBook-Air-de-Roger.local>
	<24904.26906.715083.732081@mariner.uk.xensource.com>
	<25710b42-b2e9-54a2-a510-d8e09dbb1c25@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH 4/6] tools/xenstored: use atexit to close interfaces"):
> They are. I checked when implementing LU and just rechecked.
> 
> The event-channel fd is opened explicitly without CLOEXEC in order
> to support LU (the new xenstored won't open it again, but gets its
> fd via the migration stream).

Oh, excellent.  Thanks.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:22:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:22:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190856.340663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIJG-0003Oy-8s; Mon, 20 Sep 2021 12:22:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190856.340663; Mon, 20 Sep 2021 12:22:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIJG-0003Or-5y; Mon, 20 Sep 2021 12:22:10 +0000
Received: by outflank-mailman (input) for mailman id 190856;
 Mon, 20 Sep 2021 12:22:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TWfd=OK=kernel.org=maz@srs-us1.protection.inumbo.net>)
 id 1mSIJE-0003Oj-FP
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 12:22:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5eea204a-1a0d-11ec-b871-12813bfff9fa;
 Mon, 20 Sep 2021 12:22:07 +0000 (UTC)
Received: from disco-boy.misterjones.org (disco-boy.misterjones.org
 [51.254.78.96])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.kernel.org (Postfix) with ESMTPSA id CD81460F9D;
 Mon, 20 Sep 2021 12:22:06 +0000 (UTC)
Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org)
 by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2)
 (envelope-from <maz@kernel.org>)
 id 1mSIJA-00BkwX-Th; Mon, 20 Sep 2021 13:22:04 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5eea204a-1a0d-11ec-b871-12813bfff9fa
Date: Mon, 20 Sep 2021 13:22:04 +0100
Message-ID: <87tuifv3mb.wl-maz@kernel.org>
From: Marc Zyngier <maz@kernel.org>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Sean Christopherson <seanjc@google.com>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo
 <acme@kernel.org>,
	Will Deacon <will@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>,
	Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt
 <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	x86@kernel.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org,
	linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
In-Reply-To: <662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
References: <20210828003558.713983-1-seanjc@google.com>
	<20210828201336.GD4353@worktop.programming.kicks-ass.net>
	<YUO5J/jTMa2KGbsq@google.com>
	<YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
	<662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
X-SA-Exim-Connect-IP: 185.219.108.64
X-SA-Exim-Rcpt-To: pbonzini@redhat.com, peterz@infradead.org, seanjc@google.com, mingo@redhat.com, acme@kernel.org, will@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, guoren@kernel.org, nickhu@andestech.com, green.hu@gmail.com, deanbo422@gmail.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, bp@alien8.de, x86@kernel.org, boris.ostrovsky@oracle.com, jgross@suse.com, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, hpa@zytor.com, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, sstabellini@kernel.org, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, artem.kashkanov@intel.com, like.xu.linux@gmail.com, lingsha
 n.zhu@intel.com
X-SA-Exim-Mail-From: maz@kernel.org
X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false

On Mon, 20 Sep 2021 13:05:25 +0100,
Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> On 17/09/21 09:28, Peter Zijlstra wrote:
> >> In theory, I like the idea of burying intel_pt inside x86 (and even in
> >> Intel+VMX code for the most part), but the actual implementation is a
> >> bit gross.  Because of the whole "KVM can be a module" thing,
> > 
> > ARGH!! we should really fix that. I've heard other archs have made much
> > better choices here.
> 
> I think that's only ARM, and even then it is only because of
> limitations of the hardware which mostly apply only if VHE is not in
> use.
> 
> If anything, it's ARM that should support module build in VHE mode
> (Linux would still need to know whether it will be running at EL1 or
> EL2, but KVM's functionality is as self-contained as on x86 in the VHE
> case).

I don't see this happening anytime soon. At least not before we
declare the arm64 single kernel image policy to be obsolete.

	M.

-- 
Without deviation from the norm, progress is not possible.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 12:23:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 12:23:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190863.340674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIKm-000462-Kv; Mon, 20 Sep 2021 12:23:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190863.340674; Mon, 20 Sep 2021 12:23:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIKm-00045v-Hi; Mon, 20 Sep 2021 12:23:44 +0000
Received: by outflank-mailman (input) for mailman id 190863;
 Mon, 20 Sep 2021 12:23:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIKl-00045j-74; Mon, 20 Sep 2021 12:23:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIKl-0006iy-13; Mon, 20 Sep 2021 12:23:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIKk-0001Sx-NF; Mon, 20 Sep 2021 12:23:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIKk-0000N3-Mm; Mon, 20 Sep 2021 12:23:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4Tzi1A8G0tBYUhPrkxmn4shEHwPfmrIad6yx8DlG+2c=; b=CwnCT5rjc8QvF7uGF+Tb4bEkaD
	jmS8pXREczNWMIQcDDVteAph5pGGsKP+IofMRfUFwe065Np/8+Hb6XT6PX3zK7GMwlA+0edhCrlU9
	FLuYMMJv16NfhDxvKRx4QEUxakfxHnSMq36zVwnV/p1AmC48jwXx/w62fWKZ8p3Obf5w=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165126-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165126: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=785a11cec8693de7df024aae68975dd1799b646a
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 12:23:42 +0000

flight 165126 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165126/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              785a11cec8693de7df024aae68975dd1799b646a
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  437 days
Failing since        151818  2020-07-11 04:18:52 Z  436 days  427 attempts
Testing same since   165104  2021-09-18 04:20:03 Z    2 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 73994 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 13:00:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 13:00:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190874.340689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIuD-0000F2-AL; Mon, 20 Sep 2021 13:00:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190874.340689; Mon, 20 Sep 2021 13:00:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIuD-0000Ev-6n; Mon, 20 Sep 2021 13:00:21 +0000
Received: by outflank-mailman (input) for mailman id 190874;
 Mon, 20 Sep 2021 13:00:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSIuC-0000Ep-4a
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 13:00:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fdf356a6-a1e9-4307-907f-b1bb91f20489;
 Mon, 20 Sep 2021 13:00:19 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-m2DUg56nNtqyRi0Urbb30w-2; Mon, 20 Sep 2021 15:00:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4848.eurprd04.prod.outlook.com (2603:10a6:803:55::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Mon, 20 Sep
 2021 13:00:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 13:00:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR04CA0150.eurprd04.prod.outlook.com (2603:10a6:207::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Mon, 20 Sep 2021 13:00:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdf356a6-a1e9-4307-907f-b1bb91f20489
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632142817;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=m/dP2tjFQBapFgCs14B5EAZuDuxp7XkMmschxcMQy5g=;
	b=PE2VaSRiOn6w2yCVUAYdpaLt8XKvHUbHiUSjaccT6VHP1PMWA6466SBtXRHGQrIAGcH1zU
	hbZLxfVv8dBYyHQQDoTH1hDVWRle6BRL6NhNfaua72R+rdzQzDZy2+VCCo6+8Ene5wzW2k
	TVDdHCeyAONrIxeoJ1pV+DNj65QqL9A=
X-MC-Unique: m2DUg56nNtqyRi0Urbb30w-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b/wjWg50xm8U/3kMWzge3y+3OkMS/lc4spV2lC1zw1yztAiVl0j68iRJT9kkT4mDQ1ffWV0FqicyS1zjXM1fFspzXcdX63RQcobHUH2/UZL2jdpkfR93OnfXaMTgBKmhdpmRYWaMBhS5UcF01+lGJ8Vg1xo2fTNELE2+o2M3P2qN9JCAyMXwSDHOBaWo9TYeDZA6dTLoqExKCDKYpKP5wl+Uavy2SipB+2ndtkcRaSNGjhDptSEls9TBaxUhWDWC2FpFT8DUdPMXB2/6VqCcMQIatnvj/QF+5rwPQEqS405DUf1sHyvH5Jec5R4jaov99AOHlBbTgrc/zvB0kfqLTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hm7m1lFQkBWcIUn8FkgnqbFp/XgM4RikeEDzakOZDEw=;
 b=Fzop+7iHlJaUnwsAMZrr47nLOjmeJMHXxPOA7j9nEqKt47evvQtNp6dZI2JaxUh7jOqV/Ct2kaVa6gdDRYF7SgZs7hU/VHf8riEkxLi26+8AuUTmKqY/c5+HwBW1sN/RV/AoH3LSWybbClnRSL64gSfY4TJWMaVNY7SgXdOOH+PRlKkd0HOISBckhg229Se+VD/JVeRv8sJzfIkp/pIHbt51qWxnzgpdGrxocvB6UI52ZIQFgv/8M0ZiIApf4YIr0tlkXdZ0Qpff7ZewQ83npwzT7A3DL71bot9+IcMcmDQAecKy2Dhog5RaiFyM0ELxKeJh11844+UtCNSRiFpycA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 4/6] x86/trace: Reduce stack usage from HVMTRACE_ND()
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210917084559.22673-1-andrew.cooper3@citrix.com>
 <20210917084559.22673-5-andrew.cooper3@citrix.com>
 <ef2df9df-df4a-8f8a-3f69-88a027ba66eb@suse.com>
 <e116bff4-9080-1629-221d-f2ba3255d03e@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f52efb10-7762-9828-e8c4-ce2d00acdff5@suse.com>
Date: Mon, 20 Sep 2021 15:00:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e116bff4-9080-1629-221d-f2ba3255d03e@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM3PR04CA0150.eurprd04.prod.outlook.com (2603:10a6:207::34)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f1f2677e-b903-423c-fb81-08d97c369600
X-MS-TrafficTypeDiagnostic: VI1PR04MB4848:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB484886022AA98C4F1888E85CB3A09@VI1PR04MB4848.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hHR74gDvMqrVQ0NIJ7tSt5gKGxy93eFjH75MZHOgv+6qxMyzSKgleOb1gB/8NqfjM9ez0qPB2u3u7wyCl/qDYJG4boZo3kCADeffTJOxIbLUkE+BeSgwbaX5wy5l0bJpxwvAxrdR3RwUEhrL0W4Mzg/EGQIjH8KwzTIRCnDMmDfxmH5UfrzGxBP0rippjIe27vHgNJzvi/e9etMKYxOoX6rcP2JaMF4Q2B4XvVLZX4PgYjnrx2l1YfTF3sgNuqrzZpZqneIBd0e0kaxE5CAFqPjsIPwBltjIsSTPj8PVf/6/v9Pj9UUWhfXsOAJzmS7UbYbPvx9pyVmvVEEL9ONvJkJFojC1YFP6fLjm2T0dBzwy6BSyCduVKcsDyg4xqBZS3IbgRvN0fAHLjWoFBSrjDB2BXy29FmrHAMs8K/aiiEjWC8vJfDc2TTURYglQF5FlkAg/YIoPXYIlMLe3Wpvtdy0bQ4VGvoUdLD8ZmstSqoo+uhizTRFhgRSC4nDyLm7+ycTdpB2whNRWYCBbYxXHhQBKECq320Z7y1AYBPku2jg6A5m2o2jWYe3sQsH6E4cJdjYorg+RUj1FYX4tM8SMs4jkqlNbLBFTYN2jAY0cbWFlGPMUOnLlcK5PNKMGNjAFtuv6mgdVbKUehfkKtdyl8tAcPFmB18DewV4Y5RimpNRuT9GZ1qsfuDdCE+Bg13sM1zrlG8gZrFremNPxRKSFnlZ1GugvXKJJ4UrxL87KNDc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31686004)(5660300002)(316002)(31696002)(66556008)(38100700002)(54906003)(6486002)(16576012)(6916009)(86362001)(508600001)(53546011)(956004)(8936002)(26005)(186003)(2616005)(66476007)(8676002)(36756003)(4326008)(2906002)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?VNT7I5U53YX+gI9TXUIWNMfJmqgw2xhp3O02LtNfmt4mhxODrvCPME1pgvzQ?=
 =?us-ascii?Q?t7Vw1RhwaDsBAnc5F9CoGzQDYBKvuJ+BYC0DGnucXuAy3yULMIZ0DKw4poO4?=
 =?us-ascii?Q?ACB+KGjTOunTpxnkSwmnXu9RHz0buan+i64fPenx2SCA3spp6bTRED9DhcND?=
 =?us-ascii?Q?LZ7PDnDxp5OM9elBpb+446fA8jooA/Vi6qLw/9i6BfzCN8UpImPm6TWny0h4?=
 =?us-ascii?Q?RfQe2/8qDM0A2cXX1C/rnJ/ABWXEs3mVGSlLNflKBNL3nPri4NWJU7Mycj2P?=
 =?us-ascii?Q?8HPLB9aFXpgXtX8T9CTKarryC3clX/pHb5xfRg/wNqjqUMjVwc8bhIbeGeGb?=
 =?us-ascii?Q?oOP/Hk1QNKhvIOfJ9icLCXrnGzkounMBFXTQhgimoY/0tOGWHc4Mvo0VVlYM?=
 =?us-ascii?Q?eZWX1j0fU1eefhJn0psSur/V2KvtL4QdqMenlOUmhIw28QOlEoNSMncYZC7H?=
 =?us-ascii?Q?kJW2yYzTyQW92IvrmuxoS5RDZ9xPDnW7IgN+4QfkOV07hkUB3g+D3PrkK9Av?=
 =?us-ascii?Q?zQtwKqHtomWOVVBXUtnTth/w90VVAyIZbAacJTIDuKdEi1jW2RNa0wKL4oNk?=
 =?us-ascii?Q?x2BbOeA3bFvjxet8hPdgi+NpscH4Q35N6cSHoHvf/ELzyC7dE+A83LVbd5X0?=
 =?us-ascii?Q?vNeDkPBqkRGOPJ2bldpdc470/KxhHlERRttUv4l2UGd3Q6vW3gqAlvGsEbrc?=
 =?us-ascii?Q?zIKOPpA5PWN1frzbiUinNBcxggvq7rv4jZsWcbSqqxbxFpgXEzBq2bAAoF9D?=
 =?us-ascii?Q?2e95uRW7xR9LxfuB2G5d2Hd+TMIi/12obI6skB3fBb2+wS13QTLZ/bShCFwA?=
 =?us-ascii?Q?ibDBff1gBM7qeo9g5CeixFjMAeWW1E1gPBl+7Fc995CWC0hzEKX7PcuvElj2?=
 =?us-ascii?Q?tb/e5X5YddjmnzYgm6JLa9yDH7dIyoz1nVxjl0UMlo1PIeImp4C5xDtlrMfr?=
 =?us-ascii?Q?J6c0CIe7srfJiFwGo5JDKhh56UbPsh40vOc3BC7rLw0Nj9r6FoRbqy1IbR4Y?=
 =?us-ascii?Q?P7F2jdBysErGvzhixAZG0DZC5yFFv455YUjWd99bSknpKt089fQCljgkiiGE?=
 =?us-ascii?Q?7lz07nYZz/PGUq3bykosaB4KQlph3l/v9R/3QLY1VOrLzUHRNqtBsbgRKPBc?=
 =?us-ascii?Q?cwSACCS//zqwT2iqakFY1/d6044KkiEhlufaVEEjr3rMJupI2nCjpisCr0Kw?=
 =?us-ascii?Q?+RpqZYDOSfkNvISy7DpRy7wG0ju6/rm6BSkT5pfbMA9Pxb8IyjqQUKK0L6E6?=
 =?us-ascii?Q?ovzBTPxjv7gMZR4lcxwZui/3DQ3RJB85cHOf2LX9HYLrt37bQvSXoZiyG8cZ?=
 =?us-ascii?Q?W52e9RM5FfTZDTkFnc5jZ33a?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f1f2677e-b903-423c-fb81-08d97c369600
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 13:00:13.9589
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hz8JSNBmv5VOwo/8qjDJW2HrrUzcCcy9W0warBG5rjNnCoc09UiuYXdBK+aVG6hkAPbwsHlPMn2LQ1nEhm4AgQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4848

On 20.09.2021 13:02, Andrew Cooper wrote:
> On 20/09/2021 10:05, Jan Beulich wrote:
>> On 17.09.2021 10:45, Andrew Cooper wrote:
>>> It is pointless to write all 6 entries and only consume the useful subs=
et.
>>> bloat-o-meter shows quite how obscene the overhead is in vmx_vmexit_han=
dler(),
>>> weighing in at 11% of the function arranging unread zeroes on the stack=
, and
>>> 8% for svm_vmexit_handler().
>>>
>>>   add/remove: 0/0 grow/shrink: 0/20 up/down: 0/-1867 (-1867)
>>>   Function                                     old     new   delta
>>>   hvm_msr_write_intercept                     1049    1033     -16
>>>   vmx_enable_intr_window                       238     214     -24
>>>   svm_enable_intr_window                       337     313     -24
>>>   hvmemul_write_xcr                            115      91     -24
>>>   hvmemul_write_cr                             350     326     -24
>>>   hvmemul_read_xcr                             115      91     -24
>>>   hvmemul_read_cr                              146     122     -24
>>>   hvm_mov_to_cr                                438     414     -24
>>>   hvm_mov_from_cr                              253     229     -24
>>>   vmx_intr_assist                             1150    1118     -32
>>>   svm_intr_assist                              459     427     -32
>>>   hvm_rdtsc_intercept                          138     106     -32
>>>   hvm_msr_read_intercept                       898     866     -32
>>>   vmx_vmenter_helper                          1142    1094     -48
>>>   vmx_inject_event                             813     765     -48
>>>   svm_vmenter_helper                           238     190     -48
>>>   hvm_hlt                                      197     146     -51
>>>   svm_inject_event                            1678    1614     -64
>>>   svm_vmexit_handler                          5880    5416    -464
>>>   vmx_vmexit_handler                          7281    6473    -808
>>>   Total: Before=3D3644184, After=3D3642317, chg -0.05%
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>=20
> Thanks, but this is buggy.=C2=A0 There are direct callers of HVMTRACE_ND(=
)
> which need adjustments too.

Is this really "buggy" and "need" and not merely "incomplete" and
"want"? (Just for my own understanding in terms of what I may have
overlooked.)

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 13:01:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 13:01:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190882.340699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIv7-0000sK-N7; Mon, 20 Sep 2021 13:01:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190882.340699; Mon, 20 Sep 2021 13:01:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSIv7-0000sD-KE; Mon, 20 Sep 2021 13:01:17 +0000
Received: by outflank-mailman (input) for mailman id 190882;
 Mon, 20 Sep 2021 13:01:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIv6-0000rz-HQ; Mon, 20 Sep 2021 13:01:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIv6-0007L8-AZ; Mon, 20 Sep 2021 13:01:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIv6-0003GQ-19; Mon, 20 Sep 2021 13:01:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSIv6-0002wg-0c; Mon, 20 Sep 2021 13:01:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sOTOKo3ZFxPWKPUv328IJumQixEtha580ds40d+TXFQ=; b=5JIduKI4hAwDazig86JBDnMzln
	YoUxfsHlNF9SQgn0umPZyp2TwK9GS5SujefIkZmBxeRSuFq+MxRPU+ip6J5qFa0A3NoxZoOjggRdw
	13OFlVC4163YfW0MdbSZpdt+Zb/vMhOGXsboGX9E6G4NJwT8KjhCpl4KL0lRh0LedFSY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165128-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165128: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
X-Osstest-Versions-That:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 13:01:16 +0000

flight 165128 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165128/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
baseline version:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8

Last test of basis   165097  2021-09-17 20:01:39 Z    2 days
Testing same since   165128  2021-09-20 09:01:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   437e88417b..a3dd33e63c  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 13:18:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 13:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190910.340731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJBu-0003mc-LM; Mon, 20 Sep 2021 13:18:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190910.340731; Mon, 20 Sep 2021 13:18:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJBu-0003mV-IV; Mon, 20 Sep 2021 13:18:38 +0000
Received: by outflank-mailman (input) for mailman id 190910;
 Mon, 20 Sep 2021 13:18:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rsqB=OK=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSJBu-0003mP-7b
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 13:18:38 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 9e7fe3a9-a9f3-4781-971f-765864a276c0;
 Mon, 20 Sep 2021 13:18:36 +0000 (UTC)
Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com
 [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-584-fk4cY7LrPpmLWm9Lm7_lEQ-1; Mon, 20 Sep 2021 09:18:34 -0400
Received: by mail-wr1-f69.google.com with SMTP id
 x7-20020a5d6507000000b0015dada209b1so6031937wru.15
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 06:18:34 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id o12sm13359543wms.15.2021.09.20.06.18.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 Sep 2021 06:18:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e7fe3a9-a9f3-4781-971f-765864a276c0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632143916;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XZbiHti4vCOqgWKM54l6a3juvV92A2baM8UIqCN0ZRQ=;
	b=gBUs1cCExa5AwU18pM/5jF6Axf25NmtDAir8l1/5EPd+yjKEzXkPTZJy7jfhrLmKfv3DuP
	LNPOA/KIsr4f1c97q+3Wz2SVFHfZSNPs50Ktup5c1sXY4Vcqk/HCMAoLHu6Ag8Uj6T1Gdb
	ttj9umuCWYVoOVuZQmVrWyHGxBCZToE=
X-MC-Unique: fk4cY7LrPpmLWm9Lm7_lEQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=XZbiHti4vCOqgWKM54l6a3juvV92A2baM8UIqCN0ZRQ=;
        b=4bPcBJSADUFtrmDYGMgH3o1Ztd9MO1IWXDrbklGbJKaoZZjLZviynpnNAOggUF2kz/
         CwzaTyNfNmGxDhUHgsqbN46HJ00of1Q4EZ+DgY5cHadKBourCQLtqQZL4nlLTtkYajvW
         jTHVduRrudYwPQqHGbWTPsGl7VJfTN3MTI9M5q4v8FUMcV8h4JKFNLdibbxZejCagbwN
         RUzYhKEz885MI/Zb7sA9VNslmZY/F6h1yqYy8XCzRUb9g61HIS5tbzOj/HXuiA+Li/Kq
         rRN3GY1mL6nzXxb44fnAWead5QWOyhLNFxLDSN1pnygQdNP/73xV5KG+GUxv1RG49ToQ
         4E3A==
X-Gm-Message-State: AOAM533/l4TIB3JMornOSXuKo3oVq9aJjbTxO8nQUaFM0lqKVXtMrU74
	+n3iaQgKKhfAUQjj9s+tVHcTQ8oPraJo6JHXluBk86muzp4L2N5ut4R9GAS8gWN1mNI/Tp/WlDG
	igWRx1CxxNAspFxOIVCW6X74NxIE=
X-Received: by 2002:a5d:6c6d:: with SMTP id r13mr13376912wrz.439.1632143913678;
        Mon, 20 Sep 2021 06:18:33 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJz7N3hxz36qRlOw1/rL/DwL4aDROJ8XxG8xzFR2ylC3WRZ+AXi5MycER2AoBKQtMUNziJuirw==
X-Received: by 2002:a5d:6c6d:: with SMTP id r13mr13376850wrz.439.1632143913429;
        Mon, 20 Sep 2021 06:18:33 -0700 (PDT)
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
To: Marc Zyngier <maz@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
 Sean Christopherson <seanjc@google.com>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>,
 Catalin Marinas <catalin.marinas@arm.com>, Guo Ren <guoren@kernel.org>,
 Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
 Vincent Chen <deanbo422@gmail.com>, Paul Walmsley
 <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 Albert Ou <aou@eecs.berkeley.edu>, Thomas Gleixner <tglx@linutronix.de>,
 Borislav Petkov <bp@alien8.de>, x86@kernel.org,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>, "H. Peter Anvin" <hpa@zytor.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
 <YUO5J/jTMa2KGbsq@google.com>
 <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
 <662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
 <87tuifv3mb.wl-maz@kernel.org>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <7a5825d1-d6e9-8ac8-5df2-cce693525da7@redhat.com>
Date: Mon, 20 Sep 2021 15:18:30 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <87tuifv3mb.wl-maz@kernel.org>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 20/09/21 14:22, Marc Zyngier wrote:
>> I think that's only ARM, and even then it is only because of
>> limitations of the hardware which mostly apply only if VHE is not in
>> use.
>>
>> If anything, it's ARM that should support module build in VHE mode
>> (Linux would still need to know whether it will be running at EL1 or
>> EL2, but KVM's functionality is as self-contained as on x86 in the VHE
>> case).
> I don't see this happening anytime soon. At least not before we
> declare the arm64 single kernel image policy to be obsolete.

--verbose please. :)  I am sure you're right, but I don't understand the 
link between the two.

Paolo



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 13:40:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 13:40:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190916.340742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJX2-00072N-Ep; Mon, 20 Sep 2021 13:40:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190916.340742; Mon, 20 Sep 2021 13:40:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJX2-00072G-Bw; Mon, 20 Sep 2021 13:40:28 +0000
Received: by outflank-mailman (input) for mailman id 190916;
 Mon, 20 Sep 2021 13:40:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=PJ00=OK=intel.com=tamas.lengyel@srs-us1.protection.inumbo.net>)
 id 1mSJX1-00072A-EK
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 13:40:27 +0000
Received: from mga05.intel.com (unknown [192.55.52.43])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 51f96f64-d30d-42de-add0-92d3bd1e1b81;
 Mon, 20 Sep 2021 13:40:24 +0000 (UTC)
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 20 Sep 2021 06:40:22 -0700
Received: from tlengyel-mobl.amr.corp.intel.com (HELO localhost.localdomain)
 ([10.213.166.36])
 by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 20 Sep 2021 06:40:21 -0700
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51f96f64-d30d-42de-add0-92d3bd1e1b81
X-IronPort-AV: E=McAfee;i="6200,9189,10112"; a="308677197"
X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; 
   d="scan'208";a="308677197"
X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; 
   d="scan'208";a="532167160"
From: Tamas K Lengyel <tamas.lengyel@intel.com>
To: xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas.lengyel@intel.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2] x86/mem_sharing: don't lock parent during fork reset
Date: Mon, 20 Sep 2021 09:40:11 -0400
Message-Id: <8fa3c3e10446f99b75e3d4aef10624cfa2fdf72c.1632145015.git.tamas.lengyel@intel.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

During fork reset operation the parent domain doesn't need to be gathered using
rcu_lock_live_remote_domain_by_id, the fork already has the parent pointer.

Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
---
v2: NULL the parent pointer early in the fork error path
---
 xen/arch/x86/mm/mem_sharing.c | 15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 8d5d44bdbc..74d2869c0e 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -1861,9 +1861,9 @@ static int fork(struct domain *cd, struct domain *d)
  done:
     if ( rc && rc != -ERESTART )
     {
+        cd->parent = NULL;
         domain_unpause(d);
         put_domain(d);
-        cd->parent = NULL;
     }
 
     return rc;
@@ -1879,9 +1879,10 @@ static int fork(struct domain *cd, struct domain *d)
  * footprints the hypercall continuation should be implemented (or if this
  * feature needs to be become "stable").
  */
-static int mem_sharing_fork_reset(struct domain *d, struct domain *pd)
+static int mem_sharing_fork_reset(struct domain *d)
 {
     int rc;
+    struct domain *pd = d->parent;
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     struct page_info *page, *tmp;
 
@@ -2226,8 +2227,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)
 
     case XENMEM_sharing_op_fork_reset:
     {
-        struct domain *pd;
-
         rc = -EINVAL;
         if ( mso.u.fork.pad || mso.u.fork.flags )
             goto out;
@@ -2236,13 +2235,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)
         if ( !d->parent )
             goto out;
 
-        rc = rcu_lock_live_remote_domain_by_id(d->parent->domain_id, &pd);
-        if ( rc )
-            goto out;
-
-        rc = mem_sharing_fork_reset(d, pd);
-
-        rcu_unlock_domain(pd);
+        rc = mem_sharing_fork_reset(d);
         break;
     }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 13:40:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 13:40:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190919.340754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJXI-0007PZ-Nm; Mon, 20 Sep 2021 13:40:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190919.340754; Mon, 20 Sep 2021 13:40:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSJXI-0007PP-Jn; Mon, 20 Sep 2021 13:40:44 +0000
Received: by outflank-mailman (input) for mailman id 190919;
 Mon, 20 Sep 2021 13:40:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TWfd=OK=kernel.org=maz@srs-us1.protection.inumbo.net>)
 id 1mSJXH-0007NC-PY
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 13:40:43 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b5da75c-d1f3-4d90-be8e-8414f264d9b2;
 Mon, 20 Sep 2021 13:40:43 +0000 (UTC)
Received: from disco-boy.misterjones.org (disco-boy.misterjones.org
 [51.254.78.96])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mail.kernel.org (Postfix) with ESMTPSA id 32B9260F21;
 Mon, 20 Sep 2021 13:40:42 +0000 (UTC)
Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org)
 by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2)
 (envelope-from <maz@kernel.org>)
 id 1mSJXE-00Bm01-5z; Mon, 20 Sep 2021 14:40:40 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b5da75c-d1f3-4d90-be8e-8414f264d9b2
Date: Mon, 20 Sep 2021 14:40:39 +0100
Message-ID: <87o88nuzzc.wl-maz@kernel.org>
From: Marc Zyngier <maz@kernel.org>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Sean Christopherson <seanjc@google.com>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>,
	Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley
 <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	x86@kernel.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross
 <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org,
	linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
In-Reply-To: <7a5825d1-d6e9-8ac8-5df2-cce693525da7@redhat.com>
References: <20210828003558.713983-1-seanjc@google.com>
	<20210828201336.GD4353@worktop.programming.kicks-ass.net>
	<YUO5J/jTMa2KGbsq@google.com>
	<YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
	<662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
	<87tuifv3mb.wl-maz@kernel.org>
	<7a5825d1-d6e9-8ac8-5df2-cce693525da7@redhat.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
X-SA-Exim-Connect-IP: 185.219.108.64
X-SA-Exim-Rcpt-To: pbonzini@redhat.com, peterz@infradead.org, seanjc@google.com, mingo@redhat.com, acme@kernel.org, will@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, guoren@kernel.org, nickhu@andestech.com, green.hu@gmail.com, deanbo422@gmail.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, bp@alien8.de, x86@kernel.org, boris.ostrovsky@oracle.com, jgross@suse.com, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, hpa@zytor.com, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, sstabellini@kernel.org, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, artem.kashkanov@intel.com, like.xu.linux@gmail.com, lingsha
 n.zhu@intel.com
X-SA-Exim-Mail-From: maz@kernel.org
X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false

On Mon, 20 Sep 2021 14:18:30 +0100,
Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> On 20/09/21 14:22, Marc Zyngier wrote:
> >> I think that's only ARM, and even then it is only because of
> >> limitations of the hardware which mostly apply only if VHE is not in
> >> use.
> >> 
> >> If anything, it's ARM that should support module build in VHE mode
> >> (Linux would still need to know whether it will be running at EL1 or
> >> EL2, but KVM's functionality is as self-contained as on x86 in the VHE
> >> case).
> > I don't see this happening anytime soon. At least not before we
> > declare the arm64 single kernel image policy to be obsolete.
> 
> --verbose please. :)  I am sure you're right, but I don't understand
> the link between the two.

To start making KVM/arm64 modular, you'd have to build it such as
there is no support for the nVHE hypervisor anymore. Which would mean
two different configs (one that can only work with VHE, and one for
the rest) and contradicts the current single kernel image policy.

It is bad enough that we have to support 3 sets of page sizes.
Doubling the validation space for the sake of being able to unload KVM
seems a dubious prospect.

	M.

-- 
Without deviation from the norm, progress is not possible.


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 14:12:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 14:12:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190939.340769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSK1g-0002xM-7Z; Mon, 20 Sep 2021 14:12:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190939.340769; Mon, 20 Sep 2021 14:12:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSK1g-0002xF-4K; Mon, 20 Sep 2021 14:12:08 +0000
Received: by outflank-mailman (input) for mailman id 190939;
 Mon, 20 Sep 2021 14:12:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSK1e-0002x9-MM
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 14:12:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bb12df1a-1a1c-11ec-b875-12813bfff9fa;
 Mon, 20 Sep 2021 14:12:04 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-WaL_l8gnOOaqUlZZoGgDjw-1;
 Mon, 20 Sep 2021 16:12:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5599.eurprd04.prod.outlook.com (2603:10a6:803:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 14:12:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 14:12:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0195.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 14:12:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb12df1a-1a1c-11ec-b875-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632147124;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=VuX2g+UX7ZSv6cPH9po0gV2was2+PnkUBViG0eX7vuw=;
	b=CVabM7AQQWLprEHp8SbUR17ZKcHKugYzkrO11sJPXx74XfYmLNbawQTGZN/xkMsPg0BMU9
	51XU5xQBJ5/I2PY6S8mBRdtrcjnN9ESDJUYOZjIWHTmCpUgu1QFRtSiYmiNAzv2o4r4g7q
	IMCb9zq/scWJ9F2i4OTKkivWiOXUp3Y=
X-MC-Unique: WaL_l8gnOOaqUlZZoGgDjw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fWc1E5uVqWHepMl8WJYvacqXIuallh0fccQgk8oe5yxr3Hf3+Ahwe3HLEa3iBNaPwRLYhpKsnMthIlQ7gCoRXGgpSSPuojCIDo66ILslpMflT7Kc7pegKnnJl1GZp2mrG72CfysHnREf2veCd8a5uofYVa4O6NEwLpWmcbd75MqQVzkUcXFDtRcMn5kZ7K8vvzHwA1oW/L74EB7sYE1T8RhtFg8A1s2zZwCEvgibgnkDIhaTejkEetNyD69HnRrfIS7oNdsn/kNTZqH3h7H4GTiENgNcmrFNnKNqfShTLuXdTkXHXJVl+RcmsnlqQzAssKyf8PRG3v5j89gcKJegNQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VuX2g+UX7ZSv6cPH9po0gV2was2+PnkUBViG0eX7vuw=;
 b=HDOoRo73McqsIszaWbFoaJw5fWG1lxuChQlQjVi1CK11nngsZVZWscA+Mul3mbreq9a/a7bmGs1i7v1mZVQw/6awXVRGqqE+2bYrb4aiptjnsKytVikRbeo6+NmFM323t51dhBxLYzv6Vx0TfnJCnaMAT2WEc1Xv2KMkKFxKVJVDOmrUKZPrDi4yY5jawRVbtvT4/dilr42gNK5nJI5Tq7bUtuLP/KqTYA0OsAco6Cvht46jGcPHF+YsT7ZO2EkugKpHAChtddOPDZttHT7WlkpDDgGkRijHsrnaoxmLLvjP/9+HXkeRAf4dQSJSkMEqi9YHX5nkGkHzCHokl2Og6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: Linux kernel's msi_domain_deactivate() operation
To: jiang.liu@linux.intel.com, Thomas Gleixner <tglx@linutronix.de>
Message-ID: <19fa9eaf-3c29-1ace-a16d-4ba3adf8a2e5@suse.com>
Date: Mon, 20 Sep 2021 16:11:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0195.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 64042419-a9e4-4145-9bab-08d97c409d23
X-MS-TrafficTypeDiagnostic: VI1PR04MB5599:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB559929554B95429AD9460D09B3A09@VI1PR04MB5599.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UJ6FtyVTK2PVyXNmWn2YAoP4HmDg08SUhTPTvks6YT2dHJMZzM5lGBQELpmJMm/7/JwcBQNk3ya809jsI+K1aqcKyRw72K1SsYBLFFEwyPOcOtllJGk2UlNGBYnQcFXdiVE5PuuRtGBw+fZ0jFtEEhKQJ2nG3SH1UkkxlrwhzTXXJm++SqJr+zcrzT8FOjwdmDiZvqtZy6bLhWHYl5iEsvuieO4qXB13PPj8eijBe1j/Tcf96rrDfnYEtBwmBDiIBZBGThmSMuk/j0yD//Sw81BL8SEE76RJiauqBpmpREltcLnB6QiWjXg0QCxAb03DHrcEK99Hy7kL3DM8vO5CfHy/i82SEWGmTw02cU1mpoBZxJgOV4shjycupqlT+NOVyr6b8L84ops0GC4EurinYovwo8vfHub6iokR3F3jqgRaP94ZfAhqlZ5SUXZ9j+ko6HIlHEpcAZkMdwc/dRpl8VSVAQHQ3xQGFKuD7UUn1dFPR+H584yZYdiGEsaPvaDdcMURJQ4Dz2aTxtRW9q9YaBoBK2Z1+sM0fCBpRMGvFKGSpKaqFdf6c2pKwi+8Wb066kEtsuW0w85Dorgk6sEMl3S+MJzCdY4V0gGmnjUhAlTcBi2l6/qU4KfAGML0TXlLa3dsJA/ssBt75QpAs7bDpOwWRDgq/NK7qGv7NplPUiqzP7+VxliptCk+ag4GkTcAW0g54Hq5GWuY1QeGqdzFPOV1eH4InpoVetITbx02VX4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(8676002)(66946007)(316002)(16576012)(83380400001)(508600001)(956004)(2616005)(6916009)(66556008)(186003)(8936002)(38100700002)(86362001)(31686004)(31696002)(2906002)(5660300002)(26005)(6486002)(36756003)(66476007)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cThxbVcwUG8xNHdmNEc0NkN2THM3OVVKZ0RLakJwRWN3QlRqYXVJck5oSHN3?=
 =?utf-8?B?WTY1SVM0RHlEYXgrdlNXLzdLLzl4ZExQSGpnWVlGbzJWY2xZRW03akVFQnF3?=
 =?utf-8?B?SE5mVmdaeUpxVTUxS0pFT2cweUhUUnQ2aUkzVVMrcXpnM21kUFRFdUwvaHJS?=
 =?utf-8?B?QitZT2hSbGVHN3gvNTJTZFdySk1YWHlBbG5BU2Q4bjZ5OXEvVnhobXl1K3JF?=
 =?utf-8?B?cW9IT09oOUR4NUEra0JMVnNMdVpCbEFZTTVXY0ZTeFJjZUJVVGxrd0MrL0dm?=
 =?utf-8?B?Y3Z2eWVXUVlJOEs4WWhPbUlOVFBQVnhOTG1PcGxMbGFZdGY0clBJbmJZNEdq?=
 =?utf-8?B?Wkd4SGVCTUZmTGZ5NmdsMVAxUFhOcUNtZjByQklCWDJJT1dpUEZjZkVTS3BW?=
 =?utf-8?B?eUd3bWRqRGUzN0IyOCtEbXZPV3JWTnk2azZyUDYyZkhKcVhqRVV2NXNQTk10?=
 =?utf-8?B?a3h6dTE0dDNQVWRycG1maktkWnFkVmZZMndFQ2Z0ZGVVNzRHYTB5d0UzQi9D?=
 =?utf-8?B?cFZUVTR1L1IrSlAwVnhBbzVoaUNyWS9CL0FuOWk4aWFPRW85ZEJxN0RpQ04z?=
 =?utf-8?B?VTYxZ3BGZ2N4V2Nxa3hVUWcvWjhnNmxFWU9hRGEzb2ErMlNyOC9iZEh0b3Nx?=
 =?utf-8?B?bVNOckZoa1RtSG5GdFNXTWh1MkY5YU1DSzJVRGRPMjJxNmIrNVhGcFkrVk8y?=
 =?utf-8?B?bFZlK2ZEWktXMTRjQVdMQi9TSmRyWUUvdW11ckxvQXg1SVpSWTlPVGxkb3di?=
 =?utf-8?B?K3pMa3NDWGRhNS9FWjZaNGQ2L1JIZWdYK3h2dVI4M3JJdVlDc2ovcDBFNk9L?=
 =?utf-8?B?b2I3TXQ1N2hvaHMxR09FcXV1ZHk3NzQ4NU9JQVdxWllEZU9DV2xRSHFIcFRk?=
 =?utf-8?B?eVhiZmE1ZmM1MDcrVEVWU3I4cS9RYUxvMGpvN0R5ZXlQbkJtQ3MzdWpzMWtv?=
 =?utf-8?B?c2kwVCswTnhCSkI4YzlaeS83NDlEOXhHTEFmb2ErQUp4RjZNZ01GMWxYTzV1?=
 =?utf-8?B?SkRObXNDSU5BdGtaTmdlRVF2b1FYTWdOTDlsUWtSMTNJcU9odlRvSTk3dzla?=
 =?utf-8?B?a09lL1JyY3d0L2Y3SFMrWk9tbEhXZUZQck5lcnBtLzB1eWlxTzAzZkpGT0Jq?=
 =?utf-8?B?YzZwTmlNWVZBendQYUlMcEVoN1o3ZklhWE00bEM0ZnF0cjBhYzlwSlRsNWZ4?=
 =?utf-8?B?SjcxaVZGY0JMYTFOOGVQRTB1WjRNM2F6Zk96NWFmUS9xTlpYazhMSlE2OXJo?=
 =?utf-8?B?akVOTVVaNU1ScDZzSVU2bDhiQmR2dmpYeEZZcTI4ZUdNZmQrUTljQkNmVjBW?=
 =?utf-8?B?UkR1cEtIT0h3TDFFL1ZhVHYxdWtqZDRlZXg1dkJmMUxKYkMrSk9tWlBWOFFo?=
 =?utf-8?B?OFVYMk5YdzRmOEVmN0ZxbzJMOHoycGtkZ0tGTzYyeCtkNGdEU090NmJCNThl?=
 =?utf-8?B?R0xrTEZTb3VxM1c4N2JKaFAxMVh1VC9yR1A2YVVBUTJPUlFOOU5EMjRmdmpi?=
 =?utf-8?B?cWtUL2IvbS9WSDBSMzNmQXBxaDdEdG9RbUc4RzVTQ3NiVzgxeDVWcHVsU0NR?=
 =?utf-8?B?MkNkellKT25uRG5uUWRFVWh2cHRpa2JaTzZDQkxVRnkrU09hQmFkdEhOUFlN?=
 =?utf-8?B?TEJFNDdCWENuRzFvMXI0MXdKc29yVnN6ZGdqdmRrWFdSZmdLa2xaeXRybUNW?=
 =?utf-8?B?K2RiMkFabEJNMUtmOXFPNG9tWVBmdnN2WW85cDRLRmtaSHV6WGNCL2tYSnNm?=
 =?utf-8?Q?25b+2Y1VzGlT+undaA0LfEhHGor2pHSBRiWq+j+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 64042419-a9e4-4145-9bab-08d97c409d23
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 14:12:00.6942
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GiXq6+skDJPSv1BCqk9KEHRYdruB91i2ALyMZRYJDcbnDdRJIIww/DVpNLrxNPAsr3QBFQawXnUNyaSH1NJQsQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5599

Hello,

I notice that from its introduction in 3.19 this function simply clears
message address and data. I wonder in how far this is "deactivation":
Aiui the device might still signal interrupts this way, by writing
32 bits of zeroes to address zero. (That's not going to result in an
interrupt, of course.) Shouldn't deactivation rather mean masking (if
the entry is maskable; looks like this is happening in an earlier step
when possible) or disabling of MSI (which unfortunately would then also
affect other interrupts, if multiple are in use) or some software means
replacing masking?

Background: For PVH Dom0 under Xen we check (in Xen) whether address
fields written actually fall in the designated 0xFEEnnnnn range (as
long as MSI / MSI-X is enabled at the time of writing of these fields),
logging outliers. As it turns out the message appears quite a few times
per session, primarily - but not only - during shutdown. We log these
instances not because they're wrong (as said above, this is in principle
a valid mode of operation), but because proper handling would require
further code in Xen which we were hoping we would never have to write.
(Thinking about this again, it shouldn't be all that much code.)

In a small subset of the cases the operation actually occurs with the
respective entry masked (in all other cases there's no maskbit support);
we may want to consider suppressing the log message in such cases, yet
then we may need to watch for this case the next time the mask bit gets
cleared.

Of course in the kernel you may have reassurance that the device driver
has actually disabled the interrupt source at the device before this
deactivation. We don't have such secondary knowledge in the hypervisor,
i.e. we can't use that to suppress the log message.

Any insight would be much appreciated, including the possible pointing
out of us not having understood the underlying idea.

Thanks, Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 14:15:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 14:15:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190947.340780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSK4T-0003er-Qj; Mon, 20 Sep 2021 14:15:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190947.340780; Mon, 20 Sep 2021 14:15:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSK4T-0003ek-NI; Mon, 20 Sep 2021 14:15:01 +0000
Received: by outflank-mailman (input) for mailman id 190947;
 Mon, 20 Sep 2021 14:15:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSK4S-0003ec-Py
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 14:15:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 28b56783-5816-42d6-aee8-f5201fb81db9;
 Mon, 20 Sep 2021 14:14:59 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-MC_0zQUzPwCCcnLEdLJ6Lw-1; Mon, 20 Sep 2021 16:14:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5599.eurprd04.prod.outlook.com (2603:10a6:803:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 14:14:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 14:14:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0080.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Mon, 20 Sep 2021 14:14:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28b56783-5816-42d6-aee8-f5201fb81db9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632147299;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ioWPBDicJ37FWygQgxAY5/QbpM1cnDtK6Z5h+U06auQ=;
	b=U3MwM7r4nFbN6nV7nVc/2v/WZIgR3OatDxrr8EO+HiYLmM8PFsEJ9fSpeideuaqekNS+O1
	ulb5COk5yYAW74oQSXOO3M8eN4qrsFS7cWbC1BVPUIozOQRHdj2MCDI8RZZaLZJ2Q8Ceml
	AD214f3cXIWshvQNf9RBruzldz8kM2g=
X-MC-Unique: MC_0zQUzPwCCcnLEdLJ6Lw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j+a/EVF2YFny8aXVsGpSh//2c56fFI52hSqHNRYkbvMojcm5o1Zx313gvllI4XG0O9HiRNx0trJFftCJgvO+rExnZ/7niGuvNluedEGhW/8lVjbyb9DOEPLJu8hRVkGBxIxX8w7MPVF7Me0G3aCvPzdhr2Z9u8X7CpUrqpdWLIrjgbTqQ/FMpFROoFjQYvdSSwccuo/krsvqlmLhu3/Nb6/DKyzICNmSFvGbkK85g5F4yGIrw/45pCn0MUsyBHcVA8qaZO5NNkUp5LJ/nQgMAax1T6upCszTrx4AxDNAoEUOr+p40YZWTii+pjrY3wR6g2G5gYUkMiZfX7iSK6Bq+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ioWPBDicJ37FWygQgxAY5/QbpM1cnDtK6Z5h+U06auQ=;
 b=avrHeVRTIFa3IqHU5qPmuwrfaM3FQqdJ+9Z7FTPHzar1x8bX5+n6rGpGKoPKop8WNZ7rW4DGgBBagbLOPK8g6ritY8glb706mi3e8ukls7yZismuZ1u33hP/yKwt10klDFJvHGZZWBOL1NtjoYmmuK8yWz/D5p/uEliTf9hXxYoavYmfkQ3sLR8SgyTrttnuOEjLktTL552rQSiOtGVan8WWt0Y9R4aLAYybzdvSRAE8FBlaCSkBHzHgv/uxkjkMr7NRHo9ZZCfjs3keQo/WcjPBeyhsPDxA8pCe7/IhfSCGH2RF2gwjeEgfffHG+ECizkcewUoHyF6fSOavkpR0mw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] x86/mem_sharing: don't lock parent during fork reset
To: Tamas K Lengyel <tamas.lengyel@intel.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <8fa3c3e10446f99b75e3d4aef10624cfa2fdf72c.1632145015.git.tamas.lengyel@intel.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <342dd9a2-88b3-34d5-d2ee-b288f8d81f5e@suse.com>
Date: Mon, 20 Sep 2021 16:14:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <8fa3c3e10446f99b75e3d4aef10624cfa2fdf72c.1632145015.git.tamas.lengyel@intel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0080.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f941d8fc-c0d9-4d0c-91ca-08d97c4105d2
X-MS-TrafficTypeDiagnostic: VI1PR04MB5599:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5599B788FD5CFF01397521A0B3A09@VI1PR04MB5599.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1751;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xHfguBfiBHBBW+k2VVQ4daTMP7Wpap5H5dJwlcaq0BzS4laEiu+99Lw1OPEs7Oglmak1kOJXOOb3zTFWV7x9FDKqkqfq6PjjXU9Gx1/RMZ33zi2EelN1bp7FkXRFll8qS/FWsVfdyKbBTExkyfOkrZDalvmcP6BoMc65i5BD9Y0spgRnwv/UcZ0qMKT5Urq9ldzOoU85RmuhivcyNkSa92+f40wNWG19kYCqez7Ok56XLLTfYjxgvJeXZH9tnROXpI7m5BPsIxXe8omemZl5V768PVBT/RlyBFduTFtYNnrlfRMAwQRa7tPuQKADsShiJ4JTKYB2JWxpYviCYJzs4SMl+ZftqustiSrswAfCOrZBi/vgcpLHk0gQ912UAdWf807dZdOHt7/4/D4msuPp9JWGgwLTHxrk3Ugo4btSFuNwmOxw16Q5DVZPThwNWujQd87C5izvE32r9fALBhLXmMPUYfx1lCj6AzBl3OePUiTknLif8C12kLw41ZmBmLw0aGF5/t8qJ6prtnuNXR81gdTypbnO2CgZ/g1V0Hqpq4XjRQ2iZG/DvEbHZ5KybFrbsmaEYvvG2G9xlARrwKJLJPc29+IaCSHPr8tlf+pd/V+JBiGVgTCbGdVjyNG+1Gz8lbulv3wMyFZ85uqoOu8JaupZnRL7wV2n1FPCPeSt2NOok0s1+XrkchboryzfSg4Y6rFO/sjMd4i/NJT8xL168T3C+uwiAwoTcmTUemt8ti4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4744005)(54906003)(8676002)(66946007)(316002)(16576012)(83380400001)(508600001)(956004)(2616005)(6916009)(66556008)(186003)(53546011)(8936002)(38100700002)(86362001)(31686004)(31696002)(2906002)(5660300002)(26005)(6486002)(36756003)(66476007)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?U2JCNE05UkVIWDBiUEFlMEs2djZ0YUEveE1FRXpqcW1XYU1iN0ZRbllCZDRC?=
 =?utf-8?B?dGxTOUNwZU9iemVSY1pJSTh5d1JheVBVWTJxbXhOemU4VzRub25kSG0vK2h2?=
 =?utf-8?B?NDR0Q2M3MTRSa0xFWTlaR1cvaXE2RHQrY0Z2b2JPc2x0ZlNIeGNHa0pIalNi?=
 =?utf-8?B?eVczcVl2Z0FGRVpVYmttck9KL1lXMDhNQ2V6V05iZmJGcDlzUUxodE93OHhx?=
 =?utf-8?B?ekhWc3NvQmNQK2E5T3V1elM2ajd4SUJRN2dpTEZydTZ5TTlJWVRRMThYa2VE?=
 =?utf-8?B?ZmZVUFh2WGxJa3pZZU5lSW1tTjg5SWVhVWNsd2lsL1hrWUdCTWJQQ0pCOTMv?=
 =?utf-8?B?ZGQvVHAvc01SVkVVSGlFck1UcG4rdXc1MHFuSmVBZlNCdFdiSG1mYythS0Rx?=
 =?utf-8?B?RmRFTjdRNnNpWUZITEk5S3h3QXlldGxPcHl4TmpKbGxyQlBTTkFaVVVnKzVO?=
 =?utf-8?B?dEhtNUlJR0Z4UTRGUVBvSFJ6VWpJZ3pRcVJIRVlZNnlLaE9EL3BPUm04SlNv?=
 =?utf-8?B?eGwzZTJSTERVK3VZUENMRWhISWlESTcvdkR6NVdsQ0hESFozcGk0eHc2MENh?=
 =?utf-8?B?QjQrVktMWmh3NG5IWW9xekRtUmdJcldubjRGTUlYbGdjM3MyZUU0UE9LT3hC?=
 =?utf-8?B?dFFUQXk2SGNzOVc4U0pNNVpMYk1KRmx0WkFDTHR0Q1J6TEpmYkN3cDV1RGNp?=
 =?utf-8?B?M1UzT1psNGs5K3FoQVZjYjJEdTZQTHRlV3djYmozbnR2L1lRQVlxbk5aa29s?=
 =?utf-8?B?MFVyK0pXL0M2WE9yRngyUkdxWG5PdTFiYmRnSVZXaVBicjVIaEdUTkVOcGZn?=
 =?utf-8?B?VDVHSmV1Z2EzR1lXVE4ycFZaYWJNeTVjVCs5bDhtbFRva2VYZDhhcjRZNzN2?=
 =?utf-8?B?SWpnd2paQ1ZHcW0yMUY4aXJMUmpLUDQzdFRPS2xlQ2M0R3NwMG1LYmZzdnBC?=
 =?utf-8?B?bElzaHp2c2tVSDF0YUVCb2xOcHdMN3lMY09Wa2hWSlNWQURPeFdhdDJiNk40?=
 =?utf-8?B?NlBCaEcwSTQ5cytpeUtvV1dqL1pYZjduWityQlQ5OGlxNWsvR0ZvSmpXbFha?=
 =?utf-8?B?WkxSb0Rva0ozU1NtL0w1cEFsUkhrcm5tcG1yNlpXK1ZhMlgyS1czc21mdm56?=
 =?utf-8?B?TkNmOFFBMkZ1VkRYd251STBoTHRhTHN1WkV5OWNGSHNyTUhDYzl6NjNZc3kx?=
 =?utf-8?B?L2EvYjloNnlWR21hVWxoUUZmUmhxOVNYM2Q1YXlKa1pLZndZeW0wNnZtbnBV?=
 =?utf-8?B?Tk9vbm9jRTRIaXFEV3NUY3Zqa0RKbVhNRWczV0gwSXpaM1NFMXNVazhmeVRJ?=
 =?utf-8?B?amZBbW14alFWa2lJbUlLOG5lMDJ4OEl4ajdXTUwzSFp2WUYvTkJQclN4SzZL?=
 =?utf-8?B?ckw0ejFsamVZWUF4NFlvTlJDbUVWa3ZobHNXVndnYXMyMEFFdEZ2VmpLR2or?=
 =?utf-8?B?QUhVdVRtTXpzaU9DVjVPd3dKZzFvbzlsdzd5UlAvSkFMUXI2UHZQRm9iYnVC?=
 =?utf-8?B?c3lHdkk2OXplRmNldTRxUG8rcEUvcnFpSDA2bDFNOW9RZ0FNSHY0Q3ZFS0I3?=
 =?utf-8?B?V2IyQ3lrVDd4a28xQWRiMlNIWGdIRFdKODgwL3NyZUZOWGljUmhnaDdHUVNO?=
 =?utf-8?B?a0VIWXNON2VTWVlXZHlRelppNVBGYXp1ZC8rTjZYaTBaZ3UrVXF0c2laOTBn?=
 =?utf-8?B?aXRxWjRQQ1laWTNpSTFRdVZrMldiRkxuWGNBcThtQ3lyQVp1d1FBSUZoVjAv?=
 =?utf-8?Q?xL3C7NgaZODIKUe7K8eLhzogJvoNymGGVx6XEM0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f941d8fc-c0d9-4d0c-91ca-08d97c4105d2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 14:14:56.3199
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: N8MY84bX6i15+O6kVdT2ug0qj9aL2viaiettI90sDErK51LcIL/zotYt5Auje5abpNxeGsfq/mDOf8HlUQDf9A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5599

On 20.09.2021 15:40, Tamas K Lengyel wrote:
> During fork reset operation the parent domain doesn't need to be gathered using
> rcu_lock_live_remote_domain_by_id, the fork already has the parent pointer.
> 
> Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:18:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:18:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190964.340791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL3n-0001pI-Da; Mon, 20 Sep 2021 15:18:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190964.340791; Mon, 20 Sep 2021 15:18:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL3n-0001pB-AX; Mon, 20 Sep 2021 15:18:23 +0000
Received: by outflank-mailman (input) for mailman id 190964;
 Mon, 20 Sep 2021 15:18:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSL3m-0001p5-1q
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:18:22 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fcf848d0-1a25-11ec-b876-12813bfff9fa;
 Mon, 20 Sep 2021 15:18:20 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E81D521F17;
 Mon, 20 Sep 2021 15:18:19 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C064213A61;
 Mon, 20 Sep 2021 15:18:19 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id J0WJLTumSGGKHgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 15:18:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcf848d0-1a25-11ec-b876-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632151099; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=0mIeLZaSMwTeZ9TavvQTLwfuOguzPlIZjeInUwea/9Q=;
	b=DI+v2JHoc8AY51gvMNcgs7SLKYZmig3VBWrhJqfJUuy1aItbNvpB35lvVM37ET3lKNBK3J
	K+GLpG6P8yfvhW4dtvdD+CKER4iBgLriB4BG1uDTp2oDeDqK022xKkVNGJpNlt6yeWmbRb
	/2pOSW7xuE+5htID246WYDPgHQOwAZA=
Subject: Re: [PATCH] xen/x86: drop redundant zeroing from
 cpu_initialize_context()
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <425ba7a3-9a1e-7ff5-f6de-ba9969116941@suse.com>
Date: Mon, 20 Sep 2021 17:18:19 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="POxhCh9cDNfQXHpNNVoHMXj9HrJL1resm"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--POxhCh9cDNfQXHpNNVoHMXj9HrJL1resm
Content-Type: multipart/mixed; boundary="sXjXXzKmDfQ3KZ30to8imRN1YIZJsiqcG";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <425ba7a3-9a1e-7ff5-f6de-ba9969116941@suse.com>
Subject: Re: [PATCH] xen/x86: drop redundant zeroing from
 cpu_initialize_context()
References: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>
In-Reply-To: <14881835-a48e-29fa-0870-e177b10fcf65@suse.com>

--sXjXXzKmDfQ3KZ30to8imRN1YIZJsiqcG
Content-Type: multipart/mixed;
 boundary="------------A0BD770B8B15DA09A26DD48C"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A0BD770B8B15DA09A26DD48C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 16.09.21 17:05, Jan Beulich wrote:
> Just after having obtained the pointer from kzalloc() there's no reason=

> at all to set part of the area to all zero yet another time. Similarly
> there's no point explicitly clearing "ldt_ents".
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Pushed to xen/tip.git for-linus-5.15b


Juergen

--------------A0BD770B8B15DA09A26DD48C
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A0BD770B8B15DA09A26DD48C--

--sXjXXzKmDfQ3KZ30to8imRN1YIZJsiqcG--

--POxhCh9cDNfQXHpNNVoHMXj9HrJL1resm
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIpjsFAwAAAAAACgkQsN6d1ii/Ey+s
8Qf9HJwDtAJLXp1o4sVOAw0dkjQjSsqQVO4F3xNZpqMVGvlvStjQ+yOFyGaV6mIIEg8Os0QaZtOw
uZ/Erms71Lx313yd3Lj3rach+kZSKXFGnR/ndVWTgGhFfK9kIrqgqbuGAfGvq9VeIZ+9RdovG3RO
kNHEOiIAs1pS923biCJ2JTh2jtT8ahUHRpyQ7I5jQDpX0are60hpGMFSWAHR8U0v4VDtk6KAaEYt
r/DvmY8r3Cx4YIid1ujymYtPW7Y6u1XaS7oiHVN7npfs0u9fOvvpC3qPXpa5IxpehIU52+TExdm7
m/SDoPtIEGC5u6sxcVZBO2kePj0F6a4F52EbbCBftw==
=uOZY
-----END PGP SIGNATURE-----

--POxhCh9cDNfQXHpNNVoHMXj9HrJL1resm--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:18:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:18:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190967.340802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL4I-0002Kx-OO; Mon, 20 Sep 2021 15:18:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190967.340802; Mon, 20 Sep 2021 15:18:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL4I-0002Kq-KR; Mon, 20 Sep 2021 15:18:54 +0000
Received: by outflank-mailman (input) for mailman id 190967;
 Mon, 20 Sep 2021 15:18:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSL4G-0002I7-Sn
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:18:52 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7144a6c8-2b57-4e90-be5f-3a46e592ddfa;
 Mon, 20 Sep 2021 15:18:51 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 3344F200F0;
 Mon, 20 Sep 2021 15:18:50 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 013E813A61;
 Mon, 20 Sep 2021 15:18:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id BuCwOlmmSGHIHgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 15:18:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7144a6c8-2b57-4e90-be5f-3a46e592ddfa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632151130; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=pkTjru/PAEkB2P2EV7YbVR87Hnj75AS/hOEMYeht3Rg=;
	b=lEfJ+Wy0xwPd+eiCGASf3r0840uJ3ZKvD4MyLIaKTbI0H3OFu5XDgHTvohY8m4+qkCvpWR
	4sBZvBVMHUzL/bLdS7rebpTPGO0WwChM1E4fxbD8BRzjCndlxFkf+2+1mmROwvjs/uJnLT
	BbdkchYFgdnhE1AhxnSIRbQNJNl7CO4=
Subject: Re: [PATCH] Xen/gntdev: don't ignore kernel unmapping error
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <8084a911-b768-cc6d-8b06-ae0c358466d2@suse.com>
Date: Mon, 20 Sep 2021 17:18:49 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="AE1KLi7Ljs1cLKslA7vFh8Z6YryVAZisk"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--AE1KLi7Ljs1cLKslA7vFh8Z6YryVAZisk
Content-Type: multipart/mixed; boundary="nNntOiF3zUqSEDLgpGT46Xk2If5nO0koO";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <8084a911-b768-cc6d-8b06-ae0c358466d2@suse.com>
Subject: Re: [PATCH] Xen/gntdev: don't ignore kernel unmapping error
References: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>
In-Reply-To: <c2513395-74dc-aea3-9192-fd265aa44e35@suse.com>

--nNntOiF3zUqSEDLgpGT46Xk2If5nO0koO
Content-Type: multipart/mixed;
 boundary="------------BBCF014F1A6D2869C05A0BCE"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------BBCF014F1A6D2869C05A0BCE
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 08:13, Jan Beulich wrote:
> While working on XSA-361 and its follow-ups, I failed to spot another
> place where the kernel mapping part of an operation was not treated the=

> same as the user space part. Detect and propagate errors and add a 2nd
> pr_debug().
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Pushed to xen/tip.git for-linus-5.15b


Juergen

--------------BBCF014F1A6D2869C05A0BCE
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------BBCF014F1A6D2869C05A0BCE--

--nNntOiF3zUqSEDLgpGT46Xk2If5nO0koO--

--AE1KLi7Ljs1cLKslA7vFh8Z6YryVAZisk
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIplkFAwAAAAAACgkQsN6d1ii/Ey8w
0gf/bsCaTiT9ct7po6hwInUR2oDfihHjmvv8cM6hFTsWSwoi9AGs6UIQIXzr0Tlcx/DHPhyxkidg
YZM02TT0jCgiisUUNCBZCHrLevq/Navq3I2/aMI8oKwlJ7S7jmms2IdlI7u+hhzdC13jqUa8a4U5
elDpsabjjUn5mVoDaSqxeBkqieojC06qogUNnr7bCFu2Ubxm+iHjM9NS3wFSJb91s5etntGEr47n
Z8/NW3Ygce2CksQlu4ZW/UF+Xbw4xvu78TeQLJqaHokuvJZ/IfYdQxkycsI17/eu+9ddGi9UTY1j
RZeocr3wZuX6/X+JjES8G+WvIqeVUl20vTNZqBSV8w==
=HjA1
-----END PGP SIGNATURE-----

--AE1KLi7Ljs1cLKslA7vFh8Z6YryVAZisk--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:19:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:19:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190975.340813 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL4q-0002wC-0t; Mon, 20 Sep 2021 15:19:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190975.340813; Mon, 20 Sep 2021 15:19:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL4p-0002w5-Tm; Mon, 20 Sep 2021 15:19:27 +0000
Received: by outflank-mailman (input) for mailman id 190975;
 Mon, 20 Sep 2021 15:19:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5pAY=OK=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSL4o-0002va-9u
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:19:26 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 395f4f93-176f-404f-92ce-46aeebef3755;
 Mon, 20 Sep 2021 15:19:25 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C454F2200F;
 Mon, 20 Sep 2021 15:19:24 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8D56013A61;
 Mon, 20 Sep 2021 15:19:24 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 0Vw/IXymSGElHwAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 20 Sep 2021 15:19:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 395f4f93-176f-404f-92ce-46aeebef3755
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632151164; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=oF4uywDoz1MkcVWOc8nMivEHhs75yAhfHJf39oCzPgg=;
	b=h+7nGk35C41c38kU0AMOgFsTfkG/WZCVvvFHco8ZJTLTpoitOBr2WhMSSB79c+evGNvnUZ
	ncDUp120VVXWaKi0gaDsKiQxxY4MYa/ydCtcgwf2bQy1F5IuFLKOypIENiPE7VgjRyLUQf
	Tk2YwQA1Jr8/ODwSTkFrM/kfsRBRXMk=
Subject: Re: [PATCH v2 0/4] swiotlb-xen: remaining fixes and adjustments
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <c4e260c9-6129-1916-5eb2-3e060dbe47d5@suse.com>
Date: Mon, 20 Sep 2021 17:19:23 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="EvbroSQDJKtThYMCIXWpHh3Qch2odPsZ9"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--EvbroSQDJKtThYMCIXWpHh3Qch2odPsZ9
Content-Type: multipart/mixed; boundary="xxxRpcm2fCnpW4erSmvy2q75E96nioyOj";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "hch@lst.de" <hch@lst.de>, Konrad Wilk <konrad.wilk@oracle.com>,
 "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Message-ID: <c4e260c9-6129-1916-5eb2-3e060dbe47d5@suse.com>
Subject: Re: [PATCH v2 0/4] swiotlb-xen: remaining fixes and adjustments
References: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>
In-Reply-To: <164e58ff-2edd-2c99-ac3d-e18eb06ff731@suse.com>

--xxxRpcm2fCnpW4erSmvy2q75E96nioyOj
Content-Type: multipart/mixed;
 boundary="------------BEF8B08AD90F049DF3049A8F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------BEF8B08AD90F049DF3049A8F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 12:43, Jan Beulich wrote:
> The primary intention really was the last patch, there you go (on top
> of what is already in xen/tip.git for-linus-5.15) ...
>=20
> 1: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
> 2: PCI: only build xen-pcifront in PV-enabled environments
> 3: xen/pci-swiotlb: reduce visibility of symbols
> 4: swiotlb-xen: this is PV-only on x86

All 4 patches pushed to xen/tip.git for-linus-5.15b


Juergen

--------------BEF8B08AD90F049DF3049A8F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------BEF8B08AD90F049DF3049A8F--

--xxxRpcm2fCnpW4erSmvy2q75E96nioyOj--

--EvbroSQDJKtThYMCIXWpHh3Qch2odPsZ9
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFIpnsFAwAAAAAACgkQsN6d1ii/Ey+r
pgf/e1d5GEut8n++UJYE9qfcdKF2V8/ZF0N6fYIV2xpPMWFjEBIz7lQkzO/DhseFmOVqqYeqLSS0
9dXsvCpZVtAR7Q/9h0Z0yHMbNTi81y4wuGY1/69J2nRQiw0NLFjk2v4NDHTMP8itwQlzoU4A4Czu
vZtnDiTBewfpf68WM2cELYGMYzNc+2U1ZQoRg+pnrS6jWmdMO79rvCbvPUy3bPSRgDUtotDBXWVk
hXSFHjsXcNi2uYTNNlBn5eAOsJaavXsXHv3NxLl7df+baOvkIIO7O3O0UbjZ/CqtXoKSvRemNNhD
OMA+NEtvh92B1t2voCWVYDPHHpyomhiIHbIb9LHmfQ==
=ek21
-----END PGP SIGNATURE-----

--EvbroSQDJKtThYMCIXWpHh3Qch2odPsZ9--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:23:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:23:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190984.340824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL8J-0004Su-KH; Mon, 20 Sep 2021 15:23:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190984.340824; Mon, 20 Sep 2021 15:23:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSL8J-0004Sn-HS; Mon, 20 Sep 2021 15:23:03 +0000
Received: by outflank-mailman (input) for mailman id 190984;
 Mon, 20 Sep 2021 15:23:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2sKa=OK=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mSL8H-0004Sg-V0
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:23:02 +0000
Received: from forward501j.mail.yandex.net (unknown [5.45.198.251])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7d045627-b172-4198-9eb7-a8f151544945;
 Mon, 20 Sep 2021 15:22:58 +0000 (UTC)
Received: from sas2-599c596a5fe6.qloud-c.yandex.net
 (sas2-599c596a5fe6.qloud-c.yandex.net
 [IPv6:2a02:6b8:c08:b999:0:640:599c:596a])
 by forward501j.mail.yandex.net (Yandex) with ESMTP id 8753A62314C;
 Mon, 20 Sep 2021 18:22:56 +0300 (MSK)
Received: from 2a02:6b8:c08:3c28:0:640:43b4:598a
 (2a02:6b8:c08:3c28:0:640:43b4:598a [2a02:6b8:c08:3c28:0:640:43b4:598a])
 by sas2-599c596a5fe6.qloud-c.yandex.net (mxback/Yandex) with HTTP id
 rMevW95EX8c1-MsEGETRU; Mon, 20 Sep 2021 18:22:55 +0300
Received: by sas2-43b4598adf75.qloud-c.yandex.net with HTTP;
 Mon, 20 Sep 2021 18:22:54 +0300
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d045627-b172-4198-9eb7-a8f151544945
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1632151375;
	bh=nHoYkVBWzGYo9IEZ2moORPpMl0GNP7lENmvA/d+C5Mk=;
	h=References:Date:Message-Id:Cc:Subject:In-Reply-To:To:From;
	b=lOtSxG4m1AgsyTU7xPy7QeLT4HNh4eR/mmpv3bsrwyOY6XhroNgcNPFeQ5r6uLZyx
	 0QX1GMa0EK8Qj/yPtNbZGYED/tm+6vudYqPR74+1fuyqAwjngcHWOtdJdXTwc5Li2t
	 bjLCxxVceUTZsuR8myUbRL2itNHcHx+XNQLHZ3Kg=
Authentication-Results: sas2-599c596a5fe6.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: =?utf-8?B?0JTQvNC40YLRgNC40Lkg0JjRgdCw0LnQutC40L0=?= <isaikin-dmitry@yandex.ru>
To: Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"rjstone@amazon.co.uk" <rjstone@amazon.co.uk>,
	"raphning@amazon.co.uk" <raphning@amazon.co.uk>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
In-Reply-To: <12214401632126774@myt6-4204cefb5b39.qloud-c.yandex.net>
References: <2e7044de3cd8a6768a20250e61fe262f3a018724.1631790362.git.isaikin-dmitry@yandex.ru>
	 <22a4b3c7-7faa-607b-244a-0ab99d7ccfa9@suse.com>
	 <16d43ba8-d45f-021c-0e19-e233cb967938@citrix.com>
	 <f1dda72c-695f-34d3-619e-ec2aa7696bce@suse.com>
	 <01ec71a5-a1f1-b7db-d467-bc4c734db096@xen.org>
	 <0fa0d369-c67c-1cdd-0ff8-1542487ffb8a@citrix.com>
	 <0c860901-0992-74df-4a53-d75a0971d1f3@suse.com>
	 <f6225dc6-0590-3456-8c48-7ab29aa00200@xen.org> <c242e3cc-d4ed-73b7-8e38-f94a8c479d20@suse.com> <12214401632126774@myt6-4204cefb5b39.qloud-c.yandex.net>
Subject: Re: [PATCH v1] domctl: hold domctl lock while domain is destroyed
MIME-Version: 1.0
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Mon, 20 Sep 2021 18:22:54 +0300
Message-Id: <461721632145365@mail.yandex.ru>
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=utf-8

PGRpdj54ZW5sb2NrcHJvZiB3aXRoIG15IHBhdGNoIChzb21lIHplcm8gbGluZXMgaGF2ZSBiZWVu
IHJlbW92ZWQgZm9yIGJyZXZpdHkpOjwvZGl2PjxkaXY+wqA8L2Rpdj48ZGl2PjxkaXY+PGRpdj4k
IHN1ZG8geGVubG9ja3Byb2YgLXIgOyBmb3IgaSBpbiAkKHNlcSAxIDUpIDsgZG8gdGltZSBzdWRv
IC91c3IvbGliL3hlbi00LjE0L2Jpbi94bCBkZXN0cm95IHRlc3Qtdm0tJHtpfSAmYW1wOyBkb25l
IDsgd2FpdCAkKGpvYnMgLXApIDsgc3VkbyB4ZW5sb2NrcHJvZjwvZGl2PjxkaXY+wqA8L2Rpdj48
ZGl2PnJlYWzCoMKgIMKgMG0xLDIyNHM8L2Rpdj48ZGl2PnVzZXLCoMKgIMKgMG0wLDAxM3M8L2Rp
dj48ZGl2PnN5c8KgwqAgwqAwbTAsNzEwczwvZGl2PjxkaXY+wqA8L2Rpdj48ZGl2PnJlYWzCoMKg
IMKgMG0xLDI2MnM8L2Rpdj48ZGl2PnVzZXLCoMKgIMKgMG0wLDAwNnM8L2Rpdj48ZGl2PnN5c8Kg
wqAgwqAwbTAsNzYyczwvZGl2PjxkaXY+wqA8L2Rpdj48ZGl2PnJlYWzCoMKgIMKgMG0xLDMxOXM8
L2Rpdj48ZGl2PnVzZXLCoMKgIMKgMG0wLDAxN3M8L2Rpdj48ZGl2PnN5c8KgwqAgwqAwbTAsODAw
czwvZGl2PjxkaXY+wqA8L2Rpdj48ZGl2PnJlYWzCoMKgIMKgMG0xLDQwMHM8L2Rpdj48ZGl2PnVz
ZXLCoMKgIMKgMG0wLDAxNXM8L2Rpdj48ZGl2PnN5c8KgwqAgwqAwbTAsODg5czwvZGl2PjxkaXY+
wqA8L2Rpdj48ZGl2PnJlYWzCoMKgIMKgMG0xLDQyNHM8L2Rpdj48ZGl2PnVzZXLCoMKgIMKgMG0w
LDAwM3M8L2Rpdj48ZGl2PnN5c8KgwqAgwqAwbTAsOTMyczwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sg
cGxhdGZvcm1fdGltZXJfbG9jayDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA6IGxvY2s6IMKg
IMKgIMKgIMKgIMKgIDIoIMKgIMKgIMKgIMKgIDAuMDAwMDAxNTk5cyksIGJsb2NrOiDCoCDCoCDC
oCDCoCDCoCAwKCDCoCDCoCDCoCDCoCAwLjAwMDAwMDAwMHMpPC9kaXY+PGRpdj5nbG9iYWwgbG9j
ayBmbHVzaF9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgOiBs
b2NrOiDCoCDCoCDCoCA0MjUzOCggwqAgwqAgwqAgwqAgMC4zNjc1MDc3NjJzKSwgYmxvY2s6IMKg
IMKgIMKgIDEwODc1KCDCoCDCoCDCoCDCoCAwLjI2NTkwMjk1OXMpPC9kaXY+PGRpdj5nbG9iYWwg
bG9jayBwY2lfY29uZmlnX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBs
b2NrOiDCoCDCoCDCoCDCoCDCoDE0KCDCoCDCoCDCoCDCoCAwLjAwMDAwNzA2MHMpLCBibG9jazog
wqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xv
YmFsIGxvY2sgdnBtdV9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIDogbG9jazogwqAgwqAgwqAgwqAgwqAxMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAzMTRzKSwg
YmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAoIMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rpdj48
ZGl2Pmdsb2JhbCBsb2NrIGNwdXBvb2xfbG9jayDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoDogbG9jazogwqAgwqAgwqAgwqAgwqAxMCggwqAgwqAgwqAgwqAgMC4wMDAwMDEw
NjFzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAoIMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8
L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIHZtX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDM1KCDCoCDCoCDCoCDCoCAw
LjAwMDAwMjkzNnMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAgMC4wMDAw
MDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgdGFza2xldF9sb2NrIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDMwKCDCoCDCoCDC
oCDCoCAwLjAwMDAwMzMyMnMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAg
MC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgc3lzY3RsX2xvY2sgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDgyKCDC
oCDCoCDCoCDCoCAwLjAwMDMwOTE5M3MpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAg
wqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgY2FsbF9sb2NrIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAgwqAg
wqAzNCggwqAgwqAgwqAgwqAgMC4wMDAwODgxODZzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDEo
IMKgIMKgIMKgIMKgIDAuMDAwMDAyNTkxcyk8L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIGhlYXBfbG9j
ayDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA6IGxvY2s6IMKgIMKg
IDI3ODg3MDQoIMKgIMKgIMKgIMKgIDAuMzYyMzExNzIzcyksIGJsb2NrOiDCoCDCoCDCoDIyMjY4
MSggwqAgwqAgwqAgwqAgMC4wOTExNTIyNzZzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgZ2xvYmFs
X3ZpcnFfaGFuZGxlcnNfbG9jayDCoCDCoCDCoCDCoCDCoCDCoCA6IGxvY2s6IMKgIMKgIMKgIMKg
IDUxOSggwqAgwqAgwqAgwqAgMC4wMDAwNDU4NjNzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAo
IMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIGRvbWxpc3Rf
dXBkYXRlX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDCoCDCoCDC
oCDCoCA1KCDCoCDCoCDCoCDCoCAwLjAwMDAwMDU4OHMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAg
MCggwqAgwqAgwqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgZG9tY3Rs
X2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDC
oCDCoCDCoCA1NDQoIMKgIMKgIMKgIMKgIDAuODcyNzkxMTg3cyksIGJsb2NrOiDCoCDCoCDCoCDC
oCDCoCAwKCDCoCDCoCDCoCDCoCAwLjAwMDAwMDAwMHMpPC9kaXY+PGRpdj5kb21haW4gMCBsb2Nr
IGV2ZW50X2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA6IGxvY2s6
IMKgIMKgIMKgIMKgIMKgNTEoIMKgIMKgIMKgIMKgIDAuMDAwMDM2MDM1cyksIGJsb2NrOiDCoCDC
oCDCoCDCoCDCoCAwKCDCoCDCoCDCoCDCoCAwLjAwMDAwMDAwMHMpPC9kaXY+PGRpdj5kb21haW4g
MCBsb2NrIHBhZ2VfYWxsb2NfbG9jayDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA6IGxv
Y2s6IMKgIMKgIMKgIMKgIMKgIDEoIMKgIMKgIMKgIMKgIDAuMDAwMDAwMTM1cyksIGJsb2NrOiDC
oCDCoCDCoCDCoCDCoCAwKCDCoCDCoCDCoCDCoCAwLjAwMDAwMDAwMHMpPC9kaXY+PGRpdj50b3Rh
bCBwcm9maWxpbmcgdGltZTogwqAgwqAgwqAgwqAgwqAwLjAwMDAwMDAwMHM8L2Rpdj48ZGl2PnRv
dGFsIGxvY2tlZCB0aW1lOiDCoCDCoCDCoCDCoCDCoCDCoCAxLjYwMzEwNjk2NHM8L2Rpdj48ZGl2
PnRvdGFsIGJsb2NrZWQgdGltZTogwqAgwqAgwqAgwqAgwqAgwqAwLjM1NzA1NzgyNnM8L2Rpdj48
L2Rpdj48L2Rpdj48ZGl2PsKgPC9kaXY+PGRpdj5XaXRob3V0IG15IHBhdGNoOjwvZGl2PjxkaXY+
wqA8L2Rpdj48ZGl2PjxkaXY+PGRpdj48ZGl2PjxkaXY+JCBzdWRvIHhlbmxvY2twcm9mIC1yIDsg
Zm9yIGkgaW4gJChzZXEgMSA1KSA7IGRvIHRpbWUgc3VkbyAvdXNyL2xpYi94ZW4tNC4xNC9iaW4v
eGwgZGVzdHJveSB0ZXN0LXZtLSR7aX0gJmFtcDsgZG9uZSA7IHdhaXQgJChqb2JzIC1wKSA7IHN1
ZG8geGVubG9ja3Byb2Y8L2Rpdj48ZGl2PsKgPC9kaXY+PGRpdj5yZWFswqDCoCDCoDBtMyw5Mzdz
PC9kaXY+PGRpdj51c2VywqDCoCDCoDBtMCwwMTBzPC9kaXY+PGRpdj5zeXPCoMKgIMKgMG0zLDQz
M3M8L2Rpdj48ZGl2PsKgPC9kaXY+PGRpdj5yZWFswqDCoCDCoDBtNCwzMjFzPC9kaXY+PGRpdj51
c2VywqDCoCDCoDBtMCwwMDZzPC9kaXY+PGRpdj5zeXPCoMKgIMKgMG0zLDgxN3M8L2Rpdj48ZGl2
PsKgPC9kaXY+PGRpdj5yZWFswqDCoCDCoDBtNCwzMjZzPC9kaXY+PGRpdj51c2VywqDCoCDCoDBt
MCwwMTZzPC9kaXY+PGRpdj5zeXPCoMKgIMKgMG0zLDgwNXM8L2Rpdj48ZGl2PsKgPC9kaXY+PGRp
dj5yZWFswqDCoCDCoDBtNCwzMjNzPC9kaXY+PGRpdj51c2VywqDCoCDCoDBtMCwwMTNzPC9kaXY+
PGRpdj5zeXPCoMKgIMKgMG0zLDgwM3M8L2Rpdj48ZGl2PsKgPC9kaXY+PGRpdj5yZWFswqDCoCDC
oDBtNCwzMzBzPC9kaXY+PGRpdj51c2VywqDCoCDCoDBtMCwwMTFzPC9kaXY+PGRpdj5zeXPCoMKg
IMKgMG0zLDgxNHM8L2Rpdj48ZGl2PsKgPC9kaXY+PGRpdj5nbG9iYWwgbG9jayBwbGF0Zm9ybV90
aW1lcl9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAgwqAg
wqAgOCggwqAgwqAgwqAgwqAgMC4wMDAwMDYyMTlzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAo
IMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIGZsdXNoX2xv
Y2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA6IGxvY2s6IMKgIMKg
IMKgIDI5NTc2KCDCoCDCoCDCoCDCoCAwLjI0NTQ1ODg4MnMpLCBibG9jazogwqAgwqAgwqAgwqA2
NDE1KCDCoCDCoCDCoCDCoCAwLjE3MjQzMjY0NHMpPC9kaXY+PGRpdj5nbG9iYWwgbG9jayBwY2lf
Y29uZmlnX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDC
oCDCoCDCoCDCoDUyKCDCoCDCoCDCoCDCoCAwLjAwMDA0Mjk3NXMpLCBibG9jazogwqAgwqAgwqAg
wqAgwqAgMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sg
dmVjdG9yX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2Nr
OiDCoCDCoCDCoCDCoCDCoCA0KCDCoCDCoCDCoCDCoCAwLjAwMDAxMjg2NHMpLCBibG9jazogwqAg
wqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFs
IGxvY2sgdnBtdV9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IDogbG9jazogwqAgwqAgwqAgwqAgwqAxMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAzMjNzKSwgYmxv
Y2s6IMKgIMKgIMKgIMKgIMKgIDAoIMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rpdj48ZGl2
Pmdsb2JhbCBsb2NrIGNwdXBvb2xfbG9jayDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoDogbG9jazogwqAgwqAgwqAgwqAgwqAxMCggwqAgwqAgwqAgwqAgMC4wMDAwMDA3NDBz
KSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAoIMKgIMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rp
dj48ZGl2Pmdsb2JhbCBsb2NrIHZtX2xvY2sgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDM1KCDCoCDCoCDCoCDCoCAwLjAw
MDAwMTc5OXMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAgMC4wMDAwMDAw
MDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgdGFza2xldF9sb2NrIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDMwKCDCoCDCoCDCoCDC
oCAwLjAwMDAwMjg3N3MpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAgwqAgMC4w
MDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgc3lzY3RsX2xvY2sgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgOiBsb2NrOiDCoCDCoCDCoCDCoCDCoDgwKCDCoCDC
oCDCoCDCoCAwLjAwMDI5MzI1NXMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAgMCggwqAgwqAgwqAg
wqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+Z2xvYmFsIGxvY2sgY2FsbF9sb2NrIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAgwqAgwqAz
MCggwqAgwqAgwqAgwqAgMC4wMDAxOTExNTNzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDEoIMKg
IMKgIMKgIMKgIDAuMDAwMDAyODk5cyk8L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIGhlYXBfbG9jayDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCA6IGxvY2s6IMKgIMKgIDMz
NDIzNTcoIMKgIMKgIMKgIMKgIDIuMjY4Mjk1NTA1cyksIGJsb2NrOiDCoCDCoCAzMjYzODUzKCDC
oCDCoCDCoCDCoDE4LjU1NjY1MDc5N3MpPC9kaXY+PGRpdj5nbG9iYWwgbG9jayBnbG9iYWxfdmly
cV9oYW5kbGVyc19sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAgwqA5ODI1
KCDCoCDCoCDCoCDCoCAwLjAwMDYxNTYyMnMpLCBibG9jazogwqAgwqAgwqAgwqAgNTA0KCDCoCDC
oCDCoCDCoCAwLjAwMDUzMjk2N3MpPC9kaXY+PGRpdj5nbG9iYWwgbG9jayBkb21saXN0X3VwZGF0
ZV9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAgwqAgwqAg
NSggwqAgwqAgwqAgwqAgMC4wMDAwMDAzMzJzKSwgYmxvY2s6IMKgIMKgIMKgIMKgIMKgIDAoIMKg
IMKgIMKgIMKgIDAuMDAwMDAwMDAwcyk8L2Rpdj48ZGl2Pmdsb2JhbCBsb2NrIGRvbWN0bF9sb2Nr
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDogbG9jazogwqAgwqAgwqAg
wqA5ODUwKCDCoCDCoCDCoCDCoCAwLjAwMDQyNDMxMnMpLCBibG9jazogwqAgwqAgwqAgwqAgwqAg
MCggwqAgwqAgwqAgwqAgMC4wMDAwMDAwMDBzKTwvZGl2PjxkaXY+ZG9tYWluIDAgbG9jayBldmVu
dF9sb2NrIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgOiBsb2NrOiDCoCDC
oCDCoCDCoCDCoDU5KCDCoCDCoCDCoCDCoCAwLjAwMDAzMzU1N3MpLCBibG9jazogwqAgwqAgwqAg
wqAgwqAgMyggwqAgwqAgwqAgwqAgMC4wMDAwMDA3MjNzKTwvZGl2PjxkaXY+dG90YWwgcHJvZmls
aW5nIHRpbWU6IMKgIMKgIMKgIMKgIMKgMC4wMDAwMDAwMDBzPC9kaXY+PGRpdj50b3RhbCBsb2Nr
ZWQgdGltZTogwqAgwqAgwqAgwqAgwqAgwqAgMi41MTUzODA0MTVzPC9kaXY+PGRpdj50b3RhbCBi
bG9ja2VkIHRpbWU6IMKgIMKgIMKgIMKgIMKgIDE4LjcyOTYyMDAzMHM8L2Rpdj48L2Rpdj48L2Rp
dj48L2Rpdj48L2Rpdj48ZGl2PjxkaXY+wqA8L2Rpdj48ZGl2PjxkaXY+RnJvbSBhYm92ZSB0d28g
bGlzdGluZ3MsIGl0IGNhbiBiZSBzZWVuIHRoYXQgdGhlIG1haW4gZGlmZmVyZW5jZSBpcyB0aGUg
bG9jayB3YWl0aW5nIHRpbWUgKCJibG9ja2VkIHRpbWUiKSBvbiB0aGUgImdsb2JhbCBsb2NrIGhl
YXBfbG9jayIgc3BpbmxvY2suPC9kaXY+PC9kaXY+PGRpdj7CoDwvZGl2PjwvZGl2PjxkaXY+MjAu
MDkuMjAyMSwgMTE6MzQsICJEbWl0cnkgSXNhaWtpbiIgJmx0O2lzYWlraW4tZG1pdHJ5QHlhbmRl
eC5ydSZndDs6PC9kaXY+PGJsb2NrcXVvdGU+SSB0aGluayBpdCBjb250ZW5kIG9uIHBhZ2UgaW5z
ZXJ0aW9uIGludG8gZnJlZSBwYWdlcyBsaXN0LiBCdXQgSSBkb24ndCBjb25maXJtIG9yIGRlbmll
ZCB0aGlzIHlldC4gSSdtIG5ldyBpbiBoeXBlcnZpc29ycy4uLjxiciAvPjxiciAvPjExOjE5LCAy
MCDRgdC10L3RgtGP0LHRgNGPIDIwMjEg0LMuLCBKYW4gQmV1bGljaCAmbHQ7PGEgaHJlZj0ibWFp
bHRvOmpiZXVsaWNoQHN1c2UuY29tIiByZWw9Im5vb3BlbmVyIG5vcmVmZXJyZXIiPmpiZXVsaWNo
QHN1c2UuY29tPC9hPiZndDs6PGJsb2NrcXVvdGU+PHA+T24gMTcuMDkuMjAyMSAxODowMSwgSnVs
aWVuIEdyYWxsIHdyb3RlOjwvcD48YmxvY2txdW90ZT7CoEkgd2lsbCBxdW90ZSB3aGF0IEhvbmd5
YW4gd3JvdGUgYmFjayBvbiB0aGUgZmlyc3QgcmVwb3J0OjxiciAvPsKgPGJyIC8+wqAiPGJyIC8+
wqBUaGUgYmVzdCBzb2x1dGlvbiBpcyB0byBtYWtlIHRoZSBoZWFwIHNjYWxhYmxlIGluc3RlYWQg
b2YgYSBnbG9iYWw8YnIgLz7CoGxvY2ssIGJ1dCB0aGF0IGlzIG5vdCBnb2luZyB0byBiZSB0cml2
aWFsLjxiciAvPsKgPGJyIC8+wqBPZiBjb3Vyc2UsIGFub3RoZXIgc29sdXRpb24gaXMgdG8ga2Vl
cCB0aGUgZG9tY3RsIGxvY2sgZHJvcHBlZCBpbjxiciAvPsKgZG9tYWluX2tpbGwoKSBidXQgaGF2
ZSBhbm90aGVyIGRvbWFpbl9raWxsIGxvY2sgc28gdGhhdCBjb21wZXRpbmc8YnIgLz7CoGRvbWFp
bl9raWxsKClzIHdpbGwgdHJ5IHRvIHRha2UgdGhhdCBsb2NrIGFuZCBiYWNrIG9mZiB3aXRoIGh5
cGVyY2FsbDxiciAvPsKgY29udGludWF0aW9uLiBCdXQgdGhpcyBpcyBraW5kIG9mIGhhY2t5ICh3
ZSBpbnRyb2R1Y2UgYSBsb2NrIHRvIHJlZHVjZTxiciAvPsKgc3BpbmxvY2sgY29udGVudGlvbiBl
bHNld2hlcmUpLCB3aGljaCBpcyBwcm9iYWJseSBub3QgYSBzb2x1dGlvbiBidXQgYTxiciAvPsKg
d29ya2Fyb3VuZC48YnIgLz7CoCI8L2Jsb2NrcXVvdGU+PHA+PGJyIC8+SW50ZXJlc3RpbmcuIElz
IGNvbnRlbnRpb24gb24gdGhlIGhlYXAgbG9jayBtZXJlbHkgc3VzcGVjdGVkIG9yPGJyIC8+d2Fz
IHRoYXQgbG9jayBwcm92ZW4gdG8gYmUgdGhlIHByb2JsZW0gb25lPzxiciAvPjxiciAvPkphbjxi
ciAvPsKgPC9wPjwvYmxvY2txdW90ZT48YnIgLz48YnIgLz4tLTxiciAvPtCe0YLQv9GA0LDQstC7
0LXQvdC+INC40Lcg0LzQvtCx0LjQu9GM0L3QvtCz0L4g0L/RgNC40LvQvtC20LXQvdC40Y8g0K/Q
vdC00LXQutGBLtCf0L7Rh9GC0Ys8L2Jsb2NrcXVvdGU+


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:27:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:27:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190991.340835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLCX-0005FB-5c; Mon, 20 Sep 2021 15:27:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190991.340835; Mon, 20 Sep 2021 15:27:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLCX-0005F4-2Y; Mon, 20 Sep 2021 15:27:25 +0000
Received: by outflank-mailman (input) for mailman id 190991;
 Mon, 20 Sep 2021 15:27:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSLCV-0005Ey-El
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:27:23 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 91b039d0-aeb2-458d-9527-ff1d2b050976;
 Mon, 20 Sep 2021 15:27:22 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-Hp1YB1QYMzqPWNkWfdNeiw-1; Mon, 20 Sep 2021 17:27:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5472.eurprd04.prod.outlook.com (2603:10a6:803:d3::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Mon, 20 Sep
 2021 15:27:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 15:27:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0046.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Mon, 20 Sep 2021 15:27:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91b039d0-aeb2-458d-9527-ff1d2b050976
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632151641;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6hru8/lXqXLGXNNtX+i6pB/Pt1GzcKkUt1FUwYH+4Bw=;
	b=eK8sPH+erm4D5NFsybLxjvXEDb+O7NWwarNypbQjR06nr7CjIyxb/NpeMPHvJ/IhHD/7VA
	TzJDdlosmz1+eg+T9/Hbr+eS+nvhZHuUOMS5l4TAq0lG3u9QicfQ7ilz22gsQIBlOUB/Iu
	BKwX0Hkfz6cmN9kSRNdiaoulAmcNTKM=
X-MC-Unique: Hp1YB1QYMzqPWNkWfdNeiw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E25hpfpD9fizYHQHS5010nJggpjTf47Oogy1teD56Bo7GHKIMX0I23VbZ5hEB8APr+3Ft7XDto1HLakymC3EIfLDI2s/QnQ188sg/rJbKqsnZLSAP67WG5xmZcMAZ+K2zLUahWQHNFd/mLd2jv2U4aF1K1H9/FvOEmMVdXmOX8fP30J8axPDllCkjP8dP0ytT0j4+lFcNPvxfpuc/oZSWD8P9Us77eTovZ/K3+5ICGt7SZl35SOUHRWmNb2NqH6q4IJt2aCR0t7OELS0hYbFXqYoOFrLQXV/0ZKZgibp78XAdN/rwbOEdoL2XuyeJGQ9sZPFkHASS6K4ROS+whp89Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9xHMM1ePrv4Vr/9K70ZCf92wFZaFxdeMKR/JT3xeIxM=;
 b=FDTHpZ4IhBcTZhCGh7KrgfjI5sOiUm2/POrrX+A+B6Z6ln8Fe6t36zxazFilwAuwoZtO8EZqxrm1XKODOV3bbNxXeueiJnLcBNyqEb0XTQBd+7xA9sx6P0VqNszOa39IelUxSt9nja6NOYqTiD0shMBtvqB5p5SAAA0UoWHRk2mqSVxSMgg7q6VpXR3QgB4dCnxKZ18gb9KyjZOgmMqxXRtNXH/pR4n6v5ESgF4pIIEOMdzmVz9m6epXmdN+78E6+VkBUE2NGoGhsoegv83wAA/kDCBJXEjZHWE/0MS3+qMLE0oAvuLeqey9eguSjfWSmZSn28yE0KTW0dydFGv2yA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
Date: Mon, 20 Sep 2021 17:27:17 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0046.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bcfc9444-a51d-460a-11fb-08d97c4b21b1
X-MS-TrafficTypeDiagnostic: VI1PR04MB5472:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5472B36CE062AA8F493DBA36B3A09@VI1PR04MB5472.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	y4aA1BeRvu05Z00NcR4VWTKl5yKCoLNlYUdPtf5wdCmDTlztMWlq3YlKnG7vNNyLnhtGhsAr6+kUSwflvXz+UAaBIDE30dKcA4vVk9fFeag0jHi6FlmHBVid3rPgwdyLzZQ0M8Nf/KaKgOZE+vGSmkARhELNhhWTTIetHVYMinbvOIMirm/jsE27cP2Zg9Tax0ZAMTr5G+IWBUfW/S+3J0d5EamNOeN+dP7W2M+gQEdX4IVTurfePLsRmxfjZBdZSw1A+IJQEDmIzFuXCFo0DPjoEAU2jfIL2zAwt16HaQHErJJJQ/jdx2PBf9jVnWlJ4lXAT0OYpqwGr4wRqpTw8u/L9MpAsj3WVN4hxQ9v5AcDFnYpt9DPuZc7GqaiLtqxNEHWicGAwODpkNB+0mHAG+PpnVhRK1XFGNosWHNTFUPySBqPEjoTlF9Y1rpPUGL85u/jUP7FHIhzKXFMrrIYOgMYvn4Ap0ePwcEIxp5sZLuoIP1x+9L4dWtdHTVAzARb0SADUOijZshD9QjwSJWuoFk4Y5jo7ERJUHl2Pe42Cew6p/pvD5wQioh1axAUlaMHq8z+BruJUiESOUEt+nyTGo/9DEykdKCroBZey7WMJNKEhuQb5U51WTXVRg/wc8zbbgEl4cSRhZkoROlcblB4Nj7LsHsJyXcdkSw4UnUVAvgXKEc48WMnmVxe7qOnVslAZ8KpV6YL4ZDPVGIvQ5Ei2rdcB14T2jRIR0j/V/BiFOdxQpi8YbYgNezljWaHunA5
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(376002)(136003)(346002)(39860400002)(396003)(66476007)(66556008)(86362001)(66946007)(6486002)(36756003)(54906003)(31686004)(478600001)(316002)(8676002)(31696002)(53546011)(5660300002)(4326008)(38100700002)(2616005)(956004)(8936002)(83380400001)(6916009)(186003)(16576012)(2906002)(26005)(76704002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?yTb5LrO1hvuZ38deHzL1rklfQ5kfp5pA45zHhbyuvpuuPpO9GlnSFEy3c7Zm?=
 =?us-ascii?Q?fOV0XNPmInonMoKYGGjbal2tfiw2im5hpzNDu+DwD5Tb4oVWwxgpb95wviOv?=
 =?us-ascii?Q?Gr+9ZMzAqqMbowI/jwfGZn5Zh9aFzNEG0A0hgVzEDwP8WfP1XfFi1jo4eY2H?=
 =?us-ascii?Q?BWO1NCgbOkdF5vBhJe6T5V8f1NlSKzAbTnXNWFjbma7Ti2dEJjibJnzvgEUw?=
 =?us-ascii?Q?9RBBPhDxtNpd59EdTGvwlPT3q7/xrCRKdTfg7fGgau7wnn+nu5ZyGleei3Uh?=
 =?us-ascii?Q?o7e4gZCwUzzuLcTAUVHR4p243yxcz2U7r33Is28nvkphALAn/AxewY9W7Bsf?=
 =?us-ascii?Q?Qw+gX2u8ocMsD0c4IgFyDV6Ypv/XnfX3uXFqUvAgV+oSdIJfQovV7R06xv8H?=
 =?us-ascii?Q?mdpoB6lie0q/pLsRvPAByIE4wynUuY/1kuAVWRYjy0THeCmMBLDkmZsNrirD?=
 =?us-ascii?Q?lRGHM8oCeoZCCsFeOnVTlFP5prSGspd0X2RIReOsy4YbpGTWH42SVPlzjJq4?=
 =?us-ascii?Q?/Cv+36G4srlvYwY5+UNq0XvLZ9yXHVJR/htgUZYgbdxk+mWTGWrSIpWvnBZ2?=
 =?us-ascii?Q?4kQ2NUB8eatsnmZJ4Ud4UTxbBSxbZ2zrJAhUGxwt5fMoYL7E28ZXhUxOmcDl?=
 =?us-ascii?Q?ClC3CX8p0v4sBPNYvLRavxBZ64FBkVsNOVeqyeTCDzFQugdtAL6WLbXyO4QF?=
 =?us-ascii?Q?/r2D2AsGfduunMQXohFn7sNDM8Y3+DrmxLOhQxtDWtQJzVGRyZ+8UGCpf7bK?=
 =?us-ascii?Q?az4T/ucSwh+uUoVkIDqyqWjtLzREXhv51VB7lwv373wLVEAHlpy7XcdW02RC?=
 =?us-ascii?Q?2wdokuc3nzJZr70wqRXCufLLJOlB4ki21As18ITqflE0Ts684aAtuj2mtEJV?=
 =?us-ascii?Q?30NHwDLOTj/61ZZEvk3Uuv7k+bHtVFLkoyMAVpVSTvh0RPgKXBX/UtNtGX7A?=
 =?us-ascii?Q?Zxzb+ZPa+gAYoIJOFyd9E44FKxjfKj/QcQ60z/7YtX3ks26+U7UkyHosR9LK?=
 =?us-ascii?Q?lVSDHba0ntvfQwFKMxe7o1UG2tdspfuTvzep5up2YiIgVdr1sOE3dRUWSfVQ?=
 =?us-ascii?Q?3zF8PGKLYiOr2l5x63GtLMk0M5M9vuUsuPiPK6SPsSzcVFtC5KssJqjUjAas?=
 =?us-ascii?Q?pnP9jZRDKyD/HBMVtMWHMNHwSitlqafEmPwM7biLivp3q9iG9G7GkS30geHf?=
 =?us-ascii?Q?/Z2qXTarHJn6QGPHzpq0aR92JKyp2IU56jPlO/LYa9F6AEENTZsJ1TGOnLNH?=
 =?us-ascii?Q?2onEdES3vCU6WlQ8HjH122O5KSa+H71u+Yy4w+F0SeHhE9ChTclaWeoN5AXR?=
 =?us-ascii?Q?Yzrj1pXTt2kDu1PX8sTFAk+E?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bcfc9444-a51d-460a-11fb-08d97c4b21b1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 15:27:18.0509
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3jy61KaO99DK5viqAXb/mnl8VXQvrEfl71cOSauuQ5IfX7xMAnUogbJG5oP7IkcexIrqAbwBRZnLojvgzcj1tA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5472

On 20.09.2021 12:20, Roger Pau Monn=C3=A9 wrote:
> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>> --- a/xen/include/asm-arm/grant_table.h
>> +++ b/xen/include/asm-arm/grant_table.h
>> @@ -71,11 +71,17 @@ int replace_grant_host_mapping(unsigned
>>          XFREE((gt)->arch.status_gfn);                                  =
  \
>>      } while ( 0 )
>> =20
>> -#define gnttab_set_frame_gfn(gt, st, idx, gfn)                         =
  \
>> -    do {                                                               =
  \
>> -        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =3D=
    \
>> -            (gfn);                                                     =
  \
>> -    } while ( 0 )
>> +#define gnttab_set_frame_gfn(gt, st, idx, gfn, mfn)                    =
  \
>> +    ({                                                                 =
  \
>> +        int rc_ =3D 0;                                                 =
    \
>> +        gfn_t ogfn =3D gnttab_get_frame_gfn(gt, st, idx);              =
    \
>=20
> Newline maybe? Not sure whether we decided that macros should also
> follow coding style regarding variable definition followed by newline.

To be honest in macros I'm always on the fence: A long line of all blanks
and just a trailing backslash is about as ugly to me as the missing
separator. I think normally I opt for the way chosen above, but I'm not
going to claim to know I'm consistent with this.

>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||         =
  \
>=20
> I'm slightly confused by this checks, don't you need to check for
> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
> guest_physmap_remove_page?

Why? It's ogfn which gets passed to the function. And it indeed is the
prior GFN's mapping that we want to remove here.

> Or assuming that ogfn is not invalid can be used to imply a removal?

That implication can be (and on x86 is) used for the incoming argument,
i.e. "gfn". I don't think "ogfn" can serve this purpose.

> Also the check for ogfn =3D=3D gfn is only used on Arm, while I would
> assume a similar one would be needed on x86 to guarantee the same
> behavior?

Things are sufficiently different on Arm and x86. As said above, on
x86 I'm indeed using "gfn" being INVALID_GFN as an indication that a
removal is requested. This is simply transforming the behavior from
prior to this change, with the function invocation moved into the
per-arch macros. It may be relevant to note that
gnttab_unpopulate_status_frames() calls gnttab_set_frame_gfn() with
INVALID_GFN only when gnttab_get_frame_gfn() didn't return
INVALID_GFN, i.e. the gnttab_get_frame_gfn() used in
gnttab_set_frame_gfn() won't return this value either (we've not
dropped any locks in between). And the only other caller of
gnttab_set_frame_gfn() guarantees "gfn" not to be INVALID_GFN. A
little fragile (towards hypothetical further callers of the macro/
function), yes, but I couldn't think of anything substantially
better.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:44:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:44:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.190998.340846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLT3-0007bQ-N7; Mon, 20 Sep 2021 15:44:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 190998.340846; Mon, 20 Sep 2021 15:44:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLT3-0007bJ-IW; Mon, 20 Sep 2021 15:44:29 +0000
Received: by outflank-mailman (input) for mailman id 190998;
 Mon, 20 Sep 2021 15:44:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSLT2-0007bD-0O
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:44:28 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSLT1-0001oW-Uw
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:44:27 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSLT1-0002MQ-Tr
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:44:27 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSLSy-0006C3-EV; Mon, 20 Sep 2021 16:44:24 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=h3YRUvAwKt79Fbvv83uHFMpcbe7bIspjTspKr5hz528=; b=x/4KDNmxU1MeUTbLwMrlibeDw5
	oanBbsXAlchMaSlV2fr9mKHDDnhL0Vap2jOXy9yXsXSZmPlCIph0gi01wwDyr8j68D3Wq1i055TEJ
	P6BKr54feEXA7L6q1vXJpbxdpHcjA/pUFi2/ytsAPHXQ0OCAbvcKIbuDFdtEWr5HonDc=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24904.44119.940679.241639@mariner.uk.xensource.com>
Date: Mon, 20 Sep 2021 16:44:23 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org,
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
References: <osstest-164996-mainreport@xen.org>
	<d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> As per
> 
> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
> 
> the system doesn't look to really be out of memory; as per
> 
> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
> 
> there even look to be a number of higher order pages available (albeit
> without digging I can't tell what "(C)" means). Nevertheless order-4
> allocations aren't really nice.

The host history suggests this may possibly be related to a qemu update.

http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html

> What I can't see is why this may have started triggering recently. Was
> the kernel updated in osstest? Is 512Mb of memory perhaps a bit too
> small for a Dom0 on this system (with 96 CPUs)? Going through the log
> I haven't been able to find crucial information like how much memory
> the host has or what the hypervisor command line was.

Logs from last host examination, including a dmesg:

http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.examine/

Re the command line, does Xen not print it ?

The bootloader output seems garbled in the serial log.

Anyway, I think Xen is being booted EFI judging by the grub cfg:

http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--grub.cfg.1

which means that it is probaly reading this:

http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--xen.cfg

which gives this specification of the command line:

  options=placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  

The grub cfg has this:

 multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}

It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".

Ian


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 15:59:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 15:59:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191009.340857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLhD-00014U-3U; Mon, 20 Sep 2021 15:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191009.340857; Mon, 20 Sep 2021 15:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLhC-00014N-Vw; Mon, 20 Sep 2021 15:59:06 +0000
Received: by outflank-mailman (input) for mailman id 191009;
 Mon, 20 Sep 2021 15:59:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSLhA-00014H-V7
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 15:59:04 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ad0c7200-1a2b-11ec-b878-12813bfff9fa;
 Mon, 20 Sep 2021 15:59:03 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-gjhvfbWHMXKIE8fe8wfIYA-1; Mon, 20 Sep 2021 17:59:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 15:58:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 15:58:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0031.eurprd01.prod.exchangelabs.com (2603:10a6:102::44) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17 via Frontend
 Transport; Mon, 20 Sep 2021 15:58:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad0c7200-1a2b-11ec-b878-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632153542;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4phuOynMIFldPo8ylpEV+JxDP7Hpa57DkW7ngd4oF7o=;
	b=cvKPptb4PIFDlef3a/VgAw97pl48MMXsf3o3MdMW1momXiVudLf1QJ1qqfuafmJ8TVOgXY
	nncJgbntPMxIaipJ2rcc7AJ+tVonVcIx9Nh9i3LcwCqgr4NgLrd9HsgRoi/F8GPr5aaG/X
	BjrVLT96jJnhSIZxthEmE8phKO/7MiU=
X-MC-Unique: gjhvfbWHMXKIE8fe8wfIYA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YVEh96IOhhIk/uYchMOrfn82XJ4hE0Z6v8NFeUtt6abs8nxbbVQIS2Pdht3AgABenDSJSm7Vctt00Q6Pwd2J15OwVemv6WjOhIcRgjB0HDHWIdjHb2/bHFJ3xj4iNCQHf2mUS5QWqeXRFDFS1kFQ/gdJL0WZiO3aIZ0eAdpLklYJJ3cqC0QD1SbfyvVP151T0qzMwb9daByHKQ1wO9/16s4xxCNwDjAIz2LCAMO3jREx4++I0VdMgjvrREKSq18FVDOzs5QfQO3fgY11XtbF0YB6mdU5wU1tG19icLPAMCTmvGaHHauQ+QLIzyDT87yFKziDDw+uNjz50Oe3AHZUsg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4phuOynMIFldPo8ylpEV+JxDP7Hpa57DkW7ngd4oF7o=;
 b=m2MxQzqc+n/dozAWlCB0tYkd7DT+PF/o3rMrauIjPjGovQOathXV/2si4oVM0k1Ui8nfganqGHb0fUCbp+a4ESLG4fVlxo3nDbx3BnYNLi4Dw1R53VHApj7Ck++iEX/Ip7aQ8JgQJjleF6yHmS1ZGfHOaGVsJMPLb3vX5+EookPy0KXVacAyXCdsZHAwfeaXNF/ClnWGRTpmMvpturPQuDqTYOI54uh84BwQqAgMMc/6vSfFmgtHlarBRjLm+xp071mk6jhu31E4cmeBpLmmQHr0MCiRhA4bYbvi2PL3mhJBTuaZIUF9LOmv/fe/v/94WmlGPEf0Zyj4LnaS1YSszA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4b10bf2d-bb14-8333-7ece-bba0b1bb7fdc@suse.com>
Date: Mon, 20 Sep 2021 17:58:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24904.44119.940679.241639@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0031.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::44) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 606619a3-dc2d-4ba7-ba46-08d97c4f8f33
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7150A892A7B8D6A366BF639CB3A09@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	X8Ow0DUmELH/osHxbaBApb//2CixQbwDeqsgP1x926uxjAZO5OYPwNYoGEfLTmv6egn0RTCU8L3DROTdxCrSgo/rCsmUQRiPr8LATl0qQ+4N5cu2zotdqqwfVzp+lUhmTtPtdIF2H1wzc46BD4YOkN00zTKWuQTN/UIWUpC3Z+xcAkoUIY6XAFAnmJUdB/u6vFsNHOtCfgFGq+VaCUBaNUmTSMEZ2RNxARP6LknXZdnyG9p6b+OwV57lDK0xnczLMk1wGHJXmS6Os8eJ3MN3nqJQIjr76219dRag2C+b/GlwqEJUq7Z4jCnLqZImQu2rr0eNssZfCRf3vx3g4jVuvp+HPctjIkgnfXhtHSydZ7FMBc0QjT54KrjOTnQiwUN2Fco8czbWk46qMg6wTvuMGJsVeYVlZYMaSFUhoKzJCou68dNUVIaGVetVjpY1Bk+czTnpEN1E5T8Rc831BKG1tSUTKajgCvRjx0sP03Q+kxY9D4iif1nwDVs5PUY11VsW5j+fSGA6LpuHWt99/yt7rLr+5o8fMQvLEiKWTUTR2yYGhO0hXm7UkUqeD1aO+pcRg1pvB/Jbl2Hj171LzXvM8yvM0XN2+DhOcZH8VZOaIepO18SyXQLwO2PRsT+R5G0Mm/9OI6C8boMHgY0tQpE7MGoZqXSHfzeJ4mc1z006paZQHMRgkYZduQ4IBNYTVlKKc4bcsEZW6hHfUauFKMHn4IQXPZJGoLlOeC7DmYTDb7CMCV0LbF6eZNpUEmyI0JkY3a7BwXlIu3cRPaR1NSk+5IyZe1aIzAnlS/Jm55FbZDq/WEzOZMhlPmtmFFWu8H5egekICdTV682OJ/P66RvWOw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(346002)(376002)(366004)(39860400002)(316002)(16576012)(31686004)(5660300002)(2906002)(31696002)(4326008)(86362001)(8936002)(36756003)(8676002)(2616005)(83380400001)(53546011)(478600001)(186003)(66556008)(66476007)(6486002)(26005)(66946007)(38100700002)(956004)(966005)(6916009)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MXVBWmgyY0xhMXk4bnhRbVhKM1pzQ2hDVHFRby9Nb2k1OVFFMG5ZUjhDNU5W?=
 =?utf-8?B?THQ4ZXM4OUJsSUQ4UG4zV1EvMFV6cW5TTGIvZ2VXVGhMNlRVQmNGSlhwUlpo?=
 =?utf-8?B?NHcvano0QkYrbHN3ZitDcE4yWmtkcng4eVFzbWU0bnNwT0RQd29hVUw1WFp1?=
 =?utf-8?B?WHMrU1F6WkF1THBnYU9LSjJWZVFsT0J3ZHFWUEE2bDdQaU9RM0lvTVlnMWM5?=
 =?utf-8?B?amNBempPMEVCLzAzZ2VvT1pwYU92Q1hZWEpQSFZnZXMvVGZ2UjQwREE5MWlt?=
 =?utf-8?B?NGx6TUk2RU1HemdRL25KQWt0SDVJcXFobHRDYkVub3BpOTB6NjY3R1BKWG5p?=
 =?utf-8?B?U0NGc0tDTmJTOXJHRHczSC9DRDlMNjF4VlZySlBBdXIzTGdja0tudTRqYnBE?=
 =?utf-8?B?OWcydVlIam54L2dUN1ZkakRiY0RNRFVkYTZObFBIUWVRNXhPaE9KZ0hLZHpM?=
 =?utf-8?B?dHVrejdlMG42V001ZFJHeXpkcmdaQXhicHRZR2prTlVHMEFpNEJWdVJaYlVw?=
 =?utf-8?B?NWNDRXVmV3psZTJKbG9qV2JmajlCWHNsVG5Vd1JVOER6Y0lpR2YzbTdsUG15?=
 =?utf-8?B?eGJDQ1BSeUNjRm5OQWZjdTU5ZXY5KzhMeWx6L0RGZndVNG9Wd2JZUGxqOGtv?=
 =?utf-8?B?UEFNY3BIdjE5Q0N1N2U3dGtnK1o2cXQ2ZEZXemxSRTVzcHF2VDc0YXVJOVFS?=
 =?utf-8?B?aTZDOEUrWGFaaGhtSGRHZy9nZWQzSHZZRzR2ckZmWFZaSlFXWTRqdlNyY3pU?=
 =?utf-8?B?dmlYSTdqQlEybWdyejNkSjJKb1FiOTJ4OWJuSC9YTVllbDZwNXUyUnBZTEVr?=
 =?utf-8?B?akdmNGgrbzZmRy9JUHRyRDFPaDFocEpiVm82K0dzZXN0OHZleGd1aUZkc1NI?=
 =?utf-8?B?S0JhRGZWSjJLdWJKeHRBVk8vVEFEK1A3QkhHa2dQbjh1MVBqWHVBZHlkT2lB?=
 =?utf-8?B?WTBYOGNzVzA4SWxCMjhmNkFyUFA0ZHlFU1ZXcnh0c1FWTlZ4ZytWQTNJY3Nu?=
 =?utf-8?B?ZnQrQnZnb3llK2tHWmd3Ni8zVkVMa1Q0VEh2TzVSa256bFBQTjNxYlMvWVNz?=
 =?utf-8?B?T2VUWmhDTzh3YnlzbFlicmdpaVJFQ0pnTm1KZ2pkQU00WHEzakpwNUU4dCt3?=
 =?utf-8?B?L3F3WVRycVp5L3ZaQlBST04rbmdTUHlXL2xEalBXTXRIQlEvWmZxUFlUOVVV?=
 =?utf-8?B?d1JOOEJmNWNyMDBiSnVvaHVHRnJ4VTl3bjlRbDE4bGZRQ083cWR5Z2tycGFv?=
 =?utf-8?B?Mk9ENXpPcTh2MkRLS1htdmhwVXlFaXpxeGlqdUhGRExDaTRwVVFoMm9nN25l?=
 =?utf-8?B?ZXhxSkthK2xRUmgxTUM4aXpwR2M4Y3pCS2k5OGhMbE5NWWYwUGxZMDlMU3ph?=
 =?utf-8?B?bnI4eFlyY29WYzFRT3Q1cWRjdEtwa0JDTkJSd21MTDltdzY3eUhLUWM4aXdj?=
 =?utf-8?B?QmtHNnVLeXg3bTJTN3ZZcnMveGZGR3JUd2FRRnVEM3lVbTlCUUVwSXd1TTdN?=
 =?utf-8?B?a3VpZDJkMktnMkRSNXJWVkF1REhXVDZpaEhuaWFCNEtHMTFVYXRRbzIwTW8x?=
 =?utf-8?B?bnpCQlpOVmZvTVA1WGV6Zk4xR05BWmFOdHlqelhXL2l5MG5uV0IrMDBSZUtB?=
 =?utf-8?B?a21NV3VpL2YrQVNoekZFZTlJMXhPZlVXVVJQcXg1Uk9zclNjQlEzVUNaRDR6?=
 =?utf-8?B?YUp1TVhvUUwrSWd1MGxJbXg3dlhqRFE1dURFVFVXSXZsSkx2bnFjVXNTQnpz?=
 =?utf-8?Q?JbwnF7nDMbMzvw3AsZ0kZq4C6GG8KAeBv9hU/gv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 606619a3-dc2d-4ba7-ba46-08d97c4f8f33
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 15:58:59.7947
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cZTOd6Vi26NZarNT0ufMsSJ8jXN+WdmJDFYN+be+v3OtaehGcq2Aneb3ooddxCfn96XeGZR4MctE3MWhBFZxFw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

On 20.09.2021 17:44, Ian Jackson wrote:
> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
>> As per
>>
>> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
>> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
>> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
>> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
>> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
>> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
>> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
>>
>> the system doesn't look to really be out of memory; as per
>>
>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
>>
>> there even look to be a number of higher order pages available (albeit
>> without digging I can't tell what "(C)" means). Nevertheless order-4
>> allocations aren't really nice.
> 
> The host history suggests this may possibly be related to a qemu update.
> 
> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
> 
>> What I can't see is why this may have started triggering recently. Was
>> the kernel updated in osstest? Is 512Mb of memory perhaps a bit too
>> small for a Dom0 on this system (with 96 CPUs)? Going through the log
>> I haven't been able to find crucial information like how much memory
>> the host has or what the hypervisor command line was.
> 
> Logs from last host examination, including a dmesg:
> 
> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.examine/
> 
> Re the command line, does Xen not print it ?
> 
> The bootloader output seems garbled in the serial log.
> 
> Anyway, I think Xen is being booted EFI judging by the grub cfg:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--grub.cfg.1

Also judging by output seen in the log file.

> which means that it is probaly reading this:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--xen.cfg
> 
> which gives this specification of the command line:
> 
>   options=placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  

Funny - about half of this look to be x86-only options.

But yes, this confirms my suspicion on this Dom0 getting limited to
512M of RAM.

> The grub cfg has this:
> 
>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
> 
> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".

Which wouldn't matter - the two options are x86-only again, and hence
would (if anything) trigger log messages about unknown options. Such
log messages would be seen in the ring buffer only though, not on the
serial console (for getting issued too early).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 16:15:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 16:15:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191016.340868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLwt-0003wx-Ge; Mon, 20 Sep 2021 16:15:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191016.340868; Mon, 20 Sep 2021 16:15:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSLwt-0003wq-DL; Mon, 20 Sep 2021 16:15:19 +0000
Received: by outflank-mailman (input) for mailman id 191016;
 Mon, 20 Sep 2021 16:15:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Spoc=OK=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSLws-0003wk-Az
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 16:15:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f1467fd6-1a2d-11ec-b878-12813bfff9fa;
 Mon, 20 Sep 2021 16:15:17 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-V1gM9exIMDKGjqoKRo1uDQ-2; Mon, 20 Sep 2021 18:15:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3534.eurprd04.prod.outlook.com (2603:10a6:803:8::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 20 Sep
 2021 16:15:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 16:15:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR04CA0141.eurprd04.prod.outlook.com (2603:10a6:207::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Mon, 20 Sep 2021 16:15:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1467fd6-1a2d-11ec-b878-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632154516;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=WWTOVr6n18LVQWTFZ5IgrK+XEg6IjbR5yJNW3Rr67PA=;
	b=GSzwG1sRjmUWzNm3sJkrmFF0+JJzvWfraVBjnaGJ3oz/sCL5kht1+TzGYDNdzihzgX0fak
	xFdLcJ1z+fDe6hGr6fI1geNAWieB1qXam97xS+6ea5d8bTr2R5h2opAy6YJgeebONE4j4e
	HdA/hCvsRFCJ9q3T4WZHBAmf9PMaagg=
X-MC-Unique: V1gM9exIMDKGjqoKRo1uDQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HnacVZSZLA6zNFiwykV0C43C/WgHxUJ9XeIxIy+upi30PoWB6o37wkrNvyS+z49GmrXR7zmw+qrL/Etcd9HNoXxobVQxJ9uMa8CO1up38Jhu+g++Yq1D48oLKw0SO+C7IC3EUs1i5cdfopZpPMw3ZlX1VRszixQYNWn1PJhIhDKn2hzalBl7zdvDhqkVbNF9gY5ML6L2y4Ss14z3KFIKdfewWLvzIl/fpkt30GRgOmDr9fnptW8aVJ5fCA5qKXU6t4z6OUhUNZZ2KJu9j5M1Ip8yFWSorUT6xZQLRnFUOvw5ue5mxLsfwVPUARgMZafZmbnG9wwb128ms1jrN7J1vA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WWTOVr6n18LVQWTFZ5IgrK+XEg6IjbR5yJNW3Rr67PA=;
 b=AVG841M2Mzt1tsK8PkNOGAtGxy2RtpQ1wN1/5zZDfCCbEJB2iEAYJtLrr7o6bxweDQuXU9rhV7p89grYMzaawnk+zX0dANA4er6eaBArFJdHyH4qU4SnHXwvb+4G5xrj5chDUXE/rCfaehaMOEcKZ8tt0j7ENv218Fwus84Fz1CMArClCUPnYNuleynZi5C7mQv2d64y78+j+ibu81DglHnJST6kwrlTThSnJ3Y0HM6ysUpkw8Mu70s2ilfwhc6xT7hQtJq6LtPPThAnuA/+Hqal6nECnYBaPEWCEt2B9b5zqO41Xsb0tcJNW2U15EM5btWk9qq/OUOtOdfzvsuocA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2] xen/x86: fix PV trap handling on secondary processors
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
Date: Mon, 20 Sep 2021 18:15:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR04CA0141.eurprd04.prod.outlook.com (2603:10a6:207::25)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 651d9b1d-876b-436e-cece-08d97c51d326
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3534:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB35343DA76F38D518C40A3A68B3A09@VI1PR0402MB3534.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UYw56eljep7LbKo7WpuJUD+P4hXiBK+59UCb8zKDcJYKHWw52dv+2z1JgfpLwlHvgYuCwC4vHn+oaLFqcTcSfh9kdM/v7y5Qvs7p355PhapGSHGMUTn2SQjkMNbk7AfYk/PRYVrmX0Kt2Ggmwi03t4znmiS1wka5Po57skqX92hOTvx0AuCJmdF5O/WKTmf+idy55XupVQN6ZsnPWSWfmxQImwkdD8CTJ8Sfq3NB9uUCTWmzOlujWlnJ3XpxCIpHdWQT7bZh/SVagDJpuQjnPRGk9hmuaRgQQm23RgpWAqIjTAF/c1Lt6v6Rd7XhZW/+dAhMpvY8DOIz5eXDKEtzsAeHc1VHdAuDCAOKlQTl93GHGqpOzjyLPglOlpcdqG2q07Byue+RCjW10CHCEge+o0NCIH4iuHySj5/Yh0eNdKpFCP5P0nOf2zQzGkRCZbgnlCQ9bsHuRahx3MG87k5d1lBL8Yza1t3lm86BISnIJ9oF0iLSra364MJAQQyZIP4wosuGDyQhsmk4/1650Yg28NcgsjYaaE7XZoWt7GCwS8RYfQvRTBPTu1pQsd8gOOqRoEawS61aqCIgsszlrIl1kzLpK7ZoDn1F0ci2L3u6TJlYsijJwlsW6DFaCfWz2IEpGepJz9c0IGbjdNpY0HMpLVQHbkDSlKhwVSROieOISEExVVQwshe1/hxUodDaApKca/JFisNQQpYUY3hS0YEI3R9sG5PH9uXCxllqH0jeHw4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(346002)(39860400002)(136003)(376002)(2906002)(5660300002)(6486002)(31686004)(38100700002)(66946007)(2616005)(478600001)(956004)(316002)(8676002)(83380400001)(186003)(66476007)(8936002)(66556008)(16576012)(31696002)(26005)(4326008)(86362001)(110136005)(36756003)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MVpGczdNTENRM2pBa0F3SS8xL0szNkFyTUZQUG1PSnQvOVZ2VjdhakRkZ2JG?=
 =?utf-8?B?d0lPVFgrRVVvK1VZZFk2Q2x3cnNvZ2xzbmdySGZ2RjBWNVZ6cEJWM1dKaXlz?=
 =?utf-8?B?bWc0QzIrU25aZEpVMnBaV1RjaXRaRFUrYmlVTm00Ynl4Nk1KNkc3UU9Hc1dS?=
 =?utf-8?B?THRCeHZIR2tNQTl1cnZHZDZuZGlWakJyZ3QvK2ExVmY4Uk5Pd3ZpNUVuaTFm?=
 =?utf-8?B?TFZQMzlKV21iSEZXUW4reERScXoxemVabG40RUZWQktlSkdoYnVWVm4rQi9u?=
 =?utf-8?B?MlBYbEZER0lyRXZCejgrQU1BTHlxNDZkUE84bk41bk9selJ2RW1JUmdhSXhs?=
 =?utf-8?B?Q3dZUXdCSHNtMUpkY3hzYzVuL1ZWZDVlVjVBREFoTWxHQ3pKd2IrelI3MUM0?=
 =?utf-8?B?ejJKZDBMdWorY2xYSDJkWTNIYWJNWGVQemdnbkxBNWRRWFNBWFZ2WjBEMjlS?=
 =?utf-8?B?Wi90dVlORjhodjhSRk5WUHhub3JyWlZ3TDZxeU9mWEhxb2xjUG5rU0wwYXVh?=
 =?utf-8?B?eFE2UEpnWUI2c2NHUUJaR3ZqanpXdEVIWUxTUXFHTWRTUnhYMncwU24xbFk5?=
 =?utf-8?B?VzErZ24yelBaSHJ4ckJhMGFleWZVUWJ0WGFNOWI5UURQYTdkSXdEQzVVRkk0?=
 =?utf-8?B?dm1ua0djSGt6blllb3U4YVFWTVMyTDVkWmtONUVXTzRPNldIaGdkRGk2MzV1?=
 =?utf-8?B?cDdmOWsvSm5acmcrVVFob0FSR1VHMURmZG5ma2pub3NXeEFncTBXdjVNSUVS?=
 =?utf-8?B?NGxPaVAwTXk0NXp0WmZ6d2oxMy9ybytmbEVkcnJNOWtodGlIbHJBUXdZOWhj?=
 =?utf-8?B?aEEyb1NtaU1HeU5IM2J6TlVrbW9YeGtKQVpPTWx6b0lkbHpxUzJ0Y3NpcXhL?=
 =?utf-8?B?dXRUWDlpRGpXdUM5aUNLb0Ftb0lNTDBaeExvVEdiSnBXR2h1T3F4L2hmOG0x?=
 =?utf-8?B?SG9wM0szVm5TZ0N4UVhjNFVNUjFrVlAyS0h1SXhCZCtRR2dNOTJpZGVqbWY4?=
 =?utf-8?B?SXp6QklBMXBpMC9laXVTK3Z0b01SeFdnVFV3NnpRclVSNXAzeEVtMEluUXVQ?=
 =?utf-8?B?Y3dnQ09QT2l6ejlxdDU5alNJL2RVMlp5UTRHVXVHUnZLNkdjSmlHYXlCUFZR?=
 =?utf-8?B?dEFvcXNpTUh0TEc0dEFuZ3JNYmJBa3FNeEF6YVJ1Mk84cVNqOVpLZVpETmdi?=
 =?utf-8?B?bWlDUjJnam5SN1NyY0ZrTldVTlVWaDRIZHZqQWhHZGYvbkVVR082bGVrbzF6?=
 =?utf-8?B?YzBoYnVaNnJBMUQ4Y0JNcFIxbm13eDNWMlpBUFh0M0ZHa2NTaXFjanpHQmRS?=
 =?utf-8?B?bEFXOXpWRDMwRko5TkFSVlpRYlZ4SEUzN1N5SEc5Q3Q4dFNOTHNzeXgzUzVB?=
 =?utf-8?B?OGFaUGg1dFdQYU8yR0lZNU1uMVlPa3NnMG9qK3pWbmJ4eUdUVGZlRzh3aTA0?=
 =?utf-8?B?UUdxZVVUaTZTalBGa0VaWTlJdlZEdmQ1VHByY2lMMGJjUllSU2ZpVEQ5NkRt?=
 =?utf-8?B?VWZoU2FGOEFOWHcyYkhrSjBKVGlwNk1RQy91L2NiaDNsOUZLL0M1aHZpT2NI?=
 =?utf-8?B?ZEU2aFlqT2g3b3paZlhlcTVnTW9IaUs5NE9wMWNyL3BldjF0NW5xbm9CVFg2?=
 =?utf-8?B?NmU4cEFrdzh0M0pZMGhVTE5NVHpCOWk1dUFQS25YYmR0TU5wNHc4TG1HZHZy?=
 =?utf-8?B?SmIzNWFNR3duSCtsZHlUMDRLdmRkWm81ZEt5S0xEYzRyVmtiOUxKanNCZWVs?=
 =?utf-8?Q?5zhk5zkTCu1kRVXI7M02FM0R/3Tk/yZFftJmCWV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 651d9b1d-876b-436e-cece-08d97c51d326
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 16:15:12.7766
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BKbQ9X1KldTfnMsBJAsLgyDk+0IeDyntYFJsBaQJQT5MSAzCvakUU4IJbStOXv46LCSiOxzWp3/SwLMzghDjPw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3534

The initial observation was that in PV mode under Xen 32-bit user space
didn't work anymore. Attempts of system calls ended in #GP(0x402). All
of the sudden the vector 0x80 handler was not in place anymore. As it
turns out up to 5.13 redundant initialization did occur: Once from
cpu_initialize_context() (through its VCPUOP_initialise hypercall) and a
2nd time while each CPU was brought fully up. This 2nd initialization is
now gone, uncovering that the 1st one was flawed: Unlike for the
set_trap_table hypercall, a full virtual IDT needs to be specified here;
the "vector" fields of the individual entries are of no interest. With
many (kernel) IDT entries still(?) (i.e. at that point at least) empty,
the syscall vector 0x80 ended up in slot 0x20 of the virtual IDT, thus
becoming the domain's handler for vector 0x20.

Make xen_convert_trap_info() fit for either purpose, leveraging the fact
that on the xen_copy_trap_info() path the table starts out zero-filled.
This includes moving out the writing of the sentinel, which would also
have lead to a buffer overrun in the xen_copy_trap_info() case if all
(kernel) IDT entries were populated. Convert the writing of the sentinel
to clearing of the entire table entry rather than just the address
field.

(I didn't bother trying to identify the commit which uncovered the issue
in 5.14; the commit named below is the one which actually introduced the
bad code.)

Fixes: f87e4cac4f4e ("xen: SMP guest support")
Cc: stable@vger.kernel.org
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
In how far it is correct to use the current CPU's IDT is unclear to me.
Looks at least like another latent trap.
---
v2: Switch back to using xen_convert_trap_info().

--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -755,8 +755,8 @@ static void xen_write_idt_entry(gate_des
 	preempt_enable();
 }
 
-static void xen_convert_trap_info(const struct desc_ptr *desc,
-				  struct trap_info *traps)
+static unsigned xen_convert_trap_info(const struct desc_ptr *desc,
+				      struct trap_info *traps, bool full)
 {
 	unsigned in, out, count;
 
@@ -766,17 +766,18 @@ static void xen_convert_trap_info(const
 	for (in = out = 0; in < count; in++) {
 		gate_desc *entry = (gate_desc *)(desc->address) + in;
 
-		if (cvt_gate_to_trap(in, entry, &traps[out]))
+		if (cvt_gate_to_trap(in, entry, &traps[out]) || full)
 			out++;
 	}
-	traps[out].address = 0;
+
+	return out;
 }
 
 void xen_copy_trap_info(struct trap_info *traps)
 {
 	const struct desc_ptr *desc = this_cpu_ptr(&idt_desc);
 
-	xen_convert_trap_info(desc, traps);
+	xen_convert_trap_info(desc, traps, true);
 }
 
 /* Load a new IDT into Xen.  In principle this can be per-CPU, so we
@@ -786,6 +787,7 @@ static void xen_load_idt(const struct de
 {
 	static DEFINE_SPINLOCK(lock);
 	static struct trap_info traps[257];
+	unsigned out;
 
 	trace_xen_cpu_load_idt(desc);
 
@@ -793,7 +795,8 @@ static void xen_load_idt(const struct de
 
 	memcpy(this_cpu_ptr(&idt_desc), desc, sizeof(idt_desc));
 
-	xen_convert_trap_info(desc, traps);
+	out = xen_convert_trap_info(desc, traps, false);
+	memset(&traps[out], 0, sizeof(traps[0]));
 
 	xen_mc_flush();
 	if (HYPERVISOR_set_trap_table(traps))



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:24:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:24:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191037.340883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN1o-000302-E9; Mon, 20 Sep 2021 17:24:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191037.340883; Mon, 20 Sep 2021 17:24:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN1o-0002zv-AU; Mon, 20 Sep 2021 17:24:28 +0000
Received: by outflank-mailman (input) for mailman id 191037;
 Mon, 20 Sep 2021 17:24:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSN1m-0002zg-PO; Mon, 20 Sep 2021 17:24:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSN1m-00043I-KE; Mon, 20 Sep 2021 17:24:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSN1m-0000cw-93; Mon, 20 Sep 2021 17:24:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSN1m-0007oj-8M; Mon, 20 Sep 2021 17:24:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zfiGToQEfZKQxQ/BwEnEXm7lH+5ZFQ9zdcjswrNMXrE=; b=VilsBrkM5r1pq18JAkQ0eRJZAQ
	lQV1fcKt757Q/nxaRiTouGBFyA/qOAxlNp1xLMUA9obGU+FAg1lhe5Uig8WF6weJjTXOvdQDAoZEN
	dxSnOQ6HeiD6Xx9OD7uPITinP8F0EUTg06Cx0qJ2O1ItP31+HaXtapmrXCkS/MDE2TDQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165127-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165127: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-arm64:<job status>:broken:regression
    xen-unstable:build-arm64-xsm:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:<job status>:broken:regression
    xen-unstable:build-arm64-pvops:host-install(4):broken:regression
    xen-unstable:build-arm64:host-install(4):broken:regression
    xen-unstable:build-arm64-xsm:host-install(4):broken:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-arm64-arm64-xl-seattle:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:build-arm64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 17:24:26 +0000

flight 165127 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165127/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                     <job status>                 broken  in 165119
 build-arm64-xsm                 <job status>                 broken  in 165119
 build-arm64-pvops               <job status>                 broken  in 165119
 build-arm64-pvops          4 host-install(4) broken in 165119 REGR. vs. 164945
 build-arm64                4 host-install(4) broken in 165119 REGR. vs. 164945
 build-arm64-xsm            4 host-install(4) broken in 165119 REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165119

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-seattle   1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl           1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)           blocked in 165119 n/a
 build-arm64-libvirt           1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)           blocked in 165119 n/a
 test-arm64-arm64-examine      1 build-check(1)           blocked in 165119 n/a
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165119 like 164945
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z    9 days
Failing since        164951  2021-09-12 00:14:36 Z    8 days   17 attempts
Testing same since   165103  2021-09-18 01:56:00 Z    2 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64 broken
broken-job build-arm64-xsm broken
broken-job build-arm64-pvops broken

Not pushing.

(No revision log; it would be 639 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191046.340897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN2y-0003d3-WA; Mon, 20 Sep 2021 17:25:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191046.340897; Mon, 20 Sep 2021 17:25:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN2y-0003cw-T1; Mon, 20 Sep 2021 17:25:40 +0000
Received: by outflank-mailman (input) for mailman id 191046;
 Mon, 20 Sep 2021 17:25:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN2x-0003ck-Jm
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:39 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 288e7a11-764c-4528-9450-fb71bdffc74f;
 Mon, 20 Sep 2021 17:25:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 288e7a11-764c-4528-9450-fb71bdffc74f
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158738;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=X2OvFsDnk6zSkjs6xUlTPuHsZnVpWG/gEpynVHga6Vg=;
  b=HxexOTzyLOX5K81+NCiRuygkWPboy0+HBqlTKAmC6a0Mtfzbwfa9hqWp
   eFgUH/EhCXGANHs7Cd3xItLubgAoeH93MPmTk6dfsPLZ9ZHuFobXllSpv
   sCX760ethuFjHr1O/eE/U1jaTv2S9PYWET+Hob/PDYAIUv8tliRNnsuvD
   8=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: ggiTRXzvxA+1hkc/Vsc3ofkjT/EEw0m8NFIkyDG7SWNufK8rejv13u076MXQeJ++j5JMOHu7D5
 Pg+FU2nb9WtqF/x6HP+Xmr2phHmnJ/2EFDtikXI12N9pbzP3ianUbqyEhQwcXu3EFUe84Ok5LU
 89M2JooR+G36QNLKjoUBQjvcSJOV6VVg+rctTlub3ZBw7UywZAUYPU2dfOZ3a3tT1cpfqm/GeC
 7mQ/OpxSkSDN3AsSHnjQh0pVLfaNnr4EBBF+1aqVfpFSkR8VmNLIsbshcwIatMpihH6tVPR96f
 1dQIBaW0u6sHlnFiH9tT0HkX
X-SBRS: 5.1
X-MesageID: 54938583
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KFvEq6LfxIqoUY27FE+ROZIlxSXFcZb7ZxGr2PjKsXjdYENS3mECm
 DAaXWHTaf+PYWb3Ldp0b4+2pE8F6MDSzYRgSgRlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5w7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2vn8Bgm
 eVJ7KaVVAd0B47Qur49SQRxRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gps15seQKaFD
 yYfQSMwahj6bFpSAXswLr4VoPqKuX+4aTIN/Tp5ooJoujOOnWSdyoPFMsfRe9GMbdVYmACfv
 G2u10bTDwweNdef4SGY6X/qjejK9Qv3VosdG7y/8v9Cm0CIyyoYDxh+fUS/iem0jAi5Qd03A
 1wZ/G8ioLY/8GSvT8LhRFuorXicpBkeVtFMVeog52mly7XWyxaUAHAeSTxMY8Bgs9U5LRQxz
 UOAld7tAT1psZWWRGib+7PSqim9URX5NkdbO3VCF1FcpYC+/sdj1XojU+qPDoaTjs3kSAH0m
 gqvsShjvpYPr5QrzbiSqAWvby2XmnTZcuIkzlyJBTv1tVIoPNTNi5+AsgeAvK0ZRGqNZhzY5
 iFVxZLGhAwbJczVzESwrPMx8KZFDhpvGAbVh0JmV7Il/i6kk5JIVdENuGwiTKuF38BtRNMIX
 KMxkVgKjHOwFCHzBUOSX25WI55xpZUM7fy/CpjpgiNmO/CdjjNrGR2CgmbLhQjQfLUEy/lja
 f93j+71VSty5VtbIMqeGL5GjO5DKtEW7mLPX5HrpylLIpLHPyX9dFvxC3PXNrpRxPrd+G39q
 o8DX+PXm0Q3eLCvOUH/rN9MRW3m2FBmXPgaXeQMLbXdSuencUl8Y8LsLUQJId09w/sNybeWo
 RlQmCZwkTLCuJEOEi3SAlgLVV8ldcwXQasTMXN+MFC29WIkZIrzvq4Te4FuJess9fB5zO4yR
 P4AIp3SDvNKQzXB2jIccZii89AyKEX13VqDb3i/fTwyX598XAiVqNXqSRTiqXsVBS2tuMpg/
 7D5jlHHQYAOThhJBdrNbK791Eu4uHUQwbogX0bBLtRJVl/r9Yxmd374gvMtepleIhTf3DqKk
 Q2RBE5A9+XKpoY09vjPhLyF8Nj1Q7cvQBICEjCCv7isNCTc8m6y+qN6Ub6FLWLHSWf52KS+f
 uEJnfvyB+IKwQRRuI1mHrc1ka9nv4nzp6VXxxhPFWnQawj5EatpJ3SL0JUdtqBJwbMF6wK6V
 ljWp4tfMLSNfsjkDEQQNEwuaeHajaMYnTzb7PIUJkTm5XAooOrbABsKZxTc2jZAKLZVMZ8+x
 bZzscEb3AWzlx42P4vUlStT7WmNciQNXqhPWkv222M3Zt7HEm1/XKE=
IronPort-HdrOrdr: A9a23:e3G0saobkfJMYFGT8X2hZ4YaV5opeYIsimQD101hICG8cqSj+f
 xG/c5rrCMc5wxwZJhNo7y90ey7MBbhHP1OkO8s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpM
 BdmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54938583"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH v2 02/12] xen/memory: Remove tail padding from TRC_MEM_* records
Date: Mon, 20 Sep 2021 18:25:19 +0100
Message-ID: <20210920172529.24932-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

Four TRC_MEM_* records supply custom structures with tail padding, leaking
stack rubble into the trace buffer.  Three of the records were fine in 32-bit
builds of Xen, due to the relaxed alignment of 64-bit integers, but
POD_SUPERPAGE_SPLITER was broken right from the outset.

We could pack the datastructures to remove the padding, but xentrace_format
has no way of rendering the upper half of a 16-bit field.  Instead, expand all
16-bit fields to 32-bit.

For POD_SUPERPAGE_SPLINTER, introduce an order field as it is relevant
information, and to match DECREASE_RESERVATION, and so it doesn't require a
__packed attribute to drop tail padding.

Update xenalyze's structures to match, and introduce xentrace_format rendering
which was absent previously.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>

The xentrace_format script isn't remotely Py3 compatible, and was another
script missed by our previous efforts.
---
 tools/xentrace/formats    |  4 ++++
 tools/xentrace/xenalyze.c | 12 ++++++------
 xen/arch/x86/mm/p2m-pod.c | 17 +++++++++--------
 xen/common/memory.c       |  4 ++--
 4 files changed, 21 insertions(+), 16 deletions(-)

diff --git a/tools/xentrace/formats b/tools/xentrace/formats
index deac4d8598b0..0fcc327a4078 100644
--- a/tools/xentrace/formats
+++ b/tools/xentrace/formats
@@ -136,6 +136,10 @@
 0x0010f001  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_map      [ domid = %(1)d ]
 0x0010f002  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_unmap    [ domid = %(1)d ]
 0x0010f003  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_transfer [ domid = %(1)d ]
+0x0010f005  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  decrease_reservation   [ d%(3)d gfn 0x%(2)08x%(1)08x, order %(4)u ]
+0x0010f010  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_populate           [ d%(5)d gfn 0x%(2)08x%(1)08x => mfn 0x%(4)08x%(3)08x, order %(6)u ]
+0x0010f011  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_zero_reclaim       [ d%(5)d gfn 0x%(2)08x%(1)08x => mfn 0x%(4)08x%(3)08x, order %(6)u ]
+0x0010f012  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  pod_superpage_splinter [ d%(3)d gfn 0x%(2)08x%(1)08x, order %(4)u ]
 
 0x00201001  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  hypercall  [ eip = 0x%(1)08x, eax = 0x%(2)08x ]
 0x00201101  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  hypercall  [ rip = 0x%(2)08x%(1)08x, eax = 0x%(3)08x ]
diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 5de167031e01..12dcca964645 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -8121,7 +8121,7 @@ void mem_pod_zero_reclaim_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn, mfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( v && v->hvm.vmexit_valid )
@@ -8171,7 +8171,7 @@ void mem_pod_populate_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn, mfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
@@ -8204,14 +8204,14 @@ void mem_pod_superpage_splinter_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn;
-        int d:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
     {
-        printf(" %s pod_spage_splinter d%d g %llx\n",
+        printf(" %s pod_spage_splinter d%d o%d g %"PRIx64"\n",
                ri->dump_header,
-               r->d, (unsigned long long)r->gfn);
+               r->d, r->order, r->gfn);
     }
 }
 
@@ -8255,7 +8255,7 @@ void mem_decrease_reservation_process(struct pcpu_info *p)
 
     struct {
         uint64_t gfn;
-        int d:16,order:16;
+        uint32_t d, order;
     } *r = (typeof(r))ri->d;
 
     if ( opt.dump_all )
diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c
index 8abc57265c10..90f02ae765f6 100644
--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -819,8 +819,8 @@ p2m_pod_zero_check_superpage(struct p2m_domain *p2m, gfn_t gfn)
     if ( tb_init_done )
     {
         struct {
-            u64 gfn, mfn;
-            int d:16,order:16;
+            uint64_t gfn, mfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
@@ -987,8 +987,8 @@ p2m_pod_zero_check(struct p2m_domain *p2m, const gfn_t *gfns, unsigned int count
             if ( tb_init_done )
             {
                 struct {
-                    u64 gfn, mfn;
-                    int d:16,order:16;
+                    uint64_t gfn, mfn;
+                    uint32_t d, order;
                 } t;
 
                 t.gfn = gfn_x(gfns[i]);
@@ -1217,8 +1217,8 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
     if ( tb_init_done )
     {
         struct {
-            u64 gfn, mfn;
-            int d:16,order:16;
+            uint64_t gfn, mfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
@@ -1260,12 +1260,13 @@ p2m_pod_demand_populate(struct p2m_domain *p2m, gfn_t gfn,
     if ( tb_init_done )
     {
         struct {
-            u64 gfn;
-            int d:16;
+            uint64_t gfn;
+            uint32_t d, order;
         } t;
 
         t.gfn = gfn_x(gfn);
         t.d = d->domain_id;
+        t.order = order;
 
         __trace_var(TRC_MEM_POD_SUPERPAGE_SPLINTER, 0, sizeof(t), &t);
     }
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 63642278fda9..8fd88ccb70bf 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -450,8 +450,8 @@ static void decrease_reservation(struct memop_args *a)
         if ( tb_init_done )
         {
             struct {
-                u64 gfn;
-                int d:16,order:16;
+                uint64_t gfn;
+                uint32_t d, order;
             } t;
 
             t.gfn = gmfn;
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191047.340902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN2z-0003gR-Cv; Mon, 20 Sep 2021 17:25:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191047.340902; Mon, 20 Sep 2021 17:25:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN2z-0003fd-5o; Mon, 20 Sep 2021 17:25:41 +0000
Received: by outflank-mailman (input) for mailman id 191047;
 Mon, 20 Sep 2021 17:25:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN2y-0003cq-Dt
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:40 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c5b8efdf-1a37-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:25:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5b8efdf-1a37-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158739;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=B3S41rd9NvRv/dJsqHom9/Nn++paIgMCBBiyWnP7azs=;
  b=HHnSgUOBv1+n+vifGJTe2DMx8CmQv/WEjcJ2Yug2gjvmjG8hSRHYmLVO
   IIIuwCj854TSRWl/8scXkRXtovkCY8u158cv76ZcZCdiDh4PhRptktp2y
   8XGBBSNp46pNlQ6QJaonhF7jtOCdccEdFghda6g2GQeQjKFCt2ChJqOWe
   c=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: cQzWNDkh5IkZR9+XiUtaYKn0iozqNVqpiYTOvyeehkAvNB1ho6SRQBju2AtSst2FFLFy6pTKQR
 V8q4pPYNb0gH/IgEglB/HkfNo3/0uG/j/W39KA8YQXKpKAefWH90Cg4XntakCo6uzbT/fCCYCl
 4lAeApubmgR3UQXuaWR+G0YC1IUxJBNDBAls5rhHlreXhFbfl+dX4HqrZyDJnrb+jpjSCGMPzm
 p7EaNLvPoodbgbBRM/pnwtCdv3Rn9sALk9T0QXQcqbMijcDC5z0faf6X8IiDEQBr3FivxySrEB
 TrNgEepkUtGaF4WkEQHoFBSA
X-SBRS: 5.1
X-MesageID: 52741153
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kbRaLKlkvZaovmL7isg4Zh/o5gwHIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIcXGrTMvyCNmf3KdEjbYu3pxwOu5fWmoQwHQNr/HgwFiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 M4Ult+ydSgNAu7vpuI0ahNaKhA5JJQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQKyBO
 5NANGEHgBLoJEMREVYUNaAHpNzvqWbyYiVftUnJjP9ii4TU5FMoi+W8WDbPQfSob8hImkeTp
 krd4n/0RBodMbS3yyeB83+qrv/Cm2X8Qo16PLi18PF6nXWYx3dVFQUbU139rPWk4mayVdtQJ
 E0T/isGtrUp+QqgSdyVdxynolaUsxgEQd1SHuYmrgaXxcL88wufQ2QJUDNFQNgnr9MtAywn0
 EeTmNHkDiApt6eaIVqC8p+EoDX0PjIaRUcZfjMNRwYB59jloakwgwjJQ9IlF7S65uAZAhmpn
 WrM9nJnwexO04hbjM1X4GwrnRq3/7uKTDIawjmUfVyjxzpzZreUIJWRvA2zAel7EGqJcrWQl
 CFawJHOt7FfVcvleD+lG7pWTerwjxqRGHiF2wc+QcN5n9i40yP7JehtDCdCyFCF2yruUQTgZ
 lPa8ShV7YVaVJdBRf4qO9/tYyjGIK6JKDgEahw2RoEVCnSSXFXelM2LWaJ39zq2+HXAaYllZ
 f+mnT+EVB7285iLKQZaoM9Gi9cWKt0WnzuPFfgXMTz+ief2iIGppUctbwLVM7FRAFKsiwTJ6
 ddPX/ZmOD0GC7aWX8UjyqZKdQpiBSFiXfje8pULHsbeclsOMDxwUJf5nOJ+E7GJaowIz48kC
 FnmARQGoLc+7FWaQTi3hodLMuK3Ackn8iJlYkTB/z+AghAeXGpm149HH7NfQFXt3LALISdcQ
 6ZXdsOeLO5ITzibqT0RYYOk9N5pdQixhBLINC2gOWBtc5llTg3P29nlYgqwq3VeUnvp7ZMz8
 ++6ywfWYZsfXAA+XszYX+2ikgGqtn8HleMsA0aReotPeF/h+ZRBIjDqiqNlONkFLBjOn2PI1
 wufDRoCi/PKpos5rIvAiaye9t/7GOpiBEtKWWLc6O/uZyXd+2Oix65GUfqJIm+BBD+lpv36a
 LwMnf/mMfABkFJbiKZGEu5mnfAk+t/ih75G1QA4Tn/FWEumV+F7KX6c0MgR6qAUnu1FuRG7U
 16k88VBPenbI9vsFVMcKVZ3bumH0v1IyDDe4e5sfRf/7S5zurGGTV9TL1+HjykEdOl5N4Ysw
 OEAvs8K6lPg1kp2Y4je1i0EpX6RKnEgUrk8ssBICYDmvQMn11VebMGOESTx+pyON41BP0RCz
 uV4X0Yea2CwHnb/Tkc=
IronPort-HdrOrdr: A9a23:V4h1F6456LKLtZGgAAPXwDjXdLJyesId70hD6qkQc3Fom62j5q
 eTdZEgvyMc5wx/ZJhNo7690ey7MBDhHP1OkO0s1NWZPDUO0VHARO1fBMnZsl/d8kXFndK1vp
 0QFpSWZueQMbB75/yKnDVREbwbsaa6GHbDv5ah859vJzsaGp2J921Ce2Cm+tUdfng9OXI+fq
 Dsn/Zvln6bVlk8SN+0PXUBV/irnay3qHq3CSR2fyLO8WO1/EiV1II=
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="52741153"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Meng Xu <mengxu@cis.upenn.edu>
Subject: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Date: Mon, 20 Sep 2021 18:25:18 +0100
Message-ID: <20210920172529.24932-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

In the case that 'extra' isn't a multiple of uint32_t, the calculation rounds
the number of bytes up, causing later logic to read unrelated bytes beyond the
end of the object.

Also, asserting that the object is within TRACE_EXTRA_MAX, but truncating it
in release builds is rude.  Instead, reject any out-of-spec records, leaving
enough of a message to identify the faulty caller.

There is one buggy race record, TRC_RTDS_BUDGET_BURN.  As it must remain
__packed (as cur_budget is misaligned), change bool has_extratime to uint32_t
to compensate.

The new printk() can also be hit by HVMOP_xentrace, although no over-read is
possible.  This has no business being a hypercall in the first place, as it
can't be used outside of custom local debugging, so extend the uint32_t
requirement to HVMOP_xentrace too.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
CC: Meng Xu <mengxu@cis.upenn.edu>

v2:
 * Adjust HVMOP_xentrace to avoid hitting the printk()
 * Fix TRC_RTDS_BUDGET_BURN
 * Adjust wording in __trace_var()
---
 xen/arch/x86/hvm/hvm.c |  5 +++--
 xen/common/sched/rt.c  | 24 +++++++++++++-----------
 xen/common/trace.c     | 21 ++++++++++-----------
 3 files changed, 26 insertions(+), 24 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7b48a1b925bb..09cf6330ad26 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&tr, arg, 1 ) )
             return -EFAULT;
 
-        if ( tr.extra_bytes > sizeof(tr.extra)
-             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
+        if ( tr.extra_bytes % sizeof(uint32_t) ||
+             tr.extra_bytes > sizeof(tr.extra) ||
+             tr.event >> TRC_SUBCLS_SHIFT )
             return -EINVAL;
 
         /* Cycles will be taken at the vmexit and vmenter */
diff --git a/xen/common/sched/rt.c b/xen/common/sched/rt.c
index c24cd2ac3200..c58edca0de84 100644
--- a/xen/common/sched/rt.c
+++ b/xen/common/sched/rt.c
@@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct rt_unit *svc, s_time_t now)
     /* TRACE */
     {
         struct __packed {
-            unsigned unit:16, dom:16;
+            uint16_t unit, dom;
             uint64_t cur_budget;
-            int delta;
-            unsigned priority_level;
-            bool has_extratime;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.cur_budget = (uint64_t) svc->cur_budget;
-        d.delta = delta;
-        d.priority_level = svc->priority_level;
-        d.has_extratime = svc->flags & RTDS_extratime;
+            uint32_t delta;
+            uint32_t priority_level;
+            uint32_t has_extratime;
+        } d = {
+            .unit            = svc->unit->unit_id,
+            .dom             = svc->unit->domain->domain_id,
+            .cur_budget      = svc->cur_budget,
+            .delta           = delta,
+            .priority_level  = svc->priority_level,
+            .has_extratime   = !!(svc->flags & RTDS_extratime),
+        };
+
         trace_var(TRC_RTDS_BUDGET_BURN, 1,
                   sizeof(d),
                   (unsigned char *) &d);
diff --git a/xen/common/trace.c b/xen/common/trace.c
index a2a389a1c7c3..4297ff505fb9 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -686,22 +686,21 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     unsigned long flags;
     u32 bytes_to_tail, bytes_to_wrap;
     unsigned int rec_size, total_size;
-    unsigned int extra_word;
     bool_t started_below_highwater;
 
     if( !tb_init_done )
         return;
 
-    /* Convert byte count into word count, rounding up */
-    extra_word = (extra / sizeof(u32));
-    if ( (extra % sizeof(u32)) != 0 )
-        extra_word++;
-    
-    ASSERT(extra_word <= TRACE_EXTRA_MAX);
-    extra_word = min_t(int, extra_word, TRACE_EXTRA_MAX);
-
-    /* Round size up to nearest word */
-    extra = extra_word * sizeof(u32);
+    /*
+     * extra data needs to be an exact multiple of uint32_t to prevent the
+     * later logic over-reading the object.  Reject out-of-spec records.  Any
+     * failure here is an error in the caller.
+     */
+    if ( extra % sizeof(uint32_t) ||
+         extra / sizeof(uint32_t) > TRACE_EXTRA_MAX )
+        return printk_once(XENLOG_WARNING
+                           "Trace event %#x bad size %u, discarding\n",
+                           event, extra);
 
     if ( (tb_event_mask & event) == 0 )
         return;
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191048.340919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN33-0004DJ-Ix; Mon, 20 Sep 2021 17:25:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191048.340919; Mon, 20 Sep 2021 17:25:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN33-0004D6-F1; Mon, 20 Sep 2021 17:25:45 +0000
Received: by outflank-mailman (input) for mailman id 191048;
 Mon, 20 Sep 2021 17:25:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN32-0003ck-Ip
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:44 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 86b3c526-b7fb-4aca-b34f-30ec78337ff1;
 Mon, 20 Sep 2021 17:25:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86b3c526-b7fb-4aca-b34f-30ec78337ff1
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158738;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=zbv4TRtvbYbmEr6riLgsvffP+odFSfYHHa0VIndkMik=;
  b=czcX31EqGTGZ3IsHSbQkA8hJFnS9o8lBr4M9FKTRwp2kVz/bH9I6/m/N
   0hy7rjE3067uXX1iO5UkLN1FNG/EIEwUuSPpxpPLkZvjayT9ifiJm9ksA
   aRM3u5gRHVm7HVc4xdLZmCgILOmkHpCjyUgFXQk5K1E306svtwZm8MNiH
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 0Ea6ZAnbzQ6qbl6elFK2DEpm081owM6O7Abkn5hvmJwOYOeDTH+CNPorjHTYd6EOQMJ0fZQHE+
 UAeoH5yOlXKzVBZ97qJUwcnKiBd3Tnz/DTmcUKq6Bxb0+YXeY+kHqQBJGv3bL9sewPuNaONJ2z
 8Y9VTsI6lD1rypsBrIlMGMgXfl1qwwNHziUovvPyyNImCCXwVinTOZvqEkuBp5OG08PjTOuRRQ
 CNAnphP92Ln9cLBh1vu5Z5entE58H6vgvs8WdKYeXEEOEBUZ+gK4h8d7MpZeOutO+dknu6dL3K
 8Y0fr8kjSHiR+bWFdTU3Segs
X-SBRS: 5.1
X-MesageID: 53560991
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:WsyjXar0QE3yKivrDjUrU7dzjexeBmKtYhIvgKrLsJaIsI4StFCzt
 garIBmAbPvbZjOhLdhyOYy09RhSuZXRz4drQQJpqyE8RXlBopuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0ILhW1nlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnZy0TSwIMqqWotQ6TjoFSQ1SAvZUxqCSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFKoZtmtt0nfyCvE+TIqYa67L+cVZzHE7gcUm8fP2O
 pFEN2I+PU2ojxtnanAYKp88hPySglLFWDJzsWOSn5Uxyj2GpOB2+Oe0a4eEEjCQfu1ZkE+Xq
 WPB+2XRGQwBOZqUzj/t2lCGi/LLnCj7cJkPD7D+/flv6HWRyXYUEwY+TkagrL+yjUvWc9dSL
 UME5wIlrLN06VSmSN+7UhGlyFa6uRobV8tVAvcNwgiHwarJ4C6UHmEBCDVGbbQOtdA0bSwn0
 EeTmNHkDiApt6eaIVqN+7HRoT6sNCw9KW4ZeTRCXQYD+8Pkoow4klTIVNkLOLWuktT/FDX0w
 jaLhCsznbMeiYgMzarT1VLahzOhoLDZQwhz4R/YNkq66itpaYjjYJangWU39t4ZctzfFAPY+
 iFZxY7Ot4jiEK1higSNROEmPb2uvcy1FyD/3ARDM6Z65QqyrivLkZ9r3N1uGKt4Gp9aIma5O
 x6C4Vk5CIx7ZyTxPPQuC26lI4Fzl/GxS469PhzBRocWOvBMmBm7EDaCjKJ691vkllQl2Yo7M
 I2SGSpHJSdHUfk7pNZaquF07FPK+szc7TiILXwY507+uVZ7WJJyYe1eWGZilshjsMu5TPz9q
 r6zzfdmLimzt8WlOEHqHXM7dwhWfRDX+7iv85E/mhG/zvpORzh6Vq65LUIJUI15haVF/tokD
 VnkARQw9bYLvlWecV/iQik6MNvHBM8jxVpmbX1EFQv5gBALPNfwhJrzgrNqJNHLAsQ4lqUqJ
 xTEEu3daslypsPvoWVFMcal89w8K3xGR2umZkKYXdT2RLY4LyShxzMuVlGHGPAmAnXluM0gj
 aen0w+HE5MPSx47VJTdae61zkP3tn8YwbogU0zNK9hVWUPt7Ik1dHCh0q5pe5kBeUfZ2z+X9
 weKGhNE9+PDlJA4rYvSjqeeoob3T+YnRhhGH3PW5KqdPDXB+jbx2pdJVeuFJGiPVG799Ki4S
 /9SyvXwbK8OkFpQ6tIuGLd316MuodDoouYCnAhjGXzKaXWtC69hfSbajZUe6PUVy+YA6wWsW
 0+J9t1LApmzOZvoQAwLOQ4oTuWfzvVIyDPc2us4fRfh7yht8bvZDUgLZ0uQiDZQJadeOZ8+x
 btzo9Yf7gGyh0Z4MtuCiSwIpW2AImZZDvciv5AeRoTqlhAq2hdJZpmFUn3655SGatNtNEg2I
 2DL2Pqe1uoEnkeSIWAuEXXt3PZGgcVcsR9H+1YOOlCVl4eXnfQwxhBQrWw6Qwk9Is+rCA6v1
 ryH73FIGJg=
IronPort-HdrOrdr: A9a23:7xHaKapl5PDXnqiXUegI6d0aV5rbeYIsimQD101hICG9Evb0qy
 nOpoV96faQslwssR4b9uxoVJPvfZqYz+8X3WBzB8bHYOCFgguVxehZhOOP/9SjIVydygc078
 xdmsNFebjN5DZB7PoT4GODYqodKNvsytHWuQ8JpU0dMz2DaMtbnnZE4h7wKDwReOHfb6BJbq
 Z14KB81kOdUEVSVOuXLF8fUdPOotXa/aiWHCLvV3YcmXGzZSrD0s+ALySl
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53560991"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Juergen Gross <jgross@suse.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Meng Xu <mengxu@cis.upenn.edu>
Subject: [PATCH v2 00/12] xen/trace: Fix leakage of uninitialised stack into the tracebuffer
Date: Mon, 20 Sep 2021 18:25:17 +0100
Message-ID: <20210920172529.24932-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Patches 1-3 fix bugs causing uninitialised stack to leak into the trace
buffers.  Xentrace is a developer/debugging activity restricted to fully
privileged entities, so the leaking of uninitialised stack contents is not a
security concern here.

Patches 4 and 5 are cleanup worthy of backporting, because their knock-on
effects in release builds.

Patches 6 and later are cleanup and probably not for backporting.  They
convert all trace records to using fixed types, and move some PV-specifics to
only be built for PV && TRACEBUFFER, and removing stub files from all
architectures.

I have yet more cleanup in progress making most of the macros disappear, but
this series is getting long enough already, (and taking time I don't really
have).

Andrew Cooper (12):
  xen/trace: Don't over-read trace objects
  xen/memory: Remove tail padding from TRC_MEM_* records
  xen/credit2: Remove tail padding from TRC_CSCHED2_* records
  x86/hvm: Reduce stack usage from HVMTRACE_ND()
  x86/hvm: Remove duplicate calls caused by tracing
  xen/credit2: Clean up trace handling
  xen/rt: Clean up trace handling
  xen/sched: Clean up trace handling
  xen/trace: Minor code cleanup
  x86/pv: Move x86/trace.c to x86/pv/trace.c
  xen/arch: Drop asm-*/trace.h
  x86/trace: Clean up trace handling

 tools/xentrace/formats               |   4 +
 tools/xentrace/xenalyze.c            |  12 +-
 xen/arch/x86/Makefile                |   1 -
 xen/arch/x86/hvm/hpet.c              |  15 +-
 xen/arch/x86/hvm/hvm.c               |   5 +-
 xen/arch/x86/hvm/svm/svm.c           |   8 +-
 xen/arch/x86/hvm/vlapic.c            |  23 ++-
 xen/arch/x86/hvm/vmx/vmx.c           |   9 +-
 xen/arch/x86/hvm/vpic.c              |   9 +-
 xen/arch/x86/irq.c                   |   4 +-
 xen/arch/x86/mm/p2m-pod.c            |  17 +-
 xen/arch/x86/mm/p2m-pt.c             |   6 +-
 xen/arch/x86/mm/shadow/multi.c       |   2 +-
 xen/arch/x86/pv/Makefile             |   1 +
 xen/arch/x86/pv/emul-inv-op.c        |   2 +-
 xen/arch/x86/pv/emul-priv-op.c       |   1 +
 xen/arch/x86/pv/ro-page-fault.c      |   2 +-
 xen/arch/x86/pv/trace.c              | 141 ++++++++++++++
 xen/arch/x86/pv/traps.c              |   2 +-
 xen/arch/x86/trace.c                 | 159 ----------------
 xen/arch/x86/traps.c                 |   3 +-
 xen/common/memory.c                  |   4 +-
 xen/common/sched/core.c              |   4 +-
 xen/common/sched/credit.c            |  38 ++--
 xen/common/sched/credit2.c           | 344 +++++++++++++++++------------------
 xen/common/sched/null.c              |  42 +++--
 xen/common/sched/rt.c                | 121 ++++++------
 xen/common/trace.c                   |  60 +++---
 xen/include/asm-arm/trace.h          |  12 --
 xen/include/asm-x86/hvm/trace.h      |  30 ++-
 xen/include/asm-x86/{ => pv}/trace.h |   8 +-
 xen/include/xen/trace.h              |  13 +-
 32 files changed, 537 insertions(+), 565 deletions(-)
 create mode 100644 xen/arch/x86/pv/trace.c
 delete mode 100644 xen/arch/x86/trace.c
 delete mode 100644 xen/include/asm-arm/trace.h
 rename xen/include/asm-x86/{ => pv}/trace.h (92%)

-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191049.340930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN34-0004VK-Qe; Mon, 20 Sep 2021 17:25:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191049.340930; Mon, 20 Sep 2021 17:25:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN34-0004V7-Mz; Mon, 20 Sep 2021 17:25:46 +0000
Received: by outflank-mailman (input) for mailman id 191049;
 Mon, 20 Sep 2021 17:25:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN33-0003cq-CU
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:45 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c5b8efe2-1a37-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:25:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5b8efe2-1a37-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158739;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=ZTBK/i8V2Aw3dyyoK4GveFgWRMnudCEdHrs7LHz7jJM=;
  b=ffYKYLXmdMK4xnNfiLN16nCAdivYhR1M2GMqA92dDa4xyr1Eftfc+gMz
   j6xK8dGyzxoFh0ePGYQPKV4w4MEE3zzU/zR7MPCVepPIpBm/++bYHYwmF
   AQVGQzX9CZNBoKSqrOGVarnblQKtMxzObKf+L3XLF5iEE5XVsfm4iAm2g
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: mo97rJIQ3QkUuROXYXDlf11JLR7AbsVApbznlGFG92hVQq+caeExaYNCld71797jQ2A0Hiw0nW
 DppHwsPdEVUzc3IQuxza5RGnaSDXsUO6nRDCg0PRHvVSFb4Nq9PsbwYgRc5PGAg3UoecSBZlCa
 LgP8AY4AhT5k3f5Td9/U2HDgQShZrk8pEmiGHDMYOBmpVhEf6Bi9Z1MysOWy5oGTjuTaVyTM4Y
 cpBQFIrj1e7A3c8YMzQhmTpWXkHxo9+SxwSXlCLnv3yCLg7uZYMVoEXp14UQFi60F2RatjJq9W
 6A3mSP5+Z8grvzpafuVHJdtY
X-SBRS: 5.1
X-MesageID: 53560992
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:6NOjcatpx8vlFxNX/Ky1EVYXAefnVE5ZMUV32f8akzHdYApBsoF/q
 tZmKT/TOa6IM2GnfIt0bNvioRhSvJ7XnIRiTgpvqy89EX8T+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524DhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpltZi0YiYlOff2qMMAcEQCPw1BZr945+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DP
 JdHMWs+NnwsZTUQAmY1DLgxs9ySn1b1bSVbg32onKgOtj27IAtZj+G2bYu9lsaxbdpRtlaVo
 CTB5WuRKjMwOcGbyDGF2mmxneKJliT+MKoCGbv9+vN0jVm7wm0IFAZQRVa9ueO+iEO1R5RYM
 UN8x8Y1hfFsrgrxFIC7BkDm5i7f1vIBZzZOO+0+7Cqsxq2O2Ta+G0tZFBJtV4c0itBjEFTGy
 WS1c8PV6S1H6ePOECLGp+bM91teKgBOcjRTPnZsoR8tpoC5+dBt1EqnosNLTfbt5uAZDw0c1
 NxjQMIWvLwVkcdD/KGy51mvb9mE98WRE1JdCuk6WAuYAuJFiGyNPNfABbvzt68owGOlor6p5
 iNspiRmxLpSZaxhbQTUKAn3IF1M2xpiGGaG6WOD4rF7r2j9k5JdVdkIvVmS23uFwu5bIGS0M
 Sc/SCt65YNJPWvCUEOES9vqUKwXIVzbPY29DJj8N4MWCrAoLVPv1Hw+NCa4gjG2+GBxwP5XB
 HtuWZv1ZZrsIf88l2TeqiZ0+eJD+x3SMkuJFMinkEz4geTBDJNXIJ9cWGazgikCxPvsiG3oH
 xx3bqNmEj1TD7/zZDf564kWIQxYJHQ3H8mu+cdWavSCMkxtH2R4U63dxrYoeopEmaVJl7iXo
 iHhCxEAkFev12faLQiqa2x4bO+9V5hIsn9mbzcnOkyl2iZ/bN/3vrsfbZY+YZIu6PdnkaxvV
 /AAdsjZWqZPRz3L9i4zd574qIA+Jh2niRjXZ3iuYSQlfo4mTAvMo4e2cgzq/SgILyy2qcph/
 OHwilKFGcIOHl0wAtzXZfSjy0KKkUIcwO8iDVHVJtRzeVn39NQ4ISLGkfJqcdoHLg/Ox2XG2
 l/OUwsYv+TEv6Q87MLN2fKft46sHuZzQhhaEm3c4erkPCXW5DP+k4pJUeLOdjHBTmLkvq6lY
 LwNnf37NfQGmndMspZ9TOk3nf5vuYO3qu8I1BlgEVXKc0+vW+FpLXSx1MVSsrFAm+1CsgysV
 0PTotRXNN1l4i8+/IL98Ob9Utm+6A==
IronPort-HdrOrdr: A9a23:pGiDi6kiQF6IEeU6vajOw+xg70npDfLc3DAbv31ZSRFFG/Fw9/
 rCoB17726StN91YhsdcL+7V5VoLUmzyXcx2/hzAV7AZniDhILLFuFfBOLZqlWNJ8S9zJ8+6U
 4JScND4bbLfD1HZKjBgTVRE7wbsaW6GKLDv5ag85+6JzsaFZ2J7G1Ce3em+lUdfnghOXKgfq
 DsnPauoVCbCA0qR/X+PFYpdc7ZqebGkZr3CCR2eiLOuGG1/EuVAKeRKWni4isj
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53560992"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH v2 05/12] x86/hvm: Remove duplicate calls caused by tracing
Date: Mon, 20 Sep 2021 18:25:22 +0100
Message-ID: <20210920172529.24932-6-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

1) vpic_ack_pending_irq() calls vlapic_accept_pic_intr() twice, once in the
   TRACE_2D() instantiation and once "for real".  Make the call only once.

2) vlapic_accept_pic_intr() similarly calls __vlapic_accept_pic_intr() twice,
   although this is more complicated to disentangle.

   v cannot be NULL because it has already been dereferenced in the function,
   causing the ternary expression to always call __vlapic_accept_pic_intr().
   However, the return expression of the function takes care to skip the call
   if this vCPU isn't the PIC target.  As __vlapic_accept_pic_intr() is far
   from trivial, make the TRACE_2D() semantics match the return semantics by
   only calling __vlapic_accept_pic_intr() when the vCPU is the PIC target.

3) hpet_set_timer() duplicates calls to hpet_tick_to_ns().  Pull the logic out
   which simplifies both the TRACE and create_periodic_time() calls.

4) lapic_rearm() makes multiple calls to vlapic_lvtt_period().  Pull it out
   into a local variable.

vlapic_accept_pic_intr() is called on every VMEntry, so this is a reduction in
VMEntry complexity across the board.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v2:
 * New
---
 xen/arch/x86/hvm/hpet.c   | 15 +++++++++------
 xen/arch/x86/hvm/vlapic.c | 23 +++++++++++++++--------
 xen/arch/x86/hvm/vpic.c   |  9 +++++----
 3 files changed, 29 insertions(+), 18 deletions(-)

diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index ee756abb824d..475c3f8bf471 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -240,6 +240,7 @@ static void hpet_set_timer(HPETState *h, unsigned int tn,
     uint64_t tn_cmp, cur_tick, diff;
     unsigned int irq;
     unsigned int oneshot;
+    s_time_t diff_ns, period_ns = 0;
 
     ASSERT(tn < HPET_TIMER_NUM);
     ASSERT(rw_is_write_locked(&h->lock));
@@ -311,13 +312,15 @@ static void hpet_set_timer(HPETState *h, unsigned int tn,
      * status register) before another interrupt can be delivered.
      */
     oneshot = !timer_is_periodic(h, tn) || timer_level(h, tn);
+    diff_ns = hpet_tick_to_ns(h, diff);
+    if ( !oneshot )
+        period_ns = hpet_tick_to_ns(h, h->hpet.period[tn]);
+
     TRACE_2_LONG_4D(TRC_HVM_EMUL_HPET_START_TIMER, tn, irq,
-                    TRC_PAR_LONG(hpet_tick_to_ns(h, diff)),
-                    TRC_PAR_LONG(oneshot ? 0LL :
-                                 hpet_tick_to_ns(h, h->hpet.period[tn])));
-    create_periodic_time(vhpet_vcpu(h), &h->pt[tn],
-                         hpet_tick_to_ns(h, diff),
-                         oneshot ? 0 : hpet_tick_to_ns(h, h->hpet.period[tn]),
+                    TRC_PAR_LONG(diff_ns),
+                    TRC_PAR_LONG(period_ns));
+
+    create_periodic_time(vhpet_vcpu(h), &h->pt[tn], diff_ns, period_ns,
                          irq, timer_level(h, tn) ? hpet_timer_fired : NULL,
                          timer_level(h, tn) ? (void *)(unsigned long)tn : NULL,
                          timer_level(h, tn));
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 5e21fb4937d9..b8c84458ffdc 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1267,15 +1267,18 @@ static int __vlapic_accept_pic_intr(struct vcpu *v)
 
 int vlapic_accept_pic_intr(struct vcpu *v)
 {
+    bool target, accept = false;
+
     if ( vlapic_hw_disabled(vcpu_vlapic(v)) || !has_vpic(v->domain) )
         return 0;
 
-    TRACE_2D(TRC_HVM_EMUL_LAPIC_PIC_INTR,
-             (v == v->domain->arch.hvm.i8259_target),
-             v ? __vlapic_accept_pic_intr(v) : -1);
+    target = v == v->domain->arch.hvm.i8259_target;
+    if ( target )
+        accept = __vlapic_accept_pic_intr(v);
+
+    TRACE_2D(TRC_HVM_EMUL_LAPIC_PIC_INTR, target, accept);
 
-    return ((v == v->domain->arch.hvm.i8259_target) &&
-            __vlapic_accept_pic_intr(v));
+    return target && accept;
 }
 
 void vlapic_adjust_i8259_target(struct domain *d)
@@ -1449,6 +1452,7 @@ static void lapic_rearm(struct vlapic *s)
 {
     unsigned long tmict;
     uint64_t period, tdt_msr;
+    bool is_periodic;
 
     s->pt.irq = vlapic_get_reg(s, APIC_LVTT) & APIC_VECTOR_MASK;
 
@@ -1464,12 +1468,15 @@ static void lapic_rearm(struct vlapic *s)
 
     period = ((uint64_t)APIC_BUS_CYCLE_NS *
               (uint32_t)tmict * s->hw.timer_divisor);
+    is_periodic = vlapic_lvtt_period(s);
+
     TRACE_2_LONG_3D(TRC_HVM_EMUL_LAPIC_START_TIMER, TRC_PAR_LONG(period),
-             TRC_PAR_LONG(vlapic_lvtt_period(s) ? period : 0LL), s->pt.irq);
+             TRC_PAR_LONG(is_periodic ? period : 0LL), s->pt.irq);
+
     create_periodic_time(vlapic_vcpu(s), &s->pt, period,
-                         vlapic_lvtt_period(s) ? period : 0,
+                         is_periodic ? period : 0,
                          s->pt.irq,
-                         vlapic_lvtt_period(s) ? vlapic_pt_cb : NULL,
+                         is_periodic ? vlapic_pt_cb : NULL,
                          &s->timer_last_update, false);
     s->timer_last_update = s->pt.last_plt_gtime;
 }
diff --git a/xen/arch/x86/hvm/vpic.c b/xen/arch/x86/hvm/vpic.c
index af988a868c8a..91c2c6983393 100644
--- a/xen/arch/x86/hvm/vpic.c
+++ b/xen/arch/x86/hvm/vpic.c
@@ -512,14 +512,15 @@ void vpic_irq_negative_edge(struct domain *d, int irq)
 
 int vpic_ack_pending_irq(struct vcpu *v)
 {
-    int irq;
+    int irq, accept;
     struct hvm_hw_vpic *vpic = &v->domain->arch.hvm.vpic[0];
 
     ASSERT(has_vpic(v->domain));
 
-    TRACE_2D(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, vlapic_accept_pic_intr(v),
-             vpic->int_output);
-    if ( !vlapic_accept_pic_intr(v) || !vpic->int_output )
+    accept = vlapic_accept_pic_intr(v);
+
+    TRACE_2D(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, accept, vpic->int_output);
+    if ( !accept || !vpic->int_output )
         return -1;
 
     irq = vpic_intack(vpic);
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191050.340941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN39-0004wj-CF; Mon, 20 Sep 2021 17:25:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191050.340941; Mon, 20 Sep 2021 17:25:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN39-0004wW-89; Mon, 20 Sep 2021 17:25:51 +0000
Received: by outflank-mailman (input) for mailman id 191050;
 Mon, 20 Sep 2021 17:25:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN37-0003ck-J8
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:49 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3da059ef-c2a7-4fd9-b9e7-9e82b9762130;
 Mon, 20 Sep 2021 17:25:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3da059ef-c2a7-4fd9-b9e7-9e82b9762130
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158739;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=PjTVtROVmKL1HBneR1Ooz1mqoxckEyucfjsCTUE+XFg=;
  b=Ozj3BsnLMabdSHAFo6BAi6ZQqTe6xpnmvvY0GQqeb4liiUFN/uoDrkiO
   TS4TKkM3Kvitti43Jquc+6kZOVPIbgfB7uAIPjsFVVWdURY3mvlGvniOe
   9rJb1B1gbqfh454qYuB09y1zoQ6jwbLm8SUKbeeL9YF4WzVXAXPtK1w8H
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: fvWHELj4aqtuu/oBdfmJOefbJx7VCae4/OwcTFEZMZczMk6E9cSGfCSi9x35xRvWPVX6O3+JJ6
 8ANefdZqc0Np+xKeqRwAC2bQtPHOCIY9ZO82c+HfQsupQucvdlYWd58NOsM+TfhrjTXk4pzVJc
 8sOR1NyCvC8aUa9Z+G1ff0lM45j14MVZUDw7vklht6BCHYtKxet0NU3bOz47gKWDVHSDIJCvPG
 k0YdaQSxdEybFhYNn9P7X8oQiz0Zo6nlUf52k6oFQItpyKV+gnIZB6FUrTl9/Q8aRrLOq7juz7
 j/fDYSty/TjUDm0JaBHTfxGa
X-SBRS: 5.1
X-MesageID: 54938584
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ajyFH6zuT4yA1B8UMGx6t+exwSrEfRIJ4+MujC+fZmUNrF6WrkVUz
 WpNDDuGaf6OYDD9LtAgYNm+8U4PupfTx9A2Sgo6+SAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRh2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt/Zx2
 MQchZjpcyI0PvD1u/g/CTlCDggraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25wfRqaDN
 5ZxhTxHPTTNQTlrHg0tCswnu8fyuD7vYRpltwfAzUYwyzeKl1EguFT3C/L3UNGXQcRenm6Du
 3nLuW/+B3kyNsGbyDeD2mKhgKnIhyyTcIcfErKo69ZhgUbV23QeDhtQWFemydG1hEexVNNYL
 0084Tc1oO4580nDZtjiXTWorXicpBkeVtFMVeog52ml2qfSpgqUGGUAZjpAc8A98t87QyQw0
 V2ElM+vAiZg2JWKTVqN+7HSqim9UQAONnMLbyIASQoD4vHgrZs1gxaJScxseJNZlfWsR2u2m
 WrT6nFj2fND1qbnyplX43jhgyK996jNFzQQ+wWKZ0G5zFlZfpOMMtnABUfg0RpQEGqIZgDf5
 yFUwJnPtLpm4YKlz3PWEb5UdF29z7PcamSN3wQ3d3U032n1oxaekZZsDCaSzauDGv0NfyPgK
 GTXsBlYjHO4FCr3NfIrC25d5sJD8EQBKTgHfquPBjatSsIoHONiwM2JTRTLt10BaGB2zckC1
 W6zKK5A90r27JiLKxLtHY8gPUIDnHhilQs/u7iil0nPPUWiiI69Fu5ebQrmghERx6KYugTFm
 +uzxOPTkE43bQEKWQGOqdR7BQlTdRATXMmqw+QKJr/rClc3QwkJVq6OqY7NjqQ4xsy5YM+Tp
 SrjMqKZoXKi7UD6xfKiMCE7NOy3AsYh8hrW/0UEZD6V5pTqWq73hI93Snf9VeNPGDVLwaEmQ
 v8bVd+HB/gTGD3L9y5ENcv2rZB4dQTtjgWLZnL3bD86dp9mZgrI5t67IVe/qHhQVnK65Zkkv
 rmt9gLHWp5fFQ5sO9nbNaC0xFSrsHlDxO8rBxnUIsNecVnH+ZRxL3Cjlec+JswBcE2RxjaT2
 wuMLw0foO3B/908/NXT3PjWpIa1CepuWEFdGjCDv7qxMCDb+EulwJNBD7nULWyMCjus9fz7N
 +tPzvz6PPkWp3pwstJxQ+Rx0KYzx9rzvLsGnA5qK2rGMgawAbR6L3jYgcQW7v9RxqVUsBedU
 16U/oUIIq2APc7oHQJDJAchaejfh/gYliOLsKYwKUT+oiR24KCGQQNZOBzV0H5RK758MYUEx
 +Y9uZFJt1zj20RyatvW3DpJ82msL2AbV/R1v54XN4bnlw43xwwQepfbECL3vMmCZtgk3pPG+
 dNIaH4uX4hh+3c=
IronPort-HdrOrdr: A9a23:IYghL6GX72G5OYV6pLqE6seALOsnbusQ8zAXP0AYc3Jom62j5r
 mTdZsgtSMc5Ax8ZJhko6HkBEDiewK7yXcW2/hzAV7KZmCP0wHEEGgh1/qH/9SJIVyYygc378
 ZdmsZFZ+EYdWIK7/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54938584"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Meng Xu <mengxu@cis.upenn.edu>
Subject: [PATCH v2 07/12] xen/rt: Clean up trace handling
Date: Mon, 20 Sep 2021 18:25:24 +0100
Message-ID: <20210920172529.24932-8-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

Most uses of bitfields and __packed are unnecessary.  There is also no need to
cast 'd' to (unsigned char *) before passing it to a function taking void *.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
CC: Meng Xu <mengxu@cis.upenn.edu>

v2:
 * New
---
 xen/common/sched/rt.c | 97 ++++++++++++++++++++++++---------------------------
 1 file changed, 46 insertions(+), 51 deletions(-)

diff --git a/xen/common/sched/rt.c b/xen/common/sched/rt.c
index c58edca0de84..d5da35cac75e 100644
--- a/xen/common/sched/rt.c
+++ b/xen/common/sched/rt.c
@@ -455,21 +455,21 @@ rt_update_deadline(s_time_t now, struct rt_unit *svc)
     svc->cur_budget = svc->budget;
     svc->priority_level = 0;
 
-    /* TRACE */
+    if ( unlikely(tb_init_done) )
     {
-        struct __packed {
-            unsigned unit:16, dom:16;
-            unsigned priority_level;
+        struct {
+            uint16_t unit, dom;
+            uint32_t priority_level;
             uint64_t cur_deadline, cur_budget;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.priority_level = svc->priority_level;
-        d.cur_deadline = (uint64_t) svc->cur_deadline;
-        d.cur_budget = (uint64_t) svc->cur_budget;
-        trace_var(TRC_RTDS_BUDGET_REPLENISH, 1,
-                  sizeof(d),
-                  (unsigned char *) &d);
+        } d = {
+            .dom             = svc->unit->domain->domain_id,
+            .unit            = svc->unit->unit_id,
+            .priority_level  = svc->priority_level,
+            .cur_deadline    = svc->cur_deadline,
+            .cur_budget      = svc->cur_budget,
+        };
+
+        __trace_var(TRC_RTDS_BUDGET_REPLENISH, 1, sizeof(d), &d);
     }
 
     return;
@@ -965,7 +965,7 @@ burn_budget(const struct scheduler *ops, struct rt_unit *svc, s_time_t now)
         }
     }
 
-    /* TRACE */
+    if ( unlikely(tb_init_done) )
     {
         struct __packed {
             uint16_t unit, dom;
@@ -982,9 +982,7 @@ burn_budget(const struct scheduler *ops, struct rt_unit *svc, s_time_t now)
             .has_extratime   = !!(svc->flags & RTDS_extratime),
         };
 
-        trace_var(TRC_RTDS_BUDGET_BURN, 1,
-                  sizeof(d),
-                  (unsigned char *) &d);
+        __trace_var(TRC_RTDS_BUDGET_BURN, 1, sizeof(d), &d);
     }
 }
 
@@ -1019,22 +1017,19 @@ runq_pick(const struct scheduler *ops, const cpumask_t *mask, unsigned int cpu)
         break;
     }
 
-    /* TRACE */
+    if ( unlikely(tb_init_done) && svc )
     {
-        if( svc != NULL )
-        {
-            struct __packed {
-                unsigned unit:16, dom:16;
-                uint64_t cur_deadline, cur_budget;
-            } d;
-            d.dom = svc->unit->domain->domain_id;
-            d.unit = svc->unit->unit_id;
-            d.cur_deadline = (uint64_t) svc->cur_deadline;
-            d.cur_budget = (uint64_t) svc->cur_budget;
-            trace_var(TRC_RTDS_RUNQ_PICK, 1,
-                      sizeof(d),
-                      (unsigned char *) &d);
-        }
+        struct __packed {
+            uint16_t unit, dom;
+            uint64_t cur_deadline, cur_budget;
+        } d = {
+            .unit          = svc->unit->unit_id,
+            .dom           = svc->unit->domain->domain_id,
+            .cur_deadline  = svc->cur_deadline,
+            .cur_budget    = svc->cur_budget,
+        };
+
+        __trace_var(TRC_RTDS_RUNQ_PICK, 1, sizeof(d), &d);
     }
 
     return svc;
@@ -1055,18 +1050,19 @@ rt_schedule(const struct scheduler *ops, struct sched_unit *currunit,
     struct rt_unit *snext = NULL;
     bool migrated = false;
 
-    /* TRACE */
+    if ( unlikely(tb_init_done) )
     {
-        struct __packed {
-            unsigned cpu:16, tasklet:8, tickled:4, idle:4;
-        } d;
-        d.cpu = cur_cpu;
-        d.tasklet = tasklet_work_scheduled;
-        d.tickled = cpumask_test_cpu(sched_cpu, &prv->tickled);
-        d.idle = is_idle_unit(currunit);
-        trace_var(TRC_RTDS_SCHEDULE, 1,
-                  sizeof(d),
-                  (unsigned char *)&d);
+        struct {
+            uint16_t cpu;
+            uint8_t tasklet, tickled:4, idle:4;
+        } d = {
+            .cpu      = cur_cpu,
+            .tasklet  = tasklet_work_scheduled,
+            .tickled  = cpumask_test_cpu(sched_cpu, &prv->tickled),
+            .idle     = is_idle_unit(currunit),
+        };
+
+        __trace_var(TRC_RTDS_SCHEDULE, 1, sizeof(d), &d);
     }
 
     /* clear ticked bit now that we've been scheduled */
@@ -1223,16 +1219,15 @@ runq_tickle(const struct scheduler *ops, const struct rt_unit *new)
     SCHED_STAT_CRANK(tickled_no_cpu);
     return;
  out:
-    /* TRACE */
+    if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, pad:16;
-        } d;
-        d.cpu = cpu_to_tickle;
-        d.pad = 0;
-        trace_var(TRC_RTDS_TICKLE, 1,
-                  sizeof(d),
-                  (unsigned char *)&d);
+            uint16_t cpu, _pad;
+        } d = {
+            .cpu = cpu_to_tickle,
+        };
+
+        __trace_var(TRC_RTDS_TICKLE, 1, sizeof(d), &d);
     }
 
     cpumask_set_cpu(cpu_to_tickle, &prv->tickled);
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191051.340946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN39-00051d-RE; Mon, 20 Sep 2021 17:25:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191051.340946; Mon, 20 Sep 2021 17:25:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN39-0004zs-Ly; Mon, 20 Sep 2021 17:25:51 +0000
Received: by outflank-mailman (input) for mailman id 191051;
 Mon, 20 Sep 2021 17:25:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN38-0003cq-Cg
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:50 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c5e97c3b-1a37-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:25:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5e97c3b-1a37-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158740;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=op3mN7Ssn2+qHvMncFrr9eY7JKq/qdFKvePUbUZAdVE=;
  b=RcVOOoaze2uwR7P9cXIuOh6dtW2+TpF9fVjIuYhA13ckgTV3TFdk/sUY
   LEXbWRfFhiKkFy+RQMRQpBZBVp0g0urKzrvln7NnKC0Mz68L8w3XmVnIV
   AN5yiovRy7qCvhwf0Z7lCZxWwIV/mPGfxhxgVNSTkqJDrAlEvl4iCmi2/
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: eDVWBXpWkoIOoo1AZVM4liBpfZcmugDKNRYzyrxPPYjrE3mqsirArDaUXgCfQrbvgBVttQ+g0L
 cfkXCAwNbQllMw4iSPZrM2oKdbiBYaZhueWW5XOqphEN3V0+YbuWu6f5ZdO5ymNNC17D4L8WIe
 gwJWeHXsDR6DdWnMJJmXekeID0oLhgdmFtHTEH5AcbADiI6w0AdqMLkOTcLs5Jr+kYJo8h56jM
 y6DLYBwuuPnM4SpxTq6x+ufRoWFdcE27uXuhSf1r08j475mY/knmRA1WrDJVWtq+g5L1IohsmV
 ThmAn1z6meN8JTqlTq0hP1wv
X-SBRS: 5.1
X-MesageID: 54938585
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:VbwWZqwB1MSix56KN1h6t+ffwSrEfRIJ4+MujC+fZmUNrF6WrkVTy
 zEbWW/XMq2MNjGheNF2btzkpkJTvZDcztZiTAFprSAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRh2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt/Zx2
 MQchZjpcyI0PvD1u/g/CTlCDggraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25wfRqaDN
 5dxhTxHXBvtXC1CZHMrUtEYpcio2nPRfiVHpwfAzUYwyzeKl1EguFT3C/LXZ9iLSMN9jkue4
 GXc8AzRIDsXKdiewjqt6W+3i6nEmiaTcI4cHbK39/lugXWI23ceThYRUDOTvv2RmkO4HdVFJ
 CQ86ico6KQ/6kGvZt38RAGj5m6JuAYGXNhdGPF87xuCooLV/ASxFmUCViRGatEtqIkxXzNC/
 kSSg9rjCDhrsbuUYXGQ7LGZqXW1Iyd9EIMZTXZaF01fuYCl+dxtyEKUJjp+LEKrpvv1CSCh8
 zTTlg0jh+gqrtIpjYWh4HmS1lpAuaP1oh4JChT/Bzz+t1MiNd/9PeRE+nCAsq0Rd9/xok2p+
 SFewpnAtrhm4YSlyXTVKNjhCo1F8Bps3Nf0uldpA5Bp3DCk4XfLkWt4sWwmeRsB3irpf1bUj
 K7vVeF5v8Q70JiCN/Yfj2eN5yMCl/OIKDgdfqqIBueim7AoHON9wM2LWaJ39zu3+HXAbIllY
 cvLGSpSJS9CVMyLMwZat89CiOR2l0jSNEv4RIzhzgTP7FZtTCfOEt843K+1Rrlhtsus+VyNm
 /4Gbpfi40gPAYXWP3iMmaZOfA9iEJTOLc2vwyChXrXYeVQO9aBII6K5/I7NjKQ/zvwJyb+Xr
 i/iMqKaoXKm7UD6xcyxQigLQNvSsVxX9hrX5AQgYgSl3WYNe4Gq4PtNfpc7Z+B/pudi0eR1X
 78OfMDZWqZDTTHO+jI8a5jhrdM9KET31FzWZyf1MiIie5NARhDS/oO2dAXY6yRTXDG8stEzo
 uP821qDE4YDXQlrEO3fdOmrkwGqpXEYle8rBxnIL9BfdV/C6o9vLyCt3PY7L9tVcUfIxyeA1
 hbQChAd/LGfr4gw+djPpKaFs4b2TLcuQhsERzHWtO/kOzPb82yvxZ57fNyJJT2NBnnp/KiCZ
 PlOy62uOvMwg1sX4ZF3FKxmzPxi6oK39aNa1AltAF7Cc0+vVuF7OnCD0MRC6v9Ny7tetVfkU
 06D4IAHa7CAOcejG18NPgs1KO+E0KhMyDXV6P00JmT85TN2o+XbARkDYUHUhXwPNqZxPaMk3
 fwl6Zwf5AGIgxY3NsqL03JP/GOWI31cC6gqu/n22mMwZtbHHr2aXaHhNw==
IronPort-HdrOrdr: A9a23:fepU9qq6Y6YRGtz1OIBx+ngaV5opeYIsimQD101hICG8cqSj+f
 xG/c5rrCMc5wxwZJhNo7y90ey7MBbhHP1OkO8s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpM
 BdmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54938585"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH v2 09/12] xen/trace: Minor code cleanup
Date: Mon, 20 Sep 2021 18:25:26 +0100
Message-ID: <20210920172529.24932-10-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

 * Delete trailing whitespace
 * Replace an opencoded DIV_ROUND_UP()
 * Drop bogus smp_rmb() - spin_lock_irqsave() has full smp_mb() semantics.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/common/trace.c              | 37 +++++++++++++++----------------------
 xen/include/asm-x86/hvm/trace.h |  2 +-
 2 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/xen/common/trace.c b/xen/common/trace.c
index 4297ff505fb9..41a3c170446b 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -75,10 +75,6 @@ static cpumask_t tb_cpu_mask;
 /* which tracing events are enabled */
 static u32 tb_event_mask = TRC_ALL;
 
-/* Return the number of elements _type necessary to store at least _x bytes of data
- * i.e., sizeof(_type) * ans >= _x. */
-#define fit_to_type(_type, _x) (((_x)+sizeof(_type)-1) / sizeof(_type))
-
 static int cpu_callback(
     struct notifier_block *nfb, unsigned long action, void *hcpu)
 {
@@ -96,8 +92,8 @@ static struct notifier_block cpu_nfb = {
 
 static uint32_t calc_tinfo_first_offset(void)
 {
-    int offset_in_bytes = offsetof(struct t_info, mfn_offset[NR_CPUS]);
-    return fit_to_type(uint32_t, offset_in_bytes);
+    return DIV_ROUND_UP(offsetof(struct t_info, mfn_offset[NR_CPUS]),
+                        sizeof(uint32_t));
 }
 
 /**
@@ -148,7 +144,7 @@ static int calculate_tbuf_size(unsigned int pages, uint16_t t_info_first_offset)
         pages = max_pages;
     }
 
-    /* 
+    /*
      * NB this calculation is correct, because t_info_first_offset is
      * in words, not bytes
      */
@@ -167,7 +163,7 @@ static int calculate_tbuf_size(unsigned int pages, uint16_t t_info_first_offset)
  * trace buffers.  The trace buffers are then available for debugging use, via
  * the %TRACE_xD macros exported in <xen/trace.h>.
  *
- * This function may also be called later when enabling trace buffers 
+ * This function may also be called later when enabling trace buffers
  * via the SET_SIZE hypercall.
  */
 static int alloc_trace_bufs(unsigned int pages)
@@ -401,7 +397,7 @@ int tb_control(struct xen_sysctl_tbuf_op *tbc)
         break;
     case XEN_SYSCTL_TBUFOP_enable:
         /* Enable trace buffers. Check buffers are already allocated. */
-        if ( opt_tbuf_size == 0 ) 
+        if ( opt_tbuf_size == 0 )
             rc = -EINVAL;
         else
             tb_init_done = 1;
@@ -438,7 +434,7 @@ int tb_control(struct xen_sysctl_tbuf_op *tbc)
     return rc;
 }
 
-static inline unsigned int calc_rec_size(bool_t cycles, unsigned int extra) 
+static inline unsigned int calc_rec_size(bool_t cycles, unsigned int extra)
 {
     unsigned int rec_size = 4;
 
@@ -597,7 +593,7 @@ static inline void __insert_record(struct t_buf *buf,
         rec->u.cycles.cycles_lo = (uint32_t)tsc;
         rec->u.cycles.cycles_hi = (uint32_t)(tsc >> 32);
         dst = rec->u.cycles.extra_u32;
-    } 
+    }
 
     if ( extra_data && extra )
         memcpy(dst, extra_data, extra);
@@ -717,9 +713,6 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     if ( !cpumask_test_cpu(smp_processor_id(), &tb_cpu_mask) )
         return;
 
-    /* Read tb_init_done /before/ t_bufs. */
-    smp_rmb();
-
     spin_lock_irqsave(&this_cpu(t_lock), flags);
 
     buf = this_cpu(t_bufs);
@@ -735,14 +728,14 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
 
     /* Calculate the record size */
     rec_size = calc_rec_size(cycles, extra);
- 
+
     /* How many bytes are available in the buffer? */
     bytes_to_tail = calc_bytes_avail(buf);
-    
+
     /* How many bytes until the next wrap-around? */
     bytes_to_wrap = calc_bytes_to_wrap(buf);
-    
-    /* 
+
+    /*
      * Calculate expected total size to commit this record by
      * doing a dry-run.
      */
@@ -756,7 +749,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
         {
             total_size += bytes_to_wrap;
             bytes_to_wrap = data_size;
-        } 
+        }
         total_size += LOST_REC_SIZE;
         bytes_to_wrap -= LOST_REC_SIZE;
 
@@ -768,7 +761,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     if ( rec_size > bytes_to_wrap )
     {
         total_size += bytes_to_wrap;
-    } 
+    }
     total_size += rec_size;
 
     /* Do we have enough space for everything? */
@@ -781,7 +774,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
     }
 
     /*
-     * Now, actually write information 
+     * Now, actually write information
      */
     bytes_to_wrap = calc_bytes_to_wrap(buf);
 
@@ -791,7 +784,7 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
         {
             insert_wrap_record(buf, LOST_REC_SIZE);
             bytes_to_wrap = data_size;
-        } 
+        }
         insert_lost_records(buf);
         bytes_to_wrap -= LOST_REC_SIZE;
 
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
index 145b59f6ac65..696e42eb9499 100644
--- a/xen/include/asm-x86/hvm/trace.h
+++ b/xen/include/asm-x86/hvm/trace.h
@@ -52,7 +52,7 @@
 #define DO_TRC_HVM_CLTS        DEFAULT_HVM_MISC
 #define DO_TRC_HVM_LMSW        DEFAULT_HVM_MISC
 #define DO_TRC_HVM_LMSW64      DEFAULT_HVM_MISC
-#define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC 
+#define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC
 #define DO_TRC_HVM_TRAP             DEFAULT_HVM_MISC
 #define DO_TRC_HVM_TRAP_DEBUG       DEFAULT_HVM_MISC
 #define DO_TRC_HVM_VLAPIC           DEFAULT_HVM_MISC
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191052.340963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3E-0005og-4m; Mon, 20 Sep 2021 17:25:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191052.340963; Mon, 20 Sep 2021 17:25:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3E-0005ni-08; Mon, 20 Sep 2021 17:25:56 +0000
Received: by outflank-mailman (input) for mailman id 191052;
 Mon, 20 Sep 2021 17:25:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN3C-0003ck-JO
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:54 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 860d3042-ecd5-46c3-b376-ddfe26a9aaff;
 Mon, 20 Sep 2021 17:25:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 860d3042-ecd5-46c3-b376-ddfe26a9aaff
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158740;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=z2fpwHvRQ55rgQU2q9tyaPf+/02t7tCkN4LhCvt/vE0=;
  b=IxZRA/1El0MKEngsfLYz7Y25wmVxkSdGuXwnZ1zQ+TVAxKcBiQDAGNQm
   ZzxCd+1f0GYsO1/tqB3UljIlRqocDLQCH1GGgG/qWA2SUJTLh7pvS/CHz
   S7MfMPmOtDFdQ4utec3eczA14ff/V0T6nGD1ytsl9m6d9+b6Y/87VHzZb
   I=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: dG2VgwG7HkkRDOBtHJ9Dj0/LE8KAeXmD9IpTllINph2ZL2Oif/r6rHLNMCruAfRUaV1kV/Utna
 DiuomtbjEwIguLXGxeiog8HomkXP5uMeEU3skfEdmFcpUcXutLuxsG6+mIfgglnYUn0yKT67if
 btp9pm5KtW7cqH74OTfV/0ebtSe9jM9ZI5qrTvrT5boIRQYzsmmuh40Z8vhAnUh89aht/GULlF
 eWhQMppWI6kOuknXzZYo4aSyTXYjzypAN/buCE8LQ/V5evyjC0/xyjQFi409E8OAbSBj64JsFW
 EfCdAgWfQqC2dBbPhyun00dX
X-SBRS: 5.1
X-MesageID: 53560995
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:m6JGvam46DlwBKrjhoQ5Ypfo5gxpIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIXXmiHa6vfMWemcttxYdmx/BgC6MOEndMwHgtorHwwEiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NVJsbeRRQBzBJ3rhcdACTV0LHpsA7ITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQK2DO
 ZJIOGcHgBLoURIIFQxIU5UEgf6KqXWnKWV5kWjLnP9ii4TU5FMoi+W8WDbPQfSIWMFUk0Cwt
 m/AuWPjDXkyFvaS1D6E+XKEnfLUkGXwX4d6PLG2//FlgVqXx0QIFQYbE1C8pJGRlUqWS99Zb
 UsO9UIGvaU0sUCmUNT5dxm5u2Kf+A4RXcJKFO834x3LzbDbiy6YGWosXjNHcMYhtsI9WXotz
 FDht8jyGTVlvbmRSHSc3rSZtzW/PW4SN2BqWMMfZVJbuZ+5+th110+RCIY4eEKosjHrMTH0x
 xuroC9kvYU0kO1WjYmXwgvVvS358/AlUTUJChXrsnONt10iPd7+Nt3wszA3/t4bc93IFQDpU
 Gws3pHEtbFQV8nleDmlHb1VdIxF8cppJ9E1bbRHJJAn6z3l0HqqZ4k4DNpWdRoxb5psldMEZ
 ibuVeJtCH17ZyDCgUxfOdvZ5yEWIU/IT4+NaxwsRoASCqWdjSfelM2UWaJ144wKuBN2+ZzTx
 L/BKZr8ZZrkIf0/kVJauNvxIZd0n3tjlAs/tLjQzgi90Kr2WZJmYe5eazOzghQCxPrc+m39q
 o8HX+PTkkk3eLCuM0H/rN9IRXhXfCdTOHwDg5EOHgJ1ClE9Qz9J5j646e5JRrGJaIwPx7+Uo
 SniChEHoLc97FWeQTi3hrlYQOuHdf5CQbgTZETA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:yDiwv68g32XHn8dvub1uk+DkI+orL9Y04lQ7vn2ZLiYlFvBw9v
 re+cjzuiWE6wr5NEtApTniAse9qBHnhPlICOAqVN/JMTUO0FHYSr2KhrGSoQEIdRefygd179
 YYT0AgY+eaMbEBt6nHCaODYq4dKaK8nJyVuQ==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53560995"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH v2 06/12] xen/credit2: Clean up trace handling
Date: Mon, 20 Sep 2021 18:25:23 +0100
Message-ID: <20210920172529.24932-7-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

There is no need for bitfields anywhere - use more sensible types.  There is
also no need to cast 'd' to (unsigned char *) before passing it to a function
taking void *.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>

v2:
 * Fix whitespace.
---
 xen/common/sched/credit2.c | 315 ++++++++++++++++++++++-----------------------
 1 file changed, 153 insertions(+), 162 deletions(-)

diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index 41312158ec63..9b943e5168f0 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -1080,13 +1080,13 @@ static void update_max_weight(struct csched2_runqueue_data *rqd, int new_weight,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned rqi:16, max_weight:16;
-        } d;
-        d.rqi = rqd->id;
-        d.max_weight = rqd->max_weight;
-        __trace_var(TRC_CSCHED2_RUNQ_MAX_WEIGHT, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t rqi, max_weight;
+        } d = {
+            .rqi        = rqd->id,
+            .max_weight = rqd->max_weight,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_MAX_WEIGHT, 1, sizeof(d), &d);
     }
 }
 
@@ -1114,9 +1114,7 @@ _runq_assign(struct csched2_unit *svc, struct csched2_runqueue_data *rqd)
             .rqi  = rqd->id,
         };
 
-        __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1, sizeof(d), &d);
     }
 
 }
@@ -1342,16 +1340,14 @@ update_runq_load(const struct scheduler *ops,
             uint8_t  rq_id, shift;
             uint32_t _pad;
         } d = {
-            .rq_avgload  = rqd->avgload,
-            .b_avgload   = rqd->b_avgload,
-            .rq_load     = rqd->load,
-            .rq_id       = rqd->id,
-            .shift       = P,
+            .rq_avgload = rqd->avgload,
+            .b_avgload  = rqd->b_avgload,
+            .rq_load    = rqd->load,
+            .rq_id      = rqd->id,
+            .shift      = P,
         };
 
-        __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1, sizeof(d), &d);
     }
 }
 
@@ -1401,16 +1397,16 @@ update_svc_load(const struct scheduler *ops,
     {
         struct {
             uint64_t v_avgload;
-            unsigned unit:16, dom:16;
-            unsigned shift;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.v_avgload = svc->avgload;
-        d.shift = P;
-        __trace_var(TRC_CSCHED2_UPDATE_UNIT_LOAD, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t shift;
+        } d = {
+            .v_avgload = svc->avgload,
+            .unit      = svc->unit->unit_id,
+            .dom       = svc->unit->domain->domain_id,
+            .shift     = P,
+        };
+
+        __trace_var(TRC_CSCHED2_UPDATE_UNIT_LOAD, 1, sizeof(d), &d);
     }
 }
 
@@ -1457,15 +1453,15 @@ static void runq_insert(struct csched2_unit *svc)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned pos;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.pos = pos;
-        __trace_var(TRC_CSCHED2_RUNQ_POS, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t pos;
+        } d = {
+            .unit = svc->unit->unit_id,
+            .dom  = svc->unit->domain->domain_id,
+            .pos  = pos,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_POS, 1, sizeof(d), &d);
     }
 }
 
@@ -1557,16 +1553,16 @@ static s_time_t tickle_score(const struct scheduler *ops, s_time_t now,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            int credit, score;
-        } d;
-        d.dom = cur->unit->domain->domain_id;
-        d.unit = cur->unit->unit_id;
-        d.credit = cur->credit;
-        d.score = score;
-        __trace_var(TRC_CSCHED2_TICKLE_CHECK, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t credit, score;
+        } d = {
+            .unit   = cur->unit->unit_id,
+            .dom    = cur->unit->domain->domain_id,
+            .credit = cur->credit,
+            .score  = score,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE_CHECK, 1, sizeof(d), &d);
     }
 
     return score;
@@ -1604,17 +1600,16 @@ runq_tickle(const struct scheduler *ops, struct csched2_unit *new, s_time_t now)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned processor;
-            int credit;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.processor = cpu;
-        d.credit = new->credit;
-        __trace_var(TRC_CSCHED2_TICKLE_NEW, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t processor, credit;
+        } d = {
+            .dom       = unit->domain->domain_id,
+            .unit      = unit->unit_id,
+            .processor = cpu,
+            .credit    = new->credit,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE_NEW, 1, sizeof(d), &d);
     }
 
     /*
@@ -1753,12 +1748,12 @@ runq_tickle(const struct scheduler *ops, struct csched2_unit *new, s_time_t now)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, pad:16;
-        } d;
-        d.cpu = ipid; d.pad = 0;
-        __trace_var(TRC_CSCHED2_TICKLE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t cpu, _pad;
+        } d = {
+            .cpu = ipid,
+        };
+
+        __trace_var(TRC_CSCHED2_TICKLE, 1, sizeof(d), &d);
     }
 
     tickle_cpu(ipid, rqd);
@@ -1834,16 +1829,16 @@ static void reset_credit(int cpu, s_time_t now, struct csched2_unit *snext)
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-                int credit_start, credit_end;
-            } d;
-            d.dom = svc->unit->domain->domain_id;
-            d.unit = svc->unit->unit_id;
-            d.credit_start = start_credit;
-            d.credit_end = svc->credit;
-            __trace_var(TRC_CSCHED2_CREDIT_RESET, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+                uint32_t credit_start, credit_end;
+            } d = {
+                .unit         = svc->unit->unit_id,
+                .dom          = svc->unit->domain->domain_id,
+                .credit_start = start_credit,
+                .credit_end   = svc->credit,
+            };
+
+            __trace_var(TRC_CSCHED2_CREDIT_RESET, 1, sizeof(d), &d);
         }
     }
 
@@ -1889,18 +1884,17 @@ void burn_credits(struct csched2_runqueue_data *rqd,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            int credit, budget;
-            int delta;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.credit = svc->credit;
-        d.budget = has_cap(svc) ?  svc->budget : INT_MIN;
-        d.delta = delta;
-        __trace_var(TRC_CSCHED2_CREDIT_BURN, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t credit, budget, delta;
+        } d = {
+            .unit   = svc->unit->unit_id,
+            .dom    = svc->unit->domain->domain_id,
+            .credit = svc->credit,
+            .budget = has_cap(svc) ? svc->budget : INT_MIN,
+            .delta  = delta,
+        };
+
+        __trace_var(TRC_CSCHED2_CREDIT_BURN, 1, sizeof(d), &d);
     }
 }
 
@@ -2545,17 +2539,17 @@ csched2_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
     {
         struct {
             uint64_t b_avgload;
-            unsigned unit:16, dom:16;
-            unsigned rq_id:16, new_cpu:16;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.rq_id = min_rqd ? min_rqd->id : -1;
-        d.b_avgload = min_avgload;
-        d.new_cpu = new_cpu;
-        __trace_var(TRC_CSCHED2_PICKED_CPU, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint16_t rq_id, new_cpu;
+        } d = {
+            .b_avgload = min_avgload,
+            .unit      = unit->unit_id,
+            .dom       = unit->domain->domain_id,
+            .rq_id     = min_rqd ? min_rqd->id : -1,
+            .new_cpu   = new_cpu,
+        };
+
+        __trace_var(TRC_CSCHED2_PICKED_CPU, 1, sizeof(d), &d);
     }
 
     return get_sched_res(new_cpu);
@@ -2616,16 +2610,16 @@ static void migrate(const struct scheduler *ops,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned rqi:16, trqi:16;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.rqi = svc->rqd->id;
-        d.trqi = trqd->id;
-        __trace_var(TRC_CSCHED2_MIGRATE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint16_t rqi, trqi;
+        } d = {
+            .unit = unit->unit_id,
+            .dom  = unit->domain->domain_id,
+            .rqi  = svc->rqd->id,
+            .trqi = trqd->id,
+        };
+
+        __trace_var(TRC_CSCHED2_MIGRATE, 1, sizeof(d), &d);
     }
 
     if ( svc->flags & CSFLAG_scheduled )
@@ -2762,15 +2756,15 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned lrq_id:16, orq_id:16;
-                unsigned load_delta;
-            } d;
-            d.lrq_id = st.lrqd->id;
-            d.orq_id = st.orqd->id;
-            d.load_delta = st.load_delta;
-            __trace_var(TRC_CSCHED2_LOAD_CHECK, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t lrq_id, orq_id;
+                uint32_t load_delta;
+            } d = {
+                .lrq_id     = st.lrqd->id,
+                .orq_id     = st.orqd->id,
+                .load_delta = st.load_delta,
+            };
+
+            __trace_var(TRC_CSCHED2_LOAD_CHECK, 1, sizeof(d), &d);
         }
 
         /*
@@ -2808,15 +2802,13 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
             uint16_t lrq_id, orq_id;
             uint32_t _pad;
         } d = {
-            .lb_avgload  = st.lrqd->b_avgload,
-            .ob_avgload  = st.orqd->b_avgload,
-            .lrq_id      = st.lrqd->id,
-            .orq_id      = st.orqd->id,
+            .lb_avgload = st.lrqd->b_avgload,
+            .ob_avgload = st.orqd->b_avgload,
+            .lrq_id     = st.lrqd->id,
+            .orq_id     = st.orqd->id,
         };
 
-        __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+        __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1, sizeof(d), &d);
     }
 
     SCHED_STAT_CRANK(acct_load_balance);
@@ -3402,15 +3394,15 @@ runq_candidate(struct csched2_runqueue_data *rqd,
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-                unsigned runtime;
-            } d;
-            d.dom = scurr->unit->domain->domain_id;
-            d.unit = scurr->unit->unit_id;
-            d.runtime = now - scurr->unit->state_entry_time;
-            __trace_var(TRC_CSCHED2_RATELIMIT, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+                uint32_t runtime;
+            } d = {
+                .unit    = scurr->unit->unit_id,
+                .dom     = scurr->unit->domain->domain_id,
+                .runtime = now - scurr->unit->state_entry_time,
+            };
+
+            __trace_var(TRC_CSCHED2_RATELIMIT, 1, sizeof(d), &d);
         }
         return scurr;
     }
@@ -3463,13 +3455,13 @@ runq_candidate(struct csched2_runqueue_data *rqd,
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-            } d;
-            d.dom = svc->unit->domain->domain_id;
-            d.unit = svc->unit->unit_id;
-            __trace_var(TRC_CSCHED2_RUNQ_CAND_CHECK, 1,
-                        sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+            } d = {
+                .unit = svc->unit->unit_id,
+                .dom  = svc->unit->domain->domain_id,
+            };
+
+            __trace_var(TRC_CSCHED2_RUNQ_CAND_CHECK, 1, sizeof(d), &d);
         }
 
         /*
@@ -3537,17 +3529,16 @@ runq_candidate(struct csched2_runqueue_data *rqd,
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned tickled_cpu;
-            int credit;
-        } d;
-        d.dom = snext->unit->domain->domain_id;
-        d.unit = snext->unit->unit_id;
-        d.credit = snext->credit;
-        d.tickled_cpu = snext->tickled_cpu;
-        __trace_var(TRC_CSCHED2_RUNQ_CANDIDATE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t unit, dom;
+            uint32_t tickled_cpu, credit;
+        } d = {
+            .unit        = snext->unit->unit_id,
+            .dom         = snext->unit->domain->domain_id,
+            .tickled_cpu = snext->tickled_cpu,
+            .credit      = snext->credit,
+        };
+
+        __trace_var(TRC_CSCHED2_RUNQ_CANDIDATE, 1, sizeof(d), &d);
     }
 
     if ( unlikely(snext->tickled_cpu != -1 && snext->tickled_cpu != cpu) )
@@ -3603,18 +3594,18 @@ static void csched2_schedule(
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, rq_id:16;
-            unsigned tasklet:8, idle:8, smt_idle:8, tickled:8;
-        } d;
-        d.cpu = cur_cpu;
-        d.rq_id = c2r(sched_cpu);
-        d.tasklet = tasklet_work_scheduled;
-        d.idle = is_idle_unit(currunit);
-        d.smt_idle = cpumask_test_cpu(sched_cpu, &rqd->smt_idle);
-        d.tickled = tickled;
-        __trace_var(TRC_CSCHED2_SCHEDULE, 1,
-                    sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t cpu, rq_id;
+            uint8_t tasklet, idle, smt_idle, tickled;
+        } d = {
+            .cpu      = cur_cpu,
+            .rq_id    = c2r(sched_cpu),
+            .tasklet  = tasklet_work_scheduled,
+            .idle     = is_idle_unit(currunit),
+            .smt_idle = cpumask_test_cpu(sched_cpu, &rqd->smt_idle),
+            .tickled  = tickled,
+        };
+
+        __trace_var(TRC_CSCHED2_SCHEDULE, 1, sizeof(d), &d);
     }
 
     /* Update credits (and budget, if necessary). */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:25:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:25:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191053.340968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3E-0005xc-TT; Mon, 20 Sep 2021 17:25:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191053.340968; Mon, 20 Sep 2021 17:25:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3E-0005vR-NX; Mon, 20 Sep 2021 17:25:56 +0000
Received: by outflank-mailman (input) for mailman id 191053;
 Mon, 20 Sep 2021 17:25:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN3D-0003cq-D3
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:55 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c7499935-1a37-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:25:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7499935-1a37-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158741;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=kRgS+FI6cDfW+xtWOVJXQyBa4B8IEHbmPbHmbF/E9Yc=;
  b=goB6YPScwF17n/KBWnnCeAhyLpaWQr0oIcaoT0mpQnWQOXGQNz6ifBsS
   dmNKUEgYxK/VtTgE4wc81skxqocZj2oBp83TUX8Oa9olACGILUytbzva5
   mnkiUMQ4zrCz9lfY1HAr80ppczjdJLb+3NSwv4OdRrViIRHl7TXdoYO7J
   Q=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 2aliCzhyOh0fDvZKu3MysIljbnxpZikxNlp9VbAgcwK44GiRR5MeWVrVLox707BH4fnTulyh8/
 JSK7rvRSftpOVrrVXRjqRZEnjgZN4X0GyzaO2aKiy1sAiYBbf2tBRglWrhYmorWnLIU77L407n
 5h7PUo0cLpAhZfFuKe9Psq3dwaNr3wAIujY4FxWMmpIE+5pHIEMTEd+eHlwNasE5E1y2VtWg6y
 otcHW0jFmKCrr1VQLkkTr8SoU5zjF13jCx6B9n893QksObBgkG6fQ7rZ+k/bODaCwlyK3E5kXo
 W7MCoiG+qFuZnboTg41p3mle
X-SBRS: 5.1
X-MesageID: 53560996
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KkDS6aBM/cbTDRVW/8Hkw5YqxClBgxIJ4kV8jS/XYbTApGxw32MBx
 2YWUDuDP/yCNjSnKt5yPIy3oUhQ6MSBzYJrQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00o7wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/izGNtPF9z
 493hbutalpzEoHzw/YjTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFh2pu3pEfQ54yY
 eIDU2R0N0zMfyRBYAknOdV5tbuPuHzwJmgwRFW9+vNsvjm7IBZK+LrwNNvYfPSaSMMTmVyXz
 krk1WnkBhARNPSE1CGItHmrg4fnkS3yV40TE7m53uV3m1DVzWsWYDUJUXOrrP//jVSxM/pPJ
 kpR9icwoKwa8E2wUsK7TxC+uGSDvBMXR5xXCeJSwACSyILE7gCBHG8GQzVdLts8u6ceXic23
 1WEm9foAz1Hs7CPT3+ZsLCOoluP1TM9dDFYI3VeFE1cvoel8NpbYg/zoshLEKqxpfz5HmvK8
 xeXj3Vkq5ks1tw1yPDulbzYuA5AtqQlXyZsuF6ODjn0v1sjDGK2T9f3sgmAtJ6sOK7cFwPY7
 SZew6By+chTVcnlqcCbfAka8FhFDd6+OTvAiBZEG5A7/lxBEFbyINgNvFmSyKpvW/vomAMFg
 meI4mu9B7cJZRNGiJObhKrrUKwXIVDIT4iNaxwtRoMmjmJNmOq7EMZGPhT44owQuBJ0zfFX1
 WmzKJ7xZZrlNUiX5GXvHLpMuVPa7gs/2XnSVfjGI+ePiOHFDEN5vYwtaQPUBshgtfvsiFyMr
 753apvboz0CAbaWSnSGruYuwaUicCFT6Wbe8JcMKIZu42NORQkcNhMm6elwI9A6x/sFyLigE
 7PUchYw9WcTTEbvcW2iAk2Popu2NXqmhX5kbyEqI3iy3H0vPdSm4KsFLsNldrg77u1zi/VzS
 qBdKcmHB/1OTBXB+igcMsah/NAzKkzziFLcJTehbRg+Y4VkG17D9Oj7c1a97yIJFCe265cz+
 uXyygPBTJMfbA1+F8KKOum3xla8sCFFyuJ/VkfFOPdJf0Do/NQ4IiD9lKZvccoNNQ/C1n2R0
 APPWUUUouzEookU9tjVhP/b89f1QrUmRkcDRjvV97e7MyXe71GP+44YXbbaZy3ZWUP15L6mO
 bdfwcbjPaBVh11NqYd9TepmlPps+9v1qrZG5Q14B3GXPU+zA7ZtL3Taj8lCsqpBmu1QtQesA
 x/d/9BbPfOCOd//EU5XLw0gN7zR2fYRkzjUzPI0PESlu3MnoOvZCR1fb0uWlShQDLppK4d0k
 +4utfkf5xG7lhd3YM2NiTpZ9jjUI3ENO0n9Wkr23GM/ZtIX92x/
IronPort-HdrOrdr: A9a23:d53ceKqbkf5TLc2PlQ0NeR0aV5oXeYIsimQD101hICG9Ffbo8/
 xG/c5rsCMc5wxhO03I9ergBEDiex3hHPxOkO4s1N6ZNWGN1VdARLsSi7cKqAeQeREWmNQ86U
 5ISdkGNDWuZmIQsS+B2maF+nwbsaG6GduT6dvj8w==
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53560996"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>
Subject: [PATCH v2 03/12] xen/credit2: Remove tail padding from TRC_CSCHED2_* records
Date: Mon, 20 Sep 2021 18:25:20 +0100
Message-ID: <20210920172529.24932-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

All three of these records have tail padding, leaking stack rubble into the
trace buffer.  Introduce an explicit _pad field and have the compiler zero the
padding automatically.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/common/sched/credit2.c | 47 +++++++++++++++++++++++++++-------------------
 1 file changed, 28 insertions(+), 19 deletions(-)

diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index 6396b38e044c..41312158ec63 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -1106,12 +1106,14 @@ _runq_assign(struct csched2_unit *svc, struct csched2_runqueue_data *rqd)
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned unit:16, dom:16;
-            unsigned rqi:16;
-        } d;
-        d.dom = svc->unit->domain->domain_id;
-        d.unit = svc->unit->unit_id;
-        d.rqi=rqd->id;
+            uint16_t unit, dom;
+            uint16_t rqi, _pad;
+        } d = {
+            .unit = svc->unit->unit_id,
+            .dom  = svc->unit->domain->domain_id,
+            .rqi  = rqd->id,
+        };
+
         __trace_var(TRC_CSCHED2_RUNQ_ASSIGN, 1,
                     sizeof(d),
                     (unsigned char *)&d);
@@ -1336,13 +1338,17 @@ update_runq_load(const struct scheduler *ops,
     {
         struct {
             uint64_t rq_avgload, b_avgload;
-            unsigned rq_load:16, rq_id:8, shift:8;
-        } d;
-        d.rq_id = rqd->id;
-        d.rq_load = rqd->load;
-        d.rq_avgload = rqd->avgload;
-        d.b_avgload = rqd->b_avgload;
-        d.shift = P;
+            uint16_t rq_load;
+            uint8_t  rq_id, shift;
+            uint32_t _pad;
+        } d = {
+            .rq_avgload  = rqd->avgload,
+            .b_avgload   = rqd->b_avgload,
+            .rq_load     = rqd->load,
+            .rq_id       = rqd->id,
+            .shift       = P,
+        };
+
         __trace_var(TRC_CSCHED2_UPDATE_RUNQ_LOAD, 1,
                     sizeof(d),
                     (unsigned char *)&d);
@@ -2799,12 +2805,15 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
     {
         struct {
             uint64_t lb_avgload, ob_avgload;
-            unsigned lrq_id:16, orq_id:16;
-        } d;
-        d.lrq_id = st.lrqd->id;
-        d.lb_avgload = st.lrqd->b_avgload;
-        d.orq_id = st.orqd->id;
-        d.ob_avgload = st.orqd->b_avgload;
+            uint16_t lrq_id, orq_id;
+            uint32_t _pad;
+        } d = {
+            .lb_avgload  = st.lrqd->b_avgload,
+            .ob_avgload  = st.orqd->b_avgload,
+            .lrq_id      = st.lrqd->id,
+            .orq_id      = st.orqd->id,
+        };
+
         __trace_var(TRC_CSCHED2_LOAD_BALANCE, 1,
                     sizeof(d),
                     (unsigned char *)&d);
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:26:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:26:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191055.340985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3J-0006na-G7; Mon, 20 Sep 2021 17:26:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191055.340985; Mon, 20 Sep 2021 17:26:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3J-0006nF-9r; Mon, 20 Sep 2021 17:26:01 +0000
Received: by outflank-mailman (input) for mailman id 191055;
 Mon, 20 Sep 2021 17:25:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN3H-0003ck-Jq
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:25:59 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 498bb826-dae4-419f-ac12-f9a647422097;
 Mon, 20 Sep 2021 17:25:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 498bb826-dae4-419f-ac12-f9a647422097
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158741;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=J0c0EKnncyso937b/nTJN9IuK7CtJgbwEqGNFEbbFdw=;
  b=XMLZZqrr0xpZvQ+ZtRLizLDBiFLs1uqWKFDjte7izRhO7XyvgvfOV8gt
   RXXqAjjkR3BC1Qx+nsqU6qszje/xmB0iBgLsrNw47ntghRlzS0VWNnv43
   +KHofx5kvA9NgLHv6cFBX1xuMy45H2l5SGAbUsB8JXbD6sQctCjBuNXe7
   k=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: QB/yb4BU8YtWfEDbbD4DxptDgnCuvdzoDh+lOslCjvNzO4tKfOWSn+1prs7uogfJ6ihuV5Ue7n
 cVTlTdvqGC0rqqfOyb/D3PSWb/h/vwShtTZwcQwimY0kS90tOPM9mUt80vZzkUOmn7rwb7N+aG
 Bvhpy1R5//SFzhQa2H6/XCsl/Q1Ngpxk9euAg+ReM8896N6Cjkt1p/LPZJZeX3R0bd8MpU7qwa
 np9qMPUp8QBV0usth7HbOPa0fDTNw6zsIOKIjouzDm3NLNupZOp6TMQxAD5h87tAxRmARwKTQi
 5Kex866Cmmg2MLCEGmo1oM+s
X-SBRS: 5.1
X-MesageID: 54938587
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:4OdDPKMtBZYO/N/vrR29kMFynXyQoLVcMsEvi/4bfWQNrUoj3zAGn
 TBNUGGGbvmKazehLt0jbN/ioR8O6JDczdBgSQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Eg7w7RRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYoy+MgPBI5
 txPjpDqUgg5ALTTkdwaTjANRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgWth3ZATTam2i
 8wxWBVQNT7fajNzHEZQE80gwMiQo2j0WmgNwL6SjfVuuDWCpOBr65DyNPLFd9rMQt9a9m66j
 G/b+2XyAjkBKceSjzGC9xqEluLJ2C/2Ro8WPLm57eJxxk2ewHQJDx8bXkf9puO24nNSQPoGd
 RZSoHB36/Fvqgr7FbERQiFUvlaeuwE8YvRZEdY2shqC2pTN/A+hAXUtG2sphMMdiCMmedA7/
 gbXxIq5Xm0+6Of9pWG1rejP/GjrUcQBBSpbP3ZVE1FdizX2iNxr1nryosBf/LlZZzEfMQr5x
 SyD5AM6jq8a5SLg//TmpQ2b695AS56gc+LU2uk1djn+hu+aTNT8D2BN1bQ8xawbRLt1tnHb4
 BA5dzG2tYji961hcRBhp81WRtmUCwutamWA0TaD4bF4r2nFF4GfkXB4v2gleRYB3jcsUj71e
 k7D0T69F7cKZyDCUEODWKroU55C5fG5TbzND6mIBvITMskZXFLWp0lGOB/Pt10BZWBxyMnTz
 7/AKp3yZZvbYIw6pAeLqxA1i+5zmn9mmjONGvgWDX2PiNKjWZJccp9dWHPmUwzzxPjsTNz9/
 4kNOs2U5Q9YVeGiMCDb/ZRKdQIBLGQhBICwoMtSL7bRLg1jEWAnKvnQ3bJ+JNA1w/ULzr/Fr
 iOnR0tV6Fvjnnmbew+EXW9uNeH0VpFloHNlYSF1ZQS022IuaJqE5bsEc8dlZqEu8eFulKYmT
 /QMd8iaLO5ITzDLp2YUYZXn9dQwfxW3nwOeeSGiZWFnLZJnQgXI/P7ifxfuq3ZSXnbm65Nmr
 uT5hA3BQJcFSwBzN+rsaaqinwGroHwQuONuRE+UcNNdT1rhrdpxICvrg/5pf8xVcUffxiGX3
 hq9CAsDobWfuJc89dTEiPzWr4qtFOciTENWE3OCsOSzPCjeuGGi3ZVBQKCDejWEDDH4/6CrZ
 ON0yfDgMaJYwAYW4tQkS7s7n7gj49bPpqNBylU2FXrGWF2nF7d8LyTUxsJIrKBMmudUtAbet
 phjITWG1WFl4P/YLWM=
IronPort-HdrOrdr: A9a23:N/op3a9jBN2bb8cCXCVuk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P
 re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM
 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="54938587"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH v2 04/12] x86/hvm: Reduce stack usage from HVMTRACE_ND()
Date: Mon, 20 Sep 2021 18:25:21 +0100
Message-ID: <20210920172529.24932-5-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

It is pointless to write all 6 entries and only consume the useful subset.
bloat-o-meter shows quite how obscene the overhead is in vmx_vmexit_handler(),
weighing in at 12% of the function arranging unread zeroes on the stack, and
8% for svm_vmexit_handler().

  add/remove: 0/0 grow/shrink: 0/20 up/down: 0/-1929 (-1929)
  Function                                     old     new   delta
  hvm_msr_write_intercept                     1049    1033     -16
  vmx_enable_intr_window                       238     214     -24
  svm_enable_intr_window                       337     313     -24
  hvmemul_write_xcr                            115      91     -24
  hvmemul_write_cr                             350     326     -24
  hvmemul_read_xcr                             115      91     -24
  hvmemul_read_cr                              146     122     -24
  hvm_mov_to_cr                                438     414     -24
  hvm_mov_from_cr                              253     229     -24
  vmx_intr_assist                             1150    1118     -32
  svm_intr_assist                              459     427     -32
  hvm_rdtsc_intercept                          138     106     -32
  hvm_msr_read_intercept                       898     866     -32
  vmx_vmenter_helper                          1142    1094     -48
  vmx_inject_event                             813     765     -48
  svm_vmenter_helper                           238     187     -51
  hvm_hlt                                      197     146     -51
  svm_inject_event                            1678    1614     -64
  svm_vmexit_handler                          5880    5392    -488
  vmx_vmexit_handler                          7281    6438    -843
  Total: Before=3644277, After=3642348, chg -0.05%

Adjust all users of HVMTRACE_ND(), using TRC_PAR_LONG() where appropriate
instead of opencoding it.

The 0 case needs a little help.  All object in C must have a unique address
and _d is passed by pointer.  Explicitly permit the optimiser to drop the
array.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v2:
 * Adjust callers of HVMTRACE_ND() too
 * Drop _d[] for the 0 case.

Normally I wouldn't recommend patches like this for backport, but
{vmx,svm}_vmexit_handler() are fastpaths and this is a *lot* of I-cache lines
dropped...
---
 xen/arch/x86/hvm/svm/svm.c      |  8 +++-----
 xen/arch/x86/hvm/vmx/vmx.c      |  9 ++++-----
 xen/include/asm-x86/hvm/trace.h | 28 ++++++++++------------------
 3 files changed, 17 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index afb1ccb342c2..f0e10dec046e 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1052,7 +1052,7 @@ void svm_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(tb_init_done) )
         HVMTRACE_ND(VMENTRY,
                     nestedhvm_vcpu_in_guestmode(curr) ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 0, 0, 0, 0, 0, 0, 0);
+                    1/*cycles*/);
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
@@ -2565,12 +2565,10 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
 
     if ( hvm_long_mode_active(v) )
         HVMTRACE_ND(VMEXIT64, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 3, exit_reason,
-                    regs->eip, regs->rip >> 32, 0, 0, 0);
+                    1/*cycles*/, exit_reason, TRC_PAR_LONG(regs->rip));
     else
         HVMTRACE_ND(VMEXIT, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, 2, exit_reason,
-                    regs->eip, 0, 0, 0, 0);
+                    1/*cycles*/, exit_reason, regs->eip);
 
     if ( vcpu_guestmode )
     {
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index b0a42d05f86a..d403e2d8060a 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3864,11 +3864,10 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
     __vmread(VM_EXIT_REASON, &exit_reason);
 
     if ( hvm_long_mode_active(v) )
-        HVMTRACE_ND(VMEXIT64, 0, 1/*cycles*/, 3, exit_reason,
-                    regs->eip, regs->rip >> 32, 0, 0, 0);
+        HVMTRACE_ND(VMEXIT64, 0, 1/*cycles*/, exit_reason,
+                    TRC_PAR_LONG(regs->rip));
     else
-        HVMTRACE_ND(VMEXIT, 0, 1/*cycles*/, 2, exit_reason,
-                    regs->eip, 0, 0, 0, 0);
+        HVMTRACE_ND(VMEXIT, 0, 1/*cycles*/, exit_reason, regs->eip);
 
     perfc_incra(vmexits, exit_reason);
 
@@ -4645,7 +4644,7 @@ bool vmx_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(curr->arch.hvm.vmx.lbr_flags & LBR_FIXUP_MASK) )
         lbr_fixup();
 
-    HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/, 0, 0, 0, 0, 0, 0, 0);
+    HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/);
 
     __vmwrite(GUEST_RIP,    regs->rip);
     __vmwrite(GUEST_RSP,    regs->rsp);
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
index 5cd459b855b7..145b59f6ac65 100644
--- a/xen/include/asm-x86/hvm/trace.h
+++ b/xen/include/asm-x86/hvm/trace.h
@@ -67,38 +67,30 @@
 #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
     TRACE_6D(_e, d1, d2, d3, d4)
 
-#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d6) \
+#define HVMTRACE_ND(evt, modifier, cycles, ...)                           \
     do {                                                                  \
         if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )               \
         {                                                                 \
-            struct {                                                      \
-                u32 d[6];                                                 \
-            } _d;                                                         \
-            _d.d[0]=(d1);                                                 \
-            _d.d[1]=(d2);                                                 \
-            _d.d[2]=(d3);                                                 \
-            _d.d[3]=(d4);                                                 \
-            _d.d[4]=(d5);                                                 \
-            _d.d[5]=(d6);                                                 \
+            uint32_t _d[] = { __VA_ARGS__ };                              \
             __trace_var(TRC_HVM_ ## evt | (modifier), cycles,             \
-                        sizeof(*_d.d) * count, &_d);                      \
+                        sizeof(_d), sizeof(_d) ? _d : NULL);              \
         }                                                                 \
     } while(0)
 
 #define HVMTRACE_6D(evt, d1, d2, d3, d4, d5, d6)    \
-    HVMTRACE_ND(evt, 0, 0, 6, d1, d2, d3, d4, d5, d6)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5, d6)
 #define HVMTRACE_5D(evt, d1, d2, d3, d4, d5)        \
-    HVMTRACE_ND(evt, 0, 0, 5, d1, d2, d3, d4, d5,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5)
 #define HVMTRACE_4D(evt, d1, d2, d3, d4)            \
-    HVMTRACE_ND(evt, 0, 0, 4, d1, d2, d3, d4,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4)
 #define HVMTRACE_3D(evt, d1, d2, d3)                \
-    HVMTRACE_ND(evt, 0, 0, 3, d1, d2, d3,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2, d3)
 #define HVMTRACE_2D(evt, d1, d2)                    \
-    HVMTRACE_ND(evt, 0, 0, 2, d1, d2,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1, d2)
 #define HVMTRACE_1D(evt, d1)                        \
-    HVMTRACE_ND(evt, 0, 0, 1, d1,  0,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0, d1)
 #define HVMTRACE_0D(evt)                            \
-    HVMTRACE_ND(evt, 0, 0, 0,  0,  0,  0,  0,  0,  0)
+    HVMTRACE_ND(evt, 0, 0)
 
 #define HVMTRACE_LONG_1D(evt, d1)                  \
                    HVMTRACE_2D(evt ## 64, (d1) & 0xFFFFFFFF, (d1) >> 32)
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:26:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:26:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191056.340995 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3L-0007Cq-1H; Mon, 20 Sep 2021 17:26:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191056.340995; Mon, 20 Sep 2021 17:26:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSN3K-0007Ao-NE; Mon, 20 Sep 2021 17:26:02 +0000
Received: by outflank-mailman (input) for mailman id 191056;
 Mon, 20 Sep 2021 17:26:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSN3I-0003cq-DE
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:26:00 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c7800014-1a37-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:25:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7800014-1a37-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632158742;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=blg5bSVn1FtPA4eRCL98DmLL9LKNBmU4Jtzbra9P1k8=;
  b=D5uqt+QsDawWW35aCg8jt4mqlc+UIIlq0CvqVejgTOwLVPddQOjUZjkc
   oV4m9KbCx4J8Qk7WJdtPqGsZLqPfPV3Hek6vm3znzEIm2S2z7L9taUSPI
   wWqYgXy5tio6pdzmCocF52mlztBiQvh14LjzlXBo7i6WrjISTy2jZWHow
   c=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: +usJDprPJ7Vo3ZALqmwYSVG+CjURABaRCqpMYhYpw8Y1BL2bHBRRV1/FFKSMBeKjLhz5GNjKP9
 02ntmey2s4xsN5Q2UseyFRm+/f46R8Z7iJy5XBiGfGQwB3zXSgyehGxeTgY6SGeBZ+ipWbqJBQ
 NPQ2qC2dvck5YnvBnCIfr3hVDrS1XGbrDYf03SAy7w8N9DHlGerYcfYPLboY7Wvi7Eo5d7widn
 +yBj+tYJKKjSPO3tGTu05DP6U9JqUHpHos426uKNHoXiCaAJL8N4qTYUcqwIgpT7IAQk1mkjIq
 zABYCEpTLQC1uFLLt5sNtKVa
X-SBRS: 5.1
X-MesageID: 53560997
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:sdl5R6mWI1Kde09Wk8JvH3To5gx6IURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJODG3TM/+PYWDzLtFyat7j9BsG75bSy95lGgFlqCg0HiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NYdu7ibRiIIApHRhO0jciVjMTolGaITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQK2DO
 ZJIOGUHgBLoMixmCEUWGYsHssCRt2GveGdhhU2FnP9ii4TU5FMoi+W8WDbPQfSob8hImkeTp
 krd4n/0RBodMbS3yyeB83+qrv/Cm2X8Qo16PLG2//FlgVqXx0QIFQYbE1C8pJGRiEGkXMlEA
 1cJ4Sdopq83nGSqUdvVTxC+uGSDvBMXR5xXCeJSwBGAzO/Y7hiUAkAATyVdc5o2uckuXzso2
 1SV2dTzClRHr7m9WX+bsLCOoluaIjMJJGUPYSsFSwot4NT5pow3yBXVQb5e/LWd14OvX2uqm
 nbT8XZ41+57YdM3O7uT/gDJpxmjg7n1chMp7zv2YlmH9Vhda9vwD2C30mQ3/cqsPa7AEALY7
 CNVypDEhAwdJcrSz33WGY3hCJnsvqzcYWOG2TaDCrF8r2zFxpK1QWxHDNiSzm9SO8AYcHfCZ
 EbJsGu9D7cCYSP3Mcebj2+3YvnGLJQM9/y+DZg4jfIUO/CdkTNrGwk0PiatM5jFyhRErE3GE
 c7znTyQ4ZMm5UNPl2Deegvg+eVzmnBWKZ37HMimp/hY7VZuTCHMEupUWLd/Rss48LmFsG3oH
 yV3bpDRoyizpNbWO3GNmaZKdAhiBSFiWfje9pwGHsbec1EOMDxwVJfsLUYJJtUNc1J9zbyTo
 BlQmyZwlTLCuJEwAV/bMiw8MOy/A8kXQLBSFXVEAGtEEkMLOe6HhJrzvbNuFVX+3OA8n/NyU
 dcffMCMXqZGRjjdomxPZpjhto1yMh+sgFvWbSaiZTE+eb9mRhDIpYC4Lle+qnFWA3rlr9Y6r
 p2hyhjfHcgJSTN9AZuEc/mo1V6w4yQQwbogQ0vSL9BPU0zw64w2eTfph/o6LphUexXOzzeXz
 SiMBhIcqbWfqoM56oCR16uFs52oA611GU8DRzvX6rO/NC/7+Gu/wNAfDLbULG6FDG6tofesf
 +RYyf34IcYrplcSvtouCatvwII/+8Dr++1QwDN7ESiZdF+sEL5hfCWLhJEdqq1Xy7ZFkgKqQ
 UbTqMJCMLCENc65Ql4cIA0pMraK2f0Ow2SA6P00JAPx5TNt/arBWkJXZkHehCtYJbpzEYUk3
 eZ+55JGt13h0kInYoSckyRZ12WQNXhRAawou6YTDJLvlgd2mEpJZobRC3Ou7ZyCAzmW3pLG/
 tNAaHL+uolh
IronPort-HdrOrdr: A9a23:zzdLT6PsDXUPYcBcTv2jsMiBIKoaSvp037BN7TEUdfU1SL38qy
 nApoV56faZslcssRIb9+xoWpPwJk80nKQdieN9AV7LZniBhILCFvAB0WKN+V3d8gTFh5dgPf
 gKScND4afLYmSSJ/yKmDVQaOxN/OW6
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53560997"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Juergen Gross <jgross@suse.com>,
	Dario Faggioli <dfaggioli@suse.com>
Subject: [PATCH v2 08/12] xen/sched: Clean up trace handling
Date: Mon, 20 Sep 2021 18:25:25 +0100
Message-ID: <20210920172529.24932-9-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

There is no need for bitfields anywhere - use more sensible types.  There is
also no need to cast 'd' to (unsigned char *) before passing it to a function
taking void *.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Juergen Gross <jgross@suse.com>
CC: Dario Faggioli <dfaggioli@suse.com>

v2:
 * New
---
 xen/common/sched/core.c   |  4 ++--
 xen/common/sched/credit.c | 38 ++++++++++++++++++--------------------
 xen/common/sched/null.c   | 42 +++++++++++++++++++++++++-----------------
 3 files changed, 45 insertions(+), 39 deletions(-)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8f4b1ca10d1c..fe133cbf117c 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -205,7 +205,7 @@ static inline struct scheduler *vcpu_scheduler(const struct vcpu *v)
 
 static inline void trace_runstate_change(const struct vcpu *v, int new_state)
 {
-    struct { uint32_t vcpu:16, domain:16; } d;
+    struct { uint16_t vcpu, domain; } d;
     uint32_t event;
 
     if ( likely(!tb_init_done) )
@@ -223,7 +223,7 @@ static inline void trace_runstate_change(const struct vcpu *v, int new_state)
 
 static inline void trace_continue_running(const struct vcpu *v)
 {
-    struct { uint32_t vcpu:16, domain:16; } d;
+    struct { uint16_t vcpu, domain; } d;
 
     if ( likely(!tb_init_done) )
         return;
diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c
index d0aa017c643e..f277fa37a8b1 100644
--- a/xen/common/sched/credit.c
+++ b/xen/common/sched/credit.c
@@ -1828,21 +1828,18 @@ static void csched_schedule(
     SCHED_STAT_CRANK(schedule);
     CSCHED_UNIT_CHECK(unit);
 
-    /*
-     * Here in Credit1 code, we usually just call TRACE_nD() helpers, and
-     * don't care about packing. But scheduling happens very often, so it
-     * actually is important that the record is as small as possible.
-     */
     if ( unlikely(tb_init_done) )
     {
         struct {
-            unsigned cpu:16, tasklet:8, idle:8;
-        } d;
-        d.cpu = cur_cpu;
-        d.tasklet = tasklet_work_scheduled;
-        d.idle = is_idle_unit(unit);
-        __trace_var(TRC_CSCHED_SCHEDULE, 1, sizeof(d),
-                    (unsigned char *)&d);
+            uint16_t cpu;
+            uint8_t tasklet, idle;
+        } d = {
+            .cpu     = cur_cpu,
+            .tasklet = tasklet_work_scheduled,
+            .idle    = is_idle_unit(unit),
+        };
+
+        __trace_var(TRC_CSCHED_SCHEDULE, 1, sizeof(d), &d);
     }
 
     runtime = now - unit->state_entry_time;
@@ -1904,14 +1901,15 @@ static void csched_schedule(
         if ( unlikely(tb_init_done) )
         {
             struct {
-                unsigned unit:16, dom:16;
-                unsigned runtime;
-            } d;
-            d.dom = unit->domain->domain_id;
-            d.unit = unit->unit_id;
-            d.runtime = runtime;
-            __trace_var(TRC_CSCHED_RATELIMIT, 1, sizeof(d),
-                        (unsigned char *)&d);
+                uint16_t unit, dom;
+                uint32_t runtime;
+            } d = {
+                .dom     = unit->domain->domain_id,
+                .unit    = unit->unit_id,
+                .runtime = runtime,
+            };
+
+            __trace_var(TRC_CSCHED_RATELIMIT, 1, sizeof(d), &d);
         }
 
         goto out;
diff --git a/xen/common/sched/null.c b/xen/common/sched/null.c
index 82d5d1baab85..deb59747fbe8 100644
--- a/xen/common/sched/null.c
+++ b/xen/common/sched/null.c
@@ -329,10 +329,12 @@ pick_res(const struct null_private *prv, const struct sched_unit *unit)
         struct {
             uint16_t unit, dom;
             uint32_t new_cpu;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.new_cpu = new_cpu;
+        } d = {
+            .unit    = unit->unit_id,
+            .dom     = unit->domain->domain_id,
+            .new_cpu = new_cpu,
+        };
+
         __trace_var(TRC_SNULL_PICKED_CPU, 1, sizeof(d), &d);
     }
 
@@ -357,10 +359,12 @@ static void unit_assign(struct null_private *prv, struct sched_unit *unit,
         struct {
             uint16_t unit, dom;
             uint32_t cpu;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.cpu = cpu;
+        } d = {
+            .unit = unit->unit_id,
+            .dom  = unit->domain->domain_id,
+            .cpu  = cpu,
+        };
+
         __trace_var(TRC_SNULL_UNIT_ASSIGN, 1, sizeof(d), &d);
     }
 }
@@ -388,10 +392,12 @@ static bool unit_deassign(struct null_private *prv, const struct sched_unit *uni
         struct {
             uint16_t unit, dom;
             uint32_t cpu;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.cpu = cpu;
+        } d = {
+            .unit = unit->unit_id,
+            .dom  = unit->domain->domain_id,
+            .cpu  = cpu,
+        };
+
         __trace_var(TRC_SNULL_UNIT_DEASSIGN, 1, sizeof(d), &d);
     }
 
@@ -691,11 +697,13 @@ static void null_unit_migrate(const struct scheduler *ops,
         struct {
             uint16_t unit, dom;
             uint16_t cpu, new_cpu;
-        } d;
-        d.dom = unit->domain->domain_id;
-        d.unit = unit->unit_id;
-        d.cpu = sched_unit_master(unit);
-        d.new_cpu = new_cpu;
+        } d = {
+            .unit    = unit->unit_id,
+            .dom     = unit->domain->domain_id,
+            .cpu     = sched_unit_master(unit),
+            .new_cpu = new_cpu,
+        };
+
         __trace_var(TRC_SNULL_MIGRATE, 1, sizeof(d), &d);
     }
 
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:34:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:34:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191092.341006 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNBg-0002PC-6j; Mon, 20 Sep 2021 17:34:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191092.341006; Mon, 20 Sep 2021 17:34:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNBg-0002P5-3k; Mon, 20 Sep 2021 17:34:40 +0000
Received: by outflank-mailman (input) for mailman id 191092;
 Mon, 20 Sep 2021 17:34:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSNBe-0002Oz-Sr
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:34:38 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3aa76012-671c-46b2-b7a2-2af9ef69522c;
 Mon, 20 Sep 2021 17:34:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3aa76012-671c-46b2-b7a2-2af9ef69522c
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632159276;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=bEpl8O1bP+VDWCJVrreYPoTbBZNOjPBNBltwePd93CU=;
  b=KF5GMFAE6TRaE/3VSJfM+NcTNBHSQ94Xl6twuYgkrqIxIhECxhURbpHQ
   otn6N/He5dOJZ4qPy0Fz4Mo0767et1gbX/6KdQUMfspkt7fnPS5mrRRDW
   9pSHuKmcAujEgFttZvpi1ms13ggBlKywOskFaQ32zdYh0omwIvOvXpn80
   w=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: jgyIU6088m3DDnn0RqsxcHn+/fcrkyd6geGy4fk9c2sF5hI+Xad3Tn7YU41JSyyJaW5ck7Gd+g
 w5TGS1/+aZZ0ZzEuft50O1FXgLn4AQt0+7Cl5dyyPakWc+5749KEIgg462kis+2bqP0PsPDpvi
 z32BucEwxDuVqS8msx2P/9A5/weFq770xh52KUNknDB44QyjQ1xW0dP68RQU2mBQaOV099mRax
 STU++DjoGaMe6l0VqO+gvsgMlf+XfKut3VaZBrJ2/ic40tEwhZSitfoMXWMGP9S0OiAh3z+rd0
 EJZSVcw4pA3wMh9pkHZNNpVO
X-SBRS: 5.1
X-MesageID: 53136338
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:LIKyZ6r0JKXuBZcPfpBy+qXNNPBeBmIAYhIvgKrLsJaIsI4StFCzt
 garIBnUOa2INGWkctF1boXno00EucXWm982SQM5pCkxHypB8ZuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0ILhW1nlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnb+gVlk7YI2WpMU2VQIfIShSG71MyIaSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFKoZtmtt0nfyCvE+TIqYa67L+cVZzHE7gcUm8fP2O
 pFAMmQ0N0SojxtnIFYUBs0TgemSoXS8KTJRiG3IofcV7D2GpOB2+Oe0a4eEEjCQfu1Kmm6Iq
 2SA+H72ajk4HtGCzTuO8lq3m/TC2yj8Xeo6BLC+s/JnnlCX7mgSEwENE0u2p+GjjUyzUM4ZL
 FYbkhfCtoBrqhbtFIOkGUTl/jjU5XbwRua8DcUZ+B7T2PT0uTq4LW4fCQxIRPo2vv8PEGlCO
 kCyoz/5OdB+mOTLEivNrebF9Wna1Ts9djBZNHRdJecRy5y6+thi0EiXJjp2OPPt1rXI9SfML
 ydmRcTUr44ai9ICn46/9ErO695HjsmUFlNpjuk7s2TM0++YWGJHT9fzgbQ4xawZRGp8crVml
 CJf8/VyFMhUUfmweNWlGY3h5o1FAspp1xWH2zZS82QJrWzxqxZPg6gJuGwWyLhV3jYsJmayP
 R67VfJ5z55PJnq6BZKbkKroUJ9C8EQULvy8Dqq8RoMXOvBZLVbblAkzNR/49z28yyAEzPBgU
 ap3hO7xVB72/4w8l2HoLwrcuJd2rh0DKZT7H8Gjk0v/jurAOxZ4i94taTOzUwzw14vcyC29z
 jqVH5LiJ8x3XLKsby/J35QUKFxWf3E3CYqv855cd/KZIxogE2YkUqeDzbQkco1jvqJUiuaXo
 S3tBh4GkALy1S/dNAGHSnF/c7ezD5xxmm02YH43NlGy1nl9PYv2tPUDd4E6dKUM/fB4yaImV
 OEMfsiNW6wdSjnO9zkHQ4P6qYhuKEaiiQ6UZnL3azkjZZ9wAQfO/4a8LAfo8SAPCAuxtNc//
 OL8hl+KH8JbSl07XsjMaf+pw1eghlQnmbp/DxnSP91eWETw64w2eSb/ueA6fpMXIhLZyzrEi
 wvPWUUEpfPAqpMe+cXSgfzWtJ+gFuZzExYIH2Tf6rrqZyDW8nD6nN1FWeeMOzvcSHn16OOpY
 uAMl6PwN/gOnVBrtYtgEuk0kfJitoW3/7IKnB55GHjrbkiwDuIyK3aL6sBDq6lRy+ILogCxQ
 E+OpoFXNLjh1BkJy7LNyN7Jtti+6Mw=
IronPort-HdrOrdr: A9a23:9esM3KBYDB6bG3XlHemU55DYdb4zR+YMi2TC1yhKJyC9Ffbo7v
 xG/c5rsyMc5wxwZJhNo7y90ey7MBbhHP1OkO4s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpN
 9dmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,308,1624334400"; 
   d="scan'208";a="53136338"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH v2 10/12] x86/pv: Move x86/trace.c to x86/pv/trace.c
Date: Mon, 20 Sep 2021 18:25:27 +0100
Message-ID: <20210920172529.24932-11-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

This entire file is pv-only, and not excluded from the build by
CONFIG_TRACEBUFFER.  Move it into the pv/ directory, build it conditionally,
and drop unused includes.

Also move the contents of asm/trace.h to asm/pv/trace.h to avoid the functions
being declared across the entire hypervisor.

One caller in fixup_page_fault() is effectively PV only, but is not subject to
dead code elimination.  Add an additional IS_ENABLED(CONFIG_PV) to keep the
build happy.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v2:
 * New
---
 xen/arch/x86/Makefile                |  1 -
 xen/arch/x86/pv/Makefile             |  1 +
 xen/arch/x86/pv/emul-inv-op.c        |  2 +-
 xen/arch/x86/pv/emul-priv-op.c       |  1 +
 xen/arch/x86/pv/ro-page-fault.c      |  2 +-
 xen/arch/x86/{ => pv}/trace.c        | 13 +++++------
 xen/arch/x86/pv/traps.c              |  2 +-
 xen/arch/x86/traps.c                 |  3 ++-
 xen/include/asm-x86/{ => pv}/trace.h |  8 ++++---
 xen/include/asm-x86/trace.h          | 42 ------------------------------------
 10 files changed, 17 insertions(+), 58 deletions(-)
 rename xen/arch/x86/{ => pv}/trace.c (95%)
 copy xen/include/asm-x86/{ => pv}/trace.h (92%)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 202d4d27823d..82dd4e18bd36 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -61,7 +61,6 @@ obj-y += spec_ctrl.o
 obj-y += srat.o
 obj-y += string.o
 obj-y += time.o
-obj-y += trace.o
 obj-y += traps.o
 obj-y += tsx.o
 obj-y += usercopy.o
diff --git a/xen/arch/x86/pv/Makefile b/xen/arch/x86/pv/Makefile
index 75b01b0062d8..6cda354cc41f 100644
--- a/xen/arch/x86/pv/Makefile
+++ b/xen/arch/x86/pv/Makefile
@@ -12,6 +12,7 @@ obj-y += misc-hypercalls.o
 obj-y += mm.o
 obj-y += ro-page-fault.o
 obj-$(CONFIG_PV_SHIM) += shim.o
+obj-$(CONFIG_TRACEBUFFER) += trace.o
 obj-y += traps.o
 
 obj-bin-y += dom0_build.init.o
diff --git a/xen/arch/x86/pv/emul-inv-op.c b/xen/arch/x86/pv/emul-inv-op.c
index b15f302bcaad..2c07eed9a092 100644
--- a/xen/arch/x86/pv/emul-inv-op.c
+++ b/xen/arch/x86/pv/emul-inv-op.c
@@ -19,7 +19,7 @@
  * along with this program; If not, see <http://www.gnu.org/licenses/>.
  */
 
-#include <xen/trace.h>
+#include <asm/pv/trace.h>
 
 #include "emulate.h"
 
diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index 11467a1e3a90..d0df5bc616c0 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -30,6 +30,7 @@
 #include <asm/hypercall.h>
 #include <asm/mc146818rtc.h>
 #include <asm/pv/domain.h>
+#include <asm/pv/trace.h>
 #include <asm/shared.h>
 
 #include <xsm/xsm.h>
diff --git a/xen/arch/x86/pv/ro-page-fault.c b/xen/arch/x86/pv/ro-page-fault.c
index 335aa8dc5dc0..ac5b66870c8c 100644
--- a/xen/arch/x86/pv/ro-page-fault.c
+++ b/xen/arch/x86/pv/ro-page-fault.c
@@ -20,7 +20,7 @@
  * along with this program; If not, see <http://www.gnu.org/licenses/>.
  */
 
-#include <xen/trace.h>
+#include <asm/pv/trace.h>
 #include <asm/shadow.h>
 
 #include "emulate.h"
diff --git a/xen/arch/x86/trace.c b/xen/arch/x86/pv/trace.c
similarity index 95%
rename from xen/arch/x86/trace.c
rename to xen/arch/x86/pv/trace.c
index 4a953c5b2f1e..550c22765bae 100644
--- a/xen/arch/x86/trace.c
+++ b/xen/arch/x86/pv/trace.c
@@ -1,9 +1,6 @@
-#include <xen/init.h>
-#include <xen/kernel.h>
-#include <xen/lib.h>
-#include <xen/domain.h>
 #include <xen/sched.h>
-#include <xen/trace.h>
+
+#include <asm/pv/trace.h>
 
 void __trace_pv_trap(int trapnr, unsigned long eip,
                      int use_error_code, unsigned error_code)
@@ -21,7 +18,7 @@ void __trace_pv_trap(int trapnr, unsigned long eip,
         d.trapnr = trapnr;
         d.error_code = error_code;
         d.use_error_code=!!use_error_code;
-                
+
         __trace_var(TRC_PV_TRAP, 1, sizeof(d), &d);
     }
     else
@@ -38,7 +35,7 @@ void __trace_pv_trap(int trapnr, unsigned long eip,
         d.trapnr = trapnr;
         d.error_code = error_code;
         d.use_error_code=!!use_error_code;
-                
+
         event = TRC_PV_TRAP;
         event |= TRC_64_FLAG;
         __trace_var(event, 1, sizeof(d), &d);
@@ -58,7 +55,7 @@ void __trace_pv_page_fault(unsigned long addr, unsigned error_code)
         d.eip = eip;
         d.addr = addr;
         d.error_code = error_code;
-                
+
         __trace_var(TRC_PV_PAGE_FAULT, 1, sizeof(d), &d);
     }
     else
diff --git a/xen/arch/x86/pv/traps.c b/xen/arch/x86/pv/traps.c
index 7439b76df88d..764773c02104 100644
--- a/xen/arch/x86/pv/traps.c
+++ b/xen/arch/x86/pv/traps.c
@@ -22,10 +22,10 @@
 #include <xen/event.h>
 #include <xen/hypercall.h>
 #include <xen/lib.h>
-#include <xen/trace.h>
 #include <xen/softirq.h>
 
 #include <asm/apic.h>
+#include <asm/pv/trace.h>
 #include <asm/shared.h>
 #include <asm/traps.h>
 #include <irq_vectors.h>
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 4a0e498b4c21..0cc1ee95cb5b 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -82,6 +82,7 @@
 #include <xsm/xsm.h>
 #include <asm/mach-default/irq_vectors.h>
 #include <asm/pv/traps.h>
+#include <asm/pv/trace.h>
 #include <asm/pv/mm.h>
 
 /*
@@ -1480,7 +1481,7 @@ static int fixup_page_fault(unsigned long addr, struct cpu_user_regs *regs)
     {
         int ret = paging_fault(addr, regs);
 
-        if ( ret == EXCRET_fault_fixed )
+        if ( IS_ENABLED(CONFIG_PV) && ret == EXCRET_fault_fixed )
             trace_trap_two_addr(TRC_PV_PAGING_FIXUP, regs->rip, addr);
         return ret;
     }
diff --git a/xen/include/asm-x86/trace.h b/xen/include/asm-x86/pv/trace.h
similarity index 92%
copy from xen/include/asm-x86/trace.h
copy to xen/include/asm-x86/pv/trace.h
index e65b5de6eec4..c616206eeb15 100644
--- a/xen/include/asm-x86/trace.h
+++ b/xen/include/asm-x86/pv/trace.h
@@ -1,5 +1,7 @@
-#ifndef __ASM_TRACE_H__
-#define __ASM_TRACE_H__
+#ifndef XEN_X86_PV_TRACE_H
+#define XEN_X86_PV_TRACE_H
+
+#include <xen/trace.h>
 
 #include <asm/page.h>
 
@@ -43,4 +45,4 @@ static inline void trace_ptwr_emulation(unsigned long addr, l1_pgentry_t npte)
         __trace_ptwr_emulation(addr, npte);
 }
 
-#endif /* __ASM_TRACE_H__ */
+#endif /* XEN_X86_PV_TRACE_H */
diff --git a/xen/include/asm-x86/trace.h b/xen/include/asm-x86/trace.h
index e65b5de6eec4..edef1bb099d4 100644
--- a/xen/include/asm-x86/trace.h
+++ b/xen/include/asm-x86/trace.h
@@ -1,46 +1,4 @@
 #ifndef __ASM_TRACE_H__
 #define __ASM_TRACE_H__
 
-#include <asm/page.h>
-
-void __trace_pv_trap(int trapnr, unsigned long eip,
-                     int use_error_code, unsigned error_code);
-static inline void trace_pv_trap(int trapnr, unsigned long eip,
-                                 int use_error_code, unsigned error_code)
-{
-    if ( unlikely(tb_init_done) )
-        __trace_pv_trap(trapnr, eip, use_error_code, error_code);
-}
-
-void __trace_pv_page_fault(unsigned long addr, unsigned error_code);
-static inline void trace_pv_page_fault(unsigned long addr,
-                                       unsigned error_code)
-{
-    if ( unlikely(tb_init_done) )
-        __trace_pv_page_fault(addr, error_code);
-}
-
-void __trace_trap_one_addr(unsigned event, unsigned long va);
-static inline void trace_trap_one_addr(unsigned event, unsigned long va)
-{
-    if ( unlikely(tb_init_done) )
-        __trace_trap_one_addr(event, va);
-}
-
-void __trace_trap_two_addr(unsigned event, unsigned long va1,
-                           unsigned long va2);
-static inline void trace_trap_two_addr(unsigned event, unsigned long va1,
-                                       unsigned long va2)
-{
-    if ( unlikely(tb_init_done) )
-        __trace_trap_two_addr(event, va1, va2);
-}
-
-void __trace_ptwr_emulation(unsigned long addr, l1_pgentry_t npte);
-static inline void trace_ptwr_emulation(unsigned long addr, l1_pgentry_t npte)
-{
-    if ( unlikely(tb_init_done) )
-        __trace_ptwr_emulation(addr, npte);
-}
-
 #endif /* __ASM_TRACE_H__ */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:34:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:34:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191094.341019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNBx-0002rP-Ic; Mon, 20 Sep 2021 17:34:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191094.341019; Mon, 20 Sep 2021 17:34:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNBx-0002rI-DB; Mon, 20 Sep 2021 17:34:57 +0000
Received: by outflank-mailman (input) for mailman id 191094;
 Mon, 20 Sep 2021 17:34:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSNBw-0002pz-QL
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:34:56 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 119f6c42-1a39-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:34:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 119f6c42-1a39-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632159295;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=l7ek/uy2/EYPLwKguMBv7A4G/dnRwY7hjtQmVQ7Fv4Y=;
  b=g3abrO1VCA0jyGONhkJuYTz0CEzQ7pBuyhlyovV/ojfTpFior+6xeXas
   C/I0WgU/3pc38d1WwHUAlRDfkbvQcD2cv0rgoKDABX3MbV7kspijRE3ew
   pWsahSIftfP7/JneJ6GJw83icRr0oAxPXtW4vIk8Ko3i1dh0U/HctB+Wg
   4=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: uox8gRHU445v+MNpqPTbMwaBLEBftuG4qoxsBtQ7m5ew+SILQgpY2ghUvWiMG7wokWS+PXHCM7
 d3BfdUpu9K+l/BTASTYcnfpKo8Z8yH/Mg3cCdhpPF3fnBsuSO7iS7xvhh2FWBnC9itLOWfwhiT
 CUMA0F4S00m773kfjEtB8g7IyHNth73ZcNlxjzOZ92+q3nycrzUFE6Zk7B1opy5OSnSE0M4oGp
 Zj5f1GcS5mirop/W93XDln7PlxzDWqY6HCKdcJPtNfANc85r6V3yIWHYTMgqo+fAeliwvOt4jK
 92eY2c6ZmKOhrq06HbwZf4EP
X-SBRS: 5.1
X-MesageID: 52741883
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:JN8Eial0pzg3B9CRD2GFXQ3o5gxmIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJNUD+Fbq7eYjCnKo9zYIm08x8GvMTSzdJhGgA++SFnEyMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qw2YPR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NdipazqZDdzBbfnseohVSdBDjtCPLITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBODtMJkSpTdLyjbBAOx9aZvCX7/L9ZlT2zJYasVmQKyBO
 5NJOWEHgBLoJBZ3amkZEL0Fjr23nCCjLwIB9FPInP9ii4TU5FMoi+W8WDbPQfSVQe1Fk0Deo
 XjJl0z7HxUbOdq32TeDtHW2iYfnoyT/X44DEayiwdRjilaT2287BQUfUB2wpvzRol6zXZdTJ
 lIZ/gIqrLMu7wq7Q9/lRRq6rXWY+BkGVLJ4Mcc39QWMwar8+BuCCy4PSTspQMc9qMY8SDgu1
 1mIt9DkHzpitPuSU331y1uPhWrsY25PdzZEPHJaC1teizX+nG0tphDgXuY5PZ7vtcfeMD/Mx
 Q6TsTIChqpG2KbnyJ6HEUD7byOE/8aSF15ut1SPBApJ/SsiO9X0PNXABUzzqK8addfHFAHpU
 G0sxpDGhN3iG61hg8BkrA8lJ7iv+//NGznVm1cH83IJpmn1piLLkWy95lhDyKZV3iQsImSBj
 Kz741o5CHpv0JyCN/QfXm5JI552pZUM7Py8PhwuUjarXnSWXFTdlByCmGbKhzy9+KTSufhnZ
 P93jvpA/V5FUP86nVJats820KMxxzBW+I8gbcmglHyaPU6lTCfNE98taQLWBshgtf/siFiFo
 r53aprRoz0CAbKWX8Ui2dNKRbz8BSNgXs6eRg0+XrPrHzeK70l7WqeNnut+K9I690mX/8+Rl
 kyAtoZj4AKXrRX6xc+iMyoLhGrHUcktoHQlEzYrOFr0iXEvbZz2tPUUdoctfKlh/+tmlKYmQ
 /4AcsSGI/JOVjWYpGhNMcij9NRvJEaxmAaDHyu5ezxjLZRucBPEp43/dQz1+ShQUifu7Zkio
 6et3x/wSIYYQ1gwF97fbf+ilgvjvXUUlO9ocVHPJ91fJBfl/IRwcnSjhf4rOcAcbx7Ew2LCh
 QqRBB4Zo8jLopM0r4aV1fzV8d/xHrInTERAHmTd4bKnDgXg/zKukd1aTeKFXTHBT2eoqq+sU
 vpYkqPnO/odkVcU74clS+R3zbgz7sfErqNBylg2B23CalmmB+8yInSC2sUT5KRByqUA5FmzU
 0OLvNJbJa+IKIXuF1tIfFgpaeGK1Pc1nDjO7KtqfBWmtXEvpLfXA19POxSsiTBGKOonOYwo9
 u4tpcoK5lHtkREtKNuH0nhZ+mnkwqbsiEn7WkX22LPWtzc=
IronPort-HdrOrdr: A9a23:+Onija+OkwhFDIOs/Jluk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P
 re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM
 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="52741883"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 11/12] xen/arch: Drop asm-*/trace.h
Date: Mon, 20 Sep 2021 18:25:28 +0100
Message-ID: <20210920172529.24932-12-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

The feature is x86 and pv-dom0 specific, and each architecture's main trace.h
files are empty.  Don't force all new architectures to create an empty file.

While moving the declaration of tb_init_done, change from int to bool.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

v2:
 * New
---
 xen/common/trace.c          |  2 +-
 xen/include/asm-arm/trace.h | 12 ------------
 xen/include/asm-x86/trace.h |  4 ----
 xen/include/xen/trace.h     | 13 +++++--------
 4 files changed, 6 insertions(+), 25 deletions(-)
 delete mode 100644 xen/include/asm-arm/trace.h
 delete mode 100644 xen/include/asm-x86/trace.h

diff --git a/xen/common/trace.c b/xen/common/trace.c
index 41a3c170446b..a5dc468296d6 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -67,7 +67,7 @@ static DEFINE_PER_CPU(unsigned long, lost_records_first_tsc);
 
 /* a flag recording whether initialization has been done */
 /* or more properly, if the tbuf subsystem is enabled right now */
-int tb_init_done __read_mostly;
+bool __read_mostly tb_init_done;
 
 /* which CPUs tracing is enabled on */
 static cpumask_t tb_cpu_mask;
diff --git a/xen/include/asm-arm/trace.h b/xen/include/asm-arm/trace.h
deleted file mode 100644
index e06def61f6f3..000000000000
--- a/xen/include/asm-arm/trace.h
+++ /dev/null
@@ -1,12 +0,0 @@
-#ifndef __ASM_TRACE_H__
-#define __ASM_TRACE_H__
-
-#endif /* __ASM_TRACE_H__ */
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/include/asm-x86/trace.h b/xen/include/asm-x86/trace.h
deleted file mode 100644
index edef1bb099d4..000000000000
--- a/xen/include/asm-x86/trace.h
+++ /dev/null
@@ -1,4 +0,0 @@
-#ifndef __ASM_TRACE_H__
-#define __ASM_TRACE_H__
-
-#endif /* __ASM_TRACE_H__ */
diff --git a/xen/include/xen/trace.h b/xen/include/xen/trace.h
index 3cbb542af888..055883287e8c 100644
--- a/xen/include/xen/trace.h
+++ b/xen/include/xen/trace.h
@@ -21,19 +21,14 @@
 #ifndef __XEN_TRACE_H__
 #define __XEN_TRACE_H__
 
-/* Put 'tb_init_done' here because 'asm/trace.h' may use it */
-#ifdef CONFIG_TRACEBUFFER
-extern int tb_init_done;
-#else
-#define tb_init_done false
-#endif
-
 #include <xen/types.h>
 #include <public/sysctl.h>
 #include <public/trace.h>
-#include <asm/trace.h>
 
 #ifdef CONFIG_TRACEBUFFER
+
+extern bool tb_init_done;
+
 /* Used to initialise trace buffer functionality */
 void init_trace_bufs(void);
 
@@ -58,6 +53,8 @@ void __trace_hypercall(uint32_t event, unsigned long op,
 
 #include <xen/errno.h>
 
+#define tb_init_done false
+
 static inline void init_trace_bufs(void) {}
 static inline int tb_control(struct xen_sysctl_tbuf_op *tbc)
 {
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 17:35:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 17:35:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191095.341029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNC2-0003CF-R7; Mon, 20 Sep 2021 17:35:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191095.341029; Mon, 20 Sep 2021 17:35:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNC2-0003C8-ME; Mon, 20 Sep 2021 17:35:02 +0000
Received: by outflank-mailman (input) for mailman id 191095;
 Mon, 20 Sep 2021 17:35:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSNC1-0002pz-NW
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 17:35:01 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 119f6c43-1a39-11ec-b87a-12813bfff9fa;
 Mon, 20 Sep 2021 17:34:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 119f6c43-1a39-11ec-b87a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632159296;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=GcABwI8oWHgu530a+/KgqxWauKwNAPSdO5EboEi94Ao=;
  b=Yg74MBquc/o5k5Q+yVrZvcVvjRK6IaWTq3GXWJg/cxo700e/cQfaB535
   qRKIzxVl1FmHmj4j7l+zh5Xz4mEubpEHGRaVggUNvsn53PBBSl4vor6yq
   unpP3XxHiS5qmtJXwUhJCvatk5daTcQ4bJkRxGjhxcjhiO/JNjG5v3ljb
   c=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: P7famdpIYSd9Iat7FzhqXhX4HUaZbz0EeCQRaQ6rqyFmwr5Za3YaAPyV85GzKrtJQ9tu5bJl42
 RWVeXfrsmVfZfnTu7+4KmVQ2weBwttctvKGxwvm8JsPlsch+c4pgYAi4Kfj46ORWsLxZT2SWXG
 0XP/YPlEZ+z0Sh/xibisxBH9+J9OS2VSH93KzVr/acBK4a4l6A84U1zZJmLO8lxvCw8aExVx3C
 xI0WocMQ1MB1IhLy/QX/L0zYgZgNJ83cB/CziO4B4k1sDumXMDRtXm4qL8BpP5aZLB9Viore6k
 Wkzi1RxUY9KpossjblwH6mP/
X-SBRS: 5.1
X-MesageID: 52741880
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:/bhr6KxIECZE6TC6NVJ6t+cvwSrEfRIJ4+MujC+fZmUNrF6WrkUHz
 GVJX22BPfqJZWChedgna47noBwD7J6Gn9NrHgo+rCAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRh2tUAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt4tr5
 8sWsKOgdSd3Br/JmuI6QgRpTRgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25oRQa+ON
 5JxhTxHZwXcaRsRHnAtVYNjxtmUtmDVI21RkQfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krk1WnkBhARNPSE1CGItHmrg4fnjS79HY4fCrC83vprm0GIgHweDgUMUlm2quX/jVSxM++zM
 GRNpHBo9/JrshX2EJ+tBHVUvUJooDZHQfVOSOgF+D2tlJLG5gSfIXkZf2B4PYlOWNANedA66
 rOYt4q3XmU/6+zEESj1GqS89mzpaHNMRYMWTWpdF1JUvYO7yG0mpk+XFr5e/LiJYsoZ8N0a6
 wuDqjQ3z54XhNQCv0lQ1QGa22/wznQlowhc2+k2Yo5HxlgiDGJGT9bxgbQ+0RqnBNzCJmRtR
 FBex6CjABkmVPlhbhBhpdnh+5nyvZ643MD02wYzT/HNCRz0oyPLkX9sDMFWex4yb5dslc7BS
 07PowJBjKK/z1PzNvQfXm5FMOxzlfKIPY28Dpj8N4MSCrAsJF7v1Hw/Pia4gjGy+HXAZIliY
 P93h+73Vi1EYUmmpRLrL9ogPUgDnHFmmTyLGsClkHxKE9O2PRaodFvMC3PWBshR0U9OiFS9H
 w93O5TYxhNBfvf5ZyWLo4cfIUpTdSowBIzsqtwRfemGe1I0FGYkAv7X4LUgZ406wPgFyraWp
 imwCh1C1V7ypXzbMgHWOHptX6ziAMRkpnUhMC1yYVvxgyo/YZyi5bs0focseeV17/RqyPN5F
 qFXe8iJDvlVZC7A/jARMcv0oIB4LUz5jgOSJSu1JjM4esc4FQDO/9bleCrp9TUPUXXr5Zdv/
 eX421qCE5QZRglkAMLHU96Vzgu87SoHheZ/f0rUOd0PKk/ix5dndn7qhfgtLsBSdRianmmG1
 xybCAszrPXWp9Nn68HAgK2Jotv7E+Z6GUYGTWDX4azvaHvf92unh4RBTPyJbXbWU2atoPeuY
 uBczvfdNvwbnQkV79ogQugzla9utcHyo7J6zxh/GCSZZlumPbpsP32a0JQdraZK3LJY5VO7V
 0/nFgO24llV1BcJyGIsGTc=
IronPort-HdrOrdr: A9a23:c5rIWqCFmITTuWvlHemU55DYdb4zR+YMi2TC1yhKJyC9Ffbo7v
 xG/c5rsyMc5wxwZJhNo7y90ey7MBbhHP1OkO4s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpN
 9dmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="52741880"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Subject: [PATCH v2 12/12] x86/trace: Clean up trace handling
Date: Mon, 20 Sep 2021 18:25:29 +0100
Message-ID: <20210920172529.24932-13-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Use more appropriate types.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v2:
 * New
---
 xen/arch/x86/irq.c             |   4 +-
 xen/arch/x86/mm/p2m-pt.c       |   6 +-
 xen/arch/x86/mm/shadow/multi.c |   2 +-
 xen/arch/x86/pv/trace.c        | 159 +++++++++++++++++++----------------------
 4 files changed, 78 insertions(+), 93 deletions(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index a1693f92dd92..67cbf6b979dc 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -130,8 +130,8 @@ static void _trace_irq_mask(uint32_t event, int irq, int vector,
                             const cpumask_t *mask)
 {
     struct {
-        unsigned int irq:16, vec:16;
-        unsigned int mask[6];
+        uint16_t irq, vec;
+        uint32_t mask[6];
     } d = {
        .irq = irq,
        .vec = vector,
diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index 5a0c0f5aceff..09c99d78aa40 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -609,9 +609,9 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
     if ( tb_init_done )
     {
         struct {
-            u64 gfn, mfn;
-            int p2mt;
-            int d:16,order:16;
+            uint64_t gfn, mfn;
+            uint32_t p2mt;
+            uint16_t d, order;
         } t;
 
         t.gfn = gfn;
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 8bb028c2e2fa..15265fc81dca 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2118,7 +2118,7 @@ static inline void trace_shadow_emulate(guest_l1e_t gl1e, unsigned long va)
                so put it first for alignment sake. */
             guest_l1e_t gl1e, write_val;
             guest_va_t va;
-            unsigned flags:29, emulation_count:3;
+            uint32_t flags:29, emulation_count:3;
         } d;
         u32 event;
 
diff --git a/xen/arch/x86/pv/trace.c b/xen/arch/x86/pv/trace.c
index 550c22765bae..a952fbc1eb0f 100644
--- a/xen/arch/x86/pv/trace.c
+++ b/xen/arch/x86/pv/trace.c
@@ -7,38 +7,35 @@ void __trace_pv_trap(int trapnr, unsigned long eip,
 {
     if ( is_pv_32bit_vcpu(current) )
     {
-        struct __packed {
-            unsigned eip:32,
-                trapnr:15,
-                use_error_code:1,
-                error_code:16;
-        } d;
-
-        d.eip = eip;
-        d.trapnr = trapnr;
-        d.error_code = error_code;
-        d.use_error_code=!!use_error_code;
+        struct {
+            uint32_t eip;
+            uint16_t trapnr:15;
+            bool use_error_code:1;
+            uint16_t error_code;
+        } d = {
+            .eip            = eip,
+            .trapnr         = trapnr,
+            .use_error_code = use_error_code,
+            .error_code     = error_code,
+        };
 
         __trace_var(TRC_PV_TRAP, 1, sizeof(d), &d);
     }
     else
     {
         struct __packed {
-            unsigned long eip;
-            unsigned trapnr:15,
-                use_error_code:1,
-                error_code:16;
-        } d;
-        unsigned event;
-
-        d.eip = eip;
-        d.trapnr = trapnr;
-        d.error_code = error_code;
-        d.use_error_code=!!use_error_code;
-
-        event = TRC_PV_TRAP;
-        event |= TRC_64_FLAG;
-        __trace_var(event, 1, sizeof(d), &d);
+            uint64_t rip;
+            uint16_t trapnr:15;
+            bool use_error_code:1;
+            uint16_t error_code;
+        } d = {
+            .rip            = eip,
+            .trapnr         = trapnr,
+            .use_error_code = use_error_code,
+            .error_code     = error_code,
+        };
+
+        __trace_var(TRC_PV_TRAP | TRC_64_FLAG, 1, sizeof(d), &d);
     }
 }
 
@@ -48,30 +45,28 @@ void __trace_pv_page_fault(unsigned long addr, unsigned error_code)
 
     if ( is_pv_32bit_vcpu(current) )
     {
-        struct __packed {
-            u32 eip, addr, error_code;
-        } d;
-
-        d.eip = eip;
-        d.addr = addr;
-        d.error_code = error_code;
+        struct {
+            uint32_t eip, addr, error_code;
+        } d = {
+            .eip = eip,
+            .addr = addr,
+            .error_code = error_code,
+        };
 
         __trace_var(TRC_PV_PAGE_FAULT, 1, sizeof(d), &d);
     }
     else
     {
         struct __packed {
-            unsigned long eip, addr;
-            u32 error_code;
-        } d;
-        unsigned event;
-
-        d.eip = eip;
-        d.addr = addr;
-        d.error_code = error_code;
-        event = TRC_PV_PAGE_FAULT;
-        event |= TRC_64_FLAG;
-        __trace_var(event, 1, sizeof(d), &d);
+            uint64_t eip, addr;
+            uint32_t error_code;
+        } d = {
+            .eip = eip,
+            .addr = addr,
+            .error_code = error_code,
+        };
+
+        __trace_var(TRC_PV_PAGE_FAULT | TRC_64_FLAG, 1, sizeof(d), &d);
     }
 }
 
@@ -83,10 +78,7 @@ void __trace_trap_one_addr(unsigned event, unsigned long va)
         __trace_var(event, 1, sizeof(d), &d);
     }
     else
-    {
-        event |= TRC_64_FLAG;
-        __trace_var(event, 1, sizeof(va), &va);
-    }
+        __trace_var(event | TRC_64_FLAG, 1, sizeof(va), &va);
 }
 
 void __trace_trap_two_addr(unsigned event, unsigned long va1,
@@ -94,22 +86,25 @@ void __trace_trap_two_addr(unsigned event, unsigned long va1,
 {
     if ( is_pv_32bit_vcpu(current) )
     {
-        struct __packed {
-            u32 va1, va2;
-        } d;
-        d.va1=va1;
-        d.va2=va2;
+        struct {
+            uint32_t va1, va2;
+        } d = {
+            .va1 = va1,
+            .va2 = va2,
+        };
+
         __trace_var(event, 1, sizeof(d), &d);
     }
     else
     {
-        struct __packed {
-            unsigned long va1, va2;
-        } d;
-        d.va1=va1;
-        d.va2=va2;
-        event |= TRC_64_FLAG;
-        __trace_var(event, 1, sizeof(d), &d);
+        struct {
+            uint64_t va1, va2;
+        } d = {
+            .va1 = va1,
+            .va2 = va2,
+        };
+
+        __trace_var(event | TRC_64_FLAG, 1, sizeof(d), &d);
     }
 }
 
@@ -117,40 +112,30 @@ void __trace_ptwr_emulation(unsigned long addr, l1_pgentry_t npte)
 {
     unsigned long eip = guest_cpu_user_regs()->rip;
 
-    /* We have a couple of different modes to worry about:
-     * - 32-on-32: 32-bit pte, 32-bit virtual addresses
-     * - pae-on-pae, pae-on-64: 64-bit pte, 32-bit virtual addresses
-     * - 64-on-64: 64-bit pte, 64-bit virtual addresses
-     * pae-on-64 is the only one that requires extra code; in all other
-     * cases, "unsigned long" is the size of a guest virtual address.
-     */
-
     if ( is_pv_32bit_vcpu(current) )
     {
-        struct __packed {
-            l1_pgentry_t pte;
-            u32 addr, eip;
-        } d;
-        d.addr = addr;
-        d.eip = eip;
-        d.pte = npte;
+        struct {
+            uint64_t pte;
+            uint32_t addr, eip;
+        } d = {
+            .pte  = l1e_get_intpte(npte),
+            .addr = addr,
+            .eip  = eip,
+        };
 
         __trace_var(TRC_PV_PTWR_EMULATION_PAE, 1, sizeof(d), &d);
     }
     else
     {
         struct {
-            l1_pgentry_t pte;
-            unsigned long addr, eip;
-        } d;
-        unsigned event;
-
-        d.addr = addr;
-        d.eip = eip;
-        d.pte = npte;
-
-        event = TRC_PV_PTWR_EMULATION;
-        event |= TRC_64_FLAG;
-        __trace_var(event, 1/*tsc*/, sizeof(d), &d);
+            uint64_t pte;
+            uint64_t addr, rip;
+        } d = {
+            .pte  = l1e_get_intpte(npte),
+            .addr = addr,
+            .rip  = eip,
+        };
+
+        __trace_var(TRC_PV_PTWR_EMULATION | TRC_64_FLAG, 1, sizeof(d), &d);
     }
 }
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 18:12:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 18:12:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191132.341040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNme-0008AF-26; Mon, 20 Sep 2021 18:12:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191132.341040; Mon, 20 Sep 2021 18:12:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNmd-0008A8-Uh; Mon, 20 Sep 2021 18:12:51 +0000
Received: by outflank-mailman (input) for mailman id 191132;
 Mon, 20 Sep 2021 18:12:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AtBo=OK=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mSNmc-0008A2-50
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 18:12:50 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 434a2773-1b39-400d-a801-82b7f1f201a5;
 Mon, 20 Sep 2021 18:12:47 +0000 (UTC)
Received: from pps.filterd (m0246630.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18KGUZaW023314; 
 Mon, 20 Sep 2021 18:12:46 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b66gn3rvv-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Mon, 20 Sep 2021 18:12:46 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18KI1mSX089210;
 Mon, 20 Sep 2021 18:12:45 GMT
Received: from nam02-bn1-obe.outbound.protection.outlook.com
 (mail-bn1nam07lp2041.outbound.protection.outlook.com [104.47.51.41])
 by userp3030.oracle.com with ESMTP id 3b557vwxqg-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Mon, 20 Sep 2021 18:12:44 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BL0PR10MB2835.namprd10.prod.outlook.com (2603:10b6:208:31::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Mon, 20 Sep
 2021 18:12:43 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 18:12:43 +0000
Received: from [10.74.117.74] (138.3.201.10) by
 SA9PR13CA0084.namprd13.prod.outlook.com (2603:10b6:806:23::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Mon, 20 Sep 2021 18:12:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 434a2773-1b39-400d-a801-82b7f1f201a5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=zrmD74ywt/8la0rTIv+A0YDuPs5jmd98jJp1pDuLQbU=;
 b=QoqNLoywbXEhLVZ8DcWMBllhf0Iu4MAD8oN8FQJaRejoR4YKxNP6C+wE7FLARQAiIZu3
 eQ0gg3FGV+ui+Yf8o6yd2bPqQgvnAJSUHHtbZ2Rvv4XlS9VDt74YoH7SPUt/NFPpKnnC
 Lll1HYhPatydFcj90BBM2iAWq2FJOupAs2zXYAENjdTEVLPDMSoXdNsHTWIW8Y4SslS3
 U/whn6562P7uopnhZwkHAhbuEn22UDLFI7vF2acLF3cKekqZshPvMl4TweZ6YMLfKyX6
 YUbtuOlI4y6NWPbsRmcUrXrOGwDj72vTv+L4L/o+ggesPhj46JE4/zkoLa4coS+fJv2d GQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cVZSd3ggMoQJT1LPxO7NZGWGFVxU9bfxaPPu35vcvxzNXZAa2TUB611/gzF8aS3l0ipSAqDc0kYOlTy9jXRNg1J94VcHRuLqdK/XwYBr31yrRVSVEY12w6rp601UJ8cY8Q9l171SAWq5CMIo0pF7kSNCHjMRM3tX9PXvUBo79y++kkAbxfVkUErCjvE6HXHJnFqkrhjR3dHdiEHoeeyrWm78AE8ked4Ma2XnvUq7Q/6E+G41P5zbyZr3VnmDnnWAOxrcOty1ym7QKOBb0ZcDIvmhSgq0bnkZoDGeI3cCvQI48CQHaghh/F93FLfeLeoiHqYJmsklX6iqnfXk1GVgvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zrmD74ywt/8la0rTIv+A0YDuPs5jmd98jJp1pDuLQbU=;
 b=ik0/uMarPCYMN2cJGiKgV0laSihccdiTO+ZtjlVMBC0Jse1zG6akv8Tkn8H8knuDAZYvKf5905IZdWLlBWQeS+2IQUQ+16QEeCnAfgUuFwKIJz2DCn7g2RFcBg3cG34eVSZ70k56GFOrb44p8hqfqIwKm0y61WvoFfa6a2H30UaaW+zvHmdN/2131XL+waNHFtTTc5YPHqi7XqKF/5+pQdXfZAlt+/P0WciLtV/8CWF61yT204RsJ4K2ueqF48ewceXdcQQV6m9pUPi/NSWzMo8c9/JMg1FDdT/k6c15G04/66CpwlhduYGQpy0NOhoI/hV5C2oz3Psrk/p+IZQyzg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zrmD74ywt/8la0rTIv+A0YDuPs5jmd98jJp1pDuLQbU=;
 b=hh0eRdF69eHKUgehG+6yy7uC2V090fo/L9QOq5KsXwd2lfzZj2s0LH/iv4onkFC+Q0Id/uV5wxN44UQU/YHM5VPFwCSObgs+527+CwylwRkupW53slXfUvWADaasAms7HrZkdReinxkheDTXwH2FhCc+wCidfnK/04MuoeNFWsw=
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=oracle.com;
Subject: Re: [PATCH v2] xen/x86: fix PV trap handling on secondary processors
To: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <bbd4188c-32d1-7a5c-110e-18cf2b3223d4@oracle.com>
Date: Mon, 20 Sep 2021 14:12:39 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SA9PR13CA0084.namprd13.prod.outlook.com
 (2603:10b6:806:23::29) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8945a0eb-a59a-418c-610b-08d97c623d81
X-MS-TrafficTypeDiagnostic: BL0PR10MB2835:
X-Microsoft-Antispam-PRVS: 
	<BL0PR10MB283552BE51C83FEBF40AC2838AA09@BL0PR10MB2835.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	a9zEjhCgsTB7jWJmLfoJjyxN113ts5WAHLZNt0bU7OuL0ey9DwXzKYE6D1oIivxODlhu0SpDmXJc10ZVI1nuSsmtgdIeA/4q4XlwSayTUIC0yxKXXC2OJSXGLWxjcNKn49m3rMdxWQSVBemh0fFIAL3R5pq6NMFOnju1FaubxdaEZN7fG/b0P45BIZM8RTELho4+GsOByWfwNHbB+jPAB6mczKYWElX3SW5DHy1dbr/0exQSXcCTPtsE50Ijy7xJ0h3EX6/LKnZLg1LNvpfyQfy8LyHMhWtrBvkWJVmSXpn0fC+1nnoaKQSCPZwiZ33+uUMHfPCdrGrmZFDrW7TeuwZqKlMdGXVVv/ehKhvN1WcZfSrE/80HvMkFQZ8PUjJzgvnNfzo2NfzcD7NitStzjxBjGIA9JPO0z8fK8A8c5XLIoLKvxv8RNkQmv8Sm5sD3EYjeQghJcID5atJP+g794nBee3waydPGqCHJQnkj/n9X8qA3GVHlQqWaTCtshvJ7RoDnshPMzfBguVropH8QYCe6s/h+ut5CacmKCumYClVii7DJTEvFL9ajH6b4HGl6BjRchd7+szB5NQHAL+J1pILDtBo2sQ6D0oRsemPCIkAXC89NURLuosSJJ19khDrEfhewlguyh/ugYQ9tAKUX66n53b4v6UDXPwcWmj/523hCTYVfM2UyfCebuxNoCTvi0+saGvB+IkUwF970mCFag6Mw/qQ6fT5Z8P6LkrWNaY+89haLUQlurlDTd4yySWal
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(38100700002)(31696002)(36756003)(66476007)(66946007)(6486002)(31686004)(8676002)(6666004)(5660300002)(316002)(26005)(44832011)(53546011)(110136005)(54906003)(8936002)(508600001)(956004)(4326008)(86362001)(2616005)(186003)(2906002)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?SUVva3IzZXdIeXh4SDJ2MEdNQ281T1h4Mko0aHdlN0NXREN4YVU0c3JkMERF?=
 =?utf-8?B?T2wxQVJxRlJ5WDR4alhRQ2VuVk5YSzdkTzBoRkpsRmlyeU45L1NGQk12Tkt3?=
 =?utf-8?B?TFgxVU4wMkw4YkN2Z1RkcFJSbFdhdHovUkhPTVUxYkdnMXVndTh3UjR4Q0lq?=
 =?utf-8?B?c1pIcXZKYXNVVzV1bnF4TmhlQnlyMzNKRHdpNU1HeXZrL0JDaS9IRllYZGUy?=
 =?utf-8?B?V1JlekEzOVZCaklEUytKa3YrN2VkZHl4QlF2YWMzS1YzZzBlNXBTTmM2S0JD?=
 =?utf-8?B?enFNZXl5bGlZUjNxekNLTlZzanhtSFNuT05vM3ZnYWNHc0s0eHdsZmJBUUU1?=
 =?utf-8?B?WGJLNUdLSFZjaTF6R0ZwdnQweG1YZUFDOVNZSDdJN3dXZE1NWnI0QTVCamJ1?=
 =?utf-8?B?aytLVTl6S2hnWG1NanVGdVV6VnNxV2h3b0tDS2oxNGxIRTdVQUlUYzkrTTJz?=
 =?utf-8?B?WWovYmhWQkpVTlJ6STZzZk9wczkzV3EvSkJTRHFJQm0zSnFyb09obTB6NGpT?=
 =?utf-8?B?dWJHaTM1cFd6RCtQZHBjQk1PZFUrNWxyaS83b01kZ0hkSGFsSUtnOWNEUlBv?=
 =?utf-8?B?dEVFbjBCSGlSQlUrVVpvMVV4NFBieHNwSmp5MkZjTS9obnJ3QUp0dXBCZzhv?=
 =?utf-8?B?QU9xeWJxY0RBNHBKZkhUdjVBb2w0RVdGeGYzbUd2ZlV4TzUxK0xrNkZGQmFs?=
 =?utf-8?B?UkxsRURYVXNlVk9pVnR2NWMrNzRvU255YnpYMGpoOEtLMGJlMkV2L1V0bGlj?=
 =?utf-8?B?V2lid2NlV3dPZm5kVW1Xd01BaWtEbHo1VE9XWjRLVWdYVmxZcVU4L0IxUGRn?=
 =?utf-8?B?WUtyTDNIVnpjNkhpMmZtM0tSWDhrSDM4dEFGbVpyUlZWdEdXbVVQZ3F1Nmd3?=
 =?utf-8?B?L1NMRVJ1TWxpMUdtbkcvQzhKeGtsNmJFV3R0YUgrRXgzcmZhRUVXUnU4eGJ5?=
 =?utf-8?B?QjBjSjlIYnFsdFlLbXRNaVFIZkVuRG9Ebkx3eGlOdlgySHFMaGZpRmxGYndO?=
 =?utf-8?B?TTRocVVUdXNnTFlzdjQ0eFBPemRyVHJ2Rm05bVExWFNqaHdrbHhnNWJMY3Rn?=
 =?utf-8?B?VW1YMTR3U05xYUIrZkdRV0Uxb29nd0ZKWnd6V2ZWU3JWYVlxM216Z2VldzR2?=
 =?utf-8?B?N1BoWlQraUF3clBTWDE4RDZDMDFIRGZJNzY2MHFDTFhkUUV6ZCt0V3hBREFL?=
 =?utf-8?B?V1Z4SUpaS0RpeEtvVlN3TENzTnFIeWhnZnhPM0lQbXN5cHdOQVU5LzVYc3No?=
 =?utf-8?B?SUM3T1B4NGV2OGxKTENrRkNZL0pZZzdBRDBwOUJiMFMrLzZ6aEpsZW1BZHM5?=
 =?utf-8?B?cGpsU0QwT3ZCQnRPamNVSXl0ajlXemV3dEdiT0VvVVAyTVUrTjU2ODJNMnFN?=
 =?utf-8?B?TnQxYkdsZTVQLyt3a0t3cTZYM1AzR21TZ1lUb2tGMG93MHlHN2tlSWxic3E5?=
 =?utf-8?B?WmU0b1o2N3FEN2w3Qi9vR2NPVmJJaUk0L0cxaW9YTm5RN3Y5VTdZbVpQRUda?=
 =?utf-8?B?MnVOU1c5Y3R2WTBieVA1eC91TlErUmtiaFZDRmdQMHNhME8rbEFCeHh4VXdR?=
 =?utf-8?B?L0Vha09EUlNkQVBhZFp2T0pEZHIxem9OeE1kVGxmTWJCVGd4U2w4cEI2bmMy?=
 =?utf-8?B?dGZCVTR4amN4QVhGZXZ5QWp6ajZ1bm5pRy9xMERxRjVWUjVrUWxsVldnYkMr?=
 =?utf-8?B?aXFPa3I4aW5qdWE2d2lNZDlXd3dxMUJ1L1lJVlhSRFJLWDdiSWkzN3ZDVkxB?=
 =?utf-8?Q?5NsBnYEZwZNdaTyKDoeJFLWyx+LvtJWBxFPQfu9?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8945a0eb-a59a-418c-610b-08d97c623d81
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 18:12:43.1787
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Lma5l6P12UIm9ENV18O7DJH6xNS/YSFjh8dGwdilD5d63i8LxBZOCF3w1IrcjEdf36eUhWNqjvURaSFA/uvxHmS0Esmr0CHn3XPmS1hCGkA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB2835
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10113 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 suspectscore=0 spamscore=0
 bulkscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001
 definitions=main-2109200108
X-Proofpoint-GUID: ALKlUnr9OjrLwS62HuMtlXehF7Ooipoy
X-Proofpoint-ORIG-GUID: ALKlUnr9OjrLwS62HuMtlXehF7Ooipoy


On 9/20/21 12:15 PM, Jan Beulich wrote:
> The initial observation was that in PV mode under Xen 32-bit user space
> didn't work anymore. Attempts of system calls ended in #GP(0x402). All
> of the sudden the vector 0x80 handler was not in place anymore. As it
> turns out up to 5.13 redundant initialization did occur: Once from
> cpu_initialize_context() (through its VCPUOP_initialise hypercall) and a
> 2nd time while each CPU was brought fully up. This 2nd initialization is
> now gone, uncovering that the 1st one was flawed: Unlike for the
> set_trap_table hypercall, a full virtual IDT needs to be specified here;
> the "vector" fields of the individual entries are of no interest. With
> many (kernel) IDT entries still(?) (i.e. at that point at least) empty,
> the syscall vector 0x80 ended up in slot 0x20 of the virtual IDT, thus
> becoming the domain's handler for vector 0x20.
>
> Make xen_convert_trap_info() fit for either purpose, leveraging the fact
> that on the xen_copy_trap_info() path the table starts out zero-filled.
> This includes moving out the writing of the sentinel, which would also
> have lead to a buffer overrun in the xen_copy_trap_info() case if all
> (kernel) IDT entries were populated. Convert the writing of the sentinel
> to clearing of the entire table entry rather than just the address
> field.
>
> (I didn't bother trying to identify the commit which uncovered the issue
> in 5.14; the commit named below is the one which actually introduced the
> bad code.)
>
> Fixes: f87e4cac4f4e ("xen: SMP guest support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Jan Beulich <jbeulich@suse.com>


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 18:23:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 18:23:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191138.341050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNwR-0001Pv-06; Mon, 20 Sep 2021 18:22:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191138.341050; Mon, 20 Sep 2021 18:22:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSNwQ-0001Po-TP; Mon, 20 Sep 2021 18:22:58 +0000
Received: by outflank-mailman (input) for mailman id 191138;
 Mon, 20 Sep 2021 18:22:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rsqB=OK=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSNwP-0001Pi-Jr
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 18:22:57 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 10e7496c-0347-4098-a46e-409bae5ae674;
 Mon, 20 Sep 2021 18:22:56 +0000 (UTC)
Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com
 [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-539-zimdzyCeOMOkOnJnx38K6g-1; Mon, 20 Sep 2021 14:22:55 -0400
Received: by mail-ed1-f69.google.com with SMTP id
 n5-20020a05640206c500b003cf53f7cef2so13744980edy.12
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 11:22:55 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id w25sm7281095edi.22.2021.09.20.11.22.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 Sep 2021 11:22:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10e7496c-0347-4098-a46e-409bae5ae674
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632162176;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vRI2avQuBNPO4mZTVFj4yY0DXqukEShEdAk6taZEnJk=;
	b=cV+rEkxVVpi8hgrpFdw8Oj9BL4m66kM4kVd3EyEFhjQLUr5rb3Se5XI9amNgg5bZPN8AS6
	GW5HMHgVscLcHGQERaRfgeuELNomHJxNu4lXg/s2P6om1IgbiZLDJ1/wtAussDC3xFrSST
	BxwWvG2BpeXuEYebkdWe/9JMTYiKnUg=
X-MC-Unique: zimdzyCeOMOkOnJnx38K6g-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:to:cc:references:from:subject:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=vRI2avQuBNPO4mZTVFj4yY0DXqukEShEdAk6taZEnJk=;
        b=5ah5ye+2t01gTfFgATI3IMYdsl+Xg4uMYAXXiz2YXpZcIa3BTpCzucpTFbJDGc3EhM
         zgBHy9u21B+qyEtNnZKFS/ZL2UfRTo2ECysTwpLYBe8hzEI78k0tMHiCXSIdI4SziAwY
         AQXK42dDBV3Iqycwz0H8JOhFrXPH0ZzyJdq1Kz8NKLhhZOJ20/Ldt3Tw1wcnLqSPniZW
         RvERAJURXzwx+ou1g2Pn4b8yw5kXVei80tZEFzao8aOAoTcYyYs+MiSunpsIp6/sFyP1
         gB/OrKFO91nO5MB6cSRIYEBqA0xS+yWWmetTCfMTJLdJbCGJ0GpWWmfXNS00xWzdQUIr
         Bg8g==
X-Gm-Message-State: AOAM533OEo99p6UBfccgxAQUasV3KmXXnKcsu45JiL2y1cZ2FvWFzKCx
	zx0tB/ROFvllCbukMeLJPZ6kvGUaTlsEdPGaNHJDjVf56+NRYK5vrGmLUUAgu9Orvm3/pIncwIm
	pB9rcmPAgZciyIWCv4JgWCyXaU5A=
X-Received: by 2002:a17:907:76b2:: with SMTP id jw18mr31101793ejc.120.1632162173786;
        Mon, 20 Sep 2021 11:22:53 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzSoRFsPRM/7K22YQ1jPmATj4IuvLfgFOqCYFtPnEIPU80lrUpqlQ2FpuvfaQh3tH7XvhJmFg==
X-Received: by 2002:a17:907:76b2:: with SMTP id jw18mr31101728ejc.120.1632162173428;
        Mon, 20 Sep 2021 11:22:53 -0700 (PDT)
To: Marc Zyngier <maz@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
 Sean Christopherson <seanjc@google.com>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>,
 Catalin Marinas <catalin.marinas@arm.com>, Guo Ren <guoren@kernel.org>,
 Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
 Vincent Chen <deanbo422@gmail.com>, Paul Walmsley
 <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 Albert Ou <aou@eecs.berkeley.edu>, Thomas Gleixner <tglx@linutronix.de>,
 Borislav Petkov <bp@alien8.de>, x86@kernel.org,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>, "H. Peter Anvin" <hpa@zytor.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828201336.GD4353@worktop.programming.kicks-ass.net>
 <YUO5J/jTMa2KGbsq@google.com>
 <YURDqVZ1UXKCiKPV@hirez.programming.kicks-ass.net>
 <662e93f9-e858-689d-d203-742731ecad2c@redhat.com>
 <87tuifv3mb.wl-maz@kernel.org>
 <7a5825d1-d6e9-8ac8-5df2-cce693525da7@redhat.com>
 <87o88nuzzc.wl-maz@kernel.org>
From: Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH v2 00/13] perf: KVM: Fix, optimize, and clean up callbacks
Message-ID: <0eb2d4b1-23a4-c318-9f91-8dce78c6c8ad@redhat.com>
Date: Mon, 20 Sep 2021 20:22:50 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <87o88nuzzc.wl-maz@kernel.org>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 20/09/21 15:40, Marc Zyngier wrote:
>>> At least not before we
>>> declare the arm64 single kernel image policy to be obsolete.
>>
>> --verbose please.:)   I am sure you're right, but I don't understand
>> the link between the two.
>
> To start making KVM/arm64 modular, you'd have to build it such as
> there is no support for the nVHE hypervisor anymore. Which would mean
> two different configs (one that can only work with VHE, and one for
> the rest) and contradicts the current single kernel image policy.

Ah okay, I interpreted the policy as "it's possible to build a single 
kernel image but it would be possible to build an image for a subset of 
the features as well".

In that case you could have one config that can work either with or 
without VHE (and supports y/n) and one config that can only work with 
VHE (and supports y/m/n).  The code to enter VHE EL2 would of course 
always be builtin.

> It is bad enough that we have to support 3 sets of page sizes.
> Doubling the validation space for the sake of being able to unload KVM
> seems a dubious prospect.

It's not even a configuration that matches kconfig very well, since it 
does have a way to build something *only as a module*, but not a way to 
build something only as built-in.

That said, if you had the possibility to unload/reload KVM, you'll 
quickly become unable to live without it. :)

Paolo



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 19:30:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 19:30:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191145.341062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSOzI-00087n-TI; Mon, 20 Sep 2021 19:30:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191145.341062; Mon, 20 Sep 2021 19:30:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSOzI-00087g-Q4; Mon, 20 Sep 2021 19:30:00 +0000
Received: by outflank-mailman (input) for mailman id 191145;
 Mon, 20 Sep 2021 19:29:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSOzH-00087a-7b
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 19:29:59 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 23a86384-1a49-11ec-b881-12813bfff9fa;
 Mon, 20 Sep 2021 19:29:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23a86384-1a49-11ec-b881-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632166197;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=+TlaTCIqGu8j0bYpxe1cgsh+qhECUrKtrVKQp6Z2N0o=;
  b=QZkOXM4NAEHLBw7r9wuHZ7UveyJWhxI/2im/B11QFiiJbupRZFavtBdC
   SXrew94f6MjmhYJxlScM/6Ofr7GU5qbOAyUYnsGCymck4wqeNpqsf8mDx
   R/smI3Iwpte9JDfaHCbA6h1jmRTmFL0eWOOqHh5hR2SaVoSGaghVHwPV1
   0=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 5c/xeiQ5NPwPiOWQO4WXo1wfogul/PN0wI86yPDPZyqgwApt3jFFtRbrwhPA8EjMPz6J3W1ZCU
 y8wSlQ2rgHRvGfaP8S6NVqfSPMo227cNmMKLFHY2fJeRQZocgPPaO0yyPqmvQMlLL/JWtqhqay
 AWdysyG/X9pL+ALbNnCYZsazWhF7qgMZkJXlLFpwi6Pgkq5kVazQWy195yEitjI9ARZ4+aNIit
 Y2PziFeENV3Yw8H/2CKegugLlmIPRLoT2gBNQnyDV0uYBhBiX0UUAa0H6BefWWUgwDyCkwwast
 PekvgmkSNoZDtE+rEQXESWLB
X-SBRS: 5.1
X-MesageID: 52750854
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:4lRU1K2taxFwZqDMgPbD5Qx2kn2cJEfYwER7XKvMYLTBsI5bp2dTm
 DAYDWuGOvvYNGSgc4wlPd6z9EgDscXQnNZqG1NlpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbZh2eaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhpd11l
 pJc9qaKWyA4OYLzm8U9VBl4DHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t3J8TRaaDO
 6L1bxI/XBHSTgdqeW48BYkajMalg3THQh5h/Qf9Sa0fvDGIkV0ZPKLWGMHOZtWASMFRn0CZj
 mHL5WL0BlcdLtP34SWB2mKhgKnIhyyTcIAPELy18NZ6jVvVwXYcYDUGWF3+rfSnh0qWX9NEN
 1dS6icotbI19kGgUp/6RRLQiH+bvDYMVtxICeo45QqRjK3O7G6xJEIJUzpAY9wOr9ItSHoh0
 Vrhoj/yLWUx6vvPEyvbr+rK62PpUcQIEYMcTRUJSVVC8prZmpMUnxXgcdVCMKm1rsKgTFkc3
 Au2QDgCa6Q71JBQjvXgrAGf2VpAtbCSEVVkvVy/snaNq1ojPd/7PdTABU3zsK4YRLt1WGVtq
 5TtdyK21+kIEZjFvyiEWuxl8FqBtqvdbWG0bbKCGfAcG9WRF5yLJts4DNJWfh4B3iM4ldjBO
 hS7hO+pzMUPVEZGlIcuC25LNyjP8UQHPY+/Ps04k/IUOsQhHON51Hg2OCZ8IFwBYGBzyPpia
 P93gO6HDGoACLQP8dZFb75GitcWKtQF7TqLH/jTlk3/uZLHPSL9YepVYTOmM7FihIvZ8Vq9z
 jqqH5bTo/mpeLalOXe/HE96BQ1iEEXX8ris+pQLKbbcclQ5cIzjYteIqY4cl0Vet/w9vo/1E
 ruVAxMwJIPXiSKVJAOURGpkbb+zD59zoWhiZX4nPEqy2mhlaoGqtf9Ne5wydLgh1epi0f8rE
 KVVJ5TeWqxCGmbd5jAQTZjht4g+Jh6lsh2DYni+az8lcp8+GwGQoo34fhHi/TUlBzassZdsu
 KWp0w7WGMJRRwlrAMvMRuioyle94SoUlO5oBhOaKdhPYkT8toNtLnWp3PMwJsgNLzTFxyebi
 FnKUUtJ+7GVrtZsotfThK2Co4O4KMdEHxJXTzvB8LK7FSjG5W7/k4VOZ/mFIGLGX2Tu9aT8O
 egMl6PgMOcKlUphupZnF+o51ro34tbiquMIzglgG3mXPV2nBqk5fyuD1MhL8KZM2qVYqU29X
 UfWootWPrCAOcXEFl8NJVV6MrTfhK9MwjSCv+4oJEja5TNs+OvVWEpfCBCAlShBIeYnK4gi2
 +og5JYb5gHXZsDG6TpaYvS4L1ixE0E=
IronPort-HdrOrdr: A9a23:czrrI62UQLhBOomIE6s/pQqjBLYkLtp133Aq2lEZdPRUGvb4qy
 nOpoVi6faaskdzZJhNo7+90ey7MBfhHP1OkPAs1NWZLWvbUQKTRekIh+aP/9SjIVyYygc079
 YHT0EUMr3N5DZB4/oSmDPIduod/A==
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="52750854"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>
Subject: [PATCH v2.1 13/12] xen/trace: Introduce new API
Date: Mon, 20 Sep 2021 20:29:39 +0100
Message-ID: <20210920192939.16483-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

This will be used to clean up mess of macros which exists throughout the
codebase.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

v2.1:
 * New
---
 xen/include/xen/trace.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/xen/include/xen/trace.h b/xen/include/xen/trace.h
index 055883287e8c..72c20550f6a6 100644
--- a/xen/include/xen/trace.h
+++ b/xen/include/xen/trace.h
@@ -74,6 +74,30 @@ static inline void __trace_hypercall(uint32_t event, unsigned long op,
                                      const xen_ulong_t *args) {}
 #endif /* CONFIG_TRACEBUFFER */
 
+/*
+ * Create a trace record, packaging up to 7 additional parameters into a
+ * uint32_t array.
+ */
+#define TRACE_INTERNAL(_e, _c, ...)                                     \
+    do {                                                                \
+        if ( unlikely(tb_init_done) )                                   \
+        {                                                               \
+            uint32_t _d[] = { __VA_ARGS__ };                            \
+            BUILD_BUG_ON(ARRAY_SIZE(_d) > TRACE_EXTRA_MAX);             \
+            __trace_var(_e, _c, sizeof(_d), sizeof(_d) ? _d : NULL);    \
+        }                                                               \
+    } while ( 0 )
+
+/* Split a uint64_t into two adjacent uint32_t's for a trace record. */
+#define TRACE_PARAM64(p)    (uint32_t)(p), ((p) >> 32)
+
+/* Create a trace record with time included. */
+#define TRACE_TIME(_e, ...) TRACE_INTERNAL(_e, true,  ##__VA_ARGS__)
+
+/* Create a trace record with no time included. */
+#define TRACE(_e, ...)      TRACE_INTERNAL(_e, false, ##__VA_ARGS__)
+
+
 /* Convenience macros for calling the trace function. */
 #define TRACE_0D(_e)                            \
     do {                                        \
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 19:30:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 19:30:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191146.341073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSOzO-0000cb-5m; Mon, 20 Sep 2021 19:30:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191146.341073; Mon, 20 Sep 2021 19:30:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSOzO-0000cU-2Q; Mon, 20 Sep 2021 19:30:06 +0000
Received: by outflank-mailman (input) for mailman id 191146;
 Mon, 20 Sep 2021 19:30:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AtBo=OK=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mSOzM-0000Ji-3m
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 19:30:04 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e4031643-1cdd-44e6-86c4-3dae62ba97a0;
 Mon, 20 Sep 2021 19:30:01 +0000 (UTC)
Received: from pps.filterd (m0246627.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18KI44sm026691; 
 Mon, 20 Sep 2021 19:30:00 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b6426cf7q-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Mon, 20 Sep 2021 19:30:00 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18KJJuGo041061;
 Mon, 20 Sep 2021 19:29:59 GMT
Received: from nam11-bn8-obe.outbound.protection.outlook.com
 (mail-bn8nam11lp2172.outbound.protection.outlook.com [104.47.58.172])
 by userp3020.oracle.com with ESMTP id 3b5svr5n48-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Mon, 20 Sep 2021 19:29:59 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4160.namprd10.prod.outlook.com (2603:10b6:208:1df::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep
 2021 19:29:57 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%7]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021
 19:29:57 +0000
Received: from [10.74.117.74] (138.3.201.10) by
 SA9PR13CA0133.namprd13.prod.outlook.com (2603:10b6:806:27::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Mon, 20 Sep 2021 19:29:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4031643-1cdd-44e6-86c4-3dae62ba97a0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=twx/poF39noCLQs4aAkBnBA9IKpiTltwMBDfusJ7u+c=;
 b=Js04bF9R3n3f8X65KxGtwf4q/4ZEqYrv8MMsZsSw+x+JqYc813FVQVqaOCEExmkJXH3U
 n9y2sCV9L0sd3VCc7LdeqVGYJ88cpnQFEgfyIA68f7f+r5euO2TIBvtLtH9BrGLS0oCf
 c5I7ynDxMnUU15VBm7kSF4aWkC+VGau7i2iHgVOMIbFkqS85RpGz2TlaJJ6GTL8lvvm7
 vVhsbdQK4yf04yXvyTGQnkAsMdoJpkMYrCRLELMx+RysXZDiu56VsJHf3VwHsIf7uvCb
 Cjj2IdtDTI6wDas6uJN0ZOD0blGpiLq6grIDu6Ri03IDK1Ssc6s3Lf8KKDQog+cyEbxq uA== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YTN0oDPsNO7zVLU+nh6/JcEc65bV5TopVBBna/TYRUNGPW67FV5mw6EnBUrp3A3qAFOLpyjLXHaFF0ReA2/Za87MG/3Ita8kmAuXo79am225QvkiOXHXxBikOlDQRrYyNbv0L482LMjRdcinN7RHeedKRblHhzN/gN8oQpZsipCFEuYWumAj7WknN/uU6ueC73piOQc/HOfTzXJsY2KuqRKWHzB/s8FXPqn8xn03J8+L2Oy1ICJv86ZnNcfZ1zV+/SUa5J2dkwJsXXqiOWkQZ/KdIJ6W7kuQawKkeVBkDxWFMRV/bBukDhzTo0dkwRy6/g6nqygDFBeBU+xhRFCFcw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=twx/poF39noCLQs4aAkBnBA9IKpiTltwMBDfusJ7u+c=;
 b=gkhp67FtZoNaPT4fn/MMbonagAbedxFeqZsd9voWTHVDpJ5EDlfkogf9OysWGlcHnZVrgDAaCAN5URQx64hUqIBemksnTyXAwyWD4Ldszh7feSOaLaYgrBNC0EHo+W3kapGrex8tFJdDEQYpRTq5E0iH3faiD54/0suoJ1YB0JjPsOaZT7qobXSX3rm/IzdEOiOk5QwBcvPK1uBfn4O4fY1sGb/8U0PJHpxKj88Tx20ci9JDmoZYRL5YOP0wXehGJD1a5k8fUMVj88mLrheQPZVBt8oBFCUlrOng3A2M0VTc1gONJAcmNS00jZvYnPjkj5Jk5BRKuWGHLZFq5M6fYQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=twx/poF39noCLQs4aAkBnBA9IKpiTltwMBDfusJ7u+c=;
 b=b49yOdN+VEcegujH0+nSWKIWshjqD1Eelb6s51Q4Z2BBlyIrMGfQxU057gh6jV+cKRtT5QX4io98v/6tlzZigQNb95vSdgtYBpzyY7TBscTfrCqNna5daeH0Gvhou7xxtBON9cCTljUykV1rgVJ/NDRTOMkwUY9BjOBVxm1kpgg=
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH] xen/balloon: fix balloon kthread freezing
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        linux-kernel@vger.kernel.org
Cc: Stefano Stabellini <sstabellini@kernel.org>
References: <20210920100345.21939-1-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <1b13768c-d2ba-196b-c3ea-8f9375bb94b8@oracle.com>
Date: Mon, 20 Sep 2021 15:29:53 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210920100345.21939-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SA9PR13CA0133.namprd13.prod.outlook.com
 (2603:10b6:806:27::18) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4a317567-b3c4-4ffa-5855-08d97c6d07b6
X-MS-TrafficTypeDiagnostic: MN2PR10MB4160:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB41606534071285001ABD41B98AA09@MN2PR10MB4160.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:318;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	RjZqus4HPjkDDRmxbWY8tgtBpAeGNEODsiCwh6urgqWVGZ4QAoG+veaG0ReisTU4mIVzhy8Nkp6ZIwg25vk4P3zvumMfevCXlTS5ZE5Ck+FDlWQL34/9nO3TKxTkHB3aNii0Oktkq7ESAXOy+/3QZaaFVJ7zW1rX6onPF/Pb7oBxLUnm5yrB/uqwSG/krNaUiiPkpV5xFPGGiHpnRqFDxLJ6vtKPcgwOa+eM7wvbMFFNzndcc+S2qb1eSLojYLmzCpkiBVkCkSZ32XZZbHq69fdNGmNwLqvjd/99OGZESfHnEiNjwSbi7+5gPfPKxqrEdScLggpYK7qC9qBd2fF6B3v/e702hiIjR6suztVr8mHBmfzjAh45aDqkHO22/1H06A70y5L4fF+HLyPfSEkCsfKwuPfM/91DFhEZUfXUbzBd2e9khElH8oDXFAclXOcAjC/9rieygChWW7lLn1/yqo6XIJBB2ZKABt+MJFIp8LqM0jtgLXkRJUeAAN0WwG4APXSUzj9Lsrrd6/1gUekxeeS77lWkubNs9XdHODS6b1WfmL9hXCYeaeMWQQAjeixq9rRcfOijTCbumlIuSAuWJctUOR/nxlLR5s8dcoug0Igs+wH0aGC8ugu8by636i7TU0CJTDasxAUpqRrnKG9vUi9NWO9JqXSw2x2SL9a4HoEKFN3KIWQLqY++wLu29GK+valxHEgnfR+doJpe71nrqD3KIGOQVncz64VqiVbugkEMmEnw7U3vO7Zkg1yE+Gja
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(376002)(39860400002)(396003)(346002)(478600001)(44832011)(31686004)(36756003)(5660300002)(186003)(6666004)(66476007)(66556008)(2616005)(956004)(2906002)(6486002)(31696002)(4744005)(53546011)(4326008)(66946007)(316002)(86362001)(16576012)(8936002)(83380400001)(8676002)(38100700002)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?em5GU2FHMzIrUWFkM3cxamhhUjN1TmxCQW0xelpNOU9aQWdJNHhqMytQVGZP?=
 =?utf-8?B?c3VDV0JPK3pPeEtmVkZFNTVCUmdnamZ6ZlRPZkF4bmVmSVVBaTl5YkxSbXlX?=
 =?utf-8?B?eFNKNkhDejZub25kT0xLcWVJZUxKNUdmRW9lSXk5a1Qxd01HQm1HQUw2NHFx?=
 =?utf-8?B?MThXRHVZNEVPV1lGQWFOK28rRjZ3SWhnTjdKNjhFZlN0ZzhMd1B3am1Ld1Fz?=
 =?utf-8?B?RHFrVlZJZHUxbysxNjgrajNySnVSaDFXZGsyTlBza2JTTDB4Q0FLOHJOYWhU?=
 =?utf-8?B?akg5UUpVRWtCem1XOXpLR0ptMDdJVmY5SWNWellyaURaLy9NdXFjRmU3T0RR?=
 =?utf-8?B?bHFpOGNJYVBiYW1zdG5Td0xMeUxncWdBMERnYTJZdnY1dmgvb0hCUWlBblg3?=
 =?utf-8?B?Mmh3SXB1dElYOGh3dXZqbko1eUMzU0N6SWQzd3ZNMjJRT2lLRTBlM3h2ZU9I?=
 =?utf-8?B?K0doaDQ5UzhRV0h0VjVkeEoyM3BaQmdlNXNzNG81eEJzanJYSWx2QlBGbHlP?=
 =?utf-8?B?TDBJMHZZN1dxMnRqWFVyUURQamxaVElnNVpRRWtwdmwrUnlaTG43emJMOFZr?=
 =?utf-8?B?dURnRS9pYVVCd1dHWXZIckxId0pwOGozVnlwaWk1emRRTi84UHlrcHRaTTJh?=
 =?utf-8?B?Y3B0Z1ovWGVtR1I2VHpxMHpaOVRsSTBMRk9IcjRuL29oMUVtOTl1am1Ea3oz?=
 =?utf-8?B?YUNKVDVSQXBZZ1lzQnNlbHV0a3ZlSjVEQTVpcjNBTVhsREh4QzBZb2tqUDVP?=
 =?utf-8?B?cTAvUkZKRXNFT0VOdEhCWXhjZHZJRVdreE94L0YwQlQ4ZExwdFQrRVYzVWVv?=
 =?utf-8?B?QWxlRmNnd1ZKMHdKR2JSNit5ZmVUa01WMUdJZlpUVjRmZ3o3VEpzQ3ZsNi93?=
 =?utf-8?B?QlpNZGZacnRKNGVZQVhSNVA2ZlBRdVhSdTgzU2FOYkZkbllSakI3NHBwUUlO?=
 =?utf-8?B?UEwzcGtYK3JIbkNzQ2tiamMzamIxaUZPdUxLcEFES2RmSDE4Z3VQS08yQnNs?=
 =?utf-8?B?ZDZwM2tjYjN1eEJLeWRVZ0d3TlBzWjh4YXAwVFJQbkRTdU13c1ZPeGh6aytE?=
 =?utf-8?B?eG1HODJmQXM4QXM5QnNUaTVxeGNtbXpZL0c2L3YwREdxMGRhK3IvWTVQT1Zq?=
 =?utf-8?B?aS9ySUlNVWdnR2diYUpOT2loM3ltVkNITnNNWXAvV2R5NmpmOTMrMWFRSkhz?=
 =?utf-8?B?NHd5ZVIyaWRVVGlhd2FXWUlJTmJJcEVjNXJLcVROYmNWQnJPYkdiVnJGSDZ2?=
 =?utf-8?B?SlhPdXpBZU1MUHlRWnJFcVRPblo0NG8wTldTTkwrYk1kWmtUVzFtUG4wK1NK?=
 =?utf-8?B?RjhhdDh1MWJYRC9RRTNDWUN6ZEpSNmRhalpwMnM0NHZUd2tHMEk4V25VeFd3?=
 =?utf-8?B?YWFEckdNM3h3dDUrc0RwdFVqREUyQ014d1ppYTlLMkNRc2ZxdVRJaGNCcXkx?=
 =?utf-8?B?SGFmaWNQMWNPRkZIUThZeW5CN2I4UWZMRWhsWm8wK1ZPeDBUSnVXVFAveVB2?=
 =?utf-8?B?dWk5SEtuNEZKUFc4ZFJHYm1rUFF3NDRvNDZJTTg5U0IwMU94cW9zZEIzNDVy?=
 =?utf-8?B?RDFTaVFhVGFDamFaYmcvMUQ4RW9MMU5YQ2tja2FiREpSdGJzVE4ySFp2T1hl?=
 =?utf-8?B?N3lSc2prRUJTVFFUS2JtTmFLNW1hUWFCUFRpcEgrMkt6aHBwZnpjL2RSSk12?=
 =?utf-8?B?dnIvUW1JSWhXUldXbXhvbGRadTQwSitMMkdMMFNrQngzM3dnSHorS3RuNDZK?=
 =?utf-8?Q?BShtY4oQ8T7qNXNUgKTF+jq06CEThc/FAZoJn8k?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a317567-b3c4-4ffa-5855-08d97c6d07b6
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 19:29:57.4028
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HJALA8IrJeEP53x88yiDwtcvML5aS0XQ1KtPW72iZt+6llzTqfuwg/A43XDsBczHwfShaimBt2tAnuNKm3o5dciCLKRtdUBhlfAkX45famw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4160
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10113 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0
 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109200112
X-Proofpoint-ORIG-GUID: MuX1xbbT9VFEqIxjO6dnjIflt9JJNKMR
X-Proofpoint-GUID: MuX1xbbT9VFEqIxjO6dnjIflt9JJNKMR


On 9/20/21 6:03 AM, Juergen Gross wrote:
> Commit 8480ed9c2bbd56 ("xen/balloon: use a kernel thread instead a
> workqueue") switched the Xen balloon driver to use a kernel thread.
> Unfortunately the patch omitted to call try_to_freeze() or to use
> wait_event_freezable_timeout(), causing a system suspend to fail.
>
> Fixes: 8480ed9c2bbd56 ("xen/balloon: use a kernel thread instead a workqueue")
> Signed-off-by: Juergen Gross <jgross@suse.com>



Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 19:32:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 19:32:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191158.341084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP21-0001fv-Nb; Mon, 20 Sep 2021 19:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191158.341084; Mon, 20 Sep 2021 19:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP21-0001fo-Jb; Mon, 20 Sep 2021 19:32:49 +0000
Received: by outflank-mailman (input) for mailman id 191158;
 Mon, 20 Sep 2021 19:32:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSP20-0001fg-Td
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 19:32:48 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 87813f0c-1a49-11ec-b881-12813bfff9fa;
 Mon, 20 Sep 2021 19:32:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87813f0c-1a49-11ec-b881-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632166365;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=KHngWxU5NGMuCF7kJ9kXOWLSo5HfGZpL7wbVlWdiHGs=;
  b=HhmC59QoWoS+0IrCTn0l+OufQSvdRKRjrr+7n3hjqPF8ECsrZRS5QXbT
   yerPnhIJJl30oc+dVJZ7oQyRftD7qzeDFJZb4C4EyeBImhsMiZCjDZJud
   +nDRaa2p0U0W73LN4lI1WbGqxonbANqYEX16meks6x3EvKC1YrbxmQIIk
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: pi1s2iXKKe6xDaf4/X8woomz9od6Bo+reevhKRa8KloSU62lyelURCQEOxNRez0SrqnxNnyfOO
 EWuDLbaVVcsmdEOv4cUmVsAMSLM6Vipx9E+T6OL48+1mmCz9VZzw2poWxgDRhwukluQDcR2WC9
 U+29uH840FAGd8GmTPjEiyPlv8aJ2UHblcUVvWO0aLN43T/wGeqI/r7e7gH8L7hSnqLY8IfrUP
 hCtxZZBWJ7CQ7P6LQvqGu4su4ErOFL+V10k2l5kAKhT3sduKv3hwadz6NcK3kjw7vt9PcgxRms
 hqPHXJsBTtz5uhjm6hS6ilae
X-SBRS: 5.1
X-MesageID: 54949765
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KFRwu6jDGXmH/etxcr1NOqxKX161vBcKZh0ujC45NGQN5FlHY01je
 htvWWCDMviNMWf2f9sna4SyoEsF757QyYJrTwturH9mRHgb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y2YDia++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1vuoS0SB0YA5TDo/sPeQRnSCN5Za1ZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHCOo8Ft24m5jbeFfs8GrjIQrnQ5M8e1zA17ixLNauCN
 pZINmQyBPjGSxNmP0YyLpcUp+KXn0vPImRR8l+Oh5NitgA/yyQuieOwYbI5YOeiXt5Jl0yVo
 mbH+WXRARwAMtGbjz2f/RqEmevnjS79HoUIG9WQ9ONugVCV7nweDlsRT1TTiem0jAuyVsxSL
 2QQ+zEytu4i+UqzVN7/Uhak5nmesXY0WsVSO/037hmXzajZ6BrfAXILJhZjQtE7sM49RRQxy
 0SE2djuAFRSXKa9ECzHsO3O9HXrZHZTfTRqiTI4oRUtwdb7pIApvy/2Qe18AoSosYb3Hmnw3
 GXfxMQhvIn/nfLnxo3iowue02n998CTJuImzl6IBTP+t2uVcKbgPtbxuAaBtZ6sOa7EFgHpg
 ZQSpySJAAni57m2nSqRSa0mGLiz7p5p2xWN3AYyQ/HNG9mrkkNPnLy8Ahklfy+F0e5eIFcFh
 XM/XisLv/du0IOCN/MfXm5II51CIVLc+THZuhf8NYAmX3SMXFXfoHEGibC4hjiw+KTTrU3PE
 cjCKpv9ZZrrIY9m0CC3V48gPUwDn3tlrV4/savTlkz9uZLHPSb9Ye5cbDOmM7BohIvZ8V692
 4sOaKO3J+B3DbSWjt//qtVIczjn7BETWPjLliCgXrTSelY5QD5xVaC5LHFIU9UNopm5X9zgp
 hmVMnK0AnKm7ZEeAQnVOH1ldp31WpNz8SAyMSA2ZA760Hk/e4e/qqwYcsJvL7Ug8eViy99yT
 uUEJJrcUqgeFGyf9mRPd4T5oaxjaA+v2VCEMR26bWVtZJVnXQHIpIPpJ1O96CkUAyOrnsIiu
 Ln8hBjDSJ8OSl06XsbbYf6i1X2run0ZlL4gVkfEOIALKk7t7JJrO2r6ifpue5MALhDKxz270
 QeKAEhH+bmR8tFtqNSQ3PKKtYakFed6D3F2JWiD4ObkLzTe80qi3ZREDLSCcwfCWT6m466lf
 +hUka3xaaVVgFZQvoNgOL931qZitcD3rrpXwwk4TnXGa1OnVuFpLnWchJQds6RMwvlSuBesW
 1LJ8d5fYO3bNMTgGV8XBQwkcuXciq1ExmiMtaw4cBfg+St63LubSkEDbRCDhRtUIKZxLI54k
 /wqv9Qb6lDnhxcnWjpcYvu4K4hYwqQ8bpga
IronPort-HdrOrdr: A9a23:fEdAEa01z0WvkSNGVUST+AqjBLAkLtp133Aq2lEZdPRUGvb3qy
 nOpoVj6faaslYssR0b9exofZPwJE80lqQFh7X5X43SPzUO0VHAROoJgLcKgQeQfxEWntQtsp
 uIGJIeNDSfNzdHZL7BkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="54949765"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>
Subject: [PATCH v2.1 14/12] xen: Switch to new TRACE() API
Date: Mon, 20 Sep 2021 20:32:34 +0100
Message-ID: <20210920193234.19409-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

(Almost) no functional change.

irq_move_cleanup_interrupt() changes two smp_processor_id() calls to the 'me'
local variable which manifests as a minor code improvement.  All other
differences in the compiled binary are to do with line numbers changing.

Some conversion notes:
 * HVMTRACE_LONG_[234]D() and TRACE_2_LONG_[234]D() were latently buggy.  They
   blindly discard extra parameters, but luckily no users are impacted.  They
   are also obfuscated wrappers, depending on exactly one or two parameters
   being TRC_PAR_LONG() to compile successfully.
 * HVMTRACE_LONG_1D() behaves unlike its named companions, and takes exactly
   one 64bit parameter which it splits manually.  It's one user,
   vmx_cr_access()'s LMSW path, is gets adjusted to use TRACE_PARAM64().
 * TRACE_?D() and TRACE_2_LONG_*() change to TRACE_TIME() as cycles is always.
 * HVMTRACE_ND() is opencoded for VMENTRY/VMEXIT records to include cycles.
   These are converted to TRACE_TIME(), with the old modifier parameter
   expressed as an OR at the callsite.  One callsite, svm_vmenter_helper() had
   a nested tb_init_done check, which is dropped.  (The optimiser also spotted
   this, which is why it doesn't manifest as a binary difference.)
 * All HVMTRACE_?D() change to TRACE() as cycles is explicitly skipped.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

v2.1:
 * New

I'm in two minds as to whether to split this up by subsystem or not.  It is
95% x86, and isn't a massive patch.
---
 xen/arch/x86/acpi/cpu_idle.c  | 12 +++++-----
 xen/arch/x86/compat.c         |  4 ++--
 xen/arch/x86/cpu/mwait-idle.c |  6 ++---
 xen/arch/x86/emul-i8254.c     | 14 +++++------
 xen/arch/x86/hvm/emulate.c    |  8 +++----
 xen/arch/x86/hvm/hpet.c       |  6 ++---
 xen/arch/x86/hvm/hvm.c        | 16 ++++++-------
 xen/arch/x86/hvm/rtc.c        | 12 +++++-----
 xen/arch/x86/hvm/svm/intr.c   |  6 ++---
 xen/arch/x86/hvm/svm/svm.c    | 40 +++++++++++++++-----------------
 xen/arch/x86/hvm/vlapic.c     | 29 +++++++++++------------
 xen/arch/x86/hvm/vmx/intr.c   |  6 ++---
 xen/arch/x86/hvm/vmx/vmx.c    | 54 +++++++++++++++++++++----------------------
 xen/arch/x86/hvm/vpic.c       | 13 +++++------
 xen/arch/x86/irq.c            | 14 +++++------
 xen/arch/x86/traps.c          |  2 +-
 xen/common/domain.c           |  4 ++--
 xen/common/grant_table.c      |  6 ++---
 xen/common/sched/core.c       | 48 ++++++++++++++++++--------------------
 xen/common/sched/credit.c     | 30 +++++++++++-------------
 xen/drivers/cpufreq/utility.c |  2 +-
 21 files changed, 159 insertions(+), 173 deletions(-)

diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index d788c8bffc84..98cbb8dd8316 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -752,7 +752,7 @@ static void acpi_processor_idle(void)
             /* Get start time (ticks) */
             t1 = alternative_call(cpuidle_get_tick);
             /* Trace cpu idle entry */
-            TRACE_4D(TRC_PM_IDLE_ENTRY, cx->idx, t1, exp, pred);
+            TRACE_TIME(TRC_PM_IDLE_ENTRY, cx->idx, t1, exp, pred);
 
             update_last_cx_stat(power, cx, t1);
 
@@ -762,8 +762,8 @@ static void acpi_processor_idle(void)
             t2 = alternative_call(cpuidle_get_tick);
             trace_exit_reason(irq_traced);
             /* Trace cpu idle exit */
-            TRACE_6D(TRC_PM_IDLE_EXIT, cx->idx, t2,
-                     irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
+            TRACE_TIME(TRC_PM_IDLE_EXIT, cx->idx, t2,
+                       irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
             /* Update statistics */
             update_idle_stats(power, cx, t1, t2);
             /* Re-enable interrupts */
@@ -783,7 +783,7 @@ static void acpi_processor_idle(void)
         /* Get start time (ticks) */
         t1 = alternative_call(cpuidle_get_tick);
         /* Trace cpu idle entry */
-        TRACE_4D(TRC_PM_IDLE_ENTRY, cx->idx, t1, exp, pred);
+        TRACE_TIME(TRC_PM_IDLE_ENTRY, cx->idx, t1, exp, pred);
 
         update_last_cx_stat(power, cx, t1);
 
@@ -838,8 +838,8 @@ static void acpi_processor_idle(void)
         cstate_restore_tsc();
         trace_exit_reason(irq_traced);
         /* Trace cpu idle exit */
-        TRACE_6D(TRC_PM_IDLE_EXIT, cx->idx, t2,
-                 irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
+        TRACE_TIME(TRC_PM_IDLE_EXIT, cx->idx, t2,
+                   irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
 
         /* Update statistics */
         update_idle_stats(power, cx, t1, t2);
diff --git a/xen/arch/x86/compat.c b/xen/arch/x86/compat.c
index 58b202f701d5..e4e09f34b87e 100644
--- a/xen/arch/x86/compat.c
+++ b/xen/arch/x86/compat.c
@@ -42,8 +42,8 @@ long do_sched_op_compat(int cmd, unsigned long arg)
         return fn(cmd, guest_handle_from_ptr(NULL, void));
 
     case SCHEDOP_shutdown:
-        TRACE_3D(TRC_SCHED_SHUTDOWN,
-                 current->domain->domain_id, current->vcpu_id, arg);
+        TRACE_TIME(TRC_SCHED_SHUTDOWN,
+                   current->domain->domain_id, current->vcpu_id, arg);
         domain_shutdown(current->domain, (u8)arg);
         break;
 
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index f0c6ff9d5229..43567865764b 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -808,7 +808,7 @@ static void mwait_idle(void)
 		lapic_timer_off();
 
 	before = alternative_call(cpuidle_get_tick);
-	TRACE_4D(TRC_PM_IDLE_ENTRY, cx->type, before, exp, pred);
+	TRACE_TIME(TRC_PM_IDLE_ENTRY, cx->type, before, exp, pred);
 
 	update_last_cx_stat(power, cx, before);
 
@@ -819,8 +819,8 @@ static void mwait_idle(void)
 
 	cstate_restore_tsc();
 	trace_exit_reason(irq_traced);
-	TRACE_6D(TRC_PM_IDLE_EXIT, cx->type, after,
-		irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
+	TRACE_TIME(TRC_PM_IDLE_EXIT, cx->type, after,
+		   irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
 
 	/* Now back in C0. */
 	update_idle_stats(power, cx, before, after);
diff --git a/xen/arch/x86/emul-i8254.c b/xen/arch/x86/emul-i8254.c
index 050c784702af..19b5c95af216 100644
--- a/xen/arch/x86/emul-i8254.c
+++ b/xen/arch/x86/emul-i8254.c
@@ -159,7 +159,7 @@ static int pit_get_gate(PITState *pit, int channel)
 static void pit_time_fired(struct vcpu *v, void *priv)
 {
     uint64_t *count_load_time = priv;
-    TRACE_0D(TRC_HVM_EMUL_PIT_TIMER_CB);
+    TRACE_TIME(TRC_HVM_EMUL_PIT_TIMER_CB);
     *count_load_time = get_guest_time(v);
 }
 
@@ -189,19 +189,19 @@ static void pit_load_count(PITState *pit, int channel, int val)
     case 2:
     case 3:
         /* Periodic timer. */
-        TRACE_2D(TRC_HVM_EMUL_PIT_START_TIMER, period, period);
+        TRACE_TIME(TRC_HVM_EMUL_PIT_START_TIMER, period, period);
         create_periodic_time(v, &pit->pt0, period, period, 0, pit_time_fired, 
                              &pit->count_load_time[channel], false);
         break;
     case 1:
     case 4:
         /* One-shot timer. */
-        TRACE_2D(TRC_HVM_EMUL_PIT_START_TIMER, period, 0);
+        TRACE_TIME(TRC_HVM_EMUL_PIT_START_TIMER, period, 0);
         create_periodic_time(v, &pit->pt0, period, 0, 0, pit_time_fired,
                              &pit->count_load_time[channel], false);
         break;
     default:
-        TRACE_0D(TRC_HVM_EMUL_PIT_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_PIT_STOP_TIMER);
         destroy_periodic_time(&pit->pt0);
         break;
     }
@@ -385,7 +385,7 @@ void pit_stop_channel0_irq(PITState *pit)
     if ( !has_vpit(current->domain) )
         return;
 
-    TRACE_0D(TRC_HVM_EMUL_PIT_STOP_TIMER);
+    TRACE_TIME(TRC_HVM_EMUL_PIT_STOP_TIMER);
     spin_lock(&pit->lock);
     destroy_periodic_time(&pit->pt0);
     spin_unlock(&pit->lock);
@@ -454,7 +454,7 @@ void pit_reset(struct domain *d)
 
     if ( is_hvm_domain(d) )
     {
-        TRACE_0D(TRC_HVM_EMUL_PIT_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_PIT_STOP_TIMER);
         destroy_periodic_time(&pit->pt0);
         pit->pt0.source = PTSRC_isa;
     }
@@ -499,7 +499,7 @@ void pit_deinit(struct domain *d)
 
     if ( is_hvm_domain(d) )
     {
-        TRACE_0D(TRC_HVM_EMUL_PIT_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_PIT_STOP_TIMER);
         destroy_periodic_time(&pit->pt0);
     }
 }
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 425c8ddd9779..f863a19df311 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -2183,7 +2183,7 @@ static int hvmemul_read_cr(
     case 3:
     case 4:
         *val = current->arch.hvm.guest_cr[reg];
-        HVMTRACE_LONG_2D(CR_READ, reg, TRC_PAR_LONG(*val));
+        TRACE(TRC_HVM_CR_READ64, reg, TRACE_PARAM64(*val));
         return X86EMUL_OKAY;
     default:
         break;
@@ -2199,7 +2199,7 @@ static int hvmemul_write_cr(
 {
     int rc;
 
-    HVMTRACE_LONG_2D(CR_WRITE, reg, TRC_PAR_LONG(val));
+    TRACE(TRC_HVM_CR_WRITE64, reg, TRACE_PARAM64(val));
     switch ( reg )
     {
     case 0:
@@ -2244,7 +2244,7 @@ static int hvmemul_read_xcr(
     int rc = x86emul_read_xcr(reg, val, ctxt);
 
     if ( rc == X86EMUL_OKAY )
-        HVMTRACE_LONG_2D(XCR_READ, reg, TRC_PAR_LONG(*val));
+        TRACE(TRC_HVM_XCR_READ64, reg, TRACE_PARAM64(*val));
 
     return rc;
 }
@@ -2254,7 +2254,7 @@ static int hvmemul_write_xcr(
     uint64_t val,
     struct x86_emulate_ctxt *ctxt)
 {
-    HVMTRACE_LONG_2D(XCR_WRITE, reg, TRC_PAR_LONG(val));
+    TRACE(TRC_HVM_XCR_WRITE64, reg, TRACE_PARAM64(val));
 
     return x86emul_write_xcr(reg, val, ctxt);
 }
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 8267f0b8a278..5873bb8a506d 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -212,7 +212,7 @@ static void hpet_stop_timer(HPETState *h, unsigned int tn,
 {
     ASSERT(tn < HPET_TIMER_NUM);
     ASSERT(rw_is_write_locked(&h->lock));
-    TRACE_1D(TRC_HVM_EMUL_HPET_STOP_TIMER, tn);
+    TRACE_TIME(TRC_HVM_EMUL_HPET_STOP_TIMER, tn);
     destroy_periodic_time(&h->pt[tn]);
     /* read the comparator to get it updated so a read while stopped will
      * return the expected value. */
@@ -316,8 +316,8 @@ static void hpet_set_timer(HPETState *h, unsigned int tn,
     if ( !oneshot )
         period_ns = hpet_tick_to_ns(h, h->hpet.period[tn]);
 
-    TRACE_2_LONG_4D(TRC_HVM_EMUL_HPET_START_TIMER, tn, irq,
-                    TRC_PAR_LONG(diff_ns), TRC_PAR_LONG(period_ns));
+    TRACE_TIME(TRC_HVM_EMUL_HPET_START_TIMER, tn, irq,
+               TRACE_PARAM64(diff_ns), TRACE_PARAM64(period_ns));
 
     create_periodic_time(vhpet_vcpu(h), &h->pt[tn], diff_ns, period_ns,
                          irq, timer_level(h, tn) ? hpet_timer_fired : NULL,
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 09cf6330ad26..fa211d14afd8 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1679,7 +1679,7 @@ void hvm_hlt(unsigned int eflags)
 
     do_sched_op(SCHEDOP_block, guest_handle_from_ptr(NULL, void));
 
-    HVMTRACE_1D(HLT, /* pending = */ vcpu_runnable(curr));
+    TRACE(TRC_HVM_HLT, /* pending = */ vcpu_runnable(curr));
 }
 
 void hvm_triple_fault(void)
@@ -2136,7 +2136,7 @@ int hvm_mov_to_cr(unsigned int cr, unsigned int gpr)
     unsigned long val = *decode_gpr(guest_cpu_user_regs(), gpr);
     int rc;
 
-    HVMTRACE_LONG_2D(CR_WRITE, cr, TRC_PAR_LONG(val));
+    TRACE(TRC_HVM_CR_WRITE64, cr, TRACE_PARAM64(val));
     HVM_DBG_LOG(DBG_LEVEL_1, "CR%u, value = %lx", cr, val);
 
     switch ( cr )
@@ -2201,7 +2201,7 @@ int hvm_mov_from_cr(unsigned int cr, unsigned int gpr)
     }
 
     *reg = val;
-    HVMTRACE_LONG_2D(CR_READ, cr, TRC_PAR_LONG(val));
+    TRACE(TRC_HVM_CR_READ64, cr, TRACE_PARAM64(val));
     HVM_DBG_LOG(DBG_LEVEL_VMMU, "CR%u, value = %lx", cr, val);
 
     return X86EMUL_OKAY;
@@ -3463,7 +3463,7 @@ int hvm_vmexit_cpuid(struct cpu_user_regs *regs, unsigned int inst_len)
     }
 
     guest_cpuid(curr, leaf, subleaf, &res);
-    HVMTRACE_6D(CPUID, leaf, subleaf, res.a, res.b, res.c, res.d);
+    TRACE(TRC_HVM_CPUID, leaf, subleaf, res.a, res.b, res.c, res.d);
 
     regs->rax = res.a;
     regs->rbx = res.b;
@@ -3477,7 +3477,7 @@ void hvm_rdtsc_intercept(struct cpu_user_regs *regs)
 {
     msr_split(regs, hvm_get_guest_tsc(current));
 
-    HVMTRACE_2D(RDTSC, regs->eax, regs->edx);
+    TRACE(TRC_HVM_RDTSC, regs->eax, regs->edx);
 }
 
 int hvm_msr_read_intercept(unsigned int msr, uint64_t *msr_content)
@@ -3584,8 +3584,7 @@ int hvm_msr_read_intercept(unsigned int msr, uint64_t *msr_content)
     }
 
  out:
-    HVMTRACE_3D(MSR_READ, msr,
-                (uint32_t)*msr_content, (uint32_t)(*msr_content >> 32));
+    TRACE(TRC_HVM_MSR_READ, msr, TRACE_PARAM64(*msr_content));
     return ret;
 
  gp_fault:
@@ -3601,8 +3600,7 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,
     struct domain *d = v->domain;
     int ret;
 
-    HVMTRACE_3D(MSR_WRITE, msr,
-               (uint32_t)msr_content, (uint32_t)(msr_content >> 32));
+    TRACE(TRC_HVM_MSR_WRITE, msr, TRACE_PARAM64(msr_content));
 
     if ( may_defer && unlikely(monitored_msr(v->domain, msr)) )
     {
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index 3150f5f1479b..6c133eabf5d4 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -92,7 +92,7 @@ static void rtc_pf_callback(struct vcpu *v, void *opaque)
          && ++(s->pt_dead_ticks) >= 10 )
     {
         /* VM is ignoring its RTC; no point in running the timer */
-        TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_RTC_STOP_TIMER);
         destroy_periodic_time(&s->pt);
         s->period = 0;
     }
@@ -155,7 +155,7 @@ static void rtc_timer_update(RTCState *s)
                     delta = period - ((now - s->start_time) % period);
                 if ( s->hw.cmos_data[RTC_REG_B] & RTC_PIE )
                 {
-                    TRACE_2D(TRC_HVM_EMUL_RTC_START_TIMER, delta, period);
+                    TRACE_TIME(TRC_HVM_EMUL_RTC_START_TIMER, delta, period);
                     create_periodic_time(v, &s->pt, delta, period,
                                          RTC_IRQ, rtc_pf_callback, s, false);
                 }
@@ -166,7 +166,7 @@ static void rtc_timer_update(RTCState *s)
         }
         /* fall through */
     default:
-        TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_RTC_STOP_TIMER);
         destroy_periodic_time(&s->pt);
         s->period = 0;
         break;
@@ -519,7 +519,7 @@ static int rtc_ioport_write(void *opaque, uint32_t addr, uint32_t data)
         rtc_update_irq(s);
         if ( (data ^ orig) & RTC_PIE )
         {
-            TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
+            TRACE_TIME(TRC_HVM_EMUL_RTC_STOP_TIMER);
             destroy_periodic_time(&s->pt);
             s->period = 0;
             rtc_timer_update(s);
@@ -802,7 +802,7 @@ void rtc_reset(struct domain *d)
     if ( !has_vrtc(d) )
         return;
 
-    TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
+    TRACE_TIME(TRC_HVM_EMUL_RTC_STOP_TIMER);
     destroy_periodic_time(&s->pt);
     s->period = 0;
     s->pt.source = PTSRC_isa;
@@ -877,7 +877,7 @@ void rtc_deinit(struct domain *d)
 
     spin_barrier(&s->lock);
 
-    TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
+    TRACE_TIME(TRC_HVM_EMUL_RTC_STOP_TIMER);
     destroy_periodic_time(&s->pt);
     kill_timer(&s->update_timer);
     kill_timer(&s->update_timer2);
diff --git a/xen/arch/x86/hvm/svm/intr.c b/xen/arch/x86/hvm/svm/intr.c
index 7f815d230785..122c9d7fbf14 100644
--- a/xen/arch/x86/hvm/svm/intr.c
+++ b/xen/arch/x86/hvm/svm/intr.c
@@ -101,8 +101,8 @@ static void svm_enable_intr_window(struct vcpu *v, struct hvm_intack intack)
         }
     }
 
-    HVMTRACE_3D(INTR_WINDOW, intack.vector, intack.source,
-                vmcb->event_inj.v ? vmcb->event_inj.vector : -1);
+    TRACE(TRC_HVM_INTR_WINDOW, intack.vector, intack.source,
+          vmcb->event_inj.v ? vmcb->event_inj.vector : -1);
 
     /*
      * Create a dummy virtual interrupt to intercept as soon as the
@@ -217,7 +217,7 @@ void svm_intr_assist(void)
     }
     else
     {
-        HVMTRACE_2D(INJ_VIRQ, intack.vector, /*fake=*/ 0);
+        TRACE(TRC_HVM_INJ_VIRQ, intack.vector, /*fake=*/ 0);
         svm_inject_extint(v, intack.vector);
         pt_intr_post(v, intack);
     }
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index f0e10dec046e..b98a2dd71d1e 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1049,10 +1049,8 @@ void svm_vmenter_helper(const struct cpu_user_regs *regs)
 
     svm_asid_handle_vmrun();
 
-    if ( unlikely(tb_init_done) )
-        HVMTRACE_ND(VMENTRY,
-                    nestedhvm_vcpu_in_guestmode(curr) ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/);
+    TRACE_TIME(TRC_HVM_VMENTRY |
+               (nestedhvm_vcpu_in_guestmode(curr) ? TRC_HVM_NESTEDFLAG : 0));
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
@@ -1424,10 +1422,10 @@ static void svm_inject_event(const struct x86_event *event)
 
     if ( _event.vector == TRAP_page_fault &&
          _event.type == X86_EVENTTYPE_HW_EXCEPTION )
-        HVMTRACE_LONG_2D(PF_INJECT, _event.error_code,
-                         TRC_PAR_LONG(_event.cr2));
+        TRACE(TRC_HVM_PF_INJECT64, _event.error_code,
+              TRACE_PARAM64(_event.cr2));
     else
-        HVMTRACE_2D(INJ_EXC, _event.vector, _event.error_code);
+        TRACE(TRC_HVM_INJ_EXC, _event.vector, _event.error_code);
 }
 
 static bool svm_event_pending(const struct vcpu *v)
@@ -1787,7 +1785,7 @@ static void svm_dr_access(struct vcpu *v, struct cpu_user_regs *regs)
 {
     struct vmcb_struct *vmcb = vcpu_nestedhvm(v).nv_n1vmcx;
 
-    HVMTRACE_0D(DR_WRITE);
+    TRACE(TRC_HVM_DR_WRITE);
     __restore_debug_registers(vmcb, v);
 }
 
@@ -2436,7 +2434,7 @@ static void svm_invlpga_intercept(
 
 static void svm_invlpg_intercept(unsigned long linear)
 {
-    HVMTRACE_LONG_2D(INVLPG, 0, TRC_PAR_LONG(linear));
+    TRACE(TRC_HVM_INVLPG64, 0, TRACE_PARAM64(linear));
     paging_invlpg(current, linear);
 }
 
@@ -2564,11 +2562,11 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     exit_reason = vmcb->exitcode;
 
     if ( hvm_long_mode_active(v) )
-        HVMTRACE_ND(VMEXIT64, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, exit_reason, TRC_PAR_LONG(regs->rip));
+        TRACE_TIME(TRC_HVM_VMEXIT64 | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
+                   exit_reason, TRACE_PARAM64(regs->rip));
     else
-        HVMTRACE_ND(VMEXIT, vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0,
-                    1/*cycles*/, exit_reason, regs->eip);
+        TRACE_TIME(TRC_HVM_VMEXIT | (vcpu_guestmode ? TRC_HVM_NESTEDFLAG : 0),
+                   exit_reason, regs->eip);
 
     if ( vcpu_guestmode )
     {
@@ -2660,17 +2658,17 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     {
     case VMEXIT_INTR:
         /* Asynchronous event, handled when we STGI'd after the VMEXIT. */
-        HVMTRACE_0D(INTR);
+        TRACE(TRC_HVM_INTR);
         break;
 
     case VMEXIT_NMI:
         /* Asynchronous event, handled when we STGI'd after the VMEXIT. */
-        HVMTRACE_0D(NMI);
+        TRACE(TRC_HVM_NMI);
         break;
 
     case VMEXIT_SMI:
         /* Asynchronous event, handled when we STGI'd after the VMEXIT. */
-        HVMTRACE_0D(SMI);
+        TRACE(TRC_HVM_SMI);
         break;
 
     case VMEXIT_ICEBP:
@@ -2758,9 +2756,9 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
             if ( trace_will_trace_event(TRC_SHADOW) )
                 break;
             if ( hvm_long_mode_active(v) )
-                HVMTRACE_LONG_2D(PF_XEN, regs->error_code, TRC_PAR_LONG(va));
+                TRACE(TRC_HVM_PF_XEN64, regs->error_code, TRACE_PARAM64(va));
             else
-                HVMTRACE_2D(PF_XEN, regs->error_code, va);
+                TRACE(TRC_HVM_PF_XEN, regs->error_code, va);
             break;
         }
 
@@ -2769,7 +2767,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     }
 
     case VMEXIT_EXCEPTION_AC:
-        HVMTRACE_1D(TRAP, TRAP_alignment_check);
+        TRACE(TRC_HVM_TRAP, TRAP_alignment_check);
         hvm_inject_hw_exception(TRAP_alignment_check, vmcb->exitinfo1);
         break;
 
@@ -2779,7 +2777,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
 
     /* Asynchronous event, handled when we STGI'd after the VMEXIT. */
     case VMEXIT_EXCEPTION_MC:
-        HVMTRACE_0D(MCE);
+        TRACE(TRC_HVM_MCE);
         svm_vmexit_mce_intercept(v, regs);
         break;
 
@@ -2917,7 +2915,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
         if ( (insn_len = svm_get_insn_len(v, INSTR_VMCALL)) == 0 )
             break;
         BUG_ON(vcpu_guestmode);
-        HVMTRACE_1D(VMMCALL, regs->eax);
+        TRACE(TRC_HVM_VMMCALL, regs->eax);
 
         if ( hvm_hypercall(regs) == HVM_HCALL_completed )
             __update_guest_eip(regs, insn_len);
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index b8c84458ffdc..30fd3b46ec4d 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -692,7 +692,7 @@ int guest_rdmsr_x2apic(const struct vcpu *v, uint32_t msr, uint64_t *val)
 
 static void vlapic_pt_cb(struct vcpu *v, void *data)
 {
-    TRACE_0D(TRC_HVM_EMUL_LAPIC_TIMER_CB);
+    TRACE_TIME(TRC_HVM_EMUL_LAPIC_TIMER_CB);
     *(s_time_t *)data = hvm_get_guest_time(v);
 }
 
@@ -746,9 +746,8 @@ static void vlapic_update_timer(struct vlapic *vlapic, uint32_t lvtt,
             delta = delta * vlapic->hw.timer_divisor / old_divisor;
         }
 
-        TRACE_2_LONG_3D(TRC_HVM_EMUL_LAPIC_START_TIMER, TRC_PAR_LONG(delta),
-                        TRC_PAR_LONG(is_periodic ? period : 0),
-                        vlapic->pt.irq);
+        TRACE_TIME(TRC_HVM_EMUL_LAPIC_START_TIMER, TRACE_PARAM64(delta),
+                   TRACE_PARAM64(is_periodic ? period : 0), vlapic->pt.irq);
 
         create_periodic_time(current, &vlapic->pt, delta,
                              is_periodic ? period : 0, vlapic->pt.irq,
@@ -768,7 +767,7 @@ static void vlapic_update_timer(struct vlapic *vlapic, uint32_t lvtt,
     }
     else
     {
-        TRACE_0D(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
+        TRACE_TIME(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
         destroy_periodic_time(&vlapic->pt);
         /*
          * From now, TMCCT should return 0 until TMICT is set again.
@@ -1202,8 +1201,8 @@ void vlapic_tdt_msr_set(struct vlapic *vlapic, uint64_t value)
 
         vlapic->hw.tdt_msr = value;
         /* .... reprogram tdt timer */
-        TRACE_2_LONG_3D(TRC_HVM_EMUL_LAPIC_START_TIMER, TRC_PAR_LONG(delta),
-                        TRC_PAR_LONG(0LL), vlapic->pt.irq);
+        TRACE_TIME(TRC_HVM_EMUL_LAPIC_START_TIMER, TRACE_PARAM64(delta),
+                   TRACE_PARAM64(0LL), vlapic->pt.irq);
         create_periodic_time(v, &vlapic->pt, delta, 0,
                              vlapic->pt.irq, vlapic_tdt_pt_cb,
                              &vlapic->timer_last_update, false);
@@ -1216,8 +1215,8 @@ void vlapic_tdt_msr_set(struct vlapic *vlapic, uint64_t value)
         /* trigger a timer event if needed */
         if ( value > 0 )
         {
-            TRACE_2_LONG_3D(TRC_HVM_EMUL_LAPIC_START_TIMER, TRC_PAR_LONG(0LL),
-                            TRC_PAR_LONG(0LL), vlapic->pt.irq);
+            TRACE_TIME(TRC_HVM_EMUL_LAPIC_START_TIMER, TRACE_PARAM64(0LL),
+                       TRACE_PARAM64(0LL), vlapic->pt.irq);
             create_periodic_time(v, &vlapic->pt, 0, 0,
                                  vlapic->pt.irq, vlapic_tdt_pt_cb,
                                  &vlapic->timer_last_update, false);
@@ -1226,7 +1225,7 @@ void vlapic_tdt_msr_set(struct vlapic *vlapic, uint64_t value)
         else
         {
             /* .... stop tdt timer */
-            TRACE_0D(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
+            TRACE_TIME(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
             destroy_periodic_time(&vlapic->pt);
         }
 
@@ -1276,7 +1275,7 @@ int vlapic_accept_pic_intr(struct vcpu *v)
     if ( target )
         accept = __vlapic_accept_pic_intr(v);
 
-    TRACE_2D(TRC_HVM_EMUL_LAPIC_PIC_INTR, target, accept);
+    TRACE_TIME(TRC_HVM_EMUL_LAPIC_PIC_INTR, target, accept);
 
     return target && accept;
 }
@@ -1427,7 +1426,7 @@ static void vlapic_do_init(struct vlapic *vlapic)
     vlapic_set_reg(vlapic, APIC_SPIV, 0xff);
     vlapic->hw.disabled |= VLAPIC_SW_DISABLED;
 
-    TRACE_0D(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
+    TRACE_TIME(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
     destroy_periodic_time(&vlapic->pt);
 }
 
@@ -1470,8 +1469,8 @@ static void lapic_rearm(struct vlapic *s)
               (uint32_t)tmict * s->hw.timer_divisor);
     is_periodic = vlapic_lvtt_period(s);
 
-    TRACE_2_LONG_3D(TRC_HVM_EMUL_LAPIC_START_TIMER, TRC_PAR_LONG(period),
-             TRC_PAR_LONG(is_periodic ? period : 0LL), s->pt.irq);
+    TRACE_TIME(TRC_HVM_EMUL_LAPIC_START_TIMER, TRACE_PARAM64(period),
+               TRACE_PARAM64(is_periodic ? period : 0LL), s->pt.irq);
 
     create_periodic_time(vlapic_vcpu(s), &s->pt, period,
                          is_periodic ? period : 0,
@@ -1650,7 +1649,7 @@ void vlapic_destroy(struct vcpu *v)
         return;
 
     tasklet_kill(&vlapic->init_sipi.tasklet);
-    TRACE_0D(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
+    TRACE_TIME(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
     destroy_periodic_time(&vlapic->pt);
     unmap_domain_page_global(vlapic->regs);
     free_domheap_page(vlapic->regs_page);
diff --git a/xen/arch/x86/hvm/vmx/intr.c b/xen/arch/x86/hvm/vmx/intr.c
index 80bfbb478782..303d07dc0a3a 100644
--- a/xen/arch/x86/hvm/vmx/intr.c
+++ b/xen/arch/x86/hvm/vmx/intr.c
@@ -79,8 +79,8 @@ static void vmx_enable_intr_window(struct vcpu *v, struct hvm_intack intack)
         unsigned long intr;
 
         __vmread(VM_ENTRY_INTR_INFO, &intr);
-        HVMTRACE_3D(INTR_WINDOW, intack.vector, intack.source,
-                    (intr & INTR_INFO_VALID_MASK) ? intr & 0xff : -1);
+        TRACE(TRC_HVM_INTR_WINDOW, intack.vector, intack.source,
+              (intr & INTR_INFO_VALID_MASK) ? intr & 0xff : -1);
     }
 
     if ( (intack.source == hvm_intsrc_nmi) && cpu_has_vmx_vnmi )
@@ -402,7 +402,7 @@ void vmx_intr_assist(void)
     }
     else
     {
-        HVMTRACE_2D(INJ_VIRQ, intack.vector, /*fake=*/ 0);
+        TRACE(TRC_HVM_INJ_VIRQ, intack.vector, /*fake=*/ 0);
         vmx_inject_extint(intack.vector, intack.source);
         pt_intr_post(v, intack);
     }
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 974a25b34c41..47a5df20ebc1 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1875,10 +1875,10 @@ static void vmx_inject_event(const struct x86_event *event)
 
     if ( (_event.vector == TRAP_page_fault) &&
          (_event.type == X86_EVENTTYPE_HW_EXCEPTION) )
-        HVMTRACE_LONG_2D(PF_INJECT, _event.error_code,
-                         TRC_PAR_LONG(curr->arch.hvm.guest_cr[2]));
+        TRACE(TRC_HVM_PF_INJECT64, _event.error_code,
+              TRACE_PARAM64(curr->arch.hvm.guest_cr[2]));
     else
-        HVMTRACE_2D(INJ_EXC, _event.vector, _event.error_code);
+        TRACE(TRC_HVM_INJ_EXC, _event.vector, _event.error_code);
 }
 
 static bool vmx_event_pending(const struct vcpu *v)
@@ -2783,7 +2783,7 @@ static void noinline vmx_dr_access(unsigned long exit_qualification,
 {
     struct vcpu *v = current;
 
-    HVMTRACE_0D(DR_WRITE);
+    TRACE(TRC_HVM_DR_WRITE);
 
     if ( !v->arch.hvm.flag_dr_dirty )
         __restore_debug_registers(v);
@@ -2795,7 +2795,7 @@ static void noinline vmx_dr_access(unsigned long exit_qualification,
 
 static void noinline vmx_invlpg_intercept(unsigned long linear)
 {
-    HVMTRACE_LONG_2D(INVLPG, /*invlpga=*/ 0, TRC_PAR_LONG(linear));
+    TRACE(TRC_HVM_INVLPG64, /*invlpga=*/ 0, TRACE_PARAM64(linear));
     paging_invlpg(current, linear);
 }
 
@@ -2843,7 +2843,7 @@ static int vmx_cr_access(cr_access_qual_t qual)
         hvm_monitor_crX(CR0, value, old);
         curr->arch.hvm.guest_cr[0] = value;
         vmx_update_guest_cr(curr, 0, 0);
-        HVMTRACE_0D(CLTS);
+        TRACE(TRC_HVM_CLTS);
         break;
     }
 
@@ -2856,7 +2856,7 @@ static int vmx_cr_access(cr_access_qual_t qual)
         value = (value & ~(X86_CR0_MP|X86_CR0_EM|X86_CR0_TS)) |
                 (qual.lmsw_data &
                  (X86_CR0_PE|X86_CR0_MP|X86_CR0_EM|X86_CR0_TS));
-        HVMTRACE_LONG_1D(LMSW, value);
+        TRACE(TRC_HVM_LMSW64, TRACE_PARAM64(value));
 
         if ( (rc = hvm_set_cr0(value, true)) == X86EMUL_EXCEPTION )
             hvm_inject_hw_exception(TRAP_gp_fault, 0);
@@ -3522,7 +3522,7 @@ static void vmx_do_extint(struct cpu_user_regs *regs)
     BUG_ON(!(vector & INTR_INFO_VALID_MASK));
 
     vector &= INTR_INFO_VECTOR_MASK;
-    HVMTRACE_1D(INTR, vector);
+    TRACE(TRC_HVM_INTR, vector);
 
     regs->entry_vector = vector;
     do_IRQ(regs);
@@ -3662,7 +3662,7 @@ static void vmx_failed_vmentry(unsigned int exit_reason,
 
     case EXIT_REASON_MCE_DURING_VMENTRY:
         printk("MCE\n");
-        HVMTRACE_0D(MCE);
+        TRACE(TRC_HVM_MCE);
         /* Already handled. */
         break;
 
@@ -3703,7 +3703,7 @@ static int vmx_handle_eoi_write(void)
     {
         update_guest_eip(); /* Safe: APIC data write */
         vlapic_EOI_set(vcpu_vlapic(current));
-        HVMTRACE_0D(VLAPIC);
+        TRACE(TRC_HVM_VLAPIC);
         return 1;
     }
 
@@ -3864,10 +3864,9 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
     __vmread(VM_EXIT_REASON, &exit_reason);
 
     if ( hvm_long_mode_active(v) )
-        HVMTRACE_ND(VMEXIT64, 0, 1/*cycles*/, exit_reason,
-                    TRC_PAR_LONG(regs->rip));
+        TRACE_TIME(TRC_HVM_VMEXIT64, exit_reason, TRACE_PARAM64(regs->rip));
     else
-        HVMTRACE_ND(VMEXIT, 0, 1/*cycles*/, exit_reason, regs->eip);
+        TRACE_TIME(TRC_HVM_VMEXIT, exit_reason, regs->eip);
 
     perfc_incra(vmexits, exit_reason);
 
@@ -3969,7 +3968,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
         default:
                 perfc_incr(realmode_exits);
                 v->arch.hvm.vmx.vmx_emulate = 1;
-                HVMTRACE_0D(REALMODE_EMULATE);
+                TRACE(TRC_HVM_REALMODE_EMULATE);
                 return;
             }
         case EXIT_REASON_EXTERNAL_INTERRUPT:
@@ -4032,7 +4031,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
              * Table 23-1, "Exit Qualification for Debug Exceptions").
              */
             __vmread(EXIT_QUALIFICATION, &exit_qualification);
-            HVMTRACE_1D(TRAP_DEBUG, exit_qualification);
+            TRACE(TRC_HVM_TRAP_DEBUG, exit_qualification);
             __restore_debug_registers(v);
             write_debugreg(6, exit_qualification | DR_STATUS_RESERVED_ONE);
 
@@ -4094,7 +4093,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
                 domain_pause_for_debugger();
             break;
         case TRAP_int3:
-            HVMTRACE_1D(TRAP, vector);
+            TRACE(TRC_HVM_TRAP, vector);
             if ( !v->domain->debugger_attached )
             {
                 unsigned long insn_len;
@@ -4119,7 +4118,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
             }
             break;
         case TRAP_no_device:
-            HVMTRACE_1D(TRAP, vector);
+            TRACE(TRC_HVM_TRAP, vector);
             vmx_fpu_dirty_intercept();
             break;
         case TRAP_page_fault:
@@ -4137,37 +4136,36 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
                 if ( trace_will_trace_event(TRC_SHADOW) )
                     break;
                 if ( hvm_long_mode_active(v) )
-                    HVMTRACE_LONG_2D(PF_XEN, regs->error_code,
-                                     TRC_PAR_LONG(exit_qualification) );
+                    TRACE(TRC_HVM_PF_XEN64, regs->error_code,
+                          TRACE_PARAM64(exit_qualification));
                 else
-                    HVMTRACE_2D(PF_XEN,
-                                regs->error_code, exit_qualification );
+                    TRACE(TRC_HVM_PF_XEN, regs->error_code, exit_qualification);
                 break;
             }
 
             hvm_inject_page_fault(regs->error_code, exit_qualification);
             break;
         case TRAP_alignment_check:
-            HVMTRACE_1D(TRAP, vector);
+            TRACE(TRC_HVM_TRAP, vector);
             vmx_propagate_intr(intr_info);
             break;
         case TRAP_nmi:
             if ( MASK_EXTR(intr_info, INTR_INFO_INTR_TYPE_MASK) !=
                  X86_EVENTTYPE_NMI )
                 goto exit_and_crash;
-            HVMTRACE_0D(NMI);
+            TRACE(TRC_HVM_NMI);
             /* Already handled above. */
             break;
         case TRAP_machine_check:
-            HVMTRACE_0D(MCE);
+            TRACE(TRC_HVM_MCE);
             /* Already handled above. */
             break;
         case TRAP_invalid_op:
-            HVMTRACE_1D(TRAP, vector);
+            TRACE(TRC_HVM_TRAP, vector);
             hvm_ud_intercept(regs);
             break;
         default:
-            HVMTRACE_1D(TRAP, vector);
+            TRACE(TRC_HVM_TRAP, vector);
             goto exit_and_crash;
         }
         break;
@@ -4255,7 +4253,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
         break;
 
     case EXIT_REASON_VMCALL:
-        HVMTRACE_1D(VMMCALL, regs->eax);
+        TRACE(TRC_HVM_VMMCALL, regs->eax);
 
         if ( hvm_hypercall(regs) == HVM_HCALL_completed )
             update_guest_eip(); /* Safe: VMCALL */
@@ -4644,7 +4642,7 @@ bool vmx_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(curr->arch.hvm.vmx.lbr_flags & LBR_FIXUP_MASK) )
         lbr_fixup();
 
-    HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/);
+    TRACE_TIME(TRC_HVM_VMENTRY);
 
     __vmwrite(GUEST_RIP,    regs->rip);
     __vmwrite(GUEST_RSP,    regs->rsp);
diff --git a/xen/arch/x86/hvm/vpic.c b/xen/arch/x86/hvm/vpic.c
index 91c2c6983393..1360f40b1044 100644
--- a/xen/arch/x86/hvm/vpic.c
+++ b/xen/arch/x86/hvm/vpic.c
@@ -99,8 +99,7 @@ static void vpic_update_int_output(struct hvm_hw_vpic *vpic)
     ASSERT(vpic_is_locked(vpic));
 
     irq = vpic_get_highest_priority_irq(vpic);
-    TRACE_3D(TRC_HVM_EMUL_PIC_INT_OUTPUT, vpic->int_output, vpic->is_master,
-             irq);
+    TRACE_TIME(TRC_HVM_EMUL_PIC_INT_OUTPUT, vpic->int_output, vpic->is_master, irq);
     if ( vpic->int_output == (!vpic->init_state && irq >= 0) )
         return;
 
@@ -119,7 +118,7 @@ static void vpic_update_int_output(struct hvm_hw_vpic *vpic)
 
             if ( v != NULL )
             {
-                TRACE_1D(TRC_HVM_EMUL_PIC_KICK, irq);
+                TRACE_TIME(TRC_HVM_EMUL_PIC_KICK, irq);
                 vcpu_kick(v);
             }
         }
@@ -144,7 +143,7 @@ static void __vpic_intack(struct hvm_hw_vpic *vpic, int irq)
 
     ASSERT(vpic_is_locked(vpic));
 
-    TRACE_2D(TRC_HVM_EMUL_PIC_INTACK, vpic->is_master, irq);
+    TRACE_TIME(TRC_HVM_EMUL_PIC_INTACK, vpic->is_master, irq);
     /* Edge-triggered: clear the IRR (forget the edge). */
     if ( !(vpic->elcr & mask) )
         vpic->irr &= ~mask;
@@ -483,7 +482,7 @@ void vpic_irq_positive_edge(struct domain *d, int irq)
     ASSERT(irq <= 15);
     ASSERT(vpic_is_locked(vpic));
 
-    TRACE_1D(TRC_HVM_EMUL_PIC_POSEDGE, irq);
+    TRACE_TIME(TRC_HVM_EMUL_PIC_POSEDGE, irq);
     if ( irq == 2 )
         return;
 
@@ -501,7 +500,7 @@ void vpic_irq_negative_edge(struct domain *d, int irq)
     ASSERT(irq <= 15);
     ASSERT(vpic_is_locked(vpic));
 
-    TRACE_1D(TRC_HVM_EMUL_PIC_NEGEDGE, irq);
+    TRACE_TIME(TRC_HVM_EMUL_PIC_NEGEDGE, irq);
     if ( irq == 2 )
         return;
 
@@ -519,7 +518,7 @@ int vpic_ack_pending_irq(struct vcpu *v)
 
     accept = vlapic_accept_pic_intr(v);
 
-    TRACE_2D(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, accept, vpic->int_output);
+    TRACE_TIME(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, accept, vpic->int_output);
     if ( !accept || !vpic->int_output )
         return -1;
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 67cbf6b979dc..df85932f44b4 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -236,7 +236,7 @@ static void _clear_irq_vector(struct irq_desc *desc)
     for_each_cpu(cpu, tmp_mask)
     {
         ASSERT(per_cpu(vector_irq, cpu)[old_vector] == irq);
-        TRACE_3D(TRC_HW_IRQ_MOVE_FINISH, irq, old_vector, cpu);
+        TRACE_TIME(TRC_HW_IRQ_MOVE_FINISH, irq, old_vector, cpu);
         per_cpu(vector_irq, cpu)[old_vector] = ~irq;
     }
 
@@ -799,13 +799,11 @@ void irq_move_cleanup_interrupt(struct cpu_user_regs *regs)
                 goto unlock;
             }
             send_IPI_self(IRQ_MOVE_CLEANUP_VECTOR);
-            TRACE_3D(TRC_HW_IRQ_MOVE_CLEANUP_DELAY,
-                     irq, vector, smp_processor_id());
+            TRACE_TIME(TRC_HW_IRQ_MOVE_CLEANUP_DELAY, irq, vector, me);
             goto unlock;
         }
 
-        TRACE_3D(TRC_HW_IRQ_MOVE_CLEANUP,
-                 irq, vector, smp_processor_id());
+        TRACE_TIME(TRC_HW_IRQ_MOVE_CLEANUP, irq, vector, me);
 
         per_cpu(vector_irq, me)[vector] = ~irq;
         desc->arch.move_cleanup_count--;
@@ -1939,7 +1937,7 @@ void do_IRQ(struct cpu_user_regs *regs)
                     spin_unlock(&desc->lock);
                 }
             }
-            TRACE_1D(TRC_HW_IRQ_UNMAPPED_VECTOR, vector);
+            TRACE_TIME(TRC_HW_IRQ_UNMAPPED_VECTOR, vector);
         }
         goto out_no_unlock;
     }
@@ -1980,7 +1978,7 @@ void do_IRQ(struct cpu_user_regs *regs)
 
         tsc_in = tb_init_done ? get_cycles() : 0;
         do_IRQ_guest(desc, vector);
-        TRACE_3D(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
+        TRACE_TIME(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
         goto out_no_end;
     }
 
@@ -2004,7 +2002,7 @@ void do_IRQ(struct cpu_user_regs *regs)
 
         tsc_in = tb_init_done ? get_cycles() : 0;
         action->handler(irq, action->dev_id, regs);
-        TRACE_3D(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
+        TRACE_TIME(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
 
         spin_lock_irq(&desc->lock);
     }
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 0cc1ee95cb5b..ca8374eadf9e 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1881,7 +1881,7 @@ void do_device_not_available(struct cpu_user_regs *regs)
         curr->arch.pv.ctrlreg[0] &= ~X86_CR0_TS;
     }
     else
-        TRACE_0D(TRC_PV_MATH_STATE_RESTORE);
+        TRACE_TIME(TRC_PV_MATH_STATE_RESTORE);
 #else
     ASSERT_UNREACHABLE();
 #endif
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0c2..e8672f86cc82 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -575,7 +575,7 @@ struct domain *domain_create(domid_t domid,
         hardware_domain = d;
     }
 
-    TRACE_1D(TRC_DOM0_DOM_ADD, d->domain_id);
+    TRACE_TIME(TRC_DOM0_DOM_ADD, d->domain_id);
 
     lock_profile_register_struct(LOCKPROF_TYPE_PERDOM, d, domid);
 
@@ -1141,7 +1141,7 @@ void domain_destroy(struct domain *d)
     if ( atomic_cmpxchg(&d->refcnt, 0, DOMAIN_DESTROYED) != 0 )
         return;
 
-    TRACE_1D(TRC_DOM0_DOM_REM, d->domain_id);
+    TRACE_TIME(TRC_DOM0_DOM_REM, d->domain_id);
 
     /* Delete from task list and task hashtable. */
     spin_lock(&domlist_update_lock);
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index fe1fc11b228e..7f1b2b010bc1 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1271,7 +1271,7 @@ map_grant_ref(
             goto undo_out;
     }
 
-    TRACE_1D(TRC_MEM_PAGE_GRANT_MAP, op->dom);
+    TRACE_TIME(TRC_MEM_PAGE_GRANT_MAP, op->dom);
 
     /*
      * All maptrack entry users check mt->flags first before using the
@@ -1398,7 +1398,7 @@ unmap_common(
         return;
     }
 
-    TRACE_1D(TRC_MEM_PAGE_GRANT_UNMAP, dom);
+    TRACE_TIME(TRC_MEM_PAGE_GRANT_UNMAP, dom);
 
     rgt = rd->grant_table;
 
@@ -2384,7 +2384,7 @@ gnttab_transfer(
         put_gfn(d, gop.mfn);
 #endif
 
-        TRACE_1D(TRC_MEM_PAGE_GRANT_TRANSFER, e->domain_id);
+        TRACE_TIME(TRC_MEM_PAGE_GRANT_TRANSFER, e->domain_id);
 
         /* Tell the guest about its new page frame. */
         grant_read_lock(e->grant_table);
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index fe133cbf117c..37e1a17a888b 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -827,7 +827,7 @@ int sched_init_domain(struct domain *d, unsigned int poolid)
         return ret;
 
     SCHED_STAT_CRANK(dom_init);
-    TRACE_1D(TRC_SCHED_DOM_ADD, d->domain_id);
+    TRACE_TIME(TRC_SCHED_DOM_ADD, d->domain_id);
 
     rcu_read_lock(&sched_res_rculock);
 
@@ -850,7 +850,7 @@ void sched_destroy_domain(struct domain *d)
     if ( d->cpupool )
     {
         SCHED_STAT_CRANK(dom_destroy);
-        TRACE_1D(TRC_SCHED_DOM_REM, d->domain_id);
+        TRACE_TIME(TRC_SCHED_DOM_REM, d->domain_id);
 
         rcu_read_lock(&sched_res_rculock);
 
@@ -891,7 +891,7 @@ void vcpu_sleep_nosync(struct vcpu *v)
     unsigned long flags;
     spinlock_t *lock;
 
-    TRACE_2D(TRC_SCHED_SLEEP, v->domain->domain_id, v->vcpu_id);
+    TRACE_TIME(TRC_SCHED_SLEEP, v->domain->domain_id, v->vcpu_id);
 
     rcu_read_lock(&sched_res_rculock);
 
@@ -920,7 +920,7 @@ void vcpu_wake(struct vcpu *v)
     spinlock_t *lock;
     struct sched_unit *unit = v->sched_unit;
 
-    TRACE_2D(TRC_SCHED_WAKE, v->domain->domain_id, v->vcpu_id);
+    TRACE_TIME(TRC_SCHED_WAKE, v->domain->domain_id, v->vcpu_id);
 
     rcu_read_lock(&sched_res_rculock);
 
@@ -1429,7 +1429,7 @@ void vcpu_block(void)
     }
     else
     {
-        TRACE_2D(TRC_SCHED_BLOCK, v->domain->domain_id, v->vcpu_id);
+        TRACE_TIME(TRC_SCHED_BLOCK, v->domain->domain_id, v->vcpu_id);
         raise_softirq(SCHEDULE_SOFTIRQ);
     }
 }
@@ -1502,7 +1502,7 @@ static long do_poll(struct sched_poll *sched_poll)
     if ( sched_poll->timeout != 0 )
         set_timer(&v->poll_timer, sched_poll->timeout);
 
-    TRACE_2D(TRC_SCHED_BLOCK, d->domain_id, v->vcpu_id);
+    TRACE_TIME(TRC_SCHED_BLOCK, d->domain_id, v->vcpu_id);
     raise_softirq(SCHEDULE_SOFTIRQ);
 
     return 0;
@@ -1530,7 +1530,7 @@ long vcpu_yield(void)
 
     SCHED_STAT_CRANK(vcpu_yield);
 
-    TRACE_2D(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
+    TRACE_TIME(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
     raise_softirq(SCHEDULE_SOFTIRQ);
     return 0;
 }
@@ -1888,9 +1888,8 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
-        TRACE_3D(TRC_SCHED_SHUTDOWN,
-                 current->domain->domain_id, current->vcpu_id,
-                 sched_shutdown.reason);
+        TRACE_TIME(TRC_SCHED_SHUTDOWN, current->domain->domain_id,
+                   current->vcpu_id, sched_shutdown.reason);
         ret = domain_shutdown(current->domain, (u8)sched_shutdown.reason);
 
         break;
@@ -1905,8 +1904,8 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
-        TRACE_3D(TRC_SCHED_SHUTDOWN_CODE,
-                 d->domain_id, current->vcpu_id, sched_shutdown.reason);
+        TRACE_TIME(TRC_SCHED_SHUTDOWN_CODE, d->domain_id, current->vcpu_id,
+                   sched_shutdown.reason);
 
         spin_lock(&d->shutdown_lock);
         if ( d->shutdown_code == SHUTDOWN_CODE_INVALID )
@@ -2069,7 +2068,7 @@ long sched_adjust(struct domain *d, struct xen_domctl_scheduler_op *op)
     rcu_read_lock(&sched_res_rculock);
 
     if ( (ret = sched_adjust_dom(dom_scheduler(d), d, op)) == 0 )
-        TRACE_1D(TRC_SCHED_ADJDOM, d->domain_id);
+        TRACE_TIME(TRC_SCHED_ADJDOM, d->domain_id);
 
     rcu_read_unlock(&sched_res_rculock);
 
@@ -2164,14 +2163,13 @@ static void sched_switch_units(struct sched_resource *sr,
         sr->curr = next;
         sr->prev = prev;
 
-        TRACE_3D(TRC_SCHED_SWITCH_INFPREV, prev->domain->domain_id,
-                 prev->unit_id, now - prev->state_entry_time);
-        TRACE_4D(TRC_SCHED_SWITCH_INFNEXT, next->domain->domain_id,
-                 next->unit_id,
-                 (next->vcpu_list->runstate.state == RUNSTATE_runnable) ?
-                 (now - next->state_entry_time) : 0, prev->next_time);
-        TRACE_4D(TRC_SCHED_SWITCH, prev->domain->domain_id, prev->unit_id,
-                 next->domain->domain_id, next->unit_id);
+        TRACE_TIME(TRC_SCHED_SWITCH_INFPREV, prev->domain->domain_id,
+                   prev->unit_id, now - prev->state_entry_time);
+        TRACE_TIME(TRC_SCHED_SWITCH_INFNEXT, next->domain->domain_id, next->unit_id,
+                   (next->vcpu_list->runstate.state == RUNSTATE_runnable) ?
+                   (now - next->state_entry_time) : 0, prev->next_time);
+        TRACE_TIME(TRC_SCHED_SWITCH, prev->domain->domain_id, prev->unit_id,
+                   next->domain->domain_id, next->unit_id);
 
         ASSERT(!unit_running(next));
 
@@ -2363,10 +2361,10 @@ static void sched_context_switch(struct vcpu *vprev, struct vcpu *vnext,
 {
     if ( unlikely(vprev == vnext) )
     {
-        TRACE_4D(TRC_SCHED_SWITCH_INFCONT,
-                 vnext->domain->domain_id, vnext->sched_unit->unit_id,
-                 now - vprev->runstate.state_entry_time,
-                 vprev->sched_unit->next_time);
+        TRACE_TIME(TRC_SCHED_SWITCH_INFCONT,
+                   vnext->domain->domain_id, vnext->sched_unit->unit_id,
+                   now - vprev->runstate.state_entry_time,
+                   vprev->sched_unit->next_time);
         sched_context_switched(vprev, vnext);
 
         /*
diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c
index f277fa37a8b1..0ea7145172ee 100644
--- a/xen/common/sched/credit.c
+++ b/xen/common/sched/credit.c
@@ -842,8 +842,7 @@ _csched_cpu_pick(const struct scheduler *ops, const struct sched_unit *unit,
     if ( commit && spc )
        spc->idle_bias = cpu;
 
-    TRACE_3D(TRC_CSCHED_PICKED_CPU, unit->domain->domain_id, unit->unit_id,
-             cpu);
+    TRACE_TIME(TRC_CSCHED_PICKED_CPU, unit->domain->domain_id, unit->unit_id, cpu);
 
     return cpu;
 }
@@ -887,8 +886,8 @@ __csched_unit_acct_start(struct csched_private *prv, struct csched_unit *svc)
         }
     }
 
-    TRACE_3D(TRC_CSCHED_ACCOUNT_START, sdom->dom->domain_id,
-             svc->unit->unit_id, sdom->active_unit_count);
+    TRACE_TIME(TRC_CSCHED_ACCOUNT_START, sdom->dom->domain_id,
+               svc->unit->unit_id, sdom->active_unit_count);
 
     spin_unlock_irqrestore(&prv->lock, flags);
 }
@@ -913,8 +912,8 @@ __csched_unit_acct_stop_locked(struct csched_private *prv,
         list_del_init(&sdom->active_sdom_elem);
     }
 
-    TRACE_3D(TRC_CSCHED_ACCOUNT_STOP, sdom->dom->domain_id,
-             svc->unit->unit_id, sdom->active_unit_count);
+    TRACE_TIME(TRC_CSCHED_ACCOUNT_STOP, sdom->dom->domain_id,
+               svc->unit->unit_id, sdom->active_unit_count);
 }
 
 static void
@@ -937,8 +936,8 @@ csched_unit_acct(struct csched_private *prv, unsigned int cpu)
     if ( svc->pri == CSCHED_PRI_TS_BOOST )
     {
         svc->pri = CSCHED_PRI_TS_UNDER;
-        TRACE_2D(TRC_CSCHED_BOOST_END, svc->sdom->dom->domain_id,
-                 svc->unit->unit_id);
+        TRACE_TIME(TRC_CSCHED_BOOST_END, svc->sdom->dom->domain_id,
+                   svc->unit->unit_id);
     }
 
     /*
@@ -1145,8 +1144,7 @@ csched_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
     if ( !migrating && svc->pri == CSCHED_PRI_TS_UNDER &&
          !test_bit(CSCHED_FLAG_UNIT_PARKED, &svc->flags) )
     {
-        TRACE_2D(TRC_CSCHED_BOOST_START, unit->domain->domain_id,
-                 unit->unit_id);
+        TRACE_TIME(TRC_CSCHED_BOOST_START, unit->domain->domain_id, unit->unit_id);
         SCHED_STAT_CRANK(unit_boost);
         svc->pri = CSCHED_PRI_TS_BOOST;
     }
@@ -1647,8 +1645,8 @@ csched_runq_steal(int peer_cpu, int cpu, int pri, int balance_step)
         if ( __csched_unit_is_migrateable(prv, unit, cpu, cpumask_scratch) )
         {
             /* We got a candidate. Grab it! */
-            TRACE_3D(TRC_CSCHED_STOLEN_UNIT, peer_cpu,
-                     unit->domain->domain_id, unit->unit_id);
+            TRACE_TIME(TRC_CSCHED_STOLEN_UNIT, peer_cpu,
+                       unit->domain->domain_id, unit->unit_id);
             SCHED_UNIT_STAT_CRANK(speer, migrate_q);
             SCHED_STAT_CRANK(migrate_queued);
             runq_remove(speer);
@@ -1751,7 +1749,7 @@ csched_load_balance(struct csched_private *prv, int cpu,
                  */
                 if ( CSCHED_PCPU(peer_cpu)->nr_runnable <= 1 )
                 {
-                    TRACE_2D(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* skipp'n */ 0);
+                    TRACE_TIME(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* skipp'n */ 0);
                     goto next_cpu;
                 }
 
@@ -1767,11 +1765,11 @@ csched_load_balance(struct csched_private *prv, int cpu,
                 if ( !lock )
                 {
                     SCHED_STAT_CRANK(steal_trylock_failed);
-                    TRACE_2D(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* skip */ 0);
+                    TRACE_TIME(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* skip */ 0);
                     goto next_cpu;
                 }
 
-                TRACE_2D(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* checked */ 1);
+                TRACE_TIME(TRC_CSCHED_STEAL_CHECK, peer_cpu, /* checked */ 1);
 
                 /* Any work over there to steal? */
                 speer = cpumask_test_cpu(peer_cpu, online) ?
@@ -1939,7 +1937,7 @@ static void csched_schedule(
         /* Tasklet work (which runs in idle UNIT context) overrides all else. */
         if ( tasklet_work_scheduled )
         {
-            TRACE_0D(TRC_CSCHED_SCHED_TASKLET);
+            TRACE_TIME(TRC_CSCHED_SCHED_TASKLET);
             snext = CSCHED_UNIT(sched_idle_unit(sched_cpu));
             snext->pri = CSCHED_PRI_TS_BOOST;
         }
diff --git a/xen/drivers/cpufreq/utility.c b/xen/drivers/cpufreq/utility.c
index b93895d4dddc..bc3e62bdeadd 100644
--- a/xen/drivers/cpufreq/utility.c
+++ b/xen/drivers/cpufreq/utility.c
@@ -367,7 +367,7 @@ int __cpufreq_driver_target(struct cpufreq_policy *policy,
         retval = alternative_call(cpufreq_driver.target,
                                   policy, target_freq, relation);
         if ( retval == 0 )
-            TRACE_2D(TRC_PM_FREQ_CHANGE, prev_freq/1000, policy->cur/1000);
+            TRACE_TIME(TRC_PM_FREQ_CHANGE, prev_freq / 1000, policy->cur / 1000);
     }
 
     return retval;
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 19:33:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 19:33:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191161.341095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP2Q-0002EZ-4u; Mon, 20 Sep 2021 19:33:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191161.341095; Mon, 20 Sep 2021 19:33:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP2Q-0002EJ-1u; Mon, 20 Sep 2021 19:33:14 +0000
Received: by outflank-mailman (input) for mailman id 191161;
 Mon, 20 Sep 2021 19:33:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSP2P-000298-CJ
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 19:33:13 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 973eb1fe-1a49-11ec-b881-12813bfff9fa;
 Mon, 20 Sep 2021 19:33:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 973eb1fe-1a49-11ec-b881-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632166391;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version;
  bh=98GnhCIOxU7shHKzdhEM8QcX53MmCZ5qPUFcPyknb0s=;
  b=DClUa4xBKYFRzapTkkrXOFwvVJUBkiKUlWM7RZuGA1UpN5RClkmUqo97
   j3myFr7uR0FpCVPjDsuhrLYVvMZAMjlpeZcZJ8tLq7NvsXREvMA5UPnb0
   ngWKhs6yuyRkZhCeCCUou75sfAffEHDyeNR+fXIjg4BxBLp1v682XDOnm
   Y=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 18ghwp3hTIE02UOptrisiyJXOfrmU4+e1UJkVU82XqzW1YU9fFModr+qxHIBOYp8+ApcJtxoWM
 jsaWhlikkwleIMiRevafQlENT9Gu7RO8Y377Qd/jkaDKVRac06Q0tw7mhHFQz36veMlDzkpWnh
 ZGjZEvI26zynzkQ32EVahDUa5n0dJeHKTf6dcskPNXy5JI0p857LYyRs7uHYjM1uI9Uhc5h37a
 3VXy7AJux8LmHLJ9DBqqA20eMJRBK+/e/gx+eNtM5+7eyS7cLmIf9cysJ7EikN/+k3VWc+7q3w
 T8KlXkfW03cWGBtsG7hmkdBz
X-SBRS: 5.1
X-MesageID: 54949821
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:RwOOl6Dip7TyqhVW//Xkw5YqxClBgxIJ4kV8jS/XYbTApDoq0jAGy
 WIdWmjUMquPNGSkKd8gO4Xn9hgFsZHWyoIyQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00o7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/1hKphNxX+
 ux2n5mRdSQ4Er2Lvso3akwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFgGZs15AURJ4yY
 eI2TCReNT7ROCFsHQ42KJAwxNn0tGjgJmgwRFW9+vNsvjm7IBZK+KP2LNPfd9iORMNUtkWVv
 GTL+yL+GB5yHMOb4SqI9DSrnOCntSHmXIMfEpWo+/gsh0ecrkQDBRtTWValrP2Rjk+lR8kZO
 0ES4jApr6U56AqsVNaVdxynolaUsxgEQd1SHuYmrgaXxcL8wSyUG2wFRT5pc8E9uYk9QjlC6
 7OSt4q3X3o16uTTEC/DsOfPxd+vBcQLBTIlRTQIazk82PPmgb0Qnz7fCfBNOpfg27UZBgrML
 yC2QDkW3utI1J5QhvTjpzgrkBr3+cOYFVddChH/Gzv/t1InPtbNi5mAtACDhcusOrp1WbVoU
 JIsoMGY8OlGJpWEjiXlrA4lTezxuqrt3NExhzdS83gdG9aFoCXLkWN4umgWyKJV3iEsI2SBX
 aMrkVkNjKK/xVPzBUONX25UNyjN5fO6fekJq9iONoYeCnSPXFbfoUmCmnJ8L0iyyRNxwMnTy
 L+wcNq2DGZyNEiU5GPtHI8gPUsQ7nlmnwv7HMmjpzz+iOb2TCPFGN8tbQrVBshkvfzsnekg2
 4sGXyd8404EC7OWj+i+2dN7EG3m2lBhVMiq9JQIJrDcSuekcUl4Y8LsLXoaU9QNt8xoei3gp
 xlRg2dUlwjyg2PpMwKPZiwxYb/jR88n/3k6ITYtLRCj3H16OdSj66IWdp0We7g79bM8ka4oH
 qddI8jQUO5STjnn+igGacWvpoJVaxn21xmFODCoYWZjcsc4FRDJ4NLtYiDm6DIKUnisrcI7r
 rD5jlHbTJMPSh5MFsHTbP7znVq9sWJEwLB5XlfSI8kVc0LpqdA4Jyv0h/4xAscNNRScmWfKi
 1fIWU8V/LCfrZU0/d/FgbG/g72oS+YuTFBHG2T77KqtMXWI9GSU3oIdAv2DeirQVT2o9fz6N
 /lV1fz1LNYOgE1O79hnC79uwK8zu4nvqrtdwlg2FXnHdQ32WLZpI33A1shTrKxdgLRevFLuC
 E6I/9BbP5SPOd/kTwFNdFZ0MLzb2KFGgCTW4NQ0PF7+tX1+87ewWElPOwWB1X5GJ7xvPYJ5m
 eostab6MeBkZsbG5jpesh1pyg==
IronPort-HdrOrdr: A9a23:XMuYAapEn5Bd2Xji/GtKwiAaV5opeYIsimQD101hICG8cqSj+f
 xG/c5rrCMc5wxwZJhNo7y90ey7MBbhHP1OkO8s1NWZLWrbUQKTRekIh+bfKn/baknDH4ZmpM
 BdmsNFaeEYY2IUsS+D2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="54949821"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<George.Dunlap@eu.citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>
Subject: [PATCH v2.1 15/12] xen/trace: Drop old trace macros
Date: Mon, 20 Sep 2021 20:33:02 +0100
Message-ID: <20210920193302.20275-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain

With all users updated to the new API, drop the old API.  This includes all of
asm/hvm/trace.h, which allows us to drop some includes.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

v2.1:
 * New
---
 xen/arch/x86/hvm/emulate.c        |   1 -
 xen/arch/x86/hvm/hpet.c           |   1 -
 xen/arch/x86/hvm/hvm.c            |   1 -
 xen/arch/x86/hvm/io.c             |   1 -
 xen/arch/x86/hvm/svm/intr.c       |   1 -
 xen/arch/x86/hvm/svm/svm.c        |   1 -
 xen/arch/x86/hvm/vmx/intr.c       |   1 -
 xen/arch/x86/hvm/vmx/vmx.c        |   1 -
 xen/include/asm-x86/hvm/trace.h   | 114 --------------------------------------
 xen/include/asm-x86/hvm/vmx/vmx.h |   1 -
 xen/include/xen/trace.h           |  35 ------------
 11 files changed, 158 deletions(-)
 delete mode 100644 xen/include/asm-x86/hvm/trace.h

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index f863a19df311..a651dd395b56 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -22,7 +22,6 @@
 #include <asm/hvm/emulate.h>
 #include <asm/hvm/hvm.h>
 #include <asm/hvm/monitor.h>
-#include <asm/hvm/trace.h>
 #include <asm/hvm/support.h>
 #include <asm/hvm/svm/svm.h>
 #include <asm/iocap.h>
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 5873bb8a506d..1736b8f12583 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -19,7 +19,6 @@
 #include <asm/hvm/vpt.h>
 #include <asm/hvm/io.h>
 #include <asm/hvm/support.h>
-#include <asm/hvm/trace.h>
 #include <asm/current.h>
 #include <asm/hpet.h>
 #include <asm/mc146818rtc.h>
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index fa211d14afd8..5a81ffe9dc41 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -62,7 +62,6 @@
 #include <asm/hvm/vpt.h>
 #include <asm/hvm/support.h>
 #include <asm/hvm/cacheattr.h>
-#include <asm/hvm/trace.h>
 #include <asm/hvm/nestedhvm.h>
 #include <asm/hvm/monitor.h>
 #include <asm/hvm/viridian.h>
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 046a8eb4ed1b..9be206488043 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -40,7 +40,6 @@
 #include <asm/hvm/vpt.h>
 #include <asm/hvm/vpic.h>
 #include <asm/hvm/vlapic.h>
-#include <asm/hvm/trace.h>
 #include <asm/hvm/emulate.h>
 #include <public/sched.h>
 #include <xen/iocap.h>
diff --git a/xen/arch/x86/hvm/svm/intr.c b/xen/arch/x86/hvm/svm/intr.c
index 122c9d7fbf14..154258c8b31f 100644
--- a/xen/arch/x86/hvm/svm/intr.c
+++ b/xen/arch/x86/hvm/svm/intr.c
@@ -37,7 +37,6 @@
 #include <xen/kernel.h>
 #include <public/hvm/ioreq.h>
 #include <xen/domain_page.h>
-#include <asm/hvm/trace.h>
 
 static void svm_inject_nmi(struct vcpu *v)
 {
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index b98a2dd71d1e..1f9dadaa173a 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -55,7 +55,6 @@
 #include <asm/x86_emulate.h>
 #include <public/sched.h>
 #include <asm/hvm/vpt.h>
-#include <asm/hvm/trace.h>
 #include <asm/hap.h>
 #include <asm/apic.h>
 #include <asm/debugger.h>
diff --git a/xen/arch/x86/hvm/vmx/intr.c b/xen/arch/x86/hvm/vmx/intr.c
index 303d07dc0a3a..ed17c79996c5 100644
--- a/xen/arch/x86/hvm/vmx/intr.c
+++ b/xen/arch/x86/hvm/vmx/intr.c
@@ -36,7 +36,6 @@
 #include <asm/hvm/vlapic.h>
 #include <asm/hvm/nestedhvm.h>
 #include <public/hvm/ioreq.h>
-#include <asm/hvm/trace.h>
 #include <asm/vm_event.h>
 
 /*
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 47a5df20ebc1..4f99fda8d662 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -48,7 +48,6 @@
 #include <asm/x86_emulate.h>
 #include <asm/hvm/vpt.h>
 #include <public/hvm/save.h>
-#include <asm/hvm/trace.h>
 #include <asm/hvm/monitor.h>
 #include <asm/xenoprof.h>
 #include <asm/debugger.h>
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
deleted file mode 100644
index 696e42eb9499..000000000000
--- a/xen/include/asm-x86/hvm/trace.h
+++ /dev/null
@@ -1,114 +0,0 @@
-#ifndef __ASM_X86_HVM_TRACE_H__
-#define __ASM_X86_HVM_TRACE_H__
-
-#include <xen/trace.h>
-
-#define DEFAULT_HVM_TRACE_ON  1
-#define DEFAULT_HVM_TRACE_OFF 0
-
-#define DEFAULT_HVM_VMSWITCH   DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_PF         DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_INJECT     DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_IO         DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_REGACCESS  DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_MISC       DEFAULT_HVM_TRACE_ON
-#define DEFAULT_HVM_INTR       DEFAULT_HVM_TRACE_ON
-
-#define DO_TRC_HVM_VMENTRY     DEFAULT_HVM_VMSWITCH
-#define DO_TRC_HVM_VMEXIT      DEFAULT_HVM_VMSWITCH
-#define DO_TRC_HVM_VMEXIT64    DEFAULT_HVM_VMSWITCH
-#define DO_TRC_HVM_PF_XEN      DEFAULT_HVM_PF
-#define DO_TRC_HVM_PF_XEN64    DEFAULT_HVM_PF
-#define DO_TRC_HVM_PF_INJECT   DEFAULT_HVM_PF
-#define DO_TRC_HVM_PF_INJECT64 DEFAULT_HVM_PF
-#define DO_TRC_HVM_INJ_EXC     DEFAULT_HVM_INJECT
-#define DO_TRC_HVM_INJ_VIRQ    DEFAULT_HVM_INJECT
-#define DO_TRC_HVM_REINJ_VIRQ  DEFAULT_HVM_INJECT
-#define DO_TRC_HVM_INTR_WINDOW DEFAULT_HVM_INJECT
-#define DO_TRC_HVM_IO_READ     DEFAULT_HVM_IO
-#define DO_TRC_HVM_IO_WRITE    DEFAULT_HVM_IO
-#define DO_TRC_HVM_CR_READ     DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_CR_READ64   DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_CR_WRITE    DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_CR_WRITE64  DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_DR_READ     DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_DR_WRITE    DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_XCR_READ64  DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_XCR_WRITE64 DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_MSR_READ    DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_MSR_WRITE   DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_RDTSC       DEFAULT_HVM_REGACCESS
-#define DO_TRC_HVM_CPUID       DEFAULT_HVM_MISC
-#define DO_TRC_HVM_INTR        DEFAULT_HVM_INTR
-#define DO_TRC_HVM_NMI         DEFAULT_HVM_INTR
-#define DO_TRC_HVM_MCE         DEFAULT_HVM_INTR
-#define DO_TRC_HVM_SMI         DEFAULT_HVM_INTR
-#define DO_TRC_HVM_VMMCALL     DEFAULT_HVM_MISC
-#define DO_TRC_HVM_HLT         DEFAULT_HVM_MISC
-#define DO_TRC_HVM_INVLPG      DEFAULT_HVM_MISC
-#define DO_TRC_HVM_INVLPG64    DEFAULT_HVM_MISC
-#define DO_TRC_HVM_IO_ASSIST   DEFAULT_HVM_MISC
-#define DO_TRC_HVM_MMIO_ASSIST DEFAULT_HVM_MISC
-#define DO_TRC_HVM_CLTS        DEFAULT_HVM_MISC
-#define DO_TRC_HVM_LMSW        DEFAULT_HVM_MISC
-#define DO_TRC_HVM_LMSW64      DEFAULT_HVM_MISC
-#define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC
-#define DO_TRC_HVM_TRAP             DEFAULT_HVM_MISC
-#define DO_TRC_HVM_TRAP_DEBUG       DEFAULT_HVM_MISC
-#define DO_TRC_HVM_VLAPIC           DEFAULT_HVM_MISC
-
-
-#define TRC_PAR_LONG(par) ((par)&0xFFFFFFFF),((par)>>32)
-
-#define TRACE_2_LONG_2D(_e, d1, d2, ...) \
-    TRACE_4D(_e, d1, d2)
-#define TRACE_2_LONG_3D(_e, d1, d2, d3, ...) \
-    TRACE_5D(_e, d1, d2, d3)
-#define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
-    TRACE_6D(_e, d1, d2, d3, d4)
-
-#define HVMTRACE_ND(evt, modifier, cycles, ...)                           \
-    do {                                                                  \
-        if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )               \
-        {                                                                 \
-            uint32_t _d[] = { __VA_ARGS__ };                              \
-            __trace_var(TRC_HVM_ ## evt | (modifier), cycles,             \
-                        sizeof(_d), sizeof(_d) ? _d : NULL);              \
-        }                                                                 \
-    } while(0)
-
-#define HVMTRACE_6D(evt, d1, d2, d3, d4, d5, d6)    \
-    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5, d6)
-#define HVMTRACE_5D(evt, d1, d2, d3, d4, d5)        \
-    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4, d5)
-#define HVMTRACE_4D(evt, d1, d2, d3, d4)            \
-    HVMTRACE_ND(evt, 0, 0, d1, d2, d3, d4)
-#define HVMTRACE_3D(evt, d1, d2, d3)                \
-    HVMTRACE_ND(evt, 0, 0, d1, d2, d3)
-#define HVMTRACE_2D(evt, d1, d2)                    \
-    HVMTRACE_ND(evt, 0, 0, d1, d2)
-#define HVMTRACE_1D(evt, d1)                        \
-    HVMTRACE_ND(evt, 0, 0, d1)
-#define HVMTRACE_0D(evt)                            \
-    HVMTRACE_ND(evt, 0, 0)
-
-#define HVMTRACE_LONG_1D(evt, d1)                  \
-                   HVMTRACE_2D(evt ## 64, (d1) & 0xFFFFFFFF, (d1) >> 32)
-#define HVMTRACE_LONG_2D(evt, d1, d2, ...)              \
-                   HVMTRACE_3D(evt ## 64, d1, d2)
-#define HVMTRACE_LONG_3D(evt, d1, d2, d3, ...)      \
-                   HVMTRACE_4D(evt ## 64, d1, d2, d3)
-#define HVMTRACE_LONG_4D(evt, d1, d2, d3, d4, ...)  \
-                   HVMTRACE_5D(evt ## 64, d1, d2, d3, d4)
-
-#endif /* __ASM_X86_HVM_TRACE_H__ */
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * tab-width: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h
index 85530d2e0e26..2987d5870606 100644
--- a/xen/include/asm-x86/hvm/vmx/vmx.h
+++ b/xen/include/asm-x86/hvm/vmx/vmx.h
@@ -25,7 +25,6 @@
 #include <asm/processor.h>
 #include <asm/i387.h>
 #include <asm/hvm/support.h>
-#include <asm/hvm/trace.h>
 #include <asm/hvm/vmx/vmcs.h>
 
 extern int8_t opt_ept_exec_sp;
diff --git a/xen/include/xen/trace.h b/xen/include/xen/trace.h
index 72c20550f6a6..940be79d7b8e 100644
--- a/xen/include/xen/trace.h
+++ b/xen/include/xen/trace.h
@@ -97,39 +97,4 @@ static inline void __trace_hypercall(uint32_t event, unsigned long op,
 /* Create a trace record with no time included. */
 #define TRACE(_e, ...)      TRACE_INTERNAL(_e, false, ##__VA_ARGS__)
 
-
-/* Convenience macros for calling the trace function. */
-#define TRACE_0D(_e)                            \
-    do {                                        \
-        trace_var(_e, 1, 0, NULL);              \
-    } while ( 0 )
-
-/* Common helper for TRACE_{1..6}D() below. */
-#define TRACE_varD(_e, ...)                             \
-    do {                                                \
-        if ( unlikely(tb_init_done) )                   \
-        {                                               \
-            uint32_t _d[] = { __VA_ARGS__ };            \
-            __trace_var(_e, true, sizeof(_d), _d);      \
-        }                                               \
-    } while ( 0 )
-
-#define TRACE_1D(_e, d1) \
-    TRACE_varD(_e, d1)
-
-#define TRACE_2D(_e, d1, d2) \
-    TRACE_varD(_e, d1, d2)
-
-#define TRACE_3D(_e, d1, d2, d3) \
-    TRACE_varD(_e, d1, d2, d3)
-
-#define TRACE_4D(_e, d1, d2, d3, d4) \
-    TRACE_varD(_e, d1, d2, d3, d4)
-
-#define TRACE_5D(_e, d1, d2, d3, d4, d5) \
-    TRACE_varD(_e, d1, d2, d3, d4, d5)
-
-#define TRACE_6D(_e, d1, d2, d3, d4, d5, d6) \
-    TRACE_varD(_e, d1, d2, d3, d4, d5, d6)
-
 #endif /* __XEN_TRACE_H__ */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 19:41:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 19:41:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191173.341105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP9x-0003vd-VV; Mon, 20 Sep 2021 19:41:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191173.341105; Mon, 20 Sep 2021 19:41:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSP9x-0003vW-Sa; Mon, 20 Sep 2021 19:41:01 +0000
Received: by outflank-mailman (input) for mailman id 191173;
 Mon, 20 Sep 2021 19:41:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F8X/=OK=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSP9w-0003vQ-QA
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 19:41:00 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 44ece320-adcc-4bbb-a0aa-a55416f63440;
 Mon, 20 Sep 2021 19:40:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44ece320-adcc-4bbb-a0aa-a55416f63440
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632166859;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=Q5RosjG14jEcDjoqHjZaARZmPLrzC7t3kj65ZG4GA9g=;
  b=Dy+mwlfDbzTxGZtP8Zo7VcWgLIMBssk+wab63M26Pn/AkFWHCkr/jxpg
   qPKhlSzulWMrQL7MfuMniI4tcH9+TpcC5n5GE2tAxQAt4+r8ObubVFe54
   9uvW9VSZB+DqtHfHe2dbQIwsObADcwOFRhw4oKBjIHnwD0XgxuF/UkAP1
   c=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: URdGVb4GDUGZmnJleHI618yPWG0cdyYQiacFP0IJpJY8+pBE7XzjDkW/ypCObR2QjuRzt/RBYX
 +Kjl2PWqm9jQh4NyWoxoprF/Blj/VuKGDWBpKpX3bcTKvq+EbSeDcHwYtA3ttA5yCP5Ymz3LFK
 +zp6ZEtANWhlkRUlypCgcXPsRSWijLNx494Le+IASKcUE/bTWOEHoYRIFo9BWE/PtBTN77ElSI
 RNZ8483mEoBM1I6Sh3pwQBRF2CM92T/yiKD4ksdvLpnoHzRbsgQwJHa9hZCKGTezM+m+Rb03PT
 /OIfM9Z0VdsSf6hxvFuQ8yGf
X-SBRS: 5.1
X-MesageID: 53173829
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ybOnE6z02LYk0g8q9Md6t+fTwSrEfRIJ4+MujC+fZmUNrF6WrkVWy
 mIWUDqPM//Ya2SgfN13bI/i/UME75SAn9RkTQRv+yAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRh2tUAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt+97m
 e1DtpKec1kgHrLgmfYcVUgDNj4raMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25sXQq2OP
 ZtxhTxHayXKWThJBgYuKallhOqitnTGawdZpwfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krE4mD4DxcyJNGZjz2f/RqEuOjLmi/qXZMII5ex/PVqnV67y3QaDVsdUl7Tiem0jAuyVsxSL
 2QQ+zEytu4i+UqzVN7/Uhak5nmesXY0efBdDuk74wGl0bfP7kCSAW1sZiVadNUsucsyRDor/
 lyEhdXkAXpoqrL9dJ6G3u7K93XoY3FTdDJcI39fJecY3zX9iL453yrBTdhNKvC8rsLZGCPxx
 WHRiTdr0t3/kvU3O7WHEUHv2mz3/8iSH19tvG07TUr+sVgoP9fNi5iArAGBtKcedt7xokyp4
 SBc8/Vy+tziGn1keMale+wLALjhzPKMKjS0bbVHTsR5qmjFF5JOe+ltDNBCyKVBaZ1sldzBO
 ha7VeZtCHh7ZiDCUEOPS9jtY/nGNIC5fTgfahwxUjapSsMrHDJrAQk0PRLAt4wTuBF0zMnTx
 qt3ge7zVC1HWMyLPRKdRvsH0K9D+8zN7TqIHvjGI+Cc+ePGPha9EO5dWHPXN7xRxP7U8W39r
 ocEX+PXmko3bQELSnSOmWLlBQtRdiZT6FGfg5E/S9Nv1SI8STl9VKeOke1+E2Gn9owM/tr1E
 riGchcw4DLCabfvcG1ms1hvN+HiW4hRt3U+MXB+NFqkwSF7M42u8L0eZ908erx+rL5vyvt9T
 v8kfcScA6sQFmSbqmpFNZSt/pZ/cBmLhB6VO3b3ajYIYJM9FRfC/cXpf1Wz+XBWXDa3r8Y3v
 5apyhjfHcgYXw1nAcuPMKCvwlq9sGIzguV3W0eUcNBfdF+1qNphKjDrj+9xKMYJcE2Ryjyf3
 geQIBEZueiS/NNlrIiX3fiJ9t77HfF/E0xWG3jgwYy3bSSKrHC+xYJgUfqTeWyPXm3D56j/N
 /5eyOvxMaNbkQ8S4ZZ8Cbti0Yk3+8Dr++1B1g1hEXjGMwarB7dnLiXU1MVDrPQQlLpQuA/wU
 UOT4NhKf76OPZq9QlIWIQMkaMWF1O0VxWaOvahkfh2i6X8l5qeDXGVTIwKI2X5UI7ZCOY84x
 fss5ZwN4Aulhxt2atuLg0i4LYhXwqDsh0n/iqwnPQ==
IronPort-HdrOrdr: A9a23:l7kHParxj+UfTNl9lIUuG0UaV5oReYIsimQD101hICG8cqSj9v
 xG+85rrSMc6QxhIU3I9urwW5VoLUmyyXcx2/h0AV7AZniBhILLFvAB0WKK+VSJcEeSmtK1l5
 0QFJSWYOeAdmSS5vyb3ODXKbgdKaG8gcWVuds=
X-IronPort-AV: E=Sophos;i="5.85,309,1624334400"; 
   d="scan'208";a="53173829"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2.1 16/12] xen/trace: Restrict CONFIG_TRACEBUFFER to x86 PV
Date: Mon, 20 Sep 2021 20:40:33 +0100
Message-ID: <20210920194033.25998-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

The mapping interface can only be used by x86 PV guests.

This can and should be fixed by changing to an acquire_resource() based
interface, which is compatbile with x86 PVH and ARM dom0's, but until this
happens, don't give the impression of this feature being useable elsewhere.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
---
 xen/common/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index db687b1785e7..6b6f7139e6f0 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -429,6 +429,7 @@ config DTB_FILE
 
 config TRACEBUFFER
 	bool "Enable tracing infrastructure" if EXPERT
+	depends on PV
 	default y
 	---help---
 	  Enable tracing infrastructure and pre-defined tracepoints within Xen.
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 20:03:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 20:03:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191179.341117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSPVj-0006YU-PT; Mon, 20 Sep 2021 20:03:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191179.341117; Mon, 20 Sep 2021 20:03:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSPVj-0006YN-Ly; Mon, 20 Sep 2021 20:03:31 +0000
Received: by outflank-mailman (input) for mailman id 191179;
 Mon, 20 Sep 2021 20:03:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jCl0=OK=tklengyel.com=tamas@srs-us1.protection.inumbo.net>)
 id 1mSPVh-0006YH-LB
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 20:03:29 +0000
Received: from MTA-10-3.privateemail.com (unknown [198.54.127.62])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0484fb3-9dde-4439-b480-1c86ab7bfb28;
 Mon, 20 Sep 2021 20:03:28 +0000 (UTC)
Received: from mta-10.privateemail.com (localhost [127.0.0.1])
 by mta-10.privateemail.com (Postfix) with ESMTP id E279A18000A7
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 16:03:27 -0400 (EDT)
Received: from mail-wr1-f47.google.com (unknown [10.20.151.216])
 by mta-10.privateemail.com (Postfix) with ESMTPA id C9FD118000A2
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 16:03:27 -0400 (EDT)
Received: by mail-wr1-f47.google.com with SMTP id u15so32991155wru.6
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 13:03:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0484fb3-9dde-4439-b480-1c86ab7bfb28
X-Gm-Message-State: AOAM531d71lQXRgY8EBczaGCGZBAt02YdjXURZRXroeMTIeDV7vKnDy7
	SJ/wxJDVjXOD2lCgsj3uGbEZXYS/wJaRCg/jjRE=
X-Google-Smtp-Source: ABdhPJwvpkjCnukkyOje3BkMgoiVojuuPg7T7q9NqAW24I5VHt1EYytFGkYYZjvRRELjVCZLu+OkUB1mSRjSRI9FxAA=
X-Received: by 2002:a05:6000:d2:: with SMTP id q18mr7482121wrx.4.1632168206522;
 Mon, 20 Sep 2021 13:03:26 -0700 (PDT)
MIME-Version: 1.0
References: <8fa3c3e10446f99b75e3d4aef10624cfa2fdf72c.1632145015.git.tamas.lengyel@intel.com>
 <342dd9a2-88b3-34d5-d2ee-b288f8d81f5e@suse.com>
In-Reply-To: <342dd9a2-88b3-34d5-d2ee-b288f8d81f5e@suse.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Mon, 20 Sep 2021 16:02:50 -0400
X-Gmail-Original-Message-ID: <CABfawh=oS2oR5JKc2TB1WODAT8D4qSMEG1paV814CFCEWg8SKg@mail.gmail.com>
Message-ID: <CABfawh=oS2oR5JKc2TB1WODAT8D4qSMEG1paV814CFCEWg8SKg@mail.gmail.com>
Subject: Re: [PATCH v2] x86/mem_sharing: don't lock parent during fork reset
To: Jan Beulich <jbeulich@suse.com>
Cc: Tamas K Lengyel <tamas.lengyel@intel.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
X-Virus-Scanned: ClamAV using ClamSMTP

On Mon, Sep 20, 2021 at 10:15 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 20.09.2021 15:40, Tamas K Lengyel wrote:
> > During fork reset operation the parent domain doesn't need to be gathered using
> > rcu_lock_live_remote_domain_by_id, the fork already has the parent pointer.
> >
> > Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks!


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 20:07:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 20:07:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191185.341128 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSPa3-0007Mc-Bz; Mon, 20 Sep 2021 20:07:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191185.341128; Mon, 20 Sep 2021 20:07:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSPa3-0007MV-8p; Mon, 20 Sep 2021 20:07:59 +0000
Received: by outflank-mailman (input) for mailman id 191185;
 Mon, 20 Sep 2021 20:07:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ujlw=OK=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mSPa2-0007MP-DV
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 20:07:58 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7f35bc1c-13f8-4372-aec9-67c0eaeba79e;
 Mon, 20 Sep 2021 20:07:57 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id i25so73147155lfg.6
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 13:07:57 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id w2sm1146323lfn.298.2021.09.20.13.07.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 20 Sep 2021 13:07:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f35bc1c-13f8-4372-aec9-67c0eaeba79e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=hAx0+cboWTss++cU2Cv+F5KhT5Bih7JomoByhdbei04=;
        b=Ly9VR/XezOqcpC/y6IfrYS+pVg6iQ/e9IYnAJX51T1WmJry3VyVErbugyndwRrlSSu
         UUfit7Fa2zQFJrSIMPcVetXeLfvMmYbtxmn9hWa/p48vLAFj24I9uU43gIkKQBXK9gl/
         sj4LVZTKpZ1+qeDmutU+osi5O638B5XrEYw7/cPLkPTVv+HuRPLj4vTx3ILZ9dsOBoLb
         hg++D7eX9sCdULVaBSiV6AzwQYom0o3D9Ya8UqnxIXMtsfZYibeNJ2r75VZkLp+v266w
         UBeY6Ww9N0WEDjnfcr10X5XNfdtR0SCcvTQqf1JpVEpQv4ldNguhsi2HSoW5TZftpiDw
         RmAA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=hAx0+cboWTss++cU2Cv+F5KhT5Bih7JomoByhdbei04=;
        b=RYLbgJQmHfvUH6tloDSOx77XNCm41jNvIyVc+WdRbbGc65HPG0l1UsdDQmg0KauE5D
         enNJJzzgmjyQJRMP3uB/OUfjjqU4T8bxeQDln38vi9DxYOZGw+HCsbXMlRh3WNYjnzNZ
         ygNVypljzxTrhdFyRiEpwEOrD0sIifaVkoH8jnr7Zh/jgPcpv/Wy1v589K8itjVibyoA
         yMnUOGKrl3yl8Ci7w5+pp9b40g8cffNDAlau1I2J6VMRF33czFmM+iHzoDYO6FMccgNx
         g1YzVBUD+DfvyE/pHV4d2We1eraNsEY0Wv2R8n7/0CRnlzXJ6rYBP9BE9l7OAU9BZIMh
         0rXA==
X-Gm-Message-State: AOAM533Ah/JiVm1x5GDCd7MfqQBfg2oircihaZGvDBfLxgBr/GCA5vR0
	qx/SCuumk+LyfX4GgDCuek8=
X-Google-Smtp-Source: ABdhPJzBtAI3L6L36hEFt9cP4DPGKMoQ+h+jNXNTj9i7/APqkmnPCWU7ZIdt63wXA4XmbqOm2mfVhg==
X-Received: by 2002:a2e:901a:: with SMTP id h26mr23668633ljg.76.1632168475994;
        Mon, 20 Sep 2021 13:07:55 -0700 (PDT)
Subject: Re: [PATCH V2 3/3] libxl/arm: Add handling of extended regions for
 DomU
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 Julien Grall <julien@xen.org>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-4-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109161516350.21985@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <d5016065-fc36-d3bf-ff69-c102ede528dc@gmail.com>
Date: Mon, 20 Sep 2021 23:07:54 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109161516350.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 17.09.21 01:35, Stefano Stabellini wrote:

Hi Stefano

> On Fri, 10 Sep 2021, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The extended region (safe range) is a region of guest physical
>> address space which is unused and could be safely used to create
>> grant/foreign mappings instead of wasting real RAM pages from
>> the domain memory for establishing these mappings.
>>
>> The extended regions are chosen at the domain creation time and
>> advertised to it via "reg" property under hypervisor node in
>> the guest device-tree. As region 0 is reserved for grant table
>> space (always present), the indexes for extended regions are 1...N.
>> If extended regions could not be allocated for some reason,
>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>
>> Please note the following limitations:
>> - The extended region feature is only supported for 64-bit domain.
>> - The ACPI case is not covered.
>>
>> ***
>>
>> The algorithm to choose extended regions for non-direct mapped
>> DomU is simpler in comparison with the algorithm for direct mapped
>> Dom0. As we have a lot of unused space above 4GB, provide single
>> 1GB-aligned region from the second RAM bank taking into the account
>> the maximum supported guest address space size and the amount of
>> memory assigned to the guest. The maximum size of the region is 128GB.
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> Changes since RFC:
>>     - update patch description
>>     - drop uneeded "extended-region" DT property
>>     - clear reg array in finalise_ext_region() and add a TODO
>> ---
>>   tools/libs/light/libxl_arm.c | 89 +++++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 87 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index e3140a6..8c1d9d7 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -615,9 +615,12 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
>>                                 "xen,xen");
>>       if (res) return res;
>>   
>> -    /* reg 0 is grant table space */
>> +    /*
>> +     * reg 0 is a placeholder for grant table space, reg 1 is a placeholder
>> +     * for the extended region.
>> +     */
>>       res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>> -                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
>> +                            2, 0, 0, 0, 0);
>>       if (res) return res;
>>   
>>       /*
>> @@ -1069,6 +1072,86 @@ static void finalise_one_node(libxl__gc *gc, void *fdt, const char *uname,
>>       }
>>   }
>>   
>> +#define ALIGN_UP_TO_GB(x)   (((x) + GB(1) - 1) & (~(GB(1) - 1)))
> Why do we need to align it to 1GB when for Dom0 is aligned to 2MB? I
> think it makes sense to have the same alignment requirement.
Here, unlike with Dom0, we can provide indeed a big region (the maximum 
size is 128GB), so I decided to use maximum block size for the alignment.
ok, I will use 2MB alignment to be consistent will Dom0.


>
>
>> +
>> +#define EXT_REGION_SIZE   GB(128)
> The region needs to be described in xen/include/public/arch-arm.h like
> GUEST_RAM1_BASE/SIZE.

ok, will do


>
>
>> +static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image *dom)
>> +{
>> +    void *fdt = dom->devicetree_blob;
>> +    uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
>> +    be32 *cells = &regs[0];
>> +    uint64_t region_size = 0, region_base, bank1end_align, bank1end_max;
>> +    uint32_t gpaddr_bits;
>> +    libxl_physinfo info;
>> +    int offset, rc;
>> +
>> +    offset = fdt_path_offset(fdt, "/hypervisor");
>> +    assert(offset > 0);
>> +
>> +    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
>> +        LOG(WARN, "The extended region is only supported for 64-bit guest");
>> +        goto out;
>> +    }
>> +
>> +    rc = libxl_get_physinfo(CTX, &info);
>> +    assert(!rc);
>> +
>> +    gpaddr_bits = info.gpaddr_bits;
>> +    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
>> +
>> +    /*
>> +     * Try to allocate single 1GB-aligned extended region from the second RAM
>> +     * bank (above 4GB) taking into the account the maximum supported guest
>> +     * address space size and the amount of memory assigned to the guest.
>> +     * The maximum size of the region is 128GB.
>> +     */
>> +    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
>> +    bank1end_align = GUEST_RAM1_BASE +
>> +        ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << XC_PAGE_SHIFT);
>> +
>> +    if (bank1end_max <= bank1end_align) {
>> +        LOG(WARN, "The extended region cannot be allocated, not enough space");
>> +        goto out;
>> +    }
>> +
>> +    if (bank1end_max - bank1end_align > EXT_REGION_SIZE) {
>> +        region_base = bank1end_max - EXT_REGION_SIZE;
>> +        region_size = EXT_REGION_SIZE;
>> +    } else {
>> +        region_base = bank1end_align;
>> +        region_size = bank1end_max - bank1end_align;
>> +    }
>> +
>> +out:
>> +    /*
>> +     * The first region for grant table space must be always present.
>> +     * If we managed to allocate the extended region then insert it as
>> +     * a second region.
>> +     * TODO If we failed to allocate the region, we end up inserting
>> +     * zero-sized region. This is because we don't know in advance when
>> +     * creating hypervisor node whether it would be possible to allocate
>> +     * a region, but we have to create a placeholder anyway. The Linux driver
>> +     * is able to deal with by checking the region size. We cannot choose
>> +     * a region when creating hypervisor node because the guest memory layout
>> +     * is not know at that moment (and dom->rambank_size[1] is empty).
>> +     * We need to find a way not to expose invalid regions.
>> +     */
> This is not great -- it would be barely spec compliant.

Absolutely agree.


>   
>
> When make_hypervisor_node is called we know the max memory of the guest
> as build_info.max_memkb should be populate, right?

Right. Just a small change to pass build_info to make_hypervisor_node() 
is needed.


>
> If so, we could at least detect whether we can have an extended region
> (if not caculate the exact start address) from make_hypervisor_node.
>
> total_guest_memory = build_info.max_memkb * 1024;
> rambank1_approx = total_guest_memory - GUEST_RAM0_SIZE;
> extended_region_size = GUEST_RAM1_SIZE - rambank1_approx;
>
> if (extended_region_size >= MIN_EXT_REGION_SIZE)
>      allocate_ext_region
Good point! I will recheck that. I would prefer avoid spreading extended 
region handling (introduce finalise_ext_region())
and do everything from the make_hypervisor_node().


>
>
>> +    memset(regs, 0, sizeof(regs));
>> +    set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>> +              GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
>> +    if (region_size > 0) {
> I think we want to check against a minimum amount rather than 0. Maybe 64MB?

Sounds reasonable, will update.


>
>
>> +        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
>> +            region_base, region_base + region_size);
>> +
>> +        set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>> +                  region_base, region_size);
>> +    }
>> +
>> +    rc = fdt_setprop_inplace(fdt, offset, "reg", regs, sizeof(regs));
>> +    assert(!rc);
>> +}
>> +
>>   int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
>>                                                  uint32_t domid,
>>                                                  libxl_domain_config *d_config,
>> @@ -1109,6 +1192,8 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
>>   
>>       }
>>   
>> +    finalise_ext_region(gc, dom);
>> +
>>       for (i = 0; i < GUEST_RAM_BANKS; i++) {
>>           const uint64_t size = (uint64_t)dom->rambank_size[i] << XC_PAGE_SHIFT;
>>   
>> -- 
>> 2.7.4


Thank you.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 20 22:05:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 22:05:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191206.341139 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSRPV-0002ke-4U; Mon, 20 Sep 2021 22:05:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191206.341139; Mon, 20 Sep 2021 22:05:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSRPV-0002kX-1f; Mon, 20 Sep 2021 22:05:13 +0000
Received: by outflank-mailman (input) for mailman id 191206;
 Mon, 20 Sep 2021 22:05:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSRPT-0002kN-Qy; Mon, 20 Sep 2021 22:05:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSRPT-0000Ss-Ff; Mon, 20 Sep 2021 22:05:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSRPT-000500-3i; Mon, 20 Sep 2021 22:05:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSRPT-0003Zi-3A; Mon, 20 Sep 2021 22:05:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=owNQdE+COHnUDq8gEzRsgnQqwzg6VklO+kJyGyobw0U=; b=4RXX6cSRVEOqnhVmI9UuvlwP7o
	d/JUOnJl3P7YGSKJy/1Z4xkpVsrKdR1wQkZfuxM1lq+p3gwVnerd+4NtUGFbRQRK9j8bgrB0WmZoy
	91tjExmZZv9bjWKlIiYPlVDaIShCu2mWdXGLrHxUnIw1fjr3aQBe6gtfDX6DD8Bl9oAs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165129-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165129: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-arm64-arm64-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=7adb961995a3744f51396502b33ad04a56a317c3
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 22:05:11 +0000

flight 165129 qemu-mainline real [real]
flight 165132 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165129/
http://logs.test-lab.xenproject.org/osstest/logs/165132/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-vhd 17 guest-start/debian.repeat fail pass in 165132-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                7adb961995a3744f51396502b33ad04a56a317c3
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    9 days
Failing since        164967  2021-09-13 13:06:52 Z    7 days   15 attempts
Testing same since   165129  2021-09-20 09:28:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  John Arbuckle <programmingkidx@gmail.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3210 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 22:57:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 22:57:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191225.341156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSE4-0008Ol-6k; Mon, 20 Sep 2021 22:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191225.341156; Mon, 20 Sep 2021 22:57:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSE4-0008Oe-3j; Mon, 20 Sep 2021 22:57:28 +0000
Received: by outflank-mailman (input) for mailman id 191225;
 Mon, 20 Sep 2021 22:57:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSSE3-0008OU-53; Mon, 20 Sep 2021 22:57:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSSE2-0001Hz-TC; Mon, 20 Sep 2021 22:57:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSSE2-0007TQ-Ho; Mon, 20 Sep 2021 22:57:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSSE2-0006M8-HL; Mon, 20 Sep 2021 22:57:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EEEYBW5GQIjg9sz2SyJtXXQxshTlslLwZilZ9Y2PeyM=; b=fbkJe/iUSF48ev5twrQ28DCMN7
	ImD+Q51fKHG++usbPCiilBAU1FopYGEqduIAyPeq8OutAMRTTu3Lm/p1b/mEp1kgpH0HHVQ6/M2IK
	5RGXgUMxg3GFsOy+LBk4qPMrhEF8Gs7FHYP6PfcwV1kqypOu4AvnEh1HcyZScnbbjKGw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165130-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165130: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-armhf-armhf-xl-vhd:guest-start.2:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-localmigrate/x10:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e4e737bb5c170df6135a127739a9e6148ee3da82
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 20 Sep 2021 22:57:26 +0000

flight 165130 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165130/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail in 165125 REGR. vs. 152332
 test-armhf-armhf-xl-vhd      18 guest-start.2  fail in 165125 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-xsm      13 debian-fixup               fail pass in 165125
 test-armhf-armhf-xl-credit1  14 guest-start                fail pass in 165125
 test-armhf-armhf-xl-vhd       8 xen-boot                   fail pass in 165125

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd11-amd64 19 guest-localmigrate/x10 fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-arm64-arm64-xl-xsm     15 migrate-support-check fail in 165125 never pass
 test-arm64-arm64-xl-xsm 16 saverestore-support-check fail in 165125 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 165125 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 165125 never pass
 test-armhf-armhf-xl-vhd     14 migrate-support-check fail in 165125 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 165125 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                e4e737bb5c170df6135a127739a9e6148ee3da82
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  416 days
Failing since        152366  2020-08-01 20:49:34 Z  415 days  724 attempts
Testing same since   165125  2021-09-20 01:42:52 Z    0 days    2 attempts

------------------------------------------------------------
7321 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2262655 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 23:16:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 23:16:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191234.341171 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSWH-0002TJ-U1; Mon, 20 Sep 2021 23:16:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191234.341171; Mon, 20 Sep 2021 23:16:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSWH-0002TC-R8; Mon, 20 Sep 2021 23:16:17 +0000
Received: by outflank-mailman (input) for mailman id 191234;
 Mon, 20 Sep 2021 23:16:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I4IA=OK=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSSWG-0002T6-Uu
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 23:16:16 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a795e827-7cfa-45a1-82af-2d2fb623ec1a;
 Mon, 20 Sep 2021 23:16:15 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 07D2961211;
 Mon, 20 Sep 2021 23:16:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a795e827-7cfa-45a1-82af-2d2fb623ec1a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632179774;
	bh=lWL8fnGNKTXM4Bi+yAVp1+gpNMDcF8Ek0Z+rLqzdBB8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=YEnp5FBXjtQsJausdghHKuLJItlSCUlpC2G+ai9iRGx9pGdDBniFNbpSBrobhC0Cz
	 udviL2kOafQidCQaV1xpH53ha4GYfV8xaqKhTHHg3aM/g28kQEiFmht2+HiBCq5nFB
	 GgqQa2vfu0+UORnW/8LvucewzMCrF3iKcRVzst9Jn+OHtBs8+6XhhDJtYnupvsaYhP
	 CveIWnFLn9iU64Jv8J+Z3MLhIpBwWNSgNH2rkTgxkZYjz0PtV91H2V2cvyvvQoWGEm
	 XBA5u+LZ0fERDYZguxuCh5HEbxpt2axAUFxdc7bj27chIwpKeS7uCQCUqleJgSI1CS
	 dNpWHWgV3/m6g==
Date: Mon, 20 Sep 2021 16:16:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>, 
    Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
In-Reply-To: <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
Message-ID: <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
References: <20210917130123.1764493-1-andr2000@gmail.com> <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s> <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com> <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1040504615-1632174404=:17979"
Content-ID: <alpine.DEB.2.21.2109201454550.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1040504615-1632174404=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109201454551.17979@sstabellini-ThinkPad-T480s>

On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
> On 20.09.21 14:30, Juergen Gross wrote:
> > On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
> >> Hello, Stefano!
> >>
> >> On 18.09.21 00:45, Stefano Stabellini wrote:
> >>> Hi Oleksandr,
> >>>
> >>> Why do you want to enable pciback on ARM? Is it only to "disable" a PCI
> >>> device in Dom0 so that it can be safely assigned to a DomU?
> >> Not only that
> >>>
> >>> I am asking because actually I don't think we want to enable the PV PCI
> >>> backend feature of pciback on ARM, right? That would clash with the PCI
> >>> assignment work you have been doing in Xen. They couldn't both work at
> >>> the same time.
> >> Correct, it is not used
> >>>
> >>> If we only need pciback to "park" a device in Dom0, wouldn't it be
> >>> possible and better to use pci-stub instead?
> >>
> >> Not only that, so pci-stub is not enough
> >>
> >> The functionality which is implemented by the pciback and the toolstack
> >> and which is relevant/missing/needed for ARM:
> >>
> >> 1. pciback is used as a database for assignable PCI devices, e.g. xl
> >>      pci-assignable-{add|remove|list} manipulates that list. So, whenever the
> >>      toolstack needs to know which PCI devices can be passed through it reads
> >>      that from the relevant sysfs entries of the pciback.
> >>
> >> 2. pciback is used to hold the unbound PCI devices, e.g. when passing through
> >>      a PCI device it needs to be unbound from the relevant device driver and bound
> >>      to pciback (strictly speaking it is not required that the device is bound to
> >>      pciback, but pciback is again used as a database of the passed through PCI
> >>      devices, so we can re-bind the devices back to their original drivers when
> >>      guest domain shuts down)
> >>
> >> 3. Device reset
> >>
> >> We have previously discussed on xen-devel ML possible solutions to that as from the
> >> above we see that pciback functionality is going to be only partially used on Arm.
> >>
> >> Please see [1] and [2]:
> >>
> >> 1. It is not acceptable to manage the assignable list in Xen itself
> >>
> >> 2. pciback can be split into two parts: PCI assignable/bind/reset handling and
> >> the rest like vPCI etc.
> >>
> >> 3. pcifront is not used on Arm
> >
> > It is neither in x86 PVH/HVM guests.
> Didn't know that, thank you for pointing
> >
> >> So, limited use of the pciback is one of the bricks used to enable PCI passthrough
> >> on Arm. It was enough to just re-structure the driver and have it run on Arm to achieve
> >> all the goals above.
> >>
> >> If we still think it is desirable to break the pciback driver into "common" and "pcifront specific"
> >> parts then it can be done, yet the patch is going to be the very first brick in that building.
> >
> > Doing this split should be done, as the pcifront specific part could be
> > omitted on x86, too, in case no PV guests using PCI passthrough have to
> > be supported.
> Agree, that the final solution should have the driver split
> >
> >> So, I think this patch is still going to be needed besides which direction we take.
> >
> > Some kind of this patch, yes. It might look different in case the split
> > is done first.
> >
> > I don't mind doing it in either sequence.
> >
> With this patch we have Arm on the same page as the above mentioned x86 guests,
> 
> e.g. the driver has unused code, but yet allows Arm to function now.
> 
> At this stage of PCI passthrough on Arm it is yet enough. Long term, when
> 
> the driver gets split, Arm will benefit from that split too, but unfortunately I do not
> 
> have enough bandwidth for that piece of work at the moment.

That's fair and I don't want to scope-creep this simple patch asking for
an enormous rework. At the same time I don't think we should enable the
whole of pciback on ARM because it would be erroneous and confusing.

I am wonder if there is a simple:

if (!xen_pv_domain())
    return;

That we could add in a couple of places in pciback to stop it from
initializing the parts we don't care about. Something along these lines
(untested and probably incomplete).

What do you guys think?


diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index da34ce85dc88..991ba0a9b359 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -15,6 +15,7 @@
 #include <xen/xenbus.h>
 #include <xen/events.h>
 #include <xen/pci.h>
+#include <xen/xen.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
@@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xenbus_device *dev,
 				const struct xenbus_device_id *id)
 {
 	int err = 0;
-	struct xen_pcibk_device *pdev = alloc_pdev(dev);
+	struct xen_pcibk_device *pdev;
+
+	if (!xen_pv_domain())
+		return 0;
 
+	pdev = alloc_pdev(dev);
 	if (pdev == NULL) {
 		err = -ENOMEM;
 		xenbus_dev_fatal(dev, err,
@@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
 
 int __init xen_pcibk_xenbus_register(void)
 {
+	if (!xen_pv_domain())
+		return 0;
+
 	xen_pcibk_backend = &xen_pcibk_vpci_backend;
 	if (passthrough)
 		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
@@ -752,5 +760,7 @@ int __init xen_pcibk_xenbus_register(void)
 
 void __exit xen_pcibk_xenbus_unregister(void)
 {
+	if (!xen_pv_domain())
+		return;
 	xenbus_unregister_driver(&xen_pcibk_driver);
 }
--8323329-1040504615-1632174404=:17979--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 23:21:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 23:21:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191240.341181 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSbE-0003yN-Gg; Mon, 20 Sep 2021 23:21:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191240.341181; Mon, 20 Sep 2021 23:21:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSSbE-0003yG-DY; Mon, 20 Sep 2021 23:21:24 +0000
Received: by outflank-mailman (input) for mailman id 191240;
 Mon, 20 Sep 2021 23:21:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I4IA=OK=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSSbD-0003yA-6A
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 23:21:23 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 76e843a0-1a69-11ec-b88a-12813bfff9fa;
 Mon, 20 Sep 2021 23:21:21 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7069E61211;
 Mon, 20 Sep 2021 23:21:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76e843a0-1a69-11ec-b88a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632180080;
	bh=XpVW6WlJvF/Ucdqmeqd1pM/rlpsa+YufiWqxzvIWMNA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=k4mjxMb0FxUH3WwQOwklKO8xF/SVty+wsJncMdVPRnI48QyobxLRcniPBz8TBX/As
	 cIh0r7JXjaAXa6pgvPt6XrwqNRFbmOgnw0FDnFoa3RCFUZX0g+7Us4ArDU5Ql2+UVs
	 FVhWRSWTGXmJ9KWPsBilueOkVbqgYNFiHC8dw59PZMcG+AAQGqggFg4TBA1Lb8HjFG
	 kh9nBgFYHaNDo4SLU3t6cGXPRDQtLBOxeUiJ6DsUi3is5twRG32jGJUV8Lk/0VmOBR
	 lZGRDuujo9itkaUGNO/wIx3LXylgUNKm+qDp7h9zve69Jj9SLiwG5bE/PXT9q6MTWS
	 h76asGDuNBYRg==
Date: Mon, 20 Sep 2021 16:21:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
Message-ID: <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s> <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org> <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1942625641-1632180080=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1942625641-1632180080=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sun, 19 Sep 2021, Oleksandr wrote:
> > On 18/09/2021 03:37, Stefano Stabellini wrote:
> > > On Fri, 17 Sep 2021, Stefano Stabellini wrote:
> > > > On Fri, 17 Sep 2021, Oleksandr wrote:
> > > > > > > +
> > > > > > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > > > > > +
> > > > > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > > > > +    if ( !unalloc_mem )
> > > > > > > +        return -ENOMEM;
> > > > > > > +
> > > > > > > +    /* Start with all available RAM */
> > > > > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > > > > +    {
> > > > > > > +        start = bootinfo.mem.bank[i].start;
> > > > > > > +        end = bootinfo.mem.bank[i].start +
> > > > > > > bootinfo.mem.bank[i].size - 1;
> > > > > > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > > > > > +        if ( res )
> > > > > > > +        {
> > > > > > > +            printk(XENLOG_ERR "Failed to add:
> > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > +                   start, end);
> > > > > > > +            goto out;
> > > > > > > +        }
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    /* Remove RAM assigned to Dom0 */
> > > > > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > > > > +    {
> > > > > > > +        start = assign_mem->bank[i].start;
> > > > > > > +        end = assign_mem->bank[i].start +
> > > > > > > assign_mem->bank[i].size - 1;
> > > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > +        if ( res )
> > > > > > > +        {
> > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > +                   start, end);
> > > > > > > +            goto out;
> > > > > > > +        }
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    /* Remove reserved-memory regions */
> > > > > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > > > > > +    {
> > > > > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > +        if ( res )
> > > > > > > +        {
> > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > +                   start, end);
> > > > > > > +            goto out;
> > > > > > > +        }
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    /* Remove grant table region */
> > > > > > > +    start = kinfo->gnttab_start;
> > > > > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > +    if ( res )
> > > > > > > +    {
> > > > > > > +        printk(XENLOG_ERR "Failed to remove:
> > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > +               start, end);
> > > > > > > +        goto out;
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    start = EXT_REGION_START;
> > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > > > > +                                 add_ext_regions, ext_regions);
> > > > > > > +    if ( res )
> > > > > > > +        ext_regions->nr_banks = 0;
> > > > > > > +    else if ( !ext_regions->nr_banks )
> > > > > > > +        res = -ENOENT;
> > > > > > > +
> > > > > > > +out:
> > > > > > > +    rangeset_destroy(unalloc_mem);
> > > > > > > +
> > > > > > > +    return res;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static int __init find_memory_holes(const struct kernel_info
> > > > > > > *kinfo,
> > > > > > > +                                    struct meminfo *ext_regions)
> > > > > > > +{
> > > > > > > +    struct dt_device_node *np;
> > > > > > > +    struct rangeset *mem_holes;
> > > > > > > +    paddr_t start, end;
> > > > > > > +    unsigned int i;
> > > > > > > +    int res;
> > > > > > > +
> > > > > > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > > > > > +
> > > > > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > > > > +    if ( !mem_holes )
> > > > > > > +        return -ENOMEM;
> > > > > > > +
> > > > > > > +    /* Start with maximum possible addressable physical memory
> > > > > > > range */
> > > > > > > +    start = EXT_REGION_START;
> > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > > > > +    if ( res )
> > > > > > > +    {
> > > > > > > +        printk(XENLOG_ERR "Failed to add:
> > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > +               start, end);
> > > > > > > +        goto out;
> > > > > > > +    }
> > > > > > > +
> > > > > > > +    /* Remove all regions described by "reg" property (MMIO, RAM,
> > > > > > > etc) */
> > > > > > 
> > > > > > Well... The loop below is not going to handle all the regions
> > > > > > described in
> > > > > > the property "reg". Instead, it will cover a subset of "reg" where
> > > > > > the
> > > > > > memory is addressable.
> > > > > 
> > > > > As I understand, we are only interested in subset of "reg" where the
> > > > > memory is
> > > > > addressable.
> > > > > 
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > You will also need to cover "ranges" that will describe the BARs for
> > > > > > the PCI
> > > > > > devices.
> > > > > Good point.
> > > > 
> > > > Yes, very good point!
> > > > 
> > > > 
> > > > > Could you please clarify how to recognize whether it is a PCI
> > > > > device as long as PCI support is not merged? Or just to find any
> > > > > device nodes
> > > > > with non-empty "ranges" property
> > > > > and retrieve addresses?
> > > > 
> > > > Normally any bus can have a ranges property with the aperture and
> > > > possible address translations, including /amba (compatible =
> > > > "simple-bus"). However, in these cases dt_device_get_address already
> > > > takes care of it, see xen/common/device_tree.c:dt_device_get_address.
> > > > 
> > > > The PCI bus is special for 2 reasons:
> > > > - the ranges property has a different format
> > > > - the bus is hot-pluggable
> > > > 
> > > > So I think the only one that we need to treat specially is PCI.
> > > > 
> > > > As far as I am aware PCI is the only bus (or maybe just the only bus
> > > > that we support?) where ranges means the aperture.
> > > 
> > > Now that I think about this, there is another "hotpluggable" scenario we
> > > need to think about:
> > > 
> > > [1] https://marc.info/?l=xen-devel&m=163056546214978
> > > 
> > > Xilinx devices have FPGA regions with apertures currently not described
> > > in device tree, where things can programmed in PL at runtime making new
> > > devices appear with new MMIO regions out of thin air.
> > > 
> > > Now let me start by saying that yes, the entire programmable region
> > > aperture could probably be described in device tree, however, in
> > > reality it is not currently done in any of the device trees we use
> > > (including the upstream device trees in linux.git).
> > 
> > This is rather annoying, but not unheard. There are a couple of platforms
> > where the MMIOs are not fully described in the DT.
> > 
> > In fact, we have a callback 'specific_mappings' which create additional
> > mappings (e.g. on the omap5) for dom0.
> > 
> > > 
> > > So, we have a problem :-(
> > > 
> > > 
> > > I can work toward getting the right info on device tree, but in reality
> > > that is going to take time and for now the device tree doesn't have the
> > > FPGA aperture in it. So if we accept this series as is, it is going to
> > > stop features like [1] from working. >
> > > If we cannot come up with any better plans, I think it would be better
> > > to drop find_memory_holes, only rely on find_unallocated_memory even
> > > when the IOMMU is on. One idea is that we could add on top of the
> > > regions found by find_unallocated_memory any MMIO regions marked as
> > > xen,passthrough: they are safe because they are not going to dom0 anyway.
> > 
> > (Oleksandr, it looks like some rationale about the different approach is
> > missing in the commit message. Can you add it?)
> 
> Yes sure, but let me please clarify what is different approach in this
> context. Is it to *also* take into the account MMIO regions of the devices for
> passthrough for case when IOMMU is off (in addition to unallocated memory)? If
> yes, I wonder whether we will gain much with that according to that device's
> MMIO regions are usually not big enough and we stick to allocate extended
> regions with bigger size (> 64MB).

That's fair enough. There are a couple of counter examples where the
MMIO regions for the device to assign are quite large, for instance a
GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
but maybe they are not that common. I am not sure if it is worth
scanning the tree for xen,passthrough regions every time at boot for
this.
--8323329-1942625641-1632180080=:17979--


From xen-devel-bounces@lists.xenproject.org Mon Sep 20 23:56:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 20 Sep 2021 23:56:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191246.341193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mST8a-0007cR-8m; Mon, 20 Sep 2021 23:55:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191246.341193; Mon, 20 Sep 2021 23:55:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mST8a-0007cK-5n; Mon, 20 Sep 2021 23:55:52 +0000
Received: by outflank-mailman (input) for mailman id 191246;
 Mon, 20 Sep 2021 23:55:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=I4IA=OK=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mST8Y-0007cE-5I
 for xen-devel@lists.xenproject.org; Mon, 20 Sep 2021 23:55:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 23a5b762-d36d-4b25-ba8a-cfefd9cbb46d;
 Mon, 20 Sep 2021 23:55:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8597E60F41;
 Mon, 20 Sep 2021 23:55:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23a5b762-d36d-4b25-ba8a-cfefd9cbb46d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632182147;
	bh=rje7DP+kJuYTF0X72BmoG6z/gJaTKGY2mv9TxMd0kQA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uAgWlaXg0U84+2PCvVTrahrB9Q3xdnMqamL1o7TeH4d/7RDMKtp3GDFFzVxrMNCIT
	 qoiepuEA1cekmqSsXl0IzI2xLCbKcQs0AbsE4+zYhJd4cuA2+qNGmmDUFVUqsOpVAK
	 CEIMuz9ghuM4XTyhdO8kvEfzTiZDyUEC8N30snzcvII+xVGjCMmgcyT33LGZRpscOm
	 TGxKVaD68hFvJz2Ckka7gqooBehKEqMoTMsghlrqT/8FKyH6kkZ/3C5ceqG8o/3RkE
	 MyPobiDLTrVJgIFEr+F21JPGl2zMVlHdqbyYPfxs5YK/3kq0VBIIwz6eiJQcq5ozYD
	 BH/oaKoq54fVw==
Date: Mon, 20 Sep 2021 16:55:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr <olekstysh@gmail.com>, xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
Message-ID: <alpine.DEB.2.21.2109201621280.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s> <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-930583891-1632180475=:17979"
Content-ID: <alpine.DEB.2.21.2109201628200.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-930583891-1632180475=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109201628201.17979@sstabellini-ThinkPad-T480s>

On Sun, 19 Sep 2021, Julien Grall wrote:
> On 18/09/2021 03:37, Stefano Stabellini wrote:
> > On Fri, 17 Sep 2021, Stefano Stabellini wrote:
> > > On Fri, 17 Sep 2021, Oleksandr wrote:
> > > > > > +
> > > > > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > > > > +
> > > > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > > > +    if ( !unalloc_mem )
> > > > > > +        return -ENOMEM;
> > > > > > +
> > > > > > +    /* Start with all available RAM */
> > > > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > > > +    {
> > > > > > +        start = bootinfo.mem.bank[i].start;
> > > > > > +        end = bootinfo.mem.bank[i].start +
> > > > > > bootinfo.mem.bank[i].size - 1;
> > > > > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > > > > +        if ( res )
> > > > > > +        {
> > > > > > +            printk(XENLOG_ERR "Failed to add:
> > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > +                   start, end);
> > > > > > +            goto out;
> > > > > > +        }
> > > > > > +    }
> > > > > > +
> > > > > > +    /* Remove RAM assigned to Dom0 */
> > > > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > > > +    {
> > > > > > +        start = assign_mem->bank[i].start;
> > > > > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size
> > > > > > - 1;
> > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > +        if ( res )
> > > > > > +        {
> > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > +                   start, end);
> > > > > > +            goto out;
> > > > > > +        }
> > > > > > +    }
> > > > > > +
> > > > > > +    /* Remove reserved-memory regions */
> > > > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > > > > +    {
> > > > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > +        if ( res )
> > > > > > +        {
> > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > +                   start, end);
> > > > > > +            goto out;
> > > > > > +        }
> > > > > > +    }
> > > > > > +
> > > > > > +    /* Remove grant table region */
> > > > > > +    start = kinfo->gnttab_start;
> > > > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > +    if ( res )
> > > > > > +    {
> > > > > > +        printk(XENLOG_ERR "Failed to remove:
> > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > +               start, end);
> > > > > > +        goto out;
> > > > > > +    }
> > > > > > +
> > > > > > +    start = EXT_REGION_START;
> > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > > > +                                 add_ext_regions, ext_regions);
> > > > > > +    if ( res )
> > > > > > +        ext_regions->nr_banks = 0;
> > > > > > +    else if ( !ext_regions->nr_banks )
> > > > > > +        res = -ENOENT;
> > > > > > +
> > > > > > +out:
> > > > > > +    rangeset_destroy(unalloc_mem);
> > > > > > +
> > > > > > +    return res;
> > > > > > +}
> > > > > > +
> > > > > > +static int __init find_memory_holes(const struct kernel_info
> > > > > > *kinfo,
> > > > > > +                                    struct meminfo *ext_regions)
> > > > > > +{
> > > > > > +    struct dt_device_node *np;
> > > > > > +    struct rangeset *mem_holes;
> > > > > > +    paddr_t start, end;
> > > > > > +    unsigned int i;
> > > > > > +    int res;
> > > > > > +
> > > > > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > > > > +
> > > > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > > > +    if ( !mem_holes )
> > > > > > +        return -ENOMEM;
> > > > > > +
> > > > > > +    /* Start with maximum possible addressable physical memory
> > > > > > range */
> > > > > > +    start = EXT_REGION_START;
> > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > > > +    if ( res )
> > > > > > +    {
> > > > > > +        printk(XENLOG_ERR "Failed to add:
> > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > +               start, end);
> > > > > > +        goto out;
> > > > > > +    }
> > > > > > +
> > > > > > +    /* Remove all regions described by "reg" property (MMIO, RAM,
> > > > > > etc) */
> > > > > 
> > > > > Well... The loop below is not going to handle all the regions
> > > > > described in
> > > > > the property "reg". Instead, it will cover a subset of "reg" where the
> > > > > memory is addressable.
> > > > 
> > > > As I understand, we are only interested in subset of "reg" where the
> > > > memory is
> > > > addressable.
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > You will also need to cover "ranges" that will describe the BARs for
> > > > > the PCI
> > > > > devices.
> > > > Good point.
> > > 
> > > Yes, very good point!
> > > 
> > > 
> > > > Could you please clarify how to recognize whether it is a PCI
> > > > device as long as PCI support is not merged? Or just to find any device
> > > > nodes
> > > > with non-empty "ranges" property
> > > > and retrieve addresses?
> > > 
> > > Normally any bus can have a ranges property with the aperture and
> > > possible address translations, including /amba (compatible =
> > > "simple-bus"). However, in these cases dt_device_get_address already
> > > takes care of it, see xen/common/device_tree.c:dt_device_get_address.
> > > 
> > > The PCI bus is special for 2 reasons:
> > > - the ranges property has a different format
> > > - the bus is hot-pluggable
> > > 
> > > So I think the only one that we need to treat specially is PCI.
> > > 
> > > As far as I am aware PCI is the only bus (or maybe just the only bus
> > > that we support?) where ranges means the aperture.
> > 
> > Now that I think about this, there is another "hotpluggable" scenario we
> > need to think about:
> > 
> > [1] https://marc.info/?l=xen-devel&m=163056546214978
> > 
> > Xilinx devices have FPGA regions with apertures currently not described
> > in device tree, where things can programmed in PL at runtime making new
> > devices appear with new MMIO regions out of thin air.
> > 
> > Now let me start by saying that yes, the entire programmable region
> > aperture could probably be described in device tree, however, in
> > reality it is not currently done in any of the device trees we use
> > (including the upstream device trees in linux.git).
> 
> This is rather annoying, but not unheard. There are a couple of platforms
> where the MMIOs are not fully described in the DT.
> 
> In fact, we have a callback 'specific_mappings' which create additional
> mappings (e.g. on the omap5) for dom0.

Just for clarity this is a bit different because it is not an
MMIO-region yet. It is only a *potential* MMIO region. Basically it is
nothing until the Programmable Logic gets programmed. But the
Programmable Logic only uses addresses within a given range, thankfully,
and we know the range beforehand.

 
> > So, we have a problem :-(
> > 
> > 
> > I can work toward getting the right info on device tree, but in reality
> > that is going to take time and for now the device tree doesn't have the
> > FPGA aperture in it. So if we accept this series as is, it is going to
> > stop features like [1] from working. >
> > If we cannot come up with any better plans, I think it would be better
> > to drop find_memory_holes, only rely on find_unallocated_memory even
> > when the IOMMU is on. One idea is that we could add on top of the
> > regions found by find_unallocated_memory any MMIO regions marked as
> > xen,passthrough: they are safe because they are not going to dom0 anyway.
> 
> (Oleksandr, it looks like some rationale about the different approach is
> missing in the commit message. Can you add it?)
> 
> When the IOMMU is on, Xen will do an extra mapping with GFN == MFN for every
> grant mapping in dom0. This is because Linux will always program the device
> with the MFN as it doesn't know whether the device has been protected by the
> hypervisor.
> 
> Therefore we can't use find_unallocated_memory() with the IOMMU on as it
> stands.
> 
> > The only alternative I can think of is to have a per-board
> > enable/disable toggle for the extend region but it would be very ugly.
> At least, for your board, you seem to know the list of regions that are
> reserved for future use.
>
> So how about adding a per-board list of regions that
> should not be allocated?
> 
> This will also include anything mentioned in 'specific_mappings'.

I am OK with that. I should be able to find the potential address ranges
for Xilinx boards. However, the ranges might be different for different
boards (for different family of boards, not for every little different
revision). Hopefully, Xilinx is the worst case as the hardware is
actually programmable.
--8323329-930583891-1632180475=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 00:01:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 00:01:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191254.341204 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSTED-0001Ni-HK; Tue, 21 Sep 2021 00:01:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191254.341204; Tue, 21 Sep 2021 00:01:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSTED-0001Nb-DM; Tue, 21 Sep 2021 00:01:41 +0000
Received: by outflank-mailman (input) for mailman id 191254;
 Tue, 21 Sep 2021 00:01:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9eYo=OL=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mSTEB-0001NV-SH
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 00:01:40 +0000
Received: from wout4-smtp.messagingengine.com (unknown [64.147.123.20])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 13015cec-5035-4e35-b2d4-5cb794d9c324;
 Tue, 21 Sep 2021 00:01:38 +0000 (UTC)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42])
 by mailout.west.internal (Postfix) with ESMTP id B964E3200B5E;
 Mon, 20 Sep 2021 20:01:35 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Mon, 20 Sep 2021 20:01:36 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 20 Sep 2021 20:01:32 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13015cec-5035-4e35-b2d4-5cb794d9c324
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=elddKd
	D4BPQ0M+cJwTBaZFkbiRuFjkdQnumS8prRBsE=; b=v4Pz4Ctd6lSYg77kUzC3vv
	7tV7NeWPWhYBvJW48B09tcF4tugDz/Q+ZL+eUMY4wUKYpSahhf4SmJiONTv+ChOu
	+QhIvU3VLWYZgEExcs5GM2X1R9AN7gCk7WXert82s1zszvgsRPCutKL9jy1KBQvT
	OmbY1FSiaMiDqetNLHe2igFwMm4RXDkwPy9XaA8CWOHyorBi2bIf/MrmGb0gnkxo
	ZxDJHXCk2CS1JSUpHnTemx4vSwkxvUE+y6lVJMMrNwkhlxGsRfAqcrxDj/LPntJG
	yQEV2CWKxf7aNQIuoGIO05HQmhxlj42o6zLNCHnMDwfWB62nvMmK0E6TIjOzvFkA
	==
X-ME-Sender: <xms:3iBJYRW3dCGreU2LQrLa5VmdUqU7kTqcSYXQYwzLXuXjj9zP9-eI0g>
    <xme:3iBJYRnugGD-Or2uZSV4eEK4BOyjnzLogB9Tw9SyhV9oOCG7BDtgzglvvErALpVdu
    QamPmf-TKgC0g>
X-ME-Received: <xmr:3iBJYdYuPBELwQxKMChTDVktKmW-1_8kx-eY_PtOHoZbw2GleuRGYi50MfNUA3X8KeoAjwKUZL449si9kiLLI12Y9VGgfFQw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeifedgvdekucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeetveff
    iefghfekhffggeeffffhgeevieektedthfehveeiheeiiedtudegfeetffenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:3iBJYUWh324LKSPb-1Ky676Loi0y0Y5CsUnPzrWBIEWCYZxB8-wnoA>
    <xmx:3iBJYbls-90q_PdrkBsnPyZY49B9EBFxGBTjsyGX-0vkTE5V-hyavQ>
    <xmx:3iBJYRcdLpGjcqBgL7SpRsBAH3TyHVsHItquiG-w2GJ74itgw3w0tA>
    <xmx:3yBJYZUwVrj9amEgmY4hmE6piESnkSfYnnLxyRW8Xa0TeLBBXZSCGg>
Date: Tue, 21 Sep 2021 02:01:28 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, efault@gmx.de,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, stable@vger.kernel.org
Subject: Re: [PATCH v2] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUkg2NsVB1xwX3dI@mail-itl>
References: <20210920120421.29276-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="aUPRBE57vyxN5AhB"
Content-Disposition: inline
In-Reply-To: <20210920120421.29276-1-jgross@suse.com>


--aUPRBE57vyxN5AhB
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 21 Sep 2021 02:01:28 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, efault@gmx.de,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, stable@vger.kernel.org
Subject: Re: [PATCH v2] x86/setup: call early_reserve_memory() earlier

On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
>=20
> Move the call of early_reserve_memory() before the call of
> e820__memory_setup() in order to avoid such problems.
>=20
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservatio=
ns")
> Reported-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab=
=2Ecom>
> Signed-off-by: Juergen Gross <jgross@suse.com>

I confirm this fixes my boot issue too.

Tested-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>

> ---
> V2:
> - update comment (Jan Beulich, Boris Petkov)
> - move call down in setup_arch() (Mike Galbraith)
> ---
>  arch/x86/kernel/setup.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
>=20
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..40ed44ead063 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
> =20
>  	x86_init.oem.arch_setup();
> =20
> +	/*
> +	 * Do some memory reservations *before* memory is added to memblock, so
> +	 * memblock allocations won't overwrite it.
> +	 *
> +	 * After this point, everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not RAM in
> +	 * e820. All other memory is free game.
> +	 *
> +	 * This call needs to happen before e820__memory_setup() which calls the
> +	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
> +	 * early reservations have happened already.
> +	 */
> +	early_reserve_memory();
> +
>  	iomem_resource.end =3D (1ULL << boot_cpu_data.x86_phys_bits) - 1;
>  	e820__memory_setup();
>  	parse_setup_data();
> @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
> =20
>  	parse_early_param();
> =20
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> --=20
> 2.26.2
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--aUPRBE57vyxN5AhB
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmFJINkACgkQ24/THMrX
1yxeEgf+OTiRexZli2dhF3IwJBnJzNUlMY1c9IGqgKF1TZ0LZafQLFWuvFG0sN8Q
U+Cla7L/0wYCu+M3TLT502GYcD2B1xbA2IdugAVXCgShZoRqHPE4tiF7+UuUb1jo
DvroNdgMPgPelkQvexh/OG2XMrSwEjalNV+WH9axOAOlUC0p8ji+7DRhZX9au6fV
qQrjeOfxipfe2XL8RC8ED8dXx7c+xYzpTyphfzhdAwGwMHWpAjzmuHqbGIN2V/IP
xLNETBqQfk+M2VjVPihda8RFue3V/nc4fVL+e5X/EkSW24WC0+5h/U2U1cy0HKqI
8nwCQvL/CmtUZPZbZJCrQ6mM+rob7w==
=Cn+N
-----END PGP SIGNATURE-----

--aUPRBE57vyxN5AhB--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 01:10:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 01:10:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191263.341215 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSUIg-00070C-Dw; Tue, 21 Sep 2021 01:10:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191263.341215; Tue, 21 Sep 2021 01:10:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSUIg-000705-AV; Tue, 21 Sep 2021 01:10:22 +0000
Received: by outflank-mailman (input) for mailman id 191263;
 Tue, 21 Sep 2021 01:10:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Kw8l=OL=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mSUIf-0006zz-13
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 01:10:21 +0000
Received: from mail-ed1-x52f.google.com (unknown [2a00:1450:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9c041c80-bd5e-4e97-b10f-b4c2ca141a5b;
 Tue, 21 Sep 2021 01:10:19 +0000 (UTC)
Received: by mail-ed1-x52f.google.com with SMTP id j13so67938927edv.13
 for <xen-devel@lists.xenproject.org>; Mon, 20 Sep 2021 18:10:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c041c80-bd5e-4e97-b10f-b4c2ca141a5b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=6xsV2Snj3hmYGZRLK0SCl9aPfUPXhJvF4zDUBdiC8V8=;
        b=FphWseX+R9mo3NN4TqKJ/CROgzW8DfCc2XzEiCJcg9+diJ1RaxjFS5LUbPxzAhmML+
         so8OvAsQJ+QzW8Xa3+zJj7JXGMK1jVCBY7jJtoGsN4eacRmPvc6n8Ule3PAiXPTGPwSl
         vkIslBz/MLIBDH/n3Koe+A57EACWZ1JIMbfoZtF2xVwZ7CgcA+tKKKLKrg3m6p6j57Go
         C/xbRDm4nuowmmRMEJagW1YdZmMWBDuZBmbYEGhobfCT5JFZ1flA1AIfwkK1MYnDz70q
         7OuacPy32OPc2yYYLoQSBrb5Xlm0x4ia/oyv7DniNfM6B08QEUiWzUC3IwbWso44MQdA
         J+iQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=6xsV2Snj3hmYGZRLK0SCl9aPfUPXhJvF4zDUBdiC8V8=;
        b=WDNSeYwIhMFEZWxxAPA4vN6oT3ukPXUnkcgm39kEIRsmQcMtQ1W8huYvk9ydpRx3gv
         /fXfIgHYtJ5BxLJ9+z91we/rnyoTnix6NTP7yi/UtSvcvVTccG+2OYBaHWQaDP9B+7d5
         knRgrZtD9NFRcmWjoQc7RQVXzpoEK4GTrwCHYFZVbR83vzOuwZp5nfYU4JvLLQwask+j
         AjifW5U/EfpxgxQZ6+mSOETYo/NiLNOOIMdaWq4mGS6k8mhfM37stsZrm7gO1TfGX+BY
         xI8YexOVPfTHFFWMxUCgmuoGKK2nST918Q4eV/InJfgQ+WOc7GnbqfJvVmMvsQ251Lst
         JCgg==
X-Gm-Message-State: AOAM533mLRAWVR57ef8llehMFuKQyKgJRJ4QVO1GNOWM/Q0u7262wxuH
	j67YKkQ4OzDMvs3Pwz7CJ7MF0LchBZdjR2ICsCE=
X-Google-Smtp-Source: ABdhPJzMY2qHNp75jeWCMEO868zFthPsNebKcpJqlnex1ETEZrQWYbkEi2sVuZO9mdTOAH2nQPXQUNapfhGe6eWQVLo=
X-Received: by 2002:aa7:df92:: with SMTP id b18mr31403763edy.47.1632186618539;
 Mon, 20 Sep 2021 18:10:18 -0700 (PDT)
MIME-Version: 1.0
References: <20210920172529.24932-1-andrew.cooper3@citrix.com> <20210920194033.25998-1-andrew.cooper3@citrix.com>
In-Reply-To: <20210920194033.25998-1-andrew.cooper3@citrix.com>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Tue, 21 Sep 2021 06:10:07 +0500
Message-ID: <CAJ=z9a2VcqOow9ELDVjc4BhUQdmqTxwZho9uCowksehMKjd-bQ@mail.gmail.com>
Subject: Re: [PATCH v2.1 16/12] xen/trace: Restrict CONFIG_TRACEBUFFER to x86 PV
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Jan Beulich <JBeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Type: multipart/alternative; boundary="0000000000009af41205cc770fe2"

--0000000000009af41205cc770fe2
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Andrew,

On Tue, 21 Sep 2021, 00:41 Andrew Cooper, <andrew.cooper3@citrix.com> wrote=
:

> The mapping interface can only be used by x86 PV guests.
>

Tracebuffer works on Arm... The support was added a couple of years ago
using the foreign mapping interface.


> This can and should be fixed by changing to an acquire_resource() based
> interface, which is compatbile with x86 PVH and ARM dom0's, but until thi=
s
> happens, don't give the impression of this feature being useable elsewher=
e.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> ---
>  xen/common/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index db687b1785e7..6b6f7139e6f0 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -429,6 +429,7 @@ config DTB_FILE
>
>  config TRACEBUFFER
>         bool "Enable tracing infrastructure" if EXPERT
> +       depends on PV
>         default y
>         ---help---
>           Enable tracing infrastructure and pre-defined tracepoints withi=
n
> Xen.
> --
> 2.11.0
>
>

--0000000000009af41205cc770fe2
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div>Hi Andrew,<br><br><div class=3D"gmail_quote"><div di=
r=3D"ltr" class=3D"gmail_attr">On Tue, 21 Sep 2021, 00:41 Andrew Cooper, &l=
t;<a href=3D"mailto:andrew.cooper3@citrix.com">andrew.cooper3@citrix.com</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The mapping interface =
can only be used by x86 PV guests.<br></blockquote></div></div><div dir=3D"=
auto"><br></div><div dir=3D"auto">Tracebuffer works on Arm... The support w=
as added a couple of years ago using the foreign mapping interface.</div><d=
iv dir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote"><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex">
<br>
This can and should be fixed by changing to an acquire_resource() based<br>
interface, which is compatbile with x86 PVH and ARM dom0&#39;s, but until t=
his<br>
happens, don&#39;t give the impression of this feature being useable elsewh=
ere.<br>
<br>
Signed-off-by: Andrew Cooper &lt;<a href=3D"mailto:andrew.cooper3@citrix.co=
m" target=3D"_blank" rel=3D"noreferrer">andrew.cooper3@citrix.com</a>&gt;<b=
r>
---<br>
CC: Jan Beulich &lt;<a href=3D"mailto:JBeulich@suse.com" target=3D"_blank" =
rel=3D"noreferrer">JBeulich@suse.com</a>&gt;<br>
CC: Roger Pau Monn=C3=A9 &lt;<a href=3D"mailto:roger.pau@citrix.com" target=
=3D"_blank" rel=3D"noreferrer">roger.pau@citrix.com</a>&gt;<br>
CC: Wei Liu &lt;<a href=3D"mailto:wl@xen.org" target=3D"_blank" rel=3D"nore=
ferrer">wl@xen.org</a>&gt;<br>
CC: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.org" target=
=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;<br>
CC: Julien Grall &lt;<a href=3D"mailto:julien@xen.org" target=3D"_blank" re=
l=3D"noreferrer">julien@xen.org</a>&gt;<br>
CC: Volodymyr Babchuk &lt;<a href=3D"mailto:Volodymyr_Babchuk@epam.com" tar=
get=3D"_blank" rel=3D"noreferrer">Volodymyr_Babchuk@epam.com</a>&gt;<br>
---<br>
=C2=A0xen/common/Kconfig | 1 +<br>
=C2=A01 file changed, 1 insertion(+)<br>
<br>
diff --git a/xen/common/Kconfig b/xen/common/Kconfig<br>
index db687b1785e7..6b6f7139e6f0 100644<br>
--- a/xen/common/Kconfig<br>
+++ b/xen/common/Kconfig<br>
@@ -429,6 +429,7 @@ config DTB_FILE<br>
<br>
=C2=A0config TRACEBUFFER<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 bool &quot;Enable tracing infrastructure&quot; =
if EXPERT<br>
+=C2=A0 =C2=A0 =C2=A0 =C2=A0depends on PV<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 default y<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ---help---<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Enable tracing infrastructure and pre-de=
fined tracepoints within Xen.<br>
-- <br>
2.11.0<br>
<br>
</blockquote></div></div></div>

--0000000000009af41205cc770fe2--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 01:47:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 01:47:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191271.341226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSUsV-0002H2-BO; Tue, 21 Sep 2021 01:47:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191271.341226; Tue, 21 Sep 2021 01:47:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSUsV-0002Gv-8F; Tue, 21 Sep 2021 01:47:23 +0000
Received: by outflank-mailman (input) for mailman id 191271;
 Tue, 21 Sep 2021 01:47:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSUsT-0002Gp-WF
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 01:47:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id dbc527f2-1a7d-11ec-b893-12813bfff9fa;
 Tue, 21 Sep 2021 01:47:20 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C4D8F60F6D;
 Tue, 21 Sep 2021 01:47:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbc527f2-1a7d-11ec-b893-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632188839;
	bh=I5VKcWngTdDEZip7JAp1tRcAkJbmXfrVY13T3xHXOaw=;
	h=Date:From:To:cc:Subject:From;
	b=UArsbmKda49UVQo/ke0IZJ4ZLMBO6LDdGfa2hHqPsAg8bzzSv6AvvNvDQ6H1/Rc5a
	 6IaWCq+zlJutCd2Fl6aSGiKMJYgW6lXyFfmUobVI17nUq9u3pD7O+mvtzpgYgcSDti
	 4UpcI2RkXEFHYzpxB97pVp+gR9Xs8tAMikiLrTUD8JLJjgkYROKqKb1ruvFPR63ri+
	 pETojKK/bGQU541yqjQhetswKmPeyaDyWUbPaeiLaWcpWNOrfN2jlSGRZVJr3LP0dZ
	 R89HMt3S/5ubbWZv4YpYrN/ST4soZrY3xhNhXTC/Kpgp3fxKUYjWiUsiDydpGfqr2w
	 zsgEZFoCJRigQ==
Date: Mon, 20 Sep 2021 18:47:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, Bertrand.Marquis@arm.com
Subject: Example patch to send SGIs with a hypercall 
Message-ID: <alpine.DEB.2.21.2109201835220.17979@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-238579627-1632188254=:17979"
Content-ID: <alpine.DEB.2.21.2109201837440.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-238579627-1632188254=:17979
Content-Type: text/plain; CHARSET=US-ASCII
Content-ID: <alpine.DEB.2.21.2109201837441.17979@sstabellini-ThinkPad-T480s>

Hi all,

Next week I'll be giving a talk about VM-to-VM communication mechanisms
at Embedded Linux Conf in Seattle.

One of the techniques I'll discuss, also the simplest, is shared memory
and interrupts. Although there is no binding to share memory between
dom0less VMs yet, it is already possible to do that with a workaround.
But it is not possible to send interrupts between dom0less VMs yet.

The attached patch introduces a trivial hypercall to inject SGI
interrupts into other Dom0less VMs. It is easy to use for baremetal
guests and other kernels without Xen support (no Xen drivers needed).
I'd like to share the patch publicly in advance of the talk, so that I
can reference it during the presentation. The patch is not meant for
upstreaming at this point.

Beware that it comes with no usage restrictions (no rate limiting
and/or no target limits).

Cheers,

Stefano
--8323329-238579627-1632188254=:17979
Content-Type: text/x-diff; name=xen-sgi.patch
Content-Transfer-Encoding: BASE64
Content-ID: <alpine.DEB.2.21.2109201847190.17979@sstabellini-ThinkPad-T480s>
Content-Description: 
Content-Disposition: attachment; filename=xen-sgi.patch

RnJvbSBjMTY5ZTJjMjg3YzE2ZDZjZGVmNmYwNzNkMTJmYTAyY2JkODAwODZj
IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogU3RlZmFubyBTdGFi
ZWxsaW5pIDxzdGVmYW5vLnN0YWJlbGxpbmlAeGlsaW54LmNvbT4NCkRhdGU6
IE1vbiwgMjAgU2VwIDIwMjEgMTg6MzM6NDUgLTA3MDANClN1YmplY3Q6IFtQ
QVRDSF0geGVuL2FybTogaW50cm9kdWNlIFBIWVNERVZfc2dpX3NlbmQNCg0K
SW50cm9kdWNlIGEgc2ltcGxlIGh5cGVyY2FsbCB0byBpbmplY3QgYW4gU0dJ
IGludGVycnVwdCBpbnRvIGEgdGFyZ2V0DQpkb21haW4vdmNwdS4gUGxlYXNl
IG5vdGUgdGhhdCB0aGUgaW1wbGVtZW50YXRpb24gZG9lc24ndCBjb21lIHdp
dGggYW55DQpyYXRlIGxpbWl0aW5nLg0KDQpTaWduZWQtb2ZmLWJ5OiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHN0ZWZhbm8uc3RhYmVsbGluaUB4aWxpbnguY29t
Pg0KLS0tDQogeGVuL2FyY2gvYXJtL3BoeXNkZXYuYyAgICAgICAgICB8IDM0
ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLQ0KIHhlbi9hcmNo
L2FybS90cmFwcy5jICAgICAgICAgICAgfCAgMiArLQ0KIHhlbi9pbmNsdWRl
L2FzbS1hcm0vaHlwZXJjYWxsLmggfCAgMyArKy0NCiB4ZW4vaW5jbHVkZS9w
dWJsaWMvcGh5c2Rldi5oICAgIHwgMTkgKysrKysrKysrKysrKysrKysrDQog
NCBmaWxlcyBjaGFuZ2VkLCA1MyBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9u
cygtKQ0KDQpkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BoeXNkZXYuYyBi
L3hlbi9hcmNoL2FybS9waHlzZGV2LmMNCmluZGV4IGU5MTM1NWZlMjIuLjJl
ZTI5OTM5ZDIgMTAwNjQ0DQotLS0gYS94ZW4vYXJjaC9hcm0vcGh5c2Rldi5j
DQorKysgYi94ZW4vYXJjaC9hcm0vcGh5c2Rldi5jDQpAQCAtMTEsMTAgKzEx
LDM4IEBADQogI2luY2x1ZGUgPGFzbS9oeXBlcmNhbGwuaD4NCiANCiANCi1p
bnQgZG9fcGh5c2Rldl9vcChpbnQgY21kLCBYRU5fR1VFU1RfSEFORExFX1BB
UkFNKHZvaWQpIGFyZykNCitpbnQgZG9fcGh5c2Rldl9vcChpbnQgY21kLCB1
aW50OF90IHNnaSwgdWludDE2X3QgZG9taWQsIHVpbnQzMl90IHZjcHVpZCkN
CiB7DQotICAgIGdkcHJpbnRrKFhFTkxPR19ERUJVRywgIlBIWVNERVZPUCBj
bWQ9JWQ6IG5vdCBpbXBsZW1lbnRlZFxuIiwgY21kKTsNCi0gICAgcmV0dXJu
IC1FTk9TWVM7DQorICAgIGludCByYyA9IDA7DQorICAgIHN0cnVjdCBkb21h
aW4gKmQ7DQorDQorICAgIGlmICggY21kICE9IFBIWVNERVZfc2dpX3NlbmQg
KQ0KKyAgICB7DQorICAgICAgICBnZHByaW50ayhYRU5MT0dfREVCVUcsICJQ
SFlTREVWT1AgY21kPSVkOiBub3QgaW1wbGVtZW50ZWRcbiIsIGNtZCk7DQor
ICAgICAgICByZXR1cm4gLUVOT1NZUzsNCisgICAgfQ0KKw0KKyAgICBpZiAo
IHNnaSA+PSAxNiApDQorICAgIHsNCisgICAgICAgIGdkcHJpbnRrKFhFTkxP
R19ERUJVRywgIlBIWVNERVZPUF9zZ2lfc2VuZCBpbnZhbGlkIHNnaT0ldVxu
Iiwgc2dpKTsNCisgICAgICAgIHJldHVybiAtRUlOVkFMOw0KKyAgICB9DQor
DQorDQorICAgIGQgPSByY3VfbG9ja19kb21haW5fYnlfYW55X2lkKGRvbWlk
KTsNCisgICAgaWYgKCBkID09IE5VTEwgKQ0KKyAgICAgICAgcmV0dXJuIC1F
U1JDSDsNCisgICAgaWYgKCB2Y3B1aWQgPj0gZC0+bWF4X3ZjcHVzICkNCisg
ICAgew0KKyAgICAgICAgcmMgPSAtRUlOVkFMOw0KKyAgICAgICAgZ290byBv
dXQ7DQorICAgIH0NCisNCisgICAgdmdpY19pbmplY3RfaXJxKGQsIGQtPnZj
cHVbdmNwdWlkXSwgc2dpLCB0cnVlKTsNCisNCitvdXQ6DQorICAgIHJjdV91
bmxvY2tfZG9tYWluKGQpOw0KKyAgICByZXR1cm4gcmM7DQogfQ0KIA0KIC8q
DQpkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3RyYXBzLmMgYi94ZW4vYXJj
aC9hcm0vdHJhcHMuYw0KaW5kZXggMjE5YWIzYzNmYi4uMzc0ZWQ0OTBhMiAx
MDA2NDQNCi0tLSBhL3hlbi9hcmNoL2FybS90cmFwcy5jDQorKysgYi94ZW4v
YXJjaC9hcm0vdHJhcHMuYw0KQEAgLTEzNzcsNyArMTM3Nyw3IEBAIHN0YXRp
YyBhcm1faHlwZXJjYWxsX3QgYXJtX2h5cGVyY2FsbF90YWJsZVtdID0gew0K
ICAgICBIWVBFUkNBTEwoeHNtX29wLCAxKSwNCiAgICAgSFlQRVJDQUxMKGV2
ZW50X2NoYW5uZWxfb3AsIDIpLA0KICAgICBIWVBFUkNBTExfREVQUkVDQVRF
RChldmVudF9jaGFubmVsX29wX2NvbXBhdCwgMSksDQotICAgIEhZUEVSQ0FM
TChwaHlzZGV2X29wLCAyKSwNCisgICAgSFlQRVJDQUxMKHBoeXNkZXZfb3As
IDQpLA0KICAgICBIWVBFUkNBTExfREVQUkVDQVRFRChwaHlzZGV2X29wX2Nv
bXBhdCwgMSksDQogICAgIEhZUEVSQ0FMTChzeXNjdGwsIDIpLA0KICAgICBI
WVBFUkNBTEwoaHZtX29wLCAyKSwNCmRpZmYgLS1naXQgYS94ZW4vaW5jbHVk
ZS9hc20tYXJtL2h5cGVyY2FsbC5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9o
eXBlcmNhbGwuaA0KaW5kZXggYTBjNWEzMWEyZi4uNDI5OTU0ODkwNyAxMDA2
NDQNCi0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vaHlwZXJjYWxsLmgNCisr
KyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vaHlwZXJjYWxsLmgNCkBAIC0yLDcg
KzIsOCBAQA0KICNkZWZpbmUgX19BU01fQVJNX0hZUEVSQ0FMTF9IX18NCiAN
CiAjaW5jbHVkZSA8cHVibGljL2RvbWN0bC5oPiAvKiBmb3IgYXJjaF9kb19k
b21jdGwgKi8NCi1pbnQgZG9fcGh5c2Rldl9vcChpbnQgY21kLCBYRU5fR1VF
U1RfSEFORExFX1BBUkFNKHZvaWQpIGFyZyk7DQorDQoraW50IGRvX3BoeXNk
ZXZfb3AoaW50IGNtZCwgdWludDhfdCBzZ2ksIHVpbnQxNl90IGRvbWlkLCB1
aW50MzJfdCB2Y3B1aWQpOw0KIA0KIGxvbmcgZG9fYXJtX3ZjcHVfb3AoaW50
IGNtZCwgdW5zaWduZWQgaW50IHZjcHVpZCwgWEVOX0dVRVNUX0hBTkRMRV9Q
QVJBTSh2b2lkKSBhcmcpOw0KIA0KZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRl
L3B1YmxpYy9waHlzZGV2LmggYi94ZW4vaW5jbHVkZS9wdWJsaWMvcGh5c2Rl
di5oDQppbmRleCBkMjcxNzY2YWQwLi42NDQ1OGZlN2ExIDEwMDY0NA0KLS0t
IGEveGVuL2luY2x1ZGUvcHVibGljL3BoeXNkZXYuaA0KKysrIGIveGVuL2lu
Y2x1ZGUvcHVibGljL3BoeXNkZXYuaA0KQEAgLTM0MCw2ICszNDAsMjUgQEAg
c3RydWN0IHBoeXNkZXZfZGJncF9vcCB7DQogdHlwZWRlZiBzdHJ1Y3QgcGh5
c2Rldl9kYmdwX29wIHBoeXNkZXZfZGJncF9vcF90Ow0KIERFRklORV9YRU5f
R1VFU1RfSEFORExFKHBoeXNkZXZfZGJncF9vcF90KTsNCiANCisvKg0KKyAq
IEFSTSBvbmx5IGh5cGVyY2FsbCB0byBzZW5kIGFuIFNHSSB0byBhbm90aGVy
IFZNDQorICoNCisgKiBpbnQgcGh5c2Rldl9vcChpbnQgY21kLCB1aW50OF90
IHNnaSwgdWludDE2X3QgZG9taWQsIHVpbnQzMl90IHZjcHVpZCkNCisgKg0K
KyAqIHNnaTogU0dJIG51bWJlciBvZiB0aGUgaW50ZXJydXB0IHRvIGluamVj
dA0KKyAqIGRvbWlkOiB0YXJnZXQgZG9taWQNCisgKiB2Y3B1aWQ6IHRhcmdl
dCB2Y3B1c2lkDQorICoNCisgKiBleGFtcGxlOg0KKyAqICAgICBtb3YgeDAs
ICMzMCAgLy8gUEhZU0RFVl9zZ2lfc2VuZCBjb21tYW5kDQorICogICAgIG1v
diB4MSwgIzUgICAvLyBTR0kgbnVtYmVyIDUsIGNhbiBjaG9zZSBhbnkgaW4g
dGhlIHJhbmdlIDAtMTUNCisgKiAgICAgbW92IHgyLCAjMSAgIC8vIGRlc3Rp
bmF0aW9uIGRvbWFpbiBpZCAoZS5nLiAwIGZvciBkb20wLCAxIGZvciBmaXJz
dCBkb21VKQ0KKyAqICAgICBtb3YgeDMsICMwICAgLy8gZGVzdGluYXRpb24g
dmNwdSBpZCAobm90IHBoeXNpY2FsIGNwdSksIHR5cGljYWxseSAwDQorICog
ICAgIG1vdiB4MTYsICMzMyAvLyBoeXBlcmNhbGwgcGh5c2Rldl9vcCBudW1i
ZXIgMzMNCisgKiAgICAgaHZjIDB4RUExICAgIC8vIGh5cGVyY2FsbCBjb21t
YW5kDQorICovDQorI2RlZmluZSBQSFlTREVWX3NnaV9zZW5kIDMwDQorDQog
LyoNCiAgKiBOb3RpZnkgdGhhdCBzb21lIFBJUlEtYm91bmQgZXZlbnQgY2hh
bm5lbHMgaGF2ZSBiZWVuIHVubWFza2VkLg0KICAqICoqIFRoaXMgY29tbWFu
ZCBpcyBvYnNvbGV0ZSBzaW5jZSBpbnRlcmZhY2UgdmVyc2lvbiAweDAwMDMw
MjAyIGFuZCBpcyAqKg0KLS0gDQoyLjE3LjENCg0K

--8323329-238579627-1632188254=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 03:59:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 03:59:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191283.341236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSWwY-0007Re-RI; Tue, 21 Sep 2021 03:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191283.341236; Tue, 21 Sep 2021 03:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSWwY-0007RX-OC; Tue, 21 Sep 2021 03:59:42 +0000
Received: by outflank-mailman (input) for mailman id 191283;
 Tue, 21 Sep 2021 03:59:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Cbwx=OL=kernel.org=nathan@srs-us1.protection.inumbo.net>)
 id 1mSWwX-0007RR-5C
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 03:59:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 57eb5902-1a90-11ec-b895-12813bfff9fa;
 Tue, 21 Sep 2021 03:59:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1516B611C5;
 Tue, 21 Sep 2021 03:59:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57eb5902-1a90-11ec-b895-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632196779;
	bh=7OFVjzBNfyXXNiMg++zl6W4YbQ+GkChXNDPM6C4fm5Q=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=n3XB9+ugrNqVsTPmuetwR6tFfbxIb+a2Z7toWMqN0Dy+j2Cbd7wAA1OEPwy8tAEbZ
	 ZjOua8+cY72aS72AP8eojwewsPQkKUXi2TtXcnk2kUzalmAKD01uRtWYX+3OA8Gepk
	 B87YzxW/IBbfeVyitYV3IaTEAdLqdcLDfGb17WGpYDIka9EhrHUme/0D12blfxBtCu
	 eJPWLehJsL7qkqgkblKCHlkbCjcuZ/ek106F5zZEGKWvfpWeQoNGGFGljGLDpsIpv1
	 OlBIdUGF7H3tJPgr997lQ9+w/Fo16rZSiWOXwAFb2KxZM19MDil9rGz75CswVKOYZB
	 7O9rHggfmZiQA==
Date: Mon, 20 Sep 2021 20:59:33 -0700
From: Nathan Chancellor <nathan@kernel.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, efault@gmx.de,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, stable@vger.kernel.org,
	Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>
Subject: Re: [PATCH v2] x86/setup: call early_reserve_memory() earlier
Message-ID: <YUlYpWhGCxpJ9diw@archlinux-ax161>
References: <20210920120421.29276-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210920120421.29276-1-jgross@suse.com>

On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
> Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory
> reservations") introduced early_reserve_memory() to do all needed
> initial memblock_reserve() calls in one function. Unfortunately the
> call of early_reserve_memory() is done too late for Xen dom0, as in
> some cases a Xen hook called by e820__memory_setup() will need those
> memory reservations to have happened already.
> 
> Move the call of early_reserve_memory() before the call of
> e820__memory_setup() in order to avoid such problems.
> 
> Cc: stable@vger.kernel.org
> Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
> Reported-by: Marek Marczykowski-Grecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

I had issues on an AMD Ryzen 3 4300G based system with v1. v2 does not
trigger any boot issues on that same machine or an Intel i5-4210U based
system that I also test with.

Tested-by: Nathan Chancellor <nathan@kernel.org>

> ---
> V2:
> - update comment (Jan Beulich, Boris Petkov)
> - move call down in setup_arch() (Mike Galbraith)
> ---
>  arch/x86/kernel/setup.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 79f164141116..40ed44ead063 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
>  
>  	x86_init.oem.arch_setup();
>  
> +	/*
> +	 * Do some memory reservations *before* memory is added to memblock, so
> +	 * memblock allocations won't overwrite it.
> +	 *
> +	 * After this point, everything still needed from the boot loader or
> +	 * firmware or kernel text should be early reserved or marked not RAM in
> +	 * e820. All other memory is free game.
> +	 *
> +	 * This call needs to happen before e820__memory_setup() which calls the
> +	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
> +	 * early reservations have happened already.
> +	 */
> +	early_reserve_memory();
> +
>  	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
>  	e820__memory_setup();
>  	parse_setup_data();
> @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
>  
>  	parse_early_param();
>  
> -	/*
> -	 * Do some memory reservations *before* memory is added to
> -	 * memblock, so memblock allocations won't overwrite it.
> -	 * Do it after early param, so we could get (unlikely) panic from
> -	 * serial.
> -	 *
> -	 * After this point everything still needed from the boot loader or
> -	 * firmware or kernel text should be early reserved or marked not
> -	 * RAM in e820. All other memory is free game.
> -	 */
> -	early_reserve_memory();
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  	/*
>  	 * Memory used by the kernel cannot be hot-removed because Linux
> -- 
> 2.26.2
> 
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 04:15:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 04:15:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191291.341248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSXBo-0001hH-BQ; Tue, 21 Sep 2021 04:15:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191291.341248; Tue, 21 Sep 2021 04:15:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSXBo-0001hA-7z; Tue, 21 Sep 2021 04:15:28 +0000
Received: by outflank-mailman (input) for mailman id 191291;
 Tue, 21 Sep 2021 04:15:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSXBm-0001h0-DG; Tue, 21 Sep 2021 04:15:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSXBm-0005hM-3v; Tue, 21 Sep 2021 04:15:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSXBl-0007us-RP; Tue, 21 Sep 2021 04:15:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSXBl-0006nL-Ja; Tue, 21 Sep 2021 04:15:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dUKO4ERg44RLnr2ak3qaYrazYfh93OXfMRjIyo643CQ=; b=d+wguMC6ufYpB1hArzEsqdwTUz
	UIvaytLeW0uAkZ46ELJxSf8SiTFMC2ku3UG07AGgms1C7ytB6zGg8o9LPvA3r7Xk9h+DcLtSQfitW
	7cIYzLTJ9WEUPbtyw7Tqjx1rGW1jdggWewvs7L8GbK6A/p413gYnsr0QOORZA5cBK3v8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165131-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165131: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 04:15:25 +0000

flight 165131 xen-unstable real [real]
flight 165135 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165131/
http://logs.test-lab.xenproject.org/osstest/logs/165135/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   10 days
Failing since        164951  2021-09-12 00:14:36 Z    9 days   18 attempts
Testing same since   165131  2021-09-20 17:39:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 683 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 04:51:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 04:51:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191298.341262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSXkw-00065i-8f; Tue, 21 Sep 2021 04:51:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191298.341262; Tue, 21 Sep 2021 04:51:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSXkw-00065b-43; Tue, 21 Sep 2021 04:51:46 +0000
Received: by outflank-mailman (input) for mailman id 191298;
 Tue, 21 Sep 2021 04:51:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSXku-00065V-V1
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 04:51:45 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9db99460-1a97-11ec-b89a-12813bfff9fa;
 Tue, 21 Sep 2021 04:51:43 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L3x58l003773; 
 Tue, 21 Sep 2021 04:51:41 GMT
Received: from eur02-he1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b77eyr4ky-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 04:51:40 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3283.eurprd03.prod.outlook.com (2603:10a6:208:12::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 21 Sep
 2021 04:51:37 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 04:51:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9db99460-1a97-11ec-b89a-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CltyVyTAc/rm9nlTXsGK5wsHqX2+D7a7Ft8QwiRefObKjXtt3nmm1rAELIK+2vyuhy55HIMDmmkjNZOhXXykpcSuRNCuzHKE4CbnwPibnZe6F4ShTBrihrlEfSUxiMAQp/u5+Dy/FTzsQii6SbHUvAGqEuQrmOjSTq7ykuZDErmKTCmIcCJa8pOtGz2wdVmHeCdnXJK3YSmBhYjYJZAD+t3yX0hFZ4SVIDJmZqRYYfHJvAuzDpOavh8m+iAbMQHSTAGpLy/fSo50U0oyVG/BrssG10uh1qj2ge3oYWBvC5hAJkbSyAbpNlmu0lwTBbdNsXkfPqxYgEYkV+SG9n17tg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=e2i7coYjAxsTKODWtb7nzDIwyq0Lp0Youc4nyHsfs3U=;
 b=GsYpb3iboNW0XdnTNxT7U9PMPcHeQEHGNmPyCV+yLAWZnO+u3TZOlCEk8XDKIzm3hUk3FL056Z8bCUw0RhUmLsrXVVyEHsjoofSn7S9xuYn52PZuJgKtgLMryq88HTU1YPzQc4mIbt8jIsEcgaD8Y8xs0wo21g2DB0bTIhjKPtp02/xZQ1khdeW8acMyR+WbtHwMtRysT3cRouIgq8PEKozl0X8DBQ/Eb+l9OIvBBO+x5dI0TmiJ2X1Jh8hFlUJwyAIHZagv2jP235rIf37JdcG4fl1mziGn6guqZ+dovZAyLCnVkRNCW07HdOyfdsyEM/T3qlJH2m42eYI//BvOeg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e2i7coYjAxsTKODWtb7nzDIwyq0Lp0Youc4nyHsfs3U=;
 b=IjvMHIzSHmO2TimlFn4b15RFETQnDwExBa0rJjpaFU2SNbwIr8aFZdwE5GpSpepArnrtujPkPH6/JVuB1dIRLDbGrL00eBHRsJcbNgxMdos6bF8MGY/G7Se0QwGQbhfzrgY6fSazz+pE6Kmsi31Kbp0kClWvfbNfuJ5qQneSnMxRGfaqOyCIrFNdsPFkZ5Ya1ZCvhpTL++Y/Hsljlw9BIQ7e5MJiB4u6WG199TIbzypQA0rpXQFJNZrZnLrIQ/eyC9mP4600HAcIvUFIFGPDBlKt+NM744Mgg3+ttLfSt5GlFUViMdGj8LYNwtZez/XBY0HvhYW1yxWMNJeRG5YGtw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Juergen Gross <jgross@suse.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com"
	<boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia Lukianenko
	<Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAF25gA==
Date: Tue, 21 Sep 2021 04:51:36 +0000
Message-ID: <ddb3a17a-1fd2-7108-20ad-73487a308954@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e0149047-d499-4902-a4bc-08d97cbb7e65
x-ms-traffictypediagnostic: AM0PR0302MB3283:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR0302MB328374DE12AD380A6041593BE7A19@AM0PR0302MB3283.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 2Mfjo5m4dmQL9mrGMODX1qxa4TN74MliVUQoDKZ1gMe2PSWAJ4RGV1h4BgL0Zv899VbiLdAzxSSa1uoHKPmCB9uLmY4U8BS3lCXfVKtRswr5L4YJMtObglHXq39qpOZ0bdBOZzjkgwvzT3UlBJKSi9XtNzq7wZjEqs6EITynToQEuNE6NbNpP5IT3w72Jz+w7zgjDcxsPYSCbwJLSuFsBt8hnqv701i6t5Vif8NJd1Y7i4woD0b8H+4SbVj1OnjbNJyIh0VYp1vDscbaxmEkzj5K/7veV8oH17K0fOm2AGx/072ckFLdCe/pu6J/+zDAC868uLmTECY+p2QcT3Ic+miTyXoO3EXMmWX3qbeQnZRwiqw2moA+9+FGQFZImExgES8zb/qTdQjKHTh6yAgn2+xrRpQtA9F0Qv7K5VNaNB7Oqeou5qCTphCz7MPEIolkds8THKONH6hk3WpW0/rCjNn8PUFaw6DR2e+UwUT8oKialJSycgVKdsRmtGZ3+0HS9SLsr5JTH/B2I2G+Biz7UfIYNKGDCv7HOpE5A8/QakthxUxdKvBAkkr8nMo+OiMaGRW82ttsPuwh1RSKkDxmtRvJpxumtxal0UbRvehsN+HO1Hn7D280S3XBMXk817HNC6m7o2i/LfjeqKiaPjPSNib1QMVtXYtSyGQjqASl3xl8Hcy1w0StMxENH0q7QAeNZGzNajPDztQrch1DoVzApiIGcV/UHIVUZ1PKteEmkid3Xf8Z4A75o+Noi+z7GSX3
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(53546011)(6506007)(36756003)(2616005)(38100700002)(71200400001)(86362001)(38070700005)(31696002)(66946007)(8676002)(83380400001)(55236004)(122000001)(508600001)(6486002)(6512007)(66476007)(4326008)(31686004)(5660300002)(64756008)(186003)(8936002)(66446008)(66556008)(2906002)(54906003)(26005)(316002)(6916009)(91956017)(76116006)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eDFzbzMrWlFraENOTmV1YnZIT2tlMEJ2L21xdWFwUGl2UG1ZM2pHb2JFbCti?=
 =?utf-8?B?cVRwQVN5T1YyQU1Jb1g5aGpWVjFsbzFDVFYzTjR3UkhqZVpmMytPeUVEdi82?=
 =?utf-8?B?VUplb1ZaUlFjenVBdTJ1Q0g1bW8yZngyOEUrQ0F1ZXdlbTRpdGQ1ZFZnZ0w5?=
 =?utf-8?B?eHQ4bUtZNU1Cemh5bzlwZndLS2JueXNaWkhzaFkvU0Rvc1QzMG9NNDVJM2xi?=
 =?utf-8?B?WTZlQ3JTV25kYisrNUE0djM5SUxuMk5kY2dGdlZvWVMvNDA0RThrOWs2TFRD?=
 =?utf-8?B?WkJlVVU2aE14UXNGSG1LQng1NjdRUWZqWlNHY0FnempJUno1MFEvNEhiblNx?=
 =?utf-8?B?QlVabG9yMVF6bWI0UjZRcVhOK0EzejhnS0RKb2xUYmc3V1Y2NVhzM3E5cndE?=
 =?utf-8?B?WVNOMG41V1pTL0cvWHZXRFdNaG5oRW5LRStleHE2S3lGTU9wRnMwTDZrdlpR?=
 =?utf-8?B?R3ZmQlc2MVZTZG93Z1Q1QXBBcFhtaDliVElNUjNQYVpldWw5NUpuQ1RwVDFp?=
 =?utf-8?B?MlRZaVk4Ty9aZ3F5a3FoR1R4NGY0UE9idXQ4cUlzeFZ2R3FEMEw3aGt1cFJD?=
 =?utf-8?B?THQyRGN4ZkNzbnhMWTRZRlNjS3VDS2N3c0J4dzRLNE5SakVNTWo4NW1EZWo3?=
 =?utf-8?B?N3FkM3JnSVU2L1FVZ20ybVhKMlFRMkIyU1dNRmtBdWVMN2ZKUHEwV285R2U5?=
 =?utf-8?B?SUFRdFhyZ0tPL2t6VlZNVWY2bVRqNmxNQU5Ia3dyRGlDdER2dlNJTGVFUXVt?=
 =?utf-8?B?cS82d2g5a1hmQmV3U0I0VEltNHE4TXpsMTEwOUE2RE1JL2tnYUdzcHRNSmw5?=
 =?utf-8?B?YXNSdHJjUHdLVDBEam96d1ZqVkRyYnJtOFZVbHJrMW1iRHJjQnU4QmUzajlP?=
 =?utf-8?B?YUwrd0NmT1puUEtTRGhCR0NTM2IxVG45YWJnM3FVbDFyODg2LzhiS0RIcDFk?=
 =?utf-8?B?Mnp6bzJmclFxdWlkNWtuMGNwQ3V0Q2J5R0lIRmR4Z1IxRHEwU1FCTitLRFhl?=
 =?utf-8?B?NDRFMlplL1ppMURoamZaNkh4WURJRGxlMk4wdlh1K1hIRkdzQkJvQVRja0tx?=
 =?utf-8?B?SGpjZURscGRmUUtjZ1pTdDhKVnRGUytRTmtiWjU3eUpaalFrRjZKWUlxZ0V6?=
 =?utf-8?B?UGNBMWdaZjJPZ0RtcVhiZHNKYU55a2tzb2dNK3l0ZitwWUlnSEd4WGhzNmJk?=
 =?utf-8?B?T3ozWi9zUjFPM3hva3MwUVN4YUc4RG9yYmZzU0wydE1kUFZ5elZlZEtIN1I1?=
 =?utf-8?B?YzREZXRGdk84MVlkRW5pa0pHNG5CRDBTSE0vWTVNaWJMbEx3NGcwZTRsS3cz?=
 =?utf-8?B?bzVYUlJiQk1xblVoUm9VSnZ3V29TTWxKckl0ZllhT2p0NUJ2K3E4MEloOTdF?=
 =?utf-8?B?M2xBaTV5WUZBNkNGcHMzN3JxY0VkMEl4VXQ2TnhudnZTWjlCYTh3RHJ2dkZ4?=
 =?utf-8?B?aWpNTkU5SjdyaDRkWTlsbHdRUFJ5aCtuZHJxajFYOGtiMXFweU4wMjhmeXZZ?=
 =?utf-8?B?VHhPY24vUWFXNHE3OEYyUEUzYmhnbzhkTStrSllYSVV4eU9Dam1kcjFHc3d6?=
 =?utf-8?B?SWpkZ3pheW5BUExBdFRReWJHZ0xrdlk4RHlJZGNsdXFGaFB4U2owSkQrL2h4?=
 =?utf-8?B?V2VacVBReTh6N2Rid3o0aU00UDlRalROckRHODE5clJFdU9YZklETGFETktT?=
 =?utf-8?B?MThSRWd2QTFFRjJxbDFFdTB2QlFNV29kdC8vOUVjY2dWTU9pb0VUU285Z3o4?=
 =?utf-8?B?RzhmMzB4ZG8wQlEzU1F0dWhJTkMxYVJPN0lRclRadERibG5MMkt0OStjQ2F2?=
 =?utf-8?B?c2Jtcmd5N0piS2p2TGtGUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F7099075499B774FBB326C4296F2DDFB@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e0149047-d499-4902-a4bc-08d97cbb7e65
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 04:51:36.9989
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: wko66i4g+e5VRgHkowaoZTKrjvBMg8Z4AcY95SS0X4I+nlsMM1gcct6XuqQLTJU0gt0+lcVAh4ySOkXEtgDbLJRjU6o3QjDKN+xWodb/WOze16uKGW7u7DXWY/lguSyp
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3283
X-Proofpoint-GUID: QqEUktIZuMdPN-HuPWwSX3ViTU6BYD7D
X-Proofpoint-ORIG-GUID: QqEUktIZuMdPN-HuPWwSX3ViTU6BYD7D
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0
 clxscore=1015 suspectscore=0 lowpriorityscore=0 mlxscore=0
 priorityscore=1501 impostorscore=0 mlxlogscore=999 adultscore=0
 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210028

SGksIFN0ZWZhbm8hDQoNCk9uIDIxLjA5LjIxIDAyOjE2LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3Jv
dGU6DQo+IE9uIE1vbiwgMjAgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4gT24gMjAuMDkuMjEgMTQ6MzAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4gT24gMjAu
MDkuMjEgMDc6MjMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBIZWxsbywg
U3RlZmFubyENCj4+Pj4NCj4+Pj4gT24gMTguMDkuMjEgMDA6NDUsIFN0ZWZhbm8gU3RhYmVsbGlu
aSB3cm90ZToNCj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+DQo+Pj4+PiBXaHkgZG8geW91IHdh
bnQgdG8gZW5hYmxlIHBjaWJhY2sgb24gQVJNPyBJcyBpdCBvbmx5IHRvICJkaXNhYmxlIiBhIFBD
SQ0KPj4+Pj4gZGV2aWNlIGluIERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVk
IHRvIGEgRG9tVT8NCj4+Pj4gTm90IG9ubHkgdGhhdA0KPj4+Pj4gSSBhbSBhc2tpbmcgYmVjYXVz
ZSBhY3R1YWxseSBJIGRvbid0IHRoaW5rIHdlIHdhbnQgdG8gZW5hYmxlIHRoZSBQViBQQ0kNCj4+
Pj4+IGJhY2tlbmQgZmVhdHVyZSBvZiBwY2liYWNrIG9uIEFSTSwgcmlnaHQ/IFRoYXQgd291bGQg
Y2xhc2ggd2l0aCB0aGUgUENJDQo+Pj4+PiBhc3NpZ25tZW50IHdvcmsgeW91IGhhdmUgYmVlbiBk
b2luZyBpbiBYZW4uIFRoZXkgY291bGRuJ3QgYm90aCB3b3JrIGF0DQo+Pj4+PiB0aGUgc2FtZSB0
aW1lLg0KPj4+PiBDb3JyZWN0LCBpdCBpcyBub3QgdXNlZA0KPj4+Pj4gSWYgd2Ugb25seSBuZWVk
IHBjaWJhY2sgdG8gInBhcmsiIGEgZGV2aWNlIGluIERvbTAsIHdvdWxkbid0IGl0IGJlDQo+Pj4+
PiBwb3NzaWJsZSBhbmQgYmV0dGVyIHRvIHVzZSBwY2ktc3R1YiBpbnN0ZWFkPw0KPj4+PiBOb3Qg
b25seSB0aGF0LCBzbyBwY2ktc3R1YiBpcyBub3QgZW5vdWdoDQo+Pj4+DQo+Pj4+IFRoZSBmdW5j
dGlvbmFsaXR5IHdoaWNoIGlzIGltcGxlbWVudGVkIGJ5IHRoZSBwY2liYWNrIGFuZCB0aGUgdG9v
bHN0YWNrDQo+Pj4+IGFuZCB3aGljaCBpcyByZWxldmFudC9taXNzaW5nL25lZWRlZCBmb3IgQVJN
Og0KPj4+Pg0KPj4+PiAxLiBwY2liYWNrIGlzIHVzZWQgYXMgYSBkYXRhYmFzZSBmb3IgYXNzaWdu
YWJsZSBQQ0kgZGV2aWNlcywgZS5nLiB4bA0KPj4+PiAgwqAgwqDCoCBwY2ktYXNzaWduYWJsZS17
YWRkfHJlbW92ZXxsaXN0fSBtYW5pcHVsYXRlcyB0aGF0IGxpc3QuIFNvLCB3aGVuZXZlciB0aGUN
Cj4+Pj4gIMKgIMKgwqAgdG9vbHN0YWNrIG5lZWRzIHRvIGtub3cgd2hpY2ggUENJIGRldmljZXMg
Y2FuIGJlIHBhc3NlZCB0aHJvdWdoIGl0IHJlYWRzDQo+Pj4+ICDCoCDCoMKgIHRoYXQgZnJvbSB0
aGUgcmVsZXZhbnQgc3lzZnMgZW50cmllcyBvZiB0aGUgcGNpYmFjay4NCj4+Pj4NCj4+Pj4gMi4g
cGNpYmFjayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVuYm91bmQgUENJIGRldmljZXMsIGUuZy4gd2hl
biBwYXNzaW5nIHRocm91Z2gNCj4+Pj4gIMKgIMKgwqAgYSBQQ0kgZGV2aWNlIGl0IG5lZWRzIHRv
IGJlIHVuYm91bmQgZnJvbSB0aGUgcmVsZXZhbnQgZGV2aWNlIGRyaXZlciBhbmQgYm91bmQNCj4+
Pj4gIMKgIMKgwqAgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3BlYWtpbmcgaXQgaXMgbm90IHJlcXVp
cmVkIHRoYXQgdGhlIGRldmljZSBpcyBib3VuZCB0bw0KPj4+PiAgwqAgwqDCoCBwY2liYWNrLCBi
dXQgcGNpYmFjayBpcyBhZ2FpbiB1c2VkIGFzIGEgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJv
dWdoIFBDSQ0KPj4+PiAgwqAgwqDCoCBkZXZpY2VzLCBzbyB3ZSBjYW4gcmUtYmluZCB0aGUgZGV2
aWNlcyBiYWNrIHRvIHRoZWlyIG9yaWdpbmFsIGRyaXZlcnMgd2hlbg0KPj4+PiAgwqAgwqDCoCBn
dWVzdCBkb21haW4gc2h1dHMgZG93bikNCj4+Pj4NCj4+Pj4gMy4gRGV2aWNlIHJlc2V0DQo+Pj4+
DQo+Pj4+IFdlIGhhdmUgcHJldmlvdXNseSBkaXNjdXNzZWQgb24geGVuLWRldmVsIE1MIHBvc3Np
YmxlIHNvbHV0aW9ucyB0byB0aGF0IGFzIGZyb20gdGhlDQo+Pj4+IGFib3ZlIHdlIHNlZSB0aGF0
IHBjaWJhY2sgZnVuY3Rpb25hbGl0eSBpcyBnb2luZyB0byBiZSBvbmx5IHBhcnRpYWxseSB1c2Vk
IG9uIEFybS4NCj4+Pj4NCj4+Pj4gUGxlYXNlIHNlZSBbMV0gYW5kIFsyXToNCj4+Pj4NCj4+Pj4g
MS4gSXQgaXMgbm90IGFjY2VwdGFibGUgdG8gbWFuYWdlIHRoZSBhc3NpZ25hYmxlIGxpc3QgaW4g
WGVuIGl0c2VsZg0KPj4+Pg0KPj4+PiAyLiBwY2liYWNrIGNhbiBiZSBzcGxpdCBpbnRvIHR3byBw
YXJ0czogUENJIGFzc2lnbmFibGUvYmluZC9yZXNldCBoYW5kbGluZyBhbmQNCj4+Pj4gdGhlIHJl
c3QgbGlrZSB2UENJIGV0Yy4NCj4+Pj4NCj4+Pj4gMy4gcGNpZnJvbnQgaXMgbm90IHVzZWQgb24g
QXJtDQo+Pj4gSXQgaXMgbmVpdGhlciBpbiB4ODYgUFZIL0hWTSBndWVzdHMuDQo+PiBEaWRuJ3Qg
a25vdyB0aGF0LCB0aGFuayB5b3UgZm9yIHBvaW50aW5nDQo+Pj4+IFNvLCBsaW1pdGVkIHVzZSBv
ZiB0aGUgcGNpYmFjayBpcyBvbmUgb2YgdGhlIGJyaWNrcyB1c2VkIHRvIGVuYWJsZSBQQ0kgcGFz
c3Rocm91Z2gNCj4+Pj4gb24gQXJtLiBJdCB3YXMgZW5vdWdoIHRvIGp1c3QgcmUtc3RydWN0dXJl
IHRoZSBkcml2ZXIgYW5kIGhhdmUgaXQgcnVuIG9uIEFybSB0byBhY2hpZXZlDQo+Pj4+IGFsbCB0
aGUgZ29hbHMgYWJvdmUuDQo+Pj4+DQo+Pj4+IElmIHdlIHN0aWxsIHRoaW5rIGl0IGlzIGRlc2ly
YWJsZSB0byBicmVhayB0aGUgcGNpYmFjayBkcml2ZXIgaW50byAiY29tbW9uIiBhbmQgInBjaWZy
b250IHNwZWNpZmljIg0KPj4+PiBwYXJ0cyB0aGVuIGl0IGNhbiBiZSBkb25lLCB5ZXQgdGhlIHBh
dGNoIGlzIGdvaW5nIHRvIGJlIHRoZSB2ZXJ5IGZpcnN0IGJyaWNrIGluIHRoYXQgYnVpbGRpbmcu
DQo+Pj4gRG9pbmcgdGhpcyBzcGxpdCBzaG91bGQgYmUgZG9uZSwgYXMgdGhlIHBjaWZyb250IHNw
ZWNpZmljIHBhcnQgY291bGQgYmUNCj4+PiBvbWl0dGVkIG9uIHg4NiwgdG9vLCBpbiBjYXNlIG5v
IFBWIGd1ZXN0cyB1c2luZyBQQ0kgcGFzc3Rocm91Z2ggaGF2ZSB0bw0KPj4+IGJlIHN1cHBvcnRl
ZC4NCj4+IEFncmVlLCB0aGF0IHRoZSBmaW5hbCBzb2x1dGlvbiBzaG91bGQgaGF2ZSB0aGUgZHJp
dmVyIHNwbGl0DQo+Pj4+IFNvLCBJIHRoaW5rIHRoaXMgcGF0Y2ggaXMgc3RpbGwgZ29pbmcgdG8g
YmUgbmVlZGVkIGJlc2lkZXMgd2hpY2ggZGlyZWN0aW9uIHdlIHRha2UuDQo+Pj4gU29tZSBraW5k
IG9mIHRoaXMgcGF0Y2gsIHllcy4gSXQgbWlnaHQgbG9vayBkaWZmZXJlbnQgaW4gY2FzZSB0aGUg
c3BsaXQNCj4+PiBpcyBkb25lIGZpcnN0Lg0KPj4+DQo+Pj4gSSBkb24ndCBtaW5kIGRvaW5nIGl0
IGluIGVpdGhlciBzZXF1ZW5jZS4NCj4+Pg0KPj4gV2l0aCB0aGlzIHBhdGNoIHdlIGhhdmUgQXJt
IG9uIHRoZSBzYW1lIHBhZ2UgYXMgdGhlIGFib3ZlIG1lbnRpb25lZCB4ODYgZ3Vlc3RzLA0KPj4N
Cj4+IGUuZy4gdGhlIGRyaXZlciBoYXMgdW51c2VkIGNvZGUsIGJ1dCB5ZXQgYWxsb3dzIEFybSB0
byBmdW5jdGlvbiBub3cuDQo+Pg0KPj4gQXQgdGhpcyBzdGFnZSBvZiBQQ0kgcGFzc3Rocm91Z2gg
b24gQXJtIGl0IGlzIHlldCBlbm91Z2guIExvbmcgdGVybSwgd2hlbg0KPj4NCj4+IHRoZSBkcml2
ZXIgZ2V0cyBzcGxpdCwgQXJtIHdpbGwgYmVuZWZpdCBmcm9tIHRoYXQgc3BsaXQgdG9vLCBidXQg
dW5mb3J0dW5hdGVseSBJIGRvIG5vdA0KPj4NCj4+IGhhdmUgZW5vdWdoIGJhbmR3aWR0aCBmb3Ig
dGhhdCBwaWVjZSBvZiB3b3JrIGF0IHRoZSBtb21lbnQuDQo+IFRoYXQncyBmYWlyIGFuZCBJIGRv
bid0IHdhbnQgdG8gc2NvcGUtY3JlZXAgdGhpcyBzaW1wbGUgcGF0Y2ggYXNraW5nIGZvcg0KPiBh
biBlbm9ybW91cyByZXdvcmsuIEF0IHRoZSBzYW1lIHRpbWUgSSBkb24ndCB0aGluayB3ZSBzaG91
bGQgZW5hYmxlIHRoZQ0KPiB3aG9sZSBvZiBwY2liYWNrIG9uIEFSTSBiZWNhdXNlIGl0IHdvdWxk
IGJlIGVycm9uZW91cyBhbmQgY29uZnVzaW5nLg0KPg0KPiBJIGFtIHdvbmRlciBpZiB0aGVyZSBp
cyBhIHNpbXBsZToNCj4NCj4gaWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+ICAgICAgcmV0dXJuOw0K
Pg0KPiBUaGF0IHdlIGNvdWxkIGFkZCBpbiBhIGNvdXBsZSBvZiBwbGFjZXMgaW4gcGNpYmFjayB0
byBzdG9wIGl0IGZyb20NCj4gaW5pdGlhbGl6aW5nIHRoZSBwYXJ0cyB3ZSBkb24ndCBjYXJlIGFi
b3V0LiBTb21ldGhpbmcgYWxvbmcgdGhlc2UgbGluZXMNCj4gKHVudGVzdGVkIGFuZCBwcm9iYWJs
eSBpbmNvbXBsZXRlKS4NCj4NCj4gV2hhdCBkbyB5b3UgZ3V5cyB0aGluaz8NCg0KSSB0aGluayB0
aGF0IGl0IG5lZWRzIHRvIGJlIGFuIGFkZGl0aW9uYWwgcGF0Y2ggYW5kIHRoZSBQViBjaGVjayBz
ZWVtcw0KDQpyZWFzb25hYmxlIHRvIG1lLiBXZSBuZWVkIHRvIGNoZWNrIGlmIGdhdGluZyBvbmx5
IHBhcnQgb2YgdGhlIGluaXRpYWxpemF0aW9uDQoNCndpdGggeGVuX3B2X2RvbWFpbiBpcyBqdXN0
IGVub3VnaCwgZS5nLiBpZiB0aGUgcmVzdCBvZiB0aGUgY29kZSBpcyBvayB0aGF0DQoNCnNvbWV0
aGluZyB3YXMgbm90IGluaXRpYWxpemVkIGFuZCB3b24ndCBiZSB0b3VjaGVkIGF0IHJ1bi10aW1l
Lg0KDQpMZXQncyBzZWUgd2hhdCBvdGhlciB0aGluayBhYm91dCB0aGUgYXBwcm9hY2gNCg0KPg0K
Pg0KPiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMgYi9kcml2
ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPiBpbmRleCBkYTM0Y2U4NWRjODguLjk5MWJh
MGE5YjM1OSAxMDA2NDQNCj4gLS0tIGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMN
Cj4gKysrIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMNCj4gQEAgLTE1LDYgKzE1
LDcgQEANCj4gICAjaW5jbHVkZSA8eGVuL3hlbmJ1cy5oPg0KPiAgICNpbmNsdWRlIDx4ZW4vZXZl
bnRzLmg+DQo+ICAgI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4gKyNpbmNsdWRlIDx4ZW4veGVuLmg+
DQo+ICAgI2luY2x1ZGUgInBjaWJhY2suaCINCj4gICANCj4gICAjZGVmaW5lIElOVkFMSURfRVZU
Q0hOX0lSUSAgKC0xKQ0KPiBAQCAtNjg1LDggKzY4NiwxMiBAQCBzdGF0aWMgaW50IHhlbl9wY2li
a194ZW5idXNfcHJvYmUoc3RydWN0IHhlbmJ1c19kZXZpY2UgKmRldiwNCj4gICAJCQkJY29uc3Qg
c3RydWN0IHhlbmJ1c19kZXZpY2VfaWQgKmlkKQ0KPiAgIHsNCj4gICAJaW50IGVyciA9IDA7DQo+
IC0Jc3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXYgPSBhbGxvY19wZGV2KGRldik7DQo+ICsJ
c3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXY7DQo+ICsNCj4gKwlpZiAoIXhlbl9wdl9kb21h
aW4oKSkNCj4gKwkJcmV0dXJuIDA7DQo+ICAgDQo+ICsJcGRldiA9IGFsbG9jX3BkZXYoZGV2KTsN
Cj4gICAJaWYgKHBkZXYgPT0gTlVMTCkgew0KPiAgIAkJZXJyID0gLUVOT01FTTsNCj4gICAJCXhl
bmJ1c19kZXZfZmF0YWwoZGV2LCBlcnIsDQo+IEBAIC03NDMsNiArNzQ4LDkgQEAgY29uc3Qgc3Ry
dWN0IHhlbl9wY2lia19iYWNrZW5kICpfX3JlYWRfbW9zdGx5IHhlbl9wY2lia19iYWNrZW5kOw0K
PiAgIA0KPiAgIGludCBfX2luaXQgeGVuX3BjaWJrX3hlbmJ1c19yZWdpc3Rlcih2b2lkKQ0KPiAg
IHsNCj4gKwlpZiAoIXhlbl9wdl9kb21haW4oKSkNCj4gKwkJcmV0dXJuIDA7DQo+ICsNCj4gICAJ
eGVuX3BjaWJrX2JhY2tlbmQgPSAmeGVuX3BjaWJrX3ZwY2lfYmFja2VuZDsNCj4gICAJaWYgKHBh
c3N0aHJvdWdoKQ0KPiAgIAkJeGVuX3BjaWJrX2JhY2tlbmQgPSAmeGVuX3BjaWJrX3Bhc3N0aHJv
dWdoX2JhY2tlbmQ7DQo+IEBAIC03NTIsNSArNzYwLDcgQEAgaW50IF9faW5pdCB4ZW5fcGNpYmtf
eGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+ICAgDQo+ICAgdm9pZCBfX2V4aXQgeGVuX3BjaWJrX3hl
bmJ1c191bnJlZ2lzdGVyKHZvaWQpDQo+ICAgew0KPiArCWlmICgheGVuX3B2X2RvbWFpbigpKQ0K
PiArCQlyZXR1cm47DQo+ICAgCXhlbmJ1c191bnJlZ2lzdGVyX2RyaXZlcigmeGVuX3BjaWJrX2Ry
aXZlcik7DQo+ICAgfQ==


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 05:21:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 05:21:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191307.341272 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYD3-0001X1-MN; Tue, 21 Sep 2021 05:20:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191307.341272; Tue, 21 Sep 2021 05:20:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYD3-0001Wu-Ir; Tue, 21 Sep 2021 05:20:49 +0000
Received: by outflank-mailman (input) for mailman id 191307;
 Tue, 21 Sep 2021 05:20:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSYD2-0001Wo-1r
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 05:20:48 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 75ac21b4-18e5-4a0f-ba15-3798ce7856e1;
 Tue, 21 Sep 2021 05:20:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 5352A1FE5E;
 Tue, 21 Sep 2021 05:20:45 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0CE8113B97;
 Tue, 21 Sep 2021 05:20:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id o35lAa1rSWGbVQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 05:20:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 75ac21b4-18e5-4a0f-ba15-3798ce7856e1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632201645; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=lvbCGXyMHGQFbnwXi0D4n1ECOkvMzpMs0d59kGp6hJY=;
	b=k1yBA/sRf5q96vlAh0hL+HeRX9weFcD5hoPx6fo+ivehdP+9drdra2nLTTIkmBYY8AWTaH
	gCZ/eNf3nW/iabwGshhTXpUyOIXMgp3c3OjTY+DiV4l0wcIgeZdvdD8oLUvQXto63ndMX7
	Gp7VJY6kZhNOMj8PyHD9QOp4hOGXbYg=
To: Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Message-ID: <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
Date: Tue, 21 Sep 2021 07:20:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ESRU9eNpqUFGLK4iTRkg44aEQYVXk8525"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ESRU9eNpqUFGLK4iTRkg44aEQYVXk8525
Content-Type: multipart/mixed; boundary="iVs1Yxi1J5q6XdRjnpo9ChDEi2eCxYaSC";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>

--iVs1Yxi1J5q6XdRjnpo9ChDEi2eCxYaSC
Content-Type: multipart/mixed;
 boundary="------------1FB2498030FA73179B94032B"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------1FB2498030FA73179B94032B
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 01:16, Stefano Stabellini wrote:
> On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
>> On 20.09.21 14:30, Juergen Gross wrote:
>>> On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
>>>> Hello, Stefano!
>>>>
>>>> On 18.09.21 00:45, Stefano Stabellini wrote:
>>>>> Hi Oleksandr,
>>>>>
>>>>> Why do you want to enable pciback on ARM? Is it only to "disable" a=
 PCI
>>>>> device in Dom0 so that it can be safely assigned to a DomU?
>>>> Not only that
>>>>>
>>>>> I am asking because actually I don't think we want to enable the PV=
 PCI
>>>>> backend feature of pciback on ARM, right? That would clash with the=
 PCI
>>>>> assignment work you have been doing in Xen. They couldn't both work=
 at
>>>>> the same time.
>>>> Correct, it is not used
>>>>>
>>>>> If we only need pciback to "park" a device in Dom0, wouldn't it be
>>>>> possible and better to use pci-stub instead?
>>>>
>>>> Not only that, so pci-stub is not enough
>>>>
>>>> The functionality which is implemented by the pciback and the toolst=
ack
>>>> and which is relevant/missing/needed for ARM:
>>>>
>>>> 1. pciback is used as a database for assignable PCI devices, e.g. xl=

>>>>  =C2=A0 =C2=A0=C2=A0 pci-assignable-{add|remove|list} manipulates th=
at list. So, whenever the
>>>>  =C2=A0 =C2=A0=C2=A0 toolstack needs to know which PCI devices can b=
e passed through it reads
>>>>  =C2=A0 =C2=A0=C2=A0 that from the relevant sysfs entries of the pci=
back.
>>>>
>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when passin=
g through
>>>>  =C2=A0 =C2=A0=C2=A0 a PCI device it needs to be unbound from the re=
levant device driver and bound
>>>>  =C2=A0 =C2=A0=C2=A0 to pciback (strictly speaking it is not require=
d that the device is bound to
>>>>  =C2=A0 =C2=A0=C2=A0 pciback, but pciback is again used as a databas=
e of the passed through PCI
>>>>  =C2=A0 =C2=A0=C2=A0 devices, so we can re-bind the devices back to =
their original drivers when
>>>>  =C2=A0 =C2=A0=C2=A0 guest domain shuts down)
>>>>
>>>> 3. Device reset
>>>>
>>>> We have previously discussed on xen-devel ML possible solutions to t=
hat as from the
>>>> above we see that pciback functionality is going to be only partiall=
y used on Arm.
>>>>
>>>> Please see [1] and [2]:
>>>>
>>>> 1. It is not acceptable to manage the assignable list in Xen itself
>>>>
>>>> 2. pciback can be split into two parts: PCI assignable/bind/reset ha=
ndling and
>>>> the rest like vPCI etc.
>>>>
>>>> 3. pcifront is not used on Arm
>>>
>>> It is neither in x86 PVH/HVM guests.
>> Didn't know that, thank you for pointing
>>>
>>>> So, limited use of the pciback is one of the bricks used to enable P=
CI passthrough
>>>> on Arm. It was enough to just re-structure the driver and have it ru=
n on Arm to achieve
>>>> all the goals above.
>>>>
>>>> If we still think it is desirable to break the pciback driver into "=
common" and "pcifront specific"
>>>> parts then it can be done, yet the patch is going to be the very fir=
st brick in that building.
>>>
>>> Doing this split should be done, as the pcifront specific part could =
be
>>> omitted on x86, too, in case no PV guests using PCI passthrough have =
to
>>> be supported.
>> Agree, that the final solution should have the driver split
>>>
>>>> So, I think this patch is still going to be needed besides which dir=
ection we take.
>>>
>>> Some kind of this patch, yes. It might look different in case the spl=
it
>>> is done first.
>>>
>>> I don't mind doing it in either sequence.
>>>
>> With this patch we have Arm on the same page as the above mentioned x8=
6 guests,
>>
>> e.g. the driver has unused code, but yet allows Arm to function now.
>>
>> At this stage of PCI passthrough on Arm it is yet enough. Long term, w=
hen
>>
>> the driver gets split, Arm will benefit from that split too, but unfor=
tunately I do not
>>
>> have enough bandwidth for that piece of work at the moment.
>=20
> That's fair and I don't want to scope-creep this simple patch asking fo=
r
> an enormous rework. At the same time I don't think we should enable the=

> whole of pciback on ARM because it would be erroneous and confusing.
>=20
> I am wonder if there is a simple:
>=20
> if (!xen_pv_domain())
>      return;
>=20
> That we could add in a couple of places in pciback to stop it from
> initializing the parts we don't care about. Something along these lines=

> (untested and probably incomplete).
>=20
> What do you guys think?

Uh no, not in this way, please. This will kill pci passthrough on x86
with dom0 running as PVH. I don't think this is working right now, but
adding more code making it even harder to work should be avoided.

> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback=
/xenbus.c
> index da34ce85dc88..991ba0a9b359 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -15,6 +15,7 @@
>   #include <xen/xenbus.h>
>   #include <xen/events.h>
>   #include <xen/pci.h>
> +#include <xen/xen.h>
>   #include "pciback.h"
>  =20
>   #define INVALID_EVTCHN_IRQ  (-1)
> @@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xenbus_de=
vice *dev,
>   				const struct xenbus_device_id *id)
>   {
>   	int err =3D 0;
> -	struct xen_pcibk_device *pdev =3D alloc_pdev(dev);
> +	struct xen_pcibk_device *pdev;
> +
> +	if (!xen_pv_domain())
> +		return 0;
>  =20
> +	pdev =3D alloc_pdev(dev);

This hunk isn't needed, as with bailing out of xen_pcibk_xenbus_register
early will result in xen_pcibk_xenbus_probe never being called.

>   	if (pdev =3D=3D NULL) {
>   		err =3D -ENOMEM;
>   		xenbus_dev_fatal(dev, err,
> @@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly xen_p=
cibk_backend;
>  =20
>   int __init xen_pcibk_xenbus_register(void)
>   {
> +	if (!xen_pv_domain())
> +		return 0;
> +

Use #ifdef CONFIG_X86 instead.

>   	xen_pcibk_backend =3D &xen_pcibk_vpci_backend;
>   	if (passthrough)
>   		xen_pcibk_backend =3D &xen_pcibk_passthrough_backend;
> @@ -752,5 +760,7 @@ int __init xen_pcibk_xenbus_register(void)
>  =20
>   void __exit xen_pcibk_xenbus_unregister(void)
>   {
> +	if (!xen_pv_domain())
> +		return;

#ifdef again.

>   	xenbus_unregister_driver(&xen_pcibk_driver);
>   }
>=20


Juergen

--------------1FB2498030FA73179B94032B
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------1FB2498030FA73179B94032B--

--iVs1Yxi1J5q6XdRjnpo9ChDEi2eCxYaSC--

--ESRU9eNpqUFGLK4iTRkg44aEQYVXk8525
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJa6wFAwAAAAAACgkQsN6d1ii/Ey9M
mQf7BFzwYtokoUlkMp5SD42+/7LleJEYVw1bPFN5z5g/7KtBILe02JdnU6Hp5CZ4FHtnEhQmWp+y
4AstC6mck9avyv3uWwTy4kPxk67mf18GkO8T9c7oe1s6GnnpKV5Ulreicpqr0E0anxqr1N6WqXP1
5jEnKemD/aydluAqEqov2Jf2VyRCxNe+vpNzwSL4buGxLbT9NBEw4u0XRyl4QLA8v2UzqHBuStaC
Lr292vTAowcS8fppG4pVfpkXnUppIZaa95ptsT6MS1eEc5rRf/jLULf9I+ZlFyOfIblBbuIM5EC9
XqQyTR4JmSuG+0s2xHrJUclR4E9Q0/SijiFH/0Q+Qw==
=oukX
-----END PGP SIGNATURE-----

--ESRU9eNpqUFGLK4iTRkg44aEQYVXk8525--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 05:52:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 05:52:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191313.341284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYhN-00051W-2q; Tue, 21 Sep 2021 05:52:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191313.341284; Tue, 21 Sep 2021 05:52:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYhM-00051P-W0; Tue, 21 Sep 2021 05:52:08 +0000
Received: by outflank-mailman (input) for mailman id 191313;
 Tue, 21 Sep 2021 05:52:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSYhL-00051J-Bt
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 05:52:07 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0cd6095c-1aa0-11ec-b89b-12813bfff9fa;
 Tue, 21 Sep 2021 05:52:05 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L5mxts014878; 
 Tue, 21 Sep 2021 05:52:03 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b6xr2h9w2-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 05:52:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6067.eurprd03.prod.outlook.com (2603:10a6:208:15e::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Tue, 21 Sep
 2021 05:51:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 05:51:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cd6095c-1aa0-11ec-b89b-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fcys5ssmyzGK87bAAxQTZLJHRSCpUExoBat5mO3XLGoEpLcC0+E4vgevkj2z4cJ+n2wNj/jgRpxh56nJ8RKGlasp+ZvDSwUAhqdpLu51BEr+2cDXPbnyS58Yn0eceSsxxVP43SWlBvYKkp26n/4w34xXbDCPmdLofeAnQ0akpJbtk6jB0L8W4HNA6FZeLjKtAgExQNcIzAc1U/8DmsIsQRaduEl5FIoIT3D0ToMA8NdEy5+iXqx3sUiVORqu70V/u96+0ra9U/Y2jhyRANoPWs3SXAZI6eOu2hrhVhhWebf09QEmLvR++3j52uv7ADYDUUIIIpLt+Td9FqJr8nIhGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4b6BvHvGl6harng3BJDJHlMvVVQEncyR9TRwiV/Vvx8=;
 b=UTq2Ax2s/dk0JooqGkgmeYe5mvAZhYpVebaqNhToiA08W3jrSyVsCkJve/pNejg16Wi9w4to5AUobiwlqjACw/+0wkNEVvbl+1uWKnVPdkMQqpR3RQW5EuonyCDwI2drRNIe8ztl84YuUAyft8RxSYDdUayIGtV0WaM+NecpU9cL8Z3Wx+g6Xp0PzYCDVw/8zh+SV0LuhkEgXAJNTOAD5LigmEqxiEjEeBF1Et8tZQdwH/2gbQI3j5Fv3l7W+b5NjP0imueYYt1VkcEzVbQLGM8LqFsSVn1IVlRPWAVUynzAqUaBfKEFBPH4kcajI8tA6IbKlhiTZOTNUj6CZhRusw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4b6BvHvGl6harng3BJDJHlMvVVQEncyR9TRwiV/Vvx8=;
 b=5yPlqlZZccINQsmas4v/YR4n9+/irAI5ALB4b8HhtepeHyN6FPkoKiFrn4UNutRUHVLRfMV25+hY+VuQXP2NKUNCBSqcbXnmNTyXCX037fe/zG5/mwKTR/xujtgSV1z2UczJTUsbR1QnIrwwlvdr/83NlF+Va5UlWoAg6/2JfQcwJyAuBPUSNFksAEoCOzOU9hEVi2vfTX71xx6fKRU313syfI7I7TmYjl5Mzf3lB893F7yBiQVLADyBQJiu65KhiAInroktts9B4gMnZj0keSMsA/kKCtH860b/OIIJD49ckWK8MasRTzPbQX9rl7sKIYTbV9Q3ZBC5ZuzW4sxHfg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAGXhAIAACLuA
Date: Tue, 21 Sep 2021 05:51:59 +0000
Message-ID: <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
In-Reply-To: <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 65f234ac-1d38-4cb7-ac97-08d97cc3ed9c
x-ms-traffictypediagnostic: AM0PR03MB6067:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB6067A8116A39F801AF63A214E7A19@AM0PR03MB6067.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 EepylDbOu4trE/IJKaDkqJ9shMNzqkMI2ESn/DPXEmBEcNVCo4BB4KcIiOs6BUU3P21/bvqA+BNFdxp7BZCuk4bw0tASsEAi4uS9QP4g9PnBt1hxJW15vu2NxR/RQEzCx6p/MT7MC95K3/+nEBT4AcmRuDX4U9HbrQ92eNnJJsGPlCtKutbkrxXgKyZqN1pPMPMXhvh22hbyGTiMFgqFbiCol9TzybpadI9xtEFW7n04PaTCdjIWYWmvw3asq08Uc5rOGgp8AaKTojBF2H3DRkX2jM2ktUL91oLkj+KXyyZ9zIeHFQd52t9ksCRjynbWlndBFfs66BJCHctXnhKy2Zm/kkU59Ho8pTACGW18yKi+dL231GOiNuaADwU899uD9JlDWeG1vRM1wpuVU/AS4NFB2JjI5wn2LKzp7XNB6RvBDjL04jNklzDTUX1EqY8B9woikVsK1VE8CaJ9ZHfirz57pzc4dGvqXitdzaFX/NAjNi3c3MgwEkoHr1kXuxT3LHx+IuUjFGFA4b8rxrXOMVQBNY0VGg4uizDLuR1hyfluWdBgdOSR9H6q/Zxc/Qc0+qGJEkScjbMIAyfhxR4m1PruEGIih9ywSBmUmHFwfSC/JQSW5tIjmOip+7++29QprmhYCdWk7+bbPxva2tadrqrdJmRSRmBKl6+Txt2f5//BfgGTvfFIgoT+5GUX77wc9xzCagNrUN2dAArvHJHv77i9gzd7T6ZGf8UhmiqKUuu/wE0aVXveAyTGIOceDzI0
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(5660300002)(83380400001)(122000001)(316002)(26005)(2906002)(55236004)(53546011)(6506007)(71200400001)(2616005)(186003)(54906003)(110136005)(8936002)(36756003)(66476007)(86362001)(64756008)(66446008)(66946007)(31686004)(91956017)(31696002)(76116006)(66556008)(8676002)(4326008)(508600001)(6486002)(38070700005)(6512007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?VTNBVkRORjZ0QkY3UXRSbGdZNlZOUENQL0s5OXJ0Q3Qwd2FqWTI2SWtpbm4w?=
 =?utf-8?B?U1JRelNhMHNzY1FkcitoaThHOWUxbXQ0Y3I5dFZ0Rk1uc2hGaUVuZGpDN1BP?=
 =?utf-8?B?Y3hvRFpjcGptMU1mSitEeGcwOVdUNFBla2tpMlM4VGlkdFFHUjJNMndBRERT?=
 =?utf-8?B?dW9NcFRNSmVteWt4bS85WUNxY1U3UlkwbzVwckxLT3NzZ0FRckgrQ0xpUE12?=
 =?utf-8?B?VTBqaHNBdkdUaFpvR1Vjd0w3RXBYbHJ3Rnh6M2R1MC80Q2lqTDVDRC8yUTZy?=
 =?utf-8?B?dkc5S2s3cnVKUnhvc29mMlhJdUk1a05pUnZ1WUlHWXVRZnhjajI0V1Nmd3Nt?=
 =?utf-8?B?cU5CSEdQUHRwN0IyaHREMzlZVEs0bllTZElHejRIK24vdjh4OHJzV2gzSWdx?=
 =?utf-8?B?YVVWQ3ROM0NFZFB4UXROZnpDcUVKMkk0SktqZG10dGJmaDVsS0hkdVd6Z2ZP?=
 =?utf-8?B?TnhZSUI5QnpwUHZPYU1pYWhZaUdPbUw4TTZDN3M1Y3pqQStLcWQwbUpUZTUr?=
 =?utf-8?B?cTQweFlRQXZpclRocS8yR1dBcUh2MTJVbndDTmxQd1N6dTVMZ0ZTZC9EUlhJ?=
 =?utf-8?B?cjBkQ3pkb0JwbklncklwanMzK1JsZVlrQi9TT0ZmYnFMZ0pqVUhWMVNEaWl6?=
 =?utf-8?B?RDk3c0U5ZmNqN3BKQXRhbXd4Z1UyN1RtNFBhcmlqb1NOMDIwRFI2Zm5PcEsr?=
 =?utf-8?B?bjVRcFM2RE91a1hnMTNTdnk0RzhiNEF1SDBTelZ2cEhhYjZoWmU2elZybkFi?=
 =?utf-8?B?RjM2VjVFMnhraWM5RTVaSDdXWi9Nd1ZMQi9aOGdZVGt2Y2RCWlZiYXFjK0Uv?=
 =?utf-8?B?a3NjemJaZHVkS3FnWjBvRnNOUGJFOUpVM0M5YllRRTVSNmM3S040NnR1NUZW?=
 =?utf-8?B?OG9GZGVzYTdzQ3hDK1JQdlo5bmtZQXVVbXBIZWpkMWNBZlpLMUVEdVZqd1Bh?=
 =?utf-8?B?eEtVL1lrTVFuZW4xRHgvaEcrU2Q0ZzRSQXJsanlhQ01Ea1Zxd0YxUXhDTEhW?=
 =?utf-8?B?aTF3MEdPemQvc2ZyUUxRV2tLakdWYjA0RmJGcGc5YzJDeUh3dXk4YkhrdktP?=
 =?utf-8?B?UFNEQ2Vvb3EvemFxcEJlQVRCbTdQVVJFRUt0dmhHaDB4MDR2NENtTlF3U250?=
 =?utf-8?B?TWkxbVVXdDBxcHB6dGd0ZUNyZFA2ckY1VE1HcU5HdjJDN05IQTl1dEVIczgz?=
 =?utf-8?B?QnkxakJDWEdMZmd1SE9GTzUvN3BOWTJzeDRJa0RCTUVHbVFNV1EzRTlQYXNB?=
 =?utf-8?B?Z1pvWVhmY2pmTDlLeDVMaEZoUktLdXFncjRYbm5INThlMElUUXhlQWdid0lH?=
 =?utf-8?B?YVRmUHpGS3ZlbVBkRm8yNlAwTVZVVXVWTU90Nmt2dk04cmxVSDYzN1FML1JD?=
 =?utf-8?B?dkxhczU1NlRjcnhSY2d6SWJoaElxN0w3QmhXc3FwalZpSmpxc2Q0KzlEVTRB?=
 =?utf-8?B?SytpMFMyNkU5amJkN3JtNGxuL2lMaTlRNDBpb0oyMkhYNndHMG9WMWZMb1NT?=
 =?utf-8?B?bGJzU0lRbDZvRnhFMHpZbWE5NnFsRHpEZUlza2RQbUxLQ09Fdm1UdUg4K08x?=
 =?utf-8?B?L254RTlQR00vYVBhL2EyOVRQUVlGUFBQS2I4MlgwVmhBOUwzdFJSLytZZnpB?=
 =?utf-8?B?SWJ1bVhCTFF4R2pBaUNHbXREVTdaVzdBR3lsbmhhTnVXQVdUeFMySHNEM1pH?=
 =?utf-8?B?WkMva21QNmhGa01rTXpVL0NnWTVQaXAyUFI2bWMxakkrNGxZcHJBS2w5dlNQ?=
 =?utf-8?B?VnRPTVlzWTFnN01CLzhyU2RIYWJvem04RC95MlFDbzRUeXNhWnBOSFFkMWR6?=
 =?utf-8?B?cGVrSi96bmdteCtpMU5RUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <6413BF715B895D408A3094B32829DF99@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 65f234ac-1d38-4cb7-ac97-08d97cc3ed9c
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 05:51:59.5535
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: NT6Fp0tGlWYsAVlqOSufjUuMHcqCrc4r49ZytqNbaQKxHObq/vcnnchXYgtcwiCYBoZBRmWECfk8OQFgbk1dNbQJz9oLnZ27/Qn0Z28Oow54BFUihazokqKBkZGORgHz
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6067
X-Proofpoint-ORIG-GUID: 1Ne9NicsKgy2jpSW4D6M-ZfxcCDeUEfm
X-Proofpoint-GUID: 1Ne9NicsKgy2jpSW4D6M-ZfxcCDeUEfm
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 clxscore=1015 impostorscore=0 phishscore=0 adultscore=0 lowpriorityscore=0
 mlxlogscore=999 suspectscore=0 malwarescore=0 mlxscore=0 spamscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210037

DQpPbiAyMS4wOS4yMSAwODoyMCwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjEuMDkuMjEg
MDE6MTYsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4+IE9uIE1vbiwgMjAgU2VwIDIwMjEs
IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+IE9uIDIwLjA5LjIxIDE0OjMwLCBK
dWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+PiBPbiAyMC4wOS4yMSAwNzoyMywgT2xla3NhbmRyIEFu
ZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+PiBIZWxsbywgU3RlZmFubyENCj4+Pj4+DQo+Pj4+PiBP
biAxOC4wOS4yMSAwMDo0NSwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPj4+Pj4+IEhpIE9s
ZWtzYW5kciwNCj4+Pj4+Pg0KPj4+Pj4+IFdoeSBkbyB5b3Ugd2FudCB0byBlbmFibGUgcGNpYmFj
ayBvbiBBUk0/IElzIGl0IG9ubHkgdG8gImRpc2FibGUiIGEgUENJDQo+Pj4+Pj4gZGV2aWNlIGlu
IERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRvIGEgRG9tVT8NCj4+Pj4+
IE5vdCBvbmx5IHRoYXQNCj4+Pj4+Pg0KPj4+Pj4+IEkgYW0gYXNraW5nIGJlY2F1c2UgYWN0dWFs
bHkgSSBkb24ndCB0aGluayB3ZSB3YW50IHRvIGVuYWJsZSB0aGUgUFYgUENJDQo+Pj4+Pj4gYmFj
a2VuZCBmZWF0dXJlIG9mIHBjaWJhY2sgb24gQVJNLCByaWdodD8gVGhhdCB3b3VsZCBjbGFzaCB3
aXRoIHRoZSBQQ0kNCj4+Pj4+PiBhc3NpZ25tZW50IHdvcmsgeW91IGhhdmUgYmVlbiBkb2luZyBp
biBYZW4uIFRoZXkgY291bGRuJ3QgYm90aCB3b3JrIGF0DQo+Pj4+Pj4gdGhlIHNhbWUgdGltZS4N
Cj4+Pj4+IENvcnJlY3QsIGl0IGlzIG5vdCB1c2VkDQo+Pj4+Pj4NCj4+Pj4+PiBJZiB3ZSBvbmx5
IG5lZWQgcGNpYmFjayB0byAicGFyayIgYSBkZXZpY2UgaW4gRG9tMCwgd291bGRuJ3QgaXQgYmUN
Cj4+Pj4+PiBwb3NzaWJsZSBhbmQgYmV0dGVyIHRvIHVzZSBwY2ktc3R1YiBpbnN0ZWFkPw0KPj4+
Pj4NCj4+Pj4+IE5vdCBvbmx5IHRoYXQsIHNvIHBjaS1zdHViIGlzIG5vdCBlbm91Z2gNCj4+Pj4+
DQo+Pj4+PiBUaGUgZnVuY3Rpb25hbGl0eSB3aGljaCBpcyBpbXBsZW1lbnRlZCBieSB0aGUgcGNp
YmFjayBhbmQgdGhlIHRvb2xzdGFjaw0KPj4+Pj4gYW5kIHdoaWNoIGlzIHJlbGV2YW50L21pc3Np
bmcvbmVlZGVkIGZvciBBUk06DQo+Pj4+Pg0KPj4+Pj4gMS4gcGNpYmFjayBpcyB1c2VkIGFzIGEg
ZGF0YWJhc2UgZm9yIGFzc2lnbmFibGUgUENJIGRldmljZXMsIGUuZy4geGwNCj4+Pj4+IMKgwqAg
wqDCoCBwY2ktYXNzaWduYWJsZS17YWRkfHJlbW92ZXxsaXN0fSBtYW5pcHVsYXRlcyB0aGF0IGxp
c3QuIFNvLCB3aGVuZXZlciB0aGUNCj4+Pj4+IMKgwqAgwqDCoCB0b29sc3RhY2sgbmVlZHMgdG8g
a25vdyB3aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2ggaXQgcmVhZHMNCj4+
Pj4+IMKgwqAgwqDCoCB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVudHJpZXMgb2YgdGhl
IHBjaWJhY2suDQo+Pj4+Pg0KPj4+Pj4gMi4gcGNpYmFjayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVu
Ym91bmQgUENJIGRldmljZXMsIGUuZy4gd2hlbiBwYXNzaW5nIHRocm91Z2gNCj4+Pj4+IMKgwqAg
wqDCoCBhIFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFu
dCBkZXZpY2UgZHJpdmVyIGFuZCBib3VuZA0KPj4+Pj4gwqDCoCDCoMKgIHRvIHBjaWJhY2sgKHN0
cmljdGx5IHNwZWFraW5nIGl0IGlzIG5vdCByZXF1aXJlZCB0aGF0IHRoZSBkZXZpY2UgaXMgYm91
bmQgdG8NCj4+Pj4+IMKgwqAgwqDCoCBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1c2Vk
IGFzIGEgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSQ0KPj4+Pj4gwqDCoCDCoMKg
IGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5kIHRoZSBkZXZpY2VzIGJhY2sgdG8gdGhlaXIgb3Jp
Z2luYWwgZHJpdmVycyB3aGVuDQo+Pj4+PiDCoMKgIMKgwqAgZ3Vlc3QgZG9tYWluIHNodXRzIGRv
d24pDQo+Pj4+Pg0KPj4+Pj4gMy4gRGV2aWNlIHJlc2V0DQo+Pj4+Pg0KPj4+Pj4gV2UgaGF2ZSBw
cmV2aW91c2x5IGRpc2N1c3NlZCBvbiB4ZW4tZGV2ZWwgTUwgcG9zc2libGUgc29sdXRpb25zIHRv
IHRoYXQgYXMgZnJvbSB0aGUNCj4+Pj4+IGFib3ZlIHdlIHNlZSB0aGF0IHBjaWJhY2sgZnVuY3Rp
b25hbGl0eSBpcyBnb2luZyB0byBiZSBvbmx5IHBhcnRpYWxseSB1c2VkIG9uIEFybS4NCj4+Pj4+
DQo+Pj4+PiBQbGVhc2Ugc2VlIFsxXSBhbmQgWzJdOg0KPj4+Pj4NCj4+Pj4+IDEuIEl0IGlzIG5v
dCBhY2NlcHRhYmxlIHRvIG1hbmFnZSB0aGUgYXNzaWduYWJsZSBsaXN0IGluIFhlbiBpdHNlbGYN
Cj4+Pj4+DQo+Pj4+PiAyLiBwY2liYWNrIGNhbiBiZSBzcGxpdCBpbnRvIHR3byBwYXJ0czogUENJ
IGFzc2lnbmFibGUvYmluZC9yZXNldCBoYW5kbGluZyBhbmQNCj4+Pj4+IHRoZSByZXN0IGxpa2Ug
dlBDSSBldGMuDQo+Pj4+Pg0KPj4+Pj4gMy4gcGNpZnJvbnQgaXMgbm90IHVzZWQgb24gQXJtDQo+
Pj4+DQo+Pj4+IEl0IGlzIG5laXRoZXIgaW4geDg2IFBWSC9IVk0gZ3Vlc3RzLg0KPj4+IERpZG4n
dCBrbm93IHRoYXQsIHRoYW5rIHlvdSBmb3IgcG9pbnRpbmcNCj4+Pj4NCj4+Pj4+IFNvLCBsaW1p
dGVkIHVzZSBvZiB0aGUgcGNpYmFjayBpcyBvbmUgb2YgdGhlIGJyaWNrcyB1c2VkIHRvIGVuYWJs
ZSBQQ0kgcGFzc3Rocm91Z2gNCj4+Pj4+IG9uIEFybS4gSXQgd2FzIGVub3VnaCB0byBqdXN0IHJl
LXN0cnVjdHVyZSB0aGUgZHJpdmVyIGFuZCBoYXZlIGl0IHJ1biBvbiBBcm0gdG8gYWNoaWV2ZQ0K
Pj4+Pj4gYWxsIHRoZSBnb2FscyBhYm92ZS4NCj4+Pj4+DQo+Pj4+PiBJZiB3ZSBzdGlsbCB0aGlu
ayBpdCBpcyBkZXNpcmFibGUgdG8gYnJlYWsgdGhlIHBjaWJhY2sgZHJpdmVyIGludG8gImNvbW1v
biIgYW5kICJwY2lmcm9udCBzcGVjaWZpYyINCj4+Pj4+IHBhcnRzIHRoZW4gaXQgY2FuIGJlIGRv
bmUsIHlldCB0aGUgcGF0Y2ggaXMgZ29pbmcgdG8gYmUgdGhlIHZlcnkgZmlyc3QgYnJpY2sgaW4g
dGhhdCBidWlsZGluZy4NCj4+Pj4NCj4+Pj4gRG9pbmcgdGhpcyBzcGxpdCBzaG91bGQgYmUgZG9u
ZSwgYXMgdGhlIHBjaWZyb250IHNwZWNpZmljIHBhcnQgY291bGQgYmUNCj4+Pj4gb21pdHRlZCBv
biB4ODYsIHRvbywgaW4gY2FzZSBubyBQViBndWVzdHMgdXNpbmcgUENJIHBhc3N0aHJvdWdoIGhh
dmUgdG8NCj4+Pj4gYmUgc3VwcG9ydGVkLg0KPj4+IEFncmVlLCB0aGF0IHRoZSBmaW5hbCBzb2x1
dGlvbiBzaG91bGQgaGF2ZSB0aGUgZHJpdmVyIHNwbGl0DQo+Pj4+DQo+Pj4+PiBTbywgSSB0aGlu
ayB0aGlzIHBhdGNoIGlzIHN0aWxsIGdvaW5nIHRvIGJlIG5lZWRlZCBiZXNpZGVzIHdoaWNoIGRp
cmVjdGlvbiB3ZSB0YWtlLg0KPj4+Pg0KPj4+PiBTb21lIGtpbmQgb2YgdGhpcyBwYXRjaCwgeWVz
LiBJdCBtaWdodCBsb29rIGRpZmZlcmVudCBpbiBjYXNlIHRoZSBzcGxpdA0KPj4+PiBpcyBkb25l
IGZpcnN0Lg0KPj4+Pg0KPj4+PiBJIGRvbid0IG1pbmQgZG9pbmcgaXQgaW4gZWl0aGVyIHNlcXVl
bmNlLg0KPj4+Pg0KPj4+IFdpdGggdGhpcyBwYXRjaCB3ZSBoYXZlIEFybSBvbiB0aGUgc2FtZSBw
YWdlIGFzIHRoZSBhYm92ZSBtZW50aW9uZWQgeDg2IGd1ZXN0cywNCj4+Pg0KPj4+IGUuZy4gdGhl
IGRyaXZlciBoYXMgdW51c2VkIGNvZGUsIGJ1dCB5ZXQgYWxsb3dzIEFybSB0byBmdW5jdGlvbiBu
b3cuDQo+Pj4NCj4+PiBBdCB0aGlzIHN0YWdlIG9mIFBDSSBwYXNzdGhyb3VnaCBvbiBBcm0gaXQg
aXMgeWV0IGVub3VnaC4gTG9uZyB0ZXJtLCB3aGVuDQo+Pj4NCj4+PiB0aGUgZHJpdmVyIGdldHMg
c3BsaXQsIEFybSB3aWxsIGJlbmVmaXQgZnJvbSB0aGF0IHNwbGl0IHRvbywgYnV0IHVuZm9ydHVu
YXRlbHkgSSBkbyBub3QNCj4+Pg0KPj4+IGhhdmUgZW5vdWdoIGJhbmR3aWR0aCBmb3IgdGhhdCBw
aWVjZSBvZiB3b3JrIGF0IHRoZSBtb21lbnQuDQo+Pg0KPj4gVGhhdCdzIGZhaXIgYW5kIEkgZG9u
J3Qgd2FudCB0byBzY29wZS1jcmVlcCB0aGlzIHNpbXBsZSBwYXRjaCBhc2tpbmcgZm9yDQo+PiBh
biBlbm9ybW91cyByZXdvcmsuIEF0IHRoZSBzYW1lIHRpbWUgSSBkb24ndCB0aGluayB3ZSBzaG91
bGQgZW5hYmxlIHRoZQ0KPj4gd2hvbGUgb2YgcGNpYmFjayBvbiBBUk0gYmVjYXVzZSBpdCB3b3Vs
ZCBiZSBlcnJvbmVvdXMgYW5kIGNvbmZ1c2luZy4NCg0KQXMgdGhlIGZpcnN0IHN0YWdlIGJlZm9y
ZSB0aGUgZHJpdmVyIGlzIHNwbGl0IG9yIGlmZGVmJ3MgdXNlZCAtIGNhbiB3ZSB0YWtlIHRoZSBw
YXRjaA0KDQphcyBpcyBub3c/IEluIGVpdGhlciB3YXkgd2UgY2hvc2UgdGhpcyBuZWVkcyB0byBi
ZSBkb25lLCBlLmcuIGVuYWJsZSBjb21waWxpbmcNCg0KZm9yIG90aGVyIGFyY2hpdGVjdHVyZXMg
YW5kIGNvbW1vbiBjb2RlIG1vdmUuDQoNCj4+DQo+PiBJIGFtIHdvbmRlciBpZiB0aGVyZSBpcyBh
IHNpbXBsZToNCj4+DQo+PiBpZiAoIXhlbl9wdl9kb21haW4oKSkNCj4+IMKgwqDCoMKgIHJldHVy
bjsNCj4+DQo+PiBUaGF0IHdlIGNvdWxkIGFkZCBpbiBhIGNvdXBsZSBvZiBwbGFjZXMgaW4gcGNp
YmFjayB0byBzdG9wIGl0IGZyb20NCj4+IGluaXRpYWxpemluZyB0aGUgcGFydHMgd2UgZG9uJ3Qg
Y2FyZSBhYm91dC4gU29tZXRoaW5nIGFsb25nIHRoZXNlIGxpbmVzDQo+PiAodW50ZXN0ZWQgYW5k
IHByb2JhYmx5IGluY29tcGxldGUpLg0KPj4NCj4+IFdoYXQgZG8geW91IGd1eXMgdGhpbms/DQo+
DQo+IFVoIG5vLCBub3QgaW4gdGhpcyB3YXksIHBsZWFzZS4gVGhpcyB3aWxsIGtpbGwgcGNpIHBh
c3N0aHJvdWdoIG9uIHg4Ng0KPiB3aXRoIGRvbTAgcnVubmluZyBhcyBQVkguIEkgZG9uJ3QgdGhp
bmsgdGhpcyBpcyB3b3JraW5nIHJpZ2h0IG5vdywgYnV0DQo+IGFkZGluZyBtb3JlIGNvZGUgbWFr
aW5nIGl0IGV2ZW4gaGFyZGVyIHRvIHdvcmsgc2hvdWxkIGJlIGF2b2lkZWQuDQo+DQo+PiBkaWZm
IC0tZ2l0IGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMgYi9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gaW5kZXggZGEzNGNlODVkYzg4Li45OTFiYTBhOWIzNTkg
MTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gKysr
IGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMNCj4+IEBAIC0xNSw2ICsxNSw3IEBA
DQo+PiDCoCAjaW5jbHVkZSA8eGVuL3hlbmJ1cy5oPg0KPj4gwqAgI2luY2x1ZGUgPHhlbi9ldmVu
dHMuaD4NCj4+IMKgICNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiArI2luY2x1ZGUgPHhlbi94ZW4u
aD4NCj4+IMKgICNpbmNsdWRlICJwY2liYWNrLmgiDQo+PiDCoCDCoCAjZGVmaW5lIElOVkFMSURf
RVZUQ0hOX0lSUcKgICgtMSkNCj4+IEBAIC02ODUsOCArNjg2LDEyIEBAIHN0YXRpYyBpbnQgeGVu
X3BjaWJrX3hlbmJ1c19wcm9iZShzdHJ1Y3QgeGVuYnVzX2RldmljZSAqZGV2LA0KPj4gwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdHJ1Y3QgeGVuYnVzX2RldmljZV9p
ZCAqaWQpDQo+PiDCoCB7DQo+PiDCoMKgwqDCoMKgIGludCBlcnIgPSAwOw0KPj4gLcKgwqDCoCBz
dHJ1Y3QgeGVuX3BjaWJrX2RldmljZSAqcGRldiA9IGFsbG9jX3BkZXYoZGV2KTsNCj4+ICvCoMKg
wqAgc3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXY7DQo+PiArDQo+PiArwqDCoMKgIGlmICgh
eGVuX3B2X2RvbWFpbigpKQ0KPj4gK8KgwqDCoMKgwqDCoMKgIHJldHVybiAwOw0KPj4gwqAgK8Kg
wqDCoCBwZGV2ID0gYWxsb2NfcGRldihkZXYpOw0KPg0KPiBUaGlzIGh1bmsgaXNuJ3QgbmVlZGVk
LCBhcyB3aXRoIGJhaWxpbmcgb3V0IG9mIHhlbl9wY2lia194ZW5idXNfcmVnaXN0ZXINCj4gZWFy
bHkgd2lsbCByZXN1bHQgaW4geGVuX3BjaWJrX3hlbmJ1c19wcm9iZSBuZXZlciBiZWluZyBjYWxs
ZWQuDQo+DQo+PiDCoMKgwqDCoMKgIGlmIChwZGV2ID09IE5VTEwpIHsNCj4+IMKgwqDCoMKgwqDC
oMKgwqDCoCBlcnIgPSAtRU5PTUVNOw0KPj4gwqDCoMKgwqDCoMKgwqDCoMKgIHhlbmJ1c19kZXZf
ZmF0YWwoZGV2LCBlcnIsDQo+PiBAQCAtNzQzLDYgKzc0OCw5IEBAIGNvbnN0IHN0cnVjdCB4ZW5f
cGNpYmtfYmFja2VuZCAqX19yZWFkX21vc3RseSB4ZW5fcGNpYmtfYmFja2VuZDsNCj4+IMKgIMKg
IGludCBfX2luaXQgeGVuX3BjaWJrX3hlbmJ1c19yZWdpc3Rlcih2b2lkKQ0KPj4gwqAgew0KPj4g
K8KgwqDCoCBpZiAoIXhlbl9wdl9kb21haW4oKSkNCj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4g
MDsNCj4+ICsNCj4NCj4gVXNlICNpZmRlZiBDT05GSUdfWDg2IGluc3RlYWQuDQoNClRoZSB0aXRs
ZSBvZiB0aGlzIHBhdGNoIHNheXMgdGhhdCB3ZSB3YW50IHRvIGFsbG93IHRoaXMgZHJpdmVyIGZv
ciBvdGhlciBhcmNocw0KDQphbmQgbm93IHdlIHdhbnQgdG8gaW50cm9kdWNlICIjaWZkZWYgQ09O
RklHX1g4NiIgd2hpY2ggZG9lc24ndCBzb3VuZA0KDQpyaWdodCB3aXRoIHRoYXQgcmVzcGVjdC4g
SW5zdGVhZCwgd2UgbWF5IHdhbnQgaGF2aW5nIHNvbWV0aGluZyBsaWtlIGENCg0KZGVkaWNhdGVk
IGdhdGUgZm9yIHRoaXMsIGUuZy4gIiNpZmRlZiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EX1NV
UFBfUFYiDQoNCm9yIHNvbWV0aGluZyB3aGljaCBpcyBhcmNoaXRlY3R1cmUgYWdub3N0aWMuDQoN
CkdhdGluZyBhbHNvIG1lYW5zIHRoYXQgd2UgYXJlIG5vdCB0aGlua2luZyBhYm91dCBzcGxpdHRp
bmcgdGhlIGJhY2tlbmQgZHJpdmVyIGludG8NCg0KdHdvIGRpZmZlcmVudCBvbmVzLCBlLmcuIG9u
ZSBmb3IgImNvbW1vbiIgY29kZSBhbmQgb25lIGZvciBQViBzdHVmZi4NCg0KT3RoZXJ3aXNlIHRo
aXMgaWZkZWZlcnkgd29uJ3QgYmUgbmVlZGVkLg0KDQo+DQo+PiDCoMKgwqDCoMKgIHhlbl9wY2li
a19iYWNrZW5kID0gJnhlbl9wY2lia192cGNpX2JhY2tlbmQ7DQo+PiDCoMKgwqDCoMKgIGlmIChw
YXNzdGhyb3VnaCkNCj4+IMKgwqDCoMKgwqDCoMKgwqDCoCB4ZW5fcGNpYmtfYmFja2VuZCA9ICZ4
ZW5fcGNpYmtfcGFzc3Rocm91Z2hfYmFja2VuZDsNCj4+IEBAIC03NTIsNSArNzYwLDcgQEAgaW50
IF9faW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+PiDCoCDCoCB2b2lkIF9f
ZXhpdCB4ZW5fcGNpYmtfeGVuYnVzX3VucmVnaXN0ZXIodm9pZCkNCj4+IMKgIHsNCj4+ICvCoMKg
wqAgaWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuOw0KPg0K
PiAjaWZkZWYgYWdhaW4uDQo+DQo+PiB4ZW5idXNfdW5yZWdpc3Rlcl9kcml2ZXIoJnhlbl9wY2li
a19kcml2ZXIpOw0KPj4gwqAgfQ0KPj4NCj4NCj4NCj4gSnVlcmdlbg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 06:07:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 06:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191322.341295 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYwG-0006v6-HY; Tue, 21 Sep 2021 06:07:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191322.341295; Tue, 21 Sep 2021 06:07:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYwG-0006uz-EI; Tue, 21 Sep 2021 06:07:32 +0000
Received: by outflank-mailman (input) for mailman id 191322;
 Tue, 21 Sep 2021 06:07:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSYwF-0006ut-Kl
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 06:07:31 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b976b5b-eefb-4ab0-b5ad-707b6b550df7;
 Tue, 21 Sep 2021 06:07:30 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 23F031FE5E;
 Tue, 21 Sep 2021 06:07:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D9A9A13B93;
 Tue, 21 Sep 2021 06:07:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id a3XpM6B2SWEwZQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 06:07:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b976b5b-eefb-4ab0-b5ad-707b6b550df7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632204449; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=EMk+GC3syGEDD553AiTSALMMtJS88qIkMD4XQpesw/U=;
	b=BPOzHiGi7qbpQI4YeMZ+YMIH5PQfWTjB8ZPY+sw9YgrPLJWpacvvq6Vl2raYXH45sR6UmK
	3zR0PwIrVKC4MMQZ7Kpzh4xYPpyLT/ZXYoF9O1GI+dcwXOYqrzntzA6U2ops6i8IrIB57w
	vJLXrK9h7rLFjD4DDRLFOfuq3szsW/A=
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Message-ID: <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
Date: Tue, 21 Sep 2021 08:07:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="DlvyvRDyKovEmV7DXh8WfVHTPRjI52Ka8"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--DlvyvRDyKovEmV7DXh8WfVHTPRjI52Ka8
Content-Type: multipart/mixed; boundary="HJdGdPYCV6JJoHpoUhOSa2P0UnPvIQRni";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
In-Reply-To: <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>

--HJdGdPYCV6JJoHpoUhOSa2P0UnPvIQRni
Content-Type: multipart/mixed;
 boundary="------------D23FCEDD6DEB2FBE19FCAC30"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------D23FCEDD6DEB2FBE19FCAC30
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 07:51, Oleksandr Andrushchenko wrote:
>=20
> On 21.09.21 08:20, Juergen Gross wrote:
>> On 21.09.21 01:16, Stefano Stabellini wrote:
>>> On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
>>>> On 20.09.21 14:30, Juergen Gross wrote:
>>>>> On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
>>>>>> Hello, Stefano!
>>>>>>
>>>>>> On 18.09.21 00:45, Stefano Stabellini wrote:
>>>>>>> Hi Oleksandr,
>>>>>>>
>>>>>>> Why do you want to enable pciback on ARM? Is it only to "disable"=
 a PCI
>>>>>>> device in Dom0 so that it can be safely assigned to a DomU?
>>>>>> Not only that
>>>>>>>
>>>>>>> I am asking because actually I don't think we want to enable the =
PV PCI
>>>>>>> backend feature of pciback on ARM, right? That would clash with t=
he PCI
>>>>>>> assignment work you have been doing in Xen. They couldn't both wo=
rk at
>>>>>>> the same time.
>>>>>> Correct, it is not used
>>>>>>>
>>>>>>> If we only need pciback to "park" a device in Dom0, wouldn't it b=
e
>>>>>>> possible and better to use pci-stub instead?
>>>>>>
>>>>>> Not only that, so pci-stub is not enough
>>>>>>
>>>>>> The functionality which is implemented by the pciback and the tool=
stack
>>>>>> and which is relevant/missing/needed for ARM:
>>>>>>
>>>>>> 1. pciback is used as a database for assignable PCI devices, e.g. =
xl
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 pci-assignable-{add|remove|list} manipu=
lates that list. So, whenever the
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 toolstack needs to know which PCI devic=
es can be passed through it reads
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 that from the relevant sysfs entries of=
 the pciback.
>>>>>>
>>>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when pass=
ing through
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 a PCI device it needs to be unbound fro=
m the relevant device driver and bound
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 to pciback (strictly speaking it is not=
 required that the device is bound to
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 pciback, but pciback is again used as a=
 database of the passed through PCI
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 devices, so we can re-bind the devices =
back to their original drivers when
>>>>>>  =C2=A0=C2=A0 =C2=A0=C2=A0 guest domain shuts down)
>>>>>>
>>>>>> 3. Device reset
>>>>>>
>>>>>> We have previously discussed on xen-devel ML possible solutions to=
 that as from the
>>>>>> above we see that pciback functionality is going to be only partia=
lly used on Arm.
>>>>>>
>>>>>> Please see [1] and [2]:
>>>>>>
>>>>>> 1. It is not acceptable to manage the assignable list in Xen itsel=
f
>>>>>>
>>>>>> 2. pciback can be split into two parts: PCI assignable/bind/reset =
handling and
>>>>>> the rest like vPCI etc.
>>>>>>
>>>>>> 3. pcifront is not used on Arm
>>>>>
>>>>> It is neither in x86 PVH/HVM guests.
>>>> Didn't know that, thank you for pointing
>>>>>
>>>>>> So, limited use of the pciback is one of the bricks used to enable=
 PCI passthrough
>>>>>> on Arm. It was enough to just re-structure the driver and have it =
run on Arm to achieve
>>>>>> all the goals above.
>>>>>>
>>>>>> If we still think it is desirable to break the pciback driver into=
 "common" and "pcifront specific"
>>>>>> parts then it can be done, yet the patch is going to be the very f=
irst brick in that building.
>>>>>
>>>>> Doing this split should be done, as the pcifront specific part coul=
d be
>>>>> omitted on x86, too, in case no PV guests using PCI passthrough hav=
e to
>>>>> be supported.
>>>> Agree, that the final solution should have the driver split
>>>>>
>>>>>> So, I think this patch is still going to be needed besides which d=
irection we take.
>>>>>
>>>>> Some kind of this patch, yes. It might look different in case the s=
plit
>>>>> is done first.
>>>>>
>>>>> I don't mind doing it in either sequence.
>>>>>
>>>> With this patch we have Arm on the same page as the above mentioned =
x86 guests,
>>>>
>>>> e.g. the driver has unused code, but yet allows Arm to function now.=

>>>>
>>>> At this stage of PCI passthrough on Arm it is yet enough. Long term,=
 when
>>>>
>>>> the driver gets split, Arm will benefit from that split too, but unf=
ortunately I do not
>>>>
>>>> have enough bandwidth for that piece of work at the moment.
>>>
>>> That's fair and I don't want to scope-creep this simple patch asking =
for
>>> an enormous rework. At the same time I don't think we should enable t=
he
>>> whole of pciback on ARM because it would be erroneous and confusing.
>=20
> As the first stage before the driver is split or ifdef's used - can we =
take the patch
> as is now? In either way we chose this needs to be done, e.g. enable co=
mpiling
> for other architectures and common code move.

Fine with me in principle. I need to take a more thorough look
at the patch, though.

>=20
>>>
>>> I am wonder if there is a simple:
>>>
>>> if (!xen_pv_domain())
>>>  =C2=A0=C2=A0=C2=A0=C2=A0 return;
>>>
>>> That we could add in a couple of places in pciback to stop it from
>>> initializing the parts we don't care about. Something along these lin=
es
>>> (untested and probably incomplete).
>>>
>>> What do you guys think?
>>
>> Uh no, not in this way, please. This will kill pci passthrough on x86
>> with dom0 running as PVH. I don't think this is working right now, but=

>> adding more code making it even harder to work should be avoided.
>>
>>> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciba=
ck/xenbus.c
>>> index da34ce85dc88..991ba0a9b359 100644
>>> --- a/drivers/xen/xen-pciback/xenbus.c
>>> +++ b/drivers/xen/xen-pciback/xenbus.c
>>> @@ -15,6 +15,7 @@
>>>  =C2=A0 #include <xen/xenbus.h>
>>>  =C2=A0 #include <xen/events.h>
>>>  =C2=A0 #include <xen/pci.h>
>>> +#include <xen/xen.h>
>>>  =C2=A0 #include "pciback.h"
>>>  =C2=A0 =C2=A0 #define INVALID_EVTCHN_IRQ=C2=A0 (-1)
>>> @@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xenbus_=
device *dev,
>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const struct xenbus_device_id *id)
>>>  =C2=A0 {
>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int err =3D 0;
>>> -=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev =3D alloc_pdev(dev)=
;
>>> +=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev;
>>> +
>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>>  =C2=A0 +=C2=A0=C2=A0=C2=A0 pdev =3D alloc_pdev(dev);
>>
>> This hunk isn't needed, as with bailing out of xen_pcibk_xenbus_regist=
er
>> early will result in xen_pcibk_xenbus_probe never being called.
>>
>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (pdev =3D=3D NULL) {
>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 err =3D -ENOM=
EM;
>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xenbus_dev_fa=
tal(dev, err,
>>> @@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly xen=
_pcibk_backend;
>>>  =C2=A0 =C2=A0 int __init xen_pcibk_xenbus_register(void)
>>>  =C2=A0 {
>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>> +
>>
>> Use #ifdef CONFIG_X86 instead.
>=20
> The title of this patch says that we want to allow this driver for othe=
r archs
> and now we want to introduce "#ifdef CONFIG_X86" which doesn't sound
> right with that respect. Instead, we may want having something like a
> dedicated gate for this, e.g. "#ifdef CONFIG_XEN_PCIDEV_BACKEND_SUPP_PV=
"
> or something which is architecture agnostic.

Something like that, yes. But I'd rather use CONFIG_XEN_PCIDEV_BACKEND
acting as this gate and introduce CONFIG_XEN_PCI_STUB for the stub
functionality needed on Arm. XEN_PCIDEV_BACKEND would depend on X86 and
select XEN_PCI_STUB, while on Arm XEN_PCI_STUB could be configured if
wanted. The splitting of the driver can still be done later.

> Gating also means that we are not thinking about splitting the backend =
driver into
> two different ones, e.g. one for "common" code and one for PV stuff.
> Otherwise this ifdefery won't be needed.

I just wanted to avoid the xen_pv_domain() tests creeping in, as
they are wrong IMO.


Juergen

--------------D23FCEDD6DEB2FBE19FCAC30
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------D23FCEDD6DEB2FBE19FCAC30--

--HJdGdPYCV6JJoHpoUhOSa2P0UnPvIQRni--

--DlvyvRDyKovEmV7DXh8WfVHTPRjI52Ka8
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJdqAFAwAAAAAACgkQsN6d1ii/Ey80
lAf/ag3+BoLQ5HT8A5VbqdaqsjYmqkN/wxhA5YhgRadB54IBkZ4K+bB+lbIfFKnSO2cVoB9JFdvf
AiPTc8Urmyigf/3s9GeJzUxfseJ7CoxiiH+aW6KcOmPvGb/dHii/oylqt/EHwKIeY/r41F9OX1AX
W2ksbdxBbD94bFWhddBMjHURsHbQskdhzeK66E77MKKBr48TOFha+ibsxrbMFqDFEQEF5Z2Ree5j
c+onWJI9PXFJmnPCbeqYw4VIht55u2SR0j9fOtBTo95Ni4MPR3YctL+zZ8lfn8mu1apUrT6ddNXv
0OmdXxv8bP7ldl17FUE0OVeZZTI9kUvjZIZZUZWBIQ==
=C9P/
-----END PGP SIGNATURE-----

--DlvyvRDyKovEmV7DXh8WfVHTPRjI52Ka8--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 06:10:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 06:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191328.341305 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYyx-0008HM-Vb; Tue, 21 Sep 2021 06:10:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191328.341305; Tue, 21 Sep 2021 06:10:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSYyx-0008HF-Sg; Tue, 21 Sep 2021 06:10:19 +0000
Received: by outflank-mailman (input) for mailman id 191328;
 Tue, 21 Sep 2021 06:10:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSYyx-0008H9-2w
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 06:10:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a5d0b09a-2b2d-47b7-a668-481382c0ad75;
 Tue, 21 Sep 2021 06:10:18 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2055.outbound.protection.outlook.com [104.47.0.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-f8fItshwP_up7H6nywsZbg-1; Tue, 21 Sep 2021 08:10:15 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4350.eurprd04.prod.outlook.com (2603:10a6:803:3e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 06:10:13 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 06:10:13 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0075.eurprd04.prod.outlook.com (2603:10a6:208:be::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 06:10:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5d0b09a-2b2d-47b7-a668-481382c0ad75
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632204617;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=T1NeMOJA8ig+J/2JyYIqUyNFp0jZMGc3hhutzA+lpWg=;
	b=heUdHgAyPFtP+oHYFFEl5Yym5id5eW+JsUCY3emjAQaFFsbgCUP8bpCsCp+3eOfrlPQZYC
	9qiLolrLmLvsqlats6Cq5IMupvYELXnXCJhPIyH+PmtP0P/57oOoCGnGWZl0bqMdiGgKJV
	fI7+TFodZhYo0ENtOuifyUYU9MZbh5Y=
X-MC-Unique: f8fItshwP_up7H6nywsZbg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b+DnEv43qWQYRBZiQGYPWqpS1cZw72xYHeLNfG3kPK6tcTk1fmITnoujO424aIZ6JCbFKYuEBoDY/vs8E3NDQguo3nCY9+buAbDtML+K3ewSpHegOV4i5VDBLBkwh0N2Lx7jEAOj6W6DaPpTvRciPjM+XfMcDT2Rq57dWLMyLqp0dMiexyCnGl+KRLOi2ZdC2pXw25U2JODuuUf804q36/xgCP6RTdyMl8LodCUWi/W5T9J5UWzkh8OcGEX9Zn3csMADRndCCfpO9WpN4lZtrsPyLam+sKQNGhangBIKqwqrY5vuUtDcffHwrBhgl+MyXlmj7bJBlaabnfv5CJR5iQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=T1NeMOJA8ig+J/2JyYIqUyNFp0jZMGc3hhutzA+lpWg=;
 b=NZJZp7ofPyzDvjGXSOF73G5/uWwYneOJetojwuPkxrV6NBEimuhLjdMiNyFINJKobAtmJVd2O8l+aXheUXm+V++ikF74IORiAssQs1rSUjc+KY7LvdJ0mSum4/V0p9hIpBczOai1pJNiDPoe5WghaL5I3IG7JmDFH1D/2ZXMDodwhOjhuB/1sA0RDl8fdkwBF28BwGZdG4+DepHBX7an9teMs+vn/zVt1PdZO2B9kYL1gKKfOM9qy7WS0/4x9lHGq233HoBcQKmC9qHN+mO+wNBcDbyhGHWS504+BLTHCj/fSnxMMOqcZFor5pLOC3Y0jcRgl8xfFNV/sweIKmgl5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: =?UTF-8?Q?Ping=c2=b2=3a_=5bPATCH=5d_x86=3a_drop_a_bogus_SHARED=5fM2?=
 =?UTF-8?Q?P=28=29_check_from_Dom0_building_code?=
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <47deb99d-3ffc-61e4-26e6-7e7ab186a79a@suse.com>
 <0accd732-71a2-b122-6c7b-bb8fc8b0f3cf@suse.com>
Message-ID: <27bd50f4-2216-ceb2-437a-f9eaa8f7c019@suse.com>
Date: Tue, 21 Sep 2021 08:10:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0accd732-71a2-b122-6c7b-bb8fc8b0f3cf@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0075.eurprd04.prod.outlook.com
 (2603:10a6:208:be::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 52d33556-1655-4cb1-663c-08d97cc67992
X-MS-TrafficTypeDiagnostic: VI1PR04MB4350:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4350D432F83940984A6803D6B3A19@VI1PR04MB4350.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	RlUD9cvR2G/s4Jc1NnasOyes3W1TmTI0IMVdRguiTqkTCf3WrOvBlJJOiTwBw3yoHGuROrwCsEUqhNWS1vQVii1WlIbGsGEWFhdFOVmNO9wfa18G8D+h00T6tKQUFFUscNFFHWrr8kGAoqBkU1TLtZuOkVELoCCscge6PLG0lR6kUnp0UCu/0kaSa/nZezrVagZc8dCb98yEVQ546oRrA8IAQYXvjjMHbpCDxH6WdvKBLGI8YlU2+K9vP3Fzc+RsasaaHls3BgrlfRHjHfTH0ZOBwy5k63qnUE0D7uG5Rq2KTk5MBus6B1hbzPXHGIF+FORaSrziOfy5iUGrX7KZHpXQ0tNP8VBVdV0PGr/hxUQdqUameowDHrKCGBidHage4hmm7gRGAV6/QiZWTWHsoCSHF8cN5gbF/pw+MGLeAIGREZqiNYkS+Vawu+7vXj7jPrwOmdPE2qnqg/YOarNEQ/AbkCcdhytqspAG0UW0aBZ6mhJTXgmTb4Y6VsqLLlhQftFL3fDGlZX+UclJUip2IUg7EwfT8tysfbWnsd91/bEaWE+6f2zkwpnebRTUyQxUMwaYLxOb7rHmWltDDLDlIjDbMlBwIp8FTUfmx/0ePKGOAdXu3OTHvLpG1sw6FsMLXL00tQTUrW++VjT/F65pzUOGTtJSsfNIwtjFoVbEtbKbjpSAi+gPybCQyAXCgK26KAtl/OXjYG6qib5if9DTgexMo378gaA2HO5cqaW1psQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(508600001)(956004)(4326008)(316002)(31696002)(110136005)(38100700002)(53546011)(86362001)(36756003)(5660300002)(66476007)(31686004)(6486002)(16576012)(26005)(66556008)(186003)(2616005)(8936002)(66946007)(4744005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dFVjZnNDei9QTTVqOGFQU1ZaaU9UOGttbmo4eFgzcFJDeFJHcFhabmowSlA4?=
 =?utf-8?B?elNOeU5tc21NbVNHL09KU29aajJweEJudmNDNGlLZEpKek1Ba2hCTGJtSXA0?=
 =?utf-8?B?VXlWQVo4ZC9KYkdBU2pDWExOT0Q5dDdUaUNJTzloMVJHTFNsZW9XT2E0dUVo?=
 =?utf-8?B?VytWYUt1Q1gzMkxCRnBMRUx5UElRSDlrNUt6aW5paVFDNElLejN1YXRoczF3?=
 =?utf-8?B?SXpsbUtDT1U0ZmFaZ0NTVWo4Tk44RlBMUWpKSXRsRGNPSXNzR0duUU9lQVJw?=
 =?utf-8?B?UzRGdnJ4ZEM0SVYvMTBob295VHVBbjhLRk1pTFB5QWd1RWFITWRiQnRTaXVP?=
 =?utf-8?B?VWQ1WUd6TXc0b2hjOTFFVGU0enhhMW05NzVjN1Zjc2MvbnFucXo5YnQvcEEw?=
 =?utf-8?B?Umx1enZ6N2tqRTFkb2hUTkdFMWhraThnSUlhN2FLRlNaY3lMK0FIcVFEODlt?=
 =?utf-8?B?YzZvQkYwWWR0WnNwSGNuNU1yUkIwZnpQZWdhc2NZd3RYVTdsQUxOUnFlMzdI?=
 =?utf-8?B?T1BJeGN6ZTExU0hyTGp1THgxOUx6Z0Q5cnllZjMzTFVuc1BrUEhhRmdydnVz?=
 =?utf-8?B?OElwZlRMbVlvUmQ3d21qZURjSXJRdmxZTm5tUjJ0RHoybXpJUUUwbERHNVBv?=
 =?utf-8?B?Vmg1T0duZ0ZaT0R6ZTV3SVVtZ0JxVzErSE1KQXVJMmNkd08zSWg4bWpGV2xS?=
 =?utf-8?B?ckpZTFVHbERMb3M1ZXoySE5MZFcyM294WnVGTW11MmN0bHZPZ0FoTEVxQWph?=
 =?utf-8?B?eTE3MFhoWHRBWE0zV0YxMUl1eXBtMVZ1YzFqWTlad1BXZWYvOHU5ZGF3dm5D?=
 =?utf-8?B?dDlrL010UjBsMWdPOTlGR2lWN3VlMGlKNmZ4RDJDeklTd3pEYmxaNXRGSVhk?=
 =?utf-8?B?anlvRHBSaGIzWGVKRWZpaEdkN2orMjM1dG1Qa1JNQWZ3TXRTZFU2eUFWclVU?=
 =?utf-8?B?WFVLbXlhS0tsMDdZV3V0WWxZZEFjekpSMVZwZzdSNFVtdS9oek5ka0g0Rzkr?=
 =?utf-8?B?S3RYRWVaL1l0S1N3NTBZRWV5WnFhdkpFT3pyMDFQT200RythSDNuTnpKR2tU?=
 =?utf-8?B?VTJxK3IvcTROWXJkbWVNQ09IR004cnVjQlhZOU5zdG1Ia0JKb21seHA5NjFO?=
 =?utf-8?B?QW1NUUZuVUZwWjRqdlROdXI2aVpxYm16UWI5bFdlY05rSUhISGdwbnZBVGZD?=
 =?utf-8?B?cnB5OHU0SVVIS3JQTTRrOXRPakdRR1B1dEoxdjFtdEs1aHhHYU95OGZBRzBJ?=
 =?utf-8?B?bWpoZ2QxRlBhY0s0RERCM3hnMW51bVpQd0VSRndHQVpFSVpuVEtFTUhEVHMw?=
 =?utf-8?B?TjcwQ0dPckMvZWVudllMcGR4N3M3UW8xcnpUSzkybG9NcTdQNTIzU0VYUGp3?=
 =?utf-8?B?QktTL2t4WkhCb2xSc2RDM2tWUUo5SVFTK25RaTZoWm1CZVAwY1hSbjBaWmlo?=
 =?utf-8?B?QVN5MFpGbGdDNndQNUpLT2kwQ3ArK1doaW1QN0FGZDZKdGlOclY5UWhRZ3ND?=
 =?utf-8?B?elg2MzJBVTNJNmQ0ZTlUNkxkdVR4Z3QyYnFLcW4xMVIxaHZTVURBVDd6VVFD?=
 =?utf-8?B?cXdoWTA4RTN5dGJDMmRZRGVDbmVNT2dreUdaeVN0dWF3YVpFSVlqRTA4QkQ1?=
 =?utf-8?B?TUFTMEpUUzJabGlyOHpjQ29pSnRBalYwYlJMWmVRUHdGSVVEWEJJOFQyQkFH?=
 =?utf-8?B?K1NCVDZqK292ZjE5UVEzd1NEOUw5ejgyVkVxQzlhMTlMRkZzQ1lmb1RSYS9m?=
 =?utf-8?Q?CpatS7zy50xREk28CNJjeDI19i7xil2IDbIUWR9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 52d33556-1655-4cb1-663c-08d97cc67992
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 06:10:13.6061
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HGEwl/OZ69Na/EgoSPj3mqLDhPhADA79cQH52ML6Lh++qYxJU1UjdZZftbq7WcujFjQjviaqX97oqhkyVzv8fA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4350

On 06.07.2021 09:37, Jan Beulich wrote:
> On 28.06.2021 13:52, Jan Beulich wrote:
>> If anything, a check covering a wider range of invalid M2P entries ought
>> to be used (e.g. VALID_M2P()). But since everything is fully under Xen's
>> control at this stage, simply remove the BUG_ON().
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> I didn't expect this to be controversial, so may I please ask for an ack
> (or otherwise)?

To be quite honest, I find it very strange that even simple changes like
this one sit un-responded to for months. This isn't the only example ...

Thanks, Jan

>> --- a/xen/arch/x86/pv/dom0_build.c
>> +++ b/xen/arch/x86/pv/dom0_build.c
>> @@ -815,7 +815,6 @@ int __init dom0_construct_pv(struct doma
>>      page_list_for_each ( page, &d->page_list )
>>      {
>>          mfn = mfn_x(page_to_mfn(page));
>> -        BUG_ON(SHARED_M2P(get_gpfn_from_mfn(mfn)));
>>          if ( get_gpfn_from_mfn(mfn) >= count )
>>          {
>>              BUG_ON(compat);
>>
> 



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 06:39:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 06:39:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191339.341316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZQc-0002ey-4Y; Tue, 21 Sep 2021 06:38:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191339.341316; Tue, 21 Sep 2021 06:38:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZQc-0002er-0t; Tue, 21 Sep 2021 06:38:54 +0000
Received: by outflank-mailman (input) for mailman id 191339;
 Tue, 21 Sep 2021 06:38:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSZQb-0002el-5U
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 06:38:53 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 95052c30-1aa6-11ec-b89b-12813bfff9fa;
 Tue, 21 Sep 2021 06:38:51 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L6UtLi015352; 
 Tue, 21 Sep 2021 06:38:48 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2059.outbound.protection.outlook.com [104.47.12.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b7a6q00hj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 06:38:48 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6068.eurprd03.prod.outlook.com (2603:10a6:208:166::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 06:38:43 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 06:38:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95052c30-1aa6-11ec-b89b-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NDS3GMioLO+zUJ1AyP1O2jSfAYvcjMwOUNfsNijc/Mh31LpLaqtKbJQ9MDdAMuc9yvkuja2/eyK6kioYRPT0mQRIpfVHFSny31hbVQeQ1t0BZtU9boXHBsWwb8E+inGiwBQEGfjy/mIsyzXt6fGKhlIixTaivwpVv+q0FQZSqWk4ixG/12I6+JpyGgnISFiduvimi2tgm90z/8VWWjma19ZWAXPAPDiOP1nInZqZVKS+1XFf3JOBl8EPOdXQGFMLukEIoYVz8EjsV0Z3EvUBEp2HrdwQGhUmy7bRjVQFic5Zh2CxG0jgfa1bHu/acK9fDXBAmEqBhzvrsVz6IhSy+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=H9hg44KZ9/LdNjWTbRUEVjLA25bXz8mez1mEnkRsLVo=;
 b=aOIPG+o2nN6/J/WOpOeZ7et2ZlFdBHK/CSko2apd8t23CJDevJ7q3TbCDbo04TUWgQmEPcB4aV9ZiSO6g13Qxw3bLEmjCf85S8QftWk5aeyAnLAY6kPwHY0ZcoBjc9bZ4YLdxmql1TSwQANLuMEF8ifjs2WbBzdHPlKI18+p8e7SGh9Bq6QXCUYS9IFzrnzEDul+Dnnr9ORziKFQQQqiaARd1u1OY5998svwL/h22ckfdONTLtQbyLrB8oOhASNKwk75vzBY3LiF68ZbqQNAI7rJZiL/YWAlF1WL4197ZZ79yS9rDzAL+W6FnEIooEYW/XyQedEeZ7nq2nISbU+1wA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=H9hg44KZ9/LdNjWTbRUEVjLA25bXz8mez1mEnkRsLVo=;
 b=558r3exyH9YDLltWG8Lxpf9JSzfzmKUUgBFeLo/RkLgIqXyiqkdsIlmia5QMmS3GJVsZ01vUmVYVqAdy1ddPhlIc272NzJAJsAF9KuShKnVx8WmWBOum/2kzUpRl1YsOIZ2EXQAZ6wvGVY1f2rKEZgyG+MDeSpRTjpgu0yqmV0IWanypVYrX1MjD1MUndo0SsZ95xB3zy7D89qdAXak3r8Ot6K567b9gsvxDAd08uFvWu75QWmuMwk+2P1UF0ni1DlWbr320pA8UqwQmjAZNv4jJmZ+OnDdyYTB8eBvdhyZaPGPjbhjT4NGrNYtiLdsOu0VJ6y+pF09Q9Qu4M3CcPQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAGXhAIAACLuAgAAEUwCAAAi6AA==
Date: Tue, 21 Sep 2021 06:38:43 +0000
Message-ID: <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
In-Reply-To: <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3643d03f-cb97-403c-db8d-08d97cca74ea
x-ms-traffictypediagnostic: AM0PR03MB6068:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB60688533CF4D2A001F90C24AE7A19@AM0PR03MB6068.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 BKtb+/ISESy+yO5pwbmT4SQ7jc2sWYy5Qt4I6jDdgG0tsGhk4Euu3o6XhHe6IxwCqu76uCFyul97WFh0MLCVpf8kWevMxxFKOm+nATTZOL94qC6irvKiutLVqYU2Uu9T/452/AXCVxDaK10F32JfVBGKg3/WMJZR+fA8iEyfhiYpvowHoH+am8oQBMvqbTzftxhqgm3hXSJMWJvIFm9mxCvezu2ZtarZCSEFfxMkt+/nilzeh4wKLNWW6btVbXqSZJY65HIhoCdHqzAriPFpRZ4Wfi70uwyCZP+lxMR5/udciSPcg0Ol2I1/UtlyLKDwnoWRpJGOKxXidt/pBDzm2EXFXOdPtrwaAjzKBqwSIX3vFMIpjSJ1sXO2ukffY9jyVhJLfuylNYkI3rQlTCPOTBP4d23ciilL1+Urj+Ftn5wUqKEXg4XC+5OeO6ONZPbfdxc6retl1SaElX/hLOf2eYgLuHdyjazl4Zc2xFb+vL845HZbXpM7PmDip8NoQEG4ndN+7mWjGNGnuuYFlFqrMZpBOVp9K/iVmCyMVvB0R/GHEL3/1CfDziZwq0ITRBSlDB6EiyHhOd4J7vB4CwnM1CCIdWbAEU4MARDDKQLt9VM//DI99FWJIrGj4dY3smJLffh8jmRFSdckqJqtkdOM1guFoVeeFZGAPrb8aYUI208TyIa2JG9YBvmSCdjyKkpojM79VxQ4i5udx353XkmanHZK7i0uBKNwdxEMfEalWTf009oCm9sbp3QYvWuFMkAG
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(110136005)(55236004)(36756003)(54906003)(4326008)(8936002)(31686004)(83380400001)(8676002)(6486002)(2616005)(26005)(2906002)(316002)(38100700002)(6512007)(122000001)(71200400001)(31696002)(186003)(6506007)(508600001)(66476007)(66556008)(86362001)(53546011)(66446008)(64756008)(38070700005)(91956017)(76116006)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aTVGc1BqVW03WDZSbmR1cUpSUTg5dmsvQ29JdUJBZHJKYWc3REhGbHBMeWpI?=
 =?utf-8?B?dGhwZHkwNnVMRjM1YjZxQy9PVVVWQ3lDQWZzRHRtM25VY1czZnBPS3FMakV3?=
 =?utf-8?B?QTV0S0M1SEtaTUlUUXRFYUZjcE9nRmRSS25tci9rS3l3aU1scEoxY1JWMGRy?=
 =?utf-8?B?ZkRUSjQ2TVdlcDVRcENMWGc5ZHZqQkx2NnRhRllnMnVUa3hndFpVZlplY0po?=
 =?utf-8?B?dFQwcW1Kalp4V1FudndVQWRrTDVpVjZYaHgzeHhnQ0IwQTJObXBvU0VKOHky?=
 =?utf-8?B?MldtVTVQcEVIUnVib0NLbHR6cUVvb3F0Q09FUUxtczdKS1UvMlhpUjE1Znpp?=
 =?utf-8?B?WGY5SDZuSG0zM3FaUk5CcjRoeHAwR1hQbjdFMHIxR2lPcDFaZUlMdGdKNGVm?=
 =?utf-8?B?RVJrYVpXWm44MUM1WXF4UEFheGlLQzRRQWFYb3NqWGZsVW5mQ3JVdk02d1BI?=
 =?utf-8?B?bEdTcjdSVUUvSFRkcVE2YVpUWkcyU0ZJVEpDUWVUcVhqUTEvdXAwVnM1Zm1w?=
 =?utf-8?B?WHlNamF2R1cxaDEwR3hVa2JYdjdDRWFrbzgycko5ZWVvVmpBWVEwZldya3Av?=
 =?utf-8?B?Wm9oblVxdG5MWDNpMGtTbklVUVBsZytrWWFGSHlvVG1jNDZlM2hYTFMzSWNt?=
 =?utf-8?B?bDRZVThyTkdCdTJiZHMrRFpVcHRtVkJ6TmpVbUg5bEhrV3VIWWQ2NW1uTFVt?=
 =?utf-8?B?VXZrUWpCWlBEQUMxUzNKN2NTbEJ4UFBjTkJwUzI3TWR5RFJhRkp6Q2FzZ2c5?=
 =?utf-8?B?eTVERGlYNytvYWhqajdRWlIrT0lSZUFQQUdSdU11S3JnR1dIZHBIN09tT2VI?=
 =?utf-8?B?cDJoUHRLckYzRDlpc0lxbDlrZzhTN250YmF1VzhibUxXZFBTOVZMOGpsS2R6?=
 =?utf-8?B?cnBDWFRld2hKRjZTWDN5WHhpbVFENVNQZFpVbzE2U3VLK0g3MXR4L2tlUHR1?=
 =?utf-8?B?RGFXd2VLelR1am9xQ1pvcGJ1aFo5aHQ2czd2dExXWGNOc1hqTFJuV1FhNDUv?=
 =?utf-8?B?bGFLbUhpUmpBOGk5VkNSUmpmM3RVN3pPeVB2ZFUyTlZkNjM2cTRnSURKeS9K?=
 =?utf-8?B?QUIyQWhwdjNLY0VxdlRLT1dxcHM4aHlEWkl6YUxWSWRQeWdlblBiNjM4VkI5?=
 =?utf-8?B?R2RMRko0Wnh6K0s2YkdpUU0rSVUyc0xyRVp5cFIzZEZwK3hYNk0vQ2ZGTUFr?=
 =?utf-8?B?Z1pyN0JrUjZsdWFzOEtkdGY2SGV4NEoxZE1ieERUQTIvWklQWGNBUmNrbkJr?=
 =?utf-8?B?S2d0YVFQaXpFQk56OGMxQUhQMkFla2RnQmtpRmtYb3oyL2lrZ0NnRDRqNEtB?=
 =?utf-8?B?L0lFOXNoRkNCckRQV3ljLy84bGU2eEpERXdBQ0ZRNjE4T2p1cVN5VXk0L2xs?=
 =?utf-8?B?eEJSQ2tuTitwdENzbW9zYWdjWTl4ZEdUWU9laVhTSHkzck92bkhkQ09xazhU?=
 =?utf-8?B?VHJaR2FaejZ1d3lGNTdWSE1jVFh1M2N3amc4WklkcDJ2d2NJa0IxSzIvM08r?=
 =?utf-8?B?Y1BoeHNUOWNLdlU5cmtjZ09YUzUvcGRSRVlpTEpzTFUvc3RETFJYL2w4Ymh0?=
 =?utf-8?B?eWM0T3hSVnBlTWlPTWRiNHZ0WWQ3TEd5QmtINU5kYUQ4ODgrZmNZUDNIMTNW?=
 =?utf-8?B?Z2xDZjl5V09WL0xJVTl3NVluRERXVG5xZmd2TDVVZGs5MFVwVncvUk9OYUFP?=
 =?utf-8?B?eXk1a01PQ3N4T1dmSGlRQjdSYzZSdXo5b3BlWExhRmFReUljRENBSWlnWUNH?=
 =?utf-8?B?QnVDOVp6OXdIL0VKRDZCb1JwaklLOU5MNFhkVjNoZUNkdkVKYkNZZ05Qa2xH?=
 =?utf-8?B?eUZ2eVluaXhGK3JGZUQ1dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <261C2EFB4966614881012A5A765B0E1C@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3643d03f-cb97-403c-db8d-08d97cca74ea
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 06:38:43.4839
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: QGoCpTPjSwlDZVBp6ucaebnAtL2Dfu/Ia9MDXP/E1h05bAbUcNsjePb/R8XGYKkX0pfCG0qDh5DWYwqCCeXFDgCEZAzc2YOQnKzJE+Wsiqu5dDEBTGGkRSfInhZjzryL
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6068
X-Proofpoint-GUID: _eH4U1tvwiQHtehXSWDaWBrx5dIPMQbV
X-Proofpoint-ORIG-GUID: _eH4U1tvwiQHtehXSWDaWBrx5dIPMQbV
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 malwarescore=0 mlxscore=0 spamscore=0 impostorscore=0 mlxlogscore=999
 bulkscore=0 suspectscore=0 clxscore=1015 adultscore=0 phishscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210042

DQpPbiAyMS4wOS4yMSAwOTowNywgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjEuMDkuMjEg
MDc6NTEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDIxLjA5LjIx
IDA4OjIwLCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDIxLjA5LjIxIDAxOjE2LCBTdGVm
YW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4+IE9uIE1vbiwgMjAgU2VwIDIwMjEsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+Pj4gT24gMjAuMDkuMjEgMTQ6MzAsIEp1ZXJnZW4g
R3Jvc3Mgd3JvdGU6DQo+Pj4+Pj4gT24gMjAuMDkuMjEgMDc6MjMsIE9sZWtzYW5kciBBbmRydXNo
Y2hlbmtvIHdyb3RlOg0KPj4+Pj4+PiBIZWxsbywgU3RlZmFubyENCj4+Pj4+Pj4NCj4+Pj4+Pj4g
T24gMTguMDkuMjEgMDA6NDUsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4+Pj4+Pj4+IEhp
IE9sZWtzYW5kciwNCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiBXaHkgZG8geW91IHdhbnQgdG8gZW5hYmxl
IHBjaWJhY2sgb24gQVJNPyBJcyBpdCBvbmx5IHRvICJkaXNhYmxlIiBhIFBDSQ0KPj4+Pj4+Pj4g
ZGV2aWNlIGluIERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRvIGEgRG9t
VT8NCj4+Pj4+Pj4gTm90IG9ubHkgdGhhdA0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IEkgYW0gYXNraW5n
IGJlY2F1c2UgYWN0dWFsbHkgSSBkb24ndCB0aGluayB3ZSB3YW50IHRvIGVuYWJsZSB0aGUgUFYg
UENJDQo+Pj4+Pj4+PiBiYWNrZW5kIGZlYXR1cmUgb2YgcGNpYmFjayBvbiBBUk0sIHJpZ2h0PyBU
aGF0IHdvdWxkIGNsYXNoIHdpdGggdGhlIFBDSQ0KPj4+Pj4+Pj4gYXNzaWdubWVudCB3b3JrIHlv
dSBoYXZlIGJlZW4gZG9pbmcgaW4gWGVuLiBUaGV5IGNvdWxkbid0IGJvdGggd29yayBhdA0KPj4+
Pj4+Pj4gdGhlIHNhbWUgdGltZS4NCj4+Pj4+Pj4gQ29ycmVjdCwgaXQgaXMgbm90IHVzZWQNCj4+
Pj4+Pj4+DQo+Pj4+Pj4+PiBJZiB3ZSBvbmx5IG5lZWQgcGNpYmFjayB0byAicGFyayIgYSBkZXZp
Y2UgaW4gRG9tMCwgd291bGRuJ3QgaXQgYmUNCj4+Pj4+Pj4+IHBvc3NpYmxlIGFuZCBiZXR0ZXIg
dG8gdXNlIHBjaS1zdHViIGluc3RlYWQ/DQo+Pj4+Pj4+DQo+Pj4+Pj4+IE5vdCBvbmx5IHRoYXQs
IHNvIHBjaS1zdHViIGlzIG5vdCBlbm91Z2gNCj4+Pj4+Pj4NCj4+Pj4+Pj4gVGhlIGZ1bmN0aW9u
YWxpdHkgd2hpY2ggaXMgaW1wbGVtZW50ZWQgYnkgdGhlIHBjaWJhY2sgYW5kIHRoZSB0b29sc3Rh
Y2sNCj4+Pj4+Pj4gYW5kIHdoaWNoIGlzIHJlbGV2YW50L21pc3NpbmcvbmVlZGVkIGZvciBBUk06
DQo+Pj4+Pj4+DQo+Pj4+Pj4+IDEuIHBjaWJhY2sgaXMgdXNlZCBhcyBhIGRhdGFiYXNlIGZvciBh
c3NpZ25hYmxlIFBDSSBkZXZpY2VzLCBlLmcuIHhsDQo+Pj4+Pj4+IMKgwqDCoCDCoMKgIHBjaS1h
c3NpZ25hYmxlLXthZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQgbGlzdC4gU28sIHdo
ZW5ldmVyIHRoZQ0KPj4+Pj4+PiDCoMKgwqAgwqDCoCB0b29sc3RhY2sgbmVlZHMgdG8ga25vdyB3
aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2ggaXQgcmVhZHMNCj4+Pj4+Pj4g
wqDCoMKgIMKgwqAgdGhhdCBmcm9tIHRoZSByZWxldmFudCBzeXNmcyBlbnRyaWVzIG9mIHRoZSBw
Y2liYWNrLg0KPj4+Pj4+Pg0KPj4+Pj4+PiAyLiBwY2liYWNrIGlzIHVzZWQgdG8gaG9sZCB0aGUg
dW5ib3VuZCBQQ0kgZGV2aWNlcywgZS5nLiB3aGVuIHBhc3NpbmcgdGhyb3VnaA0KPj4+Pj4+PiDC
oMKgwqAgwqDCoCBhIFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSBy
ZWxldmFudCBkZXZpY2UgZHJpdmVyIGFuZCBib3VuZA0KPj4+Pj4+PiDCoMKgwqAgwqDCoCB0byBw
Y2liYWNrIChzdHJpY3RseSBzcGVha2luZyBpdCBpcyBub3QgcmVxdWlyZWQgdGhhdCB0aGUgZGV2
aWNlIGlzIGJvdW5kIHRvDQo+Pj4+Pj4+IMKgwqDCoCDCoMKgIHBjaWJhY2ssIGJ1dCBwY2liYWNr
IGlzIGFnYWluIHVzZWQgYXMgYSBkYXRhYmFzZSBvZiB0aGUgcGFzc2VkIHRocm91Z2ggUENJDQo+
Pj4+Pj4+IMKgwqDCoCDCoMKgIGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5kIHRoZSBkZXZpY2Vz
IGJhY2sgdG8gdGhlaXIgb3JpZ2luYWwgZHJpdmVycyB3aGVuDQo+Pj4+Pj4+IMKgwqDCoCDCoMKg
IGd1ZXN0IGRvbWFpbiBzaHV0cyBkb3duKQ0KPj4+Pj4+Pg0KPj4+Pj4+PiAzLiBEZXZpY2UgcmVz
ZXQNCj4+Pj4+Pj4NCj4+Pj4+Pj4gV2UgaGF2ZSBwcmV2aW91c2x5IGRpc2N1c3NlZCBvbiB4ZW4t
ZGV2ZWwgTUwgcG9zc2libGUgc29sdXRpb25zIHRvIHRoYXQgYXMgZnJvbSB0aGUNCj4+Pj4+Pj4g
YWJvdmUgd2Ugc2VlIHRoYXQgcGNpYmFjayBmdW5jdGlvbmFsaXR5IGlzIGdvaW5nIHRvIGJlIG9u
bHkgcGFydGlhbGx5IHVzZWQgb24gQXJtLg0KPj4+Pj4+Pg0KPj4+Pj4+PiBQbGVhc2Ugc2VlIFsx
XSBhbmQgWzJdOg0KPj4+Pj4+Pg0KPj4+Pj4+PiAxLiBJdCBpcyBub3QgYWNjZXB0YWJsZSB0byBt
YW5hZ2UgdGhlIGFzc2lnbmFibGUgbGlzdCBpbiBYZW4gaXRzZWxmDQo+Pj4+Pj4+DQo+Pj4+Pj4+
IDIuIHBjaWJhY2sgY2FuIGJlIHNwbGl0IGludG8gdHdvIHBhcnRzOiBQQ0kgYXNzaWduYWJsZS9i
aW5kL3Jlc2V0IGhhbmRsaW5nIGFuZA0KPj4+Pj4+PiB0aGUgcmVzdCBsaWtlIHZQQ0kgZXRjLg0K
Pj4+Pj4+Pg0KPj4+Pj4+PiAzLiBwY2lmcm9udCBpcyBub3QgdXNlZCBvbiBBcm0NCj4+Pj4+Pg0K
Pj4+Pj4+IEl0IGlzIG5laXRoZXIgaW4geDg2IFBWSC9IVk0gZ3Vlc3RzLg0KPj4+Pj4gRGlkbid0
IGtub3cgdGhhdCwgdGhhbmsgeW91IGZvciBwb2ludGluZw0KPj4+Pj4+DQo+Pj4+Pj4+IFNvLCBs
aW1pdGVkIHVzZSBvZiB0aGUgcGNpYmFjayBpcyBvbmUgb2YgdGhlIGJyaWNrcyB1c2VkIHRvIGVu
YWJsZSBQQ0kgcGFzc3Rocm91Z2gNCj4+Pj4+Pj4gb24gQXJtLiBJdCB3YXMgZW5vdWdoIHRvIGp1
c3QgcmUtc3RydWN0dXJlIHRoZSBkcml2ZXIgYW5kIGhhdmUgaXQgcnVuIG9uIEFybSB0byBhY2hp
ZXZlDQo+Pj4+Pj4+IGFsbCB0aGUgZ29hbHMgYWJvdmUuDQo+Pj4+Pj4+DQo+Pj4+Pj4+IElmIHdl
IHN0aWxsIHRoaW5rIGl0IGlzIGRlc2lyYWJsZSB0byBicmVhayB0aGUgcGNpYmFjayBkcml2ZXIg
aW50byAiY29tbW9uIiBhbmQgInBjaWZyb250IHNwZWNpZmljIg0KPj4+Pj4+PiBwYXJ0cyB0aGVu
IGl0IGNhbiBiZSBkb25lLCB5ZXQgdGhlIHBhdGNoIGlzIGdvaW5nIHRvIGJlIHRoZSB2ZXJ5IGZp
cnN0IGJyaWNrIGluIHRoYXQgYnVpbGRpbmcuDQo+Pj4+Pj4NCj4+Pj4+PiBEb2luZyB0aGlzIHNw
bGl0IHNob3VsZCBiZSBkb25lLCBhcyB0aGUgcGNpZnJvbnQgc3BlY2lmaWMgcGFydCBjb3VsZCBi
ZQ0KPj4+Pj4+IG9taXR0ZWQgb24geDg2LCB0b28sIGluIGNhc2Ugbm8gUFYgZ3Vlc3RzIHVzaW5n
IFBDSSBwYXNzdGhyb3VnaCBoYXZlIHRvDQo+Pj4+Pj4gYmUgc3VwcG9ydGVkLg0KPj4+Pj4gQWdy
ZWUsIHRoYXQgdGhlIGZpbmFsIHNvbHV0aW9uIHNob3VsZCBoYXZlIHRoZSBkcml2ZXIgc3BsaXQN
Cj4+Pj4+Pg0KPj4+Pj4+PiBTbywgSSB0aGluayB0aGlzIHBhdGNoIGlzIHN0aWxsIGdvaW5nIHRv
IGJlIG5lZWRlZCBiZXNpZGVzIHdoaWNoIGRpcmVjdGlvbiB3ZSB0YWtlLg0KPj4+Pj4+DQo+Pj4+
Pj4gU29tZSBraW5kIG9mIHRoaXMgcGF0Y2gsIHllcy4gSXQgbWlnaHQgbG9vayBkaWZmZXJlbnQg
aW4gY2FzZSB0aGUgc3BsaXQNCj4+Pj4+PiBpcyBkb25lIGZpcnN0Lg0KPj4+Pj4+DQo+Pj4+Pj4g
SSBkb24ndCBtaW5kIGRvaW5nIGl0IGluIGVpdGhlciBzZXF1ZW5jZS4NCj4+Pj4+Pg0KPj4+Pj4g
V2l0aCB0aGlzIHBhdGNoIHdlIGhhdmUgQXJtIG9uIHRoZSBzYW1lIHBhZ2UgYXMgdGhlIGFib3Zl
IG1lbnRpb25lZCB4ODYgZ3Vlc3RzLA0KPj4+Pj4NCj4+Pj4+IGUuZy4gdGhlIGRyaXZlciBoYXMg
dW51c2VkIGNvZGUsIGJ1dCB5ZXQgYWxsb3dzIEFybSB0byBmdW5jdGlvbiBub3cuDQo+Pj4+Pg0K
Pj4+Pj4gQXQgdGhpcyBzdGFnZSBvZiBQQ0kgcGFzc3Rocm91Z2ggb24gQXJtIGl0IGlzIHlldCBl
bm91Z2guIExvbmcgdGVybSwgd2hlbg0KPj4+Pj4NCj4+Pj4+IHRoZSBkcml2ZXIgZ2V0cyBzcGxp
dCwgQXJtIHdpbGwgYmVuZWZpdCBmcm9tIHRoYXQgc3BsaXQgdG9vLCBidXQgdW5mb3J0dW5hdGVs
eSBJIGRvIG5vdA0KPj4+Pj4NCj4+Pj4+IGhhdmUgZW5vdWdoIGJhbmR3aWR0aCBmb3IgdGhhdCBw
aWVjZSBvZiB3b3JrIGF0IHRoZSBtb21lbnQuDQo+Pj4+DQo+Pj4+IFRoYXQncyBmYWlyIGFuZCBJ
IGRvbid0IHdhbnQgdG8gc2NvcGUtY3JlZXAgdGhpcyBzaW1wbGUgcGF0Y2ggYXNraW5nIGZvcg0K
Pj4+PiBhbiBlbm9ybW91cyByZXdvcmsuIEF0IHRoZSBzYW1lIHRpbWUgSSBkb24ndCB0aGluayB3
ZSBzaG91bGQgZW5hYmxlIHRoZQ0KPj4+PiB3aG9sZSBvZiBwY2liYWNrIG9uIEFSTSBiZWNhdXNl
IGl0IHdvdWxkIGJlIGVycm9uZW91cyBhbmQgY29uZnVzaW5nLg0KPj4NCj4+IEFzIHRoZSBmaXJz
dCBzdGFnZSBiZWZvcmUgdGhlIGRyaXZlciBpcyBzcGxpdCBvciBpZmRlZidzIHVzZWQgLSBjYW4g
d2UgdGFrZSB0aGUgcGF0Y2gNCj4+IGFzIGlzIG5vdz8gSW4gZWl0aGVyIHdheSB3ZSBjaG9zZSB0
aGlzIG5lZWRzIHRvIGJlIGRvbmUsIGUuZy4gZW5hYmxlIGNvbXBpbGluZw0KPj4gZm9yIG90aGVy
IGFyY2hpdGVjdHVyZXMgYW5kIGNvbW1vbiBjb2RlIG1vdmUuDQo+DQo+IEZpbmUgd2l0aCBtZSBp
biBwcmluY2lwbGUuIEkgbmVlZCB0byB0YWtlIGEgbW9yZSB0aG9yb3VnaCBsb29rDQo+IGF0IHRo
ZSBwYXRjaCwgdGhvdWdoLg0KT2YgY291cnNlDQo+DQo+Pg0KPj4+Pg0KPj4+PiBJIGFtIHdvbmRl
ciBpZiB0aGVyZSBpcyBhIHNpbXBsZToNCj4+Pj4NCj4+Pj4gaWYgKCF4ZW5fcHZfZG9tYWluKCkp
DQo+Pj4+IMKgwqDCoMKgwqAgcmV0dXJuOw0KPj4+Pg0KPj4+PiBUaGF0IHdlIGNvdWxkIGFkZCBp
biBhIGNvdXBsZSBvZiBwbGFjZXMgaW4gcGNpYmFjayB0byBzdG9wIGl0IGZyb20NCj4+Pj4gaW5p
dGlhbGl6aW5nIHRoZSBwYXJ0cyB3ZSBkb24ndCBjYXJlIGFib3V0LiBTb21ldGhpbmcgYWxvbmcg
dGhlc2UgbGluZXMNCj4+Pj4gKHVudGVzdGVkIGFuZCBwcm9iYWJseSBpbmNvbXBsZXRlKS4NCj4+
Pj4NCj4+Pj4gV2hhdCBkbyB5b3UgZ3V5cyB0aGluaz8NCj4+Pg0KPj4+IFVoIG5vLCBub3QgaW4g
dGhpcyB3YXksIHBsZWFzZS4gVGhpcyB3aWxsIGtpbGwgcGNpIHBhc3N0aHJvdWdoIG9uIHg4Ng0K
Pj4+IHdpdGggZG9tMCBydW5uaW5nIGFzIFBWSC4gSSBkb24ndCB0aGluayB0aGlzIGlzIHdvcmtp
bmcgcmlnaHQgbm93LCBidXQNCj4+PiBhZGRpbmcgbW9yZSBjb2RlIG1ha2luZyBpdCBldmVuIGhh
cmRlciB0byB3b3JrIHNob3VsZCBiZSBhdm9pZGVkLg0KPj4+DQo+Pj4+IGRpZmYgLS1naXQgYS9k
cml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNr
L3hlbmJ1cy5jDQo+Pj4+IGluZGV4IGRhMzRjZTg1ZGM4OC4uOTkxYmEwYTliMzU5IDEwMDY0NA0K
Pj4+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4+PiArKysgYi9k
cml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4+PiBAQCAtMTUsNiArMTUsNyBAQA0K
Pj4+PiDCoMKgICNpbmNsdWRlIDx4ZW4veGVuYnVzLmg+DQo+Pj4+IMKgwqAgI2luY2x1ZGUgPHhl
bi9ldmVudHMuaD4NCj4+Pj4gwqDCoCAjaW5jbHVkZSA8eGVuL3BjaS5oPg0KPj4+PiArI2luY2x1
ZGUgPHhlbi94ZW4uaD4NCj4+Pj4gwqDCoCAjaW5jbHVkZSAicGNpYmFjay5oIg0KPj4+PiDCoMKg
IMKgICNkZWZpbmUgSU5WQUxJRF9FVlRDSE5fSVJRwqAgKC0xKQ0KPj4+PiBAQCAtNjg1LDggKzY4
NiwxMiBAQCBzdGF0aWMgaW50IHhlbl9wY2lia194ZW5idXNfcHJvYmUoc3RydWN0IHhlbmJ1c19k
ZXZpY2UgKmRldiwNCj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNv
bnN0IHN0cnVjdCB4ZW5idXNfZGV2aWNlX2lkICppZCkNCj4+Pj4gwqDCoCB7DQo+Pj4+IMKgwqDC
oMKgwqDCoCBpbnQgZXJyID0gMDsNCj4+Pj4gLcKgwqDCoCBzdHJ1Y3QgeGVuX3BjaWJrX2Rldmlj
ZSAqcGRldiA9IGFsbG9jX3BkZXYoZGV2KTsNCj4+Pj4gK8KgwqDCoCBzdHJ1Y3QgeGVuX3BjaWJr
X2RldmljZSAqcGRldjsNCj4+Pj4gKw0KPj4+PiArwqDCoMKgIGlmICgheGVuX3B2X2RvbWFpbigp
KQ0KPj4+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDA7DQo+Pj4+IMKgwqAgK8KgwqDCoCBwZGV2
ID0gYWxsb2NfcGRldihkZXYpOw0KPj4+DQo+Pj4gVGhpcyBodW5rIGlzbid0IG5lZWRlZCwgYXMg
d2l0aCBiYWlsaW5nIG91dCBvZiB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyDQo+Pj4gZWFybHkg
d2lsbCByZXN1bHQgaW4geGVuX3BjaWJrX3hlbmJ1c19wcm9iZSBuZXZlciBiZWluZyBjYWxsZWQu
DQo+Pj4NCj4+Pj4gwqDCoMKgwqDCoMKgIGlmIChwZGV2ID09IE5VTEwpIHsNCj4+Pj4gwqDCoMKg
wqDCoMKgwqDCoMKgwqAgZXJyID0gLUVOT01FTTsNCj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAg
eGVuYnVzX2Rldl9mYXRhbChkZXYsIGVyciwNCj4+Pj4gQEAgLTc0Myw2ICs3NDgsOSBAQCBjb25z
dCBzdHJ1Y3QgeGVuX3BjaWJrX2JhY2tlbmQgKl9fcmVhZF9tb3N0bHkgeGVuX3BjaWJrX2JhY2tl
bmQ7DQo+Pj4+IMKgwqAgwqAgaW50IF9faW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZv
aWQpDQo+Pj4+IMKgwqAgew0KPj4+PiArwqDCoMKgIGlmICgheGVuX3B2X2RvbWFpbigpKQ0KPj4+
PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDA7DQo+Pj4+ICsNCj4+Pg0KPj4+IFVzZSAjaWZkZWYg
Q09ORklHX1g4NiBpbnN0ZWFkLg0KPj4NCj4+IFRoZSB0aXRsZSBvZiB0aGlzIHBhdGNoIHNheXMg
dGhhdCB3ZSB3YW50IHRvIGFsbG93IHRoaXMgZHJpdmVyIGZvciBvdGhlciBhcmNocw0KPj4gYW5k
IG5vdyB3ZSB3YW50IHRvIGludHJvZHVjZSAiI2lmZGVmIENPTkZJR19YODYiIHdoaWNoIGRvZXNu
J3Qgc291bmQNCj4+IHJpZ2h0IHdpdGggdGhhdCByZXNwZWN0LiBJbnN0ZWFkLCB3ZSBtYXkgd2Fu
dCBoYXZpbmcgc29tZXRoaW5nIGxpa2UgYQ0KPj4gZGVkaWNhdGVkIGdhdGUgZm9yIHRoaXMsIGUu
Zy4gIiNpZmRlZiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EX1NVUFBfUFYiDQo+PiBvciBzb21l
dGhpbmcgd2hpY2ggaXMgYXJjaGl0ZWN0dXJlIGFnbm9zdGljLg0KPg0KPiBTb21ldGhpbmcgbGlr
ZSB0aGF0LCB5ZXMuIEJ1dCBJJ2QgcmF0aGVyIHVzZSBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5E
DQo+IGFjdGluZyBhcyB0aGlzIGdhdGUgYW5kIGludHJvZHVjZSBDT05GSUdfWEVOX1BDSV9TVFVC
IGZvciB0aGUgc3R1Yg0KPiBmdW5jdGlvbmFsaXR5IG5lZWRlZCBvbiBBcm0uIFhFTl9QQ0lERVZf
QkFDS0VORCB3b3VsZCBkZXBlbmQgb24gWDg2IGFuZA0KPiBzZWxlY3QgWEVOX1BDSV9TVFVCLCB3
aGlsZSBvbiBBcm0gWEVOX1BDSV9TVFVCIGNvdWxkIGJlIGNvbmZpZ3VyZWQgaWYNCj4gd2FudGVk
LiBUaGUgc3BsaXR0aW5nIG9mIHRoZSBkcml2ZXIgY2FuIHN0aWxsIGJlIGRvbmUgbGF0ZXIuDQoN
CkhtLCBwY2liYWNrIGlzIG5vdyBjb21waWxlZCB3aGVuIENPTkZJR19YRU5fUENJREVWX0JBQ0tF
TkTCoCBpcyBlbmFibGVkDQoNCmFuZCB3ZSB3YW50IHRvIHNraXAgc29tZSBwYXJ0cyBvZiBpdHMg
Y29kZSB3aGVuIENPTkZJR19YRU5fUENJX1NUVUIgaXMgc2V0Lg0KDQpTbywgSSBpbWFnaW5lIHRo
YXQgZm9yIHg4NiB3ZSBqdXN0IGVuYWJsZSBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EIGFuZCB0
aGUNCg0KZHJpdmVyIGNvbXBpbGVzIGluIGl0cyBjdXJyZW50IHN0YXRlLiBGb3IgQXJtIHdlIGVu
YWJsZSBib3RoIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQNCg0KYW5kIENPTkZJR19YRU5fUENJ
X1NUVUIsIHNvIHBhcnQgb2YgdGhlIGRyaXZlciBpcyBub3QgY29tcGlsZWQuDQoNCg0KPg0KPj4g
R2F0aW5nIGFsc28gbWVhbnMgdGhhdCB3ZSBhcmUgbm90IHRoaW5raW5nIGFib3V0IHNwbGl0dGlu
ZyB0aGUgYmFja2VuZCBkcml2ZXIgaW50bw0KPj4gdHdvIGRpZmZlcmVudCBvbmVzLCBlLmcuIG9u
ZSBmb3IgImNvbW1vbiIgY29kZSBhbmQgb25lIGZvciBQViBzdHVmZi4NCj4+IE90aGVyd2lzZSB0
aGlzIGlmZGVmZXJ5IHdvbid0IGJlIG5lZWRlZC4NCj4NCj4gSSBqdXN0IHdhbnRlZCB0byBhdm9p
ZCB0aGUgeGVuX3B2X2RvbWFpbigpIHRlc3RzIGNyZWVwaW5nIGluLCBhcw0KPiB0aGV5IGFyZSB3
cm9uZyBJTU8uDQo+DQpJIHVuZGVyc3RhbmQgdGhhdA0KPg0KPiBKdWVyZ2Vu


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 06:49:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 06:49:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191350.341327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZb9-0004JT-AI; Tue, 21 Sep 2021 06:49:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191350.341327; Tue, 21 Sep 2021 06:49:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZb9-0004JM-7L; Tue, 21 Sep 2021 06:49:47 +0000
Received: by outflank-mailman (input) for mailman id 191350;
 Tue, 21 Sep 2021 06:49:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSZb7-0004JG-CO
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 06:49:45 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 19d47e10-1aa8-11ec-b89c-12813bfff9fa;
 Tue, 21 Sep 2021 06:49:43 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id CF79E220BD;
 Tue, 21 Sep 2021 06:49:42 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8AE2713B97;
 Tue, 21 Sep 2021 06:49:42 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id YgKpIIaASWEtdQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 06:49:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19d47e10-1aa8-11ec-b89c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632206982; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=RS230kpcC4MLuTfJwKjFSNbCj73nsDuLNPRcJtOkj1s=;
	b=ht89Qr5QkFKFdGNFcKq7RI9aQZEwPm58xQOdqr3vDgKAwVGLJdsVxMWzlwuDGJ86dwE6rv
	bq7BC6c3w+TeaisqEkouZqoYITm2Mlx7LdQ+3Z1Q10KVCUoYBuz35C/0ClJaHRgTiU5rgs
	HzilKAcRSLdAkCwRTTWmO7D/5LBge7k=
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
Date: Tue, 21 Sep 2021 08:49:42 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="wsuxArFoeRkbtGgBU1fF54AKz0F7tz7qQ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--wsuxArFoeRkbtGgBU1fF54AKz0F7tz7qQ
Content-Type: multipart/mixed; boundary="XacVIrZmUkb7pecLSR0D6hRuZnByoOX7A";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
In-Reply-To: <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>

--XacVIrZmUkb7pecLSR0D6hRuZnByoOX7A
Content-Type: multipart/mixed;
 boundary="------------CD70FE0AF5754356184F6F62"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------CD70FE0AF5754356184F6F62
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 08:38, Oleksandr Andrushchenko wrote:
>=20
> On 21.09.21 09:07, Juergen Gross wrote:
>> On 21.09.21 07:51, Oleksandr Andrushchenko wrote:
>>>
>>> On 21.09.21 08:20, Juergen Gross wrote:
>>>> On 21.09.21 01:16, Stefano Stabellini wrote:
>>>>> On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
>>>>>> On 20.09.21 14:30, Juergen Gross wrote:
>>>>>>> On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
>>>>>>>> Hello, Stefano!
>>>>>>>>
>>>>>>>> On 18.09.21 00:45, Stefano Stabellini wrote:
>>>>>>>>> Hi Oleksandr,
>>>>>>>>>
>>>>>>>>> Why do you want to enable pciback on ARM? Is it only to "disabl=
e" a PCI
>>>>>>>>> device in Dom0 so that it can be safely assigned to a DomU?
>>>>>>>> Not only that
>>>>>>>>>
>>>>>>>>> I am asking because actually I don't think we want to enable th=
e PV PCI
>>>>>>>>> backend feature of pciback on ARM, right? That would clash with=
 the PCI
>>>>>>>>> assignment work you have been doing in Xen. They couldn't both =
work at
>>>>>>>>> the same time.
>>>>>>>> Correct, it is not used
>>>>>>>>>
>>>>>>>>> If we only need pciback to "park" a device in Dom0, wouldn't it=
 be
>>>>>>>>> possible and better to use pci-stub instead?
>>>>>>>>
>>>>>>>> Not only that, so pci-stub is not enough
>>>>>>>>
>>>>>>>> The functionality which is implemented by the pciback and the to=
olstack
>>>>>>>> and which is relevant/missing/needed for ARM:
>>>>>>>>
>>>>>>>> 1. pciback is used as a database for assignable PCI devices, e.g=
=2E xl
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 pci-assignable-{add|remove|list=
} manipulates that list. So, whenever the
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 toolstack needs to know which P=
CI devices can be passed through it reads
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 that from the relevant sysfs en=
tries of the pciback.
>>>>>>>>
>>>>>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when pa=
ssing through
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 a PCI device it needs to be unb=
ound from the relevant device driver and bound
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 to pciback (strictly speaking i=
t is not required that the device is bound to
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 pciback, but pciback is again u=
sed as a database of the passed through PCI
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 devices, so we can re-bind the =
devices back to their original drivers when
>>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 guest domain shuts down)
>>>>>>>>
>>>>>>>> 3. Device reset
>>>>>>>>
>>>>>>>> We have previously discussed on xen-devel ML possible solutions =
to that as from the
>>>>>>>> above we see that pciback functionality is going to be only part=
ially used on Arm.
>>>>>>>>
>>>>>>>> Please see [1] and [2]:
>>>>>>>>
>>>>>>>> 1. It is not acceptable to manage the assignable list in Xen its=
elf
>>>>>>>>
>>>>>>>> 2. pciback can be split into two parts: PCI assignable/bind/rese=
t handling and
>>>>>>>> the rest like vPCI etc.
>>>>>>>>
>>>>>>>> 3. pcifront is not used on Arm
>>>>>>>
>>>>>>> It is neither in x86 PVH/HVM guests.
>>>>>> Didn't know that, thank you for pointing
>>>>>>>
>>>>>>>> So, limited use of the pciback is one of the bricks used to enab=
le PCI passthrough
>>>>>>>> on Arm. It was enough to just re-structure the driver and have i=
t run on Arm to achieve
>>>>>>>> all the goals above.
>>>>>>>>
>>>>>>>> If we still think it is desirable to break the pciback driver in=
to "common" and "pcifront specific"
>>>>>>>> parts then it can be done, yet the patch is going to be the very=
 first brick in that building.
>>>>>>>
>>>>>>> Doing this split should be done, as the pcifront specific part co=
uld be
>>>>>>> omitted on x86, too, in case no PV guests using PCI passthrough h=
ave to
>>>>>>> be supported.
>>>>>> Agree, that the final solution should have the driver split
>>>>>>>
>>>>>>>> So, I think this patch is still going to be needed besides which=
 direction we take.
>>>>>>>
>>>>>>> Some kind of this patch, yes. It might look different in case the=
 split
>>>>>>> is done first.
>>>>>>>
>>>>>>> I don't mind doing it in either sequence.
>>>>>>>
>>>>>> With this patch we have Arm on the same page as the above mentione=
d x86 guests,
>>>>>>
>>>>>> e.g. the driver has unused code, but yet allows Arm to function no=
w.
>>>>>>
>>>>>> At this stage of PCI passthrough on Arm it is yet enough. Long ter=
m, when
>>>>>>
>>>>>> the driver gets split, Arm will benefit from that split too, but u=
nfortunately I do not
>>>>>>
>>>>>> have enough bandwidth for that piece of work at the moment.
>>>>>
>>>>> That's fair and I don't want to scope-creep this simple patch askin=
g for
>>>>> an enormous rework. At the same time I don't think we should enable=
 the
>>>>> whole of pciback on ARM because it would be erroneous and confusing=
=2E
>>>
>>> As the first stage before the driver is split or ifdef's used - can w=
e take the patch
>>> as is now? In either way we chose this needs to be done, e.g. enable =
compiling
>>> for other architectures and common code move.
>>
>> Fine with me in principle. I need to take a more thorough look
>> at the patch, though.
> Of course
>>
>>>
>>>>>
>>>>> I am wonder if there is a simple:
>>>>>
>>>>> if (!xen_pv_domain())
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return;
>>>>>
>>>>> That we could add in a couple of places in pciback to stop it from
>>>>> initializing the parts we don't care about. Something along these l=
ines
>>>>> (untested and probably incomplete).
>>>>>
>>>>> What do you guys think?
>>>>
>>>> Uh no, not in this way, please. This will kill pci passthrough on x8=
6
>>>> with dom0 running as PVH. I don't think this is working right now, b=
ut
>>>> adding more code making it even harder to work should be avoided.
>>>>
>>>>> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pci=
back/xenbus.c
>>>>> index da34ce85dc88..991ba0a9b359 100644
>>>>> --- a/drivers/xen/xen-pciback/xenbus.c
>>>>> +++ b/drivers/xen/xen-pciback/xenbus.c
>>>>> @@ -15,6 +15,7 @@
>>>>>  =C2=A0=C2=A0 #include <xen/xenbus.h>
>>>>>  =C2=A0=C2=A0 #include <xen/events.h>
>>>>>  =C2=A0=C2=A0 #include <xen/pci.h>
>>>>> +#include <xen/xen.h>
>>>>>  =C2=A0=C2=A0 #include "pciback.h"
>>>>>  =C2=A0=C2=A0 =C2=A0 #define INVALID_EVTCHN_IRQ=C2=A0 (-1)
>>>>> @@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xenbu=
s_device *dev,
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const struct xenbus_device_id =
*id)
>>>>>  =C2=A0=C2=A0 {
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int err =3D 0;
>>>>> -=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev =3D alloc_pdev(de=
v);
>>>>> +=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev;
>>>>> +
>>>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>>>>  =C2=A0=C2=A0 +=C2=A0=C2=A0=C2=A0 pdev =3D alloc_pdev(dev);
>>>>
>>>> This hunk isn't needed, as with bailing out of xen_pcibk_xenbus_regi=
ster
>>>> early will result in xen_pcibk_xenbus_probe never being called.
>>>>
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (pdev =3D=3D NULL) {
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 err =3D=
 -ENOMEM;
>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xenbu=
s_dev_fatal(dev, err,
>>>>> @@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly x=
en_pcibk_backend;
>>>>>  =C2=A0=C2=A0 =C2=A0 int __init xen_pcibk_xenbus_register(void)
>>>>>  =C2=A0=C2=A0 {
>>>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>>>> +
>>>>
>>>> Use #ifdef CONFIG_X86 instead.
>>>
>>> The title of this patch says that we want to allow this driver for ot=
her archs
>>> and now we want to introduce "#ifdef CONFIG_X86" which doesn't sound
>>> right with that respect. Instead, we may want having something like a=

>>> dedicated gate for this, e.g. "#ifdef CONFIG_XEN_PCIDEV_BACKEND_SUPP_=
PV"
>>> or something which is architecture agnostic.
>>
>> Something like that, yes. But I'd rather use CONFIG_XEN_PCIDEV_BACKEND=

>> acting as this gate and introduce CONFIG_XEN_PCI_STUB for the stub
>> functionality needed on Arm. XEN_PCIDEV_BACKEND would depend on X86 an=
d
>> select XEN_PCI_STUB, while on Arm XEN_PCI_STUB could be configured if
>> wanted. The splitting of the driver can still be done later.
>=20
> Hm, pciback is now compiled when CONFIG_XEN_PCIDEV_BACKEND=C2=A0 is ena=
bled
> and we want to skip some parts of its code when CONFIG_XEN_PCI_STUB is =
set.
> So, I imagine that for x86 we just enable CONFIG_XEN_PCIDEV_BACKEND and=
 the
> driver compiles in its current state. For Arm we enable both CONFIG_XEN=
_PCIDEV_BACKEND
> and CONFIG_XEN_PCI_STUB, so part of the driver is not compiled.

No, I'd rather switch to compiling xen-pciback when CONFIG_XEN_PCI_STUB
is set and compile only parts of it when CONFIG_XEN_PCIDEV_BACKEND is
not set (this will be the case on Arm).

This is another step in the right direction preparing the split.

But as said before, this is not a requirement by me to take your patch.


Juergen

--------------CD70FE0AF5754356184F6F62
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------CD70FE0AF5754356184F6F62--

--XacVIrZmUkb7pecLSR0D6hRuZnByoOX7A--

--wsuxArFoeRkbtGgBU1fF54AKz0F7tz7qQ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJgIYFAwAAAAAACgkQsN6d1ii/Ey/k
Kgf/R/E+OrRH+RURLkU1IS1biQYiccgDO1Uw79AiM8/I9qsk/yl6FSV+duIl/k0UNGQQa8il8/F3
I7ZZOyCuJdYvEszG29YZk4p40pP4gzwn/H7Omy9F0WoeGRZIFda1AHqY6EvAn61TO/GFaMYxO/Gw
oyELfp2VI9NG6j1gFBEVMYaMsZpc+CIoOWmGWm1iWPbFOjTUqnwIkxlW3DASJ0E6aMRVfn6ZYZ2x
eDLZx2quqhLfzhzqbeV+qhwNAZ8SdHxa0JyjDJL9n8sAuBTxvX2XfuiXCKATcmcAIEZYR6sabwZB
oaJdMfl6K1JwXrss980rTJkaZi+JovMbKx6RYaDCpg==
=mPy5
-----END PGP SIGNATURE-----

--wsuxArFoeRkbtGgBU1fF54AKz0F7tz7qQ--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 06:53:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 06:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191356.341339 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZez-0005fx-Sp; Tue, 21 Sep 2021 06:53:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191356.341339; Tue, 21 Sep 2021 06:53:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZez-0005fq-Pb; Tue, 21 Sep 2021 06:53:45 +0000
Received: by outflank-mailman (input) for mailman id 191356;
 Tue, 21 Sep 2021 06:53:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSZez-0005fk-1V
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 06:53:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21f6dc82-c8c4-470b-9211-9a9f8bc8d78e;
 Tue, 21 Sep 2021 06:53:43 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2053.outbound.protection.outlook.com [104.47.4.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-j-aCz5gbPMin6jJ5jpCQxQ-1; Tue, 21 Sep 2021 08:53:41 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4382.eurprd04.prod.outlook.com (2603:10a6:803:73::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 06:53:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 06:53:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR03CA0054.eurprd03.prod.outlook.com (2603:10a6:208::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Tue, 21 Sep 2021 06:53:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21f6dc82-c8c4-470b-9211-9a9f8bc8d78e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632207222;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7MgjA5UzoOR0QDM+7H7U3AOR8SIUwnhsa3GT0qjNqTE=;
	b=nX5zWrtGsM9DgteF18WTVHVIpahjfl8Bc0y0E0MSDWvmlPgvkG0bsopwahaiODxIT/nB+e
	HBErHCGP5XKFi1kV71QmGy7mNMl5JatzSe5WAHo8PfmhCrXqfVlwSbRagfaGwyws6XmJYu
	v8IbJBwnySlH9JXN1Itd+C3HsKFksbw=
X-MC-Unique: j-aCz5gbPMin6jJ5jpCQxQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=F+sRSHJ7XAXKX0RKVwFbn48vpUJWfSWU60naevsLQ9/2iqpXsJZj+qdfSYLfixI4D7IxV/Uc66wb09ITkJdoc2RanaWBToSPM81QhEZHgRq1YWJSNGWG0iIEe22NdcH/MTPOk94BqvsPjD+65PsthU8+mUZTeX9OsbyBQ0rPw71nRRFf99TGoe0E0b/hKnU4O9B8BJ27u5UHzMP4ck0QcJ3/9gs9CO7VNVIxYIvYDWphoOUzKA3e7FwM5boloW7rixsdSSnXjlqckX6pZXM6moNDl1YaDZjWQMIV1/5pEqn3WaCDDrjVXE0BFBr9Y7C2Tjk3yt1279utk6fJTj4D6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7MgjA5UzoOR0QDM+7H7U3AOR8SIUwnhsa3GT0qjNqTE=;
 b=Ytlczr6htR/YNk/n2aiA4utI1X0vCff+/yJeinDQQTmqpf7m2g/57XfcccTtoqNX26bL30rFfJLBru7DsKFOpdHnJnVT6S8rFfyvUcvb6T8X1hLut39xRFO83olZk5Qc9gUK1L0JtDOPJQ7LgihomjHU5xGuAOQlZO8odiD2Evc6Khah0EE9sqrjaJqqgWbAB5kChnkjERhy85zyzIdzO+XDNsKrTj3E8hFOAswOiGqXzkGIe6W9x8OdHYW7npK1LtG505HrEG3rgbR1Odk8zv7P6bVNoO7rr5hjUY7kDUPfFj4fL7y0k2KNEZWx+3CTZLn3AHZWZzRPgkKeVpt7bA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Meng Xu <mengxu@cis.upenn.edu>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
Date: Tue, 21 Sep 2021 08:53:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR03CA0054.eurprd03.prod.outlook.com (2603:10a6:208::31)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 79151cab-6602-4140-a7ce-08d97ccc8711
X-MS-TrafficTypeDiagnostic: VI1PR04MB4382:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB438254BB0A59019BF8EA5AB1B3A19@VI1PR04MB4382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9WowQux0IPWRIjuhP2VXXkRIWnfPq0GC8pE6/lZAJG1tMUH8UiXQQb5bIGbtu7EJdMtLzftmM44W1cQPgtC4twwqCPYW8jZ1nRWohp5cvEHKbglztVrTq4ypCHtClSAexucJZZ+AhC3mpNRYS13wc7twyncFpbRlHWTe104mAqU8hRRvf47HBCpnN5m8FFVcdpr7jdFc7YaL+3DuGOou3V0xwk2+zA4xh9laoPqoitjLwnlHkhAaN+7trd1A4UBRSplRmGhnKr3yX7XerZDWaWdQUePbDIfrPfbTSUjKhpFwQkyDN2cTVKcPukkRr6jtjpFA8T9u3rY8Jztf95Mut6fw+1AeUoSecTPJoVUw5hCdxUSSQMXvrnV12/EFvUzw+VoAP/dNVwHRylRSQ0KNAv07OvjlOx4ouQVUIbt1sfJdFDu/xjTUGpHfGam3e2Jp15ato2SJl1Nmv5bwE5mIql3l7XV/aNcBvR9FneTFoKGz+dJiOPaUv8dZBIoPE47RLjkOP+Z7A+ZdBraiXgtkKTodRCsBPIGU2rM3Om39Qszn+gSB8sMNyRJrThps1CZnR6ifX8MuzBs0dvBgwZfDdHCHU392PHO/qXSKKkXBjzapBbogRSnEbluUhPbgGT60Ju8izNomxR6KvUFrGhLLjYuRXqu8RCeLAVYRe8u8Aq6cOEQdCP4Z3kbNM7dS3lQ6Fsu9+99hR+aHF1UuxaUTzzB/57w9muulLPpnbU19R+k=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(54906003)(6916009)(31686004)(31696002)(86362001)(956004)(6486002)(2616005)(186003)(5660300002)(66476007)(66556008)(8936002)(4326008)(2906002)(38100700002)(36756003)(26005)(83380400001)(66946007)(316002)(508600001)(8676002)(53546011)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?d2kyVnVrTEhwZXJiajhJSHBSS3BHeTlOaHlsVWdyb1hUN3VXeXBLTkY3d3RN?=
 =?utf-8?B?b1ovU2hKekxZYTZid1lMNDJEd2RNWWFseEF3a0Y0R29zZzVHTkpUclJhSVNs?=
 =?utf-8?B?MGdTcVpqeW5hU0RhYTJ4bWhtNFNOaVQ5NmJPOVNJWFNjZ1ltby8xbDlVNUwr?=
 =?utf-8?B?TmpVRkl2ZGZsem44QTN0enFvVWVwdXMydnR6bzh4Nmk4RHFjakxIZmQwMVNJ?=
 =?utf-8?B?cHY1N1JrTkpvN1JyVXdWdFZLUmZCWllyU1VRdWNUWm1rWjM5Z2hsd1JkdUxj?=
 =?utf-8?B?aFBHVmdWdmljMkdMMGYzS1lNUHk0RXRFOUY0U3BVaFgvbUdheWY0VE5qL2Fo?=
 =?utf-8?B?Y1FoczNVSTI5dXFoam85dDdocFViSTlHU1RKbDVzcC9lZnl0T29ESkdLdWJB?=
 =?utf-8?B?dEcvMkwrK3ozVDY4c1hQM2FERS9kQWFRVWZGRGJzeXZJdW1OUisyRGZ4UGRF?=
 =?utf-8?B?YW9UZ2NSRFhvSVU1cXEvZlFnRXN4UDNZbURzdFg5dkNWMWZMN0d5QVA3R3pG?=
 =?utf-8?B?M21aVVR4V25FSVEyYkNEMnhpZFFrb055WEY3eXlMZG5LMVFqSzdVSXJYdGVS?=
 =?utf-8?B?UVRoMUt0ZjRiNTBLOUFUaTg5aDNITEJaSGZHY2tjQXJXU1NzclFyWEpEMjYz?=
 =?utf-8?B?Y0NBWGpZU3lnakw0NXFIYUh5eEZWV3hZK2V1REwxMzZxNGhuRkFwQjZFSitW?=
 =?utf-8?B?eUFsVkVjSkRBVVVhNVQ0KzNPY0t3VnNCckcwOFpZaXgvSjU4M1NnSTN0SmxS?=
 =?utf-8?B?UVlUeSt0K0hmMjNIa0JRRnR3ZWpnbUVianVtd0xPMEI1M2JHb1FEZWV2bXh6?=
 =?utf-8?B?T2hjMzk0OFE1TDVxV2JocDFoQ2RETUFERWdva0hNdWlpQU1rU2tBTk9DdkFE?=
 =?utf-8?B?MHcvVCtMa0dMZE9yRDdZMkFhVmd6c0VRRlFmZXlNbDhDUmZKU0FmamFndm1z?=
 =?utf-8?B?ZE04OFd0VXhZVGZyTXJWeVJHNm5kY2hpb3MvQ0kvM2lUTjRrMHFVUzI5cEcw?=
 =?utf-8?B?Ky84bmgzQ2lpSGFWK015MGErV0xnOGxzQkZCU21aVUJsQVRKVUNOZWZlWjRN?=
 =?utf-8?B?Lzl0dW9hakZMODVoclNYalFkOVJ1bGxkY0hMYVdlVjRlQWlKekRFQko2YTc5?=
 =?utf-8?B?S3Rsc3NpRkZWcm5rL3FJZFVJOFBxbW03YklPWjdJVUIzNmZzVVNad29pRGJH?=
 =?utf-8?B?RGZKZlV4eEtCR083VzZlVkxsc081NmN3cUREc0gwN0FQcjgwTUYzRzhJSEU4?=
 =?utf-8?B?WE1WM0Y4SEhvK21XTUhMeWlMaHZCTzY0Z3FKWS9mZVF5aFdEUUYzMGREa1M3?=
 =?utf-8?B?OWFoWlRHT0NlWW9QRUJZeHZla1A0NGp0OW9tcnZ1UWVJcTNCbEhqWXAyeG1R?=
 =?utf-8?B?bWsyWlNJbjVOZkRYZVB2UVZka1hoeHRvZUIxYm4xang5SXpYcm5STGNUYjl2?=
 =?utf-8?B?Vkx0RGlQUm5mc1M5c0lKOFNwd215TW1WUEJJSy9jUFlpUS9xMDEyMFl0UWhH?=
 =?utf-8?B?M2ZEVGErZjdMblNJL0FxSHpVeWU4Mi8rS3ZaZTRDeVZjRGczODlaQlRoclFH?=
 =?utf-8?B?ZFpZQUFBR1l4N2JIRHpnWStMbUN5dndZcllMbi9SKzRlL1lnd0U3RkdaejZj?=
 =?utf-8?B?OWt1Q1laR29wY0x6MnRhOVFWSm1rdHAvSVpJL0kvN2pBbGlsaEVwS0VYQTB5?=
 =?utf-8?B?ejFDNlluT3VTVHJ6b1AzOC9BbTRNVXphTC9YQ2djN2l2LzZuaUNBbDNwc1li?=
 =?utf-8?Q?U6s8V8u9jq6aLInno6fVZWaefq4HSbb3ZylI09W?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 79151cab-6602-4140-a7ce-08d97ccc8711
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 06:53:33.2268
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: x5WL9KKSGnwonJ577G6yE7cpj+l2eqPiYbmVZeQNXMr8Ij7uGrErJntK/bR7zUIlTgoV7KjN/MXtWOWgHwykYg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4382

On 20.09.2021 19:25, Andrew Cooper wrote:
> In the case that 'extra' isn't a multiple of uint32_t, the calculation rounds
> the number of bytes up, causing later logic to read unrelated bytes beyond the
> end of the object.
> 
> Also, asserting that the object is within TRACE_EXTRA_MAX, but truncating it
> in release builds is rude.  Instead, reject any out-of-spec records, leaving
> enough of a message to identify the faulty caller.
> 
> There is one buggy race record, TRC_RTDS_BUDGET_BURN.  As it must remain

Nit: I guess s/race/trace/ ?

> __packed (as cur_budget is misaligned), change bool has_extratime to uint32_t
> to compensate.
> 
> The new printk() can also be hit by HVMOP_xentrace, although no over-read is
> possible.  This has no business being a hypercall in the first place, as it
> can't be used outside of custom local debugging, so extend the uint32_t
> requirement to HVMOP_xentrace too.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Two remarks (plus further not directly related ones), nevertheless:

> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
>          if ( copy_from_guest(&tr, arg, 1 ) )
>              return -EFAULT;
>  
> -        if ( tr.extra_bytes > sizeof(tr.extra)
> -             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
> +        if ( tr.extra_bytes % sizeof(uint32_t) ||
> +             tr.extra_bytes > sizeof(tr.extra) ||
> +             tr.event >> TRC_SUBCLS_SHIFT )
>              return -EINVAL;

Despite this being a function that supposedly no-one is to really
use, you're breaking the interface here when really there would be a
way to be backwards compatible: Instead of failing, pad the data to
a 32-bit boundary. As the interface struct is large enough, this
would look to be as simple as a memset() plus aligning extra_bytes
upwards. Otherwise the deliberate breaking of potential existing
callers wants making explicit in the respective paragraph of the
description.

As an aside I think at the very least the case block wants enclosing
in "#ifdef CONFIG_TRACEBUFFER", such that builds without the support
would indicate so to callers (albeit that indication would then be
accompanied by a bogus log message in debug builds).

Seeing the adjacent HVMOP_get_time I also wonder who the intended
users of that one are.

> --- a/xen/common/sched/rt.c
> +++ b/xen/common/sched/rt.c
> @@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct rt_unit *svc, s_time_t now)
>      /* TRACE */
>      {
>          struct __packed {
> -            unsigned unit:16, dom:16;
> +            uint16_t unit, dom;
>              uint64_t cur_budget;
> -            int delta;
> -            unsigned priority_level;
> -            bool has_extratime;
> -        } d;
> -        d.dom = svc->unit->domain->domain_id;
> -        d.unit = svc->unit->unit_id;
> -        d.cur_budget = (uint64_t) svc->cur_budget;
> -        d.delta = delta;
> -        d.priority_level = svc->priority_level;
> -        d.has_extratime = svc->flags & RTDS_extratime;
> +            uint32_t delta;

The original field was plain int, and aiui for a valid reason. I
don't see why you couldn't use int32_t here.

Feel free to retain the R-b if you decide to make the two suggested
changes which are directly related to your change here.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:00:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:00:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191367.341350 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZlm-0007I0-OZ; Tue, 21 Sep 2021 07:00:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191367.341350; Tue, 21 Sep 2021 07:00:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZlm-0007Ht-L7; Tue, 21 Sep 2021 07:00:46 +0000
Received: by outflank-mailman (input) for mailman id 191367;
 Tue, 21 Sep 2021 07:00:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSZll-0007Hn-CV
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:00:45 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a38a8842-1aa9-11ec-b89e-12813bfff9fa;
 Tue, 21 Sep 2021 07:00:43 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L64ZHY007744; 
 Tue, 21 Sep 2021 07:00:42 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2058.outbound.protection.outlook.com [104.47.4.58])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b77eyrduy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 07:00:41 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5011.eurprd03.prod.outlook.com (2603:10a6:208:101::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 07:00:39 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:00:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a38a8842-1aa9-11ec-b89e-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=c1MwUlmkvIQ8upGAfqnvvqjWkf3nc8H985l2bk4UwqzUwxMTuniCBfxtslZ+Vg59sPsGhJw5A6LG6u26NITS4PNBAgrUWkBdmWzK9aBpapjyUlMTBy5bnuv1sotYCBarNh0PyZAw2QuF/dgCWh76rVu+jLLugFp0lnYdeS5yfFEn2KcD8gSv12o7Lqid5mU4p2ehnKOo5ua82ccsdYS2vkjJFE8QqDYXqmEuJGWxhYSSNnXiHkZdn+zsDWCYn2TA+VJ0hqN//WOKG327YEqL+8vvPJIblMSO5cp0+DPAuaa6bzCkEdokVXyzsEcI1up7WTzoWOU9AjZ4pXN5Sa3fYA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=armrUEo1vMEssjuJiia6IZ1nZsZE7fL6+asrzL9lEG4=;
 b=Qm75susJezlVxqJIhLDD2BMtGeiUb5JpYV00LiE0bzicejjDPw1USgCb275OhbEDRwodI/6EYg73DtspO6LrNuQ21txNg5Stz7XtWurd2qBxYd64X+dC86pXeqcujjm1vG6XI0R+RY1fJBcpnl/YYhi2S4BnR1qJn1Pg8/INHSus3ux6kCtHpMGOPB//WjvDV0qBdcEHMktUQJoPkwefrAWSHuIF8Z7AWV/3vpsnEm0K3C19rZlaabrpWesuzpDss08LEeruNJ6ikowjx0fMSCACj/EwTQVpUYXQu8S19O/uHTBtWxtG6qG3vu7usspgc5x2zvThPS/hYp2hUVEJ4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=armrUEo1vMEssjuJiia6IZ1nZsZE7fL6+asrzL9lEG4=;
 b=aPEyMV6rIZSQKKHzmAKMvEP/yYhK5LlOUzYHuvclhIKUNnAq4ekjfpgoqlhjS0Q6b8nXiQnfm+a/HUka2Op+yEQJK504aHuluGmp496df3fgFitf1CenUb/3Mmv1KzMVuvd2RjQvTb+FrKI6DOnp/MbovEjZK0ieRmdtnbqyp71IEJmDAswvrw/H9rvQ2uP4BLMXrKUypoyZby741tWUuFaoJALYJlggdpw5TWyvTp1649bZlbOaZ9Ihc7dHWE7kruRsalRkh80QJ2mclLMLv9jAD983q/5kPuaPRSGEZBgaydcokvBAOHvmUtWKCW5SVvCQN4Zjm88Jf/bQfXCV2Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAGXhAIAACLuAgAAEUwCAAAi6AIAAAxMAgAADDgA=
Date: Tue, 21 Sep 2021 07:00:39 +0000
Message-ID: <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
In-Reply-To: <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 75c8071b-86c6-478b-bc79-08d97ccd8507
x-ms-traffictypediagnostic: AM0PR03MB5011:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB50116608BDC2835D18AB7D7FE7A19@AM0PR03MB5011.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 8bf6Pe5oo87kO0Mgipy3gl/HMYXWSXk7kqD1bWcNchBhtXwxvJ8IKAJ6r9W/LIpYQjzjcgrWBZc2hpesxZFsAd20pLEind8RbAflsOY1/tGuuuBTVnnxv2D9Mcb9H5m89BaviQyMXZpOU6WfCjBcaYHqFPpBtxX5G3Feo6Oyx99dPm2WNgFQGvilr7hWyLeaC7YFO+HuO4+bZG8W5Jb7t/7T0M6dN51SgnAHVZkiKYXUUDyt5xT7huK5MMEd0gQLnfpQdsim+5r1cKPNCavQ71o+eDK3VUg4GxAKNSM8zCaAYt4QBrCAURhCp0XMeXOKlq1eNi97spwKJgdFNz+NUHvQc8F5kxWPEpXtkFY/VjaJAK2igz8bVjTb/633cy86lTxpTRU2exDfR2TCpT3E9hUb2zz0GWW+HWMQxEPgIvQl8QFtW2r3iRzC6Bc0yalsgEh+fNht7+YpzovSXzTyM7njwEXqEk2qw/AAyj1pdwdxeM6E04igQdoZrcXu0I1SclTIVcIJ9b/fhTuJoGPJAEYqvImB0cJs8gHKGKl1ALiFToNNQO8XMsqwS8pbMo5qilrsWiLq5F1XBl1+diS7B1dGtaWXNG7EJi+8TzRSdSYJaVuqcBVcNOyYpsla5zx3ymChWFl5iL/A5JsX+hgb8C4jIrG3fXyHNQm5kpbN+gdcF4ibUD8whTSyWK4I5hRHL2DJtZB31Mi5SZ0r0Smm8H25N9KDGAdGr4FCzgdi0rj46GD7+cIGKko0P/3u6tN9
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(54906003)(110136005)(31696002)(2616005)(71200400001)(6486002)(38070700005)(8676002)(55236004)(66476007)(83380400001)(53546011)(6506007)(5660300002)(36756003)(316002)(8936002)(66446008)(66556008)(508600001)(6512007)(66946007)(91956017)(76116006)(64756008)(122000001)(86362001)(38100700002)(186003)(26005)(31686004)(2906002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ejBrUE9Jd2N0aGZ0SWtCZmliM3NLcUlPTi9DcExwN2V5NnNTRUEvRzFvSy9s?=
 =?utf-8?B?WUlBWS94czUrWWJYYkgvNWJINEIvdS9HUTdvM2NFSzNLc05CSE9XMDV3Sk1z?=
 =?utf-8?B?R3BwSGh1T0tPNENVbi81RDFQOWZnNG5VMjlPVkxYU2JjY0xvcUp0cXA3eHJv?=
 =?utf-8?B?QjA1ZVV2TnM4V0I5WHNBV0k3aGxZK2pjN01LbDZ3aGhvVVFPR3ErZlNTSVMv?=
 =?utf-8?B?aVQvUVlidWZqci94RGFZendSeVRkZHdkSE05RUU2T1EraFE0enZMYTJVbzlD?=
 =?utf-8?B?Q3BwUjl4QyswaGJiajdzN3p1YkpyZlBZV2ZxQ2hjLzVVU25rbzk1NlVxOFlV?=
 =?utf-8?B?L1cxTDFqQllySlZ0U25GamVNOVRqRk53UCtsODVkdmticEFhbWJFdXJucEU0?=
 =?utf-8?B?N25TNnVkOHZJeDJtNlhCU2FKMlFYR1F1RVdFdXNGY1YvcE81UkJYblpISzVn?=
 =?utf-8?B?WEc1QzZqV2xtK3BvUktOclR5R1dzNHRQUEZzOWhBL2lDMStSeVZSZWx6ZVZq?=
 =?utf-8?B?RkpETUFZQmEzL3ppYmpnMWhyWkg4M3g0ZmRhRjA5cHhXaWJ1VTE3TXZPM0FJ?=
 =?utf-8?B?NUg0Rm9xMHU5R051S1pUWjV1Nld6c050RXBVVXVwQ1lqY2k4Wjg3ajZudVcx?=
 =?utf-8?B?SDlMMmZvM3FZbERoZGFPSnE4KzVvSUpNQk1qZ010RWJUdnhuTDU4SjNTYXlv?=
 =?utf-8?B?WEVaODdBTTQ0eDZSU0xtOWtLN0M4TjNNeHEvTlZwd0p4SnFrRWQzZ0VYUW8v?=
 =?utf-8?B?TStiNGNhVkREcGhCem1qMmZSQzFSbTU0SXpVMGhYY3lHYlc1QUF6R1pxRVJz?=
 =?utf-8?B?eDVqYWR0c0pqdWRxTmdpcEFXNVRmaThhZ3IyRWRtS3BsKzlnZ09uL3NBUGZL?=
 =?utf-8?B?OFAybGpOblp1SFAyUHpNWVhPdWdkcXJqV1JlMEU5TmdCWk9WMjBybWxtUzI5?=
 =?utf-8?B?ZFJMSllJYnBGRjU5bjhQdkVRZWh3ejJ4dkhQellrVWF0dENPL29mRGlEODAv?=
 =?utf-8?B?YTQ3OGJIUE9MaEswb0htN1NNL0VnYUdQWkg2K25uUXM0L25NT3VEOVc0N09o?=
 =?utf-8?B?dk5ZWGNleG5IYTl2LzRjWnhuNDVmNlJQaEdIeUFXQmU0NzRWbFNJMnZ2Qjg5?=
 =?utf-8?B?S2lJeWgwYTlnSFE1alpQSytNUFI1OVlYaTViUE9iVlIvbVJxSlFEMkplSGYz?=
 =?utf-8?B?d2txWGl1YzZ6cmljT20vVTFzVWV4bHkrZ0o3K1pxcldqNGM1dkRhUXdwbkdD?=
 =?utf-8?B?bDNWR1hHZWNrWVBHSEtuZk10dWNtbXJIRGxXWDJoN3hVNUVRNEVpa1N0bnJT?=
 =?utf-8?B?QkZCTnN3R1lvbElJaFUvT3dEOWNwdUJkbWpEOWtzcUVVMlQ5MCt2YW80eHZR?=
 =?utf-8?B?eStyNHZwY2V5V21TSGFZMmFBS09UQnNNaDNBQmM2Rm0wT01BcEsxeHdpeUIz?=
 =?utf-8?B?bHNESEdFM1pSdmM4ZUlIKzZBQWFOMXZLcWNUZGJmREE5TWJNdUdYZ3FJWW1T?=
 =?utf-8?B?M3p5WU10a3FWeHMxNE5yZFU1MFlsSTZDYW9aUjdRUkx5NHN6T3QvS0J4Z3R5?=
 =?utf-8?B?NERYT0pqKzZ1elFDd3F6cGNOZjhKa0UwcVZzMEhURTdvajdvcldjMUw4WWZj?=
 =?utf-8?B?MkI2Z3UwNG92SThiK0Z2aUdzR05xT04wQis3WUtrbXZMWU1mYmYzQ090UUNQ?=
 =?utf-8?B?MDJTK1Fla2doeS9KVGpibEc0WklVMGYzOTZEaTRKVVYxWkhZd2ZUOXl3a1Bx?=
 =?utf-8?B?eEJaeUR0UmhrSkhnM0pKa0Nyekl3QWI4V2hxWGZwdjMrZ2xxU2JSakMwcnFr?=
 =?utf-8?B?YUlKU04wZ1BMT3FkTkgvdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <3187AAACD18CCD418C0763966CBD457B@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 75c8071b-86c6-478b-bc79-08d97ccd8507
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 07:00:39.0706
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: XMmce2+uVHXgTDry9GiCD1ojKOL7QAPG7gd/NE003Skz5xFezP/eSKnlLlHO4yTwPwqPQ7cZu5MbeAjNave7IImIgXLXiQRA4RMGpPV+R4tB8BBbQFglkHmhILVMF1c8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5011
X-Proofpoint-GUID: 2KKReqwGrWqsba8d7_sj25Y1vQ8Js6u0
X-Proofpoint-ORIG-GUID: 2KKReqwGrWqsba8d7_sj25Y1vQ8Js6u0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0
 clxscore=1015 suspectscore=0 lowpriorityscore=0 mlxscore=0
 priorityscore=1501 impostorscore=0 mlxlogscore=999 adultscore=0
 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210045

DQpPbiAyMS4wOS4yMSAwOTo0OSwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjEuMDkuMjEg
MDg6MzgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDIxLjA5LjIx
IDA5OjA3LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDIxLjA5LjIxIDA3OjUxLCBPbGVr
c2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4NCj4+Pj4gT24gMjEuMDkuMjEgMDg6MjAs
IEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4+PiBPbiAyMS4wOS4yMSAwMToxNiwgU3RlZmFubyBT
dGFiZWxsaW5pIHdyb3RlOg0KPj4+Pj4+IE9uIE1vbiwgMjAgU2VwIDIwMjEsIE9sZWtzYW5kciBB
bmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+Pj4+PiBPbiAyMC4wOS4yMSAxNDozMCwgSnVlcmdlbiBH
cm9zcyB3cm90ZToNCj4+Pj4+Pj4+IE9uIDIwLjA5LjIxIDA3OjIzLCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+Pj4+Pj4+PiBIZWxsbywgU3RlZmFubyENCj4+Pj4+Pj4+Pg0KPj4+
Pj4+Pj4+IE9uIDE4LjA5LjIxIDAwOjQ1LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4+
Pj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4gV2h5IGRvIHlvdSB3
YW50IHRvIGVuYWJsZSBwY2liYWNrIG9uIEFSTT8gSXMgaXQgb25seSB0byAiZGlzYWJsZSIgYSBQ
Q0kNCj4+Pj4+Pj4+Pj4gZGV2aWNlIGluIERvbTAgc28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFz
c2lnbmVkIHRvIGEgRG9tVT8NCj4+Pj4+Pj4+PiBOb3Qgb25seSB0aGF0DQo+Pj4+Pj4+Pj4+DQo+
Pj4+Pj4+Pj4+IEkgYW0gYXNraW5nIGJlY2F1c2UgYWN0dWFsbHkgSSBkb24ndCB0aGluayB3ZSB3
YW50IHRvIGVuYWJsZSB0aGUgUFYgUENJDQo+Pj4+Pj4+Pj4+IGJhY2tlbmQgZmVhdHVyZSBvZiBw
Y2liYWNrIG9uIEFSTSwgcmlnaHQ/IFRoYXQgd291bGQgY2xhc2ggd2l0aCB0aGUgUENJDQo+Pj4+
Pj4+Pj4+IGFzc2lnbm1lbnQgd29yayB5b3UgaGF2ZSBiZWVuIGRvaW5nIGluIFhlbi4gVGhleSBj
b3VsZG4ndCBib3RoIHdvcmsgYXQNCj4+Pj4+Pj4+Pj4gdGhlIHNhbWUgdGltZS4NCj4+Pj4+Pj4+
PiBDb3JyZWN0LCBpdCBpcyBub3QgdXNlZA0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBJZiB3ZSBv
bmx5IG5lZWQgcGNpYmFjayB0byAicGFyayIgYSBkZXZpY2UgaW4gRG9tMCwgd291bGRuJ3QgaXQg
YmUNCj4+Pj4+Pj4+Pj4gcG9zc2libGUgYW5kIGJldHRlciB0byB1c2UgcGNpLXN0dWIgaW5zdGVh
ZD8NCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IE5vdCBvbmx5IHRoYXQsIHNvIHBjaS1zdHViIGlzIG5v
dCBlbm91Z2gNCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IFRoZSBmdW5jdGlvbmFsaXR5IHdoaWNoIGlz
IGltcGxlbWVudGVkIGJ5IHRoZSBwY2liYWNrIGFuZCB0aGUgdG9vbHN0YWNrDQo+Pj4+Pj4+Pj4g
YW5kIHdoaWNoIGlzIHJlbGV2YW50L21pc3NpbmcvbmVlZGVkIGZvciBBUk06DQo+Pj4+Pj4+Pj4N
Cj4+Pj4+Pj4+PiAxLiBwY2liYWNrIGlzIHVzZWQgYXMgYSBkYXRhYmFzZSBmb3IgYXNzaWduYWJs
ZSBQQ0kgZGV2aWNlcywgZS5nLiB4bA0KPj4+Pj4+Pj4+IMKgwqDCoMKgIMKgwqAgcGNpLWFzc2ln
bmFibGUte2FkZHxyZW1vdmV8bGlzdH0gbWFuaXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywgd2hlbmV2
ZXIgdGhlDQo+Pj4+Pj4+Pj4gwqDCoMKgwqAgwqDCoCB0b29sc3RhY2sgbmVlZHMgdG8ga25vdyB3
aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2ggaXQgcmVhZHMNCj4+Pj4+Pj4+
PiDCoMKgwqDCoCDCoMKgIHRoYXQgZnJvbSB0aGUgcmVsZXZhbnQgc3lzZnMgZW50cmllcyBvZiB0
aGUgcGNpYmFjay4NCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IDIuIHBjaWJhY2sgaXMgdXNlZCB0byBo
b2xkIHRoZSB1bmJvdW5kIFBDSSBkZXZpY2VzLCBlLmcuIHdoZW4gcGFzc2luZyB0aHJvdWdoDQo+
Pj4+Pj4+Pj4gwqDCoMKgwqAgwqDCoCBhIFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3Vu
ZCBmcm9tIHRoZSByZWxldmFudCBkZXZpY2UgZHJpdmVyIGFuZCBib3VuZA0KPj4+Pj4+Pj4+IMKg
wqDCoMKgIMKgwqAgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3BlYWtpbmcgaXQgaXMgbm90IHJlcXVp
cmVkIHRoYXQgdGhlIGRldmljZSBpcyBib3VuZCB0bw0KPj4+Pj4+Pj4+IMKgwqDCoMKgIMKgwqAg
cGNpYmFjaywgYnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBhcyBhIGRhdGFiYXNlIG9mIHRoZSBw
YXNzZWQgdGhyb3VnaCBQQ0kNCj4+Pj4+Pj4+PiDCoMKgwqDCoCDCoMKgIGRldmljZXMsIHNvIHdl
IGNhbiByZS1iaW5kIHRoZSBkZXZpY2VzIGJhY2sgdG8gdGhlaXIgb3JpZ2luYWwgZHJpdmVycyB3
aGVuDQo+Pj4+Pj4+Pj4gwqDCoMKgwqAgwqDCoCBndWVzdCBkb21haW4gc2h1dHMgZG93bikNCj4+
Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IDMuIERldmljZSByZXNldA0KPj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4g
V2UgaGF2ZSBwcmV2aW91c2x5IGRpc2N1c3NlZCBvbiB4ZW4tZGV2ZWwgTUwgcG9zc2libGUgc29s
dXRpb25zIHRvIHRoYXQgYXMgZnJvbSB0aGUNCj4+Pj4+Pj4+PiBhYm92ZSB3ZSBzZWUgdGhhdCBw
Y2liYWNrIGZ1bmN0aW9uYWxpdHkgaXMgZ29pbmcgdG8gYmUgb25seSBwYXJ0aWFsbHkgdXNlZCBv
biBBcm0uDQo+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+PiBQbGVhc2Ugc2VlIFsxXSBhbmQgWzJdOg0KPj4+
Pj4+Pj4+DQo+Pj4+Pj4+Pj4gMS4gSXQgaXMgbm90IGFjY2VwdGFibGUgdG8gbWFuYWdlIHRoZSBh
c3NpZ25hYmxlIGxpc3QgaW4gWGVuIGl0c2VsZg0KPj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4gMi4gcGNp
YmFjayBjYW4gYmUgc3BsaXQgaW50byB0d28gcGFydHM6IFBDSSBhc3NpZ25hYmxlL2JpbmQvcmVz
ZXQgaGFuZGxpbmcgYW5kDQo+Pj4+Pj4+Pj4gdGhlIHJlc3QgbGlrZSB2UENJIGV0Yy4NCj4+Pj4+
Pj4+Pg0KPj4+Pj4+Pj4+IDMuIHBjaWZyb250IGlzIG5vdCB1c2VkIG9uIEFybQ0KPj4+Pj4+Pj4N
Cj4+Pj4+Pj4+IEl0IGlzIG5laXRoZXIgaW4geDg2IFBWSC9IVk0gZ3Vlc3RzLg0KPj4+Pj4+PiBE
aWRuJ3Qga25vdyB0aGF0LCB0aGFuayB5b3UgZm9yIHBvaW50aW5nDQo+Pj4+Pj4+Pg0KPj4+Pj4+
Pj4+IFNvLCBsaW1pdGVkIHVzZSBvZiB0aGUgcGNpYmFjayBpcyBvbmUgb2YgdGhlIGJyaWNrcyB1
c2VkIHRvIGVuYWJsZSBQQ0kgcGFzc3Rocm91Z2gNCj4+Pj4+Pj4+PiBvbiBBcm0uIEl0IHdhcyBl
bm91Z2ggdG8ganVzdCByZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBhbmQgaGF2ZSBpdCBydW4gb24g
QXJtIHRvIGFjaGlldmUNCj4+Pj4+Pj4+PiBhbGwgdGhlIGdvYWxzIGFib3ZlLg0KPj4+Pj4+Pj4+
DQo+Pj4+Pj4+Pj4gSWYgd2Ugc3RpbGwgdGhpbmsgaXQgaXMgZGVzaXJhYmxlIHRvIGJyZWFrIHRo
ZSBwY2liYWNrIGRyaXZlciBpbnRvICJjb21tb24iIGFuZCAicGNpZnJvbnQgc3BlY2lmaWMiDQo+
Pj4+Pj4+Pj4gcGFydHMgdGhlbiBpdCBjYW4gYmUgZG9uZSwgeWV0IHRoZSBwYXRjaCBpcyBnb2lu
ZyB0byBiZSB0aGUgdmVyeSBmaXJzdCBicmljayBpbiB0aGF0IGJ1aWxkaW5nLg0KPj4+Pj4+Pj4N
Cj4+Pj4+Pj4+IERvaW5nIHRoaXMgc3BsaXQgc2hvdWxkIGJlIGRvbmUsIGFzIHRoZSBwY2lmcm9u
dCBzcGVjaWZpYyBwYXJ0IGNvdWxkIGJlDQo+Pj4+Pj4+PiBvbWl0dGVkIG9uIHg4NiwgdG9vLCBp
biBjYXNlIG5vIFBWIGd1ZXN0cyB1c2luZyBQQ0kgcGFzc3Rocm91Z2ggaGF2ZSB0bw0KPj4+Pj4+
Pj4gYmUgc3VwcG9ydGVkLg0KPj4+Pj4+PiBBZ3JlZSwgdGhhdCB0aGUgZmluYWwgc29sdXRpb24g
c2hvdWxkIGhhdmUgdGhlIGRyaXZlciBzcGxpdA0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+PiBTbywgSSB0
aGluayB0aGlzIHBhdGNoIGlzIHN0aWxsIGdvaW5nIHRvIGJlIG5lZWRlZCBiZXNpZGVzIHdoaWNo
IGRpcmVjdGlvbiB3ZSB0YWtlLg0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IFNvbWUga2luZCBvZiB0aGlz
IHBhdGNoLCB5ZXMuIEl0IG1pZ2h0IGxvb2sgZGlmZmVyZW50IGluIGNhc2UgdGhlIHNwbGl0DQo+
Pj4+Pj4+PiBpcyBkb25lIGZpcnN0Lg0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IEkgZG9uJ3QgbWluZCBk
b2luZyBpdCBpbiBlaXRoZXIgc2VxdWVuY2UuDQo+Pj4+Pj4+Pg0KPj4+Pj4+PiBXaXRoIHRoaXMg
cGF0Y2ggd2UgaGF2ZSBBcm0gb24gdGhlIHNhbWUgcGFnZSBhcyB0aGUgYWJvdmUgbWVudGlvbmVk
IHg4NiBndWVzdHMsDQo+Pj4+Pj4+DQo+Pj4+Pj4+IGUuZy4gdGhlIGRyaXZlciBoYXMgdW51c2Vk
IGNvZGUsIGJ1dCB5ZXQgYWxsb3dzIEFybSB0byBmdW5jdGlvbiBub3cuDQo+Pj4+Pj4+DQo+Pj4+
Pj4+IEF0IHRoaXMgc3RhZ2Ugb2YgUENJIHBhc3N0aHJvdWdoIG9uIEFybSBpdCBpcyB5ZXQgZW5v
dWdoLiBMb25nIHRlcm0sIHdoZW4NCj4+Pj4+Pj4NCj4+Pj4+Pj4gdGhlIGRyaXZlciBnZXRzIHNw
bGl0LCBBcm0gd2lsbCBiZW5lZml0IGZyb20gdGhhdCBzcGxpdCB0b28sIGJ1dCB1bmZvcnR1bmF0
ZWx5IEkgZG8gbm90DQo+Pj4+Pj4+DQo+Pj4+Pj4+IGhhdmUgZW5vdWdoIGJhbmR3aWR0aCBmb3Ig
dGhhdCBwaWVjZSBvZiB3b3JrIGF0IHRoZSBtb21lbnQuDQo+Pj4+Pj4NCj4+Pj4+PiBUaGF0J3Mg
ZmFpciBhbmQgSSBkb24ndCB3YW50IHRvIHNjb3BlLWNyZWVwIHRoaXMgc2ltcGxlIHBhdGNoIGFz
a2luZyBmb3INCj4+Pj4+PiBhbiBlbm9ybW91cyByZXdvcmsuIEF0IHRoZSBzYW1lIHRpbWUgSSBk
b24ndCB0aGluayB3ZSBzaG91bGQgZW5hYmxlIHRoZQ0KPj4+Pj4+IHdob2xlIG9mIHBjaWJhY2sg
b24gQVJNIGJlY2F1c2UgaXQgd291bGQgYmUgZXJyb25lb3VzIGFuZCBjb25mdXNpbmcuDQo+Pj4+
DQo+Pj4+IEFzIHRoZSBmaXJzdCBzdGFnZSBiZWZvcmUgdGhlIGRyaXZlciBpcyBzcGxpdCBvciBp
ZmRlZidzIHVzZWQgLSBjYW4gd2UgdGFrZSB0aGUgcGF0Y2gNCj4+Pj4gYXMgaXMgbm93PyBJbiBl
aXRoZXIgd2F5IHdlIGNob3NlIHRoaXMgbmVlZHMgdG8gYmUgZG9uZSwgZS5nLiBlbmFibGUgY29t
cGlsaW5nDQo+Pj4+IGZvciBvdGhlciBhcmNoaXRlY3R1cmVzIGFuZCBjb21tb24gY29kZSBtb3Zl
Lg0KPj4+DQo+Pj4gRmluZSB3aXRoIG1lIGluIHByaW5jaXBsZS4gSSBuZWVkIHRvIHRha2UgYSBt
b3JlIHRob3JvdWdoIGxvb2sNCj4+PiBhdCB0aGUgcGF0Y2gsIHRob3VnaC4NCj4+IE9mIGNvdXJz
ZQ0KPj4+DQo+Pj4+DQo+Pj4+Pj4NCj4+Pj4+PiBJIGFtIHdvbmRlciBpZiB0aGVyZSBpcyBhIHNp
bXBsZToNCj4+Pj4+Pg0KPj4+Pj4+IGlmICgheGVuX3B2X2RvbWFpbigpKQ0KPj4+Pj4+IMKgwqDC
oMKgwqDCoCByZXR1cm47DQo+Pj4+Pj4NCj4+Pj4+PiBUaGF0IHdlIGNvdWxkIGFkZCBpbiBhIGNv
dXBsZSBvZiBwbGFjZXMgaW4gcGNpYmFjayB0byBzdG9wIGl0IGZyb20NCj4+Pj4+PiBpbml0aWFs
aXppbmcgdGhlIHBhcnRzIHdlIGRvbid0IGNhcmUgYWJvdXQuIFNvbWV0aGluZyBhbG9uZyB0aGVz
ZSBsaW5lcw0KPj4+Pj4+ICh1bnRlc3RlZCBhbmQgcHJvYmFibHkgaW5jb21wbGV0ZSkuDQo+Pj4+
Pj4NCj4+Pj4+PiBXaGF0IGRvIHlvdSBndXlzIHRoaW5rPw0KPj4+Pj4NCj4+Pj4+IFVoIG5vLCBu
b3QgaW4gdGhpcyB3YXksIHBsZWFzZS4gVGhpcyB3aWxsIGtpbGwgcGNpIHBhc3N0aHJvdWdoIG9u
IHg4Ng0KPj4+Pj4gd2l0aCBkb20wIHJ1bm5pbmcgYXMgUFZILiBJIGRvbid0IHRoaW5rIHRoaXMg
aXMgd29ya2luZyByaWdodCBub3csIGJ1dA0KPj4+Pj4gYWRkaW5nIG1vcmUgY29kZSBtYWtpbmcg
aXQgZXZlbiBoYXJkZXIgdG8gd29yayBzaG91bGQgYmUgYXZvaWRlZC4NCj4+Pj4+DQo+Pj4+Pj4g
ZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jIGIvZHJpdmVycy94
ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMNCj4+Pj4+PiBpbmRleCBkYTM0Y2U4NWRjODguLjk5MWJh
MGE5YjM1OSAxMDA2NDQNCj4+Pj4+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5i
dXMuYw0KPj4+Pj4+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+Pj4+
Pj4gQEAgLTE1LDYgKzE1LDcgQEANCj4+Pj4+PiDCoMKgwqAgI2luY2x1ZGUgPHhlbi94ZW5idXMu
aD4NCj4+Pj4+PiDCoMKgwqAgI2luY2x1ZGUgPHhlbi9ldmVudHMuaD4NCj4+Pj4+PiDCoMKgwqAg
I2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+Pj4+PiArI2luY2x1ZGUgPHhlbi94ZW4uaD4NCj4+Pj4+
PiDCoMKgwqAgI2luY2x1ZGUgInBjaWJhY2suaCINCj4+Pj4+PiDCoMKgwqAgwqAgI2RlZmluZSBJ
TlZBTElEX0VWVENITl9JUlHCoCAoLTEpDQo+Pj4+Pj4gQEAgLTY4NSw4ICs2ODYsMTIgQEAgc3Rh
dGljIGludCB4ZW5fcGNpYmtfeGVuYnVzX3Byb2JlKHN0cnVjdCB4ZW5idXNfZGV2aWNlICpkZXYs
DQo+Pj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgY29uc3Qgc3Ry
dWN0IHhlbmJ1c19kZXZpY2VfaWQgKmlkKQ0KPj4+Pj4+IMKgwqDCoCB7DQo+Pj4+Pj4gwqDCoMKg
wqDCoMKgwqAgaW50IGVyciA9IDA7DQo+Pj4+Pj4gLcKgwqDCoCBzdHJ1Y3QgeGVuX3BjaWJrX2Rl
dmljZSAqcGRldiA9IGFsbG9jX3BkZXYoZGV2KTsNCj4+Pj4+PiArwqDCoMKgIHN0cnVjdCB4ZW5f
cGNpYmtfZGV2aWNlICpwZGV2Ow0KPj4+Pj4+ICsNCj4+Pj4+PiArwqDCoMKgIGlmICgheGVuX3B2
X2RvbWFpbigpKQ0KPj4+Pj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsNCj4+Pj4+PiDCoMKg
wqAgK8KgwqDCoCBwZGV2ID0gYWxsb2NfcGRldihkZXYpOw0KPj4+Pj4NCj4+Pj4+IFRoaXMgaHVu
ayBpc24ndCBuZWVkZWQsIGFzIHdpdGggYmFpbGluZyBvdXQgb2YgeGVuX3BjaWJrX3hlbmJ1c19y
ZWdpc3Rlcg0KPj4+Pj4gZWFybHkgd2lsbCByZXN1bHQgaW4geGVuX3BjaWJrX3hlbmJ1c19wcm9i
ZSBuZXZlciBiZWluZyBjYWxsZWQuDQo+Pj4+Pg0KPj4+Pj4+IMKgwqDCoMKgwqDCoMKgIGlmIChw
ZGV2ID09IE5VTEwpIHsNCj4+Pj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGVyciA9IC1FTk9N
RU07DQo+Pj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB4ZW5idXNfZGV2X2ZhdGFsKGRldiwg
ZXJyLA0KPj4+Pj4+IEBAIC03NDMsNiArNzQ4LDkgQEAgY29uc3Qgc3RydWN0IHhlbl9wY2lia19i
YWNrZW5kICpfX3JlYWRfbW9zdGx5IHhlbl9wY2lia19iYWNrZW5kOw0KPj4+Pj4+IMKgwqDCoCDC
oCBpbnQgX19pbml0IHhlbl9wY2lia194ZW5idXNfcmVnaXN0ZXIodm9pZCkNCj4+Pj4+PiDCoMKg
wqAgew0KPj4+Pj4+ICvCoMKgwqAgaWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+Pj4+Pj4gK8KgwqDC
oMKgwqDCoMKgIHJldHVybiAwOw0KPj4+Pj4+ICsNCj4+Pj4+DQo+Pj4+PiBVc2UgI2lmZGVmIENP
TkZJR19YODYgaW5zdGVhZC4NCj4+Pj4NCj4+Pj4gVGhlIHRpdGxlIG9mIHRoaXMgcGF0Y2ggc2F5
cyB0aGF0IHdlIHdhbnQgdG8gYWxsb3cgdGhpcyBkcml2ZXIgZm9yIG90aGVyIGFyY2hzDQo+Pj4+
IGFuZCBub3cgd2Ugd2FudCB0byBpbnRyb2R1Y2UgIiNpZmRlZiBDT05GSUdfWDg2IiB3aGljaCBk
b2Vzbid0IHNvdW5kDQo+Pj4+IHJpZ2h0IHdpdGggdGhhdCByZXNwZWN0LiBJbnN0ZWFkLCB3ZSBt
YXkgd2FudCBoYXZpbmcgc29tZXRoaW5nIGxpa2UgYQ0KPj4+PiBkZWRpY2F0ZWQgZ2F0ZSBmb3Ig
dGhpcywgZS5nLiAiI2lmZGVmIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkRfU1VQUF9QViINCj4+
Pj4gb3Igc29tZXRoaW5nIHdoaWNoIGlzIGFyY2hpdGVjdHVyZSBhZ25vc3RpYy4NCj4+Pg0KPj4+
IFNvbWV0aGluZyBsaWtlIHRoYXQsIHllcy4gQnV0IEknZCByYXRoZXIgdXNlIENPTkZJR19YRU5f
UENJREVWX0JBQ0tFTkQNCj4+PiBhY3RpbmcgYXMgdGhpcyBnYXRlIGFuZCBpbnRyb2R1Y2UgQ09O
RklHX1hFTl9QQ0lfU1RVQiBmb3IgdGhlIHN0dWINCj4+PiBmdW5jdGlvbmFsaXR5IG5lZWRlZCBv
biBBcm0uIFhFTl9QQ0lERVZfQkFDS0VORCB3b3VsZCBkZXBlbmQgb24gWDg2IGFuZA0KPj4+IHNl
bGVjdCBYRU5fUENJX1NUVUIsIHdoaWxlIG9uIEFybSBYRU5fUENJX1NUVUIgY291bGQgYmUgY29u
ZmlndXJlZCBpZg0KPj4+IHdhbnRlZC4gVGhlIHNwbGl0dGluZyBvZiB0aGUgZHJpdmVyIGNhbiBz
dGlsbCBiZSBkb25lIGxhdGVyLg0KPj4NCj4+IEhtLCBwY2liYWNrIGlzIG5vdyBjb21waWxlZCB3
aGVuIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkTCoCBpcyBlbmFibGVkDQo+PiBhbmQgd2Ugd2Fu
dCB0byBza2lwIHNvbWUgcGFydHMgb2YgaXRzIGNvZGUgd2hlbiBDT05GSUdfWEVOX1BDSV9TVFVC
IGlzIHNldC4NCj4+IFNvLCBJIGltYWdpbmUgdGhhdCBmb3IgeDg2IHdlIGp1c3QgZW5hYmxlIENP
TkZJR19YRU5fUENJREVWX0JBQ0tFTkQgYW5kIHRoZQ0KPj4gZHJpdmVyIGNvbXBpbGVzIGluIGl0
cyBjdXJyZW50IHN0YXRlLiBGb3IgQXJtIHdlIGVuYWJsZSBib3RoIENPTkZJR19YRU5fUENJREVW
X0JBQ0tFTkQNCj4+IGFuZCBDT05GSUdfWEVOX1BDSV9TVFVCLCBzbyBwYXJ0IG9mIHRoZSBkcml2
ZXIgaXMgbm90IGNvbXBpbGVkLg0KPg0KPiBObywgSSdkIHJhdGhlciBzd2l0Y2ggdG8gY29tcGls
aW5nIHhlbi1wY2liYWNrIHdoZW4gQ09ORklHX1hFTl9QQ0lfU1RVQg0KPiBpcyBzZXQgYW5kIGNv
bXBpbGUgb25seSBwYXJ0cyBvZiBpdCB3aGVuIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQgaXMN
Cj4gbm90IHNldCAodGhpcyB3aWxsIGJlIHRoZSBjYXNlIG9uIEFybSkuDQoNCkJ1dCB0aGlzIHdp
bGwgcmVxdWlyZSB0aGF0IHRoZSBleGlzdGluZyBrZXJuZWwgY29uZmlndXJhdGlvbnMgb3V0IHRo
ZXJlIGhhdmUgdG8gYWRkaXRpb25hbGx5DQoNCmRlZmluZSBDT05GSUdfWEVOX1BDSV9TVFVCIHRv
IGdldCB3aGF0IHRoZXkgaGFkIGJlZm9yZSB3aXRoIHNpbXBseSBlbmFibGluZw0KDQpDT05GSUdf
WEVOX1BDSURFVl9CQUNLRU5ELiBNeSBwb2ludCB3YXMgdGhhdCBpdCBpcyBwcm9iYWJseSBkZXNp
cmFibGUgbm90IHRvIGJyZWFrDQoNCnRoZSB0aGluZ3Mgd2hpbGUgZG9pbmcgdGhlIHNwbGl0L3Jl
LXdvcmsuDQoNCkkgYWxzbyB0aG91Z2h0IHRoYXQgImNvbXBpbGUgb25seSBwYXJ0cyBvZiBpdCB3
aGVuIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQgaXMgbm90IHNldCINCg0KbWF5IGhhdmUgbW9y
ZSBjb2RlIGdhdGVkIHJhdGhlciB0aGFuIHdpdGggZ2F0aW5nIHVud2FudGVkIGNvZGUgd2l0aCBD
T05GSUdfWEVOX1BDSV9TVFVCLg0KDQpJIGFtIG5vdCBxdWl0ZSBzdXJlIGFib3V0IHRoaXMgdGhv
dWdoLg0KDQo+DQo+IFRoaXMgaXMgYW5vdGhlciBzdGVwIGluIHRoZSByaWdodCBkaXJlY3Rpb24g
cHJlcGFyaW5nIHRoZSBzcGxpdC4NCj4NCj4gQnV0IGFzIHNhaWQgYmVmb3JlLCB0aGlzIGlzIG5v
dCBhIHJlcXVpcmVtZW50IGJ5IG1lIHRvIHRha2UgeW91ciBwYXRjaC4NClRoYW5rIHlvdQ0KPg0K
Pg0KPiBKdWVyZ2Vu


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:02:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:02:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191374.341361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZnT-0007vA-3o; Tue, 21 Sep 2021 07:02:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191374.341361; Tue, 21 Sep 2021 07:02:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZnT-0007v3-0S; Tue, 21 Sep 2021 07:02:31 +0000
Received: by outflank-mailman (input) for mailman id 191374;
 Tue, 21 Sep 2021 07:02:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSZnR-0007ut-CR
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:02:29 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e1b44f86-1aa9-11ec-b89e-12813bfff9fa;
 Tue, 21 Sep 2021 07:02:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C24531FD60;
 Tue, 21 Sep 2021 07:02:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7949113B97;
 Tue, 21 Sep 2021 07:02:27 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id tMNXHIODSWHLegAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 07:02:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1b44f86-1aa9-11ec-b89e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632207747; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=7thuIDnGv5oZ6BTCP+zSuwIv02tmpVWBhc2C3gScL0A=;
	b=jAdmKx0bi/GxFtRD1x8hHCZEDyOQ/cvhk5YcWnckIWx43Mk91SCNdODfzpafnCcTY+64Og
	vWaYMT7ZXT1Bn1xhQ/qUlHk1gIqL1CrkEwp8TVH8oVO0BWYJ+hsBQRAjLikxcKNAX3wtvH
	O9djHthfpfICEmgIIifKGpYoHOBe8xQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH] x86/xen: remove unneeded preempt_disable() from xen_irq_enable()
Date: Tue, 21 Sep 2021 09:02:26 +0200
Message-Id: <20210921070226.32021-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Disabling preemption in xen_irq_enable() is not needed. There is no
risk of missing events due to preemption, as preemption can happen
only in case an event is being received, which is just the opposite
of missing an event.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/irq.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)

diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c
index dfa091d79c2e..ba9b14a97109 100644
--- a/arch/x86/xen/irq.c
+++ b/arch/x86/xen/irq.c
@@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
 {
 	struct vcpu_info *vcpu;
 
-	/*
-	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
-	 * cleared, so disable preemption to ensure we check for
-	 * events on the VCPU we are still running on.
-	 */
-	preempt_disable();
-
 	vcpu = this_cpu_read(xen_vcpu);
 	vcpu->evtchn_upcall_mask = 0;
 
-	/* Doesn't matter if we get preempted here, because any
-	   pending event will get dealt with anyway. */
+	/*
+	 * Now preemption could happen, but this is only possible if an event
+	 * was handled, so missing an event due to preemption is not
+	 * possible at all.
+	 * The worst possible case is to be preempted and then check events
+	 * pending on the old vcpu, but this is not problematic.
+	 */
 
 	barrier(); /* unmask then check (avoid races) */
 	if (unlikely(vcpu->evtchn_upcall_pending))
 		xen_force_evtchn_callback();
-
-	preempt_enable();
 }
 PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable);
 
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:09:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:09:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191379.341371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZu1-0000LO-Rq; Tue, 21 Sep 2021 07:09:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191379.341371; Tue, 21 Sep 2021 07:09:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZu1-0000LH-Og; Tue, 21 Sep 2021 07:09:17 +0000
Received: by outflank-mailman (input) for mailman id 191379;
 Tue, 21 Sep 2021 07:09:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSZtz-0000LB-U4
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:09:15 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d3953202-1aaa-11ec-b89e-12813bfff9fa;
 Tue, 21 Sep 2021 07:09:14 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8B3CB220E2;
 Tue, 21 Sep 2021 07:09:13 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 41B2B13B97;
 Tue, 21 Sep 2021 07:09:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 8aTBDhmFSWFRfQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 07:09:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3953202-1aaa-11ec-b89e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632208153; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=NFGnYauoZVnKeDnza/GotQs/rD+/zDVxEoO8ELpY84c=;
	b=p2bsao8AkBXvRYuNGYYHVxz3c45T5B2FpFbYCFPP9DiUfiU7efiqAKij99/tZgwtYjdd80
	pofwGfOhaCvVrWYcEtHlQe7CXwSuc54oBwdFkIOGL5tk3sea/KNn0lv4AIitU9ucsHiq8K
	fXpzc0G30XmOFd4Wad539lIY2rtVWDc=
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
 <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Message-ID: <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com>
Date: Tue, 21 Sep 2021 09:09:12 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="gLQExX6LE8n8XrYC4Tp16QR4nWR6z0Lu8"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--gLQExX6LE8n8XrYC4Tp16QR4nWR6z0Lu8
Content-Type: multipart/mixed; boundary="c6BTbCTwWNpsYNT0QhNxqruPTgCxDXXHQ";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
 <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
In-Reply-To: <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>

--c6BTbCTwWNpsYNT0QhNxqruPTgCxDXXHQ
Content-Type: multipart/mixed;
 boundary="------------C8E28F828DBD3E67C23C44A3"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------C8E28F828DBD3E67C23C44A3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 09:00, Oleksandr Andrushchenko wrote:
>=20
> On 21.09.21 09:49, Juergen Gross wrote:
>> On 21.09.21 08:38, Oleksandr Andrushchenko wrote:
>>>
>>> On 21.09.21 09:07, Juergen Gross wrote:
>>>> On 21.09.21 07:51, Oleksandr Andrushchenko wrote:
>>>>>
>>>>> On 21.09.21 08:20, Juergen Gross wrote:
>>>>>> On 21.09.21 01:16, Stefano Stabellini wrote:
>>>>>>> On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
>>>>>>>> On 20.09.21 14:30, Juergen Gross wrote:
>>>>>>>>> On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
>>>>>>>>>> Hello, Stefano!
>>>>>>>>>>
>>>>>>>>>> On 18.09.21 00:45, Stefano Stabellini wrote:
>>>>>>>>>>> Hi Oleksandr,
>>>>>>>>>>>
>>>>>>>>>>> Why do you want to enable pciback on ARM? Is it only to "disa=
ble" a PCI
>>>>>>>>>>> device in Dom0 so that it can be safely assigned to a DomU?
>>>>>>>>>> Not only that
>>>>>>>>>>>
>>>>>>>>>>> I am asking because actually I don't think we want to enable =
the PV PCI
>>>>>>>>>>> backend feature of pciback on ARM, right? That would clash wi=
th the PCI
>>>>>>>>>>> assignment work you have been doing in Xen. They couldn't bot=
h work at
>>>>>>>>>>> the same time.
>>>>>>>>>> Correct, it is not used
>>>>>>>>>>>
>>>>>>>>>>> If we only need pciback to "park" a device in Dom0, wouldn't =
it be
>>>>>>>>>>> possible and better to use pci-stub instead?
>>>>>>>>>>
>>>>>>>>>> Not only that, so pci-stub is not enough
>>>>>>>>>>
>>>>>>>>>> The functionality which is implemented by the pciback and the =
toolstack
>>>>>>>>>> and which is relevant/missing/needed for ARM:
>>>>>>>>>>
>>>>>>>>>> 1. pciback is used as a database for assignable PCI devices, e=
=2Eg. xl
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 pci-assignable-{add|rem=
ove|list} manipulates that list. So, whenever the
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 toolstack needs to know=
 which PCI devices can be passed through it reads
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 that from the relevant =
sysfs entries of the pciback.
>>>>>>>>>>
>>>>>>>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when =
passing through
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 a PCI device it needs t=
o be unbound from the relevant device driver and bound
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 to pciback (strictly sp=
eaking it is not required that the device is bound to
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 pciback, but pciback is=
 again used as a database of the passed through PCI
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 devices, so we can re-b=
ind the devices back to their original drivers when
>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 guest domain shuts down=
)
>>>>>>>>>>
>>>>>>>>>> 3. Device reset
>>>>>>>>>>
>>>>>>>>>> We have previously discussed on xen-devel ML possible solution=
s to that as from the
>>>>>>>>>> above we see that pciback functionality is going to be only pa=
rtially used on Arm.
>>>>>>>>>>
>>>>>>>>>> Please see [1] and [2]:
>>>>>>>>>>
>>>>>>>>>> 1. It is not acceptable to manage the assignable list in Xen i=
tself
>>>>>>>>>>
>>>>>>>>>> 2. pciback can be split into two parts: PCI assignable/bind/re=
set handling and
>>>>>>>>>> the rest like vPCI etc.
>>>>>>>>>>
>>>>>>>>>> 3. pcifront is not used on Arm
>>>>>>>>>
>>>>>>>>> It is neither in x86 PVH/HVM guests.
>>>>>>>> Didn't know that, thank you for pointing
>>>>>>>>>
>>>>>>>>>> So, limited use of the pciback is one of the bricks used to en=
able PCI passthrough
>>>>>>>>>> on Arm. It was enough to just re-structure the driver and have=
 it run on Arm to achieve
>>>>>>>>>> all the goals above.
>>>>>>>>>>
>>>>>>>>>> If we still think it is desirable to break the pciback driver =
into "common" and "pcifront specific"
>>>>>>>>>> parts then it can be done, yet the patch is going to be the ve=
ry first brick in that building.
>>>>>>>>>
>>>>>>>>> Doing this split should be done, as the pcifront specific part =
could be
>>>>>>>>> omitted on x86, too, in case no PV guests using PCI passthrough=
 have to
>>>>>>>>> be supported.
>>>>>>>> Agree, that the final solution should have the driver split
>>>>>>>>>
>>>>>>>>>> So, I think this patch is still going to be needed besides whi=
ch direction we take.
>>>>>>>>>
>>>>>>>>> Some kind of this patch, yes. It might look different in case t=
he split
>>>>>>>>> is done first.
>>>>>>>>>
>>>>>>>>> I don't mind doing it in either sequence.
>>>>>>>>>
>>>>>>>> With this patch we have Arm on the same page as the above mentio=
ned x86 guests,
>>>>>>>>
>>>>>>>> e.g. the driver has unused code, but yet allows Arm to function =
now.
>>>>>>>>
>>>>>>>> At this stage of PCI passthrough on Arm it is yet enough. Long t=
erm, when
>>>>>>>>
>>>>>>>> the driver gets split, Arm will benefit from that split too, but=
 unfortunately I do not
>>>>>>>>
>>>>>>>> have enough bandwidth for that piece of work at the moment.
>>>>>>>
>>>>>>> That's fair and I don't want to scope-creep this simple patch ask=
ing for
>>>>>>> an enormous rework. At the same time I don't think we should enab=
le the
>>>>>>> whole of pciback on ARM because it would be erroneous and confusi=
ng.
>>>>>
>>>>> As the first stage before the driver is split or ifdef's used - can=
 we take the patch
>>>>> as is now? In either way we chose this needs to be done, e.g. enabl=
e compiling
>>>>> for other architectures and common code move.
>>>>
>>>> Fine with me in principle. I need to take a more thorough look
>>>> at the patch, though.
>>> Of course
>>>>
>>>>>
>>>>>>>
>>>>>>> I am wonder if there is a simple:
>>>>>>>
>>>>>>> if (!xen_pv_domain())
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return;
>>>>>>>
>>>>>>> That we could add in a couple of places in pciback to stop it fro=
m
>>>>>>> initializing the parts we don't care about. Something along these=
 lines
>>>>>>> (untested and probably incomplete).
>>>>>>>
>>>>>>> What do you guys think?
>>>>>>
>>>>>> Uh no, not in this way, please. This will kill pci passthrough on =
x86
>>>>>> with dom0 running as PVH. I don't think this is working right now,=
 but
>>>>>> adding more code making it even harder to work should be avoided.
>>>>>>
>>>>>>> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-p=
ciback/xenbus.c
>>>>>>> index da34ce85dc88..991ba0a9b359 100644
>>>>>>> --- a/drivers/xen/xen-pciback/xenbus.c
>>>>>>> +++ b/drivers/xen/xen-pciback/xenbus.c
>>>>>>> @@ -15,6 +15,7 @@
>>>>>>>  =C2=A0=C2=A0=C2=A0 #include <xen/xenbus.h>
>>>>>>>  =C2=A0=C2=A0=C2=A0 #include <xen/events.h>
>>>>>>>  =C2=A0=C2=A0=C2=A0 #include <xen/pci.h>
>>>>>>> +#include <xen/xen.h>
>>>>>>>  =C2=A0=C2=A0=C2=A0 #include "pciback.h"
>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0 #define INVALID_EVTCHN_IRQ=C2=A0 (-1)
>>>>>>> @@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xen=
bus_device *dev,
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const struct xenbus_devi=
ce_id *id)
>>>>>>>  =C2=A0=C2=A0=C2=A0 {
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int err =3D 0;
>>>>>>> -=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev =3D alloc_pdev(=
dev);
>>>>>>> +=C2=A0=C2=A0=C2=A0 struct xen_pcibk_device *pdev;
>>>>>>> +
>>>>>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>>>>>>  =C2=A0=C2=A0=C2=A0 +=C2=A0=C2=A0=C2=A0 pdev =3D alloc_pdev(dev);=

>>>>>>
>>>>>> This hunk isn't needed, as with bailing out of xen_pcibk_xenbus_re=
gister
>>>>>> early will result in xen_pcibk_xenbus_probe never being called.
>>>>>>
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (pdev =3D=3D NULL)=
 {
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 err =3D -ENOMEM;
>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 xenbus_dev_fatal(dev, err,
>>>>>>> @@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly=
 xen_pcibk_backend;
>>>>>>>  =C2=A0=C2=A0=C2=A0 =C2=A0 int __init xen_pcibk_xenbus_register(v=
oid)
>>>>>>>  =C2=A0=C2=A0=C2=A0 {
>>>>>>> +=C2=A0=C2=A0=C2=A0 if (!xen_pv_domain())
>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0;
>>>>>>> +
>>>>>>
>>>>>> Use #ifdef CONFIG_X86 instead.
>>>>>
>>>>> The title of this patch says that we want to allow this driver for =
other archs
>>>>> and now we want to introduce "#ifdef CONFIG_X86" which doesn't soun=
d
>>>>> right with that respect. Instead, we may want having something like=
 a
>>>>> dedicated gate for this, e.g. "#ifdef CONFIG_XEN_PCIDEV_BACKEND_SUP=
P_PV"
>>>>> or something which is architecture agnostic.
>>>>
>>>> Something like that, yes. But I'd rather use CONFIG_XEN_PCIDEV_BACKE=
ND
>>>> acting as this gate and introduce CONFIG_XEN_PCI_STUB for the stub
>>>> functionality needed on Arm. XEN_PCIDEV_BACKEND would depend on X86 =
and
>>>> select XEN_PCI_STUB, while on Arm XEN_PCI_STUB could be configured i=
f
>>>> wanted. The splitting of the driver can still be done later.
>>>
>>> Hm, pciback is now compiled when CONFIG_XEN_PCIDEV_BACKEND=C2=A0 is e=
nabled
>>> and we want to skip some parts of its code when CONFIG_XEN_PCI_STUB i=
s set.
>>> So, I imagine that for x86 we just enable CONFIG_XEN_PCIDEV_BACKEND a=
nd the
>>> driver compiles in its current state. For Arm we enable both CONFIG_X=
EN_PCIDEV_BACKEND
>>> and CONFIG_XEN_PCI_STUB, so part of the driver is not compiled.
>>
>> No, I'd rather switch to compiling xen-pciback when CONFIG_XEN_PCI_STU=
B
>> is set and compile only parts of it when CONFIG_XEN_PCIDEV_BACKEND is
>> not set (this will be the case on Arm).
>=20
> But this will require that the existing kernel configurations out there=
 have to additionally=20
> define CONFIG_XEN_PCI_STUB to get what they had before with simply enab=
ling=20
> CONFIG_XEN_PCIDEV_BACKEND. My point was that it is probably desirable n=
ot to break
> the things while doing the split/re-work.

By letting XEN_PCIDEV_BACKEND select XEN_PCI_STUB this won't happen.

> I also thought that "compile only parts of it when CONFIG_XEN_PCIDEV_BA=
CKEND is not set"
> may have more code gated rather than with gating unwanted code with CON=
FIG_XEN_PCI_STUB.
> I am not quite sure about this though.

This would be a very weird semantics of XEN_PCI_STUB, as the stub part
is needed on X86 and on Arm.

Gating could even be done with Stefano's patch just by replacing his
"!xen_pv_domain()" tests with "!IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND)".


Juergen

--------------C8E28F828DBD3E67C23C44A3
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------C8E28F828DBD3E67C23C44A3--

--c6BTbCTwWNpsYNT0QhNxqruPTgCxDXXHQ--

--gLQExX6LE8n8XrYC4Tp16QR4nWR6z0Lu8
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJhRgFAwAAAAAACgkQsN6d1ii/Ey+x
1Af8CI2CUTWu83OFQy9/F1W5GVgM4CKCqlQuW0ueF5VNpGE2pCyviSCK262OwuA6kTmxEtQnhBWG
mK3jPiOPux6UsdXRc04rzcVinU+OiLa8Hmg2HT8u6hujp7nafTYrzzHDEkMgknuCaJ6PH7yPDxsl
eQE54xzLeJSBlodGnrecaK1nqXUEaOBgo32vnXfvGpvYAt/7f1sZWW0i/syivzTJVa/Gj7ugWv5R
9iKAnG6GV23FhR0/b8dQRdKUsXAHz9SN0LIEeUzydAfEvjvSylkc/E94cMDKc+rbedz0x7lipV4l
l7dkiYusgU6lcr5KSy4LwZyG3v782BOatJr42N9fzQ==
=Y366
-----END PGP SIGNATURE-----

--gLQExX6LE8n8XrYC4Tp16QR4nWR6z0Lu8--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:15:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:15:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191387.341382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZzk-0001ny-M3; Tue, 21 Sep 2021 07:15:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191387.341382; Tue, 21 Sep 2021 07:15:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSZzk-0001nr-IQ; Tue, 21 Sep 2021 07:15:12 +0000
Received: by outflank-mailman (input) for mailman id 191387;
 Tue, 21 Sep 2021 07:15:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSZzj-0001nl-Ob
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:15:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a7912a0d-1aab-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:15:10 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-2Bzb933yMwO-WOt8yPouNw-1; Tue, 21 Sep 2021 09:15:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Tue, 21 Sep
 2021 07:15:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:15:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Tue, 21 Sep 2021 07:15:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7912a0d-1aab-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208509;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=/tLyqZXftS9tJZQtKl3aFz3f9UuO7C+fVECXMxxqC6Q=;
	b=Ya65qKbXnaxtYDfCXltEjdM6lpQ2NeF6rKJDc9WM3dxSikIL7+sah+yM0V2yrSa5yb0/lc
	lrd68KTj/HfYpLklkp5fGAW55IbzUvUWymEfh4mRgg98UPfhKme1pJ+T7qW4GZmP5pTuNX
	NWUAu9YdaueMYP2As4caVxbiQ6/wOgM=
X-MC-Unique: 2Bzb933yMwO-WOt8yPouNw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f3aA2sgRvuEP1yOoaeXz3W3Qge2ftuWfDAh9jlD6Pzb5+ofn+7afOhb26pYKIBVhLQgpzKn4bUv7CjV7b6ijCKnGOr8fI3UPGrAsr3LuFxOBtYW5hrmnd0k/rKY6XPLlk0u71LpIjZhDFFVwy4ScldA25P1J0OoP4ydq+ocIpBJIWzOjbLrtld5N9CN61LXzJAkh3cm5i22B77fzyyl/R9rug7trJad9NOJE7A/Z+II8UFPcazGaM3Tfci/Wzwxqhdl8DDTB82bPaACvPiJnYGiR7Fd4Y8vt1i0/fNbTPOZwf8Mu98JLbLczmO6bwt6f9AOg9zObDwluTQEgb4tV1w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/tLyqZXftS9tJZQtKl3aFz3f9UuO7C+fVECXMxxqC6Q=;
 b=XbUR7dNYMYwpKCe34BopX+QvKCnU0WinPVo2a+1nsCpfXETciqjnq9cZ/pRq3kEsx20fgw8PVrKAM4R8tG5C+THkCbtzxu63NkiJkW/rFScNUcQ6vjqwsPMJL8/H0sjzQQbu+mYdCZOfBrPUo/HfKWQhQ+jtxfrGah0W/xZam9LBqNvM9lGjuygta05p5GKq5jd+FCVyWs0yXS6NJl7yTrJmVDgiodEjqzlSIQM4iZLZ0LHPwm+WnWb8xCexCnEMXaTrkXO+c4iKXUHAQZVEwOdERmQPFKZ314LP9EYZ4bwlLdzzLqTSX9c44VlhlnIM/ivWmUGSkV41vGzT1WcOMw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v3 0/9] x86/PVH: Dom0 building adjustments
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Date: Tue, 21 Sep 2021 09:15:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1bd167a8-45eb-43a3-0130-08d97ccf89cf
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2445DD1BA69D3EC921A67629B3A19@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2OFzrQnmQi5L4qj/bSO9OkpwHpQkEGEj+KP9BofSgnDX51o2NzQHjshSZOks1Q3rVuVussMYb5irpHsOxZn5BC6W7jNqBZjxp9Md/ju7JgJ9I3R0gaA3t2goWwn0TYxs5JOvMja4EBa1xt2CPmubVkt4Wy3lUWlEUSeOMu9h7Bryi2A+mZcWGzknJZ7bbCtewUPyLgJDFF9k7sIu31GJaH5FRRKYjeil/F5VMEVahrAb558B6Jigr9Emj721uXt/CSOPkPCoQSISWDTcI/Wp2hxbk2RJWRApPY4HOrV9wzgwuufPY39Lv/D10U8o1uVcatdctz3zdIyqzB/u/+viR2MllcNiQtvWh8SWochiU6/BbgnuIpMDQ+Ek/B39+vftU4OG3xZ72km3tba61pNBVmRmVUZs9PF04qWYKoUGlzkHmjdPjSfgRHlXwQ4SDsQEwIdJw3XYYQMJcv42PtLKp/QQhGM5kKN2BKLr1JsSMk5XXc2wTOQouvftO4J3JseDZZdQVp8kAQE+nBzeTmjEdDfZTRYWniSRXXfNtAUTXhQLQzmRVUUXkzmEs3B8ZxriESyWJ8wX3ra81l9VY/th4w7f/NzndL28peTICUwolsr3A8nz+jktZmtWWNb+DqVnXCmP+lv4tAQMjImTGUt4LLEwYjs6w50Pds8RAGRLVoKa+tqFJAUn0GwpOqJVLhBAUcczYIgFHmcM2ktzr5LMkS3bEIfIP5xpQv+N4hrKwPE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(8676002)(186003)(54906003)(36756003)(31686004)(2906002)(16576012)(316002)(956004)(8936002)(38100700002)(2616005)(6486002)(31696002)(66556008)(66476007)(66946007)(5660300002)(4326008)(6916009)(26005)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?b0hzZElUYmNSaTVLNTVGT2RFRXZsdGQwZzZuZVlzd1cxY0haTTlMRmlaZ3Bk?=
 =?utf-8?B?bmNueGhMQi9XNE85YlJzZ21HQmpWTWtDMHZ1Q3A2QlRHaUtrNWQwMEdHeDRk?=
 =?utf-8?B?SzZwTUIycFJVZVRjVzEzV2x0RXYyRlljRjM3Qkg2ak8xVzB4MnUxL2JCT2c4?=
 =?utf-8?B?ZnBIYS9EL3dZaVhqR3ptQnp0V25wTUtqM09KbjNVbmdqYXV4OGNFbXZpNkRa?=
 =?utf-8?B?czRac1JtQU9ROTRBbDJsbEhxNDljWXpPRXg0cm9sUlBUVXBwMWNQcjd5cVlp?=
 =?utf-8?B?bHJZN1ptQnNpZ0s3S0VTVEp4VzFQK0dvNnRPNnpzUVN2Y0pVZkxYbmgzYkx0?=
 =?utf-8?B?OW4yQ1B0eDM3TDlOc1JLNG0vMXM3Y1pIVG9vMDB2QzI2WmFnY0xQajg4WURQ?=
 =?utf-8?B?STVMWm16cWRWQWg4bWNDYlJoTmNqeUxSdE0xR0NxOUFaS1puTkkvTmVGT3BV?=
 =?utf-8?B?ZmlVcGN2cjU2TUt5NW92SzRnYzJMWWQxUnl1Z25ZR2w2N24xRE5MVWh5bGVo?=
 =?utf-8?B?UDJUNE9iQ3hGYytDQ1NsNXAwSFZOdVh3NC90ZE41STNaZXp5NDB1MXNqNjht?=
 =?utf-8?B?ZWJCeTNhaitNRHV5UUtYQ2oyMEp5bEduVFRYV2syUElOTWZ4VmZkajk4TjlL?=
 =?utf-8?B?SFNLNzV5VXdrNVVMam4rWEtyTjdJR3kzL2FCZGJsWjNpYlRtL2JUMCtWeU5Q?=
 =?utf-8?B?RkplcFBucFhSVU5KSjdodm1LbXE0YlF4UzVxenlaMjZDRVZwVTZscUlJOGJ1?=
 =?utf-8?B?TWFVT2JNRVl3OWFSTysrbHl4YytkREIrdFZQQ1ZmWXZCNitBQ3c1UDJ5c1ZY?=
 =?utf-8?B?Q0NCaEpHSzd4bHhkem45OXpGa1N6bUcrOXFhN1dvRzI0akpwMTVRQW5QNWNq?=
 =?utf-8?B?Q256RTYrYnExRVRFRDRHSitiaXFxVFcydSt5SXBEOGhudDYvc3VtbURPMFVv?=
 =?utf-8?B?eW55cHJUSGJNZ3haZ212ZkZUa2UyUVpDZ3UrWTJXVlYvYlBOc1p2VU9IN2FS?=
 =?utf-8?B?VzhPdjNmMk9TNVM2dEt5bk5DdnE5aVhFbEdacXByallBMHltZzJSRnArYmwx?=
 =?utf-8?B?cWlpVFJrNVNwQlQvTDBuY01iK2kxSEF3WFNUNnQwWml5Zy94OUZGUDNGZ0ph?=
 =?utf-8?B?S2ZndFNlcndhTmVxak9jUGZQKytmN1hvOE1BaFpkTGdyVEVGV1UvejFiQVls?=
 =?utf-8?B?aVo2WjJTTmJoa29qYlNEbytxOWI3UXovWUc5TDlaNUVoOWJETkhMWlZOSXRD?=
 =?utf-8?B?Wis1NHcrY0c4TDBLL0p6bVNqOXpVZUx5Y0J3Yk5aRGRvcE9UUzY4RGdLR2Zw?=
 =?utf-8?B?cWk3bTN0ODRSblRzTVFBNTlLajg5Ymphd09PSEpVc1lrU3VrL2pLYnR4aEJa?=
 =?utf-8?B?dDdiS1dadTlyZ25RM2l4U3RnMk9uOHk2R09KOEhzb25JL0g5bmZXVld6RGRq?=
 =?utf-8?B?K2IrTWk0WmlXNmFpdnZNc1gzaXNpcmprRWRTVXNDUXE3Y25mWXk4cUlmOTcw?=
 =?utf-8?B?ZWFXcWEyYUZYOFpVZW8vTTF3OW5rT3JVK0lRTXNiNSs4Rll6cGJzY1Z5VmVR?=
 =?utf-8?B?dEY5RVJBMXVKZFF4RjJtTm04Q2cwOVJ6RUJFZThyc25TYzk5eFlmS1IxUk9k?=
 =?utf-8?B?b0EvNktGUGI4MEtDUVVDc0U3M2VNN21Wai9rNWExZXgvK29ya2tDc1V1K2kw?=
 =?utf-8?B?bTNCbGp0bWM5NEVGY1Z3M2Q1bnJtRE9peTNDOEJVSEdQcWVOWTErK1dIVHBG?=
 =?utf-8?Q?rXAVcFcHDusMCCqGTBna5m3FFTJPDI8myHLvWnv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1bd167a8-45eb-43a3-0130-08d97ccf89cf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:15:06.3204
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: stekjAhzlDSFx5FqzNod9YT1TYGaRwxTjPNmr2ht3WbJhfxMho5aej17YXnqURtpXantBRyZDnO6/YVlM9IlRA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

In the course of fixing XSA-378 fallout I ran into a number of
issues, which this series is trying to deal with. The majority of
the changes is pretty much independent of one another.

There was another rather basic issue to fight first (patch
was submitted separately as RFC): vPCI wasn't aware of hidden PCI
devices, hitting an ASSERT(). Obviously I couldn't afford not
having a functioning serial console.

Compared to v2 there are a number of new changes here, and a
controversial one has been moved to the end. See individual
patches for details.

1: PVH: improve Dom0 memory size calculation
2: PV: properly set shadow allocation for Dom0
3: PVH: permit more physdevop-s to be used by Dom0
4: PVH: provide VGA console info to Dom0
5: PVH: actually show Dom0's register state from debug key '0'
6: HVM: convert hvm_virtual_to_linear_addr() to be remote-capable
7: PVH: actually show Dom0's stacks from debug key '0'
8: HVM: skip offline vCPU-s when dumping VMCBs/VMCSes
9: P2M: relax permissions of PVH Dom0's MMIO entries

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:16:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191393.341394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1L-0002Oq-1u; Tue, 21 Sep 2021 07:16:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191393.341394; Tue, 21 Sep 2021 07:16:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1K-0002Oj-UP; Tue, 21 Sep 2021 07:16:50 +0000
Received: by outflank-mailman (input) for mailman id 191393;
 Tue, 21 Sep 2021 07:16:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSa1J-0002OT-N6
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:16:49 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2c4ff113-124b-4573-b0d8-089ff8e54235;
 Tue, 21 Sep 2021 07:16:47 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L64cJ7007778; 
 Tue, 21 Sep 2021 07:16:46 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b77eyrfd5-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 07:16:45 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6321.eurprd03.prod.outlook.com (2603:10a6:20b:157::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 07:16:42 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:16:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c4ff113-124b-4573-b0d8-089ff8e54235
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SPy5yLK9udKP0Ah6JcDVi5Nv1x6cReUO6MVvOp4hg95h9dO0RBLXzR88pKwwhdjEUoMN1F3pEbYsdUUymLDjSdsm4qvra5ioauK8KunUkUwnWGzfXaNV9uJ2M3Syd/vLKE+4uswzr4LPUx7r3z+onjpF3A9hIPsszwZoSYPGC95j0eKq4fWMRlVRvQyNftfdWEoGZhY+RxBu5X69ibfYpeqdanpdrsxSo2swXWqE1DieSEg7Gc/MCiHr4uu3U/QXwDrfZErNc93qixB+Hxmns+EVrIs1X9U3xrivBIRhzVh1cGL9qQMo0dr6WYUbyIJQUY/S+O2aIThc6AJsVu6J+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PUnqXD6qhjjrblu8XRToW/AWwzlkv5LiSH5e4snrsKU=;
 b=HLboA4E5LssDb7n8V9P52WjCslN1Wi3RefLYpSsYqggvRE7KFTuCsQxpdD7B1jJIKI07qx2R+KUlaGk+IPfRO2vKQhgOXBLJ+IUi7/Ccv0WRKG5HOXep8wOkSEtzwbZxf/u/vegksP6iyAU9XS1ADfYjIPwS0JvVtszNxkSmIqTlm+wr1owbkq+9T+tNwxrjRWyBpp/awFGzpK+gTqfcu09twq17U1VTbJaVK+7gfoALH0xFYo9L26dNwct2NZ+oqJGN4KZ1tU2TBmCaRrLj6Px6ZncSz87hjdTDKtGZ/BYMlWK9PjnMn6MjQ83N01iH8zLe1+G/cHd2fsUIqYQz8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PUnqXD6qhjjrblu8XRToW/AWwzlkv5LiSH5e4snrsKU=;
 b=GZkgwTk55s0xTMXb8wM/hcP0hXx+H9/F9kKtkCAuAS+N6Z5dA0FHAuf/9iEEEUVwbYQtgSq+pk1D6zzya1CCFP3Qk4t0JFXW3q3si5P2GZaornEVCgsCGdHWWj72fUT5fAoPdZp355ZqszImp38FmEo1cTWso1Zi7U9zEXXtxBMvOXEmvaTMtSiteWZ9CHcC7sZtiOBWdYE8fWAgrYG2OBnxQXPsXjm8rn6YO9nBgiFnc83EUYyV+GWxo2m3KSW/1Du2Tcie99bBsXiCbysTNKOhTKoG3HvxTEijG8G/HlQueKC8CN1oQQXqStryYPTSBR/iB4pwthnElx91lJFiKw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAGXhAIAACLuAgAAEUwCAAAi6AIAAAxMAgAADDgCAAAJlAIAAAheA
Date: Tue, 21 Sep 2021 07:16:41 +0000
Message-ID: <48a2ef20-02ad-99e4-a8f5-fa144692aadc@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
 <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
 <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com>
In-Reply-To: <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 04de41f2-a368-4374-3b65-08d97ccfc2dd
x-ms-traffictypediagnostic: AM0PR03MB6321:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB632171C6FE23B2EB27636BE3E7A19@AM0PR03MB6321.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 CAKMH36E06jnyNYZK2xQ8UTdXATTtHAGsoQEUNkNnQAqmBSMITgcH8bYnZjzyWmkiV4XrDuhaYKdiPXtI2xMaxaMkNs/AIpuv61iry9BbTBqPfUbhGNqkm1J0rqnKd1k5IWcdQQDpi7gmrIDQ0mbbRsqNaSvbCRBkq19nDMSnST/PCTm2AvYOxfxT04X7agqlVYiQJD6x8nBkSlYrMzHVdxzQL/fSitPG1FVsY9VQU0lu9H6v1z2aw/9vvsXL7HA3f6bWSKqIM8DNHCsjvTK1jr8Ow/3CAhl5d4mRYiWyPiOiOJyHKVN3n9UZP4z9eNCsbiG+HNTuPbm07LnsU1ufyERl72zSk/OenY1sZ0IalforUL6DDDbsfWCImi1sPgvOdIR7xRiEZkQ5M+amdXUk+qvNry7Pfdr/wXdl5No1dTUSs3r9QTvjWWOaTCYAQJgPgb2m9LvdAwoeF3UVf/aO3X2mmLQGL2gXjWYBMNS3lqETjND+a14e690/qZxr/7v11dFf0ENxh2EEjI237Ws5yqvUt1+Q4pjtqyQo+lQc5iRXvQY2qVWak1sI+GPGMy0FpN5TpIvAf3+n4ArxZ3MQLNN4P+0jH9rFr3cb5eW0VK/XJV3POEkqefMxZBhM4hGfMoUZSVY0++HOlde6/sObCkX38/dSwip+UHdqfa1LoFQrniGXJcmXzs8s4df7lRQRzc/c8HFwwkPp08fzh0chSQDJQwZcF6M2TYP2p3XyxUr3biBRTEsX3FmsmU7jWDp
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(66946007)(66556008)(38100700002)(66476007)(66446008)(83380400001)(6486002)(6512007)(8676002)(71200400001)(122000001)(508600001)(86362001)(2906002)(36756003)(186003)(54906003)(53546011)(8936002)(31686004)(4326008)(2616005)(91956017)(76116006)(64756008)(110136005)(5660300002)(55236004)(6506007)(26005)(316002)(31696002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?bSs3SFlBYmZabEk4bTM3R1gwMHYwanJtMXVuUFgyVmd2VVRBdHRUVzQzZDdr?=
 =?utf-8?B?VTJMZGVnMXNsbzBGcEdaKzVaMzlXd0NHSG45aXBqU0l3OVUxbzBlbE00SzZo?=
 =?utf-8?B?alNNQ0Z6TitnWU5DbTJFckg2c2ZOaGtHcFFOb21ycnBQNmRDSis2MTdOUFRT?=
 =?utf-8?B?emRTRk9pYm43TlQvd2t2OVFkUVBqZitCMkR0aklpUm04V0Q2KzF3L3drTk5P?=
 =?utf-8?B?Mk04aVowMThyQWtxYmtlUkh4TlZrd3NzV2RaUnh5c0QwWXlqMW5OOHRUa2xa?=
 =?utf-8?B?dmxHZW5pZFY0ZGlTcjhvUGkyc0ZLRVVSTEhDam5wVjRnTEVjRVdHOEx2N2tW?=
 =?utf-8?B?TnBoVkwzd3RJY21hMnpqRm5vbm9SdkhPYjJ3blNQVXZ3NnFPUTRtbVFPMVJS?=
 =?utf-8?B?UGZwbkZ1UlREY3JNN05mWkV2c3JVUFBBYS9ldkluVXFtVGZma2p6N0U2Z096?=
 =?utf-8?B?WHo1ZjZIeW1ycGd0bmZFRVFQRHNERytMWVhaS21oMEpmdkFIQXdXOEIrYy9z?=
 =?utf-8?B?SGtEcng5YWdiU1NFWVVCWlREcUIyb2RaM1B3NWgxNlN5ckJkWTM1ZEdzS2dL?=
 =?utf-8?B?N09Dclh1QnNSMGxCelF3dG9xaTJaWjFwN1ErMGxubU9pZlRCVGZ3ZTJWVHds?=
 =?utf-8?B?cnFoQjB3MXFuU1pZUTVHMXJCWW42TXhCTFk2bDBIUmE2TzdkZllyRXQ0elZG?=
 =?utf-8?B?KzFwcDV3LzZ0K0t6cVV3dExPeW5qQlA2MTYweFJ0T2g0MnBkbzdJVG9uUGt3?=
 =?utf-8?B?TE91eDBvU2ZXekhldE1BT1hINlZIYUFRZ0NQdFJxTWJBUi9CMTFaSWJYK0dm?=
 =?utf-8?B?Ym1nSHMzaUMyeFFIclZ0RFdJejVkU2k4QkEwcm9JN0NLWXI4SmdKeUVjWWl0?=
 =?utf-8?B?N3h0endJME9iQ0ErVk5RSUkrRTdKUUdvWjM4SkY2Umx1Wksrb0xJYzV1aGlF?=
 =?utf-8?B?bGdsdDVlL2Z1MlVObDVkdDlIanM3RTEybm9XUGVoZGZyc2YrMXd2VkhLMFY5?=
 =?utf-8?B?NW5BTTk0SmI2ZXpVQUZNem1iK1hDY3FGUG5tQVQ5MFgwblc4YndXcFFBZ05y?=
 =?utf-8?B?UU5wcERGNzlTeUc2aVU2cm41WVVzU1cxU093SGZ2cU1aNkxYdzBIZ1NiOUl3?=
 =?utf-8?B?dnhhS3MxL3FlNlYvK2FvZUVLK21DVGxXM2tRdVBScFJYZUExeGpRWWtFaWdI?=
 =?utf-8?B?U3ZLZUhxOUl4UTY5ZEdLSDFiMW12am1qWjlNNTdwaEs1czArbkkvYzhkVDVs?=
 =?utf-8?B?ZXgrcFFZQXJLSTRaZ21TdG5JbmQzbkJab1lUcHBjNllHcVM0WHk4OS9nQUJn?=
 =?utf-8?B?WndJdlIyMnljbUJSeHlxZ1JnSys0aXc0SnJTK2lETHk0dmR3T0FMVmdrVTky?=
 =?utf-8?B?ZU1nQmJZRXU2MWFPNE9CU0hveUkxV1RjNHBYazQ2WU9xSE9VRHVlRXJtZlRv?=
 =?utf-8?B?Yy9YSDMwS0xPUFdVOEgvZWFtNktsc1B2VXUzczJCbmFFdEtIRTNHSXlkZTNm?=
 =?utf-8?B?bE1JQVBqVjNZUSs4ZnUvMDRJL2VOUjE0SDBxOStXNHQ3ak4wTjI5WE1ZbHBO?=
 =?utf-8?B?Z3FMdmtDQXljVlgrU3VSd05ibU1TR0duQVZtTE9tdDExNExZaTVIbHVacTRR?=
 =?utf-8?B?MGYvM2Via3BOYmVyd2t2MHNLdzB2eFp2bmxETzlzMTA5R2x4cjVlSzBzdXBP?=
 =?utf-8?B?ZnllTzIrKyt3QjdxZ29QM2VaSE9JSVpLc25VUTNPY1F6Ym1hU0c1bmdJaDFU?=
 =?utf-8?B?TkkzZGxaRHcrUENJWXY5TWtBYlFjN1Nyems4dHY2QUd0TmllaENnalEwNTlT?=
 =?utf-8?B?RmhrUmlYU0ZUVHNoZmQxQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <41BD324A72B6EB48B60386E84C772D28@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 04de41f2-a368-4374-3b65-08d97ccfc2dd
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 07:16:41.7993
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: CzGRwJ+j3PqjQ7D6GQx5LmeNMHfAEBHCyOZ8EHEsMX/I9lLGx28gPL/eLVrBcQvOrq4XJoP8g2rjQr6gwH3+LS8uZvnFlMgCEp0+2PApqjc5Jdp86smXWqXhj7rAea/g
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6321
X-Proofpoint-GUID: OckrIBv2PsVNcHx-9a35sc30Yf2SddhB
X-Proofpoint-ORIG-GUID: OckrIBv2PsVNcHx-9a35sc30Yf2SddhB
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0
 clxscore=1015 suspectscore=0 lowpriorityscore=0 mlxscore=0
 priorityscore=1501 impostorscore=0 mlxlogscore=999 adultscore=0
 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210047

DQpPbiAyMS4wOS4yMSAxMDowOSwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjEuMDkuMjEg
MDk6MDAsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDIxLjA5LjIx
IDA5OjQ5LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDIxLjA5LjIxIDA4OjM4LCBPbGVr
c2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4NCj4+Pj4gT24gMjEuMDkuMjEgMDk6MDcs
IEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4+PiBPbiAyMS4wOS4yMSAwNzo1MSwgT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+Pj4NCj4+Pj4+PiBPbiAyMS4wOS4yMSAwODoyMCwg
SnVlcmdlbiBHcm9zcyB3cm90ZToNCj4+Pj4+Pj4gT24gMjEuMDkuMjEgMDE6MTYsIFN0ZWZhbm8g
U3RhYmVsbGluaSB3cm90ZToNCj4+Pj4+Pj4+IE9uIE1vbiwgMjAgU2VwIDIwMjEsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+Pj4+Pj4+IE9uIDIwLjA5LjIxIDE0OjMwLCBKdWVy
Z2VuIEdyb3NzIHdyb3RlOg0KPj4+Pj4+Pj4+PiBPbiAyMC4wOS4yMSAwNzoyMywgT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+Pj4+Pj4+PiBIZWxsbywgU3RlZmFubyENCj4+Pj4+
Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBPbiAxOC4wOS4yMSAwMDo0NSwgU3RlZmFubyBTdGFiZWxsaW5p
IHdyb3RlOg0KPj4+Pj4+Pj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+
Pj4+Pj4+IFdoeSBkbyB5b3Ugd2FudCB0byBlbmFibGUgcGNpYmFjayBvbiBBUk0/IElzIGl0IG9u
bHkgdG8gImRpc2FibGUiIGEgUENJDQo+Pj4+Pj4+Pj4+Pj4gZGV2aWNlIGluIERvbTAgc28gdGhh
dCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRvIGEgRG9tVT8NCj4+Pj4+Pj4+Pj4+IE5vdCBv
bmx5IHRoYXQNCj4+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4+IEkgYW0gYXNraW5nIGJlY2F1c2Ug
YWN0dWFsbHkgSSBkb24ndCB0aGluayB3ZSB3YW50IHRvIGVuYWJsZSB0aGUgUFYgUENJDQo+Pj4+
Pj4+Pj4+Pj4gYmFja2VuZCBmZWF0dXJlIG9mIHBjaWJhY2sgb24gQVJNLCByaWdodD8gVGhhdCB3
b3VsZCBjbGFzaCB3aXRoIHRoZSBQQ0kNCj4+Pj4+Pj4+Pj4+PiBhc3NpZ25tZW50IHdvcmsgeW91
IGhhdmUgYmVlbiBkb2luZyBpbiBYZW4uIFRoZXkgY291bGRuJ3QgYm90aCB3b3JrIGF0DQo+Pj4+
Pj4+Pj4+Pj4gdGhlIHNhbWUgdGltZS4NCj4+Pj4+Pj4+Pj4+IENvcnJlY3QsIGl0IGlzIG5vdCB1
c2VkDQo+Pj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4+PiBJZiB3ZSBvbmx5IG5lZWQgcGNpYmFjayB0
byAicGFyayIgYSBkZXZpY2UgaW4gRG9tMCwgd291bGRuJ3QgaXQgYmUNCj4+Pj4+Pj4+Pj4+PiBw
b3NzaWJsZSBhbmQgYmV0dGVyIHRvIHVzZSBwY2ktc3R1YiBpbnN0ZWFkPw0KPj4+Pj4+Pj4+Pj4N
Cj4+Pj4+Pj4+Pj4+IE5vdCBvbmx5IHRoYXQsIHNvIHBjaS1zdHViIGlzIG5vdCBlbm91Z2gNCj4+
Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBUaGUgZnVuY3Rpb25hbGl0eSB3aGljaCBpcyBpbXBsZW1l
bnRlZCBieSB0aGUgcGNpYmFjayBhbmQgdGhlIHRvb2xzdGFjaw0KPj4+Pj4+Pj4+Pj4gYW5kIHdo
aWNoIGlzIHJlbGV2YW50L21pc3NpbmcvbmVlZGVkIGZvciBBUk06DQo+Pj4+Pj4+Pj4+Pg0KPj4+
Pj4+Pj4+Pj4gMS4gcGNpYmFjayBpcyB1c2VkIGFzIGEgZGF0YWJhc2UgZm9yIGFzc2lnbmFibGUg
UENJIGRldmljZXMsIGUuZy4geGwNCj4+Pj4+Pj4+Pj4+IMKgwqDCoMKgwqAgwqDCoCBwY2ktYXNz
aWduYWJsZS17YWRkfHJlbW92ZXxsaXN0fSBtYW5pcHVsYXRlcyB0aGF0IGxpc3QuIFNvLCB3aGVu
ZXZlciB0aGUNCj4+Pj4+Pj4+Pj4+IMKgwqDCoMKgwqAgwqDCoCB0b29sc3RhY2sgbmVlZHMgdG8g
a25vdyB3aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2ggaXQgcmVhZHMNCj4+
Pj4+Pj4+Pj4+IMKgwqDCoMKgwqAgwqDCoCB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVu
dHJpZXMgb2YgdGhlIHBjaWJhY2suDQo+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4gMi4gcGNpYmFj
ayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVuYm91bmQgUENJIGRldmljZXMsIGUuZy4gd2hlbiBwYXNz
aW5nIHRocm91Z2gNCj4+Pj4+Pj4+Pj4+IMKgwqDCoMKgwqAgwqDCoCBhIFBDSSBkZXZpY2UgaXQg
bmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFudCBkZXZpY2UgZHJpdmVyIGFuZCBi
b3VuZA0KPj4+Pj4+Pj4+Pj4gwqDCoMKgwqDCoCDCoMKgIHRvIHBjaWJhY2sgKHN0cmljdGx5IHNw
ZWFraW5nIGl0IGlzIG5vdCByZXF1aXJlZCB0aGF0IHRoZSBkZXZpY2UgaXMgYm91bmQgdG8NCj4+
Pj4+Pj4+Pj4+IMKgwqDCoMKgwqAgwqDCoCBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1
c2VkIGFzIGEgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSQ0KPj4+Pj4+Pj4+Pj4g
wqDCoMKgwqDCoCDCoMKgIGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5kIHRoZSBkZXZpY2VzIGJh
Y2sgdG8gdGhlaXIgb3JpZ2luYWwgZHJpdmVycyB3aGVuDQo+Pj4+Pj4+Pj4+PiDCoMKgwqDCoMKg
IMKgwqAgZ3Vlc3QgZG9tYWluIHNodXRzIGRvd24pDQo+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4g
My4gRGV2aWNlIHJlc2V0DQo+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4gV2UgaGF2ZSBwcmV2aW91
c2x5IGRpc2N1c3NlZCBvbiB4ZW4tZGV2ZWwgTUwgcG9zc2libGUgc29sdXRpb25zIHRvIHRoYXQg
YXMgZnJvbSB0aGUNCj4+Pj4+Pj4+Pj4+IGFib3ZlIHdlIHNlZSB0aGF0IHBjaWJhY2sgZnVuY3Rp
b25hbGl0eSBpcyBnb2luZyB0byBiZSBvbmx5IHBhcnRpYWxseSB1c2VkIG9uIEFybS4NCj4+Pj4+
Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBQbGVhc2Ugc2VlIFsxXSBhbmQgWzJdOg0KPj4+Pj4+Pj4+Pj4N
Cj4+Pj4+Pj4+Pj4+IDEuIEl0IGlzIG5vdCBhY2NlcHRhYmxlIHRvIG1hbmFnZSB0aGUgYXNzaWdu
YWJsZSBsaXN0IGluIFhlbiBpdHNlbGYNCj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiAyLiBwY2li
YWNrIGNhbiBiZSBzcGxpdCBpbnRvIHR3byBwYXJ0czogUENJIGFzc2lnbmFibGUvYmluZC9yZXNl
dCBoYW5kbGluZyBhbmQNCj4+Pj4+Pj4+Pj4+IHRoZSByZXN0IGxpa2UgdlBDSSBldGMuDQo+Pj4+
Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4gMy4gcGNpZnJvbnQgaXMgbm90IHVzZWQgb24gQXJtDQo+Pj4+
Pj4+Pj4+DQo+Pj4+Pj4+Pj4+IEl0IGlzIG5laXRoZXIgaW4geDg2IFBWSC9IVk0gZ3Vlc3RzLg0K
Pj4+Pj4+Pj4+IERpZG4ndCBrbm93IHRoYXQsIHRoYW5rIHlvdSBmb3IgcG9pbnRpbmcNCj4+Pj4+
Pj4+Pj4NCj4+Pj4+Pj4+Pj4+IFNvLCBsaW1pdGVkIHVzZSBvZiB0aGUgcGNpYmFjayBpcyBvbmUg
b2YgdGhlIGJyaWNrcyB1c2VkIHRvIGVuYWJsZSBQQ0kgcGFzc3Rocm91Z2gNCj4+Pj4+Pj4+Pj4+
IG9uIEFybS4gSXQgd2FzIGVub3VnaCB0byBqdXN0IHJlLXN0cnVjdHVyZSB0aGUgZHJpdmVyIGFu
ZCBoYXZlIGl0IHJ1biBvbiBBcm0gdG8gYWNoaWV2ZQ0KPj4+Pj4+Pj4+Pj4gYWxsIHRoZSBnb2Fs
cyBhYm92ZS4NCj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBJZiB3ZSBzdGlsbCB0aGluayBpdCBp
cyBkZXNpcmFibGUgdG8gYnJlYWsgdGhlIHBjaWJhY2sgZHJpdmVyIGludG8gImNvbW1vbiIgYW5k
ICJwY2lmcm9udCBzcGVjaWZpYyINCj4+Pj4+Pj4+Pj4+IHBhcnRzIHRoZW4gaXQgY2FuIGJlIGRv
bmUsIHlldCB0aGUgcGF0Y2ggaXMgZ29pbmcgdG8gYmUgdGhlIHZlcnkgZmlyc3QgYnJpY2sgaW4g
dGhhdCBidWlsZGluZy4NCj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4gRG9pbmcgdGhpcyBzcGxpdCBz
aG91bGQgYmUgZG9uZSwgYXMgdGhlIHBjaWZyb250IHNwZWNpZmljIHBhcnQgY291bGQgYmUNCj4+
Pj4+Pj4+Pj4gb21pdHRlZCBvbiB4ODYsIHRvbywgaW4gY2FzZSBubyBQViBndWVzdHMgdXNpbmcg
UENJIHBhc3N0aHJvdWdoIGhhdmUgdG8NCj4+Pj4+Pj4+Pj4gYmUgc3VwcG9ydGVkLg0KPj4+Pj4+
Pj4+IEFncmVlLCB0aGF0IHRoZSBmaW5hbCBzb2x1dGlvbiBzaG91bGQgaGF2ZSB0aGUgZHJpdmVy
IHNwbGl0DQo+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBTbywgSSB0aGluayB0aGlzIHBhdGNoIGlz
IHN0aWxsIGdvaW5nIHRvIGJlIG5lZWRlZCBiZXNpZGVzIHdoaWNoIGRpcmVjdGlvbiB3ZSB0YWtl
Lg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBTb21lIGtpbmQgb2YgdGhpcyBwYXRjaCwgeWVzLiBJ
dCBtaWdodCBsb29rIGRpZmZlcmVudCBpbiBjYXNlIHRoZSBzcGxpdA0KPj4+Pj4+Pj4+PiBpcyBk
b25lIGZpcnN0Lg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBJIGRvbid0IG1pbmQgZG9pbmcgaXQg
aW4gZWl0aGVyIHNlcXVlbmNlLg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IFdpdGggdGhpcyBwYXRj
aCB3ZSBoYXZlIEFybSBvbiB0aGUgc2FtZSBwYWdlIGFzIHRoZSBhYm92ZSBtZW50aW9uZWQgeDg2
IGd1ZXN0cywNCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IGUuZy4gdGhlIGRyaXZlciBoYXMgdW51c2Vk
IGNvZGUsIGJ1dCB5ZXQgYWxsb3dzIEFybSB0byBmdW5jdGlvbiBub3cuDQo+Pj4+Pj4+Pj4NCj4+
Pj4+Pj4+PiBBdCB0aGlzIHN0YWdlIG9mIFBDSSBwYXNzdGhyb3VnaCBvbiBBcm0gaXQgaXMgeWV0
IGVub3VnaC4gTG9uZyB0ZXJtLCB3aGVuDQo+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+PiB0aGUgZHJpdmVy
IGdldHMgc3BsaXQsIEFybSB3aWxsIGJlbmVmaXQgZnJvbSB0aGF0IHNwbGl0IHRvbywgYnV0IHVu
Zm9ydHVuYXRlbHkgSSBkbyBub3QNCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+IGhhdmUgZW5vdWdoIGJh
bmR3aWR0aCBmb3IgdGhhdCBwaWVjZSBvZiB3b3JrIGF0IHRoZSBtb21lbnQuDQo+Pj4+Pj4+Pg0K
Pj4+Pj4+Pj4gVGhhdCdzIGZhaXIgYW5kIEkgZG9uJ3Qgd2FudCB0byBzY29wZS1jcmVlcCB0aGlz
IHNpbXBsZSBwYXRjaCBhc2tpbmcgZm9yDQo+Pj4+Pj4+PiBhbiBlbm9ybW91cyByZXdvcmsuIEF0
IHRoZSBzYW1lIHRpbWUgSSBkb24ndCB0aGluayB3ZSBzaG91bGQgZW5hYmxlIHRoZQ0KPj4+Pj4+
Pj4gd2hvbGUgb2YgcGNpYmFjayBvbiBBUk0gYmVjYXVzZSBpdCB3b3VsZCBiZSBlcnJvbmVvdXMg
YW5kIGNvbmZ1c2luZy4NCj4+Pj4+Pg0KPj4+Pj4+IEFzIHRoZSBmaXJzdCBzdGFnZSBiZWZvcmUg
dGhlIGRyaXZlciBpcyBzcGxpdCBvciBpZmRlZidzIHVzZWQgLSBjYW4gd2UgdGFrZSB0aGUgcGF0
Y2gNCj4+Pj4+PiBhcyBpcyBub3c/IEluIGVpdGhlciB3YXkgd2UgY2hvc2UgdGhpcyBuZWVkcyB0
byBiZSBkb25lLCBlLmcuIGVuYWJsZSBjb21waWxpbmcNCj4+Pj4+PiBmb3Igb3RoZXIgYXJjaGl0
ZWN0dXJlcyBhbmQgY29tbW9uIGNvZGUgbW92ZS4NCj4+Pj4+DQo+Pj4+PiBGaW5lIHdpdGggbWUg
aW4gcHJpbmNpcGxlLiBJIG5lZWQgdG8gdGFrZSBhIG1vcmUgdGhvcm91Z2ggbG9vaw0KPj4+Pj4g
YXQgdGhlIHBhdGNoLCB0aG91Z2guDQo+Pj4+IE9mIGNvdXJzZQ0KPj4+Pj4NCj4+Pj4+Pg0KPj4+
Pj4+Pj4NCj4+Pj4+Pj4+IEkgYW0gd29uZGVyIGlmIHRoZXJlIGlzIGEgc2ltcGxlOg0KPj4+Pj4+
Pj4NCj4+Pj4+Pj4+IGlmICgheGVuX3B2X2RvbWFpbigpKQ0KPj4+Pj4+Pj4gwqDCoMKgwqDCoMKg
wqAgcmV0dXJuOw0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IFRoYXQgd2UgY291bGQgYWRkIGluIGEgY291
cGxlIG9mIHBsYWNlcyBpbiBwY2liYWNrIHRvIHN0b3AgaXQgZnJvbQ0KPj4+Pj4+Pj4gaW5pdGlh
bGl6aW5nIHRoZSBwYXJ0cyB3ZSBkb24ndCBjYXJlIGFib3V0LiBTb21ldGhpbmcgYWxvbmcgdGhl
c2UgbGluZXMNCj4+Pj4+Pj4+ICh1bnRlc3RlZCBhbmQgcHJvYmFibHkgaW5jb21wbGV0ZSkuDQo+
Pj4+Pj4+Pg0KPj4+Pj4+Pj4gV2hhdCBkbyB5b3UgZ3V5cyB0aGluaz8NCj4+Pj4+Pj4NCj4+Pj4+
Pj4gVWggbm8sIG5vdCBpbiB0aGlzIHdheSwgcGxlYXNlLiBUaGlzIHdpbGwga2lsbCBwY2kgcGFz
c3Rocm91Z2ggb24geDg2DQo+Pj4+Pj4+IHdpdGggZG9tMCBydW5uaW5nIGFzIFBWSC4gSSBkb24n
dCB0aGluayB0aGlzIGlzIHdvcmtpbmcgcmlnaHQgbm93LCBidXQNCj4+Pj4+Pj4gYWRkaW5nIG1v
cmUgY29kZSBtYWtpbmcgaXQgZXZlbiBoYXJkZXIgdG8gd29yayBzaG91bGQgYmUgYXZvaWRlZC4N
Cj4+Pj4+Pj4NCj4+Pj4+Pj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94
ZW5idXMuYyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+Pj4+Pj4+PiBpbmRl
eCBkYTM0Y2U4NWRjODguLjk5MWJhMGE5YjM1OSAxMDA2NDQNCj4+Pj4+Pj4+IC0tLSBhL2RyaXZl
cnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+Pj4+Pj4+PiArKysgYi9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4+Pj4+Pj4gQEAgLTE1LDYgKzE1LDcgQEANCj4+Pj4+Pj4+
IMKgwqDCoMKgICNpbmNsdWRlIDx4ZW4veGVuYnVzLmg+DQo+Pj4+Pj4+PiDCoMKgwqDCoCAjaW5j
bHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4+Pj4+Pj4gwqDCoMKgwqAgI2luY2x1ZGUgPHhlbi9wY2ku
aD4NCj4+Pj4+Pj4+ICsjaW5jbHVkZSA8eGVuL3hlbi5oPg0KPj4+Pj4+Pj4gwqDCoMKgwqAgI2lu
Y2x1ZGUgInBjaWJhY2suaCINCj4+Pj4+Pj4+IMKgwqDCoMKgIMKgICNkZWZpbmUgSU5WQUxJRF9F
VlRDSE5fSVJRwqAgKC0xKQ0KPj4+Pj4+Pj4gQEAgLTY4NSw4ICs2ODYsMTIgQEAgc3RhdGljIGlu
dCB4ZW5fcGNpYmtfeGVuYnVzX3Byb2JlKHN0cnVjdCB4ZW5idXNfZGV2aWNlICpkZXYsDQo+Pj4+
Pj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNvbnN0IHN0cnVj
dCB4ZW5idXNfZGV2aWNlX2lkICppZCkNCj4+Pj4+Pj4+IMKgwqDCoMKgIHsNCj4+Pj4+Pj4+IMKg
wqDCoMKgwqDCoMKgwqAgaW50IGVyciA9IDA7DQo+Pj4+Pj4+PiAtwqDCoMKgIHN0cnVjdCB4ZW5f
cGNpYmtfZGV2aWNlICpwZGV2ID0gYWxsb2NfcGRldihkZXYpOw0KPj4+Pj4+Pj4gK8KgwqDCoCBz
dHJ1Y3QgeGVuX3BjaWJrX2RldmljZSAqcGRldjsNCj4+Pj4+Pj4+ICsNCj4+Pj4+Pj4+ICvCoMKg
wqAgaWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+Pj4+Pj4+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJu
IDA7DQo+Pj4+Pj4+PiDCoMKgwqDCoCArwqDCoMKgIHBkZXYgPSBhbGxvY19wZGV2KGRldik7DQo+
Pj4+Pj4+DQo+Pj4+Pj4+IFRoaXMgaHVuayBpc24ndCBuZWVkZWQsIGFzIHdpdGggYmFpbGluZyBv
dXQgb2YgeGVuX3BjaWJrX3hlbmJ1c19yZWdpc3Rlcg0KPj4+Pj4+PiBlYXJseSB3aWxsIHJlc3Vs
dCBpbiB4ZW5fcGNpYmtfeGVuYnVzX3Byb2JlIG5ldmVyIGJlaW5nIGNhbGxlZC4NCj4+Pj4+Pj4N
Cj4+Pj4+Pj4+IMKgwqDCoMKgwqDCoMKgwqAgaWYgKHBkZXYgPT0gTlVMTCkgew0KPj4+Pj4+Pj4g
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGVyciA9IC1FTk9NRU07DQo+Pj4+Pj4+PiDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgeGVuYnVzX2Rldl9mYXRhbChkZXYsIGVyciwNCj4+Pj4+Pj4+IEBA
IC03NDMsNiArNzQ4LDkgQEAgY29uc3Qgc3RydWN0IHhlbl9wY2lia19iYWNrZW5kICpfX3JlYWRf
bW9zdGx5IHhlbl9wY2lia19iYWNrZW5kOw0KPj4+Pj4+Pj4gwqDCoMKgwqAgwqAgaW50IF9faW5p
dCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+Pj4+Pj4+PiDCoMKgwqDCoCB7DQo+
Pj4+Pj4+PiArwqDCoMKgIGlmICgheGVuX3B2X2RvbWFpbigpKQ0KPj4+Pj4+Pj4gK8KgwqDCoMKg
wqDCoMKgIHJldHVybiAwOw0KPj4+Pj4+Pj4gKw0KPj4+Pj4+Pg0KPj4+Pj4+PiBVc2UgI2lmZGVm
IENPTkZJR19YODYgaW5zdGVhZC4NCj4+Pj4+Pg0KPj4+Pj4+IFRoZSB0aXRsZSBvZiB0aGlzIHBh
dGNoIHNheXMgdGhhdCB3ZSB3YW50IHRvIGFsbG93IHRoaXMgZHJpdmVyIGZvciBvdGhlciBhcmNo
cw0KPj4+Pj4+IGFuZCBub3cgd2Ugd2FudCB0byBpbnRyb2R1Y2UgIiNpZmRlZiBDT05GSUdfWDg2
IiB3aGljaCBkb2Vzbid0IHNvdW5kDQo+Pj4+Pj4gcmlnaHQgd2l0aCB0aGF0IHJlc3BlY3QuIElu
c3RlYWQsIHdlIG1heSB3YW50IGhhdmluZyBzb21ldGhpbmcgbGlrZSBhDQo+Pj4+Pj4gZGVkaWNh
dGVkIGdhdGUgZm9yIHRoaXMsIGUuZy4gIiNpZmRlZiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5E
X1NVUFBfUFYiDQo+Pj4+Pj4gb3Igc29tZXRoaW5nIHdoaWNoIGlzIGFyY2hpdGVjdHVyZSBhZ25v
c3RpYy4NCj4+Pj4+DQo+Pj4+PiBTb21ldGhpbmcgbGlrZSB0aGF0LCB5ZXMuIEJ1dCBJJ2QgcmF0
aGVyIHVzZSBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EDQo+Pj4+PiBhY3RpbmcgYXMgdGhpcyBn
YXRlIGFuZCBpbnRyb2R1Y2UgQ09ORklHX1hFTl9QQ0lfU1RVQiBmb3IgdGhlIHN0dWINCj4+Pj4+
IGZ1bmN0aW9uYWxpdHkgbmVlZGVkIG9uIEFybS4gWEVOX1BDSURFVl9CQUNLRU5EIHdvdWxkIGRl
cGVuZCBvbiBYODYgYW5kDQo+Pj4+PiBzZWxlY3QgWEVOX1BDSV9TVFVCLCB3aGlsZSBvbiBBcm0g
WEVOX1BDSV9TVFVCIGNvdWxkIGJlIGNvbmZpZ3VyZWQgaWYNCj4+Pj4+IHdhbnRlZC4gVGhlIHNw
bGl0dGluZyBvZiB0aGUgZHJpdmVyIGNhbiBzdGlsbCBiZSBkb25lIGxhdGVyLg0KPj4+Pg0KPj4+
PiBIbSwgcGNpYmFjayBpcyBub3cgY29tcGlsZWQgd2hlbiBDT05GSUdfWEVOX1BDSURFVl9CQUNL
RU5EIGlzIGVuYWJsZWQNCj4+Pj4gYW5kIHdlIHdhbnQgdG8gc2tpcCBzb21lIHBhcnRzIG9mIGl0
cyBjb2RlIHdoZW4gQ09ORklHX1hFTl9QQ0lfU1RVQiBpcyBzZXQuDQo+Pj4+IFNvLCBJIGltYWdp
bmUgdGhhdCBmb3IgeDg2IHdlIGp1c3QgZW5hYmxlIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQg
YW5kIHRoZQ0KPj4+PiBkcml2ZXIgY29tcGlsZXMgaW4gaXRzIGN1cnJlbnQgc3RhdGUuIEZvciBB
cm0gd2UgZW5hYmxlIGJvdGggQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORA0KPj4+PiBhbmQgQ09O
RklHX1hFTl9QQ0lfU1RVQiwgc28gcGFydCBvZiB0aGUgZHJpdmVyIGlzIG5vdCBjb21waWxlZC4N
Cj4+Pg0KPj4+IE5vLCBJJ2QgcmF0aGVyIHN3aXRjaCB0byBjb21waWxpbmcgeGVuLXBjaWJhY2sg
d2hlbiBDT05GSUdfWEVOX1BDSV9TVFVCDQo+Pj4gaXMgc2V0IGFuZCBjb21waWxlIG9ubHkgcGFy
dHMgb2YgaXQgd2hlbiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EIGlzDQo+Pj4gbm90IHNldCAo
dGhpcyB3aWxsIGJlIHRoZSBjYXNlIG9uIEFybSkuDQo+Pg0KPj4gQnV0IHRoaXMgd2lsbCByZXF1
aXJlIHRoYXQgdGhlIGV4aXN0aW5nIGtlcm5lbCBjb25maWd1cmF0aW9ucyBvdXQgdGhlcmUgaGF2
ZSB0byBhZGRpdGlvbmFsbHkgZGVmaW5lIENPTkZJR19YRU5fUENJX1NUVUIgdG8gZ2V0IHdoYXQg
dGhleSBoYWQgYmVmb3JlIHdpdGggc2ltcGx5IGVuYWJsaW5nIENPTkZJR19YRU5fUENJREVWX0JB
Q0tFTkQuIE15IHBvaW50IHdhcyB0aGF0IGl0IGlzIHByb2JhYmx5IGRlc2lyYWJsZSBub3QgdG8g
YnJlYWsNCj4+IHRoZSB0aGluZ3Mgd2hpbGUgZG9pbmcgdGhlIHNwbGl0L3JlLXdvcmsuDQo+DQo+
IEJ5IGxldHRpbmcgWEVOX1BDSURFVl9CQUNLRU5EIHNlbGVjdCBYRU5fUENJX1NUVUIgdGhpcyB3
b24ndCBoYXBwZW4uDQpJbmRlZWQNCj4NCj4+IEkgYWxzbyB0aG91Z2h0IHRoYXQgImNvbXBpbGUg
b25seSBwYXJ0cyBvZiBpdCB3aGVuIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQgaXMgbm90IHNl
dCINCj4+IG1heSBoYXZlIG1vcmUgY29kZSBnYXRlZCByYXRoZXIgdGhhbiB3aXRoIGdhdGluZyB1
bndhbnRlZCBjb2RlIHdpdGggQ09ORklHX1hFTl9QQ0lfU1RVQi4NCj4+IEkgYW0gbm90IHF1aXRl
IHN1cmUgYWJvdXQgdGhpcyB0aG91Z2guDQo+DQo+IFRoaXMgd291bGQgYmUgYSB2ZXJ5IHdlaXJk
IHNlbWFudGljcyBvZiBYRU5fUENJX1NUVUIsIGFzIHRoZSBzdHViIHBhcnQNCj4gaXMgbmVlZGVk
IG9uIFg4NiBhbmQgb24gQXJtLg0KPg0KPiBHYXRpbmcgY291bGQgZXZlbiBiZSBkb25lIHdpdGgg
U3RlZmFubydzIHBhdGNoIGp1c3QgYnkgcmVwbGFjaW5nIGhpcw0KPiAiIXhlbl9wdl9kb21haW4o
KSIgdGVzdHMgd2l0aCAiIUlTX0VOQUJMRUQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCkiLg0K
DQpNYWtlcyBzZW5zZS4NCg0KQW5vdGhlciBxdWVzdGlvbiBpZiB3ZSBkbyBub3Qgd2FudCB0aGUg
Y29kZSB0byBiZSBjb21waWxlZCBvciBub3QgZXhlY3V0ZWQ/DQoNCklmIHRoZSBsYXRlciB0aGVu
IHdlIGNhbiBkZWZpbmUgc29tZXRoaW5nIGxpa2U6DQoNCmJvb2wgbmVlZF9wdl9wYXJ0KHZvaWQp
DQoNCnsNCg0KIMKgwqDCoCByZXR1cm4gSVNfRU5BQkxFRChDT05GSUdfWEVOX1BDSURFVl9CQUNL
RU5EKTsNCg0KfQ0KDQphbmQgdGhlbiBqdXN0IHVzZSBuZWVkX3B2X3BhcnQoKSBmb3IgdGhlIGNo
ZWNrcyB3aGVyZSBpdCBpcyBuZWVkZWQuDQoNClRoaXMgYWxsb3dzIGF2b2lkaW5nIG11bHRpcGxl
IGlmZGVmJ3MgdGhyb3VnaCB0aGUgY29kZQ0KDQo+DQo+DQo+IEp1ZXJnZW4=


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:16:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191394.341405 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1M-0002fH-HR; Tue, 21 Sep 2021 07:16:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191394.341405; Tue, 21 Sep 2021 07:16:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1M-0002f6-CF; Tue, 21 Sep 2021 07:16:52 +0000
Received: by outflank-mailman (input) for mailman id 191394;
 Tue, 21 Sep 2021 07:16:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa1K-0002Od-D3
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:16:50 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e28fbe3e-1aab-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:16:49 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2055.outbound.protection.outlook.com [104.47.5.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-phiF_01_OguB7cRVDHqGJA-1; Tue, 21 Sep 2021 09:16:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3389.eurprd04.prod.outlook.com (2603:10a6:803:b::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 07:16:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:16:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0057.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Tue, 21 Sep 2021 07:16:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e28fbe3e-1aab-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208608;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4Vo4Dkwt5tDPO7OINdISKIs97Cf1OIMtewg4ko0cXdk=;
	b=YLAmMJql+qviZW2w0sm47awS0gK6CbF53dEHYt5tZAwGVP3guQ4xiK8BN+nAtRKOrpyYxX
	07MT292TFUwmr3p+X9bzeBLSSZ1oJ25mcXJO1LOx+1Hc9f7hFMTm8cuvFpsOJZkvolRn+s
	oMnxRu+pmb+49nKmikMYK6yYogbRs4Y=
X-MC-Unique: phiF_01_OguB7cRVDHqGJA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NQgFmurlxHLmi94LsZ8bEGJ7Ek0Pib+n055OwWt5ZFrNeep1aGnFLPcaqLl4u3EvOKoT6Q/+OO05Kte9sXPgziaqbu5dajxisVdILiUEoAGsTpRyGy1PkURQekqIdESU6THCf2Pz3mIopKyozCT23ruxLfG6otD9HAzmAZSGSVe8iSB9bo/TJfsgKcEp/QNfc6Lu8VvtV+5IEGMD8sTMI4ZlZj4r/4NzrAIGkzQaY3npNG8ub+9LvweRgfhiqjQmt0J2Lx4LU8XmTm7nTS8BSVkTBDQLzZZQ1wpw64lfCFzPdpVinXZeb2bqrqy8zBHbU2bjdl044DEFwbd8A5Chyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4Vo4Dkwt5tDPO7OINdISKIs97Cf1OIMtewg4ko0cXdk=;
 b=c2yTQrNPMpRuTq6SdKe6cOk5q7FGljzNqbvgLWIpJ9t5eON8icAfUDpbrF2vt/qzc1Q890t9uGVj3awpgDaGl9SOkkw3DZDIAIxxoN7fBSC5fBREi3VFd1URZnes2cCT0JNaNbZNGWymymNC6cGm30PUdHBN1o3uiB0n5yMYRZqM9o08XFLlfSs32Gk3VFCUuhf+iJZiN5HwMj/ENvVD8xe9u7/Y2i7ZNpmhRMF+pXyKkhGfQYAsZ4oFaxEyAGs1NYjl7IPYTuN4SNU7jnAyeWqKWZeq0qbB2tVOPjgOD/GHLu3VTPi0SPAawwtdBk9+NRAOzj4+DscgWRr6ASeTrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 1/9] x86/PVH: improve Dom0 memory size calculation
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <562a9ee5-102e-ee25-e4cd-721e44b7409a@suse.com>
Date: Tue, 21 Sep 2021 09:16:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0057.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c4d41e5e-993e-4b71-1358-08d97ccfc503
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3389:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3389FE26E8DA678ADC6C1580B3A19@VI1PR0402MB3389.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yf6NHN4kfoXQ01MwY35MpmQYsPfONxiyB/+OG0nf3x5frssgcHSO8zvp48//ght2cHYywtALvtBZj+PPm/U8YIZbL4K/uFov2XKc6HvSQ+MpfcHq8+fEUUQPujC+tx7Tha7mD76nRpUcOcg2NShaDg0cKI8ssT6nG7XpDF0do54+1qldebmypfUCo7AsgbcUH7RLKKyY7m44yUEAOuLNFyiGYI1ZSHGBRiT83OiKBHq0vI0DNLbV+n/cS5YFtqomEhJYepv1GiBm1KUpOm8sHUubw418MQ50eS5n1mn0FK7OHX35v+IMQinNQJ7da4JMwR3kSLvEFHEomi1wZ8sqI6u1A+OdkT5TUpGLumc/ljPZiO1ec4Gpm4b1z5Z+77toW11Pya8XyF0iMVh6OodGzMoIc4f2cBp9WiADxU/cVQQib86ZIrq3961L/FupPPo6gJjnFSoPRrKT+ABN70ujus20hJSSTx1L3edMq+rC9lCbIIA97Nq9FSA2TUaCDb216XbcqfUCZZIuN2kil/zTYxV7cVfyn6EQFuHxNYqSEjHujmj75pyHLGmJiv9iYvwe7Nh/barK2pOV5l3s8gXHvY4d1zuP/xFZ4bwA1kghMt85C5/tRWJOjsSJKSakRr2gYq1EfDoMD8lnNurMNIWYPB0D6GdTBhzJgVcMaMuAh+IWyy9myWZKxrQcG779ezds2oWK870c2fzqhSE3dDwSworNrgPZbeQxPefOr+kOcPVvEwZwN/iKvyJ0kki9cLqpqqY7hPrfAXtarJuWzU0k8A==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(6916009)(2616005)(4326008)(956004)(38100700002)(31686004)(508600001)(36756003)(86362001)(66556008)(6486002)(66946007)(66476007)(83380400001)(316002)(16576012)(5660300002)(8676002)(31696002)(2906002)(186003)(8936002)(26005)(25903002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UFBpWi9PYmQycHV0VXhhVzBDZWh1Uzhlb3U4S2srMXRvNzBVbWRuS280V0dx?=
 =?utf-8?B?aXRxN2FWam0yaFRVc2IxR3BaWWtDQW4yWnVNdmwzeEtmZDQ3bzZSTHBLOTVw?=
 =?utf-8?B?V2syT0xXcEh5blh1enRwNFZXWlpGN0xMaWMxK3RTektMRmVOQTNnU2U1NHJT?=
 =?utf-8?B?M3VPUkY2Z3p0UnZNUTBQa1hNNHJIdm42UExveXRiSS9qQ3BTMTQ5VzdEVzFq?=
 =?utf-8?B?NlZ4QmxDWFVjZlJEOE1IQldwNFB5aE8wSERLcDM3S29Kb2VxUnkrWG5jNGRz?=
 =?utf-8?B?ejR3UGg5NFlma1pSeGxTdmlTaFR2cklSVFlmRWVGRy8xVjFhd0pCekdhNFBn?=
 =?utf-8?B?aFlUR0dhVmo2U2t6SklQakIwN2FldWN5WXJHSnNuTUplRjlxWmFraUp6N255?=
 =?utf-8?B?ZUR6UkNpTWFGaXdzeEpiUVQ0aEk5SVFxNnhCYWJKV1lmMXFtVGwwSHVCenB1?=
 =?utf-8?B?cnFCUEhOQXhNRG5TdGp1emt3MjlTbzN2OGVXQkdOL3QveUltREZLSGxrYnpM?=
 =?utf-8?B?WVd3THFWVS9hb3B0cFdpN0R5aVY5ZVNrNDlxaS9ueW5qdHBVK1FxcnR0MWt1?=
 =?utf-8?B?UzhKMlJxU3lFV2RJeTRiOFdTbFpvL1poK2c4di81MkRoUkhJZ2EzbnMxTDRv?=
 =?utf-8?B?YUc3OVNLZ08xMzZPdnFzQWh5L2hqNG1FZXpTRDdsSUNVbTlqMWQ0WWpnTWo4?=
 =?utf-8?B?dTUzeU9YdytyU1FHV1FmV2Qyd3FuQW5ScUo4OTF4ZWFIOXI5UWdJczRPbGp5?=
 =?utf-8?B?d1doQ1VTVkp3S1VYbFNJUkZLYkpDNDE5clhPQ3dkRWN1bndSYVMxcUI1bjJm?=
 =?utf-8?B?Q2JzZXBzSGZwSTM5RGxBcDhodER2ZDBkMVQvYUZ0bmk5aWFLaXdBNERDRkFJ?=
 =?utf-8?B?OENMbndRUFNqeGxlUDFwV1hFYjhZc3N1V2NZZHNqWC93a3dncFQyUG02VlJo?=
 =?utf-8?B?NE9rRW9BRjlseDNibGRnaThNamNSejRBMEQ5UXZJR21tamtjREpjelAvV0Jm?=
 =?utf-8?B?S1JwdWlmTk5Tb0xTQzJudGhhVlRxQlQ3NTE2YlQyaXFLclQzalVpL3JaSy8x?=
 =?utf-8?B?TGViRGlpR0FWYXhSV2hNbExDaGJlbVdiK0RZcVdxSVR3eWhFUzdFSE93SDN4?=
 =?utf-8?B?dG9lRXRPM0RqSHJsVlBSUzlHSjVoUWtpcUp0SThmNUtuT1BUYTNwTDh6ekU4?=
 =?utf-8?B?YWFVdjFXOHpCa0dtN0FZTWczTHJJcHhRY1JZaFp4STRnaUZpTGV2NXFsMFBU?=
 =?utf-8?B?bUdxTGlGbkxRZEczZTlFRTZLMUsxTGx3TUtoZWtHQjlLOEFiL2tpV0J6YUpE?=
 =?utf-8?B?ZXV0Z051TXpnRXhrM0hmdFoxdzNKcVcxK0gxR1NVYzJOZlpQalRQdTNJL29L?=
 =?utf-8?B?Sm03ZFY4aHMrMWFMUFRXYjhuMFpEYTZRZWhIOSs3VU5YQWdtMmszeDBva3Nl?=
 =?utf-8?B?M3BvNnlYWkN6d2tMTFMvWFo2czI3SGNuTS90ZldzQzdPZXNoOUZ0bmJRcEFQ?=
 =?utf-8?B?UFo1UUs0QmJWaHJ3QXpBY2lWUm5LV3hRdDR3R204aC9aVUJiK3dGM2s3QnVt?=
 =?utf-8?B?bFFtbHF5ZUFuU2grZFZQMXZaL3pkbUhmZmZIRTAxdGdDc1ZJbjFMbnZuNjZU?=
 =?utf-8?B?N0FhWkpRUjFRbzBLQnA1cTlxUWZ6bUFwUDJWRTNZb29rRUN2RDZ4ZHEvTFJB?=
 =?utf-8?B?eVUvbGNvUUExK0sxOHpTZ1A2UFBmL0RLKzFCd3pOMk1IcG9ZUFg4ZHBFZmFr?=
 =?utf-8?Q?PfBfyfbZ6jOLdnarADD3Ny3aKQNxx/bpBRFj6zQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c4d41e5e-993e-4b71-1358-08d97ccfc503
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:16:45.6312
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YntyIQd1iNOYuPWSIaiqG8+4bSQuZKrR9zUKfLEBuSP1svBKhAIk+7aZYY0u7RhSTcacDa9fXkNHJXGpljr8bg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3389

Assuming that the accounting for IOMMU page tables will also take care
of the P2M needs was wrong: dom0_paging_pages() can determine a far
higher value, high enough for the system to run out of memory while
setting up Dom0. Hence in the case of shared page tables the larger of
the two values needs to be used (without shared page tables the sum of
both continues to be applicable).

While there also account for two further aspects in the PV case: With
"iommu=dom0-passthrough" no IOMMU page tables would get allocated, so
none need accounting for. And if shadow mode is to be enabled, setting
aside a suitable amount for the P2M pool to get populated is also
necessary (i.e. similar to the non-shared-page-tables case of PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -318,7 +318,7 @@ unsigned long __init dom0_compute_nr_pag
     struct domain *d, struct elf_dom_parms *parms, unsigned long initrd_len)
 {
     nodeid_t node;
-    unsigned long avail = 0, nr_pages, min_pages, max_pages;
+    unsigned long avail = 0, nr_pages, min_pages, max_pages, iommu_pages = 0;
     bool need_paging;
 
     /* The ordering of operands is to work around a clang5 issue. */
@@ -337,18 +337,23 @@ unsigned long __init dom0_compute_nr_pag
         avail -= d->max_vcpus - 1;
 
     /* Reserve memory for iommu_dom0_init() (rough estimate). */
-    if ( is_iommu_enabled(d) )
+    if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
     {
         unsigned int s;
 
         for ( s = 9; s < BITS_PER_LONG; s += 9 )
-            avail -= max_pdx >> s;
+            iommu_pages += max_pdx >> s;
+
+        avail -= iommu_pages;
     }
 
-    need_paging = is_hvm_domain(d) &&
-        (!iommu_use_hap_pt(d) || !paging_mode_hap(d));
+    need_paging = is_hvm_domain(d)
+                  ? !iommu_use_hap_pt(d) || !paging_mode_hap(d)
+                  : opt_dom0_shadow;
     for ( ; ; need_paging = false )
     {
+        unsigned long paging_pages;
+
         nr_pages = get_memsize(&dom0_size, avail);
         min_pages = get_memsize(&dom0_min_size, avail);
         max_pages = get_memsize(&dom0_max_size, avail);
@@ -377,11 +382,20 @@ unsigned long __init dom0_compute_nr_pag
         nr_pages = min(nr_pages, max_pages);
         nr_pages = min(nr_pages, avail);
 
-        if ( !need_paging )
-            break;
+        paging_pages = paging_mode_enabled(d) || need_paging
+                       ? dom0_paging_pages(d, nr_pages) : 0;
 
         /* Reserve memory for shadow or HAP. */
-        avail -= dom0_paging_pages(d, nr_pages);
+        if ( !need_paging )
+        {
+            if ( paging_pages <= iommu_pages )
+                break;
+
+            avail -= paging_pages - iommu_pages;
+        }
+        else
+            avail -= paging_pages;
+        iommu_pages = paging_pages;
     }
 
     if ( is_pv_domain(d) &&



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:17:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:17:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191405.341415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1o-0003gy-Oa; Tue, 21 Sep 2021 07:17:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191405.341415; Tue, 21 Sep 2021 07:17:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa1o-0003gr-Le; Tue, 21 Sep 2021 07:17:20 +0000
Received: by outflank-mailman (input) for mailman id 191405;
 Tue, 21 Sep 2021 07:17:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa1o-0003gY-3q
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:17:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d06a7d7c-9448-43b7-917e-8e7aa29e0d04;
 Tue, 21 Sep 2021 07:17:19 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-3s8Ti9AIMfarizSjot36hg-1; Tue, 21 Sep 2021 09:17:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3293.eurprd04.prod.outlook.com (2603:10a6:802:11::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:17:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:17:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0059.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Tue, 21 Sep 2021 07:17:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d06a7d7c-9448-43b7-917e-8e7aa29e0d04
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208638;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MFNiMXoiKa06wTzVJFs8SMvmLCcjsyrCF6I15nLpdgY=;
	b=T8G82C8rBXnrXx4dNUCfLqoFZgVfOZ34RFARepyjE9AYal5RVlIpojY8yE3dsRijT1U7lx
	QmBaBFzzoY/tGjs4BiGUEftoEkoJxWOpBCvH9RBAfd7F8dpCrFVoGvSDzAkGMS6rAh/qz9
	C8qiLYncYnec74vggrc4qr/kcIQl5Xc=
X-MC-Unique: 3s8Ti9AIMfarizSjot36hg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=S0pYYLv5LZzVYX05b16L7+HauN2z3XNp7O1xn0TLbo1Wby8DTpgRvZrw4KLuPs1uq98Mun/YJrZBQ2WWXuSJYWNRnxWNka4ICRkdmOxNWrp+Mv9UmJE0DbyxoKwPnYgZq5KbLzfrPzyNVSQ91cqSUiFJdnkR8VB2856ASiJovltcWztKi673ry5W5f7FPsLbgqavLEb8KJoklsiogOhh7AAHiIwRLTeiHS8mintk35CvOyj0O6H9WYnUWaavURpezkV59mA5wg/lzEo/qvkxUXbY/86h4+3aVRFu3IlOT6r8OzJtze2tvWGbiGUJl7eBMwn3shmDCPiFFy9u7Ql9Ew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MFNiMXoiKa06wTzVJFs8SMvmLCcjsyrCF6I15nLpdgY=;
 b=AHXntWIYCVqrxmvAEZmvpYPPCdZC6BrqTI3wc6Sqj5TbGEdtsFrAoBG/xOT3e3dVhneAUBWcaqsam0fNxVevcRypQc6dadj0pPOxfH53Ey4vWMBKf5znIGPE5A6+o9NWqb0cjkRDHEJ4xkr7eDctQ5GaLVsR1yDXw33Vaz0a8OA5abV0hQEl6Jgh+8jFnS6LAI7kOxrjFGa6b2yUWuzkUM96oo4EnjiHAeVOH9alUp5GgWPI399J/2cvMUA+rzghceLz2CzBx3ycFpK8Y49+4js6dtf902vKaR4mU2rOQdauiowiPEMV/7TPhk+U6G2He6smGHQEzumKIHPcR6Cx4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Tim Deegan <tim@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
Date: Tue, 21 Sep 2021 09:17:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0059.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b117088e-0409-41d4-e185-08d97ccfd73a
X-MS-TrafficTypeDiagnostic: VI1PR04MB3293:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB32932CB7619789DCD9875634B3A19@VI1PR04MB3293.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VV4IQ4OoTE21WTgek+id9N1AX6i11nUc/sdCJYlQ4Vv++TSyN5VNDFYCFuV9kB5j2GnpmJK4DQyjLz9k+3QxTl5cUpGOvo1w85S6K4IN2Jgrq6CUDraiu0eZ3yAZ5aeiitZuWXUHDprDtAyPkh8X+fvCm2Eak3ZV+wmuEwyAHGL8SIUqwj4hBLtC/VEJT8YKS+mFuVsrTY6I1/4beV5m5kZEpkvseo0CWzLABoxAfNr2mUbI5FuzrM1j8Y4fJaxLIxu7lx7EgXgDTTc7PrmvS005a+uN9eqUWnmLWbtZVm3/rT7AcDoWah4crSfsnMDP6IPS0inXaih6Ydw17MrfPQi9jGBr2uc/UIbbKqUJmnSTCV4sfgD0CM8er54a8/B+HSqxrgXnvGCSmh0OOV2kLcRIK01Ib99pd6WlSVu4AgG4c6SdVGzxUjClY8eSj1T/TXIzUcFuwW+XGmb9dfmZ53FiS+/683gHYUhAqrdpXAVVY/dkl9uJYZN6AKcLA3wYSgjrNL9U+vacVZ09+3pD3XeeSe4USMoU96NblXnZXfXtDnNLoMhj2G4ooyf0qJ1XWo7Ks2YJfjFFHWqpDCL36OdXcdwIiWwDMYFaNC6W4fyURyZmDG6lAGOnaPdB+shTWlu1/SDo/aIolUaslxTR+SD3VLjtdSiOxDqWnB2ZW+AFKUTtjBntNZFhgmHb/yp7jBVsFGw8SkRF0gYb1yuHFpfjTitWUm1UtYLhrKpRQWw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(316002)(83380400001)(31696002)(8936002)(956004)(36756003)(2906002)(86362001)(54906003)(31686004)(6486002)(508600001)(66946007)(2616005)(26005)(4326008)(38100700002)(8676002)(66556008)(66476007)(186003)(6916009)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YS9mUk9pTjhGRUp1R2FOcWxVbHA5eTRyZDdaNnpES2pvbHhtRVlRNlliZFJE?=
 =?utf-8?B?VHhvMjlCeXVuZlU4cVVKeTdUeG9BMEVIWU42TEhQSmt2VTJEeDFYT1I3SXc4?=
 =?utf-8?B?VEdGdkVzRGF3MlFVTFIwajlVSzh6RDRQZU1DMWU4dWY2SzB1MmJ2TlRFVHdT?=
 =?utf-8?B?YkY1SDlPS2NrUFVGVWxZcGNWajg4WWRIdGJVVnpWZjM3WkQ3TFFCdWkyVGdz?=
 =?utf-8?B?TGhpb3ZtNTV4VWs1NU9aZ3RlU3kyVUNxLzA0RFR3SXorTUpyYUpzM2lKMXpl?=
 =?utf-8?B?UlJkTkpCR0FqYlcyKzVFdTNpb2EwRGdPVnVGbHhNYWxlbTM0STVCcWd1dFhT?=
 =?utf-8?B?QXFCRnpSMTUvaGJxR3VTbGNlaTF0bnU4MHd1VXUvYTAvOWpJV05LMFhFeGRC?=
 =?utf-8?B?RXJGSzFLbkwzeFJFSUJmQW8vaVBtWThvZE5MRnV6d0lhLzV6WHFrUW1WTURw?=
 =?utf-8?B?K1FKaTdXUVZvYzMvUG8vY1JnNzFSOWlFL1FGcG1VOHNUU21Ec21yRTdQd1RP?=
 =?utf-8?B?SmVhSXdOQlZEZFRZZ2V3WVBmTG9CdkFMak0wUWpnMVgwdUVoU3U0MVZJZHRR?=
 =?utf-8?B?TkJMNHFYeGxMVW1SMk5WQTh6OVA4eXpSUjViMDdjdUVGeG5ObmVyaTBqUlVT?=
 =?utf-8?B?cTBvTmkzendQQUwxWHJONDJNQXdQQlZuU2VsdWljU2JaSmtscm5NMkZRTzk2?=
 =?utf-8?B?cC9IT2ptSGFEY010ZFd0U01RVUlCak9ySE16YVllWGx5MjUxcVBVTjVVRlBK?=
 =?utf-8?B?Z1hoS21WY1NrSUdEWDFIZm1GYXRZQXBwaGhjSHkrUnl1SnFsZEtFeG5Xakpz?=
 =?utf-8?B?L2MvVi8yQkdJaVhCVGlReFpmc2pQM2syUGpEL0ZWdng1ejFJQUxNcyt6bENZ?=
 =?utf-8?B?d1gwZ015WGRoTytQa0I1dmRNYm5kbWJsMk1oVktNN1ZvK2tRMldxOUlPd3dQ?=
 =?utf-8?B?VkM0c3N1ejM5Q2k0bEdKUW1XSElSb25ONDlPN2NnYXJsaSthUlBmdnBlczNT?=
 =?utf-8?B?NE1vcllUOUQxQTJVOE9lenFJbVgveHR3eERLS0g5OFpadWlnVmtVNUtUbDIw?=
 =?utf-8?B?ei8vT253a1UybktOeitOb0lYWnM0Wm41OWRKUW5KL2UvZnpKemRHdHVaT1A2?=
 =?utf-8?B?VHM2S0xLNXArQ29Fb2R6dHVzY01mTGowSFhBRjh2elJ5NFE5YUJOTjVRM3JY?=
 =?utf-8?B?c3ZaSmtjUWJwZlZIRXU2UEhTVjJYRkd0OEpnWUQ3ckRPOERFUmw3VWxjM3dK?=
 =?utf-8?B?QWU5SzZXTkNieGI5Rk9oRVB3bUg1NWg2K0lOcWhtdnFnMWV3TTdqZ2lUNEdU?=
 =?utf-8?B?Mkg3WWxLSHNXQVBXVEpTUnJaTXdaRElGeE0vSlFub1IxRE0rOWp0ek5BQm9B?=
 =?utf-8?B?bXlOOXNRSUxnNGRnV3o1bHhBbGI0MmhGVmxuajl1Y3I4b296M3JpNjU0NVd0?=
 =?utf-8?B?c1pUVFFLcnN0WFdvV001S2Nlck1obHJRSWJrdWZSTCtyOHFlS2xrVXR6aHlD?=
 =?utf-8?B?REMvWnozT2pTLzZNdGdsUk9tbHc4MXB1Y2w3cytQS2NXV0JBZmNkKzVZZXZl?=
 =?utf-8?B?NGlBQnhzczFzVWsybmpESXpOOXFUSVNSMENxU0tGYWJHYXk3WVdKdHdkTE9K?=
 =?utf-8?B?NHUvTThxZlpMUDY0RUp5cVNCaGhWVURsN1c1aEd0TThSbU12T093UWp0OHoy?=
 =?utf-8?B?L1B2dkZ1akVWUjN5Qk55WmxhWWhFeGNvYmhnYzl3RGRWcU9xSmtrcGo1NjVu?=
 =?utf-8?Q?E1HbRgthSY5SC+WqWMCV6rFXOkxGi5FblTPclpm?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b117088e-0409-41d4-e185-08d97ccfd73a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:17:16.1969
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JZ3n+cYOVUDjh4563IapgqOAZe7RFOXUImThO7pp29Y4ksrd56iE9GMzOXgm9lDW9f4tTIDR4kHhGcFfam5mBQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3293

Leaving shadow setup just to the L1TF tasklet means running Dom0 on a
minimally acceptable shadow memory pool, rather than what normally
would be used (also, for example, for PVH). Populate the pool before
triggering the tasklet, on a best effort basis (again like done for
PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
---
v2: Latch dom0_paging_pages() result.

--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1298,7 +1298,7 @@ int shadow_set_allocation(struct domain
 {
     struct page_info *sp;
 
-    ASSERT(paging_locked_by_me(d));
+    ASSERT(paging_locked_by_me(d) || system_state < SYS_STATE_active);
 
     if ( pages > 0 )
     {
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -21,6 +21,7 @@
 #include <asm/page.h>
 #include <asm/pv/mm.h>
 #include <asm/setup.h>
+#include <asm/shadow.h>
 
 /* Allow ring-3 access in long mode as guest cannot use ring 1 ... */
 #define BASE_PROT (_PAGE_PRESENT|_PAGE_RW|_PAGE_ACCESSED|_PAGE_USER)
@@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
 #ifdef CONFIG_SHADOW_PAGING
     if ( opt_dom0_shadow )
     {
+        bool preempted;
+
         printk("Switching dom0 to using shadow paging\n");
+
+        nr_pt_pages = dom0_paging_pages(d, nr_pages);
+
+        do {
+            preempted = false;
+            shadow_set_allocation(d, nr_pt_pages, &preempted);
+            process_pending_softirqs();
+        } while ( preempted );
+
         tasklet_schedule(&d->arch.paging.shadow.pv_l1tf_tasklet);
     }
 #endif



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:17:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:17:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191410.341427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa2B-0004Gd-2u; Tue, 21 Sep 2021 07:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191410.341427; Tue, 21 Sep 2021 07:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa2A-0004GW-WC; Tue, 21 Sep 2021 07:17:42 +0000
Received: by outflank-mailman (input) for mailman id 191410;
 Tue, 21 Sep 2021 07:17:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa29-0004GC-QM
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:17:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 325a3e8e-9eaf-40a0-a08c-a5a859c2f989;
 Tue, 21 Sep 2021 07:17:41 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-5-eChuMQnbM9OtR8qRh1PPBw-1; Tue, 21 Sep 2021 09:17:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3293.eurprd04.prod.outlook.com (2603:10a6:802:11::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:17:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:17:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0154.eurprd02.prod.outlook.com (2603:10a6:20b:28d::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Tue, 21 Sep 2021 07:17:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 325a3e8e-9eaf-40a0-a08c-a5a859c2f989
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208660;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Le5BJdNW9ne/Xmuzk0IFXev47snchE6r7a8FNfwFXaI=;
	b=A9l1GU7mXI5buJf81fafDwrA5TGWj8njefwNhc8hQlqQAx+o8dGt+TreZYsXJvk2H1+3TP
	F3t3i6i870w+0pTqSpCQNYCkcS9UClGqfbt4vnRT9fh21jtYoI+8jrBeK+sS6iqqdJDtou
	5vxy44IXrhqODou+c97BLSCUIV5qSE8=
X-MC-Unique: eChuMQnbM9OtR8qRh1PPBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cnlVwtVClkmwJ1yacAro2yUcn1YX3AJVFXaPTI3uISPRttF5md3bt4bzgLbfdHxlYn0ru8z6CbFbRyN+XBab0h5J4/vQx/REo4UeeOHOJdCcv5/cpASccUUFK6U8Exe3Q4Qn1Xa+BB5F/PB36dsy2ZypYZbDUePYmFC/tpK/7leFlDE/UIsYGew0392kRqXxZnjuNCDpnc6jO28SSJW3b58iZB2OlLKXYRs2ZnpKVYQxJ5PyPUzs6MfBoJxGQGGxcZHhV0px0wEZ9BzimigxlI6IVeyPqhoEWA2JimjO/j61F7zFcTM6ZS1vf6h1QqbnblSHmKfGbBQdNlbmzR0clA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Le5BJdNW9ne/Xmuzk0IFXev47snchE6r7a8FNfwFXaI=;
 b=nd1gnSPL/TPVYL3hiOj9OPt6AS2yi4U1Jk75RsVCfkJuFKdkFFMobWI7k70J/bpR7tGWHnDF48QZe8BVAS0XZ3XtWuBucPVCnwuV8BcZp0m6KGRdqKiP2SsgcDoYY0ar5NM16HAWELPt1hIDsRbqXf+6TNn2SdlWuocZa61xnjr3RNs7ITXvlU/UdilnZTTmiKdR7Spzl4U3aoTXTNeL4sf8UtOzApvnQlux/9CeizTdCf64OPWaghof0bo9bp7Gox0+SjOc5BG+de55y0czOzY9d2iifv4wmSLMlwq1c0Ap3+7HAOvtv1xqfuRZYaqSrh4Kxs7YJb61+HsiIqvAQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 3/9] x86/PVH: permit more physdevop-s to be used by Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <78c84053-3d79-b36d-be03-08b3b3a3d26c@suse.com>
Date: Tue, 21 Sep 2021 09:17:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0154.eurprd02.prod.outlook.com
 (2603:10a6:20b:28d::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 97c6805a-0f31-471f-e4ea-08d97ccfe42c
X-MS-TrafficTypeDiagnostic: VI1PR04MB3293:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB329325490E31417B3E35C420B3A19@VI1PR04MB3293.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sJ9tMjWXdmprCrWK98SbUGnUwiMblG0eEeciQSt6Dk0DTN056yNhOWYk0U5l//h/q6Nict1ck6qW1ml0QtbChX0wZxE5nmbZwDDHI1LAfqlZZguq5xMwNutb1iA0xC9/mjHxd+ctRiAlvAZMmgXMRdjuUxXaKvr56Qgl+7D7Cy+41zPHA+xvYmjk7soyNDKIaq0s0KmPd6twrqVhNl3kI1GFLfkynR2rCRFzTlBgb9kLwqD3JNVvwsgKr/WFSLY4lQPY0x/T7v17kS2lUVFVtLCbrNImIzhDC2H9hgu50lpXVsyS9dAPCSN1jwJ6fpNloeG0Iu2oo9aTHHLd5x6rRiMN8EzGcawO54rw1WREmYXvUm0w5e9UuSO3zmNVDnk6X85EivlSoTw9HQApWGx/lBS4DuoJb+YtNun46ZDIAPNpQazpbwQNkj8yaynZCVK5kBDqPwqR1ucKC28mdDf7zqQ4O9c6pEsNwpOK9AjgqaYFcAoS+hAvLqaiKAULSbBKQ52GqbyLZJoA+fpcxC4iERHm4BmiqyvRbVO8QT2BsjYblY7bNeoZf0D07hWKS8OOpGAo48WwVgc9Jg557VrbCtwRh3jwOP7U34esEpP0+nUuIYMr4jXuDk5z1WT/hkV2ahjh5IPZUd5M8vTZYwIplUh5Q+ZcKuXZuDvz/O9U7uHXxf0mEAEvHAC6Cp60LE/TIfB4i/Frqephz097CDmEQPNnTIsYreK2FUl2damey+U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(316002)(83380400001)(31696002)(8936002)(956004)(36756003)(2906002)(86362001)(54906003)(31686004)(6486002)(508600001)(66946007)(2616005)(26005)(4326008)(38100700002)(8676002)(66556008)(66476007)(186003)(6916009)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZW5FWG90clhZZzc1VjFEa1lXaUxMdUR1OUJEY2E0cDFhRUtKMWxtTkZUOHhU?=
 =?utf-8?B?Uzg1anZ0Qzcvby9wL00xSVJJclZIamNweGI2ZUZia3RGdGUwZGFVWTZXamhQ?=
 =?utf-8?B?OE0yS1FXZnRnWXRlV0xFalREbU5SUXFrNkpsVWN3NmVEUCs1VnM2cGFUZC9U?=
 =?utf-8?B?d281akVYc0pQM2NSblFyNjRyZWhhcVdybi9sdjRGclZTbnJKdzhJKy9NUWoy?=
 =?utf-8?B?M2lsTkVmbWhqbHlMbzVGaGNMY1BoOXFXTUREUkJDd0R1ajVRNnNvaTBEYXRo?=
 =?utf-8?B?cEgzRTRpT3RQTFhEODN6RnJiVENDVUJKcG1scCtWVXJ5ZURYQU5OeDZUWmZS?=
 =?utf-8?B?M0J5NkpLdGs3c3B5Ym5TWjRubWJ3Wk5ZalhyT2ZmZi8vWC9mcUZFQjg0M2Nh?=
 =?utf-8?B?S3Q5R3AwYkRPY1lyTVlJR0N3S1U0M1QwY2E2amhadnBVUTVkWXo4M2lLcG5D?=
 =?utf-8?B?b3F1ckNoRDBEMmRXNCszeFhPQXA3TjNldTE0MTVXd0lSdEluVFZiQ0h2L3hr?=
 =?utf-8?B?ekJtTW16eFNGTkR1eXBRVkFJNE01VFh4d0taWkViZHQ2dFF5TFByTUpxaFZn?=
 =?utf-8?B?UUtjK0xsOGNKYTFZcnpzYmt3Z1IyZnpRdmdlRG1XT0QxQS83SC9PNFA4Wm1O?=
 =?utf-8?B?ei9KVzdBVWFlMTQvVUhvNnh1ZHBmMjFhSnE3eVlaWUdnczFDREFubG1zZHZs?=
 =?utf-8?B?bXNERjlLRVNZS3hyaUhDcXpZT210dENWYXRsbGEzbFdpYm0wS2VGWVVJZU5u?=
 =?utf-8?B?WmY2ajlMZ2UrTkphdk1oZWY3ZCtpRXB1NkRzRzJydWk1YlZzNVJsVkFOWHFV?=
 =?utf-8?B?TUpnZTBnNk1EZllucWwyZUZFZEdpbDVmR1VnQk1vNWJINGVVMnpOelVEY2dz?=
 =?utf-8?B?WWtZYUdEMGkzT3YyMlIzSGRSMEovU0t5VzVyNzZLKzRUVTRzNHI1VzMzTW9a?=
 =?utf-8?B?NGhJUFB4VkNEc1IvcUNwdmdsdkJnTzU1YkMxZlI5S3hzMmZPY1hyMU0yY2Np?=
 =?utf-8?B?T1F5ZmZXeGFkZ2hSM0dmSFVHcDhTamowY3BTMzcvTHU1TWF5d3pZbWRmQldY?=
 =?utf-8?B?YlEwR1UrY2NVMUZaRm1uZVFVQ1h5d0I1TnNwaDd6VnlsaHlNU0VZV3E0cE51?=
 =?utf-8?B?dHhhR2dOeklJcU9pUHZVUjlsVUg5WWVhZ0tFZGMyTHpPMFRpanlZVjdac1FD?=
 =?utf-8?B?dTFtWjFGZkZ3Zys2bmJZZndpUGcxYTZGVTBDbDc3bGpMWGRTRklET3JhdnE3?=
 =?utf-8?B?VzNYQXZYQ2tlcUM4aXNzRko4eEdIYTlBbFdBa1BnTXpvSjBPMUNYZHBBaU5R?=
 =?utf-8?B?ZGo2YWptcmpXbDhheG1FOG9PZkVRdzU4dmY1cFFaak4xZXAwdlEycUhLYVZD?=
 =?utf-8?B?ZmRhc2NSWmRTaFVCNGwwL3lJaWVkUGZrUnVib05zd1pGdDdyNzlGd3FKK0li?=
 =?utf-8?B?QTg0aEN6Y2JBNGdKR3c5Ull5M3QwcXBKZ1ZWSkF4MkkzZy9zdEFTSDVOYkRE?=
 =?utf-8?B?VldMY3E2a09XeU9RYU43cEZPc3I0ZHhURU9EckRnTFp6QzdWL1dEOFkxdFdC?=
 =?utf-8?B?b3dOT2xzbjFkeFNTQjJENEtObzM3RWZ1enJxWU96Skh5amZNOE1MSGxYN3d4?=
 =?utf-8?B?b2hjcWg3ZjVvRmpPRXpZZ0RmY01IVUt5RUM4V2xlMkJic253a3pMTWVzUDNi?=
 =?utf-8?B?YnkrRVNwQXg3a0VxUkhoelpDUnU5cFR6YUNHa3J0bE9nSnFmb0taUnE5TEdz?=
 =?utf-8?Q?cOu8c3PLYnK9OEtkDsVLkMAlwwFKBQJZ1MXIF5w?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 97c6805a-0f31-471f-e4ea-08d97ccfe42c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:17:37.9216
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1SOn0nmxyIpiNa4GGLBTPAAscpU9iU2OImNjof6+o3lAirSvzjUDaO4WZOudbiZwDYqLt+hU/W9DsdgJK/JxaQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3293

Certain notifications of Dom0 to Xen are independent of the mode Dom0 is
running in. Permit further PCI related ones (only their modern forms).
Also include the USB2 debug port operation at this occasion.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm uncertain about the has_vpci() part of the check: I would think
is_hardware_domain() is both sufficient and concise. Without vPCI a PVH
Dom0 won't see any PCI devices in the first place (and hence would
effectively be non-functioning). Dropping this would in particular make
PHYSDEVOP_dbgp_op better fit in the mix.
---
v3: New.

--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -94,6 +94,12 @@ static long hvm_physdev_op(int cmd, XEN_
         break;
 
     case PHYSDEVOP_pci_mmcfg_reserved:
+    case PHYSDEVOP_pci_device_add:
+    case PHYSDEVOP_pci_device_remove:
+    case PHYSDEVOP_restore_msi_ext:
+    case PHYSDEVOP_dbgp_op:
+    case PHYSDEVOP_prepare_msix:
+    case PHYSDEVOP_release_msix:
         if ( !has_vpci(currd) || !is_hardware_domain(currd) )
             return -ENOSYS;
         break;



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:18:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:18:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191421.341438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa2e-0004yu-ET; Tue, 21 Sep 2021 07:18:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191421.341438; Tue, 21 Sep 2021 07:18:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa2e-0004yl-BE; Tue, 21 Sep 2021 07:18:12 +0000
Received: by outflank-mailman (input) for mailman id 191421;
 Tue, 21 Sep 2021 07:18:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa2c-0004y8-Fl
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:18:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c7599eb2-73d9-48c5-b499-691ebcd457a6;
 Tue, 21 Sep 2021 07:18:09 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-_46qPZCjNa6EFG6pHLvE1w-1; Tue, 21 Sep 2021 09:18:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3293.eurprd04.prod.outlook.com (2603:10a6:802:11::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:18:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:18:06 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0P190CA0029.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::39) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Tue, 21 Sep 2021 07:18:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7599eb2-73d9-48c5-b499-691ebcd457a6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208688;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=52koP07e3NaHR5tP62Xc53aym67B0vZJLqdc3ZgaWY8=;
	b=RPd46UMZtmtMZ4jckRbW6fZf6vxV68eutVjz09cPyOPaZse1TdVVafVjfWQyPhL+Xm6hbT
	+YL3zAqUKp6mXkXXKiWtgrl1nSo/8fVuWXkAVdUQ61IuoSHo+85p7IagztGFVS7tUeaBSc
	PptyrfMpu2CDMETLS5LVtFXbXPkN58c=
X-MC-Unique: _46qPZCjNa6EFG6pHLvE1w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UbRThvdDBVayNVbt2MvjeYXE6UYWlef5lu8TZc1+dNew7bOh1UkT8SSjkVbYlK9XlCrkVYE5BofDBUxzA8pnvxRk/SYBvE3QOg4OF46ADy7IVv+Tx4eiZ0xG7HCRUpJ6om0xooIshtMNNJJZUNOZ22B46loPw93laOfb0PgV+t9Rf8nUFL1WCmnR8U7iamDw6mHdaIEOchD9hbLoYVBODhBo3nHalJiuUx0ow2GAfC3uCJTxD5EBJN/4gnL2AfEniJ/f88w58qi9TyWKY1BPZpfPJKf/JUY7C+3eEtvxbsmAQqQrSGYW7hzBJLHGqhZwVCq4CR11/Y/g/5O5G1X9eA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=52koP07e3NaHR5tP62Xc53aym67B0vZJLqdc3ZgaWY8=;
 b=RvAM4xaN7snbVlZnp8LCn9br8VBxGwnzEGaZkyLLQsNqHk3lutxVSK1Jpif64ApGcsO0dfuqiZnsNtXseOmTlOocgo+2CZKAjq3bwBT1j8gUsmOHA7q127kD0vXWvtmem7GFDql3Sputr4XYts2ijciGyv2wDRGPGquHV22388Htsl3uaa13qHtXl2rzvFj+tkI52J1Ed26fnpVvLVy6jBNUbcpMHJBxd7xMqzUFCJHSnKFS7DmDBA9SDXfndqVJaz3NAboJlVdDLC3mU9cUcysaMS76ohEV83Ppp8/uJ75MHxJnVLnss5WMzy3cHxCelb9C9vDircJawsNQsnZXRg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
Date: Tue, 21 Sep 2021 09:18:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0P190CA0029.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:208:190::39) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dc55bae7-f929-421c-52d3-08d97ccff532
X-MS-TrafficTypeDiagnostic: VI1PR04MB3293:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB3293CBC749A535D7D6406A17B3A19@VI1PR04MB3293.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	82q5LXkChGB0DZawfOcqS7gbPxNFrD2ouq7+J/Xv+MiqcYUBMpOABgLxl+WQSGwHwG+n38WVrtiQOxG8lAdycMO96kpjbbSgCQufz0b+9DjFfe5vMF7VjMytFse22FTHzgdD++zNVRTcNlziIxuxS04LCVfdDIzmA4shboTvBAinlktxNY+45iqxuRlmLGdbZhcCOxMdWmgigH/pPm+5T8cJ6O9xJ0hVnRqprle2BQpilqFpj7jFkPank3GW/U7qPdBYy0+iLflLCTfweV6id05CI29VB5tB9jSyPMBEKiFXF0Wygbml5dlBlRyASR8ZdEV6UMlNQZb+LMBMtnd/Ha4oGbxl3s86CGIRFdeNfsIa3repJ8cQ0vTqyIHpJSJMaX49cl9MrsaTv9gnHixD+JNyprT3iFSMBpJhBVIRu0GHi72BPPWF8fnZrT4HAHEd78wILyCqZ+trDkbo+rE3RFoG74iVkbPFoDafep8d5ED3mEd0i0Tun8bcjF4II4BvkHZ0EUmv19vNfE4IYtY1D26abXH4Lx7HaINLxRUkUke6c5bcWfgNEtpWlpFXqZ4TEjYQgVk6qMU5gnCjnxDEdjve1RdaZork2zUd4pxzvN8QmBAF9BOsRW5EI4mYC2FZ63wdX0J22bJSwYZPJ2JsQTX48cpx/uuSH/XrA/akz6rxCi15l6lnMqThQ9ZaVs5WpJyK4pXsmUn2URRdT4k70mvoF6r8iybUjPnV/Je1VOq8sCo8RXOxlU3z1TvJpXCO
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(316002)(83380400001)(31696002)(8936002)(956004)(36756003)(2906002)(86362001)(54906003)(31686004)(6486002)(508600001)(66946007)(2616005)(26005)(4326008)(38100700002)(8676002)(66556008)(66476007)(186003)(6916009)(5660300002)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bVVuUExkQmMyNFdoR1JJNTVxcDY3YnE0UDllOEtQVzE3WDQ1U0lkSWNFWDkw?=
 =?utf-8?B?OVZ2Y3U4VmdqditiMlluNm9tZXdOdkZNTjRkSkc5TGlVMlNXSGNJaVh4TWhk?=
 =?utf-8?B?dVV3TDEwTE9lZTVhWmxGVEFmVXkzYVdhYTNqVnhoNkN0eEJyYndNQnZKZ0Js?=
 =?utf-8?B?a3c5Q1Fvc1kzaWFWQ1BlRzFyaUgxSXU4L2FRSDBQL3BLL3FmTUw2alQxWWF3?=
 =?utf-8?B?S0kvSkZOZmpuaU5CSjM3KzlBMkNxeTY4RjF1d3F1cTVVS3JuTDhDMTR5b1Rk?=
 =?utf-8?B?djJObjh4T0liZ2FCZkhtNkZhVEUzYjNqckU0OW10elB2U0ZnOW4wck80eFZv?=
 =?utf-8?B?MVArUWdVMWVrU1RsK0I1cHdsZWhaU0I2L1NINVhobHJaWFZYMlNVenZuUjh3?=
 =?utf-8?B?ZmdEa3E3NDRMMEtVeGxmcll2K3JGVFVxNm1FV3dLbHhLb3duc3VIQVQzT1hJ?=
 =?utf-8?B?K2VqbnhoeVFaVE9Ybk5uZmVCemgrdEs0Mmk1VDJHdGhFUlV0Z0VVZnZWL2hK?=
 =?utf-8?B?eUpqNjMvOXdDY21oRkhRL2dkeE5nU3U2OGFJNW9vdDAxSXo3cC84ZXRlYVJx?=
 =?utf-8?B?UmJCS1pJR3JpVjFIZnhRVlNOb3REWnErR2J0MlpWcjNDTnc5RGZwVC95YWp5?=
 =?utf-8?B?MENFNzlTK3ZQZlp3M2gxTjRoRnk5ZkVOUG92MldyTEhYL1ZPOFJVSkFISk1K?=
 =?utf-8?B?MWdYcElMa3FScTdrTWhacUdBWm9VRGE1NHUzeDBzODc5ZEo2WkRNSElQNWNu?=
 =?utf-8?B?NG1NTGt4b0N5T3JmM3J0RVA4Rm1PNG5ESCtUS1A5TmtJZXA4b05MaU01TDR3?=
 =?utf-8?B?VnYrOWZyT3ZWVExKMk9hejFPWUJoNlBJajFXeEVHcVJRVmYrVDhKQm1xWlRz?=
 =?utf-8?B?SnBWb2VsU2g5YThRL0R2bFM4bTBDQzcwbVMzZmR4ZFJQMVFpMkhqQk8xL3Qr?=
 =?utf-8?B?R05PQXI0R3cyY1Vaa1FzbmhQSjZJT2kxWWlFRFVFLzVBb2luRFB1NEEyU1ZO?=
 =?utf-8?B?aldFSHRXZHVyOVI2ak5HRW80aXg5L2NHeVIvRk8yd25vSnAyK1JFYmxBWWwr?=
 =?utf-8?B?N3lZS256YU9FK3Y1VzZpdnk1ZGphQmhGS2pZTlAyQ3V6RmpmQngrWXh3MkVT?=
 =?utf-8?B?Vkl5aHV1ZzlySTQzTmZNdTVhK1IwUGgwdTBFYmNDN001MUF5Mm1HVCttcFRN?=
 =?utf-8?B?RzRJbS92ZlZKS1lSRS9nLzZMbWRiNlcxc0l6TmJCbTUwVnRNb1V5Tkl2OUth?=
 =?utf-8?B?MkpmT2xLeW05dk9EQ285YSsweGl1NGFrZG85UmZjbHh4dVRhNTR6K05ZKzJC?=
 =?utf-8?B?Ym9ZMEFOVnZPbDNsRHBVUHcyWG5LU0tWYlJVU0Z3dG0vTlpuMTZWZ0NhMkZo?=
 =?utf-8?B?K2FtdXE2NEcwQmk0RDJGUzVaVGRJMGZ1Rlk0YUdaRDhSdnQwaVdyWm1BUkNa?=
 =?utf-8?B?SjgyWGdZRmozcnI0Uy93TU9sTnc5OXY3ZkVtd05ZSkdCdEZMY1BnWlptVHM2?=
 =?utf-8?B?WEluTWt0TG4ycFJZZDdhY2tOV1g2blk1ZG1sUEp3TGZmUUJjWXdnandIQ1hF?=
 =?utf-8?B?MHdxSXpEbURyR1BzdWsreThLT1kyc3pZMmUvVmZIV2VWWU1haUVTdGpMTHFr?=
 =?utf-8?B?bEJQMWR5Vlp4b3V6N3ROTkdLZ2s1VDFLZkY5R3V5b3hPZHpCTFNaT0t3Yjdr?=
 =?utf-8?B?aWtyL2R4empwWjFQa2ZLMVFIUjFZcDErZVBPR1dkdlkySk9YUkxMcFVvdE8x?=
 =?utf-8?Q?ZyFouIQ+G7Y0BCKrvZGbRu65TloD5Xl9Qom198M?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dc55bae7-f929-421c-52d3-08d97ccff532
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:18:06.4805
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Obme7KzxnDdnISlUP0hQRyt6iuxFFL+Ica5tJ9PqiaE9bpD5/ev5GvEcGKpJaXf74BYAuVrH/66574w6xM3WDQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3293

Like PV Dom0 in order to use the console if in a mode other than text
80x25 the kernel needs to be provided information about this mode. Bump
HVM start info's "current" version to 2 and use a previously reserved
32-bit field to provide struct dom0_vga_console_info's position and
size.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: New.

--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -19,6 +19,7 @@
  */
 
 #include <xen/acpi.h>
+#include <xen/console.h>
 #include <xen/init.h>
 #include <xen/libelf.h>
 #include <xen/multiboot.h>
@@ -549,6 +550,11 @@ static int __init pvh_load_kernel(struct
     paddr_t last_addr;
     struct hvm_start_info start_info = { 0 };
     struct hvm_modlist_entry mod = { 0 };
+#ifdef CONFIG_VIDEO
+    struct dom0_vga_console_info vga_info = { 0 };
+#else
+    struct {} __maybe_unused vga_info;
+#endif
     struct vcpu *v = d->vcpu[0];
     int rc;
 
@@ -598,7 +604,7 @@ static int __init pvh_load_kernel(struct
      * split into smaller allocations, done as a single region in order to
      * simplify it.
      */
-    last_addr = find_memory(d, &elf, sizeof(start_info) +
+    last_addr = find_memory(d, &elf, sizeof(start_info) + sizeof(vga_info) +
                             (initrd ? ROUNDUP(initrd->mod_end, PAGE_SIZE) +
                                       sizeof(mod)
                                     : 0) +
@@ -672,6 +678,22 @@ static int __init pvh_load_kernel(struct
         last_addr += sizeof(mod);
     }
 
+#ifdef CONFIG_VIDEO
+    if ( fill_console_start_info(&vga_info) )
+    {
+        rc = hvm_copy_to_guest_phys(last_addr + sizeof(start_info),
+                                    &vga_info, sizeof(vga_info), v);
+        if ( !rc )
+        {
+            start_info.version = 2;
+            start_info.vga_info.offset = sizeof(start_info);
+            start_info.vga_info.size = sizeof(vga_info);
+        }
+        else
+            printk("Unable to copy VGA info to guest\n");
+    }
+#endif
+
     start_info.magic = XEN_HVM_START_MAGIC_VALUE;
     start_info.flags = SIF_PRIVILEGED | SIF_INITDOMAIN;
     rc = hvm_copy_to_guest_phys(last_addr, &start_info, sizeof(start_info), v);
--- a/xen/include/public/arch-x86/hvm/start_info.h
+++ b/xen/include/public/arch-x86/hvm/start_info.h
@@ -33,7 +33,7 @@
  *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_VALUE
  *    |                | ("xEn3" with the 0x80 bit of the "E" set).
  *  4 +----------------+
- *    | version        | Version of this structure. Current version is 1. New
+ *    | version        | Version of this structure. Current version is 2. New
  *    |                | versions are guaranteed to be backwards-compatible.
  *  8 +----------------+
  *    | flags          | SIF_xxx flags.
@@ -55,7 +55,15 @@
  *    |                | if there is no memory map being provided. Only
  *    |                | present in version 1 and newer of the structure.
  * 52 +----------------+
- *    | reserved       | Version 1 and newer only.
+ *    | vga_info.offset| Offset of struct dom0_vga_console_info from base of
+ *    |                | struct hvm_start_info. Optional and only present in
+ *    |                | version 2 and newer of the structure when
+ *    |                | SIF_INITDOMAIN is set; zero if absent.
+ * 54 +----------------+
+ *    | vga_info.size  | Size of present parts of struct dom0_vga_console_info.
+ *    |                | Optional and only present in version 2 and newer of
+ *    |                | the structure when SIF_INITDOMAIN is set; zero if
+ *    |                | absent.
  * 56 +----------------+
  *
  * The layout of each entry in the module structure is the following:
@@ -139,7 +147,15 @@ struct hvm_start_info {
     uint32_t memmap_entries;    /* Number of entries in the memmap table.    */
                                 /* Value will be zero if there is no memory  */
                                 /* map being provided.                       */
-    uint32_t reserved;          /* Must be zero.                             */
+    /*
+     * The following sub-structure is only present in version 2 and newer
+     * when SIF_INITDOMAIN is set. It is reserved in version 1 or when
+     * SIF_INITDOMAIN is clear, and absent in version 0.
+     */
+    struct {                    /* Coord-s of struct dom0_vga_console_info.  */
+        uint16_t offset;        /* ... from base of struct hvm_start_info.   */
+        uint16_t size;          /* ... of present parts of the struct.       */
+    } vga_info;
 };
 
 struct hvm_modlist_entry {



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:19:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:19:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191428.341449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa3e-0005hB-Oo; Tue, 21 Sep 2021 07:19:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191428.341449; Tue, 21 Sep 2021 07:19:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa3e-0005h4-LT; Tue, 21 Sep 2021 07:19:14 +0000
Received: by outflank-mailman (input) for mailman id 191428;
 Tue, 21 Sep 2021 07:19:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa3c-0005gl-QK
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:19:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3ee276b3-60c2-434f-88e2-2188a7c33695;
 Tue, 21 Sep 2021 07:19:11 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2055.outbound.protection.outlook.com [104.47.9.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-IqfAnf8hMJiXfa4CvRlazA-1; Tue, 21 Sep 2021 09:19:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Tue, 21 Sep
 2021 07:19:07 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:19:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0056.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.5 via Frontend Transport; Tue, 21 Sep 2021 07:19:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ee276b3-60c2-434f-88e2-2188a7c33695
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208750;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kOHPUdSszd1R6FGaIvuWyobel0/L1w9ZbdVwz40lrjw=;
	b=Pj8RS6sueAeoZuysgJQ6xuwCOBzOoGmloV3D4sfJPAdkxhiRULGGlN0+lfJqf90B7x2ppm
	1gWrlF9bhBH7pvL72bxq7orDFQqSC4SAUmrRgOlqnPjXUwI+wf5/oZ9NqCPNWdQBmrQfDf
	KkMOOMUTRSgcLOB+IGvvG9NWNx8OR2A=
X-MC-Unique: IqfAnf8hMJiXfa4CvRlazA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YQsC5t1qnE4ToYU8KWXjKGb6ULOHbYu1/J4LvakQPdtBR3ecLi09fMTFwp+cVCRhJ6fefLvhoONDwIdaUJhSKX4mcrFgf7Vu06hO5XPTCrhCoAf/4bLcI9iB/DgBdPd+tHd4YBTSqtaKIJd1uC37kvnj0itfGj5WaTSsfat4xdhNacUCc4M62DQA9w9gpgI+ju8KRQJwgEfWU97lYe2aYy2g6BURSwxXfU3nyk5DPsaz1j8woHIFkETEkHCL0fEoR4RH1fbydU35EUfoyitAEGITXgVToHsRp14kDattxHfb5qyAqhoAp7/vOirm+ANafJRTDW8lKqe2l7aaefrT3Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kOHPUdSszd1R6FGaIvuWyobel0/L1w9ZbdVwz40lrjw=;
 b=BeUyHrsCvxNJVgi5tRdNxILXpJChThXQX55vbv4socxfSiALYJT+g9OMflrxdzHTuKh4OAAs9MwILSK/FXMIW2J6C3aoPvxdi777040g9nJm9fPWR5IHgauQA7ci1bnD5qk44L3T8dZgmauyvjm2y0SQ6u7uGamPi0uxacnDF23FqoyMjOle1dOkYgOz2qa/H43uFBTv/Lg2dzckGpYOHrmtjK4WItltjvxZQaKnloaQ/FxERzQLHOUbwX3u140R58Ofu1PX2u1QXqo3Yta/dx+LqPcn4HYt5f0vy0EWu1Crhy65TGP4+Y8rZ9Z2FI/VfP6gR2v39QGQTyV7WugzvQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 5/9] x86/PVH: actually show Dom0's register state from
 debug key '0'
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <4ae1a7ef-d6e8-75db-ddf5-987f10175196@suse.com>
Date: Tue, 21 Sep 2021 09:19:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0056.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 28c134b1-4b59-425d-1804-08d97cd01987
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41908B849B2140AF80ADE1C9B3A19@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Cn1h9ducNgRNq3NeT/9DLNgN8VLEtbmdddDU+eS84S5DQ26p7/kO2uoaUH1X84odJ1L81H4cgJrGK6xKIUYOkyuE5yctANU93ulOc/YCxcAcnht6l9uX0wAnlYSmi+teyekYE7UoImlLhCLoZJRbHZKtmLy87AlqlxKitdJRzH2TA2vQ8TfgKRdRwqfUZFFL1c2djsBq1rmREc+uv43i/Ez17R4NJ8uRFe0pM2rKfIChtBBlJ2KJHQHtQHj3CS1S6HgXBPXdKaHUWlIq4anQLS95X+Am+qPPMMIPhHF4cjrr0wUDWykYeNiMRDA3VyxT1K6d/3FbnAldNxnbASXj4G0pOHvWU3eOQ0HDwWK7qqEDUE0TuTquQu3S27XEi2vItmgm9mcfOBSCxYGqTzLXozPMzfu4cMFGkRpMihlHOOKxugXor5Ore8zI0lN+8bp++1Bab+5ai1XODvulZHnX8gOQlDAQGlrAppFfM1URhoyKENsl50wXudPlN0rh9efrA2VLWv6ItcOIP4qLOfT7JTIhVQJvNnnsARi8hgUs5+GQ5FG4H9iibVfJtuftaePafdL+8afDLL0GtSJXICqkNnLtl0huBDhVbzxjU7oCPwSmnpfworPKBksx/2ha3+0COTdkDt+xfdfdb+q908bjWchd0bC9dJiIVTnBR33Ws4FfLO03jxc6pvIG1Q1p0rcvTe5xD3ebDECgCsUD0q3BWQHnTA3Vj9oGVI5O0fEAt7E=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(26005)(8676002)(186003)(36756003)(86362001)(66946007)(66476007)(16576012)(6486002)(66556008)(38100700002)(8936002)(5660300002)(508600001)(956004)(31696002)(31686004)(4326008)(54906003)(83380400001)(2616005)(316002)(6916009)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eVlCZnRyMVp1NjhjTS9RcHNLL3BPOW1aRFhGd1I1bFJWOS9MUlpyYmhLOUUw?=
 =?utf-8?B?R1BDckt3ZUxIM1MxVmcwUWZ6R2NqYU5rUGYwYWJRcWRqbWN1Vjk5ZDEvajM2?=
 =?utf-8?B?eTVZRTZnOHY1ZFBmelNJSkpBZk1BT2k5cDFFRmtSQ3VyRjZobnRja2ZpTUUx?=
 =?utf-8?B?Y255aDFMbUFzMFp0aGFQSWRJOEUwMFRkL0I0Zmsxem9yWk9tT2R4NDM3Slo0?=
 =?utf-8?B?b3Q0WGZVTE5wUTRka0dQWHhjdnVmUCttMWpqa2FpTmxacUYvTjdybnpQTTlF?=
 =?utf-8?B?OGp5R3JHUnlqN2p1bmhHUzZPTFozWEl2Q296OFRVa1FTWkVBNHlwSlVrWWEy?=
 =?utf-8?B?QWNudEdxY2xVbnBVVW5WcklFSWVWcHhpTUdNZlphOFE1Z2czL2hUajQ3Mm1n?=
 =?utf-8?B?SFUrRmhweEdKWExlZm1HcDFaSTBjWlVFWUFjM1Q0c1BCV0QySldmSTNUdVl4?=
 =?utf-8?B?R1N0Sk1ENldEd1hhbUZpeU9neVQ5cVFYeWVzNElocGFRZHhwa0plZVJGREly?=
 =?utf-8?B?R00xRDJYNEN3TUNTUlpnM0tjU3hMVWQxSjJxNERSSVpBcGVLK1IveUlhelhv?=
 =?utf-8?B?Wmt6c2syV3NGdm82Ym1KM2gvN0NKR0F3SCtoano4clEyZVZRZVdsZi9kaW55?=
 =?utf-8?B?WW1hR015dFZHejRsc3lCRU56RXZwQXNPdzZDUnJKUVlDUHRuSU43NFBieGNL?=
 =?utf-8?B?MU1vN0xlcUhTVjdyekVtOHhQRnM1enU1NVBmajJyeVd5WmhlVFp4dUJrd0VE?=
 =?utf-8?B?K0o5V0RSSStERzNYbThranJvQXduS1Q3bHVSR3dBQllMSFdlbHNMeXA4MktG?=
 =?utf-8?B?T1pzLzh6VjN0a3dJcXM2Zk9LSEZ0aldLZldoNWZPUHFiZmtCdDE5eEU0TEJo?=
 =?utf-8?B?dEFIb2lnWHNjWklTN1FrZjM3S3BldWVxUi9ML3Z2ckoyS0dOM0lMZFlUR0s0?=
 =?utf-8?B?L0trb0piMTdEZndxdkpmQ1Y2elBuMFNESXNTaDFTM1FGNUtRNURDZitHQlY5?=
 =?utf-8?B?bkNXbm1DQjdSMEc5VVJITTZKMUlwWi9oT0JidVM2dEV3U0phY0EvVHV6QzN1?=
 =?utf-8?B?RFVrTjBHelZXN2lDcGljTFVUYmJIaitCZHlLeUxlTkI2U2dLVk5rNDBiRlB2?=
 =?utf-8?B?Q1FrRmp5U25EdXpjZmt5QUVJUlEyY0ltYmU5OUFxVDdzZWdwOS9zKzNsMTdI?=
 =?utf-8?B?d0hrS3dUbzA2eTFSSGhiYzJBZEtRSlRUcURhd2Y0c0x6UTZ1d3ZEVEVlTXRU?=
 =?utf-8?B?V2lhUmhkWFAwQSthTFFwcmt4ZFFzVmw3cWRhcVZOdVRDNzJVWHZueEZBQ253?=
 =?utf-8?B?ZDQ0dFc2eWFQMk5xb0tleGhLZCtWMWZNWUxZdHVCVWFVcm0weUdMbDhpR2pV?=
 =?utf-8?B?MWpoeVcwOEIzVjFtTmVoQWoxNEZpVTJKZU9EUVRxR09sT3BJUzY1cnNadmpB?=
 =?utf-8?B?RndhNWpDK1daNWlwT0wyemNxd1NzWmc2TWx0cjUyeTgzQVppemhiY0l6RWpo?=
 =?utf-8?B?RkZCUW1pVUlKZm9COTltbGdIWTBHOE8rcHNISXNvYmx0VzhpVzAwSGQ2S2pS?=
 =?utf-8?B?VXNnUlBhaFczZFJjeXJ2TVc4THdoVEFSeUJGZit2K3FLL1RhTTc5Mm94ZE5w?=
 =?utf-8?B?Vm52ZVpsMUhUcVpCeXJJYkpETjBTbm8vUGJrZ3YrcGlTTmc5ZTd0WXh3WDVY?=
 =?utf-8?B?OHJxUHc2V28ycm9LZFQxcC9FTkphSDhvNWJyalhSdnpwS2JVU3NOeloreXkw?=
 =?utf-8?Q?RAJM8uteG/mLXMwPWgliTZLTIAgD4SOdU7Kdysj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 28c134b1-4b59-425d-1804-08d97cd01987
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:19:07.4200
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: niCjkYQ3aKdW4oQpO0dESCxLK60zKsFooAqyUCZwDg61+IXks1lCXKIik02e1cZNs3sEyTy/QCzeLTBe5gyLcg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

vcpu_show_registers() didn't do anything for HVM so far. Note though
that some extra hackery is needed for VMX - see the code comment.

Note further that the show_guest_stack() invocation is left alone here:
While strictly speaking guest_kernel_mode() should be predicated by a
PV / !HVM check, show_guest_stack() itself will bail immediately for
HVM.

While there and despite not being PVH-specific, take the opportunity and
filter offline vCPU-s: There's not really any register state associated
with them, so avoid spamming the log with useless information while
still leaving an indication of the fact.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I was pondering whether to also have the VMCS/VMCB dumped for every
vCPU, to present full state. The downside is that for larger systems
this would be a lot of output.
---
v2: New.

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -631,6 +631,12 @@ void vcpu_show_execution_state(struct vc
 {
     unsigned long flags;
 
+    if ( test_bit(_VPF_down, &v->pause_flags) )
+    {
+        printk("*** %pv is offline ***\n", v);
+        return;
+    }
+
     printk("*** Dumping Dom%d vcpu#%d state: ***\n",
            v->domain->domain_id, v->vcpu_id);
 
@@ -642,6 +648,21 @@ void vcpu_show_execution_state(struct vc
 
     vcpu_pause(v); /* acceptably dangerous */
 
+#ifdef CONFIG_HVM
+    /*
+     * For VMX special care is needed: Reading some of the register state will
+     * require VMCS accesses. Engaging foreign VMCSes involves acquiring of a
+     * lock, which check_lock() would object to when done from an IRQs-disabled
+     * region. Despite this being a layering violation, engage the VMCS right
+     * here. This then also avoids doing so several times in close succession.
+     */
+    if ( cpu_has_vmx && is_hvm_vcpu(v) )
+    {
+        ASSERT(!in_irq());
+        vmx_vmcs_enter(v);
+    }
+#endif
+
     /* Prevent interleaving of output. */
     flags = console_lock_recursive_irqsave();
 
@@ -651,6 +672,11 @@ void vcpu_show_execution_state(struct vc
 
     console_unlock_recursive_irqrestore(flags);
 
+#ifdef CONFIG_HVM
+    if ( cpu_has_vmx && is_hvm_vcpu(v) )
+        vmx_vmcs_exit(v);
+#endif
+
     vcpu_unpause(v);
 }
 
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -49,6 +49,39 @@ static void read_registers(struct cpu_us
     crs[7] = read_gs_shadow();
 }
 
+static void get_hvm_registers(struct vcpu *v, struct cpu_user_regs *regs,
+                              unsigned long crs[8])
+{
+    struct segment_register sreg;
+
+    crs[0] = v->arch.hvm.guest_cr[0];
+    crs[2] = v->arch.hvm.guest_cr[2];
+    crs[3] = v->arch.hvm.guest_cr[3];
+    crs[4] = v->arch.hvm.guest_cr[4];
+
+    hvm_get_segment_register(v, x86_seg_cs, &sreg);
+    regs->cs = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_ds, &sreg);
+    regs->ds = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_es, &sreg);
+    regs->es = sreg.sel;
+
+    hvm_get_segment_register(v, x86_seg_fs, &sreg);
+    regs->fs = sreg.sel;
+    crs[5] = sreg.base;
+
+    hvm_get_segment_register(v, x86_seg_gs, &sreg);
+    regs->gs = sreg.sel;
+    crs[6] = sreg.base;
+
+    hvm_get_segment_register(v, x86_seg_ss, &sreg);
+    regs->ss = sreg.sel;
+
+    crs[7] = hvm_get_shadow_gs_base(v);
+}
+
 static void _show_registers(
     const struct cpu_user_regs *regs, unsigned long crs[8],
     enum context context, const struct vcpu *v)
@@ -99,27 +132,8 @@ void show_registers(const struct cpu_use
 
     if ( guest_mode(regs) && is_hvm_vcpu(v) )
     {
-        struct segment_register sreg;
+        get_hvm_registers(v, &fault_regs, fault_crs);
         context = CTXT_hvm_guest;
-        fault_crs[0] = v->arch.hvm.guest_cr[0];
-        fault_crs[2] = v->arch.hvm.guest_cr[2];
-        fault_crs[3] = v->arch.hvm.guest_cr[3];
-        fault_crs[4] = v->arch.hvm.guest_cr[4];
-        hvm_get_segment_register(v, x86_seg_cs, &sreg);
-        fault_regs.cs = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_ds, &sreg);
-        fault_regs.ds = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_es, &sreg);
-        fault_regs.es = sreg.sel;
-        hvm_get_segment_register(v, x86_seg_fs, &sreg);
-        fault_regs.fs = sreg.sel;
-        fault_crs[5] = sreg.base;
-        hvm_get_segment_register(v, x86_seg_gs, &sreg);
-        fault_regs.gs = sreg.sel;
-        fault_crs[6] = sreg.base;
-        hvm_get_segment_register(v, x86_seg_ss, &sreg);
-        fault_regs.ss = sreg.sel;
-        fault_crs[7] = hvm_get_shadow_gs_base(v);
     }
     else
     {
@@ -159,24 +173,35 @@ void show_registers(const struct cpu_use
 void vcpu_show_registers(const struct vcpu *v)
 {
     const struct cpu_user_regs *regs = &v->arch.user_regs;
-    bool kernel = guest_kernel_mode(v, regs);
+    struct cpu_user_regs aux_regs;
+    enum context context;
     unsigned long crs[8];
 
-    /* Only handle PV guests for now */
-    if ( !is_pv_vcpu(v) )
-        return;
-
-    crs[0] = v->arch.pv.ctrlreg[0];
-    crs[2] = arch_get_cr2(v);
-    crs[3] = pagetable_get_paddr(kernel ?
-                                 v->arch.guest_table :
-                                 v->arch.guest_table_user);
-    crs[4] = v->arch.pv.ctrlreg[4];
-    crs[5] = v->arch.pv.fs_base;
-    crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
-    crs[7 - !kernel] = v->arch.pv.gs_base_user;
+    if ( is_hvm_vcpu(v) )
+    {
+        aux_regs = *regs;
+        get_hvm_registers(v->domain->vcpu[v->vcpu_id], &aux_regs, crs);
+        regs = &aux_regs;
+        context = CTXT_hvm_guest;
+    }
+    else
+    {
+        bool kernel = guest_kernel_mode(v, regs);
+
+        crs[0] = v->arch.pv.ctrlreg[0];
+        crs[2] = arch_get_cr2(v);
+        crs[3] = pagetable_get_paddr(kernel ?
+                                     v->arch.guest_table :
+                                     v->arch.guest_table_user);
+        crs[4] = v->arch.pv.ctrlreg[4];
+        crs[5] = v->arch.pv.fs_base;
+        crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
+        crs[7 - !kernel] = v->arch.pv.gs_base_user;
+
+        context = CTXT_pv_guest;
+    }
 
-    _show_registers(regs, crs, CTXT_pv_guest, v);
+    _show_registers(regs, crs, context, v);
 }
 
 void show_page_walk(unsigned long addr)



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:19:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:19:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191432.341460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa48-0006GY-25; Tue, 21 Sep 2021 07:19:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191432.341460; Tue, 21 Sep 2021 07:19:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa47-0006GP-Ux; Tue, 21 Sep 2021 07:19:43 +0000
Received: by outflank-mailman (input) for mailman id 191432;
 Tue, 21 Sep 2021 07:19:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa47-0006G8-3S
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:19:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 49b1ca9e-1aac-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:19:42 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-o06BvGRINnOqe1Or22CsTQ-1;
 Tue, 21 Sep 2021 09:19:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:19:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:19:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0064.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Tue, 21 Sep 2021 07:19:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49b1ca9e-1aac-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208781;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R/fFnczmCtwlapERXxOX/XNhHEnAVEjvjTgMNNwr6tw=;
	b=d8hxIQ3+Zjs9V0ufk1WdwDW13lp3CSuRyeVi4v+7Sfe9SAbtz71Q4LHm2KdBh7NeWTTVAy
	ocevN2A6/LvLA3GTPj1+BaBGK7no2a5ZEZEW1KdOiT9BaxNoU+6e2nKMgkBfvO4uVk4AZS
	hqPSBnCB2C2yWymoGYgKs5GjGbdd0gM=
X-MC-Unique: o06BvGRINnOqe1Or22CsTQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vw0PD9v0hk/jRxeWzor461xYslltKuCVvmUzNIEgkwGAaEWQuQYTd+zjflwge69YRjZSPzPqfTArWCLE09YZyF+Sjt6v0MR0ukdPQ5Jjj7WEDgd5kXG6qrndCg6Riotfhrc/ZMkcLg50XC0hphVwC7VUPJrp2MtB1/vaU5BEl5XhphwBP+GVFz2EIl8iaKboeOdU62HvFfij8Z+toX49ovF8YQpo6ZPLMkUcx44Nn/NqIvSSsc8y/gHj9iqhPoQJhL2fHpWsBCSZaYS15AYzVcOpEuSduNRmLz8mqm2Lfqmfx4ZOBfgjBsf2Rd19I/LXR1T1dsp+dYjIZVBoaFt6ow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R/fFnczmCtwlapERXxOX/XNhHEnAVEjvjTgMNNwr6tw=;
 b=PScBIewm7KpmAyqZSfiMHy9k2LWhGe6dIBpaXajAa4/yVbYp14spCbLbVuS2TDTUiAg5ZnvSOczCYhWjcT8MfYAaAxskKwg21ZC5I1bD02Oti5xfpAZTOyZKhAztCpHM8hlIlJFqYke87rGGPlM6s+XVP+JzVdXFOM4r4dut9HgHMlW+UWRTVZgBK1mfKlHmdj0NilGBwZ16hLlJ5RYlXlzwi2iFgFp9JatSrgsot+5sIWUhXW2IuyKvawJK7+HG9E103qOnFxfEc3loC6z1YrIwTTZo0sYRRi1VK3ZeC9GKTw/AOl4jfRGKvaIVgP1DDbQsARyt3b5W4JAwXPoB8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 6/9] x86/HVM: convert hvm_virtual_to_linear_addr() to be
 remote-capable
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <3397b5ea-bc21-5a2c-ba5d-afc974a92df4@suse.com>
Date: Tue, 21 Sep 2021 09:19:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0064.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 72fb8069-fc20-4153-7ac5-08d97cd02bee
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26067874CC4144E593081B09B3A19@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TJaB1RuJ1J1TQQJVuAIWYy1Lstkt45BXC8DsTjD1KvItYVmqDHMaGVe9XhXNCDzqX4l7qDFm2Q9qxb0qWhd8wCGTeqPp6SRV0ty7bbwZGUE8nLotJ0Vs/foJQvVJXIVeFR4yYQRDdtlXKP1r0czwN+g5L3lpDmXXSm+y3xTgyEt8R+y2ycJjcswjF75kmxP18I9fRiVShgpcFARQ6d8FACbA+07mRQB+ZkdHWJg7K5uv8ExhAqj0KmgQ46aRaQ2sEoWPCK+fwz9YFKehHyoXGqj92WgqwDHyYsMVrngmJo+HP45GIg6PEnsi/PBaIOeOKd589ly5pVdTRcV7ok/3/ai2hjerGraB809PdYUIILZyV+fhkxaxlPpcKBwMDyzc1iFjtqtlnmKSrpWKzyLWAose+e+4gCgHbEzXO/pE24sLDwNV+knG6yJjk5E/q0b6ebnzP1PhCj2eDWlces5ZTCz37hipjj9q45qt2RRHw5nRRj9VRPn8VI4nGvXIl4eSvNatDkRTD9d2epVU4gsmo3MqrtBWgwqIGOSQ+0dNgla/mTWUr3/nkf2dIA6F7UyoBpv2hPny0NB5a4hqEHIfoAVkzlI3aIK1b3RnkTFJ+asIm0nsJGT8i5DjsKe8cAeKmmqidAvr2i8QlrM86LE4I0OHOnzMRCZQM45feq9g+y5VKNMy0gARvsmDzjiTRu24WQm2TEvjQvPYC+uzI6BAd5KGeXe0XEPeyhE8egt0a38=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(31696002)(508600001)(6916009)(38100700002)(2616005)(4326008)(83380400001)(66556008)(66476007)(316002)(5660300002)(16576012)(26005)(54906003)(86362001)(8676002)(31686004)(956004)(8936002)(186003)(2906002)(66946007)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VTZDZ1JSOEwySEwxZVhNY0FxelNBVW1wMUEyc1ZJbmZSRWRvc2IrbTJidHZj?=
 =?utf-8?B?ckd6cTZiaGJuVU96VmhXb1gvKzhJTi9TTUJTbVZ4ejJvOTZZK0NqTTN6MHQ5?=
 =?utf-8?B?NjRFd29BaEdHdFhsOEpqd3VTRGx1QkUwT3FjMExtSzlKU2V2L0ZuQlhqYlVh?=
 =?utf-8?B?Vk9pYnptZ2ZrWG1UdEVnU0dSMG9KMDJpdWFJTmNkYURxVlZON2d2aW1OZHg1?=
 =?utf-8?B?Z1U3NUlad1dBZUFoRFAzeTRFRDNHZFRGd3dnNzY3N1BSNXMxYmZEbjU4NnJK?=
 =?utf-8?B?ZFRKTVVtcGpiS3J6c0JqK3pXdmFabnd2NU56ZHNRcTVGd01FS2FQTTRGUFhM?=
 =?utf-8?B?aDkrQ0c1UFlpcTdpZkZFbTNQY3Z4cnVzSTFrSUozc0RoNlE5RWtFZ2I3cENq?=
 =?utf-8?B?Vkx3eG1wa0FjZFd2SHVlK3h3dFFqakFJdncxYWhPTDRYOVZYRmhPUXBQRi84?=
 =?utf-8?B?WDFJYnBmTkU4UDVtM3ZTcE8yU1loUDVSOHc5ODkxSytURW1yL2JjNm00eGty?=
 =?utf-8?B?TVN5UzM5WUt0SnlsN2xDTWlya3dIbGU1K0ZqRCtVK3ZWMmE5ODVWaG1zcHhs?=
 =?utf-8?B?a3J4VXFGUkNSTFRqY3puSy9aQng4a1VhK09IQjlTaXdsc1hlclhJY0ZFckN0?=
 =?utf-8?B?cmxDSllOY3VVYmpyUzJDaG44VENlckZybytmRkFTZ2JiYnlOYktXbis0RkRS?=
 =?utf-8?B?Y3FqR0Q2RXIya2JsakgwcCtUWlZBU2RhZ1JVS2lBQnVWUzF1aG9EdlgzRVlL?=
 =?utf-8?B?dzIwQVpOMUd4QitQZDh6bUJ4c1Ara0RBc3A2WUZCcDg1cE5rOUs3MUt1QWJK?=
 =?utf-8?B?Q2JDWlhnR2Z0cWVEY3BXR0tJNnk2OUV4SEN5UVhpZkxpS291cWthb3hKa2lL?=
 =?utf-8?B?cFNlaVBybWcxNjJqcWlvQkJOWkRQeitCekJIa0VheXF0c2o0Q0R6TTd3Uk8w?=
 =?utf-8?B?SWtjd1RIQXozeENNWTJKMkNzMENvZGtrYjhFdmRFbmxSbnB2WFlOc3VuZ0sy?=
 =?utf-8?B?N1JtZGpzd0lvdVZpd1IxVWVaQ0h1amJZZHBocWpzL0N6V3B1MytvZ05zeXBQ?=
 =?utf-8?B?N2x0SjJLV3dkNHdGdWhYTlNrU2JHaTl6MjJoL3hZRE0ra3l0dG53VmorVkdl?=
 =?utf-8?B?Sk52NHFxZXc5Z2hLU0tPYUI3V240S0RIbzhKRVVhUGtZa3JzMmFJNm8yNlhN?=
 =?utf-8?B?aUZWM1dtUHh2R08zb0VlVHlPdkZSb012RFlmMTVYclBVcW5uOFBKMEpSZUY4?=
 =?utf-8?B?MzhlUDB4Z2pVYkdOcDIrSEp4ZDZkejhCVHlaMHZ4bjZmZWx3bTZ1Q3hxM1JQ?=
 =?utf-8?B?U2VlY1lOelk2cFZETDV2aENMdy8xTXg3SWl5RFl6Z1RRR2xwSllldUVuMERE?=
 =?utf-8?B?RzRGbXZZUVVFOXZKMExqUVV0VlZmZ1FQMHhiK3ZGeWwwSGpOQnVKdUhwZjBZ?=
 =?utf-8?B?NVRDMmNNaU5aQ0pjdWtmSEpjUDZ3Qm42c3dUR1hGYTZlOG1jMlNUQnBjWU95?=
 =?utf-8?B?ci9JYnluUEI2T3M4dXMvSFB5TXQ2SmtaWnhNbWRBc0ZDOE14UnNKdjlNR2dS?=
 =?utf-8?B?N0VYZU5vaE41dUJKQ2JoTEMxRER1WGFmNjNjSGNNMmtxb1hNQVVKMHM2Wmdo?=
 =?utf-8?B?cUhEN1V3T2VnZFgwK3M1eGowaVp6ZWZrdUw3NERkY0xwTDZiK1hBQUhwcVJu?=
 =?utf-8?B?d3llNVF6NkJFaUVpVDZRM0JGQ1FjU0lnOUdXTzF1SjBZTjNYV2RLK2h3SG45?=
 =?utf-8?Q?EY8kKN4BORMZ/F6qciG0TM/27rcPNIB/HFhAgkO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 72fb8069-fc20-4153-7ac5-08d97cd02bee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:19:38.2846
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fpgVTVTSozs3rU/tqjja6EY4bOmQ/8bVed01uRMqnAGv7w4Px6N0MICPRM1KaCQzTzwZC8/c2+At0QdMUFcgdQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

While all present callers want to act on "current", stack dumping for
HVM vCPU-s will require the function to be able to act on a remote vCPU.
To avoid touching all present callers, convert the existing function to
an inline wrapper around the extend new one.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Alternatively the actual dumping patch could avoid using this more
elaborate function and, ignoring access checks, simply add in the SS
segment base itself (if needed in the first place).
---
v3: New.

--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -2526,7 +2526,8 @@ int hvm_set_cr4(unsigned long value, boo
     return X86EMUL_OKAY;
 }
 
-bool_t hvm_virtual_to_linear_addr(
+bool hvm_vcpu_virtual_to_linear(
+    struct vcpu *v,
     enum x86_segment seg,
     const struct segment_register *reg,
     unsigned long offset,
@@ -2535,8 +2536,9 @@ bool_t hvm_virtual_to_linear_addr(
     const struct segment_register *active_cs,
     unsigned long *linear_addr)
 {
-    const struct vcpu *curr = current;
     unsigned long addr = offset, last_byte;
+    const struct cpu_user_regs *regs = v == current ? guest_cpu_user_regs()
+                                                    : &v->arch.user_regs;
     bool_t okay = 0;
 
     /*
@@ -2547,7 +2549,7 @@ bool_t hvm_virtual_to_linear_addr(
      */
     ASSERT(seg < x86_seg_none);
 
-    if ( !(curr->arch.hvm.guest_cr[0] & X86_CR0_PE) )
+    if ( !(v->arch.hvm.guest_cr[0] & X86_CR0_PE) )
     {
         /*
          * REAL MODE: Don't bother with segment access checks.
@@ -2555,7 +2557,7 @@ bool_t hvm_virtual_to_linear_addr(
          */
         addr = (uint32_t)(addr + reg->base);
     }
-    else if ( (guest_cpu_user_regs()->eflags & X86_EFLAGS_VM) &&
+    else if ( (regs->eflags & X86_EFLAGS_VM) &&
               is_x86_user_segment(seg) )
     {
         /* VM86 MODE: Fixed 64k limits on all user segments. */
@@ -2564,7 +2566,7 @@ bool_t hvm_virtual_to_linear_addr(
         if ( max(offset, last_byte) >> 16 )
             goto out;
     }
-    else if ( hvm_long_mode_active(curr) &&
+    else if ( hvm_long_mode_active(v) &&
               (is_x86_system_segment(seg) || active_cs->l) )
     {
         /*
@@ -2636,7 +2638,7 @@ bool_t hvm_virtual_to_linear_addr(
         else if ( last_byte > reg->limit )
             goto out; /* last byte is beyond limit */
         else if ( last_byte < offset &&
-                  curr->domain->arch.cpuid->x86_vendor == X86_VENDOR_AMD )
+                  v->domain->arch.cpuid->x86_vendor == X86_VENDOR_AMD )
             goto out; /* access wraps */
     }
 
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -314,7 +314,9 @@ enum hvm_access_type {
     hvm_access_read,
     hvm_access_write
 };
-bool_t hvm_virtual_to_linear_addr(
+
+bool hvm_vcpu_virtual_to_linear(
+    struct vcpu *v,
     enum x86_segment seg,
     const struct segment_register *reg,
     unsigned long offset,
@@ -323,6 +325,19 @@ bool_t hvm_virtual_to_linear_addr(
     const struct segment_register *active_cs,
     unsigned long *linear_addr);
 
+static inline bool hvm_virtual_to_linear_addr(
+    enum x86_segment seg,
+    const struct segment_register *reg,
+    unsigned long offset,
+    unsigned int bytes,
+    enum hvm_access_type access_type,
+    const struct segment_register *active_cs,
+    unsigned long *linear)
+{
+    return hvm_vcpu_virtual_to_linear(current, seg, reg, offset, bytes,
+                                      access_type, active_cs, linear);
+}
+
 void *hvm_map_guest_frame_rw(unsigned long gfn, bool_t permanent,
                              bool_t *writable);
 void *hvm_map_guest_frame_ro(unsigned long gfn, bool_t permanent);



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:20:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:20:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191441.341471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa4W-0007cJ-Em; Tue, 21 Sep 2021 07:20:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191441.341471; Tue, 21 Sep 2021 07:20:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa4W-0007cC-B0; Tue, 21 Sep 2021 07:20:08 +0000
Received: by outflank-mailman (input) for mailman id 191441;
 Tue, 21 Sep 2021 07:20:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa4U-0006mE-T7
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:20:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea372172-ccdb-4c2f-90ab-494e8ddfb4fd;
 Tue, 21 Sep 2021 07:20:05 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-37--c95hKXTPcSNhYd3da-yQQ-1; Tue, 21 Sep 2021 09:20:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6671.eurprd04.prod.outlook.com (2603:10a6:803:11f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 07:20:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:20:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0085.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Tue, 21 Sep 2021 07:20:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea372172-ccdb-4c2f-90ab-494e8ddfb4fd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208804;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MZzQEVSwMvpFC6oSCiycTUs10fXX3566gSFd9Uc4olw=;
	b=JXP0PggS2U3EhnABGu1I2a2qo5wFco509ahINW49DEFxO594EwdSbfIjw62jI7CJBshKLt
	HJiWUjwu3bMdVEMCA6FPYPHKbFEqldpB2zfqsHPBNcG+gLt0Sq4X1jLXOZ035bVCeMpp4F
	4Sj78WSLK/4hHT6phu4Uj9xnDgdrPtY=
X-MC-Unique: -c95hKXTPcSNhYd3da-yQQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=m+wLz7rUWazxE3ro99PvPgrXKeZg+rN2omAHPbRjJ1TNh0fU4BpYmBQBFvsbnSBvgF07flbm+kRHW8u4U8gpxqyD0qrvFHgv6u9Zx4kBMkakvTiqvBExuf5rwEjFgIqBY6ZXOSU1QUIId1FyZLHGT9KDYrAXULO84/2pVq6jY32WbBKrieh3pg2Rbg6EiFe+fznTa6igeEskY3zlI8/0apqv6aVZZViVovqKIkoS+mt5COOWukCvXCFn/f9RQiqu2WAIgUQSJ6sfsSEUD+XSwUgAGvvnkJ+94t+AoeWk0ZnUGOnDP1YkUxV0l/P2YQyKWbI629nAxhTA9m/LeDZPjA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MZzQEVSwMvpFC6oSCiycTUs10fXX3566gSFd9Uc4olw=;
 b=K3y5uFsuNLVTcVxz8jQOixhnLy6nrnqZLhAsbdWRnVdaaIN2fz+sWSy7p6Yc4IZH3gAQUMzcj6GplTjbfk44sFgfJOromeZIDKmMufd1Z44w9sae38df8MC4ietFPPXzN3HjvoXtekk4DlAJ9cP17hQzLbWkjgDxIk9G8IFpAbx/K6s74Vh1k6WoXTHNYNkOsLjUXZgV3Vxadk6AfL+odkWkiKQR1SeycLc63Yku9ogPDh4yc9GrkHSlWsMobsei1bFe6S6tfW8k6Wz7bBJZT4Ec5i+q3ThyI6yxVDGC+aqpXU/6fR28S2oeoRMmp8c+B87VgB+1NyAwpvHk9+Hycw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 7/9] x86/PVH: actually show Dom0's stacks from debug key
 '0'
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
Date: Tue, 21 Sep 2021 09:20:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0085.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 70b068cf-b07a-4023-e6c1-08d97cd039a6
X-MS-TrafficTypeDiagnostic: VE1PR04MB6671:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB667129AED513FA65383119BAB3A19@VE1PR04MB6671.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ypZOEBrTsAT+Duv6+keWmuQFg/TsTMyfj7JJSuAvmLeKnZXxIsjHwPqRsrqt1+38Hu13BqX4I9F4fSIPmdE8pAVHA/qNGbaGl9o4+m5c56zSLIhdYE2vqIFbWl8x8pgOYM8lMsIcV9nk71H4guHg6cQzIQmwugOixAtVGVHgpRNufNEdhD+wzR8kwk5Er+UgdpwMVbXwmjMKQnnzqqLcH2GHcReUWfsU3RIQ/HH+5RihhzqpvP4wT+qtUHRO52PGiraa6ogXY9bTpUfI24kSoUiSczPCufOFk+RQfjvk1u+7rAU1q7CIImcJ9ijin2xM9d/pI1prkB2NfIM5Ub7VyiVtlJ3DNxwZ1NS7++mtU/h9s4XZoC5jeLpHReDW4Itcoa8jtjNCjxFf2bzUdS5m1YVtXHJ7JRq0d+2fUZJ5ww/CnGbgF+GMnhezRPLsqtitbLyvz5QYw/yeFt+EKdMukNns0KzP4x4EvEWR3jdTXJaOSoaDe9t9bkBaPt65HhJtpkbVwOJq8hor+FVhpOBHjpbaRc/BGY+FGmsV07t4658vKbBettuDndbjV0u2TYmsMKBLjbptE/5rJE5WsJcuSkKDcnLbu5O905bsEydC6nv3uzYtcplV/S/2zg26PKpDEkZrUOv3qlOS0oE3pQ+VzdkyEqZxJq033G2YpiM3aP8rGgdsjZIwomlnniAJur0pMIqIpmS/q5XZWhVmiqSs1nMWa3BhO0bzWISCjFK0vYc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(5660300002)(36756003)(54906003)(4326008)(8936002)(956004)(31686004)(83380400001)(8676002)(2906002)(6486002)(2616005)(26005)(316002)(38100700002)(186003)(31696002)(508600001)(66476007)(66556008)(86362001)(6916009)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M1VMbVh2eldHTlZicUtjNG91T0hLKytwTGkvRkZ5ZzRjUXQ1L1g5QVBSQzdN?=
 =?utf-8?B?RzNwcFJieWJnTHNYNnpSQVMyaHlNejUyeGZ3V2owTjBZVi9VVkJDSlF4a0dJ?=
 =?utf-8?B?VVV2T2hiNXRwY3ZrdmFOTXh3blBvT1FlWU1saTdjZGNoNy9wVS80cDFtWWIz?=
 =?utf-8?B?Y08vanZPSm1UdFl0eGxER3laSENGT1VJUldWQ3JvY3JPYmhrbWJGVC9TVmJM?=
 =?utf-8?B?UWNhaUxNckhrVDFTVHhxcjFBWlRVQmY1M3YxQndKZEQ4QjhYQUlxYVBkQTBt?=
 =?utf-8?B?WE1qRWtFK3R4djR0bDA5T2RXNmEybU93TGFCK2NGdVE3RTl1UFphTGZkOUdU?=
 =?utf-8?B?bkNOTVpuNnhKRU9NaVVBRjRWbXp6RG5PRVBZSkNPelE4R0diZ01Rc21EbHIy?=
 =?utf-8?B?UjhMallrdkRIeWJtdUpRZFUvM3pZUTBIem9pOFA5WDRna3ExZEdlbmovcmx6?=
 =?utf-8?B?QlVzUHEzbU1Hc2NVQ2NJNjlYK3JhMnhWRlBJeDk3MXNGakVBbSswM2NUeC95?=
 =?utf-8?B?YTIvV1ZCdjUyc3cyWXIra29FOUtFc0wyaFZUU05oYjZyY3QzbVlhbTlpYWxo?=
 =?utf-8?B?bzVFbUJveUJReWpTSjFib0V4S3NLaEt0clFrQ3RLNm9tcVFzWGlqWnRVRk5H?=
 =?utf-8?B?VldjRE9HMDlKL1hkdVc3L1VPZE5kVzJjVHhEQWVkYVpyYUk2dUE3K1lhMmdp?=
 =?utf-8?B?QkN4aGlIRzFSODBRV2gyNCtjV1ZWMkF1WEtrRU5KcmhGU01aOU4xdzdhRlpl?=
 =?utf-8?B?eUQxWGZ4NURMMmpvREJYZU5QMHdPT3IxSVowU05xOEVGWjdaUlZhODNySVFq?=
 =?utf-8?B?ZmhRYUhWVFRxT3ROZDR2M3VKTDJHbjBOZDRjNmw4U0VVNkZWcyt2TlhHZzYy?=
 =?utf-8?B?MXdnNmJ4dzAzUmxwaWJQNVdHYTh4V3B1Q3ZFa05YNGlDRDh6cXBraHFBY1VN?=
 =?utf-8?B?UW9SSnAwdzVsUjN6RU9QTFhlL1VVcDd1YmxmZnBaQ2hBbktveHNFS09NRHEr?=
 =?utf-8?B?RnJrZ2hYdTByZlBqYTFIVVhOcnZQaUFlZ0xPM0pZdklUeC92WjIzWW9qczNT?=
 =?utf-8?B?WWdPeDZJWmh6NWo5MVg5WldUQisvSnRnekp3SVNTZXZCc3QzTTZZR29pMkFX?=
 =?utf-8?B?T1l6SEp5QzFQN3l1MDA5N0VBTkZEb285UGpiSnVERDhLTzhTQ1NJTFI4d3NL?=
 =?utf-8?B?SFh2U1ArQmx6Y3lsbGk4OThPS1dwK2g1bFRKRTZuWmdKaHpGMlRXdk1QaDBD?=
 =?utf-8?B?empMTURLTlNJcFpkQ080SUk1OUo4L0kybzdmVVhRa0hINjk5VDgyL2gyR2Mw?=
 =?utf-8?B?cjU1ZEtmQXBCSzROTUdsTE5VSUw5UWR2cmdtR1NMUTJPWlJ5OTBNUldnYU9m?=
 =?utf-8?B?bGVhT3o1TGtYbG10VldSU1Rpb2dOYVQvQVVPWWx0UTZySmNkSWRER3ZxeHlQ?=
 =?utf-8?B?Mld3RzZTdW9ZUms5R0R5NkE3Q3NZTVl4TTZKaGRlc29OZTVkajBkbDZKUXMx?=
 =?utf-8?B?Yk9UcDM2M01NMHZnbTYyMit0UkZNdGs2Z09YNmNXTk1FRlBWaERNUTNWRUVD?=
 =?utf-8?B?SUE0VkNkUnVMc0ZZZnIyK093SXhqc2NzNll0OXNPeDhzODZiOVJyV0g0QTRm?=
 =?utf-8?B?MjM0R0t6NHJQVkJIb1YrQ2k4YkZBTWtpUVdQYitYOHVDRGlOS21vRjJRUzA1?=
 =?utf-8?B?b1A1eWNzM1RMSVNnbnJhaDdlVGV3OEs4bHF3L3cxdXZMZzM3bU4yUXlNQTZS?=
 =?utf-8?Q?pIq+z1F3D8qju8Zs/YtURyeAL4pQ8xK5QtSSEAm?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 70b068cf-b07a-4023-e6c1-08d97cd039a6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:20:01.3126
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hL/QSQ+Zpz0CjVJwdWTX2QIwh5ASu6sMr8v45kLAsiVJlOH4Uyju4S/xSCTE6SXYttuSJjGRKMa+cWg4lG/hog==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6671

show_guest_stack() does nothing for HVM. Introduce a HVM-specific
dumping function, paralleling the 64- and 32-bit PV ones. We don't know
the real stack size, so only dump up to the next page boundary.

Rather than adding a vcpu parameter to hvm_copy_from_guest_linear(),
introduce hvm_copy_from_vcpu_linear() which - for now at least - in
return won't need a "pfinfo" parameter.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: The bypassing of the output interleaving avoidance isn't nice, but
     I've not been able to think of an alternative. Avoiding the call to
     hvm_vcpu_virtual_to_linear() would be in principle possible (adding
     in the SS base directly), but one way or another we need to access
     guest memory and hence can't sensibly avoid using the P2M layer.
     However, commit 0996e0f38540 ("x86/traps: prevent interleaving of
     concurrent cpu state dumps") introduced this logic here while
     really only talking about show_execution_state().
     vcpu_show_execution_state() is imo much less prone to interleaving
     of its output: It's uses from the keyhandler are sequential already
     anyway, and the only other use is from hvm_triple_fault(). Instead
     of making the locking conditional, it may therefore be an option to
     drop it again altogether.
TBD: For now this dumps also user mode stacks. We may want to restrict
     this.
TBD: An alternative to putting this next to {,compat_}show_guest_stack()
     is to put it in hvm.c, eliminating the need to introduce
     hvm_copy_from_vcpu_linear(), but then requiring extra parameters to
     be passed.
TBD: Technically this makes unnecessary the earlier added entering/
     leaving if the VMCS. Yet to avoid a series of non-trivial
     enter/exit pairs, I think leaving that in is still beneficial. In
     which case here perhaps merely the associate comment may want
     tweaking.
---
v3: New.

--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3408,6 +3408,15 @@ enum hvm_translation_result hvm_copy_fro
                       PFEC_page_present | pfec, pfinfo);
 }
 
+enum hvm_translation_result hvm_copy_from_vcpu_linear(
+    void *buf, unsigned long addr, unsigned int size, struct vcpu *v,
+    unsigned int pfec)
+{
+    return __hvm_copy(buf, addr, size, v,
+                      HVMCOPY_from_guest | HVMCOPY_linear,
+                      PFEC_page_present | pfec, NULL);
+}
+
 unsigned int copy_to_user_hvm(void *to, const void *from, unsigned int len)
 {
     int rc;
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -364,6 +364,71 @@ static void show_guest_stack(struct vcpu
     printk("\n");
 }
 
+static void show_hvm_stack(struct vcpu *v, const struct cpu_user_regs *regs)
+{
+#ifdef CONFIG_HVM
+    unsigned long sp = regs->rsp, addr;
+    unsigned int i, bytes, words_per_line, pfec = PFEC_page_present;
+    struct segment_register ss, cs;
+
+    hvm_get_segment_register(v, x86_seg_ss, &ss);
+    hvm_get_segment_register(v, x86_seg_cs, &cs);
+
+    if ( hvm_long_mode_active(v) && cs.l )
+        i = 16, bytes = 8;
+    else
+    {
+        sp = ss.db ? (uint32_t)sp : (uint16_t)sp;
+        i = ss.db ? 8 : 4;
+        bytes = cs.db ? 4 : 2;
+    }
+
+    if ( bytes == 8 || (ss.db && !ss.base) )
+        printk("Guest stack trace from sp=%0*lx:", i, sp);
+    else
+        printk("Guest stack trace from ss:sp=%04x:%0*lx:", ss.sel, i, sp);
+
+    if ( !hvm_vcpu_virtual_to_linear(v, x86_seg_ss, &ss, sp, bytes,
+                                     hvm_access_read, &cs, &addr) )
+    {
+        printk(" Guest-inaccessible memory\n");
+        return;
+    }
+
+    if ( ss.dpl == 3 )
+        pfec |= PFEC_user_mode;
+
+    words_per_line = stack_words_per_line * (sizeof(void *) / bytes);
+    for ( i = 0; i < debug_stack_lines * words_per_line; )
+    {
+        unsigned long val = 0;
+
+        if ( (addr ^ (addr + bytes - 1)) & PAGE_SIZE )
+            break;
+
+        if ( !(i++ % words_per_line) )
+            printk("\n  ");
+
+        if ( hvm_copy_from_vcpu_linear(&val, addr, bytes, v,
+                                       pfec) != HVMTRANS_okay )
+        {
+            printk(" Fault while accessing guest memory.");
+            break;
+        }
+
+        printk(" %0*lx", 2 * bytes, val);
+
+        addr += bytes;
+        if ( !(addr & (PAGE_SIZE - 1)) )
+            break;
+    }
+
+    if ( !i )
+        printk(" Stack empty.");
+    printk("\n");
+#endif
+}
+
 /*
  * Notes for get_{stack,shstk}*_bottom() helpers
  *
@@ -629,7 +694,7 @@ void show_execution_state(const struct c
 
 void vcpu_show_execution_state(struct vcpu *v)
 {
-    unsigned long flags;
+    unsigned long flags = 0;
 
     if ( test_bit(_VPF_down, &v->pause_flags) )
     {
@@ -663,14 +728,22 @@ void vcpu_show_execution_state(struct vc
     }
 #endif
 
-    /* Prevent interleaving of output. */
-    flags = console_lock_recursive_irqsave();
+    /*
+     * Prevent interleaving of output if possible. For HVM we can't do so, as
+     * the necessary P2M lookups involve locking, which has to occur with IRQs
+     * enabled.
+     */
+    if ( !is_hvm_vcpu(v) )
+        flags = console_lock_recursive_irqsave();
 
     vcpu_show_registers(v);
-    if ( guest_kernel_mode(v, &v->arch.user_regs) )
+    if ( is_hvm_vcpu(v) )
+        show_hvm_stack(v, &v->arch.user_regs);
+    else if ( guest_kernel_mode(v, &v->arch.user_regs) )
         show_guest_stack(v, &v->arch.user_regs);
 
-    console_unlock_recursive_irqrestore(flags);
+    if ( !is_hvm_vcpu(v) )
+        console_unlock_recursive_irqrestore(flags);
 
 #ifdef CONFIG_HVM
     if ( cpu_has_vmx && is_hvm_vcpu(v) )
--- a/xen/include/asm-x86/hvm/support.h
+++ b/xen/include/asm-x86/hvm/support.h
@@ -101,6 +101,9 @@ enum hvm_translation_result hvm_copy_to_
 enum hvm_translation_result hvm_copy_from_guest_linear(
     void *buf, unsigned long addr, unsigned int size, uint32_t pfec,
     pagefault_info_t *pfinfo);
+enum hvm_translation_result hvm_copy_from_vcpu_linear(
+    void *buf, unsigned long addr, unsigned int size, struct vcpu *v,
+    unsigned int pfec);
 
 /*
  * Get a reference on the page under an HVM physical or linear address.  If



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:20:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:20:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191448.341481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa55-0008Eh-MO; Tue, 21 Sep 2021 07:20:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191448.341481; Tue, 21 Sep 2021 07:20:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa55-0008Ea-JQ; Tue, 21 Sep 2021 07:20:43 +0000
Received: by outflank-mailman (input) for mailman id 191448;
 Tue, 21 Sep 2021 07:20:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa54-0008Cx-9A
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:20:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f0407dc2-183f-450a-b795-8acad6bbd819;
 Tue, 21 Sep 2021 07:20:36 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-YjvEN7OzPWibYAIKPIA3Ig-1; Tue, 21 Sep 2021 09:20:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6671.eurprd04.prod.outlook.com (2603:10a6:803:11f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 07:20:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:20:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Tue, 21 Sep 2021 07:20:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0407dc2-183f-450a-b795-8acad6bbd819
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208835;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CbSaBreuvDjQILv0E6xjB0s8ZsBQ3dj8qUpxvESe3zI=;
	b=jPvUgl3iBo76LX2XzOrKWskJdjShK5UQYY4OcrOnztuhXqloEpItzsGulgPMJ9F/MQqWcT
	I6CIZogN73IY1ZYwIUVL9fxfE2WFvRu4pqMVVw2vxawXklT4MnmBQbNityreOcEd0MWMLb
	ZS1LayiELZk1CoxFFTrPbSOgTFho/jM=
X-MC-Unique: YjvEN7OzPWibYAIKPIA3Ig-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=B2LUX0fRf98D5C0FwNwL9PbS7MLydFyLc2nBVrxWhqNeWhyMcotO4TF5BCnrHIn1tQMdOf7ThpICfDeoKJu0/WFA7j0ux2dmw/MvSlDpg/cZbGoYnB2bdXym+xYbUURDnc84ycAvJv+S3B3PWcvTPkOEyWem/+jdOVegbm35JnOo6FAlJ2z/PSFMbchyamF7rFq+fUpSBCBXa+bqBffP1JXFR7+bl2wCPsBNsSL8DbK0z0Pov6UViUT4aOq+pOyXgXJd34gg/A+69qqJP8ftO8S6P41xChozLhegmSvONpKJt23FtrxNXYO79obOdSiwTispji0a8fQrfAT9U96w0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CbSaBreuvDjQILv0E6xjB0s8ZsBQ3dj8qUpxvESe3zI=;
 b=IkEnAcKI3UGrNqS9FrEQ3+KQroOdo4HZHmxOSqhPlqPIFEKd/hK0kRv+mMPPA1rky+dq9nD7MzLD23xOjsLQ4a7OpuBMN1Joo8OHA5woI1wX2asxkGsVbomgY8RYdT//RU/hWNheCkU1GsOmBbUs76PKEfuIxc3e5DbAZEDbDfdoT8KK/D5WHSin1453/ZvhAR1J+xVLRfT1rl9j5YzH/uznRObi8+Gx0rapo4AnAIucujWvH5ic1qbbjod+q1mdZrt4wHjxT74k0lXu+lH7A6fVW1eKrdDUycFZVx34DLt88GDG6bDFIm8gbKNvFMO5PPCtcPe1HK7hvSS8j0MDFQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 8/9] x86/HVM: skip offline vCPU-s when dumping VMCBs/VMCSes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <5cc8f7e0-399e-f6a5-bc84-823a6160a3f1@suse.com>
Date: Tue, 21 Sep 2021 09:20:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9ecc65e8-c135-4a84-0244-08d97cd04ca8
X-MS-TrafficTypeDiagnostic: VE1PR04MB6671:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6671D86909E909C3CD2104FDB3A19@VE1PR04MB6671.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:457;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PspKtaJDObcgXZuFO5ezZYYAMCqnCQ2LFc39jrxaWC9NbppUQE3K97BNrjXFFkCH3xEvrLuwfKpX81wNVBHAE+EDmUrk5oEp2IPRUfYjjJQBQCSLyIb3YtaX4DhEtGxsaMYRCaMJsFUN4sSm3a95FiSSpsUX7eVlIwWuJve8N+lOrMBCCUtbvJRc34kBdISL9vvq5dz7GqyS60MULdD4QRLSJWiF6T7rSAV3OB2Ii+8L38nJjE7tlK3qa3JmgBU+eMRaadh9DkNiL4Hrdn31mP8haTOyW+U7C4UU/sTGEI6BCL0Gq/9DEKbEs+NLPTfZf9P2iFuc9G+BVQKZ3dQ/PhFHVqbuwZxks9XSyca06bsHIiTSMDkFMq2JZkzMs6NnpIsDJWd6UZkIPqi8DlM5e5TXthuKfjikguRYAhOLWUjxhoYybnXzEAZSVkO+5Q3ppQcJ/Rg+J12znDfnH/6Yiiuwbb6R2qp270TqSbOUUnemp61SU7eAvwfjWMpiPaLiqE9cu+d0bUHMe0gYZpradihz6Sd9mZVBI2YPaTw/uIY6xEFmcegqQhR8qiE6Vj5gO1mgqKRA8SeSLMJDL4I0LJKxRdYp0WNDImfysap5ODgAHQ/URUty8P0i5B4e7nBgOEa1/kRxCSKakfcIA+N2JmAeNHblRNjQSrOi7qjDmf//Q1PhY7XmNuqIL1A+t1KE7SvSBkro196sfYcAAnjfddjSQ9xJFfnr36W0Z5iabB0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(5660300002)(36756003)(54906003)(4326008)(8936002)(956004)(31686004)(8676002)(2906002)(6486002)(2616005)(26005)(316002)(38100700002)(186003)(31696002)(508600001)(66476007)(66556008)(86362001)(6916009)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NGVYaENLcnJEbUVnRUszN3Bvb0htNnNheXQ5eGMxZnI2TWIvWUNTMDJvNDcw?=
 =?utf-8?B?RHJXSDRhblRUUVRQc1JSdktKM1lWaE5hMmt2aHRTcDEyYmNsYkgrMm5ld29U?=
 =?utf-8?B?ZjMxTGRVN2dQWmxBb01vZkFVUHZ2SzRnVlFOdzlNeHhVNHZLYVJGT1pXcXRy?=
 =?utf-8?B?RWZQSmF4SkFlbHZqYUFISUhtOXYwR2xnYUZWS3hBeCtvQW1WN29RcFR4Q2pZ?=
 =?utf-8?B?b1NwckZITEM1c2Yrd1JPY3pPRC9IZ01VWVJidEVOd0JMV3BFdlM1azdhcGxu?=
 =?utf-8?B?enI3Y09YYmc3S1NSSlNJR2VOa0ZKT0lpNmZIN3RRSWVoUTBMT1BrUGg1QUlW?=
 =?utf-8?B?OWRXOXh1aDhmaktVQXFJYUZkOGR4UDdPd3BMNCswT1VSdnpFT09vNm5BazVa?=
 =?utf-8?B?aW14R3RKZW81UGhZMXV0WlcySzlmbDRTZVRHMEpabnJnNlBzWHZtOXMvMm9E?=
 =?utf-8?B?Y2FuOE8vZ2d6bSt2UnFlMkI5R1l4c2xmZk9pc2RKS3pTT082bTgrbkF3bzZu?=
 =?utf-8?B?bEVTUmt6Lzc0YS9udTdhb2lxSUZ1a2FoU28yOXo3SlVWaXFCMDZwRlc4NWww?=
 =?utf-8?B?VGpGT3ZucjUwS3cwMU0rZWJwcHd4MjdBNGtPYkZ0ZW1GYUs4N2IzQXFhdXox?=
 =?utf-8?B?QzhCMXJvLy8zVk1VTkpiR2RXRVFObWNpS29KZUpTbWZKZ0pjcmFKRWpNREVl?=
 =?utf-8?B?UlBvOHpMeHc3eVNIc1Jzc2xOZGFuUWtodXgwTStjNDU3d2xiaUpjZE1IN3NJ?=
 =?utf-8?B?VU95aG9wdEdJTWZvQi9PVjZMR3NyU2hzR1dzZXpVYWppdEhHV0dXblI2S3k0?=
 =?utf-8?B?a0lJZU40TkhWalFlMHpuMVY1dU9JWTBha2h5VS9uMlR1aG9BMXE3VHozN094?=
 =?utf-8?B?aDBPQnhwbkJDdVVKeU84TDdIT3hlZGc5Mzd6SVU4WVhBd1VZKzNuSmdka1Vl?=
 =?utf-8?B?TndNK2dPY3YyYngxdlFNUUlFUG1PMXpwR2phQ1g5a29uYmdocXFTTUlkWUF0?=
 =?utf-8?B?Y2VrRlh3VEp5OURFa0oxNy9CSnV6dkFZVk45R2pVVmx3ZG5RSUI0aURXWW55?=
 =?utf-8?B?Qnp0d044TWpLZE1uZkVJM2xPTCs1Tm05S01QRkNZM09jTHFrVEp6MGhSWG9h?=
 =?utf-8?B?WnY3QTl2SUYycG5rOGRsUVQ5UHBWc2Q0OHdYejNyYU9vYVZucDRyd2tJbFFN?=
 =?utf-8?B?WDZsQmN0eFFORUpsY0dzbE5EV0xXR2pZZzBxbXhhb1JCTlNKTXI0Vys1RWdk?=
 =?utf-8?B?SlhyMVdDVTRIbnh3ZXFnRnlET1E5MkFod2ZiWmZVN3NQV3JpMk5BZ3RSR2l0?=
 =?utf-8?B?UTExUEFFTHhpdnF2aXhEWmh2MGR2aUVtTi9HWlpucG1kTU92OGljckdvcGlI?=
 =?utf-8?B?TEl4QzRtOEY1OUt0LzRtSmNPZ015ZlY5eGRGYkVYUUZZbmdsQU5jYjkzS29P?=
 =?utf-8?B?a0JXT2Z2dXFpU2phZFBwa083Slg0Q0ljMUQ5MlJuLzU3ZUprQ1VwWUs2N0JE?=
 =?utf-8?B?dGdqQ092TjEvc1crK3JVR1dtWHVad3p0aUNoRW5xMWRpWkhwYTZJUTc4Nllt?=
 =?utf-8?B?aFhWbWFDamNreERFTGNnK2J4cWNscStTOHJzL0ZFWktFUy9OTjg4cXhoc1Fy?=
 =?utf-8?B?TGE1WmRRdjI0RnhSZGJCRHB4QTcyWDJyNkR4Yzk1V0xaOHFMY2hvTFBtTEkx?=
 =?utf-8?B?emhPTlhMSm8zalMxaUgveFVOam9zaWhPYi90QU1iSHFVQjFRR2MveVcyQ2or?=
 =?utf-8?Q?72J/BNvL96YNSman+gYhI/51TxbhvsapIZhkLFB?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ecc65e8-c135-4a84-0244-08d97cd04ca8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:20:33.2615
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RgPfGZOkr2v98P9Xz74vU+6aCkg3Sz0TnOE1pZwNrNFjEX2/aWGdX4f0P8Whuu+Sxypi3guUlsgznY+azludWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6671

There's not really any register state associated with offline vCPU-s, so
avoid spamming the log with largely useless information while still
leaving an indication of the fact.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -242,6 +242,11 @@ static void vmcb_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( test_bit(_VPF_down, &v->pause_flags) )
+            {
+                printk("\tVCPU %u: offline\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             svm_vmcb_dump("key_handler", v->arch.hvm.svm.vmcb);
         }
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -2133,6 +2133,11 @@ static void vmcs_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( test_bit(_VPF_down, &v->pause_flags) )
+            {
+                printk("\tVCPU %u: offline\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             vmcs_dump_vcpu(v);
         }



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:21:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:21:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191453.341493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa5d-0000Pz-WD; Tue, 21 Sep 2021 07:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191453.341493; Tue, 21 Sep 2021 07:21:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSa5d-0000Ps-SZ; Tue, 21 Sep 2021 07:21:17 +0000
Received: by outflank-mailman (input) for mailman id 191453;
 Tue, 21 Sep 2021 07:21:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSa5c-0000Pc-LR
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:21:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 818cb834-1aac-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:21:15 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-NZpMd9o3NHGFH53cbEF2jQ-1; Tue, 21 Sep 2021 09:21:13 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:21:12 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:21:12 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P251CA0011.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:21b::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 07:21:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 818cb834-1aac-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632208875;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=289UvYu1z6hhybJctb/uHf8lFebZQ7uBwKVUfRIW7tA=;
	b=afw0ncX6rYa7tvfD5HOD2sPq3eF6QDjKjAvIGuFNjgIDwarx+2I952rScdokzWMfVzD8ml
	UNqpJ5Ar89JQa1q6A2AsOjdPk9pZwnadCJTugo25XP/8Ux1jVidjTO9XQoI6oOFoSR73zL
	c0ouiekY0rFJSavIDHU0aCo0UIpsOvw=
X-MC-Unique: NZpMd9o3NHGFH53cbEF2jQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P7aoENOHdv05H6wD0zheCMqSW1MX2nF+sHjcYjtaB3EO4dC2O/19CrQ4Et6uI19rpH7WXN3O6kmEZZIRRDbPfYTKEk9r+RV7PCxCw/MWIHX9fiF7FZbV6CVCnUCSEMf+bx1duV5xGu2hfPdPIDf6F6cwhjdZ8WoXJB1+XxJaJdjMwNMTnyahWN/eyi+uKBY6UAt7x0f5unB3ien31m7xdWaM9oCZBU8rpMCaDuwmdExK3dwJOiK8/dMkR2xCIfJd0vi7vVpvBG32Z06QJAPw5qRmCqZGGFWEKEbL9YJwttlhuz+m0DOXurVdbCTFUF3Py+L3cNJ3uDJKgq669MJ3iA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=289UvYu1z6hhybJctb/uHf8lFebZQ7uBwKVUfRIW7tA=;
 b=AOF75uADfuJNL8vrXQcOnddp+d/kjjIjs5X6zBymE1miaFCm91NlLXeGcqm+Ols9oV7K2FioYGZPgkHp+2uumS0cFm2sPt5gVfSClWMFvALYDtPR6vFxjw4fqXNA8ip3jTC/OAdxrCuepjICpTPYPjYaepfxpW6xC8pnRdBFlh1aIVkVHWStDidfAWwTxEjr5zCtaGK8IGrgfAY1uFoUluxerUtTvjXapxh2eHOyVxAx9g6xlyEbp17zMGVE+j50gHlsG6ih7LsAam+9PJIVoz1JMyL3zyghKRK3Uoq7KLl+jTdHOOkdZ027uRU09Q2uZFnyGAcnzGtDo0jj3xznlg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO entries
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Message-ID: <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
Date: Tue, 21 Sep 2021 09:21:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM8P251CA0011.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:20b:21b::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e7988f48-3e17-4028-d6bf-08d97cd06436
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26065121EC53CCFFD9993FD7B3A19@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GEQpZvVC4gjo4LApWhDyUqClHh7Tc+bAnChVMDRBSrkRpWXHc7wfDKhk9y/owPSexYzIPvKD6GURW9j7/4m3w0b4OuvTVAdq0boRma29bJmBCV2bsA8AWt1QkJhB5PgAXsMJC0rzMJ8B5oM4xXR3U4iTydY/4mfySF2EpGjObGSUiCoOwF272fFQaLhoO67bukLlchc6Q2mjpK2FetRXTPUt7MRS3TmlpAVz3cJZTbEZdrVAOXuWLJxI0SJWIZ+ARdkotTYXdpNjPW+LxCgxrl96IJIiGNpirxCe+GE3xRaLEaSN97CdEyEDFiiUHSaq8eRK8rjJbTMKrjTrdtZBTgcBytZvvs6q8W0Iq8C2o4nc0znfY5WySyRq+gcyIZ4lSkqaon2f2OViLnKoVqRgj36sblhUKDxAO7vztU9iB7IA4mgSUSjN+KejSowVseaBn6WYzpoEoeu93up4Rof+i3FX4fhFL42DqzEEet6oT57jekftueJ76+vBTo16jmpImzHmCiGdmD645giercuGpANwKDMPVMdxs01l1WHP0eOz1e2FOWZCWnq9fecaTV+rrSLL1sVDx/nVsYcld4JDckN57yoewJnXkGF1pU/zaFPH6XYupnRzApz/0Qi1OCmh1BIkZHh+u/OZaD+bVkQR1ss8QgxPFKizTsJtmDYIzQbsaWsQDh4FUDmnHaGK/tW7jlBlP8bElI8ACm5rwd+yUfGKoNF3Pk5+7SInFrXlius=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(31696002)(508600001)(6916009)(38100700002)(2616005)(4326008)(83380400001)(66556008)(66476007)(316002)(5660300002)(16576012)(26005)(54906003)(86362001)(8676002)(31686004)(956004)(8936002)(186003)(2906002)(66946007)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZmtDWnBFZnQxR2NtMURHSEQ5YzJQSktNc09MKzhra2F6dmsyYVRKSHRtQjQ4?=
 =?utf-8?B?UG85Tk9vaWFaTm1ORWNXd0FTbGFCMWQzRm16MXJmTHdVbjk2RW9RNElkOVZV?=
 =?utf-8?B?V0lzbFl2TWpraS9xdHlUR1dlNG9DYmFVaXpGYVpXY0JPaE5XSks1ZlZHZ1ZU?=
 =?utf-8?B?WGdUMlorbzJ1bDN5ZmFpeUdudmZQWjRpRTB6c1pFN00wbWRLRXRZYUdZOXpC?=
 =?utf-8?B?M25HUERnY09EclZkMy9iT284ZzZXRzhNQVhMZ0ZpbHJOVkg5aGRVL3R6Z092?=
 =?utf-8?B?WDV6Ym4rN3FibUJWVlB5djlzbmUvOTVxbmdDTE1nYXBnWm1JUmtkNEtPVHdI?=
 =?utf-8?B?MnlnTWUzUnFiZC9yZHM3RndNMTlqdTNlNEJwTGdXU1RDOWE1bTJtRFN4VjJF?=
 =?utf-8?B?K3hHYndpYTV5S0VUcFY3NXR0MWNQQ1N5T1VROTVHcW9iYVNHU3FLYmhqTlNI?=
 =?utf-8?B?M05JcjA4b0FVTDNKSkVLU2xVRDlxSktSOTZLWkFWNE9xMDlCM1RXZDBaZlAw?=
 =?utf-8?B?Z1BpektwODNqalo5SlV4aHlrZytxOFlzREF2eHBiS2pONGJtbGRZdGZLdXBH?=
 =?utf-8?B?NEZwOTNvdUhiSVNiY0w0ZlljYzY0aXJqM0paZWFnYThNRGJxcVRZNERZM1lj?=
 =?utf-8?B?T09tMG9Kbkh2bGF0NUdrR2UwZGxRNzFReFRwTjJYN2hCY0pyRVVDUUhhVXhG?=
 =?utf-8?B?Q0xKZ3RMcXBLenBXSFRhZjR3MU9RSFpZd01tWG9rMlRjcVdDQ0ZpdGk3dDQx?=
 =?utf-8?B?ZDVVZG1mVjdKVlhUcm9qbnNLTFhERjB5YS9JeEZueWxKSGFMOEMwY2hHL3hQ?=
 =?utf-8?B?bWV2SUR3MmtiMGttOFV4UEhoUnpvVy9kelZDYmU3MXAralJnODdCN05WZ0hT?=
 =?utf-8?B?VW9pdnpxb1R0cGt4SDMvRXlaajhKMUY1Z1FXa2hnbWNESGJzcXQ0dFFwQ3NU?=
 =?utf-8?B?V2RNR3U5eUYrc25ETTkrcUtTSmZJY3lqNjhoOUUzYmJySzVqQlNjSnozb2Q0?=
 =?utf-8?B?UnVxRW5HT0xBeUI1eENTUmJlM0RjNnBhbmRZaWlKRWIzU1F1dlNFQXJEbFlu?=
 =?utf-8?B?N1dLdWk2MUZDWko0dUttQkxSY2tyS21JVGpuOW9EaFB1SS9uVXlJdDcxRkJO?=
 =?utf-8?B?ZkNqWTg3ekZFTDRaZUVpZWsvREVPTi9ad0JSb0NIQUwvcGVkNjlCd1B4R1JX?=
 =?utf-8?B?Z1pwMlhOeWI2WHVwZnJmNGprTXFESDhtcjQxV2tuYU96cUFuZXFhQVRBczFY?=
 =?utf-8?B?ZVgvSWsvODh1aXFhb253NWE3bUtaelpaZ2MxRVU4UDFFbFp6bzFsZ1BKSDFl?=
 =?utf-8?B?LytES0hjanRESWdUREJjUU1tQzFOZHNlOWQyc2NNY1gvby9RcU00L2tHZ0VM?=
 =?utf-8?B?Z1JPVi85WTE5cjFsd1NZK1lmbklSWXVFbWRTQ0tRbG1PQmNUc29PMWdXWllq?=
 =?utf-8?B?WnUwbmdINkRCVUR3VlJ3alFKUVpsWkJFU2RYYXhKQkh6MkNEWlNKWkc3U3ZT?=
 =?utf-8?B?bCtubUNtKzk4cjRoSDMyZk9oaGRWZGM2SXJLL3g4L2VSam9Jd2pFWXVJTlRm?=
 =?utf-8?B?eERoUUV3Z3dXazN0T0oxNlNqUkVleDhFYWxPMThSRkRHaGdoT216NTNvTklU?=
 =?utf-8?B?bkdMcnM4T1NVQmN3YVlPc0hLaHZ3SjhKcTRuY01LUkR3QUI5MXcybklsTEJl?=
 =?utf-8?B?UXB4TzhmY0R5a0M2RGErL0l6d01NaXRWVk41RW5Fb0J3cEkrOHpDT2k5NktY?=
 =?utf-8?Q?+ciszDXkPJIu9GQe/D5OfDN1GSZZWD0eafRKzoW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e7988f48-3e17-4028-d6bf-08d97cd06436
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:21:12.7492
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +kS0cPmW0ktOELRYfFrSvLx3W8jO+myJKsOsj5Jca8mudtFhbJXABARVd5v4FUyQJOKhi1MKgUDXZYrYFEhTKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

To become independent of the sequence of mapping operations, permit
"access" to accumulate for Dom0, noting that there's not going to be an
introspection agent for it which this might interfere with. While e.g.
ideally only ROM regions would get mapped with X set, getting there is
quite a bit of work. Plus the use of p2m_access_* here is abusive in the
first place.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: Move last in series, for being controversial.
v2: Split off from original patch. Accumulate all of R, W, and X.

--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
             return -EPERM;
         }
 
+        /*
+         * Gross bodge, to go away again rather sooner than later:
+         *
+         * For MMIO allow access permissions to accumulate, but only for Dom0.
+         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
+         * the way they specify "access", this will allow the ultimate result
+         * to be independent of the sequence of operations.
+         */
+        if ( is_hardware_domain(d) && gfn_p2mt == p2m_mmio_direct &&
+             access <= p2m_access_rwx && a <= p2m_access_rwx )
+            access |= a;
+
         if ( access == a )
         {
             gfn_unlock(p2m, gfn, order);



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:38:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:38:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191468.341504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaLi-0002Nt-JM; Tue, 21 Sep 2021 07:37:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191468.341504; Tue, 21 Sep 2021 07:37:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaLi-0002Nm-FQ; Tue, 21 Sep 2021 07:37:54 +0000
Received: by outflank-mailman (input) for mailman id 191468;
 Tue, 21 Sep 2021 07:37:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSaLh-0002Ng-WC
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:37:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cd8aa52a-0c25-417e-aa9e-01b4b2839de1;
 Tue, 21 Sep 2021 07:37:52 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-5-gZLf6xi6O0ykX9ka3Q8vag-1; Tue, 21 Sep 2021 09:37:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3535.eurprd04.prod.outlook.com (2603:10a6:803:11::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 07:37:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:37:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0100.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 21 Sep 2021 07:37:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd8aa52a-0c25-417e-aa9e-01b4b2839de1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632209871;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+IcWAWRhFcm67BoYEBI3qzzc3i4lN/iLCloOocYgg0g=;
	b=ScJxcLkTuKLk2jIw0TjgGy3nNVMvWFFWz3znnbamojfByLRLVbz9Y6Oyh8Psq8cl9UtE0H
	rhaRGjh76Pg6Ihc27QhUrBUPYbuV8jgT8ncKH1vr5dZ/yHAUjgZdS+L9sILxuj9or7YAdt
	MVK4rvHCKpf02Nr33Tw7rNNmrWPIGiY=
X-MC-Unique: gZLf6xi6O0ykX9ka3Q8vag-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IUaZ18zzKd0/6F5awnj7yVgNaZzZgHBaQU/Kxi21TFAQ8eBK4aVTXoZgPDUUjqYmT503+NpPaMv28u7O2+guPaUHK44cZ+DCd/4WIHWjTBGtv4ghn25VZQ2jf2258WIUcI9aTE2jt8punWOKPs3s69AxRzq30qsYOciohZJJqIXEe3TiUxm6oM1U6HsbdX3XTK0q4KSbTm0Ma2Euu+GyRHZx1NONI0P+4DCHqtibuAnUELW68IrNJQUzcklpfRBxVWn6PfeG1R/0Sanoym5CpZYZ7Ospm7FfQB7sN5KrsnhsY1/7qwa6Af6Met9+24lM8txy4s5I7ChRsq13XOUpMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9hu3iAJHykloOuWfFZ32ZXNlq3ONAqlbMTDZMI5vySc=;
 b=MdBa8vD47WIZrlLJYsRhm7QwB8EL52xs0+N+xR50AE7Os9L6371vyoeWETV2paEnOToe1XZz7WInuuhGhrHiWXZR8SHpF4X/4QT6N8i+kMEmy4fzA0BwRbfkfxKeWMF+BdEYKJUcjFe0B3TnmLTgCRlEqJDJa2C2ovow3isY6an+ss6OC0VduPmW+3taTlLSlf7Hv1IOU/UL+49gvJgxjg7mJhns3uhsebFs78eoTDWRlS2UyuNoIGNtrhmsEO5JwC3tJTw7FUOY/u36/3MpcVpbPu9AzM4282Vas0dYgICiGh39IjXfO4whzI4lj1tt1iBHdys8FB8HaTJZ4ZuhJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v7 1/8] AMD/IOMMU: check / convert IVMD ranges for being /
 to be reserved
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Paul Durrant <paul@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <bf4b3be9-1274-9750-7340-08b29dcecfd3@suse.com>
 <133d60ab-0de6-4831-7387-0ba9cf6efb5b@suse.com>
 <88994b1b-a8b3-1207-5ac9-dac50c383a98@citrix.com>
 <e3fd804f-07a5-8388-9c2a-19aec82a164e@suse.com>
Message-ID: <a85d800d-d06d-534f-3e25-bfe724d72037@suse.com>
Date: Tue, 21 Sep 2021 09:37:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e3fd804f-07a5-8388-9c2a-19aec82a164e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0100.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 88d4a113-9da2-421b-47a1-08d97cd2b5d4
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3535:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB35352FBFCF6A94D2E98E986EB3A19@VI1PR0402MB3535.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dV79Sv9dpzkA87/viXISu9eNpILoEVwExX2DSq37VRYCJDve0OggWFY5+q1nsPiHl+q5R6HmLL7AAAK8ERydFhrYshS/Y72Eox8EQscq2VTq1s/YmUR4OtmGmh+nya0GR5xeqybt1Q01mBT3QL5d5AMy4E8pqMgqrR4nGW0KC3av9f/W/G2uPJcaDX1IN+5tsDoZIKRx10+3d4HcPN5soFy41BozkM0GKO2w9oirbXujfP8SOS7RNgP/SA60EGUDa6wogIx6N7gxjRLipPGYbLlGWY4EyOXI0DcNeIXss05B2ikhZ9oUm8wy7xxdrFZTpJ9tc5Hi/ZBwyV75i7qhf7KelI8LeoKlyZZ5MCY78LJ99s6zW7mZkTZZ5cCQNSwRIBrbHj1s6PlJZQ3n79mY+qyD91Tah1tD4e+aAUqxtg+7GLEZSoICBlWS+AVq5l19NczNyJq8enrFWwxj7Xw7L17wiOTNrgWqpsNvgdMZ84H6mgWccQ6majC6lAy4DLr1RGKUsr2YfChYKb4HGcklMCXPgb7diu+Moh7UFPsJzE8Z7Na0j9xUYlfu57wW86xR4o+ZJv8mtb5X31wk2hPkCqRJ/FH7K5UGSane/d4ScjZW7QipwVg5EgWEPmQXD3VUf4QrKzrNWlZrQHUrVW0wblOk5Y6CycpMBOKNQDWBGKW54nc1JR4fIPyGYUM2rLCZLHHbo03FKQ3rGOXb19cYko7l0wUi22+vGJTW0gObRYw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(8676002)(66556008)(2906002)(38100700002)(83380400001)(66476007)(86362001)(54906003)(31696002)(26005)(53546011)(316002)(31686004)(66946007)(508600001)(6916009)(8936002)(16576012)(6486002)(5660300002)(36756003)(186003)(2616005)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?8VQlboMf1ce3uZ9ADvMcMAgk4iJdrE0vbBE3aePlkCLIquHVA9VZ9BCxYO6a?=
 =?us-ascii?Q?JX9AkJJwqbzkZ+IfoHSZuti1rccyIcv2/H4FRa1IjnVxezW+BVM3Zvky6kH+?=
 =?us-ascii?Q?lfKq1IbT9aUE1tLiW5a3v1VDYYVHe8dKMl/pr0c4sbekwl2bjHP7EMqcCceN?=
 =?us-ascii?Q?fqP4SC4r6AOj1T57nqj5gODNZ6tskpIw9qw9nR+pMW6oKUGIhoRgsINvkQcb?=
 =?us-ascii?Q?QwNT6jNdFIKKvTyeObwBYgLMhhOMR/9FYGedE7AIb7Si/rrRtM2vjYJSELhe?=
 =?us-ascii?Q?VfELnEOZBhrV6FSbSqNdWo4ByLi04V2pZ4KXf57d57e0E35B24on9p65KRoS?=
 =?us-ascii?Q?mSvM1JTgICx0h3pJd2lVyt9AMvO64BmqQv78DWoDOb1flBefgwDoFDPY4D7c?=
 =?us-ascii?Q?2X1ntQWXNoxjgymJ0f3jgcojRSQYpTWG71bk6dqSZU3K188JkHvWnllahEfM?=
 =?us-ascii?Q?AMzUflWXRF5sDnrh3zQJU1idHTuZIIY6VLS5RZT+2154wXS8+xUF3HiKBrri?=
 =?us-ascii?Q?Zb/w1LnhZYhz8XfVH4F+BR15G2elbtuOxQP4xBhDQEf5hUnnrozpumwRZb7m?=
 =?us-ascii?Q?lXsnu30PEekb8c23CuN2s/FsVplkgltSD5I73E6crciYaOzS29lVramOI0nm?=
 =?us-ascii?Q?3D40vBNA9fdtPtuu7W21zaStB7X49BijTZXS5ZgVgzdG4A0xHrvdGOW2KDBp?=
 =?us-ascii?Q?KPiKmmHwLLK5kugADANWCUTix71eCUNPwfqFqqU8atxRDo+hjpuhOCj+osUc?=
 =?us-ascii?Q?nInVvdrmTs7G3Gyot6/He31/Ow+Q4fYYCARB16adacIzVKzEYKewUbtjHv+6?=
 =?us-ascii?Q?OJ1FJdf0xgPBcme1tePw0ByfFLnO9aROsYnkKj5ah4cB7yTJRcXWk9s74RKU?=
 =?us-ascii?Q?I7VrjtbcKtXIJ3CtqkVNWkAXOlhlB8Gh8yRqH4Ga8YVhI1FNpRH85Ve6SCr5?=
 =?us-ascii?Q?PqqYuxI9m87x7r9zCO2UxnXB8Ord70r+FfZozH139atukS4iDQOFv+h8xJMQ?=
 =?us-ascii?Q?izw6T4CK810bbeR9eI0ijsI52YtgKR5iUO808c8gwlhN+KreZt+cXAoA2MmN?=
 =?us-ascii?Q?FV8GpD7sdJToMzMHDN6WYIjnmD1aHlpJJfu8w+GyxQyzbi6dlfu4Koo0DsxW?=
 =?us-ascii?Q?c8jrIy3l8mVxBYFD/EJe4GoF2xXc2JpD62Duq3UvI6Kj83YAfnq65AiW9yH+?=
 =?us-ascii?Q?GsLQ2J4bJw2my2woEY8px50rGi2nZNyHEZL29iYQKUK98ACC3OrbZRmG/B3P?=
 =?us-ascii?Q?gopEeKoeXpHAMdvV06taZaHj19UYxDRDcvC4o/MtzI0eGSqvAzpNkzzCN1S9?=
 =?us-ascii?Q?du2omvh/GWY3WFuS15w/RdP1?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88d4a113-9da2-421b-47a1-08d97cd2b5d4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:37:48.6159
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uGpnElRXbEjQnfc/d9QRuy9AOJtAPG+8aNjVQDAevwmDzXhto/XzzX49BKxFXsMz6XW9oahrsF0bldPd/DtlFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3535

On 26.08.2021 14:31, Jan Beulich wrote:
> On 26.08.2021 14:10, Andrew Cooper wrote:
>> On 26/08/2021 08:23, Jan Beulich wrote:
>>> While the specification doesn't say so, just like for VT-d's RMRRs no
>>> good can come from these ranges being e.g. conventional RAM or entirely
>>> unmarked and hence usable for placing e.g. PCI device BARs. Check
>>> whether they are, and put in some limited effort to convert to reserved=
.
>>> (More advanced logic can be added if actual problems are found with thi=
s
>>> simplistic variant.)
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>> ---
>>> v7: Re-base.
>>> v5: New.
>>>
>>> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
>>> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
>>> @@ -384,6 +384,38 @@ static int __init parse_ivmd_block(const
>>>      AMD_IOMMU_DEBUG("IVMD Block: type %#x phys %#lx len %#lx\n",
>>>                      ivmd_block->header.type, start_addr, mem_length);
>>> =20
>>> +    if ( !e820_all_mapped(base, limit + PAGE_SIZE, E820_RESERVED) )
>>> +    {
>>> +        paddr_t addr;
>>> +
>>> +        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not (entirely) in reserved=
 memory\n",
>>> +                        base, limit + PAGE_SIZE);
>>> +
>>> +        for ( addr =3D base; addr <=3D limit; addr +=3D PAGE_SIZE )
>>> +        {
>>> +            unsigned int type =3D page_get_ram_type(maddr_to_mfn(addr)=
);
>>> +
>>> +            if ( type =3D=3D RAM_TYPE_UNKNOWN )
>>> +            {
>>> +                if ( e820_add_range(&e820, addr, addr + PAGE_SIZE,
>>> +                                    E820_RESERVED) )
>>> +                    continue;
>>> +                AMD_IOMMU_DEBUG("IVMD Error: Page at %lx couldn't be r=
eserved\n",
>>> +                                addr);
>>> +                return -EIO;
>>> +            }
>>> +
>>> +            /* Types which won't be handed out are considered good eno=
ugh. */
>>> +            if ( !(type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
>>> +                           RAM_TYPE_UNUSABLE)) )
>>> +                continue;
>>> +
>>> +            AMD_IOMMU_DEBUG("IVMD Error: Page at %lx can't be converte=
d\n",
>>> +                            addr);
>>
>> I think these print messages need to more than just debug.=C2=A0 The fir=
st
>> one is a warning, whereas the final two are hard errors liable to impact
>> the correct running of the system.
>=20
> Well, people would observe IOMMUs not getting put in use. I was following
> existing style in this regard on the assumption that in such an event
> people would (be told to) enable "iommu=3Ddebug". Hence ...
>=20
>> Especially as you're putting them in to try and spot problem cases, they
>> should be visible by default for when we inevitably get bug reports to
>> xen-devel saying "something changed with passthrough in Xen 4.16".
>=20
> ... I can convert to ordinary printk(), provided you're convinced the
> described model isn't reasonable and introducing a logging inconsistency
> is worth it.

Since I haven't heard back on any of the replies I gave to your comments,
I'm going to assume they were sufficient to address your concerns. I'm
therefore planning to put in the part of the series which has R-b perhaps
tomorrow (with the tiny adjustment(s) that I've made following your
comments, which iirc were just spelling issues). If you disagree, please
reply there.

As to the particular concern of yours towards visibility of error-like
log messages: There's e.g. a significant amount of pre-existing
'AMD_IOMMU_DEBUG("IVHD Error: ...", ...)'. Instead of introducing
inconsistencies here, I think I'll add a patch on top introducing
AMD_IOMMU_ERROR(), AMD_IOMMU_WARN(), and AMD_IOMMU_VERBOSE(), converting
present AMD_IOMMU_DEBUG() as I see fit (and then up for discussion).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:42:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:42:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191474.341515 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaQ0-0003kq-4k; Tue, 21 Sep 2021 07:42:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191474.341515; Tue, 21 Sep 2021 07:42:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaQ0-0003kj-1W; Tue, 21 Sep 2021 07:42:20 +0000
Received: by outflank-mailman (input) for mailman id 191474;
 Tue, 21 Sep 2021 07:42:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSaPz-0003kc-8X
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:42:19 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 71f36848-1aaf-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:42:18 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 40CDB220B0;
 Tue, 21 Sep 2021 07:42:17 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1A0F613B0A;
 Tue, 21 Sep 2021 07:42:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id RvsZBdmMSWHtDQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 07:42:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71f36848-1aaf-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632210137; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=QjwOK1fJKv8Kz5PIzKMEKPqIj+avUXaKWiL5dqbqgKU=;
	b=SpNV5yTad221k3xYW9W0ic7bJ9/7LeGvNHIV656W1XhvK5+F7i+OqHksnSP0EYlfb/J6LA
	Vdeh7dXylSfiP2voQJ1hycdNSi7d6ltoak+o7VPGAEqlSkUlAO12j/FiKeFjjN0v2Exquw
	MslBwGvKy+PuvizaMhC4sQ9nQrAAXbo=
Subject: Re: [PATCH v2] xen/x86: fix PV trap handling on secondary processors
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <6083d6d0-63b7-e102-5596-3c4f35c2e5f5@suse.com>
Date: Tue, 21 Sep 2021 09:42:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="YW2RRkX5pjUzzdWaa9A04Jt7G0232U525"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--YW2RRkX5pjUzzdWaa9A04Jt7G0232U525
Content-Type: multipart/mixed; boundary="oWEil0H4OKCQInggIaw6NNL7Krho8WNW5";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <6083d6d0-63b7-e102-5596-3c4f35c2e5f5@suse.com>
Subject: Re: [PATCH v2] xen/x86: fix PV trap handling on secondary processors
References: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>
In-Reply-To: <7a266932-092e-b68f-f2bb-1473b61adc6e@suse.com>

--oWEil0H4OKCQInggIaw6NNL7Krho8WNW5
Content-Type: multipart/mixed;
 boundary="------------A21806AE892E7707F12A4F6E"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A21806AE892E7707F12A4F6E
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 20.09.21 18:15, Jan Beulich wrote:
> The initial observation was that in PV mode under Xen 32-bit user space=

> didn't work anymore. Attempts of system calls ended in #GP(0x402). All
> of the sudden the vector 0x80 handler was not in place anymore. As it
> turns out up to 5.13 redundant initialization did occur: Once from
> cpu_initialize_context() (through its VCPUOP_initialise hypercall) and =
a
> 2nd time while each CPU was brought fully up. This 2nd initialization i=
s
> now gone, uncovering that the 1st one was flawed: Unlike for the
> set_trap_table hypercall, a full virtual IDT needs to be specified here=
;
> the "vector" fields of the individual entries are of no interest. With
> many (kernel) IDT entries still(?) (i.e. at that point at least) empty,=

> the syscall vector 0x80 ended up in slot 0x20 of the virtual IDT, thus
> becoming the domain's handler for vector 0x20.
>=20
> Make xen_convert_trap_info() fit for either purpose, leveraging the fac=
t
> that on the xen_copy_trap_info() path the table starts out zero-filled.=

> This includes moving out the writing of the sentinel, which would also
> have lead to a buffer overrun in the xen_copy_trap_info() case if all
> (kernel) IDT entries were populated. Convert the writing of the sentine=
l
> to clearing of the entire table entry rather than just the address
> field.
>=20
> (I didn't bother trying to identify the commit which uncovered the issu=
e
> in 5.14; the commit named below is the one which actually introduced th=
e
> bad code.)
>=20
> Fixes: f87e4cac4f4e ("xen: SMP guest support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Pushed to xen/tip.git for-linus-5.15b


Juergen

--------------A21806AE892E7707F12A4F6E
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A21806AE892E7707F12A4F6E--

--oWEil0H4OKCQInggIaw6NNL7Krho8WNW5--

--YW2RRkX5pjUzzdWaa9A04Jt7G0232U525
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJjNgFAwAAAAAACgkQsN6d1ii/Ey9V
0Af/Z9EVcW4cZIkNhFP/98yqYfGvs3b/+JzdoHtbKybAeD98FLFha5gxbagwk5B6xr99JJXQIfM2
lQ2WvksHdxmj9/OEsePdh22rqEFmnYuNAOmZqxzyjR+AQH9u5PLnCgP2jPwjJUTDxJ2ji8XBPJu4
oUwyAqoQUW2Y0ftAdPVjpTZR+NhkJGBN77JnpOWCXu8aRlAqOB8ViOpKvehpJiPwXabtPuaIC4T1
hlo0Xk4fiUTGMaZK3xO1SEHB47aXDSEceIidI9myHPeFd84s2bscI/Ievwt3iGp0zyDtmHGupvzc
Y3xN41Xe9X4QAVAunk82c8AX8DjSMFiEK3SwnbabSg==
=c41r
-----END PGP SIGNATURE-----

--YW2RRkX5pjUzzdWaa9A04Jt7G0232U525--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:53:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:53:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191482.341525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSab4-0005Lq-6z; Tue, 21 Sep 2021 07:53:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191482.341525; Tue, 21 Sep 2021 07:53:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSab4-0005Lm-3v; Tue, 21 Sep 2021 07:53:46 +0000
Received: by outflank-mailman (input) for mailman id 191482;
 Tue, 21 Sep 2021 07:53:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSab2-0005Lg-PU
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:53:44 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 377793ac-e0ba-4f04-8a61-2163daaf85d4;
 Tue, 21 Sep 2021 07:53:43 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-Tb2tPBaVMdSP0NQvFO7ckQ-1; Tue, 21 Sep 2021 09:53:41 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 21 Sep
 2021 07:53:40 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:53:40 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0113.eurprd02.prod.outlook.com (2603:10a6:20b:28c::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Tue, 21 Sep 2021 07:53:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 377793ac-e0ba-4f04-8a61-2163daaf85d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632210822;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=x+tEN8D5GCwudAasmF+KAlvekOlZNbe3X9JtFJBl+DU=;
	b=kt+7UacLFVFKe9LU2dFDnpw4Av2Okqb6wtoZ9ofzcOl9WgzFgJX/w3PpC7l56cyBei6ckA
	Jei+W0Ldvqc712azp2HKVQ3giy0uSfHR0AtNpsisskHolHMdB9YwcoR+wZrBh8qOlP1uK4
	bJRHfAEQ3THdwxi+ikq1ztk85TTc4bE=
X-MC-Unique: Tb2tPBaVMdSP0NQvFO7ckQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CkvOOV642MgBjfJJd1M5tXX1mQS5mU2yXyScRHJlJ0VS7jiXE+8UiKV1ldsdK5gUX0YkSxWEt9I7eCR7C7lPCZMsfwVXE5bQI20KLANVk7yyM/4nNBWmtqbQZJDGjKOw/JcRYpe9qYYLcJnKDymifjbyGXHR4inDaeSlX8tGoKOygk5jkEOdLdGxOGZhh2ICqt646X+Y8v9awriStSqbtHpcZpdgOl7yLrvfH7biIKUHQg/UJl7Ja1/KaxvGWT6sKJD9lk/lIwcwiK5OxsWMjF1ZeAff785F+hZLUGraiGYsSIamEJvWil49cA60EsJSTJI6abzLSxjVs5/ymmBV3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=x+tEN8D5GCwudAasmF+KAlvekOlZNbe3X9JtFJBl+DU=;
 b=jXbJK9TDe4pSKn2BFPtIwflx1P6Mvd+X9EqsfBc/gS8IRmQAEBydT1Lb3iWwYFLa3Ssz3c2TYXbF7Bu6MB21BEc+c0Gy6M/jc25RbTqwWmQtftvo94bLSfM5bwnMfktU8mlX84h2VeknW1qxyZIgie7ECt2GUk0t2QFsjFmZXBIHe7reBHsx568UNn/Yaf412iRWiP+F7p+Kg98Jsm08j2Pr/4bLw84Be6iCLM0r/jO8+2S6tSVp5tDOGP4DGmPA/2PaPUIQqeLahjyixBZ7c1mcMINKtQA658tVuqlKjabdjAKDuNhFaIk4xV9IKrL17Lj0Goc7EhdzlBqN0mdiqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
To: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210921070226.32021-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
Date: Tue, 21 Sep 2021 09:53:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210921070226.32021-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0113.eurprd02.prod.outlook.com
 (2603:10a6:20b:28c::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1da6b922-2679-47b8-5df9-08d97cd4eccf
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB617656AEC198E6C4F5932FC7B3A19@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zH7Kl1Jv4tCQF8p34gML8wlj6etEc+soqyZSRJH2F3FJgZp3XEqmKGPar43sh+6UM/D4RB864Gf9iNpFyykzqoc1KZpUYy7lEw3BwQUnDktVUdj+I9mOPa0mWZMdO5JT1KQi4pM/Or1aD4DHaEj2QiUbjSppGR/Oi6JvPGoDwOerob92S2a8+7q4ktnuW3Ry6Tu1v0kv7PYvYXU/3sKzdfSwSHLyEi3hF8d0YZeAtXKjeHSuIsw+mOJs+2uDMNY+xAzp/CoHOPjiSTt9mk4PRVubMtsIYEkZp6m1zKT5HufsSMpYQqiFRdrw6r5nFcNFpVCdUidmX/Jv8jw8JDUJ8eo1DAfTc8Y0lJSrD63rbhJRsEjTEnIShDeS1TG44ohE3qfwWbTbZhj0ExE+FOWTp45S+33c4RGyYtJu7BLQXwzRw2WBtQGBpo00zynGlXodBWWCv7sFhsgkn2h6prmRAVzm4CYgGj9RUe4BrV+cvHLFAcLwvCk9SWNQI0UvCCbLp0w71Tl3BmdlCZnHw6LjBUtVn24CTEA7xySeHmAGnquXCrOMGrkqv7CihF2QAkzzARPL/RdOi3+nvpy8P2vaJfY9fWXdKLPGzBIxMsFIOiepwkifgRybSeTWE4Vl2EZL3R9zBkOcEiqAm5zhH9CzEW75brPMLBkWJ4hbBYtlJg6rEFY/MWA8ycW1rFru3gJH4As4R9Us9Fi7CSiF/09hXTfXplT6K1NiyEL3p4tPi+8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(4326008)(36756003)(2906002)(6862004)(38100700002)(8936002)(5660300002)(86362001)(37006003)(6636002)(8676002)(6486002)(53546011)(54906003)(316002)(16576012)(31686004)(66946007)(186003)(508600001)(66556008)(66476007)(956004)(2616005)(83380400001)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VG9nQTVHL2k1L2R0SzNJb2s1SWVBbE9hRTJUNVlLdDVjNXBydzliRUQvNkNU?=
 =?utf-8?B?Z0pqOFJlN2t3UHVrelNLV0Zzc1daZ20vVVhXZlJ0UlkvalJkamVMVHV1dVd1?=
 =?utf-8?B?WHk1Y0pha0E4bEJabUF5SFcvbTB6bEwwMlVpMkE1SGRDMENqcVVkUTdDc3V1?=
 =?utf-8?B?ZEE1L0ZVNmlCMVBoZzlUVjRQTGY4SysvaS9MK0d5eUV1Q2MyR2o5SGwzVFN0?=
 =?utf-8?B?WDRIYlBpRVVjUzRIeEZBd3oyZUNrL1VsL2xhVDZoYkZQWENiOG9mME9qTHNF?=
 =?utf-8?B?bVhYdHdCc2RWMWVxN1lHdGlOb2VLclFXMHUvbytGTThKVklqMXRRM3F1eTZn?=
 =?utf-8?B?S3pwQ0I5d250UU1ZVUJxTldPckNqc0VtVWVhSjRsckF1N1lJay9zRjR5SDEx?=
 =?utf-8?B?aTBsTG5WazBEa0V6cm5sYktWa1U4MGdBTmQyT1plclF4emZWLzMrVkdMbWpi?=
 =?utf-8?B?TmdBVU5lcFo5ZXJHbEJUZUdvdjEwc0tyUllzQnVHSTNTcFJPVlI5dDRvMXU2?=
 =?utf-8?B?UlkrZW05TEZlVUNTQ1d3UHdpTDNnNjJINWVTNjRsK1MvVm1HdG5oc09pM0JK?=
 =?utf-8?B?WWZyZlNndnBJOGgrV1ZPVTNkbmxQZ0syMUZTeW9HWjg1ZVdHUmFIcEhIWHhE?=
 =?utf-8?B?ZWE0R3dydHo1dmFnOVpkd1M5SzZoWk9Vb3cybU9yZ1p3aWhUUythT2x5VndR?=
 =?utf-8?B?Sk5lVEYyekl1N1BvekZvQ1UwWWtCMmJ6YmFGZG9KbTJHblFjNXBqbHRja2FQ?=
 =?utf-8?B?blU4Rjh3bjBtTHhkb3B4Qld5MmhqVUZTRkdGK0tRRnAzNllmUjlQK2cvTFFu?=
 =?utf-8?B?Q0xXdHN0aC9RMmI3NFRvc2dRbS9HNHMzTzVML0dUdE9Qbkc2SGJCNk5zWWty?=
 =?utf-8?B?RlVHMzFqdmRGT2lTYnJIMVNUeC9RQTZvNnlvSmozSmQ4cDh6L0l2azFHT2hh?=
 =?utf-8?B?b2NxRDFhRitrSW1FQ2NzRUxsYmdnT3VyWlVCUldvdVdWNktFT1A5MUZrUkJR?=
 =?utf-8?B?czAreWl4SHVPTjk4d0VUbStJTUlwWFVHV3Y1K2NWYURyUDZwWXZsRjhTdmpT?=
 =?utf-8?B?M1JXQ0xTcXcybENXU2dwMEFqbmF6UWNGYUk2aEJRRnp0ak56Z1RmOUpBejJB?=
 =?utf-8?B?a0dOeGxFSmxIUEhHRFJhTkIwTFhoYXBZVnUzaHJYYUtsQThQVmxFVUhtTGRa?=
 =?utf-8?B?RG55eFFkNTRQcmdTejdaNVRoSTgxSzlUWUVwQkhpYm1IenNPdzRlMEc2dE1R?=
 =?utf-8?B?dDhDSXcxdmZ5S2FweHlNVjF4YnNWcWZ1ZUpqY2lIUFZxTlQ2bFRmbHpUcFNX?=
 =?utf-8?B?NDBkOVVaZmdOM3V4ZWFEVDdxSUxUY1AxZUpGdXh2VjIya3FTa3R5dHNTMnAz?=
 =?utf-8?B?ZEFWZlcrMUFGWG5NbnJQUUYzelBrNmR5a3l3T0R2em51UFM5OEhCRURkMzVZ?=
 =?utf-8?B?aGdmYW5ySkdHVzI0TUVFYmx2ak5SZVBQTmgrWkFWeHluVStGTzB6VmR5c2tk?=
 =?utf-8?B?MFkwQU5xRy9sS3NZUVFSVjY1S3V1NXBuQ3RtWFpOREVlTFJheE9RVk1iellN?=
 =?utf-8?B?U3pMbVJiVXlNMThwVEJqZmV2MUVESmIzQ1I0a0xMQ0t6dzVBb1A4TlRIODM4?=
 =?utf-8?B?aGY5enVpbHI5RUJkNEswY0N1Y3Q3NFF6ZnloZUx4ZUpkUGtWMlhLVkRicTF1?=
 =?utf-8?B?VVBoUTZBa3pMMEVPczJURjFVSlIrbnJjb0hjZ0NKU1FvSlp6Q3hTb3M4WUVY?=
 =?utf-8?Q?+Thz10xf2rFc310/nwuM0hA3zXmqyjS8iXYiBF2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1da6b922-2679-47b8-5df9-08d97cd4eccf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:53:39.9068
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sKRDiSHa8a2Gm0WfbdtojdwkXs4PzHNGkFSE4sxd4acePbZmbhFhDIG3Zy1lU3lNUqfopuOdWLL1A7uld0vWMA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

On 21.09.2021 09:02, Juergen Gross wrote:
> --- a/arch/x86/xen/irq.c
> +++ b/arch/x86/xen/irq.c
> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>  {
>  	struct vcpu_info *vcpu;
>  
> -	/*
> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
> -	 * cleared, so disable preemption to ensure we check for
> -	 * events on the VCPU we are still running on.
> -	 */
> -	preempt_disable();
> -
>  	vcpu = this_cpu_read(xen_vcpu);
>  	vcpu->evtchn_upcall_mask = 0;
>  
> -	/* Doesn't matter if we get preempted here, because any
> -	   pending event will get dealt with anyway. */
> +	/*
> +	 * Now preemption could happen, but this is only possible if an event
> +	 * was handled, so missing an event due to preemption is not
> +	 * possible at all.
> +	 * The worst possible case is to be preempted and then check events
> +	 * pending on the old vcpu, but this is not problematic.
> +	 */

I agree this isn't problematic from a functional perspective, but ...

>  	barrier(); /* unmask then check (avoid races) */
>  	if (unlikely(vcpu->evtchn_upcall_pending))
>  		xen_force_evtchn_callback();

... is a stray call here cheaper than ...

> -
> -	preempt_enable();

... the preempt_{dis,en}able() pair?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:54:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:54:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191487.341537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSabV-0005vq-Jt; Tue, 21 Sep 2021 07:54:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191487.341537; Tue, 21 Sep 2021 07:54:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSabV-0005vj-GQ; Tue, 21 Sep 2021 07:54:13 +0000
Received: by outflank-mailman (input) for mailman id 191487;
 Tue, 21 Sep 2021 07:54:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSabT-0005mk-JX
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:54:11 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7ada695f-5030-4d1e-bd14-c60ca4cd26be;
 Tue, 21 Sep 2021 07:54:10 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 8B0AB1FD47;
 Tue, 21 Sep 2021 07:54:09 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3908813B0A;
 Tue, 21 Sep 2021 07:54:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 5TM7DKGPSWExEwAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 07:54:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ada695f-5030-4d1e-bd14-c60ca4cd26be
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632210849; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=x2DFODHzNbeP+tC+5YNGkcdF5SILtU2iZsU6zWx71r4=;
	b=Xdfw/hTxUv8Ap/lOfYmrUzi2KB5jrWsYI068h25d4g3ok+WKpEJR/QjU87t4HUKnRHojnt
	6ATv7INfI4AC56HzfXgqcinaAWJUdcm3VmQHW8qhuzea7boUyCQsoL69GWo9WbosAtintA
	HXhTMjIk5HqfYyJv0EsiOMMtEWPAK2I=
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 jbeulich@suse.com, Oleksandr Andrushchenko
 <oleksandr_andrushchenko@epam.com>,
 Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <809c7a0c-cc8b-ce6f-bab4-97029e538053@suse.com>
Date: Tue, 21 Sep 2021 09:54:08 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210917130123.1764493-1-andr2000@gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="zjbIflcdM0RFlPchq1VJSgX1W9fScMSxn"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--zjbIflcdM0RFlPchq1VJSgX1W9fScMSxn
Content-Type: multipart/mixed; boundary="fHkvMZ5xDuOMmMrlB09mkGBha2r34g40x";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 jbeulich@suse.com, Oleksandr Andrushchenko
 <oleksandr_andrushchenko@epam.com>,
 Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Message-ID: <809c7a0c-cc8b-ce6f-bab4-97029e538053@suse.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
References: <20210917130123.1764493-1-andr2000@gmail.com>
In-Reply-To: <20210917130123.1764493-1-andr2000@gmail.com>

--fHkvMZ5xDuOMmMrlB09mkGBha2r34g40x
Content-Type: multipart/mixed;
 boundary="------------0253082D65A4A2883D8F7851"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------0253082D65A4A2883D8F7851
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 17.09.21 15:01, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>=20
> Xen-pciback driver was designed to be built for x86 only. But it
> can also be used by other architectures, e.g. Arm.
> Re-structure the driver in a way that it can be built for other
> platforms as well.
>=20
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.co=
m>
> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
>=20
> ---
> Tested on Arm and x86.
> ---
>   arch/x86/include/asm/xen/pci.h     | 24 ----------
>   arch/x86/pci/xen.c                 | 74 +----------------------------=

>   drivers/xen/Kconfig                |  2 +-
>   drivers/xen/events/events_base.c   |  1 +
>   drivers/xen/pci.c                  | 75 +++++++++++++++++++++++++++++=
+
>   drivers/xen/xen-pciback/pci_stub.c |  3 +-
>   drivers/xen/xen-pciback/xenbus.c   |  2 +-
>   include/xen/pci.h                  | 34 ++++++++++++++
>   8 files changed, 115 insertions(+), 100 deletions(-)
>   create mode 100644 include/xen/pci.h
>=20
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/=
pci.h
> index 3506d8c598c1..9ff7b49bca08 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -14,30 +14,6 @@ static inline int pci_xen_hvm_init(void)
>   	return -1;
>   }
>   #endif
> -#if defined(CONFIG_XEN_DOM0)
> -int __init pci_xen_initial_domain(void);

Why are you removing this prototype? It is X86 specific.

> -int xen_find_device_domain_owner(struct pci_dev *dev);
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t dom=
ain);
> -int xen_unregister_device_domain_owner(struct pci_dev *dev);
> -#else
> -static inline int __init pci_xen_initial_domain(void)
> -{
> -	return -1;
> -}
> -static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -static inline int xen_register_device_domain_owner(struct pci_dev *dev=
,
> -						   uint16_t domain)
> -{
> -	return -1;
> -}
> -static inline int xen_unregister_device_domain_owner(struct pci_dev *d=
ev)
> -{
> -	return -1;
> -}
> -#endif
>  =20
>   #if defined(CONFIG_PCI_MSI)
>   #if defined(CONFIG_PCI_XEN)
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 3d41a09c2c14..4a45b0bf9ae4 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -23,6 +23,7 @@
>  =20
>   #include <xen/features.h>
>   #include <xen/events.h>
> +#include <xen/pci.h>
>   #include <asm/xen/pci.h>
>   #include <asm/xen/cpuid.h>
>   #include <asm/apic.h>
> @@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
>   	}
>   	return 0;
>   }
> -
> -struct xen_device_domain_owner {
> -	domid_t domain;
> -	struct pci_dev *dev;
> -	struct list_head list;
> -};
> -
> -static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> -static struct list_head dev_domain_list =3D LIST_HEAD_INIT(dev_domain_=
list);
> -
> -static struct xen_device_domain_owner *find_device(struct pci_dev *dev=
)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	list_for_each_entry(owner, &dev_domain_list, list) {
> -		if (owner->dev =3D=3D dev)
> -			return owner;
> -	}
> -	return NULL;
> -}
> -
> -int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -	int domain =3D -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner =3D find_device(dev);
> -	if (owner)
> -		domain =3D owner->domain;
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return domain;
> -}
> -EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> -
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t dom=
ain)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	owner =3D kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL)=
;
> -	if (!owner)
> -		return -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	if (find_device(dev)) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		kfree(owner);
> -		return -EEXIST;
> -	}
> -	owner->domain =3D domain;
> -	owner->dev =3D dev;
> -	list_add_tail(&owner->list, &dev_domain_list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> -
> -int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner =3D find_device(dev);
> -	if (!owner) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		return -ENODEV;
> -	}
> -	list_del(&owner->list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	kfree(owner);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
>   #endif
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index a37eb52fb401..057ddf61ef61 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -182,7 +182,7 @@ config SWIOTLB_XEN
>  =20
>   config XEN_PCIDEV_BACKEND
>   	tristate "Xen PCI-device backend driver"
> -	depends on PCI && X86 && XEN
> +	depends on PCI && XEN
>   	depends on XEN_BACKEND
>   	default m
>   	help
> diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/even=
ts_base.c
> index a78704ae3618..35493ff0d146 100644
> --- a/drivers/xen/events/events_base.c
> +++ b/drivers/xen/events/events_base.c
> @@ -65,6 +65,7 @@
>   #include <xen/interface/vcpu.h>
>   #include <xen/xenbus.h>
>   #include <asm/hw_irq.h>
> +#include <xen/pci.h>

This wouldn't be needed if you'd let the pci_xen_initial_domain()
prototype where it has been.


Juergen

--------------0253082D65A4A2883D8F7851
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------0253082D65A4A2883D8F7851--

--fHkvMZ5xDuOMmMrlB09mkGBha2r34g40x--

--zjbIflcdM0RFlPchq1VJSgX1W9fScMSxn
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJj6AFAwAAAAAACgkQsN6d1ii/Ey9U
twf/XcEg1ogU6Ouk0Wp+4q8pEx51ZrldNzcRmrZwvbzcAy0Z3v6sx/6356qdllrkt3CUHUNUeJxi
At76G2koVtDJB331Ej3XUm5+CHkAt3ZJcnJqQJlwAXBFS4s+vwCqeVSxLCQdJgWfLqFHvhgufEgN
JdwBzATJ5ezSLErvhNytkmuf8yBKZepiCuzoZ8doMekEW/l/gOzqiaq8zD/QIULm+c3g90h1CWvA
+wiRKPUOMhSDaLejhDhI4vrCl9P+NUNMeZCC8wtWKGq2LBVxjLvV2KLNHPvxU0tT7siXZGc0iESc
ipt0EyT2aauxZ7k/l8YNmodJsybb919WKVZh4Vnl2A==
=qGyM
-----END PGP SIGNATURE-----

--zjbIflcdM0RFlPchq1VJSgX1W9fScMSxn--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:54:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:54:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191494.341548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSacD-0006Y4-Tr; Tue, 21 Sep 2021 07:54:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191494.341548; Tue, 21 Sep 2021 07:54:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSacD-0006Xx-Q5; Tue, 21 Sep 2021 07:54:57 +0000
Received: by outflank-mailman (input) for mailman id 191494;
 Tue, 21 Sep 2021 07:54:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s/tM=OL=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSacC-0006Xj-N3
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:54:56 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 07d1f8a3-fe1f-4d13-aaeb-33f39ab0b3b1;
 Tue, 21 Sep 2021 07:54:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07d1f8a3-fe1f-4d13-aaeb-33f39ab0b3b1
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632210895;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=xVOvEtuIVr+SUH+Eu2fBY3f675VT2BQYcpNoh9XwPRU=;
  b=I3r/ZuqWizNBzkW6nzhCNedcDnnyc3eenYLoZlbFn7d8e73ZgB+XB50J
   M1fuaa9TGZEmLtB1ecgtjd2A/JnYPnD/YxXsT4k1MCBPGyKp2jifup8gZ
   sXvX5nL8eEWozr1PMjcneWqG9DI2Fz9aluVw21T5eqDd7q+go4GxIjB/k
   I=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: YUz+NhDjEIic84pvVGQ3L/NtPgzrI8euDWZbeANmOq+yv1YQvWdzVYMhiaYgXy83F9nzuBgEX0
 zY27f2azEvna8OlcEtZ39bZoffLlwyv29D+xiyAEQGxXutn8gI5p3loYXaPnqsZo2sJEO9NS0+
 Y3YlxnuZIMCpUC1uowimz4gqas+US/RF3HxGIcvaeeQd0PwXfjuRnC2Hf1MEX3AD86XePbh8Vp
 X+eG02KrVqmwmP4eQzjzvh8/2QI/5vUzZrwAHN8MHWa73tlEUcfC8Fsu4gESSKzbI5Y9xWVsKR
 GdiTsOBR1ygnXQ3UrnkUmleT
X-SBRS: 5.1
X-MesageID: 53210384
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:il47w62mFr2Fn5I2fvbD5fl2kn2cJEfYwER7XKvMYLTBsI5bp2AHy
 zNKWW+Ea67YMzbyco12Po/n9UMA7cfdytBlGwA5pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970Eoywbdh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhlOAtx
 9VAtYSMZB4KBZySoLwQSB9zDHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQChW9r1pwm8fD2b
 csJYxxUdS76W0MRInMHGMIxtseziSyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1Fk0Ddq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PL+y++NugVaT7ncOExBQXly+ycRVkWbnBYgZc
 RZNvHNz8+5iryRHU+URQTWzkCKLvhMcQ+Z9Cs4B2V+1kqfP5S2GUz1soiF6VDA2iCMnbWV0j
 QbZxIK4XWYHXK69Ei3Gp+zNxd+mEW1MdzZTO3VcJecQy4S7+OkOYgTzosGP+UJfpub8Hy35i
 xuOpTI371n4pZ9Wj/jnlbwrbjTFm3QocuLXzl6MNo5GxlkgDGJAW2BPwQKAhcus1K7DEjG8U
 IEswqByFtzi6K1hcwTWGo0w8EyBva7ZYFUwf3Y2R8RJG8uRF46LItkLvWAWyLZBGccYYz75C
 HI/SisIv8Q7AZdeVocuO9jZI51zlcDITI25PtiJPosmSsUgL2evoXAxDXN8KkiwySDAZ4lkY
 szFGSttZF5HYZlaIM2eHbZFjeBzmXBgmgs+h/nTlnya7FZXX1bMIZ8tO1qSdOEpqqSCpQTe6
 dFEMMWWjR5YVYXDjuP/rOb/9HgGcio2A47YsctSerLRKwZqAjh5WfTQ3akga8pumKEMzrXE+
 XS0W0l5zlvjhCKYdVXWOy47ML6/D4xiqX8bPDA3OQr60XYUfovyvrwUcIE6fOd7+bU7n+J0V
 fQMZ+6JHu9LFmbc4z0YYJSk9N5ieR2niBigJS2gZDRjLZdsSxaQoo3vfxf19TlIBS2y7JNsr
 7ql3wLdYJwCWwU9U5qGNKPxlwu85CFPlvhzUk3EJshoVH/tqIU6eTbsivIXIt0XLUmRzDWty
 AvLUwwTovPAotFp/YCR17yEtYqgD8B3AlFeQzvA9b+zOCTXojijzItHXLradDzRTjqpqqCrZ
 OETxPDgKvwX2l1NttMkQbpsyKs/4frppqNbkVs4TCmaMQzzB+MyOGSC0OlOqrZJl+1QtgaBU
 06S/sVXZOeSM8T/HV9NfAcoY4xvDx3PduU+ORjtHHjH2Q==
IronPort-HdrOrdr: A9a23:GLNRFav67bzJauL7NcBl5BE07skC54Mji2hC6mlwRA09TyXGra
 +TdaUguSMc1gx9ZJhBo7G90KnpewK6yXdQ2/hqAV7EZniahILIFvAY0WKG+VPd8kLFh4xgPM
 tbAs1D4ZjLfCRHZKXBkXiF+rQbsaC6GcmT7I+0pRcdLnAbV0gj1XYANu/yKDwJeOAsP+teKH
 Pz3Lsim9L2Ek5nEfhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2n46ctHNZA7dw6MLmK41r2MGTx2VKpUCfa/Z48SgfwQr7Mkf4IDbKRCdY1JKVbou
 W3bLofjx9qR37T
X-IronPort-AV: E=Sophos;i="5.85,310,1624334400"; 
   d="scan'208";a="53210384"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g8OZN8zqlsdseTvzTm8b0XVADGPqS69uc7xKnr0KckNb+FD6wj/VaaXx12YrY7HmqvguiPiYDtkSNv7wFV0o4rBrKZGtMzpDh2aSOVDluTfsmP9ax76VF8swpsMn8lyxEsJ2A84iOSbQDODOizVh31t5cpROIzLJFr+HijXnaXDReLJZo2+1zwdRw39V5z/4sMD76TB+6hGDWlcX5r/9JYc6pl7v+NoUCxxizGTkUc5I2ORHPROriSe9yZnQe4z7P3XCkwMJ/kJPxMWlEQjQpefFb2nbygvOMIEcqjj9wNOg7XUPWz/iRRUdTcTjSJ8OqJ85bV7WFcLJ/0QA9neQ5w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xVOvEtuIVr+SUH+Eu2fBY3f675VT2BQYcpNoh9XwPRU=;
 b=eOn4YZhkqBAMwRRzsxfOvTDQXJy8Eofqy8yBLX6b3vXHhH8MhgJ5BtFa8NjvThAck7Q4fPMiYNCkP9Te2FZSMQyUEaxvKHRV4YWzSefi8/ZB9YIzCl1wfzfks1fWzZZ6oKgOT9coREpabubgr7E3yXjULbmxUs33/Lz7O63fDqp8SfmZcbZ1qnTCYBBGqbwGI27lnIq6/iq/JfC9eRzlJpj+KoDyAU7WYDYNztMKOVXMIlA0D8yvDskRATOgLZGwxYe1kG2w+J4UgAfRk39Cadrq8WPwWnzE3w6NC/tUXrGcLZix+wpSnuJHF3Mdtg6Z2GTrRMlnMMLLRk9YGI/1TA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xVOvEtuIVr+SUH+Eu2fBY3f675VT2BQYcpNoh9XwPRU=;
 b=folHgzbxKmPH9QEWkGTXD4fFPM5aGZnqZHCWC+0UX0r0wbXXWOSrqu7vgv2zH87AkXfe2CPDrgPnutYiiOp/NDC0T8eZjLLBGizKiqIaVkvmJCj7/5DUIM+iOcga9ZErB55tMEH/IUvb0g6jm0WfzsZZuqkqaWMoPDBmKwGRzVQ=
Date: Tue, 21 Sep 2021 09:54:44 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86: drop a bogus SHARED_M2P() check from Dom0 building
 code
Message-ID: <YUmPxEMvOzrJjWXM@MacBook-Air-de-Roger.local>
References: <47deb99d-3ffc-61e4-26e6-7e7ab186a79a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <47deb99d-3ffc-61e4-26e6-7e7ab186a79a@suse.com>
X-ClientProxiedBy: LO2P265CA0283.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a1::31) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c7093002-f347-4afa-db94-08d97cd51665
X-MS-TrafficTypeDiagnostic: DM8PR03MB6216:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM8PR03MB6216623F2CFCED46BDF9F4458FA19@DM8PR03MB6216.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: gYzHlkQhVD9KbPmdJRxD8MCE/O8t3KSiejnFMLrvzjrc3OYSPxNc2uw7PMIDi5phMK1P2EhX78clbftck7xZ+VOnScVjkxF/wyp11u6IKWmDUdUOF74mrA85dhNkfJtT9dBwf8mVOHGPfKEXNqnOZA+K3wrYE8EpLFLQZyOC+U909DXw+CUBp28A4TQJ5umUc4mAqaAPflpvEw3CsShNBiCi7XAfVyDjM8AV3uMCMGZKAdOLPbdGnOyrKvilp/wu/Zb7CjuAO/KI+Nvb82lc0EayHmZpeJCmUIzM+PhQHppUUaG+DIZri7p0L3lA3qIwe5+32FFBtVL9QvmE6FaCVEghdBnKbBnU+GqzsrzYhv8wiwZ2J22QNdwss84wx2i+vshc+fL68PpJ75dmOBkE8V3m5zgtp/QlobhABwJE5Qf+fVlmaKZ+bTcX+GVThDkJ6gAxbqjvMZ0TY+0HZo/qghsabrEDVjqtCm36xB5ypeaTc0QpzDmtr+FrhgE2LkyVxPVK+D06MUeIFtUMkamG/ST9j2cVgjpwGw20lZUjMVRU6Z2TwEuVtTYH9EwQCj7zOcnYzn/pRjv71OfeHiiXHfI5wQDQdv5ELN4F9N2+TbIXToLA8a8xwgdmKCKheVFyGVgcFpdPPC6wY+g9u1ZKiQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(8676002)(5660300002)(186003)(6496006)(54906003)(66476007)(508600001)(316002)(86362001)(2906002)(26005)(66946007)(85182001)(8936002)(66556008)(956004)(9686003)(4326008)(38100700002)(4744005)(6666004)(6486002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eEhaNEI2VHNWWDZKaTV5RzR1Z3NJZDdvSzQ0MFNJV3laandudWZXdFQwT1l5?=
 =?utf-8?B?Y2FURG1jdWhVMCtJeURsd2ZLWEdJVHJWT1NqS3AySHRZNTBHekMyRGVPQlND?=
 =?utf-8?B?aXloWmUxUnhyV3g2UkJPQ3ZmUldHMk1NRUJvYi9oQXNkZTd4Mk5pM1YrR05S?=
 =?utf-8?B?bmhBcHdscktXZmpETnVMQmhGUkg2M0ZxbEc1QURLa2hBcWZNdjhtZE9tUlNP?=
 =?utf-8?B?SWZlT1BZY3liaC9Gc1dZaVBDMHZucHMzbHpVZmh3bUlubHYxY0pNZkFNTXdx?=
 =?utf-8?B?TGJUeTVNQzBLQTlOcEJ1blZ0STR2NzZ4bTFnTXZLdEpjeWpnUVdWcnI1cUhi?=
 =?utf-8?B?UGduTjczQkp5V0pRcDN6UW0xTFBRSXVMR0JBaDJVNjQxZUZKeFZxaW1wek9G?=
 =?utf-8?B?TWdHWTl6NVJNd3hIZFR5eHRWTUF3MW5qZkh4QVNudmhLL29sa0RQMFhQRHFB?=
 =?utf-8?B?dHNZK0phRWFZYllhdHFBOU5Sck5vemk2OHROeTdEM0V5YjEwVUtOVU5mVDdp?=
 =?utf-8?B?SEVnR29BM1NKcmEvdkc0dDNScCtkcnNPN1J4OEQ4cVJwQW9lWDA0a3VUOXJ3?=
 =?utf-8?B?d01Ncm9lNmNsZ29HVjRxMVY1MUE1VWx6SGM2N2RHZU95RHlvNi9BaHkxb0JM?=
 =?utf-8?B?dlUzWXJiUjFLN2lkUDU0UkdxeHhvRnB2L1phQWpOUEJMWE5QSHNWZ21Nblk5?=
 =?utf-8?B?ME5VbVRMNGtmS3ZnZEhCclJaTDhWTTM2SVVpVWNNK1Y4NEo0UUltSlMyWDRR?=
 =?utf-8?B?dVVCbi9XQng3Kzluajdpci9WU0JSdlNoN2RKSno4dTBGc2FYZGhPNTBRRnh1?=
 =?utf-8?B?d0VZaG9jZlhMV0xET1hIeFB5dmJkTHpSdFg4Y0d1MG9wNUs4QXovSjZXaWZ6?=
 =?utf-8?B?MmNxY0ZZS3kvd1JTWnpxS0FhQjRWVm9MY1ZDYytYOWxjREVjMVBjWEo3OGF4?=
 =?utf-8?B?THBSM21JWnN3cnFyYndjOEpoMko5UWU1MEJFSExzZnhpNkNqRHZIdFYreDEr?=
 =?utf-8?B?dzNuODl6Um5RUDNOMHczR3c1M1daQ3doV3NrUmlLNmpib0hyeW0rYUNFK0lh?=
 =?utf-8?B?ZlZERUpNWWhBbkRDQ2ZLVTVGZzZwZWltU08vWmtLRStPenlIQnVKQVlhU1J5?=
 =?utf-8?B?cTB2bkMwVndXbDEzb1ArY3hiMmV2czFVd1pQclFYb3NZMXNicm0xQ3ROdGhW?=
 =?utf-8?B?UGJLRk5iNGQzVWRPa2I4SXhHck9NenhDb1JML1BacG93QXFJY3R1YUc5NFpU?=
 =?utf-8?B?VHQxMTM4YnVMaTVzZnhoaDVYckYyTEFTT2pwbEQ3QVNKOXFHcjBHRXRyZlBE?=
 =?utf-8?B?akkrTVdVMlMvUmdNWDhPY3p0V2Q4ejgvejdqaWJoOHZjdzcwemRId3RGeEpN?=
 =?utf-8?B?SlJ1d25sTVdWRVpRdkZiblZwYzhOVlBqR01EbmZXd2I2V3llYTluVHptSHZH?=
 =?utf-8?B?RkZJRnRBeE1ndVNTOEwxUyttSDlsbTh3bmRHVnR5Vm8rbXR4SXZ4UHBRVjZE?=
 =?utf-8?B?UE0zNWdSWDlKejV5N29VRUNiZzJLRFFVMVl0NCtXNEhRU3U3eFlBV2ErRWdq?=
 =?utf-8?B?OFZVQngvbUY3OW1OcjREVndHMWJIZVBvRXRiQ2xuS2o3cldwTVRsUlFGM0F2?=
 =?utf-8?B?N21IdUpPbkRIbU9ER1lIeHhjcytiWDdFbnMvbW5lVDE1bkM2ZmZPSlA3ZDJk?=
 =?utf-8?B?QTZENzVLM0dhM2oxZVQ5cVFBZEVMdHlsaUczRUdSaFV0dGRmUVUvM2NLbVg4?=
 =?utf-8?Q?kQxLq4VUVpULl92x6DrWQ7JBzSEUOG3ReJuFOdQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c7093002-f347-4afa-db94-08d97cd51665
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:54:49.5925
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6IC0d2BYkSH8Zlsm4mf+s5wFbJi46YuL8ZUbA/yiJssTqaw0NjnwPYlNqBGNer0r5TdZmTpG1aoMjr6JDwm8Mw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR03MB6216
X-OriginatorOrg: citrix.com

Could you explicitly mention PV somewhere in the subject?

On Mon, Jun 28, 2021 at 01:52:52PM +0200, Jan Beulich wrote:
> If anything, a check covering a wider range of invalid M2P entries ought
> to be used (e.g. VALID_M2P()). But since everything is fully under Xen's
> control at this stage, simply remove the BUG_ON().

AFAICT adding this BUG_ON was a wholesale change to protect against
shared frames, but I also cannot ssee an scenario where we can get
here with shared frames in the m2p.

> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:55:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191502.341559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSad8-0007Ba-7P; Tue, 21 Sep 2021 07:55:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191502.341559; Tue, 21 Sep 2021 07:55:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSad8-0007BR-4D; Tue, 21 Sep 2021 07:55:54 +0000
Received: by outflank-mailman (input) for mailman id 191502;
 Tue, 21 Sep 2021 07:55:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s/tM=OL=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSad7-0007BJ-6v
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:55:53 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 572d21f1-1ab1-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:55:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 572d21f1-1ab1-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632210951;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=pinY5uab9YMisebPyArXJODX0umXo3/vbcTOQVwGIxI=;
  b=LYkdYXvhnUKI6Dk//sJG9IiwQffMwBcGNh5yT9WRCF5rVu4Dk4WtXJnS
   BHzDNqFmvP1I0nyOromu4sALmeKCA/3t04OLVJ/CQHmlWuLgimV9V0ZQ5
   op90xxtRqExUWsdJzorX7sTce3DtqVK4qnX6rJohlyVXm/KmI6I0UTJwE
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: mHrOAWfceGdnhFEUfhEVdWymLvnc+T620aCFtaCvFMxnDnixeVnmH0PnI6jGqzsCFUQmACpqti
 QxZeVmIGK4oThQ0FtwrdfYhiOH+EeP/SKnCzeAwFciPptRQQ0PJpEftlWkomnR753BXncFl27b
 sJRk4BQVsoTiFi+REtYLMjU2Kpm5iv0EgyKzJOBCh0LQ8lTA3HL+8mnFGxmRSTQB2/hxDLvgQ/
 rq47NZ4IiO71pMdnaERUPW3Ea9aRJMrdHgxbIAFxI5IHyiiJBDy29qH/8cF6oC5Npg9vkR+nAF
 XFuh6bjyqHwwXOFZviJd7VVt
X-SBRS: 5.1
X-MesageID: 53210432
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:z+d2a6+n356JC1ea7AAPDrUDqnmTJUtcMsCJ2f8bNWPcYEJGY0x3z
 DYeDW6OM/aJNGL8c4hwaYWwpE9Vv8eEmtZrTFFo/388E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j29Yx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPgp8
 u9nioTrTTsML5zVqvg+EAFVPCZhaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFh21p3pwVR54yY
 eI0YmNudRmQaiRgJ34ZKao6teKsoifWJmgwRFW9+vNsvjm7IBZK+KjgNp/Zd8KHQe1Rn12Ev
 STW8mLhGBYYOdeDjz2f/RqEmevnjS79HoUIG9WQ9PRnnVmSzWw7EwANWB2wpvzRt6Klc4sBc
 QpOoHNo9PVsshzwJjXgY/GmiCa+hDQjQ4FUL8gr2Ri/1omX+wu2AXdRG1atd+caWN8KqS0Cj
 wHSxoOyVG036tV5WlrGqezF9mra1Tw9aDZYPH5aF1NtD8zL/dlr5i8jWOqPB0JcYjfdIjj23
 znChyw3nbx7YSUjhvjjoAyvb95BoPH0ou8JCuf/BTnNAuBRPtfNi2mUBb/zt60oEWphZgPd1
 EXoYuDHhAz0MX1oqMBqaL5XdIxFGt7faGGM6bKRN8B5qlxBBEJPjagPuWojdS+Fw+4veCPzY
 V+7hO+izMYIZxOXgVtMS9vpUawClPG4ffy8D6y8RoceM/BZKV7clAkzNBH44owYuBV1+U3JE
 czAKpjE4LdzIfkP8QdasM9HgOd3mXhhlT2MLX05pjz+uYejiLeuYe5tGHOFb/wj7bPCpwPQ8
 t1FMNCNxQkZW+r7ChQ7O6ZKRbzTBXRkV53wteJNceuPflhvFG07UqeDyrI9YY112a9Sk76Qr
 H26X0ZZznv5hGHGdlrWOiwyNuu3UMYtt28/MAwtIU2shyopb7Gw4fpNbJAwZ7QmqrBulKYmU
 /kfdsycKf1TUTCbqS8FZJzwodU6JhSmjA6DJQS/Zz07c8IyTgDF4Ia8LADu6DMPHmy8ss5n+
 ++s0QbSQJwiQQV+DZmJNKLzng3p5XVEwbB8RUrFJNVXaX7AyokyJnyjlOIzLuENNQ7HmmmQ2
 TGJDEpKvuLKuYI0roXE3PjWs4ezHuJiNUNGBG2Hv62uPCzX82f/k49NVOGEIWLUWG/uofjwY
 OxUy7f3MeEdnUYMuI15Su45waU77trphrlb0gU7QymbMwX1UuttciucwM1ClqxR3bsI6wK5V
 3WG9sReJbjUatjuF0QcJVZ9Y+mOvR3OduI+MRjhzJ3G2RJK
IronPort-HdrOrdr: A9a23:6pO8+azAnZ9S+vO1+FTHKrPxt+skLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjmfZr5z+8O3WBxB8bYYOCCggWVxe5ZnOnfKlHbakjDH6tmpN
 pdmstFeaPN5DpB/L/HCWCDer5Kqrn3k9HYuQ6d9QYUcegDUdAe0+4TMHf8LqQZfngjOXJvf6
 Dsmvav6gDQM0g/X4CePD0oTuLDr9rEmNbPZgMHPQcu7E2rgSmz4LD3PhCE1lNGOgk/jYsKwC
 zgqUjU96+ju/a0xlv10HLS1Y1fnJ/ExsFYDMKBp8AJInHHixquZq5mR7qe1QpF7d2H2RIPqp
 3hsh0gN8N85zf4eXy0mwLk303a3DMn+xbZuBalqEqmhfa8aCMxCsJHi44cWADe8VAcsNZ117
 8O936FtrJMZCmw3RjV1pztbVVHh0C0qX0tnao4lHpES7YTb7dXsMg24F5VKpEdByj3gbpXXt
 WGNPusp8q+TGnqLUww5gJUsZmRtzUIb1i7q3E5y4yoO2M8pgE886MarPZv6EvouqhNDKWtSo
 z/Q9FVfYp1P7wrhJRGdZA8qPuMexzwqC33QRCvyHTcZek60iH22tXKCItc3pDfRHVP9up1pK
 j8
X-IronPort-AV: E=Sophos;i="5.85,310,1624334400"; 
   d="scan'208";a="53210432"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DBJFW/QMPLxcAscVnu1vAXb3++1QPhA9uJu5MOrlNDKmgByXur0IfAy3LQHLixk0/zyGyqoYd4DErXcdjAhie6Am0L66SdbFPAXf4zkKaumQMUsl7mmRjPCdqJAFPKngrJDwHzBydHuWqAqcpt/AfRNn+/ETXzrzln3FQCInHBcDK9JfmM6I3Gv2Tv0uYgu83iVHgFS4lfL4zua1uutDAPGqFFhKe4L0iKNDMf8qS86n5eAD2CUyBjmJBevyobppCWCRUiZ/LoPr7IzLcSlQOCYN5gDdveNDuNebilclUD2bUufBgA1E3lhBjjVUIfYRzRPkJbnagWpFO2pw3+WwPw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MBnF5FzmPnb0gAxehp+w3uDQohymLw4+XjrSrGtkyec=;
 b=Fxd8aWCrd7skIQkA0oseYMlOXrX8ordUliQXxop48+MvB7+ZD7x1kmbH41j42chKL5JVDcylDCNl4aDMF4+2wgS3xXmNikji+1QZTDcEYDdlbGTeDhhQsqXkfcG6gLFtIFZOGdaRxZre97QizbHZ6GANwv3py0smF+u2ErmiLcLojG5NaEFxXJgHYteQ2HmKKJ7LaPdgATCnj/PCJ7ApeGpWr6aZLzUgmrhJPFOOmyR0CGR38QQLfImrOXFQSQZJxGqJXnZH8bNbtA9yrADiE+I0iGkcRjIeg035TTBuLDnCg7kvcFCeljcNaGTxfGOT/DX96FuUKsg9Q8KRfhlorw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MBnF5FzmPnb0gAxehp+w3uDQohymLw4+XjrSrGtkyec=;
 b=vE9Zf7yfUydctgeGLE94muBKx/E1S+O89ArqSz0ZX995bBqdIA/8kAiYam+ogCS4zxeqVRb8hZt8COsqRGdG7bRMX6MnP0kK+nvWoaUgHrD93VIwMJO0NSLIjKkFegpgaYdXoT8xAywsJfYFMinnwvgxUtqRpA+6/fewOorfMhA=
Date: Tue, 21 Sep 2021 09:55:44 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: =?utf-8?B?UGluZ8KyOiBbUEFUQ0hdIHg4Ng==?= =?utf-8?Q?=3A?= drop a
 bogus SHARED_M2P() check from Dom0 building code
Message-ID: <YUmQAIGJaK+Uh/UT@MacBook-Air-de-Roger.local>
References: <47deb99d-3ffc-61e4-26e6-7e7ab186a79a@suse.com>
 <0accd732-71a2-b122-6c7b-bb8fc8b0f3cf@suse.com>
 <27bd50f4-2216-ceb2-437a-f9eaa8f7c019@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <27bd50f4-2216-ceb2-437a-f9eaa8f7c019@suse.com>
X-ClientProxiedBy: LO2P265CA0161.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9::29) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 35937447-d0b0-4891-bd75-08d97cd53a16
X-MS-TrafficTypeDiagnostic: DM8PR03MB6216:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM8PR03MB6216A6D566C20718BCF566178FA19@DM8PR03MB6216.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: moaGhI8htQ21qgBkquTl01/fHEhG9ddH0DXLqR2sBLWZndbEOhlTEcNKCBM9rZSaBplLLkm9/0uEr4NlUvjSulu10wjoXO5fvq6U0RA3BzwjgLDzcXMypkTXllF5cXohlPSC/g39T/SI7LDus5FhOAP9MwuVYlM6c1hsdHhIECmSr3ZxboPfEZ4p5cECTY5CDOFj3cA+fHDUUtkNNnZ8kwKgZX/guV83a4MO9NRdHxwbmuZkbm+CuUJn4EpZIqCMW4irF4plzB0WyQdSchFK+wYuPo4uQHy2tn/PHmbuv2Sb1OPeAF6L8rhIxuGjiUpZnabYJyeGKc702LRKS+iTJB4VBolPkGK0LeDIdMs7GHVE9WUzSX7AXHWF5gcrUcn2AkFwkeQxlpjkFiVdPrkha00NVuOqiysYrZT/9MPOb2bas0QO87/un0r2jP+i4I66oWmPCXswZAotBMw4JA2CzRwzwoWdjLWSQq6AOeM8GNvw3UslqJB7w10nR0JWZMw8w7tGHbWW3mZ//AmXu5s49oaUE3AoqZbomnDhDqTkSjeFIr1+4KF0cxYob25LytUUMhO7X0X0XSzzAhGw6XxIduHfT/XRuTLAT3/dGzhzfjZaDvuauMAaZm9HsiQ+GU1xpVb56sGoAE6GkbWI+B+euA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(5660300002)(186003)(6496006)(54906003)(66476007)(508600001)(316002)(86362001)(2906002)(26005)(66946007)(85182001)(8936002)(66556008)(956004)(9686003)(4326008)(38100700002)(4744005)(6666004)(6486002)(53546011);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MGJOaVBPT0ZjbnFHdzdyTmZScitUa1R4OSs0SitrbURhOEhqVkJMNjc0cTB3?=
 =?utf-8?B?Tlg5UUxZb2xKK09UdFhmL0p4bDhDTjVaY3RkK0Q3TkFBc2NIM2RzUUhaRXRE?=
 =?utf-8?B?WWNjNWNaVUE0azUwQmxvZFR0MElsWGtaeGdQMlkyVE5FVlBwUzlPOWFVYjUz?=
 =?utf-8?B?RjRvVzdINjVBcUpzR01kaTZYSG52OEUzci9CSzIxdmsvM05CRUZjOXlHOGNx?=
 =?utf-8?B?THBVNnQxQStMYVdqSFFzRVZyRXRxT295OUNUTUFBWEVYd09XbHNtenVZSUtN?=
 =?utf-8?B?dVcyOFJYRTAyb2NSRW1CQ2xOM2hWYlpUa21LTnNyNmI2VktFTkFFSlQ1ZjRB?=
 =?utf-8?B?Q3FlOUxCOE4vTGZ3OHZjWHRQd1J6OTUxeHA4NXcvVmZWakdrb25MWXRxOTlX?=
 =?utf-8?B?cjV5ckg4MDM0K0R2djVvWG4vL3l5WmUzcks3ekY0YnIvdDQ2YTRtWVV5eG55?=
 =?utf-8?B?MldEZmNxbE5nQXUyQVBGN1UzekZhMTh3UEkyeHlMNFZmUXlLSytXREo1eVpi?=
 =?utf-8?B?a1hMSk1HM3RUMXZwTFQ4WkZuanFHMGhLanFCVHVwWXdRRzc0dWpXaFVzVmpO?=
 =?utf-8?B?Z1gzTmExL1FOODN4eGpzWHRUNk1pWWtCYW9lU0Y1SDNMVFBVK3QwZFA1dm1u?=
 =?utf-8?B?N2tmVkw3cmFKTnI3UGV6eUQ4OTdVKzVpaWtRUS9iTjN2aHZKV0xiRVVWTDJ4?=
 =?utf-8?B?cytGd1VUai9sTDJDMHFSNy96bXl2K05JRjk2S0UzVkJhLzhXNzdUaDJldnRS?=
 =?utf-8?B?d1R5cVpPeUxFMW5JWGU3RktkYUdaVkVaZmFnaU0vbEVqakN2M0NNRnExVlJM?=
 =?utf-8?B?cEFFbkNWWVF1dkQ5UFlvQXhES2JhWFk5RWszN3ZSajFubWJBaHBxaVRZa3JI?=
 =?utf-8?B?WWMvSWR1WW9qUHJ5M1dyeHk0NlZIMmVDNitCUWZFMjVYcmREbStpVUxyaEVH?=
 =?utf-8?B?SC9ZM0lXa0VYMG5HNlgxYW95WnUrM1BNNlMyOXdSVnVPdWdzcjZZd0xuQ1pU?=
 =?utf-8?B?ZlVzWGhUOHFoalpZRWM0NElhUDFhYmFSK1Q3NXhVK0h6VThoa0MxQm9vVjVZ?=
 =?utf-8?B?TXFxME9wUndlU21WSWNHZGo4ZmxpVkVCSE1Fcm1QY3dYNEJSTnRlekVDVzRK?=
 =?utf-8?B?YWkyR2JaRGxaOFZCOFg3NHFUU3VtZlRPZGg1NC9sL253bHZkcHp2bTBKMlJB?=
 =?utf-8?B?cmRFcGN0QkRYb1ZYWDdOTmZYclJPYmZHUW40Nk41WFdQb09nWUVkcFA3TTN3?=
 =?utf-8?B?bVZudm5JVldLTWNQV0tKNEVjbEJaU1pFTnBBYklidjl1UWM1UWdQMnNCZkh6?=
 =?utf-8?B?YzFTOUlHelNqSXBzeFYzU0E3YXdvK3RxdFppa3Fpb3BGdGRXZEQ0UW1HeDhB?=
 =?utf-8?B?SkxFRmk0M3MrbXhtajRSZ2EySjA4NnlBSkFQbU9EZ08waEs3WTgwUy9tR1JB?=
 =?utf-8?B?b2I4N2t1YjVuNmthdkdPUnJ1MXBYUEFmc3NTR1ZUQi9VMHdNMGtMNGxjUDFJ?=
 =?utf-8?B?Q2tDTHlYbU5iMzZjR1QxNHZETWJCeFIrN0Z6SWJEVkdIb2QwMHJEWkdoOEFh?=
 =?utf-8?B?MHZDdm5lYWRBcm1sZ0ZocnJMZCsvREEyb21hanpkRENrSDQxWGJ3RFJuNzhF?=
 =?utf-8?B?M2l1OEVqRzV0RDJZdUxEMEg0Nm5UTHNPVlhLRlBDakQzY3VjNHhudHFWc2dM?=
 =?utf-8?B?UzNHRU5CUHQ4QzlqeEVidWxwUW82a21XRUJvZHpjVkd1MGFoUnZlT3Bnbzg2?=
 =?utf-8?Q?8S13Ua98dIbikX3YdqXu7U51djiMg2Jo1sTThOB?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 35937447-d0b0-4891-bd75-08d97cd53a16
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:55:49.4720
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1roL0l2CG8YobhvfgbDdDQOhbV4HOSlDhGIYJYT0Y45Q5Q7pWrXqG2Di0CQ3NPsZgg/U3V9SYIFYRUj/EeVi5w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR03MB6216
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 08:10:12AM +0200, Jan Beulich wrote:
> On 06.07.2021 09:37, Jan Beulich wrote:
> > On 28.06.2021 13:52, Jan Beulich wrote:
> >> If anything, a check covering a wider range of invalid M2P entries ought
> >> to be used (e.g. VALID_M2P()). But since everything is fully under Xen's
> >> control at this stage, simply remove the BUG_ON().
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > I didn't expect this to be controversial, so may I please ask for an ack
> > (or otherwise)?
> 
> To be quite honest, I find it very strange that even simple changes like
> this one sit un-responded to for months. This isn't the only example ...

Sorry, I'm trying to catch up with reviews, but after being away so
long it's hard to figure out what's pending.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:58:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:58:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191511.341570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSafR-00082x-SA; Tue, 21 Sep 2021 07:58:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191511.341570; Tue, 21 Sep 2021 07:58:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSafR-00082q-N4; Tue, 21 Sep 2021 07:58:17 +0000
Received: by outflank-mailman (input) for mailman id 191511;
 Tue, 21 Sep 2021 07:58:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSafQ-00082k-0R
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:58:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed87663e-8798-4531-855b-ac831316c95d;
 Tue, 21 Sep 2021 07:58:15 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-l9tVX36FOdGSD20lT2vsqA-1; Tue, 21 Sep 2021 09:58:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 07:58:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 07:58:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0129.eurprd06.prod.outlook.com (2603:10a6:20b:467::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Tue, 21 Sep 2021 07:58:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed87663e-8798-4531-855b-ac831316c95d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632211094;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aZliVOe7Xmlc0GHxrcQt3IFb5580D1LkSDKMEF2ykdA=;
	b=ZZGx231HJGdX0t6Vg4KogVJH6yKYVNrLlUrRuNo9RvEvRHZcE+SmBNfMYw0H7brtJE/wff
	DI28gPlqW8l3LHJETw8p3br+H0jRiBKsYGn5KO4ygXiAXTtgmlrQPl3433mzGb5YqZrTrI
	3AIr6ObIGXFE/qDyhIXVi64mC4/4stU=
X-MC-Unique: l9tVX36FOdGSD20lT2vsqA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W6Ft1wh7xHVu2uM9YUNOmD2KTD5HvMhF4yDrS2JBEzjFzB1R3K850GNuhWqSAiGEo956WVBPWjk8Rx9+yleKkBqweWE/BHQzNz+odE59Oxy0gW6i44vbjN0aIs7TIWOYAH3RhOXhxCK/t3XGHfgBwHWvP/pkisFJPBdDDZ8iZyOmFc5ek+DVI8fSSbs6evSA+xTtlRtBWgtUtDO4WjzI7/wP9v380rg6y9AiX1uzl4dwkSBYYXVEPQ2u0o8elmgt5NpC3Mj6r4O1dYm6EqrRn2YOY8yN7asueFPw5csGaQmTTJb78vetC/io1ACiciYy4G942g7QBTshvjys9+50vg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5SdVMlKdK6Fp3DvkmECiR7cppxy477qPRMs5PbUQaJE=;
 b=WWtzBhb03fS+mmXDHcYeb2BwARV/9ujQ+cD4QUI0jTGVooSEyCwO91eCqZ3WIRIRV12BruNfTIix2ETRJCQUZAHrplyHybgo3MiFKtFj1GnSeOTbDWZFHDepSTxCcs1PxCIL0OmDkUeog/ontYcTYR8RtWlm7a0/dliFo7YlnLlVCCAJM/1JjGhvr541Dk8c6VlhnwInGmV+3ZLphV+kkrZzUTayued4h2Znx1LeySXaFmiuIuwTIg9bkz7Rf9E4yPAKjiitG/y2VPvmoBPClFNC+SlLCCMwQCcQHHsTzb6liTqOxgWEEJxelEACHba7oP1GtXG19ZmKAFSzWgjROA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: =?UTF-8?Q?Re=3a_Ping=c2=b2=3a_=5bPATCH=5d_x86=3a_drop_a_bogus_SHARE?=
 =?UTF-8?Q?D=5fM2P=28=29_check_from_Dom0_building_code?=
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <47deb99d-3ffc-61e4-26e6-7e7ab186a79a@suse.com>
 <0accd732-71a2-b122-6c7b-bb8fc8b0f3cf@suse.com>
 <27bd50f4-2216-ceb2-437a-f9eaa8f7c019@suse.com>
 <YUmQAIGJaK+Uh/UT@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d120fac9-4e7e-1f38-90b6-b02ebd05662b@suse.com>
Date: Tue, 21 Sep 2021 09:58:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUmQAIGJaK+Uh/UT@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS9PR06CA0129.eurprd06.prod.outlook.com
 (2603:10a6:20b:467::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ed3a01a0-dc7a-4607-82f3-08d97cd58e49
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB736020326CAF9D876123E789B3A19@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oYUjsr2eySHgvWY6Mt6Q2Jz4Uv8Giha2KWSDCm0os3YxEDP/kWpn+dTqlI52DtxPmMUQn9k+CqoKbb6myqPJG5LjAOgblewtZRWHqToYa4P9QZjOx55gAmdvf+6DcP+YchfUGIpB6mx6CCXTtgps4DtEFtekry1GD5c7EV/sxPcFIW3/3PooHqp7eOuvlZyItDtfugCMksYL9psh25Skw1hWHoM6mNdmvbjGx0zjvj5P2HEFuW/SBMIvCv3rAB3uorlNXjych1fObpUTUe5NsGpjrpm22KOblE/XBLqbpqYXK0a52tM0lAoMilZEIvHP9TkY/A0iHdRbmK4UY9ceGu3dfdYE9Vdb+twbhUwmCEN8TNA+tKoxTrzaYwLmDwJhPZCQNeHi01YBM/QjAT66XAfYRNVkkzocmhlScp4lDPamYzEfZiNGjqr83lIWaQ/Fa+R+zwlfh87V5LCwIqdhjTD0UetkZ1kyDBiOY1hkpXaQdO5NAw9JjlBLu+qT9KX6kK/Oqb7wy0LO9EaZV+c7N7Xba7KO1oQgz9FDxmLwZXFsUskkLiNQaNqsBQwG/nFS2BeynnssBgITSZxP95QD0bKrJHhTbJq3mM2aKzYZU2baaekLD2qz6SiKpBu3uSNJWXDe5XIAxoCKeLGxT0OjLibVIFMK+Mg4jFhWt9h1r5s331oc5zofQJp7IVDAxeQd1R8nRy+875CjiOaqPLcNqs445ExAxIPR62Z3XM/F0BI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(31686004)(8936002)(956004)(2906002)(2616005)(54906003)(36756003)(316002)(16576012)(508600001)(66476007)(86362001)(66556008)(66946007)(186003)(26005)(38100700002)(31696002)(53546011)(6486002)(6916009)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?OBnLotjjRLo2uh+UB3f6cqeNchgm/806braiAK+lY731hTd/Kbs3Jl7yr/bG?=
 =?us-ascii?Q?paKQZcduOhAfyJp6LEJoDD/71IzAwkBS2oV7XBPJ5VezmD61SbdgFJuxlX/n?=
 =?us-ascii?Q?5zphoT7GHhhb5pQJvDkbxGt6yoQ+XInDZERB8ZVAXTL06DLNV82PtZaBQ4V4?=
 =?us-ascii?Q?sgcPg7sDMpID/NX67dG7fla+7Tk0Vrb3puY7pcpYV0ajijKLn1Wbg0GC/+Uv?=
 =?us-ascii?Q?o81/kD5/U04JyzefOk4K6S9golAwMz6IP1Nz0EemSsSFzutIPR6oBwFj/JBo?=
 =?us-ascii?Q?Js89L10WTa0ViAlWRkkyhU/PAyPAYglJnWQLXeTQLH3/GAXVN2hgAsg9UCOq?=
 =?us-ascii?Q?hW6zdT3nMFf27Jh5HFzyHaKCktkKjzs39O+S4XSwE/yJBEUtk1drJrfBinVl?=
 =?us-ascii?Q?lcyLzSrbNMX7mBIMgCs21vAay+Lwy8X/FX5IAYe/Ad3vkeGLxiu+n50Y1lX9?=
 =?us-ascii?Q?aKQkIHqkXW1OE1c/vbyD1C9uwOoYDKXYIVWe/uBj0gD8CQWIU0YZC9wNCFyH?=
 =?us-ascii?Q?v3HKG2P2zGG4T4JCmfkrrwEAx12NnFSC2mXYMuPxadxDPTPEjvye/xYL9Te1?=
 =?us-ascii?Q?zmqnW1YhrIb3n76pnDagZrxJU4JKu1uOqxjiRLta22HMheyYF8VoefBrdB+5?=
 =?us-ascii?Q?7sgbYD2XyFtklroWSPXy1YzwuTs42c6AMvikp0pMtyP8W8R/Lnnr17MRGSdt?=
 =?us-ascii?Q?8D3Tm6dPK0Bpd+kapGIwES0aXlklT/aQtLfRiSWreE0EskVHx5XRMxkQScPq?=
 =?us-ascii?Q?wz/UYV+1R1J55RX0544Vbe1W+NWY1RAoQo7KCzad9DTWFNA6IeJMqYKweQb9?=
 =?us-ascii?Q?5A8nBKX8Dnj/6xtxO2g/nz3yuqGDnN0tyvjCIja5WGv2tKf7fTX0eAHXignP?=
 =?us-ascii?Q?YlM3XPHtir777JravDKveQ5rnJB+wMKwpebpmVSU5OYcOQBw4Jz8+1wPmxDn?=
 =?us-ascii?Q?urS/Bb1I/SCvW/vzlcpNKETkZy954d91KYTuvXULjaKKiSdu8uSUsU5WHMeW?=
 =?us-ascii?Q?SpJ4fVsrhAYxDcs+ouSlwRftMNpbWEv0vA0mqjBT6G09uQR98CAzhnVGsyUm?=
 =?us-ascii?Q?oe7lJtjdqnp6B1ChvdwAHawaj9HYSfR+fyWAtoLqh9u4jBvEK1jwtxmxDCqF?=
 =?us-ascii?Q?01nnGyKTO+qnAn/j0eT2t6CKfy61HEPeO3zur7VuG3FjKX2bW/74HvbHtTjM?=
 =?us-ascii?Q?5xT4QbIFKmSu8DBS8KQvG1KEyTixyUKuTIW0vKFgTo4ao4wPiSy8G41bySeB?=
 =?us-ascii?Q?3VTFgiVT3acioH4DwW44NnA7y69Gj8i88GvQQgUZmQIQ9VPE4Eq6H1VxHyS+?=
 =?us-ascii?Q?Om0R1JPHxQJ7W5ML2awFmMKL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ed3a01a0-dc7a-4607-82f3-08d97cd58e49
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 07:58:10.7755
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9+tqIOf383Vgtj+zBx41PEDzHS6XuoaShBTVXLiA9dHbNEx2xddMV+SxXvEOUtqpNgJ3a7hfY9DRuIla4/0Bkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 21.09.2021 09:55, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 08:10:12AM +0200, Jan Beulich wrote:
>> On 06.07.2021 09:37, Jan Beulich wrote:
>>> On 28.06.2021 13:52, Jan Beulich wrote:
>>>> If anything, a check covering a wider range of invalid M2P entries oug=
ht
>>>> to be used (e.g. VALID_M2P()). But since everything is fully under Xen=
's
>>>> control at this stage, simply remove the BUG_ON().
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> I didn't expect this to be controversial, so may I please ask for an ac=
k
>>> (or otherwise)?
>>
>> To be quite honest, I find it very strange that even simple changes like
>> this one sit un-responded to for months. This isn't the only example ...
>=20
> Sorry, I'm trying to catch up with reviews, but after being away so
> long it's hard to figure out what's pending.

Thanks for the ack, and I know you're still catching up. This wasn't
primarily targeted at you (for this reason), but it also wouldn't have
felt right to not include you in the ping.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 07:58:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 07:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191514.341581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSafk-00007Z-2l; Tue, 21 Sep 2021 07:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191514.341581; Tue, 21 Sep 2021 07:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSafj-00007S-W6; Tue, 21 Sep 2021 07:58:35 +0000
Received: by outflank-mailman (input) for mailman id 191514;
 Tue, 21 Sep 2021 07:58:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSafi-0008P3-Hn
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 07:58:34 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b46b7cd7-1ab1-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 07:58:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 14B202209C;
 Tue, 21 Sep 2021 07:58:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C75C513B9C;
 Tue, 21 Sep 2021 07:58:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id lXReL6SQSWEnFQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 07:58:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b46b7cd7-1ab1-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632211109; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=TCRoA5a4c/jtpKOQe0J4E/g6/W+lJjMnAbuFoXReAXI=;
	b=KPniQxBeYVqXHdwPFzuqyH0Io2uibebIpb0Gs1mmoYACl153OpRpCTaitlPwRQsk/fdpkP
	poel0PC2wlPdsIWDcVYIgV7FFMxhatiQoAXxWv+N2+c/oLJsrD8/DBX/m2ltid6v1DuPpc
	4rdcZ+yZriYWCh4gBPFrm8D7LAg7c5M=
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210921070226.32021-1-jgross@suse.com>
 <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
Date: Tue, 21 Sep 2021 09:58:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ITflTTCXoVwbmJMFHEzHN1ZWJTbRSWWbR"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ITflTTCXoVwbmJMFHEzHN1ZWJTbRSWWbR
Content-Type: multipart/mixed; boundary="iFHxcl1T3XjGON5fVwYJ2lTk4ZzYTewBV";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
References: <20210921070226.32021-1-jgross@suse.com>
 <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
In-Reply-To: <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>

--iFHxcl1T3XjGON5fVwYJ2lTk4ZzYTewBV
Content-Type: multipart/mixed;
 boundary="------------C1C4B23D13142A37EBB8A439"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------C1C4B23D13142A37EBB8A439
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 09:53, Jan Beulich wrote:
> On 21.09.2021 09:02, Juergen Gross wrote:
>> --- a/arch/x86/xen/irq.c
>> +++ b/arch/x86/xen/irq.c
>> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>>   {
>>   	struct vcpu_info *vcpu;
>>  =20
>> -	/*
>> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
>> -	 * cleared, so disable preemption to ensure we check for
>> -	 * events on the VCPU we are still running on.
>> -	 */
>> -	preempt_disable();
>> -
>>   	vcpu =3D this_cpu_read(xen_vcpu);
>>   	vcpu->evtchn_upcall_mask =3D 0;
>>  =20
>> -	/* Doesn't matter if we get preempted here, because any
>> -	   pending event will get dealt with anyway. */
>> +	/*
>> +	 * Now preemption could happen, but this is only possible if an even=
t
>> +	 * was handled, so missing an event due to preemption is not
>> +	 * possible at all.
>> +	 * The worst possible case is to be preempted and then check events
>> +	 * pending on the old vcpu, but this is not problematic.
>> +	 */
>=20
> I agree this isn't problematic from a functional perspective, but ...
>=20
>>   	barrier(); /* unmask then check (avoid races) */
>>   	if (unlikely(vcpu->evtchn_upcall_pending))
>>   		xen_force_evtchn_callback();
>=20
> ... is a stray call here cheaper than ...
>=20
>> -
>> -	preempt_enable();
>=20
> ... the preempt_{dis,en}able() pair?

The question is if a stray call in case of preemption (very unlikely)
is cheaper than the preempt_{dis|en}able() pair on each IRQ enabling.

I'm quite sure removing the preempt_*() calls will be a net benefit.


Juergen

--------------C1C4B23D13142A37EBB8A439
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------C1C4B23D13142A37EBB8A439--

--iFHxcl1T3XjGON5fVwYJ2lTk4ZzYTewBV--

--ITflTTCXoVwbmJMFHEzHN1ZWJTbRSWWbR
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJkKQFAwAAAAAACgkQsN6d1ii/Ey/J
iAf/UMWJ8ZejnJFFhH0R0qncPqdrq2o8IgBqB1Wjqiof3lHwTxqpRuJPG8kqIgh0zYM1EysHsKnG
PZ2pgSwO40rpY4Msi4vljJC3A4O3Nnwc3elgLLaPxy0Rg9ZVEdv0Wqx+XqqvxDGCbgrGEG2UvVAj
RJAfdFpeutt0AWYBHHZ9OY8MYNo6prUO3weBJw8hIipXBvBX9cxqDnwCpg9mlY2crB7BLXtjmcSu
vKqKMucn7dteqs7PEEWzhg9YmG3Lih01LbI1eQor9dPmDvPzgqGqZeVM8nZfv4Dg4ve7Mb1DQZxu
U8buqjJk0EFBc3fdNrIFMiJq/ypucfOAwt7A/mo3nA==
=fW4i
-----END PGP SIGNATURE-----

--ITflTTCXoVwbmJMFHEzHN1ZWJTbRSWWbR--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:01:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:01:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191527.341592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaiO-00028t-Q5; Tue, 21 Sep 2021 08:01:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191527.341592; Tue, 21 Sep 2021 08:01:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSaiO-00028m-M5; Tue, 21 Sep 2021 08:01:20 +0000
Received: by outflank-mailman (input) for mailman id 191527;
 Tue, 21 Sep 2021 08:01:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=K4gq=OL=epam.com=prvs=989865dee5=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSaiM-00028g-Vf
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:01:19 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea441e09-4fc8-4b75-b2d2-11a2ac37f763;
 Tue, 21 Sep 2021 08:01:17 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18L7m7F7022464; 
 Tue, 21 Sep 2021 08:01:15 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b7a6q08af-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 21 Sep 2021 08:01:14 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6804.eurprd03.prod.outlook.com (2603:10a6:20b:2de::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 08:01:12 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%3]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 08:01:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea441e09-4fc8-4b75-b2d2-11a2ac37f763
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JdsH4ntrXTgOyGHih38Urc3wL2maM7tWk2OAVh1mX68+Jn80QxejYUY5zEfZW0ALvUBav8jeMwvVpPTRIanbcKKBhdu8KTHA9rCNFnq1C0n5JTBc/pI614ipbdjDwtP9jSRfl6t7+Hswp2essx8Z2TzX5w17rRk6+GuURqM9DQvT66We3XV755XsmE0DtJcVA4tDffhNarDdAuhdqHUfXaJU3Em0S2sG1V01dlHGJB0PdkQdRqDcd1r/eFp+BcWaV5Wo68r1d7ax6mhkcPQLwPOJClolL/LLfq4QG/Hxs+Hs+9xjHYqOpQe50iui7VyhTSSS75xpYWE9f0vbF7b55A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+1KW62GY8Rsqb5LPfbTjUKgcC8I5lfkLxv4NyEB7Brs=;
 b=mTxdulUMU2OYhE9UHRTbhvfkRmvb5EruUA9Hu3/VAgnpOrp2LGbjvVKwWY5d1RNsAtornhhFRRfm093ej3V5oON8xWK73Fmu0rV10IZvFIvjCeLJgkcUAtsyqFI8TD3/j02IsU6qBVkBnes6eHDMIf4mrRK9jnc+oJxOKcqo9TXSkYuvQg9SIsVMET8H/8hxtgomFftj4AacqlenKNNTNpsTe15yaqqkSt2XttcPwceQgjmU1kfr/qK0wwAy0rARr9RWPqoWw/RdeVpYqmlLCdZLxVAqnRY2ins8mb1ejcTFauCfY2GL0C84++eai1wOOCI1tRrIUhHR1dp/+3N9gw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+1KW62GY8Rsqb5LPfbTjUKgcC8I5lfkLxv4NyEB7Brs=;
 b=JGj3hajcJkXcXeVzx10gvAn0aD8m0AlB/L5yR+pQ4W6zIb2RdpINu+Pa62vdx63d3SE9p5rR0juVw9Ce2gI0q13fbmDK4F/6Oe2PdwOrrT1B/6n/6PGN3YToRQOg+XHj8/sgFEbVJssXAZmrHDkhRRhx/NUaBCMrLC3wcWfNwufnPOs0SCpe46s01ueJGwjfIdXcWgXuqH4rq1C8f87CxNp6VQzbfkwuMO+eUjrAPo6SLO19gUMXvzDVSy8SFPwNggnueTQSFxMRKLoBfNWRkAEsdQNJFtyG/1LWHmnMp581xNTiB+Fqea4ivAnwBjFOZXDF+R4ENdt4RfKG9YElhA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuuJC4AgAAB+IA=
Date: Tue, 21 Sep 2021 08:01:11 +0000
Message-ID: <94dcf710-276c-a3cc-48de-1c5caaf495f5@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <809c7a0c-cc8b-ce6f-bab4-97029e538053@suse.com>
In-Reply-To: <809c7a0c-cc8b-ce6f-bab4-97029e538053@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4ffb9bd7-0d5b-4642-3fed-08d97cd5fa68
x-ms-traffictypediagnostic: AM9PR03MB6804:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM9PR03MB6804BA1CBEB637BBED3EC8DCE7A19@AM9PR03MB6804.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 IlEqLOcIdayGopolSkpP2tnPXacjlrtevh89Cxh4XUXg4wrM7PaMtR0CWFiFLo66uboAuvo6mpJpUZSAq8GTXUlai8FZ2SJBmVCIrLq8x8nh/9A8E+c+aERKt+2fR2W0VYMMO6QGHGZaX+tpe6rje7bLwSf65+ckaMd0Rybr5O9TLcYTkmr51A6E9xuFroNtVo/4tw+JnVEZ5xGDwDT63hlTeToyX0xE4dNAlJJY4hmdhDGM2pIq5HrftHvJvUvR7uSaPA8PDEmGqaxd9uOF6RNqpDX2EdRJz2M3YrnxeymHBaCEjsr1ZFjwdgefhBqu/hNr1MAevBHn27vHEbUN19MvXdko0U07KY8P6U664GABxrVxzq4gPXpm+xpDKDbCHRUozQheSjsm89vEoJLYk+wY0Qisu9SKdve/YtCcj7x+FN1K3FZUptgYZj3QEvLYvvWP7I5qjQ9F+o/lm1VBb19ki40TgkmKDomQr8Rt6QHajCl7lhmag8JmpLYSbpVCpabT1o3suFnUyrD4YxvnennkRaDnRiwSHl7p5A2kv9ReAPFIy3hOeZBIpfffpN9eAWTfCcpNd5VkYJ1kHtmWAS7x5onifQ8QIoMDahXRW1mcm9DS23MNEXgFMYDUaSKsxisUMFod7suE40bOuBWUdOn8NDKrq+ezuQ/yK7/Rsg5PYx9NJbhJ8ROvG36O749D8o9/3O8ISZKr+NKvkEn+r1EiSQ1Ip5PzxQzoiQw0CBqI6LQglbU4hddJW46nARHE
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(31686004)(66946007)(8936002)(186003)(6486002)(5660300002)(31696002)(54906003)(508600001)(2616005)(36756003)(38070700005)(66446008)(107886003)(66556008)(76116006)(66476007)(86362001)(71200400001)(4326008)(38100700002)(6506007)(53546011)(316002)(55236004)(26005)(8676002)(110136005)(122000001)(2906002)(6512007)(83380400001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aW5KMWtkbmRGRE11M1JGLzRJaXBCU1NtTTI4TkRNU2VOODBPVHliRDREcmpL?=
 =?utf-8?B?SDNEbjFFdWJmSlpFMG5tZ0VodGVQNkdKbnQxYUg0YmxrZDZKNEZaUzRNenAz?=
 =?utf-8?B?VmZiY3NYOUN0bXEvMkNlbUVjbTRRZ010ekMvUXhUMjViSFBCMDUvbHA2WkRy?=
 =?utf-8?B?YXhCd0hrNEQyaE55TEczZ0d0RlRPT1UwaHIrWUFBUzFwOXBZamw0MVZRdlpF?=
 =?utf-8?B?VWVkMXBjdE0zMjRHWm5QZUhidFVuc1JMK3Q3VTR4SmIwZGNpMGg2MHRpcE5T?=
 =?utf-8?B?YmJ3UlF1Uk5USnF0Ykx3NU5yVGdGV1RBa3VpTm1kTHJhWjZidEZLb2g3V2Fv?=
 =?utf-8?B?QUtYQ1BnMzNqMlNKN0dCWVB4a2lQYTZNc0orN09YZFk1ekQrbTk4K2ZoYVBH?=
 =?utf-8?B?Tlo4QXZGOW5DTm9vRE9zd1hrTHJ4ZFRnZVJGRmVLSEhEZ0hIam1BMlE0Wkpk?=
 =?utf-8?B?c0wyK1VvMVhkaDBBKy8rTzVSV3BZNy9iRjBCRjBHdkJMdEJNYVVwbTVPdzl3?=
 =?utf-8?B?b3Mrbmc2UHU3YnZ5SDlLMzhqRVU1K1ZqYU5Sdk5zMXd4dEFVdGFkRm5SekFu?=
 =?utf-8?B?Y2Nia0VmN3Rpc29wMmo0OEdsQTRRUDBzKzl2azBMaVpzVHBVa2ZBUjZYb01Q?=
 =?utf-8?B?OXJWMU02aCtvSXd1RGRqQWVwVEY4eS8yUHoyTXM5R20zckwwM2hKL0JON3B3?=
 =?utf-8?B?K1l1N3VsUmVQTTkwcmxPZ0xROTFvZlY4RVlqc3dqQStUSzBxL3Z1WnNqbWFY?=
 =?utf-8?B?UHE5TnhYbHhsdklUOXYvRU5hTFBWTHQrZ3o0Zm5ybnBINXA4cENpS2p0aXVS?=
 =?utf-8?B?ejBsSXZkQUgzMHlvRGhtN2RUQllVd294S3hQdGNwMnhmRm1rWFFSeW4zRzl6?=
 =?utf-8?B?dTJVV2tUaVd2M1lQMnZPblh5Mmx0VE96ajB0ZmhHak15NkgxcU00ZjZzQjhh?=
 =?utf-8?B?dWVqczVETHpPNnlpb2JjK0VKOW8zdTdZWWxVS1BtdU9jRzJCMXo3bjNGdzZX?=
 =?utf-8?B?enZLanp6aWZSUFhNWmM3d3A1Z0poenpUQTkveXRLZGtneVJFc1AreTRMTnFx?=
 =?utf-8?B?N0svSVprMVNyUnFYUHRhTlFyS2h4MjVFMGp5aXY4dmFUdlRlOUZmeUhKVHhU?=
 =?utf-8?B?V1l0SmdFNTlsYlFGcXE4WnFJc3N2aHRhd0ZoRkVPUWxGVGRoWm5DRElwQVA3?=
 =?utf-8?B?MElDelJFUE93V3FrVG9HOC9YWElaWld6cTRLSlFHeUROdE95Nmlqa3VMdUxJ?=
 =?utf-8?B?a3JpRFdGTXVEcDhUbG9rUXNuVjUvOWhpVEVzbEl1ZjNYUVhQMUNGdVFNSVFK?=
 =?utf-8?B?WitIU3g2WXlxNWRUUGZ5NHd6ZWtuSmZKdnBFZDN4MWhLcHRVMFgza1FjclJ1?=
 =?utf-8?B?L1pEUWdkZCtUVXB2dXBRMVVYOE9Ha3J5Ty9IUnZabGNIeUpGeGhnOTFzT1N4?=
 =?utf-8?B?UTBjcmdPUmgzdGVFdHFDdnlHT0FNZHBIQkZMV3lPZEw5eEF1RTJSVkoxT2Js?=
 =?utf-8?B?L2o2WTl5bHlvZmNOM3k3Wm9YU1Q2TkZHcVMvZ2lOODh2YWpNMFRoUFM0RUFP?=
 =?utf-8?B?aFlQOWFwRWpVVktzRzZHSHhxR29pbW93WWs5SmRnamlYSnNGd2c3eWU1Nkpw?=
 =?utf-8?B?ZkVIcE1BVnFzWnBBNjYxeFFvUzhDT3BWcmpabFZLcUVDZTBFeWZCT1k4alFJ?=
 =?utf-8?B?T0NMeXk2RWhpbVV0Qm5DQ0hDWE1pRFQyeERXc0dvQlN5enpGQjBtODJWK213?=
 =?utf-8?B?V255MUg3a2JYYzBOdHV1WXlUWXlKREE4eUxlc1UrZVlibVpwT2oxZXZyd01k?=
 =?utf-8?B?eEFCS293RG8zYld6UEF1UT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <9A7EFAA29137E04B938BD4B4FD35004E@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4ffb9bd7-0d5b-4642-3fed-08d97cd5fa68
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2021 08:01:11.9106
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: lZMEZVbOi1QxZUYozaR2iTQC25gIJTs7Oi88BfhPOdZ/vvZQrGp1KDYFLnlXKHxyH2KYo3yq7JTcMwG5Bu8AMIGEBgP7Tjp4vZlr3ENY2VurtFhgsgLjoXzifUGIdP1K
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6804
X-Proofpoint-GUID: G-CGN8am5RrJXAGXQfIEeLbBpdha0Hj0
X-Proofpoint-ORIG-GUID: G-CGN8am5RrJXAGXQfIEeLbBpdha0Hj0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-21_01,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 malwarescore=0 mlxscore=0 spamscore=0 impostorscore=0 mlxlogscore=833
 bulkscore=0 suspectscore=0 clxscore=1015 adultscore=0 phishscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109210052

DQpPbiAyMS4wOS4yMSAxMDo1NCwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMTcuMDkuMjEg
MTU6MDEsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+
IFhlbi1wY2liYWNrIGRyaXZlciB3YXMgZGVzaWduZWQgdG8gYmUgYnVpbHQgZm9yIHg4NiBvbmx5
LiBCdXQgaXQNCj4+IGNhbiBhbHNvIGJlIHVzZWQgYnkgb3RoZXIgYXJjaGl0ZWN0dXJlcywgZS5n
LiBBcm0uDQo+PiBSZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBpbiBhIHdheSB0aGF0IGl0IGNhbiBi
ZSBidWlsdCBmb3Igb3RoZXINCj4+IHBsYXRmb3JtcyBhcyB3ZWxsLg0KPj4NCj4+IFNpZ25lZC1v
ZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0Bl
cGFtLmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEFuYXN0YXNpaWEgTHVraWFuZW5rbyA8YW5hc3Rh
c2lpYV9sdWtpYW5lbmtvQGVwYW0uY29tPg0KPj4NCj4+IC0tLQ0KPj4gVGVzdGVkIG9uIEFybSBh
bmQgeDg2Lg0KPj4gLS0tDQo+PiDCoCBhcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmjCoMKg
wqDCoCB8IDI0IC0tLS0tLS0tLS0NCj4+IMKgIGFyY2gveDg2L3BjaS94ZW4uY8KgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgNzQgKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0N
Cj4+IMKgIGRyaXZlcnMveGVuL0tjb25maWfCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
fMKgIDIgKy0NCj4+IMKgIGRyaXZlcnMveGVuL2V2ZW50cy9ldmVudHNfYmFzZS5jwqDCoCB8wqAg
MSArDQo+PiDCoCBkcml2ZXJzL3hlbi9wY2kuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgfCA3NSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+IMKgIGRyaXZlcnMv
eGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMgfMKgIDMgKy0NCj4+IMKgIGRyaXZlcnMveGVuL3hl
bi1wY2liYWNrL3hlbmJ1cy5jwqDCoCB8wqAgMiArLQ0KPj4gwqAgaW5jbHVkZS94ZW4vcGNpLmjC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgMzQgKysrKysrKysrKysrKysNCj4+
IMKgIDggZmlsZXMgY2hhbmdlZCwgMTE1IGluc2VydGlvbnMoKyksIDEwMCBkZWxldGlvbnMoLSkN
Cj4+IMKgIGNyZWF0ZSBtb2RlIDEwMDY0NCBpbmNsdWRlL3hlbi9wY2kuaA0KPj4NCj4+IGRpZmYg
LS1naXQgYS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmggYi9hcmNoL3g4Ni9pbmNsdWRl
L2FzbS94ZW4vcGNpLmgNCj4+IGluZGV4IDM1MDZkOGM1OThjMS4uOWZmN2I0OWJjYTA4IDEwMDY0
NA0KPj4gLS0tIGEvYXJjaC94ODYvaW5jbHVkZS9hc20veGVuL3BjaS5oDQo+PiArKysgYi9hcmNo
L3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgNCj4+IEBAIC0xNCwzMCArMTQsNiBAQCBzdGF0aWMg
aW5saW5lIGludCBwY2lfeGVuX2h2bV9pbml0KHZvaWQpDQo+PiDCoMKgwqDCoMKgIHJldHVybiAt
MTsNCj4+IMKgIH0NCj4+IMKgICNlbmRpZg0KPj4gLSNpZiBkZWZpbmVkKENPTkZJR19YRU5fRE9N
MCkNCj4+IC1pbnQgX19pbml0IHBjaV94ZW5faW5pdGlhbF9kb21haW4odm9pZCk7DQo+DQo+IFdo
eSBhcmUgeW91IHJlbW92aW5nIHRoaXMgcHJvdG90eXBlPyBJdCBpcyBYODYgc3BlY2lmaWMuDQpJ
bmRlZWQgaXQgaXMuIFdpbGwgbm90IHJlbW92ZSBpdCBmb3JtIHg4NiBjb2RlDQo+DQo+PiAtaW50
IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAt
aW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYs
IHVpbnQxNl90IGRvbWFpbik7DQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5f
b3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAtI2Vsc2UNCj4+IC1zdGF0aWMgaW5saW5l
IGludCBfX2luaXQgcGNpX3hlbl9pbml0aWFsX2RvbWFpbih2b2lkKQ0KPj4gLXsNCj4+IC3CoMKg
wqAgcmV0dXJuIC0xOw0KPj4gLX0NCj4+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZp
Y2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLcKgwqDCoCBy
ZXR1cm4gLTE7DQo+PiAtfQ0KPj4gLXN0YXRpYyBpbmxpbmUgaW50IHhlbl9yZWdpc3Rlcl9kZXZp
Y2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiAtwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1aW50MTZfdCBkb21haW4pDQo+
PiAtew0KPj4gLcKgwqDCoCByZXR1cm4gLTE7DQo+PiAtfQ0KPj4gLXN0YXRpYyBpbmxpbmUgaW50
IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikN
Cj4+IC17DQo+PiAtwqDCoMKgIHJldHVybiAtMTsNCj4+IC19DQo+PiAtI2VuZGlmDQo+PiDCoCDC
oCAjaWYgZGVmaW5lZChDT05GSUdfUENJX01TSSkNCj4+IMKgICNpZiBkZWZpbmVkKENPTkZJR19Q
Q0lfWEVOKQ0KPj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L3BjaS94ZW4uYyBiL2FyY2gveDg2L3Bj
aS94ZW4uYw0KPj4gaW5kZXggM2Q0MWEwOWMyYzE0Li40YTQ1YjBiZjlhZTQgMTAwNjQ0DQo+PiAt
LS0gYS9hcmNoL3g4Ni9wY2kveGVuLmMNCj4+ICsrKyBiL2FyY2gveDg2L3BjaS94ZW4uYw0KPj4g
QEAgLTIzLDYgKzIzLDcgQEANCj4+IMKgIMKgICNpbmNsdWRlIDx4ZW4vZmVhdHVyZXMuaD4NCj4+
IMKgICNpbmNsdWRlIDx4ZW4vZXZlbnRzLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+
IMKgICNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4gwqAgI2luY2x1ZGUgPGFzbS94ZW4vY3B1
aWQuaD4NCj4+IMKgICNpbmNsdWRlIDxhc20vYXBpYy5oPg0KPj4gQEAgLTU4Myw3NyArNTg0LDQg
QEAgaW50IF9faW5pdCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpDQo+PiDCoMKgwqDCoMKg
IH0NCj4+IMKgwqDCoMKgwqAgcmV0dXJuIDA7DQo+PiDCoCB9DQo+PiAtDQo+PiAtc3RydWN0IHhl
bl9kZXZpY2VfZG9tYWluX293bmVyIHsNCj4+IC3CoMKgwqAgZG9taWRfdCBkb21haW47DQo+PiAt
wqDCoMKgIHN0cnVjdCBwY2lfZGV2ICpkZXY7DQo+PiAtwqDCoMKgIHN0cnVjdCBsaXN0X2hlYWQg
bGlzdDsNCj4+IC19Ow0KPj4gLQ0KPj4gLXN0YXRpYyBERUZJTkVfU1BJTkxPQ0soZGV2X2RvbWFp
bl9saXN0X3NwaW5sb2NrKTsNCj4+IC1zdGF0aWMgc3RydWN0IGxpc3RfaGVhZCBkZXZfZG9tYWlu
X2xpc3QgPSBMSVNUX0hFQURfSU5JVChkZXZfZG9tYWluX2xpc3QpOw0KPj4gLQ0KPj4gLXN0YXRp
YyBzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgKmZpbmRfZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpkZXYpDQo+PiAtew0KPj4gLcKgwqDCoCBzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3du
ZXIgKm93bmVyOw0KPj4gLQ0KPj4gLcKgwqDCoCBsaXN0X2Zvcl9lYWNoX2VudHJ5KG93bmVyLCAm
ZGV2X2RvbWFpbl9saXN0LCBsaXN0KSB7DQo+PiAtwqDCoMKgwqDCoMKgwqAgaWYgKG93bmVyLT5k
ZXYgPT0gZGV2KQ0KPj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIG93bmVyOw0KPj4g
LcKgwqDCoCB9DQo+PiAtwqDCoMKgIHJldHVybiBOVUxMOw0KPj4gLX0NCj4+IC0NCj4+IC1pbnQg
eGVuX2ZpbmRfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gLXsN
Cj4+IC3CoMKgwqAgc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4+IC3C
oMKgwqAgaW50IGRvbWFpbiA9IC1FTk9ERVY7DQo+PiAtDQo+PiAtwqDCoMKgIHNwaW5fbG9jaygm
ZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC3CoMKgwqAgb3duZXIgPSBmaW5kX2Rldmlj
ZShkZXYpOw0KPj4gLcKgwqDCoCBpZiAob3duZXIpDQo+PiAtwqDCoMKgwqDCoMKgwqAgZG9tYWlu
ID0gb3duZXItPmRvbWFpbjsNCj4+IC3CoMKgwqAgc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlz
dF9zcGlubG9jayk7DQo+PiAtwqDCoMKgIHJldHVybiBkb21haW47DQo+PiAtfQ0KPj4gLUVYUE9S
VF9TWU1CT0xfR1BMKHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIpOw0KPj4gLQ0KPj4gLWlu
dCB4ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2LCB1
aW50MTZfdCBkb21haW4pDQo+PiAtew0KPj4gLcKgwqDCoCBzdHJ1Y3QgeGVuX2RldmljZV9kb21h
aW5fb3duZXIgKm93bmVyOw0KPj4gLQ0KPj4gLcKgwqDCoCBvd25lciA9IGt6YWxsb2Moc2l6ZW9m
KHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciksIEdGUF9LRVJORUwpOw0KPj4gLcKgwqDC
oCBpZiAoIW93bmVyKQ0KPj4gLcKgwqDCoMKgwqDCoMKgIHJldHVybiAtRU5PREVWOw0KPj4gLQ0K
Pj4gLcKgwqDCoCBzcGluX2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtwqDC
oMKgIGlmIChmaW5kX2RldmljZShkZXYpKSB7DQo+PiAtwqDCoMKgwqDCoMKgwqAgc3Bpbl91bmxv
Y2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtwqDCoMKgwqDCoMKgwqAga2ZyZWUo
b3duZXIpOw0KPj4gLcKgwqDCoMKgwqDCoMKgIHJldHVybiAtRUVYSVNUOw0KPj4gLcKgwqDCoCB9
DQo+PiAtwqDCoMKgIG93bmVyLT5kb21haW4gPSBkb21haW47DQo+PiAtwqDCoMKgIG93bmVyLT5k
ZXYgPSBkZXY7DQo+PiAtwqDCoMKgIGxpc3RfYWRkX3RhaWwoJm93bmVyLT5saXN0LCAmZGV2X2Rv
bWFpbl9saXN0KTsNCj4+IC3CoMKgwqAgc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlu
bG9jayk7DQo+PiAtwqDCoMKgIHJldHVybiAwOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQ
TCh4ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcik7DQo+PiAtDQo+PiAtaW50IHhlbl91
bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17
DQo+PiAtwqDCoMKgIHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiAt
DQo+PiAtwqDCoMKgIHNwaW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC3C
oMKgwqAgb3duZXIgPSBmaW5kX2RldmljZShkZXYpOw0KPj4gLcKgwqDCoCBpZiAoIW93bmVyKSB7
DQo+PiAtwqDCoMKgwqDCoMKgwqAgc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9j
ayk7DQo+PiAtwqDCoMKgwqDCoMKgwqAgcmV0dXJuIC1FTk9ERVY7DQo+PiAtwqDCoMKgIH0NCj4+
IC3CoMKgwqAgbGlzdF9kZWwoJm93bmVyLT5saXN0KTsNCj4+IC3CoMKgwqAgc3Bpbl91bmxvY2so
JmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtwqDCoMKgIGtmcmVlKG93bmVyKTsNCj4+
IC3CoMKgwqAgcmV0dXJuIDA7DQo+PiAtfQ0KPj4gLUVYUE9SVF9TWU1CT0xfR1BMKHhlbl91bnJl
Z2lzdGVyX2RldmljZV9kb21haW5fb3duZXIpOw0KPj4gwqAgI2VuZGlmDQo+PiBkaWZmIC0tZ2l0
IGEvZHJpdmVycy94ZW4vS2NvbmZpZyBiL2RyaXZlcnMveGVuL0tjb25maWcNCj4+IGluZGV4IGEz
N2ViNTJmYjQwMS4uMDU3ZGRmNjFlZjYxIDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy94ZW4vS2Nv
bmZpZw0KPj4gKysrIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4gQEAgLTE4Miw3ICsxODIsNyBA
QCBjb25maWcgU1dJT1RMQl9YRU4NCj4+IMKgIMKgIGNvbmZpZyBYRU5fUENJREVWX0JBQ0tFTkQN
Cj4+IMKgwqDCoMKgwqAgdHJpc3RhdGUgIlhlbiBQQ0ktZGV2aWNlIGJhY2tlbmQgZHJpdmVyIg0K
Pj4gLcKgwqDCoCBkZXBlbmRzIG9uIFBDSSAmJiBYODYgJiYgWEVODQo+PiArwqDCoMKgIGRlcGVu
ZHMgb24gUENJICYmIFhFTg0KPj4gwqDCoMKgwqDCoCBkZXBlbmRzIG9uIFhFTl9CQUNLRU5EDQo+
PiDCoMKgwqDCoMKgIGRlZmF1bHQgbQ0KPj4gwqDCoMKgwqDCoCBoZWxwDQo+PiBkaWZmIC0tZ2l0
IGEvZHJpdmVycy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMgYi9kcml2ZXJzL3hlbi9ldmVudHMv
ZXZlbnRzX2Jhc2UuYw0KPj4gaW5kZXggYTc4NzA0YWUzNjE4Li4zNTQ5M2ZmMGQxNDYgMTAwNjQ0
DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi9ldmVudHMvZXZlbnRzX2Jhc2UuYw0KPj4gKysrIGIvZHJp
dmVycy94ZW4vZXZlbnRzL2V2ZW50c19iYXNlLmMNCj4+IEBAIC02NSw2ICs2NSw3IEBADQo+PiDC
oCAjaW5jbHVkZSA8eGVuL2ludGVyZmFjZS92Y3B1Lmg+DQo+PiDCoCAjaW5jbHVkZSA8eGVuL3hl
bmJ1cy5oPg0KPj4gwqAgI2luY2x1ZGUgPGFzbS9od19pcnEuaD4NCj4+ICsjaW5jbHVkZSA8eGVu
L3BjaS5oPg0KPg0KPiBUaGlzIHdvdWxkbid0IGJlIG5lZWRlZCBpZiB5b3UnZCBsZXQgdGhlIHBj
aV94ZW5faW5pdGlhbF9kb21haW4oKQ0KPiBwcm90b3R5cGUgd2hlcmUgaXQgaGFzIGJlZW4uDQpT
dXJlLCB3aWxsIGxlYXZlIGl0IHdoZXJlIGl0IHdhcw0KPg0KPg0KPiBKdWVyZ2Vu


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:03:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:03:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191536.341603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSak0-0002o6-9D; Tue, 21 Sep 2021 08:03:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191536.341603; Tue, 21 Sep 2021 08:03:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSak0-0002nz-62; Tue, 21 Sep 2021 08:03:00 +0000
Received: by outflank-mailman (input) for mailman id 191536;
 Tue, 21 Sep 2021 08:02:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSajy-0002np-J4
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:02:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 54918688-1ab2-11ec-b89f-12813bfff9fa;
 Tue, 21 Sep 2021 08:02:57 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-4n00tCGVMKi_9KWHnshq0w-1; Tue, 21 Sep 2021 10:02:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Tue, 21 Sep
 2021 08:02:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 08:02:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR0301CA0031.eurprd03.prod.outlook.com (2603:10a6:206:14::44) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Tue, 21 Sep 2021 08:02:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54918688-1ab2-11ec-b89f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632211376;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4S774r1OPPy/t2Aaam3xGobazJwl30XB1bp7FcSWO9k=;
	b=ZA5Artvh2Weyi3KyMITXg0c2iesEUlspJCujdkd8RrIBzTAE38gbP5lmrJgSOa1ESMfMSK
	EW7w2Kpu3woFpvxDYEJx2KtIMEkvKNEYA0ijlNglSqZVXf1Ki2VROAqLkYKzv3S3VvU/kz
	bKFkxntgejG+RPeHZLONZI3yuk0XwVI=
X-MC-Unique: 4n00tCGVMKi_9KWHnshq0w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kQOXRyWyi/xLvof80RMXlcD1/8mKNKlUvevDU6tm4A04NZR8epolyqb617Ia1V9yKZGK3TiFQhjPQf4943MSKl/h6NPBbcspreKtjVNcZrDUBk7ZubU6BcIVHO8KZeB+w8T4yD5Mx0ALyhGBkxGp8hvovfeQoCgpvxhoCLDYCsCCdWYrmE5N+Zz4kWgi3nh9faaIh/qe19tQOn7SmYSGe+Hs/YNQH+TFkKpOFl0hPhld5yUjyzXKDRkq/SRaP61r/Ab/HoEk4VY+Iueflwgt8T8OcjbNu4T0x61j1JhcTJ85QUgLKyUx1TGQHM3UxNcjmDhTUSOqeOQTgu6S8KXD+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4S774r1OPPy/t2Aaam3xGobazJwl30XB1bp7FcSWO9k=;
 b=LggVY+WtFfa5xOBQ/H0vhvo0BiSgO+gmeRI9TolTFQ+iGu78GfsUFu/qWdBEj9t6klXfkGySt1ZtjtDawcSKGRiV1A2lu+8n+0ZZz62sZ6tYNGt9l2J0PYh3Uug/r5lXmXAQSvFbVvSBlqI+GDGeB0oE+V2jkPyviLQuCfwv6AsSHOpQLRFlMvLKhb7nMpJn43ind8rsFLxITuznyrGooyQ7hkPpbZwYn5fyyOtfGe58fPyKMS4OlW76pUYAH3d8EHLBO41stP5mA0rF8A8dq1YqLntO+d62lkZnIY2+sw/dnI7tSq06v5U3xHIdzTXGkI9YX8zKVDda9IbLKJHk0A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Ping: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
Message-ID: <6aa5ed94-b00e-24ef-aae6-be2cd958c406@suse.com>
Date: Tue, 21 Sep 2021 10:02:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR0301CA0031.eurprd03.prod.outlook.com
 (2603:10a6:206:14::44) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 25e7e5de-496d-435c-b6fb-08d97cd636cc
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686134B2FB9C5ADF6D1BDF5DB3A19@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mAb9f882aEf3BfZCBXkf9X0soN4Lpz69up3dFt9CHvSYzGx4PNRJCQIblkayZyS1zWN5TMOjLksLILd0idVUsDZyjvAqkPGdLH7ZmZUNXYvAyUKLXFc7oTTjb7NRA634bWcU+f6OpHXKq4yw5onm31aG/5AYz23ROC/7r5fHi9qT74HcA/jQaaQeccGyRjP6BywBdzWtFYaNxP4GUcBEoQzUn04G6NsHrL4Z9LK/6oXiT/j/kb2Y8SPSmMFBkNruxQaJmd3ni4nA8Gbmw8Llwpv5tl8J2rN6EerhvPyXy+VZva9tbtgoi97WLUH5QmoJWftdqTw1nI8/1c1aQpe7//8s8zXkFeuEGdWDq8p+4hVmWE+ZFmMiNh8aQ9fUyEcr3FtBB/21mjp8zpqFY7KpkvwcGdRy3N6Sg2VgO7iERaOusUIlOTof9LhcMXfX9j+0Da47Tfzp31yV25uCabnQzh+gJUwZJVZiWSurXpLp2ZFHLdnqricuLnhZsfoz7YWP9+rmF17YPz1/9+THAHHuve/VvZM3JdOCeUYf5Ts/uL62xqs8Wj158CwGxMqh2+6YUgncDo0Oe+Zo1HwHN+dTfW0T2RlAw8GC738ac9j9KIm1idSvCVXqE9+jywZ3K00nq9x8gha5itXWoaEUELYg/wbBFM3IqVc4ZGGImDwOkgap9R5zH3uUtNjKw+rHBaEnT9t/IsrnAzRbpCNNsia50VYL+9sMKVeoi8EKDDIVsVQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(26005)(8936002)(6486002)(5660300002)(956004)(2906002)(31686004)(66946007)(36756003)(186003)(54906003)(66476007)(83380400001)(4326008)(316002)(66556008)(110136005)(2616005)(31696002)(8676002)(86362001)(38100700002)(508600001)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UUtZK2ZFMkZUV1l2OU5RMTgwc2RKRCtHdlpSanE3U3Q3ckdlZXlzUVFLMlcv?=
 =?utf-8?B?RHRYa1A2eFREVDFPcWYvUEtWSGE5Y3ZTOFgxdW9hWllIMFJZVmNOQlhvMEY0?=
 =?utf-8?B?Wk9BMTc1bXVlRVdSSVRZNThuRkZVazl6eHlidUhSL1lJVi9jOFhKNzlpL0RP?=
 =?utf-8?B?K1FCZjlGQkZ1UnVTU0V2L1d1M2M0N083S2JMQVUrUjAwUUEwSVlmU0FNOHVy?=
 =?utf-8?B?UEc2blg2T1ZFVFAvYXhqUHE1WGo4THVLQlhPYTZTSHZla3hCQ2EyNzlaN0lr?=
 =?utf-8?B?ejVpRG1EVGdkVzBtYWdjeC9FejJUNE9xTGcxQ2w1RDJqT1dMV1BmaXFJamhU?=
 =?utf-8?B?ekNFNEFURGs1OFlFRUkyV0EyUXNSOCs3ODVVL051TngrNDBjQ2h0NGgrbEla?=
 =?utf-8?B?T1pjeU4vanJqaVFpaEhnVEp6OTVSQTFRZWlleVBpRW5TcXhtSFBpcVowUlQw?=
 =?utf-8?B?c2FTbmFrOGE1THUweDRKbzJ4cmNtUXBrVGRCdW9JUUZDTXA5L1A3NUZoOEta?=
 =?utf-8?B?ZTVOaTdhUHVoMUx5QVAzMzNmR3R2dlRWSm1TK21EcG53b3BqcDFxZEovcUJL?=
 =?utf-8?B?ckJINnlsN1ZHbXhPK1pQUUNxRXUxS0RlcVFKRzJDRXFFWHdNb0NUQnZyM2Zv?=
 =?utf-8?B?ZzdvcTN0N0hnejVFb2wzSmxHUjlOcnBtbVBKV1ZFT3Q3NURzMG1uL2E1NDFN?=
 =?utf-8?B?NnVkckh1UEQ3ZDJyVUU3SVpEejRLVDJYZ3dSZy9TeUQ4eHBIdkw3a1g5Z09C?=
 =?utf-8?B?cHowM2wvRUpMVkt3bHRkSUFmSnBVaEtyZFowMDdNcmZTZ29JSEtLWi9QdWw5?=
 =?utf-8?B?TUdISnJDR2V0dkN4ZnIrKzJXMnJCeVQ1T0JzTWdFTHEzSjNuNk04OW9VSjRU?=
 =?utf-8?B?cWVzZ1FMVjhZT2JFVkljUkl2M3F6NTZyaXptaGZUY2RrN1V1a0N2ajVYWG5i?=
 =?utf-8?B?YXJUMmRZOHlmWnZ0VFByUWhHUmVqK3VLd3dQeitkbXNFQ0htc201b2xGd1hN?=
 =?utf-8?B?a2lHTng1bXVYaDN6dytBc2VVOFpCT0ZOYXVtTURyOWd6UUFPQVF1a01ad2p5?=
 =?utf-8?B?L1BSN1dGb3cvM3VZQnh4c0VnRnBCcEVFczI3T2czYnZ0QXRUVFhEU1lMRFQ1?=
 =?utf-8?B?MElRTWdIbklyY1Y0YzRKNkV3QTFpT082R0FjelNCMXJyZVFxL2hKcDhHZUhL?=
 =?utf-8?B?YmRidlBsZHl6cVgybzZMVDl3VmhFQUtqLzhzZ2RvQzVwb2JkQjljdzEvcU5v?=
 =?utf-8?B?M2VOOXRvREQ0eU9GaGIrK0M4WWtEb3JIc0dzc29FMmlya3VJWi9uZ0g4NTZH?=
 =?utf-8?B?dTZabHphUm5UZXFWUFE4QStkTzVLYWx6Y1JGK2hSRFg2ZitPYmZRWEdXZStn?=
 =?utf-8?B?eTBZK1dXTVJoMkdnNDlESkNmY05vZWhRQXJISWlBMnArRWdSMVNMZmVUTzZV?=
 =?utf-8?B?UUxYd3RHZGFiVUNpL1cxWHdqelJDQWIvcGNWR2xyeWFmaDFVMTJjaVZObS9U?=
 =?utf-8?B?UUJIYmNyVUdUWDZWaERKakh4RThBbTZseXNTRHgvMlBIMEN5WTZiRnZycmY5?=
 =?utf-8?B?aDBadG5vS2lTNnJ4eUM2OFVPd0NuclpXNzI5UXFuSGV2d0dPVXl4WE4zSElk?=
 =?utf-8?B?aEVTNU5waWp2SU5qa2oyaGZnS1NlNlljR2RUQTVWSmF0dWlIUVlUZkVVMXYy?=
 =?utf-8?B?ZjFOdVlDM2V4K0JRRzlXS1hLdm53Ny96bmpJRXJOL0pqR1hsWmVuM1hiQnR1?=
 =?utf-8?Q?SxB645vB4WiFvkfQ4DFB9M0a63LmEa/j2WNGofX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 25e7e5de-496d-435c-b6fb-08d97cd636cc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 08:02:53.4796
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6+sZiyFJ8WD716nKmKC7dgBHkD9z3Xt0v70LIvLvG/n9jR4UxgdJm4FoSLN6qYKBJ6dAEaJ9AxDwW+T01wi+3g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 01.09.2021 18:06, Jan Beulich wrote:
> The function may fail; it is not correct to indicate "success" in this
> case up the call stack. Mark the function must-check to prove all
> cases have been caught (and no new ones will get introduced).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> In the grant-transfer case it is not really clear to me whether we can
> stick to setting GTF_transfer_completed in the error case. Since a guest
> may spin-wait for the flag to become set, simply not setting the flag is
> not an option either. I was wondering whether we may want to slightly
> alter (extend) the ABI and allow for a GTF_transfer_committed ->
> GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
> at the same time as setting GTF_transfer_completed).

While I did get a reply from Andrew on this additional remark, the code
change itself has remained un-acked and un-responded to, despite imo
being pretty straightforward. May I please as for an ack or otherwise
an indication of what needs to be changed?

Jan

> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -2394,7 +2394,7 @@ gnttab_transfer(
>          {
>              grant_entry_v1_t *sha = &shared_entry_v1(e->grant_table, gop.ref);
>  
> -            guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
> +            rc = guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
>              if ( !paging_mode_translate(e) )
>                  sha->frame = mfn_x(mfn);
>          }
> @@ -2402,7 +2402,7 @@ gnttab_transfer(
>          {
>              grant_entry_v2_t *sha = &shared_entry_v2(e->grant_table, gop.ref);
>  
> -            guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
> +            rc = guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
>              if ( !paging_mode_translate(e) )
>                  sha->full_page.frame = mfn_x(mfn);
>          }
> @@ -2415,7 +2415,7 @@ gnttab_transfer(
>  
>          rcu_unlock_domain(e);
>  
> -        gop.status = GNTST_okay;
> +        gop.status = rc ? GNTST_general_error : GNTST_okay;
>  
>      copyback:
>          if ( unlikely(__copy_field_to_guest(uop, &gop, status)) )
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -268,7 +268,8 @@ static void populate_physmap(struct memo
>                  mfn = page_to_mfn(page);
>              }
>  
> -            guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order);
> +            if ( guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order) )
> +                goto out;
>  
>              if ( !paging_mode_translate(d) &&
>                   /* Inform the domain of the new page's machine address. */
> @@ -765,8 +766,8 @@ static long memory_exchange(XEN_GUEST_HA
>              }
>  
>              mfn = page_to_mfn(page);
> -            guest_physmap_add_page(d, _gfn(gpfn), mfn,
> -                                   exch.out.extent_order);
> +            rc = guest_physmap_add_page(d, _gfn(gpfn), mfn,
> +                                        exch.out.extent_order) ?: rc;
>  
>              if ( !paging_mode_translate(d) &&
>                   __copy_mfn_to_guest_offset(exch.out.extent_start,
> --- a/xen/include/asm-arm/p2m.h
> +++ b/xen/include/asm-arm/p2m.h
> @@ -307,10 +307,9 @@ int guest_physmap_add_entry(struct domai
>                              p2m_type_t t);
>  
>  /* Untyped version for RAM only, for compatibility */
> -static inline int guest_physmap_add_page(struct domain *d,
> -                                         gfn_t gfn,
> -                                         mfn_t mfn,
> -                                         unsigned int page_order)
> +static inline int __must_check
> +guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
> +                       unsigned int page_order)
>  {
>      return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
>  }
> --- a/xen/include/asm-x86/p2m.h
> +++ b/xen/include/asm-x86/p2m.h
> @@ -583,8 +583,8 @@ int guest_physmap_add_entry(struct domain *d, gfn_t gfn,
>                              p2m_type_t t);
>  
>  /* Untyped version for RAM only, for compatibility and PV. */
> -int guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
> -                           unsigned int page_order);
> +int __must_check guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
> +                                        unsigned int page_order);
>  
>  /* Set a p2m range as populate-on-demand */
>  int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,
> 
> 



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:11:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:11:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191542.341614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSasC-0004N8-4V; Tue, 21 Sep 2021 08:11:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191542.341614; Tue, 21 Sep 2021 08:11:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSasC-0004N1-1L; Tue, 21 Sep 2021 08:11:28 +0000
Received: by outflank-mailman (input) for mailman id 191542;
 Tue, 21 Sep 2021 08:11:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSasA-0004Mv-NS
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:11:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f4a795c9-52f3-4ff7-8e0e-b243856d2543;
 Tue, 21 Sep 2021 08:11:25 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-IIj35TnaO_GnjHAXRa2YKg-2; Tue, 21 Sep 2021 10:11:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 08:11:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 08:11:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0227.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 08:11:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4a795c9-52f3-4ff7-8e0e-b243856d2543
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632211884;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=q4DEfjy8i3majxxWRX69gfYnq7AfsHgE+0qIFCaUyk8=;
	b=E+aqtBhVIddgLXym5o/krGWW7RMR/bMXherR1Xa6Quao49L56XTquqYmbivgg6ezE0skjz
	upIcYY7BwZQQWQbW4SHaJn757M+7gXbpWVwqtU23LOwKTKTkfGbTnQ4ic+/6IYTFqLItVf
	GOBtpjy5bFhkIai+7SKj+wWmntpnKZk=
X-MC-Unique: IIj35TnaO_GnjHAXRa2YKg-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H3eWvomNzQPz7UInTvPpt3TV8/uUKAvkBiQ7CINIEAiaQ9zHM7pDU7NViFRwFrxR/8P1bzUngGTM7HIQnjH3IYbkXhx8gBhBGkCgP2N9//cC5hrYYKGBztho66FP3Nw4N7thMUReCK379HCxgWxiJJn4HhMOd9cE50Qh9OckS7PsG5RGYEL5UMDrcBc848K7fNcjUTruKoMp760wEQmK8JeeRErBUlyaA82Xy/gNKskfYVkkxbMQqcRdEHmC6ONzKPztIXkU5nsHZ+tHEEKjgzrH7xWbRvMNEn6Z1JB+FV8yR/wXgYOh7UPc9EVaPafiV6nqwgaolwwiGXG0FIclFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=q4DEfjy8i3majxxWRX69gfYnq7AfsHgE+0qIFCaUyk8=;
 b=C1BJ6PquMPSCR9wReG3akAIduEsN4m2GWuzOXyXy9xdMpRxytGOisxsD+eytzYKUs3Bk6R45aU4m3mmjxkTVaNX6WTKW4mq1H7lztdbK/9HzNTV9HnNGiGaVNe7nKBX1JI0mwhPTv8nyz5BK3JYnemb7UenNGXJoxny1FadhNNe/7n2meULrwoNangnw8ebRyBRSI/bB83JYepsuL4BqPFQaEidIm43NOugdks1yFeQOoMU19bqxnTVbcIupSU084/V3IuTQayYQ9RJapVMtvDiZIi3ZowIz8qomPf45BrnhviCA3udrrBdcn6s4Zfe166tUSlH63bLPsm2ByXpFDg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
To: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210921070226.32021-1-jgross@suse.com>
 <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
 <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6305dfb1-15bc-ce4b-cbf7-d44ef5c0456d@suse.com>
Date: Tue, 21 Sep 2021 10:11:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0227.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d6095ff-2796-4f89-e21e-08d97cd76398
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71525C5F1FD42250C1EA80C5B3A19@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ojROy3FgOMDxO6SI1kQi8nnpzENpcLinDsQpSlt9VGleDHJAkq4rZPkYpDFqGJQuE2+crXxoY2uyaKLr0N1D1D51uBQnQ0p9FIK4UaUS+gGDwyZD3zILEdzgWa7NouMHzqL9tBnHKkCG4/zAANj9mtvXBtcGy6NPti1KzXFIpoe9brP73gPamfEJd1PvTBdzu3TAcIcq1KjCtPfm1xLPAKpf4suwF9P6pSDYmyO7od6Uh1fn5917QkxFpDQvw+ITvHwc/UsMR0XbKKZceJay81vHYKxbDnuRG7IB7O3R2OCx2FKkHWEvuHsWeUeA+jNrAWK/yeFOr5u/bwgZsiMmQMpdzFqa75zBKolz/bPSQJ7BbBmX3uiWLWt/dnYy4LOHzoIKPtEiRIH7u4fKFrbWKxRNgDRGuo7GrWmz8XH4vAeWbdH0jbb8jPiSXDrPwGHcmhKRfdhy6rXoLhTOiZho0IH1uPV4/kYStnQATlitdoBAtC610Wluh3grPxe1Rv1C/CKL7djzWaxiEg7bHqB1PEpul5vnKCzn+hsZNLNwoKwcY5uF+wrfAMP089DhPU1nTkMhkwBlevSebs/1XTbtVEySMi87T9MuaNy7HnuR1ge5SjwQaoHzI99k5+uA1mzlKBbNYzrhnNsJEU06UcAgN0SZMuFyp2jznlcTh1xxLzc7QyZjpNSh8CNjiM+5YP1IkWPqVHC7u2+1QOsxHoa8MNPddijOvcilo+bDeS7CuYs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6636002)(16576012)(31686004)(37006003)(54906003)(316002)(38100700002)(6486002)(2906002)(83380400001)(26005)(36756003)(8936002)(186003)(6862004)(53546011)(4326008)(2616005)(31696002)(956004)(86362001)(66946007)(66556008)(66476007)(508600001)(8676002)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K1JDYzlvbEJDMHZNNVFzWTZLYkpFNk5VbDNIWmpmRk5BUVhCYld1SGNiSDd4?=
 =?utf-8?B?bFNlRE13K2VLL3JGNjd5bjYvYjJBbm9CS2gzSHp1cXl4WmIzNmxDV1doTTEr?=
 =?utf-8?B?bGVCTjFSVWJCbk1xN3FhR1poT0ttZEk4Ylh2TFhvS0xvZzJ0YUNaSDBSemc0?=
 =?utf-8?B?ZDlqR1VYWjFPMTRYT0M5ZDRxZ3NaOXdoUVBNbFFaNXI0RmFZekNhY0RiamdH?=
 =?utf-8?B?N29GNUtHSk9pR3ZrVkNDQnA2dVhNdi8vMFZLK0RpZXR1dTdBMFI5cHIyNXRq?=
 =?utf-8?B?Tnk1eWxoa3RtTXExVVIwRjFoUnJRaUpKVDQydUlJNUNqRGNrQ3B1OFF1eDF5?=
 =?utf-8?B?V3lhYVVPRWk0MldpRDdyNVhZWTM0ay9OUnlFYW5mbkxVN0Q1aXlPeU00V2xB?=
 =?utf-8?B?NUlEbW1pUXVES1dtWTUvMWFjeVU5aWpIYy8xaC9qTlBvd1NxejlzdW81L2tM?=
 =?utf-8?B?TlZqUC9pdytYWWQxaGt4VmlCNUFEdVdacVZNU3p1eHVYZnRqRFdSbmZYSDRO?=
 =?utf-8?B?Ui95amdUQ3VWUXc4ZjNHWlJaOHdKbEp2L1FocEZFT3hBc1ZSNGlrN01sZlc0?=
 =?utf-8?B?bm9ZeWdQZTB2bWtrbk1KUmhRbW01M1REblJBclZ4MDZsMGFiWUo1SCt0cmE4?=
 =?utf-8?B?MU5mRGhxRk9yTmpYSnlpaCtXZnAyeld4czBjNXBML2tHWU9DMnlqM0VFQmww?=
 =?utf-8?B?cGRsWmlBZWhEaTRkNk0ydjQzZUZ0ckRUY3lkMWErWU9xdHE4dkhNUmFVK0lS?=
 =?utf-8?B?NkU1T3JtbmUySFJqeG5PdXozUjJ0U2xJa21DVEtHUFgzMVR0d2pFa25BT3Av?=
 =?utf-8?B?TkYvSjZubUxMUDE3QWRCamg0bWZVTERCVGV4RkdpTkM2SHhBREFEVlRRc1B5?=
 =?utf-8?B?Nk8wZlVwUERoUzE4TWxEdDhhSWtya0VFRzVuY25sV2tTZk11UGJoZWZtOUZL?=
 =?utf-8?B?dkc5Mzd4MTAreEdSU2FXTTlYalptK2FyVUlTMFNqaUVPRzIyTkRsT1lNN1V0?=
 =?utf-8?B?emRPRk9seVdMVUxlMkFnUTUzQkNDYlF4TW5SalZkSWV3bXVxUU1xN1cwc1Zy?=
 =?utf-8?B?TXVPSzIwa2xIM1hrOEg4c1JzRXZnVGMwb3pxVHBYeFdxMkVIQzdUMWMwbVdT?=
 =?utf-8?B?MnhUR3NYWERZYy9XelR6WjdlczAwcDZzZ2p2Z1paVFNuYTFhQkxmQTRmNzdM?=
 =?utf-8?B?bmJlVkNUMlpwbFpqRVhuTk93N29OYWlqR3A0cEVISlBwU2ViK2grdFhZbE5W?=
 =?utf-8?B?UVlRMWdFWlRHLzh6TXJzeWZ2aVg1Q2txZXB1U3ZWaVBMOEtLQm9GT1BvSkRY?=
 =?utf-8?B?VENqYjdZd2E2S3lvclBVbGp5WXdmQ0Fpc2g3MktRNlJERm53ZkdRb2hQYVhP?=
 =?utf-8?B?aFpXM1lwbzczdWd0UWJwZHVGN1JHUndNR1Vud0pHY2FvYWd3bEVjbjFlU0tR?=
 =?utf-8?B?ajdXTmtUNllTSnA2UHlVTkpPN0kvOWtleEdvVkJtYzZabUpZZlE1dnlSQ0tT?=
 =?utf-8?B?Vy80WkpGN1dTSXIxU29hSG9yVHlFUDhBNXFKS0hxYmo2amtKQjIxTmVjLzBB?=
 =?utf-8?B?MEp6NDV6dGMvSDZMdWJMSW1OQUVMVDV0RmdUQ1NGSWQ5T0Z6U1lEVFAxdFpz?=
 =?utf-8?B?RFFud1RhMWJYc1N2SDdrM3dPL1pZWmhHekZLc1BWakl2U000OUFRU0h0U2dZ?=
 =?utf-8?B?ekVnakozMGo1QUltdGdTTnU2cXVxN1FDWmlvOFRDMTNYTGQ2eE01aHluSUE3?=
 =?utf-8?Q?H8uyPgvi4Hv7x6xKV64Tt/I0zbjA8f1Cu6aIyfU?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d6095ff-2796-4f89-e21e-08d97cd76398
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 08:11:18.1461
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HwE+hrgPnN352yJ+5qQVGcWkSy3VpA+m4sZxPvB5p9MSnyqViZ9i1ZJtPsiY9zICex1ZxuJR4LIfyoXKxrfbkA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

On 21.09.2021 09:58, Juergen Gross wrote:
> On 21.09.21 09:53, Jan Beulich wrote:
>> On 21.09.2021 09:02, Juergen Gross wrote:
>>> --- a/arch/x86/xen/irq.c
>>> +++ b/arch/x86/xen/irq.c
>>> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>>>   {
>>>   	struct vcpu_info *vcpu;
>>>   
>>> -	/*
>>> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
>>> -	 * cleared, so disable preemption to ensure we check for
>>> -	 * events on the VCPU we are still running on.
>>> -	 */
>>> -	preempt_disable();
>>> -
>>>   	vcpu = this_cpu_read(xen_vcpu);
>>>   	vcpu->evtchn_upcall_mask = 0;
>>>   
>>> -	/* Doesn't matter if we get preempted here, because any
>>> -	   pending event will get dealt with anyway. */
>>> +	/*
>>> +	 * Now preemption could happen, but this is only possible if an event
>>> +	 * was handled, so missing an event due to preemption is not
>>> +	 * possible at all.
>>> +	 * The worst possible case is to be preempted and then check events
>>> +	 * pending on the old vcpu, but this is not problematic.
>>> +	 */
>>
>> I agree this isn't problematic from a functional perspective, but ...
>>
>>>   	barrier(); /* unmask then check (avoid races) */
>>>   	if (unlikely(vcpu->evtchn_upcall_pending))
>>>   		xen_force_evtchn_callback();
>>
>> ... is a stray call here cheaper than ...
>>
>>> -
>>> -	preempt_enable();
>>
>> ... the preempt_{dis,en}able() pair?
> 
> The question is if a stray call in case of preemption (very unlikely)
> is cheaper than the preempt_{dis|en}able() pair on each IRQ enabling.
> 
> I'm quite sure removing the preempt_*() calls will be a net benefit.

Well, yes, I agree. It would have been nice if the description pointed
out the fact that preemption kicking in precisely here is very unlikely.
But perhaps that's considered rather obvious ... The issue I'm having
is with the prior comments: They indicated that preemption happening
before the "pending" check would be okay, _despite_ the
preempt_{dis,en}able() pair. One could view this as an indication that
this pair was put there for another reason (e.g. to avoid the stray
calls). But it may of course also be that the comment simply was stale.

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:25:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:25:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191548.341625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSb5E-00061Q-Bm; Tue, 21 Sep 2021 08:24:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191548.341625; Tue, 21 Sep 2021 08:24:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSb5E-00061J-8q; Tue, 21 Sep 2021 08:24:56 +0000
Received: by outflank-mailman (input) for mailman id 191548;
 Tue, 21 Sep 2021 08:24:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSb5D-00061D-9u
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:24:55 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 73015a49-1397-4d8e-9719-58e04d33a888;
 Tue, 21 Sep 2021 08:24:54 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 37E901FE5E;
 Tue, 21 Sep 2021 08:24:53 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D642713BB6;
 Tue, 21 Sep 2021 08:24:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id lhtwMtSWSWELIgAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 08:24:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73015a49-1397-4d8e-9719-58e04d33a888
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632212693; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=q8wjQucmD0MyIsVl3SYX49fFHgtd5lz+VR1+pnqWZ8M=;
	b=FsM/MyqoNKxflas+45NhStH0KnC1V0KAjsnKFarWTcnkEoLcyyHSQlmmIFlBtHMdeTWetL
	Zt1FDuj/erbEmSbo1Z71CYztbC/urwBGEuoDDq9cm19srPeppi1Ve2njiBsWTlBd6twEBy
	GxmX4UfWf36TteM2g78+iJzuWUcgmK4=
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
References: <20210921070226.32021-1-jgross@suse.com>
 <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
 <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
 <6305dfb1-15bc-ce4b-cbf7-d44ef5c0456d@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <ccd495a5-ba43-2541-f4d0-fb44f5516a55@suse.com>
Date: Tue, 21 Sep 2021 10:24:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <6305dfb1-15bc-ce4b-cbf7-d44ef5c0456d@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="VeEOQinUwhdk0i01Jvx5ONHNBP5jE3n80"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--VeEOQinUwhdk0i01Jvx5ONHNBP5jE3n80
Content-Type: multipart/mixed; boundary="fb8vCpwLAN3ctP5eaDNHkeJMNfBtJreZB";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Message-ID: <ccd495a5-ba43-2541-f4d0-fb44f5516a55@suse.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
References: <20210921070226.32021-1-jgross@suse.com>
 <99e6cf1d-ce6a-83e5-2e43-12f6c1a89f3f@suse.com>
 <599f4e20-a25d-6cd0-ced5-f2deaf997535@suse.com>
 <6305dfb1-15bc-ce4b-cbf7-d44ef5c0456d@suse.com>
In-Reply-To: <6305dfb1-15bc-ce4b-cbf7-d44ef5c0456d@suse.com>

--fb8vCpwLAN3ctP5eaDNHkeJMNfBtJreZB
Content-Type: multipart/mixed;
 boundary="------------979DD9AA526374DA0DD89F37"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------979DD9AA526374DA0DD89F37
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 10:11, Jan Beulich wrote:
> On 21.09.2021 09:58, Juergen Gross wrote:
>> On 21.09.21 09:53, Jan Beulich wrote:
>>> On 21.09.2021 09:02, Juergen Gross wrote:
>>>> --- a/arch/x86/xen/irq.c
>>>> +++ b/arch/x86/xen/irq.c
>>>> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>>>>    {
>>>>    	struct vcpu_info *vcpu;
>>>>   =20
>>>> -	/*
>>>> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
>>>> -	 * cleared, so disable preemption to ensure we check for
>>>> -	 * events on the VCPU we are still running on.
>>>> -	 */
>>>> -	preempt_disable();
>>>> -
>>>>    	vcpu =3D this_cpu_read(xen_vcpu);
>>>>    	vcpu->evtchn_upcall_mask =3D 0;
>>>>   =20
>>>> -	/* Doesn't matter if we get preempted here, because any
>>>> -	   pending event will get dealt with anyway. */
>>>> +	/*
>>>> +	 * Now preemption could happen, but this is only possible if an ev=
ent
>>>> +	 * was handled, so missing an event due to preemption is not
>>>> +	 * possible at all.
>>>> +	 * The worst possible case is to be preempted and then check event=
s
>>>> +	 * pending on the old vcpu, but this is not problematic.
>>>> +	 */
>>>
>>> I agree this isn't problematic from a functional perspective, but ...=

>>>
>>>>    	barrier(); /* unmask then check (avoid races) */
>>>>    	if (unlikely(vcpu->evtchn_upcall_pending))
>>>>    		xen_force_evtchn_callback();
>>>
>>> ... is a stray call here cheaper than ...
>>>
>>>> -
>>>> -	preempt_enable();
>>>
>>> ... the preempt_{dis,en}able() pair?
>>
>> The question is if a stray call in case of preemption (very unlikely)
>> is cheaper than the preempt_{dis|en}able() pair on each IRQ enabling.
>>
>> I'm quite sure removing the preempt_*() calls will be a net benefit.
>=20
> Well, yes, I agree. It would have been nice if the description pointed
> out the fact that preemption kicking in precisely here is very unlikely=
=2E
> But perhaps that's considered rather obvious ... The issue I'm having
> is with the prior comments: They indicated that preemption happening
> before the "pending" check would be okay, _despite_ the
> preempt_{dis,en}able() pair. One could view this as an indication that
> this pair was put there for another reason (e.g. to avoid the stray
> calls). But it may of course also be that the comment simply was stale.=


The comment is older than the preempt_*() calls.

Those were added 8 years ago claiming they'd prevent lost events, but
at the same time at lease one other patch was added which really
prevented lost events, so adding the preempt_*() calls might just have
been a guess at that time.

> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks,


Juergen


--------------979DD9AA526374DA0DD89F37
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------979DD9AA526374DA0DD89F37--

--fb8vCpwLAN3ctP5eaDNHkeJMNfBtJreZB--

--VeEOQinUwhdk0i01Jvx5ONHNBP5jE3n80
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJltQFAwAAAAAACgkQsN6d1ii/Ey/X
0Af/YYpnfWdqpmJKUhOyWhB8i4aIJYfRRrpF17vhNmdezr6cHNCFHUeQpl6sj6acnSpLUEOeqNjR
NiOM4HZYqA899JPMs4bonsEopNoxLGbN0LlSqc4Rv0LctW8byjR490iQHxfRP0DNaiU/HB4G8HXB
fGwJU0AzioCFmwKNaKVsHq8BMLnuMgwoYfomf17k4c8YcB+SWLSVmHcd8eqaSp8ZKdTI/mAgbPzj
W73DLII+p0C4CaYwSqiurURZkHO6AtTk7hDtXaVDHrA2bGwCQMiL5mq7KLBn2A4eVsrNIeYlFabX
w1+Nmni2IgsSp3FSB73h9LyUhVyZTu5InHSzq9zq6A==
=7u1G
-----END PGP SIGNATURE-----

--VeEOQinUwhdk0i01Jvx5ONHNBP5jE3n80--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:28:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:28:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191563.341640 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSb8s-0006wQ-2E; Tue, 21 Sep 2021 08:28:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191563.341640; Tue, 21 Sep 2021 08:28:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSb8r-0006wJ-Uj; Tue, 21 Sep 2021 08:28:41 +0000
Received: by outflank-mailman (input) for mailman id 191563;
 Tue, 21 Sep 2021 08:28:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v6EE=OL=infradead.org=peterz@srs-us1.protection.inumbo.net>)
 id 1mSb8q-0006wD-Ew
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:28:41 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e1b11eb-0899-4a09-b328-2efbc863dc62;
 Tue, 21 Sep 2021 08:28:37 +0000 (UTC)
Received: from j217100.upc-j.chello.nl ([24.132.217.100]
 helo=noisy.programming.kicks-ass.net)
 by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux))
 id 1mSb7l-003dSV-QU; Tue, 21 Sep 2021 08:27:45 +0000
Received: from hirez.programming.kicks-ass.net
 (hirez.programming.kicks-ass.net [192.168.1.225])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (Client did not present a certificate)
 by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id EFA53300252;
 Tue, 21 Sep 2021 10:27:32 +0200 (CEST)
Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000)
 id C97CD213969B1; Tue, 21 Sep 2021 10:27:32 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e1b11eb-0899-4a09-b328-2efbc863dc62
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=3pGj8C8pWUiQBpKJtGVzknsFdRDfba8Mc0AV+KNc0rs=; b=lj3fL5TA/SHrtjjBNRJLFNmF+r
	ApEI0NEfHknyBJuuqWRBOpq3IUCjYhRNG1iMmW4H9+KeW1m+2T7sIPbkGQY5y7Nz23k74qcp62ezp
	E4ILAOwxrDfVl0C88TY4pOY2hpzbWDUjNt1UqdViBmSMyCa0pTXr4SVD7jeAMKZTxWJs1+UK6TnX/
	3Xk0DGp8rheZZ1w7uFMACeKPdrqynr5hwUjL1A4svkkgPKpNa8u1Qpm1uj6GwgkcB0t29T9OSjUd9
	QCK/yhOpoevqRgJvBfe5PzdJQ1BKgc3mvum6RljFYCF3T30fHkgfYdC6Yl2G7VVWKFtJ/uO4E1zsG
	z1kNBDuQ==;
Date: Tue, 21 Sep 2021 10:27:32 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
Message-ID: <YUmXdEGhgEoukk80@hirez.programming.kicks-ass.net>
References: <20210921070226.32021-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210921070226.32021-1-jgross@suse.com>

On Tue, Sep 21, 2021 at 09:02:26AM +0200, Juergen Gross wrote:
> Disabling preemption in xen_irq_enable() is not needed. There is no
> risk of missing events due to preemption, as preemption can happen
> only in case an event is being received, which is just the opposite
> of missing an event.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  arch/x86/xen/irq.c | 18 +++++++-----------
>  1 file changed, 7 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c
> index dfa091d79c2e..ba9b14a97109 100644
> --- a/arch/x86/xen/irq.c
> +++ b/arch/x86/xen/irq.c
> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>  {
>  	struct vcpu_info *vcpu;
>  
> -	/*
> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
> -	 * cleared, so disable preemption to ensure we check for
> -	 * events on the VCPU we are still running on.
> -	 */
> -	preempt_disable();
> -
>  	vcpu = this_cpu_read(xen_vcpu);
>  	vcpu->evtchn_upcall_mask = 0;
>  
> -	/* Doesn't matter if we get preempted here, because any
> -	   pending event will get dealt with anyway. */
> +	/*
> +	 * Now preemption could happen, but this is only possible if an event
> +	 * was handled, so missing an event due to preemption is not
> +	 * possible at all.
> +	 * The worst possible case is to be preempted and then check events
> +	 * pending on the old vcpu, but this is not problematic.
> +	 */
>  
>  	barrier(); /* unmask then check (avoid races) */
>  	if (unlikely(vcpu->evtchn_upcall_pending))
>  		xen_force_evtchn_callback();
> -
> -	preempt_enable();
>  }
>  PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable);
>  
> -- 
> 2.26.2
> 

So the reason I asked about this is:

  vmlinux.o: warning: objtool: xen_irq_disable()+0xa: call to preempt_count_add() leaves .noinstr.text section
  vmlinux.o: warning: objtool: xen_irq_enable()+0xb: call to preempt_count_add() leaves .noinstr.text section

as reported by sfr here:

  https://lkml.kernel.org/r/20210920113809.18b9b70c@canb.auug.org.au

(I'm still not entirely sure why I didn't see them in my build, or why
0day didn't either)

Anyway, I can 'fix' xen_irq_disable(), see below, but I'm worried about
that still having a hole vs the preempt model. Consider:

xen_irq_disable()
  preempt_disable();
  <IRQ>
    set_tif_need_resched()
  </IRQ no preemption because preempt_count!=0>
  this_cpu_read(xen_vcpu)->evtchn_upcall_mask = 1; // IRQs are actually disabled
  preempt_enable_no_resched(); // can't resched because IRQs are disabled 

  ...

xen_irq_enable()
  preempt_disable();
  vcpu->evtch_upcall_mask = 0; // IRQs are on
  preempt_enable() // catches the resched from above


Now your patch removes that preempt_enable() and we'll have a missing
preemption.

Trouble is, because this is noinstr, we can't do schedule().. catch-22

---
Subject: x86/xen: Fixup noinstr in xen_irq_{en,dis}able()
From: Peter Zijlstra <peterz@infradead.org>
Date: Mon Sep 20 13:46:19 CEST 2021

  vmlinux.o: warning: objtool: xen_irq_disable()+0xa: call to preempt_count_add() leaves .noinstr.text section
  vmlinux.o: warning: objtool: xen_irq_enable()+0xb: call to preempt_count_add() leaves .noinstr.text section

XXX, trades it for:

  vmlinux.o: warning: objtool: xen_irq_enable()+0x5c: call to __SCT__preempt_schedule_notrace() leaves .noinstr.text section

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/xen/irq.c |   24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

--- a/arch/x86/xen/irq.c
+++ b/arch/x86/xen/irq.c
@@ -44,12 +44,18 @@ __PV_CALLEE_SAVE_REGS_THUNK(xen_save_fl,
 
 asmlinkage __visible noinstr void xen_irq_disable(void)
 {
-	/* There's a one instruction preempt window here.  We need to
-	   make sure we're don't switch CPUs between getting the vcpu
-	   pointer and updating the mask. */
-	preempt_disable();
+	/*
+	 * There's a one instruction preempt window here.  We need to
+	 * make sure we're don't switch CPUs between getting the vcpu
+	 * pointer and updating the mask.
+	 */
+	preempt_disable_notrace();
 	this_cpu_read(xen_vcpu)->evtchn_upcall_mask = 1;
-	preempt_enable_no_resched();
+	/*
+	 * We have IRQs disabled at this point, rescheduling isn't going to
+	 * happen, so no point calling into the scheduler for it.
+	 */
+	preempt_enable_no_resched_notrace();
 }
 __PV_CALLEE_SAVE_REGS_THUNK(xen_irq_disable, ".noinstr.text");
 
@@ -62,7 +68,7 @@ asmlinkage __visible noinstr void xen_ir
 	 * cleared, so disable preemption to ensure we check for
 	 * events on the VCPU we are still running on.
 	 */
-	preempt_disable();
+	preempt_disable_notrace();
 
 	vcpu = this_cpu_read(xen_vcpu);
 	vcpu->evtchn_upcall_mask = 0;
@@ -74,7 +80,11 @@ asmlinkage __visible noinstr void xen_ir
 	if (unlikely(vcpu->evtchn_upcall_pending))
 		xen_force_evtchn_callback();
 
-	preempt_enable();
+	/*
+	 * XXX if we noinstr we shouldn't be calling schedule(), OTOH we also
+	 * cannot not schedule() as that would violate PREEMPT.
+	 */
+	preempt_enable_notrace();
 }
 __PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable, ".noinstr.text");
 


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:33:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:33:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191571.341651 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbD1-0008Hj-Jk; Tue, 21 Sep 2021 08:32:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191571.341651; Tue, 21 Sep 2021 08:32:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbD1-0008Hc-GY; Tue, 21 Sep 2021 08:32:59 +0000
Received: by outflank-mailman (input) for mailman id 191571;
 Tue, 21 Sep 2021 08:32:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s/tM=OL=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSbD0-0008HW-FL
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:32:58 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 06cd7690-4cb1-49fb-bee7-6ebe993dbe68;
 Tue, 21 Sep 2021 08:32:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06cd7690-4cb1-49fb-bee7-6ebe993dbe68
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632213176;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=pKJ//icSM6kIxwhqySj9SS2NC5ckvqjD6UM0/dprF0I=;
  b=LKzc0CYUyYnIcfJFeO0YneM4o/PC+GsOaPZj/buF+OJ/QYenMRaaTGhY
   4CEdnybPyYGvg1A/MKlvcDZ38UKAupTSBuCpKz7h0qyAUwcbA+LMR4faI
   +LhBAhxDZ57NktEUW6w/AgPyV7nisAvcTuDvlbXuw0FbbW+V3Zhy7mDf2
   w=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: lbOyRzhIWoxEFJ33XLRwghfh2sZsDubz5uirx7tyQj9uG6kEXydFiPt0XzvaRx29YT4Z5iisCn
 e/s2uvs7z+D8Uk7jWWfXhkC80o0OvcOl227RgQtckZNCYiefAhdpYiIY2Q0R/vjktHgg8F4iMo
 FUdeEE/Obn11i7NoGXINNasi42zm7HeRayfiOJ1fuzkN5vnGO5UsH1HVshR2CKAz4W7rOBPROW
 yEGs40Lb5OMeleEJ+XfKJTzCDTuTuCkTeZrJIQwGj3Zju8o/VtaBrWcA4WTPnwvGW8g4yZksB0
 mRJHJ1SWoT/Qvfq99q+H1Hmq
X-SBRS: 5.1
X-MesageID: 53212629
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:VGcNYatmoMmVIdepuNulQMYWF+fnVLhZMUV32f8akzHdYApBsoF/q
 tZmKTqAPfeKZmT3eYgnao+x9RkCvZTXyIA2S1A6q3szFStE+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524HhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Nplqtuae1ovGr/1suEQbj5eMh1jArB+5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sURKyAP
 ZtxhTxHdCnYfxBfCFguOIM+nd7xhWGjLS9+pwfAzUYwyzeKl1EguFT3C/LXZ9iLSMN9jkue4
 GXc8AzRGQoGPdaSzT6E9HOEheLVmy7/HoUIG9WQ6fpCkFCVgGsJB3U+V1G2vP24gU6WQM9EJ
 gof/S9Ghbg/8gmnQ8fwWzW8oWWYpVgMVtxICeo45QqRjK3O7G6xB3cGZi5MbsQ8s807TiBs0
 UWG9+4FHhQ27ufTEyjEsO7J83XiYkD5MFPuewcbCiE3vt/HsrgZix/yadBCLLOKk+XqTGSYL
 y+xkMQuu1kCpZdVjPzrpgGW22jESovhFVFut16ONo6xxkYgPtf0OdbwgbTOxasYdO6kok+9U
 G/ociR0xNsHC42E3AeJSf8EdF1Cz6fYaGCA6bKD8p9IythMx5JBVdsLiN2dDB0wWirhRdMOS
 BWI0T69HLcJYBOXgVZfOupd8fjGKJQM8vy+DJjpgidmOMAtJGdrAgk3PRP4M5/RfLgEzvhkZ
 MbznTeEJncGE6V3pAdatM9EiuRD+8zK/kuKHcqT503+idK2PSfJIZ9YYArmRr1ot8us/VSKm
 +uzwuPXkn2zpsWlOXKJmWPSRHhXRUUG6Wfe8Z0PKrHbfVo4RwnMyZb5mNscRmCspIwM/s/g9
 XChQE5Ijl35gHzMMwKRbX5/LrjoWP5CQbgTYETA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:9duF8KE8c08H8Hc2pLqFeZHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HmBEClewKnyXcT2/htAV7CZnichILMFu9fBOTZsl/d8kHFh4tgPO
 JbAtRD4b7LfClHZKTBkXCF+r8bqbHtmsDY5pav854ud3ATV0gJ1XYGNu/xKDwReOApP+tcKH
 LKjfA32AZINE5nI/iTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1Svl
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfpGoCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8DPeiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6Np+TgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeTf003MwmM29yUkqp+1WGmLeXLzAO91a9MwY/U/WuontrdCsT9Tpe+CQd9k1wva7VBaM0od
 gtn8xT5cVzp/QtHNBA7dE6ML2K41z2MGHx2V2pUCHa/YE8SjnwQs3Mkf8IDN/DQu1+8HJ1ou
 WZbG9l
X-IronPort-AV: E=Sophos;i="5.85,310,1624334400"; 
   d="scan'208";a="53212629"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZFDA0axfIaI7rTsasuPgc3kEVa39wN8wKXjiezigJhK2H4dhs/sjQqFhhQi6vUoJ4frlxERrNyWOTqBg7am3MGszQ/cx0MaGMMtOxemzQLMtmOIUentSMTpxtPKctB6CRrO0GjFvG6w1vNu3Wi9g9wfNqLXF+/r7t4uJ90U9QE2oLYIAj+YiR5w45rE1wtv+2Ci0aNT+zJw6eDTCNdwaKxi4zu7dE6WRnPxmGh7B9KnuOeFWjitlihTgaoKnEsczAV+j8PpEY7O6vbgeo1Y7C1ZU12VIB6bKRzdwJWx4QEdhuMHB8U9muNZBknZNW+v57rh+XhydHZ+QtBFj8wKfXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aogi4k5fbzzn/66Kn/jw9xSo4g25SW0DWk2MbKqNI2s=;
 b=RlDKjSWE+7fN9XfK35mvDlI7fqOfq9paDLxUES+pzUD272zOp3jOtrcXUd9O/hcM/35dZNhDFZZ0KE/IVXX4VgQClwyVYDlARIfGwVfP6PA5XYk8CkBqjsNuWCqzo7KSXKi4yuuwbG7R336HK75uTidymmdUNXO7PQAK6dcrfSp0uQ0McpuGAHV9zVBuMTI8MH1aw58P6p9TnEvYioDfFkAnIpMOrDJloKtG2TrlhFqEF5y65fSyaRFlOPC+ChFExmifOpOGpLoxEyRiJtdXEny2HrUpR7ZffOx2oVbLQG1LxdupZUvHoUhbpBa63wxNstBYgR9gVLZd1EK5i96gZw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aogi4k5fbzzn/66Kn/jw9xSo4g25SW0DWk2MbKqNI2s=;
 b=rdJNKXdXH8cQcCrJeISdXN/cHoV+mYfyFOt/qZRyroSF9aXlH/MEEWoKHDFfwYq9VOiAy8Buu7bZX1f25YaTEfW9KqpKyA6ZfxJ6pcc7+GsZBV35278osD0F8rHopHeG4L5W8VsBGc47kabS0hPBbPrefhaqvAnW4p7UsMj8EU8=
Date: Tue, 21 Sep 2021 10:32:39 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
Message-ID: <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
X-ClientProxiedBy: LO4P265CA0047.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2ac::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8722e957-8a02-4d70-6f9f-08d97cda63f7
X-MS-TrafficTypeDiagnostic: DM6PR03MB5066:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB506676E0667D4B9BFF0BBEE38FA19@DM6PR03MB5066.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: k+kT88xcWGfBLBx2ZDdUU+gNNuStOFZHlqq6H/56F0onmR/5KUloEuQEAG/UmpU3A8GGeYMKr0cXgLNNxMtbxEqiC5Z0n+bWo8XGYg4OfftpW4ZD7sU7ZW3PZegpcvg7l5202iv9QAYjWfJO/IetZGeOguu+kwX2/EGm4HMxidNAu4yn9ovw6kcunzsQIXR0HJIT3bDesGdHhzhgeGCFzH7iQvTG/HOVgw5lpxEPxpFIbZcQ20b4YaV9AOJR0Ei3p5ZGZHqvltiAZk1K7Q4357pD51hhsGpDyszn71lT0115c3H4cFi9q5Vcwbq68roKmZgu0q/kgkyoO6Folhic1wXCYNMZiMhcfY2tj4M/RwZcPsKznADReN2H4cdj5mEfsryg/+ASzsxbqnFDJcT39JdH29v3jm7Yhk6o4OMcMwbUa/vRbEWk+/bUIoiLb/d6XjrUKhNLa46f3KuJgkMh2MIFNah1s0PqXqZA+vXpWh2q/tODwQV+s/VxRAyiS8Hly4xeTqQojKUDzq+Gxy4NBiHSrOdA6Wyk+VoXg3BNVhMayUI+tzhPwLbQFENvsV6IfOtQy/UBnGSpDsGXAK7khfymAGWVVk+72CQ3n+q4Z1aArK5xZKVqQ9sV4tUpohPeLXiF4rk+CjAhWFTmmBGTmq6H0mHn5JPkMBdJpw8/jlq0rLfJQ8YEqwtdPvwtUOi/zb1aD6YU8utEz9OmXhRbaw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(508600001)(316002)(8676002)(66946007)(4326008)(66476007)(83380400001)(26005)(86362001)(6496006)(6916009)(5660300002)(2906002)(38100700002)(9686003)(956004)(54906003)(85182001)(186003)(66556008)(6486002)(6666004)(53546011)(76704002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R1Q3QXRkUkFLY3dCbnV3b25IVWFXTENGcEg2blgrajU3RnNvZUNBTi9TL3Nj?=
 =?utf-8?B?RnNuK0g0VGtRdGoxYUk4Z1hJbXd4a21TSHFySU9lTjJra1hqL2d5T0xRRHAr?=
 =?utf-8?B?aHdLK3RxNGRkVDRQZFZoWmI3VFJsd0JqdkdMSTJRcXFDWmo2WTlXQnRPd3Bx?=
 =?utf-8?B?SHVtTzNzZjE0RGhCVE5kWTlWK0tjVGF2WVF6NlEvamZmK0JvbG5kbkp3bnhV?=
 =?utf-8?B?MjBERWhZSjVRaFpKMXdOcG5CQkNiczRCMVN5Ri8yQ01zM3I4V2pUenVxUGxi?=
 =?utf-8?B?WG1wRUVpb1IydkVFbytxWGFVcXdPYlJhWldJMTVkQnhIL2hjdlMyYzdDUjBW?=
 =?utf-8?B?T1dLRUtqWFZQZ3RXYUVHVnhrMFdiSmdwRklMWEVmVjdHMWJYNHpxUXM4bjRW?=
 =?utf-8?B?THhqcUkzamxKc2tFeUdRVlNIWlFxd2xKdDVNdExRY0E4UXBhSmhadTdZWVZ6?=
 =?utf-8?B?dVdLR2REUW9ySlA5bWdtc0RTVXJNalhkb3BycUIzdzhLTFM5RVJKaDNZZUNr?=
 =?utf-8?B?alFobXQrRkZsMFYrdkdxMTBYU29sT1VOaU1XUXVLYXJuMzBRN1g4NDVPNW1m?=
 =?utf-8?B?aFdEVkdMcy95MmhLMk5XNVlta0t4SFJ6bVROdDNsSCtUcW5yUUh0NTRnS1ZE?=
 =?utf-8?B?bTNTb0orSERZUVFvSGF6NGJsZDQ4SFM0Z1diK1JXTitiVlA3aUZ3SjVEdU5s?=
 =?utf-8?B?eEZTY0lES0VJMmRxQjN2ZDNTTTNvNkdrODJXd2E2Q1FOeFJ5Z3VKZEJKMTBQ?=
 =?utf-8?B?SXJ0MmNjN0NlQmJCSjBvWFFyU256cmh0U3lzNnBjTW1tUmRicCtTWHZ5eFdl?=
 =?utf-8?B?Z3ZSdVlPbDZGbnlMNkpsUjFyZ2QzbjZKdVQrZXR6SWhKM2VqTkY0OGNsZXgw?=
 =?utf-8?B?OW11U0lTNCtEbWdER2RyeldNOE0rOEdGNjJVNkh6cEZaY2Vqell5NkpWRDdm?=
 =?utf-8?B?blRFVUtDaGZkTG16WmNKMFU4T2lvQzQ2Z2ptUjJ2a2E5WmdLZmEyQXBOcGlk?=
 =?utf-8?B?WnJkUXJVQUEwZ3NjRjU2TW16RWRmQU9TTGdTZnRqRzJubHpiajBleTJGazJm?=
 =?utf-8?B?d0lPV0Q2Um1JdW5yZDlKL0NlZGtRR0E0d0RybEFORUtKRUZ3blh3UFk3d3dQ?=
 =?utf-8?B?WUs0UVZjVTlrMGxNRHZlZmNGd3RJWTRuc2EvSUNMTitWcjkyYzVPQnRZNkh0?=
 =?utf-8?B?MmlRV1ovMmFQUitDVDR4YTVCWWI3VmNCb0NuZzgyblFDK2xXeUk5U2JScGQx?=
 =?utf-8?B?RUVSL3lsTDFZb2FpRVIxL3Noek9aeDkxbUxUajN3UlZXR1hKM0dscE5WdlFk?=
 =?utf-8?B?UmtEd0JpYXA0dWlDczBmdjRTWmE0Wks2NDhudmxGWDdUUlByUUF4b2pkMXVH?=
 =?utf-8?B?UEVIZmgrdFNMeWtLNTM1VCtjQXlqMTIzR1VqckY2Ykt5eEtXVmlCSDhEejJF?=
 =?utf-8?B?c0Z0YjRFNVpWaDFUdnIrNUEwTDNhc0hrVDY5ZGtnWlZBRTUyYTZ4L1VMVUtY?=
 =?utf-8?B?OUlMV1JKTmNlcmFLMHM4MFFucVRnT1lVMkpvWk4yeDZpSzJNVTlVc2JEL0lh?=
 =?utf-8?B?NGFGYjBVbmEvTllSR0g4c3J2amtadHJWS3JpbmhiYWNDdkY1ZHFRRWRkVkVs?=
 =?utf-8?B?MFJteHlpa3ltTSthREhmUzhMVms0YzUvUXo3RGszaUcreXJSUWI5RmhWeG1p?=
 =?utf-8?B?eS9vU1VIRENXeXAzZmRkRTVZb25JOHNFdWlrQWdmZDcyKzFYcThjVHlnaHEv?=
 =?utf-8?Q?v3Xi5sYBdbCaozhhdds4fTQqv/uw/0gVhm9cY9s?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8722e957-8a02-4d70-6f9f-08d97cda63f7
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 08:32:47.3561
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vhCNnP8q0fHiZRey8L4GXcLehbZ4ZYKTwv3yrmqD98NkugtCtiIxoPxhFcyZEP+GTjF1gHrZ8x9RK17nO7mWCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5066
X-OriginatorOrg: citrix.com

On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
> On 20.09.2021 12:20, Roger Pau Monné wrote:
> > On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
> >> --- a/xen/include/asm-arm/grant_table.h
> >> +++ b/xen/include/asm-arm/grant_table.h
> >> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
> > 
> > I'm slightly confused by this checks, don't you need to check for
> > gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
> > guest_physmap_remove_page?
> 
> Why? It's ogfn which gets passed to the function. And it indeed is the
> prior GFN's mapping that we want to remove here.
> 
> > Or assuming that ogfn is not invalid can be used to imply a removal?
> 
> That implication can be (and on x86 is) used for the incoming argument,
> i.e. "gfn". I don't think "ogfn" can serve this purpose.

I guess I'm confused due to the ogfn checks done on the Arm side that
are not performed on x86. So on Arm you always need to explicitly
unhook the previous GFN before attempting to setup a new mapping,
while on x86 you only need to do this when it's a removal in order to
clear the entry?

So you are effectively only removing the call to
guest_physmap_remove_page in gnttab_map_frame for x86, because Arm
will still perform it in gnttab_set_frame_gfn.

This seems like a limitation of Arm's guest_physmap_add_entry.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 08:43:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 08:43:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191577.341662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbMg-0001RV-KK; Tue, 21 Sep 2021 08:42:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191577.341662; Tue, 21 Sep 2021 08:42:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbMg-0001RO-Ez; Tue, 21 Sep 2021 08:42:58 +0000
Received: by outflank-mailman (input) for mailman id 191577;
 Tue, 21 Sep 2021 08:42:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XWt1=OL=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mSbMe-0001RI-IP
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 08:42:56 +0000
Received: from mail-oi1-x232.google.com (unknown [2607:f8b0:4864:20::232])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4d514461-600e-4a08-8b8b-8ef0d48f4c91;
 Tue, 21 Sep 2021 08:42:55 +0000 (UTC)
Received: by mail-oi1-x232.google.com with SMTP id 24so7619209oix.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 01:42:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d514461-600e-4a08-8b8b-8ef0d48f4c91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=NvE6Ye47LR0TteUVXln1IsRXjg42V7WUT9XlhMTBCws=;
        b=UTZI9WhoM/P7Wkf5vlXwbbuVUEPqVQELG5W09fwrlq2jVMCl5V9Mp0MzRYwoqyNAsq
         j3m9HosZXGo6lm4dbfL/u7lY7ZmlRyL0ppZKgsM9wRC9eH6y93euNCRWHDvelzwlxDsQ
         g81vq5WGzFw535YeEyBSx5q6GKJxy5s+YkyVK+NOPgFfS0caV07MwIHvZFLzdvKzkCLQ
         HBh3utvL6pJS3LR7vaWMXiEStW8muXSgUNLbwyYHC4mTHVa6MDjwhZwwfJ7xvtxY5iNT
         FTwE/Px8l9ymJ3ImTJEG2Rr8s+ionL+ieOyLpQneSPU8rvbIfqKJ6yqWZmbFgovxMctc
         /K/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=NvE6Ye47LR0TteUVXln1IsRXjg42V7WUT9XlhMTBCws=;
        b=NWOwR8IyCOalgq+hC7vNcduZJLqCv151htTgQJwQ/JH6Zj7H5VMVwaTR5djg30bcKn
         YUKrVr2h5wvHOBRDEIVWxMDFXkxLyTcNT8FG393oPFmLB4ufxMPi+fu9e4b8BvoT3aCp
         bF8t8duszqEojCMJ53x2+MukhyNlXdhuSNmsWVXOUR3uEq2FDBOqoyiTXhbEsscQZsic
         15/vMJWCFS2CnGTywnkDx9+2LGpE/+BeTV3ss+Cw+aYp95cuhGDy69heOEfS3/OGCpec
         OiRnTuWNIGgxkCO8ZojjmUhVtaiYeVrwhXqi224ovYtVChBvJEwrDNzgyA0Zh2oScV3o
         eopw==
X-Gm-Message-State: AOAM53044z7q/Zu1gpqGGQ/5WmF7jamJLW2DLMk2BALR+Fj+c3/26nYx
	DNzMh9UPRkI6D7yWjtSWHcA+gIF+Y8m/cC32NFM=
X-Google-Smtp-Source: ABdhPJybzOXPasHC7W/GTuPP+KISuIjezig8BMq/GrM7LgAD0XO9wOGEgecvHQYPmpf8UzhkHJ0STUkUJ0UpK9Vw7bI=
X-Received: by 2002:a54:4d8f:: with SMTP id y15mr2685122oix.122.1632213774708;
 Tue, 21 Sep 2021 01:42:54 -0700 (PDT)
MIME-Version: 1.0
References: <alpine.DEB.2.21.2109131441520.10523@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109131441520.10523@sstabellini-ThinkPad-T480s>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Tue, 21 Sep 2021 01:42:43 -0700
Message-ID: <CACMJ4GZGk_Rz0T+uZpKYxwLBgwudbR7KVzP1LhY7O2W++Zq0qg@mail.gmail.com>
Subject: Re: Dom0less + Argo enablement
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Alec.Kwapis@dornerworks.com, 
	Jason Andryuk <jandryuk@gmail.com>, eric chanudet <eric.chanudet@gmail.com>, james.mckenzie@hp.com, 
	christian.limpach@hp.com, Daniel Smith <dpsmith@apertussolutions.com>, 
	Rich Persaud <persaur@gmail.com>, openxt <openxt@googlegroups.com>, 
	Nicholas Tsirakis <tsirakisn@ainfosec.com>
Content-Type: multipart/alternative; boundary="0000000000003d350c05cc7d6259"

--0000000000003d350c05cc7d6259
Content-Type: text/plain; charset="UTF-8"

On Mon, Sep 13, 2021 at 3:41 PM Stefano Stabellini <sstabellini@kernel.org>
wrote:

> Hi all,
>
> This email is for anybody interested in using Argo with Dom0less setups
> for domain-to-domain communications.
>
> Argo is a secure VM-to-VM communication mechanism based on hypercalls
> [1]. It is a good fit for Dom0less setups because Argo is lightweight
> and doesn't make use of PV frontends, backends, and xenstore. Thus, it
> is easier to enable in simple Dom0less setups where VMs are booting in
> parallel, backends cannot be assumed to be already up and running, and
> the goal is just to get two VMs to talk to each others.
>

Excellent - I am very happy to see interest in Argo for this


>
> Argo makes use of event channels for notifications. Dom0less VMs don't
> have event channels support yet, although it is work-in-progress. The
> attached patch for Linux (not ready for upstreaming) enables the
> necessary event channels initialization.
>
> In addition, you also need to add the following to the DomU device tree
> (see xen/arch/arm/domain_build.c:prepare_dtb_domU):
>
> event-channel {
>         compatible = "xen,xen-4.15", "xen,xen";
>         interrupts = <0x01 0x0f 0xf08>;
>         interrupt-parent = <0xfde8>;
> };
>
>
> The final pieces to enable Argo are:
> - CONFIG_ARGO in the Xen build
> - the Argo Linux kernel module
>   https://github.com/OpenXT/linux-xen-argo


This is an important pointer. The argo-linux directory in the repo contains
the source for the current best Linux Argo driver, which has been in use
and actively maintained in OpenXT for an extended period. It is derived
from the original XenClient v4v driver -- Nicolas Tsirakis did great work
porting all the OpenXT v4v software over to this Argo driver and the
libargo userspace component when we migrated. Jason Andryuk has recently
just completed some strong modernization work that advances this Argo
driver's use of Linux synchronization primitives.

For the curious, an alternative Linux Argo driver written by Eric Chanudet
is in also the same repository in the vsock-argo directory: it explores a
different approach for Linux to use the Argo primitives under the Vsock
address family to allow socket communication between Xen domains. A third
code base that could be of interest for developing a new Linux driver for
Argo would be a port of the HP uxen hypervisor's v4v driver, which has a
good modular structure, to Argo on Xen. The code for the starting point
(for a fairly adventurous project) is here:
https://github.com/uxen-virt/uxen/tree/ascara/vm-support/linux


> - the Yocto recipe was based on
>   https://github.com/dozylynx/meta-argo-linux


Ack - this Yocto layer shows recipes for the several components involved
with building an Argo-enabled system.


>
> - pass dom0less_domU to the domU kernel command line


>
> Then you are good to go for using Argo in your Dom0less VMs, e.g.:
>
> From receiver domain: streamTest -domid 5 -port 8022 -file test.txt
> -receive
> From sender domain: streamTest -domid 4 -port 8022 -file test.txt -send
> -connect
>
> Many thanks to Alec Kwapis from DornerWorks for all the information and
> the patch!
>

Thanks, Alec, for exploring Argo with dom0less and for developing the patch
-- excellent work! I am certain that it will be helpful.

Christopher



>
> Cheers,
>
> Stefano
>
>
> [1]
> https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen

--0000000000003d350c05cc7d6259
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Mon, Sep 13, 2021 at 3:41 PM Stefano S=
tabellini &lt;<a href=3D"mailto:sstabellini@kernel.org">sstabellini@kernel.=
org</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote class=3D=
"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2=
04,204,204);padding-left:1ex">Hi all,<br>
<br>
This email is for anybody interested in using Argo with Dom0less setups<br>
for domain-to-domain communications.<br>
<br>
Argo is a secure VM-to-VM communication mechanism based on hypercalls<br>
[1]. It is a good fit for Dom0less setups because Argo is lightweight<br>
and doesn&#39;t make use of PV frontends, backends, and xenstore. Thus, it<=
br>
is easier to enable in simple Dom0less setups where VMs are booting in<br>
parallel, backends cannot be assumed to be already up and running, and<br>
the goal is just to get two VMs to talk to each others.<br></blockquote><di=
v><br></div><div>Excellent - I am very happy to see interest in Argo for th=
is<br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">
<br>
Argo makes use of event channels for notifications. Dom0less VMs don&#39;t<=
br>
have event channels support yet, although it is work-in-progress. The<br>
attached patch for Linux (not ready for upstreaming) enables the<br>
necessary event channels initialization.<br>
<br>
In addition, you also need to add the following to the DomU device tree<br>
(see xen/arch/arm/domain_build.c:prepare_dtb_domU):<br>
<br>
event-channel {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 compatible =3D &quot;xen,xen-4.15&quot;, &quot;=
xen,xen&quot;;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 interrupts =3D &lt;0x01 0x0f 0xf08&gt;;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 interrupt-parent =3D &lt;0xfde8&gt;;<br>
};<br>
<br>
<br>
The final pieces to enable Argo are:<br>
- CONFIG_ARGO in the Xen build<br>
- the Argo Linux kernel module<br>
=C2=A0 <a href=3D"https://github.com/OpenXT/linux-xen-argo" rel=3D"noreferr=
er" target=3D"_blank">https://github.com/OpenXT/linux-xen-argo</a></blockqu=
ote><div><br></div><div>This is an important pointer. The argo-linux direct=
ory in the repo contains the source for the current best Linux Argo driver,=
 which has been in=C2=A0use and actively maintained in OpenXT for an extend=
ed period. It is derived from the original XenClient v4v driver -- Nicolas =
Tsirakis did great work porting all the OpenXT v4v software over to this Ar=
go driver and the libargo userspace component when we migrated. Jason Andry=
uk has recently just completed some strong modernization work that advances=
 this Argo driver&#39;s use of Linux synchronization primitives.=C2=A0</div=
><div><br></div><div>For the curious, an alternative Linux Argo driver writ=
ten by Eric Chanudet is in also the same repository in the vsock-argo direc=
tory: it explores a different approach for Linux to use the Argo primitives=
 under the Vsock address family to allow socket communication between Xen d=
omains. A third code base that could be of interest for developing a new Li=
nux driver for Argo would be a port of the HP uxen hypervisor&#39;s=C2=A0v4=
v driver, which has a good modular structure, to Argo on Xen. The code for =
the starting point (for a fairly adventurous project) is here:</div><div><a=
 href=3D"https://github.com/uxen-virt/uxen/tree/ascara/vm-support/linux">ht=
tps://github.com/uxen-virt/uxen/tree/ascara/vm-support/linux</a><br></div><=
div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px=
 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
- the Yocto recipe was based on<br>
=C2=A0 <a href=3D"https://github.com/dozylynx/meta-argo-linux" rel=3D"noref=
errer" target=3D"_blank">https://github.com/dozylynx/meta-argo-linux</a></b=
lockquote><div><br></div><div><div>Ack - this Yocto layer shows recipes for=
 the several components involved with building an Argo-enabled system.</div=
><div>=C2=A0<br></div></div><blockquote class=3D"gmail_quote" style=3D"marg=
in:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1e=
x"><br>
- pass dom0less_domU to the domU kernel command line=C2=A0</blockquote><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
:1px solid rgb(204,204,204);padding-left:1ex">
<br>
<br>
Then you are good to go for using Argo in your Dom0less VMs, e.g.:<br>
<br>
>From receiver domain: streamTest -domid 5 -port 8022 -file test.txt -receiv=
e <br>
>From sender domain: streamTest -domid 4 -port 8022 -file test.txt -send -co=
nnect<br>
<br>
Many thanks to Alec Kwapis from DornerWorks for all the information and<br>
the patch!<br></blockquote><div><br></div><div>Thanks, Alec, for exploring =
Argo with dom0less=C2=A0and for developing the patch -- excellent work! I a=
m certain that it will be helpful.</div><div><br></div><div>Christopher</di=
v><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">
<br>
Cheers,<br>
<br>
Stefano<br>
<br>
<br>
[1] <a href=3D"https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_E=
xchange_(HMX)_for_Xen" rel=3D"noreferrer" target=3D"_blank">https://wiki.xe=
nproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen</a></blo=
ckquote></div></div>

--0000000000003d350c05cc7d6259--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 09:20:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 09:20:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191595.341673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbxH-00063k-MM; Tue, 21 Sep 2021 09:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191595.341673; Tue, 21 Sep 2021 09:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSbxH-00063d-Ii; Tue, 21 Sep 2021 09:20:47 +0000
Received: by outflank-mailman (input) for mailman id 191595;
 Tue, 21 Sep 2021 09:20:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s/tM=OL=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSbxG-00063X-9R
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 09:20:46 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a142143b-1ac9-4c25-9624-9f2b8565b884;
 Tue, 21 Sep 2021 09:20:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a142143b-1ac9-4c25-9624-9f2b8565b884
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632216044;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=bpeOjIBrupabDN+FdD3y+xN/an7SLfG3FXIiQScb2a8=;
  b=SlB8A/FOwQZez3wg6+Jjv1oe+9fsl++db4XIQYvQ3Tir1rp58vIMlcZD
   nfdhbOeGz8roODtvlv4PhxbWvqY9S4mzZgkjNI6rHyf95XJ3J3dBO9eRS
   4m3PD6xYlbTnDOEUfy+LdH2tWH4rAly13qXfqt25atA49wKCsyzDeFJ2P
   w=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: /qXoLdOR4W/wHl2Rb1URGAWa7o5FsW6irWPgfOKMTKpDv8fyrHOMs/aA/ZYmyb6lx9tETI27E4
 w5ssWk9Qhv/7aBZm0xQ4PrkaaxUL4S6uPB3LcC6PUNttaJI3I6VorTToZoOj4aSEHmezXE+OlO
 QVpONEyhTilBpE9BnQGrMqxBHkfLnBU4jTrFuEEgz8Ckgvs3PPLDhm6MeaGW0mtGojwFF2qSlr
 sy6IhFC5b8UHqVQQxG/D/LHrio3XSuJywa6RAULYMhIXLhEpltKravnTIuEVAmPXz+nc0NcpC8
 luBe/JoX+NyoWNXEMGQYIdNf
X-SBRS: 5.1
X-MesageID: 54994351
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:PKlmnq9+P5hMspkFoe7bDrUDbnmTJUtcMsCJ2f8bNWPcYEJGY0x3x
 mFLD23Ub/nZM2akLt0ia4zn9RkHupODyoViSQc/rS48E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j29Yx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhqm
 MsVlLKUTjwQEYjQp9oYQhh7NyNxaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFgGZh2psTRJ4yY
 eInRxVVVxribyZgP2g3Eb4hgde2hlPwJmgwRFW9+vNsvjm7IBZK+LrwNNvYfPSaSMMTmVyXz
 krd5HjwCBweMN2ZyBKG/2iqi+uJmjn0MKoNEJWo+/gsh0ecrkQRAhALUVqwodGil1WzHdlYL
 iQ86ico6KQ/6kGvZt38RAGj5m6JuAYGXNhdGPF87xuCooLV/ASxFmUCViRGatEtqIkxXzNC6
 7OSt4q3X3o16uTTEC/DsOfPxd+vBcQLBXRSe3clYy8C2ODMhZ0MsBfeT/VHEqHg27UZBgrML
 yC2QDkW3utI1J5QhvTjpzgrkBr3+cOYFVddChH/Gzv/t1InPtbNi5mAtACDhcusOrp1WbVoU
 JIsoMGY8OlGJpWEjiXlrA4lTezxuqrt3NExhzdS83gdG9aFoCXLkWN4umgWyKJV3iEsI2SBX
 aMrkVkNjKK/xVPzBUONX25UNyjN5fO6fekJq9iONoYeCnSPXFbfoUmCmnJ8L0iyyRNxwMnTy
 L+wcNq2DGZyNEiU5GPtHI8gPUsQ7nlmnwv7HMmjpzz+iOb2TCPFGN8tbQrVBshkvfzsnekg2
 4sGXyd8404EC7OWj+i+2dN7EG3m2lBhVMiq9JQIJrDcSuekcUl4Y8LsLXoaU9UNt4xel/vS/
 2H7XUldyVHlgmbAJxnMYXdmAI4Dl74lxZ7iFSBzb1uuxVY5ZoOjsPUWe5ctJOF1/+1/1/9kC
 fICfpzYUPhITz3G/RUbbIX889M+JEj621rWMnr3eiU7cr5hWxfNpo3ucDzw+XRcFSGwr8Y//
 eGtj1uJXZoZSg1+J8/Kc/bznUiptH0QlbsqDUvFK9VeYmv2941uJ3Cjh/M7OZhUex7C2iGbx
 0CdBhJB/bvBpIo88d/og6GYrtj2T7siTxQCR2SCtOS4LyjX+Gan0LRsaufQcGCPTn7w9YWje
 f5Rk6P2PsoYkQsYqIF7Cbtqk/4zvoO9u79Aww14N3zXdFD3WKh4K3yL0MQT5K1AwrhV5Vm/V
 k6Vo4QIPLyIPIXuEUILJRpjZeOGjKlGlj7X5PUzAUP7+C4oo+bXDRQMZ0GB2H5HMb94EII52
 uNw6scZ5ju2hgcuLtvb3Dtf8H6BLyBYXqgq3n3A7FQHVub/Jol+XKHh
IronPort-HdrOrdr: A9a23:gt9ZBqvNCa9X2umRZCbHrxmX7skC54Mji2hC6mlwRA09TyXGra
 +TdaUguSMc1gx9ZJhBo7G90KnpewK6yXdQ2/hqAV7EZniahILIFvAY0WKG+VPd8kLFh4xgPM
 tbAs1D4ZjLfCRHZKXBkXiF+rQbsaC6GcmT7I+0pRcdLnAbV0gj1XYANu/yKDwJeOAsP+teKH
 Pz3Lsim9L2Ek5nEfhTS0N1EtTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqZmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87ysIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXoHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1nackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMG9yV0qp/FWH/ebcG0jaRny9Mww/U42uonZrdUlCvgglLJd1pAZGyHo/I6M0rt
 gsfJ4Y0o2n46ctHNZA7dw6MLmK41r2MGTx2VKpUCfa/Z48SgfwQr7Mkf4IDbKRCdY1JKVbou
 W3bLofjx9qR37T
X-IronPort-AV: E=Sophos;i="5.85,310,1624334400"; 
   d="scan'208";a="54994351"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L2GYZiNqZzwKRKQITxpa7xdO/P3pive+DVQf5is9vL5ud2nUj/rDIHxvRgP5OIXiEnJRupNHUOICh7j3G2/NYZrT/CKPQBKIr9rMB+MBhA/DXxaiJcHxw1A11FhAQd6tjgQ1LqEvxKxVVcQ3yA+cossEQsKYLCYpGeuVseSNkVhBMVVxLSpUCgMHik5h+X4klTlD3l8RhNh3tptbWX+gSkjxu9/LSbtqmKbH5hOo2dNV7bNJNP48uE9y93krCzyjVve6FI0vMSNK/q4Lso5Q01rdMgUJpQ48BNfRFJmhRqphCmImTT4AxtZadU4fTsIq1JJhiPCo+G5HmL6fZCy+dg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Mw9V19GYeYg80+2iPoM1iFhfs5y4YO6kDw5nFI2ANSE=;
 b=No+3L/sNjSdimnLZDlpJeTmoSWfOm+hkaCYqR+4Ww4K7osQcW8iJAv+DC/XtA1N6HNgGR7+fONd98pdvQS9OfWeRq+kZjkRrv0sFdPETkIn0nwMy5Fj8ptu46zgqCsUfW2AUBMSuAzDtA7XhbIbBgRyEchexECjlkdvnTMOC16hChXECv133in+KTpDagj5XYARcxadYDYD63l7i4I7l60v8iOY2j5h7gZcRgO9HJLjsxRAqX69eBzRpwjbf9PhXrQO2v4a9oq8EV3PO+4oCvvqkCMf+WpofY+ZMJMRQde5IBJJW+ENu/sOj0d5dgLu4SwUc9hB0CXp1dm39Vdbyeg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Mw9V19GYeYg80+2iPoM1iFhfs5y4YO6kDw5nFI2ANSE=;
 b=kUqKESlP9eefIfBV4MIiFyctieTZ+BFozAtlUG6b8NHo109D89bzxIUGkYBmAQKpnzxmkAAPlqXhTw5vNhwHKDHfRjriAtsCRcE9OoFJvkwehLfAGmjoDV6wjxkpynMHfik1lVIfbEP/VBUwc90/sn0wx6wsilDoDirZ4W+Am4A=
Date: Tue, 21 Sep 2021 11:20:28 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
Message-ID: <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
X-ClientProxiedBy: LO4P123CA0284.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:195::19) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: db9ffc52-37ef-484a-8d75-08d97ce110e5
X-MS-TrafficTypeDiagnostic: DM5PR03MB3370:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB3370E17DC5D2E041270C8C278FA19@DM5PR03MB3370.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: zEkHvyodOkIoeeLPCW+hzibYu9L946FDG7sU0I01Y1Fh9IsQMBcKlDokHETP1ZYs9A7Z7551yphyKF9hrow3lj7Un2vmarbxFwcZMLiGHqVxHDrUPw8kx92czRge5BSdrlC9yDUw3k4BrLtvJsd5ccACBv8oRonashq+idFB4yVBpybOUMvXLxQEWNbzDOn2zY692l4A+edHICUvSnaHue5yAq26Ws2bGVZ98+T6ha6ide7U5xoXwMNcDBRvaMhKR87nCuylpsfNRDL+BT1YoRa/N7eSHF91Dp0Cice020MAquc9ydr+D6hKP0/LMzi83PgaTD8Ddc2eZ43bpm3RFsMxAzySZfcw1pV12J3tjttRhvI/lZsbUS7KdlIhWmLwjmR4AKjDKdosPQKpdeWZ1VSyyYYHDuFPZqiG383yWY6hHZB5ExmTn5ATyGShBoUqMFJgg5gyOHRbk+MT1gcfzn4DhApYip9W1FlLnZmnhFM29Mk7hnC+1/NSz9qJn2rxFCIJxbOlTAbyoC9V1ZA6kXAHxA9pLGFMXl++1299tfD21iWwAaVn8iHhvef5lXALbzfjI/DaR4AHqRhiHp6TfAd4jgLCy3btz58kdjvsjXOrtZodcvFicI04nCUU/4RbCf5iKl1YTjlEHdsdcjxd3w==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(66476007)(186003)(6496006)(83380400001)(6666004)(8676002)(8936002)(54906003)(66556008)(6486002)(5660300002)(66946007)(316002)(2906002)(956004)(86362001)(4326008)(85182001)(9686003)(508600001)(38100700002)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?T2k1NzFYN092Zmg3c0ViSnM5SzZEQXh4b0RFN2tkSXpNWlpwRENNUm0va1hh?=
 =?utf-8?B?WWl0UFlaODgxS284RUNIbmlnKzVkeHNFNDRsM3ZEQ1BPTlJ4TGdXSUkrQkMz?=
 =?utf-8?B?STN1Y2RTMEg3ek91dXRYR2ZTcmdoVERvWEdQSkJNMXNUZzQ5c2Nyd2JzZkpN?=
 =?utf-8?B?TlliR0w1eDlGY3VmY1dYTS9vWnhGNGtlSmk2ZmhZbUQySDBqSGEvcTVlUSs1?=
 =?utf-8?B?QzVYZ1p2ZWc0MzFrVGIxQVc4TU55OXVtM0RkdW5Ua0Znd1lCVU1wSVFTZzRv?=
 =?utf-8?B?RUM3OEh5QzFDeUNFWTdaSndDK1dleHBuOUkyc1hsYkdUcjI5OWxwOTdKS2ha?=
 =?utf-8?B?bkxhajY1VUV3enl3dWVMbVZSZXRiTmlnWkxXQ1ZaN0xCWDZ2OTFVcThENGM4?=
 =?utf-8?B?Z3VleFErc2crQXh3Y0oxN0NQcVJybityV0k5Z3VST0RvQUh6UmswWUZvYUN1?=
 =?utf-8?B?WXgyVUZMdEdWY2diT2dZZGxOWEJ4R3FTUUQ2VzhIaUFEL2pWSWlSbTNsRXBt?=
 =?utf-8?B?RElHMjVENUxpU3lYYmkrUTRoTlg5MENSYktIbkJLYTVqQXJkZE12WEE4eEZJ?=
 =?utf-8?B?QzFDMTkwTVB0QXNJbU5Ray9zellDcDB2dmZsK1RDdWlsOTlIeVI4RG91REVH?=
 =?utf-8?B?eUFmOURUUWZSMmRTSGlBUlJPcVlwTFVaQVp0ZWpQZ3RSeEYveWJCdEFmSk1S?=
 =?utf-8?B?TzUzVS9EYzl5VGgxQWtVazhpS0ZSOExsZXAxaTdLMkk1U29zWVpocENTZkxt?=
 =?utf-8?B?eDhUQnZwNmlrMytxNStLMmljcEtTWTJnTCszSzNRQmtRd2lsTFdIU3dlSzVz?=
 =?utf-8?B?ZVZkdGpPWVovWjBSUmdjZGR0cjhOT3F1Vlc0alhuZ1NONzNFeHoxNW9Gd3I1?=
 =?utf-8?B?bldsZlp3Nm1EWnY4Q1pObFdGbE83Yk00NzRzdDY3K0EybEkyQ1hrRFZud015?=
 =?utf-8?B?ME9xRGhjc3FOZ1JvUFp0eDNaNWJ5RGt4b2xOMTBqd2J5UGlyeWRhalhnNmZC?=
 =?utf-8?B?S2tHanY5ZnhvK1dpTHpReVcweGdTTFVVVHZoR3B6RFlaUHcwUmduR2hZQXlr?=
 =?utf-8?B?K2RwNWFkYTMzQkFwTFlGZXlHQnE2MVJDd21sOHU5Ui80M3FFL2hoRHJ4MG0z?=
 =?utf-8?B?cUlRT1RoaUVrYTVITkp5QUtPMHIzR1gzdkV6ZnFpd0NxTzZZTUFUcHRCRmFm?=
 =?utf-8?B?ODQ1bndoZ0FySHhNbVJFUzkvY1RCaWQ1ZGh3ZVJOTUFCTlZtekZIVngzRUtX?=
 =?utf-8?B?RjJiUEszQVRXYmpkRnVmNFBHdDFKRkFxa1VWajFJZ1hvRS9RS1hCaXRNRUVv?=
 =?utf-8?B?YURBZmhWN2d1cGYxR1dMaXc4UTJSWEZ3TkpaSk5qQTdIRDFBUnZDZDc5SHQv?=
 =?utf-8?B?R0V6SGoxSFF1Y1RFam1VSVdLRG53MHpWQXpjanROeVVJTXJTc2lQYXlWZzZQ?=
 =?utf-8?B?TWhNeCtUOGZxSUxmZ0xXR3NkRVVQK2NaTHBDNStBSC9PZXZmWnhvNzJMQkEy?=
 =?utf-8?B?UjBCYzZsWUJQbkt2ZDh1MUZwOEZ2V1Z6OEdwM05qZ1Q4T05sMFNORnJVOTVW?=
 =?utf-8?B?RVFVdUNGRlM3d2Izc1dwajZDSEVWVjdCSEdoWWdnNml1ZE1zWUdoWVM1WUo0?=
 =?utf-8?B?ODF0ZUlYUm5wMUtYUW5XNjZQVXRMU0xpVTE2OTRjaldwUHh4VmQ4Z2tpU1VW?=
 =?utf-8?B?aitoZ0x0Q041aENxSk9XSGJEaCsxODRUUXNjcEVYS2xjenI1TlBzY3RWK0hU?=
 =?utf-8?Q?TTAp2vEKnj41PHeFsjrwgnfUsoMrxlerVeeMB7p?=
X-MS-Exchange-CrossTenant-Network-Message-Id: db9ffc52-37ef-484a-8d75-08d97ce110e5
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 09:20:34.4134
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Y7aDDYz55WhUcQd6+TvL/UsQRiuLY6zWSKi0DEzAYU+We+K3ag3XQ0+sql7r6v1gqWeeR+HPZtNLyqZRfyXzIg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3370
X-OriginatorOrg: citrix.com

On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
> The function may fail; it is not correct to indicate "success" in this
> case up the call stack. Mark the function must-check to prove all
> cases have been caught (and no new ones will get introduced).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Just a couple of comments, as we now handle errors in some placs where
we didn't before.

> ---
> In the grant-transfer case it is not really clear to me whether we can
> stick to setting GTF_transfer_completed in the error case. Since a guest
> may spin-wait for the flag to become set, simply not setting the flag is
> not an option either. I was wondering whether we may want to slightly
> alter (extend) the ABI and allow for a GTF_transfer_committed ->
> GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
> at the same time as setting GTF_transfer_completed).
> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -2394,7 +2394,7 @@ gnttab_transfer(
>          {
>              grant_entry_v1_t *sha = &shared_entry_v1(e->grant_table, gop.ref);
>  
> -            guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
> +            rc = guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
>              if ( !paging_mode_translate(e) )
>                  sha->frame = mfn_x(mfn);
>          }
> @@ -2402,7 +2402,7 @@ gnttab_transfer(
>          {
>              grant_entry_v2_t *sha = &shared_entry_v2(e->grant_table, gop.ref);
>  
> -            guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
> +            rc = guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, 0);
>              if ( !paging_mode_translate(e) )
>                  sha->full_page.frame = mfn_x(mfn);

Is it fine to set the frame even if updating the physmap failed?

>          }
> @@ -2415,7 +2415,7 @@ gnttab_transfer(
>  
>          rcu_unlock_domain(e);
>  
> -        gop.status = GNTST_okay;
> +        gop.status = rc ? GNTST_general_error : GNTST_okay;
>  
>      copyback:
>          if ( unlikely(__copy_field_to_guest(uop, &gop, status)) )
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -268,7 +268,8 @@ static void populate_physmap(struct memo
>                  mfn = page_to_mfn(page);
>              }
>  
> -            guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order);
> +            if ( guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order) )
> +                goto out;
>  
>              if ( !paging_mode_translate(d) &&
>                   /* Inform the domain of the new page's machine address. */
> @@ -765,8 +766,8 @@ static long memory_exchange(XEN_GUEST_HA
>              }
>  
>              mfn = page_to_mfn(page);
> -            guest_physmap_add_page(d, _gfn(gpfn), mfn,
> -                                   exch.out.extent_order);
> +            rc = guest_physmap_add_page(d, _gfn(gpfn), mfn,
> +                                        exch.out.extent_order) ?: rc;
>              if ( !paging_mode_translate(d) &&
>                   __copy_mfn_to_guest_offset(exch.out.extent_start,

Would it be worth it setting the mfn on the guest output to
INVALID_MFN or some such if the physmap addition failed?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 09:39:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 09:39:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191603.341684 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScEp-0007rf-A3; Tue, 21 Sep 2021 09:38:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191603.341684; Tue, 21 Sep 2021 09:38:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScEp-0007rY-6c; Tue, 21 Sep 2021 09:38:55 +0000
Received: by outflank-mailman (input) for mailman id 191603;
 Tue, 21 Sep 2021 09:38:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nbRU=OL=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mScEn-0007rS-NL
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 09:38:53 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 921f87ad-9d7e-4699-a595-d754c60ef4fd;
 Tue, 21 Sep 2021 09:38:50 +0000 (UTC)
Received: from FR0P281CA0056.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::6) by
 AM6PR08MB3029.eurprd08.prod.outlook.com (2603:10a6:209:48::19) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14; Tue, 21 Sep 2021 09:38:47 +0000
Received: from VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:d10:49:cafe::7c) by FR0P281CA0056.outlook.office365.com
 (2603:10a6:d10:49::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Tue, 21 Sep 2021 09:38:47 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT051.mail.protection.outlook.com (10.152.19.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 09:38:47 +0000
Received: ("Tessian outbound 2e7020cc8961:v103");
 Tue, 21 Sep 2021 09:38:46 +0000
Received: from 2c9da993f3b6.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 85545CAA-61B3-4962-81F6-0903A1D03B47.1; 
 Tue, 21 Sep 2021 09:38:35 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2c9da993f3b6.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 21 Sep 2021 09:38:35 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6639.eurprd08.prod.outlook.com (2603:10a6:102:154::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 09:38:32 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%8]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 09:38:32 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0336.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18c::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 09:38:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 921f87ad-9d7e-4699-a595-d754c60ef4fd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xb84FUmQ0EMzccCYt/0WDSY++7GO+Xayqs8qAkuNc38=;
 b=ZCR2LOSO2U+p4noifOwMDSQ75U2Zx+2c6/IZdOiD3rHiL0A6HXepTWuCnsNwAoin0oDDqOCIT6Q4pg2xX7UNu9BMOiZcE4fsnHhcqzdUTTDFb6awdOnh0xsXTRo6kumspcyr8uJKzhpcx5JICkmN4sL486X2UTWuVoj9Ov9K8Ns=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3236b2941184db0a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RcWgEcO3Le6+kqHEBLauYQbWdd55GgHPARDrwoB0GPs7i+CZ/QmKxhAPbc6FY256ZizOPKALzisJxL6owThRBPTwXHdChmA4ecdgxCoTH64E7CRNI/EFFSC9h/CyO6tv0OAp7h/xqhofVfeypP3Q/lLqqBiSqLy3jRPBrdaw7z0YjP/Erd3I7VVR3SFOiONXn4GOZJGujJal6vl2q2shAcvv1cvOFiik3M7HgimeswYPrCvaKSUrUUvKkj29YMcFT9cWDKTRO0qJRm3AGXtejfclaNozUh7RdFlm7ivQtnbCak4Sa4kty8s5DbeymuDCpM49c+LX5O/HECWh0KsOpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xb84FUmQ0EMzccCYt/0WDSY++7GO+Xayqs8qAkuNc38=;
 b=Bp7it2I4Hjortl26Bv1DeF6a5jaHPhdSA/rEA7/ZxZs35JpS3zrJfCVqSK/xeR/sklPNp1MzSGmma1+S+YEdIqSnRjn1m0lfTDD50XFgaK3PkAq4umFhZ9E7Wg7jL8w4ohgpCnJc83WSTzprLKUTKwrPwEnlInTnrfBUJ/8fOOFb6r8nlnkaeEkeifjB/uLKqLH8W9vQTyYaWXMjMuVjQ/ZB/MUik3W52RnNRy7fy5ok5yiRJTKNBHaDhFmuFHXqhQMnm8+CG+SZesJ+/WJCHKQwq9jWAv2vdkUHQS3R2QOR54696yRj9WbK18iZvP5NZgF9Yj/Kl04DAUfJi4eWyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xb84FUmQ0EMzccCYt/0WDSY++7GO+Xayqs8qAkuNc38=;
 b=ZCR2LOSO2U+p4noifOwMDSQ75U2Zx+2c6/IZdOiD3rHiL0A6HXepTWuCnsNwAoin0oDDqOCIT6Q4pg2xX7UNu9BMOiZcE4fsnHhcqzdUTTDFb6awdOnh0xsXTRo6kumspcyr8uJKzhpcx5JICkmN4sL486X2UTWuVoj9Ov9K8Ns=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <Luca.Fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109171516270.21985@sstabellini-ThinkPad-T480s>
Date: Tue, 21 Sep 2021 10:38:25 +0100
Cc: Jan Beulich <jbeulich@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <18531A94-ECF7-4840-A515-B161A4ABDEF4@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
 <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
 <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
 <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com>
 <alpine.DEB.2.21.2109171516270.21985@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0336.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::17) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9f89ede2-f185-4058-b2a4-08d97ce39c7c
X-MS-TrafficTypeDiagnostic: PAXPR08MB6639:|AM6PR08MB3029:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB30298B6A88E8FB8952AFFB2DE4A19@AM6PR08MB3029.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4KiP46nYiqWVU1dMVzxW2bc9Va/BuAWQ+xn5wukEtIw47zmWc+La0bsEZEC19O9C2QobY5UiZ1bZ5tocvs6gKV2AsqARHQrU1jGmobrjCkJ7snO+wZ87IwCt3d7K6gqdxmNc1ucfhkS9ec89vmutSzZGainiGWPPYrKF1OD1DrOSVKda5yqNCxj+Sbk00Bwwkxi/b7AaZoXAFJnEy2m/ob54R9eqOd5NfCvJ+MuBzsh0bXGQD9dFB4JlowOi9eTH86BKNkkIipJVQzQw09dWTGLwBwmkkSxCpwwYiVmeXzMhBWUAfPfwZzW70gO+CkB8AFYESWl9kTw81/L2gvuH70cR9NeRRIIt9GabpIJ1cz9W9ewdpI8kX3ZqXuAYfhA6dU3rl+1fWRvqHm+dMTebO8/2+gaqJ16XhDROdpQ4xJqBiDUAviwUBnBzQ86PgJIEP27B7fgIs6wjeHw5xot9QmyrI1Ko3O+lvFHcCOA7Z3Gc8lT8xMyYAaxYbBeqgIZsHOqr4Fy1csRgfu8EiEnx2hAoJCqdLRfwq3i6zYvRcmWZ5//Sb8cS7OOm4wAs68l655wXehd/0NmZM9utE00g6fQ9om0kUY9qM1j4ldQyND2I+8cNoiVdjE9I0oSrd+nuQax70u7v4E8R/AoGI8gHTk3RyCr7LSDs/pWduKXXnrIahqJ281SWuh2/dxjcRnhWWzd8wDFk8VpR5t/SttT/G73aQKyV0hxkQeSXCdex1WOUWAeQ0HKoyD5pewVn5NN2
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(508600001)(186003)(33656002)(5660300002)(2616005)(52116002)(26005)(83380400001)(30864003)(6506007)(38350700002)(8676002)(86362001)(53546011)(4326008)(8936002)(6486002)(956004)(38100700002)(66556008)(54906003)(6512007)(2906002)(316002)(66476007)(6666004)(66946007)(36756003)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6639
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0cd08a20-67cd-496f-5e2a-08d97ce39353
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jl12RhDsLClpt38mN+/r5J4qFUqz64cjqgGdNjJYco82CuIK7BR/yUPfuEwsqt9U3UYMh3knfIskzaAGt0e4/A8EujR6oRGPpqUR31gCQ595dpQV36G2VtAp2gjGRUp/AhtOA3+aQ7Kp+QMdfMpPtrrz2UvQ2od1XUV4g7XRHVlJ4Zjt8qMrUi7aunXO2TobqVgrhQC2EtCJIyLg1YKR+vufm8FAvUgD6DMaftHFbEo/ErcdaBt6uo0Yq07cmhoZz3M2TQg3qf8mbLejOBno875/e42OQ0GvZ/bvNl2cnx6MN4j3326/QkBKStLqavWbgwDePCE7wEFqPpWabwJVdZAqeX3e2mT/jjFtDwOn6sgp/YU5tcMp+Np7HTY+7kaIbMj/2LFPQ2yc0HtW3amFL7dDUqWu38TBQmjqmnlyXTf9deYYPrFjK2xlj30mKFrQGGtB9hVN1318yWq7AVwhrglCVM+FS6cnpZmhbNYYWnKJ/2ozDNtxwGKbmGdux4Ik4Otjv+CCGyWSJ6wLsWtWv6AK6FS/uFAx8I4t+YFub/Y+mk9L/0wiDSZsfEYykgFwOGVkSXJN9qQ+hsaNKGR+yCf5WwpCnDLvdpyJiXSJgAzdJktelMr8lb46yRj8BHyQh+xQHgnpaEV5/ew8SX5QRMbcbLH3JY0F4rybOYcNgY71CHFN0e8Kr6VFYJizwWNSmrAjAG0dqz1tgkedtnhDxifZqac1SF9/8mGXIPT3mow=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(6666004)(36860700001)(956004)(30864003)(4326008)(70586007)(2616005)(70206006)(336012)(2906002)(81166007)(316002)(6486002)(8676002)(53546011)(356005)(186003)(6506007)(5660300002)(36756003)(54906003)(82310400003)(6862004)(508600001)(26005)(33656002)(47076005)(83380400001)(6512007)(8936002)(86362001)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 09:38:47.2974
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f89ede2-f185-4058-b2a4-08d97ce39c7c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3029



> On 17 Sep 2021, at 23:33, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Fri, 17 Sep 2021, Luca Fancellu wrote:
>>> On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>>>=20
>>> On Thu, 16 Sep 2021, Jan Beulich wrote:
>>>> On 16.09.2021 17:07, Luca Fancellu wrote:
>>>>> I explain here my understanding on dom0less, this feature is used to =
start domUs at
>>>>> Xen boot in parallel, the name is misleading but it doesn=E2=80=99t r=
equire dom0 to be absent.
>>>>>=20
>>>>> So if you have a dom0 kernel embed in the image, it's completely fine=
 to start it and it=20
>>>>> doesn=E2=80=99t have to be skipped.
>>>>>=20
>>>>> Here the possible user cases:
>>>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
>>>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or emb=
edded in Xen image, domUs on DT]
>>>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen =
image) and domUs on DT]
>>>>=20
>>>> If that's the intention - fine. Stefano?
>>>=20
>>=20
>> Hi Stefano,
>>=20
>>> What do you mean by dom0 modules embedded in the Xen image? I am not
>>> familiar with it, but I imagine it doesn't involve any multiboot,module
>>> nodes in device tree, right?
>>>=20
>>> Putting aside "dom0 modules embedded in Xen image" that I didn't fully
>>> understand, there are three ways to load Dom0:
>>>=20
>>> - dom0 kernel (and ramdisk, optional) on xen.cfg
>>> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" pr=
operty
>>> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,bi=
nary" property
>>=20
>> True for the #1 and #2, the last one is not implemented. The uefi,binary=
 property
>> for now is only used to load domU modules.
>=20
> Yeah, it is no problem that is not currently implemented, but from a
> device tree binding / efi interface perspective it should be possible.
>=20
>=20
>>> Then, the other use cases are:
>>>=20
>>> - true dom0less, domUs on device tree using the "reg" property
>>> - true dom0less, domUs on device tree using the "uefi,binary" property
>>>=20
>>> And of course all the possible combinations between Dom0 and DomU
>>> loading.
>>>=20
>>>=20
>>> Currently, patch #1 checks for the presence of a Dom0 kernel node and, =
if
>>> present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
>>> node is *not* present, efi_arch_use_config_file returns true.
>>>=20
>>> However, this could be a true dom0less configuration without any Dom0
>>> kernel. If so, you might not want to load xen.cfg at all because it is
>>> not needed. In a true dom0less configuration, we probably want
>>> efi_arch_use_config_file to return false.
>>=20
>> In a true dom0less configuration we might need to read xen.cfg to retrie=
ve the
>> Xen command line,=20
>=20
> The Xen command line could also be on device tree
> (/chosen/xen,xen-bootargs).
>=20
>=20
>> but following the actual implementation of the common code
>> there is more. I=E2=80=99m going to explain.
>>=20
>> What efi_arch_use_config_file really does is not only choosing to read x=
en.cfg
>> or not. Following the common code (xen/common/efi/boot.c) and what its r=
esult activate
>> along the path, it basically decides if the UEFI stub is used as a loade=
r from filesystem
>> or not. We need the UEFI stub as a loader to be 100% UEFI and load our m=
odules.
>>=20
>> The original check basically says =E2=80=9Cif there are multiboot,module=
 in the DT, then some
>> bootloader has loaded in memory the required modules so I=E2=80=99m not =
gonna load anything
>> from the filesystem because I assume it puts everything in place for me =
to boot.=E2=80=9D
>=20
> OK, I am following. It looks like this is the source of the issue.
>=20
>=20
>>> From misc/efi.txt:
>> When booted as an EFI application, Xen requires a configuration file as =
described below unless a bootloader,
>> such as GRUB, has loaded the modules and describes them in the device tr=
ee provided to Xen. If a bootloader
>> provides a device tree containing modules then any configuration files a=
re ignored, and the bootloader is
>> responsible for populating all relevant device tree nodes.
>>=20
>> What I=E2=80=99m doing in patch #1 is restricting that check to just the=
 multiboot,module that are
>> Dom0 module, why? Because with the introduction of dom0less we need to s=
pecify
>> multiboot,modules for domUs, but the presence or not of dom0 modules is =
the only
>> Information we need to understand if the user decided to start Xen with =
everything
>> in places (modules in memory, xen command line, dtb) or if the job is de=
manded to the
>> UEFI stub and its configuration file.
>=20
> I don't think so. Imagine a case where the user has everything in device
> tree, doesn't need xen.cfg, but dom0 and domUs are specified as
> uefi,binary.
>=20
> We don't want xen.cfg but we do want to be able to load files from the
> filesystem. This might not be currently implemented but from an bindings
> perspective it should be possible.
>=20
>=20
>> By the configuration file you can also load in memory the Xen dtb, so Xe=
n can
>> be started as an EFI application without the DTB and then load it using =
the EFI stub.
>=20
> This can be very useful but it would follow the !fdt check and return
> true from efi_arch_use_config_file. So it doesn't really conflict with
> anything we would around multiboot,module and xen,cfg-loading.
>=20
>=20
>> I=E2=80=99m not against this new property =E2=80=9Cxen,cfg-loading=E2=80=
=9D, I just think it is not needed because
>> we have all the information we need without it and in any case we need t=
o read the
>> configuration file because otherwise we won=E2=80=99t have access to the=
 Xen command line.
>=20
> We don't necessarely need to read the Xen command line from xen.cfg :-)
>=20
>=20
>> Now I=E2=80=99m going to show you examples of all use cases that are val=
id with the introduction
>> of this serie:
>>=20
>> 1) Start Xen as EFI application and load only Dom0
>>=20
>>    Xen.cfg:
>>    [global]
>>    default=3Dxen_dom0
>>=20
>>    [xen_dom0]
>>    options=3D<Xen command line>
>>    kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>    ramdisk=3Dinitrd-3.0.31-0.4-xen
>>    dtb=3D<xen DTB>
>>=20
>>    DT:
>>    {no modification}
>>=20
>> 2) Start Xen as EFI application to load a true dom0less setup
>>=20
>>    Xen.cfg:
>>    [global]
>>    default=3Dxen_true_dom0less
>>=20
>>    [xen_true_dom0less]
>>    options=3D<Xen command line>
>>    dtb=3D<xen DTB>
>>=20
>>    DT:
>>    chosen {
>>        #size-cells =3D <0x1>;
>> 	#address-cells =3D <0x1>;
>>=20
>> 	domU1 {
>>            #size-cells =3D <0x1>;=20
>>            #address-cells =3D <0x1>;
>>            compatible =3D "xen,domain=E2=80=9D;
>>            cpus =3D <1>;
>>            memory =3D <0 0xC0000>;
>>=20
>>            module@1 {
>>                compatible =3D "multiboot,kernel", "multiboot,module=E2=
=80=9D;
>>                bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=
=9D;
>>                uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
>>            };
>>=20
>>            module@2 {
>>                compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,mo=
dule=E2=80=9D;
>>                uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
>>            };
>>=20
>>            module@3 {
>>                compatible =3D "multiboot,device-tree", "multiboot,module=
=E2=80=9D;
>>                uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
>>            };=20
>>        };
>>=20
>>        domU2 { <as above> };
>>    }
>>=20
>> 3) Start Xen as EFI application to load Dom0 and DomUs
>>=20
>>    Xen.cfg:
>>    [global]
>>    default=3Dxen_dom0_domUs
>>=20
>>    [xen_dom0_domUs]
>>    options=3D<Xen command line>
>>    kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>    ramdisk=3Dinitrd-3.0.31-0.4-xen
>>    dtb=3D<xen DTB>
>>=20
>>    DT:
>>    chosen {
>>        #size-cells =3D <0x1>;
>> 	#address-cells =3D <0x1>;
>>=20
>> 	domU1 {
>>            #size-cells =3D <0x1>;=20
>>            #address-cells =3D <0x1>;
>>            compatible =3D "xen,domain=E2=80=9D;
>>            cpus =3D <1>;
>>            memory =3D <0 0xC0000>;
>>=20
>>            module@1 {
>>                compatible =3D "multiboot,kernel", "multiboot,module=E2=
=80=9D;
>>                bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=
=9D;
>>                uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
>>            };
>>=20
>>            module@2 {
>>                compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,mo=
dule=E2=80=9D;
>>                uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
>>            };
>>=20
>>            module@3 {
>>                compatible =3D "multiboot,device-tree", "multiboot,module=
=E2=80=9D;
>>                uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
>>            };=20
>>        };
>>=20
>>        domU2 { <as above> };
>>    }
>>=20
>> So as you see every case is covered without the introduction of the
>> property.
>>=20
>> Please let me know what do you think.
>=20

Hi Stefano,

> I think that from an interface perspective (not a code perspective) we
> need to be able to account for cases like:
>=20
> 4) Start Xen as EFI application and load only Dom0
> no Xen.cfg
> DT:
>  xen,xen-bootargs
>  dom0/uefi,binary
>  domUs/uefi,binary
>=20
>=20
> But in any case, even disregarding this case, past experience has taught
> me that it is always better to have an explicit flag to trigger a new
> behavior, rather than relying on "guesswork". If we introduce
> "xen,cfg-loading", we are going to be a lot more future-proof that if we
> don't introduce it in terms of backward and forward compatibility in
> case we need to change anything.

I see your point, for sure the DT is a more powerful tool than the simple
text configuration file and it would be the best interface.
However I think we are moving into the direction where x86 and arm
are going to diverge even if we can have a common interface for them
(the configuration file).

For that reason I=E2=80=99m asking if you would be willing to accept a solu=
tion
where we introduce a new keyword in the configuration file:

dom0less=3D<dtb> OR domu_guests=3D<dtb> OR I=E2=80=99m open to suggestion.

Where the pointed dtb contains the domU domains:

/dts-v1/;

/ {
    /* #*cells are here to keep DTC happy */
    #address-cells =3D <2>;
    #size-cells =3D <2>;

    domU1 {
           #size-cells =3D <0x1>;=20
           #address-cells =3D <0x1>;
           compatible =3D "xen,domain=E2=80=9D;
           cpus =3D <1>;
           memory =3D <0 0xC0000>;

           module@1 {
               compatible =3D "multiboot,kernel", "multiboot,module=E2=80=
=9D;
               bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=9D=
;
               uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
           };

           module@2 {
               compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,modul=
e=E2=80=9D;
               uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
           };

           module@3 {
               compatible =3D "multiboot,device-tree", "multiboot,module=E2=
=80=9D;
               uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
           };=20
    };

    domU2 { <as above> };

};


So that the user cases we discussed are valid:

1) Start Xen and load Dom0:
  =20
   Xen.cfg:
   [global]
   default=3Dxen

   [xen]
   options=3D<Xen command line>
   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
   ramdisk=3Dinitrd-3.0.31-0.4-xen
   dtb=3D<xen DTB>

2) Start Xen and load only domUs (true dom0less)

   Xen.cfg:
   [global]
   default=3Dxen

   [xen]
   options=3D<Xen command line>
   dom0less=3D<dom0less DTB>
   dtb=3D<xen DTB>

3) Start Xen and load Dom0 and DomUs

   Xen.cfg:
   [global]
   default=3Dxen

   [xen]
   options=3D<Xen command line>
   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
   ramdisk=3Dinitrd-3.0.31-0.4-xen
   dom0less=3D<dom0less DTB>
   dtb=3D<xen DTB>


With this change we will be consistent across x86 and arm UEFI boot
start procedure, we won=E2=80=99t touch the current check on multiboot,modu=
les
because it will be valid, we will have a way to boot dom0less and it
requires less testing for the changing in the common code.

Please let me know what do you think about that.

Cheers,
Luca




From xen-devel-bounces@lists.xenproject.org Tue Sep 21 09:48:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 09:48:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191614.341694 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScNw-00014N-Cc; Tue, 21 Sep 2021 09:48:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191614.341694; Tue, 21 Sep 2021 09:48:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScNw-00014G-9Z; Tue, 21 Sep 2021 09:48:20 +0000
Received: by outflank-mailman (input) for mailman id 191614;
 Tue, 21 Sep 2021 09:48:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x95K=OL=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mScNv-00014A-QG
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 09:48:19 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5981b4f1-ee6a-4685-8616-83124753625e;
 Tue, 21 Sep 2021 09:48:18 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5BD2F220BA;
 Tue, 21 Sep 2021 09:48:17 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1E18313BC1;
 Tue, 21 Sep 2021 09:48:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id z/0YBmGqSWFMSgAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 21 Sep 2021 09:48:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5981b4f1-ee6a-4685-8616-83124753625e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632217697; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=khBP5Zy8ajCKmjZXYuuEWqwWY+FnzYb7goWddflULL4=;
	b=uP0bDIOa1vdrFWX5tbC1ORxxW51mcw2hGcNfCafYyxL/4phrpIpbDaUInwwT+m9Qb5c/g9
	bdwZ/E5pumq/H7R+NyM4nUxC77hNQT7OZKFdxCX2ZHk8amZT7P8pMwQB4Q4b7ulZwvxB70
	GmD8u9B1fmZutBjndtKoMIscvn+Pz4Y=
To: Peter Zijlstra <peterz@infradead.org>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
 linux-kernel@vger.kernel.org, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210921070226.32021-1-jgross@suse.com>
 <YUmXdEGhgEoukk80@hirez.programming.kicks-ass.net>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
Message-ID: <f1e90e32-e6b0-ee50-0687-fce941a5f267@suse.com>
Date: Tue, 21 Sep 2021 11:48:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <YUmXdEGhgEoukk80@hirez.programming.kicks-ass.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Vh7Yxb9qoqtYbSyuCWcNvkXyPfc6kq7dc"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Vh7Yxb9qoqtYbSyuCWcNvkXyPfc6kq7dc
Content-Type: multipart/mixed; boundary="05v0w8Smqz5QkDQaHGDvhROvgr5dsKewm";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
 linux-kernel@vger.kernel.org, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
Message-ID: <f1e90e32-e6b0-ee50-0687-fce941a5f267@suse.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
References: <20210921070226.32021-1-jgross@suse.com>
 <YUmXdEGhgEoukk80@hirez.programming.kicks-ass.net>
In-Reply-To: <YUmXdEGhgEoukk80@hirez.programming.kicks-ass.net>

--05v0w8Smqz5QkDQaHGDvhROvgr5dsKewm
Content-Type: multipart/mixed;
 boundary="------------378369EEDDCE49808EB6A815"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------378369EEDDCE49808EB6A815
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 10:27, Peter Zijlstra wrote:
> On Tue, Sep 21, 2021 at 09:02:26AM +0200, Juergen Gross wrote:
>> Disabling preemption in xen_irq_enable() is not needed. There is no
>> risk of missing events due to preemption, as preemption can happen
>> only in case an event is being received, which is just the opposite
>> of missing an event.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   arch/x86/xen/irq.c | 18 +++++++-----------
>>   1 file changed, 7 insertions(+), 11 deletions(-)
>>
>> diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c
>> index dfa091d79c2e..ba9b14a97109 100644
>> --- a/arch/x86/xen/irq.c
>> +++ b/arch/x86/xen/irq.c
>> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>>   {
>>   	struct vcpu_info *vcpu;
>>  =20
>> -	/*
>> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
>> -	 * cleared, so disable preemption to ensure we check for
>> -	 * events on the VCPU we are still running on.
>> -	 */
>> -	preempt_disable();
>> -
>>   	vcpu =3D this_cpu_read(xen_vcpu);
>>   	vcpu->evtchn_upcall_mask =3D 0;
>>  =20
>> -	/* Doesn't matter if we get preempted here, because any
>> -	   pending event will get dealt with anyway. */
>> +	/*
>> +	 * Now preemption could happen, but this is only possible if an even=
t
>> +	 * was handled, so missing an event due to preemption is not
>> +	 * possible at all.
>> +	 * The worst possible case is to be preempted and then check events
>> +	 * pending on the old vcpu, but this is not problematic.
>> +	 */
>>  =20
>>   	barrier(); /* unmask then check (avoid races) */
>>   	if (unlikely(vcpu->evtchn_upcall_pending))
>>   		xen_force_evtchn_callback();
>> -
>> -	preempt_enable();
>>   }
>>   PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable);
>>  =20
>> --=20
>> 2.26.2
>>
>=20
> So the reason I asked about this is:
>=20
>    vmlinux.o: warning: objtool: xen_irq_disable()+0xa: call to preempt_=
count_add() leaves .noinstr.text section
>    vmlinux.o: warning: objtool: xen_irq_enable()+0xb: call to preempt_c=
ount_add() leaves .noinstr.text section
>=20
> as reported by sfr here:
>=20
>    https://lkml.kernel.org/r/20210920113809.18b9b70c@canb.auug.org.au
>=20
> (I'm still not entirely sure why I didn't see them in my build, or why
> 0day didn't either)
>=20
> Anyway, I can 'fix' xen_irq_disable(), see below, but I'm worried about=

> that still having a hole vs the preempt model. Consider:
>=20
> xen_irq_disable()
>    preempt_disable();
>    <IRQ>
>      set_tif_need_resched()
>    </IRQ no preemption because preempt_count!=3D0>
>    this_cpu_read(xen_vcpu)->evtchn_upcall_mask =3D 1; // IRQs are actua=
lly disabled
>    preempt_enable_no_resched(); // can't resched because IRQs are disab=
led
>=20
>    ...
>=20
> xen_irq_enable()
>    preempt_disable();
>    vcpu->evtch_upcall_mask =3D 0; // IRQs are on
>    preempt_enable() // catches the resched from above
>=20
>=20
> Now your patch removes that preempt_enable() and we'll have a missing
> preemption.
>=20
> Trouble is, because this is noinstr, we can't do schedule().. catch-22

I think it is even worse. Looking at xen_save_fl() there is clearly
a missing preempt_disable().

But I think this all can be resolved by avoiding the need of disabling
preemption in those calls (xen_save_fl(), xen_irq_disable() and
xen_irq_enable()).

Right now disabling preemption is needed, because the flag to be tested
or modified is reached via a pointer (xen_vcpu) stored in the percpu
area. Looking where it might point to reveals the target address is
either an array indexed by smp_processor_id() or a percpu variable of
the local cpu (xen_vcpu_info).

Nowadays (since Xen 3.4, which is older than our minimal supported Xen
version) the array indexed by smp_processor_id() is used only during
early boot (interrupts are always off, only boot cpu is running) and
just after coming back from suspending the system (e.g. when being
live migrated). Early boot should be no problem, and the suspend case
isn't either, as that is happening under control of stop_machine()
(interrupts off on all cpus).

So I think I can switch the whole mess to only need to work on the
local percpu xen_vcpu_info instance, which will access always the
"correct" area via %gs.

Let me have a try ...


Juergen

--------------378369EEDDCE49808EB6A815
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------378369EEDDCE49808EB6A815--

--05v0w8Smqz5QkDQaHGDvhROvgr5dsKewm--

--Vh7Yxb9qoqtYbSyuCWcNvkXyPfc6kq7dc
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFJqmAFAwAAAAAACgkQsN6d1ii/Ey8H
vAf/WNiR0DBxKqcEH+SPala7BeDu0AfOvpBXrSZ/mgLNOb5ztfLP0ShNfDx1uL7fQTofiHwhKsgz
rOCQ1ymAkpwVGtziFVKYCkNdksGqFHaCLboaopdrAnuRiCJNgVpwAJ1tuJu/goVuz5UN7WiIT1+/
vxsioI0uFQUlG3p+L5OJWrGjeoMlOez0VZhRzQ63XZ2+kzkkGdU2YCT2FZeCWpZsqHT74m+E1157
Rwk1oxEjIg8H5GBdij/71nj1hd5VZKPcnuiZYqyBhQ9L2gDeBX0DzT6JHJPXJyBtKYqDavOX2uUe
rZkl9UFUhF2c2iqlKa94BEwXzN6U58JHEEZGos0Xyg==
=XCVu
-----END PGP SIGNATURE-----

--Vh7Yxb9qoqtYbSyuCWcNvkXyPfc6kq7dc--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 09:51:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 09:51:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191620.341706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScQf-0002Qf-Qi; Tue, 21 Sep 2021 09:51:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191620.341706; Tue, 21 Sep 2021 09:51:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScQf-0002QY-NP; Tue, 21 Sep 2021 09:51:09 +0000
Received: by outflank-mailman (input) for mailman id 191620;
 Tue, 21 Sep 2021 09:51:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mScQe-0002QO-6A; Tue, 21 Sep 2021 09:51:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mScQd-00044a-VM; Tue, 21 Sep 2021 09:51:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mScQd-00079W-Jy; Tue, 21 Sep 2021 09:51:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mScQd-0007ES-JU; Tue, 21 Sep 2021 09:51:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HXkyXGiDmoYGslGr8ph2Mwd8rz4x7qr2ZtxBP/g5Xrs=; b=64Hz37+asOBMitNwLP9hTEtGyI
	wGjxug8mPgXgufk0WmrmQqas2H5pyDJtCoWE5Y4Eh1Qmll45GWJ3Vi1RlG1XR+VXTnAJOJJ/gW6Y0
	r96Q6akj/jgljXnuwdjPKRnbvmBmt6AbJSunaGV8Dz42K9c0SsMseq6qQ6wToUMLTzuk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165133-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165133: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=326ff8dd09556fc2e257196c49f35009700794ac
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 09:51:07 +0000

flight 165133 qemu-mainline real [real]
flight 165138 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165133/
http://logs.test-lab.xenproject.org/osstest/logs/165138/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                326ff8dd09556fc2e257196c49f35009700794ac
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z    9 days
Failing since        164967  2021-09-13 13:06:52 Z    7 days   16 attempts
Testing same since   165133  2021-09-20 22:08:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3339 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 10:12:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 10:12:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191632.341720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScl3-00057Y-QS; Tue, 21 Sep 2021 10:12:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191632.341720; Tue, 21 Sep 2021 10:12:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mScl3-00057R-N8; Tue, 21 Sep 2021 10:12:13 +0000
Received: by outflank-mailman (input) for mailman id 191632;
 Tue, 21 Sep 2021 10:12:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mScl2-00057L-C2
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:12:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a0f3ddb2-772b-4c25-a6c6-295969fbe774;
 Tue, 21 Sep 2021 10:12:11 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-F5YSqCGxO4qaff2PcV_6WQ-1; Tue, 21 Sep 2021 12:12:09 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6672.eurprd04.prod.outlook.com (2603:10a6:803:127::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 10:12:07 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 10:12:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0087.eurprd04.prod.outlook.com (2603:10a6:208:be::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Tue, 21 Sep 2021 10:12:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0f3ddb2-772b-4c25-a6c6-295969fbe774
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632219130;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=egNgv28QnKne2cGoa+8aA2c2WhtsIi7/yf9Crdily68=;
	b=NRJklLgFbD57jJqboxXKXkj9q72ThgnDK+codHxZEhtMTrrbjG21jIQg/r2L0ebWjzcpcP
	/yAr6n3oVm6MIy1vmXT3Hnh5GcBlJT6+c59xWrwi5yr+140MxMSMW34l9tOd8DQ0rRG7uJ
	NRHi1PT5WJ01bRetZ0/HG5UzTUmHI/0=
X-MC-Unique: F5YSqCGxO4qaff2PcV_6WQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QrdM/TiPbB/PyQkrpF9hDDzHgJ1RmVGMm2x25t34AtHBN8eEqZFdNEns/YxRVSe9YMs8EafngSeD0Ekk48r9+3rjva2uu1xVYsveY/iArnKIptMMI+IcDASXlaj83cXhxeIRAy67ffJb2g9rYmtW2HlsKbktDW3hgx/6HktMD97Tk0l7R+zzVLYKn6BNhZCyGBrzQTxakvdQMfuVA+B9/BINBaNEyW93oOntRXPW31tMOsJIcLORGzx9LtTN4ForBDuRzCL8xf35ujyJgcwgrw0shkJBvIfOpVsrT0ef61iBIwhj0N1oZ4lJaGcFej521e6Vow55oipzhPO3cLUp6w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6+EB9aYIW2YQDYMOKVNo1MDsa3unHdTbAmMooPXfIzc=;
 b=h/oeeAeN8oiXz2wfgF+O0UypXXGuk27CDVhZVebWwONAqlZAxkanQuyGJKKn+u5NbESzl/wRb+LyxNs4cH6SvN3iOpYjd2T05OyOfh0TDzLHT4lPBDsejQsulVpfW3P2GWtMwZbj96O3rn+h/uyHfR50q1lHzFNV92PYNU1w+htgIWp+4Bfz9EFhnGdeB/69L44BEU70UVkookeMLtHY8uwKlPFAxYMqRJBcNvGlJymigIrE9W7mUC/rOFjVRD4djk2CamzNF7Eh5cXrX9S3ofl4SOjGvbPEquKXyH1TNplnm6iqOlFddoNC6nPkG/aQYiNYw2583g6ZVy7u+pdTCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
Date: Tue, 21 Sep 2021 12:12:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR04CA0087.eurprd04.prod.outlook.com
 (2603:10a6:208:be::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: beb1f084-202c-46a3-b0c5-08d97ce84499
X-MS-TrafficTypeDiagnostic: VE1PR04MB6672:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66726BBF9C7A7A62B030D46DB3A19@VE1PR04MB6672.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z0pkep8Fly63WcTcexenh6XebL38Sft4kgKqkCYaYQsHfFQrHrwmWZhDsteYTQTylcryNYozb/91tZfi1EH55WbRov+JF7VUz6zhPKAk3jziPp6IaOtEW7EhSemllfmgCSr3NEn/q+vJLgBUa7BXALLAl8Dn7lwyBPS9giICbpFs5r+wVlfqLvF0oIRxVEliS/atYGTaGMP4zh8kIgpWuqHtbhaDiFM97FhK6E9Rj+hzMuxHdLP61f9G0e1FL61sgU332o6xEnYmntE4kAKwHQSTJ4IYDkQHcle9+6zMdmMNcEHhxic/GqmzQl35BKjNfI4s3NztvE6GHB+qe4WpVunD9P5vVNdtR17diohcEpL5tfmwhmh/tfucofb9xTyEgpE1ENKHIeVW8oq9+hG04xHkuN+kQ2dgQhKtbesAWcwOXpy6uIr1rWeZjDwJvUw+DH4jTu4lUa5/XDAduLFIgf0rtTuFZvMeGUWVPYlyWoiFiwCWF3XLbjniFTwZeWp9G2wxWsbekCVU5dKaYrfsllZqF3WXCG23D4iQxP5nWVMRzoFW4H0M89w6Ss6ku9Nqw3gpIW+E7hfIj+pWZi6Uvqt5uKRZu7wLlyE6GwAZ4JJH+ScyJhEu6Dj9K7QCODDUY0zKFt9pOfq1iHuv9uztAkyjW7YluSmJfatRM8owVGNXiqUiHIrYycFWxCJ146Z/fQZvwddR3la8+tEfeIH5sLk1VSLwmZ9ipflhoF0Mo9TS7QbYWSYE8beCGYAwiQcb
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(16576012)(31686004)(316002)(54906003)(38100700002)(6486002)(83380400001)(2906002)(26005)(8936002)(186003)(36756003)(53546011)(4326008)(2616005)(31696002)(956004)(66946007)(86362001)(66476007)(66556008)(508600001)(8676002)(5660300002)(76704002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?3M+ZqCprnT7icWea1tzC2MCiKgvbVKtu8wWYFA/gxO4lECx6kEa0fYBX3Hah?=
 =?us-ascii?Q?CPCQsdyUOVOP8+mAPsadh76AYjU4+lR15iuIPeRdBrECaULbZn9vSiom9HWs?=
 =?us-ascii?Q?lyKGpq37HdaMZh4ocwkK6cgz0WcBGgihSu0IpZAKoGJ/IXJV5Y0yBdRoaH4k?=
 =?us-ascii?Q?0dx9LLI6PorDf3+F4JoBZTK9WB36pA9OyDETqaJsGqTb/xN73OWFx5QX/aNx?=
 =?us-ascii?Q?UfjsAyjxfKchN4artgSSosHzaIpuS0hhizmbbU1igpdFaWZLd6RKOluS8MOE?=
 =?us-ascii?Q?jkM0dsgDej/+EG/hDOZ5A9oxSWaYOzdoWqNdKuRnTymGD+g26Ru7+kw0cqpR?=
 =?us-ascii?Q?wseU4JgtltoEL3IFm2x5j1NlRUWvEX6rUI4u24zdC3K6MN3xodWu47UwaTA5?=
 =?us-ascii?Q?wM7wAZea5ICIG3ImU79jd2Sn8Ds34PPOVkLwTkWhMuq5dBTHryamtC9qjsKg?=
 =?us-ascii?Q?AnUw6fZoMjy3Sctg8X1h4iDaTeVo2srIrKvg0LkYWp1JLQmta+21m9Ln2gw2?=
 =?us-ascii?Q?9d3u8Tl1ib769PqbQ1qjiDk1zeft5FPFCZIjMuJpXUthUH2nURvDIEeSKc0e?=
 =?us-ascii?Q?+RYNvBziibjjcLpdFzfKThSMyL0H7K/k1I3QqkteAQJ7SRhXurioAubzHJAy?=
 =?us-ascii?Q?85UDyoUoJ+qk4WE5JsIdW5ey8U/cQcBgCMbDXdjA2V0sXYclTy+YoceNUJ9a?=
 =?us-ascii?Q?XiBpk91/DQ9LxjfFbngNgstiM4wYB1slK1aYKgucEj5TQpKomkBnoSCNZoby?=
 =?us-ascii?Q?cwq5vBX1gWrMPVo+sf+RbKDa/K2M72hSzl9efvx+/oD5gmOGqBG1svC1uzLW?=
 =?us-ascii?Q?O/QQj+jrUhZbaOz5InSyuooLOJc5ahuNCQ3PvnJ79fYj8NlBoyCq95+UYXyY?=
 =?us-ascii?Q?szGMyJmzWtkUMdWz/5F29VWUeOaUIkbN4VyVNUx+sPXAxQ/gihTpDTimiWzt?=
 =?us-ascii?Q?yr0y9UMJ9r3qzSteULeUMs2caK4iQY7iAU+cj6aZIttVOwIsSqwH4W4aOR4q?=
 =?us-ascii?Q?IBm5Vtm8bDUZsG2wv5wuTNB7D0EggrYBkvaKryWqHEt/QreXTCrUMAGjtMAM?=
 =?us-ascii?Q?NnyxGbLEi3m3o02uHu6fVDmbMrIlUqueo2WlssL8wJZ0imf5x9Nl+DZXiDyF?=
 =?us-ascii?Q?ADvd1IoHhdu4X0aICpWKWU41DBw7QaaspK2XOxn2jqpG8K+J2U52f7kdAmJG?=
 =?us-ascii?Q?AUYinc8vjIg7XaeOYgH5AamzUqbo9XYONl4Z/wbrv2e/QBMLCmPshqThGzui?=
 =?us-ascii?Q?x0+A8/TlcgGWMVMCYzBMmCbX7bvYbrxaZZbh9fEhur59nIUo5FiYI5je+QwH?=
 =?us-ascii?Q?L4e5l1AXWdF/RLDWrWPdn9u9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: beb1f084-202c-46a3-b0c5-08d97ce84499
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 10:12:07.5935
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: noqSlIsyOyXxXVL/Nmhp+u+j/afLyLOCZ37mdiWVB/UoYCLDFIvHAhXOfyDqzeXZjfrlq3jajs869AIJDLGkxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6672

On 21.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>> On 20.09.2021 12:20, Roger Pau Monn=C3=A9 wrote:
>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>> --- a/xen/include/asm-arm/grant_table.h
>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||       =
    \
>>>
>>> I'm slightly confused by this checks, don't you need to check for
>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>> guest_physmap_remove_page?
>>
>> Why? It's ogfn which gets passed to the function. And it indeed is the
>> prior GFN's mapping that we want to remove here.
>>
>>> Or assuming that ogfn is not invalid can be used to imply a removal?
>>
>> That implication can be (and on x86 is) used for the incoming argument,
>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>=20
> I guess I'm confused due to the ogfn checks done on the Arm side that
> are not performed on x86. So on Arm you always need to explicitly
> unhook the previous GFN before attempting to setup a new mapping,
> while on x86 you only need to do this when it's a removal in order to
> clear the entry?

The difference isn't with guest_physmap_add_entry() (both x86 and
Arm only insert a new mapping there), but with
xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
mappings. And gnttab_map_frame() gets called only from there. This
is effectively what the first paragraph of the description is about.

> So you are effectively only removing the call to
> guest_physmap_remove_page in gnttab_map_frame for x86, because Arm
> will still perform it in gnttab_set_frame_gfn.

Yes.

> This seems like a limitation of Arm's guest_physmap_add_entry.

As per above I'm viewing this as a limitation of Arm's
xenmem_add_to_physmap_one().

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 10:28:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 10:28:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191639.341731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSd0g-0006sR-7N; Tue, 21 Sep 2021 10:28:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191639.341731; Tue, 21 Sep 2021 10:28:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSd0g-0006sK-3M; Tue, 21 Sep 2021 10:28:22 +0000
Received: by outflank-mailman (input) for mailman id 191639;
 Tue, 21 Sep 2021 10:28:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSd0e-0006sE-3A
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:28:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a2cfabea-1ac6-11ec-b8b4-12813bfff9fa;
 Tue, 21 Sep 2021 10:28:18 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-DK50HmfFMHSAcA2w15ncxA-1; Tue, 21 Sep 2021 12:28:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 10:28:14 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 10:28:14 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR05CA0081.eurprd05.prod.outlook.com (2603:10a6:208:136::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Tue, 21 Sep 2021 10:28:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2cfabea-1ac6-11ec-b8b4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632220097;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lV3I1Zo8tNf8W3rweWKyBOe8Xc8sjLGTMlHqR1XG1YE=;
	b=TxK7cDrhDi6m+MzrrXCJw6S1/mnAo5awVOHx105AV1GxZFAM/r8sGXR5Dks7a3i01nVACv
	JkbKSjfGQIhgIaK+1f/XgFrj1u8Vy+DwIKIGkMuQ91N+g3BqGA14bQt0zkYFIL9qwtCSHS
	/2uRQAz+J2R9PnxWzY98nIii+Mo7+Zc=
X-MC-Unique: DK50HmfFMHSAcA2w15ncxA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kk/ly3Mo+OU9uTEiDUTHO51wtKmTYlwOJvBUCT8JSJk43kvvF7aWh5sdTUjIVePW9yLJNRQV/D0/xhHioXUmAXcFETMdH6CBuFVpgh6Qcr9NRg8LRtK/qa259XF59DA9w5ip6fK5FphEIPNEMLogKGBMaOJJqwzRT2ZVnwDjxsfe1ZkLSfRwvPKEMWeXEf/M/+72DT3B/cZ99lVklWZs9QJrShYxQdvto2RDOys8eYdUZIibPrauvvWCLj6hxxfU6i1XQpOY10IyHYzqn/oyIrXmqFpbIew1X/8XKr5r0r+pAbrrCzROw/IAGjPcpRd18WTrIq5PJEBzrenqd20l0g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+yRVntTgbnnOC9d18dQVZrXN5kZgOLcXWkhZgnh+hls=;
 b=WdTve8cjiMzgWC9qy4FYtY1d5DDjhhGooJg1keaIlde7vv5XvXzb2XAHRMAVJW20zrfhbKgKi2GaFO1g6MenhMxkHln8qEe4a/XUbGVTkaCuQ3ZNbkyqNFR56VOA1TkIFFRWCmDii0mw/1oJkuqkBHl1NkjGQWwNxvJXQ5m0UDFoQ/w9aHQh0X+R6CxO/i2e+t83e0f6F9/SJEcZu+M+idXstCOAeeTrbkb1jPPvj9bjcJfboHPs9NUuCulxKVonrA1KJkh/WWM9AuC9jKXRLkIZjQtQcSr4VLS5vIepCx6mVYtGBvT86fU1APRE4bX/ZuTPRyqBV0rVUBQQKh8iJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
 <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com>
Date: Tue, 21 Sep 2021 12:28:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR05CA0081.eurprd05.prod.outlook.com
 (2603:10a6:208:136::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4411a391-db27-498d-e525-08d97cea8510
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB39333B3E8CA1AC40B12BF2B0B3A19@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ScpmX5RcdjJmSTOVfTQTGUL7zym/zi+6yapT63Ejs1j2ykEQZ5VIxjtWlgCSN7oLV4ZTtpoySf31rCxUXqawqacbDgDFQ5rYewVkMvhXUMAXdTeGaZE7zkDWJ9f+Q5oXoWODA6bwG1twOYVpzMF8xDENRT0F96YBDCXdrTz117kysNPXE0LajwZGYbRtLLs4qoNW4LLaAzvUUY4ckMdTJgKqxl1E2K2k1mAtyJL1/pwIXMEZHniw4ytF3yWnx9aq8Sk+4lFMsfYnL28GHXQG33l+x3rWk69LA+PZTgdkTdbX4WK9E+mvv3gIeTypB4U90Otc8byzcWJgC1Wd8E1Vhtwo80AD81ohEjIbreWyj9wuPNOQErnVZQEix05ykjKLQHk2uijprTpOGbvO7ZodjVGQ8MM5F6+Ygb+S+bCrljkXoff/4qo8srsNO/G258VB3IOTkFsHCMSdh7/HmoxF3WUMOiu/zdLg81unBRWNMEX3dVDq0AYadTO+KCGStEffnXQyoPJWPMqTXstC0iNjP81qz7dXtWaKhi68QHLVvns1L27vByCW4tKNmFnpe8xOVb1ThReWHbfk893/nFUPmxts42DEO4SdAcogijJ/su1aa6nHxf0bmNEJNsCdXLL4SW7h+A5NdTPaf8y0TM9VnnQ0kIkUy5FlnF3gW5W60sxtJAEk8nfm01vli04rKI2kPYjspebgOBzTYQEH9gZP4IpYWHJdX40GR0C8qgBWzfY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(31696002)(316002)(6916009)(86362001)(66556008)(36756003)(66476007)(16576012)(4326008)(508600001)(8936002)(83380400001)(186003)(54906003)(31686004)(53546011)(6486002)(5660300002)(2906002)(8676002)(956004)(2616005)(38100700002)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?VXo1oW4/lF7lhc3l4bIwV/AhJAVK6KaY24WU/gJNllZnUo6wA1eTg3bDmK+D?=
 =?us-ascii?Q?yuv+PQWUYNOkWo/roU4MrH9Y9lYC5v9XFZu9daDziRgYjAXg9iqVpT41ApuR?=
 =?us-ascii?Q?KJmU/V5XRi4WkZ7p/eR3mP0SlDgI6IhqAnEY91EfkIl5PJxghZnoogw7psZV?=
 =?us-ascii?Q?ikb1jpPWAmk1uVZD3zVlrLxuOxNZeufq8ecA7hJ1aZU6Q9YcaBlxRbyAQ2zN?=
 =?us-ascii?Q?I6a5pOnr/aF5ZDtP1JLEe4bTv3vIgqR7pZMCoU2cfTlKFEsDiU3z+OJsiD41?=
 =?us-ascii?Q?0DJOTJAPrY1+rqWxe+iTh+HadzV6TQY/3W/lU8xmCT1AkeJjJjcO30GaW7IS?=
 =?us-ascii?Q?xjnb2J/M7+yHFV+3sS25wQKkf6ihyEENp0LM2w5VgXyI+kx50RBaOoCRzoq9?=
 =?us-ascii?Q?2/P/UsjHP+4zSEXBU4o8aSWF4rwf4fqmfZ0q5Qe6DCBi6Ul8g6JMHADICJ6p?=
 =?us-ascii?Q?EB5lcyPZU+jTJEwhcYFjWI/8s2hpWLXwPd6qmnH49DguOSx7Ag0nTMsmaNmL?=
 =?us-ascii?Q?68KnkIgAp9Zv0N1fA7GR1hSZIL916sRfvzQAuxyH1ZUD5n9uvZYePeph1rCl?=
 =?us-ascii?Q?Cosf9/hJ583OLjTCWzbcdRYlYQ8rHW4Nqy+fqWljvjYScBYCL5DR3L+oMhaf?=
 =?us-ascii?Q?tSP7acbohCia8a7ajdi3lm2AnUFczBxGW7k4s5i0matNb/KimJwbJ5zdMlEK?=
 =?us-ascii?Q?UQlqAhGsvNRwPzlGoQyGNA9XukXkFin3KToX8uONhGycKaRsIHEB85z6hdwO?=
 =?us-ascii?Q?wgV8BxrHEDj4Lazl3PRpCKX141rraUJNHd7mzpHKdqPHrnyH4D0k8UV755Fv?=
 =?us-ascii?Q?o0hKXCGKwYXV+fxGtVmNoYE9NOlJGrUfm90IL7XB8iHXpNoQ0PdNxIbFES1B?=
 =?us-ascii?Q?iZNMnnJuJu2LEyMSFznjZ55EBQ3ou09+eZtIFU+Hl2zn30OPqRHLg7aPBdO8?=
 =?us-ascii?Q?9gWtyibbVQ94sJzWCzrfiB7Vtl62zmaNoG+1d6DSZ09cEg0Sta5PR5og4mci?=
 =?us-ascii?Q?UCBfSvad2jRT3cDb/l/8OfrdnKjU2GoAgmaG5xbjo1z2cm4B5hq3Uznc9NRX?=
 =?us-ascii?Q?wQyeJZKrIfaEIQd0De2N55d/xvggTnukl/s4qkV8i7FFjeYhHTY3b+N3FAFn?=
 =?us-ascii?Q?mmtXBBRr7n/m4Tox9H0bNBWWCTVtNqEyHkqTg9bGTQKjiU2cqdyRNrXlj+Tv?=
 =?us-ascii?Q?L1BFHbYTEWRyTJthnWr6pE5bX9q8nNk1ZZe91ICBWKIw3bLg4luJdckWdPLm?=
 =?us-ascii?Q?kLTxU9q6ZngLeW4/J8SGBsvtiqab6b+5IRFTetV5vspEIUVeX/6sXynRuBv8?=
 =?us-ascii?Q?gvHaDeX4KChrUr7Ud9l6nC3K?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4411a391-db27-498d-e525-08d97cea8510
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 10:28:14.7293
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: P7pOzwy8K6pca0LQW/oekcV++yKK0IMF3ex2UJnrlpKHrZVjPJpeSLPnEYHzU5iQkHKzvzk32ZO0+GUtUbp1Fw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 21.09.2021 11:20, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
>> The function may fail; it is not correct to indicate "success" in this
>> case up the call stack. Mark the function must-check to prove all
>> cases have been caught (and no new ones will get introduced).
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> Acked-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Thanks. Albeit strictly speaking an ack here isn't enough for the change
to go in, it would need to be R-b or come from a REST maintainer.

>> ---
>> In the grant-transfer case it is not really clear to me whether we can
>> stick to setting GTF_transfer_completed in the error case. Since a guest
>> may spin-wait for the flag to become set, simply not setting the flag is
>> not an option either. I was wondering whether we may want to slightly
>> alter (extend) the ABI and allow for a GTF_transfer_committed ->
>> GTF_transfer_completed transition (i.e. clearing GTF_transfer_committed
>> at the same time as setting GTF_transfer_completed).
>>
>> --- a/xen/common/grant_table.c
>> +++ b/xen/common/grant_table.c
>> @@ -2394,7 +2394,7 @@ gnttab_transfer(
>>          {
>>              grant_entry_v1_t *sha =3D &shared_entry_v1(e->grant_table, =
gop.ref);
>> =20
>> -            guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
>> +            rc =3D guest_physmap_add_page(e, _gfn(sha->frame), mfn, 0);
>>              if ( !paging_mode_translate(e) )
>>                  sha->frame =3D mfn_x(mfn);
>>          }
>> @@ -2402,7 +2402,7 @@ gnttab_transfer(
>>          {
>>              grant_entry_v2_t *sha =3D &shared_entry_v2(e->grant_table, =
gop.ref);
>> =20
>> -            guest_physmap_add_page(e, _gfn(sha->full_page.frame), mfn, =
0);
>> +            rc =3D guest_physmap_add_page(e, _gfn(sha->full_page.frame)=
, mfn, 0);
>>              if ( !paging_mode_translate(e) )
>>                  sha->full_page.frame =3D mfn_x(mfn);
>=20
> Is it fine to set the frame even if updating the physmap failed?

Well - the page is now owned by that domain, so there's nothing outright
wrong with reporting its MFN. guest_physmap_add_page() failing in the
!paging_mode_translate() is also only possible under obscure conditions,
with the guest guessing about MFNs it is in the process of getting
assigned.

>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -268,7 +268,8 @@ static void populate_physmap(struct memo
>>                  mfn =3D page_to_mfn(page);
>>              }
>> =20
>> -            guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_order)=
;
>> +            if ( guest_physmap_add_page(d, _gfn(gpfn), mfn, a->extent_o=
rder) )
>> +                goto out;
>> =20
>>              if ( !paging_mode_translate(d) &&
>>                   /* Inform the domain of the new page's machine address=
. */
>> @@ -765,8 +766,8 @@ static long memory_exchange(XEN_GUEST_HA
>>              }
>> =20
>>              mfn =3D page_to_mfn(page);
>> -            guest_physmap_add_page(d, _gfn(gpfn), mfn,
>> -                                   exch.out.extent_order);
>> +            rc =3D guest_physmap_add_page(d, _gfn(gpfn), mfn,
>> +                                        exch.out.extent_order) ?: rc;
>>              if ( !paging_mode_translate(d) &&
>>                   __copy_mfn_to_guest_offset(exch.out.extent_start,
>=20
> Would it be worth it setting the mfn on the guest output to
> INVALID_MFN or some such if the physmap addition failed?

Like above - the page is in possession of the guest now. Once it knows
of the MFN it may be able to do something to remedy the error (at the
very least: free the page again, e.g. via decrease-reservation, where
only the MFN is needed).

Of course in both cases a prereq requirement on guest behavior would
be that they consume the output field in the first place despite the
error, which in turn requires them to prefill the field with a sentinel
allowing them to recognize whether a valid MFN was passed back.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 10:45:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 10:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191647.341741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdHQ-0000qg-Ln; Tue, 21 Sep 2021 10:45:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191647.341741; Tue, 21 Sep 2021 10:45:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdHQ-0000qZ-Iw; Tue, 21 Sep 2021 10:45:40 +0000
Received: by outflank-mailman (input) for mailman id 191647;
 Tue, 21 Sep 2021 10:45:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s/tM=OL=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSdHP-0000qT-DF
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:45:39 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0e9bc0dc-1ac9-11ec-b8bb-12813bfff9fa;
 Tue, 21 Sep 2021 10:45:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e9bc0dc-1ac9-11ec-b8bb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632221138;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=ynnlJo+YkYcBCozUKQbCY2q6mb4ySXXzjzM46igEHwQ=;
  b=ctcifkMJi8GB+9jElyXZ3RjD9jqOUvaQt8SZbfzG6zsM1Z8piQZALmO5
   H92Tsg7o64CPGDK0BCXGaK6162lVVSfMAC/XabUl3K2Q1Nmljxjxm27B1
   zfnA8t8b/IsXWqq0rloGc8L6jmbgGYcddLjO8TZ7YeyjCjCwpTbphkOhQ
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: PORsFy2C0OWrR0iEk11lTFYMFXLA6mVPpzZEYxfIkw6pvjVpjOQLNpYEWPU92pCQdElDrdL3oT
 1aH2fKAOGlUah/KiaL+tRWte9sdVc0ZruzGRPok7xuIJR3lgpiAOejx3bdcIRLs8CLnebfijmv
 WH1Q/iUCs0bGw7KYrHOReltAg8tzR3gjTDvml/zkIOwAfPmoZhasOSh6kQXu2SlrWSRYOtXQ5J
 Cr0nKxq37filNxlZ26xFj7/dEjsIxNn84Ia2QfCbLfrJmkhgi1OroWaCdrb862rQIj5WoiQOMJ
 D1Dv273FzOYTzK1s/D7lXuk6
X-SBRS: 5.1
X-MesageID: 53199071
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:mvwD4aJZMNH3E1XgFE+RP5IlxSXFcZb7ZxGr2PjKsXjdYENS0jEOm
 2oYWD+FM//bN2r3fNFyOdjk9xhQuMXQydZrHARlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wrZj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2Vj/Rd+
 YhKpKDuQCYsEKvggekdXBJhRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2XvoYJjW9v36iiG97bR
 MchcxR2PS/iekNiE1I8LdUmsteR0yyXnzpw9wvO+PtfD3Lo5A5s1LngNvLFd9rMQt9a9m6Dv
 X7P9Wn9BhAcNfScxCCD/3bqgfXA9QvkXKoCGbv+8eRl6HWRzGEODBwdVXOgvOK0zEW5Xrp3O
 0ESvyYjs6U23EiqVcXmGQ21pmaeuRwRUMYWFPc1gCmM1aj88wufHnIDTDNKdJohrsBeeNAx/
 gbXxZWzX2Up6eDLDyLGnluJkd+sETQFCUMzOCgVcVUMwtzNpq0qgxLARe82RcZZkebJ9SHML
 yGi9XZl3uVI3J9Tjc1X7nic3Gn9/cGhohodo1yNBzP7tFsRiJuNOtTwgWU3+8qsO2pworOph
 3ECh8HWx+QHF5jleMelEbhVQe3BCxpoNlThbb9T83sJrG/FF52LJ9k4DNRCyKBBaJ1sRNMRS
 BWP0T69HbcKVJdQUUOSX25WI59wpZUM6Py/DqyEBjawSsEpKWdrAx2ClWbPhjuwwSDAYIkUO
 IuBcNbEMJrpIf08l1KLqxMm+eZznEgWnDqLLbiilkjP+efONRa9FOZeWHPTP79R0U9xiFiMm
 zqpH5DRkEs3vSyXSnS/zLP/2nhQfCBnWsip+5IIHgNBSyI/cFwc5zbq6epJU6RunrhPl/eO+
 Xe4W0RCz0H4i2GBIgKPAk2Popu1NXqmhX5kbyEqI3iy3H0vPdSm4KsFLsNldrg77u1zi/VzS
 qBdKcmHB/1OTBXB+igcMsah/NAzKkzziFLcJTehbRg+Y4VkG17D9Oj7c1a97yIJFCe265cz+
 uXyygPBTJMfbA1+F8KKOum3xla8sCFFyuJ/VkfFOPdJf0Do/NQ4IiD9lKZvccoNNQ/C1n2R0
 APPWUUUouzEookU9tjVhP/b89f1QrUmRkcDRjvV97e7MyXe71GP+44YXbbaZy3ZWUP15L6mO
 bdfwcbjPaBVh11NqYd9TepmlPps+9v1qrZG5Q14B3GXPU+zA7ZtL3Taj8lCsqpBmu1QtQesA
 x/d/9BbPfOCOd//EU5XLw0gN7zR2fYRkzjUzPI0PESlu3MnoOvZCR1fb0uWlShQDLppK4d0k
 +4utfkf5xG7lhd3YM2NiTpZ9jjUI3ENO0n9Wkr23GM/ZtIX92x/
IronPort-HdrOrdr: A9a23:nUWDfaA1x+GYHQnlHemo55DYdb4zR+YMi2TDgXoBLSC9E/b5qy
 nApp8mPHPP4gr5O0tApTnjAsa9qCjnhPtICOAqVN+ftW/d1VdAR7sN0WKN+VHd84KVzJ876U
 /NGZIOa+EZrDJB/KTH3DU=
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53199071"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kCfV3RuOkIEfVXpUvswZGh9AsC6QHW3pNi8AQVQyx8x9p3o+RarByiAoGecXDjcFFW73ypOVA/BewUo0GKFf6SzEXgL6ni4PHNrSxvysJ4YEe/mwgGNbMSurjeJ5GAXltDdyM4DvDtRgfKcqlcDaoaN7PILRy8TC2/C/lN9mo2C8KpMZcWDsmUcrNRmV0BSG6FzBqJPljHFH1BgN9DJ8vDbuj00RavJX2F+4wK4ilDgGf0fHf9W+IHpC4gJS6tf7yoI4ZHbU87/ydi0FAgArkFXbbEHgSQRtJ4MQuvrmYEgKrwaKvEWazx8sbqR/kKr+sV48K5XGSs2i9vo9FQpfeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=E8xTm3XLLtFlLVUnoEW45dK7JTjoytq/BFVxDn9O/48=;
 b=SvbSFrKqk0h69lthlDaaF8JWZ8LWbbvyHlFYXblKokG4w1c3l4xuEd0kTAof2Np/PZai/JeyxQE0+NEvnDfV9RQ/D+m4bdlns9Rix/LYFdlIpW+r+1JnY2VK4wD3iW+sHonET+JOZFG/n5RMe7qSzTdWm13YnpG1JRRV1B9aTHE1xIe4GIknIGrHfl2hHdmjgNfpEJyooi1HSTKaRtmCGbWYdePuJnERXAxnHLgHayOP2dMU3au++5ayiW/cCJULFb4nIWGRpz1LNzD/gQvbNCNg1LhJNhburUAXQ9hqI6evDHW8eAtP5NiG3xsslMpgokQj0nhdndrrIVmlyLLUUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=E8xTm3XLLtFlLVUnoEW45dK7JTjoytq/BFVxDn9O/48=;
 b=Ue2TNXM7uRvm+S3nHk2ZjwDjJS7TTaVgea4dfh3bAcWrwaJ8VQfsel/x3Uhw73bEedkFpYM4VxYU8uJ7UewE5vPMTTrx9hIoUxxyAyySwfBJ4tVbOmDaeVb3Lh/SyDn1eCdvC9v/p4ZHcIr4fSKH3MpMMraJPmbV5VhCm0iEucc=
Date: Tue, 21 Sep 2021 12:45:28 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, "Stefano Stabellini" <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
Message-ID: <YUm3yLOfAfqrwgmH@MacBook-Air-de-Roger.local>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
 <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
 <58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com>
X-ClientProxiedBy: LO2P265CA0389.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:f::17) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 67b169fb-77a0-4727-0864-08d97cecf031
X-MS-TrafficTypeDiagnostic: DM5PR03MB2555:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB25551B7AA2CB1758176AC9AA8FA19@DM5PR03MB2555.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: t6MexaBsTt+esdIJ0T6zaEp9k4Qem3TOqYV8czb6vR1jWwcg3N0OteHYrWi2GbK/ZWPq7og+83e1m3FejgJl9RrarGY+mUAn6pKV2F1er0QaoqXQE7Nwz0GYUy5inK8HL3nbYNsMw9BFLA36hNMlwsyg5p1t7IOTxeWfC91FjMfMMRYeB5oFkH0FdPyZ/XDVy6X28WWgzfnfsiap03wdYbRf3eXznFtxAZ/+CGiYm2TrpZG0T+VTIOx6v/igW50hlLIqyG1kJoMArcu2P9oH2OpVRiwPXno9MtuZDNZFaig7LABdodHsijZ2aE/7Kd5XrMORu34ns0jlWtYF/nst45+aAVde8GYD8w3kEln/cN8Sv03sZ3kg2Xlaay9R2WdNlHrn0e5f/t366jjqwDKbL70zde/S5T1pI/zO8EAkwjcnhHMVpkNLt4xzF0kjXZqjK3zVX05HhNNX9jBaiFWKro2SEMO3h43Pmke5eKhWfjYQHcj6qP+pOdMjgeDklLBMDltQTKspJeQtzttiB3UmVxAPUMF8WEhGTT8M+hf5fIPKI6pox/klk4DPdNroQux8r7DN9jJcgmEdHVsSE8nPWERsx6NVX+GIA1UAXwK1fpWyfW/wd5bq8YITrTIld/gWVpf0A4+4MF/W/YSKDY26AA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66556008)(66946007)(54906003)(4326008)(53546011)(316002)(6916009)(508600001)(956004)(66476007)(86362001)(26005)(5660300002)(6486002)(186003)(4744005)(38100700002)(8676002)(6666004)(85182001)(6496006)(8936002)(9686003)(2906002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U2RQV1FJUUpnNHZVSThpWnlET2JMVUsxaTJGVXNSZktya3B2T2xlUlUyRmdP?=
 =?utf-8?B?aitFMWtFWkRKTTRHbGFuVFU0eWtvQjcvRDd1d1hValFhdFJxMEMyWUVNOHhT?=
 =?utf-8?B?RWdhNG1MTzNlQXpMaGZDa21saDZlaGhrYjB0c2RRd0dyZTNrV3Z4aFI3OVVC?=
 =?utf-8?B?N05la2lwQnA2VjZxV3NnVFpZSHhVVDFVMjdML3hQSFZFaDh3NFZJK0lpZDgv?=
 =?utf-8?B?a0FNSmVZQXpobWdoNjNBOE1uU3ZPeGhVampsSGxCWWJVVUtxbVdRWVo5Vk1q?=
 =?utf-8?B?Wldtdkl4NTJXTjJvNlNOcHI5NHpMbkw5ZWYra3l0NzNSbUdEdjAwcFJ2V3Ex?=
 =?utf-8?B?SHB6bmNySWluOUp3cGVMYVR5VlQ0QTJuQVp5S3VySC9IeW5IYzZDSWp6V0Uw?=
 =?utf-8?B?c3h5SVFmN3M5L1VxUno4eWZFK2NpWEErREhaOVpwa0JUK3d4SEZ4NEFsTXNN?=
 =?utf-8?B?OXVPYnZIdDdBWnpucGhjMHQ2L0NkcWJ2cTN1cDhYUGUvZ0hDVTZybysyWngy?=
 =?utf-8?B?VXRwWEFBcjFEZ3pOY240dExPMTVqaVpJTG0zTmxUVHdVNThpRysvWWZEeitI?=
 =?utf-8?B?YTl1UUEzdzA2WmNacG9VRFFyZldZK0lZZ1VoY1BNMDdzRFVIblg3ZWU1NGNa?=
 =?utf-8?B?bGFSd2xzSkYzSVFFMllOWDBnb0JxU2ZPekFaZy9RMS9qSGx6YS83aE54NDJL?=
 =?utf-8?B?ZmxNRGdvL2p1K281Yk5KZno1UXZOVm9hZmlFRWFESGR6KytRNFZQejFkT21m?=
 =?utf-8?B?SlhDQmFxaXNKU21iZkI3YUhRam9xSjNtNzllOE1JanB4cjBxSzhnazNaMWEx?=
 =?utf-8?B?dUlWTW1ub2U3T1Y2QWRYSDFXaURIaUJYZ3pzcEFLZFl2ZkdxdDFnK1JLc1dk?=
 =?utf-8?B?UHZBSzlVNzhMaUphRlhCcFdDc0VJbEh4b3NMY3gwRHdvbTJZcUxHcE9SdWU2?=
 =?utf-8?B?TTVWWlVTQ0tja2lNV2d1YXd5U1Zqc2cweE01MjM2RUcyWE9ZRUhXTGhrL3dM?=
 =?utf-8?B?UWkxRGloQVloeUJ3NUVkZjk3ejc3VXpYYzJlSnZvc2hOT2w1K2hQMlJCZERr?=
 =?utf-8?B?bm5seEdLSDEwLzZubStvOW1VckMxUXlncmQ3dnFla0JVZkdtcWE4VTY2RE9C?=
 =?utf-8?B?bkNIUnNtM2d5UEF5K0ZXZzhsVkFxVis1VWM4RVIrbHB0SGxjeXE5WDFGRXA1?=
 =?utf-8?B?bTJVSmh5a3hYMk00N3VHRk10Z0hLdjBOWGFHRWwwdnVKMUlhbkpuS0RYVVVh?=
 =?utf-8?B?a2lMdXRBSWhOdGxqaXJJRGgxWkZmb2NabGdoSXA3cXcxNStRY05LWVA1cmZR?=
 =?utf-8?B?WFpDVVNuQzRyQUJCNG9qME55b1o0MDZtT1ROZ2JTNEsvYldRVmhtRk5mQzNJ?=
 =?utf-8?B?WlBPSlhKT2VDNFcwVTZ1OXQrUWlyT29IcHdOeEpEajNVdmVOclpydHNtM1hx?=
 =?utf-8?B?WkJ5NmR5UDlrRnM0OFBEU3ZjSHpUVW9kTXhNUU1wclhxbWc0aWt1ancyMWVW?=
 =?utf-8?B?WmJ5MDV6aGowWTF1SGt1TGxpczFlY3FsWmNzV252TFp4bVYwWFA3cnVjdFZk?=
 =?utf-8?B?ZjJJRDM5QlVtNk1GQ2dIQlpLNERXN3lSMDZuNE5xcUZNMGx5Ni9qMjFrUlJX?=
 =?utf-8?B?dnhKSElWNXV1OWlSQS9jS0JLVlhNNXJiNi84bW5QNWc5Y3FSRkFqcUI4aUc0?=
 =?utf-8?B?SGtwRWhIRlNFUlhoSWVEVWV1c08rSTFMMzcwUDlpbTNrUm1PUHZYbkxtVWVn?=
 =?utf-8?Q?f49KFaOae/UWp+pHHw2CM+zVBIBl/wH3Jjle4II?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 67b169fb-77a0-4727-0864-08d97cecf031
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 10:45:33.4492
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bII6LxbqigGKM9KDo2Eig3KXYraB48HuQMzc5GTs5nfjkZ+L2nkeTEV3+IEKejdqwAvZaxmCnmJJtqG92ndEOw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2555
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 12:28:12PM +0200, Jan Beulich wrote:
> On 21.09.2021 11:20, Roger Pau Monné wrote:
> > On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
> >> The function may fail; it is not correct to indicate "success" in this
> >> case up the call stack. Mark the function must-check to prove all
> >> cases have been caught (and no new ones will get introduced).
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > Acked-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks. Albeit strictly speaking an ack here isn't enough for the change
> to go in, it would need to be R-b or come from a REST maintainer.

Oh, FE:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 10:49:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 10:49:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191675.341760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdLN-0001qu-Ed; Tue, 21 Sep 2021 10:49:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191675.341760; Tue, 21 Sep 2021 10:49:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdLN-0001qn-B7; Tue, 21 Sep 2021 10:49:45 +0000
Received: by outflank-mailman (input) for mailman id 191675;
 Tue, 21 Sep 2021 10:49:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSdLM-0001qh-0R
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:49:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSdLL-00058I-V6
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:49:43 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mSdLL-0005fU-UI
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 10:49:43 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mSdLG-0008WV-Hn; Tue, 21 Sep 2021 11:49:38 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=FDpYBi4xl1i7LGDy2Rlx9YN0f1CD//FY8aTJuUjZ+0Q=; b=rmGHQ+I8F5H5G2Z1joRRCuFEqa
	jpuqAvovuWjvovTMhqMBG0UdR8kh9Nvylcuwk6qqnSX+cQGp0fb5qClLrlhX3DPiCDqwHooIpqQY3
	dL+85l6SAzmv8Q52ji8cEQ28w4kttvqMqDZCro7p2LWPn790Gr9D4sOdBdmVPsLUDwnY=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24905.47298.105494.711706@mariner.uk.xensource.com>
Date: Tue, 21 Sep 2021 11:49:38 +0100
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>,
    "xen-devel\@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
    "Andrew  Cooper" <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Julien Grall <julien@xen.org>,
    "Stefano Stabellini" <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
In-Reply-To: <YUm3yLOfAfqrwgmH@MacBook-Air-de-Roger.local>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
	<YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
	<58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com>
	<YUm3yLOfAfqrwgmH@MacBook-Air-de-Roger.local>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monn writes ("Re: [PATCH] common: guest_physmap_add_page()'s return value needs checking"):
> On Tue, Sep 21, 2021 at 12:28:12PM +0200, Jan Beulich wrote:
> > On 21.09.2021 11:20, Roger Pau Monn wrote:
> > > On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
> > >> The function may fail; it is not correct to indicate "success" in this
> > >> case up the call stack. Mark the function must-check to prove all
> > >> cases have been caught (and no new ones will get introduced).
> > >>
> > >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > > 
> > > Acked-by: Roger Pau Monn <roger.pau@citrix.com>
> > 
> > Thanks. Albeit strictly speaking an ack here isn't enough for the change
> > to go in, it would need to be R-b or come from a REST maintainer.
> 
> Oh, FE:
> 
> Reviewed-by: Roger Pau Monn <roger.pau@citrix.com>

Acked-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 11:00:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 11:00:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191687.341772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdVs-0004An-Fc; Tue, 21 Sep 2021 11:00:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191687.341772; Tue, 21 Sep 2021 11:00:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdVs-0004Ag-C0; Tue, 21 Sep 2021 11:00:36 +0000
Received: by outflank-mailman (input) for mailman id 191687;
 Tue, 21 Sep 2021 11:00:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSdVr-0004Aa-7i
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 11:00:35 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f1c060f0-2c04-4a12-9543-429642db4231;
 Tue, 21 Sep 2021 11:00:34 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2052.outbound.protection.outlook.com [104.47.0.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-Kp8Pgw24OsWtIg6pJp0BAA-1; Tue, 21 Sep 2021 13:00:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 11:00:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 11:00:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0090.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Tue, 21 Sep 2021 11:00:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1c060f0-2c04-4a12-9543-429642db4231
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632222033;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/gCvTzZkj/z1tXgg7O3hcPgmWbwdjjoZyAReUsMu0cA=;
	b=TB0Wwa0vnSYUzLe9I+rsBB/DZtYXNYKoiGdE1lWTA950xwdmZ6KnQm9oTy6IK1PEKm34+4
	OUhPEDIRAndbdJbZ6xXo5yYHbQXOqTY/XHe9Hov79/g3HkSx+xXBeyQwVRoZvuMeoSznXk
	lAqvd3iguMYAMuICzLTIA2Yc5e/Pabw=
X-MC-Unique: Kp8Pgw24OsWtIg6pJp0BAA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GlyLMYuGgeJ2/eqIVv8g2NrtW4gxHlGCbSpR0TC245BT2z04hMh4gie6rzf7fiyuhQ3pgrvkj7sVWcAX3ign7qZxJMDlf4mh4yoC5v367MOBoLdtEvm1XRUPMtCY8va2MFus6CLWRlWC2JBYlt0YacSzLe74PW7S1WHa2Il6WEnt4m3o6ncmAb31rEjKw3R079OC2Lgegd2SZ1TAVpDJOhu839ytWu592j8g2yyJdMYP8LbujRGzmC9gUNm2V1McJnhZl6covxGHEpjVIq2DWZThcfKu6ghO4a7YVYo3nSgSlas8tLko7DiEvkIBCTZ7uAfE+37PL0wzHI58qtiYhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/gCvTzZkj/z1tXgg7O3hcPgmWbwdjjoZyAReUsMu0cA=;
 b=jaZPwz5B/j7SlxEatMOjDkLX6jFv1/GGa4eukjjf2NAUnQuttQbtrgTjFp6rGZ9t6O9v73+RLXS0IfZWc+Q22i2OiDwHAKvbDAXwQE8aWcp9BglphM6odLMfttbFkESGzy1yAiqPTO8m1JCqo36xznIUSs53FWVxdB+/o4C9AS34QkCCD/Fe+dB1v2+o2eY6oQIXKrCgKALegb/RsdTkEigSeM+5wWZlU23c1hN8iQrc+oVEC6QEcHbo1aItrbRpJr77qaawWXcw5FRMNBSxJo7HN/x2wQJpkxUlSZZmNDTBVMiqohvd6oMfSrPXnCJ6o16rVFKnUfRbuiKPb8Gb1g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 04/12] x86/hvm: Reduce stack usage from HVMTRACE_ND()
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-5-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1ec8d289-5c34-412c-c683-189cd5506600@suse.com>
Date: Tue, 21 Sep 2021 13:00:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-5-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0090.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b5b9e368-2db5-4288-2b73-08d97cef06cf
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7150A90F296B434AD3680418B3A19@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kgXImTHDr0DXHCV8CCKOjclExEC5pu84jyhGmAimq7MdWkgS8mjKOkGLPIAlCAOgpP7PMFv7vsTIjZkjDmpkjHUNZOF+UPs/uUZ7qpRQABW8kySYhiADNLDZCorJjKardykD0vdZ2HUR8TeBkmb7jfdzML4szAflLkm01DzAwWUAck04qNwcXCwLTHLWWVI5+jf5UUQs7nAveZflEZif8e1mv12Xadqik4w9ooKati3q4Y49vzUqBSdLsfjgXRg8Hi+UhDaBjc1Ow/oB9g5ibUcjTy8PGzww922MvEDKaikecW8pZyhN8beG0y5zX2rgYTzxfqkTujc1t78w+67YQq+4KekxmwIR5E29FUnBxwNDjhhcEZgQsIPefCC4Qh0lL0cboybFwSMfADIX2pwwSbPT9ZnHO2FiI0ptnJAdthyMr1zjPL+xETYBmvkeVwf3PPgVx3ssc/VxwmFYHUp1v+K4Xhvc5vIg0vKFi/wV2iun3O8DICCu8UssvyshdZ/66it0XuEzeXRN7FfZfwjHZLvW3uvWPdGuJ1Dvxl/C9nCpONGyGmcoyHpfxKLY0Ce4k5GYiguZkGOO9IaVEOvv+Xul++FW2spT7I43IasKOwt0UnQTgILJan2yfNU/KDD7OUeWATbA5U4dxUjRJ1GW8GHI/Mou1gtkcMfTqbKx4sgm5I/E0vwGECGaqWT/NnMGCHIjJFdBWSPSRa52s4qE7Glw0zBOE+8aJoI5DtDCaU4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(5660300002)(8936002)(31686004)(2616005)(2906002)(36756003)(316002)(16576012)(54906003)(508600001)(66476007)(8676002)(86362001)(66946007)(66556008)(26005)(38100700002)(31696002)(186003)(53546011)(6486002)(4326008)(6916009)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MEJQVHBQc1dwZnBMTHpFMWRmZCtuZG94MTcwQUJBYTNoczZuWHorNjFJUUcw?=
 =?utf-8?B?SzBqNzl2TXU4YUNTOE9MNEg3aXpGSXdCcTZwMUpuaEk4WEZWMk4xMHp1YUxQ?=
 =?utf-8?B?bjdSODVCa2dSQStLd2Y3QjF2L1d4ajk0cnVHaFU5T0hoaE9Td0RHUlIrdWF2?=
 =?utf-8?B?Tm5yNnFiUUFOVzZudW1oYkdwcEZrT0llWm9GdGZUbElTWTR4Z0dGUkJIMjdv?=
 =?utf-8?B?cDlUSkJsSmRjd2NGbDBwaWZVWWhnS3RiT0lVQzdrU1V0NDJuMmpmMFJxSUJ3?=
 =?utf-8?B?MEk5YlM4cU5FM3NhWEVyQW5TNHIyV0tGU0phSnJwenpZU2RyYjdiVFNUV054?=
 =?utf-8?B?OTQyVkhubjdQSDYwQ3g1T1BxdHhWMEs3UFVwUHlGL3l2WnVNN2poZWpST0JQ?=
 =?utf-8?B?eFFTOU1wSDB4M3B0dFR4NkR4cjYrMHlOa1I4aWhFLzZIZmp5MGl5OExTTE5T?=
 =?utf-8?B?dFJpOEFUcWV3WlZOZ01UUWNGVkRWT2RUSHFiZG9OaVlaYmJLS3hHeHpWdEVq?=
 =?utf-8?B?dDVmd2YxR0htajIrb2NuOXZGWlM4TE1WRS96MXFXSHpCMy9oM29nTnFyMDF0?=
 =?utf-8?B?aFpET01FbTlVZHNpUXN3MnA5am9wY09ZWEFiUDdHUFVHdXFFMXA2VUJpUkNL?=
 =?utf-8?B?S3JrYk9KbzZFbFhsekFKbWhaOTFXTGNVd0lpLzk2bDlBeUcrdTRhbWh5ZC8w?=
 =?utf-8?B?cWZtVlZFUURTQThFYTJMbThtNTNlbnlMVUd5WDlsZ2c4c3F6cnNQVE05c0ww?=
 =?utf-8?B?SFJiR21mUkRrWEE4RnhaMzVXWEdSL05wckVNMVNNZEtFQmNVSHFHdWh5QlBz?=
 =?utf-8?B?bHV4aFM0WVpMYnUyMTNKcHdpa0FVYlR1aG5GR2t5c21vR2JGK1ZiME1WVVZJ?=
 =?utf-8?B?bW9YZDJNRVBJZnRtK3NGaTVYLzhtNXNNSFlLRzVvT0MveENYdVowcDU1bjF2?=
 =?utf-8?B?L3AwV1l1eXFXYVVOVGVYTnpxNUc1UVpxTllWNStMU3JOTVppenRRU05qMzFM?=
 =?utf-8?B?TjNmdXdMOElmVVZvaDYzeGdqWVdqWmZKaGJwRjNCTFFpZS9VU3hkRXoxVHlW?=
 =?utf-8?B?RVdPMkxCalhXbE53eW12RDNuL044eGdsYWhVS3NqWTlMQm96eXNzRzB0Qm9P?=
 =?utf-8?B?U2pUVGZjaDV2YU40KzEwNjRwOTVlTUNicVYwMFJyVXVhOFl1MmhOMjgrb0Zk?=
 =?utf-8?B?RjdwRzVvdjVNMmd2RUpvYWJadk1PUE13WWRMc2JiaXAxMERJQmlCZFhKZFEv?=
 =?utf-8?B?UTJQVWx3dFhNTDZ3RnFQM1FnZnRSTkFHTmZURlRxVEgxckxacHYrRndoVE1P?=
 =?utf-8?B?ZHBkRk9KQTlzWUh4bnRmWmQ5eC94WTlYdlNCSndkU01iR2xhbzdCMHRLd3NC?=
 =?utf-8?B?LzBNc0RGaVdna3pZbExJZTRPUDByWVZhVEJRZjl5Wnc5OEpoVktyWjVhZFpH?=
 =?utf-8?B?aFJLYVV6ek1lTnNpcTB0VWM0c0hxYmFPUlFlNVdNWDFCRGtZYVJuUkduOXRy?=
 =?utf-8?B?bTd0Y3lnWnN1QlR0ay8wNFBSQ2dOZHlNZGZtaml5T3JvNk1xY1M5dTMxVmI5?=
 =?utf-8?B?QlFlVXgzMGIzNlh2cFFmRmtuNlo2aUs1RTJsZ2NSdmg0dkpmREpFUC9UTk10?=
 =?utf-8?B?bmpkSjhQT1p6QW5NcFZLR2RUQlNRaXI5WUZFSDN1Q3l3THp1N2YzS1R4LzRn?=
 =?utf-8?B?VDVrQWU1N3Joc2Y2ejdzZURYS2tHWnBGaHh3Z2V2VUhMNmlyaWwyMWh0M090?=
 =?utf-8?Q?70Z2emFzBAzS7rlaAyGWeD1zGd5nGnmEweN8ByF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5b9e368-2db5-4288-2b73-08d97cef06cf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 11:00:30.3893
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dfvxtIkPSSspag7WeUaPsq7H9ln/ESNOsP1lXIw0DSG4ObwL+1TgH2RGWxfinQ0+WBMgHOXI0viKi0h7XN7rgg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

On 20.09.2021 19:25, Andrew Cooper wrote:
> v2:
>  * Adjust callers of HVMTRACE_ND() too

What does this refer to? The sole difference to v1 that I can spot
is ...

>  * Drop _d[] for the 0 case.

... the one corresponding to this line, i.e. ...

> --- a/xen/include/asm-x86/hvm/trace.h
> +++ b/xen/include/asm-x86/hvm/trace.h
> @@ -67,38 +67,30 @@
>  #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
>      TRACE_6D(_e, d1, d2, d3, d4)
>  
> -#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d6) \
> +#define HVMTRACE_ND(evt, modifier, cycles, ...)                           \
>      do {                                                                  \
>          if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )               \
>          {                                                                 \
> -            struct {                                                      \
> -                u32 d[6];                                                 \
> -            } _d;                                                         \
> -            _d.d[0]=(d1);                                                 \
> -            _d.d[1]=(d2);                                                 \
> -            _d.d[2]=(d3);                                                 \
> -            _d.d[3]=(d4);                                                 \
> -            _d.d[4]=(d5);                                                 \
> -            _d.d[5]=(d6);                                                 \
> +            uint32_t _d[] = { __VA_ARGS__ };                              \
>              __trace_var(TRC_HVM_ ## evt | (modifier), cycles,             \
> -                        sizeof(*_d.d) * count, &_d);                      \
> +                        sizeof(_d), sizeof(_d) ? _d : NULL);              \

... the addition of a conditional operator here (which I assume was
something a particular compiler didn't like in v1). FAOD - I'm fine
with the change, but I fear I'm overlooking something (again).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 11:04:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 11:04:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191693.341783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdZF-0004op-0L; Tue, 21 Sep 2021 11:04:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191693.341783; Tue, 21 Sep 2021 11:04:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSdZE-0004oi-SV; Tue, 21 Sep 2021 11:04:04 +0000
Received: by outflank-mailman (input) for mailman id 191693;
 Tue, 21 Sep 2021 11:04:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSdZD-0004oM-Ka
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 11:04:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4769d6db-fcb4-49a2-b376-abeaf72c60e7;
 Tue, 21 Sep 2021 11:04:02 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-3sxkIoiFM9Guz46mZtlivA-2; Tue, 21 Sep 2021 13:04:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 11:03:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 11:03:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0019.eurprd01.prod.exchangelabs.com (2603:10a6:102::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17 via Frontend
 Transport; Tue, 21 Sep 2021 11:03:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4769d6db-fcb4-49a2-b376-abeaf72c60e7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632222241;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=o9ZHWeV00VO1M6yzaBpD0BDihqnJQoJNkdDkA7YiaxY=;
	b=fjsYSDHOPStS3HgCJRosKdXFECwqaqGqvhTMjZzJUy+aupm3nKyoERgtfmXeyNfCtmlWS4
	MAnJaELislfJmh96qeTn2ktu8QvQDC/oZ2wgYmf+kwwkp8DecvN5aAXHaNsPlDY8e4fwU6
	0yUZaOWiJU1Dz5IGqwBKTy3qc6aDbqo=
X-MC-Unique: 3sxkIoiFM9Guz46mZtlivA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kHt2S79wS6xuHYXTN3pHMKGdh4BIzkb5va0HE87ZD6vpI7gcp9Krv4NJHlAiEy+BSvp/DBeqSqS0CkdVRr6445Qm0n7Dm++aU6DybEvGQ6gTBo90ElTHI2wlrQD+UZEdGhmEtBM55/XWFvBBX/8d3TMjCL5X2pmLnJhJDTi0ujhT0wYJ6JngSQQwhED3BZFLBKVaqCeldQaTJmo8RgDZMMi5hUAMbJyIoAq03wNBAjqgNPcSc84yVrMsZGDX7mr4dgJS4a60Ngrs7IMEPYCptwyRty/U68ICHptMdKl7aEk7pTUC+xI2GFHMWKw5I+eDc3LwXo31moTjJ/P7BEY14g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=o9ZHWeV00VO1M6yzaBpD0BDihqnJQoJNkdDkA7YiaxY=;
 b=ndsXqRJ8In7kKJRzMfh6CMdFcb/StU+M2jorAhsQAynv/t+ko1thKo/C2JYGj14AQ12+/IzFBKFHCjqLz3OJOLwW+dZ9pSVaRiwSK+JGXtHros+hnTDjtCgp7VMORX0lTlYnQUDqufnXLaOncSio64RiceO1IBQGq7Y6FuNfeye3y1I8Yql8/knbTfRz/FtEbsaRVzwqHk5E5Rk9dCC9D7bFdRdwwzWOUo7s0PsP/3KwOC1+o00jxjkPMaphYqox9+Q5LK0u1v4wqg1jdV7CJggkQ8uxHSm7bJ0lpzQs23txe6CNSkk6729qF6dTucfyF72b6ZYe6ej1/sxhO3Dghg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 09/12] xen/trace: Minor code cleanup
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-10-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c34d236d-d749-5d33-207a-94fef482577f@suse.com>
Date: Tue, 21 Sep 2021 13:03:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-10-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0019.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b1b43a46-9a81-45c6-77c9-08d97cef8193
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB73604E2CDA7C9E3EB41A0429B3A19@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mZcCWJNEKX7J3t9mHd5y56w2k8JgLNapiPGDmXeJDYosuexLQ/EbkX11tyC34G4yhtHObOjBltsuysAOmVyoSLaHUblB6FAROp8KFXqk9LnkcnSSCjcx/Eki6+75g6LzE7n7UAxJ7CphwMZSI63/ESgux+RPfBZIyLBWeGBmTDC/1d3/xKmCgWv1uhrQimkmQifhPyyG9ujzg9PEPGYxOr4h/38TkNpgCss1e/gfEg/ogWAjWiJaxKGdVSsuAhcaSdwgvG06JOMxIPa5/E7NkAJzTplUMTGczAj+FdU1mTNiy7bWOrpwLG+Y1AFEPW5/fPjyYl+PLcdhHo+SFYm4KcKJ0xVZJNsivt++tQD76jbGD/qm9jUlNK2VayhsqGFOZaf1E3PZCLhcrgQCyQ1Yc0snKjNVzo5IuPO0jQVIxzb+F6CmJqNkZfPc+g+O1ICUo8e6HS8pYRsysvgbOhhnRh5JYVPKPedRccvHc+ZcJXijS/HM2L1dEofS8pn+Vqqev2E5eXSlfaWZVoc4MMG0kt/W7guqWiEGAawGA8TnQiJ+d4cJ+YzymHP9LQEv1n85fKtYBERJrmJkW00gfFBS69M8N+or5X9u/XIiwreXXrhRSwtn52JEmPgBegAKrQDOqqGvb9EFKJgRC82FqnRR3C0cDd6bT+PKz26XOIPTJiEnfNbTYzLrZNnkB2V0+dsENsnTj/dBtijIZyHHzhj3Tg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(5660300002)(8936002)(31686004)(2616005)(4744005)(2906002)(36756003)(316002)(16576012)(54906003)(508600001)(66476007)(8676002)(86362001)(66946007)(66556008)(26005)(38100700002)(31696002)(186003)(53546011)(6486002)(4326008)(6916009)(83380400001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZnFBbjNZaytPUlY3K3FwRUpXTllHdTJueDIxWi9uU0haYzlRV0RKbnlnTmd3?=
 =?utf-8?B?c3BUcEh0WklNdVd3SWNuUFVPOEpEdmxsTVpXQzFJemwydFlIMmRHblBpcDNt?=
 =?utf-8?B?akM1ZE5URkEyUzJpbFo0ZTBIdklmUDFmWE5tc2ZvRG9jZTl6ZUlPSjJVc1pt?=
 =?utf-8?B?bVJRWEhvRGpMak5XbDU2TzJ1VGlGQ0xETS9ycWtXYitONWs5bTN0dGNEWlFp?=
 =?utf-8?B?a2pIb0gySlR4MzdQUSszcjFaZklQTSs3K3hhczBXVWxTSkxIa2ovRnJodTJk?=
 =?utf-8?B?TzBKT1dOamRDNk1pQ0s0L3ZpcVN3RE1GOEpYSlJpczA3UVg4VmFCWEVSWGpx?=
 =?utf-8?B?U29VSHdFT3hObDRuUWZvTVFQK3J0SERZZzFIRVRMSklNRDRpQnZSNzYvbWRm?=
 =?utf-8?B?MHJIdkpuakEycTlJS1VCUUtyZ1lCenFISTNYVjlIS01SNGNlWEl3eHFoK2tk?=
 =?utf-8?B?MDJDVU9JYVAvS0lmMC9jV2Z4UFQ0dzBSUjRXMU51d1FBMjBoMEp1a0dqK1JL?=
 =?utf-8?B?MVJFbmtIcytRZkxESVhKNzVMVVZlRi9FUEZISDZOZWxBS3JDRTE1WWRIb2wr?=
 =?utf-8?B?eGlpcmp3SzFQbXhjUm9nQXBqUklLRmJSVmg4a3BiMlB2cE9tS3lLQkFzZno3?=
 =?utf-8?B?MEpXaisrcEJvUDNaLzF2eEdHUGxGVmk5NnB5ek1ZWi9yYmRzV29DaGNhdzhY?=
 =?utf-8?B?ZXV3RjVFK1cyWlQzeCtBSldyY2lpUDdnMmVNd2E4ZEhHZXlCT0ZPUVpqZ3VT?=
 =?utf-8?B?aGkwaFFRTkxxbHpOeWx3anRES2lSaUFEYkphRzFPMlBPeDZ0Zkh2TjZ3aEhw?=
 =?utf-8?B?aEtNalpjS3BwTC9FYTBNWEQzNGFSS0VNckI4ajhjL3FZcWNncU5hTmd3NkFm?=
 =?utf-8?B?czdwWjd0NUQ2em5IVTc1WEZPSHY2QnM1dk41ZXM2aUZQb3MvZjk3a1NPd0pz?=
 =?utf-8?B?Q2JJTVJDamd3amtiL2lpNFFIajlVR3ZObk1RYmk5eUNhdFRBQlJ3bHRXTU9V?=
 =?utf-8?B?bk1vNGdNaXRRMG5wNktyV0JiaEFSdzdwMlREa0xzYzVlY3VhVHdXTG5uQlZx?=
 =?utf-8?B?dHFMWEllcW1pbkdFTDNuMUlQb3FFbEtmQk53OVQ2TU5DR3ViV3hQKzNLbVZW?=
 =?utf-8?B?MEdIMTZqTlZZQUVaaXlRVklUdDF1a1VVNm9rWjlJU3EvOFBMQ2lUTHNxbVpT?=
 =?utf-8?B?bll5NlVXSDEzYk91UUVkS3BuRTM3WndPdjFicm9QYzBKbTl1VE0xKzdVRTFB?=
 =?utf-8?B?NDliRVdpWnVsUDd5Wkwzc0tMdldLWk9tRk5TR2p2akZyb0E3cFNOUFI3QnJz?=
 =?utf-8?B?dWZYZjVORHRwNFdvN2RnZkl0ZFJ4TEY4cWtzbnRmcHNMeUdOMENTQ3JkYzM2?=
 =?utf-8?B?VXVkQ05tQ2F3ZCtrcVJHWXc0VFpIOXJPNTZacjVjS09EMURLa3p5OUNhOFc3?=
 =?utf-8?B?Ukh5QkMxbEhkWDBSbzJJVDNpY0Exd21vNmdGYkVPWm9EelJvUnNIb1NpbG9a?=
 =?utf-8?B?Q2JSS0EvK2FiOHcxS2RNRWRyaGM3RU53cW1SbmZHM1dOTTZkOGk0SXFsdEI0?=
 =?utf-8?B?SUpFZTB1L0k1OE0zQk5oZHJzcmNHM2UwQjB6bFpoOEIvREZkbXBYTVdwb1BR?=
 =?utf-8?B?bllsc0JKK1B3QWdrZGszM1VpWnpLVWdUaTJBbjd0anAycm5PYnZRNXBrbEZ2?=
 =?utf-8?B?b2xiM0orZlA1RHdQL2hKM2QzT0htYmw1eHBHT1daVmZ0OXBOZUMzcDVISGNY?=
 =?utf-8?Q?EZ4c94yRL4g7TNAl+7abro1WDwPvOWQHUb72f3u?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b1b43a46-9a81-45c6-77c9-08d97cef8193
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 11:03:56.3548
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eRVTkOhAR3+E/wW5cIjKeEKY3aigCnkL8RlFFOqWCRxaSAA5s+p8sAkZQg/SxICDjmpTAUDMJGF43fZz/JuAwQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 20.09.2021 19:25, Andrew Cooper wrote:
>  * Delete trailing whitespace
>  * Replace an opencoded DIV_ROUND_UP()
>  * Drop bogus smp_rmb() - spin_lock_irqsave() has full smp_mb() semantics.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Like for v1: Largely
Reviewed-by: Jan Beulich <jbeulich@suse.com>
One remark:

> @@ -717,9 +713,6 @@ void __trace_var(u32 event, bool_t cycles, unsigned int extra,
>      if ( !cpumask_test_cpu(smp_processor_id(), &tb_cpu_mask) )
>          return;
>  
> -    /* Read tb_init_done /before/ t_bufs. */
> -    smp_rmb();
> -
>      spin_lock_irqsave(&this_cpu(t_lock), flags);
>  
>      buf = this_cpu(t_bufs);

I wonder whether the comment wouldn't be helpful to move down here,
in of course a slightly edited form (going from /before/ to /after/).

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 12:08:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 12:08:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191713.341794 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSeZM-0003GJ-UF; Tue, 21 Sep 2021 12:08:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191713.341794; Tue, 21 Sep 2021 12:08:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSeZM-0003GC-R7; Tue, 21 Sep 2021 12:08:16 +0000
Received: by outflank-mailman (input) for mailman id 191713;
 Tue, 21 Sep 2021 12:08:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSeZL-0003G2-UH; Tue, 21 Sep 2021 12:08:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSeZL-0006Ra-P1; Tue, 21 Sep 2021 12:08:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSeZL-0005jM-9B; Tue, 21 Sep 2021 12:08:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSeZL-0001U3-8m; Tue, 21 Sep 2021 12:08:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=e8Pt6tH6QCy5qmvfQr62QSCHppN5anKCdHzeZ9BZQjc=; b=ignU/SO7VV+n3TpD9SY/c6e/O1
	WYl0xldL7cBpIjmIryqbh6gV7rDvddT8lwt/xcupjPaWlJyQKLXlkSXO6XQ6tON9U9h91TjZ95TtP
	rh3Lb4jWmg6y0YP4CbnSQOOrwQZaeLJ+mbA7Mw0QV/NL2+9lftH/9+xfp7VtJSATcHxo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165137-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165137: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=bdac3a142cb1788846f605f3dfb365ff32b356cc
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 12:08:15 +0000

flight 165137 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165137/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              bdac3a142cb1788846f605f3dfb365ff32b356cc
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  438 days
Failing since        151818  2020-07-11 04:18:52 Z  437 days  428 attempts
Testing same since   165137  2021-09-21 04:19:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 74374 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 12:10:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 12:10:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191720.341808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSebn-0004bU-Cr; Tue, 21 Sep 2021 12:10:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191720.341808; Tue, 21 Sep 2021 12:10:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSebn-0004bN-8m; Tue, 21 Sep 2021 12:10:47 +0000
Received: by outflank-mailman (input) for mailman id 191720;
 Tue, 21 Sep 2021 12:10:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSebl-0004bB-Ix; Tue, 21 Sep 2021 12:10:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSebl-0006TZ-F6; Tue, 21 Sep 2021 12:10:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSebl-0005pP-7R; Tue, 21 Sep 2021 12:10:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSebl-0004tn-6v; Tue, 21 Sep 2021 12:10:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Jo3ioi9N20Ov4ekGmZgpG0spY/HmnkZ9K9dokWI5yqw=; b=D/+2Z6S/7qkSRZU5Q+H9tnvIjh
	Icx/jM7ZMAQqQ15sCcBLplE6u4qH3ko8L2pDrqLECELwbRDk0MIsTbhu18VrKV2Yxola3vci/OvKV
	Qoxdpy+NNDNdQL6Up3Qvq0iBYvAuImWadieypB8Dg9M5vXHdYtICd6JRRQSmdu8d5TOQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165134-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165134: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-examine:reboot:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4c17ca27923c16fd73bbb9ad033c7d749c3bcfcc
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 12:10:45 +0000

flight 165134 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165134/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 14 guest-start              fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4c17ca27923c16fd73bbb9ad033c7d749c3bcfcc
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  416 days
Failing since        152366  2020-08-01 20:49:34 Z  415 days  725 attempts
Testing same since   165134  2021-09-20 23:11:35 Z    0 days    1 attempts

------------------------------------------------------------
7321 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     fail    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2262815 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 12:19:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 12:19:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191731.341822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSejo-0005ZY-F0; Tue, 21 Sep 2021 12:19:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191731.341822; Tue, 21 Sep 2021 12:19:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSejo-0005ZR-As; Tue, 21 Sep 2021 12:19:04 +0000
Received: by outflank-mailman (input) for mailman id 191731;
 Tue, 21 Sep 2021 12:19:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSejm-0005ZL-Sz
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 12:19:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 73459a3c-8225-479c-b75e-5e90d141ed38;
 Tue, 21 Sep 2021 12:19:01 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2051.outbound.protection.outlook.com [104.47.10.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-o3gr2-NlOuS7kZlNqQsZjQ-1; Tue, 21 Sep 2021 14:18:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7151.eurprd04.prod.outlook.com (2603:10a6:800:129::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 21 Sep
 2021 12:18:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 12:18:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR08CA0008.eurprd08.prod.outlook.com (2603:10a6:208:d2::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 12:18:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73459a3c-8225-479c-b75e-5e90d141ed38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632226740;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=BPsGZx1BZ9bZll4oOKgFeSjcCZ1FW9k+MZvkR+BufVA=;
	b=BDEi13Qz553u0rnbXkPxS9YJq7B+pxen/z58/5xymkedGh+JVirfcmqzbyeWbM++6k4pZp
	5OHe7OwuUd8OL7kED1jcGhHiCR9JzCnxjDqqAyCK0H6R+xT7IfKNg/+4lu7ucsgElhDpSZ
	rc0HArnQquOJzV898Fxp6tfZhLrT7Oo=
X-MC-Unique: o3gr2-NlOuS7kZlNqQsZjQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QH1vvyo4fGgK1OkXnHuV6ioH8VkmpAQTMO0AQm2Qc15bxZJe28JUNddyc9jwGEBQkXVA/ijzjh6ZsLjZ6oaKKKCshUdsNuhrF6s/P1/6lIryPzgNLTqOFlfBB+fQn8ZMwLsglEGq7ap+HTQYHpKKtg/s+njeoEpKCTKwS/hRwNXRtb6KO/JPakNgNdRXAfyempzLKH5BWBCW6eIFgDU2Q/Zt3S/bKCn39KDhPozXwL2K8CJvlrOJin18r4jRJsgQt+EfQR8Fl9HytW/pDTZp3735i7CcrR77DZZkd3o1cSe5jx5XhF6d9OiaItCaNPMVFamM/6bwdlFXfksEwjh/ZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=BPsGZx1BZ9bZll4oOKgFeSjcCZ1FW9k+MZvkR+BufVA=;
 b=F575BJA5gLWHM87gJaB1141YsNAfXZonAewFzgCgmoBLtutCkP5ryn+EmNms2t9mfJst9ISPhxMjjt/9UbSCchSIsREo1QlUVUhRvs812uYPjNoLaAseBhtkz4pH7gyAxDJ14HnsSIi/HOqMtrr6NYFZovaQlw5XOQitRnSJg3kiv/VDyDXGCQstGKAAQAzbB2xqVLdaP0qn4XOuRaMVSd7ZQE+2mkmcVnNBKojCSY2Fb3M8h6qlESSK7QhEJH2sp04wT+DWNVLcliwfz0Eh9SXrIHnPJs0S8eZBnwBVe+R3R9ws/fjXdgflspOs8YevsgkvftJYLYXjhNHP2eH2Nw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/12] x86/hvm: Remove duplicate calls caused by
 tracing
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-6-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <45d5ba73-3099-9744-41f2-e3f53faed536@suse.com>
Date: Tue, 21 Sep 2021 14:18:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-6-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR08CA0008.eurprd08.prod.outlook.com
 (2603:10a6:208:d2::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cc1e5405-761a-4e0c-8636-08d97cf9fc28
X-MS-TrafficTypeDiagnostic: VI1PR04MB7151:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7151A86B1A79F5BD2113826DB3A19@VI1PR04MB7151.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CbrFdPVf9vbiiwEBGnwFnWqtczCEM6+1Zvz9UJtf0QYXQLCLdAhX436Nyjc03AoYWAG8a5RS2Nv8I2f1a/f18Ohfts6OboFZprkku62Wxvgu7gcG6b06kq7M3O1gt5VfvxTw8TZoK9hdBoQN1dtQuYRpF242a+e9aFHr9Ji530UzI7tFwqKolsYUtUXosP13qiJSAuNj0icmkbAYrfsegZq1i5t68Jg2131x3W8RojeDTKXb7McYO+uUqLMk0IeaNT9rU5Hvu/jdfx91MyqVU7io82psQfGV0dR7gue/KvIgkYhO6ZdPgJWiH6oQIRp4Q+8Uf6N5bUoiF4jkKdB6YSQsZpJmwIo/S7+Y61fS3vFd+NZQNLHA31BHMQmcfSazKzlObby5fe0UN/dEzKlbXF0juvMrly5lZIh5Fk6WkoJFDhnSWZZn2POLDWao3eqknU5mI11Mnr8flwkMOxzIM2ispFj9pVBwILyt55cJG4F1yUaZe6Teck/BPt1iridaQOjXPehd5iHkc43RX2XUwRDkid65rYJFifVEWN9vcCrEkX/Kq/++bAwv63BaH+KAxr9v8jPk2+HKaACaP7hjzjg3F75aYGXfsK1O8/wpgcs9E65D/edJxssvrlp6Z8dzGtINpfr0QQwocDquVeRNz14y2V7ofaUL5eigYn7ReTJv1llBB4DySOQmj/FRnT6gNc7FIeUZ7rAmbcJjtFFBCgQv6HLuHZzGojgFH3+2Prs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(956004)(2616005)(54906003)(8936002)(6916009)(4326008)(36756003)(2906002)(8676002)(31686004)(6486002)(5660300002)(26005)(316002)(38100700002)(186003)(31696002)(86362001)(508600001)(66476007)(66556008)(53546011)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SEpITmowam5pZlRTRHRoMXcvUmxSZlNvUHd1dTlKejIzb1ovVlNYdFhMOWNj?=
 =?utf-8?B?ZEFLTlVYd2VCU0g2NWRMOUpibFJnY1p3VkVhMnNsNnF1b0ZBanFFN3QyREkw?=
 =?utf-8?B?c2N5WGYzenV2bmVuc0RRcmZKZDRZY3B4aG1Qc285WFpzeTNZcXdhK01rR0NB?=
 =?utf-8?B?OUNkYWQzT3MwZVhFaW9UK0FnQkdlTmVreUdJa2FDRWV4MFRkalFHdVlQKzFv?=
 =?utf-8?B?eUUyR3RxY2lVMUp0ODNnOTlyY0xXTFZqZzdmQXhueDFqRVY1SnpzTkpaVkpF?=
 =?utf-8?B?NGI3dUY2bkh2aHFVeEdySEJhRGVoVXBDeUdmT0E1NFhNTVgxeGlZcVRCNDIw?=
 =?utf-8?B?bC9EbkRLQTRiMFVmMnF0RXRFdnczQThLdWF2c05sSHpSYVd5ZElHS25Mdkxy?=
 =?utf-8?B?WUQvR2NGMmNFenRBWmViZ2lSS1lIVlVMeGhOYmk5SVUyeGRhR2xyUHE4OEx4?=
 =?utf-8?B?VnVsRUUyVDlSbFdEemdDNzhnaVRSMXFCWTdBSEFzeEFMUmRJR1ZvYURUUnU4?=
 =?utf-8?B?emorcUNUOHk5eEJoZzVpc2pHUFBxSWFuSlBJeDQwNVd2WWZoZmc5dFl3MFdK?=
 =?utf-8?B?eDNMR2lqNnJPWldmVWRoR1U1R3hvelR4UjJoQ2U4WVNLcC9XT3ordGp5SFR6?=
 =?utf-8?B?dTdHNnA4TDdzeU5nMytTNS9nWThQeGVmRnlZcy9uR0g2TFJaV2wwYU9FTWs2?=
 =?utf-8?B?dEt6dHFRbnZIS3FYVVQ4di9UK1pWV2J0M3lrRkVTV1R4MEEwVHRtWGIyazZQ?=
 =?utf-8?B?N3RCdjBPcDgvYVY3V3RHZisxdjBaOTZYdkZhbkp6NGk1RFdINEluRU9FUzNa?=
 =?utf-8?B?ZVlYdjM0Rm83UnBsVUFRa1U0MHFNSzRWUkdmbXk5UUhRMDluMWdMNUUvMEM1?=
 =?utf-8?B?TWVKcU5mUGhTYTZCempEMzk5aUNhVEpFaEQ1VkhpK25GWkRlbExBZFVNMmNa?=
 =?utf-8?B?ZmJDL3JwdHpWa1FWUUlvTEVDZm9ISkZvaE9YYkFJaDZUU1d4TTFzd3dGWXJa?=
 =?utf-8?B?WDFiaXpYbGwzWFJnTkU0NG80TUlxRHhGbkExd2djTTBGZnUremxCWjdHbUFO?=
 =?utf-8?B?RGNpOGZReUt3QncyN21aOU1EZWg4QndtL0ZUeEc4U0J0Tkp2Q2RGdTVsNmlt?=
 =?utf-8?B?c3RUQVMzMEZpMVAvTWNPdUx5ZXZrejU1TXkzVDlyQ2R2dzd3ZmJzY3A4NitB?=
 =?utf-8?B?b1YrQWdHMzRtWjFpQ0orTytLUFN5SmlZOXROellIYllHSDZDL3BMQ2FmT0ZJ?=
 =?utf-8?B?TTFDMzNLZXRlMEkwODh3RHQxLzVHVG83TENIWG9uc2dyWVlVWVlDMWJQejVY?=
 =?utf-8?B?ODVNeEZwM1lIU0JlUzJNVEx4dUpLS3h5UmlzeDhqZnRHZjBpMDI0SGg2R1hi?=
 =?utf-8?B?MnR6aFBOd2Y2aFlkR3J0dTJLUkhEKzZDUFZkcTJ4MGZvMFBDRUNvODcxNS9i?=
 =?utf-8?B?aFJZYWtBRUNJL3BZWGtQM1pRaWpEdURhYmhWVCtBdVpqc2Zsd1hwVk1PZkZS?=
 =?utf-8?B?MStIc0tJUzNja2FtQnhWK3FKay91SXdkVkFtelJXQ0ZNaDN5cHBFZmt0U2Ey?=
 =?utf-8?B?Mk42YTZNK0R1YkFZYXYxc0tLdXBFekxweFZudDNIUUhoT3hEN1U2M2RRRmVl?=
 =?utf-8?B?NjVxZnZMeVgycVJ4UE1EUXBLZ2sxMWxLQnlhVGJvQkJlbHZWQkkyZ0NLSXFn?=
 =?utf-8?B?akpVR1U0UVo0WkhVYWhwNkx5TE9BVXFrVCsyUFBaQVloY3RMcEcyMlRsZzVR?=
 =?utf-8?Q?nOyq7GjlVaEZysqgAKIQWC9PkZjMpa+i2DOJZVn?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cc1e5405-761a-4e0c-8636-08d97cf9fc28
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 12:18:56.9990
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xb3IzypI0oV/Ob4WtXvnxh64SrePgpv8ukSJwPGbL5pO7Q9x+Y7IDadBVyQufid6GICOEzfPjK/BR7pppzF39A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7151

On 20.09.2021 19:25, Andrew Cooper wrote:
> 1) vpic_ack_pending_irq() calls vlapic_accept_pic_intr() twice, once in the
>    TRACE_2D() instantiation and once "for real".  Make the call only once.
> 
> 2) vlapic_accept_pic_intr() similarly calls __vlapic_accept_pic_intr() twice,
>    although this is more complicated to disentangle.
> 
>    v cannot be NULL because it has already been dereferenced in the function,
>    causing the ternary expression to always call __vlapic_accept_pic_intr().
>    However, the return expression of the function takes care to skip the call
>    if this vCPU isn't the PIC target.  As __vlapic_accept_pic_intr() is far
>    from trivial, make the TRACE_2D() semantics match the return semantics by
>    only calling __vlapic_accept_pic_intr() when the vCPU is the PIC target.
> 
> 3) hpet_set_timer() duplicates calls to hpet_tick_to_ns().  Pull the logic out
>    which simplifies both the TRACE and create_periodic_time() calls.
> 
> 4) lapic_rearm() makes multiple calls to vlapic_lvtt_period().  Pull it out
>    into a local variable.
> 
> vlapic_accept_pic_intr() is called on every VMEntry, so this is a reduction in
> VMEntry complexity across the board.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/arch/x86/hvm/vpic.c
> +++ b/xen/arch/x86/hvm/vpic.c
> @@ -512,14 +512,15 @@ void vpic_irq_negative_edge(struct domain *d, int irq)
>  
>  int vpic_ack_pending_irq(struct vcpu *v)
>  {
> -    int irq;
> +    int irq, accept;

Strictly speaking "accept" wants to be bool, and ...

>      struct hvm_hw_vpic *vpic = &v->domain->arch.hvm.vpic[0];
>  
>      ASSERT(has_vpic(v->domain));
>  
> -    TRACE_2D(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, vlapic_accept_pic_intr(v),
> -             vpic->int_output);
> -    if ( !vlapic_accept_pic_intr(v) || !vpic->int_output )
> +    accept = vlapic_accept_pic_intr(v);

... vlapic_accept_pic_intr() would eventually also want to be
converted to return bool.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 13:50:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 13:50:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191744.341833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSg9s-0006uK-Tx; Tue, 21 Sep 2021 13:50:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191744.341833; Tue, 21 Sep 2021 13:50:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSg9s-0006ti-Qi; Tue, 21 Sep 2021 13:50:04 +0000
Received: by outflank-mailman (input) for mailman id 191744;
 Tue, 21 Sep 2021 13:50:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dn89=OL=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mSg9q-0006aD-Pf
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 13:50:02 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.47]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df7fecfd-24de-4e70-b1de-029f8c90b680;
 Tue, 21 Sep 2021 13:49:59 +0000 (UTC)
Received: from DB8PR06CA0062.eurprd06.prod.outlook.com (2603:10a6:10:120::36)
 by PR3PR08MB5580.eurprd08.prod.outlook.com (2603:10a6:102:8a::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Tue, 21 Sep
 2021 13:49:53 +0000
Received: from DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:120:cafe::ef) by DB8PR06CA0062.outlook.office365.com
 (2603:10a6:10:120::36) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Tue, 21 Sep 2021 13:49:53 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT038.mail.protection.outlook.com (10.152.21.84) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 13:49:53 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Tue, 21 Sep 2021 13:49:52 +0000
Received: from 8f451ec1d378.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BB1B8A24-2A3F-46E2-8AA4-02F3DC85F179.1; 
 Tue, 21 Sep 2021 13:49:35 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8f451ec1d378.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 21 Sep 2021 13:49:35 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4295.eurprd08.prod.outlook.com (2603:10a6:20b:b7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 13:49:33 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%6]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 13:49:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df7fecfd-24de-4e70-b1de-029f8c90b680
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TVfTS6YALUnftVAqcmnV+Dy+a5EXbBUKlepvvAXDTUI=;
 b=Dmu8mntomQ8/4eiu7XvRclh3RMRPMY8kusB7Yu1WgO2X4jsHrBGr1HY0DRiAEfpwsYo+loINeaAny9tuuiJ3xgNUi2zh0ftMQyGXy6umgd+3Q/QPtC+JEDw1ZAQeyXTWzLkXvkaZOH2ZWZ3D+HpJp3Gpu2K8u1QBPTgb/UhyohQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8ea10c5f8929f8c6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RbCwLZW2D1NzRa2xKebm+udEHp/qX/BESCfnOn5rYSzudfMEfRa3oMg58FN1sgd9vH2qDHjHu/m89rKtBxMvZ3CXvmYsqmg739isPdem+Iv0QTXKVUUbSenHCC32O073m75OoU9lXfP1DCsk8qNXwGly1IHrdO9vf1VYgvR9GRYqD6sFDwsLIzvaeRu+d76Uv4KlFxUvdz2kaX1YobRJfMt3UaSYzNPFDTeXbs129jAQR8rZ7VIjrAWdChce+RI69BIENavT0zBqUYfGcPGCrGOPkBPsUO50cNtM4srZc3FFWXsWaU+KqjwE509JK8YH9Hqb6UCXkzY6rDjBwxGQEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TVfTS6YALUnftVAqcmnV+Dy+a5EXbBUKlepvvAXDTUI=;
 b=Z/U972zhqdmcKn7nAEyoGwy2wQTjUjokt3WMBvNxjROOOQWtgP3vy1uK3hZK2qwbQHO5twhS38bCyqrao81HYKfEvD74jKnytRnZkkjWhR9AnCvM+esq61peWJEtj++UXNP4slFf6iip+215ILT4Ruv6Y7LCsZti1WF1Av5qrEbcPsdRBAlHXuZhGNvK9BcZFfJj/Du5LyEp+/AtzFfGkNZiPddJit+m/JrbdoNqwWM7Klaai4O3v/LnOhSDCQVKmxvMshHRfTJUQy8X1B5x9aSrrdCpX81CZIHddvStZdGYoyLlncY+gpIJqT6nJxKvx5aqv3xQ+b9o3OpNwO0x9g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TVfTS6YALUnftVAqcmnV+Dy+a5EXbBUKlepvvAXDTUI=;
 b=Dmu8mntomQ8/4eiu7XvRclh3RMRPMY8kusB7Yu1WgO2X4jsHrBGr1HY0DRiAEfpwsYo+loINeaAny9tuuiJ3xgNUi2zh0ftMQyGXy6umgd+3Q/QPtC+JEDw1ZAQeyXTWzLkXvkaZOH2ZWZ3D+HpJp3Gpu2K8u1QBPTgb/UhyohQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Jan Beulich <jbeulich@suse.com>, Paul Durrant
	<paul@xen.org>, =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXlPL13zcSD0YvP0y1H1bgZa9bP6uciyMAgAofYICAAJ30AIAHbKUA
Date: Tue, 21 Sep 2021 13:49:33 +0000
Message-ID: <996263D9-27C3-43FE-A111-4BD2CE75B765@arm.com>
References: <cover.1629366665.git.rahul.singh@arm.com>
 <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s>
 <A90145B0-DD30-4300-8248-73104C782172@arm.com>
 <alpine.DEB.2.21.2109161324200.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109161324200.21985@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: f9e95cbe-5a34-4a61-55c4-08d97d06b09a
x-ms-traffictypediagnostic: AM6PR08MB4295:|PR3PR08MB5580:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PR3PR08MB5580F0FB93CA0CEF7F215A45FCA19@PR3PR08MB5580.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 dEH8uvsAuu1FyKCtG3dMbeu2927W0+z7L5Ox9q2ZtuVCL+GnwykQ99S2Y4Hl9L7CBInaEqSpr05xiJVaw2kx/B7aOQvJYG4Lyn8kcJJmVi79sM0XQyRwu985YwQ1k7c9jVrzMeO0Nfr/QmfMU7iGGcg3oN32onXS6BVnF+WsIbchiP63SQb3oDXgGJq3CtBZTjbcC9DsLagyP8VFEzr7CASAKBX8MAz/dn4iobf2ip2AVY07fJSsoToqUBJEmjaNKYqZ4z42CtUIQuBzXQACB9VGwBjhX47Jsh3wc2gj7Cyryj0s9tSgbHqnzXS7PwZHFpCJJZi/l2JMe5cYga5EVtlh7iqDa0NeidC/LyHKpA6O/4JIbXkLpFFVCt6DSfzQcWI5RjstEtaYIsQNV52YAORKZpTH0fQk59m02PUeKIOJTQedtNEzINKcyHJz6XclEh5T9lgu7ajw30BoF8W7maeIMV5US2KW2R8v8ZmFn93sjBIiug6GjPxCiODsWxDPhyYmW3xuHaKFbWjdPiIdE8p3XUwFU8TfHX5aSjCQFTj7LsUpkicq7ZJW4Fmv8Ff32AdqPgLQ6+WhFGPFqzbDh9wuRa0017uJs09nKN+8qELLAPTEulovd/pf9iHXVxHM62mJ5ZIHHt9vbojBnH9jB1atmhUwC54wiP7qLbjipYEdt69nCQtYhdOnJhIglywCucnEf47v0m5qARE+w9bFEMG1g6V1xmqT8AhVKiL+OYVthgfGrkifbcgtOMnPU30Z
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(83380400001)(122000001)(6512007)(8676002)(66556008)(508600001)(186003)(71200400001)(6916009)(38100700002)(4326008)(2906002)(8936002)(36756003)(33656002)(54906003)(5660300002)(66446008)(66946007)(64756008)(66476007)(2616005)(76116006)(26005)(38070700005)(91956017)(6506007)(53546011)(86362001)(316002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <2A9EACC27BFBED47BF630388AB2089E4@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4295
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	484fc795-f767-4e79-d307-08d97d06a468
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MVpGCDXSnvN8VL87O2OxYvPWMU1Rz8OlWUQ08HWxCsMQwJH1uoi4QTNgoiaItBByw/OY7Xsb+uOnPANOnRfjLHYhRV3smnhGA0YMV9lqDis1hyDMzeWsVO44+pfYIBt6OVZULQDeT1xSVfJKzQJjmYJ8jzPG19AxEcWa1aV5o95cbjAYnN4FFB+YDiAPhcFq73VPRX442ABG+4hmu7Vv/9xQOD3H9URIMUl0jS/PJYmkqB6abZsn9avRxojGX4MCKPPqj8tq9K/Iu0CSiDYLCah/6Z2lbiwMw/ba6FBok2/LAG29be8LbaS2HM4LCgGPivsFzi/zH0kPVuZXf7h632CJmWzybRvOLlT39DbztQLdnfKc/OO81mYRy9NOeZ9EmKK2wZSwcKJtTZjKP2EibYumC9jtieYWuEIuXBFRstEvwU4vEZX4P/Aij2nLvBycfShqREkT9J7iTT7tOLmTzX8zhxA5rymdXnbWQtdd2bBpnPmgODTHb7jq7cxmWJE3hfXLEbo+khVSmJMRT8QoJw559EKcBG5CEtWXWbQDPyAhbRO9lrblUtXEsIK8tuRDM3ZVq+Amckbsl/cLc8VxYG3gUvEXD8Zdggz10SzNMrV+rjYVE1eTmTKlw06Gr4bGwveTM8RJQgnmCFCaNadX2h7qE5/Buksm2KsfpmT7Ls9N1Ws2htFtvU/lEN3jPzve4VsV+KesV1t/H+9CbIenAw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(54906003)(508600001)(186003)(36756003)(86362001)(2906002)(6862004)(4326008)(6512007)(26005)(6506007)(53546011)(336012)(107886003)(6486002)(8936002)(82310400003)(70586007)(47076005)(70206006)(36860700001)(316002)(5660300002)(2616005)(8676002)(33656002)(83380400001)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 13:49:53.5766
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f9e95cbe-5a34-4a61-55c4-08d97d06b09a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5580

SGkgU3RlZmFubywNCg0KPiBPbiAxNiBTZXAgMjAyMSwgYXQgOToyNiBwbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFRodSwgMTYg
U2VwIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4gSGkgU3RlZmFubywNCj4+IA0KPj4+IE9u
IDEwIFNlcCAyMDIxLCBhdCAxOjI2IGFtLCBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5p
QGtlcm5lbC5vcmc+IHdyb3RlOg0KPj4+IA0KPj4+IE9uIFRodSwgMTkgQXVnIDIwMjEsIFJhaHVs
IFNpbmdoIHdyb3RlOg0KPj4+PiBUaGUgZXhpc3RpbmcgVlBDSSBzdXBwb3J0IGF2YWlsYWJsZSBm
b3IgWDg2IGlzIGFkYXB0ZWQgZm9yIEFybS4NCj4+Pj4gV2hlbiB0aGUgZGV2aWNlIGlzIGFkZGVk
IHRvIFhFTiB2aWEgdGhlIGh5cGVyIGNhbGwNCj4+Pj4g4oCcUEhZU0RFVk9QX3BjaV9kZXZpY2Vf
YWRk4oCdLCBWUENJIGhhbmRsZXIgZm9yIHRoZSBjb25maWcgc3BhY2UNCj4+Pj4gYWNjZXNzIGlz
IGFkZGVkIHRvIHRoZSBYZW4gdG8gZW11bGF0ZSB0aGUgUENJIGRldmljZXMgY29uZmlnIHNwYWNl
Lg0KPj4+IA0KPj4+IFRoaXMgaXMgZG9uZSBqdXN0IGZvciBkZXZpY2UgZGlzY292ZXJ5LCByaWdo
dD8NCj4+PiANCj4+PiBBbHRob3VnaCBpdCBpcyBjdXJyZW50bHkgbm90IGltcGxlbWVudGVkIChh
bmQgSSBhbSBub3QgYXNraW5nIHRvDQo+Pj4gaW1wbGVtZW50IGl0IG5vdywgSSBhbSBvbmx5IHRy
eWluZyB0byB1bmRlcnN0YW5kIHRoZSBhcmNoaXRlY3R1cmUpLCBpdA0KPj4+IHdvdWxkIGJlIHBv
c3NpYmxlIHRvIGRpc2NvdmVyIGFsbCBQQ0kgZGV2aWNlcyBqdXN0IGJ5IHdhbGtpbmcgZG93biB0
aGUNCj4+PiBQQ0kgaGllcmFyY2h5IGJ5IG91cnNlbHZlcyBpbiBYZW4gKG5vIERvbTAgaW50ZXJh
Y3Rpb25zKSBnaXZlbiB0aGF0IHdlDQo+Pj4gaGF2ZSBhbiBFQ0FNIGRyaXZlci4NCj4+PiANCj4+
PiBJIHRha2UgdGhhdCB3b3VsZCBiZSB0aGUgd2F5IHRvIGltcGxlbWVudCBQQ0kgc3VwcG9ydCBm
b3IgRG9tMGxlc3M/DQo+PiANCj4+IEl0IGlzIG5vdCBwb3NzaWJsZSB0byBkaXNjb3ZlciBQQ0kg
ZGV2aWNlcyBpbiBYRU4gaWYgZW51bWVyYXRpb24gaXMgbm90IGRvbmUgYmVmb3JlIFhFTiBib290
Lg0KPj4gSWYgYm9vdCBmaXJtd2FyZSBkaWQgdGhlIGVudW1lcmF0aW9uLCAgWEVOIHdpbGwgZGlz
Y292ZXIgdGhlIFBDSSBkZXZpY2UuDQo+IA0KPiBPSywgYnV0IGlmIHRoZSBib290IGZpcm13YXJl
IGRvZXMgdGhlIGVudW1lcmF0aW9uLCBob3cgd2lsbCBYZW4gZGlzY292ZXINCj4gdGhlIFBDSSBk
ZXZpY2UgZXhhY3RseT8gV2lsbCBYZW4gZGlzY292ZXIgaXQgYmVjYXVzZSB0aGUgUENJIGRldmlj
ZSB3aWxsDQo+IGJlIHByZXNlbnQgaW4gZGV2aWNlIHRyZWUgZXhwbGljaXRseSAod2lsbCBoYXZl
IGl0cyBvd24gZGV2aWNlIHRyZWUNCj4gbm9kZSk/IE9yIHdpbGwgWGVuIGRpc2NvdmVyIGl0IGJ5
IHdhbGtpbmcgdGhlIFBDSSBidXM/DQoNClllcyBYZW4gZGlzY292ZXIgaXQgYnkgd2Fsa2luZyB0
aGUgUENJIGJ1cy4gVGhlcmUgaXMgYWxyZWFkeSBhIGZ1bmN0aW9uIGluIFhFTiBzY2FuX3BjaV9k
ZXZpY2VzKCkgdGhhdCB3ZSBjYW4gdXNlIA0KdG8gZGlzY292ZXIgdGhlIFBDSSBkZXZpY2UgZm9y
IGRvbTBsZXNzIHN5c3RlbS4NCg0KUmVnYXJkcywNClJhaHVsDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:21:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:21:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191753.341844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSha7-0008BA-M8; Tue, 21 Sep 2021 15:21:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191753.341844; Tue, 21 Sep 2021 15:21:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSha7-0008B3-Iy; Tue, 21 Sep 2021 15:21:15 +0000
Received: by outflank-mailman (input) for mailman id 191753;
 Tue, 21 Sep 2021 15:21:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Wg/S=OL=gmail.com=rm.skakun@srs-us1.protection.inumbo.net>)
 id 1mSha6-0008At-Di
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 15:21:14 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 793a4c43-dc8f-4d0b-9cc9-a074aa3234ec;
 Tue, 21 Sep 2021 15:21:13 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id e15so44068446lfr.10
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 08:21:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 793a4c43-dc8f-4d0b-9cc9-a074aa3234ec
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=USVqWZAArbWUQLUtDtlQScPgCmgWeWDYUhPzzmzD9cE=;
        b=d+NekTAs6gdVeYOh1ETzFjnosk7b11Jkdp85cE2xaXNQHzds4fG/DaeBvLIWZok/bT
         8zlNB+vm+Lbp7HjZs0gwvcYkuGEzY4rYubjc8U1WV2ZAjOWge59m9LsdytVZhYAyMyh/
         kVUQqPb9S0H0ijHCFY4LkH7mywcT7P6bh7o5IlSwYZH7OErFPJB2l4n+nF1KREawE9Oh
         9pmHHaVkxCTRT5jOSCSHVffxNpywbXL6aTBdOBwwCr58FHR1JN9q9xV+0XT8s4DT/5fV
         iByIBKgKwoPVX0GWHFhW7mS1NofIUrEaPpVKSfuC2qDpTtF239YZNsypjKQ+zQwPcaL2
         R2pg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=USVqWZAArbWUQLUtDtlQScPgCmgWeWDYUhPzzmzD9cE=;
        b=43IojBEneI2Qm7APb5YpdqwRh42N5k0CVPZV2+buWD1vwJzVFfKKtK90KsKDyH7tny
         VgHs/PlwwtxrSZIOYal+b79D5+kvLHqfvRoG6NFswqaocrx4/+yTuS6rgDCnISWnvAfw
         d3m+ycBMx4okABBv2JlhE9NUcep4ucFq8hfU3prgjU+DZdXJFU8KhuVWGPpmDB6U+9PM
         7jQA04eXkyMyB/vbN18ORL2sQ9dqSFTHKNw0kznueOebQUtpgwWARQZ98CP5J4rnfzXp
         22I5oDYjdGqXLyGU+c7X/L2zQ7d0V5F/Lhr4NlhDfdZBInhBL4AeCBEGZzSQAqkttdjq
         XGXg==
X-Gm-Message-State: AOAM530rkcTsJw6QDkxf7KmQzwz69SNHhLIrR/JeVCxJeJFEaAXH+oSK
	X34SXzzsQTEo+LU11mtC49XtuMza8PzGC+ST/VQ=
X-Google-Smtp-Source: ABdhPJzthr3WA2wFCeyKG586GKIXPx3EL+AR4pi1b9V0CS7EWWEg72SpC7PGxABChTKUmIl/7el5hhGdi65vVZvbKIY=
X-Received: by 2002:a19:ca03:: with SMTP id a3mr22741135lfg.176.1632237637561;
 Tue, 21 Sep 2021 08:20:37 -0700 (PDT)
MIME-Version: 1.0
References: <20210914151016.3174924-1-Roman_Skakun@epam.com>
 <7c04db79-7de1-93ff-0908-9bad60a287b9@suse.com> <CADu_u-Ou08tMFm5xU871ae8ct+2YOuvn4rQ=83CMTbg2bx87Pg@mail.gmail.com>
 <84ef7ff7-2c9c-113a-4a2c-cef54a6ded51@suse.com> <20210915135321.GA15216@lst.de>
 <CADu_u-OZzgVj+z=iD6kUQOZxUufF5QSMR6-MmpN_hLZ9PyQJhQ@mail.gmail.com> <eb98aeac-af61-0dd6-2052-5b55921746c1@arm.com>
In-Reply-To: <eb98aeac-af61-0dd6-2052-5b55921746c1@arm.com>
From: Roman Skakun <rm.skakun@gmail.com>
Date: Tue, 21 Sep 2021 18:20:29 +0300
Message-ID: <CADu_u-O-aDk-J4up-UMfjMGi0DJkD6rhePcKdyqzt5750oo-iA@mail.gmail.com>
Subject: Re: [PATCH] swiotlb: set IO TLB segment size via cmdline
To: Robin Murphy <robin.murphy@arm.com>
Cc: Christoph Hellwig <hch@lst.de>, Jan Beulich <jbeulich@suse.com>, 
	Andrii Anisov <andrii_anisov@epam.com>, Roman Skakun <roman_skakun@epam.com>, 
	Jonathan Corbet <corbet@lwn.net>, Thomas Bogendoerfer <tsbogend@alpha.franken.de>, 
	Michael Ellerman <mpe@ellerman.id.au>, Benjamin Herrenschmidt <benh@kernel.crashing.org>, 
	Paul Mackerras <paulus@samba.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Marek Szyprowski <m.szyprowski@samsung.com>, 
	"Paul E. McKenney" <paulmck@kernel.org>, Randy Dunlap <rdunlap@infradead.org>, 
	Andrew Morton <akpm@linux-foundation.org>, Thomas Gleixner <tglx@linutronix.de>, 
	"Maciej W. Rozycki" <macro@orcam.me.uk>, Muchun Song <songmuchun@bytedance.com>, 
	Viresh Kumar <viresh.kumar@linaro.org>, Peter Zijlstra <peterz@infradead.org>, 
	Lu Baolu <baolu.lu@linux.intel.com>, Mike Rapoport <rppt@kernel.org>, 
	Will Deacon <will@kernel.org>, xen-devel@lists.xenproject.org, 
	linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, 
	linux-doc@vger.kernel.org, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>, 
	Roman Skakun <rm.skakun@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Robin,

>> I use Xen PV display. In my case, PV display backend(Dom0) allocates
>> contiguous buffer via DMA-API to
>> to implement zero-copy between Dom0 and DomU.
>>
> Well, something's gone badly wrong there - if you have to shadow the
> entire thing in a bounce buffer to import it then it's hardly zero-copy,
> is it? If you want to do buffer sharing the buffer really needs to be
> allocated appropriately to begin with, such that all relevant devices
> can access it directly. That might be something which needs fixing in Xen=
.
>

Right, in case when we want to use a zero-copy approach need to avoid
using swiotlb
bounce buffer for all devices which is potentially using this buffer.
The root of the problem is that this buffer mapped to foreign pages
and when I tried to
retrieve dma_addr for this buffer I got a foreign MFN that bigger than
32 bit and swiotlb tries to
use bounce buffer.
I understood, that, need to find a way to avoid using swiotlb in this case.
At the moment, it's unclear how to do this properly.
But, this is another story...

I guess, we can have the situation when some device like rcar-du needs
to use a sufficiently large
buffer which is greater than 256 KB (128(CURRENT_IO_TLB_SEGMENT *
2048) and need to
adjust this parameter during boot time, not compilation time.
In order to this point, this patch was created.

Thanks,
Roman

=D0=BF=D1=82, 17 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 12:44, Robin=
 Murphy <robin.murphy@arm.com>:
>
> On 2021-09-17 10:36, Roman Skakun wrote:
> > Hi, Christoph
> >
> > I use Xen PV display. In my case, PV display backend(Dom0) allocates
> > contiguous buffer via DMA-API to
> > to implement zero-copy between Dom0 and DomU.
>
> Well, something's gone badly wrong there - if you have to shadow the
> entire thing in a bounce buffer to import it then it's hardly zero-copy,
> is it? If you want to do buffer sharing the buffer really needs to be
> allocated appropriately to begin with, such that all relevant devices
> can access it directly. That might be something which needs fixing in Xen=
.
>
> Robin.
>
> > When I start Weston under DomU, I got the next log in Dom0:
> > ```
> > [ 112.554471] CPU: 0 PID: 367 Comm: weston Tainted: G O
> > 5.10.0-yocto-standard+ #312
> > [ 112.575149] Call trace:
> > [ 112.577666] dump_backtrace+0x0/0x1b0
> > [ 112.581373] show_stack+0x18/0x70
> > [ 112.584746] dump_stack+0xd0/0x12c
> > [ 112.588200] swiotlb_tbl_map_single+0x234/0x360
> > [ 112.592781] xen_swiotlb_map_page+0xe4/0x4c0
> > [ 112.597095] xen_swiotlb_map_sg+0x84/0x12c
> > [ 112.601249] dma_map_sg_attrs+0x54/0x60
> > [ 112.605138] vsp1_du_map_sg+0x30/0x60
> > [ 112.608851] rcar_du_vsp_map_fb+0x134/0x170
> > [ 112.613082] rcar_du_vsp_plane_prepare_fb+0x44/0x64
> > [ 112.618007] drm_atomic_helper_prepare_planes+0xac/0x160
> > [ 112.623362] drm_atomic_helper_commit+0x88/0x390
> > [ 112.628029] drm_atomic_nonblocking_commit+0x4c/0x60
> > [ 112.633043] drm_mode_atomic_ioctl+0x9a8/0xb0c
> > [ 112.637532] drm_ioctl_kernel+0xc4/0x11c
> > [ 112.641506] drm_ioctl+0x21c/0x460
> > [ 112.644967] __arm64_sys_ioctl+0xa8/0xf0
> > [ 112.648939] el0_svc_common.constprop.0+0x78/0x1a0
> > [ 112.653775] do_el0_svc+0x24/0x90
> > [ 112.657148] el0_svc+0x14/0x20
> > [ 112.660254] el0_sync_handler+0x1a4/0x1b0
> > [ 112.664315] el0_sync+0x174/0x180
> > [ 112.668145] rcar-fcp fea2f000.fcp: swiotlb buffer is full (sz:
> > 3686400 bytes), total 65536 (slots), used 112 (slots)
> > ```
> > The problem is happened here:
> > https://elixir.bootlin.com/linux/v5.14.4/source/drivers/gpu/drm/rcar-du=
/rcar_du_vsp.c#L202
> >
> > Sgt was created in dma_get_sgtable() by dma_common_get_sgtable() and
> > includes a single page chunk
> > as shown here:
> > https://elixir.bootlin.com/linux/v5.14.5/source/kernel/dma/ops_helpers.=
c#L18
> >
> > After creating a new sgt, we tried to map this sgt through vsp1_du_map_=
sg().
> > Internally, vsp1_du_map_sg() using ops->map_sg (e.g
> > xen_swiotlb_map_sg) to perform
> > mapping.
> >
> > I realized that required segment is too big to be fitted to default
> > swiotlb segment and condition
> > https://elixir.bootlin.com/linux/latest/source/kernel/dma/swiotlb.c#L47=
4
> > is always false.
> >
> > I know that I use a large buffer, but why can't I map this buffer in on=
e chunk?
> >
> > Thanks!
> >
> > =D1=81=D1=80, 15 =D1=81=D0=B5=D0=BD=D1=82. 2021 =D0=B3. =D0=B2 16:53, C=
hristoph Hellwig <hch@lst.de>:
> >>
> >> On Wed, Sep 15, 2021 at 03:49:52PM +0200, Jan Beulich wrote:
> >>> But the question remains: Why does the framebuffer need to be mapped
> >>> in a single giant chunk?
> >>
> >> More importantly: if you use dynamic dma mappings for your framebuffer
> >> you're doing something wrong.
> >
> >
> >



--=20
Best Regards, Roman.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:29:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:29:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191761.341854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShiB-0000eD-Js; Tue, 21 Sep 2021 15:29:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191761.341854; Tue, 21 Sep 2021 15:29:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShiB-0000e6-Gv; Tue, 21 Sep 2021 15:29:35 +0000
Received: by outflank-mailman (input) for mailman id 191761;
 Tue, 21 Sep 2021 15:29:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mShiA-0000dv-BC; Tue, 21 Sep 2021 15:29:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mShiA-0001UZ-3M; Tue, 21 Sep 2021 15:29:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mShi9-0007Ir-O7; Tue, 21 Sep 2021 15:29:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mShi9-00019u-Na; Tue, 21 Sep 2021 15:29:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=W4B8+JmOKcE93b+B1RY79EuXyMwmWFHDw6vi3n2gDns=; b=DYMFuZiYvy1HDXNrxrL46JWcbX
	RaDQm5bZYLI0pZxZQjqh+w4THFCPPk6IxqaiHhf6bXW3S84ohfopZqj37x4wBhTpwgMyjZtgiJxcE
	2TwBfl4BbxVmqMuS1fQLUJgTwTts/HRAEhiv1fo1buacD78Vb0XPN1e+rtuxv2SP8AEM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165136-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165136: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 15:29:33 +0000

flight 165136 xen-unstable real [real]
flight 165141 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165136/
http://logs.test-lab.xenproject.org/osstest/logs/165141/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   10 days
Failing since        164951  2021-09-12 00:14:36 Z    9 days   19 attempts
Testing same since   165131  2021-09-20 17:39:22 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 683 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:38:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:38:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191768.341868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShqo-0002CG-Fr; Tue, 21 Sep 2021 15:38:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191768.341868; Tue, 21 Sep 2021 15:38:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShqo-0002C9-Cw; Tue, 21 Sep 2021 15:38:30 +0000
Received: by outflank-mailman (input) for mailman id 191768;
 Tue, 21 Sep 2021 15:38:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QjZ+=OL=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mShqm-0002C3-Hc
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 15:38:28 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3d0d4a79-5ce6-406d-b78a-0ba23dc1cfc8;
 Tue, 21 Sep 2021 15:38:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d0d4a79-5ce6-406d-b78a-0ba23dc1cfc8
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632238706;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=NteqrtrplbuAWatZPBQGbZaXrPDW2ncs3CSYvcFo8BE=;
  b=SjkbddrH4dF6DD/q06hpWHupyMW2lTBdlUxRvce+L+rLZbu04YN7u5Nz
   p/d9eaQudvP1iiHMWGSdVhuAInWSWYWvVnQmG38v65inkARZArg4f4HxQ
   2HsD+6xe4ukKEb5FPxrjmvGnZ52uM4ld0zrnfBYQiC7Q0fyGL5YtVfXrz
   8=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: osL35eMDgcguAYM9jvE38O1LvAyYJVpwcH5l4yKuam+x5DsdW+gathWEJN07xZ/vJaoKMhZN1x
 bwRQqo+qiZ+1I+QPPCjLwEfUpCZLvHZAV1JgQZXjq3D8CodLtiP15s4vskU3RoK0Hhm/q7YgJd
 YwxiWQzKbLZVV4z0BdCIkKxYk9jecGrMaKIfhdLr/KwZ0qvEc2UVBI0b2fHjBKxxB7wpzd3GO8
 3C1D0cF3G3fxfA07JJlckSYAbvn8cEEkKNOEq8UPKP4hLi03ADQhsp/p/sPZczQE1pJCWv2FTJ
 sOsN9aWuSsVLLVy/BDT6IKLa
X-SBRS: 5.1
X-MesageID: 53252430
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:L/kXV62bublyV2q3dvbD5fZ2kn2cJEfYwER7XKvMYLTBsI5bp2FWm
 GAZC2GPP/rbamr2ed4lPt6w9UxTvMKHyddlTwZrpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970Eoywbdh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhr+570
 4VXt6SMSQItOLb9vt4bWTxFDHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Er8IvNsT0eqgYvWlt12rxBvc6W5HTBa7N4Le02R9t3ZoTR6qFP
 6L1bxJ3bD3NbT9tAGwsK6Adscu2xWOvSzpX/Qf9Sa0fvDGIkV0ZPKLWGNjfd8GORM5Vtl2Fv
 W+A9GP8ajkFMPSPxDzD9Wij7sffkCW+VI8MGbmQ8v9xnEbV1mEVEAcRV1awvb++kEHWZj5EA
 xVKoGx09/F0rRH1CImmN/GlnJKali4iUcEACe8C1AGIzqzP0iCjAGIhUyEUPbTKq/QKqSwWO
 k6hxo2yX2U/7uHNEBpx5Z/P8mjjYnF9wXsqIHZeFFpbuYGLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3m7ESnQkSGcICuTrsoCNtVgRiG2NPdXABb3nARFodt3xc7V5l
 CJY8/VyFchXZX13qMBofNjh4Znzv6rVWNEjvbKfN8Z4rGn8k5JSVatR/Ct/NC9UDyrwQhewO
 BW7kVoIvPd7ZSL2BYcqM9PZI5l7lsDISIW6Ps04m/ITO/Cdgifcp3owDaNRtkiw+HURfVYXY
 8bDLZn0UiZEVsyKDlOeHo8g7FPi/QhnrUv7TpHn1RW3l72YYX+eU7AeN1WSKOs+6cu5TM/9q
 b6z7uOGlEdSVvPQeC7S/dJBJFwGNyFjV5v3t9ZWZqiIJQ8/QDMtDPrYwLUAfY15nvsKyreUr
 y/lAkIImkDigXDnKBmRbiwxYr3YQpsi/2kwOjYhPAj01iF7M5qv9qoWa7A+YaIjqL541fdxQ
 vRcI5eAD/1DRy7p4TMYaZWh/oVueA7y3VCFPja/YSh5dJllHlSb9tjhdwrp1S8PEivo6pdu/
 +z+jlvWGMNRSR5jAcDabOOU42mw5XVNyvhvW0boI8VIfBm++oZdNCGs3OQ8JNsBKEufy2LCh
 RqWGxoRucLEv5QxrIvSnamBooqkT7l+E05dEzWJ5Lq6L3CHrG+qwIsGW+eUZzHNEmjz/fz6N
 +lSyvj9NtwBnUpL7NUgQ+o6k/pm6ou9vaJewyRlAG7PPgaiBb5XK3Wb2dVC6/9WzbhDtArqA
 k+C97G241lS1B8JxLLJGDcYUw==
IronPort-HdrOrdr: A9a23:Zm7gN60puGOrW6NbWYqT7QqjBSVyeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5OEtApTiBUJPwJE80hqQFnrX5Wo3SIDUO2VHYUb2KiLGN/9SOIVyHygcw79
 YGT0E6MqyLMbEYt7eL3ODbKadY/DDvysnB7o2/vhQdPj2CKZsQizuRYjzrY3GeLzM2Y6bReq
 DshPav6wDQAkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53252430"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W8DixBTsG/MOu3scJmyyGG8rkvOJ0xKOlbgonsrmqBGruICypbtv5xC/u1ISghX8aph53yPQXLeMQQL5UKNYlXnDWcVv1vBhmQK1X/rFFTUQ0DMEH+nyw/RAM8QlPp4h3oaLhMeOg5bq142ucT11VC8MSgExUB+4zT1Ns/CK4ZFOV1CdIr211Wyzy/WnAYfvQHuWEdaXF91phPR4FRG5dgZzIhy7rdAwr6W+qCgmwHi9LkdktmCl994k4RVxAO+OVazi63Hf98b1g1LBZ9cb3Z4o6FO+CwIfQ5rtpQ3Lms5HjNtNnaEJaJ4ye5DXN2wUB14D6F3LfeDCgfdlivpDBA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sML6edXhxYzBIQD6A7kp2iSgs2pWQoBFQae2K5EC6ww=;
 b=lwewnhwx08tGKjXI5PdK8+tO6EBsOmkMaXLYrZb479Hcyv02BZDK+Ot3fH9N5Ba/+ERktZ2ER5MqZrcXcctE8m3LF1rRS4Pn/LRRTQhrmwcvoPoBxfTTIORJEfg9oR0G0eakC+ii3i0tC+rYQXObgkQXIuer20mFeaxM7s9YUJS+VnsbsTwVNEpJMK0Iw9/+ZfN/FGT1YMmyGxw+Ri76CDjcxJw38QgHS9oljB20csRQYTSu/1CO8UYBzGJDAlBGdqpgp5RWKTYiregV0ozpBifUEEaYGvCF36gyBKEIdFAlHmx0RyI3BKT+SOgczzu9vJrU4QzLFzamg6afG1SYAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sML6edXhxYzBIQD6A7kp2iSgs2pWQoBFQae2K5EC6ww=;
 b=Ks2OrZrKArarrRfEcpVDFBtJn4iX3RvUUd++BArG8Jvviv6si18IqrHemMXWUrZrR1n2VCLTcAgHIJwzBY05BPkfJckzSyqBMpwrPLRUEdgVZAWIPSNIq90hbcfXwcHjZ9ckIugFykdLncTN9ONNjHEKPc3RuBov4Yu3lpFDC88=
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-5-andrew.cooper3@citrix.com>
 <1ec8d289-5c34-412c-c683-189cd5506600@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 04/12] x86/hvm: Reduce stack usage from HVMTRACE_ND()
Message-ID: <f85e82e9-b75f-5b49-b1a4-86fc90ebde84@citrix.com>
Date: Tue, 21 Sep 2021 16:38:17 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1ec8d289-5c34-412c-c683-189cd5506600@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0190.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a4::15) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 37cc925a-1f06-4b59-2f2e-08d97d15d86f
X-MS-TrafficTypeDiagnostic: BYAPR03MB4615:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4615A06A381C29F4F5615016BAA19@BYAPR03MB4615.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2043;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Wrbz4m81TK1K8EoObr5FV6eOfSu/7+kero3sZKksehVlqs21ej9KUuW7dvsr0lzV8gVNXBqiGWxB9u0cJf00EtbswJARr+T61Au4+9mMfMQmYiCV3aDCleTqC/Ck+1iJ5oUky3av4+eIIJ0w7hjIcxWcydJxIskSLtgpht61YpV1F+SWQJJRMjgc31MDJKWVs156xvWrkBdcDeiqIQzk2ifu0Y/fkB4JG8GWyvxUPSYWsekTqaYxzVjgpEJl4rf0hmr2gcMzUkTIn7R5WilA4qDCPirrGzC77F7L1/pxqlCDCsnTpLrfFDHDRM2xW2N0lWs0qksm/bHZ1AqvgXOn7JxloeGbHDHzfTzvW4M+Qw7pA4wREXK32RRn4cYARxvRkwMFYomgzmgPIbpHqY0uah/chFFkf6PxYSlLRhaZbj54Smd9cBXGgyYVh5WspwEw5LZOEBJP9CbICl7deIXgtJIM1tFxssvVPL63sd4bvIIN9/NULX0je1hDEHlP3haiTMeeReWkCK3Zd7jXznphaU4i1II9zhdiSxJ0k5HooGgrH5yWPENAoAuTAPoCrNayuzKz0xdkpTq052bMOOfikygGWk8zOkvs09vGDMUFgwcRYyYyS7dISKbOeWofjl9BGmDeI34H/MqTXqTOBULXprhWlYOv9UfJjtw68Ao2s4yPLxGFFf9r8AInD4OsvoDkpENV34tV8xtCDDTBzixsKPQHQIsKlWR+CqzDw3+Mt4A=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(36756003)(6916009)(4326008)(54906003)(86362001)(16576012)(31686004)(66476007)(38100700002)(66556008)(5660300002)(26005)(66946007)(316002)(31696002)(2616005)(508600001)(186003)(83380400001)(956004)(8676002)(55236004)(8936002)(6666004)(6486002)(53546011)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eWZwR2tqS0YxUUFBQ2ZBaTVhVHVLT3JsbURtdmkxbUU3MWQ1eUQvdHIvRnpI?=
 =?utf-8?B?OUx0YUk2SkZhZkFKemFrZVRsams3ZG80SWwyOWREN0pMSkZ1NXdPR3F3Rk1p?=
 =?utf-8?B?Z2FoV1NHVkVKelJFbnlCeWVkMnFoK09HRXdzZDNQbFFiU3RCcHlRWnE3M0t0?=
 =?utf-8?B?cGhZVzRSV1BhRVhwbzFxdXVkUkY0U0RkQTh1TGl5MHRWSzZiQy8vb0lhNVI2?=
 =?utf-8?B?QTRrY1kyNDJMbVZtTVlNTG5JdXd6aDd3OVJjd2xvdW9GVkRmeXdvc1F1TVdV?=
 =?utf-8?B?eGpqTlA2RG12NmFzMEpicG5nTXkzc3lFWU1KUUZ3ODF4OUdDcW9TblBBR3Nj?=
 =?utf-8?B?aU5DVkRFS2ZDYWRmU2RMOE43VTF3UUl5V0VjMTVwWHdkdVY0SURjOEVTRWNx?=
 =?utf-8?B?Syt1d0ZOYllUS28ybG9XN1dUZ0pnUUd1VjBBTVAzdDFJQStDM294U1NSTHJl?=
 =?utf-8?B?MCt2c3NWM3lOVVM4ZG9SejR3T21VSUF5WFZOcTRqRzlNQ0ZacUZqWTV2UjY2?=
 =?utf-8?B?S1B1bEtLVHhaaXFIK3k2REt5OTRMOE1qZXBpWXdpKzcvbXhaRFNoeGkzWGRo?=
 =?utf-8?B?Q0lpOEZsSkNYQ3R6bklzbU9naUJPZjVIQnJPUDQ2QkFncW9Gby8xdk1WNGtC?=
 =?utf-8?B?Z1VxbnRzZGN5enkyYzl3OXdHeE14bEVSQWRzS0VXdkt2ckVKWVN1U29hUm9J?=
 =?utf-8?B?UGxPR2hURHJIL3BGdThlUDFuRGhjWGU5dmUwWDNxS2dOWXZFclRISTBGUWY0?=
 =?utf-8?B?dExtNXZKcWFNRGJteTBqR1hZWGdYcHFoRjVYTU5ORXdabWRmbW96dFBHNXJj?=
 =?utf-8?B?YWd5d0RJTHpEZ2dTc3ZkckNMYSsxV1l4TWxsUXpudmFkSG5CZDQ0dC9xWjhW?=
 =?utf-8?B?bU1zQ3BKVUN4SUdmQmNLM2NhME05ZVY5YmxKeTd0dVJnU2UrM1dWcGpCRGdF?=
 =?utf-8?B?dXMzcm8rSU42MElLOHBuNEtZK3Z2bTlEZzhOKy8zTkw2REVQV3g5dWc0eEZO?=
 =?utf-8?B?Q2M3UjM3dTVWcWZ3cWdWVGl6Wm9PVDNtUU1WdVhCanY2M2YwN0t4aVdHMkt0?=
 =?utf-8?B?RkZIYzg0akh3VlAxSUhDY0dFbWREZXR1K2hWaXEvSGZZS3l4SkNUN05YdkFy?=
 =?utf-8?B?d0JnRDVXdDc4U3laYkVOUG9QWW1TNG1FMGdNRG1vZUVQaVFoNVJtVitQZEFj?=
 =?utf-8?B?YXZMZHA5ZERzcy9haC9MUkVQeno5cmxDcDdiTkJ2OHgvVS9ZTkg4Vys5MmpE?=
 =?utf-8?B?UGgyT2hGMlVBZ053SzVQaGpCNEU0RjA2bFQySFBNNTNFd1hvRkxYQ1dxY2tV?=
 =?utf-8?B?MlR4MGs2V2RIME1RR3ZkR2k4a1AwUVRLYWxlQ0tJc1ZhVEEyWG00aVBzeFo2?=
 =?utf-8?B?aitnR0IrVnMrQzdFMUowTzBaME44Ukh6VnFKVHowZkZOcXI2d0k0M1JGUXpN?=
 =?utf-8?B?cm1KNHhDQTJSR0Rxc3Y4dG8vVVdqeDFPSTFmdEc2L1hYWWFBREthY1JvcGc1?=
 =?utf-8?B?L3R0c0hKNnRVdk16MUtRVHZ5Zmg2L0tKb2RlOFBINllJRXp2Z0hZd3BMM3dN?=
 =?utf-8?B?V05BYTFEYWpLT0VmcS9ycGdOSEFnejYwdTNCNmRKVW9uditLb21QT3FBQzly?=
 =?utf-8?B?L1FPcStZdmd5dnA3V0VHUWxUM2JsSmtQNnRINjJIVGd3bDhJM2tXMUtpVVNQ?=
 =?utf-8?B?YjZxckpvNG8xK0VoQ05ZQWpsZGdhMVhLVFQ1SWNTK2VMTm8wK0xkRWp5NlV2?=
 =?utf-8?Q?bY8Xumd35G73+IgxXQpSv5poAOZ1HCmBIHSiUEM?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 37cc925a-1f06-4b59-2f2e-08d97d15d86f
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 15:38:23.0535
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +j2NEFjog2M6d39Aje2hQTnRYc0k8RrqOHSQCFjSeMbSgvVW34OKWAl2T1GQABvQzZ7Ctv+1SygEJj0A9x7uwT9Awmwb95Gp+i8TnGfe+NU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4615
X-OriginatorOrg: citrix.com

On 21/09/2021 12:00, Jan Beulich wrote:
> On 20.09.2021 19:25, Andrew Cooper wrote:
>> v2:
>>  * Adjust callers of HVMTRACE_ND() too
> What does this refer to? The sole difference to v1 that I can spot
> is ...

Oh - its me who was confused.

I thought I had failed to include the changes in vmx.c/svm.c in v1.=C2=A0 I=
n
which case, no change to that in v2.

>>  * Drop _d[] for the 0 case.
> ... the one corresponding to this line, i.e. ...
>
>> --- a/xen/include/asm-x86/hvm/trace.h
>> +++ b/xen/include/asm-x86/hvm/trace.h
>> @@ -67,38 +67,30 @@
>>  #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
>>      TRACE_6D(_e, d1, d2, d3, d4)
>> =20
>> -#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, d=
6) \
>> +#define HVMTRACE_ND(evt, modifier, cycles, ...)                        =
   \
>>      do {                                                               =
   \
>>          if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )            =
   \
>>          {                                                              =
   \
>> -            struct {                                                   =
   \
>> -                u32 d[6];                                              =
   \
>> -            } _d;                                                      =
   \
>> -            _d.d[0]=3D(d1);                                            =
     \
>> -            _d.d[1]=3D(d2);                                            =
     \
>> -            _d.d[2]=3D(d3);                                            =
     \
>> -            _d.d[3]=3D(d4);                                            =
     \
>> -            _d.d[4]=3D(d5);                                            =
     \
>> -            _d.d[5]=3D(d6);                                            =
     \
>> +            uint32_t _d[] =3D { __VA_ARGS__ };                         =
     \
>>              __trace_var(TRC_HVM_ ## evt | (modifier), cycles,          =
   \
>> -                        sizeof(*_d.d) * count, &_d);                   =
   \
>> +                        sizeof(_d), sizeof(_d) ? _d : NULL);           =
   \
> ... the addition of a conditional operator here (which I assume was
> something a particular compiler didn't like in v1).

And was covered in the commit message:

> The 0 case needs a little help.  All object in C must have a unique addre=
ss
> and _d is passed by pointer.  Explicitly permit the optimiser to drop the
> array.



> FAOD - I'm fine
> with the change, but I fear I'm overlooking something (again).

Thanks,

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:40:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:40:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191776.341880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSht4-0003cB-2A; Tue, 21 Sep 2021 15:40:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191776.341880; Tue, 21 Sep 2021 15:40:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSht3-0003c4-Uv; Tue, 21 Sep 2021 15:40:49 +0000
Received: by outflank-mailman (input) for mailman id 191776;
 Tue, 21 Sep 2021 15:40:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSht3-0003bx-6X
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 15:40:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 379b6b43-9bfd-4362-8cbb-48a850bf9658;
 Tue, 21 Sep 2021 15:40:48 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-2qndNLIGOxWuJuQT17yapA-1; Tue, 21 Sep 2021 17:40:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3389.eurprd04.prod.outlook.com (2603:10a6:803:b::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Tue, 21 Sep
 2021 15:40:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 15:40:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0113.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Tue, 21 Sep 2021 15:40:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 379b6b43-9bfd-4362-8cbb-48a850bf9658
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632238847;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=W3vLjVFg9A/hv8geZZYQE/sughQeoeEnIr/IVhSL91w=;
	b=QjLy0ngmZHnWvE9oUTnkKtiX+nsppeX7pRebZ9Xj12c/068grfsTFeiJcqlDnUo3cwvYo3
	gQUx8tu6wKedR5xegE42i735uqSralYvH7Doo/JMpzawxY/c+t2XUj7sSTUnooism0vgKa
	40LWPZUXSTb44K0y2YtSoXiel7UB3bw=
X-MC-Unique: 2qndNLIGOxWuJuQT17yapA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PBq3GQTpPQAASdlPCBPPiljXM06EDZ+d2qKjgFZCQA04Q2ibYZzs9i4BxLq64x8REciqbSbPyWPupS7yiY17x7ny9zbA8ZXsc2c1JTaPJPLQGDCkvt6GCzkIFymDoj1Ra5XS5r4+NJDopjBV6eCuacUMyL+KrbV6u52xus2g9w2nwnqgcqFZ+PKW0nmfZG0uVuaHfcKK0T2cdn/471MgqH/5pUUsl0ac6eeXgWooOhRSS4Ddl9GSJPq7FloZ0VatXMoB7ZGf7AEE56zdZRzAi8j+CE+YDTby+4dSstewBVeEBYqx4CfVsWi3Hb+P5ulg9UxdS9U+unZzBB9Q1whXIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XtJm3Q2B4CxVAL6aKgs4w3/Za+id/ONdsfb0a0NlobU=;
 b=K3clgdSdFHPTa/vC9uKdTssjQt2NBO5O9C3rSJzRUBYk8wpAxQz4ji0pnrpaWX9rbPlF0Ix+rtHJ5Q6+EhBzDLFA8sgMWq+/u1rFsCNtcolzcTspwS2r33qf/Kp5lkelTjoTa2yOlGEWbN0fBJYvRu10V645nAC8aWhGu+JQ7dkvlq6M3YHm9YDjrBMliSp1iBtJ79qoPGr3WviP2Ert+3cTDg0jcITEGHj3Rhy+AmbR+qJ78eh4hNaC9OrgLSpNF+jxktok5eAX7ZZ1MMAcJKPmVvqFGK5blfu/1wuaMbfg7kQZXEXFUosL4qDBqJiikydIc9r9aoY0fJ0lpYVZew==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 04/12] x86/hvm: Reduce stack usage from HVMTRACE_ND()
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-5-andrew.cooper3@citrix.com>
 <1ec8d289-5c34-412c-c683-189cd5506600@suse.com>
 <f85e82e9-b75f-5b49-b1a4-86fc90ebde84@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <61a56d55-6d2d-d5f7-9719-4ab2d0793fa9@suse.com>
Date: Tue, 21 Sep 2021 17:40:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f85e82e9-b75f-5b49-b1a4-86fc90ebde84@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR01CA0113.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 43a984ac-d4fb-4c97-8a77-08d97d162d58
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3389:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB33898E83463EDA910BA19C35B3A19@VI1PR0402MB3389.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1051;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	AbVOnGh9zKOcvs2VFdfrcj+n18ujJ/bsPLDOPcqHfX47l+pCZFnwUrCmJyOSXWL2gI1EsUvjPxaxDbLx1mILWCj5hN3JsMfHTF7Vy2kehtNmK0G8j+HtPdp8uvPiIDEQwmq8RhrlVgA01dAjUj52WuXyl9h9ozD79yLMVLGVx+eR2HdOM0Giklk+AzhgH1YwVCsdxzRZd5Z8Aji1d9/ooF8xe9OfZ0gwhGn/5yy9bhRWseK8PewIXsxZFH9n25biOVl6rV4K6K5U/qaOzuGWA+UYEdAIR+ooICLfYEhYbsG8Pm4bnEf3wKMad5vnjj7XI2kPVZlZS9TfOilWCfmEjVA9CqGnUkcAjf19BDBGscMTpOSzscQ14D8UIDl1CErSFWXdUhuva4gOe50bRhAOVTFQRhRo2RE3TPKbihOuyxokD8CiDF0iP6aPPFhhFa5s8kX7gPQYUQM+207kLbO5B/XbrR7Vmxg6MuPRlffEBuG5ufPNdmR0ehE+K7BJuXd8jOTghyi4Z7D+Sov0EIl8KXOtZxZ5x1o6ADMrp7t1xGzC1/zI2bW80h8XX2PqlGmqIjGwz8+W/IpOokc+Q74WdBKVynWBzkyEFzbD3qYBWigtfsrA3FDaHvx83OUtwj/ep1Ene9T1tzV+SP8phq9VrIYBstEUVV/iXk0DefrkqEeK3Pjyt6R+7JA/w6C5FOoI
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(16576012)(6916009)(956004)(36756003)(86362001)(2616005)(66946007)(66556008)(4326008)(508600001)(31686004)(6486002)(66476007)(5660300002)(316002)(83380400001)(38100700002)(2906002)(8936002)(186003)(26005)(53546011)(31696002)(8676002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?PH30N7KxAXDJQcEHtQbURjomX9buEzvwPmt1KhKU4VYUui31u+oecdwehPJN?=
 =?us-ascii?Q?ymsg1ERbDBUZeIexlmARh8GA6+2+2l85pp53s/g7TJTThrvI+fGlhYfgMaRh?=
 =?us-ascii?Q?/upTz3uplVMI0fx+vTvFJ+P74isdQA5zQP3JeFTd4+6NxAQtEiAHsiDYe4sO?=
 =?us-ascii?Q?gPW3KnR2PGx0fQEFH4CNkLqYwDJy9pmhjCoL6YdIffWEPStuihXvk5luo/t/?=
 =?us-ascii?Q?CZ8/NxfTVhWk0JbaY+ZydHRuldnG2IHjlpuhw56NHQ+/1wX9z+TLIatPonfg?=
 =?us-ascii?Q?wNRGRbFDfGXIKeoQ/EHgzV6/IlXf5MX1JS5oVNyjfCEcTmPmS9wN15cadgLe?=
 =?us-ascii?Q?DEoKEnwhTuMO/GA+yEwa8mzO1Rnl3OtIJ7z0UUKED/cWQNIecMJbmE03Udz6?=
 =?us-ascii?Q?6DS2zPwZmFYjy3Tg0DsIq6LZQQZk3LViepOn+3fAfyB8Mr4mC4AStfdBXBWm?=
 =?us-ascii?Q?hZP6uRAskE4Wl+4XD75ial0Fn2aSI0f4MOr7IEvneDaKjDoO4co74Kx3avhU?=
 =?us-ascii?Q?cOpXGdQHnJ1rFpKEBToyifv4lg2xm89ImMo/FFH25Uy1gyRqkK0iegSAK6MM?=
 =?us-ascii?Q?yS4QtLFqPfAOHSrLvpYFMkXW3aQcsu0ud8xkvzjR28qAmB1270los6SqCTzl?=
 =?us-ascii?Q?sbVnCq3FfkyKvcHyRf694Ml3/ElTqT9JBmyGKTcZQKqI+xNxUsOADNsILY4f?=
 =?us-ascii?Q?pWCumgBt9/Gmi8NB0lH7BUi5icd/KrJ3enJZTb3izO5FudBZpQsD9H97ZJpm?=
 =?us-ascii?Q?TMzJUGchf90FLmz1eeOTpmGT+rxrmEeujS2KLCPTrH0W7dlZAVjQC3TnAYbu?=
 =?us-ascii?Q?M7ibSjWUMW2t1Op1WJxo71IDt8CrJ+rHwYlFCubJ5+FoAU0/QlMEEsvJm9Is?=
 =?us-ascii?Q?EPHQgMHDx1VrfP2wUMa/L43X0tfCV1+Xza0juEMDSf44yTLkQWYSLT+7OvyZ?=
 =?us-ascii?Q?kuK5Qm9rjkNCh9+xSNhLj2+vY/S8B5hGi/SBaWNFuDLDCvmgBmgab3CK7FAo?=
 =?us-ascii?Q?ROTYSsTq5/BlVlmpXRkN40F9Dh8jCh5JJUgQBwzthHaTRUYAJCfhkzGWwArt?=
 =?us-ascii?Q?S7UO2ivYZHGgD9ZqFDAUH4ptm53xDQ32wpZulxHOXVBxiUKo2BsCfaV7ovqM?=
 =?us-ascii?Q?EbsFgPI8bysyL4SQIB7Zxg1cFwTksFViabXA4ytAk0iHGOfvee6Sjd27knZP?=
 =?us-ascii?Q?yI7otT6w0jgMCEmPhU3xvRsW1UxAenpmMmcGSx6LMXQS8JF7A3BBB9weMhtW?=
 =?us-ascii?Q?AYXDjiER7LWPnAgU6MPPffPGn1FGZ+F9qBxdekf4o+6uTrre3W4BvQvX8pAv?=
 =?us-ascii?Q?Jas/axfNuo9aNdRqYeWKrcCs?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43a984ac-d4fb-4c97-8a77-08d97d162d58
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 15:40:45.4554
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VvKhlwIwL/p6Ryi0EW5nlVefLTpXzdvmBNbMBhDO+/Vpus46E0FRrLQmnEWL65HRkL9tCtOrvmbS2QTJe7dQZg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3389

On 21.09.2021 17:38, Andrew Cooper wrote:
> On 21/09/2021 12:00, Jan Beulich wrote:
>> On 20.09.2021 19:25, Andrew Cooper wrote:
>>> v2:
>>>  * Adjust callers of HVMTRACE_ND() too
>> What does this refer to? The sole difference to v1 that I can spot
>> is ...
>=20
> Oh - its me who was confused.
>=20
> I thought I had failed to include the changes in vmx.c/svm.c in v1.=C2=A0=
 In
> which case, no change to that in v2.

Good:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

>>>  * Drop _d[] for the 0 case.
>> ... the one corresponding to this line, i.e. ...
>>
>>> --- a/xen/include/asm-x86/hvm/trace.h
>>> +++ b/xen/include/asm-x86/hvm/trace.h
>>> @@ -67,38 +67,30 @@
>>>  #define TRACE_2_LONG_4D(_e, d1, d2, d3, d4, ...) \
>>>      TRACE_6D(_e, d1, d2, d3, d4)
>>> =20
>>> -#define HVMTRACE_ND(evt, modifier, cycles, count, d1, d2, d3, d4, d5, =
d6) \
>>> +#define HVMTRACE_ND(evt, modifier, cycles, ...)                       =
    \
>>>      do {                                                              =
    \
>>>          if ( unlikely(tb_init_done) && DO_TRC_HVM_ ## evt )           =
    \
>>>          {                                                             =
    \
>>> -            struct {                                                  =
    \
>>> -                u32 d[6];                                             =
    \
>>> -            } _d;                                                     =
    \
>>> -            _d.d[0]=3D(d1);                                           =
      \
>>> -            _d.d[1]=3D(d2);                                           =
      \
>>> -            _d.d[2]=3D(d3);                                           =
      \
>>> -            _d.d[3]=3D(d4);                                           =
      \
>>> -            _d.d[4]=3D(d5);                                           =
      \
>>> -            _d.d[5]=3D(d6);                                           =
      \
>>> +            uint32_t _d[] =3D { __VA_ARGS__ };                        =
      \
>>>              __trace_var(TRC_HVM_ ## evt | (modifier), cycles,         =
    \
>>> -                        sizeof(*_d.d) * count, &_d);                  =
    \
>>> +                        sizeof(_d), sizeof(_d) ? _d : NULL);          =
    \
>> ... the addition of a conditional operator here (which I assume was
>> something a particular compiler didn't like in v1).
>=20
> And was covered in the commit message:
>=20
>> The 0 case needs a little help.  All object in C must have a unique addr=
ess
>> and _d is passed by pointer.  Explicitly permit the optimiser to drop th=
e
>> array.

Right, I had associated text and change this way. It was really just
the revision log which was confusing.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:43:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:43:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191782.341891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShvt-0004G3-GM; Tue, 21 Sep 2021 15:43:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191782.341891; Tue, 21 Sep 2021 15:43:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mShvt-0004Fw-D9; Tue, 21 Sep 2021 15:43:45 +0000
Received: by outflank-mailman (input) for mailman id 191782;
 Tue, 21 Sep 2021 15:43:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mShvs-0004Fq-I9
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 15:43:44 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id efc7d036-5aad-410d-a376-e64eced4ec9c;
 Tue, 21 Sep 2021 15:43:43 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2056.outbound.protection.outlook.com [104.47.1.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-8w8D57NuOR2LwubjXpPE9w-1; Tue, 21 Sep 2021 17:43:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Tue, 21 Sep
 2021 15:43:39 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 15:43:39 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0029.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16 via Frontend Transport; Tue, 21 Sep 2021 15:43:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efc7d036-5aad-410d-a376-e64eced4ec9c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632239022;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=hvPZWeko7LkrbiJynVatfRdxwf7pWGqz0z5S7aeVE8c=;
	b=Qza8VjCWzHUbaPCPpVHVT3+8RA65nMlblMu4mEEbYHe+6lyTeCRzdVheWgUreRviahF52T
	X5yaB2y0jWhUFWLuombYggtJaTVDQqmHevIl6BymP8fwFDaxuY/YFVRHtaNc+oYNgF1RUD
	burWC1H1r/cYSjE+QyR2h4OVggVToGE=
X-MC-Unique: 8w8D57NuOR2LwubjXpPE9w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NvkQT1rUuI4Eqz7i6WLb341pGMjAvOC1YT+qTcDNZJxNz2lK/gcfqhjUo/HrzLZOu8mv3ZzgyqbuPqrXhpUvug/LEPdDUTkO4rVVl2ksCghG+0XlJ7NQ/18DluRlHZSTB9/mwAbEM632TArkcyw/W2dT96Porvtuj/YBYgPl5mA9bf7iRWHqY5+cIQHCBV3/rVNnH530HSwhgy/a+XyB3x/AVzaVk/KwlpWQOq9g9ubY6N7DEbG9G8m6uE44pUY/p2njd274Z1QDeS4xwnCHikJNi1up0JHclMD5WSBuMOJhALOwuh/SM8xEL0PyS27x6YfBIwuyGn8lBf048VNLTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hvPZWeko7LkrbiJynVatfRdxwf7pWGqz0z5S7aeVE8c=;
 b=BrHSRWq/gxetuiDzI4Qkg4sPnjGvP/Rs4Y4c58eV58X+lkXn9Kyktzk+lfDc/GalX7l2kRAyEF89/eGtwz9NHe6pkzzLEveHPpw5SrQtIBvWc9a58LxEyC2KMsgSjYUOya+ij9vUmFoIJUbN2V3uVnZAkApX9Ob3VgpFoITXhYQsx1TdMklSSoGpHPYQCwYcsOEFbOa1NtrmZGD4/veVcRmMwH/yqHQoK7mp20LfNtWephWSrsBo9+kN9aON023SXuVxYykf79gQKFmBzdYwvKrlIBGUbIa2LhPrCazV5bGloM/XsJnwO6DqPbSosEgPTQv/jfJSIFeaqD3vbWHL+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/build: suppress EFI-related tool chain checks upon local
 $(MAKE) recursion
Message-ID: <8457d422-4db9-59b9-d0c9-663430dad955@suse.com>
Date: Tue, 21 Sep 2021 17:43:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0029.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 01547283-45f2-40a3-4f4f-08d97d169520
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70375D88A1FD2D7299745DC6B3A19@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YFtrrc658vl/wEeQejojc+DhedhYB/sZU5uZPtGbVTykOQy1SQmKXlgtaVKttbnLZHiwSm9NBrsNZ1yw6NgQNBglTucaTEVYGgbKKMh0A/5nQi325fIVPWM8Y/glLxEGAfUYraF8H2sY8/vMBAmfIcnexceVA9bW1PORhL/KbRVjli8iK7oMhHbZDvkGkJwYsIqvl6kg2vrWOVaq2nFePXQaMIiCKfRBRN4RA4dBdlJPuGotremDKQO9+m6VjRF3+mhlIO7DGfv83RNHnfWEFj4+IcvvE1We9C9XPAsbxp+282dBntRFqEQqJI4PNKr6a3HoNG/EFuYnYNpFjUjOIEX25BzELRhYROZZi28q/JKFOpo+qPribZtOHst7LyUHwuRXy3/Vs1OttBq3/+ujpcLQ2Fkrw2g+whjD3aBzDphdNTrpWH3I7jcR+x7SMvo54K8z++XJ6XQPXp1CP84b01bLzI15V+xvZGrWXcJIf51P9Svn3dlEkIbpCK/KjWq2hDRTbJ0nypELFxHyxNd7yoM50EgzSXD77pBlMTmwns0Y06wHnbo/OAekc9DCQhLRc++PBu+2EYj2mfTSgWzMDAx6DzRFBmkzxGyWUkg/jx4BRdY5AhSJw7BEsUfg0LWXURDpG50WB7STIUcky4vlocW0SVNujdL57D91cwXybmnLWb912X40VO6maaBgMIt6dxwzI4TMiS8e9iyJsJ8GiKP6feqlL0wBQNE8HPgiOlg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(26005)(66946007)(2906002)(86362001)(6486002)(186003)(8676002)(8936002)(66476007)(66556008)(31686004)(5660300002)(36756003)(316002)(2616005)(16576012)(956004)(4326008)(6916009)(54906003)(508600001)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OTVpSVNQdXFrek9USzlya2JZRDVnK2Z3VFZBeTBlZGNzVWlsanExbXhqQW9x?=
 =?utf-8?B?REgvdDlWK1cwZGtxY2pKNWxhYzdVbnV4ODFDT0lXaHJRdUI1N1VqR205L0tJ?=
 =?utf-8?B?Mm05THkxUXZGdVJ4dUNLTTNnTDZOSjRuaTNRMW52WUMwRXB5MVI5NnBBcWdX?=
 =?utf-8?B?SlE4dmpLVnc0TmtlTEEzSk84YW13WjRWM2lFWkNvNWJTU0U0enU5Y0VxSWxZ?=
 =?utf-8?B?azlpWVZnbzZJdm1uWHRlOGl3OHhEREQ4eWg3TXlKRlFSNEhKUUR5RVBPL0pp?=
 =?utf-8?B?OGdvQXN2alhtblErZy9FSTN1VURqalpxWllZVkwzd2k3ZXorTmNxMUwybHg2?=
 =?utf-8?B?ckVnRWRxV0NyM2tuVG5TN3c3WDc0VERQUW9ra085Z0x0eXJrY2tVU3FMaGd4?=
 =?utf-8?B?WDZ4MU9FVUlBbEk4NFN2Y0pQZFY3WTVlTWI5NVpBTzF5c1BJMUMvTVNhbGk2?=
 =?utf-8?B?QXBLWnNFaWRDWnc0Q09LLzJSODQ2eG1iZlFxMDh1cGhNQ0RTSlNwRENOQ04y?=
 =?utf-8?B?ZU1GaW5NN1JTYUF4SFhzeHJkNXBuZzJPTFJLRGg5a1ZQUEtldHorTlV0U0hM?=
 =?utf-8?B?cHlrTk9VWEZDbXhzczlScnM4anV2M2xQQWFmdm5xejhZVlZFcU5QUFd5RnVt?=
 =?utf-8?B?WnkvRGwyTWtaZGR2WG01RHdlbzA5TlpOaHVzKzRONDUzTCsvRjhPOURnak11?=
 =?utf-8?B?a2lZT1YxZ3FHNDlXVFZYNEVCbkNYV3RvbHJCOHlJNEh0cW5STjFWMCtaaWNp?=
 =?utf-8?B?WXN6WGt0cFR4dGxsTzV2QXRTZUVTZ3Y4RjdIckhtUmQ5eWxaWTR5UnhhZ3VB?=
 =?utf-8?B?V3AyYy9XMXBqODIvMUo4Ni9xZUFoNkhsenRFTkhOMVNsZG5ja094MUtqRzBJ?=
 =?utf-8?B?ckk3OEl1dzZHTDR1d0tHNitVcmlsTzJMVEUzWnlDbHNiTlA3RVFRcWtETUFH?=
 =?utf-8?B?SGU4bnNvK3pnNnM1b2NWbXpxUDlJVXNFYTFQVUw1V0pQSTNwaWJ5MjFKSUtG?=
 =?utf-8?B?cE50YndOZHV1NWVWYVFPYi94bmxtaCtaRVphQjZlQVFLTDlpcHpnZFBWdGRX?=
 =?utf-8?B?MXBTdkwvTmdxVjVlaHdFbUk1SGJLUUFVV3A4cldUcTh1dkJiblhGbWNyallF?=
 =?utf-8?B?aVUvdWF1UEtBTEl2cVF1ZU9MT0NDcTlrOVBtTldkUnhyWWd6U0pZK0w4a0NB?=
 =?utf-8?B?RTFFeVhQRW5iazFhL3NYU3dvb1RRS29BcTdNU3U5Z21WWEZJR0Z3MWJ4UFNO?=
 =?utf-8?B?c216YTFwdHFhSHdFNXZKdU1rdEZ2bEs4SC91S0pzWHg4MU04RGF0RkVzREo2?=
 =?utf-8?B?R296c1VHUytKckNGUG1sSjJBNTBLYUhKVXNCZWY0OGFqTDlnNzZlM3lBbjFt?=
 =?utf-8?B?VkF6emJlREtzdjFLTExrZW1oL1pJMW9Idno4TEZBM0ZmQngyeU5sQjlRSzkz?=
 =?utf-8?B?akJ6UTBBRUNhNmt6UHROTWNCcDNWTENaWkc4aXJrVXVLR0ZYZUJCZFdCWkM1?=
 =?utf-8?B?VE1IcjROcldKVE8vTG9yR1d6eXloYmk4YmVEcUo0dXp2MmdjTzdmQmlYNU9j?=
 =?utf-8?B?eFlvUjFIbktjQTRDN2lWVUkvcmpuNGVjN0Nrdk9FeHdoa2NmNUhjQ1hRN2NU?=
 =?utf-8?B?eXJ2TGtkU0VIZ2x0OXRpNDFtVWhYbHNHb3haN2ZZOGdyQ0gxT1F2OTgrNDBp?=
 =?utf-8?B?Y1AzMk5nTStvWG9MVk0rYkNlQ2xmNkcvS0lGRkNDU3NSeHk3dm5HMXFhWnlR?=
 =?utf-8?Q?oqvtHJv58cXhuSdojW6rkfG0vcsxJqgoxyWE+It?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 01547283-45f2-40a3-4f4f-08d97d169520
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 15:43:39.5650
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JpVT0XZQ49gIHeFq7xZCwPgr2DBt0Ll6SwSrtk2h9AFSja1FdJZ5bqNcDLKJo4UgIhIA/qoeDrRtDX0XARZLpg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

The xen-syms and xen.efi linking steps are serialized only when the
intermediate note.o file is necessary. Otherwise both may run in
parallel. This in turn means that the compiler / linker invocations to
create efi/check.o / efi/check.efi may also happen twice in parallel.
Obviously it's a bad idea to have multiple producers of the same output
race with one another - every once in a while one may e.g. observe

objdump: efi/check.efi: file format not recognized

We don't need this EFI related checking to occur when producing the
intermediate symbol and relocation table objects, and we have an easy
way of suppressing it: Simply pass in "efi-y=", overriding the
assignments done in the Makefile and thus forcing the tool chain checks
to be bypassed.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Obviously the real (but more involved) solution would be to do away with
the recursive $(MAKE) invocations, by breaking up the long linking
rules. Representing them instead through multiple smaller rules with
suitable dependencies is certainly possible (and might even reduce
redundancy).

--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -182,13 +182,13 @@ $(TARGET)-syms: prelink.o xen.lds
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
@@ -249,14 +249,14 @@ endif
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).0) >$(@D)/.$(@F).0r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T efi.lds -N $< \
 	                $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o $(note_file_option) -o $(@D)/.$(@F).$(base).1 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).1) >$(@D)/.$(@F).1r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T efi.lds -N $< \
 	                $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(note_file_option) -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 15:58:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 15:58:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191797.341901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiAD-00062P-RL; Tue, 21 Sep 2021 15:58:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191797.341901; Tue, 21 Sep 2021 15:58:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiAD-00062I-NQ; Tue, 21 Sep 2021 15:58:33 +0000
Received: by outflank-mailman (input) for mailman id 191797;
 Tue, 21 Sep 2021 15:58:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSiAC-00060f-5Q
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 15:58:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c3d698e8-1af4-11ec-b900-12813bfff9fa;
 Tue, 21 Sep 2021 15:58:30 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-4-RDxiLevxMSu8YIGmp2-low-1; Tue, 21 Sep 2021 17:58:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3294.eurprd04.prod.outlook.com (2603:10a6:802:3::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Tue, 21 Sep
 2021 15:58:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 15:58:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0P190CA0009.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 21 Sep 2021 15:58:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3d698e8-1af4-11ec-b900-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632239910;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YzVDtu6c47NQj1352kpAYcG/QTW8cdOFMazIOozwIr8=;
	b=Cn56iMbrFVj9xfpzi995rHf1d2LczHnzYSEvXEegeNpBHSpFKfIhB7RctEZQFtY4xPiwqa
	SOIFCuJ6bX6SDP0D5aCYRakYLF8O76kqjC8UArWJF/nO7fHaNlJOMW1uBJAWNDsS5oY6Im
	pbJrUWVIxZzo2xAqr8mvXyDDWvv5+b0=
X-MC-Unique: RDxiLevxMSu8YIGmp2-low-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YH4huxZxQfVk5Foj7iR4zVc58bFjMfBrOv7E3HytfVqzcgnqvzYtvbXJVaCg9AJOVDHMQ9GiHcTXCk9uHbG9tpd6XnjdMfu7ioWGniT1687xb4Qcusvp/XE3o9OCIqxWVOyDn1DY1oK87/N8neD2VjWdHxMt10CBPZ8dzn7pSkeolKNsWPO4f4S3fctHewVr3VB8HNqKOnLB1Qf2SYbIuprb8ke3Qt33RB4M3T/yKo9ZJc/5Bnlnbc+57lK7MiiVAOAWUKVPxF48nYEK+lHPOdsl90oOKZhHRY7GQDU5Ss86qUGureogM4n+nLzMOKvoqxNmvSBfJoymUqoD4ywRvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YzVDtu6c47NQj1352kpAYcG/QTW8cdOFMazIOozwIr8=;
 b=nCI3sqMZI7ozvplt6TQrZgqjjgjF2nSHCsYzRZPhZVzAr84T+K/iQ7ey7WES5yw8daNlR8Wu8R4efc+qj+odrqoSOb5f5YK0HR701RpOtkEbj+1VehHizZGrHzSVs9hdg4QQY9Z3H1il6+n+DGhpO15y80dlpL7avvja35ix5ldQjB/u/gl6K56PO5zssuPx7P0HG4wLWiVtngDWe0rmDZkfC+jNkFNAc7v9+DAw7eB+nZm+lK4TW8+V24kzXwAVmHdY7EiMo+N1xPTfKf3GS77SMgwUEuuB7t35Nb5TuhcmObkCfkxhOTcnBJOBwDICL5oW+j1wyW3kt/9E7Hwnkg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 10/12] x86/pv: Move x86/trace.c to x86/pv/trace.c
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-11-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c64b5e8f-7491-9afd-c107-0e48a26badc4@suse.com>
Date: Tue, 21 Sep 2021 17:58:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-11-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0P190CA0009.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:208:190::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f2319307-cc7d-4092-ed79-08d97d18a648
X-MS-TrafficTypeDiagnostic: VI1PR04MB3294:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB32946070B94CC8D51BCDF029B3A19@VI1PR04MB3294.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	34QV1tkPc2LDFUQaL6r+dXYpElbUZXiP4/YK/bNbDDbmN2JWzdfOncxwOR/jRPN6U/N6ENsJs/yJ3FgrZnN8asCC6MEnGtgVL8K1BbwiGSKH2xZq/DlWTh0U/3SDbr10hY82J3ZhFxfrwGPxqzwX8RERLw66LEbLtqQCR15Fi2lFdEeQGz+yXpTxC5uXLIKPwu16UsRcNx6K2aee9qB6r/YgHcnwnKJiskH6XnykiXVKIw8Lx1u664GKT/eSHQw31zX2vPFy9N2AgYWx6aNNEVI230amwkBbXOMX579vrnfZxC9Zd30ANCiRQ2ZMVCAvDDEzDtHaMBATvR0+Kl8X8z7BvDbSV1YI3SpRmyBIac7CjDgz5EHDl+AzNlp+OySL4N0Cvq/SGlHch/1YDySS6hHxsb0HEWvXRkK923An+ldmbrlZ05brtCAupl87zCkuPVUqpMRWoRaYD6GuLI3loRbWSNM2G1qZV76FdwBP5TTfePJ23ChnuVqvUOSeT9zTTgSTLL+WT2T7sjCyPkudmDfMKvHn0vviPLLi5uwFm1BmpU+odziDXNsW8LoL84OS0/jedAeQIURwlN47nCC4uyNQ+3ZxA1q9hlucngHhc3CmfRpK/rT+qeOTUXSMuxYFrPYQaMPOZpiQpAGuE16N2pkjL0f6v2ETS00piZfhhwIuudwR9eiAw2PddekaFscWgomoQwcnSuaPZTL5FZKCykzcN1zkEoYYVVFKDuzwsCA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(6916009)(4744005)(508600001)(2906002)(26005)(8936002)(16576012)(66476007)(5660300002)(66556008)(86362001)(53546011)(4326008)(316002)(8676002)(31696002)(66946007)(38100700002)(54906003)(36756003)(956004)(31686004)(2616005)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?K2VZQm1NeWIrVXQ2TFBldzlMbTYvbm5rVkQwaFhrNlZUeEg5Slg1WVVuTUt0?=
 =?utf-8?B?MFRXUEc2MFQ1Q1hDVWxJR2hhQjMzNjU5VFBhRE1yK3YyN3cyVzVTdlJTc053?=
 =?utf-8?B?TXlQdjI3WE9OUUlkVEdhM1NURzlGMGRpWll6RG5LN2h2Zzd4TmkyKzNmZjU1?=
 =?utf-8?B?QnpSZmZ4ZEhtb0VuejFhMUt3WHZrQTUrYk1DdnU4dGlzVFNiampQa0NJNDYw?=
 =?utf-8?B?akYrU0RSeDdLM2lJcFJlSzIzSW50WThzWkV0V1o3dEsyVFpNeEZHWWdQOUFH?=
 =?utf-8?B?S3VZQnpnMXdWeFIwSEZlUHNieVRUZ3VLZ3JubXRFNzhjRXRndXNKVDd4WHNT?=
 =?utf-8?B?cjRhTG4xUVhQMlpsR2NmY2w0UlNkR1dzcXlKTWErcjNZVGxETUt0OGlOQitJ?=
 =?utf-8?B?UERXUGNVS0xVSUd6VUNyYXlvclpVbm5wN1pkV3NWdjdLYnV6QmhnZTdSeG42?=
 =?utf-8?B?bW1lNWQxdm50ZFhaTEZMREVVYWxmTDJIcG0vdHV4RWxsUjA2N3cyZnRXb1c0?=
 =?utf-8?B?TENsdU80NlJnYjN6bFYyMXcrcEFpS0ltTXRvaTFHZ0E0YUlTOFBPUXdsOXhZ?=
 =?utf-8?B?MGh0aStEQlpkSHJFNXpGOE14d05PMnZQeVJnUTVFRTNDeUZZQVh1N0Y0T1VQ?=
 =?utf-8?B?bHRyaFhxZGFQaGZBMk93QTNXbkgvdzNXdkVrdGZ0ZG5xaUo3NC9WSGpBdW5t?=
 =?utf-8?B?VisrV2Y1c3pNcUFTQkhxdzZsUXpoRUxucUkwblZwa1JmcEszSC9jeEVKdnp2?=
 =?utf-8?B?VmRGZi9sOGNMYUR2OEdKRXVzYmQ4VXE2eEJFRUd5SlcraGJFSXRUZWRNeDNt?=
 =?utf-8?B?V0xBUXRGNnZxWHptczNjM3Q0WkszOUQ3WDgrc25UY0hkZGdzVitXMHA3czA2?=
 =?utf-8?B?Skd6TkVvYTM3NS8rcERtbDhoeEJ0K2lub0x5NkZWNnlIUWxOTkJlNjY2ekdt?=
 =?utf-8?B?L3N4ZG9id0NIU3E0eVUrSS9TRW44dSswakg4VG56MitNU2NOL1NIRFFsbm9Y?=
 =?utf-8?B?NHRUWkhQYXQvK2IzVE83ZElnTVRKb2tCWkRaZ2VPSENic3E2VjZINzFQNnM0?=
 =?utf-8?B?dVJoQmZoamdUc0taR3IxMm8ycU9HRzBSSVBIdzJzRzFMMXdJWUVMUmVRN0pS?=
 =?utf-8?B?YXhZeW1QVW00aXFuZkI5SE85bENIVm9uRGRTV09ET3JqelZHb1hZaENaTzhp?=
 =?utf-8?B?K2h0SUFPUlhkMkpTSDRML0t0SjhFVlBKamZHcE1UYjQ4c0N2R2JOVFdqSlg5?=
 =?utf-8?B?cDUyVjRBOEN5MmhDLzdVNUpWMHJvakMvUkN5VmdSUnk1L2paYkNDcjFJV0Ns?=
 =?utf-8?B?dWhIOGxta0dqQ2VDbVR5aFhpU3E5a3ZONjlLd3hpOFNFeXM0UFpYdk1rdy94?=
 =?utf-8?B?QnlndDF4SkNGd215SXlaTkg5NEdIdTNmakhvc3QrdTU4T0FwL1FjbTA2dDZp?=
 =?utf-8?B?WElRVklxak5nNzlvL1d1akE2MlFsc3loby9GN2NDcW1JQUMvZjFKT0ZzcXlT?=
 =?utf-8?B?NkRoL1FTOU51ZnoxWUkwbDREVnk4UU5qbm1qVWU1TDdKbUNKSlFRb05xaFI5?=
 =?utf-8?B?T25scGt1WE9leWZyS2FET0cwdGoyeVB3RW5sZUl5RzFyckF1WmJ4WnZBVVAx?=
 =?utf-8?B?S1ZzRE5RYktnZDRHaVMvT1Z4ZktndjFML1VBOXIzTXJ4dlhWT2FlOVYvTnNz?=
 =?utf-8?B?MnpoNTdGYzF5WmZFc2xVM0VMQ2dVeHRVQzJmSjU2QlE0Sm9RcHFWWTgzbWp2?=
 =?utf-8?Q?wdzbVgtkFROS/IKhOCJqYRWFmZoohzdS+clrohz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f2319307-cc7d-4092-ed79-08d97d18a648
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 15:58:27.3039
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: r3k6vKysNA2HGyK8yUN0PLDLMz/m+K5OysqWDAPjsNGwrdLh2j9KMz5uVdTkL693vHJg+xyUS7DFhYXa5fUUiA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3294

On 20.09.2021 19:25, Andrew Cooper wrote:
> This entire file is pv-only, and not excluded from the build by
> CONFIG_TRACEBUFFER.  Move it into the pv/ directory, build it conditionally,
> and drop unused includes.
> 
> Also move the contents of asm/trace.h to asm/pv/trace.h to avoid the functions
> being declared across the entire hypervisor.
> 
> One caller in fixup_page_fault() is effectively PV only, but is not subject to
> dead code elimination.  Add an additional IS_ENABLED(CONFIG_PV) to keep the
> build happy.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 16:01:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 16:01:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191805.341913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiDU-0007yP-F1; Tue, 21 Sep 2021 16:01:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191805.341913; Tue, 21 Sep 2021 16:01:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiDU-0007yI-Bu; Tue, 21 Sep 2021 16:01:56 +0000
Received: by outflank-mailman (input) for mailman id 191805;
 Tue, 21 Sep 2021 16:01:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSiDT-0007yC-Ck
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 16:01:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3cd92b48-1af5-11ec-b900-12813bfff9fa;
 Tue, 21 Sep 2021 16:01:54 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2053.outbound.protection.outlook.com [104.47.10.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-psT7f8bOPOWNxCAZvh2EpQ-2; Tue, 21 Sep 2021 18:01:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 21 Sep
 2021 16:01:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 16:01:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0045.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Tue, 21 Sep 2021 16:01:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3cd92b48-1af5-11ec-b900-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632240113;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=g4DoNcY/J/VsXztfC2Nh0R7m1HKKgj/Pb4dk2bN2u8Y=;
	b=H4R2zOwRbggB3mu1c1vs+2XN2EZQd0O1X+hPB8/gNnccdH++qeVsplDvUDETsJRFbUavv9
	5t+t3Vil4xb57XuA1SSi8ST7VpuAzzxgz56J/srLj6cpzGjrR5sXSqzAtB0rK4Y5NHRLLG
	t+l9X/Xu3a8TUJc0YkKBRuMTvlSyRhk=
X-MC-Unique: psT7f8bOPOWNxCAZvh2EpQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HVV/951G4GGdW0hPKF8Pw5rEXZx0zNxWlVR2EBFIpg178qhugCg3LEhuzbkNlmWxOHep6F4cp86WQ4rzVGbzW5G32oELeM++K81eb84qfntutFvsdTcXTBUbPB02/h41IpsSAL1agJDmsTW9QPrvloHbRz3Q+Q3MX536+wWt5YrOZ7o1eWpWrNnifMsq2z4aHySl3IMOd/U/ornBYF5LQw0+KV9TlcLkNAbv8Ld62iTPDoxCxqkvdjgOLECzelU0zvA9l/DlgukGl8XtUZM/NSvMwn4Zxgbp3g4hKEt7JVGxzoCcqIgmRE/hiJKr5cRB/EhMqZFJrt3wbSZD1Fn/Qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g4DoNcY/J/VsXztfC2Nh0R7m1HKKgj/Pb4dk2bN2u8Y=;
 b=Y0YucGH6Q5B9w1ZeCeUzBjewZU7iI5Sbs9bkv4yZ6ETsylB40El0662VjNaH41XrN0D0Hjzf1FG9f2BKaFFnonLCmXD6RQSPAjwg13FHKqVJDiJugr1iROoaBh+KVuGGeSZW/tQEnfznwrOooE8P3154wplud3U7bEmUK4sFs1Cc92kMKUJEdm1zQ9qhQ88zgbQnnIhX1wesdXUvJuOuk38JVBQFYaK1J1oQPvLD9/PXrX2RuUoSPE48HAE0OcECaLse/rmvTQPMoeI13YhORpsuPdlcv6vTMBj0OdJ3XYY25DIK8LsPuhv1QfsbpIbIS7q3KJLp84UDTLAzL682pA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/12] xen/arch: Drop asm-*/trace.h
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-12-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <aa361622-b4dc-a8e3-f889-f3dcb65cdd4a@suse.com>
Date: Tue, 21 Sep 2021 18:01:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-12-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0045.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1e59bac9-c30e-4773-8822-08d97d191f59
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6304E217A361924004506BA7B3A19@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PajH8ddQIuU9zcsm9eJJkxSkI+3btXiS97Dw1HFRoHrcmeEA543wsVW+3juLhV1Hb1IBzwm0JRpHLFatqZ2QVPrFbmvtk+oUDadngIqy5Bsf+FleWSJHgneEKoi5O1YBmrA1wgeu/OkajgYhKiODRodoC6Fc/81ZbngbHDNsqvnLlbMSBzyWSyeH0Cl7eGg3mdLv9ndJMOkYOswNQmtTYzHVoW8QzmxQYbmwIpjyf6LmK/sU6Gs2SsA4RdU0G250JQ31A1kBt499txjH7us52CBQiMgF72qRdDpdGTv9/0euGg6s6AYRwaIhlbvdyKclaDnJ5oZgrfjCB+qZMCI8V8B2sYb2Eq+9YU8Ns7pKMkcEaaFVvOOrcXYirZwFlwazeA24AeR1LWU8OmHOztJwhFgyHaRvTIkzHbVNaSdTfAdUucEwnYjBMg1b92GH4w2O9CAULmtIQ3hY9Y3gVtQRC2ObugrKfcsEplvwwhHOi1OM6QFLyNZGO+D9liRVxWO66siCRfolHG3YFWVLk0M4Lbkk03SUWfqfl88VD7MBph5LF1mM533g3xG6Uam+xQxceGaOljbJQCbGGX3ZuXWRUzwvJGSllNy5wgTL5acM55xAlrhmNCVuGTysBTE3o6WvNM2OUNQBMBLc1PcK1Pu1NhrZTqxBMB+l2bD+SK0nmjv0sif2H9iI/NMo66PsARVmSShjjvG0GyXszDhUF3vZ97f2viPHbE7HQnVe3XIDvVQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(956004)(4744005)(5660300002)(316002)(86362001)(6916009)(186003)(31696002)(31686004)(66476007)(53546011)(508600001)(2906002)(4326008)(6486002)(66946007)(8936002)(26005)(36756003)(2616005)(38100700002)(16576012)(66556008)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VjZvbWJwTWhBYXhuZmJubHNVQ3Z0dG00VDNSVTNQbkRTaXM2T3J5QVJOVHBS?=
 =?utf-8?B?NnNCUXI1NE9GdEV4RnlHSXBZeFNWL2l0M1FuWHhWRWh2dXR2TS94RWRyMGhF?=
 =?utf-8?B?ajFTS1I4eDR2d0pRSXRVb0JJMFlEdyt2N1Y5aGtYSm5VNms0YlcyaXRra1kx?=
 =?utf-8?B?MlU3NjVLcHdmM0ROREFyZmMrNFVmMEYrRUg3dUJqdHArNUtuNGtDZWNiZVBC?=
 =?utf-8?B?RGdoTUNIM1lIWURYdzBqU0YzZ1p6Wm9VU1kxaGRZWWdkNkJPcUFjUUNYQUtv?=
 =?utf-8?B?MnpIN2k3aFczbDBCVEpuQ013RXdGM1lGd01kc2FTOXhPTnZoMHJhTzBSVnN0?=
 =?utf-8?B?VWtMV3A1V2xwa203M3l5QnJwRThmdEg4aHdhVXpjYXc0NWZNZ0ZjMk45WlBl?=
 =?utf-8?B?UkVrSU5QNTVtN1hIZUxwOVVZYWNnRDEwclQyL1NSb1dyMnQ0TG9LVlQvYXpP?=
 =?utf-8?B?NTlLUjMxN2VrU0FsRTY2SUtqY1RJZTErTnNsK29tdGpYczloZWNHWHNjeGVT?=
 =?utf-8?B?d2RFL1BQVTZNV1RxUU5sN2NVeWY5TzczZVV4YThiNWJydHVuc1c5bFhtWHla?=
 =?utf-8?B?ZHFhRDJsNThYempKK1JQOEJYOFJtRG53VkcxSEdIa0svTDUyTXU4cEVsTkdD?=
 =?utf-8?B?QVA5UUlWU1k2eHpJdFc4L1lsSkJ3YUpCdk0yV0lLNjZkWU4xV3pRRXBibUNB?=
 =?utf-8?B?K3JWSEY1Sys1N1JQd1liQkU5bW94UDA1RWJqYVdDQk9Pc3B6YmxsUXY4Qm1l?=
 =?utf-8?B?TCtlbStGMUYzYVMyREVackVkN0FaOUwrWjVmUFVQVFJIN2NIczZ3RDBPdUpx?=
 =?utf-8?B?S2NaaFRGSkR1N3NyYnNSWitXMnNkZ2psQlNick1VeG1LempiVlNoTm5Obm9U?=
 =?utf-8?B?SGFsL3ljTVFOMTkxYzZjaEtUZXp0QzJvVzBNcWQzSDFoZ0lDWXV5ODFGKzk0?=
 =?utf-8?B?Vi90aHhmbytxTS85OWYrWHA5a0FqR0Rkb2E5OEtURkJzRDBXNk9xRTk3NTJr?=
 =?utf-8?B?MUFvaVdaR3FDMnIyaW1WakVISUF1Q0tldEtiNk1LZlVUNlRmQmNTeW9uVi9T?=
 =?utf-8?B?MEorVWRpYmQvS1VUT2htenlZanBNQ2JGNk5rWU9hZTQ3NXFQeThUNkxyejRP?=
 =?utf-8?B?R0lmNXJJMXVScUhRYVc3a2IrZXN3M3BYMnZ6QlVOMlBOUjZVSWJNMTIraXY4?=
 =?utf-8?B?ZTVBMlYyZUNNMnNOZ1RaUE52OVpUYW82TFUyeGNNNDNXWVYwZjNJZWVwYlBu?=
 =?utf-8?B?aHprMTB1bmpvSTRmRUIvVzA0MFordVhCbENvRzFHaUtqMExkSk5zRjBpMFVz?=
 =?utf-8?B?emVHQnRTQ2l5T0g0ZUJPRllZcFIvN0VFVktTM3lqY2s5R2cvWEZhSFFTWjJi?=
 =?utf-8?B?MFlqU3RSSFhZenA2emM3Njl4Q3dlVEJEbHN3czVsSW9lRmFIZ3NXc3hBcFdw?=
 =?utf-8?B?bjlqbythMGJnb2NPblhZQjNOdDNiL3RsU3MrdmQ4SzFJbWx3cTlnMTZESS9I?=
 =?utf-8?B?eHdnL0crNWhGaHg3L09zSzAyTE5iUTR1ZXdXMHpIbFRjMC9URWVLZzVoVHls?=
 =?utf-8?B?R3M2czY2b1p5WVAraVViN0RwMFpOSHlVbUQxTFFKeTFCVGlxTG9STlp1bFUv?=
 =?utf-8?B?SDkySHgrTHkyK1BlbTNiRXA3aEZTWjM5M2d3ZGl5dWhPOTFhNzVQR0U5S2Yw?=
 =?utf-8?B?SmFRWmd4ZkFHQkY3SHFSWmdZOFJmSk5SQXByVE9FQVozSkpuS2dScG9QT054?=
 =?utf-8?Q?2Xpb6QouC1tPiiBR17JAtgV/7GvkMrDQ9xv8KIY?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e59bac9-c30e-4773-8822-08d97d191f59
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 16:01:50.3911
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ajeSaNg9IU+6jlXXKTxcpaqyucYTTn4G3T5LsJN0a7M8CIcyPR887I1d2pZwF9TBhYE0CXrpV+hXqZvb0JGb8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

On 20.09.2021 19:25, Andrew Cooper wrote:
> The feature is x86 and pv-dom0 specific, and each architecture's main trace.h
> files are empty.  Don't force all new architectures to create an empty file.

The first half sentence looks to be wrong according to Julien's
reply to patch 16. But the change here doesn't seem to depend on
that aspect in any way.

> While moving the declaration of tb_init_done, change from int to bool.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

With a suitable adjustment to the text:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 16:08:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 16:08:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191812.341924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiJh-0000Lm-6o; Tue, 21 Sep 2021 16:08:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191812.341924; Tue, 21 Sep 2021 16:08:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSiJh-0000Lf-39; Tue, 21 Sep 2021 16:08:21 +0000
Received: by outflank-mailman (input) for mailman id 191812;
 Tue, 21 Sep 2021 16:08:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=n1Hg=OL=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSiJf-0000LZ-GY
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 16:08:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fb6967ec-d273-415a-a586-478e399b1f65;
 Tue, 21 Sep 2021 16:08:18 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2052.outbound.protection.outlook.com [104.47.1.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-vFHV_xWuNf-hRxxZcyxw6A-1; Tue, 21 Sep 2021 18:08:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Tue, 21 Sep
 2021 16:08:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4523.018; Tue, 21 Sep 2021
 16:08:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0160.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 16:08:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb6967ec-d273-415a-a586-478e399b1f65
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632240497;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NR0Akg+4kX+0ZzQheqIw5Txm2AdPCdvBk1SsmKfxUW0=;
	b=lAaom/y0iZsckFK14twPT2H3KpAoOdVU1W9hUA/EfXQv959P/lvdBXPy0jdhO2pMH/YO1Y
	rCUR9q8t2cMv9YiIp/4u+mHCS1VDuIf/FtPHHEE3ADVZai/M6sF/OjnKMgxH/a6n37qTED
	FcVUge3G2yDbkcnqv28NZhMOfT8Cj2I=
X-MC-Unique: vFHV_xWuNf-hRxxZcyxw6A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nhU/Huk+KzPX5T6w93G+kWRiML+laZJIUbq+ELp/GfHpahiR+ILbpspkFF4tzCUj+Eo8OwSwf5yIfYFbxFGNQwehfgdk+sSuScjv9IhaYxOxgy4IwnoNI4qFSLZz/Y8UVF2ZTbqac8XZcPprmPh55QVI9eW2s5hGoGattLTSt/kL1fTbqRX8+tvEHmRIePxdUsBuYTKejiZYeNGv/R6U4B65H7WRmCK1EXMpqFvZlRU1EmgdfZBszVg+i6aMaC7G9LteKgmK90Wk2nw7H5VO1l3agcn12wkF2dCuq+eFVd6ytfcUdUL8qqLfDP6j2lgsPNi2bGaDXce1wPLGh6hDsA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NR0Akg+4kX+0ZzQheqIw5Txm2AdPCdvBk1SsmKfxUW0=;
 b=R2jZgsOS5y8NM+UxlldHs3JX+dM9wK13pA9zXIZ3UlFrWSxI/DazMg9ak2eorvdhhrS5wzx4DV3GglnJ7DWFViYiZ0icvsyFtC8wIEYNJX+yzjVvzDmlrbQi+aD2J0acYkWjybUWomJr7XpQ0qoE3dlpRBOEtFtQZNsdI/77TBMDOJReQ10z+5Ze18e7FIX3kZ65FpZYZWowvw8CKFIRJrNpgij8Cn2mc7a+aWuv1Yn+bygGZ/I4gkyCz7WoxQEL96mjyxrr6mFvODAjEELDrF2HRSRRme7kBg4jJ3DTv3y1FQ/xE4ZNF02ZIbcKHg3o8NwrEvAbQ28Wu5p+Q87phQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 12/12] x86/trace: Clean up trace handling
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-13-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f953cc44-3da4-6860-6e55-6a5751c4b3c3@suse.com>
Date: Tue, 21 Sep 2021 18:08:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-13-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0160.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a3bb977-c766-4dfb-b27e-08d97d1a04b7
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB630405ED6892C307F896DD09B3A19@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mlFjVAd7SgD7g2XY+LzoctFUb2bKHCRGKmdctEPCHsO4B5WNOEuuiTqoA+cxeO8nF7jdMHICshy528FJqSSy6eK857Yzd6EiFpvztu9KIUSA9j8fv8coe0OFphVLgblzo7GRHfQAp4J15NkOAJLOtQELhX6MAris/9TQBv3VeluhDxlKCqlQYbzLdYaG9+cglyJfyxMI8EqRh83Yhhx2WnQqLT7kryDZheVPesJ01oCIQoxJrM5UKsvO/xrx7bu2s86rUU8qH1chOwXR3Q+NjWjHPxG7BEnTFiXXKUWHWpZzEOLEeX5/qVmoWbw2c8mpRMcAnATH2YklfKa5gQCEa/7vGNRevHJfh79QwvlBF88cpKn1jJ9Et9zb6bUjIRrlBImepozrb8QkpLBrXk8SAhiHjGHTBxVBlKwCsVkOP1YIJRDdz3JTrYJousfMTP5JJv3Fym4c3UKL0MNbZUEkEwU9lJEPJ7lI1pdPqDyjYQ9ocxL7EcX7zpaattm+Vlp6E2v1GRngKEtViMnxxC9X46t/HR7xyQVQDKjP7LIz5yFPV0N8S8DfqjPbEmoKmZSYmr4W+JrN4WbPelixKZiJ9lx+zYEh35W42NMfVwCzjxPR2w5K3gAMXXMsCEvkPNKo+lFVUFxqq79aDCSdT/aSR/qsdP6bx4OrvLNXNResIH7lHZ8+7XH4RliaFMP/2iFSHI7vD90ybFn727cABiEqv9IKKQUR/UYNrHAsF/pcfJU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(4326008)(31696002)(31686004)(66476007)(53546011)(508600001)(38100700002)(16576012)(66556008)(2616005)(54906003)(66946007)(8936002)(6486002)(26005)(36756003)(8676002)(956004)(86362001)(6916009)(186003)(5660300002)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WW51OWwvbUhxcExHc2dSN2lNN0QxYkI3S1huS0tRd2JSbFB0Y1VITmY1VDJi?=
 =?utf-8?B?WWtrd1o2QzlLNmJBUUZ5ajNRdDRaUnpyUGpsM2FjTVZmTE5nQkZ5RXJwcFBY?=
 =?utf-8?B?aE8rWWM5UEdiTUs3M0dPUHBkOEJkckM2VGRRaHFVRlVlRHJtR3NDV05janZS?=
 =?utf-8?B?ZFFVTzRGRmtCWlIwRjlyTkJnWWJwQU12b2xlc1RhdUlvdUNsNkZyMjRLQkFa?=
 =?utf-8?B?eEdONGUxdXM0SHlvMmh2QlF1R05KWEtiYk5mcGpTYldPYS95aHByWWt6b2NY?=
 =?utf-8?B?VzV2NjZJYnZJbjVaMW9QVC9CSmRsQXpiTmMyUlk3MjgvUHd0bWdiekc1ZENQ?=
 =?utf-8?B?YmFaREJwVFN0NUtxNjZid3c4amxQR24wUmFGSzBCOVYvZkRwUXBxYnBLMGpQ?=
 =?utf-8?B?a25WYWxFL3ZiUWcrdHZNbE5KN1dpek5jMHhkTkQ0blVST0s0SHFoek5LN1F2?=
 =?utf-8?B?QVpOZWNtT2lteWZTcGVzczFHRWFtVThMV0UzRGRQUjlxc1h5aFdYREkvbW45?=
 =?utf-8?B?SEdJaFNPUzFoeE85aExHOW53bk9nRkJrRjJDSGtPSThoZW5OcHFuSzJNUTh1?=
 =?utf-8?B?UE93RjJzR09DZHZzakhyTmo4dDE0Uzczb1k5NW1MN0lva2toTnpEWUJFa2xu?=
 =?utf-8?B?QU9mRWlPVlhBV29UWFlNbnE5QTNDNE9pUEsxSXlBV2hKR3NCQnpsYXZ5Smo3?=
 =?utf-8?B?RldueHdHSldBL1JpY2hGbHRuL3NKU1U5eUl5Y1h6ckE3dDFPcllOTk5IK3Mz?=
 =?utf-8?B?dmlHcHlRa21jYnBKeHhqZVViWEVIRVd2VzRnSXdhQWkvTjkwVzVxY21ORFkx?=
 =?utf-8?B?aHE1cllWZHFFS2QrQ0picjM1MmppaDlSRHJ2d1ZXVVNsUG5EVXA3b05MY05C?=
 =?utf-8?B?OHBzMUlFWjg3TlR4eFc5OXVXRm1GQWVmYWlrTm1Qd0k4d2piVkJ4M3ZlZmMr?=
 =?utf-8?B?TEJDNS9NOG1ZY0xZZG1ZczEwMk5XOExKdy9zWHFXWWpRNy9TSHhIUGdhZzJt?=
 =?utf-8?B?RkZpaXpFcXorMVFPYktwaU9LalFJRjNLdVhuYmpGRHE0ZUhwWnlJaURlT0M0?=
 =?utf-8?B?WWdDTmQ0UzI1WDdwakljUGhzU1dpazVNQmtMd1B0SFpVV1B3ZmpnMk1ERXFx?=
 =?utf-8?B?MHVNUUhaLzRibzhTbWZ3ZTJ0K1dUdTZKMXlITGMwTkRLRGN2QW1ydEFLWFR6?=
 =?utf-8?B?aHVwOWtrWlprU2I4KzhTM2lDSjlILzhYNHNORGQycVhyMWxlY1NRTFY4Tlhj?=
 =?utf-8?B?UEFWMk1ObThOZ3FLcXpHemFJMHl0bGlZS0VQMll2QTRicHl4WWJQNkNWRi9k?=
 =?utf-8?B?T1FwNEJJMTZBZlh4V3BrV1lWNTZHNUpLWWcwcEIxanpxQXZMZGZUaTloYUFH?=
 =?utf-8?B?Z2FvVGtUekVhdnJSbDFwaU9scjV2dVc2R0VBNlBvbEZHRmVDTWNKa3lvVDBR?=
 =?utf-8?B?a0FGamxSSFdNL3VtMVZXV09QY29xdm8zZk1NbVNyV0gxSk1XRUJhV3VYSk1V?=
 =?utf-8?B?bSt2bmh1QnlkelNrOFQrUGtXU0JZMFdoVmpGRTI4bUN1UUJXMDZxSWZwdUJM?=
 =?utf-8?B?ZTlrMlEzZTlBZktFVnpOT1c2MWJab0pkcHIvWS9zL05lbzlHbUtMVGEzKzdM?=
 =?utf-8?B?QitEbmticXN2S3BZeExiOFk2MG9PUEswNWZGQWFuWEcxMzk3VTl0MTBzZXcz?=
 =?utf-8?B?c3lzb0FtajdLYWQvZkM4VXZTaXpDWi94YUNnMlF2SnlWVVhiUURueWZKWDlo?=
 =?utf-8?Q?08Hnu1Mmep6DTns0AUOtLbdOITpQwR05x4kTA5S?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a3bb977-c766-4dfb-b27e-08d97d1a04b7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 16:08:15.2344
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: z6u8foeE/+c8Oo7Y/1SErLsaBy0Blg1I4k+9SLgFjn6YwnCZSpuBtQxNclNJjej+vO5Frg97b+L2qZG3dkOv2g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

On 20.09.2021 19:25, Andrew Cooper wrote:
> Use more appropriate types.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with a suggestion:

> @@ -48,30 +45,28 @@ void __trace_pv_page_fault(unsigned long addr, unsigned error_code)
>  
>      if ( is_pv_32bit_vcpu(current) )
>      {
> -        struct __packed {
> -            u32 eip, addr, error_code;
> -        } d;
> -
> -        d.eip = eip;
> -        d.addr = addr;
> -        d.error_code = error_code;
> +        struct {
> +            uint32_t eip, addr, error_code;
> +        } d = {
> +            .eip = eip,
> +            .addr = addr,
> +            .error_code = error_code,
> +        };
>  
>          __trace_var(TRC_PV_PAGE_FAULT, 1, sizeof(d), &d);
>      }
>      else
>      {
>          struct __packed {
> -            unsigned long eip, addr;
> -            u32 error_code;
> -        } d;
> -        unsigned event;
> -
> -        d.eip = eip;
> -        d.addr = addr;
> -        d.error_code = error_code;
> -        event = TRC_PV_PAGE_FAULT;
> -        event |= TRC_64_FLAG;
> -        __trace_var(event, 1, sizeof(d), &d);
> +            uint64_t eip, addr;

Like you've done in __trace_pv_trap() and __trace_ptwr_emulation(),
perhaps name the field "rip" here as well?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 16:44:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 16:44:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191824.341935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSisu-0004eI-2g; Tue, 21 Sep 2021 16:44:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191824.341935; Tue, 21 Sep 2021 16:44:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSist-0004eB-VJ; Tue, 21 Sep 2021 16:44:43 +0000
Received: by outflank-mailman (input) for mailman id 191824;
 Tue, 21 Sep 2021 16:44:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Yhcq=OL=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSiss-0004e5-4b
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 16:44:42 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 25484dc9-5de4-48d6-b73c-778bbd53b5f4;
 Tue, 21 Sep 2021 16:44:41 +0000 (UTC)
Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com
 [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-119-gQyyLU3GPISV1xCadnJOow-1; Tue, 21 Sep 2021 12:44:37 -0400
Received: by mail-wr1-f72.google.com with SMTP id
 f11-20020adfc98b000000b0015fedc2a8d4so5065866wrh.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 09:44:37 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id k1sm20272243wrz.61.2021.09.21.09.44.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 09:44:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25484dc9-5de4-48d6-b73c-778bbd53b5f4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632242680;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2MOh25Z0CXinUksqYDluCoredOuFNiGvVHCwdZh/BTA=;
	b=Y6nVQSqbJ8FqJsl2l8xxkw+qDNSMwjwmdDd6SOH70Tkxo9KP5Zk1VyKbigmYlx8egEAxmE
	WXRbu1DBL/WgrhuqV7ixXsRu6Szi4Xs6vdIIoGzO5cGDvV5uyvX4DLZwiLKwhFlVZlhhfY
	DXxDNx3C6abSViWjF0ZUUcVKgqCfYdU=
X-MC-Unique: gQyyLU3GPISV1xCadnJOow-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=2MOh25Z0CXinUksqYDluCoredOuFNiGvVHCwdZh/BTA=;
        b=zuoRkDyWLbrEGOimO3CYcW9ku8ftUet2vquBPjoN/CW2Of2YPfyKG5Yp+J02QmHPy7
         CEgyijzPqTxwFI2e0kvaEgDnXfU8kKISbvBicjasWUQalzzQvXVpkP9sQqfnIXK1r180
         x0OJzWPiOk7dyHbEyvpFSr36RX1NofaKhYF6KQX0Gx+Zwgzd7lUKZb0xCtpON+wQhfGX
         sMOtoHTuxMMlre+JQRQy4e7bAJxsmWDUGSGUdjRPRASlwXzNsiP8M4AD5rmGgJP6aVW0
         3hT8vMGMRs05W4uYehE2BHOKOBEhtjawDeyaq9uNhzn8rXiv8SkLXWAftCqnlqbPjsCA
         z+TA==
X-Gm-Message-State: AOAM532PAsVx0fzT4y4DQex4oVhlTwHmkYklbxtSJnhMAcacwHPEjnYv
	mpVMfMO7J6Ur8JgG9RPZSkR9ZouytMsVx5wT9FzMIEwOLNpGkRc28i4j/i/dxJtV6n4Wcws+mte
	u2z1qvfW+qq01eTGubKwBr+waExA=
X-Received: by 2002:adf:ef02:: with SMTP id e2mr35671914wro.401.1632242676307;
        Tue, 21 Sep 2021 09:44:36 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzsCp5ZrdIqXk735XH8pFAzorkNCbfLAUPGTe1MtIGjUyvp4uSPpnZgIUWeU9z8tn602zxqYg==
X-Received: by 2002:adf:ef02:: with SMTP id e2mr35671861wro.401.1632242676096;
        Tue, 21 Sep 2021 09:44:36 -0700 (PDT)
Subject: Re: [PATCH v2 05/13] perf: Force architectures to opt-in to guest
 callbacks
To: Peter Zijlstra <peterz@infradead.org>,
 Sean Christopherson <seanjc@google.com>
Cc: Ingo Molnar <mingo@redhat.com>, Arnaldo Carvalho de Melo
 <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>,
 Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Thomas Gleixner <tglx@linutronix.de>, Borislav Petkov <bp@alien8.de>,
 x86@kernel.org, Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Juergen Gross <jgross@suse.com>,
 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>, "H. Peter Anvin" <hpa@zytor.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828003558.713983-6-seanjc@google.com>
 <20210828194752.GC4353@worktop.programming.kicks-ass.net>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <8ee13a69-f2c4-2413-2d6c-b6c0a559286e@redhat.com>
Date: Tue, 21 Sep 2021 18:44:21 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210828194752.GC4353@worktop.programming.kicks-ass.net>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 28/08/21 21:47, Peter Zijlstra wrote:
>> +config HAVE_GUEST_PERF_EVENTS
>> +	bool
> 	depends on HAVE_KVM

It won't really do anything, since Kconfig does not detects conflicts 
between select' and 'depends on' clauses.

Rather, should the symbol be selected by KVM, instead of ARM64 and X86?

Paolo



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 17:35:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 17:35:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191832.341946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSjfx-0001ex-VD; Tue, 21 Sep 2021 17:35:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191832.341946; Tue, 21 Sep 2021 17:35:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSjfx-0001eq-S6; Tue, 21 Sep 2021 17:35:25 +0000
Received: by outflank-mailman (input) for mailman id 191832;
 Tue, 21 Sep 2021 17:35:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aS0w=OL=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mSjfw-0001ek-DT
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 17:35:24 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fae314a1-2aac-499b-a095-6be0f9346101;
 Tue, 21 Sep 2021 17:35:23 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id b15so566592lfe.7
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 10:35:22 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id z16sm2024386lji.41.2021.09.21.10.35.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 10:35:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fae314a1-2aac-499b-a095-6be0f9346101
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=/jP81gv1AslQYHUXA2ClcvKMKyvf+YRZSggmAzN4Okw=;
        b=m3EU9eaA86ZKspM7TYEt9E3CXhHrsQvPyi7mXjEaYQY/nJQee/9+ugHrf7pkATnotj
         EyDJ+WjC24LDo8Se/FTp9L7WdfZFKsCB7IV3Rln82aF0TTqmnwolM5j+gaxawn+cJQO3
         8oZjlpba7LBlv3Q2+Y8+SLMge9ILvKESvVtwRBYqYCLk4izDtWHKLMFYdaBRlgEH4Liu
         9yCw1pBeeK9AIHDWJ6WKpxkRDQmBfwmFIOOxXDPl5Kp+LOHHU7g+kyeofxm0VUJNOWOP
         rBqZO0HwAuE/oH8TiGvO+Re+U+VTe1se34ZqSJQffquKiEu9kedV7oeBNEKy2a3gAJI9
         tCuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=/jP81gv1AslQYHUXA2ClcvKMKyvf+YRZSggmAzN4Okw=;
        b=RZpu12+rDiOcjsD75SVUloVzkQX3h4gy6v37X2Tp3VkoXrniKlglGnTZsNJN9kL85/
         nmFfSg8Kb9gboirgvH2LGXqs40cByNZcNLo7sujmULmSIQSxYilHEshKGCYvzpq4uSrj
         o6fAsv59tWh8obScTQomvYSBWuNZhPTu9YOo1KIAM/yB52BGVmkTH2sq2ZOTYN6MTPS+
         Ww3xf6znYElkmqVGVV2pLWbYa7tgqHNZKS38qQw6jhLdd+7of7l4c5hsDW5aYZ0vbWix
         fjPgQ/WPR8z6eAsbGtU2fFmEFBkhx78CtXCyPh+J8bmYGKunSlrwa65lZK3KYouT9EzL
         yjrw==
X-Gm-Message-State: AOAM531F//E3vwvYo3D8LNKwx7Eh5N2hwYBINEiU9f9klR65vccyTn4j
	2gjL8ff+qfQw3JZHKnIlxg0=
X-Google-Smtp-Source: ABdhPJwkTbRmYMgJv3DIAg73LAjr6EeEywdlWgNBf5kaYNo87eNpkOMu+Il2abr/YVuRCWRS6gmSPg==
X-Received: by 2002:ac2:5442:: with SMTP id d2mr23601454lfn.275.1632245721737;
        Tue, 21 Sep 2021 10:35:21 -0700 (PDT)
Subject: Re: [PATCH V2 3/3] libxl/arm: Add handling of extended regions for
 DomU
From: Oleksandr <olekstysh@gmail.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 Julien Grall <julien@xen.org>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-4-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109161516350.21985@sstabellini-ThinkPad-T480s>
 <d5016065-fc36-d3bf-ff69-c102ede528dc@gmail.com>
Message-ID: <5e9ea5d5-0784-9697-94a2-e528cf705614@gmail.com>
Date: Tue, 21 Sep 2021 20:35:20 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <d5016065-fc36-d3bf-ff69-c102ede528dc@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


Hi Stefano

[snip]



>
>>
>>
>>> +static void finalise_ext_region(libxl__gc *gc, struct xc_dom_image 
>>> *dom)
>>> +{
>>> +    void *fdt = dom->devicetree_blob;
>>> +    uint32_t regs[(GUEST_ROOT_ADDRESS_CELLS + 
>>> GUEST_ROOT_SIZE_CELLS) * 2];
>>> +    be32 *cells = &regs[0];
>>> +    uint64_t region_size = 0, region_base, bank1end_align, 
>>> bank1end_max;
>>> +    uint32_t gpaddr_bits;
>>> +    libxl_physinfo info;
>>> +    int offset, rc;
>>> +
>>> +    offset = fdt_path_offset(fdt, "/hypervisor");
>>> +    assert(offset > 0);
>>> +
>>> +    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
>>> +        LOG(WARN, "The extended region is only supported for 64-bit 
>>> guest");
>>> +        goto out;
>>> +    }
>>> +
>>> +    rc = libxl_get_physinfo(CTX, &info);
>>> +    assert(!rc);
>>> +
>>> +    gpaddr_bits = info.gpaddr_bits;
>>> +    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
>>> +
>>> +    /*
>>> +     * Try to allocate single 1GB-aligned extended region from the 
>>> second RAM
>>> +     * bank (above 4GB) taking into the account the maximum 
>>> supported guest
>>> +     * address space size and the amount of memory assigned to the 
>>> guest.
>>> +     * The maximum size of the region is 128GB.
>>> +     */
>>> +    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + 
>>> GUEST_RAM1_SIZE);
>>> +    bank1end_align = GUEST_RAM1_BASE +
>>> +        ALIGN_UP_TO_GB((uint64_t)dom->rambank_size[1] << 
>>> XC_PAGE_SHIFT);
>>> +
>>> +    if (bank1end_max <= bank1end_align) {
>>> +        LOG(WARN, "The extended region cannot be allocated, not 
>>> enough space");
>>> +        goto out;
>>> +    }
>>> +
>>> +    if (bank1end_max - bank1end_align > EXT_REGION_SIZE) {
>>> +        region_base = bank1end_max - EXT_REGION_SIZE;
>>> +        region_size = EXT_REGION_SIZE;
>>> +    } else {
>>> +        region_base = bank1end_align;
>>> +        region_size = bank1end_max - bank1end_align;
>>> +    }
>>> +
>>> +out:
>>> +    /*
>>> +     * The first region for grant table space must be always present.
>>> +     * If we managed to allocate the extended region then insert it as
>>> +     * a second region.
>>> +     * TODO If we failed to allocate the region, we end up inserting
>>> +     * zero-sized region. This is because we don't know in advance 
>>> when
>>> +     * creating hypervisor node whether it would be possible to 
>>> allocate
>>> +     * a region, but we have to create a placeholder anyway. The 
>>> Linux driver
>>> +     * is able to deal with by checking the region size. We cannot 
>>> choose
>>> +     * a region when creating hypervisor node because the guest 
>>> memory layout
>>> +     * is not know at that moment (and dom->rambank_size[1] is empty).
>>> +     * We need to find a way not to expose invalid regions.
>>> +     */
>> This is not great -- it would be barely spec compliant.
>
> Absolutely agree.
>
>
>>
>> When make_hypervisor_node is called we know the max memory of the guest
>> as build_info.max_memkb should be populate, right?
>
> Right. Just a small change to pass build_info to 
> make_hypervisor_node() is needed.
>
>
>>
>> If so, we could at least detect whether we can have an extended region
>> (if not caculate the exact start address) from make_hypervisor_node.
>>
>> total_guest_memory = build_info.max_memkb * 1024;
>> rambank1_approx = total_guest_memory - GUEST_RAM0_SIZE;
>> extended_region_size = GUEST_RAM1_SIZE - rambank1_approx;
>>
>> if (extended_region_size >= MIN_EXT_REGION_SIZE)
>>      allocate_ext_region
> Good point! I will recheck that. I would prefer avoid spreading 
> extended region handling (introduce finalise_ext_region())
> and do everything from the make_hypervisor_node().
I experimented with that, so we can indeed calculate the address and 
size of extended region from make_hypervisor_node(),
there is no need for finalise_ext_region(). So, there won't be any 
zero-sized region anymore (due to unpopulated placeholder) if we fail to 
allocate extended region.
Thanks for the idea!

[snip]


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 17:51:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 17:51:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191843.341957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSjvb-0004Bc-G1; Tue, 21 Sep 2021 17:51:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191843.341957; Tue, 21 Sep 2021 17:51:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSjvb-0004BV-BU; Tue, 21 Sep 2021 17:51:35 +0000
Received: by outflank-mailman (input) for mailman id 191843;
 Tue, 21 Sep 2021 17:51:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QjZ+=OL=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSjva-0004BP-HY
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 17:51:34 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ec0e93ad-c7f1-452d-b2c7-eefa4c88dc38;
 Tue, 21 Sep 2021 17:51:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec0e93ad-c7f1-452d-b2c7-eefa4c88dc38
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632246692;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=flGLSCuWczAnpB2wymbe+X+ZNSzpuwgVrl9APoZYs+I=;
  b=WKJyV/kwB7+skz9hbDxosBF6w8cEPPOlflUzSgBLk357z8DOlbNHVKc8
   eSXWfxYeMGWqQQyXgPmXPHEKa8TZ3Z5/3H2eNRr8ptyv859accxvJYlVd
   DadxdGvd1aKfO3gnwYp1KBrSyhnP9sCKuhi94XKC2661fjp4tf48euAqK
   A=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: KqbLzqBhpEPFOZHmiscBjdJa5b3URDtanGTW7cWiXR4cNO4OiOHjKF8VL9isgpgOcrRAMJ2/y1
 bZU+izv0vSXyLy6pVVFy8rm7Zz1hQ6xT7jDYrZKmWsWj1tAPEhhp9Ga4jjJvz97yvQDg8B65aL
 YjpJz5vpUitof3D+vmb1a94uThnpbyEpoSIFjD/SO95w58UpdKEwXA1Dg4ktxqym2yHP8RkRoA
 SICFvhNhFXi+GMVnm3HJXnSHmmukDc18QY/1zU36hgHXSqp5G94m5zTTqxIhIJ2d8q5UIJuSPK
 kGCdw80wOl8nhmIad6k6ev9q
X-SBRS: 5.1
X-MesageID: 53241365
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:uzKuOaLD9qFHvHvlFE+RUpIlxSXFcZb7ZxGr2PjKsXjdYENS32cGz
 jEYXzjSa/iNNzOgctp3O9vg8h8AuMTRz4BkSgtlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wrZj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2Zhdk29
 OVE7aaTRB8gLofrxvsYbwRHRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gpr3JwXRqiDD
 yYfQQtBbznQcjBlAW1JOqolruSZplijTxQN/Tp5ooJoujOOnWSdyoPFK8HJc9aHQcFUmEewp
 W/c+Wn9RBYAO7S31j6t4n+qwOjVkkvTWogfCbm5/f5Cm0CIyyoYDxh+fVqko9Gph0imQdVdJ
 kcIvC00osAa8UGtQcngdxa5uziZphMaXZxdH/BSwBGAzO/Y7hiUAkAATyVdc5o2uckuXzso2
 1SV2dTzClRHsqCRSH+b3qeZq3W1Iyd9BWMMbCALTAwB4vH4vZo+yBnIS75e/LWd14OvX2uqm
 nbT8XZ41+57YdM3O7uT7UrOxA6cr8DzbQMt7xX9eTy66z9ke9vwD2C30mQ3/cqsPa7AEALb5
 Shax5DHhAwdJcrSz33WGY3hCJnsvqzcYWOG2TaDCrF8r2zFxpK1QWxHDNiSzm9SO8AYcHfCZ
 EbJsGu9D7cCYSP3Mcebj2+3YvnGLJQM9/y+DZg4jfIUO/CdkTNrGwk0PiatM5jFyhRErE3GE
 c7znTyQ4ZMmNEia5GDuG7d1PUAXKtAWmjqIGMGTI+WP+ruCfn+FIYo43K+1Rrlhtsus+VyNm
 /4Gbpfi40gPAYXWP3iMmaZOfA9iEJTOLc2vwyChXrXYeVQO9aBII6K5/I7NjKQ/zvwJyb+Xr
 i/iMqKaoXKm7UD6xcyxQikLQJvkXIplrGJ9OiopPF2y3GMkb5rp56AaH6bbt5F+nAC65fIrH
 fQDZeuaBfFDFmbO9zgHNMGvp41+bhW7wwmJOnP9MjQ4epdhQS3P+8PlIVSzpHVfUHLvuJtsu
 aCk2yPaXYEHG1ZoAvHJZa/91Fi2p3Ucxr5/BhOaPtlJdUzw24F2MCit3OQvKsQBJEyblDuX3
 gqbGzkCouzJr9Nn+dXFn/nc/YyoD/F/DgxRGGyCteS6MizT/2yCx45cUbnXIWCBBT2soKj7P
 Ldb1fDxNvEDjW1miYskHuY517866vvuu6ReklZuEkLUYgn5EbhnOHSHg5VC7/Uf2r9DtAKqc
 UuT4d0Ga66RMcboHVNNdgooauOPiaMdljXItKlnJUz74Gl8/aadUFUUNB6J0XQPILxwOYIj4
 OEgpM9JtFDv1kt0ao6L3nJO6mCBDn0cSKF25JgVDbjihhcv1lwfM4fXDTX74c3XZthBWqXwz
 uR4WEYWa2xg+3f/
IronPort-HdrOrdr: A9a23:DCl5uq6zzzaENZYvzgPXwV+BI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwXZVoMkmsiqKdhrNhQYtKPTOWxVdASbsN0WKM+UyZJ8STzJ876U
 4kSdkFNDSSNykLsS+Z2njALz9I+rDum8rJ9ISuvEuFDzsaD52Ihz0JezpzeXcGIjWua6BJdq
 Z0qvA33AZJLh8sH7qG7zQ+LqT+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+uemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lgdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNwN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wmJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABOnhkjizyxSKeGXLzAO9k/seDlEhiXV6UkWoJlB9Tpb+CRF9U1wsq7USPF/lq
 z52+pT5ehzpmJ/V9MLOA47e7rDNoX6e2OEDIujGyWUKEg5AQO4l3fW2sR+2Aj4Qu1E8HMN8K
 6xJm+w81RCI37TNQ==
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53241365"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OxKD7uWNY87koeu5wl3r4yoqGJydEK7Ib9TD3sf2BLiYN2myqDA3f8TuJv359lsme4GEzyFVun1nATsYtp6ZOLzmcMq388XUjziuPHAVPs4PBT7QwrrSKMWvA5lHT6SNtWt+g5nEzS/fLgX8afRYTcbEcaosOTMSHo8ZOrF4QE4M7g+h5PvzzKpS9amtsWdKtP8YIn5lgZUlY803sBV+UBfPP0fb+l17BE6FzCO0e6sGZNPgbYLxovIqg0DjC+fAVEXvgUEoCgW4REFx/yN3dgMIkceeAoNz8CmjrkZQ7ZW/Wm1mOqjBWzpRLyODP4BQAGlbRCctOLmv1UNQy0NkOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=K96rXa87K4HaHxHM6TIZjQQ6d9h87yfh/rTKv1iNmJE=;
 b=n5N7j7Qfc/OIW8pLYNWEubmobmEJd8TsJ0efBFav9MS8OeNP3DtkMs0TVX9WOE8lNwQWX7joIFgOsiQVe4+KKp1qCk/FBIjQr5NHT/8jm2KfLyy3VsrnMrhIEeSuxBb6Rp6BNkDBouRUwPHRRuqRS4EFSvVocLCpvfRlxLtHOah9ggCczOi+gDsa8XXLWlkWH/nIUjgJAwExc6k+sqKME4zyKrPmoBeXJEdsHn/iRW0MNljoZHE+dWYX+f+qm/H39IrGBUy3qCzwTdSngR7D0Xhsm5irAU3rLwVjDfI+7Qkvxd65HPKydUQdh4PrpDIfLS2DDVxIu6c69gkp8kB7Yw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=K96rXa87K4HaHxHM6TIZjQQ6d9h87yfh/rTKv1iNmJE=;
 b=NHr6CzNqlvgCKW8MYIJ4i3Ice49ylDoQgLwEieq47AeyWueoQNla8tYEmQvpgeWrLND0oldrb7bIQgrFwlqIxPzQZnbiS/H7izl6DTa0Uva3J/Txa/sFsAj1HCWNUn88DYPYVFitv4KfIeykPrJDI0wBQrfNbyl+RNb7d//CM9w=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Meng Xu <mengxu@cis.upenn.edu>, Xen-devel
	<xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
 <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Message-ID: <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
Date: Tue, 21 Sep 2021 18:51:20 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0424.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::28) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae12b132-7b06-4256-0faf-08d97d286f94
X-MS-TrafficTypeDiagnostic: SJ0PR03MB6271:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB6271D7CB77C92D11C5674473BAA19@SJ0PR03MB6271.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PqYag20tYqIPR2GgCTL49+uFsTZQ+f3vPPc8Seg/qpaX/51+yb6to4JeOb1gIw4Ota+A9gWRFo/Xn7/azApWBZ40IXJOSjArRUF2N9MB3kyjMw79T98TBRzsli2hH10jix4yDMG5JKMzNmZRgGjm+PM5rM6fl5E5CaCkL5arZF5L4U/A8CbenfY4FVzasSErp7QFImLw68xy3WIytZYXJf6vOpSpmxtmrcyYpEMf4+9vX3orbdT9M93p1FGBMDxRUvcU/5SU0lRCBQ9MtiyhXmXpHS//gqh6kmiHj83/MO1JBf58E6VcTsZ7ofoSL0+R4k9wrM2CmyJ6KggvXsxgsaPfEZZbAAS/HW4M02DQElkadEUPE2Kv6+MaQmZXuof9A1tKc9DC82DIy+N5EtdTaj9sQ+MJ5QzH07Nz04+zadWzapQv/EieM7u4pmtsiAzM5tGiN9zTK2MM0fAwIvvDCFNCX7pcLVyr3VFuVqZmOURsck9QQlGDUhRKWncWZgC89U6Uz8CKEaSR8JccjlUhAWOoVHcKJnqCFiVNkSDJLFqdsnAduOQe4rshchwhR9CNqjNGSp71LiytpPLsQ8Oatu5UiuXIoZIXNTgMXSjqDb6Nf/6+/iEix/fXE6ULRoobsJiM7xRg+GPSfhkChw6EjBGjcU+O6WlLV10QzTlZy7bCbZSv/D6C0cENqn9V/iA53dt1qdCQlO82MszUs8leS72hrH0VDVKUMSyNeCfS3As=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55236004)(508600001)(6486002)(54906003)(5660300002)(86362001)(6916009)(31686004)(6666004)(26005)(186003)(83380400001)(956004)(31696002)(2616005)(53546011)(66946007)(16576012)(66476007)(66556008)(2906002)(38100700002)(316002)(36756003)(8676002)(4326008)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eGFPeW9PaU56azljcWkzbzFkTmxRUCtIVnZYcVpaRm1IZkFKUkM0SExnZmlP?=
 =?utf-8?B?VU41SmNQU3dnNU1zVk8wTlluMlJYTnJGWXF6SkllcStISW5RYUE2aytHS0da?=
 =?utf-8?B?cFMwM2J2cGxsQ2JyTzNWWE0wems1d1NnRG81RWwrNHZ4QlRkWFhPS3pGeUds?=
 =?utf-8?B?UFBUK1BFNjZZbjhITENxTTM3SklObXFndTJXNE1LUmF0MXFQcWhGK25MVG9Q?=
 =?utf-8?B?VEw4cC9lTlVnNk8wRUVqRU02bUFiS2YyRkRQa3NEUGZ4SzdILzUzQnNUcEVh?=
 =?utf-8?B?QlhiOUhZUUpNdXFzUmtDVE9JYzFVOHBKZTNMRlpkYmJYcVJWdVhvYVpwTURU?=
 =?utf-8?B?Y1JvQml4RlBtMWYxaVJMRjJDRjlRYVE4d3A3QmpPaVB1b1V6OEVKTFUyakJK?=
 =?utf-8?B?WW1hZU5hMld1ZjdPNHVIRVBaWkRzc2xWT0ZlcHoxNVUyRXpqSlp5c0FzZkdF?=
 =?utf-8?B?M3MvTmNKaEZkRmYyNS9wRHI5K3ZucWVaUUlWTTdVZzhjZXJNUzRGUmxrdFdp?=
 =?utf-8?B?SUtuQkJJMTI2dTRna0RrVTVMWmNBb3JGdmllc1R5ZlJjTk5hZEF4Tk5rMStU?=
 =?utf-8?B?V2dMbzRNa0VZV3NnalpzamcybUpuRUFsaSs4UUNOb1p6Q3FxRDVTcml6WDNO?=
 =?utf-8?B?Qkc4ZEs1b3BHdzQ0UTVWejM0czNhUDFEZU4vRWxsMTM3cDkvdDB1UytNNkRP?=
 =?utf-8?B?QjBiYlNITWhVLythZTJUSGlsRFUyQVpLK0FFRWdyeGpvenVDRWF2TjFia0Z5?=
 =?utf-8?B?NStaeVZtc2tWbit1UXFQc1hnMi9MeGMvOGZEVzU3aGI4d3RDMkZlRE5SeGQ1?=
 =?utf-8?B?VmVFU1ZTeGtpR09Xb0p1LzBHL1JrbTR0Sm1rTWRleU1Bd1JDWUZPU3h5WnFZ?=
 =?utf-8?B?OUlBd2oyb3pPN21oNU1HMzJIL0VrMmJRMFYzdVgrQ2xKamNrNVk5U1lidXBn?=
 =?utf-8?B?aXVuSjhNQnEwNjI1RitJSUhTeGpST3Q0TmtNbnNmRzBVUitqaTgyOVVnUVI4?=
 =?utf-8?B?UHhhbkt0RU9RU0hQRlVTRHdsQ3pRTy9PRTFIQ1lFUE81S3hFQmY1WXpGUDZK?=
 =?utf-8?B?ajQ1cGtUUzdQdXNSZk5aQjNtZE15YjBhY29lQ0pjVWpTNzlpbTVMU1pnRUVO?=
 =?utf-8?B?SnhTQnN4cHNvbUxYWDJ2S0U1bU1pNkpiM3YzekR2aUgzcWN4dDNuSDVaMml2?=
 =?utf-8?B?blNyVmdJSjVCWGtMSzlJNnhnUVI3alhLcGl2enY3c0xiQU9aM1IrNytqL0JB?=
 =?utf-8?B?OVhtWm0xNFBBNm9hU1RyT0lsSzhiU1ZHNVNxTjNPT2kyc0RabGJWNlZIS21C?=
 =?utf-8?B?Tm1Mb2lvRERZVVQ3MEpUbEpYVmh4MkxjVEp6Zkw2R2xLVXVPaGt5QVlVamJ2?=
 =?utf-8?B?dUVqYlUzbytuU3MxU1lCL2J4QzcvNWFNM2xzUnBaK2xNWG1rZ1NDcGxNcWwx?=
 =?utf-8?B?REFCNlE0MlRST2N5UEpNMjBuNm5Mc0JKdGhXZURHNDl3T3RHdEtSV3o2NUVs?=
 =?utf-8?B?NlJvYUs3aDVkNW5YSFdKU01pQzFOWkpJSDNydEp1S2gyQUdGaXBzendUTEpK?=
 =?utf-8?B?NktwcmJpaDJGbU1uT1VobFM3c3NZTnVqNFJVL202OUd3OVE2SGczWGd0N0xI?=
 =?utf-8?B?K1F0U1FNeWJiekYxUEcrSy92T2Z1czJ0cWcrUXNDS1ArR3JxdjN2ekRBUDBV?=
 =?utf-8?B?VVdWZzcxRzllcVo1blgyUGF1MEZkcmpISE1jdEVWdE5LdHNHeG05dEQ4R011?=
 =?utf-8?Q?QHXsmtmMeVxegGT7Vy+9bEpMn6BpLquYByiHrR4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ae12b132-7b06-4256-0faf-08d97d286f94
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 17:51:27.5602
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8kP7ehyJ6bq8JxhcIJTwFZidU1RAgUVS3AScNSDDOQq4VJBxEAKgyejck6RWkt8r8wfqdP03Gj30ikxpFwIr4RW1wn+umQdF3vVqZnF+XHA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6271
X-OriginatorOrg: citrix.com

On 21/09/2021 07:53, Jan Beulich wrote:
> On 20.09.2021 19:25, Andrew Cooper wrote:
>> In the case that 'extra' isn't a multiple of uint32_t, the calculation r=
ounds
>> the number of bytes up, causing later logic to read unrelated bytes beyo=
nd the
>> end of the object.
>>
>> Also, asserting that the object is within TRACE_EXTRA_MAX, but truncatin=
g it
>> in release builds is rude.  Instead, reject any out-of-spec records, lea=
ving
>> enough of a message to identify the faulty caller.
>>
>> There is one buggy race record, TRC_RTDS_BUDGET_BURN.  As it must remain
> Nit: I guess s/race/trace/ ?

Oops yes.

>
>> __packed (as cur_budget is misaligned), change bool has_extratime to uin=
t32_t
>> to compensate.
>>
>> The new printk() can also be hit by HVMOP_xentrace, although no over-rea=
d is
>> possible.  This has no business being a hypercall in the first place, as=
 it
>> can't be used outside of custom local debugging, so extend the uint32_t
>> requirement to HVMOP_xentrace too.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
> Two remarks (plus further not directly related ones), nevertheless:
>
>> --- a/xen/arch/x86/hvm/hvm.c
>> +++ b/xen/arch/x86/hvm/hvm.c
>> @@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_=
PARAM(void) arg)
>>          if ( copy_from_guest(&tr, arg, 1 ) )
>>              return -EFAULT;
>> =20
>> -        if ( tr.extra_bytes > sizeof(tr.extra)
>> -             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
>> +        if ( tr.extra_bytes % sizeof(uint32_t) ||
>> +             tr.extra_bytes > sizeof(tr.extra) ||
>> +             tr.event >> TRC_SUBCLS_SHIFT )
>>              return -EINVAL;
> Despite this being a function that supposedly no-one is to really
> use, you're breaking the interface here when really there would be a
> way to be backwards compatible: Instead of failing, pad the data to
> a 32-bit boundary. As the interface struct is large enough, this
> would look to be as simple as a memset() plus aligning extra_bytes
> upwards. Otherwise the deliberate breaking of potential existing
> callers wants making explicit in the respective paragraph of the
> description.

It is discussed, along with a justification for why an ABI change is fine.

But I could do

tr.extra_bytes =3D ROUNDUP(tr.extra_bytes, sizeof(uint32_t));

if you'd really prefer.


> As an aside I think at the very least the case block wants enclosing
> in "#ifdef CONFIG_TRACEBUFFER", such that builds without the support
> would indicate so to callers (albeit that indication would then be
> accompanied by a bogus log message in debug builds).

That message really needs deleting.

As a better alternative,

if ( !IS_ENABLED(CONFIG_TRACEBUFFER) )
=C2=A0=C2=A0=C2=A0 return -EOPNOTSUPP;

The call to __trace_var() is safe in !CONFIG_TRACEBUFFER builds.

>
> Seeing the adjacent HVMOP_get_time I also wonder who the intended
> users of that one are.

There is a very large amount of junk in hvm_op(), and to a first
approximation, I would include HVMOP_get_time in that category.

But c/s b91391491c58ac40a935e10cf0703b87d8733c38 explains why it is
necessary.=C2=A0 This just goes to demonstrate how broken our time handling
is.=C2=A0 I'll add this to the pile of things needing fixing in ABI-v2.

>
>> --- a/xen/common/sched/rt.c
>> +++ b/xen/common/sched/rt.c
>> @@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct rt=
_unit *svc, s_time_t now)
>>      /* TRACE */
>>      {
>>          struct __packed {
>> -            unsigned unit:16, dom:16;
>> +            uint16_t unit, dom;
>>              uint64_t cur_budget;
>> -            int delta;
>> -            unsigned priority_level;
>> -            bool has_extratime;
>> -        } d;
>> -        d.dom =3D svc->unit->domain->domain_id;
>> -        d.unit =3D svc->unit->unit_id;
>> -        d.cur_budget =3D (uint64_t) svc->cur_budget;
>> -        d.delta =3D delta;
>> -        d.priority_level =3D svc->priority_level;
>> -        d.has_extratime =3D svc->flags & RTDS_extratime;
>> +            uint32_t delta;
> The original field was plain int, and aiui for a valid reason. I
> don't see why you couldn't use int32_t here.

delta can't be negative, because there is a check earlier in the function.

What is a problem is the 63=3D>32 bit truncation, and uint32_t here is
half as bad as int32_t.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 18:04:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 18:04:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191850.341968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSk83-0005rN-JF; Tue, 21 Sep 2021 18:04:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191850.341968; Tue, 21 Sep 2021 18:04:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSk83-0005rG-G4; Tue, 21 Sep 2021 18:04:27 +0000
Received: by outflank-mailman (input) for mailman id 191850;
 Tue, 21 Sep 2021 18:04:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QjZ+=OL=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSk82-0005rA-AJ
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 18:04:26 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5a9ecd0c-1b06-11ec-b916-12813bfff9fa;
 Tue, 21 Sep 2021 18:04:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a9ecd0c-1b06-11ec-b916-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632247465;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=HRt1v91/WQFroHXi/UO+MT+WBi8L4E38GL3fT0PiUvE=;
  b=CIfblkn35QomBLXt+izYJZqzGmkdA7aOPR01ClmhbNlD+ooO2CygTrFw
   m4a5sycOdbueo+WpgOGztVEgchdbGazcJVlEBZkA/fN5qaqSdPP9nDc8h
   jI11o1e49PAAJX9Mrk/4tSOsHqgNF6AzWN459ci9XZsIA7iAVuXQA+7jr
   g=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: L9HRYVWMfxZAt2+3Hs869N6hTOeP0OfGrh7LAPonUgQMH4TK4UMFaZHYGJhHe7Ov8bQ/X+7aQj
 mpr+zd9yRdJSFY0uGhK4InFdnhR9MjL2szZzdeKEUagLMfcuEFfP1Be3W944NddrXP3jYZ6V0f
 L4t9Jdgm4lWtKWsnpN9fb9VYh2N/jIvO3dWQYk0WH1GGU5GgaqJZraJpPL01KOfi7EYk5OS+2I
 PD75UvNpWHKemi8q56oZnB2fn15DRk76XUfcRkUT0fLMtvzqg3+wxQFhlrGgJRuAgcsHnAasc2
 v/c73GX4X8H0n4eu4RXZgGln
X-SBRS: 5.1
X-MesageID: 53668010
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:P4hVX672bECpINIowwIBuAxRtB3AchMFZxGqfqrLsTDasY5as4F+v
 mtMWW7SPqrbNzamf9l1Yd61/UgC65SDm99rTQZs/HtnHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2Ncw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z2
 vRu6bavDjYTJqzFuOkNdzN1PwVdMvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALBc/nJo4A/FpnyinUF60OSpHfWaTao9Rf2V/cg+gTRqiAN
 5JAMVKDajzvRwFpZV0TD6scwua3gHnjSmZJ9GOs8P9fD2/7k1UqjemF3MDuUt6ASNhRn02Yj
 nnb5Gm/CRYfXPSAzRKV/3TqgfXA9QvrVYRXGLCm+/pChFyI2ndVGBAQTUG8o/Sylgi5Qd03F
 qAP0nNw9+5orhXtF4SjGU3jyJKZgvICc8hxMLEQ90a88LLV6iaUFkwuYxJlS9Nz4afaWgcWO
 k+1c8LBXGI06uTFFy7Fr994vhvpZnNEdjZqiTssCFJfuoi9+tlbYgfnE447eJNZmOEZDt0ZL
 9qilyE4m7xbpsoCzazTEbvv0m/0+8ShouLY4GzqsoOZAuFRP9XNi2+AswGzARN8wGGxFALpg
 ZT8s5LChN3i9LnU/MB3fAnoIIxFGt7faGGM6bKQI3XR32v0oCPyFWyhyBp/OF1oIq45RNMdW
 2eK4Vk5zMYKZBOCNPYrC6rsW5VC5fWxTrzNC6GLBueil7AsLWdrCgk1PhXOt40s+WBx+ZwC1
 WCzK5f0USlCVvQ5k1JbhY41iNcW+8z3/kuKLbjTxBW7y7uOInmTTLYOKlyVae4lqqiDpW3oH
 xx3bqNmEj1TD7/zZDf564kWIQxYJHQ3H8mu+cdWavSCMkxtH2R4U63dxrYoeopEmaVJl7iXo
 iHhCxEAkFev12faLQiqa2x4bO+9V5hIsn9mbzcnOkyl2iZ/bN/3vrsfbZY+YZIu6PdnkaxvV
 /AAdsjZWqZPRz3L9i4zd574qIA+Jh2niRjXZ3iuYSQlfo4mTAvMo4e2cgzq/SgILyy2qcph/
 OHwilKFGcIOHl0wAtzXZfSjy0KKkUIcwO8iDVHVJtRzeVn39NQ4ISLGkfJqcdoHLg/Ox2XG2
 l/OUwsYv+TEv6Q87MLN2fKft46sHuZzQhhaEm3c4erkPCXW5DP+k4pJUeLOdjHBTmLkvq6lY
 LwNnf37NfQGmndMspZ9TOk3nf5vuYO3qu8I1BlgEVXKc0+vW+FpLXSx1MVSsrFAm+1CsgysV
 0PTotRXNN1l4i8+/IL98Ob9Utm+6A==
IronPort-HdrOrdr: A9a23:hi9E6qCrXayuaQTlHeg7sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LS90dq7MAzhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSuwEIdxeOkNK1kJ
 0QCZSWa+eAfWSS7/yKmTVQeuxIqLLskNHK9JTjJjVWPGZXgslbnnZE422gYy9LrWd9dP8E/d
 anl7F6T23KQwVoUi33PAhIY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX222oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iCnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDw4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWAoqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocXTbqjVQGdgoBT+q3pYpxqdS32BXTq+/blkgS+pUoJjXfxn6ck7zE9HJFUcegN2w
 2LCNUwqFniJvVmGp6VP91xNPdfPFa9CC4kAFjiU2gPK5t3T04li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53668010"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MP4IFygMV26Gz0PN/lDbwkmWBXKuiGAmwjNU3b379neUfzjtMzjWcHwaKiDWJLqVWorZs6FGcdLoTyRSZDPh5lo3Q1ohIUktUF864+rQRKHLJiWjLSmYqUuK5KE8JtaBoqmZYDwVSgOE+qAAW7vHr9AAiECkA1VNWStT9Cv03ccNj66HdbGMN7GA4I3kb8cRSDqC1zdkPa4u+8OuWQ3DcJ3ZQM1P87/3c+sM03FXkhU5kChl0UxyWdu/UYAMA7wQU6lho2n74fGnQLDmnxK9p1pR5XsRv7KzhDkTVfZtLUBOKyKN0YO7lR658YxWe8bMK3F0AAV5+wcrKjpBFvgqcQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pyvufv6Aa+O0JfFy11wVbUELWG4JkMJG89GlTmp0M3o=;
 b=c242c272fBYKEhoYOrbdQJo44Tc4Fa4Ab2M5wxYlM4eN5aAK+BqnhS3K70j0wiCQl8kEmTE2HPZCnuoy7LEotzTpGDWNOiax9k4akWt7PEMI8HAq+PFt8Ca5TnAYf1DVkC+N8TqsE1FRD1NvOJYr01wrVcsb3xuer0qlM+9u9JpW1C9i+KEd+sNkyZAE6IbHjGOKAnyUbte/SK0GXvbmEqbix+80q0xATGzhaFzWRuschFJ9ttge/PiFtrGW1IRlbX5CFe9dormjjr5YFkqYnkopR3bUhBvFohg3LjrmI8ufETGfUUFQYwibAxt/Gp59xMLwkPMzJ4gBWZWgFtFlKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pyvufv6Aa+O0JfFy11wVbUELWG4JkMJG89GlTmp0M3o=;
 b=vQQ995GRWVeReBu+a7SlEfYUOFyDJ3trA++YND2BUCU8vCKG+x5kWxkFuDgk6yRlBXwagDl3+aoDk5J/Nwkav/8Bm5BcCbX0n0bD0RexHeHiWGPryDcNNkTpS/JI+qR4BkYh2SbD7aB6xxQpa2Tm0KmTx3LvvhFndooAZQk0JDo=
Subject: Re: [PATCH v2 05/12] x86/hvm: Remove duplicate calls caused by
 tracing
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-6-andrew.cooper3@citrix.com>
 <45d5ba73-3099-9744-41f2-e3f53faed536@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <c651b2ae-b664-c838-62eb-7d3609cdce68@citrix.com>
Date: Tue, 21 Sep 2021 19:04:16 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <45d5ba73-3099-9744-41f2-e3f53faed536@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0374.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a3::26) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d9cc6523-7923-4509-a337-08d97d2a3d5f
X-MS-TrafficTypeDiagnostic: BYAPR03MB4165:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB41657FD950806152B8E374D4BAA19@BYAPR03MB4165.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: kB//dxsyheToMsTMrs6c9/LXNmgGMrnpecR7szNswEwFN4285aMeaHpDn7Lsno6kEivbgbu1wLA8/bBnk3V8xuZPR7OxjF8Z+Z0/bJYdNCPqJXdevStA08p6Vy7cHAmrjGg8JUZe6d8RC9YFruVLTaYC/e/i/q50/+FWdhg/tQej0q+ExZePMu8t0IRzKjmTlhEaxEmTxblm58jLhmUFJczCvjXEmOCXj1UmmP/tAvST/vR5jm+32TaLcIq5Q6nIEIWroRyJ3I8vDw594zWZQ9RtNZWiKr1tHORRlD5H/0AixmXQJNsVHkqAdiZ/9eYm3lJJ+JvjDYRayru3i49SYEeejgABi0EID1AopDHsJJ23TfKfD453yJgdhLLbfw8+jIfSvwkpQuIZy5Jt2PwlZAQ6ZnLaiddASeLkxHzfCE7xilEBJ0dh03K1Vx3NVBWD3YJOzSionczd8mMPQlWmK3fja76jwKRC5WPZmQyHnHL4Fgroxl1Tvmn6MmNTkaEZW9kT6kuq8FLAz/tfI5mL6qx3QMLjYS4Pq1XDh/7O/GgBKuOt1pQnw8vWRWGMSQTXPMrfhheG+hZW+UoixDYQ7RPrqTpZf0lz76a8U+2vy1GYOUQUfVT+tjec4wtgeGn5Ssybcf8Dk93XTcwqdWrm+nPea0UtK2azBE5cyEhWGPHY7N0jhXcPvjiJCjxNVFzwb8vLCOEsJOQZy6WV0mn9ODN8TMaYtnS/2BxitNtqng0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(86362001)(31696002)(316002)(38100700002)(53546011)(66476007)(508600001)(66556008)(66946007)(36756003)(6916009)(4326008)(55236004)(16576012)(956004)(2616005)(6486002)(5660300002)(54906003)(26005)(2906002)(6666004)(31686004)(8936002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NVVPUG5ZWE1IV0JhcmVIVW45MUpTYlB2NE1FZ1hBY3FjUjlUYVlHRnNKWVNI?=
 =?utf-8?B?V1NPbU5OM1MyS0t3RG5SMzhGMXVPcGtRWmE5THdGN1lpcFFEZU5IdGVKM0hx?=
 =?utf-8?B?QWdIVEs5T3F1akpiWFVNbUxsdHVRbGt3SnNHTTJlYkNaTkNYekoxenlNMUVP?=
 =?utf-8?B?TTdGb0N2UGJkNVlZWGsvVWpNckI1dk5RWkNEN0FsSmJkcFIrNTBmbGpBQTAy?=
 =?utf-8?B?S0t4bkVyZ25RR0NQZzVLQ01sZUNYNmlTTnRGbW11RXg5cnVkK1M2eEQxWCtC?=
 =?utf-8?B?Wk5VVFFUa0lJeHdGYThySXc2TDE2bkZicUpBcWJOclcwUmNUa3FyUWN6K3VR?=
 =?utf-8?B?NmxaTFRTb2Q5Sm4wM2owajY0TzkyZklCY2pXdUliVnBqSjNnSFhGNHNIckFi?=
 =?utf-8?B?T05iSTNKSWRVajdMbjYxSWVDaEhoUzI0SHQrenRxbHZLUFprdFYwQ2Q1UFlG?=
 =?utf-8?B?ZzU0NGkzYk1ZUC9lYTFtVHRSN1Z1MnRkd1JNK3pjQmRILzJHK3JZem5EVVor?=
 =?utf-8?B?cENCM0UyK2lxdGFtL1lRNHlORCtwRTNIUU5FakM1eEpWUEYvWkYySGtJQldi?=
 =?utf-8?B?SlgyZFEyR1lvQWxRREJnL2NXbWVjeTB2UFJpMHVQVnlVeHV2MWpnVk1rWnUr?=
 =?utf-8?B?eG9uZXY3SHVkek1FTHVFQ01mZjZmVnZzREs0eFNkMjBENE1BdXAwR3BkZDZG?=
 =?utf-8?B?YzFncElsQnBqTXBDWkxRQ2Z1MXlCWXp0MDcxTnRXUDJGdkU2R0RFZmxuOTBQ?=
 =?utf-8?B?TnQ1S2pLZjVJd3JoZW0zdUEvd29PbzY2MWpTdEdpSlYzUVUvYVdrOUtOeG91?=
 =?utf-8?B?QWNRQVlpM0tFL1doUzFZSVd0S0FQdld3UUU1dG9rT1ZyeFE0SjYwaU5zWEtN?=
 =?utf-8?B?UklaTVM4V0lsWEZ6a3Zsd1I3UmpzZldDU2w4VWhBclgyZU9pa2VJd0NrNExS?=
 =?utf-8?B?d245NmwxNU9qUWI2RU03d2ZHR20zcDdLbjdmWDdRdzZmYnFjNUtkamtudHJD?=
 =?utf-8?B?V21jQlk2VVJURk5CMXk3ZnBvdGZGejBHTldDRHNoT05tc0dVVk1iMDF3QVQ4?=
 =?utf-8?B?V0ZOODN4LzlPSUprZXFuR1RqMy9YYU9WWFlQOW9wait5SkFCaG9tZTREc01T?=
 =?utf-8?B?WW9vWW93SnpWZTBXaHMzREdIdmxwRzFaRjdBRlREdVVROXB1ZHBxcW10alFz?=
 =?utf-8?B?OVMxZlREdDhQUUZ4TTAyV2lScFQxODhJQS84ZmkwbEpyRjdnTHZmTzFpdVI3?=
 =?utf-8?B?Z1ZmYytPejdybFdmSktrSWRFMWNQKzVVcitiQlc2OVpubjkzclUyUWlXazUw?=
 =?utf-8?B?U1cvTVQ2aVVlRSs3bmoza2dsWnl5MEppLzRZak41a0EzbWZBbGNqY1QrVjRl?=
 =?utf-8?B?WVhjRzdoU0tVY0pwbkQrRmlVY3duMFE0cmNuMWFFQ21nRGdBUDAxeU5nR3lo?=
 =?utf-8?B?cVpsME1FMXZ6NUZhd2Q3K2NUS2pabjVDWFplR2tzMWRjMUdqVlplZkpLdFU2?=
 =?utf-8?B?aGtmcnZuejhPTnVScjJYTVhnSjRNaGU2ZEZHYzEyVDVNcEMrNFcvVEU2THpM?=
 =?utf-8?B?STBYdjdtWnRHMElQWHNhSWxkMUc1eVZGMEwwUk1vODdWYVNnY2QyMEZqRWd1?=
 =?utf-8?B?cXRObU5MK1lUZDNGOW4rdTMwamRHK09WeU5KV1ovMllGaWYrYXp1WXBjVXh2?=
 =?utf-8?B?TlZXbVdGMWNZTW01YUZ5SGZjREpFNkZRNmpmd0FKd0F1UnFkbGpqQi82ZDN4?=
 =?utf-8?Q?HuC1ImRuwFhcW9ZOE1EdWpf/erdwRVpG5LKRFwq?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d9cc6523-7923-4509-a337-08d97d2a3d5f
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 18:04:22.2991
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PqiSJW2C/nB2Z9TVtQzgEelWPxbKBFZwW3iyAKJg1bJUHbpuWnVeL7rLfGj8XmgS7S7AZCXgyqp3cRAINYmyHmCiIuPsXsAg1VYpZ9esGJw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4165
X-OriginatorOrg: citrix.com

On 21/09/2021 13:18, Jan Beulich wrote:
> On 20.09.2021 19:25, Andrew Cooper wrote:
>> 1) vpic_ack_pending_irq() calls vlapic_accept_pic_intr() twice, once in the
>>    TRACE_2D() instantiation and once "for real".  Make the call only once.
>>
>> 2) vlapic_accept_pic_intr() similarly calls __vlapic_accept_pic_intr() twice,
>>    although this is more complicated to disentangle.
>>
>>    v cannot be NULL because it has already been dereferenced in the function,
>>    causing the ternary expression to always call __vlapic_accept_pic_intr().
>>    However, the return expression of the function takes care to skip the call
>>    if this vCPU isn't the PIC target.  As __vlapic_accept_pic_intr() is far
>>    from trivial, make the TRACE_2D() semantics match the return semantics by
>>    only calling __vlapic_accept_pic_intr() when the vCPU is the PIC target.
>>
>> 3) hpet_set_timer() duplicates calls to hpet_tick_to_ns().  Pull the logic out
>>    which simplifies both the TRACE and create_periodic_time() calls.
>>
>> 4) lapic_rearm() makes multiple calls to vlapic_lvtt_period().  Pull it out
>>    into a local variable.
>>
>> vlapic_accept_pic_intr() is called on every VMEntry, so this is a reduction in
>> VMEntry complexity across the board.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

>
>> --- a/xen/arch/x86/hvm/vpic.c
>> +++ b/xen/arch/x86/hvm/vpic.c
>> @@ -512,14 +512,15 @@ void vpic_irq_negative_edge(struct domain *d, int irq)
>>  
>>  int vpic_ack_pending_irq(struct vcpu *v)
>>  {
>> -    int irq;
>> +    int irq, accept;
> Strictly speaking "accept" wants to be bool, and ...
>
>>      struct hvm_hw_vpic *vpic = &v->domain->arch.hvm.vpic[0];
>>  
>>      ASSERT(has_vpic(v->domain));
>>  
>> -    TRACE_2D(TRC_HVM_EMUL_PIC_PEND_IRQ_CALL, vlapic_accept_pic_intr(v),
>> -             vpic->int_output);
>> -    if ( !vlapic_accept_pic_intr(v) || !vpic->int_output )
>> +    accept = vlapic_accept_pic_intr(v);
> ... vlapic_accept_pic_intr() would eventually also want to be
> converted to return bool.

Yeah, but given the potential for backport here, I specifically avoided
unrelated changed.  We've got loads of functions wanting to change to bool.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 18:14:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 18:14:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191858.341979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSkHt-0007Ti-Ni; Tue, 21 Sep 2021 18:14:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191858.341979; Tue, 21 Sep 2021 18:14:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSkHt-0007Tb-JS; Tue, 21 Sep 2021 18:14:37 +0000
Received: by outflank-mailman (input) for mailman id 191858;
 Tue, 21 Sep 2021 18:14:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aS0w=OL=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mSkHr-0007TV-HQ
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 18:14:35 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cdf1946f-3479-44e8-9677-0401946ef21d;
 Tue, 21 Sep 2021 18:14:33 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id i25so1075210lfg.6
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 11:14:33 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id f6sm208184lfe.190.2021.09.21.11.14.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 11:14:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf1946f-3479-44e8-9677-0401946ef21d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=V3C059kkt+NTM+c5RApfFbNMPOs7ZaqZoV0G7Ax1Wms=;
        b=RLofSCca6lMVSQghYIYZNS8X3FG6zi4UafBl7mWtr4BKKZxYrdbHKsghxsy8jKESos
         hJ3vAtmKveQ417+jqJM7txQk2q75oT/sJPCzjRAsAQYoWTranuBr6AYISp24TTg/ElEF
         cOBQiyp98fcTf/e8FFWxb/7i7fewX4wyYBJdSQl5wCPbh0Q1nYj8WAuDDXnAvBnQKux+
         6DewwHxbNKiZCXSF1yVbikVS0FCzQQ4kBVcODem91y2uABWCmeVqU5QqARWG2G3w/3o4
         yLJ0iTAzZ2lFrpGHAutZGJVDTskzZ60XNCCls4aF+sO/E5yzOVwxyo9ZGsgQuSPkTQ23
         di+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=V3C059kkt+NTM+c5RApfFbNMPOs7ZaqZoV0G7Ax1Wms=;
        b=OfuBAiNIfhMOjgG2JadIf8BQJLLtxMd8Z+rJFRmZZrw0qQJe4IB1LwwagVD7HvJkmj
         z4E9vFpZI9PslpELV/UH/SA1/FB28Jw2a3n09e2wL1gIuHBZ8zk20mlkW13JWwsiSrqQ
         b4yX1MUNJpMiNWW1LaA+VvO+JUM2r78ZmvODnHXVlwxHIwdeIDuIGc9Cj/4+9eYtoI3M
         CzARo61UJFBOMrTHWkc/Nw/uJSbKd+XE0V45zFh9gkfgvQ+c2kvqFlww8mIsNuuSX8x/
         Rlr1GuFdj7BAZwWipUJnPUph2hDtTLMouixFFjxkGMKsJYJl4eou0S85D5dH32nUsb9E
         +Mfw==
X-Gm-Message-State: AOAM532vGBVYLNDilfGZz0d0a/JNVTDfnDel3WX24YGfjF35pO+AYfHJ
	YO3bKnKjr9Qc0TRWZgBeNg8=
X-Google-Smtp-Source: ABdhPJxY/PQMbnf+3iXp9VlU/MPrfEzGDIwxkDXrnjQy/JK8r8fFzuj0iivj/1/wmflqsC9DPNC0sg==
X-Received: by 2002:a2e:a604:: with SMTP id v4mr12014386ljp.258.1632248072605;
        Tue, 21 Sep 2021 11:14:32 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
 <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
 <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
 <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
Date: Tue, 21 Sep 2021 21:14:31 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 21.09.21 02:21, Stefano Stabellini wrote:

Hi Stefano

> On Sun, 19 Sep 2021, Oleksandr wrote:
>>> On 18/09/2021 03:37, Stefano Stabellini wrote:
>>>> On Fri, 17 Sep 2021, Stefano Stabellini wrote:
>>>>> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>>>>>> +
>>>>>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>>>>>> +
>>>>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>>>>> +    if ( !unalloc_mem )
>>>>>>>> +        return -ENOMEM;
>>>>>>>> +
>>>>>>>> +    /* Start with all available RAM */
>>>>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>>>>> +    {
>>>>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>>>>> +        end = bootinfo.mem.bank[i].start +
>>>>>>>> bootinfo.mem.bank[i].size - 1;
>>>>>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>>>>>> +        if ( res )
>>>>>>>> +        {
>>>>>>>> +            printk(XENLOG_ERR "Failed to add:
>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>> +                   start, end);
>>>>>>>> +            goto out;
>>>>>>>> +        }
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>>>>> +    {
>>>>>>>> +        start = assign_mem->bank[i].start;
>>>>>>>> +        end = assign_mem->bank[i].start +
>>>>>>>> assign_mem->bank[i].size - 1;
>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>> +        if ( res )
>>>>>>>> +        {
>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>> +                   start, end);
>>>>>>>> +            goto out;
>>>>>>>> +        }
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    /* Remove reserved-memory regions */
>>>>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>>>>>> +    {
>>>>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>> +        if ( res )
>>>>>>>> +        {
>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>> +                   start, end);
>>>>>>>> +            goto out;
>>>>>>>> +        }
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    /* Remove grant table region */
>>>>>>>> +    start = kinfo->gnttab_start;
>>>>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>> +    if ( res )
>>>>>>>> +    {
>>>>>>>> +        printk(XENLOG_ERR "Failed to remove:
>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>> +               start, end);
>>>>>>>> +        goto out;
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>>>>> +                                 add_ext_regions, ext_regions);
>>>>>>>> +    if ( res )
>>>>>>>> +        ext_regions->nr_banks = 0;
>>>>>>>> +    else if ( !ext_regions->nr_banks )
>>>>>>>> +        res = -ENOENT;
>>>>>>>> +
>>>>>>>> +out:
>>>>>>>> +    rangeset_destroy(unalloc_mem);
>>>>>>>> +
>>>>>>>> +    return res;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static int __init find_memory_holes(const struct kernel_info
>>>>>>>> *kinfo,
>>>>>>>> +                                    struct meminfo *ext_regions)
>>>>>>>> +{
>>>>>>>> +    struct dt_device_node *np;
>>>>>>>> +    struct rangeset *mem_holes;
>>>>>>>> +    paddr_t start, end;
>>>>>>>> +    unsigned int i;
>>>>>>>> +    int res;
>>>>>>>> +
>>>>>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>>>>>> +
>>>>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>>>>> +    if ( !mem_holes )
>>>>>>>> +        return -ENOMEM;
>>>>>>>> +
>>>>>>>> +    /* Start with maximum possible addressable physical memory
>>>>>>>> range */
>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>>>>> +    if ( res )
>>>>>>>> +    {
>>>>>>>> +        printk(XENLOG_ERR "Failed to add:
>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>> +               start, end);
>>>>>>>> +        goto out;
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM,
>>>>>>>> etc) */
>>>>>>> Well... The loop below is not going to handle all the regions
>>>>>>> described in
>>>>>>> the property "reg". Instead, it will cover a subset of "reg" where
>>>>>>> the
>>>>>>> memory is addressable.
>>>>>> As I understand, we are only interested in subset of "reg" where the
>>>>>> memory is
>>>>>> addressable.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> You will also need to cover "ranges" that will describe the BARs for
>>>>>>> the PCI
>>>>>>> devices.
>>>>>> Good point.
>>>>> Yes, very good point!
>>>>>
>>>>>
>>>>>> Could you please clarify how to recognize whether it is a PCI
>>>>>> device as long as PCI support is not merged? Or just to find any
>>>>>> device nodes
>>>>>> with non-empty "ranges" property
>>>>>> and retrieve addresses?
>>>>> Normally any bus can have a ranges property with the aperture and
>>>>> possible address translations, including /amba (compatible =
>>>>> "simple-bus"). However, in these cases dt_device_get_address already
>>>>> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>>>>>
>>>>> The PCI bus is special for 2 reasons:
>>>>> - the ranges property has a different format
>>>>> - the bus is hot-pluggable
>>>>>
>>>>> So I think the only one that we need to treat specially is PCI.
>>>>>
>>>>> As far as I am aware PCI is the only bus (or maybe just the only bus
>>>>> that we support?) where ranges means the aperture.
>>>> Now that I think about this, there is another "hotpluggable" scenario we
>>>> need to think about:
>>>>
>>>> [1] https://marc.info/?l=xen-devel&m=163056546214978
>>>>
>>>> Xilinx devices have FPGA regions with apertures currently not described
>>>> in device tree, where things can programmed in PL at runtime making new
>>>> devices appear with new MMIO regions out of thin air.
>>>>
>>>> Now let me start by saying that yes, the entire programmable region
>>>> aperture could probably be described in device tree, however, in
>>>> reality it is not currently done in any of the device trees we use
>>>> (including the upstream device trees in linux.git).
>>> This is rather annoying, but not unheard. There are a couple of platforms
>>> where the MMIOs are not fully described in the DT.
>>>
>>> In fact, we have a callback 'specific_mappings' which create additional
>>> mappings (e.g. on the omap5) for dom0.
>>>
>>>> So, we have a problem :-(
>>>>
>>>>
>>>> I can work toward getting the right info on device tree, but in reality
>>>> that is going to take time and for now the device tree doesn't have the
>>>> FPGA aperture in it. So if we accept this series as is, it is going to
>>>> stop features like [1] from working. >
>>>> If we cannot come up with any better plans, I think it would be better
>>>> to drop find_memory_holes, only rely on find_unallocated_memory even
>>>> when the IOMMU is on. One idea is that we could add on top of the
>>>> regions found by find_unallocated_memory any MMIO regions marked as
>>>> xen,passthrough: they are safe because they are not going to dom0 anyway.
>>> (Oleksandr, it looks like some rationale about the different approach is
>>> missing in the commit message. Can you add it?)
>> Yes sure, but let me please clarify what is different approach in this
>> context. Is it to *also* take into the account MMIO regions of the devices for
>> passthrough for case when IOMMU is off (in addition to unallocated memory)? If
>> yes, I wonder whether we will gain much with that according to that device's
>> MMIO regions are usually not big enough and we stick to allocate extended
>> regions with bigger size (> 64MB).
> That's fair enough. There are a couple of counter examples where the
> MMIO regions for the device to assign are quite large, for instance a
> GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
> but maybe they are not that common. I am not sure if it is worth
> scanning the tree for xen,passthrough regions every time at boot for
> this.

ok, I will add a few sentences to commit message about this different 
approach for now. At least this could be implemented later on if there 
is a need.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 18:35:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 18:35:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191864.341990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSkbd-0001ZO-CC; Tue, 21 Sep 2021 18:35:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191864.341990; Tue, 21 Sep 2021 18:35:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSkbd-0001ZH-8f; Tue, 21 Sep 2021 18:35:01 +0000
Received: by outflank-mailman (input) for mailman id 191864;
 Tue, 21 Sep 2021 18:34:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QjZ+=OL=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSkbb-0001ZB-Ku
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 18:34:59 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9f51ff89-1b0a-11ec-b91d-12813bfff9fa;
 Tue, 21 Sep 2021 18:34:58 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f51ff89-1b0a-11ec-b91d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632249298;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=iItkPCfMhnWf4rVQ+PVkFGSFpt4Aw0qHJg4Hn657cy8=;
  b=KmDkQqdtuOtiZ9k4wn1gMRjmlm7h/M1Smnm/y+dzKmKPgRW0qhUiioFX
   xzJxXtbi3xMU7Zwzwatz4pnPgA4Ep+4kJ9TCZvCMlA7polxleuwG2dWBr
   HuaFvmZk+e5t0E1W+XmWkC+6N+n9CLueiGP7IDGupuq3IuF4aGjcvLruO
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: yB2/bV/RSp5kNm/+gJ6v/NZEfHk/xHIOAtXikvThKhlqQqjGxAyvUYpBgH0g7qApK9wxVT/XpY
 PQmo9xdFfyOCXL6PpRKs/SaTqiJ19AvEuZ/f6j43pp56LnatbCsZOxkWAjvVW6EH1VmEF+fJVf
 bMf6XQ51uIWxWXMVAF+1IVD/keOOyqPg6IlRInDngW2iTr3iU6L6SgqTVaWM/bChvxQrmJzYuW
 OuKKR3TYlM2VHl/M48QMf5bDCxXQFtOiPs6R6cKKwnz+T748OABcrDhdCtB3+00iSuh1J1fqW1
 FXSSU4qdo6/AluBMCulhS4Zz
X-SBRS: 5.1
X-MesageID: 53670609
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:8v7IwK5NPyLw+UbQ5M6nLQxRtB3AchMFZxGqfqrLsTDasY5as4F+v
 jYdCGiAOPuDYGOmeo1xb9++phhVu5ODz4djHFdsriBgHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2Ncw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z5
 I5hvLOyEB8QAvecovg8XQRxLSdDBPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALBc/nJo4A/FpnyinUF60OSpHfWaTao9Rf2V/cg+gTRqiBP
 5RBOFKDajzgeExyEwsoFKs8xr3vhCHjVRYBlHOa8P9fD2/7k1UqjemF3MDuUt6ASNhRn02Yj
 nnb5Gm/CRYfXPSAzRKV/3TqgfXA9QvrVYRXGLCm+/pChFyI2ndVGBAQTUG8o/Sylgi5Qd03F
 qAP0nNw9+5orhXtF4SjGU3jyJKZgvICc/YLP+QV6liV85aOszyFJ2YZZwBHVeVz4afaWgcWO
 k+1c8LBXGI06uTFFy7Fr994vhvpZnNEdjZqiTssCFJfuoi9+tlbYgfnE447eJNZmOEZDt0ZL
 9qilyE4m7xbpsoCzazTEbvv0m/0+8ShouLY4GzqsoOZAuFRP9XNi2+AswGzARN8wGGxFALpg
 ZT8s5LChN3i9LnU/MB3fAnoIIxFGt7faGGM6bKQI3XR32v0oCPyFWyhyBp/OF1oIq45RNMdW
 2eK4Vk5zMYKZBOCNPYrC6rsW5VC5fWxTrzNC6GLBueil7AsLWdrCgk1PhXOt40s+WBx+ZwC1
 WCzK5f0USlCVvQ5k1JbhY41iNcW+8z3/kuKLbjTxBW7y7uOInmTTLYOKlyVae4lqqiDpW3oH
 xx3bKNmEj1TD7/zZDf564kWIQxYJHQ3H8mu+cdWavSCMkxtH2R4U63dxrYoeopEmaVJl7iXo
 iHhCxEAkFev12faLQiqa2x4bO+9V5hIsn9mbzcnOkyl2iZ/bN/3vrsfbZY+YZIu6PdnkaxvV
 /AAdsjZWqZPRz3L9i4zd574qIA+Jh2niRjXZ3iuYSQlfo4mTAvMo4e2cgzq/SgILyy2qcph/
 OHwilKFGcIOHl0wAtzXZfSjy0KKkUIcwO8iDVHVJtRzeVn39NQ4ISLGkfJqcdoHLg/Ox2XG2
 l/OUwsYv+TEv6Q87MLN2fKft46sHuZzQhhaEm3c4erkPCXW5DP+k4pJUeLOdjHBTmLkvq6lY
 LwNnf37NfQGmndMspZ9TOk3nf5vuYO3qu8I1BlgEVXKc0+vW+FpLXSx1MVSsrFAm+1CsgysV
 0PTotRXNN1l4i8+/IL98Ob9Utm+6A==
IronPort-HdrOrdr: A9a23:yEq0waw//2uU24l13qB/KrPwIr1zdoMgy1knxilNoH1uHvBw8v
 rEoB1173DJYVoqNk3I++rhBEDwexLhHPdOiOF6UItKNzOW21dAQrsSiLfK8nnNHDD/6/4Y9Y
 oISdkbNDQoNykZsfrH
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53670609"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jpx/hpUcdiBkMxcB5GulAYLB64sxjBgdvjzO3Uu5FiO30DzYnH66hJHxm1lx72Nt0DLRWl/zQdToK2urNDstCxXdDuxYR0IfgsFuK+Vd797hmDTNqHzE3atY7HL9+2yAlvgapcy7dR0IXI1PFD06waxryhRAqnVEWt3cusYRSaFXVxdZl4we7uS9s7remhgGN77Qirxxpuyi+PhdYp/mZjaXGzRorFQHkhK/SEYS1eqfHHQH1//CADBsH9XzzG0qq1FKGa4OWYTtu4Mmrosf/MDVMIm5K7q2YdXEMSDnLlGKeVqqfACV55QNsGRdxYEz+mH2UXAdE64AMX8jd/sO2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ddGu/6OK1sBG/G8q1B+cUaTB/vRhwOjCZyqPZ/DTBSM=;
 b=ckLfnLtWl8QkZqLmp1hSMWCsZIj5pKtkVL/UE8JwQesLiZsaAu0WqVbCsu4R+Rcdp8oY32TZ/NOMufAl+FNnVkvWJFjXdzX4Vlh8L7HraOvVLNOmxcg/Qe2oiI0Pw642tB5Vc9v6SzpCPOpcDEEBOWp9NvNY21OxrccXRuuihY5aWA3uMfZJVMxDHClNj/AsC7q8+WwU9RiR35s9EbW2VaMKSyetD5KW5DhArm0Pd8XC/l/1VG2j0X7dGDLQpp7UXnn9UQAAEBDxHDuVvkV3hU5ca8FHjkO3GJhzl1+XROzpfFnjmZmXLl7YGcrjpGQejW74+1mXnapruhrha4RvZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ddGu/6OK1sBG/G8q1B+cUaTB/vRhwOjCZyqPZ/DTBSM=;
 b=CL+DIQUKH2TkVERWo76sxMr2KRxAwOHPLxOEkZby80wQebqpGo+ADHY01jnb5qPxRDyAxMgyAHwrw8QvCQt/hLjMfHWmmK4OQPkAMcmaZLD4HjzddD/NPRshP7HFi9CgAy1FNwt5OzMId+0iJdgndWc1/Xcl48cFprraK1FNngM=
Subject: Re: [PATCH v2 12/12] x86/trace: Clean up trace handling
To: Jan Beulich <jbeulich@suse.com>
CC: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-13-andrew.cooper3@citrix.com>
 <f953cc44-3da4-6860-6e55-6a5751c4b3c3@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <29cd2906-64cc-3eeb-a487-c71e7d1d0df0@citrix.com>
Date: Tue, 21 Sep 2021 19:34:48 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f953cc44-3da4-6860-6e55-6a5751c4b3c3@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0092.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:191::7) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 12c7ccb2-b29c-4f4b-a946-08d97d2e8208
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5776:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB57769205354811DF35422F57BAA19@SJ0PR03MB5776.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: sHMn2aUsEeNtDoTn9INlwZL1Y76RUZP3BX8227MA2e3wwUvrn+TC6I4Kzt42CEk9eFSXEnfDYh2qJs7ZzdLhCPYekreMX6auNFbaiQ4P2iVBqW6exUwim4NTahSWAnlH6B+WVUqh/jocjKKvj/VFK2GRQIQTQzRpAbVuLKaR6ys/qKtrHi4Rfi5wWhqax7XXFg1EGohniW3fUbPgoxklqNwS8XmSpeCmbHfTwdVIbjLB946fL3nW7/htDgKTBnHXt68PLzJ5At+eId1DxHrRt7O8+uWs8jDIAP3d6S3StRXy4WcuzQhxNGLS1DQ4b+9oahEE4grhvpX4ld1bfctqo7YTHW6ozf64pqTEpd+90CZG8bRaHzeVeDnKiU90I4pr50MeaPEiijYnJFGkPCnU7VeoLXVztk4E8d/0lWjBQqv4ZW/fosM1zmeu6p3T5p9fqxlSg8/wlYLHSgoJw1o3I7FAxRsCQ2fquN4YxzaRjFyGRDYtH94yt1qECWhOWeXo4H5eTDNyYFgGybG+Knra7qWppQU/wkyEahn14qUANrsCU7pGL3rJYfDqAvNIlqR9jWTwj1m2eG2AEUMhXva+eoF6MRvvGgDr703Qgu2haauNqKiA0s1KxHou23dS8NMesFV+HmD/cURJPZCGrZnz5Zf/HkM7r3um/NQ7mIeD+Zb3w+DF/iyFI90AC1ZzXl1k387peDpBwvdrVt4tsqovXkj80CBKWVfeAtsVbNYFJBg=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(508600001)(186003)(38100700002)(6666004)(53546011)(2616005)(66946007)(31686004)(16576012)(8936002)(31696002)(956004)(54906003)(316002)(4326008)(26005)(66476007)(66556008)(6916009)(5660300002)(86362001)(8676002)(55236004)(2906002)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?THA1SUdIb3l2UnBudWlweUpoWjROcDI5Y0FoMGhsQ1djWEEwUk9ZcUdRenhG?=
 =?utf-8?B?UTRNUlFiOHFKbmV5ZmRPaDlBdjFHL203dUJpOVdzdzNMWHRjTThRNVYrZU9O?=
 =?utf-8?B?YUZQSk1DbUZtZllyemFrSW0vUWR3R2d5VTVhSkZya3BVZERxaWFQMEFOMTFq?=
 =?utf-8?B?eUpHYVl1SWF2ZzJ2RkRsSUQ1WVJ6dGlqY0FmMDNaZzBSYnZuemhBdU16YW8r?=
 =?utf-8?B?NS9CVGxHUGVSUVVscWJUeGo2QUFXNWhaR29xbEgwcUNuYzRuYndpK2N6NURF?=
 =?utf-8?B?ZmlDbytCcXhFZGhjS1JPcHN6ZlNyVlNEWHB5TFdpZnhyN0pvMTBNTEJ1T1Fo?=
 =?utf-8?B?RG1uN2ZCeUpoTmhqeU0zdjkxcjNCVHhieG80cVBOQ25Ldm5TbDFhQlRBdkVT?=
 =?utf-8?B?Si81MkpJajd0S2ZMczVqRDRtZU1WMXVMckhFb2k5Mk1MekY4Nzlqc0Vvb0pP?=
 =?utf-8?B?dnVjalJkQVpQVUFHTEVFOVRWTVMwaXdXUHVhdDBGdlpjaHJ6dUhOUHVTL01l?=
 =?utf-8?B?MjR3YWpKVitEb2lmeUt5ZXI2Y2VLaEtVVzcvTkVMTjZoSXBLUmdZU2svb3hC?=
 =?utf-8?B?eXR4VVZYRDNoaGt5c1ZPbmlDQm5sODF0a1Qzd1B3Z29ac2tlK3AzdnJmSHd1?=
 =?utf-8?B?NEhFM2QwR2hKMHZtQ2k1eXN0WlQ2Ty9lSENtUnBHV0ZzZjdUV1lrN3BkdWth?=
 =?utf-8?B?NTQ3M0JCODU2U3ZJU01KakxRV2tsTkNNcUhUWTEyMFRUZlBOZUp1YXhhQzNh?=
 =?utf-8?B?MXlJMG1kMElyWUk2eGRPelVyNEZDbTZIVUZmcnJ2WUhkL09saE9GUjlqSHN2?=
 =?utf-8?B?UDREQ2FlOHV0U0VNOWRoUm8vYkZGQ1h6TmlJenMvNGQ1aDFMWWR5VDk1cmFy?=
 =?utf-8?B?UDRZWGtsRitqY251eXYzVm9RaDR0SU9EN2FBT21nV1VTclJwTDB4TU04N3Vj?=
 =?utf-8?B?SW5kYTdYU3NzRFZSb2dweUtDdFgycjN2bHVoL0NjK25oSGJDY3c4aDBxZmhZ?=
 =?utf-8?B?cjJFTG90bTJRaWlCdjIxblQrRWhZQmdRSHNJR2tHaGNhTHNjaSs4UTdGOVZX?=
 =?utf-8?B?RkJNaXZyeCtSMFJ1V2cxZVZUN1N3R0ozaXdydGlsamZodHhabHhIZFdNSTly?=
 =?utf-8?B?czBsaU1meFplODNWOGdkaWhkdDVuaytVNDVkUm0zRUx5b3ZNZUluVnNCOFpk?=
 =?utf-8?B?VDdRK2FUdHRTMk9QZ2ZrbGpLdmdxalQvZzlwbmJCak1TeHRnRWJ5aFFvWHJI?=
 =?utf-8?B?aGdvYVFKbyt1bWw3RmhwWndWUFFGMmdZd3preGphalhtOEp3UStCK0tUYmhm?=
 =?utf-8?B?OXpid3d4OXdlQ3RqZGQ1cm1tWlk0U2FWOG1acnpjTnZQSTJBMW42N3NvSVB1?=
 =?utf-8?B?SURLR0RQZVkxVWYyT2JuSWdNTGZWKzZ5SExLT0lEWlVwdDFOQnc4anJoS3N1?=
 =?utf-8?B?dzJKUkxJc1ArSU5NT05KMGpJZlB4dVRSN0tEc2ZlelZkWHUwbjR2NlNjTi96?=
 =?utf-8?B?dG9RYjRrZk11UTROWVlqM2FhUFhRU1A4eEdKWERSVVdydjQ3VktFbmF4eGI0?=
 =?utf-8?B?aTVaRDdTQTBZczZrNU9MM1RLMWhMc1QwN25XQVRYcXU4K1lNSmN2Qi9MN2lW?=
 =?utf-8?B?Q2VpUDFCOG5kVkZuWUFBbElqWFNSVnplL1lnUXBqYU5nTlNLcytRQ2RlSExv?=
 =?utf-8?B?cFBVNXlSOTY4RDVJZUpGVy83YnlQYU9PM3RtOGhvMWNESDNZSms3enpTZW5F?=
 =?utf-8?Q?ERTpXhcuxiSBugekgMXpD/fqMy0RMcq32Q1JvWU?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 12c7ccb2-b29c-4f4b-a946-08d97d2e8208
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 18:34:55.4838
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kRLylGWPTbm0v+e83K2WxQ1XAyAjdJfKXArJLRJ4EMeyUpFDnKzENsvR5Q7LqUQD2r1/g8DR0MuXPGo+HdpWEMwbqivmtXUqgad8E0IZur8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5776
X-OriginatorOrg: citrix.com

On 21/09/2021 17:08, Jan Beulich wrote:
> On 20.09.2021 19:25, Andrew Cooper wrote:
>> Use more appropriate types.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with a suggestion:
>
>> @@ -48,30 +45,28 @@ void __trace_pv_page_fault(unsigned long addr, unsigned error_code)
>>  
>>      if ( is_pv_32bit_vcpu(current) )
>>      {
>> -        struct __packed {
>> -            u32 eip, addr, error_code;
>> -        } d;
>> -
>> -        d.eip = eip;
>> -        d.addr = addr;
>> -        d.error_code = error_code;
>> +        struct {
>> +            uint32_t eip, addr, error_code;
>> +        } d = {
>> +            .eip = eip,
>> +            .addr = addr,
>> +            .error_code = error_code,
>> +        };
>>  
>>          __trace_var(TRC_PV_PAGE_FAULT, 1, sizeof(d), &d);
>>      }
>>      else
>>      {
>>          struct __packed {
>> -            unsigned long eip, addr;
>> -            u32 error_code;
>> -        } d;
>> -        unsigned event;
>> -
>> -        d.eip = eip;
>> -        d.addr = addr;
>> -        d.error_code = error_code;
>> -        event = TRC_PV_PAGE_FAULT;
>> -        event |= TRC_64_FLAG;
>> -        __trace_var(event, 1, sizeof(d), &d);
>> +            uint64_t eip, addr;
> Like you've done in __trace_pv_trap() and __trace_ptwr_emulation(),
> perhaps name the field "rip" here as well?

Fixed.  (Honestly - these all start to blur together given how large the
series is.)

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 19:43:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 19:43:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191875.342016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSlfb-0000b9-DM; Tue, 21 Sep 2021 19:43:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191875.342016; Tue, 21 Sep 2021 19:43:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSlfb-0000b2-Aa; Tue, 21 Sep 2021 19:43:11 +0000
Received: by outflank-mailman (input) for mailman id 191875;
 Tue, 21 Sep 2021 19:43:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aS0w=OL=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mSlfa-0000ad-4U
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 19:43:10 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6c28d9ec-10e7-4e97-a77b-7bb88d92e38c;
 Tue, 21 Sep 2021 19:43:08 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id y28so2287016lfb.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 12:43:08 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id y32sm1589020lfa.171.2021.09.21.12.43.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 12:43:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c28d9ec-10e7-4e97-a77b-7bb88d92e38c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=Gn/9MMD+lIm5s/bvzqWXUO99oDTImbcpJ6nNxz3pWUY=;
        b=IYvy+pKq+jVPWuXihjEuNNhU5SUrrLxqfECkYSBvQC43m4XNm+3IzLGBopysvAODen
         iGhnpOVKc4AaW1lias53UuPcm0HqB4eliYJrL9y0bw8lhAvttkUBsWB33OJjUhphteRy
         e+j4AluLf8tnlGU+Ikp5LOuCRWMybzYEUBrT4nu6UGWLU10iBVh86Ox+keZW//F5yDQo
         j0qYa/TPnGljHITZLfDy1VBWIpn/wlrs2u21FX+PUkwkuM+BZgNaDyxBeO9qU+18Q51e
         OEG9x+VG4iZKcrD2Wc9YN6iboGXoKrfWjKQXtP2MrFgU2pGNUNgnsa3dQdSozlMYffBf
         rbDg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=Gn/9MMD+lIm5s/bvzqWXUO99oDTImbcpJ6nNxz3pWUY=;
        b=66uHOh8Sl5QKreX2+Bud0/DRhBvJuQ0lybGiHIe+s9P2OKqvMzKMZGW2A74ZRZnR0b
         lvRiU8/36CHfiSgWMcisw8Wq/Fqcp6koMJx/0+2d4XgtktpnuMH5zs+mSCFEJ9W5zksY
         aeLRTlG7/nz4BOY9KDtP3Q7yx0u2w7QBC3M7ASTyapVIJ8bqFdLNncasI7PdTVFgn+Z6
         2zz+8Hp46fApdv1rR2Lkgnrpmt4YN3j+fBrn3u5fcMovJfOc/f1NPGXjKc7L/sbkY5f4
         qVnbHeA8GItRH2rOAbf6a73zQjcy4kHoeo2WfUMDy7vzxRt/BGHz6ryYCENYcLooCMJC
         XsSw==
X-Gm-Message-State: AOAM531IBSjPZ4iZJwCvQ0v8hlfcdh4mzFrLQiRVY4tgbxExwr1taj7+
	OWkdg263iA2I3iIGUeFBo28=
X-Google-Smtp-Source: ABdhPJxo8Dzr7HhmpYwOAWh4FSnBDNUC2xpiyznNVZBRDzUoTO4oAK3+K0qwB9R/pkwEJYca1w0kUg==
X-Received: by 2002:a2e:2a06:: with SMTP id q6mr28912275ljq.424.1632253387596;
        Tue, 21 Sep 2021 12:43:07 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <a993466f-1b7d-ceb3-aa3b-16f5f145b2fb@gmail.com>
Date: Tue, 21 Sep 2021 22:43:06 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 18.09.21 00:56, Stefano Stabellini wrote:

Hi Stefano

> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>> +
>>>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>>>> +
>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>> +    if ( !unalloc_mem )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    /* Start with all available RAM */
>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>> +    {
>>>> +        start = bootinfo.mem.bank[i].start;
>>>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size - 1;
>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove RAM assigned to Dom0 */
>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>> +    {
>>>> +        start = assign_mem->bank[i].start;
>>>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size - 1;
>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove reserved-memory regions */
>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>> +    {
>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +        if ( res )
>>>> +        {
>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>> +                   start, end);
>>>> +            goto out;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /* Remove grant table region */
>>>> +    start = kinfo->gnttab_start;
>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>> +    if ( res )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>>> +               start, end);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    start = EXT_REGION_START;
>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>> +                                 add_ext_regions, ext_regions);
>>>> +    if ( res )
>>>> +        ext_regions->nr_banks = 0;
>>>> +    else if ( !ext_regions->nr_banks )
>>>> +        res = -ENOENT;
>>>> +
>>>> +out:
>>>> +    rangeset_destroy(unalloc_mem);
>>>> +
>>>> +    return res;
>>>> +}
>>>> +
>>>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>>>> +                                    struct meminfo *ext_regions)
>>>> +{
>>>> +    struct dt_device_node *np;
>>>> +    struct rangeset *mem_holes;
>>>> +    paddr_t start, end;
>>>> +    unsigned int i;
>>>> +    int res;
>>>> +
>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>> +
>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>> +    if ( !mem_holes )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    /* Start with maximum possible addressable physical memory range */
>>>> +    start = EXT_REGION_START;
>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>> +    if ( res )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>>> +               start, end);
>>>> +        goto out;
>>>> +    }
>>>> +
>>>> +    /* Remove all regions described by "reg" property (MMIO, RAM, etc) */
>>> Well... The loop below is not going to handle all the regions described in
>>> the property "reg". Instead, it will cover a subset of "reg" where the
>>> memory is addressable.
>> As I understand, we are only interested in subset of "reg" where the memory is
>> addressable.
>>
>>
>>>
>>> You will also need to cover "ranges" that will describe the BARs for the PCI
>>> devices.
>> Good point.
> Yes, very good point!
>
>
>> Could you please clarify how to recognize whether it is a PCI
>> device as long as PCI support is not merged? Or just to find any device nodes
>> with non-empty "ranges" property
>> and retrieve addresses?
> Normally any bus can have a ranges property with the aperture and
> possible address translations, including /amba (compatible =
> "simple-bus"). However, in these cases dt_device_get_address already
> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>
> The PCI bus is special for 2 reasons:
> - the ranges property has a different format
> - the bus is hot-pluggable
>
> So I think the only one that we need to treat specially is PCI.
>
> As far as I am aware PCI is the only bus (or maybe just the only bus
> that we support?) where ranges means the aperture.
Thank you for the clarification. I need to find device node with 
non-empty ranges property
(and make sure that device_type property is "pci"), after that I need to 
read the context of ranges property and translate it.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 20:08:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 20:08:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191887.342027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSm3x-0003SN-Js; Tue, 21 Sep 2021 20:08:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191887.342027; Tue, 21 Sep 2021 20:08:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSm3x-0003SG-Gx; Tue, 21 Sep 2021 20:08:21 +0000
Received: by outflank-mailman (input) for mailman id 191887;
 Tue, 21 Sep 2021 20:08:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QjZ+=OL=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSm3v-0003SA-IX
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 20:08:19 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bc9e650f-1b3c-4feb-8bbc-a80e198b4684;
 Tue, 21 Sep 2021 20:08:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc9e650f-1b3c-4feb-8bbc-a80e198b4684
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632254898;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=/RcOwvX6G+A6yZfYmGyj4oEr/iP4SJj3cNoYoxqbAv8=;
  b=Smyvg2igo3ZTj3j+/Wnl+Sa2vRX+W1p727jssftOJiiHYq+V0F2hTHT0
   LmTP2rRRBAexVlm+CLMtw+8aidlHNI9kQyfxwETcFf6HTGj+1DvbhFPBc
   JcJMfTZoW255Xz09pLgEdd+/Sg+mKbzVbdKHmDxBJiozqR4sdrGElgxA2
   o=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: FJLzR/KIWUxgLnQrbub6wgBYMkXjlar5/HiZAaj/picWDuoT1+n/e9gcCiF5LWkfl6xfYxxCN+
 tbj/LCvv2FJX5XPOhXnio8Ry9YdA98dlheWdAspWvu32nGPOxHnZPNAeCrTNbXdhkbYakHNSPQ
 dw6F8gRy7abBtoTn8A6xArI0vx0mV4RxqRDujwEQHXmtDIpUMvez3Wl/ZjvKI+G4KYMHXrMQSi
 IZa+Nqorj7O94uQe+8vKMjhIi5Rc8IQw7x1uyRTqbVaUPISgmlC8t5tYAHIZwsfxDszxj4WinY
 o5NWYOAqCAnVluV/BxA73FTd
X-SBRS: 5.1
X-MesageID: 53252835
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:6Y9hu6srlIRBzc9zwyzKuptJOefnVOBZMUV32f8akzHdYApBsoF/q
 tZmKW/TM/6MZWDyKo13Odyz9h4DuZ/RnNQ3GwFqpXgwRHlG+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524HhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpljbWRSzYMfYr3quUUbTwCDHwmN45Z0eqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DP
 JBEM2o0MXwsZTV1a3EJUbdkk96YqVLwYTBg8gmPmaEetj27IAtZj+G2bYu9lsaxbd5Ogk+Sq
 2bC/mL4KhIXLtqSzXyC6H3ErvDLtTP2XsQVDrLQ3txAjUCXx2cTIAYLTlb9qv684mauVtQaJ
 0EK9y4Gqakp6FftXtT7Rwe/onOPolgbQdU4O+wl6imdx6zM+QGbC2MYCDlbZ7QOisgyQjA70
 06TqPngDzdvrb69RGqU8/GfqjbaES0YK2ASeAcPTBBD/sHupod1gx7SJv5hH7SylcbdAizrz
 naBqy1Wr7cOic8G0Y2r8FaBhCijzrDFQAc66QTQWmON9R5iaciuYInA1LTAxa8edsDDFADH5
 SVa3ZjFhAwTMX2TvBSkYec8OfKO3MSEOQP520EyR5MC6B34rhZPYrtsDCFCyFZBa5hfI2a5P
 hOL4Gu997cIYyD7NvYfj5aZTp1wlPm+T4yNuuX8M4IWCqWdYjNr682HiaS45GnrjEFkuqU2I
 57znS2EXCtCVPgPINZbQY4gPV4XKsIWnji7qXPTlU3PPV+iiJm9E+xtDbd2RrplhJ5oWS2Mm
 zqlCydv9yizrcWkOnWHmWLsEbz6BSdiXs2nwyCmXsWCPhBnCAkcNhMl+pt4I9YNt/0Mzo/gp
 yjhMmcFmAuXrSCWcm2iNyE8AI4DqL4i9BrXywR3Zg32s5XiCK7yhJoim2wfJ+h/qLM7kq4qE
 JHouayoW5xyd9gOwBxEBbHVp41+bhW7wwWIOiuuej8keJB8AQfO/7fZksHHrkHi1wK76pkzp
 aOOzATeTcZRTghuFp+OOvmu00mwrT4Wn+crBxnEJdxaeUPN9ol2KnOu0q9rcp9UcRiTlCGH0
 wu2AAsDobWfqYEC79SU17uPqJ2kErUiExMCTXXb97u/KQLT4nGnnd1bSO+NcD2EDDH09ayua
 P971fb5NPFbzl9Gv5AlS+RgzL4k5suprLhfl1w2EHLOZlWtK7VhPnjZgpUf6vwTnudU4FLkV
 FiO999WPaSyFPnkSFNBdhA4aumj1O0PnmWA5/oCP0intjR8+6CKUBsOMkDU2jBdNrZ8LKgs3
 fwl5JwN8wW6hxcnboSGgyRT+zjeJ3AMSfx65JQTAYutgQs30FBSJ5fbD3ausp2IbtxNNGgsI
 yOV2/We1+gNmBKafiphD2XJ0MpcmY8K6UJDw1I1LliUnsbI260s1xpL/DVrFglYw32rCQ6o1
 rSH46GtGZizwg==
IronPort-HdrOrdr: A9a23:W6qbCKFjUwHxiGTtpLqFf5HXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HkBEDyewKiyXcT2/hsAV7CZniahILMFu9fBOTZskXd8kHFh4lgPO
 JbAtJD4b7LfChHZKTBkXCF+r8bqbHtmsDY5pas854ud3APV0gJ1XYJNu/xKDwReOApP+taKH
 PR3Ls9m9L2Ek5nEPhTS0N1E9TrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqJmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87CsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXkHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1XackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMW9yV0qp+1WH/ebcGkjaRny9Mw8/U42uonlrdUlCvgklLJd1pAZHyHo/I6M0r9
 gsfJ4YzY2n46ctHNVA7dw6ML6K41r2MFvx2VKpUCba/Z48SgbwQr7Mkf8IDbKRCdE1JKVbou
 W2bLofjx9rR37T
X-IronPort-AV: E=Sophos;i="5.85,311,1624334400"; 
   d="scan'208";a="53252835"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=coX+2saowqZIDlN2NH7rKzFlHjReiRobcXBD/4x54Jx7fIH4Umumcq9TsgfJdmwVkipd9tMZMKefexCS/GA502/Mm1SP+BnnKvS6Zybek2i16Hef1i3nM0aWxRnz59iT7sK36r5ec3/NnjvL4YS53IQ7ez90VaZWTVNH7Hlrynxa5HENewYvmXioYTtKYZnid2cevdwUnRTiLhTCEJDXrhlmaLgDY2OmkhYEO6Pfz0slqJi2xVmDLKlt8NSFKO9uh9Ut5V6IgrePS5cCNDZgzRBvUoeaB6pTqJH34UKKF/vyy/uCol5XAjqFfR4rPkGgWLtFP2xwQesgWzutSWHKag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/RcOwvX6G+A6yZfYmGyj4oEr/iP4SJj3cNoYoxqbAv8=;
 b=GqCO0fWZIO54dqan6hCpxwN/9ihhn56gM5oInxLKaS2fUD5z64cTkVdcQdGmTG6AXjwg85+bJHoB6U9e0ttQvQTFoleri3lM1FDgTQdB6TC2aNHgQpCueCSUdwN+yIS+k8e1ke6zWvaJ0wD0dqiiWUdPcXx5ocNANuBM6E/eUXwWqLi3CoNArjoWfMbyQeJJxIKxwllDSlK7QQM29ejNPgELalcKYP4O0vjhoYG547/mTu5i5xB1xaQSvxvtHEebOLYkcubhm8xBc6uZdpjYMLOS9qvRcEHTElNanM2IbFHH7hQx30Nv+oMV8g9PUhmbGTuXRpplVTsZJ6kKilGIbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/RcOwvX6G+A6yZfYmGyj4oEr/iP4SJj3cNoYoxqbAv8=;
 b=VJhm6c0yFX08BLoPc1ObO6HAXPcyLtu5oMdTMjJQJUSKIqOJMXm4mI0GW9tz4r4BWZNius9EYXpr6wRnsF+zz4UlnqrpbKmGJLkYDmrOjabG5yj0c84Y0WyPav5As0b0+JPTx28Luieuf01ECrfY968KWI/RnXW6ygOpnFWY4/Y=
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Jan Beulich <JBeulich@suse.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
	<julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>, Juergen Gross
	<jgross@suse.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Meng Xu
	<mengxu@cis.upenn.edu>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2.1 RFC 17/12] xen/trace: Drop cycles parameter
Message-ID: <2bd51843-de50-d442-dd5f-7fcc340c3f26@citrix.com>
Date: Tue, 21 Sep 2021 21:08:05 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920172529.24932-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0210.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a5::17) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fd7a6898-e703-45f9-4ece-08d97d3b8acd
X-MS-TrafficTypeDiagnostic: BY5PR03MB5346:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB5346798F7AAD2E05C7FDCFAABAA19@BY5PR03MB5346.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: cPWkHuSRbtT17p6JFiGs8hmT8DUkMaH3uR16yL8xvqF+fDiDc0BUIhH/ArTH5310GPA65rfemlAdpnvA4zwNkXMY1wWbUIavW+QUsGpYYC73xhiliv63PdnMfm4SK/Zk/Y8rQPPWThW4BrgcdyQD/XhSEC1etD6T02/8uqoMomGXJrADa/Apd8NwBnpQ9dSFKd3rtBnYrY11YscEDE1QC8q+SX0r2mvheeTB23X4qLEEdymdAW3izGRd1QxeOOb/ZxrVSJCj762STZ6jLejj4JG6wDH3OPIlqz0jDiuujX8NSWV7ho3YyNbJfamW9hnbHwqpOZC84sl2N8K65ox79wcPsazQTGgPHqwPA3dJcK5E+kGiYWGPcsDVBVhgReI0pV5qVr3XIqeZrwVrZuP9Eq5wcmy3frVRn8N4EgXC1lGeX3AAWIGFVOM/bVnsf+4VA+gcshlRD3T0hCoTupay54yasv1xu3yLDl5toLuXYk5ZOcJ+6af4BXw3ZiYGbWRRnQEb4PwXMr0OAVGhgIMUJmjkaNY6lNr3bAN+Slgu9K+AVp1/X8ra0yO87xwV+URxFKkp4YGc9qA9NMCJ5ob6dwAX94XEZiVSMqzE0ovnS49tzDM+Z8tijIj/6jGVve1KViYsb6+V2+KAH80XOGg0P7lyFIrWkKeu6oq2eQ1l6A69sg3gkO8lJiaLi76qlxb1SdOKo4YWBr9XJg0Tds0+Hfy9VZKnmxIjOlJAXUAblC0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(6666004)(54906003)(36756003)(86362001)(5660300002)(55236004)(66476007)(38100700002)(6916009)(16576012)(66556008)(8936002)(316002)(8676002)(31686004)(7416002)(6486002)(4326008)(956004)(31696002)(2616005)(508600001)(26005)(186003)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eDhIcWlLV2k4a0ZiNlZSdFE4dnJjakt5a1kwQlVHZW5yNVU5UkVnV3ovWDNH?=
 =?utf-8?B?bmw4R0F0QnlIamhHSjU3M2RQaXM4Vng2Q3A1YU5uUTlxTUFFR2djaUM0ZUI3?=
 =?utf-8?B?eGpBc1J5bzhodTFjMnZJQm1EYjVrdXZ5T3ZuOEVsTFg4NnR6L1pQb0lvUHNB?=
 =?utf-8?B?bmlyNjFrTGp1Q0swcGlMRGVZai9vNFNLa0RpK2JrZnQwdVNYWDdzeXFGaW1G?=
 =?utf-8?B?K3p2WEExQ2JVL0hydUs1RjdJMFRzR3F3Y0l3ZW9iZVJheFM1STBXNWpVWmxY?=
 =?utf-8?B?eXU0Q2JWR1JyYy9HVTBXYkoxMFdraFNYaGp6V3NYNjdQbmJIQ0ZXcDJUY1A0?=
 =?utf-8?B?U3BXaWVXSjZDUXBPN29OdEZIK1E1Y3lyRXNKdU1TQnpQUitVS244cTYyYjIw?=
 =?utf-8?B?YnU1UXd1MU40NWJvV3ZOeVNvL0V6a0wyT0hFVjZKUHFEbjA4dG1HQ0hTOVBy?=
 =?utf-8?B?WnZpeDEvSUVTT3htMlhPSjNiejlCMUdNVHlWUmRiVnQvNml6VS9MK0IxRnhI?=
 =?utf-8?B?OG5SNTR1Sy9Tb1RFZFRpNDV2Nmh2cWphUWNxSUpZN2NKakZaSUFwK2VZaXZV?=
 =?utf-8?B?R1JmQ21JSWVsR0ZwejQwcEYxVDlBWkxTQUpLd3Uwd0Y0UnlRenFrMjZ1Njkz?=
 =?utf-8?B?YmJ3VDZOdFgreCtPMU9HT21tSVc0QjdmUWExMzkyZDg4VENXVFhjZTJIbDZt?=
 =?utf-8?B?ZWlvNmF4SU9VTDZzQVloRXhsNi9HblppOWhaWFNLbE9CUG01N3dqUC93dVBO?=
 =?utf-8?B?c2NXMmdRUVBJR1UxR3UyTUNjYWQ2Q3IzM3UzdTFHSG43c3JIazQxblVPVnJQ?=
 =?utf-8?B?SVU0TFg0dFczeXpKYy8wSFp3SGhOZEgwclJYcGMrcUxXVnZyS2V0MXBZQWsy?=
 =?utf-8?B?TGM2cC9palh0eE5KM252a0JjemxYOSsraEhSbngxbTFFNDRjb3MvczVsWUZU?=
 =?utf-8?B?YUNYeVdSMEVMdmVKcXVJNXdhVVUwUG5KN0F4SzMxRytHZzdaZllENXFlZFRh?=
 =?utf-8?B?TUE4NUZxejVjOHV3cHYwVXAvQmJkNlVtRk9pMW5RbXVUOExyREN6czhTK0VY?=
 =?utf-8?B?ejJldXJvdWtxMWhxcmRDRVM5WVVRNFk0N01CYWIzMG5UTFNaempJZVVrR05V?=
 =?utf-8?B?NU56bmpFdHhiOFBnNnRjRTAwQ3BHMndqZzM2U3lmMC8xVjFxZmhDR3p2MUx3?=
 =?utf-8?B?L2sxVWRDOU9hakxON2ZySXpRUTgxbE1mNTN2U1p5bktqb2tUMTZFKzFpS00w?=
 =?utf-8?B?Y3pLekxBOVZ4QXh2Ukh1VXdUTEw4czVtRGsxd0FuaFJkYVY0M1Y1SHUwekI2?=
 =?utf-8?B?dDgrN3Q1M2ljZmFyTXM4YmJLMi93UHdjaEh5SmxGUFhjSlFWMHRVTWNPLzN4?=
 =?utf-8?B?NC96MitDQ0ZvTzg1SEpRMjhXdkZKMzBLUnFZNnVML2VySi9yWnJkZFJvdzls?=
 =?utf-8?B?VStLVHBjemh0RXpCMmlvbW43NlVkemJCbmhuVUNZc2dWdStwdTgxTThJQm1Z?=
 =?utf-8?B?cG9kL3JJVERuR01TUXNhakMyUVZROTVPTzNndkg2YzNtdVkwak1xV3h0bm84?=
 =?utf-8?B?Z292OFRxdVhFaUlqUWxORTIzTnhxVktVVGxiMlRyRVB5dlBkUFFkbjJBc1M5?=
 =?utf-8?B?Kzl0djBWa1ZZZXJiM0JIYzkrOFZnSHhaTUV2cEM0b3RNU1JZczhyT1pjQk10?=
 =?utf-8?B?T1Z5T3d5NHNrZzhkQnd3QmVpalBuT3ZzV01iZ1Q4cmlSR0pVeXduZFdIeG4r?=
 =?utf-8?Q?FnccQgBiTnTJbcMteQHiv3PzHO5NUpeJSvVHU3i?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fd7a6898-e703-45f9-4ece-08d97d3b8acd
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 20:08:13.6448
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: k92JiFgrsT9Hm0mfa1Klf0Nh/HvW7gzfTH/tLFYTmdyE7TJeqhAwppSOGv/8ucqdc88FbsPybI+XoBf8ekzrC61DjlxHGzexq2n3a1xgJKE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5346
X-OriginatorOrg: citrix.com

Not a patch, but an RFC idea for one...

It occurred to me that the cycles parameter from __trace_var() and
friends is pointless, as the cycles bit is encoded in the top bit of the
event field anyway, and passed in the adjacent parameter.

Dropping the cycles parameter results in +85/-1029 (-944) net change.

The common change in callers is e.g. from:

lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rcx
mov=C2=A0=C2=A0=C2=A0 $0x4,%edx
mov=C2=A0=C2=A0=C2=A0 $0x1,%esi
mov=C2=A0=C2=A0=C2=A0 $0x10f002,%edi
mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
callq=C2=A0 ffff82d04022ea20 <__trace_var>

to this:

lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rdx
mov=C2=A0=C2=A0=C2=A0 $0x4,%esi
mov=C2=A0=C2=A0=C2=A0 $0x8010f002,%edi
mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
callq=C2=A0 ffff82d04022ea20 <__trace_var>


Just sprinkling "| TRC_HD_CYCLE_FLAG" over the place makes things a
little bit unwieldy.

Instead, I was thinking of implementing trace() (and a thin trace_time()
wrapper) mirroring the "new API" in patch 14.=C2=A0 Half of the trace_var()
users should be __trace_var() already because of living inside a
tb_init_done conditional, and the rest are actually opencoded TRACE()
taking no extra data.

Thoughts?

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 21 20:09:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 20:09:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191893.342039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSm5K-00043w-W0; Tue, 21 Sep 2021 20:09:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191893.342039; Tue, 21 Sep 2021 20:09:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSm5K-00043p-Rd; Tue, 21 Sep 2021 20:09:46 +0000
Received: by outflank-mailman (input) for mailman id 191893;
 Tue, 21 Sep 2021 20:09:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSm5J-00043f-Gm; Tue, 21 Sep 2021 20:09:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSm5J-0006za-B2; Tue, 21 Sep 2021 20:09:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSm5I-000409-VF; Tue, 21 Sep 2021 20:09:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSm5I-0006V8-Ui; Tue, 21 Sep 2021 20:09:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PI0PPfB5d7k0XNvJuFEGrQIU9lqJpHPgmqS5Yrp4TTM=; b=1Vl2XixUvLdk7N7DJcojKk4orV
	okMNK5MI1q4jGfu9gRyTbGkMxKUoFF+dTLKmkYJaJug1AhH/QD2EuHdifE0hvsSodxQOl54iYAAcx
	L+W4iFKvuVyYKDeoQV3C1FC91OXv39KJy9D62bNR3yP1Yido7TOYo3tUipBkeriYJX4o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165139-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165139: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=326ff8dd09556fc2e257196c49f35009700794ac
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 20:09:44 +0000

flight 165139 qemu-mainline real [real]
flight 165144 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165139/
http://logs.test-lab.xenproject.org/osstest/logs/165144/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                326ff8dd09556fc2e257196c49f35009700794ac
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   10 days
Failing since        164967  2021-09-13 13:06:52 Z    8 days   17 attempts
Testing same since   165133  2021-09-20 22:08:14 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3339 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 20:44:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 20:44:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191903.342053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSmcd-0008Or-SX; Tue, 21 Sep 2021 20:44:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191903.342053; Tue, 21 Sep 2021 20:44:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSmcd-0008Ok-PY; Tue, 21 Sep 2021 20:44:11 +0000
Received: by outflank-mailman (input) for mailman id 191903;
 Tue, 21 Sep 2021 20:44:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSmcc-0008OO-Me
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 20:44:11 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id aa50cbf1-1b1c-11ec-b92d-12813bfff9fa;
 Tue, 21 Sep 2021 20:44:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3EA8E61156;
 Tue, 21 Sep 2021 20:44:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa50cbf1-1b1c-11ec-b92d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632257048;
	bh=Ck7yVJFIXzTFvAFx0p96/tMpeKNlfNntmCEPNWDueOg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fvCrNTLnkeVpEZENUSbcIL3JRM/o/OG+6d9GKn5LIoXiDA6Gf8ZSe4X222Vfl1kRf
	 BsfiSEbG24XluNMRo9/FWlVjdeP8umRwX8wCowvRDbkAdUp/irjqBrW/EMhjmYLxQa
	 /gjOh8GquvqcaUfiGi6uMUikfkcgpi01u7o5AuLopaY7CNjLOiorGSMCrSXTJB73rT
	 qxUs6EBdM/vNyglAZ+/LRS7pIJ6KLTmMFfWJwjuzADzOco2JOJ8PbY4Ne5balbfLO1
	 AYFMFZpcI/ugwREgOLPHqTq0NFEHcyMcxvelcmVNVVjLHyD6Hc7oqUbA0Es/0WDyPW
	 mdCNQdNpDBBHQ==
Date: Tue, 21 Sep 2021 13:44:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>, 
    Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
In-Reply-To: <48a2ef20-02ad-99e4-a8f5-fa144692aadc@epam.com>
Message-ID: <alpine.DEB.2.21.2109211340470.17979@sstabellini-ThinkPad-T480s>
References: <20210917130123.1764493-1-andr2000@gmail.com> <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s> <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com> <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com> <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s> <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com> <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com> <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com> <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com> <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com> <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com> <48a2ef20-02ad-99e4-a8f5-fa144692aadc@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-722727834-1632257047=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-722727834-1632257047=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 21 Sep 2021, Oleksandr Andrushchenko wrote:
> On 21.09.21 10:09, Juergen Gross wrote:
> > On 21.09.21 09:00, Oleksandr Andrushchenko wrote:
> >>
> >> On 21.09.21 09:49, Juergen Gross wrote:
> >>> On 21.09.21 08:38, Oleksandr Andrushchenko wrote:
> >>>>
> >>>> On 21.09.21 09:07, Juergen Gross wrote:
> >>>>> On 21.09.21 07:51, Oleksandr Andrushchenko wrote:
> >>>>>>
> >>>>>> On 21.09.21 08:20, Juergen Gross wrote:
> >>>>>>> On 21.09.21 01:16, Stefano Stabellini wrote:
> >>>>>>>> On Mon, 20 Sep 2021, Oleksandr Andrushchenko wrote:
> >>>>>>>>> On 20.09.21 14:30, Juergen Gross wrote:
> >>>>>>>>>> On 20.09.21 07:23, Oleksandr Andrushchenko wrote:
> >>>>>>>>>>> Hello, Stefano!
> >>>>>>>>>>>
> >>>>>>>>>>> On 18.09.21 00:45, Stefano Stabellini wrote:
> >>>>>>>>>>>> Hi Oleksandr,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Why do you want to enable pciback on ARM? Is it only to "disable" a PCI
> >>>>>>>>>>>> device in Dom0 so that it can be safely assigned to a DomU?
> >>>>>>>>>>> Not only that
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am asking because actually I don't think we want to enable the PV PCI
> >>>>>>>>>>>> backend feature of pciback on ARM, right? That would clash with the PCI
> >>>>>>>>>>>> assignment work you have been doing in Xen. They couldn't both work at
> >>>>>>>>>>>> the same time.
> >>>>>>>>>>> Correct, it is not used
> >>>>>>>>>>>>
> >>>>>>>>>>>> If we only need pciback to "park" a device in Dom0, wouldn't it be
> >>>>>>>>>>>> possible and better to use pci-stub instead?
> >>>>>>>>>>>
> >>>>>>>>>>> Not only that, so pci-stub is not enough
> >>>>>>>>>>>
> >>>>>>>>>>> The functionality which is implemented by the pciback and the toolstack
> >>>>>>>>>>> and which is relevant/missing/needed for ARM:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. pciback is used as a database for assignable PCI devices, e.g. xl
> >>>>>>>>>>>          pci-assignable-{add|remove|list} manipulates that list. So, whenever the
> >>>>>>>>>>>          toolstack needs to know which PCI devices can be passed through it reads
> >>>>>>>>>>>          that from the relevant sysfs entries of the pciback.
> >>>>>>>>>>>
> >>>>>>>>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when passing through
> >>>>>>>>>>>          a PCI device it needs to be unbound from the relevant device driver and bound
> >>>>>>>>>>>          to pciback (strictly speaking it is not required that the device is bound to
> >>>>>>>>>>>          pciback, but pciback is again used as a database of the passed through PCI
> >>>>>>>>>>>          devices, so we can re-bind the devices back to their original drivers when
> >>>>>>>>>>>          guest domain shuts down)
> >>>>>>>>>>>
> >>>>>>>>>>> 3. Device reset
> >>>>>>>>>>>
> >>>>>>>>>>> We have previously discussed on xen-devel ML possible solutions to that as from the
> >>>>>>>>>>> above we see that pciback functionality is going to be only partially used on Arm.
> >>>>>>>>>>>
> >>>>>>>>>>> Please see [1] and [2]:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. It is not acceptable to manage the assignable list in Xen itself
> >>>>>>>>>>>
> >>>>>>>>>>> 2. pciback can be split into two parts: PCI assignable/bind/reset handling and
> >>>>>>>>>>> the rest like vPCI etc.
> >>>>>>>>>>>
> >>>>>>>>>>> 3. pcifront is not used on Arm
> >>>>>>>>>>
> >>>>>>>>>> It is neither in x86 PVH/HVM guests.
> >>>>>>>>> Didn't know that, thank you for pointing
> >>>>>>>>>>
> >>>>>>>>>>> So, limited use of the pciback is one of the bricks used to enable PCI passthrough
> >>>>>>>>>>> on Arm. It was enough to just re-structure the driver and have it run on Arm to achieve
> >>>>>>>>>>> all the goals above.
> >>>>>>>>>>>
> >>>>>>>>>>> If we still think it is desirable to break the pciback driver into "common" and "pcifront specific"
> >>>>>>>>>>> parts then it can be done, yet the patch is going to be the very first brick in that building.
> >>>>>>>>>>
> >>>>>>>>>> Doing this split should be done, as the pcifront specific part could be
> >>>>>>>>>> omitted on x86, too, in case no PV guests using PCI passthrough have to
> >>>>>>>>>> be supported.
> >>>>>>>>> Agree, that the final solution should have the driver split
> >>>>>>>>>>
> >>>>>>>>>>> So, I think this patch is still going to be needed besides which direction we take.
> >>>>>>>>>>
> >>>>>>>>>> Some kind of this patch, yes. It might look different in case the split
> >>>>>>>>>> is done first.
> >>>>>>>>>>
> >>>>>>>>>> I don't mind doing it in either sequence.
> >>>>>>>>>>
> >>>>>>>>> With this patch we have Arm on the same page as the above mentioned x86 guests,
> >>>>>>>>>
> >>>>>>>>> e.g. the driver has unused code, but yet allows Arm to function now.
> >>>>>>>>>
> >>>>>>>>> At this stage of PCI passthrough on Arm it is yet enough. Long term, when
> >>>>>>>>>
> >>>>>>>>> the driver gets split, Arm will benefit from that split too, but unfortunately I do not
> >>>>>>>>>
> >>>>>>>>> have enough bandwidth for that piece of work at the moment.
> >>>>>>>>
> >>>>>>>> That's fair and I don't want to scope-creep this simple patch asking for
> >>>>>>>> an enormous rework. At the same time I don't think we should enable the
> >>>>>>>> whole of pciback on ARM because it would be erroneous and confusing.
> >>>>>>
> >>>>>> As the first stage before the driver is split or ifdef's used - can we take the patch
> >>>>>> as is now? In either way we chose this needs to be done, e.g. enable compiling
> >>>>>> for other architectures and common code move.
> >>>>>
> >>>>> Fine with me in principle. I need to take a more thorough look
> >>>>> at the patch, though.
> >>>> Of course
> >>>>>
> >>>>>>
> >>>>>>>>
> >>>>>>>> I am wonder if there is a simple:
> >>>>>>>>
> >>>>>>>> if (!xen_pv_domain())
> >>>>>>>>         return;
> >>>>>>>>
> >>>>>>>> That we could add in a couple of places in pciback to stop it from
> >>>>>>>> initializing the parts we don't care about. Something along these lines
> >>>>>>>> (untested and probably incomplete).
> >>>>>>>>
> >>>>>>>> What do you guys think?
> >>>>>>>
> >>>>>>> Uh no, not in this way, please. This will kill pci passthrough on x86
> >>>>>>> with dom0 running as PVH. I don't think this is working right now, but
> >>>>>>> adding more code making it even harder to work should be avoided.
> >>>>>>>
> >>>>>>>> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> >>>>>>>> index da34ce85dc88..991ba0a9b359 100644
> >>>>>>>> --- a/drivers/xen/xen-pciback/xenbus.c
> >>>>>>>> +++ b/drivers/xen/xen-pciback/xenbus.c
> >>>>>>>> @@ -15,6 +15,7 @@
> >>>>>>>>      #include <xen/xenbus.h>
> >>>>>>>>      #include <xen/events.h>
> >>>>>>>>      #include <xen/pci.h>
> >>>>>>>> +#include <xen/xen.h>
> >>>>>>>>      #include "pciback.h"
> >>>>>>>>        #define INVALID_EVTCHN_IRQ  (-1)
> >>>>>>>> @@ -685,8 +686,12 @@ static int xen_pcibk_xenbus_probe(struct xenbus_device *dev,
> >>>>>>>>                      const struct xenbus_device_id *id)
> >>>>>>>>      {
> >>>>>>>>          int err = 0;
> >>>>>>>> -    struct xen_pcibk_device *pdev = alloc_pdev(dev);
> >>>>>>>> +    struct xen_pcibk_device *pdev;
> >>>>>>>> +
> >>>>>>>> +    if (!xen_pv_domain())
> >>>>>>>> +        return 0;
> >>>>>>>>      +    pdev = alloc_pdev(dev);
> >>>>>>>
> >>>>>>> This hunk isn't needed, as with bailing out of xen_pcibk_xenbus_register
> >>>>>>> early will result in xen_pcibk_xenbus_probe never being called.
> >>>>>>>
> >>>>>>>>          if (pdev == NULL) {
> >>>>>>>>              err = -ENOMEM;
> >>>>>>>>              xenbus_dev_fatal(dev, err,
> >>>>>>>> @@ -743,6 +748,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
> >>>>>>>>        int __init xen_pcibk_xenbus_register(void)
> >>>>>>>>      {
> >>>>>>>> +    if (!xen_pv_domain())
> >>>>>>>> +        return 0;
> >>>>>>>> +
> >>>>>>>
> >>>>>>> Use #ifdef CONFIG_X86 instead.
> >>>>>>
> >>>>>> The title of this patch says that we want to allow this driver for other archs
> >>>>>> and now we want to introduce "#ifdef CONFIG_X86" which doesn't sound
> >>>>>> right with that respect. Instead, we may want having something like a
> >>>>>> dedicated gate for this, e.g. "#ifdef CONFIG_XEN_PCIDEV_BACKEND_SUPP_PV"
> >>>>>> or something which is architecture agnostic.
> >>>>>
> >>>>> Something like that, yes. But I'd rather use CONFIG_XEN_PCIDEV_BACKEND
> >>>>> acting as this gate and introduce CONFIG_XEN_PCI_STUB for the stub
> >>>>> functionality needed on Arm. XEN_PCIDEV_BACKEND would depend on X86 and
> >>>>> select XEN_PCI_STUB, while on Arm XEN_PCI_STUB could be configured if
> >>>>> wanted. The splitting of the driver can still be done later.
> >>>>
> >>>> Hm, pciback is now compiled when CONFIG_XEN_PCIDEV_BACKEND is enabled
> >>>> and we want to skip some parts of its code when CONFIG_XEN_PCI_STUB is set.
> >>>> So, I imagine that for x86 we just enable CONFIG_XEN_PCIDEV_BACKEND and the
> >>>> driver compiles in its current state. For Arm we enable both CONFIG_XEN_PCIDEV_BACKEND
> >>>> and CONFIG_XEN_PCI_STUB, so part of the driver is not compiled.
> >>>
> >>> No, I'd rather switch to compiling xen-pciback when CONFIG_XEN_PCI_STUB
> >>> is set and compile only parts of it when CONFIG_XEN_PCIDEV_BACKEND is
> >>> not set (this will be the case on Arm).
> >>
> >> But this will require that the existing kernel configurations out there have to additionally define CONFIG_XEN_PCI_STUB to get what they had before with simply enabling CONFIG_XEN_PCIDEV_BACKEND. My point was that it is probably desirable not to break
> >> the things while doing the split/re-work.
> >
> > By letting XEN_PCIDEV_BACKEND select XEN_PCI_STUB this won't happen.
> Indeed
> >
> >> I also thought that "compile only parts of it when CONFIG_XEN_PCIDEV_BACKEND is not set"
> >> may have more code gated rather than with gating unwanted code with CONFIG_XEN_PCI_STUB.
> >> I am not quite sure about this though.
> >
> > This would be a very weird semantics of XEN_PCI_STUB, as the stub part
> > is needed on X86 and on Arm.
> >
> > Gating could even be done with Stefano's patch just by replacing his
> > "!xen_pv_domain()" tests with "!IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND)".
> 
> Makes sense.
> 
> Another question if we do not want the code to be compiled or not executed?
> 
> If the later then we can define something like:
> 
> bool need_pv_part(void)
> 
> {
> 
>      return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
> 
> }
> 
> and then just use need_pv_part() for the checks where it is needed.
> 
> This allows avoiding multiple ifdef's through the code

This is even better.

For my clarity, Oleksandr, are you OK with adding a few need_pv_part()
checks through the code as part of this series so that the PV PCI
backend is not initialized?

I don't have a good test environment ready for this, so I cannot really
volunteer myself.

I would prefer if we made this change as part of this series so that the
PV PCI backend features doesn't get enabled on ARM, not even temporarily.
--8323329-722727834-1632257047=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 21:30:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 21:30:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191910.342064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnKv-0004dm-9g; Tue, 21 Sep 2021 21:29:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191910.342064; Tue, 21 Sep 2021 21:29:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnKv-0004df-6Q; Tue, 21 Sep 2021 21:29:57 +0000
Received: by outflank-mailman (input) for mailman id 191910;
 Tue, 21 Sep 2021 21:29:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=uuut=OL=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mSnKt-0004dW-IZ
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 21:29:55 +0000
Received: from mail-pj1-x1033.google.com (unknown [2607:f8b0:4864:20::1033])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b956dcc-2d48-4d64-93c4-6905fbb6f63b;
 Tue, 21 Sep 2021 21:29:54 +0000 (UTC)
Received: by mail-pj1-x1033.google.com with SMTP id dw14so546266pjb.1
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 14:29:54 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id w142sm103618pfc.47.2021.09.21.14.29.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 Sep 2021 14:29:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b956dcc-2d48-4d64-93c4-6905fbb6f63b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=LSn/kFygvp1Gc7LYRPSl9USN2eKVUYYqQK80C4S3HbY=;
        b=btxgeKmkbIeI+ZpOc6VlwSC4J1M0NxXCmAKO/nawCIYSv8cbGywGkAH3QCnwAqLAUI
         IOyxlnQF3u2reCho4aOS27m8AURN66z+f1gzEN4gfB+N1/w2JLIXtC4yGrRdYBUrXdWC
         Vx/SNzr0zgBg66Mt5tqVTWXtqNRo1cMW1HRk7P6/aIgQitN5dojnf/rmx3fJsNEBiKh4
         2oG4JdRFZ1cCImM5O4stFWaGUi4+BBk4LEDc8ZYoSTyQQ09FyLfXbzxkQRIwj2f3Y1ux
         qkOUO+HshbP+1GDpYX5TBWF68F9+FWvzry5+DIxEzwl/5EC5N4EFZsnTQtNpn8FiRBGK
         Np7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=LSn/kFygvp1Gc7LYRPSl9USN2eKVUYYqQK80C4S3HbY=;
        b=70UGbx1MaPktxKoVGjiCDqLF6qJqd9nsKq8mS1KBbuce1/XppzLrSJoIZG6stnNWT2
         X7C4K4VAnmOCzzSEGEuXfTTfh+QrTo5kybnTzpk5rqWiG7/gN4N2w0/RDjW6VHSVBYlE
         OUGB2uBaFRD9Y5o92/zVrQ03NdwZ+nqcJRdC9IKl+/Bo9Sv9xLCu8tpvceRwwPi+TMEb
         Wmkz70AKpkbhBxmQd60I/brIirRecwEZWsN5biXuvNZm4chrOQ9s64H2Ne0FhDvpv2Fz
         +5+YFPH/o2tMEM3hC5+i7TUeb6XnDiFRfDHBxVJFt89nrJzOLXdOPsLmdVYdnSyXh60i
         tNtg==
X-Gm-Message-State: AOAM533cPGa9tms5RvF1kDwkYUkIZECoKNpG7ARzC5vkqMiH3SwbzzT+
	do+ZKf+oH7X8FHbJRHj2ir8S+g==
X-Google-Smtp-Source: ABdhPJzMgcR9UgCsbvPuA/02v4FkJblQW9htJs47A2YRdbvRMs4AQG7O660EDIsBZdZMmGrIa23JJg==
X-Received: by 2002:a17:902:bf42:b0:13d:b79f:a893 with SMTP id u2-20020a170902bf4200b0013db79fa893mr8518959pls.1.1632259793364;
        Tue, 21 Sep 2021 14:29:53 -0700 (PDT)
Date: Tue, 21 Sep 2021 21:29:48 +0000
From: Sean Christopherson <seanjc@google.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>, x86@kernel.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v2 05/13] perf: Force architectures to opt-in to guest
 callbacks
Message-ID: <YUpOzFW3K3iJAoWa@google.com>
References: <20210828003558.713983-1-seanjc@google.com>
 <20210828003558.713983-6-seanjc@google.com>
 <20210828194752.GC4353@worktop.programming.kicks-ass.net>
 <8ee13a69-f2c4-2413-2d6c-b6c0a559286e@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8ee13a69-f2c4-2413-2d6c-b6c0a559286e@redhat.com>

On Tue, Sep 21, 2021, Paolo Bonzini wrote:
> On 28/08/21 21:47, Peter Zijlstra wrote:
> > > +config HAVE_GUEST_PERF_EVENTS
> > > +	bool
> > 	depends on HAVE_KVM
> 
> It won't really do anything, since Kconfig does not detects conflicts
> between select' and 'depends on' clauses.

It does throw a WARN, though the build doesn't fail.

WARNING: unmet direct dependencies detected for HAVE_GUEST_PERF_EVENTS
  Depends on [n]: HAVE_KVM [=n] && HAVE_PERF_EVENTS [=y]
  Selected by [y]:
  - ARM64 [=y]

WARNING: unmet direct dependencies detected for HAVE_GUEST_PERF_EVENTS
  Depends on [n]: HAVE_KVM [=n] && HAVE_PERF_EVENTS [=y]
  Selected by [y]:
  - ARM64 [=y]

WARNING: unmet direct dependencies detected for HAVE_GUEST_PERF_EVENTS
  Depends on [n]: HAVE_KVM [=n] && HAVE_PERF_EVENTS [=y]
  Selected by [y]:
  - ARM64 [=y]

> Rather, should the symbol be selected by KVM, instead of ARM64 and X86?

By KVM, you mean KVM in arm64 and x86, correct?  Because HAVE_GUEST_PERF_EVENTS
should not be selected for s390, PPC, or MIPS.

Oh, and Xen also uses the callbacks on x86, which means the HAVE_KVM part is
arguabably wrong, even though it's guaranteed to be true for the XEN_PV case.
I'll drop that dependency and send out a separate series to clean up the arm64
side of HAVE_KVM.

The reason I didn't bury HAVE_GUEST_PERF_EVENTS under KVM (and XEN_PV) is that
there are number of references to the callbacks throught perf and I didn't want
to create #ifdef hell.

But I think I figured out a not-awful solution.  If there are wrappers+stubs for
the guest callback users, then the new Kconfig can be selected on-demand instead
of unconditionally by arm64 and x86.  That has the added bonus of eliminating
the relevant code paths for !KVM (and !XEN_PV on x86), with or without static_call.
It also obviates the needs for __KVM_WANT_GUEST_PERF_EVENTS or whatever I called
that thing.

It more or less requires defining the static calls in generic perf, but I think
that actually ends up being good thing as it consolidates more code without
introducing more #ifdefs.  The diffstats for the static_call() conversions are
also quite nice.

 include/linux/perf_event.h | 28 ++++++----------------------
 kernel/events/core.c       | 15 +++++++++++++++
 2 files changed, 21 insertions(+), 22 deletions(-)

I'll try to get a new version out today or tomorrow.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 21:34:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 21:34:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191916.342075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnPj-00060n-UU; Tue, 21 Sep 2021 21:34:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191916.342075; Tue, 21 Sep 2021 21:34:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnPj-00060g-Qg; Tue, 21 Sep 2021 21:34:55 +0000
Received: by outflank-mailman (input) for mailman id 191916;
 Tue, 21 Sep 2021 21:34:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSnPi-00060a-Dv
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 21:34:54 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 24f33a68-3de6-4ef2-8838-a85b6ff52eb4;
 Tue, 21 Sep 2021 21:34:52 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6AA8561131;
 Tue, 21 Sep 2021 21:34:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24f33a68-3de6-4ef2-8838-a85b6ff52eb4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632260091;
	bh=bubcWIJ0yTrVSlfj4OdrNlW4r0JbYqS6VNPXSUauRdI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MT8bU6epGmWlrZZZZ3sFZg17praJtFoPsosNLVdPk4MV9RjkNMnj9FqteOlt/X+g3
	 cntNuD+xvfbQLrpz2jT//0hHPEzUgId+qby8qrLnDa+jyf9QHCX8FuEZGf4hc6rNHu
	 GBlXpyLSJXmcavV/852+X4bVCP18NMZ8smbIiJ9cFtZ/RegUiy3ZM1toXrJ2dATUpX
	 Tz66RD6SIKF9gPg/W9Nw1j/nOxB8HD0m6mIyT56exmMIG29wRGbWNWpg1RFYtbashA
	 4fizXuubEiFlsbJ32ZWwDj6hH0n/m3/foaOCyY9hp60C8RF0wiagyyevH5f1+qNSuN
	 wCIw7u84epXig==
Date: Tue, 21 Sep 2021 14:34:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <Luca.Fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
In-Reply-To: <18531A94-ECF7-4840-A515-B161A4ABDEF4@arm.com>
Message-ID: <alpine.DEB.2.21.2109211357280.17979@sstabellini-ThinkPad-T480s>
References: <20210915142602.42862-1-luca.fancellu@arm.com> <20210915142602.42862-3-luca.fancellu@arm.com> <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com> <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com> <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com> <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com> <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s> <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com> <alpine.DEB.2.21.2109171516270.21985@sstabellini-ThinkPad-T480s>
 <18531A94-ECF7-4840-A515-B161A4ABDEF4@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-2057376181-1632257905=:17979"
Content-ID: <alpine.DEB.2.21.2109211358480.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2057376181-1632257905=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109211358481.17979@sstabellini-ThinkPad-T480s>

On Tue, 21 Sep 2021, Luca Fancellu wrote:
> > On 17 Sep 2021, at 23:33, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > On Fri, 17 Sep 2021, Luca Fancellu wrote:
> >>> On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>> 
> >>> On Thu, 16 Sep 2021, Jan Beulich wrote:
> >>>> On 16.09.2021 17:07, Luca Fancellu wrote:
> >>>>> I explain here my understanding on dom0less, this feature is used to start domUs at
> >>>>> Xen boot in parallel, the name is misleading but it doesn’t require dom0 to be absent.
> >>>>> 
> >>>>> So if you have a dom0 kernel embed in the image, it's completely fine to start it and it 
> >>>>> doesn’t have to be skipped.
> >>>>> 
> >>>>> Here the possible user cases:
> >>>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image]
> >>>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded in Xen image, domUs on DT]
> >>>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image) and domUs on DT]
> >>>> 
> >>>> If that's the intention - fine. Stefano?
> >>> 
> >> 
> >> Hi Stefano,
> >> 
> >>> What do you mean by dom0 modules embedded in the Xen image? I am not
> >>> familiar with it, but I imagine it doesn't involve any multiboot,module
> >>> nodes in device tree, right?
> >>> 
> >>> Putting aside "dom0 modules embedded in Xen image" that I didn't fully
> >>> understand, there are three ways to load Dom0:
> >>> 
> >>> - dom0 kernel (and ramdisk, optional) on xen.cfg
> >>> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" property
> >>> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,binary" property
> >> 
> >> True for the #1 and #2, the last one is not implemented. The uefi,binary property
> >> for now is only used to load domU modules.
> > 
> > Yeah, it is no problem that is not currently implemented, but from a
> > device tree binding / efi interface perspective it should be possible.
> > 
> > 
> >>> Then, the other use cases are:
> >>> 
> >>> - true dom0less, domUs on device tree using the "reg" property
> >>> - true dom0less, domUs on device tree using the "uefi,binary" property
> >>> 
> >>> And of course all the possible combinations between Dom0 and DomU
> >>> loading.
> >>> 
> >>> 
> >>> Currently, patch #1 checks for the presence of a Dom0 kernel node and, if
> >>> present, it decides not to proceed with xen.cfg. So if the Dom0 kernel
> >>> node is *not* present, efi_arch_use_config_file returns true.
> >>> 
> >>> However, this could be a true dom0less configuration without any Dom0
> >>> kernel. If so, you might not want to load xen.cfg at all because it is
> >>> not needed. In a true dom0less configuration, we probably want
> >>> efi_arch_use_config_file to return false.
> >> 
> >> In a true dom0less configuration we might need to read xen.cfg to retrieve the
> >> Xen command line, 
> > 
> > The Xen command line could also be on device tree
> > (/chosen/xen,xen-bootargs).
> > 
> > 
> >> but following the actual implementation of the common code
> >> there is more. I’m going to explain.
> >> 
> >> What efi_arch_use_config_file really does is not only choosing to read xen.cfg
> >> or not. Following the common code (xen/common/efi/boot.c) and what its result activate
> >> along the path, it basically decides if the UEFI stub is used as a loader from filesystem
> >> or not. We need the UEFI stub as a loader to be 100% UEFI and load our modules.
> >> 
> >> The original check basically says “if there are multiboot,module in the DT, then some
> >> bootloader has loaded in memory the required modules so I’m not gonna load anything
> >> from the filesystem because I assume it puts everything in place for me to boot.”
> > 
> > OK, I am following. It looks like this is the source of the issue.
> > 
> > 
> >>> From misc/efi.txt:
> >> When booted as an EFI application, Xen requires a configuration file as described below unless a bootloader,
> >> such as GRUB, has loaded the modules and describes them in the device tree provided to Xen. If a bootloader
> >> provides a device tree containing modules then any configuration files are ignored, and the bootloader is
> >> responsible for populating all relevant device tree nodes.
> >> 
> >> What I’m doing in patch #1 is restricting that check to just the multiboot,module that are
> >> Dom0 module, why? Because with the introduction of dom0less we need to specify
> >> multiboot,modules for domUs, but the presence or not of dom0 modules is the only
> >> Information we need to understand if the user decided to start Xen with everything
> >> in places (modules in memory, xen command line, dtb) or if the job is demanded to the
> >> UEFI stub and its configuration file.
> > 
> > I don't think so. Imagine a case where the user has everything in device
> > tree, doesn't need xen.cfg, but dom0 and domUs are specified as
> > uefi,binary.
> > 
> > We don't want xen.cfg but we do want to be able to load files from the
> > filesystem. This might not be currently implemented but from an bindings
> > perspective it should be possible.
> > 
> > 
> >> By the configuration file you can also load in memory the Xen dtb, so Xen can
> >> be started as an EFI application without the DTB and then load it using the EFI stub.
> > 
> > This can be very useful but it would follow the !fdt check and return
> > true from efi_arch_use_config_file. So it doesn't really conflict with
> > anything we would around multiboot,module and xen,cfg-loading.
> > 
> > 
> >> I’m not against this new property “xen,cfg-loading”, I just think it is not needed because
> >> we have all the information we need without it and in any case we need to read the
> >> configuration file because otherwise we won’t have access to the Xen command line.
> > 
> > We don't necessarely need to read the Xen command line from xen.cfg :-)
> > 
> > 
> >> Now I’m going to show you examples of all use cases that are valid with the introduction
> >> of this serie:
> >> 
> >> 1) Start Xen as EFI application and load only Dom0
> >> 
> >>    Xen.cfg:
> >>    [global]
> >>    default=xen_dom0
> >> 
> >>    [xen_dom0]
> >>    options=<Xen command line>
> >>    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> >>    ramdisk=initrd-3.0.31-0.4-xen
> >>    dtb=<xen DTB>
> >> 
> >>    DT:
> >>    {no modification}
> >> 
> >> 2) Start Xen as EFI application to load a true dom0less setup
> >> 
> >>    Xen.cfg:
> >>    [global]
> >>    default=xen_true_dom0less
> >> 
> >>    [xen_true_dom0less]
> >>    options=<Xen command line>
> >>    dtb=<xen DTB>
> >> 
> >>    DT:
> >>    chosen {
> >>        #size-cells = <0x1>;
> >> 	#address-cells = <0x1>;
> >> 
> >> 	domU1 {
> >>            #size-cells = <0x1>; 
> >>            #address-cells = <0x1>;
> >>            compatible = "xen,domain”;
> >>            cpus = <1>;
> >>            memory = <0 0xC0000>;
> >> 
> >>            module@1 {
> >>                compatible = "multiboot,kernel", "multiboot,module”;
> >>                bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
> >>                uefi,binary = “domU_kernel1.bin"
> >>            };
> >> 
> >>            module@2 {
> >>                compatible = “multiboot,ramdisk", "multiboot,module”;
> >>                uefi,binary = “domU_ramdisk1.bin"
> >>            };
> >> 
> >>            module@3 {
> >>                compatible = "multiboot,device-tree", "multiboot,module”;
> >>                uefi,binary = “domU_passthrough1.dtb"
> >>            }; 
> >>        };
> >> 
> >>        domU2 { <as above> };
> >>    }
> >> 
> >> 3) Start Xen as EFI application to load Dom0 and DomUs
> >> 
> >>    Xen.cfg:
> >>    [global]
> >>    default=xen_dom0_domUs
> >> 
> >>    [xen_dom0_domUs]
> >>    options=<Xen command line>
> >>    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> >>    ramdisk=initrd-3.0.31-0.4-xen
> >>    dtb=<xen DTB>
> >> 
> >>    DT:
> >>    chosen {
> >>        #size-cells = <0x1>;
> >> 	#address-cells = <0x1>;
> >> 
> >> 	domU1 {
> >>            #size-cells = <0x1>; 
> >>            #address-cells = <0x1>;
> >>            compatible = "xen,domain”;
> >>            cpus = <1>;
> >>            memory = <0 0xC0000>;
> >> 
> >>            module@1 {
> >>                compatible = "multiboot,kernel", "multiboot,module”;
> >>                bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
> >>                uefi,binary = “domU_kernel1.bin"
> >>            };
> >> 
> >>            module@2 {
> >>                compatible = “multiboot,ramdisk", "multiboot,module”;
> >>                uefi,binary = “domU_ramdisk1.bin"
> >>            };
> >> 
> >>            module@3 {
> >>                compatible = "multiboot,device-tree", "multiboot,module”;
> >>                uefi,binary = “domU_passthrough1.dtb"
> >>            }; 
> >>        };
> >> 
> >>        domU2 { <as above> };
> >>    }
> >> 
> >> So as you see every case is covered without the introduction of the
> >> property.
> >> 
> >> Please let me know what do you think.
> > 
> 
> Hi Stefano,
> 
> > I think that from an interface perspective (not a code perspective) we
> > need to be able to account for cases like:
> > 
> > 4) Start Xen as EFI application and load only Dom0
> > no Xen.cfg
> > DT:
> >  xen,xen-bootargs
> >  dom0/uefi,binary
> >  domUs/uefi,binary
> > 
> > 
> > But in any case, even disregarding this case, past experience has taught
> > me that it is always better to have an explicit flag to trigger a new
> > behavior, rather than relying on "guesswork". If we introduce
> > "xen,cfg-loading", we are going to be a lot more future-proof that if we
> > don't introduce it in terms of backward and forward compatibility in
> > case we need to change anything.
> 
> I see your point, for sure the DT is a more powerful tool than the simple
> text configuration file and it would be the best interface.
> However I think we are moving into the direction where x86 and arm
> are going to diverge even if we can have a common interface for them
> (the configuration file).

Consider that the configuration file is not the only interface to Xen
any longer. There is also HyperLaunch and I was trying to align to it:
https://marc.info/?l=xen-devel&m=162096368626246 The DT-based approached
would be very well aligned with HyperLaunch.


> For that reason I’m asking if you would be willing to accept a solution
> where we introduce a new keyword in the configuration file:
> 
> dom0less=<dtb> OR domu_guests=<dtb> OR I’m open to suggestion.
> 
> Where the pointed dtb contains the domU domains:
> 
> /dts-v1/;
> 
> / {
>     /* #*cells are here to keep DTC happy */
>     #address-cells = <2>;
>     #size-cells = <2>;
> 
>     domU1 {
>            #size-cells = <0x1>; 
>            #address-cells = <0x1>;
>            compatible = "xen,domain”;
>            cpus = <1>;
>            memory = <0 0xC0000>;
> 
>            module@1 {
>                compatible = "multiboot,kernel", "multiboot,module”;
>                bootargs = "console=ttyAMA0 root=/dev/ram0 rw”;
>                uefi,binary = “domU_kernel1.bin"
>            };
> 
>            module@2 {
>                compatible = “multiboot,ramdisk", "multiboot,module”;
>                uefi,binary = “domU_ramdisk1.bin"
>            };
> 
>            module@3 {
>                compatible = "multiboot,device-tree", "multiboot,module”;
>                uefi,binary = “domU_passthrough1.dtb"
>            }; 
>     };
> 
>     domU2 { <as above> };
> 
> };
> 
> 
> So that the user cases we discussed are valid:
> 
> 1) Start Xen and load Dom0:
>    
>    Xen.cfg:
>    [global]
>    default=xen
> 
>    [xen]
>    options=<Xen command line>
>    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>    ramdisk=initrd-3.0.31-0.4-xen
>    dtb=<xen DTB>
> 
> 2) Start Xen and load only domUs (true dom0less)
> 
>    Xen.cfg:
>    [global]
>    default=xen
> 
>    [xen]
>    options=<Xen command line>
>    dom0less=<dom0less DTB>
>    dtb=<xen DTB>
> 
> 3) Start Xen and load Dom0 and DomUs
> 
>    Xen.cfg:
>    [global]
>    default=xen
> 
>    [xen]
>    options=<Xen command line>
>    kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>    ramdisk=initrd-3.0.31-0.4-xen
>    dom0less=<dom0less DTB>
>    dtb=<xen DTB>
> 
> 
> With this change we will be consistent across x86 and arm UEFI boot
> start procedure, we won’t touch the current check on multiboot,modules
> because it will be valid, we will have a way to boot dom0less and it
> requires less testing for the changing in the common code.
> 
> Please let me know what do you think about that.

My order of preference from best to worst is:
1) my previous suggestion, e.g. xen,cfg-loading
2) your previous suggestion, e.g. change the multiboot,modules check
   without adding xen,cfg-loading
3) this proposal


Let me explain the reason behind this. This proposal still requires a
device tree but it has to be loaded from the config file, which is
limiting compared to the other approaches. From a user perspective is
just as complex (just as difficult to write) but less flexible because
it prevents using the device tree alone for UEFI booting in the future.
Given that with the two other alternatives the config file could still
be used anyway, I don't think that adding the "dom0less" parameters to
the config file buys us much in terms of alignment with x86. This is
why this is my least favorite option.

Your previous approach is actually quite good. Same complexity but much
more flexible. Similar alignment with x86 in my view. The only
correction I suggested is the addition of xen,cfg-loading to make the
efi_arch_use_config_file check more robust. However, I still prefer your
prevous approach even without xen,cfg-loading.


Let me make one more suggestion based on your previous approach (I mean
this version of the patch series):

- You have already removed the panic for ARM in case a dom0 kernel is
  not specifid in patch #2. We can go farther than that and make the
  absence of xen.cfg not a fatal failure on ARM because it is fine not
  to have dom0 in true dom0less configurations and the xen cmdline is
  optional anyway.

- If the absence of xen.cfg is not a fatal failure, then we don't need
  efi_arch_use_config_file anymore. Let's try to load xen.cfg always. We
  error out if xen.cfg specifies a dom0 kernel and we already have a
  dom0 kernel in DT.

- In the future a "don't load xen.cfg" option (the opposite of
  xen,cfg-loading) could be added but it is not necessary now


This should be a minimal change compared to this version of the patch
series, enable all the use-cases you care about, and also be more
flexible for the future.
--8323329-2057376181-1632257905=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 21:38:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 21:38:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191924.342085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnTW-0006qg-Iu; Tue, 21 Sep 2021 21:38:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191924.342085; Tue, 21 Sep 2021 21:38:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnTW-0006qZ-FT; Tue, 21 Sep 2021 21:38:50 +0000
Received: by outflank-mailman (input) for mailman id 191924;
 Tue, 21 Sep 2021 21:38:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSnTU-0006qT-HK
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 21:38:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4d5b4242-1b24-11ec-b936-12813bfff9fa;
 Tue, 21 Sep 2021 21:38:47 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9EF5D61159;
 Tue, 21 Sep 2021 21:38:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d5b4242-1b24-11ec-b936-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632260326;
	bh=fkRPSYovAxIu4VtHOAXk2ccKoJ7CG5f+c6hivf+rAto=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fc+LvZeDBql1S1SHhQk0mVmQ0oICimrV20oOpO3ZDPLfsB/ekCGFQ0P8UVqP9Cgj4
	 oLj/kQWMborgnnmRVIzlccKKseQG6J+Ifvb3h/KGO0kPOrww2lJrUBPll3mBZBjGvm
	 mIHkzyVdlw1/OYjx6MN16sv2zHTHJwwBqbtvOLtLpruVp2XLbX78ddGTo3f4FjKS2b
	 CNvuVTRDdkcYWM5HnF8Bd2m0f+RVOoqZXpVGVByRKyr/EdNHW5ysd1zh/u46IuVq7I
	 WqlQ+Ru6uazeF5JjMrz8fR3Rjx5qqCU72kWv4s27393sPibMv4e4Nk120UcmbdHdGn
	 xxvUPnsItFLDg==
Date: Tue, 21 Sep 2021 14:38:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Jan Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
In-Reply-To: <996263D9-27C3-43FE-A111-4BD2CE75B765@arm.com>
Message-ID: <alpine.DEB.2.21.2109211437420.17979@sstabellini-ThinkPad-T480s>
References: <cover.1629366665.git.rahul.singh@arm.com> <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109091720490.10523@sstabellini-ThinkPad-T480s> <A90145B0-DD30-4300-8248-73104C782172@arm.com>
 <alpine.DEB.2.21.2109161324200.21985@sstabellini-ThinkPad-T480s> <996263D9-27C3-43FE-A111-4BD2CE75B765@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-830528443-1632260326=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-830528443-1632260326=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 21 Sep 2021, Rahul Singh wrote:
> Hi Stefano,
> 
> > On 16 Sep 2021, at 9:26 pm, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 16 Sep 2021, Rahul Singh wrote:
> >> Hi Stefano,
> >> 
> >>> On 10 Sep 2021, at 1:26 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>> 
> >>> On Thu, 19 Aug 2021, Rahul Singh wrote:
> >>>> The existing VPCI support available for X86 is adapted for Arm.
> >>>> When the device is added to XEN via the hyper call
> >>>> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> >>>> access is added to the Xen to emulate the PCI devices config space.
> >>> 
> >>> This is done just for device discovery, right?
> >>> 
> >>> Although it is currently not implemented (and I am not asking to
> >>> implement it now, I am only trying to understand the architecture), it
> >>> would be possible to discover all PCI devices just by walking down the
> >>> PCI hierarchy by ourselves in Xen (no Dom0 interactions) given that we
> >>> have an ECAM driver.
> >>> 
> >>> I take that would be the way to implement PCI support for Dom0less?
> >> 
> >> It is not possible to discover PCI devices in XEN if enumeration is not done before XEN boot.
> >> If boot firmware did the enumeration,  XEN will discover the PCI device.
> > 
> > OK, but if the boot firmware does the enumeration, how will Xen discover
> > the PCI device exactly? Will Xen discover it because the PCI device will
> > be present in device tree explicitly (will have its own device tree
> > node)? Or will Xen discover it by walking the PCI bus?
> 
> Yes Xen discover it by walking the PCI bus. There is already a function in XEN scan_pci_devices() that we can use 
> to discover the PCI device for dom0less system.

OK. Please add a statement to the commit message on how dom0less support
could be implemented. It doesn't have to be implemented now of course
but it would be good to keep a note for future reference.
--8323329-830528443-1632260326=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 21:41:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 21:41:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191938.342101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnWJ-0008G5-1Q; Tue, 21 Sep 2021 21:41:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191938.342101; Tue, 21 Sep 2021 21:41:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnWI-0008Fy-UP; Tue, 21 Sep 2021 21:41:42 +0000
Received: by outflank-mailman (input) for mailman id 191938;
 Tue, 21 Sep 2021 21:41:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSnWI-0008Fo-BO; Tue, 21 Sep 2021 21:41:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSnWH-0008VF-Kg; Tue, 21 Sep 2021 21:41:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSnWH-0008KN-BM; Tue, 21 Sep 2021 21:41:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSnWH-0006Mc-As; Tue, 21 Sep 2021 21:41:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hh2KgkEsZuI9K3OWnMqL8T6mX82MtTVrix4Qn6K3Qcs=; b=gwSo8/6+/N6fcNqDplwQKMcsi5
	EOSlJyZ9F2Nzrh4R3RboMzv/yJWblx8gP8uuHtiLs+c5//0mbsGzOgUmzqobtQJKzCmStatDMjNOn
	LDTP9Bmt03+Ldmx6g1PRzG2VxJVPofFqJ4QxWPZVUjRPjpugD4SpYOPwGkqoSrGIVLx8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165140-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165140: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d9fb678414c048e185eaddadd18d75f5e8832ff3
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 21:41:41 +0000

flight 165140 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165140/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-amd64-amd64-examine      4 memdisk-try-append       fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                d9fb678414c048e185eaddadd18d75f5e8832ff3
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  417 days
Failing since        152366  2020-08-01 20:49:34 Z  416 days  726 attempts
Testing same since   165140  2021-09-21 12:14:51 Z    0 days    1 attempts

------------------------------------------------------------
7322 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263466 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 22:00:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 22:00:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191948.342115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnoU-0002Ny-Ff; Tue, 21 Sep 2021 22:00:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191948.342115; Tue, 21 Sep 2021 22:00:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnoU-0002Nr-BE; Tue, 21 Sep 2021 22:00:30 +0000
Received: by outflank-mailman (input) for mailman id 191948;
 Tue, 21 Sep 2021 22:00:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSnoT-0002Nl-9n
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 22:00:29 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5427310a-1b27-11ec-b937-12813bfff9fa;
 Tue, 21 Sep 2021 22:00:27 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7C07761159;
 Tue, 21 Sep 2021 22:00:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5427310a-1b27-11ec-b937-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632261626;
	bh=nndPsp0ITq6FLrs3FnqBUmyJ6OAUaTg007lc1X5tdK0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=PygSAkVz2sC/8NZPC/Oyb9+sNVX6r8+/L6KuFhLbgzy+UvBjFp3Rea42suDUVfi5N
	 FjnsGp7Pes/EJwPw4O1mNbgZ3vixjeO6ugzx8LkEr/G39nhynxom5m364aE+6Q8WCc
	 t8dWT0MTeIkwAJMh4HBmB8krpzYMkX6pPd9I26SRty/VpvRy+CHG3K//oUzFlWqBqE
	 IkqPVsgNxr13oPRclnTxxCMfEho9YKQV/YF2+28hkK/nfJCoZVSyaRLeSfP3UXTAW3
	 t0WgeNh135OVCD1T1Vas46N0t28rj1l8Dn8eaLFJGedDLSyngCzaRMIDvBPy7wagMG
	 uk8TCRIG0sq6Q==
Date: Tue, 21 Sep 2021 15:00:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, 
    fnuv@xilinx.com
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
Message-ID: <alpine.DEB.2.21.2109211439370.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s> <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org> <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com> <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
 <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-988134037-1632260905=:17979"
Content-ID: <alpine.DEB.2.21.2109211448520.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-988134037-1632260905=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109211448521.17979@sstabellini-ThinkPad-T480s>

On Tue, 21 Sep 2021, Oleksandr wrote:
> On 21.09.21 02:21, Stefano Stabellini wrote:
> > On Sun, 19 Sep 2021, Oleksandr wrote:
> > > > On 18/09/2021 03:37, Stefano Stabellini wrote:
> > > > > On Fri, 17 Sep 2021, Stefano Stabellini wrote:
> > > > > > On Fri, 17 Sep 2021, Oleksandr wrote:
> > > > > > > > > +
> > > > > > > > > +    dt_dprintk("Find unallocated memory for extended
> > > > > > > > > regions\n");
> > > > > > > > > +
> > > > > > > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > > > > > > +    if ( !unalloc_mem )
> > > > > > > > > +        return -ENOMEM;
> > > > > > > > > +
> > > > > > > > > +    /* Start with all available RAM */
> > > > > > > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > > > > > > +    {
> > > > > > > > > +        start = bootinfo.mem.bank[i].start;
> > > > > > > > > +        end = bootinfo.mem.bank[i].start +
> > > > > > > > > bootinfo.mem.bank[i].size - 1;
> > > > > > > > > +        res = rangeset_add_range(unalloc_mem, start, end);
> > > > > > > > > +        if ( res )
> > > > > > > > > +        {
> > > > > > > > > +            printk(XENLOG_ERR "Failed to add:
> > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > +                   start, end);
> > > > > > > > > +            goto out;
> > > > > > > > > +        }
> > > > > > > > > +    }
> > > > > > > > > +
> > > > > > > > > +    /* Remove RAM assigned to Dom0 */
> > > > > > > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > > > > > > +    {
> > > > > > > > > +        start = assign_mem->bank[i].start;
> > > > > > > > > +        end = assign_mem->bank[i].start +
> > > > > > > > > assign_mem->bank[i].size - 1;
> > > > > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > > > +        if ( res )
> > > > > > > > > +        {
> > > > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > +                   start, end);
> > > > > > > > > +            goto out;
> > > > > > > > > +        }
> > > > > > > > > +    }
> > > > > > > > > +
> > > > > > > > > +    /* Remove reserved-memory regions */
> > > > > > > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > > > > > > > +    {
> > > > > > > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > > > > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > > > > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > > > > > > +        res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > > > +        if ( res )
> > > > > > > > > +        {
> > > > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > +                   start, end);
> > > > > > > > > +            goto out;
> > > > > > > > > +        }
> > > > > > > > > +    }
> > > > > > > > > +
> > > > > > > > > +    /* Remove grant table region */
> > > > > > > > > +    start = kinfo->gnttab_start;
> > > > > > > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > > > > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > > > +    if ( res )
> > > > > > > > > +    {
> > > > > > > > > +        printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > +               start, end);
> > > > > > > > > +        goto out;
> > > > > > > > > +    }
> > > > > > > > > +
> > > > > > > > > +    start = EXT_REGION_START;
> > > > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > > > > > > +                                 add_ext_regions,
> > > > > > > > > ext_regions);
> > > > > > > > > +    if ( res )
> > > > > > > > > +        ext_regions->nr_banks = 0;
> > > > > > > > > +    else if ( !ext_regions->nr_banks )
> > > > > > > > > +        res = -ENOENT;
> > > > > > > > > +
> > > > > > > > > +out:
> > > > > > > > > +    rangeset_destroy(unalloc_mem);
> > > > > > > > > +
> > > > > > > > > +    return res;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static int __init find_memory_holes(const struct kernel_info
> > > > > > > > > *kinfo,
> > > > > > > > > +                                    struct meminfo
> > > > > > > > > *ext_regions)
> > > > > > > > > +{
> > > > > > > > > +    struct dt_device_node *np;
> > > > > > > > > +    struct rangeset *mem_holes;
> > > > > > > > > +    paddr_t start, end;
> > > > > > > > > +    unsigned int i;
> > > > > > > > > +    int res;
> > > > > > > > > +
> > > > > > > > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > > > > > > > +
> > > > > > > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > > > > > > +    if ( !mem_holes )
> > > > > > > > > +        return -ENOMEM;
> > > > > > > > > +
> > > > > > > > > +    /* Start with maximum possible addressable physical
> > > > > > > > > memory
> > > > > > > > > range */
> > > > > > > > > +    start = EXT_REGION_START;
> > > > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > > > > > > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > > > > > > +    if ( res )
> > > > > > > > > +    {
> > > > > > > > > +        printk(XENLOG_ERR "Failed to add:
> > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > +               start, end);
> > > > > > > > > +        goto out;
> > > > > > > > > +    }
> > > > > > > > > +
> > > > > > > > > +    /* Remove all regions described by "reg" property (MMIO,
> > > > > > > > > RAM,
> > > > > > > > > etc) */
> > > > > > > > Well... The loop below is not going to handle all the regions
> > > > > > > > described in
> > > > > > > > the property "reg". Instead, it will cover a subset of "reg"
> > > > > > > > where
> > > > > > > > the
> > > > > > > > memory is addressable.
> > > > > > > As I understand, we are only interested in subset of "reg" where
> > > > > > > the
> > > > > > > memory is
> > > > > > > addressable.
> > > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > You will also need to cover "ranges" that will describe the BARs
> > > > > > > > for
> > > > > > > > the PCI
> > > > > > > > devices.
> > > > > > > Good point.
> > > > > > Yes, very good point!
> > > > > > 
> > > > > > 
> > > > > > > Could you please clarify how to recognize whether it is a PCI
> > > > > > > device as long as PCI support is not merged? Or just to find any
> > > > > > > device nodes
> > > > > > > with non-empty "ranges" property
> > > > > > > and retrieve addresses?
> > > > > > Normally any bus can have a ranges property with the aperture and
> > > > > > possible address translations, including /amba (compatible =
> > > > > > "simple-bus"). However, in these cases dt_device_get_address already
> > > > > > takes care of it, see
> > > > > > xen/common/device_tree.c:dt_device_get_address.
> > > > > > 
> > > > > > The PCI bus is special for 2 reasons:
> > > > > > - the ranges property has a different format
> > > > > > - the bus is hot-pluggable
> > > > > > 
> > > > > > So I think the only one that we need to treat specially is PCI.
> > > > > > 
> > > > > > As far as I am aware PCI is the only bus (or maybe just the only bus
> > > > > > that we support?) where ranges means the aperture.
> > > > > Now that I think about this, there is another "hotpluggable" scenario
> > > > > we
> > > > > need to think about:
> > > > > 
> > > > > [1] https://marc.info/?l=xen-devel&m=163056546214978
> > > > > 
> > > > > Xilinx devices have FPGA regions with apertures currently not
> > > > > described
> > > > > in device tree, where things can programmed in PL at runtime making
> > > > > new
> > > > > devices appear with new MMIO regions out of thin air.
> > > > > 
> > > > > Now let me start by saying that yes, the entire programmable region
> > > > > aperture could probably be described in device tree, however, in
> > > > > reality it is not currently done in any of the device trees we use
> > > > > (including the upstream device trees in linux.git).
> > > > This is rather annoying, but not unheard. There are a couple of
> > > > platforms
> > > > where the MMIOs are not fully described in the DT.
> > > > 
> > > > In fact, we have a callback 'specific_mappings' which create additional
> > > > mappings (e.g. on the omap5) for dom0.
> > > > 
> > > > > So, we have a problem :-(
> > > > > 
> > > > > 
> > > > > I can work toward getting the right info on device tree, but in
> > > > > reality
> > > > > that is going to take time and for now the device tree doesn't have
> > > > > the
> > > > > FPGA aperture in it. So if we accept this series as is, it is going to
> > > > > stop features like [1] from working. >
> > > > > If we cannot come up with any better plans, I think it would be better
> > > > > to drop find_memory_holes, only rely on find_unallocated_memory even
> > > > > when the IOMMU is on. One idea is that we could add on top of the
> > > > > regions found by find_unallocated_memory any MMIO regions marked as
> > > > > xen,passthrough: they are safe because they are not going to dom0
> > > > > anyway.
> > > > (Oleksandr, it looks like some rationale about the different approach is
> > > > missing in the commit message. Can you add it?)
> > > Yes sure, but let me please clarify what is different approach in this
> > > context. Is it to *also* take into the account MMIO regions of the devices
> > > for
> > > passthrough for case when IOMMU is off (in addition to unallocated
> > > memory)? If
> > > yes, I wonder whether we will gain much with that according to that
> > > device's
> > > MMIO regions are usually not big enough and we stick to allocate extended
> > > regions with bigger size (> 64MB).
> > That's fair enough. There are a couple of counter examples where the
> > MMIO regions for the device to assign are quite large, for instance a
> > GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
> > but maybe they are not that common. I am not sure if it is worth
> > scanning the tree for xen,passthrough regions every time at boot for
> > this.
> 
> ok, I will add a few sentences to commit message about this different approach
> for now. At least this could be implemented later on if there is a need.

One thing that worries me about this is that if we take an old Xen with
this series and run it on a new board, it might cause problems. At the
very least [1] wouldn't work.

Can we have a Xen command line argument to disable extended regions as
an emergecy toggle?


[1] https://marc.info/?l=xen-devel&m=163056546214978
--8323329-988134037-1632260905=:17979--


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 22:07:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 22:07:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191956.342126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnvQ-0003F8-B8; Tue, 21 Sep 2021 22:07:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191956.342126; Tue, 21 Sep 2021 22:07:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSnvQ-0003F1-8D; Tue, 21 Sep 2021 22:07:40 +0000
Received: by outflank-mailman (input) for mailman id 191956;
 Tue, 21 Sep 2021 22:07:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zi6V=OL=google.com=jwerner@srs-us1.protection.inumbo.net>)
 id 1mSnvP-0003Ev-3w
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 22:07:39 +0000
Received: from mail-qk1-x733.google.com (unknown [2607:f8b0:4864:20::733])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 517d7eb2-9cc8-4622-8457-8ef37eae907c;
 Tue, 21 Sep 2021 22:07:38 +0000 (UTC)
Received: by mail-qk1-x733.google.com with SMTP id bk29so2515013qkb.8
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 15:07:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 517d7eb2-9cc8-4622-8457-8ef37eae907c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=chromium.org; s=google;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=dmNoREXEsTQk/U8Gj8hxJgkwpIFQVacQIWPFX9WwIhE=;
        b=BmvKqba0wk3fHOJkzodZkWkwcRiEGryCgimWoPwoR/3fuUIA+dSginnReDiVPyee7m
         d0wKi27PG+BRcSjcBjJGq+L+WHKcqjtrvpWTSmYtPhmc8evGnsfH/85hixjGh0cAWjdh
         GtyZsqwLpWRGBoTJ8GV2gLHI8D56ItDf8ldGs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=dmNoREXEsTQk/U8Gj8hxJgkwpIFQVacQIWPFX9WwIhE=;
        b=xQkkRV3+3hOtalMUbcoAbrfECQZcuej9GXNClbrTIShzbkPMM9gQl7juBQ6pcVoRpI
         o+50E0CanfV8jqWuEfTrN5vr7Wb+ZG7gc/KiPNJHhM27/PAES92yOo1L/v1swgeK3CVu
         iIVnptONH/07e0qD96AprKJk55/34bHPiGHWKoE2T2pKm2U3H6t0GPhhMInOolCrtTrp
         cB6S34hhutfeN8tgvta7vVEQLfOqdlrukNVOlcZFucxS/0QN3PV1cyZACc0SiCssLbqZ
         jmqywOAnaJwFit8gEaEETYNRq8akCi3D04HnpO8R+kXFb81x1iH10BgUfBFpAqWeTktG
         QSVw==
X-Gm-Message-State: AOAM533XSghzURlH7AL33BoBXbaFU+FY7f/kdlzSsYl+L5jT7mWTEsr0
	S4IgLZZl91J9uzITw3/9GArgxpIyMT0TdqmmZmHdww==
X-Google-Smtp-Source: ABdhPJyq8q/2GRWoSd9IVWD2qjnKGMJZedg7wb9//OSKY3U1RobmMvJLmjUnkfW/rF/kZo95fKOmy8XVlElLbOkvjc8=
X-Received: by 2002:a25:afcd:: with SMTP id d13mr41652428ybj.504.1632262057588;
 Tue, 21 Sep 2021 15:07:37 -0700 (PDT)
MIME-Version: 1.0
References: <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>
In-Reply-To: <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>
From: Julius Werner <jwerner@chromium.org>
Date: Tue, 21 Sep 2021 15:07:25 -0700
Message-ID: <CAODwPW-bbHp3Nxbr9VRt0hZMX0xRnwbGb07PS=8uysXEKFs61w@mail.gmail.com>
Subject: Re: [SPECIFICATION RFC v3] The firmware and bootloader log specification
To: Alec Brown <alec.r.brown@oracle.com>
Cc: "coreboot@coreboot.org" <coreboot@coreboot.org>, "grub-devel@gnu.org" <grub-devel@gnu.org>, 
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
	"systemd-devel@lists.freedesktop.org" <systemd-devel@lists.freedesktop.org>, 
	"trenchboot-devel@googlegroups.com" <trenchboot-devel@googlegroups.com>, 
	"u-boot@lists.denx.de" <u-boot@lists.denx.de>, "x86@kernel.org" <x86@kernel.org>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
	Aleksandr Burmashev <alexander.burmashev@oracle.com>, 
	"allen.cryptic@gmail.com" <allen.cryptic@gmail.com>, 
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
	"andy.shevchenko@gmail.com" <andy.shevchenko@gmail.com>, "ardb@kernel.org" <ardb@kernel.org>, 
	"btrotter@gmail.com" <btrotter@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, 
	"dpsmith@apertussolutions.com" <dpsmith@apertussolutions.com>, Eric DeVolder <eric.devolder@oracle.com>, 
	Eric Snowberg <eric.snowberg@oracle.com>, 
	"frowand.list@gmail.com" <frowand.list@gmail.com>, "hpa@zytor.com" <hpa@zytor.com>, 
	"hun@n-dimensional.de" <hun@n-dimensional.de>, "james.dutton@gmail.com" <james.dutton@gmail.com>, 
	"javierm@redhat.com" <javierm@redhat.com>, Joao Martins <joao.m.martins@oracle.com>, 
	"jwerner@chromium.org" <jwerner@chromium.org>, Kanth Ghatraju <kanth.ghatraju@oracle.com>, 
	Konrad Wilk <konrad.wilk@oracle.com>, 
	"krystian.hebel@3mdeb.com" <krystian.hebel@3mdeb.com>, "leif@nuviainc.com" <leif@nuviainc.com>, 
	"lukasz.hawrylko@intel.com" <lukasz.hawrylko@intel.com>, "luto@amacapital.net" <luto@amacapital.net>, 
	"michal.zygowski@3mdeb.com" <michal.zygowski@3mdeb.com>, "mjg59@google.com" <mjg59@google.com>, 
	"mtottenh@akamai.com" <mtottenh@akamai.com>, "nico.h@gmx.de" <nico.h@gmx.de>, 
	"phcoder@gmail.com" <phcoder@gmail.com>, "piotr.krol@3mdeb.com" <piotr.krol@3mdeb.com>, 
	"pjones@redhat.com" <pjones@redhat.com>, "pmenzel@molgen.mpg.de" <pmenzel@molgen.mpg.de>, 
	"rasmus.villemoes@prevas.dk" <rasmus.villemoes@prevas.dk>, 
	"rdunlap@infradead.org" <rdunlap@infradead.org>, "roger.pau@citrix.com" <roger.pau@citrix.com>, 
	Ross Philipson <ross.philipson@oracle.com>, "sjg@chromium.org" <sjg@chromium.org>, 
	"trini@konsulko.com" <trini@konsulko.com>, 
	"tyhicks@linux.microsoft.com" <tyhicks@linux.microsoft.com>, 
	"ulrich.windl@rz.uni-regensburg.de" <ulrich.windl@rz.uni-regensburg.de>, 
	"wvervoorn@eltan.com" <wvervoorn@eltan.com>, "xypron.glpk@gmx.de" <xypron.glpk@gmx.de>, 
	"rharwood@redhat.com" <rharwood@redhat.com>
Content-Type: text/plain; charset="UTF-8"

> Since it doesn't seem possible to have each boot component using the same log
> format, we added a log_format and log_phys_addr fields to give flexibility in
> how logs are stored. An example of a different log format that can be used is
> the cbmem_console log format used by coreboot:

I am not exactly sure how you expect this interoperability you seem to
be suggesting here to work. Are you saying that your bf_log_header can
sometimes point to the bf_log_buffer structure you define, and
sometimes to a coreboot CBMEM console buffer? But that's a completely
different format that requires a different reader implementation, how
is that supposed to work? If this proposal is just "we define a
wrapper structure that points to everyone's custom firmware log
implementation", then I don't really see the point (the only benefit
still left then might be discovery of the log buffer, but that's the
part you leave open in your design while all those other
implementations already have working discovery mechanisms of their own
anyway).

For the other structures you have defined, the same feedback that I
think was already mentioned on the last iteration of this thread still
applies: it seems incredibly bloated for a simple firmware logging
mechanism. You have a whooping 24+n bytes of overhead *per line* which
probably comes out to somewhere between 30-50% of total space wasted
on overhead for the average log buffer. I guess there are just
fundamentally different opinions on how featureful a firmware log
mechanism needs to be so we're probably not gonna find a format that
makes everyone happy here, but at least for the coreboot project I see
little reason for us to implement something like this when we already
have a well-working existing solution with tooling and wideranged
support.


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 22:29:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 22:29:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191962.342137 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSoGo-0005mx-6F; Tue, 21 Sep 2021 22:29:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191962.342137; Tue, 21 Sep 2021 22:29:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSoGo-0005mq-35; Tue, 21 Sep 2021 22:29:46 +0000
Received: by outflank-mailman (input) for mailman id 191962;
 Tue, 21 Sep 2021 22:29:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSoGn-0005mg-1S; Tue, 21 Sep 2021 22:29:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSoGm-0000sK-RX; Tue, 21 Sep 2021 22:29:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSoGm-0002ii-Ji; Tue, 21 Sep 2021 22:29:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSoGm-0005ot-JE; Tue, 21 Sep 2021 22:29:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8H5CY55EXdi8XrPERVMLLEudY8aC84D5QZZRO/Lu3Oc=; b=Vk2Zccv8J3N68PLtiJQM3knGvb
	1cw/0SL6EnQmVzpIG9KHUzuwss3jhuSy5K49H0b3ycnC9NOiI5BjpSR1DqjG86WHzQHMSCgfVrvN3
	K0EqOPuA5yWsQxPddIfjORQ9/RlnhCfEZelsd0IHeqYhvFSRrCpwqqRHXjHEbmTMrpPg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165143-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165143: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e467d390eaf6736377766adae0166711866fac3
X-Osstest-Versions-That:
    xen=a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 21 Sep 2021 22:29:44 +0000

flight 165143 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165143/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e467d390eaf6736377766adae0166711866fac3
baseline version:
 xen                  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1

Last test of basis   165128  2021-09-20 09:01:40 Z    1 days
Testing same since   165143  2021-09-21 19:01:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a3dd33e63c..4e467d390e  4e467d390eaf6736377766adae0166711866fac3 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 23:22:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 23:22:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191971.342151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSp5e-0003Yf-6x; Tue, 21 Sep 2021 23:22:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191971.342151; Tue, 21 Sep 2021 23:22:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSp5e-0003YY-3t; Tue, 21 Sep 2021 23:22:18 +0000
Received: by outflank-mailman (input) for mailman id 191971;
 Tue, 21 Sep 2021 23:22:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=uuut=OL=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mSp5c-0003YS-Cl
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 23:22:16 +0000
Received: from mail-pg1-x534.google.com (unknown [2607:f8b0:4864:20::534])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0c30602c-60ba-4c11-b2c4-78c92b9670de;
 Tue, 21 Sep 2021 23:22:15 +0000 (UTC)
Received: by mail-pg1-x534.google.com with SMTP id w8so677547pgf.5
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 16:22:15 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id gp11sm3483849pjb.2.2021.09.21.16.22.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 21 Sep 2021 16:22:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c30602c-60ba-4c11-b2c4-78c92b9670de
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=SrgpsGNJ8rqjyV4/EJo88O9IIQbG0UxIB/O/sCFEHCQ=;
        b=Aek6S4Ex4HL+FlRX8wezQ0nSn8PZPLVSwGFM6Tuf5Vb5PRgM0t3QLU9WUHFeTsdzAU
         QtJBZLxCtSTzHjvsrBDZHuAtRs8M/UcTtn16UVPyICkA/9SmSo6/wgDRXNjilfk6+22r
         W51t+bozYpGIBCS76o+pBdu3BTXYTvAYrWg54kKw/zUklVknkRQaSR+HYh+NPP+yi8y9
         Yb/gjfeOPPT4vB//YCwqd+j/uNNgvnBzGOO9gtxuEZPi7zBRKotpsZHqTtUaIjyqVYgT
         sIUYBMdsmRG8oPPacCtBAXZP+CTj2FLeluAQ6fbuqzgy0II1EfOxhsPqM2kyF7AdcmHW
         W4uA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=SrgpsGNJ8rqjyV4/EJo88O9IIQbG0UxIB/O/sCFEHCQ=;
        b=DMPnAjSJpEetbeEbo++0eAVhYTUcTMjIYhCeQWo9cr2pS62qHBC/PVtb4CpfFAzNB1
         8NJB3ulc56uKIf/Rpkc37WHbE/0lZkkGt7WWc3mn2MKvojwEIo5YURtYa8lYGmcI372Q
         di1WSW69duxR7Wrn1p7F8v4IUEtfuuDjRHygkLo5+kqFuCkG3tMPxiYIsk3V65Sw6WlG
         /fay2lB0PknOBAgFa3KRZsKB6Ho4cE3BvFyyBVj5gwJBpQblS4IKEx3/IKQXRokaCv+k
         57t7kOz1k2jGoZWlKUBtr0Rlq7QDeXvzg9xY3785CWJCuFg0c04IvCob4d5U/+59e4r7
         xM8w==
X-Gm-Message-State: AOAM531RtQA3itZFc1x4lU+iuuK7fE9ltkny1GTy1Y387rLtk2XLuGQR
	uOntAybOanzo6MIpM/rCd8kMkg==
X-Google-Smtp-Source: ABdhPJztKWJhnn4ll+uIjZV6ppsKBX5LhsnC09hXPT7pT0HwfgdemziHQj0i0h1tKYzjFbRFxZd+Gw==
X-Received: by 2002:a63:bf07:: with SMTP id v7mr30381034pgf.333.1632266534266;
        Tue, 21 Sep 2021 16:22:14 -0700 (PDT)
Date: Tue, 21 Sep 2021 23:22:09 +0000
From: Sean Christopherson <seanjc@google.com>
To: "Zhu, Lingshan" <lingshan.zhu@intel.com>
Cc: peterz@infradead.org, pbonzini@redhat.com, bp@alien8.de,
	vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com,
	joro@8bytes.org, kan.liang@linux.intel.com, ak@linux.intel.com,
	wei.w.wang@intel.com, eranian@google.com, liuxiangdong5@huawei.com,
	linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org,
	like.xu.linux@gmail.com, boris.ostrvsky@oracle.com,
	Like Xu <like.xu@linux.intel.com>, Will Deacon <will@kernel.org>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu,
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH V10 01/18] perf/core: Use static_call to optimize
 perf_guest_info_callbacks
Message-ID: <YUppIQxphCUK6ZLS@google.com>
References: <20210806133802.3528-1-lingshan.zhu@intel.com>
 <20210806133802.3528-2-lingshan.zhu@intel.com>
 <YSfykbECnC6J02Yk@google.com>
 <186c330e-be42-4c49-545c-8f73573b5869@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <186c330e-be42-4c49-545c-8f73573b5869@intel.com>

On Wed, Sep 15, 2021, Zhu, Lingshan wrote:
> 
> 
> On 8/27/2021 3:59 AM, Sean Christopherson wrote:
> > TL;DR: Please don't merge this patch, it's broken and is also built on a shoddy
> >         foundation that I would like to fix.
> Hi Sean,Peter, Paolo
> 
> I will send out an V11 which drops this patch since it's buggy, and Sean is
> working on fix this.
> Does this sound good?

Works for me, thanks!


From xen-devel-bounces@lists.xenproject.org Tue Sep 21 23:39:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 21 Sep 2021 23:39:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191977.342162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpLn-0005Mg-LI; Tue, 21 Sep 2021 23:38:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191977.342162; Tue, 21 Sep 2021 23:38:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpLn-0005MZ-Hz; Tue, 21 Sep 2021 23:38:59 +0000
Received: by outflank-mailman (input) for mailman id 191977;
 Tue, 21 Sep 2021 23:38:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AvEq=OL=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mSpLm-0005MT-4i
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 23:38:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 167ed822-1b35-11ec-b940-12813bfff9fa;
 Tue, 21 Sep 2021 23:38:57 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2D08F604DC;
 Tue, 21 Sep 2021 23:38:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 167ed822-1b35-11ec-b940-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632267536;
	bh=ReO7nx50rtBUa/cH2PmFCMlKCCW4WPJ6VzwFL0qZO5M=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fZJH1mLEt/VHB5Pwp1iMuPDmdW4Lghk9yLUjQca7v24Y4DToHXzvl4ouUwe6zJC4V
	 EbakIoG/LSIQw5fPdn86g0AhhFPiBCon2MbDqALxwu7Ek5Ydu7AXaZdDk9xbR+87An
	 ufcm4dp3rhHbjc2ax2m5qEyQ/eeYkhqHmEGys2SJWHWR8ModLIgBX/VsXrfDospDE5
	 Dz6+26bFvYuK792G5X/jVqHLitjrtqYtUezwDjMMpXMb1YLmxvjFrLx5I5flBHR6p2
	 DwxGRW8YsXICZ6cGTnB6o0pDsyra8aiu5Pz4LdptbvBx6oA2P08paN0pimF4NlAMbT
	 fDBv2gaD7WonQ==
Date: Tue, 21 Sep 2021 16:38:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Ian Jackson <iwj@xenproject.org>
cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org, 
    dpsmith@apertussolutions.com, sstabellini@kernel.org
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <24904.44119.940679.241639@mariner.uk.xensource.com>
Message-ID: <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
References: <osstest-164996-mainreport@xen.org> <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com> <24904.44119.940679.241639@mariner.uk.xensource.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 20 Sep 2021, Ian Jackson wrote:
> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> > As per
> > 
> > Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
> > Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
> > Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
> > Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
> > Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
> > Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
> > Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
> > 
> > the system doesn't look to really be out of memory; as per
> > 
> > Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
> > 
> > there even look to be a number of higher order pages available (albeit
> > without digging I can't tell what "(C)" means). Nevertheless order-4
> > allocations aren't really nice.
> 
> The host history suggests this may possibly be related to a qemu update.
> 
> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
> 
> > What I can't see is why this may have started triggering recently. Was
> > the kernel updated in osstest? Is 512Mb of memory perhaps a bit too
> > small for a Dom0 on this system (with 96 CPUs)? Going through the log
> > I haven't been able to find crucial information like how much memory
> > the host has or what the hypervisor command line was.
> 
> Logs from last host examination, including a dmesg:
> 
> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.examine/
> 
> Re the command line, does Xen not print it ?
> 
> The bootloader output seems garbled in the serial log.
> 
> Anyway, I think Xen is being booted EFI judging by the grub cfg:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--grub.cfg.1
> 
> which means that it is probaly reading this:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/165002/test-arm64-arm64-libvirt-raw/rochester0--xen.cfg
> 
> which gives this specification of the command line:
> 
>   options=placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  
> 
> The grub cfg has this:
> 
>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
> 
> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".

I definitely recommend to increase dom0 memory, especially as I guess
the box is going to have a significant amount, far more than 4GB. I
would set it to 2GB. Also the syntax on ARM is simpler, so it should be
just: dom0_mem=2G

In addition, I also did some investigation just in case there is
actually a bug in the code and it is not a simple OOM problem.

Looking at the recent OSSTests results, the first failure is:
https://marc.info/?l=xen-devel&m=163145323631047
http://logs.test-lab.xenproject.org/osstest/logs/164951/

Indeed, the failure is the same test-arm64-arm64-libvirt-raw which is
still failing in more recent tests:
http://logs.test-lab.xenproject.org/osstest/logs/164951/test-arm64-arm64-libvirt-raw/info.html

But if we look at the commit id of flight 164951, it is
6d45368a0a89e01a3a01d156af61fea565db96cc "xsm: drop dubious xsm_op_t
type" by Daniel P. Smith (CCed).

It is interesting because:
- it is *before* all the recent ARM patch series
- it is only 4 commits after master


The 4 commits are:

2021-09-10 16:12 Daniel P. Smith   o xsm: drop dubious xsm_op_t type
2021-09-10 16:12 Daniel P. Smith   o xsm: remove remnants of xsm_memtype hook
2021-09-10 16:12 Daniel P. Smith   o xsm: remove the ability to disable flask
2021-09-10 16:12 Andrew Cooper     o xen: Implement xen/alternative-call.h for use in common code


Looking at them in details:

- "xen: Implement xen/alternative-call.h for use in common code" shouldn'
It shouldn't affect ARM at all

- "xsm: remove the ability to disable flask"
It would only affect the test case if libvirt directly or via libxl
calls FLASK_DISABLE.

- "xsm: remove remnants of xsm_memtype hook"
Shouldn't have any effects

- "xsm: drop dubious xsm_op_t type"
It doesn't look like it should have any runtime effect, only build time


So among these four, only "xsm: remove the ability to disable flask"
seems to have the potential to break a libvirt guest start test. Even
that, it is far fetched and the lack of an explicit XSM-related error
message in the logs would really point in the direction of an OOM.



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:05:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:05:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191983.342173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpli-0000q6-J6; Wed, 22 Sep 2021 00:05:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191983.342173; Wed, 22 Sep 2021 00:05:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpli-0000pz-GG; Wed, 22 Sep 2021 00:05:46 +0000
Received: by outflank-mailman (input) for mailman id 191983;
 Wed, 22 Sep 2021 00:05:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VOxS=OM=flex--seanjc.bounces.google.com=3V3NKYQYKCXgoaWjfYckkcha.Ykitaj-Zarahheopo.tajlnkfaYp.knc@srs-us1.protection.inumbo.net>)
 id 1mSplg-0000pt-V5
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:05:44 +0000
Received: from mail-qt1-x849.google.com (unknown [2607:f8b0:4864:20::849])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 55ff4fc9-b676-48ad-ba93-f8cff336c5d7;
 Wed, 22 Sep 2021 00:05:43 +0000 (UTC)
Received: by mail-qt1-x849.google.com with SMTP id
 o9-20020ac80249000000b002a0c9fd54d5so5595651qtg.4
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55ff4fc9-b676-48ad-ba93-f8cff336c5d7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:message-id:mime-version:subject:from:to:cc;
        bh=phOg6sfjUxaw2dev9sQjUWZF+Yytb7b5OFt6ForYj/8=;
        b=iUirf8Xy9HBHz1WXV3sP98cQFZc4uLbiWcks09YKieGNfFGG+gs0k72fDMm3HygsTL
         bTb1E9YXxvDETi2LPUmOcN81qVU8N8EHMfwEIhym5xr1GL4nuPP5VSQWu2ilOytZoTGS
         29N5LX0Q1nAT/Yc19LBLlvpkIdDWZ9R0T+ikDFK/YvQdlOVrTR3jtpdXiFmTcxfOUu8s
         Y4Gz2V9KUWdc0VylyfjHYjfz7ZmN9V9G49FIJvjgEkSVzn9mYiMA4K/jF7NhytdmJ3F/
         hUx1PUAps9mDw+Jed5eSqhmcHrheKl00VcwQxkkBJMKDmZCGnvuYJgWoF05Eip0DYKf0
         Y9bA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:message-id:mime-version:subject
         :from:to:cc;
        bh=phOg6sfjUxaw2dev9sQjUWZF+Yytb7b5OFt6ForYj/8=;
        b=WLJT9RBeekTGPH89aDygZ3wk5FsNyIauRG33rm+tF/mB+1XaEzPu5v9agaGyk8sqD4
         C3+qGsLJFBynSbKmAQVDkwh9YTnRZldL1daYPN0pnw+W3mMJzs1ZgjcSCGhlArQx/OgW
         q6g1slE4V5ItxGT5r9XYtZP94zHml3ra7fWLfvxyLggIFQb2TGWSMtWQMqNMBPtNhEJD
         ZC7qD3tMd12ndf1ZuqLfg7U6FkTui+DAsqdcmrw/1DJmrDelxe6eVRAX6AyU9Z/GmHGd
         GCIVQ394PekJkWax33I/GyVnacNT7siYby/GyJEfX+XNwrzXIHuXWWLGFHN94iBBYnct
         MXOQ==
X-Gm-Message-State: AOAM530A8gJsskLDZncKvgGP4gLvFKUk9yGnOeaiteaYmrq2+d8TBf0J
	XqOqy160v0gSxY71EhzTCAfBuxAZ6T0=
X-Google-Smtp-Source: ABdhPJwRFbj6xeEYgNbL67Zce9IIY/R6nA0AUChux2cSB+iV33dupsb1zwE380RUBsEvPw8un2YtVWvkUMY=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a25:598b:: with SMTP id n133mr40023097ybb.290.1632269143022;
 Tue, 21 Sep 2021 17:05:43 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:17 -0700
Message-Id: <20210922000533.713300-1-seanjc@google.com>
Mime-Version: 1.0
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 00/16] perf: KVM: Fix, optimize, and clean up callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Peter, I left the Intel PT mess as-is.  Having to pass a NULL pointer
from KVM arm64 seemed to be a lesser evil than more exports and multiple
registration paths.

This is a combination of ~2 series to fix bugs in the perf+KVM callbacks,
optimize the callbacks by employing static_call, and do a variety of
cleanup in both perf and KVM.

Patch 1 fixes a mostly-theoretical bug where perf can deref a NULL
pointer if KVM unregisters its callbacks while they're being accessed.
In practice, compilers tend to avoid problematic reloads of the pointer
and the PMI handler doesn't lose the race against module unloading,
i.e doesn't hit a use-after-free.

Patches 2 and 3 fix an Intel PT handling bug where KVM incorrectly
eats PT interrupts when PT is supposed to be owned entirely by the host.

Patches 4-9 clean up perf's callback infrastructure and switch to
static_call for arm64 and x86 (the only survivors).

Patches 10-16 clean up related KVM code and unify the arm64/x86 callbacks.

Based on "git://git.kernel.org/pub/scm/virt/kvm/kvm.git queue", commit
680c7e3be6a3 ("KVM: x86: Exit to userspace ...").

v3:
  - Add wrappers for guest callbacks to that stubs can be provided when
    GUEST_PERF_EVENTS=n.
  - s/HAVE_GUEST_PERF_EVENTS/GUEST_PERF_EVENTS and select it from KVM
    and XEN_PV instead of from top-level arm64/x86. [Paolo]
  - Drop an unnecessary synchronize_rcu() when registering callbacks. [Peter]
  - Retain a WARN_ON_ONCE() when unregistering callbacks if the caller
    didn't provide the correct pointer. [Peter]
  - Rework the static_call patch to move it all to common perf.
  - Add a patch to drop the (un)register stubs, made possible after
    having KVM+XEN_PV select GUEST_PERF_EVENTS.
  - Split dropping guest callback "support" for arm, csky, etc... to a
    separate patch, to make introducing GUEST_PERF_EVENTS cleaner.
  
v2 (relative to static_call v10):
  - Split the patch into the semantic change (multiplexed ->state) and
    introduction of static_call.
  - Don't use '0' for "not a guest RIP".
  - Handle unregister path.
  - Drop changes for architectures that can be culled entirely.

v2 (relative to v1):
  - https://lkml.kernel.org/r/20210828003558.713983-6-seanjc@google.com
  - Drop per-cpu approach. [Peter]
  - Fix mostly-theoretical reload and use-after-free with READ_ONCE(),
    WRITE_ONCE(), and synchronize_rcu(). [Peter]
  - Avoid new exports like the plague. [Peter]

v1:
  - https://lkml.kernel.org/r/20210827005718.585190-1-seanjc@google.com

v10 static_call:
  - https://lkml.kernel.org/r/20210806133802.3528-2-lingshan.zhu@intel.com


Like Xu (1):
  perf/core: Rework guest callbacks to prepare for static_call support

Sean Christopherson (15):
  perf: Ensure perf_guest_cbs aren't reloaded between !NULL check and
    deref
  KVM: x86: Register perf callbacks after calling vendor's
    hardware_setup()
  KVM: x86: Register Processor Trace interrupt hook iff PT enabled in
    guest
  perf: Stop pretending that perf can handle multiple guest callbacks
  perf: Drop dead and useless guest "support" from arm, csky, nds32 and
    riscv
  perf: Add wrappers for invoking guest callbacks
  perf: Force architectures to opt-in to guest callbacks
  perf/core: Use static_call to optimize perf_guest_info_callbacks
  KVM: x86: Drop current_vcpu for kvm_running_vcpu + kvm_arch_vcpu
    variable
  KVM: x86: More precisely identify NMI from guest when handling PMI
  KVM: Move x86's perf guest info callbacks to generic KVM
  KVM: x86: Move Intel Processor Trace interrupt handler to vmx.c
  KVM: arm64: Convert to the generic perf callbacks
  KVM: arm64: Drop perf.c and fold its tiny bits of code into arm.c /
    pmu.c
  perf: Drop guest callback (un)register stubs

 arch/arm/kernel/perf_callchain.c   | 28 ++------------
 arch/arm64/include/asm/kvm_host.h  |  9 ++++-
 arch/arm64/kernel/perf_callchain.c | 13 ++++---
 arch/arm64/kvm/Kconfig             |  1 +
 arch/arm64/kvm/Makefile            |  2 +-
 arch/arm64/kvm/arm.c               | 11 +++++-
 arch/arm64/kvm/perf.c              | 62 ------------------------------
 arch/arm64/kvm/pmu.c               |  8 ++++
 arch/csky/kernel/perf_callchain.c  | 10 -----
 arch/nds32/kernel/perf_event_cpu.c | 29 ++------------
 arch/riscv/kernel/perf_callchain.c | 10 -----
 arch/x86/events/core.c             | 13 ++++---
 arch/x86/events/intel/core.c       |  5 +--
 arch/x86/include/asm/kvm_host.h    |  7 +++-
 arch/x86/kvm/Kconfig               |  1 +
 arch/x86/kvm/pmu.c                 |  2 +-
 arch/x86/kvm/svm/svm.c             |  2 +-
 arch/x86/kvm/vmx/vmx.c             | 25 +++++++++++-
 arch/x86/kvm/x86.c                 | 58 +++++-----------------------
 arch/x86/kvm/x86.h                 | 17 ++++++--
 arch/x86/xen/Kconfig               |  1 +
 arch/x86/xen/pmu.c                 | 32 +++++++--------
 include/kvm/arm_pmu.h              |  1 +
 include/linux/kvm_host.h           | 10 +++++
 include/linux/perf_event.h         | 41 ++++++++++++++------
 init/Kconfig                       |  4 ++
 kernel/events/core.c               | 39 +++++++++++++------
 virt/kvm/kvm_main.c                | 44 +++++++++++++++++++++
 28 files changed, 235 insertions(+), 250 deletions(-)
 delete mode 100644 arch/arm64/kvm/perf.c

-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:05:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:05:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191984.342184 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSplm-00017M-WD; Wed, 22 Sep 2021 00:05:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191984.342184; Wed, 22 Sep 2021 00:05:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSplm-00017B-Su; Wed, 22 Sep 2021 00:05:50 +0000
Received: by outflank-mailman (input) for mailman id 191984;
 Wed, 22 Sep 2021 00:05:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cjvW=OM=flex--seanjc.bounces.google.com=3WXNKYQYKCXoqcYlhaemmejc.amkvcl-bctcjjgqrq.vclnpmhcar.mpe@srs-us1.protection.inumbo.net>)
 id 1mSpll-0000pt-Ox
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:05:49 +0000
Received: from mail-qk1-x74a.google.com (unknown [2607:f8b0:4864:20::74a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16782e8b-d8d9-4f90-93e1-93a8f48dc421;
 Wed, 22 Sep 2021 00:05:45 +0000 (UTC)
Received: by mail-qk1-x74a.google.com with SMTP id
 m10-20020a05620a290a00b004334bcdf1f4so6321984qkp.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16782e8b-d8d9-4f90-93e1-93a8f48dc421
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=TT46UfxgGprT2Z8Ek6HDihOky77I506i9z7f8BFaquI=;
        b=QzatdNs58QulqhqAUqMzQHeMAZPVrruf34JxSaeKsb1wzZ7fEDRzaG2VjwSuLxNaPK
         Zvx8MU5LTp23EVbbGwmSgF0zN88glOx1zuvYfg2K4E6TFt5xsqwnJecnrnxJjVAn/XSW
         p48kSYrSJol0i2asOjTgauxlp6dmcy3NEKL7ZHjXM80ZjKZWCbCifbyLIo2+1WPZVKmw
         KwytLQKUAM4T7L4vMe2mEMdZsxDaSERrNtTQF/dJnOu+S6XA2H8bffufP4PE9h73gOTr
         07a1KkkjHmEAqPeIgb2YcP1v472LGYuhkbSzi+tELp5FcP+mJWDNfaegNWBCoCBIjGmv
         td/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=TT46UfxgGprT2Z8Ek6HDihOky77I506i9z7f8BFaquI=;
        b=gJs+xPfh64V9kyv2JP7PQP+tFC+JVwQcmv+r35jiMfdaDNhKcy5IeS8GnzVsA76IHo
         UILe4y12P6ijhTV7YT8XpiZcdAZJLCMBIUjZbcd49h48KRZvtxkJ/OA8yWwxtygRvjCb
         HM7wfB1uV78IXOIu3ZMpGXskT6LV/q+8yRApBsbnHSw7zmz216sxQD1CHKNrI66lNtz1
         Dv/RGDZdXU3PUdpRzQnPOljM0TjHCbXqUzKF9Tf5Yi0EwmABBK2RK6f3GY5m6v4mVg2t
         jlo6PHX7klG10rBR85JH49q99FRq6uKZMXIojioMquekGZ0jY+iZz1536NaV/3mq/xjj
         zJ2w==
X-Gm-Message-State: AOAM532O+/8karYXuxx5awwIfhS+Mu1qsy/znzBaahYqhBQiMmwSDe+p
	7LXJfpxQNj+gFEzLIobD73y34Jfzgq0=
X-Google-Smtp-Source: ABdhPJxemEp05xYPMtMooHZ3zkiQnq8qGEaQ+AGLmyjkjkis7iXBJprrvK8Jp05dv5ZcoQEUnmmDtCEj8No=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:58f:: with SMTP id
 bx15mr6697827qvb.7.1632269145516; Tue, 21 Sep 2021 17:05:45 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:18 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-2-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 01/16] perf: Ensure perf_guest_cbs aren't reloaded between
 !NULL check and deref
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Protect perf_guest_cbs with READ_ONCE/WRITE_ONCE to ensure it's not
reloaded between a !NULL check and a dereference, and wait for all
readers via syncrhonize_rcu() to prevent use-after-free, e.g. if the
callbacks are being unregistered during module unload.  Because the
callbacks are global, it's possible for readers to run in parallel with
an unregister operation.

The bug has escaped notice because all dereferences of perf_guest_cbs
follow the same "perf_guest_cbs && perf_guest_cbs->is_in_guest()" pattern,
and it's extremely unlikely a compiler will reload perf_guest_cbs in this
sequence.  Compilers do reload perf_guest_cbs for future derefs, e.g. for
->is_user_mode(), but the ->is_in_guest() guard all but guarantees the
PMI handler will win the race, e.g. to nullify perf_guest_cbs, KVM has to
completely exit the guest and teardown down all VMs before KVM start its
module unload / unregister sequence.

But with help, unloading kvm_intel can trigger a NULL pointer derference,
e.g. wrapping perf_guest_cbs with READ_ONCE in perf_misc_flags() while
spamming kvm_intel module load/unload leads to:

  BUG: kernel NULL pointer dereference, address: 0000000000000000
  #PF: supervisor read access in kernel mode
  #PF: error_code(0x0000) - not-present page
  PGD 0 P4D 0
  Oops: 0000 [#1] PREEMPT SMP
  CPU: 6 PID: 1825 Comm: stress Not tainted 5.14.0-rc2+ #459
  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
  RIP: 0010:perf_misc_flags+0x1c/0x70
  Call Trace:
   perf_prepare_sample+0x53/0x6b0
   perf_event_output_forward+0x67/0x160
   __perf_event_overflow+0x52/0xf0
   handle_pmi_common+0x207/0x300
   intel_pmu_handle_irq+0xcf/0x410
   perf_event_nmi_handler+0x28/0x50
   nmi_handle+0xc7/0x260
   default_do_nmi+0x6b/0x170
   exc_nmi+0x103/0x130
   asm_exc_nmi+0x76/0xbf

Fixes: 39447b386c84 ("perf: Enhance perf to allow for guest statistic collection from host")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm/kernel/perf_callchain.c   | 17 +++++++++++------
 arch/arm64/kernel/perf_callchain.c | 18 ++++++++++++------
 arch/csky/kernel/perf_callchain.c  |  6 ++++--
 arch/nds32/kernel/perf_event_cpu.c | 17 +++++++++++------
 arch/riscv/kernel/perf_callchain.c |  7 +++++--
 arch/x86/events/core.c             | 17 +++++++++++------
 arch/x86/events/intel/core.c       |  9 ++++++---
 include/linux/perf_event.h         |  8 ++++++++
 kernel/events/core.c               | 11 +++++++++--
 9 files changed, 77 insertions(+), 33 deletions(-)

diff --git a/arch/arm/kernel/perf_callchain.c b/arch/arm/kernel/perf_callchain.c
index 3b69a76d341e..1626dfc6f6ce 100644
--- a/arch/arm/kernel/perf_callchain.c
+++ b/arch/arm/kernel/perf_callchain.c
@@ -62,9 +62,10 @@ user_backtrace(struct frame_tail __user *tail,
 void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct frame_tail __user *tail;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -98,9 +99,10 @@ callchain_trace(struct stackframe *fr,
 void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -111,18 +113,21 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
-		return perf_guest_cbs->get_guest_ip();
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	if (guest_cbs && guest_cbs->is_in_guest())
+		return guest_cbs->get_guest_ip();
 
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
-		if (perf_guest_cbs->is_user_mode())
+	if (guest_cbs && guest_cbs->is_in_guest()) {
+		if (guest_cbs->is_user_mode())
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c
index 4a72c2727309..86d9f2013172 100644
--- a/arch/arm64/kernel/perf_callchain.c
+++ b/arch/arm64/kernel/perf_callchain.c
@@ -102,7 +102,9 @@ compat_user_backtrace(struct compat_frame_tail __user *tail,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -147,9 +149,10 @@ static bool callchain_trace(void *data, unsigned long pc)
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe frame;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -160,18 +163,21 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
-		return perf_guest_cbs->get_guest_ip();
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	if (guest_cbs && guest_cbs->is_in_guest())
+		return guest_cbs->get_guest_ip();
 
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
-		if (perf_guest_cbs->is_user_mode())
+	if (guest_cbs && guest_cbs->is_in_guest()) {
+		if (guest_cbs->is_user_mode())
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/csky/kernel/perf_callchain.c b/arch/csky/kernel/perf_callchain.c
index ab55e98ee8f6..35318a635a5f 100644
--- a/arch/csky/kernel/perf_callchain.c
+++ b/arch/csky/kernel/perf_callchain.c
@@ -86,10 +86,11 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 
 	/* C-SKY does not support virtualization. */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
+	if (guest_cbs && guest_cbs->is_in_guest())
 		return;
 
 	fp = regs->regs[4];
@@ -110,10 +111,11 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
 	/* C-SKY does not support virtualization. */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		pr_warn("C-SKY does not support perf in guest mode!");
 		return;
 	}
diff --git a/arch/nds32/kernel/perf_event_cpu.c b/arch/nds32/kernel/perf_event_cpu.c
index 0ce6f9f307e6..f38791960781 100644
--- a/arch/nds32/kernel/perf_event_cpu.c
+++ b/arch/nds32/kernel/perf_event_cpu.c
@@ -1363,6 +1363,7 @@ void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 		    struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 	unsigned long gp = 0;
 	unsigned long lp = 0;
@@ -1371,7 +1372,7 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 
 	leaf_fp = 0;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -1479,9 +1480,10 @@ void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 		      struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -1493,20 +1495,23 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
 	/* However, NDS32 does not support virtualization */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
-		return perf_guest_cbs->get_guest_ip();
+	if (guest_cbs && guest_cbs->is_in_guest())
+		return guest_cbs->get_guest_ip();
 
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
 	/* However, NDS32 does not support virtualization */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
-		if (perf_guest_cbs->is_user_mode())
+	if (guest_cbs && guest_cbs->is_in_guest()) {
+		if (guest_cbs->is_user_mode())
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
index 0bb1854dce83..8ecfc4c128bc 100644
--- a/arch/riscv/kernel/perf_callchain.c
+++ b/arch/riscv/kernel/perf_callchain.c
@@ -56,10 +56,11 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 
 	/* RISC-V does not support perf in guest mode. */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
+	if (guest_cbs && guest_cbs->is_in_guest())
 		return;
 
 	fp = regs->s0;
@@ -78,8 +79,10 @@ static bool fill_callchain(void *entry, unsigned long pc)
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
 	/* RISC-V does not support perf in guest mode. */
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		pr_warn("RISC-V does not support perf in guest mode!");
 		return;
 	}
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 1eb45139fcc6..ffb3e6c0d367 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -2761,10 +2761,11 @@ static bool perf_hw_regs(struct pt_regs *regs)
 void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct unwind_state state;
 	unsigned long addr;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2864,10 +2865,11 @@ perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry_ctx *ent
 void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stack_frame frame;
 	const struct stack_frame __user *fp;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->is_in_guest()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2944,18 +2946,21 @@ static unsigned long code_segment_base(struct pt_regs *regs)
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest())
-		return perf_guest_cbs->get_guest_ip();
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	if (guest_cbs && guest_cbs->is_in_guest())
+		return guest_cbs->get_guest_ip();
 
 	return regs->ip + code_segment_base(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
-	if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) {
-		if (perf_guest_cbs->is_user_mode())
+	if (guest_cbs && guest_cbs->is_in_guest()) {
+		if (guest_cbs->is_user_mode())
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index fca7a6e2242f..9baa46185d94 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2786,6 +2786,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 {
 	struct perf_sample_data data;
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	struct perf_guest_info_callbacks *guest_cbs;
 	int bit;
 	int handled = 0;
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
@@ -2852,9 +2853,11 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 	 */
 	if (__test_and_clear_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT, (unsigned long *)&status)) {
 		handled++;
-		if (unlikely(perf_guest_cbs && perf_guest_cbs->is_in_guest() &&
-			perf_guest_cbs->handle_intel_pt_intr))
-			perf_guest_cbs->handle_intel_pt_intr();
+
+		guest_cbs = perf_get_guest_cbs();
+		if (unlikely(guest_cbs && guest_cbs->is_in_guest() &&
+			     guest_cbs->handle_intel_pt_intr))
+			guest_cbs->handle_intel_pt_intr();
 		else
 			intel_pt_interrupt();
 	}
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 2d510ad750ed..6b0405e578c1 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1237,6 +1237,14 @@ extern void perf_event_bpf_event(struct bpf_prog *prog,
 				 u16 flags);
 
 extern struct perf_guest_info_callbacks *perf_guest_cbs;
+static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
+{
+	/* Reg/unreg perf_guest_cbs waits for readers via synchronize_rcu(). */
+	lockdep_assert_preemption_disabled();
+
+	/* Prevent reloading between a !NULL check and dereferences. */
+	return READ_ONCE(perf_guest_cbs);
+}
 extern int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
 extern int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
 
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 464917096e73..80ff050a7b55 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6491,14 +6491,21 @@ struct perf_guest_info_callbacks *perf_guest_cbs;
 
 int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
-	perf_guest_cbs = cbs;
+	if (WARN_ON_ONCE(perf_guest_cbs))
+		return -EBUSY;
+
+	WRITE_ONCE(perf_guest_cbs, cbs);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
 
 int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
-	perf_guest_cbs = NULL;
+	if (WARN_ON_ONCE(perf_guest_cbs != cbs))
+		return -EINVAL;
+
+	WRITE_ONCE(perf_guest_cbs, NULL);
+	synchronize_rcu();
 	return 0;
 }
 EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:05:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:05:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191985.342195 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpls-0001Rx-8C; Wed, 22 Sep 2021 00:05:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191985.342195; Wed, 22 Sep 2021 00:05:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpls-0001Ro-4h; Wed, 22 Sep 2021 00:05:56 +0000
Received: by outflank-mailman (input) for mailman id 191985;
 Wed, 22 Sep 2021 00:05:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sJCr=OM=flex--seanjc.bounces.google.com=3W3NKYQYKCXwxjfsohlttlqj.htr2js-ij0jqqnxyx.2jsuwtojhy.twl@srs-us1.protection.inumbo.net>)
 id 1mSplq-0000pt-PT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:05:54 +0000
Received: from mail-qt1-x84a.google.com (unknown [2607:f8b0:4864:20::84a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c089ab9d-313d-4207-ba2f-42954b798da3;
 Wed, 22 Sep 2021 00:05:48 +0000 (UTC)
Received: by mail-qt1-x84a.google.com with SMTP id
 c21-20020a05622a025500b002a6965decfdso5077498qtx.14
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c089ab9d-313d-4207-ba2f-42954b798da3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=/Rel7IDBHL/DlNbV9YDximOR/ceEVZKr5BpHO0J3wYE=;
        b=k3BkoiZwNDQSgL3tEWZpjE6EfQkaXg/m6u3CKlmQTge7gDNDRP920npuJicDQOLGIW
         yoGIN4Z0aY9QErf1e66xqyhlAmsb29Adxkyap6pNmw+FigDAF/HwZpyD4SwfyDNjxEBI
         L3Hj7QeZ2mK2YQzpAxgJzXhidgJBh66gysCSfbdAzAVCNl0/PpYOOwZCkG0XgqdYqcPi
         DepD/Zz5nhXyEE26CXt5pzDL9NIQrBteg/3FetFaX3qIqSta7kwW9lSwibR7Yi7ZIN64
         bGtcOit8MazFfctrkSYZbjCCAGX937gmiMhEPPsCUfa343jN+qkq71vPPMrbXWva4s8x
         1MRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=/Rel7IDBHL/DlNbV9YDximOR/ceEVZKr5BpHO0J3wYE=;
        b=f0/MbFXimdBHm7YKckkKUwDa0ncy09vLJzZ9wto+MH8hTX2KGd1tMlt2r3whXYK3FY
         Cf/q79dlCRpidjrhjR/JRzQq6zWcitYiE9RyosAzxbN5I0MLdzPp5tFRP2DuF5/cdQpE
         8Q8HbLO29/bww7kHoXa0ZWBnTz/jhK5Hp/gnnel5WGQYvmumdmco8YmtfBjuM3dz6oso
         EYRNhd1nUTzeVmyPqNRAoFYcanS/7VzsWxvpmvZL5OLY+fwRX6jN9uvooR6TKUn6/hBN
         DVFn22k/fzHUM76lNeAs0x83KxoXVeskw/ZRhOZBORcXqPRDIKS6hT6/FJ0fSupBwGnT
         vFhw==
X-Gm-Message-State: AOAM531DwRGKpHKJq9gXaVvOaZSIo5E+YgjQO4G57JJQVm7P6TZdj+9R
	9RfpLda2bCnNicgn4yogAE4D2PRDyos=
X-Google-Smtp-Source: ABdhPJxOQ2RWktf9fE+8nT6XGH9bJ2bTXzkYzGz9TY7O9+ijr2BzXnZvbY3O/xwZK7tibJDN6XsSYEuLjw4=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:12af:: with SMTP id
 w15mr24363579qvu.66.1632269147772; Tue, 21 Sep 2021 17:05:47 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:19 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-3-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 02/16] KVM: x86: Register perf callbacks after calling
 vendor's hardware_setup()
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Wait to register perf callbacks until after doing vendor hardaware setup.
VMX's hardware_setup() configures Intel Processor Trace (PT) mode, and a
future fix to register the Intel PT guest interrupt hook if and only if
Intel PT is exposed to the guest will consume the configured PT mode.

Delaying registration to hardware setup is effectively a nop as KVM's perf
hooks all pivot on the per-CPU current_vcpu, which is non-NULL only when
KVM is handling an IRQ/NMI in a VM-Exit path.  I.e. current_vcpu will be
NULL throughout both kvm_arch_init() and kvm_arch_hardware_setup().

Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Artem Kashkanov <artem.kashkanov@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/x86.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 86539c1686fa..fb6015f97f9e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8426,8 +8426,6 @@ int kvm_arch_init(void *opaque)
 
 	kvm_timer_init();
 
-	perf_register_guest_info_callbacks(&kvm_guest_cbs);
-
 	if (boot_cpu_has(X86_FEATURE_XSAVE)) {
 		host_xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK);
 		supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0;
@@ -8461,7 +8459,6 @@ void kvm_arch_exit(void)
 		clear_hv_tscchange_cb();
 #endif
 	kvm_lapic_exit();
-	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
 
 	if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
 		cpufreq_unregister_notifier(&kvmclock_cpufreq_notifier_block,
@@ -11064,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque)
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
 	kvm_ops_static_call_update();
 
+	perf_register_guest_info_callbacks(&kvm_guest_cbs);
+
 	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
 		supported_xss = 0;
 
@@ -11091,6 +11090,8 @@ int kvm_arch_hardware_setup(void *opaque)
 
 void kvm_arch_hardware_unsetup(void)
 {
+	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
+
 	static_call(kvm_x86_hardware_unsetup)();
 }
 
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:06:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:06:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191986.342206 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSplx-0001oN-Gh; Wed, 22 Sep 2021 00:06:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191986.342206; Wed, 22 Sep 2021 00:06:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSplx-0001oG-Ce; Wed, 22 Sep 2021 00:06:01 +0000
Received: by outflank-mailman (input) for mailman id 191986;
 Wed, 22 Sep 2021 00:05:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bHsd=OM=flex--seanjc.bounces.google.com=3XnNKYQYKCX8vhdqmfjrrjoh.frp0hq-ghyhoolvwv.0hqsurmhfw.ruj@srs-us1.protection.inumbo.net>)
 id 1mSplv-0000pt-PV
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:05:59 +0000
Received: from mail-qk1-x749.google.com (unknown [2607:f8b0:4864:20::749])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ac59d283-ef6f-43ef-b078-c6d60780eaee;
 Wed, 22 Sep 2021 00:05:50 +0000 (UTC)
Received: by mail-qk1-x749.google.com with SMTP id
 w2-20020a3794020000b02903b54f40b442so6728543qkd.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac59d283-ef6f-43ef-b078-c6d60780eaee
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=WoefsUy8P8uVLjBhkd6fplSxv9zXtwFfZO3aWK09BqE=;
        b=W3dvAswYaFrMhdUxOnj+C5EXMMEyJrI4w2EWKYkJvgEgkHuR8IaKKfSJHg9j0xw5+k
         fgorvfW1/YzGQUtW4RZ95gDG0qA72H/qeIgYOd+4TOG6c+1wocOOu3VRxQzsB6jGSO5g
         yRDdyBni9EhmFEE/PynpUeNnGYgcYXTu6HydYJ60R/86lZ/rosIKMEDAP+lTix9jhUIs
         1g7+eQ1MkKa4HLPulfOq0mS5b56v1Wn9wTgMBlG9vvFK/yuOUIxbO5OMIO050BN1ziI7
         quI8vYbT72gL4MHo6ScRZ81Wr5yMBwO1kb48G3KB1R6pXXjpQyqrpGzih9Su7JxvCNCF
         qqhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=WoefsUy8P8uVLjBhkd6fplSxv9zXtwFfZO3aWK09BqE=;
        b=8JkYGt9AFKK9fxlN5HT0T0FQacaAuxTjAWY1k7easL+z1G0mWtfqypZJaY8QjfE62p
         x8yRU1OcNWHdyDO0QgE8dJlZT3LmKJD3uFHxJ8DtWVvVaCYCR4zCYyryKSQXr+lx25VC
         XqtdM4D8ZdIOLMVYZ8PMFWtAESqOZWipcBowtLsbvkdOwmwyGm96Ahoi2hPVD7GTfkj+
         8ub4nkVinJY4ffpwfGi9Ge2wJn+J2JMr1zkLdaArXqSj7CcBDhZYbkBcsQwGvOudCXY3
         IiqiSdqbGCV1vztCrhQr9gc2aphV//u51GL69WR1EH1qUWpnyEmTrJSGflr3pDqWTIqJ
         yN7g==
X-Gm-Message-State: AOAM532n8RXTyT2RWh/ZdgN36wt0G+5bJfYawIyt+QBZdAmGPBIopsP7
	jFgdo9+w80BclW/navCCG32c8R/8eoE=
X-Google-Smtp-Source: ABdhPJydgoIIy+PWW+KTdPOq8w+CNl9+TVcl5SdFV/4H0FYG9MktEVuetKxR7qEx7VklEO7jLEzl555CBZg=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a25:2cd6:: with SMTP id s205mr42869270ybs.445.1632269150043;
 Tue, 21 Sep 2021 17:05:50 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:20 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-4-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 03/16] KVM: x86: Register Processor Trace interrupt hook
 iff PT enabled in guest
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Override the Processor Trace (PT) interrupt handler for guest mode if and
only if PT is configured for host+guest mode, i.e. is being used
independently by both host and guest.  If PT is configured for system
mode, the host fully controls PT and must handle all events.

Fixes: 8479e04e7d6b ("KVM: x86: Inject PMI for KVM guest")
Cc: stable@vger.kernel.org
Cc: Like Xu <like.xu.linux@gmail.com>
Reported-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Reported-by: Artem Kashkanov <artem.kashkanov@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/vmx/vmx.c          | 1 +
 arch/x86/kvm/x86.c              | 5 ++++-
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 09b256db394a..1ea4943a73d7 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1494,6 +1494,7 @@ struct kvm_x86_init_ops {
 	int (*disabled_by_bios)(void);
 	int (*check_processor_compatibility)(void);
 	int (*hardware_setup)(void);
+	bool (*intel_pt_intr_in_guest)(void);
 
 	struct kvm_x86_ops *runtime_ops;
 };
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index fada1055f325..f19d72136f77 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7896,6 +7896,7 @@ static struct kvm_x86_init_ops vmx_init_ops __initdata = {
 	.disabled_by_bios = vmx_disabled_by_bios,
 	.check_processor_compatibility = vmx_check_processor_compat,
 	.hardware_setup = hardware_setup,
+	.intel_pt_intr_in_guest = vmx_pt_mode_is_host_guest,
 
 	.runtime_ops = &vmx_x86_ops,
 };
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fb6015f97f9e..ffc6c2d73508 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8305,7 +8305,7 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = {
 	.is_in_guest		= kvm_is_in_guest,
 	.is_user_mode		= kvm_is_user_mode,
 	.get_guest_ip		= kvm_get_guest_ip,
-	.handle_intel_pt_intr	= kvm_handle_intel_pt_intr,
+	.handle_intel_pt_intr	= NULL,
 };
 
 #ifdef CONFIG_X86_64
@@ -11061,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque)
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
 	kvm_ops_static_call_update();
 
+	if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest())
+		kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr;
 	perf_register_guest_info_callbacks(&kvm_guest_cbs);
 
 	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
@@ -11091,6 +11093,7 @@ int kvm_arch_hardware_setup(void *opaque)
 void kvm_arch_hardware_unsetup(void)
 {
 	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
+	kvm_guest_cbs.handle_intel_pt_intr = NULL;
 
 	static_call(kvm_x86_hardware_unsetup)();
 }
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:06:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:06:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191987.342217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpm1-0002Ff-W0; Wed, 22 Sep 2021 00:06:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191987.342217; Wed, 22 Sep 2021 00:06:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpm1-0002FU-RN; Wed, 22 Sep 2021 00:06:05 +0000
Received: by outflank-mailman (input) for mailman id 191987;
 Wed, 22 Sep 2021 00:06:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XHq9=OM=flex--seanjc.bounces.google.com=3YHNKYQYKCYExjfsohlttlqj.htr2js-ij0jqqnxyx.2jsuwtojhy.twl@srs-us1.protection.inumbo.net>)
 id 1mSpm0-0000pt-Ph
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:04 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id de9e703c-caa3-44d4-9029-60a5873a67ca;
 Wed, 22 Sep 2021 00:05:52 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 b17-20020a056214135100b0037eaf39cb1fso7685292qvw.11
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de9e703c-caa3-44d4-9029-60a5873a67ca
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=6hT+pcsgsamW9lRK8DKVDgsjod6WnUckxdik+rJgRjs=;
        b=Bj8VBEV1iZnqxOhTLHlm7aJLEpzWR4VK7K7aD+jVIOItnRKO4fixETcLObR8Fp3wpi
         qen4ySzqMv29aPJAGQYTFE7wmMMI1j5Rtph6tALMy5t0Vpqs3rbEz84bRIN0kMxKLv8c
         UqqsOh2SdMvnju4og/DFPHA2mwLIV2Kbl8jNiHyAx8ZaTn+CIt8xR9Nfe8u7VO9K9/eN
         +2x8JLd22AxcRq3kDSstvt/WqN7tY/+xYd3wvpQsDoKFQCmzoF+iYRzBtZMgOAMbDudK
         wnTLWwmn+3bAAzltBQAbNmJfxESz4LUX9Bp79xTAk2sjGVMJKof74Y995u9shdQ30fbC
         IW7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=6hT+pcsgsamW9lRK8DKVDgsjod6WnUckxdik+rJgRjs=;
        b=Yo6WmVZ4/NNSZzK/7rSKIADmJazIeY/K6kOrYpRc4Eez3kGBRMpevGTnuDjgCraXbL
         cw/1R/q7Iii7p63bs6sULS7czLLBsx41itmO03QrVVRV3y685m6c4G/UvkNbhL7MOjS5
         YAsZCAT7ewxI7UxzHXUyuEHiVGd+vYqbNzq/QR4dRMka6S41frigneKxQAYlq8h6r0Cj
         MBIcv7fGyFOjWNQy2nZTMzyR7RoP0BQ0rAiRoP4gVghpVrmNvuYjGr0GUg52kHMqQD8A
         GAerlrHrp1RGab7La1SHPrDu0mbkbMWGHcA9X6ikDM/1gfcNHIq+9wsG7QTxVVMCmb0G
         aHAA==
X-Gm-Message-State: AOAM533yJNetrg2CqKxRIfxnERiwITiI87YNOFXAkkjxNVUAKeoBMug7
	QDe52NdUyvV8PCKR3V5ofbnOR3y5CKA=
X-Google-Smtp-Source: ABdhPJw55E4dI/IAyKkR+ZKkmH8bPHTaStHFj1pC0lhS1Eeb6cv6+uuItqqgA18GAeXeV2SBtT0zyEX+kRs=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a0c:8e83:: with SMTP id x3mr34844810qvb.40.1632269152245;
 Tue, 21 Sep 2021 17:05:52 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:21 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-5-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 04/16] perf: Stop pretending that perf can handle multiple
 guest callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Drop the 'int' return value from the perf (un)register callbacks helpers
and stop pretending perf can support multiple callbacks.  The 'int'
returns are not future proofing anything as none of the callers take
action on an error.  It's also not obvious that there will ever be
co-tenant hypervisors, and if there are, that allowing multiple callbacks
to be registered is desirable or even correct.

Opportunistically rename callbacks=>cbs in the affected declarations to
match their definitions.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/include/asm/kvm_host.h |  4 ++--
 arch/arm64/kvm/perf.c             |  8 ++++----
 include/linux/perf_event.h        | 12 ++++++------
 kernel/events/core.c              | 16 ++++------------
 4 files changed, 16 insertions(+), 24 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 41911585ae0c..ed940aec89e0 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -670,8 +670,8 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len);
 int kvm_handle_mmio_return(struct kvm_vcpu *vcpu);
 int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa);
 
-int kvm_perf_init(void);
-int kvm_perf_teardown(void);
+void kvm_perf_init(void);
+void kvm_perf_teardown(void);
 
 long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu);
 gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c
index 151c31fb9860..c37c0cf1bfe9 100644
--- a/arch/arm64/kvm/perf.c
+++ b/arch/arm64/kvm/perf.c
@@ -48,15 +48,15 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = {
 	.get_guest_ip	= kvm_get_guest_ip,
 };
 
-int kvm_perf_init(void)
+void kvm_perf_init(void)
 {
 	if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled())
 		static_branch_enable(&kvm_arm_pmu_available);
 
-	return perf_register_guest_info_callbacks(&kvm_guest_cbs);
+	perf_register_guest_info_callbacks(&kvm_guest_cbs);
 }
 
-int kvm_perf_teardown(void)
+void kvm_perf_teardown(void)
 {
-	return perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
+	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
 }
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 6b0405e578c1..317d4658afe9 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1245,8 +1245,8 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
 	/* Prevent reloading between a !NULL check and dereferences. */
 	return READ_ONCE(perf_guest_cbs);
 }
-extern int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
-extern int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
+extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
+extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
 
 extern void perf_event_exec(void);
 extern void perf_event_comm(struct task_struct *tsk, bool exec);
@@ -1489,10 +1489,10 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)	{ }
 static inline void
 perf_bp_event(struct perf_event *event, void *data)			{ }
 
-static inline int perf_register_guest_info_callbacks
-(struct perf_guest_info_callbacks *callbacks)				{ return 0; }
-static inline int perf_unregister_guest_info_callbacks
-(struct perf_guest_info_callbacks *callbacks)				{ return 0; }
+static inline void perf_register_guest_info_callbacks
+(struct perf_guest_info_callbacks *cbs)					{ }
+static inline void perf_unregister_guest_info_callbacks
+(struct perf_guest_info_callbacks *cbs)					{ }
 
 static inline void perf_event_mmap(struct vm_area_struct *vma)		{ }
 
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 80ff050a7b55..d90a43572400 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6482,31 +6482,23 @@ static void perf_pending_event(struct irq_work *entry)
 		perf_swevent_put_recursion_context(rctx);
 }
 
-/*
- * We assume there is only KVM supporting the callbacks.
- * Later on, we might change it to a list if there is
- * another virtualization implementation supporting the callbacks.
- */
 struct perf_guest_info_callbacks *perf_guest_cbs;
-
-int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
+void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
 	if (WARN_ON_ONCE(perf_guest_cbs))
-		return -EBUSY;
+		return;
 
 	WRITE_ONCE(perf_guest_cbs, cbs);
-	return 0;
 }
 EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
 
-int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
+void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
 	if (WARN_ON_ONCE(perf_guest_cbs != cbs))
-		return -EINVAL;
+		return;
 
 	WRITE_ONCE(perf_guest_cbs, NULL);
 	synchronize_rcu();
-	return 0;
 }
 EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
 
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:06:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:06:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191991.342228 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmC-0002qZ-84; Wed, 22 Sep 2021 00:06:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191991.342228; Wed, 22 Sep 2021 00:06:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmC-0002qP-4s; Wed, 22 Sep 2021 00:06:16 +0000
Received: by outflank-mailman (input) for mailman id 191991;
 Wed, 22 Sep 2021 00:06:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SVWJ=OM=flex--seanjc.bounces.google.com=3YnNKYQYKCYMzlhuqjnvvnsl.jvt4lu-kl2lsspz0z.4luwyvqlj0.vyn@srs-us1.protection.inumbo.net>)
 id 1mSpmA-0000pt-Pp
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:14 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ee500b0c-3974-4a24-9c6a-6a6ae796ebe4;
 Wed, 22 Sep 2021 00:05:54 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 e8-20020a0cf348000000b0037a350958f2so7985730qvm.7
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee500b0c-3974-4a24-9c6a-6a6ae796ebe4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=uGRLtrGbpD3S6B/7zzXt3L9N6QvZuu1UjnfG14mJ8ek=;
        b=RFIhbvEyvYMragxwcAv1E8u094o6TR3aNC++WUaaLMbpX1lKaGuAbXr4UQ4DnYHp6f
         0izeMTfNi3w+g5sbRJ0zA7LvF+2NsrCBFGIUyCyWEOC9wn70vrrhnoT3jtw95SZHu/KS
         NX/NMJBJ9rL02BAnaFZcMgeOgQTyJ3QXygCXEu9hwHwMFitHgFFJeLNMjteHVUzmBk3C
         gxJ7A5VIKod2bScVarg0Ti81qTfwhFhcVvMp3krZctVE206zQUgs8jQ7G+Dz7hdiV3WH
         q42Zv28O4z7mAPyMIHk2g1A40W19Qiluvr7IjkcHSYgiI2bp3YLj+WcuS1yfVh40dAHd
         TYrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=uGRLtrGbpD3S6B/7zzXt3L9N6QvZuu1UjnfG14mJ8ek=;
        b=CPFLqGLNZwYEK1h50vrwpS8+ZLJ6WlnpOb7PcWc7Po1MV712TINIHK09BD9r112FMC
         TynkThy9thOlIFIJxwnRGwyvLz6+34pSkogM08ENlEQpBqBzi3a2U8TcaUyyeQofFNFK
         ycxm0gJ2GqbcSbhAXmUNzusXvfnMVjTM5Y+UoZ+qXKkusq5oOCYD2VUzBqu2TV3Af9Yt
         vBg4IaTVXmi0Vl5lkPzOsq9/q/uYc4DfnLWAUp9GM480FMjMt/1yus5gANfFciaVm7tk
         kuHBEUs1/e2xKbez8GAxsDRPgmxuVgen2nsSlzI+nEY/GaErimz39xBiuVqAwZkx/V9v
         dKyA==
X-Gm-Message-State: AOAM530/vRys+CmwDaMVXUlE4deCiDMUlp/ZiPb4NdlRDjdDXlcXAf59
	N3JZfjTfTQOAWfZoyBwQ2rosv0ikLmE=
X-Google-Smtp-Source: ABdhPJx1/jRpojAUuj4tkfVy2mNumpkwTOY4qlI3JrC7x6zrnCf19L4LA34wG9xOt6d1Anx3BB/t4F42q7U=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:2e7:: with SMTP id
 h7mr34209194qvu.39.1632269154568; Tue, 21 Sep 2021 17:05:54 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:22 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-6-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 05/16] perf: Drop dead and useless guest "support" from
 arm, csky, nds32 and riscv
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Drop "support" for guest callbacks from architctures that don't implement
the guest callbacks.  Future patches will convert the callbacks to
static_call; rather than churn a bunch of arch code (that was presumably
copy+pasted from x86), remove it wholesale as it's useless and at best
wasting cycles.

A future patch will also add a Kconfig to force architcture to opt into
the callbacks to make it more difficult for uses "support" to sneak in in
the future.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm/kernel/perf_callchain.c   | 33 ++++-------------------------
 arch/csky/kernel/perf_callchain.c  | 12 -----------
 arch/nds32/kernel/perf_event_cpu.c | 34 ++++--------------------------
 arch/riscv/kernel/perf_callchain.c | 13 ------------
 4 files changed, 8 insertions(+), 84 deletions(-)

diff --git a/arch/arm/kernel/perf_callchain.c b/arch/arm/kernel/perf_callchain.c
index 1626dfc6f6ce..bc6b246ab55e 100644
--- a/arch/arm/kernel/perf_callchain.c
+++ b/arch/arm/kernel/perf_callchain.c
@@ -62,14 +62,8 @@ user_backtrace(struct frame_tail __user *tail,
 void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct frame_tail __user *tail;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		/* We don't support guest os callchain now */
-		return;
-	}
-
 	perf_callchain_store(entry, regs->ARM_pc);
 
 	if (!current->mm)
@@ -99,44 +93,25 @@ callchain_trace(struct stackframe *fr,
 void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		/* We don't support guest os callchain now */
-		return;
-	}
-
 	arm_get_current_stackframe(regs, &fr);
 	walk_stackframe(&fr, callchain_trace, entry);
 }
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return guest_cbs->get_guest_ip();
-
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		if (guest_cbs->is_user_mode())
-			misc |= PERF_RECORD_MISC_GUEST_USER;
-		else
-			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
-	} else {
-		if (user_mode(regs))
-			misc |= PERF_RECORD_MISC_USER;
-		else
-			misc |= PERF_RECORD_MISC_KERNEL;
-	}
+	if (user_mode(regs))
+		misc |= PERF_RECORD_MISC_USER;
+	else
+		misc |= PERF_RECORD_MISC_KERNEL;
 
 	return misc;
 }
diff --git a/arch/csky/kernel/perf_callchain.c b/arch/csky/kernel/perf_callchain.c
index 35318a635a5f..92057de08f4f 100644
--- a/arch/csky/kernel/perf_callchain.c
+++ b/arch/csky/kernel/perf_callchain.c
@@ -86,13 +86,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 
-	/* C-SKY does not support virtualization. */
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return;
-
 	fp = regs->regs[4];
 	perf_callchain_store(entry, regs->pc);
 
@@ -111,15 +106,8 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
-	/* C-SKY does not support virtualization. */
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		pr_warn("C-SKY does not support perf in guest mode!");
-		return;
-	}
-
 	fr.fp = regs->regs[4];
 	fr.lr = regs->lr;
 	walk_stackframe(&fr, entry);
diff --git a/arch/nds32/kernel/perf_event_cpu.c b/arch/nds32/kernel/perf_event_cpu.c
index f38791960781..a78a879e7ef1 100644
--- a/arch/nds32/kernel/perf_event_cpu.c
+++ b/arch/nds32/kernel/perf_event_cpu.c
@@ -1363,7 +1363,6 @@ void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 		    struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 	unsigned long gp = 0;
 	unsigned long lp = 0;
@@ -1372,11 +1371,6 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 
 	leaf_fp = 0;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		/* We don't support guest os callchain now */
-		return;
-	}
-
 	perf_callchain_store(entry, regs->ipc);
 	fp = regs->fp;
 	gp = regs->gp;
@@ -1480,13 +1474,8 @@ void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 		      struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe fr;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		/* We don't support guest os callchain now */
-		return;
-	}
 	fr.fp = regs->fp;
 	fr.lp = regs->lp;
 	fr.sp = regs->sp;
@@ -1495,32 +1484,17 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	/* However, NDS32 does not support virtualization */
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return guest_cbs->get_guest_ip();
-
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	int misc = 0;
 
-	/* However, NDS32 does not support virtualization */
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		if (guest_cbs->is_user_mode())
-			misc |= PERF_RECORD_MISC_GUEST_USER;
-		else
-			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
-	} else {
-		if (user_mode(regs))
-			misc |= PERF_RECORD_MISC_USER;
-		else
-			misc |= PERF_RECORD_MISC_KERNEL;
-	}
+	if (user_mode(regs))
+		misc |= PERF_RECORD_MISC_USER;
+	else
+		misc |= PERF_RECORD_MISC_KERNEL;
 
 	return misc;
 }
diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
index 8ecfc4c128bc..1fc075b8f764 100644
--- a/arch/riscv/kernel/perf_callchain.c
+++ b/arch/riscv/kernel/perf_callchain.c
@@ -56,13 +56,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	unsigned long fp = 0;
 
-	/* RISC-V does not support perf in guest mode. */
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return;
-
 	fp = regs->s0;
 	perf_callchain_store(entry, regs->epc);
 
@@ -79,13 +74,5 @@ static bool fill_callchain(void *entry, unsigned long pc)
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	/* RISC-V does not support perf in guest mode. */
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		pr_warn("RISC-V does not support perf in guest mode!");
-		return;
-	}
-
 	walk_stackframe(NULL, regs, fill_callchain, entry);
 }
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:06:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:06:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191997.342238 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmM-0003Rb-Ir; Wed, 22 Sep 2021 00:06:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191997.342238; Wed, 22 Sep 2021 00:06:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmM-0003RT-F6; Wed, 22 Sep 2021 00:06:26 +0000
Received: by outflank-mailman (input) for mailman id 191997;
 Wed, 22 Sep 2021 00:06:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZQVg=OM=flex--seanjc.bounces.google.com=3ZHNKYQYKCYU1njwslpxxpun.lxv6nw-mn4nuur121.6nwy0xsnl2.x0p@srs-us1.protection.inumbo.net>)
 id 1mSpmK-0000pt-QD
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:24 +0000
Received: from mail-qv1-xf49.google.com (unknown [2607:f8b0:4864:20::f49])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed1d47d4-d448-4e03-a056-0c7e6cfc8e76;
 Wed, 22 Sep 2021 00:05:57 +0000 (UTC)
Received: by mail-qv1-xf49.google.com with SMTP id
 cf6-20020a05621403e600b0037a70d2f960so7547939qvb.13
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed1d47d4-d448-4e03-a056-0c7e6cfc8e76
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=Cp2CdTnLL6A1x2kdouAhprhgeg+5iimEQVWESNgPeTg=;
        b=im2v/L0cygljwm5grHDru5KUhJuz5mZRJLACtD7RonWb5WoFe/D5WuNqzEdPm4G7v8
         rtZr1SHPj05vTG7DdnlSfc+js4UC81QNGDJIA03zdbB9ZdKNy7ID7mCnhXtnu+34C5yg
         kUtUAZ7DnWVxLqGUghCbUvK1tlzpDYoybfavKORupjQBShzw6o2YA6YeI5IYrMg7CfgX
         eElXUTHHYzoIzH2D6fm/ZsLOYCZfPdqD00Yv1xavmD27sqD4jDFv4lzctvG3WIPr12RY
         fcZFN2Q5aGT2PHFfiGVI0yXdul/Sbxc2AMnbSor81BxbiWNV0X2MfR+Z7ZjLq99nqexs
         iEqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=Cp2CdTnLL6A1x2kdouAhprhgeg+5iimEQVWESNgPeTg=;
        b=w0Rdl/sDbvpxrNAF6AsD5IGnSLvNN0YkU7oXZdSKhl3ij7MZFc1V+cdNXwR4emczTK
         i2Wr8DOU6Bi8BDKceVcgdT3zcJdswpR9dSbtBch4vfbW2OiG89mgrMBkXNH3tC8B8qVe
         YYANMmRm2dKOIL59+dc5MCKSsfLcRFwELLC/qrQfVuu9FhXd+iq0UpFIMPGCKKTNScJb
         6wvy+fPXT5t0CLay5Tjkj+11l5G1L5r0nKG9oHhj+i+1J0ttfSLpUBz9CYGj1c8ggjVW
         lbwf3SaX3DlwRuH4ofYa0XEV+4iqj0qkvKflMBy33sMcv5e7E/UCjl74kay5usDOVWuE
         Y7kQ==
X-Gm-Message-State: AOAM531PUc2RkzuKjsZQuSbuK32kIL3IVAkzVz+BEHwt0+OA7XHr97G/
	MRUNcBt4AyRqVzFehZ8qSi0yOw90MAs=
X-Google-Smtp-Source: ABdhPJzsZI5IFhqZMKlAhv/Y45oE9l0vm7I+arslrU5gdLF3H8fHDTQL7yct2ww41HqVKvUsbfn37B7Lw9I=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:ad4:4671:: with SMTP id z17mr33495586qvv.62.1632269156742;
 Tue, 21 Sep 2021 17:05:56 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:23 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-7-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 06/16] perf/core: Rework guest callbacks to prepare for
 static_call support
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

From: Like Xu <like.xu@linux.intel.com>

To prepare for using static_calls to optimize perf's guest callbacks,
replace ->is_in_guest and ->is_user_mode with a new multiplexed hook
->state, tweak ->handle_intel_pt_intr to play nice with being called when
there is no active guest, and drop "guest" from ->is_in_guest.

Return '0' from ->state and ->handle_intel_pt_intr to indicate "not in
guest" so that DEFINE_STATIC_CALL_RET0 can be used to define the static
calls, i.e. no callback == !guest.

Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Originally-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
[sean: extracted from static_call patch, fixed get_ip() bug, wrote changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/kernel/perf_callchain.c | 13 +++++-----
 arch/arm64/kvm/perf.c              | 35 +++++++++++---------------
 arch/x86/events/core.c             | 13 +++++-----
 arch/x86/events/intel/core.c       |  5 +---
 arch/x86/include/asm/kvm_host.h    |  2 +-
 arch/x86/kvm/pmu.c                 |  2 +-
 arch/x86/kvm/x86.c                 | 40 ++++++++++++++++--------------
 arch/x86/xen/pmu.c                 | 32 ++++++++++--------------
 include/linux/perf_event.h         | 10 +++++---
 kernel/events/core.c               |  1 +
 10 files changed, 74 insertions(+), 79 deletions(-)

diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c
index 86d9f2013172..274dc3e11b6d 100644
--- a/arch/arm64/kernel/perf_callchain.c
+++ b/arch/arm64/kernel/perf_callchain.c
@@ -104,7 +104,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 {
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->state()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -152,7 +152,7 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe frame;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->state()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -165,8 +165,8 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return guest_cbs->get_guest_ip();
+	if (guest_cbs && guest_cbs->state())
+		return guest_cbs->get_ip();
 
 	return instruction_pointer(regs);
 }
@@ -174,10 +174,11 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs)
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
 	int misc = 0;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		if (guest_cbs->is_user_mode())
+	if (guest_state) {
+		if (guest_state & PERF_GUEST_USER)
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c
index c37c0cf1bfe9..3e99ac4ab2d6 100644
--- a/arch/arm64/kvm/perf.c
+++ b/arch/arm64/kvm/perf.c
@@ -13,39 +13,34 @@
 
 DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available);
 
-static int kvm_is_in_guest(void)
+static unsigned int kvm_guest_state(void)
 {
-        return kvm_get_running_vcpu() != NULL;
-}
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+	unsigned int state;
 
-static int kvm_is_user_mode(void)
-{
-	struct kvm_vcpu *vcpu;
-
-	vcpu = kvm_get_running_vcpu();
+	if (!vcpu)
+		return 0;
 
-	if (vcpu)
-		return !vcpu_mode_priv(vcpu);
+	state = PERF_GUEST_ACTIVE;
+	if (!vcpu_mode_priv(vcpu))
+		state |= PERF_GUEST_USER;
 
-	return 0;
+	return state;
 }
 
 static unsigned long kvm_get_guest_ip(void)
 {
-	struct kvm_vcpu *vcpu;
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
 
-	vcpu = kvm_get_running_vcpu();
+	if (WARN_ON_ONCE(!vcpu))
+		return 0;
 
-	if (vcpu)
-		return *vcpu_pc(vcpu);
-
-	return 0;
+	return *vcpu_pc(vcpu);
 }
 
 static struct perf_guest_info_callbacks kvm_guest_cbs = {
-	.is_in_guest	= kvm_is_in_guest,
-	.is_user_mode	= kvm_is_user_mode,
-	.get_guest_ip	= kvm_get_guest_ip,
+	.state		= kvm_guest_state,
+	.get_ip		= kvm_get_guest_ip,
 };
 
 void kvm_perf_init(void)
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index ffb3e6c0d367..3a7630fdd340 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -2765,7 +2765,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re
 	struct unwind_state state;
 	unsigned long addr;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2869,7 +2869,7 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs
 	struct stack_frame frame;
 	const struct stack_frame __user *fp;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
+	if (guest_cbs && guest_cbs->state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2948,8 +2948,8 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 
-	if (guest_cbs && guest_cbs->is_in_guest())
-		return guest_cbs->get_guest_ip();
+	if (guest_cbs && guest_cbs->state())
+		return guest_cbs->get_ip();
 
 	return regs->ip + code_segment_base(regs);
 }
@@ -2957,10 +2957,11 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs)
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
 	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
 	int misc = 0;
 
-	if (guest_cbs && guest_cbs->is_in_guest()) {
-		if (guest_cbs->is_user_mode())
+	if (guest_state) {
+		if (guest_state & PERF_GUEST_USER)
 			misc |= PERF_RECORD_MISC_GUEST_USER;
 		else
 			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 9baa46185d94..524ad1f747bd 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2855,10 +2855,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		handled++;
 
 		guest_cbs = perf_get_guest_cbs();
-		if (unlikely(guest_cbs && guest_cbs->is_in_guest() &&
-			     guest_cbs->handle_intel_pt_intr))
-			guest_cbs->handle_intel_pt_intr();
-		else
+		if (likely(!guest_cbs || !guest_cbs->handle_intel_pt_intr()))
 			intel_pt_interrupt();
 	}
 
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 1ea4943a73d7..1080166fc0cf 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1874,7 +1874,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu);
 int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err);
 void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu);
 
-int kvm_is_in_guest(void);
+unsigned int kvm_guest_state(void);
 
 void __user *__x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa,
 				     u32 size);
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 0772bad9165c..5b68d4188de0 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -87,7 +87,7 @@ static void kvm_perf_overflow_intr(struct perf_event *perf_event,
 		 * woken up. So we should wake it, but this is impossible from
 		 * NMI context. Do it from irq work instead.
 		 */
-		if (!kvm_is_in_guest())
+		if (!kvm_guest_state())
 			irq_work_queue(&pmc_to_pmu(pmc)->irq_work);
 		else
 			kvm_make_request(KVM_REQ_PMI, pmc->vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index ffc6c2d73508..6cc66466f301 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8267,44 +8267,48 @@ static void kvm_timer_init(void)
 DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu);
 EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu);
 
-int kvm_is_in_guest(void)
+unsigned int kvm_guest_state(void)
 {
-	return __this_cpu_read(current_vcpu) != NULL;
-}
+	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
+	unsigned int state;
 
-static int kvm_is_user_mode(void)
-{
-	int user_mode = 3;
+	if (!vcpu)
+		return 0;
 
-	if (__this_cpu_read(current_vcpu))
-		user_mode = static_call(kvm_x86_get_cpl)(__this_cpu_read(current_vcpu));
+	state = PERF_GUEST_ACTIVE;
+	if (static_call(kvm_x86_get_cpl)(vcpu))
+		state |= PERF_GUEST_USER;
 
-	return user_mode != 0;
+	return state;
 }
 
-static unsigned long kvm_get_guest_ip(void)
+static unsigned long kvm_guest_get_ip(void)
 {
-	unsigned long ip = 0;
+	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
 
-	if (__this_cpu_read(current_vcpu))
-		ip = kvm_rip_read(__this_cpu_read(current_vcpu));
+	if (WARN_ON_ONCE(!vcpu))
+		return 0;
 
-	return ip;
+	return kvm_rip_read(vcpu);
 }
 
-static void kvm_handle_intel_pt_intr(void)
+static unsigned int kvm_handle_intel_pt_intr(void)
 {
 	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
 
+	/* '0' on failure so that the !PT case can use a RET0 static call. */
+	if (!vcpu)
+		return 0;
+
 	kvm_make_request(KVM_REQ_PMI, vcpu);
 	__set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT,
 			(unsigned long *)&vcpu->arch.pmu.global_status);
+	return 1;
 }
 
 static struct perf_guest_info_callbacks kvm_guest_cbs = {
-	.is_in_guest		= kvm_is_in_guest,
-	.is_user_mode		= kvm_is_user_mode,
-	.get_guest_ip		= kvm_get_guest_ip,
+	.state			= kvm_guest_state,
+	.get_ip			= kvm_guest_get_ip,
 	.handle_intel_pt_intr	= NULL,
 };
 
diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c
index e13b0b49fcdf..89dd6b1708b0 100644
--- a/arch/x86/xen/pmu.c
+++ b/arch/x86/xen/pmu.c
@@ -413,34 +413,29 @@ int pmu_apic_update(uint32_t val)
 }
 
 /* perf callbacks */
-static int xen_is_in_guest(void)
+static unsigned int xen_guest_state(void)
 {
 	const struct xen_pmu_data *xenpmu_data = get_xenpmu_data();
+	unsigned int state = 0;
 
 	if (!xenpmu_data) {
 		pr_warn_once("%s: pmudata not initialized\n", __func__);
-		return 0;
+		return state;
 	}
 
 	if (!xen_initial_domain() || (xenpmu_data->domain_id >= DOMID_SELF))
-		return 0;
+		return state;
 
-	return 1;
-}
+	state |= PERF_GUEST_ACTIVE;
 
-static int xen_is_user_mode(void)
-{
-	const struct xen_pmu_data *xenpmu_data = get_xenpmu_data();
-
-	if (!xenpmu_data) {
-		pr_warn_once("%s: pmudata not initialized\n", __func__);
-		return 0;
+	if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_PV) {
+		if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_USER)
+			state |= PERF_GUEST_USER;
+	} else if (xenpmu_data->pmu.r.regs.cpl & 3) {
+		state |= PERF_GUEST_USER;
 	}
 
-	if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_PV)
-		return (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_USER);
-	else
-		return !!(xenpmu_data->pmu.r.regs.cpl & 3);
+	return state;
 }
 
 static unsigned long xen_get_guest_ip(void)
@@ -456,9 +451,8 @@ static unsigned long xen_get_guest_ip(void)
 }
 
 static struct perf_guest_info_callbacks xen_guest_cbs = {
-	.is_in_guest            = xen_is_in_guest,
-	.is_user_mode           = xen_is_user_mode,
-	.get_guest_ip           = xen_get_guest_ip,
+	.state                  = xen_guest_state,
+	.get_ip			= xen_get_guest_ip,
 };
 
 /* Convert registers from Xen's format to Linux' */
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 317d4658afe9..f9be88a47434 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -26,11 +26,13 @@
 # include <asm/local64.h>
 #endif
 
+#define PERF_GUEST_ACTIVE	0x01
+#define PERF_GUEST_USER	0x02
+
 struct perf_guest_info_callbacks {
-	int				(*is_in_guest)(void);
-	int				(*is_user_mode)(void);
-	unsigned long			(*get_guest_ip)(void);
-	void				(*handle_intel_pt_intr)(void);
+	unsigned int			(*state)(void);
+	unsigned long			(*get_ip)(void);
+	unsigned int			(*handle_intel_pt_intr)(void);
 };
 
 #ifdef CONFIG_HAVE_HW_BREAKPOINT
diff --git a/kernel/events/core.c b/kernel/events/core.c
index d90a43572400..2e3dc9fbd5d9 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6483,6 +6483,7 @@ static void perf_pending_event(struct irq_work *entry)
 }
 
 struct perf_guest_info_callbacks *perf_guest_cbs;
+
 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
 	if (WARN_ON_ONCE(perf_guest_cbs))
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:06:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:06:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191999.342250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmS-00042O-1W; Wed, 22 Sep 2021 00:06:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191999.342250; Wed, 22 Sep 2021 00:06:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmR-00042H-UH; Wed, 22 Sep 2021 00:06:31 +0000
Received: by outflank-mailman (input) for mailman id 191999;
 Wed, 22 Sep 2021 00:06:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JH5Q=OM=flex--seanjc.bounces.google.com=3ZnNKYQYKCYc3plyunrzzrwp.nzx8py-op6pwwt343.8py02zupn4.z2r@srs-us1.protection.inumbo.net>)
 id 1mSpmP-0000pt-QR
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:29 +0000
Received: from mail-qk1-x74a.google.com (unknown [2607:f8b0:4864:20::74a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 63694b28-5d99-4911-bead-3bf7c454f1d3;
 Wed, 22 Sep 2021 00:05:59 +0000 (UTC)
Received: by mail-qk1-x74a.google.com with SMTP id
 ay30-20020a05620a179e00b00433294fbf97so6204076qkb.3
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:05:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63694b28-5d99-4911-bead-3bf7c454f1d3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=sNDCyCpfsN1q6fRIAZGHe8t08fZK9DQU77lycxeDgvE=;
        b=VfPFQTi2H68Pv6JmEb/KQ6WY/Mh3+l2kbL2XaDfVNNRWtfkT0wPW7iSQgW51WR5Ixu
         QaVgL8eDvDPLryj3vrQdNZbcnETMB9dGrhIzgmphuyjNlSU7aoy32NJ7ZHdZp0tssaUn
         WmTaDF7tt5GMvqHs/f51AneClTfDU4/Qx1Y7bh13Y1mJ7NTIjpZczmD07Ifh3YHzj8to
         lJYiaCZYA+JZO1GYoyMeT2BFWYalqIN+pUd4yUi7DujNd/X/ujswoWs0DebayZIqAv/y
         Meqb7X4l8tyF48vYwj84msbYaRs7+6wgTa49v9X0jrHAK7tmUtVCOQiRYMjdbLlI0bf9
         COSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=sNDCyCpfsN1q6fRIAZGHe8t08fZK9DQU77lycxeDgvE=;
        b=TxVY+vK+qfnUBplcTcyuv7sRCGjQ2f9BsLIcsXnhGM7SQ1hMqkbmEYxpjP7Ltb48Y9
         XyfTo63RqsgAAuV9CuD9I1A3tb5meS1VGwkBOBS6iiOy82UWlAlQMdv6AFtyyKR2K+aV
         LbClEY0zGwdjNUJGuTkk1RrPu7aWL8s5grnurXqERlY6cp6Q6aT8LBMjv/ih85skUebx
         zAtEZuj7fZSn9UWbcLBuNK6/Zl6gpn8OoyLj7K2Eq9d9EEGyeXM91ocWmU6aST+4rKB8
         mAuQTrv0s3RooIspSw0UT9uRsDKFiho9oLmajuOIJQF3FCuVE0FMX9uNKSTgYHY9Dd6E
         IhEg==
X-Gm-Message-State: AOAM532y1IpTvWGK1KHcjQ/48iNf7Cb1PsZ2lghbQhFXg0GAs18g4GDa
	ZhWbWGs3Nzsi18WcuyLoe3cKudBtL08=
X-Google-Smtp-Source: ABdhPJzkQ+TPTNghaiyjopK2Q9UXHXirkFCk7bz7Y2wHOEucRILtEQweuapuUXRwLMTZlbJ4rzpjCo5gfvs=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a25:1884:: with SMTP id 126mr25006798yby.114.1632269158936;
 Tue, 21 Sep 2021 17:05:58 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:24 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-8-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 07/16] perf: Add wrappers for invoking guest callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Add helpers for the guest callbacks to prepare for burying the callbacks
behind a Kconfig (it's a lot easier to provide a few stubs than to #ifdef
piles of code), and also to prepare for converting the callbacks to
static_call().  perf_instruction_pointer() in particular will have subtle
semantics with static_call(), as the "no callbacks" case will return 0 if
the callbacks are unregistered between querying guest state and getting
the IP.  Implement the change now to avoid a functional change when adding
static_call() support, and because the new helper needs to return
_something_ in this case.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/kernel/perf_callchain.c | 16 +++++-----------
 arch/x86/events/core.c             | 15 +++++----------
 arch/x86/events/intel/core.c       |  5 +----
 include/linux/perf_event.h         | 24 ++++++++++++++++++++++++
 4 files changed, 35 insertions(+), 25 deletions(-)

diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c
index 274dc3e11b6d..db04a55cee7e 100644
--- a/arch/arm64/kernel/perf_callchain.c
+++ b/arch/arm64/kernel/perf_callchain.c
@@ -102,9 +102,7 @@ compat_user_backtrace(struct compat_frame_tail __user *tail,
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	if (guest_cbs && guest_cbs->state()) {
+	if (perf_guest_state()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -149,10 +147,9 @@ static bool callchain_trace(void *data, unsigned long pc)
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stackframe frame;
 
-	if (guest_cbs && guest_cbs->state()) {
+	if (perf_guest_state()) {
 		/* We don't support guest os callchain now */
 		return;
 	}
@@ -163,18 +160,15 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	if (guest_cbs && guest_cbs->state())
-		return guest_cbs->get_ip();
+	if (perf_guest_state())
+		return perf_guest_get_ip();
 
 	return instruction_pointer(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
+	unsigned int guest_state = perf_guest_state();
 	int misc = 0;
 
 	if (guest_state) {
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 3a7630fdd340..d20e4f8d1aef 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -2761,11 +2761,10 @@ static bool perf_hw_regs(struct pt_regs *regs)
 void
 perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct unwind_state state;
 	unsigned long addr;
 
-	if (guest_cbs && guest_cbs->state()) {
+	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2865,11 +2864,10 @@ perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry_ctx *ent
 void
 perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
 	struct stack_frame frame;
 	const struct stack_frame __user *fp;
 
-	if (guest_cbs && guest_cbs->state()) {
+	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
 	}
@@ -2946,18 +2944,15 @@ static unsigned long code_segment_base(struct pt_regs *regs)
 
 unsigned long perf_instruction_pointer(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	if (guest_cbs && guest_cbs->state())
-		return guest_cbs->get_ip();
+	if (perf_guest_state())
+		return perf_guest_get_ip();
 
 	return regs->ip + code_segment_base(regs);
 }
 
 unsigned long perf_misc_flags(struct pt_regs *regs)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
+	unsigned int guest_state = perf_guest_state();
 	int misc = 0;
 
 	if (guest_state) {
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 524ad1f747bd..f5b02017ba16 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2786,7 +2786,6 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 {
 	struct perf_sample_data data;
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	struct perf_guest_info_callbacks *guest_cbs;
 	int bit;
 	int handled = 0;
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
@@ -2853,9 +2852,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 	 */
 	if (__test_and_clear_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT, (unsigned long *)&status)) {
 		handled++;
-
-		guest_cbs = perf_get_guest_cbs();
-		if (likely(!guest_cbs || !guest_cbs->handle_intel_pt_intr()))
+		if (!perf_guest_handle_intel_pt_intr())
 			intel_pt_interrupt();
 	}
 
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index f9be88a47434..c0a6eaf55fb1 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1247,6 +1247,30 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
 	/* Prevent reloading between a !NULL check and dereferences. */
 	return READ_ONCE(perf_guest_cbs);
 }
+static inline unsigned int perf_guest_state(void)
+{
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	return guest_cbs ? guest_cbs->state() : 0;
+}
+static inline unsigned long perf_guest_get_ip(void)
+{
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	/*
+	 * Arbitrarily return '0' in the unlikely scenario that the callbacks
+	 * are unregistered between checking guest state and getting the IP.
+	 */
+	return guest_cbs ? guest_cbs->get_ip() : 0;
+}
+static inline unsigned int perf_guest_handle_intel_pt_intr(void)
+{
+	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
+
+	if (guest_cbs && guest_cbs->handle_intel_pt_intr)
+		return guest_cbs->handle_intel_pt_intr();
+	return 0;
+}
 extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
 extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
 
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:07:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192015.342261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmw-0005Wu-D8; Wed, 22 Sep 2021 00:07:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192015.342261; Wed, 22 Sep 2021 00:07:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmw-0005W6-96; Wed, 22 Sep 2021 00:07:02 +0000
Received: by outflank-mailman (input) for mailman id 192015;
 Wed, 22 Sep 2021 00:07:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Sx30=OM=flex--seanjc.bounces.google.com=3aXNKYQYKCYo6so1xqu22uzs.q20Bs1-rs9szzw676.Bs1352xsq7.25u@srs-us1.protection.inumbo.net>)
 id 1mSpmZ-0000pt-Qt
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:39 +0000
Received: from mail-qk1-x749.google.com (unknown [2607:f8b0:4864:20::749])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bf15b4d9-f8b2-4814-84fa-bd9e3ba78229;
 Wed, 22 Sep 2021 00:06:01 +0000 (UTC)
Received: by mail-qk1-x749.google.com with SMTP id
 u22-20020a05620a455600b004333191e844so5516211qkp.14
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf15b4d9-f8b2-4814-84fa-bd9e3ba78229
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=hxRuLxBMmtgulqNWBhVm63T2q6otFpWhzuzOzQjwuz0=;
        b=AEH25mNfVOzaOi5d6c3BruylgDezaEPCOtesGovpuwBViHkaZN0oJqqsI3ayCNTU47
         g0/dG78HVaz39C6dylQWgabX8vR0or8V+qWDB8Hco6j0Zy1BH84lLC4DdigOee9qRXx5
         Z0GBtiUbse1uPHuGSgRHXOGN+MswC1H/vEN/9eWkP/kvKiaoxNiLyYnf9+trtkLu2SB3
         N3xMQRLYQClaWUgxFLgSR/Ws7M+M5HGzh33BkasWfYaCdmeGEgaOiZCARYtsiDGcOzZZ
         QmwqXobkE3+q1ncVsZzvdLP7llxDj6xtQ34TQ5DGMtVmRDULQ5ytAScISNgwHfdTAtDC
         Jt9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=hxRuLxBMmtgulqNWBhVm63T2q6otFpWhzuzOzQjwuz0=;
        b=DCjlIhZq05rHEBV73WqMCCmTd3kECjMalkAPA+txIbwHnM9fdErNvGtHJLnbz1pvGy
         LPppPqJkcelDjUm9w+nR5Ar635mgfAwyx68NG6nmG9BVaGQ9vHNZrQzo+0PiyRr114tl
         YBr2ERAHT1WD8Pmh6wqhj0bxyrog9CNWvQ6paS6BfH4hIDVThfwcG7kfcJr6M74iyoq+
         WIhDr9TZk5gpr1/aC7Y40yMvsF3f6AiaKNBvk5WhCQ+/TS8VRurvuh4lR97GDA/kgLwA
         q50s2GonJDTZP69anjVVUed/vOMr+YrlZ6GZqPwGerdzt6x99k0jG+puxysFgYaaXJVY
         Gx9g==
X-Gm-Message-State: AOAM5314yL2SmDEUOO/6X2WgeqpINadXPOze4WJ4WhYoSDyUQEmr50nh
	kTB4586qc/rhvc9sfB/LuIh03zTchYE=
X-Google-Smtp-Source: ABdhPJzZXZc/uBJgDh9/jjk115DCwSnPOWZBsVnqx8QA2MhIXqiGwOq328gHdkdfukulUyubJVoFUxWXEpY=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:6aa:: with SMTP id
 s10mr33034215qvz.56.1632269161091; Tue, 21 Sep 2021 17:06:01 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:25 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-9-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 08/16] perf: Force architectures to opt-in to guest callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Introduce GUEST_PERF_EVENTS and require architectures to select it to
allow registering and using guest callbacks in perf.  This will hopefully
make it more difficult for new architectures to add useless "support" for
guest callbacks, e.g. via copy+paste.

Stubbing out the helpers has the happy bonus of avoiding a load of
perf_guest_cbs when GUEST_PERF_EVENTS=n on arm64/x86.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/kvm/Kconfig     | 1 +
 arch/x86/kvm/Kconfig       | 1 +
 arch/x86/xen/Kconfig       | 1 +
 include/linux/perf_event.h | 6 ++++++
 init/Kconfig               | 4 ++++
 kernel/events/core.c       | 2 ++
 6 files changed, 15 insertions(+)

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index a4eba0908bfa..f2121404c7c6 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -37,6 +37,7 @@ menuconfig KVM
 	select HAVE_KVM_IRQ_BYPASS
 	select HAVE_KVM_VCPU_RUN_PID_CHANGE
 	select SCHED_INFO
+	select GUEST_PERF_EVENTS if PERF_EVENTS
 	help
 	  Support hosting virtualized guest machines.
 
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index ac69894eab88..699bf786fbce 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -36,6 +36,7 @@ config KVM
 	select KVM_MMIO
 	select SCHED_INFO
 	select PERF_EVENTS
+	select GUEST_PERF_EVENTS
 	select HAVE_KVM_MSI
 	select HAVE_KVM_CPU_RELAX_INTERCEPT
 	select HAVE_KVM_NO_POLL
diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
index afc1da68b06d..d07595a9552d 100644
--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -23,6 +23,7 @@ config XEN_PV
 	select PARAVIRT_XXL
 	select XEN_HAVE_PVMMU
 	select XEN_HAVE_VPMU
+	select GUEST_PERF_EVENTS
 	help
 	  Support running as a Xen PV guest.
 
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index c0a6eaf55fb1..eefa197d5354 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1238,6 +1238,7 @@ extern void perf_event_bpf_event(struct bpf_prog *prog,
 				 enum perf_bpf_event_type type,
 				 u16 flags);
 
+#ifdef CONFIG_GUEST_PERF_EVENTS
 extern struct perf_guest_info_callbacks *perf_guest_cbs;
 static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
 {
@@ -1273,6 +1274,11 @@ static inline unsigned int perf_guest_handle_intel_pt_intr(void)
 }
 extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
 extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
+#else
+static inline unsigned int perf_guest_state(void)		 { return 0; }
+static inline unsigned long perf_guest_get_ip(void)		 { return 0; }
+static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }
+#endif /* CONFIG_GUEST_PERF_EVENTS */
 
 extern void perf_event_exec(void);
 extern void perf_event_comm(struct task_struct *tsk, bool exec);
diff --git a/init/Kconfig b/init/Kconfig
index 55f9f7738ebb..acc7e8ba4563 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1776,6 +1776,10 @@ config HAVE_PERF_EVENTS
 	help
 	  See tools/perf/design.txt for details.
 
+config GUEST_PERF_EVENTS
+	bool
+	depends on HAVE_PERF_EVENTS
+
 config PERF_USE_VMALLOC
 	bool
 	help
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 2e3dc9fbd5d9..c6ec05809f54 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6482,6 +6482,7 @@ static void perf_pending_event(struct irq_work *entry)
 		perf_swevent_put_recursion_context(rctx);
 }
 
+#ifdef CONFIG_GUEST_PERF_EVENTS
 struct perf_guest_info_callbacks *perf_guest_cbs;
 
 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
@@ -6502,6 +6503,7 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 	synchronize_rcu();
 }
 EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
+#endif
 
 static void
 perf_output_sample_regs(struct perf_output_handle *handle,
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:07:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:07:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192016.342272 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmx-0005pU-MI; Wed, 22 Sep 2021 00:07:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192016.342272; Wed, 22 Sep 2021 00:07:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpmx-0005pN-I7; Wed, 22 Sep 2021 00:07:03 +0000
Received: by outflank-mailman (input) for mailman id 192016;
 Wed, 22 Sep 2021 00:07:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sKTv=OM=flex--seanjc.bounces.google.com=3a3NKYQYKCYw8uq3zsw44w1u.s42Du3-tuBu11y898.Du3574zus9.47w@srs-us1.protection.inumbo.net>)
 id 1mSpme-0000pt-Qp
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:44 +0000
Received: from mail-qk1-x74a.google.com (unknown [2607:f8b0:4864:20::74a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0c3c9709-3be7-4442-856f-1126aba6663d;
 Wed, 22 Sep 2021 00:06:03 +0000 (UTC)
Received: by mail-qk1-x74a.google.com with SMTP id
 70-20020a370b49000000b003d2f5f0dcc6so5794724qkl.9
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c3c9709-3be7-4442-856f-1126aba6663d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=lHJOof24VqkiYhFcIRNTWKzKjki/p0Ze5CnPJ3pN3mI=;
        b=NAVCfHnfawHtrL2iCYw8HgTBlFYN7yri+HxZMhsBlfVlSZIQEmh1AdRNXtYQ9FoGZS
         jRIrxb0XEtBmkG58IBs5w+mP9+jXc2C3HLEK37fIIPlowXZ4g2A0PlKvfuqb2ADJxmfT
         gWcyAqrNLMzTqwoc6qruf1z2Si6VfevO8CbFRhssYP8/HJFWfja6Iw9rCUyZd9oSQg+F
         9XbckVVG4ADd3HGel6flTZI1kY7ZlS2ePo2n4ME18ZOc0prkN5QAWF7paKBJF7TwSTpL
         +vKX7u3jx0Z8b6TwojXazIBTpdi6hKJXbUEPbqRRJ9J3bLcqDW1CgpBqnGXO30fQFOsf
         UrIw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=lHJOof24VqkiYhFcIRNTWKzKjki/p0Ze5CnPJ3pN3mI=;
        b=iZT5JDYOfE/lhBtN73esX6l8f/B74MUnF+NAQq1YwunmAlLQvsMQfiLktBeF1W8FWr
         3NNpJs2ZxziwPhIbAd1EgFdmK0mDa1haOu0/m20QXbwSYW7TgjE+5atVTFfoX8WgORgA
         FxpIs/Xd2lIiJxYfaO5CnlPXZcZxnvtHzEb2x8cr910DaTtKLeyzYuJZTFmcisY775b+
         QJeIaLexoDGLnyrt+2N2kYPwt+an6B8mj9c+/2R/aTfQ228Jn/Dt/OElXSWa7G4KE2xQ
         xhYsi/X+GZ0pdyc3PYfMRGlalzeIM1bEwi6UDEQ8EFMpTObTB28VD12kptnm2D5oTPEZ
         HYvA==
X-Gm-Message-State: AOAM531dYyYJ3pRpWGqH9fp9GCzgLbaHid9umHj5AK48KwSQzpQNe/gS
	Lny6JEaaaHJh8c+NWL6iddUVzDjhM3k=
X-Google-Smtp-Source: ABdhPJxiVysqs+seAL8tPuZXn+zjjV1XTgLeF1Hv4M206n5zRNvGf9PMhGCvhpJwuIHM26SWahcPIvZUjtg=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:732:: with SMTP id
 c18mr33750621qvz.9.1632269163200; Tue, 21 Sep 2021 17:06:03 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:26 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-10-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 09/16] perf/core: Use static_call to optimize perf_guest_info_callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Use static_call to optimize perf's guest callbacks on arm64 and x86,
which are now the only architectures that define the callbacks.  Use
DEFINE_STATIC_CALL_RET0 as the default/NULL for all guest callbacks, as
the callback semantics are that a return value '0' means "not in guest".

static_call obviously avoids the overhead of CONFIG_RETPOLINE=y, but is
also advantageous versus other solutions, e.g. per-cpu callbacks, in that
a per-cpu memory load is not needed to detect the !guest case.

Based on code from Peter and Like.

Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Like Xu <like.xu.linux@gmail.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 include/linux/perf_event.h | 28 ++++++----------------------
 kernel/events/core.c       | 15 +++++++++++++++
 2 files changed, 21 insertions(+), 22 deletions(-)

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index eefa197d5354..d582dfeb4e20 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1240,37 +1240,21 @@ extern void perf_event_bpf_event(struct bpf_prog *prog,
 
 #ifdef CONFIG_GUEST_PERF_EVENTS
 extern struct perf_guest_info_callbacks *perf_guest_cbs;
-static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
-{
-	/* Reg/unreg perf_guest_cbs waits for readers via synchronize_rcu(). */
-	lockdep_assert_preemption_disabled();
+DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs->state);
+DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
+DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
 
-	/* Prevent reloading between a !NULL check and dereferences. */
-	return READ_ONCE(perf_guest_cbs);
-}
 static inline unsigned int perf_guest_state(void)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	return guest_cbs ? guest_cbs->state() : 0;
+	return static_call(__perf_guest_state)();
 }
 static inline unsigned long perf_guest_get_ip(void)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	/*
-	 * Arbitrarily return '0' in the unlikely scenario that the callbacks
-	 * are unregistered between checking guest state and getting the IP.
-	 */
-	return guest_cbs ? guest_cbs->get_ip() : 0;
+	return static_call(__perf_guest_get_ip)();
 }
 static inline unsigned int perf_guest_handle_intel_pt_intr(void)
 {
-	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
-
-	if (guest_cbs && guest_cbs->handle_intel_pt_intr)
-		return guest_cbs->handle_intel_pt_intr();
-	return 0;
+	return static_call(__perf_guest_handle_intel_pt_intr)();
 }
 extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
 extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
diff --git a/kernel/events/core.c b/kernel/events/core.c
index c6ec05809f54..79c8ee1778a4 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6485,12 +6485,23 @@ static void perf_pending_event(struct irq_work *entry)
 #ifdef CONFIG_GUEST_PERF_EVENTS
 struct perf_guest_info_callbacks *perf_guest_cbs;
 
+DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs->state);
+DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
+DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
+
 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 {
 	if (WARN_ON_ONCE(perf_guest_cbs))
 		return;
 
 	WRITE_ONCE(perf_guest_cbs, cbs);
+	static_call_update(__perf_guest_state, cbs->state);
+	static_call_update(__perf_guest_get_ip, cbs->get_ip);
+
+	/* Implementing ->handle_intel_pt_intr is optional. */
+	if (cbs->handle_intel_pt_intr)
+		static_call_update(__perf_guest_handle_intel_pt_intr,
+				   cbs->handle_intel_pt_intr);
 }
 EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
 
@@ -6500,6 +6511,10 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
 		return;
 
 	WRITE_ONCE(perf_guest_cbs, NULL);
+	static_call_update(__perf_guest_state, (void *)&__static_call_return0);
+	static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0);
+	static_call_update(__perf_guest_handle_intel_pt_intr,
+			   (void *)&__static_call_return0);
 	synchronize_rcu();
 }
 EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:07:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:07:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192019.342283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpn1-0006Hj-WF; Wed, 22 Sep 2021 00:07:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192019.342283; Wed, 22 Sep 2021 00:07:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpn1-0006HY-RU; Wed, 22 Sep 2021 00:07:07 +0000
Received: by outflank-mailman (input) for mailman id 192019;
 Wed, 22 Sep 2021 00:07:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ag2R=OM=flex--seanjc.bounces.google.com=3bXNKYQYKCY4Aws51uy66y3w.u64Fw5-vwDw330ABA.Fw57961wuB.69y@srs-us1.protection.inumbo.net>)
 id 1mSpmo-0000pt-R4
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:06:54 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c805761d-0a5d-47c9-a408-346e5a994c2d;
 Wed, 22 Sep 2021 00:06:05 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 q17-20020a0cf5d1000000b00380e60ef363so7783033qvm.9
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c805761d-0a5d-47c9-a408-346e5a994c2d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=AVVcTJjDaboxmrWd01mlihP+thyDmwt+Wb6UJcWJ0SA=;
        b=qnB3SQiaaF6WC9mC3PxEeBeR+5lv1WF2OZ5dAEYuga9do3ZxizL42FhUWbNnGQsLJy
         kFlr7gl1wrMF1pi+KoC28DCxMnwiq7dfm7LkdmgwbfFIj+yZF87g+RiucCIIlaGk9z2L
         ul3JXkYMG1M7a8Mo36cpf3Rry/Wo1HIBeW17Zi62PJjYFnKdmXhaubK0jiBQ5d6UqeTU
         1+L7Uftzwd0fGoYUuho0kFZqMWYiIl8q0A+4IRcVGFEDRYPi1z57sEg8moGTEmHEg6dI
         5vqXyNGrS9ZLp2cByIwk1QgXLXs643+yydpAgPnozKp/YE6+BzivoiQrohhBs0bP0EIW
         FUEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=AVVcTJjDaboxmrWd01mlihP+thyDmwt+Wb6UJcWJ0SA=;
        b=LuaJOZUdBB5Vh/cA4ZQ5hopVFdWodf1ZhLa+rVm+xPKi6I0hIUACk1ANf0v1w0hKoj
         fY+CkuUKtk+4noIgPC8FAhZL6xfzBwkIDNGpFsHHpd32M1UvMt4BGlEbIdkQIiwHfxdR
         ttNTP7HqYbu/8inAOR1HNJUXEK3g2UlcxnEmdXtZnYHxXO6trZ5ompWrpYEhcEY9HvU4
         2H2I2u1VnWjG0udrhgdR7cYVMiE5mj/unGjqfT++c128+NWZ7pKbbLwVAUSOA0XFdeRw
         Yu3OjLVqxrddWvLVBHNAep7qewkffyvt9W+U2XlTI3w2b0/vIz1Bj8wjYkp9JbH2jzmq
         sTjw==
X-Gm-Message-State: AOAM533ZYoDIfM5lPbzcU6BpO2uhOG/KGNJVLCGo+1PBh5oVrJXONEv2
	xHH3KxOOEDoaulzHGF2uTXxTurDv2fg=
X-Google-Smtp-Source: ABdhPJx1BhnGeS4TuFK4bgjBOJbTLeLAb6P4HGUosvLJGU/FqZfGimhs446IdcpbQjbCIE0AbY4YgA9+Mxg=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6214:1430:: with SMTP id
 o16mr30643490qvx.66.1632269165263; Tue, 21 Sep 2021 17:06:05 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:27 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-11-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 10/16] KVM: x86: Drop current_vcpu for kvm_running_vcpu +
 kvm_arch_vcpu variable
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Use the generic kvm_running_vcpu plus a new 'handling_intr_from_guest'
variable in kvm_arch_vcpu instead of the semi-redundant current_vcpu.
kvm_before/after_interrupt() must be called while the vCPU is loaded,
(which protects against preemption), thus kvm_running_vcpu is guaranteed
to be non-NULL when handling_intr_from_guest is non-zero.

Switching to kvm_get_running_vcpu() will allows moving KVM's perf
callbacks to generic code, and the new flag will be used in a future
patch to more precisely identify the "NMI from guest" case.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h |  3 +--
 arch/x86/kvm/pmu.c              |  2 +-
 arch/x86/kvm/x86.c              | 21 ++++++++++++---------
 arch/x86/kvm/x86.h              | 10 ++++++----
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 1080166fc0cf..2d86a2dfc775 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -763,6 +763,7 @@ struct kvm_vcpu_arch {
 	unsigned nmi_pending; /* NMI queued after currently running handler */
 	bool nmi_injected;    /* Trying to inject an NMI this entry */
 	bool smi_pending;    /* SMI queued after currently running handler */
+	u8 handling_intr_from_guest;
 
 	struct kvm_mtrr mtrr_state;
 	u64 pat;
@@ -1874,8 +1875,6 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu);
 int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err);
 void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu);
 
-unsigned int kvm_guest_state(void);
-
 void __user *__x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa,
 				     u32 size);
 bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 5b68d4188de0..eef48258e50f 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -87,7 +87,7 @@ static void kvm_perf_overflow_intr(struct perf_event *perf_event,
 		 * woken up. So we should wake it, but this is impossible from
 		 * NMI context. Do it from irq work instead.
 		 */
-		if (!kvm_guest_state())
+		if (!kvm_handling_nmi_from_guest(pmc->vcpu))
 			irq_work_queue(&pmc_to_pmu(pmc)->irq_work);
 		else
 			kvm_make_request(KVM_REQ_PMI, pmc->vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6cc66466f301..24a6faa07442 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8264,15 +8264,17 @@ static void kvm_timer_init(void)
 			  kvmclock_cpu_online, kvmclock_cpu_down_prep);
 }
 
-DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu);
-EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu);
+static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu)
+{
+	return vcpu && vcpu->arch.handling_intr_from_guest;
+}
 
-unsigned int kvm_guest_state(void)
+static unsigned int kvm_guest_state(void)
 {
-	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
 	unsigned int state;
 
-	if (!vcpu)
+	if (!kvm_pmi_in_guest(vcpu))
 		return 0;
 
 	state = PERF_GUEST_ACTIVE;
@@ -8284,9 +8286,10 @@ unsigned int kvm_guest_state(void)
 
 static unsigned long kvm_guest_get_ip(void)
 {
-	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
 
-	if (WARN_ON_ONCE(!vcpu))
+	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
+	if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu)))
 		return 0;
 
 	return kvm_rip_read(vcpu);
@@ -8294,10 +8297,10 @@ static unsigned long kvm_guest_get_ip(void)
 
 static unsigned int kvm_handle_intel_pt_intr(void)
 {
-	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
 
 	/* '0' on failure so that the !PT case can use a RET0 static call. */
-	if (!vcpu)
+	if (!kvm_pmi_in_guest(vcpu))
 		return 0;
 
 	kvm_make_request(KVM_REQ_PMI, vcpu);
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 7d66d63dc55a..a9c107e7c907 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -387,18 +387,20 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm)
 	return kvm->arch.cstate_in_guest;
 }
 
-DECLARE_PER_CPU(struct kvm_vcpu *, current_vcpu);
-
 static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu)
 {
-	__this_cpu_write(current_vcpu, vcpu);
+	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 1);
 }
 
 static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu)
 {
-	__this_cpu_write(current_vcpu, NULL);
+	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 0);
 }
 
+static inline bool kvm_handling_nmi_from_guest(struct kvm_vcpu *vcpu)
+{
+	return !!vcpu->arch.handling_intr_from_guest;
+}
 
 static inline bool kvm_pat_valid(u64 data)
 {
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192041.342316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwe-0000o0-MU; Wed, 22 Sep 2021 00:17:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192041.342316; Wed, 22 Sep 2021 00:17:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwe-0000nB-HY; Wed, 22 Sep 2021 00:17:04 +0000
Received: by outflank-mailman (input) for mailman id 192041;
 Wed, 22 Sep 2021 00:17:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Dbqz=OM=flex--seanjc.bounces.google.com=3c3NKYQYKCZQG2yB704CC492.0CAL2B-12J2996GHG.L2BDFC720H.CF4@srs-us1.protection.inumbo.net>)
 id 1mSpnD-0000pt-S8
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:19 +0000
Received: from mail-qk1-x74a.google.com (unknown [2607:f8b0:4864:20::74a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 860f2a03-19c6-484d-961b-c8714f1a266e;
 Wed, 22 Sep 2021 00:06:12 +0000 (UTC)
Received: by mail-qk1-x74a.google.com with SMTP id
 h4-20020a05620a244400b004334ede5036so5562383qkn.13
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 860f2a03-19c6-484d-961b-c8714f1a266e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=y8OYsYpDs1HLDvFjdurM0d7KqhhKHNQa2gUw1mVnDLE=;
        b=VgUrfICBEU208O8puU1yf0798dMNWQPdVEv1umOOfyY7+MLL+nmtJuMQRYhGk6ACPB
         kV7DCUw8x7TThTD9JohJr12iNp9IVCl2gpkkOQsZfWo7I0JLG16fmuO2MGI3z76Slotc
         tAS3raY0P7YO93IW++jVoxXSx7bgW6LXHh9Lg2GYrEnit4mDk0PviAE/NTWZXLAHL0sx
         tQX8bpmjgklBcXInIIRO8BoGyPy1BMCOolkyTRz+cls0PNm7fOz8/SKH73cd2RwpkSM5
         C8zyjgBkrIzTe2Xfi0y9KCgCoaxdmORS5u5QyM9nNkpibluFvb/Koa8wFffrHGQL3WIt
         pSLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=y8OYsYpDs1HLDvFjdurM0d7KqhhKHNQa2gUw1mVnDLE=;
        b=IjIOTsTtBD8GeCtYKE2ttl2/szwwo801ikfR2mi1n8dCPoFAH5glNTAMTKOhJesH3p
         41f2WaBxp5Zj2NPFDx7zGFKffiFsNPrwQBD03NSwDdDFJiwIksLvsWgG/tKkUTHNn3K5
         NSXwgyp99UR/4yLbKyao8Nj6c/dA17foZYPRFFlusBfGRjsnzbC/hLn5sQCZq3NpPAtQ
         HcWIv9tdKs5L+r7TwrtduafqnTTlM97h/x36zuCouYQIc6a545xjv7YVbOTcTXPx3T5R
         67va3CPNUmWZkKHpEdzgedCGEHUY7vMtLboU6vvBwyLpFRQtLL8n5gUEQuRYtZH2fkEZ
         eSGw==
X-Gm-Message-State: AOAM530eGoAHTayqPlJmztfMgaQENqBy2wp6geefBLLHidFDTagT64Ax
	sggNJoQqDiE/UIq2KHHFSVIkaCLNYbY=
X-Google-Smtp-Source: ABdhPJyHtapTp0OfNUKTfdvcX6WW6HZmgDH9RXeTSlkheaqjZb44gygT11cJLtQ1bPei5p+RUE6ibKLU+f0=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a25:d9cc:: with SMTP id q195mr41667087ybg.15.1632269171910;
 Tue, 21 Sep 2021 17:06:11 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:30 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-14-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 13/16] KVM: x86: Move Intel Processor Trace interrupt
 handler to vmx.c
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Now that all state needed for VMX's PT interrupt handler is exposed to
vmx.c (specifically the currently running vCPU), move the handler into
vmx.c where it belongs.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h |  2 +-
 arch/x86/kvm/vmx/vmx.c          | 22 +++++++++++++++++++++-
 arch/x86/kvm/x86.c              | 20 +-------------------
 3 files changed, 23 insertions(+), 21 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 6efe4e03a6d2..d40814b57ae8 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1495,7 +1495,7 @@ struct kvm_x86_init_ops {
 	int (*disabled_by_bios)(void);
 	int (*check_processor_compatibility)(void);
 	int (*hardware_setup)(void);
-	bool (*intel_pt_intr_in_guest)(void);
+	unsigned int (*handle_intel_pt_intr)(void);
 
 	struct kvm_x86_ops *runtime_ops;
 };
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 61a4f5ff2acd..33f92febe3ce 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7687,6 +7687,20 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
 };
 
+static unsigned int vmx_handle_intel_pt_intr(void)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+	/* '0' on failure so that the !PT case can use a RET0 static call. */
+	if (!kvm_arch_pmi_in_guest(vcpu))
+		return 0;
+
+	kvm_make_request(KVM_REQ_PMI, vcpu);
+	__set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT,
+		  (unsigned long *)&vcpu->arch.pmu.global_status);
+	return 1;
+}
+
 static __init void vmx_setup_user_return_msrs(void)
 {
 
@@ -7713,6 +7727,8 @@ static __init void vmx_setup_user_return_msrs(void)
 		kvm_add_user_return_msr(vmx_uret_msrs_list[i]);
 }
 
+static struct kvm_x86_init_ops vmx_init_ops __initdata;
+
 static __init int hardware_setup(void)
 {
 	unsigned long host_bndcfgs;
@@ -7873,6 +7889,10 @@ static __init int hardware_setup(void)
 		return -EINVAL;
 	if (!enable_ept || !cpu_has_vmx_intel_pt())
 		pt_mode = PT_MODE_SYSTEM;
+	if (pt_mode == PT_MODE_HOST_GUEST)
+		vmx_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr;
+	else
+		vmx_init_ops.handle_intel_pt_intr = NULL;
 
 	setup_default_sgx_lepubkeyhash();
 
@@ -7898,7 +7918,7 @@ static struct kvm_x86_init_ops vmx_init_ops __initdata = {
 	.disabled_by_bios = vmx_disabled_by_bios,
 	.check_processor_compatibility = vmx_check_processor_compat,
 	.hardware_setup = hardware_setup,
-	.intel_pt_intr_in_guest = vmx_pt_mode_is_host_guest,
+	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vmx_x86_ops,
 };
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1bea616402e6..b79b2d29260d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8264,20 +8264,6 @@ static void kvm_timer_init(void)
 			  kvmclock_cpu_online, kvmclock_cpu_down_prep);
 }
 
-static unsigned int kvm_handle_intel_pt_intr(void)
-{
-	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
-
-	/* '0' on failure so that the !PT case can use a RET0 static call. */
-	if (!kvm_arch_pmi_in_guest(vcpu))
-		return 0;
-
-	kvm_make_request(KVM_REQ_PMI, vcpu);
-	__set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT,
-			(unsigned long *)&vcpu->arch.pmu.global_status);
-	return 1;
-}
-
 #ifdef CONFIG_X86_64
 static void pvclock_gtod_update_fn(struct work_struct *work)
 {
@@ -11031,11 +11017,7 @@ int kvm_arch_hardware_setup(void *opaque)
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
 	kvm_ops_static_call_update();
 
-	/* Temporary ugliness. */
-	if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest())
-		kvm_register_perf_callbacks(kvm_handle_intel_pt_intr);
-	else
-		kvm_register_perf_callbacks(NULL);
+	kvm_register_perf_callbacks(ops->handle_intel_pt_intr);
 
 	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
 		supported_xss = 0;
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192032.342294 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwd-0000G9-39; Wed, 22 Sep 2021 00:17:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192032.342294; Wed, 22 Sep 2021 00:17:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwd-0000G2-03; Wed, 22 Sep 2021 00:17:03 +0000
Received: by outflank-mailman (input) for mailman id 192032;
 Wed, 22 Sep 2021 00:17:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mFIr=OM=flex--seanjc.bounces.google.com=3eHNKYQYKCZkL73GC59HH9E7.5HFQ7G-67O7EEBLML.Q7GIKHC75M.HK9@srs-us1.protection.inumbo.net>)
 id 1mSpnS-0000pt-SV
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:34 +0000
Received: from mail-qv1-xf49.google.com (unknown [2607:f8b0:4864:20::f49])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1332bf12-5f7c-4a6e-a8fc-38bad2e7c2d1;
 Wed, 22 Sep 2021 00:06:17 +0000 (UTC)
Received: by mail-qv1-xf49.google.com with SMTP id
 w10-20020a0cb54a000000b0037a9848b92fso8990393qvd.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1332bf12-5f7c-4a6e-a8fc-38bad2e7c2d1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=6Icjr/ACV1XsfVUfKjKZi9JPiJa8aiN8zAxRj7zkg58=;
        b=lodyKSv5s5L9FuAs5M0G0kMx/zVrl93Ee68EIaXohxy+LQ+d8m+TXXDCkue1vWSnVV
         ViQYh0dHzV9iJNwFB6yjTYg0JxCSrOowA0Ye/c7r2V2aOqrV/LmltE8U0Zt+k8GFxu9h
         2la1t1UJWIkOSVjh2yV0uEzAZiwrdvHA0zHKk46Dq/OX6UEK0uGcCRXWfsZIO3eQTKqS
         ConZVcLZT+4wy0TFLTPjpjtLAOIGL9YLutbbFVl73zcBDJOID6vgd5v4Ln2GZlWHiqop
         o86J2gkYFymM8L2O6tTywYJkqskrpEBIkrTz73Gvpj/PmQ5jYyO0eaPS4tdU3c0It1S4
         A6OA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=6Icjr/ACV1XsfVUfKjKZi9JPiJa8aiN8zAxRj7zkg58=;
        b=RsC5Gw6fMmWMVJXp8G2aWGu5Ze4LxGdkgbGhLx9XVmK59ItRuoJ3pQKG6mJK1/VBjo
         EvJ85ntvKc/LZBIsumKDg05fmcEKytKWbWKOFNk1yaN8qdXlIFTG+toslNwq1nhFsJI2
         nR0sJlCyxmq6FZj7jQOUrMWmxi82tyQBBeLvz4g6TdP1DBMUaiaU8tHIG/vUGxVeG5tZ
         TkzRQQm9n7VCCDTE4cDFF6rPjlKBpRqcz0CypoTvTA8/7G9wch91pjBvMvC3dLkwGnec
         RvydSoxBZZHgf7lzImG9hUDQXwudZpU7AkJghYGlWmtnsJHn/92T/fyIliWGMbo5qDso
         9jeg==
X-Gm-Message-State: AOAM533RCwWVe7c5h2bXhq31HBrCBq2A9PyT8+hZRgLvSInnHv8Bg2pB
	uvnj2j3s2/K8kD4mEP4hJHmBXJegEKc=
X-Google-Smtp-Source: ABdhPJzlQAzmSdl3nY/YBM5ZcGipBomoLCiu3CqrYXxVZ0o44Jz0iqe4JnQizT8BW9SKS2QFzZIdD02YgoQ=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a25:7350:: with SMTP id o77mr39621975ybc.549.1632269176658;
 Tue, 21 Sep 2021 17:06:16 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:32 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-16-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 15/16] KVM: arm64: Drop perf.c and fold its tiny bits of
 code into arm.c / pmu.c
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Call KVM's (un)register perf callbacks helpers directly from arm.c, and
move the PMU bits into pmu.c and rename the related helper accordingly.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/include/asm/kvm_host.h |  3 ---
 arch/arm64/kvm/Makefile           |  2 +-
 arch/arm64/kvm/arm.c              |  6 ++++--
 arch/arm64/kvm/perf.c             | 27 ---------------------------
 arch/arm64/kvm/pmu.c              |  8 ++++++++
 include/kvm/arm_pmu.h             |  1 +
 6 files changed, 14 insertions(+), 33 deletions(-)
 delete mode 100644 arch/arm64/kvm/perf.c

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 828b6eaa2c56..f141ac65f4f1 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -670,9 +670,6 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len);
 int kvm_handle_mmio_return(struct kvm_vcpu *vcpu);
 int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa);
 
-void kvm_perf_init(void);
-void kvm_perf_teardown(void);
-
 #ifdef CONFIG_GUEST_PERF_EVENTS
 static inline bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu)
 {
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 989bb5dad2c8..0bcc378b7961 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -12,7 +12,7 @@ obj-$(CONFIG_KVM) += hyp/
 
 kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \
 	 $(KVM)/vfio.o $(KVM)/irqchip.o $(KVM)/binary_stats.o \
-	 arm.o mmu.o mmio.o psci.o perf.o hypercalls.o pvtime.o \
+	 arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
 	 inject_fault.o va_layout.o handle_exit.o \
 	 guest.o debug.o reset.o sys_regs.o \
 	 vgic-sys-reg-v3.o fpsimd.o pmu.o \
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2b542fdc237e..48f89d80f464 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1744,7 +1744,9 @@ static int init_subsystems(void)
 	if (err)
 		goto out;
 
-	kvm_perf_init();
+	kvm_pmu_init();
+	kvm_register_perf_callbacks(NULL);
+
 	kvm_sys_reg_table_init();
 
 out:
@@ -2160,7 +2162,7 @@ int kvm_arch_init(void *opaque)
 /* NOP: Compiling as a module not supported */
 void kvm_arch_exit(void)
 {
-	kvm_perf_teardown();
+	kvm_unregister_perf_callbacks();
 }
 
 static int __init early_kvm_mode_cfg(char *arg)
diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c
deleted file mode 100644
index 0b902e0d5b5d..000000000000
--- a/arch/arm64/kvm/perf.c
+++ /dev/null
@@ -1,27 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Based on the x86 implementation.
- *
- * Copyright (C) 2012 ARM Ltd.
- * Author: Marc Zyngier <marc.zyngier@arm.com>
- */
-
-#include <linux/perf_event.h>
-#include <linux/kvm_host.h>
-
-#include <asm/kvm_emulate.h>
-
-DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available);
-
-void kvm_perf_init(void)
-{
-	if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled())
-		static_branch_enable(&kvm_arm_pmu_available);
-
-	kvm_register_perf_callbacks(NULL);
-}
-
-void kvm_perf_teardown(void)
-{
-	kvm_unregister_perf_callbacks();
-}
diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 03a6c1f4a09a..d98b57a17043 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -7,6 +7,14 @@
 #include <linux/perf_event.h>
 #include <asm/kvm_hyp.h>
 
+DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available);
+
+void kvm_pmu_init(void)
+{
+	if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled())
+		static_branch_enable(&kvm_arm_pmu_available);
+}
+
 /*
  * Given the perf event attributes and system type, determine
  * if we are going to need to switch counters at guest entry/exit.
diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h
index 864b9997efb2..42270676498d 100644
--- a/include/kvm/arm_pmu.h
+++ b/include/kvm/arm_pmu.h
@@ -14,6 +14,7 @@
 #define ARMV8_PMU_MAX_COUNTER_PAIRS	((ARMV8_PMU_MAX_COUNTERS + 1) >> 1)
 
 DECLARE_STATIC_KEY_FALSE(kvm_arm_pmu_available);
+void kvm_pmu_init(void);
 
 static __always_inline bool kvm_arm_support_pmu_v3(void)
 {
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192034.342298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwd-0000J1-Ep; Wed, 22 Sep 2021 00:17:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192034.342298; Wed, 22 Sep 2021 00:17:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwd-0000I1-85; Wed, 22 Sep 2021 00:17:03 +0000
Received: by outflank-mailman (input) for mailman id 192034;
 Wed, 22 Sep 2021 00:17:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lXnW=OM=flex--seanjc.bounces.google.com=3b3NKYQYKCZACyu73w08805y.w86Hy7-xyFy552CDC.Hy79B83ywD.8B0@srs-us1.protection.inumbo.net>)
 id 1mSpmy-0000pt-Rb
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:04 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 235cc1ec-dfef-45ac-aaf4-28b42669b747;
 Wed, 22 Sep 2021 00:06:07 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 w10-20020a0cb54a000000b0037a9848b92fso8988476qvd.0
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 235cc1ec-dfef-45ac-aaf4-28b42669b747
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=QkPwm3zvrM5/6qTUHoNNLSAbgPAl2i0gmFlu0epOg4o=;
        b=d1KD92GmsPFOnc9Voj9I1ga0GeAS7/A1M3Ff0hwfVHHw2F6NmCxAZ84UfNg13BcPCi
         lG4uDVWzzMX9YcA/y1gf96xoTBNuPhAlxwTx2TA/MtTqi0CKnLCaTn1XKcjET/qIHw/W
         PFtn2f+5xRKyIIgk8E1s1kBonYCZbxSjMNPx2Pcoaxli0kTn68s/Bp75u+cWhB/zdDT2
         joJ0rxsuthSJgBA//ri2dU4acF1OCR0lq3uBlgPA2WynVs0lVsjJioXNEExBFYXBNxT3
         Bx/JzoAA/ewtVFiSqKGlN2hvDgXry2asA82RE64+sST1QDa4vg0dMxQ7kXfb5GF5+5GY
         /rGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=QkPwm3zvrM5/6qTUHoNNLSAbgPAl2i0gmFlu0epOg4o=;
        b=XXN9EQaZEpGqa2DOsyq8MtBSBT/RhAolC/MTo4yBuDE/8KHQRR0dEWVLk1LWZtSSBS
         D/qWcMQe/BDeAaYRDh/OnbpviB+p6l4dHNdYpfnDChZ4md4Uahzq2B2hajsqkir4mBIr
         tBvj/pIGnyDbOo5B7FZdv/k7hi7T3Io+d9A21RMp75EcDRGXXu9nz7usQl7g75zpHUtH
         zknd/NQJwoI/JgSgIOvDVyrVrd5Z3gH2AB2lAh2oZajtmt2gZSdtZJBnX2klBQbmm235
         bZebfNjVn8bdTYqJ3lywRGiYn2zm5egLg8VVO9o9PfL/VsnfIH7U7O5L5rMY1O41BJL8
         7OLw==
X-Gm-Message-State: AOAM530x9fSzhlLHrxINfuI16fK1min6aqUqzunaKRd0BVYHHhZm8tvK
	hr+PaEyJbV8vY0XLBRQ0JEA1fNCSr8Q=
X-Google-Smtp-Source: ABdhPJwFZxN2Kv1ibh5ZHwIVIA/ukWkfcwWerZdycqHkme9lr0IiGpVtUFqWcbWB5ZsvLCpW/DgRz3yzrYw=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:ad4:4a93:: with SMTP id h19mr34366283qvx.41.1632269167312;
 Tue, 21 Sep 2021 17:06:07 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:28 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-12-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 11/16] KVM: x86: More precisely identify NMI from guest
 when handling PMI
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Differntiate between IRQ and NMI for KVM's PMC overflow callback, which
was originally invoked in response to an NMI that arrived while the guest
was running, but was inadvertantly changed to fire on IRQs as well when
support for perf without PMU/NMI was added to KVM.  In practice, this
should be a nop as the PMC overflow callback shouldn't be reached, but
it's a cheap and easy fix that also better documents the situation.

Note, this also doesn't completely prevent false positives if perf
somehow ends up calling into KVM, e.g. an NMI can arrive in host after
KVM sets its flag.

Fixes: dd60d217062f ("KVM: x86: Fix perf timer mode IP reporting")
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/svm/svm.c |  2 +-
 arch/x86/kvm/vmx/vmx.c |  4 +++-
 arch/x86/kvm/x86.c     |  2 +-
 arch/x86/kvm/x86.h     | 13 ++++++++++---
 4 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 1a70e11f0487..0a0c01744b63 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -3843,7 +3843,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)
 	}
 
 	if (unlikely(svm->vmcb->control.exit_code == SVM_EXIT_NMI))
-		kvm_before_interrupt(vcpu);
+		kvm_before_interrupt(vcpu, KVM_HANDLING_NMI);
 
 	kvm_load_host_xsave_state(vcpu);
 	stgi();
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index f19d72136f77..61a4f5ff2acd 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -6344,7 +6344,9 @@ void vmx_do_interrupt_nmi_irqoff(unsigned long entry);
 static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,
 					unsigned long entry)
 {
-	kvm_before_interrupt(vcpu);
+	bool is_nmi = entry == (unsigned long)asm_exc_nmi_noist;
+
+	kvm_before_interrupt(vcpu, is_nmi ? KVM_HANDLING_NMI : KVM_HANDLING_IRQ);
 	vmx_do_interrupt_nmi_irqoff(entry);
 	kvm_after_interrupt(vcpu);
 }
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 24a6faa07442..412646b973bb 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9676,7 +9676,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 	 * interrupts on processors that implement an interrupt shadow, the
 	 * stat.exits increment will do nicely.
 	 */
-	kvm_before_interrupt(vcpu);
+	kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ);
 	local_irq_enable();
 	++vcpu->stat.exits;
 	local_irq_disable();
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index a9c107e7c907..9b26f9b09d2a 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -387,9 +387,16 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm)
 	return kvm->arch.cstate_in_guest;
 }
 
-static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu)
+enum kvm_intr_type {
+	/* Values are arbitrary, but must be non-zero. */
+	KVM_HANDLING_IRQ = 1,
+	KVM_HANDLING_NMI,
+};
+
+static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu,
+					enum kvm_intr_type intr)
 {
-	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 1);
+	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, (u8)intr);
 }
 
 static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu)
@@ -399,7 +406,7 @@ static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu)
 
 static inline bool kvm_handling_nmi_from_guest(struct kvm_vcpu *vcpu)
 {
-	return !!vcpu->arch.handling_intr_from_guest;
+	return vcpu->arch.handling_intr_from_guest == KVM_HANDLING_NMI;
 }
 
 static inline bool kvm_pat_valid(u64 data)
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192044.342327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwi-0001BY-VG; Wed, 22 Sep 2021 00:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192044.342327; Wed, 22 Sep 2021 00:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwi-0001BP-Qx; Wed, 22 Sep 2021 00:17:08 +0000
Received: by outflank-mailman (input) for mailman id 192044;
 Wed, 22 Sep 2021 00:17:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/q4x=OM=flex--seanjc.bounces.google.com=3cXNKYQYKCZIE0w95y2AA270.yA8J09-z0H0774EFE.J09BDA50yF.AD2@srs-us1.protection.inumbo.net>)
 id 1mSpn3-0000pt-Rt
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:09 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 86cde248-a02e-4154-8345-04560440bc5a;
 Wed, 22 Sep 2021 00:06:09 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 e8-20020a0cf348000000b0037a350958f2so7989448qvm.7
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86cde248-a02e-4154-8345-04560440bc5a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=UXKfzNBbNa1Oo/hLKODwyb6nbYWV7pmBiaBRuwKp1U0=;
        b=sij3IoihiiMGm0/dG5Xp7BiIKh2nJQSIypS3WrTNWrVOZHFksiSjBYstCiHLVbnJ13
         CGIedE83H/3gmCrV9RK/JpQ9g0u3tHEiQjhbTFJXXUrXV0ft34vTsOgWhRp1Wa7A8C6I
         ga/7VQ/lJSnUZA4NLjOrvJImdkqCnE8O/y4QTBssRcfWcNJ0EsLS/eKEb164VYhOZOrl
         XsSNnTRGSdZnzbutkd59kyydwUypRuPSaJDQOMl26PrdMRkBEJw/9N/QFBs8eSa/Xmr4
         6N1Q3twhtyak+avdO0kUbzl2cTxe08M4fiWduj9T2KL+up/0OzkOWDOk2cLy28m5k44C
         o7yg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=UXKfzNBbNa1Oo/hLKODwyb6nbYWV7pmBiaBRuwKp1U0=;
        b=s2x4F93lCjRA6zoxskwLEBWvtoRVoJsWQBrnKwXxAVMilhCdxVu55UB9S7ccaR8dw2
         WMaw0tf0/xiuGTwvWvQss7gzjFeGdqmPYDemMUabtEDEGa9lWT+XUVzsutvZQrRK8zTh
         iu7ygytvjcc53DYdSKU+kaMFx9Oiq2J+4gtm6+90GigUzn0J3wOkS/9N4RnJZThi/iKp
         /DOVy1uQjJllsZtmHv9grUnZJBLExSHC5RpOROETmCbNe2gdcxpLTt85QOgSwKkfv/YP
         6QA5javM66sgvSY/v6FqQZwP3kIOVbazbWdx7WFKZ0q8bij5EejpvQnel695i9e7SF8x
         IezQ==
X-Gm-Message-State: AOAM533FejEyr/HtmyWle2itwhhIfUJBMYJRqjDyNeRFzoI9nttUi+ci
	6g5Kd835XpcFBZUxqu1KPbuXTBscU7o=
X-Google-Smtp-Source: ABdhPJxYtz6r8xJfl/cejZ4zPL5NzU57zPkIfCxWD2RF7CxQQEMc8shvHNk6SkGK7OMvvH0aEjDtmqay0xg=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a0c:8061:: with SMTP id 88mr7515654qva.5.1632269169468;
 Tue, 21 Sep 2021 17:06:09 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:29 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-13-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 12/16] KVM: Move x86's perf guest info callbacks to generic KVM
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Move x86's perf guest callbacks into common KVM, as they are semantically
identical to arm64's callbacks (the only other such KVM callbacks).
arm64 will convert to the common versions in a future patch.

Implement the necessary arm64 arch hooks now to avoid having to provide
stubs or a temporary #define (from x86) to avoid arm64 compilation errors
when CONFIG_GUEST_PERF_EVENTS=y.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/include/asm/kvm_host.h |  8 +++++
 arch/arm64/kvm/arm.c              |  5 +++
 arch/x86/include/asm/kvm_host.h   |  3 ++
 arch/x86/kvm/x86.c                | 53 +++++++------------------------
 include/linux/kvm_host.h          | 10 ++++++
 virt/kvm/kvm_main.c               | 44 +++++++++++++++++++++++++
 6 files changed, 81 insertions(+), 42 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index ed940aec89e0..828b6eaa2c56 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -673,6 +673,14 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa);
 void kvm_perf_init(void);
 void kvm_perf_teardown(void);
 
+#ifdef CONFIG_GUEST_PERF_EVENTS
+static inline bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu)
+{
+	/* Any callback while a vCPU is loaded is considered to be in guest. */
+	return !!vcpu;
+}
+#endif
+
 long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu);
 gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
 void kvm_update_stolen_time(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e9a2b8f27792..2b542fdc237e 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -500,6 +500,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
 	return vcpu_mode_priv(vcpu);
 }
 
+unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu)
+{
+	return *vcpu_pc(vcpu);
+}
+
 /* Just ensure a guest exit from a particular CPU */
 static void exit_vm_noop(void *info)
 {
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 2d86a2dfc775..6efe4e03a6d2 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1543,6 +1543,9 @@ static inline int kvm_arch_flush_remote_tlb(struct kvm *kvm)
 		return -ENOTSUPP;
 }
 
+#define kvm_arch_pmi_in_guest(vcpu) \
+	((vcpu) && (vcpu)->arch.handling_intr_from_guest)
+
 int kvm_mmu_module_init(void);
 void kvm_mmu_module_exit(void);
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 412646b973bb..1bea616402e6 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8264,43 +8264,12 @@ static void kvm_timer_init(void)
 			  kvmclock_cpu_online, kvmclock_cpu_down_prep);
 }
 
-static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu)
-{
-	return vcpu && vcpu->arch.handling_intr_from_guest;
-}
-
-static unsigned int kvm_guest_state(void)
-{
-	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
-	unsigned int state;
-
-	if (!kvm_pmi_in_guest(vcpu))
-		return 0;
-
-	state = PERF_GUEST_ACTIVE;
-	if (static_call(kvm_x86_get_cpl)(vcpu))
-		state |= PERF_GUEST_USER;
-
-	return state;
-}
-
-static unsigned long kvm_guest_get_ip(void)
-{
-	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
-
-	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
-	if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu)))
-		return 0;
-
-	return kvm_rip_read(vcpu);
-}
-
 static unsigned int kvm_handle_intel_pt_intr(void)
 {
 	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
 
 	/* '0' on failure so that the !PT case can use a RET0 static call. */
-	if (!kvm_pmi_in_guest(vcpu))
+	if (!kvm_arch_pmi_in_guest(vcpu))
 		return 0;
 
 	kvm_make_request(KVM_REQ_PMI, vcpu);
@@ -8309,12 +8278,6 @@ static unsigned int kvm_handle_intel_pt_intr(void)
 	return 1;
 }
 
-static struct perf_guest_info_callbacks kvm_guest_cbs = {
-	.state			= kvm_guest_state,
-	.get_ip			= kvm_guest_get_ip,
-	.handle_intel_pt_intr	= NULL,
-};
-
 #ifdef CONFIG_X86_64
 static void pvclock_gtod_update_fn(struct work_struct *work)
 {
@@ -11068,9 +11031,11 @@ int kvm_arch_hardware_setup(void *opaque)
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
 	kvm_ops_static_call_update();
 
+	/* Temporary ugliness. */
 	if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest())
-		kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr;
-	perf_register_guest_info_callbacks(&kvm_guest_cbs);
+		kvm_register_perf_callbacks(kvm_handle_intel_pt_intr);
+	else
+		kvm_register_perf_callbacks(NULL);
 
 	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
 		supported_xss = 0;
@@ -11099,8 +11064,7 @@ int kvm_arch_hardware_setup(void *opaque)
 
 void kvm_arch_hardware_unsetup(void)
 {
-	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
-	kvm_guest_cbs.handle_intel_pt_intr = NULL;
+	kvm_unregister_perf_callbacks();
 
 	static_call(kvm_x86_hardware_unsetup)();
 }
@@ -11727,6 +11691,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
 	return vcpu->arch.preempted_in_kernel;
 }
 
+unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu)
+{
+	return kvm_rip_read(vcpu);
+}
+
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 {
 	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index e4d712e9f760..b9255a6439f2 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1163,6 +1163,16 @@ static inline bool kvm_arch_intc_initialized(struct kvm *kvm)
 }
 #endif
 
+#ifdef CONFIG_GUEST_PERF_EVENTS
+unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu);
+
+void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void));
+void kvm_unregister_perf_callbacks(void);
+#else
+static inline void kvm_register_perf_callbacks(void *ign) {}
+static inline void kvm_unregister_perf_callbacks(void) {}
+#endif /* CONFIG_GUEST_PERF_EVENTS */
+
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type);
 void kvm_arch_destroy_vm(struct kvm *kvm);
 void kvm_arch_sync_events(struct kvm *kvm);
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 3e67c93ca403..179fb110a00f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -5460,6 +5460,50 @@ struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void)
         return &kvm_running_vcpu;
 }
 
+#ifdef CONFIG_GUEST_PERF_EVENTS
+static unsigned int kvm_guest_state(void)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+	unsigned int state;
+
+	if (!kvm_arch_pmi_in_guest(vcpu))
+		return 0;
+
+	state = PERF_GUEST_ACTIVE;
+	if (!kvm_arch_vcpu_in_kernel(vcpu))
+		state |= PERF_GUEST_USER;
+
+	return state;
+}
+
+static unsigned long kvm_guest_get_ip(void)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
+	if (WARN_ON_ONCE(!kvm_arch_pmi_in_guest(vcpu)))
+		return 0;
+
+	return kvm_arch_vcpu_get_ip(vcpu);
+}
+
+static struct perf_guest_info_callbacks kvm_guest_cbs = {
+	.state			= kvm_guest_state,
+	.get_ip			= kvm_guest_get_ip,
+	.handle_intel_pt_intr	= NULL,
+};
+
+void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void))
+{
+	kvm_guest_cbs.handle_intel_pt_intr = pt_intr_handler;
+	perf_register_guest_info_callbacks(&kvm_guest_cbs);
+}
+void kvm_unregister_perf_callbacks(void)
+{
+	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
+}
+#endif
+
 struct kvm_cpu_compat_check {
 	void *opaque;
 	int *ret;
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192056.342338 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwl-0001XW-IL; Wed, 22 Sep 2021 00:17:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192056.342338; Wed, 22 Sep 2021 00:17:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwl-0001XJ-Di; Wed, 22 Sep 2021 00:17:11 +0000
Received: by outflank-mailman (input) for mailman id 192056;
 Wed, 22 Sep 2021 00:17:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cuAi=OM=flex--seanjc.bounces.google.com=3dnNKYQYKCZcJ51EA37FF7C5.3FDO5E-45M5CC9JKJ.O5EGIFA53K.FI7@srs-us1.protection.inumbo.net>)
 id 1mSpnI-0000pt-SI
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:24 +0000
Received: from mail-qv1-xf4a.google.com (unknown [2607:f8b0:4864:20::f4a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 61d8f6f0-0070-4e6d-b646-8742924dc860;
 Wed, 22 Sep 2021 00:06:14 +0000 (UTC)
Received: by mail-qv1-xf4a.google.com with SMTP id
 q17-20020a0cf5d1000000b00380e60ef363so7785089qvm.9
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61d8f6f0-0070-4e6d-b646-8742924dc860
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=hykBvrf8DnzH3Q8YHjL/Y0GXRoABX5ADBK+0/12MWfw=;
        b=jKopPzwiybt9fB96grPLhcmov4eHABK3Rn1V3+jC89gDcrIoprd65bnGYY9DP7ju4b
         85kkPs7ZPkL8WlWgMXplyIY3a/z2qIzxH+/FI3XjcdmrliPkx8nPrp2a5kX5Pjc1ywob
         ByTND1+iT+q5EbaKv8lVZV94ko5nNdJrrzXW+BlrNXXtl2FDDO+EPFHPKuUM6DjPG5xB
         A3Kj4DmZiicnBCVFqv70LMvPNdNZybwucvdKO2/8ODrWoNpkysFbo74lUlyrUvFEKlBi
         o1a2jdJRabm/xdF8RFOfzaw+a+ydqqTbWa982xz/HRYZHvnoWhQfVxgfpJMjXOW1pVY7
         0TqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=hykBvrf8DnzH3Q8YHjL/Y0GXRoABX5ADBK+0/12MWfw=;
        b=aNDw7PZ7ZEtFvBhA9Usrb7eZVYKFYWfOb9iYwAxf66uMKc9zpt5gYVkYKkaqTT3MpI
         U8q090Ivc6YuaA6nu/iowSfVon20qONc+BHs5c31WzzL+Yzwla5hIiET/2QG25yy8Kcw
         TH2QJBkaZZKCboHYIqa+fsSTi/S20eb6kgTUgr2H7Wxn16/LYZD7PRjHT/ggIlTo3H9a
         da3Lb9IloKjkME/ZK5uASZmJ79VF1IOMFH1T61k2/4MzgoEJoQBu9T14hFQUeKEho9Ov
         tGE8O5JU2ob2EmO5OcW3ti3dn0GhhhTEF68bXuG5fX36z0LVeeul76lmJN3pSuTRXdnD
         IYjQ==
X-Gm-Message-State: AOAM5337e1ErbLbJrpVm/xvzA/uaJE4vmwcyEXoad/Fqc0XmwXS2jl8H
	SFooQ0H0QpH9g71c0rS5qYMEf/7VUVA=
X-Google-Smtp-Source: ABdhPJy5GySvAYc4kjGUgxPeiS0d5MaL33kPbz4EVAybMqnGUcP/K+XpH6FqRymRPMD4ga0cIEmdn6OzCzE=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:a05:6902:1143:: with SMTP id
 p3mr39279294ybu.420.1632269174363; Tue, 21 Sep 2021 17:06:14 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:31 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-15-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 14/16] KVM: arm64: Convert to the generic perf callbacks
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Drop arm64's version of the callbacks in favor of the callbacks provided
by generic KVM, which are semantically identical.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/arm64/kvm/perf.c | 34 ++--------------------------------
 1 file changed, 2 insertions(+), 32 deletions(-)

diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c
index 3e99ac4ab2d6..0b902e0d5b5d 100644
--- a/arch/arm64/kvm/perf.c
+++ b/arch/arm64/kvm/perf.c
@@ -13,45 +13,15 @@
 
 DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available);
 
-static unsigned int kvm_guest_state(void)
-{
-	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
-	unsigned int state;
-
-	if (!vcpu)
-		return 0;
-
-	state = PERF_GUEST_ACTIVE;
-	if (!vcpu_mode_priv(vcpu))
-		state |= PERF_GUEST_USER;
-
-	return state;
-}
-
-static unsigned long kvm_get_guest_ip(void)
-{
-	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
-
-	if (WARN_ON_ONCE(!vcpu))
-		return 0;
-
-	return *vcpu_pc(vcpu);
-}
-
-static struct perf_guest_info_callbacks kvm_guest_cbs = {
-	.state		= kvm_guest_state,
-	.get_ip		= kvm_get_guest_ip,
-};
-
 void kvm_perf_init(void)
 {
 	if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled())
 		static_branch_enable(&kvm_arm_pmu_available);
 
-	perf_register_guest_info_callbacks(&kvm_guest_cbs);
+	kvm_register_perf_callbacks(NULL);
 }
 
 void kvm_perf_teardown(void)
 {
-	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
+	kvm_unregister_perf_callbacks();
 }
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 00:17:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 00:17:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192058.342349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwo-0001zu-To; Wed, 22 Sep 2021 00:17:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192058.342349; Wed, 22 Sep 2021 00:17:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSpwo-0001zh-Pi; Wed, 22 Sep 2021 00:17:14 +0000
Received: by outflank-mailman (input) for mailman id 192058;
 Wed, 22 Sep 2021 00:17:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cs+K=OM=flex--seanjc.bounces.google.com=3enNKYQYKCZsN95IE7BJJBG9.7JHS9I-89Q9GGDNON.S9IKMJE97O.JMB@srs-us1.protection.inumbo.net>)
 id 1mSpnc-0000pt-Sp
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 00:07:44 +0000
Received: from mail-qk1-x74a.google.com (unknown [2607:f8b0:4864:20::74a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c89c8772-a935-4c12-85fa-e239b1d93411;
 Wed, 22 Sep 2021 00:06:19 +0000 (UTC)
Received: by mail-qk1-x74a.google.com with SMTP id
 bm12-20020a05620a198c00b00432e14ddb99so5306451qkb.21
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 17:06:19 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c89c8772-a935-4c12-85fa-e239b1d93411
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=reply-to:date:in-reply-to:message-id:mime-version:references
         :subject:from:to:cc;
        bh=4BBDHyzTmFJD2PGtjmXQGbVqPzRCZc8ppu5cY3KZ5Y8=;
        b=i1hLF6bC7QippVnwD+QfcurzwPql0DTYdtvpNMOXEcTNGevY7UnIQHp4F8TFxhWOcC
         y5khcTlTsxcY03DM6I7OGwJdsNRusHePnZs48VJgz9iDl9BCd/WdBAHgBd08BkVlEctl
         qgkp34acvdJvQRinIA310i906+1pI4VRGGbTHUSEWOTyISF582eEBN0uohWrXglJ4Vha
         UiRiAJR9+rHm1hSoN64l3MLHm8i51hrtd5lKdh0LN16chuzJEtOj6NO2pvuQiZZnchZT
         2+/Nzn1kX3D5dvOeb3mhXnOtnZBTLnXfaqpsFhvkCaK7qjG4cSZxh2Y/t71DvQNuI5L1
         pIfA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:reply-to:date:in-reply-to:message-id
         :mime-version:references:subject:from:to:cc;
        bh=4BBDHyzTmFJD2PGtjmXQGbVqPzRCZc8ppu5cY3KZ5Y8=;
        b=JRbKpsY9gHOT9MaMelgARMT1JqTf84UIaAZLivNewsuY2JBkCA56Iy8K+4zobnRMdp
         tK4K7AypNAw0CpLpv3e1HjnsFRLAvXMp+qmjS9dNxTG3ALgc9iJ3Y4IAeyIwpd/wXdDB
         eJL5ezlsvwsrYx74N0kb3iexuK3AeFIE82ZdrgbXGv1WHtfe4ZhfEhd472Hq8fJgx10o
         97urcNE24NWiIhB1OZxpDbuK1Ri8JTNLm+A0ugozabqY+0QtXDlz6te/3kRiinBYtwYi
         RCn7WIuH74Lium/xTqXVY00t7kSr89OUlQr3HNfFmP6/7bCCcL5x0jVxfDl0otu57h4T
         6u+w==
X-Gm-Message-State: AOAM5322EB73JOy3sUe3ClvPQWl5LMtVkzX7lyLJxgPfxO8cxeJHsZwo
	KZEYJ7P6NoCS57v4W4t5rQuu3gYPhqQ=
X-Google-Smtp-Source: ABdhPJzV8/bAA+bVdBd7d54Vy+1c6C23hub/FF/gkHwdARHRitfqPelJ1Pr98qSi4K4qg/f1+Xhzcbcxu/4=
X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:b022:92d6:d37b:686c])
 (user=seanjc job=sendgmr) by 2002:ad4:466a:: with SMTP id z10mr32596974qvv.47.1632269178759;
 Tue, 21 Sep 2021 17:06:18 -0700 (PDT)
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Tue, 21 Sep 2021 17:05:33 -0700
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Message-Id: <20210922000533.713300-17-seanjc@google.com>
Mime-Version: 1.0
References: <20210922000533.713300-1-seanjc@google.com>
X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog
Subject: [PATCH v3 16/16] perf: Drop guest callback (un)register stubs
From: Sean Christopherson <seanjc@google.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, 
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>, 
	Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>, 
	Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, 
	Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>, 
	Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, 
	Namhyung Kim <namhyung@kernel.org>, James Morse <james.morse@arm.com>, 
	Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, 
	Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, 
	Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, 
	Joerg Roedel <joro@8bytes.org>, Stefano Stabellini <sstabellini@kernel.org>, 
	linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, 
	linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, 
	kvm@vger.kernel.org, xen-devel@lists.xenproject.org, 
	Artem Kashkanov <artem.kashkanov@intel.com>, Like Xu <like.xu.linux@gmail.com>, 
	Zhu Lingshan <lingshan.zhu@intel.com>
Content-Type: text/plain; charset="UTF-8"

Drop perf's stubs for (un)registering guest callbacks now that KVM
registration of callbacks is hidden behind GUEST_PERF_EVENTS=y.  The only
other user is x86 XEN_PV, and x86 unconditionally selects PERF_EVENTS.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 include/linux/perf_event.h | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index d582dfeb4e20..20327d1046bb 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1505,11 +1505,6 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)	{ }
 static inline void
 perf_bp_event(struct perf_event *event, void *data)			{ }
 
-static inline void perf_register_guest_info_callbacks
-(struct perf_guest_info_callbacks *cbs)					{ }
-static inline void perf_unregister_guest_info_callbacks
-(struct perf_guest_info_callbacks *cbs)					{ }
-
 static inline void perf_event_mmap(struct vm_area_struct *vma)		{ }
 
 typedef int (perf_ksymbol_get_name_f)(char *name, int name_len, void *data);
-- 
2.33.0.464.g1972c5931b-goog



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 03:24:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 03:24:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192104.342363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSsrc-0004Fb-BG; Wed, 22 Sep 2021 03:24:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192104.342363; Wed, 22 Sep 2021 03:24:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSsrc-0004FC-43; Wed, 22 Sep 2021 03:24:04 +0000
Received: by outflank-mailman (input) for mailman id 192104;
 Wed, 22 Sep 2021 03:24:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSsrb-0004F2-N5; Wed, 22 Sep 2021 03:24:03 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSsrb-0004hL-Cl; Wed, 22 Sep 2021 03:24:03 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSsra-0006KJ-VT; Wed, 22 Sep 2021 03:24:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSsra-0002Cy-Un; Wed, 22 Sep 2021 03:24:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mIy6x4LJ+6Xj+XjummdgTg2lb49lN2KFK0CBPPmJqc4=; b=Vp70/X2ZA1BJ6wvcWaGqOZPNgT
	quFgH8kc6b16R98K+J1F545ylmEkOiAa0D+93aKX6TCTiCnrJxQvyjWF6Y8fzn9vK1ETylfUiaWXv
	krV8KHGFk8gd0gKKTc24PtXB0ZhpFVPtlI0AiwweFL4zTYDBvKfCOaKgwX9OgGp9Fz8M=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165142-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165142: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 03:24:02 +0000

flight 165142 xen-unstable real [real]
flight 165146 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165142/
http://logs.test-lab.xenproject.org/osstest/logs/165146/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail in 165136 REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-raw 13 guest-start                fail pass in 165136

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165136 like 164945
 test-arm64-arm64-libvirt-raw 14 migrate-support-check fail in 165136 never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check fail in 165136 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a3dd33e63c2c8c51102f223e6efe2e3d3bdcbec1
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   11 days
Failing since        164951  2021-09-12 00:14:36 Z   10 days   20 attempts
Testing same since   165131  2021-09-20 17:39:22 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 683 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 04:46:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 04:46:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.191822.342377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSu8m-0004TC-Ac; Wed, 22 Sep 2021 04:45:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 191822.342377; Wed, 22 Sep 2021 04:45:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSu8m-0004T5-6r; Wed, 22 Sep 2021 04:45:52 +0000
Received: by outflank-mailman (input) for mailman id 191822;
 Tue, 21 Sep 2021 16:40:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=//zX=OL=stuge.se=peter@srs-us1.protection.inumbo.net>)
 id 1mSiok-0004a6-3w
 for xen-devel@lists.xenproject.org; Tue, 21 Sep 2021 16:40:26 +0000
Received: from foo.stuge.se (unknown [212.116.89.98])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9d6ab99b-1afa-11ec-b908-12813bfff9fa;
 Tue, 21 Sep 2021 16:40:23 +0000 (UTC)
Received: (qmail 19290 invoked by uid 1000); 21 Sep 2021 15:40:21 -0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d6ab99b-1afa-11ec-b908-12813bfff9fa
Message-ID: <20210921154021.19289.qmail@stuge.se>
Date: Tue, 21 Sep 2021 15:40:20 +0000
From: Peter Stuge <peter@stuge.se>
To: Alec Brown <alec.r.brown@oracle.com>,
  Heinrich Schuchardt <xypron.glpk@gmx.de>
Cc: "coreboot@coreboot.org" <coreboot@coreboot.org>,
  "grub-devel@gnu.org" <grub-devel@gnu.org>,
  "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
  "systemd-devel@lists.freedesktop.org" <systemd-devel@lists.freedesktop.org>,
  "trenchboot-devel@googlegroups.com" <trenchboot-devel@googlegroups.com>,
  "u-boot@lists.denx.de" <u-boot@lists.denx.de>,
  "x86@kernel.org" <x86@kernel.org>,
  "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
  Aleksandr Burmashev <alexander.burmashev@oracle.com>,
  "allen.cryptic@gmail.com" <allen.cryptic@gmail.com>,
  "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
  "andy.shevchenko@gmail.com" <andy.shevchenko@gmail.com>,
  "ardb@kernel.org" <ardb@kernel.org>,
  "btrotter@gmail.com" <btrotter@gmail.com>,
  Daniel Kiper <daniel.kiper@oracle.com>,
  "dpsmith@apertussolutions.com" <dpsmith@apertussolutions.com>,
  Eric DeVolder <eric.devolder@oracle.com>,
  Eric Snowberg <eric.snowberg@oracle.com>,
  "frowand.list@gmail.com" <frowand.list@gmail.com>,
  "hpa@zytor.com" <hpa@zytor.com>,
  "hun@n-dimensional.de" <hun@n-dimensional.de>,
  "james.dutton@gmail.com" <james.dutton@gmail.com>,
  "javierm@redhat.com" <javierm@redhat.com>,
  Joao Martins <joao.m.martins@oracle.com>,
  "jwerner@chromium.org" <jwerner@chromium.org>,
  Kanth Ghatraju <kanth.ghatraju@oracle.com>,
  Konrad Wilk <konrad.wilk@oracle.com>,
  "krystian.hebel@3mdeb.com" <krystian.hebel@3mdeb.com>,
  "leif@nuviainc.co m" <leif@nuviainc.com>,
  "lukasz.hawrylko@intel.com" <lukasz.hawrylko@intel.com>,
  "luto@amacapital.net" <luto@amacapital.net>,
  "michal.zygowski@3mdeb.com" <michal.zygowski@3mdeb.com>,
  "mjg59@google.com" <mjg59@google.com>,
  "mtottenh@akamai.com" <mtottenh@akamai.com>,
  "phcoder@gmail.com" <phcoder@gmail.com>,
  "piotr.krol@3mdeb.com" <piotr.krol@3mdeb.com>,
  "pjones@redhat.com" <pjones@redhat.com>,
  "pmenzel@molgen.mpg.de" <pmenzel@molgen.mpg.de>,
  "rasmus.villemoes@prevas.dk" <rasmus.villemoes@prevas.dk>,
  "rdunlap@infradead.org" <rdunlap@infradead.org>,
  "roger.pau@citrix.com" <roger.pau@citrix.com>,
  Ross Philipson <ross.philipson@oracle.com>,
  "sjg@chromium.org" <sjg@chromium.org>,
  "trini@konsulko.com" <trini@konsulko.com>,
  "tyhicks@linux.microsoft.com" <tyhicks@linux.microsoft.com>,
  "ulrich.windl@rz.uni-regensburg.de" <ulrich.windl@rz.uni-regensburg.de>,
  "wvervoorn@eltan.com" <wvervoorn@eltan.com>,
  "rharwood@redhat.com" <rharwood@redhat.com>
Subject: Re: [SPECIFICATION RFC v3] The firmware and bootloader log
 specification
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <A7F710D3-5148-4E92-9E3D-5D850AD0245F@gmx.de>
 <DM6PR10MB2986A960E859A744FDC3875ABCDE9@DM6PR10MB2986.namprd10.prod.outlook.com>

Alec Brown wrote:
> Below is how the layout of these logs would store their data.
>=20
> bf_log_header:
>                +-------------------+
> u32            | version           |
> u32            | size              |
> u8[64]         | producer          |
> u8[64]         | log_format        |
> u64            | flags             |
> u64            | next_bflh_addr    |
> u64            | log_addr          |
> u32            | log_size          |
>                +-------------------+

I suggest to include a .magic at least in bf_log_header and an
=2Exor_checksum or .crc32 only in bf_log_header.

=2Emagic doubles as endianess indicator when the structures are
stored on movable media. (Pick an asymmetric magic bit pattern!)

I suggest renaming .next_bflh_addr to .next_log_header and .log_addr
to .log_buffer_addr.

I suggest to remove .size and .log_size:

The rationale for .size is "to allow for backward compatibility" but
that seems redundant thanks to .version.

=2Elog_size can be calculated from the subordinate data and is thus
mostly an unneccessary source of potential inconsistency.


> bf_log_buffer:
>                +-------------------+
> u32            | version           |
> u32            | size              |
> u8[64]         | producer          |
> u32            | next_msg_off      |
> bf_log_msg[l]  | msgs              |
>                +-------------------+

I suggest replacing .size and .next_msg_off with .messages containing l:

=2Esize can then be calculated from .messages; again, reliably avoiding
inconsistency between .size and .next_msg_off.

Allocated size doesn't seem useful if writers must anyway maintain state
containing the starting address. If writers must be allowed to be completely
stateless then maybe at least rename .size to .allocated_size and see below
for discovery.

Having .messages also eliminates the need for an end-of-messages marker
when the allocated space is not yet filled.


> bf_log_msg:
>                +-------------------+
> u32            | size              |
> u64            | ts_nsec           |
> u32            | level             |
> u32            | facility          |
> u32            | msg_off           |
> u8[n]          | type              |
> u8[m]          | msg               |
>                +-------------------+

It seems inconsistent that log_header.size and log_msg.size cover only
the respective struct itself while log_buffer.size also covers all
subordinate messages. Skipping all .size in this version fixes that.

And log_msg.size is not very useful since both .type and .msg have variable
length; it's not possible to access .msg without scanning .type. Please at
a minimum add .type_size but better yet replace .size with .type_size and
=2Emsg_size.


> There is still the outstanding issue of how the logs will be sent to the =
OS. If
> UEFI is used, we can use config tables. If ACPI or Device Tree is used, w=
e can
> use bf_log_header.next_bflh_addr to present the logs. If none of these pl=
atforms
> are used, it becomes a lot trickier to solve this issue.
>=20
> Any suggestions are much appreciated and will be taken into consideration.

Having bf_log_header.magic and some bf_log_header.$checksum, a strict rule
for bf_log_header start address granularity and a strict maximum offset
for the first header from top and/or bottom of memory allows to quickly
discover a log in memory without explicit handover.


> LPC System Boot and Security Micro-conference on the 22nd of September
> at 7:50 AM PDT (14:50 UTC).

Have fun! :)


Heinrich Schuchardt wrote:
> We already the EFI_TCG2_PROTOCOL and RFC 5424 (The syslog protocol).
> Why do we need to start from scratch?

That's a good question. I guess noone wants to settle for a standard
=66rom somewhere else. ;)

I wouldn't mind if log_msg was a syslog transport, but I can understand
if that's rejected because syslog messages require a lot of parsing for
presentation while Alec's proposal seems focused on efficiency and simplici=
ty.

It's also nice to be able to strictly mandate UTF-8 for all fields.
(RFC 5424 allows MSG to be anything.)


Kind regards

//Peter


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:23:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192119.342388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvfS-0006lK-G3; Wed, 22 Sep 2021 06:23:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192119.342388; Wed, 22 Sep 2021 06:23:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvfS-0006lD-CW; Wed, 22 Sep 2021 06:23:42 +0000
Received: by outflank-mailman (input) for mailman id 192119;
 Wed, 22 Sep 2021 06:23:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvfR-0006l7-LR
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:23:41 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id a054d97e-1b6d-11ec-b95f-12813bfff9fa;
 Wed, 22 Sep 2021 06:23:39 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-539-PjkRE_TgM9aZoLw9b0os3g-1; Wed, 22 Sep 2021 02:23:36 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 i4-20020a5d5224000000b0015b14db14deso1090097wra.23
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:23:35 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id o1sm1112414wru.91.2021.09.21.23.23.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:23:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a054d97e-1b6d-11ec-b95f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632291819;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Zooka9gAajsg8DCKduDSvz7LupNnOu3u1TFHi7y68Yc=;
	b=ShF0BzKOCSqi/IdPbHkO8BChArWbOFIrCNMnt2/Y2nEpCQgEBO/K+qoW5xOb+/4M/CWemu
	2huUb5qYhTmjHJEyNPrgxQJaSI7ntL//fPEkPhmvYbJYGGdNVyl7rcj6qCRqw5ZrJMbL4+
	Fo02AOquiagsE4jQDX/9YU2OVonblJM=
X-MC-Unique: PjkRE_TgM9aZoLw9b0os3g-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=Zooka9gAajsg8DCKduDSvz7LupNnOu3u1TFHi7y68Yc=;
        b=oWCis/M/ghIIcAwVdyPmnIbO9InGV1uNRJuxGSgyPAA1nsD64jS4iDolY4iJxqMil+
         7gK7r6d0GM/XVGxHaBu9+lFcsog8/ofsh1T0wgIs5NReyUbOFP2j/2rn28C+/OwNYTlW
         J29sQkSOIORsPujqApgXGQLkhsD3l6niL99u4FSwG5oqUEiXROOOwCMse3v9d5HDU5cL
         Bos4K27Q99ZSV+OjS7nKgMzEq9MNsenjxTasSJ5LbwG6z7yefMVkzyW0rpApBn1Ys451
         QLdOqMd6vnhQnvvhdW5rb8IYCUsKE++zV8Ja/wlmyBBaoNcQTarIZOS3+RiFnI7/Frp3
         jAwA==
X-Gm-Message-State: AOAM533AjnSEhecxaRfqqJR+fzUBDCCPrMikwd+wARaWWc6XjZRCJ8T2
	3Ii0XDmdfYUnYPqPvyK+pbaLx9wIZJ3S8I4uWPqeRZc4jpKtUWqATANto0a0DyDbxJvLzjCwBk3
	7MW2uA/6QS/iFIhUDJH9gFvl8ApQ=
X-Received: by 2002:a7b:c3cc:: with SMTP id t12mr8423440wmj.68.1632291814966;
        Tue, 21 Sep 2021 23:23:34 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJx4Cxl6B3FvPynsb4jNEGdd2w6uOOFopYtreXbLymB/TUbaoz5flccfGAb2LTh0HveeWZIomg==
X-Received: by 2002:a7b:c3cc:: with SMTP id t12mr8423394wmj.68.1632291814600;
        Tue, 21 Sep 2021 23:23:34 -0700 (PDT)
Subject: Re: [PATCH v3 02/16] KVM: x86: Register perf callbacks after calling
 vendor's hardware_setup()
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-3-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <6e31f767-ae3d-ec87-9880-5a8ebc381192@redhat.com>
Date: Wed, 22 Sep 2021 08:23:25 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-3-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Wait to register perf callbacks until after doing vendor hardaware setup.
> VMX's hardware_setup() configures Intel Processor Trace (PT) mode, and a
> future fix to register the Intel PT guest interrupt hook if and only if
> Intel PT is exposed to the guest will consume the configured PT mode.
> 
> Delaying registration to hardware setup is effectively a nop as KVM's perf
> hooks all pivot on the per-CPU current_vcpu, which is non-NULL only when
> KVM is handling an IRQ/NMI in a VM-Exit path.  I.e. current_vcpu will be
> NULL throughout both kvm_arch_init() and kvm_arch_hardware_setup().
> 
> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> Cc: Artem Kashkanov <artem.kashkanov@intel.com>
> Cc: stable@vger.kernel.org
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/x86/kvm/x86.c | 7 ++++---
>   1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 86539c1686fa..fb6015f97f9e 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8426,8 +8426,6 @@ int kvm_arch_init(void *opaque)
>   
>   	kvm_timer_init();
>   
> -	perf_register_guest_info_callbacks(&kvm_guest_cbs);
> -
>   	if (boot_cpu_has(X86_FEATURE_XSAVE)) {
>   		host_xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK);
>   		supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0;
> @@ -8461,7 +8459,6 @@ void kvm_arch_exit(void)
>   		clear_hv_tscchange_cb();
>   #endif
>   	kvm_lapic_exit();
> -	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
>   
>   	if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
>   		cpufreq_unregister_notifier(&kvmclock_cpufreq_notifier_block,
> @@ -11064,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque)
>   	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
>   	kvm_ops_static_call_update();
>   
> +	perf_register_guest_info_callbacks(&kvm_guest_cbs);
> +
>   	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
>   		supported_xss = 0;
>   
> @@ -11091,6 +11090,8 @@ int kvm_arch_hardware_setup(void *opaque)
>   
>   void kvm_arch_hardware_unsetup(void)
>   {
> +	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
> +
>   	static_call(kvm_x86_hardware_unsetup)();
>   }
>   
> 

Acked-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:24:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:24:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192125.342399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvfz-0007II-PU; Wed, 22 Sep 2021 06:24:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192125.342399; Wed, 22 Sep 2021 06:24:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvfz-0007IB-MO; Wed, 22 Sep 2021 06:24:15 +0000
Received: by outflank-mailman (input) for mailman id 192125;
 Wed, 22 Sep 2021 06:24:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvfz-0007I4-39
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:24:15 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 6e9e0f13-bfbd-4926-bbdd-4c50b85607b3;
 Wed, 22 Sep 2021 06:24:14 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-494-9Tl2PZFGMLe5OcNvjGrSrA-1; Wed, 22 Sep 2021 02:24:10 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 f7-20020a5d50c7000000b0015e288741a4so1138577wrt.9
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:24:10 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id z13sm1236395wrs.90.2021.09.21.23.24.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:24:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e9e0f13-bfbd-4926-bbdd-4c50b85607b3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632291854;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1o72rZ3J4EUS5ZaXq1DHK1qEjD5sQ31Ny28qqm6SkjQ=;
	b=UuzsoFoDGTsrkYKOJFhy9YLdYY1Nx1Qnmw0mmXziWO/6mmJOSsOkleM0cV795oT45K0bYq
	91ZQTSON69/+9eVKTksFnL+rnTfWhtzCIy5x3WAxBbL+scLCHRh31/lcCtTe26r2H2JvgI
	tUZZpqn8yjoMCVtZ37i9zjkTDpN8Xps=
X-MC-Unique: 9Tl2PZFGMLe5OcNvjGrSrA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=1o72rZ3J4EUS5ZaXq1DHK1qEjD5sQ31Ny28qqm6SkjQ=;
        b=tpSpOJt2SXBANPQlskyruzHRoj+7g4Y84hLqR8JkcNTDB9mYdu17lJxxgjoOiBvzyE
         UHCew2TStxghNMGqQ1/HqvzF8i2O17GKOyLlNIbJXpq0YcGGft15MBJCQsPdWyntP/PT
         6Wy8EFpgKq5W8s8FjIaZV+Q1s0Tla4xneyZngvoLX2uW0CbxP6HaQznXsz5FVVXh18I1
         2a3yVH+8yvWMrOyurxut9VxBL/YaqK8dwqWAoHpEF/5edGaO/i8oIszjc9Sd7DiGPXYt
         x+rwRB5Mif1glwwOfvU0ue/hXRIvjxJmU1sFURNIjeuRqWXcaywmqtm0EiAYkhRf8fSQ
         x2ww==
X-Gm-Message-State: AOAM530quJjbuKFOuLJCh5ZLeVDXCc/D3rXgRvZhltVfAP9LYn8zSyTP
	gQRkegj758RoONlG/XQpUXrltG7c6/bfH9qMLK1UuT0BvmuUIPON8E1hBUVQkNS6IecK2UYNi11
	W3eB2tLr0XPPyoADCsPPnFmMyZs8=
X-Received: by 2002:a1c:448a:: with SMTP id r132mr8797120wma.105.1632291849597;
        Tue, 21 Sep 2021 23:24:09 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxQTq0zGwJ27i2S0Z0f0XPnoR50fJdudIkaF4OKBHHQ4AgCyC4Ha/kLdM9RRJcXFpLjepRj8g==
X-Received: by 2002:a1c:448a:: with SMTP id r132mr8797078wma.105.1632291849339;
        Tue, 21 Sep 2021 23:24:09 -0700 (PDT)
Subject: Re: [PATCH v3 03/16] KVM: x86: Register Processor Trace interrupt
 hook iff PT enabled in guest
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-4-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <459f2af0-d4ea-955e-fb2b-e118140ee9bf@redhat.com>
Date: Wed, 22 Sep 2021 08:24:00 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-4-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Override the Processor Trace (PT) interrupt handler for guest mode if and
> only if PT is configured for host+guest mode, i.e. is being used
> independently by both host and guest.  If PT is configured for system
> mode, the host fully controls PT and must handle all events.
> 
> Fixes: 8479e04e7d6b ("KVM: x86: Inject PMI for KVM guest")
> Cc: stable@vger.kernel.org
> Cc: Like Xu <like.xu.linux@gmail.com>
> Reported-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> Reported-by: Artem Kashkanov <artem.kashkanov@intel.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/x86/include/asm/kvm_host.h | 1 +
>   arch/x86/kvm/vmx/vmx.c          | 1 +
>   arch/x86/kvm/x86.c              | 5 ++++-
>   3 files changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 09b256db394a..1ea4943a73d7 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1494,6 +1494,7 @@ struct kvm_x86_init_ops {
>   	int (*disabled_by_bios)(void);
>   	int (*check_processor_compatibility)(void);
>   	int (*hardware_setup)(void);
> +	bool (*intel_pt_intr_in_guest)(void);
>   
>   	struct kvm_x86_ops *runtime_ops;
>   };
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index fada1055f325..f19d72136f77 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -7896,6 +7896,7 @@ static struct kvm_x86_init_ops vmx_init_ops __initdata = {
>   	.disabled_by_bios = vmx_disabled_by_bios,
>   	.check_processor_compatibility = vmx_check_processor_compat,
>   	.hardware_setup = hardware_setup,
> +	.intel_pt_intr_in_guest = vmx_pt_mode_is_host_guest,
>   
>   	.runtime_ops = &vmx_x86_ops,
>   };
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index fb6015f97f9e..ffc6c2d73508 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8305,7 +8305,7 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = {
>   	.is_in_guest		= kvm_is_in_guest,
>   	.is_user_mode		= kvm_is_user_mode,
>   	.get_guest_ip		= kvm_get_guest_ip,
> -	.handle_intel_pt_intr	= kvm_handle_intel_pt_intr,
> +	.handle_intel_pt_intr	= NULL,
>   };
>   
>   #ifdef CONFIG_X86_64
> @@ -11061,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque)
>   	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
>   	kvm_ops_static_call_update();
>   
> +	if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest())
> +		kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr;
>   	perf_register_guest_info_callbacks(&kvm_guest_cbs);
>   
>   	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
> @@ -11091,6 +11093,7 @@ int kvm_arch_hardware_setup(void *opaque)
>   void kvm_arch_hardware_unsetup(void)
>   {
>   	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
> +	kvm_guest_cbs.handle_intel_pt_intr = NULL;
>   
>   	static_call(kvm_x86_hardware_unsetup)();
>   }
> 

Acked-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:25:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:25:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192131.342410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvhc-0007vs-4u; Wed, 22 Sep 2021 06:25:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192131.342410; Wed, 22 Sep 2021 06:25:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvhc-0007vl-1M; Wed, 22 Sep 2021 06:25:56 +0000
Received: by outflank-mailman (input) for mailman id 192131;
 Wed, 22 Sep 2021 06:25:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvha-0007vX-C2
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:25:54 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id d59ba1fd-18d6-4068-af71-f413c35fec71;
 Wed, 22 Sep 2021 06:25:53 +0000 (UTC)
Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com
 [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-258-vJgMAaylMuWdhdjrEGpvKA-1; Wed, 22 Sep 2021 02:25:50 -0400
Received: by mail-wr1-f72.google.com with SMTP id
 u10-20020adfae4a000000b0016022cb0d2bso1108753wrd.19
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:25:49 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id 25sm5710788wmo.9.2021.09.21.23.25.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:25:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d59ba1fd-18d6-4068-af71-f413c35fec71
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632291953;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gdG5hvixHxpBF3qV2Nc1iurvw1d5rRrVn0cpW/kHHXU=;
	b=S7IqHfFFMo3fmhh0WXJADOV91hRXYxrSVRSsTgBWmU5uKny4aSMEDeoSho4E/tkwaPqReL
	vMzSmCQPMUtHqXI2FnwTYTCJoI6vVOA4UebkT8I6nJ+y5bJtB4skjb17Km3U0M7bH6iB2y
	uHvDF7d84CnnOuefeI+7bp8/InxMCr0=
X-MC-Unique: vJgMAaylMuWdhdjrEGpvKA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=gdG5hvixHxpBF3qV2Nc1iurvw1d5rRrVn0cpW/kHHXU=;
        b=XoOvx3SaWOT098iULu4aP8E0jO8WaGMoK9AkFdspmEiVjJfJPAs5yIjNBYQWmiqJBN
         WZHFRnaXjYrMea0EOuXYyN5IBFcP52RqkQ2NW3C8H62/cHgeg4+ErIoFFtH+JiN6lhW+
         ZCFHdesMmRiULQchGE0NH3b0xFwNt9tmPd2+udbXgijLbDDdopLf/7Vh8CPdkiWO9vS6
         O8oMPj6Lhr2vwCXVM/Lz+ZM8/zDva2Vmnby+SDDUsP2lgre5QlwQMzO/AtNjXGmEZkaW
         rH+sA22rsV4usz59VFLGwN9NYUV11cNgSOA3TCqS9yZSB39xQ6WPqo4qjmE1l5aZoI0q
         xwJA==
X-Gm-Message-State: AOAM531YGo7eu2otzF67qqDKgQ052ThcGJbpBEBmYQTfKfYfWwDmJDUy
	sHN5TJcLfvSuLj8p9r1hsj+/x06JMQoCHzvhsuthD0IlownDT62WNHcPYP2oKg1x64oq5YZfl4a
	4gp5VLtyG8r9iNd3ZKXga19vMMGg=
X-Received: by 2002:a1c:f302:: with SMTP id q2mr8427304wmq.56.1632291948895;
        Tue, 21 Sep 2021 23:25:48 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzzItVKh/AhZh5/Igm3bcU7YShmtTIldU49veASOpN+187Ot1E0vGGn3OxfbqcgagpWMesDpA==
X-Received: by 2002:a1c:f302:: with SMTP id q2mr8427271wmq.56.1632291948590;
        Tue, 21 Sep 2021 23:25:48 -0700 (PDT)
Subject: Re: [PATCH v3 04/16] perf: Stop pretending that perf can handle
 multiple guest callbacks
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-5-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <37afc465-c12f-01b9-f3b6-c2573e112d76@redhat.com>
Date: Wed, 22 Sep 2021 08:25:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-5-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Drop the 'int' return value from the perf (un)register callbacks helpers
> and stop pretending perf can support multiple callbacks.  The 'int'
> returns are not future proofing anything as none of the callers take
> action on an error.  It's also not obvious that there will ever be
> co-tenant hypervisors, and if there are, that allowing multiple callbacks
> to be registered is desirable or even correct.
> 
> Opportunistically rename callbacks=>cbs in the affected declarations to
> match their definitions.
> 
> No functional change intended.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/arm64/include/asm/kvm_host.h |  4 ++--
>   arch/arm64/kvm/perf.c             |  8 ++++----
>   include/linux/perf_event.h        | 12 ++++++------
>   kernel/events/core.c              | 16 ++++------------
>   4 files changed, 16 insertions(+), 24 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 41911585ae0c..ed940aec89e0 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -670,8 +670,8 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len);
>   int kvm_handle_mmio_return(struct kvm_vcpu *vcpu);
>   int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa);
>   
> -int kvm_perf_init(void);
> -int kvm_perf_teardown(void);
> +void kvm_perf_init(void);
> +void kvm_perf_teardown(void);
>   
>   long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu);
>   gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c
> index 151c31fb9860..c37c0cf1bfe9 100644
> --- a/arch/arm64/kvm/perf.c
> +++ b/arch/arm64/kvm/perf.c
> @@ -48,15 +48,15 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = {
>   	.get_guest_ip	= kvm_get_guest_ip,
>   };
>   
> -int kvm_perf_init(void)
> +void kvm_perf_init(void)
>   {
>   	if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled())
>   		static_branch_enable(&kvm_arm_pmu_available);
>   
> -	return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> +	perf_register_guest_info_callbacks(&kvm_guest_cbs);
>   }
>   
> -int kvm_perf_teardown(void)
> +void kvm_perf_teardown(void)
>   {
> -	return perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
> +	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
>   }
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index 6b0405e578c1..317d4658afe9 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -1245,8 +1245,8 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
>   	/* Prevent reloading between a !NULL check and dereferences. */
>   	return READ_ONCE(perf_guest_cbs);
>   }
> -extern int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
> -extern int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks);
> +extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
> +extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
>   
>   extern void perf_event_exec(void);
>   extern void perf_event_comm(struct task_struct *tsk, bool exec);
> @@ -1489,10 +1489,10 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)	{ }
>   static inline void
>   perf_bp_event(struct perf_event *event, void *data)			{ }
>   
> -static inline int perf_register_guest_info_callbacks
> -(struct perf_guest_info_callbacks *callbacks)				{ return 0; }
> -static inline int perf_unregister_guest_info_callbacks
> -(struct perf_guest_info_callbacks *callbacks)				{ return 0; }
> +static inline void perf_register_guest_info_callbacks
> +(struct perf_guest_info_callbacks *cbs)					{ }
> +static inline void perf_unregister_guest_info_callbacks
> +(struct perf_guest_info_callbacks *cbs)					{ }
>   
>   static inline void perf_event_mmap(struct vm_area_struct *vma)		{ }
>   
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 80ff050a7b55..d90a43572400 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -6482,31 +6482,23 @@ static void perf_pending_event(struct irq_work *entry)
>   		perf_swevent_put_recursion_context(rctx);
>   }
>   
> -/*
> - * We assume there is only KVM supporting the callbacks.
> - * Later on, we might change it to a list if there is
> - * another virtualization implementation supporting the callbacks.
> - */
>   struct perf_guest_info_callbacks *perf_guest_cbs;
> -
> -int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
> +void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
>   {
>   	if (WARN_ON_ONCE(perf_guest_cbs))
> -		return -EBUSY;
> +		return;
>   
>   	WRITE_ONCE(perf_guest_cbs, cbs);

Maybe you want a smp_store_release or rcu_assign_pointer here?

> -	return 0;
>   }
>   EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
>   
> -int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
> +void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
>   {
>   	if (WARN_ON_ONCE(perf_guest_cbs != cbs))
> -		return -EINVAL;
> +		return;
>   
>   	WRITE_ONCE(perf_guest_cbs, NULL);
>   	synchronize_rcu();
> -	return 0;
>   }
>   EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
>   
> 

Apart from the above,

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:26:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:26:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192139.342420 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSviM-00009y-IE; Wed, 22 Sep 2021 06:26:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192139.342420; Wed, 22 Sep 2021 06:26:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSviM-00009p-F8; Wed, 22 Sep 2021 06:26:42 +0000
Received: by outflank-mailman (input) for mailman id 192139;
 Wed, 22 Sep 2021 06:26:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSviK-00009c-HR
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:26:40 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 24565a95-85ae-4b3d-8836-86931cd91348;
 Wed, 22 Sep 2021 06:26:39 +0000 (UTC)
Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com
 [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-529-KQBiaRewM4miRd-zf4XPGA-1; Wed, 22 Sep 2021 02:26:36 -0400
Received: by mail-wr1-f70.google.com with SMTP id
 r15-20020adfce8f000000b0015df1098ccbso1156199wrn.4
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:26:36 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id m4sm1326977wrx.81.2021.09.21.23.26.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:26:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24565a95-85ae-4b3d-8836-86931cd91348
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632291999;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MBiMg3FiniiEqjDXbzG5q84o5gFLCF2xYeMjdShm8Ns=;
	b=F2V3P24BT/xxBY6gNs2FwObv3qICDLKFVDwLtkMWRgVqHQCXyHQZNxz4grHhVpiuK2GGqU
	mYK7DgwsnvZnRgerMWzzX3XIMfn++KwHA2vPlvdxJ19fcP/5JskDxzYDieHG4NGbzYsrwP
	xm8w6shYOCDWmHycydwTtK/1xzi/swA=
X-MC-Unique: KQBiaRewM4miRd-zf4XPGA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=MBiMg3FiniiEqjDXbzG5q84o5gFLCF2xYeMjdShm8Ns=;
        b=dZYOxr36q9SQAa0TolaaMoOL2g8y4TqG3dD61RT+///aRAiCz8vWznaPhr1c3d2o6D
         N89/UzKjVg7UrCqfyNxxXwkahyHpo/8zCt1fJOjJRwiM6UqxzG5ezicD15b16X5LNnW5
         jzUhvmXGosRHkeSi7t05kF7CrV2HHHB2mEobq0QxTK8SFHWAS6WHnjtRzCb+nnIyojXj
         T3k2ePJQVO9ga0kd5+56WGwGM1pUx3jI9Sse7d5quIfDru3z93wBJSkjvoftwwghP1Vd
         wpL5qnA5CkAdN+G6JToO7I2p4itopsS3sfvaoigBXAzSO899D+Ef19Ym4t2lGE2s8TjY
         23vA==
X-Gm-Message-State: AOAM532Tzm3laVnT6JWWZhBJ5nDD0NSEAGCj3rALCXjfP36ZMgbtoYbU
	gikbjPAj2R0ntrtkMpDH8VBsR11P5bIlM4lM8s8gD8vDFuL41QgUkbz9jxYqWj/nTXlFARXEa44
	sZuxY2XEN45W0n5C1BqPsMAUiB04=
X-Received: by 2002:a5d:4e90:: with SMTP id e16mr39082840wru.243.1632291995509;
        Tue, 21 Sep 2021 23:26:35 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyl20akxP3NXahRl1WcHnQKmZ1IfpNcPRYFb7QcuxtqBG6hCoEWyeXUt8EoPViJOjXQwzqFiA==
X-Received: by 2002:a5d:4e90:: with SMTP id e16mr39082789wru.243.1632291995216;
        Tue, 21 Sep 2021 23:26:35 -0700 (PDT)
Subject: Re: [PATCH v3 05/16] perf: Drop dead and useless guest "support" from
 arm, csky, nds32 and riscv
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-6-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <2462bc44-64a5-8bac-7c3c-d837c4b3f743@redhat.com>
Date: Wed, 22 Sep 2021 08:26:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-6-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Drop "support" for guest callbacks from architctures that don't implement
> the guest callbacks.  Future patches will convert the callbacks to
> static_call; rather than churn a bunch of arch code (that was presumably
> copy+pasted from x86), remove it wholesale as it's useless and at best
> wasting cycles.
> 
> A future patch will also add a Kconfig to force architcture to opt into
> the callbacks to make it more difficult for uses "support" to sneak in in
> the future.
> 
> No functional change intended.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/arm/kernel/perf_callchain.c   | 33 ++++-------------------------
>   arch/csky/kernel/perf_callchain.c  | 12 -----------
>   arch/nds32/kernel/perf_event_cpu.c | 34 ++++--------------------------
>   arch/riscv/kernel/perf_callchain.c | 13 ------------
>   4 files changed, 8 insertions(+), 84 deletions(-)
> 
> diff --git a/arch/arm/kernel/perf_callchain.c b/arch/arm/kernel/perf_callchain.c
> index 1626dfc6f6ce..bc6b246ab55e 100644
> --- a/arch/arm/kernel/perf_callchain.c
> +++ b/arch/arm/kernel/perf_callchain.c
> @@ -62,14 +62,8 @@ user_backtrace(struct frame_tail __user *tail,
>   void
>   perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct frame_tail __user *tail;
>   
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		/* We don't support guest os callchain now */
> -		return;
> -	}
> -
>   	perf_callchain_store(entry, regs->ARM_pc);
>   
>   	if (!current->mm)
> @@ -99,44 +93,25 @@ callchain_trace(struct stackframe *fr,
>   void
>   perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct stackframe fr;
>   
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		/* We don't support guest os callchain now */
> -		return;
> -	}
> -
>   	arm_get_current_stackframe(regs, &fr);
>   	walk_stackframe(&fr, callchain_trace, entry);
>   }
>   
>   unsigned long perf_instruction_pointer(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	if (guest_cbs && guest_cbs->is_in_guest())
> -		return guest_cbs->get_guest_ip();
> -
>   	return instruction_pointer(regs);
>   }
>   
>   unsigned long perf_misc_flags(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	int misc = 0;
>   
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		if (guest_cbs->is_user_mode())
> -			misc |= PERF_RECORD_MISC_GUEST_USER;
> -		else
> -			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
> -	} else {
> -		if (user_mode(regs))
> -			misc |= PERF_RECORD_MISC_USER;
> -		else
> -			misc |= PERF_RECORD_MISC_KERNEL;
> -	}
> +	if (user_mode(regs))
> +		misc |= PERF_RECORD_MISC_USER;
> +	else
> +		misc |= PERF_RECORD_MISC_KERNEL;
>   
>   	return misc;
>   }
> diff --git a/arch/csky/kernel/perf_callchain.c b/arch/csky/kernel/perf_callchain.c
> index 35318a635a5f..92057de08f4f 100644
> --- a/arch/csky/kernel/perf_callchain.c
> +++ b/arch/csky/kernel/perf_callchain.c
> @@ -86,13 +86,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
>   void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   			 struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	unsigned long fp = 0;
>   
> -	/* C-SKY does not support virtualization. */
> -	if (guest_cbs && guest_cbs->is_in_guest())
> -		return;
> -
>   	fp = regs->regs[4];
>   	perf_callchain_store(entry, regs->pc);
>   
> @@ -111,15 +106,8 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   			   struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct stackframe fr;
>   
> -	/* C-SKY does not support virtualization. */
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		pr_warn("C-SKY does not support perf in guest mode!");
> -		return;
> -	}
> -
>   	fr.fp = regs->regs[4];
>   	fr.lr = regs->lr;
>   	walk_stackframe(&fr, entry);
> diff --git a/arch/nds32/kernel/perf_event_cpu.c b/arch/nds32/kernel/perf_event_cpu.c
> index f38791960781..a78a879e7ef1 100644
> --- a/arch/nds32/kernel/perf_event_cpu.c
> +++ b/arch/nds32/kernel/perf_event_cpu.c
> @@ -1363,7 +1363,6 @@ void
>   perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   		    struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	unsigned long fp = 0;
>   	unsigned long gp = 0;
>   	unsigned long lp = 0;
> @@ -1372,11 +1371,6 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   
>   	leaf_fp = 0;
>   
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		/* We don't support guest os callchain now */
> -		return;
> -	}
> -
>   	perf_callchain_store(entry, regs->ipc);
>   	fp = regs->fp;
>   	gp = regs->gp;
> @@ -1480,13 +1474,8 @@ void
>   perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   		      struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct stackframe fr;
>   
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		/* We don't support guest os callchain now */
> -		return;
> -	}
>   	fr.fp = regs->fp;
>   	fr.lp = regs->lp;
>   	fr.sp = regs->sp;
> @@ -1495,32 +1484,17 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   
>   unsigned long perf_instruction_pointer(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	/* However, NDS32 does not support virtualization */
> -	if (guest_cbs && guest_cbs->is_in_guest())
> -		return guest_cbs->get_guest_ip();
> -
>   	return instruction_pointer(regs);
>   }
>   
>   unsigned long perf_misc_flags(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	int misc = 0;
>   
> -	/* However, NDS32 does not support virtualization */
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		if (guest_cbs->is_user_mode())
> -			misc |= PERF_RECORD_MISC_GUEST_USER;
> -		else
> -			misc |= PERF_RECORD_MISC_GUEST_KERNEL;
> -	} else {
> -		if (user_mode(regs))
> -			misc |= PERF_RECORD_MISC_USER;
> -		else
> -			misc |= PERF_RECORD_MISC_KERNEL;
> -	}
> +	if (user_mode(regs))
> +		misc |= PERF_RECORD_MISC_USER;
> +	else
> +		misc |= PERF_RECORD_MISC_KERNEL;
>   
>   	return misc;
>   }
> diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
> index 8ecfc4c128bc..1fc075b8f764 100644
> --- a/arch/riscv/kernel/perf_callchain.c
> +++ b/arch/riscv/kernel/perf_callchain.c
> @@ -56,13 +56,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry,
>   void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   			 struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	unsigned long fp = 0;
>   
> -	/* RISC-V does not support perf in guest mode. */
> -	if (guest_cbs && guest_cbs->is_in_guest())
> -		return;
> -
>   	fp = regs->s0;
>   	perf_callchain_store(entry, regs->epc);
>   
> @@ -79,13 +74,5 @@ static bool fill_callchain(void *entry, unsigned long pc)
>   void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   			   struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	/* RISC-V does not support perf in guest mode. */
> -	if (guest_cbs && guest_cbs->is_in_guest()) {
> -		pr_warn("RISC-V does not support perf in guest mode!");
> -		return;
> -	}
> -
>   	walk_stackframe(NULL, regs, fill_callchain, entry);
>   }
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:29:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192145.342432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvkb-00010V-WD; Wed, 22 Sep 2021 06:29:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192145.342432; Wed, 22 Sep 2021 06:29:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvkb-00010O-Sf; Wed, 22 Sep 2021 06:29:01 +0000
Received: by outflank-mailman (input) for mailman id 192145;
 Wed, 22 Sep 2021 06:29:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvkb-00010I-5Y
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:29:01 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 5f5e3177-1b6e-11ec-b95f-12813bfff9fa;
 Wed, 22 Sep 2021 06:29:00 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-388-CRwhY5nVO22gZ90a4b3SMA-1; Wed, 22 Sep 2021 02:28:58 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 m1-20020a056000180100b0015e1ec30ac3so1145089wrh.8
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:28:58 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id i18sm1136021wrn.64.2021.09.21.23.28.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:28:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f5e3177-1b6e-11ec-b95f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292140;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pJ7mY3KGJnt0GGObf9665ZCn7DQvGIxKlz9Hgzfe7VE=;
	b=Ah6F7aW5/exccYXC1OVMx5XAUJSgBfMU9UjQW3vy82NGWuB7IHHWUbst15q9aKgbi61m1R
	6xtU221qIfogFbksD81bH4UyMgm0TgzITsoPmxn35yTy+dSFgwUZI+QluwQbcauAFBXG8n
	5gL0qX9f/vf+qoNQX5SPW2JkhnRz5lI=
X-MC-Unique: CRwhY5nVO22gZ90a4b3SMA-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=pJ7mY3KGJnt0GGObf9665ZCn7DQvGIxKlz9Hgzfe7VE=;
        b=qKtI2wnc9MG9PoumStk7royVKZ2cGmxcaWAwqni99B8zZBzqJZanvSCHFMhno46TmO
         4/WLQYXkL4KbokXtPgQYp+F8BsWUfWGawfJ5Fs5Ehl0OTxb1rzJkubQOs3m5SN823Uyj
         9kdDcf81XV/xPce5FlMBezlRvVCedk5pQCcWqkFjUhneMHyT+bVfrUL/KKk49skeK4BV
         tsa7hVbFpbSrHV+tH/fdqkgYrFZhE5uuMTtfiahHLbTN+g6iOq3+fi/TDUBJu/zPtTBU
         4q9vEjOaYmRKveuT0nlcgY8Kqrq+c9i8qGCPP8FxekB7tM0rejK14Oq7Ndl6Dl1+odvF
         hxBA==
X-Gm-Message-State: AOAM533+QMHPH7djp3uJr/ml578UydWLo/id+ia6stj4kBeID5/N+Y2v
	t02Dwg7UPiIE2w5XU2l7oJqAEdc19fK8l5aqHhTbEV50uB7At0E5WSqVBWf81vrRqVQnJgh2oyj
	i5kX+aM7tmV010VtWIjNgv0FaBII=
X-Received: by 2002:a05:600c:3ba4:: with SMTP id n36mr8722438wms.35.1632292137444;
        Tue, 21 Sep 2021 23:28:57 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzGUh0clwu4unN1cIXXPNEQKf8qlYc4JIONnRibnq73sMmW8cPrqysrS+MXk+0qjWI0DZnAvw==
X-Received: by 2002:a05:600c:3ba4:: with SMTP id n36mr8722395wms.35.1632292137204;
        Tue, 21 Sep 2021 23:28:57 -0700 (PDT)
Subject: Re: [PATCH v3 06/16] perf/core: Rework guest callbacks to prepare for
 static_call support
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-7-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <6636ab49-e21c-601f-9d2c-34e681c14faa@redhat.com>
Date: Wed, 22 Sep 2021 08:28:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-7-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> To prepare for using static_calls to optimize perf's guest callbacks,
> replace ->is_in_guest and ->is_user_mode with a new multiplexed hook
> ->state, tweak ->handle_intel_pt_intr to play nice with being called when
> there is no active guest, and drop "guest" from ->is_in_guest.

... from ->get_guest_ip.  Code-wise,

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:29:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:29:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192151.342443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvl0-0001XD-88; Wed, 22 Sep 2021 06:29:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192151.342443; Wed, 22 Sep 2021 06:29:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvl0-0001X6-59; Wed, 22 Sep 2021 06:29:26 +0000
Received: by outflank-mailman (input) for mailman id 192151;
 Wed, 22 Sep 2021 06:29:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvky-0001Sc-OR
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:29:24 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id f891e790-5375-4533-b9e4-3427f57cdc13;
 Wed, 22 Sep 2021 06:29:23 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-96-4tPbb6voMQGrgd5LJjbOyw-1; Wed, 22 Sep 2021 02:29:22 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 c15-20020a5d4ccf000000b0015dff622f39so1107656wrt.21
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:29:21 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id n26sm4980232wmi.43.2021.09.21.23.29.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:29:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f891e790-5375-4533-b9e4-3427f57cdc13
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292163;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=E/tHH9wLLtazV94QpN+BmGuu66vagS3lkpjDcUcpVzA=;
	b=PGPf9ZzB33LZyiZilg2aozwZm5TDj5p0O5vz/gJCSjvz+UFxd649DuOMY96TZ0C7Yhswar
	Jo7WK7EKH7drHXehvlque90NgI8miKqAqdhI8I3mYPRJ1+VBPU59VAOBjhjFTrRdyNQnyy
	WXSIXoeyFvkhHyhwswuM+UhVvVnYRvw=
X-MC-Unique: 4tPbb6voMQGrgd5LJjbOyw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=E/tHH9wLLtazV94QpN+BmGuu66vagS3lkpjDcUcpVzA=;
        b=b1kBC0hcAgzTMWCAKT1GlpBxEV0XGv/EqJ13OcCH4F7p8HDl4G0txW4pMePOWuu3Qz
         pbbT6s5wSt+T7qWQBVPncjoJrxsuMH/f8U6p0TprzfftMb/g+0LiBZh7n2lyiH58B74X
         T53JV2F2wGtLJ1P92hpBYnziO+j3AmJjSlHLt3vKzSV9vLxvIGto3mPqYhTI7a2TOMyA
         htrsHB2+8Z4W+/jYQ7CwYx/I0Zj3DkAJEFZ0uGazzEJD/9mcgAWkLb0YpwJngdrGD1jj
         +J+5XTFvqcU6klZ2nrzzHqC+hPMkeK0ZHEHcD8k3DKw2XWOgo/He04VJvhd5ao2lrV8j
         azqg==
X-Gm-Message-State: AOAM531hl5tsGo+sVRSQ2R7Qy45os3xTps4MtfkGrBs2pZKE4L18NVM5
	LldDJypvfI1HNDAdVrRFHSDvAqkxYRQmSOk6JUvhGR/U40dZnukjRrF5XdnGXgpGhRBq8X3l/mE
	pPWr4Lc6qOTN/1ZMR2blZBGKMi1w=
X-Received: by 2002:a5d:64a7:: with SMTP id m7mr39506109wrp.171.1632292161047;
        Tue, 21 Sep 2021 23:29:21 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyhBqcw62XwXR9WbHiblbcM9n6+4WHbybrr5/MkhCq7T4dnVdh+qdYz2KsHuqdWFBJH3F6z5Q==
X-Received: by 2002:a5d:64a7:: with SMTP id m7mr39506073wrp.171.1632292160784;
        Tue, 21 Sep 2021 23:29:20 -0700 (PDT)
Subject: Re: [PATCH v3 16/16] perf: Drop guest callback (un)register stubs
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-17-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <74d2e94f-fbd3-522e-8fc5-da5a0a575838@redhat.com>
Date: Wed, 22 Sep 2021 08:29:17 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-17-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Drop perf's stubs for (un)registering guest callbacks now that KVM
> registration of callbacks is hidden behind GUEST_PERF_EVENTS=y.  The only
> other user is x86 XEN_PV, and x86 unconditionally selects PERF_EVENTS.
> 
> No functional change intended.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   include/linux/perf_event.h | 5 -----
>   1 file changed, 5 deletions(-)
> 
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index d582dfeb4e20..20327d1046bb 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -1505,11 +1505,6 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)	{ }
>   static inline void
>   perf_bp_event(struct perf_event *event, void *data)			{ }
>   
> -static inline void perf_register_guest_info_callbacks
> -(struct perf_guest_info_callbacks *cbs)					{ }
> -static inline void perf_unregister_guest_info_callbacks
> -(struct perf_guest_info_callbacks *cbs)					{ }
> -
>   static inline void perf_event_mmap(struct vm_area_struct *vma)		{ }
>   
>   typedef int (perf_ksymbol_get_name_f)(char *name, int name_len, void *data);
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:30:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:30:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192157.342453 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvlg-0002sM-Hd; Wed, 22 Sep 2021 06:30:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192157.342453; Wed, 22 Sep 2021 06:30:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvlg-0002sF-EQ; Wed, 22 Sep 2021 06:30:08 +0000
Received: by outflank-mailman (input) for mailman id 192157;
 Wed, 22 Sep 2021 06:30:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvlf-0002Lk-KL
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:30:07 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 0562a00c-b26c-455b-a892-da2fe970e63f;
 Wed, 22 Sep 2021 06:30:04 +0000 (UTC)
Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com
 [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-108-vdyzb6VpMb6DjCMZtBbkHw-1; Wed, 22 Sep 2021 02:30:00 -0400
Received: by mail-wr1-f70.google.com with SMTP id
 z2-20020a5d4c82000000b0015b140e0562so1158604wrs.7
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:30:00 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id k4sm947672wmj.30.2021.09.21.23.29.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:29:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0562a00c-b26c-455b-a892-da2fe970e63f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292204;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uMqrFogGHJvh/zXMgi/UYOAI5rgMXo+1p47tZLBtnbA=;
	b=DiuuH9EhrM8W/oLHFBe9os8P83K6MQtz4/owQwgTebme5DrT6IOo+6qAtUYFIQJGNipT2W
	doAjMS4w6d4kd3HE9MDawo/kagm7uPIKGETa4KDlDNgYy2tEFfSqbzit74Ed+VYAiF5osb
	uJXx7l4NwSGbmMu0l39PjvPojARZdck=
X-MC-Unique: vdyzb6VpMb6DjCMZtBbkHw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=uMqrFogGHJvh/zXMgi/UYOAI5rgMXo+1p47tZLBtnbA=;
        b=fu282/7nvAUVq4Wrr4YmQ9GQA8R9aWZP+X9Qh4gu8EpqYRLrve941XK0/O+uXnSCoI
         NfJObmzG1Uw+LsIrT9dmMW+az0uSjIqvKDFRKBNzVu7vTAGuHacBIU+DGNtmkGEtsPxH
         o9o/kPJAy2ZqLgpbJXr5Nlr+KO8Wimy651nwxp1kMtMctwmOxZl5wBAQ+Eg0fQrBFe9x
         1MSow9NyFEoGvESATlOrZ6MECV7IN395q/+JgTxCzge0H0iiHYdCWiITZcTA9k7yqazn
         C0+vwqsMLNRsdFUp8MB2itkbJchgi5hkQUnMSJN537qmUGUymAXryQMqZ2APiaLUXsV3
         1Qmg==
X-Gm-Message-State: AOAM531hYHhvADN5hM25GZr3uTV5JP/P6KIyMMSBjvCLMDI6FBsb8fjf
	gOz5rhM1DV46oMfqPADmIK6FDZeymco7edTtEtxvrB1NKkGRiKIv9ieFeFREp2Bu0uo1E2fiTFG
	KCxkUbvuUi8jXg8faBA8DjYw9jp0=
X-Received: by 2002:adf:ef92:: with SMTP id d18mr40001556wro.264.1632292199588;
        Tue, 21 Sep 2021 23:29:59 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJw6eHdpjSr8WTnz77WfeSYfS+jJfgrKqObxO+Wy3OIPCrDpgLHnqBvaUwGRJU1fc0Wlc4Hr5Q==
X-Received: by 2002:adf:ef92:: with SMTP id d18mr40001494wro.264.1632292199252;
        Tue, 21 Sep 2021 23:29:59 -0700 (PDT)
Subject: Re: [PATCH v3 07/16] perf: Add wrappers for invoking guest callbacks
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-8-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <a67e5fde-8b96-47b9-f1dd-c8be8ea9da7f@redhat.com>
Date: Wed, 22 Sep 2021 08:29:53 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-8-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Add helpers for the guest callbacks to prepare for burying the callbacks
> behind a Kconfig (it's a lot easier to provide a few stubs than to #ifdef
> piles of code), and also to prepare for converting the callbacks to
> static_call().  perf_instruction_pointer() in particular will have subtle
> semantics with static_call(), as the "no callbacks" case will return 0 if
> the callbacks are unregistered between querying guest state and getting
> the IP.  Implement the change now to avoid a functional change when adding
> static_call() support, and because the new helper needs to return
> _something_ in this case.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/arm64/kernel/perf_callchain.c | 16 +++++-----------
>   arch/x86/events/core.c             | 15 +++++----------
>   arch/x86/events/intel/core.c       |  5 +----
>   include/linux/perf_event.h         | 24 ++++++++++++++++++++++++
>   4 files changed, 35 insertions(+), 25 deletions(-)
> 
> diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c
> index 274dc3e11b6d..db04a55cee7e 100644
> --- a/arch/arm64/kernel/perf_callchain.c
> +++ b/arch/arm64/kernel/perf_callchain.c
> @@ -102,9 +102,7 @@ compat_user_backtrace(struct compat_frame_tail __user *tail,
>   void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>   			 struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	if (guest_cbs && guest_cbs->state()) {
> +	if (perf_guest_state()) {
>   		/* We don't support guest os callchain now */
>   		return;
>   	}
> @@ -149,10 +147,9 @@ static bool callchain_trace(void *data, unsigned long pc)
>   void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   			   struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct stackframe frame;
>   
> -	if (guest_cbs && guest_cbs->state()) {
> +	if (perf_guest_state()) {
>   		/* We don't support guest os callchain now */
>   		return;
>   	}
> @@ -163,18 +160,15 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
>   
>   unsigned long perf_instruction_pointer(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	if (guest_cbs && guest_cbs->state())
> -		return guest_cbs->get_ip();
> +	if (perf_guest_state())
> +		return perf_guest_get_ip();
>   
>   	return instruction_pointer(regs);
>   }
>   
>   unsigned long perf_misc_flags(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
> +	unsigned int guest_state = perf_guest_state();
>   	int misc = 0;
>   
>   	if (guest_state) {
> diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> index 3a7630fdd340..d20e4f8d1aef 100644
> --- a/arch/x86/events/core.c
> +++ b/arch/x86/events/core.c
> @@ -2761,11 +2761,10 @@ static bool perf_hw_regs(struct pt_regs *regs)
>   void
>   perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct unwind_state state;
>   	unsigned long addr;
>   
> -	if (guest_cbs && guest_cbs->state()) {
> +	if (perf_guest_state()) {
>   		/* TODO: We don't support guest os callchain now */
>   		return;
>   	}
> @@ -2865,11 +2864,10 @@ perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry_ctx *ent
>   void
>   perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
>   	struct stack_frame frame;
>   	const struct stack_frame __user *fp;
>   
> -	if (guest_cbs && guest_cbs->state()) {
> +	if (perf_guest_state()) {
>   		/* TODO: We don't support guest os callchain now */
>   		return;
>   	}
> @@ -2946,18 +2944,15 @@ static unsigned long code_segment_base(struct pt_regs *regs)
>   
>   unsigned long perf_instruction_pointer(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	if (guest_cbs && guest_cbs->state())
> -		return guest_cbs->get_ip();
> +	if (perf_guest_state())
> +		return perf_guest_get_ip();
>   
>   	return regs->ip + code_segment_base(regs);
>   }
>   
>   unsigned long perf_misc_flags(struct pt_regs *regs)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -	unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0;
> +	unsigned int guest_state = perf_guest_state();
>   	int misc = 0;
>   
>   	if (guest_state) {
> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> index 524ad1f747bd..f5b02017ba16 100644
> --- a/arch/x86/events/intel/core.c
> +++ b/arch/x86/events/intel/core.c
> @@ -2786,7 +2786,6 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
>   {
>   	struct perf_sample_data data;
>   	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
> -	struct perf_guest_info_callbacks *guest_cbs;
>   	int bit;
>   	int handled = 0;
>   	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
> @@ -2853,9 +2852,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
>   	 */
>   	if (__test_and_clear_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT, (unsigned long *)&status)) {
>   		handled++;
> -
> -		guest_cbs = perf_get_guest_cbs();
> -		if (likely(!guest_cbs || !guest_cbs->handle_intel_pt_intr()))
> +		if (!perf_guest_handle_intel_pt_intr())
>   			intel_pt_interrupt();
>   	}
>   
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index f9be88a47434..c0a6eaf55fb1 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -1247,6 +1247,30 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
>   	/* Prevent reloading between a !NULL check and dereferences. */
>   	return READ_ONCE(perf_guest_cbs);
>   }
> +static inline unsigned int perf_guest_state(void)
> +{
> +	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> +
> +	return guest_cbs ? guest_cbs->state() : 0;
> +}
> +static inline unsigned long perf_guest_get_ip(void)
> +{
> +	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> +
> +	/*
> +	 * Arbitrarily return '0' in the unlikely scenario that the callbacks
> +	 * are unregistered between checking guest state and getting the IP.
> +	 */
> +	return guest_cbs ? guest_cbs->get_ip() : 0;
> +}
> +static inline unsigned int perf_guest_handle_intel_pt_intr(void)
> +{
> +	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> +
> +	if (guest_cbs && guest_cbs->handle_intel_pt_intr)
> +		return guest_cbs->handle_intel_pt_intr();
> +	return 0;
> +}
>   extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
>   extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
>   
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:32:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:32:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192165.342465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvo2-0003bi-3I; Wed, 22 Sep 2021 06:32:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192165.342465; Wed, 22 Sep 2021 06:32:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvo1-0003bb-W7; Wed, 22 Sep 2021 06:32:33 +0000
Received: by outflank-mailman (input) for mailman id 192165;
 Wed, 22 Sep 2021 06:32:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvo0-0003bV-UT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:32:32 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 370fa1aa-b925-4b55-98a6-96c1c49e8ce6;
 Wed, 22 Sep 2021 06:32:32 +0000 (UTC)
Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com
 [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-502-KcZvdW2kPSmRlBrsfw-E6Q-1; Wed, 22 Sep 2021 02:32:30 -0400
Received: by mail-ed1-f69.google.com with SMTP id
 n5-20020a05640206c500b003cf53f7cef2so1805369edy.12
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:32:30 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id yd3sm552542ejb.6.2021.09.21.23.32.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:32:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 370fa1aa-b925-4b55-98a6-96c1c49e8ce6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292351;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=QQ8YzZrVP1svLk3AMq6cxZLez6XFnWefi9KQVDYCfgo=;
	b=Iq5ke2Pjw+g8VO0+JuQRlOxpJXRt92oFeMlAdJ+JzNyhvMG3i08Aa4UMMmvE/iKSjjP9g3
	YIYMuz62jbl4yKlL6lfjrOyQLNbXoYJQ5JYAFrWv+ImUoBurG3vMCgF9qAXtke6l1iW8Lt
	3Aie6U+FdOajeYLkbqZ5/i/oZfayGNU=
X-MC-Unique: KcZvdW2kPSmRlBrsfw-E6Q-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=QQ8YzZrVP1svLk3AMq6cxZLez6XFnWefi9KQVDYCfgo=;
        b=NkznUoi1qC7UTz4MR8z0J52gNtsimBLniEvYcRk5gwauQ5vYFmvz9ESACK3ADgOuTw
         qum5FoWNLOovn5zvjjINY1DzhYrOGjvgROiIIgWXlQWBKFbwWwbfh7zuJsCUeKXY+Yzf
         T8pUwDbTQjAspzkJavm4cmRfGWug2bJp8K5JhZrbzgCyiWJxraPmNEpm/iCvpIbf0RCH
         xCXxS6YC+1qu4aTuAZf5+fPQYeJ8+FLrL2shcZZSXI+pxsXKL5hNaczLQOw85Jkf0Mwr
         sch/JKBfuvjg3L3uKIJvUIHYSC4TNGiAlXiVBqX00VbztrV9gBenjZKKKzbX9FgOUZG/
         3l/g==
X-Gm-Message-State: AOAM531EW8xoAYj62gtpzCYHfpy9Q3a6dopOE4ghVdp/MBcC+0HdxoiJ
	FZ5pIwPFDVPLcbVRueG9cetR5iv7eFQ9iLXjCofaL+/PsyVQnUCCyvwkpcFeA5ie36yJfnReOMU
	GumAzscNMiH6+ds1ArvPaZ3Q2BSA=
X-Received: by 2002:a17:907:11c5:: with SMTP id va5mr38316003ejb.215.1632292349167;
        Tue, 21 Sep 2021 23:32:29 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJzYtbJQ8octsTI+p+irGuyf7bBUrwOfK9ScnlhWiK83zRobYLAMfdGjMGdI45oS8miuwF0xKA==
X-Received: by 2002:a17:907:11c5:: with SMTP id va5mr38315988ejb.215.1632292348952;
        Tue, 21 Sep 2021 23:32:28 -0700 (PDT)
Subject: Re: [PATCH v3 08/16] perf: Force architectures to opt-in to guest
 callbacks
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-9-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <f2ad98e2-ddfb-c688-65af-7ecbd8bc3b3d@redhat.com>
Date: Wed, 22 Sep 2021 08:32:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-9-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Introduce GUEST_PERF_EVENTS and require architectures to select it to
> allow registering and using guest callbacks in perf.  This will hopefully
> make it more difficult for new architectures to add useless "support" for
> guest callbacks, e.g. via copy+paste.
> 
> Stubbing out the helpers has the happy bonus of avoiding a load of
> perf_guest_cbs when GUEST_PERF_EVENTS=n on arm64/x86.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/arm64/kvm/Kconfig     | 1 +
>   arch/x86/kvm/Kconfig       | 1 +
>   arch/x86/xen/Kconfig       | 1 +
>   include/linux/perf_event.h | 6 ++++++
>   init/Kconfig               | 4 ++++
>   kernel/events/core.c       | 2 ++
>   6 files changed, 15 insertions(+)
> 
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index a4eba0908bfa..f2121404c7c6 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -37,6 +37,7 @@ menuconfig KVM
>   	select HAVE_KVM_IRQ_BYPASS
>   	select HAVE_KVM_VCPU_RUN_PID_CHANGE
>   	select SCHED_INFO
> +	select GUEST_PERF_EVENTS if PERF_EVENTS
>   	help
>   	  Support hosting virtualized guest machines.
>   
> diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
> index ac69894eab88..699bf786fbce 100644
> --- a/arch/x86/kvm/Kconfig
> +++ b/arch/x86/kvm/Kconfig
> @@ -36,6 +36,7 @@ config KVM
>   	select KVM_MMIO
>   	select SCHED_INFO
>   	select PERF_EVENTS
> +	select GUEST_PERF_EVENTS
>   	select HAVE_KVM_MSI
>   	select HAVE_KVM_CPU_RELAX_INTERCEPT
>   	select HAVE_KVM_NO_POLL
> diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
> index afc1da68b06d..d07595a9552d 100644
> --- a/arch/x86/xen/Kconfig
> +++ b/arch/x86/xen/Kconfig
> @@ -23,6 +23,7 @@ config XEN_PV
>   	select PARAVIRT_XXL
>   	select XEN_HAVE_PVMMU
>   	select XEN_HAVE_VPMU
> +	select GUEST_PERF_EVENTS
>   	help
>   	  Support running as a Xen PV guest.
>   
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index c0a6eaf55fb1..eefa197d5354 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -1238,6 +1238,7 @@ extern void perf_event_bpf_event(struct bpf_prog *prog,
>   				 enum perf_bpf_event_type type,
>   				 u16 flags);
>   
> +#ifdef CONFIG_GUEST_PERF_EVENTS
>   extern struct perf_guest_info_callbacks *perf_guest_cbs;
>   static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
>   {
> @@ -1273,6 +1274,11 @@ static inline unsigned int perf_guest_handle_intel_pt_intr(void)
>   }
>   extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
>   extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
> +#else
> +static inline unsigned int perf_guest_state(void)		 { return 0; }
> +static inline unsigned long perf_guest_get_ip(void)		 { return 0; }
> +static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }
> +#endif /* CONFIG_GUEST_PERF_EVENTS */

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Having perf_guest_handle_intel_pt_intr in generic code is a bit off.  Of 
course it has to be in the struct, but the wrapper might be placed in 
arch/x86/include/asm/perf_event.h as well (applies to patch 7 as well).

Paolo



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:33:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:33:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192171.342475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvon-0004BS-Ce; Wed, 22 Sep 2021 06:33:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192171.342475; Wed, 22 Sep 2021 06:33:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvon-0004BL-9h; Wed, 22 Sep 2021 06:33:21 +0000
Received: by outflank-mailman (input) for mailman id 192171;
 Wed, 22 Sep 2021 06:33:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvom-0004BF-Eo
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:33:20 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id f9e1c712-1b6e-11ec-b95f-12813bfff9fa;
 Wed, 22 Sep 2021 06:33:19 +0000 (UTC)
Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com
 [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-470-ZIKqZakFM_msiKV2G75Ncg-1; Wed, 22 Sep 2021 02:33:18 -0400
Received: by mail-ed1-f71.google.com with SMTP id
 w24-20020a056402071800b003cfc05329f8so1775141edx.19
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:33:17 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id n16sm628454edd.10.2021.09.21.23.33.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:33:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9e1c712-1b6e-11ec-b95f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292399;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hK1TAJDdleQGO4jZBAmuPgddEjoHfavTDxwu6JuYxPc=;
	b=JlDDnaak59jQNHs2dj27x7ZjvcngSDVkkv3ldNbEkXiD9kKGjTNWzgmF0fSw8cEu1Fp2po
	xswDQbrHpki0LeMJPtxu0kXg1pFWIa7ggHaudP+kRtf6TidkRg/5LohkkZRcqVtvCYQYqb
	7KDjJYrYXTwzICH/Cq4TJKcmEEN5qHo=
X-MC-Unique: ZIKqZakFM_msiKV2G75Ncg-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=hK1TAJDdleQGO4jZBAmuPgddEjoHfavTDxwu6JuYxPc=;
        b=S1BWsO0oVhybvL3rXKA10XfsrP+xy+lNHW5Ao8mLMaKcqq0a5okUHo4u05YGMVf3zI
         +A+7B7bbd3IZoJqYofuOTIyZlrAbVLRPx/L4nlrooTFf0rNzxSIXCHimKeqbIZIGtFj/
         fg+0iEi4LFeJmhsN2CjxYZnZcMJHsaihm2OtOuahkK3O3MeErMP9ZXd4Ckr5aI6EEkB7
         mrPtn+VoX9TuoT4SocnbaDHxJqJu0oCmAMSqm9FroZJV4PytTlYiXnOCs2Nt794SQ/rW
         bSgxVXryhLBPclOAJPkyazhCKl6h7djn+HzjbYmmSAjnVPCtDuF8KRRac1ETM0AFp4yr
         W1uQ==
X-Gm-Message-State: AOAM533Rmcib0KpWJNrg/Xb69IzVlS+w8ObUOE42JW+wpUPSF4amz6jz
	UjIALQVmtq12KiKORZo6EDekulzthtXuzZ/PokRyJeZEbVDFsOmJruauLQKJ1avJLstCqcX1NeG
	sVeQsoriOEuBUks3ORAoxiY81/kw=
X-Received: by 2002:a17:906:a01:: with SMTP id w1mr40417060ejf.117.1632292396944;
        Tue, 21 Sep 2021 23:33:16 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJwWUo+BlVIV+JEHRlDKOT5sLvq+4KeWDB237/AGh6qJRsKlnsARTagHguA0XIMd4G8mEtTIGA==
X-Received: by 2002:a17:906:a01:: with SMTP id w1mr40417041ejf.117.1632292396704;
        Tue, 21 Sep 2021 23:33:16 -0700 (PDT)
Subject: Re: [PATCH v3 09/16] perf/core: Use static_call to optimize
 perf_guest_info_callbacks
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-10-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <5ad3e3f9-260b-52b2-e0c8-9ab824e08fb4@redhat.com>
Date: Wed, 22 Sep 2021 08:33:14 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-10-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Use static_call to optimize perf's guest callbacks on arm64 and x86,
> which are now the only architectures that define the callbacks.  Use
> DEFINE_STATIC_CALL_RET0 as the default/NULL for all guest callbacks, as
> the callback semantics are that a return value '0' means "not in guest".
> 
> static_call obviously avoids the overhead of CONFIG_RETPOLINE=y, but is
> also advantageous versus other solutions, e.g. per-cpu callbacks, in that
> a per-cpu memory load is not needed to detect the !guest case.
> 
> Based on code from Peter and Like.
> 
> Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Cc: Like Xu <like.xu.linux@gmail.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   include/linux/perf_event.h | 28 ++++++----------------------
>   kernel/events/core.c       | 15 +++++++++++++++
>   2 files changed, 21 insertions(+), 22 deletions(-)
> 
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index eefa197d5354..d582dfeb4e20 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -1240,37 +1240,21 @@ extern void perf_event_bpf_event(struct bpf_prog *prog,
>   
>   #ifdef CONFIG_GUEST_PERF_EVENTS
>   extern struct perf_guest_info_callbacks *perf_guest_cbs;
> -static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void)
> -{
> -	/* Reg/unreg perf_guest_cbs waits for readers via synchronize_rcu(). */
> -	lockdep_assert_preemption_disabled();
> +DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs->state);
> +DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
> +DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
>   
> -	/* Prevent reloading between a !NULL check and dereferences. */
> -	return READ_ONCE(perf_guest_cbs);
> -}
>   static inline unsigned int perf_guest_state(void)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	return guest_cbs ? guest_cbs->state() : 0;
> +	return static_call(__perf_guest_state)();
>   }
>   static inline unsigned long perf_guest_get_ip(void)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	/*
> -	 * Arbitrarily return '0' in the unlikely scenario that the callbacks
> -	 * are unregistered between checking guest state and getting the IP.
> -	 */
> -	return guest_cbs ? guest_cbs->get_ip() : 0;
> +	return static_call(__perf_guest_get_ip)();
>   }
>   static inline unsigned int perf_guest_handle_intel_pt_intr(void)
>   {
> -	struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs();
> -
> -	if (guest_cbs && guest_cbs->handle_intel_pt_intr)
> -		return guest_cbs->handle_intel_pt_intr();
> -	return 0;
> +	return static_call(__perf_guest_handle_intel_pt_intr)();
>   }
>   extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
>   extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index c6ec05809f54..79c8ee1778a4 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -6485,12 +6485,23 @@ static void perf_pending_event(struct irq_work *entry)
>   #ifdef CONFIG_GUEST_PERF_EVENTS
>   struct perf_guest_info_callbacks *perf_guest_cbs;
>   
> +DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs->state);
> +DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
> +DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
> +
>   void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
>   {
>   	if (WARN_ON_ONCE(perf_guest_cbs))
>   		return;
>   
>   	WRITE_ONCE(perf_guest_cbs, cbs);
> +	static_call_update(__perf_guest_state, cbs->state);
> +	static_call_update(__perf_guest_get_ip, cbs->get_ip);
> +
> +	/* Implementing ->handle_intel_pt_intr is optional. */
> +	if (cbs->handle_intel_pt_intr)
> +		static_call_update(__perf_guest_handle_intel_pt_intr,
> +				   cbs->handle_intel_pt_intr);
>   }
>   EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
>   
> @@ -6500,6 +6511,10 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
>   		return;
>   
>   	WRITE_ONCE(perf_guest_cbs, NULL);
> +	static_call_update(__perf_guest_state, (void *)&__static_call_return0);
> +	static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0);
> +	static_call_update(__perf_guest_handle_intel_pt_intr,
> +			   (void *)&__static_call_return0);
>   	synchronize_rcu();
>   }
>   EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:38:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:38:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192177.342487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvtg-00051C-0I; Wed, 22 Sep 2021 06:38:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192177.342487; Wed, 22 Sep 2021 06:38:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvtf-000515-Sg; Wed, 22 Sep 2021 06:38:23 +0000
Received: by outflank-mailman (input) for mailman id 192177;
 Wed, 22 Sep 2021 06:38:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvte-00050z-Vv
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:38:23 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id a5342723-14c2-4c12-8492-5a7f9a9252e9;
 Wed, 22 Sep 2021 06:38:22 +0000 (UTC)
Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com
 [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-514-6IqISAJgO7GWjjpFfJ6Ikw-1; Wed, 22 Sep 2021 02:38:18 -0400
Received: by mail-wr1-f69.google.com with SMTP id
 v1-20020adfc401000000b0015e11f71e65so1187017wrf.2
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:38:18 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id j21sm1162153wrd.48.2021.09.21.23.38.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:38:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5342723-14c2-4c12-8492-5a7f9a9252e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292702;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z3XTCa1nu8hCtlYTFFK2GZognBOtLDAkX/kYxcYF1XA=;
	b=NUU64Xb1BcgagGpy9iPM2unKlGXmCCkRfB69KsWkLaF1CANMexfuGIRg2veNeBpU0TlP7C
	SMmoE3GhDeOg+/rY3Jky4LurhV/dNIJD+g2nyanA45uYOJKrgQ5LdF9C5yl9nWGruU4Fm5
	/Fx0zIef1Gp4HywOyr78b6+Q+MzXQk8=
X-MC-Unique: 6IqISAJgO7GWjjpFfJ6Ikw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=Z3XTCa1nu8hCtlYTFFK2GZognBOtLDAkX/kYxcYF1XA=;
        b=4INTo7wWrptTvwYh9dz3Kmz3h2QSViGbRU88JJeOJXjRc8Wle8Oy7cWZHXRohn2zOw
         +G+3euPoKIwPlBv84ifRtf1Db+TmZWC56iycdcDfJ/epqEACtkhPwTmkJ7K80LeBAWr2
         TjPYiIVBPEKyns9Y0S5jun7+NNRLulZswgLF0tp0sosv6pO4pwizaVoodae+goBgLuzJ
         /g4cILR2Bnxr0DmNGCK0F82wQH7Ld4aNmyv0HgR8qqKLPp0xILJLRaixVb4NbiywGxrv
         ufHgX5NkAY3uWu8rzNFZN41SAJ61/vmh+45OV5xKkEsAUWhLJizZ+qCJuN7tWGNE3lZP
         atrA==
X-Gm-Message-State: AOAM531F869Q6r0P/79Z6py+G5hXKJpNPN5A6PZG4HVnY40ZcfSaEOkS
	27pq1tYB2LupkLjkIfvDJTbS/uAHb69kiRz15er0tM2juGvM6nwxDSwXIIAJNPcWjrR4QuNKfaO
	Gw+e25Pn4g/MMTNfrFh+AKFRbrz8=
X-Received: by 2002:a05:600c:21c8:: with SMTP id x8mr8614006wmj.163.1632292697323;
        Tue, 21 Sep 2021 23:38:17 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJywCmZIrNrFYOLYwz4uLurAI8KkuEnxjxQSJf2bjooMgzy2h0qoOTiWQbU4UgbH2lcfc2iQrA==
X-Received: by 2002:a05:600c:21c8:: with SMTP id x8mr8613976wmj.163.1632292697089;
        Tue, 21 Sep 2021 23:38:17 -0700 (PDT)
Subject: Re: [PATCH v3 11/16] KVM: x86: More precisely identify NMI from guest
 when handling PMI
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-12-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <d444303f-7ee5-7ddc-8eae-64d1e6d4862a@redhat.com>
Date: Wed, 22 Sep 2021 08:38:14 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-12-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Note, this also doesn't completely prevent false positives if perf
> somehow ends up calling into KVM, e.g. an NMI can arrive in host after
> KVM sets its flag.

s/calling into KVM/being called from KVM/ ?

Not sure about what you meant though.  The code is nice, so

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Paolo



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:40:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:40:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192183.342498 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvw8-0006NV-Dt; Wed, 22 Sep 2021 06:40:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192183.342498; Wed, 22 Sep 2021 06:40:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvw8-0006NO-AY; Wed, 22 Sep 2021 06:40:56 +0000
Received: by outflank-mailman (input) for mailman id 192183;
 Wed, 22 Sep 2021 06:40:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvw7-0006NG-Fi
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:40:55 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id b8323c0e-2d15-449f-af15-5bc4b94fee1a;
 Wed, 22 Sep 2021 06:40:54 +0000 (UTC)
Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com
 [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-577-G6Wm7QKeP7SXbbA0fC1OxQ-1; Wed, 22 Sep 2021 02:40:53 -0400
Received: by mail-ed1-f70.google.com with SMTP id
 b7-20020a50e787000000b003d59cb1a923so1864567edn.5
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:40:53 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id t24sm618083edr.84.2021.09.21.23.40.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:40:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8323c0e-2d15-449f-af15-5bc4b94fee1a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292854;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Y4eS+67AEBYvtqXGAO5BJQzp3vg53OlrVoOKQgtt1lA=;
	b=R0CpLYBVOqp9E3R8+l6iSCCxDo4BW/+rizeBoeRtEpcrnR11/cJBADJq8DoPydUeQ+pbQ1
	3yklShXi9q18WULTdOhFHhA1f0CTsYi5egN7+Q8Ookasv91q+BkzEAlacq2QWE95TLiNwv
	GSRkFc/WHCAfV1YDIhDUodFVAAPaol4=
X-MC-Unique: G6Wm7QKeP7SXbbA0fC1OxQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=Y4eS+67AEBYvtqXGAO5BJQzp3vg53OlrVoOKQgtt1lA=;
        b=OnNhR4B0UexRYXyLHjJny37bMSwYNlNUKEwFZuCewgj2skWPKrjrVFzll08CjKYDwM
         xm7wWgRAN+clhK7/RpiAvWqGJkjWtVKgXAJLCgL4zEYdUdBPaXYHjLJP2KKyRIrEn4/X
         oXdsgOWMyF7o8HBqMbTHsrmPNC+GS7zJ1u6+srPTko89/4x+d3hF3rQVBQtq4X/lwaSL
         zrZNmNDIOMGOfTpVDdxNViPQ2Nwg9zKgyfnrbz2QlCUThwmbctbD3+xBScfSKoqtUF6V
         75eMkF0eNpfC4MCQeUOJ+GbNfGknBEnlyTugBfzdzGBWnFwsqvgDLUkNtD94WzSwbMn3
         71Tg==
X-Gm-Message-State: AOAM530BhawcYDfQx3zQ+EokwxmMuAIM6wgGyIRlzx1ctIfSAeVr4UKV
	K3FCveqp4OzbKcCud68tTia5Bhzu48R5f4tqb6m8tDYS+tOtnAea0rgrlLYrMNZnKXytQaNgKji
	yTgtNCu9L0iiC7Q/HGcnCD+kfrL8=
X-Received: by 2002:a50:d84c:: with SMTP id v12mr39445975edj.203.1632292851015;
        Tue, 21 Sep 2021 23:40:51 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxlo4LRB2M8qmmWv+FM3u6P6lvmJINGGsRc/N/kQ7rKED6NNzY0+dunHYjRokMbMUtgN8H1BQ==
X-Received: by 2002:a50:d84c:: with SMTP id v12mr39445930edj.203.1632292850792;
        Tue, 21 Sep 2021 23:40:50 -0700 (PDT)
Subject: Re: [PATCH v3 10/16] KVM: x86: Drop current_vcpu for kvm_running_vcpu
 + kvm_arch_vcpu variable
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-11-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <e5948ba8-3f4d-e749-e645-b9b82f405863@redhat.com>
Date: Wed, 22 Sep 2021 08:40:47 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-11-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Use the generic kvm_running_vcpu plus a new 'handling_intr_from_guest'
> variable in kvm_arch_vcpu instead of the semi-redundant current_vcpu.
> kvm_before/after_interrupt() must be called while the vCPU is loaded,
> (which protects against preemption), thus kvm_running_vcpu is guaranteed
> to be non-NULL when handling_intr_from_guest is non-zero.
> 
> Switching to kvm_get_running_vcpu() will allows moving KVM's perf
> callbacks to generic code, and the new flag will be used in a future
> patch to more precisely identify the "NMI from guest" case.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/x86/include/asm/kvm_host.h |  3 +--
>   arch/x86/kvm/pmu.c              |  2 +-
>   arch/x86/kvm/x86.c              | 21 ++++++++++++---------
>   arch/x86/kvm/x86.h              | 10 ++++++----
>   4 files changed, 20 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 1080166fc0cf..2d86a2dfc775 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -763,6 +763,7 @@ struct kvm_vcpu_arch {
>   	unsigned nmi_pending; /* NMI queued after currently running handler */
>   	bool nmi_injected;    /* Trying to inject an NMI this entry */
>   	bool smi_pending;    /* SMI queued after currently running handler */
> +	u8 handling_intr_from_guest;
>   
>   	struct kvm_mtrr mtrr_state;
>   	u64 pat;
> @@ -1874,8 +1875,6 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu);
>   int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err);
>   void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu);
>   
> -unsigned int kvm_guest_state(void);
> -
>   void __user *__x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa,
>   				     u32 size);
>   bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu);
> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> index 5b68d4188de0..eef48258e50f 100644
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -87,7 +87,7 @@ static void kvm_perf_overflow_intr(struct perf_event *perf_event,
>   		 * woken up. So we should wake it, but this is impossible from
>   		 * NMI context. Do it from irq work instead.
>   		 */
> -		if (!kvm_guest_state())
> +		if (!kvm_handling_nmi_from_guest(pmc->vcpu))
>   			irq_work_queue(&pmc_to_pmu(pmc)->irq_work);
>   		else
>   			kvm_make_request(KVM_REQ_PMI, pmc->vcpu);
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 6cc66466f301..24a6faa07442 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8264,15 +8264,17 @@ static void kvm_timer_init(void)
>   			  kvmclock_cpu_online, kvmclock_cpu_down_prep);
>   }
>   
> -DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu);
> -EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu);
> +static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu)
> +{
> +	return vcpu && vcpu->arch.handling_intr_from_guest;
> +}
>   
> -unsigned int kvm_guest_state(void)
> +static unsigned int kvm_guest_state(void)
>   {
> -	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
> +	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
>   	unsigned int state;
>   
> -	if (!vcpu)
> +	if (!kvm_pmi_in_guest(vcpu))
>   		return 0;
>   
>   	state = PERF_GUEST_ACTIVE;
> @@ -8284,9 +8286,10 @@ unsigned int kvm_guest_state(void)
>   
>   static unsigned long kvm_guest_get_ip(void)
>   {
> -	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
> +	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
>   
> -	if (WARN_ON_ONCE(!vcpu))
> +	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
> +	if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu)))
>   		return 0;
>   
>   	return kvm_rip_read(vcpu);
> @@ -8294,10 +8297,10 @@ static unsigned long kvm_guest_get_ip(void)
>   
>   static unsigned int kvm_handle_intel_pt_intr(void)
>   {
> -	struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu);
> +	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
>   
>   	/* '0' on failure so that the !PT case can use a RET0 static call. */
> -	if (!vcpu)
> +	if (!kvm_pmi_in_guest(vcpu))
>   		return 0;
>   
>   	kvm_make_request(KVM_REQ_PMI, vcpu);
> diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
> index 7d66d63dc55a..a9c107e7c907 100644
> --- a/arch/x86/kvm/x86.h
> +++ b/arch/x86/kvm/x86.h
> @@ -387,18 +387,20 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm)
>   	return kvm->arch.cstate_in_guest;
>   }
>   
> -DECLARE_PER_CPU(struct kvm_vcpu *, current_vcpu);
> -
>   static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu)
>   {
> -	__this_cpu_write(current_vcpu, vcpu);
> +	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 1);
>   }
>   
>   static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu)
>   {
> -	__this_cpu_write(current_vcpu, NULL);
> +	WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 0);
>   }
>   
> +static inline bool kvm_handling_nmi_from_guest(struct kvm_vcpu *vcpu)
> +{
> +	return !!vcpu->arch.handling_intr_from_guest;
> +}
>   
>   static inline bool kvm_pat_valid(u64 data)
>   {
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:41:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:41:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192191.342509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvwx-00070u-Sp; Wed, 22 Sep 2021 06:41:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192191.342509; Wed, 22 Sep 2021 06:41:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvwx-00070n-OY; Wed, 22 Sep 2021 06:41:47 +0000
Received: by outflank-mailman (input) for mailman id 192191;
 Wed, 22 Sep 2021 06:41:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvww-00070h-12
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:41:46 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 27276565-1b70-11ec-b960-12813bfff9fa;
 Wed, 22 Sep 2021 06:41:44 +0000 (UTC)
Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com
 [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-454-zt2zgcZnOUOewkVM_e0I2A-1; Wed, 22 Sep 2021 02:41:43 -0400
Received: by mail-ed1-f70.google.com with SMTP id
 d1-20020a50f681000000b003d860fcf4ffso1774534edn.22
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:41:43 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id kt19sm557582ejb.26.2021.09.21.23.41.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:41:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27276565-1b70-11ec-b960-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292904;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kLkDWA7zis/qBBg4IjuyViLi2gFI3fX/kzAaOlA6siY=;
	b=SBatF0FhE7qH3IrRWA/2sT5fvGVmBPKG+UK7ymZD0S35Ui8rMFwnk/+FbT8i5uc0WY+uS9
	aMThoczg7jGDxbHayNZ2OM8cEXztYyBUsSPYB5bb3HQjkEkDxGt7YWMa6fVo+A8lIW3l4w
	EkN766d0UTw1MXUIFNHjv7c/UBhSjDg=
X-MC-Unique: zt2zgcZnOUOewkVM_e0I2A-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=kLkDWA7zis/qBBg4IjuyViLi2gFI3fX/kzAaOlA6siY=;
        b=cXDHPTrVZRVdDlNaKbvTuWmNK8H6lLksg24XSwvAYFx20sLZCCP/CkBokwClfXiXvB
         SaIS25P3zSXTRrDhmHzC8RJ3/TnjvilL4W/LVwmUEf7DUF4wJbyN44zgSI2AiuNN54LR
         aRe4MwR9LFG3e1zNenrLypjaeqnwTeIuv3zTwZAV1Twx4XLWIS8RKLJE5ZFjRGdJos16
         frp8X4i/Gx2L3HveVn6qobD0X848HntzEErS0IdsrEjWSbaVtfFXzbjsPcrazDBhfaw5
         moJUDB2Mmkbe0MwlkMmqPjRcdT0M2Yrls5SLg9cw8z+VbLXaQ4KGOPs6hizPnUwzszHz
         00cg==
X-Gm-Message-State: AOAM530Bw0wWZlHfN/rbMe964S4ft90YadmmbOIEj8CmUOLfs0aGCciT
	jn1tqO8xAJP/YtBm4LYFlOclP2A0sVV1sUk3dl5mR66N7yfYQQ1/BkF87nYibBwUci0rOzdiLLM
	VLiXhprF9ptXYEkUcPZeBYPuusRw=
X-Received: by 2002:a05:6402:5186:: with SMTP id q6mr40893923edd.64.1632292902402;
        Tue, 21 Sep 2021 23:41:42 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJwP+MY9+n/one3D2apviNbkQobxFZdru25IJ6syxfQ+H3Moyz3xvNvSjJ+iSQi50cWPcONDFw==
X-Received: by 2002:a05:6402:5186:: with SMTP id q6mr40893882edd.64.1632292902156;
        Tue, 21 Sep 2021 23:41:42 -0700 (PDT)
Subject: Re: [PATCH v3 12/16] KVM: Move x86's perf guest info callbacks to
 generic KVM
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-13-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <50cda5e5-be7e-00f3-886b-21b0fdfd1236@redhat.com>
Date: Wed, 22 Sep 2021 08:41:30 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-13-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Move x86's perf guest callbacks into common KVM, as they are semantically
> identical to arm64's callbacks (the only other such KVM callbacks).
> arm64 will convert to the common versions in a future patch.
> 
> Implement the necessary arm64 arch hooks now to avoid having to provide
> stubs or a temporary #define (from x86) to avoid arm64 compilation errors
> when CONFIG_GUEST_PERF_EVENTS=y.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/arm64/include/asm/kvm_host.h |  8 +++++
>   arch/arm64/kvm/arm.c              |  5 +++
>   arch/x86/include/asm/kvm_host.h   |  3 ++
>   arch/x86/kvm/x86.c                | 53 +++++++------------------------
>   include/linux/kvm_host.h          | 10 ++++++
>   virt/kvm/kvm_main.c               | 44 +++++++++++++++++++++++++
>   6 files changed, 81 insertions(+), 42 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index ed940aec89e0..828b6eaa2c56 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -673,6 +673,14 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa);
>   void kvm_perf_init(void);
>   void kvm_perf_teardown(void);
>   
> +#ifdef CONFIG_GUEST_PERF_EVENTS
> +static inline bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu)
> +{
> +	/* Any callback while a vCPU is loaded is considered to be in guest. */
> +	return !!vcpu;
> +}
> +#endif
> +
>   long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu);
>   gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
>   void kvm_update_stolen_time(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index e9a2b8f27792..2b542fdc237e 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -500,6 +500,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
>   	return vcpu_mode_priv(vcpu);
>   }
>   
> +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu)
> +{
> +	return *vcpu_pc(vcpu);
> +}
> +
>   /* Just ensure a guest exit from a particular CPU */
>   static void exit_vm_noop(void *info)
>   {
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 2d86a2dfc775..6efe4e03a6d2 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1543,6 +1543,9 @@ static inline int kvm_arch_flush_remote_tlb(struct kvm *kvm)
>   		return -ENOTSUPP;
>   }
>   
> +#define kvm_arch_pmi_in_guest(vcpu) \
> +	((vcpu) && (vcpu)->arch.handling_intr_from_guest)
> +
>   int kvm_mmu_module_init(void);
>   void kvm_mmu_module_exit(void);
>   
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 412646b973bb..1bea616402e6 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8264,43 +8264,12 @@ static void kvm_timer_init(void)
>   			  kvmclock_cpu_online, kvmclock_cpu_down_prep);
>   }
>   
> -static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu)
> -{
> -	return vcpu && vcpu->arch.handling_intr_from_guest;
> -}
> -
> -static unsigned int kvm_guest_state(void)
> -{
> -	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
> -	unsigned int state;
> -
> -	if (!kvm_pmi_in_guest(vcpu))
> -		return 0;
> -
> -	state = PERF_GUEST_ACTIVE;
> -	if (static_call(kvm_x86_get_cpl)(vcpu))
> -		state |= PERF_GUEST_USER;
> -
> -	return state;
> -}
> -
> -static unsigned long kvm_guest_get_ip(void)
> -{
> -	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
> -
> -	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
> -	if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu)))
> -		return 0;
> -
> -	return kvm_rip_read(vcpu);
> -}
> -
>   static unsigned int kvm_handle_intel_pt_intr(void)
>   {
>   	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
>   
>   	/* '0' on failure so that the !PT case can use a RET0 static call. */
> -	if (!kvm_pmi_in_guest(vcpu))
> +	if (!kvm_arch_pmi_in_guest(vcpu))
>   		return 0;
>   
>   	kvm_make_request(KVM_REQ_PMI, vcpu);
> @@ -8309,12 +8278,6 @@ static unsigned int kvm_handle_intel_pt_intr(void)
>   	return 1;
>   }
>   
> -static struct perf_guest_info_callbacks kvm_guest_cbs = {
> -	.state			= kvm_guest_state,
> -	.get_ip			= kvm_guest_get_ip,
> -	.handle_intel_pt_intr	= NULL,
> -};
> -
>   #ifdef CONFIG_X86_64
>   static void pvclock_gtod_update_fn(struct work_struct *work)
>   {
> @@ -11068,9 +11031,11 @@ int kvm_arch_hardware_setup(void *opaque)
>   	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
>   	kvm_ops_static_call_update();
>   
> +	/* Temporary ugliness. */
>   	if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest())
> -		kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr;
> -	perf_register_guest_info_callbacks(&kvm_guest_cbs);
> +		kvm_register_perf_callbacks(kvm_handle_intel_pt_intr);
> +	else
> +		kvm_register_perf_callbacks(NULL);
>   
>   	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
>   		supported_xss = 0;
> @@ -11099,8 +11064,7 @@ int kvm_arch_hardware_setup(void *opaque)
>   
>   void kvm_arch_hardware_unsetup(void)
>   {
> -	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
> -	kvm_guest_cbs.handle_intel_pt_intr = NULL;
> +	kvm_unregister_perf_callbacks();
>   
>   	static_call(kvm_x86_hardware_unsetup)();
>   }
> @@ -11727,6 +11691,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
>   	return vcpu->arch.preempted_in_kernel;
>   }
>   
> +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu)
> +{
> +	return kvm_rip_read(vcpu);
> +}
> +
>   int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>   {
>   	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index e4d712e9f760..b9255a6439f2 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -1163,6 +1163,16 @@ static inline bool kvm_arch_intc_initialized(struct kvm *kvm)
>   }
>   #endif
>   
> +#ifdef CONFIG_GUEST_PERF_EVENTS
> +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu);
> +
> +void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void));
> +void kvm_unregister_perf_callbacks(void);
> +#else
> +static inline void kvm_register_perf_callbacks(void *ign) {}
> +static inline void kvm_unregister_perf_callbacks(void) {}
> +#endif /* CONFIG_GUEST_PERF_EVENTS */
> +
>   int kvm_arch_init_vm(struct kvm *kvm, unsigned long type);
>   void kvm_arch_destroy_vm(struct kvm *kvm);
>   void kvm_arch_sync_events(struct kvm *kvm);
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 3e67c93ca403..179fb110a00f 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -5460,6 +5460,50 @@ struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void)
>           return &kvm_running_vcpu;
>   }
>   
> +#ifdef CONFIG_GUEST_PERF_EVENTS
> +static unsigned int kvm_guest_state(void)
> +{
> +	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
> +	unsigned int state;
> +
> +	if (!kvm_arch_pmi_in_guest(vcpu))
> +		return 0;
> +
> +	state = PERF_GUEST_ACTIVE;
> +	if (!kvm_arch_vcpu_in_kernel(vcpu))
> +		state |= PERF_GUEST_USER;
> +
> +	return state;
> +}
> +
> +static unsigned long kvm_guest_get_ip(void)
> +{
> +	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
> +
> +	/* Retrieving the IP must be guarded by a call to kvm_guest_state(). */
> +	if (WARN_ON_ONCE(!kvm_arch_pmi_in_guest(vcpu)))
> +		return 0;
> +
> +	return kvm_arch_vcpu_get_ip(vcpu);
> +}
> +
> +static struct perf_guest_info_callbacks kvm_guest_cbs = {
> +	.state			= kvm_guest_state,
> +	.get_ip			= kvm_guest_get_ip,
> +	.handle_intel_pt_intr	= NULL,
> +};
> +
> +void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void))
> +{
> +	kvm_guest_cbs.handle_intel_pt_intr = pt_intr_handler;
> +	perf_register_guest_info_callbacks(&kvm_guest_cbs);
> +}
> +void kvm_unregister_perf_callbacks(void)
> +{
> +	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
> +}
> +#endif
> +
>   struct kvm_cpu_compat_check {
>   	void *opaque;
>   	int *ret;
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 06:42:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 06:42:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192197.342519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvxn-0007bq-5Q; Wed, 22 Sep 2021 06:42:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192197.342519; Wed, 22 Sep 2021 06:42:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSvxn-0007bj-2P; Wed, 22 Sep 2021 06:42:39 +0000
Received: by outflank-mailman (input) for mailman id 192197;
 Wed, 22 Sep 2021 06:42:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ybNQ=OM=redhat.com=pbonzini@srs-us1.protection.inumbo.net>)
 id 1mSvxl-0007bW-HM
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 06:42:37 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 45f3b2a4-1b70-11ec-b960-12813bfff9fa;
 Wed, 22 Sep 2021 06:42:36 +0000 (UTC)
Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com
 [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-397-87lOpWjOOziKNF9tF7pvrQ-1; Wed, 22 Sep 2021 02:42:35 -0400
Received: by mail-ed1-f69.google.com with SMTP id
 e7-20020aa7d7c7000000b003d3e6df477cso1787985eds.20
 for <xen-devel@lists.xenproject.org>; Tue, 21 Sep 2021 23:42:35 -0700 (PDT)
Received: from ?IPv6:2001:b07:6468:f312:c8dd:75d4:99ab:290a?
 ([2001:b07:6468:f312:c8dd:75d4:99ab:290a])
 by smtp.gmail.com with ESMTPSA id dh16sm622370edb.63.2021.09.21.23.42.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 21 Sep 2021 23:42:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45f3b2a4-1b70-11ec-b960-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632292956;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WYW/P0Q1oqTDlsP5noOn+TZGFQp14YjumKQL9bn9bmM=;
	b=OwDlj4XQQpZT4OsqQaH31/flFA59cMPL27LqTTDDH7+e3S6KXkHQkCB1/NpJTOXokjOvJc
	kcWkh4xmATgF6Hm6EAO9v7SyeI8C/fzLdH8EywdpI339GOnUzdJ9cNVw+Phso+FKXikfnr
	Yuq8eq/NQ7Yxhdn1JzoPP1zcqQxlt6k=
X-MC-Unique: 87lOpWjOOziKNF9tF7pvrQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=WYW/P0Q1oqTDlsP5noOn+TZGFQp14YjumKQL9bn9bmM=;
        b=MiLElagnVKy8noiOd1hhDjsZfQ+Fu0Mgyxz6nGat66xJr3Ra2saIpFLdbCUxUch3b9
         jzffQw0vBJOMFSDqbWD9LtLTJGzR+9778G2hJ6VPZFePoSK+vECGOhfdugEC7fQO/41o
         mR9HsfgL3EUHpPetUPHlsZ0yV+pMsHbgEOhNBastpSGgegApuq8IlP3A/WbzfIsyAtXh
         KX1BI/33k+Is0p5RxRJDOxXEYxoYZ5GcduBFtLreu1UeGm58HwNqMaVjao32UZxDp2T7
         vDZ5YfedE4h7ntXkj2ZQwRA9QaDBMbDh/Vmi53qpVD4SpXDQUgCXVOv/q9nXaYXb5Evx
         CeWg==
X-Gm-Message-State: AOAM533bzuNf5F4mj46Oy5AddUTlHGqBoo5tx29SxI/G1wc5zfdZLdir
	C9+zlysae/hxPfFtkc9Vtj90jwAu3bCl6HSSjWPUyS4jHQ9rLaU3UtyeEFFDkCLSLocJ4uhB/S1
	7AK2uC/s9XbREbG/uvT6IGIQYqw0=
X-Received: by 2002:a17:906:369a:: with SMTP id a26mr39372081ejc.539.1632292954038;
        Tue, 21 Sep 2021 23:42:34 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxkdDvJZEuqy4z2jeakskcrSYfL6dBq85KTjMRgyfnCoMn+hPyF2F91qCQRXVCbzc6p2Cz+kg==
X-Received: by 2002:a17:906:369a:: with SMTP id a26mr39372025ejc.539.1632292953733;
        Tue, 21 Sep 2021 23:42:33 -0700 (PDT)
Subject: Re: [PATCH v3 00/16] perf: KVM: Fix, optimize, and clean up callbacks
To: Sean Christopherson <seanjc@google.com>,
 Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>, Will Deacon <will@kernel.org>,
 Mark Rutland <mark.rutland@arm.com>, Marc Zyngier <maz@kernel.org>,
 Guo Ren <guoren@kernel.org>, Nick Hu <nickhu@andestech.com>,
 Greentime Hu <green.hu@gmail.com>, Vincent Chen <deanbo422@gmail.com>,
 Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt
 <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
 James Morse <james.morse@arm.com>,
 Alexandru Elisei <alexandru.elisei@arm.com>,
 Suzuki K Poulose <suzuki.poulose@arm.com>,
 Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>,
 Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
 linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
 linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
 kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
 Artem Kashkanov <artem.kashkanov@intel.com>,
 Like Xu <like.xu.linux@gmail.com>, Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <edecd594-fb34-f4c9-964d-75ae16eadff6@redhat.com>
Date: Wed, 22 Sep 2021 08:42:31 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922000533.713300-1-seanjc@google.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22/09/21 02:05, Sean Christopherson wrote:
> Peter, I left the Intel PT mess as-is.  Having to pass a NULL pointer
> from KVM arm64 seemed to be a lesser evil than more exports and multiple
> registration paths.
> 
> This is a combination of ~2 series to fix bugs in the perf+KVM callbacks,
> optimize the callbacks by employing static_call, and do a variety of
> cleanup in both perf and KVM.
> 
> Patch 1 fixes a mostly-theoretical bug where perf can deref a NULL
> pointer if KVM unregisters its callbacks while they're being accessed.
> In practice, compilers tend to avoid problematic reloads of the pointer
> and the PMI handler doesn't lose the race against module unloading,
> i.e doesn't hit a use-after-free.
> 
> Patches 2 and 3 fix an Intel PT handling bug where KVM incorrectly
> eats PT interrupts when PT is supposed to be owned entirely by the host.
> 
> Patches 4-9 clean up perf's callback infrastructure and switch to
> static_call for arm64 and x86 (the only survivors).
> 
> Patches 10-16 clean up related KVM code and unify the arm64/x86 callbacks.
> 
> Based on "git://git.kernel.org/pub/scm/virt/kvm/kvm.git queue", commit
> 680c7e3be6a3 ("KVM: x86: Exit to userspace ...").

Looks nice apart from a couple nits, I will gladly accept a topic branch 
with both the perf and the KVM parts.

Thanks,

Paolo

> v3:
>    - Add wrappers for guest callbacks to that stubs can be provided when
>      GUEST_PERF_EVENTS=n.
>    - s/HAVE_GUEST_PERF_EVENTS/GUEST_PERF_EVENTS and select it from KVM
>      and XEN_PV instead of from top-level arm64/x86. [Paolo]
>    - Drop an unnecessary synchronize_rcu() when registering callbacks. [Peter]
>    - Retain a WARN_ON_ONCE() when unregistering callbacks if the caller
>      didn't provide the correct pointer. [Peter]
>    - Rework the static_call patch to move it all to common perf.
>    - Add a patch to drop the (un)register stubs, made possible after
>      having KVM+XEN_PV select GUEST_PERF_EVENTS.
>    - Split dropping guest callback "support" for arm, csky, etc... to a
>      separate patch, to make introducing GUEST_PERF_EVENTS cleaner.
>    
> v2 (relative to static_call v10):
>    - Split the patch into the semantic change (multiplexed ->state) and
>      introduction of static_call.
>    - Don't use '0' for "not a guest RIP".
>    - Handle unregister path.
>    - Drop changes for architectures that can be culled entirely.
> 
> v2 (relative to v1):
>    - https://lkml.kernel.org/r/20210828003558.713983-6-seanjc@google.com
>    - Drop per-cpu approach. [Peter]
>    - Fix mostly-theoretical reload and use-after-free with READ_ONCE(),
>      WRITE_ONCE(), and synchronize_rcu(). [Peter]
>    - Avoid new exports like the plague. [Peter]
> 
> v1:
>    - https://lkml.kernel.org/r/20210827005718.585190-1-seanjc@google.com
> 
> v10 static_call:
>    - https://lkml.kernel.org/r/20210806133802.3528-2-lingshan.zhu@intel.com
> 
> 
> Like Xu (1):
>    perf/core: Rework guest callbacks to prepare for static_call support
> 
> Sean Christopherson (15):
>    perf: Ensure perf_guest_cbs aren't reloaded between !NULL check and
>      deref
>    KVM: x86: Register perf callbacks after calling vendor's
>      hardware_setup()
>    KVM: x86: Register Processor Trace interrupt hook iff PT enabled in
>      guest
>    perf: Stop pretending that perf can handle multiple guest callbacks
>    perf: Drop dead and useless guest "support" from arm, csky, nds32 and
>      riscv
>    perf: Add wrappers for invoking guest callbacks
>    perf: Force architectures to opt-in to guest callbacks
>    perf/core: Use static_call to optimize perf_guest_info_callbacks
>    KVM: x86: Drop current_vcpu for kvm_running_vcpu + kvm_arch_vcpu
>      variable
>    KVM: x86: More precisely identify NMI from guest when handling PMI
>    KVM: Move x86's perf guest info callbacks to generic KVM
>    KVM: x86: Move Intel Processor Trace interrupt handler to vmx.c
>    KVM: arm64: Convert to the generic perf callbacks
>    KVM: arm64: Drop perf.c and fold its tiny bits of code into arm.c /
>      pmu.c
>    perf: Drop guest callback (un)register stubs
> 
>   arch/arm/kernel/perf_callchain.c   | 28 ++------------
>   arch/arm64/include/asm/kvm_host.h  |  9 ++++-
>   arch/arm64/kernel/perf_callchain.c | 13 ++++---
>   arch/arm64/kvm/Kconfig             |  1 +
>   arch/arm64/kvm/Makefile            |  2 +-
>   arch/arm64/kvm/arm.c               | 11 +++++-
>   arch/arm64/kvm/perf.c              | 62 ------------------------------
>   arch/arm64/kvm/pmu.c               |  8 ++++
>   arch/csky/kernel/perf_callchain.c  | 10 -----
>   arch/nds32/kernel/perf_event_cpu.c | 29 ++------------
>   arch/riscv/kernel/perf_callchain.c | 10 -----
>   arch/x86/events/core.c             | 13 ++++---
>   arch/x86/events/intel/core.c       |  5 +--
>   arch/x86/include/asm/kvm_host.h    |  7 +++-
>   arch/x86/kvm/Kconfig               |  1 +
>   arch/x86/kvm/pmu.c                 |  2 +-
>   arch/x86/kvm/svm/svm.c             |  2 +-
>   arch/x86/kvm/vmx/vmx.c             | 25 +++++++++++-
>   arch/x86/kvm/x86.c                 | 58 +++++-----------------------
>   arch/x86/kvm/x86.h                 | 17 ++++++--
>   arch/x86/xen/Kconfig               |  1 +
>   arch/x86/xen/pmu.c                 | 32 +++++++--------
>   include/kvm/arm_pmu.h              |  1 +
>   include/linux/kvm_host.h           | 10 +++++
>   include/linux/perf_event.h         | 41 ++++++++++++++------
>   init/Kconfig                       |  4 ++
>   kernel/events/core.c               | 39 +++++++++++++------
>   virt/kvm/kvm_main.c                | 44 +++++++++++++++++++++
>   28 files changed, 235 insertions(+), 250 deletions(-)
>   delete mode 100644 arch/arm64/kvm/perf.c
> 



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 07:01:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 07:01:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192203.342530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwFi-0001rw-OB; Wed, 22 Sep 2021 07:01:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192203.342530; Wed, 22 Sep 2021 07:01:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwFi-0001rp-LI; Wed, 22 Sep 2021 07:01:10 +0000
Received: by outflank-mailman (input) for mailman id 192203;
 Wed, 22 Sep 2021 07:01:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSwFh-0001rj-GM
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 07:01:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id dba3f41a-1b72-11ec-b961-12813bfff9fa;
 Wed, 22 Sep 2021 07:01:07 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2054.outbound.protection.outlook.com [104.47.12.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-ng0v2zTHMDekmwV2LKzbDg-1;
 Wed, 22 Sep 2021 09:01:04 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3392.eurprd04.prod.outlook.com (2603:10a6:803:7::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 07:01:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 07:01:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0108.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 07:01:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dba3f41a-1b72-11ec-b961-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632294066;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0hlgxtfWB65IzWfXsyfgGnIlLW0HEBD60ix0PD4dU/A=;
	b=jZaEkGiphDmD7CVSmDn+oOnyQ6RHdQZZi4F2O2TlNdKSdFrk0V+XL9Uy34Ic99WZKambQp
	V30JLb2ROkwn8JhUBnuAVjni8y5TvldcknKpVJJ1k0boPBElXc7+UVPtZw9ZN2iSVPwJQ8
	xvPVueY7eLnO86qBl7meMQHxiukGtKA=
X-MC-Unique: ng0v2zTHMDekmwV2LKzbDg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G0jVUQ9xwgruqN+aEn0/jZ216/oxB9UMbThnkSFsrs+TNV0RAEDwX7qeUe51/4I7ZSgsR+mAuZ0s7IYKrKJcqJ+zXBUsDRfUQjsM3aqZkbm1/zxhuSzKI1MqOHo6gPWJXuFI5eqfvBg34dskVRJjyl/caktAMDflahys9OvrOzZPhNLfZVcwCecTfytprX+60llFuQZdtRKo1PpKxbKDX/IySUKPPU84uN3IZAkMFmVX+uiCmn95SeBoKi7Uw6/MTwC57gNKXLAzIDmPRZViLSqzstDEq9fYrTCY4jX0LgZwvizT3YJU4+8srxfgo3I/Yxjn7sVO1zne4a+k0eQlqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6PhIu2Syyiorg/TJVOVOlcj2U13qurfGV+jCJIahXck=;
 b=NRMWjLP3/SEb0oqsTfgjRmE+f6uoC5eJ6heNB1UFzfGgJ6BkOIJF5XEaVYkkLP6ZoNSugkR/keRoXYnQC3XaKS7ANHR1uVxSYBJiP7ba1x8I0wcvER02TQrn07YhGi9+f0XFcjn2jedP6TCnbF9zvS2kTVbPlgdoXyQ+p4JgkEqaugR7MR++hUvCmEsfCZ4f/v3/trpZHFA8qFVh6L3nMooFf/SbMbzzcnphhDZA+PDzY85z/IaEYqrlFshE7zkNeqO2rex0naLdxSzwxqXt1UrILJsi6kTGX85cj02isu/BaxrJKu8cSWOCAXA+uPiuHOG7Bm2jqZBjryOlU+dijQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Meng Xu <mengxu@cis.upenn.edu>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
 <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
 <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <69770706-85f6-6509-b5c4-bda61dfdf994@suse.com>
Date: Wed, 22 Sep 2021 09:01:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0108.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5792eb3f-31c7-4e1d-b17f-08d97d96bd1a
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3392:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3392495A7D7FE378C771261CB3A29@VI1PR0402MB3392.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	E/X/CodWgAIh5fDpBohfyOzDR+bJ4JPtMa051QQufaT4P2W4RwTP4WmlZ1/nO7PPRNs+FQ/hN6P/ZF9o/Y7riE98VxJEJOu5sLCsoL5X/wBa0Lly+peXQ8NN02LYYVltWKc9AWHexbin8xZ+jevZDHusg8hCtrJSD8mlH6iFY7C9W4B3z7xxnguUdkDqvh0kEQUIuYIC0ShJNn8Ei94z7o9tGz2iiniRpMT+YMVFwivf7pRvuLcvjNxZy+OTNsx4Pz5yJ1zZvbH3tpqbbR4lZFy32KNonQXSEUgxvCkki8YRqdWLA0tWt93dD79Zu5SB+8lPSsD18xer3/uHRRWqsaMqz6l+9Gk+eqn2inkNt7o59WoW2jqB/dH8ySzdJVWduwOGWib783FeLCPaLFgkQdyOsth4xP8iBvTdOR++wVPmqE1kABvysXB5ASyIxcPxMcEiK645Xh4TgJL1QHVl7kSHT0YcChGSKcphMh4Y0wL8i4q+wc0ICX7zw1pRivAMAXjNAXBd7US0s1OxoHsvM+qsspmFLXJxM2PGPzVncRl/XBT9HGB1rXJcZATxcAFH4HrfAOTCRT3B+h78iegWm4mpCIB2czSQcnuwa432pq5oLAz3ZmoXlF8eDIZYtjIiq4uDCLo8Bc5b06V0/ZA88I3wgAWIGKAmKvsBMFkjuGazFBrr7avZkl/3pTlTm0OAlcsVmbW6vhVnaXjrjbXc3B/pB1hlV3eXrlcm2/2BXuE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(38100700002)(86362001)(31696002)(8676002)(4326008)(8936002)(53546011)(6486002)(66476007)(66556008)(66946007)(6916009)(186003)(316002)(54906003)(36756003)(5660300002)(508600001)(16576012)(31686004)(956004)(2616005)(26005)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?qgFbLaKF9vJJcGYHSWFS36SCpwDh9mcS1i5fIup9tx8GxaGX69eZK9yRy1Hk?=
 =?us-ascii?Q?amZquPfHv4/86H6OJR3N9Vue/QeG0PsqoqKDYQaXcdeAm9o4fFAdSWhhUAps?=
 =?us-ascii?Q?x4RAhRwFFgvsm10lzX4aySqdDg8PNww5r79TvLRoVDPwKptTOxWqc65eF479?=
 =?us-ascii?Q?qCLxMvxTWlMDwdVnhFRsN4JweAZ1x6VZ8ONt6K4OCAPW7QBUOlGPF5Olchro?=
 =?us-ascii?Q?Ak95jBRyF63KxYmwiQcjGfZQ1bhMX3pgwK88zj++HCulLJMZGM9rU10xN/xC?=
 =?us-ascii?Q?lQlVHz/o/LiRd8c8dzsD7iD+zwSRuJtvYs2Z2S/3+HCznoWKsGGEaaxntIXR?=
 =?us-ascii?Q?kZb0tq58dtqFhgh+iaxpo5U7AB6lIjtbo9ZNplbNNFnS8zCYaOjr2FRy7ZxO?=
 =?us-ascii?Q?ZVoAxfM9H8jklLm/yKYJGa44zaVaG7gkq1CNh8nNeio2jeOWczK99rjjWpMy?=
 =?us-ascii?Q?BEi86lNMNzVEbCfLYHv5FwTxj/G4n2iqBcDzDwuzxPzSH9+ictZe82ZLRDXe?=
 =?us-ascii?Q?w9R3Opr5BH6joGE3Kq4Szq9fBMRAaLTmTAK92Dsmlx+Sv0Q+ysOHnjisZYjw?=
 =?us-ascii?Q?x6r/CTsTDh2IvEETMPl21I6/vV88OsnGhRBE9nXlehppIG9qcrLtKG4llFom?=
 =?us-ascii?Q?M7TxF5l+QSNZR7bKyUmGlDToo8YORQUhfyfk1zD5qr1MIlqiUfpnFmUs1WOG?=
 =?us-ascii?Q?Qm8BiXmDCK9qNNET1KutRpR9UT46hYXWaEtdGjOLAu30EMgHpaNfa2dqN3sr?=
 =?us-ascii?Q?CfaReKbIt8FouJaBpm6Zko/IFpBKNOotTupz7h+r3SPm76OHc1Rf/ETF8gfg?=
 =?us-ascii?Q?GC7fpu/syWwFNZ9GYzicChO0wHkEHxehoEjizxGjDSpsLudHu1f0AjU/2jIe?=
 =?us-ascii?Q?WW1MdzPvyyD6RTf7KemG7PKwnBV964l7LGHLSOkTUw6GJKWM8PvOwgMe3NSr?=
 =?us-ascii?Q?eCgI2un7rMrIoMB2I5AI5vFhdxArSXSu42wEoCrzzVvj0ZiSA5M0L2aWtoMH?=
 =?us-ascii?Q?KwdFasnnJSzDn+ICK2b8FaDREMTqxB5y9REz0vZGJu8zyZP7U6ZuuOdMpqxF?=
 =?us-ascii?Q?INU57l0VU5rw17/dNt6xswXSTDpMPzgmcw8qwrg+bSIRCSC7lfW/k6IKCRoW?=
 =?us-ascii?Q?o7c9b3IKWQHNgVznboOH9ITg09GAlNInQzyZ46eyEwKBInm0QsdY3QsGpsId?=
 =?us-ascii?Q?amtl/0yMGtplW44yHT90TYcbKvYGaMmMNa//rzv2S7MAHIMYm+XpL1mYx0lb?=
 =?us-ascii?Q?ak3MgsT5fC0AnXGsAIyDP0A6wMUo5kZ9KItNrzazKurd/H3+j21vJkV+hTh6?=
 =?us-ascii?Q?jKNeBKACFUBiZjv5xjNOkEyD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5792eb3f-31c7-4e1d-b17f-08d97d96bd1a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 07:01:02.2434
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3TwmoIKMX+PL+bJ0vgpaL0HtJt8uoVMeMdYEcBtXHgqSIN51q+irnsaF45veu21kQ2Gaq2DmD3QssmE0QoAONA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3392

On 21.09.2021 19:51, Andrew Cooper wrote:
> On 21/09/2021 07:53, Jan Beulich wrote:
>> On 20.09.2021 19:25, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/hvm/hvm.c
>>> +++ b/xen/arch/x86/hvm/hvm.c
>>> @@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE=
_PARAM(void) arg)
>>>          if ( copy_from_guest(&tr, arg, 1 ) )
>>>              return -EFAULT;
>>> =20
>>> -        if ( tr.extra_bytes > sizeof(tr.extra)
>>> -             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
>>> +        if ( tr.extra_bytes % sizeof(uint32_t) ||
>>> +             tr.extra_bytes > sizeof(tr.extra) ||
>>> +             tr.event >> TRC_SUBCLS_SHIFT )
>>>              return -EINVAL;
>> Despite this being a function that supposedly no-one is to really
>> use, you're breaking the interface here when really there would be a
>> way to be backwards compatible: Instead of failing, pad the data to
>> a 32-bit boundary. As the interface struct is large enough, this
>> would look to be as simple as a memset() plus aligning extra_bytes
>> upwards. Otherwise the deliberate breaking of potential existing
>> callers wants making explicit in the respective paragraph of the
>> description.
>=20
> It is discussed, along with a justification for why an ABI change is fine=
.

What you say is "This has no business being a hypercall in the first
place", yet to me that's not a justification to break an interface.
It is part of the ABI, so disallowing what was previously allowed
may break people's (questionable, yes) code.

> But I could do
>=20
> tr.extra_bytes =3D ROUNDUP(tr.extra_bytes, sizeof(uint32_t));
>=20
> if you'd really prefer.

I would, indeed, and as said ideally alongside clearing the excess
bytes in the buffer.

>> As an aside I think at the very least the case block wants enclosing
>> in "#ifdef CONFIG_TRACEBUFFER", such that builds without the support
>> would indicate so to callers (albeit that indication would then be
>> accompanied by a bogus log message in debug builds).
>=20
> That message really needs deleting.
>=20
> As a better alternative,
>=20
> if ( !IS_ENABLED(CONFIG_TRACEBUFFER) )
> =C2=A0=C2=A0=C2=A0 return -EOPNOTSUPP;
>=20
> The call to __trace_var() is safe in !CONFIG_TRACEBUFFER builds.

Fine with me (I'm inclined to say of course).

>> Seeing the adjacent HVMOP_get_time I also wonder who the intended
>> users of that one are.
>=20
> There is a very large amount of junk in hvm_op(), and to a first
> approximation, I would include HVMOP_get_time in that category.
>=20
> But c/s b91391491c58ac40a935e10cf0703b87d8733c38 explains why it is
> necessary.=C2=A0 This just goes to demonstrate how broken our time handli=
ng
> is.=C2=A0 I'll add this to the pile of things needing fixing in ABI-v2.

Urgh.

>>> --- a/xen/common/sched/rt.c
>>> +++ b/xen/common/sched/rt.c
>>> @@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct r=
t_unit *svc, s_time_t now)
>>>      /* TRACE */
>>>      {
>>>          struct __packed {
>>> -            unsigned unit:16, dom:16;
>>> +            uint16_t unit, dom;
>>>              uint64_t cur_budget;
>>> -            int delta;
>>> -            unsigned priority_level;
>>> -            bool has_extratime;
>>> -        } d;
>>> -        d.dom =3D svc->unit->domain->domain_id;
>>> -        d.unit =3D svc->unit->unit_id;
>>> -        d.cur_budget =3D (uint64_t) svc->cur_budget;
>>> -        d.delta =3D delta;
>>> -        d.priority_level =3D svc->priority_level;
>>> -        d.has_extratime =3D svc->flags & RTDS_extratime;
>>> +            uint32_t delta;
>> The original field was plain int, and aiui for a valid reason. I
>> don't see why you couldn't use int32_t here.
>=20
> delta can't be negative, because there is a check earlier in the function=
.

Oh, yes, didn't spot that.

> What is a problem is the 63=3D>32 bit truncation, and uint32_t here is
> half as bad as int32_t.

Agreed. Whether the truncation is an issue in practice is questionable,
as I wouldn't expect budget to be consumed in multiple-second individual
steps. But I didn't check whether this scheduler might allow a vCPU to
run for this long all in one go.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 07:04:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 07:04:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192211.342541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwIY-0002ZJ-9s; Wed, 22 Sep 2021 07:04:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192211.342541; Wed, 22 Sep 2021 07:04:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwIY-0002ZC-70; Wed, 22 Sep 2021 07:04:06 +0000
Received: by outflank-mailman (input) for mailman id 192211;
 Wed, 22 Sep 2021 07:04:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSwIX-0002Z6-6t
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 07:04:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bd872aa9-dfb0-4933-a383-4b13c7b4b033;
 Wed, 22 Sep 2021 07:04:04 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2056.outbound.protection.outlook.com [104.47.6.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-WDEx5p1eOJKopdT7SL9k2g-2; Wed, 22 Sep 2021 09:04:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5901.eurprd04.prod.outlook.com (2603:10a6:803:e9::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 07:03:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 07:03:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0128.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:e6::45) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 07:03:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd872aa9-dfb0-4933-a383-4b13c7b4b033
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632294243;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WrNld46FLQXMxSEDG20jPvZ658ARGRbFtLANVAASoaI=;
	b=BKZlqDMknF1lLGFnFP4wd5dfpXV1jTaQ31UNj9kmtgf5fsknnDl95s/+XtrwhIspY1ZWXQ
	sb4f/PTLgi008RVpoaHHStp080ugJ5xrFZEQ17OHf9YGAz1sTuIAt+x6NaZIHZU8azrOP6
	vpGk7wYQhbg6lD9kA/h1TL2OtX76cqM=
X-MC-Unique: WDEx5p1eOJKopdT7SL9k2g-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CiF0oMptAdRe/DUtPbfN2RRedCYGfH3pWoqTkKpCiY6kXoK9MMiRbLd0b/chJbzb2Wb3wGYtjzs8VevtXcdAU5j3WGRCtbLxHTlSUyCXqZglGHY4oti9848rWvubvZPqeIwekrt57SLgg+ZavvN7BkwA0RIonPcsiOTcHrZ0Zgy4MlXaL2pc/2T1GT4jimo7zz2EhFzYn0XBAtK49nE3S2kkUXkQvF79K1JCa9K3p8aK1mGb3dxiiNvXPTT+cHXAXWjD2oLnA2UVldNZm/tk1vhfEzCnG6jvM04okQwJ5wxRC0JXbmBDj7Ft90Nc9FHEWGMTGngybF4iJy5Y3xSl6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8hA5CF9k5Gw3YJIJbxUjLGQqn3qkUPNgnAZHoFImazg=;
 b=G0davBIf97SZ7w6dH0j8W1sEuvVgnnQ683CTYZOGkYzRAaao6RD6VwaZz18EjyNokzzudXOXhs/L6BHfjmbtnusR19y/gaueUItRRvWCruuPf2eM1OzybHfGopjVkG1QAF2uzG0WPsAfrmkCVvqNRHRjPPaHx2NtXIUyVs5djwAFro2tL7quSasPLr09lFrNFqiqfsaIgskjID64N6VWTNFYCCg+MdZgGFhqFa4yfKYkfgd+33OIrXHp5tp0CPUGEDQkvmHqt9VYe3zXnJ2ufCuyB2/vHTlRrAe9OWzxFyWW/AeZzp5SIlogn8zZwe4VGu/QuYScg/SoCnzA+LXi2A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2.1 RFC 17/12] xen/trace: Drop cycles parameter
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Meng Xu <mengxu@cis.upenn.edu>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <2bd51843-de50-d442-dd5f-7fcc340c3f26@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c56b991c-d9cb-0fbf-a29a-6bcd60b09d58@suse.com>
Date: Wed, 22 Sep 2021 09:03:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2bd51843-de50-d442-dd5f-7fcc340c3f26@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR10CA0128.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:e6::45) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 32c22205-b36d-48d9-57a2-08d97d9725d9
X-MS-TrafficTypeDiagnostic: VI1PR04MB5901:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5901CC9973D1D61DB26365D9B3A29@VI1PR04MB5901.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cFRbMYCXij2v/ljFAvXmtyS4jI9o2alJver7gnourvQ5kTbq+uXRz7b8tRPliE1zMkGrj1PGDKbjS0FNMlb6NVeXL0MTSXMTadumLZSx/MRy5G1igYivObumemJBKwz2ghAcBGo0OrKqLnjIaZuYurd/O6b6dnV6NTMYd5FmDT2bhrFFNCANJXA/qS+sDBp10VhSbau2aHUsU8upRvKh9u3AkDdSN4gvyZzcLa74PMf2qBgquZtXd6AIQEVxJNL9hzswf9U56L963YfHrOu1ednG8o9D+FmBWP5DJ2fAefpKPduRLC7D8KLO/toW1wwvT7r50t20qvaB1i7ksF7Rp1NTz9E1ImZ0RnGJGnuoEYUMGp22/J270Bl4QNn1Fehfj3OvWritkMEehYE5oVvp0FmxxmnNZq1awfJy6Nvqw/T3ik7D4OpweQi4JoEcUT1tsN+4W6Yz2GSAV1rAKleeuIgzvXsYXeTSNWX2yib9neVZ8CiSuULwRIfi8Sixs8cVOoBbrRsGtRi1HuZgkfHQ6k5SHN8ArsqJwbeznbKYDdFlFlM20kdNbC6wlaVsAdRyVCk13qyZ8bYbqPxhoLiOY06J1bRBAgRCSQQvg+Tku0FUPPAuRnqC3rF1vVG5LXw0hFOAzejOhKs0w8LNYahFgrpZWkAZnHAHhas6eFXY/gri364M3112Nb9ylA94GoAXyb2xFOpAbGCkPGr/5uco17/WKkf3MxpPi0fjpXfQqfg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66476007)(2906002)(8936002)(956004)(54906003)(26005)(86362001)(2616005)(36756003)(31686004)(5660300002)(4326008)(8676002)(316002)(6916009)(16576012)(38100700002)(53546011)(6486002)(31696002)(186003)(66946007)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?4WkhGw0PgdzLngogZLoZ31p5EEJjZTpJjjrGd1r2M32AzBgjyn6qo1QT1w5X?=
 =?us-ascii?Q?o7XPGfWx1hTDSzGj9ixCsoIPThF/E0AAyAjPVuRcEMEThfgeOqombgTGCAtL?=
 =?us-ascii?Q?uPsn/RywgD9zdfGFfsvfTtwDoqNQhyTHRydE6UglKPjk1C5WZlKvTn5Is/SK?=
 =?us-ascii?Q?sPMiO6iLCy91hSASBPJzxKcvskWwA3XOSpqJ6Zi+T2y2jG64NMSYDyRNwZ9T?=
 =?us-ascii?Q?X4FFkXF12zyxxveVoXMEo5TCZLl3Hk4XFsWKKM6KnsncFR/Dry0c58NgO/LW?=
 =?us-ascii?Q?DDz0oep9wh4OZDRviNc/kOMeNuiwU14ZQ3VM9iD4RiobxX+AXSK0OwhQU1lq?=
 =?us-ascii?Q?Oh76Fz8T4gQVIY9RK/A2raqXqXgkMXl8NOvX/crk4lnOwydDANGwl5a9jSi9?=
 =?us-ascii?Q?YHLFg+235wIQ1dmEQAW2DICX/72+Fa9BjZAQlF8kWVhtnoI80rnsUyFXxpWz?=
 =?us-ascii?Q?NzD11SAZpSniZjqMfZEEpvLkDil+2VKvR9UGN03rbYtyFPuroTKr7xBZSwd6?=
 =?us-ascii?Q?v8dMHikdzc0N29Z/SYCO3Hlqd++QbZAk89NgYlY7Ld4BJzi8yqRlJSBcn/v9?=
 =?us-ascii?Q?mQzdcxt3dP+w0oovU+7GBSGFOWaoAieMEhAqG0nstyBjZj09fcV/LU+OetdL?=
 =?us-ascii?Q?N5ox6WvPsZOfLmvdvPMoDbRybNTSMtqVgntZUTnfUGus+fFh+pOAorziukvQ?=
 =?us-ascii?Q?L/VexGdrZDEd/1g5o0zgLRVHcBzwyf98OMfy+S4lp3FlyKqGwo09/qN8aYZG?=
 =?us-ascii?Q?iKCAzU8StLgZWRFOJcI+yM2GyECgHXvLnGE53Ba9phlc+YOxACQtJdeccNph?=
 =?us-ascii?Q?DxY1IoQOZ4Vs1q8mySZMdLKO5yGo+8GvnT31IvKD3lFHjqNujfX9hOc6qyol?=
 =?us-ascii?Q?K9ExJnaQyxjRi17m67TaaWYLj/o9Z0230CGFCxZdewMqexKNPNLV6WePlhK0?=
 =?us-ascii?Q?61e+Zw2aIqlnZ5tG9F9t/Wj8mI45L4U77XGp0mtIqx9veU0u3hZ03ofAc3hg?=
 =?us-ascii?Q?fiGk2aDhPX69bEoFWUryXUr1Gl/izW9KRATunlAe53LyIJ11EjwlYIt5Blq2?=
 =?us-ascii?Q?P1YaasiSm0pEPLin0ce3j3QliIKy7v0yJjDk9JEJ2eUq98CTDdRWjkbcLEAl?=
 =?us-ascii?Q?3F49CoaFhZXgD56lxGt6ofejOoPVtmwkrOfHGhb1r4GxDxjDlyey+tXFTAUm?=
 =?us-ascii?Q?lA2jd0qMmLsLPz0R5QVqau4cRQqFJUxj/tVu9dhCSujGgkQnI9tEUIqCXds8?=
 =?us-ascii?Q?jz/G+Stk1FTZGunXG4T5uJtQPm/b3QeEPx6ahMEuZAvvdquZKeJyc7xwQif0?=
 =?us-ascii?Q?50IOkKKZIiuNziTUvnArrEof?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 32c22205-b36d-48d9-57a2-08d97d9725d9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 07:03:57.9179
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Sf4dMh+25Gj6y+0CaXeYxhsz/LpWuE9c8J9Pg5ESTLbRp2jnOIQH27wUHPbyVQQO4IHwEhjY/qwYF5aKB/UDPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5901

On 21.09.2021 22:08, Andrew Cooper wrote:
> Not a patch, but an RFC idea for one...
>=20
> It occurred to me that the cycles parameter from __trace_var() and
> friends is pointless, as the cycles bit is encoded in the top bit of the
> event field anyway, and passed in the adjacent parameter.
>=20
> Dropping the cycles parameter results in +85/-1029 (-944) net change.
>=20
> The common change in callers is e.g. from:
>=20
> lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rcx
> mov=C2=A0=C2=A0=C2=A0 $0x4,%edx
> mov=C2=A0=C2=A0=C2=A0 $0x1,%esi
> mov=C2=A0=C2=A0=C2=A0 $0x10f002,%edi
> mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
> callq=C2=A0 ffff82d04022ea20 <__trace_var>
>=20
> to this:
>=20
> lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rdx
> mov=C2=A0=C2=A0=C2=A0 $0x4,%esi
> mov=C2=A0=C2=A0=C2=A0 $0x8010f002,%edi
> mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
> callq=C2=A0 ffff82d04022ea20 <__trace_var>
>=20
>=20
> Just sprinkling "| TRC_HD_CYCLE_FLAG" over the place makes things a
> little bit unwieldy.
>=20
> Instead, I was thinking of implementing trace() (and a thin trace_time()
> wrapper) mirroring the "new API" in patch 14.=C2=A0 Half of the trace_var=
()
> users should be __trace_var() already because of living inside a
> tb_init_done conditional, and the rest are actually opencoded TRACE()
> taking no extra data.
>=20
> Thoughts?

Sounds quite reasonable to me.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 07:15:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 07:15:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192217.342553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwTV-00049t-DF; Wed, 22 Sep 2021 07:15:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192217.342553; Wed, 22 Sep 2021 07:15:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwTV-00049m-9d; Wed, 22 Sep 2021 07:15:25 +0000
Received: by outflank-mailman (input) for mailman id 192217;
 Wed, 22 Sep 2021 07:15:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSwTT-00049c-PN; Wed, 22 Sep 2021 07:15:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSwTT-0000m4-JR; Wed, 22 Sep 2021 07:15:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSwTS-0007jG-FF; Wed, 22 Sep 2021 07:15:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSwTS-0006aI-Ek; Wed, 22 Sep 2021 07:15:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=63BwA3Bg5DJie/EhsVrb1QMYrIDVqdFa+lx05h9/MOs=; b=YyYkGbs0xSLXU7tzELrUf8PmDa
	cWBb4D70M77qDcvqwGgmi4gR9eL53Pj49Gh3Xlut7QBEu+S9ZLwkyDXMrhoF5KLfThm75ZMR4HcQZ
	miwb5K4m2n1xHm3aYRwL4b0uiIWzOe43rWR1Rn0CJmHOdciMVRSRVvui9+jAs/USSVyo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165145-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165145: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:guest-start:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=92477dd1faa650e50bd3bb35a6c0b8d09198cc35
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 07:15:22 +0000

flight 165145 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165145/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 14 guest-start              fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                92477dd1faa650e50bd3bb35a6c0b8d09198cc35
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  417 days
Failing since        152366  2020-08-01 20:49:34 Z  416 days  727 attempts
Testing same since   165145  2021-09-21 22:12:58 Z    0 days    1 attempts

------------------------------------------------------------
7322 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263575 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 07:35:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 07:35:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192225.342567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwmN-0006iY-2T; Wed, 22 Sep 2021 07:34:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192225.342567; Wed, 22 Sep 2021 07:34:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSwmM-0006iR-VM; Wed, 22 Sep 2021 07:34:54 +0000
Received: by outflank-mailman (input) for mailman id 192225;
 Wed, 22 Sep 2021 07:34:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSwmL-0006iL-HR
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 07:34:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 92648bfc-1b77-11ec-b961-12813bfff9fa;
 Wed, 22 Sep 2021 07:34:52 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2054.outbound.protection.outlook.com [104.47.4.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-7dm5D0qsM7-A5hyMlKcQhQ-1; Wed, 22 Sep 2021 09:34:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3533.eurprd04.prod.outlook.com (2603:10a6:803:b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Wed, 22 Sep
 2021 07:34:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 07:34:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0163.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 07:34:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92648bfc-1b77-11ec-b961-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632296091;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=g+DxErIK80ExGUZHlpS0lXGQGa4hZ6FYuJR0+GOnBaA=;
	b=S7BjYY0chekPdw18FmBILgDoYIMmFgmYjj6THjcmUapmDVzcUyUbQxlmvM5VyR8B5++r9/
	kbBJshQEyjJcljqs8ha1oNfq1dvoVmK/lGsV9LouGotYQKmQN8fMbR6kgMSXbQnvKMVgl9
	/dgZAyjZJ+JDVV362Y8AnrVPO/8LEiY=
X-MC-Unique: 7dm5D0qsM7-A5hyMlKcQhQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LuePZIo2UQDxDvCpD9HML9nCC71wh6PVicHAvNPM1oAR5prA/+5OUDAR9tAz1G8oj0YnlgNJefGApN8EVW5J1Kmzzql/kE6Db8wVm8DRzWA956tJxl7ayYm8p8lt3RwusfTQbezSiqJU+tPAxcX4l00FsiBFzJ4VV1Tcwql6poORmBNbau8NSDa8zVB1jpjL+impxIQXbxgaw18awzqvm3YmMWmvt4Hd9ky0ntJMbMDmMXJO7cmAv6cwCo3QNfTdpVwcne5z62LKxwfXfshoPbCRd7Y5oMLZGsJHpjeSiMhbfu1o3/LH050nWJI6KMxyamT2t8isBRcyOLhoxaAe1w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g+DxErIK80ExGUZHlpS0lXGQGa4hZ6FYuJR0+GOnBaA=;
 b=PUJjpCM7DBkji0EDnhTvdwaXcNqSuNbXLoGExdSof6DAQssIGr/dJlnneP3K4swRQnAT87Iferzh4+s2EOjKSYh+zNuFAQ8WYuBwGduQ+TSwr59z1vSjFZB3lNmC3WiO0VmVfOU5JD3u7oVbflu0PGUpJGxItg7MTETR63Og1psxXc3rFkg6olJYRBJDlkAccqqf8Vey4WMb/F4jjybEcvC8Oewc1LaywuGKDvCqaOnp1aIgiBf2b4F0ltAGxDelgv4VOAwf6YXOnc6rv2FmsT/rEo1xhUjQPbgwsnW/xVSkzHVjWr28CWiJjp1zpSy8plJSPvDrUtJiSLuGaMSIIQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Stefano Stabellini <sstabellini@kernel.org>,
 Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, dpsmith@apertussolutions.com
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
 <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
Date: Wed, 22 Sep 2021 09:34:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0163.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5f123f80-1dec-4c5b-7ea1-08d97d9b7489
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3533:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB353304FA5377EBFFCA7EF3CEB3A29@VI1PR0402MB3533.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HwRjKBdWvvhpxIpzslYJAO/Ea3NDiq1rlDXJlXsE1eyQEXFJxYRocCbgaFCmk7SbeQFQZo8XX40VbADahJqt1v4lQFfCgxL6BwvWUpTw9dnf7s/kVQ7RBc4XwyApoYL69YjHttCKnyRQGw3KCkljLNP0fSyYrqNhdGBk4kT2J3taTMz7hD/obOkUnl1XTAlE6QtAIFmS9Syi655rAWwAVIu0C4cp1H6v+xjBMRy3OS5LiysGHDDAFCKIuvwSafJ0U+OxKrA6vhQf0g2gjSacWJ9g76euUf7DpF+XMyqF/K39kY+R9qEQ/EcaCGmWx5vvrJ8uRNrYUUDwA2dh4pF6eKTHavI1bibk+D3GkrQh3eOQ4SmhhdL7WaE8EO5kDW6+CeCQbq1DlDQlJrhrnjSeHHipb2ftargmE0Iwo04gWTkXsfgAhUgSCTXfhsWjXlbJO/zgOmwGbLvoJNuEaTQX9rJpuhUs2ClTx47pH4NrZeniBxp3IW761v1UKFJCYLHxDcCY5y3/sFpUao56vRRxakxvV9JueMk4cAbxh7KM8F+khIMtXfHfjhI4TUeLrjm4veflC1EN6aF7XQiRzJSotGBAvjbYec9ll5HAOxwX6gi20kbwK8te3afZ7/CpeOXZ2hVIP0nbHWlstEqv9s4+rzKTdHRoTGz64IjvQITqJztpqABd7pm33FCz6HBka3pgRz9WJ0udsNiZ+t82ywvVkMQbjlKrGNWHmlsqmuHshBB0XQLjwEjjKhkcTq0Kk4CR9Q6g3flduqCDs73lJPSgeeztQZm6Sv9P8OdQdyK5VEEGkza3NoSUVxw49Fn/WhWl
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(66476007)(8936002)(956004)(2616005)(86362001)(186003)(6486002)(31686004)(508600001)(26005)(66946007)(4326008)(316002)(5660300002)(53546011)(966005)(31696002)(8676002)(16576012)(83380400001)(36756003)(110136005)(2906002)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cGtlWTgrV2pRNEZKSjR6c0hSV2hUL0RtWEsyUE5YMW9sN1dmSUVVeWszZTg4?=
 =?utf-8?B?S1RHU3J1Qk0vNVV3cVR3L3F1MjBMWHRMU0kzSWRKV2ZpZkI0bkNhK2JqeWpN?=
 =?utf-8?B?OXdEcTI2OFd6cFZpOTV5VmJDcHM4cTc3RE8xWEpTSC9pTVNDUFdLZm01eWtz?=
 =?utf-8?B?eXdRK09rei8yOXgzV2kzajlHamFQMmpoSUlLYnhLb0psYldPSXU2N3dYVWNt?=
 =?utf-8?B?TEd0VnNQQzRRU2U1MjMrU2FOQWxJUzN0SlFISXl6MW9BQnk3WnRMYmtMUWtK?=
 =?utf-8?B?dURqM1NmNDFOdENHQ1dBcmROMzc2V1Q2YUZHc2VxMnNYU2xSdVZuNy9uMWFx?=
 =?utf-8?B?RkJGK3BnZGx1Q3pKTjhXUTA4cGJ1OEVXcXN5VHpuaURNMFhkRWdicjZkSTgy?=
 =?utf-8?B?MDVmaElhL3RrMEFYeGdXU0JTdzhwSWNFZWQzdVM2T0UwbUwzUGo5cndqM0JZ?=
 =?utf-8?B?YmZqclJhTXFYUUlRWDh4MHk3bkpKRnF4ZzFPU3FlN3AwdlZ5emVMZnU0UVRK?=
 =?utf-8?B?bm50ODlremd5S3B6UHc5MGE4UnpnWWIwRzVOZUdQbXd6V1lPd1VlNkI1cnBC?=
 =?utf-8?B?dW1FajVUMEZvNlVLRHdLZ0lkWEpVOGQ3K010MUtmdmU4ei96VGlzSEhPM1RQ?=
 =?utf-8?B?Y1FhaDRsaWtHWmdSdVhtbmt6YTF1b2VjSlRubzBLcDJtM2YyTmdHM1k2VmlC?=
 =?utf-8?B?UHp5d0VhS2ZxU1RsaFl2cXM3azhLeGVlQ0RtK2tXc3BpZWQ0YTBlU1NaM2ky?=
 =?utf-8?B?dEl4Y2lFNk9OZW9Rb2pIQVhXdjhQOUYySzNjSXp1REsySlM4RG4rZHo0bXJn?=
 =?utf-8?B?T0xMcG0zNG9VcGhwZ2ZIK0NRVHFyc3NwaEJ6U3UvUUY2R2pidFlKakNoaEgw?=
 =?utf-8?B?NE80bTRqMStHMFliS3htNmRnVHNQMTZ3dVhTTGd3T09hdFhWcWR0bElTM3pv?=
 =?utf-8?B?SmVTUmNCSW9ISnB3blJHK0ZXbXRyekhJYmlFMnNLL2V2Z0lqckxPdWpiQVJz?=
 =?utf-8?B?aXlSQnpkWXlsRms2WnNiRU9PK2cvZEEzMDRxMUVUUXdsR3dIZVpFeXgwQW42?=
 =?utf-8?B?RkhDSm1SUmU4dFp1R2ZsRURiNkZwcytWRGRkRU5lMWhXTjBZdFl1Nm5DdHVE?=
 =?utf-8?B?c0QvTDZBRW1uU2ZKYkJTSnUyZVFybFR6RlJSazZKV2duczlVU09mUU9IdExy?=
 =?utf-8?B?SXh1UlVqMlFOWkg2OXhJZGczUk5LWC9iaFdzbXptTDdSam9tUjJxTXVackMx?=
 =?utf-8?B?SldVekVJeUpkWWF4amVBMFVGejY1TlpjOTJjbTE2R0xSK1BEbTduWTFvRDgv?=
 =?utf-8?B?cHJVTVliVWF1THZpSzd6NmFLQ0NkWnYyYlRpODlockZmWlJGd1JSVlZoZE5R?=
 =?utf-8?B?NmJjME5ZZkVIODBQMEs2MUNTWlRJNUVRejRhL0k1a3RYaE5palFSNXByM3pJ?=
 =?utf-8?B?d0VlNHFSQ0lFN0N5TkpuY0krc1RweXhjTkdHR251SEhzRjZmSURtZWVFR25h?=
 =?utf-8?B?WmpkM1ZJWDh2eUVtQVVwaDRoeUwwS214cG9vV3l3NXMyV3RVODFWTys2OEFv?=
 =?utf-8?B?RWJLNEljK1BXRVp1aHlFbUlFTHZZY1ltVkZTdUNDTlN5NTdvSkRLTk9BYTFn?=
 =?utf-8?B?VlU0MWJGY29rSW9pMm9pQkV3WlNUZGVnM1dyTi8vdE9wbFR0anRiNXVzN3Vy?=
 =?utf-8?B?SGhNK09PU2liNXUzc1hvTEhjOXEvSEpTZFFQYUpXcjJPZTQ0SlpBQmJoT0tU?=
 =?utf-8?Q?P1yRtlzAWsDCME92Gm0eZfjgPc+Mo4Dx3FayL0D?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5f123f80-1dec-4c5b-7ea1-08d97d9b7489
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 07:34:47.9575
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bKOAs4R88JbdVaGOgIdA+WFoaDRVqLe4H1hymz6zUG0mI+4GPZ8vCr2yShrNLI2h4om7feKVGmmlfUqjNXi3Dg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3533

On 22.09.2021 01:38, Stefano Stabellini wrote:
> On Mon, 20 Sep 2021, Ian Jackson wrote:
>> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
>>> As per
>>>
>>> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
>>> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
>>> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
>>> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
>>> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
>>> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
>>> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
>>>
>>> the system doesn't look to really be out of memory; as per
>>>
>>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
>>>
>>> there even look to be a number of higher order pages available (albeit
>>> without digging I can't tell what "(C)" means). Nevertheless order-4
>>> allocations aren't really nice.
>>
>> The host history suggests this may possibly be related to a qemu update.
>>
>> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html

Stefano - as per some of your investigation detailed further down I
wonder whether you had seen this part of Ian's reply. (Question of
course then is how that qemu update had managed to get pushed.)

>> The grub cfg has this:
>>
>>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
>>
>> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".
> 
> I definitely recommend to increase dom0 memory, especially as I guess
> the box is going to have a significant amount, far more than 4GB. I
> would set it to 2GB. Also the syntax on ARM is simpler, so it should be
> just: dom0_mem=2G

Ian - I guess that's an adjustment relatively easy to make? I wonder
though whether we wouldn't want to address the underlying issue first.
Presumably not, because the fix would likely take quite some time to
propagate suitably. Yet if not, we will want to have some way of
verifying that an eventual fix there would have helped here.

> In addition, I also did some investigation just in case there is
> actually a bug in the code and it is not a simple OOM problem.

I think the actual issue is quite clear; what I'm struggling with is
why we weren't hit by it earlier.

As imo always, non-order-0 allocations (perhaps excluding the bringing
up of the kernel or whichever entity) are to be avoided it at possible.
The offender in this case looks to be privcmd's alloc_empty_pages().
For it to request through kcalloc() what ends up being an order-4
allocation, the original IOCTL_PRIVCMD_MMAPBATCH must specify a pretty
large chunk of guest memory to get mapped. Which may in turn be
questionable, but I'm afraid I don't have the time to try to drill
down where that request is coming from and whether that also wouldn't
better be split up.

The solution looks simple enough - convert from kcalloc() to kvcalloc().
I can certainly spin up a patch to Linux to this effect. Yet that still
won't answer the question of why this issue has popped up all of the
sudden (and hence whether there are things wanting changing elsewhere
as well).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192239.342600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVz-0004uK-Es; Wed, 22 Sep 2021 08:22:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192239.342600; Wed, 22 Sep 2021 08:22:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVz-0004uB-BN; Wed, 22 Sep 2021 08:22:03 +0000
Received: by outflank-mailman (input) for mailman id 192239;
 Wed, 22 Sep 2021 08:22:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxVx-0004pu-VM
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:22:02 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 280ab036-1b7e-11ec-b962-12813bfff9fa;
 Wed, 22 Sep 2021 08:21:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 280ab036-1b7e-11ec-b962-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298919;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=UFeBg4V7WhycC2OZ2X38kzJnMjaHRI/emKcUl+Sg2oM=;
  b=f8iCrKmvLipTef9Ynlk1NBuioH/NuxYnBoCTLpH1ZHnVutT0QNOg8exd
   iG4qhjFXsN78Rf+Z4sdgq+KvtQMZh7K9Ile9GXqlx5lKrgh9bsq5p8t4i
   J2ZfzYQlOfpTxDn5viLs8miGm+C/tbPtJl+PR4P+vz3V38z9GOox4VuAZ
   Q=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: v6ae+TdAS45AqlttK1L+/fUh7eFpq41ArRGM0X4BN9ddn15wNIksIXqFskblmGJm/Kh96+1CIx
 CvOYYTx8QpyJbDHvX8kDiy2/b7rdJpUEJPE6ARr88HB4LQpK4hRfGTn7lHqbbHoZZODEYkSwdO
 IK6qoa3w3DTXvwaUScIMWhgGPFmsUJX3Vs0EFmeuVKCImWlqgOCn52Dl0knBZQvV4t9Tbskl+r
 NQsnN0N3aYcutP2mTubtxero/VVxFoVstrqum/oiYD9c8RzCdt6NftARyZIPhmnrNQId736FiG
 gE+1PitmFm5exQsVNLUb4/cH
X-SBRS: 5.1
X-MesageID: 52894196
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:2lA926+147V4qFfNN1OSDrUDEHmTJUtcMsCJ2f8bNWPcYEJGY0x3z
 mAYXjyGb/zbZmPyLtp3a96zoEMA6sCAyYIyTgo4rSA8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j2NYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhok
 O1xsJaLSj4kYPLhu7skbhZACzNHaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFhmdh2pkfQ54yY
 eIDKjRfMy/vYCYWAVIFN5Ukjdr1jGvwJmgwRFW9+vNsvjm7IBZK+LriKt3OYfSRWN5Y2E2fo
 wru5H/lCxsXMNiezzut8X+2gOLL2yThV+o6Krq88fJ7hUyJ8UYaAhYWSFiTrOGwjwi1XNc3A
 0Ad5CcGt6U5802vCN7nUHWQomOAvxMac8pdFas98g7l4qvL4S6JC24cVDlDZdc68sgsSlQCx
 lKP2t/kGzFrmLmUUm6GsKeZqyuoPioYJnNEYjULJSMH6dT+pIA4jjrUU81uVqWyi7XdBjXY0
 z2M6i8kiN0uYdUjjvvhuwqd2nT1+8aPHlVdChjrsnyN91tBWIuvXI+R5AaEwNUfFbuWVkGTh
 S1R8ySB19ziHa1hhQTUHr5XReH3u67cWNHPqQUwRMh6rlxB71bmJNoJsW8kfC+FJ+5ZIWeBX
 aPFhe9GCHa/1lORZKlrasqaD80wxMAM/vy0C6iJMrKijnV3HTJrHR2Ch2bLhAgBc2B2yMnT3
 Kt3lu72Vh4n5VxPlmbeegvk+eZDKtoCKYbveHwG507/jer2iIGppUctbwLVM7FRAFKsiwTJ6
 ddPX/a3J+FkeLSmOEH/qNdLRXhTdCRTLc2m+qR/K7/YSiI7ST5JNhMk6e54E2CTt/8OzbmgE
 7DUchIw9WcTclWdeF3WNSg8OO2wNXu9xFpiVRER0Z+T8yFLSa6k7bsFdotxer8i9ed5yuVzQ
 eVDcMKFasmjgByek9jERZWi/oFkajqxggeCY3isbDQlJsYyTA3V4N70OADo8XBWXCawsMI/p
 Zym1x/aHsVfF1gzUp6OZaL91U61sFgchPl2AxnCLO5MdRi+64NtMSHw0KM6epleNRXZyzKG/
 A+KGhNE9/LVqoo4/YCR16CJpoukCcVkGU9eEzWJ5Lq6L3CCrGGi3ZVBQKCDejWEDDH4/6CrZ
 ON0yfDgMaJYwAYW4tQkS7sylPAw/driobNe3z9IJnSTYgT5EK5kL1mHwdJL6v9HyIhGtFbkQ
 UmI4NRbZ+mEYZu3DF4LKQM5Re2fzvVIyCLK5PE4LUimti96+L2LDRdbMxWW0XEPKbJ0NMUuw
 Ps7ud5Q4Au600J4PtGDhyFS1mKNMn1fDPl36sBEWNfm2lgx11VPQZ3AESunspiAZuJFPlQuP
 jLJ1rHJgK5Rxxaafnc+fZQXMTGxWXjaVMh28WI/
IronPort-HdrOrdr: A9a23:Sed+XqH9fmqeuynYpLqEEseALOsnbusQ8zAXPiBKJCC9vPb5qy
 nOpoV+6faQslwssR4b9uxoVJPvfZq+z+8R3WByB8bAYOCOggLBQL2KhbGI/9SKIVydygcy78
 Zdm6gVMqyMMbB55/yKnDVRxbwbsaa6GKPDv5ah8590JzsaDJ2Jd21Ce32m+ksdfnghObMJUK
 Cyy+BgvDSadXEefq2AdwM4t7iqnayzqHr+CyR2fyIa1A==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="52894196"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fWJtCZoJ3RK8BrwVuuq6W0DcE26fXCTsHbGxOQwoQbTgB7zgUH0XiomUOegXLep28Ft/uEnBAILDGCYY+jsA3b9kSV8pBfq7TSAHUR4tbb4u3Z4BIwiR1/+OdO8SkiFb2/YVEcWj4VHpj5EM8lICslQAO+FUyKsXRxYfpEeN9sdFOuktpLTPbnfISUfFCli58na7kA+ioUt8MBbIb49FNknpkB8sYRKrWCqLsLgOTFJg2LHBbFBKTzFI3gOh441nxDuWt8Z3fXE7GvgBPyc+AJJhJUL7Q2cZ4YXPD7o3l7M4g5PnFhc2diJfnqNfoCG5d9akccUyHHpbHCd5rQzPEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=s6lQCVt6prMWfBrL/PhlimqEZ3vfPawL47OaNE9m03I=;
 b=aAsvlBmhbG+9mHV3Dt8XOgq2komdnpjXLdXBrHbIQ2Or+20uXnLe244BA8UkSGTRupsGZpOz9B3WUSszuzQcV9J0MBKK+5cJgB/baYwslKOCw5vBjj07IyfMuqGeYswKuOxjHhGFz2Q6/4h6mtqynR/w1+BHWHCfa2m1RtJhEIqd0Bpeqrdv98w3az8hXd03W1QcKWadJ4xsjoBzxBwTrA9vsLoUDlwkuxpJn4ELTQ/Uh/R8jaEL929IXPePoB8adq7lnxSQSixbClOV2owkPkcWro0DfWGWAxWVMj8bSlb1yBOYI5t82hxd2Z2sYhM/zzZ5i8GXJEJMKXPVJR6B3g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=s6lQCVt6prMWfBrL/PhlimqEZ3vfPawL47OaNE9m03I=;
 b=aq5pjQM5s2Fys386chs6Cs+IkwUzU0RTX/7aEAAAxpMWWAYVNyxLYFuqXUrsPJ7zeb6N9VT/XWCkRsEP+njyfyNdK/Qu/OaPAE/ugn6JV5xNwPOQET9Y5Z2S5I5XZsGlz4nri42uvAoqQ+cVem+TRlkC6alqKdB0DirL0te/k94=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Julien
 Grall" <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
	"Anthony PERARD" <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Christian Lindig <christian.lindig@citrix.com>, David
 Scott <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 2/6] gnttab: allow setting max version per-domain
Date: Wed, 22 Sep 2021 10:21:19 +0200
Message-ID: <20210922082123.54374-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0191.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a4::16) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7b4a01ee-bfa5-49de-5113-08d97da20985
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB40151AC853AC59CF61EF84BA8FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: iSX0t8/ejFWgfv1Ko28SBs6ZcHMZjud+Ryl9xmuv/tCkU21iN30X9EGFP6d3g9OzJHIyeWBOn7Orf95fEJ/hjYlIfYwhAjXirzfmqn6/NatVWEhNU5wy5J40KDKd4EEqynY8bQ/8uXEr1M0ka95f8TReDM6JgpQwQ1RGEeBPekEBFFoyPQX7EgtvaOOrN0lZY5K0LvRRQ+KG0Q2+hLQv+TGQvZEE2GE9Vo2oKFeabEoW23f9rRNuqHVZ6/1f8wOKytMuVP0wAeyLxktaZ7HOK3m+X9QN9t9bWDt/15pdXI6NwrIEMAXdNlzhRl0yRRHNBOg8flr6w+vGYB3Do+kh74zu0mGvONYKZlj3eagHkfG2hIK5GSVuPO49pWONj4nAatHSSR6Lkv6tJF1dVm9o3If15+acAZz81sbfkL+xqi8iQ0OlF0mTtqRhNGVVm3rz4vjbR7B9K7vQdoQgrWPVmENewvmhZg3jPLYlgkvxf/d+oG47hz8mp9Uxc6GxWHje+iTTaf8e8TOMVgpFV8u6QSWstWUezqQv+WGe2k2ipxUK9Eg5ZlP1FmXVNYvVIMKNN3F14Yb5narOlbMigVZHO7ubqpqgHAuaLOJTbg/lbLggvRKzETGGqKmWGBKV8MKK43jBUPOKYBC63lAXpPxaKg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(30864003)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SXF5dDIvTG5jbERxOWh0dU9TZEJDd1pjeHJxZVVzaXdTUFFuTUFTbHVlNmlR?=
 =?utf-8?B?Y0hXaDhhcU9vbSt6VktYSlovbW9ILzV3ek14cmxOVHBhVW1rYVBoL1JmSlB3?=
 =?utf-8?B?V2Y2K1RydVhOSitMWkFwcytVa3lrTGRHKzV6SlRubDM3YlhPdmZCMnY5SXJX?=
 =?utf-8?B?bDRqMmpJaWFGMzlMdlh4cUYvZ3hsRnRidVB4RG0wdGlGQ0dlTTZ6SnVjSTFj?=
 =?utf-8?B?cE9JeGpGSkhRUDBjSXRuOUNPYmxIRTc1QVlGQmphMHFsNjd5UXZnRGFrY0la?=
 =?utf-8?B?aUNwS3BwWFV3YS9VMk5yclJHb2QrWCtiUFlwNnYrbjZ3N3EyUms2MjhOLzF6?=
 =?utf-8?B?N0hrTkVhQUp3N2g1SitZQnVFQmYrV2UrUElIVzZCbEVmREpwODExRmF1UXVO?=
 =?utf-8?B?TXJvM2d4Y0lYTlNnamQrOFJJMGpkM3NlWVhKblZJZWNlaC9GQXRWTEJ3Uzdm?=
 =?utf-8?B?d3VSeVhoNlJxczBCN25QMGYrdjltRFAzMVZOQU1BWU1qQ21hRWxiakR6UytI?=
 =?utf-8?B?Y0lUYnFUeHo5WEEzM04wb1MvK0t0S2VEUVJidDZLL1dGVlgyTHZ3Rnd4azRY?=
 =?utf-8?B?TS92K1NZR1Q0RUx3bUc2aFBMeWRCbUF5WVhJSjliYk9scEs4MmpSbVU3WGs5?=
 =?utf-8?B?T1RCdEhBdUJ6SW5HUlF4algrbUxRbWg3bWpwZjVKcGVENzBQVTV2VnNYVWZp?=
 =?utf-8?B?a0Zrc25FcjNnbVh4YXBHd0ZxWTFXclhnMGZpeVF3MC9jNUtMNW9VVXRGbWw2?=
 =?utf-8?B?THBvWlQrQmNwWjZzQVI5OUN0cjVyMG10VC81c2JkQzRqMlRnaDFwbEpYU2Jk?=
 =?utf-8?B?Mk5BMkhMWXgydWhGYkdPOFh5c1dQMnB3ZHdsbFVhQ1Q2b2ZWNzZEQUNiTEps?=
 =?utf-8?B?T0Y4WjZmVjNlRmdIRVZpZ1VvYmwwMWhpUGNmQnI4K25QVGZ6YmNXZFgrUXJF?=
 =?utf-8?B?YktGN3BnQ3FjaStsbExoeUhMRzRLZmhtRkFPZTZEV0ZIaGN2NHREQjBTajFj?=
 =?utf-8?B?Z2VUcmZQeCt5VU1iMlJhZHY2a00xaWdWcjFCRm96VDQyYVlCQVp3UjNqM0Ro?=
 =?utf-8?B?UVhKaTlPaERBbFBrOEZITUxsUkV5eDlCUWozRzhXYWRiMVBqTkV5MDV5c0V2?=
 =?utf-8?B?N1VZcWxzeWdTM3VvM3RjWFhpaE1Jc3hIdjRsWmJnRTU4ZUNaUGxxc0FuQS9N?=
 =?utf-8?B?YVhmNU9XaE9xRGdoNkluYldVckg3N0FlUCttYi9hTlBTMVZLclU5SEFXN2Fv?=
 =?utf-8?B?ZlJZbGlZSnlPN3FObGVtelBNcFdqanBXZ3BpTUxXWDU0R2JoRzl1ck1ZTmJ3?=
 =?utf-8?B?cGJINW9nZVp0d2NNNmxobHl3OUJvRVFGUFZoVDJWN2JUUXpjc09NbDI3SUJO?=
 =?utf-8?B?RDNhakdVMC8wV2hYMFBnaURLclF4MjlTNEJiTEJaK1JCcVhua3hrcThCbUZy?=
 =?utf-8?B?a2g1a3Vja2ttYnYrSHJnYTdhQ1NGK21xSEFCd1pUR003cU9WTDdGT09ER2VQ?=
 =?utf-8?B?ZUNDMVIzcVBCTVhtMFJTRm10eGxaU1ZpaXJVYkxaVWxUdFVrMVhBaFRWUU9G?=
 =?utf-8?B?aEdIaWsxaVdrYUxDYVBBUjgzUkdkUFF5UTBzSHNCSnVhZDdscEhTdy9pbFVz?=
 =?utf-8?B?cmhtM2g5SytuYjF6TlZrU0xDSnNkYklYMVd4TEpZcXN5cU10d3dSQ1RKWW5Y?=
 =?utf-8?B?cGRxcmtjNWhUNk54WitlNCtZeldlWVYxRk9zVGxJRWV5MEYxaFNZTFgyTzFV?=
 =?utf-8?Q?bE0Nd0ilAWxtPJBxUgLkGO+umE/4waBIo3spjF9?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b4a01ee-bfa5-49de-5113-08d97da20985
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:21:54.9428
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: he5AWf2yRRdGH5sjsh/G9Y5uzpLBPKuKKrw8TRzE7y7BXYekATQZ0xd9gNCBUxsy0JRvpSzF0bpPUmydZ7hjxQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Introduce a new domain create field so that toolstack can specify the
maximum grant table version usable by the domain. This is plumbed into
xl and settable by the user as max_grant_version.

Previously this was only settable on a per host basis using the
gnttab command line option.

Note the version is specified using 4 bits, which leaves room to
specify up to grant table version 14 (15 is used to signal default max
version). Given that we only have 2 grant table versions right now,
and a new version is unlikely in the near future using 4 bits seems
more than enough.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
NB: the stubdom max grant version is cloned from the domain one. Not
sure whether long term we might want to use different options for the
stubdom and the domain. In any case the attack surface will always be
max(stubdom, domain), so maybe it's just pointless to allow more fine
setting.
---
Changes since v1:
 - Introduce a grant_opts field and use the low 4 bits to specify the
   version. Remaining bits will be used for other purposes.
---
 docs/man/xl.cfg.5.pod.in             |  6 ++++++
 docs/man/xl.conf.5.pod.in            |  7 +++++++
 tools/helpers/init-xenstore-domain.c |  1 +
 tools/include/libxl.h                |  7 +++++++
 tools/libs/light/libxl_create.c      |  3 +++
 tools/libs/light/libxl_dm.c          |  1 +
 tools/libs/light/libxl_types.idl     |  1 +
 tools/ocaml/libs/xc/xenctrl.ml       |  1 +
 tools/ocaml/libs/xc/xenctrl.mli      |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c  |  7 ++++++-
 tools/xl/xl.c                        |  8 ++++++++
 tools/xl/xl.h                        |  1 +
 tools/xl/xl_parse.c                  |  9 +++++++++
 xen/arch/arm/domain_build.c          |  2 ++
 xen/arch/x86/setup.c                 |  1 +
 xen/common/domain.c                  |  3 ++-
 xen/common/grant_table.c             | 22 ++++++++++++++++++++--
 xen/include/public/domctl.h          | 12 ++++++++++--
 xen/include/xen/grant_table.h        |  5 +++--
 19 files changed, 90 insertions(+), 8 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 4b1e3028d2..a4bf2caafa 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -580,6 +580,12 @@ to have. This value controls how many pages of foreign domains can be accessed
 via the grant mechanism by this domain. The default value is settable via
 L<xl.conf(5)>.
 
+=item B<max_grant_version=NUMBER>
+
+Specify the maximum grant table version the domain is allowed to use. Current
+supported versions are 1 and 2. The default value is settable via
+L<xl.conf(5)>.
+
 =item B<nomigrate=BOOLEAN>
 
 Disable migration of this domain.  This enables certain other features
diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in
index b48e99131a..0a70698a7c 100644
--- a/docs/man/xl.conf.5.pod.in
+++ b/docs/man/xl.conf.5.pod.in
@@ -101,6 +101,13 @@ Sets the default value for the C<max_maptrack_frames> domain config value.
 Default: value of Xen command line B<gnttab_max_maptrack_frames>
 parameter (or its default value if unspecified).
 
+=item B<max_grant_version=NUMBER>
+
+Sets the default value for the C<max_grant_version> domain config value.
+
+Default: value of Xen command line B<gnttab> parameter (or its default value if
+unspecified).
+
 =item B<vif.default.script="PATH">
 
 Configures the default hotplug script used by virtual network devices.
diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 6836002f0b..7cd1aa8f7c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -88,6 +88,7 @@ static int build(xc_interface *xch)
          */
         .max_grant_frames = 4,
         .max_maptrack_frames = 128,
+        .grant_opts = 1,
     };
 
     xs_fd = open("/dev/xen/xenbus_backend", O_RDWR);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d698..7a35833312 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -502,6 +502,13 @@
  */
 #define LIBXL_HAVE_X86_MSR_RELAXED 1
 
+/*
+ * LIBXL_HAVE_MAX_GRANT_VERSION indicates libxl_domain_build_info has a
+ * max_grant_version field for setting the max grant table version per
+ * domain.
+ */
+#define LIBXL_HAVE_MAX_GRANT_VERSION 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index e356b2106d..1ee86602ae 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -606,6 +606,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
             .max_evtchn_port = b_info->event_channels,
             .max_grant_frames = b_info->max_grant_frames,
             .max_maptrack_frames = b_info->max_maptrack_frames,
+            .grant_opts = b_info->max_grant_version == -1
+                          ? XEN_DOMCTL_GRANT_version_default
+                          : b_info->max_grant_version,
             .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SHIFT),
         };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 9d93056b5c..9a8ddbe188 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -2320,6 +2320,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
 
     dm_config->b_info.max_grant_frames = guest_config->b_info.max_grant_frames;
     dm_config->b_info.max_maptrack_frames = guest_config->b_info.max_maptrack_frames;
+    dm_config->b_info.max_grant_version = guest_config->b_info.max_grant_version;
 
     dm_config->b_info.u.pv.features = "";
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff653a..37789a568c 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -518,6 +518,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 
     ("max_grant_frames",    uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_maptrack_frames", uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
+    ("max_grant_version",   integer, {'init_val': '-1'}),
     
     ("device_model_version", libxl_device_model_version),
     ("device_model_stubdomain", libxl_defbool),
diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index a5588c643f..6a8658bfec 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -83,6 +83,7 @@ type domctl_create_config =
 	max_evtchn_port: int;
 	max_grant_frames: int;
 	max_maptrack_frames: int;
+	max_grant_version: int;
 	arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 6e94940a8a..5933d32c70 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -75,6 +75,7 @@ type domctl_create_config = {
   max_evtchn_port: int;
   max_grant_frames: int;
   max_maptrack_frames: int;
+  max_grant_version: int;
   arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index ad953d36bd..1e60925069 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -188,7 +188,8 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 #define VAL_MAX_EVTCHN_PORT     Field(config, 5)
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
-#define VAL_ARCH                Field(config, 8)
+#define VAL_MAX_GRANT_VERSION   Field(config, 8)
+#define VAL_ARCH                Field(config, 9)
 
 	uint32_t domid = Int_val(wanted_domid);
 	int result;
@@ -198,6 +199,9 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 		.max_evtchn_port = Int_val(VAL_MAX_EVTCHN_PORT),
 		.max_grant_frames = Int_val(VAL_MAX_GRANT_FRAMES),
 		.max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES),
+		.grant_opts = Int_val(VAL_MAX_GRANT_VERSION) == -1
+			      ? XEN_DOMCTL_GRANT_version_default
+			      : Int_val(VAL_MAX_GRANT_VERSION),
 	};
 
 	domain_handle_of_uuid_string(cfg.handle, String_val(VAL_HANDLE));
@@ -251,6 +255,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 	}
 
 #undef VAL_ARCH
+#undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
 #undef VAL_MAX_EVTCHN_PORT
diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..0fde879cf4 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -55,6 +55,7 @@ bool progress_use_cr = 0;
 bool timestamps = 0;
 int max_grant_frames = -1;
 int max_maptrack_frames = -1;
+int max_grant_version = -1;
 libxl_domid domid_policy = INVALID_DOMID;
 
 xentoollog_level minmsglevel = minmsglevel_default;
@@ -213,6 +214,13 @@ static void parse_global_config(const char *configfile,
     else if (e != ESRCH)
         exit(1);
 
+    e = xlu_cfg_get_bounded_long (config, "max_grant_version", 0,
+                                  INT_MAX, &l, 1);
+    if (!e)
+        max_grant_version = l;
+    else if (e != ESRCH)
+        exit(1);
+
     libxl_cpu_bitmap_alloc(ctx, &global_vm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_hvm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_pv_affinity_mask, 0);
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 7e23f30192..cf12c79a9b 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -282,6 +282,7 @@ extern char *default_colo_proxy_script;
 extern char *blkdev_start;
 extern int max_grant_frames;
 extern int max_maptrack_frames;
+extern int max_grant_version;
 extern libxl_bitmap global_vm_affinity_mask;
 extern libxl_bitmap global_hvm_affinity_mask;
 extern libxl_bitmap global_pv_affinity_mask;
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd5..1206d7ea51 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1431,6 +1431,15 @@ void parse_config_data(const char *config_source,
     else
         exit(1);
 
+    e = xlu_cfg_get_bounded_long (config, "max_grant_version", 0,
+                                  INT_MAX, &l, 1);
+    if (e == ESRCH) /* not specified */
+        b_info->max_grant_version = max_grant_version;
+    else if (!e)
+        b_info->max_grant_version = l;
+    else
+        exit(1);
+
     libxl_defbool_set(&b_info->claim_mode, claim_mode);
 
     if (xlu_cfg_get_string (config, "on_poweroff", &buf, 0))
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d634c1..3beb1cbb41 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2649,6 +2649,7 @@ void __init create_domUs(void)
             .max_evtchn_port = -1,
             .max_grant_frames = -1,
             .max_maptrack_frames = -1,
+            .grant_opts = XEN_DOMCTL_GRANT_version_default,
         };
 
         if ( !dt_device_is_compatible(node, "xen,domain") )
@@ -2756,6 +2757,7 @@ void __init create_dom0(void)
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
+        .grant_opts = XEN_DOMCTL_GRANT_version_default,
     };
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b101565f14..b5b6c75447 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -750,6 +750,7 @@ static struct domain *__init create_dom0(const module_t *image,
         .max_evtchn_port = -1,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
+        .grant_opts = XEN_DOMCTL_GRANT_version_default,
         .max_vcpus = dom0_max_vcpus(),
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..6519272c47 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -669,7 +669,8 @@ struct domain *domain_create(domid_t domid,
         init_status |= INIT_evtchn;
 
         if ( (err = grant_table_init(d, config->max_grant_frames,
-                                     config->max_maptrack_frames)) != 0 )
+                                     config->max_maptrack_frames,
+                                     config->grant_opts) != 0 ) )
             goto fail;
         init_status |= INIT_gnttab;
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index fe1fc11b22..c43e9d5ee4 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -53,6 +53,7 @@ struct grant_table {
     percpu_rwlock_t       lock;
     /* Lock protecting the maptrack limit */
     spinlock_t            maptrack_lock;
+    unsigned int          max_grant_version;
     /*
      * Defaults to v1.  May be changed with GNTTABOP_set_version.  All other
      * values are invalid.
@@ -1917,11 +1918,27 @@ active_alloc_failed:
 }
 
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames)
+                     int max_maptrack_frames, unsigned int options)
 {
     struct grant_table *gt;
+    unsigned int max_grant_version = options & XEN_DOMCTL_GRANT_version_mask;
     int ret = -ENOMEM;
 
+    if ( max_grant_version == XEN_DOMCTL_GRANT_version_default )
+        max_grant_version = opt_gnttab_max_version;
+    if ( !max_grant_version )
+    {
+        dprintk(XENLOG_INFO, "Invalid grant table version 0 requested\n");
+        return -EINVAL;
+    }
+    if ( max_grant_version > opt_gnttab_max_version )
+    {
+        dprintk(XENLOG_INFO,
+                "Requested grant version (%u) greater than supported (%u)\n",
+                max_grant_version, opt_gnttab_max_version);
+        return -EINVAL;
+    }
+
     /* Default to maximum value if no value was specified */
     if ( max_grant_frames < 0 )
         max_grant_frames = opt_max_grant_frames;
@@ -1947,6 +1964,7 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     gt->gt_version = 1;
     gt->max_grant_frames = max_grant_frames;
     gt->max_maptrack_frames = max_maptrack_frames;
+    gt->max_grant_version = max_grant_version;
 
     /* Install the structure early to simplify the error path. */
     gt->domain = d;
@@ -3076,7 +3094,7 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop)
         goto out;
 
     res = -ENOSYS;
-    if ( op.version == 2 && opt_gnttab_max_version == 1 )
+    if ( op.version == 2 && gt->max_grant_version == 1 )
         goto out; /* Behave as before set_version was introduced. */
 
     res = 0;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..e2b47184a0 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -87,14 +87,22 @@ struct xen_domctl_createdomain {
     /*
      * Various domain limits, which impact the quantity of resources
      * (global mapping space, xenheap, etc) a guest may consume.  For
-     * max_grant_frames and max_maptrack_frames, < 0 means "use the
-     * default maximum value in the hypervisor".
+     * max_grant_frames, max_maptrack_frames and max_gnttab_version < 0
+     * means "use the default maximum value in the hypervisor".
      */
     uint32_t max_vcpus;
     uint32_t max_evtchn_port;
     int32_t max_grant_frames;
     int32_t max_maptrack_frames;
 
+/* Grant version, use low 4 bits. */
+#define XEN_DOMCTL_GRANT_version_mask    0xf
+#define XEN_DOMCTL_GRANT_version_default 0xf
+
+#define XEN_DOMCTLGRANT_MAX XEN_DOMCTL_GRANT_version_mask
+
+    uint32_t grant_opts;
+
     /* Per-vCPU buffer size in bytes.  0 to disable. */
     uint32_t vmtrace_size;
 
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index 41713e2726..fe6225346f 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ -36,7 +36,7 @@ extern unsigned int opt_max_grant_frames;
 
 /* Create/destroy per-domain grant table context. */
 int grant_table_init(struct domain *d, int max_grant_frames,
-                     int max_maptrack_frames);
+                     int max_maptrack_frames, unsigned int options);
 void grant_table_destroy(
     struct domain *d);
 void grant_table_init_vcpu(struct vcpu *v);
@@ -67,7 +67,8 @@ int gnttab_acquire_resource(
 
 static inline int grant_table_init(struct domain *d,
                                    int max_grant_frames,
-                                   int max_maptrack_frames)
+                                   int max_maptrack_frames,
+                                   unsigned int options)
 {
     return 0;
 }
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192238.342589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVx-0004d1-6A; Wed, 22 Sep 2021 08:22:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192238.342589; Wed, 22 Sep 2021 08:22:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVx-0004cs-2O; Wed, 22 Sep 2021 08:22:01 +0000
Received: by outflank-mailman (input) for mailman id 192238;
 Wed, 22 Sep 2021 08:22:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxVv-0004ME-Tr
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:21:59 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 65313bd1-3416-4a6e-8629-818319798ff4;
 Wed, 22 Sep 2021 08:21:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65313bd1-3416-4a6e-8629-818319798ff4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298913;
  h=from:to:cc:subject:date:message-id:
   content-transfer-encoding:mime-version;
  bh=seIBbqCeK4oo8fY2b3scK6+3FGPIRwYLW3Z4LyHTtj4=;
  b=QrfqTIximE/RIglQYLhumdSR5lP1LJ+OiW2GGZxIeqVxOquymtV5LZzX
   2vofq3OUzmWfirRf/eEczsdq/cdbhOeTCVFNI9KpNoPcHaJ+1b1YDssta
   e7v8LQCkLGaKmOPJVlYEqlk8u4kQLyPwU0TZ1aA5070FuBKvkCYaJQvat
   w=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: bR8K0e9YEc+iFEspSZ1Tta/JqIvGPQAjMQi172goDltYNV1zO85CKckGdejIjigVVNfSvrInrs
 iMX/rapohJe7lAyGvYQ0Cpug+K0rdWZUv9o6n4RZ7o2AcAZbZLPgglDQuMRY9z+pKe6sg0NM4f
 xLE+BEyib29BugnCU6g46WutWQrod9L9lBMxpbW9iSEBXNKzGvio0+PX4IypGGr613SAFI/3E3
 kd6JK6LA7McDC7zlKwx2xNM2D8ps1W4ejumixwJopXfoe95leH1lcKOnl727wIKivnNFziTxyH
 2rNOVpmtZDsFa447xSbwCwS+
X-SBRS: 5.1
X-MesageID: 53713429
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:FgLS0qhpfd7Yws9D3KnjeTlcX1618BcKZh0ujC45NGQN5FlHY01je
 htvWW+DbP2CZDH8cognOoux/UgA6pfVy9RrTgFo/io9Q3gb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y24Dja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1LsMOVYy5xBJfAic0xCBtBCGZxI7xZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9t3Z8XRqqEN
 qL1bxI+NSqbPUcMIW5NI6sgmeOxqF/hcxpx/Qf9Sa0fvDGIkV0ZPKLWGNjaYNuRXu1Og12V4
 GnB+gzRGQoGPdaSzT6E9HOEheLVmy7/HoUIG9WQyPluh1GCw30JPzcfX1C7vPqRh1a3XpRUL
 El80iYns6Ua7kGgSdj5GRqirxasvAMYWtdWO/037keK0KW8yweGBEAUQzhZctskucQqAzsw2
 Tehj97vQDBirrCRYXac7auP6yO/PzAPKm0PbjNCShEKi/HhqowuihPETv54DbW4yNbyHFnY3
 DSivCU4wbIJgqY2O76TpA6dxWj2/96QE1Bztl6/sn+ZAh1RYov6QqaBsWLh4edpAN2bV0LGs
 2YVhJ3LhAwRNq1hhBBhUc1UQurzvqbbbGWF6bJ8N8J+rGX2ohZPaagVuWsndRkzaq7obBe0O
 Be7hO9H2HNE0JJGh4dMaoStAowByaH6HLwJvdiFM4ITPvCdmOKBlRyChHJ8PUi2yyDAcollY
 P93lPpA6l5AUsxaIMKeHbt17FPS7nlWKZnvqXXH8vha+eDGOC79pUg53KumMblisfLsTPT92
 NdDLcqaoyhivBnFSnCPq+Y7dAlSRVBiXMyeg5EHJ4arf1s9cEl8WqC5/F/UU9E890ijvryTp
 S/Vt44x4AeXuEAr3i3QOyg/N+uwDcggxZ/5VAR1VWuVN7EYSd/HxI8UdoctfKlh8+pmzPVuS
 OICddnGCfNKIgkrMRxEBXUkhIA9JhmtmyyUOC+pPGo2c5J6HlSb8d74ZAr/siIJC3Pv58c5p
 rSh0CLdQIYCGFs+XJqHNqr3wgPjp2UZlcJzQ1DMfotZdnLz/dU4MCf2lPI2fZ0BcE2R2juA2
 g+KKh4Evu2R8ZQt+dzEiPnc/YekGudzBGRAGGzf4erkPCXW5DP7k4RBTPyJbXbWU2atoPeuY
 uBczvfdNvwbnQkV79ogQugzla9nvonhvb5XyAhgDU7nVVXzB+MyOGSC0OlOqrZJmu1TtzypV
 x/d4dJdI7iIZp/oSQZDOAo/Y+2f/vgIgT2Ov+8tKUD36SIrrrqKVUJeY0uFhCBHdeYnNYokx
 aEqudIM6hz5gR0va47UgidR/mWKD3oBT6R46c1KXN610lImmgNYfJjRKi7q+5XeOdxDP34jL
 iKQmKef1a9XwVDPciZrGHXAtQaHaU/iZPyeIIc+Gmm0
IronPort-HdrOrdr: A9a23:b3zhJ6zQitekjyh6jdPKKrPxyeskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjlfZq6z+8O3WBxB8bYYOCCggWVxe5ZnO3fKlHbak/DH41mpN
 hdmspFeaTN5DFB5K6QimnIcKdf/DDuytHVuQ609QYLcegFUdAE0+8vYTzrb3GeCTM2TqYRJd
 653I5qtjCgcXMYYoCSAWQEZfHKo5numIj9aRALKhY74E3W5AnYpYLSIly95FMzQjlPybAt/S
 zslBH43Lyqt7WexgXH32HewpxKkJ/Ky8dFBuaLls8JQw+cwDqAVcBEYfmvrTo1qOag5BIDl8
 TNmQ4pO4BJ53bYbgiO0G3Q8jil9Axrx27pyFeej3emi9f+XigGB81Igp8cWgfF6mI71esMnp
 5j7ia8jd56HBnAlCPy65zjTBdxjHe5pnIkjKo6k2Ffa40Dc7VcxLZvs3+9KK1wXB4S1bpXU9
 WHVKrnlbRrmBKhHjzkV1BUsZ2Rti9ZJGbCfqAA0vblpAS+0koJi3fxaaQk7wU9HakGOul5Dt
 L/Q9BVfYF1P74rhJ1GdZQ8qLOMexTwqDL3QSuvyAfcZek600ykke+D3Fxy3pDsRKA1
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53713429"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Pv7qv3PcoEO7ALpeRCQLQsCrbfndwXqaLmUr13iJncHUfGXDZ94skB4HbmDgCSNNs7Ya2LeCCK3TxOzeTc6iQAdGXDGoucDgkVLwlf1KoeOoJTrPmvy2ai16It+xYycGtPUOpTWqH1UsdHWaWMPZFvOAOZf5BigYGwT50hMXZ18qWklWSjhax8Yr7j5e/OvIVj4CnmyVU0phwV/d6fMGqvATqZLsf0VxsSwuGRbMoPqeBVK7S45zYRSYdT0ZskUgTGDav1qJOL8Pa7bY8dmr9y2evXOdbHjZKndCLdZXup7yWSTKkRW94r63qHP/8jCDZDV2XTpRIdNzem3Wy3sMdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oiIt5T8yXl8QFjL3/Lj8D33A5BGL+AQDPrB2Sf4vjAM=;
 b=B6RQjBW8EzWBi71GSmsHhMQL45/yQWIxAxU/65A8CtEG+AplAa6t+DcJgvn4aEThkxu9EUYlyR2opO9s1/T/clgaC+Bpb6ks+V/sYiV995tzQIyF7IjrSnnMgn3HdpewRuU89u7cW0Mu1nGYnJneaUqSIuwxo5rAZEHj0oIEdP3WoqAHqdDMMsd1ePf9C85bDYacGsDSiaRIY3w65UuXbgVguV0kIYBrWI6Wqr9e5VX/NYKkCiijIC60y6lc8Bx3FMgtmc3r7Q6L05qWZGQOd/zRV53A0JYJ8m6T6nJeAcsM4NM6Yqfz/6BcPwHPaLcWh4Pv8FDQs44YPxa56jxTRQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oiIt5T8yXl8QFjL3/Lj8D33A5BGL+AQDPrB2Sf4vjAM=;
 b=mAIMDMJyXRcNZl0L9u7nXMiPkCxLYSeTrCamDmHKxlztIFiMKL5Ic3DD3yB80K8A7ZwJS/43l9rhp5uUBp1SRo6miFD9CTcRchLD1mf1kZmdOg9mwT36tlcrEoo4hG1gkxsIIgZ2klUx4qUtk0Jk0aci+7Sj5yCUu2akngg1LIo=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 0/6] gnttab: add per-domain controls
Date: Wed, 22 Sep 2021 10:21:17 +0200
Message-ID: <20210922082123.54374-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0410.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::14) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3a11df1b-34ac-49ef-20ab-08d97da20266
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB40153A5DC75007CF1C5EB6538FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: b3sbx9ptMvJ51xMvcIB3C+1a4CQ3aDvwmFDj7vXUVotfldhRrr7eFVWJ+dIpHmqfxWoCHRfXzldCCMflRYcI8wk4q5r9qr//5T8VOIvqJtnzM89j8wlamhnE4WtFKrIztc/VgjJgxILibNWIK6LjgadtV6MT5qpsjtxk222xgOMRSOnPsgeXhE79bSmAlkc1PE2r4QdkTNhZC1H+uuTqPKCYlRh6mU+BQgIebPwLmwEWhnQmcqY0Vd5hlnLKMHIkcSeN9hDXky63Rhf6eupuiy5OH+F6Fajx0HwJtGmgEpbnkcallAd2gBsd45w8nGZJdcj/9g9dbkbxiXB+llEyQFup/jhf49/+hzigMtJJVAp36YXl5ugTu+Wpsh4yTjMn2aulUSzZF/cJe0C+pUiPhi7tHUP3DXLB5JevZdJ1HTPshOTduRqVSQoClpjOAp33b7nnlcZOkGRxkeZSt+QlQ9jOEbytQly03zKKss/bHzKRN4EQryvDicDAMKlAhxYATG1pYQp8h8ly+31CjfdJYqVKMHSeJ9agN9VC+wkSK5RmJmyKkFGW0OsomwfqetDh9rrraAmF3iz6yKsVVT6VvZGKsF67oTLKQU0fvoEfbXWjpqCJnJ4xWbaw6lquzCW9Nxzz5/VT9HTKpLlSvbCAUQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TndiSVBzaiszNHorbXdZdjJ1dlFNQWlFQ2tFcWVZMVFkMzA5RENZTmtoNmdH?=
 =?utf-8?B?ckdDWGFTUHRiSU40T21QcGJOdWFjTTJFajMrYUxPdWY2SVdJV1NMcTRQTWk3?=
 =?utf-8?B?K0hhQUhhSTE2bGUwaWEwSTJlRCs3OUVkTHpIUXNMZXRyT0lPOXdnNGFuTTNG?=
 =?utf-8?B?b1JycnppNDZFdDJ6WmxrVUkzRFYyb3ZUSXRQdm8wd2JDQ3VHRXVBbU5xQldZ?=
 =?utf-8?B?MWc4QkhuSFNyNWcwbW9vbE1mQWJ2NTBXUmdxRzZYbWw3cVplSzE3VHZLczl3?=
 =?utf-8?B?ZDdvSm9aanJ6L0tpdTh4L1I0aEpkbDFkbFhlcFh4b2tWaWc0bUNrR2VGN0Vl?=
 =?utf-8?B?WGZiaHZzV2hGenVjMzNxZVZDNngyM3dBcWNWckkzbVhJSlRWelpITVczWHJP?=
 =?utf-8?B?N0VHajk0RUNvVEtUd1IrdUhsUzBmZzFFbnJFSjZpYnZGR3Fic2JQbVBPbW90?=
 =?utf-8?B?SW4vTEFzWC94YnozK05KbTAvaXp5MS9USDVLRnZvcllOQzAvZnc0bi9IRTFY?=
 =?utf-8?B?WXBEVnh2UUxsaWVtWmJSRDM2RDRwTVpBUGV4RXVZYXdremhLTWxSWkdDNEM4?=
 =?utf-8?B?N0c1ZVcrT0cweUllbHU3TkszcjNKbzZDY1F5UTluZTZrNzhTQzRyTHF5RHh0?=
 =?utf-8?B?TUVaWUdwTmtmVGxxalFUemJSK0wrbFkyZXh3NEovLzd0ajRueFljRW1vb2dv?=
 =?utf-8?B?bEw0MkJMSlhDUGlDRCtwOXplZEdHc2JpZ2hOTDJSN2VQT3JrRW9xTWQ4OEtG?=
 =?utf-8?B?cm9QUGFkK05PRkhOME8xcHgyY2NaaDYxN3pTdnE0em5sM3Fqelg3dk0zY0Fj?=
 =?utf-8?B?L3JsVHFoa0EweEVJNGQrenM2VVN2NVY1Z2Yyc2R3RXV2T0RMMGVKMmk2OUJi?=
 =?utf-8?B?dUcydzB5dHlWWnlNYk9rS3o0VE5seW9IZXBJRWNPUTA3aHNRckZJSERRTWNB?=
 =?utf-8?B?R3p0MEEvc0dTWXd5Yi9iVzhHTW9ldEw0ODlMMUx3UlRaUVJCWVk0Yk1OczZM?=
 =?utf-8?B?bmxrNHlqcU8yRHZhUjg0MnJEZS8yZzJ0QklPcnpMZ1FQSXBvbERWUVFOZkYy?=
 =?utf-8?B?RG5XOTYxOTFWelMwRWxGakdOdTVzYVpPQXRSNGRWNnpHQmhsNUdCdnRlQXZj?=
 =?utf-8?B?VVdkVlFFM09uZXo2eFFucWM2eEtqMkc0SEprYUtRMzdnTmhpYml5N09TcXB2?=
 =?utf-8?B?d2lIYm52RjlCQ1RNdU5IRHhEN3NHeUI2VUNUb3M3QXBxeTBOUFVPWTZ4NUhh?=
 =?utf-8?B?YnhOVVdTSGxRMHNjTE5SRXlIRFlzbjZteVBrZE85SG0yOWNlRXFTR2FIdlNQ?=
 =?utf-8?B?L241eitVYXk5eWNoYXd1N2FvT2hpUmtNb3A5RVkvT0IrL3NxL3JjUVh5emNH?=
 =?utf-8?B?N3VWU240TjY5bUJYYVlzdHJ4alZYd25lNzRLM05HVmRLZzFEZHhmSjc5K0Fu?=
 =?utf-8?B?ckFPeUd1YlJXR0ljeStiOXBMQmlSZjBjYUQ3YllCMWp6UlFTMldTcVg4VHVE?=
 =?utf-8?B?THFPTjRIWlZBbkhEWmxYS0Q5d08wQ29Hd2xxN1h2ek9XalNTV1RXTWtRZmhp?=
 =?utf-8?B?bWhBK0VqMnI2L0F3NWxpaHZRaWhLeG45a01ncG0xaFRsWDNlOUdDZFNEbGpX?=
 =?utf-8?B?OTJocjdBcDk1dndONmhqN25MVDdOZ1ptM3FwODE3SGpEUVpaeGFDekxTRHhQ?=
 =?utf-8?B?QkhhRVhhblFLMWRzcTRSNE9hN1hFRnR0NUQyQ3BHeGNmRjVjU0ZUNUFSWkx2?=
 =?utf-8?Q?5os6t4zBVRhwLvT7kvKfgP5jlcuh+oCGKcxrPNG?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3a11df1b-34ac-49ef-20ab-08d97da20266
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:21:42.9461
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qXyq6vAuvDDaZBaAHdaYMJ0baBn18tRzlZ/oHFQlboFp5gkql0z+xevY2AjNq9h2tPJjC3EI+zszgJx0Bb0aww==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Hello,

First patch on the series is a trivial change to xenconsoled in order to
use xenforeignmemory stable library in order to map the shared console
ring instead of the unstable libxc interface. It's reviewed and ready to
go in.

Patches 2 and 3 allow setting the host wide command line `gnttab` option
on a per domain basis. That means selecting the max allowed grant table
version and whether transitive grants are allowed.

The last 3 patches attempt to implement support for creating guests
without a grant table. This requires some changes to xenstored in order
to partially support guests without a valid ring interface, as the lack
of grant table will prevent C xenstored from mapping the shared ring.
Note this is not an issue for Ocaml xenstored, as it still uses the
foreign memory interface to map the shared ring, and thus won't notice
the lack of grant table support on the domain.

Thanks, Roger.

Roger Pau Monne (6):
  tools/console: use xenforeigmemory to map console ring
  gnttab: allow setting max version per-domain
  gnttab: allow per-domain control over transitive grants
  tools/xenstored: use atexit to close interfaces
  tools/xenstored: partially handle domains without a shared ring
  gnttab: allow disabling grant table per-domain

 docs/man/xl.cfg.5.pod.in             |  12 +++
 docs/man/xl.conf.5.pod.in            |  14 +++
 tools/console/Makefile               |   4 +-
 tools/console/daemon/io.c            |  25 ++++--
 tools/helpers/init-xenstore-domain.c |   1 +
 tools/include/libxl.h                |  14 +++
 tools/libs/light/libxl_create.c      |   6 ++
 tools/libs/light/libxl_dm.c          |   2 +
 tools/libs/light/libxl_dom.c         |   2 +-
 tools/libs/light/libxl_types.idl     |   2 +
 tools/ocaml/libs/xc/xenctrl.ml       |   2 +
 tools/ocaml/libs/xc/xenctrl.mli      |   2 +
 tools/ocaml/libs/xc/xenctrl_stubs.c  |  12 ++-
 tools/xenstore/xenstored_core.h      |   2 +-
 tools/xenstore/xenstored_domain.c    |  91 ++++++++++----------
 tools/xenstore/xenstored_minios.c    |   4 +-
 tools/xl/xl.c                        |  15 ++++
 tools/xl/xl.h                        |   2 +
 tools/xl/xl_parse.c                  |  13 +++
 xen/arch/arm/domain_build.c          |   4 +
 xen/arch/x86/setup.c                 |   2 +
 xen/common/domain.c                  |   3 +-
 xen/common/grant_table.c             | 122 ++++++++++++++++++++++++++-
 xen/include/public/domctl.h          |  15 +++-
 xen/include/xen/grant_table.h        |   5 +-
 25 files changed, 310 insertions(+), 66 deletions(-)

-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192237.342578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVs-0004MR-Tw; Wed, 22 Sep 2021 08:21:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192237.342578; Wed, 22 Sep 2021 08:21:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxVs-0004MK-Qv; Wed, 22 Sep 2021 08:21:56 +0000
Received: by outflank-mailman (input) for mailman id 192237;
 Wed, 22 Sep 2021 08:21:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxVr-0004ME-4J
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:21:55 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d3f31747-cd10-4295-a5cf-4c9efeefc486;
 Wed, 22 Sep 2021 08:21:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3f31747-cd10-4295-a5cf-4c9efeefc486
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298913;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=KvlRs63Cx+Jvx4nNBqIkILrlJjUxHNtbkim4lAXRCe8=;
  b=c+gTB/FRcbo5acUYLMpatTJm5x98BGAYMSy+Q67Te4cqGfiWilAgi1Lz
   kwFHGWGW3G230pWIud+rNwUV6wY7gil6AKSm5TF/gXlTqyib5EQsWtbVz
   +pc3208rajXTXpBpi04c5cqhtrj9amN52kqmAv1xNl6pwA5B6SfoGMQOF
   E=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: IDblU9qmfXikEPp8GctaHMIy18ZtDSb5xCLFEX0CVudyjJix0RqPbadI5AgIvonUTL52uvlhl4
 a6ex4/Vfcv3bwX8e2M5zoMW94MxKbUoMJnrWiuJKSM1aj/eGQlQI5ZPLr4qeukeWTaTVlEcg/b
 2ZxHdkHhiB7nqheC9sesO2Rzmk6w0xqz7sjOWFiVMTuOEulNU46IxK/FAXAjxK0Wx3uxNVrO91
 /FBCzjb3Lzy0eqou1TpJxRy75Gn9fS0b3V5kOWgj2TK96MQsRmHiak8FHiJWJqP4+dPGvBvOXY
 8HU1BWgknW8uGpOpTPBYKEvl
X-SBRS: 5.1
X-MesageID: 53315430
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Uidazq3ONy1j7cfDBPbD5cp2kn2cJEfYwER7XKvMYLTBsI5bpzEPy
 mZKXmDTOPnYZzfyLdgiaYnk/UpXu5XWmNMyS1NtpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbRh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhvdt94
 Y9f7sSMYzwLYaLsveg2cDlHHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQDhWps3Zgm8fD2Z
 ZAhYyFMK076eSZdZVlJOYMgvriIiSyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1UgUSwt
 m/A537+ABwRKJqY0zXt2mKhgKrDkD32XKoWFaak7bh6jVuL3GsRBRYKE1yhrpGEZlWWAowFb
 RZOo2x38PZ0pBfDosTBswOQgnWIl0QOAIJsOdYYtw6t76b/wCaGCT1RJtJeU+DKpPPaVBRzi
 ATTxoO3WWQ22FGGYSnCrebP9FteLQBQdDVbNHFeFWPp9vG++Nlbs/7Zcjp0/EdZZPXOEDfsy
 nihqCEkjt3/ZuZaiv3moTgrb9+qz6UlrzLZBC2MBQpJDSsjPeZJgrBEDnCBtp59wH6xFAXpg
 ZT9s5H2ABoy4XSxqcBwaL9VQOHBCwm53M303gc0QshJG8WF0H+/Z4FAiAxDyLNSGp9cI1fBO
 RaL0SsIvcM7FCb6PMdfPtPqY+x3nPeIKDgQfq2NBja4SsMqL1HvEeAHTRP44l0BZ2B2wPljZ
 s/EIZjzZZvYYIw+pAeLqy4m+eZD7gg1xH/JRIC9yBKi0LGEY2WSR6tDO1yLBt3VJovdyOkM2
 9oAZcaM1TtFV+jyPnve/YIJdAhYJnknH5Hm7cdQc7fbcAZhHWggDd7XwK8gJNM5z/gEyL+Q8
 yHvQFJcxXr+mWbDdVeAZEd8Ze69Rp14t386Y3AhZA76x3g5bI+zx64DbJ9rL6I//eluwKcsH
 fkIcsmNGNpVTTHD92hPZJXxttU6Jh+qmRiPL2yuZz1mJ8xsQAnA+9nFeArz9XZRUnrr5JVm+
 7D5j1HVW5sOQQhmHf36UvP3wgPjp2UZlcJzQ1DMfotZdnLz/dU4MCf2lPI2fZ0BcE2R2juA2
 g+KKh4Evu2R8ZQt+dzEiK3Y/YekF+xyQhhTE2XBtOvkMCDb+iyowJNaUfbOdjfYDTum9KKnb
 ORT7vf9LPxYwwoa79siS+5mnfAk+t/ih75G1QA1TnzEYmOiBq5kPnTbj9JEsbdAx+MBtAa7M
 q5VFgK25Vld1BvZLWMs
IronPort-HdrOrdr: A9a23:5pA90KiEJv1Q1kTtcQFfJpbfs3BQX0t13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskaKdhrNhQYtKPTOWxVdASbsN0WKM+UyHJ8STzJ8+6U
 4kSdkENDSSNykKsS+Z2njALz9I+rDum8rJ9ISuv0uFDzsaEZ2Ihz0JdDpzeXcGIzWua6BJca
 Z1saF81kWdkDksH4mGL0hAe9KGi8zAlZrgbxJDLxk76DOWhTftzLLhCRCX0joXTjsKmN4ZgC
 f4uj28wp/mn+Cwyxfa2WOWx5NKmOH5wt8GIMCXkMAaJhjllw7tToV8XL+puiwzvYiUmRoXue
 iJhy1lE9V46nvXcG3wiRzx2zP42DJr0HPmwU/wuwqqneXJABYBT+ZRj4NQdRXUr2A6ustn7a
 5N12WF87JKEBLphk3Glpj1fiAvsnDxjWspkOYVgXAae5AZcqVtoYsW+14QOIscHRj99JssHI
 BVfYDhDc5tABGnhk3izyxSKITGZAV2Iv7GeDlNhiWt6UkUoJgjpHFog/D2nR87hdsAotd/lq
 L52gkBrsA7ciYsV9MOOA42e7rANoX8e2O+DIusGyWTKEh+AQO0l3fW2sR/2Aj4Qu1D8HMN8K
 6xJ2+w81RCIn7TNQ==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53315430"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZhLjhOMPDYiYOtBcyO2HEUeAYNvrcM5dhfeIUy+QLs7kaEWe52aTLcv4WaG93gCxtCIQ6StY5BuQArA8SbQ4BhUEyp9QAejSB3M9GcWXiMhwoNcki+culN/MWcxKZCqgKsDDvYuerfuji2/JWJiZXvXyRlXrsUp48JcBCJ1tQNh6AJ+wgZgUk7ONi2o0At/C/2ufiDiARIw1ta796cSd6p2XVple/2gJtiv0LnlkNEL+BowmADNKy8reIDMiRhAgLHZQlQikVmM97+DlxL60lTc16DVXT9R6qOUjhgYPCvEl/NxL3JiKwvyv5CvCMElBy/WNvVx6PGGPT5rAPsAlhg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p6U9+CcoG+mX2fECvTxGB38bv2Hcg2ZxfYzHOKtfzAE=;
 b=W6AD/rIiqS1Wdj9hwk2VvkOfrh4Vf0jCKuByWPSvg0fEhXqbwaTsO2WamWaBWnQUMqdwgX6zqx8mRV3bhfOGoYxACqfYWj13vx2Z2Z+k5QDnR3L/JQ5cMG61MexAq1y9lEpZHXU0OfIDNr5cCqo4wvVdY0yrBK0EsSCTLCBzLsvuswmrt68GS3hLsCT7H3T2vbCafpsaNHwzKSXysPMdJ5flMgO+oj2SA30BkHS2ORFyaGGFrmGJYSojSXbX2ot3nsAv/cTEFUt9xkW5moRVBHR9krxtz3+qF3m86+sX3YSM7WSw4pYsPi9b61F/Joi2ndUhuf3o4eYH+dEx1j4yxg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p6U9+CcoG+mX2fECvTxGB38bv2Hcg2ZxfYzHOKtfzAE=;
 b=QDWXLIz5g7Bjj82bvzImjkxuV0oXOJZn8vdK7+W3op157Ds2cpG73W/oymE7nh0Ju/5E38Zl5pjajwnZ418hh947+T/z234qYkfjnAbBaAmjx95Ew0UAB8fjWb2asvN0HGxJteFqSHoErU4jCe9y9AB8xN+fioQ5gVS9a4FCCyY=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 1/6] tools/console: use xenforeigmemory to map console ring
Date: Wed, 22 Sep 2021 10:21:18 +0200
Message-ID: <20210922082123.54374-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0042.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:61::30) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f5fcaec7-237b-49fd-71f0-08d97da205ff
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB4015C8DF3B853672F2A472448FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: TqDweZTz9JMoOtoEKoQPx6jqGFJUZJwquY9EgHbXTho4ZHCMjM18dJikjti3MYe8no3nHpDit77I7Ey399vRgY3p5LRQFO3oB0g9sc2lOUqjpF/HEcQ+zir12bkaOXHgvV4h0jqSJHnL4ENldXxWcqX91q4Yg5gIERvlgjs0OF3Oxz/J7nG+wak9plZLQsNAcBgg3ECwr9BSCq7domv1ENabkgHrV9Zth9BDTjYYi+GoywV+jjYLPKFhz6ohPU3bONLxILsjsPzSHZi234zOTiwZeiUu/ywMrNuucmFn6t+3F2TsrW8G7YgOcLEIcJDQXFXIP0psDVJVdrSk8qdhLvCfg6umAzGIkA8LO3A+yjvVydmEyW3pe+hvim8EPuUfqf04uX7RvRv9cHI34VtPUyytqtEiV4z1sR7g3Ez44buEOtpPgK7vIHp7DD0+aukEt5jToGhZ082+Exlz4IsLNjcgTtnKTJr9obHi04sGqn8GKetPY8rcmLHYuZ5aNyofdhy+xv3bxBrDThpogKSFdI001oj2tY7AWF+aOXqacOO0Dp7O+3onS3TkY6OK+bc42/R/yUIrjAwF4JpBTLioFjFI8BzFpBcHT8AKzczaEZKwG7nJVUE2KdJYu1FtpsTNPaU8h6Og5Lve36s08JJ0EQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bElBQStZRGRBSWhZWlVUZTF3NTZZQzlSUklIVmpVRCswSE9KTVlxRmJXaUdh?=
 =?utf-8?B?QjkwWmZHYk1sczYya1MvQjlXWmx2VHkwSlorTTlPaWhJYUJtb1ZXdUZzK3JK?=
 =?utf-8?B?NzNuWEIwRkdyaVBjbEJFaUpVVVB4bWtHVjNsUGg3T1RYWjVab3pPdjVZODdq?=
 =?utf-8?B?MjVhMVk4akpOSUwzQXBKMVMxazNKajdxcVFycmE5bHA3R0FsYnF4aXY4d2FT?=
 =?utf-8?B?RmxZVnM4K3N1aGNtZ2ljZnhEdlVUUXFVOXFMdCtjejY5cGNZb1lUM3RTNDFK?=
 =?utf-8?B?LzB5bUhSL1paa3FJRWFidGhDNXVwcHB4S1kxYk1IdFhBejM4WkNjTzZaUkxq?=
 =?utf-8?B?VDBnMnluU3dDVWpnTXlWTVEwZGdlOUtGREZCSWxpUkhGRFJOb2JIQWJjK1N6?=
 =?utf-8?B?UHA5ZUcwQnFtMlRuckhEOWdNN3RtSndGbGswS0dpVDZPdWhwekM4V3A1emRZ?=
 =?utf-8?B?Wkx4UHNJZ2RnVitsVWpUdG5lcnRYYlVGSFF3WE9YaTZhMDdrR0J4SkFkMGxY?=
 =?utf-8?B?M25pRW9mNXlNUXFnZk5qTm1vSFlYT3h3Q1ZNaytSM3JYRlJwM01YMXZ6dW1r?=
 =?utf-8?B?UFh5VGZTTVlTd1VuMWVFOW9rZ3k0eHpJSXRVeHczeDNra2hlNHVUU1NteUFC?=
 =?utf-8?B?c0xHcy9xdHhqMHorcVZSV25FYkFFMGQ4V2xXelpaM0svZk05OXFyQjV0bEti?=
 =?utf-8?B?bVkxTXhoMWJOU2cwbi9XWjNKc200Z1FzUzRRM3NGbmVLMElxQkRodFl4YW90?=
 =?utf-8?B?eXQvQjViL3lXS0IyRjVEOGFMREJVRjE1UmNwOVdpSTZyZGF2RmxCRVB1RlY1?=
 =?utf-8?B?MlI4YVNQdEpHZE9TWDJlVzJaNHd5UGRIeEs0YTNROFRUVXdVRWtSVmpUcjRW?=
 =?utf-8?B?Rks2bXhSanMyOHZ1Z0RjR0c4NTRoRVRmRnowRlpoKzRqRE5qbldKTGNxY3Vw?=
 =?utf-8?B?WDczaHVFSkZhVWFPVW9Dd1J3V25VbXdNZkRTZU5XODVnU0FYSEU5UjNPUDht?=
 =?utf-8?B?TkhROXdBNEw1UDI1cDVJaCtLWTVDM2M1d01WaXpDb0RoNlNkRXlHQ1FVUk9J?=
 =?utf-8?B?UEhUZUNnVU5PbTZKMjI0bHhqbUZHeGNhS1pRNW5MTHFTUUZmcExzOXFRTStn?=
 =?utf-8?B?WjBLdXB6bSsyelkxcDZqZHdXSVlRaVdjeXBZeGRoQkRKRU1vR0ErNWEzekZN?=
 =?utf-8?B?UUxFV3hCWTFUVys1V1VoM2ZESmY4Rm5JRHFIVnVNZ0pjMG1lckZvQ1huNklr?=
 =?utf-8?B?R3ArM3g2c1FPSzljNUVwclc1OC93ZjVzbTN3L1VoM1FCbTdtSFVaR2F5SklJ?=
 =?utf-8?B?TllHQ3NRZEd1MTdXbTIyRFZFcFBxMGEvZmpiRFVacDBhcmxiYWRtUUNaTm1u?=
 =?utf-8?B?WkdzZlBwUStsT0JhbEZGWVp6RTNHQ28wT0xld0RZZkp6cUkweWRZbHhJeE14?=
 =?utf-8?B?UTQ0SWFTbE0yUDdxMzZmcFlLZy9yNldacFg4dnlQa0R4ekJ4ZVZJb1pUaFNL?=
 =?utf-8?B?NWxBRWlhMklMV0lmUHR5b3A0UTBJZGtiV28vM01NZnB2UFozVm5JbEVTTW1F?=
 =?utf-8?B?UC81S3oxNUFVS3RpOG5JR3lvTEx4SEJMSVNPZlBSclY3N2JwRjhqdG1Wdzhi?=
 =?utf-8?B?QUFVb2cxUUlZRmZwTzBsZElmK0Y4d1pjaVFkODVyYVptQUN5Z3QzczVvUHpo?=
 =?utf-8?B?dXp2bnE5U254V2daU2hvdHBMQkRZeXhmbFQ1bXJqcVhkNDJkY2R0VVVlZVBY?=
 =?utf-8?Q?Xrg4guZefYCRcFiSznnWCSQC9fRHLGu6NDCKDR6?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f5fcaec7-237b-49fd-71f0-08d97da205ff
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:21:49.0452
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +dMTkd1udyg6R+er5SXbtfgoY0j7YbR5kiIxnCiCuvwJaetRkpy0LTNTiNynxyS4HC8hqc/DiE0Y2+N+R1umYw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

This patch replaces the usage of xc_map_foreign_range with
xenforeignmemory_map from the stable xenforeignmemory library. Note
there are still other uses of libxc functions which prevents removing
the dependency.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Ian Jackson <iwj@xenproject.org>
---
 tools/console/Makefile    |  4 ++--
 tools/console/daemon/io.c | 25 ++++++++++++++++++++-----
 2 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/tools/console/Makefile b/tools/console/Makefile
index 84796eac8f..3f4cddab03 100644
--- a/tools/console/Makefile
+++ b/tools/console/Makefile
@@ -29,9 +29,9 @@ clean:
 distclean: clean
 
 daemon/main.o: daemon/_paths.h
-daemon/io.o: CFLAGS += $(CFLAGS_libxenevtchn) $(CFLAGS_libxengnttab) $(CONSOLE_CFLAGS-y)
+daemon/io.o: CFLAGS += $(CFLAGS_libxenevtchn) $(CFLAGS_libxengnttab) $(CFLAGS_libxenforeignmemory) $(CONSOLE_CFLAGS-y)
 xenconsoled: $(patsubst %.c,%.o,$(wildcard daemon/*.c))
-	$(CC) $(LDFLAGS) $^ -o $@ $(LDLIBS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_xenconsoled) $(APPEND_LDFLAGS)
+	$(CC) $(LDFLAGS) $^ -o $@ $(LDLIBS) $(LDLIBS_libxenevtchn) $(LDLIBS_libxengnttab) $(LDLIBS_libxenforeignmemory) $(LDLIBS_xenconsoled) $(APPEND_LDFLAGS)
 
 client/main.o: client/_paths.h
 xenconsole: $(patsubst %.c,%.o,$(wildcard client/*.c))
diff --git a/tools/console/daemon/io.c b/tools/console/daemon/io.c
index 200b575d76..682c1f4008 100644
--- a/tools/console/daemon/io.c
+++ b/tools/console/daemon/io.c
@@ -22,6 +22,7 @@
 #include "utils.h"
 #include "io.h"
 #include <xenevtchn.h>
+#include <xenforeignmemory.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 #include <xen/io/console.h>
@@ -73,6 +74,7 @@ static int log_time_guest_needts = 1;
 static int log_hv_fd = -1;
 
 static xengnttab_handle *xgt_handle = NULL;
+static xenforeignmemory_handle *xfm_handle;
 
 static struct pollfd  *fds;
 static unsigned int current_array_size;
@@ -675,7 +677,7 @@ static void console_unmap_interface(struct console *con)
 	if (xgt_handle && con->ring_ref == -1)
 		xengnttab_unmap(xgt_handle, con->interface, 1);
 	else
-		munmap(con->interface, XC_PAGE_SIZE);
+		xenforeignmemory_unmap(xfm_handle, con->interface, 1);
 	con->interface = NULL;
 	con->ring_ref = -1;
 }
@@ -722,11 +724,12 @@ static int console_create_ring(struct console *con)
 		con->ring_ref = -1;
 	}
 	if (!con->interface) {
+		xen_pfn_t pfn = ring_ref;
+
 		/* Fall back to xc_map_foreign_range */
-		con->interface = xc_map_foreign_range(
-			xc, dom->domid, XC_PAGE_SIZE,
-			PROT_READ|PROT_WRITE,
-			(unsigned long)ring_ref);
+		con->interface = xenforeignmemory_map(
+			xfm_handle, dom->domid,	PROT_READ|PROT_WRITE, 1,
+			&pfn, NULL);
 		if (con->interface == NULL) {
 			err = EINVAL;
 			goto out;
@@ -1341,6 +1344,14 @@ void handle_io(void)
 		      errno, strerror(errno));
 	}
 
+	xfm_handle = xenforeignmemory_open(NULL, 0);
+	if (xfm_handle == NULL) {
+		dolog(LOG_ERR,
+		      "Failed to open xen foreign memory handle: %d (%s)",
+		      errno, strerror(errno));
+		goto out;
+	}
+
 	enum_domains();
 
 	for (;;) {
@@ -1462,6 +1473,10 @@ void handle_io(void)
 		xengnttab_close(xgt_handle);
 		xgt_handle = NULL;
 	}
+	if (xfm_handle != NULL) {
+		xenforeignmemory_close(xfm_handle);
+		xfm_handle = NULL;
+	}
 	log_hv_evtchn = -1;
 }
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192241.342611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxW4-0005Hq-VS; Wed, 22 Sep 2021 08:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192241.342611; Wed, 22 Sep 2021 08:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxW4-0005Hh-Q9; Wed, 22 Sep 2021 08:22:08 +0000
Received: by outflank-mailman (input) for mailman id 192241;
 Wed, 22 Sep 2021 08:22:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxW3-0005Fo-H6
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:22:07 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2c15c576-1b7e-11ec-b962-12813bfff9fa;
 Wed, 22 Sep 2021 08:22:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c15c576-1b7e-11ec-b962-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298926;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=jl8Da4tdx3a57Qk/VIGdkSe4VlRYgk2bnETxrUhv7Ps=;
  b=Ydg7ksKQWc3u7cch7O0kHt981kk1A2eLXc6VAbtf8ynaZc/MagHTcgQD
   onT/wZuaQz+Sls1KTk2NIGfYLkDPKd9pYFx1/+zs1+hDAv78yZX4FC4Q9
   WwvZU+Ss4Mie41sEhOg4l0KddyYXqkI8DMHZP6nWsENP1b9dWQ9bekO/J
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: XFSfHGlrHY4bDoejpoN6A72iiDwFrWClWCBZ4OdBkv8r5pezev8lt8kCNR9IxlxCIF3tTSCXgW
 0Rj6A9J7Jm+yLSCvFFhXiJJfUrefIvIKOTIZssBNQQQ7tGvuQmkEjuaoaqWinGP51wY5ZZKvaU
 E1rIhT4NTlg3aKefdW7YaNxDytvm6TPBmdOc869FJqt54AelI0fnkdFbaVF5S0f0ngn/Nie7D2
 afGDxq5IwcUJ/8adbsJ4r+tLXX5VSURfp6Bp/bCTccTKqKgd3mKlU1f//uCByctcn1UVD/1vOB
 GTXCBIOf2b+yLYe5zZExwTeP
X-SBRS: 5.1
X-MesageID: 52894204
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:hauNmqJJUDqaEwGiFE+R6ZIlxSXFcZb7ZxGr2PjKsXjdYENS0GMFz
 TBMXGHUbP7eNmqkL4h3aI6ypE9S6JeEmNYwSlRlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wbZj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2EpIB22
 IxClabrUChyGerisuomCCVXRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2Xv48JgG1o2qiiG974R
 5UnTQpGay3uIBJrPE8IVc0/gOqR0yyXnzpw9wvO+PtfD3Lo5A5+yr/2K/LOZ8eHA85Smy6wp
 H/C/mn/KgEXMpqY0zXt2m2orv/Cm2X8Qo16PK218LtmjUOewkQXCQYKTh2rrP+hkEm8VtlDb
 UsO9UIGrLcw3FymSMHnWB+1q2LCuQQTM+e8CMVjtlvLkPCNpV/EWC5UFVatdeDKquc7fmwtx
 gaGv+myHBM0l/q7Y3ymrYub+Gba1TcuEYMSWcMVZVJbuIO68N9v1kunosVLS/Hu34ascd3k6
 3XT9nFv2e9L5SIe//jjpTj6bySQSo8lp+Lfzj7eWH6spih9bZSsD2BDwQmGtascRGp1o1/og
 ZTlpyR8xLtVZX1uvHbUKAnoIF1Oz6zeWAAweXY1Q/EcG82FohZPh7y8BQ1DyLpBaJ5YKVcFn
 3M/SSsOvcQObRNGnIdcYp6rCtRC8JUM4e/ND6iOBvIXO8AZXFbeoElGOB7Bt0iwwRNEufxuZ
 v+mnTOEUC9y5VJPl2HtGY/wENYDm0gD+I8kbcqql0j4ieXOOiH9pHVsGALmU93VJZis+W392
 91eK9GL21NYVuj/aTPQ6okdMRYBKn1TOHw8g5U/mjerLlU0FWc/JeXWxL99KYVpk74MzrXD/
 22nW18ew1363CWVJQKPY3FlSbXuQZcg8i5rYX1yZQ6liyo5fIKizKYDbJ9rL7Mpw/NukKxvR
 P4fdsTeXvkWEmbb+y4QZIXWpZB5cEj5nhqHOieoOWBtf5NpSwHT1MXjew/jqHsHAiat7JNsq
 Ly8zALLB5EEQl06XsrRbfuuyXK3vGQcx70uDxeZfIELdRy1ooZwKiH3gvsmGO02KE3OlmmAy
 gKbIRYEvu2R8YU7x8bE2PKfpICzHuogQkcDRzvH7ay7PDXx93a4xdMSS/6BeD3QWT+m+KimY
 ukJnfjwPOdewQRPuot4Vb1q0bg/953koLoDllZoG3DCblKKDLJ8IybZgZkT5/MVnrIJ6xGrX
 k+v+8VBPeTbMczoJ1cdOQ45Y7nRzvoTgDTTsaw4LUiSCPWbJ1ZbvZG+5yWxtRE=
IronPort-HdrOrdr: A9a23:+A2wQqoFWjSaHv7dPBewOMwaV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBfhHPlOkPUs1NaZLXTbUQ6TQr2KgrGSpQEIdxeOlNK1kJ
 0QCJSWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHtXgslbnnlE422gYypLrWd9dP8E/M
 323Ls5m9PsQwVbUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZszU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDi1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyUfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW82/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 guMCjl3ocVTbqmVQGdgoE2q+bcGkjbXy32DHTqg/blkAS/xxtCvgwlLM92pAZIyHtycegD2w
 xoWp4Y4I2mdfVmH56VMt1xN/dfOla9Mi4kD1jiVGgPbJt3Q04li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="52894204"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AdbyU0MLp3zipdElBXPzgnhh3vHdRo4xl/AEO8OELYlNS/vYbiEcmFCzAVZqRk826vBd1ji3OZcbQL52k2ZI1OgujG9VooVhaEvP1pZbrngAOqJx0wPyZeFCdvZQ4WFACYcEXYvZfsoEQ5gsrMWnUo3PRT2QdTEWVkK8NshssX6ARcEvHCnOznZIY9iX7SKo5bpIMJGt7dxcvqfSxRbS0kvDZRKiOAkdrrMy/7EVSzi7uZ4xs+/eLdczgvFfl+orEIknGpUppuAPwMEek3XZbHpetqHgNEgvc5CKjfHhj0dja3LT7Q0VzopYOdycQZAqZyJwInsCFok/UU9fqv0jEQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cD1jNa1dkKGTraDhEkBrz9VgImKuoXQDqGBPFO2wT9s=;
 b=atl4Pw5LlfP8hZEjnz2dk2DPpuLtPAhtBwhHAV/EFOL8+NMf3vT6N05j5Sd9Q0F+E1SF0mWXAePPcRCSxgUhWUacOEps6IW/xWfGPk8FYXfh/FFFC8Dr8aM53s/n5+UGyhlVPU9Kiaxz7TkCRpjwSxaom7JL94O4yKsOMV0edM9GPqGz6flTPevje9NxUA5Bl211sOpPITxAcrItGpFV55HL2+EZaJsuP191Hkqf0SLaxMXpV7voZJT6nPbe7/Lobmjh4GqujEcLBPdkkGuFynccbU3fo9KoVbB9F3Af4ZOGX51wq26rbV/okdOOBLTe86K/n1Tg2Z4cQDCOXbz1fg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cD1jNa1dkKGTraDhEkBrz9VgImKuoXQDqGBPFO2wT9s=;
 b=wR8JCmAKn9vUj5yvepkwH+sQuUVKSjBg2R3LuJOhCr0f/uGD25DsIWO5ZzyLYtNsPiKtVD15+JhKJsOES/pHmbzGpi46QaW7wggH2psnxclAzoD4pFUAR6IwQ8rX4TijVcHJ1fKF4lDQHd9auqWaZ6zxkS1M9Vegwug5jGJ+nIE=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>, Julien Grall
	<julien@xen.org>
Subject: [PATCH v2 4/6] tools/xenstored: use atexit to close interfaces
Date: Wed, 22 Sep 2021 10:21:21 +0200
Message-ID: <20210922082123.54374-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0338.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::19) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 03569a38-9dee-4d0e-000d-08d97da20e86
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB40152857C183A409F8FBAE7C8FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 85vh3Jizza0/DKULkJZcbNfALAEoACDwEUv/YFuvMgoAtgycNk4AOrYAisHuDsDJ99M63PoBj3WdykTOP8giDA0GxlMLRWmv0hf3OZDkpBeU+zzC9oJ4wqH9Sc5kJ82Vqfi6nrP3ZuqPusdYX0vT+lEtDVotimt5GIxsMJ5qDUxOkg2HAPaY5hOHaHOIIxsV3NShHTwwV9XbdKKAn9WakvH/Hf0L3m5jkNNBB3TtwD4JibJiZludo+QnBOJQVu5swx2j7nX7ZfiQILrWnoXjj1bXzQP221B/4Lo4poCN4ImKy9c5IgnEZDPvDqlwgQKSopT0Ob7vuGjtXDOKFKCXauqV60e/HutpZkzRTgeCKVxsn6yrhue48pGrmVCpt7FqPfBK/dnwoixoOjHhInHIbZNjlAlzJfy4nRm1qYDcNw/EhHqPGalFUwbLKUbs7AhwtI8JR9+8ZI8MTWE8hW6WqeRtyf/yhpkmC7+MdXbNegAnE+8mGioSlJKrzqzvdpnoVKqHLDzQpqKhvoM1UOSVlOOpp9RKGcYq5NL9Rv8W5RiLOrGb8w38wHNoP50buvZL3BC3l1jFXyxpb5N2yw8lSFEz5yt6XCcaaN5K+9ZjDegVddOjrpxiwHk2UQ2dObPWJqdn2g5shWPSeSymQ2WE+w==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZFo1bjFoVVFLZlZ6YWIzTk9Fd3FwMWcrNy9OcUU1dEtYNGdvNzVESTZSQVRQ?=
 =?utf-8?B?UmZHTUV2VWxHY3lyZnFobW14b3dVcDhPeTAzdE1HM2NYa1dzN1ZiU1ZNSWxD?=
 =?utf-8?B?c0JtYkRYLy8yeXdXOWpqV0o4bWlHMFYxVVVLV0ZqeXNQZGgwTHlVT3RPTVBQ?=
 =?utf-8?B?bjVQVWt6VWpBeUVtUFE2cmJvdEVzTThMa3lYU1l6UlRQYk5ZOXltVFVNaHNR?=
 =?utf-8?B?NnZ3UGV1ZlRkZk1HdU42V3Nza1ZINFpybUF6Ty9tVzJndXBpdnlMaDZDRzN0?=
 =?utf-8?B?SFZDVExwVjcwVVhUZGN0OU41RkcySW5xNkFBblprN2NXNjZRTGVsRXpUZm0v?=
 =?utf-8?B?b1orM2ltYjNtMTgwN2dTT2w0U1lBWjZjazQ2enVKdWdpVDlMalVwSjcrRUht?=
 =?utf-8?B?cVdhMHRucG5MT21saS8xcUE4NU1SRnhOOXc5TjY3Y1ZxVUV3OXBmZk9nVjB0?=
 =?utf-8?B?V2c0ZmswUDFDb3ZNRFB6YkJvUjJiaE42Nk9EcWJYQXNhc3lPUlV4MkFVSGxK?=
 =?utf-8?B?SWtaN2Z2RSsyaEZCTHBIc0ZvU2JFNU5vRW1qL2tndnVSb3hLV0Z6MnBoWThL?=
 =?utf-8?B?cTBBVlA2cXdSNVQrQzVjbnVIVW5vdE1RM0grWjFETDFCcS9DVlpOVzUzUVZ0?=
 =?utf-8?B?S2dSWkhFSXNUcnVQeldyY1dtbWRGWkhrNzJSUkZRTVJWL05xNG5DTFl3YjJx?=
 =?utf-8?B?L2V2ek9maEc4U3Zvb2FiZHJ1VjZtZTUyRm1xZC9NM0RUSnFRSlJPY3JLVjEv?=
 =?utf-8?B?Um5wdDIvR1pBOFc2TnVUSGdFaTU4ZFNLa0M2VGl3WmlkbkpGM2loSVBFeER1?=
 =?utf-8?B?MlJIN1B2aWtES2Z0UFRuR04zYUpwb1ZIYWNJMjBQaURTU0k0WGJNZFZidzhK?=
 =?utf-8?B?WjdFajVyeVlTK25SbXBvLzZMRUxPRWExQ0tRTUxMcytNbVlEOW9LcHhhSUdP?=
 =?utf-8?B?RDV6MlNodlNrQ0tsMVFnWE9LMmtUS2RMaTk0UVkyR2k1RG80MDVKRlh3Qk5Z?=
 =?utf-8?B?WFV5YzhxWUpPUVBSMVRIZW11RVFZVUx3Y09mOEdCTFhiYlNqYjlhSnUwS1pC?=
 =?utf-8?B?VTVnQXpJMjZINFpFWnN6SGlja0hwOWtHTTdmVWdUUno4czFzWnVKOE1TYmJk?=
 =?utf-8?B?ZXVhVE5nQ0RkdWJHazFLaDZTWkt4c2N5SElWREN4QVFDODIzdTBiK0huRWRF?=
 =?utf-8?B?cm5WWFhrWFFaNSsvdlVvYm55TFNlYWtYMER4RCttVW5uWXl4MDhJTzZVaFhT?=
 =?utf-8?B?elV3a0xJM3MzaGRlc3ZzSHpTV1d4Y0FaT3FtenlUNGRyL2NscDdyWFFPYTBz?=
 =?utf-8?B?dnlFNjd6RlYwSVJzQklsdWpLSUhlOUVrNGpkUTdEL21jRlNIVUxNY1RJTnJE?=
 =?utf-8?B?NGN2MmovUmFiT3d4VytSL0x1U2Y4aGl3dVJFUUY4TkNpV3h0bTRXZklOSmxX?=
 =?utf-8?B?YzNNZGI5a1FIRmVvZW5GOGtqUEFtZWNYb2JQK1NiblhsNVpJbndsYUIzVVFJ?=
 =?utf-8?B?aEpGRE9KbjRKVlpOc1R0WGhKcURoMFdlSFhkeG1VTmlLZnFuWlpJTUp1ZUJt?=
 =?utf-8?B?ZS8wRkFSUmNhb0JqNUJJVCtUZmdOYVk0R0x4WDdZUU9FS3RTYUtxL2pHNWZu?=
 =?utf-8?B?VzZ2VWlCc3ZIR2EyUlJVL1hSLzBHd2o3SWV5bG1XdXE0RUZ4YnZHUWZUakNK?=
 =?utf-8?B?R3ZRUDE3TGpIa0dvb2dIdWc3MXZYblRLQ29aczZlQ3hyRG9wb2U1SCtlaEFN?=
 =?utf-8?Q?0HymaWAKcVxDEy2wieogcmGEKibiP/24WPCvXAb?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 03569a38-9dee-4d0e-000d-08d97da20e86
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:22:03.2607
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zIVgAK/7kr0uxyHLN53ht7WX4pQTz7c8gEQmpxDi2AY1v75F8w31TfCO0UZLritnjnHeUtxnjG11sWNy+mi59A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Exploiting the talloc clean up routines to close the Xen interfaces
is cumbersome, specially when atexit can be used to the same effect.

Convert xc and gnttab to use atexit which allows to drop one
indirection from the storing variables. While there also close the
event channel interface.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Fix minios xenstored build.
 - Also close event channel handle
---
 tools/xenstore/xenstored_core.h   |  2 +-
 tools/xenstore/xenstored_domain.c | 61 ++++++++++++++-----------------
 tools/xenstore/xenstored_minios.c |  4 +-
 3 files changed, 31 insertions(+), 36 deletions(-)

diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h
index 258f6ff382..a813bc5a8c 100644
--- a/tools/xenstore/xenstored_core.h
+++ b/tools/xenstore/xenstored_core.h
@@ -266,7 +266,7 @@ void init_pipe(int reopen_log_pipe[2]);
 #ifndef NO_SOCKETS
 extern const struct interface_funcs socket_funcs;
 #endif
-extern xengnttab_handle **xgt_handle;
+extern xengnttab_handle *xgt_handle;
 
 int remember_string(struct hashtable *hash, const char *str);
 
diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
index 47e9107c14..9fb78d5772 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -35,8 +35,8 @@
 #include <xenctrl.h>
 #include <xen/grant_table.h>
 
-static xc_interface **xc_handle;
-xengnttab_handle **xgt_handle;
+static xc_interface *xc_handle;
+xengnttab_handle *xgt_handle;
 static evtchn_port_t virq_port;
 
 xenevtchn_handle *xce_handle = NULL;
@@ -198,14 +198,14 @@ static const struct interface_funcs domain_funcs = {
 
 static void *map_interface(domid_t domid)
 {
-	return xengnttab_map_grant_ref(*xgt_handle, domid,
+	return xengnttab_map_grant_ref(xgt_handle, domid,
 				       GNTTAB_RESERVED_XENSTORE,
 				       PROT_READ|PROT_WRITE);
 }
 
 static void unmap_interface(void *interface)
 {
-	xengnttab_unmap(*xgt_handle, interface, 1);
+	xengnttab_unmap(xgt_handle, interface, 1);
 }
 
 static int destroy_domain(void *_domain)
@@ -240,7 +240,7 @@ static int destroy_domain(void *_domain)
 
 static bool get_domain_info(unsigned int domid, xc_dominfo_t *dominfo)
 {
-	return xc_domain_getinfo(*xc_handle, domid, 1, dominfo) == 1 &&
+	return xc_domain_getinfo(xc_handle, domid, 1, dominfo) == 1 &&
 	       dominfo->domid == domid;
 }
 
@@ -648,18 +648,6 @@ int do_reset_watches(struct connection *conn, struct buffered_data *in)
 	return 0;
 }
 
-static int close_xc_handle(void *_handle)
-{
-	xc_interface_close(*(xc_interface**)_handle);
-	return 0;
-}
-
-static int close_xgt_handle(void *_handle)
-{
-	xengnttab_close(*(xengnttab_handle **)_handle);
-	return 0;
-}
-
 /* Returns the implicit path of a connection (only domains have this) */
 const char *get_implicit_path(const struct connection *conn)
 {
@@ -737,35 +725,42 @@ void dom0_init(void)
 	xenevtchn_notify(xce_handle, dom0->port);
 }
 
+void interface_cleanup(void)
+{
+	if (xc_handle) {
+		xc_interface_close(xc_handle);
+		xc_handle = NULL;
+	}
+	if (xgt_handle) {
+		xengnttab_close(xgt_handle);
+		xgt_handle = NULL;
+	}
+	if (xce_handle) {
+		xenevtchn_close(xce_handle);
+		xce_handle = NULL;
+	}
+}
+
 void domain_init(int evtfd)
 {
-	int rc;
+	int rc = atexit(interface_cleanup);
 
-	xc_handle = talloc(talloc_autofree_context(), xc_interface*);
-	if (!xc_handle)
-		barf_perror("Failed to allocate domain handle");
+	if (rc)
+		barf_perror("Unable to register cleanup handler");
 
-	*xc_handle = xc_interface_open(0,0,0);
-	if (!*xc_handle)
+	xc_handle = xc_interface_open(0,0,0);
+	if (!xc_handle)
 		barf_perror("Failed to open connection to hypervisor");
 
-	talloc_set_destructor(xc_handle, close_xc_handle);
-
-	xgt_handle = talloc(talloc_autofree_context(), xengnttab_handle*);
+	xgt_handle = xengnttab_open(NULL, 0);
 	if (!xgt_handle)
-		barf_perror("Failed to allocate domain gnttab handle");
-
-	*xgt_handle = xengnttab_open(NULL, 0);
-	if (*xgt_handle == NULL)
 		barf_perror("Failed to open connection to gnttab");
 
 	/*
 	 * Allow max number of domains for mappings. We allow one grant per
 	 * domain so the theoretical maximum is DOMID_FIRST_RESERVED.
 	 */
-	xengnttab_set_max_grants(*xgt_handle, DOMID_FIRST_RESERVED);
-
-	talloc_set_destructor(xgt_handle, close_xgt_handle);
+	xengnttab_set_max_grants(xgt_handle, DOMID_FIRST_RESERVED);
 
 	if (evtfd < 0)
 		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
diff --git a/tools/xenstore/xenstored_minios.c b/tools/xenstore/xenstored_minios.c
index c94493e52a..9b050c7e02 100644
--- a/tools/xenstore/xenstored_minios.c
+++ b/tools/xenstore/xenstored_minios.c
@@ -49,12 +49,12 @@ evtchn_port_t xenbus_evtchn(void)
 
 void *xenbus_map(void)
 {
-	return xengnttab_map_grant_ref(*xgt_handle, xenbus_master_domid(),
+	return xengnttab_map_grant_ref(xgt_handle, xenbus_master_domid(),
 			GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE);
 }
 
 void unmap_xenbus(void *interface)
 {
-	xengnttab_unmap(*xgt_handle, interface, 1);
+	xengnttab_unmap(xgt_handle, interface, 1);
 }
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192243.342622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxW9-0005ei-9p; Wed, 22 Sep 2021 08:22:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192243.342622; Wed, 22 Sep 2021 08:22:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxW9-0005eR-4V; Wed, 22 Sep 2021 08:22:13 +0000
Received: by outflank-mailman (input) for mailman id 192243;
 Wed, 22 Sep 2021 08:22:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxW8-0005Fo-2t
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:22:12 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2ebd5d20-1b7e-11ec-b962-12813bfff9fa;
 Wed, 22 Sep 2021 08:22:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ebd5d20-1b7e-11ec-b962-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298930;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=ElWsU4ziXSiPMiGxXcaqusDkCKlRzfSDrCEyvckfErY=;
  b=VbBpVbnNzs19xm7HwCZ/RyzaTscS+lBtQvUmqP78O8bwQsGMCTWU2pl/
   hwRiZ80g75jIWqubPa3cjrUIOQEr7D70JDHib6WapVyCy+IkOfdPt37rd
   0PhMGFBe+c9i4QZumJFgLDaO9FYgChHCFlMy7x47cKKO2kZ8YshxOwTnU
   A=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: uDlN0FMrWxkQnIxn6+J++S9bhU8sC2M7mS2WehoftiQEH0+rdN+CPOAHN6r4MNPV4IatFOWEcz
 wq3oBquWAwJsK6nyjDgVx7bazsmeTGc76BmUgH2wmnJh6l4Gq06nXbNsfz7wnF3AJ+63uX7B9P
 U8PVSVz9sq39D0iEL48o5DyJQCtXU1frn87g7LobXc1nex2yViQbsXOTDDe5mLABEaloRH3Ye2
 MxzjTX99QkLnvl+MxkvxlMjFMC3VrJI4KGCXIRUPPChsXbWpkpgOhl17JQFx89miCRgMwyer+F
 0H6mI3U+7UVgSGSjqL/Fbmxi
X-SBRS: 5.1
X-MesageID: 52894211
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:q5+Xm6tv8luKKsDQIdxnSBZTQOfnVG5ZMUV32f8akzHdYApBsoF/q
 tZmKW7SP/3ZZzfyfookPovi/R9V75HUzt43HFE4qywwFywX+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524LhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Nplj5buV10xb5D1ots4Uz1bLT53AKF25+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25oeTKqEP
 pNxhTxHNkT7bwceJXMrBrkOvsyilmL5UhxdpwfAzUYwyzeKl1EguFT3C/LXdcaNXt59hVuDq
 yTN+GGRKhMHMN2SzxKV/3TqgfXA9QvkXKoCGbv+8eRl6HWDy2pWBBAIWF+TpfiillX4S99ZM
 1YT+Cclse417kPDZtjiXTWorXicpBkeVtFMVeog52mwJrH8uljDQDJeF3gYNYJg5JReqSEWO
 kGhpdm3G2Z34YCsc0249pOXkyy7BCQlMjpXDcMbdjfp8+UPsalq0EmVF4Y9TPbs5jHmMWqvm
 GHR9UDSk51W1JRShvvhpTgrlhrx/sChc+Ij2unAsotJBCtCbYi5bsSD7VHB5J6sx67IEwHc4
 BDodyWYhd3i7K1hdgTWG43h/5nzvp5p1QEwZnY1RPEcG8yFoSLLQGypyGgWyL1V3iM4RNMUS
 BWL5VM5CGBv0IuCMvYsPtPZ5zUC5qn8D9X1Ps3pgi51SsEpLmevpXg2DWbJhjyFuBV8wMkXZ
 MbAGe7xXClyNEiS5GfvLwvr+eRwnX5WKKK6bc2T8ilLJpLENSbJFu9aYAXRBg37hYvdyDjoH
 x9kH5Li4z1UUfHkYzmR9ogWLFsQKmM8C4ywoMtSHtNv6CI/cI34I/OOk74nZaJ/mKFZyrXB8
 n2nAxcKw1vjn3zXbw6Nbyk7Or/oWJ9+q1M9PDAtYgn0iyRyP9734fdNbYYzcJkm6Pdnkax+Q
 c4adpjSGf9IUDnGpWgQNMGvsIx4eR23rguSJC75MiMndpttSlWRqN/pdwfi7gcUCS+zuZdsq
 rGszFqDE5ECWx5jHIDdb/f2lwG9un0UmeRTWUrUI4YMJBWwodYycyGo1605OcABLxnH1wC27
 QfODEdKv/TJrq807MLN2fKOobC2HrYsBUFdBWTas+q7bHGI4mq5zIZce++UZjSBBnjs8aCva
 OgJnfHxNPoLwARDv4ZmSus5yKs/453kpqNAzxQiF3LONgz5BrRlK3iA/M9OqqwSmeMJ5VroA
 hqCqotAJLGEGML5C1pAdgMqY9OK2ewQhjSPv+8+J1/35XMv8beKOamI08JgVMCJwGNJDb4Y
IronPort-HdrOrdr: A9a23:w50CJaBJtCr0lzPlHeg9sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6LS90c67MA/hHP9OkPQs1NKZMjUO11HYSr2KgbGSoQEIXheOjdK1tp
 0QApSWaueAdGSS5PySiGLTc6dCsam6GeKT9J3jJh9WPH9XgspbnmBE42igYyhLrF4sP+tHKH
 PQ3LsLmxOQPVAsKuirDHgMWObO4/XNiZLdeBYDQzoq8hOHgz+E4KPzV0Hw5GZQbxp/hZMZtU
 TVmQ3w4auu99m91x/nzmfWq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczPgBkF5MWUrHo6mt
 jFpBkte+x19nPqZ2mw5SDg3gHxuQxeo0PK+Bu9uz/OsMb5TDU1B45qnoRCaCbU7EImoZVVzL
 9L93jxjesWMTrw2ADGo/TYXRBjkUS55VA4l/QIsnBZWYwCLJdMsI0k+l9PGptoJlO61GkeKp
 gtMCjg3ocSTbvDBEqp+lWHgebcEEjbJy32AXTr4aeuon9rdHMQ9TpQ+CVQpAZHyHqRI6M0k9
 gsCZ4Y5o2mePVmG56VNN1xMvdfNVa9NC4kSFjiWWgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="52894211"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kT91xiZCymbi/a9eAObbML8TTh2rbWpJraf7GPxFkP7iRma1uLVMZm9WjovCipIeMRq5riIrDwo1f3fMSVtaU5VYgymtEE3YYlQAUJaktOOgsPJ8MZqImACm6tMkA6wBhr0ANZw/Ek67TGdR8hkIavRgmE+fEbvQY6rDj44vUWvLcKXHTpcvZacPlQyWniORf7oYU4VI+ukYJtKfsYNzzUR8gyMZRZhuAXYu8Koml9Bg53NpU2R0RUAAbdP82NE4cBtDVFKJ5+FPqkGtJ7zwm6EG05B+Hkg0A4StKB2ahNSxaA5bY2xs0BxQrpHqe1hu87u6EWVFtGi1knjmn6+Vlw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JJ6k9ObCJZQ1HpQusAHVsZSz5HvaRNVf12lIJWSFRDo=;
 b=bpQsg8o/HPh+yIeApBcW+jvcIjrW9n6iOF3BLnWGBG7g1o/iI+C4uwdZBGzFT5mQldmHqU8bFbuZKP7jYMxV7iHYAYm7HWGF9OEThjKEn619rX2lRMviKvpDonKoPj89xt0W3KVwZuZL4uYhqefDAL8/aYMytNvM6ff9CKXmBAJZljzW/WOGNt35QZRibX/8+qlSKkJnbIITTEfKkpsp8JvCahLm+dOqDmTS8iCmnW8+lqZ61v5JZ1v8sugRVOANJll89DVZtSRWlkBeqyudPo7xHAHsZpTDxfyAOpNQiLbupbLbQ4wCISTnS+TukuRfDFd17whN4LwCwohd1qziCg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JJ6k9ObCJZQ1HpQusAHVsZSz5HvaRNVf12lIJWSFRDo=;
 b=lQWMqYxbpJduJTmEUEOVfXegibdEMvJ5DoR9g3nqbGHbA5tTy+RbDT9URClKXR5vdGE9B/snE+paOPBucJyrqC/vqwb3IERAtWjzEvK3jp+gVf0GiOK/EAh167rzT6dv4D/bvWq9Xtt1WB99akO2YN3u74uUOt7kzwtcq73hrQQ=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>, Julien Grall
	<julien@xen.org>
Subject: [PATCH v2 5/6] tools/xenstored: partially handle domains without a shared ring
Date: Wed, 22 Sep 2021 10:21:22 +0200
Message-ID: <20210922082123.54374-6-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO2P265CA0091.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:8::31) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f82e8c16-cf70-4f15-cb8d-08d97da2114f
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB4015529B4ECFEAD1C7476D838FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: igH50E/luNphf9V4ruhaktmmNQDXPeRdax/TSCxF/Y1Rf3nkCjgohDXioeahUV6kvOgfSmx3cRvA1iYeV2ESeuSyz7P2iyvAzT9GYYnvgWTELX4qyQD+zUnKCGV6l4zhCipI6lCzMCEHbET8EgQAgRVtxyKbpFH1y1dTg8z6I4vjI8QzU3Lph0YpAwIgPy2rfiCL91rwffm2YzKOfrDmXBDne6CuRs81CxROG+cZ9an50IKP/svtfd0m4UjWkPn4v5FW2XYqgjanF1Odo7BxIX84a4m/EvlPQ3h+XZAKWnkr2CZjGc+6A/KefwE9n7Xb/QhNKuYfRGuuayKSA7j6aAFPWSi3t8cDVX5jpRGn7GrnxJCnhPXvq0f7YGpv09lq8JfW2ulNQSeFKPdmSMYm6YNRO1Vo6PMR2HHMk7Jkx2H2nj26KrwAFX6Y0NgdH8EwN08aOgfiaTDmxkIMMmrXXuPOm0r6tRd/8sALQnBHf2yjOJkSeax2c0JZtXePrJYlP4wKvkxIbzuK/9zwVxFPDoz+nLdJXohdDiQL42JMdAue7to6h5DLjRP/22RzbwX2KTbaDeWEYn7MZ3w68bfYddnFvu4AXtzb51qcyYd0792UiDi0H02W3QTLLhIVumAnaOBqSxUwlaCL/UwXEjocZw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MEZJdUgyWVhxQ3g2ZVQzMElnNlM3V2VPaXF0YWdJbk1wQ0ZvTHJKSStoRXpl?=
 =?utf-8?B?ZlprR0JmbC85S25EaFBMSDZ3OWJjby9POEpXRzJkQkRxdDNIdFovMUg4cWt4?=
 =?utf-8?B?SVkyT21yZXREdmYwRG0zUjZjai9oOW5FVG9zZUczK0pWdnROK3YxSmc2N1Vj?=
 =?utf-8?B?M1hTV1kxZ2RiUW12QTNOditVUlZoQ0lOQU92OFNQTEI1WGNMUVFCWXhUU1ZT?=
 =?utf-8?B?UHF2eVowMFZEQTZSVTNISE1hbUNaTmZ2Y1QxYzRpcWdWd3hSUGlibWtZdmpk?=
 =?utf-8?B?ZVVrQlE4TlFib1Y4RjNnWFUxWlYyckZUdnVjNlZZODBCVmx3LzJ0VEJiZlFh?=
 =?utf-8?B?bGc5ekpPSXNiY3ZRL0h4cGhydUVZTi81VHFGOFNYcEhVOERhTkxxNS9zdWNY?=
 =?utf-8?B?ZkFSVVJCMkp4Ly9WM1NhYjFkRUUxUGphSVpKTk5HWVZCZ0l0YmhMeDZIY0pF?=
 =?utf-8?B?bkVVR21UMmNkK01ZT2Y1NVhwK1Z4RmhzYTJnTlo2MkNzUEpuM1JTaURXNXNU?=
 =?utf-8?B?T0l6cXE5Zkg0Mjl5Z3lnVTNEVVBwQ2x6OUlNRUlNOUU5ZUlmSEpRRjB3UjJQ?=
 =?utf-8?B?dm1GTkNqTjdYNmZnNnJ5ZkpCN2N6R2tCdnRmRTY0QytDdGxmQlhKU1E4ZjhS?=
 =?utf-8?B?cW1OWldQN2tjQ2w4Wm9hSVU1S0JWOFdMeDBYa0FRVDdTeUoycjVSTGZ5RVBF?=
 =?utf-8?B?V3BLUDdCeTM4dWpxbVFtQUg1MzV2L1pUeEU1VzFGbVdNT1lrQThYMERqWlEx?=
 =?utf-8?B?UlNmdjYvSFNkTWhvN0QreTFpYjlrZDMycEdrbFBnZXl6aFo0Q1hvMG1PWDA4?=
 =?utf-8?B?YVBzTHlwTjkrRXRDVWttcExSTFB3bmRoOWIzakxUd2hjMU4zNHFjSE54SEl4?=
 =?utf-8?B?Zk1STDg2QmFjS0ZSOG1wM2hkYzZKWVBHUXRsRGo0cGpaUEU0NXBrOWRsQm5P?=
 =?utf-8?B?TE1WNlVCUVVGS2xBSkRyUDBNSHpxY1gwdGJHRlZ1MGc1SHBEWHltVkVVd3Z4?=
 =?utf-8?B?d1N0aDlBWW1RblJ2OVZZQ3JtVVhBQ3FyU1BWUEphUXljSFM0d0ZpdHNJczA5?=
 =?utf-8?B?ZElOTmZYVjhhUVVybHVUMHdWdjBrL1pZUlVJYXBMME5rbSt6ejJRV0hjb1lH?=
 =?utf-8?B?UFJabHVQbUN1d2czRjVHZ0c1RWFFcWY1YkdwQUFGS2Y0bWlaazlveTFTd3Nj?=
 =?utf-8?B?N1NCdjZZL2wzOXhISmIwV2J3emlnN29yVzk5TUtma3l0Z3N5Qm1SaGJ2NDJF?=
 =?utf-8?B?ZEpoYjNMd3hrcXhtU3NUVVRIY25BK2xMMWozeGpQUG5rR2p4ODhCN0JnK0sx?=
 =?utf-8?B?bDdHeDRCb0NCK01Nc0MwMEJjMGxGNHVIQkplanZ5dUIzWHVVRWZ1L3lWb1Jp?=
 =?utf-8?B?YmhPMVlLVDA0VzlldENSQWdtY2xaNnpTOEhLaERrUlM3MXpPdWkxeGo2ZDVz?=
 =?utf-8?B?OVdrYkxOTFhwY3lkMU1lcE1UVUh1c01uWU5RdFJzc0xkVnJTTlhOSGxOMDZh?=
 =?utf-8?B?VXY0dEF4STgyNml6RFJhWjJwMW9tdnF3aXlWVU1pR2M3R3hOK0FBdU00Zktt?=
 =?utf-8?B?Y3EvcDIrNitUdDE2bFVCZEZTV1YzeDdjM2MrTjFtbkE5WmV5aUdOV2gxcVNL?=
 =?utf-8?B?L2U0YlBBMnJhUElPUmpVV0ZRVTFDaS8xL2pwUVRjNFJIWDc2b3pMU3NzbUh4?=
 =?utf-8?B?a25jblRidnVPSzVGdlI1OHBSSVp2WFpqczYxeDcrREtSdHNhOTNPZDkwVkxt?=
 =?utf-8?Q?flKobiLpyFgOZUcjOZ7YjvSHw4+iWmquSo94dNo?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f82e8c16-cf70-4f15-cb8d-08d97da2114f
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:22:08.0302
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jxwx+aTF73ovA/gYHZFT40uDNvvWyza1h+VCtGI90bpEYFFwJSO5V2eoOrp/lGK3/oaOBamNBIB/hA626cmPJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Failure to map the shared ring and thus establish a xenstore
connection with a domain shouldn't prevent the "@introduceDomain"
watch from firing, likewise with "@releaseDomain".

In order to handle such events properly xenstored should keep track of
the domains even if the shared communication ring cannot be mapped.
This was partially the case due to the restore mode, which needs to
handle domains that have been destroyed between the save and restore
period. This patch extends on the previous limited support of
temporary adding a domain without a valid interface ring, and modifies
check_domains to keep domains without an interface on the list.

Handling domains without a valid shared ring is required in order to
support domain without a grant table, since the lack of grant table
will prevent the mapping of the xenstore ring grant reference.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
oxenstored will need a similar treatment once grant mapping is used
there. For the time being it still works correctly because it uses
foreign memory to map the shared ring, and that will work in the
absence of grant tables on the domain.
---
Changes since v1:
 - New in this version.
---
 tools/xenstore/xenstored_domain.c | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
index 9fb78d5772..150c6f082e 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
 	struct xenstore_domain_interface *intf = conn->domain->interface;
 	XENSTORE_RING_IDX cons, prod;
 
+	if (!intf) {
+		errno = ENODEV;
+		return -1;
+	}
+
 	/* Must read indexes once, and before anything else, and verified. */
 	cons = intf->rsp_cons;
 	prod = intf->rsp_prod;
@@ -149,6 +154,11 @@ static int readchn(struct connection *conn, void *data, unsigned int len)
 	struct xenstore_domain_interface *intf = conn->domain->interface;
 	XENSTORE_RING_IDX cons, prod;
 
+	if (!intf) {
+		errno = ENODEV;
+		return -1;
+	}
+
 	/* Must read indexes once, and before anything else, and verified. */
 	cons = intf->req_cons;
 	prod = intf->req_prod;
@@ -176,6 +186,9 @@ static bool domain_can_write(struct connection *conn)
 {
 	struct xenstore_domain_interface *intf = conn->domain->interface;
 
+	if (!intf)
+		return false;
+
 	return ((intf->rsp_prod - intf->rsp_cons) != XENSTORE_RING_SIZE);
 }
 
@@ -183,7 +196,8 @@ static bool domain_can_read(struct connection *conn)
 {
 	struct xenstore_domain_interface *intf = conn->domain->interface;
 
-	if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
+	if ((domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0) ||
+	    !intf)
 		return false;
 
 	return (intf->req_cons != intf->req_prod);
@@ -268,14 +282,7 @@ void check_domains(void)
 				domain->shutdown = true;
 				notify = 1;
 			}
-			/*
-			 * On Restore, we may have been unable to remap the
-			 * interface and the port. As we don't know whether
-			 * this was because of a dying domain, we need to
-			 * check if the interface and port are still valid.
-			 */
-			if (!dominfo.dying && domain->port &&
-			    domain->interface)
+			if (!dominfo.dying)
 				continue;
 		}
 		if (domain->conn) {
@@ -450,8 +457,6 @@ static struct domain *introduce_domain(const void *ctx,
 	if (!domain->introduced) {
 		interface = is_master_domain ? xenbus_map()
 					     : map_interface(domid);
-		if (!interface && !restore)
-			return NULL;
 		if (new_domain(domain, port, restore)) {
 			rc = errno;
 			if (interface) {
@@ -505,7 +510,8 @@ int do_introduce(struct connection *conn, struct buffered_data *in)
 	if (!domain)
 		return errno;
 
-	domain_conn_reset(domain);
+	if (domain->interface)
+		domain_conn_reset(domain);
 
 	send_ack(conn, XS_INTRODUCE);
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192248.342633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxWJ-0006RI-Oq; Wed, 22 Sep 2021 08:22:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192248.342633; Wed, 22 Sep 2021 08:22:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxWJ-0006R9-Kk; Wed, 22 Sep 2021 08:22:23 +0000
Received: by outflank-mailman (input) for mailman id 192248;
 Wed, 22 Sep 2021 08:22:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxWH-0005Fo-TC
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:22:21 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 31895ee6-1b7e-11ec-b962-12813bfff9fa;
 Wed, 22 Sep 2021 08:22:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31895ee6-1b7e-11ec-b962-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298935;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=zAhiWJIs0wbl7XNVNT3P6InRVdgLDNm/3Htj620jk5E=;
  b=S4GPl0FEwGviTfW762IMygOPd+/uhC2iRHReN3S8kAZEUtmKVDhghhjF
   sMUHrPAp1XoSgQc8SGafJG6P164GeTSZ2ivrjqr+JFV/MHn+ufjIh7leY
   hFJkmnrqvL9DT4oHXkZ+vegiPCmtV8088oHOR33WeAsJLZomrZ/GF+G8X
   A=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: jZCjbM+N1sOj+WiRfQBGmrBCA4CGnbNwg35d1f+DKcKjXsfQlg8mowpP6A2UmtW7T6tCxabsfZ
 5maHkjYWsiusJRqEmWOdvfUPw34twYeohmxYQoNie0c+UfUPo+hFY6VIy+vRP4mUKydZY1X32N
 zaaPP260vrHlKYXrceDjA6nQ8x2Vz2oNuHImx+QL4MRnhHBT4YgbnkpCeqwtqgobZXtctFmeHB
 y+yOSdYro/fBTXVONtYBd2wF14gONmkuOSYQek40ZYhktbS3bHyzIgihzeTtXMVXaNRTwSmnpV
 S4UhMCMCOJPhfuFOZ3b0HCZc
X-SBRS: 5.1
X-MesageID: 53713461
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:h/Tv9auyTG9w8oMe8Op+7HtAdufnVLtZMUV32f8akzHdYApBsoF/q
 tZmKW+Ga/+KazT3fdkkb4W//RkO7MDdxoRqSAZt/i4zQioT+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524LhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Nplj5W1bQwgDqz2w6cDcztKVDh/PYZrweqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sRRK2CO
 ZNxhTxHVTOaURETBEUrIq0jheiwpybCSzcJtwfAzUYwyzeKl1EguFT3C/LXZ9iLSMN9jkue4
 GXc8AzRGQoGPdaSzT6E9HOEheLVmy7/HoUIG9WQ9PFwh0aI7ncOExBQXly+ydG7gEOjX9NUK
 2QP5zEj66M18SSDXtT7GhG1vnOAlhodQMZLVf037hmXzajZ6BrfAXILJhZAd94OpMIwXSYt1
 FKCg5XuHzMHmKKRYWKQ8PGTtzzaBMQOBTZcP2leF1JDuoS95tFo5v7Scjp9OJS8i/n+EAHx+
 WvJsggclfIKjM4g+pzuqDgrnAmQSoj1oh8dv1uMBDv1sF8hPeZJdKTztgOKtq8owJKxCwDb5
 SlawZD2APUmUMnV/BFhVtnhC11ACxytCzTanVcnNJ0o7T3FF5WLLN0IvW0WyKuENK85ld7Vj
 K374lg5CHx7ZiLCgUpLj2WZUZ9C8EQYPY65Ps04l/IXCnSLSONiwByCmGbKhzy9+KTTrU3PE
 cjCKpv9ZZrrIY9m0CC3V48g7FPf/QhnnTm7bcmil3yPiOPCDFbIGeZtGAbfNYgRsfLbyDg5B
 v4CbqNmPT0EC7agCsQWmKZORW03wY8TX8Gu95MPKb7dfWKL2ggJUpfs/F/oQKQ894x9nebU5
 HCtHEhezVv0n3rcLguWLHtkbdvSsVxX9xrX5AQgYgSl3WYNe4Gq4PtNfpc7Z+B/pudi0eR1X
 78OfMDZWqZDTTHO+jI8a5jhrdM9KET31FzWZyf1MiIie5NARhDS/oO2dAXY6yRTXDG8stEzo
 uP821qDE4YDXQlrEO3fdOmrkwGqpXEYle8rBxnIL9BfdV/C6o9vLyCt3PY7L9tVcUfIxyeA1
 hbQChAd/LGfr4gw+djPpKaFs4b2TLcuQhsERzHWtO/kOzPb82yvxZ57fNyJJT2NBnnp/KiCZ
 PlOy62uOvMwg1sX4ZF3FKxmzPxi6oK39aNa1AltAF7Cc0+vVuF7OnCD0MRC6v9Ny7tetVfkU
 06D4IAHa7CAOcejG18NPgs1KO+E0KhMyDXV6P00JmT85TN2o+XbARkDYUHUhXwPNqZxPaMk3
 fwl6Zwf5AGIgxY3NsqL03JP/GOWI31cC6gqu/n22mMwZtbHHr2aXaHhNw==
IronPort-HdrOrdr: A9a23:wr3VAKEqqBUDSQO8pLqFDZHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HlBEDmewKnyXcV2/hrAV7GZmfbUQSTXedfBOfZsl/d8k7Fh5FgPM
 VbAtFD4bTLZDAQ56aKg3jHLz9J+qj6zEnCv5a4854Cd3AOV0k2hD0JdzpzX3cGDzVuNN4cLt
 6x98BHrz2vdTA+adm6PGAMW6zmq8fQnJzrTBYaD1p/gTP+wA+A2frfKVy1zx0eWzRAzfML9n
 XEqRXw4uGGv+ugwhHR+mfP59B9mcfnyPFEGMuQ4/JlYAnEu0KNXsBMSreCtDc6rKWG70srqs
 DFp1MaM8F6+xrqDxeIiCqo/zOl/Ccl6nfkx1Pdq2Dku9bFSDUzDNcErZ5FczPCgnBQ8u1U4e
 Zu5Sa0ppBXBRTPkGDW/N7TTSxnkUKyvD4LjfMTtXpCSoETAYUh4rD3xHklUKvoIRiKq7zOSI
 JVfYPhDbdtAB+nhknizyhSKIfGZAV3Iv+EKnJyzfB9nQIm2EyQYiMjtb4id0w7heIAoq9/lp
 D525RT5cBzp/AtHNFA7Z86MLOK40z2MFvx2TGpUBza/J9uAQO5l3ew2sRz2N2X
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53713461"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CBeKqpKOkoDnnTv5ogE1Ik94K9FODESOmkdu/7i3cVnxJdyI2drJYrnNLJMwgqZatF3KxqkV/iPfm1j6twba9MEyuVD+6/7dKTurumodLosMbEp6KjEjNUfJGbhxnrtnmskhdNal0wWa307GOWoVKgK0MlLHXDL7tusN7LuaVyimCB5a1a5UDHw+D/osy42EXYarbHzoMvnkyK0WDh+ZWBAF+vDFPnZJ/Csya8Zn8nCjdkZ2iwCvRVCpSWG4Tpgrr3IUsJKZyhF3P76Pp5srUrgLtHhDrP/Pyhjvk1hU3idvY+5MdUCCt6BRmg8Vm4fEgtgyvpTv5HmYQaApcd//Vg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9tcyJZDVToONgtALKgRS3g/dMzuoFmclxk5371g7gig=;
 b=K39HAv+SkLpXc5vLGaOpkIfWMFxDA20beiCPlkEAy4hyHUz6/ajz14PIuUvkDUEs3yDzarRlCowrcDYrePsIHTA3T9VF90SHK8I9PwPe8i40GRBwvdTjLPGBFiCxxONV3U4HFpp+vbTiPl6Jt1dsohkbajXjBGCHGcdHwNU9f8wH6fOHkAjLf/9wAypSAShUANu7WSFAhSA8RfYlQGnH++bKt2H2JDih6KrCekXkPXTEGghBBpFkq4yMkzFPF5GcS0VRqAzw+aJTZoLeQ2iQYqxfnH8J8wkzAJu/nlXHUGcW2H4UAaL2mIjZgZV8QF3MQn4vCweAPJOWvfrE3yCSWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9tcyJZDVToONgtALKgRS3g/dMzuoFmclxk5371g7gig=;
 b=lpZQakppQw7z34vxRmWYjqq+CDq+ASwGaH/53/YJ1WJezdIY0E5SqkKtNqavmwnEhpS1ljDMnI2XzbsoDeQVhACZCVo9NUd/OfwrCwjdw9CfU01rbCPavC4yDt0yeq1ox8gwzQ/Jx6G1OJ1IogYfuADduJ/1asR+VTqIfBs5BJM=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 6/6] gnttab: allow disabling grant table per-domain
Date: Wed, 22 Sep 2021 10:21:23 +0200
Message-ID: <20210922082123.54374-7-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0110.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:192::7) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ba7b8016-b3dd-4ea7-4b30-08d97da213f4
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB4015FEC15D83049385100DC38FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: V5NPVDihWrMAPy8NDznUvuL396Fo+ZUrMqRTuVjMam7KW52N+KgTcxzFrVLev5CAuJvUIyPg820H/yyYxepJ3voDwjauX8yHX0YjMM+PTuuU8AAPM40NWw5EZ5kq10N7T1KgjpSVzdawIcRWCbKCkmkvUc1VwumvqNH/ZNZdrzjkwi0xmZrpjShb1FDLf4ubyr16EGlGOWUYNv1hoAgdfgrb4qeXYcykr7vJpMqXv4FAA+PVLFVgn/EIy5lGvIgwq7xGz9Bsvs0gQMZ6U9aMPCn49L+ZmPGtm3yTDNrnvHzzlO30L19S52aDTN3Dfw+wIgnSriHEdKNYqhhkMDEHb9aXrx+j7bGRlIMDRJvpzZ6vM/Sw0civgFnwnRVIEtTGkTN1VpzaXD3VvETnxxT7DzjwwgS2N/jL8vLuUSZaPVXmMQcNz/ltydnMqDUXExX30xtZmV1qx6/zPBLgARQ9XeIytQfuTjz98qWJigtE8u4pbCYW0ZX3ArHP9ju4uY1HE+obfmLCN+Z4POHlRklcgTvi5klhtsjb7AytqH6vxEMkm7XErkExxEOwiYuWkNOOoILRb3BY3U0gDP1XXd5g6j5zqU71i7gP95iN+48+rF1CE7WhBXaMOx5dsonTp2FwqZsLg9+6SRtcIRiTdnZteA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q3gwMElBWVBrMmxDaWNvS0xjc01JeHRzQUNFWDhZeVYvTWRVSFErcDNBZWhK?=
 =?utf-8?B?MitHb2QycCtNV0lBTk5kbmtRaktSQjFlU1E5a2haUHVhS2gxR1Bma0sxRVYz?=
 =?utf-8?B?SVFmQk1hcmQyakg3V0diVHBsV0hwVVBUYy81bnljejF6SEt6STZRK3dMd1di?=
 =?utf-8?B?Z0d2U2ZsUXRKcnRwUDFJWUluTWNjYzlCUWVFdnhLZVM3Y0FBdUR1OUtoN2N5?=
 =?utf-8?B?RW9wY3gxUGNUMmFRdmgzNzh3UEJrRGNqN0ZaSnZyYjI4b0h0RGwrdFhKRklO?=
 =?utf-8?B?N3NFeXk5dmhCZnc2QjBQM2xsUTRFMHpCeGNUQ0hBckc1UTc5c3ozWmVwSFlr?=
 =?utf-8?B?NHkraVB4alM4Wk5HVm4vYnVxL2Fxa1dnazRoTmZyeTRHYUlxTU5iVnBIcUtT?=
 =?utf-8?B?YXRlUmcwVjFlazBpZi85UE16Ui9ML1M2WU1FWlgwT2wrMWFqMHVhUTRjdUhz?=
 =?utf-8?B?V3VYM1RyQ1hRYkhxV2Nncm1oZEdBZ2U1ZWI5MHJpUE1YODdTaHIwUlU5ZWk2?=
 =?utf-8?B?bS9ieHpjVEJ5alE4OTZaOGx2VkdoMzQ5cm1SYUJpalZBZ0V6WlBnTHFhcWdT?=
 =?utf-8?B?RTQ0dW16b2RnTHU0N0xTZnAzWi9sLysrbFVGWFJtbUJwVWN3Rkw5QnZJb0hJ?=
 =?utf-8?B?K2wya2JqU28xSlA2ZEErbFRtL04ydmtra09qMTc0QVRPbXNoTStqYWdKNTVw?=
 =?utf-8?B?ZkxJckRMaEt5d2t6dU50ajV6aEVmSkZuamFZRGhkM2gyMkJyaVRJUG01a3gz?=
 =?utf-8?B?NHRPTnk3MHFnTWw1MnYxU2xEQUtJcCsxTjZhU0paL2l5RnhuZ2dMUFZYaFBx?=
 =?utf-8?B?ZkdJR0laenNpNjF2RnRYaTAzR1BEbGxKejhxNnl0Z0U5UTFVSEYzVENVNksy?=
 =?utf-8?B?dHlnbFQvRmpEK0cxdzU5MXFwYSt1SmoxUFBCQ2pFOFRKNGZtYVF0ZUhOOHhK?=
 =?utf-8?B?bWFZSlNURTBFTG1xV2VhdHppclRZUXhxUkEzOEN5UEQwYkUwN1ZUZnBZN3dZ?=
 =?utf-8?B?TldLYVRhWkZxeC94OTF2TmdySURuTERhTjhab1J6OEdzVm9YK2hYZVB3WnJP?=
 =?utf-8?B?dDlVZk1BOHBYK3ZtMEl2MEovL2VURHdRVWE1NHBjQitoVDI3TENJY05NdkRv?=
 =?utf-8?B?cDJwNGdLY0FzRGcweVVvWW93RnpXTlFUYnVsNmJYU0xnSURTYlNVUzZCQi8v?=
 =?utf-8?B?WEpza2lTSjhSbWgrbXNaRm9ReXh4RlNCN2drbWJ5MXBab0Y5NFVTMU13bWx2?=
 =?utf-8?B?bllpdU9SS0l0NWhkdWF2S2Y4QTAxZ2dNcnhhV1RUaXdtK2gvNlA1aDVDMERx?=
 =?utf-8?B?aml1SFdQald2TE9JUmN5a0UyZXFmSTRoNS9adU92b2hrbUUxS0szMTVLZ0tk?=
 =?utf-8?B?Z3FSeHk1SGlTNGZRQmV4SEZDWFh6VEt4SGV6ZHZkSDNvZ3lxRWUxaDNTS2F1?=
 =?utf-8?B?RWNONW1oT0hFUlNzcEdEWnVxTjFOR1VnSWxCY3BCN1pRbUZJcndva0xEVTlR?=
 =?utf-8?B?OXpYT0ZYR3lpcHBTVEg2R0xjenpGUnVTMTRFYTk4Ylk5QVdxR0FaLzdxRnpG?=
 =?utf-8?B?V28vUmtKd0h0YmhNVm1UREhVaGtjMm1yVW40R29HRHp1UnFzWkg3QUl4SHFl?=
 =?utf-8?B?eVh4UkxQUjBaaUtDeklyK0U1V1BGN0QwNnBCTDVYcEh1SDEwMTE5WStSNWp0?=
 =?utf-8?B?UTFtTE1RYXhMTnI3QjRUbHFJTDdRY294UHp4VnpMZDByVjlhSktPVHlWZEZa?=
 =?utf-8?Q?cm5hSwZFPEc+P+dfmeiti1bI5gE/K/klYnrNCkP?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ba7b8016-b3dd-4ea7-4b30-08d97da213f4
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:22:12.4130
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DKQBC9dv1cpFOoNwkvoWgw+1wHUeoSqDtL6/AizjGrpcIT0T7rTWq05rBfJaICSdzBhrOo6+nnYJ7Amft7Bt1A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Allow setting max_grant_version to 0 in order to disable grant table
usage by a domain. This prevents allocating the grant-table structure
inside of Xen and requires guards to be added in several functions in
order to prevent dereferencing the structure.

Note that a domain without a grant table could still use some of the
grant related hypercalls, it could for example issue a GNTTABOP_copy
of a grant reference from a remote domain into a local frame.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 docs/man/xl.cfg.5.pod.in     |   4 +-
 tools/libs/light/libxl_dom.c |   2 +-
 xen/common/grant_table.c     | 100 +++++++++++++++++++++++++++++++++--
 3 files changed, 98 insertions(+), 8 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index c5a447dfcd..d507540c2c 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -583,8 +583,8 @@ L<xl.conf(5)>.
 =item B<max_grant_version=NUMBER>
 
 Specify the maximum grant table version the domain is allowed to use. Current
-supported versions are 1 and 2. The default value is settable via
-L<xl.conf(5)>.
+supported versions are 1 and 2. Setting to 0 disables the grant table for the
+domain. The default value is settable via L<xl.conf(5)>.
 
 =item B<transitive_grants=BOOLEAN>
 
diff --git a/tools/libs/light/libxl_dom.c b/tools/libs/light/libxl_dom.c
index e9f58ee4b2..afc8b88497 100644
--- a/tools/libs/light/libxl_dom.c
+++ b/tools/libs/light/libxl_dom.c
@@ -598,7 +598,7 @@ static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "xc_dom_boot_image failed");
         goto out;
     }
-    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
+    if ( info->max_grant_version && (ret = xc_dom_gnttab_init(dom)) != 0 ) {
         LOGE(ERROR, "xc_dom_gnttab_init failed");
         goto out;
     }
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index f50e9f6a06..df01d03ce4 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1027,6 +1027,12 @@ map_grant_ref(
     }
 
     lgt = ld->grant_table;
+    if ( !lgt )
+    {
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", ld);
+        op->status = GNTST_bad_domain;
+        return;
+    }
     handle = get_maptrack_handle(lgt);
     if ( unlikely(handle == INVALID_MAPTRACK_HANDLE) )
     {
@@ -1037,6 +1043,14 @@ map_grant_ref(
     }
 
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        put_maptrack_handle(lgt, handle);
+        rcu_unlock_domain(rd);
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
     grant_read_lock(rgt);
 
     /* Bounds check on the grant ref */
@@ -1367,6 +1381,13 @@ unmap_common(
     ld = current->domain;
     lgt = ld->grant_table;
 
+    if ( !lgt )
+    {
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", ld);
+        op->status = GNTST_bad_domain;
+        return;
+    }
+
     if ( unlikely(op->handle >= lgt->maptrack_limit) )
     {
         gdprintk(XENLOG_INFO, "Bad d%d handle %#x\n",
@@ -1406,6 +1427,13 @@ unmap_common(
     TRACE_1D(TRC_MEM_PAGE_GRANT_UNMAP, dom);
 
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        rcu_unlock_domain(rd);
+        gdprintk(XENLOG_INFO, "%pd has no grant table\n", rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
 
     grant_read_lock(rgt);
 
@@ -1556,6 +1584,12 @@ unmap_common_complete(struct gnttab_unmap_common *op)
 
     rcu_lock_domain(rd);
     rgt = rd->grant_table;
+    if ( !rgt )
+    {
+        rcu_unlock_domain(rd);
+        op->status = GNTST_bad_domain;
+        return;
+    }
 
     grant_read_lock(rgt);
 
@@ -1931,10 +1965,7 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     if ( max_grant_version == XEN_DOMCTL_GRANT_version_default )
         max_grant_version = opt_gnttab_max_version;
     if ( !max_grant_version )
-    {
-        dprintk(XENLOG_INFO, "Invalid grant table version 0 requested\n");
-        return -EINVAL;
-    }
+        return 0;
     if ( max_grant_version > opt_gnttab_max_version )
     {
         dprintk(XENLOG_INFO,
@@ -2056,6 +2087,11 @@ gnttab_setup_table(
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out;
+    }
     grant_write_lock(gt);
 
     if ( unlikely(op.nr_frames > gt->max_grant_frames) )
@@ -2138,6 +2174,11 @@ gnttab_query_size(
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out;
+    }
 
     grant_read_lock(gt);
 
@@ -2302,6 +2343,13 @@ gnttab_transfer(
             goto put_gfn_and_copyback;
         }
 
+        if ( unlikely(!e->grant_table) )
+        {
+            gdprintk(XENLOG_INFO, "%pd has no grant table\n", e);
+            gop.status = GNTST_bad_domain;
+            goto unlock_and_copyback;
+        }
+
         if ( xsm_grant_transfer(XSM_HOOK, d, e) )
         {
             gop.status = GNTST_permission_denied;
@@ -2888,6 +2936,12 @@ static int gnttab_copy_claim_buf(const struct gnttab_copy *op,
 
     if ( op->flags & gref_flag )
     {
+        if ( !buf->domain->grant_table )
+        {
+            rc = GNTST_bad_domain;
+            goto out;
+        }
+
         rc = acquire_grant_for_copy(buf->domain, ptr->u.ref,
                                     current->domain->domain_id,
                                     buf->read_only,
@@ -3092,6 +3146,9 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop)
     int res;
     unsigned int i, nr_ents;
 
+    if ( !gt )
+        return -ENODEV;
+
     if ( copy_from_guest(&op, uop, 1) )
         return -EFAULT;
 
@@ -3270,6 +3327,11 @@ gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop,
     }
 
     gt = d->grant_table;
+    if ( !gt )
+    {
+        op.status = GNTST_bad_domain;
+        goto out2;
+    }
 
     op.status = GNTST_okay;
 
@@ -3332,7 +3394,11 @@ gnttab_get_version(XEN_GUEST_HANDLE_PARAM(gnttab_get_version_t) uop)
         return rc;
     }
 
-    op.version = d->grant_table->gt_version;
+    if ( d->grant_table )
+        op.version = d->grant_table->gt_version;
+    else
+        /* Use 0 to signal no grant table. */
+        op.version = 0;
 
     rcu_unlock_domain(d);
 
@@ -3351,6 +3417,12 @@ swap_grant_ref(grant_ref_t ref_a, grant_ref_t ref_b)
     struct active_grant_entry *act_b = NULL;
     s16 rc = GNTST_okay;
 
+    if ( !gt )
+    {
+        rcu_unlock_domain(d);
+        return GNTST_bad_domain;
+    }
+
     grant_write_lock(gt);
 
     /* Bounds check on the grant refs */
@@ -3872,6 +3944,9 @@ void grant_table_warn_active_grants(struct domain *d)
 
 #define WARN_GRANT_MAX 10
 
+    if ( !gt )
+        return;
+
     grant_read_lock(gt);
 
     nr_ents = nr_grant_entries(gt);
@@ -3953,6 +4028,9 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,
     int rc = 0;
     uint16_t flags = 0;
 
+    if ( !gt )
+        return -ENODEV;
+
     grant_read_lock(gt);
 
     if ( gt->gt_version < 1 )
@@ -4069,6 +4147,9 @@ unsigned int gnttab_resource_max_frames(const struct domain *d, unsigned int id)
     const struct grant_table *gt = d->grant_table;
     unsigned int nr = 0;
 
+    if ( !gt )
+        return 0;
+
     /* Don't need the grant lock.  This limit is fixed at domain create time. */
     switch ( id )
     {
@@ -4100,6 +4181,9 @@ int gnttab_acquire_resource(
     if ( !nr_frames )
         return rc;
 
+    if ( !gt )
+        return -ENODEV;
+
     final_frame = frame + nr_frames - 1;
 
     /* Grow table if necessary. */
@@ -4156,6 +4240,9 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn)
     struct grant_table *gt = d->grant_table;
     bool status = false;
 
+    if ( !gt )
+        return -ENODEV;
+
     grant_write_lock(gt);
 
     if ( evaluate_nospec(gt->gt_version == 2) && (idx & XENMAPIDX_grant_table_status) )
@@ -4200,6 +4287,9 @@ static void gnttab_usage_print(struct domain *rd)
     struct grant_table *gt = rd->grant_table;
     unsigned int nr_ents;
 
+    if ( !gt )
+        return;
+
     printk("      -------- active --------       -------- shared --------\n");
     printk("[ref] localdom mfn      pin          localdom gmfn     flags\n");
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:22:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:22:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192258.342644 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxWd-0007SZ-41; Wed, 22 Sep 2021 08:22:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192258.342644; Wed, 22 Sep 2021 08:22:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSxWd-0007SO-0U; Wed, 22 Sep 2021 08:22:43 +0000
Received: by outflank-mailman (input) for mailman id 192258;
 Wed, 22 Sep 2021 08:22:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSxWb-0005Fo-Tu
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:22:41 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 35cc969e-1b7e-11ec-b963-12813bfff9fa;
 Wed, 22 Sep 2021 08:22:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35cc969e-1b7e-11ec-b963-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632298942;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=p/2j7Qh3HRSjw1xZ6iz4Wwt7riuFKipEEnPAZ4kgGQU=;
  b=eYyMA264UqhRZxFPd+pUO0ecxiTf3lSb+TFvanhckpG9b8xlmMAxMt7H
   W37JfVxXLXsRAdBLIUrtxBoAH8JLNLRqSOjrXFP4SSibQOAciHL6bJ2xa
   MZZXBuHCrIsXKNxYwlc7cLNVveI25QDJ5vcQWXRepFt8tgi/TGB46duD+
   I=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: E9Xy6cmS7skkVJ/00AE9peAha+sSimph+icGRWBCy3G1U/wJ8eDyf3rBAytbXX3Bti/8iGc6LP
 LTwYk5FJ4w4+mG8MXycpDJbic+vulB0Du6vTfBw/ROcI6QZoayKA794RkLWEvuhz0Gs5IxrJDa
 l80ou6qxsjVjXMLqJDy1GQnSS1EZir1YOEZJA/fNgYemge8XAhteTu9o4/ku4rWpNPOX4hvnKB
 QXvqS09MTBAF0/d1Zou2OpTIBGNKVtK//g2q+d6Id5WujGVpmfqZcsI03m8zLNVZgs2gskG3p7
 5aWCv9x6HeHwuRJVUMa8aGb5
X-SBRS: 5.1
X-MesageID: 53294561
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:tf0Sc6t0Z1W2azd165+IJcju/efnVMZZMUV32f8akzHdYApBsoF/q
 tZmKTqOOP6CM2KjKtFzbIWx8x9XsZTTmtQwGlRt+C0xE39H+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524LhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npl6o6QVw0RN7f3lr4SfkV9FAxABqpb9+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sUTKqDO
 ZNxhTxHcBrGTAZ9OlYtEL0Dm973qSnGVDpxpwfAzUYwyzeKl1EguFT3C/LXdcaNXt59hVuDq
 yTN+GGRKgoBKNWVxD6B83StruzChyX2XMQVDrLQ3sBthFqf12kCEio8XFGwoeS6okOmUtcZI
 EsRkgIipLY/3FamRd78W1u/unHslhwBX9tdFcUq5QfLzbDbiy6YGWosXjNHcMYhtsI9WXotz
 FDht8PkA3ljvaOYTVqZ96yItnWiNC4NN2gAaCQYCwwf7LHLoos+kxbORdZLC7Oug5v+HjSY6
 y+OhDgzgfMUl8Fj6kmg1QmZ2XT2/MGPF1NroFWMNo640u9nTNG3R5Ki0VHB1vgaI6vFa16sg
 kMDhsfLuYjiEqqxeDyxrPQlRe/yva/VbmGE2DaDDLF6qG/8oCfLkZR4pWgkfRY3aJ5slSrBP
 ReL0T698qO/K5dDgUVfWIu3F81i5qzpD92Nuhv8P4cWP8QZmONq+khTia+sM4LFyxNEfUIXY
 87znSOQ4ZEyU/8P8dZOb71BuYLHPwhnrY8peXwe8ylLLJLEPCLFIVv6DLd+Rr9gt/7VyOkk2
 /1eK9GL231ibQEKWQGOqdR7BQlTdRATXMmqw+QKJr/rClc3QwkJVq6OqY7NjqQ4xsy5YM+Tp
 SrjMqKZoXKi7UD6xfKiMCE7NOy3AsYh8hrW/0UEZD6V5pTqWq72hI83fJorZ7g3sutlyP9/V
 f4efMucRP9IT1z6F/41MfERdaRuK0amgxygJS2gbGRtdpJsXVWRqNTlYhHu5G8FCS/u7Zkyp
 Lip1wX6R5sfRls9UJaKOaz3l17h72IAnO9SXlfTJoUBckvb74U3eTf6ieU6Ip9QJEyblCeaz
 QufHTwRufLJ/90u6NDMiK3d99WpHuJyE1B0BW7e6brqZyDW8nD6md1LUfqSfCCbX2Txof3wa
 eJQxvD6EfsGgFcV7NYsT+c1lfozvoK9qaVbwwJoGGTwQ26qUr4wcGOb2cRvt7FWwuMLswWBR
 U/SqMJRPq+EOZ25HQdJdhYldOmKydodhiLWsaYuOEz/6SJ6oOiHXEFVM0XegSBRNuIoYoYsw
 ONns88K8Q2vzBEtN4/e3CxT8m2NKF0GUrkm6c5GUNO61FJzxwEQe4HYBw/3/IqLOodFPUQdK
 zOJgLbP2uZHzU3YfntvTXXA0IKxX3jVVMymGLPaG2m0pw==
IronPort-HdrOrdr: A9a23:UlrRyK61ZP/PnGa36gPXwVmBI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwXJVoJkmsiaKdgLNhQItKOTOJhILGFvAF0WKP+UyDJ8S6zJ8n6U
 4CSdkONDSTNykCsS+S2mDReLxBsbr3gJxAx92utUuFJTsaFZ2IhD0JbTpzfHcGITWvUvECZe
 WhD4d81nedUEVSSv7+KmgOXuDFqdGOvJX6YSQeDxpizAWVlzun5JPzDhDdh34lInxy6IZn1V
 KAvx3y562lvf3+4hjA11XL55ATvNf60NNMCOGFl8BQADTxjQSDYphnRtS5zXwIidDqzGxvvM
 jHoh8mMcg2w3TNflutqR+o4AXk2CZG0Q6r9XaoxV/Y5eDpTjMzDMRMwahDdAHC1kYmtNZglI
 pWwmOwrfNsfFP9tRW4w+KNewBhl0Kyr3Znu/UUlWZjXYwXb6IUhZAD/XlSDIwLEEvBmcEa+d
 FVfYPhDcttABanhyizhBgq/DXsZAV8Iv6+eDlAhiTPuAIm20yQzCMjtYUidzk7hdQAozQt3Z
 WxDk1SrsA9ciYhV9MKOA4we7rFNoXze2O4DIvrGyWeKEgmAQOHl3el2sR+2AmVEKZ4u6fa3q
 6xCW9liQ==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53294561"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FM2BnFBSgG6pTY6Bi7x+SyBFQcp+ufv76WDvxpubLUYyJHeQ1oOM57SDAIS6Bl6zqFHu0npOFGEgtH2EkrLgUdkjw4SQKIv7BuTDtrNIAlQS7dTQsJ93t6OqU2cYLib5njubvNUU33OtYd3txLrK75xorT8eAYDvmNi+ZBhNwGl2o0D3LO0c3MhJNBdofc3jhKHGEhuAzt8ylkl0diOQz+mZBaifrfRTDadJvMZxsuJtmwHhwBAW+a/X5uBmBhDVs9wcwd7uYTd+9BtpZVuL3g24yx7jAEKi8Pp5wPMkP8dgCcKBthSSw6NUC30KQ34nmKfcOoyau3UjJ01Y8ReerA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=voNJG1FSxj3tOQiI/DkvvHkabCshBBjpOakCSWaIsGg=;
 b=aO4LVvvH6mZBIk4/cC8PEK15NQt5RhbRxJYjZz0j35uuHWofCDoK6d4CxW8FXfYegrfYY7DTeCZF+TBnOoGUsW6jCtmGfI6foQ9IuQxaqaqs8NNOsFaXbOVSv1o5siCzfREiFPXd2k6vkMXEvurG7+3eP8gw3Da0HRGJR9bLyHjUzCOOS/haJYWkE+uRbGePLK11uOaWNKx8MZEeCOhlKCSGPfqY5NnuFZhFVg3whXJB8Co/W0DLFcYq+/54m0NHdrgBe2QNq9u4I2faAsTtPv8iO631s3K3zpaMmIwP222mRryuvdIuyxfp/U9H1gYRzrIlW+SYjFl+OSTW/mb6Lw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=voNJG1FSxj3tOQiI/DkvvHkabCshBBjpOakCSWaIsGg=;
 b=FG51XV7BSa4qpXeEfpZkE/mFtMwhNSk/u9QWoW8fdXwtSKZGYytp4mb0/7hZPi15lu1hQWmZSDJrm902vrJ6REueg9IIWoyNfdYE6gIpr5TjTP8CRgBi+rOxn0xDXAYpNEexfniwhGvuLvIWyth3YaGECDThyDSOWYh6i6KbExw=
From: Roger Pau Monne <roger.pau@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Roger Pau Monne <roger.pau@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony
 PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 3/6] gnttab: allow per-domain control over transitive grants
Date: Wed, 22 Sep 2021 10:21:20 +0200
Message-ID: <20210922082123.54374-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0347.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18d::10) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7ca136c5-6a57-4cfc-589c-08d97da20c24
X-MS-TrafficTypeDiagnostic: SN6PR03MB4015:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB401518E3905D15760968E9128FA29@SN6PR03MB4015.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Fb9ovAOT/IvtJ20NdrpGwl+BRjujympR1hO2wXO3CVj2J4aUFqwsRgg6L8St5mivgw64bPjQGLpq4Ijssqs0zwfbmKH75yLoHGphzLBYJXqOiQ3kwl8SjZyfRz93HnlMpv099XxUtFXIH9WqkYSR8TtaD1xnZq5QcKyIxvcaRDR/U8BKn8qizsCKfBrx3SSv7Y8qLHZaq1dhmBvQ0GwKc1nvxJ4KaiHa9e6vTyzdH1TN3Z0InKbOmGVM1CPKuyofquX/DB5Zbt0OuU3fQxM8KxYZRbORVrrTr97eYFVZ0f57YTyAd/rEY/eT4RQHQZjFkk4jcJx5eKJR5n0epYNYRwoBt3UIRxGeb+bXkcqDthOfPBPVW+GgR/mHYjKRYBzdtCvILQe7ULW1VIHGl4wXl92CNNW/QtLSrO78lxcwdrPeof8iq0eYkN0WFuT7eFOP8ID/tK2DH2oVUwou0+sc6FshGLJdJBbOtRblQlKHB9+OVMOxfr1M9tAKBVluD9fShAO8o9EyUbRbmBTApJR2pILdhg76CS/MoKeQrD7HrDDNLQucp2zNM0dPmiPLMFvcz70NcYaJ1uyB26c5Yt0qnmNGLp9snTVw7s2xxsJU8bxg2e/DPSSLOWlFOpPhQ4iq2qCZbjMnwzyEJ6TUnwchSQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(66946007)(4326008)(38100700002)(8936002)(186003)(956004)(86362001)(2616005)(8676002)(508600001)(54906003)(83380400001)(6496006)(26005)(6916009)(30864003)(2906002)(6666004)(5660300002)(1076003)(6486002)(36756003)(316002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Rk1CdWdnbXFLK2RCQ1NUd2VtTnp4Y1pQRldSQkFQOVJkOEgydm0xaGYvV2Jl?=
 =?utf-8?B?c2srSXZFT3ZPSjFvRnlKWUpJcjNYNGx2YmYyWitXd2x0emoyL1JMVVpFOUJG?=
 =?utf-8?B?R3hNK1FUaFNTVWRYcEdxeC9pNlpiZ3dhcFdsQlUzbmxDclR2RjNUTm1hcDND?=
 =?utf-8?B?YW1JUTdWVC83M29xNi9IY0wvZmNKNForOEVteTJpbE1IOWRTeW40M1ErY1cy?=
 =?utf-8?B?b0dHa0tZRDQwTXYydXU3N2Z3Ykp0NjdzKzV1V1A1d3hBTDZYTmFQSkVUSEx2?=
 =?utf-8?B?SjBHZzBuczJFTEQrTE9sV1dBdTg5S1RQL2krd1ZCNFhlN0JQdzJ1dzRrZ1U1?=
 =?utf-8?B?b2tWVnR6ZmhGNDdueE5jUk9UKzVsUVYyaExLNVE2YzBKMjlhL0ZVUVR2QVZj?=
 =?utf-8?B?b3R1Smx2cFpVaE5LcGNORTRMeXRqVzRoYXpCZlJVR0JvRTF0bHg2TTI1R2tj?=
 =?utf-8?B?Qk5HR2NZbDNXK1ROdGprSTFjVTJVL2FvUDlCZEM0ZkJCS2V1V0pZNkNIQ01Q?=
 =?utf-8?B?TUdVb2xlbGZKaHVDNnY1WGViUW0zb3hJZTZTcVNES2phbUgrbkJRZTh5Q2dX?=
 =?utf-8?B?dzVrTFZZa0FFeEN5OEZjQ0EyL3BRUmhweGVxczV5aG0zbjdpeHVLVURCSXZV?=
 =?utf-8?B?UzFoL2R2VEV5Z2hYMmk4NnJnYmR5Z2tKVHZDRG4vdmJVZmlnc1JLZXZMSkZY?=
 =?utf-8?B?em9oQjlXdVQ0cWlzMGVzOWxwRnIvb0FIbnJ5YktJQm94Z2duTW42NzdtNGZr?=
 =?utf-8?B?ejlhTlFQT1h0b1hpbXc5THMvR1BhQkpHeS9qc2NQeGhIaHcxNDdFN1Q0SGhz?=
 =?utf-8?B?RS9TMUtPOUdDa20xYTV5Vy9HcmxtQlY2ckpTNXU4MHluQlY4aDcvV0pSSnN6?=
 =?utf-8?B?MFRTbWUyM2xZa3g4aE1nMzdZUlQwYk5HZ1dZTm1QQ21QQ3plUmpIZVhSOXF0?=
 =?utf-8?B?Y3U2OEFCSk5BdFYwdFoyaEZTaWU3YlcwSkhPQkVoRkZkR0JucmVxbzF3WFNj?=
 =?utf-8?B?QkhobnMrVUFUSitURmVTeWtCNU9Oa1NxRHNmcG9SdmdNc1A2TDZDWnpEVUZH?=
 =?utf-8?B?a05oSEFTc3Jnd2J5eUNxWGpIbXNQNnpzVWNrS2szSHc5cUYxU01oRXBhU2sy?=
 =?utf-8?B?WG1TaFZGc3ZyQlJaenZtdE5tQ2RKUDBVWXRzVVJVcWJDaTBBdmczSEJBL0xm?=
 =?utf-8?B?OS9qZnBKZERRdU5vR0ZVdmVQckNYSWhIOFVZMzFOd3Q2R3hmSHljRm5DOXpC?=
 =?utf-8?B?SlJBd24xYmZMSC9vL3hHY1lMTmJzSmowUGZVbHR2NE50NS9iZUJzcGQ4VTg0?=
 =?utf-8?B?alRVdG0wamVOa2dXWjVCYUFySHlZR3Q5OWtTNkdvOUxFQ1ZwK2JGN3pzRFo0?=
 =?utf-8?B?d29ndkhVdTBIV1NTalJ4eVpDcXA2UGVXQUl2SWlYM2FTNWFiQ1B3T3JpRzc5?=
 =?utf-8?B?TFRSRmJNWVdENkRqaWxRMm1qZzJ2bWtOVkJzVDJOcjhDQU0rOUFvS3hCUlBY?=
 =?utf-8?B?UGEyVFNzM0JiZnRDSEFkajV2blhlcENLeVEzS3Q0UHN4OHVWV1ltN3lvRGN3?=
 =?utf-8?B?RGR3RlR4dkRQVFZCU1VtQTRsb2RVeFptZlkxMEdMTFVLcEVoakNLRlpqMmpu?=
 =?utf-8?B?M2x5b1BFc0V5ekxTdFhQa0liTDZuYmROajZCRDV2OTM5M3AvZlZ0eHR4TkNi?=
 =?utf-8?B?UGdMRXBLVHljeWVNUyt2K1g3QUdFaHYrd2tMQW84N0VEN0YvUHdzcFY0ajZU?=
 =?utf-8?Q?9YA7D96jY/ECtx6kUENn1CFX4UxxmUCWb6qBT8l?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ca136c5-6a57-4cfc-589c-08d97da20c24
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 08:21:59.3101
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rKnVEWxnaLZQwW6H8sOjylWgwny0NFsk/6fte9KeEnMR01kJViYm20Lvt4Kr43/xULkKcfZ79TRJegFFM8C1Ig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB4015
X-OriginatorOrg: citrix.com

Introduce a new grant options flags field in domain create and use it
to signal whether transitive grants are allowed on the domain. This is
settable from xl using the transitive_grants option.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 docs/man/xl.cfg.5.pod.in            | 6 ++++++
 docs/man/xl.conf.5.pod.in           | 7 +++++++
 tools/include/libxl.h               | 7 +++++++
 tools/libs/light/libxl_create.c     | 3 +++
 tools/libs/light/libxl_dm.c         | 1 +
 tools/libs/light/libxl_types.idl    | 1 +
 tools/ocaml/libs/xc/xenctrl.ml      | 1 +
 tools/ocaml/libs/xc/xenctrl.mli     | 1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c | 7 ++++++-
 tools/xl/xl.c                       | 7 +++++++
 tools/xl/xl.h                       | 1 +
 tools/xl/xl_parse.c                 | 4 ++++
 xen/arch/arm/domain_build.c         | 6 ++++--
 xen/arch/x86/setup.c                | 3 ++-
 xen/common/grant_table.c            | 8 +++++++-
 xen/include/public/domctl.h         | 5 ++++-
 16 files changed, 62 insertions(+), 6 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index a4bf2caafa..c5a447dfcd 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -586,6 +586,12 @@ Specify the maximum grant table version the domain is allowed to use. Current
 supported versions are 1 and 2. The default value is settable via
 L<xl.conf(5)>.
 
+=item B<transitive_grants=BOOLEAN>
+
+Specify whether transitive grants should be available to the domain. Note such
+functionality only applies to grant table version 2. The default value is
+settable via L<xl.conf(5)>.
+
 =item B<nomigrate=BOOLEAN>
 
 Disable migration of this domain.  This enables certain other features
diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in
index 0a70698a7c..88efbee5fe 100644
--- a/docs/man/xl.conf.5.pod.in
+++ b/docs/man/xl.conf.5.pod.in
@@ -108,6 +108,13 @@ Sets the default value for the C<max_grant_version> domain config value.
 Default: value of Xen command line B<gnttab> parameter (or its default value if
 unspecified).
 
+=item B<transitive_grants=BOOLEAN>
+
+Sets the default value for the C<transitive_grants> domain config value.
+
+Default: value of Xen command line B<gnttab> parameter (or its default value if
+unspecified).
+
 =item B<vif.default.script="PATH">
 
 Configures the default hotplug script used by virtual network devices.
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 7a35833312..d23586f2cc 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -509,6 +509,13 @@
  */
 #define LIBXL_HAVE_MAX_GRANT_VERSION 1
 
+/*
+ * LIBXL_HAVE_TRANSITIVE_GRANTS indicates libxl_domain_build_info has a
+ * transitive_grants field for setting whether such functionality should be
+ * available to the domain.
+ */
+#define LIBXL_HAVE_TRANSITIVE_GRANTS 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 1ee86602ae..fa527923e4 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -635,6 +635,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
             create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
 
+        if (libxl_defbool_val(b_info->transitive_grants))
+            create.grant_opts |= XEN_DOMCTL_GRANT_transitive;
+
         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 9a8ddbe188..4ade437257 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -2321,6 +2321,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
     dm_config->b_info.max_grant_frames = guest_config->b_info.max_grant_frames;
     dm_config->b_info.max_maptrack_frames = guest_config->b_info.max_maptrack_frames;
     dm_config->b_info.max_grant_version = guest_config->b_info.max_grant_version;
+    dm_config->b_info.transitive_grants = guest_config->b_info.transitive_grants;
 
     dm_config->b_info.u.pv.features = "";
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 37789a568c..d05b5d2abc 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -519,6 +519,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("max_grant_frames",    uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_maptrack_frames", uint32, {'init_val': 'LIBXL_MAX_GRANT_DEFAULT'}),
     ("max_grant_version",   integer, {'init_val': '-1'}),
+    ("transitive_grants",   libxl_defbool),
     
     ("device_model_version", libxl_device_model_version),
     ("device_model_stubdomain", libxl_defbool),
diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index 6a8658bfec..da12b67baf 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -84,6 +84,7 @@ type domctl_create_config =
 	max_grant_frames: int;
 	max_maptrack_frames: int;
 	max_grant_version: int;
+	transitive_grants: bool;
 	arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 5933d32c70..83ca554eb7 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -76,6 +76,7 @@ type domctl_create_config = {
   max_grant_frames: int;
   max_maptrack_frames: int;
   max_grant_version: int;
+  transitive_grants: bool;
   arch: arch_domainconfig;
 }
 
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index 1e60925069..5697a8bd8b 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -189,7 +189,8 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
 #define VAL_MAX_GRANT_VERSION   Field(config, 8)
-#define VAL_ARCH                Field(config, 9)
+#define VAL_TRANSITIVE_GRANTS   Field(config, 9)
+#define VAL_ARCH                Field(config, 10)
 
 	uint32_t domid = Int_val(wanted_domid);
 	int result;
@@ -216,6 +217,9 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 		/* ! XEN_DOMCTL_IOMMU_ XEN_DOMCTL_IOMMU_MAX max */
 		(VAL_IOMMU_OPTS);
 
+	cfg.grant_opts |= Bool_val(VAL_TRANSITIVE_GRANTS)
+			  ? XEN_DOMCTL_GRANT_transitive : 0;
+
 	arch_domconfig = Field(VAL_ARCH, 0);
 	switch ( Tag_val(VAL_ARCH) )
 	{
@@ -255,6 +259,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config
 	}
 
 #undef VAL_ARCH
+#undef VAL_TRANSITIVE_GRANTS
 #undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 0fde879cf4..9bd398f8c9 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -56,6 +56,7 @@ bool timestamps = 0;
 int max_grant_frames = -1;
 int max_maptrack_frames = -1;
 int max_grant_version = -1;
+bool transitive_grants = true;
 libxl_domid domid_policy = INVALID_DOMID;
 
 xentoollog_level minmsglevel = minmsglevel_default;
@@ -221,6 +222,12 @@ static void parse_global_config(const char *configfile,
     else if (e != ESRCH)
         exit(1);
 
+    e = xlu_cfg_get_long (config, "transitive_grants", &l, 0);
+    if (!e)
+        transitive_grants = l;
+    else if (e != ESRCH)
+        exit(1);
+
     libxl_cpu_bitmap_alloc(ctx, &global_vm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_hvm_affinity_mask, 0);
     libxl_cpu_bitmap_alloc(ctx, &global_pv_affinity_mask, 0);
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index cf12c79a9b..d7f83c9abd 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -283,6 +283,7 @@ extern char *blkdev_start;
 extern int max_grant_frames;
 extern int max_maptrack_frames;
 extern int max_grant_version;
+extern bool transitive_grants;
 extern libxl_bitmap global_vm_affinity_mask;
 extern libxl_bitmap global_hvm_affinity_mask;
 extern libxl_bitmap global_pv_affinity_mask;
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 1206d7ea51..8e4e561df8 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1440,6 +1440,10 @@ void parse_config_data(const char *config_source,
     else
         exit(1);
 
+    xlu_cfg_get_defbool(config, "transitive_grants", &b_info->transitive_grants,
+                        0);
+    libxl_defbool_setdefault(&b_info->transitive_grants, transitive_grants);
+
     libxl_defbool_set(&b_info->claim_mode, claim_mode);
 
     if (xlu_cfg_get_string (config, "on_poweroff", &buf, 0))
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 3beb1cbb41..b9048aa9cc 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2649,7 +2649,8 @@ void __init create_domUs(void)
             .max_evtchn_port = -1,
             .max_grant_frames = -1,
             .max_maptrack_frames = -1,
-            .grant_opts = XEN_DOMCTL_GRANT_version_default,
+            .grant_opts = XEN_DOMCTL_GRANT_version_default |
+                          XEN_DOMCTL_GRANT_transitive,
         };
 
         if ( !dt_device_is_compatible(node, "xen,domain") )
@@ -2757,7 +2758,8 @@ void __init create_dom0(void)
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
-        .grant_opts = XEN_DOMCTL_GRANT_version_default,
+        .grant_opts = XEN_DOMCTL_GRANT_version_default |
+                      XEN_DOMCTL_GRANT_transitive,
     };
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b5b6c75447..7ce54a5a57 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -750,7 +750,8 @@ static struct domain *__init create_dom0(const module_t *image,
         .max_evtchn_port = -1,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
-        .grant_opts = XEN_DOMCTL_GRANT_version_default,
+        .grant_opts = XEN_DOMCTL_GRANT_version_default |
+                      XEN_DOMCTL_GRANT_transitive,
         .max_vcpus = dom0_max_vcpus(),
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index c43e9d5ee4..f50e9f6a06 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -74,6 +74,10 @@ struct grant_table {
      * progress.
      */
     unsigned int          maptrack_limit;
+
+    /* Allow transitive grants. Only applies to grant v2. */
+    bool                  allow_transitive:1;
+
     /* Shared grant table (see include/public/grant_table.h). */
     union {
         void **shared_raw;
@@ -1965,6 +1969,8 @@ int grant_table_init(struct domain *d, int max_grant_frames,
     gt->max_grant_frames = max_grant_frames;
     gt->max_maptrack_frames = max_maptrack_frames;
     gt->max_grant_version = max_grant_version;
+    gt->allow_transitive = !!(options & XEN_DOMCTL_GRANT_transitive) &&
+                           opt_transitive_grants;
 
     /* Install the structure early to simplify the error path. */
     gt->domain = d;
@@ -2887,7 +2893,7 @@ static int gnttab_copy_claim_buf(const struct gnttab_copy *op,
                                     buf->read_only,
                                     &buf->mfn, &buf->page,
                                     &buf->ptr.offset, &buf->len,
-                                    opt_transitive_grants);
+                                    buf->domain->grant_table->allow_transitive);
         if ( rc != GNTST_okay )
             goto out;
         buf->ptr.u.ref = ptr->u.ref;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index e2b47184a0..2f90e4c3f8 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -98,8 +98,11 @@ struct xen_domctl_createdomain {
 /* Grant version, use low 4 bits. */
 #define XEN_DOMCTL_GRANT_version_mask    0xf
 #define XEN_DOMCTL_GRANT_version_default 0xf
+/* Allow transitive grants. */
+#define _XEN_DOMCTL_GRANT_transitive     4
+#define XEN_DOMCTL_GRANT_transitive      (1U << _XEN_DOMCTL_GRANT_transitive)
 
-#define XEN_DOMCTLGRANT_MAX XEN_DOMCTL_GRANT_version_mask
+#define XEN_DOMCTLGRANT_MAX XEN_DOMCTL_GRANT_transitive
 
     uint32_t grant_opts;
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 08:57:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 08:57:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192285.342655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSy4B-0003VY-Pe; Wed, 22 Sep 2021 08:57:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192285.342655; Wed, 22 Sep 2021 08:57:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSy4B-0003VR-MX; Wed, 22 Sep 2021 08:57:23 +0000
Received: by outflank-mailman (input) for mailman id 192285;
 Wed, 22 Sep 2021 08:57:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mSy4A-0003VL-6V
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 08:57:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSy43-00031Y-8D; Wed, 22 Sep 2021 08:57:15 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSy42-0002zi-Te; Wed, 22 Sep 2021 08:57:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=66zvIqgSVOKKfKBje1fWJ3mG5BJ0OdZH/PZ9Pl5FB2w=; b=W/wKxZsU2ajgW6WoJRB2wnSGCZ
	n9y9x66FrKruwWZ1wgRNCsWUdYkvP2Sbg7c6RoeMJ4CBm2hkn5jTeVtu9Sl75U89Sv9HoBv14pVww
	OhyeXml4c9f5fXt3ru2Q7nQgUlht4xLZWV/PAHqOgZxD6EO2Qsejw9R4Wt/5sc5qNnmQ=;
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
Message-ID: <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
Date: Wed, 22 Sep 2021 13:57:02 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210922082123.54374-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 22/09/2021 13:21, Roger Pau Monne wrote:
> Hello,

Hi Roger,

> First patch on the series is a trivial change to xenconsoled in order to
> use xenforeignmemory stable library in order to map the shared console
> ring instead of the unstable libxc interface. It's reviewed and ready to
> go in.
> 
> Patches 2 and 3 allow setting the host wide command line `gnttab` option
> on a per domain basis. That means selecting the max allowed grant table
> version and whether transitive grants are allowed.
> 
> The last 3 patches attempt to implement support for creating guests
> without a grant table. This requires some changes to xenstored in order
> to partially support guests without a valid ring interface, as the lack
> of grant table will prevent C xenstored from mapping the shared ring.
> Note this is not an issue for Ocaml xenstored, as it still uses the
> foreign memory interface to map the shared ring, and thus won't notice
> the lack of grant table support on the domain.

I find a bit odd that the Xenstore support is conditional to whether 
grant table is available. Are you expecting domains with no grant table 
to have no PV drivers (including PV shutdown)?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:04:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:04:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192292.342666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyB3-0004xA-Hr; Wed, 22 Sep 2021 09:04:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192292.342666; Wed, 22 Sep 2021 09:04:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyB3-0004x3-Ei; Wed, 22 Sep 2021 09:04:29 +0000
Received: by outflank-mailman (input) for mailman id 192292;
 Wed, 22 Sep 2021 09:04:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=btef=OM=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mSyB1-0004wx-S7
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:04:28 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.82]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 157d4284-1b84-11ec-b966-12813bfff9fa;
 Wed, 22 Sep 2021 09:04:25 +0000 (UTC)
Received: from AS8PR04CA0178.eurprd04.prod.outlook.com (2603:10a6:20b:331::33)
 by VE1PR08MB4767.eurprd08.prod.outlook.com (2603:10a6:802:a6::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Wed, 22 Sep
 2021 09:04:22 +0000
Received: from AM5EUR03FT007.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:331:cafe::5f) by AS8PR04CA0178.outlook.office365.com
 (2603:10a6:20b:331::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 22 Sep 2021 09:04:22 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT007.mail.protection.outlook.com (10.152.16.145) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 09:04:21 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Wed, 22 Sep 2021 09:03:46 +0000
Received: from b5498b7ae3eb.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C6C19471-31C3-469C-B847-DF878F39C896.1; 
 Wed, 22 Sep 2021 09:03:34 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b5498b7ae3eb.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 Sep 2021 09:03:34 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6719.eurprd08.prod.outlook.com (2603:10a6:102:137::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 09:03:32 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.014; Wed, 22 Sep 2021
 09:03:32 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0040.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:61::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 09:03:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 157d4284-1b84-11ec-b966-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3RQglqc++WdG6G3/Aa49Tp5rLAbbmw/GSFFX4GGk4nc=;
 b=jCBa64H5zexRA/T1po+3yHOCwH1atD8hTGqZoOqPFaBapj+JQ0fcuXdcE5eb8lVR6qQCtRUrCXd+943wA0B4bi/ojekPOtRM4w24fepJqkTTkQeRmku0ioE612RX2oGMprmBY0fnsufQufs9aiTW/NRFV35eNCGvIyOSKKH5XKk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f5bc6b47a4f41cfa
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ctVoS2m8LdiaqKaj3tp1ELdu7ftiuBBWriGPB70Ry4XwRew3pIQR+z9JGUrDs1d2ez74RdesmYfusz4YI9eHv6kcMJtukhWXmEw4oGVeQ0A5nYQjjXSWAi1UAZeJnLG1eQvYMGFwPbnxJCj0wl1EmoQ9JpiMtLLjbl4p4EbzS9XNg3mTYjv//30Ji83Ghj8RTl7OhKdyxZ3jLFrrnB/6Iuq8DihBGktM1ioNXFS2yUupoy2iYB5oV2W/2sWWPDIaE/acQ8hX/2bViWNBRpB5tNlpkJ9ZEaaJpVvvw9BmlW5Z+EFEYI0HqOTxG4tev8SYSGq2BVONcl+x/YmJITQHyA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3RQglqc++WdG6G3/Aa49Tp5rLAbbmw/GSFFX4GGk4nc=;
 b=Wu/IWWN0eIzNx4qLC/2TGXEojXgQPUsx1DOjUzVhO01H52avN2QeZ83ybQ/trlbLqXGXcWfYe6cxzHyuJ9SFYTyU31ooS+MY3sVFUo9TI/Stayl9ygal4v+FVzkXCBaN4hqR/Mdpboh8Xe0Rrhgfiiz9zmEv9fftaiDlpA7dfU7B4CxOT90mmXG0SuUV/89hoxGiSrrQ2fvzoTw2ZP2IUgHapUtaQRQ57t3bWYOTUd021dDii9STMdhCV0e8HK2AWCygheU+hUID950BQMcIfyc6/+cJSxB5wbTIIzsObxlrPwFgVrTaeaZKXgoPbAsMdMsFiq9JnujYAcmYY0o5TA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3RQglqc++WdG6G3/Aa49Tp5rLAbbmw/GSFFX4GGk4nc=;
 b=jCBa64H5zexRA/T1po+3yHOCwH1atD8hTGqZoOqPFaBapj+JQ0fcuXdcE5eb8lVR6qQCtRUrCXd+943wA0B4bi/ojekPOtRM4w24fepJqkTTkQeRmku0ioE612RX2oGMprmBY0fnsufQufs9aiTW/NRFV35eNCGvIyOSKKH5XKk=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109211357280.17979@sstabellini-ThinkPad-T480s>
Date: Wed, 22 Sep 2021 10:03:23 +0100
Cc: Jan Beulich <jbeulich@suse.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <D171A1CD-ED8E-45F6-A311-92809158415F@arm.com>
References: <20210915142602.42862-1-luca.fancellu@arm.com>
 <20210915142602.42862-3-luca.fancellu@arm.com>
 <b3263ea5-b875-1c28-0e03-f911e0e97382@suse.com>
 <59F99E1E-C306-40BE-8B47-5D92ABF101F5@arm.com>
 <5bfb2544-402b-d6e1-9a8a-027e36a60acd@suse.com>
 <29091674-DCAE-4289-901E-9158FD029D96@arm.com>
 <125df38d-ac1f-c4a3-2ef9-1ce0f03e77c6@suse.com>
 <alpine.DEB.2.21.2109161255040.21985@sstabellini-ThinkPad-T480s>
 <28787ACF-F70E-49C8-B3E2-CFCE8E9B6AA7@arm.com>
 <alpine.DEB.2.21.2109171516270.21985@sstabellini-ThinkPad-T480s>
 <18531A94-ECF7-4840-A515-B161A4ABDEF4@arm.com>
 <alpine.DEB.2.21.2109211357280.17979@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0040.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:61::28) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 62459d39-1037-4de6-29a9-08d97da7f7c2
X-MS-TrafficTypeDiagnostic: PAXPR08MB6719:|VE1PR08MB4767:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB476788BBBA2CD94BB917BFAAE4A29@VE1PR08MB4767.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 D/MlVKPRt7YKjBoUgFQbt/VyCf7ZcklQQ7GZj4Bb49IWT/EEfT1+b6IYEn55UvsrWAfkii2NFSegB75VnHb26SWv0Zh6DtunFlqgpEjBowj23BGmA32lByRo6OilQ2tjj74eKGpjaWXHNYs6USR0/UC2Y88GsbNvlQiN6rR6nQOe4PCjB0RQXTuKL2ASwEQiq+8kU8JSuMp3GlnmTH1pPTzsaYtHbj++Jm/kmDw1gw5ud+1KTD07gaWhSLfykYiJ8yAeKlFkYK0Ouc+W53FvDUZRawqZku6rBvAvq6n7fyeDLmalivDn7yvYIM0uVywIqFLcdCydhUK6MfYKhD9sKKjtPqJMT9weBB79dsQ9d+AqL8cSC/VOofgIP3nG1MvBCgri4f7/n5jpVOz4lxq1wa9zIR+1QrgvGy2mCaP0egJ2zfqB6Q0AVpph3IGX8vQwknbItLZKzqGr5T7CXIHqtCWXa2QvhW67G9ON3BokoJEorjLwn2sJ001TrC+pZdrhV6gNV+/hRG6cdepZYHbYfsLGue3fFO9+IeHRzXSf3aVKJMPxjDUwviZVMS7OfoZj1sXDV8Bjw2UXsY+q41muXu3CtLBEmaOwPjy4oR6Nol9lgZ0+M69pu7fiH4gQEazWVwoc0SNJyojy/ZQWRhKMAXio32MWfhSvZZm4BBaEOu2vOCV7ZM5HWjih51oOy+2RCXemCWn3CCSU9CZ28Ssg58jcqKvchyX233zwynmt+i+qdHIB9IfBDhHxwxHCEUNbNovbM6PqxdkGSYPZGBpeHsXky8VQ5rCCZYVQ2lz+ftzifv2DSev1D4hSnAMnDCNoEgmmgPLFtgEsMFIDBJ0qkw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(6666004)(52116002)(54906003)(2616005)(36756003)(44832011)(956004)(26005)(53546011)(5660300002)(66476007)(33656002)(4326008)(316002)(8676002)(66946007)(6486002)(66556008)(38100700002)(966005)(8936002)(508600001)(83380400001)(6512007)(6916009)(186003)(86362001)(6506007)(30864003)(38350700002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6719
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	39a97e73-d111-47a0-dd14-08d97da7da4b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ixj3tOjMU3OWQ0kWAtP70k1gjpy/mT4THfu/3O/xf65r9wdVtdaq5X+AC9lOm6qtBeuw40ZM/re5Lazsq00c4yKKq+SoJa+ClVGweWKku64UZyllylkDCNO6v7qWrkBRcRb0l8pceGHrYquLXAjZ46TkL8ACEzR8PMs6lFWJjS0V9RkLtuSRDDGGqC1OOfs3/x8vIKkDVJGtM/kDXZ5e6HEICh4SAPi+0yjzkHm50x1vO0Djhn3zZGNhRw49FkYBY+gUAOGt2bEfuz1W/ONxUrihOytVcfwArv7bD/8hqGF0PZyDiJfC2vrC3AqNsTvavOspbG7cqqALFyVff4OfKmchYkz00fcTcCAy4vSls5C2mdISA/PIFs52dRgpjR6Y0PENeVCWIpMsdJBhsVFtby9+JCLQ4GYUH9/rrmwuotwmlNpj4h8xKTP9NJhHJf8GQVcdmERmfWFDNJrN9IyyZAGwiEIgbNpKsRv6f4Mbs9g/crufv2RHSvW5R2YHoWlJtRz6V0y9JSz+4szU5jP1nxkluHVsJYqd8QcoeSEofbYzBhk6Ktc49gjLRdcvdJsYeAPfRhPUHA5Yx9XjJQPEPdVPEKiM3VPWRnu09uK/gMJ4YvF7pOa6cyQ+GeL+P5CbHX2qXjPyq8Bdow/smn+s2zB8I+ksXp/iE+XmyYwuxJRvVir9mF0hoiMPyouNG2Lk38/0eovtFwxspYIAda+ehBXbKOikkaBmz9+APDSj8VlopAmOGWX7Pkztk9vysmsBKPyWMJCojFezJ8CcfD4p0QKsBbB9rTderzhaAvkvy7DtBu4VLYrTu+yWy0kRllLH
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(36756003)(966005)(36860700001)(53546011)(356005)(30864003)(33656002)(6666004)(6512007)(2906002)(8676002)(6486002)(6506007)(8936002)(82310400003)(83380400001)(186003)(26005)(2616005)(956004)(316002)(70206006)(336012)(86362001)(44832011)(54906003)(81166007)(4326008)(70586007)(47076005)(6862004)(5660300002)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:04:21.8513
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 62459d39-1037-4de6-29a9-08d97da7f7c2
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4767



> On 21 Sep 2021, at 22:34, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Tue, 21 Sep 2021, Luca Fancellu wrote:
>>> On 17 Sep 2021, at 23:33, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>>> On Fri, 17 Sep 2021, Luca Fancellu wrote:
>>>>> On 16 Sep 2021, at 21:16, Stefano Stabellini <sstabellini@kernel.org>=
 wrote:
>>>>>=20
>>>>> On Thu, 16 Sep 2021, Jan Beulich wrote:
>>>>>> On 16.09.2021 17:07, Luca Fancellu wrote:
>>>>>>> I explain here my understanding on dom0less, this feature is used t=
o start domUs at
>>>>>>> Xen boot in parallel, the name is misleading but it doesn=E2=80=99t=
 require dom0 to be absent.
>>>>>>>=20
>>>>>>> So if you have a dom0 kernel embed in the image, it's completely fi=
ne to start it and it=20
>>>>>>> doesn=E2=80=99t have to be skipped.
>>>>>>>=20
>>>>>>> Here the possible user cases:
>>>>>>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen imag=
e]
>>>>>>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or e=
mbedded in Xen image, domUs on DT]
>>>>>>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xe=
n image) and domUs on DT]
>>>>>>=20
>>>>>> If that's the intention - fine. Stefano?
>>>>>=20
>>>>=20
>>>> Hi Stefano,
>>>>=20
>>>>> What do you mean by dom0 modules embedded in the Xen image? I am not
>>>>> familiar with it, but I imagine it doesn't involve any multiboot,modu=
le
>>>>> nodes in device tree, right?
>>>>>=20
>>>>> Putting aside "dom0 modules embedded in Xen image" that I didn't full=
y
>>>>> understand, there are three ways to load Dom0:
>>>>>=20
>>>>> - dom0 kernel (and ramdisk, optional) on xen.cfg
>>>>> - dom0 kernel (and ramdisk, optional) on device tree using the "reg" =
property
>>>>> - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,=
binary" property
>>>>=20
>>>> True for the #1 and #2, the last one is not implemented. The uefi,bina=
ry property
>>>> for now is only used to load domU modules.
>>>=20
>>> Yeah, it is no problem that is not currently implemented, but from a
>>> device tree binding / efi interface perspective it should be possible.
>>>=20
>>>=20
>>>>> Then, the other use cases are:
>>>>>=20
>>>>> - true dom0less, domUs on device tree using the "reg" property
>>>>> - true dom0less, domUs on device tree using the "uefi,binary" propert=
y
>>>>>=20
>>>>> And of course all the possible combinations between Dom0 and DomU
>>>>> loading.
>>>>>=20
>>>>>=20
>>>>> Currently, patch #1 checks for the presence of a Dom0 kernel node and=
, if
>>>>> present, it decides not to proceed with xen.cfg. So if the Dom0 kerne=
l
>>>>> node is *not* present, efi_arch_use_config_file returns true.
>>>>>=20
>>>>> However, this could be a true dom0less configuration without any Dom0
>>>>> kernel. If so, you might not want to load xen.cfg at all because it i=
s
>>>>> not needed. In a true dom0less configuration, we probably want
>>>>> efi_arch_use_config_file to return false.
>>>>=20
>>>> In a true dom0less configuration we might need to read xen.cfg to retr=
ieve the
>>>> Xen command line,=20
>>>=20
>>> The Xen command line could also be on device tree
>>> (/chosen/xen,xen-bootargs).
>>>=20
>>>=20
>>>> but following the actual implementation of the common code
>>>> there is more. I=E2=80=99m going to explain.
>>>>=20
>>>> What efi_arch_use_config_file really does is not only choosing to read=
 xen.cfg
>>>> or not. Following the common code (xen/common/efi/boot.c) and what its=
 result activate
>>>> along the path, it basically decides if the UEFI stub is used as a loa=
der from filesystem
>>>> or not. We need the UEFI stub as a loader to be 100% UEFI and load our=
 modules.
>>>>=20
>>>> The original check basically says =E2=80=9Cif there are multiboot,modu=
le in the DT, then some
>>>> bootloader has loaded in memory the required modules so I=E2=80=99m no=
t gonna load anything
>>>> from the filesystem because I assume it puts everything in place for m=
e to boot.=E2=80=9D
>>>=20
>>> OK, I am following. It looks like this is the source of the issue.
>>>=20
>>>=20
>>>>> From misc/efi.txt:
>>>> When booted as an EFI application, Xen requires a configuration file a=
s described below unless a bootloader,
>>>> such as GRUB, has loaded the modules and describes them in the device =
tree provided to Xen. If a bootloader
>>>> provides a device tree containing modules then any configuration files=
 are ignored, and the bootloader is
>>>> responsible for populating all relevant device tree nodes.
>>>>=20
>>>> What I=E2=80=99m doing in patch #1 is restricting that check to just t=
he multiboot,module that are
>>>> Dom0 module, why? Because with the introduction of dom0less we need to=
 specify
>>>> multiboot,modules for domUs, but the presence or not of dom0 modules i=
s the only
>>>> Information we need to understand if the user decided to start Xen wit=
h everything
>>>> in places (modules in memory, xen command line, dtb) or if the job is =
demanded to the
>>>> UEFI stub and its configuration file.
>>>=20
>>> I don't think so. Imagine a case where the user has everything in devic=
e
>>> tree, doesn't need xen.cfg, but dom0 and domUs are specified as
>>> uefi,binary.
>>>=20
>>> We don't want xen.cfg but we do want to be able to load files from the
>>> filesystem. This might not be currently implemented but from an binding=
s
>>> perspective it should be possible.
>>>=20
>>>=20
>>>> By the configuration file you can also load in memory the Xen dtb, so =
Xen can
>>>> be started as an EFI application without the DTB and then load it usin=
g the EFI stub.
>>>=20
>>> This can be very useful but it would follow the !fdt check and return
>>> true from efi_arch_use_config_file. So it doesn't really conflict with
>>> anything we would around multiboot,module and xen,cfg-loading.
>>>=20
>>>=20
>>>> I=E2=80=99m not against this new property =E2=80=9Cxen,cfg-loading=E2=
=80=9D, I just think it is not needed because
>>>> we have all the information we need without it and in any case we need=
 to read the
>>>> configuration file because otherwise we won=E2=80=99t have access to t=
he Xen command line.
>>>=20
>>> We don't necessarely need to read the Xen command line from xen.cfg :-)
>>>=20
>>>=20
>>>> Now I=E2=80=99m going to show you examples of all use cases that are v=
alid with the introduction
>>>> of this serie:
>>>>=20
>>>> 1) Start Xen as EFI application and load only Dom0
>>>>=20
>>>>   Xen.cfg:
>>>>   [global]
>>>>   default=3Dxen_dom0
>>>>=20
>>>>   [xen_dom0]
>>>>   options=3D<Xen command line>
>>>>   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>>>   ramdisk=3Dinitrd-3.0.31-0.4-xen
>>>>   dtb=3D<xen DTB>
>>>>=20
>>>>   DT:
>>>>   {no modification}
>>>>=20
>>>> 2) Start Xen as EFI application to load a true dom0less setup
>>>>=20
>>>>   Xen.cfg:
>>>>   [global]
>>>>   default=3Dxen_true_dom0less
>>>>=20
>>>>   [xen_true_dom0less]
>>>>   options=3D<Xen command line>
>>>>   dtb=3D<xen DTB>
>>>>=20
>>>>   DT:
>>>>   chosen {
>>>>       #size-cells =3D <0x1>;
>>>> 	#address-cells =3D <0x1>;
>>>>=20
>>>> 	domU1 {
>>>>           #size-cells =3D <0x1>;=20
>>>>           #address-cells =3D <0x1>;
>>>>           compatible =3D "xen,domain=E2=80=9D;
>>>>           cpus =3D <1>;
>>>>           memory =3D <0 0xC0000>;
>>>>=20
>>>>           module@1 {
>>>>               compatible =3D "multiboot,kernel", "multiboot,module=E2=
=80=9D;
>>>>               bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=
=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
>>>>           };
>>>>=20
>>>>           module@2 {
>>>>               compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,m=
odule=E2=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
>>>>           };
>>>>=20
>>>>           module@3 {
>>>>               compatible =3D "multiboot,device-tree", "multiboot,modul=
e=E2=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
>>>>           };=20
>>>>       };
>>>>=20
>>>>       domU2 { <as above> };
>>>>   }
>>>>=20
>>>> 3) Start Xen as EFI application to load Dom0 and DomUs
>>>>=20
>>>>   Xen.cfg:
>>>>   [global]
>>>>   default=3Dxen_dom0_domUs
>>>>=20
>>>>   [xen_dom0_domUs]
>>>>   options=3D<Xen command line>
>>>>   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>>>   ramdisk=3Dinitrd-3.0.31-0.4-xen
>>>>   dtb=3D<xen DTB>
>>>>=20
>>>>   DT:
>>>>   chosen {
>>>>       #size-cells =3D <0x1>;
>>>> 	#address-cells =3D <0x1>;
>>>>=20
>>>> 	domU1 {
>>>>           #size-cells =3D <0x1>;=20
>>>>           #address-cells =3D <0x1>;
>>>>           compatible =3D "xen,domain=E2=80=9D;
>>>>           cpus =3D <1>;
>>>>           memory =3D <0 0xC0000>;
>>>>=20
>>>>           module@1 {
>>>>               compatible =3D "multiboot,kernel", "multiboot,module=E2=
=80=9D;
>>>>               bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=
=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
>>>>           };
>>>>=20
>>>>           module@2 {
>>>>               compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,m=
odule=E2=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
>>>>           };
>>>>=20
>>>>           module@3 {
>>>>               compatible =3D "multiboot,device-tree", "multiboot,modul=
e=E2=80=9D;
>>>>               uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
>>>>           };=20
>>>>       };
>>>>=20
>>>>       domU2 { <as above> };
>>>>   }
>>>>=20
>>>> So as you see every case is covered without the introduction of the
>>>> property.
>>>>=20
>>>> Please let me know what do you think.
>>>=20
>>=20
>> Hi Stefano,
>>=20
>>> I think that from an interface perspective (not a code perspective) we
>>> need to be able to account for cases like:
>>>=20
>>> 4) Start Xen as EFI application and load only Dom0
>>> no Xen.cfg
>>> DT:
>>> xen,xen-bootargs
>>> dom0/uefi,binary
>>> domUs/uefi,binary
>>>=20
>>>=20
>>> But in any case, even disregarding this case, past experience has taugh=
t
>>> me that it is always better to have an explicit flag to trigger a new
>>> behavior, rather than relying on "guesswork". If we introduce
>>> "xen,cfg-loading", we are going to be a lot more future-proof that if w=
e
>>> don't introduce it in terms of backward and forward compatibility in
>>> case we need to change anything.
>>=20
>> I see your point, for sure the DT is a more powerful tool than the simpl=
e
>> text configuration file and it would be the best interface.
>> However I think we are moving into the direction where x86 and arm
>> are going to diverge even if we can have a common interface for them
>> (the configuration file).
>=20
> Consider that the configuration file is not the only interface to Xen
> any longer. There is also HyperLaunch and I was trying to align to it:
> https://marc.info/?l=3Dxen-devel&m=3D162096368626246 The DT-based approac=
hed
> would be very well aligned with HyperLaunch.
>=20
>=20
>> For that reason I=E2=80=99m asking if you would be willing to accept a s=
olution
>> where we introduce a new keyword in the configuration file:
>>=20
>> dom0less=3D<dtb> OR domu_guests=3D<dtb> OR I=E2=80=99m open to suggestio=
n.
>>=20
>> Where the pointed dtb contains the domU domains:
>>=20
>> /dts-v1/;
>>=20
>> / {
>>    /* #*cells are here to keep DTC happy */
>>    #address-cells =3D <2>;
>>    #size-cells =3D <2>;
>>=20
>>    domU1 {
>>           #size-cells =3D <0x1>;=20
>>           #address-cells =3D <0x1>;
>>           compatible =3D "xen,domain=E2=80=9D;
>>           cpus =3D <1>;
>>           memory =3D <0 0xC0000>;
>>=20
>>           module@1 {
>>               compatible =3D "multiboot,kernel", "multiboot,module=E2=80=
=9D;
>>               bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw=E2=80=
=9D;
>>               uefi,binary =3D =E2=80=9CdomU_kernel1.bin"
>>           };
>>=20
>>           module@2 {
>>               compatible =3D =E2=80=9Cmultiboot,ramdisk", "multiboot,mod=
ule=E2=80=9D;
>>               uefi,binary =3D =E2=80=9CdomU_ramdisk1.bin"
>>           };
>>=20
>>           module@3 {
>>               compatible =3D "multiboot,device-tree", "multiboot,module=
=E2=80=9D;
>>               uefi,binary =3D =E2=80=9CdomU_passthrough1.dtb"
>>           };=20
>>    };
>>=20
>>    domU2 { <as above> };
>>=20
>> };
>>=20
>>=20
>> So that the user cases we discussed are valid:
>>=20
>> 1) Start Xen and load Dom0:
>>=20
>>   Xen.cfg:
>>   [global]
>>   default=3Dxen
>>=20
>>   [xen]
>>   options=3D<Xen command line>
>>   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>   ramdisk=3Dinitrd-3.0.31-0.4-xen
>>   dtb=3D<xen DTB>
>>=20
>> 2) Start Xen and load only domUs (true dom0less)
>>=20
>>   Xen.cfg:
>>   [global]
>>   default=3Dxen
>>=20
>>   [xen]
>>   options=3D<Xen command line>
>>   dom0less=3D<dom0less DTB>
>>   dtb=3D<xen DTB>
>>=20
>> 3) Start Xen and load Dom0 and DomUs
>>=20
>>   Xen.cfg:
>>   [global]
>>   default=3Dxen
>>=20
>>   [xen]
>>   options=3D<Xen command line>
>>   kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>>   ramdisk=3Dinitrd-3.0.31-0.4-xen
>>   dom0less=3D<dom0less DTB>
>>   dtb=3D<xen DTB>
>>=20
>>=20
>> With this change we will be consistent across x86 and arm UEFI boot
>> start procedure, we won=E2=80=99t touch the current check on multiboot,m=
odules
>> because it will be valid, we will have a way to boot dom0less and it
>> requires less testing for the changing in the common code.
>>=20
>> Please let me know what do you think about that.
>=20

Hi Stefano,

> My order of preference from best to worst is:
> 1) my previous suggestion, e.g. xen,cfg-loading

Thank you now I have the big picture, I will introduce the xen,cfg-load
In the v2 serie.

Cheers,
Luca

> 2) your previous suggestion, e.g. change the multiboot,modules check
>   without adding xen,cfg-loading
> 3) this proposal
>=20
>=20
> Let me explain the reason behind this. This proposal still requires a
> device tree but it has to be loaded from the config file, which is
> limiting compared to the other approaches. From a user perspective is
> just as complex (just as difficult to write) but less flexible because
> it prevents using the device tree alone for UEFI booting in the future.
> Given that with the two other alternatives the config file could still
> be used anyway, I don't think that adding the "dom0less" parameters to
> the config file buys us much in terms of alignment with x86. This is
> why this is my least favorite option.
>=20
> Your previous approach is actually quite good. Same complexity but much
> more flexible. Similar alignment with x86 in my view. The only
> correction I suggested is the addition of xen,cfg-loading to make the
> efi_arch_use_config_file check more robust. However, I still prefer your
> prevous approach even without xen,cfg-loading.
>=20
>=20
> Let me make one more suggestion based on your previous approach (I mean
> this version of the patch series):
>=20
> - You have already removed the panic for ARM in case a dom0 kernel is
>  not specifid in patch #2. We can go farther than that and make the
>  absence of xen.cfg not a fatal failure on ARM because it is fine not
>  to have dom0 in true dom0less configurations and the xen cmdline is
>  optional anyway.
>=20
> - If the absence of xen.cfg is not a fatal failure, then we don't need
>  efi_arch_use_config_file anymore. Let's try to load xen.cfg always. We
>  error out if xen.cfg specifies a dom0 kernel and we already have a
>  dom0 kernel in DT.
>=20
> - In the future a "don't load xen.cfg" option (the opposite of
>  xen,cfg-loading) could be added but it is not necessary now
>=20
>=20
> This should be a minimal change compared to this version of the patch
> series, enable all the use-cases you care about, and also be more
> flexible for the future.



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:06:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:06:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192300.342677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyCt-0005cH-3I; Wed, 22 Sep 2021 09:06:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192300.342677; Wed, 22 Sep 2021 09:06:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyCs-0005cA-Vt; Wed, 22 Sep 2021 09:06:22 +0000
Received: by outflank-mailman (input) for mailman id 192300;
 Wed, 22 Sep 2021 09:06:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=d5Wf=OM=epam.com=prvs=98995dc5f4=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mSyCq-0005c4-KD
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:06:20 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5902f6e8-1b84-11ec-b966-12813bfff9fa;
 Wed, 22 Sep 2021 09:06:18 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18M8sHNF027692; 
 Wed, 22 Sep 2021 09:06:16 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2106.outbound.protection.outlook.com [104.47.18.106])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b81cr823d-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 22 Sep 2021 09:06:16 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4435.eurprd03.prod.outlook.com (2603:10a6:208:c1::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 22 Sep
 2021 09:06:13 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 09:06:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5902f6e8-1b84-11ec-b966-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KB+i6JyYTbztBekKWkwfcraqmOuw+F+7awdsr3BKZ1FYZVmaYUeS+77kE8NHqfc3x76BaXd3s2sVmfHlsOZL1MJdohwjFsBdisNP/h+a42qw7MHUPoecLEf8F3w0FHoNRrwRGyhc+6adVmG0qBQFpdjbZHm4gSDQzBWsI9SWTlkp7VVKHfAzAFw2llkhIFqJkC2pmqFZwvRAl6vh8qiHYjc7EIvR2aHMfo6X4B7OOQ0M4r7sofYIgA9jMbBfRtofC1+N+u57f+ep+Csbmhh3RCO58J2BBNm1oBperAuRpEJfPOKGE5m1/DmoVXvzZwQXEX45vlokSMmwhIAAwFT7qQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gq9s4Gvde8S3W8XPUWTZYBzOuh5qowpHN0TABzU/wwk=;
 b=HCboOrPA7gBwoYXbWyn/bfAkK6QtkY6pFPBYXYSXxyKt6AkzZALFf3ZySuqd4oiDsSmO/o2wWP14pchEws3n+nSPZb2omuLzJ4R1ktT05FhJKgKH0/np6SU4d5j0xmJ6fLcyvkXQ7pyasYhzDbylJ6zSp/7yxLRFWlGPRcfGzLkCDnB1A9fMhFb7SygqDSEEZHsYQq4ADaQ09xBhw5BBO4jxnL+mjrgpxtI0wusWlLKGsB6GbnPWW3xy9gP7ekuyLc8Y2qwwIGRLbM4slDQ/i8hn8bca15sUIPY2wymlX/MAciv1VVxrZOEeMIVlwEos2eYyam3Cm13bdKsnTSlSLg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gq9s4Gvde8S3W8XPUWTZYBzOuh5qowpHN0TABzU/wwk=;
 b=fz/E5LDYUP8JAvcq0d+ycjQPzxpyrM2/y57EOx2KXd+ql+avfAdXTKe+rUccleCnHkJ7kiOUWh3T/fJp1s0VFGtPVoBJYXrQ7eHOKmwB+o0utxEJtMWQaGIsQFVpnP2oeq97EO8U5rUbI+zV70dCGEVi1TJySNQLGaW0vEfIsODkztNxkFvOylz+yUVMcqolNxJDJA+3kB0V86VzcDMxyeDMqttraJmMkdmYIAYUzeS0RLj9iiExhqE6+xZUoHQWIidg21m3e6WBTcXQFmpcw9ZxtTkVIS0rkdRQmTtGMTjzFDs+t3aGuU+CWhu1wx6ptaqFcNVoSP/OcjgMr36Fvw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Juergen Gross <jgross@suse.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com"
	<boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia Lukianenko
	<Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
Subject: Re: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH] xen-pciback: allow compiling on other archs than x86
Thread-Index: 
 AQHXq8QnOfGIukpGqE2hMWXmS9bdmKuow0KAgAOkhICAAGZ9AIAAAmOAgADCywCAAGXhAIAACLuAgAAEUwCAAAi6AIAAAxMAgAADDgCAAAJlAIAAAheAgADhlwCAAM9XAA==
Date: Wed, 22 Sep 2021 09:06:13 +0000
Message-ID: <9c81d293-1a30-3799-fc69-ba836751d53f@epam.com>
References: <20210917130123.1764493-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109171442070.21985@sstabellini-ThinkPad-T480s>
 <d81486bc-9a2b-8675-ba4d-828d3adc75fc@epam.com>
 <35e2e36a-bade-d801-faa1-c9953678bb9d@suse.com>
 <7f873e38-0362-1f60-7347-a490c9dc8572@epam.com>
 <alpine.DEB.2.21.2109201444040.17979@sstabellini-ThinkPad-T480s>
 <0f31a1bf-62b1-1aef-7b0f-34a1f6985fdb@suse.com>
 <82e55df9-74d3-6365-ab29-2bdfc4b74a1f@epam.com>
 <9b4962de-61ef-44dc-ffca-c54dd7990c6a@suse.com>
 <a9b98bc4-4c8a-2e7e-6abf-3a68025059c4@epam.com>
 <bb9fa2a8-9cc2-d83c-3659-c66b37781470@suse.com>
 <0b83aa77-aef0-0d98-cc0b-cf5f9c7599bd@epam.com>
 <111389e7-855d-0023-092c-f3e8bc57f4af@suse.com>
 <48a2ef20-02ad-99e4-a8f5-fa144692aadc@epam.com>
 <alpine.DEB.2.21.2109211340470.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109211340470.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2cf25e8d-605d-462a-6524-08d97da83a3b
x-ms-traffictypediagnostic: AM0PR03MB4435:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB4435E30572429BC5E961D1B3E7A29@AM0PR03MB4435.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 vOa1RoPxyf/p0ZLonNHJ0O6QqTOnANmib3Hc5RN+9Xn2OeLfPhnSPVbcQJ/ss/tDhSjJTqVAgSGYwG9SC1pUFDVS1ud5hh/GWkUS9UmqSLhp/ocmAY0n83KAVEM0pqryNkUCWA4bGrbhhGHOJX0BVAbCoTkwGybivUVwzOKGu5FdAikG5xoqEK5jRUnRCb/VnX1lze71+Q9PElArfT1snEo3Zd6lJf5ByHrS18giwlXw/A5K4wTNrFLaWgy6geMf+uooEZ8pwRcnvs1YTfoFiTSab3Imuf46mLryFjXfb3DUqwMEB/GTAF0b6TmCIEZCSua/WqIWZOlsK0GE1oO1vRDssmol+PrV1JCdRI4zwl+8+yTtOIqVyp/PKRnRfjmlMSVFMqlqLErabrm+eSdkvaPzDjaBOsKbJ6PQsOi4jcnQ8Q4iiL7Q9kGWeXj0MvPMMaHBTXQPRKfg7ZMwGTRDb4KIRv/FtqFBaV3+fLa6ERVdhKcc425nLtVRMAfEvfLrbNXYMgoiNmcmIV0oU1fqB4owiXoMrCo2nRhTzdHudhglXSq4TswkWUB6RBR0QfbA0tz0eNvUW1w4nZBpVVKBw+7M0yHZuVAKDo5Qyo/0bkfghWfY3PxPr9qIr+LFAtrIb+TgPJPhkW5YWF/DsXMJxLSE+874Xak/Y7nLQk5QQvSjgnydenBB9kpkqx+8r6NyKaStoQG6u4kKpszBiLAM89NVIwqsh0e9SWz3b18ZL0F/NQUADlRRAR/v4DzHhyVm
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(122000001)(5660300002)(2616005)(76116006)(30864003)(2906002)(6486002)(83380400001)(91956017)(4326008)(31686004)(66946007)(64756008)(31696002)(6506007)(86362001)(71200400001)(36756003)(38100700002)(54906003)(186003)(8936002)(26005)(508600001)(316002)(66476007)(8676002)(66446008)(53546011)(6916009)(66556008)(6512007)(55236004)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Rk1OQk5TRkg5Umx3VDRVU3lJRmI0QmtCOXAzR0NleWhIMm0vOUFrWkNoNTh1?=
 =?utf-8?B?ZnZIVW0xOUJFTUo2OVFMRXRnVWlhTitwQXkxVWRaaXJ0aHJWeGNoeTF3cWw2?=
 =?utf-8?B?ZHZZMHk0OHJGWDRKVUZFMlV3T3FYUkNud0U3eXJpdU96V3dYc1lJVUFGQUlx?=
 =?utf-8?B?WThtWkNaV0FkdzF4UEcrclhvZjNUNVU1MDBUU25uUzlENndGNzMxUXYvWFpo?=
 =?utf-8?B?aUR6V1FpZHUxQ016MDhGZ2MwaVdtOGtYUy9uWUxDamxnelEwUXdSMVhGUXpl?=
 =?utf-8?B?MW1ia21OeCt5TVRVRE9IZ05xeTdUS2N0U1djM2h0R0FSbTZLbGxMQThFMGdQ?=
 =?utf-8?B?L1ZmYU14d2p3YkdXUDVHazdiQkdvQ2QxS1NyMC9CUmhkWmliSWgxVXRnVkln?=
 =?utf-8?B?bG9mcXNJU2dTMk93SmM4enFlUEVMNlE1THRnblp3aUo1bXZ0cUJnYTNncDIw?=
 =?utf-8?B?Sy9uY0xrVXhGL3ExSHJRUFlsWC9ZajhDRGdXREpmKy8xS2JWbUtkeGozT0Fp?=
 =?utf-8?B?VDlCMGdwQ0ZmNzZVTTN3NkhLWUMrTlYxVTlGQmNUR0NlY0dJc0lPRndRN21C?=
 =?utf-8?B?SVpTTXRNclVDYVJodWhpMWhwUExpRHV0RjNVTHE1RFRJYUR5UjVja2hCdWhu?=
 =?utf-8?B?MEgvbi9sRExSNCtKQk9aWXZqWDBYNzFTeFJUd1FWME44c1dja3I0T2R2Y0tk?=
 =?utf-8?B?elZORWpGY1h3MnRsNkR6ekw0d2srU1U4a3RzQUxwNWJrVWR0bDVTVUxwZzFK?=
 =?utf-8?B?dDBzc05FaWgvcXI1NVNXZlh4OTdTQWtzcENpa21QTS9GQ3NZTzRWVjZMbXFx?=
 =?utf-8?B?dlIzUlJVTVZJSHBOTVNEeTRvbVRXOXBUS0R3bkZOMm5PelJlVVZzdWZ2aGhq?=
 =?utf-8?B?dS96OFdCcHJKZXNzNHIxTnJ1bVhPWS9QQ3Q5REFKZVM3VmxNay91cEc4bE5V?=
 =?utf-8?B?cTVGTDJHS2NRTWpHSUJEemtwRS96bC9tR0tXKzk4WE1hSGtKSU9LWXIxSEgr?=
 =?utf-8?B?RkY2S2s0RUZTY2tlcVJjOXpkeVZ4enFJMW9DNDhoZ2ZVYjlCakdjRWpIRWVY?=
 =?utf-8?B?MlBZcEVRbDlBVmxzcUdOOEJESVhwRTFyOEV2ZzRScEtmZE9hNXFxanFFVlFO?=
 =?utf-8?B?b0htNzNESDRNZld4d3lDblhkSTJ0TEVEZ3hHTzhFbTh4SVlOTGUwOEFzSTIz?=
 =?utf-8?B?aGtXcXN3WHJRZDFmbVI4Q0NyemxjMFJrUXp3MG1EeTdsWFlFNGN3SnF4VXJF?=
 =?utf-8?B?Rk5VM1czZjByMGpMZ0M2WVR4WjN5Wm1FcXBDcHJSVVBKMEFzamZNZUVRN0Zr?=
 =?utf-8?B?RjVjNjNRaWJyTFdZUHhWZEdjUEJWdXhKcTFWWlhSM1lTNnkwcURhWTViVkpU?=
 =?utf-8?B?b1VhTjNYNTRxdEprbno1U09yNVZ3aG1nQUFnYkV2U1Z0VzZKaTlKbnNia2hj?=
 =?utf-8?B?cDRIQ3dsTHlTVmtFNlREWEM4M205MklsOXlRaDRKODluSEVWR2VtbWwrVVpm?=
 =?utf-8?B?dnp5cGpsUlY0M3BsNE9iRU5hVjZoaWpBb0JBTGRiT2tJWitiUkRpTDdqYytT?=
 =?utf-8?B?bEdrSldEanVOaGpFRUJJTUNkd3F5c2FHbVFiREI2L3JnK3pPeFdZTWhDMlpE?=
 =?utf-8?B?UldJUG53S2czOFp3YVpMOThGSVNxcXVZU3UvcWRiMzREVE1kZDdIaElzKzE4?=
 =?utf-8?B?M0t4SEpmR3pkYjBIVlArZmtPSldJdXNTR2crTEFYeDZ5SndmTFgzNWlRb2gx?=
 =?utf-8?B?dE5mNVpMVDgySHNLWkxISWphZ1ZRTGgxWTdudlp2ZWdXQzNxMi9ObnZEeUJn?=
 =?utf-8?B?eWVsUzRwcHZJbXJUOWg1dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <0D0C20E792A0BB48A5C940A4D69973D3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2cf25e8d-605d-462a-6524-08d97da83a3b
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 09:06:13.3897
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: M3gPpeEZqG8/zKfKhvDr4ngrmdDTzzcSG7uE7NbV2xEkGCg4IV9fTVuTuBt2NyHLMv3aiLY8CtjfuR5EaG88p9aw1AxgNaMLM/0Qp7OxTC8ys41FXUwRv4pJ16YedWQ6
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4435
X-Proofpoint-ORIG-GUID: 056SdVnT0tFM4h1RxJhz7q1d8QnreGqa
X-Proofpoint-GUID: 056SdVnT0tFM4h1RxJhz7q1d8QnreGqa
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-22_03,2021-09-20_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0
 priorityscore=1501 lowpriorityscore=0 clxscore=1015 phishscore=0
 malwarescore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 bulkscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220062

DQpPbiAyMS4wOS4yMSAyMzo0NCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUdWUs
IDIxIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IE9uIDIxLjA5
LjIxIDEwOjA5LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDIxLjA5LjIxIDA5OjAwLCBP
bGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4gT24gMjEuMDkuMjEgMDk6NDksIEp1
ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4+PiBPbiAyMS4wOS4yMSAwODozOCwgT2xla3NhbmRyIEFu
ZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+Pj4gT24gMjEuMDkuMjEgMDk6MDcsIEp1ZXJnZW4gR3Jv
c3Mgd3JvdGU6DQo+Pj4+Pj4+IE9uIDIxLjA5LjIxIDA3OjUxLCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+Pj4+IE9uIDIxLjA5LjIxIDA4OjIwLCBKdWVyZ2VuIEdyb3NzIHdy
b3RlOg0KPj4+Pj4+Pj4+IE9uIDIxLjA5LjIxIDAxOjE2LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3Jv
dGU6DQo+Pj4+Pj4+Pj4+IE9uIE1vbiwgMjAgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hl
bmtvIHdyb3RlOg0KPj4+Pj4+Pj4+Pj4gT24gMjAuMDkuMjEgMTQ6MzAsIEp1ZXJnZW4gR3Jvc3Mg
d3JvdGU6DQo+Pj4+Pj4+Pj4+Pj4gT24gMjAuMDkuMjEgMDc6MjMsIE9sZWtzYW5kciBBbmRydXNo
Y2hlbmtvIHdyb3RlOg0KPj4+Pj4+Pj4+Pj4+PiBIZWxsbywgU3RlZmFubyENCj4+Pj4+Pj4+Pj4+
Pj4NCj4+Pj4+Pj4+Pj4+Pj4gT24gMTguMDkuMjEgMDA6NDUsIFN0ZWZhbm8gU3RhYmVsbGluaSB3
cm90ZToNCj4+Pj4+Pj4+Pj4+Pj4+IEhpIE9sZWtzYW5kciwNCj4+Pj4+Pj4+Pj4+Pj4+DQo+Pj4+
Pj4+Pj4+Pj4+PiBXaHkgZG8geW91IHdhbnQgdG8gZW5hYmxlIHBjaWJhY2sgb24gQVJNPyBJcyBp
dCBvbmx5IHRvICJkaXNhYmxlIiBhIFBDSQ0KPj4+Pj4+Pj4+Pj4+Pj4gZGV2aWNlIGluIERvbTAg
c28gdGhhdCBpdCBjYW4gYmUgc2FmZWx5IGFzc2lnbmVkIHRvIGEgRG9tVT8NCj4+Pj4+Pj4+Pj4+
Pj4gTm90IG9ubHkgdGhhdA0KPj4+Pj4+Pj4+Pj4+Pj4gSSBhbSBhc2tpbmcgYmVjYXVzZSBhY3R1
YWxseSBJIGRvbid0IHRoaW5rIHdlIHdhbnQgdG8gZW5hYmxlIHRoZSBQViBQQ0kNCj4+Pj4+Pj4+
Pj4+Pj4+IGJhY2tlbmQgZmVhdHVyZSBvZiBwY2liYWNrIG9uIEFSTSwgcmlnaHQ/IFRoYXQgd291
bGQgY2xhc2ggd2l0aCB0aGUgUENJDQo+Pj4+Pj4+Pj4+Pj4+PiBhc3NpZ25tZW50IHdvcmsgeW91
IGhhdmUgYmVlbiBkb2luZyBpbiBYZW4uIFRoZXkgY291bGRuJ3QgYm90aCB3b3JrIGF0DQo+Pj4+
Pj4+Pj4+Pj4+PiB0aGUgc2FtZSB0aW1lLg0KPj4+Pj4+Pj4+Pj4+PiBDb3JyZWN0LCBpdCBpcyBu
b3QgdXNlZA0KPj4+Pj4+Pj4+Pj4+Pj4gSWYgd2Ugb25seSBuZWVkIHBjaWJhY2sgdG8gInBhcmsi
IGEgZGV2aWNlIGluIERvbTAsIHdvdWxkbid0IGl0IGJlDQo+Pj4+Pj4+Pj4+Pj4+PiBwb3NzaWJs
ZSBhbmQgYmV0dGVyIHRvIHVzZSBwY2ktc3R1YiBpbnN0ZWFkPw0KPj4+Pj4+Pj4+Pj4+PiBOb3Qg
b25seSB0aGF0LCBzbyBwY2ktc3R1YiBpcyBub3QgZW5vdWdoDQo+Pj4+Pj4+Pj4+Pj4+DQo+Pj4+
Pj4+Pj4+Pj4+IFRoZSBmdW5jdGlvbmFsaXR5IHdoaWNoIGlzIGltcGxlbWVudGVkIGJ5IHRoZSBw
Y2liYWNrIGFuZCB0aGUgdG9vbHN0YWNrDQo+Pj4+Pj4+Pj4+Pj4+IGFuZCB3aGljaCBpcyByZWxl
dmFudC9taXNzaW5nL25lZWRlZCBmb3IgQVJNOg0KPj4+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4+
PiAxLiBwY2liYWNrIGlzIHVzZWQgYXMgYSBkYXRhYmFzZSBmb3IgYXNzaWduYWJsZSBQQ0kgZGV2
aWNlcywgZS5nLiB4bA0KPj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoCDCoMKgIHBjaS1hc3NpZ25h
YmxlLXthZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQgbGlzdC4gU28sIHdoZW5ldmVy
IHRoZQ0KPj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoCDCoMKgIHRvb2xzdGFjayBuZWVkcyB0byBr
bm93IHdoaWNoIFBDSSBkZXZpY2VzIGNhbiBiZSBwYXNzZWQgdGhyb3VnaCBpdCByZWFkcw0KPj4+
Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoCDCoMKgIHRoYXQgZnJvbSB0aGUgcmVsZXZhbnQgc3lzZnMg
ZW50cmllcyBvZiB0aGUgcGNpYmFjay4NCj4+Pj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4+Pj4gMi4g
cGNpYmFjayBpcyB1c2VkIHRvIGhvbGQgdGhlIHVuYm91bmQgUENJIGRldmljZXMsIGUuZy4gd2hl
biBwYXNzaW5nIHRocm91Z2gNCj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgwqDCoCBhIFBDSSBk
ZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFudCBkZXZpY2UgZHJp
dmVyIGFuZCBib3VuZA0KPj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoCDCoMKgIHRvIHBjaWJhY2sg
KHN0cmljdGx5IHNwZWFraW5nIGl0IGlzIG5vdCByZXF1aXJlZCB0aGF0IHRoZSBkZXZpY2UgaXMg
Ym91bmQgdG8NCj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgwqDCoCBwY2liYWNrLCBidXQgcGNp
YmFjayBpcyBhZ2FpbiB1c2VkIGFzIGEgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBD
SQ0KPj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoCDCoMKgIGRldmljZXMsIHNvIHdlIGNhbiByZS1i
aW5kIHRoZSBkZXZpY2VzIGJhY2sgdG8gdGhlaXIgb3JpZ2luYWwgZHJpdmVycyB3aGVuDQo+Pj4+
Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgIMKgwqAgZ3Vlc3QgZG9tYWluIHNodXRzIGRvd24pDQo+Pj4+
Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+Pj4+IDMuIERldmljZSByZXNldA0KPj4+Pj4+Pj4+Pj4+Pg0K
Pj4+Pj4+Pj4+Pj4+PiBXZSBoYXZlIHByZXZpb3VzbHkgZGlzY3Vzc2VkIG9uIHhlbi1kZXZlbCBN
TCBwb3NzaWJsZSBzb2x1dGlvbnMgdG8gdGhhdCBhcyBmcm9tIHRoZQ0KPj4+Pj4+Pj4+Pj4+PiBh
Ym92ZSB3ZSBzZWUgdGhhdCBwY2liYWNrIGZ1bmN0aW9uYWxpdHkgaXMgZ29pbmcgdG8gYmUgb25s
eSBwYXJ0aWFsbHkgdXNlZCBvbiBBcm0uDQo+Pj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+Pj4+IFBs
ZWFzZSBzZWUgWzFdIGFuZCBbMl06DQo+Pj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+Pj4+IDEuIEl0
IGlzIG5vdCBhY2NlcHRhYmxlIHRvIG1hbmFnZSB0aGUgYXNzaWduYWJsZSBsaXN0IGluIFhlbiBp
dHNlbGYNCj4+Pj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4+Pj4gMi4gcGNpYmFjayBjYW4gYmUgc3Bs
aXQgaW50byB0d28gcGFydHM6IFBDSSBhc3NpZ25hYmxlL2JpbmQvcmVzZXQgaGFuZGxpbmcgYW5k
DQo+Pj4+Pj4+Pj4+Pj4+IHRoZSByZXN0IGxpa2UgdlBDSSBldGMuDQo+Pj4+Pj4+Pj4+Pj4+DQo+
Pj4+Pj4+Pj4+Pj4+IDMuIHBjaWZyb250IGlzIG5vdCB1c2VkIG9uIEFybQ0KPj4+Pj4+Pj4+Pj4+
IEl0IGlzIG5laXRoZXIgaW4geDg2IFBWSC9IVk0gZ3Vlc3RzLg0KPj4+Pj4+Pj4+Pj4gRGlkbid0
IGtub3cgdGhhdCwgdGhhbmsgeW91IGZvciBwb2ludGluZw0KPj4+Pj4+Pj4+Pj4+PiBTbywgbGlt
aXRlZCB1c2Ugb2YgdGhlIHBjaWJhY2sgaXMgb25lIG9mIHRoZSBicmlja3MgdXNlZCB0byBlbmFi
bGUgUENJIHBhc3N0aHJvdWdoDQo+Pj4+Pj4+Pj4+Pj4+IG9uIEFybS4gSXQgd2FzIGVub3VnaCB0
byBqdXN0IHJlLXN0cnVjdHVyZSB0aGUgZHJpdmVyIGFuZCBoYXZlIGl0IHJ1biBvbiBBcm0gdG8g
YWNoaWV2ZQ0KPj4+Pj4+Pj4+Pj4+PiBhbGwgdGhlIGdvYWxzIGFib3ZlLg0KPj4+Pj4+Pj4+Pj4+
Pg0KPj4+Pj4+Pj4+Pj4+PiBJZiB3ZSBzdGlsbCB0aGluayBpdCBpcyBkZXNpcmFibGUgdG8gYnJl
YWsgdGhlIHBjaWJhY2sgZHJpdmVyIGludG8gImNvbW1vbiIgYW5kICJwY2lmcm9udCBzcGVjaWZp
YyINCj4+Pj4+Pj4+Pj4+Pj4gcGFydHMgdGhlbiBpdCBjYW4gYmUgZG9uZSwgeWV0IHRoZSBwYXRj
aCBpcyBnb2luZyB0byBiZSB0aGUgdmVyeSBmaXJzdCBicmljayBpbiB0aGF0IGJ1aWxkaW5nLg0K
Pj4+Pj4+Pj4+Pj4+IERvaW5nIHRoaXMgc3BsaXQgc2hvdWxkIGJlIGRvbmUsIGFzIHRoZSBwY2lm
cm9udCBzcGVjaWZpYyBwYXJ0IGNvdWxkIGJlDQo+Pj4+Pj4+Pj4+Pj4gb21pdHRlZCBvbiB4ODYs
IHRvbywgaW4gY2FzZSBubyBQViBndWVzdHMgdXNpbmcgUENJIHBhc3N0aHJvdWdoIGhhdmUgdG8N
Cj4+Pj4+Pj4+Pj4+PiBiZSBzdXBwb3J0ZWQuDQo+Pj4+Pj4+Pj4+PiBBZ3JlZSwgdGhhdCB0aGUg
ZmluYWwgc29sdXRpb24gc2hvdWxkIGhhdmUgdGhlIGRyaXZlciBzcGxpdA0KPj4+Pj4+Pj4+Pj4+
PiBTbywgSSB0aGluayB0aGlzIHBhdGNoIGlzIHN0aWxsIGdvaW5nIHRvIGJlIG5lZWRlZCBiZXNp
ZGVzIHdoaWNoIGRpcmVjdGlvbiB3ZSB0YWtlLg0KPj4+Pj4+Pj4+Pj4+IFNvbWUga2luZCBvZiB0
aGlzIHBhdGNoLCB5ZXMuIEl0IG1pZ2h0IGxvb2sgZGlmZmVyZW50IGluIGNhc2UgdGhlIHNwbGl0
DQo+Pj4+Pj4+Pj4+Pj4gaXMgZG9uZSBmaXJzdC4NCj4+Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+Pj4+
IEkgZG9uJ3QgbWluZCBkb2luZyBpdCBpbiBlaXRoZXIgc2VxdWVuY2UuDQo+Pj4+Pj4+Pj4+Pj4N
Cj4+Pj4+Pj4+Pj4+IFdpdGggdGhpcyBwYXRjaCB3ZSBoYXZlIEFybSBvbiB0aGUgc2FtZSBwYWdl
IGFzIHRoZSBhYm92ZSBtZW50aW9uZWQgeDg2IGd1ZXN0cywNCj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+
Pj4+PiBlLmcuIHRoZSBkcml2ZXIgaGFzIHVudXNlZCBjb2RlLCBidXQgeWV0IGFsbG93cyBBcm0g
dG8gZnVuY3Rpb24gbm93Lg0KPj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4+IEF0IHRoaXMgc3RhZ2Ug
b2YgUENJIHBhc3N0aHJvdWdoIG9uIEFybSBpdCBpcyB5ZXQgZW5vdWdoLiBMb25nIHRlcm0sIHdo
ZW4NCj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiB0aGUgZHJpdmVyIGdldHMgc3BsaXQsIEFybSB3
aWxsIGJlbmVmaXQgZnJvbSB0aGF0IHNwbGl0IHRvbywgYnV0IHVuZm9ydHVuYXRlbHkgSSBkbyBu
b3QNCj4+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+PiBoYXZlIGVub3VnaCBiYW5kd2lkdGggZm9yIHRo
YXQgcGllY2Ugb2Ygd29yayBhdCB0aGUgbW9tZW50Lg0KPj4+Pj4+Pj4+PiBUaGF0J3MgZmFpciBh
bmQgSSBkb24ndCB3YW50IHRvIHNjb3BlLWNyZWVwIHRoaXMgc2ltcGxlIHBhdGNoIGFza2luZyBm
b3INCj4+Pj4+Pj4+Pj4gYW4gZW5vcm1vdXMgcmV3b3JrLiBBdCB0aGUgc2FtZSB0aW1lIEkgZG9u
J3QgdGhpbmsgd2Ugc2hvdWxkIGVuYWJsZSB0aGUNCj4+Pj4+Pj4+Pj4gd2hvbGUgb2YgcGNpYmFj
ayBvbiBBUk0gYmVjYXVzZSBpdCB3b3VsZCBiZSBlcnJvbmVvdXMgYW5kIGNvbmZ1c2luZy4NCj4+
Pj4+Pj4+IEFzIHRoZSBmaXJzdCBzdGFnZSBiZWZvcmUgdGhlIGRyaXZlciBpcyBzcGxpdCBvciBp
ZmRlZidzIHVzZWQgLSBjYW4gd2UgdGFrZSB0aGUgcGF0Y2gNCj4+Pj4+Pj4+IGFzIGlzIG5vdz8g
SW4gZWl0aGVyIHdheSB3ZSBjaG9zZSB0aGlzIG5lZWRzIHRvIGJlIGRvbmUsIGUuZy4gZW5hYmxl
IGNvbXBpbGluZw0KPj4+Pj4+Pj4gZm9yIG90aGVyIGFyY2hpdGVjdHVyZXMgYW5kIGNvbW1vbiBj
b2RlIG1vdmUuDQo+Pj4+Pj4+IEZpbmUgd2l0aCBtZSBpbiBwcmluY2lwbGUuIEkgbmVlZCB0byB0
YWtlIGEgbW9yZSB0aG9yb3VnaCBsb29rDQo+Pj4+Pj4+IGF0IHRoZSBwYXRjaCwgdGhvdWdoLg0K
Pj4+Pj4+IE9mIGNvdXJzZQ0KPj4+Pj4+Pj4+PiBJIGFtIHdvbmRlciBpZiB0aGVyZSBpcyBhIHNp
bXBsZToNCj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4gaWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+Pj4+
Pj4+Pj4+ICDCoMKgwqDCoMKgwqDCoCByZXR1cm47DQo+Pj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+IFRo
YXQgd2UgY291bGQgYWRkIGluIGEgY291cGxlIG9mIHBsYWNlcyBpbiBwY2liYWNrIHRvIHN0b3Ag
aXQgZnJvbQ0KPj4+Pj4+Pj4+PiBpbml0aWFsaXppbmcgdGhlIHBhcnRzIHdlIGRvbid0IGNhcmUg
YWJvdXQuIFNvbWV0aGluZyBhbG9uZyB0aGVzZSBsaW5lcw0KPj4+Pj4+Pj4+PiAodW50ZXN0ZWQg
YW5kIHByb2JhYmx5IGluY29tcGxldGUpLg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBXaGF0IGRv
IHlvdSBndXlzIHRoaW5rPw0KPj4+Pj4+Pj4+IFVoIG5vLCBub3QgaW4gdGhpcyB3YXksIHBsZWFz
ZS4gVGhpcyB3aWxsIGtpbGwgcGNpIHBhc3N0aHJvdWdoIG9uIHg4Ng0KPj4+Pj4+Pj4+IHdpdGgg
ZG9tMCBydW5uaW5nIGFzIFBWSC4gSSBkb24ndCB0aGluayB0aGlzIGlzIHdvcmtpbmcgcmlnaHQg
bm93LCBidXQNCj4+Pj4+Pj4+PiBhZGRpbmcgbW9yZSBjb2RlIG1ha2luZyBpdCBldmVuIGhhcmRl
ciB0byB3b3JrIHNob3VsZCBiZSBhdm9pZGVkLg0KPj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+IGRpZmYg
LS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYyBiL2RyaXZlcnMveGVuL3hl
bi1wY2liYWNrL3hlbmJ1cy5jDQo+Pj4+Pj4+Pj4+IGluZGV4IGRhMzRjZTg1ZGM4OC4uOTkxYmEw
YTliMzU5IDEwMDY0NA0KPj4+Pj4+Pj4+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94
ZW5idXMuYw0KPj4+Pj4+Pj4+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMu
Yw0KPj4+Pj4+Pj4+PiBAQCAtMTUsNiArMTUsNyBAQA0KPj4+Pj4+Pj4+PiAgwqDCoMKgwqAgI2lu
Y2x1ZGUgPHhlbi94ZW5idXMuaD4NCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgICNpbmNsdWRlIDx4ZW4v
ZXZlbnRzLmg+DQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoCAjaW5jbHVkZSA8eGVuL3BjaS5oPg0KPj4+
Pj4+Pj4+PiArI2luY2x1ZGUgPHhlbi94ZW4uaD4NCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgICNpbmNs
dWRlICJwY2liYWNrLmgiDQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoCDCoCAjZGVmaW5lIElOVkFMSURf
RVZUQ0hOX0lSUcKgICgtMSkNCj4+Pj4+Pj4+Pj4gQEAgLTY4NSw4ICs2ODYsMTIgQEAgc3RhdGlj
IGludCB4ZW5fcGNpYmtfeGVuYnVzX3Byb2JlKHN0cnVjdCB4ZW5idXNfZGV2aWNlICpkZXYsDQo+
Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGNvbnN0
IHN0cnVjdCB4ZW5idXNfZGV2aWNlX2lkICppZCkNCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgIHsNCj4+
Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoMKgwqAgaW50IGVyciA9IDA7DQo+Pj4+Pj4+Pj4+IC3CoMKg
wqAgc3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXYgPSBhbGxvY19wZGV2KGRldik7DQo+Pj4+
Pj4+Pj4+ICvCoMKgwqAgc3RydWN0IHhlbl9wY2lia19kZXZpY2UgKnBkZXY7DQo+Pj4+Pj4+Pj4+
ICsNCj4+Pj4+Pj4+Pj4gK8KgwqDCoCBpZiAoIXhlbl9wdl9kb21haW4oKSkNCj4+Pj4+Pj4+Pj4g
K8KgwqDCoMKgwqDCoMKgIHJldHVybiAwOw0KPj4+Pj4+Pj4+PiAgwqDCoMKgwqAgK8KgwqDCoCBw
ZGV2ID0gYWxsb2NfcGRldihkZXYpOw0KPj4+Pj4+Pj4+IFRoaXMgaHVuayBpc24ndCBuZWVkZWQs
IGFzIHdpdGggYmFpbGluZyBvdXQgb2YgeGVuX3BjaWJrX3hlbmJ1c19yZWdpc3Rlcg0KPj4+Pj4+
Pj4+IGVhcmx5IHdpbGwgcmVzdWx0IGluIHhlbl9wY2lia194ZW5idXNfcHJvYmUgbmV2ZXIgYmVp
bmcgY2FsbGVkLg0KPj4+Pj4+Pj4+DQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqDCoMKgIGlmIChw
ZGV2ID09IE5VTEwpIHsNCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBlcnIg
PSAtRU5PTUVNOw0KPj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHhlbmJ1c19k
ZXZfZmF0YWwoZGV2LCBlcnIsDQo+Pj4+Pj4+Pj4+IEBAIC03NDMsNiArNzQ4LDkgQEAgY29uc3Qg
c3RydWN0IHhlbl9wY2lia19iYWNrZW5kICpfX3JlYWRfbW9zdGx5IHhlbl9wY2lia19iYWNrZW5k
Ow0KPj4+Pj4+Pj4+PiAgwqDCoMKgwqAgwqAgaW50IF9faW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3Jl
Z2lzdGVyKHZvaWQpDQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoCB7DQo+Pj4+Pj4+Pj4+ICvCoMKgwqAg
aWYgKCF4ZW5fcHZfZG9tYWluKCkpDQo+Pj4+Pj4+Pj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4g
MDsNCj4+Pj4+Pj4+Pj4gKw0KPj4+Pj4+Pj4+IFVzZSAjaWZkZWYgQ09ORklHX1g4NiBpbnN0ZWFk
Lg0KPj4+Pj4+Pj4gVGhlIHRpdGxlIG9mIHRoaXMgcGF0Y2ggc2F5cyB0aGF0IHdlIHdhbnQgdG8g
YWxsb3cgdGhpcyBkcml2ZXIgZm9yIG90aGVyIGFyY2hzDQo+Pj4+Pj4+PiBhbmQgbm93IHdlIHdh
bnQgdG8gaW50cm9kdWNlICIjaWZkZWYgQ09ORklHX1g4NiIgd2hpY2ggZG9lc24ndCBzb3VuZA0K
Pj4+Pj4+Pj4gcmlnaHQgd2l0aCB0aGF0IHJlc3BlY3QuIEluc3RlYWQsIHdlIG1heSB3YW50IGhh
dmluZyBzb21ldGhpbmcgbGlrZSBhDQo+Pj4+Pj4+PiBkZWRpY2F0ZWQgZ2F0ZSBmb3IgdGhpcywg
ZS5nLiAiI2lmZGVmIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkRfU1VQUF9QViINCj4+Pj4+Pj4+
IG9yIHNvbWV0aGluZyB3aGljaCBpcyBhcmNoaXRlY3R1cmUgYWdub3N0aWMuDQo+Pj4+Pj4+IFNv
bWV0aGluZyBsaWtlIHRoYXQsIHllcy4gQnV0IEknZCByYXRoZXIgdXNlIENPTkZJR19YRU5fUENJ
REVWX0JBQ0tFTkQNCj4+Pj4+Pj4gYWN0aW5nIGFzIHRoaXMgZ2F0ZSBhbmQgaW50cm9kdWNlIENP
TkZJR19YRU5fUENJX1NUVUIgZm9yIHRoZSBzdHViDQo+Pj4+Pj4+IGZ1bmN0aW9uYWxpdHkgbmVl
ZGVkIG9uIEFybS4gWEVOX1BDSURFVl9CQUNLRU5EIHdvdWxkIGRlcGVuZCBvbiBYODYgYW5kDQo+
Pj4+Pj4+IHNlbGVjdCBYRU5fUENJX1NUVUIsIHdoaWxlIG9uIEFybSBYRU5fUENJX1NUVUIgY291
bGQgYmUgY29uZmlndXJlZCBpZg0KPj4+Pj4+PiB3YW50ZWQuIFRoZSBzcGxpdHRpbmcgb2YgdGhl
IGRyaXZlciBjYW4gc3RpbGwgYmUgZG9uZSBsYXRlci4NCj4+Pj4+PiBIbSwgcGNpYmFjayBpcyBu
b3cgY29tcGlsZWQgd2hlbiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EIGlzIGVuYWJsZWQNCj4+
Pj4+PiBhbmQgd2Ugd2FudCB0byBza2lwIHNvbWUgcGFydHMgb2YgaXRzIGNvZGUgd2hlbiBDT05G
SUdfWEVOX1BDSV9TVFVCIGlzIHNldC4NCj4+Pj4+PiBTbywgSSBpbWFnaW5lIHRoYXQgZm9yIHg4
NiB3ZSBqdXN0IGVuYWJsZSBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EIGFuZCB0aGUNCj4+Pj4+
PiBkcml2ZXIgY29tcGlsZXMgaW4gaXRzIGN1cnJlbnQgc3RhdGUuIEZvciBBcm0gd2UgZW5hYmxl
IGJvdGggQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORA0KPj4+Pj4+IGFuZCBDT05GSUdfWEVOX1BD
SV9TVFVCLCBzbyBwYXJ0IG9mIHRoZSBkcml2ZXIgaXMgbm90IGNvbXBpbGVkLg0KPj4+Pj4gTm8s
IEknZCByYXRoZXIgc3dpdGNoIHRvIGNvbXBpbGluZyB4ZW4tcGNpYmFjayB3aGVuIENPTkZJR19Y
RU5fUENJX1NUVUINCj4+Pj4+IGlzIHNldCBhbmQgY29tcGlsZSBvbmx5IHBhcnRzIG9mIGl0IHdo
ZW4gQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCBpcw0KPj4+Pj4gbm90IHNldCAodGhpcyB3aWxs
IGJlIHRoZSBjYXNlIG9uIEFybSkuDQo+Pj4+IEJ1dCB0aGlzIHdpbGwgcmVxdWlyZSB0aGF0IHRo
ZSBleGlzdGluZyBrZXJuZWwgY29uZmlndXJhdGlvbnMgb3V0IHRoZXJlIGhhdmUgdG8gYWRkaXRp
b25hbGx5IGRlZmluZSBDT05GSUdfWEVOX1BDSV9TVFVCIHRvIGdldCB3aGF0IHRoZXkgaGFkIGJl
Zm9yZSB3aXRoIHNpbXBseSBlbmFibGluZyBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5ELiBNeSBw
b2ludCB3YXMgdGhhdCBpdCBpcyBwcm9iYWJseSBkZXNpcmFibGUgbm90IHRvIGJyZWFrDQo+Pj4+
IHRoZSB0aGluZ3Mgd2hpbGUgZG9pbmcgdGhlIHNwbGl0L3JlLXdvcmsuDQo+Pj4gQnkgbGV0dGlu
ZyBYRU5fUENJREVWX0JBQ0tFTkQgc2VsZWN0IFhFTl9QQ0lfU1RVQiB0aGlzIHdvbid0IGhhcHBl
bi4NCj4+IEluZGVlZA0KPj4+PiBJIGFsc28gdGhvdWdodCB0aGF0ICJjb21waWxlIG9ubHkgcGFy
dHMgb2YgaXQgd2hlbiBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EIGlzIG5vdCBzZXQiDQo+Pj4+
IG1heSBoYXZlIG1vcmUgY29kZSBnYXRlZCByYXRoZXIgdGhhbiB3aXRoIGdhdGluZyB1bndhbnRl
ZCBjb2RlIHdpdGggQ09ORklHX1hFTl9QQ0lfU1RVQi4NCj4+Pj4gSSBhbSBub3QgcXVpdGUgc3Vy
ZSBhYm91dCB0aGlzIHRob3VnaC4NCj4+PiBUaGlzIHdvdWxkIGJlIGEgdmVyeSB3ZWlyZCBzZW1h
bnRpY3Mgb2YgWEVOX1BDSV9TVFVCLCBhcyB0aGUgc3R1YiBwYXJ0DQo+Pj4gaXMgbmVlZGVkIG9u
IFg4NiBhbmQgb24gQXJtLg0KPj4+DQo+Pj4gR2F0aW5nIGNvdWxkIGV2ZW4gYmUgZG9uZSB3aXRo
IFN0ZWZhbm8ncyBwYXRjaCBqdXN0IGJ5IHJlcGxhY2luZyBoaXMNCj4+PiAiIXhlbl9wdl9kb21h
aW4oKSIgdGVzdHMgd2l0aCAiIUlTX0VOQUJMRUQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCki
Lg0KPj4gTWFrZXMgc2Vuc2UuDQo+Pg0KPj4gQW5vdGhlciBxdWVzdGlvbiBpZiB3ZSBkbyBub3Qg
d2FudCB0aGUgY29kZSB0byBiZSBjb21waWxlZCBvciBub3QgZXhlY3V0ZWQ/DQo+Pg0KPj4gSWYg
dGhlIGxhdGVyIHRoZW4gd2UgY2FuIGRlZmluZSBzb21ldGhpbmcgbGlrZToNCj4+DQo+PiBib29s
IG5lZWRfcHZfcGFydCh2b2lkKQ0KPj4NCj4+IHsNCj4+DQo+PiAgIMKgwqDCoCByZXR1cm4gSVNf
RU5BQkxFRChDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EKTsNCj4+DQo+PiB9DQo+Pg0KPj4gYW5k
IHRoZW4ganVzdCB1c2UgbmVlZF9wdl9wYXJ0KCkgZm9yIHRoZSBjaGVja3Mgd2hlcmUgaXQgaXMg
bmVlZGVkLg0KPj4NCj4+IFRoaXMgYWxsb3dzIGF2b2lkaW5nIG11bHRpcGxlIGlmZGVmJ3MgdGhy
b3VnaCB0aGUgY29kZQ0KPiBUaGlzIGlzIGV2ZW4gYmV0dGVyLg0KPg0KPiBGb3IgbXkgY2xhcml0
eSwgT2xla3NhbmRyLCBhcmUgeW91IE9LIHdpdGggYWRkaW5nIGEgZmV3IG5lZWRfcHZfcGFydCgp
DQo+IGNoZWNrcyB0aHJvdWdoIHRoZSBjb2RlIGFzIHBhcnQgb2YgdGhpcyBzZXJpZXMgc28gdGhh
dCB0aGUgUFYgUENJDQo+IGJhY2tlbmQgaXMgbm90IGluaXRpYWxpemVkPw0KWWVzDQo+DQo+IEkg
ZG9uJ3QgaGF2ZSBhIGdvb2QgdGVzdCBlbnZpcm9ubWVudCByZWFkeSBmb3IgdGhpcywgc28gSSBj
YW5ub3QgcmVhbGx5DQo+IHZvbHVudGVlciBteXNlbGYuDQo+DQo+IEkgd291bGQgcHJlZmVyIGlm
IHdlIG1hZGUgdGhpcyBjaGFuZ2UgYXMgcGFydCBvZiB0aGlzIHNlcmllcyBzbyB0aGF0IHRoZQ0K
PiBQViBQQ0kgYmFja2VuZCBmZWF0dXJlcyBkb2Vzbid0IGdldCBlbmFibGVkIG9uIEFSTSwgbm90
IGV2ZW4gdGVtcG9yYXJpbHkuDQpPaywgSSB3aWxsIHB1c2ggdjIgdG9kYXkgd2l0aCB0aGUgYWRk
aXRpb25hbCBwYXRjaCBmb3IgUFY=


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:07:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192307.342687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyEK-0006NC-ED; Wed, 22 Sep 2021 09:07:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192307.342687; Wed, 22 Sep 2021 09:07:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyEK-0006N5-B1; Wed, 22 Sep 2021 09:07:52 +0000
Received: by outflank-mailman (input) for mailman id 192307;
 Wed, 22 Sep 2021 09:07:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mSyEJ-0006My-Aa
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:07:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSyEI-0003Fe-5J; Wed, 22 Sep 2021 09:07:50 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSyEH-0003ki-Fk; Wed, 22 Sep 2021 09:07:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=u/l6QLgW1GiuVPFhTbV1kN5L6Uvm5lCyXLPzB1Uke/Y=; b=6EWLhpsKjKGP09dXuwsmL2TZG8
	zbobampEv8l9WdFb6n9Ef3xlXALJIbjxxLOmUriaaQsP4LrXbpQsrWXNxK6qib41RPiLZhjx4IkM5
	UUe7IplnMsnJOG2yrTHSm0KbnMzWJIEI1Ag+HkVMeTqrDwujwARilVxXfDLPkUYIk8qw=;
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Juergen Gross <jgross@suse.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
Message-ID: <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
Date: Wed, 22 Sep 2021 14:07:44 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210922082123.54374-6-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 22/09/2021 13:21, Roger Pau Monne wrote:
> Failure to map the shared ring and thus establish a xenstore
> connection with a domain shouldn't prevent the "@introduceDomain"
> watch from firing, likewise with "@releaseDomain".
> 
> In order to handle such events properly xenstored should keep track of
> the domains even if the shared communication ring cannot be mapped.
> This was partially the case due to the restore mode, which needs to
> handle domains that have been destroyed between the save and restore
> period. This patch extends on the previous limited support of
> temporary adding a domain without a valid interface ring, and modifies
> check_domains to keep domains without an interface on the list.
> 
> Handling domains without a valid shared ring is required in order to
> support domain without a grant table, since the lack of grant table
> will prevent the mapping of the xenstore ring grant reference.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> oxenstored will need a similar treatment once grant mapping is used
> there. For the time being it still works correctly because it uses
> foreign memory to map the shared ring, and that will work in the
> absence of grant tables on the domain.
> ---
> Changes since v1:
>   - New in this version.
> ---
>   tools/xenstore/xenstored_domain.c | 30 ++++++++++++++++++------------
>   1 file changed, 18 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
> index 9fb78d5772..150c6f082e 100644
> --- a/tools/xenstore/xenstored_domain.c
> +++ b/tools/xenstore/xenstored_domain.c
> @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
>   	struct xenstore_domain_interface *intf = conn->domain->interface;
>   	XENSTORE_RING_IDX cons, prod;
>   
> +	if (!intf) {
> +		errno = ENODEV;
> +		return -1;
> +	}
> +
>   	/* Must read indexes once, and before anything else, and verified. */
>   	cons = intf->rsp_cons;
>   	prod = intf->rsp_prod;
> @@ -149,6 +154,11 @@ static int readchn(struct connection *conn, void *data, unsigned int len)
>   	struct xenstore_domain_interface *intf = conn->domain->interface;
>   	XENSTORE_RING_IDX cons, prod;
>   
> +	if (!intf) {
> +		errno = ENODEV;
> +		return -1;
> +	}
> +
>   	/* Must read indexes once, and before anything else, and verified. */
>   	cons = intf->req_cons;
>   	prod = intf->req_prod;
> @@ -176,6 +186,9 @@ static bool domain_can_write(struct connection *conn)
>   {
>   	struct xenstore_domain_interface *intf = conn->domain->interface;
>   
> +	if (!intf)
> +		return false;
> +

Rather than adding an extra check, how about taking advantage of is_ignore?

>   	return ((intf->rsp_prod - intf->rsp_cons) != XENSTORE_RING_SIZE);
>   }
>   
> @@ -183,7 +196,8 @@ static bool domain_can_read(struct connection *conn)
>   {
>   	struct xenstore_domain_interface *intf = conn->domain->interface;
>   
> -	if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
> +	if ((domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0) ||
> +	    !intf)
>   		return false;
>   
>   	return (intf->req_cons != intf->req_prod);
> @@ -268,14 +282,7 @@ void check_domains(void)
>   				domain->shutdown = true;
>   				notify = 1;
>   			}
> -			/*
> -			 * On Restore, we may have been unable to remap the
> -			 * interface and the port. As we don't know whether
> -			 * this was because of a dying domain, we need to
> -			 * check if the interface and port are still valid.
> -			 */
> -			if (!dominfo.dying && domain->port &&
> -			    domain->interface)
> +			if (!dominfo.dying)
>   				continue;

This is mostly a revert on "tools/xenstore: handle dying domains in live 
update". However, IIRC, this check was necessary to release the 
connection if the domain has died in the middle of Live-Update.

So I think this check should stick here. Instead, I think, we should 
mark the "fake domain" so we can ignore them here.

>   		}
>   		if (domain->conn) {
> @@ -450,8 +457,6 @@ static struct domain *introduce_domain(const void *ctx,
>   	if (!domain->introduced) {
>   		interface = is_master_domain ? xenbus_map()
>   					     : map_interface(domid);
> -		if (!interface && !restore)
> -			return NULL;
>   		if (new_domain(domain, port, restore)) {
>   			rc = errno;
>   			if (interface) {
> @@ -505,7 +510,8 @@ int do_introduce(struct connection *conn, struct buffered_data *in)
>   	if (!domain)
>   		return errno;
>   
> -	domain_conn_reset(domain);
> +	if (domain->interface)
> +		domain_conn_reset(domain);
>   
>   	send_ack(conn, XS_INTRODUCE);
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:19:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:19:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192315.342698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyPK-00084r-J3; Wed, 22 Sep 2021 09:19:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192315.342698; Wed, 22 Sep 2021 09:19:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyPK-00084k-G8; Wed, 22 Sep 2021 09:19:14 +0000
Received: by outflank-mailman (input) for mailman id 192315;
 Wed, 22 Sep 2021 09:19:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mSyPJ-00084c-3F
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:19:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSyPH-0003S4-DO; Wed, 22 Sep 2021 09:19:11 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSyPH-0004Qc-2k; Wed, 22 Sep 2021 09:19:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=QE4ZkIUNgTO4teOroxrF/cuxphhYasHb+3psMDvHkNY=; b=lnpFdqTarUa7yFQG6MR6mfh5la
	+YnqrBqlR14IPlSp1r9xAnFdKJI91WTpek1zvWJbshAcqU7CHNKtNPi7VZzrnrRY+X0L288xW+VAq
	gD/xftpbRAP2Y8xL0G4ncjf9fUYuN5Avbo+tKvcl7Z88O3NjkeJAeozdpDbc/wyPu7LA=;
Subject: Re: [PATCH v2 6/6] gnttab: allow disabling grant table per-domain
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-7-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
Message-ID: <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>
Date: Wed, 22 Sep 2021 14:19:04 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210922082123.54374-7-roger.pau@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 22/09/2021 13:21, Roger Pau Monne wrote:
> Allow setting max_grant_version to 0 in order to disable grant table
> usage by a domain. This prevents allocating the grant-table structure
> inside of Xen and requires guards to be added in several functions in
> order to prevent dereferencing the structure.
> 
> Note that a domain without a grant table could still use some of the
> grant related hypercalls, it could for example issue a GNTTABOP_copy
> of a grant reference from a remote domain into a local frame.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>   docs/man/xl.cfg.5.pod.in     |   4 +-
>   tools/libs/light/libxl_dom.c |   2 +-
>   xen/common/grant_table.c     | 100 +++++++++++++++++++++++++++++++++--
>   3 files changed, 98 insertions(+), 8 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index c5a447dfcd..d507540c2c 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -583,8 +583,8 @@ L<xl.conf(5)>.
>   =item B<max_grant_version=NUMBER>
>   
>   Specify the maximum grant table version the domain is allowed to use. Current
> -supported versions are 1 and 2. The default value is settable via
> -L<xl.conf(5)>.
> +supported versions are 1 and 2. Setting to 0 disables the grant table for the
> +domain. The default value is settable via L<xl.conf(5)>.

Technically, the version only applies to format of the table for 
granting page. The mapping itself is version agnostic. So this feels a 
bit wrong to use max_grant_version to not allocate d->grant_table.

I also can see use-cases where we may want to allow a domain to grant 
page but not map grant (for instance, a further hardening of XSA-380). 
Therefore, I think we want to keep max_grant_version for the table 
itself and manage the mappings separately (possibly by letting the admin 
to select the max number of entries in the maptrack).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:26:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:26:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192321.342710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyWf-00013d-BY; Wed, 22 Sep 2021 09:26:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192321.342710; Wed, 22 Sep 2021 09:26:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyWf-00013W-8Z; Wed, 22 Sep 2021 09:26:49 +0000
Received: by outflank-mailman (input) for mailman id 192321;
 Wed, 22 Sep 2021 09:26:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSyWd-00013O-IB
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:26:47 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7a4066e4-9bc3-489c-8fda-8ba9de13a1e0;
 Wed, 22 Sep 2021 09:26:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a4066e4-9bc3-489c-8fda-8ba9de13a1e0
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632302805;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=Bv5lzHPLRLFDVicasBh8gMjl+/oe3pl7ljKyz7qv7QA=;
  b=JHL+j4q1daN+h/FzN+8FRbaXpvPuOKoK9U3SLOnWcIdaFO474Yyd2Cu3
   8QbPjptBviX4p6XeJoNNDPt8V57EUmDFMBJ2GGCngxGg/6AwARg4sv+dx
   CZQNhb6TPYfaKBmNE+dMjKx2Rld5ohqJM4cn5cGtuxKXDxJTIT39WGORO
   k=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: oogck7G+RwvXlYQTkvzOhp1dp/dr9RLR5nLwk+UPdHu/aslLE3wCSV0Zv+NTlH+hun38NxLJPd
 LRRv3lHPcMxWS0GnEisTu6iqpJllawky+dgLIURD7K+7yQLN9PXXxpgjCNIecI0Uz/Jy6m9Dkc
 xr4yqDo/1Y+TTvCbmoLuPvlAlZREyYR+0Shc4vBZZDXtI4nTZL6xK7T+mlq3MUYOE5gNLbApJn
 minvaPh2GNmHI1m8TL9HBfa/vTUCpW+KlBKHvgtRFqg+wsnYI9sTrloe5Dxe0essszZIJSuqbp
 +bAOGYUxq92CN0pU82trFyR3
X-SBRS: 5.1
X-MesageID: 55105045
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:tZtqPqjOr1eulYqM9kpV19+UX161shcKZh0ujC45NGQN5FlHY01je
 htvXGCFPaveNDD2ctp1PI22oU9TuJOBnIVgSgNp+ykyQSMb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y24Dja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1W5LK8Ui07ZJTUldgMTR1lK2JDJqRZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHxO4wSoDd4xCzxBvc6W5HTBa7N4Le02R9t25kWQK6CO
 qL1bxJhZh3xXQcMPms2K5AUzNuqpkjeKQdx/Qf9Sa0fvDGIkV0ZPKLWGNjIft2HQ+1Fk0Deo
 XjJl0ziGQ0TPtGbzTuD81qvi/XJkCe9X5gdfJWn8tZ6jVvVwXYcYDUUX1ampfiyimalRslSb
 UcT/0IGvaU0sUCmUNT5dxm5u2Kf+A4RXcJKFO834x3LzbDbiy6YGWosXjNHcMYhtsI9WXotz
 FDhoj/yLWUx6vvPEyvbr+rK62PpUcQIEYMcTTRVYwceycu5mZk+sgPRSutbU5Ouh8KgTFkc3
 Au2QDgCa6Q71JBQjfzrowyf2lpAtbCSEVVkvVy/snaNq1ojPd/7PdTABU3zsK4YRLt1WGVtq
 5TtdyK21+kIEZjFvyiEWuxl8FqBtqvdbWG0bbKCGfAcG9WRF5yLJts4DNJWfh4B3iM4ldjBO
 hS7hO+pzMUPVEZGlIcuC25LNyjP8ZUM6Py/DqyEBjaxXnSBXFDep3w/DaJh90vsjFItgckCB
 HtvSu71VSxyIf0+lFKeHr5BuZd2lnFW7T6CHvjTkkX4uYdykVbIEN/pxnPVNbtnhE5FyS2Im
 +ti2zyikUQHDLKmPXmJrub+7zkidBAGOHw/kOQOHsarKQt6AmAxTfjXxLIqYYt+mKpJ0OzP+
 xmAtoVwljITXFXLdleHbG5NcrTqUcotpH43J3V0b12px2IiccCk66JGL8k7erwu9epCy/9oT
 qZaJ5XcU6oXEjmXqS4AaZTdrZB5cEj5jwy5ICf4MiM0eIRtRlKV94a8LBfv7iQHEgG+qdA6/
 ++7zgreTJdaH1ZiAc/aZeiB1VS0uXRByut+U1GReotYeVn28ZgsICv016dlL8YJIBTF5z2by
 wfJXktI+biT+9c4qYCbi7qFooGlF/pFMnBbR2SLv6yrMST6/3a4xdMSWui/Yj2ABnj//7+vZ
 LsJwqikYuEHhltDr6F1D61vkfAl/9LqqrJXklZkEXHMYwj5A79sOCDbj8xGt6kLzb5FowqmH
 EmI/4ACa7mOPcrkFn8XJRYkMbvfha1FxGGK4KRnOlj+6Q924KGDABdbMBS7gSBAKKd4bdE+y
 uA7tc9KswGyh3LG6DpdYvy4I4hUEkE9bg==
IronPort-HdrOrdr: A9a23:P1MK6qmGaUI/v282B9tPH6jO14bpDfO/imdD5ihNYBxZY6Wkfp
 +V8sjzhCWatN9OYh0dcLC7WJVpQRvnhPhICK0qTMqftW7dyReVxeBZnPHfKljbehEWmdQtsJ
 uIH5IObOEYSGIK8voSgzPIY+rIouP3iJxA7N22pxwGIHAIGsMQnDuRSDzraXGeLDM2dKbRf6
 Dsn/avyQDQHkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="55105045"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZnUKcw5p9CezGq4cTIM2QhRR3dkuXDx2H6nZEd2jQkQG1rQUZA3ACOHI+AuCuavnsnZ0lxiTVMUGfafqXpE/DNCNFl9dDVFk0+u1QWwBstW/jaYMMygES+AvsuLHhwB8sVxi53xQCVpmzGv7ZmkeYfXIbgUNNmOSDJ4YvOcq8FPT3Op56OWtFU0W8DMiNnLrZpeafIlGq7a7G4jo4hBkh48GJ0FmYYsVBjEf9QMpwN4XCIpAzJeWs1z7HnJbpxqm/EyTrHQLOMwWuBwBdK7N2B0Ha2WUrDHUpnwBnNyDPWzGecO9okTZImpzz5//wQLzpBvcPDmJqMQWXVTNz/7e0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9x5Q87cIOuFleBoYcvIT6cs7a8IkDoF0kihHsHDYnhA=;
 b=oVT+kVI63H8cAQR4fJNchh75CmCO8wx886JqguVj6EClTg1zzbEsqK9EoGmcmYcwLl0kuqo8bbogUAS8/l7LOoHi3PKWlC36mViKSgRDcSBGyVA58uNrEP831TBYiI7Nowoq0eMOXukbv3HGn/OtV6lyJNeHjA7ZOcV3FHep1iCgB2+cERYwRyq2fmMReIpmToSJ7ZvvfGBMMvkyBejZ0XNMSI43aIfYvTIA1DFg6dP0/3TfAppCLWcWwwkop6GktNNr4HfsTdfEiW1G0Capx70DpUtdkLeeRqY4jFQlv4diAnnM2xoilv/XDva+t1pEaMB+BAs/4300T/3AYOwZNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9x5Q87cIOuFleBoYcvIT6cs7a8IkDoF0kihHsHDYnhA=;
 b=V2HG84YDLCZsus+9aDV1Wd225dQJS/mShHhEwkM1xpFuOk2IxtYwqDUBNhgNNNu+K+9qDPYVmL0gBtKhl4ErayCNJjBeFMOr5MWeFuVPERPhiXhlN7pxAgPlli3ysSApdxwRypf4AsV5cH8POzE+vh4Kv9Gg9/+jNELGOTHyvuA=
Date: Wed, 22 Sep 2021 11:26:37 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, "Stefano Stabellini" <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
Message-ID: <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
X-ClientProxiedBy: LO2P265CA0170.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 870d20ca-7c81-4871-15f8-08d97dab167a
X-MS-TrafficTypeDiagnostic: DM6PR03MB4140:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB414084444B2294BBD245BBB18FA29@DM6PR03MB4140.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: kUzZHoLGVvHs6Xv9EX2xbLhtJQKdJ314ZYcnwk16D1pLepbQDBW6+Lpy4tXjNB/6vQP5axjJAC4j9jWmFY/iHwan/bsrG3uhBQcBCGUMD0dBegKXyflZECDhfJtnW0qmzVOfoUgZwsqGG+ZEpXkBq6Tsd1yQ6UTgrX2W+l4gvLTUUxIt/7N5xhZqikDXVy1iU7bifaqS8USCLp036ukDeiSyPiDwGdba/woAmUR8UITOVd4VRqqoyuYECO4uvQx4XZOM62m22xdh838x3bBiFKuj0cA8c2I8H1z5VI5ksHTfZyGHKpNe27NR5tKBvWSvvq6uHbGC1W63DA3UPjv7ae494FA4tTIPjnSkM0GKAVvJAh0zXnUjOQordWhToHhGYOMxlqGi+HYCOCTr0tqLxCAAYb5sbT4Xmap4Mm49zUtOlHLtx5sZrPK4RvXFyGsTN3HJt55Rm1lvv8x/L8L3f4kPXVGthwMKTdHQmqfZ8AsWtc5MpngOv3oqeMOpPCj1EYm2tuSotNX58NSB+eIxOIdTRMxWDJulQmN+t2tDJowx/mNNzLrD5crUpEGv5HEtTnX/cC3NtwZhC2V+5rp5G7J60YEftLL+ssHbcGJLf2K9PiGoA0BD0bMkLzLASU15mZD/3KoGKLSvscRFcSwGziPyYqr4m3u7E5MECD7P5Bl5Ge2lP0TQAdC9ewkVw9Ae4gFqd6w5ZMRCtxwfmb040w==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(508600001)(38100700002)(6916009)(6666004)(9686003)(53546011)(66946007)(2906002)(316002)(86362001)(186003)(956004)(66556008)(8676002)(66476007)(6486002)(85182001)(8936002)(26005)(4326008)(6496006)(5660300002)(83380400001)(76704002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dm9uMGo0NWE3ck14V0xVRzc0OEl4QlFXeU11TjRNdmgvc0kvVmVrM3ZTYnNz?=
 =?utf-8?B?Ynd2Q2tHYVVPaWpCKzhPb2NnN2RKMFpXeGYxbGM2UHpHQmVFOG9hanFYU1JJ?=
 =?utf-8?B?Zy9xWWFVcU5uUExYY2RDdlNmejZLYzdxVG1VMHZvNmttc0NlT3VZZk9mcno3?=
 =?utf-8?B?YzliUFFFdU9sTTBlSmNDK29BNkNJR3ZpZUsrQjF2MjRMWDNHb2J6WC9oRzRu?=
 =?utf-8?B?dngvNnR2OWFwSG1jOW5LN0VHSGdvdW50eE1GMnJLR0xRRmRXSkdUUlVrTmxj?=
 =?utf-8?B?aDBKQXg1bzVRa3lYMk5xNkdBVGJYdGpYdjRnZUp3WUFkSjZKL05tWm1tVXFD?=
 =?utf-8?B?aHJSL2xpTlhTVmpFYlpsM3Fic1Y5NUEyU0NndURCODhLUDJTZUR4ajZXM2l1?=
 =?utf-8?B?aHlPdGl2YXhlVWpNUnRIY1p0dGlCNG4wR1hMczE0Sys4SklkVGJ2L2ZHUHNW?=
 =?utf-8?B?UnhENS8vakpQcisyQ3VQTUU2R1JaNUVVWllWOXB1WDRmZWJvb0gwcUs2S2t5?=
 =?utf-8?B?ZDR5OGVqc2N0QkpIczUvVU5DUG14UUJLUjdRUjgzbkp2NXVuT2g2QlN6WFAr?=
 =?utf-8?B?WjY4MjNvanJ5UVRGR3ovNndjS2Y2cDNSNm0rRG5QTktpZVgxaW5CUlhuYk5L?=
 =?utf-8?B?RTM2ZHpkMGVDVTdTbngrU2xEdFpPblJDbHdycEVFR0hGc05UNE1zNFpvTXlv?=
 =?utf-8?B?ZFpNYXBqdDhQdlVWcTFsZ3ZReE8yWHhScGNsaWpYVm96NG9ZR2psanJVRmdj?=
 =?utf-8?B?UU5FY2pTcTU0MU1PWVBmaUJxeVRrNjlUUVZWeHQxV1M1Q3dKd0NWVXcxRnF2?=
 =?utf-8?B?MDFkaVNhd3pPejRHc0ladFNVaGFrc3ZJc3Y4Y3hJZ3l3bGNmaFYvdjNyNXYw?=
 =?utf-8?B?K0d1azRQL2tWY3dYQmd0cS9ySmU3WldOTUVQVzhvbnFuNWRrb1BiSHNpczFM?=
 =?utf-8?B?L1ZoeW0vT21XZkY1NW1qUlBzWmxoZFl3ZFlQSkpnVXQ4eWVQRm4zbjh5N05s?=
 =?utf-8?B?UUdBTXJlTjFLcTlQWlhTZjd5WW5XZXJUckhXWVFhU1BTZnRXejQwTzFzMGdv?=
 =?utf-8?B?aVZuaWZXYlMrUERyek9FVUlMLzBYek1lUzNoOEhpNW1KVUR4Q1I4QzZoWldG?=
 =?utf-8?B?Rjh6MWdlVjlYenkvUVYzaXM3N1BMMHIwZDhHUzdwdzJjTUVSMmpSWkQxSVBL?=
 =?utf-8?B?eTFmYmYwUHN3cnowOVM4SmNJVGlWOVRjQ1o1RTZNK2VNWURYSFc0eUVDVk1a?=
 =?utf-8?B?STJLeFF0OWVwZ0RoVkUzZUpEUnRWQWdxMWR0ZEFjTnUxdVBmYVhEWVBINUpn?=
 =?utf-8?B?eWVjTkkwNGl4Y3hUOGZUVkVLL2kxRGhyNjkyMHVCaTdBMlN5NWRTMlRtL3pS?=
 =?utf-8?B?NXg5SXpxZVN0VGxWR0hOVWhseVdSTnFvYldnWi9UeENWWk1VaXFmbzJnUVp0?=
 =?utf-8?B?bERPQkFVblRyZk5PNEhUU0UrWnVrN1hrdWd3bVBnbFNxUFF4QXhKS2l3UHIz?=
 =?utf-8?B?MkdxcEhYVUN3Q3BQcVJoYmovZjdGQ29LMkpPRDZWaFZCZEZHUUsrZmZWbFU2?=
 =?utf-8?B?QS9QYitBc0F0dU9MTkN2U21uMUxVdlZMTnQxK3gvalpaV3c0VUpGTFRrSUgz?=
 =?utf-8?B?WGNRdlVNUXFwekpCcUVXeW01YWp4Q2tZWEQyak56Q1FPUkZQNlBleVBsYnBB?=
 =?utf-8?B?Z3RSQVVXOUNINHBoWitLWUFzK2pPb25aLzl6dmlwZmYzS2tTT2dxQTBEbnF2?=
 =?utf-8?Q?HYULlLBz9W39ibACeMeNJFEVM0ulfNaFJk/ZCG3?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 870d20ca-7c81-4871-15f8-08d97dab167a
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:26:42.1486
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: atCjGKJwrW0Op6ofq/ekROZe7aWVpnElZ7m8vJ4yWvUzVeCe96BVrVUh7Bpv9T4P3UyBSKduEtjH29GwNGYvKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4140
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
> On 21.09.2021 10:32, Roger Pau Monné wrote:
> > On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
> >> On 20.09.2021 12:20, Roger Pau Monné wrote:
> >>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
> >>>> --- a/xen/include/asm-arm/grant_table.h
> >>>> +++ b/xen/include/asm-arm/grant_table.h
> >>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
> >>>
> >>> I'm slightly confused by this checks, don't you need to check for
> >>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
> >>> guest_physmap_remove_page?
> >>
> >> Why? It's ogfn which gets passed to the function. And it indeed is the
> >> prior GFN's mapping that we want to remove here.
> >>
> >>> Or assuming that ogfn is not invalid can be used to imply a removal?
> >>
> >> That implication can be (and on x86 is) used for the incoming argument,
> >> i.e. "gfn". I don't think "ogfn" can serve this purpose.
> > 
> > I guess I'm confused due to the ogfn checks done on the Arm side that
> > are not performed on x86. So on Arm you always need to explicitly
> > unhook the previous GFN before attempting to setup a new mapping,
> > while on x86 you only need to do this when it's a removal in order to
> > clear the entry?
> 
> The difference isn't with guest_physmap_add_entry() (both x86 and
> Arm only insert a new mapping there), but with
> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
> mappings. And gnttab_map_frame() gets called only from there. This
> is effectively what the first paragraph of the description is about.

OK, sorry, it wasn't clear to me from the description. Could you
explicitly mention in the description that the removal is moved into
gnttab_set_frame_gfn on Arm in order to cope with the fact that
xenmem_add_to_physmap_one doesn't perform it.

TBH I think it would be in our best interest to try to make
xenmem_add_to_physmap_one behave as close as possible between arches.
This discrepancy between x86 and Arm regarding page removal is just
going to bring more trouble in the long term, and hiding the
differences inside gnttab_set_frame_gfn just makes it even more
obscure.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:28:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:28:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192327.342721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyYC-0001nX-Ob; Wed, 22 Sep 2021 09:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192327.342721; Wed, 22 Sep 2021 09:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyYC-0001nQ-KU; Wed, 22 Sep 2021 09:28:24 +0000
Received: by outflank-mailman (input) for mailman id 192327;
 Wed, 22 Sep 2021 09:28:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=q8VE=OM=citrix.com=christian.lindig@srs-us1.protection.inumbo.net>)
 id 1mSyYA-0001nG-Ji
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:28:22 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8b398b2d-c3a3-4fd4-92aa-ecf59c919d20;
 Wed, 22 Sep 2021 09:28:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b398b2d-c3a3-4fd4-92aa-ecf59c919d20
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632302901;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-id:content-transfer-encoding:
   mime-version;
  bh=QDQ7sUKDQ1O9y8Um8Up74jv6SVkGfrVFkpCyF8Li5HE=;
  b=UYeElKFYJIiAs89EcIkYcaLAgEgwTD7h6s5Jmnso/uSbhJLjOzjofBeq
   EwbJA5/aJBtbs3MEPCNTjDozlp6QJakDfIpZ3cn6hTPBwrbEj6MESulIC
   pwMohDhkzxfzKwX1xOM5JE7aaFG+b+nrXHRsGQl/UJfYMkoTphnGUmjrQ
   M=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: rCy14/w4YdlvEXI3jRSK6QBwboxKXEGxdGQ6QRU/0TX7H0Vdbnu73tqa6kyFX/8n0l+//yj5Xf
 ExHIM2MuadvWs+ht/7RuFYN5Xr+ZW2mOPswthVTHj5i4vuRzWLeqCaiL5+XvGd/6o/gDyGJ9lY
 WQ51nLbciAmnjVN/JlHXJYbSW38p9rgbB75brexi4Xcn02uWFuPUX+7F+Wi4xJboqQhSOKULol
 3L3SYs8c8ReRDUMm9Myt+jY1K0JfDAfDK66t2uoANEiKkOLhlI4wM7bir3jfVEt60qMW0/cOVK
 FcyDuhC0Q1NQpQYwVpkjliEk
X-SBRS: 5.1
X-MesageID: 52898958
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ysZcMqtk/wy4J1Pg1OMT5istEefnVM9ZMUV32f8akzHdYApBsoF/q
 tZmKW/XPP6KYWGjKdh/aYu+pENXupTXydNhGlc9ryAxESoX+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524LhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpljIGzaC4gE7P2pMsHCTJCGAZjM/1U5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DN4MFu2p8zDjfS/InR5zCTI3B5MNC3Sd2jcdLdRrbT
 5dDOWs/PUmQC/FJEgsLN79ggL/1vXS8bzJmuALKn45wu0GGmWSd15CyaYGIK7RmX/59nEmCo
 Xnd13/kGRxcP9uaoRKe6W6ljOLLmSL9WaoRGae++/osh0ecrkQtDxkRWUq+sOOOoEe0UNJCK
 GQZ4iMr66M18SSDQtDjUjWirXWDvxpaXMBfe8U49QWMx6z88wufQG8eQVZpacMknN87QyQw0
 V2ElM+vAiZg2JWFRHTY+rqKoDeaPSkOMXREdSICVREC4dTovMc0lB2nZtRpHbOxj9b1MSrt2
 D3Mpy87750MieYb2qP9+krI6w9AvbCQEFRzvF+OGDv4sEUpP+ZJerBE93Di8PVhBY+0UGO9k
 2JesNKVy/0AXYOSwXnlrPo2IF252xqUGGSC2gQ2T8d9r23FF22LJt8LsWonTKt9GoNdI2a4P
 hWL0e9EzMILZBOXgblLj5Vd4ijA5ZPpE8jsHtvQZ8BHCnSaXF7apHwyDaJ8MmaEraTNrU3dE
 czAGSpPJSxDYUiC8NZRb71GuYLHPghkmQvuqWnTlnxLK4ZygUKopUotagPSPojVE59oUC2Kq
 o0CZqNmOj10UfHkYzm/zLP/2WsidCBhbbiv8pQ/XrfafmJORTFwY9eMkOhJU9E0wMxoehLgo
 yjVtrlwkwGk2xUq6GyiNxheVV8Ydcwu9S5kZXN8YwnANrpKSd/H0ZrzvqAfJNEP3Odi0eR1X
 78CfcCBCe5IUTPJ53IWapyVkWCoXE7DadumM3X3bT4hUYRnQgCVqNbochG2rHsFDzattNt4q
 Lqlj1uJTZ0GTgVkLcDXdPPwkA/h4SlDwLp/DxnSP91eWETw64w2eSb/ueA6fpMXIhLZyzrEi
 wvPWUUEpfPAqpMe+cXSgfzWtJ+gFuZzRxIIH2TS4busGzPd+26vnd1JXOqSJGiPX2Lo4qSyI
 +5SyqikYvEAmV9Ltat6Eqpqkv1it4e++ecCw109TnvRblmtBrdxGVW83JFC5v9X27tUmQqqQ
 UbTqNNUDqqEZZH+G1kLKQt7MunajaMImiPf5OgeKVnh4HMl56KOVEhfMkXeiCFZK7cpYoopz
 f145ZwT4g27zBErLsyHnmZf8GHVdi4MVKAut5c7Bo73i1V0lgEeMMKEUiKmsouSb9hsM1UxJ
 m7GjaXPsL1Q207efidhDnPKx+dc2cwDtR0iIIXu/LhVdg4pXsMK4SA=
IronPort-HdrOrdr: A9a23:eefIS6qfOHYbBTJ5tXUS55MaV5oveYIsimQD101hICG9Ffbo8P
 xG/c5rsSMc7Qx7ZJhOo7y90cW7Lk80lqQU3WByB9mftWDd0QPDQb2KhrGC/xTQXwH46+5Bxe
 NBXsFFebjN5IFB/KXHCd+DYrQd/OU=
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="52898958"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eeJrL2JG9EXFnfpPo1erIlUrU2hsk3HaSEepI05hY80DVQSKdKVkJd4kBBpESWo+YONgYD+kcZ30lL+2SupsbQ9Z3xqqOeCfjAm2BlYv1wXDqV99FVIVXKwxbweKNOOWs1DMbqS1JOUqiBqrHYe2D7gwaqGTnALdU7FUYPVOuQQhaL1t9YOWF6im/1AwdsRxkDtGkYC+xcbuIGTJ3qneFbKz7Bagx8j9ZKEuekTwilvdCGgKtFeWw8kJdVo3c+LzsXxo/mlskqx7EXQe0yBt0ks2yJSXZFEk1HE/XLSqWXrFh0BXxfFNK5nXEGLIhBJOY99vjPmg67UJj5CP7ffjFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QDQ7sUKDQ1O9y8Um8Up74jv6SVkGfrVFkpCyF8Li5HE=;
 b=ngHn+zpBTOii9WnRnyclKE4LszCUtgvo8Ob1aA3d3mHq8yfz+ICTrj/IPwZ9WPjVmyFcl2gpvIj+wGnfNskzQFKaPYyDHiJF1e/6+KXWEpuxMlQ1xtJQmU9ND8epeulb9/AGfucaq82/mIFtJfwcb5Nkn9IS1uUKLwmMA69hKQh4WrEu9At/d1VegbdzWnlBJtvtigEqSUYpeSgRdosrTQ9DMVBPUFj9nyHse/SIQuOXja0tgek5qCZ4RGdWLSrVGkqGDjHEX7CCQl+XfgkOKLgQXEn80wekMmGIyjDGvcBIfF2cR44Jt23BY2q90NuTlu4O8k0bb9kDBdM5Vz5QJw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QDQ7sUKDQ1O9y8Um8Up74jv6SVkGfrVFkpCyF8Li5HE=;
 b=iuKTRqxDA7g7B3xExrzaKaCNZt7eaZw0CB7Thggp9JM6l9OdOVXbAvCCRZyLeY5h2Ieyx8ljVhtAL8LFi6O/+9q9Kkdq8B5CEBXK0wU1P5m4bF23/venWQESuxWseGrmLMcKM/+w8Jf22Xac1UdAa5emG1V1+f8AJVa3rI0P/nM=
From: Christian Lindig <christian.lindig@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Ian
 Jackson" <iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<Andrew.Cooper3@citrix.com>, George Dunlap <George.Dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony Perard
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, "Edwin
 Torok" <edvin.torok@citrix.com>
Subject: Re: [PATCH v2 3/6] gnttab: allow per-domain control over transitive
 grants
Thread-Topic: [PATCH v2 3/6] gnttab: allow per-domain control over transitive
 grants
Thread-Index: AQHXr4rqo8uhO7XRjkO2Lx1exL1r46uvyUQA
Date: Wed, 22 Sep 2021 09:28:18 +0000
Message-ID: <D28B0008-6DE4-4A5B-B4F1-78F902362635@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-4-roger.pau@citrix.com>
In-Reply-To: <20210922082123.54374-4-roger.pau@citrix.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3608.120.23.2.4)
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 59bbbc3f-51b4-4a94-74e0-08d97dab5034
x-ms-traffictypediagnostic: CO1PR03MB5891:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <CO1PR03MB5891692C4E89CAC810B8302FF6A29@CO1PR03MB5891.namprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: ZsCmmDV92l/tEE81q6olyPVsGUXGyKsTet7RzCOGbmwfE4lPR0TCqilCmrnPzuPl8V0HOS6b4J5GKpKHaG4x+clMjSskV9RrepNDcRIplSJ7Gp3k4eKqRVUnUOF0CHXB59DaIAdO9EwgFDYw0nCiCzbhvcyMQWzODjJg21Fv1bb4lSWTuaDN8lhvWG3AZ4VU8JYfr7kHuhgfQgDng0S0H3mEiVkIlYy4jEkI3wX6Hf1LbtfZMrnLY5LsdVWQoPF2iFNCHgFp9iNuEP5C2/RaMMFN6v+Czyj+WwZKZa0f2SJLgkIrZMthB7+6s8U0yYjmwq5fVVQMwYGU00GPSaNP4THWV7WcUS4/AJoulx3xi7brk4AN/1c9DsNp6OOEEO8bFlgvoX9b2wpX4PWlSEznGk4c47GZm4OXndsSqORn0RS5MtI7UlbPCDJdLA3/SAHBjKhRrLOmmtXyaEXBc9vbPgIa2wbKcIr+hCY57j4IntEhvzNMesfE3hwosLlbbZgtUzqENIKfWaGouGzwXjeGDTxZ3f6dqlAdfzehyLjxwsRarOGsT38U53oPRf+R7Hots2BLwWA6SSbNsT7nPa8vvi4LIKZLH1rjG/M/yqkGvnsdIyW0WIJUqVqcI2/+GEOSDhSHXn0GAAGNiGBBW1TwDdE+BQfD/eb4tJY6F/+doaIO5Iz/hdA9PIRR49zWyz6eAVN191IPC8XTaw6CmGBCnjSbo5Gslzx3Kw7TU0ATYMY=
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6380.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(8936002)(122000001)(44832011)(38070700005)(71200400001)(66556008)(76116006)(4326008)(91956017)(53546011)(6636002)(6506007)(6862004)(186003)(37006003)(2906002)(6486002)(54906003)(8676002)(107886003)(55236004)(83380400001)(316002)(38100700002)(86362001)(2616005)(26005)(4744005)(508600001)(33656002)(36756003)(6512007)(66476007)(66946007)(66446008)(64756008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?dlBQZENtVGZEZldFbXlod0xZTzdNdjRwRUg3OUtOamI1QWJ0cExWQUpmT0dD?=
 =?utf-8?B?SDhOMUZMSTZuaFJOUUdGeVQ0Yk0xTW5MTEt3YjFTQXpNRk1CeXB5N0d6N1My?=
 =?utf-8?B?engvaFRza0FFTHJ6SXZsSkllbEdSakZ3UkovVGdRdDZkNDNqVG5iaG10QXBo?=
 =?utf-8?B?UHFjakgzYk52N1VmN2FqWVorS2d0d0dqY1FMTGRRclhZQ3gzVG0yN1N6N0cv?=
 =?utf-8?B?aitGU2VEczZHbEp5czZRSHlsZUtMK3BtYlNqWkV4TUZIZ1p0V0o5MHhmb3pK?=
 =?utf-8?B?bGpiYnNycGlLRDJWRW5vOURqMUZBNXpENnRyU0FlT3JENTdCc0pSVU1heDVY?=
 =?utf-8?B?TlBvdUVKSzJlNXZlOFQwVXcwb2F5TTIrUnBXMzl4ODRzNU5Zd3Ntb3pMWEJn?=
 =?utf-8?B?UmVkb3JYMUVZZE82RHpjMW5BWHNkb0kvSmtva0lqYi9BWW1hTzVWRkVxa2xH?=
 =?utf-8?B?QlFPQUNNZmNwdGFMVHhRQ3Nqc0hhRHllaXJyVmRBVW1mMVBya1g5L25Od2x0?=
 =?utf-8?B?a215OC94bmhqclRDTzYrN1h5TmlSUVdYQVIwRjVlR0VPUmxlNUpFbm4wY0Vu?=
 =?utf-8?B?Nmkza1ViRmRxT1Q3dFVsWVdiQzZFMFZGd1ZEaW9wUGxaK0RraHVza2pOSFVV?=
 =?utf-8?B?Nmc0TEZXRXo5dDJIME51TEJLRnZCaXZabjZnWEtPdUNOUWZsTC9rSUgwZzB3?=
 =?utf-8?B?cUlQZmhoMFJxVS9seVBERzhXK3dueFNCeFYvdTByN3ZFMWtENlZjeUNNbHUy?=
 =?utf-8?B?dWJJZjZ1Tkp5ZGRydEd6TDBlbU8ycFlaSGpTcjNaUmExdHhkY203emxMTUxO?=
 =?utf-8?B?Ti9RQjN3ekZ1QlBrQ0lDUFlCYXFjNloyR3dJYzU4NmVuQVZBZFdFMzlIeTh6?=
 =?utf-8?B?MlhnYTB0SFhCS3h4eUFLRXpMaDgxdUpnNDdyOVNOUUZGSjg0MGx5L1MxQzhF?=
 =?utf-8?B?bnVYcjFmYW5HZjZaOU5XTDI1UzkvOWorQUR3NVBCeFVNUUlyWVNCUEdzQ24x?=
 =?utf-8?B?RmQyNlhFZWFnNDAzVGtBN3N3ZURDa0hMcit6VzNTemZoZVFiWHo0a3ZRV3ps?=
 =?utf-8?B?TGNTZDQ5L0ViL3k4RXBXMm9ScjJZbFlxTjNOM0lWNUErZjlrZjZ5UXoyMEZ1?=
 =?utf-8?B?VE5wUzJHNTdma05DbVNaRFRJMHo4NnJKRmZjaEhyUWpOdDdBYlRCYytVZTdx?=
 =?utf-8?B?THBvR0dYSW1JU3hlcGR0Qy85YWNFSUdBbGxlNnZUZk1rcW95NEs2dHFrTTVY?=
 =?utf-8?B?T0RMaWFDekRCYm5qbVFxWFczVHY2cHVDNVUvaE1MMGNOWHhNM1dGWEx5M0lH?=
 =?utf-8?B?Y0FtUC9JK2I2UkZjSXQwOHpoRTFhZ3ZvYXFEZVlWRlNMc21JZTlWRmpCeFQx?=
 =?utf-8?B?OVZCZGRkR2lxcnZmVCsreDF1LzF1cUV4OW02Z1I1c05DU0wwVmlXeWc2TjZ3?=
 =?utf-8?B?UnM0dWdHQ29YU1RIR3U3NDNvaFVRTHpQMmpJVlNHL1AzZG5tN1dQV3VGVURM?=
 =?utf-8?B?d0Vyd0Z5eDF6WHFnbDd0UjVEbC9qNEF2UkZvTHVwV0daV0M2ZXc4d3NPcWht?=
 =?utf-8?B?WTNsVjM0blpIeTB3NXdMY1lpSVhQK0hLZXpWN3Q3K3ZUeks2SDlrRk4vM2Fl?=
 =?utf-8?B?dFM0M3dkemk1dmE3SU1jY1BDRVhBRlpKakZDeWFxOFNYVUFmbVkxR3RyUE56?=
 =?utf-8?B?ZndJWnZpc2xEK2cyZGJERWlWMmsxY0hpMUVyVk1ZZExjNFhCaTdwa0pvcEhr?=
 =?utf-8?Q?aUd7F1j08+rda5YFncCBDkovOgj8G1L7eEUyPAb?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <6280667370D242438C913A210AC28899@namprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6380.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 59bbbc3f-51b4-4a94-74e0-08d97dab5034
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 09:28:18.6477
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: BZgvfx9FBk+lOiOBooWL0MNU/xU6fnAW384xCmN4ZfX4ZFm1a6XUtF0SUELW4piiL9lOSpsOX0Twn2uomAb8RI1C/vnW+hS2pgDGkO1DX4I=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR03MB5891
X-OriginatorOrg: citrix.com

DQoNCj4gT24gMjIgU2VwIDIwMjEsIGF0IDA5OjIxLCBSb2dlciBQYXUgTW9ubmUgPHJvZ2VyLnBh
dUBjaXRyaXguY29tPiB3cm90ZToNCj4gDQo+IEludHJvZHVjZSBhIG5ldyBncmFudCBvcHRpb25z
IGZsYWdzIGZpZWxkIGluIGRvbWFpbiBjcmVhdGUgYW5kIHVzZSBpdA0KPiB0byBzaWduYWwgd2hl
dGhlciB0cmFuc2l0aXZlIGdyYW50cyBhcmUgYWxsb3dlZCBvbiB0aGUgZG9tYWluLiBUaGlzIGlz
DQo+IHNldHRhYmxlIGZyb20geGwgdXNpbmcgdGhlIHRyYW5zaXRpdmVfZ3JhbnRzIG9wdGlvbi4N
Cj4gDQo+IFNpZ25lZC1vZmYtYnk6IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2VyLnBhdUBjaXRyaXgu
Y29tPg0KPiAtLS0NCj4gZG9jcy9tYW4veGwuY2ZnLjUucG9kLmluICAgICAgICAgICAgfCA2ICsr
KysrKw0KPiBkb2NzL21hbi94bC5jb25mLjUucG9kLmluICAgICAgICAgICB8IDcgKysrKysrKw0K
PiB0b29scy9pbmNsdWRlL2xpYnhsLmggICAgICAgICAgICAgICB8IDcgKysrKysrKw0KPiB0b29s
cy9saWJzL2xpZ2h0L2xpYnhsX2NyZWF0ZS5jICAgICB8IDMgKysrDQo+IHRvb2xzL2xpYnMvbGln
aHQvbGlieGxfZG0uYyAgICAgICAgIHwgMSArDQo+IHRvb2xzL2xpYnMvbGlnaHQvbGlieGxfdHlw
ZXMuaWRsICAgIHwgMSArDQo+IHRvb2xzL29jYW1sL2xpYnMveGMveGVuY3RybC5tbCAgICAgIHwg
MSArDQo+IHRvb2xzL29jYW1sL2xpYnMveGMveGVuY3RybC5tbGkgICAgIHwgMSArDQo+IHRvb2xz
L29jYW1sL2xpYnMveGMveGVuY3RybF9zdHVicy5jIHwgNyArKysrKystDQo+IHRvb2xzL3hsL3hs
LmMgICAgICAgICAgICAgICAgICAgICAgIHwgNyArKysrKysrDQo+IHRvb2xzL3hsL3hsLmggICAg
ICAgICAgICAgICAgICAgICAgIHwgMSArDQo+IHRvb2xzL3hsL3hsX3BhcnNlLmMgICAgICAgICAg
ICAgICAgIHwgNCArKysrDQo+IHhlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYyAgICAgICAgIHwg
NiArKysrLS0NCj4geGVuL2FyY2gveDg2L3NldHVwLmMgICAgICAgICAgICAgICAgfCAzICsrLQ0K
PiB4ZW4vY29tbW9uL2dyYW50X3RhYmxlLmMgICAgICAgICAgICB8IDggKysrKysrKy0NCj4geGVu
L2luY2x1ZGUvcHVibGljL2RvbWN0bC5oICAgICAgICAgfCA1ICsrKystDQo+IDE2IGZpbGVzIGNo
YW5nZWQsIDYyIGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25zKC0pDQoNCkFja2VkLWJ5OiBDaHJp
c3RpYW4gTGluZGlnIDxjaHJpc3RpYW4ubGluZGlnQGNpdHJpeC5jb20+


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:38:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:38:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192335.342732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyhm-0003K3-QG; Wed, 22 Sep 2021 09:38:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192335.342732; Wed, 22 Sep 2021 09:38:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyhm-0003Jw-ML; Wed, 22 Sep 2021 09:38:18 +0000
Received: by outflank-mailman (input) for mailman id 192335;
 Wed, 22 Sep 2021 09:38:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/3we=OM=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mSyhm-0003Jo-3I
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:38:18 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d30a9dfc-fca7-441b-b1fc-6f4ceed8d2b3;
 Wed, 22 Sep 2021 09:38:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d30a9dfc-fca7-441b-b1fc-6f4ceed8d2b3
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632303496;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=DeFHLfT6FhtD/+P5O/dO2BdBq0jqEifzE5pQ0jwkd68=;
  b=VnXQNg9xE4ag0UoIAP6eYkIz5iXcKUhWiFosEcy0F/Zo6LEalrEOQYsg
   KkAC52iS75O4xrK+Wa86Io35y5YaKZzeWwSiigl3MSDpWMZujMk3ZlDo6
   ZfLDj3aYK5cwtjmVtMPpBrz3KxDuBWJ8eBAQ11YkCl8lVv5SY/I7KQQPv
   k=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: FPdGzB7/LiMCCPAgm3RytwewjDHeCDsqwZbU138ECICK8asHPlFc75wd7ZOFRb2fppwamOkAFF
 5E2+7IgxDarqHCTYCK+aTQTxHbAapdUcAxtd/CxQkK7jFOvyS2vEj+6xhsA2Gmu6+tn8VydgE/
 Sd/YlJUI9SINH7vSke3yFk2J03Gn08fnNjlgNlQ/2MupqZRZqJSa3iZqE5XMGPkpBKzlwbUTR8
 DbYfRzFYRTNpi4A1/7yYPuok+VAamVbQ2CDFgC8jPbPDdfKF4gjFhVwvavlmoJKs1XDkYvi8Yo
 VF5E7a1GOT9bjUaZiK6etzRU
X-SBRS: 5.1
X-MesageID: 53320157
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:6+ro1qJ0DfpJeFBgFE+RZ5IlxSXFcZb7ZxGr2PjKsXjdYENS0mdRz
 GUeC2yAO/6OYmekc993btm09EpXv8OEyNBgT1dlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wbZj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2FhNBhy
 u9VmqCcWFlxNK7thbkSTghxRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFFgMg5IdatF4QYonx6lhnSDOo8QICFSKLPjTNd9Gpr3ZoWRKuBD
 yYfQR1hPFecTzFGAXg0Mag9wMjxj33+LSIN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wp
 H/C/mn/KgEXMpqY0zXt2mm3mubFkCf/WYQTPL617PhnhBuU3GN7IB8cWEa/oPK5olWjQN8ZI
 EsRkgIvsqoa5EGtVsP6XRCzvDiDpBF0c99dFeIq9SmGy7GS/h6UAG5CQzJcAPQ2uclzSTE02
 1uhm9LyGScpoLCTUWia9LqfsXW1Iyd9BVEFYSgIXA4U+e7JqYs4jg/MZtt7GavzhdrwcRn8z
 TmFoyk5g7Q7ltMQ2uOw+lWvqzCjvJ/SVSYu+x7aGGmi62tEiJWNPtLyrwKBtLAZcdjfHgLpU
 GU4d9a24PsOHZO3jAOxbPRTRLuqxuvYPyDNqAs6d3U+zAhB60JPbKgJvmokfRc3bp5dEdP6S
 BSM4lIKvfe/KFPvNPUuMtzrU6zG2IC9TYyNaxzCUjZZjnGdnie89SdybAa722n3mSDAeoluZ
 M/GLa5A4ZscYJmLLQZapc9GitfHJQhknAs/oKwXKDz9itKjiIa9E+ttDbd3RrlRAFm4TODpz
 jqiH5HSl0U3vBLCjtn/rtdIcAFiwYkTLpHqsc1HHtO+zv5dMDh5UZf5mOp5E6Q8xvg9vrqYr
 xmVBx4DoHKi1CKvFOl/Qi06AF8Zdc0k9ixT0O1FFQvA5kXPlq71vf9DKMNoLed4nAGhpNYtJ
 8Q4lwy7Kq0nYhzM+igHbIm7q4pndR+xghmJMTbjaz86F6OMjSSQkjM9VgewpiQIEAStss4y/
 ++p2g/BGMJRTAV+FsfGLvmoygrp73QanetzWWrOI8VSJxqwoNQ7dXSpg69lOdwIJDXC2iCei
 1ScDyAHqLSfuIQy6tTI2/yJ9t/7D+tkE0NGNGDH9rLqZzLC92+uzNYYAuaFdDzQTk3u/6Cma
 bkHxv3wKqRfzl1Lr5B9A/Bgyqdnv4njoLpTzwJFGnTXbgv0VuM8cyfehcQW7/9D3L5UvweyS
 3mjwNgCNOXbIt7hHX4QOBEhMraJ28YLl2SA9v8yOkj7unN6peLVTUVIMhCQoyVBN78pYpg9y
 OIstcNKuQyyjh0mboSPgixOrjneK3UBV+Mst40AAZ+tgQ0ukwkQbZvZAy7wwZeOd9QTbRV6f
 m7K3PLP1+ZG207PU3svDnycj+NSiKMHtA1O0FJfdU+CncDIh6Nv0RBcmdjtot+5EvmTPzpPB
 1VW
IronPort-HdrOrdr: A9a23:zwdNDKgoMsEa4NnRvBBeLxGZmnBQX0113DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskKKdkrNhQotKOzOWxFdATbsSkLcKpgePJ8SQzJ8k6U
 4NSdkYNDS0NykBsS+Y2njJLz9D+qj/zEnAv463pB0MPGIaGJ2IrT0JbjpzencGNTWubqBJcq
 Z0iPA3xQZINU5nFfhSURI+Lpb+TpDw5d7bSC9DIyRixBiFjDuu5rK/Ox+E3i0GWzcK5bs562
 DKnyHw+63m6piAu1Hh/l6Wy64TtMrqy9NFCsDJos8JKg/0ggLtQIh6QbWNsB08venqwlc3l9
 vnpQsmIq1ImjzsV1DwhSGo9xjr0T4o5XOn4ViEgUH7qci8fz4+A9opv/MTTjLpr24b+P1s2q
 NC2GyU87BNCwnboSj779/UEzl3i0uduxMZ4K0upk0adbFbRK5arIQZ8k8QOowHBjjG5IcuF/
 QrJN3A5cxRbUiRYxnizylSKeSXLzEO9yq9Mww/UpT/6UkQoJk59TpY+CUnpAZDyHpnIKM0od
 gtMcxT5flzp4EtHPtA7Epoe7rBNoX3e2O4DIulGyWvKEg2AQO+l3fJ2sRA2AiLQu1D8HJgou
 WNbLtn3VRCDX4GT/f+hKF2zg==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53320157"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YkFg648Oj43vv0Slrd2ZFmn3JVLLt6o5j+eDUs8GIQ26+Kb4AE4pCQBBAjsU9WCxjD9yqWBl/noKZa9DjKNQ40HVjXUu2OEgWs6pe3r4fnBKrwJIlEzBaHRqrh5hcZ4+Dug4YW6+dGqQRAD1aV7qJtf3lCHzeOlN/E5aVltw0N4pq/SM+jwdwNg77c2d0iQBZtIwMtMToT0how0MwZhU+51sReWjVRuGbXn5ci7dw8qHEZ6LeTcTfCHM45Ki36JnJKJBzkbu9BZY2CN4eJKpAYi6m7heKBIHr/th77QIFPuH/HrWoYv2UrMo6MpbIaMtaYMgj3BVIH4HXSb91bFhgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DeFHLfT6FhtD/+P5O/dO2BdBq0jqEifzE5pQ0jwkd68=;
 b=LU3rUCheTRMu6bPcFID90s+z1IBLb0BOFagHvIT/AJAjw1hZHUswpDfSMBiatH+vRPwfmvirAsuDbJr0AZIIGbKRQP2bNMA04QVZd4a+YlJLDxMYZ7NZ+ck4nx1IRzZZYnJLEfLH4453X7IoYkACNKVR88xa3y9VEJ/P8SGaHDDUtEBJ/IaPTuUmyrMQQ2uQHT9OrZtmoR2PKQ1eiool7VGiUnqtJpUUG7F3l7I3HNIuNseqojqLhwKLtC9LiJMN6VoVfPQWsRwNYMxTQEBPTxv224LykLVcKDTZJTWdlT49Xd5HfCilacqL9i+QcGvzEFHF+EBgCHxZgBwdl+HRAQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DeFHLfT6FhtD/+P5O/dO2BdBq0jqEifzE5pQ0jwkd68=;
 b=qyfAXl3yMuz2ZOKfoS+rsvRbPXlPI54JNQe34yS8n3Gzt9NlX/d5O33y+YsbgsCLkWiWwkmg00thKEurEvv3wUHFJ+ITDcOsyd7iXbehGwdwX/xyZ6SJfInKunHpZPTSu5vgDXcXBwf7Cfcwy0hzap3QFU2/pugKCWFmYhebUx0=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Juergen Gross <jgross@suse.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Meng Xu <mengxu@cis.upenn.edu>, Xen-devel
	<xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <2bd51843-de50-d442-dd5f-7fcc340c3f26@citrix.com>
 <c56b991c-d9cb-0fbf-a29a-6bcd60b09d58@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2.1 RFC 17/12] xen/trace: Drop cycles parameter
Message-ID: <47ac9e5b-abed-0201-14bf-e7ea2cd22508@citrix.com>
Date: Wed, 22 Sep 2021 10:38:06 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c56b991c-d9cb-0fbf-a29a-6bcd60b09d58@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P123CA0085.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::18) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 62744f2c-3066-44bb-9b48-08d97dacb29e
X-MS-TrafficTypeDiagnostic: BYAPR03MB4549:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4549035336089FB5043A1593BAA29@BYAPR03MB4549.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 5h7A+YXiaFy5gq61mlrsh4I2chp72ofKZqcVpjFc6w2IkFiEcz5+W/NN6ucdSjo3vXdAHXiz5wrGgt/QFL8p6yeZ4aHadMuc7CszDeyn47o4wIAJmoHAdeQrvDDSYs85JRahjGhVrFR+zmjYMJhuxSmmrkR8uIHg6S7ZVRud+5vCBxaNr1ame2Zn28iQiifvYs/R5Ia5pfKbZZc2wNDk/j2Cj5y7py9B/QOPr1ZvkaaXLQ5xPMNIDEGz1e8GxoOJdpTvQyowvbLPim/gFlh2vUD8lkf4amI8hhBH7ZjzNYu/eanZ6NaKnGCCNJrmwmuOl3FSagdjad/j/DUQtFwWAS9mu15vnfKPilOUOv0EgZHFdOIgn3pQnxc74Y7xmvPWXPMxwyomS32kKgOuqOgNUe8Mg4EqPZasbgMRHzL8DYO8/8Sm/nk7yxj9qZY6uHQM9EKf1C2o6+gseSlRmpVlcX7+thEzMfrKewqWkRjNz47yR/Ead4mTrWvkcBkOjgI5p6JUf5OJbltWYTqBtAoCWbJeoFLccw6oVsCYZ/Xfmmy9T/VqVQu8kCDWYkEr566Eoc32/czQhOWLdFTzeVaYUYXmZnGArXrH81iHbNKl4UC9hH/J6nWfeIuHKfSCADyyerDNKU0sqp56Zstl9NES80N5RJ2UOZRnbJ1bJlUxW8wyrxL+oYDFRBcTYIgWOFEyKtXnkV5y8Eq7dHFCcXMoGcNyllUJLzfKo1tcTOTgWc8=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(66476007)(66946007)(53546011)(66556008)(38100700002)(8676002)(7416002)(8936002)(55236004)(6666004)(6486002)(31696002)(2906002)(5660300002)(956004)(31686004)(26005)(86362001)(2616005)(83380400001)(6916009)(16576012)(316002)(36756003)(508600001)(54906003)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SGVBUVhNK2hqT0doMjQwdHg2VVl1Z1NVUlY3ZW50K20rSm94dnRZMlJYZHdx?=
 =?utf-8?B?S255QmFCajEvd29LQVZZSmpGZkFnajJNU3ViTEhaMk1ldGErMTA1aUgzaVZt?=
 =?utf-8?B?OVdSK0tFdFYzV244QkVuWmNtQjd1cS9OT1FuZEZXUjI5Kyt1YXhWRklyNDBU?=
 =?utf-8?B?Mmw0WjVyMlBib1pOcGFxVW1zVkNHVDNPY1p0YjFQTy9xTnltR2hBb05wVUpI?=
 =?utf-8?B?aWd3WEUyVThCZzZ3K3M5VkRNSGZTZHVzUkkrUWVwN0NQdE1xWkR3SDBaOXlz?=
 =?utf-8?B?VmhvaGRRUlloWW1qTENKYlNpNnR3MFNOcGZyNGJ6aWpmZ2taeUZhbVVLc1dZ?=
 =?utf-8?B?Q29Lc2FOL0tJKzhtd1UxNjA2bGNNbURReUVTTTBrbzJiR0N3NGNpeVpnNHI4?=
 =?utf-8?B?M0JPU0IwS1RNWEdUMHQrYmdISFN1SVY5SHM4Mk1zR3U2NFMzRExXVWMvd3Js?=
 =?utf-8?B?QWE1S25ZL2xMQ2NROCtGQmVBUEJibGZOU3dYd3I3bmMxbFVEL1MwMExMZWZI?=
 =?utf-8?B?TjdDZlBRdW9aRUVJdkJNc0RsRW1UaEhxaXJSZis5Y1RaTGYvSGkxT2NZTmxr?=
 =?utf-8?B?aEo2T1pCL2tsZ1ZOTTk0eitDODArcXg5cERqUG1IbThVRVl0NDNPZkdWYmRn?=
 =?utf-8?B?b3JFNklmQlpUdW80WGhWdEtHSG1ZSmFIWW90TmNUeFpXc0g4aGNpWUlnUFN3?=
 =?utf-8?B?MGRkeTlEY2FCQXBXRys2S1F3d1F5OU9qdTJYVXh5eklPSjNaQVJoVjlwcnVj?=
 =?utf-8?B?cFFyZElCb2MwM0JaK2N1Q05qUjZkZ3dSRk9oeVFMV3R4Q0Qzdk1JMXgzYWhW?=
 =?utf-8?B?SjlodUtQRWp3Slc4cDJWdlJLRHVUdWt1czJFRzZmN1UyUTJudHc4QzVZQWh3?=
 =?utf-8?B?VTZGalhjOVgzMFRCZlZudEJVT2FSTUR3SFQyd2QyanduSitCMlJrUXhrRkFP?=
 =?utf-8?B?eSt2TTNFWlZ1MFhJSmVTb2ZMT1NYU3ZRQVdKU3VLVmp5cW1pZm8vanlQWDRz?=
 =?utf-8?B?NXB5RXpwQlJ6QUJqT0tHblgzUDk5VlRWYXNob2hBT2ltSU1PQlJpV0NjS1Jn?=
 =?utf-8?B?VnJHZEZYbGpMUUk2N0NIQWQ2UTZpSURoOGRzN2NsTzlZQW03UTcvK0tBbGNq?=
 =?utf-8?B?NHpNUEIyNGl2UzRlbGhwbGVCTHdsSWN2Z1RXbXozSCtFTDg5UlpxZUJDM2t6?=
 =?utf-8?B?Qmp3ZklWSVN5SmpHR0NzWlhLbDA3bmFnYk9hVjV5T21UaHFPak1PMTFlTXF2?=
 =?utf-8?B?aVFNaDNKcDNiTWVIMnRNS0d4NDdQbWtXU3dseDF2ODVwVFR2TnpRcTRoQ2pI?=
 =?utf-8?B?QnQwbHM2RkRXMFNoQ3pYc0lsUkxITFE1T2J1bXoraGFhMjZodFYzclV3OHFQ?=
 =?utf-8?B?ZDgyd0YxY1IzK0FNUmEyTFZQMVdUT292clpBaHBXVUhxQ01QOVB2NXZ1SXpX?=
 =?utf-8?B?SjJSc2lUQkR1N3dVUHp6bk9QMGw1MEpueVBWZnJWNlYyanFxVWZiaklMV1Vs?=
 =?utf-8?B?ZGVtUWJBSGZUcUlZc2RxRys5T2NLN1RCQWw0OXlTc3J1Ulg0VkxpSFhpTlpV?=
 =?utf-8?B?cmpyb2dOZU1qK0d6YWk0dyt4OHNOcXZNSzFXbGVIQVF4VmtnYk12UTBYNDdz?=
 =?utf-8?B?ZnMzRGhBRFhYMUhhaTMyNmxzd0VlQktzREJTaTdWZDNMV0syZkhhTnN5cGxM?=
 =?utf-8?B?V2NSNW5DTXhDbUV1VDVWcEUyVTdraWtlMDRpTFZRRXhEMG1NcGdmS3k4M3dI?=
 =?utf-8?Q?tUSlc5IVUun78jw98L28becn51c4F3sPpl8KuLF?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 62744f2c-3066-44bb-9b48-08d97dacb29e
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:38:13.6500
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3zfBVdt3qY3ZkPJwpzmBIi4RxfU66dx8Iv90rVfhC2HS/Cwz88CXgT9w87X+kw801xkkGYVExT3F0ZRLw8HDtF3x20ny+QHz2CE3OwUIYQI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4549
X-OriginatorOrg: citrix.com

On 22/09/2021 08:03, Jan Beulich wrote:
> On 21.09.2021 22:08, Andrew Cooper wrote:
>> Not a patch, but an RFC idea for one...
>>
>> It occurred to me that the cycles parameter from __trace_var() and
>> friends is pointless, as the cycles bit is encoded in the top bit of the
>> event field anyway, and passed in the adjacent parameter.
>>
>> Dropping the cycles parameter results in +85/-1029 (-944) net change.
>>
>> The common change in callers is e.g. from:
>>
>> lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rcx
>> mov=C2=A0=C2=A0=C2=A0 $0x4,%edx
>> mov=C2=A0=C2=A0=C2=A0 $0x1,%esi
>> mov=C2=A0=C2=A0=C2=A0 $0x10f002,%edi
>> mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
>> callq=C2=A0 ffff82d04022ea20 <__trace_var>
>>
>> to this:
>>
>> lea=C2=A0=C2=A0=C2=A0 0x3c(%rsp),%rdx
>> mov=C2=A0=C2=A0=C2=A0 $0x4,%esi
>> mov=C2=A0=C2=A0=C2=A0 $0x8010f002,%edi
>> mov=C2=A0=C2=A0=C2=A0 %ebp,0x3c(%rsp)
>> callq=C2=A0 ffff82d04022ea20 <__trace_var>
>>
>>
>> Just sprinkling "| TRC_HD_CYCLE_FLAG" over the place makes things a
>> little bit unwieldy.
>>
>> Instead, I was thinking of implementing trace() (and a thin trace_time()
>> wrapper) mirroring the "new API" in patch 14.=C2=A0 Half of the trace_va=
r()
>> users should be __trace_var() already because of living inside a
>> tb_init_done conditional, and the rest are actually opencoded TRACE()
>> taking no extra data.
>>
>> Thoughts?
> Sounds quite reasonable to me.

In which case, for v3 I'll reposition "new API" towards the beginning of
the series, extend it with these two new APIs, then adjust all the
cleanup patches to avoid double-patching most of the call-sites.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:38:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:38:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192339.342743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyiC-0003mU-2l; Wed, 22 Sep 2021 09:38:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192339.342743; Wed, 22 Sep 2021 09:38:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyiB-0003mN-VM; Wed, 22 Sep 2021 09:38:43 +0000
Received: by outflank-mailman (input) for mailman id 192339;
 Wed, 22 Sep 2021 09:38:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSyiA-0003jm-C5
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:38:42 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ddc4527e-1b88-11ec-b967-12813bfff9fa;
 Wed, 22 Sep 2021 09:38:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C4AE81FD3D;
 Wed, 22 Sep 2021 09:38:38 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5E46413D6E;
 Wed, 22 Sep 2021 09:38:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id OOr5FJ75SmEWOgAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 09:38:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddc4527e-1b88-11ec-b967-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632303518; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=ZDITk7qYS46BDPvc5hGcTPTTlb+J+zDzHS0yTQOEps0=;
	b=BoBRVUN9t779M5zQ4BtUz+Sk8ipb7ZIb/PhyZ0wVfbYtcBc0suyrwTl4nd3Qg6ec7NYkpR
	zB3dDK0w2f648HiYeBT5pgfeaZEtSf3VaDL4SvZLQAOjTLLnd2Lr1EUlyrHKAnoifwqVlJ
	H9r2uscyiY+SMvJID5ClXV6eOyr3PYs=
Subject: Re: [PATCH v2 6/6] gnttab: allow disabling grant table per-domain
To: Julien Grall <julien@xen.org>, Roger Pau Monne <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-7-roger.pau@citrix.com>
 <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <c6dc7f81-0c4a-5c2f-1ac8-bb3ce5a58120@suse.com>
Date: Wed, 22 Sep 2021 11:38:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="4biEIaBNgREjbOueTenbcBFKojmVl3YtW"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--4biEIaBNgREjbOueTenbcBFKojmVl3YtW
Content-Type: multipart/mixed; boundary="atHKN1GnjO5GCPS3CqdmzqqSxsH74MF5n";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, Roger Pau Monne <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>
Message-ID: <c6dc7f81-0c4a-5c2f-1ac8-bb3ce5a58120@suse.com>
Subject: Re: [PATCH v2 6/6] gnttab: allow disabling grant table per-domain
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-7-roger.pau@citrix.com>
 <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>
In-Reply-To: <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>

--atHKN1GnjO5GCPS3CqdmzqqSxsH74MF5n
Content-Type: multipart/mixed;
 boundary="------------28E1F4F69A2E23BE3EAFEF9B"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------28E1F4F69A2E23BE3EAFEF9B
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 22.09.21 11:19, Julien Grall wrote:
> Hi Roger,
>=20
> On 22/09/2021 13:21, Roger Pau Monne wrote:
>> Allow setting max_grant_version to 0 in order to disable grant table
>> usage by a domain. This prevents allocating the grant-table structure
>> inside of Xen and requires guards to be added in several functions in
>> order to prevent dereferencing the structure.
>>
>> Note that a domain without a grant table could still use some of the
>> grant related hypercalls, it could for example issue a GNTTABOP_copy
>> of a grant reference from a remote domain into a local frame.
>>
>> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>> ---
>> =C2=A0 docs/man/xl.cfg.5.pod.in=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 =
4 +-
>> =C2=A0 tools/libs/light/libxl_dom.c |=C2=A0=C2=A0 2 +-
>> =C2=A0 xen/common/grant_table.c=C2=A0=C2=A0=C2=A0=C2=A0 | 100 ++++++++=
+++++++++++++++++++++++++--
>> =C2=A0 3 files changed, 98 insertions(+), 8 deletions(-)
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index c5a447dfcd..d507540c2c 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -583,8 +583,8 @@ L<xl.conf(5)>.
>> =C2=A0 =3Ditem B<max_grant_version=3DNUMBER>
>> =C2=A0 Specify the maximum grant table version the domain is allowed t=
o=20
>> use. Current
>> -supported versions are 1 and 2. The default value is settable via
>> -L<xl.conf(5)>.
>> +supported versions are 1 and 2. Setting to 0 disables the grant table=
=20
>> for the
>> +domain. The default value is settable via L<xl.conf(5)>.
>=20
> Technically, the version only applies to format of the table for=20
> granting page. The mapping itself is version agnostic. So this feels a =

> bit wrong to use max_grant_version to not allocate d->grant_table.
>=20
> I also can see use-cases where we may want to allow a domain to grant=20
> page but not map grant (for instance, a further hardening of XSA-380). =

> Therefore, I think we want to keep max_grant_version for the table=20
> itself and manage the mappings separately (possibly by letting the admi=
n=20
> to select the max number of entries in the maptrack).

You mean the already existing domain config option max_maptrack_frames?


Juergen

--------------28E1F4F69A2E23BE3EAFEF9B
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------28E1F4F69A2E23BE3EAFEF9B--

--atHKN1GnjO5GCPS3CqdmzqqSxsH74MF5n--

--4biEIaBNgREjbOueTenbcBFKojmVl3YtW
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFK+Z0FAwAAAAAACgkQsN6d1ii/Ey+K
gQf/TUHGBjcBE2xdsOzBhcgvuDF/enEoSTZj2O0mtSB/hBI+blSAvbwMULcnYs/dfvPvLXhWP0oA
q+sJvLOv4LSdCcwp/OPfI8P0aqhL99JuDQUSD69xrclaYokuuhJqvrDKyp5tPd56NybLLSnmJcRC
OzcAyKYHm9oZEgGgQ5MJL4MoVzhobUK7YPKv5F386ITMp6QlSGpML7emGKqZYAs5q6ihefzgXL8x
IcnNffOyf5JpuCanv0YzV9SoPs4ZON+v9L344FTkShz+CD9rOFCKWCT09HDo35wsppCqzZMt9B6N
HhoBD+WTvuOli/diX3+H6pN9rC5WF3HlCH2285QqtA==
=/4tT
-----END PGP SIGNATURE-----

--4biEIaBNgREjbOueTenbcBFKojmVl3YtW--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:39:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:39:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192347.342754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyjI-0004Vi-DN; Wed, 22 Sep 2021 09:39:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192347.342754; Wed, 22 Sep 2021 09:39:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyjI-0004Vb-AL; Wed, 22 Sep 2021 09:39:52 +0000
Received: by outflank-mailman (input) for mailman id 192347;
 Wed, 22 Sep 2021 09:39:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSyjH-0004VU-9P
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:39:51 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 07ddc50e-1b89-11ec-b967-12813bfff9fa;
 Wed, 22 Sep 2021 09:39:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07ddc50e-1b89-11ec-b967-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632303590;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=YZ8XIk1xLeJgTDygBaqUKPKxiFz8heo81bQbMKjtE9A=;
  b=ULymdDaeWiNrvz2DRoF/unCYYAy1zzOACO+fb0C8EmUVFseYgdJcRcES
   X2Acu4mzuraPwyqI3TyLycTUbGYPRdgcM+3pVP29BUaZoLAt6Ky9peoWK
   YubvbhviMhWpVzkRfm9aGclgrgYJDzULoPovN4l+9dE7wLWYeFwSTueKp
   Y=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: gNbAAjBzSZGq6moPtCGhU/hv+LACriVyTNtOFM0ork1gE+3JFazuSbT2p/W2C3/g+PRLELUcqc
 pWBtZWx9u5hJcM4enKBFdiJRkjoaG6uU4s7zV9WnEhxqvRofOFd3X9fs+DPpQK+AhKN/Ni3aOX
 f0EyMwKqDf2OBW9jSD1fNmJHfGFmZHe1KjWEaDp9unXrF4JHfMbIL8OOwnn3ysGvizo9U6WHCx
 2MUTijz59gB6t/jvX4Ud6R4FjHcHks+y1C28uIGN00rQFOWcVMHByqOSq4cmV3/9v0MOGV145Q
 35G1ovQixXNrx2cgUYnnbzDu
X-SBRS: 5.1
X-MesageID: 55105924
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:loEdNK3q2uuTYofJwPbD5UN2kn2cJEfYwER7XKvMYLTBsI5bp2cFm
 mdJXWnUaf+CM2b8edhyPIqz8k0A68fQmNExSFA9pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbRh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhsdtQ8
 8hJksGKTQ4oJbLCg9gbTDNEKnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIIBhGph3Jwm8fD2R
 swEKgpQaAz7YhR3JmgQBbwimdermSyqG9FfgA3M/vdmi4TJ9yRz36LqK8H9YcGRSINemUPwj
 nnd423zDxUeNdqe4TmI6HShgqnIhyyTcLwVELq05/t7mmq5z2YYCAAVfVajqPz/gUm7M/pTI
 lIZ0jAjpq8z8AqsVNaVdx+yrWOAvxUcc8FNCOB84waIooLE7gDcCmUaQzppbN09qNRwVTEsz
 kWOnd7iGXpoqrL9YXCA8raZqxuiNC5TKnUNDQcbSSMV7t+lp5s85jrOUdRLAKOzlsfyGzz73
 3aNtidWulkIpZdVjePhpwmB2m/y4MiSJuIo2unJdkyuzzhkYKn4XYih6EPE4+18KpmzUmDU6
 RDohPOiAPAy4YClzXLWGrxdQu3xvp5pIxWH3gU+RMBJGyCFvif5JNEOumkWyFJBb55cEQIFd
 nM/ru+4CHV7B3KscaY/SIa4Ec1CIUPIRIm9C6y8gjajZPFMmO67EMNGPhX4M4PFyhFEfUQD1
 XCzK53EMJriIf47pAdavs9EuVPR+szb+V4/uLihl0j3uVZhWJJlYehcawbfBgzIxIiFvB/U4
 75iCid+8D0GCLeWSnCOqeY7dAlWRVBmVcGeg5EGLYarf1s5cFzN/teMmNvNjaQ+xP8L/goJl
 1ngMnJlJK3X3yyfdVrSNSA9N9sCn/9X9BoGAMDlBn7xs1ALaoez9qYPMZwxeLgs7ut4yvBoC
 fICfq297j5nEVwrIhwRMsvwqpJMbhOuiV7cNiapemFnLZVhWxbI6pnveQ62rHsCCS++tM0fp
 by811yEHcpfFlo6VMuGOuiyy16RvGQGnL4gVUX/PdQOKl7n95JnKnKtg6Zvcd0MMxjK2hCTy
 x2SXUUDveDIroJsqIvJiKmIop2HCex7GkYGTWDX4azvbXvR/3a5wJ8GW+GNJGiPWGTx8aSkR
 ONU0/Cjb6FXwAcU69JxSu85w7g/6t3jo65h4j5lRHibPU62Dr5AI2Wd2ZUdvKN62bIE6xC9X
 ViC+4cGNOzRat/lClMYOCEscv+HiaMPgjDX4Pk4fBf66Stw8ObVWEleJUDR2ilULb8zO4I52
 +Yx/sUR7lXn2BYtN9+HiAFS9niNcSNcA/l26MlCDd+5kBcvx3FDfYfYW33/75y4YtlRNlUnf
 22Pj63YirUAnkfPfhLfz5QWMTaxUXjWhC138Q==
IronPort-HdrOrdr: A9a23:WGP6AK81V2atH8K48Tluk+FFdb1zdoMgy1knxilNoENuHPBwxv
 rAoB1E73PJYVYqOE3Jmbi7Sc+9qFfnhONICOgqTM2ftWzd2VdAQ7sSiLcKrweQfxEWs9QtqZ
 uIEJIOeeEYb2IK9foSiTPQe71LrajlgcLY9ds2jU0dNj2CA5sQkTuRYTzra3GeKjM2YqbQQ/
 Gnl7V6TnebCDkqR/X+IkNAc/nIptXNmp6jSRkaByQ/4A3LqT+z8rb1HzWRwx9bClp0sPgf2F
 mAtza8yrSosvm9xBOZ/2jP765OkN+k7tdYHsSDhuUcNz2poAe1Y4ZKXaGEoVkO0a2SwWdvtO
 OJjwYrPsx15X+UVmapoSH10w2l6zoq42+K8y7RvVLT5ejCAB4qActIgoxUNjHD7VA7gd162K
 VXm0qEqpt+F3r77WTAzumNcysvulu/oHIkn+JWpWdYS5EiZLhYqpFa1F9JEa0HADnx5OkcYa
 hT5fnnlbRrmG6hHjXkVjEF+q3pYp1zJGbJfqE6gL3X79AM90oJiHfxx6Qk7z49HdwGOt95D0
 mtCNUdqFh0dL5lUUtKPpZ2fSKGMB2/ffvyChPmHb3GLtBNB5ufke+83F0KjNvaD6DgiqFCwa
 j8bA==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="55105924"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RP2W6kWAGwDLtfe5vJF91gcaRT7AKb5l1C8/7czTI7wl9NzLjItF9mUkhc1fTWxQ732tgXymhrrVlGrPojatXuwdPHDpvwKxH5Gb5f9baZViFJEfbvXFJmIKYx+feoPX/3lQ4C6asnBCEnx9TYZc78PFADk33l6INY2WBZUg8QH4t9Wo7DIKmpxnDGLZvzrTzOw5hD6CuAHq+cszW0BOoXohaqWojfp0M90/QTySzN7+jTlTlQn2GBicBOwzn1Ek2toVyEQiFnx+sNKcLVadVLKpgTr9LqDfQDCcdDru3mD3LCnKufNN1T/xz0p7NFjfV90ERMR+oAqDCTjCzxd1NQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=r8QtknsmFgI2Z4VWUMIkPMcE5YErKEETJsYi0arvGUg=;
 b=L6oq8zmw4XFnZg6xoIh75xLUqRWNYolnBHVNNxBK77W+jI1yElEau52qflKY1LpJd/3mmqxfc8uqwOYuO5zcgvgdKS/N2/clR8zhSN6RuSKrROEp3kFkBfvuznd2IkpxV2kJ1msxcmsP7PVIxQDkoy3yMXGwpYRFXio498AxSO2sHnxJdkI+A2snJCTa1aUO+73ji3Q1Qswno9uM3kasw82lFGz/P+cUecfcxbBE7ebG5IRwFqTEvc3VRglY6W24i/xs/SGlNS4OvHmKxyzcSCd4TqZP5IFdPI8dVX9AJ38RkpRW012UeCXz3rElNzGwkB+018+lF3Nog0enBu8Nwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r8QtknsmFgI2Z4VWUMIkPMcE5YErKEETJsYi0arvGUg=;
 b=s92CEFzXd1ATy1APTQCgl3KbrHXaELGrYKhOtKk/Oaz+LmnM+bFc5TH4eel5qoW9IRvLv4c/vkvtsdzEL+ZH/VeIFNvzcHd0IeDU+N9zTEPTPBqcY6vIDq4ZA75Nh9KQKLBEdJtlNoHLoll1pbUYsHVgnAHQuIdTHSTM4YurJZI=
Date: Wed, 22 Sep 2021 11:39:39 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, "Wei
 Liu" <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, "Christian
 Lindig" <christian.lindig@citrix.com>, David Scott <dave@recoil.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
Message-ID: <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
X-ClientProxiedBy: LO4P123CA0274.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:195::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a5d0ff64-de66-4d83-8544-08d97dace89b
X-MS-TrafficTypeDiagnostic: DS7PR03MB5526:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5526752AE53D58D9D35639258FA29@DS7PR03MB5526.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: cPjtgZgAotRegQ5dJ1nhuVmX78/sex5qrS96TUcg37qjaS5o4W8atXSCMuERhrKzHmPpAa+zqZMGZ93BHOr2DU1KXdGeZKjw2pVOeFvxiokF7KF3Q/z8rTmRXDsZ6gnijaUm6neAXmmM5kevRa/ccrwVQaUdysiJZ7PFoNSnAquzjJN7FZaYjTDGz1XpUHUjIQI+XLPc7r8wTgD5hiSeNyw3JHYlRV4JGzz/cZbNz3ytLkMLlPckiVVyWRLXbA9hS+SREu9BMNIm33e5dcSEyNK9ebVlG7kx+fxQcBFtF1LSlUZpNdmkIZ0NnpZRyW6a06uo5GQCJ2fu1EMsuEinmiWIVLn/urU/ZyLRYQdBMfTnxL8frR5vZu1O7mHsXw2bIWfGnKxmJkNOk6WgG+YrMpRNb6ux1gBaZSjf2JXAE4j94uoZ+OGEbts+xXFec0Vvv6ZqseyK5tvZyMfvXOpT1Pn+bQfBuixtnvkztiFOeCajgGErVQd2AtaclVssiwI6xRZQb9BS5+48VhpMNtCG0xZ1lWQbkhkG7J7kBYam+FwJ8xuIEGmYKx6KABr8NovOwESlcC5GBzlUpMPqz60iN8tykaL5zQOCS+9FrSAv5SmiDyoJLL6u8KDt8GiIv+jksuAKoT2go9YUL0Sh5cKzNaNWIDLoOWBsudWhfX4AsRy9XLQFAykn/Z8m83NCQGtm9h+nwIu5Y6yjs1h71Y/meIDjBq2a/lITWSkaS8oK3uU=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(9686003)(966005)(186003)(8936002)(4326008)(66946007)(38100700002)(2906002)(54906003)(66476007)(508600001)(5660300002)(6666004)(6496006)(66556008)(8676002)(53546011)(316002)(86362001)(956004)(83380400001)(26005)(6916009)(85182001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aWd6UUR4dUhjVUo1SXRSSzB3cmI0YzdZQ1MrZU1RVlorS2xLWWFuOFVDYjA1?=
 =?utf-8?B?cnduTTRaemczVGhyRzg4UGFaOUlzRlhsTkJIa0s3b3RTWGdnOGJxVTBkTHpK?=
 =?utf-8?B?ZHM2VnoybW1NNXRHU1RKeitnaFQ5UUwyUmFqaEtkVCtqSEJNOGN2d01zSm9N?=
 =?utf-8?B?RjVpa1VneWh1aVlCMkNaUkYySFNFdk91Nzg4Z2RzMUlHaDNrNk5JczUrVGxU?=
 =?utf-8?B?VlJQVUt2MkxJWm9VdmN1S256a2QxeEtybTl3TTFuTlJhQ3ZJeWx4M0QxbW1i?=
 =?utf-8?B?UGhBckp6dlBWNEFqMTAxYk80VVBaWlU0N21BeTFKV1pLVDNhQVd5L00vakE0?=
 =?utf-8?B?NHBHQTh0Y1RDRkVNRldJeWJITWFCM0YvVk9sODhrSnZJMEpqRFZOZExwc1du?=
 =?utf-8?B?TU5ZMnZlbnl1RUR6UEdPR0MzN09iSmNjbTRGNGZNaVh2K0h5S0ZkMFkzeFJr?=
 =?utf-8?B?aGQ1SUtiNno5bFcvdVdDVkh2OWx3bjhwRE9YWitBOUE1QktXK2V3Q2JWL1lK?=
 =?utf-8?B?OWlCSmVMNlMyVVd0STZWTytvZ09ZREVlSXZYdU83TU52cXI4em9BUWs2clJ1?=
 =?utf-8?B?R0V2QTRRL21pdkJYUFdNSFd0cjBmOWhzTFYxbDFJNVFwbG1aUDhHaUMzNDRH?=
 =?utf-8?B?d3gwc2ZNZHpPLzhyS3dJbnJVMXE3Y3RNSXpwTmROb1FqN1VCOS9ycDN3dDJN?=
 =?utf-8?B?ZWVIT05DZUdDVlQyTFpvZVF5SW1Kd0VPd0RwcGVaOHJIaFd5TXVpdzVrL2JK?=
 =?utf-8?B?d0pRbTNHWTFaUVY0OWl4YTNYRFNtR0FhcVI3dFovcDhjdEVubS95eGxhdWYx?=
 =?utf-8?B?bFF4ZkZQZ1Zjek92UlV5Yi9paE8wM1pIQSsvMG5lVUJhRXZwWDZPQjhpSDFt?=
 =?utf-8?B?VkVycTdydGVuRlBaWjgwYkViU2xZTktIOXhhTjZQeVhhaXg3M1N3K1FzWitP?=
 =?utf-8?B?VTJxL0ptU28rc2FCcWVFUllOQnN5SE4zbHhNNGZ0STIwMnl5dEtlWElVWDdn?=
 =?utf-8?B?WThrWVRRbFE2clZ5UlhTSHg1KzZtU3pkOXZMZUNiSUJ2bHpwdnhFNmZHMTA3?=
 =?utf-8?B?R1JRV083TUFMUkxCVnFzTjZIbHF3bExVRFJ2S0I1dys4M0o3emw0VlN0Wllq?=
 =?utf-8?B?alhCRzNndDBEU0tFUjJhNHg5K2NLZEdVVkxuNWpKOGlMK2R5aDNwRTVSQktQ?=
 =?utf-8?B?eURKQTE1eXBrNUJJL2ZpT25GQnRyT09XWlk1NjE2SWdIVytGRU5SOGpDUmdq?=
 =?utf-8?B?K0tJcURQQTNkamRwQ2doNWt0anlZZi9GMzRISVZxR1UxU3FlbU9BVENpTmNC?=
 =?utf-8?B?VnlYSSt5VUErelRCTnRUVGdvYjRiSlAvcWkvdjJmb3I3MTZwUDFSZk5BSE9w?=
 =?utf-8?B?Y3Z5ZzdEcit2MG1OVXBSUDFqY21oMDAxVUs0V2hZOUt5dzdyZWVBY05PZUt4?=
 =?utf-8?B?dVF5UHRIdDBvZ0F6STNpalFDZ0IxRnNJSXhxVXF3c2ZDNUZ5ZDBNbi80MnE0?=
 =?utf-8?B?Q3Vha25YNDVoL1lsaXpQTUZTMGR0V2xiTXJNVlY3VUMrZXJ4Ym5NakJhWmNX?=
 =?utf-8?B?NUtsT2VnSVVXU1RWZVdmMjhDYmxBazRyZlFUVnRqVE5hOWFlT0xWYmZVYnds?=
 =?utf-8?B?c0RoZGFkL0FzdUYwem55enBVV3FZRzB1dFJ0Z0FiazQ4cUtIQ0pYTThDVDli?=
 =?utf-8?B?TXBJQWdVKzJaSUJSalpseStSb2lTenNCUWxobnBFREdrNjJWNW9XOHZDeFlk?=
 =?utf-8?Q?pEpvdlQUO9TMdfibl1A8VxjkwESIbOwMP2KfKhH?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a5d0ff64-de66-4d83-8544-08d97dace89b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:39:44.1363
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pKYDePIgoct1HGaPWQ74XS/abtgCv5aX9alHdEhdIlmMNP0JdWufeq3uIPZlJ9Mqkp0xF8SSKfc0UaVBv6GAnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5526
X-OriginatorOrg: citrix.com

On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
> 
> 
> On 22/09/2021 13:21, Roger Pau Monne wrote:
> > Hello,
> 
> Hi Roger,
> 
> > First patch on the series is a trivial change to xenconsoled in order to
> > use xenforeignmemory stable library in order to map the shared console
> > ring instead of the unstable libxc interface. It's reviewed and ready to
> > go in.
> > 
> > Patches 2 and 3 allow setting the host wide command line `gnttab` option
> > on a per domain basis. That means selecting the max allowed grant table
> > version and whether transitive grants are allowed.
> > 
> > The last 3 patches attempt to implement support for creating guests
> > without a grant table. This requires some changes to xenstored in order
> > to partially support guests without a valid ring interface, as the lack
> > of grant table will prevent C xenstored from mapping the shared ring.
> > Note this is not an issue for Ocaml xenstored, as it still uses the
> > foreign memory interface to map the shared ring, and thus won't notice
> > the lack of grant table support on the domain.
> 
> I find a bit odd that the Xenstore support is conditional to whether grant
> table is available. Are you expecting domains with no grant table to have no
> PV drivers (including PV shutdown)?

I don't really expect much, as having guests without grant table is a
developer option right now, if someone wants to make use of them for
any reason it would need some thought.

The other option would be my first proposal to restore foreign mapping
of the xenstore ring on that case:

https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.com/

But it's also arguable that a guest not having a grant table should
also likely prevent foreign mapping attempts. Plus such foreign
mapping won't work from stubdomains.

I'm fine with dropping those patches if they turn out to be too
controversial, I think it's an interesting option to be able to
disable the grant table, but I don't have a full picture of how that
could be used in practice. Maybe others have and would be willing to
pick this up.

The xenstored patch is mostly so that I can boot guests without a
grant table using xl and test it's disabled using XTF.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:42:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:42:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192355.342765 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyly-0005yx-VM; Wed, 22 Sep 2021 09:42:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192355.342765; Wed, 22 Sep 2021 09:42:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSyly-0005yq-S0; Wed, 22 Sep 2021 09:42:38 +0000
Received: by outflank-mailman (input) for mailman id 192355;
 Wed, 22 Sep 2021 09:42:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSylx-0005yk-38
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:42:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da850208-27c4-4ec6-9e6a-045a230829be;
 Wed, 22 Sep 2021 09:42:35 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-uwwGKn7bPUWryCrDMtmVMg-1; Wed, 22 Sep 2021 11:42:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4381.eurprd04.prod.outlook.com (2603:10a6:803:6d::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 22 Sep
 2021 09:42:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 09:42:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0073.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:18::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 09:42:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da850208-27c4-4ec6-9e6a-045a230829be
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632303755;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=iDjBQ38nJO8k1QEfC+dQfSOVo0hUKeEfFVKmKsaSuZA=;
	b=PvWdGu3YKGV7cRtJqw6+zNKZpObOJNWyA6umlsvEZ6HFbzGB7efsLycNxNZI4ZecyhiU0Y
	IxOXkR1yMcG0BzGV3QrP9OBGXQ0Eh/3q39m/B8kgTkdu4SJZVjAU8H9ocZbc8V8NqO7Tv4
	pb8sBhttTO1XY2qBUCv//Frkr/TclQM=
X-MC-Unique: uwwGKn7bPUWryCrDMtmVMg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DX5HxuabYYLmatJY4WB0Ni6uXcVWs17djQ4RDSRc23PYdaGsTLtArzVhmCbQkbTbAlaulphOGfLnHO7XKve8l1gHQ90r6nu2xS1fcznyQiT9lPYJJOt+sxPklXzVTUJvcqLQq84Lf/tXwOIZPMKCAa0vrRl1B0hKjhGlOKBWK4iETywlJzstQJQaiCorC6v/cjoBprYjIXfn/Idc1yAWZqqPcCgLsbDdDJBDpqEF2L0MOXi/2eDmWaq69eiQmtclQQZzYzBDIdUPH/RvuSIb4QVwJjVhn9KbLvuhOua6iLPY1IrCERmIOO6uXBD6eU75YE6cHyfxpasFcqK4cgzf9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JlO4UEtcn4rMUvF0lnKX7qcvk1YI7qoy84TsIWDScdg=;
 b=Lktjx2diPO+td9d5hf4JA20RrZP/ZS3K1vUsc+jFVwDZAuaSE6tJM9SV2Lso3rf12yvZAzlrZxY0KQt3fGYyNecw40OrGie4XwwTInYjy64YzgFnX0JQ6b+FhASZ/giWLgirNXwihMqfv5i+flLHzc3VaosFHoUyLGLy/YxukebYy7lIoUWp8Mzl6xgHaVjs+O+2zloS0dBQVYTK+DdwRYbIZxK6i2XPAeXZgGKL795pc+LYLiZ1x2C6jH8owjX3M+l5PEy2LH9c/3NTOKwW+9BR31gUiPvJzpFVhohxfsCH+3bCMUUdE2771yELIF9jIBsETtvhCb0vxjDEejfA/A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
Date: Wed, 22 Sep 2021 11:42:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0073.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:18::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 07315a81-ac3c-4e38-5ac1-08d97dad4c61
X-MS-TrafficTypeDiagnostic: VI1PR04MB4381:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB438182AAB9E2BD3CB9C20CC2B3A29@VI1PR04MB4381.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F8KpQAYtL85O3CqVTQrDEAxSoS2CP7R2nyemVQqHOW21a7lXFwaIugT7GhQ4NR2Jzl5x1bgLcfzW18eLBhEhFAKb2zHCbReL1+E5Eq42vqXXOsOwmudi/uaIwLEwuy48mh9Yy615f3At1Ho4IFLU4rhZQfE//y/EjgIu7tF6+whjxcujQ/38qYgHZeJEaOBCW8zwmY5O1gMjaFxyJz3F3GRR6uNaYVfB5+qZoSCXFIvJyNhvAtC1ddrfN+lHo1W/iLGa03fQEuUayeyxwI7ViULuW3SOIRo+EEEpjE4xs6MSt3lIFLifbIS5dSBap/vlacWL3mIftRCb8Ia8NB4bPUEkdA1oCigthZhhMEUY1hpMmnhQjP6uJnQJpfyEPGpv+IlKODhoDxuUd6mWaDgVCqLDMM8qrW4WjpVaA4/eu6s/IDv/eMmV1gFoUJAxinMsEKH8FfBNyeYWw/zlUq4HNbqhPIYPCM3tcLemiEFfE8N5KF6G6mBoHRR9WRlCLmSF7n8byCLzwuWOYXfw75aGkcEPWHct5+ILNtSbchG3c8nRpJ30W/Axz/+bx6POzqcat/iVh/jBI/BUYlonYSzM9arTxY0qlwCN0khL+3au0P3mElOUC/RGT8o4512svB0mGWm3lT94/mYCjsrqOX+fQWTiSoBAyoeyJvFcYoXE0n0aSGpZiZTAnkLt9002Y/xoebCV/yftHvNqqaa3fwZ8qhps/NxnX5/PmUIA1Npkec508EsknQbNPx6SbCDton/T2lVD+T76iBPumaNLaNe2Ew==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(86362001)(508600001)(83380400001)(16576012)(6486002)(53546011)(316002)(36756003)(8676002)(2906002)(54906003)(31696002)(4326008)(2616005)(186003)(38100700002)(26005)(66476007)(66556008)(66946007)(956004)(8936002)(31686004)(110136005)(76704002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?CM08BZww7A35lbq1kPZEw2NJS2s3bDqKI1XhuXoOu+PT2Nn4igiVKXmMiwsO?=
 =?us-ascii?Q?mOFgyUalqlJpewoaupRzKIZYY9yNenuqdHHDhKZRmxNhDAkevBuebd9G+r9w?=
 =?us-ascii?Q?w29+LlS3Ko/qG6fMrU+3iISdZh2+fMTsI4WZ5T5H9fHz9pxa+oVW2nJzIAOA?=
 =?us-ascii?Q?r5qw8NlANVvvoXV4kzXuK+tcQ67ieIJwpAmGYqqaT9gtwlYspOVl1vryxVLj?=
 =?us-ascii?Q?+L7+l8/u0tfbhcgFifpnHoQK2pi+iwKP4s7QwCtPxYFWkynMvJuz1hCnGeVu?=
 =?us-ascii?Q?J4vhbyDUPebB/0DUKXgrZAB1VyBTM5a0f4al0zP/VBj42XJmf6GwkEi0aPk5?=
 =?us-ascii?Q?Ens7wjwPfBLKOBZpnZZXm4G7HSGw/wlTok22NTYgfS4YUZM0v33xtMgKzxj3?=
 =?us-ascii?Q?WxgGlHD8YHgnPxNc+C4RQ3lurN4BPB97i+JybmdssIOcv6gRcVL9lJmyxkDC?=
 =?us-ascii?Q?7rmX4Z2X0J3CN3iEr7E/4qMs4nAd938Xol42Uzmrpc4sCi2JNAasBp20ra6Q?=
 =?us-ascii?Q?unIpECWAltxK6T8cwNDQ0RsGCfyoMj/iGc5skbLWMl8BW+P4qiPceBC/6qnk?=
 =?us-ascii?Q?D5ZjPUZEYx1wWUAjd7XO9/n/QM3DUD2kVyZt3Qbv+cmgtQKm6HHcr5KQgndE?=
 =?us-ascii?Q?eE6uIqAneYu4xITkrWBy6Ebu/jha8Q+buiwYzjNsk74PIJ0dh6PPetmbA/Si?=
 =?us-ascii?Q?djg3ptNvlUyaXFwRX72YhOf2FeQZE3w5zDMyOTSvlUiSBsNEnkWe+eXaUGKE?=
 =?us-ascii?Q?wOutnuRFgCkftS0k96KQd0dUeZw7mC8KcSDRKjfgRq/p1MvLfbNRRUTbKPSW?=
 =?us-ascii?Q?VbHtRiS99Bb7AqPJtGFsAZuZ0hGoJ0uuik9m63Y5NuIwsaxt2AuT3hp0K/Yi?=
 =?us-ascii?Q?Q/bOjhbssdt7X9WENlNCD3JUPv1z9aPQTZ1wCPgOEfYkRIYItyVizn5GkOAu?=
 =?us-ascii?Q?yhA16aILPR9EAOyqA2cIUe29Y/U98ycBptuplxuV/RBQ1UNhMWFY6XYnpOL4?=
 =?us-ascii?Q?OIPbxFJK77KXVy/56vK1qTJdmpuwLYwDtpDijJCUjYJrjVWdpBPEewvoNvQE?=
 =?us-ascii?Q?XOZLFO+MtwR/o4lB5PmjRW6HkhMkpk+vrEp93Qr9PGP6MxiVDCP/UN0jilfc?=
 =?us-ascii?Q?kq25v/shRc49kPAQsMC9miKeYruy12y6xRdH7E/nqtTvMOtLFgPv/ggKeGE8?=
 =?us-ascii?Q?16ttRb52rnwQxtdK0tQgUBgFFfnZ6hAQCdtmEHDz0F+BQOrxY4mY2kxquSoy?=
 =?us-ascii?Q?hGSgFqorCJk6oHUbSBLfq0N2/cDaOZBL5eu/9hemCm3MQCLFOV9jN7a246to?=
 =?us-ascii?Q?wvKsZWabIpFgkJfGJon3QHN8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 07315a81-ac3c-4e38-5ac1-08d97dad4c61
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:42:31.4698
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ImVMNTR6gYKHbokDdbNmMdjMBfraEpr2AOcH08CrsA4/j5aN5XC/mfXT16H9yY5j1CUb0aOs8o2+7Eqi1jdIng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4381

On 22.09.2021 11:26, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
>> On 21.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>>>> On 20.09.2021 12:20, Roger Pau Monn=C3=A9 wrote:
>>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>>>> --- a/xen/include/asm-arm/grant_table.h
>>>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||     =
      \
>>>>>
>>>>> I'm slightly confused by this checks, don't you need to check for
>>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>>>> guest_physmap_remove_page?
>>>>
>>>> Why? It's ogfn which gets passed to the function. And it indeed is the
>>>> prior GFN's mapping that we want to remove here.
>>>>
>>>>> Or assuming that ogfn is not invalid can be used to imply a removal?
>>>>
>>>> That implication can be (and on x86 is) used for the incoming argument=
,
>>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>>>
>>> I guess I'm confused due to the ogfn checks done on the Arm side that
>>> are not performed on x86. So on Arm you always need to explicitly
>>> unhook the previous GFN before attempting to setup a new mapping,
>>> while on x86 you only need to do this when it's a removal in order to
>>> clear the entry?
>>
>> The difference isn't with guest_physmap_add_entry() (both x86 and
>> Arm only insert a new mapping there), but with
>> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
>> mappings. And gnttab_map_frame() gets called only from there. This
>> is effectively what the first paragraph of the description is about.
>=20
> OK, sorry, it wasn't clear to me from the description. Could you
> explicitly mention in the description that the removal is moved into
> gnttab_set_frame_gfn on Arm in order to cope with the fact that
> xenmem_add_to_physmap_one doesn't perform it.

Well, it's not really "in order to cope" - that's true for the placement
prior to this change as well, so not a justification for the change.
Nevertheless I've tried to make this more clear by changing the 1st
paragraph to:

"Without holding appropriate locks, attempting to remove a prior mapping
 of the underlying page is pointless, as the same (or another) mapping
 could be re-established by a parallel request on another vCPU. Move the
 code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
 xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
 course this new placement doesn't improve things in any way as far as
 the security of grant status frame mappings goes (see XSA-379). Proper
 locking would be needed here to allow status frames to be mapped
 securely."

> TBH I think it would be in our best interest to try to make
> xenmem_add_to_physmap_one behave as close as possible between arches.
> This discrepancy between x86 and Arm regarding page removal is just
> going to bring more trouble in the long term, and hiding the
> differences inside gnttab_set_frame_gfn just makes it even more
> obscure.

Stefano, Julien?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 09:58:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 09:58:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192385.342788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz14-00086H-Bl; Wed, 22 Sep 2021 09:58:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192385.342788; Wed, 22 Sep 2021 09:58:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz14-00086A-7l; Wed, 22 Sep 2021 09:58:14 +0000
Received: by outflank-mailman (input) for mailman id 192385;
 Wed, 22 Sep 2021 09:58:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSz13-000861-Fe
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 09:58:13 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3aa6c4b4-5b1d-42e8-9fa0-d20add663a38;
 Wed, 22 Sep 2021 09:58:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3aa6c4b4-5b1d-42e8-9fa0-d20add663a38
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632304691;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=DTRDvWjV3s8RJWq5ihgcGzd0yOtwznkzp54/ChzgSsg=;
  b=St5bI7HABuHwccQvQdwi28uvh41cjM6Iq5for4Gf9wIWkVJ2ijWzxGQd
   QmpV9WSAsqHiMbzzkNpYFXiJ5xkvsm+wYZKv/kWcACBYHI1IsVnbpuMRG
   ER0x0D/ZZtne1ZiEKyRRh2tybrC39lu4/xgF32xSbVFLtLPmWbGsv9a7p
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 5T8Zr5SU7sMpWSQnJVtxeJK7T/IGxN8mI2WYdPJ7uhTBmqroHO7Sf8ACna9xZlfxVRw2q7T8uL
 OPdmtMTzZX9Wowd3PRuOB1rvuRTRA/mvLKi5SQEI//6zGJ9outfphwqC9sEAP1OLv3KzbYkx1k
 X0VaIiWaivEYu+Fo6xSQqmWC/IWzkVqyyQB1WAZ+7u48JoOQmyoY0zbwRZ8ZmqUfTEiK1q7D7n
 vUZYw7juOA+qQQGDDDFtEXvtJFGbqblI9xJ2/pd5t090Awa03+kinaJixLmSunNjPGxucgzMnC
 KzYzs4UOe+aUA73DAbMIohF9
X-SBRS: 5.1
X-MesageID: 53300481
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:4PATpq07erMgiyz30fbD5et2kn2cJEfYwER7XKvMYLTBsI5bpzJSy
 TccCmzUOfqPMDP2Ltt/PYW0/UwHvpXUy4IwSVRvpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbRh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhnI1/+
 ehQr5GLFiwCIKvMgMI6VyVWOnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQDhG9s1pkm8fD2f
 PA/bBxvQDX8SARfMGgYAqslv8qpiSyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1XkVyfv
 Xnu5HniD1cRM9n34Tua8Fq8i+nXhyT5VYkOUrqi+ZZCgkCXx2EVIA0bUx28u/bRol6zXZdTJ
 lIZ/gIqrLMu7wq7Q9/lRRq6rXWY+BkGVLJt//YSsV/XjPCOukDAWzZCHmUphMEaWNEeRmEk5
 2KUhd7SWhdErruVUVS68ouahGbnUcQKFlPudRPoXCNcvYK5+dFt0kqWJjpwOPXq1YyuQFkc1
 xjP9XJn1utJ1abnwo3mpQivvt66mnTeoufZDC3sV2S550tSYIe/buREAnCKsK4dcO51orSH1
 UXoevRyDshVVvlhdwTXGY3h+Y1FAN7fa1XhbaZHRcVJythU0yfLkXpsDNRCGauUGpxcJW+Bj
 LDvVfN5u8YIYSrCgV5fSIOtEcU6pZXd+SDefqmMNLJmO8EpHCfepX0GTRPAjgjFzRl3+Ylia
 MjzTCpZJStDYUiR5GHtHLl1PH5C7n1W+F4/srihnkz7juLBNCDKIVrHWXPXBt0EAGq/iFy92
 /5UNteQygUZV+v7YyLN9pUUI0xMJn8+ba0aYeQNHgJaCgY5SmwnFdHLxrYtJ95sk6hPz7+a9
 XChQE5IjlH4gCSfewmNb3libpLpXIp+8i1nbXB9Yw7w1ihxe5ur4Ycea4AzIest+tt8wKMmV
 PICYciBXKhCE2yV5zQHYJDhh4V+bxD31xmWNi+obWFnLZ5tTgDE4PH+eQ7r+HVcBya7r5Jm8
 bahyhnaUdwIQAE7VJTab/emzlWQu3kBmb0tAxuUc4cLIEi1qdpkMS38iPMzMvogExSby2vIz
 RuSDDcZufLJ/90//u7WiP3WtIyuCeZ/QBZXRjGJ8basOCDG1WO/2oscAv2QdDXQWW6oqqWvY
 eJZk6P1PPEdxQsYtoN9F/BgzL4k5suprLhfl1w2EHLOZlWtK7VhPnjZgpUf6vwTnudU6VmsR
 0aC2thGIrHYasrqHWkYKBchcuneh+ofnSPf7KhtLUj3jMOtEGFrjamG08GwtRFg
IronPort-HdrOrdr: A9a23:yR2iRaF++gSehrmYpLqFdJHXdLJyesId70hD6qkvc3Jom52j+P
 xGws526faVslYssHFJo6HnBEClewKgyXcT2/hsAV7CZnidhILMFuBfBOTZsljd8kHFh4pgPO
 JbAtdD4b7LfChHZKTBkXGF+r8bqbHtms3Y5pa9vgRQpENRGtpdBm9Ce3em+yZNNXB77PQCZf
 2hDp0tnUvfRZ1bVLX3OlA1G8z44/HbnpPvZhALQzYh9Qm1lDutrJr3CQKR0BsyWy5Ghe5Kyx
 mLryXJooGY992rwB7V0GHeq7xQhdva09NGQOiBkNIcJDnAghuhIK5hR7qBljYop/zH0idnrP
 D85zMbe+hj4XLYeW+45TPrxgnbyT4rr0TvzFeJ6EGT6fDRdXYfMY5slIhZehzW5w4Lp9dnyp
 9G2Gqfqt5+EQ7AtD6V3amJazha0m6P5VYym+8aiHJSFaEEbqVKkIAZ9ERJVL8dASPB7pw9Gu
 UGNrCc2B9vSyLZU5nlhBgr/DT1NU5DWituA3Jy9PB96gIm30yQlCAjtYsidnRpzuN1d3AL3Z
 WDDk1SrsA6ciYhV9MKOA4we7rENoXze2O5DIuzGyWtKEhVAQOGl3bIiI9Fk91CPqZ4lacPpA
 ==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53300481"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wvrf4ja4K8Dubs5Mniqt7EAlltXf7wexR9fasCOjBksLJvTwalMuTq9kPpp5r4HhPsiw34L0nRBeWMFExKFU3gPVoM3A/X5xqSS0SoYpDuMfJccqtAuo223Ex7R8bfaA6mNYrm7YzjhhuzHTIN2mrAGTYMnj7n0+i7PhhA/kBQ0WiH538H4KrKJ1XX7Ss9jdeNxmKILaHVdtzzLMuHaKOgpDO4+TyCtAR02k6VOXkNtpTsltq3tof/m34aEXPQKEW9nnmAk+OaUCr9fvi3I1JEbEEV9+95JRKpVUJ7yinEu4MvsDG5J8UycBUFeif5lJ+q6N3fvsySnexK+3GI0NIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=iOweIw6GDgJGrSJfIfCc3qcy8BGDuRSYU6DmWE4tBrg=;
 b=hCve4hT35pE5scLUtLBHQ0ecfBhSdNec6PqedYKZGTLhlh1kReopuz410utedyTYBB1+2WkM0y/gh9P8vMoBgFv3j804IyWnc7WPIztzJUNz8rtmgnMIEmFxDEpRueFMiAOF1+mtA5IxXmykckSaCZuMDlkPW8rhTwbVkymHVF3cKAtLCkTY3C6uRYr8S3WCvM+tb0ksLFYSND5PeaKgnW2j7xttGUBdpFjuTaaKSuWqnjvkKq3yN0skWI77Kk4sMyWw6oYvVCwqWmGDQg7dEUlaSKQc7H7DZm1TlbIFMGyXNeTgsMKHXtogRYqd3Z9uWqGXD5mnASuVkh/v7tSDsg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=iOweIw6GDgJGrSJfIfCc3qcy8BGDuRSYU6DmWE4tBrg=;
 b=lY6e4CdaTpCZvTb++l5dYivOZsfppzgR9Px1bTSrAQybmqPETy6M7pYkjjaOdwDTAo+Jzd4LMFQPAJLUV1jiFh1UF1eK33SCBBhwfjO++f/ewJbamHggy1CjK8fzEaFuRq6s2y15TZ1OFwAkSi1i8uyDIWk1le77TT+jjkSVWyM=
Date: Wed, 22 Sep 2021 11:58:03 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, "Wei
 Liu" <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
Message-ID: <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
X-ClientProxiedBy: LO4P123CA0036.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:151::23) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f7e2e83f-0783-449c-3e01-08d97daf7a60
X-MS-TrafficTypeDiagnostic: DM5PR03MB3370:
X-Microsoft-Antispam-PRVS: <DM5PR03MB337026DF38E8F14CFCC06E318FA29@DM5PR03MB3370.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: xFodaEwPN+EyugCyukMtg3iMxDrKwJUxKaiMnPkUwmL333HO7utu3JqXSPbpFQJbN2370uOm+4FaCgi6MpN7aPSt6XBDkCS3bZA+XJhCx5d+iE7l8ZfXy6XIV6UIu4ZrRbiJZehTEz4k2lyH4mY4ZSVIrXcHVSycD2MeA9Lek2z/wwrMMxBWtBTf4gs4XHX3JMCp4ng3HM9lDwv2ok7uAo7rvnorUCe17Hjtk/x/VhfsH07k1PEks6hN9KSDKzzCXjVJNnJQ5ZrOQJu/6eAjkE1G/0mp3ArH3Ky9X1MQejKESyusdesqC/qVNltK3KFC7g4D6J6sOt6SXZXOAr8Oiso9GfZeLMnCuC697AgTRxk2xBpJgMAYed085p3jBvCD9na4Q1EnjgIsMWb3QbPvRkhVsHKOLDMf0T3VNxzs5Yw3VOF25YPXTFCI1AuWRCtyDs4pDsrovDr3bi8/KgFM4eLDc/mDwQq3hDkZi6E7ImNea/8jEQid4iWy9sRgJH0z6e8u0Y2YBOqxEzJ8vnWoWW6E6T/62qMToudUdc954sZxrIzTik3gWtdbT6ea/6aaLWGpwyHYJ+CaTkEeHgkAq6/8nrFrnKk79Am+wuUCPFjaVnNyBe0NMAvgXmTU53gfXkv/24ePrX2jZWO3FNMQxSbcooqhw4lZm4Qh2AtLu+7uz3wzknqTfnU2/136bhkDlAeg3PCmqIUOkW51+vgajA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6496006)(66556008)(6666004)(53546011)(66946007)(26005)(66476007)(6486002)(5660300002)(83380400001)(186003)(8676002)(316002)(8936002)(54906003)(38100700002)(2906002)(956004)(86362001)(508600001)(4326008)(9686003)(85182001)(6916009)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U1J2L0N3R0xxa0lTa29CeXkvQnVyc29QODZDbkp4TGdGWFI4THR3WjgrS2Vr?=
 =?utf-8?B?TzNUa3BLSFYvL1hQMGwrWUtYb2x3ZWVCcUkzOWJVRkU5dm92M3FQZGFubytj?=
 =?utf-8?B?NFlOcDhkQU5hWmxOVDB1NUZLUlc1WmxlOVZubUhHaGVFZWVlaDVVNzd1a3pm?=
 =?utf-8?B?NlNwMGRNTmRIQnJpWUZyUWI0cWJWbmEyNGs0bGNWUDErUDFHdmdrYXlxaUtG?=
 =?utf-8?B?a1k5NUo1clQ3YzN4WTl4KzVBMDlrR1ZvOW15ZDU5UC9wNWtsQTVoK3Ywd1hT?=
 =?utf-8?B?RmJNaFkwSGVnNE1ESXFmd0xYVXFBMzY4OTQ3MVkrYkQrOHh3dXRHWkh3cnVT?=
 =?utf-8?B?ZkdBTGpoNTlwZjdKcWI4QWtUakl0VTBJM2VGNWJ5VDdvYWpsZDF5ZmlhWVl2?=
 =?utf-8?B?U0lUTzRYa29IYUEwVTRFcDVDbU9iNCs2YU83eW5aOFMxUkhtdWtKQVdvMmpW?=
 =?utf-8?B?OG9zK3RjNjZvK09sSUM0WlhvWXF3ajY1bnoreUQrbVdIN3VXb2RybEVOOVNa?=
 =?utf-8?B?bXZ0SVkxa0hhODh0eVBjbHdVN25pTm1ZMFNUVkFmNGhSV0tPTTZ2dXRnVEM1?=
 =?utf-8?B?eXpJNDZQMzduZzVGN3Q5aDBUdkp1a1E0V3hzVnVlWjI0VUZySlRvbU9pRTkx?=
 =?utf-8?B?eVpsVVZURGtTYWJKY0s4SldZYlBVVVZ3MkdQeDZ4NzcvN2Z0aHAvejluc0J1?=
 =?utf-8?B?Qlg5OTI4MjA0TE1oRGRvV1B4d1c0Qys3TVJVcEthbnRyeTBWTVJzQUIzUVJR?=
 =?utf-8?B?MHE3a08rWTdKbS8xUWttS0ZCNzExOXkwSEdmdXRueUFuMmlQMnV4RGErK3Iy?=
 =?utf-8?B?TDFqZjI4ZW9PNFc4aXptZHJVS3FZTFBBT3dyMTlFWVVWMWxVZEZ0WUlhY3BO?=
 =?utf-8?B?c0x4Y29QMzZQUkRQKzlTSVF6MnV3aHNrU3FDYlhPb3Z4SXNVTzBSbUU5MTNw?=
 =?utf-8?B?QVVmcEI4RndWZnA4ZW05UHVZaHNTRUF0R1FnNkIyZFRFQkNKTEg5ZExiaGc3?=
 =?utf-8?B?OG5xeko4NWxnSGRZY1NqREsvRytNUzZBOFh4cUN4dXJaQXRzaXlkUS80aDRs?=
 =?utf-8?B?SThDQmxvc0lwTFcrZ0lUaExEVWVPSS93RWlBd2RrU3kzUXZDdDgzVnMvS1RP?=
 =?utf-8?B?bTdPR3pySWEwMWx0QUZRaUJJa1JWVW1vTlcvSGVDM3JDZFRZaC9JeG5KOTZN?=
 =?utf-8?B?M1hJZExZU3U2VDZVakREay84VCtZM2JFSmdmRXB2T0lKZk9VeFh4TE5mYlJ4?=
 =?utf-8?B?UU9NNmhoRnhZVEM5YlI3UGd2WmdxWXQrOXA1YjkzbjJyR2x5aFRwQklkdHRm?=
 =?utf-8?B?NFhQK3laajEyMnhMQzZ4TjJEQ29Jb1NpK000UmRaQTV6RlVtZ1Z2cHhSS2xI?=
 =?utf-8?B?UlBaaUlYR3VGSGJBcnhlemNlaWdRVG01VHF5YVpyZ0Nndy9OT3ZoOFlJc2I0?=
 =?utf-8?B?TkRkRFVYUFMzeVpnSldWYkJwOXF1VThsRTJxOGw0RDl2R3RtRFZzYUgyU0Vl?=
 =?utf-8?B?QStnM3h1QUt3ZGRCaGlPR0lZdjkxZSsxRm9waUU4SVZNNHJNWTU3RDc1MEZk?=
 =?utf-8?B?eVplR0VYR0c1Vk5SaXZOdXQrL0VJOXFBT1AwSDlaL1c2M1lZeFB1UkhUdUIv?=
 =?utf-8?B?RXE4WXV4bTNpNFN0cmFjQzhtYUZ2akxNZTFUZm54UnNIRUJMWmtLRGhzSDRj?=
 =?utf-8?B?UHR6aHlNTlAyNFdBN0VxRjdtSmlRb2RYbzFqMkxFalMwTVFQUm9hQ2d5V3pK?=
 =?utf-8?Q?k0ijAKCXzpkAjSpr85LlDEHFqYTnYbMsqDkMMW2?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f7e2e83f-0783-449c-3e01-08d97daf7a60
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:58:07.7025
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /5ECqy4FzhLGhyEdLMqWMwQ9mRl+MHiIRLLxuXYeBZuDnzqMeto9T8c9wN6B+JDsXZ1HhpsNz7zuwRc/Yt812A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3370
X-OriginatorOrg: citrix.com

On Wed, Sep 22, 2021 at 02:07:44PM +0500, Julien Grall wrote:
> Hi Roger,
> 
> On 22/09/2021 13:21, Roger Pau Monne wrote:
> > Failure to map the shared ring and thus establish a xenstore
> > connection with a domain shouldn't prevent the "@introduceDomain"
> > watch from firing, likewise with "@releaseDomain".
> > 
> > In order to handle such events properly xenstored should keep track of
> > the domains even if the shared communication ring cannot be mapped.
> > This was partially the case due to the restore mode, which needs to
> > handle domains that have been destroyed between the save and restore
> > period. This patch extends on the previous limited support of
> > temporary adding a domain without a valid interface ring, and modifies
> > check_domains to keep domains without an interface on the list.
> > 
> > Handling domains without a valid shared ring is required in order to
> > support domain without a grant table, since the lack of grant table
> > will prevent the mapping of the xenstore ring grant reference.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> > oxenstored will need a similar treatment once grant mapping is used
> > there. For the time being it still works correctly because it uses
> > foreign memory to map the shared ring, and that will work in the
> > absence of grant tables on the domain.
> > ---
> > Changes since v1:
> >   - New in this version.
> > ---
> >   tools/xenstore/xenstored_domain.c | 30 ++++++++++++++++++------------
> >   1 file changed, 18 insertions(+), 12 deletions(-)
> > 
> > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
> > index 9fb78d5772..150c6f082e 100644
> > --- a/tools/xenstore/xenstored_domain.c
> > +++ b/tools/xenstore/xenstored_domain.c
> > @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
> >   	struct xenstore_domain_interface *intf = conn->domain->interface;
> >   	XENSTORE_RING_IDX cons, prod;
> > +	if (!intf) {
> > +		errno = ENODEV;
> > +		return -1;
> > +	}
> > +
> >   	/* Must read indexes once, and before anything else, and verified. */
> >   	cons = intf->rsp_cons;
> >   	prod = intf->rsp_prod;
> > @@ -149,6 +154,11 @@ static int readchn(struct connection *conn, void *data, unsigned int len)
> >   	struct xenstore_domain_interface *intf = conn->domain->interface;
> >   	XENSTORE_RING_IDX cons, prod;
> > +	if (!intf) {
> > +		errno = ENODEV;
> > +		return -1;
> > +	}
> > +
> >   	/* Must read indexes once, and before anything else, and verified. */
> >   	cons = intf->req_cons;
> >   	prod = intf->req_prod;
> > @@ -176,6 +186,9 @@ static bool domain_can_write(struct connection *conn)
> >   {
> >   	struct xenstore_domain_interface *intf = conn->domain->interface;
> > +	if (!intf)
> > +		return false;
> > +
> 
> Rather than adding an extra check, how about taking advantage of is_ignore?

Right, I just need to change the order in conn_can_read and
conn_can_write so that the is_ignored check is performed before the
can_{read,write} handler is called.

> >   	return ((intf->rsp_prod - intf->rsp_cons) != XENSTORE_RING_SIZE);
> >   }
> > @@ -183,7 +196,8 @@ static bool domain_can_read(struct connection *conn)
> >   {
> >   	struct xenstore_domain_interface *intf = conn->domain->interface;
> > -	if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
> > +	if ((domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0) ||
> > +	    !intf)
> >   		return false;
> >   	return (intf->req_cons != intf->req_prod);
> > @@ -268,14 +282,7 @@ void check_domains(void)
> >   				domain->shutdown = true;
> >   				notify = 1;
> >   			}
> > -			/*
> > -			 * On Restore, we may have been unable to remap the
> > -			 * interface and the port. As we don't know whether
> > -			 * this was because of a dying domain, we need to
> > -			 * check if the interface and port are still valid.
> > -			 */
> > -			if (!dominfo.dying && domain->port &&
> > -			    domain->interface)
> > +			if (!dominfo.dying)
> >   				continue;
> 
> This is mostly a revert on "tools/xenstore: handle dying domains in live
> update". However, IIRC, this check was necessary to release the connection
> if the domain has died in the middle of Live-Update.

But if the domain has died in the middle of live update
get_domain_info will return false, and thus the code won't get here.

If the domain is in the process of being removed (ie: dominfo.shutdown
being set for example), it would eventually get into dominfo.dying and
thus be removed normally.

I assumed those checks where needed in order to prevent having a
domain without a valid interface on the tracked list while it was on
the process of being removed. With the other changes on this patch
tracking a domain without a valid interface should be fine, and it
will eventually be removed as part of the normal flow.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:01:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:01:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192392.342798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz3v-000179-Pf; Wed, 22 Sep 2021 10:01:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192392.342798; Wed, 22 Sep 2021 10:01:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz3v-000172-MY; Wed, 22 Sep 2021 10:01:11 +0000
Received: by outflank-mailman (input) for mailman id 192392;
 Wed, 22 Sep 2021 10:01:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mSz3u-00016w-Hr
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:01:10 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0229630e-1b8c-11ec-b968-12813bfff9fa;
 Wed, 22 Sep 2021 10:01:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0229630e-1b8c-11ec-b968-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632304869;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=qQRL86pL8BNr0dgDPNorI2W3ms68R2tkI8CVwXInORE=;
  b=KnYQcExC2gOtNCSVt250iqpTUiZO2k3Erk85+zKFs/aasNVBE3xuMx5B
   eDipLyzsAiflYtgCZX2BYdEF1SxvrNRWttmN6h3uRZ+H0RzoRLGfSlonb
   P+ATIJ0t3Ix4wiAvuVpE1iAlxpcBHUNyGi+EwcjchPQoekkroZiQAO5yK
   0=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: s1lB/sIv8FDERjkYLpPxWCfXNtCvSh2t4dPeeb4Bc5lM5m0rnM1q6/26KMnUym0XCQPiKl/h9n
 dbA28Pt7SycQs0f/4f7Dwss4e8A6KpnQxNwVTV3+NVSDUMZPZmOiwiWugPn47w38EWXDuWYuEc
 wXqwkQEN6Ri+Lbkl72dC7ngyYj7REDO+XZXeVBarhisXsoOhg3oKG/QoL6h/xtpHA4R1sL0KzU
 4UCTEQKW38Dnzw8W5J75Y5lpT/yrNyXU56CchyTclGqVabX6hD7WlkqmI328fcuDTDfvp5qkY0
 kKJxpYw7Jnf+2IbHbv4Oh9RX
X-SBRS: 5.1
X-MesageID: 53321579
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ivxB4Kz0bDDOvTzvarN6t+flwSrEfRIJ4+MujC+fZmUNrF6WrkVTy
 WcbDGyPbqyNNGLxfY9xOYTlpEoOuJ+DmNBnGgpvpCAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRj2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt4Bp8
 vl2q5Krc1YwBPCQmdxFdhBGMxgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVort4kI8TxepsWp1lrzC3DDOZgSpfGK0nPzYIDh21p258fdRrYT
 +06bB5GT06eWhNKYwcIJLMmteaiwXaqJlW0r3rK/PFqsgA/1jdZwLXrddbYZNGObcFUhVqD4
 HLL+XzjBRMXP8DZziCKmlqtme3njS79QJgVFrCz6rhtmlL77nMXIA0bUx28u/bRoky0Vs9bK
 kcU0jEztqV0/0uuJvH/Qhm5rXisrhMaHd1KHIUS8xqRw6DZ5wKYAGksTTNbbtEi8sgsSlQXO
 kShxo2zQ2Y16fvMFCzbpuz8QS6O1TY9fXc4SRQaQxA/ssDZ/d0ytkrqR8hDKfvg5jHqIg0c0
 wxmvQBn2e5J1J5Vi/3ilbzUq2nz/cmSF2bZ8i2SBzj8v10jPOZJcqT1sQCz0BpWEGqOorBtV
 lA/ks6C5aglCZiXnURhq81cQen0u55p3NDa6GOD/qXNFRz2oBZPnqgKuVmSwXuF1e5eIlfUj
 Lf741852XOqFCLCgVVLj2eNNijX5fK4SYSNug/ogipmPcEqKV7vENBGTk+MxWH9+HURfVUEE
 c7DK66EVC9CYYw+lWbeb7pNgNcDm3FlrUuOFM+T8vhS+efHDJJjYexeawXmgyFQxP7snTg5B
 P4FZpPWlE0AALOhCsQVmKZKRW03wbEALcmeg+Rcd/KZIxogH2ckCvTLxqgmdZAjlKNQ/tokN
 FnkMqOB4Fag13DBNyuQbXVvNOHmUZpl9CppNi0wJ1e4nXMkZN/3vqsYcpI2e5gh9fBikqEoH
 6VUJZ3YD6QdUCnD9hQccYL58N5oeiO0iF/cJCGiejU+IcJtHlSb5t/+cwLz3yASFS7r59Amq
 rit21qDE5oOTghvFujMb/erww/jtHQRgrsqDUDJPsNSaAPn940zc379ifo+IsctLxTfx2TFi
 1bKUElA/eSU+t076tjEg6yAvryFKeomExoIBXTf4Ja3KTLeojipz7hfXbvaZjvaTm71pvmvP
 L0H0/HmPfQbt19WqI4gQa1zxKcz6taz9b9XygNoQCfCY1ixU+4yJ3CH2Y9Et7FXx68fsgyzA
 xrd9t5fMLSPGcXkDF9Oe1Z1MrXdjakZymvI8PA4AETm/ysmrrOIXHJbMwSIlCEAfqB+N5kow
 Lt5tcMbg+BlZsHG7jpSYvhoylmx
IronPort-HdrOrdr: A9a23:4c+PBqCqQSajIIvlHemo55DYdb4zR+YMi2TDsHoBLiC9E/bo8/
 xG+c5x6faaslossR0b9uxoW5PhfZq/z/BICOAqVN/JMTUO01HIEKhSqafk3j38C2nf24dmpM
 JdmnFFeb7N5I5B/KTH3DU=
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="53321579"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ch1nka1NJZVi9fqPl97WeIYuMxoENPqMSstbfRZq+blIWnT5ehYddE5ZznUZyMsX+uBkyj2pAAniMCf6hTINABeFpwuPsv4DP3UP09Pu0/tMFsoJNYB6tz6eqS/pW9qtZsk0EOTJv3VnqqmStLRX3WDmU5XFmuPJ+d5LGGiCLJNQrJPzh+eJAl6a874IEGayCLxmVJbUDGKjYYqvtdfhsksoAbMsjBW518YyTO8hec5bamxCMrH49BU3r1/w6eDw8d/LnDYJf6beqPCKp5jRHkvEZ9QHNlgWOOrnF8xZI+bJloYeQfNZaP9cNKAvBTpWLFRfFoi7wNXIZXF2mgAFBg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=USClfeoc2YZ7F2ElexwdnWbKJ1U9/jdBu7bSUjnvKEE=;
 b=b0vZfVUXmQsPX2NYRWHwsbnk6JMqm4YvGV6uvKy7eq/650BB+rqBlz7EXA6zxVgC3k3ZecDIWtF6uNTgI4lC3sUpWs8lCi0KuWq3K6HDIDPlfy+x/G52oA8VeKzwXQnR2bDHCOWDTo3NjvWnH6zTKu6QCtQgKftCxIjE1tUYAdx/8188pH5xfRFsf3oHxJ+S2HuB47ZRxUaDhHjobvfhyGKF0/XeycN+uYl0L/nsLG/qvsmHvcuP2LU8ZRUIlcE0Rh32+5QNyjhOQ9rLPmdSQv3pLJ8cMxiWvpVekXHoJj26mF32c74NLQM5RYHe0SAL54MvV7K/pxj/19vFbGfbLQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=USClfeoc2YZ7F2ElexwdnWbKJ1U9/jdBu7bSUjnvKEE=;
 b=RQzpmGsS6yNRhAjR/GwJ/WUsDtkTwYjzw/A+P5G0pTuWMfBhLQpoy+5nJMaErb1NUocZKF4FVU3tzdZK4A2JJwA7T1yNScEYA/v3p5HsT9uA3gd3R9n7bQ+ZXcVCUsfZb4j/huL7VaZyWaNHqWp6EPevt9h+52Of+OsA13SoOCw=
Date: Wed, 22 Sep 2021 12:00:59 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
Message-ID: <YUr+21K8GNqMFjKB@MacBook-Air-de-Roger.local>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
 <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
X-ClientProxiedBy: LO4P123CA0004.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:150::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a675003a-7122-4084-03f9-08d97dafe3fc
X-MS-TrafficTypeDiagnostic: DM6PR03MB5323:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB5323F8AC99F4EE25ABAB70118FA29@DM6PR03MB5323.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: tzAt6BgR/OLu/iXb3uF+rB/oe6gMUrUmJn0JnlF7uFrFq2W97AGgKc08izNFzaXi15PqNNllza9pQAOKZNilPTBzAtNoJ8GQO2vTliCOiovXSflBc0EBCy5ZOAqS2RuTZAcWunlFdrBKrBV/iK4c4aX2ZIfiySgJTs/ZRDngQTIyfGF+n6qwfFPNZGhd6m9YcLPnUoW03MvVQOR1kzHYBtwHI3+JIDNxaxB7XFtWqtVfKkSR4nMkD+PBVxzkfUhbwN73Zvl0uGkJtAfbWrlW850i4wXzoy5hEkJDG1UmiSiqf6GbMzSXtIJIoM39D5VRd8RwQ0otFVMDrxX9MAgOnqmga+DFV8smX3pQ4ZIijHoMXw+uUlZKV4l2XY3O1/FMcMPrFnzpSIGV2Co6euMAZj9myshk9xKgMiI76uq8ilZDR8A0QgszdU8tTIfYuX/52nj9A84izzhADNCG8e51c+a0j6G9yDPXXZOOcppBnM6KWXBl6HQu+RJaA2KqlvGaFeoJ7QmctyW60WrNa583ede2qlKVC2sFPlFUskXdkXh4EMi4u+bpxt3H8D9ngF4L/uJDaU3C4xGZlQ24sSWAZVvHvSJwP7w3uMd9XtD8RLnzrVxpwgfGWLq5Yo20HeCw7scGr+Jr93IB7wbVM9nJRn/d9g2BdBSmP/OlJQQo8w3j1Jb5b+NtszMkXb7dan754OLBHf3054klTCia9YDnejQILdrxCZTyQiN2FTkoIQ4=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(26005)(9686003)(956004)(6496006)(6916009)(38100700002)(83380400001)(8936002)(53546011)(6486002)(66946007)(66476007)(66556008)(186003)(54906003)(316002)(6666004)(5660300002)(85182001)(86362001)(2906002)(4326008)(508600001)(76704002)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VnlXVUtaQkhQMk12ME1tQXFYVHJjZGhsSDNCZW9VNldQWFJBSGUyWDBlVHNR?=
 =?utf-8?B?a3FQVnRrWjQrMXVnbnhuQ08vUmtZUjFmSEI5TlJFRGpaVHJId2p6OHhUaFl3?=
 =?utf-8?B?QzlpOG4raCsxWitERWpBN2w0U1lXK3cxVXBQbEJDblIyTGI1VHptdXc1Z2JV?=
 =?utf-8?B?NHRnaVZ2Vy8rSG4velVscWtGbEJuM3FNQUZZV3hZWUFrTzhrSDJSTkFsUHcw?=
 =?utf-8?B?T0NWM1ZheFlpSWcxeEppVVAvWXNmWU02UVZqR0t4d2dBbFFmOWZEeXZDVDhw?=
 =?utf-8?B?UEh2enB4aTEvZUp5YXlDZlU2clIrT2RwRTNZejBCNWQ3YVJ1TDRuaVlsWkY3?=
 =?utf-8?B?SzRLTTVQMXNkN0tvS3ZtckxQdUJvUllMdTNCTFBHWEg3WktHb1MwZGRRNEVk?=
 =?utf-8?B?TjRqamxDT1BqTitXNDVWUWZZS1ZaaDRUSVYrL2FsazBKeFZESDRheVR4cXoz?=
 =?utf-8?B?dkxyUkt0dmRHWThJTnJGcDB6QWhVSU9ZQjR1S2pYOTRPRmpCTUZqenR1c1Rq?=
 =?utf-8?B?UnRKTHdmczMvdDZtRWZ5SDk5VDVySHJwRHU5MVJYdnlVSkhTQjRlUng5cUtF?=
 =?utf-8?B?WEVSanJKTDhveCtCbmx0ZTRqZ3Z2VnJGOUxUTEVoSnJrV0dyWHJNTmVwSU9Z?=
 =?utf-8?B?Z21TMW1WL1lpaVBPYmczUThEN0I5MzdjVjVucDR6ZGFBaHdjYXViWTMvNlZC?=
 =?utf-8?B?VVp0NURkOFhCRWZqTDhBY0ZuWjBTUFFmVURqRzVtRXdHQWxxZTRaU2pMWGpM?=
 =?utf-8?B?dEpQZGFLTGl5ME5xSUhqNzNVR2lWQlQwK0VIUjhnaER3UWFCR3k1dnd4MTFy?=
 =?utf-8?B?THluWVgxVnRhb2IyZmNCRWNJZDkxNHZ6RTNLaDQrTm9Jei9sYkg3aTgxRTJh?=
 =?utf-8?B?WFBMZzFVTWwrVEJqeEJVN3ZZQVdIWDIyYk41STR0QW1JWjVPZ3VjMFBuNytB?=
 =?utf-8?B?Nk0xdXZaTTdkNmljd21aelJNSW94OGJmdER1R3J2RUVQbGNzVklIK3hQNlRV?=
 =?utf-8?B?OUcxOElwSjFONGxrOVp0TUllNTc3YzhlYmYyL3pNdFRBUWV1dVBVcFBnanBj?=
 =?utf-8?B?dXZLaG10ODk5L2FvYTF4UzRsWGt1T3V0T21YbXpaMHJGQkU1Y2tPUDBuZSt4?=
 =?utf-8?B?aUFNN3hDYU9xMUhNM3MxQkNjMExrTXhMcTVsZ2pzQklyejZINFR5ZytVVGY2?=
 =?utf-8?B?UENjWHZtdDVNSWZaUkpEV2JveUt3ZjNxZk1Ra2dhMW9JajRTMk44UkdldVpC?=
 =?utf-8?B?N1g4bzRIMStOdmt3eXN3U2c0d2RZUjJ4R0hHR0JYV2pQMmszandJeWh5NWJh?=
 =?utf-8?B?MnZlVUFjUFdCQTV5dmdRcld2UEtXd3Z2SG1sQmVhcXR5K1pRVEhXN3NEdlBK?=
 =?utf-8?B?UlhoL2hXQmg4Zjd3RjZXaGVqcTJYanNveW9RWHAwWlNaSWkxL1ZPSzdKU2gw?=
 =?utf-8?B?cXpDL1NUY2MzMDMyY2VXMTkrZ21mUFhIdFVSR25tR1JHMXRxNE8yNTR3cFlG?=
 =?utf-8?B?b3MzUm5lYVhFaGdQRWVvWXVZTHlXMURaTnl1Q2oyNzAwZlA4YjR2b1NDM25t?=
 =?utf-8?B?anhjTCtRVFk2Z2EyU3hOdVR0M0ppT0FuY1Uzd3NyeVc3YmhGcGZyUnBTRmFE?=
 =?utf-8?B?Vkl4K3RyNXoxaGs1bUdQLzdoUWw1UktCNXZXN25YcTBBTWpsT3VpTXk1M212?=
 =?utf-8?B?TmpRTU56TzY4REFBV3lQUDFvWWxGd2FSRE41aFEySDBXWkhWby83Q3ZtUGJk?=
 =?utf-8?Q?+kBimlYF7Fxw4fbu/jt52CmosGcVE46P4WdKMac?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a675003a-7122-4084-03f9-08d97dafe3fc
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:01:05.0086
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tc643bh8A5G79micnvCYALpbFv0NsTKxp8LOKw8G3siOTrC+yQrIAurau+sozv2Ok3PHtlm3iWpkVdDia5uuug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5323
X-OriginatorOrg: citrix.com

On Wed, Sep 22, 2021 at 11:42:30AM +0200, Jan Beulich wrote:
> On 22.09.2021 11:26, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
> >> On 21.09.2021 10:32, Roger Pau Monné wrote:
> >>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
> >>>> On 20.09.2021 12:20, Roger Pau Monné wrote:
> >>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
> >>>>>> --- a/xen/include/asm-arm/grant_table.h
> >>>>>> +++ b/xen/include/asm-arm/grant_table.h
> >>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
> >>>>>
> >>>>> I'm slightly confused by this checks, don't you need to check for
> >>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
> >>>>> guest_physmap_remove_page?
> >>>>
> >>>> Why? It's ogfn which gets passed to the function. And it indeed is the
> >>>> prior GFN's mapping that we want to remove here.
> >>>>
> >>>>> Or assuming that ogfn is not invalid can be used to imply a removal?
> >>>>
> >>>> That implication can be (and on x86 is) used for the incoming argument,
> >>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
> >>>
> >>> I guess I'm confused due to the ogfn checks done on the Arm side that
> >>> are not performed on x86. So on Arm you always need to explicitly
> >>> unhook the previous GFN before attempting to setup a new mapping,
> >>> while on x86 you only need to do this when it's a removal in order to
> >>> clear the entry?
> >>
> >> The difference isn't with guest_physmap_add_entry() (both x86 and
> >> Arm only insert a new mapping there), but with
> >> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
> >> mappings. And gnttab_map_frame() gets called only from there. This
> >> is effectively what the first paragraph of the description is about.
> > 
> > OK, sorry, it wasn't clear to me from the description. Could you
> > explicitly mention in the description that the removal is moved into
> > gnttab_set_frame_gfn on Arm in order to cope with the fact that
> > xenmem_add_to_physmap_one doesn't perform it.
> 
> Well, it's not really "in order to cope" - that's true for the placement
> prior to this change as well, so not a justification for the change.
> Nevertheless I've tried to make this more clear by changing the 1st
> paragraph to:
> 
> "Without holding appropriate locks, attempting to remove a prior mapping
>  of the underlying page is pointless, as the same (or another) mapping
>  could be re-established by a parallel request on another vCPU. Move the
>  code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
>  xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
>  course this new placement doesn't improve things in any way as far as
>  the security of grant status frame mappings goes (see XSA-379). Proper
>  locking would be needed here to allow status frames to be mapped
>  securely."

Thanks, this is indeed much clearer IMO:

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Albeit I still think we need to fix Arm side to do the removal in
xenmem_add_to_physmap_one (or the x86 side to not do it).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:04:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:04:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192401.342809 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz6u-0001pl-CA; Wed, 22 Sep 2021 10:04:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192401.342809; Wed, 22 Sep 2021 10:04:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSz6u-0001pe-9F; Wed, 22 Sep 2021 10:04:16 +0000
Received: by outflank-mailman (input) for mailman id 192401;
 Wed, 22 Sep 2021 10:04:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSz6s-0001pS-VA; Wed, 22 Sep 2021 10:04:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSz6s-0004Lv-Or; Wed, 22 Sep 2021 10:04:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mSz6s-0000Fm-Gy; Wed, 22 Sep 2021 10:04:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mSz6s-0004cv-GP; Wed, 22 Sep 2021 10:04:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+1Jwis8S94NdQzgqHAZwnq3eIS+HG3F8BuGpaX1im5Q=; b=K4N/3yLVQ8KmtOUhWZGvYG6+jT
	TSmDclUlqs+9yximwP/XxV6wFTj3wiwHT4i3eXQk3QwEcMN9png1QGM0cyU7VzjnziEhBrjV4R+h4
	SiEiB+MdigEWas6ersXAQMGsw7L+gczosE93QbhmfdM3OPrVUMxi6OkkI69ZffnCqz3E=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165150-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 165150: all pass - PUSHED
X-Osstest-Versions-This:
    xen=4e467d390eaf6736377766adae0166711866fac3
X-Osstest-Versions-That:
    xen=437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 10:04:14 +0000

flight 165150 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165150/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  4e467d390eaf6736377766adae0166711866fac3
baseline version:
 xen                  437e88417bdea0d5cd012fe9d69a3e0f8a0c13e8

Last test of basis   165120  2021-09-19 09:19:39 Z    3 days
Testing same since   165150  2021-09-22 09:30:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   437e88417b..4e467d390e  4e467d390eaf6736377766adae0166711866fac3 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:14:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:14:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192410.342834 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzGr-0003hj-Ja; Wed, 22 Sep 2021 10:14:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192410.342834; Wed, 22 Sep 2021 10:14:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzGr-0003hc-GJ; Wed, 22 Sep 2021 10:14:33 +0000
Received: by outflank-mailman (input) for mailman id 192410;
 Wed, 22 Sep 2021 10:14:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=X8t8=OM=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mSzGq-0003Qi-TU
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:14:32 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2921a4ad-ee42-4ba1-9b79-4d81487bfd60;
 Wed, 22 Sep 2021 10:14:27 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id u18so9615082lfd.12
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 03:14:27 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id k22sm184211ljm.18.2021.09.22.03.14.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 Sep 2021 03:14:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2921a4ad-ee42-4ba1-9b79-4d81487bfd60
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=S7YpcnCy6xLh/G7NWgBXppr94/jTIZ707mYLR+adzZ8=;
        b=MDb9bj+gV09rJeEKWwfvQ8F7islMHLhb2Zr1D7kZc7sP8kkdgMOggtYc3qWGwo9Eh6
         qMZQLSayzx1bD+4+fPjFwbIF0VJXuP5XqdJ9AkKF8oBameS5fS60pO5B8UFBl/pVsXx+
         CQnuUtkzfFIKGIIBjJZ1MeJ4XZKb0EeKi7HB4KP4vne0AQT8UYaKAzLpDEji52Io5mEr
         64P++MrKK7N0AFqKV8Y7yLVE6UVOqovI+eB+uiwXQLCUaB+E6OMDwNAe8e4mUKHfc/xj
         VbiULS1FPqJN9NzPVdXvE9hwEw1aPWK0qQ8WXnJh+hlgGaXHy5xzf0b41YGWOnuYDMtZ
         N2rw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=S7YpcnCy6xLh/G7NWgBXppr94/jTIZ707mYLR+adzZ8=;
        b=FBpZPRE8TyKhrfuWQjb/SX8fD3Prn1e9uAcHIWZhVLEJhyd+dNZsr11WHoYcQJ5o2i
         s5KlLzHbuwhIksJnlCyKrKYiIvkyKut2F4gc3EbxxfyRIeMY/uRmdUR55g7T7MdTQGMN
         HFwwHbVMVKGyEfaCqcyoPAABOjgogrmD7rNZYT+7v7bLLkDTW1BeowrgRnV3d5cykchx
         HwHsv29MjeWe/l4yAqrXCLi5ytsPS54z4NJSZERTY93MNnJrz8fWKUKNVpRsdMA3vXP4
         u4c4yD8TQk5kysQp15fY6N+7Q7KDBNjvEdmQ7xKUIBMS1Gj3q6Drh14y5whe2uinmgbc
         cg7A==
X-Gm-Message-State: AOAM532TO2WFGh5WLWNgJhXj48SF1Q5XPxgzTLzdT6a56YF2WSaE/m8p
	rIl9wzU3XBvnV6PDaJXuKSQjyshUVrPZQA==
X-Google-Smtp-Source: ABdhPJzKS19pslQy4wS9JYyX5fy6N2+1K+GIKEJvBUgsxXAmj5HYIIECMdVwL2DivXpc15UuvD9uAA==
X-Received: by 2002:a05:6512:3ee:: with SMTP id n14mr27553556lfq.589.1632305666273;
        Wed, 22 Sep 2021 03:14:26 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
Date: Wed, 22 Sep 2021 13:14:22 +0300
Message-Id: <20210922101422.2319240-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210922101422.2319240-1-andr2000@gmail.com>
References: <20210922101422.2319240-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Currently PCI backend implements multiple functionalities at a time.
To name a few:
1. it is used as a database for assignable PCI devices, e.g. xl
   pci-assignable-{add|remove|list} manipulates that list. So, whenever
   the toolstack needs to know which PCI devices can be passed through
   it reads that from the relevant sysfs entries of the pciback.
2. it is used to hold the unbound PCI devices list, e.g. when passing
   through a PCI device it needs to be unbound from the relevant device
   driver and bound to pciback (strictly speaking it is not required
   that the device is bound to pciback, but pciback is again used as a
   database of the passed through PCI devices, so we can re-bind the
   devices back to their original drivers when guest domain shuts down)
3. device reset for the devices being passed through
4. para-virtualized use-cases support

The para-virtualized part of the driver is not always needed as some
architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
model for PCI device passthrough. For such use-cases make the very
first step in splitting the xen-pciback driver into two parts: extended
PCI stub and PCI PV backend drivers. At the moment x86 platform will
continue using CONFIG_XEN_PCIDEV_BACKEND for the fully featured backend
driver and new platforms may build a driver with limited functionality
(no PV) by enabling CONFIG_XEN_PCIDEV_STUB.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
New in v2
---
 drivers/xen/Kconfig               | 26 +++++++++++++++++++++++++-
 drivers/xen/Makefile              |  2 +-
 drivers/xen/xen-pciback/Makefile  |  1 +
 drivers/xen/xen-pciback/pciback.h |  5 +++++
 drivers/xen/xen-pciback/xenbus.c  |  6 +++++-
 5 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 057ddf61ef61..6e92c6be19f1 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -180,10 +180,34 @@ config SWIOTLB_XEN
 	select DMA_OPS
 	select SWIOTLB
 
+config XEN_PCI_STUB
+	bool
+
+config XEN_PCIDEV_STUB
+	tristate "Xen PCI-device stub driver"
+	depends on PCI && !X86 && XEN
+	depends on XEN_BACKEND
+	select XEN_PCI_STUB
+	default m
+	help
+	  The PCI device stub driver provides limited version of the PCI
+	  device backend driver without para-virtualized support for guests.
+	  If you select this to be a module, you will need to make sure no
+	  other driver has bound to the device(s) you want to make visible to
+	  other guests.
+
+	  The "hide" parameter (only applicable if backend driver is compiled
+	  into the kernel) allows you to bind the PCI devices to this module
+	  from the default device drivers. The argument is the list of PCI BDFs:
+	  xen-pciback.hide=(03:00.0)(04:00.0)
+
+	  If in doubt, say m.
+
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
-	depends on PCI && XEN
+	depends on PCI && X86 && XEN
 	depends on XEN_BACKEND
+	select XEN_PCI_STUB
 	default m
 	help
 	  The PCI device backend driver allows the kernel to export arbitrary
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3434593455b2..5aae66e638a7 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
 obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
 obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
 obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
-obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
+obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
 obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
index e8d981d43235..e2cb376444a6 100644
--- a/drivers/xen/xen-pciback/Makefile
+++ b/drivers/xen/xen-pciback/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
+obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
 
 xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
 xen-pciback-y += conf_space.o conf_space_header.o \
diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
index 95e28ee48d52..9a64196e831d 100644
--- a/drivers/xen/xen-pciback/pciback.h
+++ b/drivers/xen/xen-pciback/pciback.h
@@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
 				    struct pci_dev *dev);
 void pcistub_put_pci_dev(struct pci_dev *dev);
 
+static inline bool xen_pcibk_pv_support(void)
+{
+	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
+}
+
 /* Ensure a device is turned off or reset */
 void xen_pcibk_reset_device(struct pci_dev *pdev);
 
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index da34ce85dc88..bde63ef677b8 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
 
 int __init xen_pcibk_xenbus_register(void)
 {
+	if (!xen_pcibk_pv_support())
+		return 0;
+
 	xen_pcibk_backend = &xen_pcibk_vpci_backend;
 	if (passthrough)
 		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
@@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
 
 void __exit xen_pcibk_xenbus_unregister(void)
 {
-	xenbus_unregister_driver(&xen_pcibk_driver);
+	if (xen_pcibk_pv_support())
+		xenbus_unregister_driver(&xen_pcibk_driver);
 }
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:14:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:14:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192409.342824 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzGn-0003Qv-DV; Wed, 22 Sep 2021 10:14:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192409.342824; Wed, 22 Sep 2021 10:14:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzGn-0003Qo-8G; Wed, 22 Sep 2021 10:14:29 +0000
Received: by outflank-mailman (input) for mailman id 192409;
 Wed, 22 Sep 2021 10:14:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=X8t8=OM=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mSzGl-0003Qi-VW
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:14:28 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 81ceafab-3834-45af-a5ff-6eee1c3e1f38;
 Wed, 22 Sep 2021 10:14:26 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id e15so9685285lfr.10
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 03:14:26 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id k22sm184211ljm.18.2021.09.22.03.14.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 Sep 2021 03:14:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81ceafab-3834-45af-a5ff-6eee1c3e1f38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=SvLayk7gLTLICLvUIqlFHN+Qd7wHHCUL73ogGPkKfx8=;
        b=dkrfcJFRtgM58BtsNkps8s/wg+2uBksUXZT7yMoNtS2Z7ULeyfkZ4w5Z4DEGDS8CqM
         qZDNTlKTqTfMUi3sz+S35Y6/nsbXQpGVWPkaBrrRvQ0BfCo1P0Iuw2KMLyiFQ5W7UQxi
         GLVSWFruCbLxZAWoD8Wg5K+pRM77JlbiSUuTjDIHpfVMOF0wfuhB5eYj9Uux96e0rQPe
         HhDSHZWqS1zOkWvLhvCAohmYlNGNI3roGCzGQ2ly8v5M1gSGPCbqnVSipAc6DXx6UdzN
         CfVzmKn654p7y4ejqoY6hn45KENIFY2FEa9usJqX2CCk7BLryqJgzPj3x8u4mw0glxAp
         pXnA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=SvLayk7gLTLICLvUIqlFHN+Qd7wHHCUL73ogGPkKfx8=;
        b=PXsDlah+msW6MuDytbAGLrTBf4bedjMUaCy0XyywTmxyS5Dr2H0cGU6gOas8pj2kRe
         ZF3CDzC8I9LtzZQUnsn72hYnc3blon7plWnpJVTUAVOhVasGOwicJ+YKjgGwANuSoG4O
         ebAQ/xMaNzFhdtUMx5whPMNMcWVAAPdRZdtsfOpCZNNeAjr563DnGoB+U257fijA/xtl
         aB2VKUVk61UOEwVneDVFRQMgMB4pzzPeVIyztm7qwkmv0ymCWKrhNpv0gyVUgDPh5GoT
         AuxyF0Y2OyiTVRVdLPVLvXsbrHzslyCi5g+MSxfBuX3lmTO0wGqlMESE7Ijx9Ple6STn
         BzBQ==
X-Gm-Message-State: AOAM530LrHWN9OENu5n50JD60qrnJCFecvk40NSloN4pr+UKN52cyivx
	4Xr6ZoHK+iTCdAgUpKAYwUV6oxlj270nUw==
X-Google-Smtp-Source: ABdhPJzEPEcaoU33JxJRvmxxIKFMw67F936g3tk7uJedNHMtFok0NvPUOOD9BNrp97mm81Fd5UBtoQ==
X-Received: by 2002:a2e:545a:: with SMTP id y26mr28468856ljd.225.1632305665218;
        Wed, 22 Sep 2021 03:14:25 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: [PATCH 1/2] xen-pciback: allow compiling on other archs than x86
Date: Wed, 22 Sep 2021 13:14:21 +0300
Message-Id: <20210922101422.2319240-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen-pciback driver was designed to be built for x86 only. But it
can also be used by other architectures, e.g. Arm.
Re-structure the driver in a way that it can be built for other
platforms as well.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

---
Since v1:
 - Do not move pci_xen_initial_domain as it is x86 specific
---
 arch/x86/include/asm/xen/pci.h     | 18 +------
 arch/x86/pci/xen.c                 | 74 +----------------------------
 drivers/xen/Kconfig                |  2 +-
 drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c |  3 +-
 drivers/xen/xen-pciback/xenbus.c   |  2 +-
 include/xen/pci.h                  | 28 +++++++++++
 7 files changed, 109 insertions(+), 93 deletions(-)
 create mode 100644 include/xen/pci.h

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 3506d8c598c1..2889f091f459 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -16,26 +16,10 @@ static inline int pci_xen_hvm_init(void)
 #endif
 #if defined(CONFIG_XEN_DOM0)
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
-	return -1;
-}
-static inline int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-static inline int xen_register_device_domain_owner(struct pci_dev *dev,
-						   uint16_t domain)
-{
-	return -1;
-}
-static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
+       return -1;
 }
 #endif
 
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 3d41a09c2c14..4a45b0bf9ae4 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -23,6 +23,7 @@
 
 #include <xen/features.h>
 #include <xen/events.h>
+#include <xen/pci.h>
 #include <asm/xen/pci.h>
 #include <asm/xen/cpuid.h>
 #include <asm/apic.h>
@@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
-
-struct xen_device_domain_owner {
-	domid_t domain;
-	struct pci_dev *dev;
-	struct list_head list;
-};
-
-static DEFINE_SPINLOCK(dev_domain_list_spinlock);
-static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
-
-static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	list_for_each_entry(owner, &dev_domain_list, list) {
-		if (owner->dev == dev)
-			return owner;
-	}
-	return NULL;
-}
-
-int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-	int domain = -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (owner)
-		domain = owner->domain;
-	spin_unlock(&dev_domain_list_spinlock);
-	return domain;
-}
-EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
-
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
-{
-	struct xen_device_domain_owner *owner;
-
-	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
-	if (!owner)
-		return -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	if (find_device(dev)) {
-		spin_unlock(&dev_domain_list_spinlock);
-		kfree(owner);
-		return -EEXIST;
-	}
-	owner->domain = domain;
-	owner->dev = dev;
-	list_add_tail(&owner->list, &dev_domain_list);
-	spin_unlock(&dev_domain_list_spinlock);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
-
-int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (!owner) {
-		spin_unlock(&dev_domain_list_spinlock);
-		return -ENODEV;
-	}
-	list_del(&owner->list);
-	spin_unlock(&dev_domain_list_spinlock);
-	kfree(owner);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
 #endif
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index a37eb52fb401..057ddf61ef61 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -182,7 +182,7 @@ config SWIOTLB_XEN
 
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
-	depends on PCI && X86 && XEN
+	depends on PCI && XEN
 	depends on XEN_BACKEND
 	default m
 	help
diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
index 224df03ce42e..fc8c1249d49f 100644
--- a/drivers/xen/pci.c
+++ b/drivers/xen/pci.c
@@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_XEN_DOM0
+struct xen_device_domain_owner {
+	domid_t domain;
+	struct pci_dev *dev;
+	struct list_head list;
+};
+
+static DEFINE_SPINLOCK(dev_domain_list_spinlock);
+static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
+
+static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	list_for_each_entry(owner, &dev_domain_list, list) {
+		if (owner->dev == dev)
+			return owner;
+	}
+	return NULL;
+}
+
+int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+	int domain = -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (owner)
+		domain = owner->domain;
+	spin_unlock(&dev_domain_list_spinlock);
+	return domain;
+}
+EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
+
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
+{
+	struct xen_device_domain_owner *owner;
+
+	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
+	if (!owner)
+		return -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	if (find_device(dev)) {
+		spin_unlock(&dev_domain_list_spinlock);
+		kfree(owner);
+		return -EEXIST;
+	}
+	owner->domain = domain;
+	owner->dev = dev;
+	list_add_tail(&owner->list, &dev_domain_list);
+	spin_unlock(&dev_domain_list_spinlock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
+
+int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (!owner) {
+		spin_unlock(&dev_domain_list_spinlock);
+		return -ENODEV;
+	}
+	list_del(&owner->list);
+	spin_unlock(&dev_domain_list_spinlock);
+	kfree(owner);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
+#endif
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index f8e4faa96ad6..bba527620507 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -19,7 +19,8 @@
 #include <linux/sched.h>
 #include <linux/atomic.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
+#include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index c09c7ebd6968..da34ce85dc88 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -14,7 +14,7 @@
 #include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
diff --git a/include/xen/pci.h b/include/xen/pci.h
new file mode 100644
index 000000000000..b8337cf85fd1
--- /dev/null
+++ b/include/xen/pci.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __XEN_PCI_H__
+#define __XEN_PCI_H__
+
+#if defined(CONFIG_XEN_DOM0)
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
+static inline int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+
+static inline int xen_register_device_domain_owner(struct pci_dev *dev,
+						   uint16_t domain)
+{
+	return -1;
+}
+
+static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+#endif
+
+#endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:15:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:15:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192420.342846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzHQ-0004VK-TW; Wed, 22 Sep 2021 10:15:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192420.342846; Wed, 22 Sep 2021 10:15:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzHQ-0004VD-Q4; Wed, 22 Sep 2021 10:15:08 +0000
Received: by outflank-mailman (input) for mailman id 192420;
 Wed, 22 Sep 2021 10:15:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSzHP-0003Qi-UA
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:15:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b658017-5ed3-49b3-9ed6-71c69138d307;
 Wed, 22 Sep 2021 10:14:54 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-QG0FgcmDMgu6Vkw0smaTbw-2; Wed, 22 Sep 2021 12:14:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4189.eurprd04.prod.outlook.com (2603:10a6:803:3d::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 10:14:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 10:14:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0093.eurprd02.prod.outlook.com (2603:10a6:208:154::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17 via Frontend
 Transport; Wed, 22 Sep 2021 10:14:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b658017-5ed3-49b3-9ed6-71c69138d307
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632305693;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=xeX058YAZuLlrCWZaRoSdAGTp0Uj3/VBmybKo4LJZZE=;
	b=I2TmtXtDBGokbOA3Nu7o1bZKTJSkxWFu2SXPtucSAcxEjB+X7iYRcsjnnqro626ub7LZOp
	jVObcU6Q+TQxAxsdgvBnGcFJQAwQcC/WDf0ffPMV4p5SzvsZ2w0r57hvLUGf/2uoeHGRHK
	NEyxT5kzY5NwrdoV0EmATCS4AvAYXlQ=
X-MC-Unique: QG0FgcmDMgu6Vkw0smaTbw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W6B+IyvVIKuhEo6kGSOyJpN2DaQVNfEyp1fi8Ef/L6wgywwZox7xHU2nSUGYDYpvNNWDjqD1qQnaH0FZ+EHhbhrsD7yFHaJsPWf9LVZrSQ885iVQi4dH0f6KwXOi/ZpthaSjVVPwtWkEsWILCpsf0VE7aW5OUaGKIp3gdxfkLhA5vSGBqFe50UCu3hYi/sVayl+5wW5J9k/yrp/9sHBgbJki1JsiHwqffqT/zeJjJmGxQ5GqMt5hjip5nAZMPYiL9tRq4wz32aTKmNcnJzGIH1OK7AEjwP0u+d/AQ8yGXy+LkbmaCbgg9ig5DYT5qql9ciVN29ODANOdkub5t76BeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xeX058YAZuLlrCWZaRoSdAGTp0Uj3/VBmybKo4LJZZE=;
 b=cureU2AMal3wkR4YSvKLHktYotH8m0+cCrWWSyYnAIGTKr9pQ238i0iLKX6aKkwvS4QAl3BnzMxPHAmVBtFmbNh1vWB3NAxTH6K5fimBZ7BpzNIormzgYlXFarS1BnGscw0lDfB5z/wOmYiJ7JLELIf7bsBMIlwO8N6tiONlWr+/4f7mI/+2k4LZeDf++LyhBLNZdMaHbbP2cdW3r+EGup2/lc6mOIyTg06cDlds/35gi7zkW8qaIALwRA3/7H4kukzqwTL8eEiW4qpJ+XCJdAyb3evSAWXHotnl2DGMNjkgv+++RNRyB8Ec/vDciUiZyipuNIwb9EOuR3FCLEZvyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/3] xen/privcmd: misc corrections
Message-ID: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Date: Wed, 22 Sep 2021 12:14:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0093.eurprd02.prod.outlook.com
 (2603:10a6:208:154::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c3ea3b95-1d9e-4b71-559b-08d97db1ce22
X-MS-TrafficTypeDiagnostic: VI1PR04MB4189:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41894356570686D56128829EB3A29@VI1PR04MB4189.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MhtbFWLhrDZmYZwL5r/W7e+thwRoWaWtatT/JHViMFh2z70yJ2DZGHESjQ75WdGiBTjeCY5ZPuzQD1P05tfHzjG83JsmZ4UE5R/29LSgM6u7Az+3HwEcye5wphzDkITxVpexvMb78hG7n8fHVl0fL7RXvTQrd7D4EAWa54W+LZomQjKAQQC6vhBD/HS+7sy+gp32d5/LbdMhudI5P1H+dAegk2PseW2Qc4DdPJ8y6pY3AuCApHRGHQLLIhw1x8uQ8xQvZn+fO60pA5pBBmnEFFJ/LMUAXPcc36JM6wV9Q/gZG+jZ1vYTK5y/z61Awpgl0XxAM1+Ve8xu573K1GpY9/swhYk3vYkkmfNY0nVoWacUbXC78L1dlwSraBf6T5X+2gt/jvvs9AdjWOICc1gBl7x1ObOUCRfb7lezclTAy33UhfwoOBGF9QH8eAGtjNFF259zEL1DdW2mtI4wZ70+NPDs/BenbGZQgWvJO4dfu2NzImMUOr3OhDwANc/G3niVmnt4QuYCJfq2sJT7JlBlO1iIZfVZAN0D/b6m52wb4cm7Yz5PBCXDkmje09anaFsyR5N3e2VqgNCUtQ+J6G+LBZ/JN3rs95hMoNTdKguvZe8JLc5FGgIqS3hWugc+ocYmz4ZH7syDyUEGQJwbI0PnSc8TaisBe66k6ukakpua0zfH4P0XIQYP0znuFWJQSFG+qLBdjNGY6FmyOzKOP/KR6MjpY4es6aDXgRJnGM9jOm0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(16576012)(31686004)(316002)(54906003)(36756003)(8676002)(2906002)(186003)(6486002)(8936002)(508600001)(31696002)(110136005)(26005)(38100700002)(4744005)(2616005)(956004)(5660300002)(4326008)(66946007)(66556008)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eFQ4dzlRQ21FdmNaYVVjUEFXbnFhY0FXSjFzclRacXdoTE5hQm9PQUg1aWVv?=
 =?utf-8?B?bVoraVVLWi9hQk1Zb3VtcGVyYlZMKy9vVGJ6S0Q0OTNNZjVxMEcvTDV6S09X?=
 =?utf-8?B?NzZrSjRQWTVIUkh3RGJ0dFFhdlZBSHdpYU5rVml0dGlEbVlsSy9ncjM0VkZx?=
 =?utf-8?B?ZFpDVjFtbTkvMElZMXdxdmh1Y2FDcFhSMVJQVXJHTjZvemo4NC9VSkNoYkJY?=
 =?utf-8?B?VklveENqRW5oVVlUbS9ZdHg1Z0NRVG42eTd3SWo3a2VLNHR4VzNDd1NnMHhG?=
 =?utf-8?B?V25pTTFPZjE5c2xaRkRMRW11MmgvWEVFb0J6ODV3NTJKdHBnY1VNeTZpNlFj?=
 =?utf-8?B?YmZ6djllaDAybk96TUtZS3JoNk93YTAzU1dwb25mM3haUFFVWGpHRTFaV2F3?=
 =?utf-8?B?QnFiVGVjdzJaYkswNzRSZFlMWGZMclF6Y3UxU2ZETzhXZkM4KzJzWXdTem9D?=
 =?utf-8?B?dGtKV3J3QVhIUWFpdmlkazNSMDNJUU1YOXRocUlkdFhTL25aY1ExYURPTkU1?=
 =?utf-8?B?b3IwbU1GeU1IREZUYy9Ic3JSVXNPRG10OHhjbTQ4QnJKS3U0N0pLM1lmbld0?=
 =?utf-8?B?aWNNbzVhKy9SdmpmOHZOZmhNdGVoVDE1ei95RTkyQkZaTDYvMHo1R0ZRdVk3?=
 =?utf-8?B?M0tQSEJnNEpIMmxvZlZXS1N5dll0UzE3SnpVSkUrcXhZeWpDZnhqR2lYTndD?=
 =?utf-8?B?ekFNSWhUUVhRRVZEQ2YrU2o4TldhUmhoQ2xCWjBRVFZIMGNyV3pQQmlHNUZK?=
 =?utf-8?B?TGlUYTlPUG9YYk0zN0tsMjZKYkJwTnp1K1oycW5iNWczaTIxZUs5NE52QTdU?=
 =?utf-8?B?WlFWcWVyZjVhbFAwTG13OXRvR3RpQjlBTXdqOHprUXNpQlZiM3lzcnQydk14?=
 =?utf-8?B?UG13aHA1Q2lVZ29zcHhSM1Z1bzJJUWp4YnZyRXBQd2VIMDNGMHBYM2FqbHh6?=
 =?utf-8?B?Q0c3QncydWpqWkUwejRuQzRDR2kzWHZINU1LMXduTkFDby9IQ0xtczFrMDhi?=
 =?utf-8?B?dzlOa25NNkhzOG1LSFBKWFZ5bk1YL1lyYUdVaU5XZGZ2RmRXajFLNXBiTlN4?=
 =?utf-8?B?Q1I0TUZBWGd4akFMQ3NTSnJhd3pTR0xWSUw5TCtoWkNmYUxWOFlVOGpPTlZD?=
 =?utf-8?B?VFhhS2NKSHlJVGh2djhCbnR5eVhyMmVEZ2RIUFhWZmxOZTdVYm1ncjhMejdS?=
 =?utf-8?B?WWZUNkx2cE9LdDNieloyeHR2dFpiaHF3UnQ5emhYdStmaVFnQ3BkanF2RVkv?=
 =?utf-8?B?ZFhsK3ZkUlIra2FqOVYxZ2lIWlJldWl1ekJMRUFkZUVBZ3VqdnhrYjE5RWpN?=
 =?utf-8?B?UHE4L1JkdWhTcXZGZG1qdHNGSGk1Z2d0U1JTWmFTUWUzaVdpOXBaQktCQ0xT?=
 =?utf-8?B?bW5lOTN6azNiSm8vVzE3RHdhbU9wSHNZckVDcUs5cnZIK0diUUR2K2Zpb3Qx?=
 =?utf-8?B?WFUxRjgrbUZpUTRBb3B3VE8veGVlbTJNSmxsNS90NnF2MmtpYmYzSlNyMmtJ?=
 =?utf-8?B?RTFtYlY3Y3ZnbXVMR3daK2c1TDhhWWVUZkd3V3lrV09LbHdweHJJT1psT1I0?=
 =?utf-8?B?M1hEbEhVYS9qVm1OUk9iUnJYL09RR3VUUjh0eiszeXJ0bkdrUk9iQmZsZTh5?=
 =?utf-8?B?TmllZlZQaWpoeU1WOW9UcU9teXJHK2UyOUlwRHlkMFpZVm9FdVNkK1hZQ3Fm?=
 =?utf-8?B?Z29LdDJhV2p1V0poQlQzdzlJSzIwYWN6K0VOQy9EOVBwVmtQWElmN3kzZFRX?=
 =?utf-8?Q?kYAiwN300xFOv1D8hpnDfSnTY9SZo4GHpRp9xla?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c3ea3b95-1d9e-4b71-559b-08d97db1ce22
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:14:47.1695
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LqgbeM9laLAwVbi9EzYOovfFCFxaCQcSM+Ksj0jrBS6ReAwx3OA3VoedFXlvUsad1T3xH8SXAsLtHhDdcBv5ug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4189

The three changes here are largely independent, except for a contextual
dependency between 2 and 3. Note that patch 1 will need actually testing,
on Arm.

1: replace kcalloc() by kvcalloc() when allocating empty pages
2: fix error handling in mmap-resource processing
3: drop "pages" parameter from xen_remap_pfn()

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:16:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:16:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192428.342856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzIw-0005HY-CQ; Wed, 22 Sep 2021 10:16:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192428.342856; Wed, 22 Sep 2021 10:16:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzIw-0005HR-9a; Wed, 22 Sep 2021 10:16:42 +0000
Received: by outflank-mailman (input) for mailman id 192428;
 Wed, 22 Sep 2021 10:16:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSzIu-0005HJ-SG
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:16:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2c7bc96a-1b8e-11ec-b96a-12813bfff9fa;
 Wed, 22 Sep 2021 10:16:39 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-1rM-3_mIPnu2w1oLQjm4IQ-1; Wed, 22 Sep 2021 12:16:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Wed, 22 Sep
 2021 10:16:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 10:16:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 10:16:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c7bc96a-1b8e-11ec-b96a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632305798;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ihy5AqkbKQlh/sKYZgTSszLHYrjLcDDqhUqK4o2XEtI=;
	b=K/b87FAGIGJ2izaj7QLjGENwt+VWzCnazT4oFrNZC0c0JOh9V64eMOEOrmUiOV7NyFj9p4
	7gWw5+/fAi/u4YFKmjePGzUkmd9+jcERcpTIU5ywEbS2/cMoEYkr79Fb5AirAcAUhHgzS4
	nTqVqASwh4xOoU01mw/KvQiSMbSdceY=
X-MC-Unique: 1rM-3_mIPnu2w1oLQjm4IQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=L5VQZxg717y8Nyolf0kqnaz7jbvhKbq+SsbF4PQMpjjxSb+wtykIjxpYc8yYcvO5oUR/uh/PNzQDGMiGI3COZaqYcpSgngYK7xe2FK7ujqq41hbfaHfzM0SFdT71ZLlssFWL70rdeUhA+hx9In3JVPbicusdksQV5jC/rOAdNHIO8futks2m6RQPz/JQ90HLr1PDBqZB6QMr3dDoXqF3opWPZUFVXsls/Oy9HxxSDer6ecT5hqwCENsGYpTRNoDadTeu0LGwDmMcKKZ4Qw55qaQRg1AvtpqxMQ4Idyxj1Qy+1zZ/vYCZXC69BpOweTvoDLorHrkQIAhxbZ/3J0Uk4w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ihy5AqkbKQlh/sKYZgTSszLHYrjLcDDqhUqK4o2XEtI=;
 b=fKw+T/6jhwEJrkEv7XmjK5n/vgkkuOFVrw9yDrJ/4vDywMqCizGc8v8w6WPbkZu1AzvQ3ekVFCCqqZsqBwR+8hx8ZvgjDER72MLaPXJW0eR/ludwFpBpsndtZgWoGhI4iNIHUZvn5HZLTjqd3ymBKUm9yMGXjHwilmscaVIfbxNKyPqvaqhzM3ruT2Edt/r3Hvl1fM0BPLRAODS3yrDea6Vi1TUPOGfO+BvJGMAG1lL/pNGfTyZ4LKFAZNTk00UAm3Sz5vkcYSZB4SqGsJDRzyvfQKRM4N7+CrrF+Ogk17W0mZI5rPajDehNlwUdHhRAEnlSv+qGQskfoz6X8Ivasg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH RFC 1/3] xen/privcmd: replace kcalloc() by kvcalloc() when
 allocating empty pages
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Message-ID: <6d698901-98a4-05be-c421-bcd0713f5335@suse.com>
Date: Wed, 22 Sep 2021 12:16:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0023.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4514c639-5875-4107-bfd0-08d97db20ef4
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41908127AA8CCD2E54806B13B3A29@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	jLzYEODIBUA1//UERQZqtoL1gxZrcJRrDmhsdABv6FVgoEB54z1CCXGqfls2JAMPKnFbwp1FY8QyBtFkmLAn2qvjtp07W0MjbATD0g2y2/UTC/YWWVGgPe79eFHJ8E1Ofl+tnY5LrQ1h5EJbbLhbUnMkVspsQay76Gb8BW/9YRCcozAzvzhaZzGW5sjs6oV8VnlqNcpFidrr7WhyGxA0+qDhHWFaJGhTd7psp9Z/SCMqKH/hMEO9k5QWiWdUPzEYneFsS4E8Xa6lvAdgSxrn2M7rEGSf29Ewd344mNQkBCnpSHG0IaGVNc+lM/tNBH2kEa56VwlNZhnpKXfYZMpKt5MW9CZWCA9evD9xLONioBb3bx5ILK4lAGTLdsuAKIMRKqg8plME/2PcqpIlNpjbvkzg/Q8YtBa+bLEL8u2ONHxXkeivFpg/8o451Q8cEXsL1ZCFPA3blTULInsJHtEcPX9qdH5C5oJHSFf6aTOPmDl05J6ldcSMZHd5gzCI4zRbv1jh4paDVB806DKqEo3BdSY0tfTCC5xIbe+q2IPTIfOXiaAUQw6q5MmBh+yZJxPSWWgoZL8omD1x+FSfxBgzyX82K6XPHDyy+1yQMn5/IJl7ymF6f7FSU0BkDWDs+xO0AoDv9e6RrKOCHI9DFMU6XElBst6Ji59cQEzjXTzlsYG6SyO2VS7nNElpfPbPD52cSzfdtuuuQywdKzt3/RHdb76Vok0f1jPrq1xBbqY/k28=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(316002)(26005)(8936002)(110136005)(31696002)(8676002)(6486002)(16576012)(186003)(83380400001)(508600001)(5660300002)(54906003)(31686004)(66476007)(36756003)(38100700002)(66946007)(86362001)(66556008)(2906002)(956004)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?blZTaFFseTV2endMd2s0VisrV2dYckpXb1hDWUk5VmVJNjVFQjZLeFl1L1VG?=
 =?utf-8?B?c2FUbHk4c3U3cThTNmZObVhEcXZzSkFDRjNQVFBZNnlYUTV1R21HbFMzekds?=
 =?utf-8?B?MlROWkJsVitBZ0RrQ2cwZm5pSFN0bmNyRnJMWkt3bjV5blZ1dURmV1VZWTlj?=
 =?utf-8?B?QnAzakNqOWNaTmRHSFFuV0YzU285Smg3aDd6OXkvSVV6S1BxWnpmK21iOUtJ?=
 =?utf-8?B?clA2L0N0bUNoU3NFM1N4OXlRR01RTXQ0TzFKYmhNSjFKYklCc2Fjd1RtRnhG?=
 =?utf-8?B?bmtCR0NleXVLanRRQmxWQUl4VzhmazNSR3k4czcrTGlKMmZDVmF3ZS9oMGlX?=
 =?utf-8?B?bjlIczNwaGhFTndkMzB0WERFaGFIeWJXSGJpaGIvcFVmRXA1eTlyL1VDVElZ?=
 =?utf-8?B?S1pobUc4QTkvcitWTGRDODNMQy95L2s5NXhWQzN3QXN5SkNQMzNSUEtkeEFC?=
 =?utf-8?B?WSthZXVqb1dOZlhzS3I0R0t5aDdTQVpsMGFxOG1YK3BoMXZkLzJiZU44Uk9t?=
 =?utf-8?B?SVU0aGRLbWUxbW4zbnZtaTVMQVFiRDJrWm9TR3l5ZEtvSng2WWs5Yng4WFRv?=
 =?utf-8?B?S3hHL3R6T2tsdXp6Qzhaa3BvVGZsNThrWldwTGpvZTlVUHozSFZqY3NsZXRh?=
 =?utf-8?B?di9jSi9ZZmNPdFcrMWJiU1dodjdUUkpvQ3hob3NCUVR6di80a2hFOVltRHZ1?=
 =?utf-8?B?bGJDWlBUNkpOQitHdTN2cWdJcUI1dHlBUUhFNlNteldtWFhPN05ubEEyK3Yr?=
 =?utf-8?B?K1M2U0NvQTVjajhFTTdwbmkrbnFrR0szZk1haGcwS3FFekVCT1ZWN0FNSlFu?=
 =?utf-8?B?dlJ1ZGRKdGFkbmN0TEsxQUx2NGtMWW83Y0VaTmF0QjhRNUZSRk1EdUNVV2hx?=
 =?utf-8?B?cnlzZmtnV3JsK0N0dHNzQXZ4OTNGS1RiYjhHbzZyQ3JDcElldi9HTXAwVFF2?=
 =?utf-8?B?VEZoTFYxTEM1LytMZkZNdnpqNjU5WVVJeWx1N2w0alk3QVRDMlp0ZUNYVmRk?=
 =?utf-8?B?TDd5c1NWekJOdGhCVDUrNHVkV21iVkRMaktKVlYvb0pnbkx4enR1ckorU3hw?=
 =?utf-8?B?OGlTQ0lkdVExZWdDWFRrYVpndFNBcnBJb01LT1hpY0JjaXZwbmxDRmFkc011?=
 =?utf-8?B?V3FxL0ptcUlRYkE4aUhpSFh3c0NzR25TZFBrQVdsWFFkR1J1TnlGUVcyL2lC?=
 =?utf-8?B?UnhJSWV2UE5ENVB2Vy9LZXNTNlJQMGtUcXF4bWpCNzhGL0QzRXZ4UFlBcUN3?=
 =?utf-8?B?MFZaZ3JhcG5qSHlHTm5zb25CSjdBdG9JMWcvbHNVNEMrQTdyVEJQalZxTHBL?=
 =?utf-8?B?ZTZNVnVzV1pXMVpMVEhuZ2pXazQydUZNcGFOZ3FNMGtDSkFjTnZSVjE5ak1N?=
 =?utf-8?B?U09GRW5jaTNOL1BwejN6dzF2Uy8yeGlvMEtHUDFzM3dhWFNMeEpkRnZWakVX?=
 =?utf-8?B?UjcwUVVXa3lQb1ZjSEl5RnViSE5oL25TN3I4ejBvQXB3bHdlbXRQdGVYb000?=
 =?utf-8?B?ODlCN0F2MjZ2akw3ZGdzZElXdGxoSFY3cVI0aVFhTUR5WXUvelJrRkh6ZFJp?=
 =?utf-8?B?a0U1K1ZTdVZZNERWSlVpd1c4aHlqS3lhNENLdUNFZWdPUXhOWU9meUc1R3dv?=
 =?utf-8?B?SEFPQkVPdnI1NTYwbWpQcEUyc0F2NHZ2QzZrcUlXY1FlMHVIL0pJbm41RTFo?=
 =?utf-8?B?aW5XakQySndDK1VmdUFVd3AwanZjUUFGaWxyNTRsei9QNThOTEZhK2RhTWpv?=
 =?utf-8?Q?uBZpnXUcTzZn+rSRIXZsXRHZ6s6fIFeFamEBOYf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4514c639-5875-4107-bfd0-08d97db20ef4
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:16:35.9386
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HW0YbVmdTsaANJAoAZwgIhIgF0LseVaLSA0hlmmR7vUU8Uwemvj3opwOmutQou0QlaA815pk9achUBVIzcuGJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

Osstest has been suffering test failures for a little while from order-4
allocation failures, resulting from alloc_empty_pages() calling
kcalloc(). As there's no need for physically contiguous space here,
switch to kvcalloc().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
RFC: I cannot really test this, as alloc_empty_pages() only gets used in
     the auto-translated case (i.e. on Arm or PVH Dom0, the latter of
     which I'm not trusting enough yet to actually start playing with
     guests).

There are quite a few more kcalloc() where it's not immediately clear
how large the element counts could possibly grow nor whether it would be
fine to replace them (i.e. physically contiguous space not required).

I wasn't sure whether to Cc stable@ here; the issue certainly has been
present for quite some time. But it didn't look to cause issues until
recently.

--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -420,7 +420,7 @@ static int alloc_empty_pages(struct vm_a
 	int rc;
 	struct page **pages;
 
-	pages = kcalloc(numpgs, sizeof(pages[0]), GFP_KERNEL);
+	pages = kvcalloc(numpgs, sizeof(pages[0]), GFP_KERNEL);
 	if (pages == NULL)
 		return -ENOMEM;
 
@@ -428,7 +428,7 @@ static int alloc_empty_pages(struct vm_a
 	if (rc != 0) {
 		pr_warn("%s Could not alloc %d pfns rc:%d\n", __func__,
 			numpgs, rc);
-		kfree(pages);
+		kvfree(pages);
 		return -ENOMEM;
 	}
 	BUG_ON(vma->vm_private_data != NULL);
@@ -912,7 +912,7 @@ static void privcmd_close(struct vm_area
 	else
 		pr_crit("unable to unmap MFN range: leaking %d pages. rc=%d\n",
 			numpgs, rc);
-	kfree(pages);
+	kvfree(pages);
 }
 
 static vm_fault_t privcmd_fault(struct vm_fault *vmf)



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:17:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:17:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192435.342868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzK7-00063C-Om; Wed, 22 Sep 2021 10:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192435.342868; Wed, 22 Sep 2021 10:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzK7-000635-L5; Wed, 22 Sep 2021 10:17:55 +0000
Received: by outflank-mailman (input) for mailman id 192435;
 Wed, 22 Sep 2021 10:17:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSzK6-00062x-5B
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:17:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5890668c-1b8e-11ec-b96a-12813bfff9fa;
 Wed, 22 Sep 2021 10:17:53 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-or5aBoWwPqO6SHe-yooqcw-1; Wed, 22 Sep 2021 12:17:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Wed, 22 Sep
 2021 10:17:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 10:17:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR06CA0131.eurprd06.prod.outlook.com (2603:10a6:208:ab::36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 10:17:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5890668c-1b8e-11ec-b96a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632305872;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=or78mkWgiZ0tw8GzqU9/omJtttnIwfRDs2Aq8lzGU1U=;
	b=MbKhPPGrz6A1M2QEqZIo91Seij94xONIDj+pFzlHq4axePAz0a4g6OawxZJ25IpClJqRMg
	WAlnVRUTgvP+JrG/r9JPAw3RsM4ZaKd1b5Lx/qrOZ8qPguSDdurWj5NoK27FRO6zqN66xf
	R+sOWsRIfaAkhlUiFlb+oN0CqNaYha0=
X-MC-Unique: or5aBoWwPqO6SHe-yooqcw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Pq42DDIJGOaxdlRTTXAtesDsX61o5He5iwNuSu1IvKWr0//tUA49NeO6CQT6mok2F9w2uUgT95mIo2bWc+QthDBNOZ9RrPgsE2ZZTcNaPxDLqhW6jvvpA+Xt9cB2/0kgIAIAV7BfZl7jwbxb0pDFBMTXmdXwYs2ZHCCWWg7ELb4EU+Ii+Vv7BGSh4ZXWQkt9qGbTEQU1A/MKN4z3PeAa2FDjlpu0G/fobBkaWR7fz2lLILbIHbkqBU1OzzdX54azdsqw77FJWvl28VtC7mcBxKxVcd71KoavNsSXo++W5HqPgMJl3EUngfT4PoRc1LTbyoJZuOPUXmJOPtih5bH4Yg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=or78mkWgiZ0tw8GzqU9/omJtttnIwfRDs2Aq8lzGU1U=;
 b=Bh9cC/Eybp/sXaq0Me5/3lkwwy1yg+YY1GdaRptkJWHjBn8N/n4qEQMdHtz/iWFRPawgyolq3PcNJK2S9oJr00CgEEl1sR6sMbI6GADZqqWp3komYglF2DJcWsRnDGoHw8AMdm1dXrCNEEWpAnPvs10V5fXYCpvKHeNxaavQAJd6pgVgES/75bJzZwGDTokP4XwPMlHOVqXI6ceUB+ilIf563CLbVp2H0dm/iESvC9DCQSvYCcyUUpaheUwfrv6ctfspK1BNtQ5iYs7H5nvYw/QZmtqJMfNmxjz7AZLg1RPQ7ifVJdaa+1yXa95xlvg64f6uW+0GKkny2tHPW7/tJw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/3] xen/privcmd: fix error handling in mmap-resource
 processing
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Message-ID: <aa6d6a67-6889-338a-a910-51e889f792d5@suse.com>
Date: Wed, 22 Sep 2021 12:17:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR06CA0131.eurprd06.prod.outlook.com
 (2603:10a6:208:ab::36) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d1b81141-4f67-4f24-d285-08d97db23af0
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4190F549AAEC8D89F27F8061B3A29@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vzmey8F4XxSEclZIYX0Y/bNPxOlwfO6kBi7n9FD6Z2d5z3q40SidvZQrCrNEe3cC3cyulWDOOcrQ+YQWCJ9QMzNQKsKsSJzBLfNZoWV6bv///Cppt/hGv0pBGSLA6Gk0k76ordvSQnP0ZrC1xlTREKbcrp3AXocLJQo7Yqupl43qoHfBc8XP4nwojsLXrPcjQTwy6RWzRqULJjsenaBqvm87S4s+c0K3smgBT4EEF00qAwUDxo698Wz2D53jVzBZBz+Q4swHDWHO1K+O+Sv9WCbjKYc+5iHwiH0W53QsJJm3WGXie9PS2zZs/3ZHU5HleOSYEBrlF0JteYDahY2YjWlpnhlT8Q8PNpIbifWXS0GZ/sFyvG3tDUU/rQXvbK/63vyrhTX1yVWGOxvVEos0rqAwxMM54U1I9V+XPm/MmF4vOoMZbwT0j7xG63ew1KUoyyi8kTLM0mM2inbfMqBjn2YlHUqJN/TindTWXrOpNv5aLeM9vTvWfvj+JB9IpqXv3mVLu118U8ndLYR7KHVW1mF+caDM7gLCf41a/OvExgPyB95cXXBqxZVwYrSppNy3qVnqYtd2u025kSrH7kbEZoQb802zipUuYSIYH0iU5B9WJ7HUctsdhH/oIW7EBkQ8gCbclDMv4RM2gctJnguKsmVS9RLhhqsjETnScHpiaa5g4j+P5m/czEu6QX3a6yhhsOtUfho5zcQvfVRu7sIhxCKvjhxb88WwvwEs7M+zhT4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(316002)(26005)(8936002)(110136005)(31696002)(8676002)(6486002)(16576012)(186003)(508600001)(5660300002)(54906003)(31686004)(66476007)(36756003)(38100700002)(66946007)(86362001)(66556008)(2906002)(956004)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bXVraHVkZ3VMVEt5bFFlWllWSEpVUjMwQ1RpWXZqUCs5SWtYblNuMGd0ZkJJ?=
 =?utf-8?B?dndQQW9nN2NpRyt3NllkSGRqTWZFUXZoYkIwVzNEZWlOWnZpSUtWM1pCRTBZ?=
 =?utf-8?B?OVpHbk1xeGtGMzRvNnpObHUyNWZUVnNydFdvRlYxaXl3YkFselRHNWQ4Y3hl?=
 =?utf-8?B?Qy92VEVWc1BRT2VwU204T0xSSTNjZERXUUViN3d4WTNxNnFDRjVoVE1rcmNF?=
 =?utf-8?B?WU9WaklqZDdRWVFRRGtNVnA1Ykk4TEd3L3ZMaHVzUW10aElTU0dmd2RYVExl?=
 =?utf-8?B?aDFkSThNUzZrQkVZbHRkQ1I5b3A0THVnMVlIRkIrc3dBMjFYdTBEK3JlNG1N?=
 =?utf-8?B?QjJSbENTTWYxOXNaUTJxaEdEUjloQzVQc3o2QTIranVwaVlhc1FHNEtXd0RX?=
 =?utf-8?B?SGhKY2V1MXRwN1c1U2t5QStwOGhNaDlWamRvTlFEQjhVQ2pCU2xjWFpldnBU?=
 =?utf-8?B?dnpPSWR6UjUyU25zaFdVa2NteXowd2dLMHprdE8yVHlzblVlRzNsTndhMU53?=
 =?utf-8?B?clpEZGlvaHdvTUdZcVgzbDdYVy9NbFEyZ1dZSmZxOVVvN3ZXY1ZSQ3hkZS9m?=
 =?utf-8?B?WWlML2swd0E5M1VJTHVzcEZ5NUVVSU8wNllKZW9abEZwdGpBU25KQ0xPZVdy?=
 =?utf-8?B?R3dQMFMzN01EL3Nrd01HakI5aUR2V3hXcmp6U3VsaXhjZkVxZlY0NXozRnJG?=
 =?utf-8?B?RitoZzVwRUJIQkVYWHRRVXU4Y3F6bmg5c0UrdENrU1JqeERMVmFybkEvc1Mz?=
 =?utf-8?B?NWtPSXh1RE51aDFpOVlEMlpRRFRvTllrVk9SczJUQjRLd2lzbGZ6azNnTGow?=
 =?utf-8?B?TXhTeXROMW5DbmFVNTFsWWZtdE16QkNIRHNDRjNhbXdieEk0TW1sK1Rhd0ln?=
 =?utf-8?B?VlFuZGViSnVpcDhVMVEwdThnRVFLRzAyU2d0ZEw1c1kxYmZjZWdmOVVGVysr?=
 =?utf-8?B?V3FpL1FxK1hoL1ZpQS8yaU1PeDVZVExZeklzNTZTTkczRFRndWQzZGhObGdq?=
 =?utf-8?B?T1o2Rjk1SWc4NGk0R0tlbDFReFFzS2dUNzQ4VURkUENKclVMV0M5ZjU1WmVl?=
 =?utf-8?B?eHpydFV3cWxSbmVqSGJubm1BMjYzN0ttelNLNjVQd21KRnllc1hNU2hFSFJF?=
 =?utf-8?B?c09TOFFldlhSTGYzT3ZmakV3TklkbEZDVUcrZGJiS3ZTaGJ4TW5JS096VEFG?=
 =?utf-8?B?VnUvcDgrampoQUpXOXhPc3NaMjhQS3lWOU0zYzlmSHoxUEY4YmFKcWs2SUc0?=
 =?utf-8?B?QmdFaVVpd04wUWRDQ2lzN2VkeGRCR0ZjeVdxeElmRk9ZckYxQjlxNmFzRTJM?=
 =?utf-8?B?c1pRNVhDSmRkYUcrY0ZrendUY0V1WVkzMFdBdXhBYktQMFhwT244U292Q1lN?=
 =?utf-8?B?NVlrNDZhVitVKzcwTGRrVGV3bmZ2UnlwUzB5UEg2TEt3L05UZHdyMVBVU0tx?=
 =?utf-8?B?aVk3cVQ0MkhJdDRDM3oyZ0lsYmJmY2JnSGpMSDczWUpFaXNKNnZIWUF5clIw?=
 =?utf-8?B?K1R3Qk9oUjNtL1hUMDIyNzJGTTBGVlV3MUNzSUNHbEVGaFlwc29ORHNEQUxu?=
 =?utf-8?B?SHh1VnRXR3FTMTAra3M0d1lnMDZMTUFDeCsrUWg0RjFCaTdoRXZhNlZ2c0tW?=
 =?utf-8?B?UzZDeFZVTjIwUWc4R2s5UHAyWlE3MlBEZUl4bVVUbnlseEhOYTRBVTlwUngz?=
 =?utf-8?B?UzcrUm82MkNWSk5hUlA0MThKYzEwTW9YQ285b2V2VW9xNGk0bWJPWVB3SjNt?=
 =?utf-8?Q?1pwRx5AGXYRigAASodJvomU/3WYKujQZgMSnw4I?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d1b81141-4f67-4f24-d285-08d97db23af0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:17:49.7196
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NDo1bw6H6EUu5BGCJ1Go3BIGApuV6ybSkpmaX66Us4ciSMCt68Qt/1TjQbAqHv81H+OmiaLXjUSECrn3AZGO6g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

xen_pfn_t is the same size as int only on 32-bit builds (and not even
on Arm32). Hence pfns[] can't be used directly to read individual error
values returned from xen_remap_domain_mfn_array(); every other error
indicator would be skipped/ignored on 64-bit.

Fixes: 3ad0876554ca ("xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESOURCE")
Cc: stable@vger.kernel.org
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -803,11 +803,12 @@ static long privcmd_ioctl_mmap_resource(
 		unsigned int domid =
 			(xdata.flags & XENMEM_rsrc_acq_caller_owned) ?
 			DOMID_SELF : kdata.dom;
-		int num;
+		int num, *errs = (int *)pfns;
 
+		BUILD_BUG_ON(sizeof(*errs) > sizeof(*pfns));
 		num = xen_remap_domain_mfn_array(vma,
 						 kdata.addr & PAGE_MASK,
-						 pfns, kdata.num, (int *)pfns,
+						 pfns, kdata.num, errs,
 						 vma->vm_page_prot,
 						 domid,
 						 vma->vm_private_data);
@@ -817,7 +818,7 @@ static long privcmd_ioctl_mmap_resource(
 			unsigned int i;
 
 			for (i = 0; i < num; i++) {
-				rc = pfns[i];
+				rc = errs[i];
 				if (rc < 0)
 					break;
 			}



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:18:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:18:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192441.342878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzKi-0006cc-1S; Wed, 22 Sep 2021 10:18:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192441.342878; Wed, 22 Sep 2021 10:18:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzKh-0006cV-Ua; Wed, 22 Sep 2021 10:18:31 +0000
Received: by outflank-mailman (input) for mailman id 192441;
 Wed, 22 Sep 2021 10:18:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSzKh-0006cJ-FB
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:18:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6e99877e-1b8e-11ec-b96a-12813bfff9fa;
 Wed, 22 Sep 2021 10:18:30 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-4xuDO56rM0qlH9jh8_4EFA-2; Wed, 22 Sep 2021 12:18:28 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Wed, 22 Sep
 2021 10:18:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 10:18:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR06CA0132.eurprd06.prod.outlook.com (2603:10a6:208:ab::37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 10:18:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e99877e-1b8e-11ec-b96a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632305909;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mKNtSH+m7Ctv8L/JFGkCYRCWB0Q8bcvCLMU0buPXpMY=;
	b=FdPbRCnNparaOijdU1QBj3W4YpoJnoaBNmo5zA3Mshu5LlowSD5oa4D97EH8RqrxRIoGb9
	I9JhfxpiY75EiCSgy4jV1G9zqNtTWx87WzaIVou1I1wOmNm9f6ayK0c8FWXkO/XfvW02jR
	oNa5fw8Mx61y8BFNodtVFNxwU6uek2c=
X-MC-Unique: 4xuDO56rM0qlH9jh8_4EFA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CjNCycKGV86L/ODvBIkFk1QMP8CJdyHhMCiR3RMebb3eb5WGurjawSgLjjwg7bbj1zIyrAbKkFU2bZN6Z/lnPjxSnmucDdoJ6BZifdPx7GqLTKiycQnWydGxBDQXcwHnXT4hpR3zecS8ArXiJwVoM6MnLSsEEuMQQwwEH/R6VWyUAW/+y0xlfcxsP2GUnaZ2cWZzfp9D0f7jRMyam2HZzxZrs1qIvwpDLLBukD///tGOWCp/buR/qNCsRuHZijVJ9miBcDTTZZtDvsiXiqtm2srIXAPS/qIeKw+3JNurc5eKWV/DtYtEshmjxw+reewsAe5MnjZkVr0IGOFiI2H85A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mKNtSH+m7Ctv8L/JFGkCYRCWB0Q8bcvCLMU0buPXpMY=;
 b=SbeNcyePJ6wZtaGaJMb2cXRq1Useh6PufGpSoDvHCUX7x8iA6OrGGrLdfse6AJz93pdGXAH1ddyKQKAHWgPbrA4AH99ur+l76Y1Fa/VXF1vY7Q1cd8Liy62kq9tyzs8yEDhTbVsH+po2swU7WA782HKlx3uk1EiC9PwII05Vp+bc7nSbHzXmhfQrvIeOtBM9wraaQdTyBgEL5pATVEPe5EhH0BNZMg8wNGmml6vvzpBt2TX4RbCufa0qcHh5Kje15/XvI9+7sUhymthiCbHHJ8qQq07ri9CHWraPcyNim5U719TXB2T3if/XmLotqtO0UNXkF0kbZ4MRGdg/6GN1Xw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/3] xen/privcmd: drop "pages" parameter from xen_remap_pfn()
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Message-ID: <036ad8a2-46f9-ac3d-6219-bdc93ab9e10b@suse.com>
Date: Wed, 22 Sep 2021 12:18:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR06CA0132.eurprd06.prod.outlook.com
 (2603:10a6:208:ab::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fa92bbca-4f8f-4de0-4489-08d97db250a6
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41900F73443597CCBCD4F93FB3A29@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	x/dREiUX63gOXj3JmwHyqFFVDYH6WmNYUbQe4/Xkte5PjMylVy4+bokdrN8qgG0ZaOKcko29gxjkEbXSPLvH1FOpaMwq5b157Maz6RXDB2OPPJniuthixKFR+Vr7mw3pl+R2jIUzZM8oyzEwfYXwE5qQFPkIuyd5j80XGtYSvCShPrnq/CIKLaLr1WQgzBYU5/5oVYpyrcegXtmAljeh20gUzk/qB4yFZzyqTkfuyyQ6sP44nDfBinK08fsT5+bcgvwLKtbpoR/CzgBKVPeFroC2euluY0i33G84q12o3K5Ma+RnTXrN/y4l/Gah5Sfo1lPcOr1BTGuI65wq+mokMYnq3ImvSyE+XvBP8F5lMBb+R0i3fnnWATjcY+3prx93MUA9GjHscx8vDZNNbHLPobzGWMmPlESb/SAk1S+JyztJEPZK20mfw2HBd6dYvDNvQsrTXIHExbFz0QkhduiWsDL1iWuJQgJx/ywRpvMg1DSanPW364+2G6KsMxtoFj4Qs2UjYfl/XXIQtQC/Q1vDJoitUf6fK2XGs4XN3MpFaEnFP5X+kTx12SesEz96LBfJigvqnF7gT7xVmboPdCQbtVWIXchY1N0I/YVErzDMv+g3fAcwYHtOORsIV1HHfvrdtmA4JamagiQVjN/8yBIxEs60K+X9W0dsJIiTX8eV13WzdxptJZ54DDjcWrtiHIzfsMWCYdU05eYBMqpOQs6e6YmAsSqWzzGXi6DAQ9GSchY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(83380400001)(508600001)(31686004)(5660300002)(66476007)(2906002)(66556008)(956004)(2616005)(36756003)(38100700002)(66946007)(86362001)(8936002)(316002)(26005)(4326008)(110136005)(8676002)(16576012)(186003)(6486002)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aXZGbFlML3R1c0xPU2o0cFYwb1llWE51RGlmRkxPNEpiRHhteUppRE14TWND?=
 =?utf-8?B?VS9iQm5UMnBraThkZ0tyS1N2VTNKemRQWVBTVitRVENYMDFEZG1meTlpempF?=
 =?utf-8?B?VnNhWEthT0FVdDI1QnlUcG5SYWdYZGFTZ05NNE1XKzRCUFQ5d3JwWVpiTHNQ?=
 =?utf-8?B?dXlnNTF0MENCMm5tY3V3M3A1UEtPelJsakhrdW9iaGxsOFNzTktnbHc3bDVK?=
 =?utf-8?B?QjR6UXRaNHd3NjRYNGlpT3BlYjB4bjByalo1RkNJc1p5bjhkRDJ1ZjZHZXFh?=
 =?utf-8?B?LzljYndBSUxRTWVrcnhZell1aTZ1bEJGL3dYUHNma1l5SElsbXR6YTlITVlF?=
 =?utf-8?B?U0ZOZVFqTk03RnltbWk4bE5oUHlMTVovQnBsUjZMcFVLNXd3TjZCL2d0cmtM?=
 =?utf-8?B?d0w3STNnNDdoL2FPYys0akJYLzBkdWFlZEdRRU01T2M0RGdBU0xTYXM1OVVx?=
 =?utf-8?B?S3owTWRmT1FJNDdWdWkvdWlFY2hrUkl3ZlJjNVdTcTBSNytYOElIRmJ0OFdp?=
 =?utf-8?B?aDJUY3Zjclg1My9oZzVvY3p2UGRYSmdRQXFZZnVwd1JheDZSdjdzcXVkTUdC?=
 =?utf-8?B?enJYcER2VmlKcXAzb29mUU8reVFpaVlHbkxTZitLTjBjcWpxR283MVl0WWlM?=
 =?utf-8?B?akNiVnpPRHkwRFJYWnFSOWIwZjd4bDg4azRmVisyMnJTME5HTjVLbGRIOGFD?=
 =?utf-8?B?SnMzbzZMbTlHaEtGTGhLdXI3b1BKV2h0M2w2azRNdlAzNE5acWRvM3V2bGV6?=
 =?utf-8?B?RnZMWDRWQjdwOUE5Y1lNcG4wN0RCblFSZjkyVFpXaFovY1pTNVgxY21qV3Vk?=
 =?utf-8?B?S3dSMm42M01jU3BhQ080bHhocGJldkNjSURjblN1M1N0NitjSDlXNE1lVHBI?=
 =?utf-8?B?ZkFFZndwTUgyeVJnc2RxbXc5THdZd1ZwbWU4ZkxNZHlqWTkzbmF3MHVyYUZR?=
 =?utf-8?B?OFNPakJjK3NkZWMxYjUvcVdYQ2lueEwrU2NDL0xQRk82Qm5PT2lBdkFXOWhm?=
 =?utf-8?B?ejgzV3V1TzVOVERrK01VWEJ2Mm5jMVlXc0tmWHNXZENFSGpmeHIzRDh4cEZF?=
 =?utf-8?B?N2QrTnZwdGZPKzRiVnBGcmlWZHRDbVRRSzB3NUpjaEtxMWh5M0tqMVZJZkZS?=
 =?utf-8?B?SE51ZUM5R2RidHpVRnBGYituQ05LZnI1QU8xMVY4ZDZwS3JweThEcGg0SnNC?=
 =?utf-8?B?R2Ftc1dTODVjbTZXV2NtREVXUUErOW5YN3ZBalpBemhBRTVhNVQxbjNqbmxu?=
 =?utf-8?B?K0xBZWI1VTI2bVdua2VGVDMzc3loSHlYOCtCZ2tYT291UjIzcnRvSzJNN3FV?=
 =?utf-8?B?aXF5c2tEK1RvY0QwNHk2MlVyRnY3Wmg0bnZoelViVnpPNVhyZGZzcldqM3Vy?=
 =?utf-8?B?L1BtZXlDcnR4eE1hSis0bFJFQXlhczN5eVc5UHBSVVp0djh5emhWT0tVTUZF?=
 =?utf-8?B?N3UzdmxIV0VFMlc4WVZ4YjJVUWM4S3V2MTB1RStkd3hzZHZPTE9LS3ZyUEQ4?=
 =?utf-8?B?QW43L2NrcVVYNzRyUDZZTTRJWTJqbDRZUzNTVEJ1djdYRStpSEs4czdmOG9l?=
 =?utf-8?B?LzNyM3h2dWNSM2pFb1hickttamRrMWdSMGsvSE5UYjRLa3ZyMGQ4UnRkczYy?=
 =?utf-8?B?SjFFc1lYZnNTeGFRN1ZVKytndktqQThaYlpnTm9kSVhsWU4xdXRvRG1TV1Jo?=
 =?utf-8?B?aHY0aExlQ0hjcVRqSXBlUVFlYTY2OWdUTGNudDNHeXVzQUE3eFlEZ1lFeVlq?=
 =?utf-8?Q?A/DKQNSH/EUA/TjxtX1kC01peWDaKxX0+vf6YHT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fa92bbca-4f8f-4de0-4489-08d97db250a6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:18:26.1589
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZX2I0VRntI1n/gqXn52Bnj1vW6lxkkAvmC7BCjfiQ2q+2qsEnebZ+1Fi1MmgH0JcNYvJKVGrE9vnsWvBRnncXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

The function doesn't use it and all of its callers say in a comment that
their respective arguments are to be non-NULL only in auto-translated
mode. Since xen_remap_domain_mfn_array() isn't supposed to be used by
non-PV, drop the parameter there as well. It was bogusly passed as non-
NULL (PRIV_VMA_LOCKED) by its only caller anyway. For
xen_remap_domain_gfn_range(), otoh, it's not clear at all why this
wouldn't want / might not need to gain auto-translated support down the
road, so the parameter is retained there despite now remaining unused
(and the only caller passing NULL); correct a respective comment as
well.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -2398,7 +2398,7 @@ static int remap_area_pfn_pte_fn(pte_t *
 
 int xen_remap_pfn(struct vm_area_struct *vma, unsigned long addr,
 		  xen_pfn_t *pfn, int nr, int *err_ptr, pgprot_t prot,
-		  unsigned int domid, bool no_translate, struct page **pages)
+		  unsigned int domid, bool no_translate)
 {
 	int err = 0;
 	struct remap_data rmd;
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -257,7 +257,7 @@ static long privcmd_ioctl_mmap(struct fi
 	LIST_HEAD(pagelist);
 	struct mmap_gfn_state state;
 
-	/* We only support privcmd_ioctl_mmap_batch for auto translated. */
+	/* We only support privcmd_ioctl_mmap_batch for non-auto-translated. */
 	if (xen_feature(XENFEAT_auto_translated_physmap))
 		return -ENOSYS;
 
@@ -810,8 +810,7 @@ static long privcmd_ioctl_mmap_resource(
 						 kdata.addr & PAGE_MASK,
 						 pfns, kdata.num, errs,
 						 vma->vm_page_prot,
-						 domid,
-						 vma->vm_private_data);
+						 domid);
 		if (num < 0)
 			rc = num;
 		else if (num != kdata.num) {
--- a/include/xen/xen-ops.h
+++ b/include/xen/xen-ops.h
@@ -64,12 +64,12 @@ static inline void xen_destroy_contiguou
 #if defined(CONFIG_XEN_PV)
 int xen_remap_pfn(struct vm_area_struct *vma, unsigned long addr,
 		  xen_pfn_t *pfn, int nr, int *err_ptr, pgprot_t prot,
-		  unsigned int domid, bool no_translate, struct page **pages);
+		  unsigned int domid, bool no_translate);
 #else
 static inline int xen_remap_pfn(struct vm_area_struct *vma, unsigned long addr,
 				xen_pfn_t *pfn, int nr, int *err_ptr,
 				pgprot_t prot,  unsigned int domid,
-				bool no_translate, struct page **pages)
+				bool no_translate)
 {
 	BUG();
 	return 0;
@@ -146,7 +146,7 @@ static inline int xen_remap_domain_gfn_a
 	 */
 	BUG_ON(err_ptr == NULL);
 	return xen_remap_pfn(vma, addr, gfn, nr, err_ptr, prot, domid,
-			     false, pages);
+			     false);
 }
 
 /*
@@ -158,7 +158,6 @@ static inline int xen_remap_domain_gfn_a
  * @err_ptr: Returns per-MFN error status.
  * @prot:    page protection mask
  * @domid:   Domain owning the pages
- * @pages:   Array of pages if this domain has an auto-translated physmap
  *
  * @mfn and @err_ptr may point to the same buffer, the MFNs will be
  * overwritten by the error codes after they are mapped.
@@ -169,14 +168,13 @@ static inline int xen_remap_domain_gfn_a
 static inline int xen_remap_domain_mfn_array(struct vm_area_struct *vma,
 					     unsigned long addr, xen_pfn_t *mfn,
 					     int nr, int *err_ptr,
-					     pgprot_t prot, unsigned int domid,
-					     struct page **pages)
+					     pgprot_t prot, unsigned int domid)
 {
 	if (xen_feature(XENFEAT_auto_translated_physmap))
 		return -EOPNOTSUPP;
 
 	return xen_remap_pfn(vma, addr, mfn, nr, err_ptr, prot, domid,
-			     true, pages);
+			     true);
 }
 
 /* xen_remap_domain_gfn_range() - map a range of foreign frames
@@ -200,8 +198,7 @@ static inline int xen_remap_domain_gfn_r
 	if (xen_feature(XENFEAT_auto_translated_physmap))
 		return -EOPNOTSUPP;
 
-	return xen_remap_pfn(vma, addr, &gfn, nr, NULL, prot, domid, false,
-			     pages);
+	return xen_remap_pfn(vma, addr, &gfn, nr, NULL, prot, domid, false);
 }
 
 int xen_unmap_domain_gfn_range(struct vm_area_struct *vma,



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:23:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:23:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192449.342890 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzPh-00088r-Od; Wed, 22 Sep 2021 10:23:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192449.342890; Wed, 22 Sep 2021 10:23:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzPh-00088k-LA; Wed, 22 Sep 2021 10:23:41 +0000
Received: by outflank-mailman (input) for mailman id 192449;
 Wed, 22 Sep 2021 10:23:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mSzPg-00088e-Ir
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:23:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSzPf-0004hB-5M; Wed, 22 Sep 2021 10:23:39 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSzPe-0001Rc-Pe; Wed, 22 Sep 2021 10:23:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Qgcee8m4YPfLmiL81b5V15n8qvz2mYYoGyYhoDxNCW0=; b=1Et153U0rZtO9XAgH9Uynp0LkZ
	Y4mi9QnmxUAjEHClu903z6rRqEiGo4EpAGsREOqNZEkRjVgx66OuZieBVkDFDMoR3+CubEoL1wJyT
	H/01SmqMHoHVDRKCI/CB+65qjhJKZqdp16CDjSDW/ljWXqRUF4762RRSbBV80T2HWo4s=;
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
From: Julien Grall <julien@xen.org>
Message-ID: <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
Date: Wed, 22 Sep 2021 15:23:28 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 22/09/2021 14:58, Roger Pau Monné wrote:
> On Wed, Sep 22, 2021 at 02:07:44PM +0500, Julien Grall wrote:
>> Hi Roger,
>>
>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>> Failure to map the shared ring and thus establish a xenstore
>>> connection with a domain shouldn't prevent the "@introduceDomain"
>>> watch from firing, likewise with "@releaseDomain".
>>>
>>> In order to handle such events properly xenstored should keep track of
>>> the domains even if the shared communication ring cannot be mapped.
>>> This was partially the case due to the restore mode, which needs to
>>> handle domains that have been destroyed between the save and restore
>>> period. This patch extends on the previous limited support of
>>> temporary adding a domain without a valid interface ring, and modifies
>>> check_domains to keep domains without an interface on the list.
>>>
>>> Handling domains without a valid shared ring is required in order to
>>> support domain without a grant table, since the lack of grant table
>>> will prevent the mapping of the xenstore ring grant reference.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>> ---
>>> oxenstored will need a similar treatment once grant mapping is used
>>> there. For the time being it still works correctly because it uses
>>> foreign memory to map the shared ring, and that will work in the
>>> absence of grant tables on the domain.
>>> ---
>>> Changes since v1:
>>>    - New in this version.
>>> ---
>>>    tools/xenstore/xenstored_domain.c | 30 ++++++++++++++++++------------
>>>    1 file changed, 18 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c
>>> index 9fb78d5772..150c6f082e 100644
>>> --- a/tools/xenstore/xenstored_domain.c
>>> +++ b/tools/xenstore/xenstored_domain.c
>>> @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
>>>    	struct xenstore_domain_interface *intf = conn->domain->interface;
>>>    	XENSTORE_RING_IDX cons, prod;
>>> +	if (!intf) {
>>> +		errno = ENODEV;
>>> +		return -1;
>>> +	}
>>> +
>>>    	/* Must read indexes once, and before anything else, and verified. */
>>>    	cons = intf->rsp_cons;
>>>    	prod = intf->rsp_prod;
>>> @@ -149,6 +154,11 @@ static int readchn(struct connection *conn, void *data, unsigned int len)
>>>    	struct xenstore_domain_interface *intf = conn->domain->interface;
>>>    	XENSTORE_RING_IDX cons, prod;
>>> +	if (!intf) {
>>> +		errno = ENODEV;
>>> +		return -1;
>>> +	}
>>> +
>>>    	/* Must read indexes once, and before anything else, and verified. */
>>>    	cons = intf->req_cons;
>>>    	prod = intf->req_prod;
>>> @@ -176,6 +186,9 @@ static bool domain_can_write(struct connection *conn)
>>>    {
>>>    	struct xenstore_domain_interface *intf = conn->domain->interface;
>>> +	if (!intf)
>>> +		return false;
>>> +
>>
>> Rather than adding an extra check, how about taking advantage of is_ignore?
> 
> Right, I just need to change the order in conn_can_read and
> conn_can_write so that the is_ignored check is performed before the
> can_{read,write} handler is called.
> 
>>>    	return ((intf->rsp_prod - intf->rsp_cons) != XENSTORE_RING_SIZE);
>>>    }
>>> @@ -183,7 +196,8 @@ static bool domain_can_read(struct connection *conn)
>>>    {
>>>    	struct xenstore_domain_interface *intf = conn->domain->interface;
>>> -	if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
>>> +	if ((domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0) ||
>>> +	    !intf)
>>>    		return false;
>>>    	return (intf->req_cons != intf->req_prod);
>>> @@ -268,14 +282,7 @@ void check_domains(void)
>>>    				domain->shutdown = true;
>>>    				notify = 1;
>>>    			}
>>> -			/*
>>> -			 * On Restore, we may have been unable to remap the
>>> -			 * interface and the port. As we don't know whether
>>> -			 * this was because of a dying domain, we need to
>>> -			 * check if the interface and port are still valid.
>>> -			 */
>>> -			if (!dominfo.dying && domain->port &&
>>> -			    domain->interface)
>>> +			if (!dominfo.dying)
>>>    				continue;
>>
>> This is mostly a revert on "tools/xenstore: handle dying domains in live
>> update". However, IIRC, this check was necessary to release the connection
>> if the domain has died in the middle of Live-Update.
> 
> But if the domain has died in the middle of live update
> get_domain_info will return false, and thus the code won't get here.

Hmmm... I think I am mixing up a few things... I went through the 
original discussion (it was on the security ML) to find out why I wrote 
the patch like that. When going through the archives, I noticed that I 
provided a different version of this patch (see [1]) because there was 
some issue with the check here (I wrote that it would lead to zombie 
domain, but don't have the rationale :().

Juergen, I don't seem to find the reason why the patch was not replaced 
as we discussed on the security ML. Do you remember why?

Assuming this was a mistake, could someone take care of sending an 
update? If not, I could do it when I am back in October.

For the archives, the issues would appear when shutting down a domain 
during Live-Update.

[1]

commit 3d1f5b71f8565787c0cf305e5571884d6aec6865
Author: Julien Grall <jgrall@amazon.com>
Date:   Thu Jun 11 16:13:10 2020 +0200

      tools/xenstore: handle dying domains in live update

      A domain could just be dying when live updating Xenstore, so the case
      of not being able to map the ring page or to connect to the event
      channel must be handled gracefully.

      Signed-off-by: Julien Grall <jgrall@amazon.com>

diff --git a/tools/xenstore/xenstored_control.c
b/tools/xenstore/xenstored_control.c
index d1187a4346b4..92dae6be6195 100644
--- a/tools/xenstore/xenstored_control.c
+++ b/tools/xenstore/xenstored_control.c
@@ -533,6 +533,13 @@ void lu_read_state(void)
         lu_close_dump_state(&state);

         talloc_free(ctx);
+
+       /*
+        * We may have missed the VIRQ_DOM_EXC notification and a domain may
+        * have died while we were live-updating. So check all the 
domains are
+        * still alive.
+        */
+       check_domains();
   }

   static const char *lu_activate_binary(const void *ctx)
diff --git a/tools/xenstore/xenstored_domain.c
b/tools/xenstore/xenstored_domain.c
index 4976ae420800..0519e88eb819 100644
--- a/tools/xenstore/xenstored_domain.c
+++ b/tools/xenstore/xenstored_domain.c
@@ -224,7 +224,7 @@ static bool get_domain_info(unsigned int domid,
xc_dominfo_t *dominfo)
                dominfo->domid == domid;
   }

-static void domain_cleanup(void)
+void check_domains()
   {
         xc_dominfo_t dominfo;
         struct domain *domain;
@@ -248,6 +248,7 @@ static void domain_cleanup(void)
                                 domain->shutdown = true;
                                 notify = 1;
                         }
+
                         if (!dominfo.dying)
                                 continue;
                 }
@@ -274,7 +275,7 @@ void handle_event(void)
                 barf_perror("Failed to read from event fd");

         if (port == virq_port)
-               domain_cleanup();
+               check_domains();

         if (xenevtchn_unmask(xce_handle, port) == -1)
                 barf_perror("Failed to write to event fd");
@@ -359,7 +360,7 @@ static struct domain *find_or_alloc_domain(const
void *ctx, unsigned int domid)
         return domain ? : alloc_domain(ctx, domid);
   }

-static int new_domain(struct domain *domain, int port)
+static int new_domain(struct domain *domain, int port, bool restore)
   {
         int rc;

@@ -375,9 +376,10 @@ static int new_domain(struct domain *domain, int port)

         /* Tell kernel we're interested in this event. */
         rc = xenevtchn_bind_interdomain(xce_handle, domain->domid, port);
-       if (rc == -1)
+       if (rc != -1)
+               domain->port = rc;
+       else if (!restore)
                 return errno;
-       domain->port = rc;

         domain->introduced = true;

@@ -428,7 +430,7 @@ static void domain_conn_reset(struct domain *domain)

   static struct domain *introduce_domain(const void *ctx,
                                        unsigned int domid,
-                                      evtchn_port_t port, bool fire_watch)
+                                      evtchn_port_t port, bool restore)
   {
         struct domain *domain;
         int rc;
@@ -440,11 +442,12 @@ static struct domain *introduce_domain(const void
*ctx,

         if (!domain->introduced) {
                 interface = map_interface(domid);
-               if (!interface)
+               if (!interface && !restore)
                         return NULL;
-               if (new_domain(domain, port)) {
+               if (new_domain(domain, port, restore)) {
                         rc = errno;
-                       unmap_interface(interface);
+                       if (interface)
+                               unmap_interface(interface);
                         errno = rc;
                         return NULL;
                 }
@@ -453,7 +456,7 @@ static struct domain *introduce_domain(const void *ctx,
                 /* Now domain belongs to its connection. */
                 talloc_steal(domain->conn, domain);

-               if (fire_watch)
+               if (!restore)
                         fire_watches(NULL, ctx, "@introduceDomain", NULL,
                                     false, NULL);
         } else {
@@ -490,7 +493,7 @@ int do_introduce(struct connection *conn, struct
buffered_data *in)
         if (port <= 0)
                 return EINVAL;

-       domain = introduce_domain(in, domid, port, true);
+       domain = introduce_domain(in, domid, port, false);
         if (!domain)
                 return errno;

@@ -730,7 +733,7 @@ static int dom0_init(void)
         dom0 = alloc_domain(NULL, xenbus_master_domid());
         if (!dom0)
                 return -1;
-       if (new_domain(dom0, port))
+       if (new_domain(dom0, port, false))
                 return -1;

         dom0->interface = xenbus_map();
@@ -1298,10 +1301,20 @@ void read_state_connection(const void *ctx,
const void *state)
                 conn = domain->conn;
         } else {
                 domain = introduce_domain(ctx, sc->spec.ring.domid,
-                                         sc->spec.ring.evtchn, false);
+                                         sc->spec.ring.evtchn, true);
                 if (!domain)
                         barf("domain allocation error");

+               conn = domain->conn;
+
+               /*
+                * The domain is about to die if we didn't manage to
+                * map the interface or the port. Mark the domain as
+                * ignored so it will be cleaned up when the domain
+                * is dead.
+                */
+               conn->is_ignored = !(domain->port && domain->interface);
+
                 if (sc->spec.ring.tdomid != DOMID_INVALID) {
                         tdomain = find_or_alloc_domain(ctx,
 
sc->spec.ring.tdomid);
@@ -1310,7 +1323,6 @@ void read_state_connection(const void *ctx, const
void *state)
                         talloc_reference(domain->conn, tdomain->conn);
                         domain->conn->target = tdomain->conn;
                 }
-               conn = domain->conn;
         }

         conn->conn_id = sc->conn_id;
diff --git a/tools/xenstore/xenstored_domain.h
b/tools/xenstore/xenstored_domain.h
index b71ab6d8a140..ec3b95a97195 100644
--- a/tools/xenstore/xenstored_domain.h
+++ b/tools/xenstore/xenstored_domain.h
@@ -21,6 +21,8 @@

   void handle_event(void);

+void check_domains(void);
+
   /* domid, mfn, eventchn, path */
   int do_introduce(struct connection *conn, struct buffered_data *in);

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:28:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:28:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192455.342901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzUG-0000W0-Bd; Wed, 22 Sep 2021 10:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192455.342901; Wed, 22 Sep 2021 10:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzUG-0000Vt-7x; Wed, 22 Sep 2021 10:28:24 +0000
Received: by outflank-mailman (input) for mailman id 192455;
 Wed, 22 Sep 2021 10:28:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mSzUF-0000Vn-64
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:28:23 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSzUE-0004mO-CB; Wed, 22 Sep 2021 10:28:22 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mSzUD-0001fQ-Gd; Wed, 22 Sep 2021 10:28:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=CPMr5A/aryLVbQ2Idr7tUHKgDUVUTrk5R3/EN889HlU=; b=liPhpRWuzeS1lH6J2x93akwJmM
	ksdPwY2QOyIXqtk1z5O41lOEOorTd119bdZUcVAdbaNcXnKQmFY2Ub6nC4l+wc/h3LvbflRBnMkQ4
	ku6dFlSJ3pcO/cnibhV9OaWVE36MX3UX+Ipwi34y0Srrj6Y3PQ89sui2njV/CcFZQrxQ=;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
 <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <b7947619-3520-174c-f5ec-49564ee048a1@xen.org>
Date: Wed, 22 Sep 2021 15:28:15 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

On 22/09/2021 14:42, Jan Beulich wrote:
> On 22.09.2021 11:26, Roger Pau Monné wrote:
>> On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
>>> On 21.09.2021 10:32, Roger Pau Monné wrote:
>>>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>>>>> On 20.09.2021 12:20, Roger Pau Monné wrote:
>>>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>>>>> --- a/xen/include/asm-arm/grant_table.h
>>>>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
>>>>>>
>>>>>> I'm slightly confused by this checks, don't you need to check for
>>>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>>>>> guest_physmap_remove_page?
>>>>>
>>>>> Why? It's ogfn which gets passed to the function. And it indeed is the
>>>>> prior GFN's mapping that we want to remove here.
>>>>>
>>>>>> Or assuming that ogfn is not invalid can be used to imply a removal?
>>>>>
>>>>> That implication can be (and on x86 is) used for the incoming argument,
>>>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>>>>
>>>> I guess I'm confused due to the ogfn checks done on the Arm side that
>>>> are not performed on x86. So on Arm you always need to explicitly
>>>> unhook the previous GFN before attempting to setup a new mapping,
>>>> while on x86 you only need to do this when it's a removal in order to
>>>> clear the entry?
>>>
>>> The difference isn't with guest_physmap_add_entry() (both x86 and
>>> Arm only insert a new mapping there), but with
>>> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
>>> mappings. And gnttab_map_frame() gets called only from there. This
>>> is effectively what the first paragraph of the description is about.
>>
>> OK, sorry, it wasn't clear to me from the description. Could you
>> explicitly mention in the description that the removal is moved into
>> gnttab_set_frame_gfn on Arm in order to cope with the fact that
>> xenmem_add_to_physmap_one doesn't perform it.
> 
> Well, it's not really "in order to cope" - that's true for the placement
> prior to this change as well, so not a justification for the change.
> Nevertheless I've tried to make this more clear by changing the 1st
> paragraph to:
> 
> "Without holding appropriate locks, attempting to remove a prior mapping
>   of the underlying page is pointless, as the same (or another) mapping
>   could be re-established by a parallel request on another vCPU. Move the
>   code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
>   xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
>   course this new placement doesn't improve things in any way as far as
>   the security of grant status frame mappings goes (see XSA-379). Proper
>   locking would be needed here to allow status frames to be mapped
>   securely."
> 
>> TBH I think it would be in our best interest to try to make
>> xenmem_add_to_physmap_one behave as close as possible between arches.
>> This discrepancy between x86 and Arm regarding page removal is just
>> going to bring more trouble in the long term, and hiding the
>> differences inside gnttab_set_frame_gfn just makes it even more
>> obscure.
> 
> Stefano, Julien?

This would be ideal as I don't particular like the approach taken in 
this patch. But AFAICT, this would require us to implement an M2P. Or is 
there another way to do it?

In another context, I saw the suggestion to bring an M2P on Arm. But I 
am still somewhat split whether this is really worth it for the current use.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192462.342917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzWx-0001v7-38; Wed, 22 Sep 2021 10:31:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192462.342917; Wed, 22 Sep 2021 10:31:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzWw-0001tk-UP; Wed, 22 Sep 2021 10:31:10 +0000
Received: by outflank-mailman (input) for mailman id 192462;
 Wed, 22 Sep 2021 10:31:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSzWv-0001rt-Ug
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:31:09 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3271bbe8-1b90-11ec-b96b-12813bfff9fa;
 Wed, 22 Sep 2021 10:31:08 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 67E45201DF;
 Wed, 22 Sep 2021 10:31:07 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2199614283;
 Wed, 22 Sep 2021 10:31:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id wIFTBusFS2GHWAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 10:31:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3271bbe8-1b90-11ec-b96b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632306667; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ngmCVTQbKbm84c7RltVCghPRIprk7FknXT4ygf5oYdM=;
	b=Cs9Z/NNUoJCgAnW0P5o26msOSLJMDA8XbwaK3/Nc3tHJL9dmdC/ukAR6cMtPUKbsiXS2Jc
	BvD0y8ECEvijpkcGECUHWLjC69a+rDInFdAMGnG/M2yzKppdAOqMpZKPFR/l0iRnkowzVo
	wB/rUzh1W/GsSKVpF3Xm5nuVrjRy0h4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org
Cc: peterz@infradead.org,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH v2 1/2] x86/xen: remove xen_have_vcpu_info_placement flag
Date: Wed, 22 Sep 2021 12:31:01 +0200
Message-Id: <20210922103102.3589-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210922103102.3589-1-jgross@suse.com>
References: <20210922103102.3589-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The flag xen_have_vcpu_info_placement was needed to support Xen
hypervisors older than version 3.4. Today the Linux kernel requires
at least Xen 4.0 to be able to run, so xen_have_vcpu_info_placement
can be dropped.

This allows to let some functions return void now, as they can never
fail.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/enlighten.c     | 97 +++++++++---------------------------
 arch/x86/xen/enlighten_hvm.c |  6 +--
 arch/x86/xen/enlighten_pv.c  | 28 ++---------
 arch/x86/xen/smp.c           | 24 ---------
 arch/x86/xen/xen-ops.h       |  4 +-
 5 files changed, 33 insertions(+), 126 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index c79bd0af2e8c..8a7bd3f4591e 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -81,21 +81,6 @@ EXPORT_SYMBOL(xen_start_flags);
  */
 struct shared_info *HYPERVISOR_shared_info = &xen_dummy_shared_info;
 
-/*
- * Flag to determine whether vcpu info placement is available on all
- * VCPUs.  We assume it is to start with, and then set it to zero on
- * the first failure.  This is because it can succeed on some VCPUs
- * and not others, since it can involve hypervisor memory allocation,
- * or because the guest failed to guarantee all the appropriate
- * constraints on all VCPUs (ie buffer can't cross a page boundary).
- *
- * Note that any particular CPU may be using a placed vcpu structure,
- * but we can only optimise if the all are.
- *
- * 0: not available, 1: available
- */
-int xen_have_vcpu_info_placement = 1;
-
 static int xen_cpu_up_online(unsigned int cpu)
 {
 	xen_init_lock_cpu(cpu);
@@ -121,10 +106,8 @@ int xen_cpuhp_setup(int (*cpu_up_prepare_cb)(unsigned int),
 	return rc >= 0 ? 0 : rc;
 }
 
-static int xen_vcpu_setup_restore(int cpu)
+static void xen_vcpu_setup_restore(int cpu)
 {
-	int rc = 0;
-
 	/* Any per_cpu(xen_vcpu) is stale, so reset it */
 	xen_vcpu_info_reset(cpu);
 
@@ -133,11 +116,8 @@ static int xen_vcpu_setup_restore(int cpu)
 	 * be handled by hotplug.
 	 */
 	if (xen_pv_domain() ||
-	    (xen_hvm_domain() && cpu_online(cpu))) {
-		rc = xen_vcpu_setup(cpu);
-	}
-
-	return rc;
+	    (xen_hvm_domain() && cpu_online(cpu)))
+		xen_vcpu_setup(cpu);
 }
 
 /*
@@ -147,7 +127,7 @@ static int xen_vcpu_setup_restore(int cpu)
  */
 void xen_vcpu_restore(void)
 {
-	int cpu, rc;
+	int cpu;
 
 	for_each_possible_cpu(cpu) {
 		bool other_cpu = (cpu != smp_processor_id());
@@ -167,20 +147,9 @@ void xen_vcpu_restore(void)
 		if (xen_pv_domain() || xen_feature(XENFEAT_hvm_safe_pvclock))
 			xen_setup_runstate_info(cpu);
 
-		rc = xen_vcpu_setup_restore(cpu);
-		if (rc)
-			pr_emerg_once("vcpu restore failed for cpu=%d err=%d. "
-					"System will hang.\n", cpu, rc);
-		/*
-		 * In case xen_vcpu_setup_restore() fails, do not bring up the
-		 * VCPU. This helps us avoid the resulting OOPS when the VCPU
-		 * accesses pvclock_vcpu_time via xen_vcpu (which is NULL.)
-		 * Note that this does not improve the situation much -- now the
-		 * VM hangs instead of OOPSing -- with the VCPUs that did not
-		 * fail, spinning in stop_machine(), waiting for the failed
-		 * VCPUs to come up.
-		 */
-		if (other_cpu && is_up && (rc == 0) &&
+		xen_vcpu_setup_restore(cpu);
+
+		if (other_cpu && is_up &&
 		    HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL))
 			BUG();
 	}
@@ -197,7 +166,7 @@ void xen_vcpu_info_reset(int cpu)
 	}
 }
 
-int xen_vcpu_setup(int cpu)
+void xen_vcpu_setup(int cpu)
 {
 	struct vcpu_register_vcpu_info info;
 	int err;
@@ -218,44 +187,26 @@ int xen_vcpu_setup(int cpu)
 	 */
 	if (xen_hvm_domain()) {
 		if (per_cpu(xen_vcpu, cpu) == &per_cpu(xen_vcpu_info, cpu))
-			return 0;
+			return;
 	}
 
-	if (xen_have_vcpu_info_placement) {
-		vcpup = &per_cpu(xen_vcpu_info, cpu);
-		info.mfn = arbitrary_virt_to_mfn(vcpup);
-		info.offset = offset_in_page(vcpup);
+	vcpup = &per_cpu(xen_vcpu_info, cpu);
+	info.mfn = arbitrary_virt_to_mfn(vcpup);
+	info.offset = offset_in_page(vcpup);
 
-		/*
-		 * Check to see if the hypervisor will put the vcpu_info
-		 * structure where we want it, which allows direct access via
-		 * a percpu-variable.
-		 * N.B. This hypercall can _only_ be called once per CPU.
-		 * Subsequent calls will error out with -EINVAL. This is due to
-		 * the fact that hypervisor has no unregister variant and this
-		 * hypercall does not allow to over-write info.mfn and
-		 * info.offset.
-		 */
-		err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info,
-					 xen_vcpu_nr(cpu), &info);
-
-		if (err) {
-			pr_warn_once("register_vcpu_info failed: cpu=%d err=%d\n",
-				     cpu, err);
-			xen_have_vcpu_info_placement = 0;
-		} else {
-			/*
-			 * This cpu is using the registered vcpu info, even if
-			 * later ones fail to.
-			 */
-			per_cpu(xen_vcpu, cpu) = vcpup;
-		}
-	}
-
-	if (!xen_have_vcpu_info_placement)
-		xen_vcpu_info_reset(cpu);
+	/*
+	 * N.B. This hypercall can _only_ be called once per CPU.
+	 * Subsequent calls will error out with -EINVAL. This is due to
+	 * the fact that hypervisor has no unregister variant and this
+	 * hypercall does not allow to over-write info.mfn and
+	 * info.offset.
+	 */
+	err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, xen_vcpu_nr(cpu),
+				 &info);
+	if (err)
+		panic("register_vcpu_info failed: cpu=%d err=%d\n", cpu, err);
 
-	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
+	per_cpu(xen_vcpu, cpu) = vcpup;
 }
 
 void xen_reboot(int reason)
diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c
index e68ea5f4ad1c..42300941ec29 100644
--- a/arch/x86/xen/enlighten_hvm.c
+++ b/arch/x86/xen/enlighten_hvm.c
@@ -163,9 +163,9 @@ static int xen_cpu_up_prepare_hvm(unsigned int cpu)
 		per_cpu(xen_vcpu_id, cpu) = cpu_acpi_id(cpu);
 	else
 		per_cpu(xen_vcpu_id, cpu) = cpu;
-	rc = xen_vcpu_setup(cpu);
-	if (rc || !xen_have_vector_callback)
-		return rc;
+	xen_vcpu_setup(cpu);
+	if (!xen_have_vector_callback)
+		return 0;
 
 	if (xen_feature(XENFEAT_hvm_safe_pvclock))
 		xen_setup_timer(cpu);
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 349f780a1567..6719963901e5 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1018,31 +1018,13 @@ void __init xen_setup_vcpu_info_placement(void)
 	for_each_possible_cpu(cpu) {
 		/* Set up direct vCPU id mapping for PV guests. */
 		per_cpu(xen_vcpu_id, cpu) = cpu;
-
-		/*
-		 * xen_vcpu_setup(cpu) can fail  -- in which case it
-		 * falls back to the shared_info version for cpus
-		 * where xen_vcpu_nr(cpu) < MAX_VIRT_CPUS.
-		 *
-		 * xen_cpu_up_prepare_pv() handles the rest by failing
-		 * them in hotplug.
-		 */
-		(void) xen_vcpu_setup(cpu);
+		xen_vcpu_setup(cpu);
 	}
 
-	/*
-	 * xen_vcpu_setup managed to place the vcpu_info within the
-	 * percpu area for all cpus, so make use of it.
-	 */
-	if (xen_have_vcpu_info_placement) {
-		pv_ops.irq.save_fl = __PV_IS_CALLEE_SAVE(xen_save_fl_direct);
-		pv_ops.irq.irq_disable =
-			__PV_IS_CALLEE_SAVE(xen_irq_disable_direct);
-		pv_ops.irq.irq_enable =
-			__PV_IS_CALLEE_SAVE(xen_irq_enable_direct);
-		pv_ops.mmu.read_cr2 =
-			__PV_IS_CALLEE_SAVE(xen_read_cr2_direct);
-	}
+	pv_ops.irq.save_fl = __PV_IS_CALLEE_SAVE(xen_save_fl_direct);
+	pv_ops.irq.irq_disable = __PV_IS_CALLEE_SAVE(xen_irq_disable_direct);
+	pv_ops.irq.irq_enable = __PV_IS_CALLEE_SAVE(xen_irq_enable_direct);
+	pv_ops.mmu.read_cr2 = __PV_IS_CALLEE_SAVE(xen_read_cr2_direct);
 }
 
 static const struct pv_info xen_info __initconst = {
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index c1b2f764b29a..bafa61b1482f 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -121,34 +121,10 @@ int xen_smp_intr_init(unsigned int cpu)
 
 void __init xen_smp_cpus_done(unsigned int max_cpus)
 {
-	int cpu, rc, count = 0;
-
 	if (xen_hvm_domain())
 		native_smp_cpus_done(max_cpus);
 	else
 		calculate_max_logical_packages();
-
-	if (xen_have_vcpu_info_placement)
-		return;
-
-	for_each_online_cpu(cpu) {
-		if (xen_vcpu_nr(cpu) < MAX_VIRT_CPUS)
-			continue;
-
-		rc = remove_cpu(cpu);
-
-		if (rc == 0) {
-			/*
-			 * Reset vcpu_info so this cpu cannot be onlined again.
-			 */
-			xen_vcpu_info_reset(cpu);
-			count++;
-		} else {
-			pr_warn("%s: failed to bring CPU %d down, error %d\n",
-				__func__, cpu, rc);
-		}
-	}
-	WARN(count, "%s: brought %d CPUs offline\n", __func__, count);
 }
 
 void xen_smp_send_reschedule(int cpu)
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 8d7ec49a35fb..c2da84484b8d 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -75,9 +75,7 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id);
 
 bool xen_vcpu_stolen(int vcpu);
 
-extern int xen_have_vcpu_info_placement;
-
-int xen_vcpu_setup(int cpu);
+void xen_vcpu_setup(int cpu);
 void xen_vcpu_info_reset(int cpu);
 void xen_setup_vcpu_info_placement(void);
 
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192463.342934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzX1-0002R4-GT; Wed, 22 Sep 2021 10:31:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192463.342934; Wed, 22 Sep 2021 10:31:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzX1-0002Qr-Cm; Wed, 22 Sep 2021 10:31:15 +0000
Received: by outflank-mailman (input) for mailman id 192463;
 Wed, 22 Sep 2021 10:31:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSzWz-0001rn-Tf
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:31:13 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e77e6817-55e9-45f3-ba1c-791dba56408a;
 Wed, 22 Sep 2021 10:31:08 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C57DC201E0;
 Wed, 22 Sep 2021 10:31:07 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6EEA813D69;
 Wed, 22 Sep 2021 10:31:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id gNPdGesFS2GHWAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 10:31:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e77e6817-55e9-45f3-ba1c-791dba56408a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632306667; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=86firgN1jlIPP55LKir2kex+Qi/Crgc5F8hTbnkpj2U=;
	b=kZGArRO73g3w8z8IReMclRkOaaUnVYe0ECsdBZfUOlFhmaUZHgqupIR64JXeM1AHaQXDsA
	CijKASyMoETKc7tdXh1H8YtbxRn7zNVjB1cwhvD/ldCJZSZDDhSIuExXIGRBBu3MMqWqB4
	WjQ9ub50sn99ss24Wydkqxj9LgIgvOY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	virtualization@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org
Cc: peterz@infradead.org,
	Juergen Gross <jgross@suse.com>,
	Deep Shah <sdeep@vmware.com>,
	"VMware, Inc." <pv-drivers@vmware.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v2 2/2] x86/xen: switch initial pvops IRQ functions to dummy ones
Date: Wed, 22 Sep 2021 12:31:02 +0200
Message-Id: <20210922103102.3589-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210922103102.3589-1-jgross@suse.com>
References: <20210922103102.3589-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The initial pvops functions handling irq flags will only ever be called
before interrupts are being enabled.

So make the __init and switch them to be dummy functions:
- xen_save_fl() can always return 0
- xen_irq_disable() is a nop
- xen_irq_enable() can BUG()

Add some generic paravirt functions for that purpose.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/include/asm/paravirt_types.h |  2 +
 arch/x86/kernel/paravirt.c            | 13 +++++-
 arch/x86/xen/enlighten.c              | 19 +--------
 arch/x86/xen/irq.c                    | 61 ++-------------------------
 4 files changed, 20 insertions(+), 75 deletions(-)

diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index d9d6b0203ec4..fc1151e77569 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -577,7 +577,9 @@ void paravirt_leave_lazy_mmu(void);
 void paravirt_flush_lazy_mmu(void);
 
 void _paravirt_nop(void);
+void paravirt_BUG(void);
 u64 _paravirt_ident_64(u64);
+unsigned long paravirt_ret0(void);
 
 #define paravirt_nop	((void *)_paravirt_nop)
 
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 04cafc057bed..4f0ebc46a15d 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -46,6 +46,17 @@ asm (".pushsection .entry.text, \"ax\"\n"
      ".type _paravirt_nop, @function\n\t"
      ".popsection");
 
+/* stub always return ing 0. */
+asm (".pushsection .entry.text, \"ax\"\n"
+     ".global paravirt_ret0\n"
+     "paravirt_ret0:\n\t"
+     "xor %" _ASM_AX ", %" _ASM_AX ";\n\t"
+     "ret\n\t"
+     ".size paravirt_ret0, . - paravirt_ret0\n\t"
+     ".type paravirt_ret0, @function\n\t"
+     ".popsection");
+
+
 void __init default_banner(void)
 {
 	printk(KERN_INFO "Booting paravirtualized kernel on %s\n",
@@ -53,7 +64,7 @@ void __init default_banner(void)
 }
 
 /* Undefined instruction for dealing with missing ops pointers. */
-static void paravirt_BUG(void)
+void paravirt_BUG(void)
 {
 	BUG();
 }
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 8a7bd3f4591e..d96cabe34a01 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -27,25 +27,10 @@ EXPORT_SYMBOL_GPL(hypercall_page);
  * Pointer to the xen_vcpu_info structure or
  * &HYPERVISOR_shared_info->vcpu_info[cpu]. See xen_hvm_init_shared_info
  * and xen_vcpu_setup for details. By default it points to share_info->vcpu_info
- * but if the hypervisor supports VCPUOP_register_vcpu_info then it can point
- * to xen_vcpu_info. The pointer is used in __xen_evtchn_do_upcall to
- * acknowledge pending events.
- * Also more subtly it is used by the patched version of irq enable/disable
- * e.g. xen_irq_enable_direct and xen_iret in PV mode.
- *
- * The desire to be able to do those mask/unmask operations as a single
- * instruction by using the per-cpu offset held in %gs is the real reason
- * vcpu info is in a per-cpu pointer and the original reason for this
- * hypercall.
- *
+ * but during boot it is switched to point to xen_vcpu_info.
+ * The pointer is used in __xen_evtchn_do_upcall to acknowledge pending events.
  */
 DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
-
-/*
- * Per CPU pages used if hypervisor supports VCPUOP_register_vcpu_info
- * hypercall. This can be used both in PV and PVHVM mode. The structure
- * overrides the default per_cpu(xen_vcpu, cpu) value.
- */
 DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info);
 
 /* Linux <-> Xen vCPU id mapping */
diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c
index dfa091d79c2e..ae8537583102 100644
--- a/arch/x86/xen/irq.c
+++ b/arch/x86/xen/irq.c
@@ -24,60 +24,6 @@ void xen_force_evtchn_callback(void)
 	(void)HYPERVISOR_xen_version(0, NULL);
 }
 
-asmlinkage __visible unsigned long xen_save_fl(void)
-{
-	struct vcpu_info *vcpu;
-	unsigned long flags;
-
-	vcpu = this_cpu_read(xen_vcpu);
-
-	/* flag has opposite sense of mask */
-	flags = !vcpu->evtchn_upcall_mask;
-
-	/* convert to IF type flag
-	   -0 -> 0x00000000
-	   -1 -> 0xffffffff
-	*/
-	return (-flags) & X86_EFLAGS_IF;
-}
-PV_CALLEE_SAVE_REGS_THUNK(xen_save_fl);
-
-asmlinkage __visible void xen_irq_disable(void)
-{
-	/* There's a one instruction preempt window here.  We need to
-	   make sure we're don't switch CPUs between getting the vcpu
-	   pointer and updating the mask. */
-	preempt_disable();
-	this_cpu_read(xen_vcpu)->evtchn_upcall_mask = 1;
-	preempt_enable_no_resched();
-}
-PV_CALLEE_SAVE_REGS_THUNK(xen_irq_disable);
-
-asmlinkage __visible void xen_irq_enable(void)
-{
-	struct vcpu_info *vcpu;
-
-	/*
-	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
-	 * cleared, so disable preemption to ensure we check for
-	 * events on the VCPU we are still running on.
-	 */
-	preempt_disable();
-
-	vcpu = this_cpu_read(xen_vcpu);
-	vcpu->evtchn_upcall_mask = 0;
-
-	/* Doesn't matter if we get preempted here, because any
-	   pending event will get dealt with anyway. */
-
-	barrier(); /* unmask then check (avoid races) */
-	if (unlikely(vcpu->evtchn_upcall_pending))
-		xen_force_evtchn_callback();
-
-	preempt_enable();
-}
-PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable);
-
 static void xen_safe_halt(void)
 {
 	/* Blocking includes an implicit local_irq_enable(). */
@@ -95,9 +41,10 @@ static void xen_halt(void)
 }
 
 static const struct pv_irq_ops xen_irq_ops __initconst = {
-	.save_fl = PV_CALLEE_SAVE(xen_save_fl),
-	.irq_disable = PV_CALLEE_SAVE(xen_irq_disable),
-	.irq_enable = PV_CALLEE_SAVE(xen_irq_enable),
+	/* Initial interrupt flag handling only called while interrupts off. */
+	.save_fl = __PV_IS_CALLEE_SAVE(paravirt_ret0),
+	.irq_disable = __PV_IS_CALLEE_SAVE(paravirt_nop),
+	.irq_enable = __PV_IS_CALLEE_SAVE(paravirt_BUG),
 
 	.safe_halt = xen_safe_halt,
 	.halt = xen_halt,
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192461.342912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzWw-0001s6-Qb; Wed, 22 Sep 2021 10:31:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192461.342912; Wed, 22 Sep 2021 10:31:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzWw-0001rz-LY; Wed, 22 Sep 2021 10:31:10 +0000
Received: by outflank-mailman (input) for mailman id 192461;
 Wed, 22 Sep 2021 10:31:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSzWv-0001rn-59
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:31:09 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 56283d78-3dcf-4d95-bc83-5e1c85586042;
 Wed, 22 Sep 2021 10:31:08 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 4FD44201DD;
 Wed, 22 Sep 2021 10:31:07 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AD9CA13D69;
 Wed, 22 Sep 2021 10:31:06 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id D6lWJ+oFS2GHWAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 10:31:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56283d78-3dcf-4d95-bc83-5e1c85586042
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632306667; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=s1H6Hgf+92X5PDjWMvrW0ZAWQ57zgmauDgvkDjzNQEw=;
	b=YYfv4ClZEZvhsiD2N/MbxWAvzX1c20MHJQVGK2jzgdzcUQ4rvnnCJ+VZbLjoNGliQ1Hb30
	8ZdrIT9YZ1uZE6muO7O2brCrMzr8OTocfoMsuQATPSy1psgRUzeaRVcsEAIGE6FyRDvKMv
	2q13TFoSLgeO1fYkP0yDUOsUSWy/oFI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
	x86@kernel.org,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org
Cc: peterz@infradead.org,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Deep Shah <sdeep@vmware.com>,
	"VMware, Inc." <pv-drivers@vmware.com>
Subject: [PATCH v2 0/2] x86/xen: simplify irq pvops
Date: Wed, 22 Sep 2021 12:31:00 +0200
Message-Id: <20210922103102.3589-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The pvops function for Xen PV guests handling the interrupt flag are
much more complex than needed.

With the supported Xen hypervisor versions they can be simplified a
lot, especially by removing the need for disabling preemption.

Juergen Gross (2):
  x86/xen: remove xen_have_vcpu_info_placement flag
  x86/xen: switch initial pvops IRQ functions to dummy ones

 arch/x86/include/asm/paravirt_types.h |   2 +
 arch/x86/kernel/paravirt.c            |  13 ++-
 arch/x86/xen/enlighten.c              | 116 ++++++--------------------
 arch/x86/xen/enlighten_hvm.c          |   6 +-
 arch/x86/xen/enlighten_pv.c           |  28 ++-----
 arch/x86/xen/irq.c                    |  61 +-------------
 arch/x86/xen/smp.c                    |  24 ------
 arch/x86/xen/xen-ops.h                |   4 +-
 8 files changed, 53 insertions(+), 201 deletions(-)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:34:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:34:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192479.342945 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzaT-0003mC-1h; Wed, 22 Sep 2021 10:34:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192479.342945; Wed, 22 Sep 2021 10:34:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzaS-0003m5-U4; Wed, 22 Sep 2021 10:34:48 +0000
Received: by outflank-mailman (input) for mailman id 192479;
 Wed, 22 Sep 2021 10:34:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ck4H=OM=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mSzaR-0003lz-NN
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:34:47 +0000
Received: from mail-pg1-x536.google.com (unknown [2607:f8b0:4864:20::536])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 64b87b1a-26e2-48dd-a137-04ba142fe929;
 Wed, 22 Sep 2021 10:34:45 +0000 (UTC)
Received: by mail-pg1-x536.google.com with SMTP id w8so2202541pgf.5
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 03:34:45 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 e13sm2007664pfi.210.2021.09.22.03.34.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 03:34:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64b87b1a-26e2-48dd-a137-04ba142fe929
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=G3lBTKL9FoS2GeTPW9J8hEJVFPyRUNCbdl3irWEPhFg=;
        b=VoKr+BxCSzT9CO4ryTvDD3G9wZrHUE0sTtMLZK2Lm+TkGGecOgC3OOiab9jGCY9Kll
         e3Nkk6KOEavdFjpqSE8OwPf42LqGMmpIaMjrJkf5FQ1bm4DCzaHMVjrS50k81Z5PYZvm
         LaQ3W++2hMpMzkR8a6WUIhPyrHKyj2ps2MmD7UAxL5yD/CVQu0BrCvgI9ZCx/6AZ2ero
         ZA58OyFlrguXiqRw6p72MTnEgdUXrgF+j0mJ9KKnzlhlbmg5lM2sj1CZeNsTS1IjKv3i
         HdblFX8nN+LqjFQBrI25lHxMdYEM3heJ/vSWPC9owVlrfu6UaOxGQhHTnyLKg4SRLwvw
         7IOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=G3lBTKL9FoS2GeTPW9J8hEJVFPyRUNCbdl3irWEPhFg=;
        b=MvMj8WBy7v4A+WhBeJ+VKtRPU2B9a2J4IEEn/mTCFPMf4iaqjdEd86g7zvoltsQkXS
         kqPOMv0/20SO9BsDwndw++DkeK7wc6ifyB78HaUUbbp3X2lxizO/RZp1hI+esKJIPlqY
         aGibo+4XLThUoV53NAsQQhnHxu6I/wwX7anSfsTWHaceAiJxCPwS0iRxVkX05JslH13H
         sJyzhLnVqhgdAaWNKOFyYPa4lwCDCPpzRhjZb5wZMcCYb9NhlZ+tJ3OE6qE6x4p+SAdW
         OdnlN2TFuyrO62Y9FP0UjYrEYq9wH5TfAJ7hvlr/Kp4cybXpwYs6BPHp1lCce7qxq+SH
         4OxQ==
X-Gm-Message-State: AOAM533kERSheg+/1GS8gqZ3BBEMbp/4lrhOsIS7wT0fZ9uNFZmbNeHH
	c6IjKstUanx6jxzOct2jOPc=
X-Google-Smtp-Source: ABdhPJw45PYU6V+KljQsJMZub2teNBXuKQfiyKdheHtrHWEuFzCoMIpWH2xXA/VT8gGeHXUNrWQ24A==
X-Received: by 2002:a63:561a:: with SMTP id k26mr32545930pgb.144.1632306884424;
        Wed, 22 Sep 2021 03:34:44 -0700 (PDT)
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
To: Christoph Hellwig <hch@lst.de>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>,
 Michael Kelley <mikelley@microsoft.com>, KY Srinivasan <kys@microsoft.com>,
 Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>, "hch@lst.de" <hch@lst.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <43e22b84-7273-4099-42ea-54b06f398650@gmail.com>
Date: Wed, 22 Sep 2021 18:34:29 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Christoph:
     This patch follows your purposal in the previous discussion.
Could you have a look?
     "use vmap_pfn as in the current series.  But in that case I think
     we should get rid of the other mapping created by vmalloc.  I
     though a bit about finding a way to apply the offset in vmalloc
     itself, but I think it would be too invasive to the normal fast
     path.  So the other sub-option would be to allocate the pages
     manually (maybe even using high order allocations to reduce TLB
     pressure) and then remap them(https://lkml.org/lkml/2021/9/2/112)

Otherwise, I merge your previous change for swiotlb into patch 9
“x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM”
You previous change 
link.(http://git.infradead.org/users/hch/misc.git/commit/8248f295928aded3364a1e54a4e0022e93d3610c) 
Please have a look.


Thanks.


On 9/16/2021 12:21 AM, Michael Kelley wrote:
> From: Tianyu Lan <ltykernel@gmail.com>  Sent: Tuesday, September 14, 2021 6:39 AM
>>
>> In Isolation VM, all shared memory with host needs to mark visible
>> to host via hvcall. vmbus_establish_gpadl() has already done it for
>> netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
>> pagebuffer() stills need to be handled. Use DMA API to map/umap
>> these memory during sending/receiving packet and Hyper-V swiotlb
>> bounce buffer dma address will be returned. The swiotlb bounce buffer
>> has been masked to be visible to host during boot up.
>>
>> Allocate rx/tx ring buffer via alloc_pages() in Isolation VM and map
>> these pages via vmap(). After calling vmbus_establish_gpadl() which
>> marks these pages visible to host, unmap these pages to release the
>> virtual address mapped with physical address below shared_gpa_boundary
>> and map them in the extra address space via vmap_pfn().
>>
>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>> ---
>> Change since v4:
>> 	* Allocate rx/tx ring buffer via alloc_pages() in Isolation VM
>> 	* Map pages after calling vmbus_establish_gpadl().
>> 	* set dma_set_min_align_mask for netvsc driver.
>>
>> Change since v3:
>> 	* Add comment to explain why not to use dma_map_sg()
>> 	* Fix some error handle.
>> ---
>>   drivers/net/hyperv/hyperv_net.h   |   7 +
>>   drivers/net/hyperv/netvsc.c       | 287 +++++++++++++++++++++++++++++-
>>   drivers/net/hyperv/netvsc_drv.c   |   1 +
>>   drivers/net/hyperv/rndis_filter.c |   2 +
>>   include/linux/hyperv.h            |   5 +
>>   5 files changed, 296 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
>> index 315278a7cf88..87e8c74398a5 100644
>> --- a/drivers/net/hyperv/hyperv_net.h
>> +++ b/drivers/net/hyperv/hyperv_net.h
>> @@ -164,6 +164,7 @@ struct hv_netvsc_packet {
>>   	u32 total_bytes;
>>   	u32 send_buf_index;
>>   	u32 total_data_buflen;
>> +	struct hv_dma_range *dma_range;
>>   };
>>
>>   #define NETVSC_HASH_KEYLEN 40
>> @@ -1074,6 +1075,8 @@ struct netvsc_device {
>>
>>   	/* Receive buffer allocated by us but manages by NetVSP */
>>   	void *recv_buf;
>> +	struct page **recv_pages;
>> +	u32 recv_page_count;
>>   	u32 recv_buf_size; /* allocated bytes */
>>   	struct vmbus_gpadl recv_buf_gpadl_handle;
>>   	u32 recv_section_cnt;
>> @@ -1082,6 +1085,8 @@ struct netvsc_device {
>>
>>   	/* Send buffer allocated by us */
>>   	void *send_buf;
>> +	struct page **send_pages;
>> +	u32 send_page_count;
>>   	u32 send_buf_size;
>>   	struct vmbus_gpadl send_buf_gpadl_handle;
>>   	u32 send_section_cnt;
>> @@ -1731,4 +1736,6 @@ struct rndis_message {
>>   #define RETRY_US_HI	10000
>>   #define RETRY_MAX	2000	/* >10 sec */
>>
>> +void netvsc_dma_unmap(struct hv_device *hv_dev,
>> +		      struct hv_netvsc_packet *packet);
>>   #endif /* _HYPERV_NET_H */
>> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
>> index 1f87e570ed2b..7d5254bf043e 100644
>> --- a/drivers/net/hyperv/netvsc.c
>> +++ b/drivers/net/hyperv/netvsc.c
>> @@ -20,6 +20,7 @@
>>   #include <linux/vmalloc.h>
>>   #include <linux/rtnetlink.h>
>>   #include <linux/prefetch.h>
>> +#include <linux/gfp.h>
>>
>>   #include <asm/sync_bitops.h>
>>   #include <asm/mshyperv.h>
>> @@ -150,11 +151,33 @@ static void free_netvsc_device(struct rcu_head *head)
>>   {
>>   	struct netvsc_device *nvdev
>>   		= container_of(head, struct netvsc_device, rcu);
>> +	unsigned int alloc_unit;
>>   	int i;
>>
>>   	kfree(nvdev->extension);
>> -	vfree(nvdev->recv_buf);
>> -	vfree(nvdev->send_buf);
>> +
>> +	if (nvdev->recv_pages) {
>> +		alloc_unit = (nvdev->recv_buf_size /
>> +			nvdev->recv_page_count) >> PAGE_SHIFT;
>> +
>> +		vunmap(nvdev->recv_buf);
>> +		for (i = 0; i < nvdev->recv_page_count; i++)
>> +			__free_pages(nvdev->recv_pages[i], alloc_unit);
>> +	} else {
>> +		vfree(nvdev->recv_buf);
>> +	}
>> +
>> +	if (nvdev->send_pages) {
>> +		alloc_unit = (nvdev->send_buf_size /
>> +			nvdev->send_page_count) >> PAGE_SHIFT;
>> +
>> +		vunmap(nvdev->send_buf);
>> +		for (i = 0; i < nvdev->send_page_count; i++)
>> +			__free_pages(nvdev->send_pages[i], alloc_unit);
>> +	} else {
>> +		vfree(nvdev->send_buf);
>> +	}
>> +
>>   	kfree(nvdev->send_section_map);
>>
>>   	for (i = 0; i < VRSS_CHANNEL_MAX; i++) {
>> @@ -330,6 +353,108 @@ int netvsc_alloc_recv_comp_ring(struct netvsc_device *net_device, u32 q_idx)
>>   	return nvchan->mrc.slots ? 0 : -ENOMEM;
>>   }
>>
>> +void *netvsc_alloc_pages(struct page ***pages_array, unsigned int *array_len,
>> +			 unsigned long size)
>> +{
>> +	struct page *page, **pages, **vmap_pages;
>> +	unsigned long pg_count = size >> PAGE_SHIFT;
>> +	int alloc_unit = MAX_ORDER_NR_PAGES;
>> +	int i, j, vmap_page_index = 0;
>> +	void *vaddr;
>> +
>> +	if (pg_count < alloc_unit)
>> +		alloc_unit = 1;
>> +
>> +	/* vmap() accepts page array with PAGE_SIZE as unit while try to
>> +	 * allocate high order pages here in order to save page array space.
>> +	 * vmap_pages[] is used as input parameter of vmap(). pages[] is to
>> +	 * store allocated pages and map them later.
>> +	 */
>> +	vmap_pages = kmalloc_array(pg_count, sizeof(*vmap_pages), GFP_KERNEL);
>> +	if (!vmap_pages)
>> +		return NULL;
>> +
>> +retry:
>> +	*array_len = pg_count / alloc_unit;
>> +	pages = kmalloc_array(*array_len, sizeof(*pages), GFP_KERNEL);
>> +	if (!pages)
>> +		goto cleanup;
>> +
>> +	for (i = 0; i < *array_len; i++) {
>> +		page = alloc_pages(GFP_KERNEL | __GFP_ZERO,
>> +				   get_order(alloc_unit << PAGE_SHIFT));
>> +		if (!page) {
>> +			/* Try allocating small pages if high order pages are not available. */
>> +			if (alloc_unit == 1) {
>> +				goto cleanup;
>> +			} else {
> 
> The "else" clause isn't really needed because of the goto cleanup above.  Then
> the indentation of the code below could be reduced by one level.
> 
>> +				memset(vmap_pages, 0,
>> +				       sizeof(*vmap_pages) * vmap_page_index);
>> +				vmap_page_index = 0;
>> +
>> +				for (j = 0; j < i; j++)
>> +					__free_pages(pages[j], alloc_unit);
>> +
>> +				kfree(pages);
>> +				alloc_unit = 1;
> 
> This is the case where a large enough contiguous physical memory chunk could
> not be found.  But rather than dropping all the way down to single pages,
> would it make sense to try something smaller, but not 1?  For example,
> cut the alloc_unit in half and try again.  But I'm not sure of all the implications.
> 
>> +				goto retry;
>> +			}
>> +		}
>> +
>> +		pages[i] = page;
>> +		for (j = 0; j < alloc_unit; j++)
>> +			vmap_pages[vmap_page_index++] = page++;
>> +	}
>> +
>> +	vaddr = vmap(vmap_pages, vmap_page_index, VM_MAP, PAGE_KERNEL);
>> +	kfree(vmap_pages);
>> +
>> +	*pages_array = pages;
>> +	return vaddr;
>> +
>> +cleanup:
>> +	for (j = 0; j < i; j++)
>> +		__free_pages(pages[i], alloc_unit);
>> +
>> +	kfree(pages);
>> +	kfree(vmap_pages);
>> +	return NULL;
>> +}
>> +
>> +static void *netvsc_map_pages(struct page **pages, int count, int alloc_unit)
>> +{
>> +	int pg_count = count * alloc_unit;
>> +	struct page *page;
>> +	unsigned long *pfns;
>> +	int pfn_index = 0;
>> +	void *vaddr;
>> +	int i, j;
>> +
>> +	if (!pages)
>> +		return NULL;
>> +
>> +	pfns = kcalloc(pg_count, sizeof(*pfns), GFP_KERNEL);
>> +	if (!pfns)
>> +		return NULL;
>> +
>> +	for (i = 0; i < count; i++) {
>> +		page = pages[i];
>> +		if (!page) {
>> +			pr_warn("page is not available %d.\n", i);
>> +			return NULL;
>> +		}
>> +
>> +		for (j = 0; j < alloc_unit; j++) {
>> +			pfns[pfn_index++] = page_to_pfn(page++) +
>> +				(ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
>> +		}
>> +	}
>> +
>> +	vaddr = vmap_pfn(pfns, pg_count, PAGE_KERNEL_IO);
>> +	kfree(pfns);
>> +	return vaddr;
>> +}
>> +
> 
> I think you are proposing this approach to allocating memory for the send
> and receive buffers so that you can avoid having two virtual mappings for
> the memory, per comments from Christop Hellwig.  But overall, the approach
> seems a bit complex and I wonder if it is worth it.  If allocating large contiguous
> chunks of physical memory is successful, then there is some memory savings
> in that the data structures needed to keep track of the physical pages is
> smaller than the equivalent page tables might be.  But if you have to revert
> to allocating individual pages, then the memory savings is reduced.
> 
> Ultimately, the list of actual PFNs has to be kept somewhere.  Another approach
> would be to do the reverse of what hv_map_memory() from the v4 patch
> series does.  I.e., you could do virt_to_phys() on each virtual address that
> maps above VTOM, and subtract out the shared_gpa_boundary to get the
> list of actual PFNs that need to be freed.   This way you don't have two copies
> of the list of PFNs -- one with and one without the shared_gpa_boundary added.
> But it comes at the cost of additional code so that may not be a great idea.
> 
> I think what you have here works, and I don't have a clearly better solution
> at the moment except perhaps to revert to the v4 solution and just have two
> virtual mappings.  I'll keep thinking about it.  Maybe Christop has other
> thoughts.
> 
>>   static int netvsc_init_buf(struct hv_device *device,
>>   			   struct netvsc_device *net_device,
>>   			   const struct netvsc_device_info *device_info)
>> @@ -337,7 +462,7 @@ static int netvsc_init_buf(struct hv_device *device,
>>   	struct nvsp_1_message_send_receive_buffer_complete *resp;
>>   	struct net_device *ndev = hv_get_drvdata(device);
>>   	struct nvsp_message *init_packet;
>> -	unsigned int buf_size;
>> +	unsigned int buf_size, alloc_unit;
>>   	size_t map_words;
>>   	int i, ret = 0;
>>
>> @@ -350,7 +475,14 @@ static int netvsc_init_buf(struct hv_device *device,
>>   		buf_size = min_t(unsigned int, buf_size,
>>   				 NETVSC_RECEIVE_BUFFER_SIZE_LEGACY);
>>
>> -	net_device->recv_buf = vzalloc(buf_size);
>> +	if (hv_isolation_type_snp())
>> +		net_device->recv_buf =
>> +			netvsc_alloc_pages(&net_device->recv_pages,
>> +					   &net_device->recv_page_count,
>> +					   buf_size);
>> +	else
>> +		net_device->recv_buf = vzalloc(buf_size);
>> +
> 
> I wonder if it is necessary to have two different code paths here.  The
> allocating and freeing of the send and receive buffers is not perf
> sensitive, and it seems like netvsc_alloc_pages() could be used
> regardless of whether SNP Isolation is in effect.  To my thinking,
> one code path is better than two code paths unless there's a
> compelling reason to have two.
> 
>>   	if (!net_device->recv_buf) {
>>   		netdev_err(ndev,
>>   			   "unable to allocate receive buffer of size %u\n",
>> @@ -375,6 +507,27 @@ static int netvsc_init_buf(struct hv_device *device,
>>   		goto cleanup;
>>   	}
>>
>> +	if (hv_isolation_type_snp()) {
>> +		alloc_unit = (buf_size / net_device->recv_page_count)
>> +				>> PAGE_SHIFT;
>> +
>> +		/* Unmap previous virtual address and map pages in the extra
>> +		 * address space(above shared gpa boundary) in Isolation VM.
>> +		 */
>> +		vunmap(net_device->recv_buf);
>> +		net_device->recv_buf =
>> +			netvsc_map_pages(net_device->recv_pages,
>> +					 net_device->recv_page_count,
>> +					 alloc_unit);
>> +		if (!net_device->recv_buf) {
>> +			netdev_err(ndev,
>> +				   "unable to allocate receive buffer of size %u\n",
>> +				   buf_size);
>> +			ret = -ENOMEM;
>> +			goto cleanup;
>> +		}
>> +	}
>> +
>>   	/* Notify the NetVsp of the gpadl handle */
>>   	init_packet = &net_device->channel_init_pkt;
>>   	memset(init_packet, 0, sizeof(struct nvsp_message));
>> @@ -456,13 +609,21 @@ static int netvsc_init_buf(struct hv_device *device,
>>   	buf_size = device_info->send_sections * device_info->send_section_size;
>>   	buf_size = round_up(buf_size, PAGE_SIZE);
>>
>> -	net_device->send_buf = vzalloc(buf_size);
>> +	if (hv_isolation_type_snp())
>> +		net_device->send_buf =
>> +			netvsc_alloc_pages(&net_device->send_pages,
>> +					   &net_device->send_page_count,
>> +					   buf_size);
>> +	else
>> +		net_device->send_buf = vzalloc(buf_size);
>> +
>>   	if (!net_device->send_buf) {
>>   		netdev_err(ndev, "unable to allocate send buffer of size %u\n",
>>   			   buf_size);
>>   		ret = -ENOMEM;
>>   		goto cleanup;
>>   	}
>> +
>>   	net_device->send_buf_size = buf_size;
>>
>>   	/* Establish the gpadl handle for this buffer on this
>> @@ -478,6 +639,27 @@ static int netvsc_init_buf(struct hv_device *device,
>>   		goto cleanup;
>>   	}
>>
>> +	if (hv_isolation_type_snp()) {
>> +		alloc_unit = (buf_size / net_device->send_page_count)
>> +				>> PAGE_SHIFT;
>> +
>> +		/* Unmap previous virtual address and map pages in the extra
>> +		 * address space(above shared gpa boundary) in Isolation VM.
>> +		 */
>> +		vunmap(net_device->send_buf);
>> +		net_device->send_buf =
>> +			netvsc_map_pages(net_device->send_pages,
>> +					 net_device->send_page_count,
>> +					 alloc_unit);
>> +		if (!net_device->send_buf) {
>> +			netdev_err(ndev,
>> +				   "unable to allocate receive buffer of size %u\n",
>> +				   buf_size);
>> +			ret = -ENOMEM;
>> +			goto cleanup;
>> +		}
>> +	}
>> +
>>   	/* Notify the NetVsp of the gpadl handle */
>>   	init_packet = &net_device->channel_init_pkt;
>>   	memset(init_packet, 0, sizeof(struct nvsp_message));
>> @@ -768,7 +950,7 @@ static void netvsc_send_tx_complete(struct net_device *ndev,
>>
>>   	/* Notify the layer above us */
>>   	if (likely(skb)) {
>> -		const struct hv_netvsc_packet *packet
>> +		struct hv_netvsc_packet *packet
>>   			= (struct hv_netvsc_packet *)skb->cb;
>>   		u32 send_index = packet->send_buf_index;
>>   		struct netvsc_stats *tx_stats;
>> @@ -784,6 +966,7 @@ static void netvsc_send_tx_complete(struct net_device *ndev,
>>   		tx_stats->bytes += packet->total_bytes;
>>   		u64_stats_update_end(&tx_stats->syncp);
>>
>> +		netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>>   		napi_consume_skb(skb, budget);
>>   	}
>>
>> @@ -948,6 +1131,87 @@ static void netvsc_copy_to_send_buf(struct netvsc_device *net_device,
>>   		memset(dest, 0, padding);
>>   }
>>
>> +void netvsc_dma_unmap(struct hv_device *hv_dev,
>> +		      struct hv_netvsc_packet *packet)
>> +{
>> +	u32 page_count = packet->cp_partial ?
>> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
>> +		packet->page_buf_cnt;
>> +	int i;
>> +
>> +	if (!hv_is_isolation_supported())
>> +		return;
>> +
>> +	if (!packet->dma_range)
>> +		return;
>> +
>> +	for (i = 0; i < page_count; i++)
>> +		dma_unmap_single(&hv_dev->device, packet->dma_range[i].dma,
>> +				 packet->dma_range[i].mapping_size,
>> +				 DMA_TO_DEVICE);
>> +
>> +	kfree(packet->dma_range);
>> +}
>> +
>> +/* netvsc_dma_map - Map swiotlb bounce buffer with data page of
>> + * packet sent by vmbus_sendpacket_pagebuffer() in the Isolation
>> + * VM.
>> + *
>> + * In isolation VM, netvsc send buffer has been marked visible to
>> + * host and so the data copied to send buffer doesn't need to use
>> + * bounce buffer. The data pages handled by vmbus_sendpacket_pagebuffer()
>> + * may not be copied to send buffer and so these pages need to be
>> + * mapped with swiotlb bounce buffer. netvsc_dma_map() is to do
>> + * that. The pfns in the struct hv_page_buffer need to be converted
>> + * to bounce buffer's pfn. The loop here is necessary because the
>> + * entries in the page buffer array are not necessarily full
>> + * pages of data.  Each entry in the array has a separate offset and
>> + * len that may be non-zero, even for entries in the middle of the
>> + * array.  And the entries are not physically contiguous.  So each
>> + * entry must be individually mapped rather than as a contiguous unit.
>> + * So not use dma_map_sg() here.
>> + */
>> +static int netvsc_dma_map(struct hv_device *hv_dev,
>> +		   struct hv_netvsc_packet *packet,
>> +		   struct hv_page_buffer *pb)
>> +{
>> +	u32 page_count =  packet->cp_partial ?
>> +		packet->page_buf_cnt - packet->rmsg_pgcnt :
>> +		packet->page_buf_cnt;
>> +	dma_addr_t dma;
>> +	int i;
>> +
>> +	if (!hv_is_isolation_supported())
>> +		return 0;
>> +
>> +	packet->dma_range = kcalloc(page_count,
>> +				    sizeof(*packet->dma_range),
>> +				    GFP_KERNEL);
>> +	if (!packet->dma_range)
>> +		return -ENOMEM;
>> +
>> +	for (i = 0; i < page_count; i++) {
>> +		char *src = phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
>> +					 + pb[i].offset);
>> +		u32 len = pb[i].len;
>> +
>> +		dma = dma_map_single(&hv_dev->device, src, len,
>> +				     DMA_TO_DEVICE);
>> +		if (dma_mapping_error(&hv_dev->device, dma)) {
>> +			kfree(packet->dma_range);
>> +			return -ENOMEM;
>> +		}
>> +
>> +		packet->dma_range[i].dma = dma;
>> +		packet->dma_range[i].mapping_size = len;
>> +		pb[i].pfn = dma >> HV_HYP_PAGE_SHIFT;
>> +		pb[i].offset = offset_in_hvpage(dma);
> 
> With the DMA min align mask now being set, the offset within
> the Hyper-V page won't be changed by dma_map_single().  So I
> think the above statement can be removed.
> 
>> +		pb[i].len = len;
> 
> A few lines above, the value of "len" is set from pb[i].len.  Neither
> "len" nor "i" is changed in the loop, so this statement can also be
> removed.
> 
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>>   static inline int netvsc_send_pkt(
>>   	struct hv_device *device,
>>   	struct hv_netvsc_packet *packet,
>> @@ -988,14 +1252,24 @@ static inline int netvsc_send_pkt(
>>
>>   	trace_nvsp_send_pkt(ndev, out_channel, rpkt);
>>
>> +	packet->dma_range = NULL;
>>   	if (packet->page_buf_cnt) {
>>   		if (packet->cp_partial)
>>   			pb += packet->rmsg_pgcnt;
>>
>> +		ret = netvsc_dma_map(ndev_ctx->device_ctx, packet, pb);
>> +		if (ret) {
>> +			ret = -EAGAIN;
>> +			goto exit;
>> +		}
>> +
>>   		ret = vmbus_sendpacket_pagebuffer(out_channel,
>>   						  pb, packet->page_buf_cnt,
>>   						  &nvmsg, sizeof(nvmsg),
>>   						  req_id);
>> +
>> +		if (ret)
>> +			netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>>   	} else {
>>   		ret = vmbus_sendpacket(out_channel,
>>   				       &nvmsg, sizeof(nvmsg),
>> @@ -1003,6 +1277,7 @@ static inline int netvsc_send_pkt(
>>   				       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
>>   	}
>>
>> +exit:
>>   	if (ret == 0) {
>>   		atomic_inc_return(&nvchan->queue_sends);
>>
>> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
>> index 382bebc2420d..c3dc884b31e3 100644
>> --- a/drivers/net/hyperv/netvsc_drv.c
>> +++ b/drivers/net/hyperv/netvsc_drv.c
>> @@ -2577,6 +2577,7 @@ static int netvsc_probe(struct hv_device *dev,
>>   	list_add(&net_device_ctx->list, &netvsc_dev_list);
>>   	rtnl_unlock();
>>
>> +	dma_set_min_align_mask(&dev->device, HV_HYP_PAGE_SIZE - 1);
>>   	netvsc_devinfo_put(device_info);
>>   	return 0;
>>
>> diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
>> index f6c9c2a670f9..448fcc325ed7 100644
>> --- a/drivers/net/hyperv/rndis_filter.c
>> +++ b/drivers/net/hyperv/rndis_filter.c
>> @@ -361,6 +361,8 @@ static void rndis_filter_receive_response(struct net_device *ndev,
>>   			}
>>   		}
>>
>> +		netvsc_dma_unmap(((struct net_device_context *)
>> +			netdev_priv(ndev))->device_ctx, &request->pkt);
>>   		complete(&request->wait_event);
>>   	} else {
>>   		netdev_err(ndev,
>> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
>> index c94c534a944e..81e58dd582dc 100644
>> --- a/include/linux/hyperv.h
>> +++ b/include/linux/hyperv.h
>> @@ -1597,6 +1597,11 @@ struct hyperv_service_callback {
>>   	void (*callback)(void *context);
>>   };
>>
>> +struct hv_dma_range {
>> +	dma_addr_t dma;
>> +	u32 mapping_size;
>> +};
>> +
>>   #define MAX_SRV_VER	0x7ffffff
>>   extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *icmsghdrp, u8 *buf, u32 buflen,
>>   				const int *fw_version, int fw_vercnt,
>> --
>> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:46:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:46:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192489.342956 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzlq-0005S1-8D; Wed, 22 Sep 2021 10:46:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192489.342956; Wed, 22 Sep 2021 10:46:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzlq-0005Ru-3Z; Wed, 22 Sep 2021 10:46:34 +0000
Received: by outflank-mailman (input) for mailman id 192489;
 Wed, 22 Sep 2021 10:46:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mSzlo-0005Ro-Hg
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:46:32 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 029b3dc5-3605-4715-a014-fdb8f0e4ca17;
 Wed, 22 Sep 2021 10:46:31 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 85A9B1FD6B;
 Wed, 22 Sep 2021 10:46:30 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4A52F13D69;
 Wed, 22 Sep 2021 10:46:30 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id odObEIYJS2H2YAAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 10:46:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 029b3dc5-3605-4715-a014-fdb8f0e4ca17
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632307590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=xnjyJyHNhItsF8+WIk68P8ksbrihtfulOtnZqaeuDFU=;
	b=O3D76MKm2/yJYcrbjXqdjOF7m9WAxD/sbkoXKtoVnBxc7lHXxda2n4OxzXYutHuQsU2gbc
	gCAcFpCHfO2huB/5atTGPNjz/+8pGu4KwE58D1TfgNBkz2zkt0krbd3rW5o8LQIPOYK1ac
	hrhtRlj4dRLv3+6UFXf/LAGNWXV8K18=
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
To: xen-devel@lists.xenproject.org, x86@kernel.org,
 linux-kernel@vger.kernel.org
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210921070226.32021-1-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <72919edf-8b8c-25f6-ee44-61463f41e188@suse.com>
Date: Wed, 22 Sep 2021 12:46:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210921070226.32021-1-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="tk7Uw5Ml2QvPQThroYrAiWGF7iRiGlEmE"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--tk7Uw5Ml2QvPQThroYrAiWGF7iRiGlEmE
Content-Type: multipart/mixed; boundary="UzaC1WicFCVVaYrpliaphlh34sfFknQWx";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org, x86@kernel.org,
 linux-kernel@vger.kernel.org
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
Message-ID: <72919edf-8b8c-25f6-ee44-61463f41e188@suse.com>
Subject: Re: [PATCH] x86/xen: remove unneeded preempt_disable() from
 xen_irq_enable()
References: <20210921070226.32021-1-jgross@suse.com>
In-Reply-To: <20210921070226.32021-1-jgross@suse.com>

--UzaC1WicFCVVaYrpliaphlh34sfFknQWx
Content-Type: multipart/mixed;
 boundary="------------B7FB63FD456F690A5AC1297F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------B7FB63FD456F690A5AC1297F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 21.09.21 09:02, Juergen Gross wrote:
> Disabling preemption in xen_irq_enable() is not needed. There is no
> risk of missing events due to preemption, as preemption can happen
> only in case an event is being received, which is just the opposite
> of missing an event.
>=20
> Signed-off-by: Juergen Gross <jgross@suse.com>

Please ignore this patch, it is superseded now by

"[PATCH v2 0/2] x86/xen: simplify irq pvops"


Juergen

> ---
>   arch/x86/xen/irq.c | 18 +++++++-----------
>   1 file changed, 7 insertions(+), 11 deletions(-)
>=20
> diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c
> index dfa091d79c2e..ba9b14a97109 100644
> --- a/arch/x86/xen/irq.c
> +++ b/arch/x86/xen/irq.c
> @@ -57,24 +57,20 @@ asmlinkage __visible void xen_irq_enable(void)
>   {
>   	struct vcpu_info *vcpu;
>  =20
> -	/*
> -	 * We may be preempted as soon as vcpu->evtchn_upcall_mask is
> -	 * cleared, so disable preemption to ensure we check for
> -	 * events on the VCPU we are still running on.
> -	 */
> -	preempt_disable();
> -
>   	vcpu =3D this_cpu_read(xen_vcpu);
>   	vcpu->evtchn_upcall_mask =3D 0;
>  =20
> -	/* Doesn't matter if we get preempted here, because any
> -	   pending event will get dealt with anyway. */
> +	/*
> +	 * Now preemption could happen, but this is only possible if an event=

> +	 * was handled, so missing an event due to preemption is not
> +	 * possible at all.
> +	 * The worst possible case is to be preempted and then check events
> +	 * pending on the old vcpu, but this is not problematic.
> +	 */
>  =20
>   	barrier(); /* unmask then check (avoid races) */
>   	if (unlikely(vcpu->evtchn_upcall_pending))
>   		xen_force_evtchn_callback();
> -
> -	preempt_enable();
>   }
>   PV_CALLEE_SAVE_REGS_THUNK(xen_irq_enable);
>  =20
>=20


--------------B7FB63FD456F690A5AC1297F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------B7FB63FD456F690A5AC1297F--

--UzaC1WicFCVVaYrpliaphlh34sfFknQWx--

--tk7Uw5Ml2QvPQThroYrAiWGF7iRiGlEmE
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFLCYUFAwAAAAAACgkQsN6d1ii/Ey+4
4wf/R6dXrtGfTz152XMdP13VtgyhlKk8j7CgiFnPfu3DbHQoI/zUv8DMDe12v8+PbbkmTvBj2X0c
jSkCWxxx3qN356IwmDATYTsM6rWFgEL37nJJgjIg5nBnNrhDC37X1C1k5hW9qYp32lVSKHS2v1Io
rAlZctdZE6NRr4j13yK5ApuVnpy+AId4Lhekt4IyqSWeZyQParYhF+Hyt4vzHb9EFa9whKQIOyaQ
uuFdAiuAMkwSAfScbdSYWC9WvnDa5aDJS9H74R415AA5yZ+TjUrbFF2c6s5CJa2Nqnqmz6NDtacj
6lu9KiXaClcr4TOt92ISKnXSSJxeMktndx+Jq+OmDw==
=2vVr
-----END PGP SIGNATURE-----

--tk7Uw5Ml2QvPQThroYrAiWGF7iRiGlEmE--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 10:47:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 10:47:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192495.342966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzmd-0006Ai-Gv; Wed, 22 Sep 2021 10:47:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192495.342966; Wed, 22 Sep 2021 10:47:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mSzmd-0006Ab-Dw; Wed, 22 Sep 2021 10:47:23 +0000
Received: by outflank-mailman (input) for mailman id 192495;
 Wed, 22 Sep 2021 10:47:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mSzmc-0006AV-HG
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 10:47:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7641d91e-1b92-11ec-b96c-12813bfff9fa;
 Wed, 22 Sep 2021 10:47:21 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-22--k6rtIaEPCmiOWKKucxmKw-1; Wed, 22 Sep 2021 12:47:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5903.eurprd04.prod.outlook.com (2603:10a6:803:e0::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 10:47:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 10:47:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0146.eurprd02.prod.outlook.com (2603:10a6:20b:28d::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Wed, 22 Sep 2021 10:47:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7641d91e-1b92-11ec-b96c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632307640;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jQGcZWHwX8Sm2ZZVAOxECqdLDxWeG4ZvsC1rDlYUo0I=;
	b=ZFjhYLR07yUzEuZnA7CRru84O6lDoFHC6a2pADQwoAcSql3y0GfA7Cz6xPSDh4xFG90+GG
	2QwgaBk2yJZIxIvVXT22WWuI9OyS6p1tCStnr40Lp3nYlvEQPPsrSRF4S5V21RI4vIoCyv
	wJsN1fhlh4BtCR2CiItYKAmhKLNM+ys=
X-MC-Unique: -k6rtIaEPCmiOWKKucxmKw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ovk8Y+g2IzAkVJFRWbi042mReAiQNo6hebmt5b4X65O5ip+5Isa3MGCE3kVofMAMpSLrA3KFbvLYfVfo8odYI7zlNJxCemTOttNI7wyQocLAlN69SbjRK12q/fqi1tv9874l5Tl0iJ6zNjcbaWnN64qRto3TJiHPP1ciEeeMCCkgnojSFTyjW2OCMTu4UXHdBZqGD/hjgyMGcxyYn7hFTSjp/1AmgiXBvFs9t67Vw/D4BYcF3S8MAHMbyT5mtMqr5gPoTGgdjgStjusHt2UfhBpB0H/ov+CRWxnVvTXMSKCTsZX/gn+i2TO1PnCtrj/bU+FRK2EREwPI14N90lKvsg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ynmh/rxar0o+sw0+7TfSuIexXq+IETrYTVj3DFqnmVk=;
 b=RRgJEeq6QX/CCfg36x4012XNFArQCl6xceLJew9pSFU8ysx/Bz3dVLEIutR2J8beAeJoq50AlfiBZlGK0LS9iVkeb6AncCx8i7Qz88alJC3HZY8dWcT3Uz8fm1ALu5xGeP2UpSOA+PEaANO0eDCtGUVaxaf32IO8FuBZwQY+pR3dYtyolRFbAs3TRHsQnhJXqUD+bQD2TjIrq9gnjDBjpRFryD578TSYrutAw3/CYehuvYzzYbJ4MT/Gu44PBySSG4L26zp/YBZDsZgbAGzht0Ec1o8crntcThNMNnaQuPBSVzQuxqam/UdPVJ7Hrn0QvpyymrztWiRUQXnYeoTe7w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: Julien Grall <julien@xen.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
 <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
 <b7947619-3520-174c-f5ec-49564ee048a1@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <474bd675-b42e-3f9f-03d6-f7afc26e10c5@suse.com>
Date: Wed, 22 Sep 2021 12:47:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b7947619-3520-174c-f5ec-49564ee048a1@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0146.eurprd02.prod.outlook.com
 (2603:10a6:20b:28d::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0e6c22f5-da02-4e08-859c-08d97db65885
X-MS-TrafficTypeDiagnostic: VI1PR04MB5903:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5903ED74FEE354F9F698F4DCB3A29@VI1PR04MB5903.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F4lEEYIlhcMirTRRLc24GNkrJvdFp5I1IDhnJCPaVJ4kSuvGFCDYQfOFhgD11oTgs5psDI9+7rFHsFoXH5IHgwFCqTCX+dfAkiOB/RNPGynS25cYn+7wLeQmUUsPMwgWrgtHk2mZMW/8oxitizGNrnrEFSuz+Jevuj5KjSvDyVDR4DPTK3ytUHsITNQoPhKf6yQ+CXj1kP6qNrvBU1SYn/WkiBwxbGy85XQYFzgk24Dsh3kYS7EMEEBqwAHWHwaWBwKqyiFXxpE1DTVnTo9X7+ovVANKVreGuze1ZmkxrU3vzwIqY6D2j/zr1DesD4DIC9RtMbzjzz/uwasmDBa6EDZE9tySw42c3FpiYH4YIFWKn0FpHj6CRhvHa3k6fKrdJsZbyKnzgzE3M0H7D5pgOmqkKn3MFsydpHTcxCohjBOmpnge9lfBX8jlczoMYrY9sGNKoaybRunSIu9dpqjRJngFNcH3iGcDcLnDiDxCaFxGWT6O3AnlNWJL+Uc0FWSVTvg18fRXR4oxs/WDJeDrAaAbgvV5F0KXt8UMZ1BLLrzUnUH/MhtfaeiO60+uyEVKm2PkG1S8+C+eli6tozMfXCMf6khr2WVHp5lygfCvH70YscwkQquGejybRumDt/h5GTlQwqoV0Ej0t+Yu3m/aEBLJzvcHDmhR+lbL2Ym4OXnFVQtu0+PhB93Gd+gD59djmmJ2OcqHCdgCr1q7ZQv7BWetf2PzIgGWBMK7SHQMst9FL3vEqyzunpMAGrDse+9rf+ptrI1X7e+MnIernFD40w==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(956004)(186003)(8936002)(31686004)(4326008)(31696002)(38100700002)(2906002)(54906003)(36756003)(66476007)(66946007)(16576012)(316002)(508600001)(8676002)(66556008)(5660300002)(26005)(2616005)(53546011)(86362001)(83380400001)(6916009)(76704002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?dwQ+FYYdRuTwqS+B+pufpMXSvmuFsP6CFbUsRLdM6ZtKGnVq/e79qmnVQN5W?=
 =?us-ascii?Q?Csbxh1s19CKhp4R9BN88DCUqAIvhMVKVryL5G0l9RV2X5jjQHY1ttQA6nK9k?=
 =?us-ascii?Q?MaoK2UM2gNB8sLPA7J/G5mKoUQS+tkb9ag4E57AZrb82fumfX/1851Hd4EEM?=
 =?us-ascii?Q?eGLOPaFz86+HFICPDiNre7SjhDy/i3xqpLKlpYToEpHX+9aIA3hefrNaDcun?=
 =?us-ascii?Q?oH9tlNs1obdSusf9vC/YXsKQ3ENGui24pnccKuih8fMjlEcefnIkfSUHLJHX?=
 =?us-ascii?Q?1OGl5PKjbREsW+gN7CTlbVQgwMxsS3NArTf7GCIm+5HUaQBBjd8II9cZU/M4?=
 =?us-ascii?Q?td4fISMMMdjybK8KNwhTh6+5t62I3f/udnm64L3uy6Q7vROdwq4uZWxrAUGn?=
 =?us-ascii?Q?V84XKZFK6y5jmRJq9PsP14FV/1ZRjxJkv7MuLL+LrpdVnlT13fXBuOsXytBL?=
 =?us-ascii?Q?4afzbOjcaLtr6f8mYAx6zBTHOzMEdaJxyZ5iwClB0WDvyYzi5wVZEX2o7PPC?=
 =?us-ascii?Q?3iOrkNZsTqWS26rD02yCPa9yWDiOy7JDDzodlLXCvj8BFCMzt6tYyac1GmnX?=
 =?us-ascii?Q?e6g8GVODEEdgJrLz6eIiPG25VFmDVYqhu/aUsrrKLRQGlqY5KlizVKXJ1YYR?=
 =?us-ascii?Q?j45cunxTLh6BD69SnoIIjklL8iZZ6xkWf4z4xpO0xK3eNPSM1KrJGqk79JrI?=
 =?us-ascii?Q?aJPOFm9crif/1LCt/xZuO244zlARHqzpgQVQJJpAeRrIVCfuMzysYmnT288C?=
 =?us-ascii?Q?cMqmscEClK7qOp09ycpNApaFM+hFT5B+r9qaWRAptB8PKR8L8F4nsIRWTLA0?=
 =?us-ascii?Q?/yW0nGhuWPG7DfQoBJjC6DR2+w8leG+e8Oq+vXZImo2Fo1dl6S2H5Fu3TUgF?=
 =?us-ascii?Q?6MBwLYjQ76Cjn23skJuftXRElWvq4Dr2cLLKl9op49cH11TD3u6P55n1Ak2o?=
 =?us-ascii?Q?Nc3WX7lkDAvFjyTXZQA8qdQQAHpp7m3kPHBd41SdqlsvtpeTzng4+okyseJW?=
 =?us-ascii?Q?Wv4FcnHvvfIcEpSQuO/gODsDkNjlIPt3dwd2RQNgHTJnWCQsoovzn93DqNC8?=
 =?us-ascii?Q?2I2xlxkhzLBKKfNeX+myXFh7HXm7DIXT0vgKQrcyh6cjRcC5Y0/4Tf12pbWK?=
 =?us-ascii?Q?lm2wzVoCAccovH2z7A6AOdtvF2XHltXypvZA2RgXQfoTnCXa0jiqpHU39+wB?=
 =?us-ascii?Q?mha6aw0gHyxxAvUIw/POVcC3ykyjrddk5+xvtfNdHDJmAhw0VMVHP5py2UXI?=
 =?us-ascii?Q?Lb4m7E+XCYJCqAqFltcoiILGwiu/MEs9JtEhFLTvaIu8A9PPM1h9FvRpMh0b?=
 =?us-ascii?Q?nr6vuww0Y5qDTrKZw0UOp2GE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e6c22f5-da02-4e08-859c-08d97db65885
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 10:47:17.3340
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tii1DXqcgpP9jOXWcZx6qluYwhMmAvEOdyijHFM/ChbDVH+kexIHagkSbi6pit7yNubiPQmbhtItvJOh+SX++w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5903

On 22.09.2021 12:28, Julien Grall wrote:
> Hi,
>=20
> On 22/09/2021 14:42, Jan Beulich wrote:
>> On 22.09.2021 11:26, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
>>>> On 21.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>>>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>>>>>> On 20.09.2021 12:20, Roger Pau Monn=C3=A9 wrote:
>>>>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>>>>>> --- a/xen/include/asm-arm/grant_table.h
>>>>>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||   =
        \
>>>>>>>
>>>>>>> I'm slightly confused by this checks, don't you need to check for
>>>>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>>>>>> guest_physmap_remove_page?
>>>>>>
>>>>>> Why? It's ogfn which gets passed to the function. And it indeed is t=
he
>>>>>> prior GFN's mapping that we want to remove here.
>>>>>>
>>>>>>> Or assuming that ogfn is not invalid can be used to imply a removal=
?
>>>>>>
>>>>>> That implication can be (and on x86 is) used for the incoming argume=
nt,
>>>>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>>>>>
>>>>> I guess I'm confused due to the ogfn checks done on the Arm side that
>>>>> are not performed on x86. So on Arm you always need to explicitly
>>>>> unhook the previous GFN before attempting to setup a new mapping,
>>>>> while on x86 you only need to do this when it's a removal in order to
>>>>> clear the entry?
>>>>
>>>> The difference isn't with guest_physmap_add_entry() (both x86 and
>>>> Arm only insert a new mapping there), but with
>>>> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
>>>> mappings. And gnttab_map_frame() gets called only from there. This
>>>> is effectively what the first paragraph of the description is about.
>>>
>>> OK, sorry, it wasn't clear to me from the description. Could you
>>> explicitly mention in the description that the removal is moved into
>>> gnttab_set_frame_gfn on Arm in order to cope with the fact that
>>> xenmem_add_to_physmap_one doesn't perform it.
>>
>> Well, it's not really "in order to cope" - that's true for the placement
>> prior to this change as well, so not a justification for the change.
>> Nevertheless I've tried to make this more clear by changing the 1st
>> paragraph to:
>>
>> "Without holding appropriate locks, attempting to remove a prior mapping
>>   of the underlying page is pointless, as the same (or another) mapping
>>   could be re-established by a parallel request on another vCPU. Move th=
e
>>   code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
>>   xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
>>   course this new placement doesn't improve things in any way as far as
>>   the security of grant status frame mappings goes (see XSA-379). Proper
>>   locking would be needed here to allow status frames to be mapped
>>   securely."
>>
>>> TBH I think it would be in our best interest to try to make
>>> xenmem_add_to_physmap_one behave as close as possible between arches.
>>> This discrepancy between x86 and Arm regarding page removal is just
>>> going to bring more trouble in the long term, and hiding the
>>> differences inside gnttab_set_frame_gfn just makes it even more
>>> obscure.
>>
>> Stefano, Julien?
>=20
> This would be ideal as I don't particular like the approach taken in=20
> this patch. But AFAICT, this would require us to implement an M2P. Or is=
=20
> there another way to do it?

For the purpose of just XENMAPSPACE_grant_table the "restricted" M2P
(to just grant table pages), as introduced by an on-list patch, would
do afaict. That wouldn't remove all the differences between the two
implementations, but at least the one affecting the difference in how
gnttab_map_frame() needs to behave.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:21:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:21:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192509.342982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0J2-0002Fc-4T; Wed, 22 Sep 2021 11:20:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192509.342982; Wed, 22 Sep 2021 11:20:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0J2-0002FV-1R; Wed, 22 Sep 2021 11:20:52 +0000
Received: by outflank-mailman (input) for mailman id 192509;
 Wed, 22 Sep 2021 11:20:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT0J0-0002FP-Jj
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:20:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT0J0-0005h3-HX
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:20:50 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT0J0-0006Sn-GT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:20:50 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mT0Iu-00039E-6C; Wed, 22 Sep 2021 12:20:44 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=zI12UCiAWcdhToVdwvm9v56wtLvDQCvSl/0SE7uChVY=; b=BAxII3mfogjFhELlIccXuFqrP7
	kraw14+2GiYxiuBh4Z0i83wEs2Bz04CAZpsdmzD1sbGLwjLgwbxv7Sf9Ng69H7sfedZ+Vn2+opxGd
	E0cbuIJVxpYGs3mww0ahzZRM1V523ynnxDDkDCcon9U8kPctb2KWWx0KHn6aCC4Be9w4=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24907.4491.895219.37845@mariner.uk.xensource.com>
Date: Wed, 22 Sep 2021 12:20:43 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
    Ian Jackson <iwj@xenproject.org>,
    xen-devel@lists.xenproject.org,
    dpsmith@apertussolutions.com
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
References: <osstest-164996-mainreport@xen.org>
	<d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
	<24904.44119.940679.241639@mariner.uk.xensource.com>
	<alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
	<19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> On 22.09.2021 01:38, Stefano Stabellini wrote:
> > On Mon, 20 Sep 2021, Ian Jackson wrote:
> >>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
> >>>
> >>> there even look to be a number of higher order pages available (albeit
> >>> without digging I can't tell what "(C)" means). Nevertheless order-4
> >>> allocations aren't really nice.
> >>
> >> The host history suggests this may possibly be related to a qemu update.
> >>
> >> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
> 
> Stefano - as per some of your investigation detailed further down I
> wonder whether you had seen this part of Ian's reply. (Question of
> course then is how that qemu update had managed to get pushed.)

I looked for bisection results for this failure and

  http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-arm64-arm64-libvirt-xsm.guest-start--debian.repeat.html

it's a heisenbug.  Also, the tests got reorganised slightly as a
side-effect of dropping some i386 tests, so some of these tests are
"new" from osstest's pov, although their content isn't really new.

Unfortunately, with it being a heisenbug, we won't get any useful
bisection results, which would otherwise conclusively tell us which
tree the problem was in.

> >> The grub cfg has this:
> >>
> >>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
> >>
> >> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".
> > 
> > I definitely recommend to increase dom0 memory, especially as I guess
> > the box is going to have a significant amount, far more than 4GB. I
> > would set it to 2GB. Also the syntax on ARM is simpler, so it should be
> > just: dom0_mem=2G
> 
> Ian - I guess that's an adjustment relatively easy to make? I wonder
> though whether we wouldn't want to address the underlying issue first.
> Presumably not, because the fix would likely take quite some time to
> propagate suitably. Yet if not, we will want to have some way of
> verifying that an eventual fix there would have helped here.

It could propagate fairly quickly.  But I'm loathe to make this change
because it seems to me that it would be simply masking the bug.

Notably, when this goes wrong, it seems to happen after the guest has
been started once successfully already.  So there *is* enough
memory...

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:35:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:35:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192516.342992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0XN-0003up-EB; Wed, 22 Sep 2021 11:35:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192516.342992; Wed, 22 Sep 2021 11:35:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0XN-0003ui-BK; Wed, 22 Sep 2021 11:35:41 +0000
Received: by outflank-mailman (input) for mailman id 192516;
 Wed, 22 Sep 2021 11:35:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0XM-0003uc-7F
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:35:40 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 4fea4556-ad03-47a2-a4f0-479e23dddc76;
 Wed, 22 Sep 2021 11:35:37 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 53C2B11B3;
 Wed, 22 Sep 2021 04:35:37 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 209963F719;
 Wed, 22 Sep 2021 04:35:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fea4556-ad03-47a2-a4f0-479e23dddc76
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Paul Durrant <paul@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v2 00/17] PCI devices passthrough on Arm
Date: Wed, 22 Sep 2021 12:34:46 +0100
Message-Id: <cover.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello All,

The purpose of this patch series is to add PCI passthrough support to Xen on
Arm. PCI passthrough support on ARM is the collaboration work between EPAM and
ARM. ARM submitted the partial RFC [1][2] last year to get early feedback. We
tried to fix all the comments and added more features to this patch series.

Working POC with all the features can be found at [3]. Working POC is tested
on x86 so that there will be no regression on x86. Design presentation can be
found at [4]

PCI passthrough support is divided into different patches. This patch series
includes following features: 

Preparatory work to implement the PCI passthrough support for the ARM:
- Refactor MSI code.
- Fixed compilation error when HAS_PCI enabled for ARM.

Discovering PCI Host Bridge in XEN:
- PCI init to initialize the PCI driver.
- PCI host bridge discovery in XEN and map the PCI ECAM configuration space to
  the XEN memory.
- PCI access functions.

Discovering PCI devices:
- To support the PCI passthrough, XEN should be aware of the PCI
  devices.
- Hardware domain is in charge of doing the PCI enumeration and will discover
  the PCI devices and then communicate to the XEN via a hypercall to add the
  PCI devices in XEN.

Enable the existing x86 virtual PCI support for ARM:
- Add VPCI trap handler for each of the PCI device added for config space
  access.
- Register the trap handler in XEN for each of the host bridge PCI ECAM config
  space access.

Emulated PCI device tree node in libxl:
- Create a virtual PCI device tree node in libxl to enable the guest OS to
  discover the virtual PCI during guest boot.

This patch series does not inlcude the following features. Following features
will be send for review in the next version of the patch series once initial
patch series merged.

- VPCI support for DOMU guests (Non-identity mappings guest view of the BARs) 
- Virtual bus topology implementation
- IOMMU related changes (generic, SMMUv2, SMMUv3)
- MSI support for DOMU guests.
- Virual ITS support for DOMU guests

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
[3] https://gitlab.com/rahsingh/xen-integration/-/commits/pci-passthrough-upstream-all
[4] https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf

Oleksandr Andrushchenko (1):
  xen/arm: Add support for Xilinx ZynqMP PCI host controller

Rahul Singh (16):
  xen/pci: Refactor MSI code that implements MSI functionality within
    XEN
  xen/pci: solve compilation error on ARM with HAS_PCI enabled
  xen/arm: solve compilation error on ARM with ACPI && HAS_PCI
  xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
  xen/arm: Add PHYSDEVOP_pci_device_* support for ARM
  xen/device-tree: Add dt_property_read_variable_u32_array helper
  xen/device-tree: Add dt_property_read_u32_array helper
  xen/device-tree: Add dt_get_pci_domain_nr helper
  xen/arm: Add support for PCI init to initialize the PCI driver.
  xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
  xen/arm: PCI host bridge discovery within XEN on ARM
  xen:arm: Implement pci access functions
  xen/arm: Enable the existing x86 virtual PCI support for ARM.
  xen/arm: Transitional change to build HAS_VPCI on ARM.
  arm/libxl: Emulated PCI device tree node in libxl
  xen/arm: Add linux,pci-domain property for hwdom if not available.

 docs/misc/xen-command-line.pandoc   |   7 +
 tools/include/libxl.h               |   6 +
 tools/libs/ctrl/xc_domain.c         |   9 +
 tools/libs/light/libxl_arm.c        | 105 ++++++++++
 tools/libs/light/libxl_create.c     |   3 +
 tools/libs/light/libxl_types.idl    |   1 +
 tools/xl/xl_parse.c                 |   2 +
 xen/arch/arm/Makefile               |   2 +
 xen/arch/arm/device.c               |   2 +
 xen/arch/arm/domain.c               |   6 +-
 xen/arch/arm/domain_build.c         |  19 ++
 xen/arch/arm/pci/Makefile           |   7 +
 xen/arch/arm/pci/ecam.c             |  60 ++++++
 xen/arch/arm/pci/pci-access.c       | 141 +++++++++++++
 xen/arch/arm/pci/pci-host-common.c  | 294 ++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-generic.c |  42 ++++
 xen/arch/arm/pci/pci-host-zynqmp.c  |  58 ++++++
 xen/arch/arm/pci/pci.c              | 101 ++++++++++
 xen/arch/arm/physdev.c              |   5 +-
 xen/arch/arm/vpci.c                 | 102 ++++++++++
 xen/arch/arm/vpci.h                 |  36 ++++
 xen/arch/x86/Kconfig                |   1 +
 xen/arch/x86/physdev.c              |  50 +----
 xen/arch/x86/x86_64/physdev.c       |   4 +-
 xen/common/Makefile                 |   1 +
 xen/common/device_tree.c            |  73 +++++++
 xen/common/domain.c                 |   2 +-
 xen/common/physdev.c                |  87 ++++++++
 xen/drivers/passthrough/Makefile    |   1 +
 xen/drivers/passthrough/arm/iommu.c |   9 +
 xen/drivers/passthrough/msi.c       |  83 ++++++++
 xen/drivers/passthrough/pci.c       |  69 +++----
 xen/drivers/pci/Kconfig             |   4 +
 xen/drivers/vpci/Makefile           |   3 +-
 xen/drivers/vpci/header.c           |   2 +
 xen/include/asm-arm/device.h        |   3 +
 xen/include/asm-arm/domain.h        |   8 +-
 xen/include/asm-arm/hypercall.h     |   2 -
 xen/include/asm-arm/numa.h          |   5 +
 xen/include/asm-arm/pci.h           | 122 +++++++++++-
 xen/include/asm-x86/hypercall.h     |   9 +-
 xen/include/asm-x86/pci.h           |  10 +-
 xen/include/public/arch-arm.h       |  17 ++
 xen/include/public/domctl.h         |   4 +-
 xen/include/xen/device_tree.h       |  74 +++++++
 xen/include/xen/hypercall.h         |   8 +
 xen/include/xen/msi.h               |  43 ++++
 xen/include/xen/pci.h               |   2 +
 xen/xsm/flask/hooks.c               |   8 +-
 49 files changed, 1597 insertions(+), 115 deletions(-)
 create mode 100644 xen/arch/arm/pci/Makefile
 create mode 100644 xen/arch/arm/pci/ecam.c
 create mode 100644 xen/arch/arm/pci/pci-access.c
 create mode 100644 xen/arch/arm/pci/pci-host-common.c
 create mode 100644 xen/arch/arm/pci/pci-host-generic.c
 create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
 create mode 100644 xen/arch/arm/pci/pci.c
 create mode 100644 xen/arch/arm/vpci.c
 create mode 100644 xen/arch/arm/vpci.h
 create mode 100644 xen/common/physdev.c
 create mode 100644 xen/drivers/passthrough/msi.c
 create mode 100644 xen/include/xen/msi.h

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:35:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:35:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192517.343004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0XW-0004EW-QP; Wed, 22 Sep 2021 11:35:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192517.343004; Wed, 22 Sep 2021 11:35:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0XW-0004EL-N0; Wed, 22 Sep 2021 11:35:50 +0000
Received: by outflank-mailman (input) for mailman id 192517;
 Wed, 22 Sep 2021 11:35:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0XV-0004D6-38
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:35:49 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 3aaef6c8-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:35:47 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BB93A11B3;
 Wed, 22 Sep 2021 04:35:46 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BE5783F719;
 Wed, 22 Sep 2021 04:35:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3aaef6c8-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Paul Durrant <paul@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v2 01/17] xen/pci: Refactor MSI code that implements MSI functionality within XEN
Date: Wed, 22 Sep 2021 12:34:47 +0100
Message-Id: <27a7c53a862c0c8dcbe4ecef203d5efeee5746a8.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

On Arm, the initial plan is to only support GICv3 ITS which doesn't
require us to manage the MSIs because the HW will protect against
spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
for ARM.

No functional change intended.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
Change in v2: Fixed minor comments
---
 xen/arch/x86/Kconfig             |  1 +
 xen/drivers/passthrough/Makefile |  1 +
 xen/drivers/passthrough/msi.c    | 83 ++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/pci.c    | 54 +++++----------------
 xen/drivers/pci/Kconfig          |  4 ++
 xen/include/xen/msi.h            | 43 +++++++++++++++++
 xen/xsm/flask/hooks.c            |  8 +--
 7 files changed, 149 insertions(+), 45 deletions(-)
 create mode 100644 xen/drivers/passthrough/msi.c
 create mode 100644 xen/include/xen/msi.h

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 1f83518ee0..b4abfca46f 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -20,6 +20,7 @@ config X86
 	select HAS_NS16550
 	select HAS_PASSTHROUGH
 	select HAS_PCI
+	select HAS_PCI_MSI
 	select HAS_PDX
 	select HAS_SCHED_GRANULARITY
 	select HAS_UBSAN
diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile
index 445690e3e5..a5efa22714 100644
--- a/xen/drivers/passthrough/Makefile
+++ b/xen/drivers/passthrough/Makefile
@@ -7,3 +7,4 @@ obj-y += iommu.o
 obj-$(CONFIG_HAS_PCI) += pci.o
 obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
 obj-$(CONFIG_HAS_PCI) += ats.o
+obj-$(CONFIG_HAS_PCI_MSI) += msi.o
diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.c
new file mode 100644
index 0000000000..ce1a450f6f
--- /dev/null
+++ b/xen/drivers/passthrough/msi.c
@@ -0,0 +1,83 @@
+#include <xen/init.h>
+#include <xen/pci.h>
+#include <xen/msi.h>
+#include <asm/hvm/io.h>
+
+int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
+{
+    int rc;
+
+    if ( pdev->msix )
+    {
+        rc = pci_reset_msix_state(pdev);
+        if ( rc )
+            return rc;
+        msixtbl_init(d);
+    }
+
+    return 0;
+}
+
+int pdev_msi_init(struct pci_dev *pdev)
+{
+    unsigned int pos;
+
+    INIT_LIST_HEAD(&pdev->msi_list);
+
+    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
+                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSI);
+    if ( pos )
+    {
+        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
+
+        pdev->msi_maxvec = multi_msi_capable(ctrl);
+    }
+
+    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
+                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSIX);
+    if ( pos )
+    {
+        struct arch_msix *msix = xzalloc(struct arch_msix);
+        uint16_t ctrl;
+
+        if ( !msix )
+            return -ENOMEM;
+
+        spin_lock_init(&msix->table_lock);
+
+        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
+        msix->nr_entries = msix_table_size(ctrl);
+
+        pdev->msix = msix;
+    }
+
+    return 0;
+}
+
+void pdev_msi_deinit(struct pci_dev *pdev)
+{
+    XFREE(pdev->msix);
+}
+
+void pdev_dump_msi(const struct pci_dev *pdev)
+{
+    const struct msi_desc *msi;
+
+    if ( list_empty(&pdev->msi_list) )
+        return;
+
+    printk(" - MSIs < ");
+    list_for_each_entry ( msi, &pdev->msi_list, list )
+        printk("%d ", msi->irq);
+    printk(">");
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index fc4fa2e5c3..8996403161 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -32,8 +32,8 @@
 #include <xen/softirq.h>
 #include <xen/tasklet.h>
 #include <xen/vpci.h>
+#include <xen/msi.h>
 #include <xsm/xsm.h>
-#include <asm/msi.h>
 #include "ats.h"
 
 struct pci_seg {
@@ -314,6 +314,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
 {
     struct pci_dev *pdev;
     unsigned int pos;
+    int rc;
 
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
         if ( pdev->bus == bus && pdev->devfn == devfn )
@@ -327,35 +328,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     *((u8*) &pdev->bus) = bus;
     *((u8*) &pdev->devfn) = devfn;
     pdev->domain = NULL;
-    INIT_LIST_HEAD(&pdev->msi_list);
-
-    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                              PCI_CAP_ID_MSI);
-    if ( pos )
-    {
-        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
-
-        pdev->msi_maxvec = multi_msi_capable(ctrl);
-    }
 
-    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                              PCI_CAP_ID_MSIX);
-    if ( pos )
+    rc = pdev_msi_init(pdev);
+    if ( rc )
     {
-        struct arch_msix *msix = xzalloc(struct arch_msix);
-        uint16_t ctrl;
-
-        if ( !msix )
-        {
-            xfree(pdev);
-            return NULL;
-        }
-        spin_lock_init(&msix->table_lock);
-
-        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
-        msix->nr_entries = msix_table_size(ctrl);
-
-        pdev->msix = msix;
+        xfree(pdev);
+        return NULL;
     }
 
     list_add(&pdev->alldevs_list, &pseg->alldevs_list);
@@ -449,7 +427,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev)
     }
 
     list_del(&pdev->alldevs_list);
-    xfree(pdev->msix);
+    pdev_msi_deinit(pdev);
     xfree(pdev);
 }
 
@@ -1271,18 +1249,16 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev)
 static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
 {
     struct pci_dev *pdev;
-    struct msi_desc *msi;
 
     printk("==== segment %04x ====\n", pseg->nr);
 
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
     {
-        printk("%pp - %pd - node %-3d - MSIs < ",
+        printk("%pp - %pd - node %-3d",
                &pdev->sbdf, pdev->domain,
                (pdev->node != NUMA_NO_NODE) ? pdev->node : -1);
-        list_for_each_entry ( msi, &pdev->msi_list, list )
-               printk("%d ", msi->irq);
-        printk(">\n");
+        pdev_dump_msi(pdev);
+        printk("\n");
     }
 
     return 0;
@@ -1422,13 +1398,9 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
     ASSERT(pdev && (pdev->domain == hardware_domain ||
                     pdev->domain == dom_io));
 
-    if ( pdev->msix )
-    {
-        rc = pci_reset_msix_state(pdev);
-        if ( rc )
-            goto done;
-        msixtbl_init(d);
-    }
+    rc = pdev_msix_assign(d, pdev);
+    if ( rc )
+        goto done;
 
     pdev->fault.count = 0;
 
diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
index 7da03fa13b..c6a7bc8007 100644
--- a/xen/drivers/pci/Kconfig
+++ b/xen/drivers/pci/Kconfig
@@ -1,3 +1,7 @@
 
 config HAS_PCI
 	bool
+
+config HAS_PCI_MSI
+	bool
+	depends on HAS_PCI
diff --git a/xen/include/xen/msi.h b/xen/include/xen/msi.h
new file mode 100644
index 0000000000..c903d0050c
--- /dev/null
+++ b/xen/include/xen/msi.h
@@ -0,0 +1,43 @@
+#ifndef __XEN_MSI_H_
+#define __XEN_MSI_H_
+
+#include <xen/pci.h>
+
+#ifdef CONFIG_HAS_PCI_MSI
+
+#include <asm/msi.h>
+
+int pdev_msix_assign(struct domain *d, struct pci_dev *pdev);
+int pdev_msi_init(struct pci_dev *pdev);
+void pdev_msi_deinit(struct pci_dev *pdev);
+void pdev_dump_msi(const struct pci_dev *pdev);
+
+#else /* !CONFIG_HAS_PCI_MSI */
+
+static inline int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
+{
+    return 0;
+}
+
+static inline int pdev_msi_init(struct pci_dev *pdev)
+{
+    return 0;
+}
+
+static inline void pdev_msi_deinit(struct pci_dev *pdev) {}
+static inline void pci_cleanup_msi(struct pci_dev *pdev) {}
+static inline void pdev_dump_msi(const struct pci_dev *pdev) {}
+
+#endif /* CONFIG_HAS_PCI_MSI */
+
+#endif /* __XEN_MSI_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 25e87180b4..ea9a12bd71 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -21,7 +21,7 @@
 #include <xen/guest_access.h>
 #include <xen/xenoprof.h>
 #include <xen/iommu.h>
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
 #include <asm/msi.h>
 #endif
 #include <public/xen.h>
@@ -114,7 +114,7 @@ static int get_irq_sid(int irq, u32 *sid, struct avc_audit_data *ad)
         }
         return security_irq_sid(irq, sid);
     }
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     {
         struct irq_desc *desc = irq_to_desc(irq);
         if ( desc->msi_desc && desc->msi_desc->dev ) {
@@ -874,7 +874,7 @@ static int flask_map_domain_pirq (struct domain *d)
 static int flask_map_domain_msi (struct domain *d, int irq, const void *data,
                                  u32 *sid, struct avc_audit_data *ad)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     const struct msi_info *msi = data;
     u32 machine_bdf = (msi->seg << 16) | (msi->bus << 8) | msi->devfn;
 
@@ -940,7 +940,7 @@ static int flask_unmap_domain_pirq (struct domain *d)
 static int flask_unmap_domain_msi (struct domain *d, int irq, const void *data,
                                    u32 *sid, struct avc_audit_data *ad)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     const struct pci_dev *pdev = data;
     u32 machine_bdf = (pdev->seg << 16) | (pdev->bus << 8) | pdev->devfn;
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:36:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:36:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192522.343015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Xj-0004l6-36; Wed, 22 Sep 2021 11:36:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192522.343015; Wed, 22 Sep 2021 11:36:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Xi-0004kz-WA; Wed, 22 Sep 2021 11:36:03 +0000
Received: by outflank-mailman (input) for mailman id 192522;
 Wed, 22 Sep 2021 11:36:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0Xi-0004iM-32
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:36:02 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 4207b5f5-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:36:00 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C56A11B3;
 Wed, 22 Sep 2021 04:36:00 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5C6F63F719;
 Wed, 22 Sep 2021 04:35:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4207b5f5-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 02/17] xen/pci: solve compilation error on ARM with HAS_PCI enabled
Date: Wed, 22 Sep 2021 12:34:48 +0100
Message-Id: <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Compilation error is observed when HAS_PCI is enabled for ARM
architecture.

Add definition for arch_iommu_use_permitted() and
arch_pci_clean_pirqs().

pci.c: In function ‘deassign_device’:
pci.c:849:49: error: implicit declaration of function ‘pci_to_dev’;
did you mean ‘dt_to_dev’? [-Werror=implicit-function-declaration]
            pci_to_dev(pdev));
pci.c:880: undefined reference to `arch_pci_clean_pirqs’
pci.c:1392: undefined reference to `arch_iommu_use_permitted'

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Remove pci_conf_read*(..) dummy implementation
- Add in code comment for arch_pci_clean_pirqs() and arch_iommu_use_permitted()
- Fixed minor comments
---
 xen/arch/arm/Makefile               |  1 +
 xen/arch/arm/pci/Makefile           |  1 +
 xen/arch/arm/pci/pci.c              | 33 +++++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/iommu.c |  9 ++++++++
 xen/include/asm-arm/pci.h           | 31 ++++++++++++++++++++++++---
 5 files changed, 72 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/arm/pci/Makefile
 create mode 100644 xen/arch/arm/pci/pci.c

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 3d3b97b5b4..44d7cc81fa 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) += arm32/
 obj-$(CONFIG_ARM_64) += arm64/
 obj-$(CONFIG_ARM_64) += efi/
 obj-$(CONFIG_ACPI) += acpi/
+obj-$(CONFIG_HAS_PCI) += pci/
 ifneq ($(CONFIG_NO_PLAT),y)
 obj-y += platforms/
 endif
diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
new file mode 100644
index 0000000000..a98035df4c
--- /dev/null
+++ b/xen/arch/arm/pci/Makefile
@@ -0,0 +1 @@
+obj-y += pci.o
diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
new file mode 100644
index 0000000000..a7a7bc3213
--- /dev/null
+++ b/xen/arch/arm/pci/pci.c
@@ -0,0 +1,33 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+
+/*
+ * PIRQ event channels are not supported on Arm, so nothing to do.
+ */
+int arch_pci_clean_pirqs(struct domain *d)
+{
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c
index db3b07a571..ee653a9c48 100644
--- a/xen/drivers/passthrough/arm/iommu.c
+++ b/xen/drivers/passthrough/arm/iommu.c
@@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 {
 }
+
+/*
+ * Unlike x86, Arm doesn't support mem-sharing, mem-paging and log-dirty (yet).
+ * So there is no restriction to use the IOMMU.
+ */
+bool arch_iommu_use_permitted(const struct domain *d)
+{
+    return true;
+}
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index de13359f65..7dd9eb3dba 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -1,7 +1,32 @@
-#ifndef __X86_PCI_H__
-#define __X86_PCI_H__
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
 
+#ifndef __ARM_PCI_H__
+#define __ARM_PCI_H__
+
+#ifdef CONFIG_HAS_PCI
+
+#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
+
+/* Arch pci dev struct */
 struct arch_pci_dev {
+    struct device dev;
 };
 
-#endif /* __X86_PCI_H__ */
+#else   /*!CONFIG_HAS_PCI*/
+
+struct arch_pci_dev { };
+
+#endif  /*!CONFIG_HAS_PCI*/
+#endif /* __ARM_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:36:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:36:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192536.343026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0YA-0005di-DC; Wed, 22 Sep 2021 11:36:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192536.343026; Wed, 22 Sep 2021 11:36:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0YA-0005db-8z; Wed, 22 Sep 2021 11:36:30 +0000
Received: by outflank-mailman (input) for mailman id 192536;
 Wed, 22 Sep 2021 11:36:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0Y8-0005a5-Ll
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:36:28 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 18472106-50f6-4ff6-aecf-475723fd105f;
 Wed, 22 Sep 2021 11:36:27 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 580C511B3;
 Wed, 22 Sep 2021 04:36:27 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 72BB73F719;
 Wed, 22 Sep 2021 04:36:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18472106-50f6-4ff6-aecf-475723fd105f
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH v2 03/17] xen/arm: solve compilation error on ARM with ACPI && HAS_PCI
Date: Wed, 22 Sep 2021 12:34:49 +0100
Message-Id: <af2360805e6d615995a1071575546ecbb011a4d7.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

prelink.o: In function `pcie_aer_get_firmware_first’:
drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'

Compilation error is observed when ACPI and HAS_PCI is enabled for ARM
architecture. APEI not supported on ARM yet move the code under
CONFIG_X86 flag to gate the code for ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Add in code comment "APEI not supported on ARM yet"
---
 xen/drivers/passthrough/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 8996403161..d774a6154e 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1150,7 +1150,8 @@ void __hwdom_init setup_hwdom_pci_devices(
     pcidevs_unlock();
 }
 
-#ifdef CONFIG_ACPI
+/* APEI not supported on ARM yet. */
+#if defined(CONFIG_ACPI) && defined(CONFIG_X86)
 #include <acpi/acpi.h>
 #include <acpi/apei.h>
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:36:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:36:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192543.343037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0YW-0006Dj-Ka; Wed, 22 Sep 2021 11:36:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192543.343037; Wed, 22 Sep 2021 11:36:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0YW-0006Dc-HP; Wed, 22 Sep 2021 11:36:52 +0000
Received: by outflank-mailman (input) for mailman id 192543;
 Wed, 22 Sep 2021 11:36:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0YV-0006DO-IG
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:36:51 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 60b1227e-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:36:50 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7895911B3;
 Wed, 22 Sep 2021 04:36:50 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7726A3F719;
 Wed, 22 Sep 2021 04:36:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60b1227e-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 04/17] xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
Date: Wed, 22 Sep 2021 12:34:50 +0100
Message-Id: <e71044315a6d2a3fb127d62954579390cb837d10.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

ARM architecture does not implement I/O ports. Ignore this call on ARM
to avoid the overhead of making a hypercall just for Xen to return
-ENOSYS.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Instead of returning success in XEN, ignored the call in xl.
---
 tools/libs/ctrl/xc_domain.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index 23322b70b5..25c95f6596 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -1348,6 +1348,14 @@ int xc_domain_ioport_permission(xc_interface *xch,
                                 uint32_t nr_ports,
                                 uint32_t allow_access)
 {
+#if defined(__arm__) || defined(__aarch64__)
+    /*
+     * The ARM architecture does not implement I/O ports.
+     * Avoid the overhead of making a hypercall just for Xen to return -ENOSYS.
+     * It is safe to ignore this call on ARM so we just return 0.
+     */
+    return 0;
+#else
     DECLARE_DOMCTL;
 
     domctl.cmd = XEN_DOMCTL_ioport_permission;
@@ -1357,6 +1365,7 @@ int xc_domain_ioport_permission(xc_interface *xch,
     domctl.u.ioport_permission.allow_access = allow_access;
 
     return do_domctl(xch, &domctl);
+#endif
 }
 
 int xc_availheap(xc_interface *xch,
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:37:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:37:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192550.343048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Yu-0006yD-TX; Wed, 22 Sep 2021 11:37:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192550.343048; Wed, 22 Sep 2021 11:37:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Yu-0006y6-Pd; Wed, 22 Sep 2021 11:37:16 +0000
Received: by outflank-mailman (input) for mailman id 192550;
 Wed, 22 Sep 2021 11:37:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0Ys-0006xk-T3
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:37:14 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 6dea7712-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:37:13 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B83D411B3;
 Wed, 22 Sep 2021 04:37:12 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1806F3F719;
 Wed, 22 Sep 2021 04:37:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6dea7712-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for ARM
Date: Wed, 22 Sep 2021 12:34:51 +0100
Message-Id: <a1c447ced213f07797eebc9471c48df56a187033.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Hardware domain is in charge of doing the PCI enumeration and will
discover the PCI devices and then will communicate to XEN via hyper
call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.

Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.

As most of the code for PHYSDEVOP_pci_device_* is the same between x86
and ARM, move the code to a common file to avoid duplication.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Add support for PHYSDEVOP_pci_device_remove()
- Move code to common code
---
 xen/arch/arm/physdev.c          |  5 +-
 xen/arch/x86/physdev.c          | 50 +-------------------
 xen/arch/x86/x86_64/physdev.c   |  4 +-
 xen/common/Makefile             |  1 +
 xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
 xen/include/asm-arm/hypercall.h |  2 -
 xen/include/asm-arm/numa.h      |  5 ++
 xen/include/asm-x86/hypercall.h |  9 ++--
 xen/include/xen/hypercall.h     |  8 ++++
 9 files changed, 106 insertions(+), 59 deletions(-)
 create mode 100644 xen/common/physdev.c

diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
index e91355fe22..4e00b03aab 100644
--- a/xen/arch/arm/physdev.c
+++ b/xen/arch/arm/physdev.c
@@ -8,10 +8,9 @@
 #include <xen/lib.h>
 #include <xen/errno.h>
 #include <xen/sched.h>
-#include <asm/hypercall.h>
+#include <xen/hypercall.h>
 
-
-int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd);
     return -ENOSYS;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 23465bcd00..c00cc99404 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
 }
 #endif /* COMPAT */
 
-ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     int irq;
     ret_t ret;
@@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         break;
     }
 
-    case PHYSDEVOP_pci_device_add: {
-        struct physdev_pci_device_add add;
-        struct pci_dev_info pdev_info;
-        nodeid_t node;
-
-        ret = -EFAULT;
-        if ( copy_from_guest(&add, arg, 1) != 0 )
-            break;
-
-        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
-        if ( add.flags & XEN_PCI_DEV_VIRTFN )
-        {
-            pdev_info.is_virtfn = 1;
-            pdev_info.physfn.bus = add.physfn.bus;
-            pdev_info.physfn.devfn = add.physfn.devfn;
-        }
-        else
-            pdev_info.is_virtfn = 0;
-
-        if ( add.flags & XEN_PCI_DEV_PXM )
-        {
-            uint32_t pxm;
-            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
-                                sizeof(add.optarr[0]);
-
-            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
-                break;
-
-            node = pxm_to_node(pxm);
-        }
-        else
-            node = NUMA_NO_NODE;
-
-        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
-        break;
-    }
-
-    case PHYSDEVOP_pci_device_remove: {
-        struct physdev_pci_device dev;
-
-        ret = -EFAULT;
-        if ( copy_from_guest(&dev, arg, 1) != 0 )
-            break;
-
-        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
-        break;
-    }
-
     case PHYSDEVOP_prepare_msix:
     case PHYSDEVOP_release_msix: {
         struct physdev_pci_device dev;
diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c
index 0a50cbd4d8..5f72652ff7 100644
--- a/xen/arch/x86/x86_64/physdev.c
+++ b/xen/arch/x86/x86_64/physdev.c
@@ -9,9 +9,10 @@ EMIT_FILE;
 #include <compat/xen.h>
 #include <compat/event_channel.h>
 #include <compat/physdev.h>
-#include <asm/hypercall.h>
+#include <xen/hypercall.h>
 
 #define do_physdev_op compat_physdev_op
+#define arch_physdev_op arch_compat_physdev_op
 
 #define physdev_apic               compat_physdev_apic
 #define physdev_apic_t             physdev_apic_compat_t
@@ -82,6 +83,7 @@ CHECK_physdev_pci_device
 typedef int ret_t;
 
 #include "../physdev.c"
+#include "../../../common/physdev.c"
 
 /*
  * Local variables:
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 54de70d422..bcb1c8fb03 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -29,6 +29,7 @@ obj-y += notifier.o
 obj-y += page_alloc.o
 obj-$(CONFIG_HAS_PDX) += pdx.o
 obj-$(CONFIG_PERF_COUNTERS) += perfc.o
+obj-y += physdev.o
 obj-y += preempt.o
 obj-y += random.o
 obj-y += rangeset.o
diff --git a/xen/common/physdev.c b/xen/common/physdev.c
new file mode 100644
index 0000000000..8d44b20db8
--- /dev/null
+++ b/xen/common/physdev.c
@@ -0,0 +1,81 @@
+
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <xen/init.h>
+
+#ifndef COMPAT
+typedef long ret_t;
+#endif
+
+ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    ret_t ret;
+
+    switch ( cmd )
+    {
+#ifdef CONFIG_HAS_PCI
+    case PHYSDEVOP_pci_device_add: {
+        struct physdev_pci_device_add add;
+        struct pci_dev_info pdev_info;
+        nodeid_t node;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&add, arg, 1) != 0 )
+            break;
+
+        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
+        if ( add.flags & XEN_PCI_DEV_VIRTFN )
+        {
+            pdev_info.is_virtfn = 1;
+            pdev_info.physfn.bus = add.physfn.bus;
+            pdev_info.physfn.devfn = add.physfn.devfn;
+        }
+        else
+            pdev_info.is_virtfn = 0;
+
+        if ( add.flags & XEN_PCI_DEV_PXM )
+        {
+            uint32_t pxm;
+            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
+                                sizeof(add.optarr[0]);
+
+            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
+                break;
+
+            node = pxm_to_node(pxm);
+        }
+        else
+            node = NUMA_NO_NODE;
+
+        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
+        break;
+    }
+
+    case PHYSDEVOP_pci_device_remove: {
+        struct physdev_pci_device dev;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&dev, arg, 1) != 0 )
+            break;
+
+        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
+        break;
+    }
+#endif
+    default:
+        ret = arch_physdev_op(cmd, arg);;
+        break;
+    }
+
+    return ret;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/hypercall.h b/xen/include/asm-arm/hypercall.h
index a0c5a31a2f..c6e4a04fd6 100644
--- a/xen/include/asm-arm/hypercall.h
+++ b/xen/include/asm-arm/hypercall.h
@@ -2,8 +2,6 @@
 #define __ASM_ARM_HYPERCALL_H__
 
 #include <public/domctl.h> /* for arch_do_domctl */
-int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-
 long do_arm_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg);
 
 long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 31a6de4e23..dbd1c2467f 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline nodeid_t pxm_to_node(unsigned pxm)
+{
+    return 0xff;
+}
+
 #endif /* __ARCH_ARM_NUMA_H */
 /*
  * Local variables:
diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
index 0ae3b8b043..7afba98186 100644
--- a/xen/include/asm-x86/hypercall.h
+++ b/xen/include/asm-x86/hypercall.h
@@ -105,10 +105,6 @@ do_update_va_mapping(
     u64 val64,
     unsigned long flags);
 
-extern long
-do_physdev_op(
-    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-
 extern long
 do_update_va_mapping_otherdomain(
     unsigned long va,
@@ -151,6 +147,11 @@ compat_physdev_op(
     int cmd,
     XEN_GUEST_HANDLE_PARAM(void) arg);
 
+extern int
+arch_compat_physdev_op(
+    int cmd,
+    XEN_GUEST_HANDLE_PARAM(void) arg);
+
 extern int
 arch_compat_vcpu_op(
     int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg);
diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 3771487a30..5cc74f40b7 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -45,6 +45,14 @@ extern long
 do_platform_op(
     XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op);
 
+extern long
+do_physdev_op(
+    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
+
+extern long
+arch_physdev_op(
+    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
+
 /*
  * To allow safe resume of do_memory_op() after preemption, we need to know
  * at what point in the page list to resume. For this purpose I steal the
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:37:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:37:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192558.343059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ZE-0007dm-9O; Wed, 22 Sep 2021 11:37:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192558.343059; Wed, 22 Sep 2021 11:37:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ZE-0007df-6H; Wed, 22 Sep 2021 11:37:36 +0000
Received: by outflank-mailman (input) for mailman id 192558;
 Wed, 22 Sep 2021 11:37:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0ZD-0007Zf-5P
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:37:35 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id c222c13e-8bb5-4352-b475-868966b89b92;
 Wed, 22 Sep 2021 11:37:33 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B95711B3;
 Wed, 22 Sep 2021 04:37:33 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8664C3F719;
 Wed, 22 Sep 2021 04:37:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c222c13e-8bb5-4352-b475-868966b89b92
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v2 06/17] xen/device-tree: Add dt_property_read_variable_u32_array helper
Date: Wed, 22 Sep 2021 12:34:52 +0100
Message-Id: <35eb7ec429da3a9e1af626af4521c13c9683f8fd.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

Import the Linux helper of_property_read_variable_u32_array. This
function find and read an array of 32 bit integers from a property,
with bounds on the minimum and maximum array size.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Patch introduced in v2
---
 xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
 xen/include/xen/device_tree.h | 26 +++++++++++++++
 2 files changed, 87 insertions(+)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 03d25a81ce..53160d61f8 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_node *np,
     return 0;
 }
 
+/**
+ * dt_find_property_value_of_size
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @min:    minimum allowed length of property value
+ * @max:    maximum allowed length of property value (0 means unlimited)
+ * @len:    if !=NULL, actual length is written to here
+ *
+ * Search for a property in a device node and valid the requested size.
+ *
+ * Return: The property value on success, -EINVAL if the property does not
+ * exist, -ENODATA if property does not have a value, and -EOVERFLOW if the
+ * property data is too small or too large.
+ */
+static void *dt_find_property_value_of_size(const struct dt_device_node *np,
+                                            const char *propname, u32 min,
+                                            u32 max, size_t *len)
+{
+    const struct dt_property *prop = dt_find_property(np, propname, NULL);
+
+    if ( !prop )
+        return ERR_PTR(-EINVAL);
+    if ( !prop->value )
+        return ERR_PTR(-ENODATA);
+    if ( prop->length < min )
+        return ERR_PTR(-EOVERFLOW);
+    if ( max && prop->length > max )
+        return ERR_PTR(-EOVERFLOW);
+
+    if ( len )
+        *len = prop->length;
+
+    return prop->value;
+}
+
+int dt_property_read_variable_u32_array(const struct dt_device_node *np,
+                                        const char *propname, u32 *out_values,
+                                        size_t sz_min, size_t sz_max)
+{
+    size_t sz, count;
+    const __be32 *val = dt_find_property_value_of_size(np, propname,
+                        (sz_min * sizeof(*out_values)),
+                        (sz_max * sizeof(*out_values)),
+                        &sz);
+
+    if ( IS_ERR(val) )
+        return PTR_ERR(val);
+
+    if ( !sz_max )
+        sz = sz_min;
+    else
+        sz /= sizeof(*out_values);
+
+    count = sz;
+    while ( count-- )
+        *out_values++ = be32_to_cpup(val++);
+
+    return sz;
+}
+
 int dt_property_match_string(const struct dt_device_node *np,
                              const char *propname, const char *string)
 {
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index b02696be94..1693fb8e8c 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
 bool_t dt_property_read_u64(const struct dt_device_node *np,
                             const char *name, u64 *out_value);
 
+
+/**
+ * dt_property_read_variable_u32_array - Find and read an array of 32 bit
+ * integers from a property, with bounds on the minimum and maximum array size.
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @out_values: pointer to return found values.
+ * @sz_min: minimum number of array elements to read
+ * @sz_max: maximum number of array elements to read, if zero there is no
+ *      upper limit on the number of elements in the dts entry but only
+ *      sz_min will be read.
+ *
+ * Search for a property in a device node and read 32-bit value(s) from
+ * it.
+ *
+ * Return: The number of elements read on success, -EINVAL if the property
+ * does not exist, -ENODATA if property does not have a value, and -EOVERFLOW
+ * if the property data is smaller than sz_min or longer than sz_max.
+ *
+ * The out_values is modified only if a valid u32 value can be decoded.
+ */
+int dt_property_read_variable_u32_array(const struct dt_device_node *np,
+                                        const char *propname, u32 *out_values,
+                                        size_t sz_min, size_t sz_max);
+
 /**
  * dt_property_read_bool - Check if a property exists
  * @np: node to get the value
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:38:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:38:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192565.343069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Zb-0008H6-Iw; Wed, 22 Sep 2021 11:37:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192565.343069; Wed, 22 Sep 2021 11:37:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Zb-0008Gz-GA; Wed, 22 Sep 2021 11:37:59 +0000
Received: by outflank-mailman (input) for mailman id 192565;
 Wed, 22 Sep 2021 11:37:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0Za-0008Gc-Bt
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:37:58 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 2e26ecf1-1f2d-4167-9210-37c43a669938;
 Wed, 22 Sep 2021 11:37:57 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1F11011B3;
 Wed, 22 Sep 2021 04:37:57 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 39D573F719;
 Wed, 22 Sep 2021 04:37:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e26ecf1-1f2d-4167-9210-37c43a669938
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v2 07/17] xen/device-tree: Add dt_property_read_u32_array helper
Date: Wed, 22 Sep 2021 12:34:53 +0100
Message-Id: <852610e258d4c36888b68771061afdc0df95e401.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

Import the Linux helper of_property_read_u32_array. This function find
and read an array of 32 bit integers from a property.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Patch introduced in v2
---
 xen/include/xen/device_tree.h | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 1693fb8e8c..497144b8a7 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -366,7 +366,6 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
 bool_t dt_property_read_u64(const struct dt_device_node *np,
                             const char *name, u64 *out_value);
 
-
 /**
  * dt_property_read_variable_u32_array - Find and read an array of 32 bit
  * integers from a property, with bounds on the minimum and maximum array size.
@@ -392,6 +391,36 @@ int dt_property_read_variable_u32_array(const struct dt_device_node *np,
                                         const char *propname, u32 *out_values,
                                         size_t sz_min, size_t sz_max);
 
+/**
+ * dt_property_read_u32_array - Find and read an array of 32 bit integers
+ * from a property.
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @out_values: pointer to return value, modified only if return value is 0.
+ * @sz:     number of array elements to read
+ *
+ * Search for a property in a device node and read 32-bit value(s) from
+ * it.
+ *
+ * Return: 0 on success, -EINVAL if the property does not exist,
+ * -ENODATA if property does not have a value, and -EOVERFLOW if the
+ * property data isn't large enough.
+ *
+ * The out_values is modified only if a valid u32 value can be decoded.
+ */
+static inline int dt_property_read_u32_array(const struct dt_device_node *np,
+                                             const char *propname,
+                                             u32 *out_values, size_t sz)
+{
+    int ret = dt_property_read_variable_u32_array(np, propname, out_values,
+                              sz, 0);
+    if ( ret >= 0 )
+        return 0;
+    else
+        return ret;
+}
+
 /**
  * dt_property_read_bool - Check if a property exists
  * @np: node to get the value
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:38:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:38:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192573.343081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Zv-0000QO-SX; Wed, 22 Sep 2021 11:38:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192573.343081; Wed, 22 Sep 2021 11:38:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0Zv-0000QC-PO; Wed, 22 Sep 2021 11:38:19 +0000
Received: by outflank-mailman (input) for mailman id 192573;
 Wed, 22 Sep 2021 11:38:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0Zu-0008Gc-Az
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:38:18 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 4de2483e-8f72-40e6-8d6c-4f95e8c0774b;
 Wed, 22 Sep 2021 11:38:12 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6D72C11B3;
 Wed, 22 Sep 2021 04:38:12 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 881CE3F719;
 Wed, 22 Sep 2021 04:38:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4de2483e-8f72-40e6-8d6c-4f95e8c0774b
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v2 08/17] xen/device-tree: Add dt_get_pci_domain_nr helper
Date: Wed, 22 Sep 2021 12:34:54 +0100
Message-Id: <b649f9f978ed38f05927573381e23f9b3c6c24b8.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

Import the Linux helper of_get_pci_domain_nr. This function will try to
obtain the host bridge domain number by finding a property called
"linux,pci-domain" of the given device node.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Patch introduced in v2
---
 xen/common/device_tree.c      | 12 ++++++++++++
 xen/include/xen/device_tree.h | 19 +++++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 53160d61f8..ea93da1725 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
     dt_alias_scan();
 }
 
+int dt_get_pci_domain_nr(struct dt_device_node *node)
+{
+    u32 domain;
+    int error;
+
+    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
+    if ( !error )
+        return -EINVAL;
+
+    return (u16)domain;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 497144b8a7..9069040ef7 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -831,6 +831,25 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
                                const char *list_name,
                                const char *cells_name);
 
+/**
+ * dt_get_pci_domain_nr - Find the host bridge domain number
+ *            of the given device node.
+ * @node: Device tree node with the domain information.
+ *
+ * This function will try to obtain the host bridge domain number by finding
+ * a property called "linux,pci-domain" of the given device node.
+ *
+ * Return:
+ * * > 0    - On success, an associated domain number.
+ * * -EINVAL    - The property "linux,pci-domain" does not exist.
+ * * -ENODATA   - The linux,pci-domain" property does not have value.
+ * * -EOVERFLOW - Invalid "linux,pci-domain" property value.
+ *
+ * Returns the associated domain number from DT in the range [0-0xffff], or
+ * a negative value if the required property is not found.
+ */
+int dt_get_pci_domain_nr(struct dt_device_node *node);
+
 #ifdef CONFIG_DEVICE_TREE_DEBUG
 #define dt_dprintk(fmt, args...)  \
     printk(XENLOG_DEBUG fmt, ## args)
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:38:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:38:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192581.343091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0aT-00019a-4W; Wed, 22 Sep 2021 11:38:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192581.343091; Wed, 22 Sep 2021 11:38:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0aT-00019R-1g; Wed, 22 Sep 2021 11:38:53 +0000
Received: by outflank-mailman (input) for mailman id 192581;
 Wed, 22 Sep 2021 11:38:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0aR-000197-LW
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:38:51 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id a836ae52-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:38:50 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7E0A911B3;
 Wed, 22 Sep 2021 04:38:50 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7DFC93F719;
 Wed, 22 Sep 2021 04:38:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a836ae52-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 09/17] xen/arm: Add support for PCI init to initialize the PCI driver.
Date: Wed, 22 Sep 2021 12:34:55 +0100
Message-Id: <000832623dc7fb429db4b4517583f16affdba35b.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

pci_init(..) will be called during xen startup to initialize and probe
the PCI host-bridge driver.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- ACPI init function to return int
- pci_segments_init() called before dt/acpi init
---
 xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/device.h |  1 +
 2 files changed, 55 insertions(+)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index a7a7bc3213..71fa532842 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -12,6 +12,10 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/acpi.h>
+#include <xen/device_tree.h>
+#include <xen/errno.h>
+#include <xen/init.h>
 #include <xen/pci.h>
 
 /*
@@ -22,6 +26,56 @@ int arch_pci_clean_pirqs(struct domain *d)
     return 0;
 }
 
+static int __init dt_pci_init(void)
+{
+    struct dt_device_node *np;
+    int rc;
+
+    dt_for_each_device_node(dt_host, np)
+    {
+        rc = device_init(np, DEVICE_PCI, NULL);
+        if( !rc )
+            continue;
+        /*
+         * Ignore the following error codes:
+         *   - EBADF: Indicate the current is not an pci
+         *   - ENODEV: The pci device is not present or cannot be used by
+         *     Xen.
+         */
+        else if ( rc != -EBADF && rc != -ENODEV )
+        {
+            printk(XENLOG_ERR "No driver found in XEN or driver init error.\n");
+            return rc;
+        }
+    }
+
+    return 0;
+}
+
+#ifdef CONFIG_ACPI
+static int __init acpi_pci_init(void)
+{
+    printk(XENLOG_ERR "ACPI pci init not supported \n");
+    return 0;
+}
+#else
+static inline int __init acpi_pci_init(void)
+{
+    return -EINVAL;
+}
+#endif
+
+static int __init pci_init(void)
+{
+    pci_segments_init();
+
+    if ( acpi_disabled )
+        return dt_pci_init();
+    else
+        return acpi_pci_init();
+}
+__initcall(pci_init);
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index ee7cff2d44..5ecd5e7bd1 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -34,6 +34,7 @@ enum device_class
     DEVICE_SERIAL,
     DEVICE_IOMMU,
     DEVICE_GIC,
+    DEVICE_PCI,
     /* Use for error */
     DEVICE_UNKNOWN,
 };
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:39:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:39:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192589.343103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ay-0001lN-En; Wed, 22 Sep 2021 11:39:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192589.343103; Wed, 22 Sep 2021 11:39:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ay-0001lG-B1; Wed, 22 Sep 2021 11:39:24 +0000
Received: by outflank-mailman (input) for mailman id 192589;
 Wed, 22 Sep 2021 11:39:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0ax-0001l8-DW
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:39:23 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id bb17faf8-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:39:22 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 421DE11B3;
 Wed, 22 Sep 2021 04:39:22 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5CE9E3F719;
 Wed, 22 Sep 2021 04:39:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb17faf8-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 10/17] xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
Date: Wed, 22 Sep 2021 12:34:56 +0100
Message-Id: <9dcb9b3b6b6923db00d6e56da26a8503d5a4855a.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Add cmdline boot option "pci-passthrough = = <boolean>" to enable
disable the PCI passthrough support on ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Add option in xen-command-line.pandoc
- Change pci option to pci-passthrough
- modify option from custom_param to boolean param
---
 docs/misc/xen-command-line.pandoc |  7 +++++++
 xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
 xen/common/physdev.c              |  6 ++++++
 xen/include/asm-arm/pci.h         | 13 +++++++++++++
 xen/include/asm-x86/pci.h         |  8 ++++++++
 5 files changed, 48 insertions(+)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index b175645fde..c867f1cf58 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1783,6 +1783,13 @@ All numbers specified must be hexadecimal ones.
 
 This option can be specified more than once (up to 8 times at present).
 
+### pci-passthrough (arm)
+> `= <boolean>`
+
+> Default: `false`
+
+Flag to enable or disable support for PCI passthrough
+
 ### pcid (x86)
 > `= <boolean> | xpti=<bool>`
 
diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index 71fa532842..fe96a9b135 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -16,6 +16,7 @@
 #include <xen/device_tree.h>
 #include <xen/errno.h>
 #include <xen/init.h>
+#include <xen/param.h>
 #include <xen/pci.h>
 
 /*
@@ -65,8 +66,21 @@ static inline int __init acpi_pci_init(void)
 }
 #endif
 
+/*
+ * By default pci passthrough is disabled
+ */
+bool_t __read_mostly pci_passthrough_enabled = 0;
+boolean_param("pci-passthrough", pci_passthrough_enabled);
+
 static int __init pci_init(void)
 {
+    /*
+     * Enable PCI passthrough when has been enabled explicitly
+     * (pci-passthrough=on)
+     */
+    if ( !pci_passthrough_enabled)
+        return 0;
+
     pci_segments_init();
 
     if ( acpi_disabled )
diff --git a/xen/common/physdev.c b/xen/common/physdev.c
index 8d44b20db8..7390d5d584 100644
--- a/xen/common/physdev.c
+++ b/xen/common/physdev.c
@@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         struct pci_dev_info pdev_info;
         nodeid_t node;
 
+        if ( !is_pci_passthrough_enabled() )
+            return -ENOSYS;
+
         ret = -EFAULT;
         if ( copy_from_guest(&add, arg, 1) != 0 )
             break;
@@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_pci_device_remove: {
         struct physdev_pci_device dev;
 
+        if ( !is_pci_passthrough_enabled() )
+            return -ENOSYS;
+
         ret = -EFAULT;
         if ( copy_from_guest(&dev, arg, 1) != 0 )
             break;
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 7dd9eb3dba..f2f86be9bc 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -19,14 +19,27 @@
 
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
 
+extern bool_t pci_passthrough_enabled;
+
 /* Arch pci dev struct */
 struct arch_pci_dev {
     struct device dev;
 };
 
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return pci_passthrough_enabled;
+}
 #else   /*!CONFIG_HAS_PCI*/
 
+#define pci_passthrough_enabled (false)
+
 struct arch_pci_dev { };
 
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return false;
+}
+
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
index cc05045e9c..0e160c6d01 100644
--- a/xen/include/asm-x86/pci.h
+++ b/xen/include/asm-x86/pci.h
@@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned int *seg,
 extern int pci_mmcfg_config_num;
 extern struct acpi_mcfg_allocation *pci_mmcfg_config;
 
+/*
+ * Unlike ARM, PCI passthrough always enabled for x86.
+ */
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return true;
+}
+
 #endif /* __X86_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:39:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:39:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192593.343114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bL-0002JU-NN; Wed, 22 Sep 2021 11:39:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192593.343114; Wed, 22 Sep 2021 11:39:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bL-0002JL-Jp; Wed, 22 Sep 2021 11:39:47 +0000
Received: by outflank-mailman (input) for mailman id 192593;
 Wed, 22 Sep 2021 11:39:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0bK-0002Ir-9B
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:39:46 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id c8406eae-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:39:44 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 57E1711B3;
 Wed, 22 Sep 2021 04:39:44 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A0E53F719;
 Wed, 22 Sep 2021 04:39:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8406eae-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on ARM
Date: Wed, 22 Sep 2021 12:34:57 +0100
Message-Id: <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XEN during boot will read the PCI device tree node “reg” property
and will map the PCI config space to the XEN memory.

As of now only "pci-host-ecam-generic" compatible board is supported.

"linux,pci-domain" device tree property assigns a fixed PCI domain
number to a host bridge, otherwise an unstable (across boots) unique
number will be assigned by Linux. XEN access the PCI devices based on
Segment:Bus:Device:Function. Segment number in XEN is same as domain
number in Linux.Segment number and domain number has to be in sync
to access the correct PCI devices.

XEN will read the “linux,pci-domain” property from the device tree node
and configure the host bridge segment number accordingly. If this
property is not available XEN will allocate the unique segment number
to the host bridge.

dt_pci_bus_find_domain_nr(..) imported from the Linux source tree with
slight modification based on tag Linux v5.14.2
commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Add more info in commit msg
- Add callback to parse register index.
- Merge patch pci_ecam_operation into this patch to avoid confusion
- Add new struct in struct device for match table
---
 xen/arch/arm/device.c               |   2 +
 xen/arch/arm/pci/Makefile           |   5 +
 xen/arch/arm/pci/ecam.c             |  60 +++++++
 xen/arch/arm/pci/pci-access.c       |  83 +++++++++
 xen/arch/arm/pci/pci-host-common.c  | 254 ++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-generic.c |  42 +++++
 xen/include/asm-arm/device.h        |   2 +
 xen/include/asm-arm/pci.h           |  59 +++++++
 8 files changed, 507 insertions(+)
 create mode 100644 xen/arch/arm/pci/ecam.c
 create mode 100644 xen/arch/arm/pci/pci-access.c
 create mode 100644 xen/arch/arm/pci/pci-host-common.c
 create mode 100644 xen/arch/arm/pci/pci-host-generic.c

diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 70cd6c1a19..197bb3c6e8 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -44,6 +44,8 @@ int __init device_init(struct dt_device_node *dev, enum device_class class,
         {
             ASSERT(desc->init != NULL);
 
+            dev->dev.of_match_table = desc->dt_match;
+
             return desc->init(dev, data);
         }
 
diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
index a98035df4c..e86f2b46fd 100644
--- a/xen/arch/arm/pci/Makefile
+++ b/xen/arch/arm/pci/Makefile
@@ -1 +1,6 @@
 obj-y += pci.o
+obj-y += pci-access.o
+obj-y += pci.o
+obj-y += pci-host-generic.o
+obj-y += pci-host-common.o
+obj-y += ecam.o
diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
new file mode 100644
index 0000000000..9b88b1ceda
--- /dev/null
+++ b/xen/arch/arm/pci/ecam.c
@@ -0,0 +1,60 @@
+/*
+ * Based on Linux drivers/pci/ecam.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+#include <xen/sched.h>
+
+/*
+ * Function to implement the pci_ops ->map_bus method.
+ */
+void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
+                               uint32_t sbdf, uint32_t where)
+{
+    const struct pci_config_window *cfg = bridge->cfg;
+    const struct pci_ecam_ops *ops = bridge->sysdata;
+    unsigned int devfn_shift = ops->bus_shift - 8;
+    void __iomem *base;
+
+    unsigned int busn = PCI_BUS(sbdf);
+
+    if ( busn < cfg->busn_start || busn > cfg->busn_end )
+        return NULL;
+
+    busn -= cfg->busn_start;
+    base = cfg->win + (busn << ops->bus_shift);
+
+    return base + (PCI_DEVFN2(sbdf) << devfn_shift) + where;
+}
+
+/* ECAM ops */
+const struct pci_ecam_ops pci_generic_ecam_ops = {
+    .bus_shift  = 20,
+    .pci_ops    = {
+        .map_bus                = pci_ecam_map_bus,
+        .read                   = pci_generic_config_read,
+        .write                  = pci_generic_config_write,
+    }
+};
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
new file mode 100644
index 0000000000..04fe9fbf92
--- /dev/null
+++ b/xen/arch/arm/pci/pci-access.c
@@ -0,0 +1,83 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+#include <asm/io.h>
+
+#define INVALID_VALUE (~0U)
+
+int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t *value)
+{
+    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+
+    if ( !addr )
+    {
+        *value = INVALID_VALUE;
+        return -ENODEV;
+    }
+
+    switch ( len )
+    {
+    case 1:
+        *value = readb(addr);
+        break;
+    case 2:
+        *value = readw(addr);
+        break;
+    case 4:
+        *value = readl(addr);
+        break;
+    default:
+        ASSERT_UNREACHABLE();
+    }
+
+    return 0;
+}
+
+int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t value)
+{
+    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+
+    if ( !addr )
+        return -ENODEV;
+
+    switch ( len )
+    {
+    case 1:
+        writeb(value, addr);
+        break;
+    case 2:
+        writew(value, addr);
+        break;
+    case 4:
+        writel(value, addr);
+        break;
+    default:
+        ASSERT_UNREACHABLE();
+    }
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
new file mode 100644
index 0000000000..4beec14f2f
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -0,0 +1,254 @@
+/*
+ * Based on Linux drivers/pci/ecam.c
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/init.h>
+#include <xen/pci.h>
+#include <xen/rwlock.h>
+#include <xen/sched.h>
+#include <xen/vmap.h>
+
+/*
+ * List for all the pci host bridges.
+ */
+
+static LIST_HEAD(pci_host_bridges);
+
+static atomic_t domain_nr = ATOMIC_INIT(-1);
+
+static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
+{
+    return ioremap_nocache(start, len);
+}
+
+static void pci_ecam_free(struct pci_config_window *cfg)
+{
+    if ( cfg->win )
+        iounmap(cfg->win);
+
+    xfree(cfg);
+}
+
+static struct pci_config_window * __init
+gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
+{
+    int err, cfg_reg_idx;
+    u32 bus_range[2];
+    paddr_t addr, size;
+    struct pci_config_window *cfg;
+
+    cfg = xzalloc(struct pci_config_window);
+    if ( !cfg )
+        return NULL;
+
+    err = dt_property_read_u32_array(dev, "bus-range", bus_range,
+                                     ARRAY_SIZE(bus_range));
+    if ( err ) {
+        cfg->busn_start = 0;
+        cfg->busn_end = 0xff;
+        printk(XENLOG_INFO "%s: No bus range found for pci controller\n",
+               dt_node_full_name(dev));
+    } else {
+        cfg->busn_start = bus_range[0];
+        cfg->busn_end = bus_range[1];
+        if ( cfg->busn_end > cfg->busn_start + 0xff )
+            cfg->busn_end = cfg->busn_start + 0xff;
+    }
+
+    if ( ops->cfg_reg_index )
+    {
+        cfg_reg_idx = ops->cfg_reg_index(dev);
+        if ( cfg_reg_idx < 0 )
+            goto err_exit;
+    }
+    else
+        cfg_reg_idx = 0;
+
+    /* Parse our PCI ecam register address */
+    err = dt_device_get_address(dev, cfg_reg_idx, &addr, &size);
+    if ( err )
+        goto err_exit;
+
+    cfg->phys_addr = addr;
+    cfg->size = size;
+
+    /*
+     * On 64-bit systems, we do a single ioremap for the whole config space
+     * since we have enough virtual address range available.  On 32-bit, we
+     * ioremap the config space for each bus individually.
+     * As of now only 64-bit is supported 32-bit is not supported.
+     *
+     * TODO: For 32-bit implement the ioremap/iounmap of config space
+     * dynamically for each read/write call.
+     */
+    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
+    if ( !cfg->win )
+        goto err_exit_remap;
+
+    printk("ECAM at [mem 0x%"PRIpaddr"-0x%"PRIpaddr"] for [bus %x-%x] \n",
+            cfg->phys_addr, cfg->phys_addr + cfg->size - 1,
+            cfg->busn_start, cfg->busn_end);
+
+    if ( ops->init )
+    {
+        err = ops->init(cfg);
+        if (err)
+            goto err_exit;
+    }
+
+    return cfg;
+
+err_exit_remap:
+    printk(XENLOG_ERR "ECAM ioremap failed\n");
+err_exit:
+    pci_ecam_free(cfg);
+
+    return NULL;
+}
+
+struct pci_host_bridge *pci_alloc_host_bridge(void)
+{
+    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
+
+    if ( !bridge )
+        return NULL;
+
+    INIT_LIST_HEAD(&bridge->node);
+    bridge->bus_start = UINT8_MAX;
+    bridge->bus_end = UINT8_MAX;
+
+    return bridge;
+}
+
+void pci_add_host_bridge(struct pci_host_bridge *bridge)
+{
+    list_add_tail(&bridge->node, &pci_host_bridges);
+}
+
+static int pci_get_new_domain_nr(void)
+{
+    return atomic_inc_return(&domain_nr);
+}
+
+static int pci_bus_find_domain_nr(struct dt_device_node *dev)
+{
+    static int use_dt_domains = -1;
+    int domain;
+
+    domain = dt_get_pci_domain_nr(dev);
+
+    /*
+     * Check DT domain and use_dt_domains values.
+     *
+     * If DT domain property is valid (domain >= 0) and
+     * use_dt_domains != 0, the DT assignment is valid since this means
+     * we have not previously allocated a domain number by using
+     * pci_get_new_domain_nr(); we should also update use_dt_domains to
+     * 1, to indicate that we have just assigned a domain number from
+     * DT.
+     *
+     * If DT domain property value is not valid (ie domain < 0), and we
+     * have not previously assigned a domain number from DT
+     * (use_dt_domains != 1) we should assign a domain number by
+     * using the:
+     *
+     * pci_get_new_domain_nr()
+     *
+     * API and update the use_dt_domains value to keep track of method we
+     * are using to assign domain numbers (use_dt_domains = 0).
+     *
+     * All other combinations imply we have a platform that is trying
+     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
+     * which is a recipe for domain mishandling and it is prevented by
+     * invalidating the domain value (domain = -1) and printing a
+     * corresponding error.
+     */
+    if ( domain >= 0 && use_dt_domains )
+    {
+        use_dt_domains = 1;
+    }
+    else if ( domain < 0 && use_dt_domains != 1 )
+    {
+        use_dt_domains = 0;
+        domain = pci_get_new_domain_nr();
+    }
+    else
+    {
+        domain = -1;
+    }
+
+    return domain;
+}
+
+int pci_host_common_probe(struct dt_device_node *dev, const void *data)
+{
+    struct pci_host_bridge *bridge;
+    struct pci_config_window *cfg;
+    struct pci_ecam_ops *ops;
+    const struct dt_device_match *of_id;
+    int err;
+
+    if ( dt_device_for_passthrough(dev) )
+        return 0;
+
+    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
+    ops = (struct pci_ecam_ops *) of_id->data;
+
+    bridge = pci_alloc_host_bridge();
+    if ( !bridge )
+        return -ENOMEM;
+
+    /* Parse and map our Configuration Space windows */
+    cfg = gen_pci_init(dev, ops);
+    if ( !cfg )
+    {
+        err = -ENOMEM;
+        goto err_exit;
+    }
+
+    bridge->dt_node = dev;
+    bridge->cfg = cfg;
+    bridge->sysdata = ops;
+    bridge->ops = &ops->pci_ops;
+    bridge->bus_start = cfg->busn_start;
+    bridge->bus_end = cfg->busn_end;
+
+    bridge->segment = pci_bus_find_domain_nr(dev);
+    if ( bridge->segment < 0 )
+    {
+        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
+        BUG();
+    }
+    pci_add_host_bridge(bridge);
+
+    return 0;
+
+err_exit:
+    xfree(bridge);
+
+    return err;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
new file mode 100644
index 0000000000..6b3288d6f3
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -0,0 +1,42 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/device.h>
+#include <xen/pci.h>
+#include <asm/pci.h>
+
+static const struct dt_device_match gen_pci_dt_match[] = {
+    { .compatible = "pci-host-ecam-generic",
+      .data =       &pci_generic_ecam_ops },
+
+    { },
+};
+
+DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
+.dt_match = gen_pci_dt_match,
+.init = pci_host_common_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 5ecd5e7bd1..582119c31e 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -16,6 +16,8 @@ struct device
     enum device_type type;
 #ifdef CONFIG_HAS_DEVICE_TREE
     struct dt_device_node *of_node; /* Used by drivers imported from Linux */
+    /* Used by drivers imported from Linux */
+    const struct dt_device_match *of_match_table;
 #endif
     struct dev_archdata archdata;
     struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index f2f86be9bc..4b32c7088a 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -26,6 +26,65 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/*
+ * struct to hold the mappings of a config space window. This
+ * is expected to be used as sysdata for PCI controllers that
+ * use ECAM.
+ */
+struct pci_config_window {
+    paddr_t         phys_addr;
+    paddr_t         size;
+    uint8_t         busn_start;
+    uint8_t         busn_end;
+    void __iomem    *win;
+};
+
+/*
+ * struct to hold pci host bridge information
+ * for a PCI controller.
+ */
+struct pci_host_bridge {
+    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
+    struct list_head node;           /* Node in list of host bridges */
+    uint16_t segment;                /* Segment number */
+    uint8_t bus_start;               /* Bus start of this bridge. */
+    uint8_t bus_end;                 /* Bus end of this bridge. */
+    struct pci_config_window* cfg;   /* Pointer to the bridge config window */
+    void *sysdata;                   /* Pointer to the config space window*/
+    const struct pci_ops *ops;
+};
+
+struct pci_ops {
+    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf,
+                             uint32_t offset);
+    int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf,
+                uint32_t reg, uint32_t len, uint32_t *value);
+    int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
+                 uint32_t reg, uint32_t len, uint32_t value);
+};
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_ecam_ops {
+    unsigned int            bus_shift;
+    struct pci_ops          pci_ops;
+    int (*cfg_reg_index)(struct dt_device_node *dev);
+    int (*init)(struct pci_config_window *);
+};
+
+/* Default ECAM ops */
+extern const struct pci_ecam_ops pci_generic_ecam_ops;
+
+int pci_host_common_probe(struct dt_device_node *dev, const void *data);
+int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t *value);
+int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t value);
+void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
+                               uint32_t sbdf, uint32_t where);
+
 static always_inline bool is_pci_passthrough_enabled(void)
 {
     return pci_passthrough_enabled;
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:39:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:39:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192597.343125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bW-0002j2-4x; Wed, 22 Sep 2021 11:39:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192597.343125; Wed, 22 Sep 2021 11:39:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bW-0002iv-1A; Wed, 22 Sep 2021 11:39:58 +0000
Received: by outflank-mailman (input) for mailman id 192597;
 Wed, 22 Sep 2021 11:39:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0bU-0002Ir-82
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:39:56 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ce3e93e4-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:39:54 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 70A3011B3;
 Wed, 22 Sep 2021 04:39:54 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 55E663F719;
 Wed, 22 Sep 2021 04:39:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce3e93e4-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host controller
Date: Wed, 22 Sep 2021 12:34:58 +0100
Message-Id: <4b2b04ab1dbc1a5c52bf54c399cdb6cf454183ea.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add support for Xilinx ZynqMP PCI host controller to map the PCI config
space to the XEN memory.

Patch helps to understand how the generic infrastructure for PCI
host-bridge discovery will be used for future references.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Change on v2:
- Add more info in commit msg
---
 xen/arch/arm/pci/Makefile          |  1 +
 xen/arch/arm/pci/pci-host-zynqmp.c | 58 ++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
 create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c

diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
index e86f2b46fd..0572c68bba 100644
--- a/xen/arch/arm/pci/Makefile
+++ b/xen/arch/arm/pci/Makefile
@@ -4,3 +4,4 @@ obj-y += pci.o
 obj-y += pci-host-generic.o
 obj-y += pci-host-common.o
 obj-y += ecam.o
+obj-y += pci-host-zynqmp.o
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
new file mode 100644
index 0000000000..c27b4ea9f0
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -0,0 +1,58 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ * Based on xen/arch/arm/pci/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/device.h>
+#include <xen/pci.h>
+#include <asm/pci.h>
+
+int nwl_cfg_reg_index(struct dt_device_node *np)
+{
+    return dt_property_match_string(np, "reg-names", "cfg");
+}
+
+/* ECAM ops */
+const struct pci_ecam_ops nwl_pcie_ops = {
+    .bus_shift  = 20,
+    .cfg_reg_index = nwl_cfg_reg_index,
+    .pci_ops    = {
+        .map_bus                = pci_ecam_map_bus,
+        .read                   = pci_generic_config_read,
+        .write                  = pci_generic_config_write,
+    }
+};
+
+static const struct dt_device_match nwl_pcie_dt_match[] = {
+    { .compatible = "xlnx,nwl-pcie-2.11",
+      .data =       &nwl_pcie_ops },
+    { },
+};
+
+DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
+.dt_match = nwl_pcie_dt_match,
+.init = pci_host_common_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:40:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:40:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192611.343136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bx-0004Hv-DW; Wed, 22 Sep 2021 11:40:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192611.343136; Wed, 22 Sep 2021 11:40:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0bx-0004Ho-AJ; Wed, 22 Sep 2021 11:40:25 +0000
Received: by outflank-mailman (input) for mailman id 192611;
 Wed, 22 Sep 2021 11:40:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0bv-0004AV-Ue
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:40:23 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 5d221a6b-b1a8-40fb-9927-c7c79fe36240;
 Wed, 22 Sep 2021 11:40:18 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B735D11B3;
 Wed, 22 Sep 2021 04:40:18 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B5B3C3F719;
 Wed, 22 Sep 2021 04:40:17 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d221a6b-b1a8-40fb-9927-c7c79fe36240
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 13/17] xen:arm: Implement pci access functions
Date: Wed, 22 Sep 2021 12:34:59 +0100
Message-Id: <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

Implement generic pci access functions to read/write the configuration
space.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Fixed comments 
---
 xen/arch/arm/pci/pci-access.c      | 58 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
 xen/include/asm-arm/pci.h          |  2 ++
 3 files changed, 79 insertions(+)

diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
index 04fe9fbf92..45500cec2a 100644
--- a/xen/arch/arm/pci/pci-access.c
+++ b/xen/arch/arm/pci/pci-access.c
@@ -16,6 +16,7 @@
 #include <asm/io.h>
 
 #define INVALID_VALUE (~0U)
+#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
 
 int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
                             uint32_t reg, uint32_t len, uint32_t *value)
@@ -72,6 +73,63 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
     return 0;
 }
 
+static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
+                                unsigned int len)
+{
+    uint32_t val = PCI_ERR_VALUE(len);
+
+    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+
+    if ( unlikely(!bridge) )
+        return val;
+
+    if ( unlikely(!bridge->ops->read) )
+        return val;
+
+    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);
+
+    return val;
+}
+
+static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
+                             unsigned int len, uint32_t val)
+{
+    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+
+    if ( unlikely(!bridge) )
+        return;
+
+    if ( unlikely(!bridge->ops->write) )
+        return;
+
+    bridge->ops->write(bridge, (uint32_t) sbdf.sbdf, reg, len, val);
+}
+
+/*
+ * Wrappers for all PCI configuration access functions.
+ */
+
+#define PCI_OP_WRITE(size, type)                            \
+    void pci_conf_write##size(pci_sbdf_t sbdf,              \
+                              unsigned int reg, type val)   \
+{                                                           \
+    pci_config_write(sbdf, reg, size / 8, val);             \
+}
+
+#define PCI_OP_READ(size, type)                             \
+    type pci_conf_read##size(pci_sbdf_t sbdf,               \
+                              unsigned int reg)             \
+{                                                           \
+    return pci_config_read(sbdf, reg, size / 8);            \
+}
+
+PCI_OP_READ(8, uint8_t)
+PCI_OP_READ(16, uint16_t)
+PCI_OP_READ(32, uint32_t)
+PCI_OP_WRITE(8, uint8_t)
+PCI_OP_WRITE(16, uint16_t)
+PCI_OP_WRITE(32, uint32_t)
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 4beec14f2f..3bdc336268 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -243,6 +243,25 @@ err_exit:
     return err;
 }
 
+/*
+ * This function will lookup an hostbridge based on the segment and bus
+ * number.
+ */
+struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
+{
+    struct pci_host_bridge *bridge;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        if ( bridge->segment != segment )
+            continue;
+        if ( (bus < bridge->bus_start) || (bus > bridge->bus_end) )
+            continue;
+        return bridge;
+    }
+
+    return NULL;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 4b32c7088a..5406daecda 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -18,6 +18,7 @@
 #ifdef CONFIG_HAS_PCI
 
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
+#define PRI_pci "%04x:%02x:%02x.%u"
 
 extern bool_t pci_passthrough_enabled;
 
@@ -84,6 +85,7 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
                             uint32_t reg, uint32_t len, uint32_t value);
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                uint32_t sbdf, uint32_t where);
+struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:40:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:40:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192613.343147 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0c3-0004cP-Lm; Wed, 22 Sep 2021 11:40:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192613.343147; Wed, 22 Sep 2021 11:40:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0c3-0004cI-I1; Wed, 22 Sep 2021 11:40:31 +0000
Received: by outflank-mailman (input) for mailman id 192613;
 Wed, 22 Sep 2021 11:40:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0c2-0004bg-Pr
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:40:30 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e2d797c4-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:40:29 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 089086D;
 Wed, 22 Sep 2021 04:40:29 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 41F453F719;
 Wed, 22 Sep 2021 04:40:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2d797c4-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI support for ARM.
Date: Wed, 22 Sep 2021 12:35:00 +0100
Message-Id: <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The existing VPCI support available for X86 is adapted for Arm.
When the device is added to XEN via the hyper call
“PHYSDEVOP_pci_device_add”, VPCI handler for the config space
access is added to the Xen to emulate the PCI devices config space.

A MMIO trap handler for the PCI ECAM space is registered in XEN
so that when guest is trying to access the PCI config space,XEN
will trap the access and emulate read/write using the VPCI and
not the real PCI hardware.

For Dom0less systems scan_pci_devices() would be used to discover the
PCI device in XEN and VPCI handler will be added during XEN boots.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Add new XEN_DOMCTL_CDF_vpci flag
- modify has_vpci() to include XEN_DOMCTL_CDF_vpci
- enable vpci support when pci-passthough option is enabled.
---
 xen/arch/arm/Makefile         |   1 +
 xen/arch/arm/domain.c         |   6 +-
 xen/arch/arm/domain_build.c   |   3 +
 xen/arch/arm/vpci.c           | 102 ++++++++++++++++++++++++++++++++++
 xen/arch/arm/vpci.h           |  36 ++++++++++++
 xen/common/domain.c           |   2 +-
 xen/drivers/passthrough/pci.c |  12 ++++
 xen/include/asm-arm/domain.h  |   8 ++-
 xen/include/asm-x86/pci.h     |   2 -
 xen/include/public/arch-arm.h |   7 +++
 xen/include/public/domctl.h   |   4 +-
 xen/include/xen/pci.h         |   2 +
 12 files changed, 179 insertions(+), 6 deletions(-)
 create mode 100644 xen/arch/arm/vpci.c
 create mode 100644 xen/arch/arm/vpci.h

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 44d7cc81fa..fb9c976ea2 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
 obj-y += platforms/
 endif
 obj-$(CONFIG_TEE) += tee/
+obj-$(CONFIG_HAS_VPCI) += vpci.o
 
 obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
 obj-y += bootfdt.init.o
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d..f7ed130023 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -39,6 +39,7 @@
 #include <asm/vgic.h>
 #include <asm/vtimer.h>
 
+#include "vpci.h"
 #include "vuart.h"
 
 DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
@@ -623,7 +624,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
     unsigned int max_vcpus;
 
     /* HVM and HAP must be set. IOMMU may or may not be */
-    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
+    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci) !=
          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
@@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
     if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
         goto fail;
 
+    if ( (rc = domain_vpci_init(d)) != 0 )
+        goto fail;
+
     return 0;
 
 fail:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d634c1..5eb83b12a1 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2773,6 +2773,9 @@ void __init create_dom0(void)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
+    if ( pci_passthrough_enabled )
+        dom0_cfg.flags |= XEN_DOMCTL_CDF_vpci;
+
     dom0 = domain_create(0, &dom0_cfg, true);
     if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
         panic("Error creating domain 0\n");
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
new file mode 100644
index 0000000000..76c12b9281
--- /dev/null
+++ b/xen/arch/arm/vpci.c
@@ -0,0 +1,102 @@
+/*
+ * xen/arch/arm/vpci.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+#include <xen/sched.h>
+
+#include <asm/mmio.h>
+
+#define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
+
+/* Do some sanity checks. */
+static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
+{
+    /* Check access size. */
+    if ( len > 8 )
+        return false;
+
+    /* Check that access is size aligned. */
+    if ( (reg & (len - 1)) )
+        return false;
+
+    return true;
+}
+
+static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
+                          register_t *r, void *p)
+{
+    unsigned int reg;
+    pci_sbdf_t sbdf;
+    unsigned long data = ~0UL;
+    unsigned int size = 1U << info->dabt.size;
+
+    sbdf.sbdf = MMCFG_BDF(info->gpa);
+    reg = REGISTER_OFFSET(info->gpa);
+
+    if ( !vpci_mmio_access_allowed(reg, size) )
+        return 0;
+
+    data = vpci_read(sbdf, reg, min(4u, size));
+    if ( size == 8 )
+        data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
+
+    *r = data;
+
+    return 1;
+}
+
+static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
+                           register_t r, void *p)
+{
+    unsigned int reg;
+    pci_sbdf_t sbdf;
+    unsigned long data = r;
+    unsigned int size = 1U << info->dabt.size;
+
+    sbdf.sbdf = MMCFG_BDF(info->gpa);
+    reg = REGISTER_OFFSET(info->gpa);
+
+    if ( !vpci_mmio_access_allowed(reg, size) )
+        return 0;
+
+    vpci_write(sbdf, reg, min(4u, size), data);
+    if ( size == 8 )
+        vpci_write(sbdf, reg + 4, 4, data >> 32);
+
+    return 1;
+}
+
+static const struct mmio_handler_ops vpci_mmio_handler = {
+    .read  = vpci_mmio_read,
+    .write = vpci_mmio_write,
+};
+
+int domain_vpci_init(struct domain *d)
+{
+    if ( !has_vpci(d) )
+        return 0;
+
+    register_mmio_handler(d, &vpci_mmio_handler,
+                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
+
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
new file mode 100644
index 0000000000..d8a7b0e3e8
--- /dev/null
+++ b/xen/arch/arm/vpci.h
@@ -0,0 +1,36 @@
+/*
+ * xen/arch/arm/vpci.h
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ARCH_ARM_VPCI_H__
+#define __ARCH_ARM_VPCI_H__
+
+#ifdef CONFIG_HAS_VPCI
+int domain_vpci_init(struct domain *d);
+#else
+static inline int domain_vpci_init(struct domain *d)
+{
+    return 0;
+}
+#endif
+
+#endif /* __ARCH_ARM_VPCI_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..40d67ec342 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
-           XEN_DOMCTL_CDF_nested_virt) )
+           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
     {
         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
         return -EINVAL;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index d774a6154e..633e89ac13 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
     else
         iommu_enable_device(pdev);
 
+#ifdef CONFIG_ARM
+    /*
+     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
+     * Dom0 inform XEN to add the PCI devices in XEN.
+     */
+    ret = vpci_add_handlers(pdev);
+    if ( ret ) {
+        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
+        goto out;
+    }
+#endif
+
     pci_enable_acs(pdev);
 
 out:
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c9277b5c6d..5fb4e24adf 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -2,6 +2,7 @@
 #define __ASM_DOMAIN_H__
 
 #include <xen/cache.h>
+#include <xen/nospec.h>
 #include <xen/timer.h>
 #include <asm/page.h>
 #include <asm/p2m.h>
@@ -262,7 +263,12 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
 
 #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
 
-#define has_vpci(d)    ({ (void)(d); false; })
+/*
+ * For X86 VPCI is enabled and tested for PVH DOM0 only but
+ * for ARM we enable support VPCI for guest domain also.
+ */
+#define has_vpci(d) ((void)(d), IS_ENABLED(CONFIG_HAS_VPCI) && \
+        evaluate_nospec(d->options & XEN_DOMCTL_CDF_vpci))
 
 #endif /* __ASM_DOMAIN_H__ */
 
diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
index 0e160c6d01..a5590557db 100644
--- a/xen/include/asm-x86/pci.h
+++ b/xen/include/asm-x86/pci.h
@@ -6,8 +6,6 @@
 #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
 #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
 
-#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
-
 #define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \
                         || id == 0x01268086 || id == 0x01028086 \
                         || id == 0x01128086 || id == 0x01228086 \
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 6b5a5f818a..727541a321 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -416,6 +416,13 @@ typedef uint64_t xen_callback_t;
 #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
 #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
 
+/*
+ * 256 MB is reserved for VPCI configuration space based on calculation
+ * 256 buses × 32 devices × 8 functions × 4 KB = 256 MB
+ */
+#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
+#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
+
 /* ACPI tables physical address */
 #define GUEST_ACPI_BASE xen_mk_ullong(0x20000000)
 #define GUEST_ACPI_SIZE xen_mk_ullong(0x02000000)
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..4245da3f45 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -70,9 +70,11 @@ struct xen_domctl_createdomain {
 #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
 #define _XEN_DOMCTL_CDF_nested_virt   6
 #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
+#define _XEN_DOMCTL_CDF_vpci          7
+#define XEN_DOMCTL_CDF_vpci           (1U << _XEN_DOMCTL_CDF_vpci)
 
 /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
-#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
+#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_vpci
 
     uint32_t flags;
 
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 8e3d4d9454..2b2dfb6f1b 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -41,6 +41,8 @@
 #define PCI_SBDF3(s,b,df) \
     ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
 
+#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
+
 typedef union {
     uint32_t sbdf;
     struct {
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:40:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:40:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192620.343158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0cJ-0005LS-Vb; Wed, 22 Sep 2021 11:40:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192620.343158; Wed, 22 Sep 2021 11:40:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0cJ-0005KN-SO; Wed, 22 Sep 2021 11:40:47 +0000
Received: by outflank-mailman (input) for mailman id 192620;
 Wed, 22 Sep 2021 11:40:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0cI-0005IC-Aw
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:40:46 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ec8c675e-1b99-11ec-b970-12813bfff9fa;
 Wed, 22 Sep 2021 11:40:45 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 317106D;
 Wed, 22 Sep 2021 04:40:45 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 16A963F719;
 Wed, 22 Sep 2021 04:40:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec8c675e-1b99-11ec-b970-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 15/17] xen/arm: Transitional change to build HAS_VPCI on ARM.
Date: Wed, 22 Sep 2021 12:35:01 +0100
Message-Id: <0b7654cc44452575cae43c4d1700075c3a42ea2a.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

This patch will be reverted once we add support for VPCI MSI/MSIX
support on ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2: Patch introduced in v2
---
 xen/drivers/vpci/Makefile | 3 ++-
 xen/drivers/vpci/header.c | 2 ++
 xen/include/asm-arm/pci.h | 8 ++++++++
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
index 55d1bdfda0..1a1413b93e 100644
--- a/xen/drivers/vpci/Makefile
+++ b/xen/drivers/vpci/Makefile
@@ -1 +1,2 @@
-obj-y += vpci.o header.o msi.o msix.o
+obj-y += vpci.o header.o
+obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index ba9a036202..f8cd55e7c0 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
      * FIXME: punching holes after the p2m has been set up might be racy for
      * DomU usage, needs to be revisited.
      */
+#ifdef CONFIG_HAS_PCI_MSI
     if ( map && !rom_only && vpci_make_msix_hole(pdev) )
         return;
+#endif
 
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 5406daecda..9327b7488e 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -27,6 +27,14 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/* Arch-specific MSI data for vPCI. */
+struct vpci_arch_msi {
+};
+
+/* Arch-specific MSI-X entry data for vPCI. */
+struct vpci_arch_msix_entry {
+};
+
 /*
  * struct to hold the mappings of a config space window. This
  * is expected to be used as sysdata for PCI controllers that
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:41:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192626.343169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ca-0005xz-CF; Wed, 22 Sep 2021 11:41:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192626.343169; Wed, 22 Sep 2021 11:41:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0ca-0005xq-96; Wed, 22 Sep 2021 11:41:04 +0000
Received: by outflank-mailman (input) for mailman id 192626;
 Wed, 22 Sep 2021 11:41:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0cY-0005vT-D0
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:41:02 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 671f767b-57c4-4942-a1fc-7bfb0e738a94;
 Wed, 22 Sep 2021 11:41:00 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1AF76D;
 Wed, 22 Sep 2021 04:41:00 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4778C3F719;
 Wed, 22 Sep 2021 04:40:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 671f767b-57c4-4942-a1fc-7bfb0e738a94
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 16/17] arm/libxl: Emulated PCI device tree node in libxl
Date: Wed, 22 Sep 2021 12:35:02 +0100
Message-Id: <5c66821e2333f6b27cb20a345eaf01a141a6c63b.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

libxl will create an emulated PCI device tree node in the device tree to
enable the guest OS to discover the virtual PCI during guest boot.
Emulated PCI device tree node will only be created when there is any
device assigned to guest.

A new area has been reserved in the arm guest physical map at
which the VPCI bus is declared in the device tree (reg and ranges
parameters of the node).

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain.
---
 tools/include/libxl.h            |   6 ++
 tools/libs/light/libxl_arm.c     | 105 +++++++++++++++++++++++++++++++
 tools/libs/light/libxl_create.c  |   3 +
 tools/libs/light/libxl_types.idl |   1 +
 tools/xl/xl_parse.c              |   2 +
 xen/include/public/arch-arm.h    |  10 +++
 6 files changed, 127 insertions(+)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d698..3362073b21 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -358,6 +358,12 @@
  */
 #define LIBXL_HAVE_BUILDINFO_ARM_VUART 1
 
+/*
+ * LIBXL_HAVE_BUILDINFO_ARM_VPCI indicates that the toolstack supports virtual
+ * PCI for ARM.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARM_VPCI 1
+
 /*
  * LIBXL_HAVE_BUILDINFO_GRANT_LIMITS indicates that libxl_domain_build_info
  * has the max_grant_frames and max_maptrack_frames fields.
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6e00..52f1ddce48 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
     return fdt_property(fdt, "reg", regs, sizeof(regs));
 }
 
+static int fdt_property_values(libxl__gc *gc, void *fdt,
+        const char *name, unsigned num_cells, ...)
+{
+    uint32_t prop[num_cells];
+    be32 *cells = &prop[0];
+    int i;
+    va_list ap;
+    uint32_t arg;
+
+    va_start(ap, num_cells);
+    for (i = 0 ; i < num_cells; i++) {
+        arg = va_arg(ap, uint32_t);
+        set_cell(&cells, 1, arg);
+    }
+    va_end(ap);
+
+    return fdt_property(fdt, name, prop, sizeof(prop));
+}
+
+static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
+                                    unsigned addr_cells,
+                                    unsigned size_cells,
+                                    unsigned num_regs, ...)
+{
+    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
+    be32 *cells = &regs[0];
+    int i;
+    va_list ap;
+    uint64_t arg;
+
+    va_start(ap, num_regs);
+    for (i = 0 ; i < num_regs; i++) {
+        /* Set the memory bit field */
+        arg = va_arg(ap, uint32_t);
+        set_cell(&cells, 1, arg);
+
+        /* Set the vpci bus address */
+        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
+        set_cell(&cells, addr_cells , arg);
+
+        /* Set the cpu bus address where vpci address is mapped */
+        set_cell(&cells, addr_cells, arg);
+
+        /* Set the vpci size requested */
+        arg = size_cells ? va_arg(ap, uint64_t) : 0;
+        set_cell(&cells, size_cells, arg);
+    }
+    va_end(ap);
+
+    return fdt_property(fdt, "ranges", regs, sizeof(regs));
+}
+
 static int make_root_properties(libxl__gc *gc,
                                 const libxl_version_info *vers,
                                 void *fdt)
@@ -668,6 +720,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+static int make_vpci_node(libxl__gc *gc, void *fdt,
+        const struct arch_info *ainfo,
+        struct xc_dom_image *dom)
+{
+    int res;
+    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
+    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
+    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "device_type", "pci");
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
+    if (res) return res;
+
+    res = fdt_property_values(gc, fdt, "bus-range", 2, 0, 255);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 3);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#size-cells", 2);
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "status", "okay");
+    if (res) return res;
+
+    res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+        GUEST_ROOT_SIZE_CELLS, 2,
+        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE,
+        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
+        GUEST_VPCI_PREFETCH_MEM_SIZE);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
 static const struct arch_info *get_arch_info(libxl__gc *gc,
                                              const struct xc_dom_image *dom)
 {
@@ -971,6 +1070,9 @@ next_resize:
         if (info->tee == LIBXL_TEE_TYPE_OPTEE)
             FDT( make_optee_node(gc, fdt) );
 
+        if (libxl_defbool_val(info->arch_arm.vpci))
+            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
+
         if (pfdt)
             FDT( copy_partial_fdt(gc, fdt, pfdt) );
 
@@ -1189,6 +1291,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
     /* ACPI is disabled by default */
     libxl_defbool_setdefault(&b_info->acpi, false);
 
+    /* VPCI is disabled by default */
+    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
+
     if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
         return;
 
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index e356b2106d..529e475489 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -632,6 +632,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
             create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
 
+        if ( libxl_defbool_val(b_info->arch_arm.vpci) )
+            create.flags |= XEN_DOMCTL_CDF_vpci;
+
         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff653a..78b1ddf0b8 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
+                               ("vpci", libxl_defbool),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd5..ffafbeffb4 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
         }
         if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
             libxl_defbool_set(&b_info->u.pv.e820_host, true);
+        if (d_config->num_pcidevs)
+            libxl_defbool_set(&b_info->arch_arm.vpci, true);
     }
 
     if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 727541a321..85692eca6c 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -431,6 +431,11 @@ typedef uint64_t xen_callback_t;
 #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
 #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
 
+/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
+#define GUEST_VPCI_ADDR_TYPE_MEM            xen_mk_ullong(0x02000000)
+#define GUEST_VPCI_MEM_ADDR                 xen_mk_ullong(0x22001000)
+#define GUEST_VPCI_MEM_SIZE                 xen_mk_ullong(0x10000000)
+
 /*
  * 16MB == 4096 pages reserved for guest to use as a region to map its
  * grant table in.
@@ -446,6 +451,11 @@ typedef uint64_t xen_callback_t;
 #define GUEST_RAM0_BASE   xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1GB */
 #define GUEST_RAM0_SIZE   xen_mk_ullong(0xc0000000)
 
+/* 4GB @ 4GB Prefetch Memory for VPCI */
+#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM   xen_mk_ullong(0x42000000)
+#define GUEST_VPCI_PREFETCH_MEM_ADDR        xen_mk_ullong(0x100000000)
+#define GUEST_VPCI_PREFETCH_MEM_SIZE        xen_mk_ullong(0x100000000)
+
 #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ 8GB */
 #define GUEST_RAM1_SIZE   xen_mk_ullong(0xfe00000000)
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:41:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:41:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192631.343179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0cg-0006NH-L8; Wed, 22 Sep 2021 11:41:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192631.343179; Wed, 22 Sep 2021 11:41:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0cg-0006N8-I3; Wed, 22 Sep 2021 11:41:10 +0000
Received: by outflank-mailman (input) for mailman id 192631;
 Wed, 22 Sep 2021 11:41:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hzcW=OM=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mT0ce-0005vT-MU
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:41:08 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 1c2d6418-f2a8-4893-9eda-a6f45f565183;
 Wed, 22 Sep 2021 11:41:07 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 476036D;
 Wed, 22 Sep 2021 04:41:07 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 46E213F719;
 Wed, 22 Sep 2021 04:41:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c2d6418-f2a8-4893-9eda-a6f45f565183
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	andre.przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 17/17] xen/arm: Add linux,pci-domain property for hwdom if not available.
Date: Wed, 22 Sep 2021 12:35:03 +0100
Message-Id: <e1b8d878709dc8e93a3bb340f921a24976cebb57.1632307952.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632307952.git.rahul.singh@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>

If the property is not present in the device tree node for host bridge,
XEN while creating the dtb for hwdom will create this property and
assigns the already allocated segment to the host bridge
so that XEN and linux will have the same segment for the host bridges.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v2:
- Add linux,pci-domain only when pci-passthrough command line option is enabeld
---
 xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
 xen/include/asm-arm/pci.h          |  9 +++++++++
 3 files changed, 46 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 5eb83b12a1..83ab0d52cc 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
             return res;
     }
 
+    if ( pci_passthrough_enabled && dt_device_type_is_equal(node, "pci") )
+    {
+        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
+        {
+            uint16_t segment;
+
+            res = pci_get_host_bridge_segment(node, &segment);
+            if ( res < 0 )
+                return res;
+
+            res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment);
+            if ( res )
+                return res;
+        }
+    }
+
     /*
      * Override the property "status" to disable the device when it's
      * marked for passthrough.
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 3bdc336268..a88f20175e 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -262,6 +262,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
 
     return NULL;
 }
+
+/*
+ * This function will lookup an hostbridge based on config space address.
+ */
+int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                uint16_t *segment)
+{
+    struct pci_host_bridge *bridge;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        if ( bridge->dt_node != node )
+            continue;
+
+        *segment = bridge->segment;
+        return 0;
+    }
+
+    return -EINVAL;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 9327b7488e..9edc6bc5ae 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -94,6 +94,8 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                uint32_t sbdf, uint32_t where);
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
+int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                uint16_t *segment);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
@@ -110,5 +112,12 @@ static always_inline bool is_pci_passthrough_enabled(void)
     return false;
 }
 
+static inline int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                              uint16_t *segment)
+{
+    ASSERT_UNREACHABLE();
+    return -EINVAL;
+}
+
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:45:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:45:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192649.343191 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0gh-0007ZZ-82; Wed, 22 Sep 2021 11:45:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192649.343191; Wed, 22 Sep 2021 11:45:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0gh-0007ZS-4z; Wed, 22 Sep 2021 11:45:19 +0000
Received: by outflank-mailman (input) for mailman id 192649;
 Wed, 22 Sep 2021 11:45:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NDiu=OM=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mT0gg-0007ZM-2N
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:45:18 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e7b59fc0-f8a9-4eff-bee8-43bb60a7e4de;
 Wed, 22 Sep 2021 11:45:15 +0000 (UTC)
Received: from AS8PR04CA0183.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::8)
 by AM4PR08MB2627.eurprd08.prod.outlook.com (2603:10a6:205:b::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Wed, 22 Sep
 2021 11:45:06 +0000
Received: from VE1EUR03FT022.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2f3:cafe::b9) by AS8PR04CA0183.outlook.office365.com
 (2603:10a6:20b:2f3::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Wed, 22 Sep 2021 11:45:06 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT022.mail.protection.outlook.com (10.152.18.64) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 11:45:06 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Wed, 22 Sep 2021 11:45:05 +0000
Received: from f36b91041942.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E6810B6A-DEB2-40F7-8B45-E9F1BA860C95.1; 
 Wed, 22 Sep 2021 11:44:50 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f36b91041942.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 Sep 2021 11:44:50 +0000
Received: from DB6P18901CA0002.EURP189.PROD.OUTLOOK.COM (2603:10a6:4:16::12)
 by VI1PR08MB3311.eurprd08.prod.outlook.com (2603:10a6:803:3e::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Wed, 22 Sep
 2021 11:44:39 +0000
Received: from DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:16:cafe::6a) by DB6P18901CA0002.outlook.office365.com
 (2603:10a6:4:16::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 22 Sep 2021 11:44:38 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT003.mail.protection.outlook.com (10.152.20.157) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 11:44:38 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Wed, 22 Sep
 2021 11:44:37 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Wed, 22 Sep 2021 11:44:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7b59fc0-f8a9-4eff-bee8-43bb60a7e4de
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lOVPuq3M+VKdkPUXWM5c9CrS27QS5LinUmI4Pl5ZKhw=;
 b=uNXjY5xIXF5QDQFErfQe4rN4/7XjiIQ/3j/AT/mNHWPgaddeHqcyGhyMrXXJzJMhIesaUDeJ4c/CbGXstMfOUYJDMXaYroP5ai0O9tixp/Zc+Bf34UDYpqwwUbgIQC8rUQhvgpk/yJazEsu+2XV7NfrU1menuL5GVWzq/ci3PuM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 16001ac9e78f6432
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=McIOSh/HJv1dPUE9Gia0VkU5oLfDo2qKfJZqQug8N0smaRkmM18BSykyfM4vSPsiy2Hjhxvq79oZYxkyW7OQAByeeA0Hr6kSdIAZxZJ5MJCC+iB5sbsYLd1Mg7ZLEdKu7mAn3n7g803zbj7PVZfls9POMqJSO+/jJ1coFE/qjSJxwt0K6du7l2fa2hnYyMzKG6i/+sU/bh/HmbpmiCvC1QGXm3mBapW4y1dfwpoKdJqfK4S3/tw4bLaOUI+rDBbOhdZu4HeCRIlbKQduQ84r84SPY+7RTcOOOEaAbX67o2w6NMAJ4PAt6bq8FQDgyvOpcM5TST9m42JKQUk1i2gyXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lOVPuq3M+VKdkPUXWM5c9CrS27QS5LinUmI4Pl5ZKhw=;
 b=mgY7pRP4OXXqrVnv8BbOUENB26vfRNfk2eH12XlkpAVVOkW0hdH5RhC5ks3IaLSgTdh7YecwCRe+biM33rFcm0IukFT46g7pkzUgrwKd63B7CsDUuu54gE2K1ntPaLqdv5j7KmVgE/n1whtusCi1u249ifQuw5ZieaNovtVZcupeYmlsGdXIcd9hOiKCl+ZOkGKCioXIzxsiyBVPdqVEv7SZd5FLf4wQqT4tHqvQQyVxes92Omo4gACZD2MivwhOGy2fwWDAQRsV3h1iWURDtRwAv5tExqyJg0FoQ2BPRbZ+2SjFoHW/e9gRgk92Cv43ap59iAxRl7kfvh2SYJezpg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lOVPuq3M+VKdkPUXWM5c9CrS27QS5LinUmI4Pl5ZKhw=;
 b=uNXjY5xIXF5QDQFErfQe4rN4/7XjiIQ/3j/AT/mNHWPgaddeHqcyGhyMrXXJzJMhIesaUDeJ4c/CbGXstMfOUYJDMXaYroP5ai0O9tixp/Zc+Bf34UDYpqwwUbgIQC8rUQhvgpk/yJazEsu+2XV7NfrU1menuL5GVWzq/ci3PuM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>, <jbeulich@suse.com>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH] xen: fix broken tainted value in mark_page_free
Date: Wed, 22 Sep 2021 11:44:32 +0000
Message-ID: <20210922114432.1093340-1-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 408fd37e-312f-4437-94f2-08d97dbe6c52
X-MS-TrafficTypeDiagnostic: VI1PR08MB3311:|AM4PR08MB2627:
X-Microsoft-Antispam-PRVS:
	<AM4PR08MB2627436C3FF8354B8960E587F7A29@AM4PR08MB2627.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1091;OLM:1091;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 C9BNsTFMlOqvUVF4CQMdB8VSbKpvyUnSjEzo3xtty2phvL41FXUGvA3Jlddj4GW0gbFvi64X2OPbeQiL72YStqM7ZPmSDIvbOfmZ5oStrnsHPDgV2o5xJkBACFufM2e5fz9XmWZ8cHZgMGh2ZhdDSXxGS/b3vZkXuz8rD7SpKcQvW4RuT4RSHL8NtXUlaOHB4Pa8Rw+H9UILjPXeov5luaJb2ylGIxsHDjv/Zqnr2Uo5LJ7KKpiiXw1ndsngpxuS58b0TnbIozI2fs5q2Xfz6HG5eIdqo8raIP/bfJWmyBUhXfLtCjVlNC/htDf8tWCkZF7X4yPbZZLIDXolJPJAhzanUG7axXV5ZE62Ysjpq2NIOXnI6/htFU4iPtw643nDc2iyxS3Z79MDQsCklkIunBa+AqjadSdWl75tMR2rfWtm5hn6Y2DT2XLBaUYcHuwmbNRcjJEkCnZFBrETUfZuQwpTjIqZOe2PLk+s1jyXprb6hV0DKdZuKg3FxYnflg0S29TFfnREpUy6E5fYlnJSFYXce3vB3KHN3NKkb6CfjpXBqB+WKvG8oSVG9xj2YWEjdO+P/mN20ACaJzm2Y0zGerKcxrzSn3EDmBIdOtQeMf2SePtik/9eAKcoeoP4HfnVEogO2nqPeYcM89XdaIYPUr9fyVzdGtpcUImbQV8yueNnWhsZgBNOQ8djkRYn8SAQZrdVQF/soo7UGx8+l0OoriiIWgAC00RqM8NnnCCXbVI7UuFyGhWtxIp2xUSovGsj
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(316002)(70206006)(82310400003)(356005)(36756003)(186003)(110136005)(8936002)(2906002)(26005)(6666004)(54906003)(70586007)(508600001)(86362001)(426003)(81166007)(5660300002)(1076003)(336012)(7696005)(4326008)(47076005)(8676002)(83380400001)(36860700001)(44832011)(2616005)(36900700001)(2101003);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3311
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT022.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7407ed3e-c8cc-4f6f-ba10-08d97dbe5bdb
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8JpWw5tF4ENEEay0KAMJhb07puL6zdgpGwR+aDcHMvcHfwEKt/mXU85zQsR4nG2Q2/ET/5xR1+NgFQWgATRi7NQSKewI7doO74kFoT6WpjHOJ7xig0ZxKAlzarVnrU2gKMjLndyzIgvmuDt6Tx9QMmV6ogSqyt6ISTqiOr4hJVDE4wiqf0iboo++gFn5yk1ZpSZa0KSEaMp17G8TEO2ZTxb9/P3npkbvtQ+uQjFoFx/kTxHPGsQ5kVH6aecG5oDjNVwYtiQTx1+7xKGiMqxzvclEBGzhPO3/wIkHT+tPAZtZdzKxOWEi5F5v+S3m+oknxhYsrp0ha8nLXG6mMLvSRVkmrO2Yr1Qk0PJokqv/t9CnA2QMTdTI61uQx0orzglvFvWDwRd0Awlyhzruf84FlR3pW33Sv/dUS3dsIOHwjmxMSS3LR7+SECo715SkZHH6jwNi0S0TXdJ0i/FkPNKaVzGTbT5TSErJrPpYch5sxz789CqvcsGx8FcdAKMlXMQwH63NlDBR7OoPM4QiugjnxIFRllIVXr22z+XPPMTcjC2QzgmweW4X9vaMDgeS9yIvRQQT4Kh01QvaEUBpNVdIel5ymgAlduF1Z4XSZx04joWzbz8MnCGbi2y1LiL29Tk49rr0swhhfjQ8m6U0uCn9iZK+RvfKJhBhqlHJfbPaU55xP5m9FvqQBJflFMqZT8en4R0cIQttGaC6JJDFQwtDizCTpEsr/iea/s6HoNRl5L0=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(186003)(26005)(70586007)(316002)(44832011)(47076005)(81166007)(426003)(6666004)(70206006)(36860700001)(36756003)(336012)(2616005)(7696005)(1076003)(4326008)(83380400001)(2906002)(508600001)(8676002)(86362001)(8936002)(5660300002)(110136005)(54906003)(82310400003)(2101003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 11:45:06.3192
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 408fd37e-312f-4437-94f2-08d97dbe6c52
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT022.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR08MB2627

Commit 540a637c3410780b519fc055f432afe271f642f8 defines a new
helper mark_page_free to extract common codes, while it accidently
breaks the local variable "tainted".

This patch fix it by letting mark_page_free() return bool of whether the
page is offlined and rename local variable "tainted" to "pg_offlined".

Coverity ID: 1491872

Fixes: 540a637c3410780b519fc055f432afe271f642f8
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
v2 changes:
- rename local variable "tainted" to "pg_offlined", and make it bool
---
 xen/common/page_alloc.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 6142c7bb6a..5801358b4b 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1380,8 +1380,10 @@ bool scrub_free_pages(void)
     return node_to_scrub(false) != NUMA_NO_NODE;
 }
 
-static void mark_page_free(struct page_info *pg, mfn_t mfn)
+static bool mark_page_free(struct page_info *pg, mfn_t mfn)
 {
+    bool pg_offlined = false;
+
     ASSERT(mfn_x(mfn) == mfn_x(page_to_mfn(pg)));
 
     /*
@@ -1405,7 +1407,7 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
     case PGC_state_offlining:
         pg->count_info = (pg->count_info & PGC_broken) |
                          PGC_state_offlined;
-        tainted = 1;
+        pg_offlined = true;
         break;
 
     default:
@@ -1425,6 +1427,8 @@ static void mark_page_free(struct page_info *pg, mfn_t mfn)
     /* This page is not a guest frame any more. */
     page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
     set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
+
+    return pg_offlined;
 }
 
 /* Free 2^@order set of pages. */
@@ -1433,7 +1437,7 @@ static void free_heap_pages(
 {
     unsigned long mask;
     mfn_t mfn = page_to_mfn(pg);
-    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), tainted = 0;
+    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), pg_offlined = 0;
     unsigned int zone = page_to_zone(pg);
 
     ASSERT(order <= MAX_ORDER);
@@ -1443,7 +1447,8 @@ static void free_heap_pages(
 
     for ( i = 0; i < (1 << order); i++ )
     {
-        mark_page_free(&pg[i], mfn_add(mfn, i));
+        if ( mark_page_free(&pg[i], mfn_add(mfn, i)) )
+            pg_offlined = 1;
 
         if ( need_scrub )
         {
@@ -1517,7 +1522,7 @@ static void free_heap_pages(
 
     page_list_add_scrub(pg, node, zone, order, pg->u.free.first_dirty);
 
-    if ( tainted )
+    if ( pg_offlined )
         reserve_offlined_page(pg);
 
     spin_unlock(&heap_lock);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:49:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:49:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192657.343201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0kD-0008UE-Tz; Wed, 22 Sep 2021 11:48:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192657.343201; Wed, 22 Sep 2021 11:48:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0kD-0008U7-QV; Wed, 22 Sep 2021 11:48:57 +0000
Received: by outflank-mailman (input) for mailman id 192657;
 Wed, 22 Sep 2021 11:48:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bMBL=OM=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mT0kC-0008U1-Mm
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:48:56 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe06::60d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df6f47f9-dcbc-47a9-a9ed-0f9b054c2ecb;
 Wed, 22 Sep 2021 11:48:54 +0000 (UTC)
Received: from AM5PR0402CA0004.eurprd04.prod.outlook.com
 (2603:10a6:203:90::14) by VI1PR0802MB2238.eurprd08.prod.outlook.com
 (2603:10a6:800:9a::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 11:48:52 +0000
Received: from AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:90:cafe::e8) by AM5PR0402CA0004.outlook.office365.com
 (2603:10a6:203:90::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Wed, 22 Sep 2021 11:48:52 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT008.mail.protection.outlook.com (10.152.16.123) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 11:48:51 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Wed, 22 Sep 2021 11:48:49 +0000
Received: from ade55d5d6ad9.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4D546286-697E-4D74-8FAF-DE51AA90867C.1; 
 Wed, 22 Sep 2021 11:48:37 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ade55d5d6ad9.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 22 Sep 2021 11:48:37 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6365.eurprd08.prod.outlook.com (2603:10a6:102:156::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 22 Sep
 2021 11:48:36 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 22 Sep 2021
 11:48:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df6f47f9-dcbc-47a9-a9ed-0f9b054c2ecb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6MmWJ7hHoqNYgOIHSmjWOBBxabBhm0rXpL0yo1/j+ps=;
 b=D+0NiW4VScqPtsfbbpP5jKWEW0TewfTiCgxpJhh6SAviyUREvyiWd7n4WGW7841j7SmZYchzKhu828CE47viIvda1tJTdCow3j3Qb6wyKR8JNMOMpcpeZ2mR3dG9zGZ9CCdZZ3uMq9JnIQzU3a8dpxux2D8mxxKKEDSqkdWDrV8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3deb535d217ff8a2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dLF0bFFd12PyxY15R3FDcYtiR1XouFeCj8db92wYc/V/Va36cwYHX+YflkSuoawmqnT0yDVgVPb3TTWb2iKTFOZVKtgDgIXcuglm0oIlCNhm4bX6izQDeybiap/6keQVtsJzwCjvmYeuZEx2Kj1c/V8EdFjFmNhaSE7YMMwqfC5CtK9eemng05HP3vqB6QtEOLNFfnX9E8X4Kc4reZ6D0iSBh8K7ElcLjVZdA+ZJb6wh1ucjHh4AcX/PZineA1Mn0WkszfT28dvhXA8XvCRUevJqP/Ukbc8PA2F90GhMwYl2S5nUdfx27Ue+JzuG8+CHhUORq8HUEt1jYVW6JNp7PQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6MmWJ7hHoqNYgOIHSmjWOBBxabBhm0rXpL0yo1/j+ps=;
 b=QAXoVNQf9mEksLJioFuwB3cJX95b7m+SocDUtTmy1RDtmjioQJGYYJkknobJJlXnPB2xtXEwuH45ToHX8uosIuot3rqaVAR9t0i/D0xxYQOJYhBGd5FKMi8GOGy33g8ufBRcuJs5Vv4SflSyTc80IR8ZEtYGcOEG0xe95FEn0oBnsiXJ1uctpIQQw5auLsAT5cxaanTaCSXYcqVNO9FaL6TIP33IscMdXKTuxGI7cAr9ZsUHqVqSCqkvxIRNX778rNm6ojSlauQYSj2M56N0b+60rLugSm8uoDQHiXiqAsONs2yjYNBNebmTFiQ80AT6LICk9xnxq5QKAB4m8ctQAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6MmWJ7hHoqNYgOIHSmjWOBBxabBhm0rXpL0yo1/j+ps=;
 b=D+0NiW4VScqPtsfbbpP5jKWEW0TewfTiCgxpJhh6SAviyUREvyiWd7n4WGW7841j7SmZYchzKhu828CE47viIvda1tJTdCow3j3Qb6wyKR8JNMOMpcpeZ2mR3dG9zGZ9CCdZZ3uMq9JnIQzU3a8dpxux2D8mxxKKEDSqkdWDrV8=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Penny Zheng <Penny.Zheng@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, "julien@xen.org" <julien@xen.org>,
	"jbeulich@suse.com" <jbeulich@suse.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH] xen: fix broken tainted value in mark_page_free
Thread-Topic: [PATCH] xen: fix broken tainted value in mark_page_free
Thread-Index: AQHXr6c/8OBwQWPrCkW/tzMAglitdauv8D8A
Date: Wed, 22 Sep 2021 11:48:36 +0000
Message-ID: <94F4D68E-DF7B-4613-B03B-05C1CB1154DB@arm.com>
References: <20210922114432.1093340-1-penny.zheng@arm.com>
In-Reply-To: <20210922114432.1093340-1-penny.zheng@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 6388dc97-e273-40c5-f806-08d97dbef2ca
x-ms-traffictypediagnostic: PAXPR08MB6365:|VI1PR0802MB2238:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0802MB2238D2C69786DB6A5BE2437A9DA29@VI1PR0802MB2238.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2089;OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4onHY0jXekkQNAbjHNdN9dc04xxDFD21ln+0hRWiG+kRKBX2w1yu42gKdeCTBEx/7wb23TcWGovuUylwKLx1UxqE/21mZVa6vi6BHqU4FbTzKf8xeSoo6/1etdzFxfd1n8uusMK4cFbIENiiGkFrmpYEqk7LK7ParI9sC0DPtkwlH2MOtuPB3TaTfbwjOrXRzy2sD9ianZUGcqF/Og59Q0v928yupIeAT9q8t4o/4jjh1ktRmGAJgPpUEKY6wZBJ6Pxa0fgNvxlc1gvimf8F9aBt0a9wcVo7R1dDu83p3L5aZdV2MdVacqLAv8QeWjYnlTzt3Yia021tr/MXxIyzBLuEFyOuH8GAMcjUV8djcmrMaxPvoHPOazlJXDnJpzWda4AKptvxAPsN8Dz1ru3qY2h6NP2UG32fP7I3ExlVAS89riZeJNssTGds/XIAFicjh7f1d8uRU/aMLFkJ8lFst3fywtoAuD/NrtD7w9HDbDHk/v9x8e6ow0sxuNL42BIBtk2a6k9NyHrARM7jEkkQugeSShG+0DaLO+1seagYj7WcNmFOtgk7t/QODLmUEzUM4YIuBPsfrkzj5JfuOktvNOBA8OVGx8awEFJVwBh5kZBtsH37SnrcrsNNOfinTuzBtmdoWYJYnLFqKUkTkLa5EiRUWvFVUxrdYF4Bx8NkK9IkD9JsLczk2oyh6bkMBSdVFeQQL6etndfdWMm7Lw4BguRqGjFwAeMf997KN4Tgl1M+DKbuw/m4z460GFblyaT4
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6506007)(2906002)(8936002)(53546011)(6512007)(508600001)(38100700002)(71200400001)(122000001)(33656002)(37006003)(316002)(8676002)(6636002)(5660300002)(186003)(83380400001)(6862004)(86362001)(76116006)(91956017)(54906003)(38070700005)(36756003)(66446008)(64756008)(4326008)(66556008)(26005)(6486002)(66946007)(66476007)(2616005)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <A7041D5C1EFAB246B21D4348596DA3B0@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6365
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	70206c1c-cd39-4c6f-c3cc-08d97dbee99f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	19ems8BqEnw56FpvtO34DGYuq707XZbnDsPijwOlPTbbRUn2O9OnGm7SI8qm2H1xX60TaKx8DuT3H09Lfyeyrk2Q65ES6wkOMCaSF1qOPx5QER2anticwZVjKzws2Qdq6+nZPjDpEs3FkrD7Di+RSFzgRQDz1vwAt2K1r2mhkwlxf/ent7Ju+2BU1mLhBPrYGFBjmvPQX61dixTHdJPMhiLAmMw7edyMTjzFWMJ1sdm7gfaHWZgGqlFMCQZpIHiVjxk41S5O7GirxeN4ILtOR1pDvW64QjZkfqqMpJ6I65kMQVK7eijJHL3wDR9i3h6PLnlyUJXS3ydKRVXmLPQPFrQfQf94d8t6SP0qdVmqj4bi2/m4fjKuKUEhVuhPy5q9PEiD36pFmVhAfMXHlaKrWsI537zBDF2JqyL4NsT5Lym4lMPxAUns9ind3yvlf8uc1bFj7enAvUiSwxk7utJCRXXhx7PD8kz1UDGDCDPmeV/xrf3RKPdm9MUJZFbDZAKKcsKd7Gjvq3/nHp6IMoYiX7Vt/r9PpJoAkPgohhleTVZjSlWwOmnlxW/iCntJLU6LKj656ga7CO4x9wUavelP53HLd5zR/7mJUh1Vo4p2vbHDSryZTPFBX1cppTQ1NvOsyFWLxHy+oDziB03jYdjHcRR6LSzY8tnHH5IgVRbwS5m6p4Rr0ZS0HzB4vTCc3u30Wg+hjV5B8e/6jaL44iJJpw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36860700001)(70206006)(5660300002)(70586007)(316002)(6636002)(82310400003)(4326008)(6512007)(33656002)(37006003)(2906002)(54906003)(83380400001)(36756003)(26005)(53546011)(6862004)(186003)(6486002)(6506007)(2616005)(47076005)(8676002)(8936002)(86362001)(356005)(508600001)(81166007)(336012);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 11:48:51.9712
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6388dc97-e273-40c5-f806-08d97dbef2ca
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2238

Hi Penny,

> On 22 Sep 2021, at 12:44, Penny Zheng <penny.zheng@arm.com> wrote:
>=20
> Commit 540a637c3410780b519fc055f432afe271f642f8 defines a new
> helper mark_page_free to extract common codes, while it accidently
> breaks the local variable "tainted".
>=20
> This patch fix it by letting mark_page_free() return bool of whether the
> page is offlined and rename local variable "tainted" to "pg_offlined".
>=20
> Coverity ID: 1491872
>=20
> Fixes: 540a637c3410780b519fc055f432afe271f642f8
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> v2 changes:
> - rename local variable "tainted" to "pg_offlined", and make it bool
> ---
> xen/common/page_alloc.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>=20
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 6142c7bb6a..5801358b4b 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1380,8 +1380,10 @@ bool scrub_free_pages(void)
>     return node_to_scrub(false) !=3D NUMA_NO_NODE;
> }
>=20
> -static void mark_page_free(struct page_info *pg, mfn_t mfn)
> +static bool mark_page_free(struct page_info *pg, mfn_t mfn)
> {
> +    bool pg_offlined =3D false;
> +
>     ASSERT(mfn_x(mfn) =3D=3D mfn_x(page_to_mfn(pg)));
>=20
>     /*
> @@ -1405,7 +1407,7 @@ static void mark_page_free(struct page_info *pg, mf=
n_t mfn)
>     case PGC_state_offlining:
>         pg->count_info =3D (pg->count_info & PGC_broken) |
>                          PGC_state_offlined;
> -        tainted =3D 1;
> +        pg_offlined =3D true;
>         break;
>=20
>     default:
> @@ -1425,6 +1427,8 @@ static void mark_page_free(struct page_info *pg, mf=
n_t mfn)
>     /* This page is not a guest frame any more. */
>     page_set_owner(pg, NULL); /* set_gpfn_from_mfn snoops pg owner */
>     set_gpfn_from_mfn(mfn_x(mfn), INVALID_M2P_ENTRY);
> +
> +    return pg_offlined;
> }
>=20
> /* Free 2^@order set of pages. */
> @@ -1433,7 +1437,7 @@ static void free_heap_pages(
> {
>     unsigned long mask;
>     mfn_t mfn =3D page_to_mfn(pg);
> -    unsigned int i, node =3D phys_to_nid(mfn_to_maddr(mfn)), tainted =3D=
 0;
> +    unsigned int i, node =3D phys_to_nid(mfn_to_maddr(mfn)), pg_offlined=
 =3D 0;
>     unsigned int zone =3D page_to_zone(pg);
>=20
>     ASSERT(order <=3D MAX_ORDER);
> @@ -1443,7 +1447,8 @@ static void free_heap_pages(
>=20
>     for ( i =3D 0; i < (1 << order); i++ )
>     {
> -        mark_page_free(&pg[i], mfn_add(mfn, i));
> +        if ( mark_page_free(&pg[i], mfn_add(mfn, i)) )
> +            pg_offlined =3D 1;
>=20
>         if ( need_scrub )
>         {
> @@ -1517,7 +1522,7 @@ static void free_heap_pages(
>=20
>     page_list_add_scrub(pg, node, zone, order, pg->u.free.first_dirty);
>=20
> -    if ( tainted )
> +    if ( pg_offlined )
>         reserve_offlined_page(pg);
>=20
>     spin_unlock(&heap_lock);
> --=20
> 2.25.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:51:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:51:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192663.343213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0mv-0001OR-BV; Wed, 22 Sep 2021 11:51:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192663.343213; Wed, 22 Sep 2021 11:51:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0mv-0001OK-87; Wed, 22 Sep 2021 11:51:45 +0000
Received: by outflank-mailman (input) for mailman id 192663;
 Wed, 22 Sep 2021 11:51:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT0mu-0001OA-3G; Wed, 22 Sep 2021 11:51:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT0mt-0006Ey-Tu; Wed, 22 Sep 2021 11:51:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT0mt-0004YL-LF; Wed, 22 Sep 2021 11:51:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mT0mt-0005Po-Ki; Wed, 22 Sep 2021 11:51:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Elu/6RSmn3ofEik/lT5O1xIKWCfT1kAiwE0QkeA2Jdc=; b=Gjo/33qbKsR46so9MSA6LOmha5
	l30f2s6/EJzHDr9pq6a3SkRqJmzNwT/HP7q4Zprp2bkxt5LmukrpFiph1t8/+/uuhxSMTxRAXHRFM
	fDG55KWsRiv/Mk275epwTlpDMPJXi2kzg99fNlh3RuzmjYphlgM74IdPz71t6rdvnN0s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165148-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165148: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=c066cb6dc1da2f2cca853a59a06a5754bf30eb46
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 11:51:43 +0000

flight 165148 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165148/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              c066cb6dc1da2f2cca853a59a06a5754bf30eb46
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  439 days
Failing since        151818  2020-07-11 04:18:52 Z  438 days  429 attempts
Testing same since   165148  2021-09-22 04:18:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 74862 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 11:59:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 11:59:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192671.343227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0uh-0002IN-83; Wed, 22 Sep 2021 11:59:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192671.343227; Wed, 22 Sep 2021 11:59:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0uh-0002IG-3f; Wed, 22 Sep 2021 11:59:47 +0000
Received: by outflank-mailman (input) for mailman id 192671;
 Wed, 22 Sep 2021 11:59:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT0uf-0002IA-FS
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 11:59:45 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 93150dfe-1b9c-11ec-b971-12813bfff9fa;
 Wed, 22 Sep 2021 11:59:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93150dfe-1b9c-11ec-b971-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632311984;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=0TyejDRDGc7iyeKK8R6mgvQf6yLT6LqkSaIMAzfY578=;
  b=b3km3c9fyah/MMadc8Xh7+zMwg9pTVqcFV3CzwJ/V2pbKmCPZ/0C5n5g
   OpsFMbnZlfjf0MGj84ESJsvRQxT5M8TXZtP3Ho9VMTLIXmzZetX46SmEM
   pdMoF4yjv1NYuKLR7d4N/qtmFIIzTqV3tqQ8cey9U1zgmM18YKxczyb5/
   A=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: VynMwQLnE7NCGGc0ftO6ymvXmv1wD1Po1NkQkvdWWQ1elp6vTn+Ac8DyeENdl59A5e06hsq2Mb
 i99IaWAM6vBqV4HkIUweHpATNaLWDOiugffpeGOHsdsbuaTUXnAhPY3WKChaUzdsYsio+95GrD
 lX4rTKxvlHzzQO1KGlh8LhIdyXfKxVmfSjojM+PQlPYQQokmjeF0Yn6t7RB2nLj8QAPwuxEa7T
 dYY+A5H/P06zfz7epDog7OBw1zo7fmZe/8D728XkV/5uVFpLup6RQdhavjh5mXbypFX4CVdPgQ
 jhgQPugiXjEVcNQ+xtm610hn
X-SBRS: 5.1
X-MesageID: 55114958
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:UFFDdK55dYESevzq/gqsxQxRtCbAchMFZxGqfqrLsTDasY5as4F+v
 mUaDG3XMvqKMWvwc9p/PYW1o05UucOBmIVmTFBt+X02Hi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NQw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Zw
 +lOlJq6RwYTOZLSvLgibzJnKAFuFPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWpp35wfQKa2i
 8wxQyZtS036bwF0E14wA8McxuyJllzzfGgNwL6SjfVuuDWCpOBr65DyNPLFd9rMQt9a9m6Iq
 2SD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQ3vxgjUCXx2cTIAYLTlb9qv684nNSQPoGd
 RZSoHB36/Fvqgr7FbERQiFUvlaPgh09XdBeEtc91x+plpXoyDmiJngtG2sphMMdiCMmedA7/
 gbXxImwVWIz6OT9pWG1rejP/GjrUcQBBSpbP3ZVE1FdizX2iNxr1nryosBf/LlZZzEfMQr5x
 SyD5AM6jq8a5SLg//TmpQ2b695AS56gc+LU2uk1djn+hu+aTNT8D2BN1bQ8xawbRLt1tnHb4
 BA5dzG2tYji961hcRCwrBglRun1t55pzwEwcXYwRsJ8plxBClaIfJxK4SEWGXqFxv0sIGezC
 GeK4Fs5zMYKYBOCMP8mC6rsWp9C5fWxSrzYugX8M4Mmjm5ZL1TcokmDpCe4ggjQraTbuftuY
 cvEKZn8Uyly5GYO5GPeetrxGIQDn0gW7WjSWYr631Kg17+fb2SSUrALLB2FaeVR0U9OiFy9H
 w93O5TYxhNBfvf5ZyWLo4cfIUpTdSowBIzsqtwRfemGe1I0FGYkAv7X4LUgZ406wPgFyraWp
 imwCh1C1V7ypXzbMgHWOHptX6ziAMRkpnUhMC1yYVvxgyo/YZyi5bs0focseeV17/RqyPN5F
 qFXe8iJDvlVZC7A/jARMcv0oIB4LUz5jgOSJSu1JjM4esc4FQDO/9bleCrp9TUPUXXr5Zdv/
 eX421qCE5QZRglkAMLHU96Vzgu87SoHheZ/f0rUOd0PKk/ix5dndn7qhfgtLsBSdRianmmG1
 xybCAszrPXWp9Nn68HAgK2Jotv7E+Z6GUYGTWDX4azvaHvf92unh4RBTPyJbXbWU2atoPeuY
 uBczvfdNvwbnQkV79ogQugzla9utcHyo7J6zxh/GCSZZlumPbpsP32a0JQdraZK3LJY5VO7V
 0/nFgO24llV1BcJyGIsGTc=
IronPort-HdrOrdr: A9a23:H+QAG6gbirak4j2nIJP0qPenVXBQX0F13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskqKdxbNhR4tKPTOWw1dASbsN0WKM+UyDJ8STzJ856U
 4kSdkCNDSSNykFsS+Z2njALz9I+rDum8rJ9ISuvkuFDzsaE52Ihz0JdTpzeXcGIjWua6BJcK
 Z1saF81kadkDksH46GL0hAe9KGi8zAlZrgbxJDLxk76DOWhTftzLLhCRCX0joXTjsKmN4ZgC
 T4uj28wp/mn+Cwyxfa2WOWx5NKmOH5wt8GIMCXkMAaJhjllw7tToV8XL+puiwzvYiUmRsXue
 iJhy1lE9V46nvXcG3wiRzx2zP42DJr0HPmwU/wuwqrneXJABYBT+ZRj4NQdRXUr2A6ustn7a
 5N12WF87JKEBLphk3Glpn1fiAvsnDxjWspkOYVgXAae5AZcqVtoYsW+14QOIscHRj99JssHI
 BVfY/hDc5tABCnhk3izytSKITGZAV3Iv7GeDlMhiWt6UkXoJgjpHFogPD2nR87heQAotd/lq
 P52gkBrsA6ciYsV9MPOA42e7rBNoX8e2O9DIusGyWUKEgmAQOEl3el2sR/2AmVEKZ4uKfa3q
 6xFm9liQ==
X-IronPort-AV: E=Sophos;i="5.85,313,1624334400"; 
   d="scan'208";a="55114958"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jCfynWNEghLVfmU6NvvcQq3t5d09L1ulPV3NJL40J/tMe2jV+TjIB0+CR6F1vtSn06kLndCdwFbo7m/IMjfaTe57C9Cb/ZbHr/jd8sSgOaVN3hKC8Wqbio8PENPsxd7NOmfKw1yOFm6iwREiw9+1oxQhkMCbcQSHWO8j+rpHBhc3NzN8Qq4lTTZBDB5sN3+/mLbUDeYwOijZbWOJqd7fBxITbR+uXlXoq7pOLYUaSd/4POGScWsUPWTQDgCABIDWQ8Tx4WjrylUGdq8az7tKQnEc+bPB5aJHZetnt8ARyHx5OPgmbFRvmRiVVq480cLZ2u5VGfUyxyZzBMTmR+XadA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jcxbyfjmLXpGC4OTqSxH5SWNRDCeJod75k5ifshS7ME=;
 b=DGrrB3zLtSSG3G+mPLfmzA6lelxCJEQ9GWEVq4KKdBKyIq9LeBFygm28I+XqG4c7fBYwzVsi6nYlZOrjgWOFZis/9pG/3ihgydV1fhs49MdpIQo5zFAiuTEoMa85c8pxSLx/aUPXS/lSP21lF9mJpU8XDFPY3OTe4d6PNWYSTK1PXZh5kGEJh8u2epmEehAspIaUl2GLQNkZ9u/UbqfqLneOebrrZVX98eXgOoUR1bRggv/jQMTuSOT9mRj6QacQZvHNjDLk74GTEjW421GvIwoggtLT9yV5dET4kPmjJYWYtjPBUskSDpVXTAtG36x4u4IjETms/n91M9ZEP9ivlw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jcxbyfjmLXpGC4OTqSxH5SWNRDCeJod75k5ifshS7ME=;
 b=ssGdBxmq9cKWk+XjBm1fCO94wek++uLDO0Khz7cSx+jAKllGY1E9LJjfIy1JuD1t/PWPFpJmowhpt1O3Df7QgShy7ITHiva6dVfFOs1MPn9ZRmRwtir2GtxGyCdN4+++XqED0QAq8CssNxJC+pwjBRqRZ71ksw5HOqVVxeTVcYY=
Date: Wed, 22 Sep 2021 13:59:33 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 1/9] x86/PVH: improve Dom0 memory size calculation
Message-ID: <YUsapfsfoL+ODa6+@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <562a9ee5-102e-ee25-e4cd-721e44b7409a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <562a9ee5-102e-ee25-e4cd-721e44b7409a@suse.com>
X-ClientProxiedBy: LO4P123CA0207.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a5::14) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 198073da-c64b-4fc8-5c12-08d97dc0743b
X-MS-TrafficTypeDiagnostic: DM5PR03MB2972:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB29721084AC4D6E8B1DD71E398FA29@DM5PR03MB2972.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2958;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: S8U07EXYl1veRrXjNCgPS7/kJ9xfo9zVs2QrXX6FCN+/3/jbxpN0IpmdULMDngcyMiQwNi9LEiudVJYnrh5/KLB+pIyE0h20ieFlJdLY5G48majzldR2cjugYgGuq43m9Nt3JbuMVAYRx4/G+a+QpjhGdcyX6HNJyH5jSPEPIVUpJQyzjlKBhIcYW2CYWMUROTnQj1Z0hrLaVLmoSb3BoCRix+QvOB055nOKX/E3g7qfypXZYrdkrFOlwzzwaHpa2saNW8hrpyIhFl5jClWmfc+IYL8MW1sBpTlHpd2MLDp3/ENo9FgwaYEmr6g9IUC0x8fc/ZhldcECe6xme4kpVQur6pkw4NM7Xkold0dbabUawlqKEe+64Nc34yOr0ARW5/DNpznIlAQlkG1Uib7FQP7WflzYQvp2s8HkoPRBqHmaaGG2zpGooLntVnxxt9VExqowmtZoTR8sA+aTgOz+xLoFQI4vmGqh4MxMmpzSEe9kcu0KJ8wAivtnk4llLDxf2+DJIBangm9X5JXN+y/g4GTSAAV3jTA+7spOIph4lMfBiSxhC0UXPTMA501snEsyPyjQG2sQA5Cf6EMLL/qoF1st70+Ag8rDrN+2j9FWryibV0/0KiKEOTX+K994k+/pmhF6svjm7Gnw08tKsNzaZoqqZk7JPKFFUWND5umurKaADPZ42pIcdwnB/aDnlwjrOLwEqwufpbnAucoSDi+0SQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(9686003)(2906002)(83380400001)(5660300002)(6486002)(8676002)(8936002)(6666004)(85182001)(956004)(186003)(316002)(54906003)(6496006)(38100700002)(26005)(66946007)(6916009)(508600001)(4326008)(86362001)(66556008)(66476007)(25903002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NjJtanoyU0s0ME16Vmw2TlB2dWJVUlFqdzVDRVpsckZCV1FiSkhaNjMrL1k1?=
 =?utf-8?B?cEdZakZuNmNqUDBtRU8vbDFWRmczY3RrVVN5VitRbkNJWDB2WkV0bHV4VXNv?=
 =?utf-8?B?bjl6NjB3bUpyYzc3S0EyVUE1TlE4K3ZyMHc5RzhFRFN2SC9TUWxOZlFMNXhS?=
 =?utf-8?B?dnVwUW50QW1qbkpnUjZ6ZytnYU01c0dHTDJLaldGQWVGUk0wQzN4c2FWY1VL?=
 =?utf-8?B?dzJaYmd1UmltUThKdkpSTHBvSFF0SEJiZGlUVHc1QlVkaUFKTmd4VWQrT1FV?=
 =?utf-8?B?QnFudmxLbHFVNDFFR0lFSnZEWk9xZ0xZUVVSS21TekxJWWlWRWFOdUsxOGlM?=
 =?utf-8?B?RlZud2tHSEtGUDJJODVGOHlXSVJnNUFTdjFOSjNadTlsWjNTNEd1emZWbDU1?=
 =?utf-8?B?YmZGTGxPY1YzSFNXZnRVOTgyakpiQ0pPaEFhL1BhWWRVc0tzOUl1M3hOTWxY?=
 =?utf-8?B?cnZkQVhNUGx6U2EvUVlHVlgxaUxUWjVzaUVRcmtzTnNTTGh6S2RqYW5rUkRB?=
 =?utf-8?B?S3NRemJtZzNGTWRaNVpkdm5aZFJDdzhyK0Vlc3FSakk3MzVBUXZGMUp1U1Mw?=
 =?utf-8?B?dWZGTTUwVHRPaVp0aDZvejJZU3pVMW00dVlnZWlKRldrUEhmUnRxazg1RG1l?=
 =?utf-8?B?WWFEb2MrTkhTLzQxc01wM3Vhb0luaUxBYmFGdG1HNCtpSDNnR1lUcDlqQUNm?=
 =?utf-8?B?ZUh3QWpSZ3ppbnc4d2pnRkxla1B4SDZadGRxWG9KS1ZWaTJpYnVPeGNtUzdG?=
 =?utf-8?B?eVpONWcwY0syelFEa1ZaMzlNMUdOUVBlVzU3QmdaRmhSMHNwaWVHWnMyMnNh?=
 =?utf-8?B?Um5BY0NNaW42ZC8vTE9MMVVUaXhpVVBpZHl4Y2NOckNvSXZCUkN6aEI1OWhi?=
 =?utf-8?B?cXU4WUpOcXcwOTVNWHBRQkxtbmxkNkxiNy9ITmU2ZDg4emwyNG5nRExsUWhD?=
 =?utf-8?B?RlROSGZSVi9ORVJqK2NXeUs2dHlBT3FpSWpJSVU2YU5Yclc3aCt5RWJoRjE3?=
 =?utf-8?B?c0tRTS9aanJTczZVVG1FZDViREtUbmo0TlJLYU91UkxCQmpXaGRpQ0JudWh4?=
 =?utf-8?B?L1BmdnY4cjNPK1JvcWxDZngwVmN6djhNWmljUmVNRjdlYzVpQVdQYkVLZmww?=
 =?utf-8?B?cnkzdDNjRG91NHgrM0xlODBDRzNML1IrVWd2cmwybDJEUWVnWUxtVG5CdVRG?=
 =?utf-8?B?NHdXUGVycmxCZmlmblRPZ0RrNFZxeFY5N28reWMrRXBLUW85YkRnS1U1Uk40?=
 =?utf-8?B?OGR0Uy9jRmU3M2piTXozK1NFUTBlYWJmNGdlUitFMzArcXdEeDlWRlJRd29s?=
 =?utf-8?B?Um9sbUxJRXB0RzZpWTMyTXlNNklGQkxHQ0lyUjlqY0ZvRUN3VkxsZ0tNcXNH?=
 =?utf-8?B?V0QrNHN2bnRIUzUveW9hV05vVFhvTHo0VVlpSjc5QnpJc0g5azBIcW1KNmxJ?=
 =?utf-8?B?bXlhL1lTUU16cG1xMGVqYjBtQXNGMlN0RU95V3NDM2VBUys4bnk4OWI4MENG?=
 =?utf-8?B?WmR0TmF3cEJPbDhlYURjbkZCNTE4Mk9meFFBaTUyMHVUenZWS1h6eUFNelBK?=
 =?utf-8?B?dWlXWkhNQXU3Uk8rTFVhQnQ3aWkvRE9TMzl1d1pWbExjblZ5WDFQa3lLYjRW?=
 =?utf-8?B?YjNSVHh5eGxYNFFsckZTeHoyaHdRaGVXbXB3NnZSY2s3Q3AyaXBGNUxwZjRa?=
 =?utf-8?B?b1EwNm1NSmUrbTRCM0kraVo3RTVqSGhtck41azJ1WWRqRFNvTlUvN3dmN3J1?=
 =?utf-8?Q?BBZx4DbxlbohwnOGLEi2kk6FVdol1sGQqA4Vdni?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 198073da-c64b-4fc8-5c12-08d97dc0743b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 11:59:38.8827
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IHT1HVq8ak4RBmJk4RPdZwe4ywJzTF2C5gvW9h1HqyDwtOZJD/U3svkGwiFxQqyygNZ5542ztCqcV39eW4dwPA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2972
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:16:44AM +0200, Jan Beulich wrote:
> Assuming that the accounting for IOMMU page tables will also take care
> of the P2M needs was wrong: dom0_paging_pages() can determine a far
> higher value, high enough for the system to run out of memory while
> setting up Dom0. Hence in the case of shared page tables the larger of
> the two values needs to be used (without shared page tables the sum of
> both continues to be applicable).
> 
> While there also account for two further aspects in the PV case: With
> "iommu=dom0-passthrough" no IOMMU page tables would get allocated, so
> none need accounting for. And if shadow mode is to be enabled, setting
> aside a suitable amount for the P2M pool to get populated is also
> necessary (i.e. similar to the non-shared-page-tables case of PVH).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -318,7 +318,7 @@ unsigned long __init dom0_compute_nr_pag
>      struct domain *d, struct elf_dom_parms *parms, unsigned long initrd_len)
>  {
>      nodeid_t node;
> -    unsigned long avail = 0, nr_pages, min_pages, max_pages;
> +    unsigned long avail = 0, nr_pages, min_pages, max_pages, iommu_pages = 0;
>      bool need_paging;
>  
>      /* The ordering of operands is to work around a clang5 issue. */
> @@ -337,18 +337,23 @@ unsigned long __init dom0_compute_nr_pag
>          avail -= d->max_vcpus - 1;
>  
>      /* Reserve memory for iommu_dom0_init() (rough estimate). */
> -    if ( is_iommu_enabled(d) )
> +    if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
>      {
>          unsigned int s;
>  
>          for ( s = 9; s < BITS_PER_LONG; s += 9 )
> -            avail -= max_pdx >> s;
> +            iommu_pages += max_pdx >> s;
> +
> +        avail -= iommu_pages;
>      }
>  
> -    need_paging = is_hvm_domain(d) &&
> -        (!iommu_use_hap_pt(d) || !paging_mode_hap(d));
> +    need_paging = is_hvm_domain(d)
> +                  ? !iommu_use_hap_pt(d) || !paging_mode_hap(d)
> +                  : opt_dom0_shadow;
>      for ( ; ; need_paging = false )
>      {
> +        unsigned long paging_pages;
> +
>          nr_pages = get_memsize(&dom0_size, avail);
>          min_pages = get_memsize(&dom0_min_size, avail);
>          max_pages = get_memsize(&dom0_max_size, avail);
> @@ -377,11 +382,20 @@ unsigned long __init dom0_compute_nr_pag
>          nr_pages = min(nr_pages, max_pages);
>          nr_pages = min(nr_pages, avail);
>  
> -        if ( !need_paging )
> -            break;
> +        paging_pages = paging_mode_enabled(d) || need_paging
> +                       ? dom0_paging_pages(d, nr_pages) : 0;
>  
>          /* Reserve memory for shadow or HAP. */
> -        avail -= dom0_paging_pages(d, nr_pages);
> +        if ( !need_paging )
> +        {
> +            if ( paging_pages <= iommu_pages )
> +                break;
> +
> +            avail -= paging_pages - iommu_pages;
> +        }
> +        else
> +            avail -= paging_pages;
> +        iommu_pages = paging_pages;
>      }

I always found this loop extremely confusing to reason about. Now that
we account for the iommu page tables using separate logic, do we
really need a loop here?

In fact I would suggest something like:

unsigned long cpu_pages = 0;

if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
{
    unsigned int s;

    for ( s = 9; s < BITS_PER_LONG; s += 9 )
        iommu_pages += max_pdx >> s;
}

[perform all the nr_pages adjustments]

if ( paging_mode_enabled(d) ||
     opt_dom0_shadow /* shadow paging gets enabled later for PV dom0. */ )
    cpu_pages = dom0_paging_pages(d, nr_pages);

if ( is_hvm_domain(d) && iommu_use_hap_pt(d) && paging_mode_hap(d) )
    avail -= max(iommu_pages, cpu_pages);
else
    avail -= cpu_pages + iommu_pages;

There will be a slight over estimation of cpu_pages, as the value
passed in doesn't account for the iommu pages in case they are used,
but still it's better to over estimate than to under estimate.

Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:04:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:04:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192681.343238 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0yu-0003m1-Uu; Wed, 22 Sep 2021 12:04:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192681.343238; Wed, 22 Sep 2021 12:04:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT0yu-0003lu-Rg; Wed, 22 Sep 2021 12:04:08 +0000
Received: by outflank-mailman (input) for mailman id 192681;
 Wed, 22 Sep 2021 12:04:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2gtc=OM=bombadil.srs.infradead.org=BATV+abbb41f0cefee8b8c822+6604+infradead.org+dwmw2@srs-us1.protection.inumbo.net>)
 id 1mT0ys-0003lo-6x
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:04:07 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8877f6ef-f013-4250-af35-40dd91819b10;
 Wed, 22 Sep 2021 12:04:05 +0000 (UTC)
Received: from [2001:8b0:10b:1::3ae] (helo=u3832b3a9db3152.infradead.org)
 by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux))
 id 1mT0yi-008Lp5-BX; Wed, 22 Sep 2021 12:03:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8877f6ef-f013-4250-af35-40dd91819b10
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=MIME-Version:Content-Type:References:
	In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=eHtXG/WYrcNH0lI4V2Q2GjTaV9btIIN6SVJHQb0lYHA=; b=T+irslYHM10iC/TgLWvms06B0D
	htLpACr1WbseCgZfX801ger5NuT/CyvEJ8KKoDiAs+uWiSThMYM+LHqr14v68/Kt9EEXASRKEHhSz
	srj1ocYEJAh+fvDuP66iT8dZKBiVu+TgHBy8ieIy+1e9y0cxHsyHh62ENBNEanIOASYtCtLSxxEs/
	rDW+ZLGSE65fdNutwUsScRSVn4jCtogRHt84O+OWtzVulBQ6SNCg8VzkRb8GP4U1kaMf+pFCQylJj
	zGWY74kCyJeeJaurLXiz0NuKNC0E/g0ChzN//fnza4EzWmw4SuX94U1MYFN4e4dqkZYCDDZn55I/U
	gvFOh6mw==;
Message-ID: <04272e87a8939be46acddd3c75bbffa84b0a40c1.camel@infradead.org>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
From: David Woodhouse <dwmw2@infradead.org>
To: Alex =?ISO-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>, Mathieu Poirier
 <mathieu.poirier@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>, Wei
 Liu <wl@xen.org>, Alexandru Agache <aagch@amazon.com>
Cc: "rust-vmm@lists.opendev.org" <rust-vmm@lists.opendev.org>, 
 xen-devel@lists.xenproject.org, Stratos Mailing List
 <stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
 <olekstysh@gmail.com>,  Stefano Stabellini <stefano.stabellini@xilinx.com>
Date: Wed, 22 Sep 2021 13:03:52 +0100
In-Reply-To: <87lf40vay1.fsf@linaro.org>
References: <87lf40vay1.fsf@linaro.org>
Content-Type: multipart/signed; micalg="sha-256"; protocol="application/pkcs7-signature";
	boundary="=-o8kMpVUAVB49qRJl6kfL"
User-Agent: Evolution 3.36.5-0ubuntu1 
MIME-Version: 1.0
X-SRS-Rewrite: SMTP reverse-path rewritten from <dwmw2@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html


--=-o8kMpVUAVB49qRJl6kfL
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-13 at 13:44 +0100, Alex Benn=C3=A9e wrote:
> Hi,
>=20
> As we consider the next cycle for Project Stratos I would like to make
> some more progress on hypervisor agnosticism for our virtio backends.
> While we have implemented a number of virtio vhost-user backends using C
> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
> backends does it make sense to do some enabling work in rust-vmm to
> support Xen?

I like this idea.

Somewhat separately, Alex Agache has already started some preliminary
hacking on supporting Xen guests within rust-vmm (on top of Linux/KVM):
https://github.com/alexandruag/vmm-reference/commits/xen

Being able to run on *actual* Xen would be good too. And we should also
aspire to do guest-transparent live migration between the two hosting
environments.

Where relevant, it would be great to be able to share components (like
emulation of the Xen PCI platform device, a completely single-tenant
XenStore implementation dedicated to a single guest, perhaps PV
netback/blkback and other things).=20



--=-o8kMpVUAVB49qRJl6kfL
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Disposition: attachment; filename="smime.p7s"
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCECow
ggUcMIIEBKADAgECAhEA4rtJSHkq7AnpxKUY8ZlYZjANBgkqhkiG9w0BAQsFADCBlzELMAkGA1UE
BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgG
A1UEChMRQ09NT0RPIENBIExpbWl0ZWQxPTA7BgNVBAMTNENPTU9ETyBSU0EgQ2xpZW50IEF1dGhl
bnRpY2F0aW9uIGFuZCBTZWN1cmUgRW1haWwgQ0EwHhcNMTkwMTAyMDAwMDAwWhcNMjIwMTAxMjM1
OTU5WjAkMSIwIAYJKoZIhvcNAQkBFhNkd213MkBpbmZyYWRlYWQub3JnMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAsv3wObLTCbUA7GJqKj9vHGf+Fa+tpkO+ZRVve9EpNsMsfXhvFpb8
RgL8vD+L133wK6csYoDU7zKiAo92FMUWaY1Hy6HqvVr9oevfTV3xhB5rQO1RHJoAfkvhy+wpjo7Q
cXuzkOpibq2YurVStHAiGqAOMGMXhcVGqPuGhcVcVzVUjsvEzAV9Po9K2rpZ52FE4rDkpDK1pBK+
uOAyOkgIg/cD8Kugav5tyapydeWMZRJQH1vMQ6OVT24CyAn2yXm2NgTQMS1mpzStP2ioPtTnszIQ
Ih7ASVzhV6csHb8Yrkx8mgllOyrt9Y2kWRRJFm/FPRNEurOeNV6lnYAXOymVJwIDAQABo4IB0zCC
Ac8wHwYDVR0jBBgwFoAUgq9sjPjF/pZhfOgfPStxSF7Ei8AwHQYDVR0OBBYEFLfuNf820LvaT4AK
xrGK3EKx1DE7MA4GA1UdDwEB/wQEAwIFoDAMBgNVHRMBAf8EAjAAMB0GA1UdJQQWMBQGCCsGAQUF
BwMEBggrBgEFBQcDAjBGBgNVHSAEPzA9MDsGDCsGAQQBsjEBAgEDBTArMCkGCCsGAQUFBwIBFh1o
dHRwczovL3NlY3VyZS5jb21vZG8ubmV0L0NQUzBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3Js
LmNvbW9kb2NhLmNvbS9DT01PRE9SU0FDbGllbnRBdXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWls
Q0EuY3JsMIGLBggrBgEFBQcBAQR/MH0wVQYIKwYBBQUHMAKGSWh0dHA6Ly9jcnQuY29tb2RvY2Eu
Y29tL0NPTU9ET1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1haWxDQS5jcnQwJAYI
KwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmNvbW9kb2NhLmNvbTAeBgNVHREEFzAVgRNkd213MkBpbmZy
YWRlYWQub3JnMA0GCSqGSIb3DQEBCwUAA4IBAQALbSykFusvvVkSIWttcEeifOGGKs7Wx2f5f45b
nv2ghcxK5URjUvCnJhg+soxOMoQLG6+nbhzzb2rLTdRVGbvjZH0fOOzq0LShq0EXsqnJbbuwJhK+
PnBtqX5O23PMHutP1l88AtVN+Rb72oSvnD+dK6708JqqUx2MAFLMevrhJRXLjKb2Mm+/8XBpEw+B
7DisN4TMlLB/d55WnT9UPNHmQ+3KFL7QrTO8hYExkU849g58Dn3Nw3oCbMUgny81ocrLlB2Z5fFG
Qu1AdNiBA+kg/UxzyJZpFbKfCITd5yX49bOriL692aMVDyqUvh8fP+T99PqorH4cIJP6OxSTdxKM
MIIFHDCCBASgAwIBAgIRAOK7SUh5KuwJ6cSlGPGZWGYwDQYJKoZIhvcNAQELBQAwgZcxCzAJBgNV
BAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAY
BgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRo
ZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTE5MDEwMjAwMDAwMFoXDTIyMDEwMTIz
NTk1OVowJDEiMCAGCSqGSIb3DQEJARYTZHdtdzJAaW5mcmFkZWFkLm9yZzCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBALL98Dmy0wm1AOxiaio/bxxn/hWvraZDvmUVb3vRKTbDLH14bxaW
/EYC/Lw/i9d98CunLGKA1O8yogKPdhTFFmmNR8uh6r1a/aHr301d8YQea0DtURyaAH5L4cvsKY6O
0HF7s5DqYm6tmLq1UrRwIhqgDjBjF4XFRqj7hoXFXFc1VI7LxMwFfT6PStq6WedhROKw5KQytaQS
vrjgMjpICIP3A/CroGr+bcmqcnXljGUSUB9bzEOjlU9uAsgJ9sl5tjYE0DEtZqc0rT9oqD7U57My
ECIewElc4VenLB2/GK5MfJoJZTsq7fWNpFkUSRZvxT0TRLqznjVepZ2AFzsplScCAwEAAaOCAdMw
ggHPMB8GA1UdIwQYMBaAFIKvbIz4xf6WYXzoHz0rcUhexIvAMB0GA1UdDgQWBBS37jX/NtC72k+A
CsaxitxCsdQxOzAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEF
BQcDBAYIKwYBBQUHAwIwRgYDVR0gBD8wPTA7BgwrBgEEAbIxAQIBAwUwKzApBggrBgEFBQcCARYd
aHR0cHM6Ly9zZWN1cmUuY29tb2RvLm5ldC9DUFMwWgYDVR0fBFMwUTBPoE2gS4ZJaHR0cDovL2Ny
bC5jb21vZG9jYS5jb20vQ09NT0RPUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFp
bENBLmNybDCBiwYIKwYBBQUHAQEEfzB9MFUGCCsGAQUFBzAChklodHRwOi8vY3J0LmNvbW9kb2Nh
LmNvbS9DT01PRE9SU0FDbGllbnRBdXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWlsQ0EuY3J0MCQG
CCsGAQUFBzABhhhodHRwOi8vb2NzcC5jb21vZG9jYS5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5m
cmFkZWFkLm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAC20spBbrL71ZEiFrbXBHonzhhirO1sdn+X+O
W579oIXMSuVEY1LwpyYYPrKMTjKECxuvp24c829qy03UVRm742R9Hzjs6tC0oatBF7KpyW27sCYS
vj5wbal+TttzzB7rT9ZfPALVTfkW+9qEr5w/nSuu9PCaqlMdjABSzHr64SUVy4ym9jJvv/FwaRMP
gew4rDeEzJSwf3eeVp0/VDzR5kPtyhS+0K0zvIWBMZFPOPYOfA59zcN6AmzFIJ8vNaHKy5QdmeXx
RkLtQHTYgQPpIP1Mc8iWaRWynwiE3ecl+PWzq4i+vdmjFQ8qlL4fHz/k/fT6qKx+HCCT+jsUk3cS
jDCCBeYwggPOoAMCAQICEGqb4Tg7/ytrnwHV2binUlYwDQYJKoZIhvcNAQEMBQAwgYUxCzAJBgNV
BAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAY
BgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMSswKQYDVQQDEyJDT01PRE8gUlNBIENlcnRpZmljYXRp
b24gQXV0aG9yaXR5MB4XDTEzMDExMDAwMDAwMFoXDTI4MDEwOTIzNTk1OVowgZcxCzAJBgNVBAYT
AkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNV
BAoTEUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRoZW50
aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAvrOeV6wodnVAFsc4A5jTxhh2IVDzJXkLTLWg0X06WD6cpzEup/Y0dtmEatrQPTRI5Or1u6zf
+bGBSyD9aH95dDSmeny1nxdlYCeXIoymMv6pQHJGNcIDpFDIMypVpVSRsivlJTRENf+RKwrB6vcf
WlP8dSsE3Rfywq09N0ZfxcBa39V0wsGtkGWC+eQKiz4pBZYKjrc5NOpG9qrxpZxyb4o4yNNwTqza
aPpGRqXB7IMjtf7tTmU2jqPMLxFNe1VXj9XB1rHvbRikw8lBoNoSWY66nJN/VCJv5ym6Q0mdCbDK
CMPybTjoNCQuelc0IAaO4nLUXk0BOSxSxt8kCvsUtQIDAQABo4IBPDCCATgwHwYDVR0jBBgwFoAU
u69+Aj36pvE8hI6t7jiY7NkyMtQwHQYDVR0OBBYEFIKvbIz4xf6WYXzoHz0rcUhexIvAMA4GA1Ud
DwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMBEGA1UdIAQKMAgwBgYEVR0gADBMBgNVHR8E
RTBDMEGgP6A9hjtodHRwOi8vY3JsLmNvbW9kb2NhLmNvbS9DT01PRE9SU0FDZXJ0aWZpY2F0aW9u
QXV0aG9yaXR5LmNybDBxBggrBgEFBQcBAQRlMGMwOwYIKwYBBQUHMAKGL2h0dHA6Ly9jcnQuY29t
b2RvY2EuY29tL0NPTU9ET1JTQUFkZFRydXN0Q0EuY3J0MCQGCCsGAQUFBzABhhhodHRwOi8vb2Nz
cC5jb21vZG9jYS5jb20wDQYJKoZIhvcNAQEMBQADggIBAHhcsoEoNE887l9Wzp+XVuyPomsX9vP2
SQgG1NgvNc3fQP7TcePo7EIMERoh42awGGsma65u/ITse2hKZHzT0CBxhuhb6txM1n/y78e/4ZOs
0j8CGpfb+SJA3GaBQ+394k+z3ZByWPQedXLL1OdK8aRINTsjk/H5Ns77zwbjOKkDamxlpZ4TKSDM
KVmU/PUWNMKSTvtlenlxBhh7ETrN543j/Q6qqgCWgWuMAXijnRglp9fyadqGOncjZjaaSOGTTFB+
E2pvOUtY+hPebuPtTbq7vODqzCM6ryEhNhzf+enm0zlpXK7q332nXttNtjv7VFNYG+I31gnMrwfH
M5tdhYF/8v5UY5g2xANPECTQdu9vWPoqNSGDt87b3gXb1AiGGaI06vzgkejL580ul+9hz9D0S0U4
jkhJiA7EuTecP/CFtR72uYRBcunwwH3fciPjviDDAI9SnC/2aPY8ydehzuZutLbZdRJ5PDEJM/1t
yZR2niOYihZ+FCbtf3D9mB12D4ln9icgc7CwaxpNSCPt8i/GqK2HsOgkL3VYnwtx7cJUmpvVdZ4o
gnzgXtgtdk3ShrtOS1iAN2ZBXFiRmjVzmehoMof06r1xub+85hFQzVxZx5/bRaTKTlL8YXLI8nAb
R9HWdFqzcOoB/hxfEyIQpx9/s81rgzdEZOofSlZHynoSMYIDyjCCA8YCAQEwga0wgZcxCzAJBgNV
BAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAY
BgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMT0wOwYDVQQDEzRDT01PRE8gUlNBIENsaWVudCBBdXRo
ZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEA4rtJSHkq7AnpxKUY8ZlYZjANBglghkgB
ZQMEAgEFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjEw
OTIyMTIwMzUyWjAvBgkqhkiG9w0BCQQxIgQgJ7ABb2e8/Yk1fm91qng4kBuh+Z88w6OxfzVJVBLV
hAEwgb4GCSsGAQQBgjcQBDGBsDCBrTCBlzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIg
TWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQx
PTA7BgNVBAMTNENPTU9ETyBSU0EgQ2xpZW50IEF1dGhlbnRpY2F0aW9uIGFuZCBTZWN1cmUgRW1h
aWwgQ0ECEQDiu0lIeSrsCenEpRjxmVhmMIHABgsqhkiG9w0BCRACCzGBsKCBrTCBlzELMAkGA1UE
BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgG
A1UEChMRQ09NT0RPIENBIExpbWl0ZWQxPTA7BgNVBAMTNENPTU9ETyBSU0EgQ2xpZW50IEF1dGhl
bnRpY2F0aW9uIGFuZCBTZWN1cmUgRW1haWwgQ0ECEQDiu0lIeSrsCenEpRjxmVhmMA0GCSqGSIb3
DQEBAQUABIIBAIATTFBHxMFISdIF5shrs3MDc3HuVNGp7sGQz74tEXZwSqpt4On2bistYYpa52NS
Gqi0HEuU9hcSTZry70LTMJ107eUMT53IzLbe0tn/sQ6tPrrNlePoiJMYLbFkNcFZ3s2faJp2vJ+j
dROX5Wz1rjOrGmGIUwSdFjaZuQC1s5+OwFqWneBdjPCmuLU2WoBWAJ35Cj7eF4DKWpzy/U9ZEoTY
Xllwwen7hdssRQAV1pCT/Mn7bWP8ysnFwvsXbS4iyoVrF/YCHoAzv+fd91MFjJcUovBrRf3H0zQg
ufK3whiOm6BvZ40uwK17evdhc397Lj35CO/JuVRh7zTiqJ4XuCUAAAAAAAA=


--=-o8kMpVUAVB49qRJl6kfL--



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:23:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:23:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192687.343249 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1H3-0006MY-Hm; Wed, 22 Sep 2021 12:22:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192687.343249; Wed, 22 Sep 2021 12:22:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1H3-0006MR-DW; Wed, 22 Sep 2021 12:22:53 +0000
Received: by outflank-mailman (input) for mailman id 192687;
 Wed, 22 Sep 2021 12:22:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT1H2-0006MH-B6; Wed, 22 Sep 2021 12:22:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT1H2-0006pe-3X; Wed, 22 Sep 2021 12:22:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT1H1-0005gI-S3; Wed, 22 Sep 2021 12:22:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mT1H1-0003Wu-RZ; Wed, 22 Sep 2021 12:22:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jPq7BjzryaG9KaDKcyyzs6mSdE1f7vnoLGAtMPZffWE=; b=m4qK8U0xSoz4A+0qZPIf/k3RYz
	vqfkXcS8AYq5eO7Im6HKcrTquZb96biwgj2qG2rCP0nJ5rwNGlid/XmYjRkwMRdQ2O2QQi16x5JRy
	JPslxff7lHUZgyWfmh4oTDpZFctJTlIgEmk8h/OD3IBL/ivHkvf/RgRhsPlhB5WLlMd8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165151-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165151: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=f334c5a41dc4702b1c767c6614025aa2f802f9a4
X-Osstest-Versions-That:
    ovmf=542cba73d21b7bcc1e3852a9d9843d5fffc2d173
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 12:22:51 +0000

flight 165151 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165151/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 f334c5a41dc4702b1c767c6614025aa2f802f9a4
baseline version:
 ovmf                 542cba73d21b7bcc1e3852a9d9843d5fffc2d173

Last test of basis   165113  2021-09-18 16:11:19 Z    3 days
Testing same since   165151  2021-09-22 09:30:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Zhang Xiaoqiang <xiaoqiang.zhang@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   542cba73d2..f334c5a41d  f334c5a41dc4702b1c767c6614025aa2f802f9a4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:24:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:24:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192694.343263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1J0-0006zD-Uj; Wed, 22 Sep 2021 12:24:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192694.343263; Wed, 22 Sep 2021 12:24:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1J0-0006z6-QR; Wed, 22 Sep 2021 12:24:54 +0000
Received: by outflank-mailman (input) for mailman id 192694;
 Wed, 22 Sep 2021 12:24:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT1Iz-0006z0-K3
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:24:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 15b782f2-1ba0-11ec-b973-12813bfff9fa;
 Wed, 22 Sep 2021 12:24:52 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-QGSkeAv_OviTZR1esa0JjQ-1; Wed, 22 Sep 2021 14:24:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3774.eurprd04.prod.outlook.com (2603:10a6:803:1f::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Wed, 22 Sep
 2021 12:24:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 12:24:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR10CA0087.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:15::40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Wed, 22 Sep 2021 12:24:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15b782f2-1ba0-11ec-b973-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632313491;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=bCMh9bDphYY+wjeyyZf9GP7gqy+UqEdPNOJx6gA/LDc=;
	b=a/lVFkF2/rtg/mkuptO2M51pLkcmnWmKUCYzHb5xKuc01abkPWCFDnNIcx0I1LNbd2vs5C
	LA7FjV9emy/GKfk4JCJdtu6xVIJ6aSw3OeGoz7yiyww/JaKzdAQkQwVomBHaoeB/u/Xf3b
	wMDmYP2cTOW6Q/Wj3tyT/sO05YH+xt4=
X-MC-Unique: QGSkeAv_OviTZR1esa0JjQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kt0AHIyFJea53EXhlvzsU4NobEdyVidjcGMFcBg+7xKjDJeAlyrNbJs8sBWv0Cn4IfZOG9a9YvD1iZRWACq6xs+KoXdCpJOibfDNCfDeg+nwn+V7zW4MIJH6PcM5i/et3pM9HKuSY9/5QrU1PYygG4dVbG0411rSE3hBJnKWMeU4MG1DBRaK79R96ZaqTGe8y+dlalqHPGOTVd4Jvp3iW9AKl1J1nyR74G+a3ZcbcPumQhneaPUEUxDt8Lw0otOB38nRI8Y0SLENKzeAy4VAufXvgEBRbY9kvxPyM6nGRHa2Uo5hyyVn3PtpK1QZodPyos0mR2cD0YNdlgoWiHhnRg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bCMh9bDphYY+wjeyyZf9GP7gqy+UqEdPNOJx6gA/LDc=;
 b=Pcrrynkj+l8bm/zulE7UWeSVTyaSwFaEMeYThAdD96IXGQe/oo5m/RZDsFbtTc6+lTekygt3N8dV3ASF1DkpgQcxLk3SC3aUqIg9u8sL+nkEdo+LNJ+zuC1CECkoesi8llmhhfbTNtKjbPZZ5sMcDuQlBcOXnv6Nsqb3kp27nk806kFN4oUqFMMC/3SXuw8vlDVUUchDQ1OB94yghT/mnwA1k4xLxYhSTgAR2ZHPSNVahCPR+bAfqsvd8vhYFAsMuhVmVLBXPtZFjA5b372n92UjQuifiL9NKoWy0QLw17xGeOdB86zTtjmgDpPJfuDwhZOf7UW31qkhd2WjnGW7mg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Ian Jackson <iwj@xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, dpsmith@apertussolutions.com
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
 <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
 <24907.4491.895219.37845@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <952a9c35-39bb-d229-0bb9-a1f4e893b61a@suse.com>
Date: Wed, 22 Sep 2021 14:24:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24907.4491.895219.37845@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR10CA0087.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:208:15::40) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7af1676f-0ab1-40fb-d482-08d97dc3f7da
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3774:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB37744115C910337C8996F14DB3A29@VI1PR0402MB3774.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	RnSju6is7a/PIKrfl8O3M0prz4OZC1y00xjQQE1IhFRNvT4RRru4XHkS7PdDcm8rRpLpW64JK5cb9O+jfL37xiedqDSaYmq8wF2dD+CAUrqlsduxXdainOwl3UFEhcd6xY2XpLmg3rQGQdgG/9s3qPoI7KxlFB6AGGOetJ2dSo64pUOMaULBCs8xcxRqFfyDz2ZFKI5CEUvHnxd5bzYHAOrOUfEU/pZYMvSwbs33PFmt2HKrpr0yHaDEBBJUktScJS3DUTCtlgKnVe9ZP1o8BUn/oLPb3OsgJ9M+ESdOixhL/bOHcNjZkOpX0y5z7iXlPTeAHVVI0i+f21LNow7kIa6bUP8BgOrWaZPtY9FpcxmfvDxV3pVB4IqnAQaabVVEaffT2SBTrZhAb9DzyXnKVn5ehuWT2UEROIHGnMGgyM3HEs7opi4HA2MAXNIbUa0WHJ8/mkGPUuDL2SH/oX3468oae0zJE5SPd/+bdWMNSy5vu9RpR+QalI89AyfzEmYEivDc1MJfxiopHZ+DV+FGIcAXH7wWXcoRe9AwErka/8tU1qe7jQKLJuQF/gulT7Xa/oMwtTlJbNqvM41bYW3UdW/WtJIh2oDllYlHy3TfTwy7s8i2awQt8dZqdGqGN2ImLpZsAxdW1v4h+Lg4UMlXol3j6EGB8JZ39WBAzGHkOJcx3A/8s6PB+O4kl+yimB6HKAzknIfIm27pHqpcqOIsOZDV7VZEIqEDsgw9ybm6pqIPc+lQqerKitGgxunYyHxmcg6YanMXemk7p45HanLhcsIl5B/WeUlBZTymkjdztelwuMxBbGTal3yi0PYMlnibeEUQpVrDvDl36udjs2/SELRIz1P/WI/EuNkhpXahmQs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(16576012)(966005)(8676002)(6916009)(66556008)(2906002)(83380400001)(36756003)(956004)(8936002)(2616005)(38100700002)(4326008)(316002)(26005)(66946007)(66476007)(86362001)(53546011)(5660300002)(186003)(6486002)(508600001)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Y0V5Sk1Jbjk3cHY4c05HWkNIV1RXRytQRVhoNXgxVWZvWmpVQStvWG1EZ2NX?=
 =?utf-8?B?L05nVzRBTkMzcEgwWUR0d2xwYmV6WjJYSFl2T1VDYXMweU9DMFBSSmlPYWlT?=
 =?utf-8?B?c2xBL3N3Tk1mMkh1bFEweVIrenhtSG94dllPOWNTZlViWEFxb0pncFdtU0tk?=
 =?utf-8?B?SXkzdHdFdmx0YnVWTmtOaG4yUlJ6eXF5VnlLeDRCTlRkK0V5Z2ZJZkZMenpD?=
 =?utf-8?B?U0dGVlpVWjRHVUk3RDBXZmEyRHN1YWFxZ1JQRWpjYXhpWGUzc0FZeXBqWnN5?=
 =?utf-8?B?Y05SUmZjdDBQcHdFekE5OWZSaHlZTDVvTWZ1VTRBdFpDSnJyNmRsc1NpVWZu?=
 =?utf-8?B?V3ZGR1h2NWlCVXVBZHcvWWwxVW5DREpLYjVxalZRZ1JTOHMzMEFuaXZHMmwx?=
 =?utf-8?B?c2F0ZXVYdjNpZmtnUnVGT2tzdjJhNzYzbUZDbnNiQ3BuNHE1d0xVNXdGOUxy?=
 =?utf-8?B?OURxdHpwVWpFM2NveHUwYVl5VmNyUndGWlUxUlJZQkZmSTJuNmRUcm9GTWdl?=
 =?utf-8?B?V3RqeHpha0NUZVkxV2cvdmpMcHpMN0JsQnJJUGUwbTVZUTdaR1BEemtmd09r?=
 =?utf-8?B?QkpPZzkrOUI1eEJwTExYS3FqMnpIRUNFc2duNHJGZTlQc2R5bGdLUEtlNlBr?=
 =?utf-8?B?Rk5pa1JlV0RXYjhmbWNkSTNTemtUZzRwOXZGVmwxVmtPb2dGWHRmU0k5TnhU?=
 =?utf-8?B?RTlDUFgyN01mc2s0RTk3OW5LUnZ2VmxpNnRFNmE2bksxYldYMTZHdkdZL3Bp?=
 =?utf-8?B?a0FTN28yN21DcUFTbnVGeEcrd2RlV2ZRSHdWd2psRUQyQkFpSjAwSWZWVUU4?=
 =?utf-8?B?NHoyV2dzU2UyUDJwMFZ6T01DMG5Tc3BrZ2FYR1ZNVEhsZE4xWmNITFdCc25K?=
 =?utf-8?B?VWNzSkdXNTZpSmFnRG5tRktlVEsvL3JGUjhpUFdWbS9wU0lGZjlMcVF0a0g2?=
 =?utf-8?B?Z2hXbmphOENPT2hVejRHL3RnQzhqVDROM1dYbm9QREp5M1FjYVFZRkhXTkNJ?=
 =?utf-8?B?UjJrNDlUL1ppKzRGTlcvcGFPeVdQaUpUbk1VdjIwamxnbDJJVDNnSFN5QWg0?=
 =?utf-8?B?cS9VdXhmNTNlcFpNREhmVDA2RkJ6TE5TVGRIMGkrdERZUy9wdW9mZm15dnBD?=
 =?utf-8?B?OG9obk1oaGN2Qzd4QjdubGlrT0tKazRmanplNEpWelRNQzljSVNJdFhibmV1?=
 =?utf-8?B?QzZCTEhlWDJyM2pXUzhaemxGaHdUREZFRC9MWFMrTTdKdGVQa09wbjRZSVZy?=
 =?utf-8?B?UXVwVlplM3gyeTFnQUxFa1VhdjNleTVpYnVncE5CcjA2WFdnbkJuN2M2UmFx?=
 =?utf-8?B?YWw5T3VHT3ZUVVRUOGJqai9VQmswOUFkd2k3bkNYU1RVWlhSNnNZNDN0TjFH?=
 =?utf-8?B?K1J1YU1WakJxdDJLd1lkdUFSYVZjSDhHVmhhN1YrbkN5Qi8wa0k3Q0g3MUNm?=
 =?utf-8?B?aEJSR1lJNWxPdHhLckhVU0dzdEpMbzlueGppS1hxbXByYURSeXN6LytLQXVl?=
 =?utf-8?B?ZTdTZzRSTEE2TElnd1I4bytPYVZyZW5wWDNZSkdrbE9QRVRuckMwRW5CUzll?=
 =?utf-8?B?Z3BOQ2JtOGloVlBMM21FNW9TdlFxZlQ2WjJoUGc2NUhrbVBVNzlTcXpEY1ZE?=
 =?utf-8?B?TnNNcFVPMnpaeHpib29PeVI4am1NM2pTbVVzajNKSzhUTlJpa3NHUzRHaGhm?=
 =?utf-8?B?ZmEyRm1MbWczbWxCbjAwc2ZUSmMxQnlCRXY5MkM4SXNVUytXOE5YWXRiMlB4?=
 =?utf-8?Q?Mh5Ku103n+LXpDhce0EH0f1JNodYGFpddvuQNqU?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7af1676f-0ab1-40fb-d482-08d97dc3f7da
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 12:24:48.1623
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WKW4tgLla20Uodc+DUgsNwLTVT45n7bhyWHuVtg2gCYFKZvyO5M37vjaF2n4ndDOhKN8FWovyC1dFP7CjCoaRw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3774

On 22.09.2021 13:20, Ian Jackson wrote:
> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
>> On 22.09.2021 01:38, Stefano Stabellini wrote:
>>> On Mon, 20 Sep 2021, Ian Jackson wrote:
>>>>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
>>>>>
>>>>> there even look to be a number of higher order pages available (albeit
>>>>> without digging I can't tell what "(C)" means). Nevertheless order-4
>>>>> allocations aren't really nice.
>>>>
>>>> The host history suggests this may possibly be related to a qemu update.
>>>>
>>>> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
>>
>> Stefano - as per some of your investigation detailed further down I
>> wonder whether you had seen this part of Ian's reply. (Question of
>> course then is how that qemu update had managed to get pushed.)
> 
> I looked for bisection results for this failure and
> 
>   http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-arm64-arm64-libvirt-xsm.guest-start--debian.repeat.html
> 
> it's a heisenbug.  Also, the tests got reorganised slightly as a
> side-effect of dropping some i386 tests, so some of these tests are
> "new" from osstest's pov, although their content isn't really new.
> 
> Unfortunately, with it being a heisenbug, we won't get any useful
> bisection results, which would otherwise conclusively tell us which
> tree the problem was in.

Quite unfortunate.

>>>> The grub cfg has this:
>>>>
>>>>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
>>>>
>>>> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".
>>>
>>> I definitely recommend to increase dom0 memory, especially as I guess
>>> the box is going to have a significant amount, far more than 4GB. I
>>> would set it to 2GB. Also the syntax on ARM is simpler, so it should be
>>> just: dom0_mem=2G
>>
>> Ian - I guess that's an adjustment relatively easy to make? I wonder
>> though whether we wouldn't want to address the underlying issue first.
>> Presumably not, because the fix would likely take quite some time to
>> propagate suitably. Yet if not, we will want to have some way of
>> verifying that an eventual fix there would have helped here.
> 
> It could propagate fairly quickly.

Is the Dom0 kernel used here a distro one or our own build of one of
the upstream trees? In the latter case I'd expect propagation to be
quite a bit faster than in the former case.

>  But I'm loathe to make this change
> because it seems to me that it would be simply masking the bug.
> 
> Notably, when this goes wrong, it seems to happen after the guest has
> been started once successfully already.  So there *is* enough
> memory...

Well, there is enough memory, sure, but (transiently as it seems) not
enough contiguous chunks. The likelihood of higher order allocations
failing increases with smaller overall memory amounts (in Dom0 in this
case), afaict, unless there's (aggressive) de-fragmentation.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:30:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:30:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192700.343273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1Nw-0007q5-I5; Wed, 22 Sep 2021 12:30:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192700.343273; Wed, 22 Sep 2021 12:30:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1Nw-0007py-F3; Wed, 22 Sep 2021 12:30:00 +0000
Received: by outflank-mailman (input) for mailman id 192700;
 Wed, 22 Sep 2021 12:30:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT1Nv-0007ps-VA
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:29:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT1Nv-0006y1-R7
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:29:59 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mT1Nv-0003f2-Q9
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:29:59 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mT1Nq-0003Im-Ou; Wed, 22 Sep 2021 13:29:54 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=uGnkq96b/xxN88TzuE7mpG5VNU1uPgMonjr6E+affzY=; b=gYwimMKqY3XxlLfsIrRiSYJjjb
	OMbWJf/xtiFNSdad+WONYaRlqr3+r+mOUc6y7sZt9MQoNRIi3mldbEI2xs2vc9zwRUUG3eD4nbOxs
	C6TreHk3EgkUNBDHv19zp12Kg97hOA+pxJNCMfA967qNOK8nUKrD+PR5eGfbb0vr94nA=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24907.8642.479155.35952@mariner.uk.xensource.com>
Date: Wed, 22 Sep 2021 13:29:54 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
    xen-devel@lists.xenproject.org,
    dpsmith@apertussolutions.com
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <952a9c35-39bb-d229-0bb9-a1f4e893b61a@suse.com>
References: <osstest-164996-mainreport@xen.org>
	<d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
	<24904.44119.940679.241639@mariner.uk.xensource.com>
	<alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
	<19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
	<24907.4491.895219.37845@mariner.uk.xensource.com>
	<952a9c35-39bb-d229-0bb9-a1f4e893b61a@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> Is the Dom0 kernel used here a distro one or our own build of one of
> the upstream trees? In the latter case I'd expect propagation to be
> quite a bit faster than in the former case.

It's our own build.

> >  But I'm loathe to make this change
> > because it seems to me that it would be simply masking the bug.
> > 
> > Notably, when this goes wrong, it seems to happen after the guest has
> > been started once successfully already.  So there *is* enough
> > memory...
> 
> Well, there is enough memory, sure, but (transiently as it seems) not
> enough contiguous chunks. The likelihood of higher order allocations
> failing increases with smaller overall memory amounts (in Dom0 in this
> case), afaict, unless there's (aggressive) de-fragmentation.

Indeed.

I'm not sure, though, that I fully understand the design principles
behind non-order-0 allocations, and memory sizing, and so on.  Your
earlier mail suggeted there may not be a design principle, and that
anything relying on non-order-0 atomic allocations is only working by
luck (or an embarassing excess of ram).

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:34:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:34:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192708.343285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1Si-0000rs-8j; Wed, 22 Sep 2021 12:34:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192708.343285; Wed, 22 Sep 2021 12:34:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1Si-0000rl-5k; Wed, 22 Sep 2021 12:34:56 +0000
Received: by outflank-mailman (input) for mailman id 192708;
 Wed, 22 Sep 2021 12:34:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=U2Du=OM=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mT1Sg-0000rf-Tt
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:34:54 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7244afea-8d33-4b36-81ac-c95cdf97bebf;
 Wed, 22 Sep 2021 12:34:53 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id AF2F71FE6E;
 Wed, 22 Sep 2021 12:34:52 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7FD1813D69;
 Wed, 22 Sep 2021 12:34:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id v5NjHewiS2GNIgAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 22 Sep 2021 12:34:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7244afea-8d33-4b36-81ac-c95cdf97bebf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632314092; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=iEnICj58UPLA4HvtWngBbuaRTSvu539gURlAKLeD2KY=;
	b=XjMG4XMz5YmTX/ajvs5ZljKNQ7GxqX2Ktc569vgDTL4cp3Qvbj8aiVvHfMEaPekksx95D/
	CiiINUM8Y+GJSv2j1RVLzGiZ/EN55UCPf6UXrtslMAy6rxmvi4nwzUzlGi/xggWYw2vOHO
	FbKmGgLgVpdN/VYega56/wy5aO4lQvo=
To: Julien Grall <julien@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
 <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
Message-ID: <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
Date: Wed, 22 Sep 2021 14:34:51 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Ur4F6JcuRd35CZsXnypc8tcNbPcuvTS1m"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Ur4F6JcuRd35CZsXnypc8tcNbPcuvTS1m
Content-Type: multipart/mixed; boundary="3vXXGWO8PRbnKaeqYgvEYTtXTaOdWLf8x";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
Message-ID: <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
 <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
In-Reply-To: <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>

--3vXXGWO8PRbnKaeqYgvEYTtXTaOdWLf8x
Content-Type: multipart/mixed;
 boundary="------------9FA4D8B3694414B3F7C17824"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------9FA4D8B3694414B3F7C17824
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 22.09.21 12:23, Julien Grall wrote:
> Hi Roger,
>=20
> On 22/09/2021 14:58, Roger Pau Monn=C3=A9 wrote:
>> On Wed, Sep 22, 2021 at 02:07:44PM +0500, Julien Grall wrote:
>>> Hi Roger,
>>>
>>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>>> Failure to map the shared ring and thus establish a xenstore
>>>> connection with a domain shouldn't prevent the "@introduceDomain"
>>>> watch from firing, likewise with "@releaseDomain".
>>>>
>>>> In order to handle such events properly xenstored should keep track =
of
>>>> the domains even if the shared communication ring cannot be mapped.
>>>> This was partially the case due to the restore mode, which needs to
>>>> handle domains that have been destroyed between the save and restore=

>>>> period. This patch extends on the previous limited support of
>>>> temporary adding a domain without a valid interface ring, and modifi=
es
>>>> check_domains to keep domains without an interface on the list.
>>>>
>>>> Handling domains without a valid shared ring is required in order to=

>>>> support domain without a grant table, since the lack of grant table
>>>> will prevent the mapping of the xenstore ring grant reference.
>>>>
>>>> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>>>> ---
>>>> oxenstored will need a similar treatment once grant mapping is used
>>>> there. For the time being it still works correctly because it uses
>>>> foreign memory to map the shared ring, and that will work in the
>>>> absence of grant tables on the domain.
>>>> ---
>>>> Changes since v1:
>>>> =C2=A0=C2=A0 - New in this version.
>>>> ---
>>>> =C2=A0=C2=A0 tools/xenstore/xenstored_domain.c | 30=20
>>>> ++++++++++++++++++------------
>>>> =C2=A0=C2=A0 1 file changed, 18 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/tools/xenstore/xenstored_domain.c=20
>>>> b/tools/xenstore/xenstored_domain.c
>>>> index 9fb78d5772..150c6f082e 100644
>>>> --- a/tools/xenstore/xenstored_domain.c
>>>> +++ b/tools/xenstore/xenstored_domain.c
>>>> @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xenstore_domain_interfac=
e *intf =3D=20
>>>> conn->domain->interface;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 XENSTORE_RING_IDX cons, prod;
>>>> +=C2=A0=C2=A0=C2=A0 if (!intf) {
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 errno =3D ENODEV;
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -1;
>>>> +=C2=A0=C2=A0=C2=A0 }
>>>> +
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Must read indexes once, and =
before anything else, and=20
>>>> verified. */
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cons =3D intf->rsp_cons;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 prod =3D intf->rsp_prod;
>>>> @@ -149,6 +154,11 @@ static int readchn(struct connection *conn,=20
>>>> void *data, unsigned int len)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xenstore_domain_interfac=
e *intf =3D=20
>>>> conn->domain->interface;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 XENSTORE_RING_IDX cons, prod;
>>>> +=C2=A0=C2=A0=C2=A0 if (!intf) {
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 errno =3D ENODEV;
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -1;
>>>> +=C2=A0=C2=A0=C2=A0 }
>>>> +
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Must read indexes once, and =
before anything else, and=20
>>>> verified. */
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cons =3D intf->req_cons;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 prod =3D intf->req_prod;
>>>> @@ -176,6 +186,9 @@ static bool domain_can_write(struct connection=20
>>>> *conn)
>>>> =C2=A0=C2=A0 {
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xenstore_domain_interfac=
e *intf =3D=20
>>>> conn->domain->interface;
>>>> +=C2=A0=C2=A0=C2=A0 if (!intf)
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return false;
>>>> +
>>>
>>> Rather than adding an extra check, how about taking advantage of=20
>>> is_ignore?
>>
>> Right, I just need to change the order in conn_can_read and
>> conn_can_write so that the is_ignored check is performed before the
>> can_{read,write} handler is called.
>>
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return ((intf->rsp_prod - intf-=
>rsp_cons) !=3D=20
>>>> XENSTORE_RING_SIZE);
>>>> =C2=A0=C2=A0 }
>>>> @@ -183,7 +196,8 @@ static bool domain_can_read(struct connection=20
>>>> *conn)
>>>> =C2=A0=C2=A0 {
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xenstore_domain_interfac=
e *intf =3D=20
>>>> conn->domain->interface;
>>>> -=C2=A0=C2=A0=C2=A0 if (domain_is_unprivileged(conn) && conn->domain=
->wrl_credit < 0)
>>>> +=C2=A0=C2=A0=C2=A0 if ((domain_is_unprivileged(conn) && conn->domai=
n->wrl_credit <=20
>>>> 0) ||
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 !intf)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return =
false;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return (intf->req_cons !=3D int=
f->req_prod);
>>>> @@ -268,14 +282,7 @@ void check_domains(void)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 domain->shutdown =3D true;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 notify =3D 1;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 }
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
/*
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 * On Restore, we may have been unable to remap the
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 * interface and the port. As we don't know whether
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 * this was because of a dying domain, we need to
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 * check if the interface and port are still valid.
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 */
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
if (!dominfo.dying && domain->port &&
>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 domain->interface)
>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
if (!dominfo.dying)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 continue;
>>>
>>> This is mostly a revert on "tools/xenstore: handle dying domains in l=
ive
>>> update". However, IIRC, this check was necessary to release the=20
>>> connection
>>> if the domain has died in the middle of Live-Update.
>>
>> But if the domain has died in the middle of live update
>> get_domain_info will return false, and thus the code won't get here.
>=20
> Hmmm... I think I am mixing up a few things... I went through the=20
> original discussion (it was on the security ML) to find out why I wrote=
=20
> the patch like that. When going through the archives, I noticed that I =

> provided a different version of this patch (see [1]) because there was =

> some issue with the check here (I wrote that it would lead to zombie=20
> domain, but don't have the rationale :().
>=20
> Juergen, I don't seem to find the reason why the patch was not replaced=
=20
> as we discussed on the security ML. Do you remember why?

Sorry, no, I don't.

You did send the new version for V6 of the LU series, but it seems at
least in V9 you commented on the patch requesting that a comment just
in the section being different between the two variants to be removed.

So either we both overlooked the new variant not having gone in, or we
agreed to use the old version (e.g. in a security meeting). In my IRC
logs I couldn't find anything either (the only mentioning of that patch
was before V6 of the series was sent, and that was before you sending
the new one as a reply to V6).

> Assuming this was a mistake, could someone take care of sending an=20
> update? If not, I could do it when I am back in October.
>=20
> For the archives, the issues would appear when shutting down a domain=20
> during Live-Update.

Hmm, IIRC you did quite some extensive testing of LU and didn't find
any problem in the final version.

Are you sure there really is a problem?


Juergen

--------------9FA4D8B3694414B3F7C17824
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------9FA4D8B3694414B3F7C17824--

--3vXXGWO8PRbnKaeqYgvEYTtXTaOdWLf8x--

--Ur4F6JcuRd35CZsXnypc8tcNbPcuvTS1m
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFLIusFAwAAAAAACgkQsN6d1ii/Ey8F
3wf+OOsUy+XbQCf7BYdqYpEebd0BiJi0YGzzBehVGp7cIxv4QbBS1cJX9fu/JkspdL8t7r9GKUub
ui4CwjVmfIp/SwaTG3mYwTJVHQsJzu6WkUtWuOrhgUuyV8TMM1Ced294Och49F1oHz4bA4ToX7ut
y77heHkRg06NTCTlF7LsjDWTgtysEEGEqWGIzMoAO7Qnm7+2OZQBOS7Nq8MYgfHrMv0DO86mUmwo
9i6x8i24fS7FT91s8+WDbIZNgr+8COYkloekvHgvppF3jt7Gn1Rd4hU3JXQhLv8Q1MLIN29o4Vnp
xomaeh4Tw0ab+gNqLLolqiNgIhZUC4Ir6hPRGiNJiA==
=KvVN
-----END PGP SIGNATURE-----

--Ur4F6JcuRd35CZsXnypc8tcNbPcuvTS1m--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:49:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192714.343296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1gd-0002iD-HE; Wed, 22 Sep 2021 12:49:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192714.343296; Wed, 22 Sep 2021 12:49:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1gd-0002i6-E4; Wed, 22 Sep 2021 12:49:19 +0000
Received: by outflank-mailman (input) for mailman id 192714;
 Wed, 22 Sep 2021 12:49:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C/3m=OM=infradead.org=peterz@srs-us1.protection.inumbo.net>)
 id 1mT1gb-0002hu-6I
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:49:18 +0000
Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f7e882af-83ec-4788-a36f-6668de1b2a01;
 Wed, 22 Sep 2021 12:49:14 +0000 (UTC)
Received: from j217100.upc-j.chello.nl ([24.132.217.100]
 helo=worktop.programming.kicks-ass.net)
 by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux))
 id 1mT1do-004mud-Fa; Wed, 22 Sep 2021 12:46:36 +0000
Received: by worktop.programming.kicks-ass.net (Postfix, from userid 1000)
 id 0C64A9816EA; Wed, 22 Sep 2021 14:46:24 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7e882af-83ec-4788-a36f-6668de1b2a01
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version:
	References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=oAPrbliIVMtJ6nyhmSrbiDdRqfA8JQhJf2UshZDVT88=; b=tEhxQE1Rmf1aTw5P1Xwbfp0kSE
	YhEyVwt1OomoAedgQy7bd0MGsyzpJuDb6xfOPGIy5Aam6rrlnutVUT6hpOPH+vxUDYZKFZIMGrEKh
	3gQSU0/TwUk/ouYEPzD5cUnwbDumaIhLYrnr3h/7NXqphYNEu8c7OmvWIaaA1SrqbRyZW2o1nMIiw
	DpHkDrSzUBUtp8+fMzfqTHvg1AMcupoiR7sQUybbpk52IWxGnkXmOTbMpPhMF+/HhKfq773SzI5mN
	Vn/f9OxoFZzS4dAcBzm+5xJ8UuNUR2znLUvU2M4Emmgi5x28KyDkgbQlbACPLVWxFjqR07SKJXdSc
	3aNbmczA==;
Date: Wed, 22 Sep 2021 14:46:23 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, x86@kernel.org,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>, Deep Shah <sdeep@vmware.com>,
	"VMware, Inc." <pv-drivers@vmware.com>
Subject: Re: [PATCH v2 0/2] x86/xen: simplify irq pvops
Message-ID: <20210922124623.GR4323@worktop.programming.kicks-ass.net>
References: <20210922103102.3589-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210922103102.3589-1-jgross@suse.com>

On Wed, Sep 22, 2021 at 12:31:00PM +0200, Juergen Gross wrote:
> The pvops function for Xen PV guests handling the interrupt flag are
> much more complex than needed.
> 
> With the supported Xen hypervisor versions they can be simplified a
> lot, especially by removing the need for disabling preemption.
> 
> Juergen Gross (2):
>   x86/xen: remove xen_have_vcpu_info_placement flag
>   x86/xen: switch initial pvops IRQ functions to dummy ones
> 
>  arch/x86/include/asm/paravirt_types.h |   2 +
>  arch/x86/kernel/paravirt.c            |  13 ++-
>  arch/x86/xen/enlighten.c              | 116 ++++++--------------------
>  arch/x86/xen/enlighten_hvm.c          |   6 +-
>  arch/x86/xen/enlighten_pv.c           |  28 ++-----
>  arch/x86/xen/irq.c                    |  61 +-------------
>  arch/x86/xen/smp.c                    |  24 ------
>  arch/x86/xen/xen-ops.h                |   4 +-
>  8 files changed, 53 insertions(+), 201 deletions(-)

That looks awesome, I'm totally in favour of deleting code :-)

Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 12:58:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 12:58:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192720.343307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1pu-0004Ja-GG; Wed, 22 Sep 2021 12:58:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192720.343307; Wed, 22 Sep 2021 12:58:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1pu-0004JT-B7; Wed, 22 Sep 2021 12:58:54 +0000
Received: by outflank-mailman (input) for mailman id 192720;
 Wed, 22 Sep 2021 12:58:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/3we=OM=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mT1ps-0004JM-Fo
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 12:58:52 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6360fc21-d64a-44e8-8fdd-fc1b547767dd;
 Wed, 22 Sep 2021 12:58:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6360fc21-d64a-44e8-8fdd-fc1b547767dd
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632315530;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=z/hNHmKF4p1bxeIGR29Ayx2C2Jjzremz6y3JzrNU0d4=;
  b=JtmieeoqkuQfBV+mb49RCUxUCf3ij4XhoQ2nZ4ldEqC3qands9bl94oX
   q48E0yxiCmxkwkRbruoB+tbGb5qqpeWv9472K8/5qF0GpTuBe4lvZ27m7
   KOCa25yaIXG8ad+1Re3fcAaYj8RFS9pJuptKgR9JXBBLU0CB693BMRLiy
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: OOP/gFVglvDWMjKEvtNjA0LTISCombTNPUTBNwh9sXsGF0iWOyU/lyeqhgz7sUpJxHLHvK9b+/
 fep9+rYRXesj6k14319Lo1Dw6XRs2OR1snIFrVwH/IsHlBQ8T2qCcnxzESG/2WclLB01A1RKbH
 p1NMEmseS0h+2Yp7XqA0gS4VEPU7nSxLL3xKtYhG1ZdqH/A6G7/w+YGJbZNURF2l4FxBnlXnVE
 oFdAIPtuY40Pb7ijJopqWgO4Q+2ZiOEtGqOo0QwSskteIsUpRbBACJQ8sZQuSVzQzCN51id9cf
 qGXG5nLWBb4Ly5HS0JNyykax
X-SBRS: 5.1
X-MesageID: 53732485
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Oykn0K6GERvRQOwLFnWdigxRtKfAchMFZxGqfqrLsTDasY5as4F+v
 msaWj2FMvzfYjbwc9Agb97l9B9Uu5eEyYUwGwpu+Xo1Hi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NQw2rBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z8
 9pHjYKwYggQMrDLvd0QbgN8On5AIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALBc/nJo4A/FpnyinUF60OSpHfWaTao9Rf2V/cg+gTRqmFP
 ZZJNFKDajzrSQVQMwc8EKsup/ayr1fkaRdklQ6s8P9fD2/7k1UqjemF3MDuUseRWcxfk0Kcp
 2TH12f0GBcXMJqY0zXt2m2orv/Cm2X8Qo16PL+y++NugVaT7ncOExBQXly+ydG4lUyWS99ZM
 1YT+Cclse417kPDZtzwWRKovVaPvwVaRsJdFet85Q2QooLE7gDcCmUaQzppbN09qNRwVTEsz
 kWOnd7iGXpoqrL9YXCA8raZqxuiNC5TKnUNDQcIQwIK7NjkpIAblQ/UQ5BoF6vdszHuMWium
 XbQ9nF43uhNy55Qv0mmwbzZqwmjrKDXTiU63wnKWUec4z5jQKf6WZP9vDA38s18wJalokip5
 SZfwZHOsrxWVPlhhwTWH75cR+jBC+KtdWSG2A8xRcFJGyGFpib7Fb289g2SM6uA3iwsQjbvf
 EabkgdY/pY70JCCPPIvPt7Z5yjHy8Hd+TXZuhL8NYEmjntZLlbvEMRSiam4hTuFraTUuftjU
 ap3iO71ZZrgNUiC8NZRb7xHuYLHOwhknT+DLXwF503/jNJym0J5uZ9aaQDTP4jVHYuvoRnP8
 sY3Cid540wEC4XDjt3s2ddLdzgidCFjbbiv8pA/XrPTc2JORTB6Y9eMkOxJRmCQt/kM/gs+1
 irmAREwJZuWrSCvFDhmnVg5OeuzAs4g/SxkVcHuVH7xs0UejU+UxP53X7M8fKU99fwlyvhxT
 vIffN6HDOgJQTPCkwnxp7GnxGC7XBj01w+IIQS/Zz0zI8xpSwDTo4e2dQrz7igeSCGwsJJm8
 bGn0wraR7sFRhhjU5mKOK7+kQvpsChPgv92UmvJPsJXJBfm/r91JnGjlfQwOcwNd0nOn2PIy
 waMDB4EjuDRuItposLRjKWJot7xQetzF0ZXBUfB6rOyOXWI92av29YYAu2JYSrcRCX//6D7P
 bdZyPT1MfsmmldWstUjT+Y3nPxmv9a2/u1U1AVpGnnPfm+HMLI4LynUx9RLu41M2qRd5Vm8V
 HWQ94QIIr6OIs7kTgIcfVJ3cuSZ2PgIsTDO9vBpcl7i7Sp68bfbA0VfOx6A1H5UILdvad53x
 O4gvIgd6hCliwpsOdGD13gG+2OJJ30GcqMmqpBFX9O71lt1kglPMc7GFyv7wJCTcNEdYEAlL
 widiLfGm7kBlFHJdGA+FCSV0OdQ7XjUVMumELPWy4y1p+f4
IronPort-HdrOrdr: A9a23:k4X5M62sC3UtjNmzH7DT8gqjBIwkLtp133Aq2lEZdPUzSL3+qy
 nOpoV+6faaskdyZJhNo7C90cq7IE80l6QFh7X5VI3KNGOK1FdARLsSlLcKqAeQfhEWmNQttp
 uIWpIOcOEZUjNB5voSmjPXLz+L+qj9zEnSv4jj80s=
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="53732485"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dRkjtH3WKoDiarzFD7fkkvbHRthIRib0/s10ECaT9kNJwC9C9/J2zCcZH0jy3AWUf+F+JZoBU2u7rBvnWwBXkndAdECximjic5aGrqmTY2H4fmuWUJIBqG2zy8oCGLnOxMyb2vzKaQmHLq0jDR3p38tOWbVEz1oNi+uoADegbIuxfzFpSoHHjIRx5gRkEUc7gQ5XAiyXUIJTiGMKqn9oGdOZ+FGKL5EepmYUXeq4Xu1vPUZqzVGxpJ/3d/2AW7YA+0NsQn8oAhFJqAxZfzGiQ9BKtTez9hNTjMqKNry1YE+iQDjaQcjnDy6fUG1efmXV0rfXmNqT482QMavsEplXiA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1KX05VlyxTNYGz7J4fRuoOrsB8QhhDZUG8KbfqC+gXw=;
 b=BSFyt4WcnKyJl8ANwYrJeoMHbP+qTJWhuTVvEHIu5t2Pj6T8fQsV6RnyYl1CotPpxym0cijhypnjLBIdJk7z/zTGxN6izQuq5TUukg9GHA+gARayck2rKbVwq7UKwyUrkEJ0yRw339/bahJtlh5GWY9JFSQWQZGp6NcyQpuVXpwzixcTO4+f1rS+Wy1kODtOn9z/5nTaSQ6qXhnnSdQ8GklgqYYFH3XSvX9iBgFfZ75worwMQb+48Wbln1dyJPlBfdU3iOCtsa45/eWCZea4uLhxkgO6jWW+U7X8NRi+Q9jpdXaMpRVEQlvKeW58x1ppq4ExQ+vk7E7m5rqWoLpoGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1KX05VlyxTNYGz7J4fRuoOrsB8QhhDZUG8KbfqC+gXw=;
 b=hXRTmkPVO9EGWDSOkk5DnwIFdkiQZckNhU7Hcyck1jRIdrpjS1Y5QOr+ZXr75Pqm+xxgMraSKo5d80vlJ9IfMa4cBRNtXne3IWFcIAoBod1KKDVomLfcIljW0mn2Munnffb00/v+HUmakNulLGrQxEjN5WgOYxylrm9EVKw2TJM=
To: Jan Beulich <jbeulich@suse.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Julien Grall <julien@xen.org>, Dario Faggioli
	<dfaggioli@suse.com>, Meng Xu <mengxu@cis.upenn.edu>, Xen-devel
	<xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
 <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
 <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
 <69770706-85f6-6509-b5c4-bda61dfdf994@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Message-ID: <84b780c0-7177-d1ba-fabd-9a6258985f9d@citrix.com>
Date: Wed, 22 Sep 2021 13:58:31 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <69770706-85f6-6509-b5c4-bda61dfdf994@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0399.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:189::8) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d05cf1ea-b40b-4382-0ae6-08d97dc8b2bc
X-MS-TrafficTypeDiagnostic: BY5PR03MB4998:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB499880C5FE3B445DD42EB5EBBAA29@BY5PR03MB4998.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: TCVx2uTIgEpaVPF8PZR+Wvy7TibW3N8PCdjIOHhAKdJ2Yws45wwTQXa14XkiVtNg7fWU6cVLTemz0VF6jlOt5nHO24JjBjNFLm2UWlkT/XWCIguXghgv+P4SF/YQmq94bL++L3U1F5f36M+BHw9HjvZtLuFep+QS9953dQjC6oS2wSqPpG0kG6DbIvms6cYXGsQoEJOYtY3JAajZKW1EJsWsf1wWlPBSuaecuu0xLsghpJptDOuMB0MJYP/oZ4YO5NBohU8BPI6f84THskIpg9C275k6OAMbNHimITI4LTdXk405aYwevVzCM4yY0+7q349JJtffgS1u0PJklYFlIyP/ysSbLFom2VyV/blLu/Ih4R6hP3D6o5B6Y+shbpI6/dLdRzxrYRIwi8Q4iGENHJDPVg3j7mPcBNXAWWljrQ6J3BPCTJ/tguCsu/hccuvCtKfZZsuGcaXUkWet9Dca+vlo8K0k0Qn9H/Qpv6r14O8xZ34iERvvjDwLn2P0dhzq95F6hD7pLBzG+iX7+FaEaJb554qi5G3JIKfMbaAK4htvXYC8lTmEJXgRrebWqDGM7286mOAYW4Rxbk8uJyRkt4kBczfqyORlqixOHs4ym8Vl14efybKrWt9FMJZKpk0XoskjHVoYW+8j+gelvEt1RpRgk6u3pbGNuu4JhwBVxUu15AHZZbViCPN8DYHy3rxjZ5lXpuWTk1Fjq0UiLw/5Bq+gWDw0De1qBjc93Jfat0c=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(508600001)(2906002)(86362001)(316002)(36756003)(16576012)(31696002)(5660300002)(6666004)(6916009)(53546011)(38100700002)(31686004)(8936002)(4326008)(26005)(8676002)(956004)(66476007)(66946007)(66556008)(186003)(2616005)(6486002)(55236004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TkdzZFlzV3QyQjZTbndLUEFyNXl0ZTUxTjE4cW1NRFI3UkZ2LzdRRU9lS05v?=
 =?utf-8?B?ZXF3cDdqa0xqUXhqYVRCY1RmdXl5dDVvRjE2QjZucjRoTGhGSkpaZzZsODAv?=
 =?utf-8?B?aTF6V1JNVjBzd0VLZWQxZHd4QUFxM2ZOTmphRWlYcWJxczRxdUt2Q3pwY2N3?=
 =?utf-8?B?d2dpdWp3cjdRQUhmUVhOZ29FNWxEODUzZFROM2lHWU1GaFh3ckNQQUFDWCtU?=
 =?utf-8?B?aFlaWmlSMzk1ZUIvb1AxSkhZRmNLWE5OWmFPVHdla0NUOXdjbEQzT2E0cGVL?=
 =?utf-8?B?b3l1SnUvVzJ3VDVnYjE2RFZ6SnZzMzBiRWIxYVZsOVpSem56Y0hKUlNNeVB0?=
 =?utf-8?B?YW1lMno5RDBuYkJYNThaQ2xpeHBLTHlWdFpFcWlnR0ljbExoS2d0Wm9kaGl1?=
 =?utf-8?B?OVU2KzliSU1HMUg4Tzlod1BWaDhhYnkyYkNKTUQ5WitKVzZ1YUh5MmxMYlBP?=
 =?utf-8?B?bVBiblZPK2RYYmNNUXlzdnIwckdyMi9RbmZIYTBsd0pIcjBObSt3OGhyd1RV?=
 =?utf-8?B?QWp4R3M0OGU2SzhEbUhJR2gzSEJiUERVcnUvU0ZzMWNSOUtkY0k5bFBldm1x?=
 =?utf-8?B?SmkvVVRMRzd5RW1LVjdUdU9zaTc2Q0pVOHVQQmo4YXRjSjNHRkdjdUZLZVZJ?=
 =?utf-8?B?U1YrSE1uOTVRVUdlemh5SlRpVlFsTWRDWSsreDRvbmhDY1BwKzQyb1EwMXZY?=
 =?utf-8?B?T0RSVVFNL25hRStwaHI1dWViZXpNMDlvellkQlpnRFVjc0FVYzdLQnRwamRN?=
 =?utf-8?B?c25qSVJXSXlTenJON2FTL2YrYTUvM2RSUWdOVkF6QWE5S0tLdENUZEdZeGhS?=
 =?utf-8?B?K1pEUmp2UWdNVXhLcm9HT3J0b1RrVXV6WXdMMncyeURqckdkanJUZWdOQ3d0?=
 =?utf-8?B?TGt5ME1VWEtSYjUyT1lmWmFaaTZaVXh5QXI4RzJ3S3dYcUs1dVlzWkE1RENy?=
 =?utf-8?B?aHljb1pBNnJQM1p0SlhIMEZlN0R3VVN3SXR2M2pTWDNialpoQkJmU2I0dDBQ?=
 =?utf-8?B?OTBMMHBBQVJrTXFnUmN1Z05pZHluaE03WkpueldQQUdwQW55OVhVeEJrTnVo?=
 =?utf-8?B?VWtnK0ZDZ1ZaUDhzak9SZm11NWtQZFFKRnVXUmZOZE5kc0ZaVDBEN09POFZX?=
 =?utf-8?B?cjR4VE5wTkxoSmlQdU5lUlE5TDRhOFFDL0VERGVxcUQrTmF2eHlNblBMTFMv?=
 =?utf-8?B?TUw0U29Rc0xxelJHNWpJT2xSMEx1cDl5a2dMa3E1NzBpcElsQWRrYWRKSDFV?=
 =?utf-8?B?RFpWd1Zkd2haelZMMWFadklnODBSaDlOSDMxemV4aTNTeDB4Z0QzSUVQK0tm?=
 =?utf-8?B?SzNleDlMU0hrL3BOU1dnYmlFeU9VZXFDaDJoRnJ5UzN0Mm1FYTA4WHptQTNl?=
 =?utf-8?B?RVJUeVFFdXVhbDc4NTIxcXlFcTFlSk1iQWwrc2h2NlVzUUd1QUpVTHBIaTZu?=
 =?utf-8?B?bndkZ2VzY3FRK2RhMVRDdEx4YXlpMUtOYkZuN1kzTVBET2o1RHRydlVjeTJL?=
 =?utf-8?B?MFZTUWJEOWtTQUtxRmhUWmp6MDNIZWM3SXNPUk5EMFc4VlcxVk9Mc1FWNk1o?=
 =?utf-8?B?L2FIWUtvYitWRHFzbHBlOWN2bFN4Z3NpV0ExOVZKZjFYeDR5SFhWNDBOeS9M?=
 =?utf-8?B?UVF0VzVob3hvNWZheFdubFpNZ1pxYVd6OGlBWTM5Skl6TTFvdEF1eHdhUjJ2?=
 =?utf-8?B?QWp1QlN3N3lTN2RNanp2VGMzQWdBMm1nQXZzNDNsRzhSa1FnSG5xcXF2d0J4?=
 =?utf-8?Q?sm1u+8kH3QHZeWzpjxdUholDAHB+bTR9+VNIJPQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d05cf1ea-b40b-4382-0ae6-08d97dc8b2bc
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 12:58:39.6614
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /KORh8WC0r/TdBTS5TT82PeJ1zfBBdEguX8m/BaD+CQ4jXUAIlgYGP6Oiodj5YHfQ+0J41QUd2kdH6WNvXyxvdhJ3wSGxEMCuw3tg9pFWtc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4998
X-OriginatorOrg: citrix.com

On 22/09/2021 08:01, Jan Beulich wrote:
> On 21.09.2021 19:51, Andrew Cooper wrote:
>> On 21/09/2021 07:53, Jan Beulich wrote:
>>> On 20.09.2021 19:25, Andrew Cooper wrote:
>>>> --- a/xen/arch/x86/hvm/hvm.c
>>>> +++ b/xen/arch/x86/hvm/hvm.c
>>>> @@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDL=
E_PARAM(void) arg)
>>>>          if ( copy_from_guest(&tr, arg, 1 ) )
>>>>              return -EFAULT;
>>>> =20
>>>> -        if ( tr.extra_bytes > sizeof(tr.extra)
>>>> -             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
>>>> +        if ( tr.extra_bytes % sizeof(uint32_t) ||
>>>> +             tr.extra_bytes > sizeof(tr.extra) ||
>>>> +             tr.event >> TRC_SUBCLS_SHIFT )
>>>>              return -EINVAL;
>>> Despite this being a function that supposedly no-one is to really
>>> use, you're breaking the interface here when really there would be a
>>> way to be backwards compatible: Instead of failing, pad the data to
>>> a 32-bit boundary. As the interface struct is large enough, this
>>> would look to be as simple as a memset() plus aligning extra_bytes
>>> upwards. Otherwise the deliberate breaking of potential existing
>>> callers wants making explicit in the respective paragraph of the
>>> description.
>> It is discussed, along with a justification for why an ABI change is fin=
e.
> What you say is "This has no business being a hypercall in the first
> place", yet to me that's not a justification to break an interface.

No, but "cannot be used outside of custom debugging" means there are no
users in practice, and therefore it really doesn't matter.

> It is part of the ABI, so disallowing what was previously allowed
> may break people's (questionable, yes) code.
>
>> But I could do
>>
>> tr.extra_bytes =3D ROUNDUP(tr.extra_bytes, sizeof(uint32_t));
>>
>> if you'd really prefer.
> I would, indeed, and as said ideally alongside clearing the excess
> bytes in the buffer.

Why?=C2=A0 The entire structure is copied out of guest memory, with a fixed=
 size.

It's not Xen's fault/problem if the VM didn't initialise it correctly,
and an explicit ROUNDUP() here maintains the current behaviour.

>>>> --- a/xen/common/sched/rt.c
>>>> +++ b/xen/common/sched/rt.c
>>>> @@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct =
rt_unit *svc, s_time_t now)
>>>>      /* TRACE */
>>>>      {
>>>>          struct __packed {
>>>> -            unsigned unit:16, dom:16;
>>>> +            uint16_t unit, dom;
>>>>              uint64_t cur_budget;
>>>> -            int delta;
>>>> -            unsigned priority_level;
>>>> -            bool has_extratime;
>>>> -        } d;
>>>> -        d.dom =3D svc->unit->domain->domain_id;
>>>> -        d.unit =3D svc->unit->unit_id;
>>>> -        d.cur_budget =3D (uint64_t) svc->cur_budget;
>>>> -        d.delta =3D delta;
>>>> -        d.priority_level =3D svc->priority_level;
>>>> -        d.has_extratime =3D svc->flags & RTDS_extratime;
>>>> +            uint32_t delta;
>>> The original field was plain int, and aiui for a valid reason. I
>>> don't see why you couldn't use int32_t here.
>> delta can't be negative, because there is a check earlier in the functio=
n.
> Oh, yes, didn't spot that.
>
>> What is a problem is the 63=3D>32 bit truncation, and uint32_t here is
>> half as bad as int32_t.
> Agreed. Whether the truncation is an issue in practice is questionable,
> as I wouldn't expect budget to be consumed in multiple-second individual
> steps. But I didn't check whether this scheduler might allow a vCPU to
> run for this long all in one go.

I expect it's marginal too.=C2=A0 Honestly, its not a bug I care to fix rig=
ht
about now.=C2=A0 I could leave a /* TODO: truncation? */ in place so whomev=
er
encounters weird behaviour from this trace record has a bit more help of
where to look?

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:01:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:01:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192728.343317 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1sq-0005ke-13; Wed, 22 Sep 2021 13:01:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192728.343317; Wed, 22 Sep 2021 13:01:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT1sp-0005kX-UR; Wed, 22 Sep 2021 13:01:55 +0000
Received: by outflank-mailman (input) for mailman id 192728;
 Wed, 22 Sep 2021 13:01:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT1so-0005kR-Mk
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:01:54 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 17850593-07be-400a-b6c8-bf6b295a3416;
 Wed, 22 Sep 2021 13:01:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17850593-07be-400a-b6c8-bf6b295a3416
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632315713;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=wbr1yFmtnXrXEa+6CzVRRRNX4v99308qxAdwiIpTihA=;
  b=WMMogbv/qhHLJ/UYmnCZ1HFBzUMiXlpzPIUwTeT52oL2B2HUuy0EITn/
   UiCiYRweU1ZG8X2qXXgKSW6LfH+tOpUn3zuHaLXG46v/4TnuKeWnQw77Q
   e0gW321/codSOHI+3lgQnXaA4ujG4K3MI7Iy35/WAykGf/+ixxufuWX6v
   A=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: CibKTBJh2q92zDOWJ0QCV1QRrDNrz+EichNnrRxIYJyMsFVJd4fSEwXo3zIUR4vQprMgjX6xvG
 IjmO3tSq50Gc+k+ua8k0g6VBOrI33i7Qm2NWDu4kP4bz6eNmrUEgqoNW4+4Rq8MLv+UG9sa5Ri
 TovKT7wSnXbeNxo1SsqPmsJrMbSV2wfYkH4HnrFxVzpUeJhvHJxH2f2Rdz1eBN+ArGDMaE5g8a
 aLiMRhfwCwPdTTxfO8CSIHht1DFvUyj0ZWFwSK40xeH433XtSntDBP/EhBfEiuBauaAEqt85vS
 wDIfZrd8DYirEYIMMStAeBDa
X-SBRS: 5.1
X-MesageID: 55120723
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:4s1196A4C6K0NBVW/xHkw5YqxClBgxIJ4kV8jS/XYbTApD5x0zQGy
 2IdUGmFOa2NZDakKNx/btu1/E0E75eGy99gQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00g7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/yBakustx6
 ct3mb+7ESgEZ4LGhPtMakwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHviTv4cHhmxYasZmJvDCN
 8giWwdTMBXYUyURI1QqDYIOg7L97pX4W2IB8w/EzUYt2EDMyCRh3b6rN8DaEvSWRMB9jkue4
 GXc8AzRABsXKdiewjqt6W+3i6nEmiaTcJIfEvi0++BnhHWXx3cPE1sGWF2ju/67h0WiHdVFJ
 CQpFjEG9PZoshbxF5+kAkP+8CXsUgMgt8R4UM4XzRqN67TuzyWSXkcpFxFLdMN3jZpjLdA17
 WNlj+8FFBQ27ubPECLDqu/Nxd+hEXNKdj5ZPEfoWSNAuoO68d9p1nojW/4+SPbdszHjJd3nL
 9lmRgAFjrMPhIYg06yh9DgraBr9+8CUEmbZCujRN19JDz+Vhqb+PORECnCBtJ6sybp1qXHb5
 xA5dzC2trxmMH10vHXlrB8x8FSVCxGta2e0vLKSN8N5q2TFF4CLJNgNiN2BGKuZGpldImK4C
 KMikShQ+IVSLBOXgVxfOtnqY/nGOZPITIy/PtiNN4ImSsEoKGevoXE/DWbNjjuFuBV9zskC1
 WKzLJ/E4YAyUv88klJbho41jNcW+8zJ7TmCHcykl0v3j+X2ibz8Ye5tDWZip9sRtcusiA7U7
 8xeJ42NzRBeW/f5eS7Z7cgYKlViEJTxLcmmwyCOXuLccAdgBk87DPrdneEod4B/xvwHnebU5
 HCtHERfzQOn13HALAyLbFFlaa/uAskj/S5qY3R0MAb6wWUnbKau8LwbK8k9c444+bEx1vVzV
 fQEJZmNW6wdVjTd9j0BRpDht4g+Jg+zjAeDMnP9MjgydpJtXSLT/drgcle9/SUCFHPv58A/v
 6ehxkXQRp9aH1ZuC8PfafSOyVKtvCdCxLIuDhWQetQKIRfi6olnLSD1n8QbGcBUJEWR3Cae2
 iaXHQwc+bvHrbgq/YSbnquDtYqoTbdzRxIIA2nB4L+qHiDG5W7/k5RYWeOFcD2BBmP5/KKuO
 bdcw/3maaBVmV9Lt8x3EqpxzLJ47Nzq/ucIwgNhFXTNTlKqFrI/fSXWgZgR7vVAlu1DpA+7e
 kOT4d0La7yGNfTsHEMVOAd4PP+I0usZm2WK4Pk4SKkgCPSbIFZTvZ1uAiSx
IronPort-HdrOrdr: A9a23:TKD966ADP/6OXXblHegksceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHPlOkPQs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ
 0QD5SWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHlXgslbnnlE422gYytLrWd9dP4E/M
 323Ls5m9PsQwVaUiz9bUN1K9TrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqJmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87CsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXkHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1XackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMW9yV0qp+1WH/ebcGkjaRny9Mw8/U42uonlrdUlCvgklLJd1pAZHyHo/I6M0r9
 gsfJ4YzY2n46ctHNVA7dw6MLmK41r2MFvx2VKpUCfa/Z48SgfwQr7Mkf8IDbKRCdE1JKVbou
 W2bLofjx9rR37T
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="55120723"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MZeGp0waY3+BqmNYxH0x5E/zPSvThchI/tRoY/+13ktsXAdvIu5e3lLCcae95Jh9mebliiODl5wqe+KopocMaao+PQZYB+k9Go2y33pN6r3cBkXFPgG3xyi8kIkpD6WdklzwYqcOCyQ0p+Jz6eciF1Gf+euQGoGfG+TL/sOngxgyQThQThU81QkJ97yOVJoYj4MfR82nypQ400eTHkDv69Ccegd0hYkAH2KNA33MnGZA+T29gsau30rglbeBCLQ1KYo6RfOm1mVCJW9dxVe9egY+JDSSjbcTYJ3nY0J9cV1k2KTFgjTbgLUQbK4stzixqLHtQwpSPp5dofRu8wPT/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OeEHWCyC0ZQ9x95QU/30S1iVlU/yt5tvPhmewScaa8Q=;
 b=geuYUNpbjPMTxocXKlzBNSuhJkT14RIqaQaOW5Br6lREg/RFGO4cJ1CpaZUF9HnBDu7ZIpDuhwJL+Fbqqibx0HuNT5xwsl003uawLPA2pPPAoW9WvEt3SOqaUH7BwuMFZwmeCNk9Ym4pxXwyEZluOfgSB/2RmD0/vS6nItn4lpKAAp9+zbc0+1XGEIau/gSw/89lndDH4llOwlywHA4sagVBYvZ+Jk+wOHYGuLj0TMm2IUoBKh00k2pv3/Qx7dZ+kAMeUtZ/7/m7WnBS6CuXhoQAc743Y9TNjU6+ZSIH+3zE21pj9RHGzrfVXGhfAdPeCStrC9YGxfpTl/TTdr3WFw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OeEHWCyC0ZQ9x95QU/30S1iVlU/yt5tvPhmewScaa8Q=;
 b=f9Y9gmknAHm+fWiMoB12h3vVnA0d/Fm+XKJWsGxdvhAC509usFRNGfejHwuENPWMl2w+8QNbhS2vHyZDMyi/qKCYWOZBmM7TPzwFuTxTL2mGWYqF2xsM1dROXuoVhHPy3H6D3fv+t6uW1x/kHzCd+dFktBCpiDoB4oLANLRnHlk=
Date: Wed, 22 Sep 2021 15:01:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, Tim Deegan
	<tim@xen.org>
Subject: Re: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
Message-ID: <YUspNEFBnihzVaWg@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
X-ClientProxiedBy: LO4P123CA0454.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1aa::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6e50d0d3-7154-4c81-5945-08d97dc921b6
X-MS-TrafficTypeDiagnostic: DM6PR03MB4970:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4970900081879B295DE5D0468FA29@DM6PR03MB4970.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1aYEuFdPvbjTSON+MDTsOEX3L/wfM6O7/8gv9WRFQOAptU3kxNV/JZVJn4ZwIQOuUgzQRBPjJztWwAeCcQNSRbSo6epMIV3dCubm/mxB+xgbhwgMlvZTYTkMTbRyl1H/69/E9A2iMSdyT199N43bcphZAVWD7iz2gKTNS3iFFZtUY2DoFJ+7tnj69RqXv1/pQ7WeNpuw5YuJtyOQcQ2YMRM2YJA3HXOaKOnpipIn7RGMEfQAnDaQfvXUeQzsC8WtYCQRlFeS1yJc5Oh7qn0beFFIimWxQZos3Bj+W1IgN013/eegwNIXEYa4C3G7qjwhKC6rcFH7QggvCxLk2ugs5kV55luLrHUuvE78ip5vIfB3v6ATVk4Tqyj1miotNdfpux0TNaleBYJaET/04qy7vaCANSwut2O+18yEcCT8Z1E1vaS2S2kGZ7prM/eg3CT6gbMM3qpa8XfD771r5hAMIvqxiXyFOBuKMlc3xaEekLxSv9QwqG2NtBk5g2KE0zmNw9556/xQdiMYXAY+vykxWEOW3ZBghq7u2DqSQWS3JqtOYn3/ve3guMKBGQ15kf1lE6uUpKGIX9kthrCQZ5raIKzE9fUsBskm16QTNVLe05flm3y4eHwLphRBGVW12k2O+LKe/e94lm3skRerHh7voQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(54906003)(66946007)(66476007)(66556008)(6666004)(6486002)(38100700002)(316002)(508600001)(4744005)(26005)(9686003)(85182001)(8676002)(5660300002)(86362001)(186003)(8936002)(4326008)(956004)(6496006)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?enJMQlJ0Mm0xSm9HdW9aNnRwd1dWblNuV01iSWhUMktxM1lZU0xPV3JYV1pj?=
 =?utf-8?B?YU1jdWFrUjNZdDcveXdrU0N6bkJ3NjlEOXY4L3d3cjV3R0tNTWRoRisycFZ1?=
 =?utf-8?B?Y3JXL0p3dGR1bFBzblQvY1M5LzNMRVJCY1RCR2ptdW5Lc3dXMVgzb0RsOHdm?=
 =?utf-8?B?TUViVkM2MCtnRkI1Y3RpcFQ2U1Y3d1FYWmVKc0tGRDZZUEw1cHYvSkI4S1lu?=
 =?utf-8?B?bCtFQmhsNW40TWc5L0p3WU9haVNhSkp3NkI5dDI2dlczaUMxMnB6YlF0OUFQ?=
 =?utf-8?B?TkJzWTVGcHI5Y0dDNVdZcGtjNEFidUJlb0xxNmU3eFkyUHkvTkpjM2ttVTBl?=
 =?utf-8?B?RUZZSHJtUGRJRHdLd2lpTGRtREFINm5VSVVseHVkUFdOTU51MDEvK0FYYWJU?=
 =?utf-8?B?TUVSTGNrb1RSZ1ZUdUZlZUNKQ0pYWjBXVTNnVmJDREd4clNBTTZQOGIyZjVX?=
 =?utf-8?B?K0JJTzdFdDJVUzlpeTFtSFhYQTRORnREWkEzM24yRExOKzFqRUw0VjNGL1A3?=
 =?utf-8?B?VmZMU28xc0tDWjFvYitCSU1Cdm9ja1MrbDJBdDBvYmxrbmJiNXpMeXM4WEFn?=
 =?utf-8?B?VGF2Si8vRkpuc2lFeXNzcGwyRFhNd0FXc0FKaHc5c3ZoaERwTUJEWGl4VFBW?=
 =?utf-8?B?WVJTYXdMc2pYemp5ZHJtbE94bERvditvSlU4dm0vOE95TFlNWFNhdmdoN1pX?=
 =?utf-8?B?NHJ6WUVsUWtWNXoyOGVweGRkNHBUYms1R0JVTDdzQmVHNm5SV1IyTDBvRzBJ?=
 =?utf-8?B?VkJlMmJoMVUvS29KWlJtMmV1aHlWa2FnalA3aVV4N054WGkrL3cwaytEYXgr?=
 =?utf-8?B?NW9wcmYyR2xtOG52dDMyR0s5UXkxdG9VRDJpdTZ4bmMzeFBTUXJZS2N4STc1?=
 =?utf-8?B?T2tvZjQxZERMZ3lxQ21tc2lpWnVRR0xhUllPQXN5WjE1cUV3RUpjdnNCb1VD?=
 =?utf-8?B?am9zWG1ZQm5TbEFZREJMa054dFZ5RnNqM2wzSTNVSTNjZHpzWElHS3JRVjh3?=
 =?utf-8?B?MGF6VWFIeXZLNFFkQ21DdXJYK1JPU2pqTDEzZ2FBV09sWFlaRWIrQm1Zdmph?=
 =?utf-8?B?c25CWEV5R05HZTFLcTh5M2E0bzNXeTVnRFloMHlMdjhFTUdSSnl5SDZ0dXdS?=
 =?utf-8?B?SzFzZlMyWEI0d3MvSkFwYW5JNW9uVzE1OHhtdVhtT0lGenRQUnZ2UVhDTDBB?=
 =?utf-8?B?YVhqWHArK1c2bUhTUUZ6QTlVNFBKNmZsMFQ1ZmlzeTB3bGxnbU5YaXZCRjEz?=
 =?utf-8?B?dE5aNG5xZG1OYTRDQmxCU1lpRnBwak50SDdMSVRNK0hwUkYyWFJqb3kydXZn?=
 =?utf-8?B?WUFPRml6bHltTFkwQmZRK3UzdjB2YjYxTDc1VUlLRm9sQzBHbU1wdmVVUy9O?=
 =?utf-8?B?c0dkR2lsVWxQTnRBQkJsQjA4a0xDL3JobkxnK0dmSFRmZlNKS0lqWUZuOHpM?=
 =?utf-8?B?cUR5amRFbWxId0dKcytmVGpYVG01alRTbHpZbEZ4TUFkLy90TDlnTHhRQmM3?=
 =?utf-8?B?VkxuaVhqLzhDeWdqZE9jYXI4aW9LZmFTd2E5Q3U5dnJNYloybFdxUmV0RXh1?=
 =?utf-8?B?SW4yclI4WS9JaksvRFczQTBzSkZsSGJaNkk1dy9xeHhlRU5VWVV3dEtVcHg2?=
 =?utf-8?B?Z0FLNVh3dk9XNUpVZlY4SEo5VHY2VDFJMFBwU1Zrak9kblBNU09ObWxHYVJW?=
 =?utf-8?B?N0Y3ZHk4L2R1SkhESWxnR1JyRHAzaTljMTl0MjVuUFBLZWxlTHFDczBmVU1a?=
 =?utf-8?Q?HwnpTgl2XPrZzSRwHTcyZqMgewobTligZxIJsOa?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e50d0d3-7154-4c81-5945-08d97dc921b6
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:01:45.8705
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ws0djurD9ZukrIJ8N1d9XY1b8J+xQLZ2Du0bHFhK16oWnGggqmGKql6vdi56w77JBXMYI/yKCZYG0hR+CZpm0Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4970
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:17:15AM +0200, Jan Beulich wrote:
> Leaving shadow setup just to the L1TF tasklet means running Dom0 on a
> minimally acceptable shadow memory pool, rather than what normally
> would be used (also, for example, for PVH). Populate the pool before
> triggering the tasklet, on a best effort basis (again like done for
> PVH).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Tim Deegan <tim@xen.org>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:27:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:27:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192734.343328 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2H3-0008RC-32; Wed, 22 Sep 2021 13:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192734.343328; Wed, 22 Sep 2021 13:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2H3-0008R5-08; Wed, 22 Sep 2021 13:26:57 +0000
Received: by outflank-mailman (input) for mailman id 192734;
 Wed, 22 Sep 2021 13:26:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT2H2-0008Qz-04
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:26:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c04280fc-1ba8-11ec-b980-12813bfff9fa;
 Wed, 22 Sep 2021 13:26:54 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-5Bbz8J84PbadPrR_Ncu_lw-1; Wed, 22 Sep 2021 15:26:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5470.eurprd04.prod.outlook.com (2603:10a6:803:d6::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 13:26:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:26:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0003.eurprd09.prod.outlook.com (2603:10a6:102:b7::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 13:26:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c04280fc-1ba8-11ec-b980-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632317213;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FNsIFDNt/dpD1F3IS8KbBBhMWnQiexgrzg+BE6l5F6Q=;
	b=AqJASzoqZrAEn6g8wjOtPgxIGeQduBPiM+QJitu8Yt2JtZMfXLoKAStdAWL/fY4dYtxVsh
	eOO24O7m/pxlGexBenm8Vas+xiQQU8h2vOqrxMYEEqClhnzWilJW5L+F9cgJ/PVq7o2i8M
	O/CyTyzDprxS7o5q6nZYptp5taxIklw=
X-MC-Unique: 5Bbz8J84PbadPrR_Ncu_lw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PRs5LoeUcFgae5JqpPH7QLmRX8Dh3KkDiAJ/yc1VogPzfFovEpSLT6XBup1X8fwfp5x4BYLwE1BVznCcoEWt0X/jmaD5LQH5pu5Y993GcGF6kGgCkzGuPOgbJoUitCyuUzxvmDYHefR+Vdxs65b+W4C2Shqry9mu8DWnhiLNV42zbHuYvJxJXtx0ZmpiFh+8Y1Qh0okwACR7F0ixtsZ4Lu9EPK8HwPpMxgJLoRKWUeYt2Kwmb2I+QugCdj1AthYBcnroE/1OZOrUoLm259vsQUoJecL6TU08/OvMZmDXUGCVDxgJO7mwce4JEAmwXbQ6uFUPuwTh12Idh09vTQLI/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FNsIFDNt/dpD1F3IS8KbBBhMWnQiexgrzg+BE6l5F6Q=;
 b=f7kY+Sg4Dg+w70ecf8x2dRNdslUdFG8YEn8LnxI6He8UTj26jv9ECUX3Bp+6mSuXyHFcEYVtJVdtfwArMn9geFojU52aUBChWzh25Dj4IoDXVpsrxRjRrWdLnHrfNaadzI2rgU1sCcXkf+NxoV16vxRVxDElWkt1DLO9EFewe0yVz8r5SfCZFRWvY6Bmr4hMxDaOpIsZlE6fg/YlbvORnNM/EtGcIEC2aoAM4oGsmmiX7gc5xAZvPKtHgnkM9d3sK7U7u2HralUWOqP98D3KWvNxsdw/0s1mlKmn7IkANB10rkneuskw0Bfo1Mhe+DwS9NqhrSzUkn8xiznuFPLWDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Ian Jackson <iwj@xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, dpsmith@apertussolutions.com
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
 <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
 <24907.4491.895219.37845@mariner.uk.xensource.com>
 <952a9c35-39bb-d229-0bb9-a1f4e893b61a@suse.com>
 <24907.8642.479155.35952@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <54b1b5be-9f43-9513-d274-105d28bd5a58@suse.com>
Date: Wed, 22 Sep 2021 15:26:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24907.8642.479155.35952@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0003.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dd1673a2-2128-4bf5-986a-08d97dcca24c
X-MS-TrafficTypeDiagnostic: VI1PR04MB5470:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5470842ABAD5196D658691ABB3A29@VI1PR04MB5470.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aG0Tyj33lbr3LKol/aQNjJwdE58rZxtmd0hbV+J8AcuY14zNV8cTEyFUMJejFAZALpPLK2iL5Z6qv8Y5/SR6rsz4OYxzCgTJsmwcgTWUZWkme3g0L1Mn1P306Vn4ybc6AWXIV9rm/eyblfPJ0ROql2kqHWowu8IC+odFnY+Haki3SFAFSltlRNg0YZ19Teu9hBoNF18664/t2JAkDjHfUgaWXArCjAMJJ+6WJyw/eUNrB9rG69huKqJlmNdRRfVCTi3wxXi8pXXKv7/qWPMLfPOzuWp9Cu7TAUwwi7l9/lxg/wPHEcLGF6w909q3Im7GwbxwlQzGwCZkeXMeen/AvHIbs0C1fijwaold58DJ4muXQrC1BeeJOV1ZNss0rHxTi9aRV5E/OFC6ucnl5AiVcfURT4KxLvbm933+SamPaTcorFkLQAnF6QlbrC0WrAv7E3KljbGCTpRCbeln6jP72r84FCiuK6vf0B7PtFireC6BbnMVcsB1is8BJLffaN6I4ZJ4ylfo1HqxNwR/WIF5zPOz99d/9bFsfD1gEYkrSoDrbwoCukrQ3c7U3dij3/qr4WM18NsldrqLh72oi3ZUHM96gtxtUBnYWgfY9oYihecvMPQ5VGz3xubUphghKETMDpjNh2w+gW9ApIHQSCv/QF9USBSA3RBGSFILs7dd8P9AULJ+soi6f0aZp9q5O/rT8n6JjSOG4VEvy2vEDBOzVtPda7RBghsRx9AF8P8AqTM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(8676002)(8936002)(38100700002)(186003)(2906002)(2616005)(6486002)(508600001)(53546011)(31686004)(66476007)(36756003)(66946007)(956004)(316002)(83380400001)(31696002)(6916009)(66556008)(16576012)(5660300002)(4326008)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WCt0ZmdvUFFSK01ERGlrdGpuOU10aXkzOGhoT2pzUjVxWVVMVG4xOFVIZFlz?=
 =?utf-8?B?Vlo5YkJCcStURXk3YThHNVJId3VFU3A0KzhxWXdlTzA2cUExL053K0pKcFY2?=
 =?utf-8?B?dEczMS9FUDYrT2o2Y1g1UWZQL2JvRm81ZEduREpXZDNNU1VOZnhwSTFUVVVq?=
 =?utf-8?B?V3pIYUN1bVpPZ1N3ODhiS1dTUFN1VUg2WmZza0t4bmFULytWK3gzdnhLc2J2?=
 =?utf-8?B?MEdpVGo0WVgzOGhnZkNFWFJ5YmtKZ2ZwZTBqSTdncCttNlk1cUx6R3NVUGlD?=
 =?utf-8?B?T0ZZYzNGaFZ4ZG9GZnV1MlNQSlV5aGo1OGFyL21ReEdoZUluNWZNZWlwRjZo?=
 =?utf-8?B?R2Z4ZXlPTWl6cXFFMllMTS92WjRkVjdXaGx1NGs3NXJiSGIzWUhKOVlxQ1BI?=
 =?utf-8?B?TDVSTkswbHdKcUtYTjdFNUNUV3dmVUw0bncrWDdrZm8wOUY5RnpzSi9FSnhm?=
 =?utf-8?B?UkYvdURuYzJYMWdTbkZldDdWRUJnOTNVeExkRi9CSTFyQUQrNkdiSVNsUVhs?=
 =?utf-8?B?d1VzamlsRUo1QkRaSS9UbGFvTGxVbmtka2didEtBZG10V2YwQk9BZXhnUVBZ?=
 =?utf-8?B?NEQwSmFQTXVOSXhRN3RDS3lJZ1NCWWtab2hJWEtRZ3QyejdhZ2RWb0xpTXBl?=
 =?utf-8?B?U2x3a3MzR3JzUHFhYzNvNDN4d2VCR082K0FKcUlna2FtUFR3VGNBTmRXMmQ5?=
 =?utf-8?B?ZVQxbXF1Y3J5OWkwOHVlQ3FabnVZOUFQRDIrd2MrTEhiK2pISHpSbmJyelFI?=
 =?utf-8?B?L0pmbk1Mdm95WXZUdzU2S045UlcxMVVoVmM3RWJHaXJoaktBUy9CN29KM3ho?=
 =?utf-8?B?YlBMVjc3a2VrQXZvZzJlRXhzcGk5RG0vQ00xalZTT05OckxXR0g2NE5UQlFw?=
 =?utf-8?B?UjZ1NTMyb2VtS05VaW5DcUtFU1Y3bGRlUDdEUzgxYSsxOTRIQlBmRUt3U3A4?=
 =?utf-8?B?M3B2d1VXSWk4T0krenpxajZTQWxucElvcU0yT2lOSkpTYkYzK0k1TVVZU053?=
 =?utf-8?B?aGtMVWFJanFLWlhxYTJndm5LUERVU0VTYjZuQXhHenJtdVpMRE50bFhTdU9J?=
 =?utf-8?B?ZVgxMG54cmc4Ujh3enpMOGZNbWI3MFRjaFFsclEyUG9FbHo1TmpkWVZHcGY4?=
 =?utf-8?B?SFB5VFovMVcxWTA3THFBNStCaVk1M2xuZDNJR0hndDgrWXhZWXZwaUZteXBr?=
 =?utf-8?B?RFdLTkdIdHkyUWVpcjVDdjhpTnpDUWMwblhFTVdTZkh2RXErdHVrMHFocEUw?=
 =?utf-8?B?MXpvOURhR0RDUDhaS1lxRXB0L1ZlR3dqeHUzSHMwRE93ai9uYlBTcmxyZTlu?=
 =?utf-8?B?cktKSkNmNkMrRmp1dGtCSWRLNkduaHR5L2JBM25yQVI4WHZLcWdTRFlQWWhH?=
 =?utf-8?B?OTU0aVorLzd0dnp0OWFUajVqVFZ3dVZ5YmJsN3hLQkRZazlXYXlxRmZPOEhH?=
 =?utf-8?B?OHVTRlhoZ3FUeWxiYXVFOEVWOUkwR1JockZvK1dYS1ZiaVA4R2dFc2V3L3JW?=
 =?utf-8?B?V2lVNW53dlF2MXRGek1TQWErQlRDV0t4Z0tTbTd5clRJVFZOUjNKWE03SnNs?=
 =?utf-8?B?cisycmlMYjl2NW1sODIvWTZjN0hwcGpsM0NXTUhlMlZsOCtSbzgxdG5qRHlw?=
 =?utf-8?B?cHQ1YUZCVExVUjhXYVhiNXU1SXNadjdBelJITDRRYnhEMkVwcUJmek1wajdF?=
 =?utf-8?B?UlZhWGgxTlFEdDhKdmk5aDZZeGRGdC9HTUs1OFpHY1lWQzhiYzNCNjBWRWpu?=
 =?utf-8?Q?vZyiPW0dWcn38ZTSgtszYeNlPWtF6t4QDECC4Jy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dd1673a2-2128-4bf5-986a-08d97dcca24c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:26:50.2303
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vPl9J8uC9zgAoN3bh4uJ4uKvi38ri6eycpd15P2P5E8U4bfYUSeVw3PIEFvevOuCzK5OM739OVUGcTkANFUW3Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5470

On 22.09.2021 14:29, Ian Jackson wrote:
> I'm not sure, though, that I fully understand the design principles
> behind non-order-0 allocations, and memory sizing, and so on.  Your
> earlier mail suggeted there may not be a design principle, and that
> anything relying on non-order-0 atomic allocations is only working by
> luck (or an embarassing excess of ram).

That's what I think, yes. During boot and in certain other specific
places it may be okay to use such allocations, as long as failure
leads to something non-destructive. A process (or VM) not getting
created successfully _might_ be okay; a process or VM failing when
it already runs is not okay. Just to give an example. The situation
here falls in the latter category, at least from osstest's pov. IOW
assuming that what gets tested is a goal in terms of functionality,
VM creation failing when there is enough memory (just not in the
right "shape") is not okay here. Or else the test was wrongly put
in place.

Therefore a goal I've been trying to follow in the hypervisor is to
eliminate higher order allocations wherever possible. And I think
the kernel wants to follow suit here.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:29:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:29:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192740.343340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2Jc-0000p0-IR; Wed, 22 Sep 2021 13:29:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192740.343340; Wed, 22 Sep 2021 13:29:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2Jc-0000ot-FP; Wed, 22 Sep 2021 13:29:36 +0000
Received: by outflank-mailman (input) for mailman id 192740;
 Wed, 22 Sep 2021 13:29:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mT2Ja-0000oj-AT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:29:34 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1f486152-1ba9-11ec-b980-12813bfff9fa;
 Wed, 22 Sep 2021 13:29:33 +0000 (UTC)
Received: from pps.filterd (m0246617.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MCsSKZ028874; 
 Wed, 22 Sep 2021 13:29:32 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4rbwq8-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 13:29:31 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18MDJarc107935;
 Wed, 22 Sep 2021 13:29:23 GMT
Received: from nam10-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103])
 by userp3030.oracle.com with ESMTP id 3b7q5mxwrv-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 13:29:23 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB5154.namprd10.prod.outlook.com (2603:10b6:208:328::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 22 Sep
 2021 13:29:21 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:29:21 +0000
Received: from [10.74.114.106] (138.3.201.42) by
 SA9P221CA0020.NAMP221.PROD.OUTLOOK.COM (2603:10b6:806:25::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Wed, 22 Sep 2021 13:29:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f486152-1ba9-11ec-b980-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=5/k6rm/vumjDkkH8Yl96dkTOmRNZ9ESQftaZaR+KVP0=;
 b=NVC+C1DqXlAyp7V7ofXmNWyNuKLs+kYsgmZgQJP3cKQBvYxbHPeyZP8etGTahQBCnh7Q
 zNieUbc5A/12tbyE7Omv+5Q5V5XLm0NEozRw3bxtW6NFbmDcHDRavqnz1BpS+hdZUdbw
 h27Ta+nanTz0gBGyW6vK0MTs9CvqATKuGdqzcJ7Hij7QAsdf0CM1D9c2+uIr+VsDMiD4
 MaWxum9iy9dzf9dAqQ3e5kXiFZrjXDS4EwnmfMRalEQTphUR13ve4Shq7CoaAMnMDvUR
 rUG5czgqWJrDeeI2V0Wuj5f94ogAED/zxE1ZZ64MgK2sU7CRoMmVdZomvQDS3dhwLn2F gA== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PkO9pLgnAQzgeYNh4Lnu8w5vKPa+UjrRshVspHWJPduS8wjGaUNYR1lU/x87Uko1Q2cGmQwwBfT8ZoNhMNON7DllYYBwTDUwOoRgF+GsCJ0Us3fJExR00UZJU2BFy+CAiQDf0ELoniy+fqHBE285wGnOPD27fzqkD8JobigDFrzG7LHqO0e9wOz7TvweLH1mDjjuM7jsD0CIxZmJNCMHBcAsQBMyZZQmRw2+9AhG08ylxaE3gXu/mCVPD+j2C/cB95vPzF4UAtPDoK7cXPFMnbMGKRqjgLyiKUsEkwVBMb8Q5uaoGfMp1YJCC/vvGDN7xUTQAXJ/M0vrJrK5I94cjw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5/k6rm/vumjDkkH8Yl96dkTOmRNZ9ESQftaZaR+KVP0=;
 b=RVD/PFOa1MwjpmIwZ/aSlYWKIHbjIuOQFFqKAGpg2WFu5MkYUGMlXouXhoqV3D3rPAcngaTwO5t7F1KXT89+rdMU03te/0R2PK2rVzqosU1hcbVEZFl3coQFjQYty7fPICBoePtnsTWhoJ7eYaZ8kJ41j7/1UGpmTyGKNlZzETjEKDeYydIRyMIIN1XXSTjf10FGbVio5vK1JwYy+doLRPt6paiaqltU21hzTQ8PUVLMn3QrOylz7vaB2bA6Psb2DeokbwOsdMu55GoCFJYqudAKeCuSPAKEdgCLY61t9eCB1kxKBfCfH7IugHfu80NO7AUz7PzYrCZdc0lgWTGbnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5/k6rm/vumjDkkH8Yl96dkTOmRNZ9ESQftaZaR+KVP0=;
 b=vL01eEISu6G3OTNuWYkhfrdRF2BLlswhjHAiHXUN6y6QYmGlYDnfJ0f9mdSQeVcN0CQRGxqt1nY30eMmUV0WG4cyWwEpojFUwkwLsCs4ncTkcc7XUMpBGg4t3QrhVds0W2GKytlWhIqgxdQua85JDKtm5AJhJQxKoRp5riyJ1I0=
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH 2/3] xen/privcmd: fix error handling in mmap-resource
 processing
To: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <aa6d6a67-6889-338a-a910-51e889f792d5@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <c0c84258-c2ee-f58c-ae9a-5f8bdd75f0db@oracle.com>
Date: Wed, 22 Sep 2021 09:29:17 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <aa6d6a67-6889-338a-a910-51e889f792d5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SA9P221CA0020.NAMP221.PROD.OUTLOOK.COM
 (2603:10b6:806:25::25) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bf1d85c0-8974-4c12-95e2-08d97dccfc8a
X-MS-TrafficTypeDiagnostic: BLAPR10MB5154:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB51545B9CC70422931FD15ECE8AA29@BLAPR10MB5154.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	yXPL13EOsFoZqLkjP4mKSTI/AdMNYZVNvm/Iti68+op3pLuKF4vjQRNlBF9d1lmzCXioHJ+bE2DMUsOMk3Bqhrmll4ddXLQCMhmYheIvQ6u7FzxbENg8YbNW3VFlHqHFZrXvgTelvZHxILOZx/5uGgBaTeXl0z9ZmNZxdzKJ4TrNt316eSYPjf1Q3j+FaTH0FYzMcOBvCbLblB/dK5QIoZT3kOB+wYt67Y0t98nWkCPtA6B7sLRUWYes5VEU+8Jo3O7W0Ipf0zkxvWggh+40oRGcndRkj+DIC38noIpHX01WRU/8HQPnGPh8BgpGWn0d12whFXKIpRvKYJSWp+ru//rHBwHLLVhbaKReVB1d76phcoAvtLxuHR39D22JuGNOXOrKTVk70rU/EiHZ+GO6uXx4iNzEXrAJ5rYYrpAqas3eqJ1H2P2YdHM3Lg6vKOjGtF7AJneoGE/jQMnIadJMPs+Xs6lCfWq6Woxs1PoAxiFMhk1XcuT1v/YOyU1T39h393+2nrepGTwi9sJNFmN3TgVaKnMUsdQh3waY6DrwrGUukZbNlyiyEqkGdfrx8FwjOv0aWJ/3r7L7C6Jn1wMwhJhRrx07TGV48hsTq4zDoUyC0odpc2giBt67PaMNSP4xzcUP0N/0vA+BVxrgOQ5+W3jEuXkmMkXpgw0OY54CsiN0ZWGGamMvYfzbY9KKNw1r1QdsopHtRLgC0evQybLVcega9sDLqqg26+941XRoq9HDQvdpzzBddsMPmiAeCPJq
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(31686004)(6666004)(36756003)(8936002)(86362001)(26005)(508600001)(54906003)(16576012)(5660300002)(110136005)(6486002)(2616005)(8676002)(956004)(316002)(38100700002)(66476007)(53546011)(2906002)(186003)(4326008)(66556008)(66946007)(44832011)(4744005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?djRhUGdOWE9QZW1aT0tGU3BXeW4wUDdqbGd5U1AvQTV5WURha0hrUGVVNWpr?=
 =?utf-8?B?emRSMWJzcnNtbHFLemQ1OWlZOFZseXNWK3NyN2dWQ2JQeTkxV1cxN1lYM05l?=
 =?utf-8?B?OWJYVVh6Q2sxQ3FRbEZyOEh3U3pidDdXMkdPZHI3Slc3K2JCWS9MYWxqQk1a?=
 =?utf-8?B?OS83anErVkhTWHkweDVtUnZqNkluczl1bzlraEUrejR4QityQXZRK05YQy9G?=
 =?utf-8?B?YzZjTjRFaGwrZVdrSDJLYUEvL0V2YnBCcmp6UCs4RCt1cktGWmVFRGZ5QTJP?=
 =?utf-8?B?Wm5HNHpQSlhHN1R6ZGtyZDJ0WU83N0lIaW14OXVvVktvVkJDb1I2aFFMdXlM?=
 =?utf-8?B?cGZmVVRUUUsreHk5azA4QTg3TmNMMnpTN2VTMm5tWm85YjJHWFJ4YnNCWlpC?=
 =?utf-8?B?amlnUzI4aFNhKzIwdzNXQ0luT01NMjBJUFpDOGRmeHg2VUNEODNjZnhsTHVC?=
 =?utf-8?B?YVdHU0l4bkM2bjlxL2VvM3ZBaGdidXJYWnlmUWlhSzgrUEtvZlduRStKTzVV?=
 =?utf-8?B?RE00NEtMMFJEcXVUWXc4Z2hYYjFlaTcxanRBWFdGT3JKWXRJZTh1ajRFL09i?=
 =?utf-8?B?eWovdGZJakl5MENIc3pCMlJKR2FKMEhrTCtRZ2cwcloyeWtjMFkwUC9UNGlJ?=
 =?utf-8?B?ZE9Rc0ROdFZUb2dwaXZEUDdtWDBjaFVpbmk5ZEdVbFJPdmxJcFV5blhldVc3?=
 =?utf-8?B?ZTkrU1FmTW0wYnQwcnZMeCs2S1Ara3E0UFhrWllBM2hyWjJHWFJraGloRDVD?=
 =?utf-8?B?MU44c0tVRE5NOGF3azh6c3NIelU3ajZnNG9FVGp2YlpxOW9IZTlHWDhIUW5X?=
 =?utf-8?B?aGhkTkc5cGpVVVNyeVk2OW1FTWgxQjhiV3NHaHhDdDF5bUEvdEhqd08rYTdQ?=
 =?utf-8?B?TnpGVzlvU0RIMzFoU1ArOW04NlhQMkh0NE12THdGTHFCaWlkRHZYL083WFlX?=
 =?utf-8?B?WlpBREd5L05ZTmhRbkZNS2dzWDhDVGlGYUJISGhCQzhYKzA2QnFER3NTMTMy?=
 =?utf-8?B?Z2FVQjBLTURXZWpnZm5ydTBpWnMzMnB2RU5ETG1PdFQ5allPWmlKcStDVEhq?=
 =?utf-8?B?RjM3ZlBubXJOMktQN0tyYlhyeE9ZUHZVNlB3TWVpTU1vL0R4N2xSQ0xKTWw3?=
 =?utf-8?B?Z1pFUlRYWkxvUzRrOTR4MzVBYTM2dU03VWpIdWl6WlJVWGZxNkh5VUVSYTZM?=
 =?utf-8?B?QkRxTGZJUWRsSkdKQW41QmRZcVBtRm9pYjlScWExejNjQ2s3YlA5TzFKU0tK?=
 =?utf-8?B?bloxSlROTUsxMVRvQk9PdXQ0TlgxaW1DOGU5UFBKU0FEZ1Z5Q2x5YTdWN0Q4?=
 =?utf-8?B?OWFZajNpU04vUEZFMTczOWhRdytUVGlvcE9mSFZFdzIyUER4ZHBkUzQ1V0tQ?=
 =?utf-8?B?SWhlSGFxY3o3U2NmOC9vS2pmYzUxMGxhZ1FhWHRYRTkvS2JxWGxNM29qVko4?=
 =?utf-8?B?c004S3FBUnJ0Yk9KK2hsRWZ1Y2d2Um84TjBrbllWMkwxM2VWTVY1ZUNKbUdO?=
 =?utf-8?B?L2M3TzFYUjhjRVd4cm1WZFBEakJrZ0NGb2FjWjYzdEprajB1b0Z3SlQ5NDQz?=
 =?utf-8?B?WXZZaGtGN0RNR25VekZJbjR6amtxcUtIYkd4c1Y4cE9XZGZOZHFvdWF3ODd5?=
 =?utf-8?B?aDVlUFNJVXVvSVdmUUFBM1BOUzZvNFFwWmpQcHlTYm9lai9FMDh0WGU4aDBK?=
 =?utf-8?B?OGtFR2hXQ2theVdLTUYrU2k1UDQ3MlVZRWJTOXhCSXVDamVkN2doWU9KUTlp?=
 =?utf-8?Q?hsdYfgXKAh2sI0kZLgwmF+edwfbsniRHtDZpoaB?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf1d85c0-8974-4c12-95e2-08d97dccfc8a
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:29:21.4342
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DmkuQmqJaPsbHVVLYK9iVhRnrKym2bBHJltBNHfwFp3yacFP5OObw9H9Fz1v5S5Gism+ynXyxgAMBLOJNW/oG6IW4qOY9N8tZnuegnQ3X9w=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5154
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10114 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999
 spamscore=0 malwarescore=0 adultscore=0 mlxscore=0 bulkscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220095
X-Proofpoint-GUID: x8abET5ak2bIf3aPfYFOW_NQZTVj0KYb
X-Proofpoint-ORIG-GUID: x8abET5ak2bIf3aPfYFOW_NQZTVj0KYb


On 9/22/21 6:17 AM, Jan Beulich wrote:
> @@ -817,7 +818,7 @@ static long privcmd_ioctl_mmap_resource(
>  			unsigned int i;
>  
>  			for (i = 0; i < num; i++) {
> -				rc = pfns[i];
> +				rc = errs[i];
>  				if (rc < 0)
>  					break;


Can the assignment be moved inside the 'if' statement?


I am also not sure I understand why we need error array at all. Don't we always look at the first error only? In fact, AFAICS this is the only place where we look at the value.


-boris



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:32:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:32:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192748.343350 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2MC-0002Dv-4o; Wed, 22 Sep 2021 13:32:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192748.343350; Wed, 22 Sep 2021 13:32:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2MC-0002Do-1k; Wed, 22 Sep 2021 13:32:16 +0000
Received: by outflank-mailman (input) for mailman id 192748;
 Wed, 22 Sep 2021 13:32:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/3we=OM=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mT2MB-0002Di-Cv
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:32:15 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0bd1b71a-0f4c-455e-a74f-97d7c2617258;
 Wed, 22 Sep 2021 13:32:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0bd1b71a-0f4c-455e-a74f-97d7c2617258
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632317534;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=FfkCmcAGjvXL3O0pC+0tNDIR1A3GzwRI2U1LyU76hZc=;
  b=e6QtR+oTA21KeijHw6L0fFTi8YFw/xN2PfvVEvuVMEYe2BguiOuvukuK
   dAZ0szLoeTM1GYi6wjAxFopgjdEae8aztz28EhL4DN3MQNxQnPZTRbdh0
   wDs/1wCtNEgjNqORZ6Uj6eyRCwhECTGuoh/yFcaxJv6RCzNI2Mm0LSw12
   g=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: aISvFQS+9GrrLuSrCmA5FOoN7oQh1KPlIFA1EaALcM8iJBpHn2Wnob1rf7sBfrZ5MbAtz4R9XK
 ynxwk5xl55PeiE0tPvZ6BaprULQMuD5HGNA/w2vw0YMVFg9InW5zck8lDacd74Hr4sKvDjPxta
 lWUP+aam6J3QYZqtLD97Prse78M+dd3RKLPuZei1RJzYLvGpzxhjDskjqN8kJrkcWg+4IEOHXf
 NZMOxrIkSpn46dOQFQF0uhgVb2DpnGjmNr96lerJCzw4ZzFMOKK0yDDnkCr7PadRCdY0DLnm5T
 kk7roPIKSUu94O+kOPgADfrw
X-SBRS: 5.1
X-MesageID: 52917235
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QwzW+KBMar0uiBVW/+fkw5YqxClBgxIJ4kV8jS/XYbTApDhxgzVWn
 GQaD2CEPvfYYGqjc98laIXn90pSupPTytYxQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00g7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/szu4ktR14
 9V0kcaqYBcTMbLjwt4XTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFhmZp2ZoVQJ4yY
 eI8UDdjPQ3FcSFFZAdIT81938muiEnwJmgwRFW9+vNsvjm7IBZK+KjgNp/Zd8KHQe1Rn12Ev
 STW8mLhGBYYOdeDjz2f/RqEhOXCgCf6U4I6D6Cj+7hhh1j77nASDTUGWF39puO24mahX/pPJ
 kpS/TAhxZXe72TyEIO7BUfh5ifZ4FhMALK8DtHW9im1xJvZ5wKyBlEgFCRoeOYpq+BsfzY1g
 wrhc8zSOdB/jFGEYSvDre3N/W3uYXF9wXwqPnBfHFBcizX3iMRq1EueEIw7eEKgpoCtQVnNL
 ya2QD/Sbln5pfUC0bmytXvDiiilznQiZl9ovliLNo5JAwUQWWJEW2BKwQOAhRqjBNzAJrVkg
 JTjs5LFhN3i9bnXyESwrB8x8FSVCxGta2e0vLKSN8N5q2TFF4CLJNgNiN2BGKuZGpldImK4C
 KMikShQ+IVSLBOXgVxfOtnqY/nGOZPITIy/PtiNN4ImSsEoKGevoXE/DWbNjjuFuBV9zskC1
 WKzLJ/EJWwEEp5u0Dfeb75bidfHMAhlnjiNLX06pjz6uYejiIm9E+ZdbATWMbBns8tpYmz9q
 r5iCidD8D0GOMXWaSjL648Da1cMKHkwH5ftrMJLMOWEJ2Jb9KsJW6+5LWoJd9M3kqJLuP3P+
 33hCEZUxECm3S/MKBmQa2AlY7TqBM4toXU+NC0qHFCpx3l8Ptr/sPZBL8M6Les96ehu7f9oV
 P1ZKc+ONetCF2bc8DMHYJij8IE7LEa3hRiDNjaOaSQke8IyXBTA/9LpJ1O99CQHAietm9E5p
 rmsilHSTZYZHlwwB8fKcvO/iVi2uCFFyu51WkLJJPhVeVntr9c2e3Cg0KdvLphVexvZxzac2
 wKHOjsipLHA890v7d3EpaGYtIP1QeFwKVVXQjvA5rGsOCiEomf6md1cUPyFdCz2XX/v/Pnwf
 v1cyvzxPaFVnFtOtIYgQb9nwbhnuonqrr5eiA9lAG/KfxKgDbY5eiuK2sxGt6tswL5FuFTpB
 hLTq4cCYbjZatn4FFMxJRY+arXR3P4ZrTDe8PApLRio/yRw5reGDR1fMhTkZPax91ep3FfJG
 dschfM=
IronPort-HdrOrdr: A9a23:Farl3Kg9DFVC2g4hXhNskWTT+HBQX1R13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskKKdkrNhQotKOzOWxFdATbsSkLcKpgePJ8SQzJ8k6U
 4NSdkYNDS0NykBsS+Y2njJLz9D+qj/zEnAv463pB0MPGIaGJ2IrT0JbjpzencGNTWubqBJcq
 Z0iPA3xQZINU5nFfhSURI+Lpb+TpDw5d3biVlsPW9R1CC+yReTrJLqGRmR2RkTFxtJ3LcZ6G
 DA1yj0/L+qvf2XwgLVkza71eUWpPLRjv94QOCcgMkcLTvhzi6ueYRaQrWH+BQ4uvum5loGmM
 TF5z0gI8NwwXXMeXzdm2qi5yDQlBIVr1Pyw16RhnXu5eT/WTIBEsJEwblUdxPIgnBQ9u1U4e
 Zu5Sa0ppBXBRTPkGDW/N7TTSxnkUKyvD4LjfMTp2Y3a/pdVJZh6agkuG9FGpYJGyz3rKo9Fv
 N1Mc3a7PFKNXuHcnHiuHV1ytDEZAVxIv6/eDlHhiWp6UkQoJggpHFogvD3301wtK7VcqM0ot
 gtac9T5ehzpsx/V9M3OA5Oe7ruNoRhKSi8Rl56Gm6XYZ3vDUi946If0I9Flt1CR6Z4uKfauK
 6xHW+w5lRCN34HN6W1rdR2G1b2MT6AYQg=
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="52917235"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OwDVWbvrbjh2h+GONaKrGoEF7jksxeWDatv1OC7LeMhqPjmJgSfhTvyqm0AI9fDg8cSfMtq7yZH1mqJCQcJG1yAbSQHlc2yUWESt6SjA1MjbXu3GVFG07BXiSbJzI/586BKG15NOdVIyrzY2jgjMyE+Rq1H7t8v9aYiah20n0BT+CNRM09OZ1sPTq/mH7YwyuxWk6LG7UXQbg8GR4yKaa12YfMXvgclkRQnykbl1iq8McM7D8ly5JK6KSMGRofXMa8U3j8aPNzylC9Y8tYGZCqkbPau9DCUthn2QYpxg7eoJG6aClZG6ngTMzaXu3qH4wsBijvFjJ+MqHCByzvpCNQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Z8SPcLaS2dRzpusqtPbo+7FqWUmu2TaUK7JgsfBqOT4=;
 b=bIgwMPF59dyffWJSimu7g3CQReC5NKk5hepe992mPLbYRvY2+xZM6cUk5IKCTOFvnd3EnSaZMTHG68dsIY/c1fMgNtNaJ+iSNBVW2N2zmVY0o/WvFjpyLu1mS5h6b7tSllP4Pw7UCgKF6aHjY9WyH1GuCg2zG8azz7jfTgMSJd6PjFf9UDnhnOTGZIy4hlnUwxSkXM5RgdjZBjzSUPiZCRETI2i2vAjU4w9ohWW9xuK86IPkSbfKwdhYcOfBGXecaExK76PpscGeBVaJ1gvNUqPpyvNrLN+NBRvuiphW4TC7Xd93HXr9qfkvof0apOddj2TlEYa7MSfLDo1zfmUZcg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Z8SPcLaS2dRzpusqtPbo+7FqWUmu2TaUK7JgsfBqOT4=;
 b=DEhWn3CIsmZkbhD9aA408A7YwHNhHjC7ue++ANQKiD+u/CXidAgeGPH0F0nrg3lSRJ74kLhR+Nn6PgbKo+17qY0OOB8lgN76tkjbdXiFfzHoDk3vnYiLzr3enGU6T0GMrFuTE00ntqVVzYYVsmJKIyRkRY+OqS5xC3iDzdMJkVc=
To: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
	<roger.pau@citrix.com>, Tim Deegan <tim@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
Message-ID: <b2a2fd30-225b-2b79-f160-8ed2e4ad439f@citrix.com>
Date: Wed, 22 Sep 2021 14:31:47 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0112.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:192::9) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 50622a34-c22d-45ee-5aa8-08d97dcd5784
X-MS-TrafficTypeDiagnostic: BYAPR03MB4293:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB42931FCD0ED85C200A4FEBA9BAA29@BYAPR03MB4293.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 89VE9oUk4XS5AdPWZpuC1BwOa8zPxOT1qNzIvnp0qywnnrdt3nmxzWWm/YfeG7byGxHgMlyDCdzPY72M6ioU+2Dt38LLZucksFk3ArZzE8jDVRQJTXeCvDkEzvkMRMp8x+Z2gc4L4U/SavCCSO6t7NfTKXCfVtj/e7xZp5zCqQiQe84sEeYORrP1K7Sc7jtZxnqTOZyBVOaSFpqosuoS4U9kned6+NP3Gi5dDAYYYeR/7MDOdV96mrjh1HEZOg6V4Dg0EqUjvpt2QtcvXGzCUOuK1NYPNYNTd5wxU8c7EO/uTqdjLWYFZc+Y9xyGQ6LVpDeYJgDxukI8sDUH9B8tmM8qtn36PScSalMaNqtFFswEWGwVHNiMg71BZMP2e+5zIeW2dNESrXB2ZIWntne29ti4ZHuQ92Klb5R6o45E8hd6KlJLZq9NTDesKa69mj8nDGCliqcrUJn+8N5jNTA+/NMLNzgQe+dZijKMUDUgootgM94q0zQ4tho79YrnYcifhtWh2dx52WzG78zAzu34AvENQMgyR0meILUIX2tGk/0jOtqdSFtpw3Zr9cbyWeFdgTcsHTeJIucT4nNnTiTbDdKaR8RLeME5DcU3grH/lU1SDzgS4KvDogI/OGCjUcwSEPiodoRi32ZsyYl8FhN7VthUOF8YVwjF/cbOMlFEuAeXMEJUmSxebDrTsbesUf5MTrDMEB5uFDEO8ci+ThKO6hLH0vrX4mCR0WkoXj5lQOpFTUldFCe4Bb8lUU82EfxI
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(8676002)(16576012)(2616005)(54906003)(110136005)(31696002)(956004)(508600001)(86362001)(186003)(8936002)(5660300002)(38100700002)(83380400001)(53546011)(66946007)(55236004)(66556008)(66476007)(4326008)(6666004)(31686004)(36756003)(6486002)(2906002)(26005)(45980500001)(43740500002)(473944003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RHJNNFo4WGc1cDZxb1kwc3diSFZGK3puLzBHOFhXRTdsU3pRcGd0bC9Hdnlp?=
 =?utf-8?B?UHRubjBLdHdBQzhWV29ZVzVBQ0lPcnIrbVdSc1l6ZG1ENVU2VlpoRzR0ejRI?=
 =?utf-8?B?T1F1eisweEJZb0xQWGhkaTMvb3FZZ2NVaUFsenlJOEd3aGdCbGN6Rlg0cGpU?=
 =?utf-8?B?bEdIanR6UGJSNG5KUU92dlE1UStoUkkwZUxBYlZWZFREM2JJeXVuRXY4Vks1?=
 =?utf-8?B?MmNJMFlZcTdFaGdGM2FtWmJXQVhqYVlGMkNFcEo5QjRtYzQ3cHNIVHkvcEcv?=
 =?utf-8?B?WXhhR0YxZVA4WExweTIrdWJZOUVtbGxtOUFWQjUyTm9KVklPczgxYnZHVjl4?=
 =?utf-8?B?WHpCRjdibkZmSFk1ME5ZVmpDa0p3RUJFWmg4TlI3TlR2dVlEdEFoUDhiajRu?=
 =?utf-8?B?dXZoaXlvT3JER2hzSXFtYzdCRkgvNUREeDduSytjV0s3R3J0cTNIbDBtbUVK?=
 =?utf-8?B?NXMxSTdwb3B1UXVrdDZ6SlJ4QW1BQmJ1K0NQdFlUNy9JdGxaQVhCeEd0MHQy?=
 =?utf-8?B?MGVRSThkcFRPVHBNV0E5bFVhNXdHc1Rxc051ejhHRkhPVU9CdzN0WTAxTmNE?=
 =?utf-8?B?VGtYZ0N3VkxxMFg3UXNweTVpaGtPQml5VmY5eGIrS3BiVXByQXoyR291RG5K?=
 =?utf-8?B?TkpCMURCOXJIME5zY2NMOUVZaFhBWldVYk5SQWUyZFZBV1dxUjlVMlFPWjh4?=
 =?utf-8?B?RGxUMDZyU3UyV0dFbE41eFA5eXVqc1JTTTJoV1ZJL1ZyN29NNTFub0RqVEdI?=
 =?utf-8?B?TlBrTWNpckV5Skd1bTVSQlI5S1B0N1MwaFdnWWUxTzVBOHVWTUtXQy9tT0or?=
 =?utf-8?B?NTJRMHVTem1BTzEwSis2YjlVaXlua0lYVTBIQ3V2dVhSREVGWFlsc3Zac0JO?=
 =?utf-8?B?NzM4cVZOVmZWQWQwakpCajdpQjdvcVVCd3VMbDRjNTkrVzRMamQ2OHE1b0pG?=
 =?utf-8?B?cTI5SWJDUi9mZUM4cnRybzRXSG13UXZVazQvOVAwWTNTbjR2bGVaQ1VMTHNy?=
 =?utf-8?B?YVd5bW5RcU8zQkZzZGIrcmJVODdnUU9CYkFZbk9mbU11NDZzVllwRWNqazBC?=
 =?utf-8?B?MDJzY1o3b1lGYlVPaUZyYVJiMEF5eFdHMkVIM0ZMYVoyZ0FFSzVSZ3NBNzlh?=
 =?utf-8?B?bTlNWkw0S1cyNEt5aSt5M2VUT0YxRS9WbzRvMUhkS08wUTQyb0pnemxieVAy?=
 =?utf-8?B?SDRSN2tWZEZJOFRwdlUxSno5TE5oMW93ak90N3MxbTVzT0UyVmd3eFlTVzZ0?=
 =?utf-8?B?V2prS2IzL3hEZzR4eG9uTCtCdU1FbCtTaFd2VklXUmhOQmpCenZmNFBqVEFV?=
 =?utf-8?B?K0ZpZlR4YllKNGFTamxnVDFrdjFmeWRIWmcva2F4b25pQUx3ODJTUmEwWmFa?=
 =?utf-8?B?eVlzcUVBcTVwWkZhcFVQSWxvTmtvZ1krYU43V0IvaUJxNDRqdVl6aGxpY1g5?=
 =?utf-8?B?ZzJhRFdqbWs0WGFEV3h0VUFCK0ZzNmlHNUMrR3d4ZDY0cHdQWnJ0b3JBMGZv?=
 =?utf-8?B?MU9RL2pDWlZxeHY4YlJBUzBCSHkrTXZhaDFQM21zcEJsWWc4QXJ2bmJOQml5?=
 =?utf-8?B?VEpTTnlHMDY2TkJMUGV0VFFzeGo1dTcrOGFaOTVUZGc4eW5Ma2dJckNORXNh?=
 =?utf-8?B?M3p0VnU0MnM0aUVFNXdIRG1tQjY3Wi9JaEtZd3pIUlJIckFmdnlPMlZwcUto?=
 =?utf-8?B?MnlYZlZZckZKcTBicVpYZjAvNHpxenBsL3M3SStLVndQdDYyQnMxTkkwR29a?=
 =?utf-8?Q?AysXrXjyGHgGHjuRvDP/jlmGwck03WuaY4rnqY+?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 50622a34-c22d-45ee-5aa8-08d97dcd5784
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:31:54.1544
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JmHCa8gyWlV/oh/EEsGpiBk8PCGLXv29C+UMNF6hxj/PuiKNS2wFd/MdxUDYRv6zkkYOAqXmZmpUjlypzJWMqaRnygynr6+I8xVCpkOQfRQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4293
X-OriginatorOrg: citrix.com

On 21/09/2021 08:17, Jan Beulich wrote:
> Leaving shadow setup just to the L1TF tasklet means running Dom0 on a
> minimally acceptable shadow memory pool, rather than what normally
> would be used (also, for example, for PVH). Populate the pool before
> triggering the tasklet, on a best effort basis (again like done for
> PVH).
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Tim Deegan <tim@xen.org>
> ---
> v2: Latch dom0_paging_pages() result.
>
> --- a/xen/arch/x86/mm/shadow/common.c
> +++ b/xen/arch/x86/mm/shadow/common.c
> @@ -1298,7 +1298,7 @@ int shadow_set_allocation(struct domain
>  {
>      struct page_info *sp;
> =20
> -    ASSERT(paging_locked_by_me(d));
> +    ASSERT(paging_locked_by_me(d) || system_state < SYS_STATE_active);
> =20
>      if ( pages > 0 )
>      {
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -21,6 +21,7 @@
>  #include <asm/page.h>
>  #include <asm/pv/mm.h>
>  #include <asm/setup.h>
> +#include <asm/shadow.h>
> =20
>  /* Allow ring-3 access in long mode as guest cannot use ring 1 ... */
>  #define BASE_PROT (_PAGE_PRESENT|_PAGE_RW|_PAGE_ACCESSED|_PAGE_USER)
> @@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
>  #ifdef CONFIG_SHADOW_PAGING
>      if ( opt_dom0_shadow )
>      {
> +        bool preempted;
> +
>          printk("Switching dom0 to using shadow paging\n");
> +
> +        nr_pt_pages =3D dom0_paging_pages(d, nr_pages);
> +
> +        do {
> +            preempted =3D false;
> +            shadow_set_allocation(d, nr_pt_pages, &preempted);
> +            process_pending_softirqs();
> +        } while ( preempted );

This is still broken.

The loop setting the shadow allocation needs to be outside of this
conditional, because it is not related to early activation of the l1tf
tasklet.

~Andrew



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:32:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:32:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192750.343362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2MP-0002Zu-FN; Wed, 22 Sep 2021 13:32:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192750.343362; Wed, 22 Sep 2021 13:32:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2MP-0002Zn-Ab; Wed, 22 Sep 2021 13:32:29 +0000
Received: by outflank-mailman (input) for mailman id 192750;
 Wed, 22 Sep 2021 13:32:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT2MO-0002ZN-KT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:32:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fcdde9c9-ebbf-44ae-8a01-a40a4d0d9069;
 Wed, 22 Sep 2021 13:32:27 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2173.outbound.protection.outlook.com [104.47.17.173])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-f0_X8DqKMJSIQKW5zG7RFw-1; Wed, 22 Sep 2021 15:32:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3776.eurprd04.prod.outlook.com (2603:10a6:803:18::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 13:32:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:32:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0043.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:53::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.17 via Frontend Transport; Wed, 22 Sep 2021 13:32:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcdde9c9-ebbf-44ae-8a01-a40a4d0d9069
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632317546;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lQyZzJYG4n4Q/DInJuPD6p0wKrbfJsMFM9zgITV+VVQ=;
	b=AN4a5sYLvO21iaN1ZSUHyNJ/ldV8HuuSYe2aRE8+uzYgsL5Jucs7YBqKKfC7HR7EciUsup
	P5k+qGjLfH7hco3+VPYyYt0HoKgcu+K0cROIbf0cJ6Krmapyv1+irmM+fd9WRhC7uVa6Mm
	tLINllazkqvXHZO+tdSQQFQnDhX4dWw=
X-MC-Unique: f0_X8DqKMJSIQKW5zG7RFw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RspPyOh12ZW+UJPpN8y2b8h1b+XCxY0tVHTW24WH0CvvdlEAKtBiW+CMYYke5r7pzLS9g0xc4I/7kQq35QHiARKsxW3IoUB/TSDNvffqb6h8S2+DVSieukenFRakmdpkp8gBUNt2iuojHSpFQAOqbxVhclnrjcynP193ci90rEgOcatJMuuJg2g3W2Q4TJGUFLk4A2mY2PxcTMTjRvYs/ud6nU9WGmztqXEsfYiaePKyv5WunhhZFu5hdR55ROssAvr9vaPneMi7WKKE7IdizW1Q0u6D55df+a7Qhbt1fMAfrE/jizPJw+ipDCuJvyG9YN/CTiDnUWjZuX9/ZBaPdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XL+HUY7F10vHh34JRGh2KOMVdV4ZxdX/D4lnoLJjpJ8=;
 b=KGV072eSzBZBsT9+t+CAnQ1Au6J15xSWoXra1CbHcWEj3UOfatft8aPn6LS5ol/YYv1rQQpb/YkAEVpJwFyvO2IO/NZNPFQf7K9Whc2ZTCRO2fqf2kuRDjJMlpwa4u0+rEy5UJmtC8X18tGOBcdyuDLHljXYTsRKM+ZvHZduUT5NohEb3HwoXu6nghmvt9IzNxkmbO6WuPzPQraN3HOsJJE3CUsNzlnAm58KZbStOm511Hvs5rDDpisQg2Iu51coJ/knOToZSD8Hum+X5TfsIu9t1nooHQN7WM8OBAcLe8HuGbqFwMtIFxCuQeajB+fIJXrXioCCX6ehqU/dMGYayg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Dario Faggioli <dfaggioli@suse.com>,
 Meng Xu <mengxu@cis.upenn.edu>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
 <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
 <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
 <69770706-85f6-6509-b5c4-bda61dfdf994@suse.com>
 <84b780c0-7177-d1ba-fabd-9a6258985f9d@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <c6a2b222-3298-e3a4-9842-10ca3fea2de1@suse.com>
Date: Wed, 22 Sep 2021 15:32:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <84b780c0-7177-d1ba-fabd-9a6258985f9d@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P189CA0043.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:53::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a5319605-3fe6-4721-c58a-08d97dcd6858
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3776:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3776709771103A756A59E9D6B3A29@VI1PR0402MB3776.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	e5KzaN86Cy35ssn9N6214jSjA3HYK4NTZnoFA/erZDDXtSHAJFzizQ5VXu/yVMWHFUJcJZlZW02CHqJ8ErXkgl/0InpPwXyjyDD2A+PhFiPq4IK69mCHNNZLSyDzPEe6ZM8DDM4kr8iVfNOo4PTdPL1VZE4l4yNv3/T2qW6luh+bHU2bwt3qnoY6Z+ei1xZa0fbyvgOA/2qwgeSoj1p7f7eGikpl0iykKASDGRS34m3Nyv0Q7eDL6GLK1rM2uZAbJQ/yYNXu+sIBjSYSDwEUAjU22iplSUSy53fmjGvnNuuzlhOJuRf4TM73rzq+2C0L5TVm2OgtPcIrL+6xuC6XX0DaLJ2pahKTrovi+Lzpc5Z+DHHFj0c7d33rHGohDO1ksYkdtjH/q6d5SdVI98Fb3hEfshhnAq9nGzbRG63uF8hRni2z6ELNmjZbgfzRxAh3/HcwuUTxGHtQ8/S0C+Gk9uq0OXD2ePmhiXfEaFAKNwWbjtm7v/gg/F/MOP7G83eua6xFO6YqNv8sNI6WriMmJW7lKl+Hiy386XNpg0LmJ4bvXz9/ZvfcuVcJ052Tjz3TRnHPVWS7ZhUkNb8GdFlG93Q1rrFBw4v8E361IwVaL67Mbg7j1tlANmFjn1KocJs3d0izvOEGKxZEacCZFkPeEz7PEcxwvUqwmuvGdxAA+KBEu3G+PU49OAHwVVweStKdi8zkQKKaFc+V4korN7n8RIB+tfAqFX6Dsx2zUtYpf0Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66476007)(38100700002)(5660300002)(86362001)(4326008)(6486002)(16576012)(53546011)(316002)(54906003)(956004)(66946007)(8676002)(31696002)(36756003)(31686004)(6916009)(26005)(2616005)(186003)(8936002)(508600001)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Rn+eIjnxfa5x3lH+knk/fzZgnWcsZ32sec78zq7bnfre6xVLY4XZP0z3b5tE?=
 =?us-ascii?Q?JIFXRiJwK7i2icYkvjXlrB0vUK5N1yxyAKDWNQVDJT4bOYcG7M6G7zVK0xAc?=
 =?us-ascii?Q?QwKTPwS3RCibk2axaHwhMbKwTzyQFE8qyi2ZA8pYY3n5ghPZWGLluckqlsBm?=
 =?us-ascii?Q?i/gfBvBWZtK4rHojTp0SYujMMD4VghmQ5PsZtaJsxFwoO728MtBFfqGOVxUU?=
 =?us-ascii?Q?bcQtHgNUrx/8JgCZQjWZUihCXE/wMCK97xpKpzuRGezIStKYXaQeN5SKE9GA?=
 =?us-ascii?Q?497KSNZq1oZP1cjkquWpoHgMEY2VpVSGxqP1KpyxVTmR1iJUQDK3fjC65N6N?=
 =?us-ascii?Q?JtLDU32QbjIuEVnl2acwpI/II3uVWFIbuI3QwCfIC5LLbeAKfyxs+ib0dL2C?=
 =?us-ascii?Q?Gz+Te3jCip/zeGLTYR4jo78gUWhDjLIxghunUjP5FDdPv+jsASal/vSUZ4gu?=
 =?us-ascii?Q?BlBsHWo1gjZ7ao3hCgHDNUTyULjQPGiKFpOev5u/KZMWPjY5qb7L8B4HpYP7?=
 =?us-ascii?Q?iJB2U857p8t+zJdAGraJFGpp5H6+8AUNuqSqOXslCrWvDfcN+EqUvgz9lMj0?=
 =?us-ascii?Q?nKEnWqtV+NQyDxPe+HcuGtN6TXrABbd0in6vCQIng1BmH2uBIyFlphDwTt1i?=
 =?us-ascii?Q?zhO3LCOQlInsbBPEnF/3ZvvNFyKldWWWIg7qa76aGAa9RxzW+ATxa5qsqZSr?=
 =?us-ascii?Q?g+okCQ1W5IfDyNfpnO9ZWKqbHuSuvLysow1Y6f1MLe+log74g79n9wuB6Oo9?=
 =?us-ascii?Q?r8lwtpTQFeItGn0pGiw2776kpf6ks3n3ibc00eaeY0A7B5LlhAOH46noVLF2?=
 =?us-ascii?Q?A3//zj9G5jW9WZ+szBPSy9UAl7BSQw/jQTU2+Xzis5cQu/CrxysrYYocaIgR?=
 =?us-ascii?Q?cCeveMZVCKWgxbdf+DtQoWyIZWJb3U8Nk95tpTqN3Uc1Au3M0hOPudLCpXlI?=
 =?us-ascii?Q?DMBn7FGtRLquPgj+WlhIr0pTJsZ7oxy6DVDUdqWPTaSOFX0rQkLoek9EZETz?=
 =?us-ascii?Q?q/cy2/3PnoHl4W8Fge2jxdKwmSwujygmpszPSldIRRYVMhyT0fiiaG64+FHd?=
 =?us-ascii?Q?rAC8ELb6X9jcWIBKSB2veUWExoNqvzbWCtpefwgW4ocxHm0g20PpvFPtC/VU?=
 =?us-ascii?Q?N89VNIQGEJIiw+msERDIvxmD42AJoNqwwlLhNDUVyxDsakXfE38wsRhPEAEY?=
 =?us-ascii?Q?GwfAt810HYwc259nO+27U6BqPMA/ZzvRQYiUnlnRbJcCkpZN4UV4O7Q3wdWv?=
 =?us-ascii?Q?w+8oQocO272571yaITfSgFeo3p1+YQS5NIXuH45DQCWb3WvKV97Lrr1RiY+0?=
 =?us-ascii?Q?0bIMvohn1qyWrDzQ9ejH5UJP?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a5319605-3fe6-4721-c58a-08d97dcd6858
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:32:22.2721
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4zm2Er9lAIC1z+RF2cQ67H8QATlV23ShVasvgxmgJkAYpTyGzC4Fgq9839oy8sBk0hucXOB7/83bSL+nfJ77QQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3776

On 22.09.2021 14:58, Andrew Cooper wrote:
> On 22/09/2021 08:01, Jan Beulich wrote:
>> On 21.09.2021 19:51, Andrew Cooper wrote:
>>> On 21/09/2021 07:53, Jan Beulich wrote:
>>>> On 20.09.2021 19:25, Andrew Cooper wrote:
>>>>> --- a/xen/arch/x86/hvm/hvm.c
>>>>> +++ b/xen/arch/x86/hvm/hvm.c
>>>>> @@ -5063,8 +5063,9 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HAND=
LE_PARAM(void) arg)
>>>>>          if ( copy_from_guest(&tr, arg, 1 ) )
>>>>>              return -EFAULT;
>>>>> =20
>>>>> -        if ( tr.extra_bytes > sizeof(tr.extra)
>>>>> -             || (tr.event & ~((1u<<TRC_SUBCLS_SHIFT)-1)) )
>>>>> +        if ( tr.extra_bytes % sizeof(uint32_t) ||
>>>>> +             tr.extra_bytes > sizeof(tr.extra) ||
>>>>> +             tr.event >> TRC_SUBCLS_SHIFT )
>>>>>              return -EINVAL;
>>>> Despite this being a function that supposedly no-one is to really
>>>> use, you're breaking the interface here when really there would be a
>>>> way to be backwards compatible: Instead of failing, pad the data to
>>>> a 32-bit boundary. As the interface struct is large enough, this
>>>> would look to be as simple as a memset() plus aligning extra_bytes
>>>> upwards. Otherwise the deliberate breaking of potential existing
>>>> callers wants making explicit in the respective paragraph of the
>>>> description.
>>> It is discussed, along with a justification for why an ABI change is fi=
ne.
>> What you say is "This has no business being a hypercall in the first
>> place", yet to me that's not a justification to break an interface.
>=20
> No, but "cannot be used outside of custom debugging" means there are no
> users in practice, and therefore it really doesn't matter.
>=20
>> It is part of the ABI, so disallowing what was previously allowed
>> may break people's (questionable, yes) code.
>>
>>> But I could do
>>>
>>> tr.extra_bytes =3D ROUNDUP(tr.extra_bytes, sizeof(uint32_t));
>>>
>>> if you'd really prefer.
>> I would, indeed, and as said ideally alongside clearing the excess
>> bytes in the buffer.
>=20
> Why?=C2=A0 The entire structure is copied out of guest memory, with a fix=
ed size.
>=20
> It's not Xen's fault/problem if the VM didn't initialise it correctly,
> and an explicit ROUNDUP() here maintains the current behaviour.

What I don't understand is what you derive "didn't initialise it correctly"
from. The public header says nothing. The data field being of type uint8_t[=
]
may very well suggest that any size is fine. Propagating rubbish instead of
predictable values (zeroes) seems wrong to me.

Anyway - I'm not going to insist; I merely think we should be as forgiving
as possible in situations like this.

>>>>> --- a/xen/common/sched/rt.c
>>>>> +++ b/xen/common/sched/rt.c
>>>>> @@ -968,18 +968,20 @@ burn_budget(const struct scheduler *ops, struct=
 rt_unit *svc, s_time_t now)
>>>>>      /* TRACE */
>>>>>      {
>>>>>          struct __packed {
>>>>> -            unsigned unit:16, dom:16;
>>>>> +            uint16_t unit, dom;
>>>>>              uint64_t cur_budget;
>>>>> -            int delta;
>>>>> -            unsigned priority_level;
>>>>> -            bool has_extratime;
>>>>> -        } d;
>>>>> -        d.dom =3D svc->unit->domain->domain_id;
>>>>> -        d.unit =3D svc->unit->unit_id;
>>>>> -        d.cur_budget =3D (uint64_t) svc->cur_budget;
>>>>> -        d.delta =3D delta;
>>>>> -        d.priority_level =3D svc->priority_level;
>>>>> -        d.has_extratime =3D svc->flags & RTDS_extratime;
>>>>> +            uint32_t delta;
>>>> The original field was plain int, and aiui for a valid reason. I
>>>> don't see why you couldn't use int32_t here.
>>> delta can't be negative, because there is a check earlier in the functi=
on.
>> Oh, yes, didn't spot that.
>>
>>> What is a problem is the 63=3D>32 bit truncation, and uint32_t here is
>>> half as bad as int32_t.
>> Agreed. Whether the truncation is an issue in practice is questionable,
>> as I wouldn't expect budget to be consumed in multiple-second individual
>> steps. But I didn't check whether this scheduler might allow a vCPU to
>> run for this long all in one go.
>=20
> I expect it's marginal too.=C2=A0 Honestly, its not a bug I care to fix r=
ight
> about now.=C2=A0 I could leave a /* TODO: truncation? */ in place so whom=
ever
> encounters weird behaviour from this trace record has a bit more help of
> where to look?

Would seem reasonable to me, but really needs to be answered by the
maintainers of this code.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:39:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:39:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192760.343373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2TK-0003hR-6W; Wed, 22 Sep 2021 13:39:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192760.343373; Wed, 22 Sep 2021 13:39:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2TK-0003hK-3E; Wed, 22 Sep 2021 13:39:38 +0000
Received: by outflank-mailman (input) for mailman id 192760;
 Wed, 22 Sep 2021 13:39:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT2TI-0003hE-Fj
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:39:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cb85bac1-332f-4306-baa5-62162d6f2428;
 Wed, 22 Sep 2021 13:39:35 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2051.outbound.protection.outlook.com [104.47.2.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-RRgzrCCYPhSqar_GqGZnBA-1; Wed, 22 Sep 2021 15:39:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 13:39:30 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:39:30 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3PR09CA0001.eurprd09.prod.outlook.com (2603:10a6:102:b7::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 13:39:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb85bac1-332f-4306-baa5-62162d6f2428
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632317974;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4goRYAj1C/OYhVef3+5ksvuHtwgPhezRweIBkwvA5VM=;
	b=S53CO5ZKfpu77Qve0OLCWooVoK0nUE3DpAWUKCdz+iiqVniMqureSS8UUYaMumcffizD3B
	dgggxNy381bpSz5KNYZwYIaMZVCXI1gwNekw8muILGFey5pBeo3dLjqMY8LLyK+tfqeG2z
	cb2AimH1f3oPQSzusLrgf5rp2Ze8p2E=
X-MC-Unique: RRgzrCCYPhSqar_GqGZnBA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SNGmlk27vKUmqC/jVwhRnZmM4TwujWR9bDExLO5GGzxC3LQek3qMd/AxVHYXHMqoKTp6Ea+5tJ8K7V+8Hriv90+2/oxncg3fIEDv574EcCPtON4v5C3dy1RIVNKdIdjWmqc288iuhp8ErMIiEDRUxZ2iLfQFem56d5gr8a7xlNYa0RrIqfEGaJJ/7aVtz3n7dO71MC2K5qjw/hm8SBAoh+vqtory6MBLT+h+VLM7jfsR4N5J4oB90pWvHari7TPtbL5/gVaXKVCFh03V3O17Zo3nI9jqASkM2/Oql/ux3pfm8Y1gwDeozL2jAlKQdH2e/VjyzMnMKRdNftxcTWcm0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4goRYAj1C/OYhVef3+5ksvuHtwgPhezRweIBkwvA5VM=;
 b=RYqeIeuXmYeznoTnWXdqzThGnfzslLoJvnBy4QzpmER/FDroyO+v7vtMpWHDWTeqzVlnlUBAzgPEhSCkrVY9QGgTjtSpdOv9BmUTVE+qKC/6Su5HeLJSMddiP4v7iASmkG/Md5SeNENqtpiJXvkGp/H/UA8EMno+1s3hURhWuv4pK2oWFe3MwzmZZc6NzuWFqpPTz6w7tRujOAMXj6mhO5K7/wQKziF23UkFDbCd/5SdR8BloDQwUmdGBHIidcK8+1Q1+TG4Sndoxl6oQZ5qfovpZ9wPs77ziv3SiKrh+NI4mErkcrQoDO515W/KVKjHZEmckeblpoOIKpqOFOeJYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 2/3] xen/privcmd: fix error handling in mmap-resource
 processing
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>, Juergen Gross <jgross@suse.com>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <aa6d6a67-6889-338a-a910-51e889f792d5@suse.com>
 <c0c84258-c2ee-f58c-ae9a-5f8bdd75f0db@oracle.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1374b8da-1076-63fb-bc54-5be9f1ae94d4@suse.com>
Date: Wed, 22 Sep 2021 15:39:29 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <c0c84258-c2ee-f58c-ae9a-5f8bdd75f0db@oracle.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3PR09CA0001.eurprd09.prod.outlook.com
 (2603:10a6:102:b7::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dc48c9ae-95b2-4e4b-271b-08d97dce6755
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7039BE7D471D883F0B069215B3A29@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+7N89F4Ko2eT3NhJlBGL5y2iaQ2uR/bgjBvyOK918Rd06/66CH9iWz4NAJvKFtcgr3oXPZPC2/YmPl9Y3Gf1v4bXO6JiG9sry7qqfWay+5Q5NHumo5rBzrb9xi6Hlrv951gBT3tm3S/VLE+ztrAn563H16DFubxoV1BdGGVae7qjGgy42vzQdcafoFCk3SXqUQulJ/AQTrJ/SfAjL8aM7FZwiHUHABPWb+i9Sx6iQmCuSUHh1RAnOY55PTn5r1LvjePxIZyxXRDISGtosY8S67Bsxjg7/fDTrcI9l01E1DPIo0Nn3N8F1bbqqsIqT1W6Xm0awTzDPUVah1DRFBVy6M5nY/JeSYWpgmX92ktjkq+3tjWdHPpI8mOBc4C4TgQcqnzGukqeNQly6ruPyJDIqUPuvYH/snQXP4Yf1mEqOLuQQBPgr6jTk80aRRQjyxZ+eJJJxJWryErcxrKN5bv0drh2FvERXt/m9evCcXjKjSi7w3EUrEoUhvfzs99+zZOR5pHR1Mx27XHDLQUs9SkZoc+uttP/yhqI9JpQAryhqjgpy5jaQDka+sNVxfo4u//3A9avFfOMmQLvtJE/yh+0MeVcDiegCyISCa3q+cm02ssVXi6VD+pbOumjvrvShCtLlzZwXqVGFxT2hUnA63EA7iTTY1jB5aS/oUHV1UW/VAd9D6LKxeVXVuZMv7x1qLNeObq3f4opukwCj5qqNvomay74PkhhKqtrh+hR1UF5z0Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(66476007)(66556008)(38100700002)(4326008)(66946007)(83380400001)(31696002)(316002)(5660300002)(54906003)(53546011)(107886003)(31686004)(2906002)(86362001)(16576012)(186003)(6486002)(8676002)(2616005)(956004)(6916009)(8936002)(36756003)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NXpZcmY5dWZRZEVkY3cxaWFlSWxpd2FaNkh0NE52MjdXQzZlaFF5Tmk1cDRM?=
 =?utf-8?B?NDAvUTZ6NGhYZS9YV2lQS2pvTjVGWWJVWFMrb0tjRENQQlh1VUd6YWgrbmwz?=
 =?utf-8?B?c0JZcXg0citQYW1Lc3BwclgxVXZVd1REZWFsVHovK2h3M3ppMi9BR2dVUTZt?=
 =?utf-8?B?YU9RNDVzWktGRFMxcGZMMjZxWVZwdVVId1JCVEw2ZjJNclhDUE5wemlubzEw?=
 =?utf-8?B?RnVZTEZVdTUwMERyQU9mMFl1bG13VjNINCtSUmprVlhJdDNSdVpTVGRtVndw?=
 =?utf-8?B?aU40V3Nrc2traUVEVGR6dENnV085WnBIWVl1aC9nOWlUOU1XR1ZKNXduMFdW?=
 =?utf-8?B?N1ZRRTVadGEyblFEMG5mdTZRYzh0SkdJL3RYSk5KeVNoMXhzUHNnTXFGVTFZ?=
 =?utf-8?B?TFRoZjEzK3ZPTWdMUWFxcmUyTzBITDFTSlp5c2ZTVC9tRVZYMHIvQlgwMkx3?=
 =?utf-8?B?NUtzQTJWMzEyMmtCTUUva1RuYlNXSG45cWszR1V5OEIzQ3prV204ZFhOTEJQ?=
 =?utf-8?B?bWlEMWVNQkVaem84WGtHVEtndjBhWDZPK3VFNXBTc0MzZkdhTnVsaUdFdndS?=
 =?utf-8?B?aUUxUnBXUGRTa1V5M1JnbmZoQUdPLzFIaGpYSjNVSGJpS0pNODRLbjNMUEpk?=
 =?utf-8?B?UDV6elp3WGFKTE52SURVT2xoMGRFWU9PWkp0azRYSDdWUjRuVXlFYUJZTlFs?=
 =?utf-8?B?d0E0RzFjcXUrSDFRckN4SGRlKzlXWkNrZUZtWEJPdlZ6SWxvVzk3OFlhVkVN?=
 =?utf-8?B?SzFLRUJUY28yR2cvdlY0L3lSRElqSWZuMGdqSlBVUTN6MUdtU0NyTmgrSmw5?=
 =?utf-8?B?Ui8zTmhzU3hrcVlxOEw0ZS83SmZMZWw4WGJWRW55Y2tGd0dXWFY5VCtMbmpK?=
 =?utf-8?B?MXVzZ0tZcEZURjZkWFhtOUZYVHJDVEhiWWZNUlpYZE5JRCtabmFaa00yblZ5?=
 =?utf-8?B?V0NVQmdIN3hSUmJUaDRLSC94RTlMcFNxeDVpWFV3RFNtZmp2RXhsVFZYeUJx?=
 =?utf-8?B?akVRSSsrS3BpdWowK2NXeFBlcmFFUmFTS1RlaGM1ekNEZXNsVDJ2ZWxIRnM5?=
 =?utf-8?B?aVNZNWxFbkNhQXpnTXlSSzRjazBkL3NIQVAwUUdkWFFZaUI1bENIb21DTkxX?=
 =?utf-8?B?ZGRaV0hLREpPU0xPYmtiRjBlaEtFTTlPRjJtY0Z1cXFHSEJWcHFLZzZyUmZ2?=
 =?utf-8?B?cTFJQndJbmNzQTlUTmFCQTZYVTlZVnJlYVdXNzRINWZPdjdJSVBlT3lmbXZN?=
 =?utf-8?B?eE42bW1mVzJ1MVJGdkNCSmd2ZFhUUnc5d2NKTHdYeWcwRFNCQ0xVenJaQkIv?=
 =?utf-8?B?OWw1aXhWVUdCSHRXekh4UHJKMENreVRqUlpEd1R3MVdKbGxPM2ZYdXpFaEZK?=
 =?utf-8?B?djJPU2dNZ1AvUDVBVVhWR3I0UTE0NGVvWmpGNDRTaHJpSG9jV1E3UGNmdmdt?=
 =?utf-8?B?clpVTlU4RnFpUC9BVSt3UkhobzFkK01RUFBFOVhNRXFqR01Nc3FoeFNWdkl1?=
 =?utf-8?B?dktGWGpKS01abFN0QXc3ZVZ5NWR5ck8xYm40MEE5ZlR0dmExb3RMVUU2WFRt?=
 =?utf-8?B?WDVOcFB5cDdVdzdSTDN2N2dxV1dqdnlhOGVYazNqR2VwYjkrZmZiZGRzS3BF?=
 =?utf-8?B?ZndEMnVRYUJjT1V4UGQrME4rd2ppZXlvQjk3aVJCVWxiUS9remdRMDJ0V0hM?=
 =?utf-8?B?ejhlZis1NjBUWG5jT1hFMjFhSHp0SXhNT0luSkJHdzNZTXlLc2h2RklIaFdX?=
 =?utf-8?Q?ndu5HqEnxOyzRixjLNhnodr1+MJkYeq+EbQm1aZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dc48c9ae-95b2-4e4b-271b-08d97dce6755
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:39:30.1343
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VVxl4hIccXeUj0uZ5RptOOBqE1jl9EMKz2Ejxf/yZtYltdG5VGzD0kQF4Kdx3Ihn59eiKSVC6LuwRyhuy/korg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 22.09.2021 15:29, Boris Ostrovsky wrote:
> On 9/22/21 6:17 AM, Jan Beulich wrote:
>> @@ -817,7 +818,7 @@ static long privcmd_ioctl_mmap_resource(
>>  			unsigned int i;
>>  
>>  			for (i = 0; i < num; i++) {
>> -				rc = pfns[i];
>> +				rc = errs[i];
>>  				if (rc < 0)
>>  					break;
> 
> 
> Can the assignment be moved inside the 'if' statement?

I wouldn't mind, albeit it's not the purpose of this change. Plus
generally, when I do such elsewhere, I'm frequently told to better
leave things as separate statements. IOW I'm a little surprised by
the request.

> I am also not sure I understand why we need error array at all. Don't we always look at the first error only? In fact, AFAICS this is the only place where we look at the value.

Well, to look at the first error we need to scan the array to find
one. Indeed we bail from here in once we've found a slot which has
failed.

I guess what you're trying to say is that there's room for
improvement. In which case I might agree, but would want to point
out that doing so would mean removing flexibility from the
underlying function(s) (which may or may not be fine depending on
what existing and future requirements there are). And that would
be for another day, if at all.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:46:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:46:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192768.343383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2aF-00059U-1u; Wed, 22 Sep 2021 13:46:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192768.343383; Wed, 22 Sep 2021 13:46:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2aE-00059N-VG; Wed, 22 Sep 2021 13:46:46 +0000
Received: by outflank-mailman (input) for mailman id 192768;
 Wed, 22 Sep 2021 13:46:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mT2aD-00059H-1v
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:46:45 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mT2aB-0008Ru-Ga; Wed, 22 Sep 2021 13:46:43 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mT2aA-0001aQ-4w; Wed, 22 Sep 2021 13:46:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=jp5Y4ELZ4RSHL0wMVHLANLBmsmpOcx5IG5cte0B3Fl4=; b=k+3+rMNEmECtDvk1y6WlgbkaTG
	1HCWhjH2DYEtx71u39CyRw1R0NQWEGcsa8rlQ8L+5WyPgY3hsuG9TI9Q57/gdtQdznfSINJ0mkH+s
	Z0SSbQd0/oT8dnRFg66ILDJZBd7gT4sbvwqeJWdlncWmVTP0xq5jHnlReq/bom+ccFbM=;
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
To: Juergen Gross <jgross@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, raphning@amazon.co.uk,
 "Doebel, Bjoern" <doebel@amazon.de>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
 <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
 <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <56bb55e3-ca7a-12e6-5848-84756b431846@xen.org>
Date: Wed, 22 Sep 2021 18:46:25 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

(+ Some AWS folks)

Hi Juergen,

On 22/09/2021 17:34, Juergen Gross wrote:
> On 22.09.21 12:23, Julien Grall wrote:
>> Hi Roger,
>>
>> On 22/09/2021 14:58, Roger Pau Monné wrote:
>>> On Wed, Sep 22, 2021 at 02:07:44PM +0500, Julien Grall wrote:
>>>> Hi Roger,
>>>>
>>>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>>>> Failure to map the shared ring and thus establish a xenstore
>>>>> connection with a domain shouldn't prevent the "@introduceDomain"
>>>>> watch from firing, likewise with "@releaseDomain".
>>>>>
>>>>> In order to handle such events properly xenstored should keep track of
>>>>> the domains even if the shared communication ring cannot be mapped.
>>>>> This was partially the case due to the restore mode, which needs to
>>>>> handle domains that have been destroyed between the save and restore
>>>>> period. This patch extends on the previous limited support of
>>>>> temporary adding a domain without a valid interface ring, and modifies
>>>>> check_domains to keep domains without an interface on the list.
>>>>>
>>>>> Handling domains without a valid shared ring is required in order to
>>>>> support domain without a grant table, since the lack of grant table
>>>>> will prevent the mapping of the xenstore ring grant reference.
>>>>>
>>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>>> ---
>>>>> oxenstored will need a similar treatment once grant mapping is used
>>>>> there. For the time being it still works correctly because it uses
>>>>> foreign memory to map the shared ring, and that will work in the
>>>>> absence of grant tables on the domain.
>>>>> ---
>>>>> Changes since v1:
>>>>>    - New in this version.
>>>>> ---
>>>>>    tools/xenstore/xenstored_domain.c | 30 
>>>>> ++++++++++++++++++------------
>>>>>    1 file changed, 18 insertions(+), 12 deletions(-)
>>>>>
>>>>> diff --git a/tools/xenstore/xenstored_domain.c 
>>>>> b/tools/xenstore/xenstored_domain.c
>>>>> index 9fb78d5772..150c6f082e 100644
>>>>> --- a/tools/xenstore/xenstored_domain.c
>>>>> +++ b/tools/xenstore/xenstored_domain.c
>>>>> @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
>>>>>        struct xenstore_domain_interface *intf = 
>>>>> conn->domain->interface;
>>>>>        XENSTORE_RING_IDX cons, prod;
>>>>> +    if (!intf) {
>>>>> +        errno = ENODEV;
>>>>> +        return -1;
>>>>> +    }
>>>>> +
>>>>>        /* Must read indexes once, and before anything else, and 
>>>>> verified. */
>>>>>        cons = intf->rsp_cons;
>>>>>        prod = intf->rsp_prod;
>>>>> @@ -149,6 +154,11 @@ static int readchn(struct connection *conn, 
>>>>> void *data, unsigned int len)
>>>>>        struct xenstore_domain_interface *intf = 
>>>>> conn->domain->interface;
>>>>>        XENSTORE_RING_IDX cons, prod;
>>>>> +    if (!intf) {
>>>>> +        errno = ENODEV;
>>>>> +        return -1;
>>>>> +    }
>>>>> +
>>>>>        /* Must read indexes once, and before anything else, and 
>>>>> verified. */
>>>>>        cons = intf->req_cons;
>>>>>        prod = intf->req_prod;
>>>>> @@ -176,6 +186,9 @@ static bool domain_can_write(struct connection 
>>>>> *conn)
>>>>>    {
>>>>>        struct xenstore_domain_interface *intf = 
>>>>> conn->domain->interface;
>>>>> +    if (!intf)
>>>>> +        return false;
>>>>> +
>>>>
>>>> Rather than adding an extra check, how about taking advantage of 
>>>> is_ignore?
>>>
>>> Right, I just need to change the order in conn_can_read and
>>> conn_can_write so that the is_ignored check is performed before the
>>> can_{read,write} handler is called.
>>>
>>>>>        return ((intf->rsp_prod - intf->rsp_cons) != 
>>>>> XENSTORE_RING_SIZE);
>>>>>    }
>>>>> @@ -183,7 +196,8 @@ static bool domain_can_read(struct connection 
>>>>> *conn)
>>>>>    {
>>>>>        struct xenstore_domain_interface *intf = 
>>>>> conn->domain->interface;
>>>>> -    if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
>>>>> +    if ((domain_is_unprivileged(conn) && conn->domain->wrl_credit 
>>>>> < 0) ||
>>>>> +        !intf)
>>>>>            return false;
>>>>>        return (intf->req_cons != intf->req_prod);
>>>>> @@ -268,14 +282,7 @@ void check_domains(void)
>>>>>                    domain->shutdown = true;
>>>>>                    notify = 1;
>>>>>                }
>>>>> -            /*
>>>>> -             * On Restore, we may have been unable to remap the
>>>>> -             * interface and the port. As we don't know whether
>>>>> -             * this was because of a dying domain, we need to
>>>>> -             * check if the interface and port are still valid.
>>>>> -             */
>>>>> -            if (!dominfo.dying && domain->port &&
>>>>> -                domain->interface)
>>>>> +            if (!dominfo.dying)
>>>>>                    continue;
>>>>
>>>> This is mostly a revert on "tools/xenstore: handle dying domains in 
>>>> live
>>>> update". However, IIRC, this check was necessary to release the 
>>>> connection
>>>> if the domain has died in the middle of Live-Update.
>>>
>>> But if the domain has died in the middle of live update
>>> get_domain_info will return false, and thus the code won't get here.
>>
>> Hmmm... I think I am mixing up a few things... I went through the 
>> original discussion (it was on the security ML) to find out why I 
>> wrote the patch like that. When going through the archives, I noticed 
>> that I provided a different version of this patch (see [1]) because 
>> there was some issue with the check here (I wrote that it would lead 
>> to zombie domain, but don't have the rationale :().
>>
>> Juergen, I don't seem to find the reason why the patch was not 
>> replaced as we discussed on the security ML. Do you remember why?
> 
> Sorry, no, I don't.
> 
> You did send the new version for V6 of the LU series, but it seems at
> least in V9 you commented on the patch requesting that a comment just
> in the section being different between the two variants to be removed.
> 
> So either we both overlooked the new variant not having gone in, or we
> agreed to use the old version (e.g. in a security meeting). In my IRC
> logs I couldn't find anything either (the only mentioning of that patch
> was before V6 of the series was sent, and that was before you sending
> the new one as a reply to V6).
> 
>> Assuming this was a mistake, could someone take care of sending an 
>> update? If not, I could do it when I am back in October.
>>
>> For the archives, the issues would appear when shutting down a domain 
>> during Live-Update.
> 
> Hmm, IIRC you did quite some extensive testing of LU and didn't find
> any problem in the final version.

I did extensive testing with early series but I can't remember whether I 
tested the shutdown reproducer with the latest series.

> 
> Are you sure there really is a problem?

I thought a bit more and looked at the code (I don't have access to a 
test machine at the moment). I think there is indeed a problem.

Some watchers of @releaseDomain (such as xenconsoled) will only remove a 
domain from their internal state when the domain is actually dead.

This is based on dominfo.dying which is only set when all the resources 
are relinquished and waiting for the other domains to release any 
resources for that domain.

The problem is Xenstore may fail to map the interface or the event 
channel long before the domain is actually dead. With the current check, 
we would free the allocated structure and therefore send @releaseDomain 
too early. So daemon like xenconsoled, would never cleanup for the 
domain and leave a zombie domain. Well... until the next @releaseDomain 
(or @introduceDomain for Xenconsoled) AFAICT.

The revised patch is meant to solve it by just ignoring the connection. 
With that approach, we would correctly notify watches when the domain is 
dead.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:50:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:50:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192775.343395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2dv-0006i2-Hg; Wed, 22 Sep 2021 13:50:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192775.343395; Wed, 22 Sep 2021 13:50:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2dv-0006hv-Ek; Wed, 22 Sep 2021 13:50:35 +0000
Received: by outflank-mailman (input) for mailman id 192775;
 Wed, 22 Sep 2021 13:50:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT2dt-0006ho-NL
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:50:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0d7574ee-1bac-11ec-b98e-12813bfff9fa;
 Wed, 22 Sep 2021 13:50:32 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2052.outbound.protection.outlook.com [104.47.0.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-BxfABUEYMza0WOxIjD2uVA-1;
 Wed, 22 Sep 2021 15:50:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Wed, 22 Sep
 2021 13:50:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:50:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0002.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 13:50:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d7574ee-1bac-11ec-b98e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632318631;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VTFda4+SGUG0aGYL1bV9SUmIq1XXpALnAtoQXqLCOqE=;
	b=IQ0n3/XIbChgz0ndLQ3XeSRpk9UjuYCS+ZSx7tcWASTvfrEGwjeGvA3i9qZ6DnC14mM1LW
	V8pi28ftuxaRVabjIQaajkQlUsyHvSk/L6iBV11e/CP3usJV6HEQvvtMDt/O9NBDrinOVJ
	U9hd9DPiTfRVr00f9uW3iqf84zJQ+4I=
X-MC-Unique: BxfABUEYMza0WOxIjD2uVA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YFPeItqSdl0dF9QsH3ibQBSM7aSbbLMB+WvvT33ExnKbvtgDXz0FvKIhv1ENTtK1JEKZS/8MF2lNzViV+CRlbGh3WibonW0pt7l9f+sCHBPSow0L5q5xPJ5JDDHcE0/uZ5qHa+fe41u+Ayr1fw1v0pBQWbdwLOQ4PxvLahCBZ78uTtngNujWUMOKIt4OF7yZr3spYP3jNYT6xViH3s2Ih2nkkasbZLYg8YvY+Nk+aTXhRI7bXCWLEwvHHjldeMtDC8oAPYJKDYHg/eiieT5D21ykKvVtGzcpyt068dym0Z/4s9H59eLgppZoN1KE8zGOEmu9b+RB8SIyKT86hfuyrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VTFda4+SGUG0aGYL1bV9SUmIq1XXpALnAtoQXqLCOqE=;
 b=BweWMUnheQ2VbrrqSClhnE85rKNnDpI+tQggz1P5eteqv6R0GKYYt+Q/0nUxcJODGE9aW+m80Tg3kVFW1AO4Vl4o1u/ba2uJk28GriBfcrP84DskCZGewSRiBArCN8ZVEPxKsv9qdeLRo45FXSrtXW5pAAw06FK4ilzdXKvcCo6f3AEWJ3Wwy7C88VcVpBMvgkiaH3O8B9KfpvHX6dlScH0gQiOjmaNOSvxZex+UZjDqHc9fXTiLQ9cRhMKJmuqRF3VxQG2nkVGW68wY5eSyyzf+XsGaA5N0jTtJQgYjODjqP6N3qQRW+hecbjLm1arQRTLaa0Eryg2sVZRYGUftUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Tim Deegan <tim@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
 <b2a2fd30-225b-2b79-f160-8ed2e4ad439f@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8c1ab98a-8fde-3d9e-5250-c5a40af70453@suse.com>
Date: Wed, 22 Sep 2021 15:50:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b2a2fd30-225b-2b79-f160-8ed2e4ad439f@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0002.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d9df45b7-51ec-4e80-e5ee-08d97dcfef04
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260645FB577051F955DB6F3AB3A29@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1850;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ts46yRoKb+jRDlZqR3liPXzJoeT3R+w4CwFHZiF+2CD3HOpJiGJR5TAT7xzhUHyBO00tQGBMiuPb1khWb2tYFIoqvdYW21J5qRikDUgy2id6F45w8vZaVKJZp4xHZtdXq1Jasd4cvwovQSchCqP4+0cvKfugalLy9/7RKITlUJg/s/gejL2DOPF1vBWP4C+Gz6TFaecTW3xiMFf/mSBTJ/i4BSltODuAs7fD2QTHrFw2wKY8iatD7oDIg+TOVEH23wv2iY2zWXkROT8qyMa4ZCPr1pRu7H2G97A07RLmGugIWFb6m4dNzgJ3fvQbJ2q++b+65MtdFCRflTownDOTAPO9pcJXmFXvesk84XoNkOWNAfZUtg9gI4gMX3/7A56/Hplv+9xv++IGTVhk1QD+EXX3JqYuMOOyNy37akmGzjGq1jGfJHI9T01AFzdEISkIfp/zx589XruYf2lC86ZF3LSAdDHCBq3uLamgjGSeI+BOaF6+32CNT0EQlfB9hW86vwP80ibmZv7miOlYjIAAs8fzEeEQRLmgHCzuCA/66/ijeYXzuaHG8H17R6iSae6Tttq0NcFIGAkrztCT074yhB8tjUd/yUcud5Y/g27V/mVVom0UmabLkAbtBIPGxZtLKXj0SYESpdb6inS1/FXuJcZHH1sslDBzDSm6MqD/qRHQjvIrrNK6hDteL0kJT9NnGyrBDX4UTlOzLA7TTLACLBm1xihqqrEI0OJXYuyYpPmxEmHlgHwqlBLAeSlrhH7X
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(31696002)(83380400001)(4326008)(38100700002)(508600001)(66946007)(316002)(2616005)(66476007)(66556008)(6486002)(6916009)(956004)(53546011)(8676002)(186003)(54906003)(86362001)(5660300002)(31686004)(16576012)(26005)(8936002)(2906002)(43740500002)(45980500001)(473944003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R0NMVEZXbW52aEs3ZXBCY1lJcEQrQXJ0UGNub1BSTFV4NktkMHRJMnNXRDhm?=
 =?utf-8?B?Z3g1ZTlrcGhIcEFZdzFDQXovVUl0NUtUMHRJR0J0NEUxS2tTeHVkaVVya2tP?=
 =?utf-8?B?bk5LRzF0Y1pOSlFtMXZxZWpybmg3TDUwRzcydkhzSitCKzl6K2tVYnVWNlJW?=
 =?utf-8?B?Mk41d3VLSnRHazc1OVVjdDZBQjUzOVZFMVMrWmNIeXVhSlpGWTVYZjQrVGpn?=
 =?utf-8?B?NWNydVNqcXhoTWV5TjBwMHRkNmRlQ0labWoyemo0NzRPWXdxaDM0Y1lXWnlC?=
 =?utf-8?B?Y3NJN3dFNmluTWNxUHJ4dDA1ZmpqTmZVNS95QkZkZllYMXFhMmFJL2VUWlhR?=
 =?utf-8?B?TXhqL0xNYTAzUjZnSnBCRHJnK1c5ZGt0MGRKc3lGR3NJcXlsc3UzS0NSRnMz?=
 =?utf-8?B?Q3NyT2xlYmY2TFhIVTJtSVNHVm43QmNZUDd6WVFubEFNbUNGNDFkbUFXZHIy?=
 =?utf-8?B?Vy9Zb0M3QVVHYThLWXJrK2k4NzljcmxRelJHajczVFl2bGhhWVNjOG1ZUzdB?=
 =?utf-8?B?K3NjL2xVdW5qQWN0d21TU1BlWEI2TFZKdzcvUE9saHl5cGtjM29GNy9DRGYv?=
 =?utf-8?B?bEtqR1FEeFN1bVN2VGYyVmttK01IWm05QS9YcUhFMi96Tm9KNDcydEtXRmhu?=
 =?utf-8?B?RFhuQ3ozdFdySnhjUEw5YlYxdEFNWmFQd2pKMGtFcTI5THhLVS9NVGxvNkFa?=
 =?utf-8?B?dHZpdjFrdU9NRnB4MGxpL2VsMXMzUXZwZVpyVDNWK3d0VkUvVXdZZVVzSHhV?=
 =?utf-8?B?Y1Z5Z0k4eFo4MkRDZEN4WFlGOTA4cWlBOTRCM2xxeUR4N3FBa1hpQWpiVG5y?=
 =?utf-8?B?UWpwZzQvTGNPaGF4VGQzWDhwa2I4Y25PVzlhZm9adm9LWUtMaDZqVURLTm1z?=
 =?utf-8?B?VVl3RzAxbnZObDYrakxiMVN5bTRMeGtobVBYMElWRFI1eWVoUy90eklLNGcw?=
 =?utf-8?B?YlhINVJMeUtoWmFEM2svM1lOTnFoNUFjTFZVa2NYcHBVY1J0UXhwNTRlUzA0?=
 =?utf-8?B?a0I0bTZ3V0NYS3RjaUVoU2ZqbmVHTklqRnVrWkUzUUhtNFJaNmhFdWVPTm52?=
 =?utf-8?B?RVZWbzdmS3VNVEtpZkErZzF4U01lbm91NnU1YjlQYllvek1jY0g0RnlYeVlB?=
 =?utf-8?B?MDVVVFpNZVFkNWIvQ0ZlOWNjcHR0UTl0SE8veEIwdE9DUVhWU1BBd1RxRUJ3?=
 =?utf-8?B?RVBUL3VRaWRJOWRITS9KNGN4WXNvZ05wQ2lqUnFVck9KbWQ2b0NXcEdTNG5k?=
 =?utf-8?B?bEZndEhEVnV3dGRRTFJOaWViT0xLTDdlb3RnRkR0STJjNE0rRXpNRkthKy9M?=
 =?utf-8?B?Ym9pbERpUEJ4a3ZyalA4Wi8rYjdRYXFmaE9vWFFrUmtPbXRpdGx2TkIvNmRs?=
 =?utf-8?B?WUY0ZDBiT2RQUVVjZjNTY1ZwZDFnNmV1bXFXY29yWlBBVkVDS3lTallyYmRx?=
 =?utf-8?B?RWx6Y3ljZ2ZVQmJla0ZvME1neXg2R3RhYkxTK2xZWm1vd21nRUpqSWU3bzZN?=
 =?utf-8?B?NCtrVmN4a0xJS1RvV1gwOHRxQnhpQ3AyZTFYU2haMGZhMWRUcktUSG5uWk9q?=
 =?utf-8?B?emJlOFhNdnBDYllKR3R3U1cwdVVrUXIvVi8rOTQ5Umd1ZXlkZFEvRE8zQ0dY?=
 =?utf-8?B?V2xMNlRTeVFNcTV5ODE0UXdsR0Y2bWtYNWoxWUg1WGZzaVpHVXVKK3J2cXp2?=
 =?utf-8?B?V2JPazk0MHFJWEVuWUtQZHRMMm43NVM5TFlxazdLSzhMZGxranBkbjF4T2FP?=
 =?utf-8?Q?Z/a2ZvdwAfMFER38xD9HuO0m6rPNTDfqjZF6mdC?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d9df45b7-51ec-4e80-e5ee-08d97dcfef04
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:50:27.2103
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SlhnB/Xw9nxTH8sCUqwL/Ot3b5msPCEQ1pxzhOcllPbD0lLkti+cbABcJ19y/jQahuCYutIXzYpfmTRYzI91Gg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

On 22.09.2021 15:31, Andrew Cooper wrote:
> On 21/09/2021 08:17, Jan Beulich wrote:
>> @@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
>>  #ifdef CONFIG_SHADOW_PAGING
>>      if ( opt_dom0_shadow )
>>      {
>> +        bool preempted;
>> +
>>          printk("Switching dom0 to using shadow paging\n");
>> +
>> +        nr_pt_pages = dom0_paging_pages(d, nr_pages);
>> +
>> +        do {
>> +            preempted = false;
>> +            shadow_set_allocation(d, nr_pt_pages, &preempted);
>> +            process_pending_softirqs();
>> +        } while ( preempted );
> 
> This is still broken.
> 
> The loop setting the shadow allocation needs to be outside of this
> conditional, because it is not related to early activation of the l1tf
> tasklet.

Well, I'm not sure what to say. On v1 you already said so. But then you
didn't care to reply to me responding:

"Are you suggesting to set up a (perhaps large) shadow pool just in
 case we need to enable shadow mode on Dom0? And all of this memory
 to then remain unused in the majority of cases?

 Plus even if so, I'd view this as a 2nd, independent step, largely
 orthogonal to the handling of "dom0=shadow". If somebody really
 wanted that, I think this should be driven by an explicit setting
 of the shadow pool size, indicating the admin is willing to waste
 the memory.

 I'm further puzzled by "not to retain upstream's security
 vulnerability" - are you saying upstream is vulnerable in some way,
 while perhaps you (XenServer) are not? In general I don't think I
 view downstream decisions as a driving factor for what upstream
 does, when the result is deliberately different behavior from
 upstream."

Which has left me with no justification to make the change you're
requesting. I've now got an ack by Tim and an R-b by Roger. I also
view the change as is being an improvement on its own (i.e. I
question you saying "This is still broken."), even if (later) we
were to follow what you request. For this reason I'll give it a day
or two for you to reply, but otherwise I'll commit the patch as is,
leaving further adjustments for a future change (by you, me, or
anyone else).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:53:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:53:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192781.343406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2gl-0007KR-W2; Wed, 22 Sep 2021 13:53:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192781.343406; Wed, 22 Sep 2021 13:53:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2gl-0007KK-Sh; Wed, 22 Sep 2021 13:53:31 +0000
Received: by outflank-mailman (input) for mailman id 192781;
 Wed, 22 Sep 2021 13:53:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT2gl-0007KA-Dn; Wed, 22 Sep 2021 13:53:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT2gl-00009U-5A; Wed, 22 Sep 2021 13:53:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT2gk-0001wg-Ny; Wed, 22 Sep 2021 13:53:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mT2gk-0007Ne-NQ; Wed, 22 Sep 2021 13:53:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pE+IfB/AYdPJdMwj43vUCUK/6dYgXU9riZUHryPgLLM=; b=MjMhsorEcQJ+p/s/+shdbfyL6S
	Ix3GVtApF9zDTwP+Ajas1rJ5CSI52iin1/tuS3FyH9PqalhnpFGRjMS7qGwLDKgVrqKyFwlDXd9F7
	5AmSEs/o/r0fqOS5TMiGInERxOELgY1mdxlUxzppSxN4txB6D4MUUMJ3vbQMawo7XxRo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165147-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165147: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e467d390eaf6736377766adae0166711866fac3
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 13:53:30 +0000

flight 165147 xen-unstable real [real]
flight 165154 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165147/
http://logs.test-lab.xenproject.org/osstest/logs/165154/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine      4 memdisk-try-append  fail pass in 165154-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e467d390eaf6736377766adae0166711866fac3
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   11 days
Failing since        164951  2021-09-12 00:14:36 Z   10 days   21 attempts
Testing same since   165147  2021-09-22 03:25:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 790 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 13:57:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 13:57:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192790.343419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2kG-00085q-Lt; Wed, 22 Sep 2021 13:57:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192790.343419; Wed, 22 Sep 2021 13:57:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2kG-00085j-I4; Wed, 22 Sep 2021 13:57:08 +0000
Received: by outflank-mailman (input) for mailman id 192790;
 Wed, 22 Sep 2021 13:57:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mT2kF-00085d-I2
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 13:57:07 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2c9e79f4-6cfb-46c2-8240-fef039a99a7e;
 Wed, 22 Sep 2021 13:57:05 +0000 (UTC)
Received: from pps.filterd (m0246630.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MDapWu010075; 
 Wed, 22 Sep 2021 13:57:03 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4pbu5y-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 13:57:01 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18MDtaqM126907;
 Wed, 22 Sep 2021 13:56:59 GMT
Received: from nam04-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam08lp2046.outbound.protection.outlook.com [104.47.73.46])
 by userp3030.oracle.com with ESMTP id 3b7q5n07gb-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 13:56:59 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4109.namprd10.prod.outlook.com (2603:10b6:208:11c::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 13:56:57 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 13:56:57 +0000
Received: from [10.74.114.106] (138.3.201.42) by
 SJ0PR05CA0195.namprd05.prod.outlook.com (2603:10b6:a03:330::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Wed, 22 Sep 2021 13:56:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c9e79f4-6cfb-46c2-8240-fef039a99a7e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=zhxTcPI7rWwl4C7tspU9gg/f3idfnYkP5xBPg20EQqo=;
 b=RL3VKgJVtoWw8nZl1vhUOzJESK+psG7xTRd4ABKxXnUoCp6vKAEOH78PvMKzeCK7iv8M
 3vs9opwP9OZ9Fl3k7AMqk1f8/12O1ut/wYY/gqQzXhBvEUsHkbFo2guY1CVJIRDcPGmp
 NCfOO/OOmWQNS7Y8o/lGKyLdAhCuw1hhq3R+sBqYkQ6wevAJayNTQ0bwXIQzUkCSnGr6
 CWHDym0mupfhy6TqSyhNF5ZgVnYD6iLU53SnC90JSI367QlBF9kdHaQG0TOOoJ1uXbuI
 08kYmggiU9XygLUYnnYbKLXPe8A4XuVDDBztnixcjmrwIRbN0px3cfN7Zq5eI9p6zxiU BQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hRKiGgoqEdwvGVd2BNVNylRTzfy7dPF14q9Vcak/EKcjiBkfoGpyEIMuo6Svve1q4r+87us9bq9jXvTTtJ7doDf5Ir95XxMNA9OFvCjNMqaqYo8Hi10UebS5bKYcW6ahcBoyRcwog5mrLSg9i+yJEcdz/6KGv3DxfE9Vf42/aeh+epvz+0qnn+c3gpoAfA21yHt7tnRGY9/krqPPA8pYAyHTAvhzCQyfMiScaiS0hAxxf8x+ZFKVD0CAH8rVRqjQI3t/3X9pRLpc/oVi39HAB3ZaxCV1IPkoCSjbE43pjBq7TzwrI6K4QhxRPR3nOGDAnavwXN4KbPrfrQaJUK5LoQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zhxTcPI7rWwl4C7tspU9gg/f3idfnYkP5xBPg20EQqo=;
 b=J3Y8eaDvOs18hD3QkcM+iQ2W2a0nEAwMM+KhYFZXAzu/UWhx2xWDDsiklZZVDZVzhU3mZpBGDxOvx0WQvs32dC0Q6mYrcDmzpHMCJr0Vs9weMDyXejJ3AcBJ1DgdomUQ0+SuD7MTmrpMOqVBkPcqd+0uCJjuqKndmag1aGUoEIwgetVigej8aRlm3s4aAiAaUSomelX0EkUE6oxbI/sMYsn840XVjjgZzRVlbGEIkfaPdDaroFoxAfxQnXiykWSS+P6WT8t7r2YhGScc3aK7qhRoMzwdyR51+pHHYjytEMNM3qScbSt6O+Ybz8Gb/N9g6kTPSOireds93i3IjED9QA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zhxTcPI7rWwl4C7tspU9gg/f3idfnYkP5xBPg20EQqo=;
 b=V0GXo6/VCa05/s9ozdFvcund1BXpHgjP5rc8aea29Ld4gIaz2re4qLZNS1ScSGqsUJK2FgrweX7zEfR1pDCtZ017xLieve2ehdOprnWM5Rii77PntgWPo5+SDqIpmMG06YDfSBLJal3RGsOLwj9tgE90xbhmIolIMLuyjm/6ld4=
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH 2/3] xen/privcmd: fix error handling in mmap-resource
 processing
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        lkml <linux-kernel@vger.kernel.org>, Juergen Gross <jgross@suse.com>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <aa6d6a67-6889-338a-a910-51e889f792d5@suse.com>
 <c0c84258-c2ee-f58c-ae9a-5f8bdd75f0db@oracle.com>
 <1374b8da-1076-63fb-bc54-5be9f1ae94d4@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <ccf710f1-3c2f-880c-fa89-64fabe852527@oracle.com>
Date: Wed, 22 Sep 2021 09:56:53 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <1374b8da-1076-63fb-bc54-5be9f1ae94d4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SJ0PR05CA0195.namprd05.prod.outlook.com
 (2603:10b6:a03:330::20) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ab6b2368-65e3-4b2c-2fcc-08d97dd0d7cb
X-MS-TrafficTypeDiagnostic: MN2PR10MB4109:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB410992A8D36074047B80AD418AA29@MN2PR10MB4109.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	yzkc66fbwES21RmkZ3XhFphm7sk61c36ArIFpns4JNm7Rr/K4jtrA66CvXl4tXsuGYPE+41bWmouQWgj8E52gpL9WCyj7EhOBPqSJJDCt4neL8xuzRmmPQYtIfo6TIRPN/Y9MID5ycFJqC5qC3NDVmHSdZHyoKyOsi6h5loojZ0NKThK0JToW1KriAZxjAvWcQ7GX5WC+1xnI40Za7Sl/XwPHNfsROKCqZAcnVqiuuoVgnGlqHjlZlKp09d9qPz4gS8Lk1q8n5UkqgchbkWjfV5VgDfxrRNCCWqdt3oBP7mr7JJ+BHL0UAqs0O1wtR/6xguTXGGemry4smiLv1ZDGFYChhBvreIM8f5+/ikrELz5+PkvZBfLZ7PJfuptfhAKyU1+3QNtFG8ak00p+Gp17sKaokffPK7wr0kBjdy1jAJJK4vVQt4kFV30kL2hlROAShyu+SfkFvlUIVmcjBKRtF3mCwZwegUE0o7yYTG3vwt8livuhjiwjmIWxyrhDyznUttNF4WEiu0qELSo3Z5FlJwlhBHBPfpzsJ5ETF9Ar+x6BMQuXgmR88lvcPunoPsG+SYvt/hMPhAfksEWZyizIXRl1+9q/k9HiNPvdu9E/135bT1qmXNLmfYXZ1lj3TEOmhv9Sfe0DYPINl2iBV1GdGc/B5bhg/9iaWS/5cTrE45dEnSIcawXupMeGUrGTKwg54wO5AbC8eHEeHuB7t8aW4Zy+Z4UOcLSrIP9zibU66PTKuhBlnyvYuTzAEeeiE1q
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(66946007)(31696002)(83380400001)(66476007)(8676002)(508600001)(86362001)(956004)(66556008)(31686004)(2906002)(6916009)(16576012)(4326008)(6666004)(316002)(186003)(36756003)(8936002)(54906003)(26005)(6486002)(53546011)(44832011)(38100700002)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?d1VOT01ySklBWTlKbDZKaTZsTkhObjFTakVmV05YRTlWY2FqSGw1ZTRyeTZ4?=
 =?utf-8?B?OFgzeEZ4azZUM0kzV2lUSFREVGJ3clFMN3dtTmZjWTBvNjU2djgrTjlKREc4?=
 =?utf-8?B?cUE4eXR2SlN6WUI4L2lBN2MyenJvNUVtV1RSdmRudk5seUoweTdPcXk1TjFj?=
 =?utf-8?B?c0Y4aHV1QWFqYVJqN3ZwR3A4dTUzWHpycVY2cDlUSkpJY2U1amt6TjVjSnNP?=
 =?utf-8?B?ckZJSDliQVFlTlliaXdtU2V5WExvOXpPK0E5Qkd5aDFTN1hKZEVrYlNSNWJn?=
 =?utf-8?B?eXNPeEtQWXJFdUpMbm5GL2ZkMzNVS2xVV3NKUXJWanowaEt0emJyczJwZVVw?=
 =?utf-8?B?L1duTDI5c0lmMysyUnlVYkh3VlYxYTFUNWtWam9LcnRjaTh4Nk4vekQ4ZHZv?=
 =?utf-8?B?c2ZKZms1NFB0L0w2WGRQeUcrSCtvSEoyQzZmd1hic091ZWpUMmg5S0RCaTNQ?=
 =?utf-8?B?bHJmbm00NWFSN1RVWFpBeFRIRlZPOFB5bk1kTWUxYkc5V1JaV3A1b29zOFJw?=
 =?utf-8?B?Y3lzNGNHZ1NrcTJNSCtidTdJSWc4Y3B1dUljM3FPYzgyZXhpdFZ1TTgvM2kv?=
 =?utf-8?B?Sk04aXd1SVA3MTRXTUdlQ1dvK0s1ejlyTEdMaE9EUG5IaU9ienVIakwxUW4w?=
 =?utf-8?B?NWo3Q1NLaVJTK2d1aVJGcjVFVEZkbDhMUW9HM0ZkbHY3ZVFqZVlaQ05zNUJ4?=
 =?utf-8?B?eDRQSCtNUVVDc3ZzcEFxdWFjc08xRm81RjRHNVlsaTNZVDN6eGt5OUc0OGc3?=
 =?utf-8?B?UlRnNFdvYzJHU0RXUHJUREVjV3lIUTN3QkJBNkxzRzB4OXVpZzkxTGdzL3pv?=
 =?utf-8?B?TzZuODdTVVdjaG1HUmNJNWd6N2hUazdjRzBTSUJlamFodlE3bFFFcmlCZFR6?=
 =?utf-8?B?ak5VNTQ0N01OcWkxb0xLNDBkbkpLeVdwREU1ZUNzTlV4MWxibExneWVrNVh0?=
 =?utf-8?B?bm9WcWo5MHljUjVZZTdHVWc3Q3lTaXNCVzRrNVMrcVk0a045OXZQZDlzNkk3?=
 =?utf-8?B?MEl3K1BGNEhUWWVKVjRSc3JPVlIzUkxkUTEzZE1WcFVsOGRTeFFlVWk0dyt5?=
 =?utf-8?B?MkZqREYyUm9wMFV1V2dOaUxsT0JWRVBUMnRmaW95ZmNwRG1TVVg3WTVzWFF4?=
 =?utf-8?B?RDZIdkxkMGxyWDh5TTVzV3FzeGo3YUlBMVZpVnRLTytRU2pTYTE2cjRLc1Ux?=
 =?utf-8?B?MWtObE9HQ3BqZ3NvQnlUbU9LbEJiZDliNHlnKzJ2a3JNRFBDWTFlcnN5SUZP?=
 =?utf-8?B?TVU1RVVEcytCTDZlV3VpN0dHcnA5YnRiRnEzdXpXMHp3Y3orcGpJTG9mYVYv?=
 =?utf-8?B?T3Q2MFIxWGo0UkJkLzhpbjdtNWVtVXNFK05zNTIrZXJaWWV2VHBnUXRNaUQ0?=
 =?utf-8?B?K0ZhM2Y5YXd5NG5Fcm1ya1A0T01EbHZ1bzlhMThkVWs3alVzejYyWWJrZGtX?=
 =?utf-8?B?dk5CSU5KZUZ5MTFQYnl3cFA0VEc0TlU2MmIrWnFPcEx6OU9rSXppQmdQUDNF?=
 =?utf-8?B?THBKRVk1SVVoS3A2SlZzSjhIaTM2cHhqOEFiOHVzU01QaU5QeWZVemRQcDlQ?=
 =?utf-8?B?bkJkeVpKcldYVUV4d3JrSXdocUFHdFlKYWF2eE5zS3RLQXRwMTRMTTQ0R1Ey?=
 =?utf-8?B?V201N2RMVjNxMTVmeHhBWTJhVG5kV0twdjhHSlN5ZTRBSGprMFZDRm52T3c5?=
 =?utf-8?B?YW11QlM0eDZnNHRadnpEL1dUcXZsKzh5dElYQjFTUDhlTWNpY3BQZWIzUk43?=
 =?utf-8?Q?Ip+vuNVCU+4G3SWAChuLr0QOw18JHessWC+crAn?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ab6b2368-65e3-4b2c-2fcc-08d97dd0d7cb
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 13:56:57.8116
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: adL9GHjmxFK8opL1cf68tCzx5vX3IF2TAjQDLeWSM3zkt5e+R2FV6p2OjI1FchAzgqZXyAI5A96Y5eGQq+H5PMuvxzhzBtsiUGfmegMAhpk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4109
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10114 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999
 spamscore=0 malwarescore=0 adultscore=0 mlxscore=0 bulkscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220100
X-Proofpoint-ORIG-GUID: 4gUOny7eEi5k4lSyVMf7k2yU2bhGGkBU
X-Proofpoint-GUID: 4gUOny7eEi5k4lSyVMf7k2yU2bhGGkBU


On 9/22/21 9:39 AM, Jan Beulich wrote:
> On 22.09.2021 15:29, Boris Ostrovsky wrote:
>> On 9/22/21 6:17 AM, Jan Beulich wrote:
>>> @@ -817,7 +818,7 @@ static long privcmd_ioctl_mmap_resource(
>>>  			unsigned int i;
>>>  
>>>  			for (i = 0; i < num; i++) {
>>> -				rc = pfns[i];
>>> +				rc = errs[i];
>>>  				if (rc < 0)
>>>  					break;
>>
>> Can the assignment be moved inside the 'if' statement?
> I wouldn't mind, albeit it's not the purpose of this change. Plus
> generally, when I do such elsewhere, I'm frequently told to better
> leave things as separate statements. IOW I'm a little surprised by
> the request.


Sure, can be done as a separate patch.


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>


>
>> I am also not sure I understand why we need error array at all. Don't we always look at the first error only? In fact, AFAICS this is the only place where we look at the value.
> Well, to look at the first error we need to scan the array to find
> one. Indeed we bail from here in once we've found a slot which has
> failed.
>
> I guess what you're trying to say is that there's room for
> improvement. In which case I might agree, but would want to point
> out that doing so would mean removing flexibility from the
> underlying function(s) (which may or may not be fine depending on
> what existing and future requirements there are).


We haven't needed this for a while and IMO existing code, with overloading the meaning of the pfn array, is rather confusing, unnecessarily complicated and error-prone (thus your patch).


>  And that would
> be for another day, if at all.


True.



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:02:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:02:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192796.343431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2pR-0001HG-Ad; Wed, 22 Sep 2021 14:02:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192796.343431; Wed, 22 Sep 2021 14:02:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2pR-0001H9-7J; Wed, 22 Sep 2021 14:02:29 +0000
Received: by outflank-mailman (input) for mailman id 192796;
 Wed, 22 Sep 2021 14:02:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mT2pQ-0001H3-3n
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:02:28 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b97125e8-7656-4356-81f7-5369d6edcb7d;
 Wed, 22 Sep 2021 14:02:26 +0000 (UTC)
Received: from pps.filterd (m0246627.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MD8Teu027967; 
 Wed, 22 Sep 2021 14:02:24 GMT
Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4rc5yw-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 14:02:20 +0000
Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1])
 by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18ME0rL9096014;
 Wed, 22 Sep 2021 14:02:18 GMT
Received: from nam12-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam12lp2173.outbound.protection.outlook.com [104.47.59.173])
 by aserp3020.oracle.com with ESMTP id 3b7q5aqfr0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 14:02:18 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB5201.namprd10.prod.outlook.com (2603:10b6:208:332::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 14:02:16 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:02:16 +0000
Received: from [10.74.114.106] (138.3.201.42) by
 SN4PR0401CA0019.namprd04.prod.outlook.com (2603:10b6:803:21::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Wed, 22 Sep 2021 14:02:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b97125e8-7656-4356-81f7-5369d6edcb7d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=vjKfXFb7ybIVhkg0g69KskKhyPpx49YcvOPdtRFCrm4=;
 b=i79AmjsSOBLRMEsbAAHkVlEQwF/wF1KrPsJ78rOUHXEtXHFsrFrB4jHImvkknYr+Qoof
 0hy7qiY9KW//SIBEYwI0JaqjEcLvOaC3fcYiPJOVnPPtu27Pz9P2cI+rAON5i4WSqRKL
 pzdbs7oz4LvmT+eGHMpUx6kFjed5PQGI7++dSYQgsHr0/bF37mTVtf0+/NsLIM8KUrf4
 WtJ32fMxJoSzd+j7K756FG3DXXGY0LpawlwpM2WocJZU0pOACrhW5gOAp+KJhlZmjYGS
 cLJ+/y4jPTjbG0bPHRZUd2Jd9U3CP9l8hi52trP3WhumixXqVUOcLdXcvll8n+a1xsaD uw== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kdKKwC6YsgBgQOggfkWZb9AjEgJKzAHxt36FmH7wOtNeSVOXKpwYbFonFkMBmDw+qP0lNk2GzAhdFDg8J6geKD/6Q30TYBIWs5J16vOmj9t00pPcivQBaOIFmMune/Mz0bUE1R4YnKYh2P4MRG7ODKr/EMlfqu884IRPTHSZbI4GlUGf90auZ7QIizP3UhdLxa5/WuJrtgeUmAj9nFJ7YzEfndbvZJ7BGfD2Xm5j750gSE5qGEal56f1LIL+RnPefIm8WnhYF6gQqI9ol+zLppgRpEmI7S6wiWntTDtEsCS/33z6oCRXAwBcUGBTB8ilpMu7uU+MdejQPoeC2BcBeg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vjKfXFb7ybIVhkg0g69KskKhyPpx49YcvOPdtRFCrm4=;
 b=Pe7zvmIwzilfbYq+pNXncD2MMOe6GLnvb+Yjb9CE839wXrrFhxIJojte9wdVtPiSIqYBS7ZP0bKABP3bNjSVFozkWO3T/ZFmWkgvifA0evYahQwOaFKdfq3SztbEWiorAuqCZ5WiQISxPE3sArE8aZc0K6+lQAnjK7+00C3wYJufXNEt3JEozpLCwzYJDmxYIQkoqVBLu32kl6+bSkSnABfn/uNhPKW1uxiilp34ZmSKmgdEj9uXsl9ZGX+GxWlbB5u5k69AoX0REWH2o1M8Xlas3pIRdHHYm5nk+Ut2j6Dv7TZ5sK+ZNJqG/T99A1FYD7Sy5K/sB+E0IBMO5j4kkw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vjKfXFb7ybIVhkg0g69KskKhyPpx49YcvOPdtRFCrm4=;
 b=ACTUAfh8ghMQcBCY+gJR+b/HcDLGAhEeJZg6A7Q7vQf84BB5jzQhq79RKUrRgiLRecWqhRzXB40aR0x4DRbrwSHhyGAC35q0/yvlmCpP2L8gi0mUIeUAweAFx7WxiIS1PWJmOHi5DVSPyFHBPG3Z+yJp4LvJd3z4QD0p02rlf+g=
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH 3/3] xen/privcmd: drop "pages" parameter from
 xen_remap_pfn()
To: Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <036ad8a2-46f9-ac3d-6219-bdc93ab9e10b@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <53732207-cd36-2214-94b3-be1837957d8d@oracle.com>
Date: Wed, 22 Sep 2021 10:02:12 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <036ad8a2-46f9-ac3d-6219-bdc93ab9e10b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN4PR0401CA0019.namprd04.prod.outlook.com
 (2603:10b6:803:21::29) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 10cde072-fcad-40c8-65ab-08d97dd195c8
X-MS-TrafficTypeDiagnostic: BLAPR10MB5201:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB52015EE5B096EA8B73AE71138AA29@BLAPR10MB5201.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	NfWL89EPl6jiPr8MYN3fa+FydfBe36MoCAOf9Fu8roRxqXZ/G6Rs5urPe42ArFDUR/tOeE8JHbVoe+FDxu6Ycc15Wd5MMRH0Ef8cZTh1QC+KD25bSTdIwVx0vQboxR1ri7zPlgbARnzWH8wRSf2KSWf2EV/8lL3TbfLLur6EdEU6+pWCb08tP/GFbh6qtb3MGdTplBaxn9/+8SP+YgxcRLi/xDjYFrGK8e22ugE7tmhH4wbYg+fjPB8u7dMLh/zdgfU7UA0UCHaJluCZ2S6dBYaj49Ej8TJwjZSoDsOCJBqo6zdonI7awrqOMcIYoxlmGUp1IhGWxc+eUjpKyEnlrpXSsdRlDND1RLoZ7OmFJKguX0EmH9YTXmuP8rPeqjhd4yYjOHmi5S0q6Uwygk8U4P/4MNK5tv8H1jifkVmrfILMWeU+VhNi/ZPV9tUOW3PtU64/yxa5aQF8huO+1sjn4KMN66m/8+xwMYZ86vpV7q09Zguv4nemSWefk5LCKdSksKtL7LEduvWEHYNZl3lqQsNKtUoyJEzL0PdTLLbDEV+lubVPaKNGc5yNpXoS9/cTe3H48KQl3MhV1bQ7mhK24QKfiGVOJZfipTgTG8d5OurADB9A8LDDPM1OHB8GWO92M0BOimV2vrk58A/ejljlBSj2+XqNZP7VkOKMyu5AWWnPbnCuCVfZNFX9C7E/Xaqu8IWsD/tTFrPBEHkTMJi+PmS+hAioK9WW65nnngtPwrbx+HDuONODyuRiCq42D1GC
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(956004)(38100700002)(26005)(16576012)(53546011)(5660300002)(31686004)(44832011)(186003)(66476007)(316002)(2616005)(66946007)(66556008)(6666004)(508600001)(4326008)(54906003)(110136005)(4744005)(86362001)(8936002)(2906002)(6486002)(83380400001)(8676002)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?L2NGdi91bTF2QUUwWHRYQmY0cDhmdTJmYjBqZStDVHdLL0NNbWs4dGhsS2s0?=
 =?utf-8?B?TmhMTjNLVnpROFJjWkI0Qi9ENlRENG1WL3laa3RKNzRIRVJMUmlLNk1ObW9u?=
 =?utf-8?B?Zk1WcCt1S3ZSa051cnBUVU5zT3B5WU9adHZFKzNkV3pIQ3k5aDhrTUlFTWth?=
 =?utf-8?B?UlRYbnVUOGJyeFZtaHJMNDZEWGtodWFBSk5VSFcxaVFONmI5djRjUkxIZVE5?=
 =?utf-8?B?WEtZbTk5SkkvVkdJY3dxdFFaL2l3MjYwbW5YUmhxOUhSZTlGRDEycU03N0hj?=
 =?utf-8?B?ZUlXb2IyZkRqNTc5ZG5mQWhmVlR5QWd6Vkc0cU9nZEpFTEphSXRLcUJRdzB4?=
 =?utf-8?B?Vko0eWo3bG1OTFdVK0hEL2JKNzJKZWFjQXozbWtpa0oxcEN0cUZwQjFwVm4x?=
 =?utf-8?B?RS82TXRoRXVHVXNEMHIwRFRJaUtIekNFcFBmcElPUjEramFFWTIxamY0M0F2?=
 =?utf-8?B?dStlNkxhTlVFNFlXUU9GUDNNMzRGSW9PTkpqV1RlMHdQSHgxanJISUVtbE5h?=
 =?utf-8?B?SmNjN3V2eGtRSWNzTnBzeDU2RXFLb1NSOTBhek5pRGtPQ3hzbmY4NGJ1UnNK?=
 =?utf-8?B?OVhxeHdwd1FPd2dXV0RYUHFzTkhWd3U2dlFJWmNCN1Jjbmc4RFBnMVpZV1RR?=
 =?utf-8?B?L01xQUhXTVhNRVY0bUdOOU9SVStOUDNqNjNyNUVnaXY2RFgrRkMzZ25GdWk2?=
 =?utf-8?B?WDR0VGwrUmlmQzNQcEhqNHVSb051aE0rL3cxbTYxM3gxcVFtOHhVbjJHMUZV?=
 =?utf-8?B?K0hEUzNaeXZObG5rallHZ0pza010MlI1ckxSSHcyTWUzWUdqTDhkZ0F4UE5t?=
 =?utf-8?B?N3l1WHdHNFBaSEdKWHdjWklDc2lZRUt6UDFsZHE2VXhtN2J4cjZhc0pYSkdk?=
 =?utf-8?B?SnI4VlViSFlSZVMyMnVaaTVDdVNuSmtvSG1XRWRNZVQ4VC9xdlk0SWtycnBv?=
 =?utf-8?B?cWhwZk9aWG4yVVh0dUFtSk45OUliSHVjSHYzWVNON0s1NjhKVXVRdFQwZXMx?=
 =?utf-8?B?VlJ4SEtDTXBmZ3FBWWhTOGhyQUZia3pPVWs2YmM4NGpDS0pFMnoxQ09oeU9a?=
 =?utf-8?B?MStqZUJpNXdldGt1ZkIyK0xhMDdRS0F2QXQ5dDk3WC9jR1E4ZWZiWkE2bVZU?=
 =?utf-8?B?WXBSdTViK2p4My9rMzEvR1J6ZkR4MlBtbFo4N2RkSjYwTGY2aFZjMEIyb01F?=
 =?utf-8?B?dEErdFFlSXdlRkx1aFhrOE9NdmNtZEJLOG44cHllN0ZPaGhDNTdGM2J1RGRq?=
 =?utf-8?B?a0RncnR6aWJWR0pkNUlPYlpIRU1LYkptYzFibnpKYjE5YUFNYm9iWmtXVEpN?=
 =?utf-8?B?NWUyQnI1NDVEVlBoK0xFVDRsQk5PRURsalUwUmFMWStzQk5UclJJNTgzWitR?=
 =?utf-8?B?N3pueG9Fdjd3OENWM1A1MC9OeEV1di9TNjRubldraGRQZE1qTWRNdnV4cjUy?=
 =?utf-8?B?V3g4cjlJSGJUOVFXM0Z0M1owZ3k2QXJXYkVzeFdDa01rRk5YbHhLMy9EUkR5?=
 =?utf-8?B?aFoyWVlTYUovazN2RWdBSFZvSkpIRUh5K0lmamlPeHROdHZrMlRqclFZdTNo?=
 =?utf-8?B?YjlxdkFJYzFFZlBxa3QwMmczY3F0RlNYR3FSVThlR01iQ2EwZS9zZnpqcW9l?=
 =?utf-8?B?ZzBXQWY4NkxqYzQ0ZHhoNnNoUjQyVytDOWhHbmg4SHRCSC96M1ZBUVA2TDZa?=
 =?utf-8?B?T3hJZDI0ZFBuZm11Z2htaWp5MklkR0JMZC9tS01Oc2JUS1dPK2l1TmdDTTdk?=
 =?utf-8?Q?YC5zzDaMsCdaN9ram0scRfckdA069qW8dG65uE4?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 10cde072-fcad-40c8-65ab-08d97dd195c8
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:02:16.6291
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: k8hvdfdMbSDLTyzuQddvfFiK+JWMKBkhE8SQFf2nO4M0HRE4vDnjTkFqRc1jkBHOygZWbkmtYmLGvS+N4ksApWh+ms6iaUwfPzRxgw8RQBo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5201
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10114 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0
 suspectscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220101
X-Proofpoint-GUID: cloD8LtIeUPUxAx6a11sZF801fOJ2kPS
X-Proofpoint-ORIG-GUID: cloD8LtIeUPUxAx6a11sZF801fOJ2kPS


On 9/22/21 6:18 AM, Jan Beulich wrote:
> The function doesn't use it and all of its callers say in a comment that
> their respective arguments are to be non-NULL only in auto-translated
> mode. Since xen_remap_domain_mfn_array() isn't supposed to be used by
> non-PV, drop the parameter there as well. It was bogusly passed as non-
> NULL (PRIV_VMA_LOCKED) by its only caller anyway. For
> xen_remap_domain_gfn_range(), otoh, it's not clear at all why this
> wouldn't want / might not need to gain auto-translated support down the
> road, so the parameter is retained there despite now remaining unused
> (and the only caller passing NULL); correct a respective comment as
> well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:12:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:12:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192802.343442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2zJ-0002su-9c; Wed, 22 Sep 2021 14:12:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192802.343442; Wed, 22 Sep 2021 14:12:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT2zJ-0002sn-6N; Wed, 22 Sep 2021 14:12:41 +0000
Received: by outflank-mailman (input) for mailman id 192802;
 Wed, 22 Sep 2021 14:12:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT2zI-0002sh-1k
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:12:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fea10ca2-8cbe-40de-9b7b-ad2b2322457d;
 Wed, 22 Sep 2021 14:12:39 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-t3BeueurMHO_6OdA_ETaAQ-1; Wed, 22 Sep 2021 16:12:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5902.eurprd04.prod.outlook.com (2603:10a6:803:ed::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Wed, 22 Sep
 2021 14:12:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:12:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0011.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:12:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fea10ca2-8cbe-40de-9b7b-ad2b2322457d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632319958;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Du2wDA6nvKLVsZijKk1h7XmI403NREUTTlM8111wuVI=;
	b=mYVxBJwkDt4pOqsnSqDSV0SGAPgxZz9Faowz0RVaawJKO0+w0D2AlOmAghA+TarCVbgCbQ
	gl3mOYynCocPz71sjvjcjDKX6glkrYpKmfxQ4cCJBejSNNsD9AF1xi4Fw0cXEtWY/BvAVY
	vMsJ6drSnoKM227M6mTPKaXNFvnYYBE=
X-MC-Unique: t3BeueurMHO_6OdA_ETaAQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TTCgNlgqO0DCuVx7OJJmBE3Jm9Lge1A+RwJC2niDE0AbyMOdQUfHQnXf9bpUPbBR0VW1H0EIL9co9BOGV0m9ikah2KRuq4tHuzrY+Dvm5G5hQar6VVJiDwJgkLAJVTyx2SUKEa2oH0YJsIS1LnHb/ueLaXelVmwHES8PBpYANDHlKl2IkuhUbaDNb/Xax0mmn4Yt1/47KDzyknFuIj7rDtr+/YCc4zBUYes4m3wN9+y2UNhxGlTEamVHOG4pJIbvZk+ErHB+g+Odzpp87/cyyieVQVT5J00rdyQmrL8OOyKPfmLgQpt8I4KEQkn0Jgeome9s2k9lE3h9m6GUqKK2lw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Du2wDA6nvKLVsZijKk1h7XmI403NREUTTlM8111wuVI=;
 b=lOb32P4N4AKMARc4zc8boVtR+g/qmDgw4EPG7jbZpAcOSDl94r50JVwWvcDl5p4P1UhO6wT3ddfOIYmG9uUOHzJhjL4TyaeyRRTEp037t12Zwewn5h9M/Fkw2EyOSpn9Jylf7UwdL31lIvT5I2ppf/lUoYB9MOMqY2k2VwkgTxxhggjvEPCjsPGVg04LTFDnmi/ACuFbeakT7QQkVVrAG+3W3lnKqISC/Sg0WGoxVShttoeXUMZvWw3s4kSXqjq2hGEZ2h4Pj9YMHBsNXzK5Wk2vOwb4lmAzENFIOYz+RLnH4zS1nzgbb5JErubk4j8+hTgGv7uYilmSYBbCaFZOIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] xen: fix broken tainted value in mark_page_free
To: Penny Zheng <Penny.Zheng@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "julien@xen.org" <julien@xen.org>, Wei Chen <Wei.Chen@arm.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <20210922114432.1093340-1-penny.zheng@arm.com>
 <94F4D68E-DF7B-4613-B03B-05C1CB1154DB@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4b4b59e7-d0b9-c64d-b90a-475c4152c4a9@suse.com>
Date: Wed, 22 Sep 2021 16:12:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <94F4D68E-DF7B-4613-B03B-05C1CB1154DB@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0011.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c4c5bfa9-70db-416c-428a-08d97dd305f6
X-MS-TrafficTypeDiagnostic: VI1PR04MB5902:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB59021FA2DE61B8B5DD06283BB3A29@VI1PR04MB5902.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2803;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TN8VZ8bWV/FbkgCr7OqHofu8q5188/GG7ONfTrr1hu6fEO4IjW9GYrC/yJkRdipdBfyCp3XmhUZ3TOshHOVGPgymRznU66TgQavtJwXwIhsVtevYMT38HBjLKGcMTg+DQRvUfiGP+6UEji/IMnRK0U/IKTGHSMeOIDW3H97ECooZpGAQb5MwaIwY7lyMu1F8zs842W0FTta0aaYc6kk9lFpxaASlWE2uADQcoIEHig62Fa12fOEWWMWzKcq/sX+c/jtLDegeljYSy66+mJSH5hBlyIwVIlLNY9ykoXYYLME5l46KI4nRHUuS5XdvubcVJ/lnjIM2645yFBcoL7w+gkMSWu35lTO2ol0G1pX9p5A7XM+5K+/KWHVH4i2csqVZxPau7IOmFs4P7K01iX6aAWTsc996zpxgbG+XimkGrs9mK0/oygoDGYgqpNBF1CtpbLN1d8/ETv1ME5QmkA9xWLg4V9WM220AmYXjOauVyQ9+nohC9jw0u0nN7Zrnv2WxHFLaDL60G7qqvokbIH4lWTKX+NNwV5jDITFRpLIzj0oCyj63yyNNblOtCwlx3qKqwQL4q4uuNzXKPSbPtgP/v6j/QoFGYsYKs/VyuIRo16SLW+anHO2hojC+Zatua71QvCO/MpLy7aEVjJYPwg68QUpv/GnYDdavO6KM8mRWU7N4CicpJ08KpWizTfZ2s4SdeUpqTP5r7qqLLmoG5/qm2z/ezkAL5+WpHBZKugMMh1U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(2906002)(31696002)(186003)(8676002)(86362001)(16576012)(66556008)(66476007)(66946007)(53546011)(26005)(316002)(54906003)(5660300002)(956004)(4326008)(6486002)(6916009)(508600001)(8936002)(2616005)(31686004)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R3AwMUVVMHNPRUl5MVhrWWUwd3BMU1FhNlJDNmlYYXQwNHpqaFNZN2REc1B2?=
 =?utf-8?B?ZG5peTB4cXpIY2hJNkV0RmphUVdLaGhPTzNyK2o1MlphN1B5T01XbmJXaWdM?=
 =?utf-8?B?eHdGQjlsaTdjM1I0aUxqZHhkb3VjQkhsZUo2UW4zTExKTlQwYVJHMXo2cS9D?=
 =?utf-8?B?bGVxWnRlNlZwdmFZaGl4N1E0R2JURHoxSWprR2V2ejRIQ2Q0dno3OEd0cGhR?=
 =?utf-8?B?RFBha3hyRFpSNC9EYkIwYlAzblJZY0JNcWJwUHN0M1NoSGtPZG5IdlRyVEZX?=
 =?utf-8?B?QlJPdk9ydE1LV21XNjhTczZZZUtnQWdyWjBOK3gzQXV4UTVwRWwyZWxNYVJU?=
 =?utf-8?B?WVMyK2Q2bjRTd0xMYzB0QXc0a3M1UEVDS0R0ejNLMkV6QVpIRVFQYkNkdTJ4?=
 =?utf-8?B?dUpDQ2R5ZTdTSVpBR0orMExUeFE1amhUSG15ZEhMa1JaOE9yajlxejZnUXhO?=
 =?utf-8?B?OS9ERm9QcUxQWVRFKzdHYVdXcWtPNGRoeGtLMEV6QUtqMEc4NVJ5V253aWJ6?=
 =?utf-8?B?ZFc3WHh0VG9kdUpqeHFOdXpRYTNlV1BBMW1kdUZwWjNMcm9VVHEyZmtQV2gz?=
 =?utf-8?B?MEVxKzVtcXJsS2NVSDRteXMrY3lSa1Bndm12T1Z1NG1lODVVYnVEMFhucEkr?=
 =?utf-8?B?WXY3cm5aT2k5cEEzb2s2aThnSTNGT3pyaXlweWpNNmMyUUpaTEZocG12d09u?=
 =?utf-8?B?ZTZEQi84SW5udE12R1REL05JQ2dhVVJZTlhTSXRBUnkzTDRjQ2xpbUtEeGJl?=
 =?utf-8?B?eEp1VXJjZFhxbDdXeXo1WHVoQUc1WEhSbytPamhzcit4ZlVhYnZ6S20weGpL?=
 =?utf-8?B?SDFnR0V5ZmhxY2h5QjBTaHNrTUFCZXpidjVBUGVHQ1RvaFBuMUM5YzErV05w?=
 =?utf-8?B?L0ljd2pqc2pDTGxWcEhFMzcvU0dDSDlyZk0yTWplK0gvVU9tNFRFMTNWREo4?=
 =?utf-8?B?eHpCVzRCcVRweEZZSm5vdVkvWHpJdmFFMEhPbXdQNkFSQ21EY0kvSDB1UlE2?=
 =?utf-8?B?SG5jOE9FZVIyRHZhdzlWMzBOT2lmRGlUYkJVS2FmUjdWUnRaTm84anM0L0k3?=
 =?utf-8?B?cEtwZHl0YkI0RzNYT3UyNnNoamJuZkpub2liTTZKUXdZNDFib3VSQURFbXB2?=
 =?utf-8?B?d0luV3ovTytUbjN4RWdGSHVJbVlMWjliM2s4NUNyR3U0YjI1ZHVPekNQZGNE?=
 =?utf-8?B?YW9DNEVJU1Z5QVVBeVAxM3M5OThtQ0ppWGdPV1BhM0dRUXFvNHhUcFdVUk5x?=
 =?utf-8?B?NlNnMFdyU29mcThvRG1GK1hmQWFLalQxVFdiZDhzWllDK01lQk9TelFTY214?=
 =?utf-8?B?VFhiakw4V0tNZGFTTUg4WVlKZitUemppdk9pUE01VUFVTGlRWTR5UHJMMnV1?=
 =?utf-8?B?WnRMbWttM1FEZzhTK2s2aDJZQlJxN0JGZ3J2NHB3RXhHYktFRDY4QlFuVHlI?=
 =?utf-8?B?cUk2L0tSWGwvN2p0TTNjbzlCQ0JNR0tNdkVqK1hoeXJ2bzRtS2d4K29vcDJ2?=
 =?utf-8?B?OU5wT1RWSERBa3ZSYm5NYlB1S2xISHRiRW1xWTB5MTZYM2E3WnBQNFlaTldQ?=
 =?utf-8?B?M1JvUUNLRWdSdnVQNUJQUkZnY2FBeDZlTjlGR1ZXaDNPTmdlVGJhNXFQQWZi?=
 =?utf-8?B?U041ZDlBcGpCMFRuckdSMXUxc0NmMC9NTzhRMENTTlVUNEpVM1pwbTZCQzh2?=
 =?utf-8?B?dFA5VjVIUTdBaGEvMjFPQmtXM2VxTXdyVTI4RGlWcU5xWXNCcFZFSG0rN2s1?=
 =?utf-8?Q?pbblWvJgJ9sGACgpj0JUjQW9N+QyREDi8/RpCCG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c4c5bfa9-70db-416c-428a-08d97dd305f6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:12:34.2151
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iU9ZJ/ENGoCkI/Zyg4qD/ukSYizFULxftCp9wF3Rps6jntfwQvbhW50tK5FPv3lNwTOwyXmTKzIBcax/iLLfaw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5902

On 22.09.2021 13:48, Bertrand Marquis wrote:
>> On 22 Sep 2021, at 12:44, Penny Zheng <penny.zheng@arm.com> wrote:
>>
>> Commit 540a637c3410780b519fc055f432afe271f642f8 defines a new
>> helper mark_page_free to extract common codes, while it accidently
>> breaks the local variable "tainted".
>>
>> This patch fix it by letting mark_page_free() return bool of whether the
>> page is offlined and rename local variable "tainted" to "pg_offlined".
>>
>> Coverity ID: 1491872
>>
>> Fixes: 540a637c3410780b519fc055f432afe271f642f8
>> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Albeit I would have wished that ...

>> @@ -1433,7 +1437,7 @@ static void free_heap_pages(
>> {
>>     unsigned long mask;
>>     mfn_t mfn = page_to_mfn(pg);
>> -    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), tainted = 0;
>> +    unsigned int i, node = phys_to_nid(mfn_to_maddr(mfn)), pg_offlined = 0;

... this would have become properly bool as well.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:14:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:14:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192810.343453 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30b-0003YQ-QF; Wed, 22 Sep 2021 14:14:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192810.343453; Wed, 22 Sep 2021 14:14:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30b-0003YJ-Mp; Wed, 22 Sep 2021 14:14:01 +0000
Received: by outflank-mailman (input) for mailman id 192810;
 Wed, 22 Sep 2021 14:14:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=btef=OM=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mT30a-0003YB-I4
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:14:00 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 535f4784-9f84-486f-b3be-01e0f6f5be2a;
 Wed, 22 Sep 2021 14:13:58 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7C6B5113E;
 Wed, 22 Sep 2021 07:13:58 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E9C333F59C;
 Wed, 22 Sep 2021 07:13:56 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 535f4784-9f84-486f-b3be-01e0f6f5be2a
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 0/2] arm/efi: Add dom0less support to UEFI boot
Date: Wed, 22 Sep 2021 15:13:39 +0100
Message-Id: <20210922141341.42288-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1

This serie introduces a way to start a dom0less setup when Xen is booting as EFI
application.
Using the device tree it's now possible to fetch from the disk and load in
memory all the modules needed to start any domU defined in the DT.
Dom0less for now is supported only by the arm architecture.

Luca Fancellu (2):
  arm/efi: Introduce uefi,cfg-load DT property
  arm/efi: Use dom0less configuration when using EFI boot

 docs/misc/arm/device-tree/booting.txt |  21 ++
 docs/misc/efi.pandoc                  | 205 ++++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 285 +++++++++++++++++++++++++-
 xen/arch/x86/efi/efi-boot.h           |   6 +
 xen/common/efi/boot.c                 |  36 ++--
 5 files changed, 533 insertions(+), 20 deletions(-)

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:14:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:14:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192811.343464 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30e-0003p9-1G; Wed, 22 Sep 2021 14:14:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192811.343464; Wed, 22 Sep 2021 14:14:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30d-0003p2-U8; Wed, 22 Sep 2021 14:14:03 +0000
Received: by outflank-mailman (input) for mailman id 192811;
 Wed, 22 Sep 2021 14:14:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=btef=OM=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mT30c-0003bo-4G
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:14:02 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 5536c1ea-1baf-11ec-b995-12813bfff9fa;
 Wed, 22 Sep 2021 14:14:00 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2607A11D4;
 Wed, 22 Sep 2021 07:14:00 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AFCDB3F59C;
 Wed, 22 Sep 2021 07:13:58 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5536c1ea-1baf-11ec-b995-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v2 1/2] arm/efi: Introduce uefi,cfg-load DT property
Date: Wed, 22 Sep 2021 15:13:40 +0100
Message-Id: <20210922141341.42288-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210922141341.42288-1-luca.fancellu@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>

Introduce the uefi,cfg-load DT property of /chosen
node for ARM whose presence decide whether to force
the load of the UEFI Xen configuration file.

The logic is that if any multiboot,module is found in
the DT, then the uefi,cfg-load property is used to see
if the UEFI Xen configuration file is needed.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v2 changes:
- Introduced uefi,cfg-load property
- Add documentation about the property
---
 docs/misc/efi.pandoc        |  2 ++
 xen/arch/arm/efi/efi-boot.h | 28 +++++++++++++++++++++++-----
 2 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ac3cd58cae..e289c5e7ba 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
 bootloader provides a device tree containing modules then any configuration
 files are ignored, and the bootloader is responsible for populating all
 relevant device tree nodes.
+The property "uefi,cfg-load" can be specified in the /chosen node to force Xen
+to load the configuration file even if multiboot modules are found.
 
 Once built, `make install-xen` will place the resulting binary directly into
 the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index cf9c37153f..8ceeba4ad1 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
 
 static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
 {
+    bool skip_cfg_file = false;
     /*
      * For arm, we may get a device tree from GRUB (or other bootloader)
      * that contains modules that have already been loaded into memory.  In
-     * this case, we do not use a configuration file, and rely on the
-     * bootloader to have loaded all required modules and appropriate
-     * options.
+     * this case, we search for the property uefi,cfg-load in the /chosen node
+     * to decide whether to skip the UEFI Xen configuration file or not.
      */
 
     fdt = lookup_fdt_config_table(SystemTable);
     dtbfile.ptr = fdt;
     dtbfile.need_to_free = false; /* Config table memory can't be freed. */
-    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
+
+    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
+    {
+        /* Locate chosen node */
+        int node = fdt_subnode_offset(fdt, 0, "chosen");
+        const void *cfg_load_prop;
+        int cfg_load_len;
+
+        if ( node > 0 )
+        {
+            /* Check if uefi,cfg-load property exists */
+            cfg_load_prop = fdt_getprop(fdt, node, "uefi,cfg-load",
+                                        &cfg_load_len);
+            if ( !cfg_load_prop )
+                skip_cfg_file = true;
+        }
+    }
+
+    if ( !fdt || !skip_cfg_file )
     {
         /*
          * We either have no FDT, or one without modules, so we must have a
-         * Xen EFI configuration file to specify modules.  (dom0 required)
+         * Xen EFI configuration file to specify modules.
          */
         return true;
     }
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:14:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:14:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192812.343475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30h-00048I-9j; Wed, 22 Sep 2021 14:14:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192812.343475; Wed, 22 Sep 2021 14:14:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT30h-000483-6U; Wed, 22 Sep 2021 14:14:07 +0000
Received: by outflank-mailman (input) for mailman id 192812;
 Wed, 22 Sep 2021 14:14:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=btef=OM=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mT30f-0003YB-H3
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:14:05 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 463e767a-36bb-42b2-89f6-815e4eac9028;
 Wed, 22 Sep 2021 14:14:02 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 20DE3113E;
 Wed, 22 Sep 2021 07:14:02 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.197.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5AFE93F59C;
 Wed, 22 Sep 2021 07:14:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 463e767a-36bb-42b2-89f6-815e4eac9028
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using EFI boot
Date: Wed, 22 Sep 2021 15:13:41 +0100
Message-Id: <20210922141341.42288-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210922141341.42288-1-luca.fancellu@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>

This patch introduces the support for dom0less configuration
when using UEFI boot on ARM, it permits the EFI boot to
continue if no dom0 kernel is specified but at least one domU
is found.

Introduce the new property "uefi,binary" for device tree boot
module nodes that are subnode of "xen,domain" compatible nodes.
The property holds a string containing the file name of the
binary that shall be loaded by the uefi loader from the filesystem.

Update efi documentation about how to start a dom0less
setup using UEFI

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v2:
- remove array of struct file
- fixed some int types
- Made the code use filesystem even when configuration
file is skipped.
- add documentation of uefi,binary in booting.txt
- add documentation on how to boot all configuration
for Xen using UEFI in efi.pandoc
---
 docs/misc/arm/device-tree/booting.txt |  21 +++
 docs/misc/efi.pandoc                  | 203 ++++++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 257 +++++++++++++++++++++++++-
 xen/arch/x86/efi/efi-boot.h           |   6 +
 xen/common/efi/boot.c                 |  36 ++--
 5 files changed, 508 insertions(+), 15 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 44cd9e1a9a..bc0f8913db 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -182,6 +182,13 @@ The kernel sub-node has the following properties:
 
     Command line parameters for the guest kernel.
 
+- efi,binary (UEFI boot only)
+
+    Specifies the file name to be loaded by the UEFI boot for this module. If
+    this is specified, there is no need to specify the reg property because it
+    will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 The ramdisk sub-node has the following properties:
 
 - compatible
@@ -193,6 +200,13 @@ The ramdisk sub-node has the following properties:
     Specifies the physical address of the ramdisk in RAM and its
     length.
 
+- efi,binary (UEFI boot only)
+
+    Specifies the file name to be loaded by the UEFI boot for this module. If
+    this is specified, there is no need to specify the reg property because it
+    will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 
 Example
 =======
@@ -257,6 +271,13 @@ The dtb sub-node should have the following properties:
     Specifies the physical address of the device tree binary fragment
     RAM and its length.
 
+- efi,binary (UEFI boot only)
+
+    Specifies the file name to be loaded by the UEFI boot for this module. If
+    this is specified, there is no need to specify the reg property because it
+    will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 As an example:
 
         module@0xc000000 {
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index e289c5e7ba..698196e129 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,3 +167,206 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+
+## UEFI boot and dom0less on ARM
+
+Dom0less feature is supported by ARM and it is possible to use it when Xen is
+started as an EFI application.
+The way to specify the domU domains is by Device Tree as specified in the
+[dom0less](dom0less.html) documentation page under the "Device Tree
+configuration" section, but instead of declaring the reg property in the boot
+module, the user must specify the "uefi,binary" property containing the name
+of the binary file that has to be loaded in memory.
+The UEFI stub will load the binary in memory and it will add the reg property
+accordingly.
+
+An example here:
+
+domU1 {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	compatible = "xen,domain";
+	memory = <0 0x20000>;
+	cpus = <1>;
+	vpl011;
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "console=ttyAMA0";
+	};
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "initrd-3.0.31-0.4-xen";
+	};
+	module@3 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "passthrough.dtb";
+	};
+};
+
+## How to boot different Xen setup using UEFI
+
+Here the supported user cases for Xen when UEFI boot is used:
+
+ - Boot Xen and Dom0 (minimum required)
+ - Boot Xen and DomU(s) (true dom0less, only on ARM)
+ - Boot Xen, Dom0 and DomU(s) (only on ARM)
+
+### Boot Xen and Dom0
+
+This configuration can be started using the Xen configuration file in the
+example above.
+
+### Boot Xen and DomU(s)
+
+This configuration needs the domU domain(s) specified in the /chosen node,
+examples of how to do that are provided by the documentation about dom0less
+and the example above shows how to use the "uefi,binary" property to use the
+UEFI stub for module loading.
+Providing the multiboot modules in the device tree, make Xen skip its UEFI
+configuration file, if it is needed for some reason, specify the "uefi,cfg-load"
+property in the /chosen node.
+
+Example 1 of how to boot a true dom0less configuration:
+
+Xen configuration file: skipped.
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "<Xen command line>"
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+Example 2 of how to boot a true dom0less configuration:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	uefi,cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+### Boot Xen, Dom0 and DomU(s)
+
+This configuration is a mix of the two configuration above, to boot this one
+the configuration file must be processed so the /chosen node must have the
+"uefi,cfg-load" property.
+
+Here an example:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
+ramdisk=initrd-3.0.31-0.4-xen
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	uefi,cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 8ceeba4ad1..e2b007ece0 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -8,9 +8,43 @@
 #include <asm/setup.h>
 #include <asm/smp.h>
 
+typedef struct {
+    char *name;
+    unsigned int name_len;
+    EFI_PHYSICAL_ADDRESS addr;
+    UINTN size;
+} dom0less_module_name;
+
+/*
+ * Binaries will be translated into bootmodules, the maximum number for them is
+ * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
+ */
+#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
+static struct file __initdata dom0less_file;
+static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
+static unsigned int __initdata dom0less_modules_available =
+                               MAX_DOM0LESS_MODULES;
+static unsigned int __initdata dom0less_modules_idx;
+
+#define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
+
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
 
+static int get_dom0less_file_index(const char *name, unsigned int name_len);
+static unsigned int allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
+                                           const char *name,
+                                           unsigned int name_len);
+static void handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
+                                        int module_node_offset,
+                                        int reg_addr_cells,
+                                        int reg_size_cells);
+static void handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                        int domain_node,
+                                        int addr_cells,
+                                        int size_cells);
+static bool efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle);
+
 #define DEVICE_TREE_GUID \
 {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
 
@@ -552,8 +586,218 @@ static void __init efi_arch_handle_module(const struct file *file,
                          kernel.size) < 0 )
             blexit(L"Unable to set reg property.");
     }
-    else
+    else if ( file != &dom0less_file )
+        /*
+         * If file is not a dom0 module file and it's not a domU module,
+         * stop here.
+         */
         blexit(L"Unknown module type");
+
+    /*
+     * dom0less_modules_available is decremented here because for each dom0
+     * file added, there will be an additional bootmodule, so the number
+     * of dom0less module files will be decremented because there is
+     * a maximum amount of bootmodules that can be loaded.
+     */
+    dom0less_modules_available--;
+}
+
+/*
+ * This function checks for a binary previously loaded with a give name, it
+ * returns the index of the file in the dom0less_files array or a negative
+ * number if no file with that name is found.
+ */
+static int __init get_dom0less_file_index(const char *name,
+                                          unsigned int name_len)
+{
+    unsigned int i;
+    int ret = ERROR_DOM0LESS_FILE_NOT_FOUND;
+
+    for (i = 0; i < dom0less_modules_idx; i++)
+    {
+        dom0less_module_name *mod = &dom0less_modules[i];
+        if ( (mod->name_len == name_len) &&
+             (strncmp(mod->name, name, name_len) == 0) )
+        {
+            ret = i;
+            break;
+        }
+    }
+    return ret;
+}
+
+/*
+ * This function allocates a binary and keeps track of its name, it
+ * returns the index of the file in the dom0less_files array.
+ */
+static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
+                                                  const char *name,
+                                                  unsigned int name_len)
+{
+    dom0less_module_name* file_name;
+    union string module_name;
+    unsigned int ret_idx;
+
+    /*
+     * Check if there is any space left for a domU module, the variable
+     * dom0less_modules_available is updated each time we use read_file(...)
+     * successfully.
+     */
+    if ( !dom0less_modules_available )
+        blexit(L"No space left for domU modules");
+
+    module_name.s = (char*) name;
+    ret_idx = dom0less_modules_idx;
+
+    /* Save at this index the name of this binary */
+    file_name = &dom0less_modules[ret_idx];
+
+    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
+                              (void**)&file_name->name) != EFI_SUCCESS )
+        blexit(L"Error allocating memory for dom0less binary name");
+
+    /* Save name and length of the binary in the data structure */
+    strlcpy(file_name->name, name, name_len);
+    file_name->name_len = name_len;
+
+    /* Load the binary in memory */
+    read_file(dir_handle, s2w(&module_name), &dom0less_file, NULL);
+
+    /* Save address and size */
+    file_name->addr = dom0less_file.addr;
+    file_name->size = dom0less_file.size;
+
+    /* s2w(...) allocates some memory, free it */
+    efi_bs->FreePool(module_name.w);
+
+    dom0less_modules_idx++;
+
+    return ret_idx;
+}
+
+/*
+ * This function checks for the presence of the uefi,binary property in the
+ * module, if found it loads the binary as dom0less module and sets the right
+ * address for the reg property into the module DT node.
+ */
+static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
+                                          int module_node_offset,
+                                          int reg_addr_cells,
+                                          int reg_size_cells)
+{
+    const void *uefi_name_prop;
+    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
+    int uefi_name_len, file_idx;
+    dom0less_module_name *file;
+
+    /* Read uefi,binary property to get the file name. */
+    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
+                                 &uefi_name_len);
+
+    if ( !uefi_name_prop )
+        /* Property not found */
+        return;
+
+    file_idx = get_dom0less_file_index(uefi_name_prop, uefi_name_len);
+    if (file_idx < 0)
+        file_idx = allocate_dom0less_file(dir_handle, uefi_name_prop,
+                                          uefi_name_len);
+
+    file = &dom0less_modules[file_idx];
+
+    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
+
+    /* Rename the module to be module@{address} */
+    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
+        blexit(L"Unable to add domU ramdisk FDT node.");
+
+    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
+                     file->addr, file->size) < 0 )
+        blexit(L"Unable to set reg property.");
+}
+
+/*
+ * This function checks for boot modules under the domU guest domain node
+ * in the DT.
+ */
+static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                               int domain_node,
+                                               int addr_cells,
+                                               int size_cells)
+{
+    int module_node;
+    /*
+     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
+     * inside this node
+     */
+    for ( module_node = fdt_first_subnode(fdt, domain_node);
+          module_node > 0;
+          module_node = fdt_next_subnode(fdt, module_node) )
+        if ( (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,kernel") == 0) ||
+             (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,ramdisk") == 0) ||
+             (fdt_node_check_compatible(fdt, module_node,
+                                        "multiboot,device-tree") == 0) )
+            /* The compatible is one of the strings above, check the module */
+            handle_dom0less_module_node(dir_handle, module_node, addr_cells,
+                                        size_cells);
+}
+
+/*
+ * This function checks for xen domain nodes under the /chosen node for possible
+ * domU guests to be loaded.
+ */
+static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle)
+{
+    int chosen;
+    int addr_len, size_len;
+    unsigned int i = 0;
+
+    /* Check for the chosen node in the current DTB */
+    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
+    if ( chosen < 0 )
+        blexit(L"Unable to setup chosen node");
+
+    /* Check for nodes compatible with xen,domain under the chosen node */
+    for ( int node = fdt_first_subnode(fdt, chosen);
+          node > 0;
+          node = fdt_next_subnode(fdt, node) )
+    {
+        int addr_cells, size_cells, len;
+        const struct fdt_property *prop;
+
+        if ( fdt_node_check_compatible(fdt, node, "xen,domain") != 0 )
+            continue;
+
+        /* Get or set #address-cells and #size-cells */
+        prop = fdt_get_property(fdt, node, "#address-cells", &len);
+        if ( !prop )
+            blexit(L"#address-cells not found in domain node.");
+
+        addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+        prop = fdt_get_property(fdt, node, "#size-cells", &len);
+        if ( !prop )
+            blexit(L"#size-cells not found in domain node.");
+
+        size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+        /* Found a node with compatible xen,domain; handle this node. */
+        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_cells);
+    }
+
+    /* Free dom0less file names if any */
+    for ( ; i < dom0less_modules_idx; i++ )
+    {
+        /* Free dom0less binary names */
+        efi_bs->FreePool(dom0less_modules[i].name);
+    }
+
+    if ( dom0less_modules_idx > 0 )
+        return true;
+
+    return false;
 }
 
 static void __init efi_arch_cpu(void)
@@ -562,8 +806,19 @@ static void __init efi_arch_cpu(void)
 
 static void __init efi_arch_blexit(void)
 {
+    unsigned int i = 0;
+
     if ( dtbfile.need_to_free )
         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
+    /* Free dom0less files if any */
+    for ( ; i < dom0less_modules_idx; i++ )
+    {
+        /* Free dom0less binary names */
+        efi_bs->FreePool(dom0less_modules[i].name);
+        /* Free dom0less binaries */
+        efi_bs->FreePages(dom0less_modules[i].addr,
+                          PFN_UP(dom0less_modules[i].size));
+    }
     if ( memmap )
         efi_bs->FreePool(memmap);
 }
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 9b0cc29aae..950fdf16b7 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
     efi_bs->FreePool(ptr);
 }
 
+static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle)
+{
+    /* x86 doesn't support dom0less */
+    return false;
+}
+
 static void __init efi_arch_cpu(void)
 {
     uint32_t eax = cpuid_eax(0x80000000);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 758f9d74d2..7d8734199e 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1127,15 +1127,16 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
     EFI_LOADED_IMAGE *loaded_image;
     EFI_STATUS status;
-    unsigned int i, argc;
-    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
+    unsigned int i, argc = 0;
+    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
     UINTN gop_mode = ~0;
     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
     union string section = { NULL }, name;
     bool base_video = false;
-    const char *option_str;
+    const char *option_str = NULL;
     bool use_cfg_file;
+    EFI_FILE_HANDLE dir_handle;
 
     __set_bit(EFI_BOOT, &efi_flags);
     __set_bit(EFI_LOADER, &efi_flags);
@@ -1216,9 +1217,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
     efi_arch_relocate_image(0);
 
+    /* Get the file system interface. */
+    dir_handle = get_parent_handle(loaded_image, &file_name);
+
     if ( use_cfg_file )
     {
-        EFI_FILE_HANDLE dir_handle;
         UINTN depth, cols, rows, size;
 
         size = cols = rows = depth = 0;
@@ -1229,9 +1232,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
         gop = efi_get_gop();
 
-        /* Get the file system interface. */
-        dir_handle = get_parent_handle(loaded_image, &file_name);
-
         /* Read and parse the config file. */
         if ( read_section(loaded_image, L"config", &cfg, NULL) )
             PrintStr(L"Using builtin config file\r\n");
@@ -1285,14 +1285,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
             efi_bs->FreePool(name.w);
         }
 
-        if ( !name.s )
-            blexit(L"No Dom0 kernel image specified.");
-
         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
 
-        option_str = split_string(name.s);
+        if ( name.s )
+            option_str = split_string(name.s);
 
-        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
+        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
+             name.s )
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
@@ -1361,12 +1360,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
         cfg.addr = 0;
 
-        dir_handle->Close(dir_handle);
-
         if ( gop && !base_video )
             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
     }
 
+    /*
+     * Check if a proper configuration is provided to start Xen:
+     *  - Dom0 specified (minimum required)
+     *  - Dom0 and DomU(s) specified
+     *  - DomU(s) specified
+     */
+    if ( !efi_arch_check_dom0less_boot(dir_handle) && !kernel.addr )
+        blexit(L"No Dom0 kernel image specified.");
+
+    dir_handle->Close(dir_handle);
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:22:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:22:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192835.343506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT395-0006dC-MO; Wed, 22 Sep 2021 14:22:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192835.343506; Wed, 22 Sep 2021 14:22:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT395-0006d5-JF; Wed, 22 Sep 2021 14:22:47 +0000
Received: by outflank-mailman (input) for mailman id 192835;
 Wed, 22 Sep 2021 14:22:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT393-0006bB-WF
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:22:46 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 546ea68c-4140-4c6c-990d-8625e408108f;
 Wed, 22 Sep 2021 14:22:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 546ea68c-4140-4c6c-990d-8625e408108f
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632320565;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=mEtagbVpJQVQH77X7f2fhBXuMNCzR+6xZPo0qRVCWWI=;
  b=K8KYI9t/C6dlfRbSAmJEj2uLxG0+jXZ8dShmo5RTd3ttZSWTfsztv2P7
   K6lzGVBCFtFhfvXOU/fWmoqXFXho864293TBBWHxUuUxyyoTYIFxqXVxj
   WvYa4vmB0R/qgfZHQxldLvdmdRShV55fX/D6RHDZbQHqMaLk+vJcw7Hxm
   U=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: iC38JYbCCpONcPRKe+ToDTKqPsLWITHNeyo9VOIBgM4lNyt2Ra4HNfSPUl/EEiLJowQFlmI6Vf
 qlJVpzcOVKzHTNaYa1PIMsgglBwSaBvO18xyWvefgq62l2udj51sdInCLA9G6ZapLHzGPh29Gp
 5CNBYkiBodVgUTS7YCAnw9ja1VoNDoXwe9ylEYzlM/wmgOEHR7g2NW+Nu6M4yF3osMUmVLEPHJ
 YV+WigJe2CkJr53OM5DYx4PQPXe3GrY+zPPONSWDEqQZZwncC7Vw7LJSZR/84i9tkzbNL8vRjE
 Dt3PFi47yTQIFtURTWuSTmxt
X-SBRS: 5.1
X-MesageID: 52928283
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:o4SUY6kyDvTtQCiSkcix8MLo5gyrIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIfUT+BPfrYNzCkLo0kaYWz9x4CvpXSmINjHlQ//y00FiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qy2YPR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NtKh8auFAN0B6jnwM4QCD1fVA94ZoQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpq15oeR6aFD
 yYfQRRlZRnJShljA2wGF8gmu86DvCClUBQN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wt
 m/Aumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3ntZoilCOwm0YCDUNSEC25/K+jyaDt8l3c
 hJOvHB09O5rqRLtHoKVswCETGCssBkuAcQNF7AD7SrV4KePxwHeKG5fZ2sUADA5j/PaVQDGx
 3fQwYizXGwx7OXMIZ6O3uzL9mLpYED5OUdHPHVdHFVfu7EPtalu1kqnczp1LEKiYjQZ8xnLy
 jaWpWAVg7wJhKbnPI3qoAia31pASnXPJzPZBzk7vEr+tWuVh6b/PuREDGQ3Ct4ad+6koqGp5
 iRspiRnxLlm4WuxeMmxrAIlR+vB2hp4GGeE3Q4H82cJrmzwk5JcQWygyG4nfxo4Wir1URTof
 FXSqWtsCGx7ZSDxBZKbl7mZUpxwpYC5TIyNfqmNMrJmP8ggHCfarXoGTRPBgAjQfL0EzPhX1
 WGzKp33Ux73yM1PkVKLegvq+eV6nnxhnTyDG8+TItbO+eP2WUN5gIwtaTOmRus48LmFsEPS9
 dNePNGN0BJRTKv1ZSy/zGLZBQ5iwaETCc+kpspJWPSEJwY6SmgtB+WImeEqepB/nrQTneDNp
 ynvVkhdwVv5pHvGNQTVNSwzNOKxBc5y/SAhICghHVe0wHx/M4yh27gSKsksdr49+e09kfMtF
 6sZe9+NC+hkQyjc/2hPdoH0qYFvLUz5hQ+HMye/TiI4eppsG17A9tP+J1O9/ygSFCun88A5p
 uT4hA/cRJMCQSVkDdrXN63znw/g4yBFlbsrDUXSI9RVdEH9y6RQKnT83q0tPsUBCRTf3T/Gh
 QyYNggV+LvWqIgv/diX2a3d99W1E/FzF1ZxFnXA6erkLjHT+2euzNMSUOuMejyBBmr49L/7O
 LdQxvD4dvYGgExLo8x3FLMylfAy4N7mprl7yAV4HSqUMwT3W+04enTWj9NSsqBtx6NCvVrkU
 02CzdBWJLGVNZ63C1UWPgckMryO2Px8duM+NhjpzJEWPBNKwYc=
IronPort-HdrOrdr: A9a23:fCWyIaAuRJS1K6TlHeg3sceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHP9OkPIs1NKZMjUO11HYTr2KgbGSpgEIXheOi9K1tp
 0QDZSWaueAdGSS5PySiGLTc6dCsai6GeKT9J/jJh9WPH5XgspbnmFE42igYylLrF4sP+tEKH
 PQ3LsMmxOQPVAsKuirDHgMWObO4/XNiZLdeBYDQzoq8hOHgz+E4KPzV0Hw5GZTbxp/hZMZtU
 TVmQ3w4auu99m91x/nzmfWq7BbgsHoxNdvDNGFzuIVNjLvoAC1Y5kJYczOgBkF5MWUrHo6mt
 jFpBkte+x19nPqZ2mw5SDg3gHxuQxeokPK+Bu9uz/OsMb5TDU1B45qnoRCaCbU7EImoZVVzL
 9L93jxjesXMTrw2ADGo/TYXRBjkUS55VA4l/QIsnBZWYwCLJdMsI0k+l9PGptoJlO11GkeKp
 gsMCjg3ocVTbvDBEqp+1WHgebcEUjbJy32AnTr4aeuonxrdHMQ9Tpv+CVQpAZGyHqRI6M0kt
 gsCZ4Y542mePVmGp6VNN1xMfdfNVa9My4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="52928283"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TQH9wp+/01aJc0xU61pz2tf+8pgcaQOxiE0iR0OHK2c1Vlqe6SGkcFoRreEvLkgJN8p4Lx3gyoxq5r0wsVq8x2a6J6XHiGSQd+ycLvZSAosEZmhv0xmJOLLGmtQDwrWVR/y8UJQ0EUuujuAYJvVFbm+00dBM4AqL/zH8LxdPHLfeD5IBaSGztTeUPPxNnTY7o19JNnvsiI94xUefmZiLXcyNLMUl2IpYrnKCGAJVC26CchmcN9QIC37iEzHHBRhVoHl28Bi0K9V8lHGdPFJp9keekSAAzagn3yXFg6qthTT6Ha9Qt9LtnvBgp7hbhRvs254zcv2T0fuGxTEaq9I+lA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rKTVIxbFozN0phUYlQnP1UxYWuvKfzgxgNVEXYOt3tY=;
 b=fv1aM1IAvWI5Li1rr64zShKxqqBmRKOSCpoIEbeEjTArZOebBGnghJiIh4cOSxfGFK88aPzRnsw3qrA4x6xlQ5gZDw/iWATWZdc3EBxDHtriZbC0VAJstgfEW1oZvYhUE6hhEDrF3CWG0t6kizzUTyIyDEmgaIuFkgQJXpUzFbyR5XDdExszFlwaAkEWuLot0x7VeBmLJ+hFMS6OX5Oz/w5+FCMcAldCKrsGnCWRJMfhTwP7Sge8e3VIN0SoH8Jk9qdOdpsRbw70hYmrqHEkDH/IX/gWDMLF3jHsqI08KDdp1CgGVJPAAacim2MdpP3bL/kQGiNlL334lm6fF36sXg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rKTVIxbFozN0phUYlQnP1UxYWuvKfzgxgNVEXYOt3tY=;
 b=TefVAtnhwNIu/qVU194L/u6TpNbTd0Fhq0bWpUVb/jMendc8kd1yAq5qbsZEaixkCZLEYz7T3SA/qgBjR0dn8gmDOrWbTnbMC17PQwHuZLZYYw1HPfJQQNmS0azMB9hvUUuzMVd1MPgxy9yuidnDGIRnrizAICdGVm7sTBZMrJQ=
Date: Wed, 22 Sep 2021 16:22:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 3/9] x86/PVH: permit more physdevop-s to be used by
 Dom0
Message-ID: <YUs8Jq4mei9+DDCU@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <78c84053-3d79-b36d-be03-08b3b3a3d26c@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <78c84053-3d79-b36d-be03-08b3b3a3d26c@suse.com>
X-ClientProxiedBy: LO4P123CA0339.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a6290f1-b750-4ca4-e44e-08d97dd46d19
X-MS-TrafficTypeDiagnostic: DS7PR03MB5525:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5525800F00F24363272B98A38FA29@DS7PR03MB5525.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 5BZ6ymswYWJzGnE7y2QBm5qQqcigic2WJLw7qj55bUnt5bP5MLEh5zNGzji61CctOeUaEzrWCbo5AER3Xu1Pm4VJljYQtmRUtgSenoLIIee40zmFQgEpOxZazl9FulJHYxTv+7ih3IjY3HMMrveKCHOt3n5NdAZK9baMHjwwPdiqM5MHkNvJTVPr7gxu11qgmtAUu80y+H/egJ9+Y57GU0/by6cdU6oGnc7ATmA3IUIZYROtNVzhqeo315hu/RMgJ9zZYOhZB8bByV829mcm1Ld3keB3F2iZxzX1ByNq6dkH73zqCXMBh2hTj1cm9QiIWwqlWd4zq7Vj3+vIhNjGDqqYqE1aGQIHeseAa93Kl9KDlSIJ3X6NzpXpgKoDunE4yYucSBxlfyLJ0i/US6qHzDDJcI70uYy2crsrbu7+GcgedXmqeM9jTbTa+6tW0kGzZATl3X5ErPPSieG9qY57dYfM673A4RZ1YPNFNx5UCjZgF8HtZV0Lxr88rhNF75IJPQMdanr13uEb53MhpBKA9+X8ZHfYsvy7JOFlHCXm3bDLB+nPNA4b+XTzj7C9jFvmjsI01XKmj183xClZX3DfeYSRUy3ZAwozJXqokdiT1NUPAn2uchb/HNle5KmWJOWefb30mGatVPOOrDZ8W162p1S//A0O5PzOclsvdY8TBtVDcX/GbtQtj0Op30Prv4Yo3HsOqZt2GsOh6PAHfZN3MA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(66946007)(66476007)(86362001)(9686003)(8936002)(83380400001)(6486002)(6666004)(2906002)(8676002)(316002)(85182001)(6916009)(54906003)(508600001)(6496006)(956004)(26005)(66556008)(4326008)(5660300002)(186003)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NFlvUnhGVjlVc1BjZkV3ell2ckU2ZWhDTzlmL1JZYzFqQUpPOHpTRTI3YlE1?=
 =?utf-8?B?TXFQT1RzZlVPaVJSOGdOa3RyajIvckVRNTdBNU9UOG1BdDlpRXJ3bFNzZFlU?=
 =?utf-8?B?Q2VhSlVzZTJUcEhpU01KK2I3Q1I1TXNmaU9VVTJ5MmFIN0xLcDYwWCtEOW5j?=
 =?utf-8?B?dVc1ZnRxekM4d3E5cUJhdjMwWWRxbXBicDRPZUJKYVR1QlZGUDlxNU5pZXhs?=
 =?utf-8?B?bFRmQVBiNCtEVnZtODFrblZyR2JEdXRiMDhqMCtSL1FHdkZCOXRCRVNRd1Mz?=
 =?utf-8?B?ZTVaV2RJQklTbGZHWmRpbW9rSTJ0MVowVFYySDVLekZoZXFtR2ppSjQwT3BQ?=
 =?utf-8?B?SGxwb29XL0Y0MHlBS1dON0E1UUs2TTB0ZFlNS0dTRmRSY2N5Q3c1NG1ZRHRR?=
 =?utf-8?B?N3J5UFBnRWszR3lZeU9oTHo4Vk1QK3VTQUpURDdUTHFZbUN6czBIam5KYllh?=
 =?utf-8?B?VEFSZWlWL3lpMENvZEhtRWpZYWNtdFZaRndoaXlqNGM3d0VmM1JyR25zMmQ3?=
 =?utf-8?B?aXZXNzBJUU0xNnRZSTRTNHA0aWVQVWo1V0J2TXF3U2FBdlFjdE5pNWoxaHg4?=
 =?utf-8?B?aE9kRG1OMlVKRzROVVEya2lFaUxXclJxOWVzQmFSZ0VoS25FWmhyUzlVT3Jz?=
 =?utf-8?B?VTdPSlEwMHM1VVg0aGEvbnpNY1NVczhIVkM0cWZjR2RvMkVJKzhKQi8yYmZ0?=
 =?utf-8?B?L2k5MHBRNngzaWlXdC8yeFhUVU1EVFYrSERHdmdDa2R5TDRURTB0VW5GWkxJ?=
 =?utf-8?B?YWthckFNblBnYS9MMk0xSVduR1lmbC81eWhPLzRZZ0xON2hHZko5Z0VtaVM5?=
 =?utf-8?B?K3d2MUU0cEsyL0orckJ3Z2U5TUNhMEVIS0JURE81bU5GM0krTHhnelBxcmpI?=
 =?utf-8?B?dThwVWpVRWNUMVhSTXFoT2VFYkdNRWRvdEZnYzNJL0IvakU3QUxBMlp2VFJv?=
 =?utf-8?B?Vitpai9XZFNtL1RkSll3d2tXMjgyQmJ1TlhsbGVDbU96RGZMVE9PemdXa0tn?=
 =?utf-8?B?SGpERUdlTVVQMEJ1a3JpMnVEeXdRTHlUTk04V1NsUldUMmZjdm5zKzlFWWlL?=
 =?utf-8?B?dzd2L3BhWml1dWQwbERuSmJoNzk3WmFzNUNkZTQybGFpa1RYa1AwUjZoUHBJ?=
 =?utf-8?B?OHJMMXAyaFlBdjB4RGNFcVFJcnl6ejFUS0pJTkszSHFDSkI0SndadWhPeVp6?=
 =?utf-8?B?aS9qeXM1WVkwOXdSMi9sTWpPTlN4VEQwRW1WemJxVnJJak01dlVNY3JWWG5N?=
 =?utf-8?B?S0JvcFkyckJDaHhwQjZtNHBzTjQ2VU1leUtLa1NpNGJscVE4anJhQkhLekIy?=
 =?utf-8?B?eGdUNVAveTlYc1dkY0dDaE9RTWI5SkZpc1M1V0IzN1NhTmo3UzA1ZWtpalJv?=
 =?utf-8?B?c21NWld5aUZPaHRpMC9BS0NndWlxaGpYVTRJSldybVFtWG5QbUZnbDc3WXB0?=
 =?utf-8?B?VmNMc0RpdHdSSlB6MW9BYUhJTEM1cnEraFFiMWFvcUxDSFFESk03R0lwZHov?=
 =?utf-8?B?akJKbVdkRE85MU9tbjVCL3BEczZCNGt2Y2NmTDZGR1ZwZ3lJcTU5Z3FYTjlX?=
 =?utf-8?B?MHh1aWE0dmIvMmg2SkdBTXJrOW8yaUUxeHpabEF3Q21mWkFiZDlLZTVWc3Ur?=
 =?utf-8?B?RTB6elhFWExOU3FmZkYyZTJCbnZNNE5SM2IxZ0NVTlBKaFRBMSs2Z3B1NVly?=
 =?utf-8?B?enFaRm5aOVl2M1FyMjRoeVJNZm1rMTlJZFNHUDF5M3RGRDlWK0FmU1YyZnIx?=
 =?utf-8?Q?M+r/PUx7bmNpetdKUEtZJojpI0wO6hkTleXVjUh?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a6290f1-b750-4ca4-e44e-08d97dd46d19
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:22:36.8838
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Jr7EJ4WesDAp3P0vd4q+9tzEtHI5MSrhJUcFrFlNkwCISuuEfoab5y1EC/nrz7hNJMdqXfV3h+BIivgESoEz3Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5525
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:17:37AM +0200, Jan Beulich wrote:
> Certain notifications of Dom0 to Xen are independent of the mode Dom0 is
> running in. Permit further PCI related ones (only their modern forms).
> Also include the USB2 debug port operation at this occasion.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I'm uncertain about the has_vpci() part of the check: I would think
> is_hardware_domain() is both sufficient and concise. Without vPCI a PVH
> Dom0 won't see any PCI devices in the first place (and hence would
> effectively be non-functioning). Dropping this would in particular make
> PHYSDEVOP_dbgp_op better fit in the mix.
> ---
> v3: New.
> 
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -94,6 +94,12 @@ static long hvm_physdev_op(int cmd, XEN_
>          break;
>  
>      case PHYSDEVOP_pci_mmcfg_reserved:
> +    case PHYSDEVOP_pci_device_add:
> +    case PHYSDEVOP_pci_device_remove:
> +    case PHYSDEVOP_restore_msi_ext:

Hm, I'm slightly unsure we need the restore operation. Wouldn't it be
better to just reset all device state on suspend and then let dom0
restore it's state as it does on native?

Maybe there's some wrinkle that prevents that from working properly.

> +    case PHYSDEVOP_dbgp_op:
> +    case PHYSDEVOP_prepare_msix:
> +    case PHYSDEVOP_release_msix:

Albeit I think those two operations won't strictly conflict with vPCI
usage (as they require no MSIX entries to be activ) I still wonder
whether we will end up needing them on a PVH dom0. They are used by
pciback and it's not yet clear how we will end up using pciback on a
PVH dom0, hence I would prefer if we could leave them out until
strictly required.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:25:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:25:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192843.343525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Bw-0007PP-7Y; Wed, 22 Sep 2021 14:25:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192843.343525; Wed, 22 Sep 2021 14:25:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Bw-0007PI-4S; Wed, 22 Sep 2021 14:25:44 +0000
Received: by outflank-mailman (input) for mailman id 192843;
 Wed, 22 Sep 2021 14:25:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT3Bu-0007PB-8J
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:25:42 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f6987e74-1bb0-11ec-b99c-12813bfff9fa;
 Wed, 22 Sep 2021 14:25:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6987e74-1bb0-11ec-b99c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632320741;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=mAvN3MvZGLtEfzpBndf/tMrJ5CEJZjjWPXJeTRw9xEc=;
  b=PqP9jG3tiBWfxQHoATZmDcYX9cpvdkhpqiveYXzJV40B6khq5mOjy6pu
   tSB+EovuSUX6HNVLEqwgTqLzDKjgY/gcb9BT3YnYotBWXKd1OqmdHrTkU
   QaRi13BwtUOHsihkMoTigu16GOo/aOLr8MKcW/rwTLTENkmwtH69ykNAi
   Q=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: s1IPJNiHDDrx9cLO9DTU8tY7ZohhHHOzobLNRBMxzDRdlSw0BjrjeW0YGogQDvMLOUbz4IpGPN
 MntcdJgb31Px5fbbWT9bNNPtZmN+naPhxJP0tm19a+PwSYRQJGmRgkTLjBjBe0AXWD1YTZk9Nt
 8eYIHtDk53xED0g88QbuTXtUxM2ZfBI+5pKiT1OeWr+rcSZq1b6C8oRbF2RKbp3ct53qesteKt
 4WoRLuQiS97DjUMDG14VKcBJp0pcFUYkR7h7w7qR1oyCUUNtEELZF1C7mhVDPstkHIY1xT8yeQ
 xujiozI2hVwNBnadgMrJY5TQ
X-SBRS: 5.1
X-MesageID: 53322525
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:cyW5BaI+lDUd9GiOFE+R6JIlxSXFcZb7ZxGr2PjKsXjdYENSgmFWn
 DccXm3SPKmJZmr0eI1zb4Tl9R4Fv8DVzdBrSlRlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wbZi6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2zn/xf8
 clE5aCdQCgPYaiStcseCSRxRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2XvoQChmpq26iiG97FW
 cQlSgVmXS7wbjFyNFkpNLxng+i30yyXnzpw9wvO+PtfD3Lo5BBw1ZD9PdyTfcaFLe1Fk0Ddq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PL+y++NugVaT7ncOExBQXly+ydGmjmauVtQZL
 FYbkhfCtoBrqhbtFIOkGUTl/jjU5XbwRua8DcUA6C2j87bdszq0D0ciETsfRfMl7vIfEGlCO
 kCyoz/5OdB+mOTLEijEpunI9Wra1Ts9djBZNHRdJecRy5y6+thi0EiXJjp2OPPt1rXI9SfML
 ydmRcTUr44ai9ICn46/9ErO695HjsmUFlNpjuk7s2TM0++YWGJHT9fzgbQ4xawZRGp8crVml
 CJf8/VyFMhUUfmweNWlGY3h5o1FAspp1xWH2zZS82QJrWzxqxZPg6gJuGwWyLhV3jYsJmayP
 R67VfJ5z55PJnq6BZKbkKroUJ9C8EQULvy8Dqq8RoMXOvBZLVbblAkzNR/49z28yyAEzPBgU
 ap3hO7xVB72/4w8l2HoLwrcuJd2rh0DKZT7H8Gjk0v/jurAOxZ4i94taTOzUwzw14vdyC39+
 NdDLcqajRJZVez1eC7M9oAPa1sNKBAG6Vre8aS7r8aPfVhrHn8PEfjUze9zcoBphf0NxOzJ4
 mu8SglTz1+m3S/LLgCDa3ZCbrLzXMkg8SJnbHJ0ZVv4iWI+ZYuP7bsEc8dld7cQ6+E+n+V/S
 OMIepvcD60XGCjH4TkUcbL0sJdmKEawnQuLMif8OGo/cpdsShbn4NjhegezpiACAjDu7Zk1o
 qG61xOdSp0GHlwwAMHTYfOp7lWwoXlCx74iAxqWeoFeIRy+/pJrJir9iu4MD/sNcRiTlCGH0
 wu2AAsDobWfqYEC79SU17uPqJ2kErUiExMCTXXb97u/KQLT4nGnnd1bSO+NcD3QCDH09aGla
 bkHxv3wKqRazlNDso46GLd316MuodDoouYCnAhjGXzKaXWtC69hfSbajZUe6PUVy+8LoxayV
 2KO5sJeaOeAN87SGVIMIBYoM7aY3vYOlziOtfk4LS0WPsOsEGZrhamKAySxtQ==
IronPort-HdrOrdr: A9a23:sZjylqiI0IeDYUEnsWFeKC7HW3BQX0h13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskqKdxbNhR4tKPTOWw1dASbsN0WKM+UyDJ8STzJ856U
 4kSdkCNDSSNykFsS+Z2njALz9I+rDum8rJ9ISuvkuFDzsaE52Ihz0JdTpzeXcGIjWua6BJcK
 Z1saF81kadkDksH4mG7j5vZZm3m/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTtj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZtA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQ/ZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv1nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLaU5nghBgs/DWQZAV3Iv/fKXJy/vB9kgIm0kyR9nFoh/D2xRw7hdUAo5ot3Z
 WMDk0nrsAJcie6BZgNcNvpevHHeVAldyi8Rl56EW6XZJ3vBEi92aIfwI9Fr91CK6Z4hqfbpv
 z6ISRlXCgJChvTNfE=
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="53322525"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EXWqVtxN4NynKoY2ZrdAo/KULQBHGFWQTD5u2XPRqSVsj1LU9weogo0thUkrnHDz48VtenUBOpbpkW8lVJrm9n95lD3nwrRpte1au6srjZY6NTwH/9az29W5/NYlO1rppPJGtyOdtcAJYhtP6LUBzlRCVzH8ilpbcL5kmvRAXnZZAzF/Eoip3/EqtS4rlb3YB3DI41TfeVsHfuUX6CF+3HViUGGmclceQbnt4GbE2oWRLiGhBZfzimlPkSKaHkQBmMtkcbau04nv1TQt8p53H/MauNCQU1l8U0seSCK6kgpGSht74LzBgey2Vrte1W5kjYxkWvY0ZiCEKvGTqL9Xtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qaDVag6CDTsAPeUnT5m77AzpkBzJ1YXqzl4R5UjmxmY=;
 b=aDU7mkS1c3wkbdBfKOCEnMjgzD3uVIuwYsYr4Q88QDzFyJacEPFMQtfXl/kSt3iV2u5g0/HPkntdNfKqXpC+bwgk0HZHQQcXm1oHtRRQPyDyDdzNspYBA/rVOnJeiic9eqdsrhOCznXFvkH7vh95hkYNBjfZXg8Nu0CLqjHEeYXCcZIXdVgBONDbD4tQaBjOZLlBYduS9lPXfPnJYQ8hUetssYaPl2FYAhOZvBEopoqslXuOOzgvp3mx0DCuDDwL86utY/B5U5IKXv8dLj9H9UoXSGHeiUhnKW9IJPgoqP2AGNl10LBtDnRzL3suYw4Rgikpj2GBY65uTyksOuwjTg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qaDVag6CDTsAPeUnT5m77AzpkBzJ1YXqzl4R5UjmxmY=;
 b=gPtsaQqOM2WeWY8H4SiIiIH4Plzmzy0/Qkc6i7P/OtYx5uJ+3l/2xXo0b/Hdpl7LOvj5TZFm+DJSU83Mt/kcM60snXlx+ZlgnzuOwYjuRzbM+NGjzJcKXqMu+f7rMTZJS/vYiWdKnLKOs1bqJKCbl8g/teOcTW0Uh9oR0et6Lbc=
Date: Wed, 22 Sep 2021 16:25:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, Tim Deegan <tim@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
Message-ID: <YUs82i7S0fd8r1uX@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
 <b2a2fd30-225b-2b79-f160-8ed2e4ad439f@citrix.com>
 <8c1ab98a-8fde-3d9e-5250-c5a40af70453@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <8c1ab98a-8fde-3d9e-5250-c5a40af70453@suse.com>
X-ClientProxiedBy: LO4P123CA0100.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:191::15) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7826b663-48eb-4612-daef-08d97dd4d73e
X-MS-TrafficTypeDiagnostic: DM5PR03MB2971:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2971397D5331E0223E5B54FB8FA29@DM5PR03MB2971.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: eSWDmI0MMdA49WU1nOeA+smL9vcRD/zYEIMa+S7Lj0WjKRGTm2fl5stnhRyhzI34F6aSlFRkU1PqZMuh7JWoi16xeGZF2il+gW4Rd+Q9W8mOIfXSLMyIJeO81qK0zdTwJB4L4y8+8FZEeKGOP1I/zCc2A93DjF2Tx0saArDu6e2e8iWSez9bvjF1N0XKjWvTkPzP6y1zJONOs6uwXeHMb9sucXs8alBUxpJBw8ptQNAjYeKV112AUZ/KVX+ADK/8QivICQO5qwVbt6s3g4YzaRZYHh1lClXikhlN+bnyjeyZuVpMb35dMlLQ7siyVoTJDhGRpWdJVgpafsjT5OQKESlL9n2AAWpu6JHykvEnO2ZLyhve/7Nd8rzpoHzqWuqD21QuvbTUhFcRI/RBrJJT0kUkfUBBD66uMjcyvofLGsTNDV4hCEfyinQv0ybTpGvgCawq+yedqSOc/rM1XYhMDDp3VRk56txioxFGCp3rCn94+ycF9DKRgyqkL2q7Dk4eQzKkDJkpRzCK52nSPxkm/pvBftN7lglVO2rgsvHn4sGA3SMQoqMjgXBr63Lw6I16PVUvxemH/QAGuP+N5YOSFcY4XTbBeRdYRJbH9YPadO3Nd0zS6f8Xg9NbUnnOlZ9w3fAiRA6o7MRkPKlPUa3wElq/NUy6RRhB4uuxQ1aoeqhKA+T6ent5mb2Kw7dCMt3Cf432zNxQX6H5qsQQJBR+KA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(83380400001)(6666004)(2906002)(53546011)(4326008)(38100700002)(186003)(6496006)(5660300002)(85182001)(8676002)(508600001)(6486002)(8936002)(316002)(9686003)(110136005)(54906003)(66476007)(66556008)(86362001)(6636002)(26005)(66946007)(473944003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U1BOTG9pNGUwTGVtSDNMMlFXaGloOFVUTkhPRS96NkhPejFJTjNZdWtJMGtF?=
 =?utf-8?B?cEJNbWpZcWxYeHYrOUlLaFNDbkU1KzFPZDIrWlgwQXhhSmhISEI5RlFNMWtL?=
 =?utf-8?B?OFdQMHRGbVJqK2JBNnJXTHo1enpyYUk0azA1SVNZZUZPQjYwejIzandqMGRa?=
 =?utf-8?B?enhSVVFMSzdKTDRTbkx0ek40am16ZlNmam52UDVjWFNtZER0Q0VwMVo0Vm9B?=
 =?utf-8?B?MnRHaG02bjY1d3lUaVlZdm9FUDdDMi8xWVh3MVd2TENOWUxzZEc5TStPbkNy?=
 =?utf-8?B?K3pjOTFQbXdmSjd2b0M2ZW9PbGlGY2lKTWJ2Zk05aEhSVjQ2WW9nTEFVRXFS?=
 =?utf-8?B?b1J2ZGFFT0Z4RzdUanZGWVBOZkw1Smt5Q1MvRHRhSGRsczNhUlkvMjZzQlhE?=
 =?utf-8?B?UXFDWGhXTGlzRVg1TEwvMWhEcHBTejlXcTFKUlFhb3hNYnlWejFwT2Z4Mnlt?=
 =?utf-8?B?V0EvR2EvMnRxYkVGaVVPRWF2bXhPb04vWXhQSDFqTnlNWUdmZnRGcDN4YTkz?=
 =?utf-8?B?RE5tUnFaalUyL2M3R0ZiSTVlaVZHUXVzUGJ1TUpiYTRyclFUTlMraWpwRm9X?=
 =?utf-8?B?aEY5VTdyQm8zcXFRZkFsaUhROW90NVBWSmtpOXRWMVFGcXdjWEVtWXNrVi9W?=
 =?utf-8?B?akdBOUIzOXNIYklMUjJxTTd4bG9QcndDeGFZOUxObThickVNcUtwNjZpMXE5?=
 =?utf-8?B?L3JWWkxnV2N6UFUzR1ZLV3ppM0lKamxlNENLZlJ6YXVIbXlGbnJCeTV5UUt4?=
 =?utf-8?B?Yjk3L0VnQ3BtMWwva25sbkRVYWdpTmVFc1QvY0dRRWtyYjNoSElWV0xRVGpT?=
 =?utf-8?B?Qkp5aUM0TmJwMnJwbG8rN3pBQzhZV0tZczAxZzVTZHRRUG9tM3RjNHZJOEFL?=
 =?utf-8?B?WUZaU1p1Vm45bG40RmExUFJHYTVCQ3RVNWdoVnhmWWdDRTJmWWIvb2FqaVZv?=
 =?utf-8?B?M3l0WkVSNFBReGxHVTZRd2VWZkJoRkk0RjI5bXZobjZMeFdnMU1RdkVjN3Z5?=
 =?utf-8?B?Y2tOZ2ZMMDZSTXZ5bkx4S0hhT2NjSjE4MDF5dXFQM0JrZXBCc0krcmFnamhP?=
 =?utf-8?B?d1FTTEU0L2thZHVMcDQ3bDZmNHF4dDBHUHNmVTlVNTM5UjhUSVJyL3ZLaG1p?=
 =?utf-8?B?TGlkN3ZIamVYSzFOOGdpS1ZlamtKa3ZweG9kUisxbVdDdTZlc2VlcE5vOXcz?=
 =?utf-8?B?UWFRaU9qMktnNzlvS0g2Q1BaM0h0UXJkMXVzM2U1dFJFNjNCcVRZK2FLSDhp?=
 =?utf-8?B?SElvY3FrTUdHUVZaYmNIeXFnNkFPZVlBV3gwN0JFUUdGQlZYUTR4ajlhL0J5?=
 =?utf-8?B?RVB2bm5WaTRDeHBJMkxkQWtSYlVEOVE1cWZ2cForaVVNRU9FUUUrZHRqbzYx?=
 =?utf-8?B?L3FuM2xqdUlrdFFOQkU5UXhOZ2pYbGdkU0NMWHlObnRNQnJ3TGpMd2h1THd3?=
 =?utf-8?B?djVzZUZudlFOVTFEUElPTjhXL3FVUDFLQW1UL0RiNUNYWGsxRjZua3poSHBV?=
 =?utf-8?B?cE8vN0VGT2R0bFgyQ2NxS3BQVXRaa0V3MmhVck1zeDJKcnZUMnFYOTdlb2Jq?=
 =?utf-8?B?Q1dOVlRSTmNJbTdMaThmRnV4R1htcy85anV0S0VLcHVnT1crMkRWaFZ5WWVi?=
 =?utf-8?B?WkQrN0V1OVY5OUt2bXJkMVFobFZGYmgyU1J4VDQzcjhTZ2NqNkJoaTFYN3Zm?=
 =?utf-8?B?Z3pxQkRRbVk4eXRXbWhHc3N5V0RPV216S1JJTU1EeGtHaUhtNHhaa2wyaHVW?=
 =?utf-8?Q?65lOEuTQFl7swpTFmidCS7fTG6qjLjO8Wl8la9j?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7826b663-48eb-4612-daef-08d97dd4d73e
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:25:34.9585
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: o5TNqALlWl8irXftYBzZdoK3HJ/OOmx+Ps9nc1QSby1IiNG+adC5Z04/80J0eUXc69dBnX0nfGsHESBlUv9RTg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2971
X-OriginatorOrg: citrix.com

On Wed, Sep 22, 2021 at 03:50:25PM +0200, Jan Beulich wrote:
> On 22.09.2021 15:31, Andrew Cooper wrote:
> > On 21/09/2021 08:17, Jan Beulich wrote:
> >> @@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
> >>  #ifdef CONFIG_SHADOW_PAGING
> >>      if ( opt_dom0_shadow )
> >>      {
> >> +        bool preempted;
> >> +
> >>          printk("Switching dom0 to using shadow paging\n");
> >> +
> >> +        nr_pt_pages = dom0_paging_pages(d, nr_pages);
> >> +
> >> +        do {
> >> +            preempted = false;
> >> +            shadow_set_allocation(d, nr_pt_pages, &preempted);
> >> +            process_pending_softirqs();
> >> +        } while ( preempted );
> > 
> > This is still broken.
> > 
> > The loop setting the shadow allocation needs to be outside of this
> > conditional, because it is not related to early activation of the l1tf
> > tasklet.
> 
> Well, I'm not sure what to say. On v1 you already said so. But then you
> didn't care to reply to me responding:
> 
> "Are you suggesting to set up a (perhaps large) shadow pool just in
>  case we need to enable shadow mode on Dom0? And all of this memory
>  to then remain unused in the majority of cases?
> 
>  Plus even if so, I'd view this as a 2nd, independent step, largely
>  orthogonal to the handling of "dom0=shadow". If somebody really
>  wanted that, I think this should be driven by an explicit setting
>  of the shadow pool size, indicating the admin is willing to waste
>  the memory.

Maybe an acceptable compromise would be to allocate the pool if
opt_dom0_shadow || opt_pv_l1tf_hwdom?

opt_pv_l1tf_hwdom is not enabled by default, so an admin opting to
enable it should also be willing to reserve the memory it requires in
case it needs activating during runtime.

Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:29:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:29:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192849.343536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3F1-0008Ez-NQ; Wed, 22 Sep 2021 14:28:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192849.343536; Wed, 22 Sep 2021 14:28:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3F1-0008Es-KM; Wed, 22 Sep 2021 14:28:55 +0000
Received: by outflank-mailman (input) for mailman id 192849;
 Wed, 22 Sep 2021 14:28:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3F0-0008Em-EA
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:28:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 68a0cfe4-1bb1-11ec-b99c-12813bfff9fa;
 Wed, 22 Sep 2021 14:28:52 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-ZqTC2YS4PS-a4ce0LXXQZA-1; Wed, 22 Sep 2021 16:28:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6672.eurprd04.prod.outlook.com (2603:10a6:803:127::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 14:28:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:28:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM8P191CA0016.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:28:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68a0cfe4-1bb1-11ec-b99c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632320931;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cVE+9S/hRl9E3YbcJWPwZcWaGRqaBlv07SKEMwIJtM8=;
	b=gKac/pnYXD4bkBpy6AKiMuI+wyM8j+eMMfVVvn+kI4QsFRMl/PO60iXSLImOy925KkPBHS
	HlJGRQPd2WxTzR3fWLDv8ejNqobKcaA0QU28BGoUQGzbrMYieKds1qckvaWqJRWVBKz4K+
	Z0J+gSvqftZurChaSsnbw1uuXS7kN74=
X-MC-Unique: ZqTC2YS4PS-a4ce0LXXQZA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KuWgZBbYNQrZxonekWRi1lLS6coj18ejGkjfTTqvMcpc8Kq9Waszfctwu0JO2wo3ry4rWBaxu5YcYaVspB5CBzfsTvemVe09lzxzGpOJJs4kUQldN1CapevoySWPSPVE7AkY/qicSCcNYOvOmIUOAYz/RTrYJaCQegV4vd92Fers6uPFeWHJyZey0UU7BEQCBpxtBMnEBnLKmZyGLGc/gYOEBps/P/8dC8rFoISEKcOzFS1m8M9QmJwN9qGQMfRbSskDXCF74O6SWd39rs0S7Z2zdx+uU/4I+KsrDXP832n9W6MgobHRtf6+YECgE6b+SNBDgQTs2zBzgP5m2DMkNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vlLwpVbKdDGidk/pmbCgqtPtVvafgbou6IhVrWQQ+a8=;
 b=ZXoKzjvzSCkA4UZd6lqtYX2A3BZTGqo9b8VOfyZ1NLxACTi3P4M82bn4WUt2u/TK0GJb9uoydIaXQHAGf/NUo1Dkpyo5PPrUqT6ihOvTQ3m7M7v8q+/alWo2g0nmogEHCiowhF8zHaVJIdOFnICwTAeLgnsn0KGUELkSNPq30OnkhY7JQW4k/d6wAsIvm7uti5SqCvd7bb93D2+Y5kWsIsv4VFjEWIYwB2P5HwZ8v70Gae56hwayLueY0sjqpiU9fb0Kvz6aIG7vIENp2yMxyzp4sXNzi4yhmxb2SUNY9PsHO5dPKHArI8yE7v9rRnjLNYq90aGLYSKm/UUJ1azTKg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 2/9] x86/PV: properly set shadow allocation for Dom0
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wl@xen.org>, Tim Deegan <tim@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <25958e72-40b9-30ef-a348-6ac9ef02b261@suse.com>
 <b2a2fd30-225b-2b79-f160-8ed2e4ad439f@citrix.com>
 <8c1ab98a-8fde-3d9e-5250-c5a40af70453@suse.com>
 <YUs82i7S0fd8r1uX@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a214deb4-be70-bf84-356f-5f6569224c95@suse.com>
Date: Wed, 22 Sep 2021 16:28:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUs82i7S0fd8r1uX@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM8P191CA0016.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:20b:21a::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e04a0599-edf7-4176-334a-08d97dd54adc
X-MS-TrafficTypeDiagnostic: VE1PR04MB6672:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6672010B71DB41AF886755D6B3A29@VE1PR04MB6672.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LFs/bU2wgpu9ueA+TZpmoCQB3UxsV0F2zWW+YLrqv2e0Gh3QtAeORh3e+mcHuJyvjDtIhzxZXQJihsuUcJxPCRlH7vXdAY7dHyinT3+rRtqSoKimwwYwNSX18QVZBrKUfPH9nRwhlA818fRQf896Avt+NptxFWB0Pii3xR5y+jIho3inCYB5LlL+G3Jj+rzn2F3G0bZhGrs3aCewuLQFWjnF2lilO7w+hnLPwOeXUmodj+xWyqFrxQll0EhziBcmwDAGx61WAZKdf7lSXMkW4wajs85agrQpG2+zIf11EjyHlAsOyVvMYkw/eSluw6nc5Eo8LGhgWXmhbpZYVPYASr396pR1nkvoS4cPn6ukEVzb7/12qW4v59mZb5bx6GMnfYhoZRGlArHe6IP7bnik17lZ59oK3MQ631VQXpvDNmHWwLTzOzFwLDBxLCXDhobCcNknwO5z08JsJFj9zEP2xsfuHz3B28E/io18xRR49dANT421EZnpD3WKE9U3/3shzxF/RPIlNeyo5BCWG0QmlLLseDnXKBlrT1D3Z1QdO0E4JSUZqR7Q339khUJzfO0Z854A5VixU5r7ksl7zbP4PhcOyC+tPcm4HP1y3Lpklo7Xzxc/7oBEALhKZ2ryaS3K6o6aVwmKsNAvyQ7m3w/hpXbD321F7NQJ8Rn6B92kQH0d62We39efrX80hb89LSQCV39kl/gGMdunr1WWsQdtAvBvQKyqJF6wR+a0nwTXNlyfRLgMhj53InoCyXJ+kne7
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(66476007)(508600001)(186003)(2906002)(36756003)(4326008)(66556008)(38100700002)(86362001)(8676002)(66946007)(83380400001)(53546011)(956004)(31696002)(2616005)(26005)(110136005)(16576012)(6486002)(31686004)(316002)(54906003)(5660300002)(45980500001)(43740500002)(473944003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?nj0PZIsDFCfX9zeuLLIYoYkK5UeoIiYF/XxCJmfOflHASL1gzwmPN9Cd+/qb?=
 =?us-ascii?Q?DZVE7xVoGM0/Rgzoll4RuLxkeLRbHwi+K5WPRnLlvYq3ssiS9OmdVnlMH7Bq?=
 =?us-ascii?Q?WZz2GOF0QzAW6vUgqDKL59HsLdefwVmL2AUIxiI+izNR9Xp8grLODF9uI0Jj?=
 =?us-ascii?Q?p4BB9gTdy88MA2E1H9ddIYan/AJnub8YIphIH1JQOuf8dnIEUd9/ggCpkiP/?=
 =?us-ascii?Q?vhZ59dHLcO+25j6qOiefcz+feXOllPqi1xWekVFowq4b7xRe65Bnd3RM5ash?=
 =?us-ascii?Q?cbLJlhbktJmz1Bm7mLnbjN6Gb/N8zS2OxnoLWyR2QCFi91Oliipx+h8I2N0I?=
 =?us-ascii?Q?qr9JU6+i4n3fOB8qV87RbGra06yRGkYLi5wPZosEghCFnVKXi1Br+sEEs0kg?=
 =?us-ascii?Q?7aTIjHiTiAhGqIzeomKr0BtQenSKmmreNRB9322LcoBUxApuBmxqR8kJwiPy?=
 =?us-ascii?Q?JS3HQORg3hIcHZVqmKJzNNP5S5oFrvGa+HtSvgH6f3A5KDQrur4st2wbficq?=
 =?us-ascii?Q?+eVMoFnf5ngYIor1RewoefJAJTm9z65fsVDOBuZKRGqMet6+pNJGaz3m1W1x?=
 =?us-ascii?Q?sTrQzE2XRRUR7RH8Ms5xAZdH68DjrmXp4pjdNDnmmn/cVLLQ8/k9IPuahsm8?=
 =?us-ascii?Q?uXa1Wf1TthD8Rj0yhMBg8tKpHSyBf65JsOiD8lB/ynDEaetaHeuZ1l+vZELu?=
 =?us-ascii?Q?ENGnI1G9VlXy3DRhsGYc7v/pmb1BND3l+LJr3NCxJailxB3K5kFNowgp2HUl?=
 =?us-ascii?Q?L3CM0X8sb2qFWzSCT81CgrLrDQbt08jVuEJJ7zuzhOWUrDVowkjhDlPq6wrX?=
 =?us-ascii?Q?PQ3vmuRhtKLo6hvZ68UORDg+wqFEyNl+yWR4e7Cf7vmpM9vYGqUGePUAf+mK?=
 =?us-ascii?Q?bmsITr6qcxJpADNErrPNlSmEzHtevPX1wOmJl4wpQ+0AGkvXpsshZ7dtm6O1?=
 =?us-ascii?Q?RSNaGxtuDSVU1jBO6wdd5J1mgsDQnbql1M1JDp0gpPFlptffvlqWAOlhmhyv?=
 =?us-ascii?Q?G8M06wOoE3zcs21qQz43rqPswjG+sWUwPHUtATeiwXyWOhnx4kwu1/R/q6KE?=
 =?us-ascii?Q?6xxZy7HmmM49+AWGyPtFcDgPRqbFQgZB7i6d1Uz/y3E/j1ye6CwMS08M0rw8?=
 =?us-ascii?Q?oCadOpu5iJZg8pHdXRS/QDNp0D/u5PcJ7t74oGFivS3IjzuiS+p3/gDrweMs?=
 =?us-ascii?Q?xhJq3JQ+sTob/aEeHvTpHwWfUm9ms/4T4fIsU7MEtyETLJVM47IJQiZqGv5l?=
 =?us-ascii?Q?9pKCgSYdh+sXKidOg5IS3mpT6cEzRMc1gq9YA0q+LrEL+rjxeAb6rPu3zIrp?=
 =?us-ascii?Q?BFMN9sdS6ci3zaJBpLbVLk0e?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e04a0599-edf7-4176-334a-08d97dd54adc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:28:48.7776
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 85CxnU0vNhEGFvta34bdUTTye6xPte2vHW5LWhm/wRqYM1cy+X/wEmbcj1CrCvHJ1DXE0/8aExUwzCkfe/la+Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6672

On 22.09.2021 16:25, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 22, 2021 at 03:50:25PM +0200, Jan Beulich wrote:
>> On 22.09.2021 15:31, Andrew Cooper wrote:
>>> On 21/09/2021 08:17, Jan Beulich wrote:
>>>> @@ -933,7 +934,18 @@ int __init dom0_construct_pv(struct doma
>>>>  #ifdef CONFIG_SHADOW_PAGING
>>>>      if ( opt_dom0_shadow )
>>>>      {
>>>> +        bool preempted;
>>>> +
>>>>          printk("Switching dom0 to using shadow paging\n");
>>>> +
>>>> +        nr_pt_pages =3D dom0_paging_pages(d, nr_pages);
>>>> +
>>>> +        do {
>>>> +            preempted =3D false;
>>>> +            shadow_set_allocation(d, nr_pt_pages, &preempted);
>>>> +            process_pending_softirqs();
>>>> +        } while ( preempted );
>>>
>>> This is still broken.
>>>
>>> The loop setting the shadow allocation needs to be outside of this
>>> conditional, because it is not related to early activation of the l1tf
>>> tasklet.
>>
>> Well, I'm not sure what to say. On v1 you already said so. But then you
>> didn't care to reply to me responding:
>>
>> "Are you suggesting to set up a (perhaps large) shadow pool just in
>>  case we need to enable shadow mode on Dom0? And all of this memory
>>  to then remain unused in the majority of cases?
>>
>>  Plus even if so, I'd view this as a 2nd, independent step, largely
>>  orthogonal to the handling of "dom0=3Dshadow". If somebody really
>>  wanted that, I think this should be driven by an explicit setting
>>  of the shadow pool size, indicating the admin is willing to waste
>>  the memory.
>=20
> Maybe an acceptable compromise would be to allocate the pool if
> opt_dom0_shadow || opt_pv_l1tf_hwdom?
>=20
> opt_pv_l1tf_hwdom is not enabled by default, so an admin opting to
> enable it should also be willing to reserve the memory it requires in
> case it needs activating during runtime.

I'd be fine making that change. Andrew - would that be sufficient to
address your concern?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:35:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:35:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192857.343547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3LY-0001Hp-IH; Wed, 22 Sep 2021 14:35:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192857.343547; Wed, 22 Sep 2021 14:35:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3LY-0001Hi-F5; Wed, 22 Sep 2021 14:35:40 +0000
Received: by outflank-mailman (input) for mailman id 192857;
 Wed, 22 Sep 2021 14:35:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3LX-0001Hc-Hs
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:35:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 77d6846e-d7bf-4955-ad5c-c07b5b98755a;
 Wed, 22 Sep 2021 14:35:38 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2057.outbound.protection.outlook.com [104.47.4.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-neNgteJyPzmPOnkWvFKIxw-1;
 Wed, 22 Sep 2021 16:35:36 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:35:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:35:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0072.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 14:35:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77d6846e-d7bf-4955-ad5c-c07b5b98755a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321337;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=PUIBynGRaD4WhYXwQk0I8TmQPV2fkMGcbu5cWr+n9H8=;
	b=G0u5zGv87eEYW+1AzVeR2yPxeEzvtddA1wKQIZcPNb4jQKr3pYz+txSs2d+Pxvi9wqUAkj
	nleKVYbn6tezB1LJ8NPoQwN3ErH8M0jaUcQWzpOxA+ncALguGwkY7IjvZbBY+4UEfJoSqh
	tj5ljrMI10pEbRnwM8oIfzoOi+J2Q5Q=
X-MC-Unique: neNgteJyPzmPOnkWvFKIxw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=i9wKV06h+UOk80WmcgiNf44dt9xbVtplMA+/hl0US7J+YOsk47JIQNYLekUTA5GdCZCtEW7AjYZme0ky07fW5iJES4RzsjOp1t1FXmFThP6/XKd9HfXHxELaMfePVvgyQxVciFN1FCbf596wz0kkFqIJzEbMLyXElkmzRYShPZk1kc30S/v2IlRJ6bclOs2EbQ5v2b1GKbD+RQzE90+/OEN2FyF1bz+0Qzqpt1wAdIoOtAWODm+GkXL15SRmYaPOA3IiPBbtJUexUh3kyfrG0kC4oCnbnTUNCtoeNLGz5HqN8ROPGUjy0ZGD2QX1jsUWgKzgkqJ2X1RosisQj9aEgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PUIBynGRaD4WhYXwQk0I8TmQPV2fkMGcbu5cWr+n9H8=;
 b=C0BIODhmbNDA7u4wM44W4HQlRQBKlOjtdto5vrRz/9fDjB12G6zZX+481dbojdNeauKDhjWfdgDMUYlzZBO7XCA0ocFEKtdIOJRA0566G6V/ihdt8hiEN0wmQBZMRx4EgxcOhHvyJskkxRqgb93bbfDTweJwDT427dhD6FxsAKv1oXcxQ9bO08mxwMmwLz1pjxbgs/jiVAyk9E3rM0XhYG97/xgGOatD6cWIm4OAe4YZbO4LWNHsnMbkS7Mi4s+BkLkeGRltxJ4+r2X21e5knmIgdAVmGo+ZAui9hRVAqmKrQHTe6hjzmWNWVYsbhIn+aCIpgzbJz0m5XlJM0GwugA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v8 0/6] AMD/IOMMU: further work split from XSA-378
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Message-ID: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Date: Wed, 22 Sep 2021 16:35:33 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0072.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ac2364a-d34f-4556-8d04-08d97dd63c6c
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2445CD4C4ECF6FAC3C441ACCB3A29@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QnzWUsKlyHHklouxXlcIHYPAViwWIvCMIzrItz0CRDgc5iK3dfH7FjWxYM8liozol2es4AGUAL8kWyjvZWJab3x6LMxt4iUk5uyX1+0N+hzmTzWc7dIvquPqrifh1g8xCKB+nS3jIdBu+WswKt5si01dTXHMW2Ap0UmnWHiBlkCm2R7u4HtxtS0VOl9za5CrVQoiBpYtG5sD5npvU42gdQIvSpjFi1LCOLRi7OLqZmxd/pXqnElk727DwtNeY2ZZKxLNWqa7o62wLWQzN9C2NDfsSiEN4z3FvSeKvwq7/6zRLhaplBQ/7rLF6abyrRZVlrVJuaYdrxh73X8av/Cp/8af8srqJ4HXJpYDh1RCZSLOizOEsDLM5OkkvHE/mH/HP+e+BD2fong5PCM7UOtPz8yMGeOAZkLVw3ngElZVu+jsCq6xgocw4ZZ4JkCeFRXBfmTa+giFZ0O95Mhp1U7j+a+t6c7erHVf1NyvLFOx0qoD2tCDk5us+ch5XJ+R2yq2fXifpGckUPYc29fwMg2LqaSjGTPE6ia2+B2tbl5foc4TFcVKXraCxT2ZYtIxyT5CVnnAWkdRa8NVxmMxNz4RtHL10tfQM0dIiQmlqIf2h3y2Alm38OqxU2GFFLKgQl2sdK1cHrZHbIcDAvJ7anZ1sUmNVA1w3yIRVA9ZqtLBQD+vy+DbDwFVeiqg25M3tqSFn6WjLZjrkHDCRu7B7MznBL4iBsjIormJ6t7cPSElMy8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(5660300002)(4744005)(66476007)(2906002)(38100700002)(4326008)(6486002)(83380400001)(2616005)(956004)(26005)(66556008)(8936002)(31696002)(86362001)(6916009)(186003)(508600001)(8676002)(54906003)(16576012)(316002)(36756003)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NGdDM0pPcCs4UHZmS0drSGRqMVFYc1M5VlVpZXVldnIyak9BdlFTRFZ3amtM?=
 =?utf-8?B?YWZGdkw5c1hQcWE4djZERzJvN2lTYlZnTlFvTUhQTGQvbktQTXVRa3YvV3hM?=
 =?utf-8?B?OFlXRXo0a2RjKzlmRktFOUFVU2hBMlQ2RDdEOXNFUlJiN0VzaXZiamJ3dWRv?=
 =?utf-8?B?WWt5N2FORlZKMEphYUwrV0VPWU1vclREZHljREVURFNoQ3JROTc5MWZqM1cr?=
 =?utf-8?B?c2hCakhmU2hNMlhoVWFlVElZVFFCNUJpWXZVYy9aeUlpMGhiSmVDeTFCK3Jv?=
 =?utf-8?B?M0pRdEpUREFaYVJNOHRDRmFXczNSSlZXVkZWeUxwS0k3YlB3KzRpc2lIdGJY?=
 =?utf-8?B?U0orc0Q5cGN3bmlvcnFtOHA1ZjJVeUY2WVBWVkcvYUVxeVAvZmJuaXFLWGY3?=
 =?utf-8?B?cFNDQk02QzF5QXdzSVI5Mk56T05tV0c4ZXpHS0dFUC8xOGpXZW9ESVhJNVpF?=
 =?utf-8?B?bHo4UG9TWkFlNzZLeUJmWjFFZWJjeVNnVllkS0NIeVkvVHMxMXNFUlFUejla?=
 =?utf-8?B?ZUhqMW9wb2VFSlV0SDYyZEdEYkUvVjFqZVlJdUFuUU9WTm8rQ1FBZGkzdGov?=
 =?utf-8?B?SmhGUXdqbCt3bndQeGZ2TGx1UzZ1MmpnaEZ5bGpmSEZERkdWb0JjcjRUTXZC?=
 =?utf-8?B?bG52VWp1dEE2Z1FpTTNyaEcyZWR0b1gzWFVhOS85SkRMSzJrdTBpa2w4T0E5?=
 =?utf-8?B?dStyT0JmRmwwYWZGVnRRQVRyV1Y0bWQrSWkzenBNdVBHVEtnRUhaV1BBQm4y?=
 =?utf-8?B?dFM3TG9xK0FFYUhDbHlJV0pvWnR5a2ppNHA2N2RzT2V5dkZ2U3RpaDJSR29Y?=
 =?utf-8?B?cSswVnUzY3kycVlsNEs3WGpMbDFXekhHRmc0N3h3c2NZbHNsODlPVm9TNC9j?=
 =?utf-8?B?NlFOeU1xZjFhUXd6bUV1WXVYNzhJRTJWbGVnaWtQaTQrWEY2eGx0Z3hPdUU2?=
 =?utf-8?B?UVF5d2tsYnByQ3Q5UjBLa1JVS1pYWExGN2ZwdlNkM254SlNOR29SQU1Pajk0?=
 =?utf-8?B?L08rdldYcWNqRktxbW9Cdkw2MEN4SWJUQmpHNWVWSkhDQlRKb1RuUVdtWmpz?=
 =?utf-8?B?WGx2TnhCU1AvU1ZETzg2QlVUZW50SEFOSlpZSmNOYWNrRlNmVE1VeUpDbDlB?=
 =?utf-8?B?MGdwQWRDaThyclpLUy93SlgxNXFQOHpRRlZONkQyWFhwamVaUERLK2xWb3RB?=
 =?utf-8?B?VlJwTmxsY09FOWpSN0JsSTJJV3pueHorUVdOZW1LbXZONGs1UEV2UWpnSmY1?=
 =?utf-8?B?Y3dRWGhuaktPSDFaMFVCWExud3BjMjBZeXBMZ1lnQW8xdDFqMmJHZjlQYXl3?=
 =?utf-8?B?UTk3b0h4bU5zZEdOakc1ZTdkTHFDTjhHaTAyTkdxS0VPVGFmNkVNeHd5MFZw?=
 =?utf-8?B?alZHTjZnZUM2U0h2TWFhNjA2UGZKa2hId3YvRmlycGp3czdrd2hwQ0RpSGZM?=
 =?utf-8?B?OWxHblk2UWRVUTY4d1lweXo5QXA3K1I1cWRPbFRid3doZzREanNTeGNENUZM?=
 =?utf-8?B?a0JuTWkwSnM1aTdFR3BqUHFKb21PSTkyc0krYlJ6L0tZVUo5VTVBeXFLdUJJ?=
 =?utf-8?B?cXNCYUR1OTNiekcwRHNPSnVhNnJXenFXdFoyQTNtbVkvVkdWVXpoYkhoUi81?=
 =?utf-8?B?VzdyZ2xTNmM0ME5xRzNtWndqcy9saDgrbWhUYkFzSHBuY3ZjQmF1ei9CVHhn?=
 =?utf-8?B?bit0YVNrTGZIRjdqaVN0Y0FNRzlvMGJOd2pHRW9SNWI4ejdBNHlFbWxkMnhs?=
 =?utf-8?Q?QTKcvV5n69nFWk28k3kCeLaIbHffgFHYmgErlW+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ac2364a-d34f-4556-8d04-08d97dd63c6c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:35:34.0525
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6FUthgMTT64/fIViH7IG2Oybv/qoj+a1Yz11f11QY1gG4XBnu2XOq40aSU7zSOPWAs2D0osUUesg3D1M8FHflQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

Along the pieces that were determined to have security relevance
there are quite a few more fixes / improvements (or so I hope)
which were decided to not become part of the XSA itself.

1: obtain IVHD type to use earlier
2: improve (extended) feature detection
3: check IVMD ranges against host implementation limits
4: respect AtsDisabled device flag
5: pull ATS disabling earlier
6: expose errors and warnings unconditionally

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:36:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:36:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192862.343558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Mi-0001rj-Sl; Wed, 22 Sep 2021 14:36:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192862.343558; Wed, 22 Sep 2021 14:36:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Mi-0001ra-Pe; Wed, 22 Sep 2021 14:36:52 +0000
Received: by outflank-mailman (input) for mailman id 192862;
 Wed, 22 Sep 2021 14:36:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3Mh-0001rN-RX
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:36:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8596ceb8-1bb2-11ec-b99e-12813bfff9fa;
 Wed, 22 Sep 2021 14:36:50 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-LH5A588vO8K5uVEmRWeoig-1; Wed, 22 Sep 2021 16:36:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 14:36:46 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:36:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 14:36:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8596ceb8-1bb2-11ec-b99e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321409;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ym1bnxR0U4vds7SXc2ouMI0aswya4N0rxIF16d3/+iM=;
	b=AZ9WRWLKhHmdDZGSHs8yUY75HkL9HDup3+WemIZQJJ0FLReUbfv+Y+Mw5nCCsPWFxS0UeC
	qnpM/3k2RAAWTTtQwJIxiaVDSoMEfonwPpYHF9JQob1tWi6MiQ10Y1DN/9okiV0Bxi3YHO
	69CMHFSts7DJeKZ1zytmAD+ZLX37DVk=
X-MC-Unique: LH5A588vO8K5uVEmRWeoig-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Lh8Xe1fxqTsI5KT+N8xZgj8gKILY+h5wfo1Uh0bKBhkrsr++rcUxPOoYEAdo21gpZ3q5O6OYv9x2EN4fdHRTXxPF/HLhtPdtYFu/FCJIcOLd07a1zcTIPFgebYmGeX/WPs+qYoJ7gIuPJuQrwCfBHfujr6ngXuyEf1kClRspYL1TDf+/rVipIvovx4ItjufDdx10a0V0yhUkZqanRYncjG0x+W3Cwuszfm6kWR2UPtsehPhbnjNbsIS8h4bqpqlhq9X/q4A0ZTQ4/yqreURyBRcSqYL7u0prmNX40VPBAS4drTwwEtDb1p+zrhSA8pUKxQxir6oMvO3tfIyz9C0EYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ym1bnxR0U4vds7SXc2ouMI0aswya4N0rxIF16d3/+iM=;
 b=CXvssv9xSVHt4LWK6I55TFga+dHaAn0Tpj4b39qaLcFp7k+UGKR5OjxmKnLPFfXsVLBUFqC4+QfZ0Abkl1VXQeBIXenBhCqnZoS7c17eKvEFggc3lLO7KJi8wrCuKqbt43s7Eb/cpU7k9Hz8Omzo1tGxCliQG3SxpocP47+JM9WEGjHh+KHXYdSVp9A14uEOW5RsAGmrYdIKKbj4kwZ9u84mfsZ4PH+hTvaLuqO2zncqJdMOCf1e96T1L/s2c+6RMDh7pGnIro9NWWgm5qri8CbD3tET7+fTFsUW48EM+9wchCM8UMlN8KAaNsSSfCYy3yZYGNhrrk7hkECye9OcrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 1/6] AMD/IOMMU: obtain IVHD type to use earlier
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <d5f76461-70d2-fc59-2213-99a093e3b57f@suse.com>
Date: Wed, 22 Sep 2021 16:36:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 19ac131c-b341-44fa-1e57-08d97dd66720
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB703991ED75CF2D5513CEB9FDB3A29@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XsmVvJo1yHaisw75CefUVryWMfUUYQv6QCQ+7ukYy+Br5/WHAzLmPuCP9UrtMMkbdXI7rF2bf16+2xplxKFCkGxS9KY6FBYQMVBFGzS58gSri05lNxcURJQeDdgsVv1x6Km1rAuWcwblmgM/w/cLJSq7OGzCcvWBeg2EOR8YHkdMtqYSVpMyOdGqWy0Dz49LsR+fejQtyVar0DuNrl2FqZFRrPjTLBkDAV5+WdvCecZLBBLNOKDFowy88K3iGHuZlAShp8yL4D8qO1QCYzK5nSgEhApT9111WQ5qCxwVELe9EGZ3wulSd/2KBoTP0W1JXK/mZwa61iLFRxX0HtEdrZVwsZWAlCNaymOg3ZajLWi8cudro3VogBlwUTrhLq+ikSYUmAD1L/xqY4HjRpZ/wlh31YLSCw7/AoOfwAodvEa+AttHl+CdX4rViqJ+lNRfCTpSqCw1Fl22+GbUdzNTId/VRWJZ8IfEE6YMyiIBB6WHmi48dv+BoTbA0OnPsuCXDWWXRywNMOp2pr5jZz9sHA0V4ZFsel8i0muTifK+Ap5YQ1IJ31x3E+3qjyDKn4VYCq/AU/K6LlK2BEGqVsljAGdTtwHzCvdDFPgsow1jdRphMMLQ73wCUYtxwBxncPUE4+2SXiqpo8kg7jW6NuLoFN0hK/fnqueKyNh/L3GXVvQT5Uvr2JRPp+QF0lMqQTE6sEKJTFAK3S59UXnPHvC5Zqlt1aPg7AqauD/nMGwU3Jw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(66476007)(4326008)(66946007)(83380400001)(38100700002)(31696002)(66556008)(316002)(5660300002)(54906003)(31686004)(2906002)(86362001)(16576012)(6486002)(186003)(8676002)(2616005)(956004)(36756003)(6916009)(8936002)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZisyMDFFU2Rnc05sZkkwSlB3VjJWQ2xtMXRoUk1UL0ZCQnFTbitEQ2FRWEhY?=
 =?utf-8?B?cTl5bnJlY3lIaVNVcTd5ekZuazk2TlhVd09nL1lOOEtndE45dVJCN01FN3Bx?=
 =?utf-8?B?NjZIWkZ4Mk04WFlTL1BoTHVzbUJLU3Zob09yeFh1ZXpwenQyMGhTendGeEIw?=
 =?utf-8?B?M05nbXZ2Smx4Ny85aVlESSt0bTJza1B5UmdGVW5OUUxwTlJXblQ4aTJKTmtv?=
 =?utf-8?B?WWVFUzBnbWlpaWRSbU9RdTZLUWZCYkJPblhEcXlESHNjbnVLMHM3Sm9mODM0?=
 =?utf-8?B?c0cxMmc5RG1oMHJiUU45SllKNk5DOU9DVVo4c3RpWDRSeVZrbGJjYndXMDVl?=
 =?utf-8?B?ZEhSblF3NUI1WGZLdjZlVWNrMXZ1TnF2bTlDdEowVk02ZlkwT0R1Q1lCbDZP?=
 =?utf-8?B?Wnh3K2ZVVTFmS1NtK0ZqTTZBMDYyQkVoclJ2YkF1ajlNaFg1Q1FvWmlPV1ND?=
 =?utf-8?B?UEFRK1U2cmVlM3VKaDRQS2VacXIwL2RjQzkvUC9KYVBwVWloZDQxZ1VpWkJz?=
 =?utf-8?B?ZEp3bWJHM2JPWXFrODJOZ05OWCsrRUp2dFQxRjY4UXUxNkpDNTVpRDF4Wmx3?=
 =?utf-8?B?aDJqOXROdmlQaC83S29RZEZTTlI2ME1DbVVic0V5cm9RckdlQ3owQkU5bmps?=
 =?utf-8?B?Um5GaTJ0WGdzMHlCREdjRWpCNGZLQjBaTXRCekdCeE5kY0JqMjVmVVFtWjg5?=
 =?utf-8?B?OEdESWUwd1NuUVc1WnFXdDNPV0FUTkI5T3JEZlNEUzV5eDZUWW9xZjRoZjU1?=
 =?utf-8?B?VFB0QkJ2SkdVdkR4MHliN204VTNmTVBxZ001UEhQRU5WcC81RCtjZkZKc0V4?=
 =?utf-8?B?K29zRVFJU0R1UzZ0WFkwaFIrYWNVYkFRVndoekFYL3plRS9wVzEzQmtmc3cv?=
 =?utf-8?B?alJaR1NhdmNPRlFKWWxWTzBrOG9KVURzdytJamtFK1dBV0JRUVBIaHlvcHp5?=
 =?utf-8?B?Wi9lYjJ1RXE4SjRiNGszY0tTU3VqeWRLSEU1ZUQya2N2cXlwTkpLWlpudHVQ?=
 =?utf-8?B?WVhEcWNSMVdFY2lvcXhZNXdXZGZ3cGcrOWNuZnNIS2NqN3VKV1pDR2gwS1dx?=
 =?utf-8?B?VXhmc2FEaEVNd3JUYlVxS1RtVFlSN3UzdEFlYVRBUEU1aERyeEIyNXEwcGVE?=
 =?utf-8?B?QTBST09XSGFFSTlIWXBEOTRJZFAxbTR6ZmlKR2NKZEg4aVIxNmZoTEdJYjNv?=
 =?utf-8?B?R0xFSXhJS01ZSTNoMFZMOVRTNStIL0t3dnFxOVk1dDRFa2NLTXlsUEJjSGZp?=
 =?utf-8?B?dTR4SXA4Z3RteEFTdHVBNzlaSVhQYTVIMzN2aGpUdzB0Z3dybnZ1MTlWdnlO?=
 =?utf-8?B?NnlIU2NMbGRSaDlOU3ZpUGo4NGE1OU5aSkRma3VtNWk0eXlJMkN3T29DaUZI?=
 =?utf-8?B?S21NOE9yVkZ0MG1ST2ZoOGtZdTlOTWtBc2c1MGlyR1Y1cmVyVmIzYk9YU2tl?=
 =?utf-8?B?cVNzRVlJaVkyeHg4NjRSd0pyK3ZMc2pOb2NCa0hEeVRWNFJyaytsbVRYbUxJ?=
 =?utf-8?B?dStWK1d4VVlla2VSMm1FbVdQYUtGVjhDaFVONkRLSS91NjFMRGF0RkFBTHBG?=
 =?utf-8?B?QmJ0NHFkMWk3WUhQVWMydGsrLzBDN1E5Z3lyVE52V1R1b0lROWp1VG05K0VX?=
 =?utf-8?B?c05Waml2enc4RkdjZVg5Vll2TmhRY2IrSWUrRUZjdVNWUWJDb1RiaWFBcVhD?=
 =?utf-8?B?UVEvNXNzMHdHTmdsdlhqNmFzN1BRRDUrdGVYdDFZOVZ3Uk5ZRFhGYjc4YkIr?=
 =?utf-8?Q?yUcGbp8MpXsrw91zhCyqyfBmdMEBPW9TDQalDhy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 19ac131c-b341-44fa-1e57-08d97dd66720
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:36:45.7146
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cyDf217/hxYrsNVQpZfLa3KgiemEcboMlTiQSQ8Cl/RwRWbYjCZWmBucnQZCw7BLMEwWgQqU/nWoaC5iClSOYA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

Doing this in amd_iommu_prepare() is too late for it, in particular, to
be used in amd_iommu_detect_one_acpi(), as a subsequent change will want
to do. Moving it immediately ahead of amd_iommu_detect_acpi() is
(luckily) pretty simple, (pretty importantly) without breaking
amd_iommu_prepare()'s logic to prevent multiple processing.

This involves moving table checksumming, as
amd_iommu_get_supported_ivhd_type() ->  get_supported_ivhd_type() will
now be invoked before amd_iommu_detect_acpi()  -> detect_iommu_acpi(). In
the course of doing so stop open-coding acpi_tb_checksum(), seeing that
we have other uses of this originally ACPI-private function elsewhere in
the tree.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v7: Move table checksumming.
v5: New.

--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -22,6 +22,8 @@
 
 #include <asm/io_apic.h>
 
+#include <acpi/actables.h>
+
 #include "iommu.h"
 
 /* Some helper structures, particularly to deal with ranges. */
@@ -1167,20 +1169,7 @@ static int __init parse_ivrs_table(struc
 static int __init detect_iommu_acpi(struct acpi_table_header *table)
 {
     const struct acpi_ivrs_header *ivrs_block;
-    unsigned long i;
     unsigned long length = sizeof(struct acpi_table_ivrs);
-    u8 checksum, *raw_table;
-
-    /* validate checksum: sum of entire table == 0 */
-    checksum = 0;
-    raw_table = (u8 *)table;
-    for ( i = 0; i < table->length; i++ )
-        checksum += raw_table[i];
-    if ( checksum )
-    {
-        AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum);
-        return -ENODEV;
-    }
 
     while ( table->length > (length + sizeof(*ivrs_block)) )
     {
@@ -1317,6 +1306,15 @@ get_supported_ivhd_type(struct acpi_tabl
 {
     size_t length = sizeof(struct acpi_table_ivrs);
     const struct acpi_ivrs_header *ivrs_block, *blk = NULL;
+    uint8_t checksum;
+
+    /* Validate checksum: Sum of entire table == 0. */
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(uint8_t, table), table->length);
+    if ( checksum )
+    {
+        AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum);
+        return -ENODEV;
+    }
 
     while ( table->length > (length + sizeof(*ivrs_block)) )
     {
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1398,15 +1398,9 @@ int __init amd_iommu_prepare(bool xt)
         goto error_out;
 
     /* Have we been here before? */
-    if ( ivhd_type )
+    if ( ivrs_bdf_entries )
         return 0;
 
-    rc = amd_iommu_get_supported_ivhd_type();
-    if ( rc < 0 )
-        goto error_out;
-    BUG_ON(!rc);
-    ivhd_type = rc;
-
     rc = amd_iommu_get_ivrs_dev_entries();
     if ( !rc )
         rc = -ENODEV;
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -179,9 +179,17 @@ static int __must_check amd_iommu_setup_
 
 int __init acpi_ivrs_init(void)
 {
+    int rc;
+
     if ( !iommu_enable && !iommu_intremap )
         return 0;
 
+    rc = amd_iommu_get_supported_ivhd_type();
+    if ( rc < 0 )
+        return rc;
+    BUG_ON(!rc);
+    ivhd_type = rc;
+
     if ( (amd_iommu_detect_acpi() !=0) || (iommu_found() == 0) )
     {
         iommu_intremap = iommu_intremap_off;



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:37:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:37:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192865.343569 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3N2-0002Y0-5t; Wed, 22 Sep 2021 14:37:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192865.343569; Wed, 22 Sep 2021 14:37:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3N2-0002Xt-2q; Wed, 22 Sep 2021 14:37:12 +0000
Received: by outflank-mailman (input) for mailman id 192865;
 Wed, 22 Sep 2021 14:37:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3N0-0002Vd-2a
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f7eb2009-6179-48fb-8483-cfefde1d690f;
 Wed, 22 Sep 2021 14:37:08 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-6qAaoySUPVKlzxJtqDs2lw-1;
 Wed, 22 Sep 2021 16:37:06 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:37:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:37:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0026.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Wed, 22 Sep 2021 14:37:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7eb2009-6179-48fb-8483-cfefde1d690f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321427;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+/R2l8vt0G8tNtmvPjl7fZ9gXSFTbwJmfLkNFVNLpmQ=;
	b=UhWUHBgxNCSleBHMj3V20uOMsPZX+2Yg03bHwtvIQCaGgbhiqxSYgP6nKOQxdBnSg76wlh
	2WX3i86FjqAVCnlXacwsTQjmKYcHhIEBAc2gPBEhoGBxzHLGJoJT2la5pDuhaUTuUrD6U9
	v8XNA+35bY1IE39DuErdfyU320jFEk4=
X-MC-Unique: 6qAaoySUPVKlzxJtqDs2lw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z66vj0047mTU5EESP4sULXQ1MS0tNDiLIZ+//m6n5PWRgvUIG+Q1WUaQewKNnIYaAnEyDx38+oUvm9sdslKOnqnMxjJYcfDol7VuH3bIEHJfxfhTihgw0dUD+YKuZ8ckXd7h+GcmE+vA9GYfmot4E3sjexdU0ECBACudaCG3oXY/2HocYmsxNnrkxImZJ0RNNKl1iCrI7EIeoD75q2jorDXEG4/Hm6xsRYBarL80Qs145D0Q4XAlDn5D4eTTuDq4Kb9/uYNkjMnu2T28O/Xk/xqQgK14zdMZKVltJBzsz9OUh9dsqr4Y2318/hFi0JDbxWtuYX+kVy4g1vJMvt8tgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+/R2l8vt0G8tNtmvPjl7fZ9gXSFTbwJmfLkNFVNLpmQ=;
 b=V/CEU36JtINRpARvKMBYbVRo7osmTGzMeBCn7recrtS0e/9sLuPQyo5MFibi9mNEDY7OyP2/9xWwYcP3+N0ZIRBV+QC771gf8t9ctXW1QpRGmP3NC6Y5y1G1rOSGRCmHzVVA5zWS+RnL1GKye5zPFzWnEm8h2HoXxmtmDsrHHmlHQgc31ljHbGggsf+D4AzNOB0UzDOrix77/qJUZvwWUDa11uFijvYKg5mpHcqYxPOT+ZDwWBnhBdzqsXl7w5mtxZqNQY//hH4HCuf9KGTo5taUT3jFmdMVrtw+Cw4I4U6Qkv9nDugo+9BIL9Z0fTMk/OpcwzA7gFfPLY/pmIMINA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 2/6] AMD/IOMMU: improve (extended) feature detection
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <4e2034ee-a074-a321-ccbb-9e36c74f0060@suse.com>
Date: Wed, 22 Sep 2021 16:37:04 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0026.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c0f66931-a73b-4102-a62e-08d97dd672c7
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024C103B7A1A9B7865E25B1B3A29@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Rg/HDUgGphnXEsx061sn1RpxYlO11XJv14xpB6aKFSr7OzqtW5PG49fKEU+OZAdIUWv/QCFspjiBOG2nVhtd4aL0Kt5vmQdV2KW1EZullR5ThDUOlJET1/FaTXUMjhNUdxp3DoTExtxeWW2LA4qTekl4x5TGEhNganNQCngIXb2SszuOtH77AeaMyrZtzUXK7Bqu7VFwDlJHLtNYavhLvQGYfJ4EoPMam75DuqFuVNifGkhGBnMMLowxZG1HI4qgxGEERJlUiaqH7xtXQNf+M8+/q0JQAbTp3om2uDfhTOPzQ1BAjsta+HKbtpGw7kEcSkbokKcvmOmG3GL5BB97E5B0kl02bFE0PTUOb3OqrHlhW3gYEbaV30O+nIS2oPSHbl+Q8OpV2daNnQ0b+7YHKR7M75RU0zdDHSOkBwmSmHy+M2tJwOL7RkenqdIoVkyTsDMKh0qmvTwr50F1AgtpopwAmUeHQ41RN8qRatb5d3s7MBPhIhAUsOR89LZGA1yTn3WxboUeiWB9s/JdcYbBurXIPNCEaVnZ0lTmIGLEBMNPa4S/SqvLQ6Bw3f2L0Ky5bj71nQ/oxB0e+rS0/WBlj4bq1e//QMFK4ogPBuBSNQ5A/SVnEKvNJnPAgdUBhuaO19yEyT0IA5N7epWrzvAXECsUFFmlDeidCmWdvtc/Tkz1fEjJx+TMmD93fzi1t6Vh+xhnt5KFAKvGJBIjuwYoF0LMUUAAYSv4VDONEmUA/yY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NDE5anpyMkVIUFl1RWVxRG9idlM0SHhIMVEzaWNRZXR4RTM0VTdIZCtlSmR5?=
 =?utf-8?B?V3pObldsM0s1N2crN1BmeHh2aDFGbmMyK2hHSEh0ZHhNaGgwWEpEL2pLbS9o?=
 =?utf-8?B?bDd0N1phdVRkMDNUMmNDcFovQ2kvSGVBU0RRLzZ0N1diRTJWR2hDbEhxbWlY?=
 =?utf-8?B?aDNEbnB1T3Mxay9MTGQ1NUxpUk5PVDNKWW9mUTEwcE1rWG4yM1l4Z3piU3p0?=
 =?utf-8?B?NjQ4d201UEdSUlV1QTFBSXh6U3lDUWdzcXZkcktscWlGMzMxdVNGclNkaHBi?=
 =?utf-8?B?SDNuOGxickcySFp4WTJNb3AyN1cwd0p0Y2h4LzJadDAyaUZkZE1QSk9UTFdS?=
 =?utf-8?B?bE5Va1pVSkJDUkRkSW02R1U5ZUE5QXRkUzM2elpaNUtMSHlNZnlWL1ZPeTFv?=
 =?utf-8?B?UFk0SmdCeUFXN2hZUjZUdWJtV3p5TkxCZmp4bG1CbVZuOUhscGZLU1FUZHlI?=
 =?utf-8?B?MklNMExxTGRFMkFKcXQrME5tQVE1Q3EvbVlLbFpBaFR5U3pPK0ZlOUJjRC9H?=
 =?utf-8?B?VEtRN09mQnQyaS8rR0dzeHB5Yy96NUZsZkdDK3RaR1ZnKzVvbW03TFdWeFJE?=
 =?utf-8?B?bmJkQWptOVdJeCtkRU5BakVRamVZMUorUGFTVnVuUmVobnV2R2l6dkQvZ2k1?=
 =?utf-8?B?a2RVK2lKanIrSlZMWXZaSXpSZ0lJY0t2ekxlWnZrem85RlM3cGhQOVhzeXdO?=
 =?utf-8?B?N21tVWQ2b0dyTlBNekdhTjJjNzUxRjVlRmo4RUpabFcxaDkrRmdHYTRnOXVD?=
 =?utf-8?B?VytYRG05TXVXMUFxdHFMaW5KNm11aTgrVTJmRmwzazF5c2VidU9uQXduK0s5?=
 =?utf-8?B?LzJ4UG5tYWJqMGZ1bEF0MjVDT0V5dzBDd204NHVyQnRMQ25DUVVYSkxvekp2?=
 =?utf-8?B?UGxRb1hyL2ZEejBaOURRTWFsZytmc0FJTW40Qk5maTNRRkhzRmtseVFWc3ZI?=
 =?utf-8?B?VGpITS9MWEFrYlRJWmFwbDlYalFCenFma0hSVklvTUxKdEdQYkhQTFdRQmpa?=
 =?utf-8?B?TzE0SFVaUXpmbUZqTzlWOVZYYjBMSElURzhDQmhlOHFyT2Z2a1VqMjNjZG15?=
 =?utf-8?B?TFNQT3NkTkw4YjgwVllhbi9OaUFFbU05MXZ0eStadHRSY3JWd2V0OWp4eGdw?=
 =?utf-8?B?T2JCdG1Zdy9zMXJnQW1DQ2IxOUlHYWs5VUJtWjlkeXcrMXdCNmc2a1Jma0wx?=
 =?utf-8?B?aUZocW5GOEhjVk9POFZMUUtxT3V5eCswZGJhdDBpTTZ2UFZzN2tJNlIyRm9a?=
 =?utf-8?B?Yit0dC9LYmlPTmZxU3N6VnJRK1lTQ3BUTStSWU55ejZCL0RMNTkyN2h4aUc2?=
 =?utf-8?B?YVlNdHRPbEVSTUUvUkE4RU00QyszdE82aWNQdU1SSklFSHZ0d0h2MUV0cGVN?=
 =?utf-8?B?alpDRWhSelpFS0dQMXVhb2VlN01tNGJPU2ZnNWJFb3dBb01QenF0MlZPdUEr?=
 =?utf-8?B?cmZOUGJsUFpoZGxNaEJmZ0dNanFIYUtBM1NicWJnYURMQURvdkNpaWJhM09n?=
 =?utf-8?B?Yll1bTFRSmhBZEZ6eHVEWWhIUlNjRFZYdVI4ZlNHS2l1MXdYTjBkM0JHdU02?=
 =?utf-8?B?NUpJdVdPclF4NVJuZkd6L3c3OXh4Ym5yeW5FMHhvdzZQbEc1djU2L2RwTVZG?=
 =?utf-8?B?ZytVRmVMNkUrYm1kNjcvanNpMStVd3A2WTBFMCtOSHhITlhxOWJFVFNnV0tF?=
 =?utf-8?B?V1B6MDdXWjVHOEI3c21tMWFXR2hiN3FFQVlGaGNwRlc5RUZWWTRFaUM1WU44?=
 =?utf-8?Q?wNtEcbLD4ZggM01PucIgeJCSxAPuPzsq8mXsU/w?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c0f66931-a73b-4102-a62e-08d97dd672c7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:05.2835
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KoeSJrzDEo4+5cvD0xN2/E8RnEJXjqVKUUuIl3cptCLyi5dKnjhy/sQdYijLMSbBVKpoKsTR5Sc4CUA/y0inIw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

First of all the documentation is very clear about ACPI table data
superseding raw register data. Use raw register data only if EFRSup is
clear in the ACPI tables (which may still go too far). Additionally if
this flag is clear, the IVRS type 11H table is reserved and hence may
not be recognized.

Furthermore propagate IVRS type 10H data into the feature flags
recorded, as the full extended features field is available in type 11H
only.

Note that this also makes necessary to stop the bad practice of us
finding a type 11H IVHD entry, but still processing the type 10H one
in detect_iommu_acpi()'s invocation of amd_iommu_detect_one_acpi().

Note also that the features.raw check in amd_iommu_prepare_one() needs
replacing, now that the field can also be populated by different means.
Key IOMMUv2 availability off of IVHD type not being 10H, and then move
it a function layer up, so that it would be set only once all IOMMUs
have been successfully prepared.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
---
v7: Re-base.
v5: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -306,6 +306,7 @@ extern struct hpet_sbdf {
     } init;
 } hpet_sbdf;
 
+extern unsigned int amd_iommu_acpi_info;
 extern int amd_iommu_min_paging_mode;
 
 extern void *shared_intremap_table;
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -1062,7 +1062,8 @@ static unsigned int __initdata nr_ivmd;
 static inline bool_t is_ivhd_block(u8 type)
 {
     return (type == ACPI_IVRS_TYPE_HARDWARE ||
-            type == ACPI_IVRS_TYPE_HARDWARE_11H);
+            ((amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) &&
+             type == ACPI_IVRS_TYPE_HARDWARE_11H));
 }
 
 static inline bool_t is_ivmd_block(u8 type)
@@ -1176,7 +1177,7 @@ static int __init detect_iommu_acpi(stru
         ivrs_block = (struct acpi_ivrs_header *)((u8 *)table + length);
         if ( table->length < (length + ivrs_block->length) )
             return -ENODEV;
-        if ( ivrs_block->type == ACPI_IVRS_TYPE_HARDWARE &&
+        if ( ivrs_block->type == ivhd_type &&
              amd_iommu_detect_one_acpi(to_ivhd_block(ivrs_block)) != 0 )
             return -ENODEV;
         length += ivrs_block->length;
@@ -1316,6 +1317,9 @@ get_supported_ivhd_type(struct acpi_tabl
         return -ENODEV;
     }
 
+    amd_iommu_acpi_info = container_of(table, const struct acpi_table_ivrs,
+                                       header)->info;
+
     while ( table->length > (length + sizeof(*ivrs_block)) )
     {
         ivrs_block = (struct acpi_ivrs_header *)((u8 *)table + length);
--- a/xen/drivers/passthrough/amd/iommu_detect.c
+++ b/xen/drivers/passthrough/amd/iommu_detect.c
@@ -60,14 +60,14 @@ void __init get_iommu_features(struct am
     const struct amd_iommu *first;
     ASSERT( iommu->mmio_base );
 
-    if ( !iommu_has_cap(iommu, PCI_CAP_EFRSUP_SHIFT) )
+    if ( !(amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) )
     {
-        iommu->features.raw = 0;
-        return;
-    }
+        if ( !iommu_has_cap(iommu, PCI_CAP_EFRSUP_SHIFT) )
+            return;
 
-    iommu->features.raw =
-        readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET);
+        iommu->features.raw =
+            readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET);
+    }
 
     /* Don't log the same set of features over and over. */
     first = list_first_entry(&amd_iommu_head, struct amd_iommu, list);
@@ -164,6 +164,42 @@ int __init amd_iommu_detect_one_acpi(
     iommu->cap_offset = ivhd_block->capability_offset;
     iommu->mmio_base_phys = ivhd_block->base_address;
 
+    if ( ivhd_type != ACPI_IVRS_TYPE_HARDWARE )
+        iommu->features.raw = ivhd_block->efr_image;
+    else if ( amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP )
+    {
+        union {
+            uint32_t raw;
+            struct {
+                unsigned int xt_sup:1;
+                unsigned int nx_sup:1;
+                unsigned int gt_sup:1;
+                unsigned int glx_sup:2;
+                unsigned int ia_sup:1;
+                unsigned int ga_sup:1;
+                unsigned int he_sup:1;
+                unsigned int pas_max:5;
+                unsigned int pn_counters:4;
+                unsigned int pn_banks:6;
+                unsigned int msi_num_ppr:5;
+                unsigned int gats:2;
+                unsigned int hats:2;
+            };
+        } attr = { .raw = ivhd_block->iommu_attr };
+
+        iommu->features.flds.xt_sup = attr.xt_sup;
+        iommu->features.flds.nx_sup = attr.nx_sup;
+        iommu->features.flds.gt_sup = attr.gt_sup;
+        iommu->features.flds.glx_sup = attr.glx_sup;
+        iommu->features.flds.ia_sup = attr.ia_sup;
+        iommu->features.flds.ga_sup = attr.ga_sup;
+        iommu->features.flds.pas_max = attr.pas_max;
+        iommu->features.flds.gats = attr.gats;
+        iommu->features.flds.hats = attr.hats;
+    }
+    else if ( list_empty(&amd_iommu_head) )
+        AMD_IOMMU_DEBUG("EFRSup not set in ACPI table; will fall back to hardware\n");
+
     /* override IOMMU HT flags */
     iommu->ht_flags = ivhd_block->header.flags;
 
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -29,6 +29,7 @@ static bool __initdata pci_init;
 static void do_amd_iommu_irq(void *data);
 static DECLARE_SOFTIRQ_TASKLET(amd_iommu_irq_tasklet, do_amd_iommu_irq, NULL);
 
+unsigned int __read_mostly amd_iommu_acpi_info;
 unsigned int __read_mostly ivrs_bdf_entries;
 u8 __read_mostly ivhd_type;
 static struct radix_tree_root ivrs_maps;
@@ -1375,9 +1376,6 @@ static int __init amd_iommu_prepare_one(
 
     get_iommu_features(iommu);
 
-    if ( iommu->features.raw )
-        iommuv2_enabled = true;
-
     return 0;
 }
 
@@ -1419,6 +1417,9 @@ int __init amd_iommu_prepare(bool xt)
             has_xt = false;
     }
 
+    if ( ivhd_type != ACPI_IVRS_TYPE_HARDWARE )
+        iommuv2_enabled = true;
+
     for_each_amd_iommu ( iommu )
     {
         /* NB: There's no need to actually write these out right here. */
--- a/xen/include/acpi/actbl2.h
+++ b/xen/include/acpi/actbl2.h
@@ -716,6 +716,9 @@ struct acpi_table_ivrs {
 
 /* Values for Info field above */
 
+#define ACPI_IVRS_EFR_SUP           0x00000001	/* extended feature support */
+#define ACPI_IVRS_PREBOOT_DMA_REMAP 0x00000002	/* pre-boot DMA remapping in use */
+#define ACPI_IVRS_GVA_SIZE          0x000000E0	/* 3 bits, guest VA size */
 #define ACPI_IVRS_PHYSICAL_SIZE     0x00007F00	/* 7 bits, physical address size */
 #define ACPI_IVRS_VIRTUAL_SIZE      0x003F8000	/* 7 bits, virtual address size */
 #define ACPI_IVRS_ATS_RESERVED      0x00400000	/* ATS address translation range reserved */



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:37:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:37:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192870.343580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3NN-00039m-JX; Wed, 22 Sep 2021 14:37:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192870.343580; Wed, 22 Sep 2021 14:37:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3NN-00039d-GT; Wed, 22 Sep 2021 14:37:33 +0000
Received: by outflank-mailman (input) for mailman id 192870;
 Wed, 22 Sep 2021 14:37:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3NN-00039P-3K
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9e2aad00-1bb2-11ec-b99e-12813bfff9fa;
 Wed, 22 Sep 2021 14:37:32 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-XzJ_7qFmMF64tdWfjv1idA-1; Wed, 22 Sep 2021 16:37:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:37:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:37:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0011.eurprd09.prod.outlook.com (2603:10a6:101:16::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:37:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e2aad00-1bb2-11ec-b99e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321451;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=73rBnD5E2FMoCGn+vOvsFjvD7EEVJfzmIbJDfM5L9dI=;
	b=M74c+d+aHzoTjuu9rAlta0Gd22luR8rqINPmPtk11DBXP5VlXyqcxwc/nBi0lO36VXox8c
	NPussEIkRJKZshbt4JGz0sCJkLY0XBjR3SmVQZTZvMkSIWZuqP5erXNnm35+37iUi8Mcn1
	oR3EyE+x1EZVMePAAa1Lp00UPZKd4iQ=
X-MC-Unique: XzJ_7qFmMF64tdWfjv1idA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n9ZvOcIjJjphRQ9dBFZK6l11mtyJRXLh1VrsY/ZDs9Bx87l4imURxLSYFy2zqJE7o46KqdXT/Kdp4jvwM3/BIyKUgn6JgKDgzaZ6yJWAsqGTtVY16hXGm6GWnfCIuABUMf8hTyPz5VlYKVDhBTOWZqPoN6di9PMsaGyq2hDrwObt7LOqji6OZzTwivTVnjWaqR/TDDJZUsiVlBR2dtEyMZouI2ay9GUraTB7jTsb+i8EplpGrVnK8i1emPQhVJ+RU2Jzf5wCNdNktfZn9LRgr5zzzR6WEZI23JGoekh9stE7RkPPBt2Miqmk5oILnuu4xqn2uRdIHTGFcKIPrN3f/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=73rBnD5E2FMoCGn+vOvsFjvD7EEVJfzmIbJDfM5L9dI=;
 b=QGaMDTiRfSVZNMsJDnAGFVAhQ4nZMkUZUDPpL63U4Z6x6uYsguq5U7h7lsgBqt3UBjp4zS2xlCAXwhN/fzht1bbqW5lVObvDnb++PZRJYpbwyfE/+Oju3Rc46+FpbanCWp+TVcck+GG16aZGxU3GR9t2qyIAw7fYFUOPqSzuTo9d6pOzhDyYSALyvzgVu9bl1FQZgqoHAMihofnQeyp7UfkXBdoWJB2rO5liAoh60MEN7Gc+NgTKcmkSr9DfT4ejtigqOAkq1CRkiyYiQXKzRsErwL+RPPwv6jOiu5ZOWaYm5N1bxxOZHApfq5mdDPsrKAbXergXT5Gpyp6OSX1TEg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 3/6] AMD/IOMMU: check IVMD ranges against host
 implementation limits
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <4ab0e244-e5a3-4b76-cef4-01d43345369c@suse.com>
Date: Wed, 22 Sep 2021 16:37:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2PR09CA0011.eurprd09.prod.outlook.com
 (2603:10a6:101:16::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: db108d3a-2307-40d0-fa88-08d97dd680d6
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024C40E0442993546355878B3A29@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3044;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	19KpKtvyQpYV+LoHCW8lM0qI0MvTAxYiUf9T4jzeUgyjHZQ9AC2O1k0FVMbAFeTjDk9iEqrw03vEdxELOHTdclvn3bvvjrMJFMd6JLdYiIiigQ2pCuT/4uuAK9fAhNbFncvyjaAMVjR6NU9Qa2CrRLQJp+47Rdm6WGyvk+quYMErmoSh+wGolHwxxMTZliJlC7unAvafiOko08LZdsQk20zp82Kqrpi7dIOjdHr2JboBqiqeziE9agVntvkBatexAwkd74KWPpDD1fW3jo9tsMcQE6zLxd2iNe3nMK5oHp1+GYKna1GI7fcATQIGOqX4Y+/O3aUc7y2N18F+c9Gl1mmu0AJHDNYcBaLNmXb/D0VTwLS18fuE72gh/6RHTUC9sKFhL68H26HfIqtuQB+FCxf/IIpJNg0g+JNupyfbrppLTBQVBaJebHv9ePnJqrJvl6N09OenHozaVrHMZOEBlDj+DmRtizKZTfYEhbiayqOaZajrOqeQZduyp5+f3G6U2FXiuD21JQj/lXP8QTAeXK4nHD9r4dnmQVJTXww/0Df+2PXc/2R6PiWSiX2vpTWd9udwg4D1nN4mlT1djO4bW2XNADxYAFun9LqxjhqWR3BeHubn1Y8e+2sh7juSl/qchgQlKeiFBdcSOiYa/rLjxval0nQkefkSwspGCFUE6O2H3vaTPy6h0LciahyPVyPtPtVkwnGO5d3EpyE/jefVeuuiXtL7md5IjZCIA75gaz0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Zk1PSzdYQlNtbXg1YjNSUlhpaEVrMmZ3QjdlelplWUxXdmM4WDU2Z3ZUREZM?=
 =?utf-8?B?VzNBOWxxWmc1MkFEbUNtUEp0Z3g3R1BwL25HOUFDSTlBNTB2ME01ZERHdlEv?=
 =?utf-8?B?YWhsaUllc3VDWFZicks5NzRnb01qeTEwWGw5LzhNb2EyTVBiQXI5Ny9Yek41?=
 =?utf-8?B?V0xUME5adVAyYXpFcnFOcE9ZdW5md2E4RUJZR2dJY2paNXZWU2FoeEdnaWlw?=
 =?utf-8?B?K3JkKzFNSGlXalROZS8wbnB3bExRcUFDd245aGF3WUs4ZFdjT0xtYlRDdDF0?=
 =?utf-8?B?dmllU1F3Nm1PNWVST1Q1VlhkL3NzeWhYdVg2UWFtaUtOVmw3TmZUb3FRaUFZ?=
 =?utf-8?B?bGUrZktOdTFTSkdNMWpQWVVRWDNIYWRpOEpSbENVTFptVGpocXBXbld4YWl5?=
 =?utf-8?B?Z0luMnZnNWhSN04wOEFoQmh0dWJ0bUFzUjEyL1FJOUtwVzdGK3JXRkxrc2Zw?=
 =?utf-8?B?eW1pVDlsMzdqakJXcWJlS0VhTnZjcmJyY3U4TnZrU0kreEJGVWg0NlFxTXRL?=
 =?utf-8?B?QlVzcnhIcnVrSWorOWVJNEdNVEk0aXQ4dkJZMVZLTWN6U1U3NWw2NG5xOVpR?=
 =?utf-8?B?TFdnMXFLQiszdXFHUmVEQ0ZMWWs5MWM0L0RDSXlDQVFRUFdEc2ZnYkw2eWtm?=
 =?utf-8?B?a3ZOQnJTY3BRODVBTGdEVTZsUG96SUVwOXJNTjM2ajdFSjBERG00VVhJUUNa?=
 =?utf-8?B?WGVtTHF1MG9nUmVHZTN1YndEUVdycVBCSlhOV1I1Y0JROEt5b1dLZDNqQ2Vo?=
 =?utf-8?B?NHk3bG5BVGx1MEtOTWFRSEZra2x0OFpndHBYVHYvQ0hpNVdzb1E0TzBEMkpa?=
 =?utf-8?B?c083MWM2a2hNeGdiN3doaSsvK1IyQ3cwczVUWDNnbFdnRGVGUnhHd25wMU9R?=
 =?utf-8?B?ZHZlRGlvUytBZFZQTGE1SDgzemo3Tjc5RE5xRWxkdGxHUWFEZEhka296WCt1?=
 =?utf-8?B?bnhmZHhhQ1BDWHhXMldnZ1oyV0xnOHR2TE1tdUNyeS9Xc3pDR2tCNDgzdm11?=
 =?utf-8?B?Zk16REJRQ3YzV2xXcGFiZy9tLys1TzIzcitEdjVtenBuT0xpenVCSkxFMDJS?=
 =?utf-8?B?WnorTjk4N3pmR0ZSODdrM3FDVWNuclBUaUtSbjdMenNIRUVLUnFZMWFyajBT?=
 =?utf-8?B?UlowOVJXMmNsMkF1dW5ia3JWM0kzUFdqUytsWVhsUmU0NXlianAzbHo5ODlH?=
 =?utf-8?B?T1JqUFhsK2Zva2xMcGMrWFJTd01qSWVSM2xpVFRSUFNrMlZrMXVJYWtsMklk?=
 =?utf-8?B?SnlXc2l6L1VkdjZtd1pMQ2tqUDA0SG5iUGdnamRXUFRxbzVxUUdoSkw2R21B?=
 =?utf-8?B?ckVJSHNiQ0l5eFBnZjBjVzhrTUNYNFdXdnpjRDhCaW9UZCszSVIvUEE5WEhu?=
 =?utf-8?B?YVI0YWdpbk02MnZFL1MzMUlpY3llbUNOK1VwREZsbDBZKzFmMlg2b3NsVno0?=
 =?utf-8?B?T1REcThEVVhnNjF2M1JSNGlFcFo5R3h0YTNkMm03WjYrMGlkbzEyai9yWjRs?=
 =?utf-8?B?VHlsS2tVQjY2RnFzbFI5SXhrVnhsZkJGUHZCV2F5dlhCd0ZxL09ubUw1aXRu?=
 =?utf-8?B?aS9MSS9Eb3NaWVZNRm9ycTgzeUM4dHZoSVZTSlJYdXlFQXlsRm43T2FVdHhx?=
 =?utf-8?B?dGpNVVVJVWpaUzZ5ckQwOFZGSTVzQzJpbkUwcVZQNUdRVE5INnBnZTJLSDRz?=
 =?utf-8?B?SjgvZGVTWlprYTFKakZYYUU5UGhLS1BaZUdjRjJQNWIrdjJqbWZncWpzVVQ2?=
 =?utf-8?Q?Hvvej9iQT5YiKRhwfRSUf+XbaWEEXzmCemUGz0s?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: db108d3a-2307-40d0-fa88-08d97dd680d6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:28.8281
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XcUrxD8nX1+6P/kugBiZ4pQ8sQIoq2WCdOwQAqg3q2TGGuwYQMXHO542fLbQD8Oax6+DE9zyDJX/3NW5Fifuvg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

When such ranges can't be represented as 1:1 mappings in page tables,
reject them as presumably bogus. Note that when we detect features late
(because of EFRSup being clear in the ACPI tables), it would be quite a
bit of work to check for (and drop) out of range IVMD ranges, so IOMMU
initialization gets failed in this case instead.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
---
v7: Re-base.
v6: Re-base.
v5: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -307,6 +307,7 @@ extern struct hpet_sbdf {
 } hpet_sbdf;
 
 extern unsigned int amd_iommu_acpi_info;
+extern unsigned int amd_iommu_max_paging_mode;
 extern int amd_iommu_min_paging_mode;
 
 extern void *shared_intremap_table;
@@ -360,7 +361,7 @@ static inline int amd_iommu_get_paging_m
     while ( max_frames > PTE_PER_TABLE_SIZE )
     {
         max_frames = PTE_PER_TABLE_ALIGN(max_frames) >> PTE_PER_TABLE_SHIFT;
-        if ( ++level > 6 )
+        if ( ++level > amd_iommu_max_paging_mode )
             return -ENOMEM;
     }
 
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -378,6 +378,7 @@ static int __init parse_ivmd_device_iomm
 static int __init parse_ivmd_block(const struct acpi_ivrs_memory *ivmd_block)
 {
     unsigned long start_addr, mem_length, base, limit;
+    unsigned int addr_bits;
     bool iw = true, ir = true, exclusion = false;
 
     if ( ivmd_block->header.length < sizeof(*ivmd_block) )
@@ -394,6 +395,17 @@ static int __init parse_ivmd_block(const
     AMD_IOMMU_DEBUG("IVMD Block: type %#x phys %#lx len %#lx\n",
                     ivmd_block->header.type, start_addr, mem_length);
 
+    addr_bits = min(MASK_EXTR(amd_iommu_acpi_info, ACPI_IVRS_PHYSICAL_SIZE),
+                    MASK_EXTR(amd_iommu_acpi_info, ACPI_IVRS_VIRTUAL_SIZE));
+    if ( amd_iommu_get_paging_mode(PFN_UP(start_addr + mem_length)) < 0 ||
+         (addr_bits < BITS_PER_LONG &&
+          ((start_addr + mem_length - 1) >> addr_bits)) )
+    {
+        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not IOMMU addressable\n",
+                        start_addr, start_addr + mem_length);
+        return 0;
+    }
+
     if ( !e820_all_mapped(base, limit + PAGE_SIZE, E820_RESERVED) )
     {
         paddr_t addr;
--- a/xen/drivers/passthrough/amd/iommu_detect.c
+++ b/xen/drivers/passthrough/amd/iommu_detect.c
@@ -67,6 +67,9 @@ void __init get_iommu_features(struct am
 
         iommu->features.raw =
             readq(iommu->mmio_base + IOMMU_EXT_FEATURE_MMIO_OFFSET);
+
+        if ( 4 + iommu->features.flds.hats < amd_iommu_max_paging_mode )
+            amd_iommu_max_paging_mode = 4 + iommu->features.flds.hats;
     }
 
     /* Don't log the same set of features over and over. */
@@ -200,6 +203,10 @@ int __init amd_iommu_detect_one_acpi(
     else if ( list_empty(&amd_iommu_head) )
         AMD_IOMMU_DEBUG("EFRSup not set in ACPI table; will fall back to hardware\n");
 
+    if ( (amd_iommu_acpi_info & ACPI_IVRS_EFR_SUP) &&
+         4 + iommu->features.flds.hats < amd_iommu_max_paging_mode )
+        amd_iommu_max_paging_mode = 4 + iommu->features.flds.hats;
+
     /* override IOMMU HT flags */
     iommu->ht_flags = ivhd_block->header.flags;
 
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1376,6 +1376,13 @@ static int __init amd_iommu_prepare_one(
 
     get_iommu_features(iommu);
 
+    /*
+     * Late extended feature determination may cause previously mappable
+     * IVMD ranges to become unmappable.
+     */
+    if ( amd_iommu_max_paging_mode < amd_iommu_min_paging_mode )
+        return -ERANGE;
+
     return 0;
 }
 
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -254,6 +254,7 @@ int amd_iommu_alloc_root(struct domain *
     return 0;
 }
 
+unsigned int __read_mostly amd_iommu_max_paging_mode = 6;
 int __read_mostly amd_iommu_min_paging_mode = 1;
 
 static int amd_iommu_domain_init(struct domain *d)



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:37:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:37:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192878.343591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Nj-0003nw-Sv; Wed, 22 Sep 2021 14:37:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192878.343591; Wed, 22 Sep 2021 14:37:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Nj-0003np-Pe; Wed, 22 Sep 2021 14:37:55 +0000
Received: by outflank-mailman (input) for mailman id 192878;
 Wed, 22 Sep 2021 14:37:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3Ni-0003nN-Lg
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:37:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6624fd67-275b-423e-b278-278fd68fde3a;
 Wed, 22 Sep 2021 14:37:53 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-HiJxDSOZNc2ADWDudpIfFg-1;
 Wed, 22 Sep 2021 16:37:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:37:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:37:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR05CA0108.eurprd05.prod.outlook.com (2603:10a6:207:1::34) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:37:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6624fd67-275b-423e-b278-278fd68fde3a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321472;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=J6zzTKAR8SOShvgrracy6yF82HzYmfEYILkFc9My6/I=;
	b=hEcNw9dxif5adIMTJ5Artx65HkYIYGNPKGpCktrd+Jrexk8BtwXMs61faakJA9wa3Gm6ED
	l1jnHyZ7eZuzy1umNAkvKfAki0uGz0YL0YKXH72xEVPJawuw7zLFb23k6MbLsJPlMG4X6J
	Ycu6NExkdiIu8h/Dt8GW3kNA6gDiaew=
X-MC-Unique: HiJxDSOZNc2ADWDudpIfFg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rm0mX7OpBYkhrujWIydQ9pQhVMCd0bQ6IYFoUvrIIvy3T+zX+yNmhntYkuxM6Qzg2SlCfLUdAy6XGkPbHmZVa1ggzFiCtRp2ZZkO/RIOTrQFKcXMqBCDaTdpc9ZTiQ2cWnh2LH6URP1kF942qawtLX3iq4ObEMI0D+SOMOpsmT0Wn2GiYsaKYHHtKKfIZyeRUz+QrlGdCfVvFGOQjhmdIVuW/eckpEY2gMAa7ybf5euvAz/jhcpBxpzyPdqfhEizyVW+tuZ0oZtb4cgYuva1pSGIui6coznSYqRGFHyz7tfm625oL6GfkTyw9cnSn2oijoRzP96EaLcGAk0hjCKY6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=J6zzTKAR8SOShvgrracy6yF82HzYmfEYILkFc9My6/I=;
 b=Mv1WVTtb+Lyn5safSIFGTcTMCh7KECT1ChcIpqsgIcWdxOfpYeNCs5IWi73Gi9hNmpqhVHTlBG+xOQsyuEbRyaP+lym6XZEVvAlRc+rjYdClq5w2F0TT0tIi8gHo4MXA4/B4UQGGONZKCr+pskFxGpcEwJG/0ykbd1KGK/hlIVVmpsVUgmmcFDqjqCSV59chcia85dKrMBAIc5u5vZibKqZHbnJ1zXVFalYcan9OuU+K4hF8wYLAl6GFecs4sM2ulVtjjmn3yzmJPTqi97SHM0zbYRnMH51yMDA6tWKxeofkhz1c5CJza1RmuE8AkLAPrtU9KtVXqvhK5PMOmmeAbg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 4/6] AMD/IOMMU: respect AtsDisabled device flag
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com>
Date: Wed, 22 Sep 2021 16:37:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR05CA0108.eurprd05.prod.outlook.com
 (2603:10a6:207:1::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 27edfd07-73c3-4b02-7129-08d97dd68d34
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB702448BC2F93C012BD2AE554B3A29@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dVysFPWQSZjjxAhHRCon4xmzgG/jsUxjqvQk28UpbmDuAQf3Sb2rE9JMlu94j4LooCGY2nbga7REy93SxRZwNMkAdL0dkjOplQrfCk5FRqg25QgYkKXwo7G5T6DnVhxTZDgBz5MCjmYHcvGbZtMol1Zt0cG1wPy177YbrM8zi8YyKBy/pd9EibY58IKulSDKuxliPHo9VcpDXuyn/bwRbgApBMx8Z8MtI/g8whoYQGTvfcwJd84av1+Q4MilmO0PJ+wjckWeG06x6mvg73AvB6bBSGVTnvwHzCx6KdMaUew06qUVl3YiHrQCNIbM59dpNKeqOx/QEEg3C8uVyOT3Xr2eK0KkJWsYLPMKhghnCYaYj/cPSOh8mTVKAYC0jNkZYJm48X6OTCOGb74S5hxzo8ugRZ8VL7gMZfeUmSduZ3Nd0XpoVix4/35tyojMEp2tzq0KoAL9x7SNWWgT9/fFUBuPpzXq6M2Ir+s//AFWXnCZpdD8Ea0x4U06A3H066ooLG0jRTHOUDYhpWaOHpW9P42T86wq38f3xyPJJrUvbz06qwPAyzGt3/7ncD5BpmwqszBTloIc6OYaHEu0JwSLangk3llf8fDY1jcqWgEMPqBn/fVFIBVhvSGcnXrCwPIJ5yA1ET6LtTYXPO4BtZVPoe/1c5+zEH8DJid8iswF7qEmfJOWDUrQF4TZRv7D1t/5S9ETWrvHXgmnD8QQH566pUQTuMXqU6WAiktvpHsfKws=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?T250cmViSm9iK2Vtb1BET080ZlMxWW4vMktFNjMyekNVL3NoT1luZXl4Umxk?=
 =?utf-8?B?NUpzMXpQbjhTSUs2NDdTUjd1SFVpd1NNR2dtSVg5Sm55dWV5MytLRnVnUGM1?=
 =?utf-8?B?Vk0rVlpxNDVVRjExcGVvbEt1czR3aysxNmJDYllEZGQ4aGxWKzdsMUxBT2lY?=
 =?utf-8?B?MGtWRTlZMkRmTW40elRmUEQ1c0dscDJzVmtHb2g0QjkrT1NEelMrL21wb2xJ?=
 =?utf-8?B?UU10SjlhUnNzbEloUjY5QTZKcUN4YmYxeXVnLzF2TE11WWlSRjdhWHhRVGVE?=
 =?utf-8?B?RmRjQnBTNjJMaU0ybzR1ZEJPeXJFaE5zbUl6SVBSck04SVc1V3Fzb0NqZVVH?=
 =?utf-8?B?ZDhYQVFod1J5azRpdVFMTnBOUzA3U0hUQ2p2VUVQL0t6VjZ1a0hQbHlMUjB3?=
 =?utf-8?B?RnNTd0ZYWERhNmR0Z2NUajVwenVsQTFrTHNReVVnbUhoZWtFdUlKcHpwUG1Y?=
 =?utf-8?B?TnhnK0doVVBjckdQOGpqYTRSajZBVVBHYjFiTmN2eU5PUEl1NHNLTytxV2VN?=
 =?utf-8?B?L1NBcEhmbFBoS2grK0FvMVNaaUVrM3M0Ni9KcHYvY3hjMVdmcHg5bzMyZWxW?=
 =?utf-8?B?Si9BdWJTUkl4dzRJbGw1S1hKSGM4VE92RkhuRWYxZmZWR2ZPcVFDVkpEMUgz?=
 =?utf-8?B?cFRTcVVqVVdRbVFJZldYUmc2RHJabnNQcER1MGZqbkhpN0dTYmdFMHRjOGR2?=
 =?utf-8?B?bjJaRElrV3p0Q1JLUmFEb2htU3F0U1lRQlZ6eHdMS2U4WlJvTyt4UnRPWnZZ?=
 =?utf-8?B?NWROUEQyTXJWVEtEK05xSkdxdU1RQXhmU3JWd21mZFRaNnhrdWFiaVZXQm1O?=
 =?utf-8?B?dEhWUWpQcnBrTW0yUXFISjlGZXlQaExzZGRhRFdQUWJianNkU3dMUUs3NTFO?=
 =?utf-8?B?UFpPTkFLbmlqTmNnVUh1MWg1UUlTdE1NcFhQQXVFVnBxK0xDUUE0TnVMcmxW?=
 =?utf-8?B?Q1g4NWdFZzBENkNCTlFyWnFUalYzSjRDejNnTm1EbHYrYW12WjQ1VURlNGNU?=
 =?utf-8?B?UG9kQ2V6TlB2YXdNSkEyaG1LUnVqSXR6blNBRFNkZmRxS2ZJR3VoWHd6VUx1?=
 =?utf-8?B?bkxraDFYR24yV2x0TDZFT1llSXJ0REhidDlIdzNWOW85U0xSckRsd0xOcHdB?=
 =?utf-8?B?TWtnWUt0ZXFwVnVhQVorbkFRZXVyVkl1aUgwS3diSmJYcTV0dmJEZnp2Nko4?=
 =?utf-8?B?N3JvUlNxZ2ZPeEI4SlhpS0JLdElkcG1oUkh6SDhFd0lPRTBENHF1Vk1wb3dv?=
 =?utf-8?B?dTFaSjJLZTRTRXJKYzNtM0k5MldVdGpKTnE2ME9GRUphRjFNdWhuYlBkZStk?=
 =?utf-8?B?aDVyWVZaelZ5TWg3VzFXbmw1OGdyNXgyYThNSUVRU3I2WHpZb09nSFVIVndQ?=
 =?utf-8?B?V1Njd2M3SlpZelVnRWpTbVNKT3Y1alhYWGpVNkRScGlQbVJRa2tPbWtyN0hZ?=
 =?utf-8?B?a1ZxRnFuTHN6UXJxZ0RBRUsyWDZobnNKS3d0a0svM3U2WllvQkZLc2NFQUNr?=
 =?utf-8?B?WGs2OGZyNGFoaWVHK0QwM0dNQkV5Q25HQzJhUmNEc2JlVmVKVGpzbVNQZkZB?=
 =?utf-8?B?NGE5WlMraVZ4ZkVHeUZHWCt1ZmRHOGFKTmR3R2pnbmp1bEZmUDhBM1Mwd2p1?=
 =?utf-8?B?dG43MDdRUUZ3czlDYkJOZ0QyYVV4TUFlSVpaRy9GS0RPamFEakYyTlYxN1Zo?=
 =?utf-8?B?RmI5TG5FQzNIWkUyazdkSDRGNTVZUFhjY0xON2w4YTJYTEhXbDBLbzZaWlNL?=
 =?utf-8?Q?/BYGJBGBCi8t/GgEF/pxZf0dU9h8z4/0VAziSif?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 27edfd07-73c3-4b02-7129-08d97dd68d34
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:37:49.5913
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: d2fvotjU3HAu+neEK54noldsR/hVwwaZxWxqYqL0PFcoCE4nMS7n6UZFiWxiYLe9jpBqlRrC+Sz3l/5M8QjQTA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

IVHD entries may specify that ATS is to be blocked for a device or range
of devices. Honor firmware telling us so.

While adding respective checks I noticed that the 2nd conditional in
amd_iommu_setup_domain_device() failed to check the IOMMU's capability.
Add the missing part of the condition there, as no good can come from
enabling ATS on a device when the IOMMU is not capable of dealing with
ATS requests.

For actually using ACPI_IVHD_ATS_DISABLED, make its expansion no longer
exhibit UB.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: As an alternative to adding the missing IOMMU capability check, we
     may want to consider simply using dte->i in the 2nd conditional in
     amd_iommu_setup_domain_device().
Note that while ATS enabling/disabling gets invoked without any locks
held, the two functions should not be possible to race with one another
for any individual device (or else we'd be in trouble already, as ATS
might then get re-enabled immediately after it was disabled, with the
DTE out of sync with this setting).
---
v7: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -120,6 +120,7 @@ struct ivrs_mappings {
     uint16_t dte_requestor_id;
     bool valid:1;
     bool dte_allow_exclusion:1;
+    bool block_ats:1;
 
     /* ivhd device data settings */
     uint8_t device_flags;
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -55,8 +55,8 @@ union acpi_ivhd_device {
 };
 
 static void __init add_ivrs_mapping_entry(
-    uint16_t bdf, uint16_t alias_id, uint8_t flags, bool alloc_irt,
-    struct amd_iommu *iommu)
+    uint16_t bdf, uint16_t alias_id, uint8_t flags, unsigned int ext_flags,
+    bool alloc_irt, struct amd_iommu *iommu)
 {
     struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(iommu->seg);
 
@@ -66,6 +66,7 @@ static void __init add_ivrs_mapping_entr
     ivrs_mappings[bdf].dte_requestor_id = alias_id;
 
     /* override flags for range of devices */
+    ivrs_mappings[bdf].block_ats = ext_flags & ACPI_IVHD_ATS_DISABLED;
     ivrs_mappings[bdf].device_flags = flags;
 
     /* Don't map an IOMMU by itself. */
@@ -499,7 +500,7 @@ static u16 __init parse_ivhd_device_sele
         return 0;
     }
 
-    add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, false,
+    add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, 0, false,
                            iommu);
 
     return sizeof(*select);
@@ -545,7 +546,7 @@ static u16 __init parse_ivhd_device_rang
     AMD_IOMMU_DEBUG(" Dev_Id Range: %#x -> %#x\n", first_bdf, last_bdf);
 
     for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
-        add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting,
+        add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting, 0,
                                false, iommu);
 
     return dev_length;
@@ -580,7 +581,7 @@ static u16 __init parse_ivhd_device_alia
 
     AMD_IOMMU_DEBUG(" Dev_Id Alias: %#x\n", alias_id);
 
-    add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, true,
+    add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, 0, true,
                            iommu);
 
     return dev_length;
@@ -636,7 +637,7 @@ static u16 __init parse_ivhd_device_alia
 
     for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
         add_ivrs_mapping_entry(bdf, alias_id, range->alias.header.data_setting,
-                               true, iommu);
+                               0, true, iommu);
 
     return dev_length;
 }
@@ -661,7 +662,8 @@ static u16 __init parse_ivhd_device_exte
         return 0;
     }
 
-    add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting, false, iommu);
+    add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting,
+                           ext->extended_data, false, iommu);
 
     return dev_length;
 }
@@ -708,7 +710,7 @@ static u16 __init parse_ivhd_device_exte
 
     for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
         add_ivrs_mapping_entry(bdf, bdf, range->extended.header.data_setting,
-                               false, iommu);
+                               range->extended.extended_data, false, iommu);
 
     return dev_length;
 }
@@ -800,7 +802,7 @@ static u16 __init parse_ivhd_device_spec
 
     AMD_IOMMU_DEBUG("IVHD Special: %pp variety %#x handle %#x\n",
                     &PCI_SBDF2(seg, bdf), special->variety, special->handle);
-    add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, true,
+    add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, 0, true,
                            iommu);
 
     switch ( special->variety )
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -105,6 +105,7 @@ static int __must_check amd_iommu_setup_
     int req_id, valid = 1, rc;
     u8 bus = pdev->bus;
     struct domain_iommu *hd = dom_iommu(domain);
+    const struct ivrs_mappings *ivrs_dev;
 
     if ( QUARANTINE_SKIP(domain) )
         return 0;
@@ -122,20 +123,18 @@ static int __must_check amd_iommu_setup_
     req_id = get_dma_requestor_id(iommu->seg, PCI_BDF2(bus, devfn));
     table = iommu->dev_table.buffer;
     dte = &table[req_id];
+    ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id];
 
     spin_lock_irqsave(&iommu->lock, flags);
 
     if ( !dte->v || !dte->tv )
     {
-        const struct ivrs_mappings *ivrs_dev;
-
         /* bind DTE to domain page-tables */
         amd_iommu_set_root_page_table(
             dte, page_to_maddr(hd->arch.amd.root_table),
             domain->domain_id, hd->arch.amd.paging_mode, valid);
 
         /* Undo what amd_iommu_disable_domain_device() may have done. */
-        ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id];
         if ( dte->it_root )
         {
             dte->int_ctl = IOMMU_DEV_TABLE_INT_CONTROL_TRANSLATED;
@@ -146,6 +145,7 @@ static int __must_check amd_iommu_setup_
         dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
 
         if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+             !ivrs_dev->block_ats &&
              iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
             dte->i = ats_enabled;
 
@@ -166,6 +166,8 @@ static int __must_check amd_iommu_setup_
     ASSERT(pcidevs_locked());
 
     if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+         !ivrs_dev->block_ats &&
+         iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
          !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
     {
         if ( devfn == pdev->devfn )
--- a/xen/include/acpi/actbl2.h
+++ b/xen/include/acpi/actbl2.h
@@ -851,7 +851,7 @@ struct acpi_ivrs_device8b {
 
 /* Values for extended_data above */
 
-#define ACPI_IVHD_ATS_DISABLED      (1<<31)
+#define ACPI_IVHD_ATS_DISABLED      (1u << 31)
 
 /* Type 72: 8-byte device entry */
 



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:38:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:38:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192883.343602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3O5-0004M1-86; Wed, 22 Sep 2021 14:38:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192883.343602; Wed, 22 Sep 2021 14:38:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3O5-0004Lr-3F; Wed, 22 Sep 2021 14:38:17 +0000
Received: by outflank-mailman (input) for mailman id 192883;
 Wed, 22 Sep 2021 14:38:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3O3-0004LN-L9
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:38:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a290730d-b82a-4d9c-93cb-14917531d40b;
 Wed, 22 Sep 2021 14:38:14 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-0PX2CnW2O9OHHYZEJFoY4g-1; Wed, 22 Sep 2021 16:38:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:38:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:38:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0171.eurprd02.prod.outlook.com (2603:10a6:20b:28e::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:38:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a290730d-b82a-4d9c-93cb-14917531d40b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321493;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=L4OzGEj91QdJC5vdoESTAykpaiWdGj46EwoZ9sMMZn8=;
	b=eRVJHYuSPX7jhAb0P4sKFMEHSz0L1usuj0D6+tBXwciklhM4/GeWhUFJiR/wcI6XRIp4/6
	6bi1xMl78S/RNyDFH2m+n9jPQMA/JqF6Qn90xeLKgZs92Xc7eaDugWIhFlgjq8ZxKyvynN
	wyKZe1Lv/v2igNNKG9F9RMK9GJIkYbw=
X-MC-Unique: 0PX2CnW2O9OHHYZEJFoY4g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fc/ksO43unMARgcWrNAODMghjoBwYq0onMQNlotNsMIfjU2jZUA+5pEfyyUepztttEAmCQa/U3kSXEg9EqxFJjTOnjofB+zQmibDftehe38kw0b3ogzLhgr5oLFo3BtX2XcezWzV32SYqXb5hgHklG63a1IrfI8aMfoJaUa22UBSeIpiGlB3lHUDh0tBOBbL5qw5Lwpao+oNS+J/HMsF4SEHaGKrxffLp1InGuD2b+TL5fLdW9IL0ZDCo006yGGaFw9vhnzTiccpN0zygFnQ/MlC6uGIMBDpxaHglV46RfR+/EOhXh3zC0xOChEjfeiY1ZIud+oEuFUz6BjqPMJ4Jg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=L4OzGEj91QdJC5vdoESTAykpaiWdGj46EwoZ9sMMZn8=;
 b=lwUwJSZDUNVr5P2xHsnGjGYmzVxPWT4DBJ76dX6DxO6J0NwxlkOGwEzU4TEjVgvKyT9QSHXjPHJrK+nHJ1bz3Ow0NZQIxqfAXxsQAl2cOBjhk47dLaZlslmhZBctI1OKEvDQfp783HY5ZQQiC2FQ2A8ZrkRORyJ4LaynxQrW6mnNJoQHe4SLTne/0/IxDAUZtVYA1enYGYMH9BTlPJ3Sg6bJ2qNEbt8/hvbkGf7nNRYE58xmN5JOO7A8qi8MNBvuYSz46ZcTsuyJ5mBJFctRFn2KhGRfYqhO5pK4xrRmlaLWdWIG8zaR6I5p7MDVENQRN7qEOE6MzzOscF2bk/9p/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 5/6] AMD/IOMMU: pull ATS disabling earlier
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <34c081bd-ac6d-055c-e6c1-eeac62851389@suse.com>
Date: Wed, 22 Sep 2021 16:38:10 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0171.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4a0bd19c-454f-43f6-72d5-08d97dd69a0b
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70245F229E9EBC1A5C2ACDADB3A29@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WHHKaRBBGCyTAzxb+ZeAonbYdx57SgjqKWPZgfh+06yjnPo7oafVQJVwtTZgOierleiIXe89jsyU7qMnOJUlOFa+sRKVNS4NZ9FT9g+sePZ2yfRaK7XkRjU93ESvhjelIHwE9Z2ppSXGaV3zz/mt5yCuTFDqB98UjInggXRERE+gUfY5vTShc8jzsbUZ23czTxXuFLY7dfS0EzIIOPZDc16OQiRcM+GmQMTivusumZUnngxCNLWsaS+p/NdW/+4q1uovk1RJiuIpUkEoAQ/2+h2MYOofkqonF0HutEksQSqh/Xnc1wb5ZxQ4UyjOQDKODQh/xIgh72l6hVrmGsw71NctKgsFmdTPuWa7UT+scOwTTNtz1hYNiYtmOSCM8Qg+TTCC8OhWe2Yhlx+S7zSX7LB/YXLzu/V3EhLPcY/ZGRp1qJUzy5Q3bYTWHi2HPW/1Z1Gtl+jgOmiwRouW16N56eJEE7vme5vARShrBlbSWlTRCFUKoz4QiQaRrm7PH7T6q6FJVlneGXuIPwQp/9pG0UGDZDy1jgQ93uRD3r0gr4ObpJ2QYukiw6gIg2fpCvlgpvRExiFBb8XB6mo7n0JhqmnAD+lvHHAXsG490DdDQ5v39NUHT7uSK6wR0UXciPbElhzaLZW18g6geXp0xL2KJkzcvGDupHvksemtYBRpgkPjACukRTUHL3S9jA/w+5vxLZVPVFsVsGtFUkjO4fiQz8VaziOL3AfFOu/slvBLuqc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UmNhSTR1N3I4aXRrWlpiUDNBdERHd1V5SVozVGVoSENLQjl2KzZpazVKV2JP?=
 =?utf-8?B?N2oyaHN4MC9YeUlVckNTbnNDelA4NGpzbEp3djQ1UFk4SnhCTWVEaWN1V0xC?=
 =?utf-8?B?bmRCUnhPZzJXMkRQdGFWczAzOXZwZGNpSTZrVUZxZ25ZcFRhendQbVpIbHR4?=
 =?utf-8?B?Skt3Y3ZZRmxyV3EzZ3F5ODA0YmNXRkIwN3QyS3JncTduWFZ6d0owNmF6WUVT?=
 =?utf-8?B?WEJFUHZGS0xIelpVT0lQOFkwQlpkTmZQMVpUU0p6WjZQOGZZdFQ5ay9LQzAz?=
 =?utf-8?B?U0hTZGVmYlM1bURNdExNRjIrV2xoczBNbFJ5TXRKdzhhcWVxTDdHM1R5RjFi?=
 =?utf-8?B?Tm5YYjVMeXlvTnV6Wnd1Um54Vkd5YVJDMTR5MFdRR2R6L21VMEUybW5WWlVZ?=
 =?utf-8?B?MHEzclBJa3hKS0hmOEQwUkwwb2V1OFdVY2MzSVN4U3I5eTZNZzc3V0VHTDZx?=
 =?utf-8?B?OUZkZkFnNzl1elFORWtFdmJvOElnQmRJWkUvUExGRXNYb3BXSDcxc1MzWEZL?=
 =?utf-8?B?S2YyYU9tZXRXdzRleWNrdVRoWE5FdVZQTWtwUDg4NVMvbVZyandaMXUrMCtQ?=
 =?utf-8?B?dTZEcW1PMWlWYVlUWUIzVmUrRTliTnlhc0psNU1wWVZGQWlGYURMV2J1Z1FJ?=
 =?utf-8?B?eWVvYXd4TGplV0JpS0dvcTNoVXl0TzlsckI3TktPUHI1anVZWDdoT29tVHBP?=
 =?utf-8?B?WGxNdzZDUDcrZkp2S2dSTDd1ekJrQVE3Z0IxSDVJQlNRZ0ZIRjZZMmQ2M3Mw?=
 =?utf-8?B?REZLQVpBbnlseGJ1RDIrWXlPK3ZRYm1nQjFmQUQ1UEdkUUtRUjk0WnlGbGVC?=
 =?utf-8?B?MkM3QlJYU2YxdG9jS2JCRGMvTXBXZHdDVVVYL0VhQVQ3VFNidWJzOW92ZmhS?=
 =?utf-8?B?dHl0R3U1NGJON2xNa0pwK0hySTlSTWxoVEI1RDBlTU52RzFDYzMvWWNlZ1lp?=
 =?utf-8?B?eVdUQ1Babm1Cbk00ZXBJenBIWmFaZWpzcUVCSUlNTlpabDZIdVFxTU14ZE9w?=
 =?utf-8?B?eWZ1dXZ4endTK1ptc2pyRFFnaUpDL3k4WkJFYUFYbEJrdG9PeDMvUVI1Yng5?=
 =?utf-8?B?dVNTWURPR0hIZjZKRk5qdUpmeGUvUXZMSFJXUlJQMEx6b25xQm1wL3NjYzdS?=
 =?utf-8?B?K1RiZEIvSWdJWjNkaTZCOC9UT0NERkhGVjBUZkF6TDMvc0F4Y1hYMmFCTjBn?=
 =?utf-8?B?anltWDE3L3hndWtGRTRtMzk4OVJhdVZQMzdSRnlIamdTc3VjNGcwVDR0cURm?=
 =?utf-8?B?M01hVUtNVWN0VHZoZVNtUzNVbEpnVVV0aWtZcEUySUlqVWQrL1VjaDAzRzBY?=
 =?utf-8?B?aDE0aXBVS3hLRTFzYm52Y1NtS25iSUV3dU5yTUhCdkdLVmF5MnB2bE5Nams5?=
 =?utf-8?B?NlJwVGJaWGE4Rjk3Q0prb0JjSHNiSlRhM1d5TlA0cFlyQkpjRnU1UkdtdzlV?=
 =?utf-8?B?cVpwK0E2aEFlSlgzMlBwMklnNlBvbDA2R2YyejBtZFY1OFN3aVUvVWUyY2Z4?=
 =?utf-8?B?UDIyUlVwZzRJVnJWYWtVdEtrMzNEME9hTEtUT21QQkZaZU9VY2g0QkVDOFZ1?=
 =?utf-8?B?RkEzUFpnSk9EZGZ0TzZCVTM5UUJPSlZLSEdQMm5BeEVPb1dCc0l3SXQ4VHBm?=
 =?utf-8?B?MWNFRWZtNnRLTCtVM0pZc1N2UkdjNEF6Mkp1LzRiT1Z5c0ZPWER5Q2M3VEc3?=
 =?utf-8?B?RmdENk5nMlJuZGRqTlV6UEk5MGN3ZjZUOWxxUkFtSkl2VWxSM1dNcDZJb2lU?=
 =?utf-8?Q?44S0i8i/gIJBiboW32N1RsOg8J8wLCpAzGLksz/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a0bd19c-454f-43f6-72d5-08d97dd69a0b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:38:11.1350
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KwwRN+RDChHsgpPzEKBSPp5kM7vIOcDMCggOsBmfEZx+e1lRjTZkua321RhAJmAkcoTR5Cvh6Y6tDNA5+QnZxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

Disabling should be done in the opposite order of enabling: ATS wants to
be turned off before adjusting the DTE, just like it gets enabled only
after the DTE was suitably prepared. Note that we want ATS to be
disabled as soon as any of the DTEs involved in the handling of a device
(including phantom devices) gets adjusted respectively. For this reason
the "devfn == pdev->devfn" of the original conditional gets dropped.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: This points out that for phantom devices the ordering in
     amd_iommu_setup_domain_device() may also not be fully suitable: ATS
     would better be enabled on the device only after all involved DTEs
     have got prepared. This would be a less straightforward change,
     though.
---
v8: New.

--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -313,6 +313,12 @@ static void amd_iommu_disable_domain_dev
     if ( QUARANTINE_SKIP(domain) )
         return;
 
+    ASSERT(pcidevs_locked());
+
+    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+         pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
+        disable_ats_device(pdev);
+
     BUG_ON ( iommu->dev_table.buffer == NULL );
     req_id = get_dma_requestor_id(iommu->seg, PCI_BDF2(bus, devfn));
     table = iommu->dev_table.buffer;
@@ -348,13 +354,6 @@ static void amd_iommu_disable_domain_dev
     }
     else
         spin_unlock_irqrestore(&iommu->lock, flags);
-
-    ASSERT(pcidevs_locked());
-
-    if ( devfn == pdev->devfn &&
-         pci_ats_device(iommu->seg, bus, devfn) &&
-         pci_ats_enabled(iommu->seg, bus, devfn) )
-        disable_ats_device(pdev);
 }
 
 static int reassign_device(struct domain *source, struct domain *target,



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:38:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:38:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192892.343613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3OS-000549-Mb; Wed, 22 Sep 2021 14:38:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192892.343613; Wed, 22 Sep 2021 14:38:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3OS-000542-J3; Wed, 22 Sep 2021 14:38:40 +0000
Received: by outflank-mailman (input) for mailman id 192892;
 Wed, 22 Sep 2021 14:38:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3OS-0004ys-8Q
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:38:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2b2ff7cc-b033-4d38-b88e-1507635c57dd;
 Wed, 22 Sep 2021 14:38:35 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-E95VvMe0PpS4HE8kUjRD6A-1; Wed, 22 Sep 2021 16:38:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:38:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:38:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR04CA0127.eurprd04.prod.outlook.com (2603:10a6:207::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 22 Sep 2021 14:38:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b2ff7cc-b033-4d38-b88e-1507635c57dd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632321514;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=04x1FN662zVsOx4AF8ENkQRIOoZIpNC+jLSRni5iedE=;
	b=X592sk0E9+CYPDVnDEE5dHo0LkNfOwQrWsIuSXx7WSo0sfeHiYXV1NjZ6/e2aUAfi+jfy8
	HtAR27o2aeh4tGUl+W0t9hP9ioZp0CWjH5GMdmSRDS0V0Lb4Wn4A/te7brAXkE5uF43lBS
	Q6vGvHHJv5vB36ii12BeB68mK9yE+Wk=
X-MC-Unique: E95VvMe0PpS4HE8kUjRD6A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=miQD+jlnl/Wt42LIOFroovVzmLJbrBrSyKqHXs6ZZEj1K3Wo9R11YxzmC3WSAI57HNi7iaoOvi50yxeDzb1GAUkPov7+x6+CmxE1MayrvErSD1hpp2oZxx0JEABRADkEZmB8b/iN7L5OCMRXDR8/8HIpwm8UNTHlrJkx3gh5Jh286BYCM/br9/f34O/f1DXbAC8kGItZ8DoWi5DVJzQVECgcEVBTC7DbfTWpYNvv1xrZGGyvLEk9ISQt2LYvvrbUAvWm4c4/buO4tys4Zt4qz249R92tqN4gdzMOqB0udcrkeND3pD62ukdK+2kQ6oiDQffbGRwkmbFXr77jQ/qFEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=04x1FN662zVsOx4AF8ENkQRIOoZIpNC+jLSRni5iedE=;
 b=jxECi7dC9VTTmgtgNevgYWdIGAtE9FIDT0XoZTBLfU6CrbWDJrWFSDpJAaWss088JKtDmyKvHl2AhP6eeoQaKw32u1oy4X3D+tkRWWkcqK6TMh85JswdAwbeE0m69aWL8lqczLNOFBVPwtWtaMUR0ZKbLRODD1rl1sVOGENmBc6l79bxbyW/KDTJaqnMzoC5g2RuxFBGA6G2smy35jascfVTnA2sTyVM+n/JbuvePD0aS5ktnBiuJzb8d+LXo/Un7dJv0ruPXAYrdlFTaZQn6pt7aKKpaAdJFY28Ct9bMQ1vD+hPajO49FSnpEstFV7IR4JADHUS9EEZ6UZ7fa/UQw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v8 6/6] AMD/IOMMU: expose errors and warnings unconditionally
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Message-ID: <dc0cd7f7-a313-099d-3e89-e3862ed11f43@suse.com>
Date: Wed, 22 Sep 2021 16:38:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d249a759-188a-d689-316a-4743922827e5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR04CA0127.eurprd04.prod.outlook.com (2603:10a6:207::11)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 336a2ede-e6db-43f0-4424-08d97dd6a66c
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024C8271E1FF78F03A4396CB3A29@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:82;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4rIZEWo7cpE39OwVJ+uB7CrckO0bPNZv5ffJsjxsRRkmX0B7BPM7U2iadIAm8+T5f3c5d61kQGIdGVLQmd/FD9RzNmHptOjkUhU1TEng4z2cheIhhWpW3LV2k9W63HXEv1yZL9WfE/uTh2muiQuusnMNSfnLLzFJoVysYK9BQ0Zw3NdLFK5Gzjl8qFQszf+V6JHB+EB0qBIPkmhe4eqmHZsggwm+9HtF2pY6v1V1ET6Iqz+HYxOtrpbMZwTGAxwl80yYXIthJIEf5+Ym5cPFC4b8qKpgebF18zJMJtRXrTDpBY4FVwo5ekp1Bodp+fSAIYemW+9im7+vyaBw0FowqzMZcbPHCTSSIC+8NlJNJBHUpRXr0G+j6TmVCA6+I7Oq/6TpPnPt1GqyhTec1/Y8okuupyClh/Ir/JtWzFw+5N5GeArLwA4pmYVwQ/vbie6/d1hiHgG4BEXsIPur7YcyMzCtXXIH+6fFw7Mv2r0JKNlrvii6Ij992cVI3ITYA9Erea44FlGtkY6xPdIADoZav1HRGmQph2Ue1p4CDjTcgLPdzt2PsUsoz9cBL4HqGmr7ZDR1gTtNEnxopwqaLMcA9sD+bYhCOvEX5rqg2024V9psYyXPRs+uLL7rV82502ofJqhjDipZtrYTjR/fU6o/wtLvN5i3f8R4zL55Fa66teLUDi+nsvtcofmNJvZ12XjoNUf1rPYV7iCsViXy74WDnHwflZcv5ExRTMeMD48/r20=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(8936002)(66556008)(16576012)(2616005)(316002)(6916009)(4326008)(30864003)(66476007)(2906002)(508600001)(66946007)(86362001)(5660300002)(956004)(26005)(186003)(38100700002)(54906003)(31686004)(31696002)(8676002)(36756003)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?V0RieW9MVGdNOWtBRTgrMEozK1MvRmswUjZxdlBBdHhyR3ZqMVcxb25YTUFt?=
 =?utf-8?B?bUlwS1VZTWk5Vm1jdzFhbmhtTFVSTGhtdG9RQTVnVmJlT1FubEZtWjBYdGp2?=
 =?utf-8?B?dXk2aVZzMXVEUlRadFRKRm9hSTVrSmsyNWN6UjBGbjJRSkRWSERVZkJRa2Vu?=
 =?utf-8?B?YVFXOW04aXhVdGFVeDVuNWRkNDJmaGd5WmRCYk13R2lGbTB1akRlMWxWd09N?=
 =?utf-8?B?T2JPanJrdjhMd2FoNkxxTWpNS1JVemNiTUdJRnE0RXJPb3ZrcnZYQUR5cHcz?=
 =?utf-8?B?anA5UTlONnIyUStJdVBZSWxkbHZPV0piaDFaZEk4YTNMeWwrVEhPVGVhV2Jw?=
 =?utf-8?B?SE9UUUcwTE1ORDVweUpuS2FLNlYzTGJ6UENic1B4djdrNGIxcnZFeG9hQ0tS?=
 =?utf-8?B?MUZ6cG85Y1V2bC9SWTB4TENUT1V4Q25XWWg2Qm43ejJLTmxUWUVySDlzTllj?=
 =?utf-8?B?ZGZjL0R3TnhIT1duOXh4UVE2dnFaT0hYWGdSQUQ4bGdvTnFRdWpwbDZHeGZz?=
 =?utf-8?B?T2paUXhwVlQxNW05RWNCZU5TRkROTnVndjZnbStQZ3FKTVQxdVJIUEpuRDNw?=
 =?utf-8?B?S09lVWlBa2xvQUxURVRpNXZKN3dZcFJVcmFjT0RMZzVWQXE2aWdGWXVCelZF?=
 =?utf-8?B?UnFTc0pIOGt4cHJaNHV6VVhqOXlvVDdwUml6T3dlQlF6eXlwZjhzbkVLWHhL?=
 =?utf-8?B?L0dQbUY2aG5wVGlQbTlxK0M3U3BXOGlieUZsWlRkRjllVTlZemMwQzQvSkFW?=
 =?utf-8?B?VDJRY01GQ25VdDAvb0JtZld6TC9ObVh1a1p3N3o3MlBGMDFyekxBRUJJZkYw?=
 =?utf-8?B?MVFwL0tLT1FyeFhOWGVRaldVU1FaVFdZZEwrZ0JRa1ByV0t0MXpqa2pRZ1g1?=
 =?utf-8?B?cUFiQ2xROHhST2MrdTFmUnByekZVMm5BMzFPMmpTMjZuU1F2cmNlNUdTVGFi?=
 =?utf-8?B?QzRtZnFUK1E1dWVsS3FUYmg0SkRkdzMzUjNBaGNmd09jTzBiQVFKVW9icHo2?=
 =?utf-8?B?V0NiNmVrVGc5allFNFpON1VlL2JlTXM5dTZyWGorWGhhSkU1ZC9GWjBJQllX?=
 =?utf-8?B?SU90c3hNWWVzNTVGQkp0a3VYNDhSN1N4QjFFaE5VaERvNE9OM3NqNG1ZWll0?=
 =?utf-8?B?bE1LUFhONUtzVjJkL0dROE5MeHZoZjBTd2hTeGR5cU5ORmFkUGJSdnF3UlpK?=
 =?utf-8?B?bTdtMzRRVGNlRXZtNllMUEZlckhqaXQwY1FCY2F0NEN1OXd5bFhvTVFwdGNW?=
 =?utf-8?B?UzFCbGloalZwM3FGdGRmTUtqOXpiSDl1aENNK3haa1o2VUZTOWd4Y2d0VVEy?=
 =?utf-8?B?VHIvV1FrTXFiMy95S3RUaDZoRjRCRkpSNFdFVnVlU0htQ2NoZjZWWWN5TGJP?=
 =?utf-8?B?dFJ2OEdNNnNCZ1JiZXRFN0RETUNmT3pXNEJ5ZlJ3RXNPNnVzWG5IS0hjRjk2?=
 =?utf-8?B?YVYvc2FpZ251VzVLK09sd1RWeitYR0lzd1lyVkNpZHd0bzFicFBvaFl5YkdG?=
 =?utf-8?B?SkFCQ0pWVlZVU01IY1RUUHNuejBNN2ZzY1p6aEtTL2VDYWhmNjNLWUZGZy8y?=
 =?utf-8?B?R1BHdlkxbThJMUQzYU1pZHdIblZOYkhZb2grbmN3SG1yckhrandva1Mya0xT?=
 =?utf-8?B?U3YvUG9RZldPV2R6dVpuQTZHTUNuZ0Y2MGtZZGhncTNlVlhHY1lNOElBQ3Jz?=
 =?utf-8?B?QkswaHJvNlZUOVlGRktlNUJMcmtCN3hQSzVBWlJEcnRZMTVqZGwrY25oOGZG?=
 =?utf-8?Q?/L+zaJaRDV8aqUI/eCGg43M23moT7KWNbVp/jzW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 336a2ede-e6db-43f0-4424-08d97dd6a66c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:38:31.9561
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: p5RF9mDBppMBR0c0nUUQb5mTsGRYkk8StNK2WNKBkkQovI5ijNzOJKwvojM5Z2G7gtCbqvNWfse7D1xl7zPJUg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

Making these dependent upon "iommu=debug" isn't really helpful in the
field. Where touching respective code anyway also make use of %pp and
%pd.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
While I'm adding AMD_IOMMU_VERBOSE(), there aren't any uses for now.
It's not really clear to me where to draw the boundary to
AMD_IOMMU_DEBUG().

I didn't bother touching iommu_guest.c here.
---
v8: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -203,6 +203,18 @@ struct acpi_ivrs_hardware;
 
 #define DMA_32BIT_MASK  0x00000000ffffffffULL
 
+#define AMD_IOMMU_ERROR(fmt, args...) \
+    printk(XENLOG_ERR "AMD-Vi: Error: " fmt, ## args)
+
+#define AMD_IOMMU_WARN(fmt, args...) \
+    printk(XENLOG_WARNING "AMD-Vi: Warning: " fmt, ## args)
+
+#define AMD_IOMMU_VERBOSE(fmt, args...) \
+    do { \
+        if ( iommu_verbose ) \
+            printk(XENLOG_INFO "AMD-Vi: " fmt, ## args); \
+    } while ( false )
+
 #define AMD_IOMMU_DEBUG(fmt, args...) \
     do  \
     {   \
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -174,7 +174,7 @@ static int __init reserve_unity_map_for_
         if ( unity_map->addr + unity_map->length > base &&
              base + length > unity_map->addr )
         {
-            AMD_IOMMU_DEBUG("IVMD Error: overlap [%lx,%lx) vs [%lx,%lx)\n",
+            AMD_IOMMU_ERROR("IVMD: overlap [%lx,%lx) vs [%lx,%lx)\n",
                             base, base + length, unity_map->addr,
                             unity_map->addr + unity_map->length);
             return -EPERM;
@@ -248,7 +248,7 @@ static int __init register_range_for_dev
     iommu = find_iommu_for_device(seg, bdf);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x!\n", bdf);
+        AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x\n", bdf);
         return -ENODEV;
     }
     req = ivrs_mappings[bdf].dte_requestor_id;
@@ -318,7 +318,7 @@ static int __init parse_ivmd_device_sele
     bdf = ivmd_block->header.device_id;
     if ( bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: Invalid Dev_Id %#x\n", bdf);
+        AMD_IOMMU_ERROR("IVMD: invalid Dev_Id %#x\n", bdf);
         return -ENODEV;
     }
 
@@ -335,16 +335,14 @@ static int __init parse_ivmd_device_rang
     first_bdf = ivmd_block->header.device_id;
     if ( first_bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: "
-                        "Invalid Range_First Dev_Id %#x\n", first_bdf);
+        AMD_IOMMU_ERROR("IVMD: invalid Range_First Dev_Id %#x\n", first_bdf);
         return -ENODEV;
     }
 
     last_bdf = ivmd_block->aux_data;
     if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: "
-                        "Invalid Range_Last Dev_Id %#x\n", last_bdf);
+        AMD_IOMMU_ERROR("IVMD: invalid Range_Last Dev_Id %#x\n", last_bdf);
         return -ENODEV;
     }
 
@@ -367,7 +365,7 @@ static int __init parse_ivmd_device_iomm
                                     ivmd_block->aux_data);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x Cap %#x\n",
+        AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x Cap %#x\n",
                         ivmd_block->header.device_id, ivmd_block->aux_data);
         return -ENODEV;
     }
@@ -384,7 +382,7 @@ static int __init parse_ivmd_block(const
 
     if ( ivmd_block->header.length < sizeof(*ivmd_block) )
     {
-        AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Length!\n");
+        AMD_IOMMU_ERROR("IVMD: invalid block length\n");
         return -ENODEV;
     }
 
@@ -402,8 +400,8 @@ static int __init parse_ivmd_block(const
          (addr_bits < BITS_PER_LONG &&
           ((start_addr + mem_length - 1) >> addr_bits)) )
     {
-        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not IOMMU addressable\n",
-                        start_addr, start_addr + mem_length);
+        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not IOMMU addressable\n",
+                       start_addr, start_addr + mem_length);
         return 0;
     }
 
@@ -411,8 +409,8 @@ static int __init parse_ivmd_block(const
     {
         paddr_t addr;
 
-        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
-                        base, limit + PAGE_SIZE);
+        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
+                       base, limit + PAGE_SIZE);
 
         for ( addr = base; addr <= limit; addr += PAGE_SIZE )
         {
@@ -423,7 +421,7 @@ static int __init parse_ivmd_block(const
                 if ( e820_add_range(&e820, addr, addr + PAGE_SIZE,
                                     E820_RESERVED) )
                     continue;
-                AMD_IOMMU_DEBUG("IVMD Error: Page at %lx couldn't be reserved\n",
+                AMD_IOMMU_ERROR("IVMD: page at %lx couldn't be reserved\n",
                                 addr);
                 return -EIO;
             }
@@ -433,8 +431,7 @@ static int __init parse_ivmd_block(const
                            RAM_TYPE_UNUSABLE)) )
                 continue;
 
-            AMD_IOMMU_DEBUG("IVMD Error: Page at %lx can't be converted\n",
-                            addr);
+            AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);
             return -EIO;
         }
     }
@@ -448,7 +445,7 @@ static int __init parse_ivmd_block(const
     }
     else
     {
-        AMD_IOMMU_DEBUG("IVMD Error: Invalid Flag Field!\n");
+        AMD_IOMMU_ERROR("IVMD: invalid flag field\n");
         return -ENODEV;
     }
 
@@ -471,7 +468,8 @@ static int __init parse_ivmd_block(const
                                        iw, ir, exclusion);
 
     default:
-        AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Type!\n");
+        AMD_IOMMU_ERROR("IVMD: unknown block type %#x\n",
+                        ivmd_block->header.type);
         return -ENODEV;
     }
 }
@@ -481,7 +479,7 @@ static u16 __init parse_ivhd_device_padd
 {
     if ( header_length < (block_length + pad_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
@@ -496,7 +494,7 @@ static u16 __init parse_ivhd_device_sele
     bdf = select->header.id;
     if ( bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf);
         return 0;
     }
 
@@ -515,14 +513,13 @@ static u16 __init parse_ivhd_device_rang
     dev_length = sizeof(*range);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     if ( range->end.header.type != ACPI_IVRS_TYPE_END )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: End_Type %#x\n",
+        AMD_IOMMU_ERROR("IVHD Error: invalid range: End_Type %#x\n",
                         range->end.header.type);
         return 0;
     }
@@ -530,16 +527,14 @@ static u16 __init parse_ivhd_device_rang
     first_bdf = range->start.header.id;
     if ( first_bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: First Dev_Id %#x\n", first_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf);
         return 0;
     }
 
     last_bdf = range->end.header.id;
     if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: Last Dev_Id %#x\n", last_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf);
         return 0;
     }
 
@@ -561,21 +556,21 @@ static u16 __init parse_ivhd_device_alia
     dev_length = sizeof(*alias);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     bdf = alias->header.id;
     if ( bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf);
         return 0;
     }
 
     alias_id = alias->used_id;
     if ( alias_id >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Alias Dev_Id %#x\n", alias_id);
+        AMD_IOMMU_ERROR("IVHD: invalid Alias Dev_Id %#x\n", alias_id);
         return 0;
     }
 
@@ -597,14 +592,13 @@ static u16 __init parse_ivhd_device_alia
     dev_length = sizeof(*range);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     if ( range->end.header.type != ACPI_IVRS_TYPE_END )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: End_Type %#x\n",
+        AMD_IOMMU_ERROR("IVHD: invalid range: End_Type %#x\n",
                         range->end.header.type);
         return 0;
     }
@@ -612,16 +606,14 @@ static u16 __init parse_ivhd_device_alia
     first_bdf = range->alias.header.id;
     if ( first_bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: First Dev_Id %#x\n", first_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf);
         return 0;
     }
 
     last_bdf = range->end.header.id;
     if ( last_bdf >= ivrs_bdf_entries || last_bdf <= first_bdf )
     {
-        AMD_IOMMU_DEBUG(
-            "IVHD Error: Invalid Range: Last Dev_Id %#x\n", last_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf);
         return 0;
     }
 
@@ -651,14 +643,14 @@ static u16 __init parse_ivhd_device_exte
     dev_length = sizeof(*ext);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     bdf = ext->header.id;
     if ( bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf);
         return 0;
     }
 
@@ -677,14 +669,13 @@ static u16 __init parse_ivhd_device_exte
     dev_length = sizeof(*range);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     if ( range->end.header.type != ACPI_IVRS_TYPE_END )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: End_Type %#x\n",
+        AMD_IOMMU_ERROR("IVHD: invalid range: End_Type %#x\n",
                         range->end.header.type);
         return 0;
     }
@@ -692,16 +683,14 @@ static u16 __init parse_ivhd_device_exte
     first_bdf = range->extended.header.id;
     if ( first_bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: First Dev_Id %#x\n", first_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: First Dev_Id %#x\n", first_bdf);
         return 0;
     }
 
     last_bdf = range->end.header.id;
     if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: "
-                        "Invalid Range: Last Dev_Id %#x\n", last_bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid range: Last Dev_Id %#x\n", last_bdf);
         return 0;
     }
 
@@ -789,14 +778,14 @@ static u16 __init parse_ivhd_device_spec
     dev_length = sizeof(*special);
     if ( header_length < (block_length + dev_length) )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
         return 0;
     }
 
     bdf = special->used_id;
     if ( bdf >= ivrs_bdf_entries )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf);
+        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf);
         return 0;
     }
 
@@ -844,12 +833,12 @@ static u16 __init parse_ivhd_device_spec
             {
                 if ( ioapic_sbdf[idx].bdf == bdf &&
                      ioapic_sbdf[idx].seg == seg )
-                    AMD_IOMMU_DEBUG("IVHD Warning: Duplicate IO-APIC %#x entries\n",
+                    AMD_IOMMU_WARN("IVHD: duplicate IO-APIC %#x entries\n",
                                     special->handle);
                 else
                 {
-                    printk(XENLOG_ERR "IVHD Error: Conflicting IO-APIC %#x entries\n",
-                           special->handle);
+                    AMD_IOMMU_ERROR("IVHD: conflicting IO-APIC %#x entries\n",
+                                    special->handle);
                     if ( amd_iommu_perdev_intremap )
                         return 0;
                 }
@@ -944,7 +933,7 @@ static int __init parse_ivhd_block(const
 
     if ( ivhd_block->header.length < hdr_size )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Block Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid block length\n");
         return -ENODEV;
     }
 
@@ -953,7 +942,7 @@ static int __init parse_ivhd_block(const
                                     ivhd_block->capability_offset);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: No IOMMU for Dev_Id %#x Cap %#x\n",
+        AMD_IOMMU_ERROR("IVHD: no IOMMU for Dev_Id %#x Cap %#x\n",
                         ivhd_block->header.device_id,
                         ivhd_block->capability_offset);
         return -ENODEV;
@@ -1016,7 +1005,8 @@ static int __init parse_ivhd_block(const
                 ivhd_block->header.length, block_length, iommu);
             break;
         default:
-            AMD_IOMMU_DEBUG("IVHD Error: Invalid Device Type!\n");
+            AMD_IOMMU_WARN("IVHD: unknown device type %#x\n",
+                           ivhd_device->header.type);
             dev_length = 0;
             break;
         }
@@ -1113,8 +1103,7 @@ static int __init parse_ivrs_table(struc
 
         if ( table->length < (length + ivrs_block->length) )
         {
-            AMD_IOMMU_DEBUG("IVRS Error: "
-                            "Table Length Exceeded: %#x -> %#lx\n",
+            AMD_IOMMU_ERROR("IVRS: table length exceeded: %#x -> %#lx\n",
                             table->length,
                             (length + ivrs_block->length));
             return -ENODEV;
@@ -1214,7 +1203,7 @@ static int __init get_last_bdf_ivhd(
 
     if ( ivhd_block->header.length < hdr_size )
     {
-        AMD_IOMMU_DEBUG("IVHD Error: Invalid Block Length!\n");
+        AMD_IOMMU_ERROR("IVHD: invalid block length\n");
         return -ENODEV;
     }
 
@@ -1261,7 +1250,8 @@ static int __init get_last_bdf_ivhd(
             dev_length = sizeof(ivhd_device->special);
             break;
         default:
-            AMD_IOMMU_DEBUG("IVHD Error: Invalid Device Type!\n");
+            AMD_IOMMU_WARN("IVHD: unknown device type %#x\n",
+                           ivhd_device->header.type);
             dev_length = 0;
             break;
         }
@@ -1327,7 +1317,7 @@ get_supported_ivhd_type(struct acpi_tabl
     checksum = acpi_tb_checksum(ACPI_CAST_PTR(uint8_t, table), table->length);
     if ( checksum )
     {
-        AMD_IOMMU_DEBUG("IVRS Error: Invalid Checksum %#x\n", checksum);
+        AMD_IOMMU_ERROR("IVRS: invalid checksum %#x\n", checksum);
         return -ENODEV;
     }
 
@@ -1340,8 +1330,7 @@ get_supported_ivhd_type(struct acpi_tabl
 
         if ( table->length < (length + ivrs_block->length) )
         {
-            AMD_IOMMU_DEBUG("IVRS Error: "
-                            "Table Length Exceeded: %#x -> %#lx\n",
+            AMD_IOMMU_ERROR("IVRS: table length exceeded: %#x -> %#lx\n",
                             table->length,
                             (length + ivrs_block->length));
             return -ENODEV;
--- a/xen/drivers/passthrough/amd/iommu_cmd.c
+++ b/xen/drivers/passthrough/amd/iommu_cmd.c
@@ -291,8 +291,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
 
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("%s: Can't find iommu for %pp\n",
-                        __func__, &pdev->sbdf);
+        AMD_IOMMU_WARN("can't find IOMMU for %pp\n", &pdev->sbdf);
         return;
     }
 
--- a/xen/drivers/passthrough/amd/iommu_detect.c
+++ b/xen/drivers/passthrough/amd/iommu_detect.c
@@ -141,21 +141,21 @@ int __init amd_iommu_detect_one_acpi(
 
     if ( ivhd_block->header.length < sizeof(*ivhd_block) )
     {
-        AMD_IOMMU_DEBUG("Invalid IVHD Block Length!\n");
+        AMD_IOMMU_ERROR("invalid IVHD block length\n");
         return -ENODEV;
     }
 
     if ( !ivhd_block->header.device_id ||
         !ivhd_block->capability_offset || !ivhd_block->base_address)
     {
-        AMD_IOMMU_DEBUG("Invalid IVHD Block!\n");
+        AMD_IOMMU_ERROR("invalid IVHD block\n");
         return -ENODEV;
     }
 
     iommu = xzalloc(struct amd_iommu);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("Error allocating amd_iommu\n");
+        AMD_IOMMU_ERROR("cannot allocate amd_iommu\n");
         return -ENOMEM;
     }
 
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -386,8 +386,8 @@ static void iommu_reset_log(struct amd_i
 
     if ( log_run )
     {
-        AMD_IOMMU_DEBUG("Warning: Log Run bit %d is not cleared"
-                        "before reset!\n", run_bit);
+        AMD_IOMMU_WARN("Log Run bit %d is not cleared before reset\n",
+                       run_bit);
         return;
     }
 
@@ -754,8 +754,8 @@ static bool_t __init set_iommu_interrupt
     pcidevs_unlock();
     if ( !iommu->msi.dev )
     {
-        AMD_IOMMU_DEBUG("IOMMU: no pdev for %pp\n",
-                        &PCI_SBDF2(iommu->seg, iommu->bdf));
+        AMD_IOMMU_WARN("no pdev for %pp\n",
+                       &PCI_SBDF2(iommu->seg, iommu->bdf));
         return 0;
     }
 
@@ -799,7 +799,7 @@ static bool_t __init set_iommu_interrupt
     if ( ret )
     {
         destroy_irq(irq);
-        AMD_IOMMU_DEBUG("can't request irq\n");
+        AMD_IOMMU_ERROR("can't request irq\n");
         return 0;
     }
 
@@ -992,7 +992,7 @@ static void *__init allocate_buffer(unsi
 
     if ( buffer == NULL )
     {
-        AMD_IOMMU_DEBUG("Error allocating %s\n", name);
+        AMD_IOMMU_ERROR("cannot allocate %s\n", name);
         return NULL;
     }
 
@@ -1224,7 +1224,7 @@ static int __init alloc_ivrs_mappings(u1
     ivrs_mappings = xzalloc_array(struct ivrs_mappings, ivrs_bdf_entries + 1);
     if ( ivrs_mappings == NULL )
     {
-        AMD_IOMMU_DEBUG("Error allocating IVRS Mappings table\n");
+        AMD_IOMMU_ERROR("cannot allocate IVRS Mappings table\n");
         return -ENOMEM;
     }
     IVRS_MAPPINGS_SEG(ivrs_mappings) = seg;
--- a/xen/drivers/passthrough/amd/iommu_intr.c
+++ b/xen/drivers/passthrough/amd/iommu_intr.c
@@ -377,8 +377,8 @@ void amd_iommu_ioapic_update_ire(
     iommu = find_iommu_for_device(seg, bdf);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("Fail to find iommu for ioapic device id ="
-                        " %04x:%04x\n", seg, bdf);
+        AMD_IOMMU_WARN("failed to find IOMMU for IO-APIC @ %04x:%04x\n",
+                       seg, bdf);
         __io_apic_write(apic, reg, value);
         return;
     }
@@ -747,8 +747,8 @@ bool __init iov_supports_xt(void)
         if ( !find_iommu_for_device(ioapic_sbdf[idx].seg,
                                     ioapic_sbdf[idx].bdf) )
         {
-            AMD_IOMMU_DEBUG("No IOMMU for IO-APIC %#x (ID %x)\n",
-                            apic, IO_APIC_ID(apic));
+            AMD_IOMMU_WARN("no IOMMU for IO-APIC %#x (ID %x)\n",
+                           apic, IO_APIC_ID(apic));
             return false;
         }
     }
@@ -765,14 +765,12 @@ int __init amd_setup_hpet_msi(struct msi
 
     if ( hpet_sbdf.init == HPET_NONE )
     {
-        AMD_IOMMU_DEBUG("Failed to setup HPET MSI remapping."
-                        " Missing IVRS HPET info.\n");
+        AMD_IOMMU_ERROR("failed to setup HPET MSI remapping: missing IVRS HPET info\n");
         return -ENODEV;
     }
     if ( msi_desc->hpet_id != hpet_sbdf.id )
     {
-        AMD_IOMMU_DEBUG("Failed to setup HPET MSI remapping."
-                        " Wrong HPET.\n");
+        AMD_IOMMU_ERROR("failed to setup HPET MSI remapping: wrong HPET\n");
         return -ENODEV;
     }
 
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -222,7 +222,7 @@ static int iommu_pde_from_dfn(struct dom
             table = iommu_alloc_pgtable(d);
             if ( table == NULL )
             {
-                AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n");
+                AMD_IOMMU_ERROR("cannot allocate I/O page table\n");
                 unmap_domain_page(next_table_vaddr);
                 return 1;
             }
@@ -252,7 +252,7 @@ static int iommu_pde_from_dfn(struct dom
                 table = iommu_alloc_pgtable(d);
                 if ( table == NULL )
                 {
-                    AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n");
+                    AMD_IOMMU_ERROR("cannot allocate I/O page table\n");
                     unmap_domain_page(next_table_vaddr);
                     return 1;
                 }
@@ -301,7 +301,7 @@ int amd_iommu_map_page(struct domain *d,
     if ( rc )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Root table alloc failed, dfn = %"PRI_dfn"\n",
+        AMD_IOMMU_ERROR("root table alloc failed, dfn = %"PRI_dfn"\n",
                         dfn_x(dfn));
         domain_crash(d);
         return rc;
@@ -310,7 +310,7 @@ int amd_iommu_map_page(struct domain *d,
     if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, true) || !pt_mfn )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
+        AMD_IOMMU_ERROR("invalid IO pagetable entry dfn = %"PRI_dfn"\n",
                         dfn_x(dfn));
         domain_crash(d);
         return -EFAULT;
@@ -343,7 +343,7 @@ int amd_iommu_unmap_page(struct domain *
     if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, false) )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
+        AMD_IOMMU_ERROR("invalid IO pagetable entry dfn = %"PRI_dfn"\n",
                         dfn_x(dfn));
         domain_crash(d);
         return -EFAULT;
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -367,10 +367,8 @@ static int reassign_device(struct domain
     iommu = find_iommu_for_device(pdev->seg, bdf);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("Fail to find iommu."
-                        " %04x:%02x:%x02.%x cannot be assigned to dom%d\n",
-                        pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                        target->domain_id);
+        AMD_IOMMU_WARN("failed to find IOMMU: %pp cannot be assigned to %pd\n",
+                       &pdev->sbdf, target);
         return -ENODEV;
     }
 
@@ -484,8 +482,8 @@ static int amd_iommu_add_device(u8 devfn
             return 0;
         }
 
-        AMD_IOMMU_DEBUG("No iommu for %pp; cannot be handed to d%d\n",
-                        &pdev->sbdf, pdev->domain->domain_id);
+        AMD_IOMMU_WARN("no IOMMU for %pp; cannot be handed to %pd\n",
+                        &pdev->sbdf, pdev->domain);
         return -ENODEV;
     }
 
@@ -527,9 +525,8 @@ static int amd_iommu_add_device(u8 devfn
              pdev->domain,
              ivrs_mappings[ivrs_mappings[bdf].dte_requestor_id].unity_map,
              0) )
-        AMD_IOMMU_DEBUG("%pd: unity mapping failed for %04x:%02x:%02x.%u\n",
-                        pdev->domain, pdev->seg, pdev->bus, PCI_SLOT(devfn),
-                        PCI_FUNC(devfn));
+        AMD_IOMMU_WARN("%pd: unity mapping failed for %pp\n",
+                       pdev->domain, &pdev->sbdf);
 
     return amd_iommu_setup_domain_device(pdev->domain, iommu, devfn, pdev);
 }
@@ -547,7 +544,7 @@ static int amd_iommu_remove_device(u8 de
     iommu = find_iommu_for_device(pdev->seg, bdf);
     if ( !iommu )
     {
-        AMD_IOMMU_DEBUG("Fail to find iommu. %pp cannot be removed from %pd\n",
+        AMD_IOMMU_WARN("failed to find IOMMU: %pp cannot be removed from %pd\n",
                         &pdev->sbdf, pdev->domain);
         return -ENODEV;
     }
@@ -560,9 +557,8 @@ static int amd_iommu_remove_device(u8 de
     if ( amd_iommu_reserve_domain_unity_unmap(
              pdev->domain,
              ivrs_mappings[ivrs_mappings[bdf].dte_requestor_id].unity_map) )
-        AMD_IOMMU_DEBUG("%pd: unity unmapping failed for %04x:%02x:%02x.%u\n",
-                        pdev->domain, pdev->seg, pdev->bus, PCI_SLOT(devfn),
-                        PCI_FUNC(devfn));
+        AMD_IOMMU_WARN("%pd: unity unmapping failed for %pp\n",
+                       pdev->domain, &pdev->sbdf);
 
     if ( amd_iommu_perdev_intremap &&
          ivrs_mappings[bdf].dte_requestor_id == bdf &&



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:47:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:47:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192905.343624 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Ws-0006uW-Mo; Wed, 22 Sep 2021 14:47:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192905.343624; Wed, 22 Sep 2021 14:47:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3Ws-0006uP-JJ; Wed, 22 Sep 2021 14:47:22 +0000
Received: by outflank-mailman (input) for mailman id 192905;
 Wed, 22 Sep 2021 14:47:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=cZP0=OM=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mT3Wr-0006uJ-5y
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:47:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fcbe4ba0-1bb3-11ec-b9a5-12813bfff9fa;
 Wed, 22 Sep 2021 14:47:20 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2053.outbound.protection.outlook.com [104.47.8.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-r1r-x2qxNl6fNemDLRQG_g-1; Wed, 22 Sep 2021 16:47:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2445.eurprd04.prod.outlook.com (2603:10a6:800:55::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Wed, 22 Sep
 2021 14:47:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 14:47:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0184.eurprd02.prod.outlook.com (2603:10a6:20b:28e::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Wed, 22 Sep 2021 14:47:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcbe4ba0-1bb3-11ec-b9a5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632322039;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Bfmu3HUIKlA6vAGqrhy+xV1qYBAdmE0A2SBHOmvTyYI=;
	b=OMyLr8nn2CDQOh0PAtI7Gp5EEDbRkgU0kNtJ/4i1UrHi+fA1cTI2zf3b47nTM/T0EW5TDr
	x/wUbW7CUMv6qzvcLXQTx1w0Gp1RgCdBmfSua7QZItHKEBEcK02SByIj32otQH+8HAtEff
	x2EiSxsgb3FH94xOcfNEE6+aLAUZHEM=
X-MC-Unique: r1r-x2qxNl6fNemDLRQG_g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QdU5S+H02fY4dCdvfDEWO3C+FBNVGDQsikWwy3QuFcctShXzecnZrMEt5hax/KPIHso6igvWzN9jO/2AxFluIp2mp39QcYhx38AnzlycDDbRf+pwUd9zBtr0l0g5gsDS0KzA2U+1rn6vNEmFlktgzkCkC9NAf1Fr1Sg4v3evcPpHT1HuFMIZULr7bowI4HtdMbJwXFBarYRqUa/uUJG5dqL1FihqJfGx7IF/XjJnImc8AktB+A8nnseBvINewmBCcf9cBu0WaXbK0YPJJTKN98ogNhtV+UFj/7cyXFpxpIaVHYgNyq08WcETB/oT2cmbqO7BS1ywzMmZ7YWSFb9K6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TsZfJ/e/HjHx9vRm/4cDVIru8+MI9qsCuRKIHFtqCLY=;
 b=W4P8EPuI4pWz/xCVXQPY9PfFNFHFpXvuIKOgAf6sCQ7jDSs8JfDPRGXnAz8J3clRiTzV3pSqaHvynP3wmg7TctpydtuJwUz4VkFaWjefwkt5AjeKIqoiSPOdYPJqwH4hrVpM320sE7lVeonIwmcId1nmWPtTV1f+gM+aBssJgSgl+/BAyaEw4bHt0/h+8WLauXUXYyiCBdUTAWGYY1lDPjYQz7IcVdiIuSI7X/wtbEXE4h7jTZFV0ndryDkNBaD2Vzx5t9LvZ9ePdR4YXf3f/CdGAiZbNcYs70nExnIDmlkGfQEQXJz6Ha7HofIHfxCIXdn2KkcTyKN+4gJ+ECw09g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xenproject.org; dkim=none (message not signed)
 header.d=none;xenproject.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Ian Jackson <iwj@xenproject.org>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com>
 <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local>
 <58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com>
 <YUm3yLOfAfqrwgmH@MacBook-Air-de-Roger.local>
 <24905.47298.105494.711706@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ad798aae-0f8e-219f-59d4-7ff5fe52c984@suse.com>
Date: Wed, 22 Sep 2021 16:47:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24905.47298.105494.711706@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0184.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8fd9481a-fb7e-4152-f52a-08d97dd7def9
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2445:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24458AB6F199F50DE40E5111B3A29@VI1PR0401MB2445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kHxNpvNzC9tMf7DxRn/cyv20uD0a1GcIdk3m2ImOmz/t69fiMXMy4/TP4JmjhahjcnqXqslUgPi6auFEFrp9r+T+Z+vTj4t9vyUhZPOtzpi6GBW89Gm5DO7Z4IOHu8Xp6QnMPNHylBvO0w7NQJZh/vzyWlmAcKxMyrjrF/1L1mKnHRDwfRfBG0FKhiGzPQFCAWYAKt/zBEh+NFFYOI+euopmengWPvsRV0A/szhqwxcwBIEOEUJpLJdCl1nSLw8mtVHF9VwCS0A5OChbX0jcTywxS/0ob7ZeyPLZxWXEXNfer9fbkN3jcMiNBIbxfxgrv0Sl4mb1bvgzEMiIiUs7HUVplDAS7Cti5dxfjWSoWMJGJB7wJjka2hZedyFz1g5NVgf6MXDIoKVwwomUmgxTazNHN0aeT2QmnGLEkAIBlpKrKbvEQHJEW+0jzBe4uFCZbN1+CjFke/hWLalVm0g6tpzpFb7ks8ooYgPf4r5wiU9JDhrSMBWvqGghDQogmyQcTq6rM+wM+JTbWzPs3wwliFKZRTwNU9Yu1bZQGtBs5GwMHUu1UzShLV0ukXxnhSzWrlOkzB71yx1S93Cq2alm5iKZuhLybOu07DZVW5Pqft+DIWm1uB9X4oik4PhApv3UMrFzyT4SxF593e68cuf7Ympy4xhiCdiAxD6nnO/SvaVMCdzMOQ82d7XEMgyIB/aDV9Q39GbumolzHTqq7uxweoTObssQYSDJHuPr3sGD9ak=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(8676002)(110136005)(16576012)(54906003)(186003)(316002)(31686004)(36756003)(4326008)(38100700002)(6486002)(5660300002)(66946007)(66476007)(2906002)(66556008)(26005)(31696002)(86362001)(8936002)(956004)(2616005)(53546011)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?9ze457zc2B50HIXKIZyivvJ9dgNrx8LA2JZIHd3X/Fv6g8AyRNzgPTd+hVR+?=
 =?us-ascii?Q?dNan9JzO2nDXFVa3NsKml14CC9D/S4zQS3WNn7bqUNjtNpt2jg5mVT0rq+E9?=
 =?us-ascii?Q?hQ+husQs5GN/cN88Q0YWYsO0HaIxdjKXz40MW0n7MneHtLMf6R4GA+qiqlfe?=
 =?us-ascii?Q?D5rJWkrlJ7Q2J9x3Do1lZid1moEYoERzzTpxLVWEArxRMZHwQ6cGPcVLS0tu?=
 =?us-ascii?Q?KUAq1UqQKUb+BdAKN0GlVBba0xJIyv8o9u1Wxsac7o8wASyR6mqIYfRrgn6n?=
 =?us-ascii?Q?qJCcJCoBXdo2NyctPnULGwWmB/zyEChP0R0mRZEapxwC3mViqn2MLq3uzClf?=
 =?us-ascii?Q?8HLKIM+hol+rER7RZr1CVZdJ0kh1mxK5UA9V+yywC8+uZ0+07f8077bygF1l?=
 =?us-ascii?Q?XAFCqlTddAW0pTqqqB+O+a3XI4+uE0CzLhq9gbPAb85i9UMgtfZDAFhWXuho?=
 =?us-ascii?Q?P/vIp0KOqRDEgfESkkga/KsUoavdFJFlslZ8uJnRjAyMaRuXDsOPY4SRWrqz?=
 =?us-ascii?Q?KyI5HTdPQfHP5UQoLu9o/RqAHt+xQLqoEQo53fu2f5xGdUb+p0qRmRHNfH7k?=
 =?us-ascii?Q?iJYz62ohz9lF8NNJbx7GDk4ku6tC24nRccekubFdELRO6EQfIypGw0eDD1na?=
 =?us-ascii?Q?gf1a68EWwKfvMcsimXMQxBSX/h3PXcOQW55whfDbIEk+pyZt1y2gIprgtyV/?=
 =?us-ascii?Q?JBGNgXw85YTf8t6qGuatmsRjDE9uT0ors8rvgJesMkx5FDWU3V3Qf1/RTkSc?=
 =?us-ascii?Q?YzP6DYS9S+3tEMXOUx3WD0OkmBi0coL4OEldU7JYr+vkMBQAgcze3D0Iai8U?=
 =?us-ascii?Q?HxGxFPcWAHQ2VbmVKkcYdUii488g3HxpIZPyJ32cXDau0xgtIxAVxSgD/yip?=
 =?us-ascii?Q?Km5IHMDO4g42qIG/JWKe5lvM4Vo3FVNK0KZdvZMpC2+VkJ5wlxvO8J+KySZP?=
 =?us-ascii?Q?KraagaM4BknUH6wAfCKaI1siSMzvzz88IVLZJwwY1dQbyphwGSUcwDqLOcvL?=
 =?us-ascii?Q?gM8UC05InYM3dtOH37jRNVd2MymUbJBkjgDywpR9xsTKJsD4fEt7I6428/Iv?=
 =?us-ascii?Q?LDIjtgyCU15tinqrYbkFYJyC4OR4pqjR+r+cFW554085hgO5/y3dQzjD+iFQ?=
 =?us-ascii?Q?FrahHAgDtCOA4z9OTCzQ+kkF5Lmw2IbzXZQeTEJ2MB+LJZZSH9U4tR+ZeyKu?=
 =?us-ascii?Q?oFzBj2H648BacWjAH3hEd6CvzpXb+d79MM9Njkw6zO8ekXYkcP1CtcFuoNbD?=
 =?us-ascii?Q?ZgQYPmuJlmCezpd5DGiqFtx6gENuV0mvT48SajzPznfME/WG3oA4rwPrO1kC?=
 =?us-ascii?Q?a0Yin8hqYBh7UcgoLvijIJPv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fd9481a-fb7e-4152-f52a-08d97dd7def9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 14:47:16.5597
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1M8efMzuJNWFX2eiWLHlybqbkRs2Z8jDdClGY1HTrMW6vBwsh2jB5zlVv/xeK1JVex+KgAJA0ABDEcweBC5RWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2445

On 21.09.2021 12:49, Ian Jackson wrote:
> Roger Pau Monn=C3=A9 writes ("Re: [PATCH] common: guest_physmap_add_page(=
)'s return value needs checking"):
>> On Tue, Sep 21, 2021 at 12:28:12PM +0200, Jan Beulich wrote:
>>> On 21.09.2021 11:20, Roger Pau Monn=C3=A9 wrote:
>>>> On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
>>>>> The function may fail; it is not correct to indicate "success" in thi=
s
>>>>> case up the call stack. Mark the function must-check to prove all
>>>>> cases have been caught (and no new ones will get introduced).
>>>>>
>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>> Acked-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>>>
>>> Thanks. Albeit strictly speaking an ack here isn't enough for the chang=
e
>>> to go in, it would need to be R-b or come from a REST maintainer.
>>
>> Oh, FE:
>>
>> Reviewed-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>=20
> Acked-by: Ian Jackson <iwj@xenproject.org>

With these, any chance of getting an Arm side ack here as well?

Thanks, Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 14:49:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 14:49:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192911.343635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3YX-0007Wq-3N; Wed, 22 Sep 2021 14:49:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192911.343635; Wed, 22 Sep 2021 14:49:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3YW-0007Wj-Ux; Wed, 22 Sep 2021 14:49:04 +0000
Received: by outflank-mailman (input) for mailman id 192911;
 Wed, 22 Sep 2021 14:49:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W6jU=OM=google.com=seanjc@srs-us1.protection.inumbo.net>)
 id 1mT3YV-0007Wd-7a
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 14:49:03 +0000
Received: from mail-pf1-x429.google.com (unknown [2607:f8b0:4864:20::429])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 62ead880-a853-40af-b140-5da68f9021f2;
 Wed, 22 Sep 2021 14:49:02 +0000 (UTC)
Received: by mail-pf1-x429.google.com with SMTP id n23so89400pfv.4
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 07:49:02 -0700 (PDT)
Received: from google.com (157.214.185.35.bc.googleusercontent.com.
 [35.185.214.157])
 by smtp.gmail.com with ESMTPSA id x19sm3250084pgk.37.2021.09.22.07.49.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 Sep 2021 07:49:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62ead880-a853-40af-b140-5da68f9021f2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20210112;
        h=date:from:to:cc:subject:message-id:references:mime-version
         :content-disposition:in-reply-to;
        bh=0sqc+8Rn23i9NIXRc3rzWglOlIsYoDOCKK+U0vVnVf8=;
        b=i17P/LE0kBMgmTnemIWdlFQzqY0Hs2jbI6I1/q7BOVL5p2dtOUMx0v77u9KgBVc6Wn
         aKV+BPEmJqQFBnpf2JgmIIqtssY8WUcJgn0PyaL+KfSiTDztZbnAplBfnqcfDhlXXfFn
         BjrKCa5IqUkRjskJLByrIP0vUAdGBFoOpGk78YEBagXPKCi9T3kxBnFA97BcpKw7BZcR
         bmLTSv6ivPklBCfoVzjL+iJjEelTUzY+vAJf7MF3MbJsLpnqXAXpI4Q3beusWaRqYBkI
         4OoNyyAV9qr/c1FTp2tfQsv3gnSyBPHEnLObwapHJkjxKwND1CbJf6lm69JT2lSfnhHf
         qYCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-disposition:in-reply-to;
        bh=0sqc+8Rn23i9NIXRc3rzWglOlIsYoDOCKK+U0vVnVf8=;
        b=wMvtWDROrBy8D5UHPojilm/7Fp208HwMaxR3VmbH9h7AE/cFhaNUToSFh+GWW0UJDI
         SpVWBF0EyGWtw1AVcNWoCN+k3VuzptaTO+ZKrnJguxpnuarC5VHR9NOtYNEJ8i5WRb/w
         M03LZtL/gLUGweETtKQZY45HwXuW/FwGEKwExvGph2vJsWdQa0XXvMEpc02uYVKFmLFx
         A5kaCW6QmLvvd/VjRuAPuJ96kpOv6SJmIe2BWysCtGPChYXL9Qe4ZZQ3wf30M7GSwsM0
         bPMLSaTWSX2kNP89xyBuGA5o1xQiVw2GLzP+OXCRv49q1SPdJW9yh0LsrgcVvWs4SiYT
         eBLg==
X-Gm-Message-State: AOAM531KHu5ZqHKSsfqBCBBUoVR4uQF1JWoc78ymQcd9OyV/OOcSa1mZ
	6stGPqOaLdTv8tDaxt3RTvLMtA==
X-Google-Smtp-Source: ABdhPJyY3Fv9QcnunPtvdtmn1QYeYqhfMcgkYKWmFqTAaf0DAX+dPQteKaq7uTqZ/uQmOkkrqsksfg==
X-Received: by 2002:a62:5fc7:0:b0:438:9e3a:b8c0 with SMTP id t190-20020a625fc7000000b004389e3ab8c0mr22443pfb.74.1632322141372;
        Wed, 22 Sep 2021 07:49:01 -0700 (PDT)
Date: Wed, 22 Sep 2021 14:48:56 +0000
From: Sean Christopherson <seanjc@google.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
	Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
	Vincent Chen <deanbo422@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	James Morse <james.morse@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	linux-arm-kernel@lists.infradead.org,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org,
	linux-riscv@lists.infradead.org, kvm@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Artem Kashkanov <artem.kashkanov@intel.com>,
	Like Xu <like.xu.linux@gmail.com>,
	Zhu Lingshan <lingshan.zhu@intel.com>
Subject: Re: [PATCH v3 08/16] perf: Force architectures to opt-in to guest
 callbacks
Message-ID: <YUtCWOYJwCUYDYtW@google.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-9-seanjc@google.com>
 <f2ad98e2-ddfb-c688-65af-7ecbd8bc3b3d@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <f2ad98e2-ddfb-c688-65af-7ecbd8bc3b3d@redhat.com>

On Wed, Sep 22, 2021, Paolo Bonzini wrote:
> On 22/09/21 02:05, Sean Christopherson wrote:
> > @@ -1273,6 +1274,11 @@ static inline unsigned int perf_guest_handle_intel_pt_intr(void)
> >   }
> >   extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
> >   extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
> > +#else
> > +static inline unsigned int perf_guest_state(void)		 { return 0; }
> > +static inline unsigned long perf_guest_get_ip(void)		 { return 0; }
> > +static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }
> > +#endif /* CONFIG_GUEST_PERF_EVENTS */
> 
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> Having perf_guest_handle_intel_pt_intr in generic code is a bit off.  Of
> course it has to be in the struct, but the wrapper might be placed in
> arch/x86/include/asm/perf_event.h as well (applies to patch 7 as well).

Yeah, I went with this option purely to keep everything bundled together.  I have
no strong opinion.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 15:01:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 15:01:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192917.343646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3kH-0001WW-4f; Wed, 22 Sep 2021 15:01:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192917.343646; Wed, 22 Sep 2021 15:01:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT3kH-0001WP-0q; Wed, 22 Sep 2021 15:01:13 +0000
Received: by outflank-mailman (input) for mailman id 192917;
 Wed, 22 Sep 2021 15:01:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT3kF-0001WJ-D4
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 15:01:11 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eb7578ee-1bb5-11ec-b9aa-12813bfff9fa;
 Wed, 22 Sep 2021 15:01:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb7578ee-1bb5-11ec-b9aa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632322870;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=H/XeDceVUZmrCI+i2Zrio2zJqsWUdhQduoFMbScE0EI=;
  b=REQwoioQ0nbrG31NnyX7dYMAtfTbcyD2749U+P8wKwvE6jCHr6S1T+3E
   azYYzCwm+WqChDPru9AKZ87fBIzq5gAwiDLoo9n4KwsdnVZ0833odocb2
   nVWXUPd7dCRc5SofxL/t4RIMgN+Pq9pzA03RLPJp71PrHYhHgN12B0YRj
   A=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: cNkhPGJWi+ZqP0lk0cKxYdKWTku5w3uONyK3qVOBOPF9pS/oMG4/Fa/+/cV3n4VluNejvoINzl
 jgM2aKvpnbtlot+yiwHwxrrNYGwo5+o9DLE2HR3iGLXdgj6PPryH3H+uc8gwEBZzNis7NKUpAh
 B85gVlR2ldbXXfxEhhO50ypcfmaOaZfdWjlVtbQ0wrugcBc3ccT1F6S2mx3gQSO6abyfLbFnPz
 zbZ57y/smM7YLUxJ2L05pZ/4rqIEfenXauuFCRshqoFZMhcIPjYxfBmvExsWMWBQYudgUIs629
 yX/dbTEOCkc9dfyP4535bpCe
X-SBRS: 5.1
X-MesageID: 53328899
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:NpMRe6rmzH/8K1SBAzylLTn08KxeBmItYhIvgKrLsJaIsI4StFCzt
 garIBmBbq2OMTPye9klPoS2/UgG7ZPRyNAySVY6rHszQnkU85uZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0ILjW1jlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnb+2RlwLO7DmpMMEYV5qNBEgJbF3+LCSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFJkYtXx6iynQEN4tQIzZQrWM7thdtNs1rp0VRqyON
 5tIAdZpRE7MTBprI3UGMs4jzNv0nSTeX2BH913A8MLb5ECMlVcsgdABKuH9ZdiiVchT2EGCq
 Qru72n/Rx0XKtGb4T6E6W63wP/CmzvhX4AfH6H+8eRl6HWRzGEODBwdVXOgvOK0zEW5Xrpix
 1c8o3R06/JorQryE4e7D0bQTGO4UgA0eNxfSM8/9Sux7bPmpDS+NjYcUg9BZ4lz3CMpfgDGx
 mNljvuwW2c16e3LFSrEnluHhWjtYnlOdAfucQdBFFFcsoe5+OnfmzqSFo4LLUKjsjHi9dgcK
 RixpS4ijv04iccR3s1XFniW3mrx+vAlouMzjzg7v15JDCsiP+ZJhKTysDA3CMqsy67DFTG8U
 IAswZT20Qz3JcjleNaxrAAx8FaBvKztDdEhqQQ3Q8lJG8qFoib+FWyv3N2ODBgwaZtVEdMYS
 GTSpRlQ9Pdu0IiCNPQsC79d//8ClPC6ffy8D6i8RoMXPvBZKV/WlAkzNBX49z28zyARfVQXZ
 M7znTCEVi1BV8yKDVOeGo8g7FPc7npvnT+MHc+rkUvPPHj3TCf9dIrp+WCmN4gRxKiFvB/U4
 5BYMc6LwA9YS+rwfm/c9ot7ELzABSFnbXwvg8AIJOOFPCR8H2QtV63Yzb87ItQ3lKVJjObYu
 Hq6XxYAmlb4gHTGLySMa2xiN+yzDcou8ypjMHx+J0us1lgifZ2rsPUVeawocOR17+dk1/N1E
 aUIIp3SHvRVRz3b0D0Bdp2h/pd6fRGmiFvWbSqoaTQyZbB6QAnN9oO2dwfj7nBWXCG2qdE/s
 /ur0QaCGcgPQAFrDcD3bvOzzgzu4ShBybwqB0aRe4tdYkTh9oRuOhfdtP5vLpFeMwjHyxuby
 x2SXUUSq97SrtJn69LOn62F8dukSrMsAkpAEmDHxr+qLi2GrHG7yIpNXevULzDQUGT4pPera
 elPlqyuNfQGmBBBspZmEqYtxqU7voO9q7hfxwViPXPKc1X0Ve8wfijYhZFC5v9X27tUmQqqQ
 UbeqNBVNIKANN7hDFNMdhEuaf6O1K1MlzTfhRjvzJ4WOMOjEGK7bHhv
IronPort-HdrOrdr: A9a23:Tx6Wi6Oi9wqgRcBcT1r155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/uxoHJPwO080lKQFmrX5WI3NYOCIghrLEGgP1/qG/9SkIVyCygc/79
 YfT0EdMqyIMbESt6+Ti2PZYrVQseVvsprY/ds2p00dMj2CAJsQiTuRZDzrdnGfE2J9dOYE/d
 enl4B6jgvlXU5SQtWwB3EDUeSGj9rXlKj+aRpDIxI88gGBgR6h9ba/SnGjr1sjegIK5Y1n3X
 nOkgT/6Knmm/anyiXE32uWy5hNgtPuxvZKGcTJoMkILTfHjBquee1aKvC/lQFwhNvqxEchkd
 HKrRtlF8Nv60nJdmXwmhfp0xmI6kdm11bSjXujxVfzq83wQzw3T+Bbg5hCTxff4008+Plhza
 NixQuixttqJCKFuB64y8nDVhlsmEbxi2Eli/Qvg3tWVpZbQKNNrLYY4FheHP47bWzHAbgcYa
 pT5fznlbRrmQvwVQGdgoAv+q3iYp0LJGbHfqBY0fbllwS/nxhCvj0lLYIk7zA9HD9Ucegw2w
 3+CNUaqFh5dL5gUUtMPpZwfSKJMB2+ffvtChPbHb21LtBNB5ryw6SHlIndotvaPqA18A==
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="53328899"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LfzUsxEtMltTMgVOSBjatt8jbISTriFzUXLLWSZVii0byh+h1jLPsSmd40eXXkQPAD9R+c02cU68hLMgjkAc95rB0/nbSeYehU+y++Nx6G0CbsYBtrYB+A7rxgc6uLNOhIBOTKO/8oCqGQdaXOIGNe2nnHHpKz/ojU5650nXWqjkzSVE6vvMoAS5cwZ/tYtmvMaHVhmvUMXcZq0VqBNHj9n1F156uKXJpEqkthdk1PVspW6oFatnmXdtm6DlULqTh54TmE+G46QFjp7ZlqlSLdRoNxTtUQToV+NRi0/24tIDViiY9FS0BQR2ItgLKpEBFc5sOvBmDKX1x+qDmnneLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1h4IqLri73dplB8TL9as5DYbuvW6hilkS1CEbax3ywY=;
 b=mW4qyvlMGS1i9Fpq4Bppuz+S+DCD/2NdV/AfcXSwj8ArMtCOdQ03fQn/k56oSbjoj3p7cs9E2v+NM6fIItjJGFvWC07J2sqlJcPL19vZF/8P/ThVfjdhQs04K/MMrmhCPQX+5nMEYzCknEzhNlxHOLNCp5VL24VlvDDVfkzXf+Z26ixG8eFgnwEaMkiDlLtzIMBTWBJEKf5OWOZPyq1XP8X6+FUr95FcgltDrIKUEics/L0M1fXFCycdoP8/YnOXOF4G6DIGC5lXY6722biMr1/DNJLakTSBJb0I+A4iXYtNBqJLL8SGOUkJqxbP89aM19EUlYtO2iX5bdT2O2poCw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1h4IqLri73dplB8TL9as5DYbuvW6hilkS1CEbax3ywY=;
 b=CNTZ9KMjknOOWFUj54RbfZ1vWe/CaSfqJhSUagmyK+bX4s79iCx+47h2V2cGc82q+f1gFMUn/4K9psFIlRIKst0MG8oFjCKHQBMZDtgCigExVnvkpnB3za486GEBNu3y6WQ2lILbjyuY48btWmfnxtROjRc/Yd00dMzycRXGw74=
Date: Wed, 22 Sep 2021 17:01:00 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
Message-ID: <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
X-ClientProxiedBy: LO4P123CA0356.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18d::19) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b92f659d-2ac7-41f8-4344-08d97dd9ccee
X-MS-TrafficTypeDiagnostic: DS7PR03MB5590:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5590C8BFD4CD8D5A7B3F5BDE8FA29@DS7PR03MB5590.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: hudBDJVj/ApgrdcwONBTUfogWBbtW3/jF2KpfX5SBb6NO21t2/WF41oPmAa4m2LTBHHEfOi+9/QVvwBD5U/U4T+zDtYF2sVlcHzOwPrT6WQRX4L5AgwhYZb9tIQGpobWdbPF4ER5tZxpMB6DiWXdX3BplLh6DKKksKlAsCYdEHIjzKG9seHOYCsh/tgDNcDsSJw5npZHMW4jQNKFCrgxoDkHRd/3knJ1P+RklsHcvmCMbYJiuKYBzNbgWEDEbaRH7Wc6VPtrWr+zoPbQN/sd4d9y9NbX4CrJVB5dM6X5bHWeFeNjcj6sxGqIyF5KYOFkgxp9mwelc9185Gpwfhhb/Q1hWPgWJ3pST5Nn0QRSxz9SLkXraWIriqDhEHNARkQ9b3LyXbyp9rLJ3Jp0D7/8216VKpvXghSfiFXZKoiCVEkU7K37HLFF4QqeJDCpB49Zej4UauOU5dFBoKCdDJVrXaT+VtcV+sEQb6GoAibJ+vkgKPjDflZhU+CE4TFLiNgBJ8QNbD84FLfeJ7MfOVYVHJAdTEORlJhjIeNZ3XfGZM+BXQAPyNL+TxpZ75blU4CzkgbQie/uqLLExHoaCa+fgnMSOPNxzzKeVCpN+k65708qB2UwRORjvMBMJHQQ7Uy9JqYWdudqYZ0pgH773Iehhhx+uNuVdsPgLexGaDhlTomCFlV6OknAyx+VjjAl2dFxzQBMIufTi97snHZS3IGroQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(86362001)(8676002)(9686003)(5660300002)(6666004)(2906002)(186003)(316002)(85182001)(54906003)(8936002)(6916009)(26005)(4326008)(83380400001)(956004)(508600001)(66476007)(6486002)(6496006)(66556008)(66946007)(21314003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bVRTN0ttbE1WYzYxaHdlOU9aYUZsT3JPditQcUViRDFvYXVVcU9WSUhFdDBI?=
 =?utf-8?B?WVhXTzVMVCtxV1IyVVp3dVhRanR1dVc0K0NMVk9JTHFLaGdlV01DQ0U2bC9q?=
 =?utf-8?B?YzJCcDNXS3MzY01pVmViRzFQVmFsNHFITk9iSHF5bGZ6MUhHUGh1VWV3WkYw?=
 =?utf-8?B?dUEyU3o4aXB5WVJOVlRxVW5sQUlHQy9vNDlHOGN6MEpkRXYxTFJ4NnRmWU1U?=
 =?utf-8?B?MUNDNDJNMm9pMktKV3dqai94bm5DYVpBWDY0MFh6bXFBak5lbEZxWHpGRHpS?=
 =?utf-8?B?V2gxZHlKZVlhRjJiTXVidXpjSk85ekJVSDBtb2NmTXc5aUdmWnR2b3NqOEhD?=
 =?utf-8?B?bG9VZFR2aWtUc2ROSkc4M1RvRnJ3c2I5NnJjZXZEMnVDWUcxRjNUSFlrWUEz?=
 =?utf-8?B?N0NOa1FZRFFkS2pGVU1Hek0wOUlhT2p6QXc1UndTU1cwU0FtSkp2TFZWbXla?=
 =?utf-8?B?eURabWRqTXhXTGxxY0R2R2xXT3BlcWo4TDVjZWNwcWJzeHdQVUdTU3oyQUVq?=
 =?utf-8?B?ZTBNY2JFOG0rS0JkRjNBbmN1UjR6QWo2cHZkaXI2YThxVU1TeTg5MjVha0Jo?=
 =?utf-8?B?eDBROU81LzFLc0QyRzQ3MmMwMEd6a3RacTE3ak9UL0V4Y3pLVjNSMXA4cVpR?=
 =?utf-8?B?NU5oRXF3L1pDT0t1NWRYSHkzeVBhQ1U3Wnl0cEVlN21IcFg4cWgwZUEwZVc4?=
 =?utf-8?B?QXFtOXE1SlNRc1hnYS9tbVN5Y2FVM1VZcHN2dDZ5bGZUZFFxNE5Ib0FPZzhU?=
 =?utf-8?B?ejN2UnlNcERFK3hFVGlYRjV5dDU1bkN5UXdSZ1ZwbUoyZUJwQk5YQTFCMnI0?=
 =?utf-8?B?ZnZzU3BDUlZFN01tYUtaWEVOdGtTdFhNR0hHVEkydThZRWI0TmthbWdqUUlH?=
 =?utf-8?B?OElGRkFCeDdZa3FWNlBJUHdLcWsyQ2k1RGcrK0ptSmVOSUFJWU8vc2NoZUtK?=
 =?utf-8?B?cHRLZEM1WmdCR2hEMGhzSU1CTVlISVo0VWZYVDVScWZzS3dndWJMell6Zzh0?=
 =?utf-8?B?ZjRhUnZyeUJYZzNjNGY0M2c5NnNWSWJSTFd3MjJDdGZId0RUcVVTdU5Yamdo?=
 =?utf-8?B?c3Awazc5cCs2QWZRUEQ5S0ZKNGdDZEN5QUcvVGxtc2VKTHEzZVUxc1ZDMzAw?=
 =?utf-8?B?c1E1RG4zZllZVkZiTkN6UHpheWVvNDE3RWlqSFg4UE1va2J1RGhGRWJadkZp?=
 =?utf-8?B?bGpoSmpsT3BzbHUwZVM4VnpzQmxxV3ltR0FzSk00dWs0SUNUSmVXWUtXWDkx?=
 =?utf-8?B?Lzd2emRGNit2dnMyMklIUUVaWDFBUmdla1V6WHVtdTh0Q1grbWFWTU83VU8v?=
 =?utf-8?B?b0twSHdob1lzSlJvelpoODhDcitQVjhrMktHMG9NT3RXOElBL3FjYS9keEJG?=
 =?utf-8?B?R1JMUzNtNmFQT0JTQWhqSTlhMGJ1VzJCRkFTVm1GR2VEcks3MTJkdWg1WGR1?=
 =?utf-8?B?bzZhOWpmOWhZQlBxak5Sc3VydG9qNm45ZDkyUDY0S0lJdzcvOUlURHAwZm1p?=
 =?utf-8?B?TXhkc2RBQkZNUlY1MnJ0WWIyNTloSDloSERUdFRzSk8yWnJWUlNibzczV1cx?=
 =?utf-8?B?WTdWdVVGN2YyWXM3aUlCZnRHMjFxYnIvMjd5ZWxUQjB4UU9zSlNzL2dlendE?=
 =?utf-8?B?a2RZeHhkK01JWjVMaU43N2hUUzd2T3VxNFlhMlpIdTZaMWc5MTBwTnJYb0Vr?=
 =?utf-8?B?WjV0K2Y3dUFMWUxXdE5YMHFJQWM1cFlLNUpkRGZhNjErUUxFYlhNeWlqbjd6?=
 =?utf-8?Q?Vy5ICtIGS1K3Q5ilpnYkgNSSBoSbpdazmq60CaV?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b92f659d-2ac7-41f8-4344-08d97dd9ccee
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 15:01:05.0581
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7v3VBdUoJUeg6riCsoxTLSsgaUPJND5RBmp73stMR3clR1rMmrzTE+6p/BucXmUhFF6hwuDGYS9flU1HnpoN9A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5590
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:18:05AM +0200, Jan Beulich wrote:
> Like PV Dom0 in order to use the console if in a mode other than text
> 80x25 the kernel needs to be provided information about this mode. Bump
> HVM start info's "current" version to 2 and use a previously reserved
> 32-bit field to provide struct dom0_vga_console_info's position and
> size.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v3: New.
> 
> --- a/xen/arch/x86/hvm/dom0_build.c
> +++ b/xen/arch/x86/hvm/dom0_build.c
> @@ -19,6 +19,7 @@
>   */
>  
>  #include <xen/acpi.h>
> +#include <xen/console.h>
>  #include <xen/init.h>
>  #include <xen/libelf.h>
>  #include <xen/multiboot.h>
> @@ -549,6 +550,11 @@ static int __init pvh_load_kernel(struct
>      paddr_t last_addr;
>      struct hvm_start_info start_info = { 0 };
>      struct hvm_modlist_entry mod = { 0 };
> +#ifdef CONFIG_VIDEO
> +    struct dom0_vga_console_info vga_info = { 0 };
> +#else
> +    struct {} __maybe_unused vga_info;
> +#endif
>      struct vcpu *v = d->vcpu[0];
>      int rc;
>  
> @@ -598,7 +604,7 @@ static int __init pvh_load_kernel(struct
>       * split into smaller allocations, done as a single region in order to
>       * simplify it.
>       */
> -    last_addr = find_memory(d, &elf, sizeof(start_info) +
> +    last_addr = find_memory(d, &elf, sizeof(start_info) + sizeof(vga_info) +
>                              (initrd ? ROUNDUP(initrd->mod_end, PAGE_SIZE) +
>                                        sizeof(mod)
>                                      : 0) +
> @@ -672,6 +678,22 @@ static int __init pvh_load_kernel(struct
>          last_addr += sizeof(mod);
>      }
>  
> +#ifdef CONFIG_VIDEO
> +    if ( fill_console_start_info(&vga_info) )
> +    {
> +        rc = hvm_copy_to_guest_phys(last_addr + sizeof(start_info),
> +                                    &vga_info, sizeof(vga_info), v);
> +        if ( !rc )
> +        {
> +            start_info.version = 2;
> +            start_info.vga_info.offset = sizeof(start_info);
> +            start_info.vga_info.size = sizeof(vga_info);
> +        }
> +        else
> +            printk("Unable to copy VGA info to guest\n");
> +    }
> +#endif
> +
>      start_info.magic = XEN_HVM_START_MAGIC_VALUE;
>      start_info.flags = SIF_PRIVILEGED | SIF_INITDOMAIN;
>      rc = hvm_copy_to_guest_phys(last_addr, &start_info, sizeof(start_info), v);
> --- a/xen/include/public/arch-x86/hvm/start_info.h
> +++ b/xen/include/public/arch-x86/hvm/start_info.h
> @@ -33,7 +33,7 @@
>   *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_VALUE
>   *    |                | ("xEn3" with the 0x80 bit of the "E" set).
>   *  4 +----------------+
> - *    | version        | Version of this structure. Current version is 1. New
> + *    | version        | Version of this structure. Current version is 2. New
>   *    |                | versions are guaranteed to be backwards-compatible.
>   *  8 +----------------+
>   *    | flags          | SIF_xxx flags.
> @@ -55,7 +55,15 @@
>   *    |                | if there is no memory map being provided. Only
>   *    |                | present in version 1 and newer of the structure.
>   * 52 +----------------+
> - *    | reserved       | Version 1 and newer only.
> + *    | vga_info.offset| Offset of struct dom0_vga_console_info from base of

I'm not sure we are supposed to reference external structures like
that. We took a lot of care to not depend on other headers, and to
make this as agnostic as possible (IIRC KVM is also capable of using
the PVH entry point natively, and hence depends on this header).

IF we want to add support for dom0_vga_console_info I think we need to
at least provide a binary layout for it, like all the other pieces
that are part of the HVM start info.

> + *    |                | struct hvm_start_info. Optional and only present in
> + *    |                | version 2 and newer of the structure when
> + *    |                | SIF_INITDOMAIN is set; zero if absent.

We have usually used an absolute physical address to reference other
data, and I think we should likely keep in that way for coherency.

> + * 54 +----------------+
> + *    | vga_info.size  | Size of present parts of struct dom0_vga_console_info.
> + *    |                | Optional and only present in version 2 and newer of
> + *    |                | the structure when SIF_INITDOMAIN is set; zero if
> + *    |                | absent.
>   * 56 +----------------+
>   *
>   * The layout of each entry in the module structure is the following:
> @@ -139,7 +147,15 @@ struct hvm_start_info {
>      uint32_t memmap_entries;    /* Number of entries in the memmap table.    */
>                                  /* Value will be zero if there is no memory  */
>                                  /* map being provided.                       */
> -    uint32_t reserved;          /* Must be zero.                             */

This 'Must be zero' comment troubles me a bit, I'm not convinced we
can just place data here (ie: it would no longer be 0). It's even
worse because the must be zero comment is only present in the C
representation of the struct, the layout above just denotes the field
is reserved (which would imply it's fine to use for other means in
v2).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 15:48:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 15:48:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192928.343660 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT4Tw-0006QW-UZ; Wed, 22 Sep 2021 15:48:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192928.343660; Wed, 22 Sep 2021 15:48:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT4Tw-0006QP-R9; Wed, 22 Sep 2021 15:48:24 +0000
Received: by outflank-mailman (input) for mailman id 192928;
 Wed, 22 Sep 2021 15:48:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DDUS=OM=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mT4Tv-0006Q2-UK
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 15:48:24 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 83646ea3-1bbc-11ec-b9b4-12813bfff9fa;
 Wed, 22 Sep 2021 15:48:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83646ea3-1bbc-11ec-b9b4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632325702;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=Fm/iZHQVTaG/+qDwkx2g8Ef/Oogvcsp2C7j44nofTKU=;
  b=FebmuMphC9v3IAL5ZezO4yagq6v7aGimDJj2N7eqI5wbJWvbYIn7hBAV
   pu7OCnBuPheb96yGtDmGRvRJpxg19o/0XVMdEatoVV6hw35PEe6+pWidZ
   G30oCLiEoV69omSkSHZYv76lGARnAAq9NXs1xPzr6biUFvMf9v3RlF3v2
   o=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: PpjQHeQ76o7inVIpTL1cXonMl+doGrHuu3dnq0Ou7DcqJXlaibLMHWxivaQAoTH4A07gA9btq6
 DOzm8SEuzC87bT7TijHo4gdGJCyVk8zLtHyfGNRUULc5zHMnRP1kCFEdAR3MSaRO6z9ex3VyKV
 MU3itc9PUxn7SZeThms7PDZXE0JM9JgGQ974b4ynCqTRqjEVUMPHLa4BPFsX+jZ9RwdotLdh1P
 hDWBKtMrgLBzTfOfKXdkKcm6U1++/qaayULaMQ76E3buv60FCr6MYkHMUY9AvBDxcA670n3w6e
 v+Nug5TQlFEIsSCMM4xmXgv1
X-SBRS: 5.1
X-MesageID: 53336251
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:lfYoS6LlTKwJ1CD2FE+R+5IlxSXFcZb7ZxGr2PjKsXjdYENShTJRz
 2MXWW3SbvmMYWahLdt0b97nphkOsZDcyIBmTwdlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wbZi6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB3Sx/Rh0
 u1prqWrdlgLGvHTkukzaj1HRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2XvoQDgm1t36iiG97hf
 ukCb2Aofi7mXCcTYmgyArUijaS30yyXnzpw9wvO+PtfD3Lo5BN1+KjgNpzSYNPibdVYmAOUq
 3zL+0z9AwoGL5qPxDyd6HWui+TT2yThV+o6Fre16/pri1273XEIBVsdUl7TnBWiohfgAZQFc
 RVSo3dw6/hpnKC2cjXjd0bghG6ehjoHYsFvTM8etB6Hx4TtxxnMUwDoUQV9hMwaWN4eHGJxj
 AbZwY+xXFSDo5XOFinMre78QSeafHFPdD5cP3dsoR4tvoG7yLzfmC4jWTqK/ESdtdTzBTi46
 DSDtiFWa1473JNTivnTEbwqhVuRSnn1ouwdvV6/soGNtFoRiGuZi2uAswOz0Bq4BNzFJmRtR
 VBd8yRk0AzrMX1qvHfXKNjh4Znzv6rVWNEiqQc3QvHNCAhBC1b8JNsNsVmS1W9CM9oeeC+BX
 aMgkVoKv/du0I+RRfYvOeqZUp1ypYC5TIiNfq2EP7JmP8kqHCfarX4GWKJl9z20+KTaufpkY
 snznAfFJStyNJmLOxLsFr9Bjud0ln5hrY4RLLiipymaPXOlTCf9YZ8OMUeUb/B/66WBoQ7P9
 M1YOdfMwBJaONASqAGOmWLKBVxVf3U9G77srMlbKryKLgZ8QTlzAP7N27IxPYdimv0NxOvP+
 3i8XG5eyUb+2iKbeVnbNCg7ZeO9R4t7oFI6ITcoYQSi1U88bNv996wYbZY2I+UqrbQx0f5uQ
 vAZUMycGfATGC/f8jEQYMCl/oxvfRimnyyUOC+hbGRtdpJsXVWRqNTlYhHu5G8FCS/u7Zkyp
 Lip1wX6R5sfRls9UJaKOaz3l17o5CoTguN/WUfMM+J/QkS0/dg4MTH1g982P9oIdUfJyAyF2
 lvEGhwfv+TM/dM4qYGbmaCeoo61OOJiBU4GTXLD5LO7OCSGrGquxYhMDLSBcTzHDT6m/ayjY
 aNezu3mMe1Bl1FP6tIuH7FuxKM4xt3uu74FkVg0QCSVNwymWuF6P32L/chTrakclLZWtDy/V
 l+L5tQHa66CP9noEQJJKQcoBghZOSr4RtUGASwJHXjH
IronPort-HdrOrdr: A9a23:ZH4kga1qTGYW7X4v5FXqxQqjBLwkLtp133Aq2lEZdPU1SKClfq
 WV98jzuiWatN98Yh8dcLK7WJVoMEm8yXcd2+B4V9qftWLdyQiVxe9ZnO7f6gylNyri9vNMkY
 dMGpIObOEY1GIK7/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="53336251"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Uj0gRfTQugQaDSfdJ6vZp1vXwd4efcnCiPIG2CAsqbJOwzxZxuhDH7abNVO+h5RfkOyPOt3jIjpFQptcI5BkDxqHIY23GTY6glV6lRaNfHYNUtbFGf6Tq4nmHwkjOEK/U8FRYljkeWoBrNSowPmlYehoG4qGhBhAsVNNMvuZIns7ImZ6zlV9IBbpXDJ54zOLc6o85BN0df33MRa2owrFtbOtg03jKxx7oahtGW21AhA3LMxT08qlWAC8nQy+GSLvFS8SKcWpFAbabBVAlLCdUihcF8u5s4jmrwap2nGU6KrQ4U9P6s2Yzm1N4ZnxckSGFn/TM3nnyJ3KLG4GFEUg1g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=n4NAxChxHIbz2sxh8SXGwiWz7oHmjAwd5Fcf/V1mcnY=;
 b=AXLZxi8xDwFz5D2Yr5UOb7KBj/iCHxH3OZAnucdyRK6je3jjRXyjxEVdlps+dy/UCQKT/Eb0E6Kso5OayaBTOJHeDtxWjaE/Jz8V6yCkWyQHzHKKZYdTFMkO5HC3v32sLjBT9bzlzbiirPMasshpaSi+1M3CO6VWw+JY5qugiTbBcs7FHTK9umAiW2grL38UT7DiWX6+28+eo9OlBYhA45BHpwA7KdjV+Oqz8uedexN5PF63inA1RwptYfGj6xWMT/C3v9JwfC3/dk539ABEDdo7kPNQrfBfA7yyzWiSikexl7MCMdNrYNZkiQQKr0HACUjCQkr7zqSzY7uMYoLoWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=n4NAxChxHIbz2sxh8SXGwiWz7oHmjAwd5Fcf/V1mcnY=;
 b=FVtJONRojxJgc0iq1eTT2mcbiMZKJqXn0SaGNUYfUI4IzlANp8uI2q1g1MUteEVR/9pEqiPSNDNUIPl8EAJ6eWlRbOIcpZ8/geYyl3pLDa26SfKr/KS/6kf/cpI026EiEqR5kzNUdLmpKW36FFePjUd91SigJNzuAL8IvXN64uA=
Date: Wed, 22 Sep 2021 17:48:13 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 5/9] x86/PVH: actually show Dom0's register state from
 debug key '0'
Message-ID: <YUtQPUHjZ8GnfeCE@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <4ae1a7ef-d6e8-75db-ddf5-987f10175196@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4ae1a7ef-d6e8-75db-ddf5-987f10175196@suse.com>
X-ClientProxiedBy: LO2P123CA0075.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:138::8) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f35446e9-22d1-48b2-7639-08d97de065e1
X-MS-TrafficTypeDiagnostic: DM6PR03MB4604:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4604016B86759FA448FC99408FA29@DM6PR03MB4604.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: JATlDs+FoSvkletnp8qaVFaNSPMQcJxmTi1D9HXGPB10L7rgr0dZ2biA7BfvyVzx3L/2MRUJlNNLl+CaERFaCWjaMkyCAH4XrzhVEu9o2zj5mU/p+y7W5fJCnhgvWnFDzUb3EjL4bq08t9bG49bCEb2988NQ3Pw3oYYvgDoMSqw1hSb+91CzVtO0uI3FUSFt17606C2VGoN2nsgW0bp7xwxTDIimiLRHwqR+erqZUbD+VPSxpXfAlCFlkHvwpQQah/rlw2iiR2FLnNZ8WJVj1FkBdqcM4SYoSuVFG5wdae/1yCWNChwKrGJU43JAP65cPFnXMGoerM7vw4ebibvpk90lRU4zNLY5wqiI4HJRRxxx75xfkzsGqumx7tr1n1kzMSt7mQEQCOu3fcngDBnc7/hAtRg4M1V7rRqGj5Hu1S8YNlf93ajCz9Z/CExRX0/5Bg3gDIS0RssCbDfb+sA52ykRGLnzaxQm86IgsSK5jJIEYZy5KgoR4mns0wDoFghbuhLecmQl3HvxuGi3SfVLmhUi2pfR8Y5M/dkhB+RsZUzSlru4wiSUUQ+A0zb+LMQqZZyQg1giRD+kOWn6L0yZjLb9E10VcNKAwgwiUdjiB62Cdy3S9DpRaxmb+nv2hMEvZnNgYuJR8zCgQPRJ4VxBDA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(4326008)(6496006)(26005)(86362001)(54906003)(6666004)(316002)(38100700002)(956004)(85182001)(508600001)(6486002)(5660300002)(8676002)(66556008)(66946007)(66476007)(6916009)(2906002)(9686003)(8936002)(83380400001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?L2NtelRSWnF4RWt5bnRUZkVhallwVTJRSjA4c3c3QzQ0Mjh1dXkyZmN6aGta?=
 =?utf-8?B?bTJLcmRCUGs0aWdFRFJzbTBHdWwrOTFqWFI2N3hUaDJIcGZGVnNQR2t6b0FU?=
 =?utf-8?B?KzJGTnBmUHpsMEN6bHJ4cEdSMU56aTNrZGpIWUtJeUx2R1VMcUhHT3ljY1c4?=
 =?utf-8?B?a3o3QnhyZHFyQytyOTVBOVZ5R1VJRjh5eGRweHpSeDJkK3BVUDFWalorUHUr?=
 =?utf-8?B?V3dsQWJZUTRrQURldEo1SzJ3UE9QbjJFQVRGOFJrVDFuU3plL3BhbFZzZENy?=
 =?utf-8?B?NTFkUzh5dFp6VW84STN6RWFFdkR3NXRMSWxqWkhHUkJDcVp3U0E5a0NjWWpQ?=
 =?utf-8?B?ZXd4Umd3WmEwbndUVEhCT29JTFRIREprU3NKSnEyOGlwN2FVNlFmSFFQNFF4?=
 =?utf-8?B?VG8wU0pFeTlUa2ZlTDdFVHFNZ1M1bTNVbGlJc29KSjlsVERwTmpSMU5hMDNu?=
 =?utf-8?B?cy9XUlRHQi85cGFYb0h6bGNCNE1VeVR4V3M1OTUxVXZsR3hhelRjUDhETU5o?=
 =?utf-8?B?cDE4dERIOHNibEttdjZSTHZkakVzUFV3NlZodFRJQkJjN295bzFIb1hWUlRy?=
 =?utf-8?B?WGhuMFB0bmpoaUxCem5zU3hSNmRjb2NNUEFDOGpwT2dkVThCdXNqOWx0NG9z?=
 =?utf-8?B?Ykg1eFVhUFlRR3RVWGVNb010eHVyM0gvWktORlVFRFVvWE1wM3BNYWNBMEl2?=
 =?utf-8?B?U0NFTnVYUlpMTnFLVGtDbElCWXNUZlFLRlc2d2ZVaUFVZHF5aXpwdmFBbU9r?=
 =?utf-8?B?Ky9CK3pBZDUxcHFLa0J4eFUrWlpXaExHMElsckYyb2ZrR1NtbEx6MU1MTDhh?=
 =?utf-8?B?LzE0L1kzcDczV2pkVjh3L3hDVkt0K1pvZ3RPa0VJbGNNQjJiRnBJNEVWc3Ur?=
 =?utf-8?B?SWF5S3BlSU01L2x3S3I1V290M3hUdjJGQ0UxT2xRbUFZZzVXaXVBVGZkdlFD?=
 =?utf-8?B?Q0RrZGR3WXRpakIxeU1jTDVSQWFiYWJ1bk9Zc2hZd0VLcExxQW5kWDJ4ODJS?=
 =?utf-8?B?OEQ5S2ZyK2FMaDRpWGMzVjI1NWVoVng0R1lZcWI4dXIyVTEzTytHVEMvaGN3?=
 =?utf-8?B?Sy9UQkpOU0I3UFl3MERzYXdWUW1xU2Z5ODRBVFR3bWZHK0N0Vy80Ni93TEJ0?=
 =?utf-8?B?WFQvSFJiREMvby9nNHd4WVlySzFSK21ERi94bVQrczk5V0FBY1JOb2VZVzds?=
 =?utf-8?B?WXN0RnRRRU1nbzhEWXdUT051TEhZZitQc2RZRnAzVFRQUDV4eTZacDlEdC94?=
 =?utf-8?B?aDhhdmViR04wWG54b0hFUXVXRzNoY1NiV3h0VDF5LzRDc0J6NTZnZnJvNFBJ?=
 =?utf-8?B?aUhvQXZNb1pibk9TQit2TnR0K0dEdUVjTHpjS0t1ZmVpWGtPVWt3bUJ5Tzda?=
 =?utf-8?B?NWlSNEhQdVh0YTZLTW5COC8rQW4vVU9MRCtVSUpOZndVWjhGcFE5cDlWeGIx?=
 =?utf-8?B?TDhYT1RTRWhvZXAwK1JYem5PVEtFQUIzR2R5M1VxWDB0WmZhZjZCc3lPN0lT?=
 =?utf-8?B?SnNYZlY5QVBzeTk4eVIwRzFYb2NyclNub01KZzd3OEFHcUFoTlBkaWhqdGt4?=
 =?utf-8?B?MC9idVZOTXkvUjRWTGNPdGVhMGtTcm5RU3B4dkFmamZzYjZubW51WTdKSWlF?=
 =?utf-8?B?U2o0WHNwRUpPQjJuUXJFWXFKelhaWUhxc2pHT0IvdmdsTVd3Q1I4eU51MEM1?=
 =?utf-8?B?SHpYa0pPT3JhVmh2a1MxQzl6VzVOUEpPMEZpRzVpYTZlTmtTM1F1dGxzeWYv?=
 =?utf-8?Q?wRf7ixn3161ZjxYF5HQxiNTlC+6BtuOW+OKCliM?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f35446e9-22d1-48b2-7639-08d97de065e1
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 15:48:18.6868
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VWeGLPYwJAows4+Q4WocDAHlv741mnhJGcyDNa+SH8T5M6RluhSd3pGqy7KfwBVj/zuviHrzyPHV4hkhV/jiMg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4604
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:19:06AM +0200, Jan Beulich wrote:
> vcpu_show_registers() didn't do anything for HVM so far. Note though
> that some extra hackery is needed for VMX - see the code comment.
> 
> Note further that the show_guest_stack() invocation is left alone here:
> While strictly speaking guest_kernel_mode() should be predicated by a
> PV / !HVM check, show_guest_stack() itself will bail immediately for
> HVM.
> 
> While there and despite not being PVH-specific, take the opportunity and
> filter offline vCPU-s: There's not really any register state associated
> with them, so avoid spamming the log with useless information while
> still leaving an indication of the fact.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I was pondering whether to also have the VMCS/VMCB dumped for every
> vCPU, to present full state. The downside is that for larger systems
> this would be a lot of output.

At least for Intel there's already a debug key to dump VMCS, so I'm
unsure it's worth dumping it here also, as a user can get the
information elsewhere (that's what I've always used to debug PVH
TBH).

> ---
> v2: New.
> 
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -631,6 +631,12 @@ void vcpu_show_execution_state(struct vc
>  {
>      unsigned long flags;
>  
> +    if ( test_bit(_VPF_down, &v->pause_flags) )
> +    {
> +        printk("*** %pv is offline ***\n", v);
> +        return;
> +    }
> +
>      printk("*** Dumping Dom%d vcpu#%d state: ***\n",
>             v->domain->domain_id, v->vcpu_id);
>  
> @@ -642,6 +648,21 @@ void vcpu_show_execution_state(struct vc
>  
>      vcpu_pause(v); /* acceptably dangerous */
>  
> +#ifdef CONFIG_HVM
> +    /*
> +     * For VMX special care is needed: Reading some of the register state will
> +     * require VMCS accesses. Engaging foreign VMCSes involves acquiring of a
> +     * lock, which check_lock() would object to when done from an IRQs-disabled
> +     * region. Despite this being a layering violation, engage the VMCS right
> +     * here. This then also avoids doing so several times in close succession.
> +     */
> +    if ( cpu_has_vmx && is_hvm_vcpu(v) )
> +    {
> +        ASSERT(!in_irq());
> +        vmx_vmcs_enter(v);
> +    }
> +#endif
> +
>      /* Prevent interleaving of output. */
>      flags = console_lock_recursive_irqsave();
>  
> @@ -651,6 +672,11 @@ void vcpu_show_execution_state(struct vc
>  
>      console_unlock_recursive_irqrestore(flags);
>  
> +#ifdef CONFIG_HVM
> +    if ( cpu_has_vmx && is_hvm_vcpu(v) )
> +        vmx_vmcs_exit(v);
> +#endif
> +
>      vcpu_unpause(v);
>  }
>  
> --- a/xen/arch/x86/x86_64/traps.c
> +++ b/xen/arch/x86/x86_64/traps.c
> @@ -49,6 +49,39 @@ static void read_registers(struct cpu_us
>      crs[7] = read_gs_shadow();
>  }
>  
> +static void get_hvm_registers(struct vcpu *v, struct cpu_user_regs *regs,
> +                              unsigned long crs[8])

Would this better be placed in hvm.c now that it's a HVM only
function?

> +{
> +    struct segment_register sreg;
> +
> +    crs[0] = v->arch.hvm.guest_cr[0];
> +    crs[2] = v->arch.hvm.guest_cr[2];
> +    crs[3] = v->arch.hvm.guest_cr[3];
> +    crs[4] = v->arch.hvm.guest_cr[4];
> +
> +    hvm_get_segment_register(v, x86_seg_cs, &sreg);
> +    regs->cs = sreg.sel;
> +
> +    hvm_get_segment_register(v, x86_seg_ds, &sreg);
> +    regs->ds = sreg.sel;
> +
> +    hvm_get_segment_register(v, x86_seg_es, &sreg);
> +    regs->es = sreg.sel;
> +
> +    hvm_get_segment_register(v, x86_seg_fs, &sreg);
> +    regs->fs = sreg.sel;
> +    crs[5] = sreg.base;
> +
> +    hvm_get_segment_register(v, x86_seg_gs, &sreg);
> +    regs->gs = sreg.sel;
> +    crs[6] = sreg.base;
> +
> +    hvm_get_segment_register(v, x86_seg_ss, &sreg);
> +    regs->ss = sreg.sel;
> +
> +    crs[7] = hvm_get_shadow_gs_base(v);
> +}
> +
>  static void _show_registers(
>      const struct cpu_user_regs *regs, unsigned long crs[8],
>      enum context context, const struct vcpu *v)
> @@ -99,27 +132,8 @@ void show_registers(const struct cpu_use
>  
>      if ( guest_mode(regs) && is_hvm_vcpu(v) )
>      {
> -        struct segment_register sreg;
> +        get_hvm_registers(v, &fault_regs, fault_crs);
>          context = CTXT_hvm_guest;
> -        fault_crs[0] = v->arch.hvm.guest_cr[0];
> -        fault_crs[2] = v->arch.hvm.guest_cr[2];
> -        fault_crs[3] = v->arch.hvm.guest_cr[3];
> -        fault_crs[4] = v->arch.hvm.guest_cr[4];
> -        hvm_get_segment_register(v, x86_seg_cs, &sreg);
> -        fault_regs.cs = sreg.sel;
> -        hvm_get_segment_register(v, x86_seg_ds, &sreg);
> -        fault_regs.ds = sreg.sel;
> -        hvm_get_segment_register(v, x86_seg_es, &sreg);
> -        fault_regs.es = sreg.sel;
> -        hvm_get_segment_register(v, x86_seg_fs, &sreg);
> -        fault_regs.fs = sreg.sel;
> -        fault_crs[5] = sreg.base;
> -        hvm_get_segment_register(v, x86_seg_gs, &sreg);
> -        fault_regs.gs = sreg.sel;
> -        fault_crs[6] = sreg.base;
> -        hvm_get_segment_register(v, x86_seg_ss, &sreg);
> -        fault_regs.ss = sreg.sel;
> -        fault_crs[7] = hvm_get_shadow_gs_base(v);
>      }
>      else
>      {
> @@ -159,24 +173,35 @@ void show_registers(const struct cpu_use
>  void vcpu_show_registers(const struct vcpu *v)
>  {
>      const struct cpu_user_regs *regs = &v->arch.user_regs;
> -    bool kernel = guest_kernel_mode(v, regs);
> +    struct cpu_user_regs aux_regs;
> +    enum context context;
>      unsigned long crs[8];
>  
> -    /* Only handle PV guests for now */
> -    if ( !is_pv_vcpu(v) )
> -        return;
> -
> -    crs[0] = v->arch.pv.ctrlreg[0];
> -    crs[2] = arch_get_cr2(v);
> -    crs[3] = pagetable_get_paddr(kernel ?
> -                                 v->arch.guest_table :
> -                                 v->arch.guest_table_user);
> -    crs[4] = v->arch.pv.ctrlreg[4];
> -    crs[5] = v->arch.pv.fs_base;
> -    crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
> -    crs[7 - !kernel] = v->arch.pv.gs_base_user;
> +    if ( is_hvm_vcpu(v) )
> +    {
> +        aux_regs = *regs;
> +        get_hvm_registers(v->domain->vcpu[v->vcpu_id], &aux_regs, crs);

I wonder if you could load the values directly into v->arch.user_regs,
but maybe that would taint some other info already there. I certainly
haven't looked closely.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:00:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:00:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192939.343675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5bI-0006xs-Gr; Wed, 22 Sep 2021 17:00:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192939.343675; Wed, 22 Sep 2021 17:00:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5bI-0006xX-AY; Wed, 22 Sep 2021 17:00:04 +0000
Received: by outflank-mailman (input) for mailman id 192939;
 Wed, 22 Sep 2021 17:00:03 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT5bH-0006nj-7v; Wed, 22 Sep 2021 17:00:03 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT5bG-00042t-VR; Wed, 22 Sep 2021 17:00:03 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT5bG-0003ET-Lw; Wed, 22 Sep 2021 17:00:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mT5bG-0006In-LU; Wed, 22 Sep 2021 17:00:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mlC9bquBJ71bxUECHny8Wd1M7bIVDM8MhlNy19qQoC4=; b=hQMGxmNpgGBBzrZXv/8Jk3j72m
	fjpdxntAijrDyfhpgyGruG/P+YvTxu9GCmflDE2X8zb4ap/YQH8htxUs6VFJYIFuTF2YElt/yYtq2
	aV3Msas+whzSoWzY9IGG7o1OOBomEwEp6G6KTbL8ATBXM5xAnigEpOihFoRirlGam3hA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165149-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165149: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:guest-start:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:heisenbug
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=92477dd1faa650e50bd3bb35a6c0b8d09198cc35
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 17:00:02 +0000

flight 165149 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165149/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 14 guest-start    fail in 165145 REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail in 165145 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-thunderx 13 debian-fixup               fail pass in 165145
 test-armhf-armhf-xl-credit1  14 guest-start                fail pass in 165145
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 165145

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 165145 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 165145 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                92477dd1faa650e50bd3bb35a6c0b8d09198cc35
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  417 days
Failing since        152366  2020-08-01 20:49:34 Z  416 days  728 attempts
Testing same since   165145  2021-09-21 22:12:58 Z    0 days    2 attempts

------------------------------------------------------------
7322 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263575 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:05:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:05:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192948.343688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5gZ-000812-CY; Wed, 22 Sep 2021 17:05:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192948.343688; Wed, 22 Sep 2021 17:05:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5gZ-00080v-8r; Wed, 22 Sep 2021 17:05:31 +0000
Received: by outflank-mailman (input) for mailman id 192948;
 Wed, 22 Sep 2021 17:05:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/3we=OM=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mT5gX-00080p-Ia
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:05:29 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4767a595-1bc7-11ec-b9bb-12813bfff9fa;
 Wed, 22 Sep 2021 17:05:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4767a595-1bc7-11ec-b9bb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632330328;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=LhBwEHKOX9qFl+8IkDrfjNwFs4n9Xj9U/uzgMazbTz8=;
  b=CW38rq2Jt0WTZkCYu0BVQjIDtwbeedlR6fnZKfCBqsqnHrbVO3mIoWvs
   dp5rkkgMXVA/1hX8NIK86efdpRM2eb/P9umlmMgp0G79eG/1DFHaS/GyH
   vH63Kni+/iaClj4xo9CDLQQi0OWYYY0uUuc0sUG74mAEi1kJa+iNIbIZa
   Y=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: KkbuIQ0BEQE7k+sqLut0vQPQwha2RgeexUhK8bHnVwWNvS41EEubAbyvTSvWaCWn46psPIDiSB
 AoOfoIMBZSC5PopY/h5BPYDDsliitSq9m0BI5xQvWQSGGysNZ6JPZyCWTXr60BvMMFQ2v57dNl
 72+rGY/DCT/iv+e12fx3bCWVyzzc0+HehFg/zxbDaX0udWzupRPTldSiKGCcXcK7CwK4EdnC14
 KKAX9kfcLCmZMeqSateTGuaxv4SBgQK/kl+lxVRB3Oj+TOitCYCzdJGgr/hbmV6zftJYKGdy1R
 t0hoq6gYaKvHDm4gfqiaqntv
X-SBRS: 5.1
X-MesageID: 53376522
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:HSdbEKuT7KXyfauYnMZZqR7mfOfnVGVZMUV32f8akzHdYApBsoF/q
 tZmKTqFa6mONDfwet9+bI60o0hVu5+An4NhQQM//yhjQSsb+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524LhWGthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npl6bGIWBgxZID3hOU7DgF6PjxDfox70eqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DP
 JFGN2c1NnwsZTVQM103Kp5hwdyVn1/NUG1873Osm5sOtj27IAtZj+G2bYu9lsaxbcdfk1ucp
 2nG13/kGRxcP9uaoRKV/3TpiuLRkCfTXIMJCKb+5vNsmEeUxGEYFFsRT1TTiea9ol6zXZRYM
 UN8x8Y1hfFsrgrxFIC7BkDm5i7f1vIBZzZOO7EX0BCS0vD52BqYIDUcbBNcVZ8HieZjEFTGy
 WS1c8PV6S1H6ePOEyvNqOvM9FteKgBOcjRTPnZsoR8tpoC5+dBt1EqnosNLTfbt5uAZDw0c1
 NxjQMIWvLwVkcdD/KGy51mvb9mE98WRE1JdCuk6WAuYAuJFiGyNPNfABbvzt68owGOlor6p5
 iNspiRmxLpSZaxhbQTUKAn3IF1M2xpiGGaG6WOD4rF7r2j9k5JdVdkIvVmS23uFwu5bIGS0M
 Sc/SCt65YNJPWvCUEOES9vqUKwXIVzbPY29DJj8N4MWCrAoLVPv1Hw+NCa4gjG2+GBxwP5XB
 HtuWZv1ZZrsIf88l2TeqiZ0+eJD+x3SMkuJFMinkEz4geTBDJNXIJ9cWGazgikCxPrsiC3e8
 spFNtvMzBNaUebkZTLQ/5JVJlcPRUXXz7iv8p0/mjerLlU0FWc/JeXWxL99KYVpk74MzrXD/
 22nW18ew1363CWVJQKPY3FlSbXuQZcg8i5rYX1yZQ6liyo5fIKizKYDbJ9rL7Mpw/NukKxvR
 P4fdsTeXvkWEmbb+y4QZIXWpZB5cEj5nhqHOieoOWBtf5NpSwHT1MXjew/jqHsHAiat7JNsq
 Ly8zALLB5EEQl06XsrRbfuuyXK3vGQcx70uDxeZfIELdRy1ooZwKiH3gvsmGO02KE3OlmmAy
 gKbIRYEvu2R8YU7x8bE2PKfpICzHuogQkcDRzvH7ay7PDXx93a4xdMSS/6BeD3QWT+m+KimY
 ukJnfjwPOdewQRPuot4Vb1q0bg/953koLoDllZoG3DCblKKDLJ8IybZgZkT5/MVnrIJ6xGrX
 k+v+8VBPeTbMczoJ1cdOQ45Y7nRzvoTgDTTsaw4LUiSCPWbJ1ZbvZG+5yWxtRE=
IronPort-HdrOrdr: A9a23:e3Kzoqtv0A8nUbuRPc1iR84n7skDdtV00zEX/kB9WHVpm6uj5q
 KTdZUgpHzJYVkqNU3I9ertBEDiexPhHPxOj7X5VI3KNGKNhILBFvAH0WKI+Vzd8kPFmdK1/J
 0QFpSXV7bLfD9HZGzBkW+FL+o=
X-IronPort-AV: E=Sophos;i="5.85,314,1624334400"; 
   d="scan'208";a="53376522"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=G/MjnQ3T0dogiV0shKGY9asCKQ8PGzG5EVVtNia3SPK37D2wDsd51MuFD01iPXYqNv8P7CBwZkpDtEoAoeWCXIcXtIegTBRLET6HZGvagNLFPu5nNwttVutz4G8BXXedAzIPONB67fDY+eFyJbwiym/RB5Qpu0pv64UExcdTuUuofjd+cKKIbyViChBy0oof7IW/ASRiPTppQr6O6BG4ItYaDDB2vzs1kl1G7vb/j7e75QriiNrV42LA2KqFEpU9ASSY7AChOkoXj+bLzJ33MF1yM/hJ2jHwkzf3IoJDh0U2LU86bHQBe6iFy5h2xzvOs4d8Qjhrc+qp51dKgsvnRg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5felQb1LbNmrt3F2TUVttsNTPHqdtjORDBVW1/xao/U=;
 b=GUzh1i7dPeS8AVhaALhOy7AYi6rx34b/clI7JBo1cZZNBJBlQc48gtwc1AKBd6Pk2sJOZdhQeBQQz/Ki4a/18pGbkPZjvUVz1V2oTBIwXwf9BIhvQ8mVFoJDEPpE9hp6Fnh6sLFcpCSP/XMMvA5qFt3HG48Ej2VzQBdpOoR5D88gnSk99xGpW6WTez2SshVDnWXI/ZkQ/ApBQeNh/Q9pcAV5TXQ8MNXbQHvUhjTNU4WQqybhLYDTCC1JAnOk5JdOt4me2KZ1bQWopTgbAHccIS1uP9qz0V6kf7jZwrYEmCVdZ8okwxZEeHRr9A8rrFhztb0bnwn93Cg5RnWlbthtQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5felQb1LbNmrt3F2TUVttsNTPHqdtjORDBVW1/xao/U=;
 b=d4TsH3SXH7Ijg56/fmwJ/71f5V+PZLIdqO181TjwVpYdTOQDqA41mFNSahJ506AC/aOElaAIAD9UtpFX1myfiv5Z5kgjPfME8zodSzULODdvg9tU2GcMZJqbrHwEjUvkfgYobNteLyvo+VvrDCmwFN14k43SbDIaMiT47pzKuu8=
Subject: Re: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Jan Beulich
	<jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Wei Liu
	<wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
 <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <e1def032-d8dd-f5df-c4f7-a9968083f06c@citrix.com>
Date: Wed, 22 Sep 2021 18:03:57 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0310.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::9) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: dc0ffd2f-de15-4b85-592a-08d97deafa97
X-MS-TrafficTypeDiagnostic: BYAPR03MB3992:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB39929201FDD08EFD899FEABBBAA29@BYAPR03MB3992.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: AfWF/A07cUBzWorc59B3/kOmhH1/+Dooyhiv1rhYuhPMgQkILwWIUl4Jtw28ygN0w5WQlw5nAM9dO33CdHxJHCYrEp5DLAM5lAmTU5YaRMQh1g0cKE0TYj/WvuUUC7YflpRRwrKeArFP2yk7UEra+CRDHgMWdMj6MBJRtZ1FJW30ZiKTkFJEKZ2wQPrji9MXNw9Fl6NWlsfgWMv/sdckQqIjMNVqrcYw0mEjBbMKvmbsGup6/P7Lp02LSXiyBb9RJhHxgSRcTSfRqN+gBDPyTv0a2x6PCZR89aifU/0DUSmHQ9P04jfDdq9WfjGCakJR21a5Tz6BjqkRgHaeWYjj7jIbEpU+AKJywyUp1aZ8XuFUg+CU9agE5Kb0fU5DMyMbb4EwT59IgcIxCya2bf7QxDw0R6G7B9Cpwu3OibWB4NjHb2k/4W3eqUVHYm20nI68Sv2ff2rh0Eu/VsCdrbRM07KfTbp/xH7d3tyY8aM5BcPjace3/MctaSMIS7rNQRD7Tz8bgflk9741C9eXQByVKBULxo3sKG0E+2+ynvmFcTRuC7PEPTVP6Zt/7J3yzIYAR98y7fBjknKVsi0gYGt3y9tlt9KC/tNTYfQx0P0JPs3nAWrZxSQRjYFrLhbhwlHFOjYzbVYsJGFonFbo8pG+EThfW9184Q6DgthjBoNg55NRXQfc4zGm/x8Znfut43+AaWQcfjVXNjCnCJk4caH+056ljRLzDzNcB0uMlV/Vn7M/1dMrqPn6NSeDdJ7DUrXA
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(54906003)(508600001)(86362001)(36756003)(16576012)(31696002)(5660300002)(6666004)(316002)(31686004)(4326008)(8936002)(2906002)(26005)(8676002)(956004)(66476007)(66556008)(66946007)(38100700002)(186003)(2616005)(6486002)(55236004)(53546011)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dlJtc2p4VjdESmJRTVAxT3ovYjBKUEpJalNlNERBN1hhSmR5Zm9DT2hYcWtw?=
 =?utf-8?B?MXVoOGRWRWhURUlUcTY0OW5MckxaZE1NV3RvT093MnQvOWk1dXBiOVJPemJQ?=
 =?utf-8?B?cFcwditUNDB6dHN1RFdYbEFOSW9VWXB6dFlDeGk1Sk5WK3Y0RHVybDJ6TGpC?=
 =?utf-8?B?Sy9GenI1K3VaY2E1WjViaC81YU9LWDdweTJTczRLVkxaanlLRVNKZnNZRUZD?=
 =?utf-8?B?K2pGeHZjamYxS1dOdlNsT3VtR2FZOUVnNG1FbXo4TFRXTXZwSTRia0RzWVI3?=
 =?utf-8?B?QzZyeS9BK1U2QTJmdmtxcFVpbVdkYVQwQVBQcVRmS1A3bGpPaUs4YjJwWmk4?=
 =?utf-8?B?cERzRXNodFc5c0RpaDZDQzAveDQzY3VtQmpqenVyQUJCaFFKMU1yWXArUGlC?=
 =?utf-8?B?OUM1YWpQRStoOUhTMGJ2aUNLK1cwcmVKZTZoZjlCL1JyQTUvNEpwN1NEamZX?=
 =?utf-8?B?QVJxaCtLY09ZNUZqVjMxUGEvei9LZitMNUNwNWJTVHZMcUs1alk2ZWE1b0Q3?=
 =?utf-8?B?L215WnNhRDhoTHBScENSckFlUGxJaHJiMVBwN2YrZXNzWEdkUGkvMmpONWND?=
 =?utf-8?B?ZU9FZXBzanM4MTZneHNCbGhWNGdPeklPZXJBRU50NDduYWNBQ0llZWpINWkw?=
 =?utf-8?B?dGdSUEpuZGFMNWcvMkMrWFB4bHM3TlA4ZDFjalhGZk12c0tBWXd2UHllQzBN?=
 =?utf-8?B?SnhOYWVZc2xIc29LQXFpRHloWFFqNVZRalhXdG55UGhZV1ZjcmViN3JsOExo?=
 =?utf-8?B?ZlF4ZmdtZHoyWElEQ2xJN1lJTTJUUEw0UGxPejZsUWxHNGFjU2ZsM0Z3WXJi?=
 =?utf-8?B?L2F2RExHRGh6eWhDUzU5Ti9wczFYL1ZCNHlXVmpoRXRNekhnYmkyS2ZaR3do?=
 =?utf-8?B?bUdSeWVLbitLK0k3Z09yS09Wb01vNEZaYitMT1IzSFZHTjFtZHVVOTBCK2Vp?=
 =?utf-8?B?YjcrUDhnakYrT2hxVXhYTVRiMFFQUEVPZXEybmVKK0gvYzZJSmN6eFlhQW9W?=
 =?utf-8?B?Vm1SSWF1Y0pzV011RFVoUHFHVkViVzB0cG8rRTY5bHMxT2NhZWFUb0JRN3hl?=
 =?utf-8?B?NWJQSEJIUjExaVRLMEUvZWx3T0NUWkRRL1NUK2ZlNk55MjJCMXpLYlB5dDh6?=
 =?utf-8?B?Z2RJR1JIdjZwcXhXNEJTbDEzVW1SdjZYRGpOYWVyRkRxK0QvRUVIK281QjZu?=
 =?utf-8?B?WmlLV3pRV1pCTjVuNU1mUjloS3RRTzhTdy9wNkZtbjc2QVlaTUlhZFFMYTE4?=
 =?utf-8?B?dE5EUks0Mzk4WEd1UTZ3RVJ4NStiYVFYcHFxNXE5VWwzWTNaVXNrT3A4Qnh0?=
 =?utf-8?B?ZkIyNnpxZlM2bHFuNnVYUWtOR2RZVngySEttcU41S2M0eWxZWmpicDBPR2R6?=
 =?utf-8?B?and1d1d5KzkweXdaWDNHMWJXS2tBZVBMTk8xK1FFelBteWVCMm5hbCsxYmc5?=
 =?utf-8?B?ZWVNR3djQjJTZ2hrTk5vVHNlY0FudGZ5Zm9XWWxUdDEzMGNZZXR1RnFxTHBQ?=
 =?utf-8?B?akp5cDRXUUhLNEo2eEV6clFFeVg2Z3BMb0c5S29NR2NhMHBvNFVDTHF1NEdl?=
 =?utf-8?B?SEZCM1VHdTNVZ20vblVLbnNZV2RPbWtvSVcwTkJyNHJ4Q09VdUE2bE9iekov?=
 =?utf-8?B?TGh5eDJKVTUrV01pc21ua0ZmTzlLSWFhMGZDM0VwQy9OckRLRlZCTDJ3QzJq?=
 =?utf-8?B?TDVuUDU1SVNXdDRXcVE0TmNPNkxFdE0xM08yblBSWmROM2JEdElNdFNsWmxi?=
 =?utf-8?Q?V96iOeCfHIjKjG1k+NZDkkhn6fm5H1G1dJT4jdV?=
X-MS-Exchange-CrossTenant-Network-Message-Id: dc0ffd2f-de15-4b85-592a-08d97deafa97
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 17:04:03.1966
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 35kituR7Vh0o91+wCJbRkoEogba/e348oeUZjfHHkDiZS309IQAQsnX2OQq9akIT76HXSzp9F+EPfT1d0hEW8iQbHKziYzkWut1G2AqTEkA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3992
X-OriginatorOrg: citrix.com

On 22/09/2021 16:01, Roger Pau Monné wrote:
> On Tue, Sep 21, 2021 at 09:18:05AM +0200, Jan Beulich wrote:
>> --- a/xen/include/public/arch-x86/hvm/start_info.h
>> +++ b/xen/include/public/arch-x86/hvm/start_info.h
>> @@ -33,7 +33,7 @@
>>   *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_VALUE
>>   *    |                | ("xEn3" with the 0x80 bit of the "E" set).
>>   *  4 +----------------+
>> - *    | version        | Version of this structure. Current version is 1. New
>> + *    | version        | Version of this structure. Current version is 2. New
>>   *    |                | versions are guaranteed to be backwards-compatible.
>>   *  8 +----------------+
>>   *    | flags          | SIF_xxx flags.
>> @@ -55,7 +55,15 @@
>>   *    |                | if there is no memory map being provided. Only
>>   *    |                | present in version 1 and newer of the structure.
>>   * 52 +----------------+
>> - *    | reserved       | Version 1 and newer only.
>> + *    | vga_info.offset| Offset of struct dom0_vga_console_info from base of
> I'm not sure we are supposed to reference external structures like
> that. We took a lot of care to not depend on other headers, and to
> make this as agnostic as possible (IIRC KVM is also capable of using
> the PVH entry point natively, and hence depends on this header).

Absolutely correct.  C is not an acceptable ABI description.

Furthermore, dom0_vga_console_info is a bad ABI to start with, as
demonstrated by the multiple problems we've had extending it in the past.

The MB1/2 framebuffer information would be a rather better example to
follow, but we'll surely need to pass the EDID string too (at least in
the case that there aren't EFI runtime services to use).

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:10:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:10:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192937.343699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5lm-0001BA-1E; Wed, 22 Sep 2021 17:10:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192937.343699; Wed, 22 Sep 2021 17:10:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT5ll-0001B3-UF; Wed, 22 Sep 2021 17:10:53 +0000
Received: by outflank-mailman (input) for mailman id 192937;
 Wed, 22 Sep 2021 16:31:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mT59n-0004Al-3E
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 16:31:39 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8e312bdb-1bc2-11ec-b9b9-12813bfff9fa;
 Wed, 22 Sep 2021 16:31:37 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id D4A5120787;
 Wed, 22 Sep 2021 16:15:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e312bdb-1bc2-11ec-b9b9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327333;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding;
	bh=p52VqvjsCznljFY2xO4Rt/Nf2Duz4eWTG3QEtLzwHk4=;
	b=ew6f8RpIgUvMpeGq1oW3jHrqPK8H2iO+ccW5V5LopATHnIAc/5JmP0kO1GUCNtrVUBg0cJ
	QT30XZEqF9IENJ34znQQjNMjtUpMYNa0HJ6F++N+WdqDnf2XtZ2AtpOoC8qXWKxmwJ7ZAK
	u3YYtGzUu+N4fWJfxLOXmE9BpPy7KnE=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 00/16] Initial support for machine creation via QMP
Date: Wed, 22 Sep 2021 18:13:49 +0200
Message-Id: <20210922161405.140018-1-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Hi,

The goal of this work is to bring dynamic machine creation to QEMU:
we want to setup a machine without compiling a specific machine C
code. It would ease supporting highly configurable platforms (for
example resulting from an automated design flow). The requirements
for such configuration include begin able to specify the number of
cores, available peripherals, emmory mapping, IRQ mapping, etc.

This series focuses on the first step: populating a machine with
devices during its creation. We propose patches to support this
using QMP commands. This is a working set of patches and improves
over the earlier rfc (posted in May):
https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg03706.html

Although it is working and could be merged, it is tag as an RFC:
we probably need to discuss the conditions for allowing a device to
be created at an early stage. Patches 6, 10 and 13, 15 and 16 depend
on such conditions and are subject to change. Other patches are
unrelated to this point.

We address several issues in this series. They are detailed below.

## 1. Stoping QEMU to populate the machine with devices

QEMU goes through several steps (called _machine phases_) when
creating the machine: 'no-machine', 'machine-created',
'accel-created', 'initialized', and finally 'ready'. At 'ready'
phase, QEMU is ready to start (see Paolo's page
https://wiki.qemu.org/User:Paolo_Bonzini/Machine_init_sequence for
more details).

Using the -preconfig CLI option, QEMU can be stopped today during
the 'accel-created' phase. Then the 'x-exit-preconfig' QMP command
triggers QEMU moving forwards to the completion of the machine
creation ('ready' phase).

The devices are created during the 'initialized' phase. 
In this phase the machine init() method has been executed and thus
machine properties have been handled. Although the sysbus exists and
the machine may have been populated by the init(), 
_machine_init_done_ notifiers have not been called yet. At this point
we can add more devices to a machine.

We propose to add 2 QMP commands:
+ The 'query-machine-phase' command would return the current machine
  phase.
+ The 'x-machine-init' command would advance the machine phase to
  'initialized'. 'x-exit-preconfig' could then still be used to
  advance to the last phase.

## 2. Adding devices

Right now, the user can create devices in 2 ways: using '-device' CLI
option or 'device_add' QMP command. Both are executed after the
machine is ready: such devices are hot-plugged. We propose to allow
'device_add' QMP command to be used during the 'initialized' phase.

In this series, we keep the constraint that the device must be
'user-creatable' (this is a device class flag). We do not see any
reason why a device the user can hot-plug could not be created at an
earlier stage.

This part is still RFC because, as Peter mentioned it (in this thread
https://lists.gnu.org/archive/html/qemu-devel/2021-08/msg01933.html),
we may want additional or distinct conditions for:
+ device we can hot-plug
+ device we can add in '-preconfig' (cold-plug)
We are open to suggestions. We could for example add a
'preconfig-creatable' or 'init-creatable' flag to device class, which
can identify a set of devices we can create this way.

The main addition is how we handle the case of sysbus devices. Sysbus
devices are particular because unlike, for example, pci devices, you
have to manually handle the memory mapping and interrupts wiring. So
right now, a sysbus device is dynamically creatable (using -device
CLI option or device_add QMP command) only if:
+ it is 'user_creatable' (like any other device),
+ and it is in the current machine sysbus device allow list.

In this series, we propose to relax the second constraint during the
earlier phases of machine creation so that when using -preconfig we
can create any 'user-creatable' sysbus device. When the machine
progresses to the 'ready' phase, sysbus devices creation will come
back to the legacy behavior: it will be possible only based on the
per-machine authorization basis.

For sysbus devices, wiring interrupts is not a problem as we can use
the 'qom-set' QMP command, but memory mapping is.

## 3. Memory mapping

There is no point allowing the creation sysbus devices if we cannot
map them onto the memory bus (the 'sysbus').

As far as we know, right now, there is no way to add memory mapping
for sysbus device using QMP commands. We propose a 'x-sysbus-mmio-map'
command to do this. This command would only be allowed during the
'initialized' phase when using -preconfig.

## 4. Working example

The last patches of the series add and modify devices in order to
build a working machine starting from the 'none' machine.

We add a new sysbus device modeling a simple memory (ram or rom). We
also set 'user-creatable' flag of some sysbus devices. These are
trivial patches, but they depends on the conditions we choose to allow
creating devices with -preconfig. Therefore, there is really no need
to review them until we settled on the device conditions first.

With these devices (memory, ibex_uart, ibex_plic) we can dynamically
configure a part (we did not add the timer, but we could) the
opentitan machine very easily and run firmwares which demonstrates
interrupts and memory-mapping are working.

We use the existing qmp-shell script to issue machine devices
from a qmp commands script file which contains qmp commands listed in
a file.

The following qmp commands add some memories, an interrupt controller
and an uart with an interrupt.

cat > opentitan.qmp <<EOF
x-machine-init

# ROM 0x00008000
device_add        driver=sysbus-memory id=rom size=0x4000 readonly=true
x-sysbus-mmio-map device=rom addr=32768

# FLASH 0x20000000
device_add        driver=sysbus-memory id=flash size=0x80000 readonly=true
x-sysbus-mmio-map device=flash addr=536870912

# RAM 0x10000000
device_add        driver=sysbus-memory id=ram size=0x10000
x-sysbus-mmio-map device=ram addr=268435456

# PLIC 0x41010000
device_add        driver=ibex-plic id=plic
x-sysbus-mmio-map device=plic addr=1090584576

# UART 0x40000000
device_add        driver=ibex-uart id=uart chardev=serial0
x-sysbus-mmio-map device=uart addr=1073741824
qom-set path=uart property=sysbus-irq[1] value=plic/unnamed-gpio-in[2]

x-exit-preconfig
EOF

We've put the opentitan.qmp and a firmware opentitan-echo.elf here
(among some other qmp machine files we are working on):
https://github.com/GreenSocs/qemu-qmp-machines
This firmware is just a small interrupt-based program echoing back
whatever is sent in the uart.

QEMU should be run using the following command:
qemu-system-riscv32 -preconfig -qmp unix:/tmp/qmp-socket,server \
    -display none \
    -M none -cpu lowrisc-ibex \
    -serial mon:stdio \
    -device loader,addr=0x8090,cpu-num=0 \
    -device loader,file=opentitan-hello.elf \

and in other terminal to do the configuration (grep is just here to
remove comments):
grep -v -e '^#' opentitan.qmp | qmp-shell -v /tmp/qmp-socket

Alternatively we can load the firmware on the existing machine and
observe the same behavior:
qemu-system-riscv32 -display none \
     -M opentitan \
     -serial mon:stdio \
     -kernel opentitan-echo.elf

We chose this example because it is very simple and does not need a
lot of devices.

This work has still a lot of limitations. Cpus config is done the
normal way (the C machine does that): in our example we used the
'none' machine. We have work to do for handling backend
connection (for example net/nic are complicated) because the way it
is done in machine C code does not translate easily in QMP commands.
Firmware loading is also a bit tricky. We plan to work on this in
follow-up series.

The series is organized as follows:
- Patches 1 to 3 add qmp support to stop QEMU at an early phase
  to populate the machine with devices.
- Patches 4 to 6 prepare and allow issuing device_add during this phase.
- Patches 7 to 10 prepare and allow creating sysbus device during this phase.
- Patches 11 and 12 add the x-sysbus-mmio-map QMP command
- Patch 13 add the memory sysbus device to model ram and rom 
- Patch 14 adds some documentation
- Patches 15 and 16 set 'user_creatable' flag of ibex_uart and ibex_plic.

This work is supported by Greensocs, Sifive and Xilinx.

Thanks,
--
Damien

Damien Hedde (12):
  softmmu/qdev-monitor: add error handling in qdev_set_id
  qdev-monitor: prevent conflicts between qmp/device_add and cli/-device
  hw/core/machine: add machine_class_is_dynamic_sysbus_dev_allowed
  qdev-monitor: Check sysbus device type before creating it
  hw/core/machine: Remove the dynamic sysbus devices type check
  qdev-monitor: allow adding any sysbus device before machine is ready
  softmmu/memory: add memory_region_try_add_subregion function
  add x-sysbus-mmio-map qmp command
  hw/mem/system-memory: add a memory sysbus device
  docs/system: add doc about the initialized machine phase and an
    example
  hw/char/ibex_uart: set user_creatable
  hw/intc/ibex_plic: set user_creatable

Mirela Grujic (4):
  rename MachineInitPhase enum constants for QAPI compatibility
  qapi: Implement query-machine-phase QMP command
  qapi: Implement x-machine-init QMP command
  qapi: Allow device_add to execute in machine initialized phase

 docs/system/managed-startup.rst | 77 +++++++++++++++++++++++++++++
 qapi/machine.json               | 79 ++++++++++++++++++++++++++++++
 qapi/qdev.json                  | 24 ++++++++-
 include/exec/memory.h           | 22 +++++++++
 include/hw/boards.h             | 18 ++++++-
 include/hw/mem/sysbus-memory.h  | 32 ++++++++++++
 include/hw/qdev-core.h          | 30 +-----------
 include/monitor/qdev.h          | 25 +++++++++-
 hw/char/ibex_uart.c             |  1 +
 hw/core/machine-qmp-cmds.c      | 11 ++++-
 hw/core/machine.c               | 48 ++++++------------
 hw/core/qdev.c                  |  7 ++-
 hw/core/sysbus.c                | 41 ++++++++++++++++
 hw/intc/ibex_plic.c             |  1 +
 hw/mem/sysbus-memory.c          | 83 +++++++++++++++++++++++++++++++
 hw/pci/pci.c                    |  2 +-
 hw/usb/core.c                   |  2 +-
 hw/virtio/virtio-iommu.c        |  2 +-
 hw/xen/xen-legacy-backend.c     |  3 +-
 monitor/hmp.c                   |  2 +-
 monitor/misc.c                  |  2 +-
 softmmu/memory.c                | 22 ++++++---
 softmmu/qdev-monitor.c          | 86 +++++++++++++++++++++++++++------
 softmmu/vl.c                    | 23 ++++++---
 ui/console.c                    |  3 +-
 hw/mem/meson.build              |  2 +
 26 files changed, 547 insertions(+), 101 deletions(-)
 create mode 100644 include/hw/mem/sysbus-memory.h
 create mode 100644 hw/mem/sysbus-memory.c

-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:31:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:31:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192961.343709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT65q-0003pG-ND; Wed, 22 Sep 2021 17:31:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192961.343709; Wed, 22 Sep 2021 17:31:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT65q-0003p9-Jh; Wed, 22 Sep 2021 17:31:38 +0000
Received: by outflank-mailman (input) for mailman id 192961;
 Wed, 22 Sep 2021 17:31:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mT65p-0003p3-Uf
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:31:37 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id efd3b8be-1bca-11ec-b9c0-12813bfff9fa;
 Wed, 22 Sep 2021 17:31:36 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 98B7F21ED8;
 Wed, 22 Sep 2021 16:16:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efd3b8be-1bca-11ec-b9c0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327363;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=20eMulj3Qy/DZxWmekzYSt42bv07Z1NJC97opChMppc=;
	b=0GTRQNrK+8eC4nSIBI1SyRdVgk/yz3AcWIgo1jZc+3/ixKzyuHi/rbXlOmb7SF5gwUT4je
	Kafcyz3ZizvLJ94FKvPqVu6mxe7wLyPq9T2y/XORwkJswxh9nvcCQaC5xGv/EED3DiFvN+
	S0CxATzuTlW34AF4O/BZxkpylFrb5Mg=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 16/16] hw/intc/ibex_plic: set user_creatable
Date: Wed, 22 Sep 2021 18:14:05 +0200
Message-Id: <20210922161405.140018-17-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

This patch allows to create the device using device_add
using -preconfig mode. This sysbus device still needs to
be allowed by a machine to be created after preconfig is done.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Depending on chosen condition for a device to be added, this commit
may change.
---
 hw/intc/ibex_plic.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/intc/ibex_plic.c b/hw/intc/ibex_plic.c
index edf76e4f61..8abd5ee613 100644
--- a/hw/intc/ibex_plic.c
+++ b/hw/intc/ibex_plic.c
@@ -291,6 +291,7 @@ static void ibex_plic_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->user_creatable = true;
     dc->reset = ibex_plic_reset;
     device_class_set_props(dc, ibex_plic_properties);
     dc->realize = ibex_plic_realize;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:37:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:37:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192969.343721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6Bs-0004k3-H6; Wed, 22 Sep 2021 17:37:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192969.343721; Wed, 22 Sep 2021 17:37:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6Bs-0004jw-D7; Wed, 22 Sep 2021 17:37:52 +0000
Received: by outflank-mailman (input) for mailman id 192969;
 Wed, 22 Sep 2021 17:37:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tKcP=OM=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mT6Bq-0004jq-NY
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:37:50 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ce3ffac2-1bcb-11ec-b9c1-12813bfff9fa;
 Wed, 22 Sep 2021 17:37:49 +0000 (UTC)
Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com
 [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-145-s3trJ2EeMrStDEAWK4WXKQ-1; Wed, 22 Sep 2021 13:37:47 -0400
Received: by mail-wr1-f69.google.com with SMTP id
 j16-20020adfa550000000b0016012acc443so2828563wrb.14
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 10:37:47 -0700 (PDT)
Received: from [192.168.1.36] (118.red-83-35-24.dynamicip.rima-tde.net.
 [83.35.24.118])
 by smtp.gmail.com with ESMTPSA id v10sm3041925wri.29.2021.09.22.10.37.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 10:37:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce3ffac2-1bcb-11ec-b9c1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632332269;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w1l0AiXwDhCyCu/u0vTki8Ygujm+OCz4LudDX2/nmkU=;
	b=O3ocui0qCFzw/Q8mEsUfC9BGpUhdsun9gDBL8u9hgwcjKNWX2tjdxEIpNbHE6gWTMaLU+I
	pwRit7KWC6mhJrQf7pl+wPde8Km5Xh3q/pnD50zq103BuMoawBM19flDfe4i7ce7weitsR
	P5J9euhmmCcBMe9fa3vxcjsNxLBuk3Y=
X-MC-Unique: s3trJ2EeMrStDEAWK4WXKQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=w1l0AiXwDhCyCu/u0vTki8Ygujm+OCz4LudDX2/nmkU=;
        b=5ns83/uxcmdka/TeyKaA1ck4PvwzKpxTslVnXPItLzZNKj9WZ3mSBi02VRYLW1t9wf
         w42R+BLb7eHAkmvCbVAT+zum49ICWvJe+e9UlZE7nz4EG64crSJiHp+RZnk+aGMHnxbf
         NNs9qFH4qVI0eqfPCtMNmyuRCE4LcAbtoevr/aEW5of5xa3APXh2ZoKoBsN+vXWHqAWV
         rEsdKoaklpbW5i2wZjgVH5LLfD/6exQdI7U1dh9l79ywLUeJRHcMIkP5SGbbdELRfrC4
         NTveEcTXn5+qgP0Eiq09tTwgg+cwZbbmmTl09wSd1/jZarKgmjYqJa1UzreC5dO9R6Mc
         Xcow==
X-Gm-Message-State: AOAM531zsHwb4ZHMrZTCZseyvgKh4M5Kk+8CmWQSauS5zcJZljUEHPRv
	/uWbQn6ILti9AO0ViYvLew4158DKiGaKXpao8R8M+6rw05IWPmWn1AX4pH634uA7urIND9on3QS
	MKdQ0RiW6PTVUPoVie6JqZFiYsRc=
X-Received: by 2002:a05:600c:2046:: with SMTP id p6mr12223532wmg.88.1632332266708;
        Wed, 22 Sep 2021 10:37:46 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyg6vmpdLqfWALQ+aYScr0ufGB6YbggOHa9PzO72FTcIBc1TC/o6Tu07FmCA0vADFzchZXYUQ==
X-Received: by 2002:a05:600c:2046:: with SMTP id p6mr12223500wmg.88.1632332266506;
        Wed, 22 Sep 2021 10:37:46 -0700 (PDT)
Message-ID: <d27c5829-e28f-4d17-966e-6cb86ebe00e6@redhat.com>
Date: Wed, 22 Sep 2021 19:37:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 02/16] qapi: Implement query-machine-phase QMP
 command
To: Damien Hedde <damien.hedde@greensocs.com>, qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 Peter Xu <peterx@redhat.com>, mirela.grujic@greensocs.com,
 Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Ani Sinha <ani@anisinha.ca>,
 Eric Blake <eblake@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-3-damien.hedde@greensocs.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>
In-Reply-To: <20210922161405.140018-3-damien.hedde@greensocs.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 9/22/21 18:13, Damien Hedde wrote:
> From: Mirela Grujic <mirela.grujic@greensocs.com>
> 
> The command returns current machine initialization phase.
>  From now on, the MachineInitPhase enum is generated from the
> QAPI schema.
> 
> Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>

Missing your S-o-b, otherwise:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> ---
>   qapi/machine.json          | 56 ++++++++++++++++++++++++++++++++++++++
>   include/hw/qdev-core.h     | 30 ++------------------
>   hw/core/machine-qmp-cmds.c |  9 ++++++
>   hw/core/qdev.c             |  5 ++++
>   4 files changed, 72 insertions(+), 28 deletions(-)



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:42:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:42:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192975.343732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6Gd-00067V-40; Wed, 22 Sep 2021 17:42:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192975.343732; Wed, 22 Sep 2021 17:42:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6Gd-00067O-0Z; Wed, 22 Sep 2021 17:42:47 +0000
Received: by outflank-mailman (input) for mailman id 192975;
 Wed, 22 Sep 2021 17:42:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tKcP=OM=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mT6Gb-00067I-8Z
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:42:45 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 2bb333d3-ef26-49f8-86cf-69b74f0549dc;
 Wed, 22 Sep 2021 17:42:44 +0000 (UTC)
Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com
 [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-483-ggp9NKr6NNmscOmBfPvVtQ-1; Wed, 22 Sep 2021 13:42:42 -0400
Received: by mail-wr1-f71.google.com with SMTP id
 e1-20020adfa741000000b0015e424fdd01so2851944wrd.11
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 10:42:42 -0700 (PDT)
Received: from [192.168.1.36] (118.red-83-35-24.dynamicip.rima-tde.net.
 [83.35.24.118])
 by smtp.gmail.com with ESMTPSA id w1sm6588014wmc.19.2021.09.22.10.42.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 10:42:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2bb333d3-ef26-49f8-86cf-69b74f0549dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632332564;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uVaOmuvSmYMdDMUDJJ0nLDELyFbMOLlDnDD2b93jvo8=;
	b=Htwu1n9DeDxsziaCPMz9t6wz0Mw0GtG++cWp/nBcFvG6MM6eM5UywOJvFWeioZrknvIQMD
	BBYgMXFCWrpjbf59/YJNDt/e8o6cbWAEIRHGKmLGfdTK7EBPQeGgjt1uf/R6xGfw+B7q+X
	6GVImcLQ8pK8m9UKXmhJNv9hUnuVK+E=
X-MC-Unique: ggp9NKr6NNmscOmBfPvVtQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=uVaOmuvSmYMdDMUDJJ0nLDELyFbMOLlDnDD2b93jvo8=;
        b=oitZFyFq4UYLAInIGuCa5ZDaOwFcYbFyXQl6JorvRTAlYrf8vS9eA4sxlegKK86LUU
         UW5D/gYhRqxAwvdfbtcdCn4qSeso9gv/0stCHA23gQ1cxrgYPQ4bB3/XZ4gZIn3n85gj
         lXgFvpBgoaDUUNZAE3j8p7QmB5jcoasfqy+jVlZCABzifFkX97rf/MXlwGNPP0OoqHtj
         uWnlS90B1Zg4bmMdzZO5RhNUgWIGLA8+G2pBpmJUPYzMkTRKkuKR5CihC3BM7q4rjqMf
         E50ujdWcGBLe8JoOwKVQ/NCTHdVHcnpwzKSAZY3RI0MPx0rot3GseXskfol6iM9UloJ9
         VmJw==
X-Gm-Message-State: AOAM530+9d9hW8flowghdwAN0C3dmPOjMS3fKddXxEmKqchux8d2uKaW
	ewYslyIqZQrofHjOZhBOHUbR10iuy0UOvHvClkiWjgNJXyGTjmqDdDtf1r59O+qUd2FqznMhZIg
	ZtemzrT+bix+fOP5RIZ//DFSd/u0=
X-Received: by 2002:a1c:7310:: with SMTP id d16mr326728wmb.30.1632332561639;
        Wed, 22 Sep 2021 10:42:41 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyMUMMi8eLqjRXCSalitvGzPda/oA8H6U5KYwVIDPTTCJLTISya0LAgbbYFrl2db1iXLKfbNQ==
X-Received: by 2002:a1c:7310:: with SMTP id d16mr326712wmb.30.1632332561486;
        Wed, 22 Sep 2021 10:42:41 -0700 (PDT)
Message-ID: <bc37831d-4d31-8b02-acc9-c54191bbf31e@redhat.com>
Date: Wed, 22 Sep 2021 19:42:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 11/16] softmmu/memory: add
 memory_region_try_add_subregion function
To: Damien Hedde <damien.hedde@greensocs.com>, qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 Peter Xu <peterx@redhat.com>, mirela.grujic@greensocs.com,
 Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Ani Sinha <ani@anisinha.ca>,
 Eric Blake <eblake@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-12-damien.hedde@greensocs.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>
In-Reply-To: <20210922161405.140018-12-damien.hedde@greensocs.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 9/22/21 18:14, Damien Hedde wrote:
> It allows to try to add a subregion to a memory region with error
> handling. Like memory_region_add_subregion_overlap, it handles
> priority as well.
> Apart the error handling, the behavior is the same. It can be used
> to do the simple memory_region_add_subregion() (with no overlap) by
> setting the priority parameter to 0.
> 
> This commit is a preparation to further use this function in the
> context of qmp command which needs error handling support.
> 
> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> ---
> 
> Adding a new function is obviously not ideal. But there is ~900
> occurrences of memory_region_add_subregion[_overlap] calls in the code
> base. We do not really see an alternative here.
> ---
>   include/exec/memory.h | 22 ++++++++++++++++++++++
>   softmmu/memory.c      | 22 ++++++++++++++--------
>   2 files changed, 36 insertions(+), 8 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:49:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:49:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192982.343743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6N3-00070R-RM; Wed, 22 Sep 2021 17:49:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192982.343743; Wed, 22 Sep 2021 17:49:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6N3-00070K-N0; Wed, 22 Sep 2021 17:49:25 +0000
Received: by outflank-mailman (input) for mailman id 192982;
 Wed, 22 Sep 2021 17:49:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XTvp=OM=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mT6N3-0006zv-7o
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:49:25 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 6c2d21b5-1bcd-11ec-b9c4-12813bfff9fa;
 Wed, 22 Sep 2021 17:49:24 +0000 (UTC)
Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com
 [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-28-rlQXclYWMMeNbimgPpj5Jw-1; Wed, 22 Sep 2021 13:49:22 -0400
Received: by mail-wr1-f72.google.com with SMTP id
 c2-20020adfa302000000b0015e4260febdso2825999wrb.20
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 10:49:22 -0700 (PDT)
Received: from [192.168.3.132] (p5b0c64dd.dip0.t-ipconnect.de. [91.12.100.221])
 by smtp.gmail.com with ESMTPSA id m4sm3482985wrx.81.2021.09.22.10.49.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 10:49:20 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c2d21b5-1bcd-11ec-b9c4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632332963;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8R2eWgo3mCPiEWKHImVoU3s0Qt0O78iy+gHqpXKDMHg=;
	b=Ag07l4jLOAcBPXgy08f5PMjSH8BISpFcZxHWZ3NGnZCCw7EE04gLvZDd+bE4A/TshlKUM5
	Y6Z3qfmDFq7fmG6SXXJthZfZldme8mgOvhqpi/w/LyrcsIdAnS4XuiDMVtKiCYaiwv7yB5
	0Yum6r0WiiSNefvjClMXaBHTJ4Ou5Ec=
X-MC-Unique: rlQXclYWMMeNbimgPpj5Jw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:organization
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=8R2eWgo3mCPiEWKHImVoU3s0Qt0O78iy+gHqpXKDMHg=;
        b=XphXeaQiGrN+EzwMrl3bJuNCvau9YQsSDpDWsi63N2TSBMNljMzAp8H85WzFH/gl2d
         PqOzJ1f3t7rKhrHYmfL7rOwEDw+PfVyPo1cVA+bHkYVTGAujIfJUBdz9FNhzMXDNvmn4
         j0NLTXRO4NfcVlaVEwdIyYz6a+y7bdWdud97PqiyWONPrVr8Sm0AEQxSqbaVoVXDRGHB
         LQw+fWG6fYGgzYEbpq1ejC8Yaw3MQDEHUPIkK8ZEu89fUY/x33atEwW0p2/SJfQPGTDQ
         Qgnbusvf+KDl3GB1jhMW0WwtuExRpYt4faptpcmbdXgE3c5C6lSLw8SDICSLHGuVDguX
         y97g==
X-Gm-Message-State: AOAM531uPDnGx0UX0sQFTl8ctOl0+u1iVDnt4aIX9arckGh3yjfaLBvD
	nRxP/qhtwklZqdpIZJUfDh4DPjSVQUpDtgqMlIsM6Fn1GCQfjTTQU+ett3C78rO8SfouciXLw6b
	LAeQyYgAEXIu9y0Faee7XqrckRtE=
X-Received: by 2002:a05:6000:1844:: with SMTP id c4mr187151wri.355.1632332961339;
        Wed, 22 Sep 2021 10:49:21 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJwDZycGgzlG5PLVo/6Isj0bTJcO/O5J4WdjAYiBchkHxW5ecBClNDAyi9t+1qZPrUoaXfO12g==
X-Received: by 2002:a05:6000:1844:: with SMTP id c4mr187105wri.355.1632332961083;
        Wed, 22 Sep 2021 10:49:21 -0700 (PDT)
Subject: Re: [RFC PATCH v2 11/16] softmmu/memory: add
 memory_region_try_add_subregion function
To: Damien Hedde <damien.hedde@greensocs.com>, qemu-devel@nongnu.org
Cc: Alistair Francis <Alistair.Francis@wdc.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= <berrange@redhat.com>,
 "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>,
 Ani Sinha <ani@anisinha.ca>, Gerd Hoffmann <kraxel@redhat.com>,
 Eric Auger <eric.auger@redhat.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Peter Xu <peterx@redhat.com>, "Dr. David Alan Gilbert"
 <dgilbert@redhat.com>, Markus Armbruster <armbru@redhat.com>,
 Eric Blake <eblake@redhat.com>, qemu-riscv@nongnu.org,
 xen-devel@lists.xenproject.org, mark.burton@greensocs.com,
 mirela.grujic@greensocs.com, edgari@xilinx.com,
 Peter Maydell <peter.maydell@linaro.org>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-12-damien.hedde@greensocs.com>
From: David Hildenbrand <david@redhat.com>
Organization: Red Hat
Message-ID: <8ff0d8c9-94f1-cc21-a9f9-bb5fcd1cc3e7@redhat.com>
Date: Wed, 22 Sep 2021 19:49:19 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210922161405.140018-12-damien.hedde@greensocs.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 22.09.21 18:14, Damien Hedde wrote:
> It allows to try to add a subregion to a memory region with error
> handling. Like memory_region_add_subregion_overlap, it handles
> priority as well.
> Apart the error handling, the behavior is the same. It can be used
> to do the simple memory_region_add_subregion() (with no overlap) by
> setting the priority parameter to 0.
> 
> This commit is a preparation to further use this function in the
> context of qmp command which needs error handling support.
> 
> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> ---
> 
> Adding a new function is obviously not ideal. But there is ~900
> occurrences of memory_region_add_subregion[_overlap] calls in the code
> base. We do not really see an alternative here.
> ---
>   include/exec/memory.h | 22 ++++++++++++++++++++++
>   softmmu/memory.c      | 22 ++++++++++++++--------
>   2 files changed, 36 insertions(+), 8 deletions(-)
> 
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index c3d417d317..422e1eda67 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -2162,6 +2162,28 @@ void memory_region_add_subregion_overlap(MemoryRegion *mr,
>                                            MemoryRegion *subregion,
>                                            int priority);
>   
> +/**
> + * memory_region_try_add_subregion: Add a subregion to a container
> + *                                  with error handling.
> + *
> + * Behaves like memory_region_add_subregion_overlap(), but errors are
> + * reported if the subregion cannot be added.
> + *
> + * @mr: the region to contain the new subregion; must be a container
> + *      initialized with memory_region_init().
> + * @offset: the offset relative to @mr where @subregion is added.
> + * @subregion: the subregion to be added.
> + * @priority: used for resolving overlaps; highest priority wins.
> + * @errp: pointer to Error*, to store an error if it happens.
> + *
> + * Returns: True in case of success, false otherwise.
> + */
> +bool memory_region_try_add_subregion(MemoryRegion *mr,
> +                                     hwaddr offset,
> +                                     MemoryRegion *subregion,
> +                                     int priority,
> +                                     Error **errp);
> +
>   /**
>    * memory_region_get_ram_addr: Get the ram address associated with a memory
>    *                             region
> diff --git a/softmmu/memory.c b/softmmu/memory.c
> index bfedaf9c4d..eac61f8236 100644
> --- a/softmmu/memory.c
> +++ b/softmmu/memory.c
> @@ -2513,22 +2513,28 @@ done:
>       memory_region_transaction_commit();
>   }
>   
> -static void memory_region_add_subregion_common(MemoryRegion *mr,
> -                                               hwaddr offset,
> -                                               MemoryRegion *subregion)
> +bool memory_region_try_add_subregion(MemoryRegion *mr,
> +                                     hwaddr offset,
> +                                     MemoryRegion *subregion,
> +                                     int priority,
> +                                     Error **errp)
>   {
> -    assert(!subregion->container);
> +    if (subregion->container) {
> +        error_setg(errp, "The memory region is already in another region");
> +        return false;
> +    }
> +    subregion->priority = priority;
>       subregion->container = mr;
>       subregion->addr = offset;
>       memory_region_update_container_subregions(subregion);
> +    return true;
>   }
>   
>   void memory_region_add_subregion(MemoryRegion *mr,
>                                    hwaddr offset,
>                                    MemoryRegion *subregion)
>   {
> -    subregion->priority = 0;
> -    memory_region_add_subregion_common(mr, offset, subregion);
> +    memory_region_try_add_subregion(mr, offset, subregion, 0, &error_abort);
>   }
>   
>   void memory_region_add_subregion_overlap(MemoryRegion *mr,
> @@ -2536,8 +2542,8 @@ void memory_region_add_subregion_overlap(MemoryRegion *mr,
>                                            MemoryRegion *subregion,
>                                            int priority)
>   {
> -    subregion->priority = priority;
> -    memory_region_add_subregion_common(mr, offset, subregion);
> +    memory_region_try_add_subregion(mr, offset, subregion, priority,
> +                                    &error_abort);
>   }
>   
>   void memory_region_del_subregion(MemoryRegion *mr,
> 

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:50:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:50:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192988.343753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6O9-0008IS-4b; Wed, 22 Sep 2021 17:50:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192988.343753; Wed, 22 Sep 2021 17:50:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6O9-0008IL-1k; Wed, 22 Sep 2021 17:50:33 +0000
Received: by outflank-mailman (input) for mailman id 192988;
 Wed, 22 Sep 2021 17:50:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tKcP=OM=redhat.com=philmd@srs-us1.protection.inumbo.net>)
 id 1mT6O7-0008IB-GY
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:50:31 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 93f0cb4d-1bcd-11ec-b9c6-12813bfff9fa;
 Wed, 22 Sep 2021 17:50:30 +0000 (UTC)
Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com
 [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-156-2mtzewOPMd60zIYiITAvig-1; Wed, 22 Sep 2021 13:50:26 -0400
Received: by mail-wr1-f72.google.com with SMTP id
 r5-20020adfb1c5000000b0015cddb7216fso2887997wra.3
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 10:50:26 -0700 (PDT)
Received: from [192.168.1.36] (118.red-83-35-24.dynamicip.rima-tde.net.
 [83.35.24.118])
 by smtp.gmail.com with ESMTPSA id g205sm2940929wmg.18.2021.09.22.10.50.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 10:50:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93f0cb4d-1bcd-11ec-b9c6-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632333030;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=324sILhdYxvnznF1+1US9ZnfMNECSzr9WbsW6XTulSY=;
	b=Ou0LoOxFrU8eBBGGiJ8aRjCSz6wah96mL/M/YkR81+qA9EXAtFVRDM4+UxW46Y9a/s7Js9
	41q/50PCBbRB6kA22HL32CB85fRFBVdbIvGcV++7GCmB9WfD9YEb/AmtVYCHB1v63fzYeP
	8D6LdrNEhEPyfJvvUHQ/JQO9FZxAI7U=
X-MC-Unique: 2mtzewOPMd60zIYiITAvig-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=324sILhdYxvnznF1+1US9ZnfMNECSzr9WbsW6XTulSY=;
        b=b/iXN1EOqqXwI0SpFP2O4Uzj63LwdgdIdC1PkAQry/A7cLbLyV5hbQ4ceQ7s01qPeR
         Few0/shJKqO7lrZ8rEPiteiV5ee+lHoT4Qqojcn6nHVAwENRTpum3/1I2T9zBfZ/htmE
         qkCUmrK03LKk/W/h+otrEdH1q3NYciz93VBKLjI3vHR3/zQov7EPuZz8BADbPTNfUOHG
         IAYVMB5UEGRBQWvq7lIC3JTJYQ7JRsV9prpVtICd2rtAqn870xFoN/2UC52hVAMzBnYa
         RjdKGyHF4MygKCqA/V8IrVuNMseFIvG+h47uImz9tuz3aFqKmRpO8ZzsCMdSKfDZHu1X
         XcPg==
X-Gm-Message-State: AOAM530ZH9PGafALYTSIl5GwcXkmtqtCfnApb3Tcw48Drgn2tCuMGrbS
	FcD+IbTixTVMa/1l46LsZ1kNkNxMtXHeFRrx+/ssreC3s7XfhiMAgNRYlmDCGWJc/XGpl80/6R1
	acRQSCeXd5qeilYdxAjx8QZJPm7U=
X-Received: by 2002:adf:d084:: with SMTP id y4mr217248wrh.249.1632333025507;
        Wed, 22 Sep 2021 10:50:25 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxkTAxtj4TUqlJA16lEOK2zxOqi0Lnhv+gZIrdsyM50ICcEZWWLl648/QwzUYzfSBk/7cFBmA==
X-Received: by 2002:adf:d084:: with SMTP id y4mr217213wrh.249.1632333025297;
        Wed, 22 Sep 2021 10:50:25 -0700 (PDT)
Message-ID: <a4debda9-f00b-dc99-7e7c-37ea9b4d7b5c@redhat.com>
Date: Wed, 22 Sep 2021 19:50:22 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 00/16] Initial support for machine creation via QMP
To: Damien Hedde <damien.hedde@greensocs.com>, qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 Peter Xu <peterx@redhat.com>, mirela.grujic@greensocs.com,
 Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Ani Sinha <ani@anisinha.ca>,
 Eric Blake <eblake@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Damien,

On 9/22/21 18:13, Damien Hedde wrote:
> 
> The goal of this work is to bring dynamic machine creation to QEMU:
> we want to setup a machine without compiling a specific machine C
> code. It would ease supporting highly configurable platforms (for
> example resulting from an automated design flow). The requirements
> for such configuration include begin able to specify the number of
> cores, available peripherals, emmory mapping, IRQ mapping, etc.
> 
> This series focuses on the first step: populating a machine with
> devices during its creation. We propose patches to support this
> using QMP commands. This is a working set of patches and improves
> over the earlier rfc (posted in May):
> https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg03706.html

Do you have a roadmap for the following steps? Or are you done with
this series?

Yesterday I was thinking about this, and one thing I was wondering is
if it would be possible to have DeviceClass and MachineClass implement
a populate_fdt() handler, to automatically generate custom DTB for these
custom machines.

Maybe in your case you don't need that, as your framework generating the
QEMU machine also generates the DTB, or even parse a DTB to generate the
machine... :)

Regards,

Phil.



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 17:50:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 17:50:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.192990.343764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6OM-0000HD-DX; Wed, 22 Sep 2021 17:50:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 192990.343764; Wed, 22 Sep 2021 17:50:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6OM-0000H3-Ac; Wed, 22 Sep 2021 17:50:46 +0000
Received: by outflank-mailman (input) for mailman id 192990;
 Wed, 22 Sep 2021 17:50:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pxyX=OM=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mT6OK-0000En-JT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 17:50:44 +0000
Received: from mail-wr1-x42c.google.com (unknown [2a00:1450:4864:20::42c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2b936fbb-83c3-4531-a2cb-1fe5ed750b8f;
 Wed, 22 Sep 2021 17:50:43 +0000 (UTC)
Received: by mail-wr1-x42c.google.com with SMTP id i23so9426179wrb.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 10:50:43 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id c8sm2984483wru.30.2021.09.22.10.50.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 22 Sep 2021 10:50:41 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 265891FF96;
 Wed, 22 Sep 2021 18:50:41 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b936fbb-83c3-4531-a2cb-1fe5ed750b8f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=EowvngywS72jBE6RikxFXgqAP7QU+TKgMADLoAphN+0=;
        b=cRAwE43CJ0p2nCPQMHV0nj54ejgyt3xuERBZDKA5f2t86nHrc5A1U/PJ9zkn0y0Zkt
         twTViH0WRRUj3qQTLy68RmSXltVvPu66d5aSRH9PmEAz7iliPm9JGYExcd14D+8WhZ9u
         G40GMKh4ANdiBQtu2pJjUPb1VOyxkdaK/M/mGlyZ7iEl0ttRni1uD1szE1kpORfyTJuP
         eXIjUOVRswZhaE/O49CAQbJeXmGBLF2y8gboGWSGeJrmJBwwNZINhrFAcFEhFzU3d/w5
         eY+Y/hw5v1YuSWsuzvdqFRpZ37PQ0ev6OM9fE/63KjKUx/nGJ1KrDQUR93hTXV3XM6FN
         nQAw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=EowvngywS72jBE6RikxFXgqAP7QU+TKgMADLoAphN+0=;
        b=5UfkqmUpfaWkJ68Jt0egl7XcjlIXElCnDIvteqCRT9YmuKPuoj5M/LTJcrjUbFZnds
         QWcOsoytEvRCLEeyqtU59Atg02r6e+UxeEkaTIdZBfkkAhn5gL0a/zAqZkUaw1ImMl76
         8u3sSTOsI+BeXBkBbu3C9qDh95fyLjGDOhrBUxSCZ7KQPrRgyiymFcQ2Po+AKddLkM+N
         Zm8KqbLFoIKVdY19VHxCQAOZYJWO1kc75m8qSmgX6hb6UET+nXQAhfvllI0KAb4NVbbC
         vnIcGrauQHU9x44wbuM21XbkMJDBOd+wGqD+gJNBxw0ReMRGsp1JLt5pCxqo7tZ6C8ha
         WRVg==
X-Gm-Message-State: AOAM532sSFqIQuFZYZPLnxxWjKKf9EwpUI7pr14Z8Guo+rjAaPmg2K7W
	xV8JdSXOhLkaiPoIGEtAzNoufA==
X-Google-Smtp-Source: ABdhPJzRWalyaDRlvA4ti6UQssgLp7cmcVNXyjS70Jwd6eeSvG/1b8HOaBGQYuTHbywYFos5SkbPvQ==
X-Received: by 2002:a7b:c1c5:: with SMTP id a5mr346897wmj.144.1632333042791;
        Wed, 22 Sep 2021 10:50:42 -0700 (PDT)
References: <87lf40vay1.fsf@linaro.org>
 <04272e87a8939be46acddd3c75bbffa84b0a40c1.camel@infradead.org>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: David Woodhouse <dwmw2@infradead.org>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>, Viresh Kumar
 <viresh.kumar@linaro.org>, Wei Liu <wl@xen.org>, Alexandru Agache
 <aagch@amazon.com>, "rust-vmm@lists.opendev.org"
 <rust-vmm@lists.opendev.org>, xen-devel@lists.xenproject.org, Stratos
 Mailing List <stratos-dev@op-lists.linaro.org>, Oleksandr Tyshchenko
 <olekstysh@gmail.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Re: Is it time to start implementing Xen bindings for rust-vmm?
Date: Wed, 22 Sep 2021 18:44:41 +0100
In-reply-to: <04272e87a8939be46acddd3c75bbffa84b0a40c1.camel@infradead.org>
Message-ID: <871r5go5xq.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


David Woodhouse <dwmw2@infradead.org> writes:

> [[S/MIME Signed Part:Undecided]]
> On Mon, 2021-09-13 at 13:44 +0100, Alex Benn=C3=A9e wrote:
>> Hi,
>>=20
>> As we consider the next cycle for Project Stratos I would like to make
>> some more progress on hypervisor agnosticism for our virtio backends.
>> While we have implemented a number of virtio vhost-user backends using C
>> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
>> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
>> backends does it make sense to do some enabling work in rust-vmm to
>> support Xen?
>
> I like this idea.
>
> Somewhat separately, Alex Agache has already started some preliminary
> hacking on supporting Xen guests within rust-vmm (on top of Linux/KVM):
> https://github.com/alexandruag/vmm-reference/commits/xen

I'll be sending along a more detailed post once I've finished my
work breakdown but I'm currently envisioning two parts. A xen-sys crate
for the low level access that supports both ioctl and hypercall calls.
This would be useful for other projects such as stubdomains (think a
"bare-metal" RTOS with some sort of backend, uni-kernel style). It would
also be the lowest layer that rust-vmm can use to interact with the
hypervisor.

I'm aware the HyperV solution is to present a KVM-like ioctl interface
via the host kernel. However if we want generality with type-1
hypervisors we can't assume all will get suitable translation layers in
the kernel.

Fortunately for the time being our focus is on virtio backends so we
don't need to get directly involved in the hypervisor run loop... for
now.

> Being able to run on *actual* Xen would be good too. And we should also
> aspire to do guest-transparent live migration between the two hosting
> environments.
>
> Where relevant, it would be great to be able to share components (like
> emulation of the Xen PCI platform device, a completely single-tenant
> XenStore implementation dedicated to a single guest, perhaps PV
> netback/blkback and other things).

For Stratos portable virtio backends is one of our project goals.
>
>
> [[End of S/MIME Signed Part]]


--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 18:07:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 18:07:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193002.343776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6e5-0002LA-1X; Wed, 22 Sep 2021 18:07:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193002.343776; Wed, 22 Sep 2021 18:07:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6e4-0002L3-UC; Wed, 22 Sep 2021 18:07:00 +0000
Received: by outflank-mailman (input) for mailman id 193002;
 Wed, 22 Sep 2021 18:06:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT6e3-0002Kt-Lq
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 18:06:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e077f100-1bcf-11ec-b9c8-12813bfff9fa;
 Wed, 22 Sep 2021 18:06:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3002F6103C;
 Wed, 22 Sep 2021 18:06:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e077f100-1bcf-11ec-b9c8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632334017;
	bh=kj4DT4Ve2Tgqljtt6pKuRqGj9inEbX9Imtax6vVXr8I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fmmGp3KAY2OfzHtizPuVVEvI4DHz2UvGjkOq5Mw7IilOpnFIJYlb9Njfx23cghsdv
	 GdhaUy6/E/VpOCWHatWkgv8XspEyuSQW5Fqcs9I5/vcNjPfCwqpYn/kSHXC6h8pHev
	 fEhf/FBzklrOqikHUhp79WFaVlUK9700WlSgw30qcotpsNbcjGofcu0BqW21+ZLd04
	 6Hr24h7fhRoAa5jtkOWD26iCcWR/EsTX6NAVAJSGEAL6039+P8bxprwbses5TIf9+A
	 3muanXuclN3kst/2oD2PMLE68EGaK+OI5fjZBsLYfpS6NmT9DtMVNM7UjakUNGJScX
	 CLRGnDPiYNkGQ==
Date: Wed, 22 Sep 2021 11:06:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Ian Jackson <iwj@xenproject.org>
Subject: Re: [PATCH] common: guest_physmap_add_page()'s return value needs
 checking
In-Reply-To: <ad798aae-0f8e-219f-59d4-7ff5fe52c984@suse.com>
Message-ID: <alpine.DEB.2.21.2109221105080.17979@sstabellini-ThinkPad-T480s>
References: <ea5d1c22-967c-b11c-bc6f-9a8cac9a9823@suse.com> <YUmj3P3j6f5Gl/xh@MacBook-Air-de-Roger.local> <58e2e6e2-c154-f09f-193e-6bef2b42995f@suse.com> <YUm3yLOfAfqrwgmH@MacBook-Air-de-Roger.local> <24905.47298.105494.711706@mariner.uk.xensource.com>
 <ad798aae-0f8e-219f-59d4-7ff5fe52c984@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-667531506-1632334017=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-667531506-1632334017=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 Sep 2021, Jan Beulich wrote:
> On 21.09.2021 12:49, Ian Jackson wrote:
> > Roger Pau Monné writes ("Re: [PATCH] common: guest_physmap_add_page()'s return value needs checking"):
> >> On Tue, Sep 21, 2021 at 12:28:12PM +0200, Jan Beulich wrote:
> >>> On 21.09.2021 11:20, Roger Pau Monné wrote:
> >>>> On Wed, Sep 01, 2021 at 06:06:37PM +0200, Jan Beulich wrote:
> >>>>> The function may fail; it is not correct to indicate "success" in this
> >>>>> case up the call stack. Mark the function must-check to prove all
> >>>>> cases have been caught (and no new ones will get introduced).
> >>>>>
> >>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>>
> >>>> Acked-by: Roger Pau Monné <roger.pau@citrix.com>
> >>>
> >>> Thanks. Albeit strictly speaking an ack here isn't enough for the change
> >>> to go in, it would need to be R-b or come from a REST maintainer.
> >>
> >> Oh, FE:
> >>
> >> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> > 
> > Acked-by: Ian Jackson <iwj@xenproject.org>
> 
> With these, any chance of getting an Arm side ack here as well?

Acked-by: Stefano Stabellini <sstabellini@kernel.org>
--8323329-667531506-1632334017=:17979--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 18:19:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 18:19:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193008.343787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6pl-0004BN-0X; Wed, 22 Sep 2021 18:19:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193008.343787; Wed, 22 Sep 2021 18:19:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6pk-0004BG-TZ; Wed, 22 Sep 2021 18:19:04 +0000
Received: by outflank-mailman (input) for mailman id 193008;
 Wed, 22 Sep 2021 18:19:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JCiA=OM=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mT6pk-0004BA-1g
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 18:19:04 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c754cfad-bc99-4853-84a0-f393c54364a9;
 Wed, 22 Sep 2021 18:19:02 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id y28so15381574lfb.0
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 11:19:02 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id c4sm235296lfr.59.2021.09.22.11.18.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 11:19:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c754cfad-bc99-4853-84a0-f393c54364a9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=cdYrxpSpDGHg9um55NhNlBekvRpOeiukL4q9FZQajXE=;
        b=VxZbTzo5LCN+0LPMsq7iMB2RnKDBOTx91/hktHfmUT2o+FVy/DDIug+jd0g8S3WTQ3
         JSAwNZf861jfjtebX1b1rD9AmeI80niHcfOOR3lyFCEA97Avw32MX37wDuF5/SLeaB6U
         RHLl0kbOAMZlW7i9GjxpkL/jtIetxt5CJnM5I0PoL+M9tkShhHOUFyvUX3xhBpTozuzC
         TEClfbambtr5NcEYyaey+Yg4s5ruu2YGYAv4io4iWjnVOYkK7OjHXtr8X71WPdqMnHrC
         W9qH2/GaXNe4sYn6svmUMrifx/ZIfM1Rt93eCLXUrNiEu4qN3Vbs/GIojGt5/dIWgjW8
         XSmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=cdYrxpSpDGHg9um55NhNlBekvRpOeiukL4q9FZQajXE=;
        b=GUgOUonp6OPXHUZDZudKPJxEShwKKOPCqZAt5gv9R6DQjyr9dI/cKLsJOZnG2fwlOQ
         sbeW0+16dYgHnmrxvIIoeuV7zymnbQi4avMnX0CSh+j5mFhgZlfd43LVj9eujhm47iDT
         ALqEm0bscx51cxxF0qEplnROa8PYuTi0dSzLDiZOJWcFDmHGFmD5Ou0ji3XB95fiVgl8
         BerUoC+yBhbmk14f8i5p3eNht+n+1pP+SJAwuG4+INYZf9CbCrn8urMrBXz2jAOoCfEf
         k2pvEc+cZPP99FQ4vqnYEYkyZZOxNz1aQ48MtFMVByO+GVNpnyLfpVw7W5GzrXw4kL/s
         0v1w==
X-Gm-Message-State: AOAM533QfdnniTjFkYXQj3hzpyNXfdqXJGwWg5uMd55U0QzwN7Q8UM5c
	TA0lLnMjwkqerUJlgfKHaok=
X-Google-Smtp-Source: ABdhPJxc70A3BpvdY40pWQ+iuxwUtdYUwY4Zz0aha42T6numlyxn29/up8j13uHPBX1JGalVPkNDeA==
X-Received: by 2002:a05:6512:455:: with SMTP id y21mr358847lfk.548.1632334740428;
        Wed, 22 Sep 2021 11:19:00 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
From: Oleksandr <olekstysh@gmail.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <a993466f-1b7d-ceb3-aa3b-16f5f145b2fb@gmail.com>
Message-ID: <a3f5707b-b161-1c94-ebe1-d6b8bbe11081@gmail.com>
Date: Wed, 22 Sep 2021 21:18:59 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <a993466f-1b7d-ceb3-aa3b-16f5f145b2fb@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US



Hi Stefano


[snip]


>>>
>>>
>>>>
>>>> You will also need to cover "ranges" that will describe the BARs 
>>>> for the PCI
>>>> devices.
>>> Good point.
>> Yes, very good point!
>>
>>
>>> Could you please clarify how to recognize whether it is a PCI
>>> device as long as PCI support is not merged? Or just to find any 
>>> device nodes
>>> with non-empty "ranges" property
>>> and retrieve addresses?
>> Normally any bus can have a ranges property with the aperture and
>> possible address translations, including /amba (compatible =
>> "simple-bus"). However, in these cases dt_device_get_address already
>> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>>
>> The PCI bus is special for 2 reasons:
>> - the ranges property has a different format
>> - the bus is hot-pluggable
>>
>> So I think the only one that we need to treat specially is PCI.
>>
>> As far as I am aware PCI is the only bus (or maybe just the only bus
>> that we support?) where ranges means the aperture.
> Thank you for the clarification. I need to find device node with 
> non-empty ranges property
> (and make sure that device_type property is "pci"), after that I need 
> to read the context of ranges property and translate it.
>
>

OK, I experimented with that and managed to parse ranges property for 
PCI host bridge node.

I tested on my setup where the host device tree contains two PCI host 
bridge nodes with the following:

pcie@fe000000 {
...
             ranges = <0x1000000 0x0 0x0 0x0 0xfe100000 0x0 0x100000 
0x2000000 0x0 0xfe200000 0x0 0xfe200000 0x0 0x200000 0x2000000 0x0 
0x30000000 0x0 0x30000000 0x0 0x8000000 0x42000000 0x0 0x38000000 0x0 
0x38000000 0x0 0x8000000>;
...
};

pcie@ee800000 {
...
             ranges = <0x1000000 0x0 0x0 0x0 0xee900000 0x0 0x100000 
0x2000000 0x0 0xeea00000 0x0 0xeea00000 0x0 0x200000 0x2000000 0x0 
0xc0000000 0x0 0xc0000000 0x0 0x8000000 0x42000000 0x0 0xc8000000 0x0 
0xc8000000 0x0 0x8000000>;
...
};

So Xen retrieves the *CPU addresses* from the ranges:

(XEN) dev /soc/pcie@fe000000 range_size 7 nr_ranges 4
(XEN) 0: addr=fe100000, size=100000
(XEN) 1: addr=fe200000, size=200000
(XEN) 2: addr=30000000, size=8000000
(XEN) 3: addr=38000000, size=8000000
(XEN) dev /soc/pcie@ee800000 range_size 7 nr_ranges 4
(XEN) 0: addr=ee900000, size=100000
(XEN) 1: addr=eea00000, size=200000
(XEN) 2: addr=c0000000, size=8000000
(XEN) 3: addr=c8000000, size=8000000

The code below covers ranges property in the context of finding memory 
holes (to be squashed with current patch):

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d37156a..7d20c10 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -834,6 +834,8 @@ static int __init find_memory_holes(struct meminfo 
*ext_regions)
      {
          unsigned int naddr;
          u64 addr, size;
+        const __be32 *ranges;
+        u32 len;

          naddr = dt_number_of_address(np);

@@ -857,6 +859,41 @@ static int __init find_memory_holes(struct meminfo 
*ext_regions)
                  goto out;
              }
          }
+
+        /*
+         * Also looking for non-empty ranges property which would 
likely mean
+         * that we deal with PCI host bridge device and the property here
+         * describes the BARs for the PCI devices.
+         */
+        ranges = dt_get_property(np, "ranges", &len);
+        if ( ranges && len )
+        {
+            unsigned int range_size, nr_ranges;
+            int na, ns, pna;
+
+            pna = dt_n_addr_cells(np);
+            na = dt_child_n_addr_cells(np);
+            ns = dt_child_n_size_cells(np);
+            range_size = pna + na + ns;
+            nr_ranges = len / sizeof(__be32) / range_size;
+
+            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
+            {
+                /* Skip the child address and get the parent (CPU) 
address */
+                addr = dt_read_number(ranges + na, pna);
+                size = dt_read_number(ranges + na + pna, ns);
+
+                start = addr & PAGE_MASK;
+                end = PAGE_ALIGN(addr + size);
+                res = rangeset_remove_range(mem_holes, start, end - 1);
+                if ( res )
+                {
+                    printk(XENLOG_ERR "Failed to remove: 
%#"PRIx64"->%#"PRIx64"\n",
+                           start, end);
+                    goto out;
+                }
+            }
+        }
      }

      start = 0;



-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 18:25:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 18:25:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193014.343797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6w1-0005ZQ-NI; Wed, 22 Sep 2021 18:25:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193014.343797; Wed, 22 Sep 2021 18:25:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT6w1-0005ZJ-KM; Wed, 22 Sep 2021 18:25:33 +0000
Received: by outflank-mailman (input) for mailman id 193014;
 Wed, 22 Sep 2021 18:25:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JCiA=OM=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mT6w0-0005ZC-L6
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 18:25:32 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc3eb3b6-3a9a-4f7b-92c2-8cbff5ec2eea;
 Wed, 22 Sep 2021 18:25:30 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id t10so15559116lfd.8
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 11:25:30 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id e24sm234918lfs.212.2021.09.22.11.25.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 22 Sep 2021 11:25:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc3eb3b6-3a9a-4f7b-92c2-8cbff5ec2eea
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=x/DVJr55rNK1aKExuqro/CT4+lVj6KxJmR3lsFddbiY=;
        b=jniApQSTPmYEn62iCCYhnB9fjTFoKH/z8dE7CUIzQ8g5r7aUMMjocQRSaykzsvWB/X
         E5V7LVcuICb0Lvi6ZNjdKoJrINsMaOb4tZ3KVLeBiXui0Pa5dm+jen8IxPdKxfv/ievW
         nmy5hqRTbCDLvhgZvcpTAD2i8pB3o1uqUcHz1Nc7PikX2K2yxe8fV7H/rS+NOqJjU5W2
         OJHiFJ+uHVwZdJ0V48AcjnL15x6Az8I63jkigncMrkxW5WjSxD9zh4NLdR4g1od9gJiu
         aaSEB7yJCJCbmF3oZBBYJxXlos+NOBzF+H4iFqQA6fdysSKJAZ9uGJspYTMOETB36vWD
         uU3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=x/DVJr55rNK1aKExuqro/CT4+lVj6KxJmR3lsFddbiY=;
        b=P2L/mkAZ8R2LP6Tle9Siy96u3PaqNEP+qSjQtHoOY9IDCnq8gIy9z+Thn6YRswiNHs
         Xbn1IOicX7xVL/svJcUUbLWFflZboIxLporcopzLWf8RgC+UdCBV0N4VNJ11xgL/o7fj
         KAuRG1qthfokTuvX8O6HRF8A6oQ5KIi3wRAROQkzwHKV969lhm/Owj0dT/hkuIAqYvpa
         FbKczNjpu/hmOfaX9O2bBFSFwlwGNi0EYAIRBvJcOIj1Qi0BFI1QTJgNQ063kV2FnjfC
         ZK+SaMbjLdKxAu1WbKc+EuMqCVbjYI/PepG7MKzvBd8k5EpgQTvcywNwE9WKk692TRZ7
         8/2A==
X-Gm-Message-State: AOAM530BK6ycn99L5hAaTx8qF1WuyixeJknYzQdNil17Qld6aLGrtekI
	aqsB6lg9wCyQqX//TqZkerg=
X-Google-Smtp-Source: ABdhPJwZL7xvIpkWxCYVKYd2z99sxT0Nqe9MGr3TjuLKFVsBfcqP3ef+DqLuF0lPg/WbMb3F1Avzkw==
X-Received: by 2002:a05:6512:4c5:: with SMTP id w5mr387502lfq.252.1632335129383;
        Wed, 22 Sep 2021 11:25:29 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, fnuv@xilinx.com
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
 <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
 <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
 <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
 <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
 <alpine.DEB.2.21.2109211439370.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <1ac58681-ef42-bb32-25b6-620d51d4f075@gmail.com>
Date: Wed, 22 Sep 2021 21:25:28 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109211439370.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 22.09.21 01:00, Stefano Stabellini wrote:

Hi Stefano

> On Tue, 21 Sep 2021, Oleksandr wrote:
>> On 21.09.21 02:21, Stefano Stabellini wrote:
>>> On Sun, 19 Sep 2021, Oleksandr wrote:
>>>>> On 18/09/2021 03:37, Stefano Stabellini wrote:
>>>>>> On Fri, 17 Sep 2021, Stefano Stabellini wrote:
>>>>>>> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>>>>>>>> +
>>>>>>>>>> +    dt_dprintk("Find unallocated memory for extended
>>>>>>>>>> regions\n");
>>>>>>>>>> +
>>>>>>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>>>>>>> +    if ( !unalloc_mem )
>>>>>>>>>> +        return -ENOMEM;
>>>>>>>>>> +
>>>>>>>>>> +    /* Start with all available RAM */
>>>>>>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>>>>>>> +    {
>>>>>>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>>>>>>> +        end = bootinfo.mem.bank[i].start +
>>>>>>>>>> bootinfo.mem.bank[i].size - 1;
>>>>>>>>>> +        res = rangeset_add_range(unalloc_mem, start, end);
>>>>>>>>>> +        if ( res )
>>>>>>>>>> +        {
>>>>>>>>>> +            printk(XENLOG_ERR "Failed to add:
>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>> +                   start, end);
>>>>>>>>>> +            goto out;
>>>>>>>>>> +        }
>>>>>>>>>> +    }
>>>>>>>>>> +
>>>>>>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>>>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>>>>>>> +    {
>>>>>>>>>> +        start = assign_mem->bank[i].start;
>>>>>>>>>> +        end = assign_mem->bank[i].start +
>>>>>>>>>> assign_mem->bank[i].size - 1;
>>>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>>>> +        if ( res )
>>>>>>>>>> +        {
>>>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>> +                   start, end);
>>>>>>>>>> +            goto out;
>>>>>>>>>> +        }
>>>>>>>>>> +    }
>>>>>>>>>> +
>>>>>>>>>> +    /* Remove reserved-memory regions */
>>>>>>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>>>>>>>>> +    {
>>>>>>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>>>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>>>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>>>> +        if ( res )
>>>>>>>>>> +        {
>>>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>> +                   start, end);
>>>>>>>>>> +            goto out;
>>>>>>>>>> +        }
>>>>>>>>>> +    }
>>>>>>>>>> +
>>>>>>>>>> +    /* Remove grant table region */
>>>>>>>>>> +    start = kinfo->gnttab_start;
>>>>>>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>>>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>>>> +    if ( res )
>>>>>>>>>> +    {
>>>>>>>>>> +        printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>> +               start, end);
>>>>>>>>>> +        goto out;
>>>>>>>>>> +    }
>>>>>>>>>> +
>>>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>>>>>>> +                                 add_ext_regions,
>>>>>>>>>> ext_regions);
>>>>>>>>>> +    if ( res )
>>>>>>>>>> +        ext_regions->nr_banks = 0;
>>>>>>>>>> +    else if ( !ext_regions->nr_banks )
>>>>>>>>>> +        res = -ENOENT;
>>>>>>>>>> +
>>>>>>>>>> +out:
>>>>>>>>>> +    rangeset_destroy(unalloc_mem);
>>>>>>>>>> +
>>>>>>>>>> +    return res;
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> +static int __init find_memory_holes(const struct kernel_info
>>>>>>>>>> *kinfo,
>>>>>>>>>> +                                    struct meminfo
>>>>>>>>>> *ext_regions)
>>>>>>>>>> +{
>>>>>>>>>> +    struct dt_device_node *np;
>>>>>>>>>> +    struct rangeset *mem_holes;
>>>>>>>>>> +    paddr_t start, end;
>>>>>>>>>> +    unsigned int i;
>>>>>>>>>> +    int res;
>>>>>>>>>> +
>>>>>>>>>> +    dt_dprintk("Find memory holes for extended regions\n");
>>>>>>>>>> +
>>>>>>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>>>>>>> +    if ( !mem_holes )
>>>>>>>>>> +        return -ENOMEM;
>>>>>>>>>> +
>>>>>>>>>> +    /* Start with maximum possible addressable physical
>>>>>>>>>> memory
>>>>>>>>>> range */
>>>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>>>>>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>>>>>>> +    if ( res )
>>>>>>>>>> +    {
>>>>>>>>>> +        printk(XENLOG_ERR "Failed to add:
>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>> +               start, end);
>>>>>>>>>> +        goto out;
>>>>>>>>>> +    }
>>>>>>>>>> +
>>>>>>>>>> +    /* Remove all regions described by "reg" property (MMIO,
>>>>>>>>>> RAM,
>>>>>>>>>> etc) */
>>>>>>>>> Well... The loop below is not going to handle all the regions
>>>>>>>>> described in
>>>>>>>>> the property "reg". Instead, it will cover a subset of "reg"
>>>>>>>>> where
>>>>>>>>> the
>>>>>>>>> memory is addressable.
>>>>>>>> As I understand, we are only interested in subset of "reg" where
>>>>>>>> the
>>>>>>>> memory is
>>>>>>>> addressable.
>>>>>>>>
>>>>>>>>
>>>>>>>>> You will also need to cover "ranges" that will describe the BARs
>>>>>>>>> for
>>>>>>>>> the PCI
>>>>>>>>> devices.
>>>>>>>> Good point.
>>>>>>> Yes, very good point!
>>>>>>>
>>>>>>>
>>>>>>>> Could you please clarify how to recognize whether it is a PCI
>>>>>>>> device as long as PCI support is not merged? Or just to find any
>>>>>>>> device nodes
>>>>>>>> with non-empty "ranges" property
>>>>>>>> and retrieve addresses?
>>>>>>> Normally any bus can have a ranges property with the aperture and
>>>>>>> possible address translations, including /amba (compatible =
>>>>>>> "simple-bus"). However, in these cases dt_device_get_address already
>>>>>>> takes care of it, see
>>>>>>> xen/common/device_tree.c:dt_device_get_address.
>>>>>>>
>>>>>>> The PCI bus is special for 2 reasons:
>>>>>>> - the ranges property has a different format
>>>>>>> - the bus is hot-pluggable
>>>>>>>
>>>>>>> So I think the only one that we need to treat specially is PCI.
>>>>>>>
>>>>>>> As far as I am aware PCI is the only bus (or maybe just the only bus
>>>>>>> that we support?) where ranges means the aperture.
>>>>>> Now that I think about this, there is another "hotpluggable" scenario
>>>>>> we
>>>>>> need to think about:
>>>>>>
>>>>>> [1] https://marc.info/?l=xen-devel&m=163056546214978
>>>>>>
>>>>>> Xilinx devices have FPGA regions with apertures currently not
>>>>>> described
>>>>>> in device tree, where things can programmed in PL at runtime making
>>>>>> new
>>>>>> devices appear with new MMIO regions out of thin air.
>>>>>>
>>>>>> Now let me start by saying that yes, the entire programmable region
>>>>>> aperture could probably be described in device tree, however, in
>>>>>> reality it is not currently done in any of the device trees we use
>>>>>> (including the upstream device trees in linux.git).
>>>>> This is rather annoying, but not unheard. There are a couple of
>>>>> platforms
>>>>> where the MMIOs are not fully described in the DT.
>>>>>
>>>>> In fact, we have a callback 'specific_mappings' which create additional
>>>>> mappings (e.g. on the omap5) for dom0.
>>>>>
>>>>>> So, we have a problem :-(
>>>>>>
>>>>>>
>>>>>> I can work toward getting the right info on device tree, but in
>>>>>> reality
>>>>>> that is going to take time and for now the device tree doesn't have
>>>>>> the
>>>>>> FPGA aperture in it. So if we accept this series as is, it is going to
>>>>>> stop features like [1] from working. >
>>>>>> If we cannot come up with any better plans, I think it would be better
>>>>>> to drop find_memory_holes, only rely on find_unallocated_memory even
>>>>>> when the IOMMU is on. One idea is that we could add on top of the
>>>>>> regions found by find_unallocated_memory any MMIO regions marked as
>>>>>> xen,passthrough: they are safe because they are not going to dom0
>>>>>> anyway.
>>>>> (Oleksandr, it looks like some rationale about the different approach is
>>>>> missing in the commit message. Can you add it?)
>>>> Yes sure, but let me please clarify what is different approach in this
>>>> context. Is it to *also* take into the account MMIO regions of the devices
>>>> for
>>>> passthrough for case when IOMMU is off (in addition to unallocated
>>>> memory)? If
>>>> yes, I wonder whether we will gain much with that according to that
>>>> device's
>>>> MMIO regions are usually not big enough and we stick to allocate extended
>>>> regions with bigger size (> 64MB).
>>> That's fair enough. There are a couple of counter examples where the
>>> MMIO regions for the device to assign are quite large, for instance a
>>> GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
>>> but maybe they are not that common. I am not sure if it is worth
>>> scanning the tree for xen,passthrough regions every time at boot for
>>> this.
>> ok, I will add a few sentences to commit message about this different approach
>> for now. At least this could be implemented later on if there is a need.
> One thing that worries me about this is that if we take an old Xen with
> this series and run it on a new board, it might cause problems. At the
> very least [1] wouldn't work.

I got it.


>
> Can we have a Xen command line argument to disable extended regions as
> an emergecy toggle?

I think, yes. If no preference for the argument name I will name it 
"no-ext-region".

>
>
> [1] https://marc.info/?l=xen-devel&m=163056546214978

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 18:32:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 18:32:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193020.343809 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT731-00079d-GS; Wed, 22 Sep 2021 18:32:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193020.343809; Wed, 22 Sep 2021 18:32:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT731-00079W-Cr; Wed, 22 Sep 2021 18:32:47 +0000
Received: by outflank-mailman (input) for mailman id 193020;
 Wed, 22 Sep 2021 18:32:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mT730-00079Q-6a
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 18:32:46 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7a4ac926-1bd3-11ec-b9c9-12813bfff9fa;
 Wed, 22 Sep 2021 18:32:45 +0000 (UTC)
Received: from pps.filterd (m0246627.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MI4GX5020387; 
 Wed, 22 Sep 2021 18:32:15 GMT
Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4re71w-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 18:32:14 +0000
Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1])
 by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18MIKq6j127725;
 Wed, 22 Sep 2021 18:31:58 GMT
Received: from nam10-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam10lp2102.outbound.protection.outlook.com [104.47.58.102])
 by aserp3020.oracle.com with ESMTP id 3b7q5b3e82-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 18:31:58 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BL0PR10MB2833.namprd10.prod.outlook.com (2603:10b6:208:78::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Wed, 22 Sep
 2021 18:31:56 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 18:31:56 +0000
Received: from [10.74.85.221] (138.3.201.29) by
 SN4PR0501CA0142.namprd05.prod.outlook.com (2603:10b6:803:2c::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Wed, 22 Sep 2021 18:31:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a4ac926-1bd3-11ec-b9c9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=upcw/uKfj88WDqSm8FUr1ULOeaphpyyhI9fexerZL3I=;
 b=UnyTwL74sWKlwKH1a5yP7BvpXgLZ0inIPo+8pYkxqmVbve1OABZqm6JBOryEgITmGe4d
 v3ZqJKxzqZ3wtIwtlgn47QSPQZnANardSBitueFulBSqcAp5lKBgNHfluO689/OR82JX
 mtmKyeE1V+grks6NMvl51iYGt6hn1BPdBeC7WCQ9xQdOqvMXcwa2xGTMU8oyAaf34yRu
 25TXmsxHtn8XYxVQ7GHu6W0WOpixHJbum9WGw9jvExJivzct0t1dohZi3WKgYwcg8yHZ
 sPVXS4hO7VbQ0LD/hdv/aM1ujtd3Aii0bsiSaJ9YXtLIN26RTN90jDFE9I2DslHZNBae Zg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jW8X2+wlu25SgNrag+nc0VDWlTpFbMQENLYtTy+cBle78zgeXFPelgxb6fXIY3umTkwc6dwuSe4sqHUxNbPy0hB0V6yHCdTnzYUylodSjG46teOJc9/XHb48qKkvwnizKdpJFrw/t8XsaCs+RsMcVMsW+K/Hhbs6p8Z3bJkF3T3ohlxBKcFGCwCMtUTnpFB3ZWVr6ZK41q+Koxz4mkwBQKOupF+SEKs3RabxqDKNdh9dBek8i1SV+WrBpxiUzAstsLurbQ9EhDm5CL9Y6DJVd2Zd0ygqP/zLycTQq+0Ft8Gjc50l9ZEOP4jBz3wUlNvvbAHuoFJlw9Ck8w4v4FuUVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=upcw/uKfj88WDqSm8FUr1ULOeaphpyyhI9fexerZL3I=;
 b=Jnb1GdH8BZe/XROZgFX1rIhEOckYw9ZBE2lb78LaqOhqbntmDEkczD3NNpYsq3+pkelndFFIRMxPozIvnVjHJlWBXlPirAVj22bM/FykRoIgG+OhOhrkKeB3hxt5tLyoyNQ5c+Bps3SLvRimVvZCilyxaqoQ8YBzkF9Afpn/r4qhr7gVTVOGfCs+4sAlMNWyejjvNpFfWj/LQF5LH0aK1GDGP86/RuaSuBMbFu1Q8Ij8JNXiHvFYCPJ+mzInFoVum5DDGYTQa9yhYu6bsqbv+DJ8k8dpdZeR1CsvfBYkQlIv8ejzbymzD4ZSGX4efXot+7S9DUVgKNct/PgzOXSFfA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=upcw/uKfj88WDqSm8FUr1ULOeaphpyyhI9fexerZL3I=;
 b=Oh9lt3soa2YwtcTsLsewsGqJKk0lFlyIkWM8ICXdpJ0fgy+1k+9tyMApE+tjSQ0wcgnM3YZEQgLUIhKq96falghGBkX78IRrzJdCuVHN8gb39kBav375pcJ6o37r3vxdaaxakNr/0O8uj+mBP0V1WxbCWPDuTD1EWyJsm1yL1rQ=
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v3 06/16] perf/core: Rework guest callbacks to prepare for
 static_call support
To: Sean Christopherson <seanjc@google.com>,
        Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        Marc Zyngier <maz@kernel.org>, Guo Ren <guoren@kernel.org>,
        Nick Hu <nickhu@andestech.com>, Greentime Hu <green.hu@gmail.com>,
        Vincent Chen <deanbo422@gmail.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt
 <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>, Paolo Bonzini <pbonzini@redhat.com>,
        Juergen Gross <jgross@suse.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
        James Morse <james.morse@arm.com>,
        Alexandru Elisei <alexandru.elisei@arm.com>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Stefano Stabellini <sstabellini@kernel.org>,
        linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org,
        linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu,
        linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org,
        kvm@vger.kernel.org, xen-devel@lists.xenproject.org,
        Artem Kashkanov <artem.kashkanov@intel.com>,
        Like Xu <like.xu.linux@gmail.com>,
        Zhu Lingshan <lingshan.zhu@intel.com>
References: <20210922000533.713300-1-seanjc@google.com>
 <20210922000533.713300-7-seanjc@google.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <07628e38-e865-a3b1-49bc-b4c469558147@oracle.com>
Date: Wed, 22 Sep 2021 14:31:47 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210922000533.713300-7-seanjc@google.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: SN4PR0501CA0142.namprd05.prod.outlook.com
 (2603:10b6:803:2c::20) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a7fae781-088e-4dc2-a9b2-08d97df74183
X-MS-TrafficTypeDiagnostic: BL0PR10MB2833:
X-Microsoft-Antispam-PRVS: 
	<BL0PR10MB283351F4C1691BCF73AAE2158AA29@BL0PR10MB2833.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	E4xTpRBm0zHZDQHxUELHK4o4skpgKGZaTxeR48ByZ9KpG1jacN8sVlmcSc0lVOmiUkbIrCDKKKwze9uyFo5txERSYJi2bB9P2aox/EB0V995c+AdT73iuCIKI1mvdB7KmMMnCZyxfKIcjNGgR+pR+w5BTqECk6wR1/QHlBpFoodnk436wQD8YTOSZFTgC8fv3W/zUDBhA3lhNUqnYwvSOP9tomXBwjX2YXF/JpXa9yiW5LZJSaKkhHpMwPD+vZBAjyAhVZK16s2FsLS4d6C7BZMYMqfoi1AUjoCTA12TS+T9QFWFth+73vUWCSpFqjiLBnk5Ry2YrQZj1Z8IFao12eqT//zzSpxR2yNnt370GNVdx+Z9CNq0L39LeUWHIk2Bv+d3t2yDLgJkZMsnTKMJ30+s+j72JbGm1ZrI1b8Y3EEjbNQDoBEkTe06zl0YLwyCfZ8FrlQco7xhlfIVis/8pW3p4oQlxyyanlfnmAYD1HyeSOhHB858PdDkaBc8h4Z+WadBoHxCQKXlMa/1Gs4Ga1g6XYtJ3BwsV+UCzoqD4GEV/orTFtDoaZ+OA3eqdAehf4qjMYZLuyuuIKEf/9JvGhbzQyPlDcJk5rGoxWZfP5B0GEIA4vUKweEjtbmQtCVZpve0EYDFxtushvGrSCJx6udp0FA1uh5LnewOYiz22m6IoR/x/zCYTUgue7CuYIR4wdkrIIzvll++9GBj2PxnTzQ82rnKCNRktlrHc36Wzu2BXtHfDD3V/nXQWhep4Pvz
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(6486002)(54906003)(66946007)(2906002)(16576012)(921005)(53546011)(7406005)(7416002)(316002)(110136005)(186003)(44832011)(26005)(8936002)(508600001)(36756003)(5660300002)(4744005)(86362001)(2616005)(31686004)(31696002)(8676002)(6666004)(956004)(38100700002)(66476007)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ZHVoaEhTOS83MVpGV1o2TzkyRGZrejRoOEwxNVp0dmUzdFdiamRzbEd1UnRC?=
 =?utf-8?B?TlNmaDdlL1A4dWt1SW9wdFl0U0pZNldNM1FTVFYvMThGM1o3SEV6eEE0OUNT?=
 =?utf-8?B?UFZWOGdjOGxBdmE5TG1KbVBTT1lFamdReUFzSzJIUnhmWmFZNFZxS3MwMUVH?=
 =?utf-8?B?QjM2ek1mdlRpQU9HMm5oVVFLM1gxaEh2NCtGdTRpNDR6OUdUbVdzT3Z5OTZl?=
 =?utf-8?B?WUZJQVM5NHEzSklXUkNFNUJSY2NMRGdVQVE3TGFWRXpqdC8zQXYxZU5SSGcz?=
 =?utf-8?B?ZmJ3UG1CbGNCRmxqWEJzbFFWN0R2cjVXVjdCd1kyTWgxVTRHdFNyenE2Yjlq?=
 =?utf-8?B?UDk3UVg3clYwRjlZY3lzbmxuYU5jTVZkWE00c29IalQ2cWtlUllaN2ZBWlFu?=
 =?utf-8?B?VEY5ckd6L3hZVGYvdzdMSTQ0ZkdLVUY5K3hXdkZJTzh4WUcvdVdib0FBYnh0?=
 =?utf-8?B?WWFjdTRCdktzbVUwTnk3TTdVZTNrdmFWNGd2WkJwbnRPa0c1QTJnRndMeDhH?=
 =?utf-8?B?aW5Rc2djTWxOZGYycXY5akZLdGlxcUpqWnZ6VFhNcENzdjgvdW9CRjJsdEhL?=
 =?utf-8?B?MTdxbCtvSkp2K21hY2tERmpleVljSUhFeVJyZlhXeGgyNmVZVjgyOVFwTXJV?=
 =?utf-8?B?NFpHQ09GczRWdWR6WWpYVWxFSnA2MTZKRFRUd2NCTElVdGNOdzdOeTRPRnV3?=
 =?utf-8?B?WDZQUU1FTG80NWlWVkRkMjhYcGhBSldHTHd2YVdhTDYxdHFkbnhtQmk2dkdr?=
 =?utf-8?B?Qk5EbEo3Zmp6ZU9rS2RGZjQva3dZOGdDT2x4aTU2blBCMHJhc3gvNXZFTzlI?=
 =?utf-8?B?OE1Hd3FOMnlsdTFMT0lsQmduV2cvbzg1L2V4dFRnZ1FKUXVjSlUrL1lOL2ps?=
 =?utf-8?B?ajJKdDFBZWNzUWd0bENnQTNkYmJyeWNUWVQvRm92eGVlZFFKMGJDdDVrQlNG?=
 =?utf-8?B?SUo1UEk5aEd0RHJPVlVOLzRZVGNDc0RrZUNPY0x2bjI5OEtEeDRLRWsxQXVW?=
 =?utf-8?B?Wk1BNEFjMkN0ZUpFNEp6Qys5RTEwSVNWQVhIVlIzRnFNTEpEekRMT3F0dmVw?=
 =?utf-8?B?bEtUeXNxaEtUTmVVYzA1RlBBbTV3aEUzd1FvQlZVay9La0dZYlFFeE9SNGNN?=
 =?utf-8?B?Q0hyNTlveG1JZzZpM3BFeGE2MDZ4aC9LRWUzZTVOYkNGK1cxQjQzSE9HKzlz?=
 =?utf-8?B?WU1xalJ3dmRRSWQ4UkpTYnd1N0NnYk9CVXhMbWwwOWNtTEFRNFNuNERjMmJF?=
 =?utf-8?B?WTdLell2V1pyY2VGRHpFS0czUHZJdXdMR01oaWV5SnlYK3cyWG1mMWRDalJI?=
 =?utf-8?B?Y2s2ZVNJQUJSN1BZYy9sNkpiT1FVbjRLc0VVR2xPOE96M2RQMHErbmgwRzd4?=
 =?utf-8?B?RFNLeElMTFd3bHNLTnBLQnkwMXVFcSsrUWc2a1ZuQSthMnFrRGpIYmlLOEZY?=
 =?utf-8?B?MW9zM29KUTMyUnM5QzQ2TGtUZndEL3NNMjB5L0paTFFxRmdWRDdadkVJd3da?=
 =?utf-8?B?SXlEU2liZFZnQ0g3cVI1Qk96K1ZUYlZhL2c4UnBHRjFlZVBqSFFKTittL3lH?=
 =?utf-8?B?Y2ZVQXhkcE4zL0FiVWlTUithRjdZeG1VUDlBS0xFaXc5UkhJUFhQekhKYkcz?=
 =?utf-8?B?bFpMdVIrcG1mWjdpdjNTYk91bEtlSWQwcTJFbnlQTWZLaWVsZ1F5Y2RQSEhr?=
 =?utf-8?B?OVc4T1dSSzJJLys2bGdlK2krOEVja0Q2dVFwamJiSTFjRzRiNlliNUNDTnA0?=
 =?utf-8?Q?SuheLxEqN2oagDCS7nHKWhp2dSOYSg6Er0YNcsO?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a7fae781-088e-4dc2-a9b2-08d97df74183
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 18:31:56.0621
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vnq/xQQSkGPgKE6aYSccfGxRj6QAFMebqWcfPpt1ZIuQ9swJ577x3oljRPPcYQbGw5Q/BJuBwWGvCjKEolbUu2cl3rRLDq8nWyGjYUTkLU8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB2833
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10115 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0
 suspectscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220121
X-Proofpoint-GUID: 7ZtSLXiphePv2iFgKHCg8JOXjSqu1mWK
X-Proofpoint-ORIG-GUID: 7ZtSLXiphePv2iFgKHCg8JOXjSqu1mWK


On 9/21/21 8:05 PM, Sean Christopherson wrote:
> From: Like Xu <like.xu@linux.intel.com>
>
> To prepare for using static_calls to optimize perf's guest callbacks,
> replace ->is_in_guest and ->is_user_mode with a new multiplexed hook
> ->state, tweak ->handle_intel_pt_intr to play nice with being called when
> there is no active guest, and drop "guest" from ->is_in_guest.
>
> Return '0' from ->state and ->handle_intel_pt_intr to indicate "not in
> guest" so that DEFINE_STATIC_CALL_RET0 can be used to define the static
> calls, i.e. no callback == !guest.
>
> Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Originally-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Signed-off-by: Like Xu <like.xu@linux.intel.com>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
> [sean: extracted from static_call patch, fixed get_ip() bug, wrote changelog]
> Signed-off-by: Sean Christopherson <seanjc@google.com>


For Xen bits


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>





From xen-devel-bounces@lists.xenproject.org Wed Sep 22 18:41:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 18:41:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193028.343820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT7Bd-0000QG-Ht; Wed, 22 Sep 2021 18:41:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193028.343820; Wed, 22 Sep 2021 18:41:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT7Bd-0000Q9-E3; Wed, 22 Sep 2021 18:41:41 +0000
Received: by outflank-mailman (input) for mailman id 193028;
 Wed, 22 Sep 2021 18:41:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mT7Bb-0000Q3-7Y
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 18:41:39 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b77be324-1bd4-11ec-b9cc-12813bfff9fa;
 Wed, 22 Sep 2021 18:41:37 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 879E321ED3;
 Wed, 22 Sep 2021 16:16:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b77be324-1bd4-11ec-b9cc-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327361;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Lg6JGGOuMKgOK9bkuvAzYukD8OqEWtmOgbk2hQPItz8=;
	b=ND0iqKFzTzDcZ27ooxhDOEEEV8AccARG7AJHhC9a02u1e9KnwGeBRww88INrFwYdnUiGLU
	iWgot6Scekg2EdCjx5zulK0cCIpl/uZhww0kpehf4I9kg6qVmrIhlxI6IZCWAofixBVreB
	IFn8x3hm1X67NsswL2KN8XfRLNp4fgc=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 14/16] docs/system: add doc about the initialized machine phase and an example
Date: Wed, 22 Sep 2021 18:14:03 +0200
Message-Id: <20210922161405.140018-15-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---
 docs/system/managed-startup.rst | 77 +++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/docs/system/managed-startup.rst b/docs/system/managed-startup.rst
index 9bcf98ea79..af12a10d27 100644
--- a/docs/system/managed-startup.rst
+++ b/docs/system/managed-startup.rst
@@ -32,4 +32,81 @@ machine, including but not limited to:
 - ``query-qmp-schema``
 - ``query-commands``
 - ``query-status``
+- ``x-machine-init``
 - ``x-exit-preconfig``
+
+In particular these commands allow to advance and stop qemu at different
+phases of the VM creation and finally to leave the "preconfig" state. The
+accessible phases are:
+
+- ``accel-created``
+- ``initialized``
+- ``ready``
+
+The order of the phases is enforced. It is not possible to go backwards.
+Note that other early phases exist, but they are not attainable with
+``--preconfig``. Depending on the phase, QMP commands can be issued to modify
+some part of the VM creation.
+
+accel-created phase
+-------------------
+
+Initial phase entered with ``--preconfig``.
+
+initialized phase
+-----------------
+
+``x-machine-init`` advances to ``initialized`` phase. During this phase, the
+machine is initialized and populated with buses and devices. The following QMP
+commands are available to manually populate or modify the machine:
+
+- ``device_add``
+- ``x-sysbus-mmio-map``
+- ``qom-set``
+
+ready phase
+-----------
+
+``x-exit-preconfig`` advances to the final phase. When entering this phase,
+the VM creation finishes. "preconfig" state is then done and QEMU goes to
+normal execution.
+
+Machine creation example
+------------------------
+
+The following is an example that shows how to add some devices with qmp
+commands, memory map them, and add interrupts::
+
+  x-machine-init
+
+  device_add        driver=sysbus-memory id=rom size=0x4000 readonly=true
+  x-sysbus-mmio-map device=rom addr=32768
+
+  device_add        driver=sysbus-memory id=flash size=0x80000 readonly=true
+  x-sysbus-mmio-map device=flash addr=536870912
+
+  device_add        driver=sysbus-memory id=ram size=0x10000
+  x-sysbus-mmio-map device=ram addr=268435456
+
+  device_add        driver=ibex-plic id=plic
+  x-sysbus-mmio-map device=plic addr=1090584576
+
+  device_add        driver=ibex-uart id=uart chardev=serial0
+  x-sysbus-mmio-map device=uart addr=1073741824
+  qom-set path=uart property=sysbus-irq[0] value=plic/unnamed-gpio-in[1]
+  qom-set path=uart property=sysbus-irq[1] value=plic/unnamed-gpio-in[2]
+  qom-set path=uart property=sysbus-irq[2] value=plic/unnamed-gpio-in[3]
+  qom-set path=uart property=sysbus-irq[3] value=plic/unnamed-gpio-in[4]
+
+  x-exit-preconfig
+
+These commands reproduce a subset of the riscv32 opentitan (hw/riscv/opentitan)
+machine. We can start qemu using::
+
+  qemu-sytem-riscv32 -preconfig -qmp unix:./qmp-sock,server \
+  -machine none -cpu lowriscv-ibex -serial mon:stdio ...
+
+Then we just have to issue the commands, for example using `qmp-shell`. If the
+previous commands were in a file named `machine.qmp`, we could do::
+
+  qmp-shell ./qmp-sock < machine.qmp
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 20:47:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 20:47:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193034.343831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT99R-0005FK-No; Wed, 22 Sep 2021 20:47:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193034.343831; Wed, 22 Sep 2021 20:47:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT99R-0005FD-Ko; Wed, 22 Sep 2021 20:47:33 +0000
Received: by outflank-mailman (input) for mailman id 193034;
 Wed, 22 Sep 2021 20:47:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT99R-0005F3-19; Wed, 22 Sep 2021 20:47:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT99Q-0007zE-Qd; Wed, 22 Sep 2021 20:47:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mT99Q-0004tj-Iq; Wed, 22 Sep 2021 20:47:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mT99Q-0007Fd-IL; Wed, 22 Sep 2021 20:47:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=d7XrgVjbrrlBPL86Ij/OiMtD0FBKwjVYOQl0sFghkW8=; b=3O88E8WV1obpM85OVA2tR6Dv7T
	Rb4vd/LHw5FWhmQUs1DDWdL1yjwn3F97SUmt5aa2bVR9SE0bGKyVGZEDpwoDiSS4RmjTgQCkV7ysv
	K+LCRJtKdNkxBNRYWoriXo1i6HdINbYXwtLR1ZFVUuS5tGnIUrFDZyu1CW8TJicCeIqY=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165155-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165155: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=79019c7a42287e8591ec98487db0e849384a74e1
X-Osstest-Versions-That:
    ovmf=f334c5a41dc4702b1c767c6614025aa2f802f9a4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 20:47:32 +0000

flight 165155 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165155/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 79019c7a42287e8591ec98487db0e849384a74e1
baseline version:
 ovmf                 f334c5a41dc4702b1c767c6614025aa2f802f9a4

Last test of basis   165151  2021-09-22 09:30:38 Z    0 days
Testing same since   165155  2021-09-22 12:40:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Corvin Köhne <c.koehne@beckhoff.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   f334c5a41d..79019c7a42  79019c7a42287e8591ec98487db0e849384a74e1 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 20:50:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 20:50:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193041.343844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9C4-0006bA-5U; Wed, 22 Sep 2021 20:50:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193041.343844; Wed, 22 Sep 2021 20:50:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9C4-0006b3-2Z; Wed, 22 Sep 2021 20:50:16 +0000
Received: by outflank-mailman (input) for mailman id 193041;
 Wed, 22 Sep 2021 20:50:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT9C2-0006ax-LV
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 20:50:14 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ae6aed86-1be6-11ec-b9df-12813bfff9fa;
 Wed, 22 Sep 2021 20:50:13 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C5F4961100;
 Wed, 22 Sep 2021 20:50:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae6aed86-1be6-11ec-b9df-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632343812;
	bh=EJREcLRPljhrSz2RL+864h3zsMoyYWgufd4XoER14GA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XUW9CkuKVdFUx6aqH3yRQKu+C/D72/foewbO+zc6A/W9ns0oqj9v8CJcCBAHf1sFf
	 m7xm7Rqb19w8nnJuxNW2sx+I7Oxj7F78j1Fl86eb6b1a1TzwtFzp/C82Vcg67ZA6X7
	 XuAlkwn69hbnAJ0yjDciTBaeVr97GRsHFhsDdJ3BwUZGcu3PWWZKXdDa7eV1ADhqy7
	 uVAUe2hVepvMNKhC14u2rCwAyDBguHgH3B0clAS3OJEdpbm7ewOqQkYxHzwGvHYE68
	 TIWLmQj2gI8AkQkfcJiHzoZcYy5KFLdiOGbe5436LBEEwRVqd4sNjXY2lur9263uyE
	 kdH70PViX6hlA==
Date: Wed, 22 Sep 2021 13:50:10 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, 
    fnuv@xilinx.com
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <1ac58681-ef42-bb32-25b6-620d51d4f075@gmail.com>
Message-ID: <alpine.DEB.2.21.2109221349350.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s> <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org> <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com> <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s> <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
 <alpine.DEB.2.21.2109211439370.17979@sstabellini-ThinkPad-T480s> <1ac58681-ef42-bb32-25b6-620d51d4f075@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-136063595-1632343812=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-136063595-1632343812=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 Sep 2021, Oleksandr wrote:
> On 22.09.21 01:00, Stefano Stabellini wrote:
> > On Tue, 21 Sep 2021, Oleksandr wrote:
> > > On 21.09.21 02:21, Stefano Stabellini wrote:
> > > > On Sun, 19 Sep 2021, Oleksandr wrote:
> > > > > > On 18/09/2021 03:37, Stefano Stabellini wrote:
> > > > > > > On Fri, 17 Sep 2021, Stefano Stabellini wrote:
> > > > > > > > On Fri, 17 Sep 2021, Oleksandr wrote:
> > > > > > > > > > > +
> > > > > > > > > > > +    dt_dprintk("Find unallocated memory for extended
> > > > > > > > > > > regions\n");
> > > > > > > > > > > +
> > > > > > > > > > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > > > > > > > > > +    if ( !unalloc_mem )
> > > > > > > > > > > +        return -ENOMEM;
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Start with all available RAM */
> > > > > > > > > > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > > > > > > > > > +    {
> > > > > > > > > > > +        start = bootinfo.mem.bank[i].start;
> > > > > > > > > > > +        end = bootinfo.mem.bank[i].start +
> > > > > > > > > > > bootinfo.mem.bank[i].size - 1;
> > > > > > > > > > > +        res = rangeset_add_range(unalloc_mem, start,
> > > > > > > > > > > end);
> > > > > > > > > > > +        if ( res )
> > > > > > > > > > > +        {
> > > > > > > > > > > +            printk(XENLOG_ERR "Failed to add:
> > > > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > > > +                   start, end);
> > > > > > > > > > > +            goto out;
> > > > > > > > > > > +        }
> > > > > > > > > > > +    }
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Remove RAM assigned to Dom0 */
> > > > > > > > > > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > > > > > > > > > +    {
> > > > > > > > > > > +        start = assign_mem->bank[i].start;
> > > > > > > > > > > +        end = assign_mem->bank[i].start +
> > > > > > > > > > > assign_mem->bank[i].size - 1;
> > > > > > > > > > > +        res = rangeset_remove_range(unalloc_mem, start,
> > > > > > > > > > > end);
> > > > > > > > > > > +        if ( res )
> > > > > > > > > > > +        {
> > > > > > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > > > +                   start, end);
> > > > > > > > > > > +            goto out;
> > > > > > > > > > > +        }
> > > > > > > > > > > +    }
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Remove reserved-memory regions */
> > > > > > > > > > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++
> > > > > > > > > > > )
> > > > > > > > > > > +    {
> > > > > > > > > > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > > > > > > > > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > > > > > > > > > +            bootinfo.reserved_mem.bank[i].size - 1;
> > > > > > > > > > > +        res = rangeset_remove_range(unalloc_mem, start,
> > > > > > > > > > > end);
> > > > > > > > > > > +        if ( res )
> > > > > > > > > > > +        {
> > > > > > > > > > > +            printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > > > +                   start, end);
> > > > > > > > > > > +            goto out;
> > > > > > > > > > > +        }
> > > > > > > > > > > +    }
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Remove grant table region */
> > > > > > > > > > > +    start = kinfo->gnttab_start;
> > > > > > > > > > > +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
> > > > > > > > > > > +    res = rangeset_remove_range(unalloc_mem, start, end);
> > > > > > > > > > > +    if ( res )
> > > > > > > > > > > +    {
> > > > > > > > > > > +        printk(XENLOG_ERR "Failed to remove:
> > > > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > > > +               start, end);
> > > > > > > > > > > +        goto out;
> > > > > > > > > > > +    }
> > > > > > > > > > > +
> > > > > > > > > > > +    start = EXT_REGION_START;
> > > > > > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1,
> > > > > > > > > > > EXT_REGION_END);
> > > > > > > > > > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > > > > > > > > > +                                 add_ext_regions,
> > > > > > > > > > > ext_regions);
> > > > > > > > > > > +    if ( res )
> > > > > > > > > > > +        ext_regions->nr_banks = 0;
> > > > > > > > > > > +    else if ( !ext_regions->nr_banks )
> > > > > > > > > > > +        res = -ENOENT;
> > > > > > > > > > > +
> > > > > > > > > > > +out:
> > > > > > > > > > > +    rangeset_destroy(unalloc_mem);
> > > > > > > > > > > +
> > > > > > > > > > > +    return res;
> > > > > > > > > > > +}
> > > > > > > > > > > +
> > > > > > > > > > > +static int __init find_memory_holes(const struct
> > > > > > > > > > > kernel_info
> > > > > > > > > > > *kinfo,
> > > > > > > > > > > +                                    struct meminfo
> > > > > > > > > > > *ext_regions)
> > > > > > > > > > > +{
> > > > > > > > > > > +    struct dt_device_node *np;
> > > > > > > > > > > +    struct rangeset *mem_holes;
> > > > > > > > > > > +    paddr_t start, end;
> > > > > > > > > > > +    unsigned int i;
> > > > > > > > > > > +    int res;
> > > > > > > > > > > +
> > > > > > > > > > > +    dt_dprintk("Find memory holes for extended
> > > > > > > > > > > regions\n");
> > > > > > > > > > > +
> > > > > > > > > > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > > > > > > > > > +    if ( !mem_holes )
> > > > > > > > > > > +        return -ENOMEM;
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Start with maximum possible addressable physical
> > > > > > > > > > > memory
> > > > > > > > > > > range */
> > > > > > > > > > > +    start = EXT_REGION_START;
> > > > > > > > > > > +    end = min((1ULL << p2m_ipa_bits) - 1,
> > > > > > > > > > > EXT_REGION_END);
> > > > > > > > > > > +    res = rangeset_add_range(mem_holes, start, end);
> > > > > > > > > > > +    if ( res )
> > > > > > > > > > > +    {
> > > > > > > > > > > +        printk(XENLOG_ERR "Failed to add:
> > > > > > > > > > > %#"PRIx64"->%#"PRIx64"\n",
> > > > > > > > > > > +               start, end);
> > > > > > > > > > > +        goto out;
> > > > > > > > > > > +    }
> > > > > > > > > > > +
> > > > > > > > > > > +    /* Remove all regions described by "reg" property
> > > > > > > > > > > (MMIO,
> > > > > > > > > > > RAM,
> > > > > > > > > > > etc) */
> > > > > > > > > > Well... The loop below is not going to handle all the
> > > > > > > > > > regions
> > > > > > > > > > described in
> > > > > > > > > > the property "reg". Instead, it will cover a subset of "reg"
> > > > > > > > > > where
> > > > > > > > > > the
> > > > > > > > > > memory is addressable.
> > > > > > > > > As I understand, we are only interested in subset of "reg"
> > > > > > > > > where
> > > > > > > > > the
> > > > > > > > > memory is
> > > > > > > > > addressable.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > > You will also need to cover "ranges" that will describe the
> > > > > > > > > > BARs
> > > > > > > > > > for
> > > > > > > > > > the PCI
> > > > > > > > > > devices.
> > > > > > > > > Good point.
> > > > > > > > Yes, very good point!
> > > > > > > > 
> > > > > > > > 
> > > > > > > > > Could you please clarify how to recognize whether it is a PCI
> > > > > > > > > device as long as PCI support is not merged? Or just to find
> > > > > > > > > any
> > > > > > > > > device nodes
> > > > > > > > > with non-empty "ranges" property
> > > > > > > > > and retrieve addresses?
> > > > > > > > Normally any bus can have a ranges property with the aperture
> > > > > > > > and
> > > > > > > > possible address translations, including /amba (compatible =
> > > > > > > > "simple-bus"). However, in these cases dt_device_get_address
> > > > > > > > already
> > > > > > > > takes care of it, see
> > > > > > > > xen/common/device_tree.c:dt_device_get_address.
> > > > > > > > 
> > > > > > > > The PCI bus is special for 2 reasons:
> > > > > > > > - the ranges property has a different format
> > > > > > > > - the bus is hot-pluggable
> > > > > > > > 
> > > > > > > > So I think the only one that we need to treat specially is PCI.
> > > > > > > > 
> > > > > > > > As far as I am aware PCI is the only bus (or maybe just the only
> > > > > > > > bus
> > > > > > > > that we support?) where ranges means the aperture.
> > > > > > > Now that I think about this, there is another "hotpluggable"
> > > > > > > scenario
> > > > > > > we
> > > > > > > need to think about:
> > > > > > > 
> > > > > > > [1] https://marc.info/?l=xen-devel&m=163056546214978
> > > > > > > 
> > > > > > > Xilinx devices have FPGA regions with apertures currently not
> > > > > > > described
> > > > > > > in device tree, where things can programmed in PL at runtime
> > > > > > > making
> > > > > > > new
> > > > > > > devices appear with new MMIO regions out of thin air.
> > > > > > > 
> > > > > > > Now let me start by saying that yes, the entire programmable
> > > > > > > region
> > > > > > > aperture could probably be described in device tree, however, in
> > > > > > > reality it is not currently done in any of the device trees we use
> > > > > > > (including the upstream device trees in linux.git).
> > > > > > This is rather annoying, but not unheard. There are a couple of
> > > > > > platforms
> > > > > > where the MMIOs are not fully described in the DT.
> > > > > > 
> > > > > > In fact, we have a callback 'specific_mappings' which create
> > > > > > additional
> > > > > > mappings (e.g. on the omap5) for dom0.
> > > > > > 
> > > > > > > So, we have a problem :-(
> > > > > > > 
> > > > > > > 
> > > > > > > I can work toward getting the right info on device tree, but in
> > > > > > > reality
> > > > > > > that is going to take time and for now the device tree doesn't
> > > > > > > have
> > > > > > > the
> > > > > > > FPGA aperture in it. So if we accept this series as is, it is
> > > > > > > going to
> > > > > > > stop features like [1] from working. >
> > > > > > > If we cannot come up with any better plans, I think it would be
> > > > > > > better
> > > > > > > to drop find_memory_holes, only rely on find_unallocated_memory
> > > > > > > even
> > > > > > > when the IOMMU is on. One idea is that we could add on top of the
> > > > > > > regions found by find_unallocated_memory any MMIO regions marked
> > > > > > > as
> > > > > > > xen,passthrough: they are safe because they are not going to dom0
> > > > > > > anyway.
> > > > > > (Oleksandr, it looks like some rationale about the different
> > > > > > approach is
> > > > > > missing in the commit message. Can you add it?)
> > > > > Yes sure, but let me please clarify what is different approach in this
> > > > > context. Is it to *also* take into the account MMIO regions of the
> > > > > devices
> > > > > for
> > > > > passthrough for case when IOMMU is off (in addition to unallocated
> > > > > memory)? If
> > > > > yes, I wonder whether we will gain much with that according to that
> > > > > device's
> > > > > MMIO regions are usually not big enough and we stick to allocate
> > > > > extended
> > > > > regions with bigger size (> 64MB).
> > > > That's fair enough. There are a couple of counter examples where the
> > > > MMIO regions for the device to assign are quite large, for instance a
> > > > GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
> > > > but maybe they are not that common. I am not sure if it is worth
> > > > scanning the tree for xen,passthrough regions every time at boot for
> > > > this.
> > > ok, I will add a few sentences to commit message about this different
> > > approach
> > > for now. At least this could be implemented later on if there is a need.
> > One thing that worries me about this is that if we take an old Xen with
> > this series and run it on a new board, it might cause problems. At the
> > very least [1] wouldn't work.
> 
> I got it.
> 
> 
> > 
> > Can we have a Xen command line argument to disable extended regions as
> > an emergecy toggle?
> 
> I think, yes. If no preference for the argument name I will name it
> "no-ext-region".

It is better to introduce it as ext-regions=yes/no with yes as default.
So that in the future we could extending it to ext-regions=start,size if
we wanted to.
--8323329-136063595-1632343812=:17979--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:01:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:01:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193047.343856 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9N8-0008Hf-4L; Wed, 22 Sep 2021 21:01:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193047.343856; Wed, 22 Sep 2021 21:01:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9N8-0008HY-1C; Wed, 22 Sep 2021 21:01:42 +0000
Received: by outflank-mailman (input) for mailman id 193047;
 Wed, 22 Sep 2021 21:01:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mT9N6-0008HS-B9
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:01:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 470d8a7a-1be8-11ec-b9df-12813bfff9fa;
 Wed, 22 Sep 2021 21:01:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 8859E21EC6;
 Wed, 22 Sep 2021 16:15:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 470d8a7a-1be8-11ec-b9df-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327353;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AlcBA0wPhQv7JVjfEB3jyBVlLhydcLHLk7mWNQnkSaw=;
	b=aNbmKAB3upukJdcXcsuRvB4jG/XeAjvN01JCf7FZfNOuG3Ym0/T3VOHAsNgbv3CyONvpfS
	mid9Yf6nHZMIKJQ8tXxHLaDcTTsVZesJe5TCDt+J3rfnHcViEzutj1BAJythr3C13zB3oI
	zK53p1V+hpk5h2HvFJykoc2d/cYBPPw=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 07/16] hw/core/machine: add machine_class_is_dynamic_sysbus_dev_allowed
Date: Wed, 22 Sep 2021 18:13:56 +0200
Message-Id: <20210922161405.140018-8-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Right now the allowance check for adding a sysbus device using
-device cli option (or device_add qmp command) is done well after
the device has been created. It is done during the machine init done
notifier: machine_init_notify() in hw/core/machine.c

This new function will allow us to check if a sysbus device type is
allowed to be dynamically created by the machine during the device
creation time.

Also make device_is_dynamic_sysbus() use the new function.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

In the context of our series, we need to be able to do the check at
device creation time to allow doing it depending on the current
MACHINE_INIT phase.
---
 include/hw/boards.h | 17 +++++++++++++++++
 hw/core/machine.c   | 15 ++++++++++++---
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 463a5514f9..934443c1cd 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -51,6 +51,23 @@ void machine_set_cpu_numa_node(MachineState *machine,
  */
 void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type);
 
+/**
+ * machine_class_is_dynamic_sysbus_dev_allowed: Check if type is an allowed
+ * sysbus device type for the machine class.
+ * @mc: Machine class
+ * @type: type to check (should be a subtype of TYPE_SYS_BUS_DEVICE)
+ *
+ * Returns: true if @type is a type in the machine's list of
+ * dynamically pluggable sysbus devices; otherwise false.
+ *
+ * Check if the QOM type @type is in the list of allowed sysbus device
+ * types (see machine_class_allowed_dynamic_sysbus_dev()).
+ * Note that if @type is a subtype of a type which is in the list, it is
+ * allowed too.
+ */
+bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
+                                                 const char *type);
+
 /**
  * device_is_dynamic_sysbus: test whether device is a dynamic sysbus device
  * @mc: Machine class
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 9125c9aad0..1a18912dc8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -545,18 +545,27 @@ void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type)
 
 bool device_is_dynamic_sysbus(MachineClass *mc, DeviceState *dev)
 {
-    bool allowed = false;
-    strList *wl;
     Object *obj = OBJECT(dev);
 
     if (!object_dynamic_cast(obj, TYPE_SYS_BUS_DEVICE)) {
         return false;
     }
 
+    return machine_class_is_dynamic_sysbus_dev_allowed(mc,
+            object_get_typename(obj));
+}
+
+bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
+                                                 const char *type)
+{
+    bool allowed = false;
+    strList *wl;
+    ObjectClass *klass = object_class_by_name(type);
+
     for (wl = mc->allowed_dynamic_sysbus_devices;
          !allowed && wl;
          wl = wl->next) {
-        allowed |= !!object_dynamic_cast(obj, wl->value);
+        allowed |= !!object_class_dynamic_cast(klass, wl->value);
     }
 
     return allowed;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:01:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:01:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193048.343866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9ND-00008w-G1; Wed, 22 Sep 2021 21:01:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193048.343866; Wed, 22 Sep 2021 21:01:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9ND-00008j-Cf; Wed, 22 Sep 2021 21:01:47 +0000
Received: by outflank-mailman (input) for mailman id 193048;
 Wed, 22 Sep 2021 21:01:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mT9NB-0008HS-87
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:01:45 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 470d8a7b-1be8-11ec-b9df-12813bfff9fa;
 Wed, 22 Sep 2021 21:01:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id F3DEA21CC1;
 Wed, 22 Sep 2021 16:15:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 470d8a7b-1be8-11ec-b9df-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327345;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VdnVuRz8orSzm80d4V6oo8FtHFI6VLNiM7CokKV1kg0=;
	b=FUm2HUwzttyCZe+8dhBbh50LnWpTBgfyRBKEmQweI52IibjcVgDn4DK1o5YtxElR7pK3VG
	FzivmPgZdVq7CIwOWz8C/0JHcJLoWkMZTuJkoIZq/q7EtIWQYR432EE+RLBVUQvkum1SsI
	7ZnOmjO/M8NkPARS4NSxeS/ky6/tWQE=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 01/16] rename MachineInitPhase enum constants for QAPI compatibility
Date: Wed, 22 Sep 2021 18:13:50 +0200
Message-Id: <20210922161405.140018-2-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

From: Mirela Grujic <mirela.grujic@greensocs.com>

This commit is a preparation to switch to a QAPI definition
of the MachineInitPhase enum.

QAPI will generate enumeration constants prefixed with the
MACHINE_INIT_PHASE_, so rename values accordingly.

Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
---
 include/hw/qdev-core.h     | 10 +++++-----
 hw/core/machine-qmp-cmds.c |  2 +-
 hw/core/machine.c          |  6 +++---
 hw/core/qdev.c             |  2 +-
 hw/pci/pci.c               |  2 +-
 hw/usb/core.c              |  2 +-
 hw/virtio/virtio-iommu.c   |  2 +-
 monitor/hmp.c              |  2 +-
 softmmu/qdev-monitor.c     |  9 +++++----
 softmmu/vl.c               |  6 +++---
 ui/console.c               |  3 ++-
 11 files changed, 24 insertions(+), 22 deletions(-)

diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 34c8a7506a..859fd913bb 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -841,30 +841,30 @@ bool qdev_should_hide_device(QemuOpts *opts);
 
 typedef enum MachineInitPhase {
     /* current_machine is NULL.  */
-    PHASE_NO_MACHINE,
+    MACHINE_INIT_PHASE_NO_MACHINE,
 
     /* current_machine is not NULL, but current_machine->accel is NULL.  */
-    PHASE_MACHINE_CREATED,
+    MACHINE_INIT_PHASE_MACHINE_CREATED,
 
     /*
      * current_machine->accel is not NULL, but the machine properties have
      * not been validated and machine_class->init has not yet been called.
      */
-    PHASE_ACCEL_CREATED,
+    MACHINE_INIT_PHASE_ACCEL_CREATED,
 
     /*
      * machine_class->init has been called, thus creating any embedded
      * devices and validating machine properties.  Devices created at
      * this time are considered to be cold-plugged.
      */
-    PHASE_MACHINE_INITIALIZED,
+    MACHINE_INIT_PHASE_INITIALIZED,
 
     /*
      * QEMU is ready to start CPUs and devices created at this time
      * are considered to be hot-plugged.  The monitor is not restricted
      * to "preconfig" commands.
      */
-    PHASE_MACHINE_READY,
+    MACHINE_INIT_PHASE_READY,
 } MachineInitPhase;
 
 extern bool phase_check(MachineInitPhase phase);
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 216fdfaf3a..52168a3771 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -147,7 +147,7 @@ HotpluggableCPUList *qmp_query_hotpluggable_cpus(Error **errp)
 
 void qmp_set_numa_node(NumaOptions *cmd, Error **errp)
 {
-    if (phase_check(PHASE_MACHINE_INITIALIZED)) {
+    if (phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
         error_setg(errp, "The command is permitted only before the machine has been created");
         return;
     }
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 067f42b528..9125c9aad0 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1274,7 +1274,7 @@ void machine_run_board_init(MachineState *machine)
 
     accel_init_interfaces(ACCEL_GET_CLASS(machine->accelerator));
     machine_class->init(machine);
-    phase_advance(PHASE_MACHINE_INITIALIZED);
+    phase_advance(MACHINE_INIT_PHASE_INITIALIZED);
 }
 
 static NotifierList machine_init_done_notifiers =
@@ -1283,7 +1283,7 @@ static NotifierList machine_init_done_notifiers =
 void qemu_add_machine_init_done_notifier(Notifier *notify)
 {
     notifier_list_add(&machine_init_done_notifiers, notify);
-    if (phase_check(PHASE_MACHINE_READY)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
         notify->notify(notify, NULL);
     }
 }
@@ -1306,7 +1306,7 @@ void qdev_machine_creation_done(void)
      * ok, initial machine setup is done, starting from now we can
      * only create hotpluggable devices
      */
-    phase_advance(PHASE_MACHINE_READY);
+    phase_advance(MACHINE_INIT_PHASE_READY);
     qdev_assert_realized_properly();
 
     /* TODO: once all bus devices are qdevified, this should be done
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index cefc5eaa0a..c5fc704f55 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -904,7 +904,7 @@ static void device_initfn(Object *obj)
 {
     DeviceState *dev = DEVICE(obj);
 
-    if (phase_check(PHASE_MACHINE_READY)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
         dev->hotplugged = 1;
         qdev_hot_added = true;
     }
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 23d2ae2ab2..5ed798b480 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -1102,7 +1102,7 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     address_space_init(&pci_dev->bus_master_as,
                        &pci_dev->bus_master_container_region, pci_dev->name);
 
-    if (phase_check(PHASE_MACHINE_READY)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
         pci_init_bus_master(pci_dev);
     }
     pci_dev->irq_state = 0;
diff --git a/hw/usb/core.c b/hw/usb/core.c
index 975f76250a..7a9a81c4fe 100644
--- a/hw/usb/core.c
+++ b/hw/usb/core.c
@@ -97,7 +97,7 @@ void usb_wakeup(USBEndpoint *ep, unsigned int stream)
     USBDevice *dev = ep->dev;
     USBBus *bus = usb_bus_from_device(dev);
 
-    if (!phase_check(PHASE_MACHINE_READY)) {
+    if (!phase_check(MACHINE_INIT_PHASE_READY)) {
         /*
          * This is machine init cold plug.  No need to wakeup anyone,
          * all devices will be reset anyway.  And trying to wakeup can
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 1b23e8e18c..b777a80be2 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -948,7 +948,7 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,
      * accept it. Having a different masks is possible but the guest will use
      * sub-optimal block sizes, so warn about it.
      */
-    if (phase_check(PHASE_MACHINE_READY)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
         int new_granule = ctz64(new_mask);
         int cur_granule = ctz64(cur_mask);
 
diff --git a/monitor/hmp.c b/monitor/hmp.c
index d50c3124e1..ad012b0b22 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -216,7 +216,7 @@ static bool cmd_can_preconfig(const HMPCommand *cmd)
 
 static bool cmd_available(const HMPCommand *cmd)
 {
-    return phase_check(PHASE_MACHINE_READY) || cmd_can_preconfig(cmd);
+    return phase_check(MACHINE_INIT_PHASE_READY) || cmd_can_preconfig(cmd);
 }
 
 static void help_cmd_dump_one(Monitor *mon,
diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 0705f00846..25275984bd 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -262,7 +262,7 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
 
     dc = DEVICE_CLASS(oc);
     if (!dc->user_creatable ||
-        (phase_check(PHASE_MACHINE_READY) && !dc->hotpluggable)) {
+        (phase_check(MACHINE_INIT_PHASE_READY) && !dc->hotpluggable)) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "driver",
                    "a pluggable device type");
         return NULL;
@@ -649,7 +649,8 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
         }
     }
 
-    if (phase_check(PHASE_MACHINE_READY) && bus && !qbus_is_hotpluggable(bus)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY) && bus &&
+        !qbus_is_hotpluggable(bus)) {
         error_setg(errp, QERR_BUS_NO_HOTPLUG, bus->name);
         return NULL;
     }
@@ -663,7 +664,7 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
     dev = qdev_new(driver);
 
     /* Check whether the hotplug is allowed by the machine */
-    if (phase_check(PHASE_MACHINE_READY)) {
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
         if (!qdev_hotplug_allowed(dev, errp)) {
             goto err_del_dev;
         }
@@ -1011,7 +1012,7 @@ int qemu_global_option(const char *str)
 
 bool qmp_command_available(const QmpCommand *cmd, Error **errp)
 {
-    if (!phase_check(PHASE_MACHINE_READY) &&
+    if (!phase_check(MACHINE_INIT_PHASE_READY) &&
         !(cmd->options & QCO_ALLOW_PRECONFIG)) {
         error_setg(errp, "The command '%s' is permitted only after machine initialization has completed",
                    cmd->name);
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 55ab70eb97..d2552ba8ac 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2692,7 +2692,7 @@ static void qemu_machine_creation_done(void)
 
 void qmp_x_exit_preconfig(Error **errp)
 {
-    if (phase_check(PHASE_MACHINE_INITIALIZED)) {
+    if (phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
         error_setg(errp, "The command is permitted only before machine initialization");
         return;
     }
@@ -3665,14 +3665,14 @@ void qemu_init(int argc, char **argv, char **envp)
     qemu_apply_legacy_machine_options(machine_opts_dict);
     qemu_apply_machine_options(machine_opts_dict);
     qobject_unref(machine_opts_dict);
-    phase_advance(PHASE_MACHINE_CREATED);
+    phase_advance(MACHINE_INIT_PHASE_MACHINE_CREATED);
 
     /*
      * Note: uses machine properties such as kernel-irqchip, must run
      * after qemu_apply_machine_options.
      */
     configure_accelerators(argv[0]);
-    phase_advance(PHASE_ACCEL_CREATED);
+    phase_advance(MACHINE_INIT_PHASE_ACCEL_CREATED);
 
     /*
      * Beware, QOM objects created before this point miss global and
diff --git a/ui/console.c b/ui/console.c
index eabbbc951c..b09b0f9760 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1353,7 +1353,8 @@ static QemuConsole *new_console(DisplayState *ds, console_type_t console_type,
     if (QTAILQ_EMPTY(&consoles)) {
         s->index = 0;
         QTAILQ_INSERT_TAIL(&consoles, s, next);
-    } else if (console_type != GRAPHIC_CONSOLE || phase_check(PHASE_MACHINE_READY)) {
+    } else if (console_type != GRAPHIC_CONSOLE ||
+               phase_check(MACHINE_INIT_PHASE_READY)) {
         QemuConsole *last = QTAILQ_LAST(&consoles);
         s->index = last->index + 1;
         QTAILQ_INSERT_TAIL(&consoles, s, next);
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:05:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:05:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193061.343878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9QU-0001BD-Ut; Wed, 22 Sep 2021 21:05:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193061.343878; Wed, 22 Sep 2021 21:05:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9QU-0001B6-Rj; Wed, 22 Sep 2021 21:05:10 +0000
Received: by outflank-mailman (input) for mailman id 193061;
 Wed, 22 Sep 2021 21:05:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT9QU-0001Az-E4
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:05:10 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7622092f-875d-4caa-ba0d-1549d45aee0a;
 Wed, 22 Sep 2021 21:05:09 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 652A36112F;
 Wed, 22 Sep 2021 21:05:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7622092f-875d-4caa-ba0d-1549d45aee0a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632344708;
	bh=CLlsYr/xMME+lA6Uj07qC22IdfSkZ/DckOstv95Y8GM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=R5ccNQ/UMqrSzun10aanrfvZeMsRNKRNVuR/RXiilnHZs/geRf5x+ywmtGfmRoy8X
	 CPpqR8UQuDkHboLC5FoBXSzJdJGg2tXC+b4Kg4j8UbneG2FogRfq5zckfzwpCNTsuA
	 DdY/sci08TA1dV0HSv4vs2JVSu0x4uqy12IRBRIc7EG9J/U/kyABE3CwiTPfv6eZ1R
	 Ar4GKx07aQrY+LcrvhVbF52Vdb6iPPqDDct9cc6pikSoxGG2U4Amw/vubBq7DommjL
	 tlGcs3KZMRGwK+I/jgLQK3H8JCT2Ct6hV5Z5MdRHkdFsfrE26IrD+VvVQu4BkPXpgG
	 YiE/96eCiOaWQ==
Date: Wed, 22 Sep 2021 14:05:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <a3f5707b-b161-1c94-ebe1-d6b8bbe11081@gmail.com>
Message-ID: <alpine.DEB.2.21.2109221402420.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <a993466f-1b7d-ceb3-aa3b-16f5f145b2fb@gmail.com> <a3f5707b-b161-1c94-ebe1-d6b8bbe11081@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1843995332-1632344708=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1843995332-1632344708=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 Sep 2021, Oleksandr wrote:
> > > > > You will also need to cover "ranges" that will describe the BARs for
> > > > > the PCI
> > > > > devices.
> > > > Good point.
> > > Yes, very good point!
> > > 
> > > 
> > > > Could you please clarify how to recognize whether it is a PCI
> > > > device as long as PCI support is not merged? Or just to find any device
> > > > nodes
> > > > with non-empty "ranges" property
> > > > and retrieve addresses?
> > > Normally any bus can have a ranges property with the aperture and
> > > possible address translations, including /amba (compatible =
> > > "simple-bus"). However, in these cases dt_device_get_address already
> > > takes care of it, see xen/common/device_tree.c:dt_device_get_address.
> > > 
> > > The PCI bus is special for 2 reasons:
> > > - the ranges property has a different format
> > > - the bus is hot-pluggable
> > > 
> > > So I think the only one that we need to treat specially is PCI.
> > > 
> > > As far as I am aware PCI is the only bus (or maybe just the only bus
> > > that we support?) where ranges means the aperture.
> > Thank you for the clarification. I need to find device node with non-empty
> > ranges property
> > (and make sure that device_type property is "pci"), after that I need to
> > read the context of ranges property and translate it.
> > 
> > 
> 
> OK, I experimented with that and managed to parse ranges property for PCI host
> bridge node.
> 
> I tested on my setup where the host device tree contains two PCI host bridge
> nodes with the following:
> 
> pcie@fe000000 {
> ...
>             ranges = <0x1000000 0x0 0x0 0x0 0xfe100000 0x0 0x100000 0x2000000
> 0x0 0xfe200000 0x0 0xfe200000 0x0 0x200000 0x2000000 0x0 0x30000000 0x0
> 0x30000000 0x0 0x8000000 0x42000000 0x0 0x38000000 0x0 0x38000000 0x0
> 0x8000000>;
> ...
> };
> 
> pcie@ee800000 {
> ...
>             ranges = <0x1000000 0x0 0x0 0x0 0xee900000 0x0 0x100000 0x2000000
> 0x0 0xeea00000 0x0 0xeea00000 0x0 0x200000 0x2000000 0x0 0xc0000000 0x0
> 0xc0000000 0x0 0x8000000 0x42000000 0x0 0xc8000000 0x0 0xc8000000 0x0
> 0x8000000>;
> ...
> };
> 
> So Xen retrieves the *CPU addresses* from the ranges:
> 
> (XEN) dev /soc/pcie@fe000000 range_size 7 nr_ranges 4
> (XEN) 0: addr=fe100000, size=100000
> (XEN) 1: addr=fe200000, size=200000
> (XEN) 2: addr=30000000, size=8000000
> (XEN) 3: addr=38000000, size=8000000
> (XEN) dev /soc/pcie@ee800000 range_size 7 nr_ranges 4
> (XEN) 0: addr=ee900000, size=100000
> (XEN) 1: addr=eea00000, size=200000
> (XEN) 2: addr=c0000000, size=8000000
> (XEN) 3: addr=c8000000, size=8000000
> 
> The code below covers ranges property in the context of finding memory holes
> (to be squashed with current patch):
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d37156a..7d20c10 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -834,6 +834,8 @@ static int __init find_memory_holes(struct meminfo
> *ext_regions)
>      {
>          unsigned int naddr;
>          u64 addr, size;
> +        const __be32 *ranges;
> +        u32 len;
> 
>          naddr = dt_number_of_address(np);
> 
> @@ -857,6 +859,41 @@ static int __init find_memory_holes(struct meminfo
> *ext_regions)
>                  goto out;
>              }
>          }
> +
> +        /*
> +         * Also looking for non-empty ranges property which would likely mean
> +         * that we deal with PCI host bridge device and the property here
> +         * describes the BARs for the PCI devices.
> +         */

One thing to be careful is that ranges with a valid parameter is not
only present in PCI busses. It can be present in amba and other
simple-busses too. In that case the format for ranges in simpler as it
doesn't have a "memory type" like PCI.

When you get addresses from reg, bus ranges properties are automatically
handled for you.

All of this to say that a check on "ranges" is not enough because it
might capture other non-PCI busses that have a different, simpler,
ranges format. You want to check for "ranges" under a device_type =
"pci"; node.


> +        ranges = dt_get_property(np, "ranges", &len);
> +        if ( ranges && len )
> +        {
> +            unsigned int range_size, nr_ranges;
> +            int na, ns, pna;
> +
> +            pna = dt_n_addr_cells(np);
> +            na = dt_child_n_addr_cells(np);
> +            ns = dt_child_n_size_cells(np);
> +            range_size = pna + na + ns;
> +            nr_ranges = len / sizeof(__be32) / range_size;
> +
> +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
> +            {
> +                /* Skip the child address and get the parent (CPU) address */
> +                addr = dt_read_number(ranges + na, pna);
> +                size = dt_read_number(ranges + na + pna, ns);
> +
> +                start = addr & PAGE_MASK;
> +                end = PAGE_ALIGN(addr + size);
> +                res = rangeset_remove_range(mem_holes, start, end - 1);
> +                if ( res )
> +                {
> +                    printk(XENLOG_ERR "Failed to remove:
> %#"PRIx64"->%#"PRIx64"\n",
> +                           start, end);
> +                    goto out;
> +                }
> +            }
> +        }
>      }

--8323329-1843995332-1632344708=:17979--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:07:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:07:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193068.343889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9Sp-0001zN-CB; Wed, 22 Sep 2021 21:07:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193068.343889; Wed, 22 Sep 2021 21:07:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9Sp-0001zG-8i; Wed, 22 Sep 2021 21:07:35 +0000
Received: by outflank-mailman (input) for mailman id 193068;
 Wed, 22 Sep 2021 21:07:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT9Sn-0001z8-Lu
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:07:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5fa4c250-6a28-44b9-b56b-06bba0762063;
 Wed, 22 Sep 2021 21:07:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6176E61181;
 Wed, 22 Sep 2021 21:07:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5fa4c250-6a28-44b9-b56b-06bba0762063
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632344851;
	bh=eXOu4o+FdOvYQrid4uHWIpYoP9Oa7kdMSkJBRAzCLHU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KrrP4ZbKbpX55zielSusOiQ2amnZFHotkrQlzxiYJMmLa1R2TmGk121wGGiBUnlk7
	 4wMVKG8srp/Rw+7+xtQ9bAHvL7usqNQt1j49medB0nsViPfSZzYGBxXuhEcKiUKtO+
	 8iPYVBS+JW9uUfQVZLv2zTykLGywumGao06c+mddGISdmWYYg7nC4cSuaA0WTXsM1Z
	 EPGTaV4jmau27UyLd36q1Hui5qgg1kKI1AyOdbTp7QeGF9VLg5f1rKkSCTsE8Uia1R
	 PcjBUqtIrFOYr4hQNG2BuR1maJQcv4jQtyuJhvnMoVOaFkxX9P7Ot5CDto8rsBU994
	 PQcer9ja4+4Dg==
Date: Wed, 22 Sep 2021 14:07:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org, 
    sstabellini@kernel.org, jbeulich@suse.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: Re: [PATCH 1/2] xen-pciback: allow compiling on other archs than
 x86
In-Reply-To: <20210922101422.2319240-1-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109221406560.17979@sstabellini-ThinkPad-T480s>
References: <20210922101422.2319240-1-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Xen-pciback driver was designed to be built for x86 only. But it
> can also be used by other architectures, e.g. Arm.
> Re-structure the driver in a way that it can be built for other
> platforms as well.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

This patch looks good to me but I would prefer if it came after the
other patch, not before.


> ---
> Since v1:
>  - Do not move pci_xen_initial_domain as it is x86 specific
> ---
>  arch/x86/include/asm/xen/pci.h     | 18 +------
>  arch/x86/pci/xen.c                 | 74 +----------------------------
>  drivers/xen/Kconfig                |  2 +-
>  drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
>  drivers/xen/xen-pciback/pci_stub.c |  3 +-
>  drivers/xen/xen-pciback/xenbus.c   |  2 +-
>  include/xen/pci.h                  | 28 +++++++++++
>  7 files changed, 109 insertions(+), 93 deletions(-)
>  create mode 100644 include/xen/pci.h
> 
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
> index 3506d8c598c1..2889f091f459 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -16,26 +16,10 @@ static inline int pci_xen_hvm_init(void)
>  #endif
>  #if defined(CONFIG_XEN_DOM0)
>  int __init pci_xen_initial_domain(void);
> -int xen_find_device_domain_owner(struct pci_dev *dev);
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> -int xen_unregister_device_domain_owner(struct pci_dev *dev);
>  #else
>  static inline int __init pci_xen_initial_domain(void)
>  {
> -	return -1;
> -}
> -static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> -						   uint16_t domain)
> -{
> -	return -1;
> -}
> -static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> +       return -1;
>  }
>  #endif
>  
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 3d41a09c2c14..4a45b0bf9ae4 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -23,6 +23,7 @@
>  
>  #include <xen/features.h>
>  #include <xen/events.h>
> +#include <xen/pci.h>
>  #include <asm/xen/pci.h>
>  #include <asm/xen/cpuid.h>
>  #include <asm/apic.h>
> @@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
>  	}
>  	return 0;
>  }
> -
> -struct xen_device_domain_owner {
> -	domid_t domain;
> -	struct pci_dev *dev;
> -	struct list_head list;
> -};
> -
> -static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> -static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> -
> -static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	list_for_each_entry(owner, &dev_domain_list, list) {
> -		if (owner->dev == dev)
> -			return owner;
> -	}
> -	return NULL;
> -}
> -
> -int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -	int domain = -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (owner)
> -		domain = owner->domain;
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return domain;
> -}
> -EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> -
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> -	if (!owner)
> -		return -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	if (find_device(dev)) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		kfree(owner);
> -		return -EEXIST;
> -	}
> -	owner->domain = domain;
> -	owner->dev = dev;
> -	list_add_tail(&owner->list, &dev_domain_list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> -
> -int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (!owner) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		return -ENODEV;
> -	}
> -	list_del(&owner->list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	kfree(owner);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
>  #endif
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index a37eb52fb401..057ddf61ef61 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -182,7 +182,7 @@ config SWIOTLB_XEN
>  
>  config XEN_PCIDEV_BACKEND
>  	tristate "Xen PCI-device backend driver"
> -	depends on PCI && X86 && XEN
> +	depends on PCI && XEN
>  	depends on XEN_BACKEND
>  	default m
>  	help
> diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
> index 224df03ce42e..fc8c1249d49f 100644
> --- a/drivers/xen/pci.c
> +++ b/drivers/xen/pci.c
> @@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
>  	return 0;
>  }
>  #endif
> +
> +#ifdef CONFIG_XEN_DOM0
> +struct xen_device_domain_owner {
> +	domid_t domain;
> +	struct pci_dev *dev;
> +	struct list_head list;
> +};
> +
> +static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> +static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> +
> +static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	list_for_each_entry(owner, &dev_domain_list, list) {
> +		if (owner->dev == dev)
> +			return owner;
> +	}
> +	return NULL;
> +}
> +
> +int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +	int domain = -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (owner)
> +		domain = owner->domain;
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return domain;
> +}
> +EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> +
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> +	if (!owner)
> +		return -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	if (find_device(dev)) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		kfree(owner);
> +		return -EEXIST;
> +	}
> +	owner->domain = domain;
> +	owner->dev = dev;
> +	list_add_tail(&owner->list, &dev_domain_list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> +
> +int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (!owner) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		return -ENODEV;
> +	}
> +	list_del(&owner->list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	kfree(owner);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
> +#endif
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index f8e4faa96ad6..bba527620507 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -19,7 +19,8 @@
>  #include <linux/sched.h>
>  #include <linux/atomic.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
> +#include <xen/xen.h>
>  #include <asm/xen/hypervisor.h>
>  #include <xen/interface/physdev.h>
>  #include "pciback.h"
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index c09c7ebd6968..da34ce85dc88 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -14,7 +14,7 @@
>  #include <linux/workqueue.h>
>  #include <xen/xenbus.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
>  #include "pciback.h"
>  
>  #define INVALID_EVTCHN_IRQ  (-1)
> diff --git a/include/xen/pci.h b/include/xen/pci.h
> new file mode 100644
> index 000000000000..b8337cf85fd1
> --- /dev/null
> +++ b/include/xen/pci.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __XEN_PCI_H__
> +#define __XEN_PCI_H__
> +
> +#if defined(CONFIG_XEN_DOM0)
> +int xen_find_device_domain_owner(struct pci_dev *dev);
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> +int xen_unregister_device_domain_owner(struct pci_dev *dev);
> +#else
> +static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> +						   uint16_t domain)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +#endif
> +
> +#endif
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:10:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:10:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193076.343900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9Vh-0003Pu-VK; Wed, 22 Sep 2021 21:10:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193076.343900; Wed, 22 Sep 2021 21:10:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9Vh-0003Pn-SK; Wed, 22 Sep 2021 21:10:33 +0000
Received: by outflank-mailman (input) for mailman id 193076;
 Wed, 22 Sep 2021 21:10:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT9Vf-0003Ph-WF
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:10:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4c2ea2dc-cd4e-4967-9087-3b4f1cb357cf;
 Wed, 22 Sep 2021 21:10:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D0D62611C6;
 Wed, 22 Sep 2021 21:10:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c2ea2dc-cd4e-4967-9087-3b4f1cb357cf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632345030;
	bh=d2egW6Jb1yflKGHaZY0ipvwf3m4wUUJPVblhrUTq/nU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=FZiXdRggtJpG40FywZhrpqe0wKKqCk740dWaG+e0N+ISF+s6Uh77IHsOjyNMFgTsB
	 i8wP+ItH3SX0UWV4f7RUwNd2PQnjm7sGGE44nwZD407De8e8h2a/tNH0FrP6fWJxtL
	 lMMMHWvxEYvL2ArDssMDxzMKtwHqPdlQc74+Za28CSiXbzcxdg6dU36TF+20PDkNcg
	 NtN6BSk5Cugp20eHcTypIFowdyHknuP8tIaVgSetV7FHY67vgCHZfiMOcXDczVN1r+
	 qoF+fjkYrMdnt7PU+s7M7F519CbYV1l9QAwaSp/cpAFSnz/X1PyIphwP8DI1m01qtu
	 L02QAKfzAcT9w==
Date: Wed, 22 Sep 2021 14:10:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org, 
    sstabellini@kernel.org, jbeulich@suse.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
In-Reply-To: <20210922101422.2319240-2-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
References: <20210922101422.2319240-1-andr2000@gmail.com> <20210922101422.2319240-2-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Currently PCI backend implements multiple functionalities at a time.
> To name a few:
> 1. it is used as a database for assignable PCI devices, e.g. xl
>    pci-assignable-{add|remove|list} manipulates that list. So, whenever
>    the toolstack needs to know which PCI devices can be passed through
>    it reads that from the relevant sysfs entries of the pciback.
> 2. it is used to hold the unbound PCI devices list, e.g. when passing
>    through a PCI device it needs to be unbound from the relevant device
>    driver and bound to pciback (strictly speaking it is not required
>    that the device is bound to pciback, but pciback is again used as a
>    database of the passed through PCI devices, so we can re-bind the
>    devices back to their original drivers when guest domain shuts down)
> 3. device reset for the devices being passed through
> 4. para-virtualized use-cases support
> 
> The para-virtualized part of the driver is not always needed as some
> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
> model for PCI device passthrough. For such use-cases make the very
> first step in splitting the xen-pciback driver into two parts: extended
> PCI stub and PCI PV backend drivers. At the moment x86 platform will
> continue using CONFIG_XEN_PCIDEV_BACKEND for the fully featured backend
> driver and new platforms may build a driver with limited functionality
> (no PV) by enabling CONFIG_XEN_PCIDEV_STUB.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Please make this the first patch of the series so that...

> ---
> New in v2
> ---
>  drivers/xen/Kconfig               | 26 +++++++++++++++++++++++++-
>  drivers/xen/Makefile              |  2 +-
>  drivers/xen/xen-pciback/Makefile  |  1 +
>  drivers/xen/xen-pciback/pciback.h |  5 +++++
>  drivers/xen/xen-pciback/xenbus.c  |  6 +++++-
>  5 files changed, 37 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index 057ddf61ef61..6e92c6be19f1 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -180,10 +180,34 @@ config SWIOTLB_XEN
>  	select DMA_OPS
>  	select SWIOTLB
>  
> +config XEN_PCI_STUB
> +	bool
> +
> +config XEN_PCIDEV_STUB
> +	tristate "Xen PCI-device stub driver"
> +	depends on PCI && !X86 && XEN
> +	depends on XEN_BACKEND
> +	select XEN_PCI_STUB
> +	default m
> +	help
> +	  The PCI device stub driver provides limited version of the PCI
> +	  device backend driver without para-virtualized support for guests.
> +	  If you select this to be a module, you will need to make sure no
> +	  other driver has bound to the device(s) you want to make visible to
> +	  other guests.
> +
> +	  The "hide" parameter (only applicable if backend driver is compiled
> +	  into the kernel) allows you to bind the PCI devices to this module
> +	  from the default device drivers. The argument is the list of PCI BDFs:
> +	  xen-pciback.hide=(03:00.0)(04:00.0)
> +
> +	  If in doubt, say m.
> +
>  config XEN_PCIDEV_BACKEND
>  	tristate "Xen PCI-device backend driver"
> -	depends on PCI && XEN
> +	depends on PCI && X86 && XEN
>  	depends on XEN_BACKEND

...we don't need this


> +	select XEN_PCI_STUB
>  	default m
>  	help
>  	  The PCI device backend driver allows the kernel to export arbitrary
> diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
> index 3434593455b2..5aae66e638a7 100644
> --- a/drivers/xen/Makefile
> +++ b/drivers/xen/Makefile
> @@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
>  obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
>  obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
>  obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
> -obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
> +obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
>  obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
>  obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
>  obj-$(CONFIG_XEN_EFI)			+= efi.o
> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
> index e8d981d43235..e2cb376444a6 100644
> --- a/drivers/xen/xen-pciback/Makefile
> +++ b/drivers/xen/xen-pciback/Makefile
> @@ -1,5 +1,6 @@
>  # SPDX-License-Identifier: GPL-2.0
>  obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
>  
>  xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
>  xen-pciback-y += conf_space.o conf_space_header.o \
> diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
> index 95e28ee48d52..9a64196e831d 100644
> --- a/drivers/xen/xen-pciback/pciback.h
> +++ b/drivers/xen/xen-pciback/pciback.h
> @@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
>  				    struct pci_dev *dev);
>  void pcistub_put_pci_dev(struct pci_dev *dev);
>  
> +static inline bool xen_pcibk_pv_support(void)
> +{
> +	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
> +}
> +
>  /* Ensure a device is turned off or reset */
>  void xen_pcibk_reset_device(struct pci_dev *pdev);
>  
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index da34ce85dc88..bde63ef677b8 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
>  
>  int __init xen_pcibk_xenbus_register(void)
>  {
> +	if (!xen_pcibk_pv_support())
> +		return 0;

Is this truly enough to stop the PV backend from initializing? Have you
actually tested it to make sure? If it works, amazing! I am quite happy
about this approach :-)




>  	xen_pcibk_backend = &xen_pcibk_vpci_backend;
>  	if (passthrough)
>  		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
> @@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
>  
>  void __exit xen_pcibk_xenbus_unregister(void)
>  {
> -	xenbus_unregister_driver(&xen_pcibk_driver);
> +	if (xen_pcibk_pv_support())
> +		xenbus_unregister_driver(&xen_pcibk_driver);
>  }
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:20:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:20:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193082.343911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9eq-0004K0-T3; Wed, 22 Sep 2021 21:20:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193082.343911; Wed, 22 Sep 2021 21:20:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mT9eq-0004Jt-Pn; Wed, 22 Sep 2021 21:20:00 +0000
Received: by outflank-mailman (input) for mailman id 193082;
 Wed, 22 Sep 2021 21:19:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mT9ep-0004Jn-7L
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:19:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d672e0f0-1bea-11ec-b9e3-12813bfff9fa;
 Wed, 22 Sep 2021 21:19:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E08F761107;
 Wed, 22 Sep 2021 21:19:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d672e0f0-1bea-11ec-b9e3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632345597;
	bh=M1UDbhXsb2YNGVdOS6vL4VAm7p9HjWIoQIvjCnDrze4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RUlL8pekKkkNBRmrXS+MtyiV2eRwQ/ZDR8IhLbbipbBiOHsNCFuJuxQ4HfcJeqmDh
	 fg/wTRCmHgeisG1rfqHGks/esMXo5JbyCNkTUV9OA19x4z7ogNTOCVy8XO08NwYbSs
	 /ppLSEUoXXMnMvBVvm8c2otxASTFCRGhDXccfKzoyW//5pJVnnwmX6l2Ca3H9UQTGr
	 FgFeN83/JCWlEFfrFQ1cv/LaRO6lWeSscc7ba8fuGTC8FWTV+insu0ygTpn5yPu3cL
	 lC/UUyllYEBSysO0lbAg06f7xT6Sw1tRP3PR0+Gyz1R/YusWlKRRi1jnQkUvaXvcQc
	 jRZ9/sMcLne4g==
Date: Wed, 22 Sep 2021 14:19:55 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 1/2] arm/efi: Introduce uefi,cfg-load DT property
In-Reply-To: <20210922141341.42288-2-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109221411200.17979@sstabellini-ThinkPad-T480s>
References: <20210922141341.42288-1-luca.fancellu@arm.com> <20210922141341.42288-2-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Luca Fancellu wrote:
> Introduce the uefi,cfg-load DT property of /chosen
> node for ARM whose presence decide whether to force
> the load of the UEFI Xen configuration file.
> 
> The logic is that if any multiboot,module is found in
> the DT, then the uefi,cfg-load property is used to see
> if the UEFI Xen configuration file is needed.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

The patch looks OK, just a couple of minor comments below.


> ---
> v2 changes:
> - Introduced uefi,cfg-load property
> - Add documentation about the property
> ---
>  docs/misc/efi.pandoc        |  2 ++
>  xen/arch/arm/efi/efi-boot.h | 28 +++++++++++++++++++++++-----
>  2 files changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ac3cd58cae..e289c5e7ba 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
>  bootloader provides a device tree containing modules then any configuration
>  files are ignored, and the bootloader is responsible for populating all
>  relevant device tree nodes.
> +The property "uefi,cfg-load" can be specified in the /chosen node to force Xen
> +to load the configuration file even if multiboot modules are found.

I think that, in addition to this, we also need to add the property to
docs/misc/arm/device-tree/booting.txt where our "official" device tree
bindings are maintained. I would add it below "Command lines" and before
"Creating Multiple Domains directly from Xen" maybe as a new chapter.
It could be called "Other Options" but other ideas could be valid too.

You can say that uefi,cfg-load is a boolean.


>  Once built, `make install-xen` will place the resulting binary directly into
>  the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..8ceeba4ad1 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
>  
>  static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
>  {
> +    bool skip_cfg_file = false;
>      /*
>       * For arm, we may get a device tree from GRUB (or other bootloader)
>       * that contains modules that have already been loaded into memory.  In
> -     * this case, we do not use a configuration file, and rely on the
> -     * bootloader to have loaded all required modules and appropriate
> -     * options.
> +     * this case, we search for the property uefi,cfg-load in the /chosen node
> +     * to decide whether to skip the UEFI Xen configuration file or not.
>       */
>  
>      fdt = lookup_fdt_config_table(SystemTable);
>      dtbfile.ptr = fdt;
>      dtbfile.need_to_free = false; /* Config table memory can't be freed. */
> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
> +
> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
> +    {
> +        /* Locate chosen node */
> +        int node = fdt_subnode_offset(fdt, 0, "chosen");
> +        const void *cfg_load_prop;
> +        int cfg_load_len;
> +
> +        if ( node > 0 )
> +        {
> +            /* Check if uefi,cfg-load property exists */
> +            cfg_load_prop = fdt_getprop(fdt, node, "uefi,cfg-load",
> +                                        &cfg_load_len);
> +            if ( !cfg_load_prop )
> +                skip_cfg_file = true;
> +        }
> +    }
> +
> +    if ( !fdt || !skip_cfg_file )

Just a suggestion, but rather than the double negative, wouldn't it be
simpler to define it as

    bool load_cfg_file = true;

?


>      {
>          /*
>           * We either have no FDT, or one without modules, so we must have a
> -         * Xen EFI configuration file to specify modules.  (dom0 required)
> +         * Xen EFI configuration file to specify modules.
>           */

Also mention in the commit message that you are taking the opportunity
to update this comment do remove "dom0 required".


>          return true;
>      }
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:44:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:44:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193088.343922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA29-0007kW-Q6; Wed, 22 Sep 2021 21:44:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193088.343922; Wed, 22 Sep 2021 21:44:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA29-0007kP-N7; Wed, 22 Sep 2021 21:44:05 +0000
Received: by outflank-mailman (input) for mailman id 193088;
 Wed, 22 Sep 2021 21:44:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mTA28-0007kJ-6N
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:44:04 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7e59f317-743d-49fd-8dd2-f01cc7533c02;
 Wed, 22 Sep 2021 21:44:01 +0000 (UTC)
Received: from pps.filterd (m0246617.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MKdYOt006658; 
 Wed, 22 Sep 2021 21:43:49 GMT
Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4rfck3-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 21:43:49 +0000
Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1])
 by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18MLepd8093386;
 Wed, 22 Sep 2021 21:43:48 GMT
Received: from nam12-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam12lp2169.outbound.protection.outlook.com [104.47.59.169])
 by aserp3020.oracle.com with ESMTP id 3b7q5bac5v-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 21:43:48 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4224.namprd10.prod.outlook.com (2603:10b6:208:1d1::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep
 2021 21:43:46 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 21:43:46 +0000
Received: from [10.74.85.221] (138.3.201.29) by
 SA9PR13CA0005.namprd13.prod.outlook.com (2603:10b6:806:21::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Wed, 22 Sep 2021 21:43:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e59f317-743d-49fd-8dd2-f01cc7533c02
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=/IBKupBcFxnA99WDInoeyscYGjc3/7VEYIAS9LPOIHg=;
 b=upFy+j1m4Nd88pfLpgkIUePVLNkdDKzs+33fijchFCZ60T+amR2x1Ndr2j4KUH3MpiEU
 OjMywOIdc3aGd5fjOJWca96X17O3xW4TZxG9Ax4AkyFMua7G4uqW3bG7zerFmVtVvpaN
 6ZepEAikfcOeSaP85USJWjWMUfMSrKlXS56/AyTupUgudm52FcAw9YeiS2nCJsTZEa+r
 sv+Jfn/eMMemUfYDgrvCsWk4B3gyoRlJAZDmMZSPWilfXKg7iIASABvAaYP6wV8e2Gei
 4xkU3JIcpUdo+O0PmYp3uMc2zG2ycQteJ+6tXfdphEASkF+IMoYmvGkTIktcYThKvk3q Tg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mjvgHweuJNeyV6Aeky2/DWzbcg7Vf2BApM+ip7v6Xi9DI4UNAl/WgYeGo1GJK+n0ADOQ9vmV7VmtZtOJINE5EENyD6Vyv2ZCN6d433BaI8R7fzecacbjxCayONfhM3CbKznEcLmewBKCIsg2Z6hRM6ZAoMe+SizQltFHrI43fC112o5sqj1kJQCwuaySjfaBj0l17pVjkoSMcjelzbnMXe4oLsDSgHtlEs6fTlp/q7uFnAYpmfSOpmA+vdya9dRXHC1BBI8IqmzNT3LP8wNWvMXe9kPLxXg11iBzQEwbQBMyO0XNYk0LV2WocLPhfEtFuwpP2dblLDpDrqgwbBbIkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/IBKupBcFxnA99WDInoeyscYGjc3/7VEYIAS9LPOIHg=;
 b=cZUqQ8aXsQ8oSzCrPsydmaP6aYfLN4Q4dRNLD4X9UmEv2cEjNZbeLm6iC6bFxcYF3WmLH7Hh41KnJNgFfI1crD6/Teu/OsVkzvsxD3Vf5MhwzqT3SbwmweRVRVcWPK+YhYUnEGrHM1k/q5xniB//LsQ9LLAGEdcenVLA/YI4a+tDR1jdkPWxZ9+9UoyWQ99HvhrEkM1kf5miN1NzSYFfjytGFbmdW4MvSrUnHcAPqGZ9JH6q1Yt0SNim5RBTqdM+ZzJM3MYgBqYSHUR6C02zy2U8hOfq7c23gHzDcFetHK03tS0NyNO4U7yRG8chRTfKXlc2ppxoryVjaS4N/Vh96A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/IBKupBcFxnA99WDInoeyscYGjc3/7VEYIAS9LPOIHg=;
 b=YuvWCdel4iHkJHFRJssOndvAyasCaZ+WM1a5A/qL1Xrddf1FLoHKLETIi+LXdcvs44i1yMmyrURjE/7dPfMxFDxBAE3jfWiDLO9nb8aUaR8LxeeQRX241IeGUW39ggKEwOGDLY4pXGoUXO/NZoWXlJLPeiCisb1XQs5kCL2GoGg=
Authentication-Results: zytor.com; dkim=none (message not signed)
 header.d=none;zytor.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v2 1/2] x86/xen: remove xen_have_vcpu_info_placement flag
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        x86@kernel.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Stefano Stabellini <sstabellini@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-2-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>
Date: Wed, 22 Sep 2021 17:43:41 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210922103102.3589-2-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-ClientProxiedBy: SA9PR13CA0005.namprd13.prod.outlook.com
 (2603:10b6:806:21::10) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: df98b6c0-31b9-4dcf-6d7f-08d97e120e57
X-MS-TrafficTypeDiagnostic: MN2PR10MB4224:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB4224494F85C9FDAC2630097B8AA29@MN2PR10MB4224.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1091;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	F4wsWfDh60UMGeswttrET2QoJwY8m6uKD+w9J3r7t75r0y2eqV7HdLV0hieuwNar8OmiSywqoLbLNA6M5XRxt+TifBDAXwuVkJ+gZT1hR+8D5Tr5a1sOpKRYGYIfADQoYDWkx/4Lly5uggIlyQwZBfjZ/8FwYZO3uJvpgg/JJl6sA16LSlwr4AzdMunthRt6kkY8JM1nxLoKsXpQ/tFgBSUeywyCj6O6AMGpGlTIwYI/YzZNcPNhpoCeVD3j7e5STxieKj4+/nTJio1cNkCM9kbhweCU0HmADqtcbQwZZTmuuZIr1iksfLL0wFA/rzyP3mQdkzrUHF8Ofz2mK/BdhEzQOf4YcejLws8A4VfX+B/2EdgyTi8TLf5gHb6sfqqXiyNLSxO87pJgQpQcauIuNcL70t/fEuVTTlZyKUYum1T8u+i2tzIGadxRPK9S0zyOBFyuP46qqBiHAN7wrCCOWF+i+XWzGatrsEbAjRalQXXqk65240Q6bBUN4gwL9EPEQoy8tm+WLJZGXRFgz4sDux92nGrJwrzKztmoBYf8bmioeqWBui1skQOopothaqGO9xpbny9R/32a9rDGaWxqa6bYQeFTZ808p/LmUHzDriSDw7ucomNjvoJg+Ea8MejXAyskTAwWXU6hzYq131xVr3UKBt/YtCaH/4RunVGlLCfwNwfvoDz2xZ4b9EL/dU11HGQQo8lRXAgVQfcbz1Znj+VmpKBV3v1NAD6jWxFpOdEG24lCIsl6GKYiUMh20wJ7
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7416002)(16576012)(38100700002)(86362001)(956004)(54906003)(36756003)(44832011)(66946007)(2906002)(26005)(53546011)(508600001)(83380400001)(31686004)(8676002)(5660300002)(66476007)(4326008)(66556008)(8936002)(316002)(2616005)(31696002)(186003)(6486002)(6666004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?UHh4RlRmYi8vKzFiTzd5bWJtSWhLSmIvclVDb3FEL0lYOVRhWmh4Q1MyU1Vz?=
 =?utf-8?B?SmdEM3BwMVlzcTdKK28zd0dJcUZKVkZ6SWxnTHlrM2plOUI3Qzk1a0h3Q0pQ?=
 =?utf-8?B?NExRdXVMQzRsd2E4TjJqNFMvSUNudnJzb3ZsWTR0VEtYTlh1YS9JRUYya1p0?=
 =?utf-8?B?UXRBZGozV0NGREhJOElHOE5JRkhWSTlYZ2c1RUZ0N1FkS1hmS3Z2TS9kZmlO?=
 =?utf-8?B?TStHYWMwUmUzV1Z5SkJYenFTVGE4YmFqdGR6RW1OZTRFMVVLNFQrK08xWTlK?=
 =?utf-8?B?KzlyUS9NYStZNFlPVUM5eE1tUFJqbVBXemJkclZVWWViV0dRVVd4d29qQm1m?=
 =?utf-8?B?K3YyYVZ4WUlGN3ZFd0p0WXlreWFwdmkvU1FSam45eCt1ZFBOMTBpUGdlZVZR?=
 =?utf-8?B?aXBwWk5mL3U0enN0YTNSMTRPRUxKaTNKQ2EzZ2srMks0bmZxbFlNUXNMMzU2?=
 =?utf-8?B?bGRRR1hRck5SUmdIR1lMSE43bFJIS2hhS0UvUE5ZaWFHMnRHVVVzQUpJU1Za?=
 =?utf-8?B?YnpZMnFEaERQdnpwbFdQNDlVWTBvc3pDaTR5eDFBL2wxL0xWdFRJWlpGZW1m?=
 =?utf-8?B?RzY3b0hjWE9JRnEwRlUxVFZxSm94Nk1WV1lrdFdHcitsRFo2dk1FRi9UOTFK?=
 =?utf-8?B?T25RbXNoK0lGSmN2dGM0V2hFdUdqWVZHazVaNWFlMGh1bHpqckhjRXJ4dmtD?=
 =?utf-8?B?ZFIzVWtjeFdaRlQ0M2pFMlJiQ0F4ZTJST2hOZUNGOW1keE1WNGQ5TTRLL1RT?=
 =?utf-8?B?REVabUNGZnAwWlM3M21tL0Y1VTNLY2dYYnJPeGZlV09jS2dPZGptUm9EenM4?=
 =?utf-8?B?NEVNbVZUL0s3Wm0wTW94Z1dBeWVwK3RiL2xKWktrbXBRWHIvdHNUbEZoM05h?=
 =?utf-8?B?RFNFaHBqL3E4M2swT3p5OEpjY3FCbE1TT09YdDJxQjhTMVVDb1kySnlCQTQz?=
 =?utf-8?B?cnhHUkFGKzRLQVZyRHYydUJSRUF4QVhZMXFRcUxmN3YxU1huK2o3V0JPVmJk?=
 =?utf-8?B?Wm5aRE5vSEY3M2J1ZU0wNTYrcTdjMWlqS29XR0RTdnhnN1UySXE1bkROeUIw?=
 =?utf-8?B?MGdMT3RLZExWKzBsV2d4VXlickVVWlBYY3Z6MTVyU3B6bFc2bEU2Tno2cnF4?=
 =?utf-8?B?YUxSRVQ2TTErMzZwemROQ2FSeVkzS3dldEk0cDNsdTVvME9Pa3pyaGIwckRH?=
 =?utf-8?B?emFtLytRWkRtandyYllnY21oSEhYMkh4bEVaSTlPOXRmTUkxb1k0bnlZSVE3?=
 =?utf-8?B?OE9nTFBEWTE0Q3pUTXNhTHQwRGFLVGJUSHE4Um5EL091UGtXWEc4a1VrTmRL?=
 =?utf-8?B?Ti9BUXRqZzFJT3NkTHVCODJwV0xLeHRXS1FKVmhvdU5la2FsaEkxcUp0QnFk?=
 =?utf-8?B?RnBnQ1lCTEZ6WDljT2pZWlBOQmYwemNQV0o3ZVNCV2gwbmJCK3RPWGM1M2Nj?=
 =?utf-8?B?Y29WTUYrd1hpUFVTNmdjYWY5TWdKMEVqcnBPb3pLeGxLU3JMd3ZYNE9Eclhx?=
 =?utf-8?B?VEc4YmdqTDBWSk02UkZHRUJSaFZHUGlBZnFTK3BTTzNEMHYvd1Zwck5qVUpn?=
 =?utf-8?B?VDdaUFN0OVg1bE5DbXh2OHNLQUlUbjcyNjF2Nld6Z1ZqVXVZY0hheDAreVlr?=
 =?utf-8?B?eTdNMXNPSVdqL3R0UnArWjM0QnRHYlBDaURSU1FwbmoxQjRlL2dsc0VuSWpO?=
 =?utf-8?B?TzRGSS9TOGFtWmI4cGdGU2tpZTczZHR1czgrS2dXUTI2R1V6VlMxWEJEa0tj?=
 =?utf-8?Q?yugJHjSbJSTJyWmgzXP1/b64oegQ8EocT+71czl?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: df98b6c0-31b9-4dcf-6d7f-08d97e120e57
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 21:43:46.5733
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: brHhu/MwxG+aOTlpMBiNKbCJcW2RIpfjXKuaSKYsKZDVVY8VPpTWgLGOjhPL8Dm/jwdCqz0BwFQNMwo+LGjhXfbOp/x7EwVAI+Z5MgLhi5w=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4224
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10115 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0
 suspectscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220138
X-Proofpoint-GUID: PHsEnjZ49UU51NquE_VFO0qhRzeGKNqm
X-Proofpoint-ORIG-GUID: PHsEnjZ49UU51NquE_VFO0qhRzeGKNqm


On 9/22/21 6:31 AM, Juergen Gross wrote:
>  
> -	if (xen_have_vcpu_info_placement) {
> -		vcpup = &per_cpu(xen_vcpu_info, cpu);
> -		info.mfn = arbitrary_virt_to_mfn(vcpup);
> -		info.offset = offset_in_page(vcpup);
> +	vcpup = &per_cpu(xen_vcpu_info, cpu);
> +	info.mfn = arbitrary_virt_to_mfn(vcpup);
> +	info.offset = offset_in_page(vcpup);
>  
> -		/*
> -		 * Check to see if the hypervisor will put the vcpu_info
> -		 * structure where we want it, which allows direct access via
> -		 * a percpu-variable.
> -		 * N.B. This hypercall can _only_ be called once per CPU.
> -		 * Subsequent calls will error out with -EINVAL. This is due to
> -		 * the fact that hypervisor has no unregister variant and this
> -		 * hypercall does not allow to over-write info.mfn and
> -		 * info.offset.
> -		 */
> -		err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info,
> -					 xen_vcpu_nr(cpu), &info);
> -
> -		if (err) {
> -			pr_warn_once("register_vcpu_info failed: cpu=%d err=%d\n",
> -				     cpu, err);
> -			xen_have_vcpu_info_placement = 0;
> -		} else {
> -			/*
> -			 * This cpu is using the registered vcpu info, even if
> -			 * later ones fail to.
> -			 */
> -			per_cpu(xen_vcpu, cpu) = vcpup;
> -		}
> -	}
> -
> -	if (!xen_have_vcpu_info_placement)
> -		xen_vcpu_info_reset(cpu);
> +	/*
> +	 * N.B. This hypercall can _only_ be called once per CPU.
> +	 * Subsequent calls will error out with -EINVAL. This is due to
> +	 * the fact that hypervisor has no unregister variant and this
> +	 * hypercall does not allow to over-write info.mfn and
> +	 * info.offset.
> +	 */
> +	err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, xen_vcpu_nr(cpu),
> +				 &info);
> +	if (err)
> +		panic("register_vcpu_info failed: cpu=%d err=%d\n", cpu, err);
>  


This is change in behavior. Before if the hypercall failed we still try to boot. I am not sure we need to worry about this (since it's not clear it actually works)  but I'd at least mention this in the commit message.


-boris




From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:45:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:45:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193094.343932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA3z-0008LX-4m; Wed, 22 Sep 2021 21:45:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193094.343932; Wed, 22 Sep 2021 21:45:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA3z-0008LQ-23; Wed, 22 Sep 2021 21:45:59 +0000
Received: by outflank-mailman (input) for mailman id 193094;
 Wed, 22 Sep 2021 21:45:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTA3y-0008LG-4d; Wed, 22 Sep 2021 21:45:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTA3x-0000hn-V3; Wed, 22 Sep 2021 21:45:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTA3x-0000F7-MP; Wed, 22 Sep 2021 21:45:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTA3x-00053B-Lt; Wed, 22 Sep 2021 21:45:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xVRF4n7m6P08TtxVCbpOZjKZf0QLeWZmZ3WcUgz8BXI=; b=DCT9Vku7AVRamclviX834MhQL/
	A0rpUNe5MdYrSAdK+p97bK0OXKRYu8tq7xBiE81G9gn8McyXiiPWN/VqVa8zm/U/oUyaiIT9Yg6dT
	1iGBvv5PVhHMr5R/6JwSzFkjDfhBfFi+7GvyBR9nbMPk+XsZLpvObTnoMQ+OL4IbQqCs=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165157-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165157: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=604be1b333b1b66052ab9b0133f156890549a4f0
X-Osstest-Versions-That:
    xen=4e467d390eaf6736377766adae0166711866fac3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 22 Sep 2021 21:45:57 +0000

flight 165157 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165157/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  604be1b333b1b66052ab9b0133f156890549a4f0
baseline version:
 xen                  4e467d390eaf6736377766adae0166711866fac3

Last test of basis   165143  2021-09-21 19:01:35 Z    1 days
Testing same since   165157  2021-09-22 15:01:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4e467d390e..604be1b333  604be1b333b1b66052ab9b0133f156890549a4f0 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:50:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:50:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193103.343947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA7x-0001RA-Uq; Wed, 22 Sep 2021 21:50:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193103.343947; Wed, 22 Sep 2021 21:50:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA7x-0001QU-Po; Wed, 22 Sep 2021 21:50:05 +0000
Received: by outflank-mailman (input) for mailman id 193103;
 Wed, 22 Sep 2021 21:50:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Psju=OM=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mTA7w-0001EK-MT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:50:04 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0ad733ba-1bef-11ec-b9eb-12813bfff9fa;
 Wed, 22 Sep 2021 21:50:03 +0000 (UTC)
Received: from pps.filterd (m0246632.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18MKdIcV006465; 
 Wed, 22 Sep 2021 21:49:50 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b7q4p6yeh-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 21:49:47 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18MLjbPv137684;
 Wed, 22 Sep 2021 21:49:36 GMT
Received: from nam02-dm3-obe.outbound.protection.outlook.com
 (mail-dm3nam07lp2045.outbound.protection.outlook.com [104.47.56.45])
 by userp3030.oracle.com with ESMTP id 3b7q5nky3w-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 22 Sep 2021 21:49:36 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB4866.namprd10.prod.outlook.com (2603:10b6:208:325::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 22 Sep
 2021 21:49:34 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.013; Wed, 22 Sep 2021
 21:49:34 +0000
Received: from [10.74.85.221] (138.3.201.29) by
 MN2PR07CA0018.namprd07.prod.outlook.com (2603:10b6:208:1a0::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Wed, 22 Sep 2021 21:49:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ad733ba-1bef-11ec-b9eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=rHcAdxv7gbw+A9E1/mIZdVxlid4Ee7MD5bGLZ8HFD04=;
 b=fpUHSf/roVyvHt+Hec9MQ/ViiNRlXTNzHP3KHSC1lhRlao1Reelk/XZ+pF+7/c103i8l
 U+5fJtAQV599A2LWDTLGT3AaLIP2nFIw6zH4vHoyyyDabNb64Uzf3ogRXEgzTaSzcyWP
 rXZ1pkmx3sEZO/nnnWgj2jRKrYuFVs+rvAVwCY4+hmU73tcLy4D5g41uLAno1aljGC/P
 Pur6WyzlDatOXp5dPpIBo7RBJrM8JRhZMr6irj67KDZXSBh/eEnNlt4/zFsMC3HlE2RC
 lb/yGf/5ZyGzoOWbBk6gpT21H20NobnkDudNYKYJoRJgBI9alOLOyR1IMswJyykOKfbW 1A== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U6sDD+lMWBDo0XoDRoyJbSnJc+JmOSR9LlEi0tdQ5fKDCxvznq2+3ISEFc2iIizF0NsWpLs8/FGwbGc9QFpkhuMzmcermWqTuP7H4yihFMd/SLDlXV3UIziANsEmN1q44vDOq3+ql98Qj102bTfT3DShCNGp6U8ElzJKhHWdyug/s61GcZKpLYFp/FnsZG1TSg9HFGuuk9mQnvQW2A28TiknFZyO1jV3wFhwaBrNpgwJMIAMiUWEmRPqPJSqWmTkiuaOUmITyMENYPoluKSd+JYyazLNx5awFp3o5GrVcEGIuNIn4HB15U8uKrVCac1fQFea9/tMvOLgjs2YGyIL3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rHcAdxv7gbw+A9E1/mIZdVxlid4Ee7MD5bGLZ8HFD04=;
 b=Od0sSRXzQZtDw8wLn2/0YkY8XZHy2qGXsMYBz3kCPWnjB8kBWAUBEjz9B/aTY6wR4n+NgR5u+wnAOIKsjH8MnD9zIFIXG8KONu6vLn2wPnxQMG/xNGDVRYRYP+ZAm4opU0koVe/5lQS3cpB++U6AnnOY9kmBXUe3AIbpifyDagDjBZnAvhgFoLL67WVDHI5V2skjqAxhGkk5wlRZu8qqfozt/JR/G8P6CMJcvlOEZTzj/jvPEbfO5wGCj/U+kgtSiLeTdt1CDG4Kv409rivZHc/Dxz287KQGEpf8Wvh4pusf04X3oC7DTUEOn5ldGRWmJ4KPqQXsgLEiaBeZpWcXLg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rHcAdxv7gbw+A9E1/mIZdVxlid4Ee7MD5bGLZ8HFD04=;
 b=uSmzAx/elnAawLOBOTAQfC6U2t2bykHR+naKy9LSrvKgEGJD5J+HcjTl3HCSsxy4rAVshxnPMdZYJrbacF/NhvAoEntAWOPSPoo4HNyqaf8JJSzYVGXn7A/pbakaHsq4IBQ2K8v58xhWGgGfE/Thij8H+E+YeyiCSKtN6TsqENg=
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v2 2/2] x86/xen: switch initial pvops IRQ functions to
 dummy ones
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        x86@kernel.org, virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Deep Shah <sdeep@vmware.com>,
        "VMware, Inc." <pv-drivers@vmware.com>,
        Thomas Gleixner
 <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
        Stefano Stabellini <sstabellini@kernel.org>
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-3-jgross@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <429cdc9b-b9f1-5393-8baf-cdd2046cd65a@oracle.com>
Date: Wed, 22 Sep 2021 17:49:32 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210922103102.3589-3-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: MN2PR07CA0018.namprd07.prod.outlook.com
 (2603:10b6:208:1a0::28) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1527ffbc-fa05-4bd5-2c49-08d97e12ddad
X-MS-TrafficTypeDiagnostic: BLAPR10MB4866:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB48664DAC5DC31EA29E5657A48AA29@BLAPR10MB4866.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:913;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	fBpIAUJr9JB5y9edz13noTVW4hM4VOmlYJjVgT6r5g6z5OhlHAvLh52CvbtJUx+cEI+NdYcYM8T/kBzt7JbUoi+VZZL9niFIHxnWsceJhR0m97thLqMu4f9fsAwBbACXv1t2OkpW1v+eknTjTjp9OSwGTonCOE/vsbm6g34wHLPDVsSgiSP8/oMIMDdCvLP4Hrgj5k94/2lp2q/unF4uDhowxLQvPY779bQzGh7bN3sKjdWCik7HmnyZpl25Y5foPV2HkVMcw8vav6Wp3HmvWXYjcgUNHWunPMujxbgSilgkZl5bGEYhFWoCjUX09rcvdZ810EG4RxnHK8mwvMH168zAPtg0kj3F5xyZFd1AXYaOXJEy/MOMCVal2UYgKEiedFL0/LACcsUAJv1Z+eA2BK7IVcy/6lSJtYpxMArrnWBpZg/gAEEfCsc5lOjJpO+fSRCq3ZYAB9StUDrbnzVoFeGR+y/BlTur/QAJtmAhj/pt9TFJO+C9STnY5JYUkPkE1G+JAQNI7VpClkSWK+pJSVxiD/gSr9A6o40uEJdO7WHX4tJKcnRvFMkSxaJJPTOnIbgsnJjxVUZ3+qJOzycS8K/1v/xz6AWFX5hLC56JKE2i4HV51AYYADm1blfoJfz4Abb9bm0M2BdrhXcUHD16vikONbYcoLXm3INxqgMcswWbWLnWxAwdY+6i8TgdG9jv0iYEbfry2Odi89nJKGpPO8szDA/wjMSjfAmHnjxtL/6fMHHmuFU6BeyOo3m3PVpf
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(53546011)(66946007)(66476007)(2616005)(316002)(66556008)(8936002)(16576012)(508600001)(2906002)(31696002)(7416002)(38100700002)(54906003)(44832011)(8676002)(4326008)(26005)(36756003)(186003)(5660300002)(4744005)(6486002)(956004)(86362001)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ejV2eHdNcHRxcFd2N2gzRXNUR2hneVFrUTBNb0NlNWJoZHRlK1J0Y21rUWly?=
 =?utf-8?B?bS9CaVo1QlRCT2QrZFF1Qzg0bWxEbzZsTkM5UThINEFITVE1OHJOalhqMUV4?=
 =?utf-8?B?ZEtPSnROdnpUOWYxTmdOVUJVTFcyaSsyc2lneVM3czdvUDNKZmRxQ04zQ0lh?=
 =?utf-8?B?VlZoOU44eHBsVHlUVDY5Y2hreUl4bU9SR2pscld0Z2VzSkNjOVU4cStxcGUw?=
 =?utf-8?B?Q3FFTk9Ua09GNkVCdCtMWlF6b0RHeXJFRllmd0tJU2V4SlhWVCtGZFVJNDNp?=
 =?utf-8?B?WnUwT1hocWZtZDhkYVY0T0dPRGxkOXdEN1JRbWhtcXZJWCtGUXl0S2NSS1VO?=
 =?utf-8?B?WjB5OUJvZnFzdmRLTVlZQ0cyVERoSllzbWlBQk14aTNlSXI2OHBzSlNPbUNz?=
 =?utf-8?B?cXBRUlZqUmxkdzcrcE5mZDJZdmhsRVkwV2orRElzSkROM2NNcmhFMHdZNVAz?=
 =?utf-8?B?SGxRbXM5MjFTRVpsSGhBQzVJREwwMUJGM1p6UU4zTjI5QkhzcnJBanN3V0p0?=
 =?utf-8?B?dkI1SkxCSHUxTWd5YzhxaEN4Szk5VnZhSVlhMHJmcUNlSkhESTN5VzJHQkhI?=
 =?utf-8?B?Z1JkN3dsaWJKbFF0YWRKK3hMZkx2S3Z0UFBvZFJVUWxXRE9yVU1nbmFLRXFQ?=
 =?utf-8?B?WlY1ay8reHZtVUFoWDdUcDdHdFhaYUlrbWE1V0xKZ1Rhb0RpeCs5SlBLRTdm?=
 =?utf-8?B?Ukh1b3RGV2doYWVWZFlEL0RwVEEvUVhLZlN3N2d3QlErNVg2Q1F1cVNCaEJh?=
 =?utf-8?B?NUhIbTVuenljcURvQkNxT00xclVZZWhVcHA1QUNHNXhHaCtkZFgxN0FyelVV?=
 =?utf-8?B?TGpBV0VzSHIydVROUzQ3N0FnUXF0UVJrQkRES29yWWhvdE9sbWhqczEvVXl5?=
 =?utf-8?B?cldwejUwVlVhY01SQm80V0lCN2I5ekprTXUzTzBkRU5YaUh2ZURuQkl1UGdl?=
 =?utf-8?B?WjFJU2h0ei8ybDBKMVQ0RGpXVERQZ2ZxRXdhVXpiZ1ZVMWp2WjJreEVlQXFK?=
 =?utf-8?B?K3NyVUpoQzBObXQvRno1NlBOWWhmUWs2Rk9PaEpNUnoyUVpXQ2pFVWViMHRB?=
 =?utf-8?B?R2xlVk41Zll6TzZ0QWVYSWU5MXBxSkhwTm1YOWVwNGlVeTZHczBkbUFIcHkw?=
 =?utf-8?B?dzhLZnR1OGNYaytNWS9BV21iM2VxdXRTcXpIa09KRXlQNTVNaUx1ZWJaeUdX?=
 =?utf-8?B?UXFycEJMYXg0Zi9TTXdPTlVIZCtUSU5iemJPb1lmRGhUdlAyRTcwNjA0R01M?=
 =?utf-8?B?cS9TajRYdzM1YTRxaU9WbnMya1JYdm1uTjVjMVV6dktlV3UvdFNPUDgxS3Fy?=
 =?utf-8?B?S2FOeDdRS3NyazN6Zzg2MG5lMytKREhGVndId3VFMC9KTkMyekRDbmRYNlRK?=
 =?utf-8?B?eG52eDZ5QS83U055dXJxMDEzblBna0VVWGUvYUs5QjgrN2RMRXpoeHowdVJV?=
 =?utf-8?B?bWkydm9XdjJ3bUZCdVYzcjJlMEtxcUpRRElDVHRycjZpTCtyanpkZkhPdFgy?=
 =?utf-8?B?Zi9XWUZJb2RJbW83YndwdDVWVi9EVGY2Q2RjTkNBT3JsVWFHNUdqNVRmZ0Nx?=
 =?utf-8?B?WUg1dlBtTklYbU8vaVdFT3MyNDBlZFNQSWtHMVpyVk1ZRDJqb3R4cXlqZFJx?=
 =?utf-8?B?T01vMVh4WVVwUzFLWnY5TW13VnZJK3JyYVRTcXdsWjc2VnBuTEJpYi9hL0h1?=
 =?utf-8?B?cGhDeERYSUt2TEFIV0ZjSmQ3eHNOL1pjODdqcW5YZktQeEExaTV2dDBZbE0z?=
 =?utf-8?Q?aE9A5rw1Aj+3lnqQkJot4Ku1pxFIoBWH7T9Mt+0?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1527ffbc-fa05-4bd5-2c49-08d97e12ddad
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 21:49:34.4903
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5Hvj0oHi00jglKU7oAxyceL+bKWWwR0WMuvXH1WtTryDwGvZbogiUL/8UoP+tMuEVGSsfhee4i02SauGgohzXIz2uPhCW+GfnnsWSo6yVDw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB4866
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10115 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=945
 spamscore=0 malwarescore=0 adultscore=0 mlxscore=0 bulkscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109220139
X-Proofpoint-GUID: FXW9w2_M-W0twU8OalnHWs6mbZqAVAql
X-Proofpoint-ORIG-GUID: FXW9w2_M-W0twU8OalnHWs6mbZqAVAql


On 9/22/21 6:31 AM, Juergen Gross wrote:
> The initial pvops functions handling irq flags will only ever be called
> before interrupts are being enabled.
>
> So make the __init and switch them to be dummy functions:


What are you making __init?


>  
> +/* stub always return ing 0. */


"always returning"


-boris



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 21:51:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 21:51:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193109.343958 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA9m-0002BG-Az; Wed, 22 Sep 2021 21:51:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193109.343958; Wed, 22 Sep 2021 21:51:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTA9m-0002B9-7e; Wed, 22 Sep 2021 21:51:58 +0000
Received: by outflank-mailman (input) for mailman id 193109;
 Wed, 22 Sep 2021 21:51:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTA9l-0002B3-E4
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 21:51:57 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4d24572a-1bef-11ec-b9eb-12813bfff9fa;
 Wed, 22 Sep 2021 21:51:55 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id EAB5060F24;
 Wed, 22 Sep 2021 21:51:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d24572a-1bef-11ec-b9eb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632347514;
	bh=O3nmDTs7JcZl+ngCNmYPU2k++1e7RYgX2tsBYicvZ8I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=eNclAZjJ4tKOK7vKdmKcNnAE2WiNJamlEXIOAwknYI85U/ibr03h6z1RFdtBsJTXk
	 vGQa0J86jInj1Ol/xMafU91hFe9LE9hGCg1GMzTyrPbrP+74Ovnfyq3EH9aqzpF8zv
	 ZgBfDkKIHV//PN3+tY216ywcTaNa7+AbxUTKNu6JHGDXtqtRUQJzc7N4wkG1j2W5sf
	 vxnENX1feo+O61/MRAHDHtu3PfpNOWL3eKdx4m0IOYYB05N1BIoG68wNhuUrAXDyKn
	 Lgv+loHVgLirruooWtfGxQ0X7rCTD/RHmc6KmmEoeDm530iFbvIHtoZNdc594Ij+ZJ
	 VFC0xzCKysggw==
Date: Wed, 22 Sep 2021 14:51:49 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using
 EFI boot
In-Reply-To: <20210922141341.42288-3-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s>
References: <20210922141341.42288-1-luca.fancellu@arm.com> <20210922141341.42288-3-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Luca Fancellu wrote:
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "uefi,binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Overall the patch is pretty good. I don't have any major concerns (only
one about the documentation). Some minor comments below.


> ---
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
>  docs/misc/arm/device-tree/booting.txt |  21 +++
>  docs/misc/efi.pandoc                  | 203 ++++++++++++++++++++
>  xen/arch/arm/efi/efi-boot.h           | 257 +++++++++++++++++++++++++-
>  xen/arch/x86/efi/efi-boot.h           |   6 +
>  xen/common/efi/boot.c                 |  36 ++--
>  5 files changed, 508 insertions(+), 15 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 44cd9e1a9a..bc0f8913db 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -182,6 +182,13 @@ The kernel sub-node has the following properties:
>  
>      Command line parameters for the guest kernel.
>  
> +- efi,binary (UEFI boot only)

Below it is actually uefi,binary, not efi,binary. I would use
uefi,binary consistently.


> +    Specifies the file name to be loaded by the UEFI boot for this module. If
> +    this is specified, there is no need to specify the reg property because it
> +    will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.

We need to specify that this parameter is a string.


>  The ramdisk sub-node has the following properties:
>  
>  - compatible
> @@ -193,6 +200,13 @@ The ramdisk sub-node has the following properties:
>      Specifies the physical address of the ramdisk in RAM and its
>      length.
>  
> +- efi,binary (UEFI boot only)
> +
> +    Specifies the file name to be loaded by the UEFI boot for this module. If
> +    this is specified, there is no need to specify the reg property because it
> +    will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.

same here

>  
>  Example
>  =======
> @@ -257,6 +271,13 @@ The dtb sub-node should have the following properties:
>      Specifies the physical address of the device tree binary fragment
>      RAM and its length.
>  
> +- efi,binary (UEFI boot only)
> +
> +    Specifies the file name to be loaded by the UEFI boot for this module. If
> +    this is specified, there is no need to specify the reg property because it
> +    will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.

same here


>  As an example:
>  
>          module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index e289c5e7ba..698196e129 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "uefi,binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	compatible = "xen,domain";
> +	memory = <0 0x20000>;
> +	cpus = <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "console=ttyAMA0";
> +	};
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +Here the supported user cases for Xen when UEFI boot is used:

"Supported" means different things to different people, so I would say
instead:

These are the different ways to boot a Xen system from UEFI:


> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in the
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen node,
> +examples of how to do that are provided by the documentation about dom0less
> +and the example above shows how to use the "uefi,binary" property to use the
> +UEFI stub for module loading.
> +Providing the multiboot modules in the device tree, make Xen skip its UEFI
> +configuration file, if it is needed for some reason, specify the "uefi,cfg-load"
> +property in the /chosen node.

This last sentence is not very clear and it looks like one needs to
specify "uefi,cfg-load" to skip the Xen config file when actually it is
the opposite. I would suggest instead:

When adding DomU modules to device tree, also add the property
uefi,cfg-load under chosen to have Xen load the Xen config file.
Otherwise, Xen will skip the config file and rely on device tree alone.


> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	uefi,cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this one
> +the configuration file must be processed so the /chosen node must have the
> +"uefi,cfg-load" property.
> +
> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	uefi,cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 8ceeba4ad1..e2b007ece0 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,43 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} dom0less_module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> +static struct file __initdata dom0less_file;
> +static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
> +static unsigned int __initdata dom0less_modules_available =
> +                               MAX_DOM0LESS_MODULES;
> +static unsigned int __initdata dom0less_modules_idx;

This is a lot better!

We don't need both dom0less_modules_idx and dom0less_modules_available.
You can just do:

#define dom0less_modules_available (MAX_DOM0LESS_MODULES - dom0less_modules_idx)
static unsigned int __initdata dom0less_modules_idx;

But maybe we can even get rid of dom0less_modules_available entirely?

We can change the check at the beginning of allocate_dom0less_file to:

if ( dom0less_modules_idx == dom0less_modules_available )
    blexit

Would that work?


> +#define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
> +
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int get_dom0less_file_index(const char *name, unsigned int name_len);
> +static unsigned int allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                           const char *name,
> +                                           unsigned int name_len);
> +static void handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
> +                                        int module_node_offset,
> +                                        int reg_addr_cells,
> +                                        int reg_size_cells);
> +static void handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                        int domain_node,
> +                                        int addr_cells,
> +                                        int size_cells);
> +static bool efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle);
>
>  #define DEVICE_TREE_GUID \
>  {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
>  
> @@ -552,8 +586,218 @@ static void __init efi_arch_handle_module(const struct file *file,
>                           kernel.size) < 0 )
>              blexit(L"Unable to set reg property.");
>      }
> -    else
> +    else if ( file != &dom0less_file )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>          blexit(L"Unknown module type");
> +    /*
> +     * dom0less_modules_available is decremented here because for each dom0
> +     * file added, there will be an additional bootmodule, so the number
> +     * of dom0less module files will be decremented because there is
> +     * a maximum amount of bootmodules that can be loaded.
> +     */
> +    dom0less_modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the dom0less_files array or a negative
> + * number if no file with that name is found.
> + */
> +static int __init get_dom0less_file_index(const char *name,
> +                                          unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret = ERROR_DOM0LESS_FILE_NOT_FOUND;
> +
> +    for (i = 0; i < dom0less_modules_idx; i++)
> +    {
> +        dom0less_module_name *mod = &dom0less_modules[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it
> + * returns the index of the file in the dom0less_files array.
> + */
> +static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                                  const char *name,
> +                                                  unsigned int name_len)
> +{
> +    dom0less_module_name* file_name;
> +    union string module_name;
> +    unsigned int ret_idx;
> +
> +    /*
> +     * Check if there is any space left for a domU module, the variable
> +     * dom0less_modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !dom0less_modules_available )
> +        blexit(L"No space left for domU modules");

This is the check that could be based on dom0less_modules_idx


> +    module_name.s = (char*) name;
> +    ret_idx = dom0less_modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &dom0less_modules[ret_idx];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +        blexit(L"Error allocating memory for dom0less binary name");
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len);
> +    file_name->name_len = name_len;

Sorry for the silly question but do we need to set the '\0' at the end
of the string or can we assumed the memory returned by AllocatePool is
already zeroed? Or should we call strlcpy with name_len+1 ?


> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &dom0less_file, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr = dom0less_file.addr;
> +    file_name->size = dom0less_file.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    dom0less_modules_idx++;
> +
> +    return ret_idx;
> +}
> +
> +/*
> + * This function checks for the presence of the uefi,binary property in the
> + * module, if found it loads the binary as dom0less module and sets the right
> + * address for the reg property into the module DT node.
> + */
> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
> +                                          int module_node_offset,
> +                                          int reg_addr_cells,
> +                                          int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx;
> +    dom0less_module_name *file;
> +
> +    /* Read uefi,binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return;
> +
> +    file_idx = get_dom0less_file_index(uefi_name_prop, uefi_name_len);
> +    if (file_idx < 0)
> +        file_idx = allocate_dom0less_file(dir_handle, uefi_name_prop,
> +                                          uefi_name_len);
> +
> +    file = &dom0less_modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +        blexit(L"Unable to add domU ramdisk FDT node.");
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +        blexit(L"Unable to set reg property.");
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + */
> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                               int domain_node,
> +                                               int addr_cells,
> +                                               int size_cells)
> +{
> +    int module_node;
> +    /*
> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
> +     * inside this node
> +     */
> +    for ( module_node = fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +        if ( (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,kernel") == 0) ||
> +             (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,ramdisk") == 0) ||
> +             (fdt_node_check_compatible(fdt, module_node,
> +                                        "multiboot,device-tree") == 0) )
> +            /* The compatible is one of the strings above, check the module */
> +            handle_dom0less_module_node(dir_handle, module_node, addr_cells,
> +                                        size_cells);
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + */
> +static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen;
> +    int addr_len, size_len;
> +    unsigned int i = 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +        blexit(L"Unable to setup chosen node");
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( int node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        int addr_cells, size_cells, len;
> +        const struct fdt_property *prop;
> +
> +        if ( fdt_node_check_compatible(fdt, node, "xen,domain") != 0 )
> +            continue;
> +
> +        /* Get or set #address-cells and #size-cells */
> +        prop = fdt_get_property(fdt, node, "#address-cells", &len);
> +        if ( !prop )
> +            blexit(L"#address-cells not found in domain node.");
> +
> +        addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +        prop = fdt_get_property(fdt, node, "#size-cells", &len);
> +        if ( !prop )
> +            blexit(L"#size-cells not found in domain node.");
> +
> +        size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +        /* Found a node with compatible xen,domain; handle this node. */
> +        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_cells);
> +    }
> +
> +    /* Free dom0less file names if any */
> +    for ( ; i < dom0less_modules_idx; i++ )
> +    {
> +        /* Free dom0less binary names */
> +        efi_bs->FreePool(dom0less_modules[i].name);
> +    }
> +
> +    if ( dom0less_modules_idx > 0 )
> +        return true;
> +
> +    return false;
>  }
>  
>  static void __init efi_arch_cpu(void)
> @@ -562,8 +806,19 @@ static void __init efi_arch_cpu(void)
>  
>  static void __init efi_arch_blexit(void)
>  {
> +    unsigned int i = 0;
> +
>      if ( dtbfile.need_to_free )
>          efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free dom0less files if any */
> +    for ( ; i < dom0less_modules_idx; i++ )
> +    {
> +        /* Free dom0less binary names */
> +        efi_bs->FreePool(dom0less_modules[i].name);
> +        /* Free dom0less binaries */
> +        efi_bs->FreePages(dom0less_modules[i].addr,
> +                          PFN_UP(dom0less_modules[i].size));
> +    }
>      if ( memmap )
>          efi_bs->FreePool(memmap);
>  }
> diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
> index 9b0cc29aae..950fdf16b7 100644
> --- a/xen/arch/x86/efi/efi-boot.h
> +++ b/xen/arch/x86/efi/efi-boot.h
> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
>      efi_bs->FreePool(ptr);
>  }
>  
> +static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    /* x86 doesn't support dom0less */
> +    return false;
> +}
> +
>  static void __init efi_arch_cpu(void)
>  {
>      uint32_t eax = cpuid_eax(0x80000000);
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..7d8734199e 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1127,15 +1127,16 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
>      EFI_LOADED_IMAGE *loaded_image;
>      EFI_STATUS status;
> -    unsigned int i, argc;
> -    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
> +    unsigned int i, argc = 0;
> +    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
>      UINTN gop_mode = ~0;
>      EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    EFI_FILE_HANDLE dir_handle;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1217,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>      efi_arch_relocate_image(0);
>  
> +    /* Get the file system interface. */
> +    dir_handle = get_parent_handle(loaded_image, &file_name);
> +
>      if ( use_cfg_file )
>      {
> -        EFI_FILE_HANDLE dir_handle;
>          UINTN depth, cols, rows, size;
>  
>          size = cols = rows = depth = 0;
> @@ -1229,9 +1232,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>          gop = efi_get_gop();
>  
> -        /* Get the file system interface. */
> -        dir_handle = get_parent_handle(loaded_image, &file_name);
> -
>          /* Read and parse the config file. */
>          if ( read_section(loaded_image, L"config", &cfg, NULL) )
>              PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1285,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +        if ( name.s )
> +            option_str = split_string(name.s);
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
> +             name.s )

I'd probably indent this and put it under the previous if, instead of
adding "&& name.s". But I'll leave it up to you.


>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> @@ -1361,12 +1360,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !efi_arch_check_dom0less_boot(dir_handle) && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");
> +
> +    dir_handle->Close(dir_handle);
> +
>      efi_arch_edd();
>  
>      /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:04:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:04:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193116.343968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTALq-0003xd-Jt; Wed, 22 Sep 2021 22:04:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193116.343968; Wed, 22 Sep 2021 22:04:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTALq-0003xW-Gf; Wed, 22 Sep 2021 22:04:26 +0000
Received: by outflank-mailman (input) for mailman id 193116;
 Wed, 22 Sep 2021 22:04:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTALp-0003xQ-DT
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:04:25 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 082567a7-7bfe-47b9-b941-26d9ffd77b5c;
 Wed, 22 Sep 2021 22:04:24 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0B41061019;
 Wed, 22 Sep 2021 22:04:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 082567a7-7bfe-47b9-b941-26d9ffd77b5c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632348263;
	bh=vqa3+sLgyMyBECDMtbSv1gF5D/5Z3StQ0yR5jODH2pE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=QL/ugiFdJg/o4vfd6/x+haECioSvUCpvoGsTqPRvpwHXnT0Y6R8t9fNSN9ytrQ0LD
	 v8pZnsnk6VdlaPYRFZo1B2NHjbfnUGK97rkviIw+W4ZrYSr5a1F1Ehv+mhGZU2mJ34
	 fYS4KK2Q6cWN6sWO+q9VHD2J1v0EuQ4BdIrVh32gS5yddakBj9BPItVYgwjaHo45VZ
	 yGWJYy+EewKpW8UwU6vwrtwde20oaqSO14YsXE5bctCyOJiXrvE8h1dPZc0YpaYpBj
	 E/q7IVjS/5/Tr4FBade8Mf1iUqqRp/CwqXTPqY3fnGj2XQBns3XtOIzOrr078UNul4
	 Y99GD/pUyKXeg==
Date: Wed, 22 Sep 2021 15:04:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 02/17] xen/pci: solve compilation error on ARM with
 HAS_PCI enabled
In-Reply-To: <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221503490.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1615104374-1632348263=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1615104374-1632348263=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Compilation error is observed when HAS_PCI is enabled for ARM
> architecture.
> 
> Add definition for arch_iommu_use_permitted() and
> arch_pci_clean_pirqs().
> 
> pci.c: In function ‘deassign_device’:
> pci.c:849:49: error: implicit declaration of function ‘pci_to_dev’;
> did you mean ‘dt_to_dev’? [-Werror=implicit-function-declaration]
>             pci_to_dev(pdev));
> pci.c:880: undefined reference to `arch_pci_clean_pirqs’
> pci.c:1392: undefined reference to `arch_iommu_use_permitted'
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v2:
> - Remove pci_conf_read*(..) dummy implementation
> - Add in code comment for arch_pci_clean_pirqs() and arch_iommu_use_permitted()
> - Fixed minor comments
> ---
>  xen/arch/arm/Makefile               |  1 +
>  xen/arch/arm/pci/Makefile           |  1 +
>  xen/arch/arm/pci/pci.c              | 33 +++++++++++++++++++++++++++++
>  xen/drivers/passthrough/arm/iommu.c |  9 ++++++++
>  xen/include/asm-arm/pci.h           | 31 ++++++++++++++++++++++++---
>  5 files changed, 72 insertions(+), 3 deletions(-)
>  create mode 100644 xen/arch/arm/pci/Makefile
>  create mode 100644 xen/arch/arm/pci/pci.c
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b4..44d7cc81fa 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) += arm32/
>  obj-$(CONFIG_ARM_64) += arm64/
>  obj-$(CONFIG_ARM_64) += efi/
>  obj-$(CONFIG_ACPI) += acpi/
> +obj-$(CONFIG_HAS_PCI) += pci/
>  ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y += platforms/
>  endif
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> new file mode 100644
> index 0000000000..a98035df4c
> --- /dev/null
> +++ b/xen/arch/arm/pci/Makefile
> @@ -0,0 +1 @@
> +obj-y += pci.o
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> new file mode 100644
> index 0000000000..a7a7bc3213
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci.c
> @@ -0,0 +1,33 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +
> +/*
> + * PIRQ event channels are not supported on Arm, so nothing to do.
> + */
> +int arch_pci_clean_pirqs(struct domain *d)
> +{
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c
> index db3b07a571..ee653a9c48 100644
> --- a/xen/drivers/passthrough/arm/iommu.c
> +++ b/xen/drivers/passthrough/arm/iommu.c
> @@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)
>  void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  {
>  }
> +
> +/*
> + * Unlike x86, Arm doesn't support mem-sharing, mem-paging and log-dirty (yet).
> + * So there is no restriction to use the IOMMU.
> + */
> +bool arch_iommu_use_permitted(const struct domain *d)
> +{
> +    return true;
> +}
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index de13359f65..7dd9eb3dba 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -1,7 +1,32 @@
> -#ifndef __X86_PCI_H__
> -#define __X86_PCI_H__
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
>  
> +#ifndef __ARM_PCI_H__
> +#define __ARM_PCI_H__
> +
> +#ifdef CONFIG_HAS_PCI
> +
> +#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> +
> +/* Arch pci dev struct */
>  struct arch_pci_dev {
> +    struct device dev;
>  };
>  
> -#endif /* __X86_PCI_H__ */
> +#else   /*!CONFIG_HAS_PCI*/
> +
> +struct arch_pci_dev { };
> +
> +#endif  /*!CONFIG_HAS_PCI*/
> +#endif /* __ARM_PCI_H__ */
> -- 
> 2.17.1
> 
--8323329-1615104374-1632348263=:17979--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:08:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:08:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193122.343979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAPR-0004mn-1S; Wed, 22 Sep 2021 22:08:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193122.343979; Wed, 22 Sep 2021 22:08:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAPQ-0004mg-Ug; Wed, 22 Sep 2021 22:08:08 +0000
Received: by outflank-mailman (input) for mailman id 193122;
 Wed, 22 Sep 2021 22:08:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTAPP-0004ma-U1
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:08:07 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b5175511-9645-42b9-9210-8d4af4ef1ef8;
 Wed, 22 Sep 2021 22:08:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 600E66109E;
 Wed, 22 Sep 2021 22:08:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5175511-9645-42b9-9210-8d4af4ef1ef8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632348486;
	bh=QWQVk8O80Iz9b+CDsZQa+wGLp70ISYSJPLI5YvAbbm8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=s2rmUXcGajM2QzSwiYuQCMQd6mEtF9Arm17dwzqrew1eNTLKK73MJFWfih/Ckyy5e
	 0MsXsBIJXAMOOyVG5WN439fRp74AUW1lyzHmJg824kyekMEGB0JUWvnGtTaSB/gTq/
	 xzmSM4iSoE86y4iUQ31dQ3aZkn/bJwGtKddAfjZ7nTa0MXoy3+wOqnX919syQTk3zP
	 1s0dCERX74IIWJH66A1LBdDexOGzGVJ6Gws4pus2qJE2ECIbyFXDlPuwaoL2SATtKn
	 n4mocKAHCoSDyfZmj6MqWHN08FYDNuQ/sp/NZy5ehCMnP5AOFDjfM+4w8bGuse+7Xw
	 f3m/9nh58z/2w==
Date: Wed, 22 Sep 2021 15:08:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Jan Beulich <jbeulich@suse.com>, 
    Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
In-Reply-To: <af2360805e6d615995a1071575546ecbb011a4d7.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221507580.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <af2360805e6d615995a1071575546ecbb011a4d7.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1011149910-1632348486=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1011149910-1632348486=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Wed, 22 Sep 2021, Rahul Singh wrote:
> prelink.o: In function `pcie_aer_get_firmware_first’:
> drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'
> 
> Compilation error is observed when ACPI and HAS_PCI is enabled for ARM
> architecture. APEI not supported on ARM yet move the code under
> CONFIG_X86 flag to gate the code for ARM.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v2: Add in code comment "APEI not supported on ARM yet"
> ---
>  xen/drivers/passthrough/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index 8996403161..d774a6154e 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -1150,7 +1150,8 @@ void __hwdom_init setup_hwdom_pci_devices(
>      pcidevs_unlock();
>  }
>  
> -#ifdef CONFIG_ACPI
> +/* APEI not supported on ARM yet. */
> +#if defined(CONFIG_ACPI) && defined(CONFIG_X86)
>  #include <acpi/acpi.h>
>  #include <acpi/apei.h>
>  
> -- 
> 2.17.1
> 
> 
--8323329-1011149910-1632348486=:17979--


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:11:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:11:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193129.343996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTASr-0006By-Q6; Wed, 22 Sep 2021 22:11:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193129.343996; Wed, 22 Sep 2021 22:11:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTASr-0006Bg-LA; Wed, 22 Sep 2021 22:11:41 +0000
Received: by outflank-mailman (input) for mailman id 193129;
 Wed, 22 Sep 2021 22:11:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTASq-00069l-0E
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:11:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1f3df2a5-6f41-4bbd-9754-11cdc41aeea1;
 Wed, 22 Sep 2021 22:11:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 73F7B21EC7;
 Wed, 22 Sep 2021 16:15:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f3df2a5-6f41-4bbd-9754-11cdc41aeea1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327354;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PKL+1gA6erzUqEOhaWDpJ5h6F3B//VEnmj0ACbfqSX0=;
	b=GLtOAaaPxTWsl4cHZ/jIm6GfdPz+u/QkndP3ql38H10VNAIkQ1EwIQigKoDRTjU3Cd27wF
	Z6jKSrHDiHVqytohW3K+04Wi7HCpPSBqociS+H/TTxSa5+XQbb0tINq8trisTlGjDgV8bz
	b/1Ub1hjekRXgH5VyDRqY1f0lN6lPzM=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 08/16] qdev-monitor: Check sysbus device type before creating it
Date: Wed, 22 Sep 2021 18:13:57 +0200
Message-Id: <20210922161405.140018-9-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Add an early check to test if the requested sysbus device type
is allowed by the current machine before creating the device. This
impacts both -device cli option and device_add qmp command.

Before this patch, the check was done well after the device has
been created (in a machine init done notifier). We can now report
the error right away.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---
 softmmu/qdev-monitor.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 47ccd90be8..f1c9242855 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -40,6 +40,7 @@
 #include "qemu/cutils.h"
 #include "hw/qdev-properties.h"
 #include "hw/clock.h"
+#include "hw/boards.h"
 
 /*
  * Aliases were a bad idea from the start.  Let's keep them
@@ -268,6 +269,16 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
         return NULL;
     }
 
+    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
+        /* sysbus devices need to be allowed by the machine */
+        MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
+        if (!machine_class_is_dynamic_sysbus_dev_allowed(mc, *driver)) {
+            error_setg(errp, "'%s' is not an allowed pluggable sysbus device "
+                             " type for the machine", *driver);
+            return NULL;
+        }
+    }
+
     return dc;
 }
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:11:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:11:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193128.343990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTASr-0006A3-Gk; Wed, 22 Sep 2021 22:11:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193128.343990; Wed, 22 Sep 2021 22:11:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTASr-00069w-Ds; Wed, 22 Sep 2021 22:11:41 +0000
Received: by outflank-mailman (input) for mailman id 193128;
 Wed, 22 Sep 2021 22:11:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTASq-00069k-0k
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:11:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0e48a7ee-1bf2-11ec-b9ee-12813bfff9fa;
 Wed, 22 Sep 2021 22:11:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 89D1B21ED5;
 Wed, 22 Sep 2021 16:16:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e48a7ee-1bf2-11ec-b9ee-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327362;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=V/wXO8Fw46s42gvXscl9fDzKCmjnZalAaCkoeLW9idc=;
	b=fzmPX4ObaVsIGjwR4GzoY/SbGumd3hE8zzTj5wQNt93NX3hZr8kYnEk/i6a/ahRb3BcyzH
	r4OPUKes6fAREsZ6M/d6oRgJNM/rLp/UW1rGVvosF72Xhv4ZT3ZMvT7X96ovqsT8bIwjFg
	MzDwJiVoSLVcg/Nq1RzI5wAN8p9MecQ=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 15/16] hw/char/ibex_uart: set user_creatable
Date: Wed, 22 Sep 2021 18:14:04 +0200
Message-Id: <20210922161405.140018-16-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

This patch allows to create the device using device_add
using -preconfig mode. This sysbus device still needs to
be allowed by a machine to be created after preconfig is done.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Depending on chosen condition for a device to be added, this commit
may change.
---
 hw/char/ibex_uart.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/char/ibex_uart.c b/hw/char/ibex_uart.c
index 9b0a817713..b1646422c0 100644
--- a/hw/char/ibex_uart.c
+++ b/hw/char/ibex_uart.c
@@ -546,6 +546,7 @@ static void ibex_uart_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->user_creatable = true;
     dc->reset = ibex_uart_reset;
     dc->realize = ibex_uart_realize;
     dc->vmsd = &vmstate_ibex_uart;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:13:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:13:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193140.344013 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAUp-0007Ml-6H; Wed, 22 Sep 2021 22:13:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193140.344013; Wed, 22 Sep 2021 22:13:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAUp-0007Me-25; Wed, 22 Sep 2021 22:13:43 +0000
Received: by outflank-mailman (input) for mailman id 193140;
 Wed, 22 Sep 2021 22:13:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTAUn-0007MU-UL
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:13:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 578debc4-1bf2-11ec-b9ee-12813bfff9fa;
 Wed, 22 Sep 2021 22:13:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 179C96103C;
 Wed, 22 Sep 2021 22:13:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 578debc4-1bf2-11ec-b9ee-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632348820;
	bh=S0NFt5MtrdqOEMYTxFcUac2mrdB+b4BgWYpchcIEUuc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iF61E41TkYzpbBpaLWXTaJBENqr8KFG+308MKqVKXTrLozSfevYsFalZA5nZLbPgg
	 Xl8M0p6CxG2YMPaodqXnhlGy2G2SOogAPP3PQeNgfN5wBwx8WeWOh5Ns9V2LTBnT19
	 BwxsPtsPwlJTW53ykwbJUU3BCdNofJzIwYOXxFkAa86RTH3jLWf/KzjVHvfEAux7DS
	 a/GpVBvWaTb+4fy5+3EMiImrByP/3sSZTur7hgIRopTEW1G2ZfQk4mqIRc3ztiKJM6
	 OeJi45PsQH3JO1I0TZwoFCLqK1FSj1BPgiW7XAkmn5lBrkhJ/n8mjE7eWxwT2hSLsL
	 GJQuw18aYfL0A==
Date: Wed, 22 Sep 2021 15:13:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Ian Jackson <iwj@xenproject.org>, 
    Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 04/17] xen/arm: xc_domain_ioport_permission(..) not
 supported on ARM.
In-Reply-To: <e71044315a6d2a3fb127d62954579390cb837d10.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221511320.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <e71044315a6d2a3fb127d62954579390cb837d10.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> ARM architecture does not implement I/O ports. Ignore this call on ARM
> to avoid the overhead of making a hypercall just for Xen to return
> -ENOSYS.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Instead of returning success in XEN, ignored the call in xl.
> ---
>  tools/libs/ctrl/xc_domain.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
> index 23322b70b5..25c95f6596 100644
> --- a/tools/libs/ctrl/xc_domain.c
> +++ b/tools/libs/ctrl/xc_domain.c
> @@ -1348,6 +1348,14 @@ int xc_domain_ioport_permission(xc_interface *xch,
>                                  uint32_t nr_ports,
>                                  uint32_t allow_access)
>  {
> +#if defined(__arm__) || defined(__aarch64__)
> +    /*
> +     * The ARM architecture does not implement I/O ports.
> +     * Avoid the overhead of making a hypercall just for Xen to return -ENOSYS.
> +     * It is safe to ignore this call on ARM so we just return 0.
> +     */
> +    return 0;

I think this is fine from a code change point of view.

I wonder if we want to return 0 or return an error here, but I am
thinking that 0 is OK because there is really nothing to do anyway as
they end up ignored one way or the other.

Based on that reasoning:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> +#else
>      DECLARE_DOMCTL;
>  
>      domctl.cmd = XEN_DOMCTL_ioport_permission;
> @@ -1357,6 +1365,7 @@ int xc_domain_ioport_permission(xc_interface *xch,
>      domctl.u.ioport_permission.allow_access = allow_access;
>  
>      return do_domctl(xch, &domctl);
> +#endif
>  }
>  
>  int xc_availheap(xc_interface *xch,
> -- 
> 2.17.1
> 
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 22:37:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 22:37:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193146.344024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAs3-0001sZ-4t; Wed, 22 Sep 2021 22:37:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193146.344024; Wed, 22 Sep 2021 22:37:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTAs3-0001sS-1Y; Wed, 22 Sep 2021 22:37:43 +0000
Received: by outflank-mailman (input) for mailman id 193146;
 Wed, 22 Sep 2021 22:37:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTAs1-0001ob-BM
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 22:37:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ab1f4438-d4a5-465b-a633-97a8f5ebf8dc;
 Wed, 22 Sep 2021 22:37:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A5BCF610A0;
 Wed, 22 Sep 2021 22:37:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab1f4438-d4a5-465b-a633-97a8f5ebf8dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632350259;
	bh=FnWej46FfN+To0zH+p7BjZJHLdlEhHTBY/bbrmGdwq4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=htbQH+9K8KHMJrN7+OChjn4EJvTXLKOAMUMKO7Y/c6vI1GHfSsYMmhqJ+hGEg50P4
	 elfbppmo4bVobG0RP/9QtWLMULD3FOGRxmzvFyHZdEqkCfGOMMY0ORHyT5ri5dRary
	 H9rSpuHfAqkZkYHvlJw8YAhLS03+JBairAn2i1CMV+Ie5DeV3+AohGtaC481I2fBaH
	 qMETT0ZgEkfgQpFiH9MzcVoq/Z3e4BbB0010cgsntRxmOZ4w1WId5bI46BOuavzi4O
	 Yojfczon9zKlQHlz5KsQEmVN6vEcq/tcVPeeSGlbzmSmLV7VVIlWvJPBNlSNs1KqSJ
	 /xRR96JHglrmg==
Date: Wed, 22 Sep 2021 15:37:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support
 for ARM
In-Reply-To: <a1c447ced213f07797eebc9471c48df56a187033.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221527510.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <a1c447ced213f07797eebc9471c48df56a187033.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Hardware domain is in charge of doing the PCI enumeration and will
> discover the PCI devices and then will communicate to XEN via hyper
> call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
> 
> Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
> 
> As most of the code for PHYSDEVOP_pci_device_* is the same between x86
> and ARM, move the code to a common file to avoid duplication.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Add support for PHYSDEVOP_pci_device_remove()
> - Move code to common code
> ---
>  xen/arch/arm/physdev.c          |  5 +-
>  xen/arch/x86/physdev.c          | 50 +-------------------
>  xen/arch/x86/x86_64/physdev.c   |  4 +-
>  xen/common/Makefile             |  1 +
>  xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/hypercall.h |  2 -
>  xen/include/asm-arm/numa.h      |  5 ++
>  xen/include/asm-x86/hypercall.h |  9 ++--
>  xen/include/xen/hypercall.h     |  8 ++++
>  9 files changed, 106 insertions(+), 59 deletions(-)
>  create mode 100644 xen/common/physdev.c
> 
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index e91355fe22..4e00b03aab 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -8,10 +8,9 @@
>  #include <xen/lib.h>
>  #include <xen/errno.h>
>  #include <xen/sched.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>  
> -
> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  {
>      gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd);
>      return -ENOSYS;
> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
> index 23465bcd00..c00cc99404 100644
> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
>  }
>  #endif /* COMPAT */
>  
> -ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  {
>      int irq;
>      ret_t ret;
> @@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          break;
>      }
>  
> -    case PHYSDEVOP_pci_device_add: {
> -        struct physdev_pci_device_add add;
> -        struct pci_dev_info pdev_info;
> -        nodeid_t node;
> -
> -        ret = -EFAULT;
> -        if ( copy_from_guest(&add, arg, 1) != 0 )
> -            break;
> -
> -        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
> -        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> -        {
> -            pdev_info.is_virtfn = 1;
> -            pdev_info.physfn.bus = add.physfn.bus;
> -            pdev_info.physfn.devfn = add.physfn.devfn;
> -        }
> -        else
> -            pdev_info.is_virtfn = 0;
> -
> -        if ( add.flags & XEN_PCI_DEV_PXM )
> -        {
> -            uint32_t pxm;
> -            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
> -                                sizeof(add.optarr[0]);
> -
> -            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> -                break;
> -
> -            node = pxm_to_node(pxm);
> -        }
> -        else
> -            node = NUMA_NO_NODE;
> -
> -        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
> -        break;
> -    }
> -
> -    case PHYSDEVOP_pci_device_remove: {
> -        struct physdev_pci_device dev;
> -
> -        ret = -EFAULT;
> -        if ( copy_from_guest(&dev, arg, 1) != 0 )
> -            break;
> -
> -        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
> -        break;
> -    }
> -
>      case PHYSDEVOP_prepare_msix:
>      case PHYSDEVOP_release_msix: {
>          struct physdev_pci_device dev;
> diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c
> index 0a50cbd4d8..5f72652ff7 100644
> --- a/xen/arch/x86/x86_64/physdev.c
> +++ b/xen/arch/x86/x86_64/physdev.c
> @@ -9,9 +9,10 @@ EMIT_FILE;
>  #include <compat/xen.h>
>  #include <compat/event_channel.h>
>  #include <compat/physdev.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>  
>  #define do_physdev_op compat_physdev_op
> +#define arch_physdev_op arch_compat_physdev_op
>  
>  #define physdev_apic               compat_physdev_apic
>  #define physdev_apic_t             physdev_apic_compat_t
> @@ -82,6 +83,7 @@ CHECK_physdev_pci_device
>  typedef int ret_t;
>  
>  #include "../physdev.c"
> +#include "../../../common/physdev.c"
>  
>  /*
>   * Local variables:
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 54de70d422..bcb1c8fb03 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -29,6 +29,7 @@ obj-y += notifier.o
>  obj-y += page_alloc.o
>  obj-$(CONFIG_HAS_PDX) += pdx.o
>  obj-$(CONFIG_PERF_COUNTERS) += perfc.o
> +obj-y += physdev.o
>  obj-y += preempt.o
>  obj-y += random.o
>  obj-y += rangeset.o
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> new file mode 100644
> index 0000000000..8d44b20db8
> --- /dev/null
> +++ b/xen/common/physdev.c
> @@ -0,0 +1,81 @@
> +
> +#include <xen/guest_access.h>
> +#include <xen/hypercall.h>
> +#include <xen/init.h>
> +
> +#ifndef COMPAT
> +typedef long ret_t;
> +#endif
> +
> +ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +{
> +    ret_t ret;
> +
> +    switch ( cmd )
> +    {
> +#ifdef CONFIG_HAS_PCI
> +    case PHYSDEVOP_pci_device_add: {
> +        struct physdev_pci_device_add add;
> +        struct pci_dev_info pdev_info;
> +        nodeid_t node;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&add, arg, 1) != 0 )
> +            break;
> +
> +        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
> +        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> +        {
> +            pdev_info.is_virtfn = 1;
> +            pdev_info.physfn.bus = add.physfn.bus;
> +            pdev_info.physfn.devfn = add.physfn.devfn;
> +        }
> +        else
> +            pdev_info.is_virtfn = 0;
> +
> +        if ( add.flags & XEN_PCI_DEV_PXM )
> +        {
> +            uint32_t pxm;
> +            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
> +                                sizeof(add.optarr[0]);
> +
> +            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> +                break;
> +
> +            node = pxm_to_node(pxm);
> +        }
> +        else
> +            node = NUMA_NO_NODE;
> +
> +        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
> +        break;
> +    }
> +
> +    case PHYSDEVOP_pci_device_remove: {
> +        struct physdev_pci_device dev;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&dev, arg, 1) != 0 )
> +            break;
> +
> +        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
> +        break;
> +    }
> +#endif
> +    default:
> +        ret = arch_physdev_op(cmd, arg);;
                                           ^ a typo?

The ARM and common parts are fine. I am not well-versed in the x86
compat stuff; we need one of the x86 maintainers to review the x86
changes.


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:07:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:07:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193155.344035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBK7-0005T5-Hy; Wed, 22 Sep 2021 23:06:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193155.344035; Wed, 22 Sep 2021 23:06:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBK7-0005Sy-Ee; Wed, 22 Sep 2021 23:06:43 +0000
Received: by outflank-mailman (input) for mailman id 193155;
 Wed, 22 Sep 2021 23:06:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTBK6-0005Ss-6M
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:06:42 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 640faf0a-35c6-4a92-ba47-a58f74c38b34;
 Wed, 22 Sep 2021 23:06:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 497C560FC1;
 Wed, 22 Sep 2021 23:06:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 640faf0a-35c6-4a92-ba47-a58f74c38b34
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632352000;
	bh=s3NrJKDfiEWGvShgVW8Mc4ZhvqhKhL+wxIimBi0pIo8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JhXqckojOeMqau34DdcdVg0yYGaoQ8XTrJRahO26YANLQBmtCb+s1f1/KoCefeWkY
	 aeXYyXfN7z/Iz+Xcgt1vSiKqNKpo3DYDUE2Zkq3OhKcMn2VhgesXnePVkGIRRqd5gh
	 6vUGhgnUarlqarfgsZvy3W6P0nCtQ+v3IZSyhFsLtPoh54aTbuS55LUPX/RSQzlmwv
	 9q+DgKt+/mKlCoMAaPnmc5zLD/FhQNwfujr62UKs7p3D4QDzQiYAcF/q0+Wb9bmlDN
	 fliE+WlBUfGxDh2I8zfcz7dz6VnLCj3EMKsWjIwJfUlxwg8F/iXUCGDq4CEMqdwG1Q
	 lRCX8cIs9NfBw==
Date: Wed, 22 Sep 2021 16:06:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v2 06/17] xen/device-tree: Add dt_property_read_variable_u32_array
 helper
In-Reply-To: <35eb7ec429da3a9e1af626af4521c13c9683f8fd.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221601240.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <35eb7ec429da3a9e1af626af4521c13c9683f8fd.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

I'd prefer the upstream commit id if possible, not one based on the
stable trees (the commit id doesn't resolve at all on Linus's tree).
Aside from that, the patch is fine.


> Import the Linux helper of_property_read_variable_u32_array. This
> function find and read an array of 32 bit integers from a property,
> with bounds on the minimum and maximum array size.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2: Patch introduced in v2
> ---
>  xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
>  xen/include/xen/device_tree.h | 26 +++++++++++++++
>  2 files changed, 87 insertions(+)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 03d25a81ce..53160d61f8 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_node *np,
>      return 0;
>  }
>  
> +/**
> + * dt_find_property_value_of_size
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @min:    minimum allowed length of property value
> + * @max:    maximum allowed length of property value (0 means unlimited)
> + * @len:    if !=NULL, actual length is written to here
> + *
> + * Search for a property in a device node and valid the requested size.
> + *
> + * Return: The property value on success, -EINVAL if the property does not
> + * exist, -ENODATA if property does not have a value, and -EOVERFLOW if the
> + * property data is too small or too large.
> + */
> +static void *dt_find_property_value_of_size(const struct dt_device_node *np,
> +                                            const char *propname, u32 min,
> +                                            u32 max, size_t *len)
> +{
> +    const struct dt_property *prop = dt_find_property(np, propname, NULL);
> +
> +    if ( !prop )
> +        return ERR_PTR(-EINVAL);
> +    if ( !prop->value )
> +        return ERR_PTR(-ENODATA);
> +    if ( prop->length < min )
> +        return ERR_PTR(-EOVERFLOW);
> +    if ( max && prop->length > max )
> +        return ERR_PTR(-EOVERFLOW);
> +
> +    if ( len )
> +        *len = prop->length;
> +
> +    return prop->value;
> +}
> +
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_values,
> +                                        size_t sz_min, size_t sz_max)
> +{
> +    size_t sz, count;
> +    const __be32 *val = dt_find_property_value_of_size(np, propname,
> +                        (sz_min * sizeof(*out_values)),
> +                        (sz_max * sizeof(*out_values)),
> +                        &sz);
> +
> +    if ( IS_ERR(val) )
> +        return PTR_ERR(val);
> +
> +    if ( !sz_max )
> +        sz = sz_min;
> +    else
> +        sz /= sizeof(*out_values);
> +
> +    count = sz;
> +    while ( count-- )
> +        *out_values++ = be32_to_cpup(val++);
> +
> +    return sz;
> +}
> +
>  int dt_property_match_string(const struct dt_device_node *np,
>                               const char *propname, const char *string)
>  {
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index b02696be94..1693fb8e8c 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
>  bool_t dt_property_read_u64(const struct dt_device_node *np,
>                              const char *name, u64 *out_value);
>  
> +
> +/**
> + * dt_property_read_variable_u32_array - Find and read an array of 32 bit
> + * integers from a property, with bounds on the minimum and maximum array size.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return found values.
> + * @sz_min: minimum number of array elements to read
> + * @sz_max: maximum number of array elements to read, if zero there is no
> + *      upper limit on the number of elements in the dts entry but only
> + *      sz_min will be read.
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: The number of elements read on success, -EINVAL if the property
> + * does not exist, -ENODATA if property does not have a value, and -EOVERFLOW
> + * if the property data is smaller than sz_min or longer than sz_max.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_values,
> +                                        size_t sz_min, size_t sz_max);
> +
>  /**
>   * dt_property_read_bool - Check if a property exists
>   * @np: node to get the value
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:21:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:21:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193161.344046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYa-00085N-Qz; Wed, 22 Sep 2021 23:21:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193161.344046; Wed, 22 Sep 2021 23:21:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYa-00085G-O7; Wed, 22 Sep 2021 23:21:40 +0000
Received: by outflank-mailman (input) for mailman id 193161;
 Wed, 22 Sep 2021 23:21:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTBYZ-000855-NI
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:21:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b3a0fff-eaee-44ed-89e8-1c05f0426f58;
 Wed, 22 Sep 2021 23:21:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 52F9C21EBA;
 Wed, 22 Sep 2021 16:15:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b3a0fff-eaee-44ed-89e8-1c05f0426f58
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327348;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DGnYGLYTZlt4UtXZbiZtPB9v0oVww6LTMwGhigIhC4w=;
	b=UPbC1fzbJrfdLTp7ZSQj2uQcWDXflOXpR/G4vVRGW9kdVFULzUF67xhe1kfuNYSz/JpOy1
	wbygKgvKMOyqx7pbbTbb6Y8z+9IiPW9zihN9nMV81AsEGbTgqsQrzORivur2m5ElixWh2H
	hKLkReYRjURA8IUJP7ZqkVTFDvxTEQs=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 03/16] qapi: Implement x-machine-init QMP command
Date: Wed, 22 Sep 2021 18:13:52 +0200
Message-Id: <20210922161405.140018-4-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

From: Mirela Grujic <mirela.grujic@greensocs.com>

The x-machine-init QMP command is available only if the -preconfig option
is used and the current machine initialization phase is accel-created.

The command triggers QEMU to enter machine initialized phase and wait
for the QMP configuration. In future commits, we will add the possiblity
to create devices at this point.

To exit the initialized phase use the x-exit-preconfig QMP command.

Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
---
 qapi/machine.json | 23 +++++++++++++++++++++++
 softmmu/vl.c      | 19 +++++++++++++++----
 2 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/qapi/machine.json b/qapi/machine.json
index 969d37fb03..56330c0e8e 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1368,3 +1368,26 @@
 ##
 { 'command': 'query-machine-phase', 'returns': 'MachineInitPhaseStatus',
              'allow-preconfig': true }
+
+##
+# @x-machine-init:
+#
+# Enter machine initialized phase
+#
+# Since: 6.2
+#
+# Returns: If successful, nothing
+#
+# Notes: This command will trigger QEMU to execute initialization steps
+#        that are required to enter the machine initialized phase. The command
+#        is available only if the -preconfig command line option was passed and
+#        if the machine is currently in the accel-created phase. To exit the
+#        machine initialized phase use the x-exit-preconfig command.
+#
+# Example:
+#
+# -> { "execute": "x-machine-init" }
+# <- { "return": {} }
+#
+##
+{ 'command': 'x-machine-init', 'allow-preconfig': true }
diff --git a/softmmu/vl.c b/softmmu/vl.c
index d2552ba8ac..84c5132ad7 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -123,6 +123,7 @@
 #include "qapi/qapi-visit-qom.h"
 #include "qapi/qapi-commands-ui.h"
 #include "qapi/qmp/qdict.h"
+#include "qapi/qapi-commands-machine.h"
 #include "qapi/qmp/qerror.h"
 #include "sysemu/iothread.h"
 #include "qemu/guest-random.h"
@@ -2610,10 +2611,16 @@ static void qemu_init_displays(void)
     }
 }
 
-static void qemu_init_board(void)
+void qmp_x_machine_init(Error **errp)
 {
     MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
 
+    if (phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
+        error_setg(errp, "The command is permitted only before "
+                         "the machine is initialized");
+        return;
+    }
+
     if (machine_class->default_ram_id && current_machine->ram_size &&
         numa_uses_legacy_mem() && !current_machine->ram_memdev_id) {
         create_default_memdev(current_machine, mem_path);
@@ -2692,12 +2699,16 @@ static void qemu_machine_creation_done(void)
 
 void qmp_x_exit_preconfig(Error **errp)
 {
-    if (phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
-        error_setg(errp, "The command is permitted only before machine initialization");
+    if (phase_check(MACHINE_INIT_PHASE_READY)) {
+        error_setg(errp, "The command is permitted only before "
+                         "the machine is ready");
         return;
     }
 
-    qemu_init_board();
+    if (!phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
+        qmp_x_machine_init(errp);
+    }
+
     qemu_create_cli_devices();
     qemu_machine_creation_done();
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:21:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:21:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193162.344050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYb-00088l-4U; Wed, 22 Sep 2021 23:21:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193162.344050; Wed, 22 Sep 2021 23:21:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYa-00087M-W6; Wed, 22 Sep 2021 23:21:40 +0000
Received: by outflank-mailman (input) for mailman id 193162;
 Wed, 22 Sep 2021 23:21:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTBYZ-000854-KW
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:21:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d5839110-1bfb-11ec-b9f2-12813bfff9fa;
 Wed, 22 Sep 2021 23:21:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id F1D0E21EC1;
 Wed, 22 Sep 2021 16:15:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5839110-1bfb-11ec-b9f2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327350;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jPWHcx8frG0Bxx16IuAMBta1shU2F7IxandpxY+0m3o=;
	b=R2JacUH5kcSgL3a0GBx59skiUZLwlJrwuwsPCJPKp/jfsGnSFIs+eudHgACISfwNN8yzup
	PCA4EJ5Gc/WoYe//HHjbUL42WxmqWvbtS6uTYjxX75b2H+nsza1Ny7Oyp1hqgAG32pAcXI
	W+J++3yJDsAwelYtREZpkVK9EkRGLEc=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 05/16] qdev-monitor: prevent conflicts between qmp/device_add and cli/-device
Date: Wed, 22 Sep 2021 18:13:54 +0200
Message-Id: <20210922161405.140018-6-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

This commit prepares to extend device_add qmp command when using
-preconfig option.

In order to avoid conflicts with the cli -device option handling, we
need to handle some special case with the QemuOpts.
The qemu_device_opts is traversed when switching from
MACHINE_INIT_PHASE_INITIALIZED to MACHINE_INIT_PHASE_READY in order
to create any device specified by cli -device. Until now any
device_add qmp command was issued after that point so there was no
problem.

If we execute the qmp command before the MACHINE_INIT_PHASE_READY
phase we need to discard the QemuOpts from the qemu_device_opts in
order to avoid the cli -device code to try to create the device
again.

This commit preserves the opts behavior regarding the devices added
in 'ready' phase by the QMP command device_add.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Although we keep the original behavior for QMP commands issued when
the machine is ready (only authorized case so far), we are not sure
it is necessary: keeping the opts in the list is not needed anymore
to ensure the id uniqueness of devices but it has the 2 following
consequences:

1. the device opts stay in the QemuOptsList. Is this list needed
   after traversing the device cli options?

2. the DeviceState "opts" field is set. Do we need to keep it after
   the device is realized ?

Any information on this will be appreciated.
---
 softmmu/qdev-monitor.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 0007698ff3..834f2b56b5 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -848,6 +848,23 @@ void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp)
     if (!dev) {
         qemu_opts_del(opts);
         return;
+    } else if (!phase_check(MACHINE_INIT_PHASE_READY)) {
+        /*
+         * Always delete the related opts in case the device was created
+         * before handling of cli -device arguments:
+         * We do not want a device added by the qmp command to be handled
+         * again by the cli -device creation code. This does not break
+         * the ID uniqueness because it is checked in qdev_device_add().
+         *
+         * Note: We check the phase in order to keep the legacy behavior:
+         * in the machine ready phase case, the QemuOpts remains in the list
+         * (and the dev->opts field is kept).
+         * If it happens it was done only to ensure the ID uniqueness and
+         * the QemuOpts is never used after this point: then we could
+         * remove QemuOpts in any phase.
+         */
+        dev->opts = NULL;
+        qemu_opts_del(opts);
     }
     object_unref(OBJECT(dev));
 }
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:21:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193163.344067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYg-0000Ei-Ct; Wed, 22 Sep 2021 23:21:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193163.344067; Wed, 22 Sep 2021 23:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYg-0000EZ-9c; Wed, 22 Sep 2021 23:21:46 +0000
Received: by outflank-mailman (input) for mailman id 193163;
 Wed, 22 Sep 2021 23:21:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTBYe-000854-GD
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:21:44 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d5839112-1bfb-11ec-b9f2-12813bfff9fa;
 Wed, 22 Sep 2021 23:21:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 3C58E21ECD;
 Wed, 22 Sep 2021 16:15:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5839112-1bfb-11ec-b9f2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327356;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=83vzCH47KASlUcuFQINQpuff64c+MWa6MS19Qg8Yxnk=;
	b=P7hcyu+pxVIWKnhY3fU1nC0/ApGUVDeZbSeiHxcpgkCZWg35zWms2xJ3anjKnSVUoSAD8c
	H+qh2umF75EUBiBBC7qMvIrQtuaCSddKC0OjRXsV4MQS+CTdyIeI2BmpDSKe3DXTKRIt0Q
	VCthFEhyqAq/QRJQzHUCvH2QWSOp70s=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 11/16] softmmu/memory: add memory_region_try_add_subregion function
Date: Wed, 22 Sep 2021 18:14:00 +0200
Message-Id: <20210922161405.140018-12-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

It allows to try to add a subregion to a memory region with error
handling. Like memory_region_add_subregion_overlap, it handles
priority as well.
Apart the error handling, the behavior is the same. It can be used
to do the simple memory_region_add_subregion() (with no overlap) by
setting the priority parameter to 0.

This commit is a preparation to further use this function in the
context of qmp command which needs error handling support.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Adding a new function is obviously not ideal. But there is ~900
occurrences of memory_region_add_subregion[_overlap] calls in the code
base. We do not really see an alternative here.
---
 include/exec/memory.h | 22 ++++++++++++++++++++++
 softmmu/memory.c      | 22 ++++++++++++++--------
 2 files changed, 36 insertions(+), 8 deletions(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index c3d417d317..422e1eda67 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -2162,6 +2162,28 @@ void memory_region_add_subregion_overlap(MemoryRegion *mr,
                                          MemoryRegion *subregion,
                                          int priority);
 
+/**
+ * memory_region_try_add_subregion: Add a subregion to a container
+ *                                  with error handling.
+ *
+ * Behaves like memory_region_add_subregion_overlap(), but errors are
+ * reported if the subregion cannot be added.
+ *
+ * @mr: the region to contain the new subregion; must be a container
+ *      initialized with memory_region_init().
+ * @offset: the offset relative to @mr where @subregion is added.
+ * @subregion: the subregion to be added.
+ * @priority: used for resolving overlaps; highest priority wins.
+ * @errp: pointer to Error*, to store an error if it happens.
+ *
+ * Returns: True in case of success, false otherwise.
+ */
+bool memory_region_try_add_subregion(MemoryRegion *mr,
+                                     hwaddr offset,
+                                     MemoryRegion *subregion,
+                                     int priority,
+                                     Error **errp);
+
 /**
  * memory_region_get_ram_addr: Get the ram address associated with a memory
  *                             region
diff --git a/softmmu/memory.c b/softmmu/memory.c
index bfedaf9c4d..eac61f8236 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -2513,22 +2513,28 @@ done:
     memory_region_transaction_commit();
 }
 
-static void memory_region_add_subregion_common(MemoryRegion *mr,
-                                               hwaddr offset,
-                                               MemoryRegion *subregion)
+bool memory_region_try_add_subregion(MemoryRegion *mr,
+                                     hwaddr offset,
+                                     MemoryRegion *subregion,
+                                     int priority,
+                                     Error **errp)
 {
-    assert(!subregion->container);
+    if (subregion->container) {
+        error_setg(errp, "The memory region is already in another region");
+        return false;
+    }
+    subregion->priority = priority;
     subregion->container = mr;
     subregion->addr = offset;
     memory_region_update_container_subregions(subregion);
+    return true;
 }
 
 void memory_region_add_subregion(MemoryRegion *mr,
                                  hwaddr offset,
                                  MemoryRegion *subregion)
 {
-    subregion->priority = 0;
-    memory_region_add_subregion_common(mr, offset, subregion);
+    memory_region_try_add_subregion(mr, offset, subregion, 0, &error_abort);
 }
 
 void memory_region_add_subregion_overlap(MemoryRegion *mr,
@@ -2536,8 +2542,8 @@ void memory_region_add_subregion_overlap(MemoryRegion *mr,
                                          MemoryRegion *subregion,
                                          int priority)
 {
-    subregion->priority = priority;
-    memory_region_add_subregion_common(mr, offset, subregion);
+    memory_region_try_add_subregion(mr, offset, subregion, priority,
+                                    &error_abort);
 }
 
 void memory_region_del_subregion(MemoryRegion *mr,
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:21:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193164.344073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYg-0000Hx-Pp; Wed, 22 Sep 2021 23:21:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193164.344073; Wed, 22 Sep 2021 23:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBYg-0000Gu-Hv; Wed, 22 Sep 2021 23:21:46 +0000
Received: by outflank-mailman (input) for mailman id 193164;
 Wed, 22 Sep 2021 23:21:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=D6HN=OM=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTBYe-000855-JW
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:21:44 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 469b1cce-97d8-456d-bb97-7342932611b2;
 Wed, 22 Sep 2021 23:21:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id B455E21ED2;
 Wed, 22 Sep 2021 16:15:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 469b1cce-97d8-456d-bb97-7342932611b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327360;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cfebBfH6GVdfSl+a3M3Z0yLytuwrCU0s61tJ+ZXVGyA=;
	b=4+57fmZZrqqfiMZEPQV/HtYezQjVslO9aiAmXDY9Q9U4w+pBohuF9d/PLrpAJIwjoBNO49
	4DN2/DjmOHhL4T4VsHSwLEPNnD/2srj0nVTR7h5YqWSOh3iImVQU3JbPlA8mSUE2Ei2E1r
	K/SqfsawzVu+ASUu//lxvgY7OHHBmXc=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 13/16] hw/mem/system-memory: add a memory sysbus device
Date: Wed, 22 Sep 2021 18:14:02 +0200
Message-Id: <20210922161405.140018-14-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

This device can be used to create some memories using standard
device_add qmp command.

This device has one property 'readonly' which allows
to choose between a ram or a rom.
The device holds the adequate memory region and can be put on
the sysbus.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Should we add a related CONFIG_ variable in the build system ?

Depending on the chosen condition to add a device, the commit may
change.
---
 include/hw/mem/sysbus-memory.h | 32 +++++++++++++
 hw/mem/sysbus-memory.c         | 83 ++++++++++++++++++++++++++++++++++
 hw/mem/meson.build             |  2 +
 3 files changed, 117 insertions(+)
 create mode 100644 include/hw/mem/sysbus-memory.h
 create mode 100644 hw/mem/sysbus-memory.c

diff --git a/include/hw/mem/sysbus-memory.h b/include/hw/mem/sysbus-memory.h
new file mode 100644
index 0000000000..3e1271dbfd
--- /dev/null
+++ b/include/hw/mem/sysbus-memory.h
@@ -0,0 +1,32 @@
+/*
+ * QEMU memory SysBusDevice
+ *
+ * Copyright (c) 2021 Greensocs
+ *
+ * Author:
+ * + Damien Hedde
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef HW_SYSBUS_MEMORY_H
+#define HW_SYSBUS_MEMORY_H
+
+#include "hw/sysbus.h"
+#include "qom/object.h"
+
+#define TYPE_SYSBUS_MEMORY "sysbus-memory"
+OBJECT_DECLARE_SIMPLE_TYPE(SysBusMemoryState, SYSBUS_MEMORY)
+
+struct SysBusMemoryState {
+    /* <private> */
+    SysBusDevice parent_obj;
+    uint64_t size;
+    bool readonly;
+
+    /* <public> */
+    MemoryRegion mem;
+};
+
+#endif /* HW_SYSBUS_MEMORY_H */
diff --git a/hw/mem/sysbus-memory.c b/hw/mem/sysbus-memory.c
new file mode 100644
index 0000000000..897fa154f0
--- /dev/null
+++ b/hw/mem/sysbus-memory.c
@@ -0,0 +1,83 @@
+/*
+ * QEMU memory SysBusDevice
+ *
+ * Copyright (c) 2021 Greensocs
+ *
+ * Author:
+ * + Damien Hedde
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/mem/sysbus-memory.h"
+#include "hw/qdev-properties.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "qapi/error.h"
+
+static Property sysbus_memory_properties[] = {
+    DEFINE_PROP_UINT64("size", SysBusMemoryState, size, 0),
+    DEFINE_PROP_BOOL("readonly", SysBusMemoryState, readonly, false),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void sysbus_memory_realize(DeviceState *dev, Error **errp)
+{
+    SysBusMemoryState *s = SYSBUS_MEMORY(dev);
+    gchar *name;
+
+    if (!s->size) {
+        error_setg(errp, "'size' must be non-zero.");
+        return;
+    }
+
+    /*
+     * We impose having an id (which is unique) because we need to generate
+     * a unique name for the memory region.
+     * memory_region_init_ram/rom() will abort() (in qemu_ram_set_idstr()
+     * function if 2 system-memory devices are created with the same name
+     * for the memory region).
+     */
+    if (!dev->id) {
+        error_setg(errp, "system-memory device must have an id.");
+        return;
+    }
+    name = g_strdup_printf("%s.region", dev->id);
+
+    if (s->readonly) {
+        memory_region_init_rom(&s->mem, OBJECT(dev), name, s->size, errp);
+    } else {
+        memory_region_init_ram(&s->mem, OBJECT(dev), name, s->size, errp);
+    }
+
+    g_free(name);
+
+    if (!*errp) {
+        sysbus_init_mmio(SYS_BUS_DEVICE(s), &s->mem);
+    }
+}
+
+static void sysbus_memory_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->user_creatable = true;
+    dc->realize = sysbus_memory_realize;
+    device_class_set_props(dc, sysbus_memory_properties);
+}
+
+static const TypeInfo sysbus_memory_info = {
+    .name          = TYPE_SYSBUS_MEMORY,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(SysBusMemoryState),
+    .class_init    = sysbus_memory_class_init,
+};
+
+static void sysbus_memory_register_types(void)
+{
+    type_register_static(&sysbus_memory_info);
+}
+
+type_init(sysbus_memory_register_types)
diff --git a/hw/mem/meson.build b/hw/mem/meson.build
index 3c8fdef9f9..81e2de1d34 100644
--- a/hw/mem/meson.build
+++ b/hw/mem/meson.build
@@ -7,3 +7,5 @@ mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c'))
 softmmu_ss.add_all(when: 'CONFIG_MEM_DEVICE', if_true: mem_ss)
 
 softmmu_ss.add(when: 'CONFIG_FUZZ', if_true: files('sparse-mem.c'))
+
+softmmu_ss.add(files('sysbus-memory.c'))
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:44:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:44:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193187.344090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBut-0004Ez-Qs; Wed, 22 Sep 2021 23:44:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193187.344090; Wed, 22 Sep 2021 23:44:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTBut-0004Es-NE; Wed, 22 Sep 2021 23:44:43 +0000
Received: by outflank-mailman (input) for mailman id 193187;
 Wed, 22 Sep 2021 23:44:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTBut-0004Em-3Q
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:44:43 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6fa8aa3c-b181-4f1f-83f6-68f45072fc17;
 Wed, 22 Sep 2021 23:44:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D5A8860F6B;
 Wed, 22 Sep 2021 23:44:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fa8aa3c-b181-4f1f-83f6-68f45072fc17
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632354281;
	bh=2yTFJatK4dcvuO+FzHRx0aWMeKEun7B2AeCN7fYwW5c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=j5haLBwlTJPO7TKipXfqXqk5XFbBow2zopf9SDZmVV4v4X9SHMv3GmzlEO1adn2zS
	 l2gwFV26rbTH5lJ6uLD/QB+S3S28lKvTkn7Bjm6cga/40ixYDJoXzfLKRw5jaPI2uu
	 oSQ3+hqxDF0jTGZRQIIIfsQBkwEXRyaynl7rP7ImpSme/V0RpLSFqjXoCxLc8gc2WW
	 KNubZ5w+SF3CsRQkzdtyPPmjhEfkf0RH6nlx9ZAO8Cp10nGAzwvam+SO5wqKY/pVgC
	 gerr7f1Gvxj+FGnlZAA07V149/NAAfQIwKtZcm1A/ZRFZFO84+zhbcN4D7sxqVzrGu
	 ONanYY9Mp2/Fg==
Date: Wed, 22 Sep 2021 16:44:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v2 07/17] xen/device-tree: Add dt_property_read_u32_array
 helper
In-Reply-To: <852610e258d4c36888b68771061afdc0df95e401.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221644150.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <852610e258d4c36888b68771061afdc0df95e401.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

Please use a commit it from Linux upstream, not stable


> Import the Linux helper of_property_read_u32_array. This function find
> and read an array of 32 bit integers from a property.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2: Patch introduced in v2
> ---
>  xen/include/xen/device_tree.h | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 1693fb8e8c..497144b8a7 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -366,7 +366,6 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
>  bool_t dt_property_read_u64(const struct dt_device_node *np,
>                              const char *name, u64 *out_value);
>  
> -
>  /**
>   * dt_property_read_variable_u32_array - Find and read an array of 32 bit
>   * integers from a property, with bounds on the minimum and maximum array size.

Spurious change


> @@ -392,6 +391,36 @@ int dt_property_read_variable_u32_array(const struct dt_device_node *np,
>                                          const char *propname, u32 *out_values,
>                                          size_t sz_min, size_t sz_max);
>  
> +/**
> + * dt_property_read_u32_array - Find and read an array of 32 bit integers
> + * from a property.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return value, modified only if return value is 0.
> + * @sz:     number of array elements to read
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: 0 on success, -EINVAL if the property does not exist,
> + * -ENODATA if property does not have a value, and -EOVERFLOW if the
> + * property data isn't large enough.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +static inline int dt_property_read_u32_array(const struct dt_device_node *np,
> +                                             const char *propname,
> +                                             u32 *out_values, size_t sz)
> +{
> +    int ret = dt_property_read_variable_u32_array(np, propname, out_values,
> +                              sz, 0);
> +    if ( ret >= 0 )
> +        return 0;
> +    else
> +        return ret;
> +}
> +
>  /**
>   * dt_property_read_bool - Check if a property exists
>   * @np: node to get the value
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 22 23:50:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 22 Sep 2021 23:50:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193193.344101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTC0J-0005p6-Fe; Wed, 22 Sep 2021 23:50:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193193.344101; Wed, 22 Sep 2021 23:50:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTC0J-0005oz-Bz; Wed, 22 Sep 2021 23:50:19 +0000
Received: by outflank-mailman (input) for mailman id 193193;
 Wed, 22 Sep 2021 23:50:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ilSY=OM=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTC0I-0005ot-5u
 for xen-devel@lists.xenproject.org; Wed, 22 Sep 2021 23:50:18 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d6409e82-1bff-11ec-b9f3-12813bfff9fa;
 Wed, 22 Sep 2021 23:50:17 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2EECE61131;
 Wed, 22 Sep 2021 23:50:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6409e82-1bff-11ec-b9f3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632354616;
	bh=zJ3WjaxN2JEd2pJJPs4Gnnct7HIdWR52lI2GSPP+WLE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=oJSmAMxa7gzi9zCa0oLEOKeUvFaJRnGSedKeD8Wwl6kMSUv6FadQZrM1J3iL5YGU4
	 hM4/iLN+I/JWkYBJ46Lig3jnskWa84rKnquadlZildMSG7RnvKEqqnEGWJwmAYwjjt
	 /o69tZt09JZIhh7qLVRfoqDpz7n2KdAquzTDvGZ6cPU+KN9SSBoUTbib0jZI+pOyFn
	 7E+Bj/6yzMa2vDcGIy3uLyAugoqNOzk4I0bH+Flg8lHVca0T4m/qJq5uU68C+aXPrP
	 Y12ujGBaFb/a0h1D8Iaqe3V4jyPj3cLI5IInKw/B9+dSeaDndmO6/hw4eako76baJM
	 xB+WSnUH5EMLg==
Date: Wed, 22 Sep 2021 16:50:14 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v2 08/17] xen/device-tree: Add dt_get_pci_domain_nr
 helper
In-Reply-To: <b649f9f978ed38f05927573381e23f9b3c6c24b8.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221649260.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <b649f9f978ed38f05927573381e23f9b3c6c24b8.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc

Please use an upstream commit id


> Import the Linux helper of_get_pci_domain_nr. This function will try to
> obtain the host bridge domain number by finding a property called
> "linux,pci-domain" of the given device node.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2: Patch introduced in v2
> ---
>  xen/common/device_tree.c      | 12 ++++++++++++
>  xen/include/xen/device_tree.h | 19 +++++++++++++++++++
>  2 files changed, 31 insertions(+)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 53160d61f8..ea93da1725 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
>      dt_alias_scan();
>  }
>  
> +int dt_get_pci_domain_nr(struct dt_device_node *node)
> +{
> +    u32 domain;
> +    int error;
> +
> +    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
> +    if ( !error )
> +        return -EINVAL;
> +
> +    return (u16)domain;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 497144b8a7..9069040ef7 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -831,6 +831,25 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
>                                 const char *list_name,
>                                 const char *cells_name);
>  
> +/**
> + * dt_get_pci_domain_nr - Find the host bridge domain number
> + *            of the given device node.
> + * @node: Device tree node with the domain information.
> + *
> + * This function will try to obtain the host bridge domain number by finding
> + * a property called "linux,pci-domain" of the given device node.
> + *
> + * Return:
> + * * > 0    - On success, an associated domain number.
> + * * -EINVAL    - The property "linux,pci-domain" does not exist.
> + * * -ENODATA   - The linux,pci-domain" property does not have value.
> + * * -EOVERFLOW - Invalid "linux,pci-domain" property value.

This doesn't match reality for us because it can only return EINVAL


> + * Returns the associated domain number from DT in the range [0-0xffff], or
> + * a negative value if the required property is not found.
> + */
> +int dt_get_pci_domain_nr(struct dt_device_node *node);
> +
>  #ifdef CONFIG_DEVICE_TREE_DEBUG
>  #define dt_dprintk(fmt, args...)  \
>      printk(XENLOG_DEBUG fmt, ## args)
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:03:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:03:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193199.344112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCDG-000875-0Q; Thu, 23 Sep 2021 00:03:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193199.344112; Thu, 23 Sep 2021 00:03:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCDF-00086y-T7; Thu, 23 Sep 2021 00:03:41 +0000
Received: by outflank-mailman (input) for mailman id 193199;
 Thu, 23 Sep 2021 00:03:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTCDE-00086s-CR
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:03:40 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d9371d51-c5af-40e7-b69b-4a642f969a0e;
 Thu, 23 Sep 2021 00:03:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4FE1E61040;
 Thu, 23 Sep 2021 00:03:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9371d51-c5af-40e7-b69b-4a642f969a0e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632355418;
	bh=0gLP7gQB9I4DdUjrcPOD3CPCp+uA/6+96tfHuOreojo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=irvutLhBTJuUOXs0e9g3SKNOuhiL+xgz0qC90rSc5wmL8chu0uddOMYuJzphwab5t
	 hgwH87KbhfguJQZew+GdTsfCLix9X/e3eZNkrPhiL1K4FuKenFIJGccLpFgU6ju2lu
	 aBz06swGzrU9jhGJkL5hf+nmxs9mzt7+Mc7Z31N9CHlMqIMfBLghQuAakKoLCiAdnF
	 elgEsrxsnQ0n8pyl1TAy8VZJirxaH1k/8Bwr3j9aR8X0hM1uAIcaOKEUKFIOoMzvDA
	 5+QdCKfNnyxwWxi2mBS66iuJ/krs06qK1OMtMTGqBcUgx/otrlhdDDPaIovwlHlkll
	 RjSooh4wWyaDw==
Date: Wed, 22 Sep 2021 17:03:37 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
In-Reply-To: <000832623dc7fb429db4b4517583f16affdba35b.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221657080.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <000832623dc7fb429db4b4517583f16affdba35b.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> pci_init(..) will be called during xen startup to initialize and probe
> the PCI host-bridge driver.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - ACPI init function to return int
> - pci_segments_init() called before dt/acpi init
> ---
>  xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/device.h |  1 +
>  2 files changed, 55 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index a7a7bc3213..71fa532842 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -12,6 +12,10 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>  
> +#include <xen/acpi.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
>  #include <xen/pci.h>
>  
>  /*
> @@ -22,6 +26,56 @@ int arch_pci_clean_pirqs(struct domain *d)
>      return 0;
>  }
>  
> +static int __init dt_pci_init(void)
> +{
> +    struct dt_device_node *np;
> +    int rc;
> +
> +    dt_for_each_device_node(dt_host, np)
> +    {
> +        rc = device_init(np, DEVICE_PCI, NULL);
> +        if( !rc )
> +            continue;

This is a NIT, so feel free to code it as you prefer, but I would write
it as follows for simplicity:


/* comment about EBADF and ENODEV */
if ( !rc || rc == -EBADF || rc == -ENODEV )
    continue;
return rc;


> +        /*
> +         * Ignore the following error codes:
> +         *   - EBADF: Indicate the current is not an pci
                                             ^ device ^a   ^ device


> +         *   - ENODEV: The pci device is not present or cannot be used by
> +         *     Xen.
> +         */
> +        else if ( rc != -EBADF && rc != -ENODEV )
> +        {
> +            printk(XENLOG_ERR "No driver found in XEN or driver init error.\n");
> +            return rc;
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +#ifdef CONFIG_ACPI
> +static int __init acpi_pci_init(void)
> +{
> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
> +    return 0;

Should return ENOSYS or EOPNOTSUPP?


> +}
> +#else
> +static inline int __init acpi_pci_init(void)

Not sure I would inline it but OK either way


> +{
> +    return -EINVAL;
> +}
> +#endif
> +
> +static int __init pci_init(void)
> +{
> +    pci_segments_init();
> +
> +    if ( acpi_disabled )
> +        return dt_pci_init();
> +    else
> +        return acpi_pci_init();
> +}
> +__initcall(pci_init);
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index ee7cff2d44..5ecd5e7bd1 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -34,6 +34,7 @@ enum device_class
>      DEVICE_SERIAL,
>      DEVICE_IOMMU,
>      DEVICE_GIC,
> +    DEVICE_PCI,
>      /* Use for error */
>      DEVICE_UNKNOWN,
>  };
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:14:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:14:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193207.344123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCNr-0001L4-1g; Thu, 23 Sep 2021 00:14:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193207.344123; Thu, 23 Sep 2021 00:14:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCNq-0001Kx-V2; Thu, 23 Sep 2021 00:14:38 +0000
Received: by outflank-mailman (input) for mailman id 193207;
 Thu, 23 Sep 2021 00:14:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTCNp-0001Kr-ND
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:14:37 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3bf8a6b8-1c03-11ec-b9f4-12813bfff9fa;
 Thu, 23 Sep 2021 00:14:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1A1C361211;
 Thu, 23 Sep 2021 00:14:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bf8a6b8-1c03-11ec-b9f4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632356075;
	bh=5PXi0iXkZLIcUn9AG/uHf3EMU3TwdNwyR6t8Sv7IOT4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=MBzblnQRB8ygBo1Y1+AiNKO7ggD+eahXnCAVl1sKayoklZUo3a3EmKPrTLbtd4lgW
	 jKVDE+uyTSZqSKTJCBnPcqB5b1R8CIrRPxF7+tfGYWh7lCvQXQp3At9gF/QvvS7QB/
	 k2qbxf0pcowcXCcwFu87ggRPGXFIiYTAmr782aCcgVf+4hX4tfuhhLcF2DUL49wHZQ
	 CT8xhyKZF5DfNFAFKDQ5UAvU0YFbAN5o0FzZBthUjR2bzY/CH2nrotgvS1Y8OEfubu
	 HwQM+Fr4Z4xY7gMNaqci7PnLWxOwh+SA01eRszNdejHye0A7oT/sLEVrmuC4FKOSqa
	 uGfqYvwma1qog==
Date: Wed, 22 Sep 2021 17:14:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 10/17] xen/arm: Add cmdline boot option "pci-passthrough
 = <boolean>"
In-Reply-To: <9dcb9b3b6b6923db00d6e56da26a8503d5a4855a.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221708500.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <9dcb9b3b6b6923db00d6e56da26a8503d5a4855a.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> Add cmdline boot option "pci-passthrough = = <boolean>" to enable
> disable the PCI passthrough support on ARM.
 ^ or disable

> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Add option in xen-command-line.pandoc
> - Change pci option to pci-passthrough
> - modify option from custom_param to boolean param
> ---
>  docs/misc/xen-command-line.pandoc |  7 +++++++
>  xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
>  xen/common/physdev.c              |  6 ++++++
>  xen/include/asm-arm/pci.h         | 13 +++++++++++++
>  xen/include/asm-x86/pci.h         |  8 ++++++++
>  5 files changed, 48 insertions(+)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index b175645fde..c867f1cf58 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1783,6 +1783,13 @@ All numbers specified must be hexadecimal ones.
>  
>  This option can be specified more than once (up to 8 times at present).
>  
> +### pci-passthrough (arm)
> +> `= <boolean>`
> +
> +> Default: `false`
> +
> +Flag to enable or disable support for PCI passthrough
> +
>  ### pcid (x86)
>  > `= <boolean> | xpti=<bool>`
>  
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index 71fa532842..fe96a9b135 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -16,6 +16,7 @@
>  #include <xen/device_tree.h>
>  #include <xen/errno.h>
>  #include <xen/init.h>
> +#include <xen/param.h>
>  #include <xen/pci.h>
>  
>  /*
> @@ -65,8 +66,21 @@ static inline int __init acpi_pci_init(void)
>  }
>  #endif
>  
> +/*
> + * By default pci passthrough is disabled
> + */
> +bool_t __read_mostly pci_passthrough_enabled = 0;

I think we are using bool rather than bool_t nowadays. Also could do =
false.


> +boolean_param("pci-passthrough", pci_passthrough_enabled);
> +
>  static int __init pci_init(void)
>  {
> +    /*
> +     * Enable PCI passthrough when has been enabled explicitly
> +     * (pci-passthrough=on)
> +     */
> +    if ( !pci_passthrough_enabled)

missing space


> +        return 0;
> +
>      pci_segments_init();
>  
>      if ( acpi_disabled )
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> index 8d44b20db8..7390d5d584 100644
> --- a/xen/common/physdev.c
> +++ b/xen/common/physdev.c
> @@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          struct pci_dev_info pdev_info;
>          nodeid_t node;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>          ret = -EFAULT;
>          if ( copy_from_guest(&add, arg, 1) != 0 )
>              break;
> @@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_pci_device_remove: {
>          struct physdev_pci_device dev;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>          ret = -EFAULT;
>          if ( copy_from_guest(&dev, arg, 1) != 0 )
>              break;
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 7dd9eb3dba..f2f86be9bc 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -19,14 +19,27 @@
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>  
> +extern bool_t pci_passthrough_enabled;
> +
>  /* Arch pci dev struct */
>  struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return pci_passthrough_enabled;
> +}
>  #else   /*!CONFIG_HAS_PCI*/
>  
> +#define pci_passthrough_enabled (false)

We don't need to define it, do we?


>  struct arch_pci_dev { };
>  
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return false;
> +}
> +
>  #endif  /*!CONFIG_HAS_PCI*/
>  #endif /* __ARM_PCI_H__ */
> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
> index cc05045e9c..0e160c6d01 100644
> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned int *seg,
>  extern int pci_mmcfg_config_num;
>  extern struct acpi_mcfg_allocation *pci_mmcfg_config;
>  
> +/*
> + * Unlike ARM, PCI passthrough always enabled for x86.
                                 ^ is


> + */
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return true;
> +}
> +
>  #endif /* __X86_PCI_H__ */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:31:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:31:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193212.344133 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeL-0003xg-GQ; Thu, 23 Sep 2021 00:31:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193212.344133; Thu, 23 Sep 2021 00:31:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeL-0003xZ-Da; Thu, 23 Sep 2021 00:31:41 +0000
Received: by outflank-mailman (input) for mailman id 193212;
 Thu, 23 Sep 2021 00:31:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTCeJ-0003xN-Tn
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:31:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9ce9660e-1c05-11ec-b9f4-12813bfff9fa;
 Thu, 23 Sep 2021 00:31:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 615DE21EC9;
 Wed, 22 Sep 2021 16:15:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ce9660e-1c05-11ec-b9f4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327354;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PDCjVxFBi4vz75aeR9pd9AbmoW6STuPuMbNBE/FSM7I=;
	b=ICuAAVCXm0R5uk9SjJ8PoZaJwYoy4CElxlYLonWdbRGOal6uenKQyWFhRGYtaMcpNGrB6U
	a6eNGZVWTngTm/nJTjQ651qUUGJcEfSF62QKzWyQ8J+LWw+rY7XTtY9hl/Kg9JWezPiQT2
	mj0ltS02PcDOghYVnG1qEo0ZJHBn8H4=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 09/16] hw/core/machine: Remove the dynamic sysbus devices type check
Date: Wed, 22 Sep 2021 18:13:58 +0200
Message-Id: <20210922161405.140018-10-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Now that we check sysbus device types during device creation, we
can remove the check done in the machine init done notifier.
This was the only thing done by this notifier, so we remove the
whole sysbus_notifier structure of the MachineState.

Note: This notifier was checking all /peripheral and /peripheral-anon
sysbus devices. Now we only check those added by -device cli option or
device_add qmp command when handling the command/option. So if there
are some devices added in one of these containers manually (eg in
machine C code), these will not be checked anymore.
This use case does not seem to appear apart from
hw/xen/xen-legacy-backend.c (it uses qdev_set_id() and in this case,
not for a sysbus device, so it's ok).

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---
 include/hw/boards.h |  1 -
 hw/core/machine.c   | 27 ---------------------------
 2 files changed, 28 deletions(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 934443c1cd..ccbc40355a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -311,7 +311,6 @@ typedef struct CpuTopology {
 struct MachineState {
     /*< private >*/
     Object parent_obj;
-    Notifier sysbus_notifier;
 
     /*< public >*/
 
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 1a18912dc8..521438e90a 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -571,18 +571,6 @@ bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
     return allowed;
 }
 
-static void validate_sysbus_device(SysBusDevice *sbdev, void *opaque)
-{
-    MachineState *machine = opaque;
-    MachineClass *mc = MACHINE_GET_CLASS(machine);
-
-    if (!device_is_dynamic_sysbus(mc, DEVICE(sbdev))) {
-        error_report("Option '-device %s' cannot be handled by this machine",
-                     object_class_get_name(object_get_class(OBJECT(sbdev))));
-        exit(1);
-    }
-}
-
 static char *machine_get_memdev(Object *obj, Error **errp)
 {
     MachineState *ms = MACHINE(obj);
@@ -598,17 +586,6 @@ static void machine_set_memdev(Object *obj, const char *value, Error **errp)
     ms->ram_memdev_id = g_strdup(value);
 }
 
-static void machine_init_notify(Notifier *notifier, void *data)
-{
-    MachineState *machine = MACHINE(qdev_get_machine());
-
-    /*
-     * Loop through all dynamically created sysbus devices and check if they are
-     * all allowed.  If a device is not allowed, error out.
-     */
-    foreach_dynamic_sysbus_device(validate_sysbus_device, machine);
-}
-
 HotpluggableCPUList *machine_query_hotpluggable_cpus(MachineState *machine)
 {
     int i;
@@ -1030,10 +1007,6 @@ static void machine_initfn(Object *obj)
                                         "Table (HMAT)");
     }
 
-    /* Register notifier when init is done for sysbus sanity checks */
-    ms->sysbus_notifier.notify = machine_init_notify;
-    qemu_add_machine_init_done_notifier(&ms->sysbus_notifier);
-
     /* default to mc->default_cpus */
     ms->smp.cpus = mc->default_cpus;
     ms->smp.max_cpus = mc->default_cpus;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:31:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:31:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193213.344140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeL-00041B-SI; Thu, 23 Sep 2021 00:31:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193213.344140; Thu, 23 Sep 2021 00:31:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeL-00040J-M2; Thu, 23 Sep 2021 00:31:41 +0000
Received: by outflank-mailman (input) for mailman id 193213;
 Thu, 23 Sep 2021 00:31:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTCeK-0003xO-8a
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:31:40 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 333be660-3c1a-4c30-818c-689c9aa597fb;
 Thu, 23 Sep 2021 00:31:39 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 29DC821ECF;
 Wed, 22 Sep 2021 16:15:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 333be660-3c1a-4c30-818c-689c9aa597fb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327359;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YFwTgaEumzc2XGbjK4hpWpdj4pZUzdb1skxz4Vmg8i8=;
	b=J+Wmqk4a8kuzHCMVCeIGgJjNaAhGIf80oYCh+/fvuoSlwd8WukKbcCqGi6wVcYUIS4uQKG
	/a8dur7Nk0YA6/FfoYgSVEqf/P0POAC01x1RLhg1ZUWOh0CgUNQ4C0nG1dSJsSu+dGdi1N
	f1rziCvvT/vfiWxVDVTEbOMcSei2Y0c=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 12/16] add x-sysbus-mmio-map qmp command
Date: Wed, 22 Sep 2021 18:14:01 +0200
Message-Id: <20210922161405.140018-13-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

This command allows to map an mmio region of sysbus device onto
the system memory. Its behavior mimics the sysbus_mmio_map()
function apart from the automatic unmap (the C function unmaps
the region if it is already mapped).
For the qmp function we consider it is an error to try to map
an already mapped function. If unmapping is required, it is
probably better to add a sysbus-mmip-unmap function.

This command is still experimental (hence the x prefix), as it
is related to the sysbus device creation through qmp commands.

In future, we expect to have to handle the overlap/priority
parameter but also multiple mapping of one mmio. For some
devices, one mmio is mapped several times at different addresses on
the bus (which is not supported by sysbus_mmio_map() function and
requires the use of memory region aliases).

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

Note: this qmp command is required to be able to build a machine from
scratch as there is no qmp-way of doing a memory mapping today.

We've added the command into qapi/qdev.json section. It does not seem to
have any really adequate section yet. Any idea ? should we create for
example a new one: qapi/sysbus.json or qapi/memory.json ?
---
 qapi/qdev.json   | 21 +++++++++++++++++++++
 hw/core/sysbus.c | 41 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)

diff --git a/qapi/qdev.json b/qapi/qdev.json
index ad669ae175..dfc1104197 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -125,3 +125,24 @@
 ##
 { 'event': 'DEVICE_DELETED',
   'data': { '*device': 'str', 'path': 'str' } }
+
+##
+# @x-sysbus-mmio-map:
+#
+# Map a sysbus device mmio onto the main system bus.
+#
+# @device: the device's QOM path
+#
+# @mmio: The mmio number to be mapped (defaults to 0).
+#
+# @addr: The base address for the mapping.
+#
+# Since: 6.2
+#
+# Returns: Nothing on success
+#
+##
+
+{ 'command': 'x-sysbus-mmio-map',
+  'data': {'device': 'str', '*mmio': 'uint8', 'addr': 'uint64'},
+  'allow-preconfig' : true }
diff --git a/hw/core/sysbus.c b/hw/core/sysbus.c
index aaae8e23cc..b0891f37b6 100644
--- a/hw/core/sysbus.c
+++ b/hw/core/sysbus.c
@@ -23,6 +23,7 @@
 #include "hw/sysbus.h"
 #include "monitor/monitor.h"
 #include "exec/address-spaces.h"
+#include "qapi/qapi-commands-qdev.h"
 
 static void sysbus_dev_print(Monitor *mon, DeviceState *dev, int indent);
 static char *sysbus_get_fw_dev_path(DeviceState *dev);
@@ -154,6 +155,46 @@ static void sysbus_mmio_map_common(SysBusDevice *dev, int n, hwaddr addr,
     }
 }
 
+void qmp_x_sysbus_mmio_map(const char *device, bool has_mmio, uint8_t mmio,
+                           uint64_t addr, Error **errp)
+{
+    Object *obj = object_resolve_path_type(device, TYPE_SYS_BUS_DEVICE, NULL);
+    SysBusDevice *dev;
+
+    if (phase_get() != MACHINE_INIT_PHASE_INITIALIZED) {
+        error_setg(errp, "The command is permitted only when "
+                         "the machine is in initialized phase");
+        return;
+    }
+
+    if (obj == NULL) {
+        error_setg(errp, "Device '%s' not found", device);
+        return;
+    }
+
+    dev = SYS_BUS_DEVICE(obj);
+    if (!has_mmio) {
+        mmio = 0;
+    }
+    if (mmio >= dev->num_mmio) {
+        error_setg(errp, "MMIO index '%u' is out of range", mmio);
+        return;
+    }
+
+    if (dev->mmio[mmio].addr != (hwaddr)-1) {
+        error_setg(errp, "MMIO index '%u' is already mapped", mmio);
+        return;
+    }
+
+    if (!memory_region_try_add_subregion(get_system_memory(), addr,
+                                         dev->mmio[mmio].memory, 0,
+                                         errp)) {
+        return;
+    }
+
+    dev->mmio[mmio].addr = addr;
+}
+
 void sysbus_mmio_unmap(SysBusDevice *dev, int n)
 {
     assert(n >= 0 && n < dev->num_mmio);
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:31:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:31:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193214.344155 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeP-0004Va-76; Thu, 23 Sep 2021 00:31:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193214.344155; Thu, 23 Sep 2021 00:31:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeP-0004VR-3d; Thu, 23 Sep 2021 00:31:45 +0000
Received: by outflank-mailman (input) for mailman id 193214;
 Thu, 23 Sep 2021 00:31:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTCeO-0003xN-Mb
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:31:44 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9ce81cae-1c05-11ec-b9f4-12813bfff9fa;
 Thu, 23 Sep 2021 00:31:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 6B2F521EB7;
 Wed, 22 Sep 2021 16:15:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ce81cae-1c05-11ec-b9f4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327347;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uO+rxeU8HZHEUI7sNPCnX4N20yxgThgHilFwN01tQgY=;
	b=4nwd0mKYOh88Bq4/k8c2ah+UaLuLdmXU2GInH88kJxYAg7EbL+YtSjZJK2c1dGQBJKmftg
	4vAVxgxK4VmkfrqkOGJ0zDbZ2nOewcMfN23UNSDTJ+uvKx1YBbYpqTbVYD/e/IUguDE62+
	uTYcsOtlMDJL+LNvRXe+uEKGS6FD+Ew=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 02/16] qapi: Implement query-machine-phase QMP command
Date: Wed, 22 Sep 2021 18:13:51 +0200
Message-Id: <20210922161405.140018-3-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

From: Mirela Grujic <mirela.grujic@greensocs.com>

The command returns current machine initialization phase.
>From now on, the MachineInitPhase enum is generated from the
QAPI schema.

Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
---
 qapi/machine.json          | 56 ++++++++++++++++++++++++++++++++++++++
 include/hw/qdev-core.h     | 30 ++------------------
 hw/core/machine-qmp-cmds.c |  9 ++++++
 hw/core/qdev.c             |  5 ++++
 4 files changed, 72 insertions(+), 28 deletions(-)

diff --git a/qapi/machine.json b/qapi/machine.json
index 157712f006..969d37fb03 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1312,3 +1312,59 @@
      '*cores': 'int',
      '*threads': 'int',
      '*maxcpus': 'int' } }
+
+##
+# @MachineInitPhase:
+#
+# Enumeration of machine initialization phases.
+#
+# @no-machine: machine does not exist
+#
+# @machine-created: machine is created, but its accelerator is not
+#
+# @accel-created: accelerator is created, but the machine properties have not
+#                 been validated and machine initialization is not done yet
+#
+# @initialized: machine is initialized, thus creating any embedded devices and
+#               validating machine properties. Devices created at this time are
+#               considered to be cold-plugged.
+#
+# @ready: QEMU is ready to start CPUs and devices created at this time are
+#         considered to be hot-plugged. The monitor is not restricted to
+#         "preconfig" commands.
+#
+# Since: 6.2
+##
+{ 'enum': 'MachineInitPhase',
+  'data': [ 'no-machine', 'machine-created', 'accel-created', 'initialized',
+            'ready' ] }
+
+##
+# @MachineInitPhaseStatus:
+#
+# Information about machine initialization phase
+#
+# @phase: the machine initialization phase
+#
+# Since: 6.2
+##
+{ 'struct': 'MachineInitPhaseStatus',
+  'data': { 'phase': 'MachineInitPhase' } }
+
+##
+# @query-machine-phase:
+#
+# Return machine initialization phase
+#
+# Since: 6.2
+#
+# Returns: MachineInitPhaseStatus
+#
+# Example:
+#
+# -> { "execute": "query-machine-phase" }
+# <- { "return": { "phase": "initialized" } }
+#
+##
+{ 'command': 'query-machine-phase', 'returns': 'MachineInitPhaseStatus',
+             'allow-preconfig': true }
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 859fd913bb..800eda8f54 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -1,6 +1,7 @@
 #ifndef QDEV_CORE_H
 #define QDEV_CORE_H
 
+#include "qapi/qapi-types-machine.h"
 #include "qemu/queue.h"
 #include "qemu/bitmap.h"
 #include "qemu/rcu.h"
@@ -839,35 +840,8 @@ void device_listener_unregister(DeviceListener *listener);
  */
 bool qdev_should_hide_device(QemuOpts *opts);
 
-typedef enum MachineInitPhase {
-    /* current_machine is NULL.  */
-    MACHINE_INIT_PHASE_NO_MACHINE,
-
-    /* current_machine is not NULL, but current_machine->accel is NULL.  */
-    MACHINE_INIT_PHASE_MACHINE_CREATED,
-
-    /*
-     * current_machine->accel is not NULL, but the machine properties have
-     * not been validated and machine_class->init has not yet been called.
-     */
-    MACHINE_INIT_PHASE_ACCEL_CREATED,
-
-    /*
-     * machine_class->init has been called, thus creating any embedded
-     * devices and validating machine properties.  Devices created at
-     * this time are considered to be cold-plugged.
-     */
-    MACHINE_INIT_PHASE_INITIALIZED,
-
-    /*
-     * QEMU is ready to start CPUs and devices created at this time
-     * are considered to be hot-plugged.  The monitor is not restricted
-     * to "preconfig" commands.
-     */
-    MACHINE_INIT_PHASE_READY,
-} MachineInitPhase;
-
 extern bool phase_check(MachineInitPhase phase);
 extern void phase_advance(MachineInitPhase phase);
+extern MachineInitPhase phase_get(void);
 
 #endif
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 52168a3771..d3b9a04855 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -204,3 +204,12 @@ MemdevList *qmp_query_memdev(Error **errp)
     object_child_foreach(obj, query_memdev, &list);
     return list;
 }
+
+MachineInitPhaseStatus *qmp_query_machine_phase(Error **errp)
+{
+    MachineInitPhaseStatus *status = g_malloc0(sizeof(*status));
+
+    status->phase = phase_get();
+
+    return status;
+}
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index c5fc704f55..d83f1c029a 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -1150,6 +1150,11 @@ void phase_advance(MachineInitPhase phase)
     machine_phase = phase;
 }
 
+MachineInitPhase phase_get(void)
+{
+    return machine_phase;
+}
+
 static const TypeInfo device_type_info = {
     .name = TYPE_DEVICE,
     .parent = TYPE_OBJECT,
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 00:31:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 00:31:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193215.344167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeQ-0004nW-Hj; Thu, 23 Sep 2021 00:31:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193215.344167; Thu, 23 Sep 2021 00:31:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTCeQ-0004n4-CN; Thu, 23 Sep 2021 00:31:46 +0000
Received: by outflank-mailman (input) for mailman id 193215;
 Thu, 23 Sep 2021 00:31:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTCeP-0003xO-6B
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 00:31:45 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f1eaeec4-e2e9-4cc1-87ac-df7a2884e1e9;
 Thu, 23 Sep 2021 00:31:38 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 16B1021EBD;
 Wed, 22 Sep 2021 16:15:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1eaeec4-e2e9-4cc1-87ac-df7a2884e1e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327349;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XKa7YpvCaHNP7B/Gx81xxfSEDEFxmlko5ypNT6F1RPs=;
	b=wtOmf1TGRi5+LsCZ7dTaWkdafLDUIQXjxY7cZKMSJPwbDnsncS8770qaxVmTeU8tVxHdsy
	98gXhs3KUKW1OrVqW4NQVrvc+Q9klOB56ADRcupW0/lzsumJ9QGaU1grDQEcXmiK54ZKOs
	5X4IsCXiqP+1fadiUoWHXRzy0Q+v+Wo=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 04/16] softmmu/qdev-monitor: add error handling in qdev_set_id
Date: Wed, 22 Sep 2021 18:13:53 +0200
Message-Id: <20210922161405.140018-5-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

qdev_set_id() is mostly used when the user adds a device (using
-device cli option or device_add qmp command). This commit adds
an error parameter to handle the case where the given id is
already taken.

Also document the function and add a return value in order to
be able to capture success/failure: the function now returns the
id in case of success, or NULL in case of failure.

The commit modifies the 2 calling places (qdev-monitor and
xen-legacy-backend) to add the error object parameter.

Note that the id is, right now, guaranteed to be unique because
all ids came from the "device" QemuOptsList where the id is used
as key. This addition is a preparation for a future commit which
will relax the uniqueness.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---
 include/monitor/qdev.h      | 25 +++++++++++++++++++++++-
 hw/xen/xen-legacy-backend.c |  3 ++-
 softmmu/qdev-monitor.c      | 38 +++++++++++++++++++++++++++----------
 3 files changed, 54 insertions(+), 12 deletions(-)

diff --git a/include/monitor/qdev.h b/include/monitor/qdev.h
index eaa947d73a..23c31f5296 100644
--- a/include/monitor/qdev.h
+++ b/include/monitor/qdev.h
@@ -9,6 +9,29 @@ void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp);
 
 int qdev_device_help(QemuOpts *opts);
 DeviceState *qdev_device_add(QemuOpts *opts, Error **errp);
-void qdev_set_id(DeviceState *dev, const char *id);
+
+/**
+ * qdev_set_id: parent the device and set its id if provided.
+ * @dev: device to handle
+ * @id: id to be given to the device, or NULL.
+ *
+ * Returns: the id of the device in case of success; otherwise NULL.
+ *
+ * @dev must be unrealized, unparented and must not have an id.
+ *
+ * If @id is non-NULL, this function tries to setup @dev qom path as
+ * "/peripheral/id". If @id is already taken, it fails. If it succeeds,
+ * the id field of @dev is set to @id (@dev now owns the given @id
+ * parameter).
+ *
+ * If @id is NULL, this function generates a unique name and setups @dev
+ * qom path as "/peripheral-anon/name". This name is not set as the id
+ * of @dev.
+ *
+ * Upon success, it returns the id/name (generated or provided). The
+ * returned string is owned by the corresponding child property and must
+ * not be freed by the caller.
+ */
+const char *qdev_set_id(DeviceState *dev, const char *id, Error **errp);
 
 #endif
diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index dd8ae1452d..f541cfa0e9 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -276,7 +276,8 @@ static struct XenLegacyDevice *xen_be_get_xendev(const char *type, int dom,
     xendev = g_malloc0(ops->size);
     object_initialize(&xendev->qdev, ops->size, TYPE_XENBACKEND);
     OBJECT(xendev)->free = g_free;
-    qdev_set_id(DEVICE(xendev), g_strdup_printf("xen-%s-%d", type, dev));
+    qdev_set_id(DEVICE(xendev), g_strdup_printf("xen-%s-%d", type, dev),
+                &error_fatal);
     qdev_realize(DEVICE(xendev), xen_sysbus, &error_fatal);
     object_unref(OBJECT(xendev));
 
diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 25275984bd..0007698ff3 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -578,22 +578,34 @@ static BusState *qbus_find(const char *path, Error **errp)
     return bus;
 }
 
-void qdev_set_id(DeviceState *dev, const char *id)
+const char *qdev_set_id(DeviceState *dev, const char *id, Error **errp)
 {
+    ObjectProperty *prop;
+
+    assert(!dev->id && !dev->realized);
+
+    /*
+     * object_property_[try_]add_child() below will assert the device
+     * has no parent
+     */
     if (id) {
-        dev->id = id;
-    }
-
-    if (dev->id) {
-        object_property_add_child(qdev_get_peripheral(), dev->id,
-                                  OBJECT(dev));
+        prop = object_property_try_add_child(qdev_get_peripheral(), id,
+                                             OBJECT(dev), NULL);
+        if (prop) {
+            dev->id = id;
+        } else {
+            error_setg(errp, "Duplicate device ID '%s'", id);
+            return NULL;
+        }
     } else {
         static int anon_count;
         gchar *name = g_strdup_printf("device[%d]", anon_count++);
-        object_property_add_child(qdev_get_peripheral_anon(), name,
-                                  OBJECT(dev));
+        prop = object_property_add_child(qdev_get_peripheral_anon(), name,
+                                         OBJECT(dev));
         g_free(name);
     }
+
+    return prop->name;
 }
 
 DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
@@ -677,7 +689,13 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **errp)
         }
     }
 
-    qdev_set_id(dev, qemu_opts_id(opts));
+    /*
+     * set dev's parent and register its id.
+     * If it fails it means the id is already taken.
+     */
+    if (!qdev_set_id(dev, qemu_opts_id(opts), errp)) {
+        goto err_del_dev;
+    }
 
     /* set properties */
     if (qemu_opt_foreach(opts, set_property, dev, errp)) {
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 01:10:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 01:10:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193234.344178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDFd-0000Rh-Gd; Thu, 23 Sep 2021 01:10:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193234.344178; Thu, 23 Sep 2021 01:10:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDFd-0000Ra-Dh; Thu, 23 Sep 2021 01:10:13 +0000
Received: by outflank-mailman (input) for mailman id 193234;
 Thu, 23 Sep 2021 01:10:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTDFc-0000RU-3C
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 01:10:12 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff9a8972-1c0a-11ec-b9fc-12813bfff9fa;
 Thu, 23 Sep 2021 01:10:10 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A9786109E;
 Thu, 23 Sep 2021 01:10:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff9a8972-1c0a-11ec-b9fc-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632359410;
	bh=2mcxG55qKaKMA20YKNot3P1eLyODJ/0MAZ99tHgpG5s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=lyexlZ4lBUwrFW3Vk4IwygVIaPjezGV33BVlzg07ifZdrh64pBdnyw8eXB0DVRMrq
	 91287bNVrTt1Y5DGQI7IG+UMgIA1m8QyvQDyD6n8Hm3y74voh3WIgHH4CySqDCVoOx
	 Yfnw8XrPRRWiYm9VK5LrInMvvf/G2551X9CLA+piz0j2eoHPiaFg/7GEcroqHx7+KJ
	 3q+3okObUkKY64YZI8sMaNxjoiaLCPSy0GjWfU3V+PB2pI7B1AHrYErWuZJGSdpyHu
	 +e+iYou6dD/ZLTrff3xdzix2YH316rK9LBTG5qt/j3hy6xDybZOc1BDqc4xxwp94C/
	 X/V6nP1M8E2AA==
Date: Wed, 22 Sep 2021 18:10:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Ian Jackson <iwj@xenproject.org>, xen-devel@lists.xenproject.org, 
    dpsmith@apertussolutions.com
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
Message-ID: <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
References: <osstest-164996-mainreport@xen.org> <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com> <24904.44119.940679.241639@mariner.uk.xensource.com> <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Jan Beulich wrote:
> On 22.09.2021 01:38, Stefano Stabellini wrote:
> > On Mon, 20 Sep 2021, Ian Jackson wrote:
> >> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> >>> As per
> >>>
> >>> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
> >>> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
> >>> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
> >>> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
> >>> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
> >>> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
> >>> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
> >>>
> >>> the system doesn't look to really be out of memory; as per
> >>>
> >>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
> >>>
> >>> there even look to be a number of higher order pages available (albeit
> >>> without digging I can't tell what "(C)" means). Nevertheless order-4
> >>> allocations aren't really nice.
> >>
> >> The host history suggests this may possibly be related to a qemu update.
> >>
> >> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
> 
> Stefano - as per some of your investigation detailed further down I
> wonder whether you had seen this part of Ian's reply. (Question of
> course then is how that qemu update had managed to get pushed.)
> 
> >> The grub cfg has this:
> >>
> >>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
> >>
> >> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".
> > 
> > I definitely recommend to increase dom0 memory, especially as I guess
> > the box is going to have a significant amount, far more than 4GB. I
> > would set it to 2GB. Also the syntax on ARM is simpler, so it should be
> > just: dom0_mem=2G
> 
> Ian - I guess that's an adjustment relatively easy to make? I wonder
> though whether we wouldn't want to address the underlying issue first.
> Presumably not, because the fix would likely take quite some time to
> propagate suitably. Yet if not, we will want to have some way of
> verifying that an eventual fix there would have helped here.
> 
> > In addition, I also did some investigation just in case there is
> > actually a bug in the code and it is not a simple OOM problem.
> 
> I think the actual issue is quite clear; what I'm struggling with is
> why we weren't hit by it earlier.
> 
> As imo always, non-order-0 allocations (perhaps excluding the bringing
> up of the kernel or whichever entity) are to be avoided it at possible.
> The offender in this case looks to be privcmd's alloc_empty_pages().
> For it to request through kcalloc() what ends up being an order-4
> allocation, the original IOCTL_PRIVCMD_MMAPBATCH must specify a pretty
> large chunk of guest memory to get mapped. Which may in turn be
> questionable, but I'm afraid I don't have the time to try to drill
> down where that request is coming from and whether that also wouldn't
> better be split up.
> 
> The solution looks simple enough - convert from kcalloc() to kvcalloc().
> I can certainly spin up a patch to Linux to this effect. Yet that still
> won't answer the question of why this issue has popped up all of the
> sudden (and hence whether there are things wanting changing elsewhere
> as well).

Also, I saw your patches for Linux. Let's say that the patches are
reviewed and enqueued immediately to be sent to Linus at the next
opportunity. It is going to take a while for them to take effect in
OSSTest, unless we import them somehow in the Linux tree used by OSSTest
straight away, right?

Should we arrange for one test OSSTest flight now with the patches
applied to see if they actually fix the issue? Otherwise we might end up
waiting for nothing...


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 01:38:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 01:38:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193241.344189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDge-0003Tl-SC; Thu, 23 Sep 2021 01:38:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193241.344189; Thu, 23 Sep 2021 01:38:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDge-0003Td-MT; Thu, 23 Sep 2021 01:38:08 +0000
Received: by outflank-mailman (input) for mailman id 193241;
 Thu, 23 Sep 2021 01:38:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTDgd-0003TT-KK; Thu, 23 Sep 2021 01:38:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTDgd-0003D6-En; Thu, 23 Sep 2021 01:38:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTDgc-000682-UV; Thu, 23 Sep 2021 01:38:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTDgc-0005uO-U2; Thu, 23 Sep 2021 01:38:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XbOreYHHtuZEhftQYfIueU9zKoblyBCvwwKcokOV5Sc=; b=lEq0V7GLUkCT+ijURmlj06F8Na
	/P+obAPZEiYZzDKSBkVFR5beQKLgmXrVy889yXxNGVxJIMnD2bLSSRX3rnTUamNpB5soXwqiflsut
	G1CCzrRfcb6g9CHFsq/imia3hFi5GkNFK5tlq02nCa6PfAgvLfx6IJL8DW6cipx6goAk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165153-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165153: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=81ceb36b965c9d5ed5b1eb0ed80e23705802de15
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 01:38:06 +0000

flight 165153 qemu-mainline real [real]
flight 165159 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165153/
http://logs.test-lab.xenproject.org/osstest/logs/165159/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                81ceb36b965c9d5ed5b1eb0ed80e23705802de15
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   11 days
Failing since        164967  2021-09-13 13:06:52 Z    9 days   18 attempts
Testing same since   165153  2021-09-22 12:08:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Andrew Jeffery <andrew@aj.id.au>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4240 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 01:41:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 01:41:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193247.344208 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDk7-0004u3-NY; Thu, 23 Sep 2021 01:41:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193247.344208; Thu, 23 Sep 2021 01:41:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDk7-0004tZ-I6; Thu, 23 Sep 2021 01:41:43 +0000
Received: by outflank-mailman (input) for mailman id 193247;
 Thu, 23 Sep 2021 01:41:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTDk4-0004qi-R7
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 01:41:41 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 64ff03ca-1c0f-11ec-ba03-12813bfff9fa;
 Thu, 23 Sep 2021 01:41:39 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 4B88C21EC3;
 Wed, 22 Sep 2021 16:15:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64ff03ca-1c0f-11ec-ba03-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327352;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pOou7ntZwf6jxxWFcKHppVKqRu0dT3fZc00S/uH+MyA=;
	b=5bk6PS2OqJmvZzhaI9CocH6KgDJLjQ0sST8dAep+XnkbKqazJzdfin3fRbNcoCFIIJ4LDG
	GoegIeSg8nXubLp4NmRiWtEy9I5diF4x3MSZmpTdsP6bl5Lp7T8uGdnf7mcsqHi+kAZIk3
	qGWjcdZLReG9SyIDa9sXiR3hURGJtcE=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 06/16] qapi: Allow device_add to execute in machine initialized phase
Date: Wed, 22 Sep 2021 18:13:55 +0200
Message-Id: <20210922161405.140018-7-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

From: Mirela Grujic <mirela.grujic@greensocs.com>

To configure a machine using QMP we need the device_add command to
execute at machine initialized phase.

Note: for device_add command in qdev.json adding the 'allow-init-config'
option has no effect because the command appears to bypass QAPI (see
TODO at qapi/qdev.json:61). The option is added there solely to document
the intent.
For the same reason, the flags have to be explicitly set in
monitor_init_qmp_commands() when the device_add command is registered.

Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
---

The commit is fine, but we may add intermediate commits before this one
in order to add or change the condition for a device type to be accepted
in the 'initialized' state (see the cover-letter of the series).
---
 qapi/qdev.json         | 3 ++-
 monitor/misc.c         | 2 +-
 softmmu/qdev-monitor.c | 6 ++++++
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/qapi/qdev.json b/qapi/qdev.json
index b83178220b..ad669ae175 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -67,7 +67,8 @@
 ##
 { 'command': 'device_add',
   'data': {'driver': 'str', '*bus': 'str', '*id': 'str'},
-  'gen': false } # so we can get the additional arguments
+  'gen': false, # so we can get the additional arguments
+  'allow-preconfig': true }
 
 ##
 # @device_del:
diff --git a/monitor/misc.c b/monitor/misc.c
index ffe7966870..2c476de316 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -231,7 +231,7 @@ static void monitor_init_qmp_commands(void)
     qmp_init_marshal(&qmp_commands);
 
     qmp_register_command(&qmp_commands, "device_add", qmp_device_add,
-                         QCO_NO_OPTIONS);
+                         QCO_ALLOW_PRECONFIG);
 
     QTAILQ_INIT(&qmp_cap_negotiation_commands);
     qmp_register_command(&qmp_cap_negotiation_commands, "qmp_capabilities",
diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 834f2b56b5..47ccd90be8 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -824,6 +824,12 @@ void qmp_device_add(QDict *qdict, QObject **ret_data, Error **errp)
     QemuOpts *opts;
     DeviceState *dev;
 
+    if (!phase_check(MACHINE_INIT_PHASE_INITIALIZED)) {
+        error_setg(errp, "The command is permitted only after "
+                         "the machine is initialized");
+        return;
+    }
+
     opts = qemu_opts_from_qdict(qemu_find_opts("device"), qdict, errp);
     if (!opts) {
         return;
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 01:41:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 01:41:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193248.344203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDk7-0004r0-Dj; Thu, 23 Sep 2021 01:41:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193248.344203; Thu, 23 Sep 2021 01:41:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTDk7-0004qt-A2; Thu, 23 Sep 2021 01:41:43 +0000
Received: by outflank-mailman (input) for mailman id 193248;
 Thu, 23 Sep 2021 01:41:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTDk4-0004qh-Nh
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 01:41:41 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b5d6653f-99af-4080-bc90-d4366a303960;
 Thu, 23 Sep 2021 01:41:39 +0000 (UTC)
Received: from crumble.bar.greensocs.com (unknown [172.17.10.6])
 by beetle.greensocs.com (Postfix) with ESMTPS id 4E73021ECB;
 Wed, 22 Sep 2021 16:15:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5d6653f-99af-4080-bc90-d4366a303960
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632327355;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ow5aMrqN/i5D/dv1vm+J5tz7Z/sAyFJH5Io1/i7HEmw=;
	b=cHbJePzpcvHXQEkqOdmaeWcRNrJ837qUy9Smnwsf4FkYlw79QNCGGyVPvnpY0eWxfDYkrY
	Gcyfi8h/ljklnuDrFdhj+SC0Q9suuCMFH6X6PkgUXB/DYK4rXoUWqJXJJmrOrjy7EPcEAj
	CWsJ/7O0ROaTwhJk7Qw6cuIAJX3Xrb8=
From: Damien Hedde <damien.hedde@greensocs.com>
To: qemu-devel@nongnu.org
Cc: Damien Hedde <damien.hedde@greensocs.com>,
	Alistair Francis <Alistair.Francis@wdc.com>,
	=?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Ani Sinha <ani@anisinha.ca>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Eric Auger <eric.auger@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Peter Xu <peterx@redhat.com>,
	David Hildenbrand <david@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Eric Blake <eblake@redhat.com>,
	qemu-riscv@nongnu.org,
	xen-devel@lists.xenproject.org,
	mark.burton@greensocs.com,
	mirela.grujic@greensocs.com,
	edgari@xilinx.com,
	Peter Maydell <peter.maydell@linaro.org>
Subject: [RFC PATCH v2 10/16] qdev-monitor: allow adding any sysbus device before machine is ready
Date: Wed, 22 Sep 2021 18:13:59 +0200
Message-Id: <20210922161405.140018-11-damien.hedde@greensocs.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210922161405.140018-1-damien.hedde@greensocs.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam: Yes

Skip the sysbus device type per-machine allow-list check before the
MACHINE_INIT_PHASE_READY phase.

This patch permits adding any sysbus device (it still needs to be
user_creatable) when using the -preconfig experimental option.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---

This commit is RFC. Depending on the condition to allow a device
to be added, it may change.
---
 softmmu/qdev-monitor.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index f1c9242855..73b991adda 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -269,8 +269,13 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
         return NULL;
     }
 
-    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
-        /* sysbus devices need to be allowed by the machine */
+    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE) &&
+        phase_check(MACHINE_INIT_PHASE_READY)) {
+        /*
+         * Sysbus devices need to be allowed by the machine.
+         * We only check that after the machine is ready in order to let
+         * us add any user_creatable sysbus device during machine creation.
+         */
         MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
         if (!machine_class_is_dynamic_sysbus_dev_allowed(mc, *driver)) {
             error_setg(errp, "'%s' is not an allowed pluggable sysbus device "
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 01:59:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 01:59:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193260.344224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTE0q-0007SN-Aj; Thu, 23 Sep 2021 01:59:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193260.344224; Thu, 23 Sep 2021 01:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTE0q-0007SG-7j; Thu, 23 Sep 2021 01:59:00 +0000
Received: by outflank-mailman (input) for mailman id 193260;
 Thu, 23 Sep 2021 01:58:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTE0o-0007S6-SU; Thu, 23 Sep 2021 01:58:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTE0o-0003YK-KG; Thu, 23 Sep 2021 01:58:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTE0o-0006wE-AQ; Thu, 23 Sep 2021 01:58:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTE0o-0007eG-9v; Thu, 23 Sep 2021 01:58:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2zTk2DoU8abXDE3jdXIXiP6dCQUfwISwcQIRwXFby7E=; b=mxVmd5FG2/RMaP+G6BNmKb1usT
	HKR9FRjfQLJhaYidiHmwQ+RPJpbU1uoBLFFYF553YCFedobOb8x6HYmrqJeladEaWHawrSzqi2V/I
	DskizJrJf7MgKNDyxkH4nH8ZpTWvm42U+eC++J5Iw6GiUgLdG18OxZE/fi9RnxBPgcAE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165152-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 165152: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=07e5f23d3fa6ca98457d1a2177a735fcc65923c2
X-Osstest-Versions-That:
    linux=48a24510c328b3b3d7775377494b4ad4f58d189a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 01:58:58 +0000

flight 165152 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165152/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 165021
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 165021
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 165021
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 165021
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 165021
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 165021
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 165021
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 165021
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 165021
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 165021
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 165021
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 165021
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                07e5f23d3fa6ca98457d1a2177a735fcc65923c2
baseline version:
 linux                48a24510c328b3b3d7775377494b4ad4f58d189a

Last test of basis   165021  2021-09-17 02:50:54 Z    5 days
Testing same since   165152  2021-09-22 10:44:14 Z    0 days    1 attempts

------------------------------------------------------------
328 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   48a24510c328..07e5f23d3fa6  07e5f23d3fa6ca98457d1a2177a735fcc65923c2 -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:07:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193266.344239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTE92-00011e-6T; Thu, 23 Sep 2021 02:07:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193266.344239; Thu, 23 Sep 2021 02:07:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTE92-00011X-3U; Thu, 23 Sep 2021 02:07:28 +0000
Received: by outflank-mailman (input) for mailman id 193266;
 Thu, 23 Sep 2021 02:07:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nnny=ON=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mTE90-00011R-SF
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:07:26 +0000
Received: from mail-ed1-x52f.google.com (unknown [2a00:1450:4864:20::52f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fc4d5898-ac11-42db-af2f-d7a0c487c853;
 Thu, 23 Sep 2021 02:07:25 +0000 (UTC)
Received: by mail-ed1-x52f.google.com with SMTP id y89so6797781ede.2
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 19:07:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc4d5898-ac11-42db-af2f-d7a0c487c853
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=cdt1FGOHqROmUnm/Tq+/DCK4YuTmqt2i34HvBS17N7w=;
        b=Tk8VTmS5FNlQpN6WIjCAkZ0wHSkkHGxyGTdy8w+0R8nxGHGVQM78BrLmOLfb3Jpo2N
         Ip49RSH8Zf2rX89whOuQ1XnE6j869JZW+UVDc7/MRgqP9Az5L4VSuR/JDpR3PT5/e3HN
         83mcTMUry2rWN4i137e/6ZWyJD0uxuE43lE2cqzM2K5dXDefT152YvZzTzdACG2k7gIh
         HnzgxNWO9yJ/7fHffgQF2z2CYd3rTJvO/oGA6Os7w7G6dXtTfvMSBmgLmiIQc4RWwrag
         y2hTDQkoRRgK+mI7eNA991Zk0rJOwNUcXZzWSJFU5oxFna449U03xEavFLfs0665oxiW
         vpLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=cdt1FGOHqROmUnm/Tq+/DCK4YuTmqt2i34HvBS17N7w=;
        b=6T746zjrX1TxLs+nlQ3pr5wd8uFPTTOfSr6ea8IQWkld4wGzi4ktW4g6O8DSsD3LH0
         wpLLAj/lLBID8bKHI25JwWi1tRX4pOIOCE9iwtp/u8ccd/aS2YLRu8sYSar3CNHTcAdV
         8ggJPe95WD9b66oA06AsypD46eU+AVQ1zlBRzMds2YbLcnK9Itqhxc9m7qY2KRMqMf4E
         l+39xQBndDIxzENClBlAOStuBJftKrz7zfmGUHVnVCyCKXThoXd/BmWV2Wgdfj+etnBk
         P8+c+EsYSNe59gEbs5ykiEyHzpao2Enz2SgawTiXOlNTJqwtSFqJ0FZLjrXZi+TeXC64
         LByA==
X-Gm-Message-State: AOAM532zZ4iKAtku0K348eucJvGuLByB4te3XNgRcAxclTzhVEPNAbUc
	M/G5OZtYQvz/AcWYoJ6ip6N6atG1KulVgZjIN3o=
X-Google-Smtp-Source: ABdhPJzTciH6flz/xE6K5fQlnGD/ZOmH1G2TM81wrTKkKCLDKt8EqIzKsjMk9gIJ/SFviH7ch1/sieFFjoN7Jys61N8=
X-Received: by 2002:a50:cf41:: with SMTP id d1mr2707716edk.219.1632362843954;
 Wed, 22 Sep 2021 19:07:23 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1632307952.git.rahul.singh@arm.com> <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
In-Reply-To: <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Thu, 23 Sep 2021 07:07:12 +0500
Message-ID: <CAJ=z9a32OsxxMbPnOEO4cEJweiBkF61U0niZDX2Z77gv+-zAQQ@mail.gmail.com>
Subject: Re: [PATCH v2 02/17] xen/pci: solve compilation error on ARM with
 HAS_PCI enabled
To: Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Andre Przywara <andre.przywara@arm.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Type: multipart/alternative; boundary="00000000000075605e05cca017ec"

--00000000000075605e05cca017ec
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

On Wed, 22 Sep 2021, 16:36 Rahul Singh, <rahul.singh@arm.com> wrote:

> Compilation error is observed when HAS_PCI is enabled for ARM
> architecture.


In general, when I read "compilation error" I interpret as a user can
trigger it in the current staging.

However, without the full series applied, HAS_PCI cannot be selected on Arm=
.

So I think the message is a bit misleading because this is more
implementing stubs to enable a feature rather than fixing compilation
errors (AFAICT all of them are not fixed here).

How about the following commit message:

xen/arm: pci: Add stubs to allow selecting HAS_PCI

In a follow-up we will enable PCI support in Xen on Arm (i.e select
HAS_PCI).

The generic code expects the arch to implement a few functions:

<List the functions>

Note that this is not yet sufficient to enable HAS_PCI and will be
addressed in follow-ups.


> Add definition for arch_iommu_use_permitted() and
> arch_pci_clean_pirqs().
>
> pci.c: In function =E2=80=98deassign_device=E2=80=99:
> pci.c:849:49: error: implicit declaration of function =E2=80=98pci_to_dev=
=E2=80=99;
> did you mean =E2=80=98dt_to_dev=E2=80=99? [-Werror=3Dimplicit-function-de=
claration]
>             pci_to_dev(pdev));
> pci.c:880: undefined reference to `arch_pci_clean_pirqs=E2=80=99
> pci.c:1392: undefined reference to `arch_iommu_use_permitted'
>
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Remove pci_conf_read*(..) dummy implementation
> - Add in code comment for arch_pci_clean_pirqs() and
> arch_iommu_use_permitted()
> - Fixed minor comments
> ---
>  xen/arch/arm/Makefile               |  1 +
>  xen/arch/arm/pci/Makefile           |  1 +
>  xen/arch/arm/pci/pci.c              | 33 +++++++++++++++++++++++++++++
>  xen/drivers/passthrough/arm/iommu.c |  9 ++++++++
>  xen/include/asm-arm/pci.h           | 31 ++++++++++++++++++++++++---
>  5 files changed, 72 insertions(+), 3 deletions(-)
>  create mode 100644 xen/arch/arm/pci/Makefile
>  create mode 100644 xen/arch/arm/pci/pci.c
>
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b4..44d7cc81fa 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) +=3D arm32/
>  obj-$(CONFIG_ARM_64) +=3D arm64/
>  obj-$(CONFIG_ARM_64) +=3D efi/
>  obj-$(CONFIG_ACPI) +=3D acpi/
> +obj-$(CONFIG_HAS_PCI) +=3D pci/
>  ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y +=3D platforms/
>  endif
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> new file mode 100644
> index 0000000000..a98035df4c
> --- /dev/null
> +++ b/xen/arch/arm/pci/Makefile
> @@ -0,0 +1 @@
> +obj-y +=3D pci.o
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> new file mode 100644
> index 0000000000..a7a7bc3213
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci.c
> @@ -0,0 +1,33 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +
> +/*
> + * PIRQ event channels are not supported on Arm, so nothing to do.
> + */
> +int arch_pci_clean_pirqs(struct domain *d)
> +{
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/arm/iommu.c
> b/xen/drivers/passthrough/arm/iommu.c
> index db3b07a571..ee653a9c48 100644
> --- a/xen/drivers/passthrough/arm/iommu.c
> +++ b/xen/drivers/passthrough/arm/iommu.c
> @@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)
>  void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  {
>  }
> +
> +/*
> + * Unlike x86, Arm doesn't support mem-sharing, mem-paging and log-dirty
> (yet).
> + * So there is no restriction to use the IOMMU.
> + */
> +bool arch_iommu_use_permitted(const struct domain *d)
> +{
> +    return true;
> +}
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index de13359f65..7dd9eb3dba 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -1,7 +1,32 @@
> -#ifndef __X86_PCI_H__
> -#define __X86_PCI_H__
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
>
> +#ifndef __ARM_PCI_H__
> +#define __ARM_PCI_H__
> +
> +#ifdef CONFIG_HAS_PCI
> +
> +#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> +
> +/* Arch pci dev struct */
>  struct arch_pci_dev {
> +    struct device dev;
>  };
>
> -#endif /* __X86_PCI_H__ */
> +#else   /*!CONFIG_HAS_PCI*/
> +
> +struct arch_pci_dev { };
> +
> +#endif  /*!CONFIG_HAS_PCI*/
> +#endif /* __ARM_PCI_H__ */
> --
> 2.17.1
>
>

--00000000000075605e05cca017ec
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div>Hi,<br><br><div class=3D"gmail_quote"><div dir=3D"lt=
r" class=3D"gmail_attr">On Wed, 22 Sep 2021, 16:36 Rahul Singh, &lt;<a href=
=3D"mailto:rahul.singh@arm.com">rahul.singh@arm.com</a>&gt; wrote:<br></div=
><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=
px #ccc solid;padding-left:1ex">Compilation error is observed when HAS_PCI =
is enabled for ARM<br>
architecture.</blockquote></div></div><div dir=3D"auto"><br></div><div dir=
=3D"auto">In general, when I read &quot;compilation error&quot; I interpret=
 as a user can trigger it in the current staging.</div><div dir=3D"auto"><b=
r></div><div dir=3D"auto">However, without the full series applied, HAS_PCI=
 cannot be selected on Arm.</div><div dir=3D"auto"><br></div><div dir=3D"au=
to">So I think the message is a bit misleading because this is more impleme=
nting stubs to enable a feature rather than fixing compilation errors (AFAI=
CT all of them are not fixed here).</div><div dir=3D"auto"><br></div><div d=
ir=3D"auto">How about the following commit message:</div><div dir=3D"auto">=
<br></div><div dir=3D"auto">xen/arm: pci: Add stubs to allow selecting HAS_=
PCI</div><div dir=3D"auto"><br></div><div dir=3D"auto">In a follow-up we wi=
ll enable PCI support in Xen on Arm (i.e select HAS_PCI).</div><div dir=3D"=
auto"><br></div><div dir=3D"auto">The generic code expects the arch to impl=
ement a few functions:</div><div dir=3D"auto"><br></div><div dir=3D"auto">&=
lt;List the functions&gt;</div><div dir=3D"auto"><br></div><div dir=3D"auto=
">Note that this is not yet sufficient to enable HAS_PCI and will be addres=
sed in follow-ups.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div =
class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Add definition for arch_iommu_use_permitted() and<br>
arch_pci_clean_pirqs().<br>
<br>
pci.c: In function =E2=80=98deassign_device=E2=80=99:<br>
pci.c:849:49: error: implicit declaration of function =E2=80=98pci_to_dev=
=E2=80=99;<br>
did you mean =E2=80=98dt_to_dev=E2=80=99? [-Werror=3Dimplicit-function-decl=
aration]<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pci_to_dev(pdev));<br>
pci.c:880: undefined reference to `arch_pci_clean_pirqs=E2=80=99<br>
pci.c:1392: undefined reference to `arch_iommu_use_permitted&#39;<br>
<br>
Signed-off-by: Rahul Singh &lt;<a href=3D"mailto:rahul.singh@arm.com" targe=
t=3D"_blank" rel=3D"noreferrer">rahul.singh@arm.com</a>&gt;<br>
---<br>
Change in v2:<br>
- Remove pci_conf_read*(..) dummy implementation<br>
- Add in code comment for arch_pci_clean_pirqs() and arch_iommu_use_permitt=
ed()<br>
- Fixed minor comments<br>
---<br>
=C2=A0xen/arch/arm/Makefile=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0|=C2=A0 1 +<br>
=C2=A0xen/arch/arm/pci/Makefile=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=
=C2=A0 1 +<br>
=C2=A0xen/arch/arm/pci/pci.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 | 33 +++++++++++++++++++++++++++++<br>
=C2=A0xen/drivers/passthrough/arm/iommu.c |=C2=A0 9 ++++++++<br>
=C2=A0xen/include/asm-arm/pci.h=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 3=
1 ++++++++++++++++++++++++---<br>
=C2=A05 files changed, 72 insertions(+), 3 deletions(-)<br>
=C2=A0create mode 100644 xen/arch/arm/pci/Makefile<br>
=C2=A0create mode 100644 xen/arch/arm/pci/pci.c<br>
<br>
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile<br>
index 3d3b97b5b4..44d7cc81fa 100644<br>
--- a/xen/arch/arm/Makefile<br>
+++ b/xen/arch/arm/Makefile<br>
@@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) +=3D arm32/<br>
=C2=A0obj-$(CONFIG_ARM_64) +=3D arm64/<br>
=C2=A0obj-$(CONFIG_ARM_64) +=3D efi/<br>
=C2=A0obj-$(CONFIG_ACPI) +=3D acpi/<br>
+obj-$(CONFIG_HAS_PCI) +=3D pci/<br>
=C2=A0ifneq ($(CONFIG_NO_PLAT),y)<br>
=C2=A0obj-y +=3D platforms/<br>
=C2=A0endif<br>
diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile<br>
new file mode 100644<br>
index 0000000000..a98035df4c<br>
--- /dev/null<br>
+++ b/xen/arch/arm/pci/Makefile<br>
@@ -0,0 +1 @@<br>
+obj-y +=3D pci.o<br>
diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c<br>
new file mode 100644<br>
index 0000000000..a7a7bc3213<br>
--- /dev/null<br>
+++ b/xen/arch/arm/pci/pci.c<br>
@@ -0,0 +1,33 @@<br>
+/*<br>
+ * This program is free software; you can redistribute it and/or modify<br=
>
+ * it under the terms of the GNU General Public License version 2 as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This program is distributed in the hope that it will be useful,<br>
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of<br>
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.=C2=A0 See the<br>
+ * GNU General Public License for more details.<br>
+ *<br>
+ * You should have received a copy of the GNU General Public License<br>
+ * along with this program.=C2=A0 If not, see &lt;<a href=3D"http://www.gn=
u.org/licenses/" rel=3D"noreferrer noreferrer" target=3D"_blank">http://www=
.gnu.org/licenses/</a>&gt;.<br>
+ */<br>
+<br>
+#include &lt;xen/pci.h&gt;<br>
+<br>
+/*<br>
+ * PIRQ event channels are not supported on Arm, so nothing to do.<br>
+ */<br>
+int arch_pci_clean_pirqs(struct domain *d)<br>
+{<br>
+=C2=A0 =C2=A0 return 0;<br>
+}<br>
+<br>
+/*<br>
+ * Local variables:<br>
+ * mode: C<br>
+ * c-file-style: &quot;BSD&quot;<br>
+ * c-basic-offset: 4<br>
+ * tab-width: 4<br>
+ * indent-tabs-mode: nil<br>
+ * End:<br>
+ */<br>
diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/=
arm/iommu.c<br>
index db3b07a571..ee653a9c48 100644<br>
--- a/xen/drivers/passthrough/arm/iommu.c<br>
+++ b/xen/drivers/passthrough/arm/iommu.c<br>
@@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)<br>
=C2=A0void __hwdom_init arch_iommu_hwdom_init(struct domain *d)<br>
=C2=A0{<br>
=C2=A0}<br>
+<br>
+/*<br>
+ * Unlike x86, Arm doesn&#39;t support mem-sharing, mem-paging and log-dir=
ty (yet).<br>
+ * So there is no restriction to use the IOMMU.<br>
+ */<br>
+bool arch_iommu_use_permitted(const struct domain *d)<br>
+{<br>
+=C2=A0 =C2=A0 return true;<br>
+}<br>
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h<br>
index de13359f65..7dd9eb3dba 100644<br>
--- a/xen/include/asm-arm/pci.h<br>
+++ b/xen/include/asm-arm/pci.h<br>
@@ -1,7 +1,32 @@<br>
-#ifndef __X86_PCI_H__<br>
-#define __X86_PCI_H__<br>
+/*<br>
+ * This program is free software; you can redistribute it and/or modify<br=
>
+ * it under the terms of the GNU General Public License version 2 as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This program is distributed in the hope that it will be useful,<br>
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of<br>
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.=C2=A0 See the<br>
+ * GNU General Public License for more details.<br>
+ *<br>
+ * You should have received a copy of the GNU General Public License<br>
+ * along with this program.=C2=A0 If not, see &lt;<a href=3D"http://www.gn=
u.org/licenses/" rel=3D"noreferrer noreferrer" target=3D"_blank">http://www=
.gnu.org/licenses/</a>&gt;.<br>
+ */<br>
<br>
+#ifndef __ARM_PCI_H__<br>
+#define __ARM_PCI_H__<br>
+<br>
+#ifdef CONFIG_HAS_PCI<br>
+<br>
+#define pci_to_dev(pcidev) (&amp;(pcidev)-&gt;<a href=3D"http://arch.dev" =
rel=3D"noreferrer noreferrer" target=3D"_blank">arch.dev</a>)<br>
+<br>
+/* Arch pci dev struct */<br>
=C2=A0struct arch_pci_dev {<br>
+=C2=A0 =C2=A0 struct device dev;<br>
=C2=A0};<br>
<br>
-#endif /* __X86_PCI_H__ */<br>
+#else=C2=A0 =C2=A0/*!CONFIG_HAS_PCI*/<br>
+<br>
+struct arch_pci_dev { };<br>
+<br>
+#endif=C2=A0 /*!CONFIG_HAS_PCI*/<br>
+#endif /* __ARM_PCI_H__ */<br>
-- <br>
2.17.1<br>
<br>
</blockquote></div></div></div>

--00000000000075605e05cca017ec--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:10:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:10:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193273.344250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEBT-0001jX-Nv; Thu, 23 Sep 2021 02:09:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193273.344250; Thu, 23 Sep 2021 02:09:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEBT-0001jQ-Kl; Thu, 23 Sep 2021 02:09:59 +0000
Received: by outflank-mailman (input) for mailman id 193273;
 Thu, 23 Sep 2021 02:09:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTEBS-0001jK-LY
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:09:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 58e4ea92-1c13-11ec-ba04-12813bfff9fa;
 Thu, 23 Sep 2021 02:09:56 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C0B3360F6F;
 Thu, 23 Sep 2021 02:09:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58e4ea92-1c13-11ec-ba04-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632362996;
	bh=n3CFCvWleyY3w7fbyTsbKm3EbMrA4k89gghybVgsBVU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bJDkYxSjrHQDhyyQtEpsnnq8RazNnMVB0j595BFeRg7oyFadN2QC3wZIMkCXLqZ71
	 6u6XU4a/BvXNgEQRC8VWNyaYAQ/0TbRYUSq6k1dHq02btbQe+J1bw6TQPSmlYU6I0Z
	 PCxxjvJ2j+6OgNITjLtTRuZFgg9zUelMG9FxxGwI00bImFfDmeXtJ5q3PQ3mlVBBCI
	 CLwZg3maRDikRBdh5OeDrHECMS5z9dLXBN2qXVqpGhAxxbQSOyrlnMcxdOWrvEHWdi
	 d5UFykZ/eC/NaEyzovnXR3oqdQyV85aDuCKL3/0/ZhpZUiYvaDMzDVHviMPCiKCni/
	 iaqGpdllUmKuQ==
Date: Wed, 22 Sep 2021 19:09:55 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-251838594-1632361499=:17979"
Content-ID: <alpine.DEB.2.21.2109221846390.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-251838594-1632361499=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109221846391.17979@sstabellini-ThinkPad-T480s>

On Wed, 22 Sep 2021, Rahul Singh wrote:
> XEN during boot will read the PCI device tree node “reg” property
> and will map the PCI config space to the XEN memory.
> 
> As of now only "pci-host-ecam-generic" compatible board is supported.
> 
> "linux,pci-domain" device tree property assigns a fixed PCI domain
> number to a host bridge, otherwise an unstable (across boots) unique
> number will be assigned by Linux. XEN access the PCI devices based on
> Segment:Bus:Device:Function. Segment number in XEN is same as domain
                               ^ A segment             ^ the    ^ a


> number in Linux.Segment number and domain number has to be in sync
                  ^ " "

> to access the correct PCI devices.
> 
> XEN will read the “linux,pci-domain” property from the device tree node
> and configure the host bridge segment number accordingly. If this
> property is not available XEN will allocate the unique segment number
> to the host bridge.
> 
> dt_pci_bus_find_domain_nr(..) imported from the Linux source tree with
> slight modification based on tag Linux v5.14.2
> commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3fc.

dt_pci_bus_find_domain_nr is not introduced by this patch any longer


> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Add more info in commit msg
> - Add callback to parse register index.
> - Merge patch pci_ecam_operation into this patch to avoid confusion
> - Add new struct in struct device for match table
> ---
>  xen/arch/arm/device.c               |   2 +
>  xen/arch/arm/pci/Makefile           |   5 +
>  xen/arch/arm/pci/ecam.c             |  60 +++++++
>  xen/arch/arm/pci/pci-access.c       |  83 +++++++++
>  xen/arch/arm/pci/pci-host-common.c  | 254 ++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-generic.c |  42 +++++
>  xen/include/asm-arm/device.h        |   2 +
>  xen/include/asm-arm/pci.h           |  59 +++++++
>  8 files changed, 507 insertions(+)
>  create mode 100644 xen/arch/arm/pci/ecam.c
>  create mode 100644 xen/arch/arm/pci/pci-access.c
>  create mode 100644 xen/arch/arm/pci/pci-host-common.c
>  create mode 100644 xen/arch/arm/pci/pci-host-generic.c
> 
> diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
> index 70cd6c1a19..197bb3c6e8 100644
> --- a/xen/arch/arm/device.c
> +++ b/xen/arch/arm/device.c
> @@ -44,6 +44,8 @@ int __init device_init(struct dt_device_node *dev, enum device_class class,
>          {
>              ASSERT(desc->init != NULL);
>  
> +            dev->dev.of_match_table = desc->dt_match;
> +
>              return desc->init(dev, data);
>          }
>  
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index a98035df4c..e86f2b46fd 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -1 +1,6 @@
>  obj-y += pci.o
> +obj-y += pci-access.o
> +obj-y += pci.o

Added twice?


> +obj-y += pci-host-generic.o
> +obj-y += pci-host-common.o
> +obj-y += ecam.o
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> new file mode 100644
> index 0000000000..9b88b1ceda
> --- /dev/null
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -0,0 +1,60 @@
> +/*
> + * Based on Linux drivers/pci/ecam.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/sched.h>
> +
> +/*
> + * Function to implement the pci_ops ->map_bus method.
> + */
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               uint32_t sbdf, uint32_t where)
> +{
> +    const struct pci_config_window *cfg = bridge->cfg;
> +    const struct pci_ecam_ops *ops = bridge->sysdata;
> +    unsigned int devfn_shift = ops->bus_shift - 8;
> +    void __iomem *base;
> +
> +    unsigned int busn = PCI_BUS(sbdf);
> +
> +    if ( busn < cfg->busn_start || busn > cfg->busn_end )
> +        return NULL;
> +
> +    busn -= cfg->busn_start;
> +    base = cfg->win + (busn << ops->bus_shift);
> +
> +    return base + (PCI_DEVFN2(sbdf) << devfn_shift) + where;
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops pci_generic_ecam_ops = {
> +    .bus_shift  = 20,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> new file mode 100644
> index 0000000000..04fe9fbf92
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -0,0 +1,83 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <asm/io.h>
> +
> +#define INVALID_VALUE (~0U)
> +
> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +
> +    if ( !addr )
> +    {
> +        *value = INVALID_VALUE;
> +        return -ENODEV;
> +    }
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        *value = readb(addr);
> +        break;
> +    case 2:
> +        *value = readw(addr);
> +        break;
> +    case 4:
> +        *value = readl(addr);
> +        break;
> +    default:
> +        ASSERT_UNREACHABLE();
> +    }
> +
> +    return 0;
> +}
> +
> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value)
                               ^ code style


> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +
> +    if ( !addr )
> +        return -ENODEV;
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        writeb(value, addr);
> +        break;
> +    case 2:
> +        writew(value, addr);
> +        break;
> +    case 4:
> +        writel(value, addr);
> +        break;
> +    default:
> +        ASSERT_UNREACHABLE();
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> new file mode 100644
> index 0000000000..4beec14f2f
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -0,0 +1,254 @@
> +/*
> + * Based on Linux drivers/pci/ecam.c
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <xen/rwlock.h>
> +#include <xen/sched.h>
> +#include <xen/vmap.h>
> +
> +/*
> + * List for all the pci host bridges.
> + */
> +
> +static LIST_HEAD(pci_host_bridges);
> +
> +static atomic_t domain_nr = ATOMIC_INIT(-1);
> +
> +static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
> +{
> +    return ioremap_nocache(start, len);
> +}
> +
> +static void pci_ecam_free(struct pci_config_window *cfg)
> +{
> +    if ( cfg->win )
> +        iounmap(cfg->win);
> +
> +    xfree(cfg);
> +}
> +
> +static struct pci_config_window * __init
> +gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
> +{
> +    int err, cfg_reg_idx;
> +    u32 bus_range[2];
> +    paddr_t addr, size;
> +    struct pci_config_window *cfg;
> +
> +    cfg = xzalloc(struct pci_config_window);
> +    if ( !cfg )
> +        return NULL;
> +
> +    err = dt_property_read_u32_array(dev, "bus-range", bus_range,
> +                                     ARRAY_SIZE(bus_range));
> +    if ( err ) {
> +        cfg->busn_start = 0;
> +        cfg->busn_end = 0xff;
> +        printk(XENLOG_INFO "%s: No bus range found for pci controller\n",
> +               dt_node_full_name(dev));
> +    } else {
> +        cfg->busn_start = bus_range[0];
> +        cfg->busn_end = bus_range[1];
> +        if ( cfg->busn_end > cfg->busn_start + 0xff )
> +            cfg->busn_end = cfg->busn_start + 0xff;
> +    }
> +
> +    if ( ops->cfg_reg_index )
> +    {
> +        cfg_reg_idx = ops->cfg_reg_index(dev);
> +        if ( cfg_reg_idx < 0 )
> +            goto err_exit;
> +    }
> +    else
> +        cfg_reg_idx = 0;
> +
> +    /* Parse our PCI ecam register address */
> +    err = dt_device_get_address(dev, cfg_reg_idx, &addr, &size);
> +    if ( err )
> +        goto err_exit;
> +
> +    cfg->phys_addr = addr;
> +    cfg->size = size;
> +
> +    /*
> +     * On 64-bit systems, we do a single ioremap for the whole config space
> +     * since we have enough virtual address range available.  On 32-bit, we
> +     * ioremap the config space for each bus individually.
> +     * As of now only 64-bit is supported 32-bit is not supported.
> +     *
> +     * TODO: For 32-bit implement the ioremap/iounmap of config space
> +     * dynamically for each read/write call.
> +     */
> +    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
> +    if ( !cfg->win )
> +        goto err_exit_remap;
> +
> +    printk("ECAM at [mem 0x%"PRIpaddr"-0x%"PRIpaddr"] for [bus %x-%x] \n",
> +            cfg->phys_addr, cfg->phys_addr + cfg->size - 1,
> +            cfg->busn_start, cfg->busn_end);
> +
> +    if ( ops->init )
> +    {
> +        err = ops->init(cfg);
> +        if (err)

code style


> +            goto err_exit;
> +    }

This is unnecessary at the moment, right? Can we get rid of ops->init ?


> +    return cfg;
> +
> +err_exit_remap:
> +    printk(XENLOG_ERR "ECAM ioremap failed\n");

No need for err_exit_remap as pci_ecam_free can take care of both cases.

> +err_exit:
> +    pci_ecam_free(cfg);
> +
> +    return NULL;
> +}
> +
> +struct pci_host_bridge *pci_alloc_host_bridge(void)
> +{
> +    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
> +
> +    if ( !bridge )
> +        return NULL;
> +
> +    INIT_LIST_HEAD(&bridge->node);
> +    bridge->bus_start = UINT8_MAX;
> +    bridge->bus_end = UINT8_MAX;
> +
> +    return bridge;
> +}
> +
> +void pci_add_host_bridge(struct pci_host_bridge *bridge)
> +{
> +    list_add_tail(&bridge->node, &pci_host_bridges);
> +}
> +
> +static int pci_get_new_domain_nr(void)
> +{
> +    return atomic_inc_return(&domain_nr);
> +}
> +
> +static int pci_bus_find_domain_nr(struct dt_device_node *dev)
> +{
> +    static int use_dt_domains = -1;
> +    int domain;
> +
> +    domain = dt_get_pci_domain_nr(dev);
> +
> +    /*
> +     * Check DT domain and use_dt_domains values.
> +     *
> +     * If DT domain property is valid (domain >= 0) and
> +     * use_dt_domains != 0, the DT assignment is valid since this means
> +     * we have not previously allocated a domain number by using
> +     * pci_get_new_domain_nr(); we should also update use_dt_domains to
> +     * 1, to indicate that we have just assigned a domain number from
> +     * DT.
> +     *
> +     * If DT domain property value is not valid (ie domain < 0), and we
> +     * have not previously assigned a domain number from DT
> +     * (use_dt_domains != 1) we should assign a domain number by
> +     * using the:
> +     *
> +     * pci_get_new_domain_nr()
> +     *
> +     * API and update the use_dt_domains value to keep track of method we
> +     * are using to assign domain numbers (use_dt_domains = 0).
> +     *
> +     * All other combinations imply we have a platform that is trying
> +     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
> +     * which is a recipe for domain mishandling and it is prevented by
> +     * invalidating the domain value (domain = -1) and printing a
> +     * corresponding error.
> +     */
> +    if ( domain >= 0 && use_dt_domains )
> +    {
> +        use_dt_domains = 1;
> +    }
> +    else if ( domain < 0 && use_dt_domains != 1 )
> +    {
> +        use_dt_domains = 0;
> +        domain = pci_get_new_domain_nr();
> +    }
> +    else
> +    {
> +        domain = -1;
> +    }
> +
> +    return domain;
> +}
> +
> +int pci_host_common_probe(struct dt_device_node *dev, const void *data)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct pci_config_window *cfg;
> +    struct pci_ecam_ops *ops;
> +    const struct dt_device_match *of_id;
> +    int err;
> +
> +    if ( dt_device_for_passthrough(dev) )
> +        return 0;
> +
> +    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
> +    ops = (struct pci_ecam_ops *) of_id->data;

Do we really need dt_match_node and dev->dev.of_match_table to get
dt_device_match.data?

data is passed as a parameter to pci_host_common_probe, isn't it enough
to do:

ops = (struct pci_ecam_ops *) data;


> +    bridge = pci_alloc_host_bridge();
> +    if ( !bridge )
> +        return -ENOMEM;
> +
> +    /* Parse and map our Configuration Space windows */
> +    cfg = gen_pci_init(dev, ops);
> +    if ( !cfg )
> +    {
> +        err = -ENOMEM;
> +        goto err_exit;
> +    }
> +
> +    bridge->dt_node = dev;
> +    bridge->cfg = cfg;
> +    bridge->sysdata = ops;
> +    bridge->ops = &ops->pci_ops;
> +    bridge->bus_start = cfg->busn_start;
> +    bridge->bus_end = cfg->busn_end;
> +
> +    bridge->segment = pci_bus_find_domain_nr(dev);
> +    if ( bridge->segment < 0 )
> +    {
> +        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
> +        BUG();
> +    }
> +    pci_add_host_bridge(bridge);
> +
> +    return 0;
> +
> +err_exit:
> +    xfree(bridge);
> +
> +    return err;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> new file mode 100644
> index 0000000000..6b3288d6f3
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -0,0 +1,42 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static const struct dt_device_match gen_pci_dt_match[] = {
> +    { .compatible = "pci-host-ecam-generic",
> +      .data =       &pci_generic_ecam_ops },
> +
> +    { },
> +};
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> +.dt_match = gen_pci_dt_match,
> +.init = pci_host_common_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index 5ecd5e7bd1..582119c31e 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -16,6 +16,8 @@ struct device
>      enum device_type type;
>  #ifdef CONFIG_HAS_DEVICE_TREE
>      struct dt_device_node *of_node; /* Used by drivers imported from Linux */
> +    /* Used by drivers imported from Linux */
> +    const struct dt_device_match *of_match_table;
>  #endif
>      struct dev_archdata archdata;
>      struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index f2f86be9bc..4b32c7088a 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,65 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/*
> + * struct to hold the mappings of a config space window. This
> + * is expected to be used as sysdata for PCI controllers that
> + * use ECAM.
> + */
> +struct pci_config_window {
> +    paddr_t         phys_addr;
> +    paddr_t         size;
> +    uint8_t         busn_start;
> +    uint8_t         busn_end;
> +    void __iomem    *win;
> +};
> +
> +/*
> + * struct to hold pci host bridge information
> + * for a PCI controller.
> + */
> +struct pci_host_bridge {
> +    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
> +    struct list_head node;           /* Node in list of host bridges */
> +    uint16_t segment;                /* Segment number */
> +    uint8_t bus_start;               /* Bus start of this bridge. */
> +    uint8_t bus_end;                 /* Bus end of this bridge. */

bus_start and bus_end are both here and also under pci_config_window.
Should we remove them from here (if not, then can we removed them from
struct pci_config_window)?


> +    struct pci_config_window* cfg;   /* Pointer to the bridge config window */
> +    void *sysdata;                   /* Pointer to the config space window*/
> +    const struct pci_ops *ops;

It looks like sysdata is unnecessary because we can get the right
pointer from ops, given that ops is pointing to a member of the struct
point by sysdata. In other words, if you use container_of(ops, struct
pci_ecam_ops, pci_ops) it should work?


> +};
> +
> +struct pci_ops {
> +    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                             uint32_t offset);
> +    int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                uint32_t reg, uint32_t len, uint32_t *value);
> +    int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                 uint32_t reg, uint32_t len, uint32_t value);
> +};
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_ecam_ops {
> +    unsigned int            bus_shift;
> +    struct pci_ops          pci_ops;
> +    int (*cfg_reg_index)(struct dt_device_node *dev);
> +    int (*init)(struct pci_config_window *);

init is unused, can we get rid of it?


> +};
> +
> +/* Default ECAM ops */
> +extern const struct pci_ecam_ops pci_generic_ecam_ops;

If we use container_of and get rid of sysdata, I wonder if we get avoid
exporting pci_generic_ecam_ops.


> +int pci_host_common_probe(struct dt_device_node *dev, const void *data);

This should be static and not exported


> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value);

also this


> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t value);

also this


> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               uint32_t sbdf, uint32_t where);

also this

>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
>      return pci_passthrough_enabled;
> -- 
> 2.17.1
> 
--8323329-251838594-1632361499=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:11:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:11:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193278.344260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEDL-00033j-3G; Thu, 23 Sep 2021 02:11:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193278.344260; Thu, 23 Sep 2021 02:11:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEDL-00033c-0N; Thu, 23 Sep 2021 02:11:55 +0000
Received: by outflank-mailman (input) for mailman id 193278;
 Thu, 23 Sep 2021 02:11:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTEDJ-00033P-21
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:11:53 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9da2e36e-1c13-11ec-ba04-12813bfff9fa;
 Thu, 23 Sep 2021 02:11:52 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0584160F6F;
 Thu, 23 Sep 2021 02:11:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9da2e36e-1c13-11ec-ba04-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632363111;
	bh=xh93hQYHLTrmEWpKa/RRvkSMxY/bjEQtAb+dCCClAnw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=agXw60tK1DbRlICfzlzF6x0AQXXor78GwIYozAtKYl6aB/M7tr4NjHS/X0Qz8UiEC
	 yDreiuLLZBePeRDd6Eb1z/XNxtQGHsCcUy6GT9MSrVuTiM9UYgHiTDhJ/gg0CFu2JE
	 Q03ozGT7yoUU3Sg7P2yyUug97rgxi88gjkea9r/O1Xn1LDo5Oc3gfuJ5xWPLH6q8uQ
	 PZrXtXb661m0NXXHT6w2bTw9qQ2U30iiOkGJRsuGuy1Tu/8ptCT/p+m7G+/1g6g0mT
	 C1dEWKKWJ99QR1KEa86tkvyxYWddXXqeaw235bXy2cCp3quzwYH59Roivq/PqwJO+r
	 qlzSuQ18+ee7A==
Date: Wed, 22 Sep 2021 19:11:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
In-Reply-To: <4b2b04ab1dbc1a5c52bf54c399cdb6cf454183ea.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221911020.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <4b2b04ab1dbc1a5c52bf54c399cdb6cf454183ea.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
> space to the XEN memory.
> 
> Patch helps to understand how the generic infrastructure for PCI
> host-bridge discovery will be used for future references.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
> Change on v2:
> - Add more info in commit msg
> ---
>  xen/arch/arm/pci/Makefile          |  1 +
>  xen/arch/arm/pci/pci-host-zynqmp.c | 58 ++++++++++++++++++++++++++++++
>  2 files changed, 59 insertions(+)
>  create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index e86f2b46fd..0572c68bba 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -4,3 +4,4 @@ obj-y += pci.o
>  obj-y += pci-host-generic.o
>  obj-y += pci-host-common.o
>  obj-y += ecam.o
> +obj-y += pci-host-zynqmp.o
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> new file mode 100644
> index 0000000000..c27b4ea9f0
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -0,0 +1,58 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +int nwl_cfg_reg_index(struct dt_device_node *np)

static


> +{
> +    return dt_property_match_string(np, "reg-names", "cfg");
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops nwl_pcie_ops = {
> +    .bus_shift  = 20,
> +    .cfg_reg_index = nwl_cfg_reg_index,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +static const struct dt_device_match nwl_pcie_dt_match[] = {
> +    { .compatible = "xlnx,nwl-pcie-2.11",
> +      .data =       &nwl_pcie_ops },
> +    { },
> +};
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
> +.dt_match = nwl_pcie_dt_match,
> +.init = pci_host_common_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:23:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:23:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193283.344271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEOl-0004ky-5w; Thu, 23 Sep 2021 02:23:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193283.344271; Thu, 23 Sep 2021 02:23:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEOl-0004kr-30; Thu, 23 Sep 2021 02:23:43 +0000
Received: by outflank-mailman (input) for mailman id 193283;
 Thu, 23 Sep 2021 02:23:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTEOk-0004kl-7e
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:23:42 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e2d54a80-b764-40b5-9ba2-91756671de3a;
 Thu, 23 Sep 2021 02:23:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4594D6105A;
 Thu, 23 Sep 2021 02:23:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2d54a80-b764-40b5-9ba2-91756671de3a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632363820;
	bh=hT1JcbC4JuV/bBEFDsCl1PT6WUlMzyXEud51b8zCBD8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XZzBIuHHr6eMUB52GSBWzGqDnCIbgGf3Oo756i2nBUk1cN3ZbmWSj8nLN7qi5ML7I
	 I31tb/ZfGN53n/6pxNHvP70gsmYCoMe4yDTH/LM9zwD58I/3DTko8Nbc550K00+Fvx
	 zQga718sUklko0gIblcaOWX7rgDRj/QnHTrlPG6BCh2QBjBWTm4p0OoIZg9IYfQ+Ry
	 lggnWaZ08VeiqwIprJ69BnwY5lo1rBsflLaV9Cxkl10xm5XzimMS+RPvTqN2OuglDZ
	 IFHrV/LdKUQ/s+vplqHXCoIHO8MScwuek6n0mxE0pJBcA8u3ofCRHRrLZB8m8II/cO
	 DOXkp5bV0FcUg==
Date: Wed, 22 Sep 2021 19:23:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
In-Reply-To: <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Subject should have xen/arm


On Wed, 22 Sep 2021, Rahul Singh wrote:
> Implement generic pci access functions to read/write the configuration
> space.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2: Fixed comments 
> ---
>  xen/arch/arm/pci/pci-access.c      | 58 ++++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
>  xen/include/asm-arm/pci.h          |  2 ++
>  3 files changed, 79 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index 04fe9fbf92..45500cec2a 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -16,6 +16,7 @@
>  #include <asm/io.h>
>  
>  #define INVALID_VALUE (~0U)
> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>  
>  int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
>                              uint32_t reg, uint32_t len, uint32_t *value)
> @@ -72,6 +73,63 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>      return 0;
>  }
>  
> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
> +                                unsigned int len)
> +{
> +    uint32_t val = PCI_ERR_VALUE(len);
> +

No blank line


> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return val;
> +
> +    if ( unlikely(!bridge->ops->read) )
> +        return val;
> +
> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);

The more I look at these casts the less I like them :-D

One idea is to move the definition of pci_sbdf_t somewhere else
entirely, for instance xen/include/xen/types.h, then we can use
pci_sbdf_t everywhere


> +    return val;
> +}
> +
> +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
> +                             unsigned int len, uint32_t val)
> +{
> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return;
> +
> +    if ( unlikely(!bridge->ops->write) )
> +        return;
> +
> +    bridge->ops->write(bridge, (uint32_t) sbdf.sbdf, reg, len, val);
> +}
> +
> +/*
> + * Wrappers for all PCI configuration access functions.
> + */
> +
> +#define PCI_OP_WRITE(size, type)                            \
> +    void pci_conf_write##size(pci_sbdf_t sbdf,              \
> +                              unsigned int reg, type val)   \
> +{                                                           \
> +    pci_config_write(sbdf, reg, size / 8, val);             \
> +}
> +
> +#define PCI_OP_READ(size, type)                             \
> +    type pci_conf_read##size(pci_sbdf_t sbdf,               \
> +                              unsigned int reg)             \
> +{                                                           \
> +    return pci_config_read(sbdf, reg, size / 8);            \
> +}
> +
> +PCI_OP_READ(8, uint8_t)
> +PCI_OP_READ(16, uint16_t)
> +PCI_OP_READ(32, uint32_t)
> +PCI_OP_WRITE(8, uint8_t)
> +PCI_OP_WRITE(16, uint16_t)
> +PCI_OP_WRITE(32, uint32_t)
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 4beec14f2f..3bdc336268 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -243,6 +243,25 @@ err_exit:
>      return err;
>  }
>  
> +/*
> + * This function will lookup an hostbridge based on the segment and bus
> + * number.
> + */
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->segment != segment )
> +            continue;
> +        if ( (bus < bridge->bus_start) || (bus > bridge->bus_end) )
> +            continue;
> +        return bridge;
> +    }
> +
> +    return NULL;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 4b32c7088a..5406daecda 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -18,6 +18,7 @@
>  #ifdef CONFIG_HAS_PCI
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> +#define PRI_pci "%04x:%02x:%02x.%u"

This is added in this patch but it is unused here


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:41:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:41:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193291.344282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEfz-0007TS-OT; Thu, 23 Sep 2021 02:41:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193291.344282; Thu, 23 Sep 2021 02:41:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEfz-0007TJ-LZ; Thu, 23 Sep 2021 02:41:31 +0000
Received: by outflank-mailman (input) for mailman id 193291;
 Thu, 23 Sep 2021 02:41:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTEfy-0007Su-5t
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:41:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6c72f810-d02f-4b04-9af5-167d89ebe6f2;
 Thu, 23 Sep 2021 02:41:28 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D91760F70;
 Thu, 23 Sep 2021 02:41:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c72f810-d02f-4b04-9af5-167d89ebe6f2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632364887;
	bh=IXRhmptxkJbaYtOuw0aSN4IAF1z9IJstey2yZCM3F3k=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Rn9AFJ6wIyhKP/lnt7bnpJgS+5l60aXDHKT3xPf7GLKiqQX1GTaEJ+oiY8FuJLIcT
	 DsU/DKu9FNO9W7yqoapGIK6FSaYjGR8dVkSRP/qAsmPPPM2xu/8mwDb86gMbfE26W+
	 4yxAiCWBZQssKleeM0h5HlbzRCRCstK4KoY3GslqPcKAV6jj/bJTNbUkzdf3t8e1pa
	 fYFyKyEoJ3znNrMjHywZqq4Mq3LgV1GgJgPaKeX3G47uPXQN/b7XG+pwmPlqYu98mb
	 V02McxDsMqEtd0HwxZEa3FUL5FTR4WsbOOYiz3g/L+Mhjl/D7PkJHi/P1Nz+4BVInN
	 bybRnFkJc86+Q==
Date: Wed, 22 Sep 2021 19:41:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    Paul Durrant <paul@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
In-Reply-To: <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1608248556-1632364432=:17979"
Content-ID: <alpine.DEB.2.21.2109221935580.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1608248556-1632364432=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109221935581.17979@sstabellini-ThinkPad-T480s>

On Wed, 22 Sep 2021, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> access is added to the Xen to emulate the PCI devices config space.
> 
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.
> 
> For Dom0less systems scan_pci_devices() would be used to discover the
> PCI device in XEN and VPCI handler will be added during XEN boots.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Add new XEN_DOMCTL_CDF_vpci flag
> - modify has_vpci() to include XEN_DOMCTL_CDF_vpci
> - enable vpci support when pci-passthough option is enabled.
> ---
>  xen/arch/arm/Makefile         |   1 +
>  xen/arch/arm/domain.c         |   6 +-
>  xen/arch/arm/domain_build.c   |   3 +
>  xen/arch/arm/vpci.c           | 102 ++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vpci.h           |  36 ++++++++++++
>  xen/common/domain.c           |   2 +-
>  xen/drivers/passthrough/pci.c |  12 ++++
>  xen/include/asm-arm/domain.h  |   8 ++-
>  xen/include/asm-x86/pci.h     |   2 -
>  xen/include/public/arch-arm.h |   7 +++
>  xen/include/public/domctl.h   |   4 +-
>  xen/include/xen/pci.h         |   2 +
>  12 files changed, 179 insertions(+), 6 deletions(-)
>  create mode 100644 xen/arch/arm/vpci.c
>  create mode 100644 xen/arch/arm/vpci.h
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 44d7cc81fa..fb9c976ea2 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y += platforms/
>  endif
>  obj-$(CONFIG_TEE) += tee/
> +obj-$(CONFIG_HAS_VPCI) += vpci.o
>  
>  obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>  obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..f7ed130023 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -39,6 +39,7 @@
>  #include <asm/vgic.h>
>  #include <asm/vtimer.h>
>  
> +#include "vpci.h"
>  #include "vuart.h"
>  
>  DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
> @@ -623,7 +624,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>      unsigned int max_vcpus;
>  
>      /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
> +    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci) !=
>           (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>      {
>          dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
> @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
>      if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
>          goto fail;
>  
> +    if ( (rc = domain_vpci_init(d)) != 0 )
> +        goto fail;
> +
>      return 0;
>  
>  fail:
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d634c1..5eb83b12a1 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2773,6 +2773,9 @@ void __init create_dom0(void)
>      if ( iommu_enabled )
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
> +    if ( pci_passthrough_enabled )

we should use the accessor function instead is_pci_passthrough_enabled


> +        dom0_cfg.flags |= XEN_DOMCTL_CDF_vpci;
> +
>      dom0 = domain_create(0, &dom0_cfg, true);
>      if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>          panic("Error creating domain 0\n");
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> new file mode 100644
> index 0000000000..76c12b9281
> --- /dev/null
> +++ b/xen/arch/arm/vpci.c
> @@ -0,0 +1,102 @@
> +/*
> + * xen/arch/arm/vpci.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +#include <xen/sched.h>
> +
> +#include <asm/mmio.h>
> +
> +#define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
> +
> +/* Do some sanity checks. */
> +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
> +{
> +    /* Check access size. */
> +    if ( len > 8 )
> +        return false;
> +
> +    /* Check that access is size aligned. */
> +    if ( (reg & (len - 1)) )
> +        return false;
> +
> +    return true;
> +}
> +
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                          register_t *r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = ~0UL;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    data = vpci_read(sbdf, reg, min(4u, size));
> +    if ( size == 8 )
> +        data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
> +
> +    *r = data;
> +
> +    return 1;
> +}
> +
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                           register_t r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = r;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    vpci_write(sbdf, reg, min(4u, size), data);
> +    if ( size == 8 )
> +        vpci_write(sbdf, reg + 4, 4, data >> 32);
> +
> +    return 1;
> +}
> +
> +static const struct mmio_handler_ops vpci_mmio_handler = {
> +    .read  = vpci_mmio_read,
> +    .write = vpci_mmio_write,
> +};
> +
> +int domain_vpci_init(struct domain *d)
> +{
> +    if ( !has_vpci(d) )
> +        return 0;
> +
> +    register_mmio_handler(d, &vpci_mmio_handler,
> +                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> +
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> new file mode 100644
> index 0000000000..d8a7b0e3e8
> --- /dev/null
> +++ b/xen/arch/arm/vpci.h
> @@ -0,0 +1,36 @@
> +/*
> + * xen/arch/arm/vpci.h
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ARCH_ARM_VPCI_H__
> +#define __ARCH_ARM_VPCI_H__
> +
> +#ifdef CONFIG_HAS_VPCI
> +int domain_vpci_init(struct domain *d);
> +#else
> +static inline int domain_vpci_init(struct domain *d)
> +{
> +    return 0;
> +}
> +#endif
> +
> +#endif /* __ARCH_ARM_VPCI_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6ee5d033b0..40d67ec342 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>           ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>             XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>             XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> -           XEN_DOMCTL_CDF_nested_virt) )
> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
>      {
>          dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>          return -EINVAL;
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index d774a6154e..633e89ac13 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>      else
>          iommu_enable_device(pdev);
>  
> +#ifdef CONFIG_ARM
> +    /*
> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
> +     * Dom0 inform XEN to add the PCI devices in XEN.
> +     */

I take this is not needed on x86 because for vpci is only used by Dom0
PVH?  It would be good to clarify either way.


> +    ret = vpci_add_handlers(pdev);
> +    if ( ret ) {
> +        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
> +        goto out;
> +    }
> +#endif
> +
>      pci_enable_acs(pdev);
>  
>  out:
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..5fb4e24adf 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -2,6 +2,7 @@
>  #define __ASM_DOMAIN_H__
>  
>  #include <xen/cache.h>
> +#include <xen/nospec.h>
>  #include <xen/timer.h>
>  #include <asm/page.h>
>  #include <asm/p2m.h>
> @@ -262,7 +263,12 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
>  
>  #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
>  
> -#define has_vpci(d)    ({ (void)(d); false; })
> +/*
> + * For X86 VPCI is enabled and tested for PVH DOM0 only but
> + * for ARM we enable support VPCI for guest domain also.
> + */
> +#define has_vpci(d) ((void)(d), IS_ENABLED(CONFIG_HAS_VPCI) && \
> +        evaluate_nospec(d->options & XEN_DOMCTL_CDF_vpci))

With the new options & XEN_DOMCTL_CDF_vpci check in place, do we even
need IS_ENABLED(CONFIG_HAS_VPCI) any longer?


>  #endif /* __ASM_DOMAIN_H__ */
>  
> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
> index 0e160c6d01..a5590557db 100644
> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -6,8 +6,6 @@
>  #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
>  #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
>  
> -#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
> -
>  #define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \
>                          || id == 0x01268086 || id == 0x01028086 \
>                          || id == 0x01128086 || id == 0x01228086 \
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 6b5a5f818a..727541a321 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -416,6 +416,13 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
>  #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
>  
> +/*
> + * 256 MB is reserved for VPCI configuration space based on calculation
> + * 256 buses × 32 devices × 8 functions × 4 KB = 256 MB
> + */
> +#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
> +#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
> +
>  /* ACPI tables physical address */
>  #define GUEST_ACPI_BASE xen_mk_ullong(0x20000000)
>  #define GUEST_ACPI_SIZE xen_mk_ullong(0x02000000)
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842..4245da3f45 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -70,9 +70,11 @@ struct xen_domctl_createdomain {
>  #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
>  #define _XEN_DOMCTL_CDF_nested_virt   6
>  #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
> +#define _XEN_DOMCTL_CDF_vpci          7
> +#define XEN_DOMCTL_CDF_vpci           (1U << _XEN_DOMCTL_CDF_vpci)
>  
>  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_vpci
>  
>      uint32_t flags;
>  
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index 8e3d4d9454..2b2dfb6f1b 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -41,6 +41,8 @@
>  #define PCI_SBDF3(s,b,df) \
>      ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
>  
> +#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
> +
>  typedef union {
>      uint32_t sbdf;
>      struct {
> -- 
> 2.17.1
> 
--8323329-1608248556-1632364432=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:52:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:52:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193296.344294 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEqr-0000hs-Qf; Thu, 23 Sep 2021 02:52:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193296.344294; Thu, 23 Sep 2021 02:52:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEqr-0000hl-MV; Thu, 23 Sep 2021 02:52:45 +0000
Received: by outflank-mailman (input) for mailman id 193296;
 Thu, 23 Sep 2021 02:52:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTEqq-0000hf-NE
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:52:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 84bc358e-4583-4aba-8619-22e5f81b14dc;
 Thu, 23 Sep 2021 02:52:43 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B7934600AA;
 Thu, 23 Sep 2021 02:52:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84bc358e-4583-4aba-8619-22e5f81b14dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632365563;
	bh=wV5ec62jAYRtdlBuwMHsb0HWYZs7rNB59pnUkneMLJU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uoLUGlcP4ow2JTzlEAtu7Z7D22lzH2Xkk5NynPzFI84AvE7Zwr0h1EFQlD+dcAmza
	 yKMEumUi8lyERSLrjBzIrWjsbLEd2JPvgKKZCK/01dBwGO+nH9aqZ5aUJr2BN9ypt0
	 G/XWJt3LJwASnNyXbOynkgXmsvgwhqhQsDi9KLw4K+rcc/5w/oifkEXwapn+Vmj7hL
	 RgYOt4e41AULqzOkWBoHBkLz0CF5KOT89iYnJ82tu9Czpqk5yd28HBe/KCGinfsD5H
	 4jGD9P0eDm8pMBKLMQbQ2o3PbrmsKfLiC63lE+D+VMk1EBqs5q65BqKjLEDHZOVgtX
	 gjmHfyrej/GWA==
Date: Wed, 22 Sep 2021 19:52:41 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 17/17] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
In-Reply-To: <e1b8d878709dc8e93a3bb340f921a24976cebb57.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109221948300.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <e1b8d878709dc8e93a3bb340f921a24976cebb57.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> If the property is not present in the device tree node for host bridge,
> XEN while creating the dtb for hwdom will create this property and
> assigns the already allocated segment to the host bridge
> so that XEN and linux will have the same segment for the host bridges.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - Add linux,pci-domain only when pci-passthrough command line option is enabeld
> ---
>  xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
>  xen/include/asm-arm/pci.h          |  9 +++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 5eb83b12a1..83ab0d52cc 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>              return res;
>      }
>  
> +    if ( pci_passthrough_enabled && dt_device_type_is_equal(node, "pci") )

Please use the accessor function is_pci_passthrough_enabled


> +    {
> +        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
> +        {
> +            uint16_t segment;
> +
> +            res = pci_get_host_bridge_segment(node, &segment);
> +            if ( res < 0 )
> +                return res;
> +
> +            res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment);
> +            if ( res )
> +                return res;
> +        }
> +    }
> +
>      /*
>       * Override the property "status" to disable the device when it's
>       * marked for passthrough.
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 3bdc336268..a88f20175e 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -262,6 +262,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
>  
>      return NULL;
>  }
> +
> +/*
> + * This function will lookup an hostbridge based on config space address.
> + */
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->dt_node != node )
> +            continue;
> +
> +        *segment = bridge->segment;
> +        return 0;
> +    }
> +
> +    return -EINVAL;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 9327b7488e..9edc6bc5ae 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -94,6 +94,8 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>  void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                 uint32_t sbdf, uint32_t where);
>  struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment);
>  
>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
> @@ -110,5 +112,12 @@ static always_inline bool is_pci_passthrough_enabled(void)
>      return false;
>  }
>  
> +static inline int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                              uint16_t *segment)
> +{
> +    ASSERT_UNREACHABLE();
> +    return -EINVAL;
> +}
> +
>  #endif  /*!CONFIG_HAS_PCI*/
>  #endif /* __ARM_PCI_H__ */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 02:57:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 02:57:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193301.344305 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEv2-0001Z9-DI; Thu, 23 Sep 2021 02:57:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193301.344305; Thu, 23 Sep 2021 02:57:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTEv2-0001Z2-8F; Thu, 23 Sep 2021 02:57:04 +0000
Received: by outflank-mailman (input) for mailman id 193301;
 Thu, 23 Sep 2021 02:57:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nnny=ON=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mTEv0-0001Yw-PB
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 02:57:02 +0000
Received: from mail-ed1-x52b.google.com (unknown [2a00:1450:4864:20::52b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 339bf0ee-0313-4566-88fc-712d2db902af;
 Thu, 23 Sep 2021 02:57:01 +0000 (UTC)
Received: by mail-ed1-x52b.google.com with SMTP id v10so13162679edj.10
 for <xen-devel@lists.xenproject.org>; Wed, 22 Sep 2021 19:57:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 339bf0ee-0313-4566-88fc-712d2db902af
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=stYRffpGhZzuPtYL+nsFafXU6FVqJNReuCUTT5KVYcY=;
        b=PFjyQ5QHnDAkBksCNwazeUl98k7zu+zjT4Q8Vzdh7O2RoZ1ZdKV1QxbcBAukOpGbPP
         7SJOCC1N8FZE/9s29rGHw/i5wMx7bIA2WqjknOwiFA38wQdeKNqv4tBLVV82BEqtVTrE
         hMii/RsZtDljcRd2m+9puxayZqzaXPLGehfxs8hvW7moN9yA8ACXixyjGY/4CL95LXmW
         3mekNO7zf1i0zl0CG1KN3Ls9v2FfpgJNNHxjnhCUkdoMdUs59o4LmjKpuXiQFfuZdJxX
         KjbG1TA72Ewzki0yHGK08sVOXRXJztlhMFc7GFCNQ23IOKHcTvaWqQbHglDwJj5ChSkP
         JGwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=stYRffpGhZzuPtYL+nsFafXU6FVqJNReuCUTT5KVYcY=;
        b=FpL7F84lqfsfKfnGT4VfVkgyFKLdbiHPV3TW2TaYVJdfK7EK0S/qTxTln6YE7XGhoO
         MkEd1V2v8nurHrpZjZOpiHST0vi0Tb/yLjqq9shAOrmePB54g16lyk8VUAC86e25iw6E
         fDxQWfHMEzCOoz5ekOCz1+lUK5d5ZkOqgqMm/zx18DOO6jV5a5u+j+r/xnn9m4ixWSUt
         bxpgkMhLGtLMRfYULjRsKEdbA7YdyRZcjDabq/jSU6hwBTwtfbemdF0whhXwl5HxQRlO
         A6mwWDbXSMsXSeHXnzrYJltDr+o8f6lT2OsrZDJUG4dWC9oGY91DR6uv/p0r6k9O5KJS
         tL6A==
X-Gm-Message-State: AOAM533Bo32d8Wdm8oKNH4raF8w6PkGyYRSBkUMIMqGFXu4JgLmKqGdV
	vavsDrKSe7jSMu7TG1x2Ox0JECCwI6Nw3aLQYpI=
X-Google-Smtp-Source: ABdhPJxqUIQ7WKhJ/OE0Bohn++ZsqNcpsSC5OrCJt2i2aOzMsMYaOXyxXxXYDxI2fJLQWQ+aNyomhjiQhQ3VV0gBqMo=
X-Received: by 2002:a17:907:2639:: with SMTP id aq25mr2717035ejc.138.1632365820261;
 Wed, 22 Sep 2021 19:57:00 -0700 (PDT)
MIME-Version: 1.0
References: <osstest-164996-mainreport@xen.org> <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com> <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com> <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Thu, 23 Sep 2021 07:56:48 +0500
Message-ID: <CAJ=z9a1W9BqeOGFu3F2+qbXM6UNo8ZO=0y-zYsnK01MNSNdbkw@mail.gmail.com>
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, Ian Jackson <iwj@xenproject.org>, 
	xen-devel <xen-devel@lists.xenproject.org>, dpsmith@apertussolutions.com
Content-Type: multipart/alternative; boundary="000000000000dc38e905cca0c88b"

--000000000000dc38e905cca0c88b
Content-Type: text/plain; charset="UTF-8"

Hi,

Sorry for the formatting.


On Thu, 23 Sep 2021, 06:10 Stefano Stabellini, <sstabellini@kernel.org>
wrote:

> On Wed, 22 Sep 2021, Jan Beulich wrote:
> > On 22.09.2021 01:38, Stefano Stabellini wrote:
> > > On Mon, 20 Sep 2021, Ian Jackson wrote:
> > >> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions -
> FAIL"):
> > >>> As per
> > >>>
> > >>> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
> > >>> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639
> inactive_anon:15857 isolated_anon:0
> > >>> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286
> inactive_file:11182 isolated_file:0
> > >>> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30
> writeback:0 unstable:0
> > >>> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922
> slab_unreclaimable:30234
> > >>> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975
> pagetables:401 bounce:0
> > >>> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100
> free_cma:1650
> > >>>
> > >>> the system doesn't look to really be out of memory; as per
> > >>>
> > >>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB
> (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C)
> 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
> > >>>
> > >>> there even look to be a number of higher order pages available
> (albeit
> > >>> without digging I can't tell what "(C)" means). Nevertheless order-4
> > >>> allocations aren't really nice.
> > >>
> > >> The host history suggests this may possibly be related to a qemu
> update.
> > >>
> > >>
> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
> >
> > Stefano - as per some of your investigation detailed further down I
> > wonder whether you had seen this part of Ian's reply. (Question of
> > course then is how that qemu update had managed to get pushed.)
> >
> > >> The grub cfg has this:
> > >>
> > >>  multiboot /xen placeholder conswitch=x watchdog noreboot
> async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan
> ${xen_rm_opts}
> > >>
> > >> It's not clear to me whether xen_rm_opts is "" or "no-real-mode
> edd=off".
> > >
> > > I definitely recommend to increase dom0 memory, especially as I guess
> > > the box is going to have a significant amount, far more than 4GB. I
> > > would set it to 2GB. Also the syntax on ARM is simpler, so it should be
> > > just: dom0_mem=2G
> >
> > Ian - I guess that's an adjustment relatively easy to make? I wonder
> > though whether we wouldn't want to address the underlying issue first.
> > Presumably not, because the fix would likely take quite some time to
> > propagate suitably. Yet if not, we will want to have some way of
> > verifying that an eventual fix there would have helped here.
> >
> > > In addition, I also did some investigation just in case there is
> > > actually a bug in the code and it is not a simple OOM problem.
> >
> > I think the actual issue is quite clear; what I'm struggling with is
> > why we weren't hit by it earlier.
> >
> > As imo always, non-order-0 allocations (perhaps excluding the bringing
> > up of the kernel or whichever entity) are to be avoided it at possible.
> > The offender in this case looks to be privcmd's alloc_empty_pages().
> > For it to request through kcalloc() what ends up being an order-4
> > allocation, the original IOCTL_PRIVCMD_MMAPBATCH must specify a pretty
> > large chunk of guest memory to get mapped. Which may in turn be
> > questionable, but I'm afraid I don't have the time to try to drill
> > down where that request is coming from and whether that also wouldn't
> > better be split up.
> >
> > The solution looks simple enough - convert from kcalloc() to kvcalloc().
> > I can certainly spin up a patch to Linux to this effect. Yet that still
> > won't answer the question of why this issue has popped up all of the
> > sudden (and hence whether there are things wanting changing elsewhere
> > as well).
>
> Also, I saw your patches for Linux. Let's say that the patches are
> reviewed and enqueued immediately to be sent to Linus at the next
> opportunity. It is going to take a while for them to take effect in
> OSSTest, unless we import them somehow in the Linux tree used by OSSTest
> straight away, right?
>

For Arm testing we don't use a branch provided by Linux upstream. So your
wait will be forever :).


> Should we arrange for one test OSSTest flight now with the patches
> applied to see if they actually fix the issue? Otherwise we might end up
> waiting for nothing...


We could push the patch in the branch we have. However the Linux we use is
not fairly old (I think I did a push last year) and not even the latest
stable.

I can't remember whether we still have some patches on top of Linux to run
on arm (specifically 32-bit). So maybe we should start to track upstream
instead?

This will have the benefits to pick any new patches.

Cheers,

.





>

--000000000000dc38e905cca0c88b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div>Hi,<div dir=3D"auto"><br></div><div dir=3D"auto">Sor=
ry for the formatting.</div><br><br><div class=3D"gmail_quote"><div dir=3D"=
ltr" class=3D"gmail_attr">On Thu, 23 Sep 2021, 06:10 Stefano Stabellini, &l=
t;<a href=3D"mailto:sstabellini@kernel.org">sstabellini@kernel.org</a>&gt; =
wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex">On Wed, 22 Sep 2021, Jan Beu=
lich wrote:<br>
&gt; On 22.09.2021 01:38, Stefano Stabellini wrote:<br>
&gt; &gt; On Mon, 20 Sep 2021, Ian Jackson wrote:<br>
&gt; &gt;&gt; Jan Beulich writes (&quot;Re: [xen-unstable test] 164996: reg=
ressions - FAIL&quot;):<br>
&gt; &gt;&gt;&gt; As per<br>
&gt; &gt;&gt;&gt;<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 in=
active_anon:15857 isolated_anon:0<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.514480 [ 1613.324918]=C2=A0 active_file:1=
3286 inactive_file:11182 isolated_file:0<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.514545 [ 1613.324918]=C2=A0 unevictable:0=
 dirty:30 writeback:0 unstable:0<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.526477 [ 1613.324918]=C2=A0 slab_reclaima=
ble:10922 slab_unreclaimable:30234<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.526540 [ 1613.324918]=C2=A0 mapped:11277 =
shmem:10975 pagetables:401 bounce:0<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.538474 [ 1613.324918]=C2=A0 free:8364 fre=
e_pcp:100 free_cma:1650<br>
&gt; &gt;&gt;&gt;<br>
&gt; &gt;&gt;&gt; the system doesn&#39;t look to really be out of memory; a=
s per<br>
&gt; &gt;&gt;&gt;<br>
&gt; &gt;&gt;&gt; Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UM=
EC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*2=
56kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB =3D 33456kB<br>
&gt; &gt;&gt;&gt;<br>
&gt; &gt;&gt;&gt; there even look to be a number of higher order pages avai=
lable (albeit<br>
&gt; &gt;&gt;&gt; without digging I can&#39;t tell what &quot;(C)&quot; mea=
ns). Nevertheless order-4<br>
&gt; &gt;&gt;&gt; allocations aren&#39;t really nice.<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt; The host history suggests this may possibly be related to a q=
emu update.<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt; <a href=3D"http://logs.test-lab.xenproject.org/osstest/result=
s/host/rochester0.html" rel=3D"noreferrer noreferrer" target=3D"_blank">htt=
p://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html</a><b=
r>
&gt; <br>
&gt; Stefano - as per some of your investigation detailed further down I<br=
>
&gt; wonder whether you had seen this part of Ian&#39;s reply. (Question of=
<br>
&gt; course then is how that qemu update had managed to get pushed.)<br>
&gt; <br>
&gt; &gt;&gt; The grub cfg has this:<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt;=C2=A0 multiboot /xen placeholder conswitch=3Dx watchdog noreb=
oot async-show-all console=3Ddtuart dom0_mem=3D512M,max:512M ucode=3Dscan=
=C2=A0 ${xen_rm_opts}<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt; It&#39;s not clear to me whether xen_rm_opts is &quot;&quot; =
or &quot;no-real-mode edd=3Doff&quot;.<br>
&gt; &gt; <br>
&gt; &gt; I definitely recommend to increase dom0 memory, especially as I g=
uess<br>
&gt; &gt; the box is going to have a significant amount, far more than 4GB.=
 I<br>
&gt; &gt; would set it to 2GB. Also the syntax on ARM is simpler, so it sho=
uld be<br>
&gt; &gt; just: dom0_mem=3D2G<br>
&gt; <br>
&gt; Ian - I guess that&#39;s an adjustment relatively easy to make? I wond=
er<br>
&gt; though whether we wouldn&#39;t want to address the underlying issue fi=
rst.<br>
&gt; Presumably not, because the fix would likely take quite some time to<b=
r>
&gt; propagate suitably. Yet if not, we will want to have some way of<br>
&gt; verifying that an eventual fix there would have helped here.<br>
&gt; <br>
&gt; &gt; In addition, I also did some investigation just in case there is<=
br>
&gt; &gt; actually a bug in the code and it is not a simple OOM problem.<br=
>
&gt; <br>
&gt; I think the actual issue is quite clear; what I&#39;m struggling with =
is<br>
&gt; why we weren&#39;t hit by it earlier.<br>
&gt; <br>
&gt; As imo always, non-order-0 allocations (perhaps excluding the bringing=
<br>
&gt; up of the kernel or whichever entity) are to be avoided it at possible=
.<br>
&gt; The offender in this case looks to be privcmd&#39;s alloc_empty_pages(=
).<br>
&gt; For it to request through kcalloc() what ends up being an order-4<br>
&gt; allocation, the original IOCTL_PRIVCMD_MMAPBATCH must specify a pretty=
<br>
&gt; large chunk of guest memory to get mapped. Which may in turn be<br>
&gt; questionable, but I&#39;m afraid I don&#39;t have the time to try to d=
rill<br>
&gt; down where that request is coming from and whether that also wouldn&#3=
9;t<br>
&gt; better be split up.<br>
&gt; <br>
&gt; The solution looks simple enough - convert from kcalloc() to kvcalloc(=
).<br>
&gt; I can certainly spin up a patch to Linux to this effect. Yet that stil=
l<br>
&gt; won&#39;t answer the question of why this issue has popped up all of t=
he<br>
&gt; sudden (and hence whether there are things wanting changing elsewhere<=
br>
&gt; as well).<br>
<br>
Also, I saw your patches for Linux. Let&#39;s say that the patches are<br>
reviewed and enqueued immediately to be sent to Linus at the next<br>
opportunity. It is going to take a while for them to take effect in<br>
OSSTest, unless we import them somehow in the Linux tree used by OSSTest<br=
>
straight away, right?<br></blockquote></div></div><div dir=3D"auto"><br></d=
iv><div dir=3D"auto">For Arm testing we don&#39;t use a branch provided by =
Linux upstream. So your wait will be forever :).</div><div dir=3D"auto"><br=
></div><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gm=
ail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-le=
ft:1ex">
<br>
Should we arrange for one test OSSTest flight now with the patches<br>
applied to see if they actually fix the issue? Otherwise we might end up<br=
>
waiting for nothing...</blockquote></div></div><div dir=3D"auto"><br></div>=
<div dir=3D"auto">We could push the patch in the branch we have. However th=
e Linux we use is not fairly old (I think I did a push last year) and not e=
ven the latest stable.</div><div dir=3D"auto"><br></div><div dir=3D"auto">I=
 can&#39;t remember whether we still have some patches on top of Linux to r=
un on arm (specifically 32-bit). So maybe we should start to track upstream=
 instead?</div><div dir=3D"auto"><br></div><div dir=3D"auto">This will have=
 the benefits to pick any new patches.</div><div dir=3D"auto"><br></div><di=
v dir=3D"auto">Cheers,</div><div dir=3D"auto"><br></div><div dir=3D"auto">.=
</div><div dir=3D"auto"><br></div><div dir=3D"auto"></div><div dir=3D"auto"=
><br></div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div><div dir=
=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
</blockquote></div></div></div>

--000000000000dc38e905cca0c88b--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:11:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193309.344315 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9G-0004AQ-ND; Thu, 23 Sep 2021 03:11:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193309.344315; Thu, 23 Sep 2021 03:11:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9G-0004AJ-JS; Thu, 23 Sep 2021 03:11:46 +0000
Received: by outflank-mailman (input) for mailman id 193309;
 Thu, 23 Sep 2021 03:11:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9F-0004AD-EB
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:11:45 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.43]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0a9ce09e-c628-4d60-80c5-337842705fda;
 Thu, 23 Sep 2021 03:11:42 +0000 (UTC)
Received: from DU2PR04CA0313.eurprd04.prod.outlook.com (2603:10a6:10:2b5::18)
 by PAXPR08MB6638.eurprd08.prod.outlook.com (2603:10a6:102:df::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 03:11:39 +0000
Received: from DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::80) by DU2PR04CA0313.outlook.office365.com
 (2603:10a6:10:2b5::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:39 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT019.mail.protection.outlook.com (10.152.20.163) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:39 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 03:11:38 +0000
Received: from e1e21acaab68.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3C225115-2288-43D4-BBD2-EDE3A62BF579.1; 
 Thu, 23 Sep 2021 03:11:25 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e1e21acaab68.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:25 +0000
Received: from FR0P281CA0060.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::17)
 by PAXPR08MB7188.eurprd08.prod.outlook.com (2603:10a6:102:20a::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:23 +0000
Received: from VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:d10:49:cafe::bb) by FR0P281CA0060.outlook.office365.com
 (2603:10a6:d10:49::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:23 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT004.mail.protection.outlook.com (10.152.18.106) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:22 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:21 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a9ce09e-c628-4d60-80c5-337842705fda
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UAeAMVF5g6MVjQKqmQi4d01qiiA0tPwK6j/l9Ubz7bc=;
 b=7ifj2UjVWtACj8FNlNiFQj2z0Czaqa6uCMEriQwsRASyNiQi04+q2n8SAatsWG7oV2G6M5x5sO98PMpYtwrOtRCpPhr1tX4Dr7rQsaGyure8wmkJBi9/Y/gmNYCN6PLnAZxYRgaemc1stFoCoVVdnri3YxdvgjqSy8fzodPGkyY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f50ebabb71f23e03
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gnWi49FVyhZfhVGKqJN6eObkVj2pjqinxpwZuQhMi+lzt+HZ7Kxa5NHmMcKatxOQaNQCRJ5RKdKmOwC6R538ZiD6+h0vVbFBpQvXCvP8J6pdyEWMAWle9faqzJ2g56y4rDPVqeJHL2TbTkJ9n5iPFABU5SmcN+0/vHKElxA5mwvFIFxSu7k10dy31a3wx2D5Q4vQnnPmLBkiSyZtXt24GZ22eZOxcci9a9s+A6ONE0ILdSwKW6ZW4fWdry9LUF+afPiSM68ngW4YYMRc+EtY1xwnvbMSnTLSap3X6OJ5zBPtkZg/1t91YbFg/jnbjW9Y1JmancymEoqtF/UYUG1CmA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UAeAMVF5g6MVjQKqmQi4d01qiiA0tPwK6j/l9Ubz7bc=;
 b=l4jZ0ylVLBSshdnKzuOk+6a8MEEYor4TK7hkByV6kZhPEPFqSMcVjezyX0dK0kMxiSfvGTsveQw9DNDXS17y4/bPc4G7jzQjgzUmg893VH5MibSg5w/k6a37E9gsNA/gagZbavNBuogGaVRVBIRjB3dFg/Al2TsimVMp3mLeTcTiLZ3ZtVxN1VeAteSV3Yt/cwucwQ1fD70a3VdkdpBbIaeaZXIFhBAJkEGfhAkQcCYHFK2VzcEtJXcmXlQY86eWxTIKE53bmkAzxCtVqanijMFrcdp1h5QbIOu9x+V+L+IHu4NPUvyMOwCoWkYx3JGpn4hZRWrS1PymXsa2Jca//g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UAeAMVF5g6MVjQKqmQi4d01qiiA0tPwK6j/l9Ubz7bc=;
 b=7ifj2UjVWtACj8FNlNiFQj2z0Czaqa6uCMEriQwsRASyNiQi04+q2n8SAatsWG7oV2G6M5x5sO98PMpYtwrOtRCpPhr1tX4Dr7rQsaGyure8wmkJBi9/Y/gmNYCN6PLnAZxYRgaemc1stFoCoVVdnri3YxdvgjqSy8fzodPGkyY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 00/11] 1:1 direct-map memory map
Date: Thu, 23 Sep 2021 03:11:04 +0000
Message-ID: <20210923031115.1429719-1-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 56ef7f50-d3be-4561-68e1-08d97e3fdc60
X-MS-TrafficTypeDiagnostic: PAXPR08MB7188:|PAXPR08MB6638:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB663897C90127CB2E9493852CF7A39@PAXPR08MB6638.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 S3rdXHINb2KiAHWnOVyWOcyCMRXg/ATHFPyFjx0/u4f5ra8hBMUVzrmhdftBERiSMKT4j87TMBaM3WTIe3cUhiEw/WufpJ8MtQcSuzSEyXB8QNAkpkY5F1c67r7B+A8BCeVPG6iMotEzCi2nrqa0hf28Aw4R+r4soOUDk7c5gUnPfRPEvfVfT0NU9dL/fsVumvOXk1n55bEeTJZ2Az7w1D7QiiYS9+pMNDdYngmtmBCZF2Z/vXq3xBRb7bsrN8nvEzHAFIfffHUngD3xtdCv92S3UIxFs/69oE9hQGOfY7TtdgSWiABIC6dZOBcVQMl5X/6sC1Efu/weWC+HxDr9DsfIiaS8CWZXWQFGSwektYiqdp/P7Ke5xXfBcHZw98nePLLocHPVSbzbN5ZrhnhqIbZJHRpXjdbIEM7ifLnvQEX0bEnlfzoTjcQsyr0D8EAJGuKAMT/m5q5Gpv8TCC0hGcCvF4VmagwaxwN+8tzk3FYumgXg9SzPCJ0JrYV23zMHbsg8aBkxtPxzesTx7iRyK1PKVmv+bvBpf1WYVC6YilHUUzTzxSGWDPz0YpjgtupnQI8yf57JQAFO16+UxKsJdjd7f2BYuzAu0qp0EOSKBfZNbY2MUXNTh0KR2DsRV4hqUHtdCFp8aPbkpja0tDrmXWq5M1QWAVB202AEwGWxntrEbsO3+qi8tOfq16tMF2iTrqhvkScurDz/uGq0oCDw9h8QKvwCUdkXjHYDNmSOot6nXo3tMdmJU6xIPgmdLClfCb9m+EUUQYy7qlEyez6kMAO3aZxbB5kkXaBr/ruLhwStSwwY32eIa9xk4SPYn6Q8sG5GC8vU8X5VvMzgz1BdzDhLqIBRslU+DBt1n8bLShI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(83380400001)(6666004)(70206006)(7696005)(2906002)(36756003)(8676002)(81166007)(356005)(508600001)(110136005)(966005)(8936002)(47076005)(1076003)(54906003)(86362001)(26005)(336012)(36860700001)(5660300002)(316002)(4326008)(44832011)(70586007)(2616005)(186003)(426003)(82310400003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7188
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	88378815-0ee5-47a6-3559-08d97e3fd273
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4WffGx/xVimjsSiPPHUU0M3/L+4OHBYfdNxHdly5LY+B1fyT0zmaanJwP8u6FfSjEhvmLIDN99pBkjN8Ae0bX+MMt48adIP/K4fVPFQ8m6XYoyZDfp6IXlqlfAmo+xm/5lh1Cqf8gFp+gkuCuuGfpQfcVZxst1hNLZWx+e1y98+scwj/1dTG/2Fjztva8Sq/Hd5bjP7pSMbDxlu6/eK8mFC07NoAp2lGOd5Z0unh4gLw4gAA8jcLzrXzca6TISe+sdKcTElCoxo2QasszxgiJB3F7o3ZkIpA27Lbg+KGXS26vw2ILtC7Om0klRP5LZx6NO+w0B8P6z9kQm7yocvYGDQ+q3vs38HedeRdHxwsnlGBRZ9Nc1NFSaDBeFV39S9YhHjQ2aJ/qTuVz942tQd7AQP9snsBjq3Cp2acwx+QJiFP9z2CI0fsC9UA/vE4CyrhWtj5auBs571cIYbE4p6yG2YMOcmo3cnfhomaKAW+jjfEzsI0jqBJFu/ozIjJkZkD9YIgxioNkhVp//czljdNaNUmyIvXbM9OXfYuwQhKoH4/QcDQlGaLSYGdOS3sAUmfV+uiUK6m70dOozZKm9QIdIzI+C+hAUJS3uJn8sk6bFb7BQiyzH6pstGVPR+XE9swr77E8gplkVWhYk7EX0d4NET5ocivYAMdvhKGI4i6WM+13rM5HGFM3ot+fKuME/7iNNXNziwxtoe4TeLYHS3CmPW4VliPUejbbp5pL/q/m8gghksuMTWxq3ybE43K0w3GyARHWXtPFHDARemB9NLPLCHjyKeRD3Rqi3odD0Pg1m09TrtG6yR6JakulQOHSi0QcPd1G/prdsG+ENJ6gGITdw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(110136005)(336012)(70586007)(81166007)(966005)(186003)(36860700001)(82310400003)(1076003)(36756003)(54906003)(2906002)(7696005)(8936002)(86362001)(70206006)(8676002)(508600001)(5660300002)(6666004)(26005)(316002)(83380400001)(2616005)(426003)(47076005)(44832011)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:39.4921
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 56ef7f50-d3be-4561-68e1-08d97e3fdc60
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6638

Cases where domU needs 1:1 direct-map memory map:
  * IOMMU not present in the system.
  * IOMMU disabled if it doesn't cover a specific device and all the guests
are trusted. Thinking a mixed scenario, where a few devices with IOMMU and
a few without, then guest DMA security still could not be totally guaranteed.
So users may want to disable the IOMMU, to at least gain some performance
improvement from IOMMU disabled.
  * IOMMU disabled as a workaround when it doesn't have enough bandwidth.
To be specific, in a few extreme situation, when multiple devices do DMA
concurrently, these requests may exceed IOMMU's transmission capacity.
  * IOMMU disabled when it adds too much latency on DMA. For example,
TLB may be missing in some IOMMU hardware, which may bring latency in DMA
progress, so users may want to disable it in some realtime scenario.

*WARNING:
Users should be aware that it is not always secure to assign a device without
IOMMU protection.
When the device is not protected by the IOMMU, the administrator should make
sure that:
 1. The device is assigned to a trusted guest.
 2. Users have additional security mechanism on the platform.

Requesting 1:1 memory mapping for the domain, when IOMMU is absent from the
system or it is disabled (status = "disabled" in device tree). In which
case, "direct-map" property is added under the appropriate domain node.

Right now, 1:1 direct-map is only supported when domain on Static Allocation,
that is, "xen,static-mem" is also necessary in the domain configuration.

Looking into related [design link](
https://lists.xenproject.org/archives/html/xen-devel/2021-05/msg00882.html)
for more details.

The whole design is about Static Allocation and 1:1 direct-map, and this
Patch Serie only covers parts of it, which are 1:1 direct-map memory map.
Other features will be delievered through different patch series.

See https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00855.html
for Domain on Static Allocation.

Penny Zheng (1):
  xen/arm: device assignment on 1:1 direct-map domain

Stefano Stabellini (10):
  xen: reserve flags for internal usage in xen_domctl_createdomain
  xen/arm: introduce XEN_DOMCTL_INTERNAL_directmap
  xen/arm: introduce 1:1 direct-map for domUs
  xen/arm: introduce accessors for vgic dist, cpu, and rdist base
    addresses
  xen/arm: vgic: introduce vgic.cbase
  xen/arm: new vgic: update vgic_cpu_base
  xen/arm: if 1:1 direct-map domain use native addresses for GICv2
  xen/arm: if 1:1 direct-map domain use native addresses for GICv3
  xen/arm: if 1:1 direct-map domain use native UART address and IRQ
    number for vPL011
  xen/docs: add a document to explain how to do passthrough without
    IOMMU

 docs/misc/arm/device-tree/booting.txt |   9 ++
 docs/misc/arm/passthrough-noiommu.txt |  54 +++++++
 xen/arch/arm/domain.c                 |   3 +-
 xen/arch/arm/domain_build.c           | 219 ++++++++++++++++++--------
 xen/arch/arm/vgic-v2.c                |  26 ++-
 xen/arch/arm/vgic-v3.c                |  10 +-
 xen/arch/arm/vgic/vgic-v2.c           |  27 +++-
 xen/arch/arm/vpl011.c                 |  34 +++-
 xen/arch/x86/setup.c                  |   4 +-
 xen/common/domain.c                   |  19 ++-
 xen/common/domctl.c                   |   3 +-
 xen/common/sched/core.c               |   2 +-
 xen/include/asm-arm/domain.h          |   9 +-
 xen/include/asm-arm/new_vgic.h        |  24 +++
 xen/include/asm-arm/vgic.h            |  42 +++++
 xen/include/asm-arm/vpl011.h          |   2 +
 xen/include/public/domctl.h           |   3 +
 xen/include/xen/domain.h              |   8 +
 xen/include/xen/sched.h               |   3 +-
 19 files changed, 394 insertions(+), 107 deletions(-)
 create mode 100644 docs/misc/arm/passthrough-noiommu.txt

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:11:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193310.344327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9K-0004RQ-VR; Thu, 23 Sep 2021 03:11:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193310.344327; Thu, 23 Sep 2021 03:11:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9K-0004RJ-S0; Thu, 23 Sep 2021 03:11:50 +0000
Received: by outflank-mailman (input) for mailman id 193310;
 Thu, 23 Sep 2021 03:11:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9J-0004Qj-IH
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:11:49 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.50]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fcac06c7-1c1b-11ec-ba0a-12813bfff9fa;
 Thu, 23 Sep 2021 03:11:48 +0000 (UTC)
Received: from AM6PR0502CA0045.eurprd05.prod.outlook.com
 (2603:10a6:20b:56::22) by AM6PR08MB3301.eurprd08.prod.outlook.com
 (2603:10a6:209:47::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:45 +0000
Received: from AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:56:cafe::61) by AM6PR0502CA0045.outlook.office365.com
 (2603:10a6:20b:56::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:45 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT059.mail.protection.outlook.com (10.152.17.193) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:45 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 03:11:44 +0000
Received: from 7c3ccbc109e1.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3601AE50-2D27-433A-9DD1-36EFAED7F40C.1; 
 Thu, 23 Sep 2021 03:11:37 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7c3ccbc109e1.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:37 +0000
Received: from AM5PR1001CA0044.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::21) by HE1PR0802MB2300.eurprd08.prod.outlook.com
 (2603:10a6:3:c5::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 03:11:35 +0000
Received: from VE1EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:15:cafe::20) by AM5PR1001CA0044.outlook.office365.com
 (2603:10a6:206:15::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:35 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT046.mail.protection.outlook.com (10.152.19.226) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:35 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:33 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcac06c7-1c1b-11ec-ba0a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8mymiVU6RzNe2URN+GFVNzglxdb0Bxcz77lG7C4aSLI=;
 b=dxA0TeTBVgUDOEWhgjUD7WpUCdGHpIG0fdiIBqQueFcErs6GiHr6N34IvznhF2mocSJFrS1Imsm3INSOTicGcbEvSm/PMmdzdyvpdxasMWptvh0t0jqFpqaYQjigg5Lfqu2qBQ79rx6kK2lWwiivilLybPDHlrJjlF6NiaIu8xw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 78d99569b4e49507
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kp7JW3SPkZTY5hxQi8HAeL0s7uw8wnulE0FaPPL8AjcvN0DrZoKlufiGxpu9UKXI09K3q9J+y70unC9EFlYkqlWDQXHcLlYcFakMHcCFY/hANqI2+/29MbNiq8Om+F/ytolv9hTyWlKxRADUDP14ef01+JvR1b6+5C94zcvBjQTq0uV4q8k+Ixlblvmwy64kJIPyUB9sGtArg1HHK2cPbA4eSOz+yVOIcGuWI3udj5nei6slZH2P6ZRquHT7P7j8d/tJf+hzOzxlPau02UIuxUif695u90IX2k4ZSUYVusW84+5Q5m2xTiXYPQi3QvHYxx/g2r3FDBMSqe8CPID5IA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8mymiVU6RzNe2URN+GFVNzglxdb0Bxcz77lG7C4aSLI=;
 b=TCiv/3ClcRi+R9WXzCpnoN3MYeWqX5KUQeED5q7b9AwXS/WIG4mElCxKsQb4RWW8Iw03BbL+ATCIiaeYcxiiPt6FFCDvzgsSz0cdB5oDuUkuN307Gh8ZxD0KlBS52Ax4T2byVXjOiGUdA37BUzV3ydPBaeGQXe+UgdMrtBsKklZN8dEpKhQ9oqsZzLaY+oRE7Y7lr46AjavlSdRkw3GuHBmqdUdjDtNzFevfWgw0UvCGbsI1DWWgHKBBGhtUAU9Oe07fxs+m7ot18IFmrKYw7BEyY2KcQd3jaiETPoAkqPz/oKqr2YRVwufmNq448qqc2Opwi81t3LdvNLlSUS5Xgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8mymiVU6RzNe2URN+GFVNzglxdb0Bxcz77lG7C4aSLI=;
 b=dxA0TeTBVgUDOEWhgjUD7WpUCdGHpIG0fdiIBqQueFcErs6GiHr6N34IvznhF2mocSJFrS1Imsm3INSOTicGcbEvSm/PMmdzdyvpdxasMWptvh0t0jqFpqaYQjigg5Lfqu2qBQ79rx6kK2lWwiivilLybPDHlrJjlF6NiaIu8xw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 05/11] xen/arm: vgic: introduce vgic.cbase
Date: Thu, 23 Sep 2021 03:11:09 +0000
Message-ID: <20210923031115.1429719-6-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 44e16a65-030b-4bfe-3ac7-08d97e3fdfe5
X-MS-TrafficTypeDiagnostic: HE1PR0802MB2300:|AM6PR08MB3301:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB3301B46AA588A0A6228DA53FF7A39@AM6PR08MB3301.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 0DVkdkx1a7AggDmFp3WKDFBdv2YkJ7Juar9krB/+s6hCz3JySXZv6AtIHDPw9pwFWwdDthDdantcFNtUoQIjbaVZr6hSuHRpC6CMu2tZvZ9ie4y7VBjwKEDvDrYYydLH9r4z4Y3GBmziLPySaP+Yh2kbjzfWRzXuTts6n+T2qq1y0749Sv0uU/BGvSSZHFBwGPSId0fEefZTJLRTRQK/JYkxTQ1HsEt0oitVN0CCt8Lv4SsTrOsKJPhKoTbePRULUtm4OXBLvRDUjonYNJHzk/hh737zNLfoLbmKw4MBs/EFY/qo9R4eDBdH3x35p3XvnrgTSU+h1O+L1D4XWOvAeuZV0hZVHiBCLiE349N4d48UR3KeW0/VQNZZL3qJ+kbWsJ7qjhDMkzlxTFAvH/TgzxfGCsT0HEQA9iAlBUa5l1rXVnLoH7nh/RJ74Sc5J7/EsEBWm7jDPmBBDi8h3v7jOfLLtM5soY7NYsfgiPiZOKcc0oTG9LBYq9sC8tBFjhbibqLGDjRVeERldBJd4S8hlS6Ib94ra5KInZfVV29vfun3EJYvfvOfShRw1T/Rq6MDYve2dFTL9RkUWEw6acUjk/zr6a9DWANwuc9AO1EzaOXI+fd4XeLlSgZWKMmMBZxNq38ViMjm3cEInq1Ac1J4+Ylw2XDECJ5QpNzUz1g4aLMkcVjPO9EsVOCnT57QFofoW82+uB5sh/kNS22MhN/EqadvkfRJIaQta8U0gH40Tbo=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(2616005)(47076005)(36860700001)(81166007)(26005)(7696005)(36756003)(356005)(508600001)(8936002)(186003)(83380400001)(110136005)(70206006)(8676002)(44832011)(426003)(54906003)(336012)(4326008)(1076003)(6666004)(2906002)(82310400003)(70586007)(316002)(5660300002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2300
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	413c93a0-b8ba-4117-b2d3-08d97e3fd9db
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ssivSYRW/dvVt5d8EiHGb5YaRrbbDmchSLxVgoD9BheiAJ0aAr35JUaf+GXEqMPOtjzg2IBeEMGPfzgOCygdBtrng++wlmHL09FmalHks7MHKOcFevm16CbjhksQijn/S2H0Y1KR7SRaATKbjFma3nzORHIAlgOIxck+o56+c3R8QhcIrI+biQNPN8yMh8K4od16kkPrvtSkAIsLKnp08eA+eGZ4jOKjG997UjpzlAZO9MoBQWc5kELtONuLGQIea+TCvSlGlXBky1H6Wq6kq2tj5Z5zr8oUwBObgKhBN8mzBI6M3VcChLxY0mNJ7WhMI0Qyngyrcwl2RuY20HTzTDHBw2c2zSHGpzYpHfFc69REuEEd6vOmM1Omkf6gk4iHZr3/xAqmkx9CrrYXQvxct6eeB3Nv0VLdf3NlkMXFXa/dO2aNkytmzuZZ6VVAtSDwbQAslJXf6qT/PyYuBBXXSyd8RKeRZydyJ97S09bk8RaqOWaWZ0jj3HAV5cgzkPaD/D8Yz+EpcLSJx6UQUtcbVfhqSuTyEb6MAynUDQ/CTHWkkqk9asPMIC4d5+s5TrnPvl4idCUOLbMvvJ+OGlIDAcmwTFTU8CHCOoA9ZtmUrgCb1//IAJQEIoRtsAQI0PexWdsXfMSuXP0Kag55x/3JJGzCPCynU4ucRrndEOwbuomNYdtj0ixJDJyByBNJB6Xqo2cd8ONH0QsThETQ//Qp4A==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(54906003)(8676002)(70206006)(83380400001)(316002)(426003)(26005)(82310400003)(336012)(110136005)(7696005)(70586007)(44832011)(2616005)(5660300002)(186003)(36756003)(81166007)(47076005)(86362001)(1076003)(36860700001)(2906002)(8936002)(6666004)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:45.3373
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 44e16a65-030b-4bfe-3ac7-08d97e3fdfe5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3301

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Add a field in struct vgic_dist to store the cpu interface base address,
similar to the existing dbase field.

Use the field in vgic_v2_domain_init, instead of original local variable.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/vgic-v2.c     | 10 +++++-----
 xen/include/asm-arm/vgic.h |  1 +
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index b2da886adc..b34ec88106 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -652,7 +652,7 @@ static int vgic_v2_vcpu_init(struct vcpu *v)
 static int vgic_v2_domain_init(struct domain *d)
 {
     int ret;
-    paddr_t cbase, csize;
+    paddr_t csize;
     paddr_t vbase;
 
     /*
@@ -669,7 +669,7 @@ static int vgic_v2_domain_init(struct domain *d)
          * Note that we assume the size of the CPU interface is always
          * aligned to PAGE_SIZE.
          */
-        cbase = vgic_v2_hw.cbase;
+        d->arch.vgic.cbase = vgic_v2_hw.cbase;
         csize = vgic_v2_hw.csize;
         vbase = vgic_v2_hw.vbase;
     }
@@ -683,7 +683,7 @@ static int vgic_v2_domain_init(struct domain *d)
          * region.
          */
         BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
-        cbase = GUEST_GICC_BASE;
+        d->arch.vgic.cbase = GUEST_GICC_BASE;
         csize = GUEST_GICC_SIZE;
         vbase = vgic_v2_hw.vbase + vgic_v2_hw.aliased_offset;
     }
@@ -692,8 +692,8 @@ static int vgic_v2_domain_init(struct domain *d)
      * Map the gic virtual cpu interface in the gic cpu interface
      * region of the guest.
      */
-    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
-                           maddr_to_mfn(vbase));
+    ret = map_mmio_regions(d, gaddr_to_gfn(d->arch.vgic.cbase),
+                           csize / PAGE_SIZE, maddr_to_mfn(vbase));
     if ( ret )
         return ret;
 
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index e1bc5113da..d5ad1f387b 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -152,6 +152,7 @@ struct vgic_dist {
     struct pending_irq *pending_irqs;
     /* Base address for guest GIC */
     paddr_t dbase; /* Distributor base address */
+    paddr_t cbase; /* CPU interface base address */
 #ifdef CONFIG_GICV3
     /* GIC V3 addressing */
     /* List of contiguous occupied by the redistributors */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:11:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:11:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193311.344338 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9Q-0004kw-8v; Thu, 23 Sep 2021 03:11:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193311.344338; Thu, 23 Sep 2021 03:11:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9Q-0004km-4v; Thu, 23 Sep 2021 03:11:56 +0000
Received: by outflank-mailman (input) for mailman id 193311;
 Thu, 23 Sep 2021 03:11:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9O-0004Qj-DU
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:11:54 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.72]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff7c5433-1c1b-11ec-ba0a-12813bfff9fa;
 Thu, 23 Sep 2021 03:11:52 +0000 (UTC)
Received: from AM6PR0502CA0043.eurprd05.prod.outlook.com
 (2603:10a6:20b:56::20) by DBBPR08MB4361.eurprd08.prod.outlook.com
 (2603:10a6:10:c8::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:41 +0000
Received: from AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:56:cafe::b3) by AM6PR0502CA0043.outlook.office365.com
 (2603:10a6:20b:56::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:41 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT059.mail.protection.outlook.com (10.152.17.193) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:41 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 03:11:39 +0000
Received: from 762f163a0586.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 CA37131F-64B9-409C-B00E-9178C79C7F7C.1; 
 Thu, 23 Sep 2021 03:11:27 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 762f163a0586.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:27 +0000
Received: from AS8PR04CA0068.eurprd04.prod.outlook.com (2603:10a6:20b:313::13)
 by DB9PR08MB7050.eurprd08.prod.outlook.com (2603:10a6:10:2bd::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:25 +0000
Received: from VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::d5) by AS8PR04CA0068.outlook.office365.com
 (2603:10a6:20b:313::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:25 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT039.mail.protection.outlook.com (10.152.19.196) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:25 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:24 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 03:11:23 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff7c5433-1c1b-11ec-ba0a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PxKTjjjfvwxRnsbZJKvjVjurClpiQegEoLdVtMIbE48=;
 b=qElmi03Ic9TwxR4HBSDJDhHEUyaifwF/YbkCtycl0YqqkWwGzfw2pp9sl8lenQflnfejBPLpeUMWhollJPPMpBf9fbtEpMIfh7Oz6U/4fy68ej7jOLJF1MYicEoQGzVOBdLhHy9O3+51QBZ4AnCYcfkiFK+5YzL7i/JRLtTdCFk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8097e4d4a45c9b09
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z/4sG/owxdNQwWePWv3Hu+IOHrYJI4SEQ0kMGZ33kq6KPEjC3V54VRPb7vohA45yATFGQG/ZSJCo5zLB/nkreEN4CdPi9ILYrYXjpAb6To9VFH4R6xPjhNin69WtwEOckwrX4FSnJnvkxj59goxbMymIswcPwjL3QBDw3YYC/Ikk9YXMJPOxwsose8WwkkPVsCzWj5YtVPWkjcgxqdgWvlv1qAy/K8g3/bjasuvun2tLohTW1dsTjGm+7aViotCEXQaT7xNZx6jis8jWwAQ0CT5PrxtxgJD1BGm2iaCVt70XV4YZRdrclrKmQEj1CISgJ1WV/Blx0d7w4bjyZffGiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PxKTjjjfvwxRnsbZJKvjVjurClpiQegEoLdVtMIbE48=;
 b=MInrOKf3N5TExZXY6p7tgfpXiMKVUmNsk5GzrQrPBrHcEq4D7Fq+c3zvLC6HmwzX15VjImnymsRTpQE3yC/Ohjw6joDKkyqc3hXlarodaaxrNXr5s7CJn0yHGMMm30Xfte2OhLVDrQIV8JJDuyRFqYbIb70RQq/f5pTNj8DQr71pDbwo6OINjiNHYkGRP+0VWey0qvhe4E6HCXsjNqkImf1QftZ5JFmEbmQia2ntnyNwentSsKYQ6ibuFLVSPd/wDNDcFfy8WMV4TMI4mzQxvcr6j33cjdAFixPXGQgcVcOfu/WRr/3TFWIvWYQjN2J0Mog0vHlrb9vUHVGE3cmbAw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PxKTjjjfvwxRnsbZJKvjVjurClpiQegEoLdVtMIbE48=;
 b=qElmi03Ic9TwxR4HBSDJDhHEUyaifwF/YbkCtycl0YqqkWwGzfw2pp9sl8lenQflnfejBPLpeUMWhollJPPMpBf9fbtEpMIfh7Oz6U/4fy68ej7jOLJF1MYicEoQGzVOBdLhHy9O3+51QBZ4AnCYcfkiFK+5YzL7i/JRLtTdCFk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 01/11] xen: reserve flags for internal usage in xen_domctl_createdomain
Date: Thu, 23 Sep 2021 03:11:05 +0000
Message-ID: <20210923031115.1429719-2-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d8459c6c-cee4-4041-51bf-08d97e3fdd75
X-MS-TrafficTypeDiagnostic: DB9PR08MB7050:|DBBPR08MB4361:
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB43616343828A71DE52FEE5A8F7A39@DBBPR08MB4361.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 WIiGZflZU0eUsgMGCiC8xo9mr7vlf+kCGfnakbLFMgl7ZAZI2SE4mpkQ/0ToA07uOytpDqexWeNyvk9GDowhWwPsjbw0G5HjVAACXYyewYCXVLMIobrIpKcbdUIcJ2zEZ2vmvHWoJHQDtaxG3+E2j9GQI4zdBWJnOism3ZCy3O6hNDDtxlrVH07tE+/ZkJ+g6/dLqghS8K0rbRw9U1ZXec7aZRRea0XANvUhpSoAO7BJijyLLgy2/2KivJDybVQESLp5KrUGkex9ykz5m+AwTRfVe8zZmz1T6zudqxOuUO3yiJCGfxrP2hPIoZz71DFwZCbE0WwwGr1CUhp8slliL2YKd7KYki2PZsJuCO8llv3XLA3Z/P1MkIL67Puycipf1jFmhCZMBHoa+TbPc/NZ8ijsbvPhUO46/sYPA4KQMXHq2ozyFArKX4J+64zkQLmId4Ippk5kPKriZHL8hjNU8PFIBpRnkgeLb1UwjgLWvP2d8/Z55L02TWz+V3ZXPxlsu9EFc2zXQ3L8rbad/HQXbhYCjT0BM1Peox1Ye+nNX5rnCSVx+D9mAofFZzOFcB8srTvEDYZGZTV5S7JRrlqGOpcFb51mm8ILQJdtuI0JHwAICeulGLEjrt+QISfgHNZO+CUR2NBcedn6O90DLVeZZ4sOP3Z5VuOSamJmJuDEqfO2Y5WsZION++xxymhXDTacrO/XaqQTgM6yprWzV2lHOd8jM9fweyPYHP1YVOtuUE8=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70586007)(2906002)(356005)(316002)(36756003)(7696005)(8936002)(508600001)(70206006)(4326008)(186003)(81166007)(26005)(426003)(1076003)(47076005)(5660300002)(2616005)(82310400003)(336012)(6666004)(86362001)(8676002)(36860700001)(44832011)(54906003)(83380400001)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7050
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b13e7494-34b7-4986-5ef0-08d97e3fd422
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/q4d3YMs/iKQ0glHBwwvNkQoyTgRxELhoB+Joo1bAQYgNV9LdJ48XaouZeuDezpnlh4AnOtz7Ptnya/HreqIMo7vBKU5uykGoKuRQ3Ip3Ar/1xhKAqtOeJVK8uBMcXBwHORt811fE5YcEzNXHu0ijRIUe1EdLTdcdrUJisp53zyXj0Madn6nQgJ6crtzQ4Ucix0gsjyQP+i3JB3xroi//CrRpJCv5WQ8XW57b4RmCG0MWnlgMJyK2Qkn001zReFcc6XGohIanA0uOXRMc7C9/oyNIsTEMgkv1MeULw8nDo6LRMEW5ugG+ROdkNNOr1Qqs/K8Br6o6XjcJvC/4ivVEyXsVNQpOWc1cUq3UWpFA4O7r58W0mLeKYfOrmLQbd/LHYXwa5v/2cp4c0WZkomnuXqYZnP1H+aZR5c9uPJUGT77rbfs/1is/cWuw11PCMKlUpcOlOsBgcwenJQdYqTohe6ZsPiYo16y8JHFhhwdaFEZEk9Wa2KFAJrW+7q4moEfunU5qA5++YFH74OOUCUU3fjDsjHiHHWr+PRQ/QOpBmtghaL0KU6uJ0EF9A812ayfmPBBRIqNaoUMZU37D8VyFMBhoEx/9++d+8FG1TSBwVyZ1me4FCuwNgev5P2QXsGkuNdk0OZF2dP2o8Bjujp6sh4mU7IXesDXK9pxhQvxElTkX1IHyB2af+eBPTbi+dlqq1yfy1bB6D2aDsWbvlvd4Q==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(83380400001)(70586007)(7696005)(316002)(70206006)(426003)(82310400003)(4326008)(110136005)(508600001)(54906003)(44832011)(26005)(336012)(81166007)(86362001)(5660300002)(2616005)(186003)(36756003)(47076005)(1076003)(8676002)(36860700001)(8936002)(6666004)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:41.2280
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d8459c6c-cee4-4041-51bf-08d97e3fdd75
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT059.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4361

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

We are passing an extra special boolean flag at domain creation to
specify whether we want to the domain to be privileged (i.e. dom0) or
not. Another flag will be introduced later in this series.

Reserve bits 16-31 from the existing flags bitfield in struct
xen_domctl_createdomain for internal Xen usage.

Allocate bit 16 for XEN_DOMCTL_INTERNAL_ispriv: whether a domain is
privileged or not.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
CC: andrew.cooper3@citrix.com
CC: jbeulich@suse.com
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Wei Liu <wl@xen.org>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: George Dunlap <george.dunlap@eu.citrix.com>
CC: Dario Faggioli <dfaggioli@suse.com>
---
 xen/arch/arm/domain.c       |  2 +-
 xen/arch/arm/domain_build.c |  7 ++++---
 xen/arch/x86/setup.c        |  4 +++-
 xen/common/domain.c         | 19 +++++++++----------
 xen/common/domctl.c         |  3 ++-
 xen/common/sched/core.c     |  2 +-
 xen/include/public/domctl.h |  3 +++
 xen/include/xen/domain.h    |  4 ++++
 xen/include/xen/sched.h     |  3 +--
 9 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d..7922249d26 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -623,7 +623,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
     unsigned int max_vcpus;
 
     /* HVM and HAP must be set. IOMMU may or may not be */
-    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
+    if ( ((config->flags & XEN_DOMCTL_CDF_MASK) & ~XEN_DOMCTL_CDF_iommu) !=
          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d634c1..8cc4c800e9 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2679,7 +2679,7 @@ void __init create_domUs(void)
          * very important to use the pre-increment operator to call
          * domain_create() with a domid > 0. (domid == 0 is reserved for Dom0)
          */
-        d = domain_create(++max_init_domid, &d_cfg, false);
+        d = domain_create(++max_init_domid, &d_cfg);
         if ( IS_ERR(d) )
             panic("Error creating domain %s\n", dt_node_name(node));
 
@@ -2752,7 +2752,8 @@ void __init create_dom0(void)
 {
     struct domain *dom0;
     struct xen_domctl_createdomain dom0_cfg = {
-        .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap,
+        .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
+                 XEN_DOMCTL_INTERNAL_ispriv,
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
@@ -2773,7 +2774,7 @@ void __init create_dom0(void)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
-    dom0 = domain_create(0, &dom0_cfg, true);
+    dom0 = domain_create(0, &dom0_cfg);
     if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
         panic("Error creating domain 0\n");
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b101565f14..6b7a1a3994 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -770,9 +770,11 @@ static struct domain *__init create_dom0(const module_t *image,
 
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
+    if ( !pv_shim )
+        dom0_cfg.flags |= XEN_DOMCTL_INTERNAL_ispriv;
 
     /* Create initial domain 0. */
-    d = domain_create(get_initial_domain_id(), &dom0_cfg, !pv_shim);
+    d = domain_create(get_initial_domain_id(), &dom0_cfg);
     if ( IS_ERR(d) || (alloc_dom0_vcpu0(d) == NULL) )
         panic("Error creating domain 0\n");
 
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..5fcca9b018 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -475,11 +475,11 @@ static void _domain_destroy(struct domain *d)
 
 static int sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
-    bool hvm = config->flags & XEN_DOMCTL_CDF_hvm;
-    bool hap = config->flags & XEN_DOMCTL_CDF_hap;
-    bool iommu = config->flags & XEN_DOMCTL_CDF_iommu;
+    bool hvm = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_hvm;
+    bool hap = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_hap;
+    bool iommu = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_iommu;
 
-    if ( config->flags &
+    if ( (config->flags & XEN_DOMCTL_CDF_MASK) &
          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
@@ -536,8 +536,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
 }
 
 struct domain *domain_create(domid_t domid,
-                             struct xen_domctl_createdomain *config,
-                             bool is_priv)
+                             struct xen_domctl_createdomain *config)
 {
     struct domain *d, **pd, *old_hwdom = NULL;
     enum { INIT_watchdog = 1u<<1,
@@ -563,7 +562,7 @@ struct domain *domain_create(domid_t domid,
     }
 
     /* Sort out our idea of is_control_domain(). */
-    d->is_privileged = is_priv;
+    d->is_privileged = config ? config->flags & XEN_DOMCTL_INTERNAL_ispriv : 0;
 
     /* Sort out our idea of is_hardware_domain(). */
     if ( domid == 0 || domid == hardware_domid )
@@ -756,7 +755,7 @@ void __init setup_system_domains(void)
      * Hidden PCI devices will also be associated with this domain
      * (but be [partly] controlled by Dom0 nevertheless).
      */
-    dom_xen = domain_create(DOMID_XEN, NULL, false);
+    dom_xen = domain_create(DOMID_XEN, NULL);
     if ( IS_ERR(dom_xen) )
         panic("Failed to create d[XEN]: %ld\n", PTR_ERR(dom_xen));
 
@@ -766,7 +765,7 @@ void __init setup_system_domains(void)
      * array. Mappings occur at the priv of the caller.
      * Quarantined PCI devices will be associated with this domain.
      */
-    dom_io = domain_create(DOMID_IO, NULL, false);
+    dom_io = domain_create(DOMID_IO, NULL);
     if ( IS_ERR(dom_io) )
         panic("Failed to create d[IO]: %ld\n", PTR_ERR(dom_io));
 
@@ -775,7 +774,7 @@ void __init setup_system_domains(void)
      * Initialise our COW domain.
      * This domain owns sharable pages.
      */
-    dom_cow = domain_create(DOMID_COW, NULL, false);
+    dom_cow = domain_create(DOMID_COW, NULL);
     if ( IS_ERR(dom_cow) )
         panic("Failed to create d[COW]: %ld\n", PTR_ERR(dom_cow));
 #endif
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 12d6144d28..2ec6d454dd 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -431,7 +431,8 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
             rover = dom;
         }
 
-        d = domain_create(dom, &op->u.createdomain, false);
+        op->u.createdomain.flags &= XEN_DOMCTL_CDF_MASK;
+        d = domain_create(dom, &op->u.createdomain);
         if ( IS_ERR(d) )
         {
             ret = PTR_ERR(d);
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8f4b1ca10d..27d5bc2259 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -3021,7 +3021,7 @@ void __init scheduler_init(void)
         sched_ratelimit_us = SCHED_DEFAULT_RATELIMIT_US;
     }
 
-    idle_domain = domain_create(DOMID_IDLE, NULL, false);
+    idle_domain = domain_create(DOMID_IDLE, NULL);
     BUG_ON(IS_ERR(idle_domain));
     BUG_ON(nr_cpu_ids > ARRAY_SIZE(idle_vcpu));
     idle_domain->vcpu = idle_vcpu;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..4d3fcd3bcb 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -74,6 +74,9 @@ struct xen_domctl_createdomain {
 /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
 #define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
 
+/* Flags from (1U<<16) to (1U<<31) are reserved for internal usage */
+#define XEN_DOMCTL_CDF_MASK           (0xffff)
+
     uint32_t flags;
 
 #define _XEN_DOMCTL_IOMMU_no_sharept  0
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 1708c36964..7ed0b62b78 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -28,6 +28,10 @@ void getdomaininfo(struct domain *d, struct xen_domctl_getdomaininfo *info);
 void arch_get_domain_info(const struct domain *d,
                           struct xen_domctl_getdomaininfo *info);
 
+/* Flags are reserved for internal usage */
+#define _XEN_DOMCTL_INTERNAL_ispriv         16
+#define XEN_DOMCTL_INTERNAL_ispriv          (1U<<_XEN_DOMCTL_INTERNAL_ispriv)
+
 /*
  * Arch-specifics.
  */
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404..a659b25dea 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -664,8 +664,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config);
  * (domid < DOMID_FIRST_RESERVED).
  */
 struct domain *domain_create(domid_t domid,
-                             struct xen_domctl_createdomain *config,
-                             bool is_priv);
+                             struct xen_domctl_createdomain *config);
 
 /*
  * rcu_lock_domain_by_id() is more efficient than get_domain_by_id().
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:11:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:11:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193313.344349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9S-00054T-MK; Thu, 23 Sep 2021 03:11:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193313.344349; Thu, 23 Sep 2021 03:11:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9S-00054G-HY; Thu, 23 Sep 2021 03:11:58 +0000
Received: by outflank-mailman (input) for mailman id 193313;
 Thu, 23 Sep 2021 03:11:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9Q-0004kC-Mt
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:11:56 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe07::623])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 778e0bce-ad3f-4456-8a34-228c3e56888f;
 Thu, 23 Sep 2021 03:11:54 +0000 (UTC)
Received: from DB6PR07CA0101.eurprd07.prod.outlook.com (2603:10a6:6:2c::15) by
 PA4PR08MB5887.eurprd08.prod.outlook.com (2603:10a6:102:f2::23) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 03:11:53 +0000
Received: from DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2c:cafe::30) by DB6PR07CA0101.outlook.office365.com
 (2603:10a6:6:2c::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:53 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT060.mail.protection.outlook.com (10.152.21.231) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:53 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 03:11:52 +0000
Received: from 91027c4583da.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0A2E9B97-84CC-48F1-B068-6DF3A2608A30.1; 
 Thu, 23 Sep 2021 03:11:45 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 91027c4583da.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:45 +0000
Received: from AS9PR06CA0264.eurprd06.prod.outlook.com (2603:10a6:20b:45f::9)
 by AM0PR08MB3137.eurprd08.prod.outlook.com (2603:10a6:208:64::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:43 +0000
Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::cd) by AS9PR06CA0264.outlook.office365.com
 (2603:10a6:20b:45f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:43 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:43 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:38 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 778e0bce-ad3f-4456-8a34-228c3e56888f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e41YLk3dIbgdmwmpIt94ilkD1EszsH0czIzcF48m2hU=;
 b=DpeA9lX6eC9cJYKShAaXaM0mb2wSy4r7+a/sgcrVzireNfDh9QhtjsyEMQr98V1+/tsw2J6MSJA6M9feJfJtofmubAdRkusSnO1QgTWFF5rqmb3mrHh9/B1T6AfvHNGEgBBkS6rp+lAEFclfPn9rJOyTTM2wzAMsL/NC2g2KQaE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2e3f64499d581046
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lOgeWG4uMUCwoCIp0NFyr9RuM1AhSE6MHUBa89vTqSxCSDJJWPulZqY9vCZaMkMiH7wGCD40UWDWQ28FlqVU8fCoaaJL+DR9TFDTemRo4MGMCHXsHrROUdyJFLnBxjvarRur9/+QEmBMarM2i0RHdG7ZehXvrQxJy4JamRNyuED4Nsf4EfF+zbkUr7hfulHYJCIWs7/9S9/A+A3zE9U19QeE4jh/0S7+j3N2cvKYo2W/D/uurZ/Opg/IS104+zPqcqybEfWUSoUi/TYwHBZ4thrxLlxvdGmUQd8QhhHtiSgRivX6OTYJo4l2OTSzNPvsefoxsp4JNIXrSBCcEnlfjg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=e41YLk3dIbgdmwmpIt94ilkD1EszsH0czIzcF48m2hU=;
 b=jYzuEPIVgk3hWoT0P6/rYgZfZlkV+qQJ1W7E/Kw5WhvuSgk15Y1PqJDXKEALKXjARQ6OSKLSzDU8n2RX0I/e9lKDnLlbU20+QEWG4Ipzp6rmWRIDP1ILpgMqQS7zipCykZ/bPHyjOi4sMVk+aZbmftI6YlHwueryphYqLBvCUPuUPzgjac13Q2Ehm6g7UFRykS27O5H6/5UL/AJPPZvl5n+X7FN9YN2PfXA6sA3FXGO0EtodaMllcg+rbJFCJUKNT7o2BmJb591sOBEDaN75PUZHIHlBGCX1LxSuYGng8r549y5bM8jvMwsh7GQRCFWlw35ciBNWnRzBjpsr8qJblg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e41YLk3dIbgdmwmpIt94ilkD1EszsH0czIzcF48m2hU=;
 b=DpeA9lX6eC9cJYKShAaXaM0mb2wSy4r7+a/sgcrVzireNfDh9QhtjsyEMQr98V1+/tsw2J6MSJA6M9feJfJtofmubAdRkusSnO1QgTWFF5rqmb3mrHh9/B1T6AfvHNGEgBBkS6rp+lAEFclfPn9rJOyTTM2wzAMsL/NC2g2KQaE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 07/11] xen/arm: if 1:1 direct-map domain use native addresses for GICv2
Date: Thu, 23 Sep 2021 03:11:11 +0000
Message-ID: <20210923031115.1429719-8-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6142d902-90fa-4d54-2c0e-08d97e3fe47a
X-MS-TrafficTypeDiagnostic: AM0PR08MB3137:|PA4PR08MB5887:
X-Microsoft-Antispam-PRVS:
	<PA4PR08MB5887B37A8575CE966E26F472F7A39@PA4PR08MB5887.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 FLiKy1Ir1H1QB01hK7TC2sRyFE97xkd/sUGW2vJEg9VBp+lbrcJs9sV7PvWJTcNIU08VsDlCVD6AsXQ/UdiwiWrBH424rHh2xwokZk333SW0wUubIWSpCg8MIbuqL4UimhGBjR+CxeAi6IzZyrsMEq389TRKo53ETecWGgnBWxY9+ZOUr1EUIo9mOQ+rKM+zwUOpsK54FPW2ndtY2cmc9guostmSV+Qx6cQRQ5/DJcRi1FBk01mxfe2FjOuKyuKDrVPKXf1Q+7drg+ZgZlXbcMFeeRGajd9TYv/iw4QueIGEQthYUynawkV+ZoXk2YCNl7Au1GuDgSkkTx6NYG+x6Umfp+Wfn/HuTqqth2sCimWtEiD4XH5NMLarTQDVJA4YB2Pf237JPrVCsJccl3iw0sbke2TBD1h7wGpJ6vOhXjDMRE0V+HfDOhjPXsjRmodmOLU9+ZaFcd68ePGmoer052B3hBlnlVfNPqaEfqfyv7uZ0CxHsEZNQ+V8K0NdVBbE1ivwC6hGn/9EXnSPtFseV0Pl9lJozAs0n/csyhLsIgCqIZLpKlnqSUoCfwIsKQf7oY4CHOJyQ2MqRG6vQ5HFuonn2VwMxDFVK2fv8o0kTCwUtB2W9pZCW+aZY8F4jbITLr3w1jIQAA212vTaxnedX1BEVCxE8WNn2/K3S0jek35MMJ8QL8RKUlgiZlfhHeM889HIssstCuGg2bLgOwvWmSDY5gXkgAH5AtNc+xguVgM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(2906002)(2616005)(1076003)(36756003)(82310400003)(4326008)(316002)(54906003)(5660300002)(8936002)(426003)(356005)(508600001)(26005)(6666004)(7696005)(86362001)(36860700001)(70586007)(186003)(8676002)(81166007)(70206006)(44832011)(83380400001)(336012)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3137
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1c52c898-0a8f-42ee-5ee4-08d97e3fdece
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Jb2aR2oLJpNDTd2Tej4BUq0E7WiabSnCIXC1K01W1KfpFLOH2Zls0ngOIBzaGodYcA5mdeKxhB6I1J9SWDwbo34gTml/YYpGaurmJb75sMlizWlBTvC4il1eqF7A1QhcuLczXZ4DNGSsyxG6Ig8otGhaWGcV0gEODf+k2+tK1jbaBuYAstTzO037+dGzdlIvIwwaivMw6bsabErx2TEcg2pwIVjODnO9dyVxKK5T9dg5FTgR0pEsXJUVnNgZwegV2xLXgwEti4thF/dDW2i1zXPtRL3zb0lJjk0LqJnQNuuBG4PyMvTxXSMP++jaTFKdKork1Um294C1KWNAcwb9yS8SIm2DMMD9AmV+bw8SvmgDwrIP1R5zxxBhdS+rIecPEnDfs6SXJnzXgru6voGBg56Gm2K5xL9NYR4SIs2BFxQR6yb/BVmV4X1aCOfl4cqQcvK0RVApJcll4XOn04jujUmonrGuVkl2a9ozd61jyJyRhxEkc48uLLJ/j7iloKCMAOu9QJFG04GjHYGCgTwnHsUU5dwDALsi9trUyuJawBEShRH+fyKs6sgkvLezcaP1Mh+FFi1C4wsCiNOUTZesOLujruMcrrqEnyA7iRlntCoaL6MmheDYR+SBNcNIXIgbX6NRyzXkKn8L8mi+x2j+mjBAuD837H3CCUExYuORzvCfAKcK8bipYYCQItuNVX75QKdJcZ3ZwZBynvQK1WxA4A==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(186003)(26005)(7696005)(70586007)(4326008)(81166007)(36756003)(8936002)(5660300002)(426003)(336012)(44832011)(70206006)(82310400003)(8676002)(2906002)(110136005)(2616005)(316002)(508600001)(54906003)(1076003)(6666004)(86362001)(47076005)(83380400001)(36860700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:53.0817
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6142d902-90fa-4d54-2c0e-08d97e3fe47a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5887

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Today we use native addresses to map the GICv2 for Dom0 and fixed
addresses for DomUs.

This patch changes the behavior so that native addresses are used for
all domains that are 1:1 direct-map(including Dom0).

Update the accessors to use the struct vgic variables to provide the
updated addresses.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/vgic-v2.c         | 16 ++++++++++++++++
 xen/arch/arm/vgic/vgic-v2.c    | 17 +++++++++++++++++
 xen/include/asm-arm/new_vgic.h |  4 ++--
 xen/include/asm-arm/vgic.h     |  4 ++--
 4 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index b34ec88106..a8cf8173d0 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -673,6 +673,22 @@ static int vgic_v2_domain_init(struct domain *d)
         csize = vgic_v2_hw.csize;
         vbase = vgic_v2_hw.vbase;
     }
+    else if ( is_domain_direct_mapped(d) )
+    {
+        /*
+         * For non-dom0 direct_mapped guests we only map a 8kB CPU
+         * interface but we make sure it is at a location occupied by
+         * the physical GIC in the host device tree.
+         *
+         * We need to add an offset to the virtual CPU interface base
+         * address when the GIC is aliased to get a 8kB contiguous
+         * region.
+         */
+        d->arch.vgic.dbase = vgic_v2_hw.dbase;
+        d->arch.vgic.cbase = vgic_v2_hw.cbase + vgic_v2_hw.aliased_offset;
+        csize = GUEST_GICC_SIZE;
+        vbase = vgic_v2_hw.vbase + vgic_v2_hw.aliased_offset;
+    }
     else
     {
         d->arch.vgic.dbase = GUEST_GICD_BASE;
diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
index fd452fcb5a..ce1f6e4373 100644
--- a/xen/arch/arm/vgic/vgic-v2.c
+++ b/xen/arch/arm/vgic/vgic-v2.c
@@ -280,6 +280,23 @@ int vgic_v2_map_resources(struct domain *d)
         csize = gic_v2_hw_data.csize;
         vbase = gic_v2_hw_data.vbase;
     }
+    else if ( is_domain_direct_mapped(d) )
+    {
+        d->arch.vgic.vgic_dist_base = gic_v2_hw_data.dbase;
+        /*
+         * For non-dom0 direct_mapped guests we only map a 8kB CPU
+         * interface but we make sure it is at a location occupied by
+         * the physical GIC in the host device tree.
+         *
+         * We need to add an offset to the virtual CPU interface base
+         * address when the GIC is aliased to get a 8kB contiguous
+         * region.
+         */
+        d->arch.vgic.vgic_cpu_base = gic_v2_hw_data.cbase +
+                                     gic_v2_hw_data.aliased_offset;
+        csize = GUEST_GICC_SIZE;
+        vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
+    }
     else
     {
         d->arch.vgic.vgic_dist_base = GUEST_GICD_BASE;
diff --git a/xen/include/asm-arm/new_vgic.h b/xen/include/asm-arm/new_vgic.h
index 9097522b27..0d16f878eb 100644
--- a/xen/include/asm-arm/new_vgic.h
+++ b/xen/include/asm-arm/new_vgic.h
@@ -188,12 +188,12 @@ struct vgic_cpu {
 
 static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
 {
-    return GUEST_GICC_BASE;
+    return vgic->vgic_cpu_base;
 }
 
 static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
 {
-    return GUEST_GICD_BASE;
+    return vgic->vgic_dist_base;
 }
 
 static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index d5ad1f387b..92f6a2d15d 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -280,12 +280,12 @@ enum gic_sgi_mode;
 
 static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
 {
-    return GUEST_GICC_BASE;
+    return vgic->cbase;
 }
 
 static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
 {
-    return GUEST_GICD_BASE;
+    return vgic->dbase;
 }
 
 #ifdef CONFIG_GICV3
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193314.344360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9U-0005Mf-0j; Thu, 23 Sep 2021 03:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193314.344360; Thu, 23 Sep 2021 03:11:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9T-0005LQ-RM; Thu, 23 Sep 2021 03:11:59 +0000
Received: by outflank-mailman (input) for mailman id 193314;
 Thu, 23 Sep 2021 03:11:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9T-0004Qj-DW
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:11:59 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.75]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ffdad977-1c1b-11ec-ba0a-12813bfff9fa;
 Thu, 23 Sep 2021 03:11:53 +0000 (UTC)
Received: from DU2PR04CA0237.eurprd04.prod.outlook.com (2603:10a6:10:2b1::32)
 by AM8PR08MB6562.eurprd08.prod.outlook.com (2603:10a6:20b:355::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 03:11:43 +0000
Received: from DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b1:cafe::74) by DU2PR04CA0237.outlook.office365.com
 (2603:10a6:10:2b1::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:43 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT028.mail.protection.outlook.com (10.152.20.99) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:42 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 03:11:42 +0000
Received: from 328040812834.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3AB28AAE-394B-455B-9593-66F767380AEF.1; 
 Thu, 23 Sep 2021 03:11:30 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 328040812834.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:30 +0000
Received: from AS8PR04CA0080.eurprd04.prod.outlook.com (2603:10a6:20b:313::25)
 by DBBPR08MB4521.eurprd08.prod.outlook.com (2603:10a6:10:cd::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:27 +0000
Received: from VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:313:cafe::3c) by AS8PR04CA0080.outlook.office365.com
 (2603:10a6:20b:313::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:27 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT039.mail.protection.outlook.com (10.152.19.196) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:27 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:26 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffdad977-1c1b-11ec-ba0a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qHVKimCebnfTnXl/cdcWvwlQTjDe3a3vxMZKNxFvmJA=;
 b=i1VoeOZq/COXIp7ZbnpyZQVqsu5uUNZnHJaRzwrtdj9mPh/FV/aFfxwDlsBlGYBVCFKmrq7aGrF9PmuP909Zdbgb26BDFxzf5JbK0Pf7E+aItb8kJiPD1jxSwtm9NtRq1JpDJao7tsaaK2w6WM9D6vUETRL9G1xeHLv2M4RcrAA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3c0ad97f2e027594
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hwY3/gCgm8FFI+//Y8bWgllcA4uiJCy5D//le6d3gRFdQ80qyseUmsGnxVdB2mwJJlWkUK1bfzDbopug7eYM66xknZHK2sz0tYd9i5aHjlDs48rnKIkoLKozMUQ7S1bNNtcb+Hn+6DhbLhYKh9fjqXjc3GzC6ornpAtRnfRM43yC6xxXBOH9BCG7e0Rghhvh+FeiCAo/Z5vYOx3XIok/diyWYKS156/fzPoB1xLMQqwQVunhnHWZtizM+YevLz5dc0d4OiDF2G7TlDLvbmVHqbgpIW1qI5yyImP4hHA4jt5i6pWZzOwo6IKeHPBrH3AxRXsmq/92UOJAvNS+en86vA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qHVKimCebnfTnXl/cdcWvwlQTjDe3a3vxMZKNxFvmJA=;
 b=AfgfBHyZVQf58vqVzf95ri3+Z+E0vfIenu7+wtaGtpvp10Kk8nUuV6AjKKdXTw+JOg7tEyHaFflftiClfBKVqMk6KL1CcACg+vyUNGccGyfvV/ewRBmMtx+eTvEDeWMTpNq/2vKLJJvvXpGH72ftIiBeBpXvd4YIRVQ4ot8mypQx5qQu8GAD3Zd5yFlYS0i2r1MvYnVxOrfUjgKf0vPeseIeom6S9ANHXDgFWC4Y+X8mzj3/1txiL38VObTKovXNVt5AHLgbC837DJ+qG+qFWDJQ9hG7b3MH2fBfCaHRou2a2USEeTikySxAlQJoV+HSusFZ6R7O/tQG7rpf4ewxog==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qHVKimCebnfTnXl/cdcWvwlQTjDe3a3vxMZKNxFvmJA=;
 b=i1VoeOZq/COXIp7ZbnpyZQVqsu5uUNZnHJaRzwrtdj9mPh/FV/aFfxwDlsBlGYBVCFKmrq7aGrF9PmuP909Zdbgb26BDFxzf5JbK0Pf7E+aItb8kJiPD1jxSwtm9NtRq1JpDJao7tsaaK2w6WM9D6vUETRL9G1xeHLv2M4RcrAA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 02/11] xen/arm: introduce XEN_DOMCTL_INTERNAL_directmap
Date: Thu, 23 Sep 2021 03:11:06 +0000
Message-ID: <20210923031115.1429719-3-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d7638033-1e47-4487-6928-08d97e3fde68
X-MS-TrafficTypeDiagnostic: DBBPR08MB4521:|AM8PR08MB6562:
X-Microsoft-Antispam-PRVS:
	<AM8PR08MB6562B705C129DDDE7A1B604AF7A39@AM8PR08MB6562.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:229;OLM:229;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 jtH2pJ/rqcJUjKN21pHyIOZmf6cu17r3p9OEYfYSXgyVmrpktCeFWsahkl/Zwy25iwD3zpX5b8luqiJ6o8vIM3Mvw8wHDzSGXg7ZA14nNL+f9xHAlSWgZaRyqkdMBOZWccd/Awnid/bFv/4jbkyJsrWOElkdfHHNQRXA/bzv5aFRBbJlq0x2y/HgWzdzEcb33DacVx/6hD1m8jG9+9ZmHvhw7sfhQ5fbOfcJ36lca8NlYLKG/tBc5EFVmfAMwUxDT1GISY4z15uGemW4GXoC4XwTtnzlF7NbCxIMW3piceTmkTkh+mzFEVHqVp8fh/7vkhYG7gQtGDaloKCyASyMXw4bgMgATBF7v/c1noeAkMLPScdxK8WtZWWAw3MpdzBqCOuF+4EWnEfswfz9DD4AVwZOIhXdTxKpJBRlAfPMGPOdIEdXifFDBH5hPyJ77rfWT72cwKwZM64zvLqdO81louCYpnIxDTNwxfj/c/fiRKTfMl5/j5WUOokRA/66AsSVeKRIx88RbDRpAEVddjORoH3fVYQVvXEDDm4Ue0Ogs0eJhQHAzHw7Gn7Vu5MSaVsNUsOaTjX2bRsZuFfrP7Alid8jFRr3s0ombp3PhAug3J6C/IDlFIKRNHdzKi3zJIXaherUBde8v2ykAidhvBP815z3oujNcE3lssGaztVEb90PDlBtscjc3GDsv/lQ4d/ITJW56oFvshpahH8j4qbrL7K1Nrc3QSlGuci0YssLtns=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(70586007)(54906003)(83380400001)(70206006)(316002)(426003)(82310400003)(4326008)(110136005)(7696005)(44832011)(2616005)(5660300002)(186003)(36756003)(81166007)(356005)(6666004)(47076005)(86362001)(1076003)(8676002)(36860700001)(8936002)(2906002)(336012)(26005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4521
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	09e2341e-c755-40cc-640b-08d97e3fd528
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YhFj2w7HnknwFrSWixp24/IGxrJuy/fM8XHWkDJfWrUMVmKJRD+ZfjPq/kAi2SR/mwjEpZJHKjeIo4lNN/cWvhEz6jNbiRo3PxuBLJudhfzkI6Hqw3j6kk2kblCjD4Zx0T5a+wzDyBeoA4UqPkOFra5dbauhWkrDROwfMLFh625bBrZEegO6k5/PbNEWreGUJxiI4bomlLqqGeMoA2yK6iZdBB6PQnHwRbWLRMVJaURmjBZRDcosj4hHFVer/Ka3GKXMYiGhdHy315abftfYPgFv90NjsaGCxR13KcdDD5281Fdk5WV6pqVQOUzBCNfbu+DMYamWZH7ZwIiTv//SXoNlmV+Dj9l6IE075ICMkZ3NWuDDvhFBYeB0/Q4NVMmVG2jpW3dKvEAHYMjY1/NOTWsZta9fQAPhcbIN/LEyIyij0hHo70v4m1bWndkAi14D/G7Jq/ixgfxyo08tk54AEE57ZnZBfUf92b8aJYBKx1lyZXVM7iQ2Q9T7vJUsmyXMtRq3g15pbUEbNohe7KWGBi/T2siQF6c/92Q2hArVUHXgnF8hO4wGty3hWauH/CDM2ayhWzRNAKXDZpJWTXDiFruq06uncditSBIBLBMKIHqNYNlCZgvokrKpFR+aqRKEATBxpOwqQw2kiyrdhCCz0B2AB/7/v/d8G5hshV0RxFXucM0yEihmV42Vb9HJMoH62dLx+5xINjwstvsBbqaRVQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(8676002)(81166007)(36860700001)(70586007)(186003)(8936002)(70206006)(82310400003)(5660300002)(86362001)(83380400001)(426003)(26005)(110136005)(7696005)(2616005)(47076005)(36756003)(6666004)(508600001)(54906003)(336012)(4326008)(1076003)(316002)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:42.8989
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d7638033-1e47-4487-6928-08d97e3fde68
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6562

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

This commit introduces a new ARM-specific flag to specify that the
domain should be 1:1 directly mapped (guest physical addresses ==
physical addresses).

Also, add a direct_map flag under struct arch_domain and use it to
implement is_domain_direct_mapped.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
CC: andrew.cooper3@citrix.com
CC: jbeulich@suse.com
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Wei Liu <wl@xen.org>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
---
 xen/arch/arm/domain.c        | 1 +
 xen/arch/arm/domain_build.c  | 2 +-
 xen/include/asm-arm/domain.h | 9 +++++++--
 xen/include/xen/domain.h     | 4 ++++
 4 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 7922249d26..0b3cff8a40 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d,
         return 0;
 
     ASSERT(config != NULL);
+    d->arch.direct_map = config->flags & XEN_DOMCTL_INTERNAL_directmap;
 
 #ifdef CONFIG_IOREQ_SERVER
     ioreq_domain_init(d);
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 8cc4c800e9..21d8a559af 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2753,7 +2753,7 @@ void __init create_dom0(void)
     struct domain *dom0;
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
-                 XEN_DOMCTL_INTERNAL_ispriv,
+                 XEN_DOMCTL_INTERNAL_ispriv | XEN_DOMCTL_INTERNAL_directmap,
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = -1,
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c9277b5c6d..a74ee5720c 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -29,8 +29,11 @@ enum domain_type {
 #define is_64bit_domain(d) (0)
 #endif
 
-/* The hardware domain has always its memory direct mapped. */
-#define is_domain_direct_mapped(d) is_hardware_domain(d)
+/*
+ * The hardware domain has always its memory direct mapped. And DOM0 shall
+ * be always been set as 1:1 direct-map domain.
+ */
+#define is_domain_direct_mapped(d) (d)->arch.direct_map
 
 struct vtimer {
     struct vcpu *v;
@@ -89,6 +92,8 @@ struct arch_domain
 #ifdef CONFIG_TEE
     void *tee;
 #endif
+
+    bool direct_map;
 }  __cacheline_aligned;
 
 struct arch_vcpu
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 7ed0b62b78..6c2b07eb42 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -32,6 +32,10 @@ void arch_get_domain_info(const struct domain *d,
 #define _XEN_DOMCTL_INTERNAL_ispriv         16
 #define XEN_DOMCTL_INTERNAL_ispriv          (1U<<_XEN_DOMCTL_INTERNAL_ispriv)
 
+/* This flag is ARM specific */
+#define _XEN_DOMCTL_INTERNAL_directmap      17
+#define XEN_DOMCTL_INTERNAL_directmap       (1U<<_XEN_DOMCTL_INTERNAL_directmap)
+
 /*
  * Arch-specifics.
  */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193315.344371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9V-0005hH-Im; Thu, 23 Sep 2021 03:12:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193315.344371; Thu, 23 Sep 2021 03:12:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9V-0005gE-9y; Thu, 23 Sep 2021 03:12:01 +0000
Received: by outflank-mailman (input) for mailman id 193315;
 Thu, 23 Sep 2021 03:12:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9U-0004kC-B3
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:00 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0c::624])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e6959f0f-a5b0-421c-a025-dc6fb36d9bcd;
 Thu, 23 Sep 2021 03:11:54 +0000 (UTC)
Received: from AM6P194CA0029.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:90::42)
 by HE1PR0802MB2540.eurprd08.prod.outlook.com (2603:10a6:3:d4::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 03:11:52 +0000
Received: from AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:90:cafe::c7) by AM6P194CA0029.outlook.office365.com
 (2603:10a6:209:90::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:52 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT054.mail.protection.outlook.com (10.152.16.212) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:51 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Thu, 23 Sep 2021 03:11:49 +0000
Received: from 02875dfcb4fc.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 54E4DF64-BF4C-4AFF-B8B0-218451B1D66B.1; 
 Thu, 23 Sep 2021 03:11:34 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 02875dfcb4fc.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:34 +0000
Received: from AM6PR01CA0042.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::19) by DB6PR0801MB1944.eurprd08.prod.outlook.com
 (2603:10a6:4:73::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 03:11:30 +0000
Received: from VE1EUR03FT043.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:e0:cafe::fe) by AM6PR01CA0042.outlook.office365.com
 (2603:10a6:20b:e0::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:30 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT043.mail.protection.outlook.com (10.152.19.122) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:30 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:28 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6959f0f-a5b0-421c-a025-dc6fb36d9bcd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cXV5Lffa19mU+2iwdP2TQ82Nx4ZrJiOJhVjvupxNFvU=;
 b=u3vK5WpREiAZkA12VE6wgC1OepCyGY4Eulp+pSMoc1/08chn2y1uadcFgtfMXrxKEWgRxUfQX39hohbxkfiA1Htg6F3ljf6XlF9j79iVRfy0NgM66u95OmuW/XH/JxOqEmMFZDZ4VXWkQd635kQ/k3w2EL7q9P8f5UnPoakEjwc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c826ebf4bbbf4bb0
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PJVKCqT/pGIrTwjTY1VJGO5SmMpxKe+V28PjeZD35AaZxe07UsS4CFGrcQZ6tmSlUo6fJIkkTxxW14DAFx0yKjT2FBhWv+yZxwog/lqQCF0yZ8PDP4940kqIOhHKhLxJG8LTx+0PtsRjnAEeLXF+hvFMA0m8cqFTJ5Sa8gDXBsqLfIn7RuXROEiCDuFN+/fCeusPwaY0n8FGIXmgL9woKQLdiykUtmlnPAKHr7vFami4U67mEf0pXD8+4kl31JKrqznvVLxltQ+jx1+txGZNooJgE1zjMw0uH4Jf8j8LBxbF258YSoy0KqJgoptqvh0FukKDERgtQqIZEKDWdvVFQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cXV5Lffa19mU+2iwdP2TQ82Nx4ZrJiOJhVjvupxNFvU=;
 b=Kl4h/Joyu9Z40IJx4ZSl0IvqTEOYscNOh8JUjfezIlb6mlmFSrlUAjefcKnkBT2u4ZLjLGBl6kDsYYS6E8w/hiyZPAzE09u0aRnCn3gXLfTmBDZrIgsYkWxD63Atr56sab0YoL0+GOhsvxJ0jeLR4E6LkrmoLpCPNuTCzJmfK49mTVKLzHlezy8NEeEHfxkqh6F0A22jlIuFLj7hi92ENIpPV6VAmkoUgHEYmoN/OrK17lREKLjBhRGCX6Hw1VsYSNdsaeSjDEtMYPJm5FctRYV6D86yhFCiumf9UvQg5Jajg6Z0L1avO+3PDAI+T/2sHJ2BjZ7oblAt9lL/+3JyUQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cXV5Lffa19mU+2iwdP2TQ82Nx4ZrJiOJhVjvupxNFvU=;
 b=u3vK5WpREiAZkA12VE6wgC1OepCyGY4Eulp+pSMoc1/08chn2y1uadcFgtfMXrxKEWgRxUfQX39hohbxkfiA1Htg6F3ljf6XlF9j79iVRfy0NgM66u95OmuW/XH/JxOqEmMFZDZ4VXWkQd635kQ/k3w2EL7q9P8f5UnPoakEjwc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 03/11] xen/arm: introduce 1:1 direct-map for domUs
Date: Thu, 23 Sep 2021 03:11:07 +0000
Message-ID: <20210923031115.1429719-4-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e7c77710-aad9-4476-f8b8-08d97e3fe3b9
X-MS-TrafficTypeDiagnostic: DB6PR0801MB1944:|HE1PR0802MB2540:
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB25408CC4CDCA8644CDC7703FF7A39@HE1PR0802MB2540.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1775;OLM:1775;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 6AQ8kthB+tJZm9n6CUjNlHXgh+/MTN9V7hUEu4Us+XL60lMqpWrwpR/yXgDSxa+4PBzukkbP77oSJ8ORD7VrtKW1zLoEUd39BNpBc99PP8CeoUk25azucU+/UUREtPVYbfO+faVi+Otvs8DZeYo+qWh5LHG6uV3inwUUwqhPviiUJebkTPwleioVIg0QyRI49ml67SIWG3fHzfHyPhCpqzPmgYixZDFVfYDKzm5qcecG52lGMtC9LDH/ETI8fFd3/cB63uNHcyjgUvwBX+ZhhevFSoI3HwOEIL77yve93TwyjbEHi9JC3QuapxJYg6G/5ZpPvY/ZJUjHrL8s7+knQsmfssp/8NpiyQIsLjfUYn/Z9T7wq7oxV7bsOLkP/0ROiA0MS+naYm9pYHyEeUog4TObSbnLk9iFbH5PYAguQ91ChlJt2cMd8cy1n0ppp6R7BvlKHWYKnUA2RK2YWPnMBVeT0rgkk6WKaYw2MGBA8e1BH4gz210GVlOjHAyOlBGgjaH4DICT/JXDUYYniFIolqW/80Ew86QWNr8egN72FMS/SyayAZ6povPcHWne0AVg5Qvs3nq0Yb9AVfvzqvzOY2xKWPkjmOPR9xAa1UsnAzMT32i6b9VEenS3twuhS1AGpkfNWW0wSgaCQaEch6aN7ZzGUUZmkMTS+e8HmA4wTJs/7LFomrqgGZ+r1Ngio290mb2gfMO9C3shYAhoPeV3yvkCjlNQ16ReeBWQDXzcbeg=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36860700001)(2906002)(8936002)(5660300002)(508600001)(316002)(82310400003)(44832011)(186003)(8676002)(6666004)(1076003)(7696005)(54906003)(47076005)(83380400001)(86362001)(81166007)(70586007)(70206006)(36756003)(26005)(426003)(2616005)(4326008)(356005)(336012)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1944
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6bef1812-7ebd-4191-99dd-08d97e3fd710
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Qon82hJ+m+y/Qe7jLVr3phHxpqfnDfPTE2IB8xRnY91ep3vsp1HzXPw/Tp//zWT9115JPbPjn0NQmVA1IqzHhC9XPuEwB+u1kpZme1EUUxG3byC7amXccvYBknL9+1HhSYuzEkcqYiOJoMT1GOLdx/bmFbU+9LohpNcW74Hw9Zoq8LJuMF0Ha96pfAyMIe7hAYRc8+m03iNApO0A/kBkjktY6F4iw3PPpTTuq55Cl2LK0KamHA0djFyULJsTacetZzt6UuUFi8UtPObKrF0RUMWxT1yBe1IH08dRKIEpoLWk05LUeK/rR4rvuaLKy8++QMo1g8SghhxMC/MXtYduY2t7PJoi9EIFpBQ9Z0j2KxQfPgEKTepbjB/DFrxot3RCCkaG2ySFe1J0iW/rxp31nUxRKmEUTGQJVUd6dYG59EV6ApePm3glyGYPYknPTqvoWu9LsHhEXWGT4R7GPsmZ1gNiS3LbL65fEVInJacs9jrCmetjgEJYWuMmyTGTfLXtAbeYqH9QNcEjKYhpbAm7Q0f3WMiU4WOlZWEtQ+HqeSbuOOIwD9IBVPZcimyLc9rpHapicgoeVqYdgqxXOtwVy3ZCHqt6uwNYzIn5fbaI22kPxlR+/xgguEbOFthMZaMmVEBfJ2Hfqrkp+MBhH/A5LAtCUAJ/vB8ran58CcyONolBE0Jz1tTXo+ZGULunMdU/JJ8TAVEaxTt9PlQMmZ5Kgw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(508600001)(110136005)(36756003)(82310400003)(186003)(83380400001)(26005)(2616005)(1076003)(6666004)(47076005)(316002)(54906003)(426003)(70206006)(86362001)(70586007)(4326008)(8676002)(2906002)(336012)(44832011)(7696005)(81166007)(36860700001)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:51.7431
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e7c77710-aad9-4476-f8b8-08d97e3fe3b9
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2540

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Cases where domU needs 1:1 direct-map memory map:
  * IOMMU not present in the system.
  * IOMMU disabled if it doesn't cover a specific device and all the guests
are trusted. Thinking a mixed scenario, where a few devices with IOMMU and
a few without, then guest DMA security still could not be totally guaranteed.
So users may want to disable the IOMMU, to at least gain some performance
improvement from IOMMU disabled.
  * IOMMU disabled as a workaround when it doesn't have enough bandwidth.
To be specific, in a few extreme situation, when multiple devices do DMA
concurrently, these requests may exceed IOMMU's transmission capacity.
  * IOMMU disabled when it adds too much latency on DMA. For example,
TLB may be missing in some IOMMU hardware, which may bring latency in DMA
progress, so users may want to disable it in some realtime scenario.

*WARNING:
Users should be aware that it is not always secure to assign a device without
IOMMU protection.
When the device is not protected by the IOMMU, the administrator should make
sure that:
 1. The device is assigned to a trusted guest.
 2. Users have additional security mechanism on the platform.

Given that with direct-map, the IOMMU could be used but it is not required.
This commit avoids setting XEN_DOMCTL_CDF_iommu when the IOMMU is disabled and
direct_map is requested.

Since, for now, 1:1 direct-map is only supported when domain on Static
Allocation, that is, "xen.static-mem" is defined in the domain configuration.

This commit also re-implements allocate_static_memory to allocate static memory
as guest RAM for 1:1 direct-map domain.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 docs/misc/arm/device-tree/booting.txt |   9 ++
 xen/arch/arm/domain_build.c           | 117 +++++++++++++++++---------
 2 files changed, 85 insertions(+), 41 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 44cd9e1a9a..3d637c747e 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -164,6 +164,15 @@ with the following properties:
     Both #address-cells and #size-cells need to be specified because
     both sub-nodes (described shortly) have reg properties.
 
+- direct-map
+
+    Optional for Domain on Static Allocation.
+    An empty property to request the memory of the domain to be 1:1
+    direct-map(guest physical address == physical address).
+    WARNING: Users must be aware of this risk, that guests having access
+    to hardware with DMA capacity must be trusted, or it could use the
+    DMA engine to access any other memory area.
+
 Under the "xen,domain" compatible node, one or more sub-nodes are present
 for the DomU kernel and ramdisk.
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 21d8a559af..213ad017dc 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -488,8 +488,14 @@ static bool __init append_static_memory_to_bank(struct domain *d,
 {
     int res;
     unsigned int nr_pages = PFN_DOWN(size);
-    /* Infer next GFN. */
-    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
+    gfn_t sgfn;
+
+    if ( !is_domain_direct_mapped(d) )
+        /* Infer next GFN when GFN != MFN. */
+        sgfn = gaddr_to_gfn(bank->start + bank->size);
+    else
+        sgfn = gaddr_to_gfn(mfn_to_maddr(smfn));
+
 
     res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
     if ( res )
@@ -537,14 +543,17 @@ static void __init allocate_static_memory(struct domain *d,
     }
     reg_cells = addr_cells + size_cells;
 
-    /*
-     * The static memory will be mapped in the guest at the usual guest memory
-     * addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
-     * xen/include/public/arch-arm.h.
-     */
-    gbank = 0;
-    gsize = ramsize[gbank];
-    kinfo->mem.bank[gbank].start = rambase[gbank];
+    if ( !is_domain_direct_mapped(d) )
+    {
+        /*
+         * The static memory will be mapped in the guest at the usual guest
+         * memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
+         * xen/include/public/arch-arm.h.
+         */
+        gbank = 0;
+        gsize = ramsize[gbank];
+        kinfo->mem.bank[gbank].start = rambase[gbank];
+    }
 
     cell = (const __be32 *)prop->value;
     nr_banks = (prop->length) / (reg_cells * sizeof (u32));
@@ -572,42 +581,58 @@ static void __init allocate_static_memory(struct domain *d,
         printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
                d, bank, pbase, pbase + psize);
 
-        while ( 1 )
+        if ( !is_domain_direct_mapped(d) )
         {
-            /* Map as much as possible the static range to the guest bank */
-            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
-                                               min(psize, gsize)) )
-                goto fail;
-
-            /*
-             * The current physical bank is fully mapped.
-             * Handle the next physical bank.
-             */
-            if ( gsize >= psize )
+            while ( 1 )
             {
-                gsize = gsize - psize;
-                break;
+                /* Map as much as possible the static range to the guest bank */
+                if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank],
+                                                   smfn, min(psize, gsize)) )
+                    goto fail;
+
+                /*
+                 * The current physical bank is fully mapped.
+                 * Handle the next physical bank.
+                 */
+                if ( gsize >= psize )
+                {
+                    gsize = gsize - psize;
+                    break;
+                }
+                /*
+                 * When current guest bank is not enough to map, exhaust
+                 * the current one and seek to the next.
+                 * Before seeking to the next, check if we still have available
+                 * guest bank.
+                 */
+                else if ( (gbank + 1) >= GUEST_RAM_BANKS )
+                {
+                    printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
+                    goto fail;
+                }
+                else
+                {
+                    psize = psize - gsize;
+                    smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
+                    /* Update to the next guest bank. */
+                    gbank++;
+                    gsize = ramsize[gbank];
+                    kinfo->mem.bank[gbank].start = rambase[gbank];
+                }
             }
+        }
+        else /* 1:1 direct-map. */
+        {
             /*
-             * When current guest bank is not enough to map, exhaust
-             * the current one and seek to the next.
-             * Before seeking to the next, check if we still have available
-             * guest bank.
+             * One guest memory bank is matched with one physical
+             * memory bank.
              */
-            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
-            {
-                printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
+            gbank = bank;
+            kinfo->mem.bank[gbank].start = pbase;
+
+            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank],
+                                               smfn, psize) )
                 goto fail;
-            }
-            else
-            {
-                psize = psize - gsize;
-                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
-                /* Update to the next guest bank. */
-                gbank++;
-                gsize = ramsize[gbank];
-                kinfo->mem.bank[gbank].start = rambase[gbank];
-            }
         }
 
         tot_size += psize;
@@ -2638,6 +2663,7 @@ void __init create_domUs(void)
 {
     struct dt_device_node *node;
     const struct dt_device_node *chosen = dt_find_node_by_path("/chosen");
+    bool direct_map = false;
 
     BUG_ON(chosen == NULL);
     dt_for_each_child_node(chosen, node)
@@ -2658,8 +2684,17 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));
 
+        direct_map = dt_property_read_bool(node, "direct-map");
+        d_cfg.flags |= direct_map ? XEN_DOMCTL_INTERNAL_directmap : 0;
         if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") )
-            d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
+        {
+            if ( iommu_enabled )
+                d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
+            else if ( !direct_map )
+                panic("Assign a device but IOMMU and direct-map are all disabled\n");
+            else
+                warning_add("Please be sure of having trusted guests, when doing device assignment without IOMMU protection\n");
+        }
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
         {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193316.344382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9Z-0006CX-SP; Thu, 23 Sep 2021 03:12:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193316.344382; Thu, 23 Sep 2021 03:12:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9Z-0006CM-NP; Thu, 23 Sep 2021 03:12:05 +0000
Received: by outflank-mailman (input) for mailman id 193316;
 Thu, 23 Sep 2021 03:12:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9Y-0004Qj-Di
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:04 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.47]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 021986ce-1c1c-11ec-ba0a-12813bfff9fa;
 Thu, 23 Sep 2021 03:11:56 +0000 (UTC)
Received: from AM5PR0701CA0023.eurprd07.prod.outlook.com
 (2603:10a6:203:51::33) by DB6PR08MB2853.eurprd08.prod.outlook.com
 (2603:10a6:6:1d::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4373.21; Thu, 23 Sep
 2021 03:11:54 +0000
Received: from AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:51:cafe::bd) by AM5PR0701CA0023.outlook.office365.com
 (2603:10a6:203:51::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:54 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT043.mail.protection.outlook.com (10.152.17.43) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:54 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 03:11:53 +0000
Received: from 411708d2965c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 DD49CD68-F329-4C2A-A160-F534F2D25831.1; 
 Thu, 23 Sep 2021 03:11:47 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 411708d2965c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:47 +0000
Received: from AS9PR06CA0264.eurprd06.prod.outlook.com (2603:10a6:20b:45f::9)
 by AM0PR08MB3137.eurprd08.prod.outlook.com (2603:10a6:208:64::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:45 +0000
Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::cd) by AS9PR06CA0264.outlook.office365.com
 (2603:10a6:20b:45f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:45 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:45 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:41 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 021986ce-1c1c-11ec-ba0a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5lLs6y6VFPvynadLJUgwDgvXXXwXXf0+4A9GvM2TqEE=;
 b=RBFJYUdcqwvg5nHkBFWQhi9Qy96m0cWqQhOZ+zwSx8cAyY879YhxDPAexj58TwvuUZ8iBpXP8g7WsCnCrP9HNQRbTAgfhnXuwMwHVvkq+LMi+tIQq9Gx8yvoqldQIhxbbTzvfwxP8uTbGXD9ZgIrWb4cMnPRP1yv0trxW+tXFEg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 884e471e58809d3d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DjNUESi2rHxnEPZeGfnLKd4GgpOc9UfXRO1QwVVKOZD9FTQ/q1rnMRXRCzvzH9m0QycM7/1RfXQiaEqfagYd8qhZWV4DBJd2cqsnsAbIW6HF8Qmth56j35wd27xJsHBYth68r/ORmUzX3EFOynxiZelbGjRkxhu5HUpavLo8AZWNtTtVHcQZuOktT7CJAWTUqyBnntoj6LwzulFxLpXDV+ReeSZSsD0tZD3moZrS/W/0lvQb7702+qm9jtA5TuaFr/kPBXkJP80rzU/gtM0NZymiRZ0RSaMNKK/berYCAZTB5eY20JVkwRr3c3BTV1QS8ZxUSfZwlWI+4Sc7r5ZXkQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5lLs6y6VFPvynadLJUgwDgvXXXwXXf0+4A9GvM2TqEE=;
 b=mSnK3DeJFCdlx03HfEF8Z5Eccl9ZChoY6EH2sL8vKJ00CMoKp3pQvm5yAUgm508u7WZURnt4nHdWF35BzwAoL+SECT5qSSV9S+xB740apKLsiNZrxgiUtZpJck2BkE7BSaGc1Q9AsEYDag+tR4yYRxrsiCh0DhmTgZoEed6ASbtQ9VxFjLw8HHGwZgB2XONKpAYfoip/QynIn2/g1J/vvoNP1kOyD24hsqZ5UbWcBag6P+LFZ1KV1oR4SBeDtTDGON+dItcH0WNGxaPGwDP13hlfW2v8KMULp/lvfyS7maqjP5fon0tHl6w3wyjVivDXnVXwPjpTTA3ATDNvck6StQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5lLs6y6VFPvynadLJUgwDgvXXXwXXf0+4A9GvM2TqEE=;
 b=RBFJYUdcqwvg5nHkBFWQhi9Qy96m0cWqQhOZ+zwSx8cAyY879YhxDPAexj58TwvuUZ8iBpXP8g7WsCnCrP9HNQRbTAgfhnXuwMwHVvkq+LMi+tIQq9Gx8yvoqldQIhxbbTzvfwxP8uTbGXD9ZgIrWb4cMnPRP1yv0trxW+tXFEg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 08/11] xen/arm: if 1:1 direct-map domain use native addresses for GICv3
Date: Thu, 23 Sep 2021 03:11:12 +0000
Message-ID: <20210923031115.1429719-9-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7bb2667c-5ca2-406e-e8d2-08d97e3fe51f
X-MS-TrafficTypeDiagnostic: AM0PR08MB3137:|DB6PR08MB2853:
X-Microsoft-Antispam-PRVS:
	<DB6PR08MB28530620D2646E4D63349D7CF7A39@DB6PR08MB2853.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2276;OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 D7UrfBhVRKuFyLkrALiRKovZ1ZO8JfuXMVO4SNvQhhn20j/cWSi58TkyM0XyCNDU0n/caJusRTAgifPaAyyjMml3MYgyERh0BJrUdjXPnX6jwH4c3uYzhJUzLU/r9Ul4ThjJkvf4kG2e6V6PXw82aXo/p6UhjYBVUbzyueJ7GnqLv8toGagMBFllVU3baxWlUmD1s2CYV+aOTI7SIWSOCSd3lsVkUcXLn3aC9JO1SWLpC+CiJEFUd7CLjjchUwyKThD5lqXe0umPukJodUY7hb4GXZ1Js4J2MCmvDz7HXq+KHEsU1aoNOU/iNRpi2SlIs6Yhn1YaEk90ZvCJHmNS4VHKGv/ZFc0APULdwpJ3F0M8iz7RtLiQXR9XonvJ+0aKyypWpudbH2NH7NQ2bGn3EK6BsaYijXeXOO/NsmDvC5yckhAxL++pz06O0y3gCLgbNtiUBk2Uu0HjRL5r+itJxIgWpP28R01ZY5zsPYO3c1GyTw+AV2ikWLDeBU9aGlr1YZ5AzLHj+TzWHMEl0JYlE8ByK9bXygAL3hcpnhM20gssjlf1K/o8l371plRb2qHcWk9ufuGp7zBKLxqf7RU5C9i7gdaC9/Lgf+8nK3ayJWGyqEK+3YmrOXMRvEjATCY8Wayu4tSqWy5tai/TZK0FeYsR5cFa6SP6Wbjiy6+UCYNNu6C0HCgipjqUIWQU7i551lUyomLNDT/QkCSB+99W0I0Bey9p3t0rOd9dkZCjP88=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(2906002)(2616005)(1076003)(36756003)(82310400003)(4326008)(316002)(54906003)(5660300002)(8936002)(426003)(356005)(508600001)(26005)(6666004)(7696005)(86362001)(36860700001)(70586007)(186003)(8676002)(81166007)(70206006)(44832011)(83380400001)(336012)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3137
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	96aa237e-049f-4675-d87e-08d97e3fe01c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F44Juwe6mcWX8MwUU3kdEOwM3Qethn/8DixsoH+IwUq6tTSt/FHf8q9g0f0A+SufREBJSVU91kIanPGKtIFReCXquzzi1tEy2EEbbFHDKa8VXW4tVCTkat6C263aNK1D9ZN5RQzgLS3Pqm5bMBrG2zOkVxBVpdSHs+jvJucqAGbmHieI+AC2qDMvN607EnuTMs+cu42uxPWXPXjfAnyBTwxEYEl3lTs4TyKwdmnHNSrrEb1wVmOpiyK1+FWNlFSFlg4WieqhnHVRe3Nd8vEu2wCZ6kxQ/FeUO+mCd5/8jjiFT/+Hgjb+sKnssgYZ6+NbJoSd4ZHL5OQE+J5k/DykkKa4Gx0qU5RHJaPan5W/iOms6xqIJGGB1FSlG7OM+Xmx6UPur1lqmf78obQr/Q3JUlN/pZM3RQgsqJx3EGrmKq+BvkhFyoGhgtECymTWkTID0Y6NpUzxCyf7C2b6IVgJNwwFtMtOun+u01rvlu+QIF0ZnwvGJ5/sS2siOJ88abA5eI36SmuQqrU4DmIDscrfetllqd4nxtm04p0fXLDF7IKxW2eNWfZQ/LMQ2etQNz48xwkC/See3RD+ZG4J4Op1xMeztz7WvjLXWpj0Cl72OMwilqWgsnIvPA3ZiEXQ5bfgSFndAoJbzKSSHeJZDp0aK0sVAKtMx6o/r7P7BzO2GKX8ZCE/tXC2T7CYx8IAbpkpeGabnAwIoZoXTKFbRvKLcw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70586007)(8936002)(316002)(426003)(4326008)(82310400003)(1076003)(5660300002)(6666004)(2906002)(7696005)(86362001)(44832011)(8676002)(54906003)(36860700001)(70206006)(508600001)(2616005)(110136005)(36756003)(47076005)(26005)(186003)(81166007)(336012)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:54.1079
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7bb2667c-5ca2-406e-e8d2-08d97e3fe51f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT043.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2853

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Today we use native addresses to map the GICv3 for Dom0 and fixed
addresses for DomUs.

This patch changes the behavior so that native addresses are used for
all domains that are 1:1 direct-map(including Dom0).

Update the rdist accessor to use the struct vgic variables to provide
the updated addresses.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/domain_build.c | 32 ++++++++++++++++++++++++--------
 xen/arch/arm/vgic-v3.c      | 10 +++++-----
 xen/include/asm-arm/vgic.h  |  6 +++---
 3 files changed, 32 insertions(+), 16 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d5f201f73e..120f1ae575 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1854,10 +1854,11 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
 {
     void *fdt = kinfo->fdt;
     int res = 0;
-    __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
+    __be32 *reg;
     __be32 *cells;
     struct domain *d = kinfo->d;
     char buf[38];
+    unsigned int i, len = 0;
 
     snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
              vgic_dist_base(&d->arch.vgic));
@@ -1881,27 +1882,42 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
     if ( res )
         return res;
 
+    /* reg specifies all re-distributors and Distributor. */
+    len = (GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) *
+          (vgic_rdist_nr(&d->arch.vgic) + 1) * sizeof(__be32);
+    reg = xmalloc_bytes(len);
+    if ( reg == NULL )
+        return -ENOMEM;
+
     cells = &reg[0];
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
                        vgic_dist_base(&d->arch.vgic), GUEST_GICV3_GICD_SIZE);
-    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                       vgic_rdist_base(&d->arch.vgic, 0),
-                       vgic_rdist_size(&d->arch.vgic, 0));
+    for ( i = 0;
+          i < vgic_rdist_nr(&d->arch.vgic);
+          i++, cells += (GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) )
+    {
+        dt_child_set_range(&cells,
+                           GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                           vgic_rdist_base(&d->arch.vgic, i),
+                           vgic_rdist_size(&d->arch.vgic, i));
+    }
 
-    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    res = fdt_property(fdt, "reg", reg, len);
     if (res)
-        return res;
+        goto out;
 
     res = fdt_property_cell(fdt, "linux,phandle", kinfo->phandle_gic);
     if (res)
-        return res;
+        goto out;
 
     res = fdt_property_cell(fdt, "phandle", kinfo->phandle_gic);
     if (res)
-        return res;
+        goto out;
 
     res = fdt_end_node(fdt);
 
+ out:
+    xfree(reg);
     return res;
 }
 
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index cb5a70c42e..83f996b46c 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1647,8 +1647,8 @@ static inline unsigned int vgic_v3_max_rdist_count(struct domain *d)
      * However DomU get a constructed memory map, so we can go with
      * the architected single redistributor region.
      */
-    return is_hardware_domain(d) ? vgic_v3_hw.nr_rdist_regions :
-               GUEST_GICV3_RDIST_REGIONS;
+    return is_domain_direct_mapped(d) ? vgic_v3_hw.nr_rdist_regions :
+                                        GUEST_GICV3_RDIST_REGIONS;
 }
 
 static int vgic_v3_domain_init(struct domain *d)
@@ -1670,10 +1670,10 @@ static int vgic_v3_domain_init(struct domain *d)
     radix_tree_init(&d->arch.vgic.pend_lpi_tree);
 
     /*
-     * Domain 0 gets the hardware address.
-     * Guests get the virtual platform layout.
+     * 1:1 direct-map domain (including Dom0) gets the hardware address.
+     * Other guests get the virtual platform layout.
      */
-    if ( is_hardware_domain(d) )
+    if ( is_domain_direct_mapped(d) )
     {
         unsigned int first_cpu = 0;
 
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 92f6a2d15d..0f7cb32c58 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -291,17 +291,17 @@ static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
 #ifdef CONFIG_GICV3
 static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
 {
-    return GUEST_GICV3_RDIST_REGIONS;
+    return vgic->nr_regions;
 }
 
 static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
 {
-    return GUEST_GICV3_GICR0_BASE;
+    return vgic->rdist_regions[i].base;
 }
 
 static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
 {
-    return GUEST_GICV3_GICR0_SIZE;
+    return vgic->rdist_regions[i].size;
 }
 #else
 static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193319.344393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9f-0006sJ-EK; Thu, 23 Sep 2021 03:12:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193319.344393; Thu, 23 Sep 2021 03:12:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9f-0006rv-9A; Thu, 23 Sep 2021 03:12:11 +0000
Received: by outflank-mailman (input) for mailman id 193319;
 Thu, 23 Sep 2021 03:12:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9e-0004kC-BV
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:10 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::620])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d7b3318f-daef-48a0-8d6d-121684e888aa;
 Thu, 23 Sep 2021 03:11:57 +0000 (UTC)
Received: from AS9PR06CA0111.eurprd06.prod.outlook.com (2603:10a6:20b:465::17)
 by AM5PR0801MB1650.eurprd08.prod.outlook.com (2603:10a6:203:2f::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 03:11:47 +0000
Received: from VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::d4) by AS9PR06CA0111.outlook.office365.com
 (2603:10a6:20b:465::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:47 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT050.mail.protection.outlook.com (10.152.19.209) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:47 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 03:11:45 +0000
Received: from fd11e5c1008d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1A0D787A-399B-4616-AB80-4A7C3AA939A5.1; 
 Thu, 23 Sep 2021 03:11:34 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id fd11e5c1008d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:34 +0000
Received: from AS9PR06CA0295.eurprd06.prod.outlook.com (2603:10a6:20b:45a::12)
 by AM6PR08MB3221.eurprd08.prod.outlook.com (2603:10a6:209:49::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 23 Sep
 2021 03:11:33 +0000
Received: from AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45a:cafe::42) by AS9PR06CA0295.outlook.office365.com
 (2603:10a6:20b:45a::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:33 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT023.mail.protection.outlook.com (10.152.16.169) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:33 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:31 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 03:11:30 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7b3318f-daef-48a0-8d6d-121684e888aa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IvsXP25FrtNtNR/vVLA1gpBc4hkPTM/WhSOqFIm620Q=;
 b=55QgxqE2cFizlZ8kNtpKAh568tZfqhHm2ey89kHnlpZvQUxIPEtL5sGBmTmzdTYu7pPdAD7UIXroCDLXBwz6H5GPisevhKsqK3Xx6LwtI/VNuavLNb5AG2nTqsaBE4SJ1PyOQIezMUnSxwVZPKroAlgeTp14z2RGkekgkEAyj8k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 988e15c28dea971f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nLzx9KM4JSp1NdRqDOXe8Q6BKLxzJlTJRqf0UQ0DmwlSFDcKT56CJJ5d1+6kogBd1FGsWzD8F343u4MaFk71+kcqd93S9gmX9SIyb8W3rMd8mt20sT2JOXqbRjAnoIk1IBj1nhdeciwuOvSzpNcCprTZgJQySbyaAbvemX9eCVx5ZiSvHJPpfqwl01upB7o7wSQXHqymfeEut8/wLodEtZfDPnQFUn1Ta21WFqGwe+4nje6ZOpjmi/yfpirRbRgQMN2DZ7RCSF0c8S+p6sp23A3WMHGn9XEV0pNzvNP/t65/X9z7AbETAwpeZXq3LCy88x/KTwwMssfpW7IdhmRyeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IvsXP25FrtNtNR/vVLA1gpBc4hkPTM/WhSOqFIm620Q=;
 b=K5STUHFGi5ZICjkb9l7fa0giC5YJIwXMPV9i0sKzOgBnp0VTO2/byidot0W6jCUAi0v8BFglVVdNIyKX4vE1CNFCCJ+Rw2wo33MbAuAtzF02KOVFgM1aDqZoC5hkrpx9YqtXPcpyNleg18nNTYKeyMyP5bKe3zLLV9l8CuZTG3PXxjTlI1P1MZBsxR2iYxP5g4BkeGmiZWMoDGcTHgO04xL2n2Jbq9ZN2jaRsqVMNy/q0c5hvy/Pwq7ypaN/qx9aj6Ycr8zKvlaKVInJkm39q0BIhY5AXypvyz1ywIGzq2ERDhpVKe4NqiVf3QehmY3Axn9bHV4hIXkIMmNQ3pkgfQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IvsXP25FrtNtNR/vVLA1gpBc4hkPTM/WhSOqFIm620Q=;
 b=55QgxqE2cFizlZ8kNtpKAh568tZfqhHm2ey89kHnlpZvQUxIPEtL5sGBmTmzdTYu7pPdAD7UIXroCDLXBwz6H5GPisevhKsqK3Xx6LwtI/VNuavLNb5AG2nTqsaBE4SJ1PyOQIezMUnSxwVZPKroAlgeTp14z2RGkekgkEAyj8k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 04/11] xen/arm: introduce accessors for vgic dist, cpu, and rdist base addresses
Date: Thu, 23 Sep 2021 03:11:08 +0000
Message-ID: <20210923031115.1429719-5-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8a3d896b-0906-4419-ddd4-08d97e3fe0fb
X-MS-TrafficTypeDiagnostic: AM6PR08MB3221:|AM5PR0801MB1650:
X-Microsoft-Antispam-PRVS:
	<AM5PR0801MB1650E97531EF0A21E146E896F7A39@AM5PR0801MB1650.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:163;OLM:163;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 3oj82O+hzAwwN+0MZhWDjXpDvx2bWfSrU2AMJiVz+xCzac2sRZY8/N8yVEficc1fb6/VJ55+IE1M6b/XNhFxQ6NjhYh7WAg8bDShvhBaISaeeDPaCTNboXj6tkNdpAQuh+KEVgbmYElWr07GqHpdBQghNh1ZBOY+8LFjgo2/UEnwmyu4265d4FiVySO9njjidb3ZNdFEy2SjtxXIE5/Fr5STDgcNQD27XtRjNJoVr9qs7ckYWbpIDuH+ubbUQbNhhSE4K75y9ZlySUyESWGnFazvXG4jaCl8TCdyqKOB1WS3tzWr/lzhbk9aW/T6S9xbTRDOtKtzi5ID7UMF4pdCnX/LTmLQV7JQgFIdri8Q7Q2TtNm3iL4Fxjh9+G8aDfvT2jzZJoAvRzNDEyJ38y3jNkH1Zj80VR1dM4w5hfhzMZbEeIaejvkTagx2654bN10WZncLt5DW79kT6hruB8k1lJJPJuXs3kcrP1OPoGMU6RrQT50ck4TG3rEqWfPb2SjGUdVmaH8vvkYaFJ4SpMWL3P9Xe95O7/adxbJzFEtvAkIenaXKpi/wX1/TdP4ypdMLPRClb6z9tr6hIfeAl/1Wyw+z/vU4I1U7UGIh3rY5J82laYXBhAtE3/MoZvhZPp83Q9SQ+t/EPUf4YJcfYli+xiaGVAhiJGgdQGccBg0D6B1Xk2uvLEEM8hjnEEIzeRn+yNneQOScmCmnOhuHKOmpDxCHxeUnZdOYLyGFJSIod3A=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(83380400001)(44832011)(26005)(2906002)(5660300002)(2616005)(81166007)(4326008)(336012)(8676002)(70206006)(70586007)(36756003)(110136005)(54906003)(356005)(508600001)(7696005)(1076003)(186003)(8936002)(82310400003)(36860700001)(426003)(47076005)(6666004)(316002)(86362001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3221
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e09a20f9-2044-4964-ab9d-08d97e3fd89e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WGI6Mv/R8w+zpoI9En/0iMdIRs0ImFzletsm3W3349YfDALldtd6QdjrccedLsXpxGm/N7nIHSWpm9vvv83bwUMGJ2psCp9zSzmrTS5d45G6yV8JQAzr2h+bU+s6/HlaCVdkfNOHVsbFqHbpXbKnAyLaiP9IeZmP2AhBJ23GdmLt3z5zZ4NO80rhlkFQ9D7ZpXn3Og8Q9j4dEIuwIWU6Iu3oO0rC7Nje7mbA8TmXzgSMvRfUmm7xdrKKezAULDfPSpG1nafg9EUdGDi3VXmnheZkl23hfJJ2x1C5XpEH3XPzSB8mlOQZbjAX1jOb0aEd4CN6kWuGxIXmHnxaquKjUANJLV3Y7zZ6R5IfLSpqxkvVg76OAoecYB9bzS3Qm1d/TO2bcOI1tyOwLWamiIG+BB2OB5U+SeVhP9QJJLLj+5PWfdAFGa7aFYuMBHwm9bsF6FGPMlXXrtPAosuTgfmki7KMHeQqZHDXgmPR2xz3C6oJAgc9bCwvqfoWRCXaxhT9TdAo6BKAODsalWVYOSlynYoyksr16zgZRiEX1rhbYLxC47EgN1dIe7Vd3+aRUwY7cV1mOrxnvyRTk7+RMV7W7ambwncwKqZjiKoFmbTSBy/w/3pdGM33msw/SmdudQhbxHw+FiGSG2fcU1/9rFu+rq00H/DPamJFNPzEpiamBOHTsPhzPTrCNWv3A97C1JMCcl/vj307E3xI5KJ/ErSsdg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(336012)(426003)(2616005)(2906002)(8936002)(6666004)(5660300002)(110136005)(47076005)(508600001)(81166007)(83380400001)(8676002)(4326008)(36860700001)(7696005)(316002)(44832011)(186003)(1076003)(86362001)(36756003)(26005)(82310400003)(70586007)(54906003)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:47.1228
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8a3d896b-0906-4419-ddd4-08d97e3fe0fb
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1650

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Introduce accessors for vgic dist, cpu, and rdist base addresses, on
all gic types.

Use the accessors when making gic node for device tree of domU.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/domain_build.c    | 21 ++++++++++++-----
 xen/include/asm-arm/new_vgic.h | 24 ++++++++++++++++++++
 xen/include/asm-arm/vgic.h     | 41 ++++++++++++++++++++++++++++++++++
 3 files changed, 80 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 213ad017dc..d5f201f73e 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1802,8 +1802,12 @@ static int __init make_gicv2_domU_node(struct kernel_info *kinfo)
     int res = 0;
     __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
     __be32 *cells;
+    struct domain *d = kinfo->d;
+    char buf[38];
 
-    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICD_BASE));
+    snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
+             vgic_dist_base(&d->arch.vgic));
+    res = fdt_begin_node(fdt, buf);
     if ( res )
         return res;
 
@@ -1825,9 +1829,9 @@ static int __init make_gicv2_domU_node(struct kernel_info *kinfo)
 
     cells = &reg[0];
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                       GUEST_GICD_BASE, GUEST_GICD_SIZE);
+                       vgic_dist_base(&d->arch.vgic), GUEST_GICD_SIZE);
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                       GUEST_GICC_BASE, GUEST_GICC_SIZE);
+                       vgic_cpu_base(&d->arch.vgic), GUEST_GICC_SIZE);
 
     res = fdt_property(fdt, "reg", reg, sizeof(reg));
     if (res)
@@ -1852,8 +1856,12 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
     int res = 0;
     __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
     __be32 *cells;
+    struct domain *d = kinfo->d;
+    char buf[38];
 
-    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICV3_GICD_BASE));
+    snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
+             vgic_dist_base(&d->arch.vgic));
+    res = fdt_begin_node(fdt, buf);
     if ( res )
         return res;
 
@@ -1875,9 +1883,10 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
 
     cells = &reg[0];
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                       GUEST_GICV3_GICD_BASE, GUEST_GICV3_GICD_SIZE);
+                       vgic_dist_base(&d->arch.vgic), GUEST_GICV3_GICD_SIZE);
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                       GUEST_GICV3_GICR0_BASE, GUEST_GICV3_GICR0_SIZE);
+                       vgic_rdist_base(&d->arch.vgic, 0),
+                       vgic_rdist_size(&d->arch.vgic, 0));
 
     res = fdt_property(fdt, "reg", reg, sizeof(reg));
     if (res)
diff --git a/xen/include/asm-arm/new_vgic.h b/xen/include/asm-arm/new_vgic.h
index 97d622bff6..9097522b27 100644
--- a/xen/include/asm-arm/new_vgic.h
+++ b/xen/include/asm-arm/new_vgic.h
@@ -186,6 +186,30 @@ struct vgic_cpu {
     uint32_t num_id_bits;
 };
 
+static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
+{
+    return GUEST_GICC_BASE;
+}
+
+static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
+{
+    return GUEST_GICD_BASE;
+}
+
+static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
+{
+    return 0;
+}
+
+static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
+{
+    return INVALID_PADDR;
+}
+
+static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
+{
+    return 0;
+}
 #endif /* __ASM_ARM_NEW_VGIC_H */
 
 /*
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 62c2ae538d..e1bc5113da 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -277,6 +277,47 @@ enum gic_sgi_mode;
  */
 #define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
 
+static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
+{
+    return GUEST_GICC_BASE;
+}
+
+static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
+{
+    return GUEST_GICD_BASE;
+}
+
+#ifdef CONFIG_GICV3
+static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
+{
+    return GUEST_GICV3_RDIST_REGIONS;
+}
+
+static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
+{
+    return GUEST_GICV3_GICR0_BASE;
+}
+
+static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
+{
+    return GUEST_GICV3_GICR0_SIZE;
+}
+#else
+static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
+{
+    return 0;
+}
+
+static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
+{
+    return INVALID_PADDR;
+}
+
+static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
+{
+    return INVALID_PADDR;
+}
+#endif
 
 extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
 extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193321.344404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9k-0007eY-Pm; Thu, 23 Sep 2021 03:12:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193321.344404; Thu, 23 Sep 2021 03:12:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9k-0007d9-K7; Thu, 23 Sep 2021 03:12:16 +0000
Received: by outflank-mailman (input) for mailman id 193321;
 Thu, 23 Sep 2021 03:12:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9j-0004kC-Bo
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:15 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1b::621])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f8101ccc-0dfd-4e35-a006-8d585399b83c;
 Thu, 23 Sep 2021 03:12:00 +0000 (UTC)
Received: from AM6P192CA0048.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:82::25)
 by DB9PR08MB6876.eurprd08.prod.outlook.com (2603:10a6:10:2ae::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:57 +0000
Received: from AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:82:cafe::33) by AM6P192CA0048.outlook.office365.com
 (2603:10a6:209:82::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:57 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT008.mail.protection.outlook.com (10.152.16.123) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:57 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Thu, 23 Sep 2021 03:11:55 +0000
Received: from b3afd8bf3d71.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 FCDD71CF-1E59-4F18-8D71-18350386DE32.1; 
 Thu, 23 Sep 2021 03:11:48 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b3afd8bf3d71.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:48 +0000
Received: from AS9PR06CA0264.eurprd06.prod.outlook.com (2603:10a6:20b:45f::9)
 by AM0PR08MB3137.eurprd08.prod.outlook.com (2603:10a6:208:64::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:46 +0000
Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::ae) by AS9PR06CA0264.outlook.office365.com
 (2603:10a6:20b:45f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:46 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:46 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:43 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8101ccc-0dfd-4e35-a006-8d585399b83c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bAbBmE5YC0L/wCZCeTP5xeKcnHHT51Doj9fGx2jLAUo=;
 b=ybdTvmcD1KnSg0dXZBdL2ulsqEH70PJ0fBkuFVwN45EZaQBLH6DwUCuQRckO7VaXWdFI1qN0tYvGytfHZdQ0SapoTANkFbEyi0QW/AFiqS8qLXN34b3GXBwurE7hsF3f4CP2TA2Wzm6DgylgHYdkwtA0cQv32bo9jRW8q6Mwa5Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a97663ed7ac83a69
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Zaz15hzZZSlMhQ4YFfEthi4nY9dElSqwFpV2sHFsYWlxwwjFGm4VcYfh5f5nDjVhSOg3Ae7IPuQMj4rWB1S3HHmb1jEjl0Etft+uf54B3piG+E16TrHfCaoy5C5Og2uBNkZh0iFLRCP1gcosc5IKpX5hpLI5sHdFtSN3VUYEiATrLg2QlX2FCkj3jqG9hrf/3KLOifAmUTFqyVuU8NoYd6q4hDhdlgYYVB/Au0JBmRqdo35TR3Ht12HIuljrFnj2hXjB+DaYz4zgXa4Nce7fBYAi8ox1y72W1Bg9C2aEd++6RYSioQxwAT+X95Hu4RRYiTOk0xQ2lHshnZ9WldlqCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bAbBmE5YC0L/wCZCeTP5xeKcnHHT51Doj9fGx2jLAUo=;
 b=KXAf/Oa/kU0o/AzsXWmE8EzUSqCo1gzQVDv+XP6pELLejgt7LdcI0jErGYiwfi9RNbhTWtfqMb+MK0JZY4yfbxfDutZtt4ZP8qM5fXdwt73jlPojHdjjgr2cY3Y7CjX9rdtPPr8L1ZJQqZVXqX0XcoHcw/4/mXZilLQ3nq43gru98EWeAo1RzGxdi4voaR+tHEwvs21+U0X0EGutVL8R4XtD10yHyHU0pGPWCfgLRvg9XWCi56rDBxqBhDVNBcI1B3RCFezx2XitgA5mMiW9KO5BRD2dT+kSOOjIdegUSBqkYPrv2/ocUKZlu11tIg8Llq0Iw4RU0iNCCA0tSGp+hQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bAbBmE5YC0L/wCZCeTP5xeKcnHHT51Doj9fGx2jLAUo=;
 b=ybdTvmcD1KnSg0dXZBdL2ulsqEH70PJ0fBkuFVwN45EZaQBLH6DwUCuQRckO7VaXWdFI1qN0tYvGytfHZdQ0SapoTANkFbEyi0QW/AFiqS8qLXN34b3GXBwurE7hsF3f4CP2TA2Wzm6DgylgHYdkwtA0cQv32bo9jRW8q6Mwa5Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 09/11] xen/arm: if 1:1 direct-map domain use native UART address and IRQ number for vPL011
Date: Thu, 23 Sep 2021 03:11:13 +0000
Message-ID: <20210923031115.1429719-10-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 35b66eb1-e189-4e68-bb77-08d97e3fe6f4
X-MS-TrafficTypeDiagnostic: AM0PR08MB3137:|DB9PR08MB6876:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB687684EFEDFE3F1EB58A8048F7A39@DB9PR08MB6876.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +gvpXnSrQEDEXBET3ZqY/6IWO7ZXSB6V8r7f9mJ/vxxWKulcYBDZQZPItP/x0FubJB3sT+m7wWKuEsAnDy9asxpXhtVkOFujEjA+FdTJL+vJYR3b5EE2TLVfOtIsFeI5E5CfjDSAyQOIT6booKtYnk0Jizh+FfNX0hXoFJPHXelZqM/1pQWaGWAv1bsg0PKIu6blYDKWZ+aATvnbxKRSj5RgHhcJeAu5NfTyrGTXeTTT5n1gEZRKbQE1kNIzPnq9xHtzzb0TbcFnBlPhDjwB3w0FIgycFGyPTxjtUfz9Y8aiGK2uvmNd2aOnMhWh6vp0Sc4TgSEe4/juxGMY/RMKLZHrQjAogJSCkORi6+BZ4MwFxi2nFbLbToibCcp4LVlL4teSZAyBRfi7tmookYvtdcxZyS8iYYCx4CPq/IDVYk1ttqxs6k+JzQtJEpFLbh7cNpgIfxZ0+U8a5ob3KvbTrT1jlFn0KfC4NipfGhTie+xOg3ufFCzHPQLG2gOuL2AD0gt+CQgVaA3vvM6VfhTLe285DfQzVgPDFoQF6L36zXhZC62KEVI4X2L1giRfALEa0qA1IYGX7zU1iwIzWtkXx+bL1Zg4FAx7UR1iwiFlW6jg460vbo4OMnWWmXtueOv4vV8XzGxRJdfuKz2FsMpxuIW7J/iMSEYDxXlKZFL0jHL7nKkZFAV+kYaKLNbtLT9WPuWj1e4kPQKQr3OZG4eNAjHyjVCet7FIgYQweLhAjKk=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(2906002)(2616005)(1076003)(36756003)(82310400003)(4326008)(316002)(54906003)(5660300002)(8936002)(426003)(356005)(508600001)(26005)(6666004)(7696005)(86362001)(36860700001)(70586007)(186003)(8676002)(81166007)(70206006)(44832011)(83380400001)(336012)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3137
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a16817f1-73a7-44f2-73f6-08d97e3fe098
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	BdtXs0k4ZcwUxSCMSJ2ycoT7Tj4RYPiJDnzof3W/VGq+GiOK9NLqP/xu4trwAVggUUXrj4MQ4SCNfGklBVtB+FUSsKq+zLr1CaCaa3hd5/J3E48z5ceQNemk1YSTXQcdrOUae0kVWp15UxVNBnVHfDuoHeMCFkjR5nGzyez3xDpXBtO0Udz5oO40u6NhlgADfjORHpIJHX5A1G2RjjFmfBkWDMKk3Vov8QhAkuItjXDBjsERoSSeD/qFqsVklhzkEtJwwGX9RZDc1QEhuvfeXfVZGnteFIrPbObap/tEdUiWvW5Ce0eyRIOuAd7ElNrbHPAwx09ABNpdJxJvPV9RAA7CE+wikBaxJAQblSssE+/9SVRJKudwxNOaZWjKtoAbhPCH3+veTmS2bflCjYBucA7dJpZxitm11vhHY1z2jcM8OvuWGjMpsWlAJfdoysKzVjs2G1BavAlJsMV6F/l+bS8kUGUdzEPNKwiWUXKV9giOcY60v4zUryvCeWk8ZBfDHD65QWsZxfveMcDBQmL4PCRF4WuIxVJfJ2peZy1QzgOLudEZPpdTQSQCSPkNwYkkl5xvFZdtBiv8YOfaQ6TwhCkiql+2tupVBYXYFs36mXSxZBj9juHWw58HA6iG4aCXAIw6G9YNcj/8eJktC+M8isaZlBwkpPcNDo+qz14qrK6LPPTS6VAqeGtXTyafqpWXM+WAh24ZHRuD4vh+iagIxQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(110136005)(54906003)(316002)(44832011)(2906002)(186003)(86362001)(508600001)(81166007)(26005)(7696005)(2616005)(5660300002)(4326008)(70586007)(36756003)(336012)(426003)(8676002)(70206006)(82310400003)(8936002)(36860700001)(6666004)(47076005)(1076003)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:57.1714
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 35b66eb1-e189-4e68-bb77-08d97e3fe6f4
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6876

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

We always use a fix address to map the vPL011 to domains. The address
could be a problem for domains that are directly mapped.

So, for domains that are directly mapped, reuse the address of the
physical UART on the platform to avoid potential clashes.

Do the same for the virtual IRQ number: instead of always using
GUEST_VPL011_SPI, try to reuse the physical SPI number if possible.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/domain_build.c  | 34 +++++++++++++++++++++++++++-------
 xen/arch/arm/vpl011.c        | 34 +++++++++++++++++++++++++++-------
 xen/include/asm-arm/vpl011.h |  2 ++
 3 files changed, 56 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 120f1ae575..c92e510ae7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -30,6 +30,7 @@
 
 #include <xen/irq.h>
 #include <xen/grant_table.h>
+#include <xen/serial.h>
 
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
@@ -1942,8 +1943,11 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
     gic_interrupt_t intr;
     __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
     __be32 *cells;
+    struct domain *d = kinfo->d;
+    char buf[27];
 
-    res = fdt_begin_node(fdt, "sbsa-uart@"__stringify(GUEST_PL011_BASE));
+    snprintf(buf, sizeof(buf), "sbsa-uart@%"PRIx64, d->arch.vpl011.base_addr);
+    res = fdt_begin_node(fdt, buf);
     if ( res )
         return res;
 
@@ -1953,14 +1957,14 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
 
     cells = &reg[0];
     dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS,
-                       GUEST_ROOT_SIZE_CELLS, GUEST_PL011_BASE,
+                       GUEST_ROOT_SIZE_CELLS, d->arch.vpl011.base_addr,
                        GUEST_PL011_SIZE);
 
     res = fdt_property(fdt, "reg", reg, sizeof(reg));
     if ( res )
         return res;
 
-    set_interrupt(intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+    set_interrupt(intr, d->arch.vpl011.virq, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
 
     res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
     if ( res )
@@ -2670,6 +2674,13 @@ static int __init construct_domU(struct domain *d,
     else
         allocate_static_memory(d, &kinfo, node);
 
+    /*
+     * Initialization before creating its device
+     * tree node in prepare_dtb_domU.
+     */
+    if ( kinfo.vpl011 )
+        rc = domain_vpl011_init(d, NULL);
+
     rc = prepare_dtb_domU(d, &kinfo);
     if ( rc < 0 )
         return rc;
@@ -2678,9 +2689,6 @@ static int __init construct_domU(struct domain *d,
     if ( rc < 0 )
         return rc;
 
-    if ( kinfo.vpl011 )
-        rc = domain_vpl011_init(d, NULL);
-
     return rc;
 }
 
@@ -2723,15 +2731,27 @@ void __init create_domUs(void)
 
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
         {
+            unsigned int vpl011_virq = GUEST_VPL011_SPI;
             d_cfg.arch.nr_spis = gic_number_lines() - 32;
 
+            /*
+             * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map in
+             * set, in which case we'll try to match the hardware.
+             *
+             * Typically, d->arch.vpl011.virq has the vpl011 irq number
+             * but at this point of the boot sequence it is not
+             * initialized yet.
+             */
+            if ( direct_map && serial_irq(SERHND_DTUART) > 0 )
+                vpl011_virq = serial_irq(SERHND_DTUART);
+
             /*
              * vpl011 uses one emulated SPI. If vpl011 is requested, make
              * sure that we allocate enough SPIs for it.
              */
             if ( dt_property_read_bool(node, "vpl011") )
                 d_cfg.arch.nr_spis = MAX(d_cfg.arch.nr_spis,
-                                         GUEST_VPL011_SPI - 32 + 1);
+                                         vpl011_virq - 32 + 1);
         }
 
         /*
diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 895f436cc4..10df25f098 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -29,6 +29,7 @@
 #include <xen/mm.h>
 #include <xen/sched.h>
 #include <xen/console.h>
+#include <xen/serial.h>
 #include <public/domctl.h>
 #include <public/io/console.h>
 #include <asm/pl011-uart.h>
@@ -71,11 +72,11 @@ static void vpl011_update_interrupt_status(struct domain *d)
      * status bit has been set since the last time.
      */
     if ( uartmis & ~vpl011->shadow_uartmis )
-        vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
+        vgic_inject_irq(d, NULL, vpl011->virq, true);
 
     vpl011->shadow_uartmis = uartmis;
 #else
-    vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, uartmis);
+    vgic_inject_irq(d, NULL, vpl011->virq, uartmis);
 #endif
 }
 
@@ -347,7 +348,8 @@ static int vpl011_mmio_read(struct vcpu *v,
                             void *priv)
 {
     struct hsr_dabt dabt = info->dabt;
-    uint32_t vpl011_reg = (uint32_t)(info->gpa - GUEST_PL011_BASE);
+    uint32_t vpl011_reg = (uint32_t)(info->gpa -
+                                     v->domain->arch.vpl011.base_addr);
     struct vpl011 *vpl011 = &v->domain->arch.vpl011;
     struct domain *d = v->domain;
     unsigned long flags;
@@ -430,7 +432,8 @@ static int vpl011_mmio_write(struct vcpu *v,
                              void *priv)
 {
     struct hsr_dabt dabt = info->dabt;
-    uint32_t vpl011_reg = (uint32_t)(info->gpa - GUEST_PL011_BASE);
+    uint32_t vpl011_reg = (uint32_t)(info->gpa -
+                                     v->domain->arch.vpl011.base_addr);
     struct vpl011 *vpl011 = &v->domain->arch.vpl011;
     struct domain *d = v->domain;
     unsigned long flags;
@@ -622,10 +625,27 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
 {
     int rc;
     struct vpl011 *vpl011 = &d->arch.vpl011;
+    const struct vuart_info *uart = serial_vuart_info(SERHND_DTUART);
 
     if ( vpl011->backend.dom.ring_buf )
         return -EINVAL;
 
+    vpl011->base_addr = GUEST_PL011_BASE;
+    vpl011->virq = GUEST_VPL011_SPI;
+    if ( is_domain_direct_mapped(d) )
+    {
+        if ( uart != NULL && serial_irq(SERHND_DTUART) > 0 )
+        {
+            vpl011->base_addr = uart->base_addr;
+            vpl011->virq = serial_irq(SERHND_DTUART);
+        }
+        else
+            printk(XENLOG_ERR
+                   "Unable to reuse physical UART address and irq for vPL011.\n"
+                   "Defaulting to addr %#"PRIpaddr" and IRQ %u\n",
+                   vpl011->base_addr, vpl011->virq);
+    }
+
     /*
      * info is NULL when the backend is in Xen.
      * info is != NULL when the backend is in a domain.
@@ -661,7 +681,7 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
         }
     }
 
-    rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
+    rc = vgic_reserve_virq(d, vpl011->virq);
     if ( !rc )
     {
         rc = -EINVAL;
@@ -673,12 +693,12 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
     spin_lock_init(&vpl011->lock);
 
     register_mmio_handler(d, &vpl011_mmio_handler,
-                          GUEST_PL011_BASE, GUEST_PL011_SIZE, NULL);
+                          vpl011->base_addr, GUEST_PL011_SIZE, NULL);
 
     return 0;
 
 out2:
-    vgic_free_virq(d, GUEST_VPL011_SPI);
+    vgic_free_virq(d, vpl011->virq);
 
 out1:
     if ( vpl011->backend_in_domain )
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index e6c7ab7381..c09abcd7a9 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -53,6 +53,8 @@ struct vpl011 {
     uint32_t    uarticr;        /* Interrupt clear register */
     uint32_t    uartris;        /* Raw interrupt status register */
     uint32_t    shadow_uartmis; /* shadow masked interrupt register */
+    paddr_t     base_addr;
+    unsigned int virq;
     spinlock_t  lock;
     evtchn_port_t evtchn;
 };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:12:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:12:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193328.344415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9u-0000AB-Bt; Thu, 23 Sep 2021 03:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193328.344415; Thu, 23 Sep 2021 03:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTF9u-00009n-7B; Thu, 23 Sep 2021 03:12:26 +0000
Received: by outflank-mailman (input) for mailman id 193328;
 Thu, 23 Sep 2021 03:12:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTF9t-0004kC-CQ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:25 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.63]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3eef12a7-de91-4816-ab8e-198bc8c41755;
 Thu, 23 Sep 2021 03:12:02 +0000 (UTC)
Received: from DB6PR07CA0054.eurprd07.prod.outlook.com (2603:10a6:6:2a::16) by
 VI1PR08MB4573.eurprd08.prod.outlook.com (2603:10a6:803:e7::20) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16; Thu, 23 Sep 2021 03:11:58 +0000
Received: from DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2a:cafe::50) by DB6PR07CA0054.outlook.office365.com
 (2603:10a6:6:2a::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:58 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT016.mail.protection.outlook.com (10.152.20.141) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:58 +0000
Received: ("Tessian outbound b324a1dbd9be:v103");
 Thu, 23 Sep 2021 03:11:57 +0000
Received: from 198723602d40.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F9D104AC-E4B8-4AFA-86B6-D6B946F524CE.1; 
 Thu, 23 Sep 2021 03:11:46 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 198723602d40.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:46 +0000
Received: from AM6P195CA0016.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:81::29)
 by AM0PR08MB5153.eurprd08.prod.outlook.com (2603:10a6:208:163::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:37 +0000
Received: from AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:81:cafe::35) by AM6P195CA0016.outlook.office365.com
 (2603:10a6:209:81::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:37 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT021.mail.protection.outlook.com (10.152.16.105) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:37 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:36 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 03:11:35 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3eef12a7-de91-4816-ab8e-198bc8c41755
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qC0WX9sJbstRrjXlzSjejZBynjOxc/CK8y9qTDR6lQ4=;
 b=Gfs4QB5l8MAA0dweywDLLlgFiSLA2/af2fo75Ni1hwK/0nc+4YFdMpa+r6jvArtfXllOEO9A/39VUJFykBbZFZZXx0sEhn1+Q6daBkUgBX4ejafzwGaiKj8VWrNe7KdGS7VLNZ9Wku10UUl/ylz7DlJhDAnbCGjmRBUyNqmx0mw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: eb2696ad9cb63629
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PxXRhu7kduSPdz13U+dMnxRE6p4Yw0a1TbfNuvbJFtDLttKzocU4WnaxLb1ocJS6zrGcXiCWMoTCYgm2pjmkxB09Vu0TRg9cvZJZSPtqeWUtfgalqlmXKI6inwmpEXMYv+4QWh5nYWlNvvG4LXxnKPUMvOcyKgOb/fuQLg8nZql7MkhL92nlWyaZc4/NBtUA0PDeFE7let5ha1QVHCEoFZex2168adlyzeA/XmMLreW11w1vg8urwLjWtl0hqLIra+D9Y7vjS8zsuhKoal82HcZ6/ksTblFdfvVjY806cLa6dsQBCxbmIdmBxv3udTVrYr7JxF0bNAO7FZyAgvg89Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qC0WX9sJbstRrjXlzSjejZBynjOxc/CK8y9qTDR6lQ4=;
 b=kFknQQB/tiF61OPW+ctwU2+t2YP49sbpNus/4Fv90UhC3uDKrVdqJS0INicbYoJiP4pJGSrW6CNM5IdZF6wcY8wGQ+jaYMKXdJXpU/VCnbu13GxLOHkvL8JU0GxPCHGnavw7fom15zUaj3V+O/Mc8VJuLzwAwprhX1vSVjc/57/HGo0ltowpKrLh5m8gQK2z2JHvrYwE88ZMwWWqJArwFI6vLnEY/qz/dbwjLFDV0GaB79y148Js/k0IfXjRMH7DqOaY2CsRef/B9X+/C7EffWJf48fXfmoEnJy69zn+bHdzXxWmN0duDW8qsn6SwxAU0ZGN2prTU7fuOjJf57GYFg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qC0WX9sJbstRrjXlzSjejZBynjOxc/CK8y9qTDR6lQ4=;
 b=Gfs4QB5l8MAA0dweywDLLlgFiSLA2/af2fo75Ni1hwK/0nc+4YFdMpa+r6jvArtfXllOEO9A/39VUJFykBbZFZZXx0sEhn1+Q6daBkUgBX4ejafzwGaiKj8VWrNe7KdGS7VLNZ9Wku10UUl/ylz7DlJhDAnbCGjmRBUyNqmx0mw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 06/11] xen/arm: new vgic: update vgic_cpu_base
Date: Thu, 23 Sep 2021 03:11:10 +0000
Message-ID: <20210923031115.1429719-7-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8b464054-658e-4d32-8440-08d97e3fe794
X-MS-TrafficTypeDiagnostic: AM0PR08MB5153:|VI1PR08MB4573:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB45732942D0756ECDB76E918FF7A39@VI1PR08MB4573.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4M0NxTjWlG1Ilt6az62awmewDr3GItBbAXJHmqHmB/cxObbHKzfmFn3mul+vTirRds0AXVJfnLbselQdSkaXQcre/gs0hwnyZT6OML7IW1CtB7hqhbIbKEt8vQStDTrgrSi48zPxG1vWiT/wFM4Fu6UlLd+3jOm5QnCKmKJNNHUWi3jC9gKAAQdUCHdG912HlSFMsrtCa3G/49OhEkMOYxracn1bHFQ74M/SOKCDCRq6jeGp2JuoewLZq4/lGi1WWNvUl4cYUuDynFCkG1que+I86HqjCk9bfLZ0O76GuBxcXT4xv63wklxAVDOCdDpKCZ/IA291T6be3Iq9XsXoY825v7XMhwGEZ38VXIr7M/jRs6LWfHQUoTdJppbYRxSC3xG9sXY0wO8PSxUh+ylUJQ/d914U3WqakO5imbDtOPiDKBtY2CiBGKQqmhhOtYR8vS4lF3Vaje7horSPDcR2BdzrojJoJlw5UOn7NDbdG5GhYhpoEC0igSoNuRqIsiXY4JOWeVfMcURVqCWTAecQOKYIV9azpEi6dogn/q0EwqKzDN71eFlvLqtg4BTDDiGm3t8K6i1fQ8v1TY8Tr/Vy2MNj3JoO5y/1CPlYCAJcNlu/qozb0P0K9gR3iSjyTwawlS5oB/c2S9eZN/8X8bT55cTdN8AC11DtQ1rvxYLSqgQSKP21FiyfutgBj67rKeJEGDxa1Qk+Yok0FQ6ZF765DjGyGpahUTm03e1s6wJj6EM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(316002)(2616005)(1076003)(54906003)(110136005)(8676002)(426003)(336012)(82310400003)(508600001)(5660300002)(70206006)(81166007)(356005)(70586007)(7696005)(8936002)(186003)(83380400001)(4326008)(86362001)(36860700001)(26005)(47076005)(2906002)(6666004)(36756003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5153
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3f7393c9-fdd9-4779-f8f4-08d97e3fdb28
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Lpmazvuv7ipXPDHPZv3SEPH25U5DLAB3SNflQiGP1x8PQwoB0USRVfE6o5U494AVO3JqVPA0zZZDRRwcDe2Zxc/7xAbxmPuAkd1YQWWm+k1DOikZLmfr89DJOcOBn+D/PUS0Q3ScXOvwvfMwnynwiez/WFQcUoszbTsDBo38sTqpGETkXLf2L5IFqQr3y6D5g9lE4ZfCwmgjLh6JjIft92hci1fAjY/Y6eF3ccarcpbVGTHNS2EeQiaqIccqWJqTDrGpSh0ferrRPNEoFKjWtn9zg4i2BPEIG8KHmRjic5lT98LUuhIaIKlLndtNwWE1rAJn7KP+bBB6qymz633Lsx7VDeQk/rKvjd8FzBdVsSnifqJ0ld7rnSSTjW5UBL9IWMBvB5pChO7FwfX4kUHLRsX91y35xSUrgPYXWVKybGIa719oKft7akmtXa3wIqyTdF65AnQu82Xz0AdkbwX1lUv76y+QhH9WBIwB0yMFd5I0BPcJBcbyUjyECpXSZ1ElFV0IctCttqMvwo3Lf+fjKsyxivF5xbirvpzc+Yjp8aDcdTuRSoSdJOQJSrVW0C9vEFI0x4kpcwmnaR7x6VGpRGS+0wKilfov8Qf6yksBH7u8hak1Za84vO2ZnG8sk16eoF6uqJKb2dZ2JTQAjL7ZgO/iWI+ZxPHhbAZW1twS7+Lz8gaw4PoYA1+y3pk3HYBNEdPkXikVjmPRd7F34sVTiA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36860700001)(86362001)(5660300002)(82310400003)(316002)(26005)(70586007)(7696005)(83380400001)(8936002)(44832011)(508600001)(1076003)(336012)(4326008)(2616005)(426003)(81166007)(2906002)(47076005)(186003)(70206006)(8676002)(110136005)(6666004)(54906003)(36756003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:11:58.2885
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b464054-658e-4d32-8440-08d97e3fe794
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4573

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

In vgic_v2_map_resources, use the new vgic_cpu_base field of struct
vgic_dist instead of original local variable.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/vgic/vgic-v2.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
index b5ba4ace87..fd452fcb5a 100644
--- a/xen/arch/arm/vgic/vgic-v2.c
+++ b/xen/arch/arm/vgic/vgic-v2.c
@@ -258,7 +258,7 @@ void vgic_v2_enable(struct vcpu *vcpu)
 int vgic_v2_map_resources(struct domain *d)
 {
     struct vgic_dist *dist = &d->arch.vgic;
-    paddr_t cbase, csize;
+    paddr_t csize;
     paddr_t vbase;
     int ret;
 
@@ -276,7 +276,7 @@ int vgic_v2_map_resources(struct domain *d)
          * Note that we assume the size of the CPU interface is always
          * aligned to PAGE_SIZE.
          */
-        cbase = gic_v2_hw_data.cbase;
+        d->arch.vgic.vgic_cpu_base = gic_v2_hw_data.cbase;
         csize = gic_v2_hw_data.csize;
         vbase = gic_v2_hw_data.vbase;
     }
@@ -290,7 +290,7 @@ int vgic_v2_map_resources(struct domain *d)
          * region.
          */
         BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
-        cbase = GUEST_GICC_BASE;
+        d->arch.vgic.vgic_cpu_base = GUEST_GICC_BASE;
         csize = GUEST_GICC_SIZE;
         vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
     }
@@ -308,8 +308,8 @@ int vgic_v2_map_resources(struct domain *d)
      * Map the gic virtual cpu interface in the gic cpu interface
      * region of the guest.
      */
-    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
-                           maddr_to_mfn(vbase));
+    ret = map_mmio_regions(d, gaddr_to_gfn(d->arch.vgic.vgic_cpu_base),
+                           csize / PAGE_SIZE, maddr_to_mfn(vbase));
     if ( ret )
     {
         gdprintk(XENLOG_ERR, "Unable to remap VGIC CPU to VCPU\n");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:17:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193343.344426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFEN-00026N-0v; Thu, 23 Sep 2021 03:17:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193343.344426; Thu, 23 Sep 2021 03:17:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFEM-00025l-RL; Thu, 23 Sep 2021 03:17:02 +0000
Received: by outflank-mailman (input) for mailman id 193343;
 Thu, 23 Sep 2021 03:17:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTFAX-0004kC-Dn
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:13:05 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fb3b812b-6dd4-4371-b27f-310d578d9eb4;
 Thu, 23 Sep 2021 03:12:10 +0000 (UTC)
Received: from DB6PR0601CA0019.eurprd06.prod.outlook.com (2603:10a6:4:7b::29)
 by DB9PR08MB6428.eurprd08.prod.outlook.com (2603:10a6:10:263::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 03:12:07 +0000
Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:7b:cafe::10) by DB6PR0601CA0019.outlook.office365.com
 (2603:10a6:4:7b::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 03:12:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:12:07 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Thu, 23 Sep 2021 03:12:06 +0000
Received: from f22e634b0734.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BD0D7A2A-0C9C-4596-B7FE-4BD63411E029.1; 
 Thu, 23 Sep 2021 03:11:55 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f22e634b0734.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:55 +0000
Received: from AS9PR06CA0254.eurprd06.prod.outlook.com (2603:10a6:20b:45f::29)
 by DB7PR08MB3259.eurprd08.prod.outlook.com (2603:10a6:5:1f::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 03:11:47 +0000
Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::6f) by AS9PR06CA0254.outlook.office365.com
 (2603:10a6:20b:45f::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:47 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:47 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:45 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb3b812b-6dd4-4371-b27f-310d578d9eb4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5SA4d2GeBF/c/7lvVqfUDdcRMEVLTNcBLtAK0ks8Cfg=;
 b=Ur4Mcgrca3EVT13PDYApcCW+O3mXljiraAtQZOSqoZWUnCSSoZhb0OSuPKkpBzjt7FplnRASaZ84gtsr5P2a6UGQoVSYxDrV5/11dLvkdS+rafDKzYzbgoykxfIyKgueJCWzZj7bSOVQqk+y8W6TgRxJv9jpRK5f+QJ9GorDqt0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cf5a9cc62c8cf42a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Utw3mh2tL9u54gxSJBfNFCpBgsU32aT5P4B4Jk7bJsV426RPe1lcZi6dP/BtGat2QDEczgHJNq41eBxAmJDcSEVYZ0N0uuBIC50qEcxcnZHlj76P/FDjer+Umky9nld0J9PMxWLRl/mj4uwtAD6wmkq3ntzMItbDIuPMBMNgapyIBu0Z7LIzA6jfOge87o5vp2yXuXL7lA/6HFt6vGypk8z4TIS74HZwQDds98DI9PiTctA9sWR7whoLa15jz0yhlsBX+AgdmjOG6SUvoCQ7M5k3PsNJI7DHFl3/7AP7+noPPddieH3YDw8hE008gxKdwcLfrLtlImEUL7hohWfkCA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5SA4d2GeBF/c/7lvVqfUDdcRMEVLTNcBLtAK0ks8Cfg=;
 b=cg0STODoC+kwN6BBhu+tY1r8ABimttGv0/VdcXdREYEbl9W3JwdBQGaacTncHMwXpE3DyncCt2vi63IAdM8hMqPcTNQaIoPLl/rrN4ryplZQr+8FzHxZGf/dv2mJqxKkEwHoGQIwWH1cKJ/e6XvB6t0fu3+WnAilIyJW0KbBoreof/JODSwhiWCE+W3nqcA24yMysssbTnQSQkQGxHD6rixBDVCXs67TgfeF6hL2ulQEnaUhAPWreNCHWJa/OysbNcCh9tDjgrHLjSDvM/SYgWoELYwBwhh2EgcCBuE2R2wwp0pjgmC2Kp4cD+R+AbaBcPBwl5YWGsRz+TOLYB9/4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5SA4d2GeBF/c/7lvVqfUDdcRMEVLTNcBLtAK0ks8Cfg=;
 b=Ur4Mcgrca3EVT13PDYApcCW+O3mXljiraAtQZOSqoZWUnCSSoZhb0OSuPKkpBzjt7FplnRASaZ84gtsr5P2a6UGQoVSYxDrV5/11dLvkdS+rafDKzYzbgoykxfIyKgueJCWzZj7bSOVQqk+y8W6TgRxJv9jpRK5f+QJ9GorDqt0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 10/11] xen/arm: device assignment on 1:1 direct-map domain
Date: Thu, 23 Sep 2021 03:11:14 +0000
Message-ID: <20210923031115.1429719-11-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a2df8bc4-31ff-4b02-8d2f-08d97e3fed10
X-MS-TrafficTypeDiagnostic: DB7PR08MB3259:|DB9PR08MB6428:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB6428A1CF676E76A321978571F7A39@DB9PR08MB6428.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 mC71vN8PmQhpFwfMtuEw25+9nkVlkF1H6N8g4warMvgpphEbiMth5DFk7ZddUe3Ei37ELoJuRVHrGEx1NHMTiYTN46MpNR95YXL42EblJaIk5fO/e75lN3DNzgrnY74lN7nf2YBClspzZo/dvA3QUqR/vdbyzOdJZcn+i2b260pikRQTxeQb6dOFuNts+0g11PS4x7YeibFMhaAxotzTJE+X93MMPdk1zmVclR918DrDlVtsOeEvfytc6hHDvE9XkVsguavhTbct4aiJRP23sicz0X/RtImu5tGBOMU8liH5wb46uBVVxEOjud+l47uHH3vOIVMVLSJ01RihWoKmdVFevKk4Cx9rmyoBcNn/m1EkBUOcFYOk0252ytfEryP/ClMLuzpLtf6A66nI6J9GtjwdFucN3Dgp9hLEVtvoDqgx1cIQeQj2u9O7ZXx7oK7QOLlf+/+CY5P9cFo39PxWLxg2hOXwU8Ypb1D4uz5Iw93mAla1Klf/zkOEdwu0uyOZeJ8EA24vY/ai23VuINrNzy9PH4WDmFcmRJgfXmS392jPXj3Ci4wM8p7LzJw6yumc3YcF7kIEbV66xR25LNfrxfA6A3qg0vUgamcM1TPPAa5pFUyb74Pp+KG9eWYLXmuY0vo3JiPIEj0PpTKU2f/juKG/ys0ylA2UEbv4FwrTYlC75ZRIvheu3FQU1bxm0zYQ7ul8Y+M18gdPc86N5cYRMCvxoTXQW96OIGbH2HoD5XE=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(82310400003)(2616005)(7696005)(47076005)(336012)(4326008)(2906002)(426003)(86362001)(44832011)(8676002)(316002)(36860700001)(1076003)(8936002)(5660300002)(6666004)(83380400001)(110136005)(70586007)(36756003)(26005)(186003)(81166007)(356005)(70206006)(54906003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3259
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	4008d624-2b61-497f-b4cd-08d97e3fe12f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+mXLZhU8nfvo6aDPST20DoqJwMn/6reOzzOmVo5Nw0yMJDc9X8B8eDKs+MgNYI6mCDT/rW3KELft2aQJuW/51rtK5oqYEX13+Khktdb+GGUAcmcX3F66x1x/SfBHmPSq4c+eF77UlTatDUY9flktSo2kncYv1CDNXZFCXsAzHIkIM/AcQbuUPEC5AzBKS+xJyZbI0Rn8Agn7EbqlGnEo4OphI0vn8W98y/6uJLouc0xTaYh4/lRtnucOF0OCEb/ypcfCc3kGDJyVa02Myy596a4fM4I0qZHzpvip1V7IplEkTcCN8jyS277eUms8GnQ/qCzrRRKsk8D1f9sUsco31fpbHC2zHSU5j8eiIbCpqVcxUil7OOxjkwL0BTE2MbWZ7ncY1dzvVlLZ3K1DsM0T5szLOxvkIlEh/RLqAUeODF/kvcm/bCHHfNQpTzV9HF+JmDiDS9dZehjs7VsoFtaGkVdnTx8cYZd1RcQA2hq5bZxqf0gSLNS01LVpqBW1qimP3xnQF3RZ0j4i0yPTRz7sS7o7ZIIPlNYgR49mehhk5LdbJkI7wIB/o2xYx27Hdh8/37OGMqZW2OruIpkZwfco/n6rmMfrXsMVKXQhFo1gf9PErQlj9um2hN79ITEE5929N84Km2hdLldLHtu1xWvBGbxjZX7ObSSoMJwKbPG/hSDGG01xJ58yYhgB40Al2L0RyZxqQVUnTiJrHyvtZNrbLw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70206006)(70586007)(81166007)(508600001)(426003)(54906003)(83380400001)(336012)(316002)(36756003)(8676002)(82310400003)(110136005)(5660300002)(2906002)(6666004)(26005)(44832011)(36860700001)(7696005)(4326008)(86362001)(1076003)(8936002)(186003)(47076005)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:12:07.4890
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a2df8bc4-31ff-4b02-8d2f-08d97e3fed10
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6428

User could do device passthrough, with "xen,force-assign-without-iommu" in
the device tree snippet, on trusted guest through 1:1 direct-map,
if IOMMU absent or disabled on hardware.

In order to achieve that, this patch adds 1:1 direct-map check and disables
iommu-related action.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 xen/arch/arm/domain_build.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index c92e510ae7..9a9d2522b7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2070,14 +2070,18 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo,
     if ( res < 0 )
         return res;
 
+    /*
+     * If xen_force, we allow assignment of devices without IOMMU protection.
+     * And if IOMMU is disabled or absent, 1:1 direct-map is necessary
+     */
+    if ( xen_force && is_domain_direct_mapped(kinfo->d) &&
+         !dt_device_is_protected(node) )
+        return 0;
+
     res = iommu_add_dt_device(node);
     if ( res < 0 )
         return res;
 
-    /* If xen_force, we allow assignment of devices without IOMMU protection. */
-    if ( xen_force && !dt_device_is_protected(node) )
-        return 0;
-
     return iommu_assign_dt_device(kinfo->d, node);
 }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:17:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193365.344437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFEV-0002Xt-7b; Thu, 23 Sep 2021 03:17:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193365.344437; Thu, 23 Sep 2021 03:17:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFEV-0002Xh-3o; Thu, 23 Sep 2021 03:17:11 +0000
Received: by outflank-mailman (input) for mailman id 193365;
 Thu, 23 Sep 2021 03:17:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qvJY=ON=arm.com=Penny.Zheng@srs-us1.protection.inumbo.net>)
 id 1mTFAI-0004kC-DH
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:12:50 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1440ab9d-f2de-4053-afbd-a762e60e0d86;
 Thu, 23 Sep 2021 03:12:08 +0000 (UTC)
Received: from PR2P264CA0035.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::23) by
 VE1PR08MB4927.eurprd08.prod.outlook.com (2603:10a6:802:b1::14) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 03:12:05 +0000
Received: from VE1EUR03FT006.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:101:1:cafe::f0) by PR2P264CA0035.outlook.office365.com
 (2603:10a6:101:1::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:12:05 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT006.mail.protection.outlook.com (10.152.18.116) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:12:04 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 03:12:03 +0000
Received: from 7325449563a6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3642A666-F715-4B71-9D5D-AA48DAE3704D.1; 
 Thu, 23 Sep 2021 03:11:52 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7325449563a6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 03:11:52 +0000
Received: from AM5PR04CA0005.eurprd04.prod.outlook.com (2603:10a6:206:1::18)
 by AM9PR08MB6644.eurprd08.prod.outlook.com (2603:10a6:20b:30e::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 03:11:50 +0000
Received: from AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:1:cafe::35) by AM5PR04CA0005.outlook.office365.com
 (2603:10a6:206:1::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:50 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT014.mail.protection.outlook.com (10.152.16.130) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 03:11:50 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 03:11:49 +0000
Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 03:11:47 +0000
Received: from penny.shanghai.arm.com (10.169.190.66) by mail.arm.com
 (10.251.24.32) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 03:11:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1440ab9d-f2de-4053-afbd-a762e60e0d86
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RUJuKpAtpsPU2vd6yUZACXFm8wINo0vkIbxoIx5eFVM=;
 b=ypSDbTU8t9Xwb5caDPUzAwZ6mMmObt51t5Wce17gQJ36XEHQoO4UF1jS24nL43tqIISmAurDVGgT6RXUZQ4s+o8EUoQWElq814kCrgnrOJ98g/VYlgE3ddXvF5i7VkBTPSecieN7ELAXfM53SDWgzLSwQp5P0WNlobjqvw8pAJM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 487e76dfad220cde
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Bq2Zpaw/e9e1p1xcN78eXrjRSnjtprwT6B7MaIiPQ38naQONa81rVczoNTbhMzi1n/iwKWKjrXnJrDp+y4XqayaS3yDY37EHAWMDdfXi3oDP70vYT7md+7QiHzu3O/Ojm/zIQJcd6+xmwByXLt4xLVVEyRM1vlgNrDYcMXcTbbhj8oaoQTM/QNh+q/JN5NA6j1ovikwgvuSCCsmNrDwBcx5m+bOollfoqNTWeeh4fx9vs7QrHy+NPbi2U7jcs1Zftc9b9MXgQEqWY/8kMZpiAWzeQaYCquaCB4PicelM+wSpY884BP6JwJVSISmDpAleL5V97imKWQfRWC7Cv1/gmA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=RUJuKpAtpsPU2vd6yUZACXFm8wINo0vkIbxoIx5eFVM=;
 b=h9rJhd93Bpcl7698BBThJLkAX3cxhPcMzqD8I8smmAqbx9JM8Rh6tUMjmQP+pors0wzNC9zOO6vCMhj77ns15Fj58tUt3wDsd25VgNMYtFsTeiJQ5RpxKIAFWK2V7M7FuMwMa0tSejXe+cKVCMR44NkbuB+4flC0Q3Wg7amew8FwTEJJ75VrAyYJQbTCqNJCv13WyISAX2v84YCRp/sHEa2UWak7TfJmPnCRxEnFmsvQxUSfGQGlkDr+l5HoCgiCWRwZKioCXMBxgZmiW1N70GYHs4piSH6+ll0wctMc+IBgtl1f+9l/CI6s7DAnq6Z/mwoVjL00Uh+g91PDcQeX0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RUJuKpAtpsPU2vd6yUZACXFm8wINo0vkIbxoIx5eFVM=;
 b=ypSDbTU8t9Xwb5caDPUzAwZ6mMmObt51t5Wce17gQJ36XEHQoO4UF1jS24nL43tqIISmAurDVGgT6RXUZQ4s+o8EUoQWElq814kCrgnrOJ98g/VYlgE3ddXvF5i7VkBTPSecieN7ELAXfM53SDWgzLSwQp5P0WNlobjqvw8pAJM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Penny Zheng <penny.zheng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Penny.Zheng@arm.com>, <Wei.Chen@arm.com>
Subject: [PATCH 11/11] xen/docs: add a document to explain how to do passthrough without IOMMU
Date: Thu, 23 Sep 2021 03:11:15 +0000
Message-ID: <20210923031115.1429719-12-penny.zheng@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923031115.1429719-1-penny.zheng@arm.com>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d862402-9c47-41b0-0b5b-08d97e3feb73
X-MS-TrafficTypeDiagnostic: AM9PR08MB6644:|VE1PR08MB4927:
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB49272BB9CE5A81EB801A9458F7A39@VE1PR08MB4927.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 S0mO2j1P9agY2idP2mQTWaiKi+dMeW0y3fZg2Ubl9IzhnhYufsj3KwoXioheiXCUHt+eS80R0WjPY4o9fCMgM5iEFuIR8Lr4sbacS2sfQXDQu+erv0aOZ3Mps0bxvxnEonlOPJvuxuhzRvXmNTS8CuItWiJPuYxwvGkJnYp4UtxTIuUIYgsNCaYIbo7sP/vLZrrvm2PFLoJogREBk4jxsmkmLzXlTWQ42EdO93n4BeetvKon/kA62ZiPov8AjPtALtV24GlHmYHawGn/Y7EaV8qYVPaK20lHwVALaXEwSHN+Gtn15JMT7TdmU3PnhM0U7gXEWpA4ZfUMAkeI5V4cFZZl/vm4wgK0kx4uAYB9wLD0/svpDzYrPnBKY+sL1idwD45yqR6GucLSu8M1wJpcazYt8h9yCk2wHgszHY0F9YGq7RW6p6fuVU4k7gRl9zCxn7MiSLfQSuH0dS9d3fHrAKWe8rECyuoVp0GdoxThnS5LAcnPAXunTcc6JMsHp/yJpTHpHdyXtlpeVUcoxv8kMLnDtOdSobE9koz8N+fQEKe31ck6hN/6lmjAMpZIjsjMHYN5fCars2a5OuBDzj7a4kylH1+rLOFDuRa5zt8gb+ue5z95o89OFTMPt8AOi5tT88hD+WubnXEf+6Dm8Ib5CyLOkI5v08K9shAjH2W4eAm8FkYelwwCDzcC7EzTAFQ3fOu5q6Ke9yaULajFS1wtiy9/Pi5Oqs+MdNtVZC/DG2A=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70206006)(47076005)(2906002)(2616005)(8676002)(356005)(7696005)(4326008)(82310400003)(70586007)(5660300002)(6666004)(44832011)(336012)(508600001)(83380400001)(36860700001)(81166007)(54906003)(186003)(8936002)(1076003)(316002)(36756003)(86362001)(426003)(110136005)(26005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6644
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT006.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	db3771a0-69b3-494f-7eac-08d97e3fe2c4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sIx/sxz/+QFN0YlTZd9vWuRFe4Oz3B2WdnIzdbXHfErI5uJa/4l1lbg1iurV9AC33foj5twiXemyyol3dQnEp+zVFdKMjhtKaabm2PXiuma9Jxb2NVnAdQkiEk+LwvpvKzP9RlXiQFaCdBV0yif/PwO/93P96ZTahsqRZow1B3lHOQkr4EX1ckAs+oOCn7g3cn5QgrGUMaVP86cZ4hbyXSFkGG1aQNxX6etWWEu3+HjWI0e0EqvfZ9jqsUjC/hyj44gqpY/uM0On+h78Li4/xaH2ZbLcpCDzM1d0VRZJboCc+kkH4Rmgequ9F/RvEKd8wmWRWQaB9o4Awpfr/jA74gN5gkFeRanhKkIOqYBcOaj0uvOOowyB0Kn+uP7HRT2OZnMc9YkllxHaSPVi1pcAB4t7xHAiFr2tnZd0Xv5GDHKQNkNsJG2T//oOFXj8q1708Se5XHJE5E0yE/H1Fq2ngDlG3OrgYOwJYU0SioTEpxtURPCkmNbfJoRQlYrvNLgs0AyNR3cx/+LuMGRPkIM6FFzEVnsvXfj3lZliahZl/EnIz5bXIJf760fPlBW2y9uyAjzgVGHpIpE28Je9EAqqck74fuYQ+d0u2w88rHZyig29Dvl4gJFic0avJDaru8EWTepfFMCTpuH7PJvh4dO+tgQoQ7Qzmbl+ngoxi4uvQPU83ER2hFx5pcudMbmDugGcnraUXFfAlPM91Dp2mNKynw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(7696005)(81166007)(2616005)(44832011)(36860700001)(8936002)(8676002)(2906002)(5660300002)(83380400001)(1076003)(82310400003)(86362001)(110136005)(47076005)(316002)(36756003)(6666004)(26005)(54906003)(426003)(186003)(4326008)(70206006)(70586007)(336012);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 03:12:04.6851
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d862402-9c47-41b0-0b5b-08d97e3feb73
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT006.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4927

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Make sure to start with a WARNING about security.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
---
 docs/misc/arm/passthrough-noiommu.txt | 54 +++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100644 docs/misc/arm/passthrough-noiommu.txt

diff --git a/docs/misc/arm/passthrough-noiommu.txt b/docs/misc/arm/passthrough-noiommu.txt
new file mode 100644
index 0000000000..61aeb8a5cd
--- /dev/null
+++ b/docs/misc/arm/passthrough-noiommu.txt
@@ -0,0 +1,54 @@
+Request Device Assignment without IOMMU support
+===============================================
+
+*WARNING:
+Users should be aware that it is not always secure to assign a device without
+IOMMU protection.
+When the device is not protected by the IOMMU, the administrator should make
+sure that:
+ 1. The device is assigned to a trusted guest.
+ 2. Users have additional security mechanism on the platform.
+
+
+This document assumes that the IOMMU is absent from the system or it is
+disabled (status = "disabled" in device tree).
+
+
+Add xen,force-assign-without-iommu; to the device tree snippet:
+
+ethernet: ethernet@ff0e0000 {
+	compatible = "cdns,zynqmp-gem";
+	xen,path = "/amba/ethernet@ff0e0000";
+	xen,reg = <0x0 0xff0e0000 0x1000 0x0 0xff0e0000>;
+	xen,force-assign-without-iommu;
+};
+
+Request 1:1 memory mapping for the domain on static allocation
+==============================================================
+
+Add a direct-map property under the appropriate /chosen/domU node which
+is also statically allocated with physical memory ranges through
+xen,static-mem property as its guest RAM.
+
+Below is an example on how to specify the 1:1 memory mapping for the domain
+on static allocation in the device-tree:
+
+/ {
+	chosen {
+		domU1 {
+			compatible = "xen,domain";
+			#address-cells = <0x2>;
+			#size-cells = <0x2>;
+			cpus = <2>;
+			memory = <0x0 0x80000>;
+			#xen,static-mem-address-cells = <0x1>;
+			#xen,static-mem-size-cells = <0x1>;
+			xen,static-mem = <0x30000000 0x20000000>;
+			direct-map;
+			...
+		};
+	};
+};
+
+Besides reserving a 512MB region starting at the host physical address
+0x30000000 to DomU1, it also requests 1:1 memory mapping.
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:39:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:39:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193388.344447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFZl-0005eR-5W; Thu, 23 Sep 2021 03:39:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193388.344447; Thu, 23 Sep 2021 03:39:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFZl-0005eK-2X; Thu, 23 Sep 2021 03:39:09 +0000
Received: by outflank-mailman (input) for mailman id 193388;
 Thu, 23 Sep 2021 03:39:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/Dm3=ON=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mTFZj-0005eE-7O
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:39:07 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e6bf3fe9-917d-4077-8dee-43f625dac08f;
 Thu, 23 Sep 2021 03:39:05 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7D76661038;
 Thu, 23 Sep 2021 03:39:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6bf3fe9-917d-4077-8dee-43f625dac08f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632368345;
	bh=h2ALf3eQQBmDwmQHsBGzHUpUw0tJVsZIOA7VWESL6G4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=uxCRnGLtrpwI+P2QqmPQxzjVoJx7sjLA51zAdHw09QeDSDKBhdx5fXZoGJXs3na5a
	 KVGBYBfgCLOqhpk0LVB6YjT3jpA4PVZjJTtCRMCBTmkfNqp2qkSsHz8nedMUDWBdL+
	 jV2K9O9Fie0/jJgia1RPPpfBIs4tVDTBnvjB2RGQXbZtT5G4r9yEwkY50iE3pft1vx
	 6ZmMlfyHtnWIaRu4Q3C0sgLP6GlZugUqraUSlGAy4Ni6OQxVqqRuMyfxjw+5XgE1X2
	 MujFmPAL6jrHgmFeBZ4UlYjaCdTupy9ycBIFVGaeKpIEO/njIeDA8fT4KzUvVHGIn4
	 rKlVfQmiCajJg==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.4 07/19] xen/balloon: use a kernel thread instead a workqueue
Date: Wed, 22 Sep 2021 23:38:41 -0400
Message-Id: <20210923033853.1421193-7-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210923033853.1421193-1-sashal@kernel.org>
References: <20210923033853.1421193-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 8480ed9c2bbd56fc86524998e5f2e3e22f5038f6 ]

Today the Xen ballooning is done via delayed work in a workqueue. This
might result in workqueue hangups being reported in case of large
amounts of memory are being ballooned in one go (here 16GB):

BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 64s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    in-flight: 229:balloon_process
    pending: cache_reap
workqueue events_freezable_power_: flags=0x84
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: disk_events_workfn
workqueue mm_percpu_wq: flags=0x8
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: vmstat_update
pool 12: cpus=6 node=0 flags=0x0 nice=0 hung=64s workers=3 idle: 2222 43

This can easily be avoided by using a dedicated kernel thread for doing
the ballooning work.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/20210827123206.15429-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/xen/balloon.c | 62 +++++++++++++++++++++++++++++++------------
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index ebb05517b6aa..2762d246991b 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -43,6 +43,8 @@
 #include <linux/sched.h>
 #include <linux/cred.h>
 #include <linux/errno.h>
+#include <linux/freezer.h>
+#include <linux/kthread.h>
 #include <linux/mm.h>
 #include <linux/memblock.h>
 #include <linux/pagemap.h>
@@ -117,7 +119,7 @@ static struct ctl_table xen_root[] = {
 #define EXTENT_ORDER (fls(XEN_PFN_PER_PAGE) - 1)
 
 /*
- * balloon_process() state:
+ * balloon_thread() state:
  *
  * BP_DONE: done or nothing to do,
  * BP_WAIT: wait to be rescheduled,
@@ -132,6 +134,8 @@ enum bp_state {
 	BP_ECANCELED
 };
 
+/* Main waiting point for xen-balloon thread. */
+static DECLARE_WAIT_QUEUE_HEAD(balloon_thread_wq);
 
 static DEFINE_MUTEX(balloon_mutex);
 
@@ -146,10 +150,6 @@ static xen_pfn_t frame_list[PAGE_SIZE / sizeof(xen_pfn_t)];
 static LIST_HEAD(ballooned_pages);
 static DECLARE_WAIT_QUEUE_HEAD(balloon_wq);
 
-/* Main work function, always executed in process context. */
-static void balloon_process(struct work_struct *work);
-static DECLARE_DELAYED_WORK(balloon_worker, balloon_process);
-
 /* When ballooning out (allocating memory to return to Xen) we don't really
    want the kernel to try too hard since that can trigger the oom killer. */
 #define GFP_BALLOON \
@@ -383,7 +383,7 @@ static void xen_online_page(struct page *page, unsigned int order)
 static int xen_memory_notifier(struct notifier_block *nb, unsigned long val, void *v)
 {
 	if (val == MEM_ONLINE)
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	return NOTIFY_OK;
 }
@@ -508,18 +508,43 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 }
 
 /*
- * As this is a work item it is guaranteed to run as a single instance only.
+ * Stop waiting if either state is not BP_EAGAIN and ballooning action is
+ * needed, or if the credit has changed while state is BP_EAGAIN.
+ */
+static bool balloon_thread_cond(enum bp_state state, long credit)
+{
+	if (state != BP_EAGAIN)
+		credit = 0;
+
+	return current_credit() != credit || kthread_should_stop();
+}
+
+/*
+ * As this is a kthread it is guaranteed to run as a single instance only.
  * We may of course race updates of the target counts (which are protected
  * by the balloon lock), or with changes to the Xen hard limit, but we will
  * recover from these in time.
  */
-static void balloon_process(struct work_struct *work)
+static int balloon_thread(void *unused)
 {
 	enum bp_state state = BP_DONE;
 	long credit;
+	unsigned long timeout;
+
+	set_freezable();
+	for (;;) {
+		if (state == BP_EAGAIN)
+			timeout = balloon_stats.schedule_delay * HZ;
+		else
+			timeout = 3600 * HZ;
+		credit = current_credit();
 
+		wait_event_interruptible_timeout(balloon_thread_wq,
+				 balloon_thread_cond(state, credit), timeout);
+
+		if (kthread_should_stop())
+			return 0;
 
-	do {
 		mutex_lock(&balloon_mutex);
 
 		credit = current_credit();
@@ -546,12 +571,7 @@ static void balloon_process(struct work_struct *work)
 		mutex_unlock(&balloon_mutex);
 
 		cond_resched();
-
-	} while (credit && state == BP_DONE);
-
-	/* Schedule more work if there is some still to be done. */
-	if (state == BP_EAGAIN)
-		schedule_delayed_work(&balloon_worker, balloon_stats.schedule_delay * HZ);
+	}
 }
 
 /* Resets the Xen limit, sets new target, and kicks off processing. */
@@ -559,7 +579,7 @@ void balloon_set_new_target(unsigned long target)
 {
 	/* No need for lock. Not read-modify-write updates. */
 	balloon_stats.target_pages = target;
-	schedule_delayed_work(&balloon_worker, 0);
+	wake_up(&balloon_thread_wq);
 }
 EXPORT_SYMBOL_GPL(balloon_set_new_target);
 
@@ -664,7 +684,7 @@ void free_xenballooned_pages(int nr_pages, struct page **pages)
 
 	/* The balloon may be too large now. Shrink it if needed. */
 	if (current_credit())
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	mutex_unlock(&balloon_mutex);
 }
@@ -696,6 +716,8 @@ static void __init balloon_add_region(unsigned long start_pfn,
 
 static int __init balloon_init(void)
 {
+	struct task_struct *task;
+
 	if (!xen_domain())
 		return -ENODEV;
 
@@ -739,6 +761,12 @@ static int __init balloon_init(void)
 	}
 #endif
 
+	task = kthread_run(balloon_thread, NULL, "xen-balloon");
+	if (IS_ERR(task)) {
+		pr_err("xen-balloon thread could not be started, ballooning will not work!\n");
+		return PTR_ERR(task);
+	}
+
 	/* Init the xen-balloon driver. */
 	xen_balloon_init();
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:39:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:39:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193392.344459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFaH-0006At-E7; Thu, 23 Sep 2021 03:39:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193392.344459; Thu, 23 Sep 2021 03:39:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFaH-0006Ak-B2; Thu, 23 Sep 2021 03:39:41 +0000
Received: by outflank-mailman (input) for mailman id 193392;
 Thu, 23 Sep 2021 03:39:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/Dm3=ON=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mTFaG-00069S-4y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:39:40 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e1036082-1c1f-11ec-ba0e-12813bfff9fa;
 Thu, 23 Sep 2021 03:39:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8F26661248;
 Thu, 23 Sep 2021 03:39:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1036082-1c1f-11ec-ba0e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632368378;
	bh=AktRUjpwPPGgdtxBKplsDvSBKllnb1XsrQVvgw+muAU=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=FrsLDuKQXxkenK6EhC2eb+qygeVlP2dA+9TziYgcyERPlAxj3JgcQJJCbbY9rTUzo
	 CJrpsZ4SZU0Y9xSY1bh6LUhX6cvp+ruw7WISDSVrJMHHHIoMpR3PUaE01TMKjalvqz
	 NGsvwXvKxdlcIpPZRJeDaEgAj43tVfQOMXkUDjJaWB0VYwVIC68lVjB3qznDaLBVQH
	 ztmpEKMPUyFXrURss9NSYPi++VC5zoVmVX5RVihUIQxcky3ZtJ7+q2K3tv0q/0UYzl
	 k2fkrVoayVuvnqLKDGdBJ0yqcJLWXDhK8I9ulOUqd4pm8DZ+NfV10VAWQkG8Vnq4dY
	 IKKOIoGT580tQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 4.19 05/15] xen/balloon: use a kernel thread instead a workqueue
Date: Wed, 22 Sep 2021 23:39:19 -0400
Message-Id: <20210923033929.1421446-5-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210923033929.1421446-1-sashal@kernel.org>
References: <20210923033929.1421446-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 8480ed9c2bbd56fc86524998e5f2e3e22f5038f6 ]

Today the Xen ballooning is done via delayed work in a workqueue. This
might result in workqueue hangups being reported in case of large
amounts of memory are being ballooned in one go (here 16GB):

BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 64s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    in-flight: 229:balloon_process
    pending: cache_reap
workqueue events_freezable_power_: flags=0x84
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: disk_events_workfn
workqueue mm_percpu_wq: flags=0x8
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: vmstat_update
pool 12: cpus=6 node=0 flags=0x0 nice=0 hung=64s workers=3 idle: 2222 43

This can easily be avoided by using a dedicated kernel thread for doing
the ballooning work.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/20210827123206.15429-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/xen/balloon.c | 62 +++++++++++++++++++++++++++++++------------
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index b23edf64c2b2..643dbe5620e8 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -43,6 +43,8 @@
 #include <linux/sched.h>
 #include <linux/cred.h>
 #include <linux/errno.h>
+#include <linux/freezer.h>
+#include <linux/kthread.h>
 #include <linux/mm.h>
 #include <linux/bootmem.h>
 #include <linux/pagemap.h>
@@ -120,7 +122,7 @@ static struct ctl_table xen_root[] = {
 #define EXTENT_ORDER (fls(XEN_PFN_PER_PAGE) - 1)
 
 /*
- * balloon_process() state:
+ * balloon_thread() state:
  *
  * BP_DONE: done or nothing to do,
  * BP_WAIT: wait to be rescheduled,
@@ -135,6 +137,8 @@ enum bp_state {
 	BP_ECANCELED
 };
 
+/* Main waiting point for xen-balloon thread. */
+static DECLARE_WAIT_QUEUE_HEAD(balloon_thread_wq);
 
 static DEFINE_MUTEX(balloon_mutex);
 
@@ -149,10 +153,6 @@ static xen_pfn_t frame_list[PAGE_SIZE / sizeof(xen_pfn_t)];
 static LIST_HEAD(ballooned_pages);
 static DECLARE_WAIT_QUEUE_HEAD(balloon_wq);
 
-/* Main work function, always executed in process context. */
-static void balloon_process(struct work_struct *work);
-static DECLARE_DELAYED_WORK(balloon_worker, balloon_process);
-
 /* When ballooning out (allocating memory to return to Xen) we don't really
    want the kernel to try too hard since that can trigger the oom killer. */
 #define GFP_BALLOON \
@@ -383,7 +383,7 @@ static void xen_online_page(struct page *page)
 static int xen_memory_notifier(struct notifier_block *nb, unsigned long val, void *v)
 {
 	if (val == MEM_ONLINE)
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	return NOTIFY_OK;
 }
@@ -508,18 +508,43 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 }
 
 /*
- * As this is a work item it is guaranteed to run as a single instance only.
+ * Stop waiting if either state is not BP_EAGAIN and ballooning action is
+ * needed, or if the credit has changed while state is BP_EAGAIN.
+ */
+static bool balloon_thread_cond(enum bp_state state, long credit)
+{
+	if (state != BP_EAGAIN)
+		credit = 0;
+
+	return current_credit() != credit || kthread_should_stop();
+}
+
+/*
+ * As this is a kthread it is guaranteed to run as a single instance only.
  * We may of course race updates of the target counts (which are protected
  * by the balloon lock), or with changes to the Xen hard limit, but we will
  * recover from these in time.
  */
-static void balloon_process(struct work_struct *work)
+static int balloon_thread(void *unused)
 {
 	enum bp_state state = BP_DONE;
 	long credit;
+	unsigned long timeout;
+
+	set_freezable();
+	for (;;) {
+		if (state == BP_EAGAIN)
+			timeout = balloon_stats.schedule_delay * HZ;
+		else
+			timeout = 3600 * HZ;
+		credit = current_credit();
 
+		wait_event_interruptible_timeout(balloon_thread_wq,
+				 balloon_thread_cond(state, credit), timeout);
+
+		if (kthread_should_stop())
+			return 0;
 
-	do {
 		mutex_lock(&balloon_mutex);
 
 		credit = current_credit();
@@ -546,12 +571,7 @@ static void balloon_process(struct work_struct *work)
 		mutex_unlock(&balloon_mutex);
 
 		cond_resched();
-
-	} while (credit && state == BP_DONE);
-
-	/* Schedule more work if there is some still to be done. */
-	if (state == BP_EAGAIN)
-		schedule_delayed_work(&balloon_worker, balloon_stats.schedule_delay * HZ);
+	}
 }
 
 /* Resets the Xen limit, sets new target, and kicks off processing. */
@@ -559,7 +579,7 @@ void balloon_set_new_target(unsigned long target)
 {
 	/* No need for lock. Not read-modify-write updates. */
 	balloon_stats.target_pages = target;
-	schedule_delayed_work(&balloon_worker, 0);
+	wake_up(&balloon_thread_wq);
 }
 EXPORT_SYMBOL_GPL(balloon_set_new_target);
 
@@ -664,7 +684,7 @@ void free_xenballooned_pages(int nr_pages, struct page **pages)
 
 	/* The balloon may be too large now. Shrink it if needed. */
 	if (current_credit())
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	mutex_unlock(&balloon_mutex);
 }
@@ -698,6 +718,8 @@ static void __init balloon_add_region(unsigned long start_pfn,
 
 static int __init balloon_init(void)
 {
+	struct task_struct *task;
+
 	if (!xen_domain())
 		return -ENODEV;
 
@@ -741,6 +763,12 @@ static int __init balloon_init(void)
 	}
 #endif
 
+	task = kthread_run(balloon_thread, NULL, "xen-balloon");
+	if (IS_ERR(task)) {
+		pr_err("xen-balloon thread could not be started, ballooning will not work!\n");
+		return PTR_ERR(task);
+	}
+
 	/* Init the xen-balloon driver. */
 	xen_balloon_init();
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:40:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:40:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193398.344469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFan-0007Uj-Nt; Thu, 23 Sep 2021 03:40:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193398.344469; Thu, 23 Sep 2021 03:40:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFan-0007Uc-KS; Thu, 23 Sep 2021 03:40:13 +0000
Received: by outflank-mailman (input) for mailman id 193398;
 Thu, 23 Sep 2021 03:40:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=/Dm3=ON=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mTFam-0006s7-Mu
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 03:40:12 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b84f1ecf-ac66-426b-8b18-a11a841ab308;
 Thu, 23 Sep 2021 03:40:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0E5026124D;
 Thu, 23 Sep 2021 03:40:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b84f1ecf-ac66-426b-8b18-a11a841ab308
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632368406;
	bh=htC7IBlUxwUYSSB+pZhwJu+KsbGIN7ATgsvr5svQKEw=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=YaZOeAwGs9WIUzsCAxHUSP54VFWvZO0xcOD+zD480mZ9hDTBHJviML7fW29m8iuYD
	 Z2FCzju95yVL6GwSDAYbX2GsJR8YLG/j0zutSC67VYiPIDaX3jbDGw9H8fqeoNrYn2
	 z62Ltcj68XyVBdmdRdtnSI+1C2JNGlsls4Q8jKROrC3NjSntWCOAwUbt4xwlHCCL1p
	 tv0cN9FkyBPFGt4DhxhIdgQ5qLdRhmSehdcH1/B4Lz+7fx7bcpawSwy/lUUNhLQfQB
	 lCVHpyNTj3p9sDAVf2j+6E+VCUWB6KZuWcPTNTeNZmAnMKzOr7dFsDzoj9Ud+CCaEF
	 Wic11KYIl2Smg==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Sasha Levin <sashal@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 4.14 04/13] xen/balloon: use a kernel thread instead a workqueue
Date: Wed, 22 Sep 2021 23:39:50 -0400
Message-Id: <20210923033959.1421662-4-sashal@kernel.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210923033959.1421662-1-sashal@kernel.org>
References: <20210923033959.1421662-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Juergen Gross <jgross@suse.com>

[ Upstream commit 8480ed9c2bbd56fc86524998e5f2e3e22f5038f6 ]

Today the Xen ballooning is done via delayed work in a workqueue. This
might result in workqueue hangups being reported in case of large
amounts of memory are being ballooned in one go (here 16GB):

BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 64s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    in-flight: 229:balloon_process
    pending: cache_reap
workqueue events_freezable_power_: flags=0x84
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: disk_events_workfn
workqueue mm_percpu_wq: flags=0x8
  pwq 12: cpus=6 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: vmstat_update
pool 12: cpus=6 node=0 flags=0x0 nice=0 hung=64s workers=3 idle: 2222 43

This can easily be avoided by using a dedicated kernel thread for doing
the ballooning work.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/20210827123206.15429-1-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/xen/balloon.c | 62 +++++++++++++++++++++++++++++++------------
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index a8e0836dffd4..b9abe3ef2188 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -43,6 +43,8 @@
 #include <linux/sched.h>
 #include <linux/cred.h>
 #include <linux/errno.h>
+#include <linux/freezer.h>
+#include <linux/kthread.h>
 #include <linux/mm.h>
 #include <linux/bootmem.h>
 #include <linux/pagemap.h>
@@ -119,7 +121,7 @@ static struct ctl_table xen_root[] = {
 #define EXTENT_ORDER (fls(XEN_PFN_PER_PAGE) - 1)
 
 /*
- * balloon_process() state:
+ * balloon_thread() state:
  *
  * BP_DONE: done or nothing to do,
  * BP_WAIT: wait to be rescheduled,
@@ -134,6 +136,8 @@ enum bp_state {
 	BP_ECANCELED
 };
 
+/* Main waiting point for xen-balloon thread. */
+static DECLARE_WAIT_QUEUE_HEAD(balloon_thread_wq);
 
 static DEFINE_MUTEX(balloon_mutex);
 
@@ -148,10 +152,6 @@ static xen_pfn_t frame_list[PAGE_SIZE / sizeof(xen_pfn_t)];
 static LIST_HEAD(ballooned_pages);
 static DECLARE_WAIT_QUEUE_HEAD(balloon_wq);
 
-/* Main work function, always executed in process context. */
-static void balloon_process(struct work_struct *work);
-static DECLARE_DELAYED_WORK(balloon_worker, balloon_process);
-
 /* When ballooning out (allocating memory to return to Xen) we don't really
    want the kernel to try too hard since that can trigger the oom killer. */
 #define GFP_BALLOON \
@@ -389,7 +389,7 @@ static void xen_online_page(struct page *page)
 static int xen_memory_notifier(struct notifier_block *nb, unsigned long val, void *v)
 {
 	if (val == MEM_ONLINE)
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	return NOTIFY_OK;
 }
@@ -571,18 +571,43 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 }
 
 /*
- * As this is a work item it is guaranteed to run as a single instance only.
+ * Stop waiting if either state is not BP_EAGAIN and ballooning action is
+ * needed, or if the credit has changed while state is BP_EAGAIN.
+ */
+static bool balloon_thread_cond(enum bp_state state, long credit)
+{
+	if (state != BP_EAGAIN)
+		credit = 0;
+
+	return current_credit() != credit || kthread_should_stop();
+}
+
+/*
+ * As this is a kthread it is guaranteed to run as a single instance only.
  * We may of course race updates of the target counts (which are protected
  * by the balloon lock), or with changes to the Xen hard limit, but we will
  * recover from these in time.
  */
-static void balloon_process(struct work_struct *work)
+static int balloon_thread(void *unused)
 {
 	enum bp_state state = BP_DONE;
 	long credit;
+	unsigned long timeout;
+
+	set_freezable();
+	for (;;) {
+		if (state == BP_EAGAIN)
+			timeout = balloon_stats.schedule_delay * HZ;
+		else
+			timeout = 3600 * HZ;
+		credit = current_credit();
 
+		wait_event_interruptible_timeout(balloon_thread_wq,
+				 balloon_thread_cond(state, credit), timeout);
+
+		if (kthread_should_stop())
+			return 0;
 
-	do {
 		mutex_lock(&balloon_mutex);
 
 		credit = current_credit();
@@ -609,12 +634,7 @@ static void balloon_process(struct work_struct *work)
 		mutex_unlock(&balloon_mutex);
 
 		cond_resched();
-
-	} while (credit && state == BP_DONE);
-
-	/* Schedule more work if there is some still to be done. */
-	if (state == BP_EAGAIN)
-		schedule_delayed_work(&balloon_worker, balloon_stats.schedule_delay * HZ);
+	}
 }
 
 /* Resets the Xen limit, sets new target, and kicks off processing. */
@@ -622,7 +642,7 @@ void balloon_set_new_target(unsigned long target)
 {
 	/* No need for lock. Not read-modify-write updates. */
 	balloon_stats.target_pages = target;
-	schedule_delayed_work(&balloon_worker, 0);
+	wake_up(&balloon_thread_wq);
 }
 EXPORT_SYMBOL_GPL(balloon_set_new_target);
 
@@ -727,7 +747,7 @@ void free_xenballooned_pages(int nr_pages, struct page **pages)
 
 	/* The balloon may be too large now. Shrink it if needed. */
 	if (current_credit())
-		schedule_delayed_work(&balloon_worker, 0);
+		wake_up(&balloon_thread_wq);
 
 	mutex_unlock(&balloon_mutex);
 }
@@ -761,6 +781,8 @@ static void __init balloon_add_region(unsigned long start_pfn,
 
 static int __init balloon_init(void)
 {
+	struct task_struct *task;
+
 	if (!xen_domain())
 		return -ENODEV;
 
@@ -804,6 +826,12 @@ static int __init balloon_init(void)
 	}
 #endif
 
+	task = kthread_run(balloon_thread, NULL, "xen-balloon");
+	if (IS_ERR(task)) {
+		pr_err("xen-balloon thread could not be started, ballooning will not work!\n");
+		return PTR_ERR(task);
+	}
+
 	/* Init the xen-balloon driver. */
 	xen_balloon_init();
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 03:51:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 03:51:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193403.344480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFlS-0000oF-Pe; Thu, 23 Sep 2021 03:51:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193403.344480; Thu, 23 Sep 2021 03:51:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTFlS-0000o8-MJ; Thu, 23 Sep 2021 03:51:14 +0000
Received: by outflank-mailman (input) for mailman id 193403;
 Thu, 23 Sep 2021 03:51:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTFlR-0000nu-IZ; Thu, 23 Sep 2021 03:51:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTFlR-0005xM-Af; Thu, 23 Sep 2021 03:51:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTFlQ-0002j7-W8; Thu, 23 Sep 2021 03:51:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTFlQ-0007Y8-Vc; Thu, 23 Sep 2021 03:51:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5ziqsY4f5K5/xWMWkJ/Htnk1NvEWqvLzewiAVkeJwcI=; b=yGH9ZMMAkGAMGfDJ22y2IIUKNG
	VYtU7KwEkEgXRlolAU1VDrGDuiHm0RTjwJ7uhz5PgZZL0eP1n6OoohEo1KffMkCt68P4FiZJD/TEf
	tEN3AkGS9MdZnK3W3Zz+kfdGKozAciF569WbRsBjRTsaghzmvYpkjoEPm9jDCF7NK+KI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165156-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165156: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-libvirt-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e467d390eaf6736377766adae0166711866fac3
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 03:51:12 +0000

flight 165156 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165156/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine    4 memdisk-try-append fail in 165147 pass in 165156
 test-amd64-amd64-libvirt-vhd 19 guest-start/debian.repeat  fail pass in 165147

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e467d390eaf6736377766adae0166711866fac3
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   12 days
Failing since        164951  2021-09-12 00:14:36 Z   11 days   22 attempts
Testing same since   165147  2021-09-22 03:25:57 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 fail    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 790 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 04:45:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 04:45:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193411.344495 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTGbQ-0006tE-05; Thu, 23 Sep 2021 04:44:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193411.344495; Thu, 23 Sep 2021 04:44:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTGbP-0006t7-T0; Thu, 23 Sep 2021 04:44:55 +0000
Received: by outflank-mailman (input) for mailman id 193411;
 Thu, 23 Sep 2021 04:44:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTGbO-0006t1-2d
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 04:44:54 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fda70712-1c28-11ec-ba13-12813bfff9fa;
 Thu, 23 Sep 2021 04:44:52 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C5C5822304;
 Thu, 23 Sep 2021 04:44:51 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7D2B113DBB;
 Thu, 23 Sep 2021 04:44:51 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id TWNPHUMGTGHlNwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 04:44:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fda70712-1c28-11ec-ba13-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632372291; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=6DS1OShxkN081ActC9VohxvjjWhXQsUmGfkvB+j0Sq4=;
	b=NxKyAbJyMgEPCr4dSNHUkVYCcpBq8h4ByundRC9cvA4lsE4KPrue+SiaBi7yzhPxl8HhrZ
	jITXbjboLxn7r1+fA/d5DKm+Zm0Xfc1CWQa4nDujoUZwGkWwpVQayUeMYhV+ZlPw+OjWsd
	TrSZkpkSTvUiO0G1dTM319PHna12LRo=
Subject: Re: [PATCH v2 1/2] x86/xen: remove xen_have_vcpu_info_placement flag
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-2-jgross@suse.com>
 <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <7de79902-e31b-899d-44bb-f9daabb2ecf0@suse.com>
Date: Thu, 23 Sep 2021 06:44:50 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="7p42MbmcR2R1URc5b0nLq1zIkTD8u7F9m"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--7p42MbmcR2R1URc5b0nLq1zIkTD8u7F9m
Content-Type: multipart/mixed; boundary="IlXSuJXAwFX31QSGBlKtSGJwPbIgwtFQl";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Stefano Stabellini <sstabellini@kernel.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
Message-ID: <7de79902-e31b-899d-44bb-f9daabb2ecf0@suse.com>
Subject: Re: [PATCH v2 1/2] x86/xen: remove xen_have_vcpu_info_placement flag
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-2-jgross@suse.com>
 <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>
In-Reply-To: <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>

--IlXSuJXAwFX31QSGBlKtSGJwPbIgwtFQl
Content-Type: multipart/mixed;
 boundary="------------2D82B1FDA2C0BF04A0178389"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2D82B1FDA2C0BF04A0178389
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 22.09.21 23:43, Boris Ostrovsky wrote:
>=20
> On 9/22/21 6:31 AM, Juergen Gross wrote:
>>  =20
>> -	if (xen_have_vcpu_info_placement) {
>> -		vcpup =3D &per_cpu(xen_vcpu_info, cpu);
>> -		info.mfn =3D arbitrary_virt_to_mfn(vcpup);
>> -		info.offset =3D offset_in_page(vcpup);
>> +	vcpup =3D &per_cpu(xen_vcpu_info, cpu);
>> +	info.mfn =3D arbitrary_virt_to_mfn(vcpup);
>> +	info.offset =3D offset_in_page(vcpup);
>>  =20
>> -		/*
>> -		 * Check to see if the hypervisor will put the vcpu_info
>> -		 * structure where we want it, which allows direct access via
>> -		 * a percpu-variable.
>> -		 * N.B. This hypercall can _only_ be called once per CPU.
>> -		 * Subsequent calls will error out with -EINVAL. This is due to
>> -		 * the fact that hypervisor has no unregister variant and this
>> -		 * hypercall does not allow to over-write info.mfn and
>> -		 * info.offset.
>> -		 */
>> -		err =3D HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info,
>> -					 xen_vcpu_nr(cpu), &info);
>> -
>> -		if (err) {
>> -			pr_warn_once("register_vcpu_info failed: cpu=3D%d err=3D%d\n",
>> -				     cpu, err);
>> -			xen_have_vcpu_info_placement =3D 0;
>> -		} else {
>> -			/*
>> -			 * This cpu is using the registered vcpu info, even if
>> -			 * later ones fail to.
>> -			 */
>> -			per_cpu(xen_vcpu, cpu) =3D vcpup;
>> -		}
>> -	}
>> -
>> -	if (!xen_have_vcpu_info_placement)
>> -		xen_vcpu_info_reset(cpu);
>> +	/*
>> +	 * N.B. This hypercall can _only_ be called once per CPU.
>> +	 * Subsequent calls will error out with -EINVAL. This is due to
>> +	 * the fact that hypervisor has no unregister variant and this
>> +	 * hypercall does not allow to over-write info.mfn and
>> +	 * info.offset.
>> +	 */
>> +	err =3D HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, xen_vcpu_nr(cp=
u),
>> +				 &info);
>> +	if (err)
>> +		panic("register_vcpu_info failed: cpu=3D%d err=3D%d\n", cpu, err);
>>  =20
>=20
>=20
> This is change in behavior. Before if the hypercall failed we still try=
 to boot. I am not sure we need to worry about this (since it's not clear=
 it actually works)=C2=A0 but I'd at least mention this in the commit mes=
sage.

Hmm, maybe I should have been more explicit saying that the hypercall
was introduced in Xen 3.4, and only reason of failure is either an
illegal vcpu, an invalid mapping specification, or a try to reissue the
hypercall for a vcpu. None of those should ever happen.


Juergen

--------------2D82B1FDA2C0BF04A0178389
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2D82B1FDA2C0BF04A0178389--

--IlXSuJXAwFX31QSGBlKtSGJwPbIgwtFQl--

--7p42MbmcR2R1URc5b0nLq1zIkTD8u7F9m
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMBkIFAwAAAAAACgkQsN6d1ii/Ey/C
QAf+MW8NU9ZWXXfs6ZXLLjQIV2BF+hUkXZKRQdnZdeNvIr2UQrI40xq5SihwSro9Tnr9vLldkD/y
2Krat7LKnzMF/MfK5XXm2v9iuTK3Rc/1AYCnovaiDbRJCd+h+3cWWccEyhNEtsjV7m70m9+E9oYV
zx0iarlETkSzZfilELCEFvgn037PJveprAhaODuxL8h7JV0zFwkGyUOowQz5TiY/iQ3Nmzv2GzF8
3WS/lzTV8NhJgMqwSVecAUBJkdRG/hCzxBuoTuPuaSp8YBSWwfgCtfWL3Cz1f3aLsBtGH4wi18L5
Wifa075NKRESCVJDxHEM2tbFJVfYiXXe4Wgxg9LwFw==
=c79E
-----END PGP SIGNATURE-----

--7p42MbmcR2R1URc5b0nLq1zIkTD8u7F9m--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 04:45:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 04:45:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193415.344506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTGcO-0007Ro-AS; Thu, 23 Sep 2021 04:45:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193415.344506; Thu, 23 Sep 2021 04:45:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTGcO-0007Rh-6V; Thu, 23 Sep 2021 04:45:56 +0000
Received: by outflank-mailman (input) for mailman id 193415;
 Thu, 23 Sep 2021 04:45:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTGcN-0007RV-Dg
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 04:45:55 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 22852d3e-1c29-11ec-ba14-12813bfff9fa;
 Thu, 23 Sep 2021 04:45:54 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C69CA1FD6F;
 Thu, 23 Sep 2021 04:45:53 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 781FA13DBB;
 Thu, 23 Sep 2021 04:45:53 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id o24jHIEGTGFwOAAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 04:45:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22852d3e-1c29-11ec-ba14-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632372353; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=HmN9x6rySyJ9lUi9QYfe8S0ZHeO+Ta1jHzRHG0MMrbg=;
	b=dOZsXn5pKF/mnbvYLocKNOzoAe6pDthx4PJwpOd2WGWLD3xvRgdrcTKMQjzNFWqFq2K0vc
	Uz8xurjQ6hRNnB3wdQKaR66e9/3yk6u40ZQj7Lkr7H0IcimRIZTC4599/ByHgVjmyRmMTB
	zKr3RQDF91/CHewQuLbz14b5s4NSf3U=
Subject: Re: [PATCH v2 2/2] x86/xen: switch initial pvops IRQ functions to
 dummy ones
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org,
 virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Deep Shah <sdeep@vmware.com>,
 "VMware, Inc." <pv-drivers@vmware.com>, Thomas Gleixner
 <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-3-jgross@suse.com>
 <429cdc9b-b9f1-5393-8baf-cdd2046cd65a@oracle.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <aa634c0c-d949-de8f-c1a3-668b83271c14@suse.com>
Date: Thu, 23 Sep 2021 06:45:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <429cdc9b-b9f1-5393-8baf-cdd2046cd65a@oracle.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="6k9FXmeDukZ9ZMu6ldsw95ygj9YXpZQo4"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--6k9FXmeDukZ9ZMu6ldsw95ygj9YXpZQo4
Content-Type: multipart/mixed; boundary="7roZvm7HEwotfUhEtHFwEXWgMEbTw8Sr9";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 xen-devel@lists.xenproject.org, x86@kernel.org,
 virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Deep Shah <sdeep@vmware.com>,
 "VMware, Inc." <pv-drivers@vmware.com>, Thomas Gleixner
 <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Message-ID: <aa634c0c-d949-de8f-c1a3-668b83271c14@suse.com>
Subject: Re: [PATCH v2 2/2] x86/xen: switch initial pvops IRQ functions to
 dummy ones
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-3-jgross@suse.com>
 <429cdc9b-b9f1-5393-8baf-cdd2046cd65a@oracle.com>
In-Reply-To: <429cdc9b-b9f1-5393-8baf-cdd2046cd65a@oracle.com>

--7roZvm7HEwotfUhEtHFwEXWgMEbTw8Sr9
Content-Type: multipart/mixed;
 boundary="------------E8E80534100E141BF91E57C6"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------E8E80534100E141BF91E57C6
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 22.09.21 23:49, Boris Ostrovsky wrote:
>=20
> On 9/22/21 6:31 AM, Juergen Gross wrote:
>> The initial pvops functions handling irq flags will only ever be calle=
d
>> before interrupts are being enabled.
>>
>> So make the __init and switch them to be dummy functions:
>=20
>=20
> What are you making __init?

Oh, sorry, that's a leftover from a previous version.

>> +/* stub always return ing 0. */
>=20
>=20
> "always returning"

Yes.


Juergen

--------------E8E80534100E141BF91E57C6
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------E8E80534100E141BF91E57C6--

--7roZvm7HEwotfUhEtHFwEXWgMEbTw8Sr9--

--6k9FXmeDukZ9ZMu6ldsw95ygj9YXpZQo4
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMBoAFAwAAAAAACgkQsN6d1ii/Ey92
gQf+LEMpK732kRBHeHq+3Y9FgdNcrAmxTA0hegwAC3XGOd3xp8EtUpI7z9SpsJ3vREMnOebdOaRH
S+S+7L3yt72EKy8XI1qteFS322jgz2SJ+z3U5DrH0xD4XRjNt+6BhiMmno6904+oDIG6QOQkFs+T
WG8MfT5baQopElSTmzAFsrI1/vHKQdutvpaoOKv+Pq+v77zjkbBQsV7g7c+T8dVC7ueCeFHZOrPN
gifwHMaWx+r8vJg2WqOszNOkclM7iTgApwl37doNJuztDUoYj06yIPYrJxcLentm7NdsPiUU1OlT
y26L/cXNn/4eGp9NkyL6SJkktNx6EseE7PUnpaoGVw==
=5cLA
-----END PGP SIGNATURE-----

--6k9FXmeDukZ9ZMu6ldsw95ygj9YXpZQo4--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 06:15:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 06:15:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193423.344517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTI0e-0001RX-Ui; Thu, 23 Sep 2021 06:15:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193423.344517; Thu, 23 Sep 2021 06:15:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTI0e-0001RQ-Re; Thu, 23 Sep 2021 06:15:04 +0000
Received: by outflank-mailman (input) for mailman id 193423;
 Thu, 23 Sep 2021 06:15:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=o0JA=ON=arm.com=Hongda.Deng@srs-us1.protection.inumbo.net>)
 id 1mTI0d-0001RK-Ql
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 06:15:03 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c8e0e5c9-4c03-4101-b638-2cce0b8f3316;
 Thu, 23 Sep 2021 06:15:00 +0000 (UTC)
Received: from DB6PR07CA0178.eurprd07.prod.outlook.com (2603:10a6:6:43::32) by
 DB9PR08MB6779.eurprd08.prod.outlook.com (2603:10a6:10:2a1::21) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15; Thu, 23 Sep 2021 06:14:58 +0000
Received: from DB5EUR03FT027.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:43:cafe::c0) by DB6PR07CA0178.outlook.office365.com
 (2603:10a6:6:43::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.6 via Frontend
 Transport; Thu, 23 Sep 2021 06:14:58 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT027.mail.protection.outlook.com (10.152.20.121) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 06:14:57 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 06:14:57 +0000
Received: from c4187ebfdf87.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D077FA00-2652-4B72-963E-E7F2DABD831B.1; 
 Thu, 23 Sep 2021 06:14:48 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c4187ebfdf87.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 06:14:48 +0000
Received: from DB8PR04CA0027.eurprd04.prod.outlook.com (2603:10a6:10:110::37)
 by AM9PR08MB7055.eurprd08.prod.outlook.com (2603:10a6:20b:412::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 06:14:41 +0000
Received: from DB5EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:110:cafe::87) by DB8PR04CA0027.outlook.office365.com
 (2603:10a6:10:110::37) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 06:14:41 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT004.mail.protection.outlook.com (10.152.20.128) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 06:14:41 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 06:14:41 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 06:14:39 +0000
Received: from hongda-OptiPlex-7070.shanghai.arm.com (10.169.190.17) by
 mail.arm.com (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via
 Frontend Transport; Thu, 23 Sep 2021 06:14:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8e0e5c9-4c03-4101-b638-2cce0b8f3316
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=69C0iju/EYJemdr7CnsUN2JOzLyOYq1kPVmEocvSqf4=;
 b=fAl/4w8cmTXdae+mm+G8fLCeNRCZTpzqk9vSvNZ9G5Jntiqyihh6KDXg+OnOi+3OTJUqUj9qv4w3NQmKmuFTh/0MRwVEkl+2/PG8kTlKY9Dif4SGOFyqYiuBcxV2EMz1cuSLMAr1sdScwDmzoXKatRyVvfzIo/N+EJI+2Gxaf78=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 71420fc32d36208e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nvMgSzTQtFVBjRJX3dxRW9Rops7il4akQL9b2rmXwE/EY5iwt9aQxl049agu6YKGF3PRUVXX2Mj9MjnvtvvJzippOzNBx/K6kz7OuOXa6sVRERVEPN4N14M2tMDgcBIqRAhhk3gdlbMDA1Cp5gfBSv7m8uHs03KYTJkrI6afJG819hbyjb5ZMMKOlbhPI4chYOMGC5SdbwipAwjSyvjHk6UIfR5jpAdis6SV5jEb1fPT6ihOcvwbrOjVHbTg0iXTLfU/RDxQ8cB+kXVpVaKDW6PaYUf74FQNmlI5bxWrnW21soU/rAwEA2XaUSwhJbv5xR8bog8t4lbfm1rMiklodw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=69C0iju/EYJemdr7CnsUN2JOzLyOYq1kPVmEocvSqf4=;
 b=h3s51cdKU517z5z9bJ5LmFGavmVK7280zqJeWrvT7Lt5GexDg6x9rLA8Q0TvfkJWzUaSv+XXDRTpcCoSYAtVHV54hLSwo5R6CB/q6nAMC6wCZa8tnqB7d1WRs010q2ZHXQY5kk2RsKsp/jjjn+zun2pKedBw0IZtFzrCprbvc5VTWoTAG13y2yGpw9RQ9UuNOmqI/+YUQxAfONul33I0dPDCvTlObAa035oSgZv5iHRFBQxllYflb8N0AfYoieAO4oiuD/MxcAQxlBwRZUPljF3nWWF9IvkggPMfHRmajyHLQkagWn0MbblZMW4u/BjFUBhnBnBrKvoEmnoI9tSV8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=69C0iju/EYJemdr7CnsUN2JOzLyOYq1kPVmEocvSqf4=;
 b=fAl/4w8cmTXdae+mm+G8fLCeNRCZTpzqk9vSvNZ9G5Jntiqyihh6KDXg+OnOi+3OTJUqUj9qv4w3NQmKmuFTh/0MRwVEkl+2/PG8kTlKY9Dif4SGOFyqYiuBcxV2EMz1cuSLMAr1sdScwDmzoXKatRyVvfzIo/N+EJI+2Gxaf78=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Hongda Deng <Hongda.Deng@arm.com>
To: <xen-devel@lists.xenproject.org>, <sstabellini@kernel.org>,
	<julien@xen.org>
CC: <Bertrand.Marquis@arm.com>, <Wei.Chen@arm.com>, <Hongda.Deng@arm.com>
Subject: [PATCH] xen/arm: vgic to ignore GICD ICPENRn registers access
Date: Thu, 23 Sep 2021 14:14:29 +0800
Message-ID: <20210923061429.16361-1-Hongda.Deng@arm.com>
X-Mailer: git-send-email 2.17.1
MIME-Version: 1.0
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1e8a6041-2daf-4c57-6697-08d97e5977ec
X-MS-TrafficTypeDiagnostic: AM9PR08MB7055:|DB9PR08MB6779:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB67796C2F268EAB24CBB0CB0CE6A39@DB9PR08MB6779.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2399;OLM:2399;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 uGbBTgWTJ5jCTAwrisik+Ucc414aU65veeZpe/pJkVfUiSt7KB8ZXic/n5wodRqXov8DjslzEbF1ojCOtJrrHj4jvoaYEoD+RwxrkzBawx3LCGGGm6XakrVSkSH2UdNfxZWabFO59zDSQaYI+1k/yZqmaO+ph8uySMpXONXMt7B32azyuqT4MCJNK9L/oR+47V5bm91bxh9PqHfqmsBVSViDT0tzzatGkCZcqHodmpxHG8UFIaiwOJOewLGrSsTXPBTlIaTYNbCOXNyp69MY7bEcY9eGJWvW3QqmLUSdOeiB36W8m+lib0ECqq7iaiueB55ewr9s1XP20MhtETGj39EFOWJVDX1qJgOf33peUIhp6VTC/YtBKwk820sxerKSFNxgy/NlFg8mS5g8F+Iz72pe23rB6MqJ7ywj2Pqboxsg1uVb2iWWViRmEzcRY+CM2qMRtoXhoAQ7nZCaDpqdpxdW9W01xF3iR5L52UtkY8CoEt99HDQFk6cpaJPsWBlZhB92Avh2sSaOtbSupKbVoS2BM4YeCqqSqPocWTJeaKPW7NelDP0zyop9YizUzD3a7vLOZPUeeLLCQj+dXRmEeCQMphsKQFJdcGHB0wFKIkvlftZlaMEyQb5CUIdPg8oz40l6PGqbxqkMJDQjSVDhhBK5Jkho7YriQ2f5CvpjZwpka89QbFf6JKI6ziMplThtkOo33GvtpJLLWrICO8g2KESZGn78tzIdbFVI08yHUFCtJ2uNopd6/cOE+ZSDmV5e0W5ak5Qvozk1VsLr9gLDeaKTlh6RAjakleidKJ9N6iJ3P+K2INpgJJZIOFvwY4uj
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2906002)(508600001)(5660300002)(36756003)(8936002)(81166007)(356005)(2616005)(36860700001)(1076003)(8676002)(26005)(82310400003)(47076005)(7696005)(966005)(316002)(54906003)(83380400001)(336012)(70206006)(4326008)(186003)(110136005)(6666004)(86362001)(426003)(70586007)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB7055
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT027.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d5d0a739-f3be-485d-850a-08d97e596de2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	anpc1yN7ObQ/JitSac+5MVS2P/DhfUHDVCLPuLNvgBw1HAyBUYvwZOvqui7PsVy7N/HP0wwfVgALSfbh9Jxp9zcSZLYNFSPLDSuvrlBDlNj8gwe/FgEapSRDQuLeZxotoBQMqofXjqG+5GmmXQwzF6W3HO6UKgjyc+o4QZ8otPFc9BX/NJ38/E3rvHIYpcjX2XjxYvUyFYQP1sr/E/sS+L4VgUzc0dfMnZeFS0ie5AbHxmWAbLoG0VZ6tuTZXBwRHmdRJLr7HZfFRCHwS8wsnnq2CTT3aY1/WqG00NAQ3hKwyKDF5VHKija2+f6uuriaMS1K8dN7+xv5vcqECxTxcJli0w9IsWbXJcZlZT1qEEsbXhFgqRXqFx3wwMsULol+Q3VIpBXA70t35LBRO2pMaJ5sJKXKL/j+mb5qbeUGLaGnQ9sdF4vDxoJl6YmDV2UGmqxQxLBRbkj2M1kFqW0zlP2AGVxSc9ekhL/B0vEbR97k2GMfAd0x/4Hh5gyV6SNAx3Q1WwjF2uDkA+335RqPP9vN7kPJrMJDNtz6Iq/CyFI7iBtYJOBz7wPDAlH618jzhDtGtHd0SxASmwPEMk2aARSASb5Yfc/w9lF+TRT/SF3m4tRl3M9l9JVKOjk7plRfejXDRodLUaVeU0DunbGBZwbDlkuX1KB+uCQI9B0eHTeI0v1atkl0U8HT1grAXPtFUnL1lT18miXFQmvxY0VYZkZfSNWmWNoAEr3c6gUBJE1Ktm0B1TJNEtQsLrlhxk/lr44seX0SZQsQKDzKkP0bk6IRfKbfQgU8KHeyBupxQpU=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(8936002)(70586007)(54906003)(110136005)(82310400003)(4326008)(5660300002)(81166007)(6666004)(2906002)(316002)(8676002)(508600001)(36860700001)(47076005)(36756003)(1076003)(186003)(26005)(7696005)(86362001)(966005)(336012)(2616005)(426003)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 06:14:57.8736
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e8a6041-2daf-4c57-6697-08d97e5977ec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT027.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6779

Currently, Xen will return IO unhandled when guests access GICD ICPENRn
registers. This will raise a data abort inside guest. For Linux Guest,
these virtual registers will not be accessed. But for Zephyr, in its
GIC initilization code, these virtual registers will be accessed. And
zephyr guest will get an IO dataabort in initilization stage and enter
fatal error. Emulating ICPENDR is not easy with the existing vGIC, so
we currently ignore these virtual registers access and print a message
about whether they are already pending instead of returning unhandled.
More details can be found at [1].

[1] https://lists.xenproject.org/archives/html/xen-devel/2021-09/
msg00744.html

Signed-off-by: Hongda Deng <hongda.deng@arm.com>
---
 xen/arch/arm/vgic-v2.c     | 10 +++++++---
 xen/arch/arm/vgic-v3.c     | 29 +++++++++++++++++------------
 xen/arch/arm/vgic.c        | 37 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/vgic.h |  2 ++
 4 files changed, 63 insertions(+), 15 deletions(-)

diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index b2da886adc..644c62757c 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -481,10 +481,14 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
 
     case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
         if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+
         printk(XENLOG_G_ERR
-               "%pv: vGICD: unhandled word write %#"PRIregister" to ICPENDR%d\n",
-               v, r, gicd_reg - GICD_ICPENDR);
-        return 0;
+               "%pv: vGICD: unhandled word write %#"PRIregister" to ICPENDR%d, and current pending state is: %s\n",
+               v, r, gicd_reg - GICD_ICPENDR,
+               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");
+        goto write_ignore_32;
 
     case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
         if ( dabt.size != DABT_WORD ) goto bad_width;
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index cb5a70c42e..c94e33ff4f 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -817,10 +817,14 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
 
     case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
         if ( dabt.size != DABT_WORD ) goto bad_width;
+        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
+        if ( rank == NULL ) goto write_ignore;
+
         printk(XENLOG_G_ERR
-               "%pv: %s: unhandled word write %#"PRIregister" to ICPENDR%d\n",
-               v, name, r, reg - GICD_ICPENDR);
-        return 0;
+               "%pv: %s: unhandled word write %#"PRIregister" to ICPENDR%d, and current pending state is: %s\n",
+               v, name, r, reg - GICD_ICPENDR,
+               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");
+        goto write_ignore_32;
 
     case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
         if ( dabt.size != DABT_WORD ) goto bad_width;
@@ -978,19 +982,20 @@ static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu *v, mmio_info_t *info,
     case VREG32(GICR_ICFGR1):
     case VRANGE32(GICR_IPRIORITYR0, GICR_IPRIORITYR7):
     case VREG32(GICR_ISPENDR0):
-         /*
-          * Above registers offset are common with GICD.
-          * So handle common with GICD handling
-          */
+        /*
+         * Above registers offset are common with GICD.
+         * So handle common with GICD handling
+         */
         return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
                                                  info, gicr_reg, r);
 
     case VREG32(GICR_ICPENDR0):
-        if ( dabt.size != DABT_WORD ) goto bad_width;
-        printk(XENLOG_G_ERR
-               "%pv: vGICR: SGI: unhandled word write %#"PRIregister" to ICPENDR0\n",
-               v, r);
-        return 0;
+        /*
+         * Above registers offset are common with GICD.
+         * So handle common with GICD handling
+         */
+        return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
+                                                 info, gicr_reg, r);
 
     case VREG32(GICR_IGRPMODR0):
         /* We do not implement security extensions for guests, write ignore */
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 8f9400a519..29a1aa5056 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -470,6 +470,43 @@ void vgic_set_irqs_pending(struct vcpu *v, uint32_t r, unsigned int rank)
     }
 }
 
+bool vgic_get_irqs_pending(struct vcpu *v, uint32_t r, unsigned int rank)
+{
+    const unsigned long mask = r;
+    unsigned int i;
+    /* The first rank is always per-vCPU */
+    bool private = rank == 0;
+    bool is_pending = false;
+
+    /* LPIs status will never be retrieved via this function */
+    ASSERT(!is_lpi(32 * rank + 31));
+
+    for_each_set_bit( i, &mask, 32 )
+    {
+        unsigned int irq = i + 32 * rank;
+
+        if ( !private )
+        {
+            struct pending_irq *p = spi_to_pending(v->domain, irq);
+
+            if ( p->desc != NULL )
+            {
+                unsigned long flags;
+
+                spin_lock_irqsave(&p->desc->lock, flags);
+                is_pending = gic_read_pending_state(p->desc);
+                spin_unlock_irqrestore(&p->desc->lock, flags);
+            }
+        }
+
+        /* When get pending state, break the loop in time */
+        if ( is_pending )
+            break;
+    }
+
+    return is_pending;
+}
+
 bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode,
                  int virq, const struct sgi_target *target)
 {
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 62c2ae538d..ffc841e5f5 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -290,6 +290,8 @@ extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, int n);
 extern void vgic_set_irqs_pending(struct vcpu *v, uint32_t r,
                                   unsigned int rank);
+extern bool vgic_get_irqs_pending(struct vcpu *v, uint32_t r,
+                                  unsigned int rank);
 extern void register_vgic_ops(struct domain *d, const struct vgic_ops *ops);
 int vgic_v2_init(struct domain *d, int *mmio_count);
 int vgic_v3_init(struct domain *d, int *mmio_count);
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:23:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:23:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193431.344528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJ4s-000179-4z; Thu, 23 Sep 2021 07:23:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193431.344528; Thu, 23 Sep 2021 07:23:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJ4s-000172-1c; Thu, 23 Sep 2021 07:23:30 +0000
Received: by outflank-mailman (input) for mailman id 193431;
 Thu, 23 Sep 2021 07:23:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTJ4q-00016w-1n
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:23:28 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 24126841-1c3f-11ec-ba1a-12813bfff9fa;
 Thu, 23 Sep 2021 07:23:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24126841-1c3f-11ec-ba1a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632381805;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=jivZlARPbfDHiXmS1tCj5pdBOxyHVp0XI8yxs7i89dA=;
  b=SePGp5D1TYYX18taFYc61gWFml9WyMvffkXnBJsxMTs2uQSsZNepdESp
   8vJ62Tki4k4t3I4hdzxXBnLlPdr5EhkSHsVPf85tPaCvJBfc102rnQ56C
   Pp4N6tXiSKvjZFMm/smJntuspLLwW4tBQuSZp3oIqRqDZpGBynhIvQCD1
   Y=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: CW5gxOjWoC9Oi5dL8MxWCPd1ZWAvIZ+FmMuvfwqn4L++19JXCFa4JUFrkKyoUYAZ47NmVg73N5
 KXFb9cMc6dvtu9HUPZAHHUMxxcl1R3r7cCFszIPEWtKw7eRapzMywJnrrR4yD2/UcN+8u0jAuw
 XGPf1tvwNGmGCUfBEMdgjHQ8SuzTLgQW05QbRc7pmyblsWbreUiuTtdi8fI8j5hrqwdF+9HaGw
 1pQMzQMNywAVV3v8W3Z4eeBuy7KlN41/CG+mHuby8za33SpBSJPfLXHkWoSGQ66klz3nKSUkS3
 FIFpSHB1dcqOUQOYh3O2DCF8
X-SBRS: 5.1
X-MesageID: 53403100
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:uDKYSqyM5Sov9Cj2kbN6t+eUwSrEfRIJ4+MujC+fZmUNrF6WrkVUy
 2IYUWuFOPiOZ2ryf9p+O9u39EsPuceGmtE2HlBtpSAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRi2tcAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt80ty
 dBdr66LcAtqE7/0ptkldRtDUAgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVort4kI8TxepsWp1lrzC3DDOZgSpfGK0nPzYIDgG9r35gWdRrYT
 /cJSytWfT76WEJSK10ZB9UO29uJpXaqJlW0r3rK/PFqsgA/1jdZz7zFINfTPNuQSq19hE+Ap
 mTH+WvRCxQTJtuZjzGCtG+v7sfDmi7xVYY6Hbix5PlsxlGerkQSFx8+RVa9ueO+iEO1R5RYM
 UN80igzqak/8mS7Q9+7WAe3yFaGsQQbQMF4CPAh5UeGza+8yxaUAC0IQyBMbPQitdQqXno62
 1mRhdTrCDdz9rqPRhqgGqy89G3of3JPdClbOHFCHVBtD8TfTJ8bk07OS8Z5IpWMgp7YCwnO/
 gisrjN9ruBG5SIU7JmT8VfCijOqg5HGSA8p+wnaNl6YAhNFiJ2NPNPztAKFhRpUBMPAFADQ4
 SRd8ySLxL1WVfmweDqxrPLh9V1Dz82MNiHVyXVrFoMon9hG0y/+Jd0IiN2SyUEADyrlRdMLS
 BOI0e+yzMUKVJdPUUORS9jqYyjN5fK8fekJrtiOMrJzjmFZLWdrBh1Ga0+KxHzKm0Mxi6w5M
 przWZ/yVixBUvw/l2TsFrZ1PVoXKsYWnzi7qXfTlUjP7FZjTCTNFedt3KWmNIjVE59oUC2Kq
 o0CZqNmOj1UUfHkYzm/zGLgBQpiEJTPPriv85Y/XrfaemJOQTh9Y9eMkeJJU9E0xMx9y7aXl
 kxRr2cFkTITc1Wccl7UAp2iAZuyNatCQYUTZ3dxYgr4hyZ5Me5CLs43LvMKQFXuz8Q6pdZcR
 PgZYcSQRPNJTzXM4TMGapfh6odlcXyWacimZkJJuRAzIMxtQRLn4Njhcle9/SUCFHPv58A/v
 6ehxkXQRp9aH1ZuC8PfafSOyVKtvCdCxLIuDhWQetQDKl/x9IVKKjDqiqNlKc87NhielCCR0
 BybAElEqLCV8ZM16tTAmYuNs5ytT7llBkNfEmSCteS2OCDW83CN24hFVOrULznRWHmtoPepZ
 PlPzuG6O/oCxQ4Yv415Grdt7KQ/+9qw+OMKklU6RC3GNg35BKlhL3+K2dh0mpdMnrIJ6xGrX
 k+v+8VBPenbMs3SD1NMdhEuaf6O1K9Il2CKv+g1Okjz+AR+4KGDDRdJJxCJhSFQcOl1PYciz
 btzscIa8VXi2B8jM9LAhSFI7WWcaHcHVvx/5J0dBYbqjCsty01DPsOAWnOnvsnXZoUeKFQuL
 x+VmLHG1uZVyUf1enYuEWTAgLhGjpMUtREWlFIPKjxlQDYeaiPbCPGJzQkKcw==
IronPort-HdrOrdr: A9a23:vBDNSKESjsZn6t88pLqFcpHXdLJyesId70hD6qkvc3Nom52j+/
 xGws536faVslcssHFJo6HmBEClewKnyXcT2/htAV7CZnichILMFu9fBOTZsl/d8kHFh4tgPO
 JbAtRD4b7LfClHZKTBkXCF+r8bqbHtmsDY5pav854ud3ATV0gJ1XYGNu/xKDwReOApP+tcKH
 LKjfA32AZINE5nJPiTNz0gZazuttfLnJXpbVovAAMm0hCHiXeN5KThGxaV8x8CW3cXqI1SvV
 Ttokjc3OGOovu7whjT2yv66IlXosLozp9mCNaXgsYYBz3wgkKDZZhnWZeFoDcpydvfpWoCoZ
 3pmVMNLs5z43TeciWcpgbs4RDp1HIU53rr2Taj8DLeiP28YAh/J9tKhIpffBecwVEnpstA3K
 VC2H/cn4ZLDDvb9R6NpuTgZlVPrA6ZsHAimekcgzh0So0FcoJcqoQZ4Qd8DIoAJiTn84oqed
 MeQv003MwmMm9yUkqp/FWGmLeXLzEO91a9Mwc/U/WuonhrdCsT9Tpd+CQd9k1wgq7VBaM0oN
 gsCZ4Y5o2mePVmGp6VNN1xMvdfNVa9NC4kEFjiaWgPR5t3cE4klfbMkcEIDaeRCdo18Kc=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53403100"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ls6zfH39QrhwpjVy7aoO1/2kPyPVbWkjfzjY9Eoeeay0EVTLtylONrkQ1nVkleDSCpoWyjJ2RTC51DDF+B4pzZuUFsggCL9k60OlUEErH2Ew56T7a8PQerW+nG3mP/Qov82wKK7mNpdeHsmAJTZOEPWMxjT+F4jVlVo8nR3R/xKPs9858zxBKpuPbkjjJc+WkFseGBZTDRGxO/ogkKFN7LupnOGCxl86rU/RlkUehSgju3bbsco+q1BmywaFzm/5CXd/Zs2zxOM7VJ++2vTqMVy15ExOgpWhlWjhDVl0MJAsgs54XDDKJNMNXsVJYA8IwBuvAX5vpfFlft2OXuFzWg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Z3sBADxbHmgf+gWA0mJP8l4iNdX4Q4e+8btyNSbBulU=;
 b=HY4BiAQ8OAcI+VwLb5wMgdmKdIJZvndB+YNqpgy8rc/Zm6tVLcWKOYqlXjSHgW+wTTQp4iBJXa4rLQjQH9oDcwjDqDHCGY8LoLKYz3u5hXLizPiR90K3fdfXilVEubDan2pAReF0RIKQFy8iIPNxKylsxmm1Gh9RgQRYtlMLE/TJXfDVSlftfBIYTAmgWFL35f1zpx5iYFaW/ig4etrCwyrgA2OzGHAHbWuW11YHcJ4u43PFVtuGGXj3m3u3ArrJPtpVJdQ0CDtVVe0oXPBx3jqmLoXek/OYnzE3BPXCK4T2RMRxHFqt73K+1EZc9Y9SZHu1zG1A5b6jm7q3dpFijQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Z3sBADxbHmgf+gWA0mJP8l4iNdX4Q4e+8btyNSbBulU=;
 b=XcVwn2LvO6zusvq+cczayePhnVHfTL5eCHN20/z0qyH7J505cELuRemz1vF2l55GX1RmZmJpCuAztSSDWfnZoc+5sBpK5R5mv0czbiRlky68ArlMJ3gH36Ge5UJQZ5Cs3Wr3yOhLWSSFAkJD+2GZRohTKK5aGT95x9BxWcJeXVo=
Date: Thu, 23 Sep 2021 09:23:17 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
CC: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>, Ian
 Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, <raphning@amazon.co.uk>,
	"Doebel, Bjoern" <doebel@amazon.de>
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
Message-ID: <YUwrZS3FOh+hCQle@MacBook-Air-de-Roger.local>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
 <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
 <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
 <56bb55e3-ca7a-12e6-5848-84756b431846@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <56bb55e3-ca7a-12e6-5848-84756b431846@xen.org>
X-ClientProxiedBy: LO4P123CA0258.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:194::11) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7c3a7603-7f8e-406f-5655-08d97e63063b
X-MS-TrafficTypeDiagnostic: DM4PR03MB6192:
X-Microsoft-Antispam-PRVS: <DM4PR03MB61920586447420DE701BC2398FA39@DM4PR03MB6192.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: yLB7tOsOrxRPC02NQAJ6tpKaUwAtZ8h/s1U0RbEFSxLnWEERwTEJ3djl9mV5uiK99Ltj+21doPo9fhxjDWyUZ49R/RSYxGZQc9r1uFGekHWbsHPvSiU4Cpc655B86shcZshwATkd4zT6GQ4vM7XYDmsnqfA7ic+wcmqLgp6bg1iB8pEsei8Z/0VSCqpo/XhJ5b/SNgU+6wlO7EOUlVlFrMJat7gKcz8SvbSWSvHuJIl8SGZvzj7FVS3cEMUPCatrfYYHBX5iTwjJLiPfvv7TjKQEvFLydA1+J/C+vORZcChh93K6OpaxXNnCi6aFXhiG4NH3IP6Eqcx46bJWSICrNwLJcF6QvZkRadnRrPs5za37aMaa+405h0nN1TgTe9WPcnWfS6wBSlUeWvf2xSfkJGT+dbW0Jewfy5gdXMRo6KjLaghV7HyX0WsuObNSkaArGux76sRFaxk91RqvOv7F7CFd57VLYv1NuDfuCn8MbnyLqlTO+h947JEq/a6sb7mHGajErdl0Q9h7mBvPpVZO6k9YIENZoM9npfp1kgGnJVg8nwedW1SgGzUT/G8qgm85iHtRKzJQy3jYaNm22kD8gp/MoKM5FZeKcBCbDdYuI/gEngIhluXt2AILoigPxIokfUME7Dr4vxLau2Ac5kPigCVNlJuLeTjtGz4gYneEAqOglrd5RB+NgAERkO3Exd7ILBmiJidokwEbv2MkXJhdFg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(316002)(6496006)(83380400001)(66556008)(6486002)(6666004)(66476007)(8676002)(9686003)(2906002)(66946007)(508600001)(6916009)(956004)(8936002)(186003)(53546011)(54906003)(38100700002)(5660300002)(26005)(86362001)(85182001)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cUJQb2NCa25UYTh0QmJnREs5UmNaQjY2aVBjYmJJbCsrSGZZYldseHRPSkxF?=
 =?utf-8?B?b05ONDVqVFVia3gvZGNKMk1YMmNPZ0dGS25TMEFqL0w3dUNlQ2FGWEZ3bGxG?=
 =?utf-8?B?UWg1bVY0UmdWUFJaeDFMT0VaUGNTc1JWQmpRQjFCRnRDSmpGYXN0RTBJVVhz?=
 =?utf-8?B?eGt1MFAvWXFuUDJGMFVtZmFIVng3YndRWkpia29WWElNRm5wNUdXdzE0dXQ3?=
 =?utf-8?B?NFBoVGdVNHcrcmllOURpVU5tWm56U01CeThUbW5kNDB5WWJ2NGFJS284cmwr?=
 =?utf-8?B?dmxTRU1STzlhb0xFVEhoMkNIRWhoaGViVEFsOGI0QmpWby9UYkVUQlUrVy90?=
 =?utf-8?B?ZUpibDc4bndQRTBIcjRYVzlmcmhFQXZjcnQzQUFqaDlyaXpId1FhYVRBQkdM?=
 =?utf-8?B?ZU9venluMmhzanJUMWMySDVKZzdvdEtsRmR5b09LaHRsS1MyRmxvbnAwRGZL?=
 =?utf-8?B?ZngvT1U1eG9YN2FSTDJDbjRDc3lVQ2x6dUYzL29yVGVncUNhSVdJNlNmNkEx?=
 =?utf-8?B?MnFHdXQ4VjllZ2lTa1V5Z3BtMGNzR2M2NDU4eUF4UnBoVlNUd3lJbWU1SUFO?=
 =?utf-8?B?SjdrR1h5eE0wOVIzbG9DTGRzRU5YM0RWZjIxd0JDazdkT0d1UEpEUnY5Y1RT?=
 =?utf-8?B?M29hWUhCOFpKR1UyZHlBdTFOVGxWT2FZN2J4ZlF1bUlXNVA2U20xRGp1ellN?=
 =?utf-8?B?b3ZicDhtNC9pSEZ6UmZBc2FiYzBNOXBpMEIxRmltVCtrSHo5M05GczFxRDBz?=
 =?utf-8?B?SUVVa0VGYW5ZUFFQMTI2bVp0bDRGTURocVhiVngxQzZFVlFxNjRBYm5BZlZm?=
 =?utf-8?B?dDRiak1WZmtMdVhCWDVKaVpidmRtTVA3K0NWL01HYUpGOFZ4SUw5aDA2NUh5?=
 =?utf-8?B?ODgvZGJkK0E2NGZNWnFqOXhCMDFLY2Z4cHdISEUyc0xQSnJwcEtSVk82WUxL?=
 =?utf-8?B?SlFWN0RHSXJPZ3hRbDRLMzNHM2dJUHVDQW1FczNlOWdvQ2FpOUI2KzVDVkVh?=
 =?utf-8?B?eDcrTzVoYzAxbkNjMFNzYkk3eUt1bUZBbUVlSWoyTkRJQzJHZm5lQUxTTkNq?=
 =?utf-8?B?M1QrUS8wZytOaVd0c2RXZ3g1U1ppZXhJeEdjS0o1cVdZNTZjSndyYlM5REh0?=
 =?utf-8?B?Q3dMS3djY0RoekQ5TXFPVHlkUElJSVgzNEZMUUkrNGFEZ1gwNzRHU1kwQ0dv?=
 =?utf-8?B?bjlpODI5bnhhZmdYTTJueUZwcFppcG0zS05DZzZBRnpuQS9ySHJEMXZybDRY?=
 =?utf-8?B?dnpxdjkzek52blNORzgyRzVyRWt4Qk1zMkhKTVZHRHVLbkZSQTM4dTRWcExt?=
 =?utf-8?B?SXVIcVl1U0FpSDJsZUFZNUZpUjFEZzNmQThwb1RHb2ZJRW5BUCtjcTg4WWpw?=
 =?utf-8?B?cGsrQldSc2lqQW9VQ1E4c0FQODRDV2tKWDFKcXZpd2NoYWJ2dXZ0MWtGbjBV?=
 =?utf-8?B?ZDUrVzNsV0o0ZVBObmg5bTNqNW5wbkZMU29FaEhUVHFLZmJRZ2RYM1hBM04w?=
 =?utf-8?B?enFQQktlNXg1cHErQXdVSGFBdDR1THd5aW5ub1RwSGI5NWxzdjRxdzZrWnRF?=
 =?utf-8?B?MzFVWWtLNzVMN2l0OEhiYURvOC9VZUZ4b3dxb0NFWkl2dHFVeWtmQThIbFUz?=
 =?utf-8?B?cFMrbzR3ckdIQ3QwZ3VwS1VrQ01ObXZIbnY3WEo1emJENUQ2VlN2blYwaFhl?=
 =?utf-8?B?Qy82SGtPTlpFQktTRkptZGd5aGo0MUUwRHl2dWNJVG05LzFCWm4zdU5sOEhH?=
 =?utf-8?Q?foQrrf71XTBroU3vd9XubMmXuYZdYanSAvQvRZA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c3a7603-7f8e-406f-5655-08d97e63063b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 07:23:22.2642
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GpFOtccEpdECaymzyuYC9YsZvKg96xyw6KXufWTzCKjxEKnfDmAJ0pd03l0j3JqCtg58vSJuANRn7hiAWwsfzA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6192
X-OriginatorOrg: citrix.com

On Wed, Sep 22, 2021 at 06:46:25PM +0500, Julien Grall wrote:
> (+ Some AWS folks)
> 
> Hi Juergen,
> 
> On 22/09/2021 17:34, Juergen Gross wrote:
> > On 22.09.21 12:23, Julien Grall wrote:
> > > Hi Roger,
> > > 
> > > On 22/09/2021 14:58, Roger Pau Monné wrote:
> > > > On Wed, Sep 22, 2021 at 02:07:44PM +0500, Julien Grall wrote:
> > > > > Hi Roger,
> > > > > 
> > > > > On 22/09/2021 13:21, Roger Pau Monne wrote:
> > > > > > Failure to map the shared ring and thus establish a xenstore
> > > > > > connection with a domain shouldn't prevent the "@introduceDomain"
> > > > > > watch from firing, likewise with "@releaseDomain".
> > > > > > 
> > > > > > In order to handle such events properly xenstored should keep track of
> > > > > > the domains even if the shared communication ring cannot be mapped.
> > > > > > This was partially the case due to the restore mode, which needs to
> > > > > > handle domains that have been destroyed between the save and restore
> > > > > > period. This patch extends on the previous limited support of
> > > > > > temporary adding a domain without a valid interface ring, and modifies
> > > > > > check_domains to keep domains without an interface on the list.
> > > > > > 
> > > > > > Handling domains without a valid shared ring is required in order to
> > > > > > support domain without a grant table, since the lack of grant table
> > > > > > will prevent the mapping of the xenstore ring grant reference.
> > > > > > 
> > > > > > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > > > > > ---
> > > > > > oxenstored will need a similar treatment once grant mapping is used
> > > > > > there. For the time being it still works correctly because it uses
> > > > > > foreign memory to map the shared ring, and that will work in the
> > > > > > absence of grant tables on the domain.
> > > > > > ---
> > > > > > Changes since v1:
> > > > > >    - New in this version.
> > > > > > ---
> > > > > >    tools/xenstore/xenstored_domain.c | 30
> > > > > > ++++++++++++++++++------------
> > > > > >    1 file changed, 18 insertions(+), 12 deletions(-)
> > > > > > 
> > > > > > diff --git a/tools/xenstore/xenstored_domain.c
> > > > > > b/tools/xenstore/xenstored_domain.c
> > > > > > index 9fb78d5772..150c6f082e 100644
> > > > > > --- a/tools/xenstore/xenstored_domain.c
> > > > > > +++ b/tools/xenstore/xenstored_domain.c
> > > > > > @@ -119,6 +119,11 @@ static int writechn(struct connection *conn,
> > > > > >        struct xenstore_domain_interface *intf =
> > > > > > conn->domain->interface;
> > > > > >        XENSTORE_RING_IDX cons, prod;
> > > > > > +    if (!intf) {
> > > > > > +        errno = ENODEV;
> > > > > > +        return -1;
> > > > > > +    }
> > > > > > +
> > > > > >        /* Must read indexes once, and before anything
> > > > > > else, and verified. */
> > > > > >        cons = intf->rsp_cons;
> > > > > >        prod = intf->rsp_prod;
> > > > > > @@ -149,6 +154,11 @@ static int readchn(struct
> > > > > > connection *conn, void *data, unsigned int len)
> > > > > >        struct xenstore_domain_interface *intf =
> > > > > > conn->domain->interface;
> > > > > >        XENSTORE_RING_IDX cons, prod;
> > > > > > +    if (!intf) {
> > > > > > +        errno = ENODEV;
> > > > > > +        return -1;
> > > > > > +    }
> > > > > > +
> > > > > >        /* Must read indexes once, and before anything
> > > > > > else, and verified. */
> > > > > >        cons = intf->req_cons;
> > > > > >        prod = intf->req_prod;
> > > > > > @@ -176,6 +186,9 @@ static bool domain_can_write(struct
> > > > > > connection *conn)
> > > > > >    {
> > > > > >        struct xenstore_domain_interface *intf =
> > > > > > conn->domain->interface;
> > > > > > +    if (!intf)
> > > > > > +        return false;
> > > > > > +
> > > > > 
> > > > > Rather than adding an extra check, how about taking
> > > > > advantage of is_ignore?
> > > > 
> > > > Right, I just need to change the order in conn_can_read and
> > > > conn_can_write so that the is_ignored check is performed before the
> > > > can_{read,write} handler is called.
> > > > 
> > > > > >        return ((intf->rsp_prod - intf->rsp_cons) !=
> > > > > > XENSTORE_RING_SIZE);
> > > > > >    }
> > > > > > @@ -183,7 +196,8 @@ static bool domain_can_read(struct
> > > > > > connection *conn)
> > > > > >    {
> > > > > >        struct xenstore_domain_interface *intf =
> > > > > > conn->domain->interface;
> > > > > > -    if (domain_is_unprivileged(conn) && conn->domain->wrl_credit < 0)
> > > > > > +    if ((domain_is_unprivileged(conn) &&
> > > > > > conn->domain->wrl_credit < 0) ||
> > > > > > +        !intf)
> > > > > >            return false;
> > > > > >        return (intf->req_cons != intf->req_prod);
> > > > > > @@ -268,14 +282,7 @@ void check_domains(void)
> > > > > >                    domain->shutdown = true;
> > > > > >                    notify = 1;
> > > > > >                }
> > > > > > -            /*
> > > > > > -             * On Restore, we may have been unable to remap the
> > > > > > -             * interface and the port. As we don't know whether
> > > > > > -             * this was because of a dying domain, we need to
> > > > > > -             * check if the interface and port are still valid.
> > > > > > -             */
> > > > > > -            if (!dominfo.dying && domain->port &&
> > > > > > -                domain->interface)
> > > > > > +            if (!dominfo.dying)
> > > > > >                    continue;
> > > > > 
> > > > > This is mostly a revert on "tools/xenstore: handle dying
> > > > > domains in live
> > > > > update". However, IIRC, this check was necessary to release
> > > > > the connection
> > > > > if the domain has died in the middle of Live-Update.
> > > > 
> > > > But if the domain has died in the middle of live update
> > > > get_domain_info will return false, and thus the code won't get here.
> > > 
> > > Hmmm... I think I am mixing up a few things... I went through the
> > > original discussion (it was on the security ML) to find out why I
> > > wrote the patch like that. When going through the archives, I
> > > noticed that I provided a different version of this patch (see [1])
> > > because there was some issue with the check here (I wrote that it
> > > would lead to zombie domain, but don't have the rationale :().
> > > 
> > > Juergen, I don't seem to find the reason why the patch was not
> > > replaced as we discussed on the security ML. Do you remember why?
> > 
> > Sorry, no, I don't.
> > 
> > You did send the new version for V6 of the LU series, but it seems at
> > least in V9 you commented on the patch requesting that a comment just
> > in the section being different between the two variants to be removed.
> > 
> > So either we both overlooked the new variant not having gone in, or we
> > agreed to use the old version (e.g. in a security meeting). In my IRC
> > logs I couldn't find anything either (the only mentioning of that patch
> > was before V6 of the series was sent, and that was before you sending
> > the new one as a reply to V6).
> > 
> > > Assuming this was a mistake, could someone take care of sending an
> > > update? If not, I could do it when I am back in October.
> > > 
> > > For the archives, the issues would appear when shutting down a
> > > domain during Live-Update.
> > 
> > Hmm, IIRC you did quite some extensive testing of LU and didn't find
> > any problem in the final version.
> 
> I did extensive testing with early series but I can't remember whether I
> tested the shutdown reproducer with the latest series.
> 
> > 
> > Are you sure there really is a problem?
> 
> I thought a bit more and looked at the code (I don't have access to a test
> machine at the moment). I think there is indeed a problem.
> 
> Some watchers of @releaseDomain (such as xenconsoled) will only remove a
> domain from their internal state when the domain is actually dead.
> 
> This is based on dominfo.dying which is only set when all the resources are
> relinquished and waiting for the other domains to release any resources for
> that domain.
> 
> The problem is Xenstore may fail to map the interface or the event channel
> long before the domain is actually dead. With the current check, we would
> free the allocated structure and therefore send @releaseDomain too early. So
> daemon like xenconsoled, would never cleanup for the domain and leave a
> zombie domain. Well... until the next @releaseDomain (or @introduceDomain
> for Xenconsoled) AFAICT.
> 
> The revised patch is meant to solve it by just ignoring the connection. With
> that approach, we would correctly notify watches when the domain is dead.

I think the patch provided by Julien is indeed better than the current
code, or else you will miss @releaseDomain events in corner cases for
dominfo.dying.

I think however the patch is missing a change in the order of the
checks in conn_can_{read,write}, so that the is_ignored check is
performed before calling can_{read,write} which will try to poke at
the interface and trigger a fault if not mapped.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:36:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:36:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193436.344539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJHi-0002oy-9Y; Thu, 23 Sep 2021 07:36:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193436.344539; Thu, 23 Sep 2021 07:36:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJHi-0002or-6H; Thu, 23 Sep 2021 07:36:46 +0000
Received: by outflank-mailman (input) for mailman id 193436;
 Thu, 23 Sep 2021 07:36:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/6DS=ON=epam.com=prvs=9900839950=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTJHg-0002ol-AX
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:36:44 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 97e869a5-73c2-49a6-b9f8-e9a77a0e4e37;
 Thu, 23 Sep 2021 07:36:41 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18N62G6n023681; 
 Thu, 23 Sep 2021 07:36:40 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b8jer8fu0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 07:36:40 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3713.eurprd03.prod.outlook.com (2603:10a6:208:41::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 23 Sep
 2021 07:36:37 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 07:36:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97e869a5-73c2-49a6-b9f8-e9a77a0e4e37
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LJ3lDlt1eLxZtK/2ytbHBdMztrEkHyv0VMn2b5fhG5cQ3heyPG2WcaNQVr0y7wPSfR0z+Ev7ow6Tz+U/nvnabAm0fsLi0SSa7fqEG0wJDpm3YPZQVk1DMzPc2o2R2636+4IzyrVV8er2Ij5LMA/ZJifw18nLgMPshR7KgaJx8Yps02pgSOoqKLl9C+rrdgp3Qo7PGUrwV1NbUmfjbb9/zvR+vxJzMllVXd+zQx20YS3INI0t6q8P0XaI4r2GC45e4VQQRgWPlfbOu0kg9drIh0JMiI+8908+OnOYjlehf1LsuafewdFwQVCZ3mvaR8RlUqIbmlQKnzJK+PcxEEMekg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QGlfgEYuZT3QkP0Wi0tAMUOS9t2BlP683x7RuIKvrfc=;
 b=Nj3rz+9mDKqc9szx4UhWsTegcNFN0CIUIr4VsHy07J7HU5bDDvuQI6osJk2plCX4jn+iR3t77kq6EspGyCKfVVGYvXUMsTbHcTlpT+DJVpKxJolPvlrBwi9uoi2diEHsPQ4zDmDzRCec0XGHR3CkRfFdvwgPRfV8w5ff+gx2zI7Z2CmZVnKLNzz/ORSLQznMoLqPs+9mnLQFKx0vYkvixbfSNFtVta1bceKRIAbFDt38k+GiTle1270/bB5FzWIaHyuTQmGsYsUKc+YkjyMCHyE5J4bKNtRDx0WXWXsZmLuiKK9AN+qyyz6i8pkLrJWEz6P3ogtOXDvKaSTbfipRew==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QGlfgEYuZT3QkP0Wi0tAMUOS9t2BlP683x7RuIKvrfc=;
 b=sG3ox2zlVUa+jl/c1vc4jIjMdKoW9y7aBQputOCoKqjKXF/DfR2U4Kq6HSGsTozj1r6VK5wqQVn4JGbGIUpUQq6DtovWgdqr/Q08DDofqDWVCvpfKJCU3eAfBXmHvv8I5AHunIlAE39Nj8UEnLMbVGPv9KduNKJTAq5xKivBqXz559TjziClmQ2lpv1UZrNlqxI+NpO6yk/Yqfv9RKLN3aTVmw8uzCRs/ON6uAodA0+/FbX76NQ3UOSTX2LzolRCBGDcHjNbTb5AUVHFR5l+Z0trY8gsBMk8lFrOku0bSd4ICyXk8asBdf6gOyrynLlCDRwoEyWTXcfcz1aYHAUXEg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com"
	<jgross@suse.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>,
        Anastasiia Lukianenko
	<Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH 1/2] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH 1/2] xen-pciback: allow compiling on other archs than x86
Thread-Index: AQHXr5qigrpThOcX+k6nYL2UAHzZIquwjH8AgACvxIA=
Date: Thu, 23 Sep 2021 07:36:36 +0000
Message-ID: <11a4dcb0-31c5-9fa5-3c99-b8907b11f19a@epam.com>
References: <20210922101422.2319240-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109221406560.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221406560.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8f37be86-a85b-43b4-2da0-08d97e64e013
x-ms-traffictypediagnostic: AM0PR03MB3713:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB3713327E874562CB6EFD2936E7A39@AM0PR03MB3713.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 AJVtgpr4ISpVI1koE5u+OpXA/LPEPAFmAYW64Zs11AbbGavSCVjfaly3Q9VVjUT2VxRSzKpEoJY/X7uUgdYFGE2LqoR+vUY9RvlOsljCQJZMaM5lMpQ+Bwl0AMvCGql/56lq9NwOgPJMW0IcsSwtr9mUqoQDbRDMietDPaHqcwx3R2sYM7owvNhcxQUyCmSftvBAmVwAkKsymOB55WqXIPwZViqNweAHz54/XQSj2nzNtV2z0b0Wy+5xBZtR7ztjURmlS1XhTcyAlMM7X3R1GwLpEVlG74eCqj6BAnaZoZycObWH9Lqc/o68H8PXtgm59xijsSu4/CdKD2ds5HtWHGJ7HkeUtglod5b3n15KX3d49QymgWkY3tI+AFxY/bUq+U6sdUXomP9LmcL2MV6ZM5va/7WrUVgetg4qqT0xQxWJenzQbYkxXzUV5qjPUcHEmbagtNuqKRar95xaf9r6l8+vGYMLCQ+u9Z/27WqQZFrLzVE3Zb4F7ClmFj8rixAFEAu07sT+RckoYPDedq2dzuujfbidClk7D5CoY9gBvPU2EG8xqrPYPtUUGyl7WZXZ2YlJndy6pJLqUgb8smFHlzF41VeZkYk1vaoxdKrJfTX0keYY4kYvPAVpkviGslY872CcjvDO6GlcDCu4yYdT6Ruh5FDkyJ9JSxbuv8qdpp+wYoOdjjOf7Lw72aoQI28mkPK9VrKF5PT4CR4SAx/52MKRkQBEvgrCIjWF8tR2rSQzUmLGEmOElUs7+demqRVD
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(107886003)(186003)(86362001)(2906002)(66446008)(64756008)(76116006)(36756003)(66556008)(38070700005)(71200400001)(66476007)(31686004)(6916009)(2616005)(6512007)(66946007)(4326008)(6486002)(8676002)(31696002)(5660300002)(54906003)(26005)(83380400001)(8936002)(316002)(38100700002)(53546011)(122000001)(508600001)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NldPMjZlTzNXMEVaRHZNTHdRYnlTSFA4aEVSYlVKVTg1eFJPOGlLeDN3UkhE?=
 =?utf-8?B?aTJCNVZ0dmY5UEcwU0lsTFl4emVpRjh3emI0V0Q5T0N2OHUxM0RzeU9NYTk5?=
 =?utf-8?B?V1BHSFlCcEVPWVFLM2dBc3RMbXVTeE1vblBFQ0RaOGM0eGVTcWZUZUxyeWcx?=
 =?utf-8?B?ekEwUmNUNEVoRWs5STkreXJqRHdPOWM5WXU0ak5za1V6Wmp4Q0h3dFBBUlIv?=
 =?utf-8?B?QkhtMXFEZXFQVFJCR3N5aW1jM01vSUZEUk05VjFkMWNzNGptYUQ2WTl0ZG0z?=
 =?utf-8?B?cUM2SXpLZDRWWGlrQ3lpQ2Z1VUR1dW5JbDZkRGRkSkZFZlJaS01rM3ptaXZ1?=
 =?utf-8?B?ZTNCNi9hbndZd0F1dlF3TG1UV0pVVW81UmVqUEYrZGlrT1FscnJYdEhieUY3?=
 =?utf-8?B?RGxGQkpWQ1FyU3l3MVJocUVMc1NBWTRzWUV3N2RGS1NiRW15cWlKRHltK3BN?=
 =?utf-8?B?SWluRDQ0d0ZjT01NM0tmVTJKa04wRmszeHV2emxQTk5YNWNkdnh3V0RoY2Yv?=
 =?utf-8?B?SDFJZ1hNcVYxU2JiUUpPS2IyRVdSV2VZRUVXTFVKdnp6NGdJRlIva2JwUUgy?=
 =?utf-8?B?a3E0c0xqMTluV0hjaFZBMnJyK2pUWEloUlZDS0EzNjg3enZvQW4zaStJbE9D?=
 =?utf-8?B?eldkVWZjME44VGRWUlpZbm10bmtIRFNQMkRoVUVoVjZPT0lIU1A5L0JzNXgx?=
 =?utf-8?B?MGl6VThPQllEbGp0eWs4aS9MMmdRTitXbDlRWHNqWER6a2VZZ1lrK2tmVmhl?=
 =?utf-8?B?cjNVTThpN254WlBNa1V2RTIyR1BGWS9ZaW43Q3ZycjdpWGhiTUkyZjJuVUhz?=
 =?utf-8?B?UVdSUFdOVUs4SUs1cWYzRlZaMVZLYStBOU1OZm12MWdpOUpKdGg5U2FPMWM5?=
 =?utf-8?B?VndDdmVmVmJzZ29sR29heUc2YmR0cGVobW5SbkNhTTVDRW9POSttbm5aeFk4?=
 =?utf-8?B?Z2NER2xSTEw0NjVwc2puS2RxVHJaU0c4VW1lcElWT01aYTlLcVhQeUdDdFda?=
 =?utf-8?B?VnJtTmgwMDhVR09yNHVhWTdBelMyNXhrVWNUQSszM2NOZ2hjSnUrWU5VY0cw?=
 =?utf-8?B?ZDIxd0hFaFdQZGw0VWtwYVhiN1BobjNoS3BLb2pHL05yQjhyYTFmVE9IRVhK?=
 =?utf-8?B?d1J5MEhWU1BQKzdtY1ZrSE5SanZVVllIWEVoZW9NRGs0dDBrdWdrb3RnM3cr?=
 =?utf-8?B?TDNMQzN2Z1ZpdHlHbmJjbW5WLzIrZzF2OTZDN1hvdDNnbE5XdEtEK1J4UEV2?=
 =?utf-8?B?VW1VNDdaODBLNFhjbGJpSThHd1pDN3RQVHZKVnF2VWJ2K1F0cmVWMHdXeUhT?=
 =?utf-8?B?RUlabDYwYTRUN1VHVG4wTWJZbGUxeXdDVnRBQUZrNFYwZUE0djVkR08rNStF?=
 =?utf-8?B?QS9lN1VlRDFITStzdTFKblNVMmlWWHZLQnpGMzJSN2Z0ejVac2hkZ0VqUXR4?=
 =?utf-8?B?SmwwZVUwMkt0WU9mSTBNS21GNGRTakZqemhFbkcwNExMZ3ZVVlRNTUlBWnFp?=
 =?utf-8?B?cXB2eFBaSGhUMUd4SHhDS2R2SUphNlAyQ1JWcG5IbXZhdkxveTVuL3k5RSs5?=
 =?utf-8?B?d3NUTVZtTVJLc0NYR1pqWE5BblkrQTdjWDZrcllvVzZtQ01vbDhVTFo1RjRh?=
 =?utf-8?B?dGt0WkRBNFNBV0F0cTRMcUUrZis4NGJlbTJkK3hXUEVlYm5KcTUxeHNSQURl?=
 =?utf-8?B?SzJIYjMxZW1wMlErVlo3Y0FYUGozOVhNbDlabFJQalNEOXpoblhXakxVK0NT?=
 =?utf-8?Q?nzJKG1luAWygxJYiq/xBnQfFb8gUvlpqir/N8RP?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A38536C9B966A5428A85D3431B1442AC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f37be86-a85b-43b4-2da0-08d97e64e013
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 07:36:36.9494
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: i5lp26K5HxsXYJTSuH4nNZr2qY3SkzjQaCktOKDF09PnuZBLBjifaINvAAPNKTPnY5dDPlXkOTnX7GewmB8rMD0t/kmM2RuMSZ4kaZm/YOacgZSgkRiZzZd+OgHX61VS
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3713
X-Proofpoint-ORIG-GUID: K03Xf-y37qU_o1CmlBuA80DS7FPwTX9c
X-Proofpoint-GUID: K03Xf-y37qU_o1CmlBuA80DS7FPwTX9c
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_02,2021-09-22_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0
 phishscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999
 priorityscore=1501 malwarescore=0 spamscore=0 suspectscore=0 clxscore=1015
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230045

DQpPbiAyMy4wOS4yMSAwMDowNywgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBXZWQs
IDIyIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBYZW4tcGNpYmFjayBkcml2ZXIgd2FzIGRlc2lnbmVkIHRvIGJlIGJ1aWx0IGZvciB4
ODYgb25seS4gQnV0IGl0DQo+PiBjYW4gYWxzbyBiZSB1c2VkIGJ5IG90aGVyIGFyY2hpdGVjdHVy
ZXMsIGUuZy4gQXJtLg0KPj4gUmUtc3RydWN0dXJlIHRoZSBkcml2ZXIgaW4gYSB3YXkgdGhhdCBp
dCBjYW4gYmUgYnVpbHQgZm9yIG90aGVyDQo+PiBwbGF0Zm9ybXMgYXMgd2VsbC4NCj4+DQo+PiBT
aWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hj
aGVua29AZXBhbS5jb20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBBbmFzdGFzaWlhIEx1a2lhbmVua28g
PGFuYXN0YXNpaWFfbHVraWFuZW5rb0BlcGFtLmNvbT4NCj4gVGhpcyBwYXRjaCBsb29rcyBnb29k
IHRvIG1lIGJ1dCBJIHdvdWxkIHByZWZlciBpZiBpdCBjYW1lIGFmdGVyIHRoZQ0KPiBvdGhlciBw
YXRjaCwgbm90IGJlZm9yZS4NCk9rLCB3aWxsIHN3YXANCj4NCj4NCj4+IC0tLQ0KPj4gU2luY2Ug
djE6DQo+PiAgIC0gRG8gbm90IG1vdmUgcGNpX3hlbl9pbml0aWFsX2RvbWFpbiBhcyBpdCBpcyB4
ODYgc3BlY2lmaWMNCj4+IC0tLQ0KPj4gICBhcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgg
ICAgIHwgMTggKy0tLS0tLQ0KPj4gICBhcmNoL3g4Ni9wY2kveGVuLmMgICAgICAgICAgICAgICAg
IHwgNzQgKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4+ICAgZHJpdmVycy94ZW4vS2Nv
bmZpZyAgICAgICAgICAgICAgICB8ICAyICstDQo+PiAgIGRyaXZlcnMveGVuL3BjaS5jICAgICAg
ICAgICAgICAgICAgfCA3NSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+ICAgZHJp
dmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0dWIuYyB8ICAzICstDQo+PiAgIGRyaXZlcnMveGVu
L3hlbi1wY2liYWNrL3hlbmJ1cy5jICAgfCAgMiArLQ0KPj4gICBpbmNsdWRlL3hlbi9wY2kuaCAg
ICAgICAgICAgICAgICAgIHwgMjggKysrKysrKysrKysNCj4+ICAgNyBmaWxlcyBjaGFuZ2VkLCAx
MDkgaW5zZXJ0aW9ucygrKSwgOTMgZGVsZXRpb25zKC0pDQo+PiAgIGNyZWF0ZSBtb2RlIDEwMDY0
NCBpbmNsdWRlL3hlbi9wY2kuaA0KPj4NCj4+IGRpZmYgLS1naXQgYS9hcmNoL3g4Ni9pbmNsdWRl
L2FzbS94ZW4vcGNpLmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgNCj4+IGluZGV4
IDM1MDZkOGM1OThjMS4uMjg4OWYwOTFmNDU5IDEwMDY0NA0KPj4gLS0tIGEvYXJjaC94ODYvaW5j
bHVkZS9hc20veGVuL3BjaS5oDQo+PiArKysgYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNp
LmgNCj4+IEBAIC0xNiwyNiArMTYsMTAgQEAgc3RhdGljIGlubGluZSBpbnQgcGNpX3hlbl9odm1f
aW5pdCh2b2lkKQ0KPj4gICAjZW5kaWYNCj4+ICAgI2lmIGRlZmluZWQoQ09ORklHX1hFTl9ET00w
KQ0KPj4gICBpbnQgX19pbml0IHBjaV94ZW5faW5pdGlhbF9kb21haW4odm9pZCk7DQo+PiAtaW50
IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAt
aW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYs
IHVpbnQxNl90IGRvbWFpbik7DQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5f
b3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAgICNlbHNlDQo+PiAgIHN0YXRpYyBpbmxp
bmUgaW50IF9faW5pdCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpDQo+PiAgIHsNCj4+IC0J
cmV0dXJuIC0xOw0KPj4gLX0NCj4+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZpY2Vf
ZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLQlyZXR1cm4gLTE7
DQo+PiAtfQ0KPj4gLXN0YXRpYyBpbmxpbmUgaW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWlu
X293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiAtCQkJCQkJICAgdWludDE2X3QgZG9tYWlu
KQ0KPj4gLXsNCj4+IC0JcmV0dXJuIC0xOw0KPj4gLX0NCj4+IC1zdGF0aWMgaW5saW5lIGludCB4
ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+
PiAtew0KPj4gLQlyZXR1cm4gLTE7DQo+PiArICAgICAgIHJldHVybiAtMTsNCj4+ICAgfQ0KPj4g
ICAjZW5kaWYNCj4+ICAgDQo+PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYvcGNpL3hlbi5jIGIvYXJj
aC94ODYvcGNpL3hlbi5jDQo+PiBpbmRleCAzZDQxYTA5YzJjMTQuLjRhNDViMGJmOWFlNCAxMDA2
NDQNCj4+IC0tLSBhL2FyY2gveDg2L3BjaS94ZW4uYw0KPj4gKysrIGIvYXJjaC94ODYvcGNpL3hl
bi5jDQo+PiBAQCAtMjMsNiArMjMsNyBAQA0KPj4gICANCj4+ICAgI2luY2x1ZGUgPHhlbi9mZWF0
dXJlcy5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4v
cGNpLmg+DQo+PiAgICNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4gICAjaW5jbHVkZSA8YXNt
L3hlbi9jcHVpZC5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL2FwaWMuaD4NCj4+IEBAIC01ODMsNzcg
KzU4NCw0IEBAIGludCBfX2luaXQgcGNpX3hlbl9pbml0aWFsX2RvbWFpbih2b2lkKQ0KPj4gICAJ
fQ0KPj4gICAJcmV0dXJuIDA7DQo+PiAgIH0NCj4+IC0NCj4+IC1zdHJ1Y3QgeGVuX2RldmljZV9k
b21haW5fb3duZXIgew0KPj4gLQlkb21pZF90IGRvbWFpbjsNCj4+IC0Jc3RydWN0IHBjaV9kZXYg
KmRldjsNCj4+IC0Jc3RydWN0IGxpc3RfaGVhZCBsaXN0Ow0KPj4gLX07DQo+PiAtDQo+PiAtc3Rh
dGljIERFRklORV9TUElOTE9DSyhkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLXN0YXRp
YyBzdHJ1Y3QgbGlzdF9oZWFkIGRldl9kb21haW5fbGlzdCA9IExJU1RfSEVBRF9JTklUKGRldl9k
b21haW5fbGlzdCk7DQo+PiAtDQo+PiAtc3RhdGljIHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9v
d25lciAqZmluZF9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXN0cnVj
dCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiAtDQo+PiAtCWxpc3RfZm9yX2Vh
Y2hfZW50cnkob3duZXIsICZkZXZfZG9tYWluX2xpc3QsIGxpc3QpIHsNCj4+IC0JCWlmIChvd25l
ci0+ZGV2ID09IGRldikNCj4+IC0JCQlyZXR1cm4gb3duZXI7DQo+PiAtCX0NCj4+IC0JcmV0dXJu
IE5VTEw7DQo+PiAtfQ0KPj4gLQ0KPj4gLWludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVy
KHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLQlzdHJ1Y3QgeGVuX2RldmljZV9kb21h
aW5fb3duZXIgKm93bmVyOw0KPj4gLQlpbnQgZG9tYWluID0gLUVOT0RFVjsNCj4+IC0NCj4+IC0J
c3Bpbl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQlvd25lciA9IGZpbmRf
ZGV2aWNlKGRldik7DQo+PiAtCWlmIChvd25lcikNCj4+IC0JCWRvbWFpbiA9IG93bmVyLT5kb21h
aW47DQo+PiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQly
ZXR1cm4gZG9tYWluOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fZmluZF9kZXZp
Y2VfZG9tYWluX293bmVyKTsNCj4+IC0NCj4+IC1pbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21h
aW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwgdWludDE2X3QgZG9tYWluKQ0KPj4gLXsNCj4+
IC0Jc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4+IC0NCj4+IC0Jb3du
ZXIgPSBremFsbG9jKHNpemVvZihzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIpLCBHRlBf
S0VSTkVMKTsNCj4+IC0JaWYgKCFvd25lcikNCj4+IC0JCXJldHVybiAtRU5PREVWOw0KPj4gLQ0K
Pj4gLQlzcGluX2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtCWlmIChmaW5k
X2RldmljZShkZXYpKSB7DQo+PiAtCQlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5s
b2NrKTsNCj4+IC0JCWtmcmVlKG93bmVyKTsNCj4+IC0JCXJldHVybiAtRUVYSVNUOw0KPj4gLQl9
DQo+PiAtCW93bmVyLT5kb21haW4gPSBkb21haW47DQo+PiAtCW93bmVyLT5kZXYgPSBkZXY7DQo+
PiAtCWxpc3RfYWRkX3RhaWwoJm93bmVyLT5saXN0LCAmZGV2X2RvbWFpbl9saXN0KTsNCj4+IC0J
c3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtCXJldHVybiAwOw0K
Pj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9v
d25lcik7DQo+PiAtDQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIo
c3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFp
bl9vd25lciAqb3duZXI7DQo+PiAtDQo+PiAtCXNwaW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3Nw
aW5sb2NrKTsNCj4+IC0Jb3duZXIgPSBmaW5kX2RldmljZShkZXYpOw0KPj4gLQlpZiAoIW93bmVy
KSB7DQo+PiAtCQlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC0J
CXJldHVybiAtRU5PREVWOw0KPj4gLQl9DQo+PiAtCWxpc3RfZGVsKCZvd25lci0+bGlzdCk7DQo+
PiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQlrZnJlZShv
d25lcik7DQo+PiAtCXJldHVybiAwOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5f
dW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4+ICAgI2VuZGlmDQo+PiBkaWZmIC0t
Z2l0IGEvZHJpdmVycy94ZW4vS2NvbmZpZyBiL2RyaXZlcnMveGVuL0tjb25maWcNCj4+IGluZGV4
IGEzN2ViNTJmYjQwMS4uMDU3ZGRmNjFlZjYxIDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy94ZW4v
S2NvbmZpZw0KPj4gKysrIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4gQEAgLTE4Miw3ICsxODIs
NyBAQCBjb25maWcgU1dJT1RMQl9YRU4NCj4+ICAgDQo+PiAgIGNvbmZpZyBYRU5fUENJREVWX0JB
Q0tFTkQNCj4+ICAgCXRyaXN0YXRlICJYZW4gUENJLWRldmljZSBiYWNrZW5kIGRyaXZlciINCj4+
IC0JZGVwZW5kcyBvbiBQQ0kgJiYgWDg2ICYmIFhFTg0KPj4gKwlkZXBlbmRzIG9uIFBDSSAmJiBY
RU4NCj4+ICAgCWRlcGVuZHMgb24gWEVOX0JBQ0tFTkQNCj4+ICAgCWRlZmF1bHQgbQ0KPj4gICAJ
aGVscA0KPj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3BjaS5jIGIvZHJpdmVycy94ZW4vcGNp
LmMNCj4+IGluZGV4IDIyNGRmMDNjZTQyZS4uZmM4YzEyNDlkNDlmIDEwMDY0NA0KPj4gLS0tIGEv
ZHJpdmVycy94ZW4vcGNpLmMNCj4+ICsrKyBiL2RyaXZlcnMveGVuL3BjaS5jDQo+PiBAQCAtMjU0
LDMgKzI1NCw3OCBAQCBzdGF0aWMgaW50IHhlbl9tY2ZnX2xhdGUodm9pZCkNCj4+ICAgCXJldHVy
biAwOw0KPj4gICB9DQo+PiAgICNlbmRpZg0KPj4gKw0KPj4gKyNpZmRlZiBDT05GSUdfWEVOX0RP
TTANCj4+ICtzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgew0KPj4gKwlkb21pZF90IGRv
bWFpbjsNCj4+ICsJc3RydWN0IHBjaV9kZXYgKmRldjsNCj4+ICsJc3RydWN0IGxpc3RfaGVhZCBs
aXN0Ow0KPj4gK307DQo+PiArDQo+PiArc3RhdGljIERFRklORV9TUElOTE9DSyhkZXZfZG9tYWlu
X2xpc3Rfc3BpbmxvY2spOw0KPj4gK3N0YXRpYyBzdHJ1Y3QgbGlzdF9oZWFkIGRldl9kb21haW5f
bGlzdCA9IExJU1RfSEVBRF9JTklUKGRldl9kb21haW5fbGlzdCk7DQo+PiArDQo+PiArc3RhdGlj
IHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqZmluZF9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKmRldikNCj4+ICt7DQo+PiArCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3du
ZXI7DQo+PiArDQo+PiArCWxpc3RfZm9yX2VhY2hfZW50cnkob3duZXIsICZkZXZfZG9tYWluX2xp
c3QsIGxpc3QpIHsNCj4+ICsJCWlmIChvd25lci0+ZGV2ID09IGRldikNCj4+ICsJCQlyZXR1cm4g
b3duZXI7DQo+PiArCX0NCj4+ICsJcmV0dXJuIE5VTEw7DQo+PiArfQ0KPj4gKw0KPj4gK2ludCB4
ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiArew0K
Pj4gKwlzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPj4gKwlpbnQgZG9t
YWluID0gLUVOT0RFVjsNCj4+ICsNCj4+ICsJc3Bpbl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3Bp
bmxvY2spOw0KPj4gKwlvd25lciA9IGZpbmRfZGV2aWNlKGRldik7DQo+PiArCWlmIChvd25lcikN
Cj4+ICsJCWRvbWFpbiA9IG93bmVyLT5kb21haW47DQo+PiArCXNwaW5fdW5sb2NrKCZkZXZfZG9t
YWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gKwlyZXR1cm4gZG9tYWluOw0KPj4gK30NCj4+ICtFWFBP
UlRfU1lNQk9MX0dQTCh4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4+ICsNCj4+ICtp
bnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwg
dWludDE2X3QgZG9tYWluKQ0KPj4gK3sNCj4+ICsJc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293
bmVyICpvd25lcjsNCj4+ICsNCj4+ICsJb3duZXIgPSBremFsbG9jKHNpemVvZihzdHJ1Y3QgeGVu
X2RldmljZV9kb21haW5fb3duZXIpLCBHRlBfS0VSTkVMKTsNCj4+ICsJaWYgKCFvd25lcikNCj4+
ICsJCXJldHVybiAtRU5PREVWOw0KPj4gKw0KPj4gKwlzcGluX2xvY2soJmRldl9kb21haW5fbGlz
dF9zcGlubG9jayk7DQo+PiArCWlmIChmaW5kX2RldmljZShkZXYpKSB7DQo+PiArCQlzcGluX3Vu
bG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJCWtmcmVlKG93bmVyKTsNCj4+
ICsJCXJldHVybiAtRUVYSVNUOw0KPj4gKwl9DQo+PiArCW93bmVyLT5kb21haW4gPSBkb21haW47
DQo+PiArCW93bmVyLT5kZXYgPSBkZXY7DQo+PiArCWxpc3RfYWRkX3RhaWwoJm93bmVyLT5saXN0
LCAmZGV2X2RvbWFpbl9saXN0KTsNCj4+ICsJc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9z
cGlubG9jayk7DQo+PiArCXJldHVybiAwOw0KPj4gK30NCj4+ICtFWFBPUlRfU1lNQk9MX0dQTCh4
ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcik7DQo+PiArDQo+PiAraW50IHhlbl91bnJl
Z2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+ICt7DQo+
PiArCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiArDQo+PiArCXNw
aW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJb3duZXIgPSBmaW5kX2Rl
dmljZShkZXYpOw0KPj4gKwlpZiAoIW93bmVyKSB7DQo+PiArCQlzcGluX3VubG9jaygmZGV2X2Rv
bWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJCXJldHVybiAtRU5PREVWOw0KPj4gKwl9DQo+PiAr
CWxpc3RfZGVsKCZvd25lci0+bGlzdCk7DQo+PiArCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xp
c3Rfc3BpbmxvY2spOw0KPj4gKwlrZnJlZShvd25lcik7DQo+PiArCXJldHVybiAwOw0KPj4gK30N
Cj4+ICtFWFBPUlRfU1lNQk9MX0dQTCh4ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVy
KTsNCj4+ICsjZW5kaWYNCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9w
Y2lfc3R1Yi5jIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0dWIuYw0KPj4gaW5kZXgg
ZjhlNGZhYTk2YWQ2Li5iYmE1Mjc2MjA1MDcgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay9wY2lfc3R1Yi5jDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9w
Y2lfc3R1Yi5jDQo+PiBAQCAtMTksNyArMTksOCBAQA0KPj4gICAjaW5jbHVkZSA8bGludXgvc2No
ZWQuaD4NCj4+ICAgI2luY2x1ZGUgPGxpbnV4L2F0b21pYy5oPg0KPj4gICAjaW5jbHVkZSA8eGVu
L2V2ZW50cy5oPg0KPj4gLSNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDx4
ZW4vcGNpLmg+DQo+PiArI2luY2x1ZGUgPHhlbi94ZW4uaD4NCj4+ICAgI2luY2x1ZGUgPGFzbS94
ZW4vaHlwZXJ2aXNvci5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2ludGVyZmFjZS9waHlzZGV2Lmg+
DQo+PiAgICNpbmNsdWRlICJwY2liYWNrLmgiDQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4v
eGVuLXBjaWJhY2sveGVuYnVzLmMgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0K
Pj4gaW5kZXggYzA5YzdlYmQ2OTY4Li5kYTM0Y2U4NWRjODggMTAwNjQ0DQo+PiAtLS0gYS9kcml2
ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gKysrIGIvZHJpdmVycy94ZW4veGVuLXBj
aWJhY2sveGVuYnVzLmMNCj4+IEBAIC0xNCw3ICsxNCw3IEBADQo+PiAgICNpbmNsdWRlIDxsaW51
eC93b3JrcXVldWUuaD4NCj4+ICAgI2luY2x1ZGUgPHhlbi94ZW5idXMuaD4NCj4+ICAgI2luY2x1
ZGUgPHhlbi9ldmVudHMuaD4NCj4+IC0jaW5jbHVkZSA8YXNtL3hlbi9wY2kuaD4NCj4+ICsjaW5j
bHVkZSA8eGVuL3BjaS5oPg0KPj4gICAjaW5jbHVkZSAicGNpYmFjay5oIg0KPj4gICANCj4+ICAg
I2RlZmluZSBJTlZBTElEX0VWVENITl9JUlEgICgtMSkNCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRl
L3hlbi9wY2kuaCBiL2luY2x1ZGUveGVuL3BjaS5oDQo+PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0K
Pj4gaW5kZXggMDAwMDAwMDAwMDAwLi5iODMzN2NmODVmZDENCj4+IC0tLSAvZGV2L251bGwNCj4+
ICsrKyBiL2luY2x1ZGUveGVuL3BjaS5oDQo+PiBAQCAtMCwwICsxLDI4IEBADQo+PiArLyogU1BE
WC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAgKi8NCj4+ICsNCj4+ICsjaWZuZGVmIF9fWEVO
X1BDSV9IX18NCj4+ICsjZGVmaW5lIF9fWEVOX1BDSV9IX18NCj4+ICsNCj4+ICsjaWYgZGVmaW5l
ZChDT05GSUdfWEVOX0RPTTApDQo+PiAraW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIo
c3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAraW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWlu
X293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsIHVpbnQxNl90IGRvbWFpbik7DQo+PiAraW50IHhl
bl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+
PiArI2Vsc2UNCj4+ICtzdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293
bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+PiArfQ0K
Pj4gKw0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293
bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiArCQkJCQkJICAgdWludDE2X3QgZG9tYWluKQ0K
Pj4gK3sNCj4+ICsJcmV0dXJuIC0xOw0KPj4gK30NCj4+ICsNCj4+ICtzdGF0aWMgaW5saW5lIGlu
dCB4ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYp
DQo+PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+PiArfQ0KPj4gKyNlbmRpZg0KPj4gKw0KPj4gKyNl
bmRpZg0KPj4gLS0gDQo+PiAyLjI1LjENCj4+


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:43:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:43:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193444.344550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOW-0004WW-4u; Thu, 23 Sep 2021 07:43:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193444.344550; Thu, 23 Sep 2021 07:43:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOW-0004WP-1e; Thu, 23 Sep 2021 07:43:48 +0000
Received: by outflank-mailman (input) for mailman id 193444;
 Thu, 23 Sep 2021 07:43:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=64r8=ON=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mTJOV-0004WJ-6K
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:43:47 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fb5aa7f2-1c41-11ec-ba1a-12813bfff9fa;
 Thu, 23 Sep 2021 07:43:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D07E60EC0;
 Thu, 23 Sep 2021 07:43:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb5aa7f2-1c41-11ec-ba1a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632383025;
	bh=3uVNPtFfbPBZRQaNfQKwTwtev9YVKUZqny4BWnp3ztY=;
	h=From:To:Cc:Subject:Date:From;
	b=Vold8Z9hLOut5Y57D9YqGfAPpL4eeU2GGKe6N2/G2K+rSChme5BK2bVJUGQMhNnD5
	 tlc7pIoxBTA8mtBmkSbcvxOA4ol2fbYKLvd84RUIoei3PfOrgW7d8UvcqmHNvAAnqH
	 2rTOLkAFW6DycEheiJuAByIZSW1Mb0ht/+wW3EsUTQK66rudMrHlsuHTSWtaF9pCtZ
	 zz2eXREGQLaTYVqwKNRZd1xLM2W36l4O1x9wg+UbZ8351+5wCVj9XzBQPxb3HkA2G0
	 kjff1rDe9w+gBCM2bpBWj7skIwT6XMoMhVQl3RcYgUgyL/ulxZSnczTeB3j1RHlvfS
	 d5Zxibdoq1rzA==
From: Mike Rapoport <rppt@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: [PATCH 0/3] memblock: cleanup memblock_free interface
Date: Thu, 23 Sep 2021 10:43:32 +0300
Message-Id: <20210923074335.12583-1-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Hi,

Following the discussion on [1] this is the fix for memblock freeing APIs
mismatch. 

The first patch is a cleanup of numa_distance allocation in arch_numa I've
spotted during the conversion.
The second patch is a fix for Xen memory freeing on some of the error
paths.

The core change is in the third patch that makes memblock_free() a
counterpart of memblock_alloc() and adds memblock_phys_alloc() to be a
counterpart of memblock_phys_alloc().

Since scripts/get_maintainer.pl returned more than 100 addresses I've
trimmed the distribution list only to the relevant lists.

[1] https://lore.kernel.org/all/CAHk-=wj9k4LZTz+svCxLYs5Y1=+yKrbAUArH1+ghyG3OLd8VVg@mail.gmail.com

Mike Rapoport (3):
  arch_numa: simplify numa_distance allocation
  xen/x86: free_p2m_page: use memblock_free_ptr() to free a virtual pointer
  memblock: cleanup memblock_free interface

 arch/alpha/kernel/core_irongate.c         |  2 +-
 arch/arc/mm/init.c                        |  2 +-
 arch/arm/mach-hisi/platmcpm.c             |  2 +-
 arch/arm/mm/init.c                        |  2 +-
 arch/arm64/mm/mmu.c                       |  4 ++--
 arch/mips/mm/init.c                       |  2 +-
 arch/mips/sgi-ip30/ip30-setup.c           |  6 +++---
 arch/powerpc/kernel/dt_cpu_ftrs.c         |  2 +-
 arch/powerpc/kernel/paca.c                |  4 ++--
 arch/powerpc/kernel/setup-common.c        |  2 +-
 arch/powerpc/kernel/setup_64.c            |  2 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |  2 +-
 arch/powerpc/platforms/pseries/svm.c      |  4 +---
 arch/riscv/kernel/setup.c                 |  4 ++--
 arch/s390/kernel/setup.c                  |  8 ++++----
 arch/s390/kernel/smp.c                    |  4 ++--
 arch/s390/kernel/uv.c                     |  2 +-
 arch/s390/mm/kasan_init.c                 |  2 +-
 arch/sh/boards/mach-ap325rxa/setup.c      |  2 +-
 arch/sh/boards/mach-ecovec24/setup.c      |  4 ++--
 arch/sh/boards/mach-kfr2r09/setup.c       |  2 +-
 arch/sh/boards/mach-migor/setup.c         |  2 +-
 arch/sh/boards/mach-se/7724/setup.c       |  4 ++--
 arch/sparc/kernel/smp_64.c                |  2 +-
 arch/um/kernel/mem.c                      |  2 +-
 arch/x86/kernel/setup.c                   |  4 ++--
 arch/x86/kernel/setup_percpu.c            |  2 +-
 arch/x86/mm/init.c                        |  2 +-
 arch/x86/mm/kasan_init_64.c               |  4 ++--
 arch/x86/mm/numa.c                        |  2 +-
 arch/x86/mm/numa_emulation.c              |  2 +-
 arch/x86/xen/mmu_pv.c                     |  6 +++---
 arch/x86/xen/p2m.c                        |  2 +-
 arch/x86/xen/setup.c                      |  6 +++---
 drivers/base/arch_numa.c                  | 10 ++++------
 drivers/firmware/efi/memmap.c             |  2 +-
 drivers/macintosh/smu.c                   |  2 +-
 drivers/of/kexec.c                        |  2 +-
 drivers/of/of_reserved_mem.c              |  4 ++--
 drivers/s390/char/sclp_early.c            |  2 +-
 drivers/usb/early/xhci-dbc.c              | 10 +++++-----
 drivers/xen/swiotlb-xen.c                 |  2 +-
 include/linux/memblock.h                  | 16 ++--------------
 init/initramfs.c                          |  2 +-
 init/main.c                               |  2 +-
 kernel/dma/swiotlb.c                      |  2 +-
 kernel/printk/printk.c                    |  4 ++--
 lib/bootconfig.c                          |  2 +-
 lib/cpumask.c                             |  2 +-
 mm/cma.c                                  |  2 +-
 mm/memblock.c                             | 20 ++++++++++----------
 mm/memory_hotplug.c                       |  2 +-
 mm/percpu.c                               |  8 ++++----
 mm/sparse.c                               |  2 +-
 tools/bootconfig/include/linux/memblock.h |  2 +-
 55 files changed, 94 insertions(+), 110 deletions(-)


base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:43:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:43:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193445.344561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOb-0004ny-Dc; Thu, 23 Sep 2021 07:43:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193445.344561; Thu, 23 Sep 2021 07:43:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOb-0004np-AY; Thu, 23 Sep 2021 07:43:53 +0000
Received: by outflank-mailman (input) for mailman id 193445;
 Thu, 23 Sep 2021 07:43:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=64r8=ON=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mTJOa-0004n9-CZ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:43:52 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id abdc063d-89e2-4baf-8c07-42e417cee739;
 Thu, 23 Sep 2021 07:43:51 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E807D61038;
 Thu, 23 Sep 2021 07:43:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abdc063d-89e2-4baf-8c07-42e417cee739
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632383031;
	bh=ISUQEAeabD8TFuzFwCipkjqJm558AV8oII+raBjMcR4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=qp4PlSy1I2qLdHfiL+6LZq+DtUAW0BZNW8wWana91BkCU23YNyluYXPn3qRTLPtZv
	 nKeiyUiksdKxzzSpxVSgam85AgBf4/Oi9IBa05SDCS8Vhd7zIdAhFgm+m6hDjh46y+
	 zgSAUfDAyTkvZojIS/HqTJ8bHsBPWFPJqCUkPfRiYtndeE5czEFfrL7O0vHSE7G8ps
	 raokz57MNdzvGevCOJ7eHnUIMwrQwINX3LVGYRMX+O8ir8PikcZnvpGJvgoQZnchBJ
	 rPqRejbi67XM5nP4AeGEbzjUVLZdJqRjdH9jyG3N+kSrB2M0hOUOUOmwRt0L5pnKJd
	 hgPAAvvP+k1Gw==
From: Mike Rapoport <rppt@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: [PATCH 1/3] arch_numa: simplify numa_distance allocation
Date: Thu, 23 Sep 2021 10:43:33 +0300
Message-Id: <20210923074335.12583-2-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210923074335.12583-1-rppt@kernel.org>
References: <20210923074335.12583-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Memory allocation of numa_distance uses memblock_phys_alloc_range() without
actual range limits, converts the returned physical address to virtual and
then only uses the virtual address for further initialization.

Simplify this by replacing memblock_phys_alloc_range() with
memblock_alloc().

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 drivers/base/arch_numa.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index 00fb4120a5b3..f6d0efd01188 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -275,15 +275,13 @@ void __init numa_free_distance(void)
 static int __init numa_alloc_distance(void)
 {
 	size_t size;
-	u64 phys;
 	int i, j;
 
 	size = nr_node_ids * nr_node_ids * sizeof(numa_distance[0]);
-	phys = memblock_phys_alloc_range(size, PAGE_SIZE, 0, PFN_PHYS(max_pfn));
-	if (WARN_ON(!phys))
+	numa_distance = memblock_alloc(size, PAGE_SIZE);
+	if (WARN_ON(!numa_distance))
 		return -ENOMEM;
 
-	numa_distance = __va(phys);
 	numa_distance_cnt = nr_node_ids;
 
 	/* fill with the default distances */
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:43:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:43:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193447.344572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOh-0005AE-NQ; Thu, 23 Sep 2021 07:43:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193447.344572; Thu, 23 Sep 2021 07:43:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOh-0005A7-JL; Thu, 23 Sep 2021 07:43:59 +0000
Received: by outflank-mailman (input) for mailman id 193447;
 Thu, 23 Sep 2021 07:43:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=64r8=ON=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mTJOg-000583-1r
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:43:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0216b996-1c42-11ec-ba1a-12813bfff9fa;
 Thu, 23 Sep 2021 07:43:57 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9933660EC0;
 Thu, 23 Sep 2021 07:43:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0216b996-1c42-11ec-ba1a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632383036;
	bh=iWKfUnKlQ9cPEYiRCmYD11oUFDahtkWK+8zPXcxbHfw=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=H1kaX42zyV3QIA+qDp4knzrt9WyS+qws3w1e8o4EW+yTnl/4fM57n5Dphy5IjQHJ1
	 6E+ppDgbmXTDPAKAQlraJB76u0SOF1v5mXkUUakV90s32cAhMAYyDsPNx4Sc1n4wvi
	 W2oWhofnItmWkCPORqhQx9dfBnxoDuP3YwVe/8TymJVaQJp3MdfdmJm4lZw7qje1Mr
	 1e4A83pakvGWhdUmI+pTan/SrNA9KzDSPNoli0MAkH0aR0LyfCR5iZGE12ruwS9Z1D
	 HQ9EitT9EFvxvF9zxe+XZQX3fVc+GHOu2xA+uaYHGpyeNsZPbdbM0oQVVJVjEdP4Km
	 NSNLeM6nYHuFw==
From: Mike Rapoport <rppt@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: [PATCH 2/3] xen/x86: free_p2m_page: use memblock_free_ptr() to free a virtual pointer
Date: Thu, 23 Sep 2021 10:43:34 +0300
Message-Id: <20210923074335.12583-3-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210923074335.12583-1-rppt@kernel.org>
References: <20210923074335.12583-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

free_p2m_page() wrongly passes a virtual pointer to memblock_free() that
treats it as a physical address.

Call memblock_free_ptr() instead that gets a virtual address to free the
memory.

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 arch/x86/xen/p2m.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index 5e6e236977c7..141bb9dbd2fb 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -197,7 +197,7 @@ static void * __ref alloc_p2m_page(void)
 static void __ref free_p2m_page(void *p)
 {
 	if (unlikely(!slab_is_available())) {
-		memblock_free((unsigned long)p, PAGE_SIZE);
+		memblock_free_ptr(p, PAGE_SIZE);
 		return;
 	}
 
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:44:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:44:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193449.344583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOp-0005fJ-0u; Thu, 23 Sep 2021 07:44:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193449.344583; Thu, 23 Sep 2021 07:44:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJOo-0005fA-TO; Thu, 23 Sep 2021 07:44:06 +0000
Received: by outflank-mailman (input) for mailman id 193449;
 Thu, 23 Sep 2021 07:44:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=64r8=ON=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mTJOo-0005cf-F8
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:44:06 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 05dad936-1c42-11ec-ba1a-12813bfff9fa;
 Thu, 23 Sep 2021 07:44:03 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4CE7161211;
 Thu, 23 Sep 2021 07:43:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05dad936-1c42-11ec-ba1a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632383043;
	bh=mJeWK+xuVoCeSQUmLH9lpzgLfVM1zzfE4HrawQix1ug=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=KDrYudBp8kqpKSZtmmIHXjlbky0zSU6IgZkAYAHvWX3IrY14LXrpS/dHIHT3SN6Vl
	 nOzJWRJpdmurdxYM3sO6Mr7Ndo5qlnPMAEIZcjWgg7YxhGrKK5ymp2HkxU1pJChfIe
	 1gOq6WMR0+7vnM78DS1+gp3E/ehMmeo//uouFbUGEdsXAJx7LvOBUF5q/LZTHSngpS
	 jdps5qhMB1HGsJgLHs4d7tSq5Pa63rQVhYgjYBr5cVtRuPqTvwKn6l3Zg2kw0ciJXF
	 6VGEcMAMuiRhUgLN59eMAPomyCoYcKW6Hv+t4savcjbNBrFB3Em9Pc6MU4TcVVPuti
	 qv1IzAd/rH19A==
From: Mike Rapoport <rppt@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: [PATCH 3/3] memblock: cleanup memblock_free interface
Date: Thu, 23 Sep 2021 10:43:35 +0300
Message-Id: <20210923074335.12583-4-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210923074335.12583-1-rppt@kernel.org>
References: <20210923074335.12583-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

For ages memblock_free() interface dealt with physical addresses even
despite the existence of memblock_alloc_xx() functions that return a
virtual pointer.

Introduce memblock_phys_free() for freeing physical ranges and repurpose
memblock_free() to free virtual pointers to make the following pairing
abundantly clear:

	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);

	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
	void memblock_free(void *ptr, size_t size);

Replace intermediate memblock_free_ptr() with memblock_free() and drop
unnecessary aliases memblock_free_early() and memblock_free_early_nid().

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 arch/alpha/kernel/core_irongate.c         |  2 +-
 arch/arc/mm/init.c                        |  2 +-
 arch/arm/mach-hisi/platmcpm.c             |  2 +-
 arch/arm/mm/init.c                        |  2 +-
 arch/arm64/mm/mmu.c                       |  4 ++--
 arch/mips/mm/init.c                       |  2 +-
 arch/mips/sgi-ip30/ip30-setup.c           |  6 +++---
 arch/powerpc/kernel/dt_cpu_ftrs.c         |  2 +-
 arch/powerpc/kernel/paca.c                |  4 ++--
 arch/powerpc/kernel/setup-common.c        |  2 +-
 arch/powerpc/kernel/setup_64.c            |  2 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |  2 +-
 arch/powerpc/platforms/pseries/svm.c      |  4 +---
 arch/riscv/kernel/setup.c                 |  4 ++--
 arch/s390/kernel/setup.c                  |  8 ++++----
 arch/s390/kernel/smp.c                    |  4 ++--
 arch/s390/kernel/uv.c                     |  2 +-
 arch/s390/mm/kasan_init.c                 |  2 +-
 arch/sh/boards/mach-ap325rxa/setup.c      |  2 +-
 arch/sh/boards/mach-ecovec24/setup.c      |  4 ++--
 arch/sh/boards/mach-kfr2r09/setup.c       |  2 +-
 arch/sh/boards/mach-migor/setup.c         |  2 +-
 arch/sh/boards/mach-se/7724/setup.c       |  4 ++--
 arch/sparc/kernel/smp_64.c                |  2 +-
 arch/um/kernel/mem.c                      |  2 +-
 arch/x86/kernel/setup.c                   |  4 ++--
 arch/x86/kernel/setup_percpu.c            |  2 +-
 arch/x86/mm/init.c                        |  2 +-
 arch/x86/mm/kasan_init_64.c               |  4 ++--
 arch/x86/mm/numa.c                        |  2 +-
 arch/x86/mm/numa_emulation.c              |  2 +-
 arch/x86/xen/mmu_pv.c                     |  6 +++---
 arch/x86/xen/p2m.c                        |  2 +-
 arch/x86/xen/setup.c                      |  6 +++---
 drivers/base/arch_numa.c                  |  4 ++--
 drivers/firmware/efi/memmap.c             |  2 +-
 drivers/macintosh/smu.c                   |  2 +-
 drivers/of/kexec.c                        |  2 +-
 drivers/of/of_reserved_mem.c              |  4 ++--
 drivers/s390/char/sclp_early.c            |  2 +-
 drivers/usb/early/xhci-dbc.c              | 10 +++++-----
 drivers/xen/swiotlb-xen.c                 |  2 +-
 include/linux/memblock.h                  | 16 ++--------------
 init/initramfs.c                          |  2 +-
 init/main.c                               |  2 +-
 kernel/dma/swiotlb.c                      |  2 +-
 kernel/printk/printk.c                    |  4 ++--
 lib/bootconfig.c                          |  2 +-
 lib/cpumask.c                             |  2 +-
 mm/cma.c                                  |  2 +-
 mm/memblock.c                             | 20 ++++++++++----------
 mm/memory_hotplug.c                       |  2 +-
 mm/percpu.c                               |  8 ++++----
 mm/sparse.c                               |  2 +-
 tools/bootconfig/include/linux/memblock.h |  2 +-
 55 files changed, 92 insertions(+), 106 deletions(-)

diff --git a/arch/alpha/kernel/core_irongate.c b/arch/alpha/kernel/core_irongate.c
index 72af1e72d833..6b8ed12936b6 100644
--- a/arch/alpha/kernel/core_irongate.c
+++ b/arch/alpha/kernel/core_irongate.c
@@ -233,7 +233,7 @@ albacore_init_arch(void)
 			unsigned long size;
 
 			size = initrd_end - initrd_start;
-			memblock_free(__pa(initrd_start), PAGE_ALIGN(size));
+			memblock_free((void *)initrd_start, PAGE_ALIGN(size));
 			if (!move_initrd(pci_mem))
 				printk("irongate_init_arch: initrd too big "
 				       "(%ldK)\ndisabling initrd\n",
diff --git a/arch/arc/mm/init.c b/arch/arc/mm/init.c
index 699ecf119641..59408f6a02d4 100644
--- a/arch/arc/mm/init.c
+++ b/arch/arc/mm/init.c
@@ -173,7 +173,7 @@ static void __init highmem_init(void)
 #ifdef CONFIG_HIGHMEM
 	unsigned long tmp;
 
-	memblock_free(high_mem_start, high_mem_sz);
+	memblock_phys_free(high_mem_start, high_mem_sz);
 	for (tmp = min_high_pfn; tmp < max_high_pfn; tmp++)
 		free_highmem_page(pfn_to_page(tmp));
 #endif
diff --git a/arch/arm/mach-hisi/platmcpm.c b/arch/arm/mach-hisi/platmcpm.c
index 96a484095194..258586e31333 100644
--- a/arch/arm/mach-hisi/platmcpm.c
+++ b/arch/arm/mach-hisi/platmcpm.c
@@ -339,7 +339,7 @@ static int __init hip04_smp_init(void)
 err_sysctrl:
 	iounmap(relocation);
 err_reloc:
-	memblock_free(hip04_boot_method[0], hip04_boot_method[1]);
+	memblock_phys_free(hip04_boot_method[0], hip04_boot_method[1]);
 err:
 	return ret;
 }
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6162a070a410..6d0cb0f7bc54 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -158,7 +158,7 @@ phys_addr_t __init arm_memblock_steal(phys_addr_t size, phys_addr_t align)
 		panic("Failed to steal %pa bytes at %pS\n",
 		      &size, (void *)_RET_IP_);
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	return phys;
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index cfd9deb347c3..f68c2d953617 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -738,8 +738,8 @@ void __init paging_init(void)
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
 	init_mm.pgd = swapper_pg_dir;
 
-	memblock_free(__pa_symbol(init_pg_dir),
-		      __pa_symbol(init_pg_end) - __pa_symbol(init_pg_dir));
+	memblock_phys_free(__pa_symbol(init_pg_dir),
+			   __pa_symbol(init_pg_end) - __pa_symbol(init_pg_dir));
 
 	memblock_allow_resize();
 }
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 19347dc6bbf8..325e1552cbea 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -529,7 +529,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/arch/mips/sgi-ip30/ip30-setup.c b/arch/mips/sgi-ip30/ip30-setup.c
index 44b1607e964d..75a34684e704 100644
--- a/arch/mips/sgi-ip30/ip30-setup.c
+++ b/arch/mips/sgi-ip30/ip30-setup.c
@@ -69,10 +69,10 @@ static void __init ip30_mem_init(void)
 		total_mem += size;
 
 		if (addr >= IP30_REAL_MEMORY_START)
-			memblock_free(addr, size);
+			memblock_phys_free(addr, size);
 		else if ((addr + size) > IP30_REAL_MEMORY_START)
-			memblock_free(IP30_REAL_MEMORY_START,
-				     size - IP30_MAX_PROM_MEMORY);
+			memblock_phys_free(IP30_REAL_MEMORY_START,
+					   size - IP30_MAX_PROM_MEMORY);
 	}
 	pr_info("Detected %luMB of physical memory.\n", MEM_SHIFT(total_mem));
 }
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
index 358aee7c2d79..705757f1a809 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -1095,7 +1095,7 @@ static int __init dt_cpu_ftrs_scan_callback(unsigned long node, const char
 
 	cpufeatures_setup_finished();
 
-	memblock_free(__pa(dt_cpu_features),
+	memblock_free(dt_cpu_features,
 			sizeof(struct dt_cpu_feature)*nr_dt_cpu_features);
 
 	return 0;
diff --git a/arch/powerpc/kernel/paca.c b/arch/powerpc/kernel/paca.c
index 9bd30cac852b..4e3bde0e0679 100644
--- a/arch/powerpc/kernel/paca.c
+++ b/arch/powerpc/kernel/paca.c
@@ -322,7 +322,7 @@ void __init free_unused_pacas(void)
 
 	new_ptrs_size = sizeof(struct paca_struct *) * nr_cpu_ids;
 	if (new_ptrs_size < paca_ptrs_size)
-		memblock_free(__pa(paca_ptrs) + new_ptrs_size,
+		memblock_free(paca_ptrs + new_ptrs_size,
 					paca_ptrs_size - new_ptrs_size);
 
 	paca_nr_cpu_ids = nr_cpu_ids;
@@ -331,7 +331,7 @@ void __init free_unused_pacas(void)
 #ifdef CONFIG_PPC_BOOK3S_64
 	if (early_radix_enabled()) {
 		/* Ugly fixup, see new_slb_shadow() */
-		memblock_free(__pa(paca_ptrs[boot_cpuid]->slb_shadow_ptr),
+		memblock_free(paca_ptrs[boot_cpuid]->slb_shadow_ptr,
 				sizeof(struct slb_shadow));
 		paca_ptrs[boot_cpuid]->slb_shadow_ptr = NULL;
 	}
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index b1e43b69a559..6b1338db8779 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -825,7 +825,7 @@ static void __init smp_setup_pacas(void)
 		set_hard_smp_processor_id(cpu, cpu_to_phys_id[cpu]);
 	}
 
-	memblock_free(__pa(cpu_to_phys_id), nr_cpu_ids * sizeof(u32));
+	memblock_free(cpu_to_phys_id, nr_cpu_ids * sizeof(u32));
 	cpu_to_phys_id = NULL;
 }
 #endif
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index eaa79a0996d1..1777e992b20b 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -812,7 +812,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 static int pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 3dd35c327d1c..004cd6a96c8a 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2981,7 +2981,7 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 	if (!phb->hose) {
 		pr_err("  Can't allocate PCI controller for %pOF\n",
 		       np);
-		memblock_free(__pa(phb), sizeof(struct pnv_phb));
+		memblock_free(phb, sizeof(struct pnv_phb));
 		return;
 	}
 
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 87f001b4c4e4..aa79a96a2749 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -55,9 +55,7 @@ void __init svm_swiotlb_init(void)
 	if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
 		return;
 
-
-	memblock_free_early(__pa(vstart),
-			    PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+	memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
 	panic("SVM: Cannot allocate SWIOTLB buffer");
 }
 
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index b9620e5f00ba..b42bfdc67482 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -230,13 +230,13 @@ static void __init init_resources(void)
 
 	/* Clean-up any unused pre-allocated resources */
 	if (res_idx >= 0)
-		memblock_free(__pa(mem_res), (res_idx + 1) * sizeof(*mem_res));
+		memblock_free(mem_res, (res_idx + 1) * sizeof(*mem_res));
 	return;
 
  error:
 	/* Better an empty resource tree than an inconsistent one */
 	release_child_resources(&iomem_resource);
-	memblock_free(__pa(mem_res), mem_res_sz);
+	memblock_free(mem_res, mem_res_sz);
 }
 
 
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 67e5fff96ee0..d09895bc66cc 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -693,7 +693,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	if (register_memory_notifier(&kdump_mem_nb)) {
-		memblock_free(crash_base, crash_size);
+		memblock_phys_free(crash_base, crash_size);
 		return;
 	}
 
@@ -748,7 +748,7 @@ static void __init free_mem_detect_info(void)
 
 	get_mem_detect_reserved(&start, &size);
 	if (size)
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 }
 
 static const char * __init get_mem_info_source(void)
@@ -793,7 +793,7 @@ static void __init check_initrd(void)
 	if (initrd_data.start && initrd_data.size &&
 	    !memblock_is_region_memory(initrd_data.start, initrd_data.size)) {
 		pr_err("The initial RAM disk does not fit into the memory\n");
-		memblock_free(initrd_data.start, initrd_data.size);
+		memblock_phys_free(initrd_data.start, initrd_data.size);
 		initrd_start = initrd_end = 0;
 	}
 #endif
@@ -890,7 +890,7 @@ static void __init setup_randomness(void)
 
 	if (stsi(vmms, 3, 2, 2) == 0 && vmms->count)
 		add_device_randomness(&vmms->vm, sizeof(vmms->vm[0]) * vmms->count);
-	memblock_free((unsigned long) vmms, PAGE_SIZE);
+	memblock_phys_free((unsigned long) vmms, PAGE_SIZE);
 }
 
 /*
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 1a04e5bdf655..37826d8c4f74 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
 			/* Get the CPU registers */
 			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
 	}
-	memblock_free(page, PAGE_SIZE);
+	memblock_phys_free(page, PAGE_SIZE);
 	diag_amode31_ops.diag308_reset();
 	pcpu_set_smt(0);
 }
@@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
 
 	/* Add CPUs present at boot */
 	__smp_rescan_cpus(info, true);
-	memblock_free_early((unsigned long)info, sizeof(*info));
+	memblock_free(info, sizeof(*info));
 }
 
 /*
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 5a656c7b7a67..d57457b16fe5 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -64,7 +64,7 @@ void __init setup_uv(void)
 	}
 
 	if (uv_init(uv_stor_base, uv_info.uv_base_stor_len)) {
-		memblock_free(uv_stor_base, uv_info.uv_base_stor_len);
+		memblock_phys_free(uv_stor_base, uv_info.uv_base_stor_len);
 		goto fail;
 	}
 
diff --git a/arch/s390/mm/kasan_init.c b/arch/s390/mm/kasan_init.c
index 3e4735168019..483b9dbe0970 100644
--- a/arch/s390/mm/kasan_init.c
+++ b/arch/s390/mm/kasan_init.c
@@ -399,5 +399,5 @@ void __init kasan_copy_shadow_mapping(void)
 
 void __init kasan_free_early_identity(void)
 {
-	memblock_free(pgalloc_pos, pgalloc_freeable - pgalloc_pos);
+	memblock_phys_free(pgalloc_pos, pgalloc_freeable - pgalloc_pos);
 }
diff --git a/arch/sh/boards/mach-ap325rxa/setup.c b/arch/sh/boards/mach-ap325rxa/setup.c
index bac8a058ebd7..c77b5f00a66a 100644
--- a/arch/sh/boards/mach-ap325rxa/setup.c
+++ b/arch/sh/boards/mach-ap325rxa/setup.c
@@ -560,7 +560,7 @@ static void __init ap325rxa_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-ecovec24/setup.c b/arch/sh/boards/mach-ecovec24/setup.c
index bab91a99124e..2b22ce792147 100644
--- a/arch/sh/boards/mach-ecovec24/setup.c
+++ b/arch/sh/boards/mach-ecovec24/setup.c
@@ -1502,7 +1502,7 @@ static void __init ecovec_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU0 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu0_dma_membase = phys;
 
@@ -1510,7 +1510,7 @@ static void __init ecovec_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU1 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu1_dma_membase = phys;
 }
diff --git a/arch/sh/boards/mach-kfr2r09/setup.c b/arch/sh/boards/mach-kfr2r09/setup.c
index eeb5ce341efd..20f4db778ed6 100644
--- a/arch/sh/boards/mach-kfr2r09/setup.c
+++ b/arch/sh/boards/mach-kfr2r09/setup.c
@@ -633,7 +633,7 @@ static void __init kfr2r09_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-migor/setup.c b/arch/sh/boards/mach-migor/setup.c
index 6703a2122c0d..f60061283c48 100644
--- a/arch/sh/boards/mach-migor/setup.c
+++ b/arch/sh/boards/mach-migor/setup.c
@@ -633,7 +633,7 @@ static void __init migor_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-se/7724/setup.c b/arch/sh/boards/mach-se/7724/setup.c
index 8d6541ba0186..8bbf5a6aa423 100644
--- a/arch/sh/boards/mach-se/7724/setup.c
+++ b/arch/sh/boards/mach-se/7724/setup.c
@@ -966,7 +966,7 @@ static void __init ms7724se_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU0 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu0_dma_membase = phys;
 
@@ -974,7 +974,7 @@ static void __init ms7724se_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU1 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu1_dma_membase = phys;
 }
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 0224d8f19ed6..b98a7bbe6728 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -1567,7 +1567,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 static int __init pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c
index 8e636ce02949..0039771eb01c 100644
--- a/arch/um/kernel/mem.c
+++ b/arch/um/kernel/mem.c
@@ -47,7 +47,7 @@ void __init mem_init(void)
 	 */
 	brk_end = (unsigned long) UML_ROUND_UP(sbrk(0));
 	map_memory(brk_end, __pa(brk_end), uml_reserved - brk_end, 1, 1, 0);
-	memblock_free(__pa(brk_end), uml_reserved - brk_end);
+	memblock_free((void *)brk_end, uml_reserved - brk_end);
 	uml_reserved = brk_end;
 
 	/* this will put all low memory onto the freelists */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 79f164141116..1bbe9b6173ef 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -322,7 +322,7 @@ static void __init reserve_initrd(void)
 
 	relocate_initrd();
 
-	memblock_free(ramdisk_image, ramdisk_end - ramdisk_image);
+	memblock_phys_free(ramdisk_image, ramdisk_end - ramdisk_image);
 }
 
 #else
@@ -521,7 +521,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
-		memblock_free(crash_base, crash_size);
+		memblock_phys_free(crash_base, crash_size);
 		return;
 	}
 
diff --git a/arch/x86/kernel/setup_percpu.c b/arch/x86/kernel/setup_percpu.c
index 5afd98559193..7b65275544b2 100644
--- a/arch/x86/kernel/setup_percpu.c
+++ b/arch/x86/kernel/setup_percpu.c
@@ -135,7 +135,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size, size_t align)
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_ptr(ptr, size);
+	memblock_free(ptr, size);
 }
 
 static int __init pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 23a14d82e783..1895986842b9 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -618,7 +618,7 @@ static void __init memory_map_top_down(unsigned long map_start,
 	 */
 	addr = memblock_phys_alloc_range(PMD_SIZE, PMD_SIZE, map_start,
 					 map_end);
-	memblock_free(addr, PMD_SIZE);
+	memblock_phys_free(addr, PMD_SIZE);
 	real_end = addr + PMD_SIZE;
 
 	/* step_size need to be small so pgt_buf from BRK could cover it */
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index ef885370719a..e7b9b464a82f 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -49,7 +49,7 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr,
 			p = early_alloc(PMD_SIZE, nid, false);
 			if (p && pmd_set_huge(pmd, __pa(p), PAGE_KERNEL))
 				return;
-			memblock_free_ptr(p, PMD_SIZE);
+			memblock_free(p, PMD_SIZE);
 		}
 
 		p = early_alloc(PAGE_SIZE, nid, true);
@@ -85,7 +85,7 @@ static void __init kasan_populate_pud(pud_t *pud, unsigned long addr,
 			p = early_alloc(PUD_SIZE, nid, false);
 			if (p && pud_set_huge(pud, __pa(p), PAGE_KERNEL))
 				return;
-			memblock_free_ptr(p, PUD_SIZE);
+			memblock_free(p, PUD_SIZE);
 		}
 
 		p = early_alloc(PAGE_SIZE, nid, true);
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index 1e9b93b088db..c6b1213086d6 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -355,7 +355,7 @@ void __init numa_reset_distance(void)
 
 	/* numa_distance could be 1LU marking allocation failure, test cnt */
 	if (numa_distance_cnt)
-		memblock_free_ptr(numa_distance, size);
+		memblock_free(numa_distance, size);
 	numa_distance_cnt = 0;
 	numa_distance = NULL;	/* enable table creation */
 }
diff --git a/arch/x86/mm/numa_emulation.c b/arch/x86/mm/numa_emulation.c
index e801e30089c4..1a02b791d273 100644
--- a/arch/x86/mm/numa_emulation.c
+++ b/arch/x86/mm/numa_emulation.c
@@ -517,7 +517,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 	}
 
 	/* free the copied physical distance table */
-	memblock_free_ptr(phys_dist, phys_size);
+	memblock_free(phys_dist, phys_size);
 	return;
 
 no_emu:
diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 8d751939c6f3..0c831ad78933 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1025,7 +1025,7 @@ static void __init xen_free_ro_pages(unsigned long paddr, unsigned long size)
 	for (; vaddr < vaddr_end; vaddr += PAGE_SIZE)
 		make_lowmem_page_readwrite(vaddr);
 
-	memblock_free(paddr, size);
+	memblock_phys_free(paddr, size);
 }
 
 static void __init xen_cleanmfnmap_free_pgtbl(void *pgtbl, bool unpin)
@@ -1151,7 +1151,7 @@ static void __init xen_pagetable_p2m_free(void)
 		xen_cleanhighmap(addr, addr + size);
 		size = PAGE_ALIGN(xen_start_info->nr_pages *
 				  sizeof(unsigned long));
-		memblock_free(__pa(addr), size);
+		memblock_free((void *)addr, size);
 	} else {
 		xen_cleanmfnmap(addr);
 	}
@@ -1955,7 +1955,7 @@ void __init xen_relocate_p2m(void)
 		pfn_end = p2m_pfn_end;
 	}
 
-	memblock_free(PFN_PHYS(pfn), PAGE_SIZE * (pfn_end - pfn));
+	memblock_phys_free(PFN_PHYS(pfn), PAGE_SIZE * (pfn_end - pfn));
 	while (pfn < pfn_end) {
 		if (pfn == p2m_pfn) {
 			pfn = p2m_pfn_end;
diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index 141bb9dbd2fb..58db86f7b384 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -197,7 +197,7 @@ static void * __ref alloc_p2m_page(void)
 static void __ref free_p2m_page(void *p)
 {
 	if (unlikely(!slab_is_available())) {
-		memblock_free_ptr(p, PAGE_SIZE);
+		memblock_free(p, PAGE_SIZE);
 		return;
 	}
 
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 8bfc10330107..f387fc7e5250 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -153,7 +153,7 @@ static void __init xen_del_extra_mem(unsigned long start_pfn,
 			break;
 		}
 	}
-	memblock_free(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
+	memblock_phys_free(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
 }
 
 /*
@@ -719,7 +719,7 @@ static void __init xen_reserve_xen_mfnlist(void)
 		return;
 
 	xen_relocate_p2m();
-	memblock_free(start, size);
+	memblock_phys_free(start, size);
 }
 
 /**
@@ -885,7 +885,7 @@ char * __init xen_memory_setup(void)
 		xen_phys_memcpy(new_area, start, size);
 		pr_info("initrd moved from [mem %#010llx-%#010llx] to [mem %#010llx-%#010llx]\n",
 			start, start + size, new_area, new_area + size);
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 		boot_params.hdr.ramdisk_image = new_area;
 		boot_params.ext_ramdisk_image = new_area >> 32;
 	}
diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index f6d0efd01188..cad34cec3acc 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -165,7 +165,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
@@ -264,7 +264,7 @@ void __init numa_free_distance(void)
 	size = numa_distance_cnt * numa_distance_cnt *
 		sizeof(numa_distance[0]);
 
-	memblock_free_ptr(numa_distance, size);
+	memblock_free(numa_distance, size);
 	numa_distance_cnt = 0;
 	numa_distance = NULL;
 }
diff --git a/drivers/firmware/efi/memmap.c b/drivers/firmware/efi/memmap.c
index 2ff1883dc788..4df55a55da84 100644
--- a/drivers/firmware/efi/memmap.c
+++ b/drivers/firmware/efi/memmap.c
@@ -35,7 +35,7 @@ void __init __efi_memmap_free(u64 phys, unsigned long size, unsigned long flags)
 		if (slab_is_available())
 			memblock_free_late(phys, size);
 		else
-			memblock_free(phys, size);
+			memblock_phys_free(phys, size);
 	} else if (flags & EFI_MEMMAP_SLAB) {
 		struct page *p = pfn_to_page(PHYS_PFN(phys));
 		unsigned int order = get_order(size);
diff --git a/drivers/macintosh/smu.c b/drivers/macintosh/smu.c
index fe63d5ee201b..f62152111236 100644
--- a/drivers/macintosh/smu.c
+++ b/drivers/macintosh/smu.c
@@ -570,7 +570,7 @@ int __init smu_init (void)
 fail_db_node:
 	of_node_put(smu->db_node);
 fail_bootmem:
-	memblock_free_ptr(smu, sizeof(struct smu_device));
+	memblock_free(smu, sizeof(struct smu_device));
 	smu = NULL;
 fail_np:
 	of_node_put(np);
diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
index 761fd870d1db..8d07f73a4389 100644
--- a/drivers/of/kexec.c
+++ b/drivers/of/kexec.c
@@ -170,7 +170,7 @@ int ima_free_kexec_buffer(void)
 	if (ret)
 		return ret;
 
-	return memblock_free(addr, size);
+	return memblock_phys_free(addr, size);
 
 }
 
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index 59c1390cdf42..c48d0dce13eb 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -45,7 +45,7 @@ static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size,
 	if (nomap) {
 		err = memblock_mark_nomap(base, size);
 		if (err)
-			memblock_free(base, size);
+			memblock_phys_free(base, size);
 	}
 
 	return err;
@@ -282,7 +282,7 @@ void __init fdt_init_reserved_mem(void)
 				if (nomap)
 					memblock_clear_nomap(rmem->base, rmem->size);
 				else
-					memblock_free(rmem->base, rmem->size);
+					memblock_phys_free(rmem->base, rmem->size);
 			}
 		}
 	}
diff --git a/drivers/s390/char/sclp_early.c b/drivers/s390/char/sclp_early.c
index f3d5c7f4c13d..98247a4a64b7 100644
--- a/drivers/s390/char/sclp_early.c
+++ b/drivers/s390/char/sclp_early.c
@@ -139,7 +139,7 @@ int __init sclp_early_get_core_info(struct sclp_core_info *info)
 	}
 	sclp_fill_core_info(info, sccb);
 out:
-	memblock_free_early((unsigned long)sccb, length);
+	memblock_free(sccb, length);
 	return rc;
 }
 
diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index be4ecbabdd58..933d77ad0a64 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -185,7 +185,7 @@ static void __init xdbc_free_ring(struct xdbc_ring *ring)
 	if (!seg)
 		return;
 
-	memblock_free(seg->dma, PAGE_SIZE);
+	memblock_phys_free(seg->dma, PAGE_SIZE);
 	ring->segment = NULL;
 }
 
@@ -665,10 +665,10 @@ int __init early_xdbc_setup_hardware(void)
 		xdbc_free_ring(&xdbc.in_ring);
 
 		if (xdbc.table_dma)
-			memblock_free(xdbc.table_dma, PAGE_SIZE);
+			memblock_phys_free(xdbc.table_dma, PAGE_SIZE);
 
 		if (xdbc.out_dma)
-			memblock_free(xdbc.out_dma, PAGE_SIZE);
+			memblock_phys_free(xdbc.out_dma, PAGE_SIZE);
 
 		xdbc.table_base = NULL;
 		xdbc.out_buf = NULL;
@@ -987,8 +987,8 @@ static int __init xdbc_init(void)
 	xdbc_free_ring(&xdbc.evt_ring);
 	xdbc_free_ring(&xdbc.out_ring);
 	xdbc_free_ring(&xdbc.in_ring);
-	memblock_free(xdbc.table_dma, PAGE_SIZE);
-	memblock_free(xdbc.out_dma, PAGE_SIZE);
+	memblock_phys_free(xdbc.table_dma, PAGE_SIZE);
+	memblock_phys_free(xdbc.out_dma, PAGE_SIZE);
 	writel(0, &xdbc.xdbc_reg->control);
 	early_iounmap(xdbc.xhci_base, xdbc.xhci_length);
 
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 8c10edf9efe6..bad266ea3bcd 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -240,7 +240,7 @@ void __init xen_swiotlb_init_early(void)
 	 */
 	rc = xen_swiotlb_fixup(start, nslabs);
 	if (rc) {
-		memblock_free(__pa(start), PAGE_ALIGN(bytes));
+		memblock_free(start, PAGE_ALIGN(bytes));
 		if (nslabs > 1024 && repeat--) {
 			/* Min is 2MB */
 			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 34de69b3b8ba..e1f73b81d595 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -103,7 +103,7 @@ void memblock_allow_resize(void);
 int memblock_add_node(phys_addr_t base, phys_addr_t size, int nid);
 int memblock_add(phys_addr_t base, phys_addr_t size);
 int memblock_remove(phys_addr_t base, phys_addr_t size);
-int memblock_free(phys_addr_t base, phys_addr_t size);
+int memblock_phys_free(phys_addr_t base, phys_addr_t size);
 int memblock_reserve(phys_addr_t base, phys_addr_t size);
 #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
 int memblock_physmem_add(phys_addr_t base, phys_addr_t size);
@@ -117,8 +117,8 @@ int memblock_mark_mirror(phys_addr_t base, phys_addr_t size);
 int memblock_mark_nomap(phys_addr_t base, phys_addr_t size);
 int memblock_clear_nomap(phys_addr_t base, phys_addr_t size);
 
+void memblock_free(void *ptr, size_t size);
 void memblock_free_all(void);
-void memblock_free_ptr(void *ptr, size_t size);
 void reset_node_managed_pages(pg_data_t *pgdat);
 void reset_all_zones_managed_pages(void);
 
@@ -441,18 +441,6 @@ static inline void *memblock_alloc_node(phys_addr_t size,
 				      MEMBLOCK_ALLOC_ACCESSIBLE, nid);
 }
 
-static inline void memblock_free_early(phys_addr_t base,
-					      phys_addr_t size)
-{
-	memblock_free(base, size);
-}
-
-static inline void memblock_free_early_nid(phys_addr_t base,
-						  phys_addr_t size, int nid)
-{
-	memblock_free(base, size);
-}
-
 static inline void memblock_free_late(phys_addr_t base, phys_addr_t size)
 {
 	__memblock_free_late(base, size);
diff --git a/init/initramfs.c b/init/initramfs.c
index a842c0544745..c48748217c28 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -607,7 +607,7 @@ void __weak __init free_initrd_mem(unsigned long start, unsigned long end)
 	unsigned long aligned_start = ALIGN_DOWN(start, PAGE_SIZE);
 	unsigned long aligned_end = ALIGN(end, PAGE_SIZE);
 
-	memblock_free(__pa(aligned_start), aligned_end - aligned_start);
+	memblock_free((void*)aligned_start, aligned_end - aligned_start);
 #endif
 
 	free_reserved_area((void *)start, (void *)end, POISON_FREE_INITMEM,
diff --git a/init/main.c b/init/main.c
index 3f7216934441..a0714600a83f 100644
--- a/init/main.c
+++ b/init/main.c
@@ -924,7 +924,7 @@ static void __init print_unknown_bootoptions(void)
 		end += sprintf(end, " %s", *p);
 
 	pr_notice("Unknown command line parameters:%s\n", unknown_options);
-	memblock_free_ptr(unknown_options, len);
+	memblock_free(unknown_options, len);
 }
 
 asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 87c40517e822..02656d7ccbfd 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -247,7 +247,7 @@ swiotlb_init(int verbose)
 	return;
 
 fail_free_mem:
-	memblock_free_early(__pa(tlb), bytes);
+	memblock_free(tlb, bytes);
 fail:
 	pr_warn("Cannot allocate buffer");
 }
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index a8d0a58deebc..2cae1bfa6be7 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1166,9 +1166,9 @@ void __init setup_log_buf(int early)
 	return;
 
 err_free_descs:
-	memblock_free_ptr(new_descs, new_descs_size);
+	memblock_free(new_descs, new_descs_size);
 err_free_log_buf:
-	memblock_free_ptr(new_log_buf, new_log_buf_len);
+	memblock_free(new_log_buf, new_log_buf_len);
 }
 
 static bool __read_mostly ignore_loglevel;
diff --git a/lib/bootconfig.c b/lib/bootconfig.c
index 5ae248b29373..547558d80e64 100644
--- a/lib/bootconfig.c
+++ b/lib/bootconfig.c
@@ -792,7 +792,7 @@ void __init xbc_destroy_all(void)
 	xbc_data = NULL;
 	xbc_data_size = 0;
 	xbc_node_num = 0;
-	memblock_free_ptr(xbc_nodes, sizeof(struct xbc_node) * XBC_NODE_MAX);
+	memblock_free(xbc_nodes, sizeof(struct xbc_node) * XBC_NODE_MAX);
 	xbc_nodes = NULL;
 	brace_index = 0;
 }
diff --git a/lib/cpumask.c b/lib/cpumask.c
index c3c76b833384..a971a82d2f43 100644
--- a/lib/cpumask.c
+++ b/lib/cpumask.c
@@ -188,7 +188,7 @@ EXPORT_SYMBOL(free_cpumask_var);
  */
 void __init free_bootmem_cpumask_var(cpumask_var_t mask)
 {
-	memblock_free_early(__pa(mask), cpumask_size());
+	memblock_free(mask, cpumask_size());
 }
 #endif
 
diff --git a/mm/cma.c b/mm/cma.c
index 995e15480937..f9a53bc708f2 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -378,7 +378,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base,
 	return 0;
 
 free_mem:
-	memblock_free(base, size);
+	memblock_phys_free(base, size);
 err:
 	pr_err("Failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
 	return ret;
diff --git a/mm/memblock.c b/mm/memblock.c
index 184dcd2e5d99..affcfdda0ff8 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -472,7 +472,7 @@ static int __init_memblock memblock_double_array(struct memblock_type *type,
 		kfree(old_array);
 	else if (old_array != memblock_memory_init_regions &&
 		 old_array != memblock_reserved_init_regions)
-		memblock_free_ptr(old_array, old_alloc_size);
+		memblock_free(old_array, old_alloc_size);
 
 	/*
 	 * Reserve the new array if that comes from the memblock.  Otherwise, we
@@ -796,28 +796,28 @@ int __init_memblock memblock_remove(phys_addr_t base, phys_addr_t size)
 }
 
 /**
- * memblock_free_ptr - free boot memory allocation
- * @ptr: starting address of the  boot memory allocation
+ * memblock_free - free boot memory allocation
+ * @ptr: starting *virtual* address of the  boot memory allocation
  * @size: size of the boot memory block in bytes
  *
  * Free boot memory block previously allocated by memblock_alloc_xx() API.
  * The freeing memory will not be released to the buddy allocator.
  */
-void __init_memblock memblock_free_ptr(void *ptr, size_t size)
+void __init_memblock memblock_free(void *ptr, size_t size)
 {
 	if (ptr)
-		memblock_free(__pa(ptr), size);
+		memblock_phys_free(__pa(ptr), size);
 }
 
 /**
- * memblock_free - free boot memory block
- * @base: phys starting address of the  boot memory block
+ * memblock_free_phys - free boot memory block
+ * @base: starting *physical* address of the  boot memory block
  * @size: size of the boot memory block in bytes
  *
- * Free boot memory block previously allocated by memblock_alloc_xx() API.
+ * Free boot memory block previously allocated by memblock_phys_alloc_xx() API.
  * The freeing memory will not be released to the buddy allocator.
  */
-int __init_memblock memblock_free(phys_addr_t base, phys_addr_t size)
+int __init_memblock memblock_phys_free(phys_addr_t base, phys_addr_t size)
 {
 	phys_addr_t end = base + size - 1;
 
@@ -1934,7 +1934,7 @@ static void __init free_memmap(unsigned long start_pfn, unsigned long end_pfn)
 	 * memmap array.
 	 */
 	if (pg < pgend)
-		memblock_free(pg, pgend - pg);
+		memblock_phys_free(pg, pgend - pg);
 }
 
 /*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 9fd0be32a281..feffaa9423fe 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -2204,7 +2204,7 @@ static int __ref try_remove_memory(u64 start, u64 size)
 	arch_remove_memory(start, size, altmap);
 
 	if (IS_ENABLED(CONFIG_ARCH_KEEP_MEMBLOCK)) {
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 		memblock_remove(start, size);
 	}
 
diff --git a/mm/percpu.c b/mm/percpu.c
index e0a986818903..f5b2c2ea5a54 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -2472,7 +2472,7 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
  */
 void __init pcpu_free_alloc_info(struct pcpu_alloc_info *ai)
 {
-	memblock_free_early(__pa(ai), ai->__ai_size);
+	memblock_free(ai, ai->__ai_size);
 }
 
 /**
@@ -3134,7 +3134,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 out_free:
 	pcpu_free_alloc_info(ai);
 	if (areas)
-		memblock_free_early(__pa(areas), areas_size);
+		memblock_free(areas, areas_size);
 	return rc;
 }
 #endif /* BUILD_EMBED_FIRST_CHUNK */
@@ -3256,7 +3256,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size,
 		free_fn(page_address(pages[j]), PAGE_SIZE);
 	rc = -ENOMEM;
 out_free_ar:
-	memblock_free_early(__pa(pages), pages_size);
+	memblock_free(pages, pages_size);
 	pcpu_free_alloc_info(ai);
 	return rc;
 }
@@ -3286,7 +3286,7 @@ static void * __init pcpu_dfl_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_dfl_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/mm/sparse.c b/mm/sparse.c
index 120bc8ea5293..e5c84b0cf0c9 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -451,7 +451,7 @@ static void *sparsemap_buf_end __meminitdata;
 static inline void __meminit sparse_buffer_free(unsigned long size)
 {
 	WARN_ON(!sparsemap_buf || size == 0);
-	memblock_free_early(__pa(sparsemap_buf), size);
+	memblock_free(sparsemap_buf, size);
 }
 
 static void __init sparse_buffer_init(unsigned long size, int nid)
diff --git a/tools/bootconfig/include/linux/memblock.h b/tools/bootconfig/include/linux/memblock.h
index f2e506f7d57f..e6ea42e97b22 100644
--- a/tools/bootconfig/include/linux/memblock.h
+++ b/tools/bootconfig/include/linux/memblock.h
@@ -6,6 +6,6 @@
 
 #define SMP_CACHE_BYTES	0
 #define memblock_alloc(size, align)	malloc(size)
-#define memblock_free_ptr(paddr, size)	free(paddr)
+#define memblock_free(paddr, size)	free(paddr)
 
 #endif
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:50:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:50:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193466.344593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJV5-00080F-SY; Thu, 23 Sep 2021 07:50:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193466.344593; Thu, 23 Sep 2021 07:50:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJV5-000808-Pc; Thu, 23 Sep 2021 07:50:35 +0000
Received: by outflank-mailman (input) for mailman id 193466;
 Thu, 23 Sep 2021 07:50:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTJV4-0007zy-If; Thu, 23 Sep 2021 07:50:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTJV4-00020k-De; Thu, 23 Sep 2021 07:50:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTJV4-0005XC-1Z; Thu, 23 Sep 2021 07:50:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTJV4-0002yf-11; Thu, 23 Sep 2021 07:50:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OH5yYqedgoiUL7WnQRlRc0c7TxyObcr8+6PuKs4JSbg=; b=P6UlAPV9SMtCM0DkMZ9/m2h3r4
	7EHnKefhzo3C1NA+WaYaBVDB97SAS+j1TecX5gwYu79UbXIjh7Zs7Hulg+ySjgKMvsnv5j9NaWZU7
	vTXSO7vNE04OmZxzfIkdWWoHHtSyiouupP+X6GjwlvON83fMUUeHv8r44VWwiOXX7QBw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165158-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165158: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=cf1d2c3e7e2f3754fe3d6dc747f7a092b168d9cf
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 07:50:34 +0000

flight 165158 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165158/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                cf1d2c3e7e2f3754fe3d6dc747f7a092b168d9cf
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  418 days
Failing since        152366  2020-08-01 20:49:34 Z  417 days  729 attempts
Testing same since   165158  2021-09-22 17:10:57 Z    0 days    1 attempts

------------------------------------------------------------
7324 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263826 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 07:56:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 07:56:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193472.344607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJaq-0000Ek-JO; Thu, 23 Sep 2021 07:56:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193472.344607; Thu, 23 Sep 2021 07:56:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJaq-0000Ec-GX; Thu, 23 Sep 2021 07:56:32 +0000
Received: by outflank-mailman (input) for mailman id 193472;
 Thu, 23 Sep 2021 07:56:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTJap-0000EW-D0
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 07:56:31 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTJan-00026N-Kl; Thu, 23 Sep 2021 07:56:29 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTJan-0002Aa-6o; Thu, 23 Sep 2021 07:56:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=8HIs2uBNK86v6gFgWlwquG5MQ2VlUICiFiKJtauEd+k=; b=EbiWMrgp1O/byjaI2DCvp+aAF4
	/QyXUCSyZB7itIV2nC/hXewDfR5JnjJV+PXJyuPgWaClttR+5Gj5/HdhsEEwxx9PW4VDPbiyGkBBw
	2DSYKnLg/ojJvi7c1InCfnw7NkT7T488yj6j8tdsvXLaKOG6ncQ1zOFvIp/zJH/uGV64=;
Subject: Re: [PATCH v2 5/6] tools/xenstored: partially handle domains without
 a shared ring
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 raphning@amazon.co.uk, "Doebel, Bjoern" <doebel@amazon.de>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-6-roger.pau@citrix.com>
 <0aed8667-7f31-b0fb-3358-c5fd9a5734a1@xen.org>
 <YUr+K0tg6mspyOyd@MacBook-Air-de-Roger.local>
 <69285850-4168-b30c-db47-8c0649bd1ffc@xen.org>
 <09c4bad1-99e3-5d24-873e-4aca9acd6fed@suse.com>
 <56bb55e3-ca7a-12e6-5848-84756b431846@xen.org>
 <YUwrZS3FOh+hCQle@MacBook-Air-de-Roger.local>
From: Julien Grall <julien@xen.org>
Message-ID: <041bbb31-700f-0ebd-d255-47a4aec927d8@xen.org>
Date: Thu, 23 Sep 2021 12:56:22 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YUwrZS3FOh+hCQle@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 23/09/2021 12:23, Roger Pau Monné wrote:
> On Wed, Sep 22, 2021 at 06:46:25PM +0500, Julien Grall wrote:
>> I thought a bit more and looked at the code (I don't have access to a test
>> machine at the moment). I think there is indeed a problem.
>>
>> Some watchers of @releaseDomain (such as xenconsoled) will only remove a
>> domain from their internal state when the domain is actually dead.
>>
>> This is based on dominfo.dying which is only set when all the resources are
>> relinquished and waiting for the other domains to release any resources for
>> that domain.
>>
>> The problem is Xenstore may fail to map the interface or the event channel
>> long before the domain is actually dead. With the current check, we would
>> free the allocated structure and therefore send @releaseDomain too early. So
>> daemon like xenconsoled, would never cleanup for the domain and leave a
>> zombie domain. Well... until the next @releaseDomain (or @introduceDomain
>> for Xenconsoled) AFAICT.
>>
>> The revised patch is meant to solve it by just ignoring the connection. With
>> that approach, we would correctly notify watches when the domain is dead.
> 
> I think the patch provided by Julien is indeed better than the current
> code, or else you will miss @releaseDomain events in corner cases for
> dominfo.dying.
> 
> I think however the patch is missing a change in the order of the
> checks in conn_can_{read,write}, so that the is_ignored check is
> performed before calling can_{read,write} which will try to poke at
> the interface and trigger a fault if not mapped.

Ah good point. I originally moved the is_ignored check after the 
callback because the socket connection can in theory be closed from 
can_{read, write}.

However, in pratice, is_ignored is only set for socket from 
ignore_connection() that will also close the socket.

The new use will only set is_ignored for the domain connection. So I am 
guessing moving the check early on ought to be fine.

The alternative would be to call ignore_connection() but this feels a 
bit weird because most of it would be a NOP as we are introducing the 
domain.

Cheers,


-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:10:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:10:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193480.344618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJnn-0002lF-17; Thu, 23 Sep 2021 08:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193480.344618; Thu, 23 Sep 2021 08:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJnm-0002l8-Tg; Thu, 23 Sep 2021 08:09:54 +0000
Received: by outflank-mailman (input) for mailman id 193480;
 Thu, 23 Sep 2021 08:09:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTJnm-0002l2-Hh
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:09:54 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a17c69ec-1c45-11ec-ba1b-12813bfff9fa;
 Thu, 23 Sep 2021 08:09:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a17c69ec-1c45-11ec-ba1b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632384593;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=Ydpw2L3QYM2AIhLc1pzhasfeQz66ZGYQKxHkWjzM8FA=;
  b=SVgnbpjY+20WzW0pO7ZJTBtr+dElQRELGeKouJ+U4z110NBHZIT3Py7W
   uGrwWsnqa0Sms/6lvPVlp9ggO+v5vTfO/WfBO01uNTBSjmuXPxxy0BXOF
   7JGGzlvAl/R+6yYINU3iu2wVyjoTFuM+k606xpBvgdQ6q/hD1ULaer8qj
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: +k7ryQpuop7DWC/lCqxNptHRWZGcu50pOP7z9X+/4+vEFMfy3WFh/IsemCPWKWTIzOFDSAadAV
 rDYF9n66BQsjeO4ukAPrU/YtGFdlEKt85y3qrMM5mxwhi5o19Bfq0VHENJcNSUWAeIYU3mzHY3
 Y6YHmHSmMShklMoD+daAXuPHD1Algo1ytUPmT3X8NkQhJuDZ07ZEQCIcsQzDpsgUQ53GRoq7+Y
 5NdT0eLEqljFRHeD+NmvdXK6pPyB5/17+Uk/tlkJ/A1WORAfnakMQ5S0qs0Y9Gm9HAJTDa/YHj
 5mOusDXC/kEfnFFDsOm9gC1W
X-SBRS: 5.1
X-MesageID: 53412025
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ZMGUVa7fn6arX6Pq8RaBPwxRtObAchMFZxGqfqrLsTDasY5as4F+v
 mtNDDjQPPiMZGejL95zO4/loxwHu5/cxtVrSgY4q3xjHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NUw2LBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z6
 t5S6syKRgUQAYbUxONeUj1nMAVnFPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWxs35oWR6u2i
 8wxVTtjbRrYYwV0GHQJAbcloKSnuyDUbGgNwL6SjfVuuDWCpOBr65DyNPLFd9rMQt9a9m6Iq
 2SD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQ3vxgjUCXx2cTIAYLTlb9qv684nNSQPoGd
 RZSoHB36/Fvqgr7FbERQiFUvlamtQFDC+BfEdQFwxnRzenVxzm6InEtG2sphMMdiOc6Qjkj1
 1msltzvBCByvLD9dU9x5ot4vhvpZnNIdzFqiTssCFJfuoi+/NhbYgfnE447eJNZmOEZDt0ZL
 9qilyE4m7xbpsoCzazTEbvv0m/0+8ChouLY4GzqsoOZAuFRONXNi2+AswGzARN8wGCxFALpU
 J8swZT20Qz2JcvR/BFhuc1UdF1T296LMSfHnXlkFIQ7+jKm9haLJN4LumolfB8ybJxVKFcFh
 XM/XysLv/e/21PwMcdKj3+ZUZx2ncAM6/y/PhwrUja+SscoL1LWlM2fTUWRw3rsgCARfVIXY
 /+mnTKXJS9CU8xPlWPuL89EiOND7n1ulAv7GMGgpzz6gOX2WZJgYepcWLd4Rrtit/3sTcS82
 4s3CvZmPD0FC7ChPnmLr9RDRb3IRFBiba3LRwVsXrfrCiJtGX07Cu+XxrUkeod/mL9SmPuO9
 Xa4MnK0AnKk7ZEeAQnVOH1ldp31WpNz8SAyMSA2ZA760Hk/e4e/qqwYcsJvL7Ug8eViy99yT
 uUEJJrcUqgeFGyf9mRPd4T5oaxjaA+v2VCEMR26bWVtZJVnXQHIpIPpJ1O96CkUAyOrnsIiu
 Ln8hBjDSJ8OSl06XsbbYf6i1X2run0ZlL4gVkfEOIALKk7t7JJrO2r6ifpue5MALhDKxz270
 QeKAEhH+bmR8tFtqNSQ3PKKtYakFed6D3F2JWiD4ObkLzTe80qi3ZREDLSCcwfCWT6m466lf
 +hUka3xaaVVgFZQvoNgOL931qZitcD3rrpXwwk4TnXGa1OnVuFpLnWchJQds6RMwvlSuBesW
 1LJ8d5fYO3bNMTgGV8XBQwkcuXciq1ExmiMtaw4cBfg+St63LubSkEDbRCDhRtUIKZxLI54k
 /wqv9Qb6lDnhxcnWjpcYvu4K4hYwqQ8bpga
IronPort-HdrOrdr: A9a23:kE/zuK76XJbwiFLZBgPXwVWBI+orL9Y04lQ7vn2ZFiY6TiXIra
 +TdaoguSMc6AxwZJkh8erwXpVoZUmsiKKdgLNhR4tKOTOGhILGFvAG0WKP+UyFJ8S6zJ8g6U
 4CSdkONDSTNykDsS+S2mDReLxMsbr3ktHZuQ6d9QYXcegDUdAf0+4TMHfjLqQZfnggOXJvf6
 Dsmfav6gDQNEg/X4CePD0oTuLDr9rEmNbPZgMHPQcu7E2rgSmz4LD3PhCE1lNGOgk/josKwC
 zgqUjU96+ju/a0xlv10HLS1Y1fnJ/ExsFYDMKBp8AJInHHixquZq5mR7qe1QpF7N2H2RIPqp
 3hsh0gN8N85zf4eXy0mwLk303a3DMn+xbZuBelqEqmhfa8aCMxCsJHi44cWADe8VAcsNZ117
 8O936FtrJMZCmw3BjV1pztbVVHh0C0qX0tnao4lHpES7YTb7dXsMg24F5VKpEdByj3gbpXUd
 WGNPuspsq+TGnqKkww5gJUsZiRtzUIb1m7q3E5y4+oO2M8pgE/86MarPZv10vouqhNDaWtSo
 z/Q9BVfYp1P70rhJRGdZE8qPuMex/wqC33QRevyHTcZeo60iH22tTKCItc3pDcRHVP9upqpK
 j8
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53412025"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hUUph1iSzCHqpYhBDmpGS8ISxDq9c2al3tZpOkxcBuYEA/bJKIZlA+pVuQSjWAy+uI9/AtmUYzfA19F7XC65AH2xRRVFe+xoGyi0vOPAQHkTuq6ViibA23kmcOCFxttYzGFJkbTj7f/nm03ZiP4CpnraDf5X3RvnLYC5zp8KpoDYrznDKT/9k0wWb1MRwHzI+IeLcxe6bHw8Rj9w1qC+AKSWpoz17IOA01re/0bBCIBj9Dxga0oa9K6FUumzkAOFXGfvYX4W+7cNRlZ5MV9lnKTfBUC0M4Z9jZ/gCaORvUj62a8jOkciI9F/m5JFCphNC13r7BpdOfG92AeA0igTYQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eBu4pcbXT29mhKOgwOyv523CUW4myLMsPAhueB0Ul0o=;
 b=NnHYa+2oN48WH9dZWvDA6OJTsfNMeiggYzPgCeOG5TdvBbEwo7NFBGtA2Ys4h2gHoh73SUX/65vqZrJZnYl4fdRG2XNB6HbyNa0+oTv+ImR9KMXOxYd8LZzqTVqBpBgw5Z/EH/+wNj8sbo4O0cEyw0EP0BgegR1jnusUJizDF0nwjBHnOlPJ9+xdmtRp5bF9mOJfAIhYhRcZhRCwNLB9xqpZa+ay3LvGvRp01nFs+rh0DRjekJ2Pf62rXf7qLCxtLmZj5YUT3d3FUwFoxXWuKkO0+IM1LRstufawXSKSbnJyr3Yf86s1CkNT7T8O5lqSwl10ysf6jAHSk5Xq9DT80Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eBu4pcbXT29mhKOgwOyv523CUW4myLMsPAhueB0Ul0o=;
 b=gqfEPIRwM6dmmTUPtDS72wxDkMbfvI/Sv405Z2mmx0XDysBcE1FFAh5PMj3QOf0O8lS/49jUlj2qrgc5F3SrNt6nGMbPgv/yIDrtLXfxlgac89n6EC+mTrYHWwMrFPLpgIp8We9COW5fcy8rqvSUcPwkoxmPL08qmpcC4wbvaXI=
Date: Thu, 23 Sep 2021 10:09:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 6/9] x86/HVM: convert hvm_virtual_to_linear_addr() to
 be remote-capable
Message-ID: <YUw2RLtI73QNjaPE@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <3397b5ea-bc21-5a2c-ba5d-afc974a92df4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <3397b5ea-bc21-5a2c-ba5d-afc974a92df4@suse.com>
X-ClientProxiedBy: LO4P123CA0309.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::8) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b43993ec-940f-418c-2395-08d97e6981ed
X-MS-TrafficTypeDiagnostic: DS7PR03MB5445:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB54458B13263CF5FB5B7F68188FA39@DS7PR03MB5445.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: be7NVAXHMjD5JPyGbSFVdIfZOYEwfJwNiIESUgHGq9gF3ASyg+JpSto2wF5uzmq/L3HLUbav9z9lkTI9gPcT9UZuOTPbfHIQPr8AhtGHGICWF6D3rxVmrXm4QhU5V59LkDEVFDk77ovFAhSlOfjjbwb4oW/s8Ielzh4urJRefdtUUh4gT4LkgbrGSRi5Lc/XvEQfOtc9Pvcbq45h8pOZwZO0g/g78NFRQSWDSNlIILFxZknHzkUK5cchExsdcKhK7A5ajYkR35WSc0kXCbejuxNhHVB1ycU2uwWKgaCOJwHN6eWYax6omWSaCaELOpl6eaUvnSA7ye2weKZfNj2Qk/Lt9gv+BAtAwCOLiPL24w7++03vFJKD3+FVLF50RZPaNmHTyivArGdoO7oRLGHDa87GPxtIE6pd5gREqAdstDODnEW7GcchezX9lDqnUe01tIs8vdAjB26U68fV1EeV/Qjth94ES3rEZdLxdpvgdYdlGRT3T2p6Ml9ZG2uMM0lxToxWNFoGI90da7I2jIF8VhcR6MdWVnjRcm51EgFCMWCZVv12W8BDtkXCg7SpEQvIXvgtJZhIZW7sk2d6OniEhdKHhbcIT6bg84yVjrRVxWEM5VsZryPwvUW/e4dy+cNn6H4+gx+NUfrnBsdg0QT+iQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(186003)(508600001)(6666004)(86362001)(38100700002)(26005)(83380400001)(85182001)(66946007)(66476007)(66556008)(8676002)(9686003)(4326008)(6916009)(6486002)(316002)(5660300002)(2906002)(54906003)(6496006)(8936002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cXc5TG94UHlPcFV6T0hha2lYcXJpQ29iR0hXTnFpc0hrWWV3eGVnVFFZUytm?=
 =?utf-8?B?MS9aZEk4VGZsWElEWDNLTXFPZWVTMXlmMVhLdHZnM3R0WlVnSDJaVEhGWU5z?=
 =?utf-8?B?U3VNWHBxSktWVEkxTzA2d3ZwakYwVXdRMHcxaTFRL0g5K1hyS2hrTFk0OE1I?=
 =?utf-8?B?eVRNMkFOSDU1QlRiK2JHVi9VbXBLell4eXFDazcwY3VvTlFyKzNDZHV4QURP?=
 =?utf-8?B?QlFVTDBockRabWVzQXo5MGQzYWh1QlhOMVlJQmIwN1d2YUVVU1FkUVMxb1Jw?=
 =?utf-8?B?YVFvNnVydmpFNWM3Ryt4clM0VGpiSVZOaFNMa0VTK2FTRjlBeGF6Wkx2RGJO?=
 =?utf-8?B?VjZwUHhLSTYwZXZIS2VXZTNUZ3NVd0ZuWW04T1A0ZXJIbmdtbkdvZGprem1F?=
 =?utf-8?B?VzNzLzZyaE8zc2R5QXVRTnk0R21BcXlCY04vbWFYZlVieVN6QTYvUmVLZzZq?=
 =?utf-8?B?VGl0MzEzVmlzWmdGcWJub3U1NW5Pa2lGeFJRR0Z4OHBVY3g5YlJ1QjFYZjUx?=
 =?utf-8?B?VWQyc2RZMTR3a2U5S0c0d3NtRU5udTZ4YUN5RmdITlJlU2hpUGdKNkh2NXUx?=
 =?utf-8?B?bktnK0JRYlhpTEI5azVRNW5ZMkYzN0hzbFk5aDBTZXpjaG1vcEpOTXI4NTdS?=
 =?utf-8?B?SDc2R2kwY2J2Y0syTW1ZNHFyeFdkck9zek1HZWVUcmJPSXlDZHpRMGZsSkFp?=
 =?utf-8?B?WnEyMStmdkF0WTJ1NE9HNTFuV3E3OHFSbWg5ZkUzTE9rdmxpZUJQRTlwZ05C?=
 =?utf-8?B?bXRQRVpoQUxNajNFazFhREhOTVB5TGd4bUlydDlvc2VPeW1NV09uQ1dQQ0hv?=
 =?utf-8?B?dUJ2c1VxK2dJTElWZTd5QXRRMEgyMTduOTM4aVEzVnZNanBwSFN6V3VyZTBt?=
 =?utf-8?B?N01SM1d3dWdxQ3dtNm82Z0tobWJXV0FBa0FwaUJCWWtsK0VpZEp0cjNhWnF5?=
 =?utf-8?B?WmRvUG12cGYwYUsza1V2L0RCYkxXYVRkbVJmVUZEMW5aZXZtcUdBYTlMR3Vi?=
 =?utf-8?B?V3RTVHgzZDVDaU4vNnlUK1diMzFYeDJqRHI2Z3BiZFFGZmtYVFJneDJyR2sz?=
 =?utf-8?B?YldZZ0RINmdaTnRRU2lpMVVtZEdoZ0IvWXgrQnQveEkwWHZZekcxbFBybGpm?=
 =?utf-8?B?d1J3OSsrTTdPWXhFM3h0OXFtVkQwTTkvQk9pcXZYS3JlbG9uay9uNXVUZ0pi?=
 =?utf-8?B?bVlKRWxpeUJtM3dJdktCYVQvR0ZSUi9vUm8rRmt0aHhsZ0V6Zi9ybjlwazdI?=
 =?utf-8?B?Q2FZeHdDQVlIVmNtaGxTV1BLdHdtcDBDRmJ4SEowa2FVTFI4M2FBaFRKbDVl?=
 =?utf-8?B?UUJ4U09QamNIMGJSTDFwdEdGM04zbllqTDZneHhRaElwelVOaGppMHh6L0FT?=
 =?utf-8?B?RFgvTXNXcm1RNnN2U3FzYXlOVEMrV0MvZmZoSEh1cmZ0OWZCWFlHQkRQUUc5?=
 =?utf-8?B?WW1jNENLaHlaZi8wNzE4dHMzYTBZZjRkSVE4RnFiYTZkR21Cc2dJUTQvQ2Vr?=
 =?utf-8?B?WlovaDFRSWRxNDRoUHhXcE9ZNzZzL2FEZ1BJSUhEK2x0MitOV1NrYk1RY1Fy?=
 =?utf-8?B?MENGTGlYc0sveCthb2Jib1RNYTBjcGRWSEVaUmErbFQxVW1rcTRYRmhmMkg5?=
 =?utf-8?B?cTF6OHUvRHlBSFhxcWlReHhsVC9CS1plYUdweFJVNnJFcGY1MVhScjkxSllH?=
 =?utf-8?B?cExrMVQ3R1pFbCtIS3pHcG1NT3hsWnp2akgwelQ4Qk9aSENFQTFlMURONVpU?=
 =?utf-8?Q?ly2FiBMcaKV+l2KwiYhaHQ1ASrVFqTY4XMmQGlT?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b43993ec-940f-418c-2395-08d97e6981ed
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 08:09:46.8353
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Y1p5vefj8KVo4ItfhP4vYfTeJYiU8D/JjT8WMGzWuv2AySsGkLyFtTx8QfoZZjxbO7XtBZSzt10V1QyUz5Npig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5445
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:19:37AM +0200, Jan Beulich wrote:
> While all present callers want to act on "current", stack dumping for
> HVM vCPU-s will require the function to be able to act on a remote vCPU.
> To avoid touching all present callers, convert the existing function to
> an inline wrapper around the extend new one.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Alternatively the actual dumping patch could avoid using this more
> elaborate function and, ignoring access checks, simply add in the SS
> segment base itself (if needed in the first place).
> ---
> v3: New.
> 
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -2526,7 +2526,8 @@ int hvm_set_cr4(unsigned long value, boo
>      return X86EMUL_OKAY;
>  }
>  
> -bool_t hvm_virtual_to_linear_addr(
> +bool hvm_vcpu_virtual_to_linear(
> +    struct vcpu *v,
>      enum x86_segment seg,
>      const struct segment_register *reg,
>      unsigned long offset,
> @@ -2535,8 +2536,9 @@ bool_t hvm_virtual_to_linear_addr(
>      const struct segment_register *active_cs,
>      unsigned long *linear_addr)
>  {
> -    const struct vcpu *curr = current;
>      unsigned long addr = offset, last_byte;
> +    const struct cpu_user_regs *regs = v == current ? guest_cpu_user_regs()
> +                                                    : &v->arch.user_regs;
>      bool_t okay = 0;

Since you change the function return type to bool, you should also
change the type of the returned variable IMO. It's just a two line
change.

Also is it worth adding some check that the remote vCPU is paused? Or
else you might get inconsistent results by using data that's stale  by
the time Xen acts on it.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:10:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:10:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193482.344629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJo2-0003s2-C9; Thu, 23 Sep 2021 08:10:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193482.344629; Thu, 23 Sep 2021 08:10:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJo2-0003rt-8u; Thu, 23 Sep 2021 08:10:10 +0000
Received: by outflank-mailman (input) for mailman id 193482;
 Thu, 23 Sep 2021 08:10:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTJo0-0003JR-Tv
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:10:08 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ec72a713-ee26-4528-b4a8-563deea9ed5c;
 Thu, 23 Sep 2021 08:10:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1ED1B221D4;
 Thu, 23 Sep 2021 08:10:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 82C6313DC7;
 Thu, 23 Sep 2021 08:10:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id hYWfHlo2TGGbEgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 08:10:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec72a713-ee26-4528-b4a8-563deea9ed5c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632384603; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=yCmYQKyQ1CegwpuFA1biRZgl2wD61Tp1f0chBfFWd8E=;
	b=Zo9jK3BESoN14AcEe1BU9TlyBOM5vIu7qEY3xWT70OoJZfYRozRIjKKOglOdiFf+TrLBEk
	4Qe+Daxrgzlk9cBOtPI08MTMcWVKBHZOyBtMsEY1ZNIOMoFLH5eYd3sSKzrfEDby3x/F+o
	VQfB1yri9FWEgzsgQNNNznPuWHkZIp4=
Subject: Re: [PATCH 2/3] xen/x86: free_p2m_page: use memblock_free_ptr() to
 free a virtual pointer
To: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, devicetree@vger.kernel.org,
 iommu@lists.linux-foundation.org, kasan-dev@googlegroups.com,
 kvm@vger.kernel.org, linux-alpha@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org,
 linux-mm@kvack.org, linux-riscv@lists.infradead.org,
 linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
 linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org,
 linux-usb@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
 sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
 Mike Rapoport <rppt@linux.ibm.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-3-rppt@kernel.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <69c60441-d6d0-96e2-a04e-5bdf87241b4b@suse.com>
Date: Thu, 23 Sep 2021 10:10:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210923074335.12583-3-rppt@kernel.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Ou4UW1xPRLuVMrznu7nRrBvD9qxXwPwpI"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Ou4UW1xPRLuVMrznu7nRrBvD9qxXwPwpI
Content-Type: multipart/mixed; boundary="Opwz8FSwCRvDNIt0szi2Ea4VaFO0qhLYD";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, devicetree@vger.kernel.org,
 iommu@lists.linux-foundation.org, kasan-dev@googlegroups.com,
 kvm@vger.kernel.org, linux-alpha@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org,
 linux-mm@kvack.org, linux-riscv@lists.infradead.org,
 linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
 linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org,
 linux-usb@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
 sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
 Mike Rapoport <rppt@linux.ibm.com>
Message-ID: <69c60441-d6d0-96e2-a04e-5bdf87241b4b@suse.com>
Subject: Re: [PATCH 2/3] xen/x86: free_p2m_page: use memblock_free_ptr() to
 free a virtual pointer
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-3-rppt@kernel.org>
In-Reply-To: <20210923074335.12583-3-rppt@kernel.org>

--Opwz8FSwCRvDNIt0szi2Ea4VaFO0qhLYD
Content-Type: multipart/mixed;
 boundary="------------7FABCF9287BBEACE094F562B"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7FABCF9287BBEACE094F562B
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 09:43, Mike Rapoport wrote:
> From: Mike Rapoport <rppt@linux.ibm.com>
>=20
> free_p2m_page() wrongly passes a virtual pointer to memblock_free() tha=
t
> treats it as a physical address.
>=20
> Call memblock_free_ptr() instead that gets a virtual address to free th=
e
> memory.
>=20
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------7FABCF9287BBEACE094F562B
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7FABCF9287BBEACE094F562B--

--Opwz8FSwCRvDNIt0szi2Ea4VaFO0qhLYD--

--Ou4UW1xPRLuVMrznu7nRrBvD9qxXwPwpI
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMNlkFAwAAAAAACgkQsN6d1ii/Ey/u
jwf/Wc7bKSwtYpm6kgk3TsHmpeJbPgh0Zpv1wT9MtIr5veUKbFZsg/Ji0X/gETRJ/GzFem6QpjrD
qfZjjrHW84FjHmrikrdzulZV8SZVqYLSdZWQRL4dja5oWLHT7nPkOtdmNelKufQ3CxAmy1JmVzVb
Mx6gmnvelfR4gvjcbXXvmtdNZvJIKIQ3zLsqDK8z5H0AA3wt7EG/6FFaIZD/lYSqQAmpBXhbvdZe
EmLrt4FExY741RXXb6HIT7WjiQ+iFHRiGgNjAF7OZlY2xzR14fRVqNF56oq8GYvMSJU1knueE60N
Y+1NbMVNCzWoYzBjFB95UGbdvzgBy6XOzZ21ar2IPQ==
=DYUR
-----END PGP SIGNATURE-----

--Ou4UW1xPRLuVMrznu7nRrBvD9qxXwPwpI--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:15:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:15:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193492.344641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJsx-0004nZ-Vh; Thu, 23 Sep 2021 08:15:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193492.344641; Thu, 23 Sep 2021 08:15:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTJsx-0004nS-SX; Thu, 23 Sep 2021 08:15:15 +0000
Received: by outflank-mailman (input) for mailman id 193492;
 Thu, 23 Sep 2021 08:15:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTJsw-0004nK-Ky
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:15:14 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c05c6f91-2c5d-43da-bd78-76684ac40473;
 Thu, 23 Sep 2021 08:15:13 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A60422027D;
 Thu, 23 Sep 2021 08:15:12 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 13F2613DC7;
 Thu, 23 Sep 2021 08:15:12 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id ojiaA5A3TGF2FQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 08:15:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c05c6f91-2c5d-43da-bd78-76684ac40473
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632384912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=QSLhmvEk1Pq1iiswl5snzLhelQGabqOAvRi47xucKUc=;
	b=dQkZNFnVmjFTdqDDt9GU9DTIEjBGZRbIA2ofbDcmiIB8bDqVrDggJ8ZwgPcJOAAnibhKSk
	wohDp1nG/Y0sL0SW/01jfOSBK8tJNu5AzCDV/dTnleWkyjK5RMbnhh0I5Wt/IQmwJHm3zx
	ZvfeJvE8TpXAAdEAgm6wdTBm9F+eeO8=
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
To: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, devicetree@vger.kernel.org,
 iommu@lists.linux-foundation.org, kasan-dev@googlegroups.com,
 kvm@vger.kernel.org, linux-alpha@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org,
 linux-mm@kvack.org, linux-riscv@lists.infradead.org,
 linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
 linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org,
 linux-usb@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
 sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
 Mike Rapoport <rppt@linux.ibm.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <60c0d0f9-e4c6-ef66-b85b-0d091f8cba15@suse.com>
Date: Thu, 23 Sep 2021 10:15:11 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210923074335.12583-4-rppt@kernel.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="zppOVwffTjZ5Oe49Vq0wbEbRvLUF4dOID"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--zppOVwffTjZ5Oe49Vq0wbEbRvLUF4dOID
Content-Type: multipart/mixed; boundary="AxmgW1XtM0JJFTzWHwG0o4gfWFl7DFvoM";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, devicetree@vger.kernel.org,
 iommu@lists.linux-foundation.org, kasan-dev@googlegroups.com,
 kvm@vger.kernel.org, linux-alpha@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org,
 linux-mm@kvack.org, linux-riscv@lists.infradead.org,
 linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
 linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org,
 linux-usb@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
 sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
 Mike Rapoport <rppt@linux.ibm.com>
Message-ID: <60c0d0f9-e4c6-ef66-b85b-0d091f8cba15@suse.com>
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
In-Reply-To: <20210923074335.12583-4-rppt@kernel.org>

--AxmgW1XtM0JJFTzWHwG0o4gfWFl7DFvoM
Content-Type: multipart/mixed;
 boundary="------------CFB99E0866EE66F8CFC01FC3"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------CFB99E0866EE66F8CFC01FC3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 09:43, Mike Rapoport wrote:
> From: Mike Rapoport <rppt@linux.ibm.com>
>=20
> For ages memblock_free() interface dealt with physical addresses even
> despite the existence of memblock_alloc_xx() functions that return a
> virtual pointer.
>=20
> Introduce memblock_phys_free() for freeing physical ranges and repurpos=
e
> memblock_free() to free virtual pointers to make the following pairing
> abundantly clear:
>=20
> 	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
> 	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);
>=20
> 	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
> 	void memblock_free(void *ptr, size_t size);
>=20
> Replace intermediate memblock_free_ptr() with memblock_free() and drop
> unnecessary aliases memblock_free_early() and memblock_free_early_nid()=
=2E
>=20
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>

arch/x86/xen/ parts: Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------CFB99E0866EE66F8CFC01FC3
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------CFB99E0866EE66F8CFC01FC3--

--AxmgW1XtM0JJFTzWHwG0o4gfWFl7DFvoM--

--zppOVwffTjZ5Oe49Vq0wbEbRvLUF4dOID
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMN48FAwAAAAAACgkQsN6d1ii/Ey8D
fwf/WV3EUVWvjXkc64q0a0it6LMGy2AtQrh8KdDecuLV8iH5bKTnqNAZOUoV6sYTeiLsSSnRTLOt
yKKjkWsC9/gUsyuO0B8Zw/VX/zoXJqp7T57FfmW+37qcslFuLzImqvDxdU65n/jEbme+VExmw6UF
yy1ATxxxhQIxeTDXB3SfE0f6rX4Fw1DUqQc25bFNpD1wzdp1xG6qhH31/CWUI/V/frEfuzZrrN5F
Uimkqk3+xjrqqpYh2fb/Pwpd77LFOdIrV4gH0oyl0NA3x3QMNi+67FrbMtuRHZij1jnpwoY1RiUc
uVxzINJ+LJh0g8836hHAkPh5tQNBjV7C6V7LXddn6g==
=Fnkp
-----END PGP SIGNATURE-----

--zppOVwffTjZ5Oe49Vq0wbEbRvLUF4dOID--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:23:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:23:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193497.344652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTK0n-0006QV-Qr; Thu, 23 Sep 2021 08:23:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193497.344652; Thu, 23 Sep 2021 08:23:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTK0n-0006QO-NU; Thu, 23 Sep 2021 08:23:21 +0000
Received: by outflank-mailman (input) for mailman id 193497;
 Thu, 23 Sep 2021 08:23:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTK0m-0006QI-5Z
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:23:20 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 81b2aade-1c47-11ec-ba1b-12813bfff9fa;
 Thu, 23 Sep 2021 08:23:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81b2aade-1c47-11ec-ba1b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632385399;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=SVVg9QjwxI5bFvKqj21ypl6w122xBh2MGFBLxXhmFtQ=;
  b=d/bnPlGcwXZhu1AUjaxl8N3DWsjRx1HQ5BiEzOgbz2AuhRjLQ9e0ION8
   H062UjYBPTti48k4cuHQLwIzpEYJHXvZ9vnvaBEjGu6Qxv5IB6sDJST72
   rYJoa/xXVK5fZseeGekI02VMJQUYLeYSgqzQBqvacakP66Mx2v5wJtpsb
   E=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: BsvoHUPuhHF3qYKbY+Folo+4ZARk791rq5Wo1ESxVwXDk9/WWXuKOjybb3VHK/1jcmp5IfQQfM
 KedGlKUcSio2gBLfe5uCi7A53hcvivgZnD16i6KU8Q2z/7hLIo2Rvy/13frK/ASZyfg2AZ5jT3
 GmLBzEaiKOCn8b9bJSsbGIOxlssTKjRSkH4EsICsr8evYe+tmhQ/aBvAzD9snyXEGUOFnLQKFG
 oU0taD8IPt/Jwyc39pGJMBtkSy6LsGlJeTHkzvPVhL/07k5KS1PUnWb8fmrBnAUR0HM8VOUzPZ
 gw4WAOLmhFDLTttAwhcCjNVJ
X-SBRS: 5.1
X-MesageID: 53407651
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:9j2NU6Idr2KUfCzeFE+RKZIlxSXFcZb7ZxGr2PjKsXjdYENS32FRn
 zMZXGmAMv+NZTHwKNt3bN+/o00FupLSy9NqGQFlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5wLZg6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB3Zvo4y0
 fpT5aCVaiImb4HwqN0WSh9XRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsF2gcsuNo/zNZ43sXB81zDJS/0hRPgvRo2XvoMAg2lt36iiG97TV
 dAHOBwwTi7uSD4SZW01M4sEpsKR0yyXnzpw9wvO+PtfD3Lo5A98zrnrdsbUc9qiRMNJk0Ler
 WXDl0z7HxdcMtWcwDiE93uEh+nTkCe9U4UXfJW6/PN3hFyYxkQIFQYbE1C8pJGRmkO4Ht5SN
 UEQ0i4vtrQpslymSMHnWB+1q2LCuQQTM+e8CMVjtlvLkPCNpV/EWC5UFVatdeDKqudmZGJpz
 3CXz+/MJhlg4ZGpeUuW77Gb+Gba1TcuEUcOYioNTA0g6tbloZ0ugh+ncuuPAJJZnfWuRmqtm
 W7iQDwWwuxJ1J9Wi/zTEUXv3mr0zqUlWDLZ8ek+soiN1Qp/eIftTIih81GzAR1ofdvBEwHpU
 JTpnaGjAAEy4XOlyHflrAYlRujBCxO53Nv02wUH834JrWjFxpJbVdoMiAyS3W8wWir+RRfnY
 VXIpSRa74JJMX2hYMdfOtzqUZ9xlfC7S460B5g4i+aihLArL2drGwk0OSatM53FyhBwwcnTx
 7/BGSpTMZrqIfs+l2fnLwvs+bQq2jo/1QvuqWPTlXyaPU6lTCfNE98taQLWBshgtf/siFiFo
 r53aprRoz0CAbKWX8Ui2dNKRbz8BSNgXs6eRg0+XrPrHzeK70l7UKKOneN9JdY190mX/8+Rl
 kyAtoZj4AOXrVXMKBmQa2Alb7XqXJ1lqmk8MzBqNlGts0XPq671hEvGX5doL7Qh6sJ5yvt4E
 6sMd8maW6wdQTXb4TUNK5L6qdU6JhisgAuPOQujYSQ+IME8F1CYpIe8c1u97jQKAwq2qdA6/
 ++q2DTETMdRXA9lFsvXNq6ilgvjoXgHletudELUOd0PKl70+Y1nJnWp3P86Ks0BMzvZwT6e2
 1rECBsUv7CV8YQ07MPIleaPqILwS7lyGU9THm/667eqNHaFojr/kNEYCOvRJGLTTmL5/qmmd
 N559fCkPa1VhktOvqp9D61vkfA06ezwquII1Q9jBnjKMQimU+syPnmc0MBTnaRR3bsF6xCuU
 0eC99QGa7WEPMTpTAwYKAY/N7nR0PgVnn/Z7OgvIVW87yhypeLVXUJXNhiKqSpcMLoqb991n
 bZ/4JYbu16llx4nEtealSQFpW2DI0sJX7gjqpxHUpTgjRAmyw0abJHRYsMsDEpjtzmY3pEWH
 wKp
IronPort-HdrOrdr: A9a23:pgMAs6optGbW7kRbnmnBIiYaV5uhL9V00zEX/kB9WHVpm5Oj+f
 xGzc516farslossREb+expOMG7MBXhHLpOkPQs1NCZLXXbUQqTXftfBO7ZogEIdBeOk9K1uZ
 0QF5SWTeeAcmSS7vyKkDVQcexQuOVvmZrA7Yy1ogYPPGMaGJ2IrT0JcTpzencGNTWubqBJba
 Z0iPA3wAZJLh8sH7qG7zQ+Lqf+T5qhruOQXTc2QzocrCWehzKh77D3VzCewxclSjtKhZMv63
 LMnQDV7riq96jT8G6c60bjq7Bt3PfxwNpKA8KBzuATNzXXkw6tIKBsQaeLsjwZqPymrHwqjN
 7PiRE9ONkb0QKeQkiF5T/WnyXw2jcn7HHvjXeenHvYuMT8AAk3DsJQ7LgpOCfx2g4FhpVRwa
 hL12WWu958FhXbhhnw4NDOSlVDile0iWBKq59Qs1VvFa8lLJNBp40W+01YVL0aGjjh1YwhGO
 5ySOnB+fdtd0+AZXyxhBgt/DWVZAV2Iv66eDlEhiTMuAIm2kyRjnFohPD3p01wsa7UEPJ/lr
 352qcBrsAEciZZV9MkOA+tKfHHfFAleii8RV56F26XXJ3vC0i92aIf349FkN1CWKZ4gafay6
 6xHG+xiwYJCgvT4Iu1rcZ2ziw=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53407651"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DTIYaMjLpdNHwATwqpn6SSBOwNsR+fDaBBADRD8q3kpMe6kj1ruxySXm8RqXZZ9j7ZajYsyazi6oj50TgYkB/uVZj2O3CLxfUpzwrHQ6bKOg/vW7axUN5WabtrV7/sAPKrWGcoa8dPttybR12N0mpsVHQ9A+lfMf7DwGfGUClfe1BiH3W1KP+YBhegj9ZBYrVCoFkFJAcVGTseVXm5ObsP6Sfc4RHfH5bUmMC7ZTRnEZZBMEEVIQMCc9Kl6IEZ2aEzXvvokZcVrKCDJSj1nTMPnOLZ9WJgWDt9sW4k4OtxxrSnSy5jMQSfMIvf068b2eeS0Py75bNbcFHsT7XHJUMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pL5THT8DeRWbL+ZkXLmt1JQ/FRxDsykKgKZKcA1bbjw=;
 b=PJEcmji94If1/FTuQCV07cegHPB+ytC6HUvq1aBFbZTlXnRr2BRIUI+1afmNqWX5+KtmLDM5elG15v0id83QYiEjMdzX8HTAlVR7xea7h8K0uUEX8Y1CBk6FkmlcLxbFsAao4UNXY18nQOOlSCY3IFK9DsxCo0vbPdb2OJZsMPpX1hLQ1IWDBhMG3HV+UKTJRrkPoo7Bc+uCnEM7bQz8T0G+JFMnlkQt5h9C8gvwA6wciOEzS/5ToiZO5iMFbV4Ai+KPhgFqz5Rpm2YfmO/gYdrQXX90Bti8eUrvBvMYkdVP+Jx71xMYfCz31931hugd4vcAfVlbh+rEzJGFvbJZfQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pL5THT8DeRWbL+ZkXLmt1JQ/FRxDsykKgKZKcA1bbjw=;
 b=vec+Vi7Co/XreJtEWsy3qSop2LdwuWsNUPArB163yUW4Y43zuschAIaf0ndpsugK2S9YikMf/duFFDzP5JeCWTE8lb9EyuAF6ugrbOgAW7NgtePQkqj04VuOc4jBZbgUE1gamglpd8/08jy7AG9KOTjXuO2+gSONr3x6sA8vkb4=
Date: Thu, 23 Sep 2021 10:23:09 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Kevin Tian <kevin.tian@intel.com>, "Jun
 Nakajima" <jun.nakajima@intel.com>
Subject: Re: [PATCH v3 8/9] x86/HVM: skip offline vCPU-s when dumping
 VMCBs/VMCSes
Message-ID: <YUw5babRHdk6Jh8E@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <5cc8f7e0-399e-f6a5-bc84-823a6160a3f1@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5cc8f7e0-399e-f6a5-bc84-823a6160a3f1@suse.com>
X-ClientProxiedBy: LNXP265CA0022.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:5e::34) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: af49ae8c-dadb-4153-a25c-08d97e6b63d1
X-MS-TrafficTypeDiagnostic: DS7PR03MB5463:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5463060B54D92BAB1FBFEA128FA39@DS7PR03MB5463.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: OY/nJvcY1GdLOJWGrAeTioahC3H03cH0QO4H9KbO8hMQEvFlxozdCqgAdy+EzpQcUsTruW+GyAdRjw+fMB67kiEGmGj0qCobCNQzkTY3pxvbnd4brV+q1TAV92oN1DmSXiPOvxWBndTuGV2+5c7sMyC6zMKhFcBL7E41eL7ye4Wf/NUhkaKBnLm6Zkj6HG7vI+WmaygGYGkVqTqFm1PM5Sh5zzyMvk+VN+JMWA9YqEi6RmUiw9nyj2+RvKBd/nwebpGkzv6S7PtS62bqpMlSk7f0QoKDBfR1RAP6FhIykpx6bksDEAv42l2ACgDOnmYs9kRyy9nbENnvLl11L0YMmsbiyu5ET58oH787TDsp5ssSCm7hPoZCDYHIq3hFQ2V3HUQ6OrALyJEAQ5IwDueb/4QJfjXjHzBJMxDVdXBmxE0FFpx1zZVNb5QLDr6at2dQDYUt2BrcrosjZjZPs7/j0BWZ5KmzMQ3CTfe3q0DwoM06kShpHZt3a58bHVNYPjjgBOoimKiSIkw1Z/3ffSrdYltc9FbYzURuZCNoHZNqs/7BoYqJw5LVnmDtlDv+IZzyBgu3XyelxPrGS/zNY7ykugnOO8DEX/5ydzZq7FmV7DcnEfBR486sP4nzd+hay8CMBt4GTh9x4HwXenbyJZlOHw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(66556008)(66946007)(26005)(956004)(4744005)(9686003)(66476007)(85182001)(5660300002)(186003)(6496006)(6486002)(4326008)(316002)(8676002)(8936002)(2906002)(6666004)(6916009)(508600001)(54906003)(86362001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?N21HdDlhbVlnMUlYNlQ1elZMZk0rWU4vUVZIcW1OeVpSdkN2UUljaTdLTXZL?=
 =?utf-8?B?NFNGRituMitoYXNyVmVVRGcwNXVTUDVCV1B0UnJZOWpvQWNTY0Q2N3hyNUpi?=
 =?utf-8?B?K244NE5PcVdZU0wxL3B1bUZnZW5xZEJlS3BDRjVpRERGekJsbmFWcG90bU5W?=
 =?utf-8?B?UnJOWXRoN3dEbHAxdkZPZGlMZlJEY3VJcEF0ZVdFcDYxT05ZN2ZCNm5HeHAz?=
 =?utf-8?B?Sy84NXMvQ0JOdTdOb3V1cTdSbjFLQisyWHFIR1R3NFA5YS9xaWtSR3lrdGYy?=
 =?utf-8?B?L1hmOEJ5MFJWK3h2a3JnT25RM1luRmVHRXVCc0NOR0V3TXFEVXE2bjVqVlRm?=
 =?utf-8?B?SjBOSDZLNmppRXpqU0hIWlFZeFRSSEZ4Q1hPbnJWNzF4ZkxmM0FnNkIzZFZR?=
 =?utf-8?B?VjJaajlJU0RTekk0SFNKQWF4d2d4b3oyVno5WVBabUY4Z1VNcG9DRXZqOXk0?=
 =?utf-8?B?ZnRXMUNPSXFRc2FUUm1WY1VlUm9uTGZwYlJMcXlQRWZjTzRoTGhxeUJ3VmE1?=
 =?utf-8?B?dG9UTWE1alNpTEdvT2p1UVhOcitjYVNybHJMeEM4THFKWUlkQy82enJueko5?=
 =?utf-8?B?bWJYaWVRcWpBeFFNOU1YTEZTb3oxQlFhUzJ5VkQrOHJOZXNmNmRwRVlGK0NH?=
 =?utf-8?B?UWpNdnZsR0w5MjJWalROV1dZTEtQbWZHRXNqaW4yU295R0tsQkxOSlk3dVRU?=
 =?utf-8?B?eVoxc28vem9teGM1b1ZTZ1VCQWpYdXRvRTNQa0ZLek1MczcyTzZaZHE3NXFw?=
 =?utf-8?B?TzVWeUJKN0ErdW1TM2xhcnpNK3FkdnNnZHdXeTdrT3c2VCs4WmdtSmZJLzlp?=
 =?utf-8?B?cHhsQWRQd3VzOE9IS3o5L1BuRklwMU5VemNSY2lFcEhQRFdvR2JPZzRWcnRX?=
 =?utf-8?B?QmExMUFYSTNEWHIzWjJkeFlTekU0VUhYZjBieERSRlIzTnh2eXF5eXFkSmFD?=
 =?utf-8?B?ZllWQTZjWkQ2dGNGc0I5Ri9Gai9McjgySEFKZTJ4Z2lhWU1rdDEzSDJ6b3hq?=
 =?utf-8?B?MmlDSjZnQmwwcUlpT3MxczNzYU9EcktqMmwxdVJTZFNWb095dHVGVVE0bHkr?=
 =?utf-8?B?OXpJL2g3OVJmMk1HSnNsVi82N296dHZ1ZmM4Y1JvZWxVN1pvZXFZcWYwcXoy?=
 =?utf-8?B?Umc5NUJDQVJjWTFoYUV0ZTNvNFBEWm42QXVHcXpUQS9qT3JheWVFK3hlZUwz?=
 =?utf-8?B?bThRUnBMZFk3R2owdTBjS0FlQ3V3WXVKL25kQUllSzUrRHIxVjhYRHhnM2Jh?=
 =?utf-8?B?Q3RXbGt3eHN1UGo5QVNVMVFWSHByVDZrSDFrZnl2SGR3RWMxNFljbFlHU0Rv?=
 =?utf-8?B?TktaNXhzandNdm9yVlhPVkhuQ0NqdUxOU0wyMmtIM01iUjNFYkxmaEZ5Zk9s?=
 =?utf-8?B?bUMweks4NnE1Vy9Qb3EyeWh6RWtoTHZnT0xuUWJiRG9DZE9TNzBsMXlDd28z?=
 =?utf-8?B?b1UvY0NxeGllVGs0bGtRd1ZpdG1KNFVYWUdWaUdTckJDbUFHRFlVR3pCMXk2?=
 =?utf-8?B?ODdRcloxOE0wZExuNStNbnl3akdPRHFsL3g2Q05jcytNNk94QmdvYlhva1Rt?=
 =?utf-8?B?Q0ZwcDMwaXhROEduaEpZMlk2ODdrbUZsTFdpRWVzM212MzVWSVUzQnJ6WEZN?=
 =?utf-8?B?dUhOOS9Jd0NYcktGaURpWGs4c3NzMDFEaSsvR0xZd3ZkQ0dpTjFLcHFKWldQ?=
 =?utf-8?B?cVppMGJTZHpXTlB5NnFxa3FDMFFBTFdBWnRmeXBac2RmZVJhdGRNTE82a1NG?=
 =?utf-8?Q?SQaw6T2vZuUEosi2lWU4qh27Wu9yxVsfxAIocEi?=
X-MS-Exchange-CrossTenant-Network-Message-Id: af49ae8c-dadb-4153-a25c-08d97e6b63d1
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 08:23:15.2173
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 30IcUcn7kT8TALqPhz0PSq0Jbc96/Z6hebRFv/ZOivJ7cT9pLbvA7cZOvepJrRjSNGG1iNB0leKOk87eTwUJcw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5463
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:20:32AM +0200, Jan Beulich wrote:
> There's not really any register state associated with offline vCPU-s, so
> avoid spamming the log with largely useless information while still
> leaving an indication of the fact.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Is the state cleared when the vCPU is brought down? Or else it might
be interesting to print such state for debug purposes.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:40:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:40:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193504.344663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKH7-0000mY-ED; Thu, 23 Sep 2021 08:40:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193504.344663; Thu, 23 Sep 2021 08:40:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKH7-0000mR-BB; Thu, 23 Sep 2021 08:40:13 +0000
Received: by outflank-mailman (input) for mailman id 193504;
 Thu, 23 Sep 2021 08:40:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKH5-0000mL-Vd
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:40:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKH3-0003OX-VB; Thu, 23 Sep 2021 08:40:09 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKH3-0006L7-Kf; Thu, 23 Sep 2021 08:40:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=/CeiMIMdodfncL/h5MckRpUSYe9NOFuKqaP7mLNAvGw=; b=mZw+ciWDEPzP6dK5Y8zEbwov+E
	MykIUjoosD2Mcxxfrvq0Wcpmyulf4nGLOHhOu7krSZwFwS3EptbPEuG2YJAKW8nD1VF1116YstEMU
	ELlbbkHxE/9+l26E1yvIY+XcBNhwMg9UqX93nxHzyLeu1cv6I+jSM6Rj+KC1w8eDtMBA=;
Subject: Re: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call from
 gnttab_map_frame()
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
 <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
 <b7947619-3520-174c-f5ec-49564ee048a1@xen.org>
 <474bd675-b42e-3f9f-03d6-f7afc26e10c5@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <8c685481-a74d-9aa6-c1db-4d22c95a4eda@xen.org>
Date: Thu, 23 Sep 2021 13:40:04 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <474bd675-b42e-3f9f-03d6-f7afc26e10c5@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Jan,

On 22/09/2021 15:47, Jan Beulich wrote:
> On 22.09.2021 12:28, Julien Grall wrote:
>> Hi,
>>
>> On 22/09/2021 14:42, Jan Beulich wrote:
>>> On 22.09.2021 11:26, Roger Pau Monné wrote:
>>>> On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
>>>>> On 21.09.2021 10:32, Roger Pau Monné wrote:
>>>>>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>>>>>>> On 20.09.2021 12:20, Roger Pau Monné wrote:
>>>>>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>>>>>>> --- a/xen/include/asm-arm/grant_table.h
>>>>>>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>>>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||           \
>>>>>>>>
>>>>>>>> I'm slightly confused by this checks, don't you need to check for
>>>>>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>>>>>>> guest_physmap_remove_page?
>>>>>>>
>>>>>>> Why? It's ogfn which gets passed to the function. And it indeed is the
>>>>>>> prior GFN's mapping that we want to remove here.
>>>>>>>
>>>>>>>> Or assuming that ogfn is not invalid can be used to imply a removal?
>>>>>>>
>>>>>>> That implication can be (and on x86 is) used for the incoming argument,
>>>>>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>>>>>>
>>>>>> I guess I'm confused due to the ogfn checks done on the Arm side that
>>>>>> are not performed on x86. So on Arm you always need to explicitly
>>>>>> unhook the previous GFN before attempting to setup a new mapping,
>>>>>> while on x86 you only need to do this when it's a removal in order to
>>>>>> clear the entry?
>>>>>
>>>>> The difference isn't with guest_physmap_add_entry() (both x86 and
>>>>> Arm only insert a new mapping there), but with
>>>>> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
>>>>> mappings. And gnttab_map_frame() gets called only from there. This
>>>>> is effectively what the first paragraph of the description is about.
>>>>
>>>> OK, sorry, it wasn't clear to me from the description. Could you
>>>> explicitly mention in the description that the removal is moved into
>>>> gnttab_set_frame_gfn on Arm in order to cope with the fact that
>>>> xenmem_add_to_physmap_one doesn't perform it.
>>>
>>> Well, it's not really "in order to cope" - that's true for the placement
>>> prior to this change as well, so not a justification for the change.
>>> Nevertheless I've tried to make this more clear by changing the 1st
>>> paragraph to:
>>>
>>> "Without holding appropriate locks, attempting to remove a prior mapping
>>>    of the underlying page is pointless, as the same (or another) mapping
>>>    could be re-established by a parallel request on another vCPU. Move the
>>>    code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
>>>    xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
>>>    course this new placement doesn't improve things in any way as far as
>>>    the security of grant status frame mappings goes (see XSA-379). Proper
>>>    locking would be needed here to allow status frames to be mapped
>>>    securely."
>>>
>>>> TBH I think it would be in our best interest to try to make
>>>> xenmem_add_to_physmap_one behave as close as possible between arches.
>>>> This discrepancy between x86 and Arm regarding page removal is just
>>>> going to bring more trouble in the long term, and hiding the
>>>> differences inside gnttab_set_frame_gfn just makes it even more
>>>> obscure.
>>>
>>> Stefano, Julien?
>>
>> This would be ideal as I don't particular like the approach taken in
>> this patch. But AFAICT, this would require us to implement an M2P. Or is
>> there another way to do it?
> 
> For the purpose of just XENMAPSPACE_grant_table the "restricted" M2P
> (to just grant table pages), as introduced by an on-list patch, would
> do afaict. That wouldn't remove all the differences between the two
> implementations, but at least the one affecting the difference in how
> gnttab_map_frame() needs to behave.

This would work. I will add it in my todo list but I am not such when I 
will have time to look at it. I am happy if someone else wants to look 
at it.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:41:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:41:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193509.344674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKIP-0001LB-Od; Thu, 23 Sep 2021 08:41:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193509.344674; Thu, 23 Sep 2021 08:41:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKIP-0001L4-Lf; Thu, 23 Sep 2021 08:41:33 +0000
Received: by outflank-mailman (input) for mailman id 193509;
 Thu, 23 Sep 2021 08:41:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKIO-0001Ky-LJ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:41:32 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKIN-0003PT-LX; Thu, 23 Sep 2021 08:41:31 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKIN-0006UR-BN; Thu, 23 Sep 2021 08:41:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=oGTGb53KwrXJZg3OWCEohSAIq+O4UDOxgTZE1KabNu8=; b=dHjVg5h0MG4trwUFjF/moW1+Fy
	RnOgVHvXbObqh/SGYxItZAGPnDyNAGNKWvu5HotRUfKbr9BpFw2zNGvbzJQWmRrASTQ759d1rhd3/
	4gyZkz9pdBgOQcKT0ACZnvmIDkrFSQw7Ay7EYYmX2pN2P5/QLhfz4lhVwn0sCFx6G+OE=;
Subject: Re: [PATCH v2 6/6] gnttab: allow disabling grant table per-domain
To: Juergen Gross <jgross@suse.com>, Roger Pau Monne <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <20210922082123.54374-7-roger.pau@citrix.com>
 <b900f179-c6c2-00f5-e531-5110307fa491@xen.org>
 <c6dc7f81-0c4a-5c2f-1ac8-bb3ce5a58120@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <4ff81f6e-a422-3d96-0089-9c7325371937@xen.org>
Date: Thu, 23 Sep 2021 13:41:26 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <c6dc7f81-0c4a-5c2f-1ac8-bb3ce5a58120@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 22/09/2021 14:38, Juergen Gross wrote:
> On 22.09.21 11:19, Julien Grall wrote:
>> Hi Roger,
>>
>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>> Allow setting max_grant_version to 0 in order to disable grant table
>>> usage by a domain. This prevents allocating the grant-table structure
>>> inside of Xen and requires guards to be added in several functions in
>>> order to prevent dereferencing the structure.
>>>
>>> Note that a domain without a grant table could still use some of the
>>> grant related hypercalls, it could for example issue a GNTTABOP_copy
>>> of a grant reference from a remote domain into a local frame.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>> ---
>>>   docs/man/xl.cfg.5.pod.in     |   4 +-
>>>   tools/libs/light/libxl_dom.c |   2 +-
>>>   xen/common/grant_table.c     | 100 +++++++++++++++++++++++++++++++++--
>>>   3 files changed, 98 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>>> index c5a447dfcd..d507540c2c 100644
>>> --- a/docs/man/xl.cfg.5.pod.in
>>> +++ b/docs/man/xl.cfg.5.pod.in
>>> @@ -583,8 +583,8 @@ L<xl.conf(5)>.
>>>   =item B<max_grant_version=NUMBER>
>>>   Specify the maximum grant table version the domain is allowed to 
>>> use. Current
>>> -supported versions are 1 and 2. The default value is settable via
>>> -L<xl.conf(5)>.
>>> +supported versions are 1 and 2. Setting to 0 disables the grant 
>>> table for the
>>> +domain. The default value is settable via L<xl.conf(5)>.
>>
>> Technically, the version only applies to format of the table for 
>> granting page. The mapping itself is version agnostic. So this feels a 
>> bit wrong to use max_grant_version to not allocate d->grant_table.
>>
>> I also can see use-cases where we may want to allow a domain to grant 
>> page but not map grant (for instance, a further hardening of XSA-380). 
>> Therefore, I think we want to keep max_grant_version for the table 
>> itself and manage the mappings separately (possibly by letting the 
>> admin to select the max number of entries in the maptrack).
> 
> You mean the already existing domain config option max_maptrack_frames?

Yes. I didn't realize we already had an option for that :).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:47:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:47:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193514.344685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKOV-0002Hr-EC; Thu, 23 Sep 2021 08:47:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193514.344685; Thu, 23 Sep 2021 08:47:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKOV-0002Hk-AW; Thu, 23 Sep 2021 08:47:51 +0000
Received: by outflank-mailman (input) for mailman id 193514;
 Thu, 23 Sep 2021 08:47:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKOU-0002He-2c
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:47:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKOO-0003Wg-MZ; Thu, 23 Sep 2021 08:47:44 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKOO-00076k-BB; Thu, 23 Sep 2021 08:47:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=EFrIMcFb+Qzx31VA8bLct+FAFfProGkLXjBGJVlcABg=; b=MOpm3sGBCAjK/UbEBqTqEtIoY6
	h++YyRK2392PJGzh4Ipx+xZbyquPI0AHHM5YFAw0bWV8fWL9jyFufKAJWw+VuCw9HedPaPRL453Q4
	tFlKDjIx1wKq1JAxTvvVQgrNHUjkOsiyJF9o48UP4XSRL2+LHQguITtQOPXvglM9rKGA=;
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
From: Julien Grall <julien@xen.org>
Message-ID: <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
Date: Thu, 23 Sep 2021 13:47:37 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 22/09/2021 14:39, Roger Pau Monné wrote:
> On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
>>
>>
>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>> Hello,
>>
>> Hi Roger,
>>
>>> First patch on the series is a trivial change to xenconsoled in order to
>>> use xenforeignmemory stable library in order to map the shared console
>>> ring instead of the unstable libxc interface. It's reviewed and ready to
>>> go in.
>>>
>>> Patches 2 and 3 allow setting the host wide command line `gnttab` option
>>> on a per domain basis. That means selecting the max allowed grant table
>>> version and whether transitive grants are allowed.
>>>
>>> The last 3 patches attempt to implement support for creating guests
>>> without a grant table. This requires some changes to xenstored in order
>>> to partially support guests without a valid ring interface, as the lack
>>> of grant table will prevent C xenstored from mapping the shared ring.
>>> Note this is not an issue for Ocaml xenstored, as it still uses the
>>> foreign memory interface to map the shared ring, and thus won't notice
>>> the lack of grant table support on the domain.
>>
>> I find a bit odd that the Xenstore support is conditional to whether grant
>> table is available. Are you expecting domains with no grant table to have no
>> PV drivers (including PV shutdown)?
> 
> I don't really expect much, as having guests without grant table is a
> developer option right now, if someone wants to make use of them for
> any reason it would need some thought.
> 
> The other option would be my first proposal to restore foreign mapping
> of the xenstore ring on that case:
> 
> https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.com/
> 
> But it's also arguable that a guest not having a grant table should
> also likely prevent foreign mapping attempts. Plus such foreign
> mapping won't work from stubdomains.

There is another option: extend the acquire hypercall to allow xenstored 
domain to map the xenstore interface. This would require more work, but 
at least it would avoid the interesting dependency on the grant table.

> 
> I'm fine with dropping those patches if they turn out to be too
> controversial, I think it's an interesting option to be able to
> disable the grant table, but I don't have a full picture of how that
> could be used in practice. Maybe others have and would be willing to
> pick this up.

I think the current approach is probably OK as a developper option. 
However, we should at least document in the option that disabling the 
grant-table will also disable Xenstore (anything else?) support when 
using C Xenstored.

It might also be worth to clearly state in the doc that this is only 
intended for developer use and not supported.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 08:52:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 08:52:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193519.344696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKSh-0003eS-UQ; Thu, 23 Sep 2021 08:52:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193519.344696; Thu, 23 Sep 2021 08:52:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKSh-0003eL-RC; Thu, 23 Sep 2021 08:52:11 +0000
Received: by outflank-mailman (input) for mailman id 193519;
 Thu, 23 Sep 2021 08:52:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKSf-0003eF-U5
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 08:52:09 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKSf-0003bu-Kg; Thu, 23 Sep 2021 08:52:09 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKSf-0007OT-Ah; Thu, 23 Sep 2021 08:52:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=54Q2iN3Hzh5H2P1QQyjMr5dul7cZXAydmOHMnp8mxEU=; b=r304z6fqwu+glGkW7MPd/C2ZtN
	2ZLRKWcv1Jyvrevma5Dor/BOKOt7A8/3rBDpC+OddvdV4McK1Szab/Vd2hnBUpb4eXyr3K5TJWTnG
	lB98NjEhLGsuzdr6xAqHguXU7zxHmw3JpbytFbndAsxXWk298l+aRyfj9kZ3yFiielmM=;
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 andre.przywara@arm.com, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <33deb79b-54e3-c00e-653c-9c1191574b4a@xen.org>
Date: Thu, 23 Sep 2021 13:52:05 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 07:23, Stefano Stabellini wrote:
>> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
>> index 4b32c7088a..5406daecda 100644
>> --- a/xen/include/asm-arm/pci.h
>> +++ b/xen/include/asm-arm/pci.h
>> @@ -18,6 +18,7 @@
>>   #ifdef CONFIG_HAS_PCI
>>   
>>   #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>> +#define PRI_pci "%04x:%02x:%02x.%u"
> 
> This is added in this patch but it is unused here

To add on this, this is technically not arch specific. So if this is 
necessary in a follow-up patch, then the definition should be moved to a 
generic header.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:02:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:02:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193525.344707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKcL-0005LB-Ts; Thu, 23 Sep 2021 09:02:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193525.344707; Thu, 23 Sep 2021 09:02:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKcL-0005L4-Qo; Thu, 23 Sep 2021 09:02:09 +0000
Received: by outflank-mailman (input) for mailman id 193525;
 Thu, 23 Sep 2021 09:02:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/6DS=ON=epam.com=prvs=9900839950=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTKcJ-0005Ky-NC
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:02:08 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ecd2bbba-1c4c-11ec-ba1f-12813bfff9fa;
 Thu, 23 Sep 2021 09:02:06 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18N7ObkM011292; 
 Thu, 23 Sep 2021 09:02:03 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2051.outbound.protection.outlook.com [104.47.9.51])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b8n5trb59-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 09:02:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7489.eurprd03.prod.outlook.com (2603:10a6:20b:272::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 09:02:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 09:02:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecd2bbba-1c4c-11ec-ba1f-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=foUkaKBUrCzTz0UtCkOFogcXixBB13y9QGbEwvH64j2RbliXGFhTZydg5DhaAPjsV3UOMeQwOFyWzB5g3DisvHrl7u5I6Cf/V5UgZ5eIaco8BG0UKYmQoZXaw0OzYTcth7sOD+VUg7eFz4dAumHD71ukS6dQaJeH/OCxh1B4TJTM6y5t1f1bVvb4/klPMlUUFzigI/gd1LCvd5r2dgh01cLJWX2H2dfzZF9AROVjwcNgMf/FeOTLd42k2iJKJP9RhkTACHRYF+hbKk8c3NlVsYFgZB1IfWJBcr1Hi4pSjSeUHetsMpYWYQVpEkRJWsdMi/YGhdVZmm368H3g/wBs9A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JgqZ5M7Wo0ZrUjxL5jwGvpYqtkYP3LU+tdVvmSyFf28=;
 b=OLVLtVJSev1S7COvmggvFIb0X30ECsHOt1Hwl6fYNzi4BqqfmihmerHifZEbeWwEt5MmL8MV6VLPcBMqm2y3qH4NP8AkPFNuYWrQ3GIO85DleTUhkYxioPRE044rsL5i8wvB+MnDPzZDm+ZBxyRtR7bytSCAZ+00ia/+GOou5f/FdVNnoydxM/Pzegqsy4NTLRLIkBFwY6SRo32E1LXx+lI6s3xvaosUY8g/g+LEpuKzQ1h4rkDXIbkYd2MDYn72QLREHMXt2FAlFDukyfLci0Gwq6Dojz79aZYtZYf7EPqIGT0OdQc8K8zdWEdixJQz2RVcDccnExtmraDP0KVLnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JgqZ5M7Wo0ZrUjxL5jwGvpYqtkYP3LU+tdVvmSyFf28=;
 b=g7Rogwk95bJ2Nx6W13TMY9i5+yNQ3d2O/g4RdoLuxRyoNAFK/Sxhp1+i/tXj5KzuiZFyZ06gfwMxyQubXzJllvTwYTcb3/LlBqu/jXZD1XUSPvy1ebe/vrI9DMy+dDzFf6dETPbqo2zsH161xTfSY5X93ftYiB39UDKkK6xPkNOF9gTeZkst8UqyD7L2ZkHdlrKYNlpmBAd8OB5zCRXnRc28nciO6UuYk7LUOR+tIG1XFhLRc8ObPXAFoZWtLSqEeK7yeRy6r466yrGQ36LIaod6aeVr9npDDbozYQV2x0V5HSqykE+N4DSIJBqMlmv3qMZmGUj05TB4G5fOZXNfKQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        "jgross@suse.com"
	<jgross@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Oleksandr
 Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
Thread-Index: AQHXr5qiwVuFg9DS+ESEBHonrZ7zRquwjVWAgADGyoA=
Date: Thu, 23 Sep 2021 09:02:00 +0000
Message-ID: <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>
References: <20210922101422.2319240-1-andr2000@gmail.com>
 <20210922101422.2319240-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f537de7f-f0f4-428b-b893-08d97e70cdd3
x-ms-traffictypediagnostic: AM9PR03MB7489:
x-microsoft-antispam-prvs: 
 <AM9PR03MB74893DF73B20A534FDA8AE25E7A39@AM9PR03MB7489.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 a+Y0NNwf2PDAG5joQXGQhG6a2inh9QLs6Mpo0vnjsCUVC5hW/zMqacv9PavHgeYjzvECbkoavxEuNgv56pQaCcZNdHFUtMWSb7sL3Tpz1H7Rj4pRQgHiF2o/GfwJFsLVj193ngmVzt/JbZGNyLqKW1hJ76YEorBXxOaUsIFuRnen0xT4qxU0nnsAwhIvCwHChAJMP9n5gmHz8KFozyTMWnI95nJrRBUBHzPIzBJxuUPvYieOHLooQc6e4AoiA6+LRoF7cMxH+2uJVzG7IeWe95WOCEGW0gp8L7T3tJoyO8yuc/upOkwk5sqFVA0JHm69g9bx8fVOlJfzaL0r6cs+xkYKp1oyXwLGkeMtAgJyAMo4tTO7CJZnAJSWz2xEXbbNSEiZOFE23SBYMsx+92u0aSxCmr8y9Is0+GpcNoy8GHhnvaL311r3pN39PhuJEjA0LSwsEPAcv18ZFYxAfpvMfQBr3Ozib5JcOLB+OZpgLJeY2jw4owkOYYjKISAd6qMWXYEvT/hpGE0lAJYrHkjleu06pVGAheKkw2SX8ZSKE3FbxRf/mXmZc5oJ46hAnvm/ElfmtL1HbP/A14O61rml6SNyJ8b8A09gnJxdJGtlb3oRJmyzrAKjpcRV/7ijWg3CRLQ8avdTfWxJghMrX2CB8pR937aXcDsDlBbhoevVdSDLbdlIIw4PyQA5Th00kQ8XZUu+aT45VWu8aiuiU3HM5zFr6neHZ1+XJOjIDkPOce1qb/ZR8gBWlyqgmUxUFaV1
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(36756003)(316002)(6512007)(110136005)(186003)(4326008)(66476007)(66946007)(64756008)(66446008)(66556008)(76116006)(54906003)(71200400001)(8936002)(53546011)(6486002)(38070700005)(8676002)(2616005)(508600001)(31696002)(31686004)(86362001)(6506007)(2906002)(122000001)(5660300002)(83380400001)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?VnoySGtPNVdORXFTblBPZXo0Sm1rY0V1ZElpcHlBTDJ2Z3huOE1VZ0ZYaVhU?=
 =?utf-8?B?U1pDMkpvdUpIem1QWUZ1M0ZNT2dIQVcvRkdiM2tWVUFLTWhzeDZTdEQ2dmpG?=
 =?utf-8?B?N1prUUJMUDEyMENsOStxNzRXRnlZOERPRktDbzRjc1dqOHZJTHFZMmp4TStG?=
 =?utf-8?B?WHpaMHlMSW9KOExmZ1l5aWFrdGNLWnFRVDRXS1EwRklOaVp2eHdrY3l1ck8r?=
 =?utf-8?B?YUU3bWZCdEN6NmdKbHFJNXk4eUtMcGdGdlRXL1I2OXo1OUVVUE5sSjVEcnNk?=
 =?utf-8?B?U0lHL1J1UXMzc0d5SXNWaUtpZ3Y2QzhRbkEvb3lVajBoT054bGswQlhoS1Nw?=
 =?utf-8?B?YnNxQzMrUWFEZWc0bDJITWhGWmxRZkhickxXZ1czRUs1V1dQSnk1ZVBJMkNV?=
 =?utf-8?B?YUVwU3haQkl3S2t1YWR5VElKK1pvN1pHWEE3Z0w5S2dwVzlCWDBNTC93WjYz?=
 =?utf-8?B?dTlGQ1F1Z2ZqZ0RCc3gwaGVtQVhKSW9BMzBDQU5GY0szRDFlV2E4NHlTUnlC?=
 =?utf-8?B?QjdqMm9xNldoQk13UTJNN2thUXYzUHczNnkxRFo5cW5qUWFrdXNub1RveVBt?=
 =?utf-8?B?by91TTNsaDY5Q21wS2FZY0VMam1Hb2Z0Uy9yWHM4bE8wS0p3bVFoMXlpRG9n?=
 =?utf-8?B?Qk1Qa0ZjdmFZVkxwV296Um9aQXRRRXhvT3hZYXlHUDZsbE5STi9Tai9neGN3?=
 =?utf-8?B?amt0ci9oTG5RZU8yUy8yMUQxNDREU0pmUXRseWxlS2hHSElSL2tydUo5UnQ4?=
 =?utf-8?B?RHdOVlRlMEpTa1IxWVdlaUgzUEd4RmdheXFTLzB2Wk0yZnhXcGpVZGFqWStx?=
 =?utf-8?B?RVQ5eXo3TENHNU9SakZweGhaNjFGbTJGNXVNcXkxbW5yTUdxZEpmY0pybnZx?=
 =?utf-8?B?UlRMZ1dLUHV3VnFXbGYxdzB0UXVQTUdmNnJvVzBFeGx4andHNHovWWFTaUNO?=
 =?utf-8?B?RzRmNTB0UElnNitnakg0UTE4NG5CRmJGaXZSTDdJbjhzZGJ4aVcrcXRJR2l4?=
 =?utf-8?B?RTNQa0dVUEZtcHhHYXZEYlpML3JRN3NscTdUeFFNSTdnNVc3NFlBdUdmdFFu?=
 =?utf-8?B?T3J2clhxNWdtRERqRkljYVV1NU9kRFN4Sm9nMmRtanF0dVpGeEdmWEVQS1J1?=
 =?utf-8?B?ZHFwTmVIV2lUd2Jzek5ZWFgwcjNBdVprWmNIRGV4aVFob0U3ZWNoUHdtMDlV?=
 =?utf-8?B?MUlvWVhSSVBBUDJ4V1RKejR5aUlNWFY1UG9IUkM3cWJGNXIvbU42K3lhVlpG?=
 =?utf-8?B?blo2Z0xtYm01WjJZSzdpVFNTOEtsdGpQZXB3MU5BTWlSRDEza3YxMlpDZjUr?=
 =?utf-8?B?KzF0WkpUcEZSRFh1OUpNZTAvMUxrc216Tk0rVGN5UU5rWlJDcDhsMEtWVXZO?=
 =?utf-8?B?S1BtMGdUMkFIQzVhUzl0ZXVTQSs1bzcvOEZGZTkwd0gxYmRiRkFKUjkydElC?=
 =?utf-8?B?MXcrK1dRNmVMRENOQzdnejJaVTBBaFBtVW56eFE1VWY4eEFjSS9od2s5OUw0?=
 =?utf-8?B?NnIvakM1VFdlZGpDcU1yY3V4S0RPOStBSzlFeUhvaEV4cUhucS92Z21obEhZ?=
 =?utf-8?B?S3cwekhKbkZ2Ky83elBPR1ZEa25haEJPODNrTW5QL1NjaU5Ua2FUdGROdkNw?=
 =?utf-8?B?YmhORi9CQndxQzFDdmVuQ1NQMHJZVmlmQTlJcTV3dkN0VHJWWkg5Yjc2WDdN?=
 =?utf-8?B?Z0hTRWVIUjhZeGtCWTBUWGN6UFVZY2ZFVEY1bXhDbE9aWlBjdHdBT2ZWMkM5?=
 =?utf-8?Q?EkKw/Q+xpdmepbwYRIjr35SChIKUeggAZ5fnqP3?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <D0432EEE3FF88F4D8F9303EBCC48165F@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f537de7f-f0f4-428b-b893-08d97e70cdd3
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 09:02:00.3421
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /R+X5HB+w5QvqitR0TSYnFb3nAkKgng/20mzHT2lFP1KriI7Lr3lwH+Vij0ync5K+pMwMzVbrQEL/lEf5PygXvaMhMK+AbpubIyUSF+0Ss6kxHDPaEECVEbAJHiUyq3p
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7489
X-Proofpoint-ORIG-GUID: IfYYh2Vf5VhTqgRPFrRdQ2AoveT1ZE72
X-Proofpoint-GUID: IfYYh2Vf5VhTqgRPFrRdQ2AoveT1ZE72
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_03,2021-09-22_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 mlxlogscore=999 phishscore=0 mlxscore=0 priorityscore=1501
 lowpriorityscore=0 adultscore=0 impostorscore=0 bulkscore=0 malwarescore=0
 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230056

DQpPbiAyMy4wOS4yMSAwMDoxMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBXZWQs
IDIyIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBDdXJyZW50bHkgUENJIGJhY2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlv
bmFsaXRpZXMgYXQgYSB0aW1lLg0KPj4gVG8gbmFtZSBhIGZldzoNCj4+IDEuIGl0IGlzIHVzZWQg
YXMgYSBkYXRhYmFzZSBmb3IgYXNzaWduYWJsZSBQQ0kgZGV2aWNlcywgZS5nLiB4bA0KPj4gICAg
IHBjaS1hc3NpZ25hYmxlLXthZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQgbGlzdC4g
U28sIHdoZW5ldmVyDQo+PiAgICAgdGhlIHRvb2xzdGFjayBuZWVkcyB0byBrbm93IHdoaWNoIFBD
SSBkZXZpY2VzIGNhbiBiZSBwYXNzZWQgdGhyb3VnaA0KPj4gICAgIGl0IHJlYWRzIHRoYXQgZnJv
bSB0aGUgcmVsZXZhbnQgc3lzZnMgZW50cmllcyBvZiB0aGUgcGNpYmFjay4NCj4+IDIuIGl0IGlz
IHVzZWQgdG8gaG9sZCB0aGUgdW5ib3VuZCBQQ0kgZGV2aWNlcyBsaXN0LCBlLmcuIHdoZW4gcGFz
c2luZw0KPj4gICAgIHRocm91Z2ggYSBQQ0kgZGV2aWNlIGl0IG5lZWRzIHRvIGJlIHVuYm91bmQg
ZnJvbSB0aGUgcmVsZXZhbnQgZGV2aWNlDQo+PiAgICAgZHJpdmVyIGFuZCBib3VuZCB0byBwY2li
YWNrIChzdHJpY3RseSBzcGVha2luZyBpdCBpcyBub3QgcmVxdWlyZWQNCj4+ICAgICB0aGF0IHRo
ZSBkZXZpY2UgaXMgYm91bmQgdG8gcGNpYmFjaywgYnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBh
cyBhDQo+PiAgICAgZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSSBkZXZpY2VzLCBz
byB3ZSBjYW4gcmUtYmluZCB0aGUNCj4+ICAgICBkZXZpY2VzIGJhY2sgdG8gdGhlaXIgb3JpZ2lu
YWwgZHJpdmVycyB3aGVuIGd1ZXN0IGRvbWFpbiBzaHV0cyBkb3duKQ0KPj4gMy4gZGV2aWNlIHJl
c2V0IGZvciB0aGUgZGV2aWNlcyBiZWluZyBwYXNzZWQgdGhyb3VnaA0KPj4gNC4gcGFyYS12aXJ0
dWFsaXplZCB1c2UtY2FzZXMgc3VwcG9ydA0KPj4NCj4+IFRoZSBwYXJhLXZpcnR1YWxpemVkIHBh
cnQgb2YgdGhlIGRyaXZlciBpcyBub3QgYWx3YXlzIG5lZWRlZCBhcyBzb21lDQo+PiBhcmNoaXRl
Y3R1cmVzLCBlLmcuIEFybSBvciB4ODYgUFZIIERvbTAsIGFyZSBub3QgdXNpbmcgYmFja2VuZC1m
cm9udGVuZA0KPj4gbW9kZWwgZm9yIFBDSSBkZXZpY2UgcGFzc3Rocm91Z2guIEZvciBzdWNoIHVz
ZS1jYXNlcyBtYWtlIHRoZSB2ZXJ5DQo+PiBmaXJzdCBzdGVwIGluIHNwbGl0dGluZyB0aGUgeGVu
LXBjaWJhY2sgZHJpdmVyIGludG8gdHdvIHBhcnRzOiBleHRlbmRlZA0KPj4gUENJIHN0dWIgYW5k
IFBDSSBQViBiYWNrZW5kIGRyaXZlcnMuIEF0IHRoZSBtb21lbnQgeDg2IHBsYXRmb3JtIHdpbGwN
Cj4+IGNvbnRpbnVlIHVzaW5nIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQgZm9yIHRoZSBmdWxs
eSBmZWF0dXJlZCBiYWNrZW5kDQo+PiBkcml2ZXIgYW5kIG5ldyBwbGF0Zm9ybXMgbWF5IGJ1aWxk
IGEgZHJpdmVyIHdpdGggbGltaXRlZCBmdW5jdGlvbmFsaXR5DQo+PiAobm8gUFYpIGJ5IGVuYWJs
aW5nIENPTkZJR19YRU5fUENJREVWX1NUVUIuDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPiBQ
bGVhc2UgbWFrZSB0aGlzIHRoZSBmaXJzdCBwYXRjaCBvZiB0aGUgc2VyaWVzIHNvIHRoYXQuLi4N
Cj4NCj4+IC0tLQ0KPj4gTmV3IGluIHYyDQo+PiAtLS0NCj4+ICAgZHJpdmVycy94ZW4vS2NvbmZp
ZyAgICAgICAgICAgICAgIHwgMjYgKysrKysrKysrKysrKysrKysrKysrKysrKy0NCj4+ICAgZHJp
dmVycy94ZW4vTWFrZWZpbGUgICAgICAgICAgICAgIHwgIDIgKy0NCj4+ICAgZHJpdmVycy94ZW4v
eGVuLXBjaWJhY2svTWFrZWZpbGUgIHwgIDEgKw0KPj4gICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9wY2liYWNrLmggfCAgNSArKysrKw0KPj4gICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5i
dXMuYyAgfCAgNiArKysrKy0NCj4+ICAgNSBmaWxlcyBjaGFuZ2VkLCAzNyBpbnNlcnRpb25zKCsp
LCAzIGRlbGV0aW9ucygtKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi9LY29uZmln
IGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4gaW5kZXggMDU3ZGRmNjFlZjYxLi42ZTkyYzZiZTE5
ZjEgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi9LY29uZmlnDQo+PiArKysgYi9kcml2ZXJz
L3hlbi9LY29uZmlnDQo+PiBAQCAtMTgwLDEwICsxODAsMzQgQEAgY29uZmlnIFNXSU9UTEJfWEVO
DQo+PiAgIAlzZWxlY3QgRE1BX09QUw0KPj4gICAJc2VsZWN0IFNXSU9UTEINCj4+ICAgDQo+PiAr
Y29uZmlnIFhFTl9QQ0lfU1RVQg0KPj4gKwlib29sDQo+PiArDQo+PiArY29uZmlnIFhFTl9QQ0lE
RVZfU1RVQg0KPj4gKwl0cmlzdGF0ZSAiWGVuIFBDSS1kZXZpY2Ugc3R1YiBkcml2ZXIiDQo+PiAr
CWRlcGVuZHMgb24gUENJICYmICFYODYgJiYgWEVODQo+PiArCWRlcGVuZHMgb24gWEVOX0JBQ0tF
TkQNCj4+ICsJc2VsZWN0IFhFTl9QQ0lfU1RVQg0KPj4gKwlkZWZhdWx0IG0NCj4+ICsJaGVscA0K
Pj4gKwkgIFRoZSBQQ0kgZGV2aWNlIHN0dWIgZHJpdmVyIHByb3ZpZGVzIGxpbWl0ZWQgdmVyc2lv
biBvZiB0aGUgUENJDQo+PiArCSAgZGV2aWNlIGJhY2tlbmQgZHJpdmVyIHdpdGhvdXQgcGFyYS12
aXJ0dWFsaXplZCBzdXBwb3J0IGZvciBndWVzdHMuDQo+PiArCSAgSWYgeW91IHNlbGVjdCB0aGlz
IHRvIGJlIGEgbW9kdWxlLCB5b3Ugd2lsbCBuZWVkIHRvIG1ha2Ugc3VyZSBubw0KPj4gKwkgIG90
aGVyIGRyaXZlciBoYXMgYm91bmQgdG8gdGhlIGRldmljZShzKSB5b3Ugd2FudCB0byBtYWtlIHZp
c2libGUgdG8NCj4+ICsJICBvdGhlciBndWVzdHMuDQo+PiArDQo+PiArCSAgVGhlICJoaWRlIiBw
YXJhbWV0ZXIgKG9ubHkgYXBwbGljYWJsZSBpZiBiYWNrZW5kIGRyaXZlciBpcyBjb21waWxlZA0K
Pj4gKwkgIGludG8gdGhlIGtlcm5lbCkgYWxsb3dzIHlvdSB0byBiaW5kIHRoZSBQQ0kgZGV2aWNl
cyB0byB0aGlzIG1vZHVsZQ0KPj4gKwkgIGZyb20gdGhlIGRlZmF1bHQgZGV2aWNlIGRyaXZlcnMu
IFRoZSBhcmd1bWVudCBpcyB0aGUgbGlzdCBvZiBQQ0kgQkRGczoNCj4+ICsJICB4ZW4tcGNpYmFj
ay5oaWRlPSgwMzowMC4wKSgwNDowMC4wKQ0KPj4gKw0KPj4gKwkgIElmIGluIGRvdWJ0LCBzYXkg
bS4NCj4+ICsNCj4+ICAgY29uZmlnIFhFTl9QQ0lERVZfQkFDS0VORA0KPj4gICAJdHJpc3RhdGUg
IlhlbiBQQ0ktZGV2aWNlIGJhY2tlbmQgZHJpdmVyIg0KPj4gLQlkZXBlbmRzIG9uIFBDSSAmJiBY
RU4NCj4+ICsJZGVwZW5kcyBvbiBQQ0kgJiYgWDg2ICYmIFhFTg0KPj4gICAJZGVwZW5kcyBvbiBY
RU5fQkFDS0VORA0KPiAuLi53ZSBkb24ndCBuZWVkIHRoaXMNClllcywgSSBkaWRuJ3QgbGlrZSB0
aGlzIGNoYW5nZSB0b28NCj4NCj4NCj4+ICsJc2VsZWN0IFhFTl9QQ0lfU1RVQg0KPj4gICAJZGVm
YXVsdCBtDQo+PiAgIAloZWxwDQo+PiAgIAkgIFRoZSBQQ0kgZGV2aWNlIGJhY2tlbmQgZHJpdmVy
IGFsbG93cyB0aGUga2VybmVsIHRvIGV4cG9ydCBhcmJpdHJhcnkNCj4+IGRpZmYgLS1naXQgYS9k
cml2ZXJzL3hlbi9NYWtlZmlsZSBiL2RyaXZlcnMveGVuL01ha2VmaWxlDQo+PiBpbmRleCAzNDM0
NTkzNDU1YjIuLjVhYWU2NmU2MzhhNyAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL01ha2Vm
aWxlDQo+PiArKysgYi9kcml2ZXJzL3hlbi9NYWtlZmlsZQ0KPj4gQEAgLTI0LDcgKzI0LDcgQEAg
b2JqLSQoQ09ORklHX1hFTl9TWVNfSFlQRVJWSVNPUikJKz0gc3lzLWh5cGVydmlzb3Iubw0KPj4g
ICBvYmotJChDT05GSUdfWEVOX1BWSFZNX0dVRVNUKQkJKz0gcGxhdGZvcm0tcGNpLm8NCj4+ICAg
b2JqLSQoQ09ORklHX1NXSU9UTEJfWEVOKQkJKz0gc3dpb3RsYi14ZW4ubw0KPj4gICBvYmotJChD
T05GSUdfWEVOX01DRV9MT0cpCQkrPSBtY2Vsb2cubw0KPj4gLW9iai0kKENPTkZJR19YRU5fUENJ
REVWX0JBQ0tFTkQpCSs9IHhlbi1wY2liYWNrLw0KPj4gK29iai0kKENPTkZJR19YRU5fUENJX1NU
VUIpCSAgICAgICAgKz0geGVuLXBjaWJhY2svDQo+PiAgIG9iai0kKENPTkZJR19YRU5fUFJJVkNN
RCkJCSs9IHhlbi1wcml2Y21kLm8NCj4+ICAgb2JqLSQoQ09ORklHX1hFTl9BQ1BJX1BST0NFU1NP
UikJKz0geGVuLWFjcGktcHJvY2Vzc29yLm8NCj4+ICAgb2JqLSQoQ09ORklHX1hFTl9FRkkpCQkJ
Kz0gZWZpLm8NCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmls
ZSBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+PiBpbmRleCBlOGQ5ODFkNDMy
MzUuLmUyY2IzNzY0NDRhNiAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNr
L01ha2VmaWxlDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4g
QEAgLTEsNSArMSw2IEBADQo+PiAgICMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAN
Cj4+ICAgb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCkgKz0geGVuLXBjaWJhY2subw0K
Pj4gK29iai0kKENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4+ICAg
DQo+PiAgIHhlbi1wY2liYWNrLXkgOj0gcGNpX3N0dWIubyBwY2liYWNrX29wcy5vIHhlbmJ1cy5v
DQo+PiAgIHhlbi1wY2liYWNrLXkgKz0gY29uZl9zcGFjZS5vIGNvbmZfc3BhY2VfaGVhZGVyLm8g
XA0KPj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaWJhY2suaCBiL2Ry
aXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaWJhY2suaA0KPj4gaW5kZXggOTVlMjhlZTQ4ZDUyLi45
YTY0MTk2ZTgzMWQgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2li
YWNrLmgNCj4+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaWJhY2suaA0KPj4gQEAg
LTcxLDYgKzcxLDExIEBAIHN0cnVjdCBwY2lfZGV2ICpwY2lzdHViX2dldF9wY2lfZGV2KHN0cnVj
dCB4ZW5fcGNpYmtfZGV2aWNlICpwZGV2LA0KPj4gICAJCQkJICAgIHN0cnVjdCBwY2lfZGV2ICpk
ZXYpOw0KPj4gICB2b2lkIHBjaXN0dWJfcHV0X3BjaV9kZXYoc3RydWN0IHBjaV9kZXYgKmRldik7
DQo+PiAgIA0KPj4gK3N0YXRpYyBpbmxpbmUgYm9vbCB4ZW5fcGNpYmtfcHZfc3VwcG9ydCh2b2lk
KQ0KPj4gK3sNCj4+ICsJcmV0dXJuIElTX0VOQUJMRUQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VO
RCk7DQo+PiArfQ0KPj4gKw0KPj4gICAvKiBFbnN1cmUgYSBkZXZpY2UgaXMgdHVybmVkIG9mZiBv
ciByZXNldCAqLw0KPj4gICB2b2lkIHhlbl9wY2lia19yZXNldF9kZXZpY2Uoc3RydWN0IHBjaV9k
ZXYgKnBkZXYpOw0KPj4gICANCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay94ZW5idXMuYyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+PiBpbmRleCBk
YTM0Y2U4NWRjODguLmJkZTYzZWY2NzdiOCAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hl
bi1wY2liYWNrL3hlbmJ1cy5jDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5i
dXMuYw0KPj4gQEAgLTc0Myw2ICs3NDMsOSBAQCBjb25zdCBzdHJ1Y3QgeGVuX3BjaWJrX2JhY2tl
bmQgKl9fcmVhZF9tb3N0bHkgeGVuX3BjaWJrX2JhY2tlbmQ7DQo+PiAgIA0KPj4gICBpbnQgX19p
bml0IHhlbl9wY2lia194ZW5idXNfcmVnaXN0ZXIodm9pZCkNCj4+ICAgew0KPj4gKwlpZiAoIXhl
bl9wY2lia19wdl9zdXBwb3J0KCkpDQo+PiArCQlyZXR1cm4gMDsNCj4gSXMgdGhpcyB0cnVseSBl
bm91Z2ggdG8gc3RvcCB0aGUgUFYgYmFja2VuZCBmcm9tIGluaXRpYWxpemluZz8gSGF2ZSB5b3UN
Cj4gYWN0dWFsbHkgdGVzdGVkIGl0IHRvIG1ha2Ugc3VyZT8gSWYgaXQgd29ya3MsIGFtYXppbmch
IEkgYW0gcXVpdGUgaGFwcHkNCj4gYWJvdXQgdGhpcyBhcHByb2FjaCA6LSkNCg0KV2VsbCwgSSBw
dXQgc29tZSBsb2dzIGludG8gdGhlIGRyaXZlciBhbmQgc2F3IG5vdGhpbmcgb2J2aW91cyBwb2lu
dGluZw0KDQp0byBhbnkgYmFja2VuZCBhY3Rpdml0aWVzIChwcm9iYWJseSB0aGlzIGlzIGFsc28g
YmVjYXVzZSBJIGRvbid0IGhhdmUgYW55DQoNCmZyb250ZW5kKS4gSSBzZWUgdGhhdCB0aGUgeGVu
YnVzIGRyaXZlciBpcyBub3QgcmVnaXN0ZXJlZC4gSW4gWGVuU3RvcmUgSSBzZWU6DQoNCnJvb3RA
ZG9tMDp+IyB4ZW5zdG9yZS1scyAtZiB8IGdyZXAgcGNpDQovbG9jYWwvZG9tYWluLzAvYmFja2Vu
ZC9wY2kgPSAiIg0KL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIgPSAiIg0KL2xvY2FsL2Rv
bWFpbi8wL2JhY2tlbmQvcGNpLzIvMCA9ICIiDQovbG9jYWwvZG9tYWluLzAvYmFja2VuZC9wY2kv
Mi8wL2Zyb250ZW5kID0gIi9sb2NhbC9kb21haW4vMi9kZXZpY2UvcGNpLzAiDQovbG9jYWwvZG9t
YWluLzAvYmFja2VuZC9wY2kvMi8wL2Zyb250ZW5kLWlkID0gIjIiDQovbG9jYWwvZG9tYWluLzAv
YmFja2VuZC9wY2kvMi8wL29ubGluZSA9ICIxIg0KL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNp
LzIvMC9zdGF0ZSA9ICIxIg0KL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9kb21haW4g
PSAiRG9tVSINCi9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3BjaS8yLzAva2V5LTAgPSAiMDAwMDow
MzowMC4wIg0KL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9kZXYtMCA9ICIwMDAwOjAz
OjAwLjAiDQovbG9jYWwvZG9tYWluLzAvYmFja2VuZC9wY2kvMi8wL29wdHMtMCA9ICJtc2l0cmFu
c2xhdGU9MCxwb3dlcl9tZ210PTAscGVybWlzc2l2ZT0wLHJkbV9wb2xpY3k9c3RyaWN0Ig0KL2xv
Y2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9zdGF0ZS0wID0gIjEiDQovbG9jYWwvZG9tYWlu
LzAvYmFja2VuZC9wY2kvMi8wL251bV9kZXZzID0gIjEiDQovbG9jYWwvZG9tYWluLzIvZGV2aWNl
L3BjaSA9ICIiDQovbG9jYWwvZG9tYWluLzIvZGV2aWNlL3BjaS8wID0gIiINCi9sb2NhbC9kb21h
aW4vMi9kZXZpY2UvcGNpLzAvYmFja2VuZCA9ICIvbG9jYWwvZG9tYWluLzAvYmFja2VuZC9wY2kv
Mi8wIg0KL2xvY2FsL2RvbWFpbi8yL2RldmljZS9wY2kvMC9iYWNrZW5kLWlkID0gIjAiDQovbG9j
YWwvZG9tYWluLzIvZGV2aWNlL3BjaS8wL3N0YXRlID0gIjEiDQovbGlieGwvcGNpID0gIiINCi9s
aWJ4bC9wY2kvMDAwMC0wMy0wMC0wID0gIiINCi9saWJ4bC9wY2kvMDAwMC0wMy0wMC0wL2RvbWlk
ID0gIjIiDQoNCkJ1dCBJSVVJQyB0aGVzZSBjb21lIGZyb20gdGhlIHRvb2xzdGFjaw0KDQpASnVl
cmdlbiwgZG8geW91IGtub3cgaG93IHRvIGNoZWNrIGlmIHRoZSBiYWNrZW5kIGlzIGluZGVlZCBu
b3QgcnVubmluZw0KDQpvciB0aGUgYWJvdmUgc2hvdWxkIGJlIGVub3VnaCB0byBwcm92ZT8NCg0K
DQo+DQo+DQo+DQo+DQo+PiAgIAl4ZW5fcGNpYmtfYmFja2VuZCA9ICZ4ZW5fcGNpYmtfdnBjaV9i
YWNrZW5kOw0KPj4gICAJaWYgKHBhc3N0aHJvdWdoKQ0KPj4gICAJCXhlbl9wY2lia19iYWNrZW5k
ID0gJnhlbl9wY2lia19wYXNzdGhyb3VnaF9iYWNrZW5kOw0KPj4gQEAgLTc1Miw1ICs3NTUsNiBA
QCBpbnQgX19pbml0IHhlbl9wY2lia194ZW5idXNfcmVnaXN0ZXIodm9pZCkNCj4+ICAgDQo+PiAg
IHZvaWQgX19leGl0IHhlbl9wY2lia194ZW5idXNfdW5yZWdpc3Rlcih2b2lkKQ0KPj4gICB7DQo+
PiAtCXhlbmJ1c191bnJlZ2lzdGVyX2RyaXZlcigmeGVuX3BjaWJrX2RyaXZlcik7DQo+PiArCWlm
ICh4ZW5fcGNpYmtfcHZfc3VwcG9ydCgpKQ0KPj4gKwkJeGVuYnVzX3VucmVnaXN0ZXJfZHJpdmVy
KCZ4ZW5fcGNpYmtfZHJpdmVyKTsNCj4+ICAgfQ0KPj4gLS0gDQo+PiAyLjI1LjENCj4+DQpUaGFu
ayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:03:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:03:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193532.344717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKdA-0005xd-DC; Thu, 23 Sep 2021 09:03:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193532.344717; Thu, 23 Sep 2021 09:03:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKdA-0005xW-AJ; Thu, 23 Sep 2021 09:03:00 +0000
Received: by outflank-mailman (input) for mailman id 193532;
 Thu, 23 Sep 2021 09:02:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKd9-0005xI-N1
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:02:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKd9-0003oW-ER; Thu, 23 Sep 2021 09:02:59 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKd9-0000Ap-4Q; Thu, 23 Sep 2021 09:02:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=HfN1IGFIrJppc5fvqGb+ktVGm+nPKJ2a+PRtFdhGhM4=; b=pKDkULY9qDol/kEkgunI2Lfgce
	Sn6I5076r1Czj87o0mIlxkHCH87ytn5GBxVuMi8fc8XK6xv2tBZXeDrU6CziueC6oXoShp8oPC1ey
	aupHcQtbdrrripkKwkM9AifBUn0gqNezLpMt5tUIaTEnD8fq/xlc8SSGohLuhODJJYrA=;
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 andre.przywara@arm.com, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien@xen.org>
Message-ID: <bcb7c35b-cad1-8da1-8723-82f1f79d4f74@xen.org>
Date: Thu, 23 Sep 2021 14:02:54 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 23/09/2021 07:23, Stefano Stabellini wrote:
> Subject should have xen/arm
> 
> 
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Implement generic pci access functions to read/write the configuration
>> space.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2: Fixed comments
>> ---
>>   xen/arch/arm/pci/pci-access.c      | 58 ++++++++++++++++++++++++++++++
>>   xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
>>   xen/include/asm-arm/pci.h          |  2 ++
>>   3 files changed, 79 insertions(+)
>>
>> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
>> index 04fe9fbf92..45500cec2a 100644
>> --- a/xen/arch/arm/pci/pci-access.c
>> +++ b/xen/arch/arm/pci/pci-access.c
>> @@ -16,6 +16,7 @@
>>   #include <asm/io.h>
>>   
>>   #define INVALID_VALUE (~0U)
>> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>>   
>>   int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
>>                               uint32_t reg, uint32_t len, uint32_t *value)
>> @@ -72,6 +73,63 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>>       return 0;
>>   }
>>   
>> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>> +                                unsigned int len)
>> +{
>> +    uint32_t val = PCI_ERR_VALUE(len);
>> +
> 
> No blank line
> 
> 
>> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
>> +
>> +    if ( unlikely(!bridge) )
>> +        return val;
>> +
>> +    if ( unlikely(!bridge->ops->read) )
>> +        return val;
>> +
>> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);
> 
> The more I look at these casts the less I like them :-D

I really dislike them. This is kind of defeating the purpose of trying 
to be more typesafe.

> 
> One idea is to move the definition of pci_sbdf_t somewhere else
> entirely, for instance xen/include/xen/types.h, then we can use
> pci_sbdf_t everywhere

AFAIU, the problem is the prototype helpers are defined in asm/pci.h 
which is included by xen/pci.h before defining sbdf_t. Is it correct?

If so there are two options:
   1) define sbdf_t and then include asm/pci.h.
   2) Name the union and then pre-declare it.

Option 1 is probably nicer is we have more types in the future that are 
used by arch specific but defined in the common headers. We have a few 
places that uses this approach.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:05:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:05:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193538.344729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKfD-0006dB-Ro; Thu, 23 Sep 2021 09:05:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193538.344729; Thu, 23 Sep 2021 09:05:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKfD-0006d4-No; Thu, 23 Sep 2021 09:05:07 +0000
Received: by outflank-mailman (input) for mailman id 193538;
 Thu, 23 Sep 2021 09:05:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTKfC-0006cw-3s
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:05:06 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b18149d-86b2-4347-a0e7-0c03ee8279bf;
 Thu, 23 Sep 2021 09:05:05 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5525922346;
 Thu, 23 Sep 2021 09:05:04 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1C26413DCA;
 Thu, 23 Sep 2021 09:05:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id H9zeBEBDTGE/MQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 09:05:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b18149d-86b2-4347-a0e7-0c03ee8279bf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632387904; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=SlpT/lxAM3jzsI+Mgdob12kTvoXiVmms10P//QNB5AM=;
	b=iJTsPXCZYqt7+JPZVlPz+d97attND1bUg4UgMjSGHD6dztJbvcmzF6AFh5gEbvKFmXg0uJ
	lCq2nlY6Q4mbQ43BR52LvI0eD26NVg+7csnAhVBROg2WN95lA0HbDAAyOxYsz1KQJnjw5N
	pJ7ELKTtB5XYS6h387+HTsjodpvSzc0=
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210922101422.2319240-1-andr2000@gmail.com>
 <20210922101422.2319240-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
 <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <7880dee9-7372-5a25-db55-018f21e8b08c@suse.com>
Date: Thu, 23 Sep 2021 11:05:03 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Raagvfdh0L7U0w5qqCjIvjKl35tyxY7y3"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Raagvfdh0L7U0w5qqCjIvjKl35tyxY7y3
Content-Type: multipart/mixed; boundary="GNX9GoSgpMQWceOML5nwDmElT2X4pKTAT";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Message-ID: <7880dee9-7372-5a25-db55-018f21e8b08c@suse.com>
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
References: <20210922101422.2319240-1-andr2000@gmail.com>
 <20210922101422.2319240-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
 <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>
In-Reply-To: <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>

--GNX9GoSgpMQWceOML5nwDmElT2X4pKTAT
Content-Type: multipart/mixed;
 boundary="------------FEAB4E1D867B0D1D35FC67BF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------FEAB4E1D867B0D1D35FC67BF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 11:02, Oleksandr Andrushchenko wrote:
>=20
> On 23.09.21 00:10, Stefano Stabellini wrote:
>> On Wed, 22 Sep 2021, Oleksandr Andrushchenko wrote:
>>> --- a/drivers/xen/xen-pciback/xenbus.c
>>> +++ b/drivers/xen/xen-pciback/xenbus.c
>>> @@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen=
_pcibk_backend;
>>>   =20
>>>    int __init xen_pcibk_xenbus_register(void)
>>>    {
>>> +	if (!xen_pcibk_pv_support())
>>> +		return 0;
>> Is this truly enough to stop the PV backend from initializing? Have yo=
u
>> actually tested it to make sure? If it works, amazing! I am quite happ=
y
>> about this approach :-)
>=20
> Well, I put some logs into the driver and saw nothing obvious pointing
>=20
> to any backend activities (probably this is also because I don't have a=
ny
>=20
> frontend). I see that the xenbus driver is not registered. In XenStore =
I see:
>=20
> root@dom0:~# xenstore-ls -f | grep pci
> /local/domain/0/backend/pci =3D ""
> /local/domain/0/backend/pci/2 =3D ""
> /local/domain/0/backend/pci/2/0 =3D ""
> /local/domain/0/backend/pci/2/0/frontend =3D "/local/domain/2/device/pc=
i/0"
> /local/domain/0/backend/pci/2/0/frontend-id =3D "2"
> /local/domain/0/backend/pci/2/0/online =3D "1"
> /local/domain/0/backend/pci/2/0/state =3D "1"
> /local/domain/0/backend/pci/2/0/domain =3D "DomU"
> /local/domain/0/backend/pci/2/0/key-0 =3D "0000:03:00.0"
> /local/domain/0/backend/pci/2/0/dev-0 =3D "0000:03:00.0"
> /local/domain/0/backend/pci/2/0/opts-0 =3D "msitranslate=3D0,power_mgmt=
=3D0,permissive=3D0,rdm_policy=3Dstrict"
> /local/domain/0/backend/pci/2/0/state-0 =3D "1"
> /local/domain/0/backend/pci/2/0/num_devs =3D "1"
> /local/domain/2/device/pci =3D ""
> /local/domain/2/device/pci/0 =3D ""
> /local/domain/2/device/pci/0/backend =3D "/local/domain/0/backend/pci/2=
/0"
> /local/domain/2/device/pci/0/backend-id =3D "0"
> /local/domain/2/device/pci/0/state =3D "1"
> /libxl/pci =3D ""
> /libxl/pci/0000-03-00-0 =3D ""
> /libxl/pci/0000-03-00-0/domid =3D "2"
>=20
> But IIUIC these come from the toolstack
>=20
> @Juergen, do you know how to check if the backend is indeed not running=

>=20
> or the above should be enough to prove?

I don't see how the backend could be running without being registered
with xenbus. It won't receive any watches, so there is no way a
connection with a frontend could be established.


Juergen


--------------FEAB4E1D867B0D1D35FC67BF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------FEAB4E1D867B0D1D35FC67BF--

--GNX9GoSgpMQWceOML5nwDmElT2X4pKTAT--

--Raagvfdh0L7U0w5qqCjIvjKl35tyxY7y3
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMQz8FAwAAAAAACgkQsN6d1ii/Ey95
CAf+LMrSF13F920AxmTtvMQbhQu/0abJ9OnAxlSGWpWb/n/qBvHE80H3GU69e4jRMCxQdru9w4F9
z9IUDYL7w2ypVjWVmKszhCw3kDxP5SwH/SWGEu83cPIyfVW9ew3x0/wGIwEoFEOI/51iKvLIrwcw
00tjR7RsSDAtHtc8zbLo87aYVYwo2J+NeKknTwYBDG2Poo+y9yNq+wrTxYTlPb3PF4TIfmcxm6Wu
ZCAL2Gx0I6iEAp9pj+IOyErh7yCS4twkwP85gF8N/2N2FdC5bCuYIKyqgpCseef833dSmPsQvWIv
kIfkrFqZ0Ph9X716CVtHvtS5I1XDX8TrjFN5tF6k9w==
=mvh6
-----END PGP SIGNATURE-----

--Raagvfdh0L7U0w5qqCjIvjKl35tyxY7y3--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:10:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:10:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193544.344739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKk6-0008GE-EG; Thu, 23 Sep 2021 09:10:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193544.344739; Thu, 23 Sep 2021 09:10:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKk6-0008G7-Aq; Thu, 23 Sep 2021 09:10:10 +0000
Received: by outflank-mailman (input) for mailman id 193544;
 Thu, 23 Sep 2021 09:10:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/6DS=ON=epam.com=prvs=9900839950=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTKk5-0008G1-1i
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:10:09 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0b9e7e20-1c4e-11ec-ba21-12813bfff9fa;
 Thu, 23 Sep 2021 09:10:07 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18N65CAL024936; 
 Thu, 23 Sep 2021 09:10:05 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2051.outbound.protection.outlook.com [104.47.4.51])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b8j7krutu-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 09:10:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7425.eurprd03.prod.outlook.com (2603:10a6:20b:260::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 09:09:56 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 09:09:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b9e7e20-1c4e-11ec-ba21-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ARyM5bd/USn88D76eCPLIFDXD8FunBn/zfkCei5z7RrGOqmss8UP/g5okLOwwuXyibA8lWRAgQROI9vkGAbeqFaL39EojmOJfFWmCEv4ScF7k13bw9et12h/RVGhhJI6z0Px9mXMsFD6X6RCYyzpyVYJWyopqQMgur/GJdO1+EKmW0O0i67UB8chva6ns67ndAsQFY+KmgO8n/C2Cy1b8zlbL8MXFp+J+Ek/fDfdYq295WOfMkPgj2kiyvTNoR578ysntc+RcOowPldQOwOG4zvWdA1DitRwLWKaFPD8jHefNswCp9yazgwMc7QED8GM1wbNck/88UBM0l7pAPoucQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jxQI2uEP5ck7Lt6/B2VRAhJxpExuP54d/COIh2GQwp4=;
 b=DaTyTmGMnOgZvz9KuqUF+U3s30CrVS3TGEuE/zt6GoCb0J2o4WeR45gFPt3JsPHYphzrQ/eJUce/MwLZeoQ/LblfGuoQv+vuFwEDLtK0lrGrl6yhk5X0Oe8eFPSl2E3IJHK3f5OQH1XsToSkwPvhm4VIkw6vWK2vQSK2jzj8tW1a88ngVGnt5l6oZIzKqBoCF2CJBDgKcKOsft0hSJ0+wLtGZKJ4QkD9xGc3pv4HizuFufZZDBIITkAaogyCfXUL47ySTgqTVtMOig/urLxPDn6hQoVOqdz9OZlIlUdWKejT+wGE9Vv35ThPaDnLn+XBRwQRR6xmWpYqC/9lA+5fYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jxQI2uEP5ck7Lt6/B2VRAhJxpExuP54d/COIh2GQwp4=;
 b=l6tHIURxeDsjZJn3vvfddKRHng4O25m7MRj6LT2Xe9JGEDwagkjHCRnaexfRHPd8O4t2fTqnJO9I7frgQQsVbR8uwUnU120DTsTkuP/jQEhvIYF3IPYmuhr0gHOQCnOHl2A2jL3KM7XKVsH5LS1JSP/y5ruRaJ6Fcc41O03+YpeWBhfe5w+ykwZR2JSPdpU2n6AsSNhWM8lyhhJxFgR62aBRkT19pNS3WT3wlggAQKtgEiiZwc26zPGSlDHoZphdeE/PRedZ99ofTiWk3CpKReVUGnSg9FiC/s+ZoBEYr2/hcEUcZOtfnCbEf3j2xIlDS8HjixYZIblZFtrL6eu5oQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Oleksandr
 Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
Thread-Index: AQHXr5qiwVuFg9DS+ESEBHonrZ7zRquwjVWAgADGyoCAAADcgIAAAVyA
Date: Thu, 23 Sep 2021 09:09:56 +0000
Message-ID: <98e432f0-6be1-6394-1c06-ac5e726c708d@epam.com>
References: <20210922101422.2319240-1-andr2000@gmail.com>
 <20210922101422.2319240-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s>
 <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com>
 <7880dee9-7372-5a25-db55-018f21e8b08c@suse.com>
In-Reply-To: <7880dee9-7372-5a25-db55-018f21e8b08c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3e8d5731-b9f9-4bae-b044-08d97e71e985
x-ms-traffictypediagnostic: AM9PR03MB7425:
x-microsoft-antispam-prvs: 
 <AM9PR03MB7425461DD20150C3BB000773E7A39@AM9PR03MB7425.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ZzHOQAwQ1vrA6qbxqPAZPlBuXjiPsTfJ2hY5h595Ibz4zXplB47QH5B3fxOcKPjWkV4nLA8U0VLjwffg3veRVBrJ7BqjOPJ8KYP/HE3bbvmCfYDWp61mBdgt8KVK/RQS81dkx5HGTt4a7mZeYJaXyjMelmYxKuFPi/OBhw/t6ppDlmGi7092xk8g05jYysQB9DOAazzpPn988EU6mOVkfhGe+E/9WUjqUftSKqzEvkTsdx4TvGsoC+tlAnxJ1p8vL5kx92Zyq6rdA5R0lwrKudRxdSINlKnsyCZyeWNyV4CCoGgH8U2dVvUty1S2YH314l1xb261Wm5wo0aEBc3KimDK0z+gzrF74dTKT7rFnsO6/LqcARrKLKDX89nqqHTCxyyducfP/P2z0NtIrerf3IvEqScU/AJ86vhTUxx+fJcUHyGvaVTtXs+22slXPsxnB1/DHm2E1+JfUuQp7+WsuUgnmnaDP8DCgfKFWyq2xMN3JDuiRbTyldezHvyfjEPUWYCwGjCeyn/aSQ+kK3HbfYrN6U6hjkW0VbNMDkvf138/G+em6ozh3sxBS1f4ckaHjDlBz66+jW1tOc4ojMArdpJV4KsczPgyqvbN2x/VPrDh0/+QwosFPO3+q0uoZ1BA9/Rz6M0LI08saSQmFBbWXGZiCp+ciKFn9WLumAE/b3V8HotMYlVzOHY+WACHc1lhNbfEgdkkRhLnNRoPY8ljoBJfDuxPghPZKKUIs78pOp9+OWAiUP7fTBb+7fhq1CY5
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(4326008)(76116006)(66476007)(316002)(66556008)(31696002)(2906002)(6512007)(6486002)(31686004)(66446008)(8676002)(2616005)(122000001)(5660300002)(8936002)(36756003)(64756008)(66946007)(110136005)(26005)(186003)(53546011)(38100700002)(6506007)(71200400001)(86362001)(54906003)(38070700005)(83380400001)(508600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dFRHcFVaYlRiVGhFMDEzM2dMdUlFZmYzODdsTDVLK1NMeTlVZHJ2UWV6YmtG?=
 =?utf-8?B?U3hmZ0ZCQWhpcGxOUDlRMjNRejdnYjdKOUpTZ3ZucHlvTUlOVU0zVFRKV2c1?=
 =?utf-8?B?dkhOUU9hYU14VGxBRk5kclNwUlpVZU9aa1ZwbVRWQTJUbmhGUjZVNkZHanlD?=
 =?utf-8?B?NG9ZbzdDeE44RjdTZEdIOXVqdEpydk81dFI4d20yQWNIMXNDbUI4QXJYWGo1?=
 =?utf-8?B?cXl1REU1T3FpZGlTelhvbXpYV1Q4SVZsTkFBMVVyaXdOaGozdlEwVE1yRTBv?=
 =?utf-8?B?aUV5VmZVa3U0WWZqV3JmVlNla3pIVWZXZzk1UU42SWE5dXRwTHVsdXZiUnNu?=
 =?utf-8?B?MTlaR0ViUkZWTnRTek4yQmpZSVN5VWE4OTFYc2pCV0NNTWJ6djRtVTNiUEs3?=
 =?utf-8?B?di8wNWd0ZjQ4U1BlSzBXYm9QOURLOTVaejRDcHY3Ni80cVhudDF2c0ttOEdB?=
 =?utf-8?B?YTRYeGFRTGVDTzNhMzZVdzR5dzI0N2VjbkluZXoycjRxZWxYN2xNalI5Z0VM?=
 =?utf-8?B?d2pUSWx6R1RodTRIV1ArSkhjaG5hWmcydnFzaUJ1Y1ZyaDl3N0RDOHgvNUcz?=
 =?utf-8?B?VE9YQWpocUFvU3JZN1g5d3lkVUxyclJjR3kvSUkvNkZUelNNcUNmWkJZb2Vh?=
 =?utf-8?B?b3dXL3ZRZ3BCaENkaXIwby9TY3pKc1Z0bXRjZHBPTjN0OGtCOTNXWGRLWktW?=
 =?utf-8?B?RzNTRVkyRHdIaTQ0RVRJbDBJTDBmUVovejhJQ0hnNkUyaExacXAzSEpEa3Zr?=
 =?utf-8?B?MUMwbHFLd1UvcnlzVWNwbEpHMkxrQ2pLS0lSSDRVL3hBK1ZwMXZtQVo4b0Vo?=
 =?utf-8?B?NUZBaXNybldxY2tTTll1WWJjbWZNUWlSeWlhcGxzNm95RHhZamRWZWFmUFRy?=
 =?utf-8?B?NWpQOHlrSGJhUUJmWUY4QlhmcitpcXQ4MkNscWltODV2Q1VnUnRKb3FpYWRs?=
 =?utf-8?B?T08zS2FDcWFMSExuWEtmeU51U0JBajhEdEJJZS9hK2NhUXZDSVRQa0xCSkUz?=
 =?utf-8?B?MEFIelhMaUtVVjZjbStNMWIrTm11ait3UXUwdVdtOFpUZWoxdFlJSDc3WUxz?=
 =?utf-8?B?UlVGaXowbTdZcytuZmhkQUk3S2JEUm5LSFlFU2VxMGcxRmkrMjIzU1Qvb09F?=
 =?utf-8?B?cVZ2dVJUd0dhL0R5eldIOXhYTThhSnhLdWpobmJvUWN3Q0pjOTFqbTN5SU5k?=
 =?utf-8?B?eTZ2d1h3Y3R5bEN3UXc1K0c3dkFSSFRrUWpWVzVITUF3Z0k1VllXL2R5S3Bh?=
 =?utf-8?B?YlZYT2RnU3owOGdHNktOYzB2dkxaNERrdC82Q21ubjU1MnM0N3phWVRySXdO?=
 =?utf-8?B?c1h4UmluYUhyNzEydkJPaVNjOFk1bUgwM0VTMnNXaE9SU0F2SmxhYWxCWlN3?=
 =?utf-8?B?QnFYa0M2QWZBaHFWc3JGOVA5RjdUKzVvUzVQR1ZOOHByb2RtdjlTZi83TGV0?=
 =?utf-8?B?QXVmbmtwM2Vab0NwWndKNlNzQlpJTENMd012SXN6L0p5SGloU1hkb0c4U2dL?=
 =?utf-8?B?VThpd3pSZktvMERyNjRXNlhySmlYRHc4Vi9GUEFsWU1hb3hFZWNxUVp0TUlM?=
 =?utf-8?B?RWdnazA2N25NUkMxa0NuWUg5R2hXbWtSeUV6NGk4RmhDZWdFZy94KzdHWXFl?=
 =?utf-8?B?Qjl3cm1OU0dQdVM1d1QzWjRWWGFZYXptWG5SRVRXZi9wMVdjenhrMXlSMkpH?=
 =?utf-8?B?b1dLZjhNTG56dStFaWthWDhMMnlieWtLMmVIeG9wbnhyd0k0UjRuRXNJbTRh?=
 =?utf-8?Q?b3Ht1ky3unoef3gQPFk2lzgBjcTcJdK1438KUGV?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <F2E7B7B4902C7B4B9407F91DECC9D645@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3e8d5731-b9f9-4bae-b044-08d97e71e985
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 09:09:56.3345
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jxeEUKx1fYbNlDOmgQUCaMevy9y+qgev4HnB1L+8ywO2M+EGHwKB0MGbk6OVJ7TXgMGLxD+pNpmqPLHTc5X4ilxq+3jUoGkXkKqUshDxJF/a1F2kD/1vKj4GDbLFFjIE
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7425
X-Proofpoint-ORIG-GUID: P1bSan3MJh1pTPuimyaE6QEzuOU6HUD5
X-Proofpoint-GUID: P1bSan3MJh1pTPuimyaE6QEzuOU6HUD5
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_03,2021-09-22_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015
 priorityscore=1501 mlxlogscore=999 mlxscore=0 suspectscore=0
 impostorscore=0 bulkscore=0 adultscore=0 phishscore=0 malwarescore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230057

DQpPbiAyMy4wOS4yMSAxMjowNSwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjMuMDkuMjEg
MTE6MDIsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDIzLjA5LjIx
IDAwOjEwLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4gT24gV2VkLCAyMiBTZXAgMjAy
MSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IC0tLSBhL2RyaXZlcnMveGVu
L3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+Pj4+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNr
L3hlbmJ1cy5jDQo+Pj4+IEBAIC03NDMsNiArNzQzLDkgQEAgY29uc3Qgc3RydWN0IHhlbl9wY2li
a19iYWNrZW5kICpfX3JlYWRfbW9zdGx5IHhlbl9wY2lia19iYWNrZW5kOw0KPj4+PiDCoMKgIMKg
wqAgaW50IF9faW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+Pj4+IMKgwqAg
ew0KPj4+PiArwqDCoMKgIGlmICgheGVuX3BjaWJrX3B2X3N1cHBvcnQoKSkNCj4+Pj4gK8KgwqDC
oMKgwqDCoMKgIHJldHVybiAwOw0KPj4+IElzIHRoaXMgdHJ1bHkgZW5vdWdoIHRvIHN0b3AgdGhl
IFBWIGJhY2tlbmQgZnJvbSBpbml0aWFsaXppbmc/IEhhdmUgeW91DQo+Pj4gYWN0dWFsbHkgdGVz
dGVkIGl0IHRvIG1ha2Ugc3VyZT8gSWYgaXQgd29ya3MsIGFtYXppbmchIEkgYW0gcXVpdGUgaGFw
cHkNCj4+PiBhYm91dCB0aGlzIGFwcHJvYWNoIDotKQ0KPj4NCj4+IFdlbGwsIEkgcHV0IHNvbWUg
bG9ncyBpbnRvIHRoZSBkcml2ZXIgYW5kIHNhdyBub3RoaW5nIG9idmlvdXMgcG9pbnRpbmcNCj4+
DQo+PiB0byBhbnkgYmFja2VuZCBhY3Rpdml0aWVzIChwcm9iYWJseSB0aGlzIGlzIGFsc28gYmVj
YXVzZSBJIGRvbid0IGhhdmUgYW55DQo+Pg0KPj4gZnJvbnRlbmQpLiBJIHNlZSB0aGF0IHRoZSB4
ZW5idXMgZHJpdmVyIGlzIG5vdCByZWdpc3RlcmVkLiBJbiBYZW5TdG9yZSBJIHNlZToNCj4+DQo+
PiByb290QGRvbTA6fiMgeGVuc3RvcmUtbHMgLWYgfCBncmVwIHBjaQ0KPj4gL2xvY2FsL2RvbWFp
bi8wL2JhY2tlbmQvcGNpID0gIiINCj4+IC9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3BjaS8yID0g
IiINCj4+IC9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3BjaS8yLzAgPSAiIg0KPj4gL2xvY2FsL2Rv
bWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9mcm9udGVuZCA9ICIvbG9jYWwvZG9tYWluLzIvZGV2aWNl
L3BjaS8wIg0KPj4gL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9mcm9udGVuZC1pZCA9
ICIyIg0KPj4gL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9vbmxpbmUgPSAiMSINCj4+
IC9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3BjaS8yLzAvc3RhdGUgPSAiMSINCj4+IC9sb2NhbC9k
b21haW4vMC9iYWNrZW5kL3BjaS8yLzAvZG9tYWluID0gIkRvbVUiDQo+PiAvbG9jYWwvZG9tYWlu
LzAvYmFja2VuZC9wY2kvMi8wL2tleS0wID0gIjAwMDA6MDM6MDAuMCINCj4+IC9sb2NhbC9kb21h
aW4vMC9iYWNrZW5kL3BjaS8yLzAvZGV2LTAgPSAiMDAwMDowMzowMC4wIg0KPj4gL2xvY2FsL2Rv
bWFpbi8wL2JhY2tlbmQvcGNpLzIvMC9vcHRzLTAgPSAibXNpdHJhbnNsYXRlPTAscG93ZXJfbWdt
dD0wLHBlcm1pc3NpdmU9MCxyZG1fcG9saWN5PXN0cmljdCINCj4+IC9sb2NhbC9kb21haW4vMC9i
YWNrZW5kL3BjaS8yLzAvc3RhdGUtMCA9ICIxIg0KPj4gL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQv
cGNpLzIvMC9udW1fZGV2cyA9ICIxIg0KPj4gL2xvY2FsL2RvbWFpbi8yL2RldmljZS9wY2kgPSAi
Ig0KPj4gL2xvY2FsL2RvbWFpbi8yL2RldmljZS9wY2kvMCA9ICIiDQo+PiAvbG9jYWwvZG9tYWlu
LzIvZGV2aWNlL3BjaS8wL2JhY2tlbmQgPSAiL2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvcGNpLzIv
MCINCj4+IC9sb2NhbC9kb21haW4vMi9kZXZpY2UvcGNpLzAvYmFja2VuZC1pZCA9ICIwIg0KPj4g
L2xvY2FsL2RvbWFpbi8yL2RldmljZS9wY2kvMC9zdGF0ZSA9ICIxIg0KPj4gL2xpYnhsL3BjaSA9
ICIiDQo+PiAvbGlieGwvcGNpLzAwMDAtMDMtMDAtMCA9ICIiDQo+PiAvbGlieGwvcGNpLzAwMDAt
MDMtMDAtMC9kb21pZCA9ICIyIg0KPj4NCj4+IEJ1dCBJSVVJQyB0aGVzZSBjb21lIGZyb20gdGhl
IHRvb2xzdGFjaw0KPj4NCj4+IEBKdWVyZ2VuLCBkbyB5b3Uga25vdyBob3cgdG8gY2hlY2sgaWYg
dGhlIGJhY2tlbmQgaXMgaW5kZWVkIG5vdCBydW5uaW5nDQo+Pg0KPj4gb3IgdGhlIGFib3ZlIHNo
b3VsZCBiZSBlbm91Z2ggdG8gcHJvdmU/DQo+DQo+IEkgZG9uJ3Qgc2VlIGhvdyB0aGUgYmFja2Vu
ZCBjb3VsZCBiZSBydW5uaW5nIHdpdGhvdXQgYmVpbmcgcmVnaXN0ZXJlZA0KPiB3aXRoIHhlbmJ1
cy4gSXQgd29uJ3QgcmVjZWl2ZSBhbnkgd2F0Y2hlcywgc28gdGhlcmUgaXMgbm8gd2F5IGENCj4g
Y29ubmVjdGlvbiB3aXRoIGEgZnJvbnRlbmQgY291bGQgYmUgZXN0YWJsaXNoZWQuDQoNClRoaXMg
aXMgbXkgdW5kZXJzdGFuZGluZyB0b28sIHNvIHRoZSBvbmx5IGNoYW5nZSBJJ3ZlIHB1dCBpbiBw
YXRjaCBJIHJlbW92ZWQNCg0KcmVnaXN0ZXIvdW5yZWdpc3Rlci4gSXQgc2VlbXMgdGhpcyBpcyBq
dXN0IGVub3VnaCBhbmQgdGhlIHBhdGNoIHNob3VsZCBiZSBvayBhcyBpcw0KDQo+DQo+DQo+IEp1
ZXJnZW4NCj4NClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:16:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:16:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193552.344751 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKqN-0000Z7-8Y; Thu, 23 Sep 2021 09:16:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193552.344751; Thu, 23 Sep 2021 09:16:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKqN-0000Z0-5G; Thu, 23 Sep 2021 09:16:39 +0000
Received: by outflank-mailman (input) for mailman id 193552;
 Thu, 23 Sep 2021 09:16:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTKqL-0000Yu-Rk
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:16:37 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKqK-00044Z-AY; Thu, 23 Sep 2021 09:16:36 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTKqJ-0001SO-Ui; Thu, 23 Sep 2021 09:16:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=DosqaWJh+KATWTaJjQCJvGRIAS0XWqA62VkXi8Iwe30=; b=YA0BhrGP/0bGeP/mBtleTljDTs
	5okFpj3UHSs2f2pCc9eXvvlNTlTehEJedrVzenLGBZrnLMXTZIYUkdCJwOBH9E099TFyib2a4atii
	Y+/9tU7gESTpXp0xf40RULnG+PMc5mfLiJRpkweQM7wzrGR03RnE1AimtZYHRuEj+xLw=;
Subject: Re: [PATCH RFC 2/4] xen: add bitmap to indicate per-domain state
 changes
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210914123600.1626-1-jgross@suse.com>
 <20210914123600.1626-3-jgross@suse.com>
From: Julien Grall <julien@xen.org>
Message-ID: <cde0c694-a6be-e7c9-99af-fc5602e85cd7@xen.org>
Date: Thu, 23 Sep 2021 14:16:31 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210914123600.1626-3-jgross@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 14/09/2021 17:35, Juergen Gross wrote:
> Add a bitmap with one bit per possible domid indicating the respective
> domain has changed its state (created, deleted, dying, crashed,
> shutdown).
> 
> Registering the VIRQ_DOM_EXC event will result in setting the bits for
> all existing domains and resetting all other bits.
> 
> Resetting a bit will be done in a future patch.
> 
> This information is needed for Xenstore to keep track of all domains.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>   xen/common/domain.c        | 22 ++++++++++++++++++++++
>   xen/common/event_channel.c |  2 ++
>   xen/include/xen/sched.h    |  2 ++
>   3 files changed, 26 insertions(+)
> 
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 755349b93f..14a427e2ef 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -87,6 +87,22 @@ bool __read_mostly vmtrace_available;
>   /* Unique domain identifier, protected by domctl lock. */
>   static uint64_t unique_id;
>   
> +static DECLARE_BITMAP(dom_state_changed, DOMID_MASK + 1);
> +
> +void domain_reset_states(void)
> +{
> +    struct domain *d;
> +
> +    bitmap_zero(dom_state_changed, DOMID_MASK + 1);
> +
> +    rcu_read_lock(&domlist_read_lock);
> +
> +    for_each_domain ( d )
> +        set_bit(d->domain_id, dom_state_changed);
> +
> +    rcu_read_unlock(&domlist_read_lock);
> +}
> +
>   static void __domain_finalise_shutdown(struct domain *d)
>   {
>       struct vcpu *v;
> @@ -101,6 +117,7 @@ static void __domain_finalise_shutdown(struct domain *d)
>               return;
>   
>       d->is_shut_down = 1;
> +    set_bit(d->domain_id, dom_state_changed);

Looking at the follow-up patch, I think you want a smp_wmb() before 
set_bit() otherwise this could be re-ordered on Arm (set_bit() doesn't 
guarantee any ordering). This is to avoid any issue if the new hypercall 
run concurrently.

>       if ( (d->shutdown_code == SHUTDOWN_suspend) && d->suspend_evtchn )
>           evtchn_send(d, d->suspend_evtchn);
>       else
> @@ -720,6 +737,8 @@ struct domain *domain_create(domid_t domid,
>           rcu_assign_pointer(domain_hash[DOMAIN_HASH(domid)], d);
>           spin_unlock(&domlist_update_lock);
>   
> +        set_bit(d->domain_id, dom_state_changed);

Here you should not need one because spin_unlock() contains one.

> +
>           memcpy(d->handle, config->handle, sizeof(d->handle));
>       }
>   
> @@ -954,6 +973,7 @@ int domain_kill(struct domain *d)
>           /* Mem event cleanup has to go here because the rings
>            * have to be put before we call put_domain. */
>           vm_event_cleanup(d);
> +        set_bit(d->domain_id, dom_state_changed);

You might want an smp_wmb() here as well.

>           put_domain(d);
>           send_global_virq(VIRQ_DOM_EXC);
>           /* fallthrough */
> @@ -1141,6 +1161,8 @@ static void complete_domain_destroy(struct rcu_head *head)
>   
>       xfree(d->vcpu);
>   
> +    set_bit(d->domain_id, dom_state_changed);
> +

I think this wants to be moved *after* _domain_destroy() with an 
smp_wmb() between the two. So you when the bit is set you know the 
domain is dead.

>       _domain_destroy(d);
>   
>       send_global_virq(VIRQ_DOM_EXC);
> diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
> index da88ad141a..e2a416052b 100644
> --- a/xen/common/event_channel.c
> +++ b/xen/common/event_channel.c
> @@ -1218,6 +1218,8 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>           rc = evtchn_bind_virq(&bind_virq, 0);
>           if ( !rc && __copy_to_guest(arg, &bind_virq, 1) )
>               rc = -EFAULT; /* Cleaning up here would be a mess! */
> +        if ( !rc && bind_virq.virq == VIRQ_DOM_EXC )
> +            domain_reset_states();
>           break;
>       }
>   
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index b827c5af8d..2ae26bc50e 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -730,6 +730,8 @@ void domain_resume(struct domain *d);
>   
>   int domain_soft_reset(struct domain *d, bool resuming);
>   
> +void domain_reset_states(void);
> +
>   int vcpu_start_shutdown_deferral(struct vcpu *v);
>   void vcpu_end_shutdown_deferral(struct vcpu *v);
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:24:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:24:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193557.344762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKxq-0002DL-0y; Thu, 23 Sep 2021 09:24:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193557.344762; Thu, 23 Sep 2021 09:24:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTKxp-0002DE-UB; Thu, 23 Sep 2021 09:24:21 +0000
Received: by outflank-mailman (input) for mailman id 193557;
 Thu, 23 Sep 2021 09:24:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTKxp-0002D8-2q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:24:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c772bf84-5cf8-4869-b1e7-d5b57c2b4fb5;
 Thu, 23 Sep 2021 09:24:19 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-uOx-q_-GMNWLUp22WaTgaw-1; Thu, 23 Sep 2021 11:24:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3293.eurprd04.prod.outlook.com (2603:10a6:802:11::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 09:24:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 09:24:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0042.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:48::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Thu, 23 Sep 2021 09:24:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c772bf84-5cf8-4869-b1e7-d5b57c2b4fb5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632389058;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=a1fFr37HGDC+ZBiUGHJDEvBCp02MANs3D58TdZg3h5A=;
	b=VPUM/P1OmY5flANP8DMGwz+7XB1ckbIYTFO4toX6n9apjC/gL/XfvWL3YXD6HRtZtgUOVA
	xdAf1sv16dRCidtFFuH9VCYEj67oV9JyYsofDEdBRBysbv+XIIwIjAb8ws/IA+/KKugFLD
	oQizDRyypohE3fBJJ+w+0o6waWXIdpw=
X-MC-Unique: uOx-q_-GMNWLUp22WaTgaw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=McxchkuwyM6C9fw+16jxCNjI+2EyTT5cJN1HnH1+ztlxpE/DfVkPlveHl8FzMl69x1ZfAwoVsmvAHhw60WgF3Rf5iAh91qzP3QEzBKrdxveVnIvM3BWFogqSf+okYKq3mnDgn+u2j+DgziNq6VRPFE46EyUoth0nPZ0TBpCH9ShU8q4oLNNI4cBqE/J/EFlTZcOchXQY96r7w4HKaKjCydYn2SZipYo2Cn94Kn4YFJcOMRNC/FgC5nLLij731dN4V8E8bguLTfrDJs6M1Uz8MVCp1e6K9BYBr55LcVPlbE2QttAukl8xLuZ9FN4LpmUvD8WbyUuV54VjNfa1CvD02w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=a1fFr37HGDC+ZBiUGHJDEvBCp02MANs3D58TdZg3h5A=;
 b=Vt5S+4LDSC8blQ2Nij4cn1dwx6+9iGecjgsoLnnEY49tD/2JiENqv1HV3wTa3ZrwUv9gI6Xti+ff8ZtwfpDNoJeF1P1MLUjV69icwZ/Bv/Dlr0xy7lRZ7U00xT2dzEMR9OA/lpDDmzEbZEzp7wt12Cp3mIJ3fyDXAR0EFOMILVzQ8+HO+GZMdfuxWvY8okKr2s5JDAfliitSCd/mwzFMMAJj4V0cEQu2PdPOrOPYQubrunFokggifl2JidN9emP7Ob2oWZOyB2ozNo7BBv27/g1CYbdRKuMfya1VZneNFgvxLngMONAbs6HE2mxvd6hhkfsLglDT/sWSoRb6hntKfA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Stefano Stabellini <sstabellini@kernel.org>,
 Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, dpsmith@apertussolutions.com
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
 <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
 <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <93fe57d7-0315-c5b9-0edb-c7d02a2b825c@suse.com>
Date: Thu, 23 Sep 2021 11:24:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0042.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:48::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 907ef154-f76a-4578-230b-08d97e73e96c
X-MS-TrafficTypeDiagnostic: VI1PR04MB3293:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB3293473D4CB613B74F3C66D5B3A39@VI1PR04MB3293.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xyt/DO2+4aELjxzBkcrDXzWzIdb3PyiN4jFXvgBZmiqqsYNwXHreRh7TWbBLgOd8t4K2G4YyryymS9m7Yqauubj/0VDovfHk81bvAF5REdT7QDgA/8WQzZWXfHJ/Cg5/ceIU1oAF8zoIblQ0W8ska0wnTl3m0GMopuudW9yLKStxvdjBpPFvubS3ZXe7HRFXO65xabgfe/bxikBtg8niHsYa+cn84qK+KehRS03zk2zNHKEfwiCA5ig92/UpBnb585DCFNwBmHoL8R3/OS4x2v3txOlIGZieYgeW28QnYU6sqkeKRUC44TfULDpM4SkfAln4dOcnDMIyYBtbkUmSvZu8SBtRYRQ/I+YMDsmuNGnPQyHX8gi3mdWJEHGlBgrhfD6IuGiNUEhvklRw9loL2agMXuxig9qUsz48I3qz5QUxkrlktNy1tt9ERtGkXdoO8Or0vd4iUJ9YR5MfPWMhxNvSoBpLuHXpTnWTSnkZ1uZyKmYSnyd28F0XVLzSVu9BAP/pc56BdfSLP06pqld4VLSOGb5Khf7EyvkZy2wE/9czos9yoFhWTp7LStfQYko58KBdh8XoNeDwITbO7CZ0wpvELH1DKVYlhOOIQhOTWPOBPBJxKuixOrSUVoxgThPGdpnvzEocsQzLv/9ycep+r+Q7R2RrB+jGxcKIYO5P6FpScPuhtJBr3I1kDtdt/WQ3dnYLqhspdJYX7IGUdccqWy4dmAKNDR0DQpCWjR/coyMVno8eZwlRZ82PqwC3znbJV5wvie901Le2JtjaM0xoQJWT+k9TCyJWH72HJG+TQqikBcZZyp8kIFieUOxrbeEW
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(110136005)(83380400001)(16576012)(316002)(86362001)(6486002)(956004)(2616005)(36756003)(966005)(31686004)(31696002)(8936002)(66556008)(66476007)(8676002)(38100700002)(508600001)(4326008)(2906002)(66946007)(186003)(26005)(53546011)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z2FQai81d1hZRC9yeFNmcFZwaVlpMHZFNXAzUkFnb0V1dFBrNC9ENHlRQ2Mr?=
 =?utf-8?B?RzM4ZGlQYWRSY2VPV2k0ZjlEOHNEWU9JSjVYV0RnRm9PYXI0S2hzakVqcUp4?=
 =?utf-8?B?L29YeW8vdnBWMkl2YzFHOW5ML3ZvdFVYU25rZ0h3Nkc5RDFocFk5QVlJT0kr?=
 =?utf-8?B?ZVo1NFhMQVZMRnZhT2Qyb0YrTzRlZU5FMzR2eHQxOVcrVzEvbHRTQWNQU1RN?=
 =?utf-8?B?TnZaOEFQM0hnY1Y3MGhMb0N1RzJaSDlSdjhhME5nOFFCQ1gxS3M0K0FObC9T?=
 =?utf-8?B?a0NTQUlBbFNtakV0TUlBcEJrOXVvR0JjbGUzaVVwVElPeHNWV1VNS0ZyS29U?=
 =?utf-8?B?YmU3WkVPSk9XQ0pDSGI5ZlB5UUsyN01yR3ZkZUEvYUE2Q0hKUnU2Z3FnczBF?=
 =?utf-8?B?MVc3NzFFOWNGaUtqRCtNZ29MRE43emIvMTRYTmVpTzdHOEFZODZhb1c4YmQ3?=
 =?utf-8?B?bUFraHV0aUJUYkFWN3NKanNEVXJVN0NYVnJDOURoai9TRWNQQnBNYjI5Zk56?=
 =?utf-8?B?aVZXak1SOURxYWJwL3M3VU9oNy8ycTh4eEtNbE11a3M4bXozRlJJR2dwS25K?=
 =?utf-8?B?aEVQTlROWnpCczQ5cGRJNmJYNEM2Mi9VRUVQWWkvME13ODJpKzVzTUtHSUoz?=
 =?utf-8?B?OTcwTUhuNnpoVitHZXgrcms2bUNBVDB6enpsMkNPTCtjVko2Ni9iSC93c1pJ?=
 =?utf-8?B?VEluZSt3UnpwSkhLT05kSlo4M2E0ZDV6eFAxWXZiaEJ5ampuQkNJN1F3U1g3?=
 =?utf-8?B?ZnJEVEIyWklTQ1ZlQjlocWkxSmdSV01yVEplSVhSa0xFRXdmOEdIY2ZGYzBr?=
 =?utf-8?B?TzN0VXFzakNSMVdJaFRTM0xzRGgydXRUUjFFQlhrV2psL2dHcytVYWxXOUp2?=
 =?utf-8?B?S3ducDZFOHBDVU00c1JmZXlKSStnZ3pTd2hMOEQvM0h4TUVGWi9ibHg3ZUoz?=
 =?utf-8?B?a1QyQkZqUjBRZGRqU2hNd0xMTEFUand0KzdRWVp3OGdGSHYrZ3U1WVNqL3Zo?=
 =?utf-8?B?YnBnL1NUV0l5VEZKNk1tNHIySW14S01XTTdFakNibGJCaWU1WlpjRXJVQ24v?=
 =?utf-8?B?TW5sR0kzQ3M0YkdVdUtXMHptYkZMVjJaemJyNFk0V3dTVndOdEFiemluY2VH?=
 =?utf-8?B?S2RDWXJPUCt0NkRLdkdkVWEzZHgyajFXaFl4ZUtpcFlBTTdGWlhUazNRbGVY?=
 =?utf-8?B?ejNoVldKblZWc0s0N3BMTVZJWXlSSExzYm9xdlZ5bHJ6N0wwanRKQWpUVm5M?=
 =?utf-8?B?MS9PMXpMVnNZNlZ4RjBuVWJGRnpJbUxvNURTenUzZ1B4YjNwN3dSVEpKeUUx?=
 =?utf-8?B?SzFSSklyRE5nSVZlNTlPeTdmdXIwZG5YMlVXb0E2Wm1STHh6TXh5K2FFOW0z?=
 =?utf-8?B?eXhXVjBNRmxGTldLRGlLcEpkQldrbWFqa3VIRGZGRWVtZUxIakFNbXY4Nndw?=
 =?utf-8?B?NlJ4UVRabk9XcEdWbDkzNGZ2b1YxckRUM2lRa2ZtOVQ1MnNseEluY1IyRFZP?=
 =?utf-8?B?VFJReUtucUxnUlZBemplRUFSbC9RaVEzOURxWjBEYkZBd0ZZT2J0OGUreFhR?=
 =?utf-8?B?WVlPWVB1dEtxTmpjc1lMSlpCSnFmM3ZYSndXNGoxbEh5a1l2R2RtUVVhWU9n?=
 =?utf-8?B?WE5qcXhqbzZyQjk0ZHB6aFBhM3JsVUdFQ1NiaXdSOXIrMjc4c1hULzRaVGN5?=
 =?utf-8?B?dGdUMVFKUVlKcjlFQkpvMHMvRkJnaGlwUitzUWxLSmtJWjRyY2UvRjZNYi81?=
 =?utf-8?Q?XM2tczcJ4rIE3BKM/QUUF8YqKXFIkLxX+7GtH+W?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 907ef154-f76a-4578-230b-08d97e73e96c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 09:24:15.4431
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oWEJwTHU3IYpDk2Oe6JLlC7ZIYfz67wVDwnV12+ss05b58xbXZ0LDb5DeIKB38hmmUB2KOyHStiU2OBhYWFUjw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3293

On 23.09.2021 03:10, Stefano Stabellini wrote:
> On Wed, 22 Sep 2021, Jan Beulich wrote:
>> On 22.09.2021 01:38, Stefano Stabellini wrote:
>>> On Mon, 20 Sep 2021, Ian Jackson wrote:
>>>> Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
>>>>> As per
>>>>>
>>>>> Sep 15 14:44:55.502598 [ 1613.322585] Mem-Info:
>>>>> Sep 15 14:44:55.502643 [ 1613.324918] active_anon:5639 inactive_anon:15857 isolated_anon:0
>>>>> Sep 15 14:44:55.514480 [ 1613.324918]  active_file:13286 inactive_file:11182 isolated_file:0
>>>>> Sep 15 14:44:55.514545 [ 1613.324918]  unevictable:0 dirty:30 writeback:0 unstable:0
>>>>> Sep 15 14:44:55.526477 [ 1613.324918]  slab_reclaimable:10922 slab_unreclaimable:30234
>>>>> Sep 15 14:44:55.526540 [ 1613.324918]  mapped:11277 shmem:10975 pagetables:401 bounce:0
>>>>> Sep 15 14:44:55.538474 [ 1613.324918]  free:8364 free_pcp:100 free_cma:1650
>>>>>
>>>>> the system doesn't look to really be out of memory; as per
>>>>>
>>>>> Sep 15 14:44:55.598538 [ 1613.419061] DMA32: 2788*4kB (UMEC) 890*8kB (UMEC) 497*16kB (UMEC) 36*32kB (UMC) 1*64kB (C) 1*128kB (C) 9*256kB (C) 7*512kB (C) 0*1024kB 0*2048kB 0*4096kB = 33456kB
>>>>>
>>>>> there even look to be a number of higher order pages available (albeit
>>>>> without digging I can't tell what "(C)" means). Nevertheless order-4
>>>>> allocations aren't really nice.
>>>>
>>>> The host history suggests this may possibly be related to a qemu update.
>>>>
>>>> http://logs.test-lab.xenproject.org/osstest/results/host/rochester0.html
>>
>> Stefano - as per some of your investigation detailed further down I
>> wonder whether you had seen this part of Ian's reply. (Question of
>> course then is how that qemu update had managed to get pushed.)
>>
>>>> The grub cfg has this:
>>>>
>>>>  multiboot /xen placeholder conswitch=x watchdog noreboot async-show-all console=dtuart dom0_mem=512M,max:512M ucode=scan  ${xen_rm_opts}
>>>>
>>>> It's not clear to me whether xen_rm_opts is "" or "no-real-mode edd=off".
>>>
>>> I definitely recommend to increase dom0 memory, especially as I guess
>>> the box is going to have a significant amount, far more than 4GB. I
>>> would set it to 2GB. Also the syntax on ARM is simpler, so it should be
>>> just: dom0_mem=2G
>>
>> Ian - I guess that's an adjustment relatively easy to make? I wonder
>> though whether we wouldn't want to address the underlying issue first.
>> Presumably not, because the fix would likely take quite some time to
>> propagate suitably. Yet if not, we will want to have some way of
>> verifying that an eventual fix there would have helped here.
>>
>>> In addition, I also did some investigation just in case there is
>>> actually a bug in the code and it is not a simple OOM problem.
>>
>> I think the actual issue is quite clear; what I'm struggling with is
>> why we weren't hit by it earlier.
>>
>> As imo always, non-order-0 allocations (perhaps excluding the bringing
>> up of the kernel or whichever entity) are to be avoided it at possible.
>> The offender in this case looks to be privcmd's alloc_empty_pages().
>> For it to request through kcalloc() what ends up being an order-4
>> allocation, the original IOCTL_PRIVCMD_MMAPBATCH must specify a pretty
>> large chunk of guest memory to get mapped. Which may in turn be
>> questionable, but I'm afraid I don't have the time to try to drill
>> down where that request is coming from and whether that also wouldn't
>> better be split up.
>>
>> The solution looks simple enough - convert from kcalloc() to kvcalloc().
>> I can certainly spin up a patch to Linux to this effect. Yet that still
>> won't answer the question of why this issue has popped up all of the
>> sudden (and hence whether there are things wanting changing elsewhere
>> as well).
> 
> Also, I saw your patches for Linux. Let's say that the patches are
> reviewed and enqueued immediately to be sent to Linus at the next
> opportunity. It is going to take a while for them to take effect in
> OSSTest, unless we import them somehow in the Linux tree used by OSSTest
> straight away, right?

Yes.

> Should we arrange for one test OSSTest flight now with the patches
> applied to see if they actually fix the issue? Otherwise we might end up
> waiting for nothing...

Not sure how easy it is to do one-off Linux builds then to be used in
hypervisor tests. Ian?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:36:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:36:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193562.344773 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTL9h-0003te-4p; Thu, 23 Sep 2021 09:36:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193562.344773; Thu, 23 Sep 2021 09:36:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTL9h-0003tX-1W; Thu, 23 Sep 2021 09:36:37 +0000
Received: by outflank-mailman (input) for mailman id 193562;
 Thu, 23 Sep 2021 09:36:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTL9g-0003tR-33
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:36:36 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTL9f-0004ON-Tt; Thu, 23 Sep 2021 09:36:35 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTL9f-0003Cx-Iq; Thu, 23 Sep 2021 09:36:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=j+fUunU+E7Qz1EZZojv+haB+IVBX0/IWAy4Wv+P1qOM=; b=MjvuqNJE3rMtwHRW8x3BZvvb9S
	8L5QT4cWpPN1+wrs7gutf1VKsNiJRBNTQLNRxfi8dEC5wvU4Etn1Vj9VhMkljIWGCKRaZKcIpnOIX
	kjy7OiR1f+3z9yHZkP+uWkdh4+1LYrYVW10eP/VIfNCyFlghUVtiqThTobm7IpJ8cgSA=;
Subject: Re: [PATCH] xen/arm: vgic to ignore GICD ICPENRn registers access
To: Hongda Deng <Hongda.Deng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923061429.16361-1-Hongda.Deng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <0b0ede18-b944-8693-dede-616c3386e965@xen.org>
Date: Thu, 23 Sep 2021 14:36:32 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923061429.16361-1-Hongda.Deng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 11:14, Hongda Deng wrote:
> Currently, Xen will return IO unhandled when guests access GICD ICPENRn
> registers. This will raise a data abort inside guest. For Linux Guest,
> these virtual registers will not be accessed. But for Zephyr, in its
> GIC initilization code, these virtual registers will be accessed. And
> zephyr guest will get an IO dataabort in initilization stage and enter

s/dataabort/data abort/
s/initilization/initialization/

> fatal error. Emulating ICPENDR is not easy with the existing vGIC, so
> we currently ignore these virtual registers access and print a message
> about whether they are already pending instead of returning unhandled.
> More details can be found at [1].
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2021-09/
> msg00744.html
> 
> Signed-off-by: Hongda Deng <hongda.deng@arm.com>
> ---
>   xen/arch/arm/vgic-v2.c     | 10 +++++++---
>   xen/arch/arm/vgic-v3.c     | 29 +++++++++++++++++------------
>   xen/arch/arm/vgic.c        | 37 +++++++++++++++++++++++++++++++++++++
>   xen/include/asm-arm/vgic.h |  2 ++
>   4 files changed, 63 insertions(+), 15 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index b2da886adc..644c62757c 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -481,10 +481,14 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
>   
>       case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
>           if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;



> +
>           printk(XENLOG_G_ERR
> -               "%pv: vGICD: unhandled word write %#"PRIregister" to ICPENDR%d\n",
> -               v, r, gicd_reg - GICD_ICPENDR);
> -        return 0;
> +               "%pv: vGICD: unhandled word write %#"PRIregister" to ICPENDR%d, and current pending state is: %s\n",
> +               v, r, gicd_reg - GICD_ICPENDR,
> +               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");

Each register contain the information for multiple pending interrupts. 
So it is a bit confusing to say whether the state is on/off. Instead, it 
would be better to state which interrupt is pending.

Also, I would rather avoid printing a message if there are no interrupts 
pending because there are no issues if this is happening.

> +        goto write_ignore_32;
>   
>       case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>           if ( dabt.size != DABT_WORD ) goto bad_width;
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index cb5a70c42e..c94e33ff4f 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -817,10 +817,14 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
>   
>       case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
>           if ( dabt.size != DABT_WORD ) goto bad_width;
> +        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> +        if ( rank == NULL ) goto write_ignore;
> +
>           printk(XENLOG_G_ERR
> -               "%pv: %s: unhandled word write %#"PRIregister" to ICPENDR%d\n",
> -               v, name, r, reg - GICD_ICPENDR);
> -        return 0;
> +               "%pv: %s: unhandled word write %#"PRIregister" to ICPENDR%d, and current pending state is: %s\n",
> +               v, name, r, reg - GICD_ICPENDR,
> +               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");
> +        goto write_ignore_32;
>   
>       case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>           if ( dabt.size != DABT_WORD ) goto bad_width;
> @@ -978,19 +982,20 @@ static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu *v, mmio_info_t *info,
>       case VREG32(GICR_ICFGR1):
>       case VRANGE32(GICR_IPRIORITYR0, GICR_IPRIORITYR7):
>       case VREG32(GICR_ISPENDR0):
> -         /*
> -          * Above registers offset are common with GICD.
> -          * So handle common with GICD handling
> -          */
> +        /*
> +         * Above registers offset are common with GICD.
> +         * So handle common with GICD handling
> +         */
>           return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
>                                                    info, gicr_reg, r);
>   
>       case VREG32(GICR_ICPENDR0):
> -        if ( dabt.size != DABT_WORD ) goto bad_width;
> -        printk(XENLOG_G_ERR
> -               "%pv: vGICR: SGI: unhandled word write %#"PRIregister" to ICPENDR0\n",
> -               v, r);
> -        return 0;
> +        /*
> +         * Above registers offset are common with GICD.
> +         * So handle common with GICD handling
> +         */
> +        return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
> +                                                 info, gicr_reg, r);
>   
>       case VREG32(GICR_IGRPMODR0):
>           /* We do not implement security extensions for guests, write ignore */
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 8f9400a519..29a1aa5056 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -470,6 +470,43 @@ void vgic_set_irqs_pending(struct vcpu *v, uint32_t r, unsigned int rank)
>       }
>   }
>   
> +bool vgic_get_irqs_pending(struct vcpu *v, uint32_t r, unsigned int rank)
> +{
> +    const unsigned long mask = r;
> +    unsigned int i;
> +    /* The first rank is always per-vCPU */
> +    bool private = rank == 0;
> +    bool is_pending = false;
> +
> +    /* LPIs status will never be retrieved via this function */
> +    ASSERT(!is_lpi(32 * rank + 31));
> +
> +    for_each_set_bit( i, &mask, 32 )
> +    {
> +        unsigned int irq = i + 32 * rank;
> +
> +        if ( !private )

It is not clear to me why you not handling PPIs/SGIs and ...

> +        {
> +            struct pending_irq *p = spi_to_pending(v->domain, irq);
> +
> +            if ( p->desc != NULL )

... emulated SPIs (e.g. PL011).

> +            {
> +                unsigned long flags;
> +
> +                spin_lock_irqsave(&p->desc->lock, flags);
> +                is_pending = gic_read_pending_state(p->desc);
> +                spin_unlock_irqrestore(&p->desc->lock, flags);

What you are reading here is the pending state from the HW. This is not 
the same as the pending state from the VM PoV. In fact, in the most 
common case, the interrupt will be pending from the VM PoV, but simply 
active from the HW PoV (it is deactivated once the interrupt has been 
handled by the guest).

I think what you want to check is whether the flag GIC_IRQ_GUEST_QUEUED 
is set in p->status (Stefano ?).

This is technically still a bit racy as Xen may still think the 
interrupt is pending while the it may be actually active in the guest. 
AFAIK, the other way around (i.e. not pending in Xen but pending in the 
guest) cannot happen.

Anyway, this is just a message, so it is still better than crashing :).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:47:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:47:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193569.344784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLJh-0005cn-8i; Thu, 23 Sep 2021 09:46:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193569.344784; Thu, 23 Sep 2021 09:46:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLJh-0005cg-5Y; Thu, 23 Sep 2021 09:46:57 +0000
Received: by outflank-mailman (input) for mailman id 193569;
 Thu, 23 Sep 2021 09:46:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTLJf-0005ca-FI
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:46:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2e89faff-1c53-11ec-ba25-12813bfff9fa;
 Thu, 23 Sep 2021 09:46:54 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-2-XS5A3k-FPDOfb-LrZXSeMA-1; Thu, 23 Sep 2021 11:46:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6175.eurprd04.prod.outlook.com (2603:10a6:803:fb::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 09:46:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 09:46:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0022.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 09:46:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e89faff-1c53-11ec-ba25-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632390413;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sehrv+WIrqmEJeJg6YrVAHVxuo1aqIIyqYLvE9GL9fg=;
	b=dGy7BT7jmIh4WJmQecA2f533BcNYGVODu6bg4oepCmvG2fQYQs9GaQORxp2k8EQFB9uyer
	BFT7NM51Br8iyl2GXHJYtwlKdAUcx6IpnmKV1hAyJBM4dfS3e59yzVb5rgqHK/pN4d7uPa
	pU9RJPLilT5vEggYNkCbGkSzZdxy+4I=
X-MC-Unique: XS5A3k-FPDOfb-LrZXSeMA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WAzO+oXXOQ9LXQoBGOk8swCCYH4E1FsnK/FNmiYlhtjcSfnMCiuZcUvnUkfNEewud1A1maqNKcmzsQG0Wa/83Gyutuk1rZ+6pKDFCti5CRU2b5eYWU+FH442SKVEfUnZjSY/W1cjrMBVa3l6+hZZ4pVGVeTkGOk3iit+21bWWfMeFtx8hLWFyrx9Fg5vYLJFkJ00uK47yk3JwB5P5CLb73rDLxHGMwijt9dC1PDM+CDTeKTBJe3/r6Gd8Rw07XDR0sluHiaDxm9oE4PYIfq2XD8u0GK/iWjKeFvhowqCgvQ2VDSL6sBgWAPRHVWood/uwHiM6y7ju3yl4yh9ciE6yw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=D8xb1ivZ1MN5iPoVGL2oFTZn7e65u64/qlhkhMvCkoQ=;
 b=LvjKarFPDh2Kz3atfL6dRBTfOANLWjmlHW2gluwpCt0waqcs8QFQ3FUT97JdWdH0AfLXnXd9RQCEquHZAUqDc9yZ93lvfzcMfLvUE1UGmpNo2ISB3l0EhpZjAMBONrE/c7BmDs5iuP5LOfEWnFcHnrJ+PxV0ekVC1EEyOZRZhogkb6WB9qDaJUWk64hn7x1y4wlcYbuTJWc1A3GWLLP/0BjeLfHoBrckK8DpAMQkWl010d3mEWjEqx86w5COlRWva4UrdZL+/o3JPjB+G1NBT5mJoFTBszUlSzzG97IsP2ZBXLkYUI3+TQNHXbXU/O+J0Ct0wtv5Rt713JU0S3KMNg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
 <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <86c22215-923f-e8ce-e0a9-c76d90da4d77@suse.com>
Date: Thu, 23 Sep 2021 11:46:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P193CA0022.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 84545998-79b7-4147-eb7a-08d97e771108
X-MS-TrafficTypeDiagnostic: VI1PR04MB6175:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6175982FF2F45C73F3B8E475B3A39@VI1PR04MB6175.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2n9agiQ+0RH3T77DFUl2tQJ6sY0n7p9OqwKjro/O2qa7ekKL5U5E8u5TDbIOMjvX4yOMjiza1jrOqZgmtkVAlUvB4+AmIpDe0pgqVJPIaat8ww4OyjMLfWWHlzZybRcvVfj2S0WFzIGtQH518871Rq3bXzjNuQwVhkFAlVApfq0/0/oodl3U/ajq1wNcjPHGrrWWaJyuo6L3C9nLNgCbSEDemx3YCoFv3u3+D02j6klcERvjNPthLbxuVm25jgyNJ0RbjOQGGhAY2pjey+XLo4NvCWKXKH3j8x2SHo0bpyEIWbo/SqPL2O1HHEpCRI5gDKwxn1uOfqLAOBq3wtcxpJ+aKtVcOKWFrohwtILplq0jgM8bk6tU0mdaBg5zGt9drq5ceqPiihIKo7BopKGkfXg3s1MtDjRQgVQoPxSTngndUOLN92ALcL87kinjaLYzvN6AxFbgLyJlR622esfFFbtP6Vh3LyEOtu/3w7PAs/3hg0PjqwzJTzVSWDWEVC2COGaiGX65Wcuun7NC4sD7R5fZmbILfr4BARACtcCqxEDqa0FLwGl6ygWGFlKItyj2mLPivvOtX46DSYmQ+c0OXyvqOntsHbyAcwcQdrdHZSMgsfTlC4TgBlt0owHq2IMxrujrG91VXSiAb5r0SdBA8ertGCkq3o3mevPUGFsy3VqJFuJGVeLQON7OhVuQqwu5yAggkmodr3NIa62cBWpo0iX/rU6lx8CRstJtwO4W6h9Q6r42Ge8eu0/+L35t2FqDP3Y94K5Qmea+QNWo2eeIyw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(316002)(31696002)(16576012)(54906003)(66946007)(53546011)(5660300002)(8936002)(6486002)(66476007)(66556008)(2616005)(83380400001)(956004)(186003)(8676002)(26005)(4326008)(508600001)(38100700002)(31686004)(2906002)(36756003)(6916009)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?rKzcgy4yHd5xugXB5Ers8uhnykkZ4de51ecziV6fs5SsjbDkdYrbPBVtC8fy?=
 =?us-ascii?Q?pU/zXgciRcDxIHqU3SWIjo4ykQDuzPYLT5IZiAgaIbJ2aT/fNiLH9Q0lscga?=
 =?us-ascii?Q?p0pGZIgIg/wMF72nphov+SomIVOzbYlZm/O8OioNhS/GjbYBBVgeePLRThfq?=
 =?us-ascii?Q?O4c837Zuxf5GzyLwHxdVr/ti9MIOKPTeV58Q/z7uSvZ0k9v3UlF3YDOTXwH6?=
 =?us-ascii?Q?NfY85vftD/xHl9tiGsuuEQpnPusz85lbKBJ8eSFzVCibPdXssXU1oO5oV3WS?=
 =?us-ascii?Q?8/pF06SiFSg58d8lPyhwuhhu8ZC667kuY1OgDyEVytWMYceOz+JDSfLvK12P?=
 =?us-ascii?Q?8tW+e/+XsF38LWt10ZpHBQpPWGsTHFhCCHYJFtLQj6nDHXTr8Klp/YcB9AoV?=
 =?us-ascii?Q?0Rl9xXvjdUwc51rf67rxvSI4FroGUtFdVvSijHrytJZThZAO37t91aIcTeJH?=
 =?us-ascii?Q?43sIDm/hRLttANpxYcIXBz8DWJeHTR1JmfAIF2OCk6mh76g77xRVTv1H9LrJ?=
 =?us-ascii?Q?6Cc0HyvR/RH5Q4SmiLsa5UNgUTOreRUA3zszTY1noZcbqZVqyENNYWRuohbH?=
 =?us-ascii?Q?tAlZazfGIJG7rObFplxrkyufzdXeDEyv2wKGRkqkVTVVcxSOXlxNMdSS34+B?=
 =?us-ascii?Q?31b61AST0n8YSjAukihsNE2dWbeQusC1P4fwHsE+nvpTkDAbuMogXEZk+ud3?=
 =?us-ascii?Q?RpOzKu9MUYtYp+QygKH9Xy4TKCRJBNwLX068qrngv5LkA6De5gJofh16N9MC?=
 =?us-ascii?Q?KjDahE1fYGZY6Az3MlY6TU0XYU4dTp6Yd70OKk4ST+Pnd2ie6oXOzFJt3cM7?=
 =?us-ascii?Q?rQxUHgHn4u7vwSldZ6TFXifkQnVxy8hsIgOajvv/OCgoTXrqatQ10QWTeWFD?=
 =?us-ascii?Q?+fUh7XxzGvJ98W4D7bePW0DV6SrFxuypWJjHHEjHE2LVl4yxesoUAvtG+aDv?=
 =?us-ascii?Q?HTlxMWQIvqa24f/SFRSv6rwjj3rW2RLssGgWY5QXaCxCsQ6vyI904qJkw1v7?=
 =?us-ascii?Q?AD+Vy7Jxaj4zXTQu6xzwzCZ0JoDvN2SffRaHj3wgSd72b1rNYrU7zwMVDeBA?=
 =?us-ascii?Q?6ual/yqMDWjK9DcfS45BsfK/OHnTl3++OTNgm947KPpUVCqbO1rcQggM0qbM?=
 =?us-ascii?Q?vyIQKgkrUW5tY/dfijmBD6D8ZMvNhd2ginR6kbh3QEDOs8IZV9Z4+im/bz8l?=
 =?us-ascii?Q?uQaEGWwK2DqMMNpVZCPZtX0Jpg+DSiGzBVPZzEEkipN5u/ZWA4CJao/x6slk?=
 =?us-ascii?Q?JcokVfmIid9l84uO7Zj/GSYPdnA2YQI+ZHB1qyH5JD3XKf+e/3cHj7J2PCss?=
 =?us-ascii?Q?Hm4D7eRkW7/KJXVPKkIQASmG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 84545998-79b7-4147-eb7a-08d97e771108
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 09:46:50.2760
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DAsk70rnPA9f73GHBdEbeJRkamWmpZ5RqaT8Lh3aqIVNy7oWE/3FatCzRk1Q005NxYzjk0FiXI5peaLPJXyjFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6175

On 22.09.2021 17:01, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:18:05AM +0200, Jan Beulich wrote:
>> --- a/xen/include/public/arch-x86/hvm/start_info.h
>> +++ b/xen/include/public/arch-x86/hvm/start_info.h
>> @@ -33,7 +33,7 @@
>>   *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_V=
ALUE
>>   *    |                | ("xEn3" with the 0x80 bit of the "E" set).
>>   *  4 +----------------+
>> - *    | version        | Version of this structure. Current version is =
1. New
>> + *    | version        | Version of this structure. Current version is =
2. New
>>   *    |                | versions are guaranteed to be backwards-compat=
ible.
>>   *  8 +----------------+
>>   *    | flags          | SIF_xxx flags.
>> @@ -55,7 +55,15 @@
>>   *    |                | if there is no memory map being provided. Only
>>   *    |                | present in version 1 and newer of the structur=
e.
>>   * 52 +----------------+
>> - *    | reserved       | Version 1 and newer only.
>> + *    | vga_info.offset| Offset of struct dom0_vga_console_info from ba=
se of
>=20
> I'm not sure we are supposed to reference external structures like
> that. We took a lot of care to not depend on other headers, and to
> make this as agnostic as possible (IIRC KVM is also capable of using
> the PVH entry point natively, and hence depends on this header).

But KVM wouldn't be using a Dom0-only part of the interface, would
it? (I'm aware of the possible re-using of the entry point.)

> IF we want to add support for dom0_vga_console_info I think we need to
> at least provide a binary layout for it, like all the other pieces
> that are part of the HVM start info.

Which then means we can't sensibly re-use the existing structure,
as that doesn't have as strict rules as the hvm_start_info one.
Which in turn means Linux can't re-use the code converting
dom0_vga_console_info, resulting in two places needing updating
whenever information gets add to (then) both structures (what
information they carry will, after all, want to remain in sync).

>> + *    |                | struct hvm_start_info. Optional and only prese=
nt in
>> + *    |                | version 2 and newer of the structure when
>> + *    |                | SIF_INITDOMAIN is set; zero if absent.
>=20
> We have usually used an absolute physical address to reference other
> data, and I think we should likely keep in that way for coherency.

Hmm. (See below.)

>> + * 54 +----------------+
>> + *    | vga_info.size  | Size of present parts of struct dom0_vga_conso=
le_info.
>> + *    |                | Optional and only present in version 2 and new=
er of
>> + *    |                | the structure when SIF_INITDOMAIN is set; zero=
 if
>> + *    |                | absent.
>>   * 56 +----------------+
>>   *
>>   * The layout of each entry in the module structure is the following:
>> @@ -139,7 +147,15 @@ struct hvm_start_info {
>>      uint32_t memmap_entries;    /* Number of entries in the memmap tabl=
e.    */
>>                                  /* Value will be zero if there is no me=
mory  */
>>                                  /* map being provided.                 =
      */
>> -    uint32_t reserved;          /* Must be zero.                       =
      */
>=20
> This 'Must be zero' comment troubles me a bit, I'm not convinced we
> can just place data here (ie: it would no longer be 0). It's even
> worse because the must be zero comment is only present in the C
> representation of the struct, the layout above just denotes the field
> is reserved (which would imply it's fine to use for other means in
> v2).

I thought the textual description was meant to be the ABI spec. The C
comment should therefore be viewed as if missing "in version 1" or
"presently".

Taking into account also Andrew's reply, I have to admit that I'm
inclined to request that one of the two of you fix this obvious
shortcoming in both Xen and Linux. I'm not really willing to be the one
to introduce a 2nd layout for the same set of data just for the purpose
of "playing nice" in an area where that, affecting Dom0 only, doesn't
seem to matter all this much. My goal was rather to keep the impact on
hvm_start_info as low as possible (and in particular avoid changing its
size, as strictly speaking Linux'es consumer implementation is buggy:
It would always copy as much data as it knows _may_ be present, not as
little data as may have been _actually_ provided; whoever implemented
this did only consider one half of the compatibility requirements,
quite likely simply because in the design this aspect was also missed,
or else the structure would have had a length field right from its
introduction).

IOW I'm afraid I may not be seeing the "big picture" here ...

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:47:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:47:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193574.344795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLKd-0006Oz-J1; Thu, 23 Sep 2021 09:47:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193574.344795; Thu, 23 Sep 2021 09:47:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLKd-0006O8-Ff; Thu, 23 Sep 2021 09:47:55 +0000
Received: by outflank-mailman (input) for mailman id 193574;
 Thu, 23 Sep 2021 09:47:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lEPm=ON=csgroup.eu=christophe.leroy@srs-us1.protection.inumbo.net>)
 id 1mTLKc-0006O2-PH
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:47:54 +0000
Received: from pegase2.c-s.fr (unknown [93.17.235.10])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5219fa5a-1c53-11ec-ba25-12813bfff9fa;
 Thu, 23 Sep 2021 09:47:53 +0000 (UTC)
Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67])
 by localhost (Postfix) with ESMTP id 4HFVhS1HTcz9sTZ;
 Thu, 23 Sep 2021 11:47:52 +0200 (CEST)
Received: from pegase2.c-s.fr ([172.26.127.65])
 by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id N2iojAvTUjrW; Thu, 23 Sep 2021 11:47:52 +0200 (CEST)
Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192])
 by pegase2.c-s.fr (Postfix) with ESMTP id 4HFVhS0G8Vz9sTX;
 Thu, 23 Sep 2021 11:47:52 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by messagerie.si.c-s.fr (Postfix) with ESMTP id E65698B775;
 Thu, 23 Sep 2021 11:47:51 +0200 (CEST)
Received: from messagerie.si.c-s.fr ([127.0.0.1])
 by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023)
 with ESMTP id HEg50hnqH7U5; Thu, 23 Sep 2021 11:47:51 +0200 (CEST)
Received: from PO20335.IDSI0.si.c-s.fr (unknown [192.168.202.200])
 by messagerie.si.c-s.fr (Postfix) with ESMTP id 224928B763;
 Thu, 23 Sep 2021 11:47:50 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5219fa5a-1c53-11ec-ba25-12813bfff9fa
X-Virus-Scanned: amavisd-new at c-s.fr
X-Virus-Scanned: amavisd-new at c-s.fr
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
To: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>
Cc: devicetree@vger.kernel.org, linux-efi@vger.kernel.org,
 Mike Rapoport <rppt@linux.ibm.com>, kvm@vger.kernel.org,
 linux-s390@vger.kernel.org, linux-sh@vger.kernel.org,
 linux-um@lists.infradead.org, linux-kernel@vger.kernel.org,
 kasan-dev@googlegroups.com, linux-mips@vger.kernel.org, linux-mm@kvack.org,
 iommu@lists.linux-foundation.org, linux-usb@vger.kernel.org,
 linux-alpha@vger.kernel.org, sparclinux@vger.kernel.org,
 xen-devel@lists.xenproject.org, Andrew Morton <akpm@linux-foundation.org>,
 linux-snps-arc@lists.infradead.org, linuxppc-dev@lists.ozlabs.org,
 linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
From: Christophe Leroy <christophe.leroy@csgroup.eu>
Message-ID: <1101e3c7-fcb7-a632-8e22-47f4a01ea02e@csgroup.eu>
Date: Thu, 23 Sep 2021 11:47:48 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210923074335.12583-4-rppt@kernel.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: fr-FR
Content-Transfer-Encoding: 8bit



Le 23/09/2021 à 09:43, Mike Rapoport a écrit :
> From: Mike Rapoport <rppt@linux.ibm.com>
> 
> For ages memblock_free() interface dealt with physical addresses even
> despite the existence of memblock_alloc_xx() functions that return a
> virtual pointer.
> 
> Introduce memblock_phys_free() for freeing physical ranges and repurpose
> memblock_free() to free virtual pointers to make the following pairing
> abundantly clear:
> 
> 	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
> 	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);
> 
> 	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
> 	void memblock_free(void *ptr, size_t size);
> 
> Replace intermediate memblock_free_ptr() with memblock_free() and drop
> unnecessary aliases memblock_free_early() and memblock_free_early_nid().
> 
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
> ---

> diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
> index 1a04e5bdf655..37826d8c4f74 100644
> --- a/arch/s390/kernel/smp.c
> +++ b/arch/s390/kernel/smp.c
> @@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
>   			/* Get the CPU registers */
>   			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
>   	}
> -	memblock_free(page, PAGE_SIZE);
> +	memblock_phys_free(page, PAGE_SIZE);
>   	diag_amode31_ops.diag308_reset();
>   	pcpu_set_smt(0);
>   }
> @@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
>   
>   	/* Add CPUs present at boot */
>   	__smp_rescan_cpus(info, true);
> -	memblock_free_early((unsigned long)info, sizeof(*info));
> +	memblock_free(info, sizeof(*info));
>   }
>   
>   /*

I'm a bit lost. IIUC memblock_free_early() and memblock_free() where 
identical.

In the first hunk memblock_free() gets replaced by memblock_phys_free()
In the second hunk memblock_free_early() gets replaced by memblock_free()

I think it would be easier to follow if you could split it in several 
patches:
- First patch: Create memblock_phys_free() and change all relevant 
memblock_free() to memblock_phys_free() - Or change memblock_free() to 
memblock_phys_free() and make memblock_free() an alias of it.
- Second patch: Make memblock_free_ptr() become memblock_free() and 
change all remaining callers to the new semantics (IIUC 
memblock_free(__pa(ptr)) becomes memblock_free(ptr) and make 
memblock_free_ptr() an alias of memblock_free()
- Fourth patch: Replace and drop memblock_free_ptr()
- Fifth patch: Drop memblock_free_early() and memblock_free_early_nid() 
(All users should have been upgraded to memblock_free_phys() in patch 1 
or memblock_free() in patch 2)

Christophe


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:53:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:53:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193579.344805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLQO-0007sg-7b; Thu, 23 Sep 2021 09:53:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193579.344805; Thu, 23 Sep 2021 09:53:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLQO-0007sZ-4l; Thu, 23 Sep 2021 09:53:52 +0000
Received: by outflank-mailman (input) for mailman id 193579;
 Thu, 23 Sep 2021 09:53:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTLQM-0007sT-Sm
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:53:50 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b08002f3-3fa3-4d93-b60a-9e760c96c8b0;
 Thu, 23 Sep 2021 09:53:49 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id z24so24771090lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 02:53:49 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id h8sm582728ljh.27.2021.09.23.02.53.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 02:53:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b08002f3-3fa3-4d93-b60a-9e760c96c8b0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=XKbdAC9VjfymO1XEcySU7Tw5k0/nPtdPrfAPQBhF/eI=;
        b=EaN30mIuXR5Rmmhi8XIrA5BIqwjyKhuOJBSAk2VcNeY4K1tBY0tgaHieE/ye2t82+D
         Q03cnCumMLi3eyVFVEunCZa+oqw1Al+N/71hBirVc4c99EVkX+868M/GTzYZvI8dZIy/
         hjxq/euJjhLVBUIv3OJL49sbfpCxt0FgRjlhw+iF4Ez3fVC+0hKTSkvMUTIgN8vw5Ghg
         exH7i1K+rZapglH0iwkb8f4TzuTM4xN4vk3sSWHrBB1NK563LtDR6ftlCG4Q5dDwkoyi
         YKVysVmkjclfMtAC42KSJRWioQ/Di1EaOWk3Gjwc7plyFZ0kh692M4ttcLFKqiZrSh9s
         ILuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=XKbdAC9VjfymO1XEcySU7Tw5k0/nPtdPrfAPQBhF/eI=;
        b=NdT9Hk8MQ6vpewTz7mshJvEHmGiV5VKAagDkLs+9tv2/A45WoS3SfCDarHZS6YmcKn
         nYoznlVTQRWhY9D1M6eJIbjgnbgqxEgvz+hB1pZCFZE9B23wMNbLIeNvEgN+qLrx5w36
         Z9AOkECZZAjrKZDtM8JAQEhFbgasAq/wziEhSKIhsVT2Iv6Cgq+ONh8Liap4I1bdf7qv
         dEOi8kfhd+EDIHZmG7ylzIONs0/A2uYH33R7Q3pAd7Snz3goQV1flMOf2RyCKJbkRlGj
         aEr+iLbaZNrClvFofKzSBPHyaAJGtow3ix3kNtEnpZz/IOZa4AAi8NsEOEgQU5qbWgfW
         2GmA==
X-Gm-Message-State: AOAM530q7vBNIijpA4qz1c8wyzu3gYrCiukFuljOkRAlx3aI6xB2ahai
	knNJ8rgr/xiGfs0tD94HSGDsO8Dlx/5+nw==
X-Google-Smtp-Source: ABdhPJw1DwVwjIkSssbPvDibBogq/KzZbYAShfAkUUgv4FoFnDI/9v5+fJ6FluE9vBqLU89rFHQuvA==
X-Received: by 2002:a05:6512:12c6:: with SMTP id p6mr3288566lfg.271.1632390828064;
        Thu, 23 Sep 2021 02:53:48 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
Date: Thu, 23 Sep 2021 12:53:44 +0300
Message-Id: <20210923095345.185489-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Currently PCI backend implements multiple functionalities at a time.
To name a few:
1. It is used as a database for assignable PCI devices, e.g. xl
   pci-assignable-{add|remove|list} manipulates that list. So, whenever
   the toolstack needs to know which PCI devices can be passed through
   it reads that from the relevant sysfs entries of the pciback.
2. It is used to hold the unbound PCI devices list, e.g. when passing
   through a PCI device it needs to be unbound from the relevant device
   driver and bound to pciback (strictly speaking it is not required
   that the device is bound to pciback, but pciback is again used as a
   database of the passed through PCI devices, so we can re-bind the
   devices back to their original drivers when guest domain shuts down)
3. Device reset for the devices being passed through
4. Para-virtualized use-cases support

The para-virtualized part of the driver is not always needed as some
architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
model for PCI device passthrough. For such use-cases make the very
first step in splitting the xen-pciback driver into two parts: extended
PCI stub and PCI PV backend drivers. At the moment x86 platform will
continue using CONFIG_XEN_PCIDEV_BACKEND for the fully featured backend
driver and new platforms may build a driver with limited functionality
(no PV) by enabling CONFIG_XEN_PCIDEV_STUB.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Changes since v2:
 - swap the patch order
New in v2
---
 drivers/xen/Kconfig               | 24 ++++++++++++++++++++++++
 drivers/xen/Makefile              |  2 +-
 drivers/xen/xen-pciback/Makefile  |  1 +
 drivers/xen/xen-pciback/pciback.h |  5 +++++
 drivers/xen/xen-pciback/xenbus.c  |  6 +++++-
 5 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index a37eb52fb401..6e92c6be19f1 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -180,10 +180,34 @@ config SWIOTLB_XEN
 	select DMA_OPS
 	select SWIOTLB
 
+config XEN_PCI_STUB
+	bool
+
+config XEN_PCIDEV_STUB
+	tristate "Xen PCI-device stub driver"
+	depends on PCI && !X86 && XEN
+	depends on XEN_BACKEND
+	select XEN_PCI_STUB
+	default m
+	help
+	  The PCI device stub driver provides limited version of the PCI
+	  device backend driver without para-virtualized support for guests.
+	  If you select this to be a module, you will need to make sure no
+	  other driver has bound to the device(s) you want to make visible to
+	  other guests.
+
+	  The "hide" parameter (only applicable if backend driver is compiled
+	  into the kernel) allows you to bind the PCI devices to this module
+	  from the default device drivers. The argument is the list of PCI BDFs:
+	  xen-pciback.hide=(03:00.0)(04:00.0)
+
+	  If in doubt, say m.
+
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
 	depends on PCI && X86 && XEN
 	depends on XEN_BACKEND
+	select XEN_PCI_STUB
 	default m
 	help
 	  The PCI device backend driver allows the kernel to export arbitrary
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3434593455b2..5aae66e638a7 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
 obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
 obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
 obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
-obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
+obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
 obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
index e8d981d43235..e2cb376444a6 100644
--- a/drivers/xen/xen-pciback/Makefile
+++ b/drivers/xen/xen-pciback/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
+obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
 
 xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
 xen-pciback-y += conf_space.o conf_space_header.o \
diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
index 95e28ee48d52..9a64196e831d 100644
--- a/drivers/xen/xen-pciback/pciback.h
+++ b/drivers/xen/xen-pciback/pciback.h
@@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
 				    struct pci_dev *dev);
 void pcistub_put_pci_dev(struct pci_dev *dev);
 
+static inline bool xen_pcibk_pv_support(void)
+{
+	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
+}
+
 /* Ensure a device is turned off or reset */
 void xen_pcibk_reset_device(struct pci_dev *pdev);
 
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index c09c7ebd6968..f8ba2903a3ff 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
 
 int __init xen_pcibk_xenbus_register(void)
 {
+	if (!xen_pcibk_pv_support())
+		return 0;
+
 	xen_pcibk_backend = &xen_pcibk_vpci_backend;
 	if (passthrough)
 		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
@@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
 
 void __exit xen_pcibk_xenbus_unregister(void)
 {
-	xenbus_unregister_driver(&xen_pcibk_driver);
+	if (xen_pcibk_pv_support())
+		xenbus_unregister_driver(&xen_pcibk_driver);
 }
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:53:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:53:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193580.344817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLQT-0008Ae-G3; Thu, 23 Sep 2021 09:53:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193580.344817; Thu, 23 Sep 2021 09:53:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLQT-0008AX-CQ; Thu, 23 Sep 2021 09:53:57 +0000
Received: by outflank-mailman (input) for mailman id 193580;
 Thu, 23 Sep 2021 09:53:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTLQR-0007sT-PU
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:53:55 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f43e3efd-5c3e-4466-b64a-72e4c312b17b;
 Thu, 23 Sep 2021 09:53:50 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id x27so24420001lfu.5
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 02:53:50 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id h8sm582728ljh.27.2021.09.23.02.53.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 02:53:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f43e3efd-5c3e-4466-b64a-72e4c312b17b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=273vid3FwOFmMn1y/17lnNlcloyGCFpA/el4JOMAC3U=;
        b=Juu/Im9RD3cI9irkzuhsmZeaw8gRRC6kPFUBQgM4JwZXFIBFOTfVsDft4D0U/dcJZI
         UdJ44+XnIrinP3TUKP6H5jKoT8LtM/5hWzcvFdN+nwWilx9PFkXcLWtdYLpE5QooB6+k
         RW60Irb8Jo2SYIwQo3ZeIuGyQXFyGfvouKnUrZ92vmXXdonuR6nbXTG8lAlSq39VBnK2
         f7njqHsPDSH1C/Lvsb+R5rK+53nGHvUX381pJcJdcdEAXsMcT9LibNHqz8uTovmVeqUe
         M8PeD421lnAiYpcWsSk2pt0gUiKBpRXysprrmXBkSmBeoVBXO9aTmzYgbkWqXmbB+bO+
         TQbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=273vid3FwOFmMn1y/17lnNlcloyGCFpA/el4JOMAC3U=;
        b=CU1X67019r6XtqFX3IEnhlFRLpdbzIk5Dd61nJh5sqTY1weOhb9OT3B121BWCuk+CU
         Gpn3bPPrMNpzd0U+19qcm/nwIEbn0PITsiBG62NUbA+SK/KYJMFx2U3XNpQMBvgiEk19
         GT28fF+C5QCtknG3+1J2OlWKuPC0PoDeYc3NuqMnE89vMiQdPIDxQ+iOJ2T+CjwzmgNr
         QaCmPsKUONm2Ov7iDcRGWVwpoVAZw+K25PVgd2f9+CrSyeyh7fgpixLyCamFxz1s6Npk
         1oO7J2d2IsCb3DLi0XOXEYW2dn/TLqGep2C33aW9V5Y7TAl/oTsEX5mR7Z4eGonjWnxw
         jfMw==
X-Gm-Message-State: AOAM532pLbhqrXg1+AovQjeNlbdnc7msTMJDEhew2zcNLB122XGnKwDt
	SC4XgIqMt6U4FZSCkL2Pv8je6GthZRn8AQ==
X-Google-Smtp-Source: ABdhPJx+k3CIZCEk4gFeEHeef0x4jLVb/OCMJEojIojU9GJLDihF2tRdEz2uyoeLdSjtF8nY1vh6Kg==
X-Received: by 2002:a05:6512:e98:: with SMTP id bi24mr3188886lfb.497.1632390829240;
        Thu, 23 Sep 2021 02:53:49 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than x86
Date: Thu, 23 Sep 2021 12:53:45 +0300
Message-Id: <20210923095345.185489-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923095345.185489-1-andr2000@gmail.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen-pciback driver was designed to be built for x86 only. But it
can also be used by other architectures, e.g. Arm.
Re-structure the driver in a way that it can be built for other
platforms as well.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

---
Since v2:
 - swap the patch order
Since v1:
 - Do not move pci_xen_initial_domain as it is x86 specific
---
 arch/x86/include/asm/xen/pci.h     | 18 +------
 arch/x86/pci/xen.c                 | 74 +----------------------------
 drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c |  3 +-
 drivers/xen/xen-pciback/xenbus.c   |  2 +-
 include/xen/pci.h                  | 28 +++++++++++
 6 files changed, 108 insertions(+), 92 deletions(-)
 create mode 100644 include/xen/pci.h

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 3506d8c598c1..2889f091f459 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -16,26 +16,10 @@ static inline int pci_xen_hvm_init(void)
 #endif
 #if defined(CONFIG_XEN_DOM0)
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
-	return -1;
-}
-static inline int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-static inline int xen_register_device_domain_owner(struct pci_dev *dev,
-						   uint16_t domain)
-{
-	return -1;
-}
-static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
+       return -1;
 }
 #endif
 
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 3d41a09c2c14..4a45b0bf9ae4 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -23,6 +23,7 @@
 
 #include <xen/features.h>
 #include <xen/events.h>
+#include <xen/pci.h>
 #include <asm/xen/pci.h>
 #include <asm/xen/cpuid.h>
 #include <asm/apic.h>
@@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
-
-struct xen_device_domain_owner {
-	domid_t domain;
-	struct pci_dev *dev;
-	struct list_head list;
-};
-
-static DEFINE_SPINLOCK(dev_domain_list_spinlock);
-static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
-
-static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	list_for_each_entry(owner, &dev_domain_list, list) {
-		if (owner->dev == dev)
-			return owner;
-	}
-	return NULL;
-}
-
-int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-	int domain = -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (owner)
-		domain = owner->domain;
-	spin_unlock(&dev_domain_list_spinlock);
-	return domain;
-}
-EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
-
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
-{
-	struct xen_device_domain_owner *owner;
-
-	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
-	if (!owner)
-		return -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	if (find_device(dev)) {
-		spin_unlock(&dev_domain_list_spinlock);
-		kfree(owner);
-		return -EEXIST;
-	}
-	owner->domain = domain;
-	owner->dev = dev;
-	list_add_tail(&owner->list, &dev_domain_list);
-	spin_unlock(&dev_domain_list_spinlock);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
-
-int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (!owner) {
-		spin_unlock(&dev_domain_list_spinlock);
-		return -ENODEV;
-	}
-	list_del(&owner->list);
-	spin_unlock(&dev_domain_list_spinlock);
-	kfree(owner);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
 #endif
diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
index 224df03ce42e..fc8c1249d49f 100644
--- a/drivers/xen/pci.c
+++ b/drivers/xen/pci.c
@@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_XEN_DOM0
+struct xen_device_domain_owner {
+	domid_t domain;
+	struct pci_dev *dev;
+	struct list_head list;
+};
+
+static DEFINE_SPINLOCK(dev_domain_list_spinlock);
+static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
+
+static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	list_for_each_entry(owner, &dev_domain_list, list) {
+		if (owner->dev == dev)
+			return owner;
+	}
+	return NULL;
+}
+
+int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+	int domain = -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (owner)
+		domain = owner->domain;
+	spin_unlock(&dev_domain_list_spinlock);
+	return domain;
+}
+EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
+
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
+{
+	struct xen_device_domain_owner *owner;
+
+	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
+	if (!owner)
+		return -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	if (find_device(dev)) {
+		spin_unlock(&dev_domain_list_spinlock);
+		kfree(owner);
+		return -EEXIST;
+	}
+	owner->domain = domain;
+	owner->dev = dev;
+	list_add_tail(&owner->list, &dev_domain_list);
+	spin_unlock(&dev_domain_list_spinlock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
+
+int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (!owner) {
+		spin_unlock(&dev_domain_list_spinlock);
+		return -ENODEV;
+	}
+	list_del(&owner->list);
+	spin_unlock(&dev_domain_list_spinlock);
+	kfree(owner);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
+#endif
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index f8e4faa96ad6..bba527620507 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -19,7 +19,8 @@
 #include <linux/sched.h>
 #include <linux/atomic.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
+#include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index f8ba2903a3ff..bde63ef677b8 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -14,7 +14,7 @@
 #include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
diff --git a/include/xen/pci.h b/include/xen/pci.h
new file mode 100644
index 000000000000..b8337cf85fd1
--- /dev/null
+++ b/include/xen/pci.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __XEN_PCI_H__
+#define __XEN_PCI_H__
+
+#if defined(CONFIG_XEN_DOM0)
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
+static inline int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+
+static inline int xen_register_device_domain_owner(struct pci_dev *dev,
+						   uint16_t domain)
+{
+	return -1;
+}
+
+static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+#endif
+
+#endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:54:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:54:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193591.344828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLRP-0000eq-UW; Thu, 23 Sep 2021 09:54:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193591.344828; Thu, 23 Sep 2021 09:54:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLRP-0000ej-RM; Thu, 23 Sep 2021 09:54:55 +0000
Received: by outflank-mailman (input) for mailman id 193591;
 Thu, 23 Sep 2021 09:54:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTLRN-0000eZ-V2
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:54:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTLRN-0004hs-Qc; Thu, 23 Sep 2021 09:54:53 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTLRN-0004in-DI; Thu, 23 Sep 2021 09:54:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=F0GhiiQdRJLWMGAG5etNZqqaQ1QMtZlfF191vHmxTEc=; b=HoEPW2PsB9DD3OOKtRCPLHp0G9
	dqZ2egQkEQekB2RF5nWexNkKMZ3tmA2V4CL65dEB78xmEnMBnVKizyggaEQIfejIw+mrSRds3U0Yw
	0tBBmy6JEWPBfNHuHSvKQdKUBby/TR+wSbz0bLfYDoDGI3T7k5hiRwV8AcGCxPoyZ6zI=;
Subject: Re: [PATCH 01/11] xen: reserve flags for internal usage in
 xen_domctl_createdomain
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-2-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <7da75d7c-5640-305d-d742-d64b8c219949@xen.org>
Date: Thu, 23 Sep 2021 14:54:48 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-2-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> We are passing an extra special boolean flag at domain creation to
> specify whether we want to the domain to be privileged (i.e. dom0) or
> not. Another flag will be introduced later in this series.
> 
> Reserve bits 16-31 from the existing flags bitfield in struct
> xen_domctl_createdomain for internal Xen usage.

I am a bit split with this approach. This feels a bit of a hack to 
reserve bits for internal purpose in external headers. But at the same 
time I can see how this is easier to deal with it over repurposing the 
last argument of domain_create().

I would suggest to add an extra sentence to explain the goal is to 
consolidate all the information for a domain in a single structure.

>  > Allocate bit 16 for XEN_DOMCTL_INTERNAL_ispriv: whether a domain is
> privileged or not.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> CC: andrew.cooper3@citrix.com
> CC: jbeulich@suse.com
> CC: George Dunlap <George.Dunlap@eu.citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: "Roger Pau Monné" <roger.pau@citrix.com>
> CC: George Dunlap <george.dunlap@eu.citrix.com>
> CC: Dario Faggioli <dfaggioli@suse.com>
> ---
>   xen/arch/arm/domain.c       |  2 +-
>   xen/arch/arm/domain_build.c |  7 ++++---
>   xen/arch/x86/setup.c        |  4 +++-
>   xen/common/domain.c         | 19 +++++++++----------
>   xen/common/domctl.c         |  3 ++-
>   xen/common/sched/core.c     |  2 +-
>   xen/include/public/domctl.h |  3 +++
>   xen/include/xen/domain.h    |  4 ++++
>   xen/include/xen/sched.h     |  3 +--
>   9 files changed, 28 insertions(+), 19 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..7922249d26 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -623,7 +623,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>       unsigned int max_vcpus;
>   
>       /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
> +    if ( ((config->flags & XEN_DOMCTL_CDF_MASK) & ~XEN_DOMCTL_CDF_iommu) !=
>            (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>       {
>           dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d634c1..8cc4c800e9 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2679,7 +2679,7 @@ void __init create_domUs(void)
>            * very important to use the pre-increment operator to call
>            * domain_create() with a domid > 0. (domid == 0 is reserved for Dom0)
>            */
> -        d = domain_create(++max_init_domid, &d_cfg, false);
> +        d = domain_create(++max_init_domid, &d_cfg);
>           if ( IS_ERR(d) )
>               panic("Error creating domain %s\n", dt_node_name(node));
>   
> @@ -2752,7 +2752,8 @@ void __init create_dom0(void)
>   {
>       struct domain *dom0;
>       struct xen_domctl_createdomain dom0_cfg = {
> -        .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap,
> +        .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
> +                 XEN_DOMCTL_INTERNAL_ispriv,
>           .max_evtchn_port = -1,
>           .max_grant_frames = gnttab_dom0_frames(),
>           .max_maptrack_frames = -1,
> @@ -2773,7 +2774,7 @@ void __init create_dom0(void)
>       if ( iommu_enabled )
>           dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>   
> -    dom0 = domain_create(0, &dom0_cfg, true);
> +    dom0 = domain_create(0, &dom0_cfg);
>       if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>           panic("Error creating domain 0\n");
>   
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index b101565f14..6b7a1a3994 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -770,9 +770,11 @@ static struct domain *__init create_dom0(const module_t *image,
>   
>       if ( iommu_enabled )
>           dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> +    if ( !pv_shim )
> +        dom0_cfg.flags |= XEN_DOMCTL_INTERNAL_ispriv;
>   
>       /* Create initial domain 0. */
> -    d = domain_create(get_initial_domain_id(), &dom0_cfg, !pv_shim);
> +    d = domain_create(get_initial_domain_id(), &dom0_cfg);
>       if ( IS_ERR(d) || (alloc_dom0_vcpu0(d) == NULL) )
>           panic("Error creating domain 0\n");
>   
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6ee5d033b0..5fcca9b018 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -475,11 +475,11 @@ static void _domain_destroy(struct domain *d)
>   
>   static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>   {
> -    bool hvm = config->flags & XEN_DOMCTL_CDF_hvm;
> -    bool hap = config->flags & XEN_DOMCTL_CDF_hap;
> -    bool iommu = config->flags & XEN_DOMCTL_CDF_iommu;
> +    bool hvm = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_hvm;
> +    bool hap = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_hap;
> +    bool iommu = (config->flags & XEN_DOMCTL_CDF_MASK) & XEN_DOMCTL_CDF_iommu;

Adding "& XEN_DOMCTL_CDF_MASK" in the 3 lines above looks rather pointless.

>   
> -    if ( config->flags &
> +    if ( (config->flags & XEN_DOMCTL_CDF_MASK) &
>            ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>              XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>              XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> @@ -536,8 +536,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>   }
>   
>   struct domain *domain_create(domid_t domid,
> -                             struct xen_domctl_createdomain *config,
> -                             bool is_priv)
> +                             struct xen_domctl_createdomain *config)
>   {
>       struct domain *d, **pd, *old_hwdom = NULL;
>       enum { INIT_watchdog = 1u<<1,
> @@ -563,7 +562,7 @@ struct domain *domain_create(domid_t domid,
>       }
>   
>       /* Sort out our idea of is_control_domain(). */
> -    d->is_privileged = is_priv;
> +    d->is_privileged = config ? config->flags & XEN_DOMCTL_INTERNAL_ispriv : 0;

config->flags will be stored in d->options. Given there is a single 
caller for d->is_privileged, I would drop the field and replace the use 
with d->options & XEN_DOMCTL_INTERAL_ispriv?

>   
>       /* Sort out our idea of is_hardware_domain(). */
>       if ( domid == 0 || domid == hardware_domid )
> @@ -756,7 +755,7 @@ void __init setup_system_domains(void)
>        * Hidden PCI devices will also be associated with this domain
>        * (but be [partly] controlled by Dom0 nevertheless).
>        */
> -    dom_xen = domain_create(DOMID_XEN, NULL, false);
> +    dom_xen = domain_create(DOMID_XEN, NULL);
>       if ( IS_ERR(dom_xen) )
>           panic("Failed to create d[XEN]: %ld\n", PTR_ERR(dom_xen));
>   
> @@ -766,7 +765,7 @@ void __init setup_system_domains(void)
>        * array. Mappings occur at the priv of the caller.
>        * Quarantined PCI devices will be associated with this domain.
>        */
> -    dom_io = domain_create(DOMID_IO, NULL, false);
> +    dom_io = domain_create(DOMID_IO, NULL);
>       if ( IS_ERR(dom_io) )
>           panic("Failed to create d[IO]: %ld\n", PTR_ERR(dom_io));
>   
> @@ -775,7 +774,7 @@ void __init setup_system_domains(void)
>        * Initialise our COW domain.
>        * This domain owns sharable pages.
>        */
> -    dom_cow = domain_create(DOMID_COW, NULL, false);
> +    dom_cow = domain_create(DOMID_COW, NULL);
>       if ( IS_ERR(dom_cow) )
>           panic("Failed to create d[COW]: %ld\n", PTR_ERR(dom_cow));
>   #endif
> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> index 12d6144d28..2ec6d454dd 100644
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -431,7 +431,8 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>               rover = dom;
>           }
>   
> -        d = domain_create(dom, &op->u.createdomain, false);
> +        op->u.createdomain.flags &= XEN_DOMCTL_CDF_MASK;

I think it is a bad idea to silently ignore the reserved bits for 
internal purpose. Instead, we should check and return and error if they 
are set.

> +        d = domain_create(dom, &op->u.createdomain);
>           if ( IS_ERR(d) )
>           {
>               ret = PTR_ERR(d);
> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
> index 8f4b1ca10d..27d5bc2259 100644
> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -3021,7 +3021,7 @@ void __init scheduler_init(void)
>           sched_ratelimit_us = SCHED_DEFAULT_RATELIMIT_US;
>       }
>   
> -    idle_domain = domain_create(DOMID_IDLE, NULL, false);
> +    idle_domain = domain_create(DOMID_IDLE, NULL);
>       BUG_ON(IS_ERR(idle_domain));
>       BUG_ON(nr_cpu_ids > ARRAY_SIZE(idle_vcpu));
>       idle_domain->vcpu = idle_vcpu;
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842..4d3fcd3bcb 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -74,6 +74,9 @@ struct xen_domctl_createdomain {
>   /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
>   #define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
>   
> +/* Flags from (1U<<16) to (1U<<31) are reserved for internal usage */
> +#define XEN_DOMCTL_CDF_MASK           (0xffff)

I would suggest to add U at the end to make clear this is a unsigned value.

> +
>       uint32_t flags;
>   
>   #define _XEN_DOMCTL_IOMMU_no_sharept  0
> diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
> index 1708c36964..7ed0b62b78 100644
> --- a/xen/include/xen/domain.h
> +++ b/xen/include/xen/domain.h
> @@ -28,6 +28,10 @@ void getdomaininfo(struct domain *d, struct xen_domctl_getdomaininfo *info);
>   void arch_get_domain_info(const struct domain *d,
>                             struct xen_domctl_getdomaininfo *info);
>   
> +/* Flags are reserved for internal usage */
> +#define _XEN_DOMCTL_INTERNAL_ispriv         16
> +#define XEN_DOMCTL_INTERNAL_ispriv          (1U<<_XEN_DOMCTL_INTERNAL_ispriv)

Coding style: space before and after <<.

> +
>   /*
>    * Arch-specifics.
>    */
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 28146ee404..a659b25dea 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -664,8 +664,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config);
>    * (domid < DOMID_FIRST_RESERVED).
>    */
>   struct domain *domain_create(domid_t domid,
> -                             struct xen_domctl_createdomain *config,
> -                             bool is_priv);
> +                             struct xen_domctl_createdomain *config);
>   
>   /*
>    * rcu_lock_domain_by_id() is more efficient than get_domain_by_id().
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 09:58:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 09:58:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193596.344839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLV0-0001bd-EJ; Thu, 23 Sep 2021 09:58:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193596.344839; Thu, 23 Sep 2021 09:58:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLV0-0001bW-BB; Thu, 23 Sep 2021 09:58:38 +0000
Received: by outflank-mailman (input) for mailman id 193596;
 Thu, 23 Sep 2021 09:58:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTLUz-0001bQ-66
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 09:58:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b5de0de3-2041-4c49-9524-17e001ebd5ca;
 Thu, 23 Sep 2021 09:58:36 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-k_lmU9kDMnCB2eyoKPSt_w-1; Thu, 23 Sep 2021 11:58:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 09:58:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 09:58:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FRYP281CA0007.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Thu, 23 Sep 2021 09:58:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5de0de3-2041-4c49-9524-17e001ebd5ca
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632391115;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZUJVWCi178wU2K6APwOZWBURHqQ6lSLk4CxpqjOY//Y=;
	b=VvP4EX1Gc15DpnCB/pgO2AEFwtEqltYDntD1aWY2V3j2va0mbl14wsS7gvr9knKYFSxitQ
	HtXbZuwHzzbg5Op6/51Fq9f/s0rgzqlWHI4pNC0M8hQNLbWyH5/LQPxLzaaTwdAlENZ8ch
	UNwYECxTXxmfjs/cYk9EwlS0e2f1Y/I=
X-MC-Unique: k_lmU9kDMnCB2eyoKPSt_w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mQtCuUWIn/2i6cXNLHshDgx9MulkZBYF/z6qv2tdw6c678f76QV0DMO+LJuEssSu9vegYqqzGvDXELjQ7fUntCESaNoraPsXa2N+CQx0Dlgn7FMrvjpwkA8/JCjm2YcXKxGMu5tXxJKwjYaQxqYVnHxYV9UW+EWdBGAj79ZCZI4aewE0BW9AF/3A58FgfQlWaVis3CzQLFqdnYCtJmbgli2XAxJbG+GOsqkeXBpcrskUxk9eTQ8dFF5hrv7VvHHt6mG3r2Ey1+ozfsR9RxtF+HOVY/VhiQZ3L4MmGumjj0gBNjJ+ao71u6cJK9BbJSUyzn9uEs1BVqCmBqWW5uUX2w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eOMrY5KR7MGvqsNHWNxoLZPlaZKGm2IB8vaftL/+ERk=;
 b=O2fFsfROYxEu8ojJxh5Z9CLB64h9gtkAy4rGnYBS+t5vw2sZ3/k4vvX5F83h6EmbnbCwT/W/chPBzdN7ezGyb4+HuXllklGyVqU3ebNJWQnHibmc8x3Ron387kp6SvuXTp1nYDjhV9c8FXxDKl3oF89gq/XNUonyetc7DO5tdF7Fx8pJDJfqaMHfq7UfWeoONMT6AnZsvs78CGJJ2l+e+eknUqCcd1+Nujx6YI8NQJCx6xkK6j0QSDkZ1dgJRaYYArx9/gFBqKR+VqG+Mtqvy125jHAsB+2sqjWdHkupFQii8/A9GNeiDeD6PK/yoLobGXgHWR9ufzhDjRYVr2puGw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
 <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
 <e1def032-d8dd-f5df-c4f7-a9968083f06c@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4438c535-b83f-8b15-91e9-1815382180dc@suse.com>
Date: Thu, 23 Sep 2021 11:58:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e1def032-d8dd-f5df-c4f7-a9968083f06c@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FRYP281CA0007.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::17)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 69f558f3-9473-41da-670a-08d97e78b30c
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2606EF263EB20E8BAC1C3068B3A39@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8NroSra/PMnZikpOmQ3FKzz/sDVopOQE5fkkPDsd3SwRtgLfxK4ByOKo54STaUsuSccOKLsPH05Z6POcsLnCzRrDSixYfeAQpe34vB3ImUZXB2Wui9KziJgZJdpZLJCbfl9mAoUnZbJoJB/OEzyVN3wJo9Fnv2RjHeYlu7YjavWmgL7btp525/VW4n39H95KF/mVuNu1QzEGAed7IcfC7XJsT1s1Pv/UWNjT3FWmeVlgbrFgKWWa4lCHolgDA/7YcwTg70s43fLPsf0uMuNn1VJpOr4BkJFZ2+iwQSHMK/jvPHNRYy8OdQ0xCIOV3Qtv80bGGSHUclwxbjrX7W2fQjpspsOhAD2r9ECK+xWJroJYY5YG83ZBP7cvK/alo2NcKPysd5CXnZwWWVW25Ws3BIF8lNtaImDJgOLTmcctJqXciLST6NzFb/zdB5PaPh/+vbMTKic3lrF5IDf8UpIZ878noOmbN+8UakBmI4v2jSFF3K8+mW2gT3e5Ko7cKhZkKv90RtX1IwWHSE/ku9i9/rRBP/73bdzegHHjV0cTDE19SbZUOoP+IR+wtcAaR/uAtkhNXsPbsFVBXoYj8kWQGObKEu3xHLfcEpXWjUg4COi1ndOTbWvHlSmcpiQbyloV9uhPHHprLrHWnLA/LudBxPxy2wecPOQTxEu5+DozI7tMEDR/j/UIdNS680TrZ1H5y5Q9BRcgNHgcVhUZpaCGXBaF57wir6CND06ZaerIVVFbCHGoifsbZ1mS7/tD8yWj4r8aPFzIRFb9M/F6nn1JtQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(16576012)(8676002)(186003)(54906003)(31686004)(5660300002)(4326008)(8936002)(2906002)(26005)(53546011)(956004)(66556008)(2616005)(31696002)(36756003)(66476007)(6486002)(66946007)(508600001)(316002)(6916009)(38100700002)(21314003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?eSf2Ur9tLFqOZP5xFuEl/mr8w5acZliQeOqwYizZ9LpdtcaUsY+N5zY5yGIs?=
 =?us-ascii?Q?0pKShnCAa5Y3LZbzSjvhwqM63k1vB7Cx+ARrylwpZCoWcqmpmrq3jiJC05qY?=
 =?us-ascii?Q?SB1zR0NI39RHWnkpUtbTt257WGkpiZpA0vC2dsgTB1TMW02X5w+gbJinNO/Z?=
 =?us-ascii?Q?8yVzRzyjXfNv8uQpl3SoX7KWmmkoNWKUKp+yH1bU9V6QS+JVYQBjnptSQKh+?=
 =?us-ascii?Q?2VV7iVHO3MSUAr6/qVMqnjaYMnIItolN29m8dZii/Qdd9m1947iHzc8arhj1?=
 =?us-ascii?Q?laab6n6DQF9HiN62GxFRQ08wPi9+jjxBilwrPHbF32t92GrEPEvjjTpRCKUU?=
 =?us-ascii?Q?MrUOaOtt6aIkL+8hWBtzrxJdLorjt1hWNZxODzi0WT1ElNTbWF8rsuHWUP2Y?=
 =?us-ascii?Q?koZfFhgrXbwDSupiYY9uEyXt7Z2/cRIZHONN66H7+oX+V61gksiGETdRGqLe?=
 =?us-ascii?Q?QSYlIN2Dchev6YDVFaah/ryHOkg+jwjz9hKkhMMrlxF/ULLzgg91dRQF9in1?=
 =?us-ascii?Q?x6wBGc/2ThtjpPb2H6TtF5ZsTC8Wp0DEWPR/GeGlqaRcOvKS6Cv0cNOXFLwE?=
 =?us-ascii?Q?bgA+UsMFoVg36YTSgDw6N8RvHNA3CkFbq/FR4BRrHy9BxSxIx+jKaVrjjI0V?=
 =?us-ascii?Q?YE+cViCAXGt8kYfio+IVfBmiq0kiA2qs7DRVvjRAl8+m0BMsneGNaGRILhtE?=
 =?us-ascii?Q?CfEMPantnk1CXO2ZgUBVmYQc+y01okUI6IobETwS5gW1C5IkYIEcKKkA+r4C?=
 =?us-ascii?Q?Qm/RUtFXr4SB+dpje6241e1+2iq48h1DTgk9/wZDfiBJqarIrVg3UwsncfzH?=
 =?us-ascii?Q?vFu+qWIDHcx/jM6ttcPdv44zLnBqlH+asct8hfZX1dBrXdNfZn0BFkHmn4H0?=
 =?us-ascii?Q?ug2ow2lfw5dAnfQ3ixSpWHr+zq2IqyvDTWtHkFSsF7azMvLhiPvCALWl9v9t?=
 =?us-ascii?Q?dbQCHa3pd+WfM2N7wnom7gPzkadCwS/z6nCQSNNHS9/oBlkvI5gidasUAB/H?=
 =?us-ascii?Q?aC1vhLHGwmr7aqxAFuorHtc8Eq5MKEdlRf0ZQueywLAy8tTX8t0QOERxZLbH?=
 =?us-ascii?Q?KGO8g4MYu/9srmplwzsWwCyX74A6eSN9SQ0qEzrlzyGh2tmuDDiVCNWB1Dl0?=
 =?us-ascii?Q?l/60kHYFVlX5vk3g9Fx2/g/sdhcFaXe2DXy01cRcFAk2pUYW9mwcms9wAgzs?=
 =?us-ascii?Q?p1vasuYLu1vr8tpsSOX9E7KhsCQsl0aKYKjIkgG2+7OlJhANiOIKvtBHGJtU?=
 =?us-ascii?Q?EnSa7gzVjgYKSVQv3e+RWyVQ+6uOqjc7HCzwX/e71GIOlSQjFKWisL31y1VI?=
 =?us-ascii?Q?3gNQdZO39Pd9IlJMSbi17rnD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 69f558f3-9473-41da-670a-08d97e78b30c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 09:58:31.5595
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 18w7lkAmADHAYGc6t1FqR6CVo1aP8RcxqqRFZ5Ek5xZ/XZGR6beHQtusKlCErsuQxMcfaH4ePyE7pPNjzOXPeg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

On 22.09.2021 19:03, Andrew Cooper wrote:
> On 22/09/2021 16:01, Roger Pau Monn=C3=A9 wrote:
>> On Tue, Sep 21, 2021 at 09:18:05AM +0200, Jan Beulich wrote:
>>> --- a/xen/include/public/arch-x86/hvm/start_info.h
>>> +++ b/xen/include/public/arch-x86/hvm/start_info.h
>>> @@ -33,7 +33,7 @@
>>>   *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_=
VALUE
>>>   *    |                | ("xEn3" with the 0x80 bit of the "E" set).
>>>   *  4 +----------------+
>>> - *    | version        | Version of this structure. Current version is=
 1. New
>>> + *    | version        | Version of this structure. Current version is=
 2. New
>>>   *    |                | versions are guaranteed to be backwards-compa=
tible.
>>>   *  8 +----------------+
>>>   *    | flags          | SIF_xxx flags.
>>> @@ -55,7 +55,15 @@
>>>   *    |                | if there is no memory map being provided. Onl=
y
>>>   *    |                | present in version 1 and newer of the structu=
re.
>>>   * 52 +----------------+
>>> - *    | reserved       | Version 1 and newer only.
>>> + *    | vga_info.offset| Offset of struct dom0_vga_console_info from b=
ase of
>> I'm not sure we are supposed to reference external structures like
>> that. We took a lot of care to not depend on other headers, and to
>> make this as agnostic as possible (IIRC KVM is also capable of using
>> the PVH entry point natively, and hence depends on this header).
>=20
> Absolutely correct.=C2=A0 C is not an acceptable ABI description.

See my reply to Roger's earlier mail.

> Furthermore, dom0_vga_console_info is a bad ABI to start with, as
> demonstrated by the multiple problems we've had extending it in the past.

I don't view this as "problems", nor do I think we couldn't extend it
further that same way, if need be.

> The MB1/2 framebuffer information would be a rather better example to
> follow,

Maybe, but I'm not sure - it doesn't look any better extensibility-wise
than dom0_vga_console_info. Also MB1 doesn't really have separate
structures, so if anything it would need to be MB2.

> but we'll surely need to pass the EDID string too (at least in
> the case that there aren't EFI runtime services to use).

According to the understanding I've gained while putting together the
patch to retrieve EDID info when running under EFI, there's no way to
obtain these via runtime services. Yet I also don't see why we would
need to pass this here - we've got XENPF_firmware_info to retrieve
this data, and I didn't think we need PVH to behave differently from
PV in such regards.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:00:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:00:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193601.344849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLWp-00030Q-QN; Thu, 23 Sep 2021 10:00:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193601.344849; Thu, 23 Sep 2021 10:00:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLWp-00030J-NT; Thu, 23 Sep 2021 10:00:31 +0000
Received: by outflank-mailman (input) for mailman id 193601;
 Thu, 23 Sep 2021 10:00:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTLWo-00030B-Qg
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:00:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTLWo-0004ud-FJ; Thu, 23 Sep 2021 10:00:30 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTLWo-0005TV-5O; Thu, 23 Sep 2021 10:00:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=a+mRos+FxOv5H2KRZ2STHOwwYTDpULtpygDmSpWT4wo=; b=bV1Ouviu6sHJ1yHi7zFVSNbvJL
	eJJt9G87xVr3eAaUUyU3oieV7TVee8GNxpi6X4pnkPRdi1kzaE+E1amgdmnI78Hfrtp0pdeUZpIzz
	EFwJbUUj0Estqe3IdBRSHy1c1mbNZbtTXr6Jb6EpQhavbx7bwjFxLQXdDXh7/TPUcdvY=;
Subject: Re: [PATCH 02/11] xen/arm: introduce XEN_DOMCTL_INTERNAL_directmap
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-3-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <e059836c-d3f4-70d4-c4f6-cb15b1b787cc@xen.org>
Date: Thu, 23 Sep 2021 15:00:26 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-3-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> This commit introduces a new ARM-specific flag to specify that the
> domain should be 1:1 directly mapped (guest physical addresses ==

"1:1" and "directly" means the exactly the same. Please remove either.

> physical addresses).
> 
> Also, add a direct_map flag under struct arch_domain and use it to
> implement is_domain_direct_mapped.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> CC: andrew.cooper3@citrix.com
> CC: jbeulich@suse.com
> CC: George Dunlap <George.Dunlap@eu.citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: "Roger Pau Monné" <roger.pau@citrix.com>
> ---
>   xen/arch/arm/domain.c        | 1 +
>   xen/arch/arm/domain_build.c  | 2 +-
>   xen/include/asm-arm/domain.h | 9 +++++++--
>   xen/include/xen/domain.h     | 4 ++++
>   4 files changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 7922249d26..0b3cff8a40 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d,
>           return 0;
>   
>       ASSERT(config != NULL);
> +    d->arch.direct_map = config->flags & XEN_DOMCTL_INTERNAL_directmap;
>   
>   #ifdef CONFIG_IOREQ_SERVER
>       ioreq_domain_init(d);
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 8cc4c800e9..21d8a559af 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2753,7 +2753,7 @@ void __init create_dom0(void)
>       struct domain *dom0;
>       struct xen_domctl_createdomain dom0_cfg = {
>           .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
> -                 XEN_DOMCTL_INTERNAL_ispriv,
> +                 XEN_DOMCTL_INTERNAL_ispriv | XEN_DOMCTL_INTERNAL_directmap,
>           .max_evtchn_port = -1,
>           .max_grant_frames = gnttab_dom0_frames(),
>           .max_maptrack_frames = -1,
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..a74ee5720c 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -29,8 +29,11 @@ enum domain_type {
>   #define is_64bit_domain(d) (0)
>   #endif
>   
> -/* The hardware domain has always its memory direct mapped. */
> -#define is_domain_direct_mapped(d) is_hardware_domain(d)
> +/*
> + * The hardware domain has always its memory direct mapped. And DOM0 shall
> + * be always been set as 1:1 direct-map domain.
> + */

I think this comment should be moved on top of dom0_cfg.flags. This will 
prevent in stall comment if in the future we decide to remove the direct 
map (I know that cache coloring will want to drop it).

> +#define is_domain_direct_mapped(d) (d)->arch.direct_map
>   
>   struct vtimer {
>       struct vcpu *v;
> @@ -89,6 +92,8 @@ struct arch_domain
>   #ifdef CONFIG_TEE
>       void *tee;
>   #endif
> +
> +    bool direct_map;

We already store the flag in d->options. So this is a bit redundant.

>   }  __cacheline_aligned;
>   
>   struct arch_vcpu
> diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
> index 7ed0b62b78..6c2b07eb42 100644
> --- a/xen/include/xen/domain.h
> +++ b/xen/include/xen/domain.h
> @@ -32,6 +32,10 @@ void arch_get_domain_info(const struct domain *d,
>   #define _XEN_DOMCTL_INTERNAL_ispriv         16
>   #define XEN_DOMCTL_INTERNAL_ispriv          (1U<<_XEN_DOMCTL_INTERNAL_ispriv)
>   
> +/* This flag is ARM specific */
> +#define _XEN_DOMCTL_INTERNAL_directmap      17
> +#define XEN_DOMCTL_INTERNAL_directmap       (1U<<_XEN_DOMCTL_INTERNAL_directmap)

Coding style: space before and after <<.

> +
>   /*
>    * Arch-specifics.
>    */
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:10:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:10:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193608.344860 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLgA-0004kh-SM; Thu, 23 Sep 2021 10:10:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193608.344860; Thu, 23 Sep 2021 10:10:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLgA-0004ka-PW; Thu, 23 Sep 2021 10:10:10 +0000
Received: by outflank-mailman (input) for mailman id 193608;
 Thu, 23 Sep 2021 10:10:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTLg8-0004kT-Th
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:10:08 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a8b089b8-8929-41d8-980d-71910081bfd3;
 Thu, 23 Sep 2021 10:10:07 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id e15so24421161lfr.10
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 03:10:07 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id q2sm322992lfm.306.2021.09.23.03.10.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 03:10:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8b089b8-8929-41d8-980d-71910081bfd3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=lOLKsC7QU6uW8UPH3QxwQddTYgFfgaG2iDWPw7J8H98=;
        b=OlJMvU4ZCXrM3UsHVnf9+ZcErpogjdDpGTxh99kuEufDK/3JUXUIpkBS+zIwZIH1cS
         PgR5C19VzKFbIfi/ir7Eydr/nURhxSHOcyuKNXweAxXwbFOG6kbKv2eijp2Zl9tpzvxP
         /s1QK3z1x8+xmm85CY2h2NZvxUWK2l3Zhva4AluZQpccF7lSLk6JjFrbq7Bo3/IFBr1O
         9luTcYZznpxU+x1UGrvY6tc9ZUoX1z3vni3vJ+QXmdWDSskIVs1/mxS96CvDB2xed8iN
         d2oWW2ifWj/+rGlZEJwchq4BPand+W/IKiwVXgldvlnbXWHMNwyB2StwZWiisCwcQIsI
         /KMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=lOLKsC7QU6uW8UPH3QxwQddTYgFfgaG2iDWPw7J8H98=;
        b=BHSoCQAAsxlmQC+ePFunsqgkL1YF0zDSQzfVgsOZX4dWyTEqXsyirkRpaX19xpc8bx
         hpDd2wRhTBsspM3JTQaPCs/4rimskZJpBYDqKEfzG1hwsNU2vGXrpImrKasD9oEKfxWt
         q7SmMLHpfxXAZZFsPP8TapN+URCrMOislJWwJQuXomSf8tAxX51LIuIo9oAU62gcJzC0
         btCDq0F/OGpXmIL7gbsd73dtSHrF4KL2taKxbbpFFLJtzSkOor9o0XfUyQwMDNleVWax
         VCFSRYBgvaFetYB9RwZMQOdWj+jqbjPaJzN1KxxhHJmG+vox4bT/d+ulaQFZdNY3ZMU/
         O88g==
X-Gm-Message-State: AOAM530Kkufqg3vV8P9gkW6Yy3UvYHiITwV3mI5EcOfZebsMyvZ3zNiV
	Knbt0kU6wet8qgDzkXyNTt4=
X-Google-Smtp-Source: ABdhPJxjKo/smQaTt2FedPGvDnyq0u1oWSXNTOJ5e+8VMD45ubQG6sBSN0YYQK9BXX9oNowfIMkEQg==
X-Received: by 2002:a05:651c:88b:: with SMTP id d11mr4328212ljq.126.1632391805873;
        Thu, 23 Sep 2021 03:10:05 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>, fnuv@xilinx.com
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109171459200.21985@sstabellini-ThinkPad-T480s>
 <2a1cf877-60a3-c8c5-8982-51ef3606b977@xen.org>
 <1a2cd4d2-7ec3-266a-9cba-e4ab49c3aaef@gmail.com>
 <alpine.DEB.2.21.2109201619020.17979@sstabellini-ThinkPad-T480s>
 <df9e2f08-b21c-902c-673a-1d690088a98b@gmail.com>
 <alpine.DEB.2.21.2109211439370.17979@sstabellini-ThinkPad-T480s>
 <1ac58681-ef42-bb32-25b6-620d51d4f075@gmail.com>
 <alpine.DEB.2.21.2109221349350.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <5e2dd68c-0d17-a564-fd87-e635044a8cbf@gmail.com>
Date: Thu, 23 Sep 2021 13:10:04 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109221349350.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 22.09.21 23:50, Stefano Stabellini wrote:

Hi Stefano.

> On Wed, 22 Sep 2021, Oleksandr wrote:
>> On 22.09.21 01:00, Stefano Stabellini wrote:
>>> On Tue, 21 Sep 2021, Oleksandr wrote:
>>>> On 21.09.21 02:21, Stefano Stabellini wrote:
>>>>> On Sun, 19 Sep 2021, Oleksandr wrote:
>>>>>>> On 18/09/2021 03:37, Stefano Stabellini wrote:
>>>>>>>> On Fri, 17 Sep 2021, Stefano Stabellini wrote:
>>>>>>>>> On Fri, 17 Sep 2021, Oleksandr wrote:
>>>>>>>>>>>> +
>>>>>>>>>>>> +    dt_dprintk("Find unallocated memory for extended
>>>>>>>>>>>> regions\n");
>>>>>>>>>>>> +
>>>>>>>>>>>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>>>>>>>>>>>> +    if ( !unalloc_mem )
>>>>>>>>>>>> +        return -ENOMEM;
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Start with all available RAM */
>>>>>>>>>>>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>>>>>>>>>>> +    {
>>>>>>>>>>>> +        start = bootinfo.mem.bank[i].start;
>>>>>>>>>>>> +        end = bootinfo.mem.bank[i].start +
>>>>>>>>>>>> bootinfo.mem.bank[i].size - 1;
>>>>>>>>>>>> +        res = rangeset_add_range(unalloc_mem, start,
>>>>>>>>>>>> end);
>>>>>>>>>>>> +        if ( res )
>>>>>>>>>>>> +        {
>>>>>>>>>>>> +            printk(XENLOG_ERR "Failed to add:
>>>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>>>> +                   start, end);
>>>>>>>>>>>> +            goto out;
>>>>>>>>>>>> +        }
>>>>>>>>>>>> +    }
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Remove RAM assigned to Dom0 */
>>>>>>>>>>>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>>>>>>>>>>> +    {
>>>>>>>>>>>> +        start = assign_mem->bank[i].start;
>>>>>>>>>>>> +        end = assign_mem->bank[i].start +
>>>>>>>>>>>> assign_mem->bank[i].size - 1;
>>>>>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start,
>>>>>>>>>>>> end);
>>>>>>>>>>>> +        if ( res )
>>>>>>>>>>>> +        {
>>>>>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>>>> +                   start, end);
>>>>>>>>>>>> +            goto out;
>>>>>>>>>>>> +        }
>>>>>>>>>>>> +    }
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Remove reserved-memory regions */
>>>>>>>>>>>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++
>>>>>>>>>>>> )
>>>>>>>>>>>> +    {
>>>>>>>>>>>> +        start = bootinfo.reserved_mem.bank[i].start;
>>>>>>>>>>>> +        end = bootinfo.reserved_mem.bank[i].start +
>>>>>>>>>>>> +            bootinfo.reserved_mem.bank[i].size - 1;
>>>>>>>>>>>> +        res = rangeset_remove_range(unalloc_mem, start,
>>>>>>>>>>>> end);
>>>>>>>>>>>> +        if ( res )
>>>>>>>>>>>> +        {
>>>>>>>>>>>> +            printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>>>> +                   start, end);
>>>>>>>>>>>> +            goto out;
>>>>>>>>>>>> +        }
>>>>>>>>>>>> +    }
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Remove grant table region */
>>>>>>>>>>>> +    start = kinfo->gnttab_start;
>>>>>>>>>>>> +    end = kinfo->gnttab_start + kinfo->gnttab_size - 1;
>>>>>>>>>>>> +    res = rangeset_remove_range(unalloc_mem, start, end);
>>>>>>>>>>>> +    if ( res )
>>>>>>>>>>>> +    {
>>>>>>>>>>>> +        printk(XENLOG_ERR "Failed to remove:
>>>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>>>> +               start, end);
>>>>>>>>>>>> +        goto out;
>>>>>>>>>>>> +    }
>>>>>>>>>>>> +
>>>>>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1,
>>>>>>>>>>>> EXT_REGION_END);
>>>>>>>>>>>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>>>>>>>>>>>> +                                 add_ext_regions,
>>>>>>>>>>>> ext_regions);
>>>>>>>>>>>> +    if ( res )
>>>>>>>>>>>> +        ext_regions->nr_banks = 0;
>>>>>>>>>>>> +    else if ( !ext_regions->nr_banks )
>>>>>>>>>>>> +        res = -ENOENT;
>>>>>>>>>>>> +
>>>>>>>>>>>> +out:
>>>>>>>>>>>> +    rangeset_destroy(unalloc_mem);
>>>>>>>>>>>> +
>>>>>>>>>>>> +    return res;
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> +static int __init find_memory_holes(const struct
>>>>>>>>>>>> kernel_info
>>>>>>>>>>>> *kinfo,
>>>>>>>>>>>> +                                    struct meminfo
>>>>>>>>>>>> *ext_regions)
>>>>>>>>>>>> +{
>>>>>>>>>>>> +    struct dt_device_node *np;
>>>>>>>>>>>> +    struct rangeset *mem_holes;
>>>>>>>>>>>> +    paddr_t start, end;
>>>>>>>>>>>> +    unsigned int i;
>>>>>>>>>>>> +    int res;
>>>>>>>>>>>> +
>>>>>>>>>>>> +    dt_dprintk("Find memory holes for extended
>>>>>>>>>>>> regions\n");
>>>>>>>>>>>> +
>>>>>>>>>>>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>>>>>>>>>>>> +    if ( !mem_holes )
>>>>>>>>>>>> +        return -ENOMEM;
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Start with maximum possible addressable physical
>>>>>>>>>>>> memory
>>>>>>>>>>>> range */
>>>>>>>>>>>> +    start = EXT_REGION_START;
>>>>>>>>>>>> +    end = min((1ULL << p2m_ipa_bits) - 1,
>>>>>>>>>>>> EXT_REGION_END);
>>>>>>>>>>>> +    res = rangeset_add_range(mem_holes, start, end);
>>>>>>>>>>>> +    if ( res )
>>>>>>>>>>>> +    {
>>>>>>>>>>>> +        printk(XENLOG_ERR "Failed to add:
>>>>>>>>>>>> %#"PRIx64"->%#"PRIx64"\n",
>>>>>>>>>>>> +               start, end);
>>>>>>>>>>>> +        goto out;
>>>>>>>>>>>> +    }
>>>>>>>>>>>> +
>>>>>>>>>>>> +    /* Remove all regions described by "reg" property
>>>>>>>>>>>> (MMIO,
>>>>>>>>>>>> RAM,
>>>>>>>>>>>> etc) */
>>>>>>>>>>> Well... The loop below is not going to handle all the
>>>>>>>>>>> regions
>>>>>>>>>>> described in
>>>>>>>>>>> the property "reg". Instead, it will cover a subset of "reg"
>>>>>>>>>>> where
>>>>>>>>>>> the
>>>>>>>>>>> memory is addressable.
>>>>>>>>>> As I understand, we are only interested in subset of "reg"
>>>>>>>>>> where
>>>>>>>>>> the
>>>>>>>>>> memory is
>>>>>>>>>> addressable.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> You will also need to cover "ranges" that will describe the
>>>>>>>>>>> BARs
>>>>>>>>>>> for
>>>>>>>>>>> the PCI
>>>>>>>>>>> devices.
>>>>>>>>>> Good point.
>>>>>>>>> Yes, very good point!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Could you please clarify how to recognize whether it is a PCI
>>>>>>>>>> device as long as PCI support is not merged? Or just to find
>>>>>>>>>> any
>>>>>>>>>> device nodes
>>>>>>>>>> with non-empty "ranges" property
>>>>>>>>>> and retrieve addresses?
>>>>>>>>> Normally any bus can have a ranges property with the aperture
>>>>>>>>> and
>>>>>>>>> possible address translations, including /amba (compatible =
>>>>>>>>> "simple-bus"). However, in these cases dt_device_get_address
>>>>>>>>> already
>>>>>>>>> takes care of it, see
>>>>>>>>> xen/common/device_tree.c:dt_device_get_address.
>>>>>>>>>
>>>>>>>>> The PCI bus is special for 2 reasons:
>>>>>>>>> - the ranges property has a different format
>>>>>>>>> - the bus is hot-pluggable
>>>>>>>>>
>>>>>>>>> So I think the only one that we need to treat specially is PCI.
>>>>>>>>>
>>>>>>>>> As far as I am aware PCI is the only bus (or maybe just the only
>>>>>>>>> bus
>>>>>>>>> that we support?) where ranges means the aperture.
>>>>>>>> Now that I think about this, there is another "hotpluggable"
>>>>>>>> scenario
>>>>>>>> we
>>>>>>>> need to think about:
>>>>>>>>
>>>>>>>> [1] https://marc.info/?l=xen-devel&m=163056546214978
>>>>>>>>
>>>>>>>> Xilinx devices have FPGA regions with apertures currently not
>>>>>>>> described
>>>>>>>> in device tree, where things can programmed in PL at runtime
>>>>>>>> making
>>>>>>>> new
>>>>>>>> devices appear with new MMIO regions out of thin air.
>>>>>>>>
>>>>>>>> Now let me start by saying that yes, the entire programmable
>>>>>>>> region
>>>>>>>> aperture could probably be described in device tree, however, in
>>>>>>>> reality it is not currently done in any of the device trees we use
>>>>>>>> (including the upstream device trees in linux.git).
>>>>>>> This is rather annoying, but not unheard. There are a couple of
>>>>>>> platforms
>>>>>>> where the MMIOs are not fully described in the DT.
>>>>>>>
>>>>>>> In fact, we have a callback 'specific_mappings' which create
>>>>>>> additional
>>>>>>> mappings (e.g. on the omap5) for dom0.
>>>>>>>
>>>>>>>> So, we have a problem :-(
>>>>>>>>
>>>>>>>>
>>>>>>>> I can work toward getting the right info on device tree, but in
>>>>>>>> reality
>>>>>>>> that is going to take time and for now the device tree doesn't
>>>>>>>> have
>>>>>>>> the
>>>>>>>> FPGA aperture in it. So if we accept this series as is, it is
>>>>>>>> going to
>>>>>>>> stop features like [1] from working. >
>>>>>>>> If we cannot come up with any better plans, I think it would be
>>>>>>>> better
>>>>>>>> to drop find_memory_holes, only rely on find_unallocated_memory
>>>>>>>> even
>>>>>>>> when the IOMMU is on. One idea is that we could add on top of the
>>>>>>>> regions found by find_unallocated_memory any MMIO regions marked
>>>>>>>> as
>>>>>>>> xen,passthrough: they are safe because they are not going to dom0
>>>>>>>> anyway.
>>>>>>> (Oleksandr, it looks like some rationale about the different
>>>>>>> approach is
>>>>>>> missing in the commit message. Can you add it?)
>>>>>> Yes sure, but let me please clarify what is different approach in this
>>>>>> context. Is it to *also* take into the account MMIO regions of the
>>>>>> devices
>>>>>> for
>>>>>> passthrough for case when IOMMU is off (in addition to unallocated
>>>>>> memory)? If
>>>>>> yes, I wonder whether we will gain much with that according to that
>>>>>> device's
>>>>>> MMIO regions are usually not big enough and we stick to allocate
>>>>>> extended
>>>>>> regions with bigger size (> 64MB).
>>>>> That's fair enough. There are a couple of counter examples where the
>>>>> MMIO regions for the device to assign are quite large, for instance a
>>>>> GPU, Xilinx AIEngine, or the PCIe Root Complex with the entire aperture,
>>>>> but maybe they are not that common. I am not sure if it is worth
>>>>> scanning the tree for xen,passthrough regions every time at boot for
>>>>> this.
>>>> ok, I will add a few sentences to commit message about this different
>>>> approach
>>>> for now. At least this could be implemented later on if there is a need.
>>> One thing that worries me about this is that if we take an old Xen with
>>> this series and run it on a new board, it might cause problems. At the
>>> very least [1] wouldn't work.
>> I got it.
>>
>>
>>> Can we have a Xen command line argument to disable extended regions as
>>> an emergecy toggle?
>> I think, yes. If no preference for the argument name I will name it
>> "no-ext-region".
> It is better to introduce it as ext-regions=yes/no with yes as default.
> So that in the future we could extending it to ext-regions=start,size if
> we wanted to.

ok, will do


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:11:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:11:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193613.344872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLhO-0005K9-8H; Thu, 23 Sep 2021 10:11:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193613.344872; Thu, 23 Sep 2021 10:11:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLhO-0005K2-3h; Thu, 23 Sep 2021 10:11:26 +0000
Received: by outflank-mailman (input) for mailman id 193613;
 Thu, 23 Sep 2021 10:11:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTLhN-0005Jw-Ef
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:11:25 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 708c158c-bf1e-4faa-986c-5c2c35e1d155;
 Thu, 23 Sep 2021 10:11:24 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id e15so24433785lfr.10
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 03:11:24 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id f9sm415449lfp.273.2021.09.23.03.11.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 03:11:22 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 708c158c-bf1e-4faa-986c-5c2c35e1d155
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=58SO+4woagQ86uv4XxkGyEU8aiY2WBIAciBZUmjfyug=;
        b=qcVcAJQL15o6NrXEqJPmGIba+258GyoDtWJwDuXzHlwzhyQo9a+q59/KZ2RYGMl68L
         gSPF0TxZf3FcXHk5DXmz6wmsXD9dg57PGg4USFKFrKxqupkL9YNl+AfpxVCbIUtfu/kg
         h7G4vJD4H7ptsUreuZUassCcJTo3lxD3opkim0ljpJNsFhncGGnqtMS6ydUsOvPgxHi8
         brMAYsK3q8KM52fA+5S7MR+0PGX+EUHOnv+vI/vKuNw5KBrT2YGM0Xh4hHX6+7NVkuAB
         Th/vhcsbp2BjiJ9KGiCMDFmZYP29cZIVIVBJyglrAL0l4twEkRJTIoQzCz9PgALDEmoV
         cWOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=58SO+4woagQ86uv4XxkGyEU8aiY2WBIAciBZUmjfyug=;
        b=P0ZHWWb201Z4NcoVz76qcneWuqUNKoID8SIOEPtjlwJyvAUcIKlCdBbYYXfvr8GBpk
         rUR6ENAs+UFUXypF4buZ4Mja+ZpbD5yCljNky/hz+kAt03n9CVS3GMzKfUDUt2z8D0QD
         gwYiqopXWG33TLPEXTj+52xkqY/kNVr9yBB3gO5Chx1l8IOXff1SKF9zxcrx9UUWJJeS
         4dkLIssNxC1E+tqXXPybdHT5834ku3hHlh/JbAlyzuvvRUpBRTpkzZ6zG/jKW5mppgbl
         ELVFz7BKixfyrgnLRMTKF2Cc69apU/JBZ6rPFkH4MboBdYLlqA5CEeEIhLlNqHMJspgc
         6deQ==
X-Gm-Message-State: AOAM532460+g6qZVmamvs1hMvureNbyV+cmK9Yh4wiIu5jt8cp39tzt0
	zElBeRY+IUG7ZQ2KMI4MqlGvBa733WU=
X-Google-Smtp-Source: ABdhPJxSevTTO33/ebEkxxR+OtI6fwGm4CN/VcYyiJHTl8C3q5NnS33dWI/AHtiHCqQjdjlcLRzqXg==
X-Received: by 2002:a2e:80ca:: with SMTP id r10mr4213837ljg.347.1632391883160;
        Thu, 23 Sep 2021 03:11:23 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <alpine.DEB.2.21.2109171451230.21985@sstabellini-ThinkPad-T480s>
 <a993466f-1b7d-ceb3-aa3b-16f5f145b2fb@gmail.com>
 <a3f5707b-b161-1c94-ebe1-d6b8bbe11081@gmail.com>
 <alpine.DEB.2.21.2109221402420.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <90071fcd-e7d8-3075-e235-44580d1434e1@gmail.com>
Date: Thu, 23 Sep 2021 13:11:22 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109221402420.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 23.09.21 00:05, Stefano Stabellini wrote:

Hi Stefano

> On Wed, 22 Sep 2021, Oleksandr wrote:
>>>>>> You will also need to cover "ranges" that will describe the BARs for
>>>>>> the PCI
>>>>>> devices.
>>>>> Good point.
>>>> Yes, very good point!
>>>>
>>>>
>>>>> Could you please clarify how to recognize whether it is a PCI
>>>>> device as long as PCI support is not merged? Or just to find any device
>>>>> nodes
>>>>> with non-empty "ranges" property
>>>>> and retrieve addresses?
>>>> Normally any bus can have a ranges property with the aperture and
>>>> possible address translations, including /amba (compatible =
>>>> "simple-bus"). However, in these cases dt_device_get_address already
>>>> takes care of it, see xen/common/device_tree.c:dt_device_get_address.
>>>>
>>>> The PCI bus is special for 2 reasons:
>>>> - the ranges property has a different format
>>>> - the bus is hot-pluggable
>>>>
>>>> So I think the only one that we need to treat specially is PCI.
>>>>
>>>> As far as I am aware PCI is the only bus (or maybe just the only bus
>>>> that we support?) where ranges means the aperture.
>>> Thank you for the clarification. I need to find device node with non-empty
>>> ranges property
>>> (and make sure that device_type property is "pci"), after that I need to
>>> read the context of ranges property and translate it.
>>>
>>>
>> OK, I experimented with that and managed to parse ranges property for PCI host
>> bridge node.
>>
>> I tested on my setup where the host device tree contains two PCI host bridge
>> nodes with the following:
>>
>> pcie@fe000000 {
>> ...
>>              ranges = <0x1000000 0x0 0x0 0x0 0xfe100000 0x0 0x100000 0x2000000
>> 0x0 0xfe200000 0x0 0xfe200000 0x0 0x200000 0x2000000 0x0 0x30000000 0x0
>> 0x30000000 0x0 0x8000000 0x42000000 0x0 0x38000000 0x0 0x38000000 0x0
>> 0x8000000>;
>> ...
>> };
>>
>> pcie@ee800000 {
>> ...
>>              ranges = <0x1000000 0x0 0x0 0x0 0xee900000 0x0 0x100000 0x2000000
>> 0x0 0xeea00000 0x0 0xeea00000 0x0 0x200000 0x2000000 0x0 0xc0000000 0x0
>> 0xc0000000 0x0 0x8000000 0x42000000 0x0 0xc8000000 0x0 0xc8000000 0x0
>> 0x8000000>;
>> ...
>> };
>>
>> So Xen retrieves the *CPU addresses* from the ranges:
>>
>> (XEN) dev /soc/pcie@fe000000 range_size 7 nr_ranges 4
>> (XEN) 0: addr=fe100000, size=100000
>> (XEN) 1: addr=fe200000, size=200000
>> (XEN) 2: addr=30000000, size=8000000
>> (XEN) 3: addr=38000000, size=8000000
>> (XEN) dev /soc/pcie@ee800000 range_size 7 nr_ranges 4
>> (XEN) 0: addr=ee900000, size=100000
>> (XEN) 1: addr=eea00000, size=200000
>> (XEN) 2: addr=c0000000, size=8000000
>> (XEN) 3: addr=c8000000, size=8000000
>>
>> The code below covers ranges property in the context of finding memory holes
>> (to be squashed with current patch):
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index d37156a..7d20c10 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -834,6 +834,8 @@ static int __init find_memory_holes(struct meminfo
>> *ext_regions)
>>       {
>>           unsigned int naddr;
>>           u64 addr, size;
>> +        const __be32 *ranges;
>> +        u32 len;
>>
>>           naddr = dt_number_of_address(np);
>>
>> @@ -857,6 +859,41 @@ static int __init find_memory_holes(struct meminfo
>> *ext_regions)
>>                   goto out;
>>               }
>>           }
>> +
>> +        /*
>> +         * Also looking for non-empty ranges property which would likely mean
>> +         * that we deal with PCI host bridge device and the property here
>> +         * describes the BARs for the PCI devices.
>> +         */
> One thing to be careful is that ranges with a valid parameter is not
> only present in PCI busses. It can be present in amba and other
> simple-busses too. In that case the format for ranges in simpler as it
> doesn't have a "memory type" like PCI.
>
> When you get addresses from reg, bus ranges properties are automatically
> handled for you.
>
> All of this to say that a check on "ranges" is not enough because it
> might capture other non-PCI busses that have a different, simpler,
> ranges format. You want to check for "ranges" under a device_type =
> "pci"; node.

ok, will do.


>
>
>> +        ranges = dt_get_property(np, "ranges", &len);
>> +        if ( ranges && len )
>> +        {
>> +            unsigned int range_size, nr_ranges;
>> +            int na, ns, pna;
>> +
>> +            pna = dt_n_addr_cells(np);
>> +            na = dt_child_n_addr_cells(np);
>> +            ns = dt_child_n_size_cells(np);
>> +            range_size = pna + na + ns;
>> +            nr_ranges = len / sizeof(__be32) / range_size;
>> +
>> +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
>> +            {
>> +                /* Skip the child address and get the parent (CPU) address */
>> +                addr = dt_read_number(ranges + na, pna);
>> +                size = dt_read_number(ranges + na + pna, ns);
>> +
>> +                start = addr & PAGE_MASK;
>> +                end = PAGE_ALIGN(addr + size);
>> +                res = rangeset_remove_range(mem_holes, start, end - 1);
>> +                if ( res )
>> +                {
>> +                    printk(XENLOG_ERR "Failed to remove:
>> %#"PRIx64"->%#"PRIx64"\n",
>> +                           start, end);
>> +                    goto out;
>> +                }
>> +            }
>> +        }
>>       }

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:21:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:21:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193618.344882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLrT-0006pE-6P; Thu, 23 Sep 2021 10:21:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193618.344882; Thu, 23 Sep 2021 10:21:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTLrT-0006p7-2k; Thu, 23 Sep 2021 10:21:51 +0000
Received: by outflank-mailman (input) for mailman id 193618;
 Thu, 23 Sep 2021 10:21:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTLrR-0006p1-TU
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:21:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0f2f9240-1c58-11ec-ba25-12813bfff9fa;
 Thu, 23 Sep 2021 10:21:48 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-cbpoAIkrNb-bo3_13yAlxA-1; Thu, 23 Sep 2021 12:21:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6671.eurprd04.prod.outlook.com (2603:10a6:803:11f::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 10:21:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 10:21:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0167.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 10:21:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f2f9240-1c58-11ec-ba25-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632392507;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JrgImIEQSTJMYdhj/jza9NFuxXx8r4A++zE1f1dQ9+c=;
	b=ns2jo6l+r38FGF15zHYATHRVxm7x99pjC3hle4XlgZyJDqlCoxHNcd1k8tOWWVW0RIW0lJ
	muj0s8S4acoSCTotBgCom/xz4/Wb3h4L6Vy+0cJ1ZDkphOzlbiBvCbDwBPy1WRcSmLb9XM
	AtxOHlnlRkGaH+fTfV5RcJNOTi9yvm4=
X-MC-Unique: cbpoAIkrNb-bo3_13yAlxA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n1YFijOdOmhgTceZlvK6Ihwe58MXzfQQPCf1CMfuBqJUotrJ1q/4tASsFqAQ5q2h9Rf6ljflzc0JUYvOfB5Zm1mfK3UjtOv++vM2o293N7sx4wpT3bHK7yPeGC3AODpuE6uXio1DMzyVuwcXXv2p2wE/yHm+uPHD9vHRSwScKMmWYjrYmnVbQa3MnhT6uYtOnU51p/xreyoYX6B7a4EVghOxtm/Dv+GGBRplKI87AysXFQW4+Rv6VAV2pPq3mz8WpDWQ1nH2M1+qpdiOiKMGg6HUMAry5MbVHNnW+ZY5QHkbd0CHDzu3yI4i3gS5m96C8UFsDZl1oDNhXIGMJ8rUpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NiOTVezOCSVmFKxp430lQNuv8bpFt1RKty7SyjiGnVo=;
 b=ZL20qdFnntRplvNgjwW40dDMgaRJQds5jUrMmYRcVfYIk9GQTFoXanSbEmExcHOnwcLpqQeDqgEhNZGaqj9U7QuZfUALfw+cRVfcFbvSQy/Ze/VXWeuyNUFNdTQ1qPmQyLqZN4agtCbcQmV0irQNaNJ9GYbEs+Yxf8taCMMPeDI3lo9VTvRqEuptUQmrNKVmvfchtJjepMar/Zrimifg4G+Qhfay77d2aRUO7JqmsPU7INtpU+w1p5L9b2b1S//d7/WG8sa89QOcfvmFSKSJsSywWmQF7D/MAhneWXTcDDMMTjCaonPApuuPY6GsJs1EUOlQwrKK/bOpmzT+Akm/Gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 5/9] x86/PVH: actually show Dom0's register state from
 debug key '0'
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <4ae1a7ef-d6e8-75db-ddf5-987f10175196@suse.com>
 <YUtQPUHjZ8GnfeCE@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d58fdb48-0452-321a-5ded-d949740682c5@suse.com>
Date: Thu, 23 Sep 2021 12:21:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUtQPUHjZ8GnfeCE@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0167.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f3140388-e855-4087-5cd7-08d97e7bf0a2
X-MS-TrafficTypeDiagnostic: VE1PR04MB6671:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66712D1FD667D57034C712DAB3A39@VE1PR04MB6671.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eL8oJ3A+BVnt1M8vXs1UIhOIpLKMvt8v1QWfYo5M9GodvQA54tdY+VYoumERLFPaaVfVtqe04WtX6E3J2lh6oqtc6lxXCETLoxQt7LIANw22TFzbxjQ+a9uPaY38pfi3T+TFmKr8t2FfIv6ODTx1qC8OyEope6mBV7+/A0MvPF4Q1LnR1atip/rXfvP02rVPFwHjzV8o6Mi1lXCGH9kvbIiWO539jX1j19EpFXrQz9pu9X9HrJFH+ZTBLLZITbpTdqumLlFaY0J1v08DelrYUSDs9oN5GpQ1vImNm8IxowvjYd7oKGNyoZ2sxNZxRQQfaw7040D0XyMulNYoHdaGFaM1+ueAqGnwN+Ps++sPkv4ISiA8JR8F8uTLi9GJiaZ/65JazO/XxjogYDOxdeQOLNFWX0aMipxPVs7FhtHqqH3t6n3P8T7VNTfvVj4kR7tg4xnnd9c5fk0oGLpsehQorIBiirbudGJADRDKwewRkYfo+zPy/vpKQrQpm747KSXpj3xfTh/mTH3AIC8Rv+0QIN+AR9nIlp9NpCL8TjUPMyHB6lf5E/XwIy9eq70uix37G5a+/mwlJHPkeNudtjoS7rujrDzwQFbmu8lxL4+ya7rnA4wO7ND0eN66naydAIELivJgzWSGdsXCSui7VucmaZLKEAUIHplI+S2HsZM18Kv6UoKBtmwto1zVagea2xCFkGbVsSk+pQABnaeasjvFy2iYCCDzy0igwOfpPSEImdg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(16576012)(53546011)(8936002)(8676002)(36756003)(66946007)(6916009)(2906002)(4326008)(54906003)(956004)(2616005)(86362001)(83380400001)(66556008)(316002)(26005)(31686004)(5660300002)(508600001)(186003)(31696002)(66476007)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?P6R4TG81A0bv5CfmXqoCW7DbFDtHuvwhH/dN2of7P1iRR2H1zEuapfR3sgu2?=
 =?us-ascii?Q?bNxGE2+nPMnxawotlxt3fyKxVTWFO5XbNG3nA+reXJVkLwXkw02OsQcrx1BQ?=
 =?us-ascii?Q?2nKoqEa79OAM0Hsyhz/TikWpdZ4BlKPFw0mJCllKsMGKJ2OYzkZooycYUiwX?=
 =?us-ascii?Q?BRGk8o3sBsrMpTlPqr3qPr1Jf6w5cEfLlM6AU1tkUDZ/O+KNITcMnw5w1nXW?=
 =?us-ascii?Q?VLbuu4eNlmdeEZANv8BDxtksYqRfZoz18uh2WgxaRYvCAVIPq8kRP1d3fR9p?=
 =?us-ascii?Q?0occV1+0Fa0z33uuCsKXy6O7fuWCfy++krx96yhhXxx5vDVFmVy/blw05Lhg?=
 =?us-ascii?Q?zc64hNksyw3bwgFszTTBFvXg5O8zWEvOYvyOc3SSyQLqohKLyFggVaHr3NhI?=
 =?us-ascii?Q?TCCp5een6r4rOsEU8/xiT8mV5LyfFkcl9DCFDn7w6eXPaUDJfw4RD8y0xcHM?=
 =?us-ascii?Q?z0mY3vBqhCivtzYs/m0VsQTkZqqh2AShFmROGu8wKk/ZWwDWoaunmJ0FczC6?=
 =?us-ascii?Q?mqQd5w0hVV8nmbRpv0Ts3InDrwOKxA8O2RdYwa+B0TzZKWLerr/UDSpXfsSA?=
 =?us-ascii?Q?nsuDhaOcyWfvFl692bKUd3mvmRz3Vi6dCCsxaI/9UCRVuIcyECtyTumjwBMD?=
 =?us-ascii?Q?8PkMBCfsH96whjCjbKRVoMSHdyc/nRfGTDNCIlPD6xo3/MEHc8ii/n/QE0pq?=
 =?us-ascii?Q?D9Tp94TV9SovRjS64glQI76jPt5P767kWuZXRbmER3JJb3dZFyy+lHV3fHSG?=
 =?us-ascii?Q?plYjZqpfnTHQuJAWGBCS2sm+NFf9CkCI50DqXswMdea5Vkke3kV5kspEHxcy?=
 =?us-ascii?Q?BqHlpBdG+gc7g4FqXD8SlQNrRRKn9kUgQ8pNJDTB+YgeuvouaojRsc0WQA3W?=
 =?us-ascii?Q?cux+L9i+w2vmtoZQ2L4go4e20l+QHuWFeNvUfk0NEmBbHJH15nUkxNbNiaVa?=
 =?us-ascii?Q?7MPyENlTGt5eo5IfzUoBv3eKosGzOgH03nmUJn3EIt+ithsCpOdZ92RfVWxr?=
 =?us-ascii?Q?FUhtWcD2otGcAgOvlDtRbL+77pzYxOICkaXuy2JieaaIFXX3cmUW89XthtkJ?=
 =?us-ascii?Q?tSn6Q2pLpYWhJh+3yz8Fz5OhNqXtvqN5Gsi7x5kVz9ROv3s+19CCCcrIV4G+?=
 =?us-ascii?Q?uw9OZIEO7CY5gh8vUIT9C/hEpWHSMkDKO4KVqL0PgzYssHnW3xApECaaIjRP?=
 =?us-ascii?Q?GLJeCoteVitULYA3qqq/I5F0VbhS0eFtJx9Ivqeb07aCGUBF/i20In7iZVhb?=
 =?us-ascii?Q?tacRDbSOhLOTuBndOkOEixvnXC6lBLKEhea/c7xP1aAhKyUD2VIhQpSIM+ar?=
 =?us-ascii?Q?AQS/aEtW/dKFCuexvTKEdPDF?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f3140388-e855-4087-5cd7-08d97e7bf0a2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:21:43.4004
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: laA7kdw5RTk+5xQ3cfdp1hcJFjBmQLE3IhxQw8CH21+VHr8K72SpyZQcbRKLnPPSeul5YROW/tJkHBxJhUC0lA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6671

On 22.09.2021 17:48, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:19:06AM +0200, Jan Beulich wrote:
>> vcpu_show_registers() didn't do anything for HVM so far. Note though
>> that some extra hackery is needed for VMX - see the code comment.
>>
>> Note further that the show_guest_stack() invocation is left alone here:
>> While strictly speaking guest_kernel_mode() should be predicated by a
>> PV / !HVM check, show_guest_stack() itself will bail immediately for
>> HVM.
>>
>> While there and despite not being PVH-specific, take the opportunity and
>> filter offline vCPU-s: There's not really any register state associated
>> with them, so avoid spamming the log with useless information while
>> still leaving an indication of the fact.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I was pondering whether to also have the VMCS/VMCB dumped for every
>> vCPU, to present full state. The downside is that for larger systems
>> this would be a lot of output.
>=20
> At least for Intel there's already a debug key to dump VMCS, so I'm
> unsure it's worth dumping it here also, as a user can get the
> information elsewhere (that's what I've always used to debug PVH
> TBH).

I know there is a respective debug key. That dumps _all_ VMCSes, though,
so might be quite verbose on a big system (where Dom0's output alone
may already be quite verbose).

>> --- a/xen/arch/x86/x86_64/traps.c
>> +++ b/xen/arch/x86/x86_64/traps.c
>> @@ -49,6 +49,39 @@ static void read_registers(struct cpu_us
>>      crs[7] =3D read_gs_shadow();
>>  }
>> =20
>> +static void get_hvm_registers(struct vcpu *v, struct cpu_user_regs *reg=
s,
>> +                              unsigned long crs[8])
>=20
> Would this better be placed in hvm.c now that it's a HVM only
> function?

I was asking myself this question, but decided that the placement here
is perhaps at least no bigger of a problem than putting it there.
Factors played into this:
- the specifics of the usage of the crs[8] array,
- the fact that the PV function also lives here, not under pv/,
- the desire to keep the function static.

I can certainly be talked into moving the code, but I will want to see
convincing arguments that none of the three items above (and possible
other ones I may have missed) are really a problem then.

>> @@ -159,24 +173,35 @@ void show_registers(const struct cpu_use
>>  void vcpu_show_registers(const struct vcpu *v)
>>  {
>>      const struct cpu_user_regs *regs =3D &v->arch.user_regs;

Please note this in addition to my response below.

>> -    bool kernel =3D guest_kernel_mode(v, regs);
>> +    struct cpu_user_regs aux_regs;
>> +    enum context context;
>>      unsigned long crs[8];
>> =20
>> -    /* Only handle PV guests for now */
>> -    if ( !is_pv_vcpu(v) )
>> -        return;
>> -
>> -    crs[0] =3D v->arch.pv.ctrlreg[0];
>> -    crs[2] =3D arch_get_cr2(v);
>> -    crs[3] =3D pagetable_get_paddr(kernel ?
>> -                                 v->arch.guest_table :
>> -                                 v->arch.guest_table_user);
>> -    crs[4] =3D v->arch.pv.ctrlreg[4];
>> -    crs[5] =3D v->arch.pv.fs_base;
>> -    crs[6 + !kernel] =3D v->arch.pv.gs_base_kernel;
>> -    crs[7 - !kernel] =3D v->arch.pv.gs_base_user;
>> +    if ( is_hvm_vcpu(v) )
>> +    {
>> +        aux_regs =3D *regs;
>> +        get_hvm_registers(v->domain->vcpu[v->vcpu_id], &aux_regs, crs);
>=20
> I wonder if you could load the values directly into v->arch.user_regs,
> but maybe that would taint some other info already there. I certainly
> haven't looked closely.

I had it that other way first, wondering whether altering the structure
there might be safe. It felt wrong to fiddle with the live registers,
and the "const" above than was the final bit that convinced me I should
go the chosen route. Yet again - I can be talked into going the route
you outline via convincing arguments. Don't forget that we e.g.
deliberately poison the selector values in debug builds (see
hvm_invalidate_regs_fields()) - that poisoning would get undermined if
we wrote directly into the structure.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:31:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:31:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193625.344894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM14-0000Lh-9d; Thu, 23 Sep 2021 10:31:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193625.344894; Thu, 23 Sep 2021 10:31:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM14-0000La-5p; Thu, 23 Sep 2021 10:31:46 +0000
Received: by outflank-mailman (input) for mailman id 193625;
 Thu, 23 Sep 2021 10:31:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTM13-0000LU-1F
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:31:45 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7100ac42-1c59-11ec-ba28-12813bfff9fa;
 Thu, 23 Sep 2021 10:31:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7100ac42-1c59-11ec-ba28-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632393102;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=w4wXSGWdH9fRfy2+tUQaMA8SXmauZ2tsLoU+bZJrnBw=;
  b=Bv0hLcnM+5hGX6VqIg2tm2YNGkBbj08sINYZwvrqzv2uODNJSCvUgL78
   uNuMlbgRbkBamQhHRNUxSMA59VuUFoFqqzjL/MzAHOeaPHJ8PhQkCj7nF
   ltTJkzn6Oe3lLdyucqiS9DCg1nnq0GLULYpfk2jxDajywxINz2SGSUnJD
   o=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 4wDtFBSorgUvQZhrBievh8Cv31VdzqJq0xlYbB4Z81U7Oeh686ocEpJIL6c8oKExRfemfdFJKV
 R06++WMwn8EskYfBUdpEdrXe+qaF0sdWVc5bZGaVrxQ0w6zOADwj582Zvpyq9SV+gwJ5LNy9NT
 22bK4a6wfpubjAm/D3gbBAXgLG0ldF2lxlIw/n44h1cwmRUgqe2JKyrN30EUS9xQ8clkw0FGQ9
 iMKQhLYfYg7fpz/x/eeEgcscI4JWkpIGyNoPQIzM6Kj46aI5jtbKfSukRYraL3XSp92KsRP/uz
 qifeMh4PD9cgCeb28xDz0jfr
X-SBRS: 5.1
X-MesageID: 53442998
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kz9+N6v9Og86gtVw1e4a3Bd5eufnVLdZMUV32f8akzHdYApBsoF/q
 tZmKWnQP6zZNjGhct91at7j9k4O6MXdm4BgHFFqrCBgRXkT+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524PhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npll727dj8gHY3wxMcwDSJkDzleAYpsweqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sSQayPN
 ppxhTxHYgrDbiRUG0orUq19ruethHDPKxpApwfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krW8mK8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECqF+Zy3EXCRYWfUCmuvT/gUm7M++zM
 GRNpHBo9/JrshX2EJ+tBHVUvUJooDY+etFvK+QZyD3R6YeK2DqFHkg4aRR4PYlOWNANeRQm0
 VqAntXMDDNpsaGIRX/1yop4vQ9eKgBOcjRfP3FsoR8tpoC5+dBu0kunosNLTfbt5uAZDw0c1
 NxjQMIWvLwVkcdD/KGy51mvb9mE98WRE1ZdCuk6WAuYAuJFiGyNO9fABbvzt68owGOlor+p5
 iNsdy+2trxmMH11vHbRKNjh5o2B6fefKyH7ilVyBZQn/DnF0yf9JtoLu2kmfxkzY5dsldrVj
 Kn741g5CHh7ZifCUEOKS9jpV5RCIVbIT7wJqcw4nvIRO8MsJWdrDQllZFKK3nCFraTfuftXB
 HtvSu71VSxyIf0+lFKeHr5BuZd2lnFW7T6CHvjTkkX4uYdykVbIEN8tKkWVVOkl4cus+UONm
 zqpH5DRkEs3vSyXSnS/zLP/2nhQdiVkWcCq9ZYHHgNBSyI/cFwc5zbq6epJU6RunrhPl/eO+
 Xe4W0RCz0H4i2GBIgKPAk2PopuwNXqmhX5kbyEqI3iy3H0vPdSm4KsFLsNldrg77u1zi/VzS
 qBdKcmHB/1OTBXB+igcMsah/NAzKkzziFLcJTehbRg+Y4VkG17D9Oj7c1a97yIJFCe265cz+
 uXyygPBTJMfbA1+F8KKOum3xla8sCFFyuJ/VkfFOPdJf0Do/NQ4IiD9lKZvccoNNQ/C1n2R0
 APPWUUUouzEookU9tjVhP/b89f1QrUmRkcDRjvV97e7MyXe71GP+44YXbbaZy3ZWUP15L6mO
 bdfwcbjPaBVh11NqYd9TepmlPps+9v1qrZG5Q14B3GXPU+zA7ZtL3Taj8lCsqpBmu1QtQesA
 x/d/9BbPfOCOd//EU5XLw0gN7zR2fYRkzjUzPI0PESlu3MnoOvZCR1fb0uWlShQDLppK4d0k
 +4utfkf5xG7lhd3YM2NiTpZ9jjUI3ENO0n9Wkr23GM/ZtIX92x/
IronPort-HdrOrdr: A9a23:iWzKOazdQI8gAe7Z2VCvKrPxseskLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjnfZr5z+8J3WBxB8bZYOCCggqVxe5ZnO7fKlHbaknDH6tmpN
 tdmstFeazN5DpB/L7HCWCDer5KqrT3k9HL9JfjJjVWPHpXgslbnnlE422gYzRLrWd9dP0E/M
 323Ls5m9PsQwVbUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZszU
 H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDi1H8a1P
 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXoyUfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16
 pNwiawq4dXJQmoplW82/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp
 guMCjl3ocVTbqmVQGdgoE2q+bcGkjbXy32DHTqg/blkAS/xxtCvgwlLM92pAZIyHtycegD2w
 xoWp4Y4I2mdfVmH56VMt1xN/dfOla9Mi4kD1jiVGgPNJt3cE4l+KSHqonc2omRCes1Jd0J6c
 38bG8=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53442998"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ed6+vWV+hTVZv12jarhkajVQTlsGpC5ziOcFCYLFj/L7+EZkp1qx2LC3h3u812qjkZDrAHHNUf2W47b70gLtOoFmOA4PeewFDLF8sIWO5q+VSNZyI6wSpS7A3rGfd5fswc1ibONlRi9BlAXPMQhqAKfJIOgu/wML6nbs+r0sGPQTrSaU1a2r9e8k1L9AOAz4qev0rgJYko5l409wtMl/WGCOEWyHKHcClya5V7FScXe7xWJqtMUy7zBHKUQm6mlMBogLYRLJqoBiBnUDAPz5TCzR296bMVkMG+S7Mq9+4MRN7qPOgm4jQLkpPxXS1dP9w52dzexawnKpCYc9fDqRvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7GQI1cTIkFaS2g0GrtYec+AGREcjYeMRkGmLhZEcrjc=;
 b=E+NsjRiwNzKZmGCcYC0syerilK4+Heo8Jvr0NwEEGRcgUksbsq/ilv9GfQw0yi6/+Y+tInKNPN1u0m6rqLloJDD3SJ4TQnDs3a2okH1HLzvaXTf6WwTuyM7muu5ouViwXzI+cW4YcrB1kkyM42ecZYLWuQawGuJBi86+Xc+29okmwWs0uLQlbC8/C+JcIoi+mXEN3fGwSq1fvghRYukTI/Gn9NXr1pEapTlC2kAJ9H+D1+TFixIXNbQ65gx1SfkxcUTN57VPt6wlg4KweLWeEMGbBmALuKLu7DjSoVLySBcOipXssTZuDHjyr6v0jHi31ELVVBnHC886p3q6LPCWVA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7GQI1cTIkFaS2g0GrtYec+AGREcjYeMRkGmLhZEcrjc=;
 b=mFka8ZJH8HaNXXRpuOLXAweUPUuUVqEA1k0ageoIrQEKqNmI/y5yJiGq5p2W7BbnjShTfV/1uN39zOAHUiXXjNvNuAb2tUCFAGII5KD+reWMiFTZWoTtiZ+cCHchtaWtchwzzjVAaHP4eGdh5ZesadkWBDIMnVo+pCVpq64RVv4=
Date: Thu, 23 Sep 2021 12:31:14 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 7/9] x86/PVH: actually show Dom0's stacks from debug
 key '0'
Message-ID: <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
X-ClientProxiedBy: LO4P123CA0452.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1aa::7) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4de520e1-f6ef-44e0-ec01-08d97e7d48de
X-MS-TrafficTypeDiagnostic: DM4PR03MB6094:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM4PR03MB609458C06AE26CF981BE9E478FA39@DM4PR03MB6094.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: /fyPj4fpYbI9W7wMW32pF2InD1oLpsm0lY4I2HDs+Oq13Dg62ISfvokA1UgyVbW5Rea/WcHY2hAL5HRJ0Xfam7icg9D6TOVxsgoi3VIM9vhCQ9kX+KgYyLAFKEvK5OwUrbfm2pyNYBqej2jtDQD5DnZaH/qU9PxILvHFHifRfFIomgp2JQbIZaHK0UWqS80UFhkmE3lbx0Kn7fYNLQScec0KBvzSiL2ij4RR+md+RS4+51ikcsJrswGv4kBf2v1kqQrAuDfYvF5z/XP91pMf7gUAPXr4Q+MTH3lcL+462tKF2UEeopRBJ0ubDRtSxxdEKg5RwbzgNoySZQDruVE9V1bOUqUf2sdIl8lfV+YNxaTiNnwDlMF6DXeuYPG+y/ay/Awd8U4b/6Q2Ty/B14s4CINXnmUcdwvrut+VZiCR6AGlTEg3fycimLfTeV+wet0hWjQMMK6o8BVok8p2TmbITqTU2iiAEwkrBpeTsPnPmMAW98lCbBLxraoWflfBeuM7uLxnCjytSQlyNZJzJJHLe7/03GkhKBJrC0IiOucLRELbRgRe40yf492/NngzijBygGAMBGHEbSGZcQfTnTF9kOvmhdjLaMcyln7UIYtW8j7hu/zGL3TEAOt6fx12R+weAb2YT51gPFW190mVzPUG/g==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(38100700002)(86362001)(186003)(2906002)(66556008)(6916009)(5660300002)(66476007)(66946007)(26005)(956004)(6666004)(4326008)(85182001)(8936002)(54906003)(316002)(9686003)(8676002)(6496006)(83380400001)(508600001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S3ZvQXVWWkFjTTRGZy94V2laVnNKTEJpS0pjVkNtZlRXZlBGUmd5S2xCV0FD?=
 =?utf-8?B?Wit2eTVJVzF6a05vTjRpVWpkMllXMTFSenc4NVRVeC9NczVPc1dRUENYdWhI?=
 =?utf-8?B?RTJSZ2tjczNad2JwVW9nUTBaZ2V6VjBaa0thdktPcW0yaDh2eG5RR0I5ZzNT?=
 =?utf-8?B?cTl2YUJ1RkdRVzNWWEtQM0cwZ0NvTGpqSWt5VEZGUCtVeG1xOUNBS0NSbFcy?=
 =?utf-8?B?ZUF5aG1wM2RmV2l6S1hXYU42TDlDWUJvaUtQcStKZnc5VW5xSTJKUVU5V0tQ?=
 =?utf-8?B?ZEl0T3dnZXJkTE9WUWFIOUl5R1BkWm91cDdqcm5KQmtGU3ovQm1lMGhwd2pN?=
 =?utf-8?B?TUQyaUxpd1Rwd0NtSlhVRjNxTUl2eWJ2MGZzMmJNN201NnBXRTZFQ2krZ08y?=
 =?utf-8?B?ZC8zWjlJZFJESXh2a0QzNS9wcVoySXlLdlhLRk15NDlPN0hJUXE5ZGtsRWVJ?=
 =?utf-8?B?SGdIY2g0TGpDL013a1NRQ3QxZHJXNnpDcTdNZzJFQW41aCtjWjhyK0E1YzA0?=
 =?utf-8?B?TldoQmFwejFjUXZIMi9wWTA2UDh2VWpZY0lDczJMNThHUDZ6b3BPL1dmWEQw?=
 =?utf-8?B?c253dTU0NjBZSmVOTy9nNnZsVW5Da1M5WVRiU0llUGxGc0xIK0RtSW1RQitk?=
 =?utf-8?B?Y1dkUUdCQW5ON0IwZ2w3cWZOZ3I3eWthcTJtb3poem5LeWtNRmhlbzZOQnZy?=
 =?utf-8?B?bEJLdGNyZ2gwYU9vTXlmeUtWcTJOd3dJZjZSOC9HOEphSFJPNS96SXRDaktV?=
 =?utf-8?B?WGtOakZ2djdlWlFqdGRKVE9zc01tSGE5ZVUzN1NLbDkwUUZIb2JqbEpSc1Zu?=
 =?utf-8?B?dko2MXFKcWhJRVlkMGYvOEJrUTRtOHRPVGlPdk9ldTVPSVVMR1ZpbWQrd0NW?=
 =?utf-8?B?MjEyUm1zNGJFSXNLY012NXVhMENzd3hGUDVJQlA2U0xNQnhiOFUzMVB3bTVW?=
 =?utf-8?B?MnRIcHZrcG8vNkhkSmsrYXNhak4zS3BoakUrbWNOZURhLzJVUXB2OUhLWVBC?=
 =?utf-8?B?dTdGZHBkL2Q1N1ZSTEtxbjQ1SFpTV3M4dmV6YWpBRnFxaFhRdnNMQmozTmlM?=
 =?utf-8?B?cEI3Mk4vdFlCSzNnUzhrMEVvLzc4enNvakQzU2Z1VWxoRnVFWkh0cklwY0Fl?=
 =?utf-8?B?bGdvWkE2OFlWZUVlN2RwbUJkSjc5bFBrdS8zdjdlbXFqS2hrSk1ONUZDVHhy?=
 =?utf-8?B?a28rSUpPT1JDcFUyOXA2eENRRU9QL2ZRRmtkZElyYWZQT09KM05nM0RFVG1u?=
 =?utf-8?B?WWYwUVlZRkJ5UGREaGVWUTY5ZkJwbVRNbkFNaUpseDBWdTN6SzJoOUs1Ymow?=
 =?utf-8?B?VVlENUJOeWRMTzl2NTE1alJUcjNxd0NHMVRZcjdkUjBlKzRwV3cwdXJwUWd5?=
 =?utf-8?B?WkxQVmpPRXB3TjVPdWhRaWxQWlVvM204cklCRFJ4NEJpRzU0VzNTcUNhdndG?=
 =?utf-8?B?czdSbS9uNkhkY0ZTWU1KTDR5UmxZbWpWMFJiMTYrems2d2hneEx2d0EyUXIw?=
 =?utf-8?B?M1gwYnd1TXlsakphYUtBdHV4QmNyL0dIclh3S0ZiMFRsMmRKS0FhRXVvQ1Yz?=
 =?utf-8?B?OU9qQzBvQWpjbjF0RCtUWlpHbk5YcVJWeXpzVHNDdFV3bU5uWVRzRy9JRHl3?=
 =?utf-8?B?cmZJZmw3aTU0ZVV0ZEE2eUJOYmU4MExnQ0NuQS9qN2ZhdDN2N0N5RGRKbW50?=
 =?utf-8?B?byt2WVBtckI1UXpWR0ZxMWFIdnYxN0tBWjZOQm5pM00wdnVNREdDY0I3S3g4?=
 =?utf-8?Q?Jbkv+bXLYOEUrULIVu/0SP3Ht4pWVTFAme2b0qp?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4de520e1-f6ef-44e0-ec01-08d97e7d48de
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:31:21.0008
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ceQKcTGqjsw4ECHExZ8eSaRK0pYEC5i+obkHAjLCLtVC8y59cq6j5lWqikQ9kkOSCriB22Ls03d17IJwcOlgqQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6094
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:20:00AM +0200, Jan Beulich wrote:
> show_guest_stack() does nothing for HVM. Introduce a HVM-specific
> dumping function, paralleling the 64- and 32-bit PV ones. We don't know
> the real stack size, so only dump up to the next page boundary.
> 
> Rather than adding a vcpu parameter to hvm_copy_from_guest_linear(),
> introduce hvm_copy_from_vcpu_linear() which - for now at least - in
> return won't need a "pfinfo" parameter.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> TBD: The bypassing of the output interleaving avoidance isn't nice, but
>      I've not been able to think of an alternative. Avoiding the call to
>      hvm_vcpu_virtual_to_linear() would be in principle possible (adding
>      in the SS base directly), but one way or another we need to access
>      guest memory and hence can't sensibly avoid using the P2M layer.
>      However, commit 0996e0f38540 ("x86/traps: prevent interleaving of
>      concurrent cpu state dumps") introduced this logic here while
>      really only talking about show_execution_state().
>      vcpu_show_execution_state() is imo much less prone to interleaving
>      of its output: It's uses from the keyhandler are sequential already
>      anyway, and the only other use is from hvm_triple_fault(). Instead
>      of making the locking conditional, it may therefore be an option to
>      drop it again altogether.
> TBD: For now this dumps also user mode stacks. We may want to restrict
>      this.
> TBD: An alternative to putting this next to {,compat_}show_guest_stack()
>      is to put it in hvm.c, eliminating the need to introduce
>      hvm_copy_from_vcpu_linear(), but then requiring extra parameters to
>      be passed.
> TBD: Technically this makes unnecessary the earlier added entering/
>      leaving if the VMCS. Yet to avoid a series of non-trivial
>      enter/exit pairs, I think leaving that in is still beneficial. In
>      which case here perhaps merely the associate comment may want
>      tweaking.
> ---
> v3: New.
> 
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -3408,6 +3408,15 @@ enum hvm_translation_result hvm_copy_fro
>                        PFEC_page_present | pfec, pfinfo);
>  }
>  
> +enum hvm_translation_result hvm_copy_from_vcpu_linear(
> +    void *buf, unsigned long addr, unsigned int size, struct vcpu *v,
> +    unsigned int pfec)

Even if your current use case doesn't need it, would it be worth
adding a pagefault_info_t parameter?

> +{
> +    return __hvm_copy(buf, addr, size, v,
> +                      HVMCOPY_from_guest | HVMCOPY_linear,
> +                      PFEC_page_present | pfec, NULL);
> +}
> +
>  unsigned int copy_to_user_hvm(void *to, const void *from, unsigned int len)
>  {
>      int rc;
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -364,6 +364,71 @@ static void show_guest_stack(struct vcpu
>      printk("\n");
>  }
>  
> +static void show_hvm_stack(struct vcpu *v, const struct cpu_user_regs *regs)
> +{
> +#ifdef CONFIG_HVM
> +    unsigned long sp = regs->rsp, addr;
> +    unsigned int i, bytes, words_per_line, pfec = PFEC_page_present;
> +    struct segment_register ss, cs;
> +
> +    hvm_get_segment_register(v, x86_seg_ss, &ss);
> +    hvm_get_segment_register(v, x86_seg_cs, &cs);
> +
> +    if ( hvm_long_mode_active(v) && cs.l )
> +        i = 16, bytes = 8;
> +    else
> +    {
> +        sp = ss.db ? (uint32_t)sp : (uint16_t)sp;
> +        i = ss.db ? 8 : 4;
> +        bytes = cs.db ? 4 : 2;
> +    }
> +
> +    if ( bytes == 8 || (ss.db && !ss.base) )
> +        printk("Guest stack trace from sp=%0*lx:", i, sp);
> +    else
> +        printk("Guest stack trace from ss:sp=%04x:%0*lx:", ss.sel, i, sp);
> +
> +    if ( !hvm_vcpu_virtual_to_linear(v, x86_seg_ss, &ss, sp, bytes,
> +                                     hvm_access_read, &cs, &addr) )
> +    {
> +        printk(" Guest-inaccessible memory\n");
> +        return;
> +    }
> +
> +    if ( ss.dpl == 3 )
> +        pfec |= PFEC_user_mode;
> +
> +    words_per_line = stack_words_per_line * (sizeof(void *) / bytes);
> +    for ( i = 0; i < debug_stack_lines * words_per_line; )
> +    {
> +        unsigned long val = 0;
> +
> +        if ( (addr ^ (addr + bytes - 1)) & PAGE_SIZE )
> +            break;
> +
> +        if ( !(i++ % words_per_line) )
> +            printk("\n  ");
> +
> +        if ( hvm_copy_from_vcpu_linear(&val, addr, bytes, v,
> +                                       pfec) != HVMTRANS_okay )

I think I'm confused, but what about guests without paging enabled?
Don't you need to use hvm_copy_from_guest_phys (likely transformed
into hvm_copy_from_vcpu_phys)?

> +        {
> +            printk(" Fault while accessing guest memory.");
> +            break;
> +        }
> +
> +        printk(" %0*lx", 2 * bytes, val);
> +
> +        addr += bytes;
> +        if ( !(addr & (PAGE_SIZE - 1)) )
> +            break;
> +    }
> +
> +    if ( !i )
> +        printk(" Stack empty.");
> +    printk("\n");
> +#endif
> +}
> +
>  /*
>   * Notes for get_{stack,shstk}*_bottom() helpers
>   *
> @@ -629,7 +694,7 @@ void show_execution_state(const struct c
>  
>  void vcpu_show_execution_state(struct vcpu *v)
>  {
> -    unsigned long flags;
> +    unsigned long flags = 0;
>  
>      if ( test_bit(_VPF_down, &v->pause_flags) )
>      {
> @@ -663,14 +728,22 @@ void vcpu_show_execution_state(struct vc
>      }
>  #endif
>  
> -    /* Prevent interleaving of output. */
> -    flags = console_lock_recursive_irqsave();
> +    /*
> +     * Prevent interleaving of output if possible. For HVM we can't do so, as
> +     * the necessary P2M lookups involve locking, which has to occur with IRQs
> +     * enabled.
> +     */
> +    if ( !is_hvm_vcpu(v) )
> +        flags = console_lock_recursive_irqsave();
>  
>      vcpu_show_registers(v);
> -    if ( guest_kernel_mode(v, &v->arch.user_regs) )
> +    if ( is_hvm_vcpu(v) )
> +        show_hvm_stack(v, &v->arch.user_regs);

Would it make sense to unlock in show_hvm_stack, and thus keep the
printing of vcpu_show_registers locked even when in HVM context?

TBH I've never found the guest stack dump to be helpful for debugging
purposes, but maybe others do.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:34:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:34:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193630.344904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM49-0000zZ-Np; Thu, 23 Sep 2021 10:34:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193630.344904; Thu, 23 Sep 2021 10:34:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM49-0000zS-Ki; Thu, 23 Sep 2021 10:34:57 +0000
Received: by outflank-mailman (input) for mailman id 193630;
 Thu, 23 Sep 2021 10:34:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTM48-0000zJ-7K
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:34:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e41cf942-1c59-11ec-ba28-12813bfff9fa;
 Thu, 23 Sep 2021 10:34:55 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-IFSlWXeTMp6wriZKts0jMg-1;
 Thu, 23 Sep 2021 12:34:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2447.eurprd04.prod.outlook.com (2603:10a6:800:53::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 10:34:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 10:34:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0052.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:53::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 10:34:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e41cf942-1c59-11ec-ba28-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632393294;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xMGtfv2GIbmnQ7Y8ZAx6bsOoV5Jz9FESpz+hQv/7GUw=;
	b=IjXU4bkLhf7PRYusuMA90T1lrRze0T4joJ04wXQYmJT3KUYsZ5RrUT3ODmq3kDhQ8ZkHcj
	jrwWhE8MseKDwfubmJ78hskMGt5CuZ2q2/+sG9RNwaUPd+dOHWCidY10Q4c1M2nS7NFSuk
	KAeM+4PPhsOEUeQ5MaKSNUv7poRle/c=
X-MC-Unique: IFSlWXeTMp6wriZKts0jMg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=I0aGng4EfmqVXinLUonr6tbR7QFH3bWMi9NvYWrBc2z2ieBG4mdP1z0ZOMQkXMt/tML1YjhzD6Ok5hZUp/+W0xGxvZmni62WveKVIpZd/sV7wLt+v+wlVu6G5xLtiIQDBIVkqX8JymCUSHlkHFBREmYAFHG49JXnn+fx20JGe/TpditOsf2MiNUN8XzKJe8qAvDCvMW1NLVmp1qM1Y+F9Wcu2ZtqINWJ3VL9l68JcgIeCSHaZ9hKL9a6S56SoVpWo0O9vqxP+5GBXfNo8zxMd/p0D9hau32pf48agU08fbXrLHFOpG1nYItCc0Kwz8BP7vWyT1J7J+4Gk4kWxLM7rw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xsJR4mBTsBcRuYBVyViNw9g/7NDrTvKyGR8IzJp0MYY=;
 b=IfB3iJUNnrog4hT9XiynxOCYzAFiRHBdpkGWZxDvd0bcD3Z/8JuK/ctBKwJs4r09gI2902m5TV9yz8Ww/zSRqf0oG12bFKmOIbjsQgMCpMH8n+su5Q9bVMQ8220U6JgSr/QHzHd49ahjajVCSxIS+prHKCX0Pu62HUgyhK282+i3MqUAbCYGIuvxq5Q/nBOX29oWJ6PCEVHIakp+cj6BcYF63rNMooWzdnseJtQ9q54i3saWEppkRRQNSS/QCgpiR7V+zK9jQeAAujGifs9x2cnV2rTLCZK+xngPP6ehOCZcN17xMYGf3a5AV8HHxG5DvhXcZvIY/hDV5CDaPnM/lg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 6/9] x86/HVM: convert hvm_virtual_to_linear_addr() to
 be remote-capable
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <3397b5ea-bc21-5a2c-ba5d-afc974a92df4@suse.com>
 <YUw2RLtI73QNjaPE@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <46f90ac6-a4f0-d00a-2956-a0e4b351f698@suse.com>
Date: Thu, 23 Sep 2021 12:34:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUw2RLtI73QNjaPE@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P189CA0052.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:53::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ac148389-cae7-4242-e40a-08d97e7dc538
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2447:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24478BB240EC3D23A66C5087B3A39@VI1PR0401MB2447.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HVUUxpmL3bnVEyinhoBrYzjc/zkMzk5aj/1uh6BuQw4gdcdTa6/PZZa6U0/CxLGPh4i/Rwy4qjonU6KDUm3oPBY/m+4ubaXlmxIQWx1cKjBUIRlj9vtE3z+FjZyjknZghMC3CXxbOkIdKVjOjR7VXYMbxmfFdtqQ43sA5dfDISxrnIeCIRaG3PkSTdkCl1eSRXr15GqW153j4Cq3i+LwtpdeBEOIyx0dgOVxc6ZSdOK/+Ij5/TIJcJh3Zr7D1Oo6bwsGc0861gfT/bd/ALmi1g2CeewPaQcJeUQkm0MOaoYVHjv2uGGA0LOOJeZAGh3/oG42F163PMSnfljS2ltk8ruoRAjo7n83bAc/ndLFq1Togo9AqlJwKFwXSpfGIajrRRG2P5I3L+FM32bIe8gpUXx8B9ij50x/9jUzyCBsQwdlaxgE2dJcMfQqSOJVT64o6sjgxiwoH4S0MEtMwnFXfdvsZT3ZIP8B4lbpBIOJHW6tMvI6ft7NZKg1i1IRK0/HxD+QntbPzwRenQ93lxvHDT011RlX6/OItjBRBCq0shz+YKkG6I/opm1CvgN9aHZ8OuspYaY3McEj4XkB1OX/XpjDkE0Kxb5C7GPNQM0zOa3GCUaF40g9S7ZzRgyRlfoxRBoQZEfjC7AwA/zymR2fnx5zagYtPdINUcHGh17aAKdKX5LbLaOFpWXz1qSl6OW/59UAobqX9l4GyueZnTOSk5/9T6IobUdn5CCTBbodNbQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(16576012)(186003)(31696002)(66476007)(5660300002)(2616005)(66556008)(956004)(2906002)(8936002)(38100700002)(6486002)(66946007)(86362001)(6916009)(53546011)(54906003)(31686004)(8676002)(83380400001)(508600001)(316002)(4326008)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?iE2J/buITkiD/XBKV+CHVdd36F2mNAoVP0Csit18/FtVt/zXlu2hmiPoz6co?=
 =?us-ascii?Q?UM56P4jEcnUGWmmYkwLATflICBP4FLs3H7e7dKlAfCYNv4eUd5tiFIstJd1z?=
 =?us-ascii?Q?XZydoZ+Y0o8sUs/YSTXIb5AyY4DbUUsIu6d94DWtCJI/pW5mPhKsjfqCswy6?=
 =?us-ascii?Q?DHCa2nyeIXCs6vkIsjAvyRfrdtOFW6OZfzLWk4jULFbT0fCPaRsHz0ShZpIU?=
 =?us-ascii?Q?pd/VqsWF/SJk4I8HNo1jRMuuAaYqy8dSKAn8pHNWWYZfEuWQHKrO1rrVGJkK?=
 =?us-ascii?Q?OWl8A2JjGKN+ZgGhuvT+VWrzIaC3KXesDd18XQ7YBRFT4PsYDNKGqV9CGWqq?=
 =?us-ascii?Q?1yJDff2EgiGl7YpKmX5KhnN+0olR4kOJxAGjoLqH/L6ZFu8HSAwIDWxeRuLZ?=
 =?us-ascii?Q?p9QWvUKv6CP2ba28tgz5kiaSPsELyjrldSiFcjm+kVqvd2IsTZK//YoQTMvh?=
 =?us-ascii?Q?oucUfN+rQK+c+KGZIhKQnlffNYLawJRl2l2rFRZcfguXoLZH8B4N7BwQo1Fb?=
 =?us-ascii?Q?CvRt0tNRiRMK/N7eO6A+4XhDsEoqtjNYQ45VpSEJT1WZO7uAEoiwokDXVv3u?=
 =?us-ascii?Q?xA3HREksnmaFNaiyi43iVhwt0hSkaALpRQWwiPKz1f6VQcwAjbU2+lOr367q?=
 =?us-ascii?Q?Ho2wcO+LrLu4r0s3G0rG5VEBjc610Wp2BS9RcHpezHqwzJRyXpx4NTKgCBz2?=
 =?us-ascii?Q?h6t4hy64GTlelWhk51dLBGW/NYSxUz29RXtK7M7Cud2RY5B9xDekANPfxKVG?=
 =?us-ascii?Q?gAHEjaeKZFXsEjtzMp7ZyU+cqRBSMDywwiGU+a6UpSNPDH422h12jSPVb51W?=
 =?us-ascii?Q?TaJlUQblb/L93Lx63fCYJm7szjt2TYj0UZOJ6zv6ta0M7naGIHj8iUmBx31D?=
 =?us-ascii?Q?C9kQ7rxNMYtKmC0WD7S2XseWVYNdUVoIq5f21KyZaRRCWgmwjw3bLTLWwtsl?=
 =?us-ascii?Q?3OvqBz+7T9VlRAOeSub240jEQUuK3kuys+/pSnjDq6pwa19pEUCTAMwUmXrz?=
 =?us-ascii?Q?bAvUL0QX27QOeZN5u8myl31fpgdyFPkAcEvro0WLwR/nnsQVC0gzHIznbu6X?=
 =?us-ascii?Q?E03Zd0Sk33I9x+UAguY03Xg8oUFA3y0fyTV+FhkiHVBAbtyn7BExhSnTAesS?=
 =?us-ascii?Q?LcLJfIYnarWnuxYAIq8iEpPJdPHO1gfRJhJnXaGJnf6mQ+1nrFev92iwmTND?=
 =?us-ascii?Q?TH4MfWm4N/QWOaY8+LkUUZ+IvmmsbVy7vuuzQBeBJAvDt0W6zYoRn76ZYY3C?=
 =?us-ascii?Q?yokt+z/oCPBNqHDsTWTSV5leYNJOXsC3PwMb4oYbySZWFvcFfx89xcvftGk1?=
 =?us-ascii?Q?e7lWbJyj+zrgnFTTW5MGS2cV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ac148389-cae7-4242-e40a-08d97e7dc538
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:34:49.6266
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OsxqyAxVL1ybwjSV29ShT8VN/3HtPSvjrH3cvgjSaSG4jLM0+ERgVgKGb2F/5fIwJz0jTtMlSuF2xiNkoc62RQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2447

On 23.09.2021 10:09, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:19:37AM +0200, Jan Beulich wrote:
>> --- a/xen/arch/x86/hvm/hvm.c
>> +++ b/xen/arch/x86/hvm/hvm.c
>> @@ -2526,7 +2526,8 @@ int hvm_set_cr4(unsigned long value, boo
>>      return X86EMUL_OKAY;
>>  }
>> =20
>> -bool_t hvm_virtual_to_linear_addr(
>> +bool hvm_vcpu_virtual_to_linear(
>> +    struct vcpu *v,
>>      enum x86_segment seg,
>>      const struct segment_register *reg,
>>      unsigned long offset,
>> @@ -2535,8 +2536,9 @@ bool_t hvm_virtual_to_linear_addr(
>>      const struct segment_register *active_cs,
>>      unsigned long *linear_addr)
>>  {
>> -    const struct vcpu *curr =3D current;
>>      unsigned long addr =3D offset, last_byte;
>> +    const struct cpu_user_regs *regs =3D v =3D=3D current ? guest_cpu_u=
ser_regs()
>> +                                                    : &v->arch.user_reg=
s;
>>      bool_t okay =3D 0;
>=20
> Since you change the function return type to bool, you should also
> change the type of the returned variable IMO. It's just a two line
> change.

Can do; I would have viewed this as an unrelated change: While the
first change needed indeed is on an adjacent line (above), the other
one isn't.

> Also is it worth adding some check that the remote vCPU is paused? Or
> else you might get inconsistent results by using data that's stale  by
> the time Xen acts on it.

I did ask myself the same question. I did conclude that even if the
remote vCPU is paused at the time of the check, it may not be anymore
immediately after, so the information returned might be stale anyway.
I further looked at {hvm,vmx}_get_segment_register() to see what they
did - nothing. It is only now that I've also checked
svm_get_segment_register(), which - interestingly - has such a check.
I will copy the ASSERT() used there.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:36:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:36:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193635.344916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM5X-0001aR-2A; Thu, 23 Sep 2021 10:36:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193635.344916; Thu, 23 Sep 2021 10:36:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM5W-0001aK-VM; Thu, 23 Sep 2021 10:36:22 +0000
Received: by outflank-mailman (input) for mailman id 193635;
 Thu, 23 Sep 2021 10:36:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTM5W-0001aA-8P
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:36:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTM5V-0005VS-Qk; Thu, 23 Sep 2021 10:36:21 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTM5U-00008b-VY; Thu, 23 Sep 2021 10:36:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=vorgyAnbI5+9T68OIx6ZYiApShBPdVYjqchNEhNY5iU=; b=2spFwcbr5eGbwb4s6fjk6/tGnW
	3OonzNNRsw8X0J19o7VjLqdq926DJF1HuwVqI77JRztCUPr+lOXIz3FVCn4yRJDE+DRQfd0xVvMLa
	OD4AXjwC8kqPsG8xV3nJ+ZE3myOJhCzqce+rwX03EJf+MNuNBif0DWUsvbQAFiMRZrnE=;
Subject: Re: [PATCH 03/11] xen/arm: introduce 1:1 direct-map for domUs
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-4-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <27b7e2c9-168e-68ef-e372-0571b81676fc@xen.org>
Date: Thu, 23 Sep 2021 15:36:02 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-4-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> Cases where domU needs 1:1 direct-map memory map:

"1:1" and "direct-map" means pretty much the same. Given that the 
property is name "direct-map", then I would drop "1:1".

>    * IOMMU not present in the system.
>    * IOMMU disabled if it doesn't cover a specific device and all the guests
> are trusted. Thinking a mixed scenario, where a few devices with IOMMU and
> a few without, then guest DMA security still could not be totally guaranteed.
> So users may want to disable the IOMMU, to at least gain some performance
> improvement from IOMMU disabled.
>    * IOMMU disabled as a workaround when it doesn't have enough bandwidth.
> To be specific, in a few extreme situation, when multiple devices do DMA
> concurrently, these requests may exceed IOMMU's transmission capacity.
>    * IOMMU disabled when it adds too much latency on DMA. For example,
> TLB may be missing in some IOMMU hardware, which may bring latency in DMA
> progress, so users may want to disable it in some realtime scenario.
> 
> *WARNING:
> Users should be aware that it is not always secure to assign a device without
> IOMMU protection.
> When the device is not protected by the IOMMU, the administrator should make
> sure that:
>   1. The device is assigned to a trusted guest.
>   2. Users have additional security mechanism on the platform.

The two requirements are only necessary for device that are DMA-capable. 
For device that can't do DMA, it will likely be fine to assign to 
non-trusted guest.

> 
> Given that with direct-map, the IOMMU could be used but it is not required.

I can't parse it.

> This commit avoids setting XEN_DOMCTL_CDF_iommu when the IOMMU is disabled and
> direct_map is requested.
> 
> Since, for now, 1:1 direct-map is only supported when domain on Static

I think "Since" seems unnecessary.

> Allocation, that is, "xen.static-mem" is defined in the domain configuration.
> 
> This commit also re-implements allocate_static_memory to allocate static memory
> as guest RAM for 1:1 direct-map domain.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   docs/misc/arm/device-tree/booting.txt |   9 ++
>   xen/arch/arm/domain_build.c           | 117 +++++++++++++++++---------
>   2 files changed, 85 insertions(+), 41 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 44cd9e1a9a..3d637c747e 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -164,6 +164,15 @@ with the following properties:
>       Both #address-cells and #size-cells need to be specified because
>       both sub-nodes (described shortly) have reg properties.
>   
> +- direct-map
> +
> +    Optional for Domain on Static Allocation.
> +    An empty property to request the memory of the domain to be 1:1
> +    direct-map(guest physical address == physical address).
> +    WARNING: Users must be aware of this risk, that guests having access
> +    to hardware with DMA capacity must be trusted, or it could use the
> +    DMA engine to access any other memory area.

The WARNING is only applicable if the device is not protected by an 
IOMMU. So this should be clarified because one may want still want to 
use the direct-map (e.g. because the OS relies on the host layout) and 
have IOMMU enabled.

> +
>   Under the "xen,domain" compatible node, one or more sub-nodes are present
>   for the DomU kernel and ramdisk.
>   
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 21d8a559af..213ad017dc 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -488,8 +488,14 @@ static bool __init append_static_memory_to_bank(struct domain *d,
>   {
>       int res;
>       unsigned int nr_pages = PFN_DOWN(size);
> -    /* Infer next GFN. */
> -    gfn_t sgfn = gaddr_to_gfn(bank->start + bank->size);
> +    gfn_t sgfn;
> +
> +    if ( !is_domain_direct_mapped(d) )
> +        /* Infer next GFN when GFN != MFN. */
> +        sgfn = gaddr_to_gfn(bank->start + bank->size);
> +    else
> +        sgfn = gaddr_to_gfn(mfn_to_maddr(smfn));
> +
>   
>       res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages);
>       if ( res )
> @@ -537,14 +543,17 @@ static void __init allocate_static_memory(struct domain *d,
>       }

This function was already pretty difficult to read. So I would rather 
not add more complexity in it. Instead, I would look to split the common 
code in a separate helper or possibly duplicate it.

>       reg_cells = addr_cells + size_cells;
>   
> -    /*
> -     * The static memory will be mapped in the guest at the usual guest memory
> -     * addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> -     * xen/include/public/arch-arm.h.
> -     */
> -    gbank = 0;
> -    gsize = ramsize[gbank];
> -    kinfo->mem.bank[gbank].start = rambase[gbank];
> +    if ( !is_domain_direct_mapped(d) )
> +    {
> +        /*
> +         * The static memory will be mapped in the guest at the usual guest
> +         * memory addresses (GUEST_RAM0_BASE, GUEST_RAM1_BASE) defined by
> +         * xen/include/public/arch-arm.h.
> +         */
> +        gbank = 0;
> +        gsize = ramsize[gbank];
> +        kinfo->mem.bank[gbank].start = rambase[gbank];
> +    } >
>       cell = (const __be32 *)prop->value;
>       nr_banks = (prop->length) / (reg_cells * sizeof (u32));
> @@ -572,42 +581,58 @@ static void __init allocate_static_memory(struct domain *d,
>           printk(XENLOG_INFO "%pd: STATIC BANK[%u] %#"PRIpaddr"-%#"PRIpaddr"\n",
>                  d, bank, pbase, pbase + psize);
>   
> -        while ( 1 )
> +        if ( !is_domain_direct_mapped(d) )
>           {
> -            /* Map as much as possible the static range to the guest bank */
> -            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank], smfn,
> -                                               min(psize, gsize)) )
> -                goto fail;
> -
> -            /*
> -             * The current physical bank is fully mapped.
> -             * Handle the next physical bank.
> -             */
> -            if ( gsize >= psize )
> +            while ( 1 )
>               {
> -                gsize = gsize - psize;
> -                break;
> +                /* Map as much as possible the static range to the guest bank */
> +                if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank],
> +                                                   smfn, min(psize, gsize)) )
> +                    goto fail;
> +
> +                /*
> +                 * The current physical bank is fully mapped.
> +                 * Handle the next physical bank.
> +                 */
> +                if ( gsize >= psize )
> +                {
> +                    gsize = gsize - psize;
> +                    break;
> +                }
> +                /*
> +                 * When current guest bank is not enough to map, exhaust
> +                 * the current one and seek to the next.
> +                 * Before seeking to the next, check if we still have available
> +                 * guest bank.
> +                 */
> +                else if ( (gbank + 1) >= GUEST_RAM_BANKS )
> +                {
> +                    printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
> +                    goto fail;
> +                }
> +                else
> +                {
> +                    psize = psize - gsize;
> +                    smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
> +                    /* Update to the next guest bank. */
> +                    gbank++;
> +                    gsize = ramsize[gbank];
> +                    kinfo->mem.bank[gbank].start = rambase[gbank];
> +                }
>               }
> +        }
> +        else /* 1:1 direct-map. */
> +        {
>               /*
> -             * When current guest bank is not enough to map, exhaust
> -             * the current one and seek to the next.
> -             * Before seeking to the next, check if we still have available
> -             * guest bank.
> +             * One guest memory bank is matched with one physical
> +             * memory bank.
>                */
> -            else if ( (gbank + 1) >= GUEST_RAM_BANKS )
> -            {
> -                printk(XENLOG_ERR "Exhausted all possible guest banks.\n");
> +            gbank = bank;
> +            kinfo->mem.bank[gbank].start = pbase;
> +
> +            if ( !append_static_memory_to_bank(d, &kinfo->mem.bank[gbank],
> +                                               smfn, psize) )
>                   goto fail;
> -            }
> -            else
> -            {
> -                psize = psize - gsize;
> -                smfn = mfn_add(smfn, gsize >> PAGE_SHIFT);
> -                /* Update to the next guest bank. */
> -                gbank++;
> -                gsize = ramsize[gbank];
> -                kinfo->mem.bank[gbank].start = rambase[gbank];
> -            }
>           }
>   
>           tot_size += psize;
> @@ -2638,6 +2663,7 @@ void __init create_domUs(void)
>   {
>       struct dt_device_node *node;
>       const struct dt_device_node *chosen = dt_find_node_by_path("/chosen");
> +    bool direct_map = false;

This is a bit redundant for just a couple of use. Instead, you could 
directly use d_cfg.flags & XEN_DOMCTL_INTERNAL_directmap.

>   
>       BUG_ON(chosen == NULL);
>       dt_for_each_child_node(chosen, node)
> @@ -2658,8 +2684,17 @@ void __init create_domUs(void)
>               panic("Missing property 'cpus' for domain %s\n",
>                     dt_node_name(node));
>   
> +        direct_map = dt_property_read_bool(node, "direct-map");
> +        d_cfg.flags |= direct_map ? XEN_DOMCTL_INTERNAL_directmap : 0;
>           if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") )
> -            d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> +        {
> +            if ( iommu_enabled )
> +                d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> +            else if ( !direct_map )
> +                panic("Assign a device but IOMMU and direct-map are all disabled\n");
> +            else
> +                warning_add("Please be sure of having trusted guests, when doing device assignment without IOMMU protection\n");
> +        }
>   
>           if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
>           {
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:39:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193642.344926 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM87-0002Y9-LQ; Thu, 23 Sep 2021 10:39:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193642.344926; Thu, 23 Sep 2021 10:39:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTM87-0002Y2-IK; Thu, 23 Sep 2021 10:39:03 +0000
Received: by outflank-mailman (input) for mailman id 193642;
 Thu, 23 Sep 2021 10:39:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTM86-0002Xw-4s
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:39:02 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0651cc4b-16d7-4693-af5b-e8dd0d60eced;
 Thu, 23 Sep 2021 10:39:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0651cc4b-16d7-4693-af5b-e8dd0d60eced
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632393540;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=czW0SM3LqxlQ6bHGBfoSgj8BE1sxKKvRihHh40ND2BM=;
  b=NFiWnaoJte8J6T+o9PWr9hFEtomNZoNdD9SI4o8MRssVWqDMk64R5MLA
   BqjGhLaRLjo9ENJZf9O1+YMaUH0EoXwenZewR3CbXnmoHCdSKXOVbA4yQ
   9SuAMrW9iVR75ZqAVscXTTmNOSChHqtsb85WyrHgpVZ7wmYsStf21SsLP
   0=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 5gNW6APMgNzWFtZUud4PvTyln0NUrDmHWUYUHSQXLhH6qpe3LuuqF50jnDNWjLRWIWK0pn9xLr
 e7MxI0KoquJMaYQKRK3JLzpZI9A3D5u8JskeqVHYWAqtGfG7ow/LvCL4QcQPhCqMt6P/5roYYc
 xDflBcz4YuQdkTjIVdt6/sQjBeKzrs9JneL2O/KT01eOYlV2jMizSWShfmRx2lM6aT0ltB0kMl
 B7lTJawG49/q8eUreqFT9v6CRvMACn3JVJ+/RnPtwa7+AGg3N0zBZ8eus5bnriIijnRhhze1iO
 D6ryswT7EpXSfo8uJJ1/VO+z
X-SBRS: 5.1
X-MesageID: 53020167
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fxB6C61zMUV3iZQTgPbD5Q12kn2cJEfYwER7XKvMYLTBsI5bpzIFn
 TAbDGqAbv3fNGKhLo9+bou18U8AucSByoBmQVBtpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbVh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhgvZTi
 /BNiKOKTSwrO5XClesYCUV0DHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQAhm9p2J8m8fD2Q
 M8mV2ooMxD6UyJmEFMTM7Bkm+C5mSyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1Fk0Ddq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PL+y++NugVaT7ncOExBQXly+ycRVkWbnBYgZc
 RZNvHNz8+5iryRHU+URQTWlhlzHmx4hZOBUOO8/0A+Axa+OvyqGUz1soiF6VPQqs8o/RDoP3
 1CPns/0CTEHjIB5WU5x5Z/P8mjvY3Z9wXsqIHZeFFpYv4CLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3mrESnQkSGcICuTrsoCNslgRiG2NPdXABb3nARBodt3xor6p5
 iRspiRmxLpSZaxhbQTUKAn3IJmn5uyeLBrXikN1Ep8q+lyFoiD4J9sLuGkleh04Y67onAMFh
 meJ4mu9A7cJYBOXgVJfOdrtW6zGM4C6fTgaahwkRoUXOcUgHON21CpveVSRzwjQfLsEy8kC1
 WOgWZ/0Vx4yUP0/pBLvHrt1+eJ7l0gWmDKILbimnkvP7FZrTCPMIVvzGADVNb5RAWLtiFi9z
 uuzwOPQkEgADL2iPHOHmWPRRHhTRUUG6VnNg5U/XsaIIxZ8GXFnDPnUwLg7fJdikbgTneDNl
 kxRkGcBoLYmrXGYewiMdF55b7bjAcR2oX4hZHR+Nle0wXkzJ42o6f5HJZcweLAm8s1lzOJ1E
 KZZK5nRXKwXR2SV4SkZYLn8sJdmKEahizWRMnf3ezM4ZZNhGVDEo4e2Ygv1+SASJSOrrs9i8
 aa43wbWTMNbFQRvBcrbcty1yFa1sSRPke5+RRKQcNJSZF/t4M5hLCmo1q07JMQFKBPiwDqG1
 lnJXUdE9LeV+4JsqYvHn6GJqYutAtBSJEsCEjmJ96uyOAnb4nGnnd1KXtGXcG2PT2jz4qijO
 7lYlqmuLP0dkV9WmINgCLI3n7km7t7iqrIGnARpGHLHMwaiBr96eyTU2MBOsutGx6NDuBvwU
 UWKo4EINbKMMcLjMVgQOAt6MbjTiaBKwmHfvaYvPUH3xC5r577WA0xdMi6FhDFZMLYoYpgux
 v0suZJO5gGy4vbw3g1qUsyAG7ywE0E9
IronPort-HdrOrdr: A9a23:Hv9lWaP7idwCE8BcT1D155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/uxoHJPwO080kqQFnLX5XI3SJzUO3VHHEGgM1/qB/9SNIVyaygcZ79
 YdT0EcMqyAMbEZt7eC3ODQKb9Jq7PmgcPY9ds2jU0dNT2CA5sQkTuRYTzrdHGeKjM2YabQQ/
 Gnl7V6TnebCDkqR/X+IkNAc/nIptXNmp6jSRkaByQ/4A3LqT+z8rb1HzWRwx9bClp0sPgf2F
 mAtza8yrSosvm9xBOZ/2jP765OkN+k7tdYHsSDhuUcNz2poAe1Y4ZKXaGEoVkO0a2SwWdvtO
 OJjwYrPsx15X+UVmapoSH10w2l6zoq42+K8y7RvVLT5ejCAB4qActIgoxUNjHD7VA7gd162K
 VXm0qEqpt+F3r77WTAzumNcysvulu/oHIkn+JWpWdYS5EiZLhYqpFa1F9JEa0HADnx5OkcYa
 hT5fnnlbRrmG6hHjXkVjEF+q3pYp1zJGbJfqE6gL3X79AM90oJiHfxx6Qk7z49HdwGOt95D0
 mtCNUdqFh0dL5lUUtKPpZ2fSKGMB2/ffvyChPmHb3GLtBNB5ufke+83F0KjNvaD6DgiqFCwa
 j8bA==
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53020167"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Bpliu182gJ1z4Qb2oNMnb6ivSg25nIR7JzMJFxWZ5l5dmiudhVUIk8oj8VgF4iQgyY8oVw7FGoZH7gM0ZehfxKEpQeU07tnSlqY82+elSWn300+hEtdXQFWpnJSN/2jYsmCOA4Aj92duJVrTyvdKIPaAXyQibKBrBTYIRbKjbNRPRL+aL3Eo0W2bbIOxT5W5OE5FqE2maPQwf0vAeYli91Hp1HsYrIXkeOOiofYNQKV6zNV8l7s15I46BSgd0QKdBvLLdVhHhPDKmrKRHF9X7h+Gsgf/Oe2ptqhVwffCgzIfkwWQgpCWMOkY+hV9N9scv6t3zAu8EEA7CiLhcJ/ahQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gS7M4mv+sgTTxlL68snpTZ4ltNxdQSv0wHuPxWzPSN8=;
 b=cDDbQpHo7QGe+vEedOxmMVQxX3dHr+iAaGWrKiYazrujD0gT/bvuW94kBMSNritX2Cyld3AfyEyS2QL5aAnuCGAc06qutg+58zalag04qx3nZKuomY/vHOLkL/SuYgdVDf8AIpHeSovDl7CgdPRomRvUTso/oQKQmH2zyz2OfboQTf6kRmnAvy0vyNI1hTAi3iVh1H4CZCxj16MvfRxvnS8xrAoiOFxSnWGKLJSDowgqmjxQmVQRnT6chcn6KUYPS2op06VrauUpCpJngI48yqDcR8no/GG8hA0k2En/8ukDUBii9b1cXw1x4a671exmJyc25H2qNy38egYehpUZTQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gS7M4mv+sgTTxlL68snpTZ4ltNxdQSv0wHuPxWzPSN8=;
 b=ZKf6TnJ0V2C+oA1n3o33KmP4hNa1vs2L1wz1fGMB512cHVLI51jiVlB51d8mlcI9+M6Z/jLzp6GyHQdP/STltjCeoLcCtsT4WEBur5PFwvlCNxrMgfKydqfsUCEhYbj/ttqsSfaU04/i8QBKLD+3IWU9brST37mtoZhzIhuV8ys=
Date: Thu, 23 Sep 2021 12:38:52 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 7/9] x86/PVH: actually show Dom0's stacks from debug
 key '0'
Message-ID: <YUxZPK89MaabmXBd@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
 <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
X-ClientProxiedBy: LO4P123CA0321.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:197::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 59966ca8-bdb9-45a6-7fcf-08d97e7e587c
X-MS-TrafficTypeDiagnostic: DS7PR03MB5445:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5445BDB389318155539D327D8FA39@DS7PR03MB5445.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: HbqNO5OmqJepGQxlwjBihhMeZlQwnYsTb30VkCbHiqLZmhh/CMB2JIuyBJclfCL2hh+AAdb3WAYxIG0w8zi5M/BiHhMb7UgpdjquOBKCFJqvw24WC6iPV21yo7PjgFbpcswe428E8bQfnAsjdqOHl/koaRqE6o0ONBy5Hy8l6rZr1/3HjsRfJnJZ3hC1RHJ+Wxwkog4UqjRQ84T2Z6lXpo/El398ktragu0Fziejt3ir8HqxvGtGO41tBxCXRsNt2gw6baKNrcuWqAPNJi+gQEUIdFCQNOoFxMfeZA0HuOUcB7omgtGz1L+aCsoHEsChDOU+JqoDFf58X8nWWcxG0Fy5M5w5GTGkRN//lONLIKMhUOx1HsZF3NoGWJNJgEwtZMliFs2VXsba9q050kY/fwNjUjXTcENRjOf2Zti7xM37z/PdxSbOd7FQQC+ggouDXEdG5uAkRzuWiOUsLaaydBZImUhSg+ndZiFLnTqeBzf0+ig0NjmuXfUCN7empxHLyH7l13XRQSDkdYq5yqEceohhxna6+U+dgrqE8NsW2RQbGekoaH1am4CjmOKHu8lNrpwnt47LuE6Kyh2JDtwAkYUUFeHbih1XjUmq22jJZgudLd8X/v7IaBBuRdk3VTrnATv9WdrwsYemFT7u2kGz/A==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(508600001)(86362001)(186003)(6666004)(38100700002)(26005)(83380400001)(85182001)(4744005)(66476007)(66946007)(9686003)(8676002)(4326008)(66556008)(6916009)(6486002)(316002)(5660300002)(54906003)(2906002)(8936002)(6496006);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S3pPMnFCL3ZGVWVyR2ZlWXJBWmVxK3JoRHM5dVVxWUlMSGozaHQ0dHJ6UFNG?=
 =?utf-8?B?VGp5aUxrN3U3OGN4S0pwYXJEOEpOT3pvWSt3KzI5bDh1WElqL1YwejlNL1Jo?=
 =?utf-8?B?QW9MblM3LzFFN2NGd2hCSGFNeDgvSGptL1pjRHN1ZzdsQlBYOFpLdVU2dVIz?=
 =?utf-8?B?d3YzMFE1OWdNWFZaK3JVMWUrSzJVdW5DNUovQUlqTVBVYm0vSmRSL1NjM3Vx?=
 =?utf-8?B?MXpJVkxLMStTQzNtZ0p3b09FdWROZVJYS2R2dTN6MWkweXNLWXNQdlFsNTNq?=
 =?utf-8?B?NU9oeGFuU3UrSmdrTlY4dE1mMGRzcm5TandpVlV1d04zY1o4OVBzTTFicHBP?=
 =?utf-8?B?Y2RSNHYxeWwrZEQvUjBMS2toRzZGaHloYjRmNkcrNnVWYVVYbTZWUldLZ2FS?=
 =?utf-8?B?OE1Dd1BuK0RBUGVKdlNwSWdIdDNncTdwdVMxTmRwMWIyR1BUNzhoUFlYOTFD?=
 =?utf-8?B?eldiVDI2ajJoRGZTeE1nS3BhZXRHZkZjNHNBdnpEamU0T0JVMXdyNjd3bllL?=
 =?utf-8?B?YUhDQnNqc2RST3VQQkY5VGFvMUhlRVkzRytDWW5ZMmhOd3NldGdOWk53VXdM?=
 =?utf-8?B?dHF1MklQSWo0OFVicjlIdm9jQ0JWdkl4dlRuNUkyWkFSSC9LLzNNUHA3QmNw?=
 =?utf-8?B?MlliRU9JYitGZEhUbnk5Qys1cmNGVk9nQlBnUmxGVkhoSWVyZFZsRFRtck5z?=
 =?utf-8?B?bm03OE05bU1aTDF6VzBjVlRXN1Nsc3NoRVNHMDFzMHlZaXVtd2tiMDlXakhv?=
 =?utf-8?B?MVQ5WTJ6MHRZcVRVNURZL1ovM3pvZDhuSEFuVkNaUmdseUVDSjJBRVZLTExv?=
 =?utf-8?B?SEg3cVpHL2ZYRE1lRkxYS0VnaUVIYTBITTVSK1FVKzQxRXg0TTRlbElDYWlR?=
 =?utf-8?B?YmZCazdIRmtKYkxILzdMVHF1Y2xKZk9vNDY2REdZdnFDdko3ei9VQTZMSVJH?=
 =?utf-8?B?S2c5ZXAwc1IySExORjUvaHpjNGlrUFk0bmsvUDczZkY3NjUvUmYzUGtiV1Na?=
 =?utf-8?B?Wi8xSUNJS2xTMlhNYWJPOEIvYXZJOWp0dDdJSWVJSzZ5bmlYYVhRZk1ZazF6?=
 =?utf-8?B?UjA4SU5GWnl0YVhyY0YxZmliOHZCWEFqQ2tVL0h5NHFvaDNyREd1eWxDdDND?=
 =?utf-8?B?VU9QMFVQKzhYZjQyUzVTTGhwTTZmS0d3ZE9qYWRQM1g3TUdaZzRvcEVoTHVn?=
 =?utf-8?B?RE0vbjJTT1F6Yi9jc1NvWTdPSGduWlRaMFY2V0l4MjJPYUtINW5tVmVXdWN3?=
 =?utf-8?B?UUFMV1hpL0VWbjI4bjV6MTVUbWdQRG0rQTNReHFXQWEvZGtMN3FEV0NOb2dC?=
 =?utf-8?B?ajc3Mk1wSmdGb3N0T1ZLVkJjUG5NcTNYd0RmWldVODdGVFBZcUIrK2Qxekhl?=
 =?utf-8?B?Vm9reEwrSTJXN253RWpiejRtZnJkaXQwczh1RFptQjJERitkdmJUK242VkVy?=
 =?utf-8?B?Q1RHdmhRbVZ3VTI1WjNMKzl0NThNYThETmdLQUw4aGxNc3RpUFlydHJTRXA1?=
 =?utf-8?B?dzJvQXdvV1ZNbm1OWE50TmZwODdxV0xrZmROUnZmc1RRSGpqcFdoeWVUd25n?=
 =?utf-8?B?a0p4ZmtrcnlpKzZwQ1ZMVmh2Wml1NDJKSW5aMURzd2p6U2dlRll6NDlnbk0r?=
 =?utf-8?B?ZFN0M1dWak9ra1oxbkI1ODVIMjJEYzBGN0VPSmZZOVpDV0VrYzJubEdyWTBO?=
 =?utf-8?B?Zk1YL3JLd1NiNkxRM0RPbStmdWxKRCtFa0NNSW83VFZDR3JZb29SM2NnNHlV?=
 =?utf-8?Q?fUaGxnnVlvn0DEKub9f7A6qQRB9u7DmfBzaJMa6?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 59966ca8-bdb9-45a6-7fcf-08d97e7e587c
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:38:56.8514
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GPSS05U0dKcxr+g0mAPEy1Ez1cLi7UXuQxagDwWTrm4+7EIte8ugPJUqXJCbbG2XO7liGG9a8170a72E39Ih0w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5445
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 12:31:14PM +0200, Roger Pau Monné wrote:
> On Tue, Sep 21, 2021 at 09:20:00AM +0200, Jan Beulich wrote:
> > -    /* Prevent interleaving of output. */
> > -    flags = console_lock_recursive_irqsave();
> > +    /*
> > +     * Prevent interleaving of output if possible. For HVM we can't do so, as
> > +     * the necessary P2M lookups involve locking, which has to occur with IRQs
> > +     * enabled.
> > +     */
> > +    if ( !is_hvm_vcpu(v) )
> > +        flags = console_lock_recursive_irqsave();
> >  
> >      vcpu_show_registers(v);
> > -    if ( guest_kernel_mode(v, &v->arch.user_regs) )
> > +    if ( is_hvm_vcpu(v) )
> > +        show_hvm_stack(v, &v->arch.user_regs);
> 
> Would it make sense to unlock in show_hvm_stack, and thus keep the
> printing of vcpu_show_registers locked even when in HVM context?

To clarify, the unlock should be limited around the call to
show_hvm_stack, not to be performed inside of the function itself
(since we would have to pass flags around then).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:41:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:41:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193647.344938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMAg-0003sz-3T; Thu, 23 Sep 2021 10:41:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193647.344938; Thu, 23 Sep 2021 10:41:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMAg-0003ss-0Q; Thu, 23 Sep 2021 10:41:42 +0000
Received: by outflank-mailman (input) for mailman id 193647;
 Thu, 23 Sep 2021 10:41:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTMAe-0003sm-Pj
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:41:40 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id af60a608-fdcb-4a8d-9e26-b6a1c63334c0;
 Thu, 23 Sep 2021 10:41:39 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id i25so25036866lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 03:41:39 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id e28sm581123ljo.63.2021.09.23.03.41.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 03:41:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af60a608-fdcb-4a8d-9e26-b6a1c63334c0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=kLw0CGs448/djjAYxedYF/ESH4j1jZJzDssPc1coj74=;
        b=lolFy9reeK3R7+zLNy4DJi9pJy8L9LWtfRg2M9taCBAOPjzS0xLgkozyChs/zawY/L
         4BYCES3JjiSOhhbi/f8UnOciBSrMDsdmppTIuqMj4qijSXzjFHm68p+BEYjZv5iPnL2L
         lbEUlUzYdOjhoaBkYGyQ1u0EGUYr4wecjN2X8bGuvJYENnfe0yTnvyFWEiRxNfES00nC
         hzwDR1nfApklFJO9YC0tqMInXd4SXN5KNLY+lfWt/SEgcYjcPy2HDNmO2nOdSYYArrFs
         ha7CLyVWB4kjBLf1pRlqwJiKv9JV8NaI9XxEgtjnjhQQmb1nrnIwbyH++F8QJ5QeVUYH
         lWSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=kLw0CGs448/djjAYxedYF/ESH4j1jZJzDssPc1coj74=;
        b=lpCpCIh1V0hLflo9O9VQXkAnTgwSEM3wWXdzj33owbQF3jvYa9ozNX28XeA1FBPUGr
         CC8x6MEBtXiMRXSAhcsjfkXkYsfXH6gJHCj2CBx+wfpM+Y9VUMOXqwhxSDEJVQHeb8Un
         uVe7Oh8kXRoNKG072bxcIPLU6mGzbhxSo1EzczPrwV24AJkbh8x4dlWeGcodGlrjco0x
         u0/zO2KXoIXWMMswwO+cDORtB2PCGNvg/A5TTWt3hc0mZg78KgB+TmhnLncSD+U+xWho
         mHvxujGUGLxSs9wTwHrrxxvXv/Tbd3lbKIRXqyK1MVpBVFW+Ge4f1WC+fH/269sdhho7
         oIsg==
X-Gm-Message-State: AOAM5304/+nYjCy/DtBTtoqLBviuiMl9ybkA/t3nwMVBXsZahXD2U1jk
	juQ5N6dJpJUZVxI2O5SumKw=
X-Google-Smtp-Source: ABdhPJzSnTAvJrb/JengHJ1dzNzATaxINBJ0gLTZPoe3qW9tdia9iFWbXtJFSX4S4/C2yhU5Vl9TCg==
X-Received: by 2002:a05:6512:12c8:: with SMTP id p8mr3481462lfg.40.1632393697874;
        Thu, 23 Sep 2021 03:41:37 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
From: Oleksandr <olekstysh@gmail.com>
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <cc5ee8cc-84ac-a27f-af99-ac0ba3ab8d68@gmail.com>
Message-ID: <8665a10b-60b1-1551-0aab-e6725f4c6ff1@gmail.com>
Date: Thu, 23 Sep 2021 13:41:36 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <cc5ee8cc-84ac-a27f-af99-ac0ba3ab8d68@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


Hi Stefano, Julien


On 18.09.21 19:59, Oleksandr wrote:
>
> Hi Julien.
>
>
> [snip]
>
>
>>>
>>>
>>>> +#define EXT_REGION_END 0x80003fffffffULL
>>>> +
>>>> +static int __init find_unallocated_memory(const struct kernel_info 
>>>> *kinfo,
>>>> +                                          struct meminfo 
>>>> *ext_regions)
>>>> +{
>>>> +    const struct meminfo *assign_mem = &kinfo->mem;
>>>> +    struct rangeset *unalloc_mem;
>>>> +    paddr_t start, end;
>>>> +    unsigned int i;
>>>> +    int res;
>>>
>>> We technically already know which range of memory is unused. This is 
>>> pretty much any region in the freelist of the page allocator. So how 
>>> about walking the freelist instead?
>>
>> ok, I will investigate the page allocator code (right now I have no 
>> understanding of how to do that). BTW, I have just grepped "freelist" 
>> through the code and all page context related appearances are in x86 
>> code only.
>>
>>>
>>> The advantage is we don't need to worry about modifying the function 
>>> when adding new memory type.
>>>
>>> One disavantage is this will not cover *all* the unused memory as 
>>> this is doing. But I think this is an acceptable downside.
>
> I did some investigations and create test patch. Although, I am not 
> 100% sure this is exactly what you meant, but I will provide results 
> anyway.
>
> 1. Below the extended regions (unallocated memory, regions >=64MB ) 
> calculated by my initial method (bootinfo.mem - kinfo->mem - 
> bootinfo.reserved_mem - kinfo->gnttab):
>
> (XEN) Extended region 0: 0x48000000->0x54000000
> (XEN) Extended region 1: 0x57000000->0x60000000
> (XEN) Extended region 2: 0x70000000->0x78000000
> (XEN) Extended region 3: 0x78200000->0xc0000000
> (XEN) Extended region 4: 0x500000000->0x580000000
> (XEN) Extended region 5: 0x600000000->0x680000000
> (XEN) Extended region 6: 0x700000000->0x780000000
>
> 2. Below the extended regions (unallocated memory, regions >=64MB) 
> calculated by new method (free memory in page allocator):
>
> (XEN) Extended region 0: 0x48000000->0x54000000
> (XEN) Extended region 1: 0x58000000->0x60000000
> (XEN) Extended region 2: 0x70000000->0x78000000
> (XEN) Extended region 3: 0x78200000->0x84000000
> (XEN) Extended region 4: 0x86000000->0x8a000000
> (XEN) Extended region 5: 0x8c200000->0xc0000000
> (XEN) Extended region 6: 0x500000000->0x580000000
> (XEN) Extended region 7: 0x600000000->0x680000000
> (XEN) Extended region 8: 0x700000000->0x765e00000
>
> Some thoughts regarding that.
>
> 1. A few ranges below 4GB are absent in resulting extended regions. I 
> assume, this is because of the modules:
>
> (XEN) Checking for initrd in /chosen
> (XEN) Initrd 0000000084000040-0000000085effc48
> (XEN) RAM: 0000000048000000 - 00000000bfffffff
> (XEN) RAM: 0000000500000000 - 000000057fffffff
> (XEN) RAM: 0000000600000000 - 000000067fffffff
> (XEN) RAM: 0000000700000000 - 000000077fffffff
> (XEN)
> (XEN) MODULE[0]: 0000000078080000 - 00000000781d74c8 Xen
> (XEN) MODULE[1]: 0000000057fe7000 - 0000000057ffd080 Device Tree
> (XEN) MODULE[2]: 0000000084000040 - 0000000085effc48 Ramdisk
> (XEN) MODULE[3]: 000000008a000000 - 000000008c000000 Kernel
> (XEN) MODULE[4]: 000000008c000000 - 000000008c010000 XSM
> (XEN)  RESVD[0]: 0000000084000040 - 0000000085effc48
> (XEN)  RESVD[1]: 0000000054000000 - 0000000056ffffff
>
> 2. Also, it worth mentioning that relatively large chunk (~417MB) of 
> memory above 4GB is absent (to be precise, at the end of last RAM 
> bank), which I assume, used for Xen internals.
> We could really use it for extended regions.
> Below free regions in the heap (for last RAM bank) just in case:
>
> (XEN) heap[node=0][zone=23][order=5] 0x00000765ec0000-0x00000765ee0000
> (XEN) heap[node=0][zone=23][order=6] 0x00000765e80000-0x00000765ec0000
> (XEN) heap[node=0][zone=23][order=7] 0x00000765e00000-0x00000765e80000
> (XEN) heap[node=0][zone=23][order=9] 0x00000765c00000-0x00000765e00000
> (XEN) heap[node=0][zone=23][order=10] 0x00000765800000-0x00000765c00000
> (XEN) heap[node=0][zone=23][order=11] 0x00000765000000-0x00000765800000
> (XEN) heap[node=0][zone=23][order=12] 0x00000764000000-0x00000765000000
> (XEN) heap[node=0][zone=23][order=14] 0x00000760000000-0x00000764000000
> (XEN) heap[node=0][zone=23][order=17] 0x00000740000000-0x00000760000000
> (XEN) heap[node=0][zone=23][order=18] 0x00000540000000-0x00000580000000
> (XEN) heap[node=0][zone=23][order=18] 0x00000500000000-0x00000540000000
> (XEN) heap[node=0][zone=23][order=18] 0x00000640000000-0x00000680000000
> (XEN) heap[node=0][zone=23][order=18] 0x00000600000000-0x00000640000000
> (XEN) heap[node=0][zone=23][order=18] 0x00000700000000-0x00000740000000
>
> Yes, you already pointed out this disadvantage, so if it is an 
> acceptable downside, I am absolutely OK.
>
>
> 3. Common code updates. There is a question how to properly make a 
> connection between common allocator internals and Arm's code for 
> creating DT. I didn’t come up with anything better
> than creating for_each_avail_page() for invoking a callback with page 
> and its order.
>
> **********
>
> Below the proposed changes on top of the initial patch, would this be 
> acceptable in general?
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 523eb19..1e58fc5 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -753,16 +753,33 @@ static int __init add_ext_regions(unsigned long 
> s, unsigned long e, void *data)
>      return 0;
>  }
>
> +static int __init add_unalloc_mem(struct page_info *page, unsigned 
> int order,
> +                                  void *data)
> +{
> +    struct rangeset *unalloc_mem = data;
> +    paddr_t start, end;
> +    int res;
> +
> +    start = page_to_maddr(page);
> +    end = start + pfn_to_paddr(1UL << order);
> +    res = rangeset_add_range(unalloc_mem, start, end - 1);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        return res;
> +    }
> +
> +    return 0;
> +}
> +
>  #define EXT_REGION_START   0x40000000ULL
>  #define EXT_REGION_END     0x80003fffffffULL
>
> -static int __init find_unallocated_memory(const struct kernel_info 
> *kinfo,
> -                                          struct meminfo *ext_regions)
> +static int __init find_unallocated_memory(struct meminfo *ext_regions)
>  {
> -    const struct meminfo *assign_mem = &kinfo->mem;
>      struct rangeset *unalloc_mem;
>      paddr_t start, end;
> -    unsigned int i;
>      int res;
>
>      dt_dprintk("Find unallocated memory for extended regions\n");
> @@ -771,59 +788,9 @@ static int __init find_unallocated_memory(const 
> struct kernel_info *kinfo,
>      if ( !unalloc_mem )
>          return -ENOMEM;
>
> -    /* Start with all available RAM */
> -    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> -    {
> -        start = bootinfo.mem.bank[i].start;
> -        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> -        res = rangeset_add_range(unalloc_mem, start, end - 1);
> -        if ( res )
> -        {
> -            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> -                   start, end);
> -            goto out;
> -        }
> -    }
> -
> -    /* Remove RAM assigned to Dom0 */
> -    for ( i = 0; i < assign_mem->nr_banks; i++ )
> -    {
> -        start = assign_mem->bank[i].start;
> -        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> -        if ( res )
> -        {
> -            printk(XENLOG_ERR "Failed to remove: 
> %#"PRIx64"->%#"PRIx64"\n",
> -                   start, end);
> -            goto out;
> -        }
> -    }
> -
> -    /* Remove reserved-memory regions */
> -    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> -    {
> -        start = bootinfo.reserved_mem.bank[i].start;
> -        end = bootinfo.reserved_mem.bank[i].start +
> -            bootinfo.reserved_mem.bank[i].size;
> -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> -        if ( res )
> -        {
> -            printk(XENLOG_ERR "Failed to remove: 
> %#"PRIx64"->%#"PRIx64"\n",
> -                   start, end);
> -            goto out;
> -        }
> -    }
> -
> -    /* Remove grant table region */
> -    start = kinfo->gnttab_start;
> -    end = kinfo->gnttab_start + kinfo->gnttab_size;
> -    res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +    res = for_each_avail_page(add_unalloc_mem, unalloc_mem);
>      if ( res )
> -    {
> -        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> -               start, end);
>          goto out;
> -    }
>
>      start = EXT_REGION_START;
>      end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> @@ -840,8 +807,7 @@ out:
>      return res;
>  }
>
> -static int __init find_memory_holes(const struct kernel_info *kinfo,
> -                                    struct meminfo *ext_regions)
> +static int __init find_memory_holes(struct meminfo *ext_regions)
>  {
>      struct dt_device_node *np;
>      struct rangeset *mem_holes;
> @@ -961,9 +927,9 @@ static int __init make_hypervisor_node(struct 
> domain *d,
>      else
>      {
>          if ( !is_iommu_enabled(d) )
> -            res = find_unallocated_memory(kinfo, ext_regions);
> +            res = find_unallocated_memory(ext_regions);
>          else
> -            res = find_memory_holes(kinfo, ext_regions);
> +            res = find_memory_holes(ext_regions);
>
>          if ( res )
>              printk(XENLOG_WARNING "Failed to allocate extended 
> regions\n");
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 8fad139..7cd1020 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1572,6 +1572,40 @@ static int reserve_heap_page(struct page_info *pg)
>
>  }
>
> +/* TODO heap_lock? */
> +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, 
> void *),
> +                        void *data)
> +{
> +    unsigned int node, zone, order;
> +    int ret;
> +
> +    for ( node = 0; node < MAX_NUMNODES; node++ )
> +    {
> +        if ( !avail[node] )
> +            continue;
> +
> +        for ( zone = 0; zone < NR_ZONES; zone++ )
> +        {
> +            for ( order = 0; order <= MAX_ORDER; order++ )
> +            {
> +                struct page_info *head, *tmp;
> +
> +                if ( page_list_empty(&heap(node, zone, order)) )
> +                    continue;
> +
> +                page_list_for_each_safe ( head, tmp, &heap(node, 
> zone, order) )
> +                {
> +                    ret = cb(head, order, data);
> +                    if ( ret )
> +                        return ret;
> +                }
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +
>  int offline_page(mfn_t mfn, int broken, uint32_t *status)
>  {
>      unsigned long old_info = 0;
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 667f9da..64dd3e2 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -123,6 +123,9 @@ unsigned int online_page(mfn_t mfn, uint32_t 
> *status);
>  int offline_page(mfn_t mfn, int broken, uint32_t *status);
>  int query_page_offline(mfn_t mfn, uint32_t *status);
>
> +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, 
> void *),
> +                        void *data);
> +
>  void heap_init_late(void);
>
>  int assign_pages(


I am sorry, but may I please clarify regarding that? Whether we will go 
this new direction (free memory in page allocator) or leave things as 
they are (bootinfo.mem - kinfo->mem - bootinfo.reserved_mem - 
kinfo->gnttab). This is only one still unclear moment to me in current 
patch before preparing V3.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:43:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:43:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193652.344948 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMC6-0004TM-EL; Thu, 23 Sep 2021 10:43:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193652.344948; Thu, 23 Sep 2021 10:43:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMC6-0004TF-BN; Thu, 23 Sep 2021 10:43:10 +0000
Received: by outflank-mailman (input) for mailman id 193652;
 Thu, 23 Sep 2021 10:43:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QBZb=ON=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mTMC4-0004T9-No
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:43:08 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.56]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 099f7838-1c5b-11ec-ba28-12813bfff9fa;
 Thu, 23 Sep 2021 10:43:07 +0000 (UTC)
Received: from AM3PR05CA0107.eurprd05.prod.outlook.com (2603:10a6:207:1::33)
 by AM6PR08MB3238.eurprd08.prod.outlook.com (2603:10a6:209:45::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 10:42:56 +0000
Received: from VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:207:1:cafe::e0) by AM3PR05CA0107.outlook.office365.com
 (2603:10a6:207:1::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 10:42:56 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT039.mail.protection.outlook.com (10.152.19.196) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 10:42:55 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 10:42:52 +0000
Received: from 0d0de202793a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 859F5F06-91C3-4BA1-85D0-B8936D608975.1; 
 Thu, 23 Sep 2021 10:42:45 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0d0de202793a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 10:42:45 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR3PR08MB5579.eurprd08.prod.outlook.com (2603:10a6:102:82::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 10:42:43 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 10:42:42 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0415.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18b::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 10:42:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 099f7838-1c5b-11ec-ba28-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=moOsx1sdIhrnMVxdu08lygWSOT5+XJcJk6731bVt/Is=;
 b=0/BFkK3v++N0znxVo4we64NigCX9FqpyGm+1sV4NhcNBH32hzXrSMF9SXFie69MBRlwNULyhxd3rIaPn9mC8r6HEFF7TrwEmYVwKTFLpgLFUz8TuKg46HbXam+H1Jmyzf4ks99oaq5k4DgNL+RLtN1xAAwBBa5rXiPGOLZq0x8E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6a3034e2b863e2f3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wo0PioaQrF27kWDzUpGDy3GkN3nXnN8VL5gCzfQpFr3excwjXJ3/yApVyPE0MTNKDkM6ZJNmIislekP7OpvLkPLP0OJIxKhLtJNNQrpkEN1/xPEEROVcLF/rAsTasHA2kzZ75tOZhxbnMZexn2TS9yIcBvPvP6LpD7QpQXb1Zu7RwxD3sY8AHkFW1hz5zuWkDoyllq0RYnawGG5+1i7l4AdgS3gdmF4jLnuzuVP+5GhMtaCqYjgnAPz/x4fdqHevjIQbs2gI+fiuednmrYnZh5ECGBBjH/eJqBjY6o4DyFohh85rZvRDywu+7uoF/dom8D+LlTrDUyaOIsSPCBKbRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=moOsx1sdIhrnMVxdu08lygWSOT5+XJcJk6731bVt/Is=;
 b=g4svbwQs8HUZQ9eDWovYOQhbKRFFrjs54zuKDZSv/M+XzjUgvRoYuSwXjgCqZ1Gy8X28NZbz41S0Dcij5EYoQZo4TCX1tPY/3vSYxD94GJfdOdQ+5kIZ0mksbx0t5LOXHhN23RqRuYhXMN0GHlPLrqJv8aZwC+cJaj5uUA+Ul8iZTPHZFlPnvMIrsmHmrQSfPFK4rX9vewkv8fb9kz+9QORFks5YFizp8w66eSaGs8ifPrvlT5lN14LAydsOOatgGFqSM6G5ddWq8n1r98lfZW0mBzMRIRHpGOfkNpcWx000IAAnzn2QoSooUlc+na+RYob0FiKDJ56L1NQYL6FRSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=moOsx1sdIhrnMVxdu08lygWSOT5+XJcJk6731bVt/Is=;
 b=0/BFkK3v++N0znxVo4we64NigCX9FqpyGm+1sV4NhcNBH32hzXrSMF9SXFie69MBRlwNULyhxd3rIaPn9mC8r6HEFF7TrwEmYVwKTFLpgLFUz8TuKg46HbXam+H1Jmyzf4ks99oaq5k4DgNL+RLtN1xAAwBBa5rXiPGOLZq0x8E=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v2 1/2] arm/efi: Introduce uefi,cfg-load DT property
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109221411200.17979@sstabellini-ThinkPad-T480s>
Date: Thu, 23 Sep 2021 11:42:34 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <4804F311-5341-4AC0-9408-1CFF1A2F23DE@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>
 <20210922141341.42288-2-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109221411200.17979@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0415.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18b::6) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0408b62b-528c-4c7b-c98e-08d97e7ee742
X-MS-TrafficTypeDiagnostic: PR3PR08MB5579:|AM6PR08MB3238:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB323831C6E715C372C72EDBE1E4A39@AM6PR08MB3238.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 nIzuknwvKsVFnLkIDpjXGKg7gHPaisuI1TxPiDf6fSeV+XYjWxwQ3Y/DoOKQLHU1Wn22esRqA1cZjZpiDj8MHoM6d5eFoEqx+tyyAH6tsQGsj9ilXej9nDGLtAZo2DV847091FdLrZPOhSr2qyWAMEBzaLsbtUGMM4NTTFR7a7+dphsO1raNEThjeDK3yNIeWk1hJ6fWc4AWuWoJ/2JO7Hc8F67p3sCX+6VCeoiVa6S0GuQZuzfbQ5vB3+9FxIZwHPKKTCFwJ3rSMUe2Tspv5DWZyyPXPg1DjxaB6orv+kHznsJ/rXvpjUcNg+QDEgb68FWFER70jUL/b8+9LNH1c30yYcrVUERE7DxVj/WDAXH55thVuWGPf9Mb5oZmBh9k+6liVu5XWtV+KYYBSwJrtnnk+kytdeC+0D7rQy//ulFwWZr+xijqi6KcyF7fHZH0s6LcjHJuuYgx/OSTXifXHmb08wcT/betIuiBqj9FHSyiJgQ+WclPBV6U3qzNsRiU+RcD9WND4eEuILAfS8v20JnXU90cjQ62SJ/t4yojkzgvk2xLYz8FrLkl9Cu6V+Y2le4rRczmqouoEYemg5xMD0hJbxCXuWzbFPDj7gHq+uagKoWRIrkwJYHOEmzQPBVDZ6BZvQdUk+l2FkL0Il1sSyBBHuwsVcq3Xou1S4Be4G9uX5vKSx1K1liF/kxSS/4IvBjjexUdNc26MXVeycid32IMuEUtc2X20RDByYzzKM4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(5660300002)(2616005)(86362001)(53546011)(956004)(508600001)(33656002)(6916009)(66556008)(66476007)(6512007)(54906003)(316002)(8936002)(83380400001)(4326008)(26005)(6486002)(8676002)(52116002)(186003)(36756003)(6506007)(38100700002)(6666004)(38350700002)(2906002)(44832011)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5579
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f79a8db5-9464-4ce0-3357-08d97e7edf32
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9pxrKaoU+d2XYvp0vqkQQJIQupSYSeC07aYxRtmnvpnOLF/ZU+jtg7N6jAnm9mSECjB+wX8d9x58uiMMm8JwjoDGaqJyP8zprm19Ws7v9abtBPwSLbPVhMCJmc+qZiHGEh9CL9j5b8FesoC1yGJFfor4t45mdvyi/HsEZ6QziWQkb6X+Q8A9Ls3fT7lYUUQP5x6VjvkKqZJ78UVVwm7D6lxJzeXlUA9h1EfIb5hW/tFO86ePUgkdU0YK6/LpYBemBUGJV268yZolKwg73ucWDgEhRahQodIBCkCQQ7Ckj3yyJvXPm2DLmbGLbdBU8xtjINWWk5Tn+eSsxDPyF4a0MKnQCMYWtIJOPMN2ux9OhmbyucfbGUlrfs3OxNbZgc8VBvhCqkV2Q3sxSjdrGtqnSjFeNmJqRipEq6mK3HtValitDviUrZXscO7IyhGMYFuzLJqjo4tBIaVHzYFNIR7FX/OQR732Ph+fhVWnxK1+tJQpKwiuQf3fLmait4fCNQfeZ55Agf6yhhzqpzqfOOLfv2Yf/5Az9TSzuo6jv2Fev0l7Jh2cZvnf/qib/q4aGW6wM5baxR3TvKGEXe+RtsXj60gMocTsgPTRF/ovREdLfP/iwWYEalzNIxn1yT26tF3aDrp6hE2BFRu/aFTperqzzrPhP1MP3x4scl6zv5VbkcWelxt1Btr3qmZ4HgcVjLBgUgGuQ4y7EhBP5hzOdHVw3g==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(54906003)(316002)(36756003)(107886003)(44832011)(6862004)(47076005)(5660300002)(508600001)(86362001)(6666004)(81166007)(82310400003)(8936002)(956004)(26005)(8676002)(70206006)(70586007)(36860700001)(33656002)(53546011)(6506007)(186003)(6512007)(2616005)(336012)(356005)(83380400001)(6486002)(2906002)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:42:55.9469
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0408b62b-528c-4c7b-c98e-08d97e7ee742
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT039.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3238



> On 22 Sep 2021, at 22:19, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Wed, 22 Sep 2021, Luca Fancellu wrote:
>> Introduce the uefi,cfg-load DT property of /chosen
>> node for ARM whose presence decide whether to force
>> the load of the UEFI Xen configuration file.
>>=20
>> The logic is that if any multiboot,module is found in
>> the DT, then the uefi,cfg-load property is used to see
>> if the UEFI Xen configuration file is needed.
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>=20
> The patch looks OK, just a couple of minor comments below.
>=20
>=20
>> ---
>> v2 changes:
>> - Introduced uefi,cfg-load property
>> - Add documentation about the property
>> ---
>> docs/misc/efi.pandoc        |  2 ++
>> xen/arch/arm/efi/efi-boot.h | 28 +++++++++++++++++++++++-----
>> 2 files changed, 25 insertions(+), 5 deletions(-)
>>=20
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index ac3cd58cae..e289c5e7ba 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tr=
ee provided to Xen.  If a
>> bootloader provides a device tree containing modules then any configurat=
ion
>> files are ignored, and the bootloader is responsible for populating all
>> relevant device tree nodes.
>> +The property "uefi,cfg-load" can be specified in the /chosen node to fo=
rce Xen
>> +to load the configuration file even if multiboot modules are found.
>=20

Hi Stefano,

> I think that, in addition to this, we also need to add the property to
> docs/misc/arm/device-tree/booting.txt where our "official" device tree
> bindings are maintained. I would add it below "Command lines" and before
> "Creating Multiple Domains directly from Xen" maybe as a new chapter.
> It could be called "Other Options" but other ideas could be valid too.
>=20
> You can say that uefi,cfg-load is a boolean.

Sure, I will add in v3, what about this:

UEFI boot and DT
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

When Xen is booted using UEFI, it doesn't read the configuration file if an=
y
multiboot module is specified. To force Xen to load the configuration file,=
 the
boolean property uefi,cfg-load must be declared in the /chosen node.

>=20
>> Once built, `make install-xen` will place the resulting binary directly =
into
>> the EFI boot partition, provided `EFI_VENDOR` is set in the environment =
(and
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index cf9c37153f..8ceeba4ad1 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LO=
ADED_IMAGE *loaded_image)
>>=20
>> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTabl=
e)
>> {
>> +    bool skip_cfg_file =3D false;
>>     /*
>>      * For arm, we may get a device tree from GRUB (or other bootloader)
>>      * that contains modules that have already been loaded into memory. =
 In
>> -     * this case, we do not use a configuration file, and rely on the
>> -     * bootloader to have loaded all required modules and appropriate
>> -     * options.
>> +     * this case, we search for the property uefi,cfg-load in the /chos=
en node
>> +     * to decide whether to skip the UEFI Xen configuration file or not=
.
>>      */
>>=20
>>     fdt =3D lookup_fdt_config_table(SystemTable);
>>     dtbfile.ptr =3D fdt;
>>     dtbfile.need_to_free =3D false; /* Config table memory can't be free=
d. */
>> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,modul=
e") < 0 )
>> +
>> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 =
)
>> +    {
>> +        /* Locate chosen node */
>> +        int node =3D fdt_subnode_offset(fdt, 0, "chosen");
>> +        const void *cfg_load_prop;
>> +        int cfg_load_len;
>> +
>> +        if ( node > 0 )
>> +        {
>> +            /* Check if uefi,cfg-load property exists */
>> +            cfg_load_prop =3D fdt_getprop(fdt, node, "uefi,cfg-load",
>> +                                        &cfg_load_len);
>> +            if ( !cfg_load_prop )
>> +                skip_cfg_file =3D true;
>> +        }
>> +    }
>> +
>> +    if ( !fdt || !skip_cfg_file )
>=20
> Just a suggestion, but rather than the double negative, wouldn't it be
> simpler to define it as
>=20
>    bool load_cfg_file =3D true;
>=20
> ?

Sure I will modify it.

>=20
>=20
>>     {
>>         /*
>>          * We either have no FDT, or one without modules, so we must hav=
e a
>> -         * Xen EFI configuration file to specify modules.  (dom0 requir=
ed)
>> +         * Xen EFI configuration file to specify modules.
>>          */
>=20
> Also mention in the commit message that you are taking the opportunity
> to update this comment do remove "dom0 required".

Yes I will add it in the commit message

Cheers,
Luca

>=20
>=20
>>         return true;
>>     }
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:45:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:45:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193659.344960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMEH-0005BX-1r; Thu, 23 Sep 2021 10:45:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193659.344960; Thu, 23 Sep 2021 10:45:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMEG-0005BQ-Sl; Thu, 23 Sep 2021 10:45:24 +0000
Received: by outflank-mailman (input) for mailman id 193659;
 Thu, 23 Sep 2021 10:45:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMEF-0005BK-OJ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:45:23 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMEF-0005gV-Fy; Thu, 23 Sep 2021 10:45:23 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMEF-00017L-5O; Thu, 23 Sep 2021 10:45:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=uTIOvtDFhgTQagOCLz/7ETcHasUNW8MPG5ftpDFLmmo=; b=s6ufslsD7cdXAjLoyofnn7uihe
	F7vgyybdDFcWYwq0c6TATwCCd8UQtVAN85uE7WufSrdIorKi7Lu0XeRGP11HPWdzfbFNOWtx/Evjj
	MEFZKVRW0cMY5dr2lOQQlL6HRuiSz022zHbeba6PJgODfw2BASuUrOcGcOEgCXygWPLI=;
Subject: Re: [PATCH 04/11] xen/arm: introduce accessors for vgic dist, cpu,
 and rdist base addresses
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-5-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <cebb55da-7a24-fd51-1fdc-b40f2ed437d7@xen.org>
Date: Thu, 23 Sep 2021 15:45:19 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-5-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> Introduce accessors for vgic dist, cpu, and rdist base addresses, on
> all gic types.
> 
> Use the accessors when making gic node for device tree of domU.

Please explain in the commit message why you need them.

however, I am not entirely convined of the usefulness of the helpers. It 
will call to bits that are already directly accessible and you could 
simply #ifdef make_gicv3_domU_node.

> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/domain_build.c    | 21 ++++++++++++-----
>   xen/include/asm-arm/new_vgic.h | 24 ++++++++++++++++++++
>   xen/include/asm-arm/vgic.h     | 41 ++++++++++++++++++++++++++++++++++
>   3 files changed, 80 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 213ad017dc..d5f201f73e 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1802,8 +1802,12 @@ static int __init make_gicv2_domU_node(struct kernel_info *kinfo)
>       int res = 0;
>       __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
>       __be32 *cells;
> +    struct domain *d = kinfo->d;
> +    char buf[38];
>   
> -    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICD_BASE));
> +    snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
> +             vgic_dist_base(&d->arch.vgic));
> +    res = fdt_begin_node(fdt, buf);
>       if ( res )
>           return res;
>   
> @@ -1825,9 +1829,9 @@ static int __init make_gicv2_domU_node(struct kernel_info *kinfo)
>   
>       cells = &reg[0];
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                       GUEST_GICD_BASE, GUEST_GICD_SIZE);
> +                       vgic_dist_base(&d->arch.vgic), GUEST_GICD_SIZE);
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                       GUEST_GICC_BASE, GUEST_GICC_SIZE);
> +                       vgic_cpu_base(&d->arch.vgic), GUEST_GICC_SIZE);
>   
>       res = fdt_property(fdt, "reg", reg, sizeof(reg));
>       if (res)
> @@ -1852,8 +1856,12 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
>       int res = 0;
>       __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
>       __be32 *cells;
> +    struct domain *d = kinfo->d;
> +    char buf[38];
>   
> -    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICV3_GICD_BASE));
> +    snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
> +             vgic_dist_base(&d->arch.vgic));
> +    res = fdt_begin_node(fdt, buf);
>       if ( res )
>           return res;
>   
> @@ -1875,9 +1883,10 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
>   
>       cells = &reg[0];
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                       GUEST_GICV3_GICD_BASE, GUEST_GICV3_GICD_SIZE);
> +                       vgic_dist_base(&d->arch.vgic), GUEST_GICV3_GICD_SIZE);
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                       GUEST_GICV3_GICR0_BASE, GUEST_GICV3_GICR0_SIZE);
> +                       vgic_rdist_base(&d->arch.vgic, 0),
> +                       vgic_rdist_size(&d->arch.vgic, 0));
>   
>       res = fdt_property(fdt, "reg", reg, sizeof(reg));
>       if (res)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 62c2ae538d..e1bc5113da 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -277,6 +277,47 @@ enum gic_sgi_mode;
>    */
>   #define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
>   
> +static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)

The names of the helpers suggest that if we pass any domain (including 
dom0), then we would return the correct address. However, here this is 
only going to return the address for the guests.

This looks like to be solved by the follow-up patches (in particular 
patch #7, #8). So I think it would be best to first introduced all the 
fields and then use it directly here.

> +{
> +    return GUEST_GICC_BASE;
> +}
> +
> +static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
> +{
> +    return GUEST_GICD_BASE;
> +}
> +
> +#ifdef CONFIG_GICV3
> +static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
> +{
> +    return GUEST_GICV3_RDIST_REGIONS;
> +}
> +
> +static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
> +{
> +    return GUEST_GICV3_GICR0_BASE;
> +}
> +
> +static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
> +{
> +    return GUEST_GICV3_GICR0_SIZE;
> +}
> +#else
> +static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
> +{
> +    return 0;
> +}
> +
> +static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
> +{
> +    return INVALID_PADDR;
> +}
> +
> +static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
> +{
> +    return INVALID_PADDR;
> +}
> +#endif
>   
>   extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int virq);
>   extern void vgic_remove_irq_from_queues(struct vcpu *v, struct pending_irq *p);
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:47:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:47:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193664.344970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMG6-0005qE-Al; Thu, 23 Sep 2021 10:47:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193664.344970; Thu, 23 Sep 2021 10:47:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMG6-0005q7-7t; Thu, 23 Sep 2021 10:47:18 +0000
Received: by outflank-mailman (input) for mailman id 193664;
 Thu, 23 Sep 2021 10:47:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMG5-0005q1-33
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:47:17 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMG4-0005i8-TU; Thu, 23 Sep 2021 10:47:16 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMG4-0001HB-I1; Thu, 23 Sep 2021 10:47:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=n5w3XaparN2cwLfd1PJuST3kkmtv9IuXlkEHjqY5efs=; b=Kv1WCwSzjD0QzsM8/ZybBU7iXC
	fDmW9kxytsf4z7CkNey5zXmqqVM7cE43qbVyIbyCouIjBF8DHvOnVwEKKX7l2ty2vnLKafmr1HHLm
	DibH4dhYbyVH7i6dAVKLwhhn9pXMAfiysX1/rEOMXew+a7obAWcE74VLFqSX0XREuJrM=;
Subject: Re: [PATCH 05/11] xen/arm: vgic: introduce vgic.cbase
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-6-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <71709a54-6bb1-1981-d661-61c54d26f888@xen.org>
Date: Thu, 23 Sep 2021 15:47:12 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-6-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi Penny,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> Add a field in struct vgic_dist to store the cpu interface base address,
> similar to the existing dbase field.
> 
> Use the field in vgic_v2_domain_init, instead of original local variable.

Please explain in the commit message why this is necessary.

> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/vgic-v2.c     | 10 +++++-----
>   xen/include/asm-arm/vgic.h |  1 +
>   2 files changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index b2da886adc..b34ec88106 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -652,7 +652,7 @@ static int vgic_v2_vcpu_init(struct vcpu *v)
>   static int vgic_v2_domain_init(struct domain *d)
>   {
>       int ret;
> -    paddr_t cbase, csize;
> +    paddr_t csize;
>       paddr_t vbase;
>   
>       /*
> @@ -669,7 +669,7 @@ static int vgic_v2_domain_init(struct domain *d)
>            * Note that we assume the size of the CPU interface is always
>            * aligned to PAGE_SIZE.
>            */
> -        cbase = vgic_v2_hw.cbase;
> +        d->arch.vgic.cbase = vgic_v2_hw.cbase;
>           csize = vgic_v2_hw.csize;
>           vbase = vgic_v2_hw.vbase;
>       }
> @@ -683,7 +683,7 @@ static int vgic_v2_domain_init(struct domain *d)
>            * region.
>            */
>           BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
> -        cbase = GUEST_GICC_BASE;
> +        d->arch.vgic.cbase = GUEST_GICC_BASE;
>           csize = GUEST_GICC_SIZE;

Shouldn't we also stash d->arch.vgic.csize?

>           vbase = vgic_v2_hw.vbase + vgic_v2_hw.aliased_offset;
>       }
> @@ -692,8 +692,8 @@ static int vgic_v2_domain_init(struct domain *d)
>        * Map the gic virtual cpu interface in the gic cpu interface
>        * region of the guest.
>        */
> -    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
> -                           maddr_to_mfn(vbase));
> +    ret = map_mmio_regions(d, gaddr_to_gfn(d->arch.vgic.cbase),
> +                           csize / PAGE_SIZE, maddr_to_mfn(vbase));
>       if ( ret )
>           return ret;
>   
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index e1bc5113da..d5ad1f387b 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -152,6 +152,7 @@ struct vgic_dist {
>       struct pending_irq *pending_irqs;
>       /* Base address for guest GIC */
>       paddr_t dbase; /* Distributor base address */
> +    paddr_t cbase; /* CPU interface base address */
>   #ifdef CONFIG_GICV3
>       /* GIC V3 addressing */
>       /* List of contiguous occupied by the redistributors */
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:47:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:47:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193667.344982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMGP-0006T3-Jj; Thu, 23 Sep 2021 10:47:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193667.344982; Thu, 23 Sep 2021 10:47:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMGP-0006Sw-GB; Thu, 23 Sep 2021 10:47:37 +0000
Received: by outflank-mailman (input) for mailman id 193667;
 Thu, 23 Sep 2021 10:47:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTMGO-0006Ix-47
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:47:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a71e0192-1c5b-11ec-ba29-12813bfff9fa;
 Thu, 23 Sep 2021 10:47:32 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6-aji_W9-9MweHJFIQ7iCnzQ-1; Thu, 23 Sep 2021 12:47:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4351.eurprd04.prod.outlook.com (2603:10a6:803:49::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 10:47:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 10:47:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0227.eurprd02.prod.outlook.com (2603:10a6:20b:28f::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 10:47:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a71e0192-1c5b-11ec-ba29-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632394051;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6ynZOPV3oqoTxSvfyfMk4y35fb66QaVR9wjA/3b3zu0=;
	b=kFQZ0izESg1efd8p1HCPhHHQ8YZ5/Skry12TGLKqGP8VcC7tgQvSFAXYxq9SxzZiEAwE1d
	gLQjEtbMWx96dKSrqasLGoopUgGPHVnHecdonWuevzr8VDe9zN0XM/kx3MiYy9pNL0fZcO
	7m/7agZtaISe5d2bHMD0GtEDS2nfb14=
X-MC-Unique: aji_W9-9MweHJFIQ7iCnzQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cjB8HT/PJvGCNegWBAIwBEmhMqH2HTha3ktr9PMo5a9Bv/3/qZL3YVUEfwiCCMSexi80iot9wiJvqbIK/BQNWrIdCEe+wCQw6td9tzWe++vjzEp96bzpWpWm60sakkiJIX19qzHHza7FbgUDJ7GonHfBr4HCYyHr5VhtvJXNBFrtIjHBRM/cKjfu5KH65QCsNPcwwCnVNTJ3lXbp+EIVCXb5CyHH7fyuQhJN2W/DNKBQDq/V8hcnu2kgFWTiwkY25pqlt8DkhDyTlQYzq7tsOobLY9QwE8+g3fzBmlrUz7dbEarGEA13V8to7h4B3yimkk4rhfcBgaTTGpsLsDr3DQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dWTt9Hzwf1WGDtN+3AbYodjdeBnm9iihDjOXU4Qg7/Q=;
 b=DomRDal7hziBIjgduZyZ/50onsKGcWBpcLQDTU4Yy24Mtds+9gykbZdJqHnB184qkcj6haeGc+T0qDuU70SqeSIbUeFbVg1KpjfIGEUrB8uMY4ch4aolx/6yiXwafgTiN+QkYV/Fm6FcRGMlfFBKcmKZpi/AfIjgOT4WbBZxIpTpVLiiVlfpIDg1E/6QqJ9tN89vQXj+j33N3ukGEH+xsyvkFCszocb6cWoZxo7pcgC0vreS1s92BP41BJ9j2NMNxIMSEle/jfBMoZH+Yh0wUdRfFeZSQOBilBN931u0IL5vSkJdUQozz6ZLQxCCedz896VW/sXqYxI53aBlLSmQmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 7/9] x86/PVH: actually show Dom0's stacks from debug
 key '0'
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
 <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3b292c10-e346-4af0-1575-acdb060f0c41@suse.com>
Date: Thu, 23 Sep 2021 12:47:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0227.eurprd02.prod.outlook.com
 (2603:10a6:20b:28f::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 77e334f4-4f06-47cc-fab1-08d97e7f8965
X-MS-TrafficTypeDiagnostic: VI1PR04MB4351:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43516685E41121EDDFB349C6B3A39@VI1PR04MB4351.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	U6VoRI24exTj/wQNRdlP1EJxec9B0F/VYIfMyZ2fzH/qvNi4FkyWOUGAHSVXBe4Z99x1ydZhkSDlzliiNFxMVkp7AwxuzH6AV/Ue7V4aDvLqaMc+W/ceppnvF4/zcb32ITIaZzT/IgbYci9RpgiDuLuHo9+5q6yUS7ijSS6bK31y0y0B2Xky1YqsKdIwkt/ToVHOcnooDylj/bOtE5ngWDdKN1SMUvx15MOOFIroA52kjvC/5pSJlmL/vBtm3qRidK86+YUl+Z2++kgYUhMs6UzlBQ3dzUYIaSv/yEs+Se/6fPuy2hbln8+XhdruVosvxk8n9orv9Mgql6Qak8CdD0PFEbW/Jtdy77gkPuKZ4f5mUM+JV7jhxas76sx2MTYGOOaKMZ78y+nn2g31OL5vh9IheYGlLkaQPjbbKFMfnzdw7tJGzszEGvF2hPlKk20FdZBvJM6Q0QHoxmVK9PhFmrS0KiIJK2l/kuqpMnZTgNceWVD9oS8OCMKmTu6mLTS8iss6ZbFK2pjkTf9/9ZZU9Y553dQUvgYb+kEkTyn36/XOpIHrmYg9f81AVcErCFZJZUOiOhVC1+RFSZoDZDBbYIk3VlnVxuoCuYvXKSe3lZUs9F45kD/rJJacTyZ9rMISIngI4o4pj0nOWDjeP3vv6jKsgoNSH53Hq2/WiaYew94ZhJ4BGnnQlN/78pS9H+DKo/RvTdtBjZIOLOBBzq77WNjaGX2GadUza3YozU18684=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(86362001)(956004)(16576012)(508600001)(2616005)(6916009)(316002)(8936002)(4326008)(83380400001)(38100700002)(66946007)(31696002)(6486002)(31686004)(36756003)(53546011)(54906003)(26005)(66476007)(5660300002)(66556008)(186003)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Q5LoQPKmtsjQfviw8q6JQ849IbzI9blUBC8CGdAr2a//6B+EXgRCGjtBJJJu?=
 =?us-ascii?Q?awT6M85/PzlaSTkYVWxhYJy3kHQgGmBWhW56KvPYLZ3zwAXjliv+evAUs4k2?=
 =?us-ascii?Q?8MsDa1JbsHseX5VvodvUjDBvp+06fTeOrDe4d5s72TZKcee7zicR1l5TYDgd?=
 =?us-ascii?Q?ijAq3z8a689BndjgTDAlkB0V+ZZx9dBx8HQ9DvjhxwYVCAbOqC8nnkrXWtac?=
 =?us-ascii?Q?kJ1S7S49gQF/43gVW8VWiEo/eYrvxnrAMcyiwFebrf1Y+NoqLT+5fzKBp0qE?=
 =?us-ascii?Q?ou7jV3MIS6frfPTVil2l0k3NrMncZqR6TDuEKO57kc8ch1GMmFc4bqssrN2u?=
 =?us-ascii?Q?4gJ3sAad35gBqQ3UghDEr4mHpQhz11DuUGw9LRyCNMoXa8E3NBxKbHCwVYgG?=
 =?us-ascii?Q?ORbnqhZ61SEBsa1qIvKOnkaT4QzOPH43/nokAjBsUH9k65+UrLeCpf8FNhwF?=
 =?us-ascii?Q?wg/nt4xbFJHT1QdciD1iIpLk/sg1lFox84Hqe3i48eynrjcZfQ0ARlgrE2p/?=
 =?us-ascii?Q?ZdzquMV3YS0wQMW58mLljaf50bYLHeRGgrx//X+u6YVwX+Z3OjrGF8gD3Ggp?=
 =?us-ascii?Q?kI0fR6/D96xxQYsAuTlDCGC0Si2ckCIF6KmoVJayC/1vP8fLNIHsPTMls1eA?=
 =?us-ascii?Q?hTSeaMA2QZyYsG81SxMx3RoYqn1q14pq99n7PNYxRkALaSgomjpBZj1QskXy?=
 =?us-ascii?Q?Bw71LsJTp8ePxHT0NbOyLlB9l7cUL7ZpEwysLwA/VBuBDm+5otbAwkPL6eYZ?=
 =?us-ascii?Q?By9yQgPpE6MA4Ge6R/tfOhsvT37kglra+/2tACFoPRNzkK+Z9ZwrlqY6fxqt?=
 =?us-ascii?Q?2u13DNUPwEkpZMGLVE9iefyKPc2xojz1Rsgu3JQvj5ElhPHKWlTs1iKLPQxU?=
 =?us-ascii?Q?98dvwKPwK46nhR9h1yZwsrI3d2EtXcV4aXBucl5RL08Jw8yJZ+ZVrHOUD7Lt?=
 =?us-ascii?Q?WwNZnXTYHmWZgRiCPuwYU7i4Z5JsGP9VUOOEBd5AGJtXCZ39fNrwIYmcahpT?=
 =?us-ascii?Q?1bRwTc/qmAee7E7p5BEEo7JAzPCUITiZthhzJ7npg6NOuoE6uZJFS2h/rLLE?=
 =?us-ascii?Q?utFGI0k0/UCCdD419dDF4a/Jftw20X2fLyjOXxl+RKPCxQkfwfyDgs9IPtIi?=
 =?us-ascii?Q?U5J4nQ0IGspY7HKj5R6Wq501vIPTFVdQvKFQu1RqLOv2dBhW1/mhktwDaaCG?=
 =?us-ascii?Q?FUDmMkEUijwLDy/9lGtLKXgVJAlh6QaI3TGN+lJ8dBCkg034b+j2Ww/CMvav?=
 =?us-ascii?Q?4hEMAvg7Y/5X4YOR4NGnJjNvu+LjMmY7qw9tONYLREIlv5Ttc6vy1pD/sMds?=
 =?us-ascii?Q?yt5hp0/nfbazXhRypZvCVn9p?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 77e334f4-4f06-47cc-fab1-08d97e7f8965
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 10:47:28.1413
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: C2LVpZKxNFKzwNP3c25v6LTid2Mhtqf6kNhHXJ+ovj5ta8Ilk4tMsWn4g6R5j8J0ejPAhEZnegeJB5tes6G0sw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4351

On 23.09.2021 12:31, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:20:00AM +0200, Jan Beulich wrote:
>> --- a/xen/arch/x86/hvm/hvm.c
>> +++ b/xen/arch/x86/hvm/hvm.c
>> @@ -3408,6 +3408,15 @@ enum hvm_translation_result hvm_copy_fro
>>                        PFEC_page_present | pfec, pfinfo);
>>  }
>> =20
>> +enum hvm_translation_result hvm_copy_from_vcpu_linear(
>> +    void *buf, unsigned long addr, unsigned int size, struct vcpu *v,
>> +    unsigned int pfec)
>=20
> Even if your current use case doesn't need it, would it be worth
> adding a pagefault_info_t parameter?

I'd prefer to add such parameters only once they become necessary.

>> --- a/xen/arch/x86/traps.c
>> +++ b/xen/arch/x86/traps.c
>> @@ -364,6 +364,71 @@ static void show_guest_stack(struct vcpu
>>      printk("\n");
>>  }
>> =20
>> +static void show_hvm_stack(struct vcpu *v, const struct cpu_user_regs *=
regs)
>> +{
>> +#ifdef CONFIG_HVM
>> +    unsigned long sp =3D regs->rsp, addr;
>> +    unsigned int i, bytes, words_per_line, pfec =3D PFEC_page_present;
>> +    struct segment_register ss, cs;
>> +
>> +    hvm_get_segment_register(v, x86_seg_ss, &ss);
>> +    hvm_get_segment_register(v, x86_seg_cs, &cs);
>> +
>> +    if ( hvm_long_mode_active(v) && cs.l )
>> +        i =3D 16, bytes =3D 8;
>> +    else
>> +    {
>> +        sp =3D ss.db ? (uint32_t)sp : (uint16_t)sp;
>> +        i =3D ss.db ? 8 : 4;
>> +        bytes =3D cs.db ? 4 : 2;
>> +    }
>> +
>> +    if ( bytes =3D=3D 8 || (ss.db && !ss.base) )
>> +        printk("Guest stack trace from sp=3D%0*lx:", i, sp);
>> +    else
>> +        printk("Guest stack trace from ss:sp=3D%04x:%0*lx:", ss.sel, i,=
 sp);
>> +
>> +    if ( !hvm_vcpu_virtual_to_linear(v, x86_seg_ss, &ss, sp, bytes,
>> +                                     hvm_access_read, &cs, &addr) )
>> +    {
>> +        printk(" Guest-inaccessible memory\n");
>> +        return;
>> +    }
>> +
>> +    if ( ss.dpl =3D=3D 3 )
>> +        pfec |=3D PFEC_user_mode;
>> +
>> +    words_per_line =3D stack_words_per_line * (sizeof(void *) / bytes);
>> +    for ( i =3D 0; i < debug_stack_lines * words_per_line; )
>> +    {
>> +        unsigned long val =3D 0;
>> +
>> +        if ( (addr ^ (addr + bytes - 1)) & PAGE_SIZE )
>> +            break;
>> +
>> +        if ( !(i++ % words_per_line) )
>> +            printk("\n  ");
>> +
>> +        if ( hvm_copy_from_vcpu_linear(&val, addr, bytes, v,
>> +                                       pfec) !=3D HVMTRANS_okay )
>=20
> I think I'm confused, but what about guests without paging enabled?
> Don't you need to use hvm_copy_from_guest_phys (likely transformed
> into hvm_copy_from_vcpu_phys)?

__hvm_copy() calls hvm_translate_get_page() telling it whether the
input is a linear or physical address. hvm_translate_get_page() will
use paging_gva_to_gfn() in this case. The HAP backing function
changes when the guest {en,dis}ables paging, while shadow code deals
with paging disabled by installing an identity mapping
(d->arch.paging.shadow.unpaged_pagetable) which it would then end up
(needlessly) walking.

It really is - afaict - intentional for callers to not have to deal
with the special case.

>> @@ -663,14 +728,22 @@ void vcpu_show_execution_state(struct vc
>>      }
>>  #endif
>> =20
>> -    /* Prevent interleaving of output. */
>> -    flags =3D console_lock_recursive_irqsave();
>> +    /*
>> +     * Prevent interleaving of output if possible. For HVM we can't do =
so, as
>> +     * the necessary P2M lookups involve locking, which has to occur wi=
th IRQs
>> +     * enabled.
>> +     */
>> +    if ( !is_hvm_vcpu(v) )
>> +        flags =3D console_lock_recursive_irqsave();
>> =20
>>      vcpu_show_registers(v);
>> -    if ( guest_kernel_mode(v, &v->arch.user_regs) )
>> +    if ( is_hvm_vcpu(v) )
>> +        show_hvm_stack(v, &v->arch.user_regs);
>=20
> Would it make sense to unlock in show_hvm_stack, and thus keep the
> printing of vcpu_show_registers locked even when in HVM context?

Perhaps not _in_, but before calling it, yet - why not.

> TBH I've never found the guest stack dump to be helpful for debugging
> purposes, but maybe others do.

I can't count how many times I did use the stack dumps of Dom0 to
actually pinpoint problems.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:47:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:47:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193671.344992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMGh-00075J-SW; Thu, 23 Sep 2021 10:47:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193671.344992; Thu, 23 Sep 2021 10:47:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMGh-000759-PL; Thu, 23 Sep 2021 10:47:55 +0000
Received: by outflank-mailman (input) for mailman id 193671;
 Thu, 23 Sep 2021 10:47:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMGf-000714-Rs
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:47:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMGf-0005iY-Ol; Thu, 23 Sep 2021 10:47:53 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMGf-0001JI-Ed; Thu, 23 Sep 2021 10:47:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=AsQaybsN+y6K9nSwJw8z39y9Zr8Ja6WqTb2Egp8zowk=; b=LJLXolA/St+7XfSzRfMn6lIHVX
	Z9CeQJAYi4MXjpwGA+W+v8EMh872Hx6tu7TgGvaYcyk+JtPsfxM0zC5mPXGbR5iQ8E65gu2wQGgw5
	MnvzkTsJv4IZHBgXOToW0OzG9WO26tB9bM35ARhMJdrzqIkScDtB/m0ksRZLOnvpDxdQ=;
Subject: Re: [PATCH 06/11] xen/arm: new vgic: update vgic_cpu_base
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-7-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <b8c3659d-e1d5-0957-6e4a-89e3222a0480@xen.org>
Date: Thu, 23 Sep 2021 15:47:47 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-7-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> In vgic_v2_map_resources, use the new vgic_cpu_base field of struct
> vgic_dist instead of original local variable.

I think this wants to be folded in patch #5.

> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/vgic/vgic-v2.c | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
> index b5ba4ace87..fd452fcb5a 100644
> --- a/xen/arch/arm/vgic/vgic-v2.c
> +++ b/xen/arch/arm/vgic/vgic-v2.c
> @@ -258,7 +258,7 @@ void vgic_v2_enable(struct vcpu *vcpu)
>   int vgic_v2_map_resources(struct domain *d)
>   {
>       struct vgic_dist *dist = &d->arch.vgic;
> -    paddr_t cbase, csize;
> +    paddr_t csize;
>       paddr_t vbase;
>       int ret;
>   
> @@ -276,7 +276,7 @@ int vgic_v2_map_resources(struct domain *d)
>            * Note that we assume the size of the CPU interface is always
>            * aligned to PAGE_SIZE.
>            */
> -        cbase = gic_v2_hw_data.cbase;
> +        d->arch.vgic.vgic_cpu_base = gic_v2_hw_data.cbase;
>           csize = gic_v2_hw_data.csize;
>           vbase = gic_v2_hw_data.vbase;
>       }
> @@ -290,7 +290,7 @@ int vgic_v2_map_resources(struct domain *d)
>            * region.
>            */
>           BUILD_BUG_ON(GUEST_GICC_SIZE != SZ_8K);
> -        cbase = GUEST_GICC_BASE;
> +        d->arch.vgic.vgic_cpu_base = GUEST_GICC_BASE;
>           csize = GUEST_GICC_SIZE;
>           vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
>       }
> @@ -308,8 +308,8 @@ int vgic_v2_map_resources(struct domain *d)
>        * Map the gic virtual cpu interface in the gic cpu interface
>        * region of the guest.
>        */
> -    ret = map_mmio_regions(d, gaddr_to_gfn(cbase), csize / PAGE_SIZE,
> -                           maddr_to_mfn(vbase));
> +    ret = map_mmio_regions(d, gaddr_to_gfn(d->arch.vgic.vgic_cpu_base),
> +                           csize / PAGE_SIZE, maddr_to_mfn(vbase));
>       if ( ret )
>       {
>           gdprintk(XENLOG_ERR, "Unable to remap VGIC CPU to VCPU\n");
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:52:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:52:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193683.345004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMKm-0000G6-Lp; Thu, 23 Sep 2021 10:52:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193683.345004; Thu, 23 Sep 2021 10:52:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMKm-0000Fz-Ij; Thu, 23 Sep 2021 10:52:08 +0000
Received: by outflank-mailman (input) for mailman id 193683;
 Thu, 23 Sep 2021 10:52:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMKl-0000Ft-DP
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:52:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMKl-0005oC-5s; Thu, 23 Sep 2021 10:52:07 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMKk-0001dh-RS; Thu, 23 Sep 2021 10:52:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=Q6dwjDIOEVQxb8x7Gyad+iCjguS1FstWTow7Js2xGwI=; b=c4ZWhFBRTo7XMXx5V11NRCxVnT
	SvAZPfQny4IkOgHQvdkIP3UUipmZGjjn4J7+W3NWV8VdmY/vth6c8X6TSZB9EY/rict2zt33S4EDB
	QtBUUxMcqy2xNJYBngBRkQT2wm2lAqxBR36m8ZJVg/w17XYHfSxBZtKmS3huzCV8TJyg=;
Subject: Re: [PATCH 07/11] xen/arm: if 1:1 direct-map domain use native
 addresses for GICv2
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-8-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <ac986e10-0ecf-3a2a-65a7-4cfe6567c64f@xen.org>
Date: Thu, 23 Sep 2021 15:52:00 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-8-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> Today we use native addresses to map the GICv2 for Dom0 and fixed
> addresses for DomUs.
> 
> This patch changes the behavior so that native addresses are used for
> all domains that are 1:1 direct-map(including Dom0).
> 
> Update the accessors to use the struct vgic variables to provide the
> updated addresses.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/vgic-v2.c         | 16 ++++++++++++++++
>   xen/arch/arm/vgic/vgic-v2.c    | 17 +++++++++++++++++
>   xen/include/asm-arm/new_vgic.h |  4 ++--
>   xen/include/asm-arm/vgic.h     |  4 ++--
>   4 files changed, 37 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index b34ec88106..a8cf8173d0 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -673,6 +673,22 @@ static int vgic_v2_domain_init(struct domain *d)
>           csize = vgic_v2_hw.csize;
>           vbase = vgic_v2_hw.vbase;
>       }
> +    else if ( is_domain_direct_mapped(d) )
> +    {
> +        /*
> +         * For non-dom0 direct_mapped guests we only map a 8kB CPU
> +         * interface but we make sure it is at a location occupied by
> +         * the physical GIC in the host device tree.
> +         *
> +         * We need to add an offset to the virtual CPU interface base
> +         * address when the GIC is aliased to get a 8kB contiguous
> +         * region.
> +         */

So I agree that we need to differentiate between dom0 and other direct 
mapped domains. However, I think it would be good to explainm why given 
that in other places you treat dom0 and direct mapped domain the same way.

> +        d->arch.vgic.dbase = vgic_v2_hw.dbase;
> +        d->arch.vgic.cbase = vgic_v2_hw.cbase + vgic_v2_hw.aliased_offset;
> +        csize = GUEST_GICC_SIZE;
> +        vbase = vgic_v2_hw.vbase + vgic_v2_hw.aliased_offset;
> +    }
>       else
>       {
>           d->arch.vgic.dbase = GUEST_GICD_BASE;
> diff --git a/xen/arch/arm/vgic/vgic-v2.c b/xen/arch/arm/vgic/vgic-v2.c
> index fd452fcb5a..ce1f6e4373 100644
> --- a/xen/arch/arm/vgic/vgic-v2.c
> +++ b/xen/arch/arm/vgic/vgic-v2.c
> @@ -280,6 +280,23 @@ int vgic_v2_map_resources(struct domain *d)
>           csize = gic_v2_hw_data.csize;
>           vbase = gic_v2_hw_data.vbase;
>       }
> +    else if ( is_domain_direct_mapped(d) )
> +    {
> +        d->arch.vgic.vgic_dist_base = gic_v2_hw_data.dbase;
> +        /*
> +         * For non-dom0 direct_mapped guests we only map a 8kB CPU
> +         * interface but we make sure it is at a location occupied by
> +         * the physical GIC in the host device tree.
> +         *
> +         * We need to add an offset to the virtual CPU interface base
> +         * address when the GIC is aliased to get a 8kB contiguous
> +         * region.
> +         */
> +        d->arch.vgic.vgic_cpu_base = gic_v2_hw_data.cbase +
> +                                     gic_v2_hw_data.aliased_offset;
> +        csize = GUEST_GICC_SIZE;
> +        vbase = gic_v2_hw_data.vbase + gic_v2_hw_data.aliased_offset;
> +    }
>       else
>       {
>           d->arch.vgic.vgic_dist_base = GUEST_GICD_BASE;
> diff --git a/xen/include/asm-arm/new_vgic.h b/xen/include/asm-arm/new_vgic.h
> index 9097522b27..0d16f878eb 100644
> --- a/xen/include/asm-arm/new_vgic.h
> +++ b/xen/include/asm-arm/new_vgic.h
> @@ -188,12 +188,12 @@ struct vgic_cpu {
>   
>   static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
>   {
> -    return GUEST_GICC_BASE;
> +    return vgic->vgic_cpu_base;
>   }
>   
>   static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
>   {
> -    return GUEST_GICD_BASE;
> +    return vgic->vgic_dist_base;
>   }
>   
>   static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index d5ad1f387b..92f6a2d15d 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -280,12 +280,12 @@ enum gic_sgi_mode;
>   
>   static inline paddr_t vgic_cpu_base(struct vgic_dist *vgic)
>   {
> -    return GUEST_GICC_BASE;
> +    return vgic->cbase;
>   }
>   
>   static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
>   {
> -    return GUEST_GICD_BASE;
> +    return vgic->dbase;
>   }
>   
>   #ifdef CONFIG_GICV3 >

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:52:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:52:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193681.345015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTML4-0000fr-VC; Thu, 23 Sep 2021 10:52:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193681.345015; Thu, 23 Sep 2021 10:52:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTML4-0000fk-SF; Thu, 23 Sep 2021 10:52:26 +0000
Received: by outflank-mailman (input) for mailman id 193681;
 Thu, 23 Sep 2021 10:51:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZpLY=ON=anisinha.ca=ani@srs-us1.protection.inumbo.net>)
 id 1mTMK4-0000Eq-KF
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:51:24 +0000
Received: from mail-pl1-x634.google.com (unknown [2607:f8b0:4864:20::634])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 11461a33-4c5f-4c4b-b0b0-1094f814d6d0;
 Thu, 23 Sep 2021 10:51:23 +0000 (UTC)
Received: by mail-pl1-x634.google.com with SMTP id y1so3720435plk.10
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 03:51:23 -0700 (PDT)
Received: from anisinha-lenovo ([115.96.156.170])
 by smtp.googlemail.com with ESMTPSA id e13sm5257765pfi.210.2021.09.23.03.51.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 03:51:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11461a33-4c5f-4c4b-b0b0-1094f814d6d0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=anisinha-ca.20210112.gappssmtp.com; s=20210112;
        h=from:date:to:cc:subject:in-reply-to:message-id:references
         :user-agent:mime-version;
        bh=4fJXlclBbnkVQPBywtqy0QsfK3GLzEGhebPQ86PZ4yM=;
        b=UwDv30mJvlcUbc5vsqv5ShtJ3oE0rdWz5cuSzKVT1VMx7oOAhptBwIZn+62mHKdIda
         CcTYo+EH+KpGcDyZdaNnrJToFa4sWqSUcsxSlx9OQMQwM6QNX2Bww7OSG0RQXXbD3De5
         /pWCE/0WJrdF+Yon0++dBCaOxfyz5xkXytI1DQ+bx6EBoEMTn9XJoS68bJ+H7GOwtgB8
         qoSCK6s0PZy1TP/XYoD8RYoHjlNP3fM2zc0dTANnUrIbLemr+exTR/fR3LouYTH11512
         q0A2gT3vXM5gNC++K3kAcqiW6ZU/MP54Ed6BdACd1DXgZye8fizrfC83ybMpvRifxhXA
         lPMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:date:to:cc:subject:in-reply-to:message-id
         :references:user-agent:mime-version;
        bh=4fJXlclBbnkVQPBywtqy0QsfK3GLzEGhebPQ86PZ4yM=;
        b=VXIpjNXwcQ4dC8pW25Ecqfz+r5N47avqQvSKRZwqj+8qRBdQYGKcgfs4WpxqrQFOR1
         nb6pAux0UA7DBrH8Y8ysGR1GLDgRNrQ8zw/3FMtLgp+TiFT7HjXBgf3VgQ3FoV9Ywps1
         Ddez/dtmi2FxiI1QmHiyEQWGkPvzHqnGK2iUDjfBTDhUWylwhFfLAckeMrTd0ZvBsKoE
         yOst9ZZAwLM3r7YgrzBh/w2LDhpNIaHqxtj/DHYaFyeWrzb4GAbIWqrHyJ/WIF3yB2k8
         J5GrpbDNQIEAQSbj8EyZfVceu1EBbOvQuzqcqK5RnVkq7Pitql+fY6iYZmaUPxoBwoni
         sufg==
X-Gm-Message-State: AOAM5339eKGXUUwHByCHSTAfQJWFRVtz2hKmpSuVD4m6mRZefRAfHDqr
	u1ZVUwN1iEuRNnvHFmmCLtutVg==
X-Google-Smtp-Source: ABdhPJzia+90/vmutfv9+OVhfiSOiEAHqZONBLVWc1gCXXoo34eh7DSYhO2/LLO7Y/O8VFiKodZgxw==
X-Received: by 2002:a17:90b:4d8f:: with SMTP id oj15mr4553491pjb.13.1632394282429;
        Thu, 23 Sep 2021 03:51:22 -0700 (PDT)
From: Ani Sinha <ani@anisinha.ca>
X-Google-Original-From: Ani Sinha <anisinha@anisinha.ca>
Date: Thu, 23 Sep 2021 16:21:11 +0530 (IST)
X-X-Sender: anisinha@anisinha-lenovo
To: Damien Hedde <damien.hedde@greensocs.com>
cc: qemu-devel@nongnu.org, Alistair Francis <Alistair.Francis@wdc.com>, 
    =?ISO-8859-15?Q?Marc-Andr=E9_Lureau?= <marcandre.lureau@redhat.com>, 
    Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, 
    Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, 
    =?ISO-8859-15?Q?Daniel_P=2E_Berrang=E9?= <berrange@redhat.com>, 
    "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, 
    Ani Sinha <ani@anisinha.ca>, Gerd Hoffmann <kraxel@redhat.com>, 
    Eric Auger <eric.auger@redhat.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, 
    "Dr. David Alan Gilbert" <dgilbert@redhat.com>, 
    Markus Armbruster <armbru@redhat.com>, Eric Blake <eblake@redhat.com>, 
    qemu-riscv@nongnu.org, xen-devel@lists.xenproject.org, 
    mark.burton@greensocs.com, mirela.grujic@greensocs.com, edgari@xilinx.com, 
    Peter Maydell <peter.maydell@linaro.org>
Subject: Re: [RFC PATCH v2 07/16] hw/core/machine: add
 machine_class_is_dynamic_sysbus_dev_allowed
In-Reply-To: <20210922161405.140018-8-damien.hedde@greensocs.com>
Message-ID: <alpine.DEB.2.22.394.2109231559230.630@anisinha-lenovo>
References: <20210922161405.140018-1-damien.hedde@greensocs.com> <20210922161405.140018-8-damien.hedde@greensocs.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII



On Wed, 22 Sep 2021, Damien Hedde wrote:

> Right now the allowance check for adding a sysbus device using
> -device cli option (or device_add qmp command) is done well after
> the device has been created. It is done during the machine init done
> notifier: machine_init_notify() in hw/core/machine.c
>
> This new function will allow us to check if a sysbus device type is
> allowed to be dynamically created by the machine during the device
> creation time.
>
> Also make device_is_dynamic_sysbus() use the new function.
>
> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> ---
>
> In the context of our series, we need to be able to do the check at
> device creation time to allow doing it depending on the current
> MACHINE_INIT phase.
> ---
>  include/hw/boards.h | 17 +++++++++++++++++
>  hw/core/machine.c   | 15 ++++++++++++---
>  2 files changed, 29 insertions(+), 3 deletions(-)
>
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 463a5514f9..934443c1cd 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -51,6 +51,23 @@ void machine_set_cpu_numa_node(MachineState *machine,
>   */
>  void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type);
>
> +/**
> + * machine_class_is_dynamic_sysbus_dev_allowed: Check if type is an allowed
> + * sysbus device type for the machine class.
> + * @mc: Machine class
> + * @type: type to check (should be a subtype of TYPE_SYS_BUS_DEVICE)
> + *
> + * Returns: true if @type is a type in the machine's list of
> + * dynamically pluggable sysbus devices; otherwise false.
> + *
> + * Check if the QOM type @type is in the list of allowed sysbus device
> + * types (see machine_class_allowed_dynamic_sysbus_dev()).
> + * Note that if @type is a subtype of a type which is in the list, it is
> + * allowed too.
> + */
> +bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
> +                                                 const char *type);
> +

How about renaming this to device_type_is_allowed_dynamic_sysbus() ?

>  /**
>   * device_is_dynamic_sysbus: test whether device is a dynamic sysbus device
>   * @mc: Machine class
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 9125c9aad0..1a18912dc8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -545,18 +545,27 @@ void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type)
>
>  bool device_is_dynamic_sysbus(MachineClass *mc, DeviceState *dev)
>  {
> -    bool allowed = false;
> -    strList *wl;
>      Object *obj = OBJECT(dev);
>
>      if (!object_dynamic_cast(obj, TYPE_SYS_BUS_DEVICE)) {
>          return false;
>      }
>
> +    return machine_class_is_dynamic_sysbus_dev_allowed(mc,
> +            object_get_typename(obj));
> +}
> +
> +bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
> +                                                 const char *type)
> +{
> +    bool allowed = false;
> +    strList *wl;
> +    ObjectClass *klass = object_class_by_name(type);
> +
>      for (wl = mc->allowed_dynamic_sysbus_devices;
>           !allowed && wl;
>           wl = wl->next) {
> -        allowed |= !!object_dynamic_cast(obj, wl->value);
> +        allowed |= !!object_class_dynamic_cast(klass, wl->value);
>      }
>
>      return allowed;
> --
> 2.33.0
>
>


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 10:59:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 10:59:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193692.345026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMS6-0001no-OT; Thu, 23 Sep 2021 10:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193692.345026; Thu, 23 Sep 2021 10:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMS6-0001nh-Kw; Thu, 23 Sep 2021 10:59:42 +0000
Received: by outflank-mailman (input) for mailman id 193692;
 Thu, 23 Sep 2021 10:59:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMS4-0001nb-W1
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 10:59:41 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMS4-0005wS-Pn; Thu, 23 Sep 2021 10:59:40 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMS4-0002Vj-Fo; Thu, 23 Sep 2021 10:59:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=xQRgn9oeecq7YDlbjVCHlp9AqlPzYJ0NNSTWpfkmSHs=; b=CLKQ74khakJNGPdHnh/h7kxl0g
	/QRxGXgxDZcrIL7VyOLTKMwWZC7n8hvf+tG6Zg3g8ws+LYkSmEwhMm0rMaq6Gz84NU967ThonKbdx
	BmtXleJ6Rj6cCPHgcm7jETNlJnTspH6fPc+JZiAw9lgTO4Jc0JlmTqbtSkX7Yfo/ZQuI=;
Subject: Re: [PATCH 08/11] xen/arm: if 1:1 direct-map domain use native
 addresses for GICv3
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-9-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <3efa0620-0a55-bce7-1e2c-66f3449af7cc@xen.org>
Date: Thu, 23 Sep 2021 15:59:35 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-9-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> Today we use native addresses to map the GICv3 for Dom0 and fixed
> addresses for DomUs.
> 
> This patch changes the behavior so that native addresses are used for
> all domains that are 1:1 direct-map(including Dom0).
> 
> Update the rdist accessor to use the struct vgic variables to provide
> the updated addresses.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/domain_build.c | 32 ++++++++++++++++++++++++--------
>   xen/arch/arm/vgic-v3.c      | 10 +++++-----
>   xen/include/asm-arm/vgic.h  |  6 +++---
>   3 files changed, 32 insertions(+), 16 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d5f201f73e..120f1ae575 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1854,10 +1854,11 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
>   {
>       void *fdt = kinfo->fdt;
>       int res = 0;
> -    __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
> +    __be32 *reg;
>       __be32 *cells;
>       struct domain *d = kinfo->d;
>       char buf[38];
> +    unsigned int i, len = 0;
>   
>       snprintf(buf, sizeof(buf), "interrupt-controller@%"PRIx64,
>                vgic_dist_base(&d->arch.vgic));
> @@ -1881,27 +1882,42 @@ static int __init make_gicv3_domU_node(struct kernel_info *kinfo)
>       if ( res )
>           return res;
>   
> +    /* reg specifies all re-distributors and Distributor. */
> +    len = (GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) *
> +          (vgic_rdist_nr(&d->arch.vgic) + 1) * sizeof(__be32);
> +    reg = xmalloc_bytes(len);
> +    if ( reg == NULL )
> +        return -ENOMEM;
> +
>       cells = &reg[0];
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
>                          vgic_dist_base(&d->arch.vgic), GUEST_GICV3_GICD_SIZE);
> -    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                       vgic_rdist_base(&d->arch.vgic, 0),
> -                       vgic_rdist_size(&d->arch.vgic, 0));
> +    for ( i = 0;
> +          i < vgic_rdist_nr(&d->arch.vgic);
> +          i++, cells += (GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) )
> +    {
> +        dt_child_set_range(&cells,
> +                           GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> +                           vgic_rdist_base(&d->arch.vgic, i),
> +                           vgic_rdist_size(&d->arch.vgic, i));
> +    }
>   
> -    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    res = fdt_property(fdt, "reg", reg, len);
>       if (res)
> -        return res;
> +        goto out;
>   
>       res = fdt_property_cell(fdt, "linux,phandle", kinfo->phandle_gic);
>       if (res)
> -        return res;
> +        goto out;
>   
>       res = fdt_property_cell(fdt, "phandle", kinfo->phandle_gic);
>       if (res)
> -        return res;
> +        goto out;
>   
>       res = fdt_end_node(fdt);
>   
> + out:
> +    xfree(reg);
>       return res;
>   }
>   
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index cb5a70c42e..83f996b46c 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1647,8 +1647,8 @@ static inline unsigned int vgic_v3_max_rdist_count(struct domain *d)
>        * However DomU get a constructed memory map, so we can go with
>        * the architected single redistributor region.
>        */

The comment needs to be updated.

> -    return is_hardware_domain(d) ? vgic_v3_hw.nr_rdist_regions :
> -               GUEST_GICV3_RDIST_REGIONS;
> +    return is_domain_direct_mapped(d) ? vgic_v3_hw.nr_rdist_regions :
> +                                        GUEST_GICV3_RDIST_REGIONS;

Let's avoid to make the assumption that dom0 is always direct mapped. So 
this should be "is_domain_direct_mapped(d) || is_hardware_domain(d)".

I would actually suggest to introduce a new helper 
"is_domain_use_host_layout()" (or similar) that wraps the two check. 
This would make the code a bit more flexible.

>   }
>   
>   static int vgic_v3_domain_init(struct domain *d)
> @@ -1670,10 +1670,10 @@ static int vgic_v3_domain_init(struct domain *d)
>       radix_tree_init(&d->arch.vgic.pend_lpi_tree);
>   
>       /*
> -     * Domain 0 gets the hardware address.
> -     * Guests get the virtual platform layout.
> +     * 1:1 direct-map domain (including Dom0) gets the hardware address.
> +     * Other guests get the virtual platform layout.
>        */
> -    if ( is_hardware_domain(d) )
> +    if ( is_domain_direct_mapped(d) )

Same here.

>       {
>           unsigned int first_cpu = 0;
>   
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 92f6a2d15d..0f7cb32c58 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -291,17 +291,17 @@ static inline paddr_t vgic_dist_base(struct vgic_dist *vgic)
>   #ifdef CONFIG_GICV3
>   static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
>   {
> -    return GUEST_GICV3_RDIST_REGIONS;
> +    return vgic->nr_regions;
>   }
>   
>   static inline paddr_t vgic_rdist_base(struct vgic_dist *vgic, unsigned int i)
>   {
> -    return GUEST_GICV3_GICR0_BASE;
> +    return vgic->rdist_regions[i].base;
>   }
>   
>   static inline paddr_t vgic_rdist_size(struct vgic_dist *vgic, unsigned int i)
>   {
> -    return GUEST_GICV3_GICR0_SIZE;
> +    return vgic->rdist_regions[i].size;
>   }

I think this change in vgic.h should have belong to the patch 
introducing the helpers (if we still plan to use them).

>   #else
>   static inline unsigned int vgic_rdist_nr(struct vgic_dist *vgic)
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:04:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:04:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193697.345037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMWW-0003CX-B0; Thu, 23 Sep 2021 11:04:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193697.345037; Thu, 23 Sep 2021 11:04:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMWW-0003CQ-7y; Thu, 23 Sep 2021 11:04:16 +0000
Received: by outflank-mailman (input) for mailman id 193697;
 Thu, 23 Sep 2021 11:04:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZpLY=ON=anisinha.ca=ani@srs-us1.protection.inumbo.net>)
 id 1mTMWV-0003CK-0Z
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:04:15 +0000
Received: from mail-pl1-x635.google.com (unknown [2607:f8b0:4864:20::635])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0c47e261-347c-4037-9779-92573c3ef51e;
 Thu, 23 Sep 2021 11:04:13 +0000 (UTC)
Received: by mail-pl1-x635.google.com with SMTP id w11so3728624plz.13
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 04:04:13 -0700 (PDT)
Received: from anisinha-lenovo ([115.96.156.170])
 by smtp.googlemail.com with ESMTPSA id 126sm6890764pgi.86.2021.09.23.04.04.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 04:04:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c47e261-347c-4037-9779-92573c3ef51e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=anisinha-ca.20210112.gappssmtp.com; s=20210112;
        h=from:date:to:cc:subject:in-reply-to:message-id:references
         :user-agent:mime-version;
        bh=1RxHU6ohg85S859gdTN8SmlrMapEypB3sVeaylhNzKc=;
        b=xp676m60NgY6Pdk+CpLb9v69Dv7cEYtiwmfwoFGZ7BHxLWSfF5ZC3a2ws4uSjGuOPf
         4TZzaTD+knuMaYTye3KpEj338NPh5Ff76gGjAjREmiiyTFOoTaBAbUb8MkDxQH90wkYg
         xlFeVbThhDtvCrKK+wJ+Dy+E2elCJ6oW1f75VWa0LVveVn6aP92rVU4ar8U4sIi1N2a/
         ymrDK13CEBTLq5G7O4r16iQe5Wgb97+pgZpswQaZjm2PZb6URvfdknz0qcX2ENrqjP8T
         amsF0+DqaiUPp74kE1Nku+dzo/fKWirhwT9B3cjbaPY7tNoqTusSwBX+bu8idJDfmvWd
         sJjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:date:to:cc:subject:in-reply-to:message-id
         :references:user-agent:mime-version;
        bh=1RxHU6ohg85S859gdTN8SmlrMapEypB3sVeaylhNzKc=;
        b=RqLgoHUk9Hc18eNA+NCtS8RLn83Oepo2N34HOqT/s9/4SXbRDqh9Z1COI3CXYDRz4u
         AY7rr3rhpGrvsFQsGTzE1GkmBbTJYvEKaEm7T3volUhfXTd3HhJQ3kjPCTyZrA58WTLj
         TrNp5Wp0fcAHnfclb6kN7mR3/vQvVtnjXqxxx1cdIMMZR6amPdheX9TxaB+zCeGCHYsw
         56cTqrsr0WRkRlCwGwSdp41OllDNCURu8JXleB94Dm2IqeiXpWykYSi11JPi8rDKLdWi
         QoNkZ3dueyi8Ecr2+4H0bR5jwDP6wF/TNG0kERqzSDR+RQnQNK1j0ycV8Cxe10XXoPDt
         tYyQ==
X-Gm-Message-State: AOAM532FPzRCIs5pPEza56zaJoAr23OAoujlF5GRF6p7tsruXNzqhATm
	izjBdvZ61CCv9Wcob4aYsJPkkA==
X-Google-Smtp-Source: ABdhPJxBS6AGfLPmzLRaQHRFmVjDVoRSyb6vHulAN4aGINKZzpCa+OGlRLA+YQyj5ZUQtE0f9I/NeA==
X-Received: by 2002:a17:90b:4a4c:: with SMTP id lb12mr16930916pjb.55.1632395052788;
        Thu, 23 Sep 2021 04:04:12 -0700 (PDT)
From: Ani Sinha <ani@anisinha.ca>
X-Google-Original-From: Ani Sinha <anisinha@anisinha.ca>
Date: Thu, 23 Sep 2021 16:34:01 +0530 (IST)
X-X-Sender: anisinha@anisinha-lenovo
To: Damien Hedde <damien.hedde@greensocs.com>
cc: qemu-devel@nongnu.org, Alistair Francis <Alistair.Francis@wdc.com>, 
    =?ISO-8859-15?Q?Marc-Andr=E9_Lureau?= <marcandre.lureau@redhat.com>, 
    Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, 
    Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, 
    =?ISO-8859-15?Q?Daniel_P=2E_Berrang=E9?= <berrange@redhat.com>, 
    "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, 
    Ani Sinha <ani@anisinha.ca>, Gerd Hoffmann <kraxel@redhat.com>, 
    Eric Auger <eric.auger@redhat.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, 
    "Dr. David Alan Gilbert" <dgilbert@redhat.com>, 
    Markus Armbruster <armbru@redhat.com>, Eric Blake <eblake@redhat.com>, 
    qemu-riscv@nongnu.org, xen-devel@lists.xenproject.org, 
    mark.burton@greensocs.com, mirela.grujic@greensocs.com, edgari@xilinx.com, 
    Peter Maydell <peter.maydell@linaro.org>
Subject: Re: [RFC PATCH v2 10/16] qdev-monitor: allow adding any sysbus device
 before machine is ready
In-Reply-To: <20210922161405.140018-11-damien.hedde@greensocs.com>
Message-ID: <alpine.DEB.2.22.394.2109231628280.630@anisinha-lenovo>
References: <20210922161405.140018-1-damien.hedde@greensocs.com> <20210922161405.140018-11-damien.hedde@greensocs.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII



On Wed, 22 Sep 2021, Damien Hedde wrote:

> Skip the sysbus device type per-machine allow-list check before the
> MACHINE_INIT_PHASE_READY phase.
>
> This patch permits adding any sysbus device (it still needs to be
> user_creatable) when using the -preconfig experimental option.
>
> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> ---
>
> This commit is RFC. Depending on the condition to allow a device
> to be added, it may change.
> ---
>  softmmu/qdev-monitor.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
> index f1c9242855..73b991adda 100644
> --- a/softmmu/qdev-monitor.c
> +++ b/softmmu/qdev-monitor.c
> @@ -269,8 +269,13 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
>          return NULL;
>      }
>
> -    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
> -        /* sysbus devices need to be allowed by the machine */
> +    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE) &&
> +        phase_check(MACHINE_INIT_PHASE_READY)) {
> +        /*
> +         * Sysbus devices need to be allowed by the machine.
> +         * We only check that after the machine is ready in order to let
> +         * us add any user_creatable sysbus device during machine creation.
> +         */
>          MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
>          if (!machine_class_is_dynamic_sysbus_dev_allowed(mc, *driver)) {
>              error_setg(errp, "'%s' is not an allowed pluggable sysbus device "

Since now you are adding the state of the machine creation in the
valiation condition, the failure error message becomes misleading.
Better to do this I think :

if (object class is TYPE_SYS_BUS_DEVICE)
{
  if (!phase_check(MACHINE_INIT_PHASE_READY))
    {
      // error out here saying the state of the machine creation is too
early
    }

    // do the other check on dynamic sysbus

}



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:08:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:08:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193704.345048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMas-0004AG-2Z; Thu, 23 Sep 2021 11:08:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193704.345048; Thu, 23 Sep 2021 11:08:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMar-0004A9-Tg; Thu, 23 Sep 2021 11:08:45 +0000
Received: by outflank-mailman (input) for mailman id 193704;
 Thu, 23 Sep 2021 11:08:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTMaq-0004A3-7I
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:08:44 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 94471309-08c8-470a-b0f8-0990909f649c;
 Thu, 23 Sep 2021 11:08:42 +0000 (UTC)
Received: from [192.168.15.189] (unknown [195.68.53.70])
 by beetle.greensocs.com (Postfix) with ESMTPSA id E523C20777;
 Thu, 23 Sep 2021 11:08:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94471309-08c8-470a-b0f8-0990909f649c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632395320;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pKGcNnmlj8pflXWMttiwz2KP8cVDOcviRK9BnXLJgmg=;
	b=Yzw8VOoeXtchb31M5o7ihWMzh115d5LEkUD/IjkknpVcRv+Wpiik2gMc7ix0A9ZGEFXtUd
	BNML4R9Kt/vkAR74LOFxs0dragUKbR0yYkvXfUnQ3YLvVm937WLOKlcaHHrQZDXvPolsfg
	qZfCuKoudgXjuISaTcUD5bKxvcPzjAk=
Message-ID: <d080506d-04a0-0af1-5426-6162091ef55f@greensocs.com>
Date: Thu, 23 Sep 2021 13:08:38 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 00/16] Initial support for machine creation via QMP
Content-Language: en-US-large
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>,
 qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 Peter Xu <peterx@redhat.com>, mirela.grujic@greensocs.com,
 Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Ani Sinha <ani@anisinha.ca>,
 Eric Blake <eblake@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <a4debda9-f00b-dc99-7e7c-37ea9b4d7b5c@redhat.com>
From: Damien Hedde <damien.hedde@greensocs.com>
In-Reply-To: <a4debda9-f00b-dc99-7e7c-37ea9b4d7b5c@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 9/22/21 19:50, Philippe Mathieu-Daudé wrote:
> Hi Damien,
> 
> On 9/22/21 18:13, Damien Hedde wrote:
>>
>> The goal of this work is to bring dynamic machine creation to QEMU:
>> we want to setup a machine without compiling a specific machine C
>> code. It would ease supporting highly configurable platforms (for
>> example resulting from an automated design flow). The requirements
>> for such configuration include begin able to specify the number of
>> cores, available peripherals, emmory mapping, IRQ mapping, etc.
>>
>> This series focuses on the first step: populating a machine with
>> devices during its creation. We propose patches to support this
>> using QMP commands. This is a working set of patches and improves
>> over the earlier rfc (posted in May):
>> https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg03706.html
> 
> Do you have a roadmap for the following steps? Or are you done with
> this series?

Hi Philippe,

We would like to stick to this scope ("creating devices in a machine") 
for this particular series otherwise it will become very big and cover a 
huge scope.

We have some patches to "migrate" more devices to be early 
user-creatable (like the last 2 patches of this series) so that we can 
use more device when building a machine. But these are trivial and only 
depends on what is the condition to allow this. We plan to submit these 
when this series is done.

We plan to address other issues we have in others series of patches. We 
do not put a roadmap somewhere. But we can details this in a page in our 
github or in the qemu wiki if you think this is a good idea.
Here are the main remaining issues:
+ the base machine (we are using "none" here because it is "almost" 
empty and fit our needs but it has some limitations)
+ adding cpus
+ non-trivial memory mappings
+ solving backend (eg: net) connection issues

> 
> Yesterday I was thinking about this, and one thing I was wondering is
> if it would be possible to have DeviceClass and MachineClass implement
> a populate_fdt() handler, to automatically generate custom DTB for these
> custom machines.
> 
> Maybe in your case you don't need that, as your framework generating the
> QEMU machine also generates the DTB, or even parse a DTB to generate the
> machine... :)

You are right, we do not need this. We indeed use a device tree file to 
describe the platform but this is an "extended" device tree (it has more 
info than needed for linux). If it was not the case, I think it would 
still be easier to generate it from the source platform description than 
using qemu as an intermediate.

It is probably possible but it is tricky to generate the dtb: mapping in 
dtb are not standardized and really depends on the node types.

For example, to generate the interrupt-map property of a device node. 
You need to know the interrupt mapping (which interrupt line goes in 
which interrupt controller) and also have info about the interrupt 
controller's cells format (eg: how many bytes do we need to specify the 
interrupt). For example for some controller, you have specify the 
interrupt kind (rising or falling edge, high or low active level).

So you'll probably need some special "get_dtb_interrupt_cell" method in 
interrupt controllers to generate these entries for you so that a device 
can ask dtb data to its controller.

Bus mappings also depend on the bus type, but since qemu devices are 
already organized on a bus-type basis, this is probably easier to solve.

You'll have similar issues with every mapping. But bus and interrupt are 
the most important ones.

Thanks,
Damien


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:10:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:10:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193709.345058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMcr-0005Td-CE; Thu, 23 Sep 2021 11:10:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193709.345058; Thu, 23 Sep 2021 11:10:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMcr-0005TW-9Q; Thu, 23 Sep 2021 11:10:49 +0000
Received: by outflank-mailman (input) for mailman id 193709;
 Thu, 23 Sep 2021 11:10:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zMzq=ON=dingwall.me.uk=james@srs-us1.protection.inumbo.net>)
 id 1mTMcq-0005TQ-0S
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:10:48 +0000
Received: from smarthost01a.sbp.mail.zen.net.uk (unknown [212.23.1.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c9dac02b-4d0e-48ad-b64c-fcfd1cedb045;
 Thu, 23 Sep 2021 11:10:46 +0000 (UTC)
Received: from [217.155.64.189] (helo=mail0.xen.dingwall.me.uk)
 by smarthost01a.sbp.mail.zen.net.uk with esmtpsa
 (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1)
 (envelope-from <james@dingwall.me.uk>) id 1mTMcn-0002Jj-2O
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:10:45 +0000
Received: from localhost (localhost [IPv6:::1])
 by mail0.xen.dingwall.me.uk (Postfix) with ESMTP id 67DDF172026
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 12:10:44 +0100 (BST)
Received: from mail0.xen.dingwall.me.uk ([IPv6:::1])
 by localhost (mail0.xen.dingwall.me.uk [IPv6:::1]) (amavisd-new, port 10024)
 with ESMTP id I78hNExp92bf for <xen-devel@lists.xenproject.org>;
 Thu, 23 Sep 2021 12:10:44 +0100 (BST)
Received: from ghoul.dingwall.me.uk (ghoul.dingwall.me.uk
 [IPv6:2a02:8010:698e:302::c0a8:1c8])
 by dingwall.me.uk (Postfix) with ESMTP id 425B9172023
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 12:10:44 +0100 (BST)
Received: by ghoul.dingwall.me.uk (Postfix, from userid 1000)
 id 2D41C816; Thu, 23 Sep 2021 12:10:44 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9dac02b-4d0e-48ad-b64c-fcfd1cedb045
X-Virus-Scanned: Debian amavisd-new at dingwall.me.uk
Date: Thu, 23 Sep 2021 12:10:44 +0100
From: James Dingwall <james-xen@dingwall.me.uk>
To: xen-devel@lists.xenproject.org
Subject: domain never exits after using 'xl save'
Message-ID: <20210923111044.GA3849950@dingwall.me.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Originating-smarthost01a-IP: [217.155.64.189]
Feedback-ID: 217.155.64.189

Hi,

This is an issue that was observed on 4.11.3 but I have reproduced on 4.14.3.
After using the `xl save` command the associated `xl create` process exits
which later results in the domain not being cleaned up when the guest is
shutdown.

e.g.:

# xl list -v | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
guest01  15  2048     3     -b----    1555.9 d13cc54d-dcb8-4337-9dfe-3b04f671b16a        - system_u:system_r:migrate_domU_t

# ps -ef | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
root     18694     1  0 Sep22 ?        00:00:00 /usr/sbin/xl create -p /etc/xen/config/d13cc54d-dcb8-4337-9dfe-3b04f671b16a.cfg

# xl save -p guest01 /vmsave/guest01.mem
Saving to /vmsave/guest01.mem new xl format (info 0x3/0x0/2900)
xc: info: Saving domain 15, type x86 HVM
xc: Frames: 1044480/1044480  100%
xc: End of stream: 0/0    0%

# xl list -v | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
guest01  15  2048     3     --p---    1558.3 d13cc54d-dcb8-4337-9dfe-3b04f671b16a        - system_u:system_r:migrate_domU_t

# ps -ef | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
- no matches -

# xl unpause guest01

# xl list -v | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
guest01  15  2048     3     -b----    1559.0 d13cc54d-dcb8-4337-9dfe-3b04f671b16a        - system_u:system_r:migrate_domU_t

# xl shutdown guest01

# xl list -v | grep d13cc54d-dcb8-4337-9dfe-3b04f671b16
guest01  15  2048     3     ---s--    1575.8 d13cc54d-dcb8-4337-9dfe-3b04f671b16a        0 system_u:system_r:migrate_domU_t


What we would expect is that the `xl create` process remains running so that
when the domain is later shutdown then it gets cleaned up without having to
manually `xl destroy`.

tools/xl/xl_vmcontrol.c handle_domain_death() has (0 == DOMAIN_RESTART_NONE in xl.h)

    case LIBXL_SHUTDOWN_REASON_SUSPEND:
        LOG("Domain has suspended.");
        return 0;

The while(1) loop of create_domain() has a switch statement which handles this
return value with:

            case DOMAIN_RESTART_NONE:
                LOG("Done. Exiting now");
                libxl_event_free(ctx, event);
                ret = 0;
                goto out;


Is this the expected behaviour?  Would an approach to getting the behaviour we
want be to change the return value from handle_domain_death() to one which
doesn't trigger the exit?

Thanks,
James


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:10:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:10:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193710.345070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMcu-0005kg-Ku; Thu, 23 Sep 2021 11:10:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193710.345070; Thu, 23 Sep 2021 11:10:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMcu-0005kZ-HN; Thu, 23 Sep 2021 11:10:52 +0000
Received: by outflank-mailman (input) for mailman id 193710;
 Thu, 23 Sep 2021 11:10:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTMct-0005jh-2R
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:10:51 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e89bb3c8-1c5e-11ec-ba2a-12813bfff9fa;
 Thu, 23 Sep 2021 11:10:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e89bb3c8-1c5e-11ec-ba2a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632395449;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=Rjvg1g1MiNfMPHB5E9DE3X/pJjj2vLy6IeghbUb9V6U=;
  b=C0jhYbxJ6GoD1lZYjh6+ZTwBNx61fZfLzL1EuGBSpnIRZf2RV+kiwzcJ
   NM9fFL4NDZX/9Cmy8l/ptRDdGriiK67y2IIWQ9CgL8L0GN/r/F74EYAe+
   eWNUSMC9dtflzzg4UkJn73HALGf4tt2iWAK+86/2NucwJWiLxEu5/F7gz
   M=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: irpx3oFTrN59n0nP8PpNYti4d7IxnlLCj96BRyCPIncQq/fnsmuLZjqqnC4bGCJ3hM+G8O9gWV
 S484bQQKEDc9uOEgalGBxtDqtUfArBmWFKsqpUBNbldq/7hfMY7zU32lQjmp6uN4GBIYvohRpp
 d/kidudfv4fp7XBRyRoHF+UdNSNbvLgrl53aEjmPRQkaH7Lgdp9I5cN8YIifhy4KFiNeZL/0Ke
 RJP/5Jp4xlXCqD6mNch+LTpihYK8sfM+96csly2/dkJvzj19vSj+YHL1gwrhZJNxRdRA2XnWs8
 GErOWQA6LQH4hyPFYnYYg+35
X-SBRS: 5.1
X-MesageID: 53021683
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:I4BrWau4KzKotsrxkMir6Z6dc+fnVIhZMUV32f8akzHdYApBsoF/q
 tZmKT+PM6zYNGKged0gPI20o0NSscTUn4A1HFM9qi81RHwT+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524PhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npl6KaQaSIiAfP2lLpMDkZ7Aw9kHfNX5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sWR6+AN
 5FxhTxHK0vnWg1rCFsrULEC29jyn1vhYyR2twfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krW8mK8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECqF+Zy3EXCRYWfUCmuvT/gUm7M++zM
 GRNpHBo9/JrshX2EJ+tBHVUvUJooDZBYfpwNuYFtjuBlPuJvCWJKHIkEC9ePYlOWNANeRQm0
 VqAntXMDDNpsaGIRX/1yop4vQ9eKgBOcjRfP3FsoR8tpoC5+dBu0kunosNLTfbt5uAZDw0c1
 NxjQMIWvLwVkcdD/KGy51mvb9mE98WRE1ZdCuk6WAuYAuJFiGyNO9fABbvzt68owGOlor+p5
 iNsdy+2trxmMH11vHbRKNjh5Znwjxp/DBXSgER0A74q/Cm39niocOh4uW8lfhwyap5fJ2W4M
 Sc/XD+9ArcJZxNGioctP+qM5zkCl/C8RbwJqNiOBjaxXnSBXFDep3w/DaJh92vsjFItgckC1
 WSzK66R4YIhIf0/llKeHr5FuZdyn3xW7T6DFPjTkkX8uZLDNSH9dFvwGAbXBgzPxPjf+1u9H
 hc2H5bi9iizp8WkOXSIrd5PcwpaRZX5bLivw/Fqmie4ClMOMEkqCuPLwKNnfIpgnq9PkfzP8
 G37UUhdoGcTT1WeQelTQnw8Or7pQ7hlqnc3YX4lMVqygiBxaoez9qYPMZAweOB/puBkyPd1S
 dgDetmBXasTGmiWpWxFYMmvtpFmeTSqmRmKY3ivbg8gcsMyXAfO4NLlIFfirXFcEiqtuMIii
 LS8zQeHE4EbTgFvAZ+OOvKixl+8p1YHn+d2UxeaK9VfYhy0ooNrNzbwnrk8JMRVcUfPwT6T1
 gC3BxYEpLaS/99poYeR3a3d9tWnCepzGEZeDlL317fuOHmI5HenzK9BTP2MIWLXWlTr9fjwf
 u5S1fz9bqEKxQ4Yr4pmHr935qsi/N+z9aRCxwFpEXiXPVSmDrRsfiuP0cVV7/Afw7ZYvU29W
 16V+8kcMrKMYZu3HFkULQsjT+KCyfBLxWWCsaVreB33tH1t4b6KcUROJB3d2iVSIYx8PJ4h3
 ep86tUd7Bayi0ZyP9uL5syOG79g8pDUv30bi6wn
IronPort-HdrOrdr: A9a23:aVw5kK6/mCiJ95ndjwPXwVOBI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwX5VoZUmsj6KdgLNhRotKOTOJhILGFvAB0WKP+UyEJ8S6zJ8h6U
 4CSdkBNDSTNykCsS+S2mDReLxBsbr3gZxAx92ut0uFJTsaFJ2IhD0JbDpzfHcGIDWvUvECZe
 ahD4d81nCdUEVSSv7+KmgOXuDFqdGOvJX6YSQeDxpizAWVlzun5JPzDhDdh34lIn9y6IZn1V
 KAvx3y562lvf3+4hjA11XL55ATvNf60NNMCOGFl8BQADTxjQSDYphnRtS5zX0IidDqzGxvvM
 jHoh8mMcg2w3TNflutqR+o4AXk2CZG0Q6q9XaoxV/Y5eDpTjMzDMRMwahDdAHC1kYmtNZglI
 pWwmOwrfNsfFL9tRW4w+KNewBhl0Kyr3Znu/UUlWZjXYwXb6IUhZAD/XlSDIwLEEvBmc4a+d
 FVfYLhDcttABGnhyizhBgr/DXsZAV9Iv6+eDlDhiTPuAIm2EyQzCMjtboidzk7hdUAozQt3Z
 WwDk1SrsA8ciYhV9MLOA4we7rGNoXze2O/DIuzGyWuKEhVAQOHl3bIiI9FkN1CPqZ4iqcPpA
 ==
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53021683"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YzuGAKM2V1GV2HMOyck7wMXmtEaVrh1A8r6Wj5H26kJs7lrO/+Lp8fk46Fc0GyrC7sZVoSHGOZpplWZ2Muqa6l7G9G/CDGH8t5e6UC/l5ONSoDyBsM1eS+jP2TWgJM40dM75mY1AX+fLZwDYSGObEBt+8X8RyW1Q9jifTVdYWfpXOC40aS4smb3xFJKc1FmBU8P8/aVLDktGM6XrIkGecUMOVmZzu5GTfUD5xy37Ylc/sFRx1doFfKzi5Ve6wT5aWyOZJgMIGcFd1lR0gwv5ao5CGIHzkpO2R9cVzjr6npG4OPoFUtHoR3PX8e2YcQyhHs4j2pQm7eurXrCgzZPP0g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QCMVIrccNaP+l9SSMJIa+u7dt+224mLkUJX64Qpo0J4=;
 b=MB9fzhiQUoOGM84T/Yh8Dv2+S3J9XD2A2bYZaEntVXeu6kuchELYmCGyNlSQCOpq72epZyjIaBKy+XN7SjaR9iOEsxlQxTkJMCavg3j/gT4C4wFEkEZ3D7kC63X7Gb7LFzcKHGXKYQch+8hzbM6bPhcRAS94/uBa+w0cMBom1wkeicYgqtvtGArbrx+jUh8mXNpGaWjR3V+wkAcfqvlwKiVBWkbeVlHXsL1fZ25zCIpNgDIr2s+S2e56NuNDMkwJxKqXDnjLkzuHRz+MONBtOa+DmoWVZ08cJ+JvMg8tpbuZv1Sa/ipOo3EV9JZ7L9PcVmXFeu+W/Uj7L6JUQki0pg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QCMVIrccNaP+l9SSMJIa+u7dt+224mLkUJX64Qpo0J4=;
 b=a5/2m2gbDxEbU0H/45HfqBdDLh1zc/f+NWtI7FSXRIf72Rd3jjgiJxDs9CPoSUVd5k5q/0IJq9P3NXiA9jGkx7WONMAmdc2tNDIyZAti4fpJZxk/hD5JjveDXkFeOvTE71Jlsonah1fn+ukrINKIIIKGxFcDQH1QjRpN6SGpYfw=
Date: Thu, 23 Sep 2021 13:10:40 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
Message-ID: <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
X-ClientProxiedBy: LO4P265CA0009.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2ad::12) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9fd14a30-d877-449a-7fcc-08d97e82c9b0
X-MS-TrafficTypeDiagnostic: SN6PR03MB3822:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB38224CD17663E7E1D7CBD85A8FA39@SN6PR03MB3822.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: jTPlX5fp6zVgn2Po/KFzYZIXlkhQe5Q6tnT+o4AFBo+eo4L7gj4b6YbGF/loLNYEFf9TMPfM+rtalzTbfw7BUVEe+7wjqHtsuqmOqnqrIzvNdz6kk5ufUmP2P5KpU0HS1zh48GFiD+K790olsZ13eV3U0Pj3ThcSNs72YqvmhyC8j9ShPqje2Mkl346ub/m7fFfsfb0priveC6B9Fk+OpWUPe0LomhyG9LknVzh3XA7H3Sg02KogOaFDHhMs2tIoiHLuWF1Dyc5SEf9vtmIUOjaFtInt4CKgguXNtIei72Peq6LmugR03CViAtHRpxprQB75NTjiMOOUh1iAuAEhl2hc3fzp74rUVSU+3QIyLmW6VLiE6b7mJXvGp7qkzYgzLiPN+hQQJ8hjC6bA7xbiz1rfvzpEPUi+c7aPxxpKNkkFCXa1HCQJHo6f3myzhT/XMf+JVSGTDS/sytQbvc8JDAa5EBacww9uby7csMT5Ne3Oho+Lk1SJGHeaZJhUBIL1jX4tPeoHtJsbX6OztIROxwB80cZDLPXuxDfy/UDtgvskGgYkT0Sal/Wx+1d558TuGleiBwjwwuIz0oJo9VO4SljOzNcg/JnpT2iJJXn7FTnHZ44SGmsxaYWM2Hs3PnjCru5lx5fAk6vMYWfyzbpLBQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(54906003)(66946007)(8676002)(85182001)(83380400001)(66556008)(6916009)(4326008)(26005)(6486002)(956004)(5660300002)(66476007)(8936002)(9686003)(2906002)(508600001)(38100700002)(6666004)(107886003)(6496006)(316002)(186003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?anUrcWM5S2lNaXQxTWNEU2l4N2NjYUt1R21IbTE2VDhaVWJIbmxLUHJ4SS9t?=
 =?utf-8?B?MWNZNjdYM3czSWd3cG9ySW0zMTVMYTFkQ20yVjF3b05XQ2ZVWFE1WVZzZGlE?=
 =?utf-8?B?VXZRNzJaeVBJdFVBaFVMcDlScWIwb2hUeGdmMGJLY1FUaWtUSkNad3JoVkxy?=
 =?utf-8?B?WE40RXg2ak1pVWJtTTlvME04aCtjaFR3YWVlYldtS1ZVZ0ZRS01ZSWxkeGNw?=
 =?utf-8?B?ak5RSVpzTndydHUxVmUyVnM4QXJKVEFGbi91cHl1Zlg5cUlNRFBLY2FYKzBS?=
 =?utf-8?B?TWc0bSt3c2dSK0pZSDhJdUwvUWY3VGM2citkOHRZU1Q1cmJPNkJpRVkvTWg0?=
 =?utf-8?B?WUlhUjl5SDJlYWpsVlVmYlNvUDhXTXM1QzhJa3FwT0FpMTdJRTdNVTdXYU9U?=
 =?utf-8?B?M01ueFFlRHlaUUdOeWtEQllBdnpoWmtGTW0vRW13SjVKaVFNWVRHVW9hb2JW?=
 =?utf-8?B?RXBVQWQzcnFkRVVHWDFJaUU0UGVZUTRZY3JQTFhkTFhBSmtGN1VEL05iOWkv?=
 =?utf-8?B?WHRMSW1GNjg4Tjl5SCtpRnIxV1FPV2Rra3FUYVpLaUVtNHlwV3ovYVVsb2tM?=
 =?utf-8?B?eWtXeXpqU29ZUzhyVXU1VXQ0NHBGc3BKWkhKN1pycjllSnZCRHU2ZVY2OXho?=
 =?utf-8?B?d21EOWRtT2czWWlETUxlYTZCTnB6VytDckIzVExsbGUyUExwN1dDSVlCTkdH?=
 =?utf-8?B?ckZQdEJna2lMSUE2blJnbEZaaFlWOXZpOVp3c09mN0ZsRDJoOXRyZGhnY1VV?=
 =?utf-8?B?OHh2ZG41K0c2TU5DNzM5NHJWOS84WTl5SzkxZ0RVL20xcDNwbzNsR3lGc3Qz?=
 =?utf-8?B?R3pPVG1SZDhLTnpYRlVFTEJkWmMzYVNXVUhqOVpuUWtRMU4xUENwNFN5K0xQ?=
 =?utf-8?B?MW5nQ2didTV5RmRQVm5iaG80RlJ3RHY3ZnJhV1N4aG43ZUJzbkRUQjdYd0ly?=
 =?utf-8?B?aFdsMjBlajFON1I0cHV5VkRjUUU1UXFWL3h2SkYrR0tCSWNLQVBndW13OE5R?=
 =?utf-8?B?dWt0eVZPZUFrWDFGbW5HV0ZzMHM5WUFKVG1SYmRJOFI1WUJjUkdQd29PcFFD?=
 =?utf-8?B?WU8xNzhDK1QySkxEYW9CQ2dKcUlBR1A0R3E5T29vb2NYc2F3UEF0TWRoalp6?=
 =?utf-8?B?ck5KRTBYeElINmhHbDZJeWR5dUowdGErZU5WOFRRcUFFMnY1NnlDSVRrc2I5?=
 =?utf-8?B?eGFEdjBSaENvWkVpTU9qTzVHK0YxZEFsY0tsVEZleVBQR3lMUGRiS1R0dHlP?=
 =?utf-8?B?Ti9jQlNCWTZObnQrWEE4ZmhpVG9XeTZzejlRbEhnRFJQT2dxSzBibW1FemtT?=
 =?utf-8?B?OUl2c0tvSkRMQnRhcWE2ZHlxOE9jdnd4U0xZSVZnbUdRRGlEVnU3U1EvNGVn?=
 =?utf-8?B?S1dtcmdUMGx6WlExNTdyS1pIZENnRFRVYnJuSmp3UjFyYzNPY0RVOERvTk1F?=
 =?utf-8?B?QW9pb2g5eDAwYmdnb2trSHZ1amZVdjgvb0R0QjdCMzM1cmNNZS9aUUFPSzI3?=
 =?utf-8?B?U2lLNEg0ZHByZ1pTYWNKeEFOZGZFNGREUm8rblRFU3cwT2sxWEVTWnYyNEFS?=
 =?utf-8?B?VXI3RHJNSWtJNEE5MzJ6VDdyZXo3TzJzVlJLVDNMSUZ1dGVxYXN2MExOd2Fw?=
 =?utf-8?B?d3Y0QVRDZ3RPUzJFZzhmT3lnY052Z2htdHVJdnBsejZzOStTVUhpemR2azBY?=
 =?utf-8?B?QmhrOEZaaS94Y05XejdoRXZmZmFQQ2xYNldDbjgzVUgrM0NYckt6OFhwOC82?=
 =?utf-8?Q?U/NARZ6IMTOChE53vjG5btoRPUzDVWkTw4SE/cS?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9fd14a30-d877-449a-7fcc-08d97e82c9b0
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:10:44.6750
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MYHuGEWZY/AaOPG49ey2VXDnXcj7ybqH867BRZSs7toKIXK2EkQ4M7341ABqtgZxWck7HwPiRjtCtdGkqTYbMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB3822
X-OriginatorOrg: citrix.com

On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
> To become independent of the sequence of mapping operations, permit
> "access" to accumulate for Dom0, noting that there's not going to be an
> introspection agent for it which this might interfere with. While e.g.
> ideally only ROM regions would get mapped with X set, getting there is
> quite a bit of work. Plus the use of p2m_access_* here is abusive in the
> first place.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v3: Move last in series, for being controversial.
> v2: Split off from original patch. Accumulate all of R, W, and X.
> 
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
>              return -EPERM;
>          }
>  
> +        /*
> +         * Gross bodge, to go away again rather sooner than later:
> +         *
> +         * For MMIO allow access permissions to accumulate, but only for Dom0.
> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
> +         * the way they specify "access", this will allow the ultimate result
> +         * to be independent of the sequence of operations.

Wouldn't it be better to 'fix' those operations so that they can work
together?

It's my understanding that set_identity_p2m_entry is the one that has
strong requirements regarding the access permissions, as on AMD ACPI
tables can specify how should regions be mapped.

A possible solution might be to make set_mmio_p2m_entry more tolerant
to how present mappings are handled. For once that function doesn't
let callers specify access permissions, so I would consider that if a
mapping is present on the gfn and it already points to the requested
mfn no error should be returned to the caller. At the end the 'default
access' for that gfn -> mfn relation is the one established by
set_identity_p2m_entry and shouldn't be subject to the p2m default
access.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:11:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:11:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193712.345081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMd2-00068q-US; Thu, 23 Sep 2021 11:11:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193712.345081; Thu, 23 Sep 2021 11:11:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMd2-00068j-Q7; Thu, 23 Sep 2021 11:11:00 +0000
Received: by outflank-mailman (input) for mailman id 193712;
 Thu, 23 Sep 2021 11:10:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTMd0-00066O-VO
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:10:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ed32efe6-1c5e-11ec-ba2a-12813bfff9fa;
 Thu, 23 Sep 2021 11:10:58 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2054.outbound.protection.outlook.com [104.47.10.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-_6HQlhhEODys7yKR7h9Evg-2; Thu, 23 Sep 2021 13:10:56 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:10:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 11:10:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0008.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Thu, 23 Sep 2021 11:10:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed32efe6-1c5e-11ec-ba2a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632395457;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=5n8E4qDNOXyPRZiTn4zsnzmSBAjyQQDHUXlXehU+4/E=;
	b=I1MlNhdguwdpR7/beWYOCX/SeWd4lee3+jyskoNnNYD/7sKAh1045Zb2QkvkqqQAwEIdDU
	XaadVzncmFU//9kGcLkAtDnp8mJocUlklFty5tWLAO7Dvk4SGM+/qgQGkEkFS0EWbAUmY3
	GLrK1COCvEzKlTYBH3EooTYDdRnr8Nc=
X-MC-Unique: _6HQlhhEODys7yKR7h9Evg-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Aj2dgfk1cZqcAYpsJ3ql+31AmQTVIY40+Kt4ht+l1WKq54rZKOu6uGXPHZgXuiQw7WG+3G9kitx6FGRoo1Th5SbdEASxOAXgi4iVykdLqPuA+IlE9GV00+P4gSbQCNjdmtcIFB3b2I1mZzUx1NroamMvLyMvMJuV3nEL8MtMhIUGM1Y3vR9NXP1ScnVmEjGwj1AeHM9CYzWUPxb1kg7M1JQ0AJNbt+L1A+0j194QfBq8KC6Y0LH3BDhsMruCLlhNJ3nuFMJo54abtHv/ZIwXqMaAcO0sEWDZ1b1DonfUmIBzIwzjGJ95XeFBIXaKJYs5g//4hbCqZHgbHewdmNoUMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5n8E4qDNOXyPRZiTn4zsnzmSBAjyQQDHUXlXehU+4/E=;
 b=b0ANod3y0yT1cVjQcXawCokCazyeuvRwJY69aMvkSjugSjREOlVJIgabXtWI9f9cxEn4RNlI4oP5pqQq/XG3rixqNH4YVE8zZuzISGA5KrSJcBydUiIBMuwW1PIkYZ3cUOYdmXDIWvuHA55lee1vOMdlvwedXEN7khHoZnUq24i5I2lkfNSvX5aFq1Kg6uNbxPkAblFLi1Qfj16iK/vWDYDDPcxY2kFpHhLGxhEiTBl93wk8Si6J2OjGqQURlqHquz73gdDGOb/nipucoC81G/HHuGEpMOaRU0w5sPjN4FM6nejBbF0KsKMQyUCV9f18BRfR6zkvkMAzbRn75oYpGA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org,
 sstabellini@kernel.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20210923095345.185489-1-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
Date: Thu, 23 Sep 2021 13:10:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923095345.185489-1-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0008.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 62403514-8f2a-4f56-2a16-08d97e82cdfb
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383C01842F656D72C8890ABB3A39@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sQPeRfzDZ0+qRqLtZTLy4yuM88tx9z5WtFdeqDKk3Sf6atduPIJXavj1N4iedtd37EQKcMZ2UdWQNUYQcrHixS+ZY+0kpZQ695htoIWUdiaDelcP+atjCjEu8tdecYnq6YWQWh7K5AMvtOmMgE7jKbb7aV7Xpn1bkzM1CByMukAJjomJcDXYvnuHgabcxht0qptM7sbVThpGsAPEu/KRlkzQv+9XF7vEADtg0F6k0iX8P8G1V1KJ+D//M57oxUBAANN5zMrzwEx7qEzC4R6DgGIaAhYMzK19asoL74+uwJHsWDQqVbh/SisGbxQty/Yhrj1ywDJbWFcr4ytgwHWvowtOv4hK2ZcU8i3YmVOHmsdN41d0GdV8nBKNH9+bpQcNuPRUdQATvruRvSHu1IJEKKSScdqAkxnCigfYRbQwW8b2To0TYh0JifMjGadkYlRdCHTVmjckxC692sTWY5Ye+tLYXTAkEbeU24V2WsUvVKwr+UBTK0XHUuzjtCIWfIVjOcduL8A+ZpzXg4hTjIcqiVctNv8fgmR+Y8fWgjH2I81hP3AcsUws8cN/98cB1ju+l7e3wsaYjz/7CErMldBdXpJ2c4w6pqAs1ITkQJUY5dWMfsU0Ng384bwNtsyL/B0gLeawdK1CfDTYmLSozv0zEd2daqf86E8PVMm5QsPQBhvJ7Yar9gprvt2krqZ/+kPvr9YBB7FYGhcrAej5Lv6PHgLUlEHbLOna4679YPZNv/Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(26005)(66556008)(956004)(186003)(8936002)(6916009)(6486002)(4326008)(316002)(86362001)(66476007)(2906002)(31696002)(2616005)(83380400001)(8676002)(53546011)(508600001)(31686004)(5660300002)(38100700002)(16576012)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?d1FpMVFzRmx1ZjZrMk5aSGFHaXE1cVI1TkFybHkybEhSQ1gvRER5TlFMeDFM?=
 =?utf-8?B?OFNzS3V0dUpYY3A1aURrTnRCRGpCZ3RsREJCN2d4UnM1NXFqTm1Ba3htTTZF?=
 =?utf-8?B?MWh3cThGY21WOW9FRXFtdGM5aFZOQ0V0QUNEU0NwMzVFTVNraDY5MUtpaStS?=
 =?utf-8?B?WmtLSHJJSUUyOW1PYTE2Wm8wa3JCTFlMR3dwRDlNd0tmVWpWTEUzZHVvNHdk?=
 =?utf-8?B?VTRYNmloNUMrMCtkeUJrOFlRYkdLN3NFZFNGMGFzSDByZXkvYmNQeHZjdTlt?=
 =?utf-8?B?WWF6UGlPSmhqVDl0TEZydEs0LzFrMlhzK25UWTdPU1NrQ0ZFZGU2RDh0MzRE?=
 =?utf-8?B?L0REbXNJYVpIK2Rpd2RmcUF5cFJjdXlzV3VMbmRERGswTTlpMTZmYVRlR1VG?=
 =?utf-8?B?TlpuYlI5N3N0TnNkOHc0RXhBRHlJeHBaa1BBMnhPWkRDV3JQcDM1VjNEc0pD?=
 =?utf-8?B?Vk9DMFFLbncwVURiSSsrY3E2NGU5OUlqVUVSdnlpcnN4UHVRb2xlL3ZTYzJF?=
 =?utf-8?B?cmFUTHNaeWZoeUZlbUxRejNCclV5Q0xVYVl0YjVPeFlGaTIxWUpqSnM5ZHlv?=
 =?utf-8?B?R1YvSUhnazdKRWhvL0p6MGw1YnJmWHFnOWV4Z0pSU2pWYS9BZmlTWXdxWDBG?=
 =?utf-8?B?Zm9uSldJQnRKWUlkKzd4TnhRaDhUd3dHWURwSUhrbkFqRlZZTXExdnRtNHhK?=
 =?utf-8?B?NS96SVkyWXQ1R0NoWEd4OTRvcHIxME15QVMrZU1pNjJGZzd6dGJQQS9NNCsv?=
 =?utf-8?B?a0hnYjJoejlWOC9oODlUaHJsUFQwRzhYMXpDdW9FQU9Ed3htejU5bUEwWFFE?=
 =?utf-8?B?UDVkV1RmbXc0MFB0R3NkMC9ETXUyckxuQnFLSzRCUS9vc25rQi93UGp2aWtM?=
 =?utf-8?B?NmJTSFRIcmNEWFVNQk11MGpvZUJ3MmxTWEVkZXdjVGxFMGFHQ3BubDJLVVRp?=
 =?utf-8?B?VFNRV3NHNkdYVEtRcFN6MHVmcm5xWUtDM2dpaEVUZkt1Y0FRQnFxZ3QrZ3gw?=
 =?utf-8?B?T25oMEZKckVwd2thYVV4TmpGS1FVc1FocXlTZ2pKbVNTNWRHYTRPL3Z0c0JE?=
 =?utf-8?B?OXRLZVBlenZPaXRqT1U1VVAvUXROM1FJbFlpY2w4NjJTMkZWaTJ5K0J5R1ZZ?=
 =?utf-8?B?bnptdktESitiSFBySnRUSDIxSHZCZDNnaEJqejRTYkdLWGcvWEVkOHFYa2Vm?=
 =?utf-8?B?bGs4d3FTcTFNYWc4SGw4N1hwM1JOcWFBTlovOGMwYVhKTjNZOUd0b2F0STBq?=
 =?utf-8?B?ZThxTSthVCsyYm5Ic3V4a0NuLzluYzVrTU8valROdFVMNEtPQWJVbkEvc3VQ?=
 =?utf-8?B?RGs0YkN5UHRxaTRrSGNQS1gvMDl3S0tvZXFZM1RHU2xNMlpPSkpsdE9XU3Jr?=
 =?utf-8?B?Nk9vQ0FtbjBPN2lxRjU3cjNwM1g4R2s1UHVMSWVaUU1uWW51T3dFK21rVitV?=
 =?utf-8?B?RWpTTXFzUjRsMys3d285ZGlKYUJPd0t6bEx3OHk0bjR1SXVQRzg1RXRsWWJM?=
 =?utf-8?B?VnByenhPTzAvVGxya3B2KzIxdUlvTTAvV3EzZUFFakQzcHc5NjI0UHVHMWRM?=
 =?utf-8?B?aC9BV1VDSHFLT1F2ZG9pc2VmTXdhK2Q5YUl6a3Z4ZWV3cExWSld5Y2xuZUpZ?=
 =?utf-8?B?V0tDSCtQSzhtYTE1SkZYbysvSk1uTW5MK1cwd28zKzlLTllGeGZGQmRPTG5R?=
 =?utf-8?B?YTVzNmVWZFExM2VOUEFqVFk5RTRQSEsvMVVMaiszcVd1c1ByK01JcDhDUFBK?=
 =?utf-8?Q?UAEjZSWeuc0TZR4mgFM1uV2jwzNGYfUYij+L3GG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 62403514-8f2a-4f56-2a16-08d97e82cdfb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:10:51.6943
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CyaRxDGeAFY5Oe4Ia2MB52FhyYLYzc1Dk+zU6JaONFxqQaAagLYxLVjkkFbw6cB+XbUPE3SNML+eJEhr63STww==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

On 23.09.2021 11:53, Oleksandr Andrushchenko wrote:
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -180,10 +180,34 @@ config SWIOTLB_XEN
>  	select DMA_OPS
>  	select SWIOTLB
>  
> +config XEN_PCI_STUB
> +	bool
> +
> +config XEN_PCIDEV_STUB
> +	tristate "Xen PCI-device stub driver"
> +	depends on PCI && !X86 && XEN
> +	depends on XEN_BACKEND
> +	select XEN_PCI_STUB
> +	default m
> +	help
> +	  The PCI device stub driver provides limited version of the PCI
> +	  device backend driver without para-virtualized support for guests.
> +	  If you select this to be a module, you will need to make sure no
> +	  other driver has bound to the device(s) you want to make visible to
> +	  other guests.
> +
> +	  The "hide" parameter (only applicable if backend driver is compiled
> +	  into the kernel) allows you to bind the PCI devices to this module
> +	  from the default device drivers. The argument is the list of PCI BDFs:
> +	  xen-pciback.hide=(03:00.0)(04:00.0)
> +
> +	  If in doubt, say m.
> +
>  config XEN_PCIDEV_BACKEND
>  	tristate "Xen PCI-device backend driver"
>  	depends on PCI && X86 && XEN
>  	depends on XEN_BACKEND
> +	select XEN_PCI_STUB

Does kconfig not at least warn about this? The selected item has a
"depends on !X88" conflicting with the "depends on X86" here.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:12:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:12:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193726.345091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMek-0007Kp-HR; Thu, 23 Sep 2021 11:12:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193726.345091; Thu, 23 Sep 2021 11:12:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMek-0007Ki-EY; Thu, 23 Sep 2021 11:12:46 +0000
Received: by outflank-mailman (input) for mailman id 193726;
 Thu, 23 Sep 2021 11:12:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTMej-0007Ka-Ap
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:12:45 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2cb3bf9d-1c5f-11ec-ba2a-12813bfff9fa;
 Thu, 23 Sep 2021 11:12:44 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9419E1FFA5;
 Thu, 23 Sep 2021 11:12:43 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4AFA313DCD;
 Thu, 23 Sep 2021 11:12:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id +cGmECthTGEpegAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 11:12:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cb3bf9d-1c5f-11ec-ba2a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632395563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=aEVPoxQen73SrDxSP+9kD2Kz53XX95G6TqvPawceJl0=;
	b=aMIMZ3p+NoahC+IwiLLRYLcRvVU9dvfBfQn/9wyv7qpmsc66wjyLirHMsuSYegGl79SY5s
	7X/s5hh43TI0cvshY647L+Bi+i46ymVBlv985tPGD2kdbgsGwZ9zuaiivCuSe2D8C4fmBp
	enCX7AFR4JnBcWeS7WwWHxXnsjhFLF0=
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
To: Jan Beulich <jbeulich@suse.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20210923095345.185489-1-andr2000@gmail.com>
 <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <478b9175-f21f-b77a-2bc1-ad230bbdf548@suse.com>
Date: Thu, 23 Sep 2021 13:12:42 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="2lcwPdRBsDCp1LdWI77itRlY1UcSO1FMi"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--2lcwPdRBsDCp1LdWI77itRlY1UcSO1FMi
Content-Type: multipart/mixed; boundary="RabaBlNU167INrkhGG7TimY4gvZhpBVDu";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Message-ID: <478b9175-f21f-b77a-2bc1-ad230bbdf548@suse.com>
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
References: <20210923095345.185489-1-andr2000@gmail.com>
 <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
In-Reply-To: <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>

--RabaBlNU167INrkhGG7TimY4gvZhpBVDu
Content-Type: multipart/mixed;
 boundary="------------A604B18524153844DF606DEF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A604B18524153844DF606DEF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 13:10, Jan Beulich wrote:
> On 23.09.2021 11:53, Oleksandr Andrushchenko wrote:
>> --- a/drivers/xen/Kconfig
>> +++ b/drivers/xen/Kconfig
>> @@ -180,10 +180,34 @@ config SWIOTLB_XEN
>>   	select DMA_OPS
>>   	select SWIOTLB
>>  =20
>> +config XEN_PCI_STUB
>> +	bool
>> +
>> +config XEN_PCIDEV_STUB
>> +	tristate "Xen PCI-device stub driver"
>> +	depends on PCI && !X86 && XEN
>> +	depends on XEN_BACKEND
>> +	select XEN_PCI_STUB
>> +	default m
>> +	help
>> +	  The PCI device stub driver provides limited version of the PCI
>> +	  device backend driver without para-virtualized support for guests.=

>> +	  If you select this to be a module, you will need to make sure no
>> +	  other driver has bound to the device(s) you want to make visible t=
o
>> +	  other guests.
>> +
>> +	  The "hide" parameter (only applicable if backend driver is compile=
d
>> +	  into the kernel) allows you to bind the PCI devices to this module=

>> +	  from the default device drivers. The argument is the list of PCI B=
DFs:
>> +	  xen-pciback.hide=3D(03:00.0)(04:00.0)
>> +
>> +	  If in doubt, say m.
>> +
>>   config XEN_PCIDEV_BACKEND
>>   	tristate "Xen PCI-device backend driver"
>>   	depends on PCI && X86 && XEN
>>   	depends on XEN_BACKEND
>> +	select XEN_PCI_STUB
>=20
> Does kconfig not at least warn about this? The selected item has a
> "depends on !X88" conflicting with the "depends on X86" here.

XEN_PCI_STUB !=3D XEN_PCIDEV_STUB


Juergen

--------------A604B18524153844DF606DEF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A604B18524153844DF606DEF--

--RabaBlNU167INrkhGG7TimY4gvZhpBVDu--

--2lcwPdRBsDCp1LdWI77itRlY1UcSO1FMi
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMYSoFAwAAAAAACgkQsN6d1ii/Ey9F
jAf8Dbp2R2HZn0f7rIjof1tX1Os673CwlUaxnI/ciMK9PSwo6nvII9KPwHFeK+STusBQ3uAafvBB
ztdiA0zYckwVE0zjyvaHJ+oa1IY0b5Nqbbj2bBopu+uKB3ZKnX6JD4AL4ro/RudxQC4FMN8+ILrJ
rzFTLYN6ErnQztmwnkiJGDlx5xfkyMULySu8S1FDjkdo9nDHZUUZMIorLjT6XD9AWB9oW07yJA+V
S5ZtbPQL4gSHhsIRKoMLipSy3aZhWKYsqYNtan5wA0uMqKDF36fpJmQwoMMjb+FC1wDkdzv/HsQV
lJcIiLuHYA8UrA53yoQmVLEPkEaAd4WVg27q/ol6tA==
=uxBc
-----END PGP SIGNATURE-----

--2lcwPdRBsDCp1LdWI77itRlY1UcSO1FMi--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:13:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:13:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193731.345103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMfa-0007xv-TP; Thu, 23 Sep 2021 11:13:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193731.345103; Thu, 23 Sep 2021 11:13:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMfa-0007xo-Q6; Thu, 23 Sep 2021 11:13:38 +0000
Received: by outflank-mailman (input) for mailman id 193731;
 Thu, 23 Sep 2021 11:13:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/6DS=ON=epam.com=prvs=9900839950=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTMfZ-0007xi-LO
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:13:37 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4bb664a8-1c5f-11ec-ba2a-12813bfff9fa;
 Thu, 23 Sep 2021 11:13:36 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NAlmGm001951; 
 Thu, 23 Sep 2021 11:13:34 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b8pfegkp6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 11:13:34 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6884.eurprd03.prod.outlook.com (2603:10a6:20b:2de::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:13:31 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 11:13:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bb664a8-1c5f-11ec-ba2a-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dfcAZxsSdMPBBvFzpvCmDiR3U/RbP3XiWoPB2UOtThnaqG8UA2iY+J1xnV0gfWrBV4qMS86KHDMumVEUC6JBG2J8Lfnr8mxqkAxJ4cLRE3dKP6rbDDyoZuJOK/mVvBDRUIeesecE/59pRbgbOaR6+nPnjWx+oeU+QlAPQMYO1zyrvN+dnjxKVYrvgWW8XtI+aK/LHmjgqaqTG2feUIB9v0DwvxJDRfLHPfIxgJSyLZa85h+bRlopGQ6mGx9BkSds1vn1JiOODdYoAQQ7jhkPG/3AIiUfAJotTeo4FiOuw33/GK7nMZUzoe9F+Xtz2LnOlCY3EqtQ1jLrNVBNAVmp+Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bmH2R3i2RP4dHzfHu0s3hjen1sJwkocnlF0NpX9KS3U=;
 b=bTXRt/Yn1P3/zxPynK7G3HSFruDhiBZyyIZNioSBsBs9Z6Yr8QmRTibOFkBtX02J9IbAe9oMcG26In+zbyB1rcCjs6OXs6dHth/PHF5pr/xke+0bpThEhMyuCZ8wy6YDqoRpXuC1ZCfslp/GO8yWIho1yy3RqFyjnD0NPvCdonk6voOVr6YtLxEXr1oQ9ur/SeNBXw5mJ2t5zQa4kaEADIU3vCcnFtXOJ8+xWKX0voQW4pxZD9XkyMg/brg32TI5NeeYqFL3ewAP+bRaY7GcRg57SRzMa9ODoYpjbqiaoHtNQszp1FPXio8zJafkS0AWfIQhDsuVVSau2CmWbzxONA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bmH2R3i2RP4dHzfHu0s3hjen1sJwkocnlF0NpX9KS3U=;
 b=QZ7igPuNoRlyN6fe9VZsvZoOn25oA/kiuXFVrgq77Mw3Zemm0rV+agWPLhepw8YRWVPBh1AKgk17X2CmdsG1jxJOBl4d3frEYY1Ci+5BLPw9AwoSY8jd+KNCG2h7PEfrjHQ/0uvvsoCgS8NMiMkKJa9b7k8WhAoq22V3sqf8n9weorZJCDPY00BvgmWbE2EXdfK1IxWs9zX2/sdkw0JwaMBuQFF/aFe5mD+GNN5WqmWtvzNSDTxBLv4MYlOS6p+KAKMN/NBwX4wNFYu7HNMaF6APym6gDgzCrOE81bBZ72vSMt+qUf47L2FvYON/tjPOm8yXEuQ2m1NvniJsxPLm7A==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com" <jgross@suse.com>, "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org" <sstabellini@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: AQHXsGDshY1QGXQ13Ui5WskBVnVMCauxdpMAgAAAwIA=
Date: Thu, 23 Sep 2021 11:13:31 +0000
Message-ID: <4efeaf1e-26d4-a2f0-89ab-1a9d795f1f5a@epam.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
In-Reply-To: <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2e3beadf-a73b-403e-a635-08d97e832d7a
x-ms-traffictypediagnostic: AM9PR03MB6884:
x-microsoft-antispam-prvs: 
 <AM9PR03MB6884C2EB66E360F3A0012759E7A39@AM9PR03MB6884.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 DxJmgmjWZp9KxcncOdtwYajJh1AAapFALq8YaWsGFuQEENKbfR3QGeaks+uh+7mCyh0pjbfP63JW5a5GxxDhYbldf1lNdGrNkiJjTJQqISzMgaPnOcB1glXIV/W4qzGDqWD47s8bfkb9A741adlpmh+w90BnQnYO4d9OvI4GmBDS7YN8Qd4wfvrzT+l00OErO/P/FUyBl1rQtahWGtBcLMbHfGl/YG5vJMG4/idQvZdNigAN/6xssaF8pnuXSq7xtDU9zhw9Q6icSmcG+RW/aYVxj5W8OL8udSaN/l4UPa1t6Bl6oIB8Oej8z7UppR84fyaGQR/gF1SFWBVtHTu1RIz2vGbPq/7ZHzZIsio8VXmS/uE+gvj6xsz8jaQAUjAUteVffUAYtEYKEafJnbrIZdeJTFD96MrGnNGwSMojrwshcDOabBWk0Szkvw5t4geieKg3bKm79/+W9OdzyIR0VlUX5tlfwdtl7fSslWpYlq7M+5zspaVKuTEizLMs9R+GKqLoQgD0VTxtuZDKLKD5GHPApz6iwJE0H4EXGFR18uA0Ha2xtZZbUBws3QGSG0LLYP9sgL6oLvtrXElM6OqPQ1hxI5qZKUvcorW+cNNB9KRkgQVdKQBD1B5b3/g9hhhuOXXwtbdJt6IeIayr9+6xuWM2cy6h7yYmVKUBLJrIg4v2wlNNCFXI0BWbsXUxU+YjHlgMcIkIWlBeQRrgrH9OeD8OL+WEm4fxZI/XWBsvUu0=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(316002)(54906003)(83380400001)(186003)(6512007)(5660300002)(8936002)(110136005)(2906002)(26005)(31696002)(508600001)(4326008)(6506007)(2616005)(71200400001)(31686004)(53546011)(122000001)(66946007)(38070700005)(76116006)(66446008)(64756008)(91956017)(38100700002)(66476007)(66556008)(86362001)(36756003)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eXFPcnV4TFdHTlVzWk4xMFB1T1ZZQWJXMmNrcGphdUNXS0VMVzNtL01IZVIx?=
 =?utf-8?B?UHVTT0NGRHJFNmZkcWJyaFNJZEYwazVReElWdFNKMEkwSWt5eXQrT1FrWlRK?=
 =?utf-8?B?by9lTGZZZG5tdkZXM1MreDVDNmM3STNHaUdKaVJ1Mm5YNmFKcVNGbFJhUzB1?=
 =?utf-8?B?UnN6YnhYZE9mQnFPcCtGamtmOXVuSVBWQU9kU0xiUU5TNUNiYjZrQUFXbnN2?=
 =?utf-8?B?WkFXWWVCdURxbkhFRk5LL3NvdWUvYUZ4ak5JVnZSa3pmdXl1WWxDcnY0Myt3?=
 =?utf-8?B?dGJzZWdDQ2w4QmpiODk1S3BTQVVjeGFHTzRDWHlRQjQ4Vk14UkNQckxxMUw5?=
 =?utf-8?B?MmwvR0k4WWd3YVpuZG1SVDdibko2eGRyMDZxTWxzZGI2bUZaNmJCVWxWQmdS?=
 =?utf-8?B?Rll0bU9qY081M1FTUFJrRUpNWE1xdERnVEZOVkl4alU2azNUQXRISDVYOFdx?=
 =?utf-8?B?RWxpQ3d0alFzYlVMQjZsS2Y5V045anZUSVdLak9MMm1ZQncvZUsrNENlOGNC?=
 =?utf-8?B?RExzUHFEVm5VaUd1THZTTlhBNm1IWUlCMXBaSWdOa2JkdS9mQzJ1UEtlSHgr?=
 =?utf-8?B?UHVZMDdIYmdUOXQ4SG1mYlVFWnpOd1daaytHN0NKci9HaTJtcXpLZ3ROUkRE?=
 =?utf-8?B?b2tTaG5Md2xDeTB6Wi9aVlUxbTRqdGlFL016aFNNcW1EV2pmTUZMVGl6TzNC?=
 =?utf-8?B?VU9iTUJNdVpOQ29nVy9Eb2dPUkJuSnJ0V1FRZVlNaWhHWGdhTDZrMTJERUpH?=
 =?utf-8?B?a3RINWVyMVRyLzBtTmQzdXdTL0tKbDJyOTRTVXA4UFc1eGk0UWNXRUJqWnFu?=
 =?utf-8?B?THkyTHQyOUxUektKOW9ZVGx2NE4vaE9uKzhWOG1ad0RPMVpxdUdTL3hFdGhr?=
 =?utf-8?B?RERhU1BOc09nOGc3eFkxZ29SOGtZeU9zRmtUUko4TU5Ea2JSWlpqbHV2L1Zl?=
 =?utf-8?B?enp2NUxlRDl3VWY3MUFiaUQ1RkFHUUhSZTZTZFlDRXJiNXFUMWZ5OEp4dzBy?=
 =?utf-8?B?dXVPc3VHQ2lGTUVhY1VQSEE0YkptTVBUbVNMSUdkYlhhMHl1TFlrd25MRy94?=
 =?utf-8?B?U3pGRE1SQkFDdXduakVwRlZZMk9kZDJMYkJNSFU4MndOOFRHRUxmNlRDaHpv?=
 =?utf-8?B?Y3JXNGJFU2dpejlCZVY3TjBQQ1BaWjVlTnVNUUlCOXlxOVU1Umx0aURROW13?=
 =?utf-8?B?S3NoMXIzWDBvMU1BSzJBNTUyS2ZRaE56c1M3QVY4NWI4TWxIY0ZyYlNzRlhN?=
 =?utf-8?B?bjNjcFNqTFpMTm5rMzJXTW56Q0lwcXJJTGJkRjFnMDhmVUJGb0FjV3NuZzgz?=
 =?utf-8?B?cmlNcWNhaEZFbFNsWXowdEk0LzVWaE83Q2hNKzdIM3VVL3dIeWNBYTAyRVp2?=
 =?utf-8?B?TXlFMzFMTXlvZ2NSaEV6SVFUYThHK2QrNVBIamFsMUFYYWVibjYraGxzcGF5?=
 =?utf-8?B?TEhDTE5lbVc2aXh1VExvWVl4L1NCOUd4NlY3QXpUT1ZMVmc2ZzMrbm1PUC9w?=
 =?utf-8?B?YVpESzM3bEx1dE5NREJGSk14S0VtaVJodS9IdEx6Qmg5VVNFZUVNTjU0L0U2?=
 =?utf-8?B?NHBTR1JuU1h5MlphQ1NyWnZUb0FWRW9KVmFIMm9UbWo0dTlUTVJBRDFGSVVy?=
 =?utf-8?B?UXQ5cTBhQTZYZnNMOU9FUUpBcmtydkZpbVdyVXkvYWdLZVJQeHR4K2RId3pv?=
 =?utf-8?B?Uk1FVFQ0c2VNRWZwMDdjeWJiWXdwSE8xUUNsRzlsZ0daaDJtdGdTVDF4UWFp?=
 =?utf-8?Q?CA5N/rDvoeC7V4ECCgUtmyp480ykueYKQo5WYg2?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <86AA8D9B9F38F84AB71E7D5618A72D71@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e3beadf-a73b-403e-a635-08d97e832d7a
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 11:13:31.7577
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SLSPo3VMtIxktad63t9EFZ8nUDAlceuMa3BQeXGWL/HI1YBUslcPMUhs08MsZh9B/mcJMqYwTGfobRUcTK0SBTu0hxXWZx0aQNXZdB84vn1bub4ZA0r5O9jpxACOvkuV
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6884
X-Proofpoint-GUID: oSKDCu5391jkehK1iYZvSdqW8hDBFR2-
X-Proofpoint-ORIG-GUID: oSKDCu5391jkehK1iYZvSdqW8hDBFR2-
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_03,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 impostorscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501
 bulkscore=0 malwarescore=0 mlxlogscore=999 clxscore=1015 mlxscore=0
 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230070

DQpPbiAyMy4wOS4yMSAxNDoxMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDIzLjA5LjIwMjEg
MTE6NTMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEvZHJpdmVycy94
ZW4vS2NvbmZpZw0KPj4gKysrIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4gQEAgLTE4MCwxMCAr
MTgwLDM0IEBAIGNvbmZpZyBTV0lPVExCX1hFTg0KPj4gICAJc2VsZWN0IERNQV9PUFMNCj4+ICAg
CXNlbGVjdCBTV0lPVExCDQo+PiAgIA0KPj4gK2NvbmZpZyBYRU5fUENJX1NUVUINCj4+ICsJYm9v
bA0KPj4gKw0KPj4gK2NvbmZpZyBYRU5fUENJREVWX1NUVUINCj4+ICsJdHJpc3RhdGUgIlhlbiBQ
Q0ktZGV2aWNlIHN0dWIgZHJpdmVyIg0KPj4gKwlkZXBlbmRzIG9uIFBDSSAmJiAhWDg2ICYmIFhF
Tg0KPj4gKwlkZXBlbmRzIG9uIFhFTl9CQUNLRU5EDQo+PiArCXNlbGVjdCBYRU5fUENJX1NUVUIN
Cj4+ICsJZGVmYXVsdCBtDQo+PiArCWhlbHANCj4+ICsJICBUaGUgUENJIGRldmljZSBzdHViIGRy
aXZlciBwcm92aWRlcyBsaW1pdGVkIHZlcnNpb24gb2YgdGhlIFBDSQ0KPj4gKwkgIGRldmljZSBi
YWNrZW5kIGRyaXZlciB3aXRob3V0IHBhcmEtdmlydHVhbGl6ZWQgc3VwcG9ydCBmb3IgZ3Vlc3Rz
Lg0KPj4gKwkgIElmIHlvdSBzZWxlY3QgdGhpcyB0byBiZSBhIG1vZHVsZSwgeW91IHdpbGwgbmVl
ZCB0byBtYWtlIHN1cmUgbm8NCj4+ICsJICBvdGhlciBkcml2ZXIgaGFzIGJvdW5kIHRvIHRoZSBk
ZXZpY2UocykgeW91IHdhbnQgdG8gbWFrZSB2aXNpYmxlIHRvDQo+PiArCSAgb3RoZXIgZ3Vlc3Rz
Lg0KPj4gKw0KPj4gKwkgIFRoZSAiaGlkZSIgcGFyYW1ldGVyIChvbmx5IGFwcGxpY2FibGUgaWYg
YmFja2VuZCBkcml2ZXIgaXMgY29tcGlsZWQNCj4+ICsJICBpbnRvIHRoZSBrZXJuZWwpIGFsbG93
cyB5b3UgdG8gYmluZCB0aGUgUENJIGRldmljZXMgdG8gdGhpcyBtb2R1bGUNCj4+ICsJICBmcm9t
IHRoZSBkZWZhdWx0IGRldmljZSBkcml2ZXJzLiBUaGUgYXJndW1lbnQgaXMgdGhlIGxpc3Qgb2Yg
UENJIEJERnM6DQo+PiArCSAgeGVuLXBjaWJhY2suaGlkZT0oMDM6MDAuMCkoMDQ6MDAuMCkNCj4+
ICsNCj4+ICsJICBJZiBpbiBkb3VidCwgc2F5IG0uDQo+PiArDQo+PiAgIGNvbmZpZyBYRU5fUENJ
REVWX0JBQ0tFTkQNCj4+ICAgCXRyaXN0YXRlICJYZW4gUENJLWRldmljZSBiYWNrZW5kIGRyaXZl
ciINCj4+ICAgCWRlcGVuZHMgb24gUENJICYmIFg4NiAmJiBYRU4NCj4+ICAgCWRlcGVuZHMgb24g
WEVOX0JBQ0tFTkQNCj4+ICsJc2VsZWN0IFhFTl9QQ0lfU1RVQg0KPiBEb2VzIGtjb25maWcgbm90
IGF0IGxlYXN0IHdhcm4gYWJvdXQgdGhpcz8gVGhlIHNlbGVjdGVkIGl0ZW0gaGFzIGENCj4gImRl
cGVuZHMgb24gIVg4OCIgY29uZmxpY3Rpbmcgd2l0aCB0aGUgImRlcGVuZHMgb24gWDg2IiBoZXJl
Lg0KDQpXaHk/DQoNClhFTl9QQ0lERVZfU1RVQiBkZXBlbmRzIG9uIFBDSSAmJiAhWDg2ICYmIFhF
Tg0KDQpYRU5fUENJX1NUVUIgaGFzIG5vdGhpbmcsIGp1c3QgYm9vbA0KDQpYRU5fUENJREVWX0JB
Q0tFTkQgc2VsZWN0cyBYRU5fUENJX1NUVUINCg0KPg0KPiBKYW4NCj4=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:14:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:14:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193737.345114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMgA-00005G-7A; Thu, 23 Sep 2021 11:14:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193737.345114; Thu, 23 Sep 2021 11:14:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMgA-000054-3e; Thu, 23 Sep 2021 11:14:14 +0000
Received: by outflank-mailman (input) for mailman id 193737;
 Thu, 23 Sep 2021 11:14:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMg8-00004q-II
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:14:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMg8-0006Fa-69; Thu, 23 Sep 2021 11:14:12 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMg7-00048U-SI; Thu, 23 Sep 2021 11:14:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=1zAR4mzKy/VhBD2yok0W5hHTz87+eTPfnMf+q8xx1mk=; b=D5nNqcchyYQ2r5dRrivs9TiLEK
	BtydZV6pt0tdRNv8nORjnloodcVCuf2bkzlsnnXsjmuN2X9cGpZePbrA5OIRwG7QebfoGO4TlQ0O/
	CMRC2lHZy9s6JZTmCt1MDg7RTdBCT6WlTt3eNUkomM0InGxH4YjpMeBvZzjS2o8s2GwE=;
Subject: Re: [PATCH 09/11] xen/arm: if 1:1 direct-map domain use native UART
 address and IRQ number for vPL011
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-10-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <db752d34-fc23-04b3-3c84-12d4de6859e0@xen.org>
Date: Thu, 23 Sep 2021 16:14:08 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-10-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit



On 23/09/2021 08:11, Penny Zheng wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> We always use a fix address to map the vPL011 to domains. The address
> could be a problem for domains that are directly mapped.
> 
> So, for domains that are directly mapped, reuse the address of the
> physical UART on the platform to avoid potential clashes.
> 
> Do the same for the virtual IRQ number: instead of always using
> GUEST_VPL011_SPI, try to reuse the physical SPI number if possible.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/domain_build.c  | 34 +++++++++++++++++++++++++++-------
>   xen/arch/arm/vpl011.c        | 34 +++++++++++++++++++++++++++-------
>   xen/include/asm-arm/vpl011.h |  2 ++
>   3 files changed, 56 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 120f1ae575..c92e510ae7 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -30,6 +30,7 @@
>   
>   #include <xen/irq.h>
>   #include <xen/grant_table.h>
> +#include <xen/serial.h>
>   
>   static unsigned int __initdata opt_dom0_max_vcpus;
>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
> @@ -1942,8 +1943,11 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
>       gic_interrupt_t intr;
>       __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>       __be32 *cells;
> +    struct domain *d = kinfo->d;
> +    char buf[27];
>   
> -    res = fdt_begin_node(fdt, "sbsa-uart@"__stringify(GUEST_PL011_BASE));
> +    snprintf(buf, sizeof(buf), "sbsa-uart@%"PRIx64, d->arch.vpl011.base_addr);
> +    res = fdt_begin_node(fdt, buf);
>       if ( res )
>           return res;
>   
> @@ -1953,14 +1957,14 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
>   
>       cells = &reg[0];
>       dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS,
> -                       GUEST_ROOT_SIZE_CELLS, GUEST_PL011_BASE,
> +                       GUEST_ROOT_SIZE_CELLS, d->arch.vpl011.base_addr,
>                          GUEST_PL011_SIZE);
>   
>       res = fdt_property(fdt, "reg", reg, sizeof(reg));
>       if ( res )
>           return res;
>   
> -    set_interrupt(intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> +    set_interrupt(intr, d->arch.vpl011.virq, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
>   
>       res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
>       if ( res )
> @@ -2670,6 +2674,13 @@ static int __init construct_domU(struct domain *d,
>       else
>           allocate_static_memory(d, &kinfo, node);
>   
> +    /*
> +     * Initialization before creating its device
> +     * tree node in prepare_dtb_domU.
> +     */

I think it would be better to explain *why* this needs to be done before.

> +    if ( kinfo.vpl011 )
> +        rc = domain_vpl011_init(d, NULL);
> +
>       rc = prepare_dtb_domU(d, &kinfo);
>       if ( rc < 0 )
>           return rc;
> @@ -2678,9 +2689,6 @@ static int __init construct_domU(struct domain *d,
>       if ( rc < 0 )
>           return rc;
>   
> -    if ( kinfo.vpl011 )
> -        rc = domain_vpl011_init(d, NULL);
> -
>       return rc;
>   }
>   
> @@ -2723,15 +2731,27 @@ void __init create_domUs(void)
>   
>           if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
>           {
> +            unsigned int vpl011_virq = GUEST_VPL011_SPI;

Coding style: Add a newline here.

>               d_cfg.arch.nr_spis = gic_number_lines() - 32;
>   
> +            /*
> +             * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map in
> +             * set, in which case we'll try to match the hardware.
> +             *
> +             * Typically, d->arch.vpl011.virq has the vpl011 irq number
> +             * but at this point of the boot sequence it is not
> +             * initialized yet.
> +             */
> +            if ( direct_map && serial_irq(SERHND_DTUART) > 0 )
> +                vpl011_virq = serial_irq(SERHND_DTUART);

I think we should not continue if the domain is direct-mapped *and* the 
IRQ is not found. Otherwise, this will may just result to potential 
breakage if GUEST_VPL011_SPI happens to be used for an HW device.

> +
>               /*
>                * vpl011 uses one emulated SPI. If vpl011 is requested, make
>                * sure that we allocate enough SPIs for it.
>                */
>               if ( dt_property_read_bool(node, "vpl011") )
>                   d_cfg.arch.nr_spis = MAX(d_cfg.arch.nr_spis,
> -                                         GUEST_VPL011_SPI - 32 + 1);
> +                                         vpl011_virq - 32 + 1);
>           }
>   
>           /*
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index 895f436cc4..10df25f098 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -29,6 +29,7 @@
>   #include <xen/mm.h>
>   #include <xen/sched.h>
>   #include <xen/console.h>
> +#include <xen/serial.h>
>   #include <public/domctl.h>
>   #include <public/io/console.h>
>   #include <asm/pl011-uart.h>
> @@ -71,11 +72,11 @@ static void vpl011_update_interrupt_status(struct domain *d)
>        * status bit has been set since the last time.
>        */
>       if ( uartmis & ~vpl011->shadow_uartmis )
> -        vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, true);
> +        vgic_inject_irq(d, NULL, vpl011->virq, true);
>   
>       vpl011->shadow_uartmis = uartmis;
>   #else
> -    vgic_inject_irq(d, NULL, GUEST_VPL011_SPI, uartmis);
> +    vgic_inject_irq(d, NULL, vpl011->virq, uartmis);
>   #endif
>   }
>   
> @@ -347,7 +348,8 @@ static int vpl011_mmio_read(struct vcpu *v,
>                               void *priv)
>   {
>       struct hsr_dabt dabt = info->dabt;
> -    uint32_t vpl011_reg = (uint32_t)(info->gpa - GUEST_PL011_BASE);
> +    uint32_t vpl011_reg = (uint32_t)(info->gpa -
> +                                     v->domain->arch.vpl011.base_addr);
>       struct vpl011 *vpl011 = &v->domain->arch.vpl011;
>       struct domain *d = v->domain;
>       unsigned long flags;
> @@ -430,7 +432,8 @@ static int vpl011_mmio_write(struct vcpu *v,
>                                void *priv)
>   {
>       struct hsr_dabt dabt = info->dabt;
> -    uint32_t vpl011_reg = (uint32_t)(info->gpa - GUEST_PL011_BASE);
> +    uint32_t vpl011_reg = (uint32_t)(info->gpa -
> +                                     v->domain->arch.vpl011.base_addr);
>       struct vpl011 *vpl011 = &v->domain->arch.vpl011;
>       struct domain *d = v->domain;
>       unsigned long flags;
> @@ -622,10 +625,27 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>   {
>       int rc;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
> +    const struct vuart_info *uart = serial_vuart_info(SERHND_DTUART);
>   
>       if ( vpl011->backend.dom.ring_buf )
>           return -EINVAL;
>   
> +    vpl011->base_addr = GUEST_PL011_BASE;
> +    vpl011->virq = GUEST_VPL011_SPI;
> +    if ( is_domain_direct_mapped(d) )
> +    {
> +        if ( uart != NULL && serial_irq(SERHND_DTUART) > 0 )
> +        {
> +            vpl011->base_addr = uart->base_addr;
> +            vpl011->virq = serial_irq(SERHND_DTUART);

This seems a bit pointless to call serial_irq() twice. How about add a 
field in vuart_info to return the interrupt number?

> +        }
> +        else
> +            printk(XENLOG_ERR
> +                   "Unable to reuse physical UART address and irq for vPL011.\n"
> +                   "Defaulting to addr %#"PRIpaddr" and IRQ %u\n",
> +                   vpl011->base_addr, vpl011->virq);
> +    }
> +
>       /*
>        * info is NULL when the backend is in Xen.
>        * info is != NULL when the backend is in a domain.
> @@ -661,7 +681,7 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>           }
>       }
>   
> -    rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
> +    rc = vgic_reserve_virq(d, vpl011->virq);
>       if ( !rc )
>       {
>           rc = -EINVAL;
> @@ -673,12 +693,12 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>       spin_lock_init(&vpl011->lock);
>   
>       register_mmio_handler(d, &vpl011_mmio_handler,
> -                          GUEST_PL011_BASE, GUEST_PL011_SIZE, NULL);
> +                          vpl011->base_addr, GUEST_PL011_SIZE, NULL);

So you are making the assumpption that the UART region will be equal to 
(or bigger) than GUEST_PL011_SIZE. There are definitely UART out where 
the MMIO region is smaller than 4K.

Although, I don't expect them to be passthrough today. So this is 
probably fine to assume that the next 4K is free. Can you add some 
justification in-code and in the commit message?

>   
>       return 0;
>   
>   out2:
> -    vgic_free_virq(d, GUEST_VPL011_SPI);
> +    vgic_free_virq(d, vpl011->virq);
>   
>   out1:
>       if ( vpl011->backend_in_domain )
> diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> index e6c7ab7381..c09abcd7a9 100644
> --- a/xen/include/asm-arm/vpl011.h
> +++ b/xen/include/asm-arm/vpl011.h
> @@ -53,6 +53,8 @@ struct vpl011 {
>       uint32_t    uarticr;        /* Interrupt clear register */
>       uint32_t    uartris;        /* Raw interrupt status register */
>       uint32_t    shadow_uartmis; /* shadow masked interrupt register */
> +    paddr_t     base_addr;
> +    unsigned int virq;
>       spinlock_t  lock;
>       evtchn_port_t evtchn;
>   };
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:15:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:15:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193744.345125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMh4-0000mo-MN; Thu, 23 Sep 2021 11:15:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193744.345125; Thu, 23 Sep 2021 11:15:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMh4-0000mh-Ic; Thu, 23 Sep 2021 11:15:10 +0000
Received: by outflank-mailman (input) for mailman id 193744;
 Thu, 23 Sep 2021 11:15:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTMh4-0000mb-1p
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:15:10 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe02::61f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 245d3c0e-71fe-4172-b3be-10396d6c39bb;
 Thu, 23 Sep 2021 11:15:07 +0000 (UTC)
Received: from AM6PR08CA0010.eurprd08.prod.outlook.com (2603:10a6:20b:b2::22)
 by PR3PR08MB5851.eurprd08.prod.outlook.com (2603:10a6:102:85::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:15:05 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:b2:cafe::9c) by AM6PR08CA0010.outlook.office365.com
 (2603:10a6:20b:b2::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 11:15:05 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 11:15:04 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Thu, 23 Sep 2021 11:15:03 +0000
Received: from 29ef04d864ad.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 10DE5F4C-10BC-4397-9545-CADF79FECDAE.1; 
 Thu, 23 Sep 2021 11:14:52 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 29ef04d864ad.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 11:14:52 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB7026.eurprd08.prod.outlook.com (2603:10a6:20b:34d::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 11:14:50 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 11:14:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 245d3c0e-71fe-4172-b3be-10396d6c39bb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uGOPPHVn3iO3ThBFgg3V+W+0iB/6oRHnxU04VoSJulc=;
 b=OJRUVtT4VTWZU0+lPadUAM+bzahVASGVkK2fO35+jj7bhKY+mElMsvEtETeUVaFEF8ik+DyyCrPc1P7FVf1mp3S8jwIFAsChW59nK0fRtVf+k1JEgHW2LejaPbfa+7QVU81pt+EaZnxcM0Qjub49FYTyTuuqIU6TjcqqXMj+Q+A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 7ad4955db1dd7cdd
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wx6FyFJ2GzTZbL277U9TUdMqXHllcVWA4g/VsrQuyynFd3s0ZWycieWIL2Y5ivlkGP25mnDTls/Qk3xW0802Oe5U17gbgU07P5GKz2GpqhQfdq4tF6kd3i5dXhHHjR5pJSwU6fv3h//YvGXUx+HNeGtg61t56JQgygaASzhZgDTr+nii+C/hec5k10JRoDoLVGI+t4qcroFmiMkixwb6SLIYv+RoPRsvd5yz9sW9Tz9OJ66OYz/g8fq/PmMCB37/JdUk04MEywS762BSUEijCZJusFnKyWcEJAhdJ2nD99CKz25d0dt+kUc94c1fgVyeA6/vDfQe6B3ybbUvKHnRrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uGOPPHVn3iO3ThBFgg3V+W+0iB/6oRHnxU04VoSJulc=;
 b=Z0QZuQrdro6eqflNnZ9R1+lHeNgL4BPIop4idpQrlz97Dm30PIGLqEkx+ytD8nxRJg0xDegws55ah/sgSKt9Ne/d3d7SV8M75oy3udOOoj0dWixgSlftn9ZqK2eGknE0MH9C7Luidymv1dEkaoZH5oXBXzj5c21qPO+8qtajOf/fVd+GXn78g5YQpuzWNZO3x+0bzSO5KyxXgUTKVfqFxGgGd6a/fNaNg7Ez+2ay/NfluWlSYxiJqtYCMZgn1JCl4NtIh9M6a+DMjH2laAxNn/nKOosY65CDjHJy4Lj3cAvHpmbgK3SP/oaFdgVvSqVOhwI+HiyHdLZxLOjQM/gEIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uGOPPHVn3iO3ThBFgg3V+W+0iB/6oRHnxU04VoSJulc=;
 b=OJRUVtT4VTWZU0+lPadUAM+bzahVASGVkK2fO35+jj7bhKY+mElMsvEtETeUVaFEF8ik+DyyCrPc1P7FVf1mp3S8jwIFAsChW59nK0fRtVf+k1JEgHW2LejaPbfa+7QVU81pt+EaZnxcM0Qjub49FYTyTuuqIU6TjcqqXMj+Q+A=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien.grall.oss@gmail.com>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 02/17] xen/pci: solve compilation error on ARM with
 HAS_PCI enabled
Thread-Topic: [PATCH v2 02/17] xen/pci: solve compilation error on ARM with
 HAS_PCI enabled
Thread-Index: AQHXr6YM7M3wnWXhpk+HlxEaE7sJMauw4CUAgACZAIA=
Date: Thu, 23 Sep 2021 11:14:50 +0000
Message-ID: <3E7CE8E4-367F-44BF-9FE6-3A99E7301610@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <b6a9c007061f963332af63da544e5031e18a7850.1632307952.git.rahul.singh@arm.com>
 <CAJ=z9a32OsxxMbPnOEO4cEJweiBkF61U0niZDX2Z77gv+-zAQQ@mail.gmail.com>
In-Reply-To:
 <CAJ=z9a32OsxxMbPnOEO4cEJweiBkF61U0niZDX2Z77gv+-zAQQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 43c8511b-0032-49c9-9442-08d97e8364f1
x-ms-traffictypediagnostic: AS8PR08MB7026:|PR3PR08MB5851:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PR3PR08MB585182BB6173B71280003E05FCA39@PR3PR08MB5851.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:3631;OLM:3631;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AaSQp1FVkWzVR4P4BO/OCvDyD8DRnku1SKLgR5J2mNu/ozZJIZg0MVSYyjXVo1r+qij70jGMrdGFdqHgLqD7qqGb5o+sMlTqOxsW6xCiaJkcSHyf3ulxNOkGM/Yu+2rlCeZJdot0eeQIu1SD56w9xvTMVwI68Gghgf8/Kj/Olvb+BlXeswMTgt6nu8B+CmkH14AvhXW6/oeZbjUwftkMkPAOhhFziCkm9Xo2wB3/LdlemfgmNm7X+CH+rbsRLmMtYeHLfJHERwm9aQzG4Vaxz3twE6xw9hgQHkguDNep5w/6ZY2V7xLIvK4mMADQz7667UlghajvLvIoNJ4jozkerev/2HmSsakx1CfUgS8E+k9bAFcvtg+Oe/witPLX4sv94GCOHdqruBbVsXfvVVpXHc+JO0QLLHN6Ry+dF6aAUCSL8viaysMADIQEl/eCfW9qttUzFRRZiqfLtgg5vR6xFa5VIP+kESwfoVN5JFj59Lf/cBT8eE51OUdwcRpN/Q+stSAIR0tk29cdkLR8Mh3anKMYKduQjQGFlm+2iAWxtc+GFwpUyxScxfsp0TRYtjc7XYfcEqsmD3bY5+W083OhAdUG5w+hOHDD5okg+hx9epcVtPsBVwGLQ+db+iIintwAETOastfqAegNuDGF8qQt0S8t6Z4L9Wn4JpUU3I14jEgHXIYpPtzI6czahmJS2YW/NbSRk4TG+DtJefC3c4wp45+DNmfmzrsKgERkSLDH3M0=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(6486002)(122000001)(6512007)(4744005)(91956017)(508600001)(8936002)(66446008)(64756008)(66556008)(66476007)(76116006)(66946007)(33656002)(53546011)(316002)(5660300002)(86362001)(54906003)(4326008)(6506007)(71200400001)(2616005)(2906002)(186003)(26005)(83380400001)(8676002)(36756003)(6916009)(38070700005)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3CEB873553E6264CB448622A7ADF9D29@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7026
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	44cd5c64-f8d1-4da5-5ec3-08d97e835c8e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xDB6e4+elvXKEqzFrsERurE5qehjn5sdeTiV2can+0vQKY2k2pcr3dZvmnxF286yh9KRShpEHvadf5Kq8ilo/L6/IAlIfchNG1CapzwaSfme5cC7zaY86CF3KebZOttr7+z374qje1Kiy7Q86fi6bHkkeQeg26b7Tz+aPHxMozPzTrOIjCQ0RRxSjmbeXECfQglixPOnsOOIp6FJYhZl6T5PCrLwyp84kj6oTp9yhCPlHbj8P1nA7JLt1aKDPgrK2QmNCsZ4QsiT8dvpKi+rM/Y7xCsdIhLDBdBumZgb+bt8ExoD0zfbhAvlQ4LZxwYOz7Nmsnaa/2x1LwFytTiyb/8rWldVd/NbJQGOcLK/WcoPsrvv/Yd/+8UyOijysPJpgVYQj0W7vyotnjmaCFY4XkivKdOqmwrRqgiMqm3NPGtCWYuzwTb4IvJF8bNui9Jselkhuz+hV2qNjc8TMaYdF36AVy+phuXy6BkRDsD4HHF2aV/0r1FZwCUr6QtyU6CElHA+Un3/Aam0gCuVVW8gckEu05IzJsduAukwZPuY7Jf6l8nYJlFy7S2e4deozqSg9wkQiuC3MR4xNT5N9xlShWu7u6en76wfOZqD6iPugPKnNslkojQFUYKIKl9ClusoZDP/xpY5JpUCRRXLf5pW7FpUFQSBkUj3PfOjc4evRwNco+owDTSXzq9qqtey0yOdMFschNe+bXDfY6J0Gp0JPQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(5660300002)(8936002)(8676002)(336012)(107886003)(53546011)(36860700001)(4326008)(70586007)(83380400001)(70206006)(86362001)(82310400003)(47076005)(6486002)(186003)(6506007)(54906003)(316002)(6512007)(508600001)(81166007)(26005)(6862004)(2906002)(2616005)(33656002)(356005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:15:04.8395
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 43c8511b-0032-49c9-9442-08d97e8364f1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5851

Hi Julien

> On 23 Sep 2021, at 3:07 am, Julien Grall <julien.grall.oss@gmail.com> wro=
te:
>=20
> Hi,
>=20
> On Wed, 22 Sep 2021, 16:36 Rahul Singh, <rahul.singh@arm.com> wrote:
> Compilation error is observed when HAS_PCI is enabled for ARM
> architecture.
>=20
> In general, when I read "compilation error" I interpret as a user can tri=
gger it in the current staging.
>=20
> However, without the full series applied, HAS_PCI cannot be selected on A=
rm.
>=20
> So I think the message is a bit misleading because this is more implement=
ing stubs to enable a feature rather than fixing compilation errors (AFAICT=
 all of them are not fixed here).
>=20
> How about the following commit message:
>=20
> xen/arm: pci: Add stubs to allow selecting HAS_PCI
>=20
> In a follow-up we will enable PCI support in Xen on Arm (i.e select HAS_P=
CI).
>=20
> The generic code expects the arch to implement a few functions:
>=20
> <List the functions>
>=20
> Note that this is not yet sufficient to enable HAS_PCI and will be addres=
sed in follow-ups.
>=20

Ok. I will modify the commit message in next version.

Regards,
Rahul
 =


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:18:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:18:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193749.345136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMjv-0001hM-4k; Thu, 23 Sep 2021 11:18:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193749.345136; Thu, 23 Sep 2021 11:18:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMjv-0001hF-1H; Thu, 23 Sep 2021 11:18:07 +0000
Received: by outflank-mailman (input) for mailman id 193749;
 Thu, 23 Sep 2021 11:18:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTMjt-0001h9-SI
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:18:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eb968836-1c5f-11ec-ba2a-12813bfff9fa;
 Thu, 23 Sep 2021 11:18:04 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-H_AUkDNRNbyR7QDuPPWtSQ-1;
 Thu, 23 Sep 2021 13:18:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5741.eurprd04.prod.outlook.com (2603:10a6:803:df::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 11:18:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 11:18:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0232.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 11:18:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb968836-1c5f-11ec-ba2a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632395884;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=J81t8+yaR4LN6D9aRF2BIKOcrCWettxIPfYEDkKGzCQ=;
	b=OZHFiOoQJqQqVkwG7mXgQtfpFAfaEY6wxtBSWS224ZJFbnyVYE97OmYbP+78Sq/HFG/mC+
	usxzw7QhkkNmkjfmwVsqNTZ9tBH+SSEI8+XoJdRkhcau5oZ1nh57+mzenFAVwwwykfuD2j
	vq0jroiukCyKXutq+rwqkYd+435/Pvs=
X-MC-Unique: H_AUkDNRNbyR7QDuPPWtSQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aNgY4my2hahXCig+MMTk5riAbbxuXPnJGe/ebnQoVwl4QkgPjA3H5V/TX8O4QwhFH5akqezB3p0cL7B+4CqN95JHB2fW12YNnqYLNqT/hUiy6NzIIQuSGY0gFNwRKjCxyekzESmtRJsktphqZrtJCJSLR+vRa00DXJLRB2MloXP+gOY+oyoLT9Vjj8J7hxPMNurREuD+KnhEjcRt6I/9Pd8CSeHZEA1I/kAcPcrE1KiAP0HsANN6vR4IrREw/7jhYPKl/qValQ6sRtvV7hDcqFp/xZPu5J63qyRGpo4r+nbwmQWmiHlVdQqZKDiD+SdicZQNCM3FRYqiUpRh1kNolA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=J81t8+yaR4LN6D9aRF2BIKOcrCWettxIPfYEDkKGzCQ=;
 b=aV+fdTy60kq1ziVszaj3rcrxDLnXGUw/Rn9l3oRS0jdQbYloeKC70Idc0Je3ii5L0DsQCDTDCSx8MUSE2QrUfzBur5f9Nl4LeAPTbAOR8LIcaG1+tnHXNZ2d3FfdxJD5vjKEcurvhgKij5tmXczkd3lKKS7K9SQg/JZpUWSp6bKNs8KQvsJzgdJtM78gS9C3BWRPQAs+td3QLwSxPco/932r1rqAo6/HdqduJ5Mx/DZPYkOQ34Orgcau5QS+YypDXhliy2f8olvhInnQeyrJnptyzDL2unJSfN1frzpXlJYrdYyQIf1HnXtQQRWHG7sVSIFt2g+NO9ziMw3cXoJtaw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
To: Juergen Gross <jgross@suse.com>
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 Oleksandr Andrushchenko <andr2000@gmail.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <d12b0bcd-e998-d4c5-e673-9c13a864eea4@suse.com>
 <478b9175-f21f-b77a-2bc1-ad230bbdf548@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <87bb76ba-c8b4-5ab0-a8c3-13e8ab4af3b9@suse.com>
Date: Thu, 23 Sep 2021 13:17:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <478b9175-f21f-b77a-2bc1-ad230bbdf548@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0232.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 38f43355-5a85-49c2-620d-08d97e83cdbc
X-MS-TrafficTypeDiagnostic: VI1PR04MB5741:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB57412606ADB588077C14D425B3A39@VI1PR04MB5741.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Beeqt/OZ1hyEHduJZDfMjyppdL7+nsSaSSRmO4TE8zqt2lhBKIC6NkkQrX7/mKVKmIoPWItawQ7WKLmmzGY66/5KgMyFHOmkkY6lxrp6X8y+AA4RQ+9gCKb3Rtufi9B88xqOn8b6TKky+IvoLZdcCKF5omxQlOK9ug6ygpul4nAmEPSU7mL/zKzdnYGhul6oo1POB0IAANocScsDgwUzVyH0ls+HL5PPIZba3yGgnrZWUsUOqZXVMXtf48b/0SHDo0VN8fUKzPN/zC7fvEV72u+VJ1zyguIFOduiknYVw+D+4hh3JguAK93Y+YGJZHqcAA8mOMoe8fbyjbqcQM28O0FtvJ07y6ZpcYn+wJHu69/tUngdDQ6z/WxVS5qAeCS0Evn9JjzkqdxnAJFl+AjTchdZ9R1pNnkhh6pNIHehNWHqsz5GsMqArDM3uE77G83P0kaIxO/+2EfpjzDY94JNVPESGZXstsVUYavmSbLaYZzVkrH8qR7O3ET4gbgBxSzIH0u0wSUfQbZt8ZcaOJjXU2uJjM2EjMCUhZcV2Z2t1wMz3UfWLPoiIE1Rys9Zfb2TbvjEvscndutaRDGP5Yks3TFelBp9Ja3IBTARY/Vk8XWrC4745yna+571JqE7KaFT9FgzjJZhW08Sc42A1JArkUlCIEHVk2KhPyAzbX98ZlsnYM18+h3LwiJtbxG5d5nyC89uRkrWeUBnM2+MuhIHOwrdCYPd2/VdNq+PEMSHs5s=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6862004)(186003)(36756003)(4326008)(26005)(6486002)(8676002)(2906002)(38100700002)(31696002)(83380400001)(6636002)(31686004)(53546011)(86362001)(508600001)(956004)(5660300002)(66946007)(2616005)(16576012)(37006003)(8936002)(316002)(66556008)(66476007)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OUVjdUg5MjdxUXFWQmRSWWVFR0xtRCt4cGFHRDJGZ2pHQUdta1Iyb3V1SXdE?=
 =?utf-8?B?WjZnenZ0YWNZMGFydTRHTmFLaWhjd1BpL053akttRitFZlh2VUoySmRJN2pH?=
 =?utf-8?B?Tk44RUxnOXJXODEycHVPdlpDRzVrR0MvcWVkZVh1T0h3SHBaMFVhZDJxYmZm?=
 =?utf-8?B?YlcydGFoOGZqN01YSndHUDBubEpBa0hmQmY5Nk9rbGZZd3h0Rm9kbVNkdjVh?=
 =?utf-8?B?N2RON0lMeVR1OGpXQTZFdW5KdFBJK2xrZWEwQXdMdkY5cmlVSkdUVW5YM0d2?=
 =?utf-8?B?bjJMU0dtOFpnNEErcXZSUW5pOFBQT1BDTWpzdGJuckFDTUVNTW9VcUpOelVU?=
 =?utf-8?B?YjlVWTBLcnRRR1QxVEh6c1V6TXJFK3hkbXEzYUNtV1lqbGFMaXpEUGMwOXVq?=
 =?utf-8?B?S0pjbDNQMFp1YmR0dTJXbXV5YjRsT25TOW9Xdy9xS1orMnQxUnNGWnpXbDl3?=
 =?utf-8?B?RC9Lb2huUDE0Y1JNVlplNk03NHdvK1ZlcWxZVCtRNlVsK2N6ZDN0NzdoN3U4?=
 =?utf-8?B?cW9BOUN2VHBZdmpQZ01BaVVxcFVQWjJ2aFpHMndqajZxeFlLcEtxanFPNGhC?=
 =?utf-8?B?ejRyclB6MUxDbUdkNVYzSHVRbTVkYjhvZlRlL1dpb1RoQnUrNlBzVUpxWmph?=
 =?utf-8?B?bHZWTjMvRkZxdDJreG4xQ2hvTzg2eXozWk1WWUplKzRPNjlKOUo4bG84ZHht?=
 =?utf-8?B?STJmbXFUWnZpMEtWZzN2MzQvdVdNSS9YVlVLYWxZU2JQNStyWEtRWVcrY2xs?=
 =?utf-8?B?SC80RFh2OXpRcTR4Mk9zSGRoa2NpMDVKTWZ0ZjNjWTh3a2RBdWtIeHFKOEIv?=
 =?utf-8?B?NlZ4cDJyRnp5aWZianRFb0R4a2FuOGRkOVZwSzRQUCs4d3AxOE1xZkJCdFFo?=
 =?utf-8?B?NENCTDAvci9ZT3lWZVFFbmFueXdFd2I1NG03citmRTlMdk5pN2l1MjJvZG1w?=
 =?utf-8?B?YTFPZUZ0RVlHRkpJMFg4d1BHNHpwMSsxSFJublZleThKTVl0OFY1cXlQU3pp?=
 =?utf-8?B?Z2VWWXBGdFI1ai90M1RaVTlMaE54WEF2NUJ1V0VFcnVXWVg1VkZORGt1aVNO?=
 =?utf-8?B?N3UrQWowSXIrRldLQUdnZGI4TnFTK3RaVVl2MXVITlhMb0RVQldhTzgrcXRa?=
 =?utf-8?B?WnBuT3pLMDd0WW03VmV3YTUyZm00RmFLNG1aeGNkSGxxcHlFcEVKUnNheWh1?=
 =?utf-8?B?U1YzczJ4N1NCUmFvY0tKRW5zUUEyaXlubG1QOWNkL05KdnVFSDBVcU4wV3Nt?=
 =?utf-8?B?NVN1UFR2bjU2MnQ0d3ZVOXdNVHZXVDQvaFdZc1pyckt5azFkN3pmMExPREdP?=
 =?utf-8?B?ZkVaR2tGTElWTlJmaDEwbVN1ak9WemV5Nm9GTy9jTlBrb1RXV01IcWw1M0lY?=
 =?utf-8?B?ZjB2SlppSEtyamY3Q2p2RUtKREpKRjRMbjBsbCt2RzczTkdCcmlEUHJ4T0Jq?=
 =?utf-8?B?MXpCcm5jblpzVUlWR0N1a3pYMnIvWHNLMkNJWDd6djY0NEp2UDBGc3lvTTR1?=
 =?utf-8?B?Mm5UUkgxZFJwNGRYaWNKZ0RQUDVmRnlZdElsNVp0Tm56bzE2WjBOVmVjbzVT?=
 =?utf-8?B?a3lZc2pCOENkZ1QySm96UjRSNERabTlITGUyZGd4czJOZEtNN2N5b0VlaDcv?=
 =?utf-8?B?c3N2OW82WWNoc00wbEZmdnFXQlN6V1JTRDgxNEg0K3Q4d0RELzdvYlExeTM1?=
 =?utf-8?B?L08vMlFBZThuV05QWDdySzlFamR3QW5RZUFMcUd3a0xJaDBxZC9LaVBXYUFC?=
 =?utf-8?Q?eLMFKvNIrC9u7gZCLSdcecTpWrkCm4xpNXYIn2c?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 38f43355-5a85-49c2-620d-08d97e83cdbc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:18:00.7826
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AHFVMvqamEqSEnqY3qQEGHs4J4ja+9HYem0/+C0cBrLIA0BZeVLQJn2gIM3OJ0ZczIOz1B34jlQG5bOcwUdS9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5741

On 23.09.2021 13:12, Juergen Gross wrote:
> On 23.09.21 13:10, Jan Beulich wrote:
>> On 23.09.2021 11:53, Oleksandr Andrushchenko wrote:
>>> --- a/drivers/xen/Kconfig
>>> +++ b/drivers/xen/Kconfig
>>> @@ -180,10 +180,34 @@ config SWIOTLB_XEN
>>>   	select DMA_OPS
>>>   	select SWIOTLB
>>>   
>>> +config XEN_PCI_STUB
>>> +	bool
>>> +
>>> +config XEN_PCIDEV_STUB
>>> +	tristate "Xen PCI-device stub driver"
>>> +	depends on PCI && !X86 && XEN
>>> +	depends on XEN_BACKEND
>>> +	select XEN_PCI_STUB
>>> +	default m
>>> +	help
>>> +	  The PCI device stub driver provides limited version of the PCI
>>> +	  device backend driver without para-virtualized support for guests.
>>> +	  If you select this to be a module, you will need to make sure no
>>> +	  other driver has bound to the device(s) you want to make visible to
>>> +	  other guests.
>>> +
>>> +	  The "hide" parameter (only applicable if backend driver is compiled
>>> +	  into the kernel) allows you to bind the PCI devices to this module
>>> +	  from the default device drivers. The argument is the list of PCI BDFs:
>>> +	  xen-pciback.hide=(03:00.0)(04:00.0)
>>> +
>>> +	  If in doubt, say m.
>>> +
>>>   config XEN_PCIDEV_BACKEND
>>>   	tristate "Xen PCI-device backend driver"
>>>   	depends on PCI && X86 && XEN
>>>   	depends on XEN_BACKEND
>>> +	select XEN_PCI_STUB
>>
>> Does kconfig not at least warn about this? The selected item has a
>> "depends on !X88" conflicting with the "depends on X86" here.
> 
> XEN_PCI_STUB != XEN_PCIDEV_STUB

Oh, sorry.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:20:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:20:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193754.345147 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMlq-0002zo-IR; Thu, 23 Sep 2021 11:20:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193754.345147; Thu, 23 Sep 2021 11:20:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMlq-0002zh-ES; Thu, 23 Sep 2021 11:20:06 +0000
Received: by outflank-mailman (input) for mailman id 193754;
 Thu, 23 Sep 2021 11:20:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTMlo-0002ka-Iu
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:20:04 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 31fd677c-1c60-11ec-ba2c-12813bfff9fa;
 Thu, 23 Sep 2021 11:20:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31fd677c-1c60-11ec-ba2c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632396002;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=vQOhuimDY2OitzZaBklOacvXpkqW1b3phNRavn8XKPQ=;
  b=S7Dl9eOp64T9FjnesVID6X67L6qSmAOB/Ir3Xyt0x1byn3ytE0PtDVRK
   muQsmsjY/NsnabWvmxYeFD21urfD8mS7/+3FBhVZZDfPP2V0g0QcxjNkF
   UDe0I8akGwNy9g6mzvYSM63nIzqD/f50DtgV/UafKXqbgPqw0eGPMGBSe
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: OC+pEBFdVQTpi4PNsDt5LIsrxzyDBVRuk6PowLN7HobNhdsnqkL6Rdv7PPDCqzqGKk3wXQz0vq
 YheaEDr+ZUmyLYLcpdCzk1rEdRQbnq/t3+o5LKTchJzCmmLoKyJlgCdsfxZvjafD7+l7ClHd0g
 rUi6C2ewx9GoKQpmandEqLymp8dHRzMOX4Q7uEi87cEBMi4qBHeFxHrHbln8pf9EYdX2ITqruZ
 +TQhczhGnGAPfpQW28r6BecpaWD6bqe5kfHQQ0eJ2vBVG0l7ULXGVuxNcvzfYCACW0XHA0xK55
 i2cFb2hNPdC/9oa9QF+hjnLD
X-SBRS: 5.1
X-MesageID: 53022233
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:OFGPb67bOQid2xdFhNwAjwxRtGjBchMFZxGqfqrLsTDasY5as4F+v
 mAWUW/SOP2CZWHwc4p+Oomy8kwOvsPVxtYxGwo/+C9gHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NUw2LBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z1
 el2jYyfYAcQZazHiv8scTxjNigmIvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALNs7kMZlZonh95TrYEewnUdbIRKCiCdpwgWxo3JoURq22i
 8wxYjxJUyr7eBZ2OxQnLLUAvdiCt0TcWmgNwL6SjfVuuDWCpOBr65DvP8DSYceiXthOkwCTo
 Weu13/iHhgQOdibyDyE2nGhnOnCmWX8Qo16PIO/8vlmkViC3Fs5ARcdVUa4ifShg0v4UNVaQ
 2QY8zQjhbI//0uqSp/6RRLQiHyOswMYWtFQO/Yn8wzLwa3Riy6JC25BQjNfZdgOsM4tWSdsx
 lKPh8nuBzFkrPuSU3313rWJqTK/PwAFIGlEYjULJSMa5/HzrYd1iQjAJv5iD6ydntDzASv3w
 T2BsG45nbp7sCIQ//zlpxad2Wvq/8WXCF5ujunKYo67xh5FX7a1SKWo0APa5uRbHaS8dXyI5
 GdRzqBy89syJZ2KkSWMRsAEE7eo++uJPVXgvLJ/I3Uy32/yoCP5I+i89Bk7fR0wY5tYIFcFd
 WeO4Vs52XNFAJe9gUabiaqKAsI2xODLEd3/X5g4hfIfP8AsLGdrEMxoDHN8PlwBcmBwysnT2
 r/BKK5A6Er274w9l1JaoM9HjdcWKtgWnz+7eHwC503PPUCiTHCUU6wZF1CFc/o06qiJyC2Mr
 Y0FbJDSk04BCryvCsUyzWL1BQpRRZTcLcqqw/G7i8bZelY2cI3fI6W5LUwdl3xNwP0Oy7agE
 oCVUU5E0lvv7UAr2i3QAk2PnIjHBM4lxVpiZHREFQ/xhxALPNb+hI9CJsBfVeR2q4ReIQtcE
 qBtlzOoWa8UFFwqOl01MPHAkWCVXE7w2F3Sb3b7OmhXklwJb1Whx+IItzDHrUEmJiG2qdE/s
 /un0AbaSoAEXANsEIDdb/fH8r97lSN1dDtaUxSaL99NVl/r9YQ2eSX9guVue5MHKAnZxyvc3
 AGTWE9Kqe7Iqo4z0d/ImaHb8Nv5T7ogRhJXTzvB8LK7FSjG5W7/k4VOZ/mFIGLGX2Tu9aT8O
 egMl6PgMOcKlUphupZnF+o51ro34tbi/ucIzgltEHjRQU6sD7dsfiuP0cVV7/Ufzb5FowqmH
 EmI/4ACa7mOPcrkFn8XJRYkMbvfha1FxGGK4K1sckvg5SJx8L6WamloPkGB2H5HMb94EII52
 uN96sQY3BOy10gxOdGcgyEKq2nVdi4cU78qv40xCZPwjlZ50UlLZJHRB3Ok4JyLbNkQYEAmL
 iXN2fjHjrVYgEHDb2AyBT7G2u8E3cYCvxVDzVkjIVWVm4Wa2q9rjUMJqTlnHB5Iyhhn0v5oP
 jk5PkJ4EqyC4jN0iZURRGurAQxAWEWU90GZJ4HlT4EFo51EjlDwEVA=
IronPort-HdrOrdr: A9a23:/Q4yPKwFCq9SvuX5sJOzKrPxv+skLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjnfZr5z+8J3WB3B8bfYOCGghrTEGgG1+rfKlLbakjDH4JmpM
 Ndmu1FeaLN5DtB/LbHCWuDYq4dKbC8mcjC74qurAYOPHJXguNbnnxE426gYzxLrWJ9dOME/f
 Snl616T23KQwVoUi33PAhIY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX222oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iCnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDw4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWAoqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocXTbqjVQGdgoBT+q3pYpxqdS32BXTq+/blkgS+pUoJjXfxn6ck7zE9HJFUcegN2w
 2LCNUwqFniJvVmGp6VP91xNPdfPFa9CC4kAFjiU2gPK5t3T04li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53022233"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YtoBd1HwCFP1G3xsIikhM+pqv6N45zxpjk/fAJjnRRSzYKGV48rjg6IRLOUFvQqSMalg84J/Fg+cCE77R0jxc5yac91g4WEjucweD8eUfBQEv3f+wgrWkgic8N3ROgjnWVA5udcmB8FTojdX3QawXGttxtQy7rDWGX0L9FbG0vvwSxThM5PzuWedpikY0g9d5Snc7rZS60jLpD62XZNbUdmX+LEnJIvu7aWDOxW1jYxDGPjULcQ00+rh2OrKkxQtlYO0lgtIBAgZw/XvSW4WIGu1p2i810PM3Ylr3aGBPjR69iNLde9Jr71bH/z4mjecsJpYULKHu99bFTNvK9W0Jg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dO+dY5cvH+56MllVutdEH/SCEq0JNQk2yLi4P15abpE=;
 b=mFfU+U575m2PTvZUfkJfLHcMl8PGEF0WSeMjXHANggisExMbJnG3GmQ94g6Q3dbMeq3tsUaOrLw6oCBpE0hF/hQ75UitIPcNif3QtQ964cRfu0bv5CydsFYqC/QyLpkRbCvtcucK/MZyE7GMIK893SoWKVjtamw3466SUeulDNIQijytveGTDKNIVUiT86HdH28ehJLXuW5NJY+EhzB6KxuKP+uYYTm0M+spglJ0rhtbjc8yR4HeC7B1sgtH1PwE++ctaqH6lAH0Pq1Z8hglnD7YjQPKMcLFp+HVIUlGUMo20vWXDv4BOUjbvwbGh5RKayOMazjEVnMvgfxRBG83qA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dO+dY5cvH+56MllVutdEH/SCEq0JNQk2yLi4P15abpE=;
 b=Qd+wEij/PyOKXqqerf1/Nd0bmUP9r9ccb2ksFQjQjB7D/9B3UCs4m3KCjaHmJ/RRBXdItA9g1yHArmlFn7pawUqzfEyicIkRmawn0C1b00joKfjZPVUXeULfJ8cEZ5FWdy5WQFlJmJ1IzdBhUvL4tGzOFCS2Eof9eYksCPAHztU=
Date: Thu, 23 Sep 2021 13:19:52 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, "Wei
 Liu" <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, "Christian
 Lindig" <christian.lindig@citrix.com>, David Scott <dave@recoil.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
Message-ID: <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
 <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
X-ClientProxiedBy: LO2P265CA0418.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::22) To SA0PR03MB5610.namprd03.prod.outlook.com
 (2603:10b6:806:b2::9)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6a5b965f-1f8f-4ff6-2f8d-08d97e84135e
X-MS-TrafficTypeDiagnostic: SN6PR03MB3744:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SN6PR03MB3744738B120719013C68701A8FA39@SN6PR03MB3744.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1HcKAVmJqr6Xf2UhaZDNR2iDEuVgk4y3vFfa4htmTy5CLmMRMZ2XpvhbP7k4Ayv86xLaejRaUw9riMV/W05OAtkPmkEr9lgFIUd8Xgjx20nGURsgQaEQusyvxny9pb4UI7jXjdiuP4JKhLsAfc/gpCZ1HmIvTitArBcnregikEw95XnQjRwAp1PUau1Jg6B4U21NvgQ46KMFH2ZUvvJrRmudT5e+tRhodCCR1oTuCZ8FxSGCqVTltqil95iBJa8tfBy182/WIlaSAiH5niskV6JHKUqf4B0hHW5SD5ftgmHOt8+i+o2aIGZG2ZsuwZIpW87/gokb6zuZeAUkmGYvLkXQeBbDYCu0xYm3WSOI43olzCC84zUrSBv50b1Kqu7jI8RipK+yt4x3EzokiK2OAnOKwk0jiF9RpBmeldide/UF+5sohdUpm87uD9uFE1fRcoL/xkB2GjuMXna77Teu2ooa3SpLTlkbOuilRrQgAHD3gBuOnR6k4NFzfbcXoDO3bw2nwYTV/Nwyft/QGorYZUFZn0n2e6585fnDqvDw1tL6dlkl0E4DfnIZmA6I+CQFzShnbTa4BlXK/ijWyMQFDYGM+SV546ZdFx3sBfLwl9DSjdL8IBDMTgSJzYFKIuguniPDH1amcJ2ERinshbIlLy71a2DQRS3GBRjrzvpfphpP0pj9KRVADA4uMUwUt4gwNjpEqejLTff5IhXgPZhcwk4X1Q3sHtQfIDoVTVRtCUc=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA0PR03MB5610.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(6486002)(54906003)(26005)(4326008)(186003)(8676002)(316002)(6666004)(8936002)(6496006)(38100700002)(6916009)(9686003)(53546011)(5660300002)(66556008)(66946007)(86362001)(2906002)(66476007)(85182001)(83380400001)(508600001)(966005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?L05EUlcyeWFhZ2Q2Q01UNHlEV2lUTVU2M09oQ1NGWTZUNWVSd1RPQmpibjVG?=
 =?utf-8?B?eHhGbVFqQ1FWaWFLR2JjQlFWYlI5anB1UDR2QWRQYk9UaWd0SEN3Q1RrOGVH?=
 =?utf-8?B?TlR5TU14cUs5Z3doRWo1bVpvbnQvUGJua3BGVm5aVWRoMHFhdHUwY1dpeHR6?=
 =?utf-8?B?ZUhrVFlxN0xBMDBqSkxlcEprdTZqQTRKUWVVcittNEZHemVxdjRVakc2MGpO?=
 =?utf-8?B?VjZUVW9lQitIZFBMemc4bkp0SkdxMWYzVkJIVGJLZTJkNnJvK1dpcWt3T3M3?=
 =?utf-8?B?M1RNQ1IxR1VqUU1obUpFOHZwblFseXVlVjBlNEtOY2dzblpoTjhULzVpcHVC?=
 =?utf-8?B?Q0dybUs4VStDcnRTMEUwaHlzaVpLL0xvekpvZmFDSENNRkl1YjNYNTlSSWg1?=
 =?utf-8?B?dVcxZ0lPTWl2UXpXRHBPUWFUNTA3VlJlM0pZaTdtK2wwckhNNjBUbUpBRFlS?=
 =?utf-8?B?MWNlazd3TiswK2EzMDk1Q3dwSlEvaUpGWTNrM0dRTWltTExqOC9RMG42anFm?=
 =?utf-8?B?Y2E1RXRKVnZhRlR0MXZJN1ZJTUhoV1NtWGI2NHZwODBYeExTTlRlVHdVK2Za?=
 =?utf-8?B?RjhEV3J0VHlCMkRrZ1pQQjNEZFpBK1M3NGZNWTVaLzZaOGJ4YldMV2doSTFH?=
 =?utf-8?B?NWJZOXdZa2syR20vTmtNVXpFOHNyVEx3clNoMDVQeEQxTGxqaVdDQ2pRMHhY?=
 =?utf-8?B?Znp2RXZhcUR4SU9EVVNnaWV6S3NheENJZFBidHpMZ20weTlQZmtMSUc0bmNE?=
 =?utf-8?B?S0orcG5FUUkyaEpEZDVsNEQzMHpORjYrT1Jtd3oxVWk3VzJvQ2dNeStWbjVs?=
 =?utf-8?B?T0k5YnZ1UDZMWmIzaG9lMTBKTGMwcEx1OWxadDhKdUYwOXFEdG9reTJvdGpy?=
 =?utf-8?B?YzM5SGx3UGN1ZHk1V09FYXlScHFMbWNZNzRwM2VCbStjZmx0Y2kvSThKSkwr?=
 =?utf-8?B?WGNmREJpR0xON3NibmdQbGtxbzYzc2FsZjc5eUdCNWxQYXhUZzl6VURvbWFQ?=
 =?utf-8?B?N0hwdVFnUGxCZUEydEVKVEx6YW1LRzhYZHIyZGREZ2JnZ0gvQWFsLzJ1N2ww?=
 =?utf-8?B?TWxFdWpYcFVHemFiL3hXMENXZlFyT3JzVFdCSW9XTDJGckFCbWxtV1AvRkNi?=
 =?utf-8?B?WnB4MG5pNEVUei80c3hpdlg3OVhMOTNabi9HT00wbitaWThFUmxxSUlsWGZt?=
 =?utf-8?B?TzlwVTlLSU1STHREMFFHTlQweWd2VHFuc2Z0SW9TbVliNmxpWG15Y3IxRlhL?=
 =?utf-8?B?clJLaHBZTTMvNkpETzlsSHEyVEt2NWxkeHZWdEVndlo2VmdOdFpKVzRvVjJS?=
 =?utf-8?B?U1ZjbjU0cXZqcGRCd1ByZElVUlNPV3Q1QXZ5OVhDN3VhY0pFZG9ReUhORk12?=
 =?utf-8?B?RmVZWmRJOWVkcXZqd1pvWGdMV3RKTDVtbWtBTW9ISDFwT09OQWtVSERja0M4?=
 =?utf-8?B?eDFDVUtXcGRrNmpQNTZGMXVINkdmSTZGdVdMZ1lDU3dQYW1Hemc5Y3orVUUv?=
 =?utf-8?B?UTIrc044cFFVKzAraTA1YTRBVW8vUWlFYS9EQjNQTWEvWXBHU0JHV0tmb29S?=
 =?utf-8?B?UXFhOGxqVFlWNkJDME5yU3ZFS0dRMjZDWjFyWUlsd0RnUnI4SkoxNEV0RGZB?=
 =?utf-8?B?eUtNOXJzV3czQzFadlIxVUNSWFBRMDRtV1Rtc2xVK2Ztbm9yblVhM3Zhb1c1?=
 =?utf-8?B?d3QyeDF6U08yY1UrSXpLNnZ5L0lqbUozanptaHRSaUo4czdWWHhlY295SWV3?=
 =?utf-8?Q?jELdfTsORK/I9+PaSaFkeKylGDvi7prGwBZRAtf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a5b965f-1f8f-4ff6-2f8d-08d97e84135e
X-MS-Exchange-CrossTenant-AuthSource: SA0PR03MB5610.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:19:57.7060
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: E+DgiLG1AaHhriTrWWVon9di6q/ckcFsEneyqOs0KefH0QnGm6pMSEyX7c1eO9CJwpORfnstmbLXpN3zc7xawQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR03MB3744
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
> Hi Roger,
> 
> On 22/09/2021 14:39, Roger Pau Monné wrote:
> > On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
> > > 
> > > 
> > > On 22/09/2021 13:21, Roger Pau Monne wrote:
> > > > Hello,
> > > 
> > > Hi Roger,
> > > 
> > > > First patch on the series is a trivial change to xenconsoled in order to
> > > > use xenforeignmemory stable library in order to map the shared console
> > > > ring instead of the unstable libxc interface. It's reviewed and ready to
> > > > go in.
> > > > 
> > > > Patches 2 and 3 allow setting the host wide command line `gnttab` option
> > > > on a per domain basis. That means selecting the max allowed grant table
> > > > version and whether transitive grants are allowed.
> > > > 
> > > > The last 3 patches attempt to implement support for creating guests
> > > > without a grant table. This requires some changes to xenstored in order
> > > > to partially support guests without a valid ring interface, as the lack
> > > > of grant table will prevent C xenstored from mapping the shared ring.
> > > > Note this is not an issue for Ocaml xenstored, as it still uses the
> > > > foreign memory interface to map the shared ring, and thus won't notice
> > > > the lack of grant table support on the domain.
> > > 
> > > I find a bit odd that the Xenstore support is conditional to whether grant
> > > table is available. Are you expecting domains with no grant table to have no
> > > PV drivers (including PV shutdown)?
> > 
> > I don't really expect much, as having guests without grant table is a
> > developer option right now, if someone wants to make use of them for
> > any reason it would need some thought.
> > 
> > The other option would be my first proposal to restore foreign mapping
> > of the xenstore ring on that case:
> > 
> > https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.com/
> > 
> > But it's also arguable that a guest not having a grant table should
> > also likely prevent foreign mapping attempts. Plus such foreign
> > mapping won't work from stubdomains.
> 
> There is another option: extend the acquire hypercall to allow xenstored
> domain to map the xenstore interface. This would require more work, but at
> least it would avoid the interesting dependency on the grant table.

Xen isn't aware of the shared xenstore ring page currently, so that
would mean introducing more knowledge to the hypervisor that what's
strictly required IMO, as Xen has no business in knowing such details.

The grant table slot used by the xenstore shared page is just an
agreement at the toolstack level, but not known to the hypervisor so
far.

> > 
> > I'm fine with dropping those patches if they turn out to be too
> > controversial, I think it's an interesting option to be able to
> > disable the grant table, but I don't have a full picture of how that
> > could be used in practice. Maybe others have and would be willing to
> > pick this up.
> 
> I think the current approach is probably OK as a developper option. However,
> we should at least document in the option that disabling the grant-table
> will also disable Xenstore (anything else?) support when using C Xenstored.
> 
> It might also be worth to clearly state in the doc that this is only
> intended for developer use and not supported.

Sure, adding it to xl.cfg man page is likely the best place. Will do
when updating the patches.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:20:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:20:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193757.345158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMmA-0003YB-U4; Thu, 23 Sep 2021 11:20:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193757.345158; Thu, 23 Sep 2021 11:20:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMmA-0003XX-Py; Thu, 23 Sep 2021 11:20:26 +0000
Received: by outflank-mailman (input) for mailman id 193757;
 Thu, 23 Sep 2021 11:20:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTMm8-0003Qq-LL
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:20:24 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0d::60d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 916ab67a-bcb6-4f4c-9e93-14184f5cbd2a;
 Thu, 23 Sep 2021 11:20:18 +0000 (UTC)
Received: from AM5PR0201CA0002.eurprd02.prod.outlook.com
 (2603:10a6:203:3d::12) by VI1PR08MB4301.eurprd08.prod.outlook.com
 (2603:10a6:803:f7::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 11:20:00 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:3d:cafe::a) by AM5PR0201CA0002.outlook.office365.com
 (2603:10a6:203:3d::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 11:20:00 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 11:20:00 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Thu, 23 Sep 2021 11:19:58 +0000
Received: from 3195baacd5a9.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 713CBD3E-6E15-47F2-AA04-266BDB8E2876.1; 
 Thu, 23 Sep 2021 11:19:42 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3195baacd5a9.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 11:19:42 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1939.eurprd08.prod.outlook.com (2603:10a6:203:4c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 11:19:30 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 11:19:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 916ab67a-bcb6-4f4c-9e93-14184f5cbd2a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lj1GC+tv5scfGOtQDJe05RRjCb7Emx137fY9thBEOkU=;
 b=v2cN8SwNLER6mQ50fMVQIvJn6rLZaRzzazk3IWdzqGPQn9uR/bGhCJisHXZxjprQ5L/SFIyCUTE40Lp6OdyGdAqXaBW36ueeBCy04For9GtYbLj1J7+FTqx3Ebrwi9j4EtIERqt/m6YH6kNoqa14No0oqYAmomF14eGCZYjZWPE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e8861d3b3784b02b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kmBIiDDaMiUWRQCd6cISu4L0yI+u1DVtbVfSd76XeTFh+ApwNnPMgNdgMXjOxqQCK2xmfPh9kuCcS5omwutph1BUYkI92xfFMky6NmqAFb60og3DO6JXy9GymH3stg0aNU26Wlpxnlv/UTWpSh9sfx29U4t8ghl55n0zssIc7FGUYzcKQBhEblmOuHxeGAwpmz6Hz1I+Dadfk5fQnNumCW5EEWgyv6mh8IvX5WS+k9iGkLB5SbdJunEpW8ATclUBaqBbEwHwvmGJf46y6Dn8rpQQ2I/aZRKC8QQqtH7Z+ZiL2rOL/XbWQACiRGJw6buWvN/7filwlGE8FHo62bSOEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Lj1GC+tv5scfGOtQDJe05RRjCb7Emx137fY9thBEOkU=;
 b=LG9pCO9Nqk7rIxNLKdUPKr6dgjj/fpvEy2Q3uIy+s2AupdJCp4jKQRU50M+9bL1Gtsj52KkGJga6QBFI9KpCXNSRyt/oaN4kKtzh7k0gPc0Ckj46DOZ2mZJj34/yFHQH0nP7M/2npg94W1y4W8xsYjU9YiD4lxRB5N9DKrW930TdpdCNo1yyQH2XaK13P1xDJ1Mei0t0YVaa3RMbQSY8EfHMKgWjpm1iTFS+p5tXCkL5582K7tPVH8jXA4JmakDMn72VFP3SmRfts+Vb+vkohACGGNXGImjdAmrJXM/H2ZJR94Kxkvif0xm7vCLDADi8JwNQ4QDV8BIdYWUZp68yew==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Lj1GC+tv5scfGOtQDJe05RRjCb7Emx137fY9thBEOkU=;
 b=v2cN8SwNLER6mQ50fMVQIvJn6rLZaRzzazk3IWdzqGPQn9uR/bGhCJisHXZxjprQ5L/SFIyCUTE40Lp6OdyGdAqXaBW36ueeBCy04For9GtYbLj1J7+FTqx3Ebrwi9j4EtIERqt/m6YH6kNoqa14No0oqYAmomF14eGCZYjZWPE=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>
Subject: Re: [PATCH v2 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for
 ARM
Thread-Topic: [PATCH v2 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for
 ARM
Thread-Index: AQHXr6Y8iuh9n6XE8kGpK3+HP56+I6uwpZaAgADU3YA=
Date: Thu, 23 Sep 2021 11:19:30 +0000
Message-ID: <59B54064-1DE0-44ED-9B50-F1709B77E56B@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <a1c447ced213f07797eebc9471c48df56a187033.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221527510.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221527510.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 65c2d1b8-0848-4cf2-0f79-08d97e84150f
x-ms-traffictypediagnostic: AM5PR0801MB1939:|VI1PR08MB4301:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB43018BFF16B762CBF7D90BD8FCA39@VI1PR08MB4301.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rv09D2lp3+r8ELMn40Lj2Wew2Hp9P4WH9kWSLnFenKkxlamGq0UoekbB2UJ9rtl5XFGQ/k46n9XgoM1k+wugL7FCE4tgBZrAvQm+Ot3maSMbCLy68m6eSPIpESBXQBCdA12oxw1NTRgv9umtKAwpWXMeLXiQG7MxvGM4Ew2cPCvbm4VhJTLKWdGU9MX14kctzlx3qybXJJ5lwP00tDJDTGO/fGFI2f55RHe/cyuZWPoHLyUSt4hn76wG4MdqjPw5CJb75GqirwFrJsUdlsQjTNBo65gzaGCTeCWnm1js8f7XEEdGffZqhnFHSu/PXDpfwxU7/m5nCt90oFfI0FITjgpGIZE25CF6jBYs9G7JEr0Ed6oXGyGu91E5hYbqOZ0PG9+87a6nDAUTmuZ5pQQmAw3i8pBYYLXQBFQ/cLLXluq50XJNOwWL3NVvuzCZt6h3Xcd3F5h8MBwvgbVb6FoYOoZtr95vKcL8Ia7CJCLNIFUTH71+074XMDe58psc99pNtn3orAmcr908LJ5EoQIuoddUwWT5hRxSy14oAKbtN2ks7g7sMJlDMQjkULin7sVAFKTw6dOksv2n2j6ZJnE1T8oCHQo0TgUww7Jo+zCfeV7G/LJeUESrCuG/Pn1hyno3DG9AqL9XKd8bspDk9D8j45SR9Vmr7AhUGCMZkPfrfLJVCQVj0yWeHP4tKJ49lQD1/pgcUCt4i7y4BbvbP+FolMN+t48PiX27dHWBVvUpPOfSttlhsPTRLM1BhvfkzV7D
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(91956017)(83380400001)(38070700005)(6512007)(86362001)(6486002)(8936002)(71200400001)(64756008)(122000001)(6506007)(38100700002)(5660300002)(2906002)(186003)(8676002)(76116006)(53546011)(33656002)(66476007)(6916009)(36756003)(2616005)(54906003)(66446008)(316002)(66946007)(26005)(7416002)(4326008)(66556008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <695C343121B23B44BD2B70DB5FC4D837@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1939
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	91909f59-f5dc-4e7b-34b0-08d97e840319
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rWZ+g9J2vukW68WOas0KZsDEY4tBkTk1rfentweVabW5a9qcrUcOGkM2g3ul6Mfl/G/+mOBTz8iVXpWIVcqLZNLf1EPuDoRWQIKD/S9GDR8s2oph0kdrmOpUvR+DbuJt27U8F6M3HBRsSAD3eYx5irZZEaqhW0WRIqJjjLbLBq/3uWG8nhPKtzAj4/UvkDLM/nLycXN4EZCAIUsYFVlDNvLr30TFv40MOCeX/5DBA6jr1fvY0YQgB0ULjAASCvCxHJP/0l5Vkt0v80+XsJfGTIrReUzeDV1XT8rSRtSXlGEknZAgxSrnTQ7zUrscIOH3QobOeqLakSCQ32LgB7peM+lmCnJmheQkgopC4tWw7eNaLleV0f8ys8PpMWZrCO8zLnlNr/BDFpKuzALK0RZQupltaWkrEvKlE5+jiWiGF49Awpfb43OQlQ+X5D08mjGPdjYnHGHvvwbTTbPeMBveJztmwKqB6IYKMIB988SfNyc4eHdhjLhCeAzMoM8gEMnnbch5q2K8ybdCkFaqmRoq95G7r//Z62t8ipcxBje4m1vHv75LJUkkmhv5ExoUlGCr/ow8SbBcYnfmoZC8Ut/KhQmNM45ZopdxxFGLuCz51KwGNYiBz9zyJmFip8kQHXt1PqrTll1OB52udeDIlzkiAmT+CQtvA3QDIb5CwpXPpdVlgtthxHJvnsHvmQh0nitBQv4YIFhabdv+NXMyIVQ8VA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70206006)(70586007)(81166007)(54906003)(508600001)(6512007)(83380400001)(336012)(316002)(8676002)(36756003)(82310400003)(5660300002)(2906002)(107886003)(26005)(33656002)(6506007)(36860700001)(4326008)(86362001)(6862004)(47076005)(53546011)(186003)(8936002)(356005)(2616005)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:20:00.1568
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 65c2d1b8-0848-4cf2-0f79-08d97e84150f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4301

Hi Stefano,

> On 22 Sep 2021, at 11:37 pm, Stefano Stabellini <sstabellini@kernel.org> =
wrote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Hardware domain is in charge of doing the PCI enumeration and will
>> discover the PCI devices and then will communicate to XEN via hyper
>> call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
>>=20
>> Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
>>=20
>> As most of the code for PHYSDEVOP_pci_device_* is the same between x86
>> and ARM, move the code to a common file to avoid duplication.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2:
>> - Add support for PHYSDEVOP_pci_device_remove()
>> - Move code to common code
>> ---
>> xen/arch/arm/physdev.c          |  5 +-
>> xen/arch/x86/physdev.c          | 50 +-------------------
>> xen/arch/x86/x86_64/physdev.c   |  4 +-
>> xen/common/Makefile             |  1 +
>> xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/hypercall.h |  2 -
>> xen/include/asm-arm/numa.h      |  5 ++
>> xen/include/asm-x86/hypercall.h |  9 ++--
>> xen/include/xen/hypercall.h     |  8 ++++
>> 9 files changed, 106 insertions(+), 59 deletions(-)
>> create mode 100644 xen/common/physdev.c
>>=20
>> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
>> index e91355fe22..4e00b03aab 100644
>> --- a/xen/arch/arm/physdev.c
>> +++ b/xen/arch/arm/physdev.c
>> @@ -8,10 +8,9 @@
>> #include <xen/lib.h>
>> #include <xen/errno.h>
>> #include <xen/sched.h>
>> -#include <asm/hypercall.h>
>> +#include <xen/hypercall.h>
>>=20
>> -
>> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>> +long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>> {
>>     gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=3D%d: not implemented\n", cmd)=
;
>>     return -ENOSYS;
>> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
>> index 23465bcd00..c00cc99404 100644
>> --- a/xen/arch/x86/physdev.c
>> +++ b/xen/arch/x86/physdev.c
>> @@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
>> }
>> #endif /* COMPAT */
>>=20
>> -ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>> +ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>> {
>>     int irq;
>>     ret_t ret;
>> @@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM=
(void) arg)
>>         break;
>>     }
>>=20
>> -    case PHYSDEVOP_pci_device_add: {
>> -        struct physdev_pci_device_add add;
>> -        struct pci_dev_info pdev_info;
>> -        nodeid_t node;
>> -
>> -        ret =3D -EFAULT;
>> -        if ( copy_from_guest(&add, arg, 1) !=3D 0 )
>> -            break;
>> -
>> -        pdev_info.is_extfn =3D !!(add.flags & XEN_PCI_DEV_EXTFN);
>> -        if ( add.flags & XEN_PCI_DEV_VIRTFN )
>> -        {
>> -            pdev_info.is_virtfn =3D 1;
>> -            pdev_info.physfn.bus =3D add.physfn.bus;
>> -            pdev_info.physfn.devfn =3D add.physfn.devfn;
>> -        }
>> -        else
>> -            pdev_info.is_virtfn =3D 0;
>> -
>> -        if ( add.flags & XEN_PCI_DEV_PXM )
>> -        {
>> -            uint32_t pxm;
>> -            size_t optarr_off =3D offsetof(struct physdev_pci_device_ad=
d, optarr) /
>> -                                sizeof(add.optarr[0]);
>> -
>> -            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
>> -                break;
>> -
>> -            node =3D pxm_to_node(pxm);
>> -        }
>> -        else
>> -            node =3D NUMA_NO_NODE;
>> -
>> -        ret =3D pci_add_device(add.seg, add.bus, add.devfn, &pdev_info,=
 node);
>> -        break;
>> -    }
>> -
>> -    case PHYSDEVOP_pci_device_remove: {
>> -        struct physdev_pci_device dev;
>> -
>> -        ret =3D -EFAULT;
>> -        if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
>> -            break;
>> -
>> -        ret =3D pci_remove_device(dev.seg, dev.bus, dev.devfn);
>> -        break;
>> -    }
>> -
>>     case PHYSDEVOP_prepare_msix:
>>     case PHYSDEVOP_release_msix: {
>>         struct physdev_pci_device dev;
>> diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev=
.c
>> index 0a50cbd4d8..5f72652ff7 100644
>> --- a/xen/arch/x86/x86_64/physdev.c
>> +++ b/xen/arch/x86/x86_64/physdev.c
>> @@ -9,9 +9,10 @@ EMIT_FILE;
>> #include <compat/xen.h>
>> #include <compat/event_channel.h>
>> #include <compat/physdev.h>
>> -#include <asm/hypercall.h>
>> +#include <xen/hypercall.h>
>>=20
>> #define do_physdev_op compat_physdev_op
>> +#define arch_physdev_op arch_compat_physdev_op
>>=20
>> #define physdev_apic               compat_physdev_apic
>> #define physdev_apic_t             physdev_apic_compat_t
>> @@ -82,6 +83,7 @@ CHECK_physdev_pci_device
>> typedef int ret_t;
>>=20
>> #include "../physdev.c"
>> +#include "../../../common/physdev.c"
>>=20
>> /*
>>  * Local variables:
>> diff --git a/xen/common/Makefile b/xen/common/Makefile
>> index 54de70d422..bcb1c8fb03 100644
>> --- a/xen/common/Makefile
>> +++ b/xen/common/Makefile
>> @@ -29,6 +29,7 @@ obj-y +=3D notifier.o
>> obj-y +=3D page_alloc.o
>> obj-$(CONFIG_HAS_PDX) +=3D pdx.o
>> obj-$(CONFIG_PERF_COUNTERS) +=3D perfc.o
>> +obj-y +=3D physdev.o
>> obj-y +=3D preempt.o
>> obj-y +=3D random.o
>> obj-y +=3D rangeset.o
>> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
>> new file mode 100644
>> index 0000000000..8d44b20db8
>> --- /dev/null
>> +++ b/xen/common/physdev.c
>> @@ -0,0 +1,81 @@
>> +
>> +#include <xen/guest_access.h>
>> +#include <xen/hypercall.h>
>> +#include <xen/init.h>
>> +
>> +#ifndef COMPAT
>> +typedef long ret_t;
>> +#endif
>> +
>> +ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>> +{
>> +    ret_t ret;
>> +
>> +    switch ( cmd )
>> +    {
>> +#ifdef CONFIG_HAS_PCI
>> +    case PHYSDEVOP_pci_device_add: {
>> +        struct physdev_pci_device_add add;
>> +        struct pci_dev_info pdev_info;
>> +        nodeid_t node;
>> +
>> +        ret =3D -EFAULT;
>> +        if ( copy_from_guest(&add, arg, 1) !=3D 0 )
>> +            break;
>> +
>> +        pdev_info.is_extfn =3D !!(add.flags & XEN_PCI_DEV_EXTFN);
>> +        if ( add.flags & XEN_PCI_DEV_VIRTFN )
>> +        {
>> +            pdev_info.is_virtfn =3D 1;
>> +            pdev_info.physfn.bus =3D add.physfn.bus;
>> +            pdev_info.physfn.devfn =3D add.physfn.devfn;
>> +        }
>> +        else
>> +            pdev_info.is_virtfn =3D 0;
>> +
>> +        if ( add.flags & XEN_PCI_DEV_PXM )
>> +        {
>> +            uint32_t pxm;
>> +            size_t optarr_off =3D offsetof(struct physdev_pci_device_ad=
d, optarr) /
>> +                                sizeof(add.optarr[0]);
>> +
>> +            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
>> +                break;
>> +
>> +            node =3D pxm_to_node(pxm);
>> +        }
>> +        else
>> +            node =3D NUMA_NO_NODE;
>> +
>> +        ret =3D pci_add_device(add.seg, add.bus, add.devfn, &pdev_info,=
 node);
>> +        break;
>> +    }
>> +
>> +    case PHYSDEVOP_pci_device_remove: {
>> +        struct physdev_pci_device dev;
>> +
>> +        ret =3D -EFAULT;
>> +        if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
>> +            break;
>> +
>> +        ret =3D pci_remove_device(dev.seg, dev.bus, dev.devfn);
>> +        break;
>> +    }
>> +#endif
>> +    default:
>> +        ret =3D arch_physdev_op(cmd, arg);;
>                                           ^ a typo?
Ack.
>=20
> The ARM and common parts are fine. I am not well-versed in the x86
> compat stuff; we need one of the x86 maintainers to review the x86
> changes.

Yes. I need one of the x86 maintainers to review the code.
Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:22:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:22:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193766.345169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMnv-0004Jx-A5; Thu, 23 Sep 2021 11:22:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193766.345169; Thu, 23 Sep 2021 11:22:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMnv-0004Jq-69; Thu, 23 Sep 2021 11:22:15 +0000
Received: by outflank-mailman (input) for mailman id 193766;
 Thu, 23 Sep 2021 11:22:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTMnt-0004Jk-Vc
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:22:14 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe05::618])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5fbd94c6-a732-4048-a695-8cec5a3abce0;
 Thu, 23 Sep 2021 11:22:12 +0000 (UTC)
Received: from AM7PR04CA0001.eurprd04.prod.outlook.com (2603:10a6:20b:110::11)
 by DU2PR08MB7270.eurprd08.prod.outlook.com (2603:10a6:10:2d5::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:22:08 +0000
Received: from AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:110:cafe::f8) by AM7PR04CA0001.outlook.office365.com
 (2603:10a6:20b:110::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 11:22:08 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT042.mail.protection.outlook.com (10.152.17.168) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 11:22:07 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Thu, 23 Sep 2021 11:22:06 +0000
Received: from 59b5cb761934.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D1354835-735E-46F1-9A2A-82A382476037.1; 
 Thu, 23 Sep 2021 11:21:57 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 59b5cb761934.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 11:21:57 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0802MB2499.eurprd08.prod.outlook.com (2603:10a6:203:a1::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 11:21:53 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 11:21:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5fbd94c6-a732-4048-a695-8cec5a3abce0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JIZt/hsSKmEAvOlJm6ihDHlyDQUvKbLmTARD9HJTuiA=;
 b=d9dAoqKe++gO/YfN3fT/gwWBRvXf2EIZ9fKFsFjMXP07YV2m7x/lTAt4R817C1ux61f4kJLi1w2BbAqYSP0H8X8B5rXa93INVlxQsyDxLCicLe8ijk/Aey3ldIbdBWqUvyg3z8mtssURr8OPNqUitUQGdf6MGLL9BIdTgQ7oliI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: af0e7891f03c5b7f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vz6CjvnGrpG/7KjQ1L9vYOgiEUSEecRncedvbIJ3AP7WPJQ34pZaNNv3nUgwChxe/iuAnaMFBRf8NCZCXh12MDvADUs/q9zHTSPGAAjNCjFGJw5x2ySt5HDdhIk3Jjwp/XImHwoXSqAPJYz4WtWHbJ56lE7pqmPKrHJyefnSb1nTnVlN9DtiYX26XvUSqNUMzg3hGi+5dj58yyzbnBzW57n4r2TPGJjHd7LFMnC3U3ROHIGEep67EESyiY6llCkinyGp88rNHUW+EuECIqUJwQA65KtjAasBUhdDAHtIJz0jP3TlJz/goreT/5eREDOe6kPlxQG0bgSU9HBPhT5ZwQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JIZt/hsSKmEAvOlJm6ihDHlyDQUvKbLmTARD9HJTuiA=;
 b=C6A34WR7FMuemcr/iSZhA65RRPyU1IMi6/T8mRG0KZowpCssA1W8sUfSD/5WHcaEIdWHFsBuPjyo5zkAsHN2+K1Ckldnmm6uOpMB3W7f9rdTp+SCDtAzcZzvxX9aUWvFsnJxHfwOi1pAZvW4zAlMTjppC2Lfw6ie7m/xmjJ62KxJm9neWUTvjsuaUGVC3kdL34mhK+3f8zf0DGOhSP/V4HeGLSD6r50YtzzhbTrNC1jzTxax5UOpTwV8jvOLeYUDEKZX4B79heevsU1Cz3zQDb263j2lhl66fXO1AdmeNg+BfjZ41yjS4NKvIjQG2D5tWrdHp/nyVpM/DKZvU2X0NQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JIZt/hsSKmEAvOlJm6ihDHlyDQUvKbLmTARD9HJTuiA=;
 b=d9dAoqKe++gO/YfN3fT/gwWBRvXf2EIZ9fKFsFjMXP07YV2m7x/lTAt4R817C1ux61f4kJLi1w2BbAqYSP0H8X8B5rXa93INVlxQsyDxLCicLe8ijk/Aey3ldIbdBWqUvyg3z8mtssURr8OPNqUitUQGdf6MGLL9BIdTgQ7oliI=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>
Subject: Re: [PATCH v2 06/17] xen/device-tree: Add
 dt_property_read_variable_u32_array helper
Thread-Topic: [PATCH v2 06/17] xen/device-tree: Add
 dt_property_read_variable_u32_array helper
Thread-Index: AQHXr6ZA9+m8mQY/6E2BJMEbqtI5eKuwrbKAgADNawA=
Date: Thu, 23 Sep 2021 11:21:53 +0000
Message-ID: <8448D6A9-2152-40B8-8159-65C1AA3D781D@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <35eb7ec429da3a9e1af626af4521c13c9683f8fd.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221601240.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221601240.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 22645990-f420-4796-c788-08d97e846101
x-ms-traffictypediagnostic: AM5PR0802MB2499:|DU2PR08MB7270:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DU2PR08MB7270FDA10590F813C50788ABFCA39@DU2PR08MB7270.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2733;OLM:2733;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 a2L3B3NoUyPIAXD60atMSft9WU5+v+edLoIQ+38DCCt9WVI7gL/BKlV97XFo3hexjuh6d0O40UfjKzewL3Ns28YHes3yzeU4n+tDU5EKnKLGEDtEz1UyacpIpaJQRLxpYF4Er2rOL26tdVb+nCJdLJ7Tg+yussjydVYlzYYBoJ7MvIPtXI9JT0LKWQfNf5De/Of4PZaN8UHjOPyK0N/VJdP5fHA+XqrITw0VF7aQuGJ+vwIMzKDdR4eQPcnjmHcEf3DInrz5VxVlrCHYhju2hREUtP7jBEFdQXSHyAby5nPHg/icjKsUestf2v6mBUwZB/qQKf7KAjc/DnWDIbfeVO8qaQXQHY4mbwIKkan3O8btNe7xUC/IilQwt3i42pcORvZ2TUjzp+z4PCxA+EjOZGjB7CshDNQxm/8cpom7WNdkWX1HJMPX/0GnsUnK0OgbaKtsP15C6/eoljBhAlaGp32z3SHCYJgReV1uU8OalbOtEnr3whq/S5EjyhNuLlvZXPmeIxJGljWDTknn3E8gRp0D5kU6EMUavfNjp49jZ1XVRr/lnTY4PAIDOO8p8WPyinby2KM24aBhLmZJQXECuA0sI/pHdGfgOhg5kUc4VnSUbjSIRqzI/9ge9wzKI7ud3GmEWt4KqFYiJ3WVF2JPy0GDgSKWgWZ3xbtA7SLtqEFSjR888vSxmnUW9UKbtAwR6p25Fg4xRTd/Qfwfx/tgv7E51faR1ygwadhheYLPXuYFjRBX1e9liswedQP/r+cS
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(36756003)(2616005)(508600001)(86362001)(186003)(71200400001)(38100700002)(122000001)(38070700005)(33656002)(26005)(83380400001)(66476007)(66946007)(6512007)(8676002)(4326008)(66556008)(66446008)(6916009)(76116006)(6486002)(316002)(91956017)(5660300002)(54906003)(2906002)(53546011)(8936002)(6506007)(64756008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <410ACCC7E89BC94EA4817D171A56BBB3@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2499
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	749a7ea9-cce1-41d0-c234-08d97e84585c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XX8Q+SeOH6WNHAxLd4LwYfF/c600+448YYVN0DFWHg2IZwDoX0P5JnpStxoapXp+XxmDN/MRXlSxEZmNMSnC5c2OCFBosGil+QChA5lUgGhVybdrOXqIOxc5ODZs2r78dqTWRIwEPAbjFtOnKul3jiK7PS7Uaenl1ZyL44DQMu5GL62dINrMJvO6socatZiDq18t+IniyoMbsa8AJaqx5QiGz7AImPXMgJ/eSD/xP4tWUeOAIP1Ds379J+GtVKHZ4hW7EJoihhwSLpsKv3GVX3BvVN8apU5n+uiloy907KkRmvYWv5XWEaWM2Fto4ehsSDalFqxYTkuffw+Mk50J9TxS3XhC+/qr0R0IgEYYx+ZixiimmGP7Sz5usH75OSNUnCACbfYkBBed1cYUNAhhxQYqE5joeebahCYVadSZkg3fzZCHnUrF69g1KRAfDfoSgA4zVF/pRG0+qrZqErxZdXG34iMcZRyz9e7GBfu0l/DvzGmszr9ykU8FF0kVGm6bSNRI69/L0f9bTSEtM2yiOVZBQfbxYzO/51RyC0KkliWxvCmapr5J8HHmanS4xAM9fDq+J/jxG1TDVrmlhEGS6H56i9il89VRnmKnZjRQ7yWAapq2HAZkOg+rvrqDCGDZ2tEVJbKkf7pdEDdcZOeMo6VJOW2y9LH/NmOuyl7HZbVqeGj2GA2pElb9Zv459A5jYDiW+px8PTueTrL/Oodrkg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2906002)(356005)(82310400003)(2616005)(83380400001)(54906003)(47076005)(33656002)(36860700001)(53546011)(186003)(8936002)(70586007)(70206006)(6862004)(26005)(81166007)(508600001)(4326008)(86362001)(316002)(6506007)(5660300002)(6512007)(8676002)(36756003)(336012)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:22:07.7355
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 22645990-f420-4796-c788-08d97e846101
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB7270

Hi Stefano,

> On 23 Sep 2021, at 12:06 am, Stefano Stabellini <sstabellini@kernel.org> =
wrote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3=
fc
>=20
> I'd prefer the upstream commit id if possible, not one based on the
> stable trees (the commit id doesn't resolve at all on Linus's tree).
> Aside from that, the patch is fine.
>=20

Ok. I will change the stable commit id to  upstream commit id.

Regards,
Rahul
>=20
>> Import the Linux helper of_property_read_variable_u32_array. This
>> function find and read an array of 32 bit integers from a property,
>> with bounds on the minimum and maximum array size.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2: Patch introduced in v2
>> ---
>> xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
>> xen/include/xen/device_tree.h | 26 +++++++++++++++
>> 2 files changed, 87 insertions(+)
>>=20
>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>> index 03d25a81ce..53160d61f8 100644
>> --- a/xen/common/device_tree.c
>> +++ b/xen/common/device_tree.c
>> @@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_=
node *np,
>>     return 0;
>> }
>>=20
>> +/**
>> + * dt_find_property_value_of_size
>> + *
>> + * @np:     device node from which the property value is to be read.
>> + * @propname:   name of the property to be searched.
>> + * @min:    minimum allowed length of property value
>> + * @max:    maximum allowed length of property value (0 means unlimited=
)
>> + * @len:    if !=3DNULL, actual length is written to here
>> + *
>> + * Search for a property in a device node and valid the requested size.
>> + *
>> + * Return: The property value on success, -EINVAL if the property does =
not
>> + * exist, -ENODATA if property does not have a value, and -EOVERFLOW if=
 the
>> + * property data is too small or too large.
>> + */
>> +static void *dt_find_property_value_of_size(const struct dt_device_node=
 *np,
>> +                                            const char *propname, u32 m=
in,
>> +                                            u32 max, size_t *len)
>> +{
>> +    const struct dt_property *prop =3D dt_find_property(np, propname, N=
ULL);
>> +
>> +    if ( !prop )
>> +        return ERR_PTR(-EINVAL);
>> +    if ( !prop->value )
>> +        return ERR_PTR(-ENODATA);
>> +    if ( prop->length < min )
>> +        return ERR_PTR(-EOVERFLOW);
>> +    if ( max && prop->length > max )
>> +        return ERR_PTR(-EOVERFLOW);
>> +
>> +    if ( len )
>> +        *len =3D prop->length;
>> +
>> +    return prop->value;
>> +}
>> +
>> +int dt_property_read_variable_u32_array(const struct dt_device_node *np=
,
>> +                                        const char *propname, u32 *out_=
values,
>> +                                        size_t sz_min, size_t sz_max)
>> +{
>> +    size_t sz, count;
>> +    const __be32 *val =3D dt_find_property_value_of_size(np, propname,
>> +                        (sz_min * sizeof(*out_values)),
>> +                        (sz_max * sizeof(*out_values)),
>> +                        &sz);
>> +
>> +    if ( IS_ERR(val) )
>> +        return PTR_ERR(val);
>> +
>> +    if ( !sz_max )
>> +        sz =3D sz_min;
>> +    else
>> +        sz /=3D sizeof(*out_values);
>> +
>> +    count =3D sz;
>> +    while ( count-- )
>> +        *out_values++ =3D be32_to_cpup(val++);
>> +
>> +    return sz;
>> +}
>> +
>> int dt_property_match_string(const struct dt_device_node *np,
>>                              const char *propname, const char *string)
>> {
>> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree=
.h
>> index b02696be94..1693fb8e8c 100644
>> --- a/xen/include/xen/device_tree.h
>> +++ b/xen/include/xen/device_tree.h
>> @@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_=
node *np,
>> bool_t dt_property_read_u64(const struct dt_device_node *np,
>>                             const char *name, u64 *out_value);
>>=20
>> +
>> +/**
>> + * dt_property_read_variable_u32_array - Find and read an array of 32 b=
it
>> + * integers from a property, with bounds on the minimum and maximum arr=
ay size.
>> + *
>> + * @np:     device node from which the property value is to be read.
>> + * @propname:   name of the property to be searched.
>> + * @out_values: pointer to return found values.
>> + * @sz_min: minimum number of array elements to read
>> + * @sz_max: maximum number of array elements to read, if zero there is =
no
>> + *      upper limit on the number of elements in the dts entry but only
>> + *      sz_min will be read.
>> + *
>> + * Search for a property in a device node and read 32-bit value(s) from
>> + * it.
>> + *
>> + * Return: The number of elements read on success, -EINVAL if the prope=
rty
>> + * does not exist, -ENODATA if property does not have a value, and -EOV=
ERFLOW
>> + * if the property data is smaller than sz_min or longer than sz_max.
>> + *
>> + * The out_values is modified only if a valid u32 value can be decoded.
>> + */
>> +int dt_property_read_variable_u32_array(const struct dt_device_node *np=
,
>> +                                        const char *propname, u32 *out_=
values,
>> +                                        size_t sz_min, size_t sz_max);
>> +
>> /**
>>  * dt_property_read_bool - Check if a property exists
>>  * @np: node to get the value
>> --=20
>> 2.17.1
>>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:26:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:26:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193773.345180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMsG-00053L-WB; Thu, 23 Sep 2021 11:26:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193773.345180; Thu, 23 Sep 2021 11:26:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMsG-00053E-Sw; Thu, 23 Sep 2021 11:26:44 +0000
Received: by outflank-mailman (input) for mailman id 193773;
 Thu, 23 Sep 2021 11:26:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTMsF-000538-Qr
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:26:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMsF-0006TB-M8; Thu, 23 Sep 2021 11:26:43 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTMsF-0005IO-CG; Thu, 23 Sep 2021 11:26:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=qsT6T22e61w6/AfoiiKbCCIYFz/PbjvAVA6oSo1g8CQ=; b=vPuhyePzRQi+7lhrZNrHbINgo5
	7IegUyeYzQHA+RiRYjc4nFS4uSypdmQX6l+EZSfMSz7aeKCsO/SQekhUjQ25cnDuBenzChE5vSmr2
	EOm6eqJuVhWjtfHNRBhBBh4J1RWvxqeq/RplY+RoZqdxDNklV8uVIW+Uece6ny6YNKbE=;
Subject: Re: [PATCH 10/11] xen/arm: device assignment on 1:1 direct-map domain
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-11-penny.zheng@arm.com>
From: Julien Grall <julien@xen.org>
Message-ID: <c6730ec5-53c7-9bf2-4620-eefbc7c7c4a3@xen.org>
Date: Thu, 23 Sep 2021 16:26:40 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210923031115.1429719-11-penny.zheng@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit

Hi,

On 23/09/2021 08:11, Penny Zheng wrote:
> User could do device passthrough, with "xen,force-assign-without-iommu" in
> the device tree snippet, on trusted guest through 1:1 direct-map,
> if IOMMU absent or disabled on hardware.

At the moment, it would be possible to passthrough a non-DMA capable 
device with direct-mapping. After this patch, this is going to be forbidden.

> 
> In order to achieve that, this patch adds 1:1 direct-map check and disables
> iommu-related action.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/arch/arm/domain_build.c | 12 ++++++++----
>   1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index c92e510ae7..9a9d2522b7 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2070,14 +2070,18 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo,
>       if ( res < 0 )
>           return res;
>   
> +    /*
> +     * If xen_force, we allow assignment of devices without IOMMU protection.
> +     * And if IOMMU is disabled or absent, 1:1 direct-map is necessary > +     */
> +    if ( xen_force && is_domain_direct_mapped(kinfo->d) &&
> +         !dt_device_is_protected(node) )

dt_device_is_protected() will be always false unless the device is 
protected behing an SMMU using the legacy binding. So I don't think this 
is correct to move this check ahead. In fact..

> +        return 0;
> +
>       res = iommu_add_dt_device(node);

... the call should already be a NOP when the IOMMU is disabled or the 
device is not behind an IOMMU. So can you explain what you are trying to 
prevent here?

>       if ( res < 0 )
>           return res;
>   
> -    /* If xen_force, we allow assignment of devices without IOMMU protection. */
> -    if ( xen_force && !dt_device_is_protected(node) )
> -        return 0;
> -
>       return iommu_assign_dt_device(kinfo->d, node);
>   }
>   
> 

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:27:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:27:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193778.345190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMt8-0005f7-8o; Thu, 23 Sep 2021 11:27:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193778.345190; Thu, 23 Sep 2021 11:27:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMt8-0005f0-5v; Thu, 23 Sep 2021 11:27:38 +0000
Received: by outflank-mailman (input) for mailman id 193778;
 Thu, 23 Sep 2021 11:27:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTMt6-0005eo-W3
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:27:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b49c7b7-5fb2-4eba-bbae-fad3dcc29654;
 Thu, 23 Sep 2021 11:27:36 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2057.outbound.protection.outlook.com [104.47.9.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-4jEBeSl3Ot2NMScZYB6Qmg-1; Thu, 23 Sep 2021 13:27:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:27:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 11:27:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0045.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:53::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 11:27:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b49c7b7-5fb2-4eba-bbae-fad3dcc29654
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632396455;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hLv34NzNfECH3TDbVe8AtqUakpYCaTSoad0Jnx87XMg=;
	b=bFZBT4Thg8OKDm7IGMgG8rOKe1sp9uzKOo8Q/P0IdTuqu7utXGZQTvUTwgsLNjZ9BS/cmN
	iiajJszbtLIj85dfdretJE6RLrFybRdWs4cHwHSdDIpzxOkaIRyE8y3wlfj17SldiLtBfP
	ju2Qbu0AOA0JrLmmYvG/AeCQgVCnz5I=
X-MC-Unique: 4jEBeSl3Ot2NMScZYB6Qmg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YFy9sZcAmNbRUznYTTWXyi8z9eaw6BO2giM0KY5J/ROsF9CB0a27Vw02m8N1esVgMaUOXN2WkjSNb6ORfm9yOOz2QdVLQWSRkvSZ7n95exKorU7xRI1r1h42IiqA5EIjIqx6pHIN99ONxqf14yeJx3waPzxdy8+F5aVKRrXAEh+6VQns6D2uaJFIcmEAvCjJEdXUHeRm5IBxIClaaMiNUX7ALBjBOCulrzZ9CnABgioUKqIghjjw1sU2u7VAWR1H3alHY8MerwbFNuFj6FqPRE3KmFIIrgy2wa+z1uXhGRdz1MsHtTH/+f9CPokGp8WD8etGhijZ1evhJtBzvCuo2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vykuahmkcVEa45h5O0kikFNVm23Ro8CWi14oldtTsS8=;
 b=cezzSvsFj7POJ8C1jmmOZQ0TgBXCFhgsM6AuxkaynfP5D6Iv+rE0+gMl6AWS9+prk3QdCEV66HERO5UF5tdmM2mGGIWzemPHyLgwzXD4M5ZiLR3OocRln/ERXb/CU1tcXfQmnz1gsQh95WdG0GFJIcoqHBv/5J3WsvVdNebPAtGE5AVgvBvP2gDxVDGuT2WgR6Pjz7To3KGy3Rrsj1u9pmhEfS0kiOqQ1KawOpeZ7FUKP05P6ldilkYKxuZICwxEZQeBbrq9qPb/e0g5YgwfwsNKl71lJALyGmv82w5tqVMjBxNP22mFaq3XAYjMSZyjsovYamb41MJQn+biObpIkg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 8/9] x86/HVM: skip offline vCPU-s when dumping
 VMCBs/VMCSes
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Kevin Tian
 <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <5cc8f7e0-399e-f6a5-bc84-823a6160a3f1@suse.com>
 <YUw5babRHdk6Jh8E@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b1c14b85-e65a-d563-bf51-0cb4483fde0b@suse.com>
Date: Thu, 23 Sep 2021 13:27:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUw5babRHdk6Jh8E@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P189CA0045.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:53::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a87c4f8d-212a-4cf8-f883-08d97e852230
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686330F9342F88998969E335B3A39@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dWC+aMNsGxZqdWPN8IfToqBtvR+uwaPmw1kYu/ncal8nHuKJ5yaQnXyPU207tkXTaWBGeL3GJIuMNxahwR+rO+976tiS6rA0r0sNJgjktT+mMEcpAPzO0xwtg1P2pTiUZq59xw2xHX5GJuOKbKmcVNj6OM+eW1xqOXBlROtALHOkWKA6Jfbp0vzuwEZBJWlPo3CKy2TgNgvmMj/wai37x81Q7mwSAlNOLrQk+qCHufLjxyHvcTGHWv0lFm5r1asLZa1lGaRAJtD+nGJvaCcHNBjN/8az+rgt/4zflH/7/zJpi8Iu2W9bGEWPdz2kAoQj/GedHYjhOXlr6fdBInXsinoSoIrly63HS6cHQ8Z2bGlJb7VU2aX40XPa4VZ7WcuRjgccuYOCKtLwR0VFnC9SBMnd+V6UuxVbvVMzpWg7jReNgxCB1wd1lsvimsRWYu/G3RTTloepU9bAsGU6icxR6oZWQAoRywOlCfMnozgAhSAMC1bPtcXUDak2rsMelxrydqbXUlaUquWCogzWdKJyAM+YoqRcKtljhbvBjkSbXsNcs6TR2OLOraQeZVsfNiHFt++5i5mOAtZtoedI9a7u+r5ZGxX0eQf5JD5rgUarLDqrlcA/xAMOa8d2Y2RF3+L/c/sZlxPi3+daEyo15FQVNxjUYnftFQF4UbmTlYRrWLhoLAKoZnySiQPvU7FeiBUKqPWOMsZO78GeqJptkZFkBgFKybNQ8H8nt41Lr5LX3HE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(53546011)(54906003)(31686004)(6486002)(36756003)(4744005)(66946007)(66556008)(26005)(66476007)(2616005)(6916009)(16576012)(956004)(8676002)(86362001)(508600001)(316002)(5660300002)(4326008)(2906002)(8936002)(186003)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?cT7cHv6MWIIrhTUMAo3/Sa0bta/ECUDr6XZImG7YPKNHJwjzusni5WRELQs4?=
 =?us-ascii?Q?zehmPmBLpDpJCb3eO01t3BPC262Bay+wlXvzNXzyc1ZVzZCGw9oWY1P9IwCZ?=
 =?us-ascii?Q?iDXnPQDIuqtnSvPdsVrCHmOafefg//kLmlIrHpiCcRZciAsA2KIOLN8BdRFk?=
 =?us-ascii?Q?ryFWDfFwxaAH6dVXfdtjne8bkC3rfpu9cWwwytXBSEm75QYvKRAI/OQSg7Nn?=
 =?us-ascii?Q?nW9m0UdxTE2hpKbuckX6Pp6jo9NKa1NnJKGc0c93zjCtRknPDvHAGHz8Nv+y?=
 =?us-ascii?Q?EDbQP1gH5Iq+QX2B+vP/dvvwD1VpRawpFWHIKodA0Pqag3JGdLozjO8wNM8x?=
 =?us-ascii?Q?5kJLkpFiHNyGRe1Rz9ZxSkj3X4ShIpu6pp6GHhGueN31c08dj+h8rU1+9Zcq?=
 =?us-ascii?Q?PP/ynsSPyFWNw/6WmNn9mHbnxGj4pscn6zmK0mSkEo6YKuuhunQbDMcZ7aaZ?=
 =?us-ascii?Q?y2XlWGLWSkbC53V1Zz97pgSXi26CKazqUjK8xwfH458LzBr1IWLNqp6ZTs90?=
 =?us-ascii?Q?53DKWZQUSGfv6+gFzb6PRIAJQfCO+Okt2VZqarK+fxZFLvHSLJScTf6Ha/rU?=
 =?us-ascii?Q?j91seKI23akTPK1H4tjf7dhMlqRY4SWmgdt2Ih+M4C2WaTCxMiav/S+dQf7y?=
 =?us-ascii?Q?InkigFRs14zY5wpUUQZFjxVFx8svfYB3YXAdMXkX1vj00sAHZPm/nMDvDmjn?=
 =?us-ascii?Q?1jKYGDIvPGeRfeXI85HXqUhttx7I/e7KD/87pSriKOSWPGjfWgQC8Wy3K+Lx?=
 =?us-ascii?Q?G+yzFYa1B1OaL7HeWyXEYAgEIQtbAzjcf2Zp3Xn86ARLSUEKcB3oCDEoFxyn?=
 =?us-ascii?Q?0l78mqabaVnkonLxwcUFmreQnbGSJgkZnHiLUNtinJ+LS3iVl+siMjHi7M3q?=
 =?us-ascii?Q?9Ywoqczvx2f9M9Bchhyq6Kl/Fy0sZHgQzgRS7lJgg8OKMqlYadrw8JRZjBuR?=
 =?us-ascii?Q?Jcrw2mamAAcGEWpjZREtEeA3o07S+qA93L4iUJculTUkVHooqEN74V50uVaS?=
 =?us-ascii?Q?LTTtnlV1ONVI/gCeR1siDCGsYmWszNrznRJSenoImWgNxY8C/7vH1+tE5nKd?=
 =?us-ascii?Q?Z+NKxta6yTc50EVzZR4GCTJstem1QCNm1PwanC06IpXsvBViQrFMHVwOP/8J?=
 =?us-ascii?Q?GZfCPDg4LXdrS+63tf2iDITmsnNfLLLttvJlQvJMduPjM97FS2aqYB4J+OF0?=
 =?us-ascii?Q?Rnhahq0xeFT+avsFNG624vhNMd0VwXCoMMtFN+N86NCCpRfTbH/ueHFTuiXC?=
 =?us-ascii?Q?Ce5daLPKb7vWbbxSpd/RDGuVy+guRqtEH74PK6p50lmcmIopdd6QfaqbFkfB?=
 =?us-ascii?Q?GAmGHu64JoB0Jv52hXQNevhA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a87c4f8d-212a-4cf8-f883-08d97e852230
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:27:31.9740
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qTVDXPmWZJgmFAnhwQJgChin/QolX3NlKTOJVGrmV2PJmfh0gJAcraUjCtHgP7BgVxHPBGJCHYOM2CbNXko0rg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 23.09.2021 10:23, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:20:32AM +0200, Jan Beulich wrote:
>> There's not really any register state associated with offline vCPU-s, so
>> avoid spamming the log with largely useless information while still
>> leaving an indication of the fact.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> Reviewed-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Thanks.

> Is the state cleared when the vCPU is brought down? Or else it might
> be interesting to print such state for debug purposes.

Hmm, if that's considered to be potentially useful, then we'd need a
key different from VPF_down. HLT would leave state in place, and the
INIT/SIPI sequence would clobber prior state also only on the 2nd
step. v->is_initialised may be an option, but gets cleared by INIT,
not SIPI. Any other ideas?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:33:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:33:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193784.345201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMyJ-0007He-U3; Thu, 23 Sep 2021 11:32:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193784.345201; Thu, 23 Sep 2021 11:32:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTMyJ-0007HX-R7; Thu, 23 Sep 2021 11:32:59 +0000
Received: by outflank-mailman (input) for mailman id 193784;
 Thu, 23 Sep 2021 11:32:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTMyI-0007HR-EW
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:32:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff77b1a2-1c61-11ec-ba2c-12813bfff9fa;
 Thu, 23 Sep 2021 11:32:57 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2056.outbound.protection.outlook.com [104.47.2.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-OcbfMxcDOCu4iU__uNF44w-1;
 Thu, 23 Sep 2021 13:32:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 11:32:54 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 11:32:54 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0003.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 11:32:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff77b1a2-1c61-11ec-ba2c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632396776;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AytB519fHA2wBvZ3fOtCAP08HTYaYRC2kLRN7MjH3+A=;
	b=DaxWt3pWx6WUCldFiUqgH8GF0q/Wmfg4wyxXGlffIMiLypHno52SCCeLRQvugcUmLvTs6q
	Yl2JO/vlehRzXyBes47z/ZZ93dMkBjMXZCbGcDmRXuxBbsc4lGHQ7zvm/1a4BvTAJugEPl
	er4AZDJxTCZ4r9KGF+AQStNiuinnvv8=
X-MC-Unique: OcbfMxcDOCu4iU__uNF44w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cpLVaqCG4dreKkV7Mz/QXY/QQg7snz60Vnf1f8C918LxYzEKaSMRa2aftHMt2k4D/ImwYOWSqZfdyu8DFhc+9kDejy4nUgI3GRaCCZbeYi1qWdFyi4vl7gWfQsgh/YNIAOA/ZUynMBlL0UfYQ5eoD5dIvxzBFyuq16JZuq8A3JqlRLeHEjlBJIQ3WAZGE5gZRqPSJ6/QsfupjOgoG0es4ZP8HEppMwyWH01WIjVk2lFoRULpSwTUEeSFW6d7vuIWa18G2fL6mOe1s4tBokb5Qko5ARXXOBKBFX4IpPR9MhX/DRupiZXVjLFuYUfKKfEJRRmE4TbyQzWvlpTQf8B28w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NZMJ/ub7TgqiJT95OkT/1W1Q9Vvk3YKKElOuUf526wc=;
 b=CJUxWndU/+VSpELHWC2S93wdr/vk4V207KRLOLPLAXhY10BfKEwU/HEwROIaB/h9NXTDoVmFkJZ9b8CrbyhLoSChAg4bU6YBAKq8PPZvnneLUfzy/LERAVWtXz9OhIMJJ6ag1CtyReQ2OlqRx9a5tgVOEskQkHw5cLzdM0HMVj4uzFdcAyAxCzAwW5Kmqlx6JUes+LbazeUU3bA7R8DwEVeUayIokw7yb7eNtxk+ke9glyz4xPHFTaUrX5+50HekGiNS4jdTISngWnsrxiPXIQo0ba4ULd55xXQHGEZ/TqWZLxI70ym/VmnrqM7Cc3Itgj0IgFkyYJAN0547rinuzA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
Date: Thu, 23 Sep 2021 13:32:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P191CA0003.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c222b15f-0a0e-41f9-7cff-08d97e85e229
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB73601CCCA2A6508DF3AE7362B3A39@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gyhnHZFeBBwVSbJa6h0ZRy9xz1ie1C9Xwqk2F5j4OqYIPOZC+ddxnLQmQrb9/K3R9Okqlsjs2ncD0Mo0+RHNiklCO6J95YRoBzM+aPawd2dwST8+/BuY2xCiUpZiF2ZnhVSBELXO94CfRFQxutJCju+VA8F7TUpQgPkpat+jk4D85jLB13Z+pkcLZ2LVU+Y7yql0MHpx4yWRiq504KY3FvPUS/YSA4JwlJvX4S0AVkZ0ahQQ5FrSA8pZ5CYqFznuD+mdVWeG+GO8W1h7RYNWW/YWs2K3UD0b2FfJeFGgSatu4SzKGBo8pjhaDVQqsInP/w3sVWbQg1OJNWZVOW/aOa5RblbBuJlFaA84+CE6zpuux3zjKVvtV9h9oXIx+GzOj8mH2sgdJG/itQziKakiYxZ1E5soeI9IuoshgTGVSn74GN+qTAK9UOciVn3xb6CitVPD92oUINfTQ/6UQfpf7DtO5NyuIsa98sNpnyBKuEzf8+9csMy6XuTWWY7I1RmLExAHoyQFfnbDGoF5gh8Gl5WyQNDJOVjp/KlMuXlaPhoyh7RSlqIZ+VQcuv/RsU7605fHAwYTSX4h20NbeTjrrpvDbJ/t78vloT4kFjywElSUzoibUcmdzjDAKy9T8gvTtcHRP8lC+snCGfOh51yxqnmSXtg9BXa5uBBSwP47FElbTQstjDJdP8e7lurFRVxPDwv07EDXlYHn0rmh54qxWhXViIt1rI6xPLKGhvG9qtg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(2616005)(31696002)(53546011)(38100700002)(8936002)(6486002)(8676002)(83380400001)(5660300002)(31686004)(956004)(86362001)(2906002)(6916009)(316002)(36756003)(16576012)(54906003)(66556008)(66946007)(186003)(26005)(4326008)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?NkgluNKPTIMV5ZRUf2sHMyr/Ct+q/Ag3ah4k8Rcwk5P4uPBFdDms3ScZIF7L?=
 =?us-ascii?Q?PLcwX07HZViSOUTjQhlXsu6Zjw8WuBf9RSc4smnhJkWo7jV8BUoyurLCo61p?=
 =?us-ascii?Q?HQY5C8jHUYfLlZpWEbf16qeCi4Ead0TX+5X2DI3v1xPZD6sXmy4pAUY3qS2u?=
 =?us-ascii?Q?MR88mJV92Pz89xt069c7SmPhKtqtDINajI57eRAc/RdufcaVUoxcksvh7zqY?=
 =?us-ascii?Q?fytKandzRwAyzDD21iz2oOS292PhbTRPkwF6w4BEdRgZFzsqGd8GW8HXBy7h?=
 =?us-ascii?Q?Qv2nq6HQkd0vaRj3uRZh8GsYiVCFLUK/CvuROPumYAHofHad+FTc8Bf1j9DA?=
 =?us-ascii?Q?Cq4SRsNpDXdAyl51r4xoGLvL9EBHZSKtsuKxQTSbGFJ5yvWVBBe22rkirtx+?=
 =?us-ascii?Q?uKw31jFt1zDnWVnuJV3IygjJE9VBMRFA8HQvcYMxEK5eU3RghrGbmqwz+EX4?=
 =?us-ascii?Q?vTohfzPEcm2A/UXK7TqXOXI7ynF8db6AhBpFP+yi5Z8QTL4tCguJHRhLk2Cb?=
 =?us-ascii?Q?6uZBWwTxs+Y2qIMIbpKk3Lm9Wsza/FWxsCz32n64nIpeKhKpdVim1uvg42Ru?=
 =?us-ascii?Q?hPjMaNyPqMfVWtmP8+7H4zwpgxJIgkudffVn9307XUzIIij47D5mYrO2uZgn?=
 =?us-ascii?Q?gYin55/R2jmewXXFQ2G4jlrdSDJVCAk9VkMGgoeF2LU9AU7fUpAXE/sf6lQz?=
 =?us-ascii?Q?YOIjpkh89lR9eJ4UABaiipKf4LqCAs8TqJAhu/wogrwa1CopTUuEddobiHTt?=
 =?us-ascii?Q?lDqYFrO9LDvan523qBneWbaAI6zMLY4+PA1pL8dhF/U7UoCW19JbOv0mTTlD?=
 =?us-ascii?Q?3FEKvp6mRNZc10rFaA3/lUfvAfxu0tkv4+21vSM/mvoUTf5HX4ekZ679s3Tm?=
 =?us-ascii?Q?T8V7cONvlIq/B2VMirNxkHVe//AVJ2apE3xpiJ4NGOeEVo7jPF5depKmIpvK?=
 =?us-ascii?Q?isjSxBZ735obqu6zcEW4ds6Ft/+ueiD45ptwk/0lV5tZ6H0v5ZrdJ0IewF6V?=
 =?us-ascii?Q?IzQqEv6VA7/qA3EOL768vfa5QJP/DFYku7l/gXtEIZAOSjz850FWURe3ty/j?=
 =?us-ascii?Q?RzxsJAkznsiSulrbb8hKo9wZQwcn2/5LhlIggJ7eQwl6VfwTGKTcGBRCX6Sf?=
 =?us-ascii?Q?gC7YxaVeJj1K/R2w8jKLrKwD9dolYsIuVDa0IK8boir4p7pu/4MJh+OywErK?=
 =?us-ascii?Q?ikcQetDCZmiiS9bXScyON76Kj9lzN8FDbllTYXJah/4hLIBSZiu5B+MFdkf/?=
 =?us-ascii?Q?SZlWnCjXSaoBxv4oLtTZ/PU2DaxeGYUoP7Oyx1J6bfbanB6qFjLnGL8uqojp?=
 =?us-ascii?Q?6NwsYwt3jGd1XAjjzmN6KMZ8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c222b15f-0a0e-41f9-7cff-08d97e85e229
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:32:54.0495
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xIYuACyj8zAzzcfIh6xdE07selxxOJMiSN5F6AhGTpFPnAxXBmG70nFawgio9ndjzoWjAyYJcUIEY9RBqiL3bQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 23.09.2021 13:10, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
>> To become independent of the sequence of mapping operations, permit
>> "access" to accumulate for Dom0, noting that there's not going to be an
>> introspection agent for it which this might interfere with. While e.g.
>> ideally only ROM regions would get mapped with X set, getting there is
>> quite a bit of work. Plus the use of p2m_access_* here is abusive in the
>> first place.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> v3: Move last in series, for being controversial.
>> v2: Split off from original patch. Accumulate all of R, W, and X.
>>
>> --- a/xen/arch/x86/mm/p2m.c
>> +++ b/xen/arch/x86/mm/p2m.c
>> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
>>              return -EPERM;
>>          }
>> =20
>> +        /*
>> +         * Gross bodge, to go away again rather sooner than later:
>> +         *
>> +         * For MMIO allow access permissions to accumulate, but only fo=
r Dom0.
>> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() diff=
er in
>> +         * the way they specify "access", this will allow the ultimate =
result
>> +         * to be independent of the sequence of operations.
>=20
> Wouldn't it be better to 'fix' those operations so that they can work
> together?

Yes, but then we should do this properly by removing all abuse of
p2m_access_t.

> It's my understanding that set_identity_p2m_entry is the one that has
> strong requirements regarding the access permissions, as on AMD ACPI
> tables can specify how should regions be mapped.
>=20
> A possible solution might be to make set_mmio_p2m_entry more tolerant
> to how present mappings are handled. For once that function doesn't
> let callers specify access permissions, so I would consider that if a
> mapping is present on the gfn and it already points to the requested
> mfn no error should be returned to the caller. At the end the 'default
> access' for that gfn -> mfn relation is the one established by
> set_identity_p2m_entry and shouldn't be subject to the p2m default
> access.

I think further reducing access is in theory supposed to be possible.
For Dom0 all of this (including the potential of default access not
being RWX) a questionable thing though, as pointed out in earlier
discussions. After all there's no introspection (or alike) agent
supposed to be controlling Dom0.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:53:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:53:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193790.345213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNHg-0001f2-Hd; Thu, 23 Sep 2021 11:53:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193790.345213; Thu, 23 Sep 2021 11:53:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNHg-0001ev-EY; Thu, 23 Sep 2021 11:53:00 +0000
Received: by outflank-mailman (input) for mailman id 193790;
 Thu, 23 Sep 2021 11:53:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTNHf-0001ep-U1
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:53:00 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.68]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cb149d78-1c64-11ec-ba2d-12813bfff9fa;
 Thu, 23 Sep 2021 11:52:57 +0000 (UTC)
Received: from AM7PR03CA0018.eurprd03.prod.outlook.com (2603:10a6:20b:130::28)
 by AM0PR08MB3796.eurprd08.prod.outlook.com (2603:10a6:208:100::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 11:52:55 +0000
Received: from AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:130:cafe::2c) by AM7PR03CA0018.outlook.office365.com
 (2603:10a6:20b:130::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 11:52:55 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT038.mail.protection.outlook.com (10.152.17.118) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 11:52:54 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Thu, 23 Sep 2021 11:52:53 +0000
Received: from b09edc9b00eb.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 265B9692-005D-48EE-B4E5-E5CC12B9C452.1; 
 Thu, 23 Sep 2021 11:52:41 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b09edc9b00eb.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 11:52:41 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6181.eurprd08.prod.outlook.com (2603:10a6:20b:29a::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 11:52:40 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 11:52:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb149d78-1c64-11ec-ba2d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XrA1IBMBSCWmY0iQLpEx6FUR9pt03y1bOEHFTxWBqag=;
 b=hU8LjnCHevcWDYTRTDkWyBqd7m18DZW/dznWXNoqRBELAV38AxXsHNK+7EamYVvkB0k9py0LDvvgBXDKjCT0Wwcc6g1L5KYwKHNBGspJDr1JYu+Frunk+N3Ymg3ZWq6yakAu49g2DAu4PEdn8dKbTfVzqdZkbhNQ5pBosQqgzOQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 65e5f4c983969980
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZELO0Fsh22IzcAWmmIFrMhJ1x8ZxEtk5H7aqubMWalBn0qUM9KDFECcLd2QW4D9xT1QyaBYlrKXyC8noENBBmBrFI1iAn511vVUhJXaGYB3vzx3RkVAST+CgqQ73Qxd/RUhyeZ/5WPlxeolNEUXT9FzGfCcNYlE18lqtUzT2fbKMIuMEx3MfozQjmXoNjD+9IdJ1U6j9d7K6nbpfB+4dRTHs8LdeEWB1qOwcTTqMXNTZwjA1FuaKwko93T2BnZiBzhZdduCKqDp5fe+NUHOuncfPkn/+NBF1o879echsqKOZ0q1Rd9hcwxaMKh/Y1GNWClXtmgD5JU8r5lTLeFr82Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XrA1IBMBSCWmY0iQLpEx6FUR9pt03y1bOEHFTxWBqag=;
 b=nJbdv3lNm4RNnxpjBfPtcrs0PiIJ4FFN5tcea/nzWrE5++Rq62kznKmTQCa2lZ17F6ZVjmVvjn45ODQIYWohpgyHAaD8gM/pOTXuHEs83HT9IhvsYPIVKjkxcuXU9ftBrEW0Xyxmku5UIRlhnhaz9zSS6D3szXvg/4byapm+rfvxe59L+g7Bpkg3+6F9Dlr5t7pPpfTpqLaOePaFi8hb77h5Up976eXz6RHuw0TwQYO+GhK2RfKxhbb82Pewy9rsbO73HnpQaJSoc3bW2DOTOa5UFDoqcMKKBW9bylRAE297g5Dbt69iUT0fgLJK/Xp5f0/yakAyiI6h6zd/kZQGow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XrA1IBMBSCWmY0iQLpEx6FUR9pt03y1bOEHFTxWBqag=;
 b=hU8LjnCHevcWDYTRTDkWyBqd7m18DZW/dznWXNoqRBELAV38AxXsHNK+7EamYVvkB0k9py0LDvvgBXDKjCT0Wwcc6g1L5KYwKHNBGspJDr1JYu+Frunk+N3Ymg3ZWq6yakAu49g2DAu4PEdn8dKbTfVzqdZkbhNQ5pBosQqgzOQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>
Subject: Re: [PATCH v2 08/17] xen/device-tree: Add dt_get_pci_domain_nr helper
Thread-Topic: [PATCH v2 08/17] xen/device-tree: Add dt_get_pci_domain_nr
 helper
Thread-Index: AQHXr6ZY75EobBA4I0aERqDKzahV06uwueAAgADJ1gA=
Date: Thu, 23 Sep 2021 11:52:39 +0000
Message-ID: <8D1C3683-3AB7-4A83-B6A9-2442102164BB@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <b649f9f978ed38f05927573381e23f9b3c6c24b8.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221649260.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221649260.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 646f6651-849a-43ed-6bb3-08d97e88adf2
x-ms-traffictypediagnostic: AS8PR08MB6181:|AM0PR08MB3796:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB379654793133762D96D80974FCA39@AM0PR08MB3796.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2276;OLM:2276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 MJIdFoNZW7Xjxu4JjbNdecMFysP0Bj1VVRUh71qr3lrirHmUo6WqYeXUW/m8+b9hT7o1MZSRtG2UriFyJIAGN/hJ95jCCP2KK97dAoQbKa3OsiEMz26OTMbsdx3vmMNcrwcIkEIaeYpi5lPs8zb4pnWfvYeE6fjszRkZXQvJs6bDaHP0KPoWzIPgI5wRLIDy52H8iipvb7MhUuOLtTAXYdQyjSbHLoVK0l/QQ5fCPGq7QDyDLyrCUrzwIcI9cgYV/sIGwS0L0nr1764ZdTKNiW6sbamyf3XX/7tco6j2GAxvqYX4wUho+xWtNt4LcwytSRusbrhpyVFq/fINpvfiPu3wO0b+nplMhKAyA6HkxkI6+3fRdkgyVlMQRAWazuNLOfTVVFLJcgOHB9qg/xbVAb41tm++eUyLNFHbWUWwvyy5mYRun7EswE9Pdxwlc8/HSsSVJt74zJW58H1EKTbhx7TbQLYrw9xMACG2tkuesj7ejGVVA+rM3sgIS3nolUDwdC0toa4Ya9JRgd5O6/Vfl8f0hJhvC+4mCvLAT4jyjnW9pWhAO0tTcJxlLffwARqIjwwCVo4bZVZyDMmpBgw1FR0PVzORQ7pcrY3QdYhY/O5YZf7+3sL41CxXfwC49JzM+S4o05THjRlvgWQRL7zTbsAaLLe8WaT/Nk+INwEP4iRtP9TDBP2b+RGfpwynepiKEA6IqAEQgW1zsEdk/A1zY7NEUfbvw0NcwYNy7GyAui7bRfI6EaGQrjjr1bLV4eF0
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(6916009)(8676002)(33656002)(316002)(5660300002)(6512007)(8936002)(86362001)(54906003)(2616005)(71200400001)(4326008)(91956017)(76116006)(122000001)(66556008)(64756008)(38100700002)(2906002)(186003)(66446008)(53546011)(66946007)(66476007)(26005)(6506007)(36756003)(6486002)(508600001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <130275A2D43EA44FAF1B6487D441B0A2@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6181
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6203eaac-dd74-4fd8-0894-08d97e88a508
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cSthg0DYVTCS38toibG77W5tJhJpd6BsDsXhCdVqvI542nvy5V1QNs6lKfEiBk3YNA1PjGMnCxxUGK15J1gGMrwnKum0ps/WzMKAG0BaMBWrrZDdaVQq93gLKwjZiXhqhui2pl+T4okGUBG8ho88NRHa5Ck6hwiIIOq7K+Xs+J4Agk2nyD/3Arv2AwjDYLWCBEiUI4H4yqxhWW+yBzMRd/nXNWETqWtME0CdhtKU3FXQ1Bw1X223YqmmxASrQWE6uTKliIP+bFLgBTEwxPLfa+XyTOWsrafn3TgLdZSmfv3fGn9VzCJQdQJteZpLV5G+ZP3/cCagK6sZbf29zh4B9dm4qVUGy7IMY1xISBzur//2loPv9e0wlwbeFJHukncY385f6zJQE32q/r2YNsu89YFVZvGKiTjl+zaKZEj0R/xF7EM3FhxAXhtDYWc/41o8eg1gr8XkXHL0REYjaVT6ZGoOQeGfYJzqQsConpvqTMy/tP96Y9SGRKSpI6a89kZuxt5UzhErO9sejMMylPZXSgzVJsxr4kMBuJmhrZ4PK2Fa8UgnRUlOrBRP0gY0q/uYukdxt07dt7Tzg03MnoG+Wro6jG/kB5Wn6BBv3vq84McNPhZwFvcFNLSgf8pL7CWxZgmt1mOy+274kTYfHz9oh4jKJ/8Z/NyhogESGTWHawO6PAqVd1AzLAXaVFlIs0LtmbeoD50o7WGtoTH4g/w9Cw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(54906003)(70586007)(70206006)(4326008)(47076005)(316002)(36860700001)(86362001)(186003)(53546011)(26005)(508600001)(33656002)(36756003)(2906002)(6506007)(6486002)(8936002)(6512007)(2616005)(8676002)(336012)(6862004)(356005)(5660300002)(81166007)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:52:54.8053
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 646f6651-849a-43ed-6bb3-08d97e88adf2
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3796

Hi Stefano,

> On 23 Sep 2021, at 12:50 am, Stefano Stabellini <sstabellini@kernel.org> =
wrote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Based on tag Linux v5.14.2 commit bbdd3de144fc142f2f4b9834c9241cc4e7f3d3=
fc
>=20
> Please use an upstream commit id
Ack.
>=20
>=20
>> Import the Linux helper of_get_pci_domain_nr. This function will try to
>> obtain the host bridge domain number by finding a property called
>> "linux,pci-domain" of the given device node.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2: Patch introduced in v2
>> ---
>> xen/common/device_tree.c      | 12 ++++++++++++
>> xen/include/xen/device_tree.h | 19 +++++++++++++++++++
>> 2 files changed, 31 insertions(+)
>>=20
>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>> index 53160d61f8..ea93da1725 100644
>> --- a/xen/common/device_tree.c
>> +++ b/xen/common/device_tree.c
>> @@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
>>     dt_alias_scan();
>> }
>>=20
>> +int dt_get_pci_domain_nr(struct dt_device_node *node)
>> +{
>> +    u32 domain;
>> +    int error;
>> +
>> +    error =3D dt_property_read_u32(node, "linux,pci-domain", &domain);
>> +    if ( !error )
>> +        return -EINVAL;
>> +
>> +    return (u16)domain;
>> +}
>> +
>> /*
>>  * Local variables:
>>  * mode: C
>> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree=
.h
>> index 497144b8a7..9069040ef7 100644
>> --- a/xen/include/xen/device_tree.h
>> +++ b/xen/include/xen/device_tree.h
>> @@ -831,6 +831,25 @@ int dt_count_phandle_with_args(const struct dt_devi=
ce_node *np,
>>                                const char *list_name,
>>                                const char *cells_name);
>>=20
>> +/**
>> + * dt_get_pci_domain_nr - Find the host bridge domain number
>> + *            of the given device node.
>> + * @node: Device tree node with the domain information.
>> + *
>> + * This function will try to obtain the host bridge domain number by fi=
nding
>> + * a property called "linux,pci-domain" of the given device node.
>> + *
>> + * Return:
>> + * * > 0    - On success, an associated domain number.
>> + * * -EINVAL    - The property "linux,pci-domain" does not exist.
>> + * * -ENODATA   - The linux,pci-domain" property does not have value.
>> + * * -EOVERFLOW - Invalid "linux,pci-domain" property value.
>=20
> This doesn't match reality for us because it can only return EINVAL

I will remove in next version.

Regards,
Rahul
>=20
>> + * Returns the associated domain number from DT in the range [0-0xffff]=
, or
>> + * a negative value if the required property is not found.
>> + */
>> +int dt_get_pci_domain_nr(struct dt_device_node *node);
>> +
>> #ifdef CONFIG_DEVICE_TREE_DEBUG
>> #define dt_dprintk(fmt, args...)  \
>>     printk(XENLOG_DEBUG fmt, ## args)
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:55:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:55:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193797.345223 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNJr-0002Lp-2f; Thu, 23 Sep 2021 11:55:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193797.345223; Thu, 23 Sep 2021 11:55:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNJq-0002Li-Vq; Thu, 23 Sep 2021 11:55:14 +0000
Received: by outflank-mailman (input) for mailman id 193797;
 Thu, 23 Sep 2021 11:55:14 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTNJp-0002LY-Ni
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:55:14 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1b70ad8e-1c65-11ec-ba2d-12813bfff9fa;
 Thu, 23 Sep 2021 11:55:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b70ad8e-1c65-11ec-ba2d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632398112;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=3L0djTs7W8tieHKvjiA5ogm9JOdtYDpi5RzCDIyBREM=;
  b=QysV6S95saHMml3t1oXg0DtMpG0IhHm0rZhNEacPPPsYqSFyW4aTmCL2
   FUe1tO9HY5PqvfCoEWl1yMML4FaoT9d1ifuOyZ+9Sr1mZat0LyU+yRgx0
   8hC3Cw/8tuqnHNtrfED+zj9YMlam1FXNgTU6urT7F5IM7PPQfbt6Ymomy
   s=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: NcP5u+Bz3F64K0KPa/n4y+XV5ihyRP/qO/AReLsDExZ/vYED7CCS79+tG9qgZbuMPdMuz5oc3d
 iYrEoqzClfW5TjburCWnxgXqN55CtCBMnhl/tvz7hgF1cIbdjVTr8Iqi6OOsqnQyYIu//6TsWt
 hjXwZCWOsJ7PU880vWy9qqFdziNwG1eV6pKdG3lG0OPZVaVmVaRxEzUTy8U0jOuEIkecPpoMYt
 PjWak4RllCFpLmtr6HeJM2x5UTWekBQarbqCWWoB4WiZsBiZDsLCaDJ528Od8AjfzoSE2f4w+H
 4Y4tcpFacbKSkXaDsUsNowy9
X-SBRS: 5.1
X-MesageID: 53448195
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ACmKoa/sAnMpE7oXZ1ceDrUDXnmTJUtcMsCJ2f8bNWPcYEJGY0x3m
 mpOXDjQb/bYa2T2ct0iYYm3/RlV68SGy9MyT1Nq/Hg8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j2dYx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhRw
 vkSj52fSDs2ZI3phOUiQyV5DR1XaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFh2ts1pkfQJ4yY
 eJBbTpxZgT7XidsPwkvJJY1s92XvDrGJmgwRFW9+vNsvjm7IBZK+KfpGMrYfJqNX8o9tlaVo
 CfK8nr0BjkeNceD0nyV/3S0nOjNkCjnHoUIG9WQ9PRnnVmSzWw7EwANWB2wpvzRt6Klc4sBc
 QpOoHNo9PVsshzwJjXgY/GmiF6OkkEmWvVRL/JgyCSRy5XE3yeHOkFRG1atd+canMMxQDUr0
 HqAkNXoGSFjvdWpdJ6NyluHhWjtYnVPcwfucQdBFFFcsoS/+enfmzqSFo4LLUKjsjHi9dgcK
 RixpS4ijv04iccR3s1XFniW3mrx/vAlouMzjzg7v15JDCslP+ZJhKTysDA3CMqsyq7DFTFtW
 1Bex6CjABgmV83lqcB0aLxl8EuVz/iEKibAplVkAoMs8T+gk1b6I9sKu2omfBw3aJdVEdMMX
 KM1kVkKjHO0FCH2BZKbnqrrU5h6pUQePY6Nug/ogipmPcEqKV7vENBGbk+MxWH9+HXAYollU
 ap3hf2EVC5AYYw+lWLeb75EjdcDm3BvrUuOFMuT50n2jtKjiIu9FO5t3K2mNbtisstpYWz9r
 r5iCid9408EDbKiMnWJq957wJJjBSFTOK0aYvd/L4arCgFnBHsgG7nWx7YgcJZihKNbiqHD+
 XTVZ6OS4AGXaaTvJVrYZ3Z9RqnoWJoj/3s3MTZ1ZQSj2mQ5YJbp56AaLsNlcb4i/e1l7Ph1U
 /haJJnQXqUREmzKq2YHcJ3wjI1+bxD31wiACDWoPWokdJl6Sg2XptK9Jlnz9DMDBzacvNclp
 +HyzRvSRJcOHlwwDMvfZP+14Um2uHwRxLB7U0fSe4EBc0Tw6ol6bSf2i6Zvcc0LLBzCwBqc1
 hqXXkhE9bWc/ddt/YCQ166eroqvH+9vJWZgHjHWveSsKC3X3mu/2oscAuyGSi/QCTHv86K4a
 OQLk/ylaK8bnExHupZXGqpwyf5s/MPmorJXw1g2HHjPaFj3WLpsLmPfgJtKv6xJgLRYpRG3S
 gSE/dwDYeeFP8bsEVgwIgs5b7vciaFIy2eKtfllcl/n4CJX/aacVRQANhaBvyVRMb9pPd532
 uwmosMXt1SyhxdC3gxqVcyIG7Bg9kA9bpg=
IronPort-HdrOrdr: A9a23:jh29qa7TpENJ1D07NwPXwVuBI+orL9Y04lQ7vn2ZFiY6TiXIra
 +TdaoguSMc6AxwZJkh8erwXpVoZUmsiKKdhrNhQYtKPTOWwldASbsC0WKM+UyEJ8STzJ846U
 4kSdkANDSSNykLsS+Z2njBLz9I+rDum8rE9ISurQYfcegpUdAa0+4QMHfrLqQcfng+OXNWLu
 v62iIRzADQB0j/I/7LSkUtbqzmnZnmhZjmaRkJC1oO7xSPtyqh7PrfHwKD1hkTfjtTyfN6mF
 K13TDR1+GGibWW2xXc32jc49B/n8bg8MJKAIiphtIOIjvhpw60bMBKWqGEvhoyvOazgWxa3+
 XkklMFBYBe+nnRdma6rV/E3BTh6i8n7zvYxVqRkRLY0IfEbQN/L/AEqZNScxPf5UZllsp7yr
 h302WQsIcSJQ/cnQzmjuK4FC1Cpw6Rmz4PgOQTh3tQXc81c7lKt7ES+0tTDdMpAD/60oY6C+
 NjZfuspMq+SWnqKkwxg1MfhOBFBh8Ib1C7qwk5y42oOgFt7TJEJxBy/r1Yop8CnKhNA6Wsqd
 60a5iBOdl1P7grhJlGdZI8qP2MeyXwqCL3QRCvyGvcZdU60lL22tTKCeYOlayXkKJh9upFpH
 2GaiIBiVIP
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53448195"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JzQKPyq3Y2MMXbsZ54ZbP9TXwmgmCRkvBmECFvNsw5RLepTzHoADFzoVgEC6/CkHW/oC0+FPj/7f6uroD1sFH2n5gLtgCG0eQ90fiXl4/GJA+jaqKyOY+5EzxRCZh6ZnDhExeE7GPA5HsCquhz26rY6t8Q1O+iMDq1Acanl5l/ulfBa1LK6hxMcIL8GlTN4sreiWBUPLHxN3i/vnpN0ifULfr8hChZkr9p3edp4CpiD+vY+6qtDTcbSazwQRI2fhsbDAKrskj1FBiBcYR4pBd4g9xkOH9YLAV8SW8dYkZibGWmUBFVC2HEerE9A5gxBF50/L/9iyHSyaNgy3J1gOzQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zl8HNQkhaNkXceTYB3Jh85Oznx7vvJQEKqJc+/lXEcQ=;
 b=ETeA78JNUvg+r2FtXn0B0OrRLxoaw7jzsuxTRYh7Ah+hoCieHJ4it2PyvdarpK2ybtsZQNrdmD1HokuBDynxQyEOhqnbjiitw9G989x4zFkjLMvzD+FCapBY16Ycg9sz0nS7u9jZ/xgfhk1soMoFGHUZH59QiVjFDnO9D7aPanwLk/M8HAcFZCYLpvZuOPpDuLgbl2EbxJggn/O7lJMJmKuiEFpOt5gh7G50m2xHoEigzsDnBryUNKQdV7WfMPZ4WZyQA/lY8fhEqCSAzp4S9QRMq3Bt1L1QYPui4vBh1tIqnm7PRda9uK/6saPZRqIaT8MHCTbXELGRPyygjfbu+w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zl8HNQkhaNkXceTYB3Jh85Oznx7vvJQEKqJc+/lXEcQ=;
 b=hnFFUg4yMOZPOQhFw/zTdG8dLX8cH3f2tpKxzuZkCRH37IT9jMM7y5DrBlQAkhgW1vGMXAcIQS7DLwxvEES+4gXsKlmcbJIr/l9A0FnQCmKkXDcHJSsT7O1oi3zRkwwwHttqWcvY9qvABF+/tm4H46s8nty/DUQ+sFuooIQJ8aU=
Date: Thu, 23 Sep 2021 13:54:59 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
Message-ID: <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
 <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
X-ClientProxiedBy: LO4P123CA0325.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::6) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6d18372e-6dff-4a3b-e0d3-08d97e88fbc5
X-MS-TrafficTypeDiagnostic: DM5PR03MB2923:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB2923A791C4D69BB1CE126E248FA39@DM5PR03MB2923.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: BxGxY0IIaSUKIuizi4HhKREXsJuD06gNjeX7412Xgjcbpb900hyqZ7NLTBqt1HM4pKyO9RD+FnifJBBZPtd9YdXrNhqG/VhejF/pX2kFPSvFcOsf1i2ShgNN0kNtBwS5qsoqfL+ORv1u97alxJZi9uL3z+EY6qD2bQGK2CfMJKqNrWJO7cZ1el9iSpue7PT0iKpXfZuoRBuIlEcbe1ewqapJBMxKjGjuprni6zEbIcUx+Y1T/BGJvla/EcRtjWL6XUa6Au7Ah/7YYwMKHR+q4dy49ufZ4sqFw2tI3KO+9AKwEQIm2/ne0GywGNQwM+eaJbrpN+ltudpFHbKRMX/Q1GXSoboQYURvis2CVsc4rOwJyRs+1//JZPf4v0PtC8XZ3Pvamt+Fe6T1nM2WwXdSNVGn/mGchctyxnV89IZ7hxgLkiZdqEpjRS5c2zQFF8xSVWjYEQF8V6fp9sID89WzILDXf7Xei95J2CgeXXRep3V2WA8pQa/IASIpjMnvTetDOKQz2kB5EdAEkg0CTjkgwh9hEV0NIL7KcMD2cjN2djz8dm7IjHPd4Z7WB6wTtXSWN+8NK6MxJZdqZ570Y7PXsGg4NSzhBomzOHyaRHjA+rs9ibiN3zhty5jucN9APdy9VhsqDov4DPk7iIIqcvnooA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(508600001)(4326008)(26005)(85182001)(186003)(107886003)(54906003)(9686003)(66946007)(6916009)(6496006)(83380400001)(86362001)(6666004)(8676002)(316002)(66476007)(38100700002)(53546011)(8936002)(6486002)(66556008)(2906002)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WTd1NzY0ajVBVThPOGRLc0t0OFBIRTIrbGJHUkxQTHBVWVFMN1BOOTVmY2ls?=
 =?utf-8?B?a3JkSW1YdEVTM25LOW54SmVXcEw3RWpZaEttSFowczVQbmllU3M0Q0JUSUVR?=
 =?utf-8?B?RnBJVk5mcG10RmMwd0NQR0I1SkdVbmkwN3FaUHBzT0c0WjVKaXduNytabWNJ?=
 =?utf-8?B?QitmOXcrdHVMMzdQdkJmODZlM2V0K25JVjEzZTVudlpRM2NHSy9WMFpJakkv?=
 =?utf-8?B?R1U4VTNqNjREaUkrakpTNis0akNLRi94RUdYNEZ4d1ZvbWhtYU1vWnNHaDFr?=
 =?utf-8?B?Y21tUEtoemJuZWRVbnF3TW1qM0FSbGhnYUlYYkY5Z05heXAzT21MVTVzU1pK?=
 =?utf-8?B?cjRrR1JjQ2RvMVdnWDAwSUNuMk1pYnVMc0VqMm5ocU16Z3p4TEUxNWFiQ21H?=
 =?utf-8?B?QVE4NGtpc2pDRmRLdEFiZlB2aUFsYi9Tdy9EU3ljNTh2R1FoTHV1bVp3bGtu?=
 =?utf-8?B?ZnQ5dlR3L01OcEtXeFVidit2Uld3c2hHdVJTMGRlMWVWUHRFZlU3amVKN0RL?=
 =?utf-8?B?RlZBRm9iUlZVT2dQb1VUdkNneVdGYkE4Yjh5ZzNYcU1LQU1jZzVRcUlOakVz?=
 =?utf-8?B?Y0JTMHY3bk0xVHNuWGErMjhtTE1RRk1zSFZLWVVZQ1V3cnBpMHgwNGhLS1dW?=
 =?utf-8?B?NDlqZ09sdHVTcGI1UTZRc2lGZW1oUzJCWE9LOExveko4bUtGZHArekdjQkk4?=
 =?utf-8?B?QWhvNzAzMDFLeTlGSG1rTDlaalpCZ2tDSmhQVGU4VVZrVnU2azY2Ry9Wbm8x?=
 =?utf-8?B?cGxEYUJlam1sT2h2WnV6SXFZY3JIaTBrWVlxWjg1b2NheWNWbzNjWkxEQ2pT?=
 =?utf-8?B?K3Avb3RVNU9Cc3N4VGJmZGhnUUE2U094ejdkWjAxRFNsRnB5TTI2eDFSM3p5?=
 =?utf-8?B?L01TQ0hTYUVmN1FFWmZFWUZ2ZVZFakZETmtwMXJmNWs5SFBFQTMvUlB3akRu?=
 =?utf-8?B?Tnk3Z1kwNDdQWm84Q0ljd3BDRVdkT2pYQ0laRng4bzJvdEpNUUVGZEJ0eUEy?=
 =?utf-8?B?M2phVDNVaVBJSENYSWdTTUdYRVZFdW1qSUlhK1lvcUtoMU5DYlRZejJ2OXBW?=
 =?utf-8?B?VmpRR2RMT2dTbW1LTnNTR042dUtKb1pFT1hSbUJ3eHNBekp3bkRzMWh3bEJw?=
 =?utf-8?B?NlJObUZ3MXJBNHhBVjdTZ3NKaHJ3VTNnbGtSckwzeWI4emZ4c1Z3bWRKajkw?=
 =?utf-8?B?N25aRW1lVldGVm95ZDhJSTlOdmpVcmljTjFvbEltOWFUYmszaEtyb2pxR0RK?=
 =?utf-8?B?K3AwQjQ2Y1V5eU83OXhyQVEvRlhRN1c4RkJhcDQxNXExd3NGTWxOdEpyVWhL?=
 =?utf-8?B?SnNiWU5oakNGdWc3STdGbHJucFI1eGFCM0ljTWE3eXQ2OXpYZk9Sa0NzTEhS?=
 =?utf-8?B?a3JKR0FVYUI2SERJbDZlWE14TFFsT0FTUHdWRUJ2UTFsMjJFZmZyd0dGN2pR?=
 =?utf-8?B?SXMxa0xORHd6a3JaWGduYnNmRVhqWGRLaDN0WTdFTnp2MVZzZE14eW9jZERZ?=
 =?utf-8?B?cVNrWmxUMzN0L01HUTNRTWhFRk05UzNxQmZIWS9ZbTRDUmxDRVRSUmhVTjRr?=
 =?utf-8?B?NnlQWVR5aTVFNW9QK0NoNkpkVVhocHQzdjFPOFFKNm9TSnFRNkw3NzZuaS9L?=
 =?utf-8?B?ZnNFMG82Y1pSaFJ1dHJhcXlGWHc3bHRUMFpmVm1GdS9OMTQvOHZMTWw5ME1w?=
 =?utf-8?B?STBMelVsWW9ka2ttUWMxYkdxZi96a0JkVW9DNFQvOFdCbzEzOG8vTHVWWi94?=
 =?utf-8?Q?CXzav/lZrBhIgZkZRzUoOBZ7CxbI9bCjy774O6N?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6d18372e-6dff-4a3b-e0d3-08d97e88fbc5
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 11:55:05.6826
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: X+Ci536Wumb3qR9oLunSqX1z65e5j7d+LGNghlKDZVbgUnnHhxJJlwczOY0y/8qTHI80tf0xtPwaeUAy+0ZpPg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2923
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 01:32:52PM +0200, Jan Beulich wrote:
> On 23.09.2021 13:10, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
> >> To become independent of the sequence of mapping operations, permit
> >> "access" to accumulate for Dom0, noting that there's not going to be an
> >> introspection agent for it which this might interfere with. While e.g.
> >> ideally only ROM regions would get mapped with X set, getting there is
> >> quite a bit of work. Plus the use of p2m_access_* here is abusive in the
> >> first place.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> v3: Move last in series, for being controversial.
> >> v2: Split off from original patch. Accumulate all of R, W, and X.
> >>
> >> --- a/xen/arch/x86/mm/p2m.c
> >> +++ b/xen/arch/x86/mm/p2m.c
> >> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
> >>              return -EPERM;
> >>          }
> >>  
> >> +        /*
> >> +         * Gross bodge, to go away again rather sooner than later:
> >> +         *
> >> +         * For MMIO allow access permissions to accumulate, but only for Dom0.
> >> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
> >> +         * the way they specify "access", this will allow the ultimate result
> >> +         * to be independent of the sequence of operations.
> > 
> > Wouldn't it be better to 'fix' those operations so that they can work
> > together?
> 
> Yes, but then we should do this properly by removing all abuse of
> p2m_access_t.

I'm not sure I follow what that abuse is.

> > It's my understanding that set_identity_p2m_entry is the one that has
> > strong requirements regarding the access permissions, as on AMD ACPI
> > tables can specify how should regions be mapped.
> > 
> > A possible solution might be to make set_mmio_p2m_entry more tolerant
> > to how present mappings are handled. For once that function doesn't
> > let callers specify access permissions, so I would consider that if a
> > mapping is present on the gfn and it already points to the requested
> > mfn no error should be returned to the caller. At the end the 'default
> > access' for that gfn -> mfn relation is the one established by
> > set_identity_p2m_entry and shouldn't be subject to the p2m default
> > access.
> 
> I think further reducing access is in theory supposed to be possible.

Couldn't an access reduction introduce issues, kind of similar to what
would happen if the regions are unmapped? (and that XSA-378 addressed)

I think whatever permissions set_identity_p2m_entry sets should be
mandatory ones, and no changes should be allowed. That would maybe
require introducing a new p2m type (p2m_mmio_mandatory) in order to
differentiate firmware mandatory MMIO mappings from the rest.

> For Dom0 all of this (including the potential of default access not
> being RWX) a questionable thing though, as pointed out in earlier
> discussions. After all there's no introspection (or alike) agent
> supposed to be controlling Dom0.

Ideally I would prefer a solution that could be applied to both dom0
and domU, if that's possible.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 11:56:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 11:56:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193802.345235 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNKn-0002vz-CO; Thu, 23 Sep 2021 11:56:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193802.345235; Thu, 23 Sep 2021 11:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNKn-0002vs-9J; Thu, 23 Sep 2021 11:56:13 +0000
Received: by outflank-mailman (input) for mailman id 193802;
 Thu, 23 Sep 2021 11:56:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZpLY=ON=anisinha.ca=ani@srs-us1.protection.inumbo.net>)
 id 1mTNKl-0002vR-9W
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 11:56:11 +0000
Received: from mail-pf1-x42c.google.com (unknown [2607:f8b0:4864:20::42c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b7f1e4b4-490d-4c5d-9f3d-274201aa7a26;
 Thu, 23 Sep 2021 11:56:06 +0000 (UTC)
Received: by mail-pf1-x42c.google.com with SMTP id 145so5473279pfz.11
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 04:56:06 -0700 (PDT)
Received: from anisinha-lenovo ([115.96.156.170])
 by smtp.googlemail.com with ESMTPSA id 201sm5097769pfw.37.2021.09.23.04.55.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 04:56:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7f1e4b4-490d-4c5d-9f3d-274201aa7a26
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=anisinha-ca.20210112.gappssmtp.com; s=20210112;
        h=from:date:to:cc:subject:in-reply-to:message-id:references
         :user-agent:mime-version;
        bh=SY869irrxzfOoJkdWCuDuzh7ZngMT+9YxkjytmTDREU=;
        b=U7PowsmPYHknkV3fCw5NxMoC2azRQc+aWAttZ+UHqBZVvwM6dUlqw02Rm2h0zbAbI0
         xbGhtLJ9geYBLFjmTS/5oO5gArei8XSmN62/NNQFQA1+cUB9oZTURXW+AEEwQjYGwoGe
         hBw5BdJd2jktGxhnL3O8AKZdJ95pCO39/6x2yaNFPkm/oAj+jO6+sfeIxbSIkB6qsEtT
         mjDd6A1iOBomObypmypRTU8IV2hWlesozGv9gtU73ixhNgIUvTqibBGj62+H1g2CH6Yv
         O0Y0fl6s49RPCuLNCFEjmFme7dqAQUWsUdhWd8oioOQzNLIehSsGSIHAl7DSq+d4KCb4
         3L8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:date:to:cc:subject:in-reply-to:message-id
         :references:user-agent:mime-version;
        bh=SY869irrxzfOoJkdWCuDuzh7ZngMT+9YxkjytmTDREU=;
        b=v3huI2vacsc2x3vrCF8GaevaceP9BJ/tPGADfmc3LSY2Im181wUbJeackuiT4PKpfC
         Q4GxJmnw3zuexCX7wdhF2CTFXwrGIEY+TwF+PyWHGGS06sIxuzhQ9gu0kAETFrJ+tyLO
         A3JSx3oJBUGLfr12NSfERrzWsbERVgHCf5lWoXTonQlRdip++2wfbrDRBas749nkN69L
         FyhXkZMoWD80hnz1+uWzlDkINO9xKBzGfd8llkCvautj/Dt+gAABDcWHCo9gCeKXkEDt
         UJqKRwBpnB8EcC7XFSC/a8GXL4Lbs+hgHci7F1VifHMmeIr2VWlQrP5k9DD+d3nzj8LP
         Pdhg==
X-Gm-Message-State: AOAM532fNmy8JMXDHbDyc9e6vMUydS5aI2ewf3CqHGl4jni2fPIpH9R3
	Lqai5yesDpV/HSxyIK3CpUysfg==
X-Google-Smtp-Source: ABdhPJxN+kA2mTsQR+l1zBPEpE/rXxHeA2wclNVyce3HUXRFs9M3E7zp1eZW9twTe7ZJhuUZvRsFxQ==
X-Received: by 2002:a62:1ec5:0:b0:446:40ec:73b2 with SMTP id e188-20020a621ec5000000b0044640ec73b2mr4086191pfe.5.1632398165735;
        Thu, 23 Sep 2021 04:56:05 -0700 (PDT)
From: Ani Sinha <ani@anisinha.ca>
X-Google-Original-From: Ani Sinha <anisinha@anisinha.ca>
Date: Thu, 23 Sep 2021 17:25:54 +0530 (IST)
X-X-Sender: anisinha@anisinha-lenovo
To: Ani Sinha <ani@anisinha.ca>
cc: Damien Hedde <damien.hedde@greensocs.com>, qemu-devel@nongnu.org, 
    Alistair Francis <Alistair.Francis@wdc.com>, 
    =?ISO-8859-15?Q?Marc-Andr=E9_Lureau?= <marcandre.lureau@redhat.com>, 
    Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, 
    Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, 
    =?ISO-8859-15?Q?Daniel_P=2E_Berrang=E9?= <berrange@redhat.com>, 
    "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, 
    Gerd Hoffmann <kraxel@redhat.com>, Eric Auger <eric.auger@redhat.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, 
    "Dr. David Alan Gilbert" <dgilbert@redhat.com>, 
    Markus Armbruster <armbru@redhat.com>, Eric Blake <eblake@redhat.com>, 
    qemu-riscv@nongnu.org, xen-devel@lists.xenproject.org, 
    mark.burton@greensocs.com, mirela.grujic@greensocs.com, edgari@xilinx.com, 
    Peter Maydell <peter.maydell@linaro.org>
Subject: Re: [RFC PATCH v2 10/16] qdev-monitor: allow adding any sysbus device
 before machine is ready
In-Reply-To: <alpine.DEB.2.22.394.2109231628280.630@anisinha-lenovo>
Message-ID: <alpine.DEB.2.22.394.2109231723060.630@anisinha-lenovo>
References: <20210922161405.140018-1-damien.hedde@greensocs.com> <20210922161405.140018-11-damien.hedde@greensocs.com> <alpine.DEB.2.22.394.2109231628280.630@anisinha-lenovo>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII



On Thu, 23 Sep 2021, Ani Sinha wrote:

>
>
> On Wed, 22 Sep 2021, Damien Hedde wrote:
>
> > Skip the sysbus device type per-machine allow-list check before the
> > MACHINE_INIT_PHASE_READY phase.
> >
> > This patch permits adding any sysbus device (it still needs to be
> > user_creatable) when using the -preconfig experimental option.
> >
> > Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> > ---
> >
> > This commit is RFC. Depending on the condition to allow a device
> > to be added, it may change.
> > ---
> >  softmmu/qdev-monitor.c | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
> > index f1c9242855..73b991adda 100644
> > --- a/softmmu/qdev-monitor.c
> > +++ b/softmmu/qdev-monitor.c
> > @@ -269,8 +269,13 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
> >          return NULL;
> >      }
> >
> > -    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
> > -        /* sysbus devices need to be allowed by the machine */
> > +    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE) &&
> > +        phase_check(MACHINE_INIT_PHASE_READY)) {
> > +        /*
> > +         * Sysbus devices need to be allowed by the machine.
> > +         * We only check that after the machine is ready in order to let
> > +         * us add any user_creatable sysbus device during machine creation.
> > +         */
> >          MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
> >          if (!machine_class_is_dynamic_sysbus_dev_allowed(mc, *driver)) {
> >              error_setg(errp, "'%s' is not an allowed pluggable sysbus device "
>
> Since now you are adding the state of the machine creation in the
> valiation condition, the failure error message becomes misleading.
> Better to do this I think :
>
> if (object class is TYPE_SYS_BUS_DEVICE)
> {
>   if (!phase_check(MACHINE_INIT_PHASE_READY))
>     {
>       // error out here saying the state of the machine creation is too
> early
>     }
>
>     // do the other check on dynamic sysbus
>
> }

The other thing to consider is whether we should put the machine phaze
check at a higher level, at qdev_device_add() perhaps. One might envison
that different device types may be allowed to be added at different
stages of machine creation. So this check needs be more generalized for
the future.




From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:01:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:01:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193809.345246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNQ0-0004i0-6Z; Thu, 23 Sep 2021 12:01:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193809.345246; Thu, 23 Sep 2021 12:01:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNQ0-0004ht-3F; Thu, 23 Sep 2021 12:01:36 +0000
Received: by outflank-mailman (input) for mailman id 193809;
 Thu, 23 Sep 2021 12:01:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CYt9=ON=linux.ibm.com=rppt@srs-us1.protection.inumbo.net>)
 id 1mTNPy-0004hn-Ig
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:01:34 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.156.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 15912445-afe3-4f07-89d8-eb5ab1843116;
 Thu, 23 Sep 2021 12:01:33 +0000 (UTC)
Received: from pps.filterd (m0098410.ppops.net [127.0.0.1])
 by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NAdUcU003364; 
 Thu, 23 Sep 2021 08:01:15 -0400
Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com
 [159.122.73.72])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3b8p4dcmbn-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 08:01:14 -0400
Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1])
 by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18NBvwov029624;
 Thu, 23 Sep 2021 12:01:11 GMT
Received: from b06cxnps4074.portsmouth.uk.ibm.com
 (d06relay11.portsmouth.uk.ibm.com [9.149.109.196])
 by ppma06fra.de.ibm.com with ESMTP id 3b7q6ps42c-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 12:01:11 +0000
Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com
 [9.149.105.58])
 by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18NC18Jk56885734
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 23 Sep 2021 12:01:08 GMT
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 810B04C08B;
 Thu, 23 Sep 2021 12:01:08 +0000 (GMT)
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 64DD54C089;
 Thu, 23 Sep 2021 12:01:06 +0000 (GMT)
Received: from linux.ibm.com (unknown [9.145.159.121])
 by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS;
 Thu, 23 Sep 2021 12:01:06 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15912445-afe3-4f07-89d8-eb5ab1843116
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc :
 subject : message-id : references : mime-version : content-type :
 content-transfer-encoding : in-reply-to; s=pp1;
 bh=pXzEf65A4Oe5+XzfRr8terVnfyIyuO6PQs0+F8aXgGc=;
 b=ZCv5VVg5Axl96NC4/j5oW2leA+TyRu7PTszC85LX7OnyuvdR/QPrxFJvHee6+ph9/1Ah
 rUd/1Dr4tZ5yXYAiKoNA4l0g4riJzCer09eV4KmKCsnovao7Ujg6/MQzQ6IVb2C5LRSL
 6WitPRreJzNPlii4WW55mXNNAxhzlImWjS7TBI5y3E9CuYqxTL+hR1eRpZFENX5WHVgo
 3bzeScRRqqA6eAdFzB/5aWdZYhNxC/HkliRolmuo1Xv4pKsUbXkIzU8GEpyYBQunYUKJ
 0M08Qqduz9x2e5o+taGQrQG0ce7A7g963Zzijz3wHvEhl/y43e2FXQI9OwpYg11CHmgU 9g== 
Date: Thu, 23 Sep 2021 15:01:04 +0300
From: Mike Rapoport <rppt@linux.ibm.com>
To: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Mike Rapoport <rppt@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        devicetree@vger.kernel.org, linux-efi@vger.kernel.org,
        kvm@vger.kernel.org, linux-s390@vger.kernel.org,
        linux-sh@vger.kernel.org, linux-um@lists.infradead.org,
        linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com,
        linux-mips@vger.kernel.org, linux-mm@kvack.org,
        iommu@lists.linux-foundation.org, linux-usb@vger.kernel.org,
        linux-alpha@vger.kernel.org, sparclinux@vger.kernel.org,
        xen-devel@lists.xenproject.org,
        Andrew Morton <akpm@linux-foundation.org>,
        linux-snps-arc@lists.infradead.org, linuxppc-dev@lists.ozlabs.org,
        linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
Message-ID: <YUxsgN/uolhn1Ok+@linux.ibm.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
 <1101e3c7-fcb7-a632-8e22-47f4a01ea02e@csgroup.eu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1101e3c7-fcb7-a632-8e22-47f4a01ea02e@csgroup.eu>
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: fe2SFaLdoxw706nrVmE8bleSLkHdGrzC
X-Proofpoint-ORIG-GUID: fe2SFaLdoxw706nrVmE8bleSLkHdGrzC
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_04,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxscore=0
 lowpriorityscore=0 phishscore=0 clxscore=1011 bulkscore=0 spamscore=0
 mlxlogscore=613 impostorscore=0 priorityscore=1501 malwarescore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230076

On Thu, Sep 23, 2021 at 11:47:48AM +0200, Christophe Leroy wrote:
> 
> 
> Le 23/09/2021  09:43, Mike Rapoport a crit:
> > From: Mike Rapoport <rppt@linux.ibm.com>
> > 
> > For ages memblock_free() interface dealt with physical addresses even
> > despite the existence of memblock_alloc_xx() functions that return a
> > virtual pointer.
> > 
> > Introduce memblock_phys_free() for freeing physical ranges and repurpose
> > memblock_free() to free virtual pointers to make the following pairing
> > abundantly clear:
> > 
> > 	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
> > 	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);
> > 
> > 	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
> > 	void memblock_free(void *ptr, size_t size);
> > 
> > Replace intermediate memblock_free_ptr() with memblock_free() and drop
> > unnecessary aliases memblock_free_early() and memblock_free_early_nid().
> > 
> > Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> > Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
> > ---
> 
> > diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
> > index 1a04e5bdf655..37826d8c4f74 100644
> > --- a/arch/s390/kernel/smp.c
> > +++ b/arch/s390/kernel/smp.c
> > @@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
> >   			/* Get the CPU registers */
> >   			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
> >   	}
> > -	memblock_free(page, PAGE_SIZE);
> > +	memblock_phys_free(page, PAGE_SIZE);
> >   	diag_amode31_ops.diag308_reset();
> >   	pcpu_set_smt(0);
> >   }
> > @@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
> >   	/* Add CPUs present at boot */
> >   	__smp_rescan_cpus(info, true);
> > -	memblock_free_early((unsigned long)info, sizeof(*info));
> > +	memblock_free(info, sizeof(*info));
> >   }
> >   /*
> 
> I'm a bit lost. IIUC memblock_free_early() and memblock_free() where
> identical.

Yes, they were, but all calls to memblock_free_early() were using
__pa(vaddr) because they had a virtual address at hand.

> In the first hunk memblock_free() gets replaced by memblock_phys_free()
> In the second hunk memblock_free_early() gets replaced by memblock_free()

In the first hunk the memory is allocated with memblock_phys_alloc() and we
have a physical range to free. In the second hunk the memory is allocated
with memblock_alloc() and we are freeing a virtual pointer.
 
> I think it would be easier to follow if you could split it in several
> patches:

It was an explicit request from Linus to make it a single commit:

  but the actual commit can and should be just a single commit that just
  fixes 'memblock_free()' to have sane interfaces.

I don't feel strongly about splitting it (except my laziness really
objects), but I don't think doing the conversion in several steps worth the
churn.

> - First patch: Create memblock_phys_free() and change all relevant
> memblock_free() to memblock_phys_free() - Or change memblock_free() to
> memblock_phys_free() and make memblock_free() an alias of it.
> - Second patch: Make memblock_free_ptr() become memblock_free() and change
> all remaining callers to the new semantics (IIUC memblock_free(__pa(ptr))
> becomes memblock_free(ptr) and make memblock_free_ptr() an alias of
> memblock_free()
> - Fourth patch: Replace and drop memblock_free_ptr()
> - Fifth patch: Drop memblock_free_early() and memblock_free_early_nid() (All
> users should have been upgraded to memblock_free_phys() in patch 1 or
> memblock_free() in patch 2)
> 
> Christophe

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193814.345257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNRy-0005Kz-Il; Thu, 23 Sep 2021 12:03:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193814.345257; Thu, 23 Sep 2021 12:03:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNRy-0005Ks-Fk; Thu, 23 Sep 2021 12:03:38 +0000
Received: by outflank-mailman (input) for mailman id 193814;
 Thu, 23 Sep 2021 12:03:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNRw-0005KP-VW
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:37 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4e75bc9d-295c-4cf0-a32b-92e4388f63af;
 Thu, 23 Sep 2021 12:03:33 +0000 (UTC)
Received: from AS8PR04CA0191.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::16)
 by DB6PR0801MB1800.eurprd08.prod.outlook.com (2603:10a6:4:38::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:03:31 +0000
Received: from AM5EUR03FT033.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2f3:cafe::90) by AS8PR04CA0191.outlook.office365.com
 (2603:10a6:20b:2f3::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:31 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT033.mail.protection.outlook.com (10.152.16.99) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:31 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 12:03:30 +0000
Received: from 7acd3800b8dd.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D03D8F67-BB24-41A0-8CA2-6E79E5223F6E.1; 
 Thu, 23 Sep 2021 12:03:18 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7acd3800b8dd.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:18 +0000
Received: from DU2PR04CA0176.eurprd04.prod.outlook.com (2603:10a6:10:2b0::31)
 by AS8PR08MB7157.eurprd08.prod.outlook.com (2603:10a6:20b:403::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:15 +0000
Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::75) by DU2PR04CA0176.outlook.office365.com
 (2603:10a6:10:2b0::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:15 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:15 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:08 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e75bc9d-295c-4cf0-a32b-92e4388f63af
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qd6qXBT8Cd52hi6+dwZJ9jVGH1R+o+wyTmNWceQv7m4=;
 b=smRq9tFRDx4zcXtuhm3+jk8TD675jdNpA7HmJcv5SkkKkdOWAAZXqzTsGoDDPegWKzce46X16sTMWII3nDwj2HrVub2lKLLItpu3NU9WbsIC5TvndgvqDQQYEZoaur4oBZk44aak66kZuQH0+m9+QSRcGFuZDV/WwC8A0dBflII=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ba3dfc19ea7f442a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kz97nwnNcS4BreFpwo2paIkCIzmc8/k3fEI3KdrNlYirCr1fARKim5su2plyfDgaVM8XmtSknumGRA/4GL7WEsPkU0D8mEJxCF0fzeD+DtWHwbqY9tR/OD6N5WUCqP3oYc/Fr10xXIlj65Tz0sKZW2+LplCtFOiuk2jem6V9yZJxcxWoOapcViElta38ByTu5ayto90sG5CtaHnywTy6LLA/1aRNGPoT4v1AMHbFMh1IKkn1b+vWYxi1GDgHe58xD3tEcrQClttso3UFESwWuul32sEjpuomluXMOpaehB5a76Gzq4+b+fkNxYAiEkAzeW8Hu4B+5qAdvfmnP/1Ppw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qd6qXBT8Cd52hi6+dwZJ9jVGH1R+o+wyTmNWceQv7m4=;
 b=NGmBZ01Y+6f4TKAEA3zZ5/XdH2DtSQ4MiT+zdJNf1dRfXiqZVs6nfdklJY39n7cSrId/AMfZHUsKS3zP868RzGznariXn8YPGUpinLA70hZ/AoXvsGS5rl6KB2U7lXOSKXQpiz8Df4Q94E6DWQOtII+jJKBWqb2BJ0lwdgIq4LphRCTk+LZefSrx19dgmAxM9nlBv9I9xtAZ/gUO8uFhrpMxrg4mVegBAldIG4/kO0lBA2YNJJbhyAKtL0rRMzBHjU/7+hJxvfMK5N5phhNuRa2lTO3TVB75AvoCJ8DdC6jfMC5O6nMdyWLl3Nx0Uko9KpWKFXk21QOAT0mQQrd/EQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qd6qXBT8Cd52hi6+dwZJ9jVGH1R+o+wyTmNWceQv7m4=;
 b=smRq9tFRDx4zcXtuhm3+jk8TD675jdNpA7HmJcv5SkkKkdOWAAZXqzTsGoDDPegWKzce46X16sTMWII3nDwj2HrVub2lKLLItpu3NU9WbsIC5TvndgvqDQQYEZoaur4oBZk44aak66kZuQH0+m9+QSRcGFuZDV/WwC8A0dBflII=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 00/37] Add device tree based NUMA support to Arm
Date: Thu, 23 Sep 2021 20:01:59 +0800
Message-ID: <20210923120236.3692135-1-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e3c6d064-bacb-4cc4-16cb-08d97e8a2963
X-MS-TrafficTypeDiagnostic: AS8PR08MB7157:|DB6PR0801MB1800:
X-Microsoft-Antispam-PRVS:
	<DB6PR0801MB1800D172EF6091FBDAFDBF239EA39@DB6PR0801MB1800.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3276;OLM:3276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Xj5lp4hs3g4hYukSFF/ti6WlY7cd5crhuaVPoIQRFR4Uw5xRmiOg23CRvZvVyihx8ac/xx8C0+ba+rFWCrPk5uu0Z975vsqBl/WETPB4mcTDmTdzj7/DIDfNjahFh8UopyQ5CLUKkqFeWNudkFYsHMWfuqYOWZ8hvEtaiixFJdmgRHnJUyHPgqWe5xPr6x/3G0rtASXPhcHkmJBhyj9B3WgqLQVWfTSoP+oIHeXh9Mldc/YuE0XR+VzIgjJkUQMwdd2n/wJU9PRu52k5vY9/rqUJnwHKacPobD635oRMRINX3kP3lEdVK5R+5AerW1g/THmkPaTLAnE8sye0OrpI+AFWrcsQa7ZqbWiwwvTq75n4z9gEJlW1rTH/RblFueA2+4vS/L5TQtvudoo6chQOr8okJPtLqm1G5PuAVcNHSdW8fFemR7DziYOY7czUtiuGVWR0bx6XUO+6pT6uNnGPdOUEYzHHR8IzkWhXkbslQJkHA+ZbRl+JWRdR+XOatB6Q9vMH4RsgiJjjpDT3b55Q10dGxRoErqALHAaWOEmg5TOzkpG0DLQsdticmRwpeH9sS8l4+QghRQ3regTfFyJBjZFoaXJ/yya/WATAGEsNtZJN0qiQyayXjADjOgqA8hyRfuXRvdBzrrXMqNzbG96u9linxZdzIxK0FAXk6Y2sCeIkfXq0B/q61l+ogDUBLrVZVcvV6b48/f5MFAaFdhlcfFlij16jHWfKI90VAvOaVJQ=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(2906002)(2616005)(36756003)(82310400003)(70206006)(316002)(426003)(4326008)(5660300002)(8936002)(1076003)(508600001)(6666004)(356005)(8676002)(70586007)(86362001)(36860700001)(7696005)(186003)(336012)(26005)(81166007)(83380400001)(44832011)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7157
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT033.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	56dae567-b150-424e-c4ee-08d97e8a1f98
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1Ay7lN5MjACwsaotyUbdmtUmfa4q7Xe3Olm/7Sh0LRPTMlnyy7+IF6zpfM2DusoGA4Bls8ZQ/cy8VZ0dXV4eeebOYpjz2TX2IMzcpYH8Ujs+GpmJf/JVehrUCSBGnQ4N19SnWG+nqnhmbds22Slfp20PJNKrQkgvjhfHsjeUmsdw2H9Q32JGb5nEfSQmbgT6SBJ1fhSBFDzwwOad+TZt2ZjnvaTmSoy7pWpzV9/mLHTsbIozpyhsKSyRFT1w6PXDY941Q1nAeHBfWfSvQy7YJgtLciCLy0fWTXez/YIUzRuqIWONOPVADHdphtvjRTHsbrV9RvECMEs2tCxqvEwxzrgc2Vu3ZLDci4c75gRZNqsmyEdyRiq20452Nch9RXm1qQa/6LLl3HYETsQN34Sd56z+2CvLmHNGU6mSuI87wi/rm+hZX01HtqgiLTJcbNbQxH+c81ow2OCASbSrHLvXRdPT6qCdB0itkZs4MCO+8elVZHqKxy8+jqyrE6ek4Hq2OFwQh/EDrSs3+LT2lH2VLIadCwNpEuja00sy0wemlmgZGZK+QXM8uqoJ8eQ3lthASC1D7euKPZH20NzQVRQL5EgPF0NuiLrZaNRiFzDGG8jxxDJaAU0bcNNvLpMYSMkuUncBp3TAWJH6JQaV+aORtpRaXBh4rToT/p6DSg2IJMEG4n9mNo5Jlc6xLSPJOKaAZBzDKoMKZksZQLgyvkSxqA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(2616005)(4326008)(426003)(47076005)(8676002)(36860700001)(81166007)(83380400001)(86362001)(82310400003)(8936002)(110136005)(186003)(36756003)(5660300002)(1076003)(70206006)(70586007)(508600001)(336012)(6666004)(7696005)(316002)(2906002)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:31.3992
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e3c6d064-bacb-4cc4-16cb-08d97e8a2963
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT033.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1800

Xen memory allocation and scheduler modules are NUMA aware.
But actually, on x86 has implemented the architecture APIs
to support NUMA. Arm was providing a set of fake architecture
APIs to make it compatible with NUMA awared memory allocation
and scheduler.

Arm system was working well as a single node NUMA system with
these fake APIs, because we didn't have multiple nodes NUMA
system on Arm. But in recent years, more and more Arm devices
support multiple nodes NUMA system.

So now we have a new problem. When Xen is running on these Arm
devices, Xen still treat them as single node SMP systems. The
NUMA affinity capability of Xen memory allocation and scheduler
becomes meaningless. Because they rely on input data that does
not reflect real NUMA layout.

Xen still think the access time for all of the memory is the
same for all CPUs. However, Xen may allocate memory to a VM
from different NUMA nodes with different access speeds. This
difference can be amplified in workloads inside VM, causing
performance instability and timeouts. 

So in this patch series, we implement a set of NUMA API to use
device tree to describe the NUMA layout. We reuse most of the
code of x86 NUMA to create and maintain the mapping between
memory and CPU, create the matrix between any two NUMA nodes.
Except ACPI and some x86 specified code, we have moved other
code to common. In next stage, when we implement ACPI based
NUMA for Arm64, we may move the ACPI NUMA code to common too,
but in current stage, we keep it as x86 only.

This patch serires has been tested and booted well on one
Arm64 NUMA machine and one HPE x86 NUMA machine.

---
@Julien, about numa=noacpi option, I haven't remove it in this
patch series. I had tried, and this option is not easy to remove.
So I'm not going to think about doing that in this series  

rfc -> v1:
 1. Re-order the whole patch series to avoid temporary code
 2. Add detection of discontinous node memory range
 3. Fix typos in commit messages and code comments.
 4. For variables that are used in common code, we don't
    covert them to external any more. Instead, we export
    helpers to provide variable access.
 5. Revert memnodemap[0] to 0 when NUMA init failed. Change
    memnodemapsize from ARRAY_SIZE(_memnodemap) to 1 to reflect
    reality.
 6. Use arch_has_default_dmazone for page_alloc.c instead of
    changing code inside Arm
 7. Keep Kconfig options alphabetically sorted.
 8. Replace #if !defined by #ifndef
 9. Use paddr_t for addresses in NUMA node structures and function
    parameters
10. Use fw_numa to replace acpi_numa for neutrality
11. Change BIOS to Firmware in print message.
12. Promote VIRTUAL_BUG_ON to ASSERT
13. Introduce CONFIG_EFI to stub API for non-EFI architecture
14. Use EFI stub API to replace arch helper for efi_enabled
15. Use NR_MEM_BANKS for Arm's NR_NODE_MEMBLKS
16. Change matrix map default value from NUMA_REMOTE_DISTANCE to 0
17. Remove check in numa_set_node.
18. Follow the x86's method of adding CPU to NUMA
19. Use fdt prefix for all device tree NUMA parser's API
20. Check un-matched bi-direction distance in matrix map
21. Remove unless fdt type check function
22. Update doc to remove numa x86 spcific
23. Introduce Arm generic NUMA Kconfig option

Wei Chen (37):
  xen/arm: Print a 64-bit number in hex from early uart
  xen: introduce a Kconfig option to configure NUMA nodes number
  xen/x86: Initialize memnodemapsize while faking NUMA node
  xen: introduce an arch helper for default dma zone status
  xen: decouple NUMA from ACPI in Kconfig
  xen/arm: use !CONFIG_NUMA to keep fake NUMA API
  xen/x86: use paddr_t for addresses in NUMA node structure
  xen/x86: add detection of discontinous node memory range
  xen/x86: introduce two helpers to access memory hotplug end
  xen/x86: use helpers to access/update mem_hotplug
  xen/x86: abstract neutral code from acpi_numa_memory_affinity_init
  xen/x86: decouple nodes_cover_memory from E820 map
  xen/x86: decouple processor_nodes_parsed from acpi numa functions
  xen/x86: use name fw_numa to replace acpi_numa
  xen/x86: rename acpi_scan_nodes to numa_scan_nodes
  xen/x86: export srat_bad to external
  xen/x86: use CONFIG_NUMA to gate numa_scan_nodes
  xen: move NUMA common code from x86 to common
  xen/x86: promote VIRTUAL_BUG_ON to ASSERT in
  xen: introduce CONFIG_EFI to stub API for non-EFI architecture
  xen/arm: Keep memory nodes in dtb for NUMA when boot from EFI
  xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
  xen/arm: implement node distance helpers for Arm
  xen/arm: implement two arch helpers to get memory map info
  xen/arm: implement bad_srat for Arm NUMA initialization
  xen/arm: build NUMA cpu_to_node map in dt_smp_init_cpus
  xen/arm: Add boot and secondary CPU to NUMA system
  xen/arm: stub memory hotplug access helpers for Arm
  xen/arm: introduce a helper to parse device tree processor node
  xen/arm: introduce a helper to parse device tree memory node
  xen/arm: introduce a helper to parse device tree NUMA distance map
  xen/arm: unified entry to parse all NUMA data from device tree
  xen/arm: keep guest still be NUMA unware
  xen/arm: enable device tree based NUMA in system init
  xen/arm: use CONFIG_NUMA to gate node_online_map in smpboot
  xen/arm: Provide Kconfig options for Arm to enable NUMA
  docs: update numa command line to support Arm

 docs/misc/xen-command-line.pandoc |   2 +-
 xen/arch/Kconfig                  |  11 +
 xen/arch/arm/Kconfig              |  12 +
 xen/arch/arm/Makefile             |   4 +-
 xen/arch/arm/arm64/head.S         |   9 +-
 xen/arch/arm/bootfdt.c            |   8 +-
 xen/arch/arm/domain_build.c       |   6 +
 xen/arch/arm/efi/efi-boot.h       |  25 --
 xen/arch/arm/numa.c               | 155 ++++++++++
 xen/arch/arm/numa_device_tree.c   | 274 ++++++++++++++++++
 xen/arch/arm/setup.c              |  12 +
 xen/arch/arm/smpboot.c            |  39 ++-
 xen/arch/x86/Kconfig              |   3 +-
 xen/arch/x86/numa.c               | 449 ++---------------------------
 xen/arch/x86/setup.c              |   2 +-
 xen/arch/x86/srat.c               | 232 ++-------------
 xen/common/Kconfig                |  14 +
 xen/common/Makefile               |   2 +
 xen/common/numa.c                 | 450 ++++++++++++++++++++++++++++++
 xen/common/numa_srat.c            | 264 ++++++++++++++++++
 xen/common/page_alloc.c           |   2 +-
 xen/drivers/acpi/Kconfig          |   3 +-
 xen/drivers/acpi/Makefile         |   2 +-
 xen/include/asm-arm/mm.h          |  10 +
 xen/include/asm-arm/numa.h        |  50 ++++
 xen/include/asm-x86/acpi.h        |   4 -
 xen/include/asm-x86/config.h      |   1 -
 xen/include/asm-x86/mm.h          |  10 +
 xen/include/asm-x86/numa.h        |  65 +----
 xen/include/asm-x86/setup.h       |   1 -
 xen/include/xen/efi.h             |  11 +
 xen/include/xen/numa.h            |  94 ++++++-
 32 files changed, 1470 insertions(+), 756 deletions(-)
 create mode 100644 xen/arch/arm/numa.c
 create mode 100644 xen/arch/arm/numa_device_tree.c
 create mode 100644 xen/common/numa.c
 create mode 100644 xen/common/numa_srat.c

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193815.345268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS1-0005bz-Qp; Thu, 23 Sep 2021 12:03:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193815.345268; Thu, 23 Sep 2021 12:03:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS1-0005bs-Nl; Thu, 23 Sep 2021 12:03:41 +0000
Received: by outflank-mailman (input) for mailman id 193815;
 Thu, 23 Sep 2021 12:03:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNS0-0005bO-O6
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:40 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.79]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4961292b-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:39 +0000 (UTC)
Received: from DU2PR04CA0163.eurprd04.prod.outlook.com (2603:10a6:10:2b0::18)
 by VI1PR08MB5440.eurprd08.prod.outlook.com (2603:10a6:803:134::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:37 +0000
Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::ff) by DU2PR04CA0163.outlook.office365.com
 (2603:10a6:10:2b0::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:37 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:37 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 12:03:37 +0000
Received: from 19d892eff777.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B935D0E8-7C46-4B1E-960D-060BEDD1953C.1; 
 Thu, 23 Sep 2021 12:03:22 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 19d892eff777.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:22 +0000
Received: from DB6P193CA0014.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::24) by
 VI1PR08MB3837.eurprd08.prod.outlook.com (2603:10a6:803:ba::15) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 12:03:20 +0000
Received: from DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:29:cafe::2a) by DB6P193CA0014.outlook.office365.com
 (2603:10a6:6:29::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:20 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT032.mail.protection.outlook.com (10.152.20.162) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:20 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:14 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4961292b-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GITypNRUWivldCkDa8kIMuzSIJwemWwJe9dN6aK1m/A=;
 b=Bqi4g2rSfJJh7O7sF7hJH30m6VkoukrC8wtlB/nc/OESL/27P7tb4yua3NrA7h0RNriOrSeQ4DrH5P209lJYeUyVVYnqtzPDxy5WQOj1NLtqe7EqWEs6f62PsesMLYSZ5gV2Pz4cR4H6do1/vYuSjFapFCyXZ1RPuYTfuPqW4u4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 45cdde9f39e78528
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ex5eH6YsdkCyllM+8FYxI4cJTlxkKWGwBI6gubTjguBZNWCQnJJ+kZGEk3Js7DyCMiTkj6KQJUXIK/x6bztVGtp2mirab+wX9TOhXKZHyC9nXB2NqN++BZt/P0LyH14enK+aHx0Rf7xHJeSzsr9AOMnfjBPTjYc3WAR9ErhXoKRZTdWzdiX0VOpfRxtgD/T2FSwlL5ODpArVxRLIorj2ZMIeOsTpW51o8wNb+5he7/fPRPZqFKqVNLu/GL/EfID2plAqwvqpETs2cVJQ0tuKOnY5h98ePGR6AeY3J1EG3g3KyV8osvPr8KwQe0Ev3e4N7Z6PizPu0TsUYIDE5W5IdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GITypNRUWivldCkDa8kIMuzSIJwemWwJe9dN6aK1m/A=;
 b=n7GWtL+gydl6NQEfDRnAeQ8IXjWJeiqf/lh4Q0Pc8kPs6l/9DmolJbFiQ60bY+pm4ehhm0Qu6GQ/PRNjquDz8fcJI5s9cBJ+lpLSwCIUytQGVT7PBc46jar6ARAl0cYbc4O8wwYLx0Ef0nSfrMB/7T0nv1BY0AnoneNrOCZiL6KnBATZEKG3hiBF0gF9nAsafzqLGoIV+YrLKq79G+f/zKBpI5HC7rrLA9Fh9fhVrSUZRYhmDFdHiDbFK8vl6r9kN3g19O71BCIyvqM9/GVaF+27urtsxAdEUkomcu1CMvii93pL5eSbmtnbTrwMFT+fHf1j5d5J+NgBVdUvH+Pw/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GITypNRUWivldCkDa8kIMuzSIJwemWwJe9dN6aK1m/A=;
 b=Bqi4g2rSfJJh7O7sF7hJH30m6VkoukrC8wtlB/nc/OESL/27P7tb4yua3NrA7h0RNriOrSeQ4DrH5P209lJYeUyVVYnqtzPDxy5WQOj1NLtqe7EqWEs6f62PsesMLYSZ5gV2Pz4cR4H6do1/vYuSjFapFCyXZ1RPuYTfuPqW4u4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 03/37] xen/x86: Initialize memnodemapsize while faking NUMA node
Date: Thu, 23 Sep 2021 20:02:02 +0800
Message-ID: <20210923120236.3692135-4-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 18a390a7-0593-4635-5883-08d97e8a2cda
X-MS-TrafficTypeDiagnostic: VI1PR08MB3837:|VI1PR08MB5440:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB544050C4CA11E24DDC4217369EA39@VI1PR08MB5440.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 /6Mt1pU3T1qMlhujo/qYgxQjM8btZjryc7/ZChhFF55NHYVYzQlrkAzY6V39iBC49GpJn/r4gJjrvGBmfsGPbGXglIchf0MRGwZzziaawlVKXCPJeVks0B+ZO+X9BaTsNuyPjJWrUBMVt5nSEplkDAMzzcsLQEt0lrP52wZQdqdBI/iEOddItU/JXE+37kHcsJ2xUiWYjU3wsGKQM3CZcHsGgLkeKWLd0wBQ08Bt3FWICeHIfq5Zcw4qNmcTryEoi7pit9QUuLIQA+9tAtVZ7HnfgHW8fFN/oLfRhRwGz7vUTHZjkb2JyG0Aj5rh3S50XhkCaR4NFxT07Ue4tE1taExq9WNBnB7EwcN/S3V7YUfUu+r/KW2JFXi3N8DjNCNFVO/ThnD33lXI+mKQJGZvI0m0diF/fliv7Xa4l+eXNHZY67WyDtpxCB2GrcgZv7EYadh6yMmTkQZpGpT1E4rVHvQAYuqyqfx5qSPZiTGFLc7H5XQAGMNKUCAMoalXjWZClxHOHZMLUs3JSx1LmAeVuaxVuSGZGP+gCoEj3vVbf+U4GesXJV/zSb0GH/S6fuPooLRyxtJgjFMpGwvXm5QCY1SoJZHdDtDEUFSezobqKvePPC9YFxdY2si3kUvQHkdIeQLRU+NVJ1nRYbfnduG4kNGcEkwnRpaPsiT3BXxZIikjFPpgrJFujGrngclRzdssZ4nx5zHIgooV2pQSHvrA25+PFHf8dGsnoPWr8SfstfA=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(4326008)(8936002)(83380400001)(2906002)(70206006)(36860700001)(356005)(110136005)(426003)(2616005)(8676002)(47076005)(81166007)(5660300002)(508600001)(70586007)(82310400003)(86362001)(36756003)(316002)(186003)(336012)(6666004)(7696005)(1076003)(26005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3837
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3b7e8088-a045-4af3-395a-08d97e8a22a2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gNdGmT+AwqJxVwjxXhTrZjD0hq/hvQ9NcEMXvUn9yJfiE1NZYYMCVstERHUNIIF2SYR+qtu6SVRgutpU46h6+jUTmDJojz9WQ1ecAO7XLFkY9qBrHMYh/OjNXJE2TBPmD5gj6SHKBDjcjVQXrA6nZy7uz7fnIvW7ZYdZPydh2lRZ/1kvo9CC8rWFhsiba9IeFrMXrwH/wZ0rlNkgy+7iYjjIE3/e7Kd0i1KSBSnzL3UWsdsuxUO1tRYvCZK90g3Bj6VaxEFc6AwLkLmIhBB6tcw2iUBgq9vOlLFrofpCxVoYM+7mwkEUEeRNtBNGwrR8Yz0e8G9mWFPwJ48HnvyK9+lcLwPlOXOGKmB40KpPEPGAY4TcISjPBui32kiI7eRpJTGfeQyaUcn+bKmUFEB1TF7uT/x2vB2gny2Th8K3n9dIkHhkq228Z8J8pwGDpSTKs1DfgKshojD0TQYnRzz+gu4Q3Ohmicd3wcBzf92ipY3CT/V/lNP23PfOgNWIKRhdUQOH9bc1lAmsSvsL6Q/SHeKd3G3sLHHgAQHos2NAIJkPpHayFPEZgVoOrM21vGjER0YQAtOtBg0vzwx+/pMRi4foFlQ5AO/mdVKjHv+U8ySVwWWCwBDxm8QCizru/o6O//lLjYtd13CrjuW6bcE+q24z2clURpc+Pus6aoVVnRXJHvAueb8ygthYi/Bi5+0cv6y1UqqFdVvjPohKsCA5Ig==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(8936002)(4326008)(508600001)(70206006)(2906002)(426003)(110136005)(36860700001)(81166007)(2616005)(8676002)(47076005)(5660300002)(70586007)(6666004)(86362001)(186003)(336012)(316002)(82310400003)(36756003)(7696005)(83380400001)(26005)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:37.2649
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 18a390a7-0593-4635-5883-08d97e8a2cda
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5440

When system turns NUMA off or system lacks of NUMA support,
Xen will fake a NUMA node to make system works as a single
node NUMA system.

In this case the memory node map doesn't need to be allocated
from boot pages, it will use the _memnodemap directly. But
memnodemapsize hasn't been set. Xen should assert in phys_to_nid.
Because x86 was using an empty macro "VIRTUAL_BUG_ON" to replace
SSERT, this bug will not be triggered on x86.

Actually, Xen will only use 1 slot of memnodemap in this case.
So we set memnodemap[0] to 0 and memnodemapsize to 1 in this
patch to fix it.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index f1066c59c7..ce79ee44ce 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -270,6 +270,10 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
     /* setup dummy node covering all memory */
     memnode_shift = BITS_PER_LONG - 1;
     memnodemap = _memnodemap;
+    /* Dummy node only uses 1 slot in reality */
+    memnodemap[0] = 0;
+    memnodemapsize = 1;
+
     nodes_clear(node_online_map);
     node_set_online(0);
     for ( i = 0; i < nr_cpu_ids; i++ )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193816.345279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS3-0005tR-Ab; Thu, 23 Sep 2021 12:03:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193816.345279; Thu, 23 Sep 2021 12:03:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS3-0005tE-50; Thu, 23 Sep 2021 12:03:43 +0000
Received: by outflank-mailman (input) for mailman id 193816;
 Thu, 23 Sep 2021 12:03:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNS1-0005KP-UE
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:41 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0a1bd4d9-cb2e-4520-945d-6a3fa9f0c920;
 Thu, 23 Sep 2021 12:03:36 +0000 (UTC)
Received: from AS8PR04CA0176.eurprd04.prod.outlook.com (2603:10a6:20b:331::31)
 by DU2PR08MB7255.eurprd08.prod.outlook.com (2603:10a6:10:2d2::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:34 +0000
Received: from VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:331:cafe::b6) by AS8PR04CA0176.outlook.office365.com
 (2603:10a6:20b:331::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:34 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT053.mail.protection.outlook.com (10.152.19.198) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:34 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 12:03:30 +0000
Received: from ae303b40b6ee.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4F78024A-3A30-4E5E-9C73-E80203E1249C.1; 
 Thu, 23 Sep 2021 12:03:19 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ae303b40b6ee.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:19 +0000
Received: from DU2PR04CA0161.eurprd04.prod.outlook.com (2603:10a6:10:2b0::16)
 by PA4PR08MB6016.eurprd08.prod.outlook.com (2603:10a6:102:e3::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:18 +0000
Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::82) by DU2PR04CA0161.outlook.office365.com
 (2603:10a6:10:2b0::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:18 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:18 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:12 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a1bd4d9-cb2e-4520-945d-6a3fa9f0c920
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eYpFnlkmKkWIfrICpF9PJssjdhb8RblMf+bC222jYXo=;
 b=b1dYqPsaLbPvqLviSQLFGXLHi9ft/D1yAetHHVrqVT2Yjcgto5Uqk/Vmxx4o3DwXJ/GrjYesrJdEjhL4DUD5rR8ewwuZjtflaz21WAtgp+a+b+xNXY1Jc0N0JMyP0T7Zl02CGy0UBCqmusSimiEQ1GYU2dby6F589PfyoP6UyaU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 4bb2a889a10d83d5
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Je0JyCYcCahaXGSrIylgBYfOIHGv7kRCbedNWsePEU3KwaiV7Q2/OmUiTiCxNLMKr2T9vsESnKIiBfca33IfoTeWytocXYHr9jXoLcHupLpRGlTFo6Gm2TQq0NLjO8CAlM/Htr5yHZzz98uTPvIlSsv4/HLQvtSSi9kXjonC3oxmO6OuBzR7Fx3HzE5jf4UTkKnhIAdUuRS8g2/gyXLi9Vrom6xJFQ+6vR5y7HPpvrhDz1UbqChA1B+C/o4UD46TavLgAEoam+U21bTOG5+6VzTq1gu9pe0pw/dinapuVyk2yVIeWhp9Ae7t2riT3kL2WB2yp2HOSbGNBwwOPvdUtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eYpFnlkmKkWIfrICpF9PJssjdhb8RblMf+bC222jYXo=;
 b=dwnJs9g00gRFCpbil5HynnImfid70D/qOqU02x/+EJ14TSidLGaBhZ8V+wpnIqA3334OHeCmOvkFCrSt8jM9RU6AFXVhJ3me6IV6NAtFHRC9t+axyqrQuet6ARLGBnm99pk1BO78CK4clnSlqzAzwJRzgOohT+XyBEfeGWD6qV1DGrHGJUTn+q0eyVeT/wzFxF35opK/mEUpyKIghXBvqtlUAy2X54NxnS0Q/MJpd6oYMESTUv+U1cwI5mpU6BLDXRjaJpfb9OgqMGH5LLLRbT7rlYWjXJqYAyYgtqi1VkB/xskkcM59Tyh9P4tc2ZOFfAravvQoDUmsrzk3x/wwRA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eYpFnlkmKkWIfrICpF9PJssjdhb8RblMf+bC222jYXo=;
 b=b1dYqPsaLbPvqLviSQLFGXLHi9ft/D1yAetHHVrqVT2Yjcgto5Uqk/Vmxx4o3DwXJ/GrjYesrJdEjhL4DUD5rR8ewwuZjtflaz21WAtgp+a+b+xNXY1Jc0N0JMyP0T7Zl02CGy0UBCqmusSimiEQ1GYU2dby6F589PfyoP6UyaU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA nodes number
Date: Thu, 23 Sep 2021 20:02:01 +0800
Message-ID: <20210923120236.3692135-3-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c18f4d07-b7fc-4990-593b-08d97e8a2afb
X-MS-TrafficTypeDiagnostic: PA4PR08MB6016:|DU2PR08MB7255:
X-Microsoft-Antispam-PRVS:
	<DU2PR08MB7255CFF9D420EF63D9F979859EA39@DU2PR08MB7255.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 F81usdegP21yaQyztrp2wr38GQL4K/hHAit67QRV2UTmp2BIq5CcxIsbTyzGOiGDKUgFVbWZcAHdUxUdyVX7YkqLhfxEy7HAAdm0QwDkK5Q72HI0S5wieAH1ug9uhSNCYw1jDzzRxg0u65y5Mm8Ay8Eo3hzqL20g1WK/EoWs8K5HyCpKjxD4t/4CAKEPCswRtp44tpZ9fjEWO0EQNDYyw0CLPtcWSfoayt60AYIpbO94whGd70QZMjoW5U1OS42rXYSdrUa/kGMryvoWWP3BrCN/PWL2uxj3x3FYwO/VJ1LuQ/uYmk3F+NhyHi2DmHnz9F+VlJApXwVy45QK6mSH7OiNIOrj6yzNhPnUSjfm3YbV91UI3+1VizyuDtVTaDs8UeYnFYqsrQsI7nTBFbsPJbNokSybPrsS8lDwqWyjD8c9r1ERPKz/bBHpf8B4Qsji99lvRlFiUxkuytgPwi18eOUKAIriW4DL/c1pRRHBUr2vWEWevAutDOXxdmDMCsD3t4aQ3zWEGfGtfk57Ac4zS1SvAfrRCZn6jbOFUQ5D9PhIEwDC3GII3ygGV+YPrNAoOILiDg1Hs1lkrSlr6iAJeC7ft1y1ye18qPh2plSR06ogSf9nd+2LICeVSiZT6TwcARbJk7GrFZlTPULkQ4SAvRKj++Gzk7JM7hecPKVRPAFuyp5ohDjjcnmBjiGbVD2EIhrrQKbCBJijWFec4naITnhzPvS0+6w0jlzUoPZbl3Y=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(1076003)(26005)(44832011)(81166007)(86362001)(70586007)(4326008)(83380400001)(47076005)(6666004)(36860700001)(508600001)(336012)(82310400003)(356005)(110136005)(426003)(8676002)(8936002)(70206006)(186003)(316002)(5660300002)(36756003)(7696005)(2906002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6016
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	68dccb8a-de2f-41b9-5cc2-08d97e8a216f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5uY23spjg57uWBOSGRStNZnzN64CNb4URXzhjV6cyD9fhUy66RTkZEnr/Vaz2HUKKHsJpOFPKIAymPmvcWjz/Io+/GKZboVaIDhUZ658S1lh0uV70sS2/LGQtGwsqfjp2dSuFKowua+iJXbFKTw/fW7MDp3GG6wMmEWgChrXNTvtwqACumDzlvcBWyBTgznSKtRdRRyN0WRR5pau0ufkjcNR0p9aZNzvD2iNKXM5nTLcgtzJIOnSd3izcKGLoSnT4FNMBNPBMjjjk9bziMeLrJS7aQ2KWpDcYKfOSVV71NHe2qyZQH0bKx3rYc89pr5MoaWU0GUgTS37x4VGzs/qwdvJphf9r45kl9CYeB1XygypYzpwVjr0h9fXWdRlZjMyL/KCULddy1DQBkXkIACOAhZq5RtOwRBTaFj9mPpmtrR3Ur6Lfa/QzbhX2TBw4vDquBLSft8/SU+7bcLOoYTOqlJ9G/e7yjJOkvv9CLQKp7BbNnsYHry9KnF7D1aUOir3HqgEUxBQIpiY9kbljzhuPKfa6ONUUaFOp/K/qPG/2NXy1+QdicqIfvT+1vHc0nA/lSH7Mfu69oXDzFu8uoPyd5PSdBvi2lZNorLt69aZVGhxK4jD6tWLecFM/4Nc6yRu7KdHh+Vz+1i5apRLDdhP0RnaZvPkchyWGznxXwtHtHD6J6L9g5hbvjxto6Pw9W9gflAg2ha4xLtGk9Mya/UxfQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(70586007)(81166007)(7696005)(47076005)(2906002)(26005)(8936002)(82310400003)(86362001)(186003)(83380400001)(36756003)(6666004)(8676002)(2616005)(426003)(508600001)(336012)(4326008)(36860700001)(316002)(70206006)(44832011)(1076003)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:34.0269
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c18f4d07-b7fc-4990-593b-08d97e8a2afb
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB7255

Current NUMA nodes number is a hardcode configuration. This
configuration is difficult for an administrator to change
unless changing the code.

So in this patch, we introduce this new Kconfig option for
administrators to change NUMA nodes number conveniently.
Also considering that not all architectures support NUMA,
this Kconfig option only can be visible on NUMA enabled
architectures. Non-NUMA supported architectures can still
use 1 as MAX_NUMNODES.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/Kconfig           | 11 +++++++++++
 xen/include/asm-x86/numa.h |  2 --
 xen/include/xen/numa.h     | 10 +++++-----
 3 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
index f16eb0df43..8a20da67ed 100644
--- a/xen/arch/Kconfig
+++ b/xen/arch/Kconfig
@@ -17,3 +17,14 @@ config NR_CPUS
 	  For CPU cores which support Simultaneous Multi-Threading or similar
 	  technologies, this the number of logical threads which Xen will
 	  support.
+
+config NR_NUMA_NODES
+	int "Maximum number of NUMA nodes supported"
+	range 1 4095
+	default "64"
+	depends on NUMA
+	help
+	  Controls the build-time size of various arrays and bitmaps
+	  associated with multiple-nodes management. It is the upper bound of
+	  the number of NUMA nodes the scheduler, memory allocation and other
+	  NUMA-aware components can handle.
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index bada2c0bb9..3cf26c2def 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -3,8 +3,6 @@
 
 #include <xen/cpumask.h>
 
-#define NODES_SHIFT 6
-
 typedef u8 nodeid_t;
 
 extern int srat_rev;
diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
index 7aef1a88dc..52950a3150 100644
--- a/xen/include/xen/numa.h
+++ b/xen/include/xen/numa.h
@@ -3,14 +3,14 @@
 
 #include <asm/numa.h>
 
-#ifndef NODES_SHIFT
-#define NODES_SHIFT     0
-#endif
-
 #define NUMA_NO_NODE     0xFF
 #define NUMA_NO_DISTANCE 0xFF
 
-#define MAX_NUMNODES    (1 << NODES_SHIFT)
+#ifdef CONFIG_NR_NUMA_NODES
+#define MAX_NUMNODES CONFIG_NR_NUMA_NODES
+#else
+#define MAX_NUMNODES    1
+#endif
 
 #define vcpu_to_node(v) (cpu_to_node((v)->processor))
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193817.345290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS6-0006E3-KI; Thu, 23 Sep 2021 12:03:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193817.345290; Thu, 23 Sep 2021 12:03:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS6-0006Dp-GZ; Thu, 23 Sep 2021 12:03:46 +0000
Received: by outflank-mailman (input) for mailman id 193817;
 Thu, 23 Sep 2021 12:03:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNS5-0005bO-Mq
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:45 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.74]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4a5d2a18-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:40 +0000 (UTC)
Received: from AS8PR04CA0133.eurprd04.prod.outlook.com (2603:10a6:20b:127::18)
 by VI1PR08MB3823.eurprd08.prod.outlook.com (2603:10a6:803:c2::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:38 +0000
Received: from AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:127:cafe::ad) by AS8PR04CA0133.outlook.office365.com
 (2603:10a6:20b:127::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:38 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT052.mail.protection.outlook.com (10.152.17.161) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:38 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 12:03:37 +0000
Received: from 44d92c6500e8.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F6E86E5D-73A2-41B7-8718-11882F6755A2.1; 
 Thu, 23 Sep 2021 12:03:17 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 44d92c6500e8.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:17 +0000
Received: from DU2PR04CA0164.eurprd04.prod.outlook.com (2603:10a6:10:2b0::19)
 by AS8PR08MB7223.eurprd08.prod.outlook.com (2603:10a6:20b:346::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 12:03:16 +0000
Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::8e) by DU2PR04CA0164.outlook.office365.com
 (2603:10a6:10:2b0::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:16 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:16 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:10 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a5d2a18-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Jvn0ISo/HgoiboBWB5FbYbJTrYAvUyDADUFepjr42Sw=;
 b=6wUCbjwZbgV4QuMhjrJ6LpDOOGTuNUY4epgyn+Wu1jH2gVK3v34enDYKe7T8JgMVSwTOzHKqTiQ6T4QxB69tuNfvjWWS34TyywuxMCYlUrQt0xYN2YCquI5n4xYhEG9SgGZBFRxpPzwUULqA14fl8JdYs8cJKzKMW/TqZsOmKMw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2406da346ef2d9e5
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gFvw+C6x1mEYUAyAcypHj6Ny9AHhhQdKsyh58VcmC9A19pZuivMGoBM2oPhTHYGTYkHAEX/JQz95AYvzaAifLyBwA0lPyWAvxKQIBEw/dKihQacyGbPWltZ7cwky2BtN0TeMBBMTA2pgGezwevhYHzGdiJbhx7a67wvKisUi8naeRiXiZMZ3sg277RNDxjuoYK6McD3SD6u9tyC4f58uBw0WXi6fmVy4/Vc3ufStuhGvUlW7RWQWy+mlldsuXdmyOQQxiyS0xXhP29jMrayAKtvZfNgKoSDXt52t74TMki0s9t/QFukFpdqnUZy5PSwyeecKVoeME/STuojWHDsUGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Jvn0ISo/HgoiboBWB5FbYbJTrYAvUyDADUFepjr42Sw=;
 b=cb1BOiI2Kv9ixkOVfDQ79QYSsvmpXzjAuNDDzqYCTxroWBbATu6+P6VIMjc+3sv+EDmYQ7h8zPFQLam8+d9dreZRQ7Ako4xBSUtkwGfW7Yu3+4iYRp3z+cSR0NU5/kypti4XbGpUqnJOdCb1i7fE+/PNp5hno5vnlFb1K02UeDxmkfgROlTJ1jnCaOC6bLhSjYbe3TUCz7bUaRHAlNBnV+KlWmhiZN86HTKbSQz5p4LnRkdKy5tTvdrQBfWhBC5uDxsaoijnnl0uiKCViyE2UVApEf/n+6WekI4Xej6OjtfC7xErWTy5xJ73w2bdwnKkZRg1GhFWbL0xTaMi8CrQ0w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Jvn0ISo/HgoiboBWB5FbYbJTrYAvUyDADUFepjr42Sw=;
 b=6wUCbjwZbgV4QuMhjrJ6LpDOOGTuNUY4epgyn+Wu1jH2gVK3v34enDYKe7T8JgMVSwTOzHKqTiQ6T4QxB69tuNfvjWWS34TyywuxMCYlUrQt0xYN2YCquI5n4xYhEG9SgGZBFRxpPzwUULqA14fl8JdYs8cJKzKMW/TqZsOmKMw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 01/37] xen/arm: Print a 64-bit number in hex from early uart
Date: Thu, 23 Sep 2021 20:02:00 +0800
Message-ID: <20210923120236.3692135-2-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 594b5d82-14c6-47b7-4c70-08d97e8a2d85
X-MS-TrafficTypeDiagnostic: AS8PR08MB7223:|VI1PR08MB3823:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB382377CFFEE72C35A15CB0199EA39@VI1PR08MB3823.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 0gk4XAwLeFhtHLi9koYwmYtQP12yf72rkz3Wgz4BoRtum2YEeTav1FwXikW/FwyqJXAmlLPkqUI9LXPAv8BguhbN82q5ZdMcr2SSb/+OG28I1uUH16VMKuVXobeM+otoQbdl2r5X7V0h30FDh02W/riwszgorjFZFdWwFbBZgNtL0XuDiaRdJTdS2XVrV5E+Ko0XHZhRdRojg0HAX7chbqoy/Txw+IYr4OdotnTwCVPqQRdhL8x3nminmfhAuxL3vHhzAU/HmGMPEmWKJIrWO+ZnDPNaSaC0ezIlzqqQP0BomyKMgU5LrOfGmPaRrGzPBm2hQg9TRFEu227WKnbh/HPlXzdA1Ku98I2wFI8GC07hXfHx5CLIxFLS70iMKzoPdXHpjl2uqI3DajfBKuuci2XLETmtRtY6aejLXGY85GVsW+E1K5BAtJBrF6J3gw3W3K2OAqjomJxScf0wbGKgGftKPL8fellex228LK6nM0VaaJ8fJbyeETBvJBM5TbTc5yuzpqUDS6pMafpLMYAYsvTWknN7NU6L+kNcW5Qb+55CPAElbW6RslO7djqWdvNHzngNmzxLfpWTk1Zrf3DAmq2/TvHxvU131UQLWo69K8dZXirtzkrVJPDe33tvkuO4sXHTgz8YQzpVVETGYWj0MJEuSNpfQGvQar9sdlBGEoTL2RLoaCC9QpqBtAy8zu6gOl9bPTw4/86FWFXYAIoencZBxftpQYVVG5XxKkR+0TQ=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(1076003)(426003)(8936002)(70586007)(70206006)(44832011)(8676002)(26005)(186003)(356005)(2906002)(4326008)(86362001)(2616005)(5660300002)(36756003)(36860700001)(6666004)(82310400003)(336012)(110136005)(316002)(83380400001)(81166007)(508600001)(7696005)(47076005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7223
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	85b30f8a-48cd-4e87-0b32-08d97e8a205c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	082+bzUt56zH+f5i0vl1OtSc3EgiB8kJjHTrQ98DreaJ/m8DX3X03+KgyGejeoDC3K0Qz0bVGdFwV8/b1rE1JLxh/Ckjq4AVXMjp4vWQc+zcH2rfmvhsf7YjTUUBOrhPErGGoJwrvddPoD++u2GGvP2b8fAW81L/IHp8/b0UbYnY04mxQgkR7syXpf66tUAIphGvl5l4iC1XYuJKaRnzaeYqnzPDmQaFVEMLvdr9M5y0Ga2c8wu3EEIRPefrX1helxhMp8FR7RgCm/yqhmAGsK6lXp+IadyQn6uFbRBimUNl+uIpgJhsE+M3c/abD+i+PLtztxzhv1xHN8DDADMZrlLwZD4JJnSY/HvQC9aJxomjuSWxldiHtwif4lZsoPYzILgP8kxUZRX4HbH6qyKZI1yfHWNM+i/KZ68U32hj+UJX8p1QkXX7Wjm6m9gsCgOb7+Z1/apxMynR4tkepx6qGOu0rPEWprTmP/Q0yBDpAUF1RRJ1nzT2/6nMWrmIQQC4EWt/GOHufxGpTNqZ9WLa3PDxBz07ATcDso2ZsdGJUuRhW2paUl9KibccxjJ8ekooI3wMTX+02hmo0I5oBgE/R50ROIztSFPPrP2OWXO11zi6aO5LjDwHMkTG/tOUCVpdFKe688EdfECuRgpRQO2xqvFTBaflPXZYme+wR2DMNNaD8Tg9WFJ0uVVb7wPw1wEEnoj9J2NF0eUmanLB5JZpQg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70586007)(508600001)(8676002)(83380400001)(6666004)(70206006)(426003)(44832011)(82310400003)(26005)(336012)(7696005)(316002)(110136005)(1076003)(186003)(5660300002)(81166007)(36756003)(4326008)(86362001)(47076005)(36860700001)(2906002)(8936002)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:38.3369
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 594b5d82-14c6-47b7-4c70-08d97e8a2d85
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3823

Current putn function that is using for early print
only can print low 32-bit of AArch64 register. This
will lose some important messages while debugging
with early console. For example:
(XEN) Bringing up CPU5
- CPU 0000000100000100 booting -
Will be truncated to
(XEN) Bringing up CPU5
- CPU 00000100 booting -

In this patch, we increased the print loops and shift
bits to make putn print 64-bit number.

Signed-off-by: Wei Chen <wei.chen@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
 xen/arch/arm/arm64/head.S | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index aa1f88c764..d957ea377b 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -862,17 +862,18 @@ puts:
         ret
 ENDPROC(puts)
 
-/* Print a 32-bit number in hex.  Specific to the PL011 UART.
+/* Print a 64-bit number in hex.
  * x0: Number to print.
  * x23: Early UART base address
  * Clobbers x0-x3 */
+#define PRINT_MASK 0xf000000000000000
 putn:
         adr   x1, hex
-        mov   x3, #8
+        mov   x3, #16
 1:
         early_uart_ready x23, 2
-        and   x2, x0, #0xf0000000    /* Mask off the top nybble */
-        lsr   x2, x2, #28
+        and   x2, x0, #PRINT_MASK    /* Mask off the top nybble */
+        lsr   x2, x2, #60
         ldrb  w2, [x1, x2]           /* Convert to a char */
         early_uart_transmit x23, w2
         lsl   x0, x0, #4             /* Roll it through one nybble at a time */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193818.345301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS7-0006Vq-Vr; Thu, 23 Sep 2021 12:03:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193818.345301; Thu, 23 Sep 2021 12:03:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNS7-0006UY-Re; Thu, 23 Sep 2021 12:03:47 +0000
Received: by outflank-mailman (input) for mailman id 193818;
 Thu, 23 Sep 2021 12:03:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNS6-0005KP-Ua
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:46 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.46]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 94fad7ee-6acf-4700-ba31-be03a623c15e;
 Thu, 23 Sep 2021 12:03:44 +0000 (UTC)
Received: from AS8PR04CA0162.eurprd04.prod.outlook.com (2603:10a6:20b:331::17)
 by AM6PR08MB4151.eurprd08.prod.outlook.com (2603:10a6:20b:a2::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:40 +0000
Received: from VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:331:cafe::1e) by AS8PR04CA0162.outlook.office365.com
 (2603:10a6:20b:331::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:40 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT053.mail.protection.outlook.com (10.152.19.198) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:39 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 12:03:39 +0000
Received: from 82cee3a26ade.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6607CB9A-6AB8-4598-A9DB-7D96F11BFD3B.1; 
 Thu, 23 Sep 2021 12:03:23 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 82cee3a26ade.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:23 +0000
Received: from DB6P193CA0013.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::23) by
 HE1PR0801MB1787.eurprd08.prod.outlook.com (2603:10a6:3:80::14) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 12:03:21 +0000
Received: from DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:29:cafe::39) by DB6P193CA0013.outlook.office365.com
 (2603:10a6:6:29::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:21 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT032.mail.protection.outlook.com (10.152.20.162) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:21 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:16 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94fad7ee-6acf-4700-ba31-be03a623c15e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qPIL4uOxlIkWYH7gl2Qha1lsYRJ9hENlygoMT/WdWDg=;
 b=FPUwJr6Bcqw32TxYJGyhzXXnUnlgvka6mmIAwg7QWWOzKrzDBLsadKhY0GpjNKHZAgPp5PaP6VR5oQ8nepXvBiTAKg1npKDNzfVDvrT/bGYpyhbJLWd76gl50YaIqaO8+cIyvM6neu4RDlQAo7K3z+dsMCxk9WXVz4tiP8NXJfY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: aff0860eda26db16
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WguwpjDgffGzGmCOjpptiVNbdB5ARbRxQBkFoQkXWga29AVz76bmf/b4Bb3VJEjc2xn/X5tuolfiCV46JQwJmOPvKrZxw0FHK0SY8shufHrdUpDDhE8Itca1TIMoNNAaUpJjTFqtC6HAXp7RhyfkzzKaUEX5dRK6b+eAcXI3G6D8E2TzS9WFyOCE/pJLf0cFW0kHgboi4xnhG1X4ddtNLjSJT0qjRSQ5KWRfiVQtAvXN+Apg9uv4mL+ggBeu/Mkb5UEnAuDW8DVAkvjlBxIKd0Jjh2G80KDSpo+WztJjDJfcRI0Oo+Mx1GuYwfL/6xeH8oUnNT289h+XxPqSxro9qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=qPIL4uOxlIkWYH7gl2Qha1lsYRJ9hENlygoMT/WdWDg=;
 b=aG9cg9/8imlY/aA0JURYkQFoGnFEakVZtjh8WoFVwECH8cvyg1J7UAUyNpFSqUcFhLngqZx5wQU3y/BQIzA4M7slDS98yZazGwjW+Psbsz+dUI0mke8nkN58Krx3uFb7q1o9ASRTwY0u4qTX1Xrya9nmHAAE71itkmeOshe3yClGEn92xCePffFu3H8gK9E0hPDLXqj0gIEbG+DIhP2IO2jdBIENQDht3v2Y4ivhBfsU7aRyNeH8a4ppAY8LPg9bFBXQ3pPzz84SzljcRZ2nRNjetQuwjch0S06ndZphkEw1mm8Mbqd4T65H2cyiN7WIj77Qaon2tN4UMfcJdWKJIA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=qPIL4uOxlIkWYH7gl2Qha1lsYRJ9hENlygoMT/WdWDg=;
 b=FPUwJr6Bcqw32TxYJGyhzXXnUnlgvka6mmIAwg7QWWOzKrzDBLsadKhY0GpjNKHZAgPp5PaP6VR5oQ8nepXvBiTAKg1npKDNzfVDvrT/bGYpyhbJLWd76gl50YaIqaO8+cIyvM6neu4RDlQAo7K3z+dsMCxk9WXVz4tiP8NXJfY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 04/37] xen: introduce an arch helper for default dma zone status
Date: Thu, 23 Sep 2021 20:02:03 +0800
Message-ID: <20210923120236.3692135-5-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6b836271-0681-4ed1-e560-08d97e8a2e78
X-MS-TrafficTypeDiagnostic: HE1PR0801MB1787:|AM6PR08MB4151:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB415103EC7436240FECE597C89EA39@AM6PR08MB4151.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AOwDhD/TmYkLa/GO9jAAw9enNGXp5w83DjDe55s6mMacHPzb3G3SpB0sdBpw3x4oT0udK7m0Yb2LcH0wleZNSLWq2ANRhD+LUAcUhMVUEL+ZcpRfGFMTuswmkaTzoiXwjjE2zOHDVx/PtTTRmwxaDgF3K+J272WXTW36Vf3LK+2pdh4OaFXMYKdX5p2arNPMEk/0Hrj8tz5uiPnSS3yhgigdUQqqBKP5yV8T+pHKBmKiJZKFNYXRkNmCXB5VZ8tj6uq3jzznV0DOhBMEkRx32DF2vLP+2BQjLoFaPg2sRRirNmY58lhL4HXnRMMUjBmScDKexpN63qzC3UbbMy8oNINmyZAOl0NJNk6Umcb784nvKBA3bRcxNcgJq2KOkKIDeVOOqLkIEQNrbdugHHxKg4sfI/pXIX7mN/j3Q46b84e7XY3KUXCmapMdxk7lEa4xgSUKHvU/UmtmFWWtB4NqgrirOsW5BtfyfLPEAaxauWra9xizQfYR95Eta4dk42YXOTSDK9U2qqhuz0v/UAvFRbppHW1PZc/QbI6zE0AB1NuTgNwl32+8B5Fu7xscL3/uQ8XNx4Ge06sLKcQpQEbIhoZl+eYwvOYzW3ZmhtW3y+meW5f6YtV67OZFanm+2Cw5HjhYG2XKoUySQ08wGzJQXbm5brXU3Q4e+ErLUltdJzB3rhi86iS8GZugxekRT0VHfOIKQH22QjQ3Wrm93GdrNkvBC0onMwCxt6yOuzEuZvM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(8676002)(426003)(508600001)(44832011)(70586007)(82310400003)(86362001)(2616005)(316002)(336012)(81166007)(8936002)(70206006)(2906002)(4326008)(5660300002)(186003)(1076003)(36756003)(356005)(7696005)(36860700001)(6666004)(26005)(83380400001)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB1787
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	52c3eb3d-7cce-4a24-3bff-08d97e8a2333
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ulnXWjjtAl+f0X5ZF5BmbiTCTSr+5XK1fnakEfgIet6V9D04b6BkuJF1b5qYAJQ8SKMJVYNOvaHNUVWBueDcO+XEJ/fAVQ3S6FK/1PHlHthiuoux/g3n9qRbOwk70SPgBTguodYvvd1dOKM+WAQXNy0Ub2MvM5TtLtEz3+uh8LVqB9E6d5SYQ0hVmFKuDilyGE/RHknFGr95UtfF5a1Z7NW8wxya0pJrx2GhoW7a60XXDr0WS2SST+WIZDOdBUuJYSZJX0vx7eZJxmEYqSOmmdh6dBnP39mrpuFfT1z7inQJEbPqtREJuPoy0V49szpR4nGu1XshumKslqK8mjLnjqlBewXSQ3oOU/SoRyiRHd/Si5At+7M4GdzCkRPsdUtzXlAWchWhDpbIvSMJm9zQguA2MPcj/7p9X26jUQzBKOc9ka0ggLajt28SUgdUWAz7XvrzVxpQOGPrXDTbDO/w9qx+gBYTA8QgGBjFvK5u8FOk+UJBeuYwCdoeVDRwTG5eCiY5eRs5yjFIeOktXNzd4KI8/188dCAb1NsTlrFbC3P4iQocYFyvccVus6xzltDTkI3cvHNta9u65lzF0nwqgu0hBfRzfYcf9tHqoyl/ec9P/kPRzeW9NZmY8DfOYXPqblkgI4Ppd238SeU7JOlqWN8U3Awul+M4NHxHgoOk6wsJyasITQ1j5qhs4dMAYigyV3BUoXCG5OuwumsH3mrhzw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(81166007)(36860700001)(426003)(5660300002)(508600001)(336012)(82310400003)(2616005)(2906002)(6666004)(36756003)(7696005)(8676002)(26005)(83380400001)(1076003)(316002)(110136005)(44832011)(70206006)(47076005)(70586007)(8936002)(86362001)(186003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:39.8789
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6b836271-0681-4ed1-e560-08d97e8a2e78
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4151

In current code, when Xen is running in a multiple nodes NUMA
system, it will set dma_bitsize in end_boot_allocator to reserve
some low address memory for DMA.

There are some x86 implications in current implementation. Becuase
on x86, memory starts from 0. On a multiple nodes NUMA system, if
a single node contains the majority or all of the DMA memory. x86
prefer to give out memory from non-local allocations rather than
exhausting the DMA memory ranges. Hence x86 use dma_bitsize to set
aside some largely arbitrary amount memory for DMA memory ranges.
The allocations from these memory ranges would happen only after
exhausting all other nodes' memory.

But the implications are not shared across all architectures. For
example, Arm doesn't have these implications. So in this patch, we
introduce an arch_have_default_dmazone helper for arch to determine
that it need to set dma_bitsize for reserve DMA allocations or not.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c        | 5 +++++
 xen/common/page_alloc.c    | 2 +-
 xen/include/asm-arm/numa.h | 5 +++++
 xen/include/asm-x86/numa.h | 1 +
 4 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index ce79ee44ce..1fabbe8281 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -371,6 +371,11 @@ unsigned int __init arch_get_dma_bitsize(void)
                  + PAGE_SHIFT, 32);
 }
 
+unsigned int arch_have_default_dmazone(void)
+{
+    return ( num_online_nodes() > 1 ) ? 1 : 0;
+}
+
 static void dump_numa(unsigned char key)
 {
     s_time_t now = NOW();
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 5801358b4b..80916205e5 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1889,7 +1889,7 @@ void __init end_boot_allocator(void)
     }
     nr_bootmem_regions = 0;
 
-    if ( !dma_bitsize && (num_online_nodes() > 1) )
+    if ( !dma_bitsize && arch_have_default_dmazone() )
         dma_bitsize = arch_get_dma_bitsize();
 
     printk("Domain heap initialised");
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 31a6de4e23..9d5739542d 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline unsigned int arch_have_default_dmazone(void)
+{
+    return 0;
+}
+
 #endif /* __ARCH_ARM_NUMA_H */
 /*
  * Local variables:
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 3cf26c2def..8060cbf3f4 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -78,5 +78,6 @@ extern int valid_numa_range(u64 start, u64 end, nodeid_t node);
 void srat_parse_regions(u64 addr);
 extern u8 __node_distance(nodeid_t a, nodeid_t b);
 unsigned int arch_get_dma_bitsize(void);
+unsigned int arch_have_default_dmazone(void);
 
 #endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193819.345312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSD-00072B-IH; Thu, 23 Sep 2021 12:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193819.345312; Thu, 23 Sep 2021 12:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSD-000724-Dk; Thu, 23 Sep 2021 12:03:53 +0000
Received: by outflank-mailman (input) for mailman id 193819;
 Thu, 23 Sep 2021 12:03:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSB-0005KP-UZ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:51 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1eb32525-e5a8-45c0-aa12-c4acde350d6e;
 Thu, 23 Sep 2021 12:03:45 +0000 (UTC)
Received: from DU2PR04CA0157.eurprd04.prod.outlook.com (2603:10a6:10:2b0::12)
 by AM6PR08MB4849.eurprd08.prod.outlook.com (2603:10a6:20b:c3::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:03:44 +0000
Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::f4) by DU2PR04CA0157.outlook.office365.com
 (2603:10a6:10:2b0::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:43 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:43 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 12:03:42 +0000
Received: from f568312eedc8.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 95FF0764-608F-409F-816D-B7FE6DA940F1.1; 
 Thu, 23 Sep 2021 12:03:30 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f568312eedc8.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:30 +0000
Received: from DU2PR04CA0230.eurprd04.prod.outlook.com (2603:10a6:10:2b1::25)
 by AM9PR08MB6049.eurprd08.prod.outlook.com (2603:10a6:20b:2de::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:29 +0000
Received: from DB5EUR03FT033.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b1:cafe::a4) by DU2PR04CA0230.outlook.office365.com
 (2603:10a6:10:2b1::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:29 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT033.mail.protection.outlook.com (10.152.20.76) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:28 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:28 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:27 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eb32525-e5a8-45c0-aa12-c4acde350d6e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7+me4FoPz33FS1cQ5pOAkpgqjLy+g/9qUY65MaXt3uY=;
 b=rKYjq6WQv1Bgrpg0ZOkOGhWOb67ddyFqeC1zu2pWyTLWaNJgBC9WR13BAixZC/k3tt4rrmZge0AKh3UsV4X/aj9bGuP/EOtW8+SWg2FLoFlz7Xnv1jwt4HgCNQibX4d6SjdPk1EeIh+WmVHhZgAiJ5IplsTit4NlqPXeUEZr7Xk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 585b1ce00a515de2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kVNJJfMRr8Ane83w7P6tHN1EJLgkv9HCKHJM5cRUHipLxd8Yj+CUcaz0WZYzKWKAM6cCbZD8jdia51kpvTVDHNBRdGKy+Hc79jp9f4L7LTRdhMrrnHI0RBsIPkljsBX/u0jz35SEVVpcO6e1Ql0lPWKVf3ANwf6S7npYy4s4HZ1aYv9QGGV7yksSryJFyCFW78lIfzcFWoVjXHCDSNqiJZJEj/wYQRcZnOV/390WJRJ5Mz7obLb1JGNyRK2FPxjXYbUlfGGv738s3MrQyIc2GnFr74j6O51eLHeq4Z/R20ltci+yQlkmDQTqW0+ARlvFQq6D5mEhUFvIYgLUzsy+gQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7+me4FoPz33FS1cQ5pOAkpgqjLy+g/9qUY65MaXt3uY=;
 b=V4D92yE0UjSfZNXmoA/tnwYWl3sAAdUCOCjxx3RhEJ8ZL5NbfEYfZGe38eKtTO3D56Zvg79Kk/w1QYSc8uU9RB1YeI3oEo2buoCSsZ1Zmtr+hWE8356Mgq8kjKb5BFnYCcs3aNE+lR8uH/ET6vs2vPN2w4/lPvsALEFN5ntkm6qdbgbwEy/3ILCVEK5tCZW3aa4cYQR2tcJSwTK+2akoV489CIR2Nz96t1HxmMaKwnapY1qBWfxjKq77OGw0fe3rbrcVvQ7Wp07shqv9NI9v02jx+/+VSP4tOgUMqK46akUx/g3r6NwZI88q4Wix/64f2USZQSF7va0crXpwvRoMcA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7+me4FoPz33FS1cQ5pOAkpgqjLy+g/9qUY65MaXt3uY=;
 b=rKYjq6WQv1Bgrpg0ZOkOGhWOb67ddyFqeC1zu2pWyTLWaNJgBC9WR13BAixZC/k3tt4rrmZge0AKh3UsV4X/aj9bGuP/EOtW8+SWg2FLoFlz7Xnv1jwt4HgCNQibX4d6SjdPk1EeIh+WmVHhZgAiJ5IplsTit4NlqPXeUEZr7Xk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 09/37] xen/x86: introduce two helpers to access memory hotplug end
Date: Thu, 23 Sep 2021 20:02:08 +0800
Message-ID: <20210923120236.3692135-10-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 76cdc768-a892-4049-c2d5-08d97e8a3086
X-MS-TrafficTypeDiagnostic: AM9PR08MB6049:|AM6PR08MB4849:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4849678C6864AE8122FDA8589EA39@AM6PR08MB4849.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 RuUUUdBNTw2mQqdB6hKctYEwo7orXlwKISmDAvYkHG0pG/DMP8fazMz0ffjgrqpRVqmdcL4V5XniTIlsDuX4YaUI+XH+Rhj8COl0jqQ96CTm0mYEfiziEn+uw7N8catOFxnub38JZy/zQe2VyjHqRpxEQ7neS3Z1yBmo+WBDQ6pt8xpBVDgOTUpHuZ28vCXp1cRngGS9V9ITr0eTsz1zq0srBjh8t65zL/OjFn8rp5yIW2VfukswBOtxFWMfUAQcR5xvf7DSbx0okWajFkLaj8y7yVlSGTjhFofu49IhyTmeUBAIgaIfh9X7K8/1KDsbvpw/0iptgcbq2VuigbYzVFcITpipa1qwtn9/3xUwjfcByD+8k8c8DviPNjItB5xjAJrozPdbZIqYLcp40F3Ii9CX4hh6zqlN24om2jg5Y2EqSvxTzY92k2iu0g1fOnCArgsBip0Ch8ws9wkGqm9Ph3JVNdMADiqrMOYbV8KAOurv9bBYWlN37O1FJoGxB1H8HRVt7E6bBBmr8qHQ0UANtafxWGyiwQfnV92eH9IGtZS3dsvLxF84ctAkTzEUzhqoAO1DX5zNvDtAedChGHiGjCNqQh0ekGL/q0/1QZaqGWGkztKW2SBtmdhrgRVc32nhmhiKwlIwF+ZFc0rPgAWTPRe/u4cRgslYyuFYOghDzsPC60bKJrmYX1nJPgfomdGw5zlkiQcocZXRjBppz+dA5PlepjYRlXW1mca05NVnXUg=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(356005)(508600001)(186003)(2616005)(1076003)(81166007)(8936002)(36756003)(26005)(4326008)(426003)(8676002)(86362001)(336012)(47076005)(83380400001)(44832011)(36860700001)(70586007)(70206006)(7696005)(2906002)(110136005)(82310400003)(316002)(6666004)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6049
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f642fd07-b7f4-4bc2-282f-08d97e8a27e5
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kL8SGwFY1oL/Yhchkippm4uPktKmc/NMabRtjfKqZRK7yliz0ZQINv9+Mwy7dLmG7mYD8p+QRWei3dTVDN10qWmktis5aygau4kvbV+ZrO3BUkBKmMxR71AfGVE7gAYv1cPlZZF9Msp14mSzJyW1WkNByytHBe/R5ZPAQZPmroRqww/v6HfpAQJDupa05ajRTD/YByAVQQ7pkexOta7r5atpd1AmifJi3nine2z5lLAviPVPlTtV82seqwUVyIa43rOUDNSxn5cIVcjKXj+kDaHJxn237V76bxVrKSYojJfZWC3jFiEcu5sl/4MStRWcmxNGB2qM8GW34RKPoBUR03AXwZgm6pd5sn/wnlrOr9QySVGqSCkJd+OkBrwOzfaq9yyKG0cG2IoX3mhliT+TrE61/QWrVuV145XOzx0Et3CBRJ9V62z1MZZWXhl4nu+Mg0ipNx2FvNIxCjhPdV4F93ANDXb5DvyphIuDBJs9aiLm1/dV1PTk1VSr9lsEfXIVm7Y/t7QJlbl+JR7Jps/ynrHgMu/ooGe1YHk2IHdE123/bwWRPXGA/juRL5YUcrdhb0SDW+rYH8MEv+er+IDmUufjKdjItjqvmxp5M25ccoVCxn5hTfm+nPn3FyO3vjMwRvYB3OTHUIMkEak8VCM7kMh95on8RvPv+dbRbXTbHmqoXILBzm/0OqQHhPmFjA+pZZuidintkCOqnWg3TmtxZQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(1076003)(86362001)(7696005)(36860700001)(2616005)(8936002)(47076005)(186003)(508600001)(81166007)(426003)(83380400001)(336012)(70206006)(70586007)(5660300002)(2906002)(6666004)(44832011)(26005)(316002)(36756003)(82310400003)(110136005)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:43.4270
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 76cdc768-a892-4049-c2d5-08d97e8a3086
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4849

x86 provides a mem_hotplug to maintain the end of memory hotplug
end address. This variable can be accessed out of mm.c. We want
some code out of mm.c can be reused by other architectures without
memory hotplug ability. So in this patch, we introduce these two
helpers to replace mem_hotplug direct access. This will give the
ability to stub these two API.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/include/asm-x86/mm.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index cb90527499..af2fc4b0cd 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -475,6 +475,16 @@ static inline int get_page_and_type(struct page_info *page,
 
 extern paddr_t mem_hotplug;
 
+static inline void mem_hotplug_update_boundary(paddr_t end)
+{
+    mem_hotplug = end;
+}
+
+static inline paddr_t mem_hotplug_boundary(void)
+{
+    return mem_hotplug;
+}
+
 /******************************************************************************
  * With shadow pagetables, the different kinds of address start
  * to get get confusing.
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:03:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193820.345318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSE-00076u-40; Thu, 23 Sep 2021 12:03:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193820.345318; Thu, 23 Sep 2021 12:03:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSD-00075r-Q4; Thu, 23 Sep 2021 12:03:53 +0000
Received: by outflank-mailman (input) for mailman id 193820;
 Thu, 23 Sep 2021 12:03:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSC-0006xV-8h
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:03:52 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.87]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5095996a-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:51 +0000 (UTC)
Received: from DU2PR04CA0162.eurprd04.prod.outlook.com (2603:10a6:10:2b0::17)
 by VI1PR08MB4158.eurprd08.prod.outlook.com (2603:10a6:803:e6::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:49 +0000
Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::9d) by DU2PR04CA0162.outlook.office365.com
 (2603:10a6:10:2b0::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:49 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:49 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 12:03:48 +0000
Received: from a328cebf5fe4.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 52144751-B383-4D99-A1E9-13435DC9C057.1; 
 Thu, 23 Sep 2021 12:03:36 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a328cebf5fe4.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:36 +0000
Received: from AM6PR0202CA0059.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::36) by DB6PR0802MB2375.eurprd08.prod.outlook.com
 (2603:10a6:4:87::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:28 +0000
Received: from AM5EUR03FT009.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:3a:cafe::ac) by AM6PR0202CA0059.outlook.office365.com
 (2603:10a6:20b:3a::36) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:28 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT009.mail.protection.outlook.com (10.152.16.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:28 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:21 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5095996a-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GcILUNZkNyOp7caaG1E8DDdtnvtY8zrWXypkM6x7c0E=;
 b=yTWwiwJFNWT7FbZnbCIXO3KF02rJVoinulqD4DyfnWLhZRg2t+u19F8f42gTE94xTBeTwgimq/gOVTbJOlm6WTmWN7mwc9O10oR1vdqF6GrfblYAVfhihfKMyx3VqhmCwCL3gVI4dcHQOXU1k9gW95oc419xR0bLwuiiOP1Ba14=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 4ff5c38518d203d8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CUxJnp4KkB22S450kfX906L/zN8PU/Nn4NrXnjcRgsFCyn4DnZUE+aj99WTW3GTRz4dLMhibkf7V0jRW0ouOex2ZaBZf8a0XNvAPp4oQT31RASAjFtcajgQypzRMAWxZoEjZFWKUfc9EzqcB8uPJfiFTrPyBrK6QM9r3F4VuWPYaujrQLTvzdCRSRZAi+WjGnhGoNCHCmsNnAtyhBAs5RTkGM95BQXpx0n6qiAg8fDBKGniui2b2JBrvWHYbD4lp8CuANecF+kFsMmnMwaHR3K8G+V9asM+kbzHDJeJ61pjdzquciNCWfHYQlOmIeGS2PPIT0xQs+v4R8EPLdWxeVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GcILUNZkNyOp7caaG1E8DDdtnvtY8zrWXypkM6x7c0E=;
 b=O6lWhgrMBq+jFAp3hZ3UZFzzQ8hjXpFkWrEAkf1avrJ5jxnIarpFQeO12n5JC8nm54bDyys2FycIR9a9j+WLVXS68LYxOOcB08l7oI0M+HdntUNyehLS1Gs+XgXSlFXSNDWVqO8WSger31mV54c1loQUNcAeh6YJBTHXjbDKPPEJvnpz4dfpacXD+4poZQxV7A7f1oSjYw483dwpTIXnsmfxDK0W1GHtW4yhyf5RNVkUmqCCzPtx4EEPOaJjDgJYw4xifBmZsquVkaLOdFNwsHb5Ds2gQ+l5xOqB38rkpmKmMQAFCMN6a4+r41YhCsr6KgQ1rW7Xn6qgrG+douvV2Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GcILUNZkNyOp7caaG1E8DDdtnvtY8zrWXypkM6x7c0E=;
 b=yTWwiwJFNWT7FbZnbCIXO3KF02rJVoinulqD4DyfnWLhZRg2t+u19F8f42gTE94xTBeTwgimq/gOVTbJOlm6WTmWN7mwc9O10oR1vdqF6GrfblYAVfhihfKMyx3VqhmCwCL3gVI4dcHQOXU1k9gW95oc419xR0bLwuiiOP1Ba14=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 06/37] xen/arm: use !CONFIG_NUMA to keep fake NUMA API
Date: Thu, 23 Sep 2021 20:02:05 +0800
Message-ID: <20210923120236.3692135-7-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 953a3fb4-ef0c-4d47-4e80-08d97e8a33ef
X-MS-TrafficTypeDiagnostic: DB6PR0802MB2375:|VI1PR08MB4158:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB4158519935F134F6E99A50179EA39@VI1PR08MB4158.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2512;OLM:2512;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 IYJzVLVHJm9ARQVk3lQE2eMUzEsiFSRl2kZ19AG1hqOiozAcNViMzhbrPLGE174snrq4QcV7XhGtb3qMUzIdfuZZUO0Uze12f2sEG/RfsWytY1SA3gENfaPtnI+C+ZP+teG3ws6s8p6NlTnn89WL+Kwov4XcAAQIkj/tPPghGTHFodVVI4vAyJDZrtgVg45KCQbN/8KToXoFpUDQlq59HYzrazT8MFnuu90r1nUGQiGOoqEm02m3nXdeArEp8NSZfjwEzTpDG/+zUqy/dtTI3oKnrt1uWRMKWIHpwcEYemuDkw95asvLzCqmtY52FhACu1S6bn9S92/uUQFxNaihqMPAlOcoY5qgPuMAH+mee67yP7D9vvyX4/7+PSl/24AUhp/kFG/PmaWAto2tolM7eLwD5+lnCr4XVo9A2vFyrViLcYgKGphF65wC8khBM1w2BXkJ6uFuCu9874nj7NTKwHbKQY24oqrAGp/kMMSpmstuFJbR2VdEv0lR0iYdEiF+WKqZ5a8Veui+f4qHwpji9RT8Mcp6/XYIxAdQf9qJAHo/p+jpboQCFSO+1RV3GrLB9oV7t0J4qhhwOmZVllyNCJvhzZHVw89oYy6UofH8BcvigS2AIvWLPe0x+SRnaeM36fp9AujnHK7QUBT3WOy/wTP25NlrnDpYNewZOHllt9JEOO1wg3u5GjWzCxFGVZ5WHTfKaSBt2AHn5zmCLErl3tNmGUEtU6svKn2f53Ni4eI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(1076003)(7696005)(426003)(86362001)(186003)(508600001)(36756003)(36860700001)(336012)(2616005)(47076005)(83380400001)(4326008)(82310400003)(70206006)(110136005)(70586007)(6666004)(2906002)(8936002)(81166007)(356005)(5660300002)(316002)(8676002)(44832011)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2375
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	92c66c39-fdd7-40be-a709-08d97e8a27a0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	85r/UrjY0Bxj2pO3G5DuhAd1UdkRlv6Mj72cOkMjf2Kd3HWvinUncAH+XvV6oniL2bizFGXm1WBzVWlbd67NC/cI25WgEAASw/2zYxWeaEAAM8+816AT4kd349c1FK/qQN3x4Cfy16QtQ6apArBqSgR0pQsRTD7Xnreq6qahV4qVO9d5Ar3vSOxRUw1UR7a2pdexekGFYzlalT6z6TamvzsYDS1DdUi1hK9ERyyJVvabhE/ZeqfxzboGpN3beiuiXuymt4GQ2owtc320RhPZZNTB6ScUwUXJ/BsCcTYIorc2Oa5y/WNSymkGwRPOWshnC7Gh+uCVEz3rv+z0nr3J56fDFikhzuTTvQpeMy34cTFJxqIsR9KhbvNS5VWCxUlUWiaU7QXZwlEgrWOFciK0GjuEPu5qQ1wKaZrknWjXAh4TH7nWJtUyhoFrW95yiUGD2oAMMxxCTGQNeb/Frjddi6pAaS5F7WKc72L57uKLaBrWXqfanLMa7xLMmSqCqhIyZpmM2f7+WgbkuLi5/ftWRZ2rW7Un4wfSHFf3KdJJUyn6vRpogd/LgWAFZxLfPSXqiQE5ksBi3Fx26DWFh92ASjmC0fGhf8WiJIz1KKGLoe2OhsTP+Eejdj8wXV6khVx8rLPYjqoUTSWXVdAcc+cNmxOWPmapkIwsCCZjAjfIstdj6WV2LI5Lv70FT2T1iaQLFsgi0nuFvHplKIXxEOPcWg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6666004)(336012)(2616005)(316002)(5660300002)(86362001)(110136005)(426003)(4326008)(36756003)(47076005)(186003)(81166007)(2906002)(44832011)(26005)(8676002)(83380400001)(8936002)(1076003)(82310400003)(36860700001)(7696005)(70586007)(70206006)(508600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:49.1453
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 953a3fb4-ef0c-4d47-4e80-08d97e8a33ef
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4158

We have introduced CONFIG_NUMA in previous patch. And this
option is enabled only on x86 in current stage. In a follow
up patch, we will enable this option for Arm. But we still
want users can disable the CONFIG_NUMA through Kconfig. In
this case, keep current fake NUMA API, will make Arm code
still can work with NUMA aware memory allocation and scheduler.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/include/asm-arm/numa.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 9d5739542d..8f1c67e3eb 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -5,6 +5,8 @@
 
 typedef u8 nodeid_t;
 
+#ifndef CONFIG_NUMA
+
 /* Fake one node for now. See also node_online_map. */
 #define cpu_to_node(cpu) 0
 #define node_to_cpumask(node)   (cpu_online_map)
@@ -25,6 +27,8 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+#endif
+
 static inline unsigned int arch_have_default_dmazone(void)
 {
     return 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193823.345334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSM-0008FK-FW; Thu, 23 Sep 2021 12:04:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193823.345334; Thu, 23 Sep 2021 12:04:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSM-0008Ea-A9; Thu, 23 Sep 2021 12:04:02 +0000
Received: by outflank-mailman (input) for mailman id 193823;
 Thu, 23 Sep 2021 12:04:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSK-0006xV-Nj
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:00 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.62]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 51a1c4be-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:52 +0000 (UTC)
Received: from PR0P264CA0139.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::31)
 by AM9PR08MB6915.eurprd08.prod.outlook.com (2603:10a6:20b:30f::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:49 +0000
Received: from VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1a:cafe::d9) by PR0P264CA0139.outlook.office365.com
 (2603:10a6:100:1a::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:49 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT064.mail.protection.outlook.com (10.152.19.210) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:49 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Thu, 23 Sep 2021 12:03:48 +0000
Received: from ec9bdfbe0915.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 04BA987C-A848-48A6-BA73-490F24ED3F12.1; 
 Thu, 23 Sep 2021 12:03:35 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ec9bdfbe0915.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:35 +0000
Received: from DB6PR07CA0021.eurprd07.prod.outlook.com (2603:10a6:6:2d::31) by
 AM0PR08MB4322.eurprd08.prod.outlook.com (2603:10a6:208:148::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:33 +0000
Received: from DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2d:cafe::cc) by DB6PR07CA0021.outlook.office365.com
 (2603:10a6:6:2d::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.8 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:33 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT019.mail.protection.outlook.com (10.152.20.163) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:32 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:33 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:31 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51a1c4be-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rv+ePpm3yVoMrcs6MF1PccZI0uYzJEbD1D1+NGjLnZY=;
 b=Anai8pLH5joOUnO6M+9wv1bFAgkUmXITxrBlIzNr2FQAJFNbJNwUk6SZVD9FaxlmMiXecqHF+8dVALOMAT4jRsEE1WVbJrAWOZVEFSIverbAKX1RS/SIRwUL3sSKE+AbqHmQPJD0fezhoD8ne5HTqpm88u9+g/EnHXBMn64hQYY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5c9c0cfae6dac015
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=caz+a442gJEyjiBHQq/LjUEUYkTHZCKBOrNweA+RyRafJ0Nt3tRT2YLZyWlg+q/Ggfe8/+N6aeBsKInC0UFIJeqLK/Cqgd8p0af0mlE/6FetTY5+DEM2SG9s87K2tlziY8B9+41VggDlKwsYTaVGnaKQ2lY83aZSwDiZPZHMpkatiHuiPTIJ9bFbparH9vcGElwXW0eRc5eEGPE85rrEYzpGLbJc9C2524LCfnjUKTiU20lWKV7Gb0segajXw2vP+rWEwr1nrTcEvOzJ2JJELlY9Obx1by0Ne4D+Fp+eeaQwCdFS29YGf7ooZAaC4haU8M2krF3ki6yF2qy5Ez6nNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Rv+ePpm3yVoMrcs6MF1PccZI0uYzJEbD1D1+NGjLnZY=;
 b=NrrE/cJ4Cja/BXbkht3VAPTzmNsykgoit364WxV6FfrvwD6jLy/VuehNjyPDMQln8SoDSC6NCY575yo2CquWVrneU2z8PO3rRTeUtje0DFqaHLEaGTwcxoBPX5UsIQJK+bTjccdhbCgHpHZuqwulE6L8ePzsm5gZiSJfnCmWkZ05bBx1FGqO44z5+3rHJYF2OTDmcH6pxGZjAuEH4PrMfK9qKKuo/WGcyGPQU1nAsLV7BtiuBhTstekI0CZl2WhF2U5/p4JoMJpq0U9OaLsHk91mZvKlqwixgIW7quVWeameZNpL63NR+jFHqiHmfq9SDcAFiUaAEkQlYI4MFqM1aQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rv+ePpm3yVoMrcs6MF1PccZI0uYzJEbD1D1+NGjLnZY=;
 b=Anai8pLH5joOUnO6M+9wv1bFAgkUmXITxrBlIzNr2FQAJFNbJNwUk6SZVD9FaxlmMiXecqHF+8dVALOMAT4jRsEE1WVbJrAWOZVEFSIverbAKX1RS/SIRwUL3sSKE+AbqHmQPJD0fezhoD8ne5HTqpm88u9+g/EnHXBMn64hQYY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 11/37] xen/x86: abstract neutral code from acpi_numa_memory_affinity_init
Date: Thu, 23 Sep 2021 20:02:10 +0800
Message-ID: <20210923120236.3692135-12-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 89db300a-2116-4361-844c-08d97e8a3428
X-MS-TrafficTypeDiagnostic: AM0PR08MB4322:|AM9PR08MB6915:
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB69159010AE55D53F0FD95E8A9EA39@AM9PR08MB6915.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3276;OLM:3276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 0BvzLm+wBVJPHsPB+W4VPN4D5VgiLLjEMy6E4snzslUHLMoaPiqc+YEVJwTqXiQXXgvswNJier6CZBSzUlhkHSiP00j3oxhxzL0BLuP1IiM9ul+gVWdDuA2nCyD3izu6Ci65Oh93oc5XsiBY0gUwQcQLwGcXD8PI1EXgjh/QWeQCSrA2wT69dJtK+cdehtisXfGqhQK0M5Otqgv2LeNUnEqDCzMIIO2ImyM+koYV9uZH8smc4lVIAsh2/+zAprwD92N+LSO6zCxtqdAoGHtp7WApNRsMQVvuYXdMoaGrbE3Zx1k41aEPmNdk+eFEXD38sPptRbcDE9BQul9oV3TxNPJdeCJhNIpF9wSqAkMPVE1v209R0LaBBLtUXkpo3lH1rfIrzgKLRXeD0kVjyNowkxfk6XEbKARFsmqONWtPpzPFlZLnMDn7x9MC5Kha76qzdR8Qiaf4xHCnbv2C4e9Swb6xhsteNPKywf5+h0dOsTSZ5t6pIMblOASt3/8o6FUaKtZu5NiA8D2xP3xqkxrltcycWatYgzbRjvlIJodT+REpepdIiEaafVN0Cm2nvPJiIGjq2sKI4nWa1VE3dHSpBdzG7ZYXDqBEAQgf1GbDKeuh0knMuNUJvdn4pzhgcsI66Dj/7WgY5DMwtHG3G9XublxzS8GqK0ot1Ag2+YXiboDM5M5cOOjG1geL4fOicWv1ShAsNk+4V4UVx6xbJvyLOuQ7Pc5twH3vB0EjE2dg6tw=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(426003)(47076005)(8676002)(44832011)(86362001)(186003)(4326008)(336012)(2616005)(83380400001)(508600001)(1076003)(70586007)(316002)(6666004)(5660300002)(36860700001)(356005)(70206006)(7696005)(81166007)(26005)(82310400003)(8936002)(110136005)(2906002)(36756003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4322
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	fbca8517-66ee-4a60-4ffb-08d97e8a2a34
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/8u5CeDp/RMSFVHId4o0u3V9GZpWYthLVF2m3o6F2MpOwtfdOPYlG1xtI8iX7eo1CHhCjdzz3upyFkdIuwJKsZOg97YI2MYtgs816Vn2TCiyC0kPAbv3xWKMPXvBDJRKnBkHskcigphP/7+V23pGWjgEkaRkXCIlExnJCIpGROrpEZPgh+dxPXiVC8eZBGVEYCDgp8JhfcE9yGK0rUGjU2yo2tXvyw+uVTnFdEd8/q6O2kBzcsYmeZ4UUkQ0muimWROpdPkk2VlL3a6WLLlu4LtiXhRZWpnoH39DN/I6SbvgydFVVH51Fz5hQQWMsGyXF5Vhz/PB67OTmwWP3g2jPLDzXKNeous7DAK9rairq2GvcQ+e40Hix5Kx8xd3NFlKwH+eOMhkdJQuJhel656H2NK8b9H+TueAyA7cmN8A0FLk9MMY7WLbFf5/IsfE8gybXCmHrRrxhyuKF4b58t7eLAFalRdzSdL6qeuMfXR/mpwXWZJW0HA7Oa6ydksw9WWVWAvFCQCwgKAVT+L/KmSmYeAJ2684SquJ1L9tyilYElIY4ryuDkqRtxBKx00b1fOt0ki5yaa5GWu2vFx/NZrsHNklpYQlQpKQilSrWw5U6Y45idtHialTsvcRRaMu83jxgzDZWkguIyJ9xanzPYf+nYn1z7kjVBq7tRe6UImo43EIJsjB1GRw/aKZl9eHF36xs1rYRit2gpdefg5O6SfZzQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(1076003)(26005)(44832011)(81166007)(86362001)(70586007)(4326008)(83380400001)(47076005)(6666004)(36860700001)(508600001)(336012)(82310400003)(110136005)(426003)(8676002)(8936002)(70206006)(186003)(316002)(5660300002)(36756003)(7696005)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:49.4274
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 89db300a-2116-4361-844c-08d97e8a3428
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT064.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6915

There is some code in acpi_numa_memory_affinity_init to update node
memory range and update node_memblk_range array. This code is not
ACPI specific, it can be shared by other NUMA implementation, like
device tree based NUMA implementation.

So in this patch, we abstract this memory range and blocks relative
code to a new function. This will avoid exporting static variables
like node_memblk_range. And the PXM in neutral code print messages
have been replaced by NODE, as PXM is ACPI specific.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/srat.c        | 131 +++++++++++++++++++++----------------
 xen/include/asm-x86/numa.h |   3 +
 2 files changed, 77 insertions(+), 57 deletions(-)

diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 3334ede7a5..18bc6b19bb 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -104,6 +104,14 @@ nodeid_t setup_node(unsigned pxm)
 	return node;
 }
 
+bool __init numa_memblks_available(void)
+{
+	if (num_node_memblks < NR_NODE_MEMBLKS)
+		return true;
+
+	return false;
+}
+
 int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
 {
 	int i;
@@ -301,69 +309,35 @@ static bool __init is_node_memory_continuous(nodeid_t nid,
 	return true;
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-void __init
-acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
+/* Neutral NUMA memory affinity init function for ACPI and DT */
+int __init numa_update_node_memblks(nodeid_t node,
+		paddr_t start, paddr_t size, bool hotplug)
 {
-	paddr_t start, end;
-	unsigned pxm;
-	nodeid_t node;
+	paddr_t end = start + size;
 	int i;
 
-	if (srat_disabled())
-		return;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
-		bad_srat();
-		return;
-	}
-	if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
-		return;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	/* Supplement the heuristics in l1tf_calculations(). */
-	l1tf_safe_maddr = max(l1tf_safe_maddr, ROUNDUP(end, PAGE_SIZE));
-
-	if (num_node_memblks >= NR_NODE_MEMBLKS)
-	{
-		dprintk(XENLOG_WARNING,
-                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
-		bad_srat();
-		return;
-	}
-
-	pxm = ma->proximity_domain;
-	if (srat_rev < 2)
-		pxm &= 0xff;
-	node = setup_node(pxm);
-	if (node == NUMA_NO_NODE) {
-		bad_srat();
-		return;
-	}
-	/* It is fine to add this area to the nodes data it will be used later*/
+	/* It is fine to add this area to the nodes data it will be used later */
 	i = conflicting_memblks(start, end);
 	if (i < 0)
 		/* everything fine */;
 	else if (memblk_nodeid[i] == node) {
-		bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
-		                !test_bit(i, memblk_hotplug);
+		bool mismatch = !hotplug != !test_bit(i, memblk_hotplug);
 
-		printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
-		       mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
+		printk("%sSRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
+		       mismatch ? KERN_ERR : KERN_WARNING, node, start, end,
 		       node_memblk_range[i].start, node_memblk_range[i].end);
 		if (mismatch) {
-			bad_srat();
-			return;
+			return -1;
 		}
 	} else {
 		printk(KERN_ERR
-		       "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
-		       pxm, start, end, node_to_pxm(memblk_nodeid[i]),
+		       "SRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
+		       node, start, end, memblk_nodeid[i],
 		       node_memblk_range[i].start, node_memblk_range[i].end);
-		bad_srat();
-		return;
+		return -1;
 	}
-	if (!(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)) {
+
+	if (!hotplug) {
 		struct node *nd = &nodes[node];
 
 		if (!node_test_and_set(node, memory_nodes_parsed)) {
@@ -375,26 +349,69 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 			if (nd->end < end)
 				nd->end = end;
 
-			/* Check whether this range contains memory for other nodes */
-			if (!is_node_memory_continuous(node, nd->start, nd->end)) {
-				bad_srat();
-				return;
-			}
+			if (!is_node_memory_continuous(node, nd->start, nd->end))
+				return -1;
 		}
 	}
-	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
-	       node, pxm, start, end,
-	       ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
+
+	printk(KERN_INFO "SRAT: Node %u %"PRIpaddr"-%"PRIpaddr"%s\n",
+	       node, start, end, hotplug ? " (hotplug)" : "");
 
 	node_memblk_range[num_node_memblks].start = start;
 	node_memblk_range[num_node_memblks].end = end;
 	memblk_nodeid[num_node_memblks] = node;
-	if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) {
+	if (hotplug) {
 		__set_bit(num_node_memblks, memblk_hotplug);
 		if (end > mem_hotplug_boundary())
 			mem_hotplug_update_boundary(end);
 	}
 	num_node_memblks++;
+
+	return 0;
+}
+
+/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
+void __init
+acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
+{
+	unsigned pxm;
+	nodeid_t node;
+	int ret;
+
+	if (srat_disabled())
+		return;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
+		bad_srat();
+		return;
+	}
+	if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
+		return;
+
+	/* Supplement the heuristics in l1tf_calculations(). */
+	l1tf_safe_maddr = max(l1tf_safe_maddr,
+			ROUNDUP((ma->base_address + ma->length), PAGE_SIZE));
+
+	if (!numa_memblks_available())
+	{
+		dprintk(XENLOG_WARNING,
+                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
+		bad_srat();
+		return;
+	}
+
+	pxm = ma->proximity_domain;
+	if (srat_rev < 2)
+		pxm &= 0xff;
+	node = setup_node(pxm);
+	if (node == NUMA_NO_NODE) {
+		bad_srat();
+		return;
+	}
+
+	ret = numa_update_node_memblks(node, ma->base_address, ma->length,
+					ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE);
+	if (ret != 0)
+		bad_srat();
 }
 
 /* Sanity check to catch more bad SRATs (they are amazingly common).
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 50cfd8e7ef..5772a70665 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -74,6 +74,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
 				 NODE_DATA(nid)->node_spanned_pages)
 
 extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
+extern bool numa_memblks_available(void);
+extern int numa_update_node_memblks(nodeid_t node,
+		paddr_t start, paddr_t size, bool hotplug);
 
 void srat_parse_regions(paddr_t addr);
 extern u8 __node_distance(nodeid_t a, nodeid_t b);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193826.345345 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSQ-0000QV-V5; Thu, 23 Sep 2021 12:04:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193826.345345; Thu, 23 Sep 2021 12:04:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSQ-0000QA-Pf; Thu, 23 Sep 2021 12:04:06 +0000
Received: by outflank-mailman (input) for mailman id 193826;
 Thu, 23 Sep 2021 12:04:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSP-0006xV-Nt
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:05 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.42]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 51e1b6dc-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:53 +0000 (UTC)
Received: from PR0P264CA0183.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::27)
 by PAXPR08MB6525.eurprd08.prod.outlook.com (2603:10a6:102:154::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:51 +0000
Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::29) by PR0P264CA0183.outlook.office365.com
 (2603:10a6:100:1c::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:51 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:03:47 +0000
Received: from caa7b4dcd7cb.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A11E3031-0ACD-43CA-A9D7-E16F6FA992ED.1; 
 Thu, 23 Sep 2021 12:03:29 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id caa7b4dcd7cb.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:29 +0000
Received: from AM6PR0202CA0050.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::27) by AM9PR08MB7117.eurprd08.prod.outlook.com
 (2603:10a6:20b:41d::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:03:27 +0000
Received: from AM5EUR03FT009.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:3a:cafe::3e) by AM6PR0202CA0050.outlook.office365.com
 (2603:10a6:20b:3a::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:27 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT009.mail.protection.outlook.com (10.152.16.110) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:27 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:19 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51e1b6dc-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Bj0MiZUkjRpuLNYU7r2jj8EyNrfRU+quQC40vND4xs=;
 b=a0GfpOWRG7JknM/pMBuTF26K0ly9DQCzJedWLlJUKC4Ly+Pn1E71QNRDXypogtDNd5rUXsGJqF94cS5uLQXmx1x5oVfG6oHbnI0aLi4WOkDbwrJAPvKQOYhpWdx48/sJAFSc9zwjMt/FRgSh3vwuhUb9WcT2dXRWtfV4wESvx4Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: cf82ffb760e15b2e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=da8nWAOCURge37dfwRPFct2SZ1BQ4LF/AgWfRK6q//W6UUeFwOKhgJEwvLMoqqWqyqVtZpQpkpeBP7/vPAUuLzIHIfigQDktKw014jymWKxChM7v/lfgho/ayDfg9oLH4eMSqFesEj4qsRQdFHXWVjYgmZETVeuxU93/58DXXEsvWZBkHE7rNoH8HHoBWxvueSXK7w72tZcRRZRJvbpKxiNOGlsU2mm51T/+AjWy+M1mc0XX8kUIQNYqv4VoAliYeDi19SSC88OKEgLJnAvv1jy1YbxzyaOE9XltvIm9l58SWCkspLxJBDGhMlpdWiJDmzuyJ2XzaRmSa9NzAyaVbQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5Bj0MiZUkjRpuLNYU7r2jj8EyNrfRU+quQC40vND4xs=;
 b=hPP6iOn6UOFbpgVxkaV+k1XPMNueoirTjkG1WtdQ0E0sPtPr9IrRtK5eg23Ov1Vmn/0CuNWzVR1WCacAQ0QQhid2qd27bpxh/VZ+gxC0qK877s2bxCYAbYvJwL1VO/2FI8b1xRxOiEyOEwUP6S6SST4NeZ6ITbd4GYnJcpMEhGtWr8Yq6OGxx72spHNMpZw2uGqxvuPodcehK4ePY7j7GmmJqs3vGM7xpO6OzgM50hXnUDRHfRa3R0lTlVxjAgBrdFmUP8KiFKnaKwluvVXifzDR6BY6Jaz7P3PAdrJbp0QtG5DjY+pCu2MAsDnJH1Cm3zAGemm5GR02bewRtj3G1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5Bj0MiZUkjRpuLNYU7r2jj8EyNrfRU+quQC40vND4xs=;
 b=a0GfpOWRG7JknM/pMBuTF26K0ly9DQCzJedWLlJUKC4Ly+Pn1E71QNRDXypogtDNd5rUXsGJqF94cS5uLQXmx1x5oVfG6oHbnI0aLi4WOkDbwrJAPvKQOYhpWdx48/sJAFSc9zwjMt/FRgSh3vwuhUb9WcT2dXRWtfV4wESvx4Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 05/37] xen: decouple NUMA from ACPI in Kconfig
Date: Thu, 23 Sep 2021 20:02:04 +0800
Message-ID: <20210923120236.3692135-6-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e70ab8b5-14b4-481b-383f-08d97e8a353f
X-MS-TrafficTypeDiagnostic: AM9PR08MB7117:|PAXPR08MB6525:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB65254EAD4A5884D750EB3E439EA39@PAXPR08MB6525.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:935;OLM:935;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 WGTXvtcGlJfOrMskUH8PFEP99JJ+k79pMI0AmoBzPCRXFzGyFmpvSTboI5WWNpo+18J38XAotRpBPiWQ7R60Zp4olsVc4HwgyuxzZrUPtPjq5+oIluSwDxnMccHrxFRtpIpU/CA2J3UWzT1bvIg1aCf91N7kyEST+qcFcuXjOvaOvDsvwNzqYaoa9bcGd+wi1VQ3bio2yOCPIatKIHuwai7tBC4Vn5s6JfkJnZReDLRSoOLilw5O3qrBewKAvqecLVTnX5sGVtAGH9t1v/hAoOF4QvbXgWW6TgRxeJMPPd993OPy7qneiVrvLGeeMhDo8EjypxvzFuBdWwIAEahePglVfQzME0VhLmmSMwNg4t5iQyFYl60fP70qsA7T7JVymr+Dq6wKWAqR0IcQ0PoRdXlthU9k13s/BrkgHl/u9naxIjhUbd5BloaolMxLcDowi7rNI6cQEDzZM1/6D+MEqeTvBv/NsbtMYQDinS1Y736lgB5VeDjX48CMOge/14ZCs7nOO8WdG21cg80EXt1mQpjIdvNljPw07DIQMSuRcG2nBnBU42XkM0EtDw3Tk7X7skIoPcAI6R1Zm4C6u3mq0qNPkXXJVxYNyUYF2TGPsGZY6KsmoVF5ncAMABcPJpjd0nvbf2NCmvfgtalg9Db1WcaFl9qds3iNG8OBCngEyyHp66Ip1CYY1hAOriNC/fwmR6RrxtPlepQv2JRdir7z60AwqMcR5k4FgRRhUqBw8Eg=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(7696005)(8936002)(2906002)(356005)(81166007)(508600001)(5660300002)(36756003)(26005)(2616005)(82310400003)(36860700001)(1076003)(47076005)(316002)(8676002)(70206006)(83380400001)(336012)(4326008)(6666004)(186003)(110136005)(86362001)(44832011)(70586007)(426003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB7117
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	cb3a5a9e-9c97-487d-af5d-08d97e8a270b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3RPlFvkCFNZMclfgKD9d4sKfv9SKt6SNqSfe78HlUj/FHF/bFngypu0ANtE1fyDkjlOUT9v0PESi3eUgUi0q0C9xUe4OHxnMLkZpcS424838v101OqmM+X2TCLoCi1N1OBCXrTL4bvrDaL3z6Zg51cI9czmsdWw8j7291o3qo6JekqnvGDpIpOJbzwBUla8qLRGIrSUlH2T+3ofGP0/3csYjWsAMwWJyTkfhxJs5wVRTVdt/Bk7ORCD3N5wf2dlHcOZAVHxDIUB+hA/C1ZvBGyZXOOaE1r254QIZ4REM+92oRdb+tgEPlPdAQxZtdYaNZGBj1jrpW4bf1OQChBvLj2yAMThL1tfaz6MffMoXewg8NeTPhjF1pQ7RgEmkvlQkw68USjv+kT23Z+rhp5jTfeuDRhAhJ2mbYmBLUT9vYB0+prhDaKoSImjNDsccvSAl4f4JCD+Z2TtRfXiQlXCfzi4JDBLmJHObCC7UAfktusIVP4QaEzDEEFMNv129xV4IPPDJ0qfOKF0wUEYqMSNysGknxjoA94PYFlJJPOtpITpMvnlj+5nObrc2uLpANedZIUMNqTskviGMEtg17uditfG84K10L/zKz5JIcG0Y8pfjeVXwoDMcd65x7D4VAqvPXQzN5h7Yr/2+FRYZcO1eiVbg+HqAlVhCE3tQj+/ZW4IJvzTl4P+1eOemsowF/7u4YRIkF5zOrN6SVXH9/eSk9g==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(186003)(36756003)(26005)(8676002)(7696005)(6666004)(426003)(82310400003)(44832011)(4326008)(336012)(83380400001)(2906002)(86362001)(508600001)(81166007)(1076003)(5660300002)(70586007)(36860700001)(2616005)(8936002)(316002)(70206006)(47076005)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:51.2448
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e70ab8b5-14b4-481b-383f-08d97e8a353f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6525

In current Xen code only implements x86 ACPI-based NUMA support.
So in Xen Kconfig system, NUMA equals to ACPI_NUMA. x86 selects
NUMA by default, and CONFIG_ACPI_NUMA is hardcode in config.h.

In a follow-up patch, we will introduce support for NUMA using
the device tree. That means we will have two NUMA implementations,
so in this patch we decouple NUMA from ACPI based NUMA in Kconfig.
Make NUMA as a common feature, that device tree based NUMA also
can select it.

Signed-off-by: Wei Chen <wei.chen@arm.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/x86/Kconfig         | 2 +-
 xen/common/Kconfig           | 3 +++
 xen/drivers/acpi/Kconfig     | 3 ++-
 xen/drivers/acpi/Makefile    | 2 +-
 xen/include/asm-x86/config.h | 1 -
 5 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 1f83518ee0..28d13b9705 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,6 +6,7 @@ config X86
 	def_bool y
 	select ACPI
 	select ACPI_LEGACY_TABLES_LOOKUP
+	select ACPI_NUMA
 	select ALTERNATIVE_CALL
 	select ARCH_SUPPORTS_INT128
 	select CORE_PARKING
@@ -25,7 +26,6 @@ config X86
 	select HAS_UBSAN
 	select HAS_VPCI if HVM
 	select NEEDS_LIBELF
-	select NUMA
 
 config ARCH_DEFCONFIG
 	string
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index db687b1785..9ebb1c239b 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -70,6 +70,9 @@ config MEM_ACCESS
 config NEEDS_LIBELF
 	bool
 
+config NUMA
+	bool
+
 config STATIC_MEMORY
 	bool "Static Allocation Support (UNSUPPORTED)" if UNSUPPORTED
 	depends on ARM
diff --git a/xen/drivers/acpi/Kconfig b/xen/drivers/acpi/Kconfig
index b64d3731fb..e3f3d8f4b1 100644
--- a/xen/drivers/acpi/Kconfig
+++ b/xen/drivers/acpi/Kconfig
@@ -5,5 +5,6 @@ config ACPI
 config ACPI_LEGACY_TABLES_LOOKUP
 	bool
 
-config NUMA
+config ACPI_NUMA
 	bool
+	select NUMA
diff --git a/xen/drivers/acpi/Makefile b/xen/drivers/acpi/Makefile
index 4f8e97228e..2fc5230253 100644
--- a/xen/drivers/acpi/Makefile
+++ b/xen/drivers/acpi/Makefile
@@ -3,7 +3,7 @@ obj-y += utilities/
 obj-$(CONFIG_X86) += apei/
 
 obj-bin-y += tables.init.o
-obj-$(CONFIG_NUMA) += numa.o
+obj-$(CONFIG_ACPI_NUMA) += numa.o
 obj-y += osl.o
 obj-$(CONFIG_HAS_CPUFREQ) += pmstat.o
 
diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h
index 883c2ef0df..9a6f0a6edf 100644
--- a/xen/include/asm-x86/config.h
+++ b/xen/include/asm-x86/config.h
@@ -31,7 +31,6 @@
 /* Intel P4 currently has largest cache line (L2 line size is 128 bytes). */
 #define CONFIG_X86_L1_CACHE_SHIFT 7
 
-#define CONFIG_ACPI_NUMA 1
 #define CONFIG_ACPI_SRAT 1
 #define CONFIG_ACPI_CSTATE 1
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193828.345356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSW-0001BC-Cq; Thu, 23 Sep 2021 12:04:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193828.345356; Thu, 23 Sep 2021 12:04:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSW-0001Aq-6J; Thu, 23 Sep 2021 12:04:12 +0000
Received: by outflank-mailman (input) for mailman id 193828;
 Thu, 23 Sep 2021 12:04:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSU-0006xV-O6
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:10 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.78]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 52d26e10-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:54 +0000 (UTC)
Received: from DB6PR0501CA0042.eurprd05.prod.outlook.com (2603:10a6:4:67::28)
 by AS8PR08MB7063.eurprd08.prod.outlook.com (2603:10a6:20b:34b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:50 +0000
Received: from DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:67:cafe::7f) by DB6PR0501CA0042.outlook.office365.com
 (2603:10a6:4:67::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:50 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT060.mail.protection.outlook.com (10.152.21.231) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:50 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 12:03:49 +0000
Received: from efa9648030a2.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1D931F74-A602-4166-9477-DCDECB6C177E.1; 
 Thu, 23 Sep 2021 12:03:39 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id efa9648030a2.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:39 +0000
Received: from AM5PR0602CA0002.eurprd06.prod.outlook.com
 (2603:10a6:203:a3::12) by AS8PR08MB6056.eurprd08.prod.outlook.com
 (2603:10a6:20b:299::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:03:37 +0000
Received: from AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:a3:cafe::2e) by AM5PR0602CA0002.outlook.office365.com
 (2603:10a6:203:a3::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:37 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT042.mail.protection.outlook.com (10.152.17.168) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:36 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:30 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52d26e10-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=R3iJbN9Z/D/BKsINPG8vU2TydxYMORgZ/IZECSVdK5M=;
 b=Ggp0q+0NQWfK5pP+hhB//iP7G7iXZPfX1DGS9RMH/wtqHgoV4DaqFVbqFH2TnHPJBXDMCqFt4X3k5RofMo49hcwz703ZrH2x8QX5QcaYXcNQSlAYvPjfhmzYCAwD4oPmxdpM/X7DkdtSlJV7OhHyGIdo9G6N809Gw8tgxPY4Xyw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 83fdd838382d77e9
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DX++omQpgcZwp+kr6tHlHTcpyAB+Vvy8DJGSREttDOQk6NE+8iBhIOiS4BowpabmLtc61cC6IuuCInIHKekfli7HziZVb9L0Mej2volYkKVUPcYK1jARPf8s6PALywczjmLshesI8q1ktBBO7xSFGvkME/jDP+qLUdpTSurUOWlHZfBmziaIyTp5DcrgwugISlenUsWM72DBBS/wWvJRccX6wYvU8OR5cV3F+FhMXAcmCgE6a1SBSwPa1dEui3lGq3KHwQMT3wdqqz5KF3SxpxKpa4kPaZJYc1IBE09KOW953LmPRN/wxFWLc2F+QcMKkpxWLRiOwpYfj4+UZnHJxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R3iJbN9Z/D/BKsINPG8vU2TydxYMORgZ/IZECSVdK5M=;
 b=oOE3geR1EzoEqzzhmxhA9fj2MJ0BhJrv8c9LbCSw2WvOdfpye2b8+pBF3rLP2uLLXBiUQaBUp5pc5wnJw5x2a06Vn4PEEUWBniB+vT1p9YN+1InL+T14zy5QPNvMTlNhxYZ7Np/e0GUhPRjDZ0TJ4XXzqDMl0p1UcjrNk00WFEym7+wjahWYPTRpIpeTPdRtmj10sD5gOPsPHDOw/BsodzYliOnJ9tM8MHP9uMSss9/OI+dm6vIBzs6ieUou8VWKcSjMSM+FXgDEWbXOUqzgY1M8cI3N8qdrseS6D2PepI/qnz8ICMAF9VRYwotajqyoeQrLysLulZjSX5Hf/HtSdw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=R3iJbN9Z/D/BKsINPG8vU2TydxYMORgZ/IZECSVdK5M=;
 b=Ggp0q+0NQWfK5pP+hhB//iP7G7iXZPfX1DGS9RMH/wtqHgoV4DaqFVbqFH2TnHPJBXDMCqFt4X3k5RofMo49hcwz703ZrH2x8QX5QcaYXcNQSlAYvPjfhmzYCAwD4oPmxdpM/X7DkdtSlJV7OhHyGIdo9G6N809Gw8tgxPY4Xyw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 10/37] xen/x86: use helpers to access/update mem_hotplug
Date: Thu, 23 Sep 2021 20:02:09 +0800
Message-ID: <20210923120236.3692135-11-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7b2db7cb-3fd0-4c98-015a-08d97e8a34a7
X-MS-TrafficTypeDiagnostic: AS8PR08MB6056:|AS8PR08MB7063:
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB70639749A0D05D07A11FF28E9EA39@AS8PR08MB7063.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1201;OLM:1201;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 vrNke+rC7Y4uRjIOePPoBbp0K10EFlfOAa1JteBra5a220WkdIVq2PDEmG1rSqpPJrhuaek0f2saxSDPVm3isW3nc7GbGWICj12hO5OQhINKl8z4w8bgq+5et0dIE6yNkuuPfo2gCsVlt34kQDgf+ninOfUI95aGZ4VHOIIPM0rsVwLfD2vtxNbnU3cL14zOg4PvmXxOC/I4vA2aX76guSRdNPORVGAP4ai2lqGkrFpu0O6gM2qy+31AsqVz0A8V8UxOKIUEhz+mCWbIiTAl/2vkaGaaIJuWzxTqIzzBEnavZZ1LC1j3SG3byBfnxQxsJ8wyOOdKCo7Q6U063J7pFgoPNEXHPzllFqkuBFc258BO2TFjDwTuh/5rMvT+wXTzS8mOX7bvgL3JXQk66ituSM/A7ZEq/RDzoFWioNUPY7U4feKuWnx8uZijgoOQes+FnVpKNHPlMM5LRy2lOJmat5ecAsni9VNqLUaDk4v0Udhrm9dg/jYpscVUGUteUga4PmKjA6Yy6myxnTngR5r3e0ueRNX0vhS3ZIJfJBM2pkFgzPYdGxxmXoKS7qJBD765NBrDhVM5yWBCPpgELLKRgPyrsGrMpezk619H71iJQ3zH+hj2pIOIiF3ktnoSb8diMiecw/KDXzvxnWTu+GvFWS6uQMBSwtxuzFL+a/YPm6YRbKc1hAe35auEK1v0PL4gHBLn7jaLDTDmnJwA+WSuASn6gNkl9gawd5g3PQ40bdI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8936002)(2616005)(4326008)(36756003)(316002)(186003)(356005)(86362001)(82310400003)(8676002)(6666004)(110136005)(426003)(336012)(44832011)(7696005)(2906002)(4744005)(508600001)(47076005)(5660300002)(36860700001)(70586007)(81166007)(26005)(70206006)(83380400001)(1076003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6056
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	db419007-5be9-4e60-97a2-08d97e8a2ca3
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ddIXjuWvIR4rJulL5ser2ASyFimea4KxEVUk3CRnfdjJGL78aSdPMGA9H/4uTm/d1nZcE3PjaCMCSTyHmWs3Q3OO0RNYMRncHvy+Mkayyz/L4g574MFHCfy5WYoKe2VagK+i0w2sZNMHCGrSJTjuTru510HV/ue1uhijIQ5LxFRSQA56QY1R5JQx0crunJYMr2EnlrjC2ZpIRBXmwB20DC9ULeguw1rPZq3aYOjhRf9s5HbxD10U7V8W2CFxFy7NPhnCUXUcNUZOxm/vjjSjAV2pfnQ0kQU5AGNYl9usyvCdliWkyHN994iL4IWzEDjIXNniXnbaFmgIZ3o4O3hyrKXnRTUjcqRue6QNimq/zxME63Tz11SGU4ix3C2CSF2fGs9GlSvFMV7qhKcUd7Xk5LcpbtjySwvxAPT7LonYBRPWfofIKuA2cqm2EP5KWO1eDRvKhaL3kxl559M5/5z/nTvkkeLY3CqfU7d0EPwnXzHAkKfX0Bpy3cf/29p9oXNJij3wk8lDQDqfom/aOY5EogumrL7euvhL3+ZrEG4gykSjDXwEmI2UBnFcxthhzU0OuNYCVoJckPKUa/1Fbjv8nnhlkMC92/fSJ21tKaBLwGeJA0gcvviMOyVnjiWUTp8pBD4TeGZvYpNlPMM5Trt92b43li2JA5kMGSj179kOrjJTAQb5Ca9cL+yarz9D29sd/v5/xmz7SO66gR+7ohxnWg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(82310400003)(1076003)(36860700001)(4744005)(508600001)(70586007)(7696005)(70206006)(426003)(316002)(5660300002)(8936002)(86362001)(110136005)(4326008)(6666004)(336012)(2616005)(81166007)(2906002)(44832011)(26005)(186003)(83380400001)(8676002)(36756003)(47076005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:50.3502
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b2db7cb-3fd0-4c98-015a-08d97e8a34a7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7063

We want to abstract code from acpi_numa_memory_affinity_init.
But mem_hotplug is coupled with x86. In this patch, we use
helpers to repace mem_hotplug direct accessing. This will
allow most code can be common.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/srat.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 2f08fa4660..3334ede7a5 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -391,8 +391,8 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 	memblk_nodeid[num_node_memblks] = node;
 	if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) {
 		__set_bit(num_node_memblks, memblk_hotplug);
-		if (end > mem_hotplug)
-			mem_hotplug = end;
+		if (end > mem_hotplug_boundary())
+			mem_hotplug_update_boundary(end);
 	}
 	num_node_memblks++;
 }
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193829.345367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSX-0001V9-OM; Thu, 23 Sep 2021 12:04:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193829.345367; Thu, 23 Sep 2021 12:04:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSX-0001Sr-Iq; Thu, 23 Sep 2021 12:04:13 +0000
Received: by outflank-mailman (input) for mailman id 193829;
 Thu, 23 Sep 2021 12:04:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSV-0005KP-VY
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:12 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::612])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d394736b-4044-490a-81d9-6cb0a001eed6;
 Thu, 23 Sep 2021 12:03:50 +0000 (UTC)
Received: from DU2PR04CA0312.eurprd04.prod.outlook.com (2603:10a6:10:2b5::17)
 by DB9PR08MB6444.eurprd08.prod.outlook.com (2603:10a6:10:23c::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Thu, 23 Sep
 2021 12:03:46 +0000
Received: from DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::79) by DU2PR04CA0312.outlook.office365.com
 (2603:10a6:10:2b5::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:46 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT049.mail.protection.outlook.com (10.152.20.191) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:46 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Thu, 23 Sep 2021 12:03:45 +0000
Received: from b4c9112bdd82.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 2B438CB3-7A90-4A1A-8800-96D502AF91C5.1; 
 Thu, 23 Sep 2021 12:03:38 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b4c9112bdd82.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:38 +0000
Received: from DB6P18901CA0018.EURP189.PROD.OUTLOOK.COM (2603:10a6:4:16::28)
 by PAXPR08MB7155.eurprd08.prod.outlook.com (2603:10a6:102:208::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:35 +0000
Received: from DB5EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:16:cafe::b) by DB6P18901CA0018.outlook.office365.com
 (2603:10a6:4:16::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:35 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT048.mail.protection.outlook.com (10.152.21.28) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:35 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:35 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:33 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d394736b-4044-490a-81d9-6cb0a001eed6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=c+6OH5SULDNhvZukoMondUWr5/a4eP4oQVaqRYS7JR4=;
 b=fbTILkyCZL7qIulNFWBwMrxcGvVjZAIMRcmTcKRFTRtReussNbGbSsGm1JMUsTgOlXkeJr6dr792trAI65njeodFr7dnUOlZLHhCbL/zlvhAe56KLOxAjFNvHoWof24CVcTLO1Xto7QDqinfBppmB81CAr6SeBlObLMtsIx4hGw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 285d174cee0d49fe
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XcZUeDkPntqMvBzx0dIDmC9IZ1qiRAyQCpQoOIWpLh8SWkSV4nZCsRKgyheWfdfHbsXiJwFi+8O1Jdy4Nrrx3kDSvDeFtgKrqCcFRcNOxxivQXoZoiFj67gO+XiGVDVescKY8n4Cm43AhJUwe9PCvMPDk8sILfPOK662gPsqDRshI60YtP4lx2FeC3jfP95HJr/XdAPtf33S17o920uP/LsaSTlqmb5TzPErOvHfTCKomL7BEQfOZyactmfAPCSClJBRYKoFqQC1D8kzfQiy2jdazvY2yVl6YKtBexPcbDk6UnRpLME2do7u4GaDyk3Sr68NSgX/xbuoG67Dv0kNxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=c+6OH5SULDNhvZukoMondUWr5/a4eP4oQVaqRYS7JR4=;
 b=iVhlRH7sIToqiLy9ZMX16nhC63sb5bbhR+N9N5xO39OnH1nOqkyv29hgGKrkxTQe8ZU4ntray/Jdpy3EAqpYK3/VwDm37zzfZquKPNDHtc9qimafG63ONF7NQgMnXTIntdOv00iG+iWTXdi3H0eCMm1gUj8sXFr9hunl741m55tnwpUeKVwG5rD7Xg4pjQ3xcFuwUd2Uf7FU7ww8FD3NEb1B2cZa/nFNEmsybndnfgApZ0HYdl1EgiKMGEjRjXs0ZR3Nb5lN/6456Pj1wkjmGDPLTX41K+kFMpbsWZ3RARDNA86fbvplXSa2WPKKEtRUXMQBrIagpGZu4dUnWw2YBg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=c+6OH5SULDNhvZukoMondUWr5/a4eP4oQVaqRYS7JR4=;
 b=fbTILkyCZL7qIulNFWBwMrxcGvVjZAIMRcmTcKRFTRtReussNbGbSsGm1JMUsTgOlXkeJr6dr792trAI65njeodFr7dnUOlZLHhCbL/zlvhAe56KLOxAjFNvHoWof24CVcTLO1Xto7QDqinfBppmB81CAr6SeBlObLMtsIx4hGw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 12/37] xen/x86: decouple nodes_cover_memory from E820 map
Date: Thu, 23 Sep 2021 20:02:11 +0800
Message-ID: <20210923120236.3692135-13-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d901cae-12cb-4f5a-b7a4-08d97e8a3225
X-MS-TrafficTypeDiagnostic: PAXPR08MB7155:|DB9PR08MB6444:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB644490D90E3E65882AD216D39EA39@DB9PR08MB6444.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4941;OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Y/8E4fLFkv2vvubolkdZlZbupIXtLOZqH8jdRUw9zou9+e8KrxHLiY4maUMwvvlDHflLfW0rfFbi+5VIGYE0qOsTTKkdewxxYTeYOTHCM3pW7CHrAC0LoYSID5TAQNlW1kzMdOIugoLzYwfH0aCFqMAnj1LmGVVg0ogvJg9jd4PBoFvlBC9CQtW6NgROthYJJbmcydFIL/b/nhUpatuOq07eUmZhr0ffqBM78D4iQNhQzuxWxnmsh6MMwAPEKiQ+n0UzzBTY4sJyi39Vmqsr4RAdfONQCbIn+1PcajPj6tIxlM6fK0cLJ7end4oYwTy7x2IJp2KgbCEQq8oBn0ygtQG1U0tO87nqztTa2QYcXiWPjmEO7MtaH/aMS1t2OMU/quN0SYNUjd7tUTclvzIPVaBCwRlJHibWjA53f1GAOdubwlPPp8zGB5WCL1LbpsXVOdWZqbUqAhHVidtIIszrsQ4Di3t5zuRoiO3j6K3CQx7ZIWd7tpzuwPezobJCQJIm4iSQ7jDN12N5VB1hMqRhHck41OcCDDiHIXGCe2VcFXim6bXCLg7pe58iIpNsnhCKrYhRErNNgT1OGixLGx8mnhrydUHMcPyHe6byHDNBvOCAMHTGq1X60LJQehL8aalN4PFsDMLCD+5c3uJvA/QIFLlgd7bM+UZm5htEBxClgCdPkMOp4n95vXOV7dusfHLeYd1Jmb0B0UQ+3IxgWjcd6SdhJxMmy7GQ/3dy2xXhnIM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(81166007)(36860700001)(356005)(8676002)(47076005)(316002)(5660300002)(8936002)(44832011)(86362001)(110136005)(6666004)(2616005)(4326008)(70206006)(336012)(426003)(82310400003)(1076003)(83380400001)(2906002)(186003)(70586007)(26005)(36756003)(7696005)(508600001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7155
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6d3c7c15-4802-472f-917c-08d97e8a2ba8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	c5Lu+AcCBJ8cKTEWK0C9eChhatlTtswOacl7oUAcJGHJtoeiHsm46SD6r20Jf5SQyEdqgzbbt43E7Fj9ZvaXc5T/pil16Pf8b4ohwUjNYO6KiKn6M1/6HGqZtQYC6b6H4Qk6vBHWkoG+8c6Uj542MgVsgKVfbyYi6JlHCZZ8TPqfeQR6/rR+K4f0lPVpURf+ljl85M6l7GtbcwM146uRWTXSGqPDoJ+fztjwzrGWo2Y0tcIRTdq8+seNdddQhbT8gQgG3wM4DazRDWNX6TaEKZHgsjOX3nqjQGrHk9xt1xWtaPzqmq8rM1jOrP6si6QEVlGe/AMQsiLRCXDBRGZgbjNHdRkcKsJRoyKAQcjV1/bh8N3yBtZ2dgPm6dsTfUM+qcjko5r/2zLDfG/0Xr4lrkQ2jiISqLeLmfkGfL0ewtIiJKoJjBNM6esvdj6UFLQRdRAqjqeZtjR5ihwXBDkPfJ7ajH7AMr4p6ZjphhPPuvNElRciTVpONiFdlpgBG078iSDYmfpKohhZKh2N9gPV97ITLxdXNub+y6LzmXCorUXRXK0FLVc0GG6tnbNmmQc5Gb5a8VtjBuQQgoQNoD2EmLMxQMMjtuLkHG2ktMPb03xPf7wwP2igDiWRcVPlDwltVSzU/NSFEiAY1eCNAzALbPXwlX5x5zT5C+OA+OtS4Ispd3GEU4udMZgFlMJjrhetvkKAR2yc8kdoIjPZ1kR5Xg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2906002)(316002)(2616005)(44832011)(508600001)(8936002)(36860700001)(70206006)(47076005)(4326008)(7696005)(6666004)(336012)(426003)(5660300002)(186003)(36756003)(26005)(8676002)(70586007)(1076003)(81166007)(83380400001)(86362001)(82310400003)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:46.1480
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d901cae-12cb-4f5a-b7a4-08d97e8a3225
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6444

We will reuse nodes_cover_memory for Arm to check its bootmem
info. So we introduce two arch helpers to get memory map's
entry number and specified entry's range:
    arch_get_memory_bank_number
    arch_get_memory_bank_range

Depends above two helpers, we make nodes_cover_memory become
architecture independent. And the only change from an x86
perspective is the additional checks:
  !start || !end

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c        | 18 ++++++++++++++++++
 xen/arch/x86/srat.c        | 11 ++++-------
 xen/include/asm-x86/numa.h |  3 +++
 3 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 6337bbdf31..6bc4ade411 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -378,6 +378,24 @@ unsigned int arch_have_default_dmazone(void)
     return ( num_online_nodes() > 1 ) ? 1 : 0;
 }
 
+uint32_t __init arch_meminfo_get_nr_bank(void)
+{
+	return e820.nr_map;
+}
+
+int __init arch_meminfo_get_ram_bank_range(uint32_t bank,
+	paddr_t *start, paddr_t *end)
+{
+	if (e820.map[bank].type != E820_RAM || !start || !end) {
+		return -1;
+	}
+
+	*start = e820.map[bank].addr;
+	*end = e820.map[bank].addr + e820.map[bank].size;
+
+	return 0;
+}
+
 static void dump_numa(unsigned char key)
 {
     s_time_t now = NOW();
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 18bc6b19bb..aa07a7e975 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -419,17 +419,14 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 static int __init nodes_cover_memory(void)
 {
 	int i;
+	uint32_t nr_banks = arch_meminfo_get_nr_bank();
 
-	for (i = 0; i < e820.nr_map; i++) {
+	for (i = 0; i < nr_banks; i++) {
 		int j, found;
 		paddr_t start, end;
 
-		if (e820.map[i].type != E820_RAM) {
+		if (arch_meminfo_get_ram_bank_range(i, &start, &end))
 			continue;
-		}
-
-		start = e820.map[i].addr;
-		end = e820.map[i].addr + e820.map[i].size;
 
 		do {
 			found = 0;
@@ -448,7 +445,7 @@ static int __init nodes_cover_memory(void)
 		} while (found && start < end);
 
 		if (start < end) {
-			printk(KERN_ERR "SRAT: No PXM for e820 range: "
+			printk(KERN_ERR "SRAT: No NODE for memory map range: "
 				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
 			return 0;
 		}
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 5772a70665..78e044a390 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -82,5 +82,8 @@ void srat_parse_regions(paddr_t addr);
 extern u8 __node_distance(nodeid_t a, nodeid_t b);
 unsigned int arch_get_dma_bitsize(void);
 unsigned int arch_have_default_dmazone(void);
+extern uint32_t arch_meminfo_get_nr_bank(void);
+extern int arch_meminfo_get_ram_bank_range(uint32_t bank,
+    paddr_t *start, paddr_t *end);
 
 #endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193831.345378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSb-0001yO-Bo; Thu, 23 Sep 2021 12:04:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193831.345378; Thu, 23 Sep 2021 12:04:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSb-0001y6-6y; Thu, 23 Sep 2021 12:04:17 +0000
Received: by outflank-mailman (input) for mailman id 193831;
 Thu, 23 Sep 2021 12:04:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSZ-0006xV-OJ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:15 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.46]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 54ef45f6-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:58 +0000 (UTC)
Received: from DB6PR0501CA0015.eurprd05.prod.outlook.com (2603:10a6:4:8f::25)
 by VI1PR0801MB1744.eurprd08.prod.outlook.com (2603:10a6:800:5c::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 12:03:53 +0000
Received: from DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:8f:cafe::ed) by DB6PR0501CA0015.outlook.office365.com
 (2603:10a6:4:8f::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:53 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT058.mail.protection.outlook.com (10.152.20.255) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:53 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Thu, 23 Sep 2021 12:03:53 +0000
Received: from b65a2952c961.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C7961B90-1680-4FBB-BC9A-94D410AC05B1.1; 
 Thu, 23 Sep 2021 12:03:42 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b65a2952c961.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:42 +0000
Received: from AM6PR05CA0028.eurprd05.prod.outlook.com (2603:10a6:20b:2e::41)
 by DBAPR08MB5846.eurprd08.prod.outlook.com (2603:10a6:10:1b0::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:03:40 +0000
Received: from AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2e:cafe::6e) by AM6PR05CA0028.outlook.office365.com
 (2603:10a6:20b:2e::41) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:40 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT055.mail.protection.outlook.com (10.152.17.214) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:40 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:37 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:36 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54ef45f6-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r4IUu2+SwohNK8zI3PdrUOxfoLsLYpgk2ynTlN2bdKg=;
 b=EjDf5zi/Z4tc5ADYbjfC0Ds++dmooCIotDopvA8DqDVXfFgCjNDmCuNbdF4pRSgGcG+ZLkMpUF5qKrKGJ9JcDibbjdUfS5k9Vu1V4c6w7DWg8ejK/8Y6nd9YZ3wazAx7UvpY9z0688EOQv4Y2J9d26xcZBFt30gdXXzbmIzUiTs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 91fc362807295dac
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MUPfU67MOThvum4MqrB4LDEr0CsJPB0GaNUHxOhO+iDY6ZoTFWLxhU30gL8OedBww41a8cjrIAdZj4/+H3hTaCl67PlWVSTtFMtD9MtTpVVgTBn8+VUjapjTERJNGvGN+nHkX/lqqwxnUiuVYfo5MfPupOI95Mm8gwyiUFQFn6Q5F+d4C2DyHSgKE22rUjavKYUqljjBgQ+9t349TqmRKr7jlAtk3EI6hqsNm7Cv9teicSfSifvNs3XgR3e/zD/LYQOJMx6m2omL7A0Y16MHYlFuzhpm32XNLmqRSoXZNuDbbuw8RHFdHN7v3UFbVYBmEfWopVTuPyAwARXHHwPDGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=r4IUu2+SwohNK8zI3PdrUOxfoLsLYpgk2ynTlN2bdKg=;
 b=fy6femylwVDfk4J9U7BlSfQguQgIKrJEN0GZ42AhquyzhmS0nNXoov35Ygl+hm2yAbOxkW7qqlW+7gcMu+QGRhnNmpVvrYmeaTXrlY4C4O2G0TR+BkdoYDMn21Bk1fonB59MZvFTrsZ2pv9AoXENAm5ZdRRty0BwA6EZXaBsqRFUjKYnRNAI8c3xcxYPvZpfgdEY7i6KdoM3Yaq8WpQRFGZK7SAS3pEPxnw08saS6ButYYufZkDnBDAggbfqIwO27bBdy2epbWKk3VqsFAiNX3zzY08X36NtabalxMLqZn6N7OvgPTItvnK3LttI99J3Xb2VxEqSpdfyl42R5hbLgQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=r4IUu2+SwohNK8zI3PdrUOxfoLsLYpgk2ynTlN2bdKg=;
 b=EjDf5zi/Z4tc5ADYbjfC0Ds++dmooCIotDopvA8DqDVXfFgCjNDmCuNbdF4pRSgGcG+ZLkMpUF5qKrKGJ9JcDibbjdUfS5k9Vu1V4c6w7DWg8ejK/8Y6nd9YZ3wazAx7UvpY9z0688EOQv4Y2J9d26xcZBFt30gdXXzbmIzUiTs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 13/37] xen/x86: decouple processor_nodes_parsed from acpi numa functions
Date: Thu, 23 Sep 2021 20:02:12 +0800
Message-ID: <20210923120236.3692135-14-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a30b010c-b72f-4bff-d840-08d97e8a3689
X-MS-TrafficTypeDiagnostic: DBAPR08MB5846:|VI1PR0801MB1744:
X-Microsoft-Antispam-PRVS:
	<VI1PR0801MB174403D036B709FD2504658E9EA39@VI1PR0801MB1744.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4125;OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 FO4jxbgM/yNpujNsJKRzt70AqOeRUqUE4m0Jcw4z8lMARTNioC6noYT9NE+jd4jCqiWg+y7Dru/pPOWlqVs+dXcDcTdfUL7xCQ3DTapMuD6a9J6/wVVkZP+XqtGF93fV3zbOsPKVygqOo4ClDgvg2sX816TRZN1ONSazwVqwrJniEKAj32Xu5jGGczwrJ4Fd1MUOa84zO1mQpjpZercexw8KO2Bjbmvg2zZIMeC3b/o/ix1aAlJAtZvKMjF70Ela3RS9VPZS8IaNNADrXsRuhQ9NGccKwMXpq3G1AW4hHTyxxEAHWO8i3cqnP1Aij5Xs/WJYloOOb6STwS7gIK3PoeJULJ4iEE2NGkSLxrumkyrzMDCR5ceGxr7aqyVDxULM5NLWGlhzBwuU1+CTGDP4/IJnEhhLhE9pTEqbWpYsr5VUDT3MdTspIAIic73NAgNnl8HIw5ILEWpuM16rTA4diijAKLTbwF2IAp33kzrziBbGQsA2IQC/4wwTMiXbafpmCBUSx7Pf4d/h3xZMnCnUuGRCU3FOm2mxlEmrG3K0hhWDMj3S12jGkLya2R/Fl3vn2Gr6uwhagUiq+7HXgnHfzERk44IR0I0zF/dSohKQor4y64gDOG2gSJhGhOpfR92yB2eIyDP3jLabW8an6h34QuWv0nJAfYKqEWQ8iq53ZEjIFpuG4YPofQqS9javgjrOqI7nMUDuFj+XIoGB42eEEw6FopIMc72dXkoGaBxe7hM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(356005)(86362001)(44832011)(83380400001)(508600001)(36756003)(6666004)(8676002)(70586007)(81166007)(2616005)(70206006)(2906002)(336012)(8936002)(110136005)(5660300002)(1076003)(186003)(4326008)(47076005)(316002)(7696005)(36860700001)(82310400003)(26005)(426003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5846
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	022efe4a-3379-42b8-8a70-08d97e8a2e87
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TGlGaLLtZs8qe1Yf7d4+NDZYGhQjlc/ISMVqswE8If63wFUGrLZRZpWoydJX0U0mdzRHIxJLhPVNZ2NegQO56nPMqQTzWRIf2rWUwk/n3rCJlaz287yhAL9ouCi7OncrvL99C4R3EWm2BlLr0Z+lRaEJx+1Pu1llGJ1I2kXr3B87X8Hiqo2YqQD0I6tQba1Fug4vvtAX0EY/v8iFRWyE3blE6mOkHTd9TnMsSZI0Mfa5H/ikE7D/6yyUdHwdewFKPQLBMo0Ds3qFLHPxmwHlgASTIpn3e47Ro61N1z/dTAsFUt7vwOyeKth4qrn5qZJq+raYgj2O0Ul1xkTSJkZ/bmYBkaZqt2oGlxEUXe6ds57aefzNL+hSb6K+4EZLEYOZJkt+cAP+8emIcUnYAA1qmhYvMn/eSUk3HzVgAirET/9X21OBhXgzCh1vVjKgB1dKtwfARS90U37Fer9KdddNH8B6AnWcuMhxR41xAEKhiiG0OgOt4sGUQtWU+DZKlbMot8gbUmDDIYAOYNBPMNwIXA5yh2VufEbjN9TkSy/egEiysaTSF1R/lL+WJBHB+6Ybv5fOrjZxqsLWRa/buOrqFYXHmxIkTF6RaCyyXkJjHvfHUQc5RL5UQy5Cduvqbg0FGGL7f6am9yh+bN/eN+vrvotV2Lc73qJYPjVhij9V5TT2Gz07exA7zUxqOoxGuU1wGav7JojRm9aNi8KjT+8Fhw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8936002)(2616005)(2906002)(36756003)(81166007)(82310400003)(7696005)(336012)(6666004)(8676002)(5660300002)(83380400001)(70586007)(70206006)(4326008)(426003)(44832011)(186003)(1076003)(26005)(508600001)(47076005)(316002)(110136005)(36860700001)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:53.5136
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a30b010c-b72f-4bff-d840-08d97e8a3689
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1744

Xen is using processor_nodes_parsed to record parsed processor nodes
from ACPI table or other firmware provided resource table. This
variable is used in ACPI numa functions directly. In follow-up
patchs, neutral NUMA code will be abstracted and move to other files.
So in this patch, we introduce numa_set_processor_nodes_parsed helper
to decouple processor_nodes_parsed from acpi numa functions.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/srat.c        | 9 +++++++--
 xen/include/asm-x86/numa.h | 1 +
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index aa07a7e975..9276a52138 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -104,6 +104,11 @@ nodeid_t setup_node(unsigned pxm)
 	return node;
 }
 
+void  __init numa_set_processor_nodes_parsed(nodeid_t node)
+{
+	node_set(node, processor_nodes_parsed);
+}
+
 bool __init numa_memblks_available(void)
 {
 	if (num_node_memblks < NR_NODE_MEMBLKS)
@@ -236,7 +241,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 
 	apicid_to_node[pa->apic_id] = node;
-	node_set(node, processor_nodes_parsed);
+	numa_set_processor_nodes_parsed(node);
 	acpi_numa = 1;
 
 	if (opt_acpi_verbose)
@@ -271,7 +276,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
 		return;
 	}
 	apicid_to_node[pa->apic_id] = node;
-	node_set(node, processor_nodes_parsed);
+	numa_set_processor_nodes_parsed(node);
 	acpi_numa = 1;
 
 	if (opt_acpi_verbose)
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 78e044a390..295f875a51 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -77,6 +77,7 @@ extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
 extern bool numa_memblks_available(void);
 extern int numa_update_node_memblks(nodeid_t node,
 		paddr_t start, paddr_t size, bool hotplug);
+extern void numa_set_processor_nodes_parsed(nodeid_t node);
 
 void srat_parse_regions(paddr_t addr);
 extern u8 __node_distance(nodeid_t a, nodeid_t b);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:04:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:04:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193834.345389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSg-0002hO-Oe; Thu, 23 Sep 2021 12:04:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193834.345389; Thu, 23 Sep 2021 12:04:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNSg-0002h9-Jx; Thu, 23 Sep 2021 12:04:22 +0000
Received: by outflank-mailman (input) for mailman id 193834;
 Thu, 23 Sep 2021 12:04:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSe-0006xV-OV
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:20 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5583a53f-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:03:59 +0000 (UTC)
Received: from AS9PR06CA0065.eurprd06.prod.outlook.com (2603:10a6:20b:464::30)
 by AM0PR08MB3476.eurprd08.prod.outlook.com (2603:10a6:208:dd::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:54 +0000
Received: from VE1EUR03FT038.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:464:cafe::83) by AS9PR06CA0065.outlook.office365.com
 (2603:10a6:20b:464::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:54 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT038.mail.protection.outlook.com (10.152.19.112) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:54 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Thu, 23 Sep 2021 12:03:53 +0000
Received: from 12313445366e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3FCA74CE-884C-4689-80D6-F74825E26883.1; 
 Thu, 23 Sep 2021 12:03:45 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 12313445366e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:45 +0000
Received: from AM6PR05CA0031.eurprd05.prod.outlook.com (2603:10a6:20b:2e::44)
 by DB9PR08MB6889.eurprd08.prod.outlook.com (2603:10a6:10:2a1::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:03:44 +0000
Received: from AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2e:cafe::63) by AM6PR05CA0031.outlook.office365.com
 (2603:10a6:20b:2e::44) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:44 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT055.mail.protection.outlook.com (10.152.17.214) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:43 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:41 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5583a53f-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bHB3OIRxraBI99dHCnbQBTwh5FbIOCKQZXZ1E5VSHqs=;
 b=iAT2nzsxmP6X58+jMrjioleryYdFbsMP9rbS0X+6K6yjYBs6CYuCOTbjjzUjqwbojD/1CMlU2d3IK9KwwlPHvK9XKglsra7zkiRT93h4vPKmA6N2vWnJch0QZrOhShYS2I/Yqb98WNFQWiISx6MXyFNX0Nguvj0ZpypfuO+AS/I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d9f2c0d4b272f3ef
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f73aX07EY08DBRiQt9WhG5PBNrclcaCHk/aG6jHqvhqXEetNDVNVTRL2uQuzz0AJUuCD6djTC6Xi60SLMB4nMTbMDEi5I+5RO5VRi9c5486znTb4cC0F955FeNqXWpDD7wp2BrSYZbtn2x2HArHnnw7CM/GF65Zb18ta9sgWifDmvzh0el+YEyogFLmsmh0KPirTQbiZJFzZcaT0RkFdSIiydJljLQs9Nq7oOsj7hS7Qts9YvOSnuLYDC+DUL0AQeN9Ln9p2HlhyndwweFlIAKKviditU21WsGPCdx8Uxa/L0Km105EDL+5oA5IIMuigy3y+J1HIEYwkEkeTsIiY8A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bHB3OIRxraBI99dHCnbQBTwh5FbIOCKQZXZ1E5VSHqs=;
 b=IXr8/KOhipoXLnD4GOlPYbDCPIbbPodevluYN5i1me4KxYE432PlUmFEPL/E/sl+oYgfbTS2XshPdV6530M0MSJo32IWxiWBvcqtIltjwtCQ32L5oMXI7ee6miMXKU46t3P08e8+wdh5pt0lWQpkKtxbLaCv1WAepwrqyFdwSs2/72ZnvR+WpT1XRxYfWQ1km+Wjp0xRLCKxJY28kMiMyfoZU1A6ynRZ1eYZWTcHHo3LbhKXM4velYss30L19+kX1eaH2rbkiTCL8cLZvd83R1gCndgNcC4cG9Y4+NgjNGH76FQTU2jUB/dvsoBETrlWXF3oHnpsutasIR29G9FG8A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bHB3OIRxraBI99dHCnbQBTwh5FbIOCKQZXZ1E5VSHqs=;
 b=iAT2nzsxmP6X58+jMrjioleryYdFbsMP9rbS0X+6K6yjYBs6CYuCOTbjjzUjqwbojD/1CMlU2d3IK9KwwlPHvK9XKglsra7zkiRT93h4vPKmA6N2vWnJch0QZrOhShYS2I/Yqb98WNFQWiISx6MXyFNX0Nguvj0ZpypfuO+AS/I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 15/37] xen/x86: rename acpi_scan_nodes to numa_scan_nodes
Date: Thu, 23 Sep 2021 20:02:14 +0800
Message-ID: <20210923120236.3692135-16-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 197158ef-8198-40e7-3780-08d97e8a36f1
X-MS-TrafficTypeDiagnostic: DB9PR08MB6889:|AM0PR08MB3476:
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB34761ED70BCA144E7A83D3939EA39@AM0PR08MB3476.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NzGNuz8Y1xXknlT+8r0RnU1UtHo1PdP5WGHYvhOEcKq0uhsl8JNQyOldKxHp+eoLISCB1G/Xt/UpyM2sQ7ro5/fFad6wdtOEnUCPT7hYrBfdp+I7Q22oss1ZcoNIO8yqMjC4EPwoQNvFJfHiYRoFmtPEMK/r2JfE1rmmwvAOiDSZbbi82xJ533wBDIedOo166741Cltr1JFdiIo3UPJor1Ttbfpwlymsx8VG67Rj9+KZXp9OqO3B/pKBgCpm7idGISchC7tSMa+6vYvBGkMfIPDEdbKHzgK7g5/rHDKzC+L8Ro6TPDx7YeIDTz3gupE8B929Yijm5uT8nWa+wtMH2ExU4deroiesooYLGjbSV5bfQYy6GJTi3TO69zjuU2Dk9bTKaFVu5dQTvHgsKaNRryE6QbjoADyBHudJF5eIzW+JWW1wtTPcJLamC12NAyfgMP4nkh93wd8PTeL1UvFAqq4oLZAzfkUri6bY+GuYEQWf+pIQJlLDz/bGV3RV0pcl60fJrNWFSSUr7E2MhYjk1cXZ9QNPjK8uxW/31vgv/h6FD7uA1ZmELdgxTnu2TGOWmkPHl+oWWaDuVRUkaG4OVYNiurngQE0O3Em/I+Kmcbsqu5qCB3kSqCPnVL51M+ycem8Jh8BPtAzSXI3hrQ7AD55iDpnJFuG3VWSmQ1oHCHFwSGiAoeVhPrN8VFaygbA7YsanGGFpJSsA1qQgog5UZRSYBErF11FdXSk/Mxbhy0Q=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(8936002)(110136005)(1076003)(186003)(2906002)(336012)(82310400003)(426003)(26005)(47076005)(4326008)(316002)(7696005)(36860700001)(508600001)(36756003)(6666004)(356005)(44832011)(83380400001)(86362001)(81166007)(2616005)(70206006)(70586007)(8676002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6889
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8faa9ec5-04b5-4c8e-65bc-08d97e8a30dd
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Cp3kYtAT3YABE2yJbPpKSixL52n4S1hRk5RWPzDmW+rPmpm4B7uTRM3kYvs0oTDlyZUPXG3BPQ5zG9iUuxIc9kASOqfpktLHfmK/AGfypHVVn8Z7s53KfPFWkGsEMImvG9KLCc6BkUzo1CxNSW+QgZwDMeeRBZRt6EpSkDmwzc3Nxsjsj7Pw8oCVcbx4TjH4X7YEpCA2Dv++mZSPzNtNpAWeHmSWno5InQDjdLQTLQOeg6BodnDFRkeNSCsGB5tukuzJiaB+F5JJqLLLsRL07UGroAKOgjXOAGBpbqkOP7I13XmFPT7ws4OflLE6hc55kx29H1CdgN6eidrLVKktdkt9ITjccZwqkJiIFZa6aZV5BYPpLo+rnTzP3cnaykSxGGmezdMEPR/x6hPw2jEgZ1X1mSASdUX8A3oqpYUC/vR1K50Ez76/y9fdt7uMmG4opadfUmpz57daz/NSVGgtyDCZ7Dt4EpkSZhnXjep8aQjxuBWQh7YCZLBxsp1Vb3oOEqsdzxp+RKR7ZxLYv1OjyuCSyHUODeI5toW/AmbhVuD1cvcIJcLSdGvxOUf9QG8BMtfcUK6P3er9PCBS+ZIDyIY3ZXmdboBUPzUltFmczbUWijPDiOR9HJ5xl1ieg2OXk9Pw82RfFgy1709DkYsJN30xMVnVNJIOjABGAWVlF6Z8WR1/CMVQQEK0r0Jjyhd/3dBxXPXT+oiPrV4TLMIjPQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(83380400001)(36756003)(70206006)(6666004)(426003)(82310400003)(7696005)(81166007)(8676002)(2906002)(1076003)(508600001)(44832011)(86362001)(36860700001)(4326008)(26005)(47076005)(336012)(2616005)(5660300002)(316002)(110136005)(186003)(70586007)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:54.0941
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 197158ef-8198-40e7-3780-08d97e8a36f1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3476

The most code in acpi_scan_nodes can be reused by other NUMA
implementation. Rename acpi_scan_nodes to a more generic name
numa_scan_nodes, and replace BIOS to Firmware in print message,
as BIOS is x86 specific name.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c        | 2 +-
 xen/arch/x86/srat.c        | 4 ++--
 xen/include/asm-x86/acpi.h | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 2ef385ae3f..8a4710df39 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -261,7 +261,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
     end = pfn_to_paddr(end_pfn);
 
 #ifdef CONFIG_ACPI_NUMA
-    if ( !numa_off && !acpi_scan_nodes(start, end) )
+    if ( !numa_off && !numa_scan_nodes(start, end) )
         return;
 #endif
 
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 4921830f94..0b8b0b0c95 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -512,7 +512,7 @@ void __init srat_parse_regions(paddr_t addr)
 }
 
 /* Use the information discovered above to actually set up the nodes. */
-int __init acpi_scan_nodes(paddr_t start, paddr_t end)
+int __init numa_scan_nodes(paddr_t start, paddr_t end)
 {
 	int i;
 	nodemask_t all_nodes_parsed;
@@ -547,7 +547,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end)
 		paddr_t size = nodes[i].end - nodes[i].start;
 		if ( size == 0 )
 			printk(KERN_WARNING "SRAT: Node %u has no memory. "
-			       "BIOS Bug or mis-configured hardware?\n", i);
+			       "Firmware Bug or mis-configured hardware?\n", i);
 
 		setup_node_bootmem(i, nodes[i].start, nodes[i].end);
 	}
diff --git a/xen/include/asm-x86/acpi.h b/xen/include/asm-x86/acpi.h
index 83be71fec3..2add971072 100644
--- a/xen/include/asm-x86/acpi.h
+++ b/xen/include/asm-x86/acpi.h
@@ -102,7 +102,7 @@ extern unsigned long acpi_wakeup_address;
 #define ARCH_HAS_POWER_INIT	1
 
 extern s8 fw_numa;
-extern int acpi_scan_nodes(u64 start, u64 end);
+extern int numa_scan_nodes(u64 start, u64 end);
 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
 
 extern struct acpi_sleep_info acpi_sinfo;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193844.345400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVI-0004wz-Ao; Thu, 23 Sep 2021 12:07:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193844.345400; Thu, 23 Sep 2021 12:07:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVI-0004ws-65; Thu, 23 Sep 2021 12:07:04 +0000
Received: by outflank-mailman (input) for mailman id 193844;
 Thu, 23 Sep 2021 12:07:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNV9-0006xV-TT
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:55 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 730b2713-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:50 +0000 (UTC)
Received: from DB6PR0301CA0054.eurprd03.prod.outlook.com (2603:10a6:4:54::22)
 by AM6PR08MB4933.eurprd08.prod.outlook.com (2603:10a6:20b:e9::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:48 +0000
Received: from DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:54:cafe::53) by DB6PR0301CA0054.outlook.office365.com
 (2603:10a6:4:54::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:48 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT062.mail.protection.outlook.com (10.152.20.197) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:48 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 12:04:48 +0000
Received: from 4960b0822148.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 71862B50-7BB2-490E-89DE-0DCDD0B4EB35.1; 
 Thu, 23 Sep 2021 12:04:36 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4960b0822148.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:36 +0000
Received: from AM5PR0201CA0013.eurprd02.prod.outlook.com
 (2603:10a6:203:3d::23) by AM9PR08MB6981.eurprd08.prod.outlook.com
 (2603:10a6:20b:414::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:35 +0000
Received: from VE1EUR03FT047.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:3d:cafe::65) by AM5PR0201CA0013.outlook.office365.com
 (2603:10a6:203:3d::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:35 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT047.mail.protection.outlook.com (10.152.19.218) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:35 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:27 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:04:25 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 730b2713-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OBYQYm/pE+7ohJdCVC0uw11MJGdKCfmbO+hBvil1Eag=;
 b=rpVqVyZuBO29kZXODsd/4sEt5FNEPeNP7JyqAfWjH5etVBcQEETNMAU8HD/RYZpGY7wvhw3YlQoR/kaSucJvBdg0hyFGwzL3piATEu3TaZD8h/N8jCxiDip4GiJtDiltOuIjVZ5imUnlZTve9rS/w3BCsgKP69z9wBhiiLTF664=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 12b7cd9313eb12ab
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bxmzv4vZ3Wo/M+nyvaghH+LMqBD6Spae6W87H77piFdNmP8ojXeERFHHBbdEmkh2/IWi9W5744C70g0YwzMme+YjsSIUdzm9fQj0McB4Hy0Ob9nK3vkGqxpOP1s0rFKAzCKeZ0zuqIpILL/elNVaMEDpTrZ6vsjLtl+5C1/Hr+Gdd/xZqiW/GGl/9mB2DhRG2/ziJhIi275xbQpuDCZ1hz7B7OsPr+gQtpq44HJfhQUPy/Jj7fz4AWl1MloPrwwFBQbzG1sv9ajPmeTulZ2CZjDyiB6E+g2lX66fI0HKQq/YQ+6p1LrKuh22zvFo99od1A8ZhgfDxUZ7dxjP7V1ghw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OBYQYm/pE+7ohJdCVC0uw11MJGdKCfmbO+hBvil1Eag=;
 b=LhrpmwfiJ3xmDxN/rpfowx0FJexczaBxPSBA48XHWxa+YxWk3mR4C/+3Agj5+pSLZgU2xqQZhfvSW97sy46X+XHjwqEpFF3HLgHdrceiiDIHZLgYT1iB6DssrKHa1yheSSLHGcle+3fVEE7F/3YnXKaCDaa117QVFGeJ5kRzp9ww8IS63igxp9lIu96rtlhHblTjTuTaboqvcrZYAj8cqA9J8cIa5apuFadnrEJdZBJEqytOhyiEko/8/w1l/TtWPTdvtmVpUbv7mTwy+nxyV8QxwRQBwGKfoa6JhTE9FotS2OzKNln3wRHB0eHAFr8JXPtCm2weYT91Gt6LMpATYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OBYQYm/pE+7ohJdCVC0uw11MJGdKCfmbO+hBvil1Eag=;
 b=rpVqVyZuBO29kZXODsd/4sEt5FNEPeNP7JyqAfWjH5etVBcQEETNMAU8HD/RYZpGY7wvhw3YlQoR/kaSucJvBdg0hyFGwzL3piATEu3TaZD8h/N8jCxiDip4GiJtDiltOuIjVZ5imUnlZTve9rS/w3BCsgKP69z9wBhiiLTF664=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 35/37] xen/arm: use CONFIG_NUMA to gate node_online_map in smpboot
Date: Thu, 23 Sep 2021 20:02:34 +0800
Message-ID: <20210923120236.3692135-36-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f3aa601d-397a-4a47-32d0-08d97e8a5747
X-MS-TrafficTypeDiagnostic: AM9PR08MB6981:|AM6PR08MB4933:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4933DA4231B8E5FE9594D7D09EA39@AM6PR08MB4933.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3044;OLM:3044;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 XILAatikNKG7Jx3du5S+YJSwp8CjhV3EaCa8GKFjUis5U37Wrt/wISUM9pRQoQEntVvWtb1EMl8tlOvCoFQUx66cj9BJriKt4zlOUInwg8ETOyraEZISxUAVbtP47X26V+9qpjcbSYD8OOi14Oz9aOr87SbV3MP+iQCoRUodlZN6K78UiXAiGsTE7Uv/BFZiPbnWNLBdapL99m3gibErOcLAjm4kF1CIefnbnaU1TUC7t53dR+/GFlqb+r+rSU0V7SjZD1x0EEAurYyXDGtwo4eLQhLll7FsDyuwhmtZkwjBzG4CkSQD90y1IisIeA0dMbe+A4C5ZdaufBjcK10MEMjM1c3WDFeO4MinL3ROiatkzDlhA8yfV+dshiOV61Uu5fIjIVeaMwttc5sZUiAkVfVeNYaptdauD9TGToM8ZYHthHwu1zppjMhj88SeiqUtmqPbG7yeF5DEP9QognPk2T4j1uxYsZY5KLbf1s3ixyx5EfukPK7VA7bsaEAFfqyfzD047URW4XSm3+dvvbQOeoj0G2sdx017bx5lNdRChL4jOkv+JpbqIXU9vR7Abd+v3+LNH3ERbQs+RosxE11gTyFUEmdaCSA7YgvxKYx4cVcqfrqtycl9S5zoIBSIMSrJ+3GyRgxPL8PE+7zHnl+M+B/VqEMyeR1rSKYymrko8E1VZIvsuT6YUjmr7+nEmf/y2w6FlRre+kATbb95TFgzsLlOhlFGEFQIrhUR6lzPqDU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(110136005)(316002)(7696005)(81166007)(70206006)(70586007)(47076005)(36860700001)(82310400003)(8936002)(5660300002)(356005)(4744005)(186003)(2616005)(83380400001)(426003)(8676002)(26005)(4326008)(508600001)(336012)(44832011)(2906002)(6666004)(1076003)(36756003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6981
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c7dfaaf2-46fe-46f3-cbc9-08d97e8a4f94
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yRwMWa8HdFxAD+5Ms7cLrYXBpn2By4Eqa269XVT96x5UL8Yv9TuUNIIc6l+Iq3MrHkJJzyqH3cdwDv40rw8rh8ceQGo1Rvo+HU2/xcTHIxqavTCr8RP1OKx9APjrqMMkz76P0MY8eqDjhwlwfSZ4tYBCMQ/sxRznHElFJpiC+9kJoFnQcAc0gzVGcJpnlG9TURI45LG74w3LTbSOnna0EwJtsTKq7r6k4cLxW4jVHtsNHs4Rh8Hgr3K2SdgzniyMkR/4wKHUKW2UlrmtSEdXLWrK5enNRqauVYtJpKZE873vFhBmGBBVMpR6WSvtLJ83ym1c+4BQKzAAd5KU0s0CVVgzxsI7+yvigigxMlZVrZIUN32zTtWJlGPT2Lh8p6zIDHc3F2YiwgX+lxe3oLyP+ZpU1oCrVjFA5LCmovbmfkQKnwOoXb73BgF79MJgkfwYpvZoDd2ilQbGarZz7MKqDX+AcrpPjGn9pDewzu8Q2iCkHUH7IY5/KYEkv2UgSNaYiOCSRwXzH27GtrH/q/SUia3tToX52JNyzhfLt5OXH+WuiThfpl9zmGmShd9aEevNN5sqwgL8d3B54M+JDTQ1nEcHuYfyfYRZ1IHw9dd/kIKRzHLGsc+1CBvNa67gzp2mxjqh2Se0Fcq5WgB2CAofItuKaZR5j75CrIm44L0JnOjmuQ4pms4844l4tOW2DXoaDi/NFq2Mmqu1b79XQam0Fw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(81166007)(83380400001)(508600001)(8936002)(2616005)(5660300002)(70206006)(336012)(47076005)(36860700001)(70586007)(426003)(8676002)(110136005)(86362001)(4744005)(6666004)(44832011)(2906002)(36756003)(26005)(4326008)(186003)(316002)(82310400003)(7696005)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:48.4426
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f3aa601d-397a-4a47-32d0-08d97e8a5747
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4933

node_online_map in smpboot still need for Arm when NUMA is turn
off by Kconfig.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/smpboot.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 6e3cc8d3cc..216c8144b4 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -46,8 +46,10 @@ struct cpuinfo_arm cpu_data[NR_CPUS];
 /* CPU logical map: map xen cpuid to an MPIDR */
 register_t __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = MPIDR_INVALID };
 
+#ifndef CONFIG_NUMA
 /* Fake one node for now. See also include/asm-arm/numa.h */
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+#endif
 
 /* Xen stack for bringing up the first CPU. */
 static unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193849.345411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVP-0005Hh-LK; Thu, 23 Sep 2021 12:07:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193849.345411; Thu, 23 Sep 2021 12:07:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVP-0005HV-Hy; Thu, 23 Sep 2021 12:07:11 +0000
Received: by outflank-mailman (input) for mailman id 193849;
 Thu, 23 Sep 2021 12:07:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSv-0005KP-0E
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:37 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.5.64]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 38991186-efbb-4ccb-9f64-24a4c1aa4c7b;
 Thu, 23 Sep 2021 12:03:54 +0000 (UTC)
Received: from AM6PR04CA0044.eurprd04.prod.outlook.com (2603:10a6:20b:f0::21)
 by VI1PR0801MB1727.eurprd08.prod.outlook.com (2603:10a6:800:5a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:51 +0000
Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:f0:cafe::9c) by AM6PR04CA0044.outlook.office365.com
 (2603:10a6:20b:f0::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:51 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:51 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 12:03:49 +0000
Received: from 9cf742a5989a.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A8CAD156-9C5C-46D6-864B-12E3A182BFB0.1; 
 Thu, 23 Sep 2021 12:03:35 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9cf742a5989a.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:35 +0000
Received: from AM5PR0602CA0024.eurprd06.prod.outlook.com
 (2603:10a6:203:a3::34) by DBBPR08MB6251.eurprd08.prod.outlook.com
 (2603:10a6:10:208::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:32 +0000
Received: from AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:a3:cafe::72) by AM5PR0602CA0024.outlook.office365.com
 (2603:10a6:203:a3::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:32 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT042.mail.protection.outlook.com (10.152.17.168) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:32 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:26 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:25 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38991186-efbb-4ccb-9f64-24a4c1aa4c7b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qq6wTyOtVX+3FOXJpOSjlfoDJNYhe/IRFtzFw5fVKj4=;
 b=IPXnMSf4VYMZpzXVA9fCiTHr2TseQttPgKx36J3MFvKVJgRl8XhW6BQ7fCI4Jx1uozmN8DRFhGErfTZrp1xd08AZX6+HkUNnMlucr959QsJIIIItcShfpqFZQnjnvLmm3sUBkXyuJxIPcXidS5Votg/FO3xlBA3c08I2HQJwOtA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 075236bcbe98196f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AIo3EqQhsCeV/fCeHZB6X314PnTI8tW7HR+6YpKLwuZGwSrFxWj1bjfJvdo98x05icNgXHF4WUW+heqCMTxpOZEwxSN9WCdIbEg/G5raUMp292pHZbAbuiA+hHQr8syAARNYJzALEesc2xkQIxtxeXihVw791GeLZGx5TTbU5J5Elu3SQoUThIVCW7FZLrzZn1/4DJ8vWbIgPqJQ9hV31HCz2g7hmoyS9Ws3wrSCagPry7/p5xRh13xgWsg/BDm0AyrwP6dbGQdPb3x2azqMuhPbOmWY/OizICFXPIvRygaXslqi53eUzTSZi6AV7StOpyAyz3G8ckJ9Ui4aDJJ6ig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Qq6wTyOtVX+3FOXJpOSjlfoDJNYhe/IRFtzFw5fVKj4=;
 b=OK0lF41h6aqkbdbqVCbk7GySxh5KaybLYwM7qpM6r9Vzo+mUQEw2/lO23UhR7YOavfAl2DFA8GSl71v5OL/y8qSDnbmvA2M1x++f6XnF0xy/J/cZyDyR3kiGibQz4yv6vIRoCPWWGCMORndSJw8P8qPJeH7DGcVb7en1oKF+zTJ9dr4Lv+XAOIeXeNJbK+VdDQz0uk/h79voyF4W5aOKQ49QQNkkLup9P+1lANHWkzP9fkirnjEgPFlR0JdsqpyYOR7atpSCnnq5GUl7QBfRo2yIDfZuJ++BshFQs9b18eAcO8/V6E+MfXzXKjvrnFvdflG8TJ0SepVHXFVo7xuZVg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qq6wTyOtVX+3FOXJpOSjlfoDJNYhe/IRFtzFw5fVKj4=;
 b=IPXnMSf4VYMZpzXVA9fCiTHr2TseQttPgKx36J3MFvKVJgRl8XhW6BQ7fCI4Jx1uozmN8DRFhGErfTZrp1xd08AZX6+HkUNnMlucr959QsJIIIItcShfpqFZQnjnvLmm3sUBkXyuJxIPcXidS5Votg/FO3xlBA3c08I2HQJwOtA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 08/37] xen/x86: add detection of discontinous node memory range
Date: Thu, 23 Sep 2021 20:02:07 +0800
Message-ID: <20210923120236.3692135-9-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b31a1b0e-3918-4675-0fd5-08d97e8a3521
X-MS-TrafficTypeDiagnostic: DBBPR08MB6251:|VI1PR0801MB1727:
X-Microsoft-Antispam-PRVS:
	<VI1PR0801MB172740517A9ADADE655951F99EA39@VI1PR0801MB1727.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 uy7r7emIWxUurtOpRer+cKjJzZDL7auIjL+c2A4QEVJS6VpcoLTrqdDyNVc8LHZwnP0Oi8X7fXg0dano/SodwiZMRJeXjPha0aM+9lzyxK9tAPpTEHdKZ9SdjEA29fPYkzYrolmQ8+AyslPF9aQa2T2NYw00hLnD6GzjW9KpFYuG6dPW0aWieMZgBrteWAEanZQsiqUgWIsn53Ddvn/4jEZIeXlvze4Wga8nDaUYrxyw7cHBIGP/ajdVKl1tn2QeCASd+wbTAJTi90WyzT2YtXKr8KbV6UO69YIJxkZ+MG5IiGaeYeKMG+90iBUf/PJnZ/nqr/OpImOX8aHFcqTm/68K4F0WkLhbNkQryynr0aTwcA0UNrAWf48gWHyixG6+y3weD/yB4Wr1RCNcINkZlMzQKEgQ4qyArj0SAYF3L2vTWxOAwcf8XuBniDULFmZjLiZENeMiVKF3owQSU8ZR4WfoJgkQNN0ClHWGI9+/4PQ6Bo7wW3N7CGJKoeGtqWGIawQO6LcA43opLNl6V5YJgRzBpnohgqiHtV10kO/6fxaXuOtTRUpYT+AA4J8IBmh/tBV9wn2/WrWZgejlcJBCZpodiVAZDxvh65J35dFZ62fzwKnn8TE1IXOImrDXtbaWIeDk4BcSqzdaRz0XrJ19jFIpJNTXJwqH0EatTWEJxHuMmWLFdIxs84yxCNWZgjf7+uLeSbnd1LE1HEv/GcZ2EDgGSO4EvQQx31ZAwj9G4/Y=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(81166007)(36860700001)(356005)(8676002)(47076005)(316002)(5660300002)(8936002)(44832011)(86362001)(110136005)(6666004)(2616005)(4326008)(70206006)(336012)(426003)(82310400003)(1076003)(83380400001)(2906002)(186003)(70586007)(26005)(36756003)(7696005)(508600001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6251
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d5d596cd-dc95-4c7b-ab3e-08d97e8a29fb
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9UMxvly7c9LfuHITUmQhy3K2KZ1efgz8BFrzECO+BYdSUItM6dNRhG8qnNu/pPfzT33ecGHokezDjFKx6kOhb+aSaO5b4EvIXoJ+iUFH8PXBk7t4zEBcQ0x3B7yBJCMRnIlJApjyN6Q2lRhUoe+jVwXesR14SNnifVeOt7wpTEEpw9UfpNMOfq+ALU/SPKkiT8t50eRIH1sgIFYDN/D5WIDNcRgNPQGAXGKRVRpqZz3tfrHuftiQGWgXAndAXi12d29ULJt7GMSxUeW8ZVjlkYi2jTJ8fPXAfw+sbEoz4QgsRTLw7PnXqzP1hSMrFaGBS7txeSPFnZTStY5etnjhwzBtcuGAAmLtId918cfsOWlqi5Vm562X3xunqPNXl6XUHWupVIX8fTU+NrSEN091Tm/fS4qt9qPl8EqcW38gbtIHQtagprPN1OkuIltr98CELnCStcfZDZ5J9UM4DWZTqaMt0cudTOxGofnY/qiAopZZ6FTUY+VIlhwiteB6bTT3fkWffSBkH2jYJA5qaf/3NvTeNBlbe3eaFc4QrCgxOjTqo5ek324JDnbe8FiS/Py5kVIhJvmXkhSTPeA2gDzWx3IrADI7mJ/+D9zpJxGhzY3u1UNrAM5Ef3Lna5lD0MV5OzGLqFa6w/L3dEse6iBRh8MHvrJ/XisHdrrBmAarwXk/8KgtOKdzlcsHUJsx0zHGpVrN+vC1SdUyFr1gIKYqwA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8936002)(186003)(86362001)(4326008)(508600001)(83380400001)(70586007)(426003)(82310400003)(2906002)(26005)(81166007)(7696005)(70206006)(47076005)(6666004)(44832011)(36756003)(36860700001)(316002)(1076003)(2616005)(8676002)(5660300002)(336012)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:51.0522
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b31a1b0e-3918-4675-0fd5-08d97e8a3521
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1727

One NUMA node may contain several memory blocks. In current Xen
code, Xen will maintain a node memory range for each node to cover
all its memory blocks. But here comes the problem, in the gap of
one node's two memory blocks, if there are some memory blocks don't
belong to this node (remote memory blocks). This node's memory range
will be expanded to cover these remote memory blocks.

One node's memory range contains othe nodes' memory, this is obviously
not very reasonable. This means current NUMA code only can support
node has continous memory blocks. However, on a physical machine, the
addresses of multiple nodes can be interleaved.

So in this patch, we add code to detect discontinous memory blocks
for one node. NUMA initializtion will be failed and error messages
will be printed when Xen detect such hardware configuration.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/srat.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 7d20d7f222..2f08fa4660 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -271,6 +271,36 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
 		       pxm, pa->apic_id, node);
 }
 
+/*
+ * Check to see if there are other nodes within this node's range.
+ * We just need to check full contains situation. Because overlaps
+ * have been checked before by conflicting_memblks.
+ */
+static bool __init is_node_memory_continuous(nodeid_t nid,
+    paddr_t start, paddr_t end)
+{
+	nodeid_t i;
+
+	struct node *nd = &nodes[nid];
+	for_each_node_mask(i, memory_nodes_parsed)
+	{
+		/* Skip itself */
+		if (i == nid)
+			continue;
+
+		nd = &nodes[i];
+		if (start < nd->start && nd->end < end)
+		{
+			printk(KERN_ERR
+			       "NODE %u: (%"PRIpaddr"-%"PRIpaddr") intertwine with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
+			       nid, start, end, i, nd->start, nd->end);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 void __init
 acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
@@ -344,6 +374,12 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 				nd->start = start;
 			if (nd->end < end)
 				nd->end = end;
+
+			/* Check whether this range contains memory for other nodes */
+			if (!is_node_memory_continuous(node, nd->start, nd->end)) {
+				bad_srat();
+				return;
+			}
 		}
 	}
 	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193865.345422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVi-00064C-V2; Thu, 23 Sep 2021 12:07:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193865.345422; Thu, 23 Sep 2021 12:07:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVi-000645-Rr; Thu, 23 Sep 2021 12:07:30 +0000
Received: by outflank-mailman (input) for mailman id 193865;
 Thu, 23 Sep 2021 12:07:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNUQ-0006xV-Sl
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:10 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.67]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6adcd31a-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:35 +0000 (UTC)
Received: from DB6PR0601CA0027.eurprd06.prod.outlook.com (2603:10a6:4:17::13)
 by DBBPR08MB4252.eurprd08.prod.outlook.com (2603:10a6:10:c2::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:32 +0000
Received: from DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:17:cafe::25) by DB6PR0601CA0027.outlook.office365.com
 (2603:10a6:4:17::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:32 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT011.mail.protection.outlook.com (10.152.20.95) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:32 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 12:04:31 +0000
Received: from e65c92c8e861.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B81359B3-0BE5-450D-A9D8-FE51EB833D47.1; 
 Thu, 23 Sep 2021 12:04:19 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e65c92c8e861.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:19 +0000
Received: from AS9PR06CA0105.eurprd06.prod.outlook.com (2603:10a6:20b:465::27)
 by AM4PR0802MB2177.eurprd08.prod.outlook.com (2603:10a6:200:5d::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:18 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::5a) by AS9PR06CA0105.outlook.office365.com
 (2603:10a6:20b:465::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:18 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:18 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:06 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6adcd31a-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=24TzNeXkluknvjlUOjwTTRzo7llaAUU14RNeMwlWz6E=;
 b=Rle0oijAtTyX8gF6xwDUp77hGrbcGNjm/dLMTBs3Vd9UIlItm6OAyJOaNjNsoG7iZLv/R+BcLsK4uYGCbP5H75MAfR/LTR/UYBeN71RO4+xybFX5VVSVyoJpCUw/mwZuzaaUfp0gDp4flMzjWjiTgJUsRMSkXjgfiFQeMrYqk+I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a61ea4807183f0ca
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PwTcW4HxqTP+6Iz2oF+74jfTpOdpTyplmQtI0EBsyQV8Cj7eKIdNPkiQB+mMocXm95eGMcBKeLfZed+DWg+ESj0hhOVTUai92FWnHvRQtSH0HtnmzS/2Dt/ckFuyPCmT4neFD5kXhSaH9jnYp3m21i6m2SJByctudXP5wSethg8cRvXGPGZGWWnuBvpuw4Z9vI33RxhwygO5D44bKNCGJLnN/ppEUkPYWr3jDEkrGuVvD62o8bp0hvPe1SpGytczhodGQFgrR9cv2AiqvhHuj/amcqo3MRoUIjIE40+WkQdkgX37Bgw9bK078oTIPc7yWZa6eUwf/jgyGr7asdilxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=24TzNeXkluknvjlUOjwTTRzo7llaAUU14RNeMwlWz6E=;
 b=Bqi2qcBh2Su8jV2PFYwVsb4nv9e8+nBZ4JnXsJ8Bjda0SUfFS4yuWJfuuebcw2crHzp4V7+c95WfREi0kE3lnDDYjwcYdfoFAsI7nWiY+ctu1ndIjJSVOX8k/HCUzW/R46fpOtH50AcIMdBNnO7aRw6QskATQ3329SnHrI4sTntWdi1+HOOf2NfihE8fMXsMJHgnfRrHI0BQhd4ZUz61R0FHnFH7hNfz0pFckA/blPfKUGtb5DFE+qhtCCQzLlwn6H57uHt7TNqjrSXXRyRoAFv9spfpBDnFZunSTTJfXvrQouxwxzXwwp0UUWGyjRaL6FkgDDTEznhqTeDyhtNsIQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=24TzNeXkluknvjlUOjwTTRzo7llaAUU14RNeMwlWz6E=;
 b=Rle0oijAtTyX8gF6xwDUp77hGrbcGNjm/dLMTBs3Vd9UIlItm6OAyJOaNjNsoG7iZLv/R+BcLsK4uYGCbP5H75MAfR/LTR/UYBeN71RO4+xybFX5VVSVyoJpCUw/mwZuzaaUfp0gDp4flMzjWjiTgJUsRMSkXjgfiFQeMrYqk+I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 26/37] xen/arm: build NUMA cpu_to_node map in dt_smp_init_cpus
Date: Thu, 23 Sep 2021 20:02:25 +0800
Message-ID: <20210923120236.3692135-27-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e9b1f90-1cdc-4885-ebc2-08d97e8a4d81
X-MS-TrafficTypeDiagnostic: AM4PR0802MB2177:|DBBPR08MB4252:
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB42523378052DBB690DFF35919EA39@DBBPR08MB4252.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GStpNBt+BE9ZbmniAnLail5zd1teo6+J6nP1GQLsN76b4i7pu+rRRdWK3ePMZ1f/yY7PXJkXAjETVB+hfa/Xskg8DqW2H4CFV1YOz/a7TI3thGRzLSgZArncCBFE4jFMC/cGnh/3mMu2LomYlaUJuJaAgQ9RHuawmhkuRc2u+C2p9KbELTdRZdJ2DNohd5lxrye+uDQ7R2MFx4bqSlu/b/FkO2Dz177c8+jeCenac433BcubDr437T8MFt3XdfLrxBUSQhGeMGVrOyw6G8P4cyM7mgrxePTjWnbz3H15t+XNbaezZCbOkOU0/T8kKuodKLEH5TGmfJ/9PGc+jTFWJCyotQVSjRT4R4SOnnAlxxU9p5x4j9zytuiMRlpRCic6/yHZyWjh+HV3wAmOIPfbrmcBfugPzmy+8W1fUxQpJ7nkJKJE2+b9OFYlFToPUSMoXbS1WSQK0q7V8vephaWRARmT25+RFsnzT6LY6g1XytCvANZh0xCSyysI4lXM8h/96IeEetvnCvUnJ3U4Ita/CVz9mqQX+UqRTOmsIyzCgXuGHZSjGpLuJk3jKgMgIWcizXC8YyPf59SKQz+gDG7Kfs9nxWBHjtpatv6P3tmyjwdbTxIiuyHGyf9psHfnmEn9F22GzrD6CoC1J8LGbw0n/EtSKBs5eaSQ/Yb/EZ5ucWWKYXZ8agQ8mDDAtJRwxjwJZbLmsnq5f/H44/GhkTsEfp3y6b1MAdHYZATsbiuOzlU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(356005)(508600001)(336012)(186003)(2616005)(81166007)(1076003)(36756003)(8936002)(8676002)(86362001)(426003)(4326008)(26005)(44832011)(36860700001)(70206006)(70586007)(7696005)(83380400001)(2906002)(110136005)(6666004)(82310400003)(316002)(47076005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2177
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c3f9b71d-4c33-473c-5ddb-08d97e8a4584
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ozgF1EfbjuLDjlnQtcn6oISBztjGhmoXbiGaOtQexdXodaHTyqAqqPG34VnQSluOhz367JkkhwREjTqQPVMh3kUTu0HD8HW0uygYiK3d83izjIzHNWoZKxTvh5d/U7G0xtSYTSQhSBLEfDMe6XlhkK27pdN2zZ7/0S6/sx6qRenEg2zZgVDty/a0Jn8JB6zf5MZC7kaET6UOdl8zpdg9plhr4LOVOIPZ1zCGZ7jvAlUsjFUcGQG5ZHJQpahWr09Bl0DV1Vuw2zfL6UJV5dnPlHf6CRSJ79I+aC1QjnSCdE3RzEgQAQh42OoQG5cJimQ/XsXZhy1FkVbj+twk6/K6wODXnypWqmihfOrODVcohYQCliNwjCYZI6cWO/H+13oW6r++wq/Tt0d/sAhJ146WPMa9D9mhIqkEKtvIFE/QYHL5iT0TRmJltik1a5gDgu4c3ih2vHZXwAi1rrchJSW6bMDWIBDFCVoGRQrb0/lUjhy43saWUozGtBtgtwijK/iWFTv0USPSuArWLVxKjfQj4hukIQK1HMDVs1J5/Kk47p14+Vt8zoBwLpTk3aTrIQXXEemeyoPBHczLomofFOwXuR4FhJWSlMr7y6tcaDaIYfr4NpfNyrfZGf5mucm/aJ4c2PTY0eA9BhNwKC0AejVRLSm8PuUfz5tXA3eMPGlWHs+W+XV5G48JI+epnTVdgirs4PM/k7lSuqEgXifEWvVmew==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(83380400001)(5660300002)(2906002)(6666004)(36860700001)(86362001)(44832011)(70586007)(316002)(81166007)(508600001)(8676002)(8936002)(4326008)(2616005)(110136005)(82310400003)(426003)(26005)(7696005)(186003)(70206006)(336012)(47076005)(36756003)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:32.0477
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e9b1f90-1cdc-4885-ebc2-08d97e8a4d81
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4252

NUMA implementation has a cpu_to_node array to store CPU to NODE
map. Xen is using CPU logical ID in runtime components, so we
use CPU logical ID as CPU index in cpu_to_node.

In device tree case, cpu_logical_map is created in dt_smp_init_cpus.
So, when NUMA is enabled, dt_smp_init_cpus will fetch CPU NUMA id
at the same time for cpu_to_node.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/smpboot.c     | 37 ++++++++++++++++++++++++++++++++++++-
 xen/include/asm-arm/numa.h |  5 +++++
 2 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 60c0e82fc5..6e3cc8d3cc 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -121,7 +121,12 @@ static void __init dt_smp_init_cpus(void)
     {
         [0 ... NR_CPUS - 1] = MPIDR_INVALID
     };
+    static nodeid_t node_map[NR_CPUS] __initdata =
+    {
+        [0 ... NR_CPUS - 1] = NUMA_NO_NODE
+    };
     bool bootcpu_valid = false;
+    uint32_t nid = 0;
     int rc;
 
     mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
@@ -172,6 +177,28 @@ static void __init dt_smp_init_cpus(void)
             continue;
         }
 
+        if ( IS_ENABLED(CONFIG_NUMA) )
+        {
+            /*
+             * When CONFIG_NUMA is set, try to fetch numa infomation
+             * from CPU dts node, otherwise the nid is always 0.
+             */
+            if ( !dt_property_read_u32(cpu, "numa-node-id", &nid) )
+            {
+                printk(XENLOG_WARNING
+                       "cpu[%d] dts path: %s: doesn't have numa information!\n",
+                       cpuidx, dt_node_full_name(cpu));
+                /*
+                 * During the early stage of NUMA initialization, when Xen
+                 * found any CPU dts node doesn't have numa-node-id info, the
+                 * NUMA will be treated as off, all CPU will be set to a FAKE
+                 * node 0. So if we get numa-node-id failed here, we should
+                 * set nid to 0.
+                 */
+                nid = 0;
+            }
+        }
+
         /*
          * 8 MSBs must be set to 0 in the DT since the reg property
          * defines the MPIDR[23:0]
@@ -231,9 +258,12 @@ static void __init dt_smp_init_cpus(void)
         {
             printk("cpu%d init failed (hwid %"PRIregister"): %d\n", i, hwid, rc);
             tmp_map[i] = MPIDR_INVALID;
+            node_map[i] = NUMA_NO_NODE;
         }
-        else
+        else {
             tmp_map[i] = hwid;
+            node_map[i] = nid;
+        }
     }
 
     if ( !bootcpu_valid )
@@ -249,6 +279,11 @@ static void __init dt_smp_init_cpus(void)
             continue;
         cpumask_set_cpu(i, &cpu_possible_map);
         cpu_logical_map(i) = tmp_map[i];
+
+        nid = node_map[i];
+        if ( nid >= MAX_NUMNODES )
+            nid = 0;
+        numa_set_node(i, nid);
     }
 }
 
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 758eafeb05..8a4ad379e0 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -46,6 +46,11 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline void numa_set_node(int cpu, nodeid_t node)
+{
+
+}
+
 #endif
 
 static inline unsigned int arch_have_default_dmazone(void)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193867.345427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVj-00067C-Df; Thu, 23 Sep 2021 12:07:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193867.345427; Thu, 23 Sep 2021 12:07:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVj-000669-5w; Thu, 23 Sep 2021 12:07:31 +0000
Received: by outflank-mailman (input) for mailman id 193867;
 Thu, 23 Sep 2021 12:07:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNSk-0005KP-W2
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:27 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::620])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8e392c35-b95e-4c30-8195-80009b707bb9;
 Thu, 23 Sep 2021 12:03:54 +0000 (UTC)
Received: from PR0P264CA0184.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::28)
 by HE1PR08MB2873.eurprd08.prod.outlook.com (2603:10a6:7:30::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:03:44 +0000
Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::e5) by PR0P264CA0184.outlook.office365.com
 (2603:10a6:100:1c::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:44 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:44 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:03:42 +0000
Received: from 9c69dda446e7.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B77D722B-197B-4BEF-98E4-0803B71C25E7.1; 
 Thu, 23 Sep 2021 12:03:26 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9c69dda446e7.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:26 +0000
Received: from DB6P193CA0002.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::12) by
 DB6PR0801MB1926.eurprd08.prod.outlook.com (2603:10a6:4:6c::20) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.16; Thu, 23 Sep 2021 12:03:24 +0000
Received: from DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:29:cafe::67) by DB6P193CA0002.outlook.office365.com
 (2603:10a6:6:29::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:24 +0000
Received: from nebula.arm.com (40.67.248.234) by
 DB5EUR03FT032.mail.protection.outlook.com (10.152.20.162) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:24 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:24 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:22 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e392c35-b95e-4c30-8195-80009b707bb9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hKGWyOZlxxh43KHp7ZOxfxEZJMhlD15ad4VHvSvtHzE=;
 b=bc4CJPDNJeBqmaNVOzPGFxoBH34QB5MFYuLELOKkalfMzHKJranlDRzX0bdvODm2Cqwhkd6C+BtDhC5mXnlBdfTR28J/y2HRhX6GpSRkUwpgBqPpGft97/yJcDsXKH+O8aNhTClmHb2JqwKtAR97GhNZ29gbXLvcxQ5s7EX3DJs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: bfeb19c4adcec2df
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HfX+etrn9Uey+4KzLB96tj0eHEBNpInNru9o6XbSGLQ3jy1KuJRXbchW8AnkBav7m3yGUdxqkYXHsIFExSKxhoNUudRh7dBaHDvIGPceoijbxImTxI4yVyAGWxrUM7oTy9QDry/QTqDCV/2kzNH4UWlYsrvR7W0wU0mXERx/2o6gJdDWTVCy8PYglBc0mygdXkXeKOezwQssjf13rvB9mDwWVHp6MUHoiWITLV+H+caYaVMCYJi9VKmZmv/ExQmVDJAnAIfkSF51GQ8/U0pijzvVENIRz8cZoKvz4usV71RfJilqL5GDqZrXdF6lrrAbdoyLJqlav6T7996UM/aspA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hKGWyOZlxxh43KHp7ZOxfxEZJMhlD15ad4VHvSvtHzE=;
 b=dwLHbxyTLgfSdq9MnnactGv8eO/9dCt6W/ee5BVhgGghjCZjSXJU5UtEwPsNdlJnJoI20brbirH08mqxUUax9v1pcL1E2JG4eckuCDTIkik/jyx2pKi0YuPxvBTLgTCoVjW7UgU3Xf2wPVzH2Qa+9pMnST0/1wMNxJ9UfFFgRQOfDqgJhinpWUYuB77ebcgA5PjqDXlZUwpjpLJtYfSz3GP/lIbpVKTmE7mwAwxCDMOLEYcR+FDa0L1Lt0PT3OxT9VKkMnfnD8mAeOaYXmJc4ehM03nI9SHJEjmjNmUhUaeIHho+9riWGvZTay9mCxajs8KsBxj/GMFFjKkkZnuCOg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hKGWyOZlxxh43KHp7ZOxfxEZJMhlD15ad4VHvSvtHzE=;
 b=bc4CJPDNJeBqmaNVOzPGFxoBH34QB5MFYuLELOKkalfMzHKJranlDRzX0bdvODm2Cqwhkd6C+BtDhC5mXnlBdfTR28J/y2HRhX6GpSRkUwpgBqPpGft97/yJcDsXKH+O8aNhTClmHb2JqwKtAR97GhNZ29gbXLvcxQ5s7EX3DJs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node structure
Date: Thu, 23 Sep 2021 20:02:06 +0800
Message-ID: <20210923120236.3692135-8-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ab2834c-4178-4f9b-5bb7-08d97e8a3124
X-MS-TrafficTypeDiagnostic: DB6PR0801MB1926:|HE1PR08MB2873:
X-Microsoft-Antispam-PRVS:
	<HE1PR08MB2873F0BD54D0A53FB98E8EAE9EA39@HE1PR08MB2873.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:935;OLM:935;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Nnf+ibO55AWQQfBEFFAOvG+NflpSS8wWS7KOIrJmFExDxAFPE6TGxPzSk4wlb9pLY/j09CPafUFbfZbcfKqtWsEmDArxH0d0ULpukbifbz9i3Ev/OwenSWV/LCbC9nMRGnk46kzOGcp4X3TBwWMh4Y4E5zCbnF7Wg4JOoIpx7ct8xk8yeCMJ7OWH4+ivy78LBx4MpI7hbore9EFD3DVOQDpHaUF6lGleJZ9Wyopy34Ix51kVAA77xQZzxfEyM7Fo0eeYGYPUkAOZDdOzJ9Yax4k7ZDk2+Ip+BfyXDN0B400Dg999fBN550iKJdGGCXg7QQQzgk0H8RdYLAZ+QqIed7bWBGLWfXP2itLvSPjF8buk7A/oa5Y+7J8ji0nMnFaUR4seEUB/ITCTXaLsXYbmITcAQDEEFhKsKcYW3Zk0XM5f/YpVw1Q31ZzbYxVWDwdzA4IqsWSspFP5a3qNtCDmEnqdHwGLi7+2dWUNSSo8/ABTh0m11+ZoOh6W6jb2BNIs1DQ8m3Sps8zcFn8soCf5RdnyeMkJHPkfOL/2weO1rL6CjkS8vYdSScAQzfEsfjNbTGeggq7TeMP5cDCJidUL4+MQvP8alsosdAiA9TL1Pfk9kwEkMLr7WVS2rK661Xy6ZRklQQnXc2NhF3xZnT+yB9bvshgNFOmqWJL7eG9+bNMB2aG2zTU1evJ4I4VeZiIp0P23anvB+jTZNskg0387NgJqBCTV6Kxm71YdihpvPU8=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(83380400001)(508600001)(426003)(336012)(186003)(86362001)(82310400003)(70206006)(36860700001)(110136005)(7696005)(5660300002)(36756003)(81166007)(356005)(44832011)(316002)(70586007)(2906002)(8936002)(4326008)(47076005)(6666004)(2616005)(1076003)(26005)(8676002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1926
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e30fb11a-b275-4250-4e12-08d97e8a255d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	bVxWPuK1X2fgvk3Mu093LbsVC5OSkzfzl2vXnaJ4v67rED2pzcGZBnu9SyqnoTO8UnW4L+v1bYHg0g5o5hd9lsMcm/uIMdJIU9rfHxjCJm+HiQlf/7NzeeGaPpaJOo4mr5EmJ0yjdet/B4/wnzbjg/fY1MA+pUyyRAFKbpmoSCUIcoS9WMT8VhAJu4frZ+JjATpq16K68NnhVUSuQsMIvL6cjtoat66QhwXwBtog4gKsKWiI/IGOkL1YZirNhngQOjFA5k/nxq8pt6ce+2/nRQWUcGYr0zpV1T8Qhviaq3SevSJ84CT4wnEMOcl24zsBTIQ1HbPvVJL4pA3OWwBuogoDS1+i/AUVinOBVnC//NGNUsmlkj0WDLAR9lEAQexnp6MO/uuLSdsRmmOmyR2JPrbTIpB8oo/EXVpfM90IzF+DkPp83NLxGgQ+EGWvN9MDB40fEVlpq/KzPNFfeGZYXPNWoG++UIid6rdHUYYBRD67H4oICOPdGHjo+XYu1D2YLsP8VCiTgyo2KY1+/kjLnxRBK1JLDLiIPHZmzB+OlmjRQGFXFWY96eD/gWQ+OHPXA6bLOnshFLaokCt/nAHFLFRWtYuMV51uZ39pio4ShuxFM2gPPn5xv2ckFMYxKXVlN0Dz/KErmb7X5vOItxce/CNx4PN25AS5tN1rkpGG/uG+D6Sc15621yrHmb7vebMlHG91ogWKXuTLeq//P3ermA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(186003)(81166007)(316002)(82310400003)(110136005)(83380400001)(508600001)(44832011)(86362001)(7696005)(70586007)(4326008)(336012)(36860700001)(1076003)(5660300002)(426003)(2616005)(47076005)(8676002)(26005)(36756003)(6666004)(2906002)(8936002)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:44.3269
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ab2834c-4178-4f9b-5bb7-08d97e8a3124
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR08MB2873

NUMA node structure "struct node" is using u64 as node memory
range. In order to make other architectures can reuse this
NUMA node relative code, we replace the u64 to paddr_t. And
use pfn_to_paddr and paddr_to_pfn to replace explicit shift
operations. The relate PRIx64 in print messages have been
replaced by PRIpaddr at the same time.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c        | 32 +++++++++++++++++---------------
 xen/arch/x86/srat.c        | 26 +++++++++++++-------------
 xen/include/asm-x86/numa.h |  8 ++++----
 3 files changed, 34 insertions(+), 32 deletions(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 1fabbe8281..6337bbdf31 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -165,12 +165,12 @@ int __init compute_hash_shift(struct node *nodes, int numnodes,
     return shift;
 }
 /* initialize NODE_DATA given nodeid and start/end */
-void __init setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end)
-{ 
+void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end)
+{
     unsigned long start_pfn, end_pfn;
 
-    start_pfn = start >> PAGE_SHIFT;
-    end_pfn = end >> PAGE_SHIFT;
+    start_pfn = paddr_to_pfn(start);
+    end_pfn = paddr_to_pfn(end);
 
     NODE_DATA(nodeid)->node_start_pfn = start_pfn;
     NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn;
@@ -201,11 +201,12 @@ void __init numa_init_array(void)
 static int numa_fake __initdata = 0;
 
 /* Numa emulation */
-static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
+static int __init numa_emulation(unsigned long start_pfn,
+                                 unsigned long end_pfn)
 {
     int i;
     struct node nodes[MAX_NUMNODES];
-    u64 sz = ((end_pfn - start_pfn)<<PAGE_SHIFT) / numa_fake;
+    u64 sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake;
 
     /* Kludge needed for the hash function */
     if ( hweight64(sz) > 1 )
@@ -221,9 +222,9 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
     memset(&nodes,0,sizeof(nodes));
     for ( i = 0; i < numa_fake; i++ )
     {
-        nodes[i].start = (start_pfn<<PAGE_SHIFT) + i*sz;
+        nodes[i].start = pfn_to_paddr(start_pfn) + i*sz;
         if ( i == numa_fake - 1 )
-            sz = (end_pfn<<PAGE_SHIFT) - nodes[i].start;
+            sz = pfn_to_paddr(end_pfn) - nodes[i].start;
         nodes[i].end = nodes[i].start + sz;
         printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n",
                i,
@@ -249,24 +250,26 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
 void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
 { 
     int i;
+    paddr_t start, end;
 
 #ifdef CONFIG_NUMA_EMU
     if ( numa_fake && !numa_emulation(start_pfn, end_pfn) )
         return;
 #endif
 
+    start = pfn_to_paddr(start_pfn);
+    end = pfn_to_paddr(end_pfn);
+
 #ifdef CONFIG_ACPI_NUMA
-    if ( !numa_off && !acpi_scan_nodes((u64)start_pfn << PAGE_SHIFT,
-         (u64)end_pfn << PAGE_SHIFT) )
+    if ( !numa_off && !acpi_scan_nodes(start, end) )
         return;
 #endif
 
     printk(KERN_INFO "%s\n",
            numa_off ? "NUMA turned off" : "No NUMA configuration found");
 
-    printk(KERN_INFO "Faking a node at %016"PRIx64"-%016"PRIx64"\n",
-           (u64)start_pfn << PAGE_SHIFT,
-           (u64)end_pfn << PAGE_SHIFT);
+    printk(KERN_INFO "Faking a node at %016"PRIpaddr"-%016"PRIpaddr"\n",
+           start, end);
     /* setup dummy node covering all memory */
     memnode_shift = BITS_PER_LONG - 1;
     memnodemap = _memnodemap;
@@ -279,8 +282,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
     for ( i = 0; i < nr_cpu_ids; i++ )
         numa_set_node(i, 0);
     cpumask_copy(&node_to_cpumask[0], cpumask_of(0));
-    setup_node_bootmem(0, (u64)start_pfn << PAGE_SHIFT,
-                    (u64)end_pfn << PAGE_SHIFT);
+    setup_node_bootmem(0, start, end);
 }
 
 void numa_add_cpu(int cpu)
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 6b77b98201..7d20d7f222 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -104,7 +104,7 @@ nodeid_t setup_node(unsigned pxm)
 	return node;
 }
 
-int valid_numa_range(u64 start, u64 end, nodeid_t node)
+int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
 {
 	int i;
 
@@ -119,7 +119,7 @@ int valid_numa_range(u64 start, u64 end, nodeid_t node)
 	return 0;
 }
 
-static __init int conflicting_memblks(u64 start, u64 end)
+static __init int conflicting_memblks(paddr_t start, paddr_t end)
 {
 	int i;
 
@@ -135,7 +135,7 @@ static __init int conflicting_memblks(u64 start, u64 end)
 	return -1;
 }
 
-static __init void cutoff_node(int i, u64 start, u64 end)
+static __init void cutoff_node(int i, paddr_t start, paddr_t end)
 {
 	struct node *nd = &nodes[i];
 	if (nd->start < start) {
@@ -275,7 +275,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
 void __init
 acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 {
-	u64 start, end;
+	paddr_t start, end;
 	unsigned pxm;
 	nodeid_t node;
 	int i;
@@ -318,7 +318,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 		bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
 		                !test_bit(i, memblk_hotplug);
 
-		printk("%sSRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with itself (%"PRIx64"-%"PRIx64")\n",
+		printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
 		       mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
 		       node_memblk_range[i].start, node_memblk_range[i].end);
 		if (mismatch) {
@@ -327,7 +327,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 		}
 	} else {
 		printk(KERN_ERR
-		       "SRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with PXM %u (%"PRIx64"-%"PRIx64")\n",
+		       "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
 		       pxm, start, end, node_to_pxm(memblk_nodeid[i]),
 		       node_memblk_range[i].start, node_memblk_range[i].end);
 		bad_srat();
@@ -346,7 +346,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 				nd->end = end;
 		}
 	}
-	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIx64"-%"PRIx64"%s\n",
+	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
 	       node, pxm, start, end,
 	       ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
 
@@ -369,7 +369,7 @@ static int __init nodes_cover_memory(void)
 
 	for (i = 0; i < e820.nr_map; i++) {
 		int j, found;
-		unsigned long long start, end;
+		paddr_t start, end;
 
 		if (e820.map[i].type != E820_RAM) {
 			continue;
@@ -396,7 +396,7 @@ static int __init nodes_cover_memory(void)
 
 		if (start < end) {
 			printk(KERN_ERR "SRAT: No PXM for e820 range: "
-				"%016Lx - %016Lx\n", start, end);
+				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
 			return 0;
 		}
 	}
@@ -432,7 +432,7 @@ static int __init srat_parse_region(struct acpi_subtable_header *header,
 	return 0;
 }
 
-void __init srat_parse_regions(u64 addr)
+void __init srat_parse_regions(paddr_t addr)
 {
 	u64 mask;
 	unsigned int i;
@@ -441,7 +441,7 @@ void __init srat_parse_regions(u64 addr)
 	    acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat))
 		return;
 
-	srat_region_mask = pdx_init_mask(addr);
+	srat_region_mask = pdx_init_mask((u64)addr);
 	acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
 			      srat_parse_region, 0);
 
@@ -457,7 +457,7 @@ void __init srat_parse_regions(u64 addr)
 }
 
 /* Use the information discovered above to actually set up the nodes. */
-int __init acpi_scan_nodes(u64 start, u64 end)
+int __init acpi_scan_nodes(paddr_t start, paddr_t end)
 {
 	int i;
 	nodemask_t all_nodes_parsed;
@@ -489,7 +489,7 @@ int __init acpi_scan_nodes(u64 start, u64 end)
 	/* Finally register nodes */
 	for_each_node_mask(i, all_nodes_parsed)
 	{
-		u64 size = nodes[i].end - nodes[i].start;
+		paddr_t size = nodes[i].end - nodes[i].start;
 		if ( size == 0 )
 			printk(KERN_WARNING "SRAT: Node %u has no memory. "
 			       "BIOS Bug or mis-configured hardware?\n", i);
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 8060cbf3f4..50cfd8e7ef 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -16,7 +16,7 @@ extern cpumask_t     node_to_cpumask[];
 #define node_to_cpumask(node)    (node_to_cpumask[node])
 
 struct node { 
-	u64 start,end; 
+	paddr_t start,end;
 };
 
 extern int compute_hash_shift(struct node *nodes, int numnodes,
@@ -36,7 +36,7 @@ extern void numa_set_node(int cpu, nodeid_t node);
 extern nodeid_t setup_node(unsigned int pxm);
 extern void srat_detect_node(int cpu);
 
-extern void setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end);
+extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end);
 extern nodeid_t apicid_to_node[];
 extern void init_cpu_to_node(void);
 
@@ -73,9 +73,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
 #define node_end_pfn(nid)       (NODE_DATA(nid)->node_start_pfn + \
 				 NODE_DATA(nid)->node_spanned_pages)
 
-extern int valid_numa_range(u64 start, u64 end, nodeid_t node);
+extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
 
-void srat_parse_regions(u64 addr);
+void srat_parse_regions(paddr_t addr);
 extern u8 __node_distance(nodeid_t a, nodeid_t b);
 unsigned int arch_get_dma_bitsize(void);
 unsigned int arch_have_default_dmazone(void);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193868.345444 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVp-0006mR-Oy; Thu, 23 Sep 2021 12:07:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193868.345444; Thu, 23 Sep 2021 12:07:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVp-0006mC-Kp; Thu, 23 Sep 2021 12:07:37 +0000
Received: by outflank-mailman (input) for mailman id 193868;
 Thu, 23 Sep 2021 12:07:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNV4-0006xV-T9
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:50 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.88]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 727f9474-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:49 +0000 (UTC)
Received: from DB8P191CA0027.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:130::37)
 by PAXPR08MB6541.eurprd08.prod.outlook.com (2603:10a6:102:15a::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:47 +0000
Received: from DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:130:cafe::5) by DB8P191CA0027.outlook.office365.com
 (2603:10a6:10:130::37) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:47 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT061.mail.protection.outlook.com (10.152.21.234) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:47 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 12:04:46 +0000
Received: from 60a7a5fd6538.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D06DFBD3-EACD-4321-8B02-FEDEB3E5EADC.1; 
 Thu, 23 Sep 2021 12:04:37 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 60a7a5fd6538.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:37 +0000
Received: from AS9PR06CA0100.eurprd06.prod.outlook.com (2603:10a6:20b:465::11)
 by AM0PR08MB4228.eurprd08.prod.outlook.com (2603:10a6:208:138::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 23 Sep
 2021 12:04:36 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::c4) by AS9PR06CA0100.outlook.office365.com
 (2603:10a6:20b:465::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:36 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:36 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:15 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 727f9474-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=V5dCi7RLHOGiZwjaffySQjpEEHSFmEIlzO2cAQzos3g=;
 b=l5Jsji4GLT7sCjHfjs2qGAYISvSiRQm7uheZnkHoMUFsd+bB5Cq3o5x7O15o/dlmVQjTJp1OOT0ErNKupDNvPvlWxiaAm2TghCWPtn0JCEVvGPlIzxDgb3QKUfjnRC4xqzkp1tGs3R170FBY6BmFgP3FShS5YuVz6xARDLEVCPo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ef6df92a1e0be266
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nWAScDte5A/sajD+TCyI8Ag9Ekykw0r5fbmaOkmDWjyRj7tuV105qi2Cx4HnlqPnpBovzMZb6N3U0IX9bRR0yXUJzFZ9zuV3lEQH97slkCeo04+KEfLrWQNRPDB94q8aZr5MBR5KohG9iGnSzM+lUVlPsafZjw9BPw7o3b3ZAzqjFHErHG3T8H2whVooAr02Yuhpw52PkIzahZjp6qJ8rh2jYCjaB0xso+42lIqsTS7QM5uoM3K4ZBuFo7sif1u3ej2VMpTRGlP0LI1ipeNAGL3HaXB7ga8lSL6OaC9D0jFWGSr3jxrBGeNeiWr3NOQDkmRXyJ56IavLMiMnBrKmQw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V5dCi7RLHOGiZwjaffySQjpEEHSFmEIlzO2cAQzos3g=;
 b=MjuN5ixREG24663ZzfVO4c2k1ZOJ78DUtTp/9CW1PgbwigKVqyB7FOUyxFgvqfKb73hDo/Wb3Poqz+a7kzKe57BUD5tmZhL2hIdA8axiIHL13f0N9F7Ete+20lYsanIqLoS+lwiphFWY9SuBRtWG4qFnfrXJV2VcVvGYRwtLtFNJ/JI/JQDb3to72jbbwbrx6VvaaODF5FzyRb84uL+33rY9CFGJVWg4m728nJtQcRpmFsLzwWAidga0PJr++7mP9lm8ixDYHwkGfOECVj4KQYlcw+QHUKd8hdDwS961xGWryiqin7KOe0rHGcZFsn7eMKo0gjivS8gkCcR+wOuWlg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=V5dCi7RLHOGiZwjaffySQjpEEHSFmEIlzO2cAQzos3g=;
 b=l5Jsji4GLT7sCjHfjs2qGAYISvSiRQm7uheZnkHoMUFsd+bB5Cq3o5x7O15o/dlmVQjTJp1OOT0ErNKupDNvPvlWxiaAm2TghCWPtn0JCEVvGPlIzxDgb3QKUfjnRC4xqzkp1tGs3R170FBY6BmFgP3FShS5YuVz6xARDLEVCPo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 30/37] xen/arm: introduce a helper to parse device tree memory node
Date: Thu, 23 Sep 2021 20:02:29 +0800
Message-ID: <20210923120236.3692135-31-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40370d81-b797-4c86-e01b-08d97e8a5681
X-MS-TrafficTypeDiagnostic: AM0PR08MB4228:|PAXPR08MB6541:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6541D5E42CD45CCF0C95AFE49EA39@PAXPR08MB6541.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YcJKU/oOfBJYCoGuRGXQFHKmi1+4wu6ZWxCym+qF4vmJajknFztJ0P2Yu3Um15CVWqQ33vn/yO6zZrkMPiq5eRelzrpC6/GZoI5kf7Ny28LeUh4npbS08QKNtrtJAtasFlfzbeaQuOi+B4lBD3+/SKuKzGXjuuyaGr6FBDuWzw9Vq2rizMSfxk00vxyy8vtIkF+GuCg2/1S4q69bUmSwiSwMjlk2MHiqlj2Aqiug8G5xfQ7C3+Agc5kyZ5Yg4a0gQKnDBzN44jm3UoHbLiLnCYSR7QGQEghtqho6z6Jyz2RcL8cpTgGtOhzKAl07mM1kBXClEg+SRuPQ3F9qdkEuzj9YXmMsPpLoltpr5gcYh6/ORY7HEwVwFXp/N2ds3TS7vdWSR4WlAy7bGk1H4T89lSZiykJd+R10Ld/hDFrPQOf59S6CrdC74BryCedR8h/X8+oyrV0363RMhUP+RCCNs4fgecfrvohu9QoASWznW28VV84ZvQw7QJHdhqNSWv9yybCsain1iG6h3SPxRx+GX4S88kKw1sv1SmQla5t3F+ad+eLrH8ZGyj5B0iJUlE6/kARPKlUqDwpnbj0MWJtwm0a36uNP6/qe6YugtnJknjLiCzswMEmkMdgGO/H7Y5zuPo2IsUYDAX+Kee4gHCRjLuD5MdqPyJnImSWRaPI30UZ1a5hojaCbNLAlq4RpQmsCcQJychAzvX74Ov0YyTW0kYlX8+WtnPy+2PS61bMxLwQ=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(36860700001)(508600001)(356005)(83380400001)(426003)(110136005)(6666004)(82310400003)(47076005)(70586007)(70206006)(316002)(5660300002)(86362001)(8676002)(8936002)(7696005)(2906002)(36756003)(26005)(1076003)(44832011)(186003)(81166007)(4326008)(336012)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4228
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	268c5f9b-2163-4497-8f2d-08d97e8a4fe4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yEd809IuaD1FS0Yo4ezVc1OxJU9ltWUh9NVeI5ovZfdXC/tugWMEMoKjrQReiH5alCo3ukITtfV9xLyRw6TuWSJhexn0J2x+Fjzd0in+SZA5hbTHEF4jiBi7FSExlhTkAQXxnYNHTC6Kb2owFxNcnAziEf2KMWZyEWfCeA4xN3ODNczdUlfboAgg9EyHSXK7BUHuxCBV+ziICv3X7uwY7JOBiP0mNK1QWb0NNXyeIunS8wAHuphYBaiZCgJD7HQbzQwVE4R/xmljcRa1OBvlSWrzvxJu1fir6pLwHUoPhUlYE6S8GbfIEJmoY+RUuzluuTHukG+bk8vAcJwnWOBqxmfFRDQrbQ2i/iCqEqhaG1rBY3qRYdkUOuu7bE0TRuSJQ3R/LnIMU27DyHNHYCaxmpqrfoR+v04Qb00dupKNWhJdRPS6j0/qQLGXtLOq0huX0VBY2M/1vJl7Un5AXh6gUVDalkkmA3mfSZljo5sjm65Xd9kjnMJLj/i914143xgshCaGq8BQ8zEsoCaCuuDOE/pCN+SDQhuxMtiv4NfQHDpgir3eJAutGTsZS+510gpIKp2PqCk7SFfPu5Zps2iCLEj6hLLGpIRRRaN41ZZ6tfd2Kil15W6yt9K5UDjorzWgQBbqIhEkAVeYfvllYhn3EH2vN95RPH+PIbgymG3icNsuKuDY9VpyOShCtNfo7g7pGg+oV3FUOsltmNB2wv1kYg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(508600001)(426003)(86362001)(336012)(316002)(1076003)(36756003)(83380400001)(186003)(26005)(70206006)(2616005)(44832011)(70586007)(47076005)(36860700001)(110136005)(7696005)(8936002)(6666004)(2906002)(8676002)(81166007)(82310400003)(5660300002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:47.1514
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 40370d81-b797-4c86-e01b-08d97e8a5681
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6541

Memory blocks' NUMA ID information is stored in device tree's
memory nodes as "numa-node-id". We need a new helper to parse
and verify this ID from memory nodes.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa_device_tree.c | 80 +++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)

diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
index 2428fbae0b..7918a397fa 100644
--- a/xen/arch/arm/numa_device_tree.c
+++ b/xen/arch/arm/numa_device_tree.c
@@ -42,6 +42,35 @@ static int __init fdt_numa_processor_affinity_init(nodeid_t node)
     return 0;
 }
 
+/* Callback for parsing of the memory regions affinity */
+static int __init fdt_numa_memory_affinity_init(nodeid_t node,
+                                paddr_t start, paddr_t size)
+{
+    int ret;
+
+    if ( srat_disabled() )
+    {
+        return -EINVAL;
+    }
+
+	if ( !numa_memblks_available() )
+	{
+		dprintk(XENLOG_WARNING,
+                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
+		bad_srat();
+		return -EINVAL;
+	}
+
+	ret = numa_update_node_memblks(node, start, size, false);
+	if ( ret != 0 )
+	{
+		bad_srat();
+	    return -EINVAL;
+	}
+
+    return 0;
+}
+
 /* Parse CPU NUMA node info */
 static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
 {
@@ -56,3 +85,54 @@ static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
 
     return fdt_numa_processor_affinity_init(nid);
 }
+
+/* Parse memory node NUMA info */
+static int __init fdt_parse_numa_memory_node(const void *fdt, int node,
+    const char *name, uint32_t addr_cells, uint32_t size_cells)
+{
+    uint32_t nid;
+    int ret = 0, len;
+    paddr_t addr, size;
+    const struct fdt_property *prop;
+    uint32_t idx, ranges;
+    const __be32 *addresses;
+
+    nid = device_tree_get_u32(fdt, node, "numa-node-id", MAX_NUMNODES);
+    if ( nid >= MAX_NUMNODES )
+    {
+        printk(XENLOG_WARNING "Node id %u exceeds maximum value\n", nid);
+        return -EINVAL;
+    }
+
+    prop = fdt_get_property(fdt, node, "reg", &len);
+    if ( !prop )
+    {
+        printk(XENLOG_WARNING
+               "fdt: node `%s': missing `reg' property\n", name);
+        return -EINVAL;
+    }
+
+    addresses = (const __be32 *)prop->data;
+    ranges = len / (sizeof(__be32)* (addr_cells + size_cells));
+    for ( idx = 0; idx < ranges; idx++ )
+    {
+        device_tree_get_reg(&addresses, addr_cells, size_cells, &addr, &size);
+        /* Skip zero size ranges */
+        if ( !size )
+            continue;
+
+        ret = fdt_numa_memory_affinity_init(nid, addr, size);
+        if ( ret ) {
+            return -EINVAL;
+        }
+    }
+
+    if ( idx == 0 )
+    {
+        printk(XENLOG_ERR
+               "bad property in memory node, idx=%d ret=%d\n", idx, ret);
+        return -EINVAL;
+    }
+
+    return 0;
+}
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193869.345455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVr-000759-4k; Thu, 23 Sep 2021 12:07:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193869.345455; Thu, 23 Sep 2021 12:07:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVr-00074w-0M; Thu, 23 Sep 2021 12:07:39 +0000
Received: by outflank-mailman (input) for mailman id 193869;
 Thu, 23 Sep 2021 12:07:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNTA-0005KP-0g
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:04:52 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe05::604])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bc4ca0cf-2426-441b-962a-1f49f29c75db;
 Thu, 23 Sep 2021 12:03:57 +0000 (UTC)
Received: from DB3PR08CA0003.eurprd08.prod.outlook.com (2603:10a6:8::16) by
 AM6PR08MB3783.eurprd08.prod.outlook.com (2603:10a6:20b:8a::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14; Thu, 23 Sep 2021 12:03:55 +0000
Received: from DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8:0:cafe::c7) by DB3PR08CA0003.outlook.office365.com
 (2603:10a6:8::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:55 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT028.mail.protection.outlook.com (10.152.20.99) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:55 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:03:51 +0000
Received: from a0dd7d4ec432.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 391A3AAE-1392-41BE-8417-481DF8F46BA4.1; 
 Thu, 23 Sep 2021 12:03:44 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a0dd7d4ec432.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:03:44 +0000
Received: from AM6PR05CA0020.eurprd05.prod.outlook.com (2603:10a6:20b:2e::33)
 by AS8PR08MB6853.eurprd08.prod.outlook.com (2603:10a6:20b:39e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:03:43 +0000
Received: from AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2e:cafe::fb) by AM6PR05CA0020.outlook.office365.com
 (2603:10a6:20b:2e::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:43 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT055.mail.protection.outlook.com (10.152.17.214) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:03:42 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:39 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc4ca0cf-2426-441b-962a-1f49f29c75db
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rxcQF/l2VBvSGxaJXCbidBt4Razr3FEN1UGpSS7VamU=;
 b=zJHNUwyB+8RXKpklFjxdwOuWnSsumjjycAfIW/1bJWf5VwowKZc1sre+OMSoW7tXPeB92hX+wgcuO6A4oW2PVL4FLOg1POsWIv08uYtADKXHkvn0Xsa1X5TdeKGTb6joTJu2fzvB+5hAp4y6MZjCxTtii0D7f7lgx2eOzYltT8s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2d6f0d1f949a01f6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R9RgDAIkQY4uwu4MiZEoxHL3nE17jV4bpg6QJmhKG2HP6XLk+fsdF5hyReIXyyCXGfZTZ9Mr9VPaNcer1ohiWJF6hQ9CWbaHt9wqx019/hPsXKru/y3Vg1iQOVcarFnSiCqQ/X9h/qIth6o0L5DiFAOVqTzv1cTk8b+TOEOWBa3oq1zMptDQZ1Isu39M7cqlTWlMaAqE8EH4DPVqcpNkD7gRC4uWaF5lKe0QbV4QOo4MfoS69kknxK2QyFO5nB+XixiR9JJJpcUHwj41aHmDCOavsbEbo4YYuudQVSKz3X+7bXAviTI3/Vp3N6QfEd7tGXtgeXuPFIrq3x4ZRQLEPw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rxcQF/l2VBvSGxaJXCbidBt4Razr3FEN1UGpSS7VamU=;
 b=jPs4euhsdxfWL5cqUbaMlYFzM+SEoJajXJ458W+ZEw7/INY3XxNsbOaZYO/9QHcAWXaV/0G06al1bni/8XPoOXUUHslRakjeRZFsAMaUWTCExTXtKo6tI357OwaOSRez0cDn+dPtDgpxlU6CC2CbPCVsk5KTWkHydxEkcogC324VAZtuRIp+zlvGb2QtAEXG53PoXo6xOVc4lPBdACqkYMrwGLX2FMq0QtZDUKTHMD5BVG307SIgbQHbwjjubUSlra18sThIWFsXoYcENx9ElfNfQbsfQbRnEm/uyDKuTerM3hPbnEu0PthHL7Q8hhewvArBQbZ5gGdUOJr+8l7gAA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rxcQF/l2VBvSGxaJXCbidBt4Razr3FEN1UGpSS7VamU=;
 b=zJHNUwyB+8RXKpklFjxdwOuWnSsumjjycAfIW/1bJWf5VwowKZc1sre+OMSoW7tXPeB92hX+wgcuO6A4oW2PVL4FLOg1POsWIv08uYtADKXHkvn0Xsa1X5TdeKGTb6joTJu2fzvB+5hAp4y6MZjCxTtii0D7f7lgx2eOzYltT8s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 14/37] xen/x86: use name fw_numa to replace acpi_numa
Date: Thu, 23 Sep 2021 20:02:13 +0800
Message-ID: <20210923120236.3692135-15-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5389e76c-62cc-46b7-f6a9-08d97e8a37bd
X-MS-TrafficTypeDiagnostic: AS8PR08MB6853:|AM6PR08MB3783:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB37839AAE5F777092F80EF60E9EA39@AM6PR08MB3783.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rLUR4/pCjIS9x7cNthiiO3Ar48SawJFdwCNSpWTHq2p+P8k0ipVUWIQMyH77YgZtW/J1X0pcVu/y8zF10t5TKxnkEex3Twscy5Kw2RJ+65t+Fhh1NNQj+Q5JGXIZR0h3Z+YyzFTc+BlyeKqf9mOhFii09LIyrlTLIuxX9D/HfxDwoJVhy4mUcuJtnhF9COUE7TOZXrJTJHX0R003/JHL1rtu27oLExGAo9JMnvEdZ29rI/VLn6EKFgcgzB3ynrpXTp6Ef4KJNbhPLXTraYNeUs0XiiUHRkCZayg8W3wFzV0gTkkKKRG5gYs9yvb9bEZhSw2BsIns334WbMe//HK2B5awmijEKTZyGZfr4fGiYj1OoOmnb22mGA/LOvAqS9gNrK5F/zihUAUwU7lS+75JHdMXzJOiG5Ngd7DQ9HGVmfJYUXiv3PhltlUGuu8APYXzc9OG4xlQwjqjqIhBhLIJ2qbGoOk2RzIkhqvJ6FVkTyFuvi58FXX3RMN0yXDi3DEeVjZvyvzi5/K4RFwqKede2VsLnBpW2/XZ962UwLZMlK2BS5jEKHZwCGWNW9LI+CccrmJZuQkss8jtSSqyUSdR1JC+CIo5DA0CLg3vKPkOMV8Tb1u/c4NXeJ0dMPOS6hpPYZYhdLLBPu99IH1/cpwtA9gxDVFytZVhjnEGpoXjI8T07g8Z5/OI/GiNPFM02jfSNZYHH8k12KYqlbT12vMih4KTfIRWt+pIKbua9dF2pKI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(83380400001)(86362001)(8936002)(426003)(5660300002)(2906002)(186003)(8676002)(7696005)(6666004)(81166007)(47076005)(1076003)(36756003)(110136005)(2616005)(82310400003)(316002)(336012)(356005)(36860700001)(26005)(70586007)(44832011)(70206006)(4326008)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6853
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	dc8688cf-ddaf-49b9-59ff-08d97e8a301a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uATJXKJVpEIAFkO+YfGTAPPv1u+Lmz4zP3WUzFKDRs4W4ilVqwIB79Y3vEQ6CcVbwrKrfT6yemzj4pUJ6tQ4ZbWH4Vse6kctqjPaM/eP0aDv+dj+ywpSiNk4wR1GOnNBFn3qJamYf28kLQDA4MWi7a/0xq4pesDc9HxBTzjflN4KG7zbHXne/rwON13kG2lWmxcP7FeQ4RLn4r81ucj4eV2BeOIwKCK9EwpJAW0vEeQH1eLWkCc8pss0n72pWQjifCE+oAbbEcWZsfVqG/QFTHDysvufJrXhXxLkd7EH1iDllHzMyVOMfCXvSz5DSTF8cH+SibT8M1SFHPQScu8ZJZY3mJuw1wnjE1rgSpSTRTgCSo/LoZYpNtIzUYuk4dRbZ24kqKWbODe/WqanFZ/jJzx5srg4QhxbrTITGUIb+aaz3dc0/ntQtV3QvbbMmzzBYPyPQMdPlLMkC27vocw2zjngFBGpIEWYiyrDFIoBh3gT/VnZoW+Fg9q4SKyjcF6knyvUMk+gw7gKfmy2AqGSTNVurb6cXljfm53o1RdeGIguWymTN5P0T3OZsjcx+lO83dyv/f90Q1BrIR50ORdIXvBSa3PV5TZmaVq4J6JcpBum+Je0d7Ppy8LPpsQ8cdwts+GEKfEnhUUzEzGx45Q8n3RWUvpKLBPfX5HS64OmxKhRNgA0x5GkhGKdgDGGFFlbhc4MDyNP1d9Zv7SO4aL4ag==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(8936002)(110136005)(1076003)(2906002)(336012)(82310400003)(426003)(26005)(47076005)(4326008)(316002)(186003)(36860700001)(7696005)(508600001)(36756003)(6666004)(44832011)(83380400001)(86362001)(81166007)(2616005)(70206006)(70586007)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:03:55.5338
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5389e76c-62cc-46b7-f6a9-08d97e8a37bd
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3783

Xen is using acpi_numa as a switch for ACPI based NUMA. We want to
use this switch logic for other firmware based NUMA implementation,
like device tree based NUMA in follow-up patches. As Xen will never
use both ACPI and device tree based NUMA at runtime. So I rename
acpi_numa to a more generic name fw_name. This will also allow to
have the code mostly common.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c        |  6 +++---
 xen/arch/x86/setup.c       |  2 +-
 xen/arch/x86/srat.c        | 10 +++++-----
 xen/include/asm-x86/acpi.h |  2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 6bc4ade411..2ef385ae3f 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -51,11 +51,11 @@ cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly;
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
 bool numa_off;
-s8 acpi_numa = 0;
+s8 fw_numa = 0;
 
 int srat_disabled(void)
 {
-    return numa_off || acpi_numa < 0;
+    return numa_off || fw_numa < 0;
 }
 
 /*
@@ -315,7 +315,7 @@ static __init int numa_setup(const char *opt)
     else if ( !strncmp(opt,"noacpi",6) )
     {
         numa_off = false;
-        acpi_numa = -1;
+        fw_numa = -1;
     }
 #endif
     else
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b101565f14..1a2093b554 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -313,7 +313,7 @@ void srat_detect_node(int cpu)
     node_set_online(node);
     numa_set_node(cpu, node);
 
-    if ( opt_cpu_info && acpi_numa > 0 )
+    if ( opt_cpu_info && fw_numa > 0 )
         printk("CPU %d APIC %d -> Node %d\n", cpu, apicid, node);
 }
 
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 9276a52138..4921830f94 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -167,7 +167,7 @@ static __init void bad_srat(void)
 {
 	int i;
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
-	acpi_numa = -1;
+	fw_numa = -1;
 	for (i = 0; i < MAX_LOCAL_APIC; i++)
 		apicid_to_node[i] = NUMA_NO_NODE;
 	for (i = 0; i < ARRAY_SIZE(pxm2node); i++)
@@ -242,7 +242,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa)
 
 	apicid_to_node[pa->apic_id] = node;
 	numa_set_processor_nodes_parsed(node);
-	acpi_numa = 1;
+	fw_numa = 1;
 
 	if (opt_acpi_verbose)
 		printk(KERN_INFO "SRAT: PXM %u -> APIC %08x -> Node %u\n",
@@ -277,7 +277,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
 	}
 	apicid_to_node[pa->apic_id] = node;
 	numa_set_processor_nodes_parsed(node);
-	acpi_numa = 1;
+	fw_numa = 1;
 
 	if (opt_acpi_verbose)
 		printk(KERN_INFO "SRAT: PXM %u -> APIC %02x -> Node %u\n",
@@ -492,7 +492,7 @@ void __init srat_parse_regions(paddr_t addr)
 	u64 mask;
 	unsigned int i;
 
-	if (acpi_disabled || acpi_numa < 0 ||
+	if (acpi_disabled || fw_numa < 0 ||
 	    acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat))
 		return;
 
@@ -521,7 +521,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end)
 	for (i = 0; i < MAX_NUMNODES; i++)
 		cutoff_node(i, start, end);
 
-	if (acpi_numa <= 0)
+	if (fw_numa <= 0)
 		return -1;
 
 	if (!nodes_cover_memory()) {
diff --git a/xen/include/asm-x86/acpi.h b/xen/include/asm-x86/acpi.h
index 7032f3a001..83be71fec3 100644
--- a/xen/include/asm-x86/acpi.h
+++ b/xen/include/asm-x86/acpi.h
@@ -101,7 +101,7 @@ extern unsigned long acpi_wakeup_address;
 
 #define ARCH_HAS_POWER_INIT	1
 
-extern s8 acpi_numa;
+extern s8 fw_numa;
 extern int acpi_scan_nodes(u64 start, u64 end);
 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193871.345466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVs-0007Mo-Ia; Thu, 23 Sep 2021 12:07:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193871.345466; Thu, 23 Sep 2021 12:07:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNVs-0007Lt-DN; Thu, 23 Sep 2021 12:07:40 +0000
Received: by outflank-mailman (input) for mailman id 193871;
 Thu, 23 Sep 2021 12:07:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNUz-0006xV-Su
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:45 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.67]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 727f9472-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:48 +0000 (UTC)
Received: from DB3PR08CA0014.eurprd08.prod.outlook.com (2603:10a6:8::27) by
 AM5PR0802MB2404.eurprd08.prod.outlook.com (2603:10a6:203:a0::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:43 +0000
Received: from DB5EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8::4) by DB3PR08CA0014.outlook.office365.com (2603:10a6:8::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT004.mail.protection.outlook.com (10.152.20.128) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Thu, 23 Sep 2021 12:04:43 +0000
Received: from e69a35bcc9b5.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E0723BDA-A389-46C5-B2F4-4B4EABD18756.1; 
 Thu, 23 Sep 2021 12:04:36 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e69a35bcc9b5.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:36 +0000
Received: from AM6P191CA0095.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::36)
 by VI1PR08MB2830.eurprd08.prod.outlook.com (2603:10a6:802:24::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:34 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::76) by AM6P191CA0095.outlook.office365.com
 (2603:10a6:209:8a::36) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:34 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:34 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:11 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:04:10 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 727f9472-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dVwXh/yVQ83FhUNQPKNpGoOH7G0T0dTGV7cr7Z6KkRQ=;
 b=5a739+q5UVFwBvAzTxEfHdG2rFG2qWGyDigpcuvJcKI4Q69rQSbLUX0rXF/q/wGx//c5iSiM/Tfp7dWQUWh/Xl+JzDON93qXdgphrijxX4AV5y54cfpea9FXqycgN3o5QjjThz/KMJaxqqADqI8bt2B+3uF4Xq3frnkWgwraG8s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c23ec75439f285af
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=irnoecishdpmIJc8M6BMlHF9sa6NxYyCHX0Hwt3UtAj6LEtndEkwKnC7v0O1a4xzKcPmQR1pcjtKrGKIYIEzHfoH0LwF6iWVsnpYLZYWTBZEouUnlDdz17hsr1xq3Y0GeKQDDU2vqyJF0slcnLON6cVtqpnJVZ75HGT7AtOKNS95u+Y5OR6hUJTxFwDU2No/OaojsAWNN25aCFxPl+V6p0QoptrbDzVbi7/hJnihQJJrJnioRUA2ES6lDFMxqJ08fEb+GZuk2LjeUVP2StsafPuI1Qwlv2Nw9Rlf1LgHL5NE33Evqgr7sQeF26pzsvV8Pcg8OPIedrAQ3HUhlOXUFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dVwXh/yVQ83FhUNQPKNpGoOH7G0T0dTGV7cr7Z6KkRQ=;
 b=IcdXDnwp8guR14FCIRj2dr4gcChjdGbpX1xhJGEz6t2jA9jV7uyg5ZuLMauvaFwC4o/D93hF4P5x2QLPGXBgj9E+sDAWZognd2MKeCBLLso+D3NCgcUKRcfkwx5fYVbqwJrYebT1e45C7y3iVtDuHhbHk3XYd41ubBMgVILf5hCn4Hjkc2fzqFQlkyssQ9ghRo8YwEhLGIhBjkxt2GoN6ywq06nOEQrgi8YyBxOSzdsEz2KGEsBcD5NDlGEqUcme8+Z8dcYEvG0q3mJ5+eqeMH4I4u/092MmC3k5xQ15axRtl+GuPnq1rleM9D4ew0rRmRdOU1XgwjjHYuQjpZ4mHA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dVwXh/yVQ83FhUNQPKNpGoOH7G0T0dTGV7cr7Z6KkRQ=;
 b=5a739+q5UVFwBvAzTxEfHdG2rFG2qWGyDigpcuvJcKI4Q69rQSbLUX0rXF/q/wGx//c5iSiM/Tfp7dWQUWh/Xl+JzDON93qXdgphrijxX4AV5y54cfpea9FXqycgN3o5QjjThz/KMJaxqqADqI8bt2B+3uF4Xq3frnkWgwraG8s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 28/37] xen/arm: stub memory hotplug access helpers for Arm
Date: Thu, 23 Sep 2021 20:02:27 +0800
Message-ID: <20210923120236.3692135-29-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e0c6df13-2ecf-4e08-189a-08d97e8a5456
X-MS-TrafficTypeDiagnostic: VI1PR08MB2830:|AM5PR0802MB2404:
X-Microsoft-Antispam-PRVS:
	<AM5PR0802MB2404D30724CD5FDDA969C0C99EA39@AM5PR0802MB2404.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 J6cxXrmwElyUZoCL+lkHU3hHmoFvkrRAnhl+dpNeB9BFEapfgUNekwQuyxZTGq5pV0Gpi0vkur4qX8GjIjuWs114Z/y/JZKnKqUfP9aNb301FX3h97ndtjGwDOhubby4ZyD/LzccSGOsqZ37oiaf4nMAJE9rKeJwM3Ls3VzHyseh0mHQmzZIRrJWhPcsZcxLGVtoN09aceM/H1n6OuT9nHchjANTllD5uqaoroTOih+Tem7dkUcvi/6R0qIHO2Wt9iGecOi72ZCUKjQEHIdsFct3XkqPOuyJ1HL9sF4bzijcdoZKcYnTFZcjBzI6E7xpi0rJDLp8l90Zx4f+Bo3gjDx1qxtapYmeAVnuGw16n7e3zuEACNA0mxKSR04KoeKYS+i4Ud+D3SmG0OpCD3uaYj1vPolNXFO1KAYyFmnCr8ETcw7Vd/7xvD8c4PrXck7NWIj9dzo9ojUL8Q7TISvIWUaVS8Zx5j3P1ZEShpvPmae5GIQWepH+ED7Wr6lC13mpzdhYMaMV2JVeaczCDT9yWHbPNxRN+sV0SY9cVb6wd3bLBGsP9C9rfA/j9/mfQMZn5FEiyNFhsGQ3hyMuDTGbKCU0oUzy8zMnobbYAjtp5qKhbiUVne7E49U/k8vLhd4/bE6EzrHsV8FpeFyuBO04/aTjwo5F6dGOEXiZSklvPB22bual6WW79e5cHe1G+imBFlxEmwLdbEWDD/O99jm+qD1MXYE+C/9Vq5f8MiF1cR4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(426003)(6666004)(36756003)(70206006)(70586007)(110136005)(336012)(2906002)(83380400001)(4744005)(2616005)(8936002)(4326008)(356005)(26005)(47076005)(86362001)(36860700001)(5660300002)(82310400003)(186003)(44832011)(7696005)(316002)(1076003)(508600001)(81166007)(8676002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB2830
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2023d138-8e0b-4346-51ce-08d97e8a4eb7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6y64pTjOq1IQ5mYHKHXlsD3asm1LquWhbeOC7MB6x3fsvyjvVhaiJwRTZXsQQgs+cI5h+I/mzrANZ2mfWHW5TOHDrrHkM1AibF1TzF3MxlNa1uG5hTfxhpQR/qVmJl80QDX+g63cThIjdSsuGM0Mh0C3Lvz9n8jQ4erQlML7NWGAHgoIKLpK6MyC8I8p/1bepZCt/lfZ6pOlV2VwhGlXqkaNlNvx4IH1dkdEuzjZJViWRUDXfhD/LMrxSndFWgFdU7vcKJ3TJvuQ4NrHoOhiVknGEVrb470Nftoo/+Kja+yoj/zsdZ3Esk9xFDYLW5i85sfkylzHrNzAIBSyGcVOojysWy4bgl7xwoPxc68t8UFxrJAYRaaJCioEOz3UuSR5eroaG8yzPN5FR1flHxnPutDrX7zRdLLz3Y/3z0QHI1GRhDXRN5TYLSZj9ShG9JQvNqiBcB51Rm9Drbmdx3DuU7whdi23YB1u7dhQLpvCO5midt+IKq4gyXy16meCwNFb7v1iW7o+0Ix8jzZKWTxVy59tx9DgQ9u0seVwhigYCWoXYkuLqMNmFLehDNvYI5PwckW5OT7Jts0rTUmFDGrK49Jfk44WIIj3Q6uUJzWBT7M+jXMWNv62f2QuJZLG6ErSHkZEV1vH8IrFWjRqVzoUHyHbEp5YIij1U52+B2J0vCywsQKJZQjTMjhV1HKJ19I+FzKaUUPSA6D+eNd3+sHpOQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(81166007)(36756003)(7696005)(508600001)(6666004)(36860700001)(2616005)(4744005)(1076003)(110136005)(316002)(8936002)(8676002)(47076005)(4326008)(86362001)(336012)(5660300002)(83380400001)(44832011)(70586007)(70206006)(426003)(2906002)(186003)(26005)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:43.5081
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e0c6df13-2ecf-4e08-189a-08d97e8a5456
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2404

Common code in NUMA need these two helpers to access/update
memory hotplug end address. Arm has not support memory hotplug
yet. So we stub these two helpers in this patch to make NUMA
common code happy.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/include/asm-arm/mm.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index 7b5e7b7f69..fc9433165d 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -362,6 +362,16 @@ void clear_and_clean_page(struct page_info *page);
 
 unsigned int arch_get_dma_bitsize(void);
 
+static inline void mem_hotplug_update_boundary(paddr_t end)
+{
+
+}
+
+static inline paddr_t mem_hotplug_boundary(void)
+{
+    return 0;
+}
+
 #endif /*  __ARCH_ARM_MM__ */
 /*
  * Local variables:
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193891.345477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNW7-0000KE-1K; Thu, 23 Sep 2021 12:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193891.345477; Thu, 23 Sep 2021 12:07:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNW6-0000K3-U1; Thu, 23 Sep 2021 12:07:54 +0000
Received: by outflank-mailman (input) for mailman id 193891;
 Thu, 23 Sep 2021 12:07:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNUf-0006xV-SO
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:25 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.79]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6921c873-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:32 +0000 (UTC)
Received: from AM5PR0402CA0010.eurprd04.prod.outlook.com
 (2603:10a6:203:90::20) by PAXPR08MB6765.eurprd08.prod.outlook.com
 (2603:10a6:102:138::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:30 +0000
Received: from AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:90:cafe::6d) by AM5PR0402CA0010.outlook.office365.com
 (2603:10a6:203:90::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:30 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT010.mail.protection.outlook.com (10.152.16.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:30 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Thu, 23 Sep 2021 12:04:28 +0000
Received: from 84eaf1236402.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 65DBA426-175E-4FCC-8368-5804DC4F84D7.1; 
 Thu, 23 Sep 2021 12:04:15 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 84eaf1236402.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:15 +0000
Received: from AS9PR06CA0167.eurprd06.prod.outlook.com (2603:10a6:20b:45c::19)
 by DB7PR08MB3130.eurprd08.prod.outlook.com (2603:10a6:5:24::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:12 +0000
Received: from VE1EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::8c) by AS9PR06CA0167.outlook.office365.com
 (2603:10a6:20b:45c::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:12 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT023.mail.protection.outlook.com (10.152.18.133) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:12 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:48 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6921c873-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xrDysX/S2+52U1YehehJRI4UKMo45On8F4OkTenZGx4=;
 b=dPKEVqbVjLK+Dd73OjXKsf1IOfOKtm/C2gsGtefZfDG5EnuHAQv8XsgZPwgr8UDUuSZElpoej2JfimPrX1MUAVMD0yJdRUJeZrvdcZAARwQxqnG4LfnplVI3LNm5iWHdgcUEqHsJaBs3GTQQdxpbTUb4fgHxGCiK8kPkSCI9JaA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 55bcb7f56a94619d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VnmvMjXtCiJiEa6bRueK6qNZ3qUfzae3nl3YhBZrOheIKbhI0mGws/iuDSj+hIxzahP0+yP63ltrtwvb2v5V8SYJ/82caHPULuWWINghYfsAiuLCb/dGlWR+kB+cIyCo321biOL/N5pWgTbXbLqnefEsDd8CAImGCczHSjuvnTl82nyrC6hf2HzY/th1d8XUl5yHaziQA8B3nX5fUVJP98sYEY2lm+naOwQ52kxjuImL81OmIeynmjo8HtdkzZCpSLkxXq2xSuxC/YtVGxtYmDaKM96tuRqyut1vd3ANJ7scLFZOC9NtexytN7H7uq/dN9BqPvrNDZD2ACFf7M3QXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xrDysX/S2+52U1YehehJRI4UKMo45On8F4OkTenZGx4=;
 b=YLerJfRyvOjaJ9r2bvTF9bFJbEuW8Ea8M0wCXszAErRP1YeXRz8rLJZWOiBlLHwiczGDKt7fD158VYsDTYKnlyV0yzoB6aa/fJuBzoNoym/nG8g7RPOI7LhJxtfecXIDIMvdLK3JV0+wxAw1/nSZulNEra5vUhCAMPJOsFlfsGFEV9KrVvacf8oznuTF8i/pYUV1POanmH1aSNpSSGCokZNPfYZYt2nfKabHno7bHoy9BB94XDZUkxSgo8JB9FYiuyC9xssbTMfmgainF5xHEFLq4CccdZW+yRTKPkv8YmRPmroWpa3SISdpVstI4RHuUY0Z9BbvN8hm/v+CqfHYMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xrDysX/S2+52U1YehehJRI4UKMo45On8F4OkTenZGx4=;
 b=dPKEVqbVjLK+Dd73OjXKsf1IOfOKtm/C2gsGtefZfDG5EnuHAQv8XsgZPwgr8UDUuSZElpoej2JfimPrX1MUAVMD0yJdRUJeZrvdcZAARwQxqnG4LfnplVI3LNm5iWHdgcUEqHsJaBs3GTQQdxpbTUb4fgHxGCiK8kPkSCI9JaA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 18/37] xen: move NUMA common code from x86 to common
Date: Thu, 23 Sep 2021 20:02:17 +0800
Message-ID: <20210923120236.3692135-19-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1e3a1952-bb9c-4e26-fccc-08d97e8a4c7a
X-MS-TrafficTypeDiagnostic: DB7PR08MB3130:|PAXPR08MB6765:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6765FDEB3DFB03D07128ECE89EA39@PAXPR08MB6765.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:126;OLM:126;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ZDoQXjTV40+pt3K4l/xC/9iOrphtf0+TOGro6MlpfwcQSN9yw/HUl0et6vsYJ2RO8TEhCKOAGxU7Tjsfk2F/unyuxUwj0Ykii8RGyzv8Z2X2Em0Xdur9ToORb0Bms2s/awyTFH2br3t/sT85TSNByfXrTHCFNF7dhy5F84XGcbLqDy+4DM9lM0h7w7VqvtMt/tsfwVOjuixQE/rP4TZ6d+8xKQSkkOxaRYLfe28RujCrVJAWhil0zXjkYoV7dtg7XlRcoaEKpHJuilH1d6vDC1onizNBJy2HXy6y6j0IvK6v4AV3G6K5oSF6woFFtgnzgYKH+49R3lR7ECuujCkveG6igtI3J/jQfPYQkN5OOrs0yEAnr+2aO+71HvVzOBRxvJOhTqStCP0o8syY7CTwZOP5BdWMvZApyLALCReMubuKM9cBEHCYkgnTq0ftkT3QLAXKdJJzLzpwvWorJWjf+4DwC7BeKsv+/+o5pf9w1lufZ4tn1acNROMd9zcIZGi1p6TlwlAq1B3nITfcq9cOzwPNE/7LsigJfccKk9c2y6xTWfPmY12gesqBjoMH/DwIk2QxwB67u2BbN7pHO5j4rg7cn5OkgIuHoJsdG7/dvrs5GpOvKxogXQflNoyX3jvJj5dagbBa02cxkjifmjyHNpZEw6/1H1dEpNPhwuaSE36mP9hDY9K0PHTQFxYilu43pbxVUGt4VHHbMuiojoRG8qKaNwBaiGzyzRYdXrv+QS4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(7696005)(36860700001)(82310400003)(6666004)(70206006)(5660300002)(8936002)(1076003)(2906002)(36756003)(47076005)(81166007)(336012)(316002)(186003)(26005)(426003)(110136005)(86362001)(83380400001)(356005)(2616005)(30864003)(70586007)(4326008)(44832011)(8676002)(36900700001)(579004)(559001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3130
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	289611dc-1bea-4388-dcc4-08d97e8a419a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0vIdfW/KIwPYK1h+64vLB5OVUBWc82zdBTzmjKBPZkSgWXAh55TDOP9wQxLB7simV0iyiTY/N2gJr1VlyCcAxgNVEkcKoDXexfjqrP0WCKF8a6VVrVNXh6a/aul60/Ah9v0IMa6YydLFfvrRgLUN+ybti9vjppKdOljJL6vp2e//LNVHmUPPursQizFxtYZp9eR+4IZiZdRemiW+BaLxyAafEGcH0070DrqhCtLe8+t0w+Op/vDsTDr7X/bC5r3axzPcFU3BZOFV3AdVTgVo2ljC/RCUx5JI1KWCokbgPCecdVlGr8x/7YjoguhmTC1niyD049zgSYEFmIe0n0mJ6WgawpZ1qdVZq+VfnZRPhlAWDgyXw4xQlkFVOZsueqPdKj1UAbvNpWzyQ9wJU8fxIY/3z5/SccnLqMaY4WG1NDgoasDFguEa+z1UUCLYtRnAk5gpq1MGs6DCi53J3soAhUZcfYNb6++FtfGxxJRQGCErD3764lgz/5qqBOyl7ooRyi2nIx9q0/utvHyn+71nUb7W1WBETzFcoRIHV3gIuGCi1krA5ed2aOcfnzzwdlPFMArf+Lvbk2p8Bbq7YUvSMXBuGkGVC5cM2+59QoupJSPm0UxbReEKJhg5Kd/41eXSBZVyDCDg4UDZ1zDV0HgL127s7tSbGEOqG8IoG9L7PhNDN2GY9nvwCd/MYyC/LpzLBDOGimWGzMm4M9UqTMCOrA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(82310400003)(8936002)(81166007)(26005)(316002)(70206006)(47076005)(4326008)(508600001)(83380400001)(70586007)(36756003)(110136005)(6666004)(8676002)(336012)(7696005)(36860700001)(86362001)(30864003)(44832011)(186003)(1076003)(5660300002)(2906002)(2616005)(426003)(559001)(579004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:30.2651
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e3a1952-bb9c-4e26-fccc-08d97e8a4c7a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6765

Some common code has been decoupled and abstracted from x86 ACPI
based NUMA implementation. In order to make this code can be reused
by other NUMA implementation, we move this code from x86 to common
folder. And this code is gated by CONFIG_NUMA, it only can be used
by those architectures that NUMA is enabled. For those architectures
do not support NUMA, they still can implementation NUMA stub API in
asm/numa.h to make NUMA-aware components happy.

In this patch, we also remove some unused include headers.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c         | 446 +----------------------------------
 xen/arch/x86/srat.c         | 253 +-------------------
 xen/common/Makefile         |   2 +
 xen/common/numa.c           | 450 ++++++++++++++++++++++++++++++++++++
 xen/common/numa_srat.c      | 264 +++++++++++++++++++++
 xen/include/asm-x86/acpi.h  |   4 -
 xen/include/asm-x86/numa.h  |  68 ------
 xen/include/asm-x86/setup.h |   1 -
 xen/include/xen/numa.h      |  82 +++++++
 9 files changed, 802 insertions(+), 768 deletions(-)
 create mode 100644 xen/common/numa.c
 create mode 100644 xen/common/numa_srat.c

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 509d2738c0..92b6bdf7b9 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -3,24 +3,13 @@
  * Copyright 2002,2003 Andi Kleen, SuSE Labs.
  * Adapted for Xen: Ryan Harper <ryanh@us.ibm.com>
  */ 
-
-#include <xen/mm.h>
-#include <xen/string.h>
 #include <xen/init.h>
-#include <xen/ctype.h>
+#include <xen/mm.h>
 #include <xen/nodemask.h>
 #include <xen/numa.h>
-#include <xen/keyhandler.h>
-#include <xen/param.h>
-#include <xen/time.h>
-#include <xen/smp.h>
-#include <xen/pfn.h>
-#include <asm/acpi.h>
 #include <xen/sched.h>
-#include <xen/softirq.h>
 
-static int numa_setup(const char *s);
-custom_param("numa", numa_setup);
+#include <asm/acpi.h>
 
 #ifndef Dprintk
 #define Dprintk(x...)
@@ -29,300 +18,12 @@ custom_param("numa", numa_setup);
 /* from proto.h */
 #define round_up(x,y) ((((x)+(y))-1) & (~((y)-1)))
 
-struct node_data node_data[MAX_NUMNODES];
-
-/* Mapping from pdx to node id */
-int memnode_shift;
-static typeof(*memnodemap) _memnodemap[64];
-unsigned long memnodemapsize;
-u8 *memnodemap;
-
-nodeid_t cpu_to_node[NR_CPUS] __read_mostly = {
-    [0 ... NR_CPUS-1] = NUMA_NO_NODE
-};
 /*
  * Keep BIOS's CPU2node information, should not be used for memory allocaion
  */
 nodeid_t apicid_to_node[MAX_LOCAL_APIC] = {
     [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE
 };
-cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly;
-
-nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
-
-bool numa_off;
-s8 fw_numa = 0;
-
-int srat_disabled(void)
-{
-    return numa_off || fw_numa < 0;
-}
-
-/*
- * Given a shift value, try to populate memnodemap[]
- * Returns :
- * 1 if OK
- * 0 if memnodmap[] too small (of shift too small)
- * -1 if node overlap or lost ram (shift too big)
- */
-static int __init populate_memnodemap(const struct node *nodes,
-                                      int numnodes, int shift, nodeid_t *nodeids)
-{
-    unsigned long spdx, epdx;
-    int i, res = -1;
-
-    memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap));
-    for ( i = 0; i < numnodes; i++ )
-    {
-        spdx = paddr_to_pdx(nodes[i].start);
-        epdx = paddr_to_pdx(nodes[i].end - 1) + 1;
-        if ( spdx >= epdx )
-            continue;
-        if ( (epdx >> shift) >= memnodemapsize )
-            return 0;
-        do {
-            if ( memnodemap[spdx >> shift] != NUMA_NO_NODE )
-                return -1;
-
-            if ( !nodeids )
-                memnodemap[spdx >> shift] = i;
-            else
-                memnodemap[spdx >> shift] = nodeids[i];
-
-            spdx += (1UL << shift);
-        } while ( spdx < epdx );
-        res = 1;
-    }
-
-    return res;
-}
-
-static int __init allocate_cachealigned_memnodemap(void)
-{
-    unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap));
-    unsigned long mfn = mfn_x(alloc_boot_pages(size, 1));
-
-    memnodemap = mfn_to_virt(mfn);
-    mfn <<= PAGE_SHIFT;
-    size <<= PAGE_SHIFT;
-    printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n",
-           mfn, mfn + size);
-    memnodemapsize = size / sizeof(*memnodemap);
-
-    return 0;
-}
-
-/*
- * The LSB of all start and end addresses in the node map is the value of the
- * maximum possible shift.
- */
-static int __init extract_lsb_from_nodes(const struct node *nodes,
-                                         int numnodes)
-{
-    int i, nodes_used = 0;
-    unsigned long spdx, epdx;
-    unsigned long bitfield = 0, memtop = 0;
-
-    for ( i = 0; i < numnodes; i++ )
-    {
-        spdx = paddr_to_pdx(nodes[i].start);
-        epdx = paddr_to_pdx(nodes[i].end - 1) + 1;
-        if ( spdx >= epdx )
-            continue;
-        bitfield |= spdx;
-        nodes_used++;
-        if ( epdx > memtop )
-            memtop = epdx;
-    }
-    if ( nodes_used <= 1 )
-        i = BITS_PER_LONG - 1;
-    else
-        i = find_first_bit(&bitfield, sizeof(unsigned long)*8);
-    memnodemapsize = (memtop >> i) + 1;
-    return i;
-}
-
-int __init compute_hash_shift(struct node *nodes, int numnodes,
-                              nodeid_t *nodeids)
-{
-    int shift;
-
-    shift = extract_lsb_from_nodes(nodes, numnodes);
-    if ( memnodemapsize <= ARRAY_SIZE(_memnodemap) )
-        memnodemap = _memnodemap;
-    else if ( allocate_cachealigned_memnodemap() )
-        return -1;
-    printk(KERN_DEBUG "NUMA: Using %d for the hash shift.\n", shift);
-
-    if ( populate_memnodemap(nodes, numnodes, shift, nodeids) != 1 )
-    {
-        printk(KERN_INFO "Your memory is not aligned you need to "
-               "rebuild your hypervisor with a bigger NODEMAPSIZE "
-               "shift=%d\n", shift);
-        return -1;
-    }
-
-    return shift;
-}
-/* initialize NODE_DATA given nodeid and start/end */
-void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end)
-{
-    unsigned long start_pfn, end_pfn;
-
-    start_pfn = paddr_to_pfn(start);
-    end_pfn = paddr_to_pfn(end);
-
-    NODE_DATA(nodeid)->node_start_pfn = start_pfn;
-    NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn;
-
-    node_set_online(nodeid);
-} 
-
-void __init numa_init_array(void)
-{
-    int rr, i;
-
-    /* There are unfortunately some poorly designed mainboards around
-       that only connect memory to a single CPU. This breaks the 1:1 cpu->node
-       mapping. To avoid this fill in the mapping for all possible
-       CPUs, as the number of CPUs is not known yet.
-       We round robin the existing nodes. */
-    rr = first_node(node_online_map);
-    for ( i = 0; i < nr_cpu_ids; i++ )
-    {
-        if ( cpu_to_node[i] != NUMA_NO_NODE )
-            continue;
-        numa_set_node(i, rr);
-        rr = cycle_node(rr, node_online_map);
-    }
-}
-
-#ifdef CONFIG_NUMA_EMU
-static int numa_fake __initdata = 0;
-
-/* Numa emulation */
-static int __init numa_emulation(unsigned long start_pfn,
-                                 unsigned long end_pfn)
-{
-    int i;
-    struct node nodes[MAX_NUMNODES];
-    u64 sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake;
-
-    /* Kludge needed for the hash function */
-    if ( hweight64(sz) > 1 )
-    {
-        u64 x = 1;
-        while ( (x << 1) < sz )
-            x <<= 1;
-        if ( x < sz/2 )
-            printk(KERN_ERR "Numa emulation unbalanced. Complain to maintainer\n");
-        sz = x;
-    }
-
-    memset(&nodes,0,sizeof(nodes));
-    for ( i = 0; i < numa_fake; i++ )
-    {
-        nodes[i].start = pfn_to_paddr(start_pfn) + i*sz;
-        if ( i == numa_fake - 1 )
-            sz = pfn_to_paddr(end_pfn) - nodes[i].start;
-        nodes[i].end = nodes[i].start + sz;
-        printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n",
-               i,
-               nodes[i].start, nodes[i].end,
-               (nodes[i].end - nodes[i].start) >> 20);
-        node_set_online(i);
-    }
-    memnode_shift = compute_hash_shift(nodes, numa_fake, NULL);
-    if ( memnode_shift < 0 )
-    {
-        memnode_shift = 0;
-        printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\n");
-        return -1;
-    }
-    for_each_online_node ( i )
-        setup_node_bootmem(i, nodes[i].start, nodes[i].end);
-    numa_init_array();
-
-    return 0;
-}
-#endif
-
-void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
-{ 
-    int i;
-    paddr_t start, end;
-
-#ifdef CONFIG_NUMA_EMU
-    if ( numa_fake && !numa_emulation(start_pfn, end_pfn) )
-        return;
-#endif
-
-    start = pfn_to_paddr(start_pfn);
-    end = pfn_to_paddr(end_pfn);
-
-#ifdef CONFIG_NUMA
-    if ( !numa_off && !numa_scan_nodes(start, end) )
-        return;
-#endif
-
-    printk(KERN_INFO "%s\n",
-           numa_off ? "NUMA turned off" : "No NUMA configuration found");
-
-    printk(KERN_INFO "Faking a node at %016"PRIpaddr"-%016"PRIpaddr"\n",
-           start, end);
-    /* setup dummy node covering all memory */
-    memnode_shift = BITS_PER_LONG - 1;
-    memnodemap = _memnodemap;
-    /* Dummy node only uses 1 slot in reality */
-    memnodemap[0] = 0;
-    memnodemapsize = 1;
-
-    nodes_clear(node_online_map);
-    node_set_online(0);
-    for ( i = 0; i < nr_cpu_ids; i++ )
-        numa_set_node(i, 0);
-    cpumask_copy(&node_to_cpumask[0], cpumask_of(0));
-    setup_node_bootmem(0, start, end);
-}
-
-void numa_add_cpu(int cpu)
-{
-    cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]);
-} 
-
-void numa_set_node(int cpu, nodeid_t node)
-{
-    cpu_to_node[cpu] = node;
-}
-
-/* [numa=off] */
-static __init int numa_setup(const char *opt)
-{
-    if ( !strncmp(opt,"off",3) )
-        numa_off = true;
-    else if ( !strncmp(opt,"on",2) )
-        numa_off = false;
-#ifdef CONFIG_NUMA_EMU
-    else if ( !strncmp(opt, "fake=", 5) )
-    {
-        numa_off = false;
-        numa_fake = simple_strtoul(opt+5,NULL,0);
-        if ( numa_fake >= MAX_NUMNODES )
-            numa_fake = MAX_NUMNODES;
-    }
-#endif
-#ifdef CONFIG_ACPI_NUMA
-    else if ( !strncmp(opt,"noacpi",6) )
-    {
-        numa_off = false;
-        fw_numa = -1;
-    }
-#endif
-    else
-        return -EINVAL;
-
-    return 0;
-} 
 
 /*
  * Setup early cpu_to_node.
@@ -395,146 +96,3 @@ int __init arch_meminfo_get_ram_bank_range(uint32_t bank,
 
 	return 0;
 }
-
-static void dump_numa(unsigned char key)
-{
-    s_time_t now = NOW();
-    unsigned int i, j, n;
-    struct domain *d;
-    struct page_info *page;
-    unsigned int page_num_node[MAX_NUMNODES];
-    const struct vnuma_info *vnuma;
-
-    printk("'%c' pressed -> dumping numa info (now = %"PRI_stime")\n", key,
-           now);
-
-    for_each_online_node ( i )
-    {
-        paddr_t pa = pfn_to_paddr(node_start_pfn(i) + 1);
-
-        printk("NODE%u start->%lu size->%lu free->%lu\n",
-               i, node_start_pfn(i), node_spanned_pages(i),
-               avail_node_heap_pages(i));
-        /* sanity check phys_to_nid() */
-        if ( phys_to_nid(pa) != i )
-            printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n",
-                   pa, phys_to_nid(pa), i);
-    }
-
-    j = cpumask_first(&cpu_online_map);
-    n = 0;
-    for_each_online_cpu ( i )
-    {
-        if ( i != j + n || cpu_to_node[j] != cpu_to_node[i] )
-        {
-            if ( n > 1 )
-                printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]);
-            else
-                printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]);
-            j = i;
-            n = 1;
-        }
-        else
-            ++n;
-    }
-    if ( n > 1 )
-        printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]);
-    else
-        printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]);
-
-    rcu_read_lock(&domlist_read_lock);
-
-    printk("Memory location of each domain:\n");
-    for_each_domain ( d )
-    {
-        process_pending_softirqs();
-
-        printk("Domain %u (total: %u):\n", d->domain_id, domain_tot_pages(d));
-
-        for_each_online_node ( i )
-            page_num_node[i] = 0;
-
-        spin_lock(&d->page_alloc_lock);
-        page_list_for_each(page, &d->page_list)
-        {
-            i = phys_to_nid(page_to_maddr(page));
-            page_num_node[i]++;
-        }
-        spin_unlock(&d->page_alloc_lock);
-
-        for_each_online_node ( i )
-            printk("    Node %u: %u\n", i, page_num_node[i]);
-
-        if ( !read_trylock(&d->vnuma_rwlock) )
-            continue;
-
-        if ( !d->vnuma )
-        {
-            read_unlock(&d->vnuma_rwlock);
-            continue;
-        }
-
-        vnuma = d->vnuma;
-        printk("     %u vnodes, %u vcpus, guest physical layout:\n",
-               vnuma->nr_vnodes, d->max_vcpus);
-        for ( i = 0; i < vnuma->nr_vnodes; i++ )
-        {
-            unsigned int start_cpu = ~0U;
-
-            if ( vnuma->vnode_to_pnode[i] == NUMA_NO_NODE )
-                printk("       %3u: pnode ???,", i);
-            else
-                printk("       %3u: pnode %3u,", i, vnuma->vnode_to_pnode[i]);
-
-            printk(" vcpus ");
-
-            for ( j = 0; j < d->max_vcpus; j++ )
-            {
-                if ( !(j & 0x3f) )
-                    process_pending_softirqs();
-
-                if ( vnuma->vcpu_to_vnode[j] == i )
-                {
-                    if ( start_cpu == ~0U )
-                    {
-                        printk("%d", j);
-                        start_cpu = j;
-                    }
-                }
-                else if ( start_cpu != ~0U )
-                {
-                    if ( j - 1 != start_cpu )
-                        printk("-%d ", j - 1);
-                    else
-                        printk(" ");
-                    start_cpu = ~0U;
-                }
-            }
-
-            if ( start_cpu != ~0U  && start_cpu != j - 1 )
-                printk("-%d", j - 1);
-
-            printk("\n");
-
-            for ( j = 0; j < vnuma->nr_vmemranges; j++ )
-            {
-                if ( vnuma->vmemrange[j].nid == i )
-                    printk("           %016"PRIx64" - %016"PRIx64"\n",
-                           vnuma->vmemrange[j].start,
-                           vnuma->vmemrange[j].end);
-            }
-        }
-
-        read_unlock(&d->vnuma_rwlock);
-    }
-
-    rcu_read_unlock(&domlist_read_lock);
-}
-
-static __init int register_numa_trigger(void)
-{
-    register_keyhandler('u', dump_numa, "dump NUMA info", 1);
-    return 0;
-}
-__initcall(register_numa_trigger);
-
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 94bd5b34da..44517c7b62 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -10,24 +10,19 @@
  * 
  * Adapted for Xen: Ryan Harper <ryanh@us.ibm.com>
  */
-
+#include <xen/acpi.h>
 #include <xen/init.h>
 #include <xen/mm.h>
-#include <xen/inttypes.h>
 #include <xen/nodemask.h>
-#include <xen/acpi.h>
 #include <xen/numa.h>
 #include <xen/pfn.h>
+
 #include <asm/e820.h>
 #include <asm/page.h>
 #include <asm/spec_ctrl.h>
 
 static struct acpi_table_slit *__read_mostly acpi_slit;
 
-static nodemask_t memory_nodes_parsed __initdata;
-static nodemask_t processor_nodes_parsed __initdata;
-static struct node nodes[MAX_NUMNODES] __initdata;
-
 struct pxm2node {
 	unsigned pxm;
 	nodeid_t node;
@@ -37,11 +32,6 @@ static struct pxm2node __read_mostly pxm2node[MAX_NUMNODES] =
 
 static unsigned node_to_pxm(nodeid_t n);
 
-static int num_node_memblks;
-static struct node node_memblk_range[NR_NODE_MEMBLKS];
-static nodeid_t memblk_nodeid[NR_NODE_MEMBLKS];
-static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS);
-
 static inline bool node_found(unsigned idx, unsigned pxm)
 {
 	return ((pxm2node[idx].pxm == pxm) &&
@@ -104,65 +94,6 @@ nodeid_t setup_node(unsigned pxm)
 	return node;
 }
 
-void  __init numa_set_processor_nodes_parsed(nodeid_t node)
-{
-	node_set(node, processor_nodes_parsed);
-}
-
-bool __init numa_memblks_available(void)
-{
-	if (num_node_memblks < NR_NODE_MEMBLKS)
-		return true;
-
-	return false;
-}
-
-int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
-{
-	int i;
-
-	for (i = 0; i < num_node_memblks; i++) {
-		struct node *nd = &node_memblk_range[i];
-
-		if (nd->start <= start && nd->end >= end &&
-			memblk_nodeid[i] == node)
-			return 1;
-	}
-
-	return 0;
-}
-
-static __init int conflicting_memblks(paddr_t start, paddr_t end)
-{
-	int i;
-
-	for (i = 0; i < num_node_memblks; i++) {
-		struct node *nd = &node_memblk_range[i];
-		if (nd->start == nd->end)
-			continue;
-		if (nd->end > start && nd->start < end)
-			return i;
-		if (nd->end == end && nd->start == start)
-			return i;
-	}
-	return -1;
-}
-
-static __init void cutoff_node(int i, paddr_t start, paddr_t end)
-{
-	struct node *nd = &nodes[i];
-	if (nd->start < start) {
-		nd->start = start;
-		if (nd->end < nd->start)
-			nd->start = nd->end;
-	}
-	if (nd->end > end) {
-		nd->end = end;
-		if (nd->start > nd->end)
-			nd->start = nd->end;
-	}
-}
-
 __init void bad_srat(void)
 {
 	int i;
@@ -284,97 +215,6 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
 		       pxm, pa->apic_id, node);
 }
 
-/*
- * Check to see if there are other nodes within this node's range.
- * We just need to check full contains situation. Because overlaps
- * have been checked before by conflicting_memblks.
- */
-static bool __init is_node_memory_continuous(nodeid_t nid,
-    paddr_t start, paddr_t end)
-{
-	nodeid_t i;
-
-	struct node *nd = &nodes[nid];
-	for_each_node_mask(i, memory_nodes_parsed)
-	{
-		/* Skip itself */
-		if (i == nid)
-			continue;
-
-		nd = &nodes[i];
-		if (start < nd->start && nd->end < end)
-		{
-			printk(KERN_ERR
-			       "NODE %u: (%"PRIpaddr"-%"PRIpaddr") intertwine with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
-			       nid, start, end, i, nd->start, nd->end);
-			return false;
-		}
-	}
-
-	return true;
-}
-
-/* Neutral NUMA memory affinity init function for ACPI and DT */
-int __init numa_update_node_memblks(nodeid_t node,
-		paddr_t start, paddr_t size, bool hotplug)
-{
-	paddr_t end = start + size;
-	int i;
-
-	/* It is fine to add this area to the nodes data it will be used later */
-	i = conflicting_memblks(start, end);
-	if (i < 0)
-		/* everything fine */;
-	else if (memblk_nodeid[i] == node) {
-		bool mismatch = !hotplug != !test_bit(i, memblk_hotplug);
-
-		printk("%sSRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
-		       mismatch ? KERN_ERR : KERN_WARNING, node, start, end,
-		       node_memblk_range[i].start, node_memblk_range[i].end);
-		if (mismatch) {
-			return -1;
-		}
-	} else {
-		printk(KERN_ERR
-		       "SRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
-		       node, start, end, memblk_nodeid[i],
-		       node_memblk_range[i].start, node_memblk_range[i].end);
-		return -1;
-	}
-
-	if (!hotplug) {
-		struct node *nd = &nodes[node];
-
-		if (!node_test_and_set(node, memory_nodes_parsed)) {
-			nd->start = start;
-			nd->end = end;
-		} else {
-			if (start < nd->start)
-				nd->start = start;
-			if (nd->end < end)
-				nd->end = end;
-
-			if (!is_node_memory_continuous(node, nd->start, nd->end))
-				return -1;
-		}
-	}
-
-	printk(KERN_INFO "SRAT: Node %u %"PRIpaddr"-%"PRIpaddr"%s\n",
-	       node, start, end, hotplug ? " (hotplug)" : "");
-
-	node_memblk_range[num_node_memblks].start = start;
-	node_memblk_range[num_node_memblks].end = end;
-	memblk_nodeid[num_node_memblks] = node;
-	if (hotplug) {
-		__set_bit(num_node_memblks, memblk_hotplug);
-		if (end > mem_hotplug_boundary())
-			mem_hotplug_update_boundary(end);
-	}
-	num_node_memblks++;
-
-	return 0;
-}
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 void __init
 acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
@@ -419,45 +259,6 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
 		bad_srat();
 }
 
-/* Sanity check to catch more bad SRATs (they are amazingly common).
-   Make sure the PXMs cover all memory. */
-static int __init nodes_cover_memory(void)
-{
-	int i;
-	uint32_t nr_banks = arch_meminfo_get_nr_bank();
-
-	for (i = 0; i < nr_banks; i++) {
-		int j, found;
-		paddr_t start, end;
-
-		if (arch_meminfo_get_ram_bank_range(i, &start, &end))
-			continue;
-
-		do {
-			found = 0;
-			for_each_node_mask(j, memory_nodes_parsed)
-				if (start < nodes[j].end
-				    && end > nodes[j].start) {
-					if (start >= nodes[j].start) {
-						start = nodes[j].end;
-						found = 1;
-					}
-					if (end <= nodes[j].end) {
-						end = nodes[j].start;
-						found = 1;
-					}
-				}
-		} while (found && start < end);
-
-		if (start < end) {
-			printk(KERN_ERR "SRAT: No NODE for memory map range: "
-				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
-			return 0;
-		}
-	}
-	return 1;
-}
-
 void __init acpi_numa_arch_fixup(void) {}
 
 static uint64_t __initdata srat_region_mask;
@@ -511,56 +312,6 @@ void __init srat_parse_regions(paddr_t addr)
 	pfn_pdx_hole_setup(mask >> PAGE_SHIFT);
 }
 
-/* Use the information discovered above to actually set up the nodes. */
-int __init numa_scan_nodes(paddr_t start, paddr_t end)
-{
-	int i;
-	nodemask_t all_nodes_parsed;
-
-	/* First clean up the node list */
-	for (i = 0; i < MAX_NUMNODES; i++)
-		cutoff_node(i, start, end);
-
-	if (fw_numa <= 0)
-		return -1;
-
-	if (!nodes_cover_memory()) {
-		bad_srat();
-		return -1;
-	}
-
-	memnode_shift = compute_hash_shift(node_memblk_range, num_node_memblks,
-				memblk_nodeid);
-
-	if (memnode_shift < 0) {
-		printk(KERN_ERR
-		     "SRAT: No NUMA node hash function found. Contact maintainer\n");
-		bad_srat();
-		return -1;
-	}
-
-	nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed);
-
-	/* Finally register nodes */
-	for_each_node_mask(i, all_nodes_parsed)
-	{
-		paddr_t size = nodes[i].end - nodes[i].start;
-		if ( size == 0 )
-			printk(KERN_WARNING "SRAT: Node %u has no memory. "
-			       "Firmware Bug or mis-configured hardware?\n", i);
-
-		setup_node_bootmem(i, nodes[i].start, nodes[i].end);
-	}
-	for (i = 0; i < nr_cpu_ids; i++) {
-		if (cpu_to_node[i] == NUMA_NO_NODE)
-			continue;
-		if (!nodemask_test(cpu_to_node[i], &processor_nodes_parsed))
-			numa_set_node(i, NUMA_NO_NODE);
-	}
-	numa_init_array();
-	return 0;
-}
-
 static unsigned node_to_pxm(nodeid_t n)
 {
 	unsigned i;
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 54de70d422..90e5bf3efb 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -26,6 +26,8 @@ obj-$(CONFIG_MEM_ACCESS) += mem_access.o
 obj-y += memory.o
 obj-y += multicall.o
 obj-y += notifier.o
+obj-$(CONFIG_NUMA) += numa.o
+obj-$(CONFIG_NUMA) += numa_srat.o
 obj-y += page_alloc.o
 obj-$(CONFIG_HAS_PDX) += pdx.o
 obj-$(CONFIG_PERF_COUNTERS) += perfc.o
diff --git a/xen/common/numa.c b/xen/common/numa.c
new file mode 100644
index 0000000000..fc6bba3981
--- /dev/null
+++ b/xen/common/numa.c
@@ -0,0 +1,450 @@
+/* 
+ * Generic VM initialization for NUMA setups.
+ * Copyright 2002,2003 Andi Kleen, SuSE Labs.
+ * Adapted for Xen: Ryan Harper <ryanh@us.ibm.com>
+ */
+#include <xen/init.h>
+#include <xen/keyhandler.h>
+#include <xen/mm.h>
+#include <xen/nodemask.h>
+#include <xen/numa.h>
+#include <xen/param.h>
+
+#include <xen/sched.h>
+#include <xen/softirq.h>
+
+static int numa_setup(const char *s);
+custom_param("numa", numa_setup);
+
+struct node_data node_data[MAX_NUMNODES];
+
+/* Mapping from pdx to node id */
+int memnode_shift;
+static typeof(*memnodemap) _memnodemap[64];
+unsigned long memnodemapsize;
+u8 *memnodemap;
+
+nodeid_t cpu_to_node[NR_CPUS] __read_mostly = {
+    [0 ... NR_CPUS-1] = NUMA_NO_NODE
+};
+
+cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly;
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+bool numa_off;
+s8 fw_numa = 0;
+
+int srat_disabled(void)
+{
+    return numa_off || fw_numa < 0;
+}
+
+/*
+ * Given a shift value, try to populate memnodemap[]
+ * Returns :
+ * 1 if OK
+ * 0 if memnodmap[] too small (of shift too small)
+ * -1 if node overlap or lost ram (shift too big)
+ */
+static int __init populate_memnodemap(const struct node *nodes,
+                                      int numnodes, int shift, nodeid_t *nodeids)
+{
+    unsigned long spdx, epdx;
+    int i, res = -1;
+
+    memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap));
+    for ( i = 0; i < numnodes; i++ )
+    {
+        spdx = paddr_to_pdx(nodes[i].start);
+        epdx = paddr_to_pdx(nodes[i].end - 1) + 1;
+        if ( spdx >= epdx )
+            continue;
+        if ( (epdx >> shift) >= memnodemapsize )
+            return 0;
+        do {
+            if ( memnodemap[spdx >> shift] != NUMA_NO_NODE )
+                return -1;
+
+            if ( !nodeids )
+                memnodemap[spdx >> shift] = i;
+            else
+                memnodemap[spdx >> shift] = nodeids[i];
+
+            spdx += (1UL << shift);
+        } while ( spdx < epdx );
+        res = 1;
+    }
+
+    return res;
+}
+
+static int __init allocate_cachealigned_memnodemap(void)
+{
+    unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap));
+    unsigned long mfn = mfn_x(alloc_boot_pages(size, 1));
+
+    memnodemap = mfn_to_virt(mfn);
+    mfn <<= PAGE_SHIFT;
+    size <<= PAGE_SHIFT;
+    printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n",
+           mfn, mfn + size);
+    memnodemapsize = size / sizeof(*memnodemap);
+
+    return 0;
+}
+
+/*
+ * The LSB of all start and end addresses in the node map is the value of the
+ * maximum possible shift.
+ */
+static int __init extract_lsb_from_nodes(const struct node *nodes,
+                                         int numnodes)
+{
+    int i, nodes_used = 0;
+    unsigned long spdx, epdx;
+    unsigned long bitfield = 0, memtop = 0;
+
+    for ( i = 0; i < numnodes; i++ )
+    {
+        spdx = paddr_to_pdx(nodes[i].start);
+        epdx = paddr_to_pdx(nodes[i].end - 1) + 1;
+        if ( spdx >= epdx )
+            continue;
+        bitfield |= spdx;
+        nodes_used++;
+        if ( epdx > memtop )
+            memtop = epdx;
+    }
+    if ( nodes_used <= 1 )
+        i = BITS_PER_LONG - 1;
+    else
+        i = find_first_bit(&bitfield, sizeof(unsigned long)*8);
+    memnodemapsize = (memtop >> i) + 1;
+    return i;
+}
+
+int __init compute_hash_shift(struct node *nodes, int numnodes,
+                              nodeid_t *nodeids)
+{
+    int shift;
+
+    shift = extract_lsb_from_nodes(nodes, numnodes);
+    if ( memnodemapsize <= ARRAY_SIZE(_memnodemap) )
+        memnodemap = _memnodemap;
+    else if ( allocate_cachealigned_memnodemap() )
+        return -1;
+    printk(KERN_DEBUG "NUMA: Using %d for the hash shift.\n", shift);
+
+    if ( populate_memnodemap(nodes, numnodes, shift, nodeids) != 1 )
+    {
+        printk(KERN_INFO "Your memory is not aligned you need to "
+               "rebuild your hypervisor with a bigger NODEMAPSIZE "
+               "shift=%d\n", shift);
+        return -1;
+    }
+
+    return shift;
+}
+/* initialize NODE_DATA given nodeid and start/end */
+void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end)
+{
+    unsigned long start_pfn, end_pfn;
+
+    start_pfn = paddr_to_pfn(start);
+    end_pfn = paddr_to_pfn(end);
+
+    NODE_DATA(nodeid)->node_start_pfn = start_pfn;
+    NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn;
+
+    node_set_online(nodeid);
+}
+
+void __init numa_init_array(void)
+{
+    int rr, i;
+
+    /* There are unfortunately some poorly designed mainboards around
+       that only connect memory to a single CPU. This breaks the 1:1 cpu->node
+       mapping. To avoid this fill in the mapping for all possible
+       CPUs, as the number of CPUs is not known yet.
+       We round robin the existing nodes. */
+    rr = first_node(node_online_map);
+    for ( i = 0; i < nr_cpu_ids; i++ )
+    {
+        if ( cpu_to_node[i] != NUMA_NO_NODE )
+            continue;
+        numa_set_node(i, rr);
+        rr = cycle_node(rr, node_online_map);
+    }
+}
+
+#ifdef CONFIG_NUMA_EMU
+static int numa_fake __initdata = 0;
+
+/* Numa emulation */
+static int __init numa_emulation(unsigned long start_pfn,
+                                 unsigned long end_pfn)
+{
+    int i;
+    struct node nodes[MAX_NUMNODES];
+    u64 sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake;
+
+    /* Kludge needed for the hash function */
+    if ( hweight64(sz) > 1 )
+    {
+        u64 x = 1;
+        while ( (x << 1) < sz )
+            x <<= 1;
+        if ( x < sz/2 )
+            printk(KERN_ERR "Numa emulation unbalanced. Complain to maintainer\n");
+        sz = x;
+    }
+
+    memset(&nodes,0,sizeof(nodes));
+    for ( i = 0; i < numa_fake; i++ )
+    {
+        nodes[i].start = pfn_to_paddr(start_pfn) + i*sz;
+        if ( i == numa_fake - 1 )
+            sz = pfn_to_paddr(end_pfn) - nodes[i].start;
+        nodes[i].end = nodes[i].start + sz;
+        printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n",
+               i,
+               nodes[i].start, nodes[i].end,
+               (nodes[i].end - nodes[i].start) >> 20);
+        node_set_online(i);
+    }
+    memnode_shift = compute_hash_shift(nodes, numa_fake, NULL);
+    if ( memnode_shift < 0 )
+    {
+        memnode_shift = 0;
+        printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\n");
+        return -1;
+    }
+    for_each_online_node ( i )
+        setup_node_bootmem(i, nodes[i].start, nodes[i].end);
+    numa_init_array();
+
+    return 0;
+}
+#endif
+
+void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
+{
+    int i;
+    paddr_t start, end;
+
+#ifdef CONFIG_NUMA_EMU
+    if ( numa_fake && !numa_emulation(start_pfn, end_pfn) )
+        return;
+#endif
+
+    start = pfn_to_paddr(start_pfn);
+    end = pfn_to_paddr(end_pfn);
+
+#ifdef CONFIG_NUMA
+    if ( !numa_off && !numa_scan_nodes(start, end) )
+        return;
+#endif
+
+    printk(KERN_INFO "%s\n",
+           numa_off ? "NUMA turned off" : "No NUMA configuration found");
+
+    printk(KERN_INFO "Faking a node at %016"PRIpaddr"-%016"PRIpaddr"\n",
+           start, end);
+    /* setup dummy node covering all memory */
+    memnode_shift = BITS_PER_LONG - 1;
+    memnodemap = _memnodemap;
+    /* Dummy node only uses 1 slot in reality */
+    memnodemap[0] = 0;
+    memnodemapsize = 1;
+
+    nodes_clear(node_online_map);
+    node_set_online(0);
+    for ( i = 0; i < nr_cpu_ids; i++ )
+        numa_set_node(i, 0);
+    cpumask_copy(&node_to_cpumask[0], cpumask_of(0));
+    setup_node_bootmem(0, start, end);
+}
+
+void numa_add_cpu(int cpu)
+{
+    cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]);
+}
+
+void numa_set_node(int cpu, nodeid_t node)
+{
+    cpu_to_node[cpu] = node;
+}
+
+
+/* [numa=off] */
+static __init int numa_setup(const char *opt)
+{
+    if ( !strncmp(opt,"off",3) )
+        numa_off = true;
+    else if ( !strncmp(opt,"on",2) )
+        numa_off = false;
+#ifdef CONFIG_NUMA_EMU
+    else if ( !strncmp(opt, "fake=", 5) )
+    {
+        numa_off = false;
+        numa_fake = simple_strtoul(opt+5,NULL,0);
+        if ( numa_fake >= MAX_NUMNODES )
+            numa_fake = MAX_NUMNODES;
+    }
+#endif
+#ifdef CONFIG_ACPI_NUMA
+    else if ( !strncmp(opt,"noacpi",6) )
+    {
+        numa_off = false;
+        fw_numa = -1;
+    }
+#endif
+    else
+        return -EINVAL;
+
+    return 0;
+}
+
+
+static void dump_numa(unsigned char key)
+{
+    s_time_t now = NOW();
+    unsigned int i, j, n;
+    struct domain *d;
+    struct page_info *page;
+    unsigned int page_num_node[MAX_NUMNODES];
+    const struct vnuma_info *vnuma;
+
+    printk("'%c' pressed -> dumping numa info (now = %"PRI_stime")\n", key,
+           now);
+
+    for_each_online_node ( i )
+    {
+        paddr_t pa = pfn_to_paddr(node_start_pfn(i) + 1);
+
+        printk("NODE%u start->%lu size->%lu free->%lu\n",
+               i, node_start_pfn(i), node_spanned_pages(i),
+               avail_node_heap_pages(i));
+        /* sanity check phys_to_nid() */
+        if ( phys_to_nid(pa) != i )
+            printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n",
+                   pa, phys_to_nid(pa), i);
+    }
+
+    j = cpumask_first(&cpu_online_map);
+    n = 0;
+    for_each_online_cpu ( i )
+    {
+        if ( i != j + n || cpu_to_node[j] != cpu_to_node[i] )
+        {
+            if ( n > 1 )
+                printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]);
+            else
+                printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]);
+            j = i;
+            n = 1;
+        }
+        else
+            ++n;
+    }
+    if ( n > 1 )
+        printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]);
+    else
+        printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]);
+
+    rcu_read_lock(&domlist_read_lock);
+
+    printk("Memory location of each domain:\n");
+    for_each_domain ( d )
+    {
+        process_pending_softirqs();
+
+        printk("Domain %u (total: %u):\n", d->domain_id, domain_tot_pages(d));
+
+        for_each_online_node ( i )
+            page_num_node[i] = 0;
+
+        spin_lock(&d->page_alloc_lock);
+        page_list_for_each(page, &d->page_list)
+        {
+            i = phys_to_nid(page_to_maddr(page));
+            page_num_node[i]++;
+        }
+        spin_unlock(&d->page_alloc_lock);
+
+        for_each_online_node ( i )
+            printk("    Node %u: %u\n", i, page_num_node[i]);
+
+        if ( !read_trylock(&d->vnuma_rwlock) )
+            continue;
+
+        if ( !d->vnuma )
+        {
+            read_unlock(&d->vnuma_rwlock);
+            continue;
+        }
+
+        vnuma = d->vnuma;
+        printk("     %u vnodes, %u vcpus, guest physical layout:\n",
+               vnuma->nr_vnodes, d->max_vcpus);
+        for ( i = 0; i < vnuma->nr_vnodes; i++ )
+        {
+            unsigned int start_cpu = ~0U;
+
+            if ( vnuma->vnode_to_pnode[i] == NUMA_NO_NODE )
+                printk("       %3u: pnode ???,", i);
+            else
+                printk("       %3u: pnode %3u,", i, vnuma->vnode_to_pnode[i]);
+
+            printk(" vcpus ");
+
+            for ( j = 0; j < d->max_vcpus; j++ )
+            {
+                if ( !(j & 0x3f) )
+                    process_pending_softirqs();
+
+                if ( vnuma->vcpu_to_vnode[j] == i )
+                {
+                    if ( start_cpu == ~0U )
+                    {
+                        printk("%d", j);
+                        start_cpu = j;
+                    }
+                }
+                else if ( start_cpu != ~0U )
+                {
+                    if ( j - 1 != start_cpu )
+                        printk("-%d ", j - 1);
+                    else
+                        printk(" ");
+                    start_cpu = ~0U;
+                }
+            }
+
+            if ( start_cpu != ~0U  && start_cpu != j - 1 )
+                printk("-%d", j - 1);
+
+            printk("\n");
+
+            for ( j = 0; j < vnuma->nr_vmemranges; j++ )
+            {
+                if ( vnuma->vmemrange[j].nid == i )
+                    printk("           %016"PRIx64" - %016"PRIx64"\n",
+                           vnuma->vmemrange[j].start,
+                           vnuma->vmemrange[j].end);
+            }
+        }
+
+        read_unlock(&d->vnuma_rwlock);
+    }
+
+    rcu_read_unlock(&domlist_read_lock);
+}
+
+static __init int register_numa_trigger(void)
+{
+    register_keyhandler('u', dump_numa, "dump NUMA info", 1);
+    return 0;
+}
+__initcall(register_numa_trigger);
diff --git a/xen/common/numa_srat.c b/xen/common/numa_srat.c
new file mode 100644
index 0000000000..7bda2ecef6
--- /dev/null
+++ b/xen/common/numa_srat.c
@@ -0,0 +1,264 @@
+/*
+ * ACPI 3.0 based NUMA setup
+ * Copyright 2004 Andi Kleen, SuSE Labs.
+ *
+ * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
+ *
+ * Called from acpi_numa_init while reading the SRAT and SLIT tables.
+ * Assumes all memory regions belonging to a single proximity domain
+ * are in one chunk. Holes between them will be included in the node.
+ *
+ * Adapted for Xen: Ryan Harper <ryanh@us.ibm.com>
+ */
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/nodemask.h>
+#include <xen/numa.h>
+
+static nodemask_t memory_nodes_parsed __initdata;
+static nodemask_t processor_nodes_parsed __initdata;
+static struct node nodes[MAX_NUMNODES] __initdata;
+
+static int num_node_memblks;
+static struct node node_memblk_range[NR_NODE_MEMBLKS];
+static nodeid_t memblk_nodeid[NR_NODE_MEMBLKS];
+static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS);
+
+void  __init numa_set_processor_nodes_parsed(nodeid_t node)
+{
+	node_set(node, processor_nodes_parsed);
+}
+
+bool __init numa_memblks_available(void)
+{
+	if (num_node_memblks < NR_NODE_MEMBLKS)
+		return true;
+
+	return false;
+}
+
+int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
+{
+	int i;
+
+	for (i = 0; i < num_node_memblks; i++) {
+		struct node *nd = &node_memblk_range[i];
+
+		if (nd->start <= start && nd->end >= end &&
+			memblk_nodeid[i] == node)
+			return 1;
+	}
+
+	return 0;
+}
+
+static __init int conflicting_memblks(paddr_t start, paddr_t end)
+{
+	int i;
+
+	for (i = 0; i < num_node_memblks; i++) {
+		struct node *nd = &node_memblk_range[i];
+		if (nd->start == nd->end)
+			continue;
+		if (nd->end > start && nd->start < end)
+			return i;
+		if (nd->end == end && nd->start == start)
+			return i;
+	}
+	return -1;
+}
+
+static __init void cutoff_node(int i, paddr_t start, paddr_t end)
+{
+	struct node *nd = &nodes[i];
+	if (nd->start < start) {
+		nd->start = start;
+		if (nd->end < nd->start)
+			nd->start = nd->end;
+	}
+	if (nd->end > end) {
+		nd->end = end;
+		if (nd->start > nd->end)
+			nd->start = nd->end;
+	}
+}
+
+/*
+ * Check to see if there are other nodes within this node's range.
+ * We just need to check full contains situation. Because overlaps
+ * have been checked before by conflicting_memblks.
+ */
+static bool __init is_node_memory_continuous(nodeid_t nid,
+    paddr_t start, paddr_t end)
+{
+	nodeid_t i;
+
+	struct node *nd = &nodes[nid];
+	for_each_node_mask(i, memory_nodes_parsed)
+	{
+		/* Skip itself */
+		if (i == nid)
+			continue;
+
+		nd = &nodes[i];
+		if (start < nd->start && nd->end < end)
+		{
+			printk(KERN_ERR
+			       "NODE %u: (%"PRIpaddr"-%"PRIpaddr") intertwine with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
+			       nid, start, end, i, nd->start, nd->end);
+			return false;
+		}
+	}
+
+	return true;
+}
+
+/* Neutral NUMA memory affinity init function for ACPI and DT */
+int __init numa_update_node_memblks(nodeid_t node,
+		paddr_t start, paddr_t size, bool hotplug)
+{
+	paddr_t end = start + size;
+	int i;
+
+	/* It is fine to add this area to the nodes data it will be used later */
+	i = conflicting_memblks(start, end);
+	if (i < 0)
+		/* everything fine */;
+	else if (memblk_nodeid[i] == node) {
+		bool mismatch = !hotplug != !test_bit(i, memblk_hotplug);
+
+		printk("%sSRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
+		       mismatch ? KERN_ERR : KERN_WARNING, node, start, end,
+		       node_memblk_range[i].start, node_memblk_range[i].end);
+		if (mismatch) {
+			return -1;
+		}
+	} else {
+		printk(KERN_ERR
+		       "SRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
+		       node, start, end, memblk_nodeid[i],
+		       node_memblk_range[i].start, node_memblk_range[i].end);
+		return -1;
+	}
+
+	if (!hotplug) {
+		struct node *nd = &nodes[node];
+
+		if (!node_test_and_set(node, memory_nodes_parsed)) {
+			nd->start = start;
+			nd->end = end;
+		} else {
+			if (start < nd->start)
+				nd->start = start;
+			if (nd->end < end)
+				nd->end = end;
+
+			if (!is_node_memory_continuous(node, nd->start, nd->end))
+				return -1;
+		}
+	}
+
+	printk(KERN_INFO "SRAT: Node %u %"PRIpaddr"-%"PRIpaddr"%s\n",
+	       node, start, end, hotplug ? " (hotplug)" : "");
+
+	node_memblk_range[num_node_memblks].start = start;
+	node_memblk_range[num_node_memblks].end = end;
+	memblk_nodeid[num_node_memblks] = node;
+	if (hotplug) {
+		__set_bit(num_node_memblks, memblk_hotplug);
+		if (end > mem_hotplug_boundary())
+			mem_hotplug_update_boundary(end);
+	}
+	num_node_memblks++;
+
+	return 0;
+}
+
+/* Sanity check to catch more bad SRATs (they are amazingly common).
+   Make sure the PXMs cover all memory. */
+static int __init nodes_cover_memory(void)
+{
+	int i;
+	uint32_t nr_banks = arch_meminfo_get_nr_bank();
+
+	for (i = 0; i < nr_banks; i++) {
+		int j, found;
+		paddr_t start, end;
+
+		if (arch_meminfo_get_ram_bank_range(i, &start, &end))
+			continue;
+
+		do {
+			found = 0;
+			for_each_node_mask(j, memory_nodes_parsed)
+				if (start < nodes[j].end
+				    && end > nodes[j].start) {
+					if (start >= nodes[j].start) {
+						start = nodes[j].end;
+						found = 1;
+					}
+					if (end <= nodes[j].end) {
+						end = nodes[j].start;
+						found = 1;
+					}
+				}
+		} while (found && start < end);
+
+		if (start < end) {
+			printk(KERN_ERR "SRAT: No NODE for memory map range: "
+				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
+			return 0;
+		}
+	}
+	return 1;
+}
+
+/* Use the information discovered above to actually set up the nodes. */
+int __init numa_scan_nodes(paddr_t start, paddr_t end)
+{
+	int i;
+	nodemask_t all_nodes_parsed;
+
+	/* First clean up the node list */
+	for (i = 0; i < MAX_NUMNODES; i++)
+		cutoff_node(i, start, end);
+
+	if (fw_numa <= 0)
+		return -1;
+
+	if (!nodes_cover_memory()) {
+		bad_srat();
+		return -1;
+	}
+
+	memnode_shift = compute_hash_shift(node_memblk_range, num_node_memblks,
+				memblk_nodeid);
+
+	if (memnode_shift < 0) {
+		printk(KERN_ERR
+		     "SRAT: No NUMA node hash function found. Contact maintainer\n");
+		bad_srat();
+		return -1;
+	}
+
+	nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed);
+
+	/* Finally register nodes */
+	for_each_node_mask(i, all_nodes_parsed)
+	{
+		paddr_t size = nodes[i].end - nodes[i].start;
+		if ( size == 0 )
+			printk(KERN_WARNING "SRAT: Node %u has no memory. "
+			       "Firmware Bug or mis-configured hardware?\n", i);
+
+		setup_node_bootmem(i, nodes[i].start, nodes[i].end);
+	}
+	for (i = 0; i < nr_cpu_ids; i++) {
+		if (cpu_to_node[i] == NUMA_NO_NODE)
+			continue;
+		if (!nodemask_test(cpu_to_node[i], &processor_nodes_parsed))
+			numa_set_node(i, NUMA_NO_NODE);
+	}
+	numa_init_array();
+	return 0;
+}
diff --git a/xen/include/asm-x86/acpi.h b/xen/include/asm-x86/acpi.h
index 2add971072..2140461ff3 100644
--- a/xen/include/asm-x86/acpi.h
+++ b/xen/include/asm-x86/acpi.h
@@ -101,10 +101,6 @@ extern unsigned long acpi_wakeup_address;
 
 #define ARCH_HAS_POWER_INIT	1
 
-extern s8 fw_numa;
-extern int numa_scan_nodes(u64 start, u64 end);
-#define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
-
 extern struct acpi_sleep_info acpi_sinfo;
 #define acpi_video_flags bootsym(video_flags)
 struct xenpf_enter_acpi_sleep;
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index a5690a7098..cd407804c8 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -7,85 +7,17 @@ typedef u8 nodeid_t;
 
 extern int srat_rev;
 
-extern nodeid_t      cpu_to_node[NR_CPUS];
-extern cpumask_t     node_to_cpumask[];
-
-#define cpu_to_node(cpu)		(cpu_to_node[cpu])
-#define parent_node(node)		(node)
-#define node_to_first_cpu(node)  (__ffs(node_to_cpumask[node]))
-#define node_to_cpumask(node)    (node_to_cpumask[node])
-
-struct node { 
-	paddr_t start,end;
-};
-
-extern int compute_hash_shift(struct node *nodes, int numnodes,
-			      nodeid_t *nodeids);
 extern nodeid_t pxm_to_node(unsigned int pxm);
 
 #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT))
-#define VIRTUAL_BUG_ON(x) 
-
-extern void numa_add_cpu(int cpu);
-extern void numa_init_array(void);
-extern bool numa_off;
 
-
-extern int srat_disabled(void);
-extern void bad_srat(void);
-extern void numa_set_node(int cpu, nodeid_t node);
 extern nodeid_t setup_node(unsigned int pxm);
-extern void srat_detect_node(int cpu);
 
-extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end);
 extern nodeid_t apicid_to_node[];
 extern void init_cpu_to_node(void);
 
-static inline void clear_node_cpumask(int cpu)
-{
-	cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]);
-}
-
-/* Simple perfect hash to map pdx to node numbers */
-extern int memnode_shift; 
-extern unsigned long memnodemapsize;
-extern u8 *memnodemap;
-
-struct node_data {
-    unsigned long node_start_pfn;
-    unsigned long node_spanned_pages;
-};
-
-extern struct node_data node_data[];
-
-static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
-{ 
-	nodeid_t nid;
-	VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize);
-	nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift]; 
-	VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]); 
-	return nid; 
-} 
-
-#define NODE_DATA(nid)		(&(node_data[nid]))
-
-#define node_start_pfn(nid)	(NODE_DATA(nid)->node_start_pfn)
-#define node_spanned_pages(nid)	(NODE_DATA(nid)->node_spanned_pages)
-#define node_end_pfn(nid)       (NODE_DATA(nid)->node_start_pfn + \
-				 NODE_DATA(nid)->node_spanned_pages)
-
-extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
-extern bool numa_memblks_available(void);
-extern int numa_update_node_memblks(nodeid_t node,
-		paddr_t start, paddr_t size, bool hotplug);
-extern void numa_set_processor_nodes_parsed(nodeid_t node);
-
 void srat_parse_regions(paddr_t addr);
-extern u8 __node_distance(nodeid_t a, nodeid_t b);
 unsigned int arch_get_dma_bitsize(void);
 unsigned int arch_have_default_dmazone(void);
-extern uint32_t arch_meminfo_get_nr_bank(void);
-extern int arch_meminfo_get_ram_bank_range(uint32_t bank,
-    paddr_t *start, paddr_t *end);
 
 #endif
diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
index 24be46115d..63838ba2d1 100644
--- a/xen/include/asm-x86/setup.h
+++ b/xen/include/asm-x86/setup.h
@@ -17,7 +17,6 @@ void early_time_init(void);
 
 void set_nr_cpu_ids(unsigned int max_cpus);
 
-void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn);
 void arch_init_memory(void);
 void subarch_init_memory(void);
 
diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
index 52950a3150..51391a2440 100644
--- a/xen/include/xen/numa.h
+++ b/xen/include/xen/numa.h
@@ -12,10 +12,92 @@
 #define MAX_NUMNODES    1
 #endif
 
+#define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
+
 #define vcpu_to_node(v) (cpu_to_node((v)->processor))
 
 #define domain_to_node(d) \
   (((d)->vcpu != NULL && (d)->vcpu[0] != NULL) \
    ? vcpu_to_node((d)->vcpu[0]) : NUMA_NO_NODE)
 
+/* The following content can be used when NUMA feature is enabled */
+#ifdef CONFIG_NUMA
+
+extern nodeid_t      cpu_to_node[NR_CPUS];
+extern cpumask_t     node_to_cpumask[];
+
+#define cpu_to_node(cpu)		(cpu_to_node[cpu])
+#define parent_node(node)		(node)
+#define node_to_first_cpu(node)  (__ffs(node_to_cpumask[node]))
+#define node_to_cpumask(node)    (node_to_cpumask[node])
+
+struct node {
+	paddr_t start,end;
+};
+
+extern int compute_hash_shift(struct node *nodes, int numnodes,
+			      nodeid_t *nodeids);
+
+#define VIRTUAL_BUG_ON(x)
+
+extern void numa_add_cpu(int cpu);
+extern void numa_init_array(void);
+extern bool numa_off;
+extern s8 fw_numa;
+
+extern int srat_disabled(void);
+extern void srat_detect_node(int cpu);
+extern void bad_srat(void);
+
+extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn);
+extern void numa_set_node(int cpu, nodeid_t node);
+extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end);
+
+static inline void clear_node_cpumask(int cpu)
+{
+	cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]);
+}
+
+/* Simple perfect hash to map pdx to node numbers */
+extern int memnode_shift;
+extern unsigned long memnodemapsize;
+extern u8 *memnodemap;
+
+struct node_data {
+    unsigned long node_start_pfn;
+    unsigned long node_spanned_pages;
+};
+
+extern struct node_data node_data[];
+
+static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
+{
+	nodeid_t nid;
+	VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize);
+	nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift];
+	VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]);
+	return nid;
+}
+
+#define NODE_DATA(nid)		(&(node_data[nid]))
+
+#define node_start_pfn(nid)	(NODE_DATA(nid)->node_start_pfn)
+#define node_spanned_pages(nid)	(NODE_DATA(nid)->node_spanned_pages)
+#define node_end_pfn(nid)       (NODE_DATA(nid)->node_start_pfn + \
+				 NODE_DATA(nid)->node_spanned_pages)
+
+extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
+extern bool numa_memblks_available(void);
+extern int numa_update_node_memblks(nodeid_t node,
+		paddr_t start, paddr_t size, bool hotplug);
+extern void numa_set_processor_nodes_parsed(nodeid_t node);
+extern int numa_scan_nodes(u64 start, u64 end);
+
+extern u8 __node_distance(nodeid_t a, nodeid_t b);
+extern uint32_t arch_meminfo_get_nr_bank(void);
+extern int arch_meminfo_get_ram_bank_range(uint32_t bank,
+    paddr_t *start, paddr_t *end);
+
+#endif
+
 #endif /* _XEN_NUMA_H */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:07:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:07:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193893.345481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNW7-0000Pk-DU; Thu, 23 Sep 2021 12:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193893.345481; Thu, 23 Sep 2021 12:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNW7-0000Of-9x; Thu, 23 Sep 2021 12:07:55 +0000
Received: by outflank-mailman (input) for mailman id 193893;
 Thu, 23 Sep 2021 12:07:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNUa-0006xV-S6
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:20 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.74]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6adcd31b-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:36 +0000 (UTC)
Received: from AM5PR0301CA0002.eurprd03.prod.outlook.com
 (2603:10a6:206:14::15) by HE1PR0802MB2315.eurprd08.prod.outlook.com
 (2603:10a6:3:ce::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:34 +0000
Received: from AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:14:cafe::16) by AM5PR0301CA0002.outlook.office365.com
 (2603:10a6:206:14::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:34 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT003.mail.protection.outlook.com (10.152.16.149) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 12:04:32 +0000
Received: from 3124b1183ca6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E577C1D4-DFD8-4C86-913D-901855A7A0C2.1; 
 Thu, 23 Sep 2021 12:04:14 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3124b1183ca6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:14 +0000
Received: from AS9PR06CA0113.eurprd06.prod.outlook.com (2603:10a6:20b:465::30)
 by AM6PR08MB4982.eurprd08.prod.outlook.com (2603:10a6:20b:e9::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:13 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::85) by AS9PR06CA0113.outlook.office365.com
 (2603:10a6:20b:465::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:13 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:13 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:50 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6adcd31b-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZFMSX7kfVCjWvqyxLu5Vopy2J4u9k63suSpDv1EU3iI=;
 b=mn8H/P0wlD9W4Jjhm6SuLAFA+LdS15mS+hUtfQigrl9Yts9BL3i7VQeqrW1eWizJFmvif0G3GboqudQGZ8YMUO27lZCcw84IdRkzis+24AEa4MBO8jEevjLdPTsbkYtRw0A940gObTlww3VUs4jnuap5469FyB6y8q6x1rjyQHg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: bc0074091b2e4c39
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VAvzdiKhJ693OkRMHlVbdKjkCvpQqnRLT6umZKZl4gzY5GPJ4judYzgsEt87afVGs7iPOZFELC1EQgkbcsD/uUSdMZg62DL9fevfwijPi8bzmP6GKx7nTkytANjuo6T8MDbT74TcqKHlq4IQQZTj5lESsSiSXF8ZdiJDysR5X1QwPJvoeofZ+GcGhKNsFuDrbtZtL+ThNl/WibyY1DiCfa9LQ60xz/7na0dgkudxZBiVcWTuiLhcfuwJrh2IDaW1ieqTeZk5TxDvOy2gkPvgixGy5s9Ih1fjWydiiHujuSHJawJgfrJG5dqkqxVfnvHXsV6ZHtFBhMcFyRTVBPXPuQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZFMSX7kfVCjWvqyxLu5Vopy2J4u9k63suSpDv1EU3iI=;
 b=A3gipSIHHCKy+RvocsRbvcCiAKQ7Z3Fh4jfItJV+Osbx37N8YpzCedezpB8RXsKL2PY/J/CPLlNe2r/y8kF2sMiDXQ9PTDcXjlle2TumoYrkh6kOD5q32EhQ/WqVQqoVFHgH8P+HKP14kuFer+Pm8hiCHphmfNsp1IAU5lOsGhA2wJ2hkm3ljCy2o8JWAdD7K5YBNQ6shrpd2KcZmgjuc21PNvauvM7qP/GK28wLkvU2qsYhF7C5zbS1f1fNnFFrfW2rkn6z4Jx4YlIMyrUaflWtVTYpice/s9P27umo5KEo/qX9ZEGWrXLwbGDPd1PNiqcri7P6qPRhcoSMLBY96Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZFMSX7kfVCjWvqyxLu5Vopy2J4u9k63suSpDv1EU3iI=;
 b=mn8H/P0wlD9W4Jjhm6SuLAFA+LdS15mS+hUtfQigrl9Yts9BL3i7VQeqrW1eWizJFmvif0G3GboqudQGZ8YMUO27lZCcw84IdRkzis+24AEa4MBO8jEevjLdPTsbkYtRw0A940gObTlww3VUs4jnuap5469FyB6y8q6x1rjyQHg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 19/37] xen/x86: promote VIRTUAL_BUG_ON to ASSERT in
Date: Thu, 23 Sep 2021 20:02:18 +0800
Message-ID: <20210923120236.3692135-20-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8764a4c3-746d-4754-faaa-08d97e8a4e6a
X-MS-TrafficTypeDiagnostic: AM6PR08MB4982:|HE1PR0802MB2315:
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB23150F08B3FCFB201DDBA2139EA39@HE1PR0802MB2315.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 n8uxqib7f3ru9cUC2VMn0teeR+5V9e0wem/w0c57WFvwf1l/SyZJRIW0Dg/7zm53LaR/pcIZsn2Tnaff0C0cJ47u9d4MdTwPjH7H+nLY1LJ8r9gE0aZHouhvOMHJOEwSBFqXt7WjB++sdUPDhzS03liZ4V5XcrHVCLIO/1IAqmOxam1bTK6raMBcNTLQbtj6ocCi2aXfuFxdDWD6YFdhPMpY7Kt9K+dZNlpo1vGTYeLKtBVs6+d/2x8I3hLL4RGQWi4Hj4Nb6NPN11FnY6nxxD/ubQFNpDTHU6KWrqkSJXt+LHBnNtGs3Fuy7c0HhTrRIPKn9VimDTzNv/HsAjtZK+m1T3arofcjAuyjjDYLBWF06y0q160cCttwzNrWlX7/mptMwt75GcvdPlphDEq5x9XRcdr8qzLSifyRwd2T+bLBVYmY0mXcqe8bJ1x8MoWAkhg520TSVc0UjU15L53GY89g6aC8qxVFS1mhomC7T26tWEBLrpmglW/uqh948cseTv/scQso89ujPeLQPDwDGgfxZsv47Jj08kYwNIh3YqLt4IzImTJOGLSmA44yzSmMweQHRBh4vDV/CPt/NjQbLEkdrNp2FTZMQ5q+sn76gvjkoRIreYp/NTVvLy4Yyn6/w0hpDyvGUveBVJJM14euZulpflNu4X8pdVZxVlQvI4j+KjhokNSmIE0Mmm6aNvDnuJqofZKXfmOWZ0Kjky5h3QbYIPAJNftcBaZNb6fJ9CU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(1076003)(7696005)(426003)(86362001)(186003)(508600001)(36756003)(36860700001)(336012)(2616005)(47076005)(83380400001)(4326008)(82310400003)(70206006)(110136005)(70586007)(6666004)(2906002)(8936002)(81166007)(356005)(5660300002)(316002)(8676002)(44832011)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4982
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	95e6c592-b26b-4360-cead-08d97e8a4241
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6S0QU/gT8VHvkWiuYcfU76N1/0d2aYlyMVvOKqFKbPdwQBE6TpzSTYBTtklh1EIm19zFU4x5pjuC1m2rMTCxkjC8ZKL/bkdCaVkBKnAm7KF2+gBBqhg6VyfYCMxY0ncBjGrvWjbT21Y4A8gqsie/+dGsmybFAgmTy0Wc6aABgwOJhzwUdNYbn/cE6eyJBPILtuq/1I4zvLrWUO0DFqhovwyV8xoUdhTUor/0G2SoW3aSt/vKwK38WvCVZJfFx8kfspTv+68oGbt1Q7ROaggcg+R3PklA3+YrM3N1tCrleHHf8kJGF1+hTsfHhQrGC/xYt1lQB8y9yaRids/S/fX1Yq4xuBlVj2DoCnPxvSa/ukXVsHDW+paMWDav7wedjCNJHrie5Ta7TR7FoLxtV9XHUFqYxEvcNU2dh2pwgbwhgip9xFlfyNnfQqCm+cBBNuEYbc3tc6daBs6HVzUoem6M37mzY+TEeexY+rnOe+p9PfoS05EqOiZ5LD+7E2Y8yC8M/PIZqkbX0s9+z6oHCyh46jRKfSp6Bf5WWgMsOmWtkjs366KeSBPUo2I+5Onb46grMqaeeyvsZl9jiGabLVrj8tYuJ1oFqePhfzExHKWmOaBzMk6eD1j0CBrKxz2/NQ0K7I96wlVnPVfaWRqeYFGULIyLYohUzlHRHYdU9e0D/vp3uv4IQ4iRHXxgrS02tvwK9kpTE0Cxr++OBN0eamgpjA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(186003)(36756003)(7696005)(26005)(8676002)(6666004)(82310400003)(4326008)(44832011)(336012)(426003)(83380400001)(2906002)(86362001)(508600001)(81166007)(1076003)(70586007)(5660300002)(36860700001)(2616005)(8936002)(316002)(70206006)(47076005)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:33.5327
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8764a4c3-746d-4754-faaa-08d97e8a4e6a
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2315

VIRTUAL_BUG_ON that is using in phys_to_nid is an empty macro. This
results in two lines of error-checking code in phys_to_nid are not
actually working. It also covers up two compilation errors:
1. error: ‘MAX_NUMNODES’ undeclared (first use in this function).
   This is because MAX_NUMNODES is defined in xen/numa.h.
   But asm/numa.h is a dependent file of xen/numa.h, we can't
   include xen/numa.h in asm/numa.h. This error has been fixed
   after we move phys_to_nid to xen/numa.h.
2. error: wrong type argument to unary exclamation mark.
   This is becuase, the error-checking code contains !node_data[nid].
   But node_data is a data structure variable, it's not a pointer.

So, in this patch, we use ASSERT in VIRTUAL_BUG_ON to enable the two
lines of error-checking code. And fix the the left compilation errors
by replacing !node_data[nid] to !node_data[nid].node_spanned_pages.

Because when node_spanned_pages is 0, this node has no memory.
numa_scan_node will print warning message for such kind of nodes:
"Firmware Bug or mis-configured hardware?". Even Xen allows to online
such kind of nodes. I still think it's impossible for phys_to_nid to
return a no memory node for a physical address.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/include/xen/numa.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
index 51391a2440..1978e2be1b 100644
--- a/xen/include/xen/numa.h
+++ b/xen/include/xen/numa.h
@@ -38,7 +38,7 @@ struct node {
 extern int compute_hash_shift(struct node *nodes, int numnodes,
 			      nodeid_t *nodeids);
 
-#define VIRTUAL_BUG_ON(x)
+#define VIRTUAL_BUG_ON(x) ASSERT(!(x))
 
 extern void numa_add_cpu(int cpu);
 extern void numa_init_array(void);
@@ -75,7 +75,7 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
 	nodeid_t nid;
 	VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize);
 	nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift];
-	VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]);
+	VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid].node_spanned_pages);
 	return nid;
 }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:08:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:08:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193902.345499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNWG-0001Pb-5Y; Thu, 23 Sep 2021 12:08:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193902.345499; Thu, 23 Sep 2021 12:08:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNWG-0001PJ-0r; Thu, 23 Sep 2021 12:08:04 +0000
Received: by outflank-mailman (input) for mailman id 193902;
 Thu, 23 Sep 2021 12:08:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNUV-0006xV-S3
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:06:15 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6aa254d0-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:35 +0000 (UTC)
Received: from AS9PR06CA0215.eurprd06.prod.outlook.com (2603:10a6:20b:45e::22)
 by AM7PR08MB5512.eurprd08.prod.outlook.com (2603:10a6:20b:de::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:33 +0000
Received: from VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45e:cafe::ee) by AS9PR06CA0215.outlook.office365.com
 (2603:10a6:20b:45e::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT004.mail.protection.outlook.com (10.152.18.106) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Thu, 23 Sep 2021 12:04:31 +0000
Received: from 8bbf188dc31c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 DB224C83-4C73-46AF-AA37-8CA59E27886A.1; 
 Thu, 23 Sep 2021 12:04:13 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8bbf188dc31c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:13 +0000
Received: from AS9PR06CA0160.eurprd06.prod.outlook.com (2603:10a6:20b:45c::23)
 by AM6PR08MB3127.eurprd08.prod.outlook.com (2603:10a6:209:43::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:11 +0000
Received: from VE1EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::7b) by AS9PR06CA0160.outlook.office365.com
 (2603:10a6:20b:45c::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:11 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT023.mail.protection.outlook.com (10.152.18.133) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:11 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:44 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:42 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6aa254d0-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nRu3DqWp6Kxy08JOeoeKkmzqLqGRX3WyhcKxeUsjENI=;
 b=f3+QNwG5gv8tBCRPDeU/ScmG2COmblcHwAtLsafJ+qPRKDWUjI3iRZlSF8venrqt5YhMk9nw1vh5gWbtZTDZvF0LKPdNsgDL2asdhEZXH51SNw3/uHH2v3Lcnk8mDDjxGt2eJ4X2S1Y8L2NX0HmTugCf7nnbp1FeMBf5ldyjA0U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d27630bc127e496c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZsWc91N7wotNCfQYJQSblyn5soI+l19yZcCcqtgbYNHZy7ujJeSUgm0ZKWXHpXafMB1Tu7xyUaduStWf3//FHKuMcZcV+5yaBvdnQJn1lj/a1yGdPF4KyuCXjk8RNI0yzJ5sCMbjmmwYwuT1MXubVwl7c3Xy/9Dbx+VW75/bCQcDe1MtMvRZ6bTFb96Bfy0HpKuzjbdqx/pHkcKVG2c/+NytDSbE531G8MvWhJBGMjm4B0PWXdyUEDjwkuFyjg99jd/kt8eb51dxJ1zT4CQeaXmVIPzVJwSTKL3zB/StCdxOSxdf8FqHA6XOlZqexMMBA8A2y/tYwXuVmRIlchGMSA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nRu3DqWp6Kxy08JOeoeKkmzqLqGRX3WyhcKxeUsjENI=;
 b=hPP2BO4ubKHJgr/aDkxkxUvexH59T69iEU+N03XeyeXoBtmkVCq1wAeemZS3cHgWy9jTcFy+Y2Z0jf4M3nmCSEPXYfd+aBBd6SJGgo3m/jWtjyCeCe6OclZZ03fkabyRkZ1+U0WnBS2adDVbNp8qX9jm3t2GLZh+fCrTFO1RRB98gXm4tvPCLan/l6o2gq3itGncsiYGGXHB/ZZyHFEHJV9tISvX8/at/cZ6HdJ2w11kHizz5hHMqrsBmaKrFs+jcf909ItK9E/nneneZPfYH9O4WjEm/g/Bxmj9quAMnksNnDUzK2fgxbwo/Zm7TjwrGS4tw6Zlog/dg7aiZfCG6A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nRu3DqWp6Kxy08JOeoeKkmzqLqGRX3WyhcKxeUsjENI=;
 b=f3+QNwG5gv8tBCRPDeU/ScmG2COmblcHwAtLsafJ+qPRKDWUjI3iRZlSF8venrqt5YhMk9nw1vh5gWbtZTDZvF0LKPdNsgDL2asdhEZXH51SNw3/uHH2v3Lcnk8mDDjxGt2eJ4X2S1Y8L2NX0HmTugCf7nnbp1FeMBf5ldyjA0U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 16/37] xen/x86: export srat_bad to external
Date: Thu, 23 Sep 2021 20:02:15 +0800
Message-ID: <20210923120236.3692135-17-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b117b6a4-261e-42c5-cf10-08d97e8a4e22
X-MS-TrafficTypeDiagnostic: AM6PR08MB3127:|AM7PR08MB5512:
X-Microsoft-Antispam-PRVS:
	<AM7PR08MB55128982062A610673F70C949EA39@AM7PR08MB5512.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4125;OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Tl4oJhYkJoAEc85m2rxUHeB/01EDB3APB/lLhCx2y3V6pcew5lf6k6356OL/dsQbcAY6G1cPn0NqyOplt2qfiXNaRSS6NijqHyguBuZU3pc8ydjdm8/AIBtAPt42lw3B5vSlWFXD0GO+Rif9yxDqhAiFT44oAiX1zbdOsfUlzymq9rYLKTAUAGUkwmtZdbSVcNgM3G28xLGOKceHXgewpEpahoTW07xM/CIMaI+3RyUJ0QKDucauw2uGnHXQHm701Jll9vJFQyX+qmgf0YpPoISGWEDqErOssi2hXlPqqjna2ks8bhK1HqPPL8ZNVPoCHWKGNCMqZVYPyWFRGjUV5QLyiu5pG+r4efKPIUEpWCkXh69q7Sbsg1w2lGjmU4Cllmu6sZ6m3mD+sDeciK22aU3O2k7nR7ektzUjkINSkQEjQJ6epa3XroDVORtRXFSiNQVabbQtmSbN+jZdk1sQSiZ037or4b8o7vHWWgCPYHUwprU4mOzf9HcOsNK829AI87MNs2AclN6WQFz2PjjFU/uZvwHo622fhvLs9crbdr6SoRVuJtGT23xuPks/8BlO/mjkMCmH528WeyB5cOKKR9pG6m/vQ42wvZrZ4klonbWY8Og3yW+Fe4v9U9w/d04XKR6B9/TSmmPQK499QPcPmjl/zCmRRlqfxWw6Ou2Qla3fGwD8I+posZP5mS/+D3vitbybfiITUZraDRoFD9rj8jXNffcaKNPjJcUwRLEg0eQ=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(336012)(316002)(4326008)(1076003)(82310400003)(186003)(36860700001)(8936002)(86362001)(70586007)(26005)(5660300002)(70206006)(426003)(2616005)(44832011)(81166007)(7696005)(8676002)(110136005)(83380400001)(6666004)(36756003)(356005)(47076005)(2906002)(508600001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3127
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6bd857c0-879b-4429-e87f-08d97e8a4157
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rh1Z+JkeW468ZBycuiSBV6I+KFI68EmSkYxPYB5bHxA9Z6BqqspB61XQQRua1VDBb+sR+tEpI1ZeIh9rrgn4FDWm5QBK8H7/M2EKzhgC4Sp/oxxaaj5qqPo/3s6vkbPRIog2XIjOzhr1CoSK/ezecn+ZC52kplUa+TpbNluiKiRJuMIgm2DCL3A4qhmHhhr0qUeI4OnOD0sxzB/IELH9/fn0IOCLZcdTnWeq+zXGHcF3CeaTk3QzL5LAVDxYN+8tvR2IjwrBrouA3gObmo8HSRbc+pW8nz8tTar7vc8xyba9xNVtwQOuofzEbuh5zl6VkKSEnMl8dAHlCliP/x9oscFaXrndZ2+EA1dChesCCOdfh88uDijBNuzp8kuSXihgEK5x76LYYKYdL8g+en9I1lEr1wDoku8bgdtMhgB7uMUcPurrHNyTYJTqednIIDxL7HIChBVfN3AaLsFmJy6elCGmSIt7+WjvCVNTFIMBjsukjiYas/5MUFI/jhSEFgp+OzgIDpNMJTkwtqe8pidhDfMVctsYxER6G9iZdqKw5F9tcdKn+KZiKbz2RZLTEm3XtaNDS2o23sQ6EtdMHO4fUKaQtie5BoDIJPg3j438SwCuUUL/KfHHqUZ2RWvcJ5CoETPS8eoAo7HbVeI5hJhIf00RYpjjBxNygb6jUUIjnkyJeB2X69iPzQxbNKuHVYWDR1OoGfYMnExSFR+F5+BJdQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(82310400003)(1076003)(36860700001)(508600001)(8936002)(70586007)(7696005)(70206006)(426003)(316002)(5660300002)(86362001)(110136005)(4326008)(6666004)(336012)(2616005)(81166007)(2906002)(44832011)(26005)(186003)(83380400001)(8676002)(36756003)(47076005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:33.0057
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b117b6a4-261e-42c5-cf10-08d97e8a4e22
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5512

srat_bad is used when NUMA initialization code scan SRAT failed.
It will turn fw_numa to disabled status. Its implementation depends
on NUMA implementation. We want every NUMA implementation to provide
this function for common initialization code.

In this patch, we export srat_bad to external. This will allow to
have the code mostly common.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/srat.c        | 2 +-
 xen/include/asm-x86/numa.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 0b8b0b0c95..94bd5b34da 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -163,7 +163,7 @@ static __init void cutoff_node(int i, paddr_t start, paddr_t end)
 	}
 }
 
-static __init void bad_srat(void)
+__init void bad_srat(void)
 {
 	int i;
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 295f875a51..a5690a7098 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -32,6 +32,7 @@ extern bool numa_off;
 
 
 extern int srat_disabled(void);
+extern void bad_srat(void);
 extern void numa_set_node(int cpu, nodeid_t node);
 extern nodeid_t setup_node(unsigned int pxm);
 extern void srat_detect_node(int cpu);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:15:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193915.345510 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNdU-0003se-VH; Thu, 23 Sep 2021 12:15:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193915.345510; Thu, 23 Sep 2021 12:15:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNdU-0003sX-SA; Thu, 23 Sep 2021 12:15:32 +0000
Received: by outflank-mailman (input) for mailman id 193915;
 Thu, 23 Sep 2021 12:15:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTNdT-0003sR-Bq
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:15:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2e7a39f0-6b60-4174-b36c-92995c46e266;
 Thu, 23 Sep 2021 12:15:29 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-vjnw-j-GPOyLHCbusI4Caw-1; Thu, 23 Sep 2021 14:15:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:15:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 12:15:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0050.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Thu, 23 Sep 2021 12:15:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e7a39f0-6b60-4174-b36c-92995c46e266
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632399328;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WdI8tDnzWG98SmRC9bSEqoNO4HHkzvd5C8Nm5429JL4=;
	b=EeUdDlcAvYJXMflCQfJEmL1lutT4VOI6JrSdhJZo7v9MlIJHjffUFKp36TnYNtz2KPQMtr
	1P9CkgVW9jAaAwKC/1T82R6k1zVWJUEGWdn/RdsPeirQiTNEIeE7h+ZNVlrqz+cYHN/8z4
	BOZG5qgv5y9/pQ3Njoycu5b/tvIDpCk=
X-MC-Unique: vjnw-j-GPOyLHCbusI4Caw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iACBpTn7+EAyj9VpDsNsuhmci9zCGAycBl8yGq30EBcFEzDkZxlEqqOvlsszL1pB+c0r88MCDAOOeqlAqkX2bNDNn2Hd3HITgWL0v5+BKokMhZUj3T+0KFzEh6V1PwzZgHzhRlSGrm0p+F+jiUNc7IHcUJmaL79PsdrEFpdrtTDJgl96TpblqGTAm35Dg1XBvHFR+4HHtuEE3Fiu1pgB4mySAF+WscMfZNRAfya4dthQGz0LL7GA6Lf6z3AnYBK+CFGfIDwAoAQuz91xDhD+q+1/RBiDM3q9AeyRadJ7+cm/t8XUwRQwjvNkUQhgJa+Tbvq7DJbPWxE5dvFq+osrAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ExwpCERfHVHuyBQ4YOWGVqCv6sGKRn/7ZIEZKUo9moE=;
 b=Wtoabq85ED9akWW9+8pgCXbs2+eKXoTp2mOMCJWX3vpxN6RM6Fok5NtjciEfAyChAL3rN3mMeafycCuAhL09l3liCVjcU0ETeISFdV0QWYav5Xk4g/WdZBNsNlkfpLJNjT0DoZqjxtdAq8s1nXsjez/ncFHlsK8gPRUgczUz5SV5fyemVaQlqw6nhzr7KTGS0jPBTnibm+H9ma6pFuAuAWVMlfbzjxH13vwWwj4oj+KQUQp5dRi5CkPlsK7jC27IoXCxF8Ipv3v96ucxHeXAZ5INL+Rcx8ptPPwyyacZbFj3+j0zPpTph1Lzt0WS/V2VpkgdSXx7ffvm1wsvfnqedg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
 <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
 <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <33abd731-7c2e-8c9b-963a-fd446a21e87a@suse.com>
Date: Thu, 23 Sep 2021 14:15:25 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0050.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a065442c-3f92-4133-deb2-08d97e8bd359
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7037D952FF2487A3E8BB3FD0B3A39@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	f6BPVM1vKtOu5NAeMEUmO3ffRI1IW9lheKoMITdmlbUsQgfMjdK/eKgfylVs0ko1U+yiya7TbkmcFbf+0oMETchtP0LTSuqCNuhNOiZHy+EaXayh7wBiBOgIkJ2cKW7fgJgIGSs2U06OGZI1QpGO43mILrCKcFfkXoOAgy5RUxZuQO4an9uoS6cIhd7/poJDXBmHTQsueIcuup2/YA0gK7tOooSbQ8mHpyBAKcylzVZzTtjhdLvP7z8HVR+FDrQSJM4QuK9r922ZFi+kA/bdi/inVNX0UpTZviqHxQn45tslHBvHtJK0HDgQRZqBmFBIFXZleY6fHLQ26jMrQ2SVwgKYen994EKVSQtbrwKgzZTfLeafNAXaCLIX4jOxy0QNUNZh7yX8oQI2stRByw4bjg9Lhk5PRpe0Llo+4qiXGZWFK/GcBg2TCTFjUo96TMDTwhICQRVOgF/t03Dppj0hl+cy1qcUFDAd4lGukux5ch6CuHGkUEqSojo8kOhJQtlBWfodKtFG+7UCRq29lzSYeiM71QqWPdXcOXmheyrA1SHZU2X46s41X2bbUm/XvqYdWd9iFRAaOHD8lhSM1Cvnn/Urr3NupiFpy0CVs7HsmGFX6h31HmzDAOTOwaT10d9Izyg7pPVVMawR/vHQcragrmvkXAHyEHWM3skK1zCwLI+eb1FFoqrmjqYaFo3YM7GJXhCHbYPDCO6vjIY+MaU0V5N4fyywUze1GfYmdD/Z43A=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(2906002)(508600001)(38100700002)(53546011)(5660300002)(31696002)(66476007)(16576012)(316002)(186003)(66946007)(54906003)(83380400001)(31686004)(8676002)(86362001)(66556008)(6486002)(36756003)(956004)(26005)(2616005)(6916009)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?5ubs4BUwoINKf1Hk35Hp6G0LqHetnykMacCGrLZxXZKJ55WKcQiYDU/op7ZZ?=
 =?us-ascii?Q?qzw4Gu6UdbgynQicha4rh3FkQF0+xXQh22btUKDlfFpeYl+g8DM6QwEQ3oXK?=
 =?us-ascii?Q?mnlrMI1R2313SrW4Mj2rYJrPrIGp/249wHG2780A6NqxGtxXQbb46mjPmlqO?=
 =?us-ascii?Q?W0lhxLNKHvGMWYJxxRUcxJvlUZ3mvuNo9+Jdge5prqm12Z/vG8/iPoE05d4v?=
 =?us-ascii?Q?0NHiNm2q1AV5pyvUSKokiucEC30aRJe6ZX7oQ5NIkx8ht7i4aiXxcIBMmQnN?=
 =?us-ascii?Q?RUr6GPUeqz93o77iC/DgS1th0FllhK2c2cEWWTwJq0rPmmYfNUJd0uNqKPQL?=
 =?us-ascii?Q?GFz8F3BLVojqWsv63qKjJevESRPtqN7FkfOZDhSwEafpDp9SPnsbfdWiLzPP?=
 =?us-ascii?Q?NcDiF9tCPjRfaW7CZ22NEFpiUOhCpoUh8KMuMX7SCzf/FfA4/AehINrp08os?=
 =?us-ascii?Q?BUjJbRVtmb0Q/E7CLXsdQO7PUKPO9RAEDLGYRHcCNAQKaimHaKVRoQ27HUDE?=
 =?us-ascii?Q?0mmcPr+nFC2LtjC8WShz1QzvBMcU/XEGpN1QUXDtE13S87+fzkNlFkE4kbOg?=
 =?us-ascii?Q?ZUi/ufwaJVtu8vcBOsYNt9ggyyxqUz3Tmc4IqQkrFTmNWOdd6d9Ri8Cwf4ax?=
 =?us-ascii?Q?lzskHrDU+nUKBiI7s2mho0hZNy2JzwaU+5YO9O4NsAhPN6UEStr2U+y6PCO/?=
 =?us-ascii?Q?4XDVkkDahhZSN7RnU416S/7Hp4B23m5zMb+1TN7SwyK+/6/e/R5Rq62wjzAP?=
 =?us-ascii?Q?MUjTHQH8XsmpUMpil9dCzxNrQBp3JK9FyE2odi4lrjCBR55U4V4F1gSWxQ7X?=
 =?us-ascii?Q?U6ZocWDpk/htee7vk3XtyE5pUGJtv9ktNflS6e2auW9HkaEp35HyU2ejQ2zI?=
 =?us-ascii?Q?w54o/FylEMQVAMa/I/nGDr1FalrM9W7jv3+l6EkZbOSK1usjrMrdkJFJ08E7?=
 =?us-ascii?Q?m2fq9r59Y0v3APIC3itTTaGRlueBVFG1BnXvmghzR4Fu2hW4nO1uPsWKnCRN?=
 =?us-ascii?Q?eXfxJh7ulDt1tExCNW2CP5ugnkZd3Y3eiqOEv5Dmc2r7ApLyiqx+lwSF4dMK?=
 =?us-ascii?Q?EMOJJCM2FAKwwrTSU3IDTNfAHMzJz6YrZcgd+msBHL+zCnSiH7CXv9AWbtWs?=
 =?us-ascii?Q?IdKc40isZgkUhfIwKwxhn1bEgFy1uAP42cZWvjIBfgrJOPN4CBqqqdf6P+Lm?=
 =?us-ascii?Q?ECjIutAOux+DxpvgK2Inu3lIkac2DrgHhhgcyFoW0RZwovlYhOzCoTiUHoCa?=
 =?us-ascii?Q?lf1ELRDyhXy8TCKSvhSeJqNzj/Nn4hBFiEFtKsnCqBo9RSbR3jqNVA9XusSE?=
 =?us-ascii?Q?G2xF/oLDO1BHLO5jNrU4nHNL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a065442c-3f92-4133-deb2-08d97e8bd359
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:15:26.2121
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OMMI8c8QJvlkaARM8hbTXZyV07u6exhONw/Lz2fZuKMN+BUoUfpFt8oTwsLJ73cDRaQsIfmIIdmp+6mTu6dNuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

On 23.09.2021 13:54, Roger Pau Monn=C3=A9 wrote:
> On Thu, Sep 23, 2021 at 01:32:52PM +0200, Jan Beulich wrote:
>> On 23.09.2021 13:10, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
>>>> --- a/xen/arch/x86/mm/p2m.c
>>>> +++ b/xen/arch/x86/mm/p2m.c
>>>> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
>>>>              return -EPERM;
>>>>          }
>>>> =20
>>>> +        /*
>>>> +         * Gross bodge, to go away again rather sooner than later:
>>>> +         *
>>>> +         * For MMIO allow access permissions to accumulate, but only =
for Dom0.
>>>> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() di=
ffer in
>>>> +         * the way they specify "access", this will allow the ultimat=
e result
>>>> +         * to be independent of the sequence of operations.
>>>
>>> Wouldn't it be better to 'fix' those operations so that they can work
>>> together?
>>
>> Yes, but then we should do this properly by removing all abuse of
>> p2m_access_t.
>=20
> I'm not sure I follow what that abuse is.

As was clarified, p2m_access_t should be solely used by e.g.
introspection agents, who are then the entity responsible for
resolving the resulting faults. Any other uses (to e.g. merely
restrict permissions for other reasons) are really abuses. That
is, if e.g. a r/o direct-MMIO mapping is needed, this should not
be expressed as (p2m_mmio_direct, p2m_access_r) tuple, but would
require a distinct p2m_mmio_direct_ro type.

>>> It's my understanding that set_identity_p2m_entry is the one that has
>>> strong requirements regarding the access permissions, as on AMD ACPI
>>> tables can specify how should regions be mapped.
>>>
>>> A possible solution might be to make set_mmio_p2m_entry more tolerant
>>> to how present mappings are handled. For once that function doesn't
>>> let callers specify access permissions, so I would consider that if a
>>> mapping is present on the gfn and it already points to the requested
>>> mfn no error should be returned to the caller. At the end the 'default
>>> access' for that gfn -> mfn relation is the one established by
>>> set_identity_p2m_entry and shouldn't be subject to the p2m default
>>> access.
>>
>> I think further reducing access is in theory supposed to be possible.
>=20
> Couldn't an access reduction introduce issues, kind of similar to what
> would happen if the regions are unmapped? (and that XSA-378 addressed)
>=20
> I think whatever permissions set_identity_p2m_entry sets should be
> mandatory ones, and no changes should be allowed. That would maybe
> require introducing a new p2m type (p2m_mmio_mandatory) in order to
> differentiate firmware mandatory MMIO mappings from the rest.

Hmm, indeed. No deviation in either direction should be permitted.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:15:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:15:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193916.345520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNda-0004Ae-AE; Thu, 23 Sep 2021 12:15:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193916.345520; Thu, 23 Sep 2021 12:15:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNda-0004AT-6i; Thu, 23 Sep 2021 12:15:38 +0000
Received: by outflank-mailman (input) for mailman id 193916;
 Thu, 23 Sep 2021 12:15:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jMA4=ON=aepfle.de=olaf@srs-us1.protection.inumbo.net>)
 id 1mTNdY-0003sR-8Y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:15:36 +0000
Received: from mo4-p00-ob.smtp.rzone.de (unknown [81.169.146.163])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1006d056-325c-4f41-88dd-63d293eecfb3;
 Thu, 23 Sep 2021 12:15:31 +0000 (UTC)
Received: from sender by smtp.strato.de (RZmta 47.33.8 AUTH)
 with ESMTPSA id j06d2fx8NCFTE2t
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits))
 (Client did not present a certificate);
 Thu, 23 Sep 2021 14:15:29 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1006d056-325c-4f41-88dd-63d293eecfb3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1632399329;
    s=strato-dkim-0002; d=aepfle.de;
    h=References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Cc:Date:
    From:Subject:Sender;
    bh=4fRQpmQT/5HywAU4pn6RVOIOFK8DO6QtaQdjaI1nwjI=;
    b=buYvwG7bJGjBD/LmkrwAA0aEWZEZWKDMStrUtRsOdn8g/oqmhd/nbPQCZsFlJXSqCz
    lJbirgJP3Eo+jj2biqRqoHbbhChqSWQWWjhfoU6mEqLFzx+nzMtwBO29LT4FyPzGWwSs
    +/Ysxw0xCxyTp45o7wvdbKMQmJonCyC9lbbxIQmjNPjMpikzFqNUpJ1XJp6Lbpv1S2wf
    NWXmEphRW5yC8J/j7W3zKk8LyqfU0eYnYtvjVjYA5+Xe2eujSoCpz8K00qKbW9yfHiRa
    t/VoEzAln9Ytm9oKYTSh2CaoONg4cVlJMW2Jc5ku4HMqb1iUChRj1igyfdg4u9ySnzWh
    RwHg==
Authentication-Results: strato.com;
    dkim=none
X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QLpd5ylWvMDX3y/OuD5rXVisEoBx6KYk3sDSS2ECuxRNydJqDgTcTmRadqe62a"
X-RZG-CLASS-ID: mo00
Date: Thu, 23 Sep 2021 14:15:18 +0200
From: Olaf Hering <olaf@aepfle.de>
To: James Dingwall <james-xen@dingwall.me.uk>
Cc: xen-devel@lists.xenproject.org
Subject: Re: domain never exits after using 'xl save'
Message-ID: <20210923141518.3e4d46a3.olaf@aepfle.de>
In-Reply-To: <20210923111044.GA3849950@dingwall.me.uk>
References: <20210923111044.GA3849950@dingwall.me.uk>
X-Mailer: Claws Mail 2021.09.16 (GTK+ 2.24.32; x86_64-suse-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/JNDON5ThbWtSJEAw9S8nZ3f";
 protocol="application/pgp-signature"; micalg=pgp-sha256

--Sig_/JNDON5ThbWtSJEAw9S8nZ3f
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Am Thu, 23 Sep 2021 12:10:44 +0100
schrieb James Dingwall <james-xen@dingwall.me.uk>:

> xl save -p

This was discussed earlier this year in "how handle domU checkpoints proper=
ly".

https://build.opensuse.org/request/show/918052 has a patch named "xl-save-p=
c.patch" to address the issue.


Olaf

--Sig_/JNDON5ThbWtSJEAw9S8nZ3f
Content-Type: application/pgp-signature
Content-Description: Digitale Signatur von OpenPGP

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE97o7Um30LT3B+5b/86SN7mm1DoAFAmFMb9YACgkQ86SN7mm1
DoBMDRAAgvUzkq1aQAvganBt15YngNbOviw5Jl2i9JvIgir8U3KSAiLPW2mT4ZlP
Vcpw2kNCpAXaDlWMMMn1xVjdZ8qewU1ViFizLBLWn/h7L8wyDgS6fSc+lWJaLOrY
/z1VnpsdiBusrxthb56aiVlaSsALKGEU+g3ZcWcW2iUTV1YL1Jia7lJBB2aMLy4e
6iqnonVxtF9L/EV0qtZQj6lb+eg4CPspJTZ/SIGN2nUw1OkjMBWjgXiQ/VZTNi/D
pYTC6zrTPi7y7wQLEgfZ9gehkkwUXnKYmwOHbt517vhLvNkiLp5Oq0VUXFtbEVM8
XgqcBWQbgsNuuQ/aXvUVwx0MCPZ71aNx7BPEFkmtkMM6kQHwEDDM0ziFpnWB8fhe
wY8vAR4xxlU4RG3dT4e1Wsw5D8t1OSQutx09o8t3JfKVueC/o4iDOdDzffmxyE68
KUZ40kefKWTR7vUtUIlgag5THHzbnhbBwKmysk2v7mnBDMfc6eTzB+LaQTsdWDLf
dBh4fj3vsWmrQhB/Z2ZrJRDUuVOnMjKTDpNlc5Kawm24186YEtEo4prneIitM38p
IIHemAZCPwAFkd2qva9B0scS43xZDrQkoNGWbQ6qi/JHQweHa6PA0uQNcFoxTjR8
EiKE6zJnivnsxYafn9Qj0laMXqo+F/6lrpXN3jTMeCTz7/w5xoU=
=6gB/
-----END PGP SIGNATURE-----

--Sig_/JNDON5ThbWtSJEAw9S8nZ3f--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193928.345532 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNev-000590-MK; Thu, 23 Sep 2021 12:17:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193928.345532; Thu, 23 Sep 2021 12:17:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNev-00058c-HR; Thu, 23 Sep 2021 12:17:01 +0000
Received: by outflank-mailman (input) for mailman id 193928;
 Thu, 23 Sep 2021 12:17:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWi-0005KP-7n
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:32 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe05::61a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1cb65bf7-a450-400a-91b1-1323ad587036;
 Thu, 23 Sep 2021 12:04:53 +0000 (UTC)
Received: from DU2PR04CA0162.eurprd04.prod.outlook.com (2603:10a6:10:2b0::17)
 by VI1PR08MB4447.eurprd08.prod.outlook.com (2603:10a6:803:f3::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:49 +0000
Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b0:cafe::9c) by DU2PR04CA0162.outlook.office365.com
 (2603:10a6:10:2b0::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:49 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:49 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Thu, 23 Sep 2021 12:04:49 +0000
Received: from 3548f08ec3d6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 68D427ED-28CA-4AFD-BC4E-C23B61B42A9C.1; 
 Thu, 23 Sep 2021 12:04:35 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3548f08ec3d6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:35 +0000
Received: from AS9PR06CA0096.eurprd06.prod.outlook.com (2603:10a6:20b:465::25)
 by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:33 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::4f) by AS9PR06CA0096.outlook.office365.com
 (2603:10a6:20b:465::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:08 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cb65bf7-a450-400a-91b1-1323ad587036
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cQXv12XCrC0ornD3XLR6+SrgHIB9Pxv0AYpsZjKo72M=;
 b=a6cHRChBhtKx/YM2LKALl/qBfS6yG+vAqwnM6DuAj/5Qzf2tCpp/paD0XXktUEWO9ZnaJ7np4A5ZB5O3G28xm6Xr8lSCklBTE3l7a03hYA1jdp0koxEQ78g46IwwjvXo3MNoDJzy5GBzFbTIAUr8MA0DTBfmx7sqJIqD9TI1QIM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e6b074adeafe187f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UxmubAxpGvAIsBj3jW5/TAqbSDAoE+BYSSbXcCaiJpCNqRp+GIHVmSm6mzd58y0ktGYzOiDeREQrvfyaHQEA/RM4qXhARyqSYcSGJv61VsfvD/Req4Tx5sr66iqNO3z/zR8xoFc0+mEobvHRyv9T7zDKZ/ev59kZFH3NU0ss5JrBSgiPzmAYWJY9OYYoqZUn5Qt/FBKPH17rd7mm1EzADuz/8BApzQP2Pg3BF2y6LzI8ylVeiQ8y2rTRMa8JwP1tDTZKdJYzak5g4o85EFNMkK82U+cSUTwKT442lsQbhT02vUeztHVHSEhCATbUSHpFjfE7ldJBOqR38CZPeWe4wg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cQXv12XCrC0ornD3XLR6+SrgHIB9Pxv0AYpsZjKo72M=;
 b=Gks5zDlFNm1dO2scoCgfCfycFecaKUXENm4ME4iqs552H6lu8DGD7TQTT8Q/5bPVn9CTXjxeH+RwHGdp3qdOHDz0gjZHBOq9a0Q7GqlIQaAV3bQOFuDF/C3/VEUbf1iLEO9HLzfkwEa72ePbFr3vs16k8oZk3XGlizZLhcE5Yd2nltnzE0auUVtUWdzSNkyKR7pqur2/fmWuSWNGs5Z1W+bcJh+ni+suMP77i0RBkDinsChGdtztBTmmPelWPuxmxFKdqU4xUvLV/Ot6NUubWl8mkUQCxOgkX8dxKfI0l4B3u0E/Q1qgmrnMFHpjQ7lMNV2WA0vx5ZgGX2cH5NqSWA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cQXv12XCrC0ornD3XLR6+SrgHIB9Pxv0AYpsZjKo72M=;
 b=a6cHRChBhtKx/YM2LKALl/qBfS6yG+vAqwnM6DuAj/5Qzf2tCpp/paD0XXktUEWO9ZnaJ7np4A5ZB5O3G28xm6Xr8lSCklBTE3l7a03hYA1jdp0koxEQ78g46IwwjvXo3MNoDJzy5GBzFbTIAUr8MA0DTBfmx7sqJIqD9TI1QIM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 27/37] xen/arm: Add boot and secondary CPU to NUMA system
Date: Thu, 23 Sep 2021 20:02:26 +0800
Message-ID: <20210923120236.3692135-28-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 29c3aa62-b93a-4e6c-587f-08d97e8a57ec
X-MS-TrafficTypeDiagnostic: AM0PR08MB5363:|VI1PR08MB4447:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB44475541191CBD530E1340FE9EA39@VI1PR08MB4447.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 9OXy2VLFXhEAEPzN/ck0kCHr3GGPjpx0WKFBXgmMQzRC4uakapqD5/ls5Mx2EXNZwiDRLSHHqYArcNi6SOCdp0V3A0VK8fwtnAqzzf10QxS4mrl9rPV+0T++29aj7N2dg7Q36Km8IMYko/HaUWVBZ6zBiqdikgJoTjyaD6GSTleyyOFsCzoYXrVkIPY6xhAvSstMX2lQqmFBR5bHcBe0pReaVL/Xk1kKsxxH9pQxlQ4UiBAkgywx/k0MBfTv1M8ywPIhEybHuf/PpiA0EkiV9sveXIlYYm+R7rqgzPQPSqSIyrYLCe/T9Ht6Cz3wI45uCA9t/KaZJ3EMUNWxs/UAdmLHwYmIvU35viPyB0ydKdNiZ6af1DC8vyk91vt+/8uh+8bb/YdUwrGDTlGYwC7MKtiP0x+cHxYFcjgvj1wrvqb5a3cFeaNKZYyEH8IcLU5zd3ry5oqV0wafQWxyVxYSnq8U02jU2/HQh5O3IMwZlE7t51VMYTSoNwdo8YNENyARu6RdxbYK2zvpik7zU63kIx5r5vIQAyyZ5cXrxGXtftD++5+u6T8phwPEo95JNfKWbnzOxmo7iMGaJe4az7JYx3O2HJBuSSSbJuIssdaxlK9Edx/7uctbVYQxZc7x7rIcapSaCQCIcShUOyUoFlf19BwK33+/WtG2IKPGymnVciTVway4AT2hCE3l7MYGi62OqyxNTl6/fw+Pz5RukwTFCtBG83FihZQzTsnOqZ+wLDc=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(316002)(1076003)(5660300002)(83380400001)(8936002)(36756003)(82310400003)(110136005)(36860700001)(2616005)(2906002)(86362001)(70206006)(4326008)(508600001)(426003)(8676002)(7696005)(70586007)(6666004)(47076005)(81166007)(336012)(44832011)(186003)(26005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5d9a9377-4a40-4df4-517d-08d97e8a4e85
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8TFpdkcx4EqOQng0e8tm1dUSqyWPpyQlk+ZJw+f9ZSqD2Qa5Tk3vxYHqT5uUuRjMjwmk1yuYzNomvcJGOMQ6CbIvpCAr2SKIRte2FHrCfm1ztkkjY6ZmhpNjfZdL4OzgsbSn+55GxQhILzYtjbnZ6H3Oyg8x1Ztn8RnNOz83GYjuSUvy/if+DUB1JJuU5Qbu/N6AF4Eo+sKxj7P/SY2glEuZCwPdHY7p8OKbaHFzqIuSjGpcQh1ukWBMs9NzrKpwhDcN2jBWgxyZC7dPxQoqItdjxBbzXta8dBLjPps5ncPYTfkZNiJ6hY+YzESMMbzUNpdCXtmzUFCEh9NnGITYEoLteaxpQjAy48JE8yw85UifJ/XnSlSCtt5SYmzTzNewXKaTp47oWHZmGTh1Q5tCCUKl1UM9p2lus3Z+FbQBFW+CsQ1t1dWg3SoE+5JaR0MRRI/XgxNSL2oXiLIlnTKJt2vEyiOQh43eogQEgrJb475qIsUqYLGE42zokZ49WARpF4Eyawk+n1X502RVvRVkgKOyzHUL+RJVhP6aDfxxNa7cjVwhGjrGvO2ee3EHFN/X66hSeWTa5GNATfuV3hRZXv7Dq36iG6QoFcfO1jO2vREm0wv0SdalI50x2m42zvHX53fpW8ALUOGY1JXxuUnnAdDz3RZS1uHS4U3pSO2cOF2vlFkQAviXFXEWepyzjDc48+6/rKWLsT2m0pJpfst0FA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6666004)(336012)(2616005)(316002)(5660300002)(86362001)(110136005)(426003)(4326008)(36756003)(47076005)(186003)(81166007)(2906002)(44832011)(26005)(8676002)(83380400001)(8936002)(1076003)(82310400003)(36860700001)(7696005)(70586007)(70206006)(508600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:49.5269
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 29c3aa62-b93a-4e6c-587f-08d97e8a57ec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4447

In this patch, we make NUMA node online and add cpu to
its NUMA node. This will make NUMA-aware components
have NUMA affinity data to support their work.

To keep the mostly the same behavior of x86, we still use
srat_detect_node to online node. The difference is that,
we have prepared cpu_to_node in dt_smp_init_cpus, so we
don't need to setup cpu_to_node in srat_detect_node.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa.c        | 10 ++++++++++
 xen/arch/arm/setup.c       |  5 +++++
 xen/include/asm-arm/numa.h | 10 ++++++++++
 3 files changed, 25 insertions(+)

diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
index 5209d3de4d..7f05299b76 100644
--- a/xen/arch/arm/numa.c
+++ b/xen/arch/arm/numa.c
@@ -32,6 +32,16 @@ __init void bad_srat(void)
     fw_numa = -1;
 }
 
+void srat_detect_node(int cpu)
+{
+    nodeid_t node = cpu_to_node[cpu];
+
+    if ( node == NUMA_NO_NODE )
+        node = 0;
+
+    node_set_online(node);
+}
+
 void __init numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance)
 {
     if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 49dc90d198..1f0fbc95b5 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -988,6 +988,11 @@ void __init start_xen(unsigned long boot_phys_offset,
 
     for_each_present_cpu ( i )
     {
+        /* Detect and online node based on cpu_to_node[] */
+        srat_detect_node(i);
+        /* Set up node_to_cpumask based on cpu_to_node[]. */
+        numa_add_cpu(i);
+
         if ( (num_online_cpus() < cpus) && !cpu_online(i) )
         {
             int ret = cpu_up(i);
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 8a4ad379e0..7675012cb7 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -46,11 +46,21 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline void numa_add_cpu(int cpu)
+{
+
+}
+
 static inline void numa_set_node(int cpu, nodeid_t node)
 {
 
 }
 
+static inline void srat_detect_node(int cpu)
+{
+
+}
+
 #endif
 
 static inline unsigned int arch_have_default_dmazone(void)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193930.345543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNew-0005QL-Tf; Thu, 23 Sep 2021 12:17:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193930.345543; Thu, 23 Sep 2021 12:17:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNew-0005QA-Pe; Thu, 23 Sep 2021 12:17:02 +0000
Received: by outflank-mailman (input) for mailman id 193930;
 Thu, 23 Sep 2021 12:17:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNVp-0005KP-6E
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:37 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1b::62b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 491d468a-579d-47bb-8f4f-d7b0ca53fc34;
 Thu, 23 Sep 2021 12:04:30 +0000 (UTC)
Received: from AM7PR04CA0021.eurprd04.prod.outlook.com (2603:10a6:20b:110::31)
 by DB8PR08MB3963.eurprd08.prod.outlook.com (2603:10a6:10:a8::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:28 +0000
Received: from AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:110:cafe::a4) by AM7PR04CA0021.outlook.office365.com
 (2603:10a6:20b:110::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:28 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT020.mail.protection.outlook.com (10.152.16.116) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:27 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Thu, 23 Sep 2021 12:04:26 +0000
Received: from a55985ad4ddb.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EDD24E0F-1EF2-41BF-8A2E-E6A5D197A8FE.1; 
 Thu, 23 Sep 2021 12:04:17 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a55985ad4ddb.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:17 +0000
Received: from AS9PR06CA0118.eurprd06.prod.outlook.com (2603:10a6:20b:465::20)
 by DB7PR08MB3756.eurprd08.prod.outlook.com (2603:10a6:10:79::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:16 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::b9) by AS9PR06CA0118.outlook.office365.com
 (2603:10a6:20b:465::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:16 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:15 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:58 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:56 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 491d468a-579d-47bb-8f4f-d7b0ca53fc34
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6dEtUaYVX87QwORy8SpB1QfVQ6OMtEh5+JCNvdfCby4=;
 b=EpUJJCYL5wAgOIM62xfO32tbLeMHjnUf4FVdUYhAKYwxCo8R0mFoPxT14t/AQ+0q/GyCPVH7MKEdQ+ItQEl6h3I88iJAhpIKDCS+XU7WlFqUuGSvTRTseszDCuuhGtXY5Vch+U7qrOr8FMKukV4/CkIs/cmqe4f1Sb500Hn/fAQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 1f3622f63e5f6b3d
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QOegBsqLDJqcFf/9zDozwwhanjaRYg/F3zPM7mf8OYU/DByb/EcCK49aY2hbuVPh8UsGTDwf0PzYqrBVcnZFtajfJSiwV43JMQe48mxgS7MxGJ9lDMxonf9pP4hls5MR86kFen7/f6GqruOdO61CbnK8bPE7TD9bAKghh30edEcYJN/beeNHeWC9WF3an3o/2GdWVJPyQ6uvSRffyfavz8EnvitWbN6N0UfIMHDkj78Epxqc9WOw6KxND4Ghp9Ix9hezMqzWc9B3697dGCUOqKlTZP+RUQBfCN9Xnm+phJ3EZ8g67cmrNAT+3wseebXcCUMbmeR9hj1YMUq72vjwGw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6dEtUaYVX87QwORy8SpB1QfVQ6OMtEh5+JCNvdfCby4=;
 b=Sk9rN/s5Ms3xycath6IGwhSNy0e3i2rjAi8eNKihEqAZuRR1g0RPRAEy8gRbGLOuIfZhIt28x4s+OwksNvqR7DG02/DHl0d6CjrjsBFk/19nED3EH23VGnmhy08x4Rngl5kU8IvGsnYa4oJXOirS59xiHQf+fL+NujDNHOI0St8CyE/M5OcZMYH17ZMq2MhROjzLcdkuDfGa7/dpECGLrGUI8bijbHDP6hEWVDkNsdDFi01THY3CMc/Epdv4MVkzicfaBvH8pe9nhhxGY+2bgi8VLIfA8wsH814aGao92HQQp8ALnwD0FhoHQIad73wKBeeAByzcWloiaDdzt7cx6g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6dEtUaYVX87QwORy8SpB1QfVQ6OMtEh5+JCNvdfCby4=;
 b=EpUJJCYL5wAgOIM62xfO32tbLeMHjnUf4FVdUYhAKYwxCo8R0mFoPxT14t/AQ+0q/GyCPVH7MKEdQ+ItQEl6h3I88iJAhpIKDCS+XU7WlFqUuGSvTRTseszDCuuhGtXY5Vch+U7qrOr8FMKukV4/CkIs/cmqe4f1Sb500Hn/fAQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
Date: Thu, 23 Sep 2021 20:02:21 +0800
Message-ID: <20210923120236.3692135-23-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7e2f21ac-8e00-4fe5-dd8b-08d97e8a4ad9
X-MS-TrafficTypeDiagnostic: DB7PR08MB3756:|DB8PR08MB3963:
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB3963C7FBC7FA757E681B1E719EA39@DB8PR08MB3963.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 dOv97TrR0v493lsXsyhbvgSbMmqbXfR+lNx3QLOXxO38q4KvfnI0aY6zl0mdBGBNJ3N55o270bA66G14aTkwMuEhbkxcsIgLQ6CyYz8oqiNqgDwpwi4jZ+DN5bzP861uTx/MSu3wpl9uIjYX8XYt076u+SE0iqXGG39rjhsKDCsY/o/UfqwINIa9hXApTwsLLhTUVjrphSOkDzfeDjuc8ICmLNsQIAAUt0F8+1L6O+A160dagclfMPsC6Vh+hSf+XAn29s01LhiKrLVLSfIcrOu8UG9j49P/f0Yg4I5haaTp7Kx6A6l3LOWH2sY3Y1T20asTAFzjEkGBljkruRuN0cNX9PVroYohSfaDdJSFoXVVD7ittg+2NaSCN/5PjP4IYxTLJ/rezrxN2vBHRiKmevLwoCovVyzKrpdqOLYDJwZidALHQUmHJ40h5a2dbTOrPST5bfbRZV+3OxWYGAYjuJy3P9L2qwNniiLc69aSlpZBZ2IMSOhcxQZPINruVD5Dh9gkoeBT/w+lDt4eMS+2PjFZXE+nht9XQ1uAHzONih/R9+Me/iAGZ+qtpKkH6knX0pCaC5WfU1enInKRgNpneC8Oj26rMyqhR3RlNJqCz/5g/OQJ0C9kjjL/EnB5EKt4aYyit1HyqpRXOPnWMDaKUvzgDvT5vTBWzAbUM/NFf3659awbFz3h6MbMkDdwp0ZSo3LbNqOo90PqsjvZ9s544jiT+cxLNTBrHxJ8fJyzHXI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(426003)(356005)(83380400001)(86362001)(508600001)(81166007)(82310400003)(316002)(336012)(36860700001)(110136005)(1076003)(2906002)(70206006)(8676002)(2616005)(8936002)(36756003)(70586007)(4326008)(6666004)(7696005)(47076005)(44832011)(26005)(186003)(5660300002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3756
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	81e2f0ec-5f3c-46bd-e2cb-08d97e8a43d6
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sccVZf+mOTc6JXx7wbfxuS927Ys6Fx81eXOigU7tsClKnV+8wrumjN6KEpQkjtJBRPq+eftg3Z3lNajHAbERCr3GGrXZRsbUL81DmTks0G9MlpJZsKWe3y2XIdXtqsXoDGY3oEQG8NJgWmR2joXCYXh5q3ZJpO5hCCB3d2W8yM4uvNFd0iC3X0eZC1CImqyoWpNmXyhTBtjSIQ3JCpDKD7Ng22GDRk9phzV1kDa4zYgNLOuidIIu8kyUkO3csDY16Y2reg9s1ymlmiJGBsldaHF+dBSj6avS+bIqDx73LImx0zQrKptflJxa2tUb54okQre+9xRVgKmcxoZebg6F4tbuejygPyixsiRr/xXaCcZsYruAwefk+jK1UoXIw9MDO9QKRlHif2BKR+HCfXEBXGEVI0KA9erOo+YljsksaBUMM3KUZCP7scHZNm57l8s1Swm8J/zRj1QiQwge+LUJ0wN+7EZ2eM57QNhSFg5WPViffhBbZmlmcn3QZhRlUGoabN4tYPMyqSCGtdngZLKkQirG4w0LnjfavVnJSUD0ovnQGuG8VyU8W0BiCJPYrQzx52Q6IuCP7Z0wAwaLndlq14y6hH6EkfWofG7kURChhUSrqofiuNyTUY2Ssg8O9RdYUKiAQwnmh8FX/NsbZEbtgmRyhv/JZF9yUdyUvVnkRG1/J836cV6fqQN79nCSPilq0wH3rphSM4KY12Ir3dMePg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(36860700001)(316002)(5660300002)(508600001)(6666004)(83380400001)(47076005)(336012)(8936002)(44832011)(426003)(2616005)(186003)(70206006)(70586007)(8676002)(36756003)(1076003)(2906002)(26005)(4326008)(110136005)(7696005)(81166007)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:27.5424
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e2f21ac-8e00-4fe5-dd8b-08d97e8a4ad9
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB3963

As a memory range described in device tree cannot be split across
multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS in
arch header. And keep default NR_NODE_MEMBLKS in common header
for those architectures NUMA is disabled.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/include/asm-arm/numa.h | 8 +++++++-
 xen/include/xen/numa.h     | 2 ++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 8f1c67e3eb..21569e634b 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -3,9 +3,15 @@
 
 #include <xen/mm.h>
 
+#include <asm/setup.h>
+
 typedef u8 nodeid_t;
 
-#ifndef CONFIG_NUMA
+#ifdef CONFIG_NUMA
+
+#define NR_NODE_MEMBLKS NR_MEM_BANKS
+
+#else
 
 /* Fake one node for now. See also node_online_map. */
 #define cpu_to_node(cpu) 0
diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
index 1978e2be1b..1731e1cc6b 100644
--- a/xen/include/xen/numa.h
+++ b/xen/include/xen/numa.h
@@ -12,7 +12,9 @@
 #define MAX_NUMNODES    1
 #endif
 
+#ifndef NR_NODE_MEMBLKS
 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
+#endif
 
 #define vcpu_to_node(v) (cpu_to_node((v)->processor))
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193941.345554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNf7-0005wR-Bd; Thu, 23 Sep 2021 12:17:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193941.345554; Thu, 23 Sep 2021 12:17:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNf7-0005wI-77; Thu, 23 Sep 2021 12:17:13 +0000
Received: by outflank-mailman (input) for mailman id 193941;
 Thu, 23 Sep 2021 12:17:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWn-0005KP-7z
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:37 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe06::600])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a96bdc96-c20a-417c-a929-905a94858260;
 Thu, 23 Sep 2021 12:04:55 +0000 (UTC)
Received: from DU2PR04CA0302.eurprd04.prod.outlook.com (2603:10a6:10:2b5::7)
 by PAXPR08MB6972.eurprd08.prod.outlook.com (2603:10a6:102:1dc::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:53 +0000
Received: from DB5EUR03FT006.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::e9) by DU2PR04CA0302.outlook.office365.com
 (2603:10a6:10:2b5::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:53 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT006.mail.protection.outlook.com (10.152.20.106) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:53 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:04:52 +0000
Received: from ab7004a01c82.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3592E839-A002-464C-8159-FF9022817EE6.1; 
 Thu, 23 Sep 2021 12:04:40 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ab7004a01c82.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:40 +0000
Received: from AM6P191CA0091.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::32)
 by PR3PR08MB5673.eurprd08.prod.outlook.com (2603:10a6:102:86::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:39 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::12) by AM6P191CA0091.outlook.office365.com
 (2603:10a6:209:8a::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:39 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:38 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:17 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a96bdc96-c20a-417c-a929-905a94858260
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2kwEsupYdQhcxT6kCZF4Bnp8YHcdU6oO4hBw9qhon1g=;
 b=CQ7Jq1j8gT8ZK7pXB3x91ubyuMcIOIRnhXX+yKhgtDStU6CH8CLJaTfRZE5SNHjj92B5jZfFtfJUaxHpJk/UY6OcjswrGfUv8oQNki/2L/qQxt8uuKdjNLW0ABaWmD2p197BEUJfAHNGLvDSvF0O5E+M+oK4H8TrboMP1GnJS3M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e27c485cd25f70f0
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QnZqa6OQT2k3vrlzKVEv7/To8p1VmmTxzOw8Wgwh6hSunYTmygsOKj67nj/q28CCuJIxzxfZeMhlVGhxX2QnJLbH25O3LtYw0a8ZwldtuWC3qguUOJbSNjKEPMtlv2rnaZM+Z+eec1OTtDT6LJUMq/7R2+OrAvSYeXR4p1fV26/68pCixOc6HpMuMhmd+LIHp3CCz4web+ZMuOesCILWZLwG1y2m6OpxbmS7alwLi0Caj23QdIxS+7Nkukck2wX7KrP4gwWb9fzGNstYVv0/1EbhKus98mfaQ6S/CMxdWz9t6hxogbL3CpX7eec+OLcbF+xcffj/yZON5FanRvibaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2kwEsupYdQhcxT6kCZF4Bnp8YHcdU6oO4hBw9qhon1g=;
 b=kyK2/b3yxutn6cCOcOBP6ggt+VOVIrssm8bhfaaWXXsaN5HT3P4t8+ObcOcDlMPloM88FEUZE7sehpgV+Zo1YYfSWdLmROSOVNzWV2f5GXT6riYW/ahNjTw8bLQAHCzPiHkti+nUojN/2LexELtoe3BrY9/etG85XXtcYr5VA5Fen8iLtBJZkbg6fizeAODI3veXCf5+ScS7jqWU/K8HV8WLMI0dmIjjP4GXDQcRJ7JCCPIQEsdg9rU/OIDBsxhXsXbhXYA4dZo9wILyGYrLRLTknOGkylWGrKLyl6Z4rOG7G+JWVyDxgWzxj2j/e79Y2efT1qKb0nbwy8KU/V+rDQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2kwEsupYdQhcxT6kCZF4Bnp8YHcdU6oO4hBw9qhon1g=;
 b=CQ7Jq1j8gT8ZK7pXB3x91ubyuMcIOIRnhXX+yKhgtDStU6CH8CLJaTfRZE5SNHjj92B5jZfFtfJUaxHpJk/UY6OcjswrGfUv8oQNki/2L/qQxt8uuKdjNLW0ABaWmD2p197BEUJfAHNGLvDSvF0O5E+M+oK4H8TrboMP1GnJS3M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 31/37] xen/arm: introduce a helper to parse device tree NUMA distance map
Date: Thu, 23 Sep 2021 20:02:30 +0800
Message-ID: <20210923120236.3692135-32-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f60a024b-3763-4276-f705-08d97e8a5a1b
X-MS-TrafficTypeDiagnostic: PR3PR08MB5673:|PAXPR08MB6972:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6972941BA98DFCCB6C80AFC69EA39@PAXPR08MB6972.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 I7HRuwLqb6rxp113nJfoi+GL1LRBKvxeVSVq/fZ1HxCVCTFy4aiFmHZGmrrsXr7D1g/KYvUi9sg/hiRpB41FKIL4l8Ka0M8a/tTwLS6kUggbcalrRnhOovfYWsVwItdjS1N0I7YlUpuNNVa1HdwOuAIVz6rPtzIjc5/I21SlswpaGeHZrjDsXRXhaFqMV6ZV7tISy3aBtdIZ2NKRSVF57b031sR/9l+ao1SEQdAkB++H1SmqoTuoIOHkWZB9qFFEE4dtf/uCdB7y14K/97uQN5mBrFmqdXhXIevf+Qv6uqBjzQreWjuX9mE05H5LkdmNJRxQpB/LXkJfSYuybGYsuCIz7P4Ef9LertVbUyWRg683Ek26M6t7dEKczXf1rfGu69blrNcQ9Ed4h4n/pQibU/Rqu79pZbrS7VogUxXzqKIEVY2YpnnhBsy53ws9sawUimWzrMSBbCOS4aLeVU5mG2/ZUOk/T2VthAL30bjY8QJ97f2T22ccnz385v+kWhhIBiwX7CRUFSlqXDE17+ALBVS9m2kd7ixL1TdDcNe/EDXtec+hq9/z7KpDteVd1xpMSJnRLVbZf51xlWFUSJJVlcLDqT1It0sYfVIQSjLdGo66/9wjKOP3M1zolzZEp8N97ub/9BrmE64UT20Q6EOZ25TCN9yQnrDjQTbY0kEB5XyIjyCE9DrmcAQYph0BfZ5Dnd/B3xU8p4n6JkANcFXTJ4/uFDqDL0hiP/E9CDNxdTAlCR9BONBtHHFUAirwBkxaWCOKTr2pwxzjEQIQCZLuIFKx2tjq+ujRmPH43uYp6m0hevcYM2PU0arlmyOeWEov
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36756003)(5660300002)(316002)(110136005)(44832011)(1076003)(47076005)(508600001)(86362001)(81166007)(6666004)(82310400003)(7696005)(8936002)(26005)(70206006)(8676002)(966005)(70586007)(36860700001)(186003)(2616005)(356005)(83380400001)(336012)(426003)(4326008)(2906002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5673
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT006.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9d6f7704-002a-44c8-68cd-08d97e8a517f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wrCUhuLO0lH7466nBb5yETRORbCbwEQvuoBGor/OFCRMkBZDcy1UIcUvSnRBJT5pYyTd+/XRiiqS2s7tX8xggjoA7gkjr/yWt8rzwZPmxWE0gCbItieodg+ToFdUZK1XtAKEHcqblGYfN3eIxsFdw1s5jD/NpxYO7cDL21DqfLT6QA3kfFjnnstw6hZN7joVL2cgJmelQibgwm2XH9W5+yeuKb8aOBc5nYkotvrk6/hjYe5qLgXAiT2u1jwNoHE+THFYXcsLJCs31itbX16qVrBuXyYlWEdUOmK/SojeJleUO+YWAFoW/Ec5Nicxzd/l9IHcVHzMHilSb6kXOobc4IPwnNZq0Jmg5vEX9nsCgkf8L+yssgPyARrPqvHgBr9dHMJLfsLUgKjADJ4yGQArkCjimmgOcDqT9/UocuMqqBtNyCtWI1xgtgRJvYPVk2eozYOIULdLxC/iNNswszWf9ATDgemYDOPi9Y+VpwgrJnzJqXPcndO9nc/s0f8ooELgAWLEPEGv+jOK55P+eelJeFDlb/JpDouaYxyRFBiPwBIXzABWBEO9SMy+fcdCmghvfLPt5jZsvUnDjgcwYvqT+9MJqpA8grjF4zJ/rSzzyRopClbCsPNRIhEyRbtEeBp/voYsBs4L78B87dn50By9dIQS/CJExxdCw/+NeI/9CXXFcolwZ8rWGwUMo6jj7lgHsQSldBxUSyU8PCGFGOw3l3+coAc7TTE1NKgHNA20/g4XqopBY/zUMoDleDNyo/AwL3G4mRP+oXPT/kV8dBeorirLK9gWbqsthEYXc5KXK8I=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(110136005)(8676002)(426003)(508600001)(336012)(82310400003)(36756003)(5660300002)(7696005)(2906002)(8936002)(316002)(70206006)(186003)(2616005)(966005)(4326008)(83380400001)(44832011)(81166007)(86362001)(70586007)(6666004)(36860700001)(47076005)(1076003)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:53.1934
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f60a024b-3763-4276-f705-08d97e8a5a1b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT006.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6972

A NUMA aware device tree will provide a "distance-map" node to
describe distance between any two nodes. This patch introduce a
new helper to parse this distance map.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa_device_tree.c | 106 ++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)

diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
index 7918a397fa..e7fa84df4c 100644
--- a/xen/arch/arm/numa_device_tree.c
+++ b/xen/arch/arm/numa_device_tree.c
@@ -136,3 +136,109 @@ static int __init fdt_parse_numa_memory_node(const void *fdt, int node,
 
     return 0;
 }
+
+
+/* Parse NUMA distance map v1 */
+static int __init fdt_parse_numa_distance_map_v1(const void *fdt, int node)
+{
+    const struct fdt_property *prop;
+    const __be32 *matrix;
+    uint32_t entry_count;
+    int len, i;
+
+    printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n");
+
+    prop = fdt_get_property(fdt, node, "distance-matrix", &len);
+    if ( !prop )
+    {
+        printk(XENLOG_WARNING
+               "NUMA: No distance-matrix property in distance-map\n");
+
+        return -EINVAL;
+    }
+
+    if ( len % sizeof(uint32_t) != 0 )
+    {
+        printk(XENLOG_WARNING
+               "distance-matrix in node is not a multiple of u32\n");
+        return -EINVAL;
+    }
+
+    entry_count = len / sizeof(uint32_t);
+    if ( entry_count == 0 )
+    {
+        printk(XENLOG_WARNING "NUMA: Invalid distance-matrix\n");
+
+        return -EINVAL;
+    }
+
+    matrix = (const __be32 *)prop->data;
+    for ( i = 0; i + 2 < entry_count; i += 3 )
+    {
+        uint32_t from, to, distance, opposite;
+
+        from = dt_next_cell(1, &matrix);
+        to = dt_next_cell(1, &matrix);
+        distance = dt_next_cell(1, &matrix);
+        if ( (from == to && distance != NUMA_LOCAL_DISTANCE) ||
+            (from != to && distance <= NUMA_LOCAL_DISTANCE) )
+        {
+            printk(XENLOG_WARNING
+                   "NUMA: Invalid distance: NODE#%u->NODE#%u:%u\n",
+                   from, to, distance);
+            return -EINVAL;
+        }
+
+        printk(XENLOG_INFO "NUMA: distance: NODE#%u->NODE#%u:%u\n",
+               from, to, distance);
+
+        /* Get opposite way distance */
+        opposite = __node_distance(from, to);
+        if ( opposite == 0 )
+        {
+            /* Bi-directions are not set, set both */
+            numa_set_distance(from, to, distance);
+            numa_set_distance(to, from, distance);
+        }
+        else
+        {
+            /*
+             * Opposite way distance has been set to a different value.
+             * It may be a firmware device tree bug?
+             */
+            if ( opposite != distance )
+            {
+                /*
+                 * In device tree NUMA distance-matrix binding:
+                 * https://www.kernel.org/doc/Documentation/devicetree/bindings/numa.txt
+                 * There is a notes mentions:
+                 * "Each entry represents distance from first node to
+                 *  second node. The distances are equal in either
+                 *  direction."
+                 *
+                 * That means device tree doesn't permit this case.
+                 * But in ACPI spec, it cares to specifically permit this
+                 * case:
+                 * "Except for the relative distance from a System Locality
+                 *  to itself, each relative distance is stored twice in the
+                 *  matrix. This provides the capability to describe the
+                 *  scenario where the relative distances for the two
+                 *  directions between System Localities is different."
+                 *
+                 * That means a real machine allows such NUMA configuration.
+                 * So, place a WARNING here to notice system administrators,
+                 * is it the specail case that they hijack the device tree
+                 * to support their rare machines?
+                 */
+                printk(XENLOG_WARNING
+                       "Un-matched bi-direction! NODE#%u->NODE#%u:%u, NODE#%u->NODE#%u:%u\n",
+                       from, to, distance, to, from, opposite);
+            }
+
+            /* Opposite way distance has been set, just set this way */
+            numa_set_distance(from, to, distance);
+        }
+    }
+
+    return 0;
+}
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193942.345565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNf8-0006E1-Nd; Thu, 23 Sep 2021 12:17:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193942.345565; Thu, 23 Sep 2021 12:17:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNf8-0006CZ-G2; Thu, 23 Sep 2021 12:17:14 +0000
Received: by outflank-mailman (input) for mailman id 193942;
 Thu, 23 Sep 2021 12:17:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNVz-0005KP-6X
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:47 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.74]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 66900bfe-957e-47f2-97f4-2bcb54fed078;
 Thu, 23 Sep 2021 12:04:34 +0000 (UTC)
Received: from AM7PR04CA0024.eurprd04.prod.outlook.com (2603:10a6:20b:110::34)
 by AM0PR08MB5171.eurprd08.prod.outlook.com (2603:10a6:208:159::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 12:04:32 +0000
Received: from AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:110:cafe::2e) by AM7PR04CA0024.outlook.office365.com
 (2603:10a6:20b:110::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:32 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT020.mail.protection.outlook.com (10.152.16.116) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:32 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Thu, 23 Sep 2021 12:04:31 +0000
Received: from 19d49da73a86.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8AF177E2-6968-4553-891E-0E442CC65920.1; 
 Thu, 23 Sep 2021 12:04:18 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 19d49da73a86.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:18 +0000
Received: from AS9PR06CA0093.eurprd06.prod.outlook.com (2603:10a6:20b:465::22)
 by DB6PR0802MB2581.eurprd08.prod.outlook.com (2603:10a6:4:a1::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:15 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::2c) by AS9PR06CA0093.outlook.office365.com
 (2603:10a6:20b:465::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:15 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:14 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:55 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:54 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66900bfe-957e-47f2-97f4-2bcb54fed078
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gvySSU4kIt7pfh5/hl+s44OlyMdIG3eGKnyLNiYLQmk=;
 b=U4tFnxshcx9DiEH5p7IZTk8JiaTvEw++j5U1xylexRQ7vuZqv8qSvfMxG9aP451kCw0ZCqRP1YSZcna49AKsHwjQA8df/wrBnwzH9hEFN+hnfFzqUqHB6EdlkgPYow1uspvjSGLqwsGXG7MkHHDO0ue7nCSN20FZAuikI5YZXko=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: b1669256a1151d9b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VGXRIjy+dAMyO+wmLYAleLbc0RZZMY5JBTTdUU2HBiLBUmaDqIdU8gQtZPYQqQja/E8WDjPtUfPYmFdM+OBsgLspjtHZb0tYIwjqIczeb628BnkE4jnKxkXCMWHUrD1kR49anRryybyZlB9mPqe4HTeWLde5+W4j+P2MqOitUMBrPZGBcxe5GKd1ZYW0/BGwFCybXtqzQUs20J6dJafPjjT5tUmiR2w8IIpRBt7cGai30NvGPBSh9/zVgNYqFvq6DXuEC7RjyajhcHTPu+dodhXWfIXeWv68PV2rJU/jTMxGSwDvzCOolC1BM76OexTGBnY4vbj3Zx/8GvNot0yHVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gvySSU4kIt7pfh5/hl+s44OlyMdIG3eGKnyLNiYLQmk=;
 b=WxFyGPY9U5PnjKc+W7AgpZ8OKEySi8lOWnjaY7Xze6Wjep7hw3K/As0/dL0YrBCR5fK6nE42P2k2g0BnBGr7H1gfpIdmLOH5aiJx9D+oW9258IyFALKGLPDSP70oG1pcIV5ymdlBKD+yHKi0C1Ea5h3NlbzrmeRf++e04qQ6zuNi12OGcU46Ki5C3P2LYODAOstEZuLqmeiLnw8cIKMxUUxLpXBoG3jGXNQd5j35priizbT2nIGSRjaDlrX7sfTDevIRD30pJgdsEmvdanSO8gkZ/oOdTcty6FqpPw3vpJbSPAIZOGBp9LLIu6bbKiMIEakjai51eJaGF45D+y3gdg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gvySSU4kIt7pfh5/hl+s44OlyMdIG3eGKnyLNiYLQmk=;
 b=U4tFnxshcx9DiEH5p7IZTk8JiaTvEw++j5U1xylexRQ7vuZqv8qSvfMxG9aP451kCw0ZCqRP1YSZcna49AKsHwjQA8df/wrBnwzH9hEFN+hnfFzqUqHB6EdlkgPYow1uspvjSGLqwsGXG7MkHHDO0ue7nCSN20FZAuikI5YZXko=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 21/37] xen/arm: Keep memory nodes in dtb for NUMA when boot from EFI
Date: Thu, 23 Sep 2021 20:02:20 +0800
Message-ID: <20210923120236.3692135-22-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1508f99f-b4a0-423e-0b75-08d97e8a4dc1
X-MS-TrafficTypeDiagnostic: DB6PR0802MB2581:|AM0PR08MB5171:
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB5171EE9F5900EB19C09464909EA39@AM0PR08MB5171.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 CJxiuM498ucDO34ghQdgHgKwtFmEQ1Abp0CumzVXo+SJgi1WNS2WUBHAZLncCQcv2LiasLAmR/IqJB9EwTwIYMUJCkvqhJr5qUtfFGD6cF6LfV+j51JsKHBFWuGQM9GwSW0HSqZhQpDObeSanQWhtAmWRg39ESCkAkZsZV7eXOUWZnHITwrwcbo9ilAN5EuCAjAH8SHnCI8gi0wjnJavgpsKRqSM2jY4jiFe1uLjJgyuqIiMSU2p6siRcXyrY7R2+inTv+M1ob0lcMsZhr8gNKUpzcif2NkoatgKgQQ+ZXOD3xw0KobMoSNv1je1X4ZWaSRcNkRRdYv0RMC4WsWuKSqCFZn1yNRqKJQQuB4jRkrcFw2DmNsj6BbFq9T0YS1MV9EnAh3hBr1D5wpffi3HZOQV8YTqwmfyMAr6F9ayiK546b0vEdBI8mOm60hX/6RwwA6pFQ7tgK4i2AdCp8inGlWL5vDp3SseHDebwxwG+TeIh3OOo6AXDHYOp0/JZSMBUkA3KgbnQHJG7CH6RsN3LaNmpDqZFSxUA9b+PkpYcPoLGxD4GjGnZ3Fbdf4kfW2iqgVpc1DMREjvIk865BGbDkPlOoHJeCzc8C34aCo4WtThmAeXVmGoYFKq+kxsYqF+gclqH1qIg5dz+SP16GWYzC1BQvleF4C8dDX97sVpHZZo9zUx4HQaB31e7CqHYscxwuBRaRdoxMAmG3/XgRKPbtfLLygcIjeqTR7P8Ylv8A0=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(7696005)(2906002)(508600001)(5660300002)(8936002)(81166007)(356005)(36756003)(2616005)(36860700001)(1076003)(47076005)(316002)(8676002)(26005)(83380400001)(336012)(82310400003)(6666004)(110136005)(4326008)(186003)(86362001)(44832011)(70586007)(70206006)(426003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2581
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a6e061a9-1254-48ef-188f-08d97e8a4355
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uZvkHcsXMCuHrxZVZnqr66DByS50wpa1AOs0b0r/ezMMWaturZJdpYEGoypyeSpoAEnxWDS4TonfEd54dt8GhbSmcBjU8ASm7P4dpDV2Z2KMVmctyzytq2520mSiE/jr88F4RiOUXJIRwbioxYHJfwAI3FesxnRX6juJnvJLWmMbqRjAf+usajYVZTiQtWmE9fZixkde0TU7HpVcqMRI9Bs5e7SKKx6g3+WU7Qcf75crVcyu+Gc+bCRJE4NXKbd+dyxt2d9odIhZHeA1owzdCsZ/vIhsjUUlAvnCcd3qeP2mKST++PGsfkMt1HPeV8lpxgBuEBRGAFIUXpr5ghL+PcMSphJAuE/tkUHl+FxNmkztPCoJie2utcGdXyzdkU9w1dfyGnQn9msHlRvzGey8r2TNpk9q6oAhgJzdxbncq+NOB233c90m/ktAhiB4WI+4kqaFza7OmwqkYBmwwd20nkVagRgwnuQ6FYIXeVhj1wkUIVCzZ8ooxdYwWbn01I+FcL1LGlmrY2RmkqR2PjG5SOQdJIg4S3GN4eT7762ok93I6iFPi192ymTYKm3mDiyzaR8nuMK+YrC08cevyIf3OmvRxF1CCSaJykWpBH3FBoDpcQ8cTOUUStzJ1bwNO/D/y74KQr3crx9ErZhoFCvMyNyhBPUtjt2su6qZn1xv+wDNenrCeG/a5taTGTARTd1LA9zhFBFTLV2OaXTExtZnuQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70586007)(508600001)(81166007)(2906002)(426003)(70206006)(36756003)(110136005)(5660300002)(316002)(4326008)(83380400001)(82310400003)(7696005)(6666004)(1076003)(336012)(186003)(26005)(8936002)(8676002)(44832011)(47076005)(36860700001)(86362001)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:32.4119
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1508f99f-b4a0-423e-0b75-08d97e8a4dc1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5171

EFI can get memory map from EFI system table. But EFI system
table doesn't contain memory NUMA information, EFI depends on
ACPI SRAT or device tree memory node to parse memory blocks'
NUMA mapping.

But in current code, when Xen is booting from EFI, it will
delete all memory nodes in device tree. So in UEFI + DTB
boot, we don't have numa-node-id for memory blocks any more.

So in this patch, we will keep memory nodes in device tree for
NUMA code to parse memory numa-node-id later.

As a side effect, if we still parse boot memory information in
early_scan_node, bootmem.info will calculate memory ranges in
memory nodes twice. So we have to prevent early_scan_node to
parse memory nodes in EFI boot.

As EFI APIs only can be used in Arm64, so we introduced a stub
API for non-EFI supported Arm32. This will prevent

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/bootfdt.c      |  8 +++++++-
 xen/arch/arm/efi/efi-boot.h | 25 -------------------------
 xen/include/xen/efi.h       |  7 +++++++
 3 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index afaa0e249b..6bc5a465ec 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -11,6 +11,7 @@
 #include <xen/lib.h>
 #include <xen/kernel.h>
 #include <xen/init.h>
+#include <xen/efi.h>
 #include <xen/device_tree.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/sort.h>
@@ -370,7 +371,12 @@ static int __init early_scan_node(const void *fdt,
 {
     int rc = 0;
 
-    if ( device_tree_node_matches(fdt, node, "memory") )
+    /*
+     * If Xen has been booted via UEFI, the memory banks will already
+     * be populated. So we should skip the parsing.
+     */
+    if ( !efi_enabled(EFI_BOOT) &&
+         device_tree_node_matches(fdt, node, "memory"))
         rc = process_memory_node(fdt, node, name, depth,
                                  address_cells, size_cells, &bootinfo.mem);
     else if ( depth == 1 && !dt_node_cmp(name, "reserved-memory") )
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index cf9c37153f..d0a9987fa4 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -197,33 +197,8 @@ EFI_STATUS __init fdt_add_uefi_nodes(EFI_SYSTEM_TABLE *sys_table,
     int status;
     u32 fdt_val32;
     u64 fdt_val64;
-    int prev;
     int num_rsv;
 
-    /*
-     * Delete any memory nodes present.  The EFI memory map is the only
-     * memory description provided to Xen.
-     */
-    prev = 0;
-    for (;;)
-    {
-        const char *type;
-        int len;
-
-        node = fdt_next_node(fdt, prev, NULL);
-        if ( node < 0 )
-            break;
-
-        type = fdt_getprop(fdt, node, "device_type", &len);
-        if ( type && strncmp(type, "memory", len) == 0 )
-        {
-            fdt_del_node(fdt, node);
-            continue;
-        }
-
-        prev = node;
-    }
-
    /*
     * Delete all memory reserve map entries. When booting via UEFI,
     * kernel will use the UEFI memory map to find reserved regions.
diff --git a/xen/include/xen/efi.h b/xen/include/xen/efi.h
index 661a48286a..b52a4678e9 100644
--- a/xen/include/xen/efi.h
+++ b/xen/include/xen/efi.h
@@ -47,6 +47,13 @@ int efi_runtime_call(struct xenpf_efi_runtime_call *);
 int efi_compat_get_info(uint32_t idx, union compat_pf_efi_info *);
 int efi_compat_runtime_call(struct compat_pf_efi_runtime_call *);
 
+#else
+
+static inline bool efi_enabled(unsigned int feature)
+{
+    return false;
+}
+
 #endif /* CONFIG_EFI*/
 
 #endif /* !__ASSEMBLY__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193950.345576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfF-0006tY-5V; Thu, 23 Sep 2021 12:17:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193950.345576; Thu, 23 Sep 2021 12:17:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfE-0006tB-W2; Thu, 23 Sep 2021 12:17:20 +0000
Received: by outflank-mailman (input) for mailman id 193950;
 Thu, 23 Sep 2021 12:17:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNW9-0005KP-6q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:57 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.75]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 09719c9d-0cd5-4463-94da-71b36f3bfc83;
 Thu, 23 Sep 2021 12:04:35 +0000 (UTC)
Received: from AS8PR04CA0191.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::16)
 by DB6PR0801MB1800.eurprd08.prod.outlook.com (2603:10a6:4:38::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 12:04:33 +0000
Received: from AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2f3:cafe::8a) by AS8PR04CA0191.outlook.office365.com
 (2603:10a6:20b:2f3::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT041.mail.protection.outlook.com (10.152.17.186) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:04:33 +0000
Received: from ef915eeccf5d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 AD7BA721-1AE9-4FA1-BC79-24EB200E25CE.1; 
 Thu, 23 Sep 2021 12:04:20 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ef915eeccf5d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:20 +0000
Received: from AS9PR06CA0094.eurprd06.prod.outlook.com (2603:10a6:20b:465::7)
 by AM6PR08MB5045.eurprd08.prod.outlook.com (2603:10a6:20b:e4::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:17 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::cf) by AS9PR06CA0094.outlook.office365.com
 (2603:10a6:20b:465::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:17 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:17 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:04 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09719c9d-0cd5-4463-94da-71b36f3bfc83
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bv/kyQCgDxJAB2ngKdqERIdGChRlUtb76o5kiaGFJyA=;
 b=8tVQGWrAjiDKX7u7KJroWaNvSMZkrVTT1RF7UmWLOvTMaMIACQW8nRvJNoTu7xhsngpM35RQ53wzsyGu8AiiMMHWQyxaxKxOrdMUs/lOSjT9BrqpvEjc/dc1PVmGjEdg60rE9Bi1jlQ6RPvre47Tq+aFae0PM7Spn2//ykCIPSs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fd2f19f131e286cb
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RgBOSq28pwyt6AIc/9cb/+Yiwc51u+QucM/hlhLHFzvwPBlukFtEwJ/j+Pw0M2OCILVOMfiT4oTL0tePlEIZQ9v2IiJFwnJkcWTXZW5oa7/MaOTbHhXboJQFSXoHUAicp4kcsqfJ25N2zAgdDI/bdTps9tSok907mH+iuhOKk2Q3QwEl3xA5sTTlRgZmPbMzXs8kskTfJyLUPmGSW5PrNyyigyqNPfwL/ANqF1rN4KN3pQm2Jqh0t1je0QRleKcKTgWnmvgyymO/wXjzZn5IWJJzu6PRJUseQP8+nEw7nk/oewhSPdd46Eq59P5/qChALveqejTm0+k1cFmOIc9v5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bv/kyQCgDxJAB2ngKdqERIdGChRlUtb76o5kiaGFJyA=;
 b=A4zyZLqf8yfFmidpoKTLZwN8XlrxQQluSLGbaFY2q0bHNKjBsPSlp15UiM6bUi7dr35WRgd9Pssh4vGNE0v8cZuZ7vcKtitkw/z+vzhbE67BLxTWL8LBB0iQLa9+JsQVeAEYhOVksfZ1co2Vkh3GEcmfoY4t9GHZqCYp8NEDFESsYmVKvlURSsiRAJfC6Fxwc63qxL13Oe0xDoKvhoqdHQCjYi5RYoYPET+ZcJKwgEJAGtXTpbllBFlowBHyulpV07jmbmw/SbqrCO/fro1reItDJ9stRNKVO/eLtyUSQjj6p6kczQaysjOSFYRoXD9lcWGvnBZVs93TRjCg9MGDjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bv/kyQCgDxJAB2ngKdqERIdGChRlUtb76o5kiaGFJyA=;
 b=8tVQGWrAjiDKX7u7KJroWaNvSMZkrVTT1RF7UmWLOvTMaMIACQW8nRvJNoTu7xhsngpM35RQ53wzsyGu8AiiMMHWQyxaxKxOrdMUs/lOSjT9BrqpvEjc/dc1PVmGjEdg60rE9Bi1jlQ6RPvre47Tq+aFae0PM7Spn2//ykCIPSs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA initialization
Date: Thu, 23 Sep 2021 20:02:24 +0800
Message-ID: <20210923120236.3692135-26-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 17e2335d-43fc-4a59-bb9e-08d97e8a4e8d
X-MS-TrafficTypeDiagnostic: AM6PR08MB5045:|DB6PR0801MB1800:
X-Microsoft-Antispam-PRVS:
	<DB6PR0801MB1800903BD03C6E6399E2C7799EA39@DB6PR0801MB1800.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 hH8Kge7MB/PPB0wKl7fKIbFtyh7/6prOeHhF6M7svzrk5jTO+EBNnhd5m7v0wFh88OpCOFpWDjduwg+r1AAjbWDHKFiGYyReT8JdPmtbJh1fMgDthqcERkGsbzz5UBhb8nO+7P8h771AiaAQ3cvULVzNmfcwKml1EOEjZ+ewBZjf3B+X3Gdyh60GD6EKETysQXPHwpnOfjl1/lTauVZ7xj9G+15MQwAx1d5QOFkQHJ1s4F/7psmToMk84zYlytYwASkpvjUChIZ+LeNCEcnClaRXbzv50nFLnYCGwpOHYiU69NPqZUpTguM5AZ452pjmYP2LWlfd1aIJHKJqhD0CFf56Brql/kO5f9YhUol2QEqNziVBMq9oPP5IaKJrWrVexbXZ14wREgYiytTE93Lh8DATi38VzVhwGSUoZbVv5hBaJA7QTSlo+ILhKTpIm1Azs4LhsZ0YIrlkUvU+aq4JD3Ap/p/77yDbdck3RM8cLGplpZQGnpcNBI0mRBxdqAm8bi6q2ZQo5kQXVkeajefIwD1U3cZW6Yh29YXPE7dp5sdyRlBu9biaz7qHoUPbXmUAHqx+7oLhU57yiXYIXxq9K9X+c5Hj440PKoEbD1nclBuWzcH5H34zgT93u9P399AFQ6MK7fNgdaeqC0iOfAMsV8TuuiFu721Gsr4Wp72JwD4CXiqqnNOYgaaZdjxnyTBOKanS9q8QWDI0KUHFqWjP5wuMO35TOE4yWp1Dye2Jvic=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36860700001)(356005)(8936002)(70206006)(110136005)(2906002)(7696005)(86362001)(336012)(26005)(81166007)(6666004)(5660300002)(82310400003)(36756003)(2616005)(4326008)(70586007)(47076005)(426003)(44832011)(8676002)(1076003)(316002)(186003)(508600001)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5045
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	767039c6-5a48-42f6-3a62-08d97e8a44db
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VHxLos1TlwEIj8PhHqYPfgpZeFNnJi8JiQtAnwJGB6T3afLb0CAgDhJYOfN23PVQU88cUGnCYr5xLSICbNjm0Y+BgDuLPG2tOE7nmHYolQ6B0t+haQEInCDrCYzl/CPYg0k2ZpQg5++MOmnX5NErrtAnIVPVUma+NxpxqwpvWCaXgztfz8yvPOXbSF58+xOgnyytuA59HHAWaXc67tuZbolJCWzDdjIldcOPrtJtAmicyyAD4mvhHChWt9kpYgWemAjJnKRYFXYZvEnAtI/brFgbsOrlbBp9BCccv+4THCUhOK2ZWANtzIUCOBF1jtmMh8hKgvhYvW3BK/UuX1RAu7bGJlJEBtAlt7Z2zVBUQUNN+aMsIGQQ9sKNirx4mZ61CxlO/yD73Ejdo91cBAKN+xG/KgsIQfxOx9T5A6BceihFK2ZmH9fHcUCOGoEHPF5BmDRr1+ldZEzOOA4W9YRvx3mjhInmzWLZvOMrdSVWIRk6ybn9aeZBzDEXPut/GYG3etUFCxqs2EJW1+prFUm5C5K/Sqf+YlmGAmye+CxCCj/PH6x6lMaj67Xvn8g0r1QCvCucNHq6Hvcn6Aw6gd7p74jdM+c8GWfwyMizejMFvRHZNmjen29SLSy2G0hIGAIw0JyF+P8znkUyP84UnhqUrCx6djOgJWdcigRs2w37PEMiM/wQT8TuCwxKjV+hVC2gYTqt90BNk9mQC8qLJSqgEw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(2616005)(4326008)(426003)(47076005)(8676002)(36860700001)(81166007)(86362001)(82310400003)(8936002)(110136005)(186003)(36756003)(5660300002)(1076003)(70206006)(70586007)(508600001)(336012)(6666004)(7696005)(316002)(2906002)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:33.7529
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 17e2335d-43fc-4a59-bb9e-08d97e8a4e8d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1800

NUMA initialization will parse information from firmware provided
static resource affinity table (ACPI SRAT or DTB). bad_srat if a
function that will be used when initialization code encounters
some unexcepted errors.

In this patch, we introduce Arm version bad_srat for NUMA common
initialization code to invoke it.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
index 3755b01ef4..5209d3de4d 100644
--- a/xen/arch/arm/numa.c
+++ b/xen/arch/arm/numa.c
@@ -18,6 +18,7 @@
  *
  */
 #include <xen/init.h>
+#include <xen/nodemask.h>
 #include <xen/numa.h>
 
 static uint8_t __read_mostly
@@ -25,6 +26,12 @@ node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
     { 0 }
 };
 
+__init void bad_srat(void)
+{
+    printk(KERN_ERR "NUMA: Firmware SRAT table not used.\n");
+    fw_numa = -1;
+}
+
 void __init numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance)
 {
     if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193959.345586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfG-0007CS-G7; Thu, 23 Sep 2021 12:17:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193959.345586; Thu, 23 Sep 2021 12:17:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfG-0007C3-AK; Thu, 23 Sep 2021 12:17:22 +0000
Received: by outflank-mailman (input) for mailman id 193959;
 Thu, 23 Sep 2021 12:17:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWY-0005KP-7Q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:22 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.42]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 763e903f-e63c-4fdd-ad36-f687d916c203;
 Thu, 23 Sep 2021 12:04:52 +0000 (UTC)
Received: from AM0PR07CA0006.eurprd07.prod.outlook.com (2603:10a6:208:ac::19)
 by DB7PR08MB4219.eurprd08.prod.outlook.com (2603:10a6:10:34::31) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:50 +0000
Received: from VE1EUR03FT030.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:ac:cafe::e6) by AM0PR07CA0006.outlook.office365.com
 (2603:10a6:208:ac::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:50 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT030.mail.protection.outlook.com (10.152.18.66) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:50 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Thu, 23 Sep 2021 12:04:49 +0000
Received: from 6a4a37fc73d1.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EAF65ACB-7FF1-43C9-93F8-7DF9683493FB.1; 
 Thu, 23 Sep 2021 12:04:41 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6a4a37fc73d1.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:41 +0000
Received: from AM6P191CA0075.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::16)
 by PAXPR08MB7187.eurprd08.prod.outlook.com (2603:10a6:102:209::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:40 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::a9) by AM6P191CA0075.outlook.office365.com
 (2603:10a6:209:8a::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:40 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:40 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:20 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:04:18 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 763e903f-e63c-4fdd-ad36-f687d916c203
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/Uy52E7uZCYgh1DxgAv2AGcqP59JVjmgUIyI38scwW0=;
 b=boAtcNAaol8Owam3CPCcBG6TXy7Pdcnsct5SYv/1A9wDlPRGKM2txYwrEtZnm1TAI+RbDT/v1qwUMKZ9eUCUBYWs7rDua14VappPdVm+/n+RufBH+CYd0tLTnaf9kvRNuXBKuL8hOrB34jSsltBuRvCNZ7y4twBIag3hgTaHebU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 9018c8cdcaaecb1f
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JjcsaJdjsQluXeD75CTF/v8sr0LRGFljDEWIYaJ6+VHxpYnBxByxpa9AptaOpnjVocUt9i+5bJDVn+vFIgA56Tlr4uQzFOVuYLaZK0zZZKRlyTEkRlGr1ECRzZt4/3e68HgIwOkDvYNmHVSAE0XwGu7VueDUXjPnnkU2IBrcfzrRqa0zafcOi1Ihqx8xaSHlIFzTE5O030V+KKIz4yyp5DDf3kkC6CjfzOHxJUSMIHc+LQ6fdYHYE3e9+g1F8nzD/kd/dskFapcsWB14C6W+CTVLiIK3ZZzo0nMzxxND4m/HnBQM/q3HQhZARxy6Fd3AhsNeIdLVCM+aQ0IvoEf6kQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/Uy52E7uZCYgh1DxgAv2AGcqP59JVjmgUIyI38scwW0=;
 b=gKPiSHe4B/QGe1oWddpxjfk4L7U4g/jMKtAhblCOf0l3oo6UV1rIQ+crjQfyjtNk/Jw+DpUnunTETYcdMpSNVzjVNGFzLVTdKBCbPu9Xl/uh35a8octOhVzHZyqbeJkajJmWBBagwEHpC+NvK8+UnBKjG7wUowf3tez6YYaILJxjjdiOera6rC87C2muTDveimR4yHDtb2iMPzXSlq0T9Hu2ag+sEVExJcMZt/6AqMlI0QYNIlc/QFZjBcIwT29giD9kT1qNMuPr7YaN+R98IIJG1COno1c9fALpwyoh3ES7rCHObAyIiyDJ/f4O/0I7plnD+AS7Uc15JOSNjXZNmA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/Uy52E7uZCYgh1DxgAv2AGcqP59JVjmgUIyI38scwW0=;
 b=boAtcNAaol8Owam3CPCcBG6TXy7Pdcnsct5SYv/1A9wDlPRGKM2txYwrEtZnm1TAI+RbDT/v1qwUMKZ9eUCUBYWs7rDua14VappPdVm+/n+RufBH+CYd0tLTnaf9kvRNuXBKuL8hOrB34jSsltBuRvCNZ7y4twBIag3hgTaHebU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from device tree
Date: Thu, 23 Sep 2021 20:02:31 +0800
Message-ID: <20210923120236.3692135-33-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d0deb3f8-9145-4d00-f441-08d97e8a5896
X-MS-TrafficTypeDiagnostic: PAXPR08MB7187:|DB7PR08MB4219:
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB4219C193E81EB523CCABF8DC9EA39@DB7PR08MB4219.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 /XyZspcenSR+Xfp/bqrbh8RJhaBTqQIxeIXfbOEXA18fWLrHld2s7bm6jQH85duPf3O7OTBe+VkabINbquY35zYYMpN4v1fz2RbfwzBusOC2a2VU+taIsNtPTIun9lgJHNirlrx/y+bFwmzeyrtbvQExnhZgwJMie+7H30NBveONWyveLy1J3tsEiezkwLNpgJeeQm2jN86emK8yzO6BqN6Xz+X7LRGczfEABK2WsOgJwuj4DyTzmzJ3hjgDSOU3EgJMRP+zyXzNuB1qjpRWJxppM9rlLfRnJZ/pXpyIM8jPDcPs2d53yjH8PmlRo1qXIBTO39e65fRaUz8IRzDIhVDihrIUQxXw8KHb6yhqOUhMYjBAA7DlIBj7LQpigu/BQZ2Ea/I5qSleXeBU7T9j8X8L2jU+hjwh5XzzVQWgAMz+vrVxLhPdaDp/NFGykfk/krAHVsP0iYOZ+/2Z/prz6WSYuCUkbKRQdr4YXokwoQJvGS8N6mk/l86moDCyTU4dvq0MIjo6WI5B6EBy4h4Lu3hdiHtpAJiGu7ddo4fm1LctjaHY2FugzuNoGkVVmWqaxptLw5gRBa9Jfs20gLc6oOvAiiEXFWJAa1w9yIUbAUbf4YoTWM1g6DNRCWtF+svpYERApK+Am94UpfQ6YC9GTutWmLUVDSQ59bh2cUTgXeRvLJYM6s4+qcLokQ6G2RwjwTuSw2uBGwpftGOqwyeB6be/Ttj6vIzhUR/OuF/NdEg=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(110136005)(2616005)(1076003)(82310400003)(316002)(336012)(47076005)(81166007)(44832011)(70206006)(4326008)(70586007)(26005)(356005)(36860700001)(86362001)(8936002)(508600001)(83380400001)(5660300002)(2906002)(186003)(6666004)(8676002)(7696005)(426003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7187
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c8b70389-a79a-4536-ebe1-08d97e8a5281
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+nuLdw1t/NfZzBnKYdvlWqFmM4OA2Hw1HnD8/QIx+SMu9csE3LTwvKgL5uSBfr/QG8vbXaHXKh5b28l3gO0zb0NsVuncN42hxyWsziD5sIiZ0/CQrzGDRT2EoHR/OaJobBYL69uU3e1HRsysAdDuDzsePffD2tFfBbiJBQVkXl87oKeWwrkSzDbNSPHhJ6XI/SNJ3KOE0rBw5zU/xs5lcXFekGQADxABXxSd3KR9jrC5IJrDslmnMyfmnwIsUy+4rjAFmZb1sQY+tUifgvo2SopWYPuPs28Lk9XzOnIgBD3bFRvbVUN35b2I7UpgP+Qg6VtmkdSFUlfYTNHUaS7NUwJBeN62mVIVj4BAnr4fDvKWaavNAxKvxfLIRDO9RQCKjL4o5sq7aE0JQHPjMHw2MTmbrgUvPGfGbJYno0udx1YLskWTrUYctqxybleJmAWdq8FrkqY9AwGTnSuoZDIhWeN7NV66YS4Tx6rWMQhX+EM/yhS7o/G2R8bxAdBtjA70Z6Trx6+WZva6inGkHhB9+OWolTLejbqXF9/2NdT7sU2U/BYGZyZ4vG0qW7fKf6VRf4qc+w0++gKJcFLGDkPqdFMEt4PG0rQtRmi/WxfS3XDF2Ktbh8/jEZRUyopfKlCRI1A2KOvWBA0WYE+sk9/NpyMgtxXPDma0epDTWLSuLE45CVo9NDiJHTcnY/xHgD53JmPysW3niWzSqr0SMhBdTA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(86362001)(508600001)(1076003)(316002)(110136005)(2616005)(5660300002)(8936002)(186003)(70586007)(36860700001)(4326008)(26005)(47076005)(336012)(6666004)(426003)(82310400003)(70206006)(83380400001)(36756003)(8676002)(81166007)(2906002)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:50.5091
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d0deb3f8-9145-4d00-f441-08d97e8a5896
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB4219

In this API, we scan whole device tree to parse CPU node id, memory
node id and distance-map. Though early_scan_node will invoke has a
handler to process memory nodes. If we want to parse memory node id
in this handler, we have to embeded NUMA parse code in this handler.
But we still need to scan whole device tree to find CPU NUMA id and
distance-map. In this case, we include memory NUMA id parse in this
API too. Another benefit is that we have a unique entry for device
tree NUMA data parse.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa_device_tree.c | 30 ++++++++++++++++++++++++++++++
 xen/include/asm-arm/numa.h      |  1 +
 2 files changed, 31 insertions(+)

diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
index e7fa84df4c..6a3fed0002 100644
--- a/xen/arch/arm/numa_device_tree.c
+++ b/xen/arch/arm/numa_device_tree.c
@@ -242,3 +242,33 @@ static int __init fdt_parse_numa_distance_map_v1(const void *fdt, int node)
 
     return 0;
 }
+
+static int __init fdt_scan_numa_nodes(const void *fdt,
+                int node, const char *uname, int depth,
+                u32 address_cells, u32 size_cells, void *data)
+{
+    int len, ret = 0;
+    const void *prop;
+
+    prop = fdt_getprop(fdt, node, "device_type", &len);
+    if (prop)
+    {
+        len += 1;
+        if ( memcmp(prop, "cpu", len) == 0 )
+            ret = fdt_parse_numa_cpu_node(fdt, node);
+        else if ( memcmp(prop, "memory", len) == 0 )
+            ret = fdt_parse_numa_memory_node(fdt, node, uname,
+                                address_cells, size_cells);
+    }
+    else if ( fdt_node_check_compatible(fdt, node,
+                                "numa-distance-map-v1") == 0 )
+        ret = fdt_parse_numa_distance_map_v1(fdt, node);
+
+    return ret;
+}
+
+/* Initialize NUMA from device tree */
+int __init numa_device_tree_init(const void *fdt)
+{
+    return device_tree_for_each_node(fdt, 0, fdt_scan_numa_nodes, NULL);
+}
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 7675012cb7..f46e8e2935 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -23,6 +23,7 @@ typedef u8 nodeid_t;
 #define NR_NODE_MEMBLKS NR_MEM_BANKS
 
 extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
+extern int numa_device_tree_init(const void *fdt);
 
 #else
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193970.345598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfS-0008I6-Us; Thu, 23 Sep 2021 12:17:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193970.345598; Thu, 23 Sep 2021 12:17:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfS-0008Hv-QZ; Thu, 23 Sep 2021 12:17:34 +0000
Received: by outflank-mailman (input) for mailman id 193970;
 Thu, 23 Sep 2021 12:17:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNX2-0005KP-8o
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:52 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8ca9c1d0-3899-468a-8635-c1a380dc1cd1;
 Thu, 23 Sep 2021 12:05:06 +0000 (UTC)
Received: from AS8PR04CA0205.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::30)
 by VE1PR08MB4990.eurprd08.prod.outlook.com (2603:10a6:803:112::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:05:04 +0000
Received: from AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2f3:cafe::63) by AS8PR04CA0205.outlook.office365.com
 (2603:10a6:20b:2f3::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:05:04 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT041.mail.protection.outlook.com (10.152.17.186) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:05:04 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 12:05:02 +0000
Received: from 03d3309475cf.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D43E9940-4716-47EF-8348-3EBC63C515C1.1; 
 Thu, 23 Sep 2021 12:04:50 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 03d3309475cf.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:50 +0000
Received: from AM6P191CA0085.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::26)
 by PA4PR08MB6256.eurprd08.prod.outlook.com (2603:10a6:102:e6::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:48 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::b7) by AM6P191CA0085.outlook.office365.com
 (2603:10a6:209:8a::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:48 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:47 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:31 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:04:30 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ca9c1d0-3899-468a-8635-c1a380dc1cd1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZVsq3Ppcw/iW83lpjuNFJbIFMRjYaGc0rT7Th+1FJeA=;
 b=ixk63QG1kWaz3bXLoDY6betbAhNtXdNLlRX8sE+M9juzzsGlDDuSpdyzEo+xnZ0JCI3La2xh2f/rPNvAxE7xMdodMKxpsfbkxbpmKTlxeqlvQqaXE/Y7HOLCcd7xD1tfUfhpd61EibdBWbAgvc+rAg6xFWYXghenk6ev5nhMvbA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 302f1b9d54d65633
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SGA3SDr9uzmdia9ASGcsVX8tnaq00WIBwuTPkwakvbClze7wp2Dnq1d4mXMQWU3HBuht7AP+YNnkSIDzlqjneR9xyR44C1y885p3XMy2mB5fCxMhGKHe8oqne+68MqjhbBdiLfADaGIzrFcMIaw48WAvzJ3zPQqFbMHOqPO1M/OKG6Z4XhZHvEo32rIUmwYAS21tAg+KiWfcPmHw6MRw+8QONgyRlyaRj7zcwO8q28/XO7gwRQSR27oysHPKNEPPglJhVuIZss5e3E5VPZxGSFMIwGLf9VZiq8YHgcBOFX09RDtJKm7731vYojbvva3++X8TJENc34bg4HnNvNsEmw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZVsq3Ppcw/iW83lpjuNFJbIFMRjYaGc0rT7Th+1FJeA=;
 b=b0108p3kECqYWhgw93J6e9EZ0DzYyKmhK5+qUjh7bCd/42AzpvyHwo/hInENPtwZoAKBxI6QmoXJPiwBIE2pBZbKe3v/OEaaymn0qoePTxXXZCiSvyjJYcm12I2fVcce78TdP03oahI/36hgMheHHJ6fRQZ95mlPdaPKhzne8dt4MNxeLLyqb3zYBzZ5Kma8euE+eIIREKuWrwhxb3VKLQcxJnydyaaPsxuhX5bxfsaEYWDmKXNdYzLGAZyaIOMcmSln8ckNkqsO0t7XBJJn66/s1Qx8F71NHFrmqjZ5K6JgFiUqSE/A6AXlxehdldEASTSolrzCuYqhGE3nQvPS6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZVsq3Ppcw/iW83lpjuNFJbIFMRjYaGc0rT7Th+1FJeA=;
 b=ixk63QG1kWaz3bXLoDY6betbAhNtXdNLlRX8sE+M9juzzsGlDDuSpdyzEo+xnZ0JCI3La2xh2f/rPNvAxE7xMdodMKxpsfbkxbpmKTlxeqlvQqaXE/Y7HOLCcd7xD1tfUfhpd61EibdBWbAgvc+rAg6xFWYXghenk6ev5nhMvbA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 37/37] docs: update numa command line to support Arm
Date: Thu, 23 Sep 2021 20:02:36 +0800
Message-ID: <20210923120236.3692135-38-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 501d9ee5-7f18-417a-1e0e-08d97e8a6097
X-MS-TrafficTypeDiagnostic: PA4PR08MB6256:|VE1PR08MB4990:
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB4990145D89D2E6DD8F5FB0509EA39@VE1PR08MB4990.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:1923;OLM:1923;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NyB0pYDfej+Hc1Jtckj9ywjtSegCo4lZBbn+mZQNDgGZviVn3Q+lufG0cJwVJHXjCrlcz1Lq5qTUHGBgzqcnA/OcjcIGe9dNMfC86pxlcV6crfDIiXoFFwxZjJ9I9uumGvzwf4cBFdHQbf7F+tFg1YcafnPibU0ElWC4LNhzTkQvDGP6m39N/0e1V4WVbmHMvSL3i+LpJhlxe+BleVeYj8/98UrTStJTZsbKKE19Z8rUMlyyl7/zBaTjsserkeMvdsYIjHL1ZFUyIxuzedrLfZYEyI0OMkYKVHpgExmN2YWZ92yxoDpnmDb0tnymBwI2/xmxNdAtg7oQG8t43TjzeOQb1xdNHbJ7BW2e5QdeHA7f1q/xklf9nlhuawOFs6ocVH6BuJcp1cZl7uE+kFUl7Eom6/zyPG8ov0qeAQ42VOtwP0ssK+tSZWmZtx8UOAALRpHvluBhUxts0Fs6k/mXvH9X0C1BnTHvhvO4eVJ/2M5LPEeaycGksvL/mbYmP7W/vYWiXzO2XfzviphjjZPv0dytdyj7dWS9p9DE8vMGmDRzy818mudNHwS33/wMPI0aIB2QvJkQOFBq5bWODvA2CeSkjY/13oG1HqpjYZWFOXgrWK9wuvD7t5VvlkI1ifTJ5/Vv0AfMNse63Ln6xVbqXPeJoAIBn5FbBZhge1MvY+ktYRaqxCxjrwoql0ozr0x44IXcc2tHMBUmS/B8AQ+J8+DDX8FjLkJrcxtr/+gqvSU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(81166007)(1076003)(70586007)(5660300002)(2906002)(86362001)(508600001)(356005)(47076005)(70206006)(110136005)(36860700001)(316002)(2616005)(8936002)(83380400001)(7696005)(8676002)(26005)(186003)(36756003)(6666004)(4744005)(336012)(426003)(82310400003)(4326008)(44832011)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6256
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a893dde5-7dbb-4cc5-1bd6-08d97e8a56d4
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xmnnYWr0A8FOmDIH440d/ddiZ1AcHHKqkoxL4qWI5Ax5req0SAefWu7+psjJIvWDkgaN7B9k0kajYDY8t/H+P3s+dTdQDBzMEq0PROM9IuHU35/GiheScg1htyE9B7BcpvkjeIxzT1FPY82eHWwhE6/adBee241R0XAPtTTHbeEZYsNQUDvLiJ7TtlN1tfIKTO8FLtoSYpin72SimSWYdwpRI7/dUut31fb9mqpOJU4mSWi96mIAQ9PaETu8SsG64IiiqzlZr3OAbc5q2J8xdhf8Cu2F09o5d3QA+fV/NpMJj+tXh/a2eG6w2aLiXsJy0CQGnLCMDz+yntumzgQkqSvyFuo/2XIB9Ra45MOr3huaMbVxaAFjl745Eqm2/t00xzY97MgUkvmytT3uOp+QLNn6SaqsCdL/Bt7s476UWaM4YecCxJybG+9rFuqED4VqShzjudNYWpszO837hfeVoAqIdjm8rnVts166Ik3uQuEKbd7qTW8QlGrblqgHftNewz9bKTiquKwVWHQpEyCi7RmnKQ0zJcywhEAcjUgjvlOWgf9DI8Gnt57rDj+eCyOQetag7LT5Vixsb4NVzLH28IjbzaMvl884ITDUU+LALPzXsuH4FjahDh6B+8pgDML+NUFDIFmfxqg44lr/3fIDFizFQHvLywJ0iD3Vt0Su3Cp+jpnTMmzmJNlPqNEe850dGsbkFhx/qtYuGg4F8/GGvQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(508600001)(83380400001)(2616005)(186003)(26005)(426003)(8676002)(6666004)(44832011)(2906002)(36756003)(1076003)(336012)(70206006)(70586007)(7696005)(86362001)(110136005)(316002)(81166007)(8936002)(5660300002)(4744005)(47076005)(82310400003)(36860700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:05:04.0124
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 501d9ee5-7f18-417a-1e0e-08d97e8a6097
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4990

Current numa command in documentation is x86 only. Remove
x86 from numa command's arch limitation in this patch.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 docs/misc/xen-command-line.pandoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 177e656f12..4f3f24eb9d 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1785,7 +1785,7 @@ i.e. a limit on the number of guests it is possible to start each having
 assigned a device sharing a common interrupt line.  Accepts values between
 1 and 255.
 
-### numa (x86)
+### numa
 > `= on | off | fake=<integer> | noacpi`
 
 > Default: `on`
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193974.345609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfb-0000Yp-EQ; Thu, 23 Sep 2021 12:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193974.345609; Thu, 23 Sep 2021 12:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfb-0000YQ-8U; Thu, 23 Sep 2021 12:17:43 +0000
Received: by outflank-mailman (input) for mailman id 193974;
 Thu, 23 Sep 2021 12:17:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNVu-0005KP-6K
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:42 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.57]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 688aa582-8a07-4358-9127-1a73f69cb49a;
 Thu, 23 Sep 2021 12:04:32 +0000 (UTC)
Received: from DBBPR09CA0018.eurprd09.prod.outlook.com (2603:10a6:10:c0::30)
 by AM6PR08MB4423.eurprd08.prod.outlook.com (2603:10a6:20b:bf::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 23 Sep
 2021 12:04:30 +0000
Received: from DB5EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:c0:cafe::cd) by DBBPR09CA0018.outlook.office365.com
 (2603:10a6:10:c0::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:30 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT015.mail.protection.outlook.com (10.152.20.145) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:30 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 12:04:29 +0000
Received: from b3dfbd92560a.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D55D4007-D26D-4147-A030-86248BA07E8C.1; 
 Thu, 23 Sep 2021 12:04:15 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b3dfbd92560a.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:15 +0000
Received: from AS9PR06CA0110.eurprd06.prod.outlook.com (2603:10a6:20b:465::29)
 by DB8PR08MB5387.eurprd08.prod.outlook.com (2603:10a6:10:115::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:14 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::4e) by AS9PR06CA0110.outlook.office365.com
 (2603:10a6:20b:465::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:14 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:13 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:53 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 688aa582-8a07-4358-9127-1a73f69cb49a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LT0n1SFAvgK3Jv0H4JFQYnKlHEfutbmGqdGdC9djIAI=;
 b=b9ufI3LQ1/Tv+xEHMCAlr66hy75m+aULqpElhnB4AEO8yffhvCMz7ujCsBy6CLJIXyQHPwBnTuZ6H+UTfbW75k+90N8pEJ94jhcUMZvw0fTOeu2QjoSknNqvcj3ERwzMZ5hgvQbwxsfxLd5uW+rPfOnD2fcov3L79isTlse+Pm0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 842a8a7af6a85634
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WPm7kFPrkfvNHLp1jVs/Dj/yQpBqiTqy9fblwKyVzTjmqsbVrTvubZydMEq/uuXqHRqx+sHv9zrETxx2yhTweWWnU47bQ2RcbhX6lSVTpm8SPWzBR4fyIDMPVNSnAPaCMNV1yir7xSjejnlUhVeN43WYIyJkmLGNgl/BAWp+YErsQj8R9dSLPxh7QSPxDwh63oBiq0zAWKefu5XMYM1HJuKze5L9mqbgi9nq9lV/8b1sUYag4ccsdcjTpIgg03ntlGQKh9gt6Q0Nl084+v4XzxkbHbval9t4upktzCfTLq11FsEbOTzJSggCeYYiOIlSaSnVg4fX4Q0Vxk/gsI+xEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LT0n1SFAvgK3Jv0H4JFQYnKlHEfutbmGqdGdC9djIAI=;
 b=PzH9vR8mxqXjux9n95YDETh9Kl8ilbDO+K/9Ewheqq4TVcx8u93my/OOvl8CnzvbNC+cRD5cPRVv8wRAJRlpHBe+SkBpuY1v1VxbZxmUHZiRRSfGY640tKTkVfDEbQQa8KcsCZC8enpiAHhVVNJNQFm/3O/qzz8dx0HJ0JVcYVSXeeT9JlOYHfSJxQNt77zlAQ5cEybz9j4DinmwU3c2LSrDFtTZKX52Iiz/dmZdY2sJZwYwVEscQ8blBdCOdhkr4JkxWC+BJgtrbS8M/0Meuloduw9SRpKonRmC2SCdAFF2v+zKpKdrL5r7gv7W7LKc1DZzRrxDgsx80K0hj/TBkQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LT0n1SFAvgK3Jv0H4JFQYnKlHEfutbmGqdGdC9djIAI=;
 b=b9ufI3LQ1/Tv+xEHMCAlr66hy75m+aULqpElhnB4AEO8yffhvCMz7ujCsBy6CLJIXyQHPwBnTuZ6H+UTfbW75k+90N8pEJ94jhcUMZvw0fTOeu2QjoSknNqvcj3ERwzMZ5hgvQbwxsfxLd5uW+rPfOnD2fcov3L79isTlse+Pm0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI architecture
Date: Thu, 23 Sep 2021 20:02:19 +0800
Message-ID: <20210923120236.3692135-21-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 526ce2ac-6d4e-44f1-b7ab-08d97e8a4c98
X-MS-TrafficTypeDiagnostic: DB8PR08MB5387:|AM6PR08MB4423:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4423D71E54E6720A49AF42EF9EA39@AM6PR08MB4423.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 9jObQxWYN9620scgtKFIYTHJuMrjm5Bk8hWQ2NRflQm3oojfiWWq5iJ5s4UxIU2uBDyHR7x3Lnytub4syAIXOJGJNgpAOx3YlsmC0LjziT3NnmZVgJyJYPNwInPv5Qq0hEFZGGo2vcdKPiuFNRq5OBG2aj34nhiPVgQ4aB3hZylLAEBGt5Tc9DuH2vSMxqZNcmP7wKiHkL/NY/BjCBqIlklXcTboFWFpoczU7a/ZxeaNNRVbSSQizwOaJu+LoDULtXH6FZUyYAieW3gpdCJFbb2U/ru9ZDs/jN/f5Y9XBlJxc3MENIjy64+Ee/5n/AXmcbYPaBj/LyxMk6m3bRNceuK5Wo5raeLcrxIbuNJBQyIpvkEgQCu1Io0wC7/fVxlop9QCjNdkRQFRLHa9zywvsmDpY8arwZ8/KANtw+TrahSCs4gIOOeGLFLxpgQ3GIpwjadhJCG5iSmfOarZqgx0Si6msjiGOIRJEDm+ySguglMAaDH/O3JBZVUdVNGk6klyGau/abfoOniteNwbFWfXX1GTIfjmtj/k9UBhnBfEfWReOxRXM+UxCHJGuE2cNJ8riMFodTibxfBPaW+VJB+HhfFBj0IgyXfRMmFHlIIvCvOkKnT/gL/8tzKEJ7MqkL7gALN7r/tzLHtGwk8iyaqHhMFl/K8yHAl6HbE7uB/nLjgq3Sz3HhH1ZpT133QBCaf94ZsFphFqqwb8ag+C0EEjDbJojqn9GCmb56Cxo1YJsVw=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(47076005)(8936002)(316002)(82310400003)(70206006)(70586007)(6666004)(36756003)(356005)(1076003)(36860700001)(8676002)(83380400001)(5660300002)(86362001)(26005)(508600001)(186003)(4326008)(2616005)(336012)(426003)(44832011)(2906002)(7696005)(110136005)(81166007)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5387
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5b603ccb-8d6c-4b1e-1089-08d97e8a428e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NJWgKKGtYEBLDdw4o2hBj8Kqs5UeS1kwojL78S/J19Tw0xqIJqMBiy+1TAEPlctUhi5p4keGur6yXVeD1EYJojvtRtOyma/EuRwZjZlrGNw7lsV0mQjEPT3Q2ZrAdZJUCA9/4boBnH9UjxFcjj8/NhsFi0aTPAMlgjfGj9c215mcT9mhYc4mBq0RPc051gRSipaI6ej1iBx7MhNwnDFCdqiq1HVEWxUibA2tYmvqO/vB4RsXrxtbTipZfw9W//7sY5WdYUdQnsLtiMn+9Clr633CeFOEmkpcie9JhOAU1kR1Y2D1yJeH31K1VIumvLzSLx992fJqf4S/0MuXU+u7k5iAgGwjtM95RE5s5ApR4TnxYbzeMmtwI0+as/5Y2aMCRyvp8isXEvOph7jluOlezSX7/dKijxzKSxS4egKJKyiYWGEfjcHrxs/+MXE08BF0zui0JRG9QBjuoN1PlWZrkQ+ZL+fTHGASxhdlPG5oKA03LQNYf0W+TAKNEVM46DYajofKiGSGnL1NTe3zSlcUA6Rw1+EvAii+xC08KHP1cKuKGl1GF/yQwXYzQRBZn8yijcr//DzSH5gt8kfis+yH9Bh+J1lCjyQllcGU+VNfBVEh95ocbqaRMEDH10WkMC0lPvl12fpeeh5CIEOE42cSOT1uItXqukKyCyLltlEPELisSuuUcOu6nbaX/2xhLty45AZ6F9JS26FNgceP4z25gw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(4326008)(110136005)(316002)(36756003)(5660300002)(81166007)(8676002)(8936002)(44832011)(2906002)(86362001)(26005)(83380400001)(186003)(1076003)(70206006)(2616005)(70586007)(47076005)(336012)(426003)(82310400003)(7696005)(36860700001)(6666004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:30.5229
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 526ce2ac-6d4e-44f1-b7ab-08d97e8a4c98
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4423

Some architectures do not support EFI, but EFI API will be used
in some common features. Instead of spreading #ifdef ARCH, we
introduce this Kconfig option to give Xen the ability of stubing
EFI API for non-EFI supported architectures.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/Kconfig  |  1 +
 xen/arch/arm/Makefile |  2 +-
 xen/arch/x86/Kconfig  |  1 +
 xen/common/Kconfig    | 11 +++++++++++
 xen/include/xen/efi.h |  4 ++++
 5 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index ecfa6822e4..865ad83a89 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -6,6 +6,7 @@ config ARM_64
 	def_bool y
 	depends on !ARM_32
 	select 64BIT
+	select EFI
 	select HAS_FAST_MULTIPLY
 
 config ARM
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 3d3b97b5b4..ae4efbf76e 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -1,6 +1,6 @@
 obj-$(CONFIG_ARM_32) += arm32/
 obj-$(CONFIG_ARM_64) += arm64/
-obj-$(CONFIG_ARM_64) += efi/
+obj-$(CONFIG_EFI) += efi/
 obj-$(CONFIG_ACPI) += acpi/
 ifneq ($(CONFIG_NO_PLAT),y)
 obj-y += platforms/
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 28d13b9705..b9ed187f6b 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -10,6 +10,7 @@ config X86
 	select ALTERNATIVE_CALL
 	select ARCH_SUPPORTS_INT128
 	select CORE_PARKING
+	select EFI
 	select HAS_ALTERNATIVE
 	select HAS_COMPAT
 	select HAS_CPUFREQ
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 9ebb1c239b..f998746a1a 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -11,6 +11,16 @@ config COMPAT
 config CORE_PARKING
 	bool
 
+config EFI
+	bool
+	---help---
+      This option provides support for runtime services provided
+      by UEFI firmware (such as non-volatile variables, realtime
+      clock, and platform reset). A UEFI stub is also provided to
+      allow the kernel to be booted as an EFI application. This
+      is only useful for kernels that may run on systems that have
+      UEFI firmware.
+
 config GRANT_TABLE
 	bool "Grant table support" if EXPERT
 	default y
@@ -196,6 +206,7 @@ config KEXEC
 
 config EFI_SET_VIRTUAL_ADDRESS_MAP
     bool "EFI: call SetVirtualAddressMap()" if EXPERT
+    depends on EFI
     ---help---
       Call EFI SetVirtualAddressMap() runtime service to setup memory map for
       further runtime services. According to UEFI spec, it isn't strictly
diff --git a/xen/include/xen/efi.h b/xen/include/xen/efi.h
index 94a7e547f9..661a48286a 100644
--- a/xen/include/xen/efi.h
+++ b/xen/include/xen/efi.h
@@ -25,6 +25,8 @@ extern struct efi efi;
 
 #ifndef __ASSEMBLY__
 
+#ifdef CONFIG_EFI
+
 union xenpf_efi_info;
 union compat_pf_efi_info;
 
@@ -45,6 +47,8 @@ int efi_runtime_call(struct xenpf_efi_runtime_call *);
 int efi_compat_get_info(uint32_t idx, union compat_pf_efi_info *);
 int efi_compat_runtime_call(struct compat_pf_efi_runtime_call *);
 
+#endif /* CONFIG_EFI*/
+
 #endif /* !__ASSEMBLY__ */
 
 #endif /* __XEN_EFI_H__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193975.345614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfb-0000bP-Sc; Thu, 23 Sep 2021 12:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193975.345614; Thu, 23 Sep 2021 12:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfb-0000a3-Ir; Thu, 23 Sep 2021 12:17:43 +0000
Received: by outflank-mailman (input) for mailman id 193975;
 Thu, 23 Sep 2021 12:17:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWT-0005KP-7N
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:17 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.79]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 69d98c97-e649-444e-b6ac-6ce67b78bbac;
 Thu, 23 Sep 2021 12:04:49 +0000 (UTC)
Received: from FR0P281CA0038.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:48::21)
 by VI1PR08MB3296.eurprd08.prod.outlook.com (2603:10a6:803:3e::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:43 +0000
Received: from VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:d10:48:cafe::d) by FR0P281CA0038.outlook.office365.com
 (2603:10a6:d10:48::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT028.mail.protection.outlook.com (10.152.18.88) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Thu, 23 Sep 2021 12:04:42 +0000
Received: from 66052b498c8c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 681A7DAD-D5D4-46CE-BE94-CA1844AD4D6B.1; 
 Thu, 23 Sep 2021 12:04:17 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 66052b498c8c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:17 +0000
Received: from AS9PR06CA0103.eurprd06.prod.outlook.com (2603:10a6:20b:465::21)
 by PAXPR08MB7203.eurprd08.prod.outlook.com (2603:10a6:102:20a::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 12:04:17 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::94) by AS9PR06CA0103.outlook.office365.com
 (2603:10a6:20b:465::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:17 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:17 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:01 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69d98c97-e649-444e-b6ac-6ce67b78bbac
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wIObFkd9jRKDqLY57XelL0JAnNTNC5COXU4N00/2A38=;
 b=74aQ+7KDAnTF9+j6xpUlrEN85gY6syIWqjKk8JqB+0q9rHbM8o52OeChSlj3+AttjAp3KHzRFx5mFZcI+mGJ1jvmYg6vSKLMqxIkonYk+ZLMobkKDmt37IS+Eyoa0NhVTvE3Es4kg/AT+bLuLE64HBqRCVv8blPc+gqMFnrNV3c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 567f3102161f69e4
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WP/zNJPO5KyDYCe5/wxO9umB3FvU5vtY2wLmCYzdV/UVFJqM/AAJyrGoumys7NZ8YzVivc42yqbzJNn6QOhXBYvPj27TivQS2E/a2/uIxyXPxjyA2UIG+SgjGz2mf4+DONnMVU6I7L8MW+Sc8VgnOHpt1lArHnP2VfCSbcRfmFCFu+I4R+IaA92WCUNYfiL/GoT0BBOz5eHSrEs66d56SzOYWiHbulrVk4yWBPxYPjOT0TEqMC08EL+QQHKxklEor/UHxolsAvVnhnCdvvG1OXclca7LOXMvw0Nt8TsRE7gmOF6WNrN+BbtPimLF9kyxPmzloAOt/A2MU3QMUEJv7Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wIObFkd9jRKDqLY57XelL0JAnNTNC5COXU4N00/2A38=;
 b=LBGWFaKxmXwylkWgvWOqpLhpoyn0jCAW8ZKB/7fLBKpZzawEdOAmjMqrsRdP/+reSNleIwRbFT+42PgeF7Wp+g6Wd5tJX/N7tfueqU7l7uVOLgMKiq+GTRH57MXcAkpImXvOH333uCNQ0apRroU1HIDBzyFvt0suwq5cyLrJnt/20OghCZ5IwWS1J7fyTcAyhIWcC3ZrFFIQxSC6qtjvJWzSSFyLC1eM8WsHVLJd92POKycWfnewJBJHt3QLPpZqQLDPWcotTQ4K/HeziSwyalOTmk3q8OyExlKVKelE93xLrTkLN4pMVi5KR+PMhhtL49tAC1sK4zLQjywMTzcIpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wIObFkd9jRKDqLY57XelL0JAnNTNC5COXU4N00/2A38=;
 b=74aQ+7KDAnTF9+j6xpUlrEN85gY6syIWqjKk8JqB+0q9rHbM8o52OeChSlj3+AttjAp3KHzRFx5mFZcI+mGJ1jvmYg6vSKLMqxIkonYk+ZLMobkKDmt37IS+Eyoa0NhVTvE3Es4kg/AT+bLuLE64HBqRCVv8blPc+gqMFnrNV3c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 24/37] xen/arm: implement two arch helpers to get memory map info
Date: Thu, 23 Sep 2021 20:02:23 +0800
Message-ID: <20210923120236.3692135-25-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9bac5a6e-2d53-4269-219b-08d97e8a545c
X-MS-TrafficTypeDiagnostic: PAXPR08MB7203:|VI1PR08MB3296:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3296161D0C0BE2E6867AB78B9EA39@VI1PR08MB3296.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:411;OLM:411;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 XLLsowjDBurWKits5jk3cvABClzJgMkFxYytr5n51No/kq7tNPXVnCNL7+LzDWFFlaX7jewOWK9o8l0bWINhDmpaKdJUKQaMpZuDPJrq1+/EquKOH+ywFtZSrpoYbvwF/mE6RGW+ORNhnzh3XSwG+8NPcWNIe95MrgU2/zHGxKemfw6R8SA1xcibumFfTPxUEfbkHq7wKRg3euJ+PJvngxZ1vJisGh9meMlzxfXLd1WvVjzYYOg+i/XzZMnJCtIwHncMG+tbFXslHq8NzsDwRH6sc9j5YyAxliYzg1lWMrnwl0VujqD5thPtoub1peRanlGfwDoao53mYO+KSWVglHYiCFmaMprgNUS7ZLRYJDEJEOcGIbuSzlVVCGLNrdZr0EEACN/1YGnvD/6Z50TuO09JIf5xdoyjD7skT4f8GvYpI2/XLlkXtGxn+Q+WunffW3ACXtJ6rluaIX6/B2DpUK/7m9oh6QYBxnsPI7NgAYpxDDDIanDnI3Rl/h7N5lzf4GHZv1UI+xanCLSL5vZ7jaSMGExxrBzMJu9LanSsHsgZV0Jj/YS3jdJFCHKONmK5qpKFFIpH6Mw1KSCjzEMrYphBvOPrnoloRG5ICuXj069lFGSTnpCvyEDgkzb1px2Qdy6bRmBZXelcTlbGLil38DNfyE57ksjgRSDQh51mApK3W1py7EUFtW4sndnHYG5UD1k9F2Y5Jq5GP7YQl3weze3tW8nYR4F4BpUolp/6Nx4=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(426003)(2616005)(508600001)(26005)(7696005)(83380400001)(81166007)(356005)(186003)(336012)(4744005)(1076003)(70586007)(70206006)(5660300002)(82310400003)(47076005)(6666004)(110136005)(36860700001)(86362001)(4326008)(2906002)(8676002)(316002)(8936002)(36756003)(44832011)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7203
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b9d3b4a0-823a-4d7d-8bde-08d97e8a4493
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	I/qx8ut2212vOdMajIPRJ9jKTwNIQOJvng4oJTay4805szLCvoZa2vJFU2cPIRMReLdNavh+EN/09ibxp1GArmd5TL/f0hfhFxqofhQxqfyMnu7kBaZBPgfs+2RQc9px6hESekjI7gG4bpFVN8Svnkd8pCPVj71aaghpkkayD+g28hYFs+8+Xg40iNAxu1rvUmLnF+gezmuDc83NEpvT1+B57ItJy44Wca84+OrRLk79to00SCqSFiyTNyqhOhw3+CJyyCDOA177gzC6Z+pQP96THfsVsv8eMJ6qJDW359sBIfrX3DesaImwiyHI0S2LtvmJpcGHeyoU0Z/ifX8vojnXIcnwcAFUOCPlaKpmVNIz+jWQdH9mk7naIb/2WpwGlFl4SoOHo6L3AoulGavMX1Cx7gTYwCM62JAgmqJ3RqVf7pbjWLV0U43RGcBJl4Qeqms/livc3THymiRuwjPIRzjRsitORmHGJh/ieXpRodZmDei6ZX8IWXZ2jGXjC/6rVbZZG3b0GnJr4bXi1CYDp67XhkEFpVf0Jr7CPGdkDQL80yRH9+zD7ocSS5/3EGzXNuGPb0oVsb8UDLFtXARX3CC6kXjVd6fZuGijTvU6IlGVtNoKhRCphf0iG4UYrdsvexQBxOPCDs27dxO3DHP34pU6rwXZApdz53iynTOpBk1mQMRmxpwI+M52P/sR+Q6cAz0oMpEwFF2sdDZLBqHgkQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(8936002)(110136005)(1076003)(186003)(2906002)(336012)(82310400003)(426003)(26005)(4744005)(47076005)(4326008)(316002)(7696005)(36860700001)(508600001)(36756003)(6666004)(44832011)(83380400001)(86362001)(81166007)(2616005)(70206006)(70586007)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:43.4497
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9bac5a6e-2d53-4269-219b-08d97e8a545c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3296

These two helpers are architecture APIs that are required by
nodes_cover_memory.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
index 3f08870d69..3755b01ef4 100644
--- a/xen/arch/arm/numa.c
+++ b/xen/arch/arm/numa.c
@@ -67,3 +67,17 @@ uint8_t __node_distance(nodeid_t from, nodeid_t to)
     return node_distance_map[from][to];
 }
 EXPORT_SYMBOL(__node_distance);
+
+uint32_t __init arch_meminfo_get_nr_bank(void)
+{
+	return bootinfo.mem.nr_banks;
+}
+
+int __init arch_meminfo_get_ram_bank_range(uint32_t bank,
+	paddr_t *start, paddr_t *end)
+{
+	*start = bootinfo.mem.bank[bank].start;
+	*end = bootinfo.mem.bank[bank].start + bootinfo.mem.bank[bank].size;
+
+	return 0;
+}
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193978.345628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfd-00011C-6K; Thu, 23 Sep 2021 12:17:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193978.345628; Thu, 23 Sep 2021 12:17:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfc-000105-Va; Thu, 23 Sep 2021 12:17:44 +0000
Received: by outflank-mailman (input) for mailman id 193978;
 Thu, 23 Sep 2021 12:17:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNX7-0005KP-8s
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:57 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.45]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b15688bc-e809-4dae-a08b-8c83425a07b6;
 Thu, 23 Sep 2021 12:05:25 +0000 (UTC)
Received: from PR0P264CA0181.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::25)
 by VE1PR08MB4783.eurprd08.prod.outlook.com (2603:10a6:802:a9::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:05:20 +0000
Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1c:cafe::f2) by PR0P264CA0181.outlook.office365.com
 (2603:10a6:100:1c::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:05:20 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:05:20 +0000
Received: ("Tessian outbound 8b24208353e0:v103");
 Thu, 23 Sep 2021 12:05:18 +0000
Received: from 6415f6476d30.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D94C7F36-7225-4215-B3AD-7B66E54708FF.1; 
 Thu, 23 Sep 2021 12:04:48 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6415f6476d30.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:48 +0000
Received: from AM6P191CA0104.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::45)
 by AM9PR08MB6195.eurprd08.prod.outlook.com (2603:10a6:20b:284::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:46 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::8b) by AM6P191CA0104.outlook.office365.com
 (2603:10a6:209:8a::45) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:46 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:46 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:24 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b15688bc-e809-4dae-a08b-8c83425a07b6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XPyMh8kRm237dXS4GzraOVjqXtVF9dvrj+OlofkeXZM=;
 b=R98OGe1zJLc+wYDSRAf9OcKgXO/XUJs+TOE2xBq6X4ipvBp6vqLnzmLt5p/1Y3QHTbtsqzWLLKbYTS/xUjWaYl2i7HxAfh3iNH6nYVCngAYJR9XzB37UvcG9ynQm7wTTa05byGHJ6zdGM3m4PXODNi74+c8al0R+AMLTEKl4TwM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 0c9621024c7910d2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LCY5mjUjcTuUzD8QLUOTelv72R9wjbolCxbFuXZ+jjMlatXBZakjvHH0JRdPRU/3OTf2Q1Fk2n60UbGZFAYBPHp+5M2sudYuv+wtUINABvgQb3bXqyUfF4IkQ8jy+5KIn9NYLvdTHmSYJhucmp8vCo/VSNA3NRH5G8pIr6fWlpnw+lSdz2bkvw7eBYWF5fxBu4m+y4g0kMRC8ZqErfGDeir3itoaW2KEJCFgYskxmzjZUT7jJS85T/P31IBu49l/zSy9+GIUAKjmBPTKmJcWjzpsSU553HmSaHs9b/kYQReTaKGqm8ZWvDFGS1QqsckgXPgaGfJBA2kmbMuFT2yrOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XPyMh8kRm237dXS4GzraOVjqXtVF9dvrj+OlofkeXZM=;
 b=kmO1HiqSCWFJKEtMN37vQMfcJuUEYfZ57Yl5U+0GMIpPxRaIuElEpFidVyqlWSIVwbGaSkWWaiO9wNTCsJ99S3HXOY3xA3iNVdIEHKLCRS472oAUzzafGaP7uwM8EeEfViyW6TPNjtN92VyA3KCbrTQ7P4uGd9k2tgmXzcA4Z1ycrrklqoKrtuusTRZZEYlHZIdtJsc8vBeFAuaQuvpGJvACLdkSClQT+ixvg4LG5UUJPCn+9Of/wBQ+sS/x8oKIVmKEkl+xin8uVelAzs/Sy9Yg6wez+2Ly84VVSo6Xvm81o8hoIJuDDFY+NUqPafwckzcnBTCFdsSiWKbNu60yvw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XPyMh8kRm237dXS4GzraOVjqXtVF9dvrj+OlofkeXZM=;
 b=R98OGe1zJLc+wYDSRAf9OcKgXO/XUJs+TOE2xBq6X4ipvBp6vqLnzmLt5p/1Y3QHTbtsqzWLLKbYTS/xUjWaYl2i7HxAfh3iNH6nYVCngAYJR9XzB37UvcG9ynQm7wTTa05byGHJ6zdGM3m4PXODNi74+c8al0R+AMLTEKl4TwM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 34/37] xen/arm: enable device tree based NUMA in system init
Date: Thu, 23 Sep 2021 20:02:33 +0800
Message-ID: <20210923120236.3692135-35-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7524a0dc-3fde-42ba-3ca9-08d97e8a6a36
X-MS-TrafficTypeDiagnostic: AM9PR08MB6195:|VE1PR08MB4783:
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB4783DABE82EFA9FE47EFEAE79EA39@VE1PR08MB4783.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:3173;OLM:3173;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 9nSKCTMaejQPMJJ+ZfEY7qDHpNBGYij1umBDDjqV+RVOaCyvJh66HoljiVU1dRVizGISfHk6Eizp9wavF7aPt1+1XV5PxsPhFko7SUH8ZRZk7F81as3i9bzNbIAUtGwZ3t7Tn6twkWRQKmUaqeerboNeBj5YCHlq68SvTlwYhMFHBZsJ1hdPq7ydWiNzYvSL6xqZI8lCjCk8s7VmXOOzG8beBJi04/+UIt/WM4TNNQ5DS9YeY1ao51khGuAGsEg0hLIVveV7nBqPmSaf8Yo+Wpb3zlfjnHIVtYCNcGk7ULzB0PUGYlflvEwAtXmKJYWngFJFgQG28/YpOcnYWnyyxazoNaH5OdqTQxVERcnC89OtFWYiElXGHzWQvZ5Xrs9mx+YrgpNQgM51R1rGogv+Ez8f5PGMRgulXqkhMsgMniaBzroDs99NRIeuYGQbukqJlfM6WXI4fPfSEpdRdKJK4YW8swMSaQVMI/B3Mo/pRrF+6Mdyk88IuFaUv0IEgvpCn1huKyg9oHJbpdHx/t5Lpbk7ko3IZp6OARDK8tnrQPQNmg4xSOTm+VmeZlU+6M5sV03OJKbtcqqVV4fDqslHhIYCMoAX7IpoBiQUwYTjCF945ypt0etZd6if8H7prIzBjwMb3wp/5ZamehRgwEVnPUywt/lnTnssCiHKkFn3+KZmH4OxNoGlTLRC47EaZhlqeDWZM+9DB9HclbsCCSGaWwlJLhpXvpo539XMx71rku8=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(44832011)(82310400003)(47076005)(2906002)(336012)(186003)(8676002)(8936002)(316002)(2616005)(1076003)(508600001)(83380400001)(356005)(36860700001)(86362001)(70206006)(5660300002)(81166007)(36756003)(110136005)(6666004)(70586007)(4326008)(426003)(26005)(7696005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6195
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d8d5cfd0-76b2-4e3c-fcc9-08d97e8a5628
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oGvCla2hn4nMMulmYaoufQDU01V5N5XzqSOHDfjXLLMGhx0ZbgRIEmDG9K0+7uDJgeMiQlYj6HX2chynuhwEk7RxdK7NeJeUENYGtXS0mvCrs7haj1qPIGUc646w0OFCJ/zgqB//U+QO5QKGb+t99uZyHH+cVORLF5lueAONLiRCrDcknCO5r1+WP/FXViVF/53bavSdb0B9EW4rbANwhT/KxhIlB7ibNSVhDfBFC/zaxSXeJ6ST7EX8dr1kQxCbHyoGlevjhZCgS1xbuCb6wkMMguBO1FicVX+n249PtFR3TSjcBzj/A0Piov5pviCEBX69cUNNk90ChFu9RX5D+Wrp3H72bK+yVbKLj5yG1jx/Ma/xghhuEZz+ZUhLer3Tx1TGXPI6UoYs8lox8WxHtvsX7BdVhmD5PDqPwpnukfeQhrGj6wPrT4z8lD5nz8d0ZUvWOOaaJA07E68YS9/g1VQWe1lm2BK2PM736cbpKX3kI0c24nCNhHCgF/EvxBsPhz6iRu0CP+a9NWNikmmYvyT9NGXInSBQ2LoTKk6ippCjHMNWI2L4BJWjaeAtprETR11Jun1azm9LbFuG9+sUrBH9vVTNUtX6Om1ykjHkAWryy9irFuI/ly+PNyUcqiKizeI6LppKFkS6GPu0VmqDqZomhbe9LAmMh+sB1Q4GtUFNFEh4eEO7HpD4Y2DUr92spd9tRC/V3uX4uvxxAax/JQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(110136005)(36756003)(336012)(426003)(86362001)(83380400001)(36860700001)(186003)(47076005)(8936002)(1076003)(4326008)(70586007)(7696005)(2906002)(81166007)(26005)(70206006)(82310400003)(316002)(6666004)(508600001)(8676002)(5660300002)(2616005)(44832011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:05:20.1072
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7524a0dc-3fde-42ba-3ca9-08d97e8a6a36
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4783

In this patch, we can start to create NUMA system that is
based on device tree.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/numa.c        | 55 ++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/setup.c       |  7 +++++
 xen/include/asm-arm/numa.h |  6 +++++
 3 files changed, 68 insertions(+)

diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
index 7f05299b76..d7a3d32d4b 100644
--- a/xen/arch/arm/numa.c
+++ b/xen/arch/arm/numa.c
@@ -18,8 +18,10 @@
  *
  */
 #include <xen/init.h>
+#include <xen/device_tree.h>
 #include <xen/nodemask.h>
 #include <xen/numa.h>
+#include <xen/pfn.h>
 
 static uint8_t __read_mostly
 node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
@@ -85,6 +87,59 @@ uint8_t __node_distance(nodeid_t from, nodeid_t to)
 }
 EXPORT_SYMBOL(__node_distance);
 
+void __init numa_init(bool acpi_off)
+{
+    uint32_t idx;
+    paddr_t ram_start = ~0;
+    paddr_t ram_size = 0;
+    paddr_t ram_end = 0;
+
+    /* NUMA has been turned off through Xen parameters */
+    if ( numa_off )
+        goto mem_init;
+
+    /* Initialize NUMA from device tree when system is not ACPI booted */
+    if ( acpi_off )
+    {
+        int ret = numa_device_tree_init(device_tree_flattened);
+        if ( ret )
+        {
+            printk(XENLOG_WARNING
+                   "Init NUMA from device tree failed, ret=%d\n", ret);
+            numa_off = true;
+        }
+    }
+    else
+    {
+        /* We don't support NUMA for ACPI boot currently */
+        printk(XENLOG_WARNING
+               "ACPI NUMA has not been supported yet, NUMA off!\n");
+        numa_off = true;
+    }
+
+mem_init:
+    /*
+     * Find the minimal and maximum address of RAM, NUMA will
+     * build a memory to node mapping table for the whole range.
+     */
+    ram_start = bootinfo.mem.bank[0].start;
+    ram_size  = bootinfo.mem.bank[0].size;
+    ram_end   = ram_start + ram_size;
+    for ( idx = 1 ; idx < bootinfo.mem.nr_banks; idx++ )
+    {
+        paddr_t bank_start = bootinfo.mem.bank[idx].start;
+        paddr_t bank_size = bootinfo.mem.bank[idx].size;
+        paddr_t bank_end = bank_start + bank_size;
+
+        ram_size  = ram_size + bank_size;
+        ram_start = min(ram_start, bank_start);
+        ram_end   = max(ram_end, bank_end);
+    }
+
+    numa_initmem_init(PFN_UP(ram_start), PFN_DOWN(ram_end));
+    return;
+}
+
 uint32_t __init arch_meminfo_get_nr_bank(void)
 {
 	return bootinfo.mem.nr_banks;
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 1f0fbc95b5..6097850682 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -905,6 +905,13 @@ void __init start_xen(unsigned long boot_phys_offset,
     /* Parse the ACPI tables for possible boot-time configuration */
     acpi_boot_table_init();
 
+    /*
+     * Try to initialize NUMA system, if failed, the system will
+     * fallback to uniform system which means system has only 1
+     * NUMA node.
+     */
+    numa_init(acpi_disabled);
+
     end_boot_allocator();
 
     /*
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index f46e8e2935..5b03dde87f 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -24,6 +24,7 @@ typedef u8 nodeid_t;
 
 extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
 extern int numa_device_tree_init(const void *fdt);
+extern void numa_init(bool acpi_off);
 
 #else
 
@@ -47,6 +48,11 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline void numa_init(bool acpi_off)
+{
+
+}
+
 static inline void numa_add_cpu(int cpu)
 {
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193980.345637 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfe-0001G1-5H; Thu, 23 Sep 2021 12:17:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193980.345637; Thu, 23 Sep 2021 12:17:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfd-0001EX-RY; Thu, 23 Sep 2021 12:17:45 +0000
Received: by outflank-mailman (input) for mailman id 193980;
 Thu, 23 Sep 2021 12:17:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWd-0005KP-7i
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:27 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.15.77]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 117f061d-c305-4d4d-b62c-bcfa36765019;
 Thu, 23 Sep 2021 12:04:53 +0000 (UTC)
Received: from AS9PR06CA0340.eurprd06.prod.outlook.com (2603:10a6:20b:466::35)
 by AM0PR08MB3762.eurprd08.prod.outlook.com (2603:10a6:208:100::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 12:04:51 +0000
Received: from VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:466:cafe::aa) by AS9PR06CA0340.outlook.office365.com
 (2603:10a6:20b:466::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:51 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT014.mail.protection.outlook.com (10.152.19.38) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:51 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 12:04:49 +0000
Received: from 9a5a85aa1274.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E0ADA806-D352-4668-A05A-41F79A4AF1E2.1; 
 Thu, 23 Sep 2021 12:04:37 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9a5a85aa1274.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:37 +0000
Received: from AS9PR06CA0099.eurprd06.prod.outlook.com (2603:10a6:20b:465::34)
 by DB6PR08MB2837.eurprd08.prod.outlook.com (2603:10a6:6:19::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 12:04:35 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::16) by AS9PR06CA0099.outlook.office365.com
 (2603:10a6:20b:465::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:35 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:35 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:13 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 117f061d-c305-4d4d-b62c-bcfa36765019
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oSRPuHv3is6t7XfRKDtZ5lXIpe0r3zEXCQvXK2ZgwaU=;
 b=i+2MFZIKCPnC3jksxYDTfeUSsWSDDWs+TMTXiGQKJyx4PKDegyugKfBbqc63vT9cleJfr5r3rfAe1tGd9Ku5MqVIhwJCpJXfetJBxKb/SBbLWJq2PTHmKYqXbStvfuny++1/8v+bpdv9bQe0X/e6AwIzu2pYOUNjhZvPoDYfy0E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 35a3dcabb8d82fd6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Umpeg/t8Y7NsxzwY1dW0o1HvlMne95z94TyxFZBfJ48dF56YeG2/Ktxi7gu1hhu1a3Uus4zsHIwiQHGh9tv+Zo5VEH9c6sNsSE62rtI1arV/pgHz/xwJ3HNiLKBLW42N4j8XJ54QtoPSUTL3vVaJFNtPIXiRn8IVQo0l96Ha6aPp47IzRlHP5dLlBlxRCQT+B2LnGofroPKCzjQ5Z9tyRWw9gEHqzaSNCoBbnkkOYoAJhl30uiRSusMb+0zwDsMQs1wTIpBb63Yt3iNaGpSh3enmYqml+9gmZWaiToQ9nlFkN0DRfEVpMRpYp/yTDFRopDsY4U5YLYDY935IcY8OfQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oSRPuHv3is6t7XfRKDtZ5lXIpe0r3zEXCQvXK2ZgwaU=;
 b=NzPYeHr5zVoNvXoM10dqNFZlWt+fVSI/IBUEax6bEKvNXfUqjdUQL0RpFENnmFtlpmrT7eSg+/9NWopx+sjjn7XK57NfTgFhdUofJfqtaqDtV+F662nlvkOOLhHwJFpT3L8ejRIHW5K6q5iZLb+Zx/I8Sw3yisOQVOodlJt6k99Ogj8tq0uI4I0TLyN6oMrSbJVns7efHB1I10iwMD/v0eAgLp5N3Rxg3u/GyBuR8+RrZJnQP123ceXZkczQSmKqnZERJofdr/vjmBQOA3bR4rbAvBx234QNGFVFpmYI7TP4fm9njzrKSUH2BadKhCmKzdF9tm81Yv4V7NVfIvE7Jw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oSRPuHv3is6t7XfRKDtZ5lXIpe0r3zEXCQvXK2ZgwaU=;
 b=i+2MFZIKCPnC3jksxYDTfeUSsWSDDWs+TMTXiGQKJyx4PKDegyugKfBbqc63vT9cleJfr5r3rfAe1tGd9Ku5MqVIhwJCpJXfetJBxKb/SBbLWJq2PTHmKYqXbStvfuny++1/8v+bpdv9bQe0X/e6AwIzu2pYOUNjhZvPoDYfy0E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 29/37] xen/arm: introduce a helper to parse device tree processor node
Date: Thu, 23 Sep 2021 20:02:28 +0800
Message-ID: <20210923120236.3692135-30-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f731d929-9509-4fd5-4de6-08d97e8a58e5
X-MS-TrafficTypeDiagnostic: DB6PR08MB2837:|AM0PR08MB3762:
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB3762CA6623A477745C29DB119EA39@AM0PR08MB3762.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4XpKmFZYp9OJloY0jEYrHDi13BYRQOtMsSkFUIzRbyhq9kQtdLFd+M5F0kyKmkxMBFOqTYViCIeD7hG+6CGVYJJlrgMeFhQPg2pYo7LvX2fsN7RtAVWjnSj/bSlYKFMya1ZNqkQo0eJcgR/Br1veP8Ok/nXiPDnh3OrfYNHiryJ20sfwqOoue1NFz06/ForN7eZ8sXxnLPZdG6lYbJi2r7FBU5YBDa6rYnoPt6r4cNO7P7MdcXaYMWjh1MnR2u5JG5CY1GfygyM5YfmRNK8iemlzw87SxuIO/1R9GUBgke5DW77lnLNCCrCX7H5BooMbskP5pcb6NLoR7Y/GxdHnjU1fWrvm7ePPXKhDzZo+q06UzM9VNfKWwHDQGBpbdIo6Xvj50ciuPoz5BrtmkmDlfmVrgmehLAbOIUggRuJORhB7hT6yTZ6SHI0YbTuY+OKj3T77yBnsmQX4nJHmuVnGHhyE7EPCPQAO6XWra2LVQF90guX244oA7fmxkVAys5jeWmwYsXi/nkL0OxKlvheXekAuYHvVXNHED/fYUgL2u68a86plQCtVx263MdWElQ9CPo1rm7vI9jKz20IK1UT1YMYmRD9SkicOpXmoMR6dM6RXi5uEIwpER/4nvxaOIVaYpd3koJiIU1WQnwPeyfptf9VCASlIrTTUtYONOQ12YU1e4cRHQ57UWb2ghDMDJb1sYS8MSARIdShENLTKUQ6XtxL6MV6mm2fG+/8uL9QnJkmlyzmJB4O8gRaGhK7jUg1ZKIYBtAdOWoby1p5IiPcXc+Q1tS3PlzGs2/dJuyI1AVG+zw9BMcSuTkKZ1gPZtTh0bqu5kY8mySAm8I1dkxhlZg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(356005)(81166007)(1076003)(7696005)(6666004)(36860700001)(316002)(186003)(110136005)(336012)(86362001)(36756003)(82310400003)(2616005)(426003)(508600001)(2906002)(44832011)(8676002)(70586007)(47076005)(70206006)(26005)(4326008)(8936002)(5660300002)(83380400001)(2004002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2837
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	97ae4213-0ad6-4049-1089-08d97e8a4f99
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LBgkHJCzstKdOkbo4DSaxD2ubk79BzuD51XflmWJAwySzt/tJ71WKLivETk7uBQVD5uhseaiRsNoiHWE1E1xQJc1de/l6xh088P4Asec+XLEldh6ODSyoRXdp23Jc5w5uvhjO3/NQ+E58odz5byogcOfRWqINHmKbXHC2BIfMg1gcpcJNkIBHiCSMVfPuJsPattf/DBTi6XebZXQND8Qyawx3+/lsXgMNV7oJRaf6zMX0m8x3+C2DpWzL/jt5AjTunHNJTFh4eFZ8Pr8rdeDCbZf4MyKohzMUKnL2RRhQ7l77oYNC6+qBXkE7jJ0zNSPr8SNOBP/TRqGxEy41QmeM1Q6Cx7UQ47BljXBQ4S7yOM4K6fYwcYBHz5oua2fQiydrhQzqAeplaUSePa078JwmC2suWQV08yIVOhtVuLovGBXLVSO6QWNLpnzm8s8Xdo/8rP1P/pcX7gg83t5x6kiKF/3wBoizJibBwMICFZqmFL5UVdydO6KLXrlzHxgawyx7of5/vE5T9C2XwhiO5M61EXwGpUeLf78eRt5uY7m51AEJBJXyDkSToLei0JVgDGqv9J1c0cdTNf8fmPHyH79jjiTYqZ19RP6YmzWUQKV9TWCrBgMtifwtH07bgadgQ5PnQcgDsT+FNxTocGRPEJ/HBLeoISPL/zKk/Mv2B20QBSazy6KSb3R+wyX6KjN9xx4EoLpMV6BUtNCNCoC3TQ1zK8g6OyDcMn+UdpoDl8pWtGD7QcN5/4TjjABz2h1fSYfh4Tvvoca/VCGPNuB98MHRIEbne0u6u9fINrLoudJYS3Pj7aHti3BTgtcrPpIgda+
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(70586007)(4326008)(316002)(82310400003)(7696005)(2906002)(110136005)(44832011)(508600001)(186003)(8936002)(86362001)(81166007)(1076003)(2616005)(36756003)(5660300002)(8676002)(26005)(6666004)(36860700001)(47076005)(336012)(83380400001)(426003)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:51.0565
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f731d929-9509-4fd5-4de6-08d97e8a58e5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3762

Processor NUMA ID information is stored in device tree's processor
node as "numa-node-id". We need a new helper to parse this ID from
processor node. If we get this ID from processor node, this ID's
validity still need to be checked. Once we got a invalid NUMA ID
from any processor node, the device tree will be marked as NUMA
information invalid.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/Makefile           |  1 +
 xen/arch/arm/numa_device_tree.c | 58 +++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
 create mode 100644 xen/arch/arm/numa_device_tree.c

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 41ca311b6b..c50df2c25d 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -36,6 +36,7 @@ obj-y += mem_access.o
 obj-y += mm.o
 obj-y += monitor.o
 obj-$(CONFIG_NUMA) += numa.o
+obj-$(CONFIG_DEVICE_TREE_NUMA) += numa_device_tree.o
 obj-y += p2m.o
 obj-y += percpu.o
 obj-y += platform.o
diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
new file mode 100644
index 0000000000..2428fbae0b
--- /dev/null
+++ b/xen/arch/arm/numa_device_tree.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm Architecture support layer for NUMA.
+ *
+ * Copyright (C) 2021 Arm Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+#include <xen/init.h>
+#include <xen/nodemask.h>
+#include <xen/numa.h>
+#include <xen/libfdt/libfdt.h>
+#include <xen/device_tree.h>
+
+/* Callback for device tree processor affinity */
+static int __init fdt_numa_processor_affinity_init(nodeid_t node)
+{
+    if ( srat_disabled() )
+        return -EINVAL;
+    else if ( node == NUMA_NO_NODE || node >= MAX_NUMNODES )
+    {
+        bad_srat();
+        return -EINVAL;
+	}
+
+    numa_set_processor_nodes_parsed(node);
+    fw_numa = 1;
+
+    printk(KERN_INFO "DT: NUMA node %"PRIu8" processor parsed\n", node);
+
+    return 0;
+}
+
+/* Parse CPU NUMA node info */
+static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
+{
+    uint32_t nid;
+
+    nid = device_tree_get_u32(fdt, node, "numa-node-id", MAX_NUMNODES);
+    if ( nid >= MAX_NUMNODES )
+    {
+        printk(XENLOG_ERR "Node id %u exceeds maximum value\n", nid);
+        return -EINVAL;
+    }
+
+    return fdt_numa_processor_affinity_init(nid);
+}
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193981.345642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfe-0001O9-PM; Thu, 23 Sep 2021 12:17:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193981.345642; Thu, 23 Sep 2021 12:17:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfe-0001Kw-Db; Thu, 23 Sep 2021 12:17:46 +0000
Received: by outflank-mailman (input) for mailman id 193981;
 Thu, 23 Sep 2021 12:17:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNVQ-0005KP-5Q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:12 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0c::621])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bc77ab59-227c-4d3e-b0a2-387a2c701b56;
 Thu, 23 Sep 2021 12:04:27 +0000 (UTC)
Received: from DB3PR08CA0035.eurprd08.prod.outlook.com (2603:10a6:8::48) by
 DB9PR08MB6586.eurprd08.prod.outlook.com (2603:10a6:10:23f::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15; Thu, 23 Sep 2021 12:04:25 +0000
Received: from DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8:0:cafe::e2) by DB3PR08CA0035.outlook.office365.com
 (2603:10a6:8::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:25 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT028.mail.protection.outlook.com (10.152.20.99) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:25 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Thu, 23 Sep 2021 12:04:25 +0000
Received: from c6c05db66536.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 AF2ED06C-7B7F-499C-8A2E-123668EAFBD0.1; 
 Thu, 23 Sep 2021 12:04:13 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c6c05db66536.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:13 +0000
Received: from AS9PR06CA0116.eurprd06.prod.outlook.com (2603:10a6:20b:465::26)
 by PR2PR08MB4732.eurprd08.prod.outlook.com (2603:10a6:101:26::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:11 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::19) by AS9PR06CA0116.outlook.office365.com
 (2603:10a6:20b:465::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:11 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:11 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:03:45 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc77ab59-227c-4d3e-b0a2-387a2c701b56
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gw2w6BJL6KcQ0ikVAlb5sou0Hf1qviJ3JlTLvaJdQt8=;
 b=RpJ9V+UMlBJjkPmu7iFrbzNjdCJrgADjbpCyW3WMef1SdV5r0ra69IqLQSOmAv7kg7OkfZgM0n2AA8hHPyeTx3GomAMV5dDM/9VCnm+N/FsqTiJ+L4rUZNBCXYQ1Waa0RKGN4SoUu28ccuA0oMdIZQ2Ls6c1nqDGQmr65V1vYS8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 91056ac04821cc0b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JnRn8B9dodBh4cqw4ukaJ3bCWbZq90l0CkzsHmIyMAvQnGCnkeq6jt8Yme1o5XQJCv2W/2yRGGTgDq+deQYs3E5HlZ5uqWejdro3C4k0cNYNuDsSH/IQqTktrylJt8HJ0ReYOSe1u5SmSuVGuouFP6Xb92PxywBwslPDwDLp2K0IJjAYLzKP0kwrg1bLpjm+rTQR3LaMsIB7wgvSq+NQPCWu8Pv2jtKP4fbSOsgL8+iFVTUxeyrA6Kwro1phjNkNUgT6EcAvcLQ1rHbdpr2UR5XmzfMe1mmQth8jBbCxK20MMBkHFJGGnuQT3KrOPG13MpD01258vGQVbHw8eBr4FQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gw2w6BJL6KcQ0ikVAlb5sou0Hf1qviJ3JlTLvaJdQt8=;
 b=OnIvQiHhcT9oXK5fj1cQYUb9SIuPH6vKjvmfAejCXebo3gfHH6tRisNwV+X2R/ChUJXuJoIfUJ7s+K2nR6PHQpn1yaoTcWknSDC/E9WjKyBKAP/iPQyn7JFp07+OoWHhlYNTa6sCT3CaSEnXp9TXrwTJNt/B6DXIuoLlaAPFLkHzesuqjMJ7HX1lPXPY71sV93GPVfdjwksfsZoJvrmQR/mEvszLgVJe/XO/DVknROnpJfPqcWXvdxZRtxHIk8FHPmxgIMzO4Yr7voDwSMBrGAWX4gv7uscpf9IywpjGZvOdwERjr+29DjTynGRd5c5QlI+2Aw+y0JwMyWLGZvo9zA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gw2w6BJL6KcQ0ikVAlb5sou0Hf1qviJ3JlTLvaJdQt8=;
 b=RpJ9V+UMlBJjkPmu7iFrbzNjdCJrgADjbpCyW3WMef1SdV5r0ra69IqLQSOmAv7kg7OkfZgM0n2AA8hHPyeTx3GomAMV5dDM/9VCnm+N/FsqTiJ+L4rUZNBCXYQ1Waa0RKGN4SoUu28ccuA0oMdIZQ2Ls6c1nqDGQmr65V1vYS8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 17/37] xen/x86: use CONFIG_NUMA to gate numa_scan_nodes
Date: Thu, 23 Sep 2021 20:02:16 +0800
Message-ID: <20210923120236.3692135-18-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 17f098ad-3726-48cd-6724-08d97e8a49cc
X-MS-TrafficTypeDiagnostic: PR2PR08MB4732:|DB9PR08MB6586:
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB658600A5557235C5AFBB2BE29EA39@DB9PR08MB6586.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 c79r8fl9QOgjUlIj0mZwd76tk/WxM3P4kH2L9gF8vHcSV+3S/AkYfmF+PrDM3FE5sgYCwNAD6jrTUrN49Sgds/tJdF1FF/JaNYP1hw75PPTcDYr2iezAcWp3qtxPx9ZalPe5U5FQnA2N0j6zigcicJ4GISw1RCh07+xfEn1xyCCb7TwZ33Gi0wX/U58LLCZ2shEMJbtPkORa5KKSFyft8QTMbgxjwIcVukIyalR5T1EPnyUSbXqeZPs8MaHVs/pgcgj2E9MBO6LJS8eA5QIjKacj9QZj56o8EFITjaFXF8jYH3dVY4Q22jUyohRImAlhYyXi2a7oBKkh4+hPAO2Z5O2JMjS9ImW5daMum34e/NFPVwxFa/8cKbXgbIfh7MRms4RlFmiIaCUqvegdDUh+mAbpfEbMRypb2eE/8Z/8srwM1bRfH2L6Oau81Eu4GLUITE9Kq3hkB7eFTm26bVnTk+lD+AQa8YWwOi5CIQOfa+MsyRwEHY9zK6eqzcKHdhRd4/ERmmvoHjwiNVkLlqfiQI3s3q6EBfUCDNM5TKU8mkwHSyopFv/p5sXXum33O7z9Bs3Z4gwd847mQLtcwlD86wKSaR+eNSb8lM1eOXmVXwZQwi1I0GllRB5xMy5/MWYBI4+XyxVggIOyzJf08jPhdSnm8jR7K4coS7bFgWdk9TaeYo68casj/9zOhjyYAhPAG9TS8lBspTWwN4i0Gu+CPnck9+8fNF5oVn6uzcN27eU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70586007)(8936002)(47076005)(316002)(6666004)(82310400003)(70206006)(356005)(36860700001)(36756003)(5660300002)(86362001)(26005)(4326008)(508600001)(186003)(2906002)(2616005)(1076003)(44832011)(336012)(426003)(81166007)(110136005)(7696005)(8676002)(83380400001)(4744005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4732
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	db5b9d9a-8ce0-4c2e-586e-08d97e8a4142
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	N/b2iDMfuLcLCpVegkCQ0u/q7s5jiZ4c77YbMM9QDXVPF5ghVtm7lDnvhADPsRionksnFl5rwYuroGK0XB7EexXsa6bh8ypkWIMdHM1/gIPo+LbIHWplx89Wlfmk5qL9XwWLoRl0V3AtoSy2/xem9CvpdrZZzrxHup5HqINf59a20gNOy4QiB2XlF0rX486OjZCoClgbHgzJ+zPchY+v7N6igA9iCFb6lusHpEZ0bwOHcQV6HjhX/h1yPZsm8EmQwj9/AiymflgeS8aGZ5G3zdj5Kb/P7EpClOJ7n5Kl1pw2SCG0c5q7B3+QZimSwJl4h1EMDgUfHNKTlhx/cRtSO+O5FSgll8dbkK4cuLhNaFsAXizhD9dNx7PrasbF3F8B3OpaxJVyYhbJd5BcJ+sZD8C5wzUzrO+Xn1UqlsWjO6NVWZNHkgnzBpiJC+XCYk/75DQ8dahS8z9f+hwbi+9pSIfPWw8HCpVhorMoXiK8XSUV1AJDaH4U4JcjZ2v1P0TamLQCAA5x7IVESCRWcNiZFNWMHnqG/aLV/KgcU8nvHvgnXbxfySSX8GwjXgCN0KypSNtfyBS+AW+/Z9faed4ozGKdNFWb3gWk90WYQN7eNfCf3GW6PpzzfIZ6RFf9IXrS2JdUHUw5mrVg+Ngb+wuVrZN41yBYGF4kNQWES/2/CLmPOnzgsLLhBmweLdXic0/WT98kWn9hY3AkKOXK3VHmiQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70586007)(6666004)(2906002)(8936002)(70206006)(110136005)(83380400001)(4326008)(82310400003)(4744005)(44832011)(5660300002)(81166007)(316002)(8676002)(508600001)(36756003)(1076003)(7696005)(26005)(186003)(426003)(86362001)(2616005)(336012)(47076005)(36860700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:25.8275
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 17f098ad-3726-48cd-6724-08d97e8a49cc
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6586

As we have turned numa_scan_nodes to neutral function. If we
still use CONFIG_ACPI_NUMA in numa_initmem_init to gate
numa_scan_nodes that doesn't make sense. As CONFIG_ACPI_NUMA
will be selected by CONFIG_NUMA for x86. So in this patch,
we replace CONFIG_ACPI_NUMA by CONFIG_NUMA.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/x86/numa.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 8a4710df39..509d2738c0 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -260,7 +260,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
     start = pfn_to_paddr(start_pfn);
     end = pfn_to_paddr(end_pfn);
 
-#ifdef CONFIG_ACPI_NUMA
+#ifdef CONFIG_NUMA
     if ( !numa_off && !numa_scan_nodes(start, end) )
         return;
 #endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193989.345664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfj-0002Ka-1v; Thu, 23 Sep 2021 12:17:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193989.345664; Thu, 23 Sep 2021 12:17:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfi-0002KF-TW; Thu, 23 Sep 2021 12:17:50 +0000
Received: by outflank-mailman (input) for mailman id 193989;
 Thu, 23 Sep 2021 12:17:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNW4-0005KP-6Z
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:52 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::626])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 49e9fde8-c21e-4a45-8924-0bf8b2f455a7;
 Thu, 23 Sep 2021 12:04:35 +0000 (UTC)
Received: from AS9PR06CA0029.eurprd06.prod.outlook.com (2603:10a6:20b:462::18)
 by PAXPR08MB7280.eurprd08.prod.outlook.com (2603:10a6:102:217::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:33 +0000
Received: from AM5EUR03FT040.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:462:cafe::35) by AS9PR06CA0029.outlook.office365.com
 (2603:10a6:20b:462::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:33 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT040.mail.protection.outlook.com (10.152.17.148) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:32 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Thu, 23 Sep 2021 12:04:30 +0000
Received: from f901f435cad7.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8933737B-2358-405F-9221-EC3CA3AD749C.1; 
 Thu, 23 Sep 2021 12:04:18 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f901f435cad7.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:18 +0000
Received: from AS9PR06CA0092.eurprd06.prod.outlook.com (2603:10a6:20b:465::10)
 by AM6PR08MB4818.eurprd08.prod.outlook.com (2603:10a6:20b:cb::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 12:04:16 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::25) by AS9PR06CA0092.outlook.office365.com
 (2603:10a6:20b:465::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:16 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:16 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:00 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:03:58 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:03:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49e9fde8-c21e-4a45-8924-0bf8b2f455a7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e3okijkGOEYgqB5JRr1n7+WErub1PozJvnVaCT3ZajM=;
 b=6bx6FARY9uEpKHTemojAM1hFDf4kcp4drbAEFa4bfZkLxzV2QjwVM3uBUguk2KOrEpnvLkiRuk8RqIgIBKiPqgJtXTxcVWEFHcnbe4nBnDTFn4A4CCocRCLY4ZssjKZLw1L0wfKzxN6rhjbtnPB9eKMSrco6Pa0sTpr4AxtQTmY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 04c7ee98cf8e44cd
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kgpeq63N531U9H6zSg1o+6kAs0DLCAfVNiJlB7ByzsIqzWZdX2q8xjj3au+2mdkvw+XLCcKAxyPsjznb7wMiV4P3ofyqXQFrsYJ9yhdJAiqoV2MyAKh0CaEIk9G9zwKwLVaPDIaefMU2uwyoQ9JwAhyztpiiKuqu9ramQy3/H30bPpezcLEB1c8gI/0FXHoZMT2kF1qRBCLpnjU7AAppWenJmwNn4lNbGxkQ0hB2ueRJ2TnLBtpQ2ytQxpOzBwZg3ePvC3EDrQieb2P5BOZuq7IMKYYXBvtZmgVbXDK4Zk3G8W8SIFOUp/M5ZFUF9azVpWX503XcmysXJjtIghytIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=e3okijkGOEYgqB5JRr1n7+WErub1PozJvnVaCT3ZajM=;
 b=aBYHeIa3BrjiEvJ3rKSZFb3p3xkMgjSReZgAOW/2gtKVbj2us6/MkKR/opYEpioCxWGJxJhg+C2ecyjblZ34Wgr1gvah/CGlQ4BRppQQSQIFHFUsk8ANB2y/AvTCnHUcNPoUm5Uc8PWqKDb+m943o6IXBseduw3hF24Q/5dAVMEuxTMuXUDzqNbdntDa003uaLDK5w9ROlB7qB0EXGU/UGsaxTSyGjub9njzdIkcm5rKp7Vuib898FgsjOage5dxk1wliCUxVSiMU5jueAvvWqC035HACn1t+YPszOHcvj5ZCQm4hsAUBUkZ+SF5eqxXm1tVwCMZc7nM6nbUsgrvMw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=e3okijkGOEYgqB5JRr1n7+WErub1PozJvnVaCT3ZajM=;
 b=6bx6FARY9uEpKHTemojAM1hFDf4kcp4drbAEFa4bfZkLxzV2QjwVM3uBUguk2KOrEpnvLkiRuk8RqIgIBKiPqgJtXTxcVWEFHcnbe4nBnDTFn4A4CCocRCLY4ZssjKZLw1L0wfKzxN6rhjbtnPB9eKMSrco6Pa0sTpr4AxtQTmY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
Date: Thu, 23 Sep 2021 20:02:22 +0800
Message-ID: <20210923120236.3692135-24-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 55e307fb-016a-436a-615c-08d97e8a4e13
X-MS-TrafficTypeDiagnostic: AM6PR08MB4818:|PAXPR08MB7280:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB7280E9BDC38441379B6597D39EA39@PAXPR08MB7280.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 MWHGhCXTPamOABmtcqxV7u5Ta0Si8iM3bgdnTb3dccK9jaC7mCO8dzrU6l95FKfEKHsnEflJ0UnUTyxQNV0z53krBjTZje3qUEhC0Hlr71shrzIfUgayXp0igRZYS4Kn3gM3SGG+qOZolOvzFXtRbQ/I15KR1Thg4t05E1NT5BXrjkR7U3W8yjBGVyXPDZRdkcp8JI1avrGsdSnJm48pd9Tfqy66GisEwh/CtyxlBOpSKzOSsC1jZSJmkfJ3jtXbUUIiseHs00VUhuo6Z9imn//rl4Lp78WeaWTqz3CGu1if8vEwb73RHFQXGse4XqPrdTrpIIvvTYxctDLUdr/mmmHS/kYzEWbVG+e0dR4SJDSzAeVj5HaQVNopIYXXQ+Ny5418EkbuvUyjvozzJ2M7QLPzYoX/f0vnYzmJDlBbebEmYSO8wyT+Nz2LZo2HEa9KO3NJxfwbitKZ2c57U7GFBrAcvW/0+vciE+Q+XhcYecaRsNv4BK6JP0XPF3/suSW/iSB3IvsqKxH7XcOvSqVBpFtoWGY4NH6NO0H9mklaN+YAr2wxJyjGXtW/3La9dVFUIpKApbSU17YbQBTfXVchu8VTVy4951kkKk0Re2pQYXT/BJV6ytu7ikxNSZkIGIK3FmXkKo8Usry5co5dUvidueHt73p2E9numolg5wW1wfKxdOBX2qc4vrK9DlGoTFnd9/oE9+ofozg9BCA0aI5V5+ljgx9fCK4emZRh5s0xOsX2Dh39VeSjkwIKSUfuHYkJnpOayz83baU78igf4M30+kzgwAhCW8dc996cb73/IHwp0jaGldBXjvSwsPjKQ3qlkNzn7r9CL3SxjS0XyhAy6w==
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(356005)(7696005)(36756003)(6666004)(8676002)(2906002)(82310400003)(36860700001)(70586007)(26005)(186003)(81166007)(83380400001)(86362001)(316002)(70206006)(1076003)(426003)(336012)(4326008)(44832011)(110136005)(2616005)(47076005)(5660300002)(8936002)(2004002)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4818
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT040.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b3c6325e-33a2-4920-c6c2-08d97e8a445e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tiwAfeZEJqwXlPkXv4xpTvpIhIhHXCQXyULjmMwLpy1EhlkpXRbguaEU/4eZZY2a+efLTigrj5Hlb5YIRAnjF41+1W2q/jH/X1JUKyZHhJWYhTACHHUO3aOrwjsLeSrzHRiuQVBeha++DcsUuZ0oOiAqJAj8mt+X1D/76fFDaHlOH83vVUgiu94XV056TRta2mibR8ISKqKvZbCEX5BZjOGezKpB+WCsMiETkdbAH0OFvx8uOG9geTGt264FnEcAUc2/qdCLOTbrnhaacuunMmtTsnG+dYoTAxGldagqXZz3GvDHqPWgK5rHu/pk6pULLGX80P5rjYe9v6jVAjmgT8q0V6LOxgvID312EqxJEuq4CZyY8tQDBUfTzeoorMYXJUPFV0KkkGfgWPDTnx+bF67nI2LAdo9yWWUL7kcpJ2Fu5Nb459iYZI5VZCZH5lYqYSBmQr7AYy8v41AjMcv0lWG15iXUsHlnyowmZXHepVoA/I7i/Ihs35yaJPslQ/ZSijLdl2XMFIuLBvoifKu8sqvpIvBNPtIZkUwvcH8wSZTvfHaRq5QXlC0bxLFpUVJFO8Ea99FHqzZT18sm1CspEhlMcw2VUQzG4d+krYdwUSt5eFE6IaaZuZXHlV/tn198465RCssUOPyCEy8UsnyP1wIAyg/rIQTwSIDfbvbaAcQ90cngdAb4OAzqh4/bKcs1eIwZIWB4LQ3U/nwh08A4O+PO/wWpzyhziJzGpDWssjM2iELEPoiTKidCuQbr2D5FppeeiaIrkN1dPzh7/r/SFBVlbZBoJ7ZqMovl3NC1/P1pHYaksry9W9UntyJ/Wj/7
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(26005)(7696005)(186003)(86362001)(426003)(1076003)(508600001)(36756003)(36860700001)(336012)(2616005)(47076005)(4326008)(82310400003)(83380400001)(70206006)(110136005)(70586007)(6666004)(2906002)(8936002)(81166007)(316002)(8676002)(5660300002)(44832011)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:32.9535
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 55e307fb-016a-436a-615c-08d97e8a4e13
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT040.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7280

We will parse NUMA nodes distances from device tree or ACPI
table. So we need a matrix to record the distances between
any two nodes we parsed. Accordingly, we provide this
node_set_distance API for device tree or ACPI table parsers
to set the distance for any two nodes in this patch.
When NUMA initialization failed, __node_distance will return
NUMA_REMOTE_DISTANCE, this will help us avoid doing rollback
for distance maxtrix when NUMA initialization failed.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/Makefile      |  1 +
 xen/arch/arm/numa.c        | 69 ++++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/numa.h | 13 +++++++
 3 files changed, 83 insertions(+)
 create mode 100644 xen/arch/arm/numa.c

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index ae4efbf76e..41ca311b6b 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -35,6 +35,7 @@ obj-$(CONFIG_LIVEPATCH) += livepatch.o
 obj-y += mem_access.o
 obj-y += mm.o
 obj-y += monitor.o
+obj-$(CONFIG_NUMA) += numa.o
 obj-y += p2m.o
 obj-y += percpu.o
 obj-y += platform.o
diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
new file mode 100644
index 0000000000..3f08870d69
--- /dev/null
+++ b/xen/arch/arm/numa.c
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm Architecture support layer for NUMA.
+ *
+ * Copyright (C) 2021 Arm Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+#include <xen/init.h>
+#include <xen/numa.h>
+
+static uint8_t __read_mostly
+node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
+    { 0 }
+};
+
+void __init numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance)
+{
+    if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
+    {
+        printk(KERN_WARNING
+               "NUMA: invalid nodes: from=%"PRIu8" to=%"PRIu8" MAX=%"PRIu8"\n",
+               from, to, MAX_NUMNODES);
+        return;
+    }
+
+    /* NUMA defines 0xff as an unreachable node and 0-9 are undefined */
+    if ( distance >= NUMA_NO_DISTANCE ||
+        (distance >= NUMA_DISTANCE_UDF_MIN &&
+         distance <= NUMA_DISTANCE_UDF_MAX) ||
+        (from == to && distance != NUMA_LOCAL_DISTANCE) )
+    {
+        printk(KERN_WARNING
+               "NUMA: invalid distance: from=%"PRIu8" to=%"PRIu8" distance=%"PRIu32"\n",
+               from, to, distance);
+        return;
+    }
+
+    node_distance_map[from][to] = distance;
+}
+
+uint8_t __node_distance(nodeid_t from, nodeid_t to)
+{
+    /* When NUMA is off, any distance will be treated as remote. */
+    if ( srat_disabled() )
+        return NUMA_REMOTE_DISTANCE;
+
+    /*
+     * Check whether the nodes are in the matrix range.
+     * When any node is out of range, except from and to nodes are the
+     * same, we treat them as unreachable (return 0xFF)
+     */
+    if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
+        return from == to ? NUMA_LOCAL_DISTANCE : NUMA_NO_DISTANCE;
+
+    return node_distance_map[from][to];
+}
+EXPORT_SYMBOL(__node_distance);
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 21569e634b..758eafeb05 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -9,8 +9,21 @@ typedef u8 nodeid_t;
 
 #ifdef CONFIG_NUMA
 
+/*
+ * In ACPI spec, 0-9 are the reserved values for node distance,
+ * 10 indicates local node distance, 20 indicates remote node
+ * distance. Set node distance map in device tree will follow
+ * the ACPI's definition.
+ */
+#define NUMA_DISTANCE_UDF_MIN   0
+#define NUMA_DISTANCE_UDF_MAX   9
+#define NUMA_LOCAL_DISTANCE     10
+#define NUMA_REMOTE_DISTANCE    20
+
 #define NR_NODE_MEMBLKS NR_MEM_BANKS
 
+extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
+
 #else
 
 /* Fake one node for now. See also node_online_map. */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193990.345669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfj-0002PE-Jq; Thu, 23 Sep 2021 12:17:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193990.345669; Thu, 23 Sep 2021 12:17:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfj-0002O4-A1; Thu, 23 Sep 2021 12:17:51 +0000
Received: by outflank-mailman (input) for mailman id 193990;
 Thu, 23 Sep 2021 12:17:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNVE-0006xV-Tj
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:07:00 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.86]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 79262836-1c66-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:04:59 +0000 (UTC)
Received: from DU2PR04CA0307.eurprd04.prod.outlook.com (2603:10a6:10:2b5::12)
 by AM6PR08MB3399.eurprd08.prod.outlook.com (2603:10a6:20b:47::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:04:57 +0000
Received: from DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::d) by DU2PR04CA0307.outlook.office365.com
 (2603:10a6:10:2b5::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:57 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT049.mail.protection.outlook.com (10.152.20.191) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:57 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Thu, 23 Sep 2021 12:04:56 +0000
Received: from b12015d85945.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 88F0757A-4AA9-40EB-8B84-257C7DA7080B.1; 
 Thu, 23 Sep 2021 12:04:45 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b12015d85945.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:45 +0000
Received: from AM6P191CA0079.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8a::20)
 by DB8PR08MB3994.eurprd08.prod.outlook.com (2603:10a6:10:a6::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 12:04:43 +0000
Received: from AM5EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8a:cafe::24) by AM6P191CA0079.outlook.office365.com
 (2603:10a6:209:8a::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: from nebula.arm.com (40.67.248.234) by
 AM5EUR03FT013.mail.protection.outlook.com (10.152.16.140) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:43 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com
 (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:21 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79262836-1c66-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vghFBlrVlMBMFKgAWPXBQGBuhDmWD5Y0b6JI6HLn9ps=;
 b=gN76vXhZxcBv5BscLifO8EYJDthjC2Ft+x3xWoNfXtF1IIuRTh/CaF4drz1j2axlQY9aDWkjul+pZBQxL3dkg0VfpFBK07fX3gf0znAhJWovzv/TUtuRu2ebi3zWTh3EI748J9xyO+kja8x1O5dArRxjN4bfb85da7kkvo5LWa0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3eea9582ea44a5ec
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H7h8agbnXbNYy4zQeLPQoCUNaCZYwXUgZClp62C/AMHfzOFeZvETHFOBn5WuvSHCXFGZPXi5HTv6E3wHotq9gT6DkYqhUjIaXAvvxamcg93LSGA3zlCVU+eAHh0DscjI8feCGxyw1hl/922SGDjzaxyxcY+DaT3L5AWNDL4euLyqlCZD83/8GAW+h9VDevNHTGyieimDmeovQzWOkA3Nkq0frh8LHonWMjwWzTnqNeL1O0CugedArEMNzqc//DU0VJ25wLFZQbM79TbjhXzPbuvtle+dFRP9s0LEhT8cGu9ffFyJVcj8l0ip6OQrjGrIx+8+lIb7rSuvhKW21X5GyA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vghFBlrVlMBMFKgAWPXBQGBuhDmWD5Y0b6JI6HLn9ps=;
 b=Q8d+573xQL/GxJ8OICSS6kZpomg+vZUa2jtLrUNKLcZqpY142658sjFAwduYdYj/71GckBwFeOEevMQK2zX9BIndUwFSscHcFr9Q/um/xnkvCju8n2GIVIPSK9/QE9qzlWC9Yi3lvC5kjXP45ljAZKfLeLI3y70SOJoNTeahjAANvxQOBJ/FY5/yG6zpfpm+i2AtBTMMvlijLbnkYOwpPzDsZcr6a08TTJ8PjbZubcJ2gTJhQDcPqxxrQ2a2sMy6fH/zYYiyMqZGRQ7XGrrE+hyRawJKU1fth30ZqgbDHISBvHApSoKVeejuaUnqcY3cyg+qGqjH+VSbl4yiNgAbSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vghFBlrVlMBMFKgAWPXBQGBuhDmWD5Y0b6JI6HLn9ps=;
 b=gN76vXhZxcBv5BscLifO8EYJDthjC2Ft+x3xWoNfXtF1IIuRTh/CaF4drz1j2axlQY9aDWkjul+pZBQxL3dkg0VfpFBK07fX3gf0znAhJWovzv/TUtuRu2ebi3zWTh3EI748J9xyO+kja8x1O5dArRxjN4bfb85da7kkvo5LWa0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 33/37] xen/arm: keep guest still be NUMA unware
Date: Thu, 23 Sep 2021 20:02:32 +0800
Message-ID: <20210923120236.3692135-34-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40fcc8c0-593e-48f1-b9c8-08d97e8a5c77
X-MS-TrafficTypeDiagnostic: DB8PR08MB3994:|AM6PR08MB3399:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB33990DC48B8D3B80428CBF749EA39@AM6PR08MB3399.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 RRHmHsEYa6UrnyhJlKbWJ58VGEEodjNSu07OgYMtAz7uK78AhzDDR5WgEAT/6AS1nypEwoIZnmIgLfJsT26zQk5qe7Dewvg090gEwJzS7CAU76a9VKvm+XYN01LceWDRLZ2Qyxz4ImU76+qoHp7yqAA5XCo9ruR9mwm+MJOvyphMmrYUBjmFyT1HdMYYrDG7mLvutmxK3wCePMfV7439s4gmVrO06W8kv7X8x+svBkCkiYZDiB2Ye2Hk/m3w4S3ZX18VFSgl7Gwh9DSV1LDfVzGDwZldMEAgnYdrkRcqPeh9yVfnuquFCyimdVxoeJ4el4p2u37Ojm4Y3YzeumuTrCtDizQvxgRmOAJtFuGEmPqZLN7i1d1Y4AP5KeF7T87Ies8+sV29or8P+S/flpF0SO67V4YmKpCb/GJfuf4Cll3z26bflgw9PpcsDs3COkxRaZxUT8MbR49h7CwJ/zy2FjB2fN7FQov4ZxOVZIgMYwyDme8Tbyhbag3/fLvn6fYgD4CkGBLRQXQisKeMcudKIbF0+To50lxva4B+NWUbpdUNZnZaL8w1ea+5nz0LF4/u9VTdl7w31/vLgTsAXB1xWLfEZUePSEnSgkHSThL9AkXJKZyA14hByfGbBMRyi24q0xKaXqg+oZt/b0nMM0El3oW2RI/Q9CDQwTVNPG7h9wsMFNLrnlH5Wnb47pNlj9dLhvW8k231emtpGfIDIoovpZNvwukA8dfHu/un4Phq6+I=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(86362001)(2906002)(81166007)(1076003)(6666004)(70586007)(7696005)(316002)(110136005)(356005)(36860700001)(4326008)(5660300002)(36756003)(8936002)(508600001)(426003)(82310400003)(47076005)(336012)(8676002)(44832011)(26005)(2616005)(186003)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB3994
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	51cab88d-6ada-46d5-f08e-08d97e8a5449
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Nps67gIzyn2E66UC8brIMXkvgIsfkNn58WoFelcdM6K5GSA09KmdUdkiFLGFdRfh4CxxPoB9SXhgd2RIPENuN4iJ6s9lES2GYUf+QDosFPchygOdtrRO0PgdsPZsCnCnJWUYrxBnOP9e97j3xHM99Cz1rOtSQEISFex7YWfm8sUQeU5YRnVagVCdVAgMpLyBMaPnhOil8pEcXhO332oqdO5Gnv3cj9FfjFdGGCEhnktvQn0iSp2s6Ip/ggRvhuRbWJPz3Z1n0jEk7BKyJxHWZAJYQYqCENRTw6ct2ExB09h4PfmTHrU/2hdFckShHAG3dbpYABUIF478UuZqU1ymcucu6kSlZ+0wseP+dq1WH2Ns/oRxvQlWWp017u8sUEHIQzUvbpb49jQqjr3cgbSGkmyCtBwc3NJKHKMo55jK3ZOlgIuKcFMJxen1pmtekUMuVmJvzEsUFsmavH4t+ZkRkrg37RbeiWkVXRKZWEJ2ZxHGwGpTMgBlenIXPrhvCLSxMIkGPp3n0Dm2JLr5sDjUyjb6XujZPg+k3o5XKSo9pq5vxwn/resNnBckQ+GKNtHRBreHQY1pjce0Tldxlo5MJr7U0E/ku3a4navrVIexeIgEtcYV37Ai+AUHoyCClJBT/R+A3fk7UqqBVul5tXO+/8vLPeFmUT3mqo98hfQs4ZpQxOSShJYbkjUzzZrFbpMVtntbYqCXsLZOG4Rq9qZvUw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36756003)(110136005)(336012)(426003)(2616005)(186003)(82310400003)(8936002)(70586007)(7696005)(2906002)(81166007)(26005)(70206006)(8676002)(6666004)(5660300002)(508600001)(316002)(47076005)(44832011)(4326008)(86362001)(36860700001)(1076003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:04:57.1502
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 40fcc8c0-593e-48f1-b9c8-08d97e8a5c77
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3399

The NUMA information provided in the host Device-Tree
are only for Xen. For dom0, we want to hide them as they
may be different (for now, dom0 is still not aware of NUMA)
The CPU and memory nodes are recreated from scratch for the
domain. So we already skip the "numa-node-id" property for
these two types of nodes.

However, some devices like PCIe may have "numa-node-id"
property too. We have to skip them as well.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/domain_build.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d634c1..6e94922238 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -737,6 +737,10 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                 continue;
         }
 
+        /* Guest is numa unaware in current stage */
+        if ( dt_property_name_is_equal(prop, "numa-node-id") )
+            continue;
+
         res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
 
         if ( res )
@@ -1607,6 +1611,8 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
         DT_MATCH_TYPE("memory"),
         /* The memory mapped timer is not supported by Xen. */
         DT_MATCH_COMPATIBLE("arm,armv7-timer-mem"),
+        /* Numa info doesn't need to be exposed to Domain-0 */
+        DT_MATCH_COMPATIBLE("numa-distance-map-v1"),
         { /* sentinel */ },
     };
     static const struct dt_device_match timer_matches[] __initconst =
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:17:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:17:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.193997.345686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfn-0003Gj-Gt; Thu, 23 Sep 2021 12:17:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 193997.345686; Thu, 23 Sep 2021 12:17:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNfn-0003Fe-79; Thu, 23 Sep 2021 12:17:55 +0000
Received: by outflank-mailman (input) for mailman id 193997;
 Thu, 23 Sep 2021 12:17:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fewu=ON=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTNWx-0005KP-8f
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:08:47 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::62a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bee594a9-9cb3-4904-b90c-9dc8498d15a9;
 Thu, 23 Sep 2021 12:05:05 +0000 (UTC)
Received: from PR3P251CA0016.EURP251.PROD.OUTLOOK.COM (2603:10a6:102:b5::8) by
 PAXPR08MB7296.eurprd08.prod.outlook.com (2603:10a6:102:217::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 12:05:03 +0000
Received: from VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:102:b5:cafe::d4) by PR3P251CA0016.outlook.office365.com
 (2603:10a6:102:b5::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:05:03 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT031.mail.protection.outlook.com (10.152.18.69) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:05:02 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 12:05:01 +0000
Received: from 6b4712ded5e1.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 EBBC38B5-AEE2-4B0F-B1F1-0EBEA0BA195B.1; 
 Thu, 23 Sep 2021 12:04:44 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6b4712ded5e1.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 12:04:44 +0000
Received: from AM6P193CA0139.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:85::44)
 by PA4PR08MB6046.eurprd08.prod.outlook.com (2603:10a6:102:e4::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:04:42 +0000
Received: from VE1EUR03FT058.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:85:cafe::a9) by AM6P193CA0139.outlook.office365.com
 (2603:10a6:209:85::44) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:42 +0000
Received: from nebula.arm.com (40.67.248.234) by
 VE1EUR03FT058.mail.protection.outlook.com (10.152.19.86) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 12:04:42 +0000
Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com
 (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.14; Thu, 23 Sep
 2021 12:04:29 +0000
Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX01.Emea.Arm.com
 (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.14; Thu, 23
 Sep 2021 12:04:27 +0000
Received: from ais-wip-ds.shanghai.arm.com (10.169.190.87) by mail.arm.com
 (10.251.24.31) with Microsoft SMTP Server id 15.1.2308.14 via Frontend
 Transport; Thu, 23 Sep 2021 12:04:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bee594a9-9cb3-4904-b90c-9dc8498d15a9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=m+rO+v6pBMqEKBz+SCDWANix+90+cnJn/n9KVtjiQo0=;
 b=2MOS5p1UyWvqnIuwlV79Ydp9CxmTOibh2arjPKx5k/B7ue5keJEiQeB6Mp9sKOWGSPR5Ac3WMem77r0WxY67YCUwTIrueh4DeN9yXgYCZB0Q2WOg0/X8ASZyU1AV773Hnf8XwHhWlQxSG4IZcB1ryoXAFgJsQt+R/kgJ8Nn+cO0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 9b7fb2e8a8847eb2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=APOBonax8lrawm65/Zr9s2OdC15waGv8jn1G9Fpf94ySafllh3yw3fwYmZcucoFxoJeqpyWuOl0ln+4723I7aJWFUUIERxspuRw5S47tk1kAzUI3hqVy6zqv+YJNe7+ondutGUVpx+/SedaZi6gHfsOqCBtju0PuEdTrDTSYus+2zb+uZdM7MU0dzuj0ZNy1QI5FRe+QSDMvcbVPYRTd6AAZsCT1b0bxHsJwJPaS6Q1hBkaPcPtQ9ldaR0i8nD2hmH6TWSxHi7Xb0KeRbHvgZOESTC1WBYlWa18/LZh2wUdTtlauy+dYOFMF8xFluk0g7s2OP9dNswR1eyEf5Xsg4w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=m+rO+v6pBMqEKBz+SCDWANix+90+cnJn/n9KVtjiQo0=;
 b=X4Ub9wlYfEAiyxgXGE6jEea0hMhLgm5UnB1WZw+gCRfl1XmkAZTVT8OgIbT8omuUtL24toBodBkxMYKhLr/c/ptsDY0btomrGIR55bjPIm87fbXam+QIxev/2o6OmsgFjFpSR2m3h7FIkEu2ODAoAMUsSymzFeb9h9rAzeIrjzWa130/3yR1D5Hw1bxOQPD17Kt/AFnteG1qDViA7AWQ7qNZIYfFgi2mebqZ/RNNUMY2+zE0dJaGH7DYG9VdcCIRPuRgX45JSU5OYo3Ruf9tmabi4h9WOY3+tryV2qnw2X6o0ziZoNIEDT3Zy1Nw3fLviiXjd9Hl9A7dsrOSpo7/ig==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=m+rO+v6pBMqEKBz+SCDWANix+90+cnJn/n9KVtjiQo0=;
 b=2MOS5p1UyWvqnIuwlV79Ydp9CxmTOibh2arjPKx5k/B7ue5keJEiQeB6Mp9sKOWGSPR5Ac3WMem77r0WxY67YCUwTIrueh4DeN9yXgYCZB0Q2WOg0/X8ASZyU1AV773Hnf8XwHhWlQxSG4IZcB1ryoXAFgJsQt+R/kgJ8Nn+cO0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=pass action=none
 header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 40.67.248.234 as permitted sender) receiver=protection.outlook.com;
 client-ip=40.67.248.234; helo=nebula.arm.com;
From: Wei Chen <wei.chen@arm.com>
To: <wei.chen@arm.com>, <xen-devel@lists.xenproject.org>,
	<sstabellini@kernel.org>, <julien@xen.org>
CC: <Bertrand.Marquis@arm.com>
Subject: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable NUMA
Date: Thu, 23 Sep 2021 20:02:35 +0800
Message-ID: <20210923120236.3692135-37-wei.chen@arm.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923120236.3692135-1-wei.chen@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 1
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2e9168a5-217d-4c96-fd69-08d97e8a5ff7
X-MS-TrafficTypeDiagnostic: PA4PR08MB6046:|PAXPR08MB7296:
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB729652596DA96F6B3BF1971E9EA39@PAXPR08MB7296.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 FWyQ/diQKU/Q8pt8an7uk40O23+6zpoDoIHpzG9wgrk26CgfBxnbGcBgpgdltepkMRpuHgRRfky7V3GF9qV8jZtWTXJJ4wsGbTITKx4g4gFtXAkJnJKWqSyGkjtHTdhI//Se9xaC4pDmRJBzyfww+iJgt6tqJ8HzZT7AowVIc7ZgX9hIJh/tDemF1z3jQFGcNgDV4KxxVFyU0wdpFK82Kr7ZE6FUo3Y0V6Iv4HqhHq4HWPo0i61HC/Y4+Mg9o+pUbNn7ToYTLrk2sse4rPsm2JClWzdlFuMvth57fqbxrb9tcos99nlZElOowfwUewiGHfDtXbkmeLnnHPtWS+k+IREs81O2mGSxczVcNfgzB3kwjf56hNZb7N/K4XxQXonX31A+mXJ5Rzw5Iqya0iXIAqzLqi5txLB2Ufl3Hg2W5ufmmY/0yMYbiXteM+fxqjR8ReAJO/Ld6ACEtWNGcICr8JlKl6ogKCojocG/cvCubcisZ78gjiYWTNDbqOsK9jcPvgw+3f/8Vh45o0atx7+zpIv1hQogRXh/2sAJQkQyj1JSNRcDpPyxL7RVLfPir5jDWtjpstYmeVwpDdXNn+yPaiXJO7yKpNnbEv17DDRQfTpw0J5u6YF4lATQQvODGX0pkiVjWLrvdPft2s5WhQt56ioyMfxkg183jYSO3YRhTHASv0Mw0a3s2AyKaYX7lsGsdEuJ+MzTNp7TYzU7bIEu6exNqk4ZDjgOmivkDc4MbrE=
X-Forefront-Antispam-Report-Untrusted:
 CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(186003)(36756003)(4326008)(26005)(8676002)(7696005)(6666004)(82310400003)(44832011)(426003)(2906002)(336012)(83380400001)(86362001)(508600001)(81166007)(1076003)(5660300002)(70586007)(2616005)(36860700001)(8936002)(316002)(70206006)(356005)(47076005)(110136005)(36900700001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6046
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7561502b-0e89-4ab2-bb09-08d97e8a53b6
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5sHuN9u5bXOmyvJOyj/s0kEhsUsin2fcEWhDSP6nbkmfCsLFJqlNqCPJ9bKGhZq23oWyEyUJjmli3u5/B7pgn+PmY8jf7ylkDVRb7SueK7La/y8lId3P5hTpkKwV7NumhD+SFJfxbXPz+jnIYc8i/iCJUPxf1fXY5SCiouxhQfCeXnyqaP9cO4mhPMJJWQiF7meG+aRFIxRwBY+mH2FIrMT5FMt3/rWNd4h0rileNLeldgFV629sYAKht1QR1oBHKQr94Sy0xLStx/UM5stgnRI8yNiAjCYFN682dVL8uUXWXVUKt2ZJrRJgJoG/+O5wjp8VkvF1UOxkeh9UCAZnDhS2S4PR7IWLJSTfLN1z7ngR1/HWAc8jI/6S14X6+NpIQV3gMpGprWD9BN8Tw4BZPs2XG6nhXL2dZZyWSbctRuav7bR9c7zVSBZy/YOxIEb6RZifQ/BufDhwaL8nx+TnRJoryv6boP8pklBP+6mKBqtNVjD8nBlRF6TfxIUwK3fqu921I580K/r9/FJLOMwwSggAf4Wnf5iYIUj78GXd4v/5cr2AMxIaLyBcA9ywdx29CvI8i3e7/Ha0iYwjzVvmLm1/4nytbw6StGDGHICFinmGj+JQK6o6cqTxHXFCKRJxkyrah4faHyNxWUHBbL1tQ3W5STY+vt9z96THr2rshkaVSUWewBAJM2NZcunEYmFeGRNCshI+MUndoreHqecXYw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(1076003)(36756003)(2906002)(83380400001)(26005)(7696005)(6666004)(36860700001)(2616005)(316002)(81166007)(336012)(44832011)(70586007)(8676002)(47076005)(82310400003)(86362001)(426003)(508600001)(186003)(5660300002)(4326008)(70206006)(8936002)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 12:05:02.9250
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e9168a5-217d-4c96-fd69-08d97e8a5ff7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7296

Arm platforms support both ACPI and device tree. We don't
want users to select device tree NUMA or ACPI NUMA manually.
We hope usrs can just enable NUMA for Arm, and device tree
NUMA and ACPI NUMA can be selected depends on device tree
feature and ACPI feature status automatically. In this case,
these two kinds of NUMA support code can be co-exist in one
Xen binary. Xen can check feature flags to decide using
device tree or ACPI as NUMA based firmware.

So in this patch, we introduce a generic option:
CONFIG_ARM_NUMA for user to enable NUMA for Arm.
And one CONFIG_DEVICE_TREE_NUMA option for ARM_NUMA
to select when HAS_DEVICE_TREE option is enabled.
Once when ACPI NUMA for Arm is supported, ACPI_NUMA
can be selected here too.

Signed-off-by: Wei Chen <wei.chen@arm.com>
---
 xen/arch/arm/Kconfig | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 865ad83a89..ded94ebd37 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -34,6 +34,17 @@ config ACPI
 	  Advanced Configuration and Power Interface (ACPI) support for Xen is
 	  an alternative to device tree on ARM64.
 
+ config DEVICE_TREE_NUMA
+	def_bool n
+	select NUMA
+
+config ARM_NUMA
+	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
+	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
+	---help---
+
+	  Enable Non-Uniform Memory Access (NUMA) for Arm architecutres
+
 config GICV3
 	bool "GICv3 driver"
 	depends on ARM_64 && !NEW_VGIC
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:30:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:30:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194070.345697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNsE-0008K8-N2; Thu, 23 Sep 2021 12:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194070.345697; Thu, 23 Sep 2021 12:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNsE-0008K1-Jm; Thu, 23 Sep 2021 12:30:46 +0000
Received: by outflank-mailman (input) for mailman id 194070;
 Thu, 23 Sep 2021 12:30:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTNsE-0008Jv-2O
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:30:46 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 12595b6a-1c6a-11ec-ba2e-12813bfff9fa;
 Thu, 23 Sep 2021 12:30:44 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E0E242235E;
 Thu, 23 Sep 2021 12:30:43 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AEA6013C62;
 Thu, 23 Sep 2021 12:30:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id NvNjKXNzTGG3JwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 12:30:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12595b6a-1c6a-11ec-ba2e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632400243; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=LhcIQcMVizxGvHBw/6VV2mKGKHrIYgyDJMe76PlbHdo=;
	b=ocX8UxG9NxyMAuSYlp6hZ/vEpdGknBAUndqPdryiJUC+KAl0JjpDXnzpFQrnke18cHK1uW
	uEL0gyhwtxyL/rS6l9UQwo01xGcvzcNWd4oJj6ee/L4vkjNfdcOT47nAEs4MpLLn642Yi/
	aksD4x0o0RQTz6vumfunHYecFKiAo4k=
Subject: Re: [PATCH RFC 1/3] xen/privcmd: replace kcalloc() by kvcalloc() when
 allocating empty pages
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <6d698901-98a4-05be-c421-bcd0713f5335@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <6e90efe0-4c0d-55d9-bff8-6da9b9614cd1@suse.com>
Date: Thu, 23 Sep 2021 14:30:43 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <6d698901-98a4-05be-c421-bcd0713f5335@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="DReaxJbM2X5uf5LruUj9ZWLgctnRzIgmv"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--DReaxJbM2X5uf5LruUj9ZWLgctnRzIgmv
Content-Type: multipart/mixed; boundary="MY6l14mAjftPp6ROEXXWrrmnZRAeYFlKL";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 lkml <linux-kernel@vger.kernel.org>
Message-ID: <6e90efe0-4c0d-55d9-bff8-6da9b9614cd1@suse.com>
Subject: Re: [PATCH RFC 1/3] xen/privcmd: replace kcalloc() by kvcalloc() when
 allocating empty pages
References: <0f0db6fa-2604-9a0d-1138-0063b5a39a87@suse.com>
 <6d698901-98a4-05be-c421-bcd0713f5335@suse.com>
In-Reply-To: <6d698901-98a4-05be-c421-bcd0713f5335@suse.com>

--MY6l14mAjftPp6ROEXXWrrmnZRAeYFlKL
Content-Type: multipart/mixed;
 boundary="------------7F2FB7307810C6841AC92D01"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7F2FB7307810C6841AC92D01
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 22.09.21 12:16, Jan Beulich wrote:
> Osstest has been suffering test failures for a little while from order-=
4
> allocation failures, resulting from alloc_empty_pages() calling
> kcalloc(). As there's no need for physically contiguous space here,
> switch to kvcalloc().
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>

> ---
> RFC: I cannot really test this, as alloc_empty_pages() only gets used i=
n
>       the auto-translated case (i.e. on Arm or PVH Dom0, the latter of
>       which I'm not trusting enough yet to actually start playing with
>       guests).
>=20
> There are quite a few more kcalloc() where it's not immediately clear
> how large the element counts could possibly grow nor whether it would b=
e
> fine to replace them (i.e. physically contiguous space not required).

I don't think those are an issue. Per default the sizes seem to be well
below a single page.

> I wasn't sure whether to Cc stable@ here; the issue certainly has been
> present for quite some time. But it didn't look to cause issues until
> recently.

I'd rather add it to stable. Its not as if the patch had a high
complexity.


Juergen

--------------7F2FB7307810C6841AC92D01
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7F2FB7307810C6841AC92D01--

--MY6l14mAjftPp6ROEXXWrrmnZRAeYFlKL--

--DReaxJbM2X5uf5LruUj9ZWLgctnRzIgmv
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMc3MFAwAAAAAACgkQsN6d1ii/Ey+0
lQf9GKjiImongEdgd0ahwBbAun2YWu+HqadhVoJKBzKa2XkXfL5RictA7VuW5J1+2XZGQzoiT1YC
VBTg5CeYvp1UoqVV2u7PcgXsfTcgMZc0VLt2pl8HpniFD34ToEFDooENtu/yxeAF557KK5gw+JQG
3ArlHZoOI2e38WXhCTj+nG79GwCbyplgZPwPqlUhAZuYtwQXu9pu6jGKdO6zUCl02aN7D7P3Vu61
ibkntBOMUorimlX5rtEF4Bryp4N8AnWI20MzFLq6FhvCH369RNfOMgz8os4hBbe2oiquQNCRpfb/
OBfZMxihbvVPqtKcImChLF+6cYBFaiBweY53CO44QQ==
=DEf8
-----END PGP SIGNATURE-----

--DReaxJbM2X5uf5LruUj9ZWLgctnRzIgmv--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:31:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:31:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194077.345708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNsu-0000UA-4i; Thu, 23 Sep 2021 12:31:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194077.345708; Thu, 23 Sep 2021 12:31:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTNsu-0000U3-09; Thu, 23 Sep 2021 12:31:28 +0000
Received: by outflank-mailman (input) for mailman id 194077;
 Thu, 23 Sep 2021 12:31:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTNss-0000Tr-Oh; Thu, 23 Sep 2021 12:31:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTNss-0007hN-Gt; Thu, 23 Sep 2021 12:31:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTNss-0004RU-7z; Thu, 23 Sep 2021 12:31:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTNss-0006Md-7V; Thu, 23 Sep 2021 12:31:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Ohm2Y52tpiTw0BGcueJnqzYYsGn4qsoA8/jiXUZb7eU=; b=4pT2dydk8FDvLJbV1rpryzqObC
	mrEbyQOqWiYFQWfFuVZXlx1CxRBKVNwbV64AkXfkaFO357l6+NOeMpZkLgzcN7VgorWmREQfXbhYC
	Vpua9VbzQNODf/uM62xkzMIu2OMsVDxHwv7iJBdsQid8NmIh6QvuCXjWD5qAPkKoFSCk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165162-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165162: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=f147634a3811567f9c77d1aecb49703bb5651a19
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 12:31:26 +0000

flight 165162 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165162/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              f147634a3811567f9c77d1aecb49703bb5651a19
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  440 days
Failing since        151818  2020-07-11 04:18:52 Z  439 days  430 attempts
Testing same since   165162  2021-09-23 04:20:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 75006 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:43:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:43:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194086.345722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTO4u-0002Ki-8Q; Thu, 23 Sep 2021 12:43:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194086.345722; Thu, 23 Sep 2021 12:43:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTO4u-0002Kb-5N; Thu, 23 Sep 2021 12:43:52 +0000
Received: by outflank-mailman (input) for mailman id 194086;
 Thu, 23 Sep 2021 12:43:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTO4s-0002KV-N4
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:43:51 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e571f165-1c6b-11ec-ba2f-12813bfff9fa;
 Thu, 23 Sep 2021 12:43:48 +0000 (UTC)
Received: from [192.168.15.189] (unknown [195.68.53.70])
 by beetle.greensocs.com (Postfix) with ESMTPSA id 14EF120786;
 Thu, 23 Sep 2021 12:43:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e571f165-1c6b-11ec-ba2f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632401026;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=74J5uL4+y3OkwakSUEbDd5pmG2azKhrylHgHqUMTlRo=;
	b=NqrdEZvIHKxKmAWJ08TREruX+6E1zwtjlIzFRKUJ8z/RjI58v7xHMihe/01vIPrVrKra7T
	niGWbOMLvL+/r6m0yCRL3pyfXGhri4PRKFkmCm8kOkiTf9h5mxH1Qh61DdQIQpDW1ISlel
	hpLot1xPahP9abOARHX6kwVUlJhMYeU=
Message-ID: <e3ab341b-3e7a-1485-90ed-dbd9d2e3588d@greensocs.com>
Date: Thu, 23 Sep 2021 14:43:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 02/16] qapi: Implement query-machine-phase QMP
 command
Content-Language: en-US-large
To: =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= <philmd@redhat.com>,
 qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 Peter Xu <peterx@redhat.com>, mirela.grujic@greensocs.com,
 Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Ani Sinha <ani@anisinha.ca>,
 Eric Blake <eblake@redhat.com>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-3-damien.hedde@greensocs.com>
 <d27c5829-e28f-4d17-966e-6cb86ebe00e6@redhat.com>
From: Damien Hedde <damien.hedde@greensocs.com>
In-Reply-To: <d27c5829-e28f-4d17-966e-6cb86ebe00e6@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 9/22/21 19:37, Philippe Mathieu-Daudé wrote:
> On 9/22/21 18:13, Damien Hedde wrote:
>> From: Mirela Grujic <mirela.grujic@greensocs.com>
>>
>> The command returns current machine initialization phase.
>>  From now on, the MachineInitPhase enum is generated from the
>> QAPI schema.
>>
>> Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
> 
> Missing your S-o-b, otherwise:

Sorry. I did not realize I had to add it since it was already done by 
Mirela. I'll add it for this patch and patches 1, 6 and 8.

> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com >
>> ---
>>   qapi/machine.json          | 56 ++++++++++++++++++++++++++++++++++++++
>>   include/hw/qdev-core.h     | 30 ++------------------
>>   hw/core/machine-qmp-cmds.c |  9 ++++++
>>   hw/core/qdev.c             |  5 ++++
>>   4 files changed, 72 insertions(+), 28 deletions(-)
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:47:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:47:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194091.345732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTO8N-00032A-Pr; Thu, 23 Sep 2021 12:47:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194091.345732; Thu, 23 Sep 2021 12:47:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTO8N-000323-Mm; Thu, 23 Sep 2021 12:47:27 +0000
Received: by outflank-mailman (input) for mailman id 194091;
 Thu, 23 Sep 2021 12:44:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=E3B8=ON=synopsys.com=Shahab.Vahedi@srs-us1.protection.inumbo.net>)
 id 1mTO5c-0002sY-PZ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:44:37 +0000
Received: from smtprelay-out1.synopsys.com (unknown [149.117.73.133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3bc1979b-58c3-40ef-bc9c-0c3a323fb43a;
 Thu, 23 Sep 2021 12:44:33 +0000 (UTC)
Received: from mailhost.synopsys.com (badc-mailhost4.synopsys.com
 [10.192.0.82])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client CN "mailhost.synopsys.com", Issuer "SNPSica2" (verified OK))
 by smtprelay-out1.synopsys.com (Postfix) with ESMTPS id 885924151D;
 Thu, 23 Sep 2021 12:44:30 +0000 (UTC)
Received: from o365relay-in.synopsys.com (us03-o365relay3.synopsys.com
 [10.4.161.139])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client CN "o365relay-in.synopsys.com",
 Issuer "Entrust Certification Authority - L1K" (verified OK))
 by mailhost.synopsys.com (Postfix) with ESMTPS id DAA1AA005C;
 Thu, 23 Sep 2021 12:44:26 +0000 (UTC)
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1anam02lp2047.outbound.protection.outlook.com [104.47.57.47])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "mail.protection.outlook.com",
 Issuer "DigiCert Cloud Services CA-1" (verified OK))
 by o365relay-in.synopsys.com (Postfix) with ESMTPS id 5410280151;
 Thu, 23 Sep 2021 12:44:22 +0000 (UTC)
Received: from CY4PR12MB1160.namprd12.prod.outlook.com (2603:10b6:903:38::12)
 by CY4PR1201MB0200.namprd12.prod.outlook.com (2603:10b6:910:1d::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 12:44:19 +0000
Received: from CY4PR12MB1160.namprd12.prod.outlook.com
 ([fe80::2d3a:44fe:fd31:11e2]) by CY4PR12MB1160.namprd12.prod.outlook.com
 ([fe80::2d3a:44fe:fd31:11e2%7]) with mapi id 15.20.4523.018; Thu, 23 Sep 2021
 12:44:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bc1979b-58c3-40ef-bc9c-0c3a323fb43a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=synopsys.com; s=mail;
	t=1632401072; bh=p8yypIUjB8D1QtzMPOfUh/V1qIMQJPRo2PqJWZ5SYsY=;
	h=From:To:CC:Subject:Date:References:In-Reply-To:From;
	b=Gvq9P4b8YntKHRuxkqZ3tF0YGuSDOH+oVCDf+VSwXXjmaGxF2cK8X33R7WTWMZdxB
	 GEf7t9gTDEWUF/5zMlhaPic0Gw9SioFA146VuyVbXAQDD0BMWeYiU9orEq2KpGk5hQ
	 i5sYd3qIUy1129LF03W2RQe+YxmxznJiJIdmrk3SdS3mqJQGmbc2z3+ie826v+rXmf
	 y2mWag8+iQ0DLDwtixGft2C9TjYEL1Y/KL5EtNn3IHbRSVbKs1VbfEHZw/WNdd8Ppi
	 MF1tAfYhy87PyuwzOIovDHwB4mk2cbJzuZ3GcC+8Od9h1/aYntxCPPc7gvFZv9ewLf
	 2ywh1lMEodcdA==
Authentication-Results: o365relay-in.synopsys.com; dmarc=pass (p=reject dis=none) header.from=synopsys.com
Authentication-Results: o365relay-in.synopsys.com; spf=pass smtp.mailfrom=shahab@synopsys.com
Authentication-Results: o365relay-in.synopsys.com;
	dkim=pass (1024-bit key; unprotected) header.d=synopsys.com header.i=@synopsys.com header.b="XjlMBsSZ";
	dkim-atps=neutral
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ql0OQazr/pNI1ie5lP8Ht8iY8eOUSaANgcsKDZeanbY5JGm2DYN7ViWFq0GSn7YuEdL/lqgPOUwfp6yIwIN7w682q4qOhfVMJJ030VSQuyLahcjVTcTLbj1waahXj9HMRlxKs4Vgg7OqUkLGVtxBHm8v4qbfy5zOmVpGOEgRFzdV+WPfmxr/w2EuS62IYorK7D6PicnCLwUU80cPNmSxzQBIwvJv9I4RLgCW7/OIo59UBGn6ngK54KflPmdMqyk3NB/if51/2tt090fXxKBAVjzRQMpUc9bdssSpKr5TVg7jvM9I1dvnobtwPMo1Rf+9jG2KB1ZQOFtNAl2+M4q1/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p8yypIUjB8D1QtzMPOfUh/V1qIMQJPRo2PqJWZ5SYsY=;
 b=lB/dfGY6qGnlPCzk6jrqPkumxuD6Zle3JmpqYxFguRB+KFcNp6CEFP4jl5p0h3ScILyRfGJGJ6iKV9p6EUIXLi3OjLbHITHcmB3i9qnwN5UvUJ+iHOJpN9M6l6uzRQOqvLaLkjNjU0E+Phxy2TgG8aeY9o8aw+1/5wA0gAX08oYdQXqbqMe4QzUPO3Eju5H0KcnbEjLg54k/UTSE87WlvYxpA0bDlhO5ZZxMYqTaMkGZjoBOqwIIH33tyS1+bGTyH3MjjzIeRu+PbYAUtyPAxkMPpz55xCAno4NSztx5WB43EDkbi0NVyhaRoI8RaQ+rK40gjT3/SxJpzhaG68Z+6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=synopsys.com; dmarc=pass action=none header.from=synopsys.com;
 dkim=pass header.d=synopsys.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=synopsys.com;
 s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p8yypIUjB8D1QtzMPOfUh/V1qIMQJPRo2PqJWZ5SYsY=;
 b=XjlMBsSZIIeMIcn3BV4fq6QTuHKmTlq2wcWFtDSMYpHRIbCRUXHI1j/6Nvc788nfl1iWUG0VjTTkqniwaPBMQvx+hnxM47SXxJ8BHG5lMxGTgpN1OoKMXR+iJA+dKngwgvBc0glcR5Dz2K32DkEYfbZDkaHe+kbDS5mZMs/JArQ=
X-SNPS-Relay: synopsys.com
From: Shahab Vahedi <Shahab.Vahedi@synopsys.com>
To: Mike Rapoport <rppt@kernel.org>, Linus Torvalds
	<torvalds@linux-foundation.org>
CC: Andrew Morton <akpm@linux-foundation.org>, "devicetree@vger.kernel.org"
	<devicetree@vger.kernel.org>, "iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>, "kasan-dev@googlegroups.com"
	<kasan-dev@googlegroups.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-alpha@vger.kernel.org" <linux-alpha@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org"
	<linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "linux-mips@vger.kernel.org"
	<linux-mips@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>,
	"linux-riscv@lists.infradead.org" <linux-riscv@lists.infradead.org>,
	"linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
	"linux-sh@vger.kernel.org" <linux-sh@vger.kernel.org>,
	"linux-snps-arc@lists.infradead.org" <linux-snps-arc@lists.infradead.org>,
	"linux-um@lists.infradead.org" <linux-um@lists.infradead.org>,
	"linux-usb@vger.kernel.org" <linux-usb@vger.kernel.org>,
	"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
	"sparclinux@vger.kernel.org" <sparclinux@vger.kernel.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Mike
 Rapoport <rppt@linux.ibm.com>
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
Thread-Topic: [PATCH 3/3] memblock: cleanup memblock_free interface
Thread-Index: AQHXsE7r2Z9Q1x5PbkCRzXvVc2eiLKuxkQqA
Date: Thu, 23 Sep 2021 12:44:18 +0000
Message-ID: <70364d6d-7569-3cd8-acce-559878ce4596@synopsys.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
In-Reply-To: <20210923074335.12583-4-rppt@kernel.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=synopsys.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c1e5f238-9c40-4f49-2da5-08d97e8fdc67
x-ms-traffictypediagnostic: CY4PR1201MB0200:
x-microsoft-antispam-prvs:
 <CY4PR1201MB02008C7112226A9E691D827CA6A39@CY4PR1201MB0200.namprd12.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2958;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 OamkW3UO1/WqFADz4A194HFP4oBQk8gtqnKNjVuDksrLnfE6l+GVJQhwsaegPUSH7gx9t8lMJ1FdmgYJCgJ4OZ3EhKf8RZTD3Ts0zaaHTLojOQ5TBH4ZPl6xEabh2XoGqBvwUOEEFr9uSp9MQZp9dSBW+UcQqhYiFNaSTPWozfT9nTMipLej1e9xdNsDzJwkWvc4Nqfx9vJUpUl1b2y3wVZ1oitLTB5EUrRSlKgJTGqIekv1SR1YFUhPDs3nw9WEL9bXMu/EvTL8nA8XhyS8T5AggLricxDpUq9wYUHjECvPt7tEppOAD3ENQzfE8EgYlSeEkDpjU97d+AXoZrfhgqvMnmxfIN2HYaS9mEsbHWU/v+kC/37zYn4F+xyL7BZ7yyVP3OXhFEbyRjeA1fu5yW3MacAsdznAi8uIZTlBVJnpBYFOXwFAXJRBkzH6xa4sV/7hMat8LxK4cZD3yPIhI0AN4eK6rNxYmtVdMz7h6nIy+EuGpBn1s07XirzbDNP3itQCWqt0TOrchOPjGN3ZLRqsKltq5PAMjMXOxRZdN2dqnUFCweBuWWfCwgrV5044fj0vvvSEUUfHpFWJLJRnyg4c3Eh2g6Ee+cAC9QDPHcsmAgajEJma1cFC1eRoIfuvfIe6apmOy46ssLpPZuMbub2YhBZ5/KLzyz/ik0j7m6nXwWbxN1QdS/7wnLeRcvfuaqggkqHcyct/8HQb0MRs9FG7LhwusDW07wtmQYsIsLg=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR12MB1160.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(71200400001)(86362001)(2906002)(83380400001)(54906003)(31686004)(110136005)(38070700005)(91956017)(66946007)(508600001)(66446008)(64756008)(66556008)(66476007)(76116006)(7416002)(5660300002)(8676002)(186003)(8936002)(31696002)(122000001)(6512007)(4744005)(36756003)(53546011)(6506007)(6486002)(2616005)(4326008)(316002)(45980500001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?Z3E1Q1p2dDJNR052VkZ1TG1nK2ZKSGJNWWxCdkIxdlgvemFlRHNZdG55SnAz?=
 =?utf-8?B?NFZnVDlEWVZHN3c5dTc1U2pCY2lTdkhTeXB3QU0xV1pocFhjTHZlUFkzSkhl?=
 =?utf-8?B?UDlnbWhORjBYYXFRUUFnNDM5ZFdHbFlZSVFEa3lIUUxnREc5d2ducFNGY0pt?=
 =?utf-8?B?V2NTVGdxdVFMNlRLemlvQ1M2VUJHMVVYL0ZkaXZ2dEVBQnc3Y0EvQVQzQmo5?=
 =?utf-8?B?dzZDbWtxejJRZnFkN210bmIwczlpMGprc0JqY3Rza01pV2JLU29CWkw0SFRJ?=
 =?utf-8?B?bnNoQVBFbzZIRnBndm5VR2tHWWo5ejdEWURZTktZbGthMkdKbU1rcFNTcWxQ?=
 =?utf-8?B?K0tyNG9pRTNuZEY5NXIxUEF6U1k3Nlg5RVlTT0hTTzlQUWYzNVdiRVlpY1ND?=
 =?utf-8?B?eHRxRmg1aHEvVE9mcTVmZU1kalNoR1NBZmwzSUZmSEFwa2xPL1ZuUWd6ZHll?=
 =?utf-8?B?TS9vNDMzNVY4SUxOVTA3S0hQVjRsODRLWC9abmp1MGg4R1c5YmhlM25sK1Jq?=
 =?utf-8?B?WE1BOEM2bWE3RDN6YWs1TUdCOUdZZm5LaW4wYUVuWVR1SFg3RVRoeXZOSTFY?=
 =?utf-8?B?MjNROHlxdHlWejJzN05uZlBQUGZPVXRSOHl5ZnRlTUFkVndJTHNyTS9CZWtn?=
 =?utf-8?B?ckYycndlMSt4bTk5TlEvZE1HOWFXZlRDRmJtOHY3MTdPZ3NhblhLVFZMYWx1?=
 =?utf-8?B?RFd1NXNBcjZCcjVYZWFyWDZGQzBTSmhzV09qZmZEUkdyOENUdFpteXNqQmFF?=
 =?utf-8?B?ejVtUldNT0swSzllQkNnc251RHhQZVJlN3JpdWlWVksyb3ZHMGZRcE0vOVJJ?=
 =?utf-8?B?U0xCQ0hnN294VDBXT1lOZXdKZEM4ZjZoOHdmVStLVnVSUElvNVZFRm1LWDBP?=
 =?utf-8?B?SVBaUjhXUHhHcEdId2hjRXorVkxiNTNISzFjTzkzb0dMMHBIRUtxZmF1aHRk?=
 =?utf-8?B?NEI0a3lZN0prRlNQTGtGTmNWT3VJSjYvVWltOWswWVAreU9BUkxaOHFhOFdq?=
 =?utf-8?B?TkZBZGdXVE9Icnlyd0YraDFNZHR5TDdrVGZmbVVJRXZDRXFtclFFc21TNEk0?=
 =?utf-8?B?WVdjOUJoRHdPdEpMY1pKMDFiK1RuMFZsbFRJMGRLVklCNTQ2bS9CanI4VENx?=
 =?utf-8?B?RkloOG1BZmU2ZnlRTmJCZjJXdDlYaFJnSzhkTFFTbFJSWFBFOGQrcS80NXVU?=
 =?utf-8?B?YjNqOHVwWVM0TkY1NktBVEVmMnVVV1ZYNWp4Z0xySVJFTC9BSm9qNkNzdmZM?=
 =?utf-8?B?MXFFOEIxWDdieU5BQ1pGZlFWWTFaZjhFbi9XUSs1Y0JHemtQUXhrNElVTzU2?=
 =?utf-8?B?K1JtOG1aQUN4UUVyR1ArNmpvbWdscWFPdlJWTi9mKytoSTN5ZUkvc2dvMDEw?=
 =?utf-8?B?c0lmWWFvcWxLRkNiUFU1VTF1QnRoMDJrVkpNK0FtN2FFUVhqKzV3ZHVVTzh4?=
 =?utf-8?B?RkxWQS9YcnZwcklycTcrZy8rWDhxSEdJalZaMllBQ0FHZEhHL1YzTTJsWUpT?=
 =?utf-8?B?MmNvbnFxYmk4ZllDcnJ5U2UycExRUHRmVEJLTEI4UHFOeDJUWjN3UjdkcU9T?=
 =?utf-8?B?RjBtOTJ0M3VPeTdNVVpSbHQxeXVsVDU3LzBMM2RnNkNCd1BuZlZEalZLbTNG?=
 =?utf-8?B?cHQwc3luVmpwc2ZJWVZEY3IzVENiRTdrRHp5NzM3b2xvRnA5LzlCcWpuTE43?=
 =?utf-8?B?ZkFzTUNlWkxVdlFGQ1NtdDBvZS95aGRTUkNaSlBIQ1NNY2FabGNSWFp5dVpr?=
 =?utf-8?Q?zxZX1W1RS474i7oyqAN1Twr11o31hFkPD6o09yZ?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <BEE0BCB4DA8E9D4FA42EB8E75B22E457@namprd12.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: synopsys.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CY4PR12MB1160.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c1e5f238-9c40-4f49-2da5-08d97e8fdc67
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 12:44:19.0739
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: c33c9f88-1eb7-4099-9700-16013fd9e8aa
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: YXVpj8MQIzNFMgREFPcSyNOhb48DBCQjuuwYd+4/FTIRz+7n+fNOo/94DmdqxSrRhwhnoAWwsgrHeewqRJ+nSA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0200

T24gOS8yMy8yMSA5OjQzIEFNLCBNaWtlIFJhcG9wb3J0IHdyb3RlOg0KPiBGcm9tOiBNaWtlIFJh
cG9wb3J0IDxycHB0QGxpbnV4LmlibS5jb20+DQo+IA0KPiBGb3IgYWdlcyBtZW1ibG9ja19mcmVl
KCkgaW50ZXJmYWNlIGRlYWx0IHdpdGggcGh5c2ljYWwgYWRkcmVzc2VzIGV2ZW4NCj4gZGVzcGl0
ZSB0aGUgZXhpc3RlbmNlIG9mIG1lbWJsb2NrX2FsbG9jX3h4KCkgZnVuY3Rpb25zIHRoYXQgcmV0
dXJuIGENCj4gdmlydHVhbCBwb2ludGVyLg0KPiANCj4gSW50cm9kdWNlIG1lbWJsb2NrX3BoeXNf
ZnJlZSgpIGZvciBmcmVlaW5nIHBoeXNpY2FsIHJhbmdlcyBhbmQgcmVwdXJwb3NlDQo+IG1lbWJs
b2NrX2ZyZWUoKSB0byBmcmVlIHZpcnR1YWwgcG9pbnRlcnMgdG8gbWFrZSB0aGUgZm9sbG93aW5n
IHBhaXJpbmcNCj4gYWJ1bmRhbnRseSBjbGVhcjoNCj4gDQo+IAlpbnQgbWVtYmxvY2tfcGh5c19m
cmVlKHBoeXNfYWRkcl90IGJhc2UsIHBoeXNfYWRkcl90IHNpemUpOw0KPiAJcGh5c19hZGRyX3Qg
bWVtYmxvY2tfcGh5c19hbGxvYyhwaHlzX2FkZHJfdCBiYXNlLCBwaHlzX2FkZHJfdCBzaXplKTsN
Cj4gDQo+IAl2b2lkICptZW1ibG9ja19hbGxvYyhwaHlzX2FkZHJfdCBzaXplLCBwaHlzX2FkZHJf
dCBhbGlnbik7DQo+IAl2b2lkIG1lbWJsb2NrX2ZyZWUodm9pZCAqcHRyLCBzaXplX3Qgc2l6ZSk7
DQo+IA0KPiBSZXBsYWNlIGludGVybWVkaWF0ZSBtZW1ibG9ja19mcmVlX3B0cigpIHdpdGggbWVt
YmxvY2tfZnJlZSgpIGFuZCBkcm9wDQo+IHVubmVjZXNzYXJ5IGFsaWFzZXMgbWVtYmxvY2tfZnJl
ZV9lYXJseSgpIGFuZCBtZW1ibG9ja19mcmVlX2Vhcmx5X25pZCgpLg0KPiANCj4gU3VnZ2VzdGVk
LWJ5OiBMaW51cyBUb3J2YWxkcyA8dG9ydmFsZHNAbGludXgtZm91bmRhdGlvbi5vcmc+DQo+IFNp
Z25lZC1vZmYtYnk6IE1pa2UgUmFwb3BvcnQgPHJwcHRAbGludXguaWJtLmNvbT4NCg0KYXJjaC9h
cmMgcGFydDogUmV2aWV3ZWQtYnk6IFNoYWhhYiBWYWhlZGkgPHNoYWhhYkBzeW5vcHN5cy5jb20+
DQoNCg0KVGhhbmtzLA0KU2hhaGFiDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:54:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:54:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194099.345744 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFQ-0004j6-NI; Thu, 23 Sep 2021 12:54:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194099.345744; Thu, 23 Sep 2021 12:54:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFQ-0004iz-K8; Thu, 23 Sep 2021 12:54:44 +0000
Received: by outflank-mailman (input) for mailman id 194099;
 Thu, 23 Sep 2021 12:54:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFP-0004it-4o
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:54:43 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3881b06-f829-4810-91a3-ac72710915f8;
 Thu, 23 Sep 2021 12:54:42 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id p29so25905886lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:42 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:40 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3881b06-f829-4810-91a3-ac72710915f8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=MNFbIgyk0Jd8aG2ETpYU3f61p4SY/XICQ38rzboeHCY=;
        b=Q3s1F2qrT4ycOqCWrlTdxbCjScIxh7VvKaITEIkGWKwodHAyjx063hx6SU0MWOrHBw
         FQBiwqr39XH0vX9VlveiL1I+WcWhCO4kXTNlhShbdIyO3d0xdiUtw94iXUjId97eixcC
         mnzHdO0GpNq/5DFzQflALpqmgZBZnMrFKi9EtqjYnCQ+qKkCvRlFKCW2o2pdzD+rtSfy
         zeaVZemgPR3YV6JiAXG2gLEkK9tF1035gCduUrq/FvjsQIDTI8jgPlhF5oDGvGOdbPg7
         6FJHWUwwW83QGmEojDgpLTI39dayUkgsUKkdxq4OqLMrzG5Ek54ypBby2BEqt6EYqMFZ
         Lc/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=MNFbIgyk0Jd8aG2ETpYU3f61p4SY/XICQ38rzboeHCY=;
        b=jQfaMQQ3vKucKW/ARFdphcfs4+zsU/9dNy5dSjG0nPJNJ2q5nQunS+4eoV7KKKdVH3
         Qm5OXYvPuCDmGeZbRS171nDqcfoxlN72PD+6k8u48tA4flfKP8+k3MVb0LF9tOJbWA9y
         A1goIcyrGaEaH9Lvd3X74DzNR3zqrYFUs6Y48+TXKY9y/diSqEO4b5yAt0eBCmSSzUZ9
         HRYnYF4XhmW/nG2U9ZaMztDhSd/mxBc5oimcTpfDVk63twU672/gKQY/KsAeDn6haJ8t
         VFAlu/RhRCFCI1pn93YlJQXwdQKB6WKcar1atIat3AT67BaEwveu0yVyhTVfydRvJA9j
         4xWg==
X-Gm-Message-State: AOAM5322LUR/IyQelCcoSO9xBIemX8IYAJixbWakBmZcyEJOkBqt9zUc
	HSlZ91FiysZZM/tJaoRGzViV+Ud0PpTv9A==
X-Google-Smtp-Source: ABdhPJyeQSYMT1AM394oHTbiIcYviBjWCf4DhHILZQpEtnl9FzOVrK6UyeMAWky5pN3O8yYhaV4nUw==
X-Received: by 2002:a05:6512:12c6:: with SMTP id p6mr3971226lfg.271.1632401680882;
        Thu, 23 Sep 2021 05:54:40 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 00/11] PCI devices passthrough on Arm, part 2
Date: Thu, 23 Sep 2021 15:54:27 +0300
Message-Id: <20210923125438.234162-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This is an assorted series of patches which aim is to make some further
basis for PCI passthrough on Arm support. The series continues the work
published earlier by Arm [1] and adds new helpers and clears the way for
vPCI changes which will follow.

RFC is at [2], [3]. Design presentation can be found at [4].

Thank you,
Oleksandr

[1] https://patchwork.kernel.org/project/xen-devel/list/?series=550933
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
[3] https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
[4] https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf


Oleksandr Andrushchenko (10):
  xen/arm: Fix dev_is_dt macro definition
  xen/arm: Add new device type for PCI
  xen/arm: Introduce pci_find_host_bridge_node helper
  xen/device-tree: Make dt_find_node_by_phandle global
  xen/arm: Mark device as PCI while creating one
  libxl: Allow removing PCI devices for all types of domains
  libxl: Only map legacy PCI IRQs if they are supported
  xen/arm: Setup MMIO range trap handlers for hardware domain
  xen/arm: Do not map PCI ECAM and MMIO space to Domain-0's p2m
  xen/arm: Process pending vPCI map/unmap operations

Oleksandr Tyshchenko (1):
  xen/domain: Call pci_release_devices() when releasing domain resources

 tools/libs/light/Makefile          |  4 ++
 tools/libs/light/libxl_pci.c       | 15 ++++-
 xen/arch/arm/domain.c              | 11 +++-
 xen/arch/arm/domain_build.c        | 38 ++++++++----
 xen/arch/arm/pci/ecam.c            | 14 +++++
 xen/arch/arm/pci/pci-host-common.c | 93 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
 xen/arch/arm/pci/pci.c             | 10 ++++
 xen/arch/arm/traps.c               | 13 +++++
 xen/arch/arm/vpci.c                | 33 +++++++++++
 xen/arch/arm/vpci.h                |  6 ++
 xen/arch/x86/hvm/hvm.c             |  6 ++
 xen/common/device_tree.c           |  2 +-
 xen/common/ioreq.c                 |  9 ---
 xen/drivers/passthrough/pci.c      |  3 +
 xen/include/asm-arm/device.h       |  6 +-
 xen/include/asm-arm/pci.h          | 24 ++++++++
 xen/include/asm-arm/setup.h        | 13 +++++
 xen/include/xen/device_tree.h      |  2 +
 19 files changed, 275 insertions(+), 28 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:54:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:54:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194100.345755 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFW-000509-0P; Thu, 23 Sep 2021 12:54:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194100.345755; Thu, 23 Sep 2021 12:54:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFV-000500-S2; Thu, 23 Sep 2021 12:54:49 +0000
Received: by outflank-mailman (input) for mailman id 194100;
 Thu, 23 Sep 2021 12:54:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFU-0004it-2X
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:54:48 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a43ebe10-5e3d-4e6b-8c3b-afdab9aaf67b;
 Thu, 23 Sep 2021 12:54:43 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id b20so26419291lfv.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:43 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a43ebe10-5e3d-4e6b-8c3b-afdab9aaf67b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=EgRiuJEADNLL58SEJDtb5O15xXHVo2JGSl0INXd+hf8=;
        b=OHBuwCKvEwlz64M6VDqvzUwbFE1xKQ+yi2WtKkHOkUZqgsJhfJMgmmD1FLTCshj50Q
         IGYcBkgbqefI4o+3v+vNixJ/NSQl4OXe21pZJeQvcrkIc7lkV9Fs3BSUuxLVmTFCibuy
         SyA2XW6bmP2vVIgFeqVNVR20h+ybhv+H+aTvtl3Rgpn6vnbF/QTsTI9wLB37Yz2QTtdA
         chANcReMP40acuUNsXl0L3CHCJi0dGz4e5DXnSqsESfPyC2uD1dqR0PFX44nCbI+iti9
         ZzGsaNQ67D0LwEXAWif/Jn98J0+wgRKoe1eJe5gpOUMQGesoKbrdN5aX3NkNVbir+qW5
         Wyiw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=EgRiuJEADNLL58SEJDtb5O15xXHVo2JGSl0INXd+hf8=;
        b=0kzRIKgGQuQCSj+w+qvTv+D1lccgryMEqRFUJ67uOHt2CHMYIGeNeIJgpNvrtjry0I
         iuaiYdHce24LVydxg534LUcvF9sNQmJZGDyZlwNupytJOx1zO6juZFo5suCeGTdWssOt
         d6ot7qBW4kibR7OQ1ft9dZxPtHLnZ90PFjQAEVVmcZUfTwz2LSE0cnvJm38GnWRffE/C
         ETriOTClW21Mk+HynmBThXORtXf1ZabTEczEIYWDs9A6isJ0YxkFSmOFE943u4Gt3R3Q
         G9pphvITV52764Ln1gdRCrRXTp4IHhU3DaOUYYFXmuS3o3gZ3B9VwJJo5YogKcVoaig2
         Hoqg==
X-Gm-Message-State: AOAM532FZ924/80aTb6YVPudISjskuEZkE7yiyxPbzWRfZbTBGRPUVmG
	C7npKfCjoaPot0SUps8Km8xQIND6PTgNfw==
X-Google-Smtp-Source: ABdhPJw9P0SMBu4FSQsqJndfx88wYC/ICJ7gNtz6XIKsYk6jZ6KYpgUuUvOI+j9Xo6N4JZZ0xRqugg==
X-Received: by 2002:a05:6512:3452:: with SMTP id j18mr4170512lfr.435.1632401681958;
        Thu, 23 Sep 2021 05:54:41 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 01/11] xen/arm: Fix dev_is_dt macro definition
Date: Thu, 23 Sep 2021 15:54:28 +0300
Message-Id: <20210923125438.234162-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This macro is not currently used, but still has an error in it:
a missing parenthesis. Fix this, so the macro is properly defined.

Fixes: 6c5d3075d97e ("xen/arm: Introduce a generic way to describe device")

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
New in v2
---
 xen/include/asm-arm/device.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 582119c31ee0..64aaa2641b7f 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -29,7 +29,7 @@ typedef struct device device_t;
 
 /* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
 #define dev_is_pci(dev) ((void)(dev), 0)
-#define dev_is_dt(dev)  ((dev->type == DEV_DT)
+#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
 
 enum device_class
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:54:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:54:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194101.345766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFa-0005J8-8c; Thu, 23 Sep 2021 12:54:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194101.345766; Thu, 23 Sep 2021 12:54:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFa-0005J1-5c; Thu, 23 Sep 2021 12:54:54 +0000
Received: by outflank-mailman (input) for mailman id 194101;
 Thu, 23 Sep 2021 12:54:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFZ-0004it-2h
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:54:53 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id caa3d2c4-a236-4c27-969b-25387d5f3147;
 Thu, 23 Sep 2021 12:54:44 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id t10so26347744lfd.8
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:44 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: caa3d2c4-a236-4c27-969b-25387d5f3147
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=Prgw1n7wTygj7XLYMSUBEwYoQObrqUDu7O/88Jp1DaU=;
        b=Aqlsdw1eztQq5Zz4qHfp+blkZsvnn/gEtDwzUtWP1FukAN2DGdE27mQxmjBhuavCg+
         3aGpJbrnPFhN1cenUqre7enhWCUHO0sdi2MJIy2pcgVZO+J7nycstYvQV1kbciSL7mil
         GAsijE497cpUGPhtZEwxOa6B8Vou9qVf9acqq4LPdWZ4MCZZ/7ixvoGyMknf/w5H2fGe
         fYk3gHbReSutMmJvoiSSIc0Gc8oUS7rTUDs5dW2A0PyIUNS/nYyYBC4QT6yuZCOcXRSM
         5SAH+hYea6vI7ybXU8lcfJqJ9ZDdlLzQJMGrlEAbYhxdXa6/PUnQ+WL4eqsz4OedjP6F
         DZCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=Prgw1n7wTygj7XLYMSUBEwYoQObrqUDu7O/88Jp1DaU=;
        b=Gz0y1djPqXNKWlWUXyO3+zp52auMxMJnrwHQ5f6zXvxSUX48V2eR97o6f72SgcyngS
         83A3FPlTx4NRgpubIii6nTzUMLRuyY4QM+V7d5Zdha0m39gwM9KKSBgPkmGO372wQWVK
         l9d9ngj9DjXGFrEZ8PFBwuyX3gi1z8W0Pyp4hU6kt6GXGnf4pBVwGZay5h9atQhuRsL/
         FY5fW/oXtbF7Qql4WqsMkBhHNRclKj9iqKvYe6tgT9jluRVn/3fVryrQYALJsXgdkHKl
         BJBVCl8+WNVZH66xbfOD1P0475HTzHFoIXwqhVH8zcu9wMLYCp8gbaE9huPfYQN8drHu
         L+4Q==
X-Gm-Message-State: AOAM532nUpQdb4cf/npntbVLSEgXLwCLW9H0SgHz235leK0CkZWihTJx
	wavgTszQ30COc74dmWfdzKanmW2TFDKEuQ==
X-Google-Smtp-Source: ABdhPJx//uwDyEpU+2t+QokFKVXyOJTKl843DWiDZIVY1xjFqAPBVhS1g6OynE9Q3ko70Y1LD8aKUQ==
X-Received: by 2002:a05:6512:2308:: with SMTP id o8mr4026378lfu.13.1632401682978;
        Thu, 23 Sep 2021 05:54:42 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 02/11] xen/arm: Add new device type for PCI
Date: Thu, 23 Sep 2021 15:54:29 +0300
Message-Id: <20210923125438.234162-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add new device type (DEV_PCI) to distinguish PCI devices from platform
DT devices, so some drivers, like IOMMU, can handle PCI devices
differently.

Also add a helper which is when given a struct device returns the
corresponding struct pci_dev which this device is a part of.

Because of the header cross-dependencies, e.g. we need both
struct pci_dev and struct arch_pci_dev at the same time, this cannot be
done with an inline. Macro can be implemented, but looks scary:

 #define dev_to_pci_dev(dev) container_of((container_of((dev), \
                        struct arch_pci_dev, dev), struct pci_dev, arch)

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - Folded new device type (DEV_PCI) into this patch.
---
 xen/arch/arm/pci/pci.c       | 10 ++++++++++
 xen/include/asm-arm/device.h |  4 ++--
 xen/include/asm-arm/pci.h    |  7 +++++++
 3 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index bb15edbccc90..e0420d0d86c1 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -27,6 +27,16 @@ int arch_pci_clean_pirqs(struct domain *d)
     return 0;
 }
 
+struct pci_dev *dev_to_pci(struct device *dev)
+{
+    struct arch_pci_dev *arch_dev;
+
+    ASSERT(dev->type == DEV_PCI);
+
+    arch_dev = container_of((dev), struct arch_pci_dev, dev);
+    return container_of(arch_dev, struct pci_dev, arch);
+}
+
 static int __init dt_pci_init(void)
 {
     struct dt_device_node *np;
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 64aaa2641b7f..12de217b36b9 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -4,6 +4,7 @@
 enum device_type
 {
     DEV_DT,
+    DEV_PCI,
 };
 
 struct dev_archdata {
@@ -27,8 +28,7 @@ typedef struct device device_t;
 
 #include <xen/device_tree.h>
 
-/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
-#define dev_is_pci(dev) ((void)(dev), 0)
+#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
 #define dev_is_dt(dev)  ((dev)->type == DEV_DT)
 
 enum device_class
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index d2728a098a11..9e366ae67e83 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -27,6 +27,13 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/*
+ * Because of the header cross-dependencies, e.g. we need both
+ * struct pci_dev and struct arch_pci_dev at the same time, this cannot be
+ * done with an inline here. Macro can be implemented, but looks scary.
+ */
+struct pci_dev *dev_to_pci(struct device *dev);
+
 /* Arch-specific MSI data for vPCI. */
 struct vpci_arch_msi {
 };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:54:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:54:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194102.345776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFe-0005ek-Hd; Thu, 23 Sep 2021 12:54:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194102.345776; Thu, 23 Sep 2021 12:54:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFe-0005ed-E9; Thu, 23 Sep 2021 12:54:58 +0000
Received: by outflank-mailman (input) for mailman id 194102;
 Thu, 23 Sep 2021 12:54:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFe-0004it-2w
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:54:58 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a781f2b3-ebc0-47c6-9470-1f1b9209b024;
 Thu, 23 Sep 2021 12:54:45 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id i4so26246242lfv.4
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:45 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:43 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a781f2b3-ebc0-47c6-9470-1f1b9209b024
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ywsVgZmg0atZGESTkNZBQfqFY254EfvmFk14qbemHso=;
        b=S/31GUMsxvRrP1XF6wl+HRwUSizur6tWU9DOrgB+Ip6QXSpropykEsQM6TJxFL5L9y
         8QMiKwqJExoqZMiRbA3aft4VOtivvFty7EZPZK1bWVH/Qw2q3KRqDQdvpNKnpH7zKX6I
         IPNHkrWWsbvafPdYrmNnfkJIe3jaSVS5w3433LAQNq0Hgas1sPNNFWL7jrQcHVxW3/h0
         4frBJ80C9L0z7cTC1nYENtGaOrJq4R7IdXFN4m2P9uwPJywJeF/eLCiXOQsbFvimVN4V
         2LBMK3iCwNMhBapteKX76nLE/2XSbnLWL32t6hu/9UwQm+4A8n2BhbVEeOO6MlK9LbMU
         87HQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ywsVgZmg0atZGESTkNZBQfqFY254EfvmFk14qbemHso=;
        b=eljRRZvQYD+1irFffV2anWJG3uoNLDSmcif609iBaRvAzvbPCN6NPGDovNabXKx1P+
         ZYx+QsjFNBbto220gIDZOUvGzxHWN8x6Pqlqy6qPuAoQIEyZq4AWMQRt/EsHmrHYvjF0
         fJbBk5CDMqXsvbMYdZcJzxn42+tbF+nppMeAB7e/A6jTcN/lbLRqHfXT/9gp8+4qc806
         zBchb1d4KjwIxgws2zgZhQ5M0UTJ5HC+/yambqji2uOPuE9CBomSa0aKdtjeDM1NOOom
         l+16HOS4tzAb8M/L6RgLTRnkTgMDbO3lMCBUOvaSrEr9JSA8qK0M4gP6z8Uc7sBNSqzr
         P4ig==
X-Gm-Message-State: AOAM530Kv9TZySRdR6g+rCcWRSedvnOWoSUOxXKi//cLqUO7OrQCHVOC
	slJ4+FGaN6WkpTJzXG3Bdc18jSI407wzmg==
X-Google-Smtp-Source: ABdhPJx2xtdoiUQBwDDZezKg8Zgt46DrsOL+/EVGJ+gC9vZkBAAVXbSkfyKXTZZgyCDRekfYMXmE+A==
X-Received: by 2002:a2e:7c0a:: with SMTP id x10mr4967251ljc.455.1632401683981;
        Thu, 23 Sep 2021 05:54:43 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 03/11] xen/arm: Introduce pci_find_host_bridge_node helper
Date: Thu, 23 Sep 2021 15:54:30 +0300
Message-Id: <20210923125438.234162-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Get host bridge node given a PCI device attached to it.

This helper will be re-used for adding PCI devices by the subsequent
patches.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
 xen/arch/arm/pci/pci-host-common.c | 17 +++++++++++++++++
 xen/include/asm-arm/pci.h          |  1 +
 2 files changed, 18 insertions(+)

diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index a88f20175ea9..1567b6e2956c 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -283,6 +283,23 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
     return -EINVAL;
 }
 
+/*
+ * Get host bridge node given a device attached to it.
+ */
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
+{
+    struct pci_host_bridge *bridge;
+    struct pci_dev *pdev = dev_to_pci(dev);
+
+    bridge = pci_find_host_bridge(pdev->seg, pdev->bus);
+    if ( unlikely(!bridge) )
+    {
+        printk(XENLOG_ERR "Unable to find PCI bridge for "PRI_pci"\n",
+               pdev->seg, pdev->bus, pdev->sbdf.dev, pdev->sbdf.fn);
+        return NULL;
+    }
+    return bridge->dt_node;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 9e366ae67e83..5b100556225e 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -103,6 +103,7 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:55:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:55:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194104.345788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFj-00069W-Rj; Thu, 23 Sep 2021 12:55:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194104.345788; Thu, 23 Sep 2021 12:55:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFj-00069K-O4; Thu, 23 Sep 2021 12:55:03 +0000
Received: by outflank-mailman (input) for mailman id 194104;
 Thu, 23 Sep 2021 12:55:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFj-0004it-36
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:03 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f9edd8dd-1b80-4a1e-8fa5-90642b8eaa89;
 Thu, 23 Sep 2021 12:54:46 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id z24so26706855lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:46 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9edd8dd-1b80-4a1e-8fa5-90642b8eaa89
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=yvjDacI417BPdKlJvGk7p+c4OF7HOtW+eGbmd4BLwuQ=;
        b=KOEKA0CpK+2fx5D8i0IiC0UcNJ/awgi6r6h3MF+wm7BRKSywc3+9cvPq9lK1ldbvVY
         EYGyrKDvqHOiz5P9ip+8adsBWwjA6oy3VhcUlYED1KOIxEdKFrLXjdoUQ9JuAXR/7PtJ
         WtISAU9iKzh3nSH9rougHWArA+9Z4dW07ff4PaOFjrDe+7a28spqEEv3ON2hZzYwfd/Q
         C3sNpeq3Z6Ab8EqyG06A3nfyN3BhBQ99XH9KjDKXzNrOKXZqIl26S3cpv5KVF6Ptca9E
         Jl50A4MC/9vspNZNPNnO4q6j9tKZWtCX2zgfkc3TeyAEB3vapwjc1PQd6r9KjzbeIHpC
         sOmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=yvjDacI417BPdKlJvGk7p+c4OF7HOtW+eGbmd4BLwuQ=;
        b=2s2X6EtyT57HVCmhiEsr0lTzfyz+rTrmBmPzzM/Rw56ucbeEYJ0rRZe2bQN1q8qlTl
         DJ5rq+k2lXhF1F3SrimBIFFL+bZw49hGMfmUikGu43yfJULJyNt7uQrP3juKXEVoiimw
         eomvl95cf+RcN7cwfUwf4lqFumRdEJB1NwR910Cce9UAzwezoxt0NXdHWNf2MsP4FPpn
         MunFb+3G7WtjcpkKNpQkbprm4BDttiNePfpXAm5Q7nyP3Wpda9bcFKc9/KULjMzoQ618
         tz5WuRPNtlEh13ZwRRszl/PqR8WSjdkxCPd4oSXgAgyfH0gO+GKk7TnzLcI20Er02mYP
         NDTw==
X-Gm-Message-State: AOAM531GsWQjAcfh1sRrOYX//hTul6ajy/anrVRb2uiFP/cnG4wYi8zj
	ZQJ6pKRE/GabCj/CwRO4JwcxscvdVRJbVg==
X-Google-Smtp-Source: ABdhPJyT22i21H5QrosLmLTvjvbY/GOQr+1/ScI3gtJHYC4bLwkBlXfvXTV5VzcTJKuzm1EhUp3eEw==
X-Received: by 2002:a19:c1c3:: with SMTP id r186mr4106670lff.400.1632401684938;
        Thu, 23 Sep 2021 05:54:44 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 04/11] xen/device-tree: Make dt_find_node_by_phandle global
Date: Thu, 23 Sep 2021 15:54:31 +0300
Message-Id: <20210923125438.234162-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Make dt_find_node_by_phandle globally visible, so it can be re-used by
other frameworks.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/common/device_tree.c      | 2 +-
 xen/include/xen/device_tree.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index ea93da1725f6..4aae281e89bf 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1047,7 +1047,7 @@ int dt_for_each_range(const struct dt_device_node *dev,
  *
  * Returns a node pointer.
  */
-static struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
 {
     struct dt_device_node *np;
 
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 9069040ef7f7..3334048d3bb5 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -850,6 +850,8 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
  */
 int dt_get_pci_domain_nr(struct dt_device_node *node);
 
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle);
+
 #ifdef CONFIG_DEVICE_TREE_DEBUG
 #define dt_dprintk(fmt, args...)  \
     printk(XENLOG_DEBUG fmt, ## args)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:55:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:55:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194106.345799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFp-0006i3-7I; Thu, 23 Sep 2021 12:55:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194106.345799; Thu, 23 Sep 2021 12:55:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFp-0006hr-30; Thu, 23 Sep 2021 12:55:09 +0000
Received: by outflank-mailman (input) for mailman id 194106;
 Thu, 23 Sep 2021 12:55:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFo-0004it-4U
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:08 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f66fc0e9-a9da-4562-ac96-1818e2faaa98;
 Thu, 23 Sep 2021 12:54:47 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id b15so25733308lfe.7
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:47 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f66fc0e9-a9da-4562-ac96-1818e2faaa98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=7U9a60NMzbH9p0acv4bSjt13eIGtF1+DTnt5NlygFls=;
        b=Ml9zeKG++Ld2AfqsBT77oaujf9ckmq47WrQi7mDfp3hwnG1gIz5YT7UhV7KYZCiP3G
         vufCvHDbRMUlQLZHIBnuzvT/unQKGI4w2EQ0IGY6Pi0gPE0Io/DwdNvSX0iiqK7s5rwS
         +6b/5MzhcwdHXCkdFkfD/SdllAO4IsGYyKEHsSd3MZlYovly8miq16qXQtQAOhwolWDS
         jUWJTJsO16yPrhvhcbDp1ErLjzeqDMHIE/WkHFQcV76aELl85nnbMFjiMK0ozg4UTt/q
         HPyYlxRyZ5tjcF9grNmNSFDCw27nLZ2D4NBfkcUzASz3N4jmTdTkVnVV1kiInULS7Rkr
         K3qg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=7U9a60NMzbH9p0acv4bSjt13eIGtF1+DTnt5NlygFls=;
        b=rNJhTzfFl+pxmtAWIbcjt0e2yXsQOTU37aSwsCYlmvAhhfk+0H5Iwyr89yAZ+m1S6Z
         x2l/wFr6Uix40uosyiZEVU9qaIOy870gE5HAe2SZ/ScCs/oTinq+gYVKexCAAt89S61x
         gS1A2XFSdAXgP/1tdWKlx5GgNQ3Da2CM17/wRB424nhzeI/Wo9sn6m4HoiGStWtvpksA
         gGgV9Qum14HkMZ1Pv5j5VeUXpt0CHQWWLBC42ra4wyzu17cNQDeqe5VGwTb+xEZbef/Y
         UgwJCDHUIb/mDhlZfGG1UVfGmpdkcs98TCk0CK1GbJ50GNpU91dgUunhXhNJ9NsiqIbR
         onbw==
X-Gm-Message-State: AOAM530SZxYSnyja0F/y8l4emfRA9AoBcWplfVfbFvY5SihTdZuzq02Z
	7Wqn3f0WeAidx+wqyAJQZosp/d2TFD7O2A==
X-Google-Smtp-Source: ABdhPJzqrMZnYsb4N9IlBnYa8xgiZ3yr1gWFMLelnAwzhGFLJ2/uDp0cj6r6FbGXmwrcmNHNKRuPng==
X-Received: by 2002:a2e:144a:: with SMTP id 10mr5292738lju.149.1632401686081;
        Thu, 23 Sep 2021 05:54:46 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Date: Thu, 23 Sep 2021 15:54:32 +0300
Message-Id: <20210923125438.234162-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

While adding a PCI device mark it as such, so other frameworks
can distinguish it form DT devices.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - Moved the assignment from iommu_add_device to alloc_pdev
---
 xen/drivers/passthrough/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 633e89ac1311..fc3469bc12dc 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     *((u8*) &pdev->bus) = bus;
     *((u8*) &pdev->devfn) = devfn;
     pdev->domain = NULL;
+#ifdef CONFIG_ARM
+    pci_to_dev(pdev)->type = DEV_PCI;
+#endif
 
     rc = pdev_msi_init(pdev);
     if ( rc )
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:55:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:55:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194108.345810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFt-0007HV-P4; Thu, 23 Sep 2021 12:55:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194108.345810; Thu, 23 Sep 2021 12:55:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFt-0007HH-JH; Thu, 23 Sep 2021 12:55:13 +0000
Received: by outflank-mailman (input) for mailman id 194108;
 Thu, 23 Sep 2021 12:55:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFt-0004it-3g
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:13 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 57490847-57d1-49bd-b0a4-403c336c5255;
 Thu, 23 Sep 2021 12:54:48 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id e15so26186690lfr.10
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:48 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57490847-57d1-49bd-b0a4-403c336c5255
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=1NF2VdRzcAopkt/ag2bsT0u3KlYDxtxCU6z2CshuS9A=;
        b=c3W4pHiP0SQbXwXikrCRW/5HNiXnuy0GiKtdlTMWgAt+q5OKH4BJh+UUyzBsXiVDS4
         zPujaViJYEAM1QBWCUMzgq0wdSiZnFKwIkmo4ONopW1JG57hen9QfipVt6XjuZv5dpHN
         +YZTLjE3or8uWGVSg/JLS4x8b8vO46ynKoDjhM76qSZMVy23quiQ7/h5HTNam0Chti+d
         cHX6/fjEQyzwcG8VH0oaZiSQfWyva9YO9Pn+bnG6KgJL3cn/+TJLLMZxWdxC67rMJN3b
         78QCugavjre0R6syfKKbFQJo858XMtYRKwARobETSbeIjp/EG7uAtioOjwSw/slcBrGg
         KlDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=1NF2VdRzcAopkt/ag2bsT0u3KlYDxtxCU6z2CshuS9A=;
        b=MIqAw/6marq3Squ39n51JKi53v5TAx7FPvbNOGOVkEkTaJ3+6NldJ016FN+7++bHqZ
         xYn+p77TsjnqH+DvuDrA9dCxhvY+HnA3v/WXfBD7d7tcz4oSPI+fVNDTXEWCPi5K4efy
         W9ua1lUu0jThWaOcvJJ3QOsnSksoHbxswNiWjlQbPgIktBLfgW8yzMelec5855Sh39k/
         aq+X37Dq7YZhN6mpeSipeMaUJODjheBoaavj70IxDEncNVp347fYyZnzVEWAsPknI5DP
         hCvsISO+LPuAO6xot2d1omY5STNdIZsSZMKb1rBKlUv8szPVTNNCwqlUWP3/iAZvTFQH
         RcaQ==
X-Gm-Message-State: AOAM533dDIMSbctiEsiG9Kj8UEovxrU00HoOAJHPR0HihxHZx1gX1deh
	YLm8bRUc5aJVgD0jkceQsHAdQogl7cRTzw==
X-Google-Smtp-Source: ABdhPJzELccAtlQYBFFcQSEcUG96n2lCiiukq2T8exQ27ScjvOLHxHZt3jzH71KXNO1heo96QkLGcw==
X-Received: by 2002:a05:651c:3c3:: with SMTP id f3mr5182916ljp.222.1632401687403;
        Thu, 23 Sep 2021 05:54:47 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com
Subject: [PATCH v2 06/11] xen/domain: Call pci_release_devices() when releasing domain resources
Date: Thu, 23 Sep 2021 15:54:33 +0300
Message-Id: <20210923125438.234162-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

This is the very same that we already do for DT devices. Moreover, x86
already calls pci_release_devices().

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

---
Since v1:
 - re-wording in the commit message
---
 xen/arch/arm/domain.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index f7ed130023d5..854e8fed0393 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -985,7 +985,8 @@ static int relinquish_memory(struct domain *d, struct page_list_head *list)
  * function which may return -ERESTART.
  */
 enum {
-    PROG_tee = 1,
+    PROG_pci = 1,
+    PROG_tee,
     PROG_xen,
     PROG_page,
     PROG_mapping,
@@ -1022,6 +1023,12 @@ int domain_relinquish_resources(struct domain *d)
 #ifdef CONFIG_IOREQ_SERVER
         ioreq_server_destroy_all(d);
 #endif
+#ifdef CONFIG_HAS_PCI
+    PROGRESS(pci):
+        ret = pci_release_devices(d);
+        if ( ret )
+            return ret;
+#endif
 
     PROGRESS(tee):
         ret = tee_relinquish_resources(d);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:55:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:55:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194110.345820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFz-0007uj-4X; Thu, 23 Sep 2021 12:55:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194110.345820; Thu, 23 Sep 2021 12:55:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOFz-0007ua-0i; Thu, 23 Sep 2021 12:55:19 +0000
Received: by outflank-mailman (input) for mailman id 194110;
 Thu, 23 Sep 2021 12:55:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOFy-0004it-3r
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:18 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1abdf6a5-2656-4396-8478-8e678bb67c52;
 Thu, 23 Sep 2021 12:54:50 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id t10so26348964lfd.8
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:50 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:48 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1abdf6a5-2656-4396-8478-8e678bb67c52
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=A/GZKDhDtUFyfAXRqux8zSAjOsg67lOWqEd8gnbAkBjf2ZcGl8o6Lt2rIX9xn/hQzZ
         rboPJFgUMKOheXmwh3W/Jd2UFU683MeJ1NIGWoN+edqTbrnPSnsl28LpC9JyN1tDT0k0
         Xf2Bn3bHXhIaitQTbOqI5ThtQH4kBtOS8Fwf46mytqTm8l9Dxe5Ewfgfan1wj69tp0wD
         pLrMJx8cFSTkjiDbWoW8yYPhsRwrjufX43JQOguZ1/NbfCO1sBLv4hFk0P8RD49NrO3Z
         wcBip5MjV3N4WQmtEH70DxZXCJOxDbkiQkdkg87+UK+st8ok6tvvp9XqAzXeYQW9fQb9
         woBw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=4G/cqHJoElX1BDeWf9th8pyS1aPy1uFMhhLbuSIFI/JfGdD3qH6nl/14dMOdUkjdhu
         luiSGvyh/tngYD7Z0nWIaXrlXSxdonYNJO9fxMQ48/SYXNSDrjpl/MLsApqmDFqDU02g
         j/8iQ9z6MsPSBDQ7w4KGObUun9BboTHGYuaiZEzTY3L0pthT+6RkwQCXqqHOKOVIZcJb
         mT5b2th6QGYtoOcp4KItNUki+ICChLRSy8qryHxyxxbivQwYbuXoudYDkUt6ZlEFxfCI
         VxIVs8Tzyj0Hw09ot264Uo4TABLyW3qN512NZlStCjP0e5ACBYNa91A5Q12y0Z1vqvoZ
         gJ3A==
X-Gm-Message-State: AOAM531/Ilh1s3X+oxBB6TfeM5/ZgBT/luJ01jMVYLKIOjTzAfnrPFnH
	zJfy0OC/ZuLtUwAF92H1L2mjXogV4PBJpw==
X-Google-Smtp-Source: ABdhPJyPiC8+p5vTdecL32CVkX94l6rOzV6gWgEpMuOEU8GwMMl02WWx8nK2vMBCUYk8LpGC2ea0bQ==
X-Received: by 2002:a2e:9c43:: with SMTP id t3mr5005948ljj.198.1632401689231;
        Thu, 23 Sep 2021 05:54:49 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 07/11] libxl: Allow removing PCI devices for all types of domains
Date: Thu, 23 Sep 2021 15:54:34 +0300
Message-Id: <20210923125438.234162-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

The PCI device remove path may now be used by PVH on ARM, so the
assert is no longer valid.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>
---
 tools/libs/light/libxl_pci.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 1a1c2630803b..59f3686fc85e 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1947,8 +1947,6 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
             goto out_fail;
         }
     } else {
-        assert(type == LIBXL_DOMAIN_TYPE_PV);
-
         char *sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", pci->domain,
                                      pci->bus, pci->dev, pci->func);
         FILE *f = fopen(sysfs_path, "r");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:55:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194112.345832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOG4-0008Uc-G7; Thu, 23 Sep 2021 12:55:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194112.345832; Thu, 23 Sep 2021 12:55:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOG4-0008UR-Aj; Thu, 23 Sep 2021 12:55:24 +0000
Received: by outflank-mailman (input) for mailman id 194112;
 Thu, 23 Sep 2021 12:55:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOG3-0004it-3p
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:23 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 915cde92-d924-4b0b-9871-d2ec30d41a07;
 Thu, 23 Sep 2021 12:54:52 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id b20so26421266lfv.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:52 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 915cde92-d924-4b0b-9871-d2ec30d41a07
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=1TrjF5t3o30buNEbCcSZ6J+pkIyzH+w8BkXrdAyRKN4=;
        b=ivM99SULTQHo3nshuAEGZKlbuXXaVaw6z/XPmvCfBqjSwhxnIlZ/bPAK2/B9U2XbZA
         DLyBvRDtzt4cCP0pPvUqlNAzh5PdOzeyNvCU3UI48NsRLk0k/OxA61v4Mq8uG7rzL/Ic
         1IbbjpU6rtXdQfuTpNU7bCaZGg8EAxCcatB07HL2y/aLj7gSeab9hWVuljnxC0134TCO
         xopJ3qRjy86VKkjVH3E6GO7a7X63/EVS9JRHF1aooXlz8Ww90py3FdtmSELFQWjaW/Fy
         dZUXUfAYDI2bfRdoLCWPyMZ6aJzakLCWbEAQ/m2EmrExUy5htTCM6caCg52zHkq18bTw
         zcsQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=1TrjF5t3o30buNEbCcSZ6J+pkIyzH+w8BkXrdAyRKN4=;
        b=fmL9ZY/Nr2Et0lyzOAjrek88FSBc2JDK3sD/yMjV7DeWBm7iKM6qzv4t/EUmmhcvZH
         tMdvzSPH6eI30YVtZh5DxId81uCsS3qTjzXHHe/a9EVK7qvCyAlVx0fla2jKVF5XOlJS
         ymYwpQ/TlTef4Jr+Y5Zd/+awukDZf4jAsZjv+7zKZ0P9YL+v2PmFknffoc25kx22jtAc
         rhUXdYaKo2GuLbxbgYxe8zRcD7GA5PJadM9nnH2Z3n5xevBizGLpGlQHIHdEbYI7mnT9
         6lMweLcoxWp3/lWAN4U0DwKsFH29Xg+50h8B4GLn4VD+pcB0zyFDZ2bFstQ6TVfVXgp9
         9Ydg==
X-Gm-Message-State: AOAM530X8pD6SyIW5O9pnsCtxsVBfg1uGa4ovSqPd6w6e6arb4a/KcHK
	H9/JER35W46bljQ4yNZGHlPAmFtSkxGHPA==
X-Google-Smtp-Source: ABdhPJzPqQaEl2/aIGz3AwANRVvhbdk4cidmrdFeJlcv41FfPhRPal5UzVCuBJYk5BalAfuXbi6WUg==
X-Received: by 2002:a05:6512:2302:: with SMTP id o2mr4119509lfu.297.1632401691504;
        Thu, 23 Sep 2021 05:54:51 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 09/11] xen/arm: Setup MMIO range trap handlers for hardware domain
Date: Thu, 23 Sep 2021 15:54:36 +0300
Message-Id: <20210923125438.234162-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

In order for vPCI to work it needs to maintain guest and hardware
domain's views on the configuration space. For example, BARs and
COMMAND registers require emulation for guests and the guest view
on the registers needs to be in sync with the real contents of the
relevant registers. For that ECAM address space needs to also be
trapped for the hardware domain, so we need to implement PCI host
bridge specific callbacks to properly setup MMIO handlers for those
ranges depending on particular host bridge implementation.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - Dynamically calculate the number of MMIO handlers required for vPCI
   and update the total number accordingly
 - s/clb/cb
 - Do not introduce a new callback for MMIO handler setup
---
 xen/arch/arm/domain.c              |  2 ++
 xen/arch/arm/pci/pci-host-common.c | 28 +++++++++++++++++++++++++
 xen/arch/arm/vpci.c                | 33 ++++++++++++++++++++++++++++++
 xen/arch/arm/vpci.h                |  6 ++++++
 xen/include/asm-arm/pci.h          |  7 +++++++
 5 files changed, 76 insertions(+)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 854e8fed0393..c7b25bc70439 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -733,6 +733,8 @@ int arch_domain_create(struct domain *d,
     if ( (rc = domain_vgic_register(d, &count)) != 0 )
         goto fail;
 
+    count += domain_vpci_get_num_mmio_handlers(d);
+
     if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 )
         goto fail;
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 1567b6e2956c..155f2a2743af 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -300,6 +300,34 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
     }
     return bridge->dt_node;
 }
+
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*cb)(struct domain *d,
+                                       struct pci_host_bridge *bridge))
+{
+    struct pci_host_bridge *bridge;
+    int err;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        err = cb(d, bridge);
+        if ( err )
+            return err;
+    }
+    return 0;
+}
+
+int pci_host_get_num_bridges(void)
+{
+    struct pci_host_bridge *bridge;
+    int count = 0;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+        count++;
+
+    return count;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 76c12b92814f..14947e975d69 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -80,17 +80,50 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
     .write = vpci_mmio_write,
 };
 
+static int vpci_setup_mmio_handler(struct domain *d,
+                                   struct pci_host_bridge *bridge)
+{
+    struct pci_config_window *cfg = bridge->cfg;
+
+    register_mmio_handler(d, &vpci_mmio_handler,
+                          cfg->phys_addr, cfg->size, NULL);
+    return 0;
+}
+
 int domain_vpci_init(struct domain *d)
 {
     if ( !has_vpci(d) )
         return 0;
 
+    if ( is_hardware_domain(d) )
+        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+
+    /* Guest domains use what is programmed in their device tree. */
     register_mmio_handler(d, &vpci_mmio_handler,
                           GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
 
     return 0;
 }
 
+int domain_vpci_get_num_mmio_handlers(struct domain *d)
+{
+    int count = 0;
+
+    if ( is_hardware_domain(d) )
+        /* For each PCI host bridge's configuration space. */
+        count += pci_host_get_num_bridges();
+    else
+        /*
+         * VPCI_MSIX_MEM_NUM handlers for MSI-X tables per each PCI device
+         * being passed through. Maximum number of supported devices
+         * is 32 as virtual bus topology emulates the devices as embedded
+         * endpoints.
+         * +1 for a single emulated host bridge's configuration space. */
+        count = VPCI_MSIX_MEM_NUM * 32 + 1;
+
+    return count;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
index d8a7b0e3e802..27a2b069abd2 100644
--- a/xen/arch/arm/vpci.h
+++ b/xen/arch/arm/vpci.h
@@ -17,11 +17,17 @@
 
 #ifdef CONFIG_HAS_VPCI
 int domain_vpci_init(struct domain *d);
+int domain_vpci_get_num_mmio_handlers(struct domain *d);
 #else
 static inline int domain_vpci_init(struct domain *d)
 {
     return 0;
 }
+
+static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
+{
+    return 0;
+}
 #endif
 
 #endif /* __ARCH_ARM_VPCI_H__ */
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 5b100556225e..7618f0b6725b 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -15,6 +15,8 @@
 #ifndef __ARM_PCI_H__
 #define __ARM_PCI_H__
 
+#include <asm/mmio.h>
+
 #ifdef CONFIG_HAS_PCI
 
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
@@ -109,6 +111,11 @@ static always_inline bool is_pci_passthrough_enabled(void)
 {
     return !!pci_passthrough_enabled;
 }
+
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*clb)(struct domain *d,
+                                        struct pci_host_bridge *bridge));
+int pci_host_get_num_bridges(void);
 #else   /*!CONFIG_HAS_PCI*/
 
 #define pci_passthrough_enabled (false)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194122.345843 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHd-0001ez-Qh; Thu, 23 Sep 2021 12:57:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194122.345843; Thu, 23 Sep 2021 12:57:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHd-0001es-ND; Thu, 23 Sep 2021 12:57:01 +0000
Received: by outflank-mailman (input) for mailman id 194122;
 Thu, 23 Sep 2021 12:57:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHV-0004it-6j
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:53 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5945c790-8869-4b99-afdc-a6c7733b4bcc;
 Thu, 23 Sep 2021 12:55:13 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id z24so26712439lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:13 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5945c790-8869-4b99-afdc-a6c7733b4bcc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=/VZhQBHA/qWsZnQaFLYfN7XFH1H4UA39toP1GBs37TM=;
        b=P8Hj9IuBQQDngliolaPP7ZmvBgMYQvbZ3Apn/ouQ8J5yPra/sFgxauc0LD/tzbp8wo
         rQVHv5c8j9RWxG5p+ChF6f6tuID8Q5zvstZjZXEgtkX07RIVDTs+GA7+lz++OaiMcLhi
         IFjpzJZaoO3dkvtRJMTIhAmVdMP2OA+9A/SwKLgTT56y7vVw7+JpwsebBeCIWEfcb1hD
         M/puTGIUrPojnOjh1Pv8I8JEqW926nKjvKg3bpiEvoid8ZqJ+/nD/+SNZGC+xtHx2zhb
         PUk7NxVks2afQC+Z3eZ0AeqP4bhyZ5n3O66dDvY+S8vQsxsiz6I0/wUTjrG+70nJjuNw
         ZCyQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=/VZhQBHA/qWsZnQaFLYfN7XFH1H4UA39toP1GBs37TM=;
        b=JmCcsFNyCR1+yeqn4tUT48yvyQ6/eojjMBv74n7WWxb2axj3GbIw8LTWCrqQT7rBz2
         kY3IUSr0IblsBKKuvSkkxs/bpdmr7WRNt7ii9X5dS9vn1pIidADsMTZxOFp+Yq+Jb2Ja
         sC/n7WSm0ucGzH/pvs4dNZ+X5SUfG8MSZYAaboGRED3lUdW8UdHyj9PxjBWfZuOzNBK2
         9duN9CI5AgiZoU9/ZBInGoM3EnusrmEYLKVhkU6Ob8NODqkFayMmIw/4/xF4bF7Px12f
         E6wcacS3lwvD3tBduD39I/sk8X8fp3XhBOofJFOPKQYxJQtWgXiY30/AHTtfV1RYBAcj
         +/og==
X-Gm-Message-State: AOAM533QoIhod2VbBZrkh0F8oAk/l1evunRrcJbxJTN7rjtqnVj1IhYv
	BXNJLWZea9MKEJDIEw2WWPaUhjvxW5snYw==
X-Google-Smtp-Source: ABdhPJxVZKYeKANBbtQt4yOeYPS1abKifRIc3+NhdRNVbgxW2Y/9Ry0MALLSfk/9SB4lbTL6En8Icg==
X-Received: by 2002:a05:6512:110a:: with SMTP id l10mr4093541lfg.550.1632401711887;
        Thu, 23 Sep 2021 05:55:11 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Date: Thu, 23 Sep 2021 15:54:57 +0300
Message-Id: <20210923125501.234252-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Take into account guest's BAR view and program its p2m accordingly:
gfn is guest's view of the BAR and mfn is the physical BAR value as set
up by the host bridge in the hardware domain.
This way hardware doamin sees physical BAR values and guest sees
emulated ones.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - s/MSI/MSI-X in comments
---
 xen/drivers/vpci/header.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 9c603d26d302..bdd18599b205 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -30,6 +30,10 @@
 
 struct map_data {
     struct domain *d;
+    /* Start address of the BAR as seen by the guest. */
+    gfn_t start_gfn;
+    /* Physical start address of the BAR. */
+    mfn_t start_mfn;
     bool map;
 };
 
@@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
                      unsigned long *c)
 {
     const struct map_data *map = data;
+    gfn_t start_gfn;
     int rc;
 
     for ( ; ; )
     {
         unsigned long size = e - s + 1;
 
+        /*
+         * Any BAR may have holes in its memory we want to map, e.g.
+         * we don't want to map MSI-X regions which may be a part of that BAR,
+         * e.g. when a single BAR is used for both MMIO and MSI-X.
+         * In this case MSI-X regions are subtracted from the mapping, but
+         * map->start_gfn still points to the very beginning of the BAR.
+         * So if there is a hole present then we need to adjust start_gfn
+         * to reflect the fact of that substraction.
+         */
+        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
+
+        printk(XENLOG_G_DEBUG
+               "%smap [%lx, %lx] -> %#"PRI_gfn" for d%d\n",
+               map->map ? "" : "un", s, e, gfn_x(start_gfn),
+               map->d->domain_id);
         /*
          * ARM TODOs:
          * - On ARM whether the memory is prefetchable or not should be passed
@@ -52,8 +72,10 @@ static int map_range(unsigned long s, unsigned long e, void *data,
          * - {un}map_mmio_regions doesn't support preemption.
          */
 
-        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
-                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
+        rc = map->map ? map_mmio_regions(map->d, start_gfn,
+                                         size, _mfn(s))
+                      : unmap_mmio_regions(map->d, start_gfn,
+                                           size, _mfn(s));
         if ( rc == 0 )
         {
             *c += size;
@@ -69,6 +91,7 @@ static int map_range(unsigned long s, unsigned long e, void *data,
         ASSERT(rc < size);
         *c += rc;
         s += rc;
+        gfn_add(map->start_gfn, rc);
         if ( general_preempt_check() )
                 return -ERESTART;
     }
@@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
             if ( !bar->mem )
                 continue;
 
+            data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ?
+                _gfn(PFN_DOWN(bar->addr)) :
+                _gfn(PFN_DOWN(bar->guest_addr));
+            data.start_mfn = _mfn(PFN_DOWN(bar->addr));
             rc = rangeset_consume_ranges(bar->mem, map_range, &data);
 
             if ( rc == -ERESTART )
@@ -194,6 +221,10 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
         if ( !bar->mem )
             continue;
 
+        data.start_gfn = is_hardware_domain(d) ?
+            _gfn(PFN_DOWN(bar->addr)) :
+            _gfn(PFN_DOWN(bar->guest_addr));
+        data.start_mfn = _mfn(PFN_DOWN(bar->addr));
         while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
                                               &data)) == -ERESTART )
             process_pending_softirqs();
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194124.345854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHn-00023Z-2N; Thu, 23 Sep 2021 12:57:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194124.345854; Thu, 23 Sep 2021 12:57:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHm-00023S-VC; Thu, 23 Sep 2021 12:57:10 +0000
Received: by outflank-mailman (input) for mailman id 194124;
 Thu, 23 Sep 2021 12:57:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOGw-0004it-5t
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:18 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b26bf262-48c3-45ed-bbc2-aa8f8a0ed16e;
 Thu, 23 Sep 2021 12:55:07 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id u18so25609793lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:07 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b26bf262-48c3-45ed-bbc2-aa8f8a0ed16e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=xjptWExU6TWM7wvE9QyWUsFVZpNpHRTTMdpxk2EhC08=;
        b=qKoY6kLGWNocUC9YkflO59mYUD/aorNBD60p3xgQ2vM+uFVxL45hsNzjZhiDVSldyt
         vYa6gNOodlA9lptZuPstRE52EhHkvCvAtsyyfp7ChzF4xhwVNMLBiRmPgWyz9imHUYv+
         7o1AywfCBf2Ok2/x1Yk8aIbBbI5sMmg/dtFM06U7oGTGSJnGptkgWmnSt07Qp78WaLZy
         MbA5gvhVyYKQH0f6s6d8rOVjetw2uTvxCEp0RK8jYTar4QidFIJemDkSl2I6yS63kJB3
         2Se4o/x4n36AIjDzDYPMStgYdii2TqjxvlyAtbyehQ5adOyKrEVeWpiEOZx0QnMUZ9WO
         R7Mw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=xjptWExU6TWM7wvE9QyWUsFVZpNpHRTTMdpxk2EhC08=;
        b=N/lXeT7cqTRc+V5RmuGAd+iB7mzzi0Wtv4/e0KVOJThM3fki9jqQmxxiyeKh4dm4uO
         B7eNPexuRvBmvNRv5XlYWiZ5BbrB5J5fd9ptyNBScCAR0v9agfaNQkAlBP6lGVK5NPeA
         dnrqxWMbe6BOSgQpyyl6CKetTV6wqj4YLEe0QdLPqlSKcEWlzM5wDMkbq9SDPuhKiTLl
         xhYQEn50IKjAGkVWlkL/Z71OzH4K+gkkntjLWc4PoWFtSvur0XlQLg833txUrnDsw7PH
         RoQuKgX30Est5bDnoZ0wwKTNUtfrTMsbb27goW52KTTVbsxvL4POsgV1oODZCWXuktFN
         EBdw==
X-Gm-Message-State: AOAM530c75IBikLNpC1KxUvGYffMBSnShg2KcoAKcfsf0raeWzgxaVJh
	uzL4aXgf4TArjXMBrqGd1DrzZ0zK4lKX3Q==
X-Google-Smtp-Source: ABdhPJxwYqEoAkmKlLJL7pG8wGA8T/J8LGPSsCQnLc2yjHBgONmMkR8ingCNcm4T6tjqEAz/b6x9Jg==
X-Received: by 2002:ac2:5467:: with SMTP id e7mr3860249lfn.537.1632401703781;
        Thu, 23 Sep 2021 05:55:03 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 00/11] PCI devices passthrough on Arm, part 3
Date: Thu, 23 Sep 2021 15:54:50 +0300
Message-Id: <20210923125501.234252-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This patch series is focusing on vPCI and adds support for non-identity
PCI BAR mappings which is required while passing through a PCI device to
a guest. The highlights are:

- Add relevant vpci register handlers when assigning PCI device to a domain
  and remove those when de-assigning. This allows having different
  handlers for different domains, e.g. hwdom and other guests.

- Emulate guest BAR register values based on physical BAR values.
  This allows creating a guest view of the registers and emulates
  size and properties probe as it is done during PCI device enumeration by
  the guest.

- Instead of handling a single range set, that contains all the memory
  regions of all the BARs and ROM, have them per BAR.

- Take into account guest's BAR view and program its p2m accordingly:
  gfn is guest's view of the BAR and mfn is the physical BAR value as set
  up by the host bridge in the hardware domain.
  This way hardware doamin sees physical BAR values and guest sees
  emulated ones.

The series also adds support for virtual PCI bus topology for guests:
 - We emulate a single host bridge for the guest, so segment is always 0.
 - The implementation is limited to 32 devices which are allowed on
   a single PCI bus.
 - The virtual bus number is set to 0, so virtual devices are seen
   as embedded endpoints behind the root complex.

The series was also tested on x86 PVH Dom0 and doesn't break it.

Thank you,
Oleksandr

Oleksandr Andrushchenko (11):
  vpci: Make vpci registers removal a dedicated function
  vpci: Add hooks for PCI device assign/de-assign
  vpci/header: Move register assignments from init_bars
  vpci/header: Add and remove register handlers dynamically
  vpci/header: Implement guest BAR register handlers
  vpci/header: Handle p2m range sets per BAR
  vpci/header: program p2m with guest BAR view
  vpci/header: Emulate PCI_COMMAND register for guests
  vpci/header: Reset the command register when adding devices
  vpci: Add initial support for virtual PCI bus topology
  xen/arm: Translate virtual PCI bus topology for guests

 xen/arch/arm/domain.c         |   1 +
 xen/arch/arm/vpci.c           |  87 ++++++-
 xen/arch/arm/vpci.h           |   3 +
 xen/common/domain.c           |   1 +
 xen/drivers/Kconfig           |   4 +
 xen/drivers/passthrough/pci.c |  91 ++++++++
 xen/drivers/vpci/header.c     | 412 +++++++++++++++++++++++++++-------
 xen/drivers/vpci/vpci.c       |  48 +++-
 xen/include/asm-arm/pci.h     |   1 +
 xen/include/xen/pci.h         |  19 ++
 xen/include/xen/sched.h       |   8 +
 xen/include/xen/vpci.h        |  34 ++-
 12 files changed, 617 insertions(+), 92 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194125.345858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHn-00027K-DR; Thu, 23 Sep 2021 12:57:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194125.345858; Thu, 23 Sep 2021 12:57:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOHn-000266-8l; Thu, 23 Sep 2021 12:57:11 +0000
Received: by outflank-mailman (input) for mailman id 194125;
 Thu, 23 Sep 2021 12:57:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHG-0004it-6L
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:38 +0000
Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d0baee5-c887-4f26-b64c-1231e4c064d0;
 Thu, 23 Sep 2021 12:55:09 +0000 (UTC)
Received: by mail-lf1-x136.google.com with SMTP id t10so26352813lfd.8
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:09 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d0baee5-c887-4f26-b64c-1231e4c064d0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=8BU9mTDeAzJlCIShSCG3B8dpXtNyqwk/IzMfbTSVhQk=;
        b=EMo4gYUd9HuWyuM57Ppq0ISKssPDVmxqd/a2Lx8YxHs+mmxcN3gEsd/EPXpnRYNWZp
         xUlW26eHqyQlqEpQH2chnUCMJNrSGZO3Olbx+hOgheIIXNim7uQ4qz1WphFudbVt50hd
         BCE6CDc6s2BR9ov02TwJG7FxwD0KVKAB8inrPSF6WAz2svtSEwAjq2zndY8KtQBCs2Bt
         HImAJ9sxfgDV5pyymroSzP67w/57I5BESX9bfWnOdMX54tMd6Ku2IEzbJ+9gjj97OMYH
         lbjW5qmWvr4ZF3gWxm6X3XoGpR5GKQmxcH2VeFEYmbXfqZJOLOnSZqJuBjHlzRFybmot
         cppw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=8BU9mTDeAzJlCIShSCG3B8dpXtNyqwk/IzMfbTSVhQk=;
        b=4a0G+kWshZWoBxi7Xqd6K1O0pb4xD9vF9QBvwoI+2kpIT30DmiXFGhsxycY2KiQi3S
         Lw5b9ssgWW9GSy9SiRzo2bRAcaAo4Yuex4wY0zslgEyQqhd2hDbJd6+O/p2UKgcuGLuf
         2Ry2oZ4bRY0rAOXt+RcG5gOSCDuJKmDzDmguPcS+Mgtippe3FZPtYI/ov8jTS8WHfkx0
         V2GrjXDjxMLpF0RHBkqAM/uyZOow8uCx3gTXnA+PUaAL3kND54isqo/kA6OMlDRfm5xO
         V0mB5IqAo6aerG/eQBCwGEITkgGZ+X4tCiZkuZOpvxuBhzUMlxsIyo5C1q14PH+n6Ju3
         wz5w==
X-Gm-Message-State: AOAM530be766ngxLmP3nQ1ryXOXn/iHx3aoT3hHskcql+DL0REAxtDYC
	uIiuKsby47M63fe6c8Zz40S0P+MwCL9U9A==
X-Google-Smtp-Source: ABdhPJyHMtn4B5EwqiLHMWf9G5BqvVikcJTSRJI2KpV/yKtKmYNsU7V2hYr/2ZAL2nqe2p+lvspzlQ==
X-Received: by 2002:a05:6512:10d0:: with SMTP id k16mr4161319lfg.530.1632401708273;
        Thu, 23 Sep 2021 05:55:08 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 04/11] vpci/header: Add and remove register handlers dynamically
Date: Thu, 23 Sep 2021 15:54:54 +0300
Message-Id: <20210923125501.234252-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add relevant vpci register handlers when assigning PCI device to a domain
and remove those when de-assigning. This allows having different
handlers for different domains, e.g. hwdom and other guests.

Use stubs for guest domains for now.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - simplify some code3. simplify
 - use gdprintk + error code instead of gprintk
 - gate vpci_bar_{add|remove}_handlers with CONFIG_HAS_VPCI_GUEST_SUPPORT,
   so these do not get compiled for x86
 - removed unneeded is_system_domain check
---
 xen/drivers/Kconfig       |  4 +++
 xen/drivers/vpci/header.c | 72 ++++++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c   |  8 +++++
 xen/include/xen/vpci.h    |  8 +++++
 4 files changed, 84 insertions(+), 8 deletions(-)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..f3e3a05a4544 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	depends on HAS_PCI
+
 endmenu
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 3d571356397a..1ce98795fcca 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -397,6 +397,17 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
     pci_conf_write32(pdev->sbdf, reg, val);
 }
 
+static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
                       uint32_t val, void *data)
 {
@@ -445,14 +456,25 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
-static int add_bar_handlers(const struct pci_dev *pdev)
+static void guest_rom_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_rom_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
+static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
 {
     unsigned int i;
     struct vpci_header *header = &pdev->vpci->header;
     struct vpci_bar *bars = header->bars;
     int rc;
 
-    /* Setup a handler for the command register. */
+    /* Setup a handler for the command register: same for hwdom and guests. */
     rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
                            2, header);
     if ( rc )
@@ -475,8 +497,13 @@ static int add_bar_handlers(const struct pci_dev *pdev)
                 rom_reg = PCI_ROM_ADDRESS;
             else
                 rom_reg = PCI_ROM_ADDRESS1;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
-                                   rom_reg, 4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                       rom_reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_rom_read, guest_rom_write,
+                                       rom_reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -485,8 +512,13 @@ static int add_bar_handlers(const struct pci_dev *pdev)
             uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
 
             /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write,
+                                       reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_bar_read, guest_bar_write,
+                                       reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -520,7 +552,7 @@ static int init_bars(struct pci_dev *pdev)
     }
 
     if ( pdev->ignore_bars )
-        return add_bar_handlers(pdev);
+        return add_bar_handlers(pdev, true);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -582,7 +614,7 @@ static int init_bars(struct pci_dev *pdev)
                               PCI_ROM_ADDRESS_ENABLE;
     }
 
-    rc = add_bar_handlers(pdev);
+    rc = add_bar_handlers(pdev, true);
     if ( rc )
     {
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
@@ -593,6 +625,30 @@ static int init_bars(struct pci_dev *pdev)
 }
 REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+int vpci_bar_add_handlers(const struct domain *d, const struct pci_dev *pdev)
+{
+    int rc;
+
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+
+    rc = add_bar_handlers(pdev, is_hardware_domain(d));
+    if ( rc )
+        gdprintk(XENLOG_ERR,
+                 "%pp: failed to add BAR handlers for dom%pd: %d\n",
+                 &pdev->sbdf, d, rc);
+    return rc;
+}
+
+int vpci_bar_remove_handlers(const struct domain *d, const struct pci_dev *pdev)
+{
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+    return 0;
+}
+#endif
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index a8fed3d2c42e..0cdc0c3f75c4 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -94,7 +94,11 @@ int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    return vpci_bar_add_handlers(d, dev);
+#else
     return 0;
+#endif
 }
 
 /* Notify vPCI that device is de-assigned from guest. */
@@ -104,7 +108,11 @@ int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    return vpci_bar_remove_handlers(d, dev);
+#else
     return 0;
+#endif
 }
 
 #endif /* __XEN__ */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index b9485b2aea1b..912cbc6aa7ad 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -63,6 +63,14 @@ uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg,
  */
 bool __must_check vpci_process_pending(struct vcpu *v);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+/* Add/remove BAR handlers for a domain. */
+int vpci_bar_add_handlers(const struct domain *d,
+                          const struct pci_dev *pdev);
+int vpci_bar_remove_handlers(const struct domain *d,
+                             const struct pci_dev *pdev);
+#endif
+
 struct vpci {
     /* List of vPCI handlers for a device. */
     struct list_head handlers;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194129.345876 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI3-00037S-2K; Thu, 23 Sep 2021 12:57:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194129.345876; Thu, 23 Sep 2021 12:57:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI2-000377-TO; Thu, 23 Sep 2021 12:57:26 +0000
Received: by outflank-mailman (input) for mailman id 194129;
 Thu, 23 Sep 2021 12:57:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHL-0004it-6g
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:43 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 96030892-7a61-4732-967d-19f6ffea7ce8;
 Thu, 23 Sep 2021 12:55:09 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id b15so25737297lfe.7
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:09 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96030892-7a61-4732-967d-19f6ffea7ce8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=TDiVVp/85Hu7qvLwikbqJ56SNZPH692g3qboO1XLKAU=;
        b=DH6sRara2lIzT7fiwygzxRyOEAe577X/U+ypPVBFF3U4XBHY3okF7NHXhAa5CI51Wv
         Fw/7ho2wGWStYpkbJMigRCdA+zzNU2ENw6FgfszkkxD9GKyiH27CqGyMG/KCrbP2FLDs
         oTfO8tIjkCLZ58BkutfSpTkZYIHxJMyLxumi5oe85jSGgg3N9yOqUjoc+QmgCWY8kxNV
         ZB8EGKcddtd4yuCpos2HcQM64WWqhJWGxWhp7FyxFSOFjVzHwpjwRw1RUkrak7yZBWxN
         zTkTxwCRoDZH2Cemj7kYlRKwv6wuP/qBCODwtaxVsC5+dedlFvIzSEZiMRSOA/u4r/Fo
         SHNg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=TDiVVp/85Hu7qvLwikbqJ56SNZPH692g3qboO1XLKAU=;
        b=FlZSmYqkQf2Tyl3See2tex9xXOdO5oHHltCsviCHS5fIWWC18VidiU9SOPJSMXnkAq
         qOvXuCxiaOPMXWpsOAoZ2ldPD7AcyqalmZhzd5Zx+aG+PkNCi4EYoyxVlBgq4nkEZm9T
         d0CvmFrPhGJHELV2YCS5DHAZu1Ab80+B4CWv3mKZuUG/wQoIz09WNrBP/HUuNtkKdf8s
         BYT5CP/RU9elfhQDEBW6ov1VaqQMNgCdGnsh2TaQG91ZO6y/nDUOYJJq3Q7JTGwBp7X4
         zbiDgIVT/pB5LCjHsMtTeOoEozBDvAXsaXcie87z+wYLaa4ZXc/Orog2nDxLORwsy4F+
         g86Q==
X-Gm-Message-State: AOAM53178bYDJmQzvewRufKUY2XifSr8uQ1IwGg2UzuSE8UfZi9V24VX
	QlXIlHKkUUSjETERgLfoItjltJ2fJFa1Dg==
X-Google-Smtp-Source: ABdhPJwtuLnpa/+icsyqakDioUzcXtFi1z998oW4N/1k6zq5qBg75ioZqd9T0FC0ynDimZ2biRkQeQ==
X-Received: by 2002:a05:6512:2e8:: with SMTP id m8mr4278108lfq.22.1632401707216;
        Thu, 23 Sep 2021 05:55:07 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 03/11] vpci/header: Move register assignments from init_bars
Date: Thu, 23 Sep 2021 15:54:53 +0300
Message-Id: <20210923125501.234252-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vPCI register
handlers depending on the domain: hwdom or guest.
The need for this step is that it is easier to have all related functionality
put at one place. When the subsequent patches add decisions on which
handlers to install, e.g. hwdom or guest handlers, then this is easily
achievable.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - constify struct pci_dev where possible
 - extend patch description
---
 xen/drivers/vpci/header.c | 83 ++++++++++++++++++++++++++-------------
 1 file changed, 56 insertions(+), 27 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index f8cd55e7c024..3d571356397a 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -445,6 +445,55 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
+static int add_bar_handlers(const struct pci_dev *pdev)
+{
+    unsigned int i;
+    struct vpci_header *header = &pdev->vpci->header;
+    struct vpci_bar *bars = header->bars;
+    int rc;
+
+    /* Setup a handler for the command register. */
+    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
+                           2, header);
+    if ( rc )
+        return rc;
+
+    if ( pdev->ignore_bars )
+        return 0;
+
+    for ( i = 0; i < PCI_HEADER_NORMAL_NR_BARS + 1; i++ )
+    {
+        if ( (bars[i].type == VPCI_BAR_IO) || (bars[i].type == VPCI_BAR_EMPTY) )
+            continue;
+
+        if ( bars[i].type == VPCI_BAR_ROM )
+        {
+            unsigned int rom_reg;
+            uint8_t header_type = pci_conf_read8(pdev->sbdf,
+                                                 PCI_HEADER_TYPE) & 0x7f;
+            if ( header_type == PCI_HEADER_TYPE_NORMAL )
+                rom_reg = PCI_ROM_ADDRESS;
+            else
+                rom_reg = PCI_ROM_ADDRESS1;
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                   rom_reg, 4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+        else
+        {
+            uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
+
+            /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
+                                   4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+    }
+    return 0;
+}
+
 static int init_bars(struct pci_dev *pdev)
 {
     uint16_t cmd;
@@ -470,14 +519,8 @@ static int init_bars(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
-    if ( rc )
-        return rc;
-
     if ( pdev->ignore_bars )
-        return 0;
+        return add_bar_handlers(pdev);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -492,14 +535,6 @@ static int init_bars(struct pci_dev *pdev)
         if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO )
         {
             bars[i].type = VPCI_BAR_MEM64_HI;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
-            if ( rc )
-            {
-                pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-                return rc;
-            }
-
             continue;
         }
 
@@ -532,14 +567,6 @@ static int init_bars(struct pci_dev *pdev)
         bars[i].addr = addr;
         bars[i].size = size;
         bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH;
-
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4,
-                               &bars[i]);
-        if ( rc )
-        {
-            pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-            return rc;
-        }
     }
 
     /* Check expansion ROM. */
@@ -553,11 +580,13 @@ static int init_bars(struct pci_dev *pdev)
         rom->addr = addr;
         header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) &
                               PCI_ROM_ADDRESS_ENABLE;
+    }
 
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg,
-                               4, rom);
-        if ( rc )
-            rom->type = VPCI_BAR_EMPTY;
+    rc = add_bar_handlers(pdev);
+    if ( rc )
+    {
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+        return rc;
     }
 
     return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194130.345881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI3-0003Bi-Cn; Thu, 23 Sep 2021 12:57:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194130.345881; Thu, 23 Sep 2021 12:57:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI3-0003Al-5o; Thu, 23 Sep 2021 12:57:27 +0000
Received: by outflank-mailman (input) for mailman id 194130;
 Thu, 23 Sep 2021 12:57:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOGI-0004it-4Q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:38 +0000
Received: from mail-lf1-x12f.google.com (unknown [2a00:1450:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 50e1c962-e474-4a32-a272-e52fa77ddbba;
 Thu, 23 Sep 2021 12:54:53 +0000 (UTC)
Received: by mail-lf1-x12f.google.com with SMTP id i4so26248019lfv.4
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:53 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50e1c962-e474-4a32-a272-e52fa77ddbba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=1/Av7G6GeJczbDfxqzlnIjolaRGsuAU++UwoP8I05TI=;
        b=RwhaNlcv18og0qKohd2m91E9PVXuI6XFKWNx/rnHegXZWaORWUqBS/43wI5i8bRO1k
         ypQSFQwAXMm+HQKJTqyn+7EPdAPy1W1/ZQ0pgTetQIzKoxJa1esuxYlR5qwBvPT4qNK0
         Wvinvok3mchwEGtguo+OSdJmUO43zcpu2KTsOGnn5V1mL/qhLR9n9vHBE1Dsz95yj5Zh
         KOOmgnr3cCMRy0ITTy4QTb8k66psaD8X5bJKcdTtXQ2R+85BZ6kFEYArZviQvWJ7LKmI
         VKVCQOaLndulJqSCN2iyIDpRc6fg3aJZ3K16iql2qxe/SVca6J12RejijfWtPLl1cvMJ
         BpTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=1/Av7G6GeJczbDfxqzlnIjolaRGsuAU++UwoP8I05TI=;
        b=6huGpiWKNfwGQFcNpjTiYXM9teRpP7plnf+EIoTnu0/wIJP9teezfKQ9p2GSE9mk+d
         L1RQD4ogIfJRALcf3/yXtRiK3jzTz0SU+MXFmSPcFBbtxPa20n8wAJ309qBahM1c9Keu
         GpYFR6H3+X0R/NdYf+VqCSDFMfFJ9U7/rUZm9L/qLIrFrtf8J2Bg3q+n+3ud5OOwIRwZ
         t1TqCFMRTI0Ecd6RFHru7bnv1jfVZjL9Xy6AWMMZROnc/nG8oZEKIhcrzky7qd3wYex9
         SygIQYTAeRiHQEiP8y2MSA9bUJmYgNLPsShpIBO5/RR/V6twGeL4W+v26wzHHpvH6ROK
         qUog==
X-Gm-Message-State: AOAM531A5CSPjdvo6MMXIOamnrEmJMlfjpd5eX/9BYzPKIIh2yEjj1zN
	N2CB27J8nmMmrL6MN9dr32mlvNnZEt7/6A==
X-Google-Smtp-Source: ABdhPJzx6pBtO2ant5XNGCw2LSXesDoDIg25KnxrYda03oE7UEIslZOTnYnpYVvYtxr9mbJCfYR3Kg==
X-Received: by 2002:a05:6512:3a89:: with SMTP id q9mr3902643lfu.515.1632401692595;
        Thu, 23 Sep 2021 05:54:52 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to Domain-0's p2m
Date: Thu, 23 Sep 2021 15:54:37 +0300
Message-Id: <20210923125438.234162-11-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

PCI host bridges are special devices in terms of implementing PCI
passthrough. According to [1] the current implementation depends on
Domain-0 to perform the initialization of the relevant PCI host
bridge hardware and perform PCI device enumeration. In order to
achieve that one of the required changes is to not map all the memory
ranges in map_range_to_domain as we traverse the device tree on startup
and perform some additional checks if the range needs to be mapped to
Domain-0.

The generic PCI host controller device tree binding says [2]:
- ranges: As described in IEEE Std 1275-1994, but must provide
          at least a definition of non-prefetchable memory. One
          or both of prefetchable Memory and IO Space may also
          be provided.

- reg   : The Configuration Space base address and size, as accessed
          from the parent bus.  The base address corresponds to
          the first bus in the "bus-range" property.  If no
          "bus-range" is specified, this will be bus 0 (the default).

>From the above none of the memory ranges from the "ranges" property
needs to be mapped to Domain-0 at startup as MMIO mapping is going to
be handled dynamically by vPCI as we assign PCI devices, e.g. each
device assigned to Domain-0/guest will have its MMIOs mapped/unmapped
as needed by Xen.

The "reg" property covers not only ECAM space, but may also have other
then the configuration memory ranges described, for example [3]:
- reg: Should contain rc_dbi, config registers location and length.
- reg-names: Must include the following entries:
   "rc_dbi": controller configuration registers;
   "config": PCIe configuration space registers.

This patch makes it possible to not map all the ranges from the
"ranges" property and also ECAM from the "reg". All the rest from the
"reg" property still needs to be mapped to Domain-0, so the PCI
host bridge remains functional in Domain-0.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00777.html
[2] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt
[3] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - Added better description of why and what needs to be mapped into
   Domain-0's p2m and what doesn't
 - Do not do any mappings for PCI devices while traversing the DT
 - Walk all the bridges and make required mappings in one go
---
 xen/arch/arm/domain_build.c        | 38 +++++++++++++++--------
 xen/arch/arm/pci/ecam.c            | 14 +++++++++
 xen/arch/arm/pci/pci-host-common.c | 48 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
 xen/include/asm-arm/pci.h          |  9 ++++++
 xen/include/asm-arm/setup.h        | 13 ++++++++
 6 files changed, 111 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 83ab0d52cce9..e72c1b881cae 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -10,7 +10,6 @@
 #include <asm/regs.h>
 #include <xen/errno.h>
 #include <xen/err.h>
-#include <xen/device_tree.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/guest_access.h>
 #include <xen/iocap.h>
@@ -47,12 +46,6 @@ static int __init parse_dom0_mem(const char *s)
 }
 custom_param("dom0_mem", parse_dom0_mem);
 
-struct map_range_data
-{
-    struct domain *d;
-    p2m_type_t p2mt;
-};
-
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
@@ -1388,9 +1381,8 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
     return 0;
 }
 
-static int __init map_range_to_domain(const struct dt_device_node *dev,
-                                      u64 addr, u64 len,
-                                      void *data)
+int __init map_range_to_domain(const struct dt_device_node *dev,
+                               u64 addr, u64 len, void *data)
 {
     struct map_range_data *mr_data = data;
     struct domain *d = mr_data->d;
@@ -1417,6 +1409,13 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
         }
     }
 
+#ifdef CONFIG_HAS_PCI
+    if ( is_pci_passthrough_enabled() &&
+         (device_get_class(dev) == DEVICE_PCI) &&
+         !mr_data->map_pci_bridge )
+        need_mapping = false;
+#endif
+
     if ( need_mapping )
     {
         res = map_regions_p2mt(d,
@@ -1450,7 +1449,11 @@ static int __init map_device_children(struct domain *d,
                                       const struct dt_device_node *dev,
                                       p2m_type_t p2mt)
 {
-    struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
+    struct map_range_data mr_data = {
+        .d = d,
+        .p2mt = p2mt,
+        .map_pci_bridge = false
+    };
     int ret;
 
     if ( dt_device_type_is_equal(dev, "pci") )
@@ -1582,7 +1585,11 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     /* Give permission and map MMIOs */
     for ( i = 0; i < naddr; i++ )
     {
-        struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
+        struct map_range_data mr_data = {
+            .d = d,
+            .p2mt = p2mt,
+            .map_pci_bridge = false
+        };
         res = dt_device_get_address(dev, i, &addr, &size);
         if ( res )
         {
@@ -2754,7 +2761,14 @@ static int __init construct_dom0(struct domain *d)
         return rc;
 
     if ( acpi_disabled )
+    {
         rc = prepare_dtb_hwdom(d, &kinfo);
+#ifdef CONFIG_HAS_PCI
+        if ( rc < 0 )
+            return rc;
+        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
+#endif
+    }
     else
         rc = prepare_acpi(d, &kinfo);
 
diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
index 9b88b1cedaa2..eae177f2cbc2 100644
--- a/xen/arch/arm/pci/ecam.c
+++ b/xen/arch/arm/pci/ecam.c
@@ -39,6 +39,19 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
     return base + (PCI_DEVFN2(sbdf) << devfn_shift) + where;
 }
 
+bool pci_ecam_need_p2m_mapping(struct domain *d,
+                               struct pci_host_bridge *bridge,
+                               uint64_t addr)
+{
+    struct pci_config_window *cfg = bridge->cfg;
+
+    /*
+     * We do not want ECAM address space to be mapped in Domain-0's p2m,
+     * so we can trap access to it.
+     */
+    return cfg->phys_addr != addr;
+}
+
 /* ECAM ops */
 const struct pci_ecam_ops pci_generic_ecam_ops = {
     .bus_shift  = 20,
@@ -46,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
         .map_bus                = pci_ecam_map_bus,
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
+        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
     }
 };
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 155f2a2743af..f350826ea26b 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -18,6 +18,7 @@
 
 #include <xen/init.h>
 #include <xen/pci.h>
+#include <asm/setup.h>
 #include <xen/rwlock.h>
 #include <xen/sched.h>
 #include <xen/vmap.h>
@@ -328,6 +329,53 @@ int pci_host_get_num_bridges(void)
     return count;
 }
 
+int __init pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt)
+{
+    struct pci_host_bridge *bridge;
+    struct map_range_data mr_data = {
+        .d = d,
+        .p2mt = p2mt,
+        .map_pci_bridge = true
+    };
+
+    /*
+     * For each PCI host bridge we need to only map those ranges
+     * which are used by Domain-0 to properly initialize the bridge,
+     * e.g. we do not want to map ECAM configuration space which lives in
+     * "reg" or "assigned-addresses" device tree property.
+     * Neither we want to map any of the MMIO ranges found in the "ranges"
+     * device tree property.
+     */
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        const struct dt_device_node *dev = bridge->dt_node;
+        int i;
+
+        for ( i = 0; i < dt_number_of_address(dev); i++ )
+        {
+            uint64_t addr, size;
+            int err;
+
+            err = dt_device_get_address(dev, i, &addr, &size);
+            if ( err )
+            {
+                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                       i, dt_node_full_name(dev));
+                return err;
+            }
+
+            if ( bridge->ops->need_p2m_mapping(d, bridge, addr) )
+            {
+                err = map_range_to_domain(dev, addr, size, &mr_data);
+                if ( err )
+                    return err;
+            }
+        }
+    }
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
index c27b4ea9f02f..adbe3627871f 100644
--- a/xen/arch/arm/pci/pci-host-zynqmp.c
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -33,6 +33,7 @@ const struct pci_ecam_ops nwl_pcie_ops = {
         .map_bus                = pci_ecam_map_bus,
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
+        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
     }
 };
 
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 7618f0b6725b..b81f66e813ef 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -19,6 +19,8 @@
 
 #ifdef CONFIG_HAS_PCI
 
+#include <asm/p2m.h>
+
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
 #define PRI_pci "%04x:%02x:%02x.%u"
 
@@ -79,6 +81,9 @@ struct pci_ops {
                 uint32_t reg, uint32_t len, uint32_t *value);
     int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
                  uint32_t reg, uint32_t len, uint32_t value);
+    bool (*need_p2m_mapping)(struct domain *d,
+                             struct pci_host_bridge *bridge,
+                             uint64_t addr);
 };
 
 /*
@@ -102,6 +107,9 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
                             uint32_t reg, uint32_t len, uint32_t value);
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                uint32_t sbdf, uint32_t where);
+bool pci_ecam_need_p2m_mapping(struct domain *d,
+                               struct pci_host_bridge *bridge,
+                               uint64_t addr);
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
@@ -116,6 +124,7 @@ int pci_host_iterate_bridges(struct domain *d,
                              int (*clb)(struct domain *d,
                                         struct pci_host_bridge *bridge));
 int pci_host_get_num_bridges(void);
+int pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt);
 #else   /*!CONFIG_HAS_PCI*/
 
 #define pci_passthrough_enabled (false)
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 95da0b7ab9cd..21863dd2bc58 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -2,6 +2,8 @@
 #define __ARM_SETUP_H_
 
 #include <public/version.h>
+#include <asm/p2m.h>
+#include <xen/device_tree.h>
 
 #define MIN_FDT_ALIGN 8
 #define MAX_FDT_SIZE SZ_2M
@@ -77,6 +79,14 @@ struct bootinfo {
 #endif
 };
 
+struct map_range_data
+{
+    struct domain *d;
+    p2m_type_t p2mt;
+    /* Set if mappings for PCI host bridges must not be skipped. */
+    bool map_pci_bridge;
+};
+
 extern struct bootinfo bootinfo;
 
 extern domid_t max_init_domid;
@@ -124,6 +134,9 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
 u32 device_tree_get_u32(const void *fdt, int node,
                         const char *prop_name, u32 dflt);
 
+int map_range_to_domain(const struct dt_device_node *dev,
+                        u64 addr, u64 len, void *data);
+
 #endif
 /*
  * Local variables:
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194134.345895 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI4-0003he-SI; Thu, 23 Sep 2021 12:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194134.345895; Thu, 23 Sep 2021 12:57:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOI4-0003g3-Nj; Thu, 23 Sep 2021 12:57:28 +0000
Received: by outflank-mailman (input) for mailman id 194134;
 Thu, 23 Sep 2021 12:57:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHa-0004it-6y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:58 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 26526be8-ef0e-4cb2-b1b7-0fa4db338ccd;
 Thu, 23 Sep 2021 12:55:13 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id y28so26182005lfb.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:13 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26526be8-ef0e-4cb2-b1b7-0fa4db338ccd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=omkszF+qjH4yhR2r1KBhAVTg1e2rgb4747WsTYEwl9M=;
        b=Diu59nZbLtVeysxoliiZuuEQkQgFilWmqQls8NsOhdkmZ5wxSw/zo1goR/KwhzT/xi
         Lt9GtjdUG3sKql9xvyV4XvyruHnLRqrfEuzQp6dU2uZ4VkJbmbkNJpih1Dxvq/u6PpQM
         iVnsRf+UWQAcCMeTMPTztOWeW38uk7TY0W0uhva/PJyLmo/PClGeXFc4w+W8ehVd7X0T
         qvaaE3u7GlSml5X/idcIN0e5cB3GvkHhkB8BzlnWuyvNg7WYz7Ldz1JTPM3cXD9EYmk2
         ZpQoQCm6hW2ptx6ahvUKLP2F+S2bhg30587oxYZ+a0qb3Wl0uuARXEpBvVujgCQyQBXm
         +rjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=omkszF+qjH4yhR2r1KBhAVTg1e2rgb4747WsTYEwl9M=;
        b=qX6lfOxIjirjQ0SNreYBcXKqyCAURSwEpM1nbqFtcdgqSGHLlb4dsQfRg5YBggKdUe
         4ASecVe7YiTt5gt3SJFI6ppG/EdN/30zOhHIGIv1xnluT5n0shUZ9RkgmVJfoIniM8ri
         Fyz85beMT9udHcQO6J9KMMpz9fcgl8jeQMppAiBnR4s718KmZg6YXbkFa2IKXuV8Nzkp
         88TYh1yYoVVNCQE6onSkEq8ouUdtYxumjYrGwdiowFxLk+rbSjZn3lgiL1OBV2kZhlkf
         VpIWwFIHDRSUwsvx5i1LADwG6IDs5d6Bj/v9txMRdLv08AlXrFoUTHQHPRVJow1IkRFZ
         A9NA==
X-Gm-Message-State: AOAM533M+wKw+RIwrjlac3iyi2yi0A/GX2b5DGh/aOoZeUKjaOX4S4WB
	zu9/4ruYDoWXG2gFMFgujYFJIoPyGPlt2g==
X-Google-Smtp-Source: ABdhPJzXcmkYQUmHLW0CXz3pHHxFBTBXyogaOKmeC1zkP4o3nmiDDwSXr7uy1B0TdVvjqkto2D8sUA==
X-Received: by 2002:a05:6512:6cb:: with SMTP id u11mr4034394lff.517.1632401710794;
        Thu, 23 Sep 2021 05:55:10 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 06/11] vpci/header: Handle p2m range sets per BAR
Date: Thu, 23 Sep 2021 15:54:56 +0300
Message-Id: <20210923125501.234252-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Instead of handling a single range set, that contains all the memory
regions of all the BARs and ROM, have them per BAR.

This is in preparation of making non-identity mappings in p2m for the
MMIOs/ROM.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 172 ++++++++++++++++++++++++++------------
 xen/include/xen/vpci.h    |   3 +-
 2 files changed, 122 insertions(+), 53 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index ec4d215f36ff..9c603d26d302 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -131,49 +131,75 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
 
 bool vpci_process_pending(struct vcpu *v)
 {
-    if ( v->vpci.mem )
+    if ( v->vpci.num_mem_ranges )
     {
         struct map_data data = {
             .d = v->domain,
             .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
         };
-        int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data);
+        struct pci_dev *pdev = v->vpci.pdev;
+        struct vpci_header *header = &pdev->vpci->header;
+        unsigned int i;
 
-        if ( rc == -ERESTART )
-            return true;
+        for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+        {
+            struct vpci_bar *bar = &header->bars[i];
+            int rc;
 
-        spin_lock(&v->vpci.pdev->vpci->lock);
-        /* Disable memory decoding unconditionally on failure. */
-        modify_decoding(v->vpci.pdev,
-                        rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
-                        !rc && v->vpci.rom_only);
-        spin_unlock(&v->vpci.pdev->vpci->lock);
+            if ( !bar->mem )
+                continue;
 
-        rangeset_destroy(v->vpci.mem);
-        v->vpci.mem = NULL;
-        if ( rc )
-            /*
-             * FIXME: in case of failure remove the device from the domain.
-             * Note that there might still be leftover mappings. While this is
-             * safe for Dom0, for DomUs the domain will likely need to be
-             * killed in order to avoid leaking stale p2m mappings on
-             * failure.
-             */
-            vpci_remove_device(v->vpci.pdev);
+            rc = rangeset_consume_ranges(bar->mem, map_range, &data);
+
+            if ( rc == -ERESTART )
+                return true;
+
+            spin_lock(&pdev->vpci->lock);
+            /* Disable memory decoding unconditionally on failure. */
+            modify_decoding(pdev,
+                            rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
+                            !rc && v->vpci.rom_only);
+            spin_unlock(&pdev->vpci->lock);
+
+            rangeset_destroy(bar->mem);
+            bar->mem = NULL;
+            v->vpci.num_mem_ranges--;
+            if ( rc )
+                /*
+                 * FIXME: in case of failure remove the device from the domain.
+                 * Note that there might still be leftover mappings. While this is
+                 * safe for Dom0, for DomUs the domain will likely need to be
+                 * killed in order to avoid leaking stale p2m mappings on
+                 * failure.
+                 */
+                vpci_remove_device(pdev);
+        }
     }
 
     return false;
 }
 
 static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
-                            struct rangeset *mem, uint16_t cmd)
+                            uint16_t cmd)
 {
     struct map_data data = { .d = d, .map = true };
-    int rc;
+    struct vpci_header *header = &pdev->vpci->header;
+    int rc = 0;
+    unsigned int i;
+
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
 
-    while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
-        process_pending_softirqs();
-    rangeset_destroy(mem);
+        if ( !bar->mem )
+            continue;
+
+        while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
+                                              &data)) == -ERESTART )
+            process_pending_softirqs();
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
     if ( !rc )
         modify_decoding(pdev, cmd, false);
 
@@ -181,7 +207,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
 }
 
 static void defer_map(struct domain *d, struct pci_dev *pdev,
-                      struct rangeset *mem, uint16_t cmd, bool rom_only)
+                      uint16_t cmd, bool rom_only, uint8_t num_mem_ranges)
 {
     struct vcpu *curr = current;
 
@@ -192,9 +218,9 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
      * started for the same device if the domain is not well-behaved.
      */
     curr->vpci.pdev = pdev;
-    curr->vpci.mem = mem;
     curr->vpci.cmd = cmd;
     curr->vpci.rom_only = rom_only;
+    curr->vpci.num_mem_ranges = num_mem_ranges;
     /*
      * Raise a scheduler softirq in order to prevent the guest from resuming
      * execution with pending mapping operations, to trigger the invocation
@@ -206,42 +232,47 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
 static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
 {
     struct vpci_header *header = &pdev->vpci->header;
-    struct rangeset *mem = rangeset_new(NULL, NULL, 0);
     struct pci_dev *tmp, *dev = NULL;
     const struct vpci_msix *msix = pdev->vpci->msix;
-    unsigned int i;
+    unsigned int i, j;
     int rc;
-
-    if ( !mem )
-        return -ENOMEM;
+    uint8_t num_mem_ranges;
 
     /*
-     * Create a rangeset that represents the current device BARs memory region
+     * Create a rangeset per BAR that represents the current device memory region
      * and compare it against all the currently active BAR memory regions. If
      * an overlap is found, subtract it from the region to be mapped/unmapped.
      *
-     * First fill the rangeset with all the BARs of this device or with the ROM
+     * First fill the rangesets with all the BARs of this device or with the ROM
      * BAR only, depending on whether the guest is toggling the memory decode
      * bit of the command register, or the enable bit of the ROM BAR register.
      */
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
-        const struct vpci_bar *bar = &header->bars[i];
+        struct vpci_bar *bar = &header->bars[i];
         unsigned long start = PFN_DOWN(bar->addr);
         unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
+        bar->mem = NULL;
+
         if ( !MAPPABLE_BAR(bar) ||
              (rom_only ? bar->type != VPCI_BAR_ROM
                        : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) )
             continue;
 
-        rc = rangeset_add_range(mem, start, end);
+        bar->mem = rangeset_new(NULL, NULL, 0);
+        if ( !bar->mem )
+        {
+            rc = -ENOMEM;
+            goto fail;
+        }
+
+        rc = rangeset_add_range(bar->mem, start, end);
         if ( rc )
         {
             printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n",
                    start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            goto fail;
         }
     }
 
@@ -252,14 +283,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
         unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) +
                                      vmsix_table_size(pdev->vpci, i) - 1);
 
-        rc = rangeset_remove_range(mem, start, end);
-        if ( rc )
+        for ( j = 0; j < ARRAY_SIZE(header->bars); j++ )
         {
-            printk(XENLOG_G_WARNING
-                   "Failed to remove MSIX table [%lx, %lx]: %d\n",
-                   start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            const struct vpci_bar *bar = &header->bars[j];
+
+            if ( !bar->mem )
+                continue;
+
+            rc = rangeset_remove_range(bar->mem, start, end);
+            if ( rc )
+            {
+                printk(XENLOG_G_WARNING
+                       "Failed to remove MSIX table [%lx, %lx]: %d\n",
+                       start, end, rc);
+                goto fail;
+            }
         }
     }
 
@@ -291,7 +329,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             unsigned long start = PFN_DOWN(bar->addr);
             unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
-            if ( !bar->enabled || !rangeset_overlaps_range(mem, start, end) ||
+            if ( !bar->enabled ||
+                 !rangeset_overlaps_range(bar->mem, start, end) ||
                  /*
                   * If only the ROM enable bit is toggled check against other
                   * BARs in the same device for overlaps, but not against the
@@ -300,13 +339,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
                  (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) )
                 continue;
 
-            rc = rangeset_remove_range(mem, start, end);
+            rc = rangeset_remove_range(bar->mem, start, end);
             if ( rc )
             {
                 printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n",
                        start, end, rc);
-                rangeset_destroy(mem);
-                return rc;
+                goto fail;
             }
         }
     }
@@ -324,12 +362,42 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
          * will always be to establish mappings and process all the BARs.
          */
         ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only);
-        return apply_map(pdev->domain, pdev, mem, cmd);
+        return apply_map(pdev->domain, pdev, cmd);
     }
 
-    defer_map(dev->domain, dev, mem, cmd, rom_only);
+    /* Find out how many memory ranges has left after MSI and overlaps. */
+    num_mem_ranges = 0;
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        if ( !rangeset_is_empty(bar->mem) )
+            num_mem_ranges++;
+    }
+
+    /*
+     * There are cases when PCI device, root port for example, has neither
+     * memory space nor IO. In this case PCI command register write is
+     * missed resulting in the underlying PCI device not functional, so:
+     *   - if there are no regions write the command register now
+     *   - if there are regions then defer work and write later on
+     */
+    if ( !num_mem_ranges )
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+    else
+        defer_map(dev->domain, dev, cmd, rom_only, num_mem_ranges);
 
     return 0;
+
+fail:
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
+    return rc;
 }
 
 static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 9eaf99f356fe..3696c73a4237 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -86,6 +86,7 @@ struct vpci {
             /* Guest view of the BAR. */
             uint64_t guest_addr;
             uint64_t size;
+            struct rangeset *mem;
             enum {
                 VPCI_BAR_EMPTY,
                 VPCI_BAR_IO,
@@ -160,9 +161,9 @@ struct vpci {
 
 struct vpci_vcpu {
     /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */
-    struct rangeset *mem;
     struct pci_dev *pdev;
     uint16_t cmd;
+    uint8_t num_mem_ranges;
     bool rom_only : 1;
 };
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194139.345909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIH-0004ea-7T; Thu, 23 Sep 2021 12:57:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194139.345909; Thu, 23 Sep 2021 12:57:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIH-0004eP-3r; Thu, 23 Sep 2021 12:57:41 +0000
Received: by outflank-mailman (input) for mailman id 194139;
 Thu, 23 Sep 2021 12:57:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOGS-0004it-4i
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:48 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bf44b32d-d9fe-4f80-a0a0-a5c01915bd04;
 Thu, 23 Sep 2021 12:54:54 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id i25so26492013lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:54 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf44b32d-d9fe-4f80-a0a0-a5c01915bd04
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=HNTngv5ONOEO5gpzLyGmdvTGyFjpzr3RFglNf5/7Wm8=;
        b=MQjwcQdvOef1lREXofpeWL8/hQj819MvOexAPoO/+yK43KhOF8+lTN/74YVCe2+PZX
         cLlHsy60bn2dnjMUUiJeS54azNeSD/fAJA1zzqv/xntDTbsGFBNzFU/Rrg1paOYaT5IU
         NMuYaT3azDD9cH9ObvJXcliJQAbvqMHPbaKj/qY8c+KDyIvbDpz2QBwnvpWTbIxWNVwW
         RQltD2fhQ3TObamBbgOjbByRtpB1fZitsEbEPyIz3icTBeF3TUi2pSGhE7+5kiwUktKi
         MGouv5VkY3Db/bu2cOZTzdogcoa8ane8TMtqA7scSmEN6UqIwQ6RLMzxIqE7QgXSEyHf
         q1pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=HNTngv5ONOEO5gpzLyGmdvTGyFjpzr3RFglNf5/7Wm8=;
        b=qWOewXG0PCc3ox5YtxTTPUHkZ69FuexJaH9vAmMgrpabceVMD5fOnZCThfzj86eKaY
         1vJiIqE5UvbVBMJnVZQMF6kC/uDQ6Hca8BgC+OXs4MoLwfiEclLd1GoO/R9tKU6lGtCB
         yfTLjtIsLxFBLBIZKoG0zqk3+lhtwSd3Rj7CUcFUielj7n/mRDyA474OL/JUyiX/2Tb8
         qcBtlvmYm74m3FUv9yRKsWlJx5ZubVWuljk2I9DX6b+U+I0s9dgwhgYtNjKsZnfXdkvb
         ZtYr5h/qGrXOVrOuc8O0MrOvD8sLzDhBoDLmHSudQpH6kvPvl4RvV6uPlRFmaJODEFgt
         pe8g==
X-Gm-Message-State: AOAM530Hll+bwNI111fplKR7l43D82E+BLlruONj6LPsZkvZEDExGq7Z
	2O6m3O7Br0LWTw/MKqPMVffd6wLqcdQ5uw==
X-Google-Smtp-Source: ABdhPJycKsS0ZD76qSoGLuJUcMkJfAL1kNCC9tMBeWWfrb3lwIIN2AJTC93++bij/sHp820gH1XtOg==
X-Received: by 2002:ac2:442f:: with SMTP id w15mr3929533lfl.491.1632401693567;
        Thu, 23 Sep 2021 05:54:53 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 11/11] xen/arm: Process pending vPCI map/unmap operations
Date: Thu, 23 Sep 2021 15:54:38 +0300
Message-Id: <20210923125438.234162-12-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

vPCI may map and unmap PCI device memory (BARs) being passed through which
may take a lot of time. For this those operations may be deferred to be
performed later, so that they can be safely preempted.
Run the corresponding vPCI code while switching a vCPU.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - Moved the check for pending vpci work from the common IOREQ code
   to hvm_do_resume on x86
 - Re-worked the code for Arm to ensure we don't miss pending vPCI work
---
 xen/arch/arm/traps.c   | 13 +++++++++++++
 xen/arch/x86/hvm/hvm.c |  6 ++++++
 xen/common/ioreq.c     |  9 ---------
 3 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 219ab3c3fbde..b246f51086e3 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -34,6 +34,7 @@
 #include <xen/symbols.h>
 #include <xen/version.h>
 #include <xen/virtual_region.h>
+#include <xen/vpci.h>
 
 #include <public/sched.h>
 #include <public/xen.h>
@@ -2304,6 +2305,18 @@ static bool check_for_vcpu_work(void)
     }
 #endif
 
+    if ( has_vpci(v->domain) )
+    {
+        bool pending;
+
+        local_irq_enable();
+        pending = vpci_process_pending(v);
+        local_irq_disable();
+
+        if ( pending )
+            return true;
+    }
+
     if ( likely(!v->arch.need_flush_to_ram) )
         return false;
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7b48a1b925bb..d32f5d572941 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -549,6 +549,12 @@ void hvm_do_resume(struct vcpu *v)
     if ( !vcpu_ioreq_handle_completion(v) )
         return;
 
+    if ( has_vpci(v->domain) && vpci_process_pending(v) )
+    {
+        raise_softirq(SCHEDULE_SOFTIRQ);
+        return;
+    }
+
     if ( unlikely(v->arch.vm_event) )
         hvm_vm_event_do_resume(v);
 
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index d732dc045df9..689d256544c8 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -25,9 +25,7 @@
 #include <xen/lib.h>
 #include <xen/paging.h>
 #include <xen/sched.h>
-#include <xen/softirq.h>
 #include <xen/trace.h>
-#include <xen/vpci.h>
 
 #include <asm/guest_atomics.h>
 #include <asm/ioreq.h>
@@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
 
 bool vcpu_ioreq_handle_completion(struct vcpu *v)
 {
-    struct domain *d = v->domain;
     struct vcpu_io *vio = &v->io;
     struct ioreq_server *s;
     struct ioreq_vcpu *sv;
     enum vio_completion completion;
     bool res = true;
 
-    if ( has_vpci(d) && vpci_process_pending(v) )
-    {
-        raise_softirq(SCHEDULE_SOFTIRQ);
-        return false;
-    }
-
     while ( (sv = get_pending_vcpu(v, &s)) != NULL )
         if ( !wait_for_io(sv, get_ioreq(s, v)) )
             return false;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194143.345919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOII-00050m-Iy; Thu, 23 Sep 2021 12:57:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194143.345919; Thu, 23 Sep 2021 12:57:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOII-000503-EB; Thu, 23 Sep 2021 12:57:42 +0000
Received: by outflank-mailman (input) for mailman id 194143;
 Thu, 23 Sep 2021 12:57:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOGD-0004it-4L
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:55:33 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b31eb32d-d0f7-4a3d-99f4-de4ecf029500;
 Thu, 23 Sep 2021 12:54:54 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id u18so25607498lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:54:54 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id l7sm453584lfk.52.2021.09.23.05.54.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:54:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b31eb32d-d0f7-4a3d-99f4-de4ecf029500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=7YmzvU66v4oKyYcFR8Qcx6xXzeJU6lQheixQJ2KXwag=;
        b=UfedQtUeDFQTn+SL64ems4Rn0J9SCZv6TQxUejF3126H4dXW34Fw4o335M+EgmuAVd
         Q+vcH55CqHDM1pXmnjXaPaOVe1VITRUSu5KXOh04yowdL3yGAtoN+5EaxqzzaU6qFi/w
         saapwqv3p3gpPNF+fCgc17fZVTb29xUu2viB7O22AyvNkzeroO4y1c/DDEzVJY32qKv5
         Fcwqf7Upq2oQFzm0tI0xEkjpjRRGSRVhZA2zwNFjvYuVwJwNZQsTsusoXy1tgf0JV05F
         IUgAvoCTNPEoVXx3FTZSPXP2TFvw6GhfjmySjOo5Y1UpfNnXInYqc8q7/HMGpyT+EPfg
         +JVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=7YmzvU66v4oKyYcFR8Qcx6xXzeJU6lQheixQJ2KXwag=;
        b=59WPZJewSNCCV6sZRSRIj5ycvpfiopsanY2e+JqnDUT7Xfw9cw6b8cX9IIAPo7jb9N
         mpCWc3rjl3lQAwK2PKes8uxQlgdb+P5azrLjgrivgKY9H3QQiR+v1lGFaD7pDxju8zad
         C/54hwEOJTmKDZckEk7s2v48iy3hdZQ2killPSK3VcNu/NhS4KGt8rRLDfEYgtwyXHEY
         g4JzNEMrRNiUoOqoMPeOz4SmuAF3GUMyckRqGFNeYvbA+H3ssyoR9T7/Xe5InvTm+swr
         OirMD/Td1rI+vh00LFZsvi/5UfgsbNu/cKD41PF/j/p/8J7AMGsZ0orQloffAXWYVRq0
         h9sw==
X-Gm-Message-State: AOAM531r2mCtl9ogTa74P9hp8fF74QdUf+aw6IcYfmI9pVfQHNazlN6Q
	s/cY2ptm0CIFVlY9cbzggt8NXD+8mRUM9Q==
X-Google-Smtp-Source: ABdhPJwQFyQoC6ZyqfOZz0jz5BMc2q86hNOrzHCE+sQ7nVhOg/JEVk7PIv0iNGvCwLta+XxiRKvQlw==
X-Received: by 2002:a2e:7f17:: with SMTP id a23mr5074537ljd.133.1632401690450;
        Thu, 23 Sep 2021 05:54:50 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 08/11] libxl: Only map legacy PCI IRQs if they are supported
Date: Thu, 23 Sep 2021 15:54:35 +0300
Message-Id: <20210923125438.234162-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125438.234162-1-andr2000@gmail.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Arm's PCI passthrough implementation doesn't support legacy interrupts,
but MSI/MSI-X. This can be the case for other platforms too.
For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
it to the CFLAGS and compile the relevant code in the toolstack only if
applicable.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>

---
Since v1:
 - Minimized #idefery by introducing pci_supp_legacy_irq function
   for relevant checks
---
 tools/libs/light/Makefile    |  4 ++++
 tools/libs/light/libxl_pci.c | 13 +++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 7d8c51d49242..bd3f6be2a183 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -46,6 +46,10 @@ CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
 	-Wno-declaration-after-statement -Wformat-nonliteral
 CFLAGS += -I.
 
+ifeq ($(CONFIG_X86),y)
+CFLAGS += -DCONFIG_PCI_SUPP_LEGACY_IRQ
+endif
+
 SRCS-$(CONFIG_X86) += libxl_cpuid.c
 SRCS-$(CONFIG_X86) += libxl_x86.c
 SRCS-$(CONFIG_X86) += libxl_psr.c
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 59f3686fc85e..4c2d7aeefbb2 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1364,6 +1364,15 @@ static void pci_add_timeout(libxl__egc *egc, libxl__ev_time *ev,
     pci_add_dm_done(egc, pas, rc);
 }
 
+static bool pci_supp_legacy_irq(void)
+{
+#ifdef CONFIG_PCI_SUPP_LEGACY_IRQ
+    return true;
+#else
+    return false;
+#endif
+}
+
 static void pci_add_dm_done(libxl__egc *egc,
                             pci_add_state *pas,
                             int rc)
@@ -1434,6 +1443,8 @@ static void pci_add_dm_done(libxl__egc *egc,
         }
     }
     fclose(f);
+    if (!pci_supp_legacy_irq())
+        goto out_no_irq;
     sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                 pci->bus, pci->dev, pci->func);
     f = fopen(sysfs_path, "r");
@@ -1983,6 +1994,8 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
         }
         fclose(f);
 skip1:
+        if (!pci_supp_legacy_irq())
+            goto skip_irq;
         sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                pci->bus, pci->dev, pci->func);
         f = fopen(sysfs_path, "r");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194144.345924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIJ-000572-3S; Thu, 23 Sep 2021 12:57:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194144.345924; Thu, 23 Sep 2021 12:57:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOII-000556-RB; Thu, 23 Sep 2021 12:57:42 +0000
Received: by outflank-mailman (input) for mailman id 194144;
 Thu, 23 Sep 2021 12:57:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOH1-0004it-6A
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:23 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 56e2b585-9c23-4014-bc02-da210cf5b44a;
 Thu, 23 Sep 2021 12:55:07 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id i4so26250580lfv.4
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:07 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56e2b585-9c23-4014-bc02-da210cf5b44a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=9Y+DU0DSrlSJm3RBHsKqMThf7inWBIT9pPD3eSNkVAE=;
        b=hYxKWCjYG91LckucdXz7kiTTL5AB09I7V3lbQ4uZfD/q/MYCdyjE2/BPyX2RGkGkdD
         LfzE8+Dq+LMIIayzkHPshcRPxLaXsNy1adj8t3KB0xIEB7dvCLotKdf5i3SbpzPtedxp
         izc40arDT+3hOiCrJkkto5MPJztcsDKNoskt7sClcJASUMvw7ijT/tgxdo2IeKV32+ku
         EHFUoUbQLhE8/6dSUTfwfUKIDtfWs3YfQIptdnsV2te7IokWTGwuBHgN30IWV3kbcbbJ
         TFLgalge/nTmIig4YTtpi5Be+40ZTTJZ/wbm4FINugBjRCfV+W/vTLDcKfhE0U7wNW5J
         Fgzw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=9Y+DU0DSrlSJm3RBHsKqMThf7inWBIT9pPD3eSNkVAE=;
        b=dFdbrSulI0lh5wU1CMbGz/KoUiC86HHuc+ShB7P56GBiwaswyR0ZcruFRXt/BpwetH
         C8p1PHajJMfNaOo9bk/j/dNdnhqm02/ZKtOd62EVNnquRynUg+gcveBtNk0ZnSIeAMZk
         ID3iR8JX3YBgejMyjzQrgtNOk8XIOWHxwe0d53hV+lkaFapkYSdmHHeXfL0f7spSEAnf
         BWlXe5VfABOGsDv2q85U2gdZQ06osozRUxYZ4aJr+dK6zhLAapyMPiaErro782SnbN9v
         DVnc9KUX3hW7ClwM40faGe4VsJxtIfs1pJsejST7xphlXXShzXRYv6PHopFqmO7/jl+n
         0FZw==
X-Gm-Message-State: AOAM532dvFyUfJbVzxEVfhnwPHJTapvH488VP/CqErlzTv0QDTk4EhEA
	1r+jhQx6h0IZxQ/+ra7zxoDgvKOhX5QErw==
X-Google-Smtp-Source: ABdhPJws+VUTSrWiIzEEm+gWdSYS1iA4cTrQf4OFeA2lEZRm4C2x+KWOsrumrELjauDBZ1A2tBy7QA==
X-Received: by 2002:ac2:5f83:: with SMTP id r3mr4179435lfe.686.1632401706071;
        Thu, 23 Sep 2021 05:55:06 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 02/11] vpci: Add hooks for PCI device assign/de-assign
Date: Thu, 23 Sep 2021 15:54:52 +0300
Message-Id: <20210923125501.234252-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

When a PCI device gets assigned/de-assigned some work on vPCI side needs
to be done for that device. Introduce a pair of hooks so vPCI can handle
that.

Please note, that in the current design the error path is handled by
the toolstack via XEN_DOMCTL_assign_device/XEN_DOMCTL_deassign_device,
so this is why it is acceptable not to de-assign devices if vPCI's
assign fails, e.g. the roll back will be handled on deassign_device when
it is called by the toolstack.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - extended the commit message
---
 xen/drivers/passthrough/pci.c |  9 +++++++++
 xen/drivers/vpci/vpci.c       | 21 +++++++++++++++++++++
 xen/include/xen/vpci.h        | 18 ++++++++++++++++++
 3 files changed, 48 insertions(+)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index fc3469bc12dc..e1da283d73ad 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -872,6 +872,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
     if ( ret )
         goto out;
 
+    ret = vpci_deassign_device(d, pdev);
+    if ( ret )
+        goto out;
+
     if ( pdev->domain == hardware_domain  )
         pdev->quarantine = false;
 
@@ -1431,6 +1435,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
         rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
     }
 
+    if ( rc )
+        goto done;
+
+    rc = vpci_assign_device(d, pdev);
+
  done:
     if ( rc )
         printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 1666402d55b8..a8fed3d2c42e 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -86,6 +86,27 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
 
     return rc;
 }
+
+/* Notify vPCI that device is assigned to guest. */
+int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
+{
+    /* It only makes sense to assign for hwdom or guest domain. */
+    if ( is_system_domain(d) || !has_vpci(d) )
+        return 0;
+
+    return 0;
+}
+
+/* Notify vPCI that device is de-assigned from guest. */
+int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
+{
+    /* It only makes sense to de-assign from hwdom or guest domain. */
+    if ( is_system_domain(d) || !has_vpci(d) )
+        return 0;
+
+    return 0;
+}
+
 #endif /* __XEN__ */
 
 static int vpci_register_cmp(const struct vpci_register *r1,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 2e910d0b1f90..b9485b2aea1b 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -26,6 +26,12 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 /* Add vPCI handlers to device. */
 int __must_check vpci_add_handlers(struct pci_dev *dev);
 
+/* Notify vPCI that device is assigned/de-assigned to/from guest. */
+int __must_check vpci_assign_device(struct domain *d,
+                                    const struct pci_dev *dev);
+int __must_check vpci_deassign_device(struct domain *d,
+                                      const struct pci_dev *dev);
+
 /* Remove all handlers and free vpci related structures. */
 void vpci_remove_device(struct pci_dev *pdev);
 /* Remove all handlers for the device given. */
@@ -220,6 +226,18 @@ static inline int vpci_add_handlers(struct pci_dev *pdev)
     return 0;
 }
 
+static inline int vpci_assign_device(struct domain *d,
+                                     const struct pci_dev *dev)
+{
+    return 0;
+};
+
+static inline int vpci_deassign_device(struct domain *d,
+                                       const struct pci_dev *dev)
+{
+    return 0;
+};
+
 static inline void vpci_dump_msi(void) { }
 
 static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194149.345942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIY-0006bt-M3; Thu, 23 Sep 2021 12:57:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194149.345942; Thu, 23 Sep 2021 12:57:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIY-0006bf-Hc; Thu, 23 Sep 2021 12:57:58 +0000
Received: by outflank-mailman (input) for mailman id 194149;
 Thu, 23 Sep 2021 12:57:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHQ-0004it-6b
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:48 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e7f5a879-b865-434e-b8f1-e906b434769c;
 Thu, 23 Sep 2021 12:55:10 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id g41so26148306lfv.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:10 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:08 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7f5a879-b865-434e-b8f1-e906b434769c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=cIPXz2mA5EJS9da7dICK94JpwcF7nJh6i2ljGMdw31k=;
        b=HeIp6l9OjTuGNCf2NE4aerZ8ypyKK4xUPQnFXvFUXP077VWYVQl01B1SqpfXR79jpp
         lumKKt3PtFC1SzIMYHOJ1Xg2K84l1AcTFIOxYWPqFqWJjzHGP9TzgWH0/yE4ulu9RSYJ
         DtMCkXFpokEB7iVvHSiyvLsZau03miegPDqQoXi+dbxgJAyzoJxd0pYptWTney0odSQR
         8f5mzPtX2eYXntjJtjpAMmS1a04xUPR9dNMOfpfvvRtPfzI6cFMmvzoeotsNMzl4sCbb
         TEyTdPUBXZLPe/XpOXC2Gjo/wkpueSH5afjhVEKZRVifB+EMxHeZ5bYVukc7cEAOJaAd
         kkzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=cIPXz2mA5EJS9da7dICK94JpwcF7nJh6i2ljGMdw31k=;
        b=MlSlWj8Qf8sXRqgSEoaa9ld7qLAzYnja+fM0PiQlP1/mUEgWv6gVeBqoMRZBYQVbKP
         WqVl9jT0F22Sqwj/1a+8CBW2BEi06XNyFTkMocSOnYPl2l93vys7HWgDO8BCzsDNqqkr
         0xasLNJ/LtrRLbtzKkRGiucNfui78K6Ny9p+NRXPXj7XPer5HUpaIQdNcXgszQBc3mry
         gF8kprtG4tLn3n8vVkB0+057G5QBhyM9BtVVoyMK1pTlLNzfHcqSB5QNaFIFWnNJs4y0
         hf7R5ch9YNKoKTuapIe7aVJ+7iw+ZbAW1SbGMcRYHhki79Be0ITrUSP6fNiHYWrLK+Sl
         /7Qg==
X-Gm-Message-State: AOAM532MICJdvEHOJNwSS0Xo+w9fgG2Ie0Ud83DnSMNQz9FSOmZD+rRw
	0yoaKXubjGG6oxkKJgH1p4kisz1JEnlYJQ==
X-Google-Smtp-Source: ABdhPJzNsCE1gRB70SpWfN2PFcPnO8jYHLcnJzCh/r0yGox61/c0i3oggOazkuEWskoC7iatqYj6qw==
X-Received: by 2002:a2e:1508:: with SMTP id s8mr5206646ljd.240.1632401709286;
        Thu, 23 Sep 2021 05:55:09 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 05/11] vpci/header: Implement guest BAR register handlers
Date: Thu, 23 Sep 2021 15:54:55 +0300
Message-Id: <20210923125501.234252-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Emulate guest BAR register values: this allows creating a guest view
of the registers and emulates size and properties probe as it is done
during PCI device enumeration by the guest.

ROM BAR is only handled for the hardware domain and for guest domains
there is a stub: at the moment PCI expansion ROM is x86 only, so it
might not be used by other architectures without emulating x86. Other
use-cases may include using that expansion ROM before Xen boots, hence
no emulation is needed in Xen itself. Or when a guest wants to use the
ROM code which seems to be rare.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - re-work guest read/write to be much simpler and do more work on write
   than read which is expected to be called more frequently
 - removed one too obvious comment

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 30 +++++++++++++++++++++++++++++-
 xen/include/xen/vpci.h    |  3 +++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 1ce98795fcca..ec4d215f36ff 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -400,12 +400,38 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
 static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
                             uint32_t val, void *data)
 {
+    struct vpci_bar *bar = data;
+    bool hi = false;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+    {
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        hi = true;
+    }
+    else
+    {
+        val &= PCI_BASE_ADDRESS_MEM_MASK;
+        val |= bar->type == VPCI_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32
+                                           : PCI_BASE_ADDRESS_MEM_TYPE_64;
+        val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
+    }
+
+    bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
+    bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
+
+    bar->guest_addr &= ~(bar->size - 1) | ~PCI_BASE_ADDRESS_MEM_MASK;
 }
 
 static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
                                void *data)
 {
-    return 0xffffffff;
+    const struct vpci_bar *bar = data;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+        return bar->guest_addr >> 32;
+
+    return bar->guest_addr;
 }
 
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
@@ -522,6 +548,8 @@ static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
             if ( rc )
                 return rc;
         }
+
+        bars[i].guest_addr = 0;
     }
     return 0;
 }
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 912cbc6aa7ad..9eaf99f356fe 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -81,7 +81,10 @@ struct vpci {
     struct vpci_header {
         /* Information about the PCI BARs of this device. */
         struct vpci_bar {
+            /* Physical view of the BAR. */
             uint64_t addr;
+            /* Guest view of the BAR. */
+            uint64_t guest_addr;
             uint64_t size;
             enum {
                 VPCI_BAR_EMPTY,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 12:57:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 12:57:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194150.345947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIZ-0006fo-2k; Thu, 23 Sep 2021 12:57:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194150.345947; Thu, 23 Sep 2021 12:57:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOIY-0006fC-RQ; Thu, 23 Sep 2021 12:57:58 +0000
Received: by outflank-mailman (input) for mailman id 194150;
 Thu, 23 Sep 2021 12:57:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOGr-0004it-5p
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:56:13 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b46d3d5b-4489-4e5a-941c-d26fc3fb88f9;
 Thu, 23 Sep 2021 12:55:06 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id p29so25910679lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:06 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b46d3d5b-4489-4e5a-941c-d26fc3fb88f9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=FKgxJK/5NyOFFpbY0xtHY00JQINZqmrpnwuzOqh/Ukk=;
        b=ea9E6tTANi/jveiexkEVCENbTHTScpXopjrod/3psE9GkasQWaKTmKmNiQdbaWuDC/
         yH7u43xdaWw6JGuAmS7VdZIbZs2VK2GJpvAX1gsdiwuVDd+TbekUS79BlpSs/bsPGdKQ
         wP7eOlSKkJfw0G+fTQSHuDwBcZKnZidY09Ob43s+XqlFLMymD65CN10T5mkq4cPwxDdv
         F82q1IJ6In4oWymTxGjH3GeMN5quVUG8HgV9f9dHpD81lh977Lm95mcWCzHZfGcvaIYS
         3ceMEDf10lcNmeqfesDK5HQo4ZBjVI5XVdBj4QNsDHEcVBEJwNF9cuMOkmMvRJYGDj2x
         XeDg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=FKgxJK/5NyOFFpbY0xtHY00JQINZqmrpnwuzOqh/Ukk=;
        b=tlGEHPq9WAcQiTQPWEZ4UJctEr7SyGlw8PBcgg3Bw5L0iBcvlrmrNhP6zXbibKbXss
         Qh6f3eAo2/hbg5xBc3cKX46ugR9thksH66fPINvqjpJk3pN88BoPDVhKWUwvsH7vrqlJ
         m9y9CjKNWXYOP+3yuuFgeeVYYmUiQdEEfyxy7nsbYK92i4JLDcoptmrjUTWc7fHbM5tW
         GAAKUuqMGkUZJbITl8UAAF8N9TK2bh4vFSZjfFU6tcZcN3HGWUDy9KcFuunExCOjPEoQ
         6LMcYgRKIzXUOGZVNUJdKi3VMQA8lq8ira9Fo7q2G5kR7tE43v+GrEok3VcFTA/pKI1M
         mzCg==
X-Gm-Message-State: AOAM532d/NTwju1TkcHaV8WSpYdFEMhgQr4slKU4Z7oDVpBsvoXZuVBR
	7wldFMiFV6TUF1ULxIq1N5aYU6srfofs2g==
X-Google-Smtp-Source: ABdhPJxISqOOWmLgWDAAOw3kzzTRX2xcHM4R9vZqpv+EYE/4AAOVpEOzHCAWnBLNdTRXDedIXQbwPg==
X-Received: by 2002:a05:6512:2244:: with SMTP id i4mr4065599lfu.219.1632401704895;
        Thu, 23 Sep 2021 05:55:04 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 01/11] vpci: Make vpci registers removal a dedicated function
Date: Thu, 23 Sep 2021 15:54:51 +0300
Message-Id: <20210923125501.234252-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vpci register
handlers depending on the domain: hwdom or guest.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - constify struct pci_dev where possible
---
 xen/drivers/vpci/vpci.c | 7 ++++++-
 xen/include/xen/vpci.h  | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index cbd1bac7fc33..1666402d55b8 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -35,7 +35,7 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
-void vpci_remove_device(struct pci_dev *pdev)
+void vpci_remove_device_registers(const struct pci_dev *pdev)
 {
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
@@ -48,6 +48,11 @@ void vpci_remove_device(struct pci_dev *pdev)
         xfree(r);
     }
     spin_unlock(&pdev->vpci->lock);
+}
+
+void vpci_remove_device(struct pci_dev *pdev)
+{
+    vpci_remove_device_registers(pdev);
     xfree(pdev->vpci->msix);
     xfree(pdev->vpci->msi);
     xfree(pdev->vpci);
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 9f5b5d52e159..2e910d0b1f90 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -28,6 +28,8 @@ int __must_check vpci_add_handlers(struct pci_dev *dev);
 
 /* Remove all handlers and free vpci related structures. */
 void vpci_remove_device(struct pci_dev *pdev);
+/* Remove all handlers for the device given. */
+void vpci_remove_device_registers(const struct pci_dev *pdev);
 
 /* Add/remove a register handler. */
 int __must_check vpci_add_register(struct vpci *vpci,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:03:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:03:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194159.345964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTONj-0001B2-Im; Thu, 23 Sep 2021 13:03:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194159.345964; Thu, 23 Sep 2021 13:03:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTONj-0001Av-FE; Thu, 23 Sep 2021 13:03:19 +0000
Received: by outflank-mailman (input) for mailman id 194159;
 Thu, 23 Sep 2021 13:03:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/6DS=ON=epam.com=prvs=9900839950=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTONh-0001An-ML
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:03:17 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9e1ff9fc-1c6e-11ec-ba32-12813bfff9fa;
 Thu, 23 Sep 2021 13:03:16 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18NChDOr000644;
 Thu, 23 Sep 2021 13:03:12 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2052.outbound.protection.outlook.com [104.47.0.52])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b8r9uggbd-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 13:03:11 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3217.eurprd03.prod.outlook.com (2603:10a6:208:2::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Thu, 23 Sep
 2021 13:03:09 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 13:03:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e1ff9fc-1c6e-11ec-ba32-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OQMsbxw4Z1DmLBnZM9hqkVbBb5XPOfwDh3aNBqBu1pdXlppOCFJVJwiOWJGOxVdt0FDrL9FExXYtld5CkY0PGwUrjADaPaQ7LN1YxpeuqdukbjbX4/G3C2s/xfRuEzyrOxilAzDlmvPwT2odSkO9E0pcBp+5CfLiJMnBFE6RVussnUlB5pMDMqHJ+76o0H+B8ogABau+sErs0rZ3mY3oLllcyInUU32uv0zva/PN4R8Ob7mS9vcnPaVG4TeePBhYJslJm9SGbyDORvfoDA2Csx5/cUYhr8pzZXqFDXXWfbb5xmrZFdeFl8zlCJ4T9UxxqcCto8kbBJMCL8G8on7n4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gDW9yuq4Iug2NRkjjA7yR7nq+UKZzimaAIQHqisNjy0=;
 b=JSyL+2J5uLI7IPcF0ulSVpRTsbvx6PzfyxW0i5tdnHJBzjpMHV02QzERHiNv9aDqmyKfbJ+7FVpThGKPMqTnv0t5ypoSrHtHh+alhxZHl8zTvhXYHk2KvBretNal1vzxos+9pPz+FNn57tcWbQ9UcmPJjwZnsocNoTYN0tQs1d2Qp5Mh+Lz9xZen4x34RYtW/rmKwJGTFKrlHmUvMzixZl5OCS7qCtvp8QPSw1u2v+UIGmggf0sxcXjm0r0SUAgsIrewH9bs7OQBHFaArK2S/lhDb5Dn8KJ1UyOzmQHzwuaxHQ49iiiijIIsU27oBdhmt+cMg3Z2vBr2Tk31HLWXaQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gDW9yuq4Iug2NRkjjA7yR7nq+UKZzimaAIQHqisNjy0=;
 b=rZIRohMUgE6mpqbJqoYiDx37RSTklyaZ4vSk30e+n92pRRddhxQUV0NFhg+Y0WN+/9p0TpJoi0VKX+YU+esx250WnwuEe44iQDAwa7PtqxZFUmUnNfP/YbGH8+Qbf7N4RCf9dCy0IR2GyBvP1pfOY/4h4PfZk8Ae0R13r4bCiRyUk3UV8zX/zQv5Pwsu5YjGGtP5O3AS5jOncxueX41ckITmdJ1OIiicFQX8fAk3Wl/Mqj1i+AGsqwcXcEDH/HboDIK+TWscrCZ5pYaJbdEIxizGMFXR2PYoDS+JGXcWWACuWkGdCmI2YCKuo60aeW373WIqEdtz6eowa9UngdqH7A==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Rahul Singh
	<rahul.singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        "andre.przywara@arm.com"
	<andre.przywara@arm.com>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXsHtbREhsUX85MESzoUZXILt9Yg==
Date: Thu, 23 Sep 2021 13:03:09 +0000
Message-ID: <e2634706-3198-3224-371c-3d13210a35c9@epam.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 37247253-d8a1-4686-ad5d-08d97e927e0c
x-ms-traffictypediagnostic: AM0PR0302MB3217:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR0302MB3217EDF130A006A2A5A6FA47E7A39@AM0PR0302MB3217.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ETU3E1/E4Bl+6vVk2nAmlQjvp05VKeb8TMQaWCn56QBUlkGjkoU2tih+4eZZEMkJiL2cZ6EWaab12qjRxXqiF5kabtwEZ2AZeqozuqidu/DUzj0j9Kj0B3qq3as1S6sTHPbH4sL19Qp++IIuqevpfllMOn90UhFTBElqzFvbXM/MYU2gdd/lQiQZ9EA3aZJResTleb5wgP4yMbWnIyjLQ5KWea2NthSLSjet6UqnrVzoIGY19qGsAkZWlH0t+W5XWO4rXd6jzEVz74Q1hpZHThnIatZ9qtOUW3xFyVEfSyFxrq77s4SEHqWPgPDhKkTjeafwNRa9iszZgtxVDgHzKhfm1/oatuaoLA6pO0Bxe3FPbgToMrzpdxZVJCWgnwmGCWxDIKigIcgpnzcg0GEUla/Lx01h4CSOvJ3FtC+3/oLEn98tho9XaRjTXbgShckLrSuHFDnT9oRH/oLTYfVYSln1ZQWossiPzwcpZgcJASsWUlRnPkavxCmLLCb8p0WDkADmxo5O7LfZcJAJTz+DZIVhbvlt8jYzKEfIqRQcV2aCSty50aYAAOJyEZM4tnGuV/FWWuvPDYOTI8i8rAAbDBGT4f4L0EWdzu0BFztFeo+50wGXoUMFNsR+I3UVk9hGyN6r47BmUiCxUMNdQGQ8/PT2MD6tFGQ+mlo3Tv3ES87qg5wi17ePWCDCpQc6jzzSoEoCVYe38zAUl6wVVSmHtviiXBv1C+10ER0gkYDOhVzwlsLir9gZu0FMEiRK9cYD
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31696002)(316002)(6506007)(54906003)(71200400001)(4326008)(2906002)(8936002)(186003)(2616005)(66446008)(91956017)(64756008)(66556008)(38100700002)(86362001)(6486002)(8676002)(31686004)(36756003)(508600001)(110136005)(6512007)(5660300002)(66476007)(38070700005)(26005)(107886003)(76116006)(122000001)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Mnk0Z0RzdCs3V2RsOFVIM1djQTNtNEM1MERZT2ZFNnhvL2ZOdnlMSTg0Szdn?=
 =?utf-8?B?KzZxZ2xSOGtVK2R3SlhwV2JjSUJ0YmF1ZFRERkluL2NQdmVYL0hHeC91Nm1F?=
 =?utf-8?B?dURHK2FaNnNJcm5yT3RmaWswS21zWmRxNGJycGdKdGtjVmlxMUgxdHBlMGxG?=
 =?utf-8?B?RWxVTkNsM0FXY05yQUJPdTJ5dHF2VXllQUVuN0pCd3hVdmtJQ0M2djRScmV0?=
 =?utf-8?B?aFNuNnNibE5NbEk2UTlLT25xVEtnTVNSRTBBamtmeC9NUHpQNjJ3c0wxQjJK?=
 =?utf-8?B?T3NoVWlQdXNUekI1VHo5b3B2U2wrYTNDWFo2dDdER0xTcS9XREJEQ3g5MjBJ?=
 =?utf-8?B?WDNJMExjQzdWbGFtUXNRZ05YRVJSeXRsNytGZklIQm5tWGJicDNieWdPRFll?=
 =?utf-8?B?Y3pxODJBWGR4S2RnYzBFbkNYVldQNUtPeGRXM0h6NlBlOG1aRkc1UzJ6UnZE?=
 =?utf-8?B?S1E4cG9pV3hNaURyUjd0ZVdrTXp1NjJlWThBSWEvNjM5Z2czVi80eW1nZUg3?=
 =?utf-8?B?RVlwQ1B5aEhoeTJGQnUxQjU1cHkzQjV4QitObWwxSkNWaFF4UWtWRG51OVdE?=
 =?utf-8?B?ZUJmVEJrMS9rMExKRmtzSkV1Ryt1K01nVVpQbnZFdkg3MUY2djJSaTFmQnRJ?=
 =?utf-8?B?YlNpaXU2OHJ5a1Q2dTdzQXo5QXp0UnEvb3V2TUErRG9GUWFYOXZPNmU1amtv?=
 =?utf-8?B?ajYxbzIveWxSMEgxN1JrY29xZ1FWTXkrMTA1UWRTV2ZNYWpUYjVwSy85Mnoz?=
 =?utf-8?B?YzZQV1Vuc005ZExZVzZXclFiUU8zQ0hXdjBmVEIrZys0RDdEMEZBUHY3dDRU?=
 =?utf-8?B?dk91T1NVVFNodnBXQW9lL1dPL01QTld5Z21MYnpoTldpVVB5UGQ0bm5xZEJ0?=
 =?utf-8?B?bit0V2JRY1k4TEJVUjRFNWVoQnBXWDViL09yT0M1dUtzZk5qb1hyY1pzU1g4?=
 =?utf-8?B?SlFBblR5NjAzVnB4VGdNd0RUdGpyZE5EcjFRZFpvN08zU3I2dndmUktrdnlQ?=
 =?utf-8?B?aUxSaVpwZUEwekRYWXFYVml5SmthbXFYMGF1eGEvNWpXVnF1Rmh0SUtwNTJE?=
 =?utf-8?B?VjhVVFgrRXZVUEdyQlpjbGZoWXhDMW5OaVVheDJKMisyRVY2dDNzekxXQ1VE?=
 =?utf-8?B?YlNRa1Jac0dVMlgyenpCeCtaYjJzdXNvbjFtbDM1SVFmd2xGOUtZelpITGpZ?=
 =?utf-8?B?bXl0UDBiNExaWmtwMHZTS3BuK3NiTW9vbHpDL3AyTjlNZUVnRDJ0UUQyTno1?=
 =?utf-8?B?N0FQVzJ4Mk1pNWZyUnViTmkrYVBSeGVUWDZtRS82NWlKTmpZajF6engzWXZ2?=
 =?utf-8?B?V1VSeGhzTS9mS2lEa256WW8yOU5TbkMyVURlSmpCenZRTlI1QVN1Wm9Hd3R5?=
 =?utf-8?B?a2dCUEJOWW16cHZiZjhsRnZFQk45RkdPMXhKYmJ4NHVpWFNPQUNzTEFNS0Ni?=
 =?utf-8?B?Lyt4WWhMY1VXVFNkWEFSeTlOOWZBcVpRQ2pPaytyZ1ZyM21yVlJEbG8zZGlK?=
 =?utf-8?B?L1ZUYmlHNlEwQ2ovTzhoK3lZOCtwSDJNTndaYm1MeEJzdkMzMC9zbDl3eW42?=
 =?utf-8?B?VmJxbTNPQXpzQTZ2S0NhdERzL1dZc2VnM01MSzhXTHR2R3RLUDM4YzJlai96?=
 =?utf-8?B?L0pIQTdtQXBYMkhLRHNBYjdFZTh2T3l4dVg2dkVETGlqZ0xoM05OUG9Jcy84?=
 =?utf-8?B?TmMraUFTeDRzZVFNN2JFZzR6WmlENUJrTGhNSkhPZzViTFE5VUlrRXRpUnVM?=
 =?utf-8?Q?ifMDo09E0cFXt6leKzMnBvG7r3amOv9eXCYXu2n?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <79945C8FC58612419D4727C4428F9BD6@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 37247253-d8a1-4686-ad5d-08d97e927e0c
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 13:03:09.3814
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SxvaWdH5yM68pKHAdyHyovMD/jaHn4v4W6URXfdvPr/N1j+dp7CmzaBjX/By7gv6z73jTPV91Re7yNZQzaSnhDX8lRv4psQ5InKpvjPi8rsNtSbd5FBNVQ3VJB+us5yv
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3217
X-Proofpoint-GUID: j_nOWS5M5mRPjH0JnH7hUKC8wh_ToXX5
X-Proofpoint-ORIG-GUID: j_nOWS5M5mRPjH0JnH7hUKC8wh_ToXX5
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_04,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0
 priorityscore=1501 phishscore=0 adultscore=0 spamscore=0 impostorscore=0
 bulkscore=0 lowpriorityscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109200000
 definitions=main-2109230082

SGksIFN0ZWZhbm8hDQoNCltzbmlwXQ0KDQoNCj4+ICt9Ow0KPj4gKw0KPj4gKy8qIERlZmF1bHQg
RUNBTSBvcHMgKi8NCj4+ICtleHRlcm4gY29uc3Qgc3RydWN0IHBjaV9lY2FtX29wcyBwY2lfZ2Vu
ZXJpY19lY2FtX29wczsNCj4gSWYgd2UgdXNlIGNvbnRhaW5lcl9vZiBhbmQgZ2V0IHJpZCBvZiBz
eXNkYXRhLCBJIHdvbmRlciBpZiB3ZSBnZXQgYXZvaWQNCj4gZXhwb3J0aW5nIHBjaV9nZW5lcmlj
X2VjYW1fb3BzLg0KPg0KPg0KPj4gK2ludCBwY2lfaG9zdF9jb21tb25fcHJvYmUoc3RydWN0IGR0
X2RldmljZV9ub2RlICpkZXYsIGNvbnN0IHZvaWQgKmRhdGEpOw0KPiBUaGlzIHNob3VsZCBiZSBz
dGF0aWMgYW5kIG5vdCBleHBvcnRlZA0KPg0KPg0KPj4gK2ludCBwY2lfZ2VuZXJpY19jb25maWdf
cmVhZChzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYsDQo+PiAr
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3QgbGVuLCB1
aW50MzJfdCAqdmFsdWUpOw0KPiBhbHNvIHRoaXMNCj4NCj4NCj4+ICtpbnQgcGNpX2dlbmVyaWNf
Y29uZmlnX3dyaXRlKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwgdWludDMyX3Qgc2Jk
ZiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgcmVnLCB1aW50MzJf
dCBsZW4sIHVpbnQzMl90IHZhbHVlKTsNCj4gYWxzbyB0aGlzDQo+DQo+DQo+PiArdm9pZCBfX2lv
bWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+
ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3Qgc2JkZiwgdWludDMyX3Qg
d2hlcmUpOw0KPiBhbHNvIHRoaXMNCj4NCj4+ICAgc3RhdGljIGFsd2F5c19pbmxpbmUgYm9vbCBp
c19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCh2b2lkKQ0KPj4gICB7DQo+PiAgICAgICByZXR1cm4g
cGNpX3Bhc3N0aHJvdWdoX2VuYWJsZWQ7DQo+PiAtLSANCj4+IDIuMTcuMQ0KDQpBbGwgdGhlIGFi
b3ZlIGlzIHN0aWxsIG5lZWQgdG8gYmUgZXhwb3J0ZWQgYXMgdGhvc2UgYXJlIGdvaW5nIHRvIGJl
IHVzZWQNCg0KYnkgb3RoZXIgYnJpZGdlJ3MgaW1wbGVtZW50YXRpb25zLCBzZWUgWnlucU1QIGZv
ciBpbnN0YW5jZSBsYXRlciBpbiB0aGUgc2VyaWVzLg0KDQpJJ2xsIHBvc3QgYSBzbmlwcGV0IGZy
b20gdGhlIGZ1dHVyZToNCg0KLyogRUNBTSBvcHMgKi8NCmNvbnN0IHN0cnVjdCBwY2lfZWNhbV9v
cHMgbndsX3BjaWVfb3BzID0gew0KIMKgwqDCoCAuYnVzX3NoaWZ0wqAgPSAyMCwNCiDCoMKgwqAg
LmNmZ19yZWdfaW5kZXggPSBud2xfY2ZnX3JlZ19pbmRleCwNCiDCoMKgwqAgLnBjaV9vcHPCoMKg
wqAgPSB7DQogwqDCoMKgwqDCoMKgwqAgLm1hcF9idXPCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgPSBwY2lfZWNhbV9tYXBfYnVzLA0KIMKgwqDCoMKgwqDCoMKgIC5yZWFkwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gcGNpX2dlbmVyaWNfY29uZmlnX3JlYWQsDQog
wqDCoMKgwqDCoMKgwqAgLndyaXRlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCA9
IHBjaV9nZW5lcmljX2NvbmZpZ193cml0ZSwNCiDCoMKgwqDCoMKgwqDCoCAubmVlZF9wMm1fbWFw
cGluZ8KgwqDCoMKgwqDCoCA9IHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmcsDQogwqDCoMKgIH0N
Cn07DQoNCkRUX0RFVklDRV9TVEFSVChwY2lfZ2VuLCAiUENJIEhPU1QgWllOUU1QIiwgREVWSUNF
X1BDSSkNCi5kdF9tYXRjaCA9IG53bF9wY2llX2R0X21hdGNoLA0KLmluaXQgPSBwY2lfaG9zdF9j
b21tb25fcHJvYmUsDQpEVF9ERVZJQ0VfRU5EDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:06:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:06:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194165.345975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOQq-0001pG-2O; Thu, 23 Sep 2021 13:06:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194165.345975; Thu, 23 Sep 2021 13:06:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOQp-0001p9-VD; Thu, 23 Sep 2021 13:06:31 +0000
Received: by outflank-mailman (input) for mailman id 194165;
 Thu, 23 Sep 2021 13:06:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTOQo-0001oy-12
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:06:30 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1072c53e-1c6f-11ec-ba33-12813bfff9fa;
 Thu, 23 Sep 2021 13:06:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 321542028E;
 Thu, 23 Sep 2021 13:06:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0E8DE13DD5;
 Thu, 23 Sep 2021 13:06:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 0mX+AdR7TGF0PgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 13:06:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1072c53e-1c6f-11ec-ba33-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632402388; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=gETyMuCCO3AVSEUMWLrDiqh3r48UCX/EnrNTKjKthVk=;
	b=d/V+WLyPoCXRhhPzVFHk0iojvnnyd0jM3XFdE5DSctSERwmMQ4jh1mMf3AT9jSb+LmHYYE
	gR/L3unsMXxMWUIw4lSPTMmhwlMcdq/tU5w7OLmAwlfBvEvt8jNqv2YA7Jss2t5cMyU3/Y
	Ow9CeuR+ehdZj78jxtN2to/RCtj1rLE=
Subject: Re: [PATCH 1/9] xen/x86: prevent PVH type from getting clobbered
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <46ec0dab-8b5e-f20f-225b-3f4c57f34a61@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <ca719eaa-892a-9baa-c9a3-136314193e7a@suse.com>
Date: Thu, 23 Sep 2021 15:06:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <46ec0dab-8b5e-f20f-225b-3f4c57f34a61@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="dYGjSbU8vl6fICgOr3YJ4Fc0FV5bSwepS"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--dYGjSbU8vl6fICgOr3YJ4Fc0FV5bSwepS
Content-Type: multipart/mixed; boundary="deraih2vZG4VkwYUGXEwF6oOHPJSNHRFY";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <ca719eaa-892a-9baa-c9a3-136314193e7a@suse.com>
Subject: Re: [PATCH 1/9] xen/x86: prevent PVH type from getting clobbered
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <46ec0dab-8b5e-f20f-225b-3f4c57f34a61@suse.com>
In-Reply-To: <46ec0dab-8b5e-f20f-225b-3f4c57f34a61@suse.com>

--deraih2vZG4VkwYUGXEwF6oOHPJSNHRFY
Content-Type: multipart/mixed;
 boundary="------------6C112F19F9003730F6CCB913"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------6C112F19F9003730F6CCB913
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:07, Jan Beulich wrote:
> Like xen_start_flags, xen_domain_type gets set before .bss gets cleared=
=2E
> Hence this variable also needs to be prevented from getting put in .bss=
,
> which is possible because XEN_NATIVE is an enumerator evaluating to
> zero. Any use prior to init_hvm_pv_info() setting the variable again
> would lead to wrong decisions; one such case is xenboot_console_setup()=

> when called as a result of "earlyprintk=3Dxen".
>=20
> Use __ro_after_init as more applicable than either __section(".data") o=
r
> __read_mostly.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------6C112F19F9003730F6CCB913
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------6C112F19F9003730F6CCB913--

--deraih2vZG4VkwYUGXEwF6oOHPJSNHRFY--

--dYGjSbU8vl6fICgOr3YJ4Fc0FV5bSwepS
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMe9MFAwAAAAAACgkQsN6d1ii/Ey9Y
7gf7Bct/QOlPYrfq/qHJYJVw9KaZg5SYiDvvTS7ZIp4fgN+Wd8TQmuZq+m0cBw7iukRuospaUTLc
U022x873+bE/Rx7ZhXe130EpuVqwtR1VCrRZjTgJQlRHp3CN1ItmUTSpjeY28TVuiV3uB7wVyPX7
RLbmbvP4O2/OeUmk6+QMRU1cr7ZbzCOKhqomUP1jGU7X6Fbm+xDj6xAt+N3aXR4qUXUqYb6ioZIf
FRNNd53KCAUNQmGiFKtXA/m8j74sNcaCUj6+hOKZX9XSiAwl6uXWdojldj38QS7mpzWK1STBHW8o
myjKiV5j2KXN+ixmy4DBGhcWUMepqsevMNIwT6kXtQ==
=onPo
-----END PGP SIGNATURE-----

--dYGjSbU8vl6fICgOr3YJ4Fc0FV5bSwepS--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:07:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:07:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194181.345986 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTORj-0002YV-IA; Thu, 23 Sep 2021 13:07:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194181.345986; Thu, 23 Sep 2021 13:07:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTORj-0002YO-ED; Thu, 23 Sep 2021 13:07:27 +0000
Received: by outflank-mailman (input) for mailman id 194181;
 Thu, 23 Sep 2021 13:07:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHp-0004it-7d
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:57:13 +0000
Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id af9177de-3a8d-49c1-bd4a-675a79d72820;
 Thu, 23 Sep 2021 12:55:14 +0000 (UTC)
Received: by mail-lf1-x136.google.com with SMTP id g41so26148978lfv.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:14 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af9177de-3a8d-49c1-bd4a-675a79d72820
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=fpEmkp363zGTNG/sREpOTD2+RNFlDcJmlTvIa8Vah1w=;
        b=bpnFyr4vGY6stFAtsMo9qX7ZkqJW7256vkP2otZyANDs5R1Qyh53a03oaQJiIO9Tfi
         VADZFcegMz3d43HptXQRRBCE9Gxx0vMyS8AniNFVc6xLaJS/vx7Kr/+07sfBFJGV6Xpy
         LWMfQ+PUwmyqY1X4GEDMUqma1EjVbZWsOyffYXaXWsC3eFY2Q3Tjq5sZYjCYc7l44EbK
         +BEtyp5dq7r4QF+GRr9FVsKAJNxYFta58KVMQ46xAP7fQOZS6a7hNPN2rK1zMl8tkN0i
         jQnXs/gd5/oBSDCzcsb/stq18qcDugzMPAdsbGVksToFDkWLkYEExgyQFAkWHJhxMcTk
         0iYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=fpEmkp363zGTNG/sREpOTD2+RNFlDcJmlTvIa8Vah1w=;
        b=XNnh9CH91uvB4PJSWypfv+a04zGMlc4T0gpFr0UW/wldx8mHS8w+AfqnBoNeEld63T
         FkMAWn85cOS+DWTIt/qDxow0sTpgNhiNo1ISDEyDaugaJIDn5iQQYEZhr55tR9ASs6X0
         YHYKxYgiIky21a+m90TD+uwgD0ZP/fcn5c/94Ff+ipXGvsiTasr+p15aq+E+A7AbaNtd
         WQ5OjQE37XOAyDcKzvvCjCbHzhFshhHDfsw+IAoWOpiaH3H2GWo+SGm4rX92idwe+fEn
         Ad60xkX8w5NC4lrVrVUwgmgecjVWElRnVd2BzQRFjDEbKyM5kzvaXrn9oM8KKPgIz2R9
         Z0vw==
X-Gm-Message-State: AOAM533t8YI7tTjNk9C3SUk3nz/tlb3MMoGhftBhiAuFhRKfAYti2fex
	DzEdRyooIXWUDc8uKlfJWcbMqWmt9HcHiQ==
X-Google-Smtp-Source: ABdhPJzdYDxTei6IVbz6wkAAT9SIcXwlU/2hhWG9paV9ZKyig0hoON0I5dQvzIu3rLdWmGK1j+rN+w==
X-Received: by 2002:a05:6512:32c6:: with SMTP id f6mr3953415lfg.503.1632401713136;
        Thu, 23 Sep 2021 05:55:13 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 08/11] vpci/header: Emulate PCI_COMMAND register for guests
Date: Thu, 23 Sep 2021 15:54:58 +0300
Message-Id: <20210923125501.234252-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add basic emulation support for guests. At the moment only emulate
PCI_COMMAND_INTX_DISABLE bit, the rest is not emulated yet and left
as TODO.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
New in v2
---
 xen/drivers/vpci/header.c | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index bdd18599b205..99f9c37dfb00 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -452,6 +452,32 @@ static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t cmd, void *data)
+{
+    /* TODO: Add proper emulation for all bits of the command register. */
+
+    if ( (cmd & PCI_COMMAND_INTX_DISABLE) == 0 )
+    {
+        /*
+         * Guest wants to enable INTx. It can't be enabled if:
+         *  - host has INTx disabled
+         *  - MSI/MSI-X enabled
+         */
+        if ( pdev->vpci->msi->enabled )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+        else
+        {
+            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, reg);
+
+            if ( current_cmd & PCI_COMMAND_INTX_DISABLE )
+                cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
+    }
+
+    cmd_write(pdev, reg, cmd, data);
+}
+
 static void bar_write(const struct pci_dev *pdev, unsigned int reg,
                       uint32_t val, void *data)
 {
@@ -599,9 +625,12 @@ static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
     struct vpci_bar *bars = header->bars;
     int rc;
 
-    /* Setup a handler for the command register: same for hwdom and guests. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    if ( is_hwdom )
+        rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write,
+                               PCI_COMMAND, 2, header);
+    else
+        rc = vpci_add_register(pdev->vpci, vpci_hw_read16, guest_cmd_write,
+                               PCI_COMMAND, 2, header);
     if ( rc )
         return rc;
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:07:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:07:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194192.345996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOS3-00039D-QZ; Thu, 23 Sep 2021 13:07:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194192.345996; Thu, 23 Sep 2021 13:07:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOS3-000396-N7; Thu, 23 Sep 2021 13:07:47 +0000
Received: by outflank-mailman (input) for mailman id 194192;
 Thu, 23 Sep 2021 13:07:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTOS2-00035D-5l
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:07:46 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id afe49a30-0ebe-4859-b677-71560ff0171f;
 Thu, 23 Sep 2021 13:07:44 +0000 (UTC)
Received: from [192.168.15.189] (unknown [195.68.53.70])
 by beetle.greensocs.com (Postfix) with ESMTPSA id 664F520786;
 Thu, 23 Sep 2021 13:07:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afe49a30-0ebe-4859-b677-71560ff0171f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632402463;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vFKXSKDeRNHY9r3dJXAy1S04EIsRqGcyMaUITIfJp/E=;
	b=Vgjrf4zf3hLAwwoZpB4MMGRryvLD4WxAgB6E7p3RGlh+GLG2sR84EQIPeqHslh9hEijOx5
	nNtoHHTIsQpYwOwgG/vY2WEuAlFPSV2ZBYrKkAib0o9e+EW4Mn4fmHDVaHLy2lNpaWMqxb
	O0dEBOioAyO97QP65z5rCdIdK0W9DF0=
Message-ID: <87197aa8-cce2-4f76-ca31-7989cbeaedba@greensocs.com>
Date: Thu, 23 Sep 2021 15:07:42 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 07/16] hw/core/machine: add
 machine_class_is_dynamic_sysbus_dev_allowed
Content-Language: en-US-large
To: Ani Sinha <ani@anisinha.ca>
Cc: qemu-devel@nongnu.org, Alistair Francis <Alistair.Francis@wdc.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= <berrange@redhat.com>,
 "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Eric Auger <eric.auger@redhat.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Markus Armbruster <armbru@redhat.com>, Eric Blake <eblake@redhat.com>,
 qemu-riscv@nongnu.org, xen-devel@lists.xenproject.org,
 mark.burton@greensocs.com, mirela.grujic@greensocs.com, edgari@xilinx.com,
 Peter Maydell <peter.maydell@linaro.org>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-8-damien.hedde@greensocs.com>
 <alpine.DEB.2.22.394.2109231559230.630@anisinha-lenovo>
From: Damien Hedde <damien.hedde@greensocs.com>
In-Reply-To: <alpine.DEB.2.22.394.2109231559230.630@anisinha-lenovo>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 9/23/21 12:51, Ani Sinha wrote:
> 
> 
> On Wed, 22 Sep 2021, Damien Hedde wrote:
> 
>> Right now the allowance check for adding a sysbus device using
>> -device cli option (or device_add qmp command) is done well after
>> the device has been created. It is done during the machine init done
>> notifier: machine_init_notify() in hw/core/machine.c
>>
>> This new function will allow us to check if a sysbus device type is
>> allowed to be dynamically created by the machine during the device
>> creation time.
>>
>> Also make device_is_dynamic_sysbus() use the new function.
>>
>> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
>> ---
>>
>> In the context of our series, we need to be able to do the check at
>> device creation time to allow doing it depending on the current
>> MACHINE_INIT phase.
>> ---
>>   include/hw/boards.h | 17 +++++++++++++++++
>>   hw/core/machine.c   | 15 ++++++++++++---
>>   2 files changed, 29 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/hw/boards.h b/include/hw/boards.h
>> index 463a5514f9..934443c1cd 100644
>> --- a/include/hw/boards.h
>> +++ b/include/hw/boards.h
>> @@ -51,6 +51,23 @@ void machine_set_cpu_numa_node(MachineState *machine,
>>    */
>>   void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type);
>>
>> +/**
>> + * machine_class_is_dynamic_sysbus_dev_allowed: Check if type is an allowed
>> + * sysbus device type for the machine class.
>> + * @mc: Machine class
>> + * @type: type to check (should be a subtype of TYPE_SYS_BUS_DEVICE)
>> + *
>> + * Returns: true if @type is a type in the machine's list of
>> + * dynamically pluggable sysbus devices; otherwise false.
>> + *
>> + * Check if the QOM type @type is in the list of allowed sysbus device
>> + * types (see machine_class_allowed_dynamic_sysbus_dev()).
>> + * Note that if @type is a subtype of a type which is in the list, it is
>> + * allowed too.
>> + */
>> +bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
>> +                                                 const char *type);
>> +
> 
> How about renaming this to device_type_is_allowed_dynamic_sysbus() ?

I followed the "machine_class_allow_dynamic_sysbus_dev" function name style.

But sure I'll rename it. We can even drop the "allowed" if we want
to stay closer to the device_is_dynamic_sysbus legacy function which if 
used to check this on a device instance.

Damien
--
Thanks


> 
>>   /**
>>    * device_is_dynamic_sysbus: test whether device is a dynamic sysbus device
>>    * @mc: Machine class
>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>> index 9125c9aad0..1a18912dc8 100644
>> --- a/hw/core/machine.c
>> +++ b/hw/core/machine.c
>> @@ -545,18 +545,27 @@ void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type)
>>
>>   bool device_is_dynamic_sysbus(MachineClass *mc, DeviceState *dev)
>>   {
>> -    bool allowed = false;
>> -    strList *wl;
>>       Object *obj = OBJECT(dev);
>>
>>       if (!object_dynamic_cast(obj, TYPE_SYS_BUS_DEVICE)) {
>>           return false;
>>       }
>>
>> +    return machine_class_is_dynamic_sysbus_dev_allowed(mc,
>> +            object_get_typename(obj));
>> +}
>> +
>> +bool machine_class_is_dynamic_sysbus_dev_allowed(MachineClass *mc,
>> +                                                 const char *type)
>> +{
>> +    bool allowed = false;
>> +    strList *wl;
>> +    ObjectClass *klass = object_class_by_name(type);
>> +
>>       for (wl = mc->allowed_dynamic_sysbus_devices;
>>            !allowed && wl;
>>            wl = wl->next) {
>> -        allowed |= !!object_dynamic_cast(obj, wl->value);
>> +        allowed |= !!object_class_dynamic_cast(klass, wl->value);
>>       }
>>
>>       return allowed;
>> --
>> 2.33.0
>>
>>


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:08:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:08:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194208.346008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSm-0003pP-3E; Thu, 23 Sep 2021 13:08:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194208.346008; Thu, 23 Sep 2021 13:08:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSm-0003pI-00; Thu, 23 Sep 2021 13:08:32 +0000
Received: by outflank-mailman (input) for mailman id 194208;
 Thu, 23 Sep 2021 13:08:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHz-0004it-87
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:57:23 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dde5d022-867a-48eb-9e7e-8b40d99586ad;
 Thu, 23 Sep 2021 12:55:17 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id b15so25738434lfe.7
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:17 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dde5d022-867a-48eb-9e7e-8b40d99586ad
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=t+TG3ZsjniyYOn1TjIrSDa/ZTsIEIfPrt6I6dPYbhyk=;
        b=eV8YVH5NC4DmSL4x9/RAgitTwlfCmkQPMgFjPMCQK0n3AeZxwnSn+Udn3ubor53chc
         UEnrd9Smm1404yKVVdaLjy3hOJiWBs7T/rVlalmkXg8d5F1EoxTvtMek+bFqTzq7ufkJ
         Ydn5qa0Q1Oxs7w9E3XPmEQi5d5dBjhmPZfK3TsooOGEDddIb3G3she0IOltWH9l+WSHp
         BoZAPk6yFF9zp3nTpBJacx+xWWVLPo3Lt6e8BYc8qB1gr7SsEU4ljwiOFgqvyHmb9Leo
         VFFmKbTyuoA/iNCivDgXGHSYRXDzRaeZbBxNiOBI6RvCi+2vKVJbQWuifzkPHZXRqUEv
         jw/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=t+TG3ZsjniyYOn1TjIrSDa/ZTsIEIfPrt6I6dPYbhyk=;
        b=a4/HE7eoZgMpj0aWHKgbDfO6ZtTPItnWKwsbJHvbMH45Q6R0cJlyjXAt+x1R+40x0b
         x59R7FZAfaSfjNbw7QCO5xsCIkOoBBNmei47TleChP1Kvtye2uOy/QykAmKzaKb1DMK5
         JZWdVDJQIb4a/tr6bmRmC/Zg4mOichC8cKiZIpuwKwfQComQbzeP4vq4o7Ax3Fn3oOIu
         IdUHAxe4LO0wl4MV0F+HXFDQT5WeNEntgUlAQ7l2jygd6KkoTYtsE0oDoJ7TFgQzKl+y
         myz5c63ww7C4SgbDcifSbDVNkUJJUdTc4JZ175psdAXDt0rJigyBp9HNwcVAwJ94p3vx
         IDdw==
X-Gm-Message-State: AOAM530xdypUlPNi81Zzfcc264FQKMSX2UEoFBFTVHR/y2k++2zUxPSW
	GNSakt2H/UpebwyaZIRsXK4MeGIOKAVgfA==
X-Google-Smtp-Source: ABdhPJyeEpTK6x6nOoWHlX/qYeM4XHS0Q00/ZB2U8VKbmN/QGEVH32sZonpdDV4deZk47lwatxUBhA==
X-Received: by 2002:a2e:bd03:: with SMTP id n3mr5073432ljq.359.1632401714161;
        Thu, 23 Sep 2021 05:55:14 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 09/11] vpci/header: Reset the command register when adding devices
Date: Thu, 23 Sep 2021 15:54:59 +0300
Message-Id: <20210923125501.234252-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reset the command register when passing through a PCI device:
it is possible that when passing through a PCI device its memory
decoding bits in the command register are already set. Thus, a
guest OS may not write to the command register to update memory
decoding, so guest mappings (guest's view of the BARs) are
left not updated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - do not write 0 to the command register, but respect host settings.
---
 xen/drivers/vpci/header.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 99f9c37dfb00..b2829b9d206b 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -452,8 +452,7 @@ static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
-static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
-                            uint32_t cmd, void *data)
+static uint32_t emulate_cmd_reg(const struct pci_dev *pdev, uint32_t cmd)
 {
     /* TODO: Add proper emulation for all bits of the command register. */
 
@@ -468,14 +467,20 @@ static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
             cmd |= PCI_COMMAND_INTX_DISABLE;
         else
         {
-            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, reg);
+            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
 
             if ( current_cmd & PCI_COMMAND_INTX_DISABLE )
                 cmd |= PCI_COMMAND_INTX_DISABLE;
         }
     }
 
-    cmd_write(pdev, reg, cmd, data);
+    return cmd;
+}
+
+static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t cmd, void *data)
+{
+    cmd_write(pdev, reg, emulate_cmd_reg(pdev, cmd), data);
 }
 
 static void bar_write(const struct pci_dev *pdev, unsigned int reg,
@@ -794,6 +799,10 @@ int vpci_bar_add_handlers(const struct domain *d, const struct pci_dev *pdev)
         gdprintk(XENLOG_ERR,
                  "%pp: failed to add BAR handlers for dom%pd: %d\n",
                  &pdev->sbdf, d, rc);
+
+    /* Reset the command register with respect to host settings. */
+    pci_conf_write16(pdev->sbdf, PCI_COMMAND, emulate_cmd_reg(pdev, 0));
+
     return rc;
 }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:08:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:08:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194209.346019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSn-00047j-CB; Thu, 23 Sep 2021 13:08:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194209.346019; Thu, 23 Sep 2021 13:08:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSn-00047Z-7h; Thu, 23 Sep 2021 13:08:33 +0000
Received: by outflank-mailman (input) for mailman id 194209;
 Thu, 23 Sep 2021 13:08:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOI4-0004it-89
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:57:28 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c1ab318b-7c38-43f1-8007-8693397c9763;
 Thu, 23 Sep 2021 12:55:17 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id i25so26496283lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:17 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1ab318b-7c38-43f1-8007-8693397c9763
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=XlpFRu8hRrvPpjMFenpSAJbBdeavkULfWEH3e9kj/3o=;
        b=pn/1WK9cTRCVMT8S8N+kHcScv8SVfp4WAmQl55y/V20VbgiCkjI6Mk421HuVf6HD5d
         gDj6lasHPMcFqrunqkhsT24D1oJD0PfKOWmQLaRUOyNed5XAR7hMxyJZ/puHpCBQKzlq
         GeA2LA9d28oc5bmqWILcBA1L3xPM08dTwe5eVd/3dhpR6mLVE9XbtOzCp18JDAg19kK1
         MSTJNsM0HppMoi6zI5afgpVAVQlaOc97sFuV7yj5jvHo+w6X6PoZrUhXvz6jZn30jTAq
         PrhFFr4qfbF33EQv2Xff6GNBP7lYEX2WiZq1luKuv6c9ezife0E9U3tQnXqFF0k7uy+G
         dwZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=XlpFRu8hRrvPpjMFenpSAJbBdeavkULfWEH3e9kj/3o=;
        b=yNttErMUSD8q+Gz2bHDy7TpdMhaYjVpufRncBVD7FfZN9qlUsfLeOAUg46bYOb3Jcw
         Mh2CyJwL2mAVahntilEtfs/oMrkjbC/iKBU+ymTFgO7oznIEGwuUEropsA7qFkow/dnF
         4mrYIEsAzQpKK2hjaDFroPbqT1K2yWI2MlQEDtnazlTpioLDuYAnvUm5QT6w0Og8i3DR
         9vAGuQgU9ZU6QrKhTlIYFjpJ/GUroR7vBFUGNH/pFLN3k7cCNj7aOhk+jj/jh49qyMVc
         +am2ZBN7MG6Ku08+BZb1gSF+4INDd5QX8lfz/Qo/4dXt4KBWzhpgd5PIH7nr49E9MRxe
         /u2w==
X-Gm-Message-State: AOAM530VvNIpAoQJabylMuoyRopAMu5aZ0RoegxAQDLf9p2EyhGoiD1k
	ThFQ7IZWx25pyeHg3rUsqlTl+ixkQl0mpg==
X-Google-Smtp-Source: ABdhPJx2iIC0KBtVCRSDDJlP12bf3jCvWEtBjbBIXUe+n2z2VKx+YKSykuDGyFp3BrRnSXWUp8dwoA==
X-Received: by 2002:a05:6512:1151:: with SMTP id m17mr4030772lfg.144.1632401716421;
        Thu, 23 Sep 2021 05:55:16 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for guests
Date: Thu, 23 Sep 2021 15:55:01 +0300
Message-Id: <20210923125501.234252-12-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
New in v2
---
 xen/arch/arm/domain.c         |  1 +
 xen/arch/arm/vpci.c           | 87 +++++++++++++++++++++++++++++++----
 xen/arch/arm/vpci.h           |  3 ++
 xen/drivers/passthrough/pci.c | 25 ++++++++++
 xen/include/asm-arm/pci.h     |  1 +
 xen/include/xen/pci.h         |  1 +
 xen/include/xen/sched.h       |  2 +
 7 files changed, 111 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index c7b25bc70439..c0ad6ad682d2 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -797,6 +797,7 @@ void arch_domain_destroy(struct domain *d)
                        get_order_from_bytes(d->arch.efi_acpi_len));
 #endif
     domain_io_free(d);
+    domain_vpci_free(d);
 }
 
 void arch_domain_shutdown(struct domain *d)
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 14947e975d69..012f958960d1 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -17,6 +17,14 @@
 
 #define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
 
+struct vpci_mmio_priv {
+    /*
+     * Set to true if the MMIO handlers were set up for the emulated
+     * ECAM host PCI bridge.
+     */
+    bool is_virt_ecam;
+};
+
 /* Do some sanity checks. */
 static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
 {
@@ -38,6 +46,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
     pci_sbdf_t sbdf;
     unsigned long data = ~0UL;
     unsigned int size = 1U << info->dabt.size;
+    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
 
     sbdf.sbdf = MMCFG_BDF(info->gpa);
     reg = REGISTER_OFFSET(info->gpa);
@@ -45,6 +54,13 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
     if ( !vpci_mmio_access_allowed(reg, size) )
         return 0;
 
+    /*
+     * For the passed through devices we need to map their virtual SBDF
+     * to the physical PCI device being passed through.
+     */
+    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v, &sbdf) )
+            return 1;
+
     data = vpci_read(sbdf, reg, min(4u, size));
     if ( size == 8 )
         data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
@@ -61,6 +77,7 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
     pci_sbdf_t sbdf;
     unsigned long data = r;
     unsigned int size = 1U << info->dabt.size;
+    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
 
     sbdf.sbdf = MMCFG_BDF(info->gpa);
     reg = REGISTER_OFFSET(info->gpa);
@@ -68,6 +85,13 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
     if ( !vpci_mmio_access_allowed(reg, size) )
         return 0;
 
+    /*
+     * For the passed through devices we need to map their virtual SBDF
+     * to the physical PCI device being passed through.
+     */
+    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v, &sbdf) )
+            return 1;
+
     vpci_write(sbdf, reg, min(4u, size), data);
     if ( size == 8 )
         vpci_write(sbdf, reg + 4, 4, data >> 32);
@@ -80,13 +104,48 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
     .write = vpci_mmio_write,
 };
 
+/*
+ * There are three  originators for the PCI configuration space access:
+ * 1. The domain that owns physical host bridge: MMIO handlers are
+ *    there so we can update vPCI register handlers with the values
+ *    written by the hardware domain, e.g. physical view of the registers/
+ *    configuration space.
+ * 2. Guest access to the passed through PCI devices: we need to properly
+ *    map virtual bus topology to the physical one, e.g. pass the configuration
+ *    space access to the corresponding physical devices.
+ * 3. Emulated host PCI bridge access. It doesn't exist in the physical
+ *    topology, e.g. it can't be mapped to some physical host bridge.
+ *    So, all access to the host bridge itself needs to be trapped and
+ *    emulated.
+ */
 static int vpci_setup_mmio_handler(struct domain *d,
                                    struct pci_host_bridge *bridge)
 {
-    struct pci_config_window *cfg = bridge->cfg;
+    struct vpci_mmio_priv *priv;
+
+    priv = xzalloc(struct vpci_mmio_priv);
+    if ( !priv )
+        return -ENOMEM;
+
+    priv->is_virt_ecam = !is_hardware_domain(d);
 
-    register_mmio_handler(d, &vpci_mmio_handler,
-                          cfg->phys_addr, cfg->size, NULL);
+    if ( is_hardware_domain(d) )
+    {
+        struct pci_config_window *cfg = bridge->cfg;
+
+        bridge->mmio_priv = priv;
+        register_mmio_handler(d, &vpci_mmio_handler,
+                              cfg->phys_addr, cfg->size,
+                              priv);
+    }
+    else
+    {
+        d->vpci_mmio_priv = priv;
+        /* Guest domains use what is programmed in their device tree. */
+        register_mmio_handler(d, &vpci_mmio_handler,
+                              GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE,
+                              priv);
+    }
     return 0;
 }
 
@@ -95,13 +154,16 @@ int domain_vpci_init(struct domain *d)
     if ( !has_vpci(d) )
         return 0;
 
-    if ( is_hardware_domain(d) )
-        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
-
-    /* Guest domains use what is programmed in their device tree. */
-    register_mmio_handler(d, &vpci_mmio_handler,
-                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
+    return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+}
 
+static int domain_vpci_free_cb(struct domain *d,
+                               struct pci_host_bridge *bridge)
+{
+    if ( is_hardware_domain(d) )
+        XFREE(bridge->mmio_priv);
+    else
+        XFREE(d->vpci_mmio_priv);
     return 0;
 }
 
@@ -124,6 +186,13 @@ int domain_vpci_get_num_mmio_handlers(struct domain *d)
     return count;
 }
 
+void domain_vpci_free(struct domain *d)
+{
+    if ( !has_vpci(d) )
+        return;
+
+    pci_host_iterate_bridges(d, domain_vpci_free_cb);
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
index 27a2b069abd2..38e5a28c0d95 100644
--- a/xen/arch/arm/vpci.h
+++ b/xen/arch/arm/vpci.h
@@ -18,6 +18,7 @@
 #ifdef CONFIG_HAS_VPCI
 int domain_vpci_init(struct domain *d);
 int domain_vpci_get_num_mmio_handlers(struct domain *d);
+void domain_vpci_free(struct domain *d);
 #else
 static inline int domain_vpci_init(struct domain *d)
 {
@@ -28,6 +29,8 @@ static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
     return 0;
 }
+
+static inline void domain_vpci_free(struct domain *d) { }
 #endif
 
 #endif /* __ARCH_ARM_VPCI_H__ */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 4552ace855e0..579c6947cc35 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -890,6 +890,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
     return 0;
 }
 
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf)
+{
+    struct domain *d = v->domain;
+    struct vpci_dev *vdev;
+    bool found = false;
+
+    pcidevs_lock();
+    list_for_each_entry ( vdev, &d->vdev_list, list )
+    {
+        if ( vdev->sbdf.sbdf == sbdf->sbdf )
+        {
+            /* Replace virtual SBDF with the physical one. */
+            *sbdf = vdev->pdev->sbdf;
+            found = true;
+            break;
+        }
+    }
+    pcidevs_unlock();
+    return found;
+}
+
 /* Caller should hold the pcidevs_lock */
 static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
                            uint8_t devfn)
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index b81f66e813ef..983a63be7572 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -72,6 +72,7 @@ struct pci_host_bridge {
     struct pci_config_window* cfg;   /* Pointer to the bridge config window */
     void *sysdata;                   /* Pointer to the config space window*/
     const struct pci_ops *ops;
+    void *mmio_priv;                 /* MMIO handler's private data. */
 };
 
 struct pci_ops {
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 35ae1d093921..0017de19e7c8 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -191,6 +191,7 @@ struct pci_dev *pci_get_real_pdev(int seg, int bus, int devfn);
 struct pci_dev *pci_get_pdev_by_domain(const struct domain *, int seg,
                                        int bus, int devfn);
 void pci_check_disable_device(u16 seg, u8 bus, u8 devfn);
+bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf);
 
 uint8_t pci_conf_read8(pci_sbdf_t sbdf, unsigned int reg);
 uint16_t pci_conf_read16(pci_sbdf_t sbdf, unsigned int reg);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index d304c7ebe766..0626820545cc 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -450,6 +450,8 @@ struct domain
      * to assign a unique SBDF to a passed through virtual PCI device.
      */
     int vpci_dev_next;
+    /* Virtual PCI MMIO handler's private data. */
+    void *vpci_mmio_priv;
 #endif
 
 #ifdef CONFIG_HAS_PASSTHROUGH
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:08:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:08:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194212.346030 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSr-0004VK-Rs; Thu, 23 Sep 2021 13:08:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194212.346030; Thu, 23 Sep 2021 13:08:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOSr-0004VD-NJ; Thu, 23 Sep 2021 13:08:37 +0000
Received: by outflank-mailman (input) for mailman id 194212;
 Thu, 23 Sep 2021 13:08:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JD/Q=ON=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mTOHu-0004it-7y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 12:57:18 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bf96aaab-a758-4d13-82cb-0f9b9f285ee2;
 Thu, 23 Sep 2021 12:55:16 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id b20so26425942lfv.3
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 05:55:16 -0700 (PDT)
Received: from localhost.localdomain ([185.199.97.5])
 by smtp.gmail.com with ESMTPSA id o12sm453010lft.254.2021.09.23.05.55.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 23 Sep 2021 05:55:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf96aaab-a758-4d13-82cb-0f9b9f285ee2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=01d6SaTi/ep3sxri+AlZsSZXy8XnKOJzH2RZEoLRhnE=;
        b=EWpEK98V3gn+BXUIniS4aeCA9MHYJ+uraGzPRHWI6TQV1B/vOoZeupB+37cr+IGN5l
         ecgShTHEtU2cFr68IVOvI7ZjrPH0mELx7q5M2RfFA8RxYf62iGZGZgB4d9huCMUKwJfI
         rGF75wr4M7zdy8Vi/3omiimpqMmoiPB++DcA182+C+cCOS5SNWsaGte1VMc4ISIdMYeU
         rvNSAtt5UvAYcglcHlgkOy6E/D+dEtx4joYE3SnnrHq3wzNfOPDbQQ5ep1vQAXvSRdWv
         ZwRm1UayVuL8hQeFccvHM3BPFjlSdpDw6ZPFeeWU6FplECWsRyCoua3q5zY1abpy29BS
         TyoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=01d6SaTi/ep3sxri+AlZsSZXy8XnKOJzH2RZEoLRhnE=;
        b=nlcDXV1u1cnc+4YFn2ba3tN+MGMyo7eUUp1AuyM5rohwKnRgY+7oW0X6RtD5eUJ7+e
         AqYMP9BBVaOlTKaGkRt4R54QFKzSbbNwPwpicQTodRd7BiSTQXxLV5kzi8lpDOm6qwkQ
         ye4PgQrSTQHy5gaIOcl/x+3T3bQi75+LN/sx/Ug7N1k8u6vrZ/mH1FbfnKjfcOx3Xfzx
         Fb5FwOgfOUnzqqAtUPZTyHdzcWB3sWimRMyKFLrSSLZrk/sAeVS1mm0kTRfM2kG4D+hx
         EQtoqiFas0VSEZSQqJBoWg5wSrk/+sAqHLBv5mWR5pZMJoQtcyDXc43gDsYdbIAPJnt4
         K88w==
X-Gm-Message-State: AOAM530ziUhSgaeuaJ6B9MN3eXhcEwn5DSVjiDSnr4jS0ZrYc0ctve/1
	zZt8XsdKY4miS6JmE6ymVZBk41ACHtlJ2g==
X-Google-Smtp-Source: ABdhPJx3puHxJXs6dd2LzzWCWWYJzbY4YRnuivwUaxA1/mtGOwJpGs6ZnDTtx7CE1RfoyWW9sphKRw==
X-Received: by 2002:a2e:6e08:: with SMTP id j8mr4786466ljc.247.1632401715286;
        Thu, 23 Sep 2021 05:55:15 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus topology
Date: Thu, 23 Sep 2021 15:55:00 +0300
Message-Id: <20210923125501.234252-11-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210923125501.234252-1-andr2000@gmail.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
New in v2
---
 xen/common/domain.c           |  1 +
 xen/drivers/passthrough/pci.c | 57 +++++++++++++++++++++++++++++++++++
 xen/drivers/vpci/vpci.c       | 18 +++++++++--
 xen/include/xen/pci.h         | 18 +++++++++++
 xen/include/xen/sched.h       |  6 ++++
 5 files changed, 97 insertions(+), 3 deletions(-)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index 40d67ec34232..b80ff2e5e2e6 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -601,6 +601,7 @@ struct domain *domain_create(domid_t domid,
 
 #ifdef CONFIG_HAS_PCI
     INIT_LIST_HEAD(&d->pdev_list);
+    INIT_LIST_HEAD(&d->vdev_list);
 #endif
 
     /* All error paths can depend on the above setup. */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index e1da283d73ad..4552ace855e0 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -833,6 +833,63 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
     return ret;
 }
 
+static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
+                                                const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    list_for_each_entry ( vdev, &d->vdev_list, list )
+        if ( vdev->pdev == pdev )
+            return vdev;
+    return NULL;
+}
+
+int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    ASSERT(!pci_find_virtual_device(d, pdev));
+
+    /* Each PCI bus supports 32 devices/slots at max. */
+    if ( d->vpci_dev_next > 31 )
+        return -ENOSPC;
+
+    vdev = xzalloc(struct vpci_dev);
+    if ( !vdev )
+        return -ENOMEM;
+
+    /* We emulate a single host bridge for the guest, so segment is always 0. */
+    *(u16*) &vdev->seg = 0;
+    /*
+     * The bus number is set to 0, so virtual devices are seen
+     * as embedded endpoints behind the root complex.
+     */
+    *((u8*) &vdev->bus) = 0;
+    *((u8*) &vdev->devfn) = PCI_DEVFN(d->vpci_dev_next++, 0);
+
+    vdev->pdev = pdev;
+    vdev->domain = d;
+
+    pcidevs_lock();
+    list_add_tail(&vdev->list, &d->vdev_list);
+    pcidevs_unlock();
+
+    return 0;
+}
+
+int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    pcidevs_lock();
+    vdev = pci_find_virtual_device(d, pdev);
+    if ( vdev )
+        list_del(&vdev->list);
+    pcidevs_unlock();
+    xfree(vdev);
+    return 0;
+}
+
 /* Caller should hold the pcidevs_lock */
 static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
                            uint8_t devfn)
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 0cdc0c3f75c4..a40a138a14f7 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -90,24 +90,36 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
 /* Notify vPCI that device is assigned to guest. */
 int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
 {
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    int rc;
+#endif
+
     /* It only makes sense to assign for hwdom or guest domain. */
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
 #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
-    return vpci_bar_add_handlers(d, dev);
-#else
-    return 0;
+    rc = vpci_bar_add_handlers(d, dev);
+    if ( rc )
+        return rc;
 #endif
+
+    return pci_add_virtual_device(d, dev);
 }
 
 /* Notify vPCI that device is de-assigned from guest. */
 int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
 {
+    int rc;
+
     /* It only makes sense to de-assign from hwdom or guest domain. */
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
+    rc = pci_remove_virtual_device(d, dev);
+    if ( rc )
+        return rc;
+
 #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
     return vpci_bar_remove_handlers(d, dev);
 #else
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 2b2dfb6f1b49..35ae1d093921 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -136,6 +136,22 @@ struct pci_dev {
     struct vpci *vpci;
 };
 
+struct vpci_dev {
+    struct list_head list;
+    /* Physical PCI device this virtual device is connected to. */
+    const struct pci_dev *pdev;
+    /* Virtual SBDF of the device. */
+    const union {
+        struct {
+            uint8_t devfn;
+            uint8_t bus;
+            uint16_t seg;
+        };
+        pci_sbdf_t sbdf;
+    };
+    struct domain *domain;
+};
+
 #define for_each_pdev(domain, pdev) \
     list_for_each_entry(pdev, &(domain)->pdev_list, domain_list)
 
@@ -165,7 +181,9 @@ int pci_add_segment(u16 seg);
 const unsigned long *pci_get_ro_map(u16 seg);
 int pci_add_device(u16 seg, u8 bus, u8 devfn,
                    const struct pci_dev_info *, nodeid_t node);
+int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev);
 int pci_remove_device(u16 seg, u8 bus, u8 devfn);
+int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev);
 int pci_ro_device(int seg, int bus, int devfn);
 int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn);
 struct pci_dev *pci_get_pdev(int seg, int bus, int devfn);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404e6..d304c7ebe766 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -444,6 +444,12 @@ struct domain
 
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
+    struct list_head vdev_list;
+    /*
+     * Current device number used by the virtual PCI bus topology
+     * to assign a unique SBDF to a passed through virtual PCI device.
+     */
+    int vpci_dev_next;
 #endif
 
 #ifdef CONFIG_HAS_PASSTHROUGH
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:23:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:23:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194251.346041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOgp-0007u2-4u; Thu, 23 Sep 2021 13:23:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194251.346041; Thu, 23 Sep 2021 13:23:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTOgp-0007tv-1g; Thu, 23 Sep 2021 13:23:03 +0000
Received: by outflank-mailman (input) for mailman id 194251;
 Thu, 23 Sep 2021 13:23:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTOgn-0007tp-01
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:23:01 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 22dd4093-1830-4084-82df-04402fb89cff;
 Thu, 23 Sep 2021 13:22:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22dd4093-1830-4084-82df-04402fb89cff
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632403379;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=2Ja6wiy9vmJC0clJluFPedYUYuglRJmxbJ233MRb8XY=;
  b=OFit1Fr7pfFTDIqP8PDLNwHuZ1dnljhcEjk2zY+Fem9xbAoY6jlf98JG
   3b6f0ooG2DUcSzNR5MvyPz0YRqOUOd43hhbDUZ/qxxV+iBPppLH5u1uqu
   3oX1mkaGVoLhfN4NnpmyWVubJj8IcvMrtlc3DI6ki5jrGlu8TzhBFtaMk
   g=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 70kYRKc/smLy3vXyZ/IOBPrcLt+tjNzLVTFOQ86dj6MgPOgQRAd0cy0PN6J26l5431DaCh4LXI
 mxhkRXyva3nxiEMjSFb85ZLbpJjkHBixUjyoy2fSlpJvKuUgC+va6PbPwbTrGEmsYFfzRian2B
 c56mm+jgRC7k6Plmw/uEfLhWIkCyYEbQtebDq1XvZ7+WeUEKLCT9uW0OziHPoK1CcU8UfQOOt3
 gh9QvN014dD+IppNpRPBi4GygAHwah/EVNriDzxq0OkAI5bDfXNgsMyReea45FiLO63HpwmNyb
 fzJdwB+lM9zEJ1VAD+Zx8wtW
X-SBRS: 5.1
X-MesageID: 53034015
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:bkYDZ6AVrXNzLRVW//fkw5YqxClBgxIJ4kV8jS/XYbTApDp012YGy
 zQWWz3VO66Ka2L0fIoiPoW38UNQvZHWz9FqQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00k7wobVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/pGqMwM570
 uV0hb+qQjoSBIjDmuQMekwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHv6SvoMAhWpYasZmQfrBN
 s8ITBhUNi+DTh91EWU2DLYEpbL97pX4W2IB8w/EzUYt2EDMyCRh3b6rN8DaEvSaSMMQkkuGq
 2bu+2XiHgpcJNGZ0SCC8H+nmqnIhyyTcIAYGaC89/VqqEaO3WFVAxoTPWZXutHg1BT4AYgGb
 RVJpGx+9sDe6XBHUPHSABjnvnSejicseMBbLtYAzQ6Pyrv9tlPx6nc/chZNb9kvtckTTDMs1
 0OUk96BOQGDoIF5WlrGqezJ923a1Tw9aDZYPH5ZEVJtD8zL/dlr5i8jWOqPB0JcYjfdIjj23
 znChyw3nbx7YSUjhvjjoAivb95BoPH0ou8JCuf/AjnNAuBRPtfNi2mUBb7zt60owGGxFAXpg
 ZT8s5LChN3i9LnU/MB3fM0DHauy+9GOOyDGjFhkEvEJrmr2oSX4JdgLvmsvdS+F1/ronxezO
 yc/XisLuPdu0IaCN/crM+pd9exwpUQfKTgVfq+NNYcfCnSAXASG4DtvdSatM5PFyiARfVUEE
 c7DK66EVC9CYYw+lWbeb7pNgNcDm3FlrUuOFM+T8vhS+efHDJJjYexeawXmgyFQxP7snTg5B
 P4FbJbVl0UDDreuCsQVmKZKRW03wbEALcmeg+Rcd/KZIxogH2ckCvTLxqgmdZAjlKNQ/tokN
 FnnMqOB4Fag13DBNyuQbXVvNOHmUZpl9CppNi0wJ1e4nXMkZN/3vqsYcpI2e5gh9fBikqEoH
 6VUJZ3YD6QdUCnD9hQccYL58N5oeiO0iF/cJCGiejU+IcJtHlSb5t/+cwLz3yASFS7r59Amq
 rit21qDE5oOTghvFujMb/erww/jtHQRgrsqDUDJPsNSaAPn940zc379ifo+IsctLxTfx2TFi
 1bKUElA/eSU+t076tjEg6yAvryFKeomExoIBXTf4Ja3KTLeojipz7hfXbvaZjvaTm71pvmvP
 L0H0/HmPfQbt19WqI4gQa1zxKcz6taz9b9XygNoQCfCY1ixU+4yJ3CH2Y9Et7FXx68fsgyzA
 xrd9t5fMLSPGcXkDF9Oe1Z1MrXdjakZymvI8PA4AETm/ysmrrOIXHJbMwSIlCEAfqB+N5kow
 Lt5tcMbg+BlZsHG7jpSYvhoylmx
IronPort-HdrOrdr: A9a23:hn75waoPv2k0riZtpAM81ksaV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NCZLXTbUQqTXfpfBO7ZrQEIdBeOlNK1uZ
 0QFpSWTeeAcWSS7vyKkTVQcexQueVvmZrA7Yy1rwYPPHFXguNbnn9E426gYzNLrWJ9dPwE/f
 Snl656T23KQwVpUi33PAhJY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX232oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iBnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDA4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWArqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocWTbqjVQGagoBT+q3oYpxqdS32BnTq+/blnQS+pUoJjHfxn6ck7zA9HJFUcegM2w
 2LCNUvqFniJvVmGZ6VP91xM/dfPFa9Ny4kAFjiUmgPK5t3Tk4li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53034015"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lLDK+QxI29Pbk35K+/+aKBR7QnZaQ6kLMI3Tv/lIlwL8DJZP954VRqfURMciK40ZKPsCHTDVzHtzpRhSHSXhhiRqpi3kmgSCu0y7QG1PXRwH7yzfv6rlQVzCYbGgUYjEzJ8wn3GyJ+Xu1FWvM10lQytNLtVU1yO40Qpy6eLwyO5pdwoGsF5P4mxJYkN6HntSEtow4aDhZ+VWdySvutC+kU7bqfbw8SFZNkgatRMictiFMWp4Gf8U1bo7Rjv2/Z3cnOLA5lqi7uLUBBMi+ucZ+R4Er5EiJSRO0EbI0E0uFSvXPz7jTrEEoRdyuyQX19tggWvNdtT61QFS7bXq5aTYMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LgSDTGtxKVdLSomQ6SlUTGGVV+6O9JYMKbeZi8zhrW0=;
 b=RfcTW63cbWeS9sWqH7r3SoU+c+9MZrKnEaoSONWIXazoShWmAJHPzBmiA81e7v7xkAeFiQrefMnOu9yan2Ry3nfLc6cwsafIWftIX4LsO4ZmGfXIYdtuKn7M8D1HNJhZHoytN6+OWVxnqItPoC0CoNU4kI4wKWHijaVrHWntmh/eO4P41Xo4hR5WOtqgXQJIdKf79RC5QK9qYcpKTv5xHVojTWkYcZEtR/59ZLxSpyKAw0DCib16BEZQqy8PVDjl1mzM0otXf8lH5q1LSdUrAPIRNHL+LyL5LqDrO9k2blmZfBiANz4sYD/D6Th/O+b4jdEhQvseSzs7GaDdZjDpOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LgSDTGtxKVdLSomQ6SlUTGGVV+6O9JYMKbeZi8zhrW0=;
 b=MnJv2njMQfLi+zLJKw8G3uvB5sOkpZfiWShhShuUXJFjax+BofirBNl0SG1Zrdz2Z4m2XImNQrK9A+WFuMK74p+zgi6KqalJL+NkR9p+eXZGpvGWbHtMHJREkBT3Oryh6pQy9UslcWlgbkjRXmn1QzFVHCB87jab7g8++i7QpBk=
Date: Thu, 23 Sep 2021 15:22:50 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 4/9] x86/PVH: provide VGA console info to Dom0
Message-ID: <YUx/qrZRKCsz7WdR@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <215e2ea4-cb49-7d94-7f97-c6b81e522a60@suse.com>
 <YUtFLB+n2piR+cHe@MacBook-Air-de-Roger.local>
 <86c22215-923f-e8ce-e0a9-c76d90da4d77@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <86c22215-923f-e8ce-e0a9-c76d90da4d77@suse.com>
X-ClientProxiedBy: LO4P123CA0454.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1aa::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a96dc406-27d3-42b1-10e9-08d97e95410e
X-MS-TrafficTypeDiagnostic: DM5PR03MB2972:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB29721FCE1281D1AB5758037B8FA39@DM5PR03MB2972.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: NkgJu+J/4mSdlOqHeQzQ8P2urdREIh58uSTXvYuQVCXIBeKuBEFCa5nvllHqhnZWUaFouVP8k25FLLix7LIgG5d5WnCQE4obncWcL1i+KIQMVMTfh55tpXwLc8A7fMQkc15jvYBy2bEeW6WGT3ub1rYEHd1f0irTpMh9TGdUT/GwRcJOCSOqb9y4o2p0EtumaFLRWR3uj2urCz+fvumH3D4dOs/jsAfGt+QbEpRrvQIuVkktlWq2fHvzXhmI1gO4qw/5idnlJTweUzlTFTYBFkVIhhp07+jjmsRiIIhpVu5pLoDTQ3QT/lijRIR2BSzdGkW85r2Etqh9jNrWpGbEXFOrpIfAwqezGcx5ZnGeH6kxQvIVvAFzqR1an164MbxfOngSya1cwYAbiLwAwqylrzm3eBqkeh5jRGr84+P+1eg2gKpj2O+X027+LjAWnsqvs7Gn/ut+UJIOiaZupIis2vKliv1j46+h+qiu+EoR2KTqS04oPhZ+THnfskFaAxq5rkQaLMpCmpUFXnjDedqzJ3FI8F1bNfgKCDrLlxD/vb1/EbnbN7lIM0Fyq1KIzw6UZ/rxUCnyGVfOlv24+QZZnhPzXjPvRY2qM5oRYsIBzbWu+T4DXPDQeTk1aUp/H+r7cECXqvxDvX1Z877DqD75JiQ2Q4+s0Sz9FpC5NsUp3o0MvmiwxfV50UGpLZDztxTuB1hYOlnXbgIiUL/v+T68Vg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(9686003)(2906002)(83380400001)(956004)(6486002)(8676002)(85182001)(5660300002)(8936002)(6496006)(38100700002)(6666004)(26005)(316002)(54906003)(4326008)(53546011)(66946007)(508600001)(86362001)(186003)(6916009)(66556008)(66476007)(21314003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZVdrN0ZqTjBtY01uM2NNekNPV2RZdVNqV0RIYnNoZUg2KzR0REl5Sy9EaGlz?=
 =?utf-8?B?SmhLbmtWUlNSc05FWGtMRHlVTitYNHBycitGU1BvdzZ4MTR5eGRjNUZXaEhv?=
 =?utf-8?B?cFllWEN6VzhYSnlsRjM5ZEI5Y3BGVEcyOEtQYmUyL1RKWEQvc1EzVDdkMS9z?=
 =?utf-8?B?Z3hKY1UxRkFUQkE5dWd6enhncEZ6S2V1N1ZZand1MnMxOUE5ZWsxZ1BuUUJF?=
 =?utf-8?B?Wk1WZHNFRUx4eGdPc0VCOStid0hMTmlkcmxiYjdYeERkWE51RkttT3VGK0kv?=
 =?utf-8?B?YU9qbmluVytZbWIzbVB4blBtczhXVXJDVlBOdjY5ZVNueHFFeHBVN1Q5T1Jz?=
 =?utf-8?B?RXJhMWN4SUczTU1VRXRnVHh1Q1Y1cWpIOXp4OFZtbThVUDRRdDdEWmlCV2VP?=
 =?utf-8?B?Umc3V21PaGMxM3NnNUN5UXZ4YnBCVEZ3RWV2VGw4SmxSSkhvaFpXVjBEOGZn?=
 =?utf-8?B?Rjl6YTlKV05MMVBYckhneVVscTJRbmQ2VmJmNjV0TkloTE9hYmlaMHpLK2FC?=
 =?utf-8?B?SEU2Sm5FVmJlcUlsUUFmTllUa09sdU9yb3p6RDhxWDkxRmhhcXhqMnZTYzhW?=
 =?utf-8?B?NllxU3NzZzM3OUlibGVzVi9SRnhXeXJPV0RKSm9JNjZtVndmcGdlYlVEditV?=
 =?utf-8?B?TnZRTG1aNjhyQnR4QUwySm9DQVUzRVdQbVl0K00vTW9GVExWbEZycURjTW9i?=
 =?utf-8?B?Skg1NjRDQ3VjU3NIb0N3T3ZnQlo0eEd6K05WbXlWYUlUR0lwcHlxTXJnUTdN?=
 =?utf-8?B?VGJjdDJOcm1PQUR0R2RjSHFDYzRUSUJPQTlrRkZJSXZzZUxnUk5IUEdHSytG?=
 =?utf-8?B?WTg0UUZSUUNxTHlXZGtpOTVnanJGWHJPa1YwRGE3RExqUlRIM05NeFVxVHpV?=
 =?utf-8?B?c3BadXFZdDluTjRQeEZoUEdCNHdFSzdlNEJuV0dsWG5OZzh3NHV4VUFEVE5k?=
 =?utf-8?B?MG96NUJsZ0g4b2x3ZVRnbStQWEJZVTJHNnBwN1NTYXkra1V4c09vcTVDanho?=
 =?utf-8?B?MG1UMGErU2pHUnhnSHNRNnZWSzBOTXdnNjVieTdaK001Ly9yZmoxcEFpR3Vq?=
 =?utf-8?B?UXZrbW5neEVKWUlsM1BGeVBmaFY5ZW4rZ0EydS9XQktkVDUyaW43WmprejBp?=
 =?utf-8?B?Q2VUYTRyWk1QNnkzWWhNWDZDOWQ1bWlwdUpTK0JVQUlqbDNBTzNkbTRUNVRO?=
 =?utf-8?B?M0trTkFocmgxTEpaM1pqdHNCM1ZsV25yYnRhUXh3RXBkSGFjSksrK1ZlWEh6?=
 =?utf-8?B?RHJMQXVvZEhNZUFhclZUZUEwV3M3c3hwdTV5QzlXNWtQWllVTjdQaWpFYWEy?=
 =?utf-8?B?enlkajNHU0hwR3VjbHpMR0NuQlNKeFNoem1UQ1RhczNUcGZBZi9YMlh4b3lt?=
 =?utf-8?B?b1dTd1duQTFCTkJvRENWUUhPTTdKQS9GSnV5T0dueTIrd3lEblMwamxuUEsr?=
 =?utf-8?B?S0oyQ1BDMERFVC83RThqdW1SbUN0OTZvQklUL2w3bG5BU2JVV1JVdFNKSzd5?=
 =?utf-8?B?UkpqS05mSFBBRUt5N1E3VHJhbnA5aFl6cml0SHliUFVVUlVPNXIvWXBEVGlU?=
 =?utf-8?B?ejkwSFhHSzhNdVorVUlDZW1oTlM4Qk1NdGJCV1FkMmJUOENLb0NpeCtVN05Q?=
 =?utf-8?B?NFBSSGtISHRMZWg4ci9WRkNpUEtRcUVoSytPWDlEd0VzdmhjQ1hIMmJ5V3Mr?=
 =?utf-8?B?TVJSTXBpaUxFcXFPbi9wMnBsZldhYXB1aE9kTkQ4VFVHQVlCZ1RUK1BjN0Vl?=
 =?utf-8?Q?K1FZs3xBLg+6KY64l/VzCdNjNpEsj0rsQYhTVQn?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a96dc406-27d3-42b1-10e9-08d97e95410e
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 13:22:55.8055
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GYOA8IlcZ9gARhvW4SNTJo0omqSEmacaHc5WrlHd5pFnpwS/Addyw08h0sJ4a95sLiOtjUu35A/DHC5L7veGoA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2972
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 11:46:48AM +0200, Jan Beulich wrote:
> On 22.09.2021 17:01, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:18:05AM +0200, Jan Beulich wrote:
> >> --- a/xen/include/public/arch-x86/hvm/start_info.h
> >> +++ b/xen/include/public/arch-x86/hvm/start_info.h
> >> @@ -33,7 +33,7 @@
> >>   *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_VALUE
> >>   *    |                | ("xEn3" with the 0x80 bit of the "E" set).
> >>   *  4 +----------------+
> >> - *    | version        | Version of this structure. Current version is 1. New
> >> + *    | version        | Version of this structure. Current version is 2. New
> >>   *    |                | versions are guaranteed to be backwards-compatible.
> >>   *  8 +----------------+
> >>   *    | flags          | SIF_xxx flags.
> >> @@ -55,7 +55,15 @@
> >>   *    |                | if there is no memory map being provided. Only
> >>   *    |                | present in version 1 and newer of the structure.
> >>   * 52 +----------------+
> >> - *    | reserved       | Version 1 and newer only.
> >> + *    | vga_info.offset| Offset of struct dom0_vga_console_info from base of
> > 
> > I'm not sure we are supposed to reference external structures like
> > that. We took a lot of care to not depend on other headers, and to
> > make this as agnostic as possible (IIRC KVM is also capable of using
> > the PVH entry point natively, and hence depends on this header).
> 
> But KVM wouldn't be using a Dom0-only part of the interface, would
> it? (I'm aware of the possible re-using of the entry point.)

I think naming it as dom0-only is part of the problem. In principle I
don't see why you couldn't use such structure to report VGA console
information when inside of a VM that has such device.

> > IF we want to add support for dom0_vga_console_info I think we need to
> > at least provide a binary layout for it, like all the other pieces
> > that are part of the HVM start info.
> 
> Which then means we can't sensibly re-use the existing structure,
> as that doesn't have as strict rules as the hvm_start_info one.
> Which in turn means Linux can't re-use the code converting
> dom0_vga_console_info, resulting in two places needing updating
> whenever information gets add to (then) both structures (what
> information they carry will, after all, want to remain in sync).

Indeed. I don't think there's a way for us to cut corners here. As
said above, I don't think we should explicitly limit the usage of this
information to Xen dom0, and hence should be a first class addition to
the start info contents.

> >> + *    |                | struct hvm_start_info. Optional and only present in
> >> + *    |                | version 2 and newer of the structure when
> >> + *    |                | SIF_INITDOMAIN is set; zero if absent.
> > 
> > We have usually used an absolute physical address to reference other
> > data, and I think we should likely keep in that way for coherency.
> 
> Hmm. (See below.)
> 
> >> + * 54 +----------------+
> >> + *    | vga_info.size  | Size of present parts of struct dom0_vga_console_info.
> >> + *    |                | Optional and only present in version 2 and newer of
> >> + *    |                | the structure when SIF_INITDOMAIN is set; zero if
> >> + *    |                | absent.
> >>   * 56 +----------------+
> >>   *
> >>   * The layout of each entry in the module structure is the following:
> >> @@ -139,7 +147,15 @@ struct hvm_start_info {
> >>      uint32_t memmap_entries;    /* Number of entries in the memmap table.    */
> >>                                  /* Value will be zero if there is no memory  */
> >>                                  /* map being provided.                       */
> >> -    uint32_t reserved;          /* Must be zero.                             */
> > 
> > This 'Must be zero' comment troubles me a bit, I'm not convinced we
> > can just place data here (ie: it would no longer be 0). It's even
> > worse because the must be zero comment is only present in the C
> > representation of the struct, the layout above just denotes the field
> > is reserved (which would imply it's fine to use for other means in
> > v2).
> 
> I thought the textual description was meant to be the ABI spec. The C
> comment should therefore be viewed as if missing "in version 1" or
> "presently".
> 
> Taking into account also Andrew's reply, I have to admit that I'm
> inclined to request that one of the two of you fix this obvious
> shortcoming in both Xen and Linux. I'm not really willing to be the one
> to introduce a 2nd layout for the same set of data just for the purpose
> of "playing nice" in an area where that, affecting Dom0 only, doesn't
> seem to matter all this much. My goal was rather to keep the impact on
> hvm_start_info as low as possible (and in particular avoid changing its
> size, as strictly speaking Linux'es consumer implementation is buggy:
> It would always copy as much data as it knows _may_ be present, not as
> little data as may have been _actually_ provided; whoever implemented
> this did only consider one half of the compatibility requirements,
> quite likely simply because in the design this aspect was also missed,
> or else the structure would have had a length field right from its
> introduction).

The isn't a strict need to have several different layouts on the
consumer, as you could easily use offsetof to copy up to the boundary
you know it's populated given a version number. But yes, a size field
would be useful to prevent the consumer from having to know the
version boundaries.

I can try to take a stab at implementing this at a later point, I
certainly don't want to force you into implementing something you
believe it's not the correct solution.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:47:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:47:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194262.346052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTP43-0002gs-VJ; Thu, 23 Sep 2021 13:47:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194262.346052; Thu, 23 Sep 2021 13:47:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTP43-0002gl-S0; Thu, 23 Sep 2021 13:47:03 +0000
Received: by outflank-mailman (input) for mailman id 194262;
 Thu, 23 Sep 2021 13:47:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ZpLY=ON=anisinha.ca=ani@srs-us1.protection.inumbo.net>)
 id 1mTP42-0002gf-2e
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:47:02 +0000
Received: from mail-ed1-x52d.google.com (unknown [2a00:1450:4864:20::52d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9cbeecaa-b1b2-4cdd-ae9a-8cc795971c54;
 Thu, 23 Sep 2021 13:47:01 +0000 (UTC)
Received: by mail-ed1-x52d.google.com with SMTP id c21so23596735edj.0
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 06:47:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cbeecaa-b1b2-4cdd-ae9a-8cc795971c54
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=anisinha-ca.20210112.gappssmtp.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=X6tPYj9oUhcOZ+uoKP+nG8NnVQLmoHX/d9g9BCvLsR4=;
        b=hhQ6niECsnZzrG0mdkN3l2WljsCA50VtPKnZq/g/2/G/ZURvGbApaDR4yM3Qu7Nedr
         yCbU2ae+DEh22mAwEsYAFVeHCmi2bWr7WdChJM2pKepbtPDure7P44nCJZ2ISY4/vdNs
         bq8yhJ5dTW5mGd3AfNM1D7FzOCp7GGd6zmvX2Sac656+re+366zG2dB+NClxqXqRq0xh
         htyslKOh7ULkt6Pot837zj5o9iN+TyPQOYd2F9w9eoRyHdRezVHYpRhoZ3Njl2piiKKY
         XjR+oCA6huAJMYDWQDZUBthnwD5zHMYYY3ylJSOJ4a0uQ9yJ3GvdPxnjKj4UhEWK172g
         MV7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=X6tPYj9oUhcOZ+uoKP+nG8NnVQLmoHX/d9g9BCvLsR4=;
        b=wHcfr42cb1QDILBnQo7IrKLqqxLyzml/JDvCP2tZwCEKdXoulwVuusYrqNumDiH3XE
         X7AeX9/VrNaA9Di1l9T8h87jx1FEJ3/x5a2xEGtvKZxn8hSBkgFHUmyS11ajII7JYV/R
         L3xKzV9gFUtvRSrVCP7ixMzsit5qgifTfOItVNyPKeybb/LkV9J4bKRiDNQHngSKRhle
         9qkTNtJujl+qxiRasx1QpSeGwjt0fPf3K0m27F/DMwRQeElnbavv0MiQmUO+vkIFZ0vZ
         kLwzr6tqVOCw/bGzhjmOWOGKDs8CYt2IluVVvb2HtaYMX+CmwqsLO8nlRjzKcFlnMTTs
         Rdfw==
X-Gm-Message-State: AOAM533DyQy4aR0kFHpnOIZaON23KCOxb1yNjxdFHHChkCPwDjaetTDy
	9vwzFKMXgerjr0YexO13YMHkh9xrJxvc9ZfT83tsOg==
X-Google-Smtp-Source: ABdhPJwICFJ0kqduDlOdG2Bm7hRT1CYmYq3zG89jCtiaURODn4W0xVzTn8I3igE6k1ZKJtFQaXUt4Jr9El0Yfcd7sdE=
X-Received: by 2002:a05:6402:3128:: with SMTP id dd8mr5448390edb.383.1632404816373;
 Thu, 23 Sep 2021 06:46:56 -0700 (PDT)
MIME-Version: 1.0
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-3-damien.hedde@greensocs.com> <d27c5829-e28f-4d17-966e-6cb86ebe00e6@redhat.com>
 <e3ab341b-3e7a-1485-90ed-dbd9d2e3588d@greensocs.com>
In-Reply-To: <e3ab341b-3e7a-1485-90ed-dbd9d2e3588d@greensocs.com>
From: Ani Sinha <ani@anisinha.ca>
Date: Thu, 23 Sep 2021 19:16:45 +0530
Message-ID: <CAARzgww_dxXLhJTm1jeWzbQxVGzT6YNxZi+uTrRDnBZ9Z1v+QA@mail.gmail.com>
Subject: Re: [RFC PATCH v2 02/16] qapi: Implement query-machine-phase QMP command
To: Damien Hedde <damien.hedde@greensocs.com>
Cc: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@redhat.com>, 
	QEMU Developers <qemu-devel@nongnu.org>, Peter Maydell <peter.maydell@linaro.org>, 
	"Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>, Peter Xu <peterx@redhat.com>, 
	mirela.grujic@greensocs.com, Alistair Francis <Alistair.Francis@wdc.com>, 
	Gerd Hoffmann <kraxel@redhat.com>, Eric Blake <eblake@redhat.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org, 
	Paul Durrant <paul@xen.org>, Markus Armbruster <armbru@redhat.com>, 
	Anthony Perard <anthony.perard@citrix.com>, 
	=?UTF-8?B?TWFyYy1BbmRyw6kgTHVyZWF1?= <marcandre.lureau@redhat.com>, 
	Eduardo Habkost <ehabkost@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, 
	Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, qemu-riscv@nongnu.org, 
	=?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= <berrange@redhat.com>, 
	mark.burton@greensocs.com, edgari@xilinx.com, 
	Igor Mammedov <imammedo@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Sep 23, 2021 at 6:13 PM Damien Hedde <damien.hedde@greensocs.com> w=
rote:
>
>
>
> On 9/22/21 19:37, Philippe Mathieu-Daud=C3=A9 wrote:
> > On 9/22/21 18:13, Damien Hedde wrote:
> >> From: Mirela Grujic <mirela.grujic@greensocs.com>
> >>
> >> The command returns current machine initialization phase.
> >>  From now on, the MachineInitPhase enum is generated from the
> >> QAPI schema.
> >>
> >> Signed-off-by: Mirela Grujic <mirela.grujic@greensocs.com>
> >
> > Missing your S-o-b, otherwise:
>
> Sorry. I did not realize I had to add it since it was already done by
> Mirela. I'll add it for this patch and patches 1, 6 and 8.

https://wiki.qemu.org/Contribute/SubmitAPatch#Patch_emails_must_include_a_S=
igned-off-by:_line
btw, it's strictly not mandatory if you are both from the same company
and you did not write the patch or contribute in anyway to it.

>
> > Reviewed-by: Philippe Mathieu-Daud=C3=A9 <philmd@redhat.com >
> >> ---
> >>   qapi/machine.json          | 56 ++++++++++++++++++++++++++++++++++++=
++
> >>   include/hw/qdev-core.h     | 30 ++------------------
> >>   hw/core/machine-qmp-cmds.c |  9 ++++++
> >>   hw/core/qdev.c             |  5 ++++
> >>   4 files changed, 72 insertions(+), 28 deletions(-)
> >


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 13:55:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 13:55:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194269.346063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPBd-0004Aj-RR; Thu, 23 Sep 2021 13:54:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194269.346063; Thu, 23 Sep 2021 13:54:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPBd-0004Ac-OH; Thu, 23 Sep 2021 13:54:53 +0000
Received: by outflank-mailman (input) for mailman id 194269;
 Thu, 23 Sep 2021 13:54:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lEPm=ON=csgroup.eu=christophe.leroy@srs-us1.protection.inumbo.net>)
 id 1mTPBc-0004AW-Et
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 13:54:52 +0000
Received: from pegase2.c-s.fr (unknown [93.17.235.10])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7945a4b4-2d5c-49ef-9317-7be62a01ecc2;
 Thu, 23 Sep 2021 13:54:51 +0000 (UTC)
Received: from localhost (mailhub3.si.c-s.fr [172.26.127.67])
 by localhost (Postfix) with ESMTP id 4HFc9P5sHPz9sV7;
 Thu, 23 Sep 2021 15:54:49 +0200 (CEST)
Received: from pegase2.c-s.fr ([172.26.127.65])
 by localhost (pegase2.c-s.fr [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id xdm1Ouzf8XXu; Thu, 23 Sep 2021 15:54:49 +0200 (CEST)
Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192])
 by pegase2.c-s.fr (Postfix) with ESMTP id 4HFc9P4lm1z9sV4;
 Thu, 23 Sep 2021 15:54:49 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by messagerie.si.c-s.fr (Postfix) with ESMTP id 8BA938B776;
 Thu, 23 Sep 2021 15:54:49 +0200 (CEST)
Received: from messagerie.si.c-s.fr ([127.0.0.1])
 by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023)
 with ESMTP id i24VgXHaAF2s; Thu, 23 Sep 2021 15:54:49 +0200 (CEST)
Received: from PO20335.IDSI0.si.c-s.fr (unknown [192.168.202.200])
 by messagerie.si.c-s.fr (Postfix) with ESMTP id ACF658B763;
 Thu, 23 Sep 2021 15:54:47 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7945a4b4-2d5c-49ef-9317-7be62a01ecc2
X-Virus-Scanned: amavisd-new at c-s.fr
X-Virus-Scanned: amavisd-new at c-s.fr
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
To: Mike Rapoport <rppt@linux.ibm.com>
Cc: Mike Rapoport <rppt@kernel.org>,
 Linus Torvalds <torvalds@linux-foundation.org>, devicetree@vger.kernel.org,
 linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org,
 linux-sh@vger.kernel.org, linux-um@lists.infradead.org,
 linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com,
 linux-mips@vger.kernel.org, linux-mm@kvack.org,
 iommu@lists.linux-foundation.org, linux-usb@vger.kernel.org,
 linux-alpha@vger.kernel.org, sparclinux@vger.kernel.org,
 xen-devel@lists.xenproject.org, Andrew Morton <akpm@linux-foundation.org>,
 linux-snps-arc@lists.infradead.org, linuxppc-dev@lists.ozlabs.org,
 linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
 <1101e3c7-fcb7-a632-8e22-47f4a01ea02e@csgroup.eu>
 <YUxsgN/uolhn1Ok+@linux.ibm.com>
From: Christophe Leroy <christophe.leroy@csgroup.eu>
Message-ID: <96e3da9f-70ff-e5c0-ef2e-cf0b636e5695@csgroup.eu>
Date: Thu, 23 Sep 2021 15:54:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <YUxsgN/uolhn1Ok+@linux.ibm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: fr-FR
Content-Transfer-Encoding: 8bit



Le 23/09/2021 à 14:01, Mike Rapoport a écrit :
> On Thu, Sep 23, 2021 at 11:47:48AM +0200, Christophe Leroy wrote:
>>
>>
>> Le 23/09/2021 à 09:43, Mike Rapoport a écrit :
>>> From: Mike Rapoport <rppt@linux.ibm.com>
>>>
>>> For ages memblock_free() interface dealt with physical addresses even
>>> despite the existence of memblock_alloc_xx() functions that return a
>>> virtual pointer.
>>>
>>> Introduce memblock_phys_free() for freeing physical ranges and repurpose
>>> memblock_free() to free virtual pointers to make the following pairing
>>> abundantly clear:
>>>
>>> 	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
>>> 	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);
>>>
>>> 	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
>>> 	void memblock_free(void *ptr, size_t size);
>>>
>>> Replace intermediate memblock_free_ptr() with memblock_free() and drop
>>> unnecessary aliases memblock_free_early() and memblock_free_early_nid().
>>>
>>> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
>>> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
>>> ---
>>
>>> diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
>>> index 1a04e5bdf655..37826d8c4f74 100644
>>> --- a/arch/s390/kernel/smp.c
>>> +++ b/arch/s390/kernel/smp.c
>>> @@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
>>>    			/* Get the CPU registers */
>>>    			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
>>>    	}
>>> -	memblock_free(page, PAGE_SIZE);
>>> +	memblock_phys_free(page, PAGE_SIZE);
>>>    	diag_amode31_ops.diag308_reset();
>>>    	pcpu_set_smt(0);
>>>    }
>>> @@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
>>>    	/* Add CPUs present at boot */
>>>    	__smp_rescan_cpus(info, true);
>>> -	memblock_free_early((unsigned long)info, sizeof(*info));
>>> +	memblock_free(info, sizeof(*info));
>>>    }
>>>    /*
>>
>> I'm a bit lost. IIUC memblock_free_early() and memblock_free() where
>> identical.
> 
> Yes, they were, but all calls to memblock_free_early() were using
> __pa(vaddr) because they had a virtual address at hand.

I'm still not following. In the above memblock_free_early() was taking 
(unsigned long)info . Was it a bug ? It looks odd to hide bug fixes in 
such a big patch, should that bug fix go in patch 2 ?

> 
>> In the first hunk memblock_free() gets replaced by memblock_phys_free()
>> In the second hunk memblock_free_early() gets replaced by memblock_free()
> 
> In the first hunk the memory is allocated with memblock_phys_alloc() and we
> have a physical range to free. In the second hunk the memory is allocated
> with memblock_alloc() and we are freeing a virtual pointer.
>   
>> I think it would be easier to follow if you could split it in several
>> patches:
> 
> It was an explicit request from Linus to make it a single commit:
> 
>    but the actual commit can and should be just a single commit that just
>    fixes 'memblock_free()' to have sane interfaces.
> 
> I don't feel strongly about splitting it (except my laziness really
> objects), but I don't think doing the conversion in several steps worth the
> churn.

The commit is quite big (55 files changed, approx 100 lines modified).

If done in the right order the change should be minimal.

It is rather not-easy to follow and review when a function that was 
existing (namely memblock_free() ) disappears and re-appears in the same 
commit but to do something different.

You do:
- memblock_free() ==> memblock_phys_free()
- memblock_free_ptr() ==> memblock_free()

At least you could split in two patches, the advantage would be that 
between first and second patch memblock() doesn't exist anymore so you 
can check you really don't have anymore user.

> 
>> - First patch: Create memblock_phys_free() and change all relevant
>> memblock_free() to memblock_phys_free() - Or change memblock_free() to
>> memblock_phys_free() and make memblock_free() an alias of it.
>> - Second patch: Make memblock_free_ptr() become memblock_free() and change
>> all remaining callers to the new semantics (IIUC memblock_free(__pa(ptr))
>> becomes memblock_free(ptr) and make memblock_free_ptr() an alias of
>> memblock_free()
>> - Fourth patch: Replace and drop memblock_free_ptr()
>> - Fifth patch: Drop memblock_free_early() and memblock_free_early_nid() (All
>> users should have been upgraded to memblock_free_phys() in patch 1 or
>> memblock_free() in patch 2)
>>
>> Christophe
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:04:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:04:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194274.346073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPKO-0005xh-O8; Thu, 23 Sep 2021 14:03:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194274.346073; Thu, 23 Sep 2021 14:03:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPKO-0005xa-L9; Thu, 23 Sep 2021 14:03:56 +0000
Received: by outflank-mailman (input) for mailman id 194274;
 Thu, 23 Sep 2021 14:03:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTPKN-0005xU-6L
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:03:55 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 15e1d750-1c77-11ec-ba42-12813bfff9fa;
 Thu, 23 Sep 2021 14:03:54 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 4B5902028E;
 Thu, 23 Sep 2021 14:03:53 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1A93213E6E;
 Thu, 23 Sep 2021 14:03:53 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id VgYoBUmJTGFxYQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:03:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15e1d750-1c77-11ec-ba42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632405833; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=QUATB7dh8yhUtunA0ZRT/TKN770jgGXBPhfsgdJXo8Y=;
	b=e+xwDbF+CEDtVLqKleQzk3ItHPuq+fr0YgpyKZ0ELyOciPMU0CjE/FfF/Q/KMniBxUi+rB
	3k4Cl2r99yv18OAJ7GMuKsBubSJltHMnejsUrg7VnpsJ9xml3658mCRJOzeCLVUUO1iRa8
	ewZBlkPOJs1rru9Hh6lONXTd1ceM81c=
Subject: Re: [PATCH 2/9] xen/x86: allow PVH Dom0 without XEN_PV=y
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Konrad Wilk <konrad.wilk@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <2254771e-68ff-e190-15b4-42ed92239712@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <8fd63619-811b-b8ca-1655-513acb3f90c2@suse.com>
Date: Thu, 23 Sep 2021 16:03:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <2254771e-68ff-e190-15b4-42ed92239712@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="0N4u14VvQrO5xlgGNKQpMtZopwsEtCxTk"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--0N4u14VvQrO5xlgGNKQpMtZopwsEtCxTk
Content-Type: multipart/mixed; boundary="QKVjVgKOLQd4fxCGqtwCbm5ABiaLzF0fU";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Konrad Wilk <konrad.wilk@oracle.com>
Message-ID: <8fd63619-811b-b8ca-1655-513acb3f90c2@suse.com>
Subject: Re: [PATCH 2/9] xen/x86: allow PVH Dom0 without XEN_PV=y
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <2254771e-68ff-e190-15b4-42ed92239712@suse.com>
In-Reply-To: <2254771e-68ff-e190-15b4-42ed92239712@suse.com>

--QKVjVgKOLQd4fxCGqtwCbm5ABiaLzF0fU
Content-Type: multipart/mixed;
 boundary="------------F02F650270AB2DDBFBB24402"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------F02F650270AB2DDBFBB24402
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:08, Jan Beulich wrote:
> Decouple XEN_DOM0 from XEN_PV, converting some existing uses of XEN_DOM=
0
> to a new XEN_PV_DOM0. (I'm not convinced all are really / should really=

> be PV-specific, but for starters I've tried to be conservative.)
>=20
> For PVH Dom0 the hypervisor populates MADT with only x2APIC entries, so=

> without x2APIC support enabled in the kernel things aren't going to wor=
k
> very well. (As opposed, DomU-s would only ever see LAPIC entries in MAD=
T
> as of now.) Note that this then requires PVH Dom0 to be 64-bit, as
> X86_X2APIC depends on X86_64.
>=20
> In the course of this xen_running_on_version_or_later() needs to be
> available more broadly. Move it from a PV-specific to a generic file,
> considering that what it does isn't really PV-specific at all anyway.
>=20
> Note that xen/interface/version.h cannot be included on its own; in
> enlighten.c, which uses SCHEDOP_* anyway, include xen/interface/sched.h=

> first to resolve the apparently sole missing type (xen_ulong_t).
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------F02F650270AB2DDBFBB24402
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------F02F650270AB2DDBFBB24402--

--QKVjVgKOLQd4fxCGqtwCbm5ABiaLzF0fU--

--0N4u14VvQrO5xlgGNKQpMtZopwsEtCxTk
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMiUgFAwAAAAAACgkQsN6d1ii/Ey+O
wwf/RqAsLsAEvROhrKoCH5PbWgbUtzCKiVJTeZM4ZLOP1+qyuub7jS8eud2KAZfk530l4mQsDUUH
Lzw5Y6u18N87tIx13Z4XfHDEMV2VOu5NTGii7pFhGR7r8WmebTRISzZ0nMAmKABoXzRyeDwpG6qS
BjgcwpISwkej6x7YbeiCgVNm5XkcUdsnlgO9QwiDtLkBtAbIVbHWJMUq1duqsUnPbtg2f5lZL/Hh
DNTCNvcHlBDTab5GGJ5c8H6P4N8oZWGrIcNCbxorAjj2CTnqiGeS8O5FbzUY5FNfHb5O3O1fJeiS
Vu3YIo4xmutol4xrF8F45Td6PhP1QbvuRiSyLLv7vw==
=0xVM
-----END PGP SIGNATURE-----

--0N4u14VvQrO5xlgGNKQpMtZopwsEtCxTk--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:04:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:04:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194279.346084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPLI-0006W2-1e; Thu, 23 Sep 2021 14:04:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194279.346084; Thu, 23 Sep 2021 14:04:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPLH-0006Vv-Uz; Thu, 23 Sep 2021 14:04:51 +0000
Received: by outflank-mailman (input) for mailman id 194279;
 Thu, 23 Sep 2021 14:04:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fO2a=ON=greensocs.com=damien.hedde@srs-us1.protection.inumbo.net>)
 id 1mTPLG-0006Vj-Kj
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:04:51 +0000
Received: from beetle.greensocs.com (unknown [5.135.226.135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 69583e72-dbba-46a7-aec2-092201a0bc3b;
 Thu, 23 Sep 2021 14:04:48 +0000 (UTC)
Received: from [192.168.15.189] (unknown [195.68.53.70])
 by beetle.greensocs.com (Postfix) with ESMTPSA id D292D20786;
 Thu, 23 Sep 2021 14:04:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69583e72-dbba-46a7-aec2-092201a0bc3b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greensocs.com;
	s=mail; t=1632405886;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UkbynlnCvJheMvgy5KHKY+xz6gcpk1nQXV5B9mUjOns=;
	b=JWNmS1MG18oEUFGyOIkmMRUIIqV+LMdU6sw4WjnCt8zL2ND+/STJkQcO4LdS33T+zi+The
	pA9rPdNQYqWx0fnxDCONrjEK8TrHMHIt18UppwpNxwrp/CIE//fy8rM3L8EFzSG3DVonuE
	xq1U1TRyGPL0wijjmNi/1ar9ll9PUxk=
Message-ID: <b14de602-9a80-cb44-9ae0-5a39a9dde5cd@greensocs.com>
Date: Thu, 23 Sep 2021 16:04:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [RFC PATCH v2 10/16] qdev-monitor: allow adding any sysbus device
 before machine is ready
Content-Language: en-US-large
To: Ani Sinha <ani@anisinha.ca>
Cc: Peter Maydell <peter.maydell@linaro.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>,
 qemu-devel@nongnu.org, Peter Xu <peterx@redhat.com>,
 mirela.grujic@greensocs.com, Alistair Francis <Alistair.Francis@wdc.com>,
 Gerd Hoffmann <kraxel@redhat.com>, Eric Blake <eblake@redhat.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org,
 Paul Durrant <paul@xen.org>, Markus Armbruster <armbru@redhat.com>,
 Anthony Perard <anthony.perard@citrix.com>,
 =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= <marcandre.lureau@redhat.com>,
 Eduardo Habkost <ehabkost@redhat.com>,
 "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
 Eric Auger <eric.auger@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>,
 qemu-riscv@nongnu.org, =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?=
 <berrange@redhat.com>, mark.burton@greensocs.com, edgari@xilinx.com,
 Igor Mammedov <imammedo@redhat.com>
References: <20210922161405.140018-1-damien.hedde@greensocs.com>
 <20210922161405.140018-11-damien.hedde@greensocs.com>
 <alpine.DEB.2.22.394.2109231628280.630@anisinha-lenovo>
 <alpine.DEB.2.22.394.2109231723060.630@anisinha-lenovo>
From: Damien Hedde <damien.hedde@greensocs.com>
In-Reply-To: <alpine.DEB.2.22.394.2109231723060.630@anisinha-lenovo>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 9/23/21 13:55, Ani Sinha wrote:
> 
> 
> On Thu, 23 Sep 2021, Ani Sinha wrote:
> 
>>
>>
>> On Wed, 22 Sep 2021, Damien Hedde wrote:
>>
>>> Skip the sysbus device type per-machine allow-list check before the
>>> MACHINE_INIT_PHASE_READY phase.
>>>
>>> This patch permits adding any sysbus device (it still needs to be
>>> user_creatable) when using the -preconfig experimental option.
>>>
>>> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
>>> ---
>>>
>>> This commit is RFC. Depending on the condition to allow a device
>>> to be added, it may change.
>>> ---
>>>   softmmu/qdev-monitor.c | 9 +++++++--
>>>   1 file changed, 7 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
>>> index f1c9242855..73b991adda 100644
>>> --- a/softmmu/qdev-monitor.c
>>> +++ b/softmmu/qdev-monitor.c
>>> @@ -269,8 +269,13 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
>>>           return NULL;
>>>       }
>>>
>>> -    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
>>> -        /* sysbus devices need to be allowed by the machine */
>>> +    if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE) &&
>>> +        phase_check(MACHINE_INIT_PHASE_READY)) {
>>> +        /*
>>> +         * Sysbus devices need to be allowed by the machine.
>>> +         * We only check that after the machine is ready in order to let
>>> +         * us add any user_creatable sysbus device during machine creation.
>>> +         */
>>>           MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
>>>           if (!machine_class_is_dynamic_sysbus_dev_allowed(mc, *driver)) {
>>>               error_setg(errp, "'%s' is not an allowed pluggable sysbus device "
>>
>> Since now you are adding the state of the machine creation in the
>> valiation condition, the failure error message becomes misleading.
>> Better to do this I think :
>>
>> if (object class is TYPE_SYS_BUS_DEVICE)
>> {
>>    if (!phase_check(MACHINE_INIT_PHASE_READY))
>>      {
>>        // error out here saying the state of the machine creation is too
>> early
>>      }
>>
>>      // do the other check on dynamic sysbus
>>
>> }
> 
> The other thing to consider is whether we should put the machine phaze
> check at a higher level, at qdev_device_add() perhaps. One might envison
> that different device types may be allowed to be added at different
> stages of machine creation. So this check needs be more generalized for
> the future.
> 

Hi Ani,

I think moving out the allowance check from qdev_get_device_class is a 
good idea. The code will be more clear even if the check is not generalized.

Thanks,
--
Damien



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:05:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:05:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194284.346095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPMM-00077q-BW; Thu, 23 Sep 2021 14:05:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194284.346095; Thu, 23 Sep 2021 14:05:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPMM-00077j-8c; Thu, 23 Sep 2021 14:05:58 +0000
Received: by outflank-mailman (input) for mailman id 194284;
 Thu, 23 Sep 2021 14:05:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTPML-00077b-0l
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:05:57 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5e9b28d4-1c77-11ec-ba46-12813bfff9fa;
 Thu, 23 Sep 2021 14:05:56 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 539212028E;
 Thu, 23 Sep 2021 14:05:55 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1D25413E6E;
 Thu, 23 Sep 2021 14:05:55 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id qTWpBcOJTGGiYgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:05:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e9b28d4-1c77-11ec-ba46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632405955; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=OfqaEdiK4gjrAA60PKFlDfHHLNeDMBRrILyWc6i5XnU=;
	b=Dizzq+EMM16+adE5r4UyJttEOvOUBrQFoKsP90AYyrhnqY+PPMvD5FnNfzJcAKekO6iTOo
	+dzxW6pUaEtugDZhTzAErX7ewmP0NkiBeIvwb4LuBEs8JUhu0j/hDyn+bjqfb0UJtETGcp
	oKWW9SHEUC35Bud51kF7E5F4b+/jlMQ=
Subject: Re: [PATCH 3/9] xen/x86: make "earlyprintk=xen" work better for PVH
 Dom0
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <ecf17c7b-09a4-29a7-6951-1e0b0dda3c67@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <5d6f47fd-67e9-4095-7db2-4609ab3c820a@suse.com>
Date: Thu, 23 Sep 2021 16:05:54 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <ecf17c7b-09a4-29a7-6951-1e0b0dda3c67@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="7Sn9KCmfryNgAdLGXxcEYApc3mfViOsla"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--7Sn9KCmfryNgAdLGXxcEYApc3mfViOsla
Content-Type: multipart/mixed; boundary="lrYWW5MdoNS1FyvqW63Bx40DFIeBWCYI3";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
Message-ID: <5d6f47fd-67e9-4095-7db2-4609ab3c820a@suse.com>
Subject: Re: [PATCH 3/9] xen/x86: make "earlyprintk=xen" work better for PVH
 Dom0
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <ecf17c7b-09a4-29a7-6951-1e0b0dda3c67@suse.com>
In-Reply-To: <ecf17c7b-09a4-29a7-6951-1e0b0dda3c67@suse.com>

--lrYWW5MdoNS1FyvqW63Bx40DFIeBWCYI3
Content-Type: multipart/mixed;
 boundary="------------BF192ED46C94EEA4CF27F1DB"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------BF192ED46C94EEA4CF27F1DB
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:09, Jan Beulich wrote:
> The xen_hvm_early_write() path better wouldn't be taken in this case;
> while port 0xE9 can be used, the hypercall path is quite a bit more
> efficient. Put that first, as it may also work for DomU-s (see also
> xen_raw_console_write()).
>=20
> While there also bail from the function when the first
> domU_write_console() failed - later ones aren't going to succeed.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------BF192ED46C94EEA4CF27F1DB
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------BF192ED46C94EEA4CF27F1DB--

--lrYWW5MdoNS1FyvqW63Bx40DFIeBWCYI3--

--7Sn9KCmfryNgAdLGXxcEYApc3mfViOsla
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMicIFAwAAAAAACgkQsN6d1ii/Ey9h
mQf8DI71q9ktz1P7PxBLbYY+ozb8GDucPgA525474ajhxRAup6NGg4M2Uq+SnYH1/tXYfDOAO+Q+
AmF3D9YJxjqMal31LSR7S9RDiIf6TCFvk2Py5s6tW4XC1HlTkS2Yrn0WZTiwu98gqSPsjcllqElP
PGPqwH0WjCr4wyv0M0+1D+vwtRZvUYRV/OcYBaCT0st0u6e0J6s6AS4EdSWKr0kiz+IpN+swffJu
ZsowOdJKrL8VN3WmzahbgY7iF1x5iS1Tcyey2WR6tH5nVIPJjMVODbqCb0JiamrIs3zCzpynEgRa
lqMBnnnKRuIkgIZolKmwNEfGQlRl3wLBEypbideAkg==
=E+MT
-----END PGP SIGNATURE-----

--7Sn9KCmfryNgAdLGXxcEYApc3mfViOsla--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:07:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:07:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194291.346106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPNN-0007zs-Pn; Thu, 23 Sep 2021 14:07:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194291.346106; Thu, 23 Sep 2021 14:07:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPNN-0007zl-Me; Thu, 23 Sep 2021 14:07:01 +0000
Received: by outflank-mailman (input) for mailman id 194291;
 Thu, 23 Sep 2021 14:07:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTPNM-0007r1-AS
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:07:00 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ac1deccb-e417-4c32-b9b0-0b5132bba7a5;
 Thu, 23 Sep 2021 14:06:59 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id AEE201FFB0;
 Thu, 23 Sep 2021 14:06:58 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6878313CD1;
 Thu, 23 Sep 2021 14:06:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id cAhJFgKKTGE9YwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:06:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac1deccb-e417-4c32-b9b0-0b5132bba7a5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632406018; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=WEHW5U42S9LvAJ008oareKv+HLyxxMprASlowcERJ4Q=;
	b=MB5oLnSY75MSxT/AWURooj8YC4ns4jixV3PM+LTOw3RdYb2vL+spx8Gf2B3YFtLW5FsSiP
	vSQdhe6myA3X8FZ5hgu2U7637aTD7Sg5lcn56ucZohgf6LFcjdYcD9+nW/g6eDdUZYRZ7l
	j+CIFNJ8PQh07MitNrQcewIXf+5yAhs=
Subject: Re: [PATCH 4/9] xen/x86: allow "earlyprintk=xen" to work for PV Dom0
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <bbf5f984-d039-97b2-cfa2-23a8a7afcbed@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <baf50925-4544-9ad9-a9d9-0175b7892f85@suse.com>
Date: Thu, 23 Sep 2021 16:06:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <bbf5f984-d039-97b2-cfa2-23a8a7afcbed@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ktVs5VwmmwDITJIdmbiOkFUQTz1ew9xRE"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ktVs5VwmmwDITJIdmbiOkFUQTz1ew9xRE
Content-Type: multipart/mixed; boundary="0CayfOTegXtxL52X91EenenGUNtl6A3mc";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <baf50925-4544-9ad9-a9d9-0175b7892f85@suse.com>
Subject: Re: [PATCH 4/9] xen/x86: allow "earlyprintk=xen" to work for PV Dom0
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <bbf5f984-d039-97b2-cfa2-23a8a7afcbed@suse.com>
In-Reply-To: <bbf5f984-d039-97b2-cfa2-23a8a7afcbed@suse.com>

--0CayfOTegXtxL52X91EenenGUNtl6A3mc
Content-Type: multipart/mixed;
 boundary="------------8BEE0A06B93E087CC496CDE3"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8BEE0A06B93E087CC496CDE3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:09, Jan Beulich wrote:
> With preferred consoles "tty" and "hvc" announced as preferred,
> registering "xenboot" early won't result in use of the console: It also=

> needs to be registered as preferred. Generalize this from being DomU-
> only so far.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------8BEE0A06B93E087CC496CDE3
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8BEE0A06B93E087CC496CDE3--

--0CayfOTegXtxL52X91EenenGUNtl6A3mc--

--ktVs5VwmmwDITJIdmbiOkFUQTz1ew9xRE
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMigAFAwAAAAAACgkQsN6d1ii/Ey/Y
tAf/S//XmkyBpyAc7uYh1Nry45qvDJ2pAiZQqrwOw6spsNerXjXeodIbeP5jJiqRj2C+KvC6aWNM
t5I3Pp99GpT0OFXpWBz2sd3FETWrISCKqQYL+Qh4uG00S9oN+MjYcdXYkwEyyWDgKUMudveim0hn
v7xNva7DdfiERBaB8AXz0mF0E34IiVLmHu/ZaNyiBC69wvvKhZlORC5ZbfOHSaVSlfDHmQnoFFbX
wz5nkr/t868R5LwArJb/nF1tPTlMUvbhL4jg6QCelZ0eFdkL0qZsVJS9Khx6wCcd0wXIqnjgFVj2
gBQP98gnYNRE35BGza5B9elbYIRZk8b26oA2Y3Nmxw==
=EHdk
-----END PGP SIGNATURE-----

--ktVs5VwmmwDITJIdmbiOkFUQTz1ew9xRE--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:08:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:08:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194297.346118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPOz-0000LO-5z; Thu, 23 Sep 2021 14:08:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194297.346118; Thu, 23 Sep 2021 14:08:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPOz-0000LH-2f; Thu, 23 Sep 2021 14:08:41 +0000
Received: by outflank-mailman (input) for mailman id 194297;
 Thu, 23 Sep 2021 14:08:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QBZb=ON=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mTPOx-0000L9-KI
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:08:39 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.57]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id be6794b4-1c77-11ec-ba46-12813bfff9fa;
 Thu, 23 Sep 2021 14:08:36 +0000 (UTC)
Received: from AS9PR06CA0156.eurprd06.prod.outlook.com (2603:10a6:20b:45c::13)
 by AS8PR08MB6710.eurprd08.prod.outlook.com (2603:10a6:20b:397::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 14:08:35 +0000
Received: from AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::d5) by AS9PR06CA0156.outlook.office365.com
 (2603:10a6:20b:45c::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 14:08:35 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT030.mail.protection.outlook.com (10.152.16.117) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 14:08:34 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Thu, 23 Sep 2021 14:08:31 +0000
Received: from 60fb0bf4bc0b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1D7A6BD6-6ABE-4E3D-9CDD-0ADA6DF368DF.1; 
 Thu, 23 Sep 2021 14:08:19 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 60fb0bf4bc0b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 14:08:19 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6192.eurprd08.prod.outlook.com (2603:10a6:102:ea::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 14:08:15 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 14:08:15 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0221.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:b::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 14:08:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be6794b4-1c77-11ec-ba46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/jPowXSl5B39OM7PYXe287zKbMR+e5YkjxwJkceWc+Y=;
 b=SXDJFQlmvFNfHG2tiq2C1xb/OmR9JVSBkyBkv0Hh2l+o0k2a+BAjOWJ0v9xyu6auSKmdSREC0+FiRFmfOhC0czCwc02sa+gx6mQ60bWLfI+l8dUQDRMTtRdv7V9a9tTI04uXz8gd1HZun8oRe7cPOlW05fXR1eyQkuibzHNjJLw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 7beca78cade72a9e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Jogt1WvOs3vaYrTYeGNAmPpUP1Jnb4hNMkcjyghSay6RHB6oXD/DqBARngYND8YHsVTDtpYc8ExtQLzY2sEvov026fDYtHNo8aBUPRIBuQRNrWLwMVTrsrC0YAdBlwznLbOM8cXZZNN1hKZdf+souF5baGzlQiguHy4ppfd2SUNEq/Mb7Gv9L9Ga3dAQU4wpaS/oCFSTEPToE/KfAl5+Sae/U+Cvp/YsocXDbSkxDepb/ajJzz7vRTicL9js5yhgJ5dDzWrtyxOJS24V4+xgMPQpSG7yH4UnZiB+UVngGKszmPylpfHvOfJKs3k3mmD4u9xidq4Hp2hdr+m1gGe3EQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/jPowXSl5B39OM7PYXe287zKbMR+e5YkjxwJkceWc+Y=;
 b=PTUkOYRldiGqvBqKQ5qUv/J6dI4BX0U9VgaPNxawTqyHrqQs1O5txXLQfx6xyrUqKwbavrykZahtXszv/6P+0HGqA9NNBGEzp89gy9LOEUpSyacYsQjnpqBHtd7C2ktjRDv+5+p7C6ziId89qt93a3lyxpuqgkRW9/UC6vS2dOpUUSEpnzv2KtkONm51qDUIpgRX0Vs31BV2YHiH8gijb9UXX3qFt4D2BX1TF6VjtGR209AQGF6x0lSeH6p29S0/DLbdR2vNNFs6xjh3t0NCNHqkw5W9sX91w8xLJPD5xQ3ZW+c/FSn18rezPJ/Vlg562KyxHEQtXo1DNcCZBhZQJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/jPowXSl5B39OM7PYXe287zKbMR+e5YkjxwJkceWc+Y=;
 b=SXDJFQlmvFNfHG2tiq2C1xb/OmR9JVSBkyBkv0Hh2l+o0k2a+BAjOWJ0v9xyu6auSKmdSREC0+FiRFmfOhC0czCwc02sa+gx6mQ60bWLfI+l8dUQDRMTtRdv7V9a9tTI04uXz8gd1HZun8oRe7cPOlW05fXR1eyQkuibzHNjJLw=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s>
Date: Thu, 23 Sep 2021 15:08:07 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>,
 =?utf-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <2C0E480C-DF1C-4AAD-89F5-99D52B016970@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>
 <20210922141341.42288-3-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0221.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:b::17) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 040c4946-d3d2-4c4b-5f85-08d97e9ba185
X-MS-TrafficTypeDiagnostic: PA4PR08MB6192:|AS8PR08MB6710:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB671025A943F36BF7DE3A7CEBE4A39@AS8PR08MB6710.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2331;OLM:2331;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 dY8mXK09zyKIJD4Gvf/aXj0g7cP+ct9w7HXFs5xfTkO78gDgVPkAdsn53d253kjClmpYVJzovgev/VhqkInQqzFM6H8UEAPFIT/6ocaN5ejEpMFRjCl55mCC62adaO1UeznmXFKKozaNp7zfRzrYH+7WYUHGjZendrQxrJzELQ5op5ZZj0KUG1F8UpY7UFgdXF0aaQR+62bkk8/aDlgiVDfJ7z5qB3sAXGqs9CCfla3DQPD7Xkb+BKiak0TJGUA2RwCiLn+lbx1GQZys2muLHONsDyKBD18uUz3N6jGFLhHZR+fD53EYG/JOT1vJFDzYBI95jBvK+C6SJP68uU9J1HcRLZ7dT4TXAPA8T6gIOwEQwNAzLlm0Q2nbmphGy5mYMYhWLUXUmCpKZNQVLnrLbGqbV3LmVMvTJigAdNgI2VURWlz0HQ+6kgCHuzipS6HifQpK+0rfkm+6t9XrT9A9AdQXwOId6mNJ410UdYLD1IFuVpdaGQG9Q0Fb3W1Sxoz1FYsd6G/8DQBztpPPNr0kuIQDRseusaISmSgRcSxNloC1Wy/Bw9yfAT+XMfv4qkGVyCzF12wWNaQTox1IjDnTjS3EtS4C/t1Cig5HJrU9vFiXANEsBnJX+FFtUSV9BkR+Dv0G1tXbk1RFCrfr0FyNE+kfOwecf1DM4emHDh3oNC3blAhwbIVPAJ+6u5CW3k5t3Aof3horJxbqtxjiwu4nHqOmKDVqd3ekVQDKPZMS2C0nZGu0hwGplFQgpB9yDDiYeCd7vjDmHNlzu3hlrCm78ArClTyQVKgWMjnI53Uy23A=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52116002)(38100700002)(38350700002)(6486002)(30864003)(6512007)(33656002)(508600001)(66556008)(66476007)(66946007)(956004)(54906003)(316002)(5660300002)(86362001)(6506007)(53546011)(4326008)(8936002)(6666004)(2616005)(2906002)(44832011)(26005)(186003)(8676002)(83380400001)(36756003)(6916009)(7416002)(32563001)(21314003)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6192
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3ee6e49e-944a-4a3f-d606-08d97e9b95af
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0CCArW9ftsiz6mwKr8LjjD/v+R70PJrvL40KZogo56PQgVGv1Z6DxFMbKM49mURIVCSP46Ffw/45jgvDonjUY7/ZxnINVDeykp4D4z0qzCFk/4dGJbd+SFguBM1gTGxmUbPR9Ts/6YNq9kheVFQBYqGNDJHG5UPeovn3c6Fx1JCb5cyiTKlsFNC1jUuSOwzmgBzqfklXD3xHG8d9rSaIUZUBIC0xFZ9Na3sgwntCF7NgMBhLslHMX87HWUy9PGn4cYm/xvY4mnriAl3p/xO+3dSDL1D3nJOVWJyF9tpcJqS4e7bcMuCyizuIxP7XYC9ltM0IrhZTpgFDfuBwBkw9yhVDaE0i/6ol9ywdvIJq5DCj7qV8dpZrdj+GVPi6vWZeEz52doVpOyU+flJcwv7NHdVDQr3/dq1K+j89y4JWQgjYxMiN3/YLsaChuZHREXwwHqXlk/gmAeuYi2oYYWPRd2t2xqcDyqblJ/ot9tndyHaFytRRFqn1lxb7wFUZmDLq1hCFeM1r8avOcaC/AyGqeqqc3lDsBdLPN0tVWB9/Ds1xHX5yKEGRPC1qogoyGAUE+qTPSUk7LXzUPLG1bPXTitlrTsLM4I6v1HR7XA6F7tsmLJn233XO0eYzzctv48tf9XK/UPXQtsWPm/itBQl6lBuO7Ye0Okokyqkz9psfHKoJAQ3rTLZwJKhqreRtu3txV/q19Kb2RXMj0QjSgcdDMNAy24FjTm0BFmKqiCqtjfA0NfmOmOso+1gyLA6p5HoF
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(54906003)(6506007)(316002)(6486002)(356005)(186003)(2616005)(33656002)(956004)(86362001)(6862004)(508600001)(81166007)(44832011)(6512007)(2906002)(26005)(53546011)(4326008)(70586007)(107886003)(8676002)(36756003)(36860700001)(336012)(30864003)(8936002)(82310400003)(6666004)(47076005)(5660300002)(70206006)(83380400001)(32563001)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:08:34.3749
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 040c4946-d3d2-4c4b-5f85-08d97e9ba185
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6710



> On 22 Sep 2021, at 22:51, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Wed, 22 Sep 2021, Luca Fancellu wrote:
>> This patch introduces the support for dom0less configuration
>> when using UEFI boot on ARM, it permits the EFI boot to
>> continue if no dom0 kernel is specified but at least one domU
>> is found.
>>=20
>> Introduce the new property "uefi,binary" for device tree boot
>> module nodes that are subnode of "xen,domain" compatible nodes.
>> The property holds a string containing the file name of the
>> binary that shall be loaded by the uefi loader from the filesystem.
>>=20
>> Update efi documentation about how to start a dom0less
>> setup using UEFI
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>=20
> Overall the patch is pretty good. I don't have any major concerns (only
> one about the documentation). Some minor comments below.
>=20
>=20
>> ---
>> Changes in v2:
>> - remove array of struct file
>> - fixed some int types
>> - Made the code use filesystem even when configuration
>> file is skipped.
>> - add documentation of uefi,binary in booting.txt
>> - add documentation on how to boot all configuration
>> for Xen using UEFI in efi.pandoc
>> ---
>> docs/misc/arm/device-tree/booting.txt |  21 +++
>> docs/misc/efi.pandoc                  | 203 ++++++++++++++++++++
>> xen/arch/arm/efi/efi-boot.h           | 257 +++++++++++++++++++++++++-
>> xen/arch/x86/efi/efi-boot.h           |   6 +
>> xen/common/efi/boot.c                 |  36 ++--
>> 5 files changed, 508 insertions(+), 15 deletions(-)
>>=20
>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/devic=
e-tree/booting.txt
>> index 44cd9e1a9a..bc0f8913db 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -182,6 +182,13 @@ The kernel sub-node has the following properties:
>>=20
>>     Command line parameters for the guest kernel.
>>=20
>> +- efi,binary (UEFI boot only)
>=20
> Below it is actually uefi,binary, not efi,binary. I would use
> uefi,binary consistently.

Yes that was a mistake, thanks for catching it

>=20
>=20
>> +    Specifies the file name to be loaded by the UEFI boot for this modu=
le. If
>> +    this is specified, there is no need to specify the reg property bec=
ause it
>> +    will be created by the UEFI stub on boot.
>> +    This option is needed only when UEFI boot is used.
>=20
> We need to specify that this parameter is a string.

ok

>=20
>=20
>> The ramdisk sub-node has the following properties:
>>=20
>> - compatible
>> @@ -193,6 +200,13 @@ The ramdisk sub-node has the following properties:
>>     Specifies the physical address of the ramdisk in RAM and its
>>     length.
>>=20
>> +- efi,binary (UEFI boot only)
>> +
>> +    Specifies the file name to be loaded by the UEFI boot for this modu=
le. If
>> +    this is specified, there is no need to specify the reg property bec=
ause it
>> +    will be created by the UEFI stub on boot.
>> +    This option is needed only when UEFI boot is used.
>=20
> same here

Ok

>=20
>>=20
>> Example
>> =3D=3D=3D=3D=3D=3D=3D
>> @@ -257,6 +271,13 @@ The dtb sub-node should have the following properti=
es:
>>     Specifies the physical address of the device tree binary fragment
>>     RAM and its length.
>>=20
>> +- efi,binary (UEFI boot only)
>> +
>> +    Specifies the file name to be loaded by the UEFI boot for this modu=
le. If
>> +    this is specified, there is no need to specify the reg property bec=
ause it
>> +    will be created by the UEFI stub on boot.
>> +    This option is needed only when UEFI boot is used.
>=20
> same here

Ok

>=20
>=20
>> As an example:
>>=20
>>         module@0xc000000 {
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index e289c5e7ba..698196e129 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -167,3 +167,206 @@ sbsign \
>> 	--output xen.signed.efi \
>> 	xen.unified.efi
>> ```
>> +
>> +## UEFI boot and dom0less on ARM
>> +
>> +Dom0less feature is supported by ARM and it is possible to use it when =
Xen is
>> +started as an EFI application.
>> +The way to specify the domU domains is by Device Tree as specified in t=
he
>> +[dom0less](dom0less.html) documentation page under the "Device Tree
>> +configuration" section, but instead of declaring the reg property in th=
e boot
>> +module, the user must specify the "uefi,binary" property containing the=
 name
>> +of the binary file that has to be loaded in memory.
>> +The UEFI stub will load the binary in memory and it will add the reg pr=
operty
>> +accordingly.
>> +
>> +An example here:
>> +
>> +domU1 {
>> +	#address-cells =3D <1>;
>> +	#size-cells =3D <1>;
>> +	compatible =3D "xen,domain";
>> +	memory =3D <0 0x20000>;
>> +	cpus =3D <1>;
>> +	vpl011;
>> +
>> +	module@1 {
>> +		compatible =3D "multiboot,kernel", "multiboot,module";
>> +		uefi,binary =3D "vmlinuz-3.0.31-0.4-xen";
>> +		bootargs =3D "console=3DttyAMA0";
>> +	};
>> +	module@2 {
>> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
>> +		uefi,binary =3D "initrd-3.0.31-0.4-xen";
>> +	};
>> +	module@3 {
>> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
>> +		uefi,binary =3D "passthrough.dtb";
>> +	};
>> +};
>> +
>> +## How to boot different Xen setup using UEFI
>> +
>> +Here the supported user cases for Xen when UEFI boot is used:
>=20
> "Supported" means different things to different people, so I would say
> instead:
>=20
> These are the different ways to boot a Xen system from UEFI:

Ok

>=20
>=20
>> +
>> + - Boot Xen and Dom0 (minimum required)
>> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
>> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
>> +
>> +### Boot Xen and Dom0
>> +
>> +This configuration can be started using the Xen configuration file in t=
he
>> +example above.
>> +
>> +### Boot Xen and DomU(s)
>> +
>> +This configuration needs the domU domain(s) specified in the /chosen no=
de,
>> +examples of how to do that are provided by the documentation about dom0=
less
>> +and the example above shows how to use the "uefi,binary" property to us=
e the
>> +UEFI stub for module loading.
>> +Providing the multiboot modules in the device tree, make Xen skip its U=
EFI
>> +configuration file, if it is needed for some reason, specify the "uefi,=
cfg-load"
>> +property in the /chosen node.
>=20
> This last sentence is not very clear and it looks like one needs to
> specify "uefi,cfg-load" to skip the Xen config file when actually it is
> the opposite. I would suggest instead:
>=20
> When adding DomU modules to device tree, also add the property
> uefi,cfg-load under chosen to have Xen load the Xen config file.
> Otherwise, Xen will skip the config file and rely on device tree alone.

Ok

>=20
>=20
>> +Example 1 of how to boot a true dom0less configuration:
>> +
>> +Xen configuration file: skipped.
>> +
>> +Device tree:
>> +
>> +```
>> +chosen {
>> +	#size-cells =3D <0x1>;
>> +	#address-cells =3D <0x1>;
>> +	xen,xen-bootargs =3D "<Xen command line>"
>> +
>> +	domU1 {
>> +		#size-cells =3D <0x1>;
>> +		#address-cells =3D <0x1>;
>> +		compatible =3D "xen,domain";
>> +		cpus =3D <0x1>;
>> +		memory =3D <0x0 0xc0000>;
>> +		vpl011;
>> +
>> +		module@1 {
>> +			compatible =3D "multiboot,kernel", "multiboot,module";
>> +			uefi,binary =3D "Image-domu1.bin";
>> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> +		};
>> +	};
>> +	domU2 {
>> +		#size-cells =3D <0x1>;
>> +		#address-cells =3D <0x1>;
>> +		compatible =3D "xen,domain";
>> +		cpus =3D <0x1>;
>> +		memory =3D <0x0 0x100000>;
>> +		vpl011;
>> +
>> +		module@2 {
>> +			compatible =3D "multiboot,kernel", "multiboot,module";
>> +			uefi,binary =3D "Image-domu2.bin";
>> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> +		};
>> +	};
>> +};
>> +```
>> +
>> +Example 2 of how to boot a true dom0less configuration:
>> +
>> +Xen configuration file:
>> +
>> +```
>> +[global]
>> +default=3Dxen
>> +
>> +[xen]
>> +options=3D<Xen command line>
>> +dtb=3D<optional DTB>
>> +```
>> +
>> +Device tree:
>> +
>> +```
>> +chosen {
>> +	#size-cells =3D <0x1>;
>> +	#address-cells =3D <0x1>;
>> +	uefi,cfg-load;
>> +
>> +	domU1 {
>> +		#size-cells =3D <0x1>;
>> +		#address-cells =3D <0x1>;
>> +		compatible =3D "xen,domain";
>> +		cpus =3D <0x1>;
>> +		memory =3D <0x0 0xc0000>;
>> +		vpl011;
>> +
>> +		module@1 {
>> +			compatible =3D "multiboot,kernel", "multiboot,module";
>> +			uefi,binary =3D "Image-domu1.bin";
>> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> +		};
>> +	};
>> +	domU2 {
>> +		#size-cells =3D <0x1>;
>> +		#address-cells =3D <0x1>;
>> +		compatible =3D "xen,domain";
>> +		cpus =3D <0x1>;
>> +		memory =3D <0x0 0x100000>;
>> +		vpl011;
>> +
>> +		module@2 {
>> +			compatible =3D "multiboot,kernel", "multiboot,module";
>> +			uefi,binary =3D "Image-domu2.bin";
>> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> +		};
>> +	};
>> +};
>> +```
>> +
>> +### Boot Xen, Dom0 and DomU(s)
>> +
>> +This configuration is a mix of the two configuration above, to boot thi=
s one
>> +the configuration file must be processed so the /chosen node must have =
the
>> +"uefi,cfg-load" property.
>> +
>> +Here an example:
>> +
>> +Xen configuration file:
>> +
>> +```
>> +[global]
>> +default=3Dxen
>> +
>> +[xen]
>> +options=3D<Xen command line>
>> +kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
>> +ramdisk=3Dinitrd-3.0.31-0.4-xen
>> +dtb=3D<optional DTB>
>> +```
>> +
>> +Device tree:
>> +
>> +```
>> +chosen {
>> +	#size-cells =3D <0x1>;
>> +	#address-cells =3D <0x1>;
>> +	uefi,cfg-load;
>> +
>> +	domU1 {
>> +		#size-cells =3D <0x1>;
>> +		#address-cells =3D <0x1>;
>> +		compatible =3D "xen,domain";
>> +		cpus =3D <0x1>;
>> +		memory =3D <0x0 0xc0000>;
>> +		vpl011;
>> +
>> +		module@1 {
>> +			compatible =3D "multiboot,kernel", "multiboot,module";
>> +			uefi,binary =3D "Image-domu1.bin";
>> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> +		};
>> +	};
>> +};
>> +```
>> +
>> +
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index 8ceeba4ad1..e2b007ece0 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -8,9 +8,43 @@
>> #include <asm/setup.h>
>> #include <asm/smp.h>
>>=20
>> +typedef struct {
>> +    char *name;
>> +    unsigned int name_len;
>> +    EFI_PHYSICAL_ADDRESS addr;
>> +    UINTN size;
>> +} dom0less_module_name;
>> +
>> +/*
>> + * Binaries will be translated into bootmodules, the maximum number for=
 them is
>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DT=
B
>> + */
>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
>> +static struct file __initdata dom0less_file;
>> +static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MO=
DULES];
>> +static unsigned int __initdata dom0less_modules_available =3D
>> +                               MAX_DOM0LESS_MODULES;
>> +static unsigned int __initdata dom0less_modules_idx;
>=20
> This is a lot better!
>=20
> We don't need both dom0less_modules_idx and dom0less_modules_available.
> You can just do:
>=20
> #define dom0less_modules_available (MAX_DOM0LESS_MODULES - dom0less_modul=
es_idx)
> static unsigned int __initdata dom0less_modules_idx;
>=20
> But maybe we can even get rid of dom0less_modules_available entirely?
>=20
> We can change the check at the beginning of allocate_dom0less_file to:
>=20
> if ( dom0less_modules_idx =3D=3D dom0less_modules_available )
>    blexit
>=20
> Would that work?

I thought about it but I think they need to stay, because dom0less_modules_=
available is the
upper bound for the additional dom0less modules (it is decremented each tim=
e a dom0 module
Is added), instead dom0less_modules_idx is the typical index for the array =
of dom0less modules.

>=20
>=20
>> +#define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
>> +
>> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>> void __flush_dcache_area(const void *vaddr, unsigned long size);
>>=20
>> +static int get_dom0less_file_index(const char *name, unsigned int name_=
len);
>> +static unsigned int allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
>> +                                           const char *name,
>> +                                           unsigned int name_len);
>> +static void handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
>> +                                        int module_node_offset,
>> +                                        int reg_addr_cells,
>> +                                        int reg_size_cells);
>> +static void handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>> +                                        int domain_node,
>> +                                        int addr_cells,
>> +                                        int size_cells);
>> +static bool efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_handle);
>>=20
>> #define DEVICE_TREE_GUID \
>> {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, =
0xe0}}
>>=20
>> @@ -552,8 +586,218 @@ static void __init efi_arch_handle_module(const st=
ruct file *file,
>>                          kernel.size) < 0 )
>>             blexit(L"Unable to set reg property.");
>>     }
>> -    else
>> +    else if ( file !=3D &dom0less_file )
>> +        /*
>> +         * If file is not a dom0 module file and it's not a domU module=
,
>> +         * stop here.
>> +         */
>>         blexit(L"Unknown module type");
>> +    /*
>> +     * dom0less_modules_available is decremented here because for each =
dom0
>> +     * file added, there will be an additional bootmodule, so the numbe=
r
>> +     * of dom0less module files will be decremented because there is
>> +     * a maximum amount of bootmodules that can be loaded.
>> +     */
>> +    dom0less_modules_available--;
>> +}
>> +
>> +/*
>> + * This function checks for a binary previously loaded with a give name=
, it
>> + * returns the index of the file in the dom0less_files array or a negat=
ive
>> + * number if no file with that name is found.
>> + */
>> +static int __init get_dom0less_file_index(const char *name,
>> +                                          unsigned int name_len)
>> +{
>> +    unsigned int i;
>> +    int ret =3D ERROR_DOM0LESS_FILE_NOT_FOUND;
>> +
>> +    for (i =3D 0; i < dom0less_modules_idx; i++)
>> +    {
>> +        dom0less_module_name *mod =3D &dom0less_modules[i];
>> +        if ( (mod->name_len =3D=3D name_len) &&
>> +             (strncmp(mod->name, name, name_len) =3D=3D 0) )
>> +        {
>> +            ret =3D i;
>> +            break;
>> +        }
>> +    }
>> +    return ret;
>> +}
>> +
>> +/*
>> + * This function allocates a binary and keeps track of its name, it
>> + * returns the index of the file in the dom0less_files array.
>> + */
>> +static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_h=
andle,
>> +                                                  const char *name,
>> +                                                  unsigned int name_len=
)
>> +{
>> +    dom0less_module_name* file_name;
>> +    union string module_name;
>> +    unsigned int ret_idx;
>> +
>> +    /*
>> +     * Check if there is any space left for a domU module, the variable
>> +     * dom0less_modules_available is updated each time we use read_file=
(...)
>> +     * successfully.
>> +     */
>> +    if ( !dom0less_modules_available )
>> +        blexit(L"No space left for domU modules");
>=20
> This is the check that could be based on dom0less_modules_idx
>=20

The only way I see to have it based on dom0less_modules_idx will be to comp=
are it
to the amount of modules still available, that is not constant because it i=
s dependent
on how many dom0 modules are loaded, so still two variables needed.
Don=E2=80=99t know if I=E2=80=99m missing something.

>=20
>> +    module_name.s =3D (char*) name;
>> +    ret_idx =3D dom0less_modules_idx;
>> +
>> +    /* Save at this index the name of this binary */
>> +    file_name =3D &dom0less_modules[ret_idx];
>> +
>> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(ch=
ar),
>> +                              (void**)&file_name->name) !=3D EFI_SUCCES=
S )
>> +        blexit(L"Error allocating memory for dom0less binary name");
>> +
>> +    /* Save name and length of the binary in the data structure */
>> +    strlcpy(file_name->name, name, name_len);
>> +    file_name->name_len =3D name_len;
>=20
> Sorry for the silly question but do we need to set the '\0' at the end
> of the string or can we assumed the memory returned by AllocatePool is
> already zeroed? Or should we call strlcpy with name_len+1 ?

Not silly at all, I will call strlcpy with name_len+1 and strlcpy will put =
the
termination.

>=20
>=20
>> +
>> +    /* Load the binary in memory */
>> +    read_file(dir_handle, s2w(&module_name), &dom0less_file, NULL);
>> +
>> +    /* Save address and size */
>> +    file_name->addr =3D dom0less_file.addr;
>> +    file_name->size =3D dom0less_file.size;
>> +
>> +    /* s2w(...) allocates some memory, free it */
>> +    efi_bs->FreePool(module_name.w);
>> +
>> +    dom0less_modules_idx++;
>> +
>> +    return ret_idx;
>> +}
>> +
>> +/*
>> + * This function checks for the presence of the uefi,binary property in=
 the
>> + * module, if found it loads the binary as dom0less module and sets the=
 right
>> + * address for the reg property into the module DT node.
>> + */
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                          int module_node_offset,
>> +                                          int reg_addr_cells,
>> +                                          int reg_size_cells)
>> +{
>> +    const void *uefi_name_prop;
>> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number +=
 \0 */
>> +    int uefi_name_len, file_idx;
>> +    dom0less_module_name *file;
>> +
>> +    /* Read uefi,binary property to get the file name. */
>> +    uefi_name_prop =3D fdt_getprop(fdt, module_node_offset, "uefi,binar=
y",
>> +                                 &uefi_name_len);
>> +
>> +    if ( !uefi_name_prop )
>> +        /* Property not found */
>> +        return;
>> +
>> +    file_idx =3D get_dom0less_file_index(uefi_name_prop, uefi_name_len)=
;
>> +    if (file_idx < 0)
>> +        file_idx =3D allocate_dom0less_file(dir_handle, uefi_name_prop,
>> +                                          uefi_name_len);
>> +
>> +    file =3D &dom0less_modules[file_idx];
>> +
>> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->ad=
dr);
>> +
>> +    /* Rename the module to be module@{address} */
>> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
>> +        blexit(L"Unable to add domU ramdisk FDT node.");
>> +
>> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_=
cells,
>> +                     file->addr, file->size) < 0 )
>> +        blexit(L"Unable to set reg property.");
>> +}
>> +
>> +/*
>> + * This function checks for boot modules under the domU guest domain no=
de
>> + * in the DT.
>> + */
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_hand=
le,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells)
>> +{
>> +    int module_node;
>> +    /*
>> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device=
-tree}
>> +     * inside this node
>> +     */
>> +    for ( module_node =3D fdt_first_subnode(fdt, domain_node);
>> +          module_node > 0;
>> +          module_node =3D fdt_next_subnode(fdt, module_node) )
>> +        if ( (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,kernel") =3D=3D 0) |=
|
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,ramdisk") =3D=3D 0) =
||
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,device-tree") =3D=3D=
 0) )
>> +            /* The compatible is one of the strings above, check the mo=
dule */
>> +            handle_dom0less_module_node(dir_handle, module_node, addr_c=
ells,
>> +                                        size_cells);
>> +}
>> +
>> +/*
>> + * This function checks for xen domain nodes under the /chosen node for=
 possible
>> + * domU guests to be loaded.
>> + */
>> +static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_han=
dle)
>> +{
>> +    int chosen;
>> +    int addr_len, size_len;
>> +    unsigned int i =3D 0;
>> +
>> +    /* Check for the chosen node in the current DTB */
>> +    chosen =3D setup_chosen_node(fdt, &addr_len, &size_len);
>> +    if ( chosen < 0 )
>> +        blexit(L"Unable to setup chosen node");
>> +
>> +    /* Check for nodes compatible with xen,domain under the chosen node=
 */
>> +    for ( int node =3D fdt_first_subnode(fdt, chosen);
>> +          node > 0;
>> +          node =3D fdt_next_subnode(fdt, node) )
>> +    {
>> +        int addr_cells, size_cells, len;
>> +        const struct fdt_property *prop;
>> +
>> +        if ( fdt_node_check_compatible(fdt, node, "xen,domain") !=3D 0 =
)
>> +            continue;
>> +
>> +        /* Get or set #address-cells and #size-cells */
>> +        prop =3D fdt_get_property(fdt, node, "#address-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#address-cells not found in domain node.");
>> +
>> +        addr_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        prop =3D fdt_get_property(fdt, node, "#size-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#size-cells not found in domain node.");
>> +
>> +        size_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        /* Found a node with compatible xen,domain; handle this node. *=
/
>> +        handle_dom0less_domain_node(dir_handle, node, addr_cells, size_=
cells);
>> +    }
>> +
>> +    /* Free dom0less file names if any */
>> +    for ( ; i < dom0less_modules_idx; i++ )
>> +    {
>> +        /* Free dom0less binary names */
>> +        efi_bs->FreePool(dom0less_modules[i].name);
>> +    }
>> +
>> +    if ( dom0less_modules_idx > 0 )
>> +        return true;
>> +
>> +    return false;
>> }
>>=20
>> static void __init efi_arch_cpu(void)
>> @@ -562,8 +806,19 @@ static void __init efi_arch_cpu(void)
>>=20
>> static void __init efi_arch_blexit(void)
>> {
>> +    unsigned int i =3D 0;
>> +
>>     if ( dtbfile.need_to_free )
>>         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
>> +    /* Free dom0less files if any */
>> +    for ( ; i < dom0less_modules_idx; i++ )
>> +    {
>> +        /* Free dom0less binary names */
>> +        efi_bs->FreePool(dom0less_modules[i].name);
>> +        /* Free dom0less binaries */
>> +        efi_bs->FreePages(dom0less_modules[i].addr,
>> +                          PFN_UP(dom0less_modules[i].size));
>> +    }
>>     if ( memmap )
>>         efi_bs->FreePool(memmap);
>> }
>> diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
>> index 9b0cc29aae..950fdf16b7 100644
>> --- a/xen/arch/x86/efi/efi-boot.h
>> +++ b/xen/arch/x86/efi/efi-boot.h
>> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const str=
uct file *file,
>>     efi_bs->FreePool(ptr);
>> }
>>=20
>> +static bool __init efi_arch_check_dom0less_boot(EFI_FILE_HANDLE dir_han=
dle)
>> +{
>> +    /* x86 doesn't support dom0less */
>> +    return false;
>> +}
>> +
>> static void __init efi_arch_cpu(void)
>> {
>>     uint32_t eax =3D cpuid_eax(0x80000000);
>> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
>> index 758f9d74d2..7d8734199e 100644
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1127,15 +1127,16 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>     static EFI_GUID __initdata shim_lock_guid =3D SHIM_LOCK_PROTOCOL_GUI=
D;
>>     EFI_LOADED_IMAGE *loaded_image;
>>     EFI_STATUS status;
>> -    unsigned int i, argc;
>> -    CHAR16 **argv, *file_name, *cfg_file_name =3D NULL, *options =3D NU=
LL;
>> +    unsigned int i, argc =3D 0;
>> +    CHAR16 **argv, *file_name =3D NULL, *cfg_file_name =3D NULL, *optio=
ns =3D NULL;
>>     UINTN gop_mode =3D ~0;
>>     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop =3D NULL;
>>     union string section =3D { NULL }, name;
>>     bool base_video =3D false;
>> -    const char *option_str;
>> +    const char *option_str =3D NULL;
>>     bool use_cfg_file;
>> +    EFI_FILE_HANDLE dir_handle;
>>=20
>>     __set_bit(EFI_BOOT, &efi_flags);
>>     __set_bit(EFI_LOADER, &efi_flags);
>> @@ -1216,9 +1217,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>>=20
>>     efi_arch_relocate_image(0);
>>=20
>> +    /* Get the file system interface. */
>> +    dir_handle =3D get_parent_handle(loaded_image, &file_name);
>> +
>>     if ( use_cfg_file )
>>     {
>> -        EFI_FILE_HANDLE dir_handle;
>>         UINTN depth, cols, rows, size;
>>=20
>>         size =3D cols =3D rows =3D depth =3D 0;
>> @@ -1229,9 +1232,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>>=20
>>         gop =3D efi_get_gop();
>>=20
>> -        /* Get the file system interface. */
>> -        dir_handle =3D get_parent_handle(loaded_image, &file_name);
>> -
>>         /* Read and parse the config file. */
>>         if ( read_section(loaded_image, L"config", &cfg, NULL) )
>>             PrintStr(L"Using builtin config file\r\n");
>> @@ -1285,14 +1285,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>             efi_bs->FreePool(name.w);
>>         }
>>=20
>> -        if ( !name.s )
>> -            blexit(L"No Dom0 kernel image specified.");
>> -
>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>>=20
>> -        option_str =3D split_string(name.s);
>> +        if ( name.s )
>> +            option_str =3D split_string(name.s);
>>=20
>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str=
) )
>> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str=
) &&
>> +             name.s )
>=20
> I'd probably indent this and put it under the previous if, instead of
> adding "&& name.s". But I'll leave it up to you.

If name.s is not specified we might want to check if the kernel is in the i=
mage section anyway.
When read_section is executed, it it returns true, then the kernel is loade=
d even if the name.s
Is NULL (it is NULL because we didn=E2=80=99t specify kernel=3D<image> <arg=
s>)
The code there is a little bit tricky I can tell.

>=20
>=20
>>         {
>>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>>             efi_bs->FreePool(name.w);
>> @@ -1361,12 +1360,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>>         cfg.addr =3D 0;
>>=20
>> -        dir_handle->Close(dir_handle);
>> -
>>         if ( gop && !base_video )
>>             gop_mode =3D efi_find_gop_mode(gop, cols, rows, depth);
>>     }
>>=20
>> +    /*
>> +     * Check if a proper configuration is provided to start Xen:
>> +     *  - Dom0 specified (minimum required)
>> +     *  - Dom0 and DomU(s) specified
>> +     *  - DomU(s) specified
>> +     */
>> +    if ( !efi_arch_check_dom0less_boot(dir_handle) && !kernel.addr )
>> +        blexit(L"No Dom0 kernel image specified.");
>> +
>> +    dir_handle->Close(dir_handle);
>> +
>>     efi_arch_edd();
>>=20
>>     /* XXX Collect EDID info. */
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:08:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:08:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194298.346128 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPP7-0000fe-Hq; Thu, 23 Sep 2021 14:08:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194298.346128; Thu, 23 Sep 2021 14:08:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPP7-0000fX-Eu; Thu, 23 Sep 2021 14:08:49 +0000
Received: by outflank-mailman (input) for mailman id 194298;
 Thu, 23 Sep 2021 14:08:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTPP5-0000ef-Tm
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:08:47 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c49292b2-1c77-11ec-ba46-12813bfff9fa;
 Thu, 23 Sep 2021 14:08:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 715D21FFB0;
 Thu, 23 Sep 2021 14:08:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3A95513E76;
 Thu, 23 Sep 2021 14:08:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id UmAiDW6KTGEmZAAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:08:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c49292b2-1c77-11ec-ba46-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632406126; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=YpdVqII+a+2ign5Gw5YEwLEqZL7daQpZ14uDvZt0kn8=;
	b=qfJDn5jN8BgsSg7vRDzNUS39VlaWgwGyf3Vbx1+zAsTn6/16Cb0JKkKKrHllAWLLrOXoDB
	pGnVvvcY7OkKY82+pdpanZaDvWLFbeq4HGQ/YwTntjYDW487JlykHYAVHDSJ8Qnsn+ipH8
	iROnSTlVuXdn33JLlYGF4g1ujiYpgOA=
Subject: Re: [PATCH 5/9] xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5e9ff16e-85f0-50ea-f053-37e17351a0cc@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <8d9625ce-e92d-61e0-76c3-98b373ac58b4@suse.com>
Date: Thu, 23 Sep 2021 16:08:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <5e9ff16e-85f0-50ea-f053-37e17351a0cc@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="rcxj8k18kJK4Rt3BNqZmdkTMVC38Ha0Gi"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--rcxj8k18kJK4Rt3BNqZmdkTMVC38Ha0Gi
Content-Type: multipart/mixed; boundary="dTkucEr8pCYBdB1EMrSXQ2wqaNLRyP23c";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
Message-ID: <8d9625ce-e92d-61e0-76c3-98b373ac58b4@suse.com>
Subject: Re: [PATCH 5/9] xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5e9ff16e-85f0-50ea-f053-37e17351a0cc@suse.com>
In-Reply-To: <5e9ff16e-85f0-50ea-f053-37e17351a0cc@suse.com>

--dTkucEr8pCYBdB1EMrSXQ2wqaNLRyP23c
Content-Type: multipart/mixed;
 boundary="------------F71F769F00374A0B92B7ECAA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------F71F769F00374A0B92B7ECAA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:10, Jan Beulich wrote:
> xenboot_write_console() is dealing with these quite fine so I don't see=

> why xenboot_console_setup() would return -ENOENT in this case.
>=20
> Adjust documentation accordingly.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------F71F769F00374A0B92B7ECAA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------F71F769F00374A0B92B7ECAA--

--dTkucEr8pCYBdB1EMrSXQ2wqaNLRyP23c--

--rcxj8k18kJK4Rt3BNqZmdkTMVC38Ha0Gi
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMim0FAwAAAAAACgkQsN6d1ii/Ey/J
7gf/RKO2W7fozhOgDLmvzri9NwREBxZtrCgj4Ri8XOSLct0/S2u8vSkP8Js3/IPDjM//JzqCXpTj
4czZwneK9yEQ614n5OpGZ06SE0NZr9OGDHccHT17VTmaqVu6Wtu2TrdB8dXcjJwVxyZz1OiSIrvH
Iaz/S19KcsqmusKYumEjMjj9cOrdEMShG/bxRoHlc3Y/HmZxYA6LvPtVrVHmFoDskTp2qv6VR8YL
WcQtv/ucbDjio+C6ZgpQny5hoYcJ4hwCBKeh/d1V/GaQ/DkNDTUD5rqqzVYUzcT+L6moYGXYduN6
jq0lHz1evvZHzhvV7pK4zLms9uadtVgeVrCzjrJwqQ==
=lu/s
-----END PGP SIGNATURE-----

--rcxj8k18kJK4Rt3BNqZmdkTMVC38Ha0Gi--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:10:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:10:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194309.346140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPQm-0002KL-Ux; Thu, 23 Sep 2021 14:10:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194309.346140; Thu, 23 Sep 2021 14:10:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPQm-0002KE-RZ; Thu, 23 Sep 2021 14:10:32 +0000
Received: by outflank-mailman (input) for mailman id 194309;
 Thu, 23 Sep 2021 14:10:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wGVh=ON=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mTPQl-0002K4-GG
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:10:31 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 020b33a6-1c78-11ec-ba47-12813bfff9fa;
 Thu, 23 Sep 2021 14:10:30 +0000 (UTC)
Received: from pps.filterd (m0246617.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NDqZN7007034; 
 Thu, 23 Sep 2021 14:10:08 GMT
Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71])
 by mx0b-00069f02.pphosted.com with ESMTP id 3b8neb2bnk-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 23 Sep 2021 14:10:00 +0000
Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1])
 by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18NE73cx147870;
 Thu, 23 Sep 2021 14:09:59 GMT
Received: from nam10-dm6-obe.outbound.protection.outlook.com
 (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106])
 by aserp3030.oracle.com with ESMTP id 3b7q5dsfre-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 23 Sep 2021 14:09:59 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by BLAPR10MB4931.namprd10.prod.outlook.com (2603:10b6:208:331::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 14:09:58 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::f520:b987:b36e:618f%8]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 14:09:58 +0000
Received: from [10.74.119.142] (138.3.201.14) by
 MN2PR16CA0019.namprd16.prod.outlook.com (2603:10b6:208:134::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 14:09:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 020b33a6-1c78-11ec-ba47-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=jkDtPb1T0sU5FbY+CiYBQbSOYKSIUZIPL2BFDe0J36I=;
 b=wD9yLZp7FnMvrwHZf+j99va+yc52085wMo4aDFnsrSxbNG9PJgzG9TVSeaHgoC+r5R+/
 P/4ZlVq8v+1/o4yC8e88KgLajeXclXwoiWJ+W/mjrvaAKsomOGXyTiNZ359BVpIweTD8
 Yu0lcpuiO/1NL8nkasjNgMseRRvtaF+mMeAEmnfPme+sxMMpOM8WXbHjKRJia82WNk2q
 6msJVMBxI2Z1XvFQBfCrOSPS36sRxfxvBKCfgdg8dWbxBs0QPFYsfEwHIvDWBgtu9oKU
 tCKMVfgJaZgrr+Hp4i9MUGQ9L0BQnMWey9hk1VcLq130t1OW/u+8DIJeJjt/g8b8ZXTm Mg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZrSV/1aAySU3Z21yZRq3db2wvQnABx6rJeBDeXDPu+0JaG5ahHdwTxR5Uf+/Vm9C+HZB7QoMlMJY9FnttiawI5A84kwxN+ik3pF+RDzRk2GCl14T1aM/uUZYxKHj5j9t8nT8IOZgJJh9g6Cw+8EFiIM8PDOvaSiObaN8jy3WzlTU++LjBXphuH2JY51glF7/IkEAMNgVZ/Xm4YyH+DWCeY1vImskYgwbVi6aZSmtQjT0Hv87lKqDizfkRs65tYC8I/iEjRj7RyP/nSxjRYhfvA0CjrlIuQ0B5B9fwdYQr/9l7O9LTbCbhM012HQYbyBKvfP6VnHJaWUzmmFyLVjInA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jkDtPb1T0sU5FbY+CiYBQbSOYKSIUZIPL2BFDe0J36I=;
 b=M8SjOf+KMU9W7gKJeusdvDMT8lARc0zkiJE/4cDw2W8VTrEs9ci52yV1Jr5CBfgXnMIlMzKyEPYbpKn3yZr9gIoX7btIdMJ/32HLen5+BDHStJJLFc3JEeVw0rH42QeBvRwt0GcvwEfzAz2qjqc6haQdXkjDctmBCvjRyyhZ384XEKXEir8oIzcDzE1JXRIkHhE2pFWw5J5rW9XrF13cekm4DzuVF2ewITR2eSlH4FcLqDAQx7us8ga/c/ayiVShTzbe7RwClqzSYecB1J5fo6Rw3J4dd+WeVSC3Wc/4JWRcGm0Txu2APhsVAU+KV4fKwxvFUfphy3RkAiGj4aV03g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=jkDtPb1T0sU5FbY+CiYBQbSOYKSIUZIPL2BFDe0J36I=;
 b=eaVXRq9cifzhamMPpiDUXbOEqTaw5/J36X5+3IZ0smQ2o28ItsUH0ylrlkRAzCppsl0EEY0yoHEz3xw6FM4zND1g4qkKRDXZfEpvev5HFaeDR4INAhM2bhnsUWEp04y8ltpjMQBJJp1ZWOxGyLbuHF1CD0U9fqwDZAERbEzpRIs=
Authentication-Results: zytor.com; dkim=none (message not signed)
 header.d=none;zytor.com; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v2 1/2] x86/xen: remove xen_have_vcpu_info_placement flag
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
        x86@kernel.org, linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, Stefano Stabellini <sstabellini@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>
References: <20210922103102.3589-1-jgross@suse.com>
 <20210922103102.3589-2-jgross@suse.com>
 <212d31cd-650d-27c6-b523-fd4f686872d1@oracle.com>
 <7de79902-e31b-899d-44bb-f9daabb2ecf0@suse.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <154d8fd5-1c31-97b2-8d8e-d50c9ff7a51f@oracle.com>
Date: Thu, 23 Sep 2021 10:09:55 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <7de79902-e31b-899d-44bb-f9daabb2ecf0@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: MN2PR16CA0019.namprd16.prod.outlook.com
 (2603:10b6:208:134::32) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2074d78b-faf8-4d84-dbe3-08d97e9bd33d
X-MS-TrafficTypeDiagnostic: BLAPR10MB4931:
X-Microsoft-Antispam-PRVS: 
	<BLAPR10MB49316794D3EE8C2DE6A090B28AA39@BLAPR10MB4931.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:336;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	zifko0Ij+HI9t6Fq9bvXLNGjo8c7p2qbhzaqpgRIOo9DFxybwXWbQ2tfj+auvm++aW49q/Fe9cO1frGe2tga+ALTu3LpZUmBDmW9JZSXPZwk6B9iR3rVlSa0Uyd47L8cSDF4eGz5j8od3Qw0jEK3ufl4jyQ55fXbQsscr4cWwJjyYH0Y5a7C80b2sJSxLIJLiLXcFiDum59LoGtUNASRmchGyAQRE65qwXe0t0yu0zUO5oTfx2+Zq0k6iV9klW9S1YICcyLlyoKHRwmlYjrTTI9LaIum5iw2Makb/zMFxHtomt+ji+Dy5suHdv7OyAZuBETZGivajnPN3mdxBwKxjgJh/dMpSO/dP485ZGrLBDeKYxfSGouuf/4Ym8wvxcLTiKW2mAeN483QkgvW4TzYF+/BNDMDklsAFTyisoUKsngBSZei1ncqLJQk1NLLc91mRUw+/B9PdB2HEs2HrG/RgESqPlLkj7fJM7WWzSUZSR2gG/9rWt327z2Etf8vL178/hiclFr/TTzkZM7ahMXhpIlDwfr/1YHm/l7VXisZfrThP1rxNS+n6WQ5e3D3zLYwx4Q0mN+RjfrNyth/SfULHFiRIG+GhTweuot8dCUYopy2W6SP+qbTqvBjMh6oqIC+E+7Zk3iRhyw1Jxd2SZ3Jk6HsM0VpK9b+vWA0A4XWrDUkfDoSlDzyJQi6qr/os60Rp0andWuY6QlRuoNXtt92H755jVE+CoGc4fBP6NY/rWKxzQJ3mtv/zD15F6rVVtPQ
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(2616005)(4326008)(956004)(31686004)(186003)(508600001)(4744005)(26005)(38100700002)(44832011)(316002)(16576012)(7416002)(8676002)(6486002)(5660300002)(8936002)(54906003)(2906002)(53546011)(66476007)(66556008)(66946007)(36756003)(83380400001)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?enFobldFVUhrWlcvRlhkRVZKVThPdFZqUGpRUnduOVp5bUhZa2IzS0tEMHk3?=
 =?utf-8?B?SVk3eGcvbjlQVEdzZStFM2ZFc2phTDQzbG9xZWpLbzZLRmdRajN0azc5dFN5?=
 =?utf-8?B?M1VWck4zaS9JNW5YRFgxRVRoOGYxOUF1SDRISEFDdWZaWk56ZEF3Tm8ycGx1?=
 =?utf-8?B?OE9HSUdtN0ljK2FFRm44VEh3anF6eHAxdDMxbk1oMmNsNTdNWGpwVmZ0cUFn?=
 =?utf-8?B?N0N4SCtZVC9JYnlQZnJINytNMVp1MlBPTTVaSGc1VHN5TDV2ZGpDVjVYWlF4?=
 =?utf-8?B?V3ZBUHNaYjNkT2VqSTl0NC9yeE9Bdng4Sm9QWXpsbi9nSXJvc3pBN1J4elZ3?=
 =?utf-8?B?RGNZdTVKNjYwNVJQTGF2dzk5bVRvM3VnSmRmWUVJMHgyeVl4MWZEbnBBK2lW?=
 =?utf-8?B?S0l6Rk1zZU1TN3pVQjVBRlhNSnE0TTIvVnM4R0JvbFMyWkhXc25GdU4rN1kw?=
 =?utf-8?B?VGVKQWxRTzJGc2orRmdERmV6eWtFYmhkeW1GelRVZHBrVEJjUkJJcUpZM3J0?=
 =?utf-8?B?bkJzSzA3R1A2N09zVTRIZmxuUHdpckcvKzVzTGg1SGRDZ3BsVlY1VDRiOUtx?=
 =?utf-8?B?UG5uT0V5Njl6eXpaK1ZrOG42cmgzbDVrOVdRdDZJblZ1MjVFa0p4dmpjRW11?=
 =?utf-8?B?SlRqdExqTjNMbllaNnZYdThSN0kzVlB1by8wKzQxM0t2b0d0S0RLMzlpRWRq?=
 =?utf-8?B?MHQwTEUwbHIzS0t4cTRKUHMyQjM0a2hyTUlJbm1VczdXNFFxRU81aGxnUENh?=
 =?utf-8?B?VTJrejBTOFZRVFVZUVZBZFJuTjFuUkZ6YUIva2xvK2Q4OGFnN2p3dzduM0k3?=
 =?utf-8?B?VlpSMkZnUS9YNkMxR0x3SDljUUJ6Mk1GQnhrMFFocEdRZUhaQS9rVkFiQ1RC?=
 =?utf-8?B?N2lJcGFXZkthTG5CbnowbFlYT0pVQytNdjBnQVc4RDFlajVUQ1RBMFh5OG41?=
 =?utf-8?B?eCtXVm82SGdlS0I5SE9vQSswTHhaSVJVS1JzV3AwYTFwalk4N0oxYmlmeDNN?=
 =?utf-8?B?QlpwcUJVYWJESVlsaWtFTWRmaU1oQWIrcmw1ZFpHazNGcWt1YU5jL1VzMkU3?=
 =?utf-8?B?amZDSkp2VkU2UE43UzZtWXlmZTh4Snh0ZmVGL0hBY0ZGZDl4RkwvUkhhOWRJ?=
 =?utf-8?B?VGNTaktmWUluWkhIZ3RlUE0yNkYzUTlkcnRGeE8vRU12WHd4WS9WWnhrZmRX?=
 =?utf-8?B?Tmk0aHBqTHUrTU5DNytLdkswd0V4d0cwTExPWk56bEVqb3Bob240ZUZPSm9w?=
 =?utf-8?B?NGdPQ0FXem16Z1IwMWhlUzZiaVF1aFlUZXhLOUExWVgzRXVsOXJSbi8yK2dW?=
 =?utf-8?B?QjJBL3JpbXUvbEtLY3pTVitIbXFtVHNGZWFUK1RtOE5XbG80VVIzZVAwaEM4?=
 =?utf-8?B?bGZDeWsyOGdsMGZhc2JiU0Zra3B2U2FBa2xjZ0N3QVlGbU1NV2tnd3pGaStR?=
 =?utf-8?B?d3l5K0toaFBEZlN3N2tNRmdZekdDVHFCUVY4U3BIcDFKcWFFaWlXWTJkZXJz?=
 =?utf-8?B?ZGRrYlhnV1F1RVZwUExHQTZSdUJHVTVLU0doM2libndpZno5ZnhYb3ZYcGhQ?=
 =?utf-8?B?cTZJRmNiT2N5SGtNNE5selhlQ2c2ZDl5MUZWRHlxT2xuVmxTQm9CNFp4QUVi?=
 =?utf-8?B?SGNTUlIxc3l2RTFOQ1JiQnk3b3ZoeFdIZkhlTzNYNWZUOXB5NFJNVlVBSjA1?=
 =?utf-8?B?NldYbnRlS3FDVzVIcjlEclAwclZjMFNhdm9HdWNFODdhTkwyVFh0RmlscWxm?=
 =?utf-8?Q?Ez99+r0C/qy66BlCrFnW/vcFp7Z3Y1vOpdss4qu?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2074d78b-faf8-4d84-dbe3-08d97e9bd33d
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:09:57.9877
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bu+XPjhqin/BmTfvFLj6R9eBrA0umBVK/zZpZ5nPyXW7Gh9ub8BHt0E/7MDMgiDCb2yo7QkuurQYNCfulysrNNoIAPpNpvYwH7x9y53eA6k=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB4931
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10115 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0
 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109200000 definitions=main-2109230089
X-Proofpoint-GUID: CedxHG7Pz8_Xx3hoSw6Mr5tVO9aQ9zRo
X-Proofpoint-ORIG-GUID: CedxHG7Pz8_Xx3hoSw6Mr5tVO9aQ9zRo


On 9/23/21 12:44 AM, Juergen Gross wrote:
>
> Hmm, maybe I should have been more explicit saying that the hypercall
> was introduced in Xen 3.4, and only reason of failure is either an
> illegal vcpu, an invalid mapping specification, or a try to reissue the
> hypercall for a vcpu. None of those should ever happen.
>

That last sentence -- famous last words ;-) But yes, sure.


Assuming both patches adjust their commit messages and the typo


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:17:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:17:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194314.346151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPXb-0003JH-Ih; Thu, 23 Sep 2021 14:17:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194314.346151; Thu, 23 Sep 2021 14:17:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPXb-0003JA-FX; Thu, 23 Sep 2021 14:17:35 +0000
Received: by outflank-mailman (input) for mailman id 194314;
 Thu, 23 Sep 2021 14:17:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTPXZ-0003J0-LU; Thu, 23 Sep 2021 14:17:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTPXZ-0001Ss-D2; Thu, 23 Sep 2021 14:17:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTPXZ-0000wy-0N; Thu, 23 Sep 2021 14:17:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTPXY-0000DA-W6; Thu, 23 Sep 2021 14:17:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5FZlgBcE/8gK2hMubckIn1/S66oKb9lVggyplmXHDpA=; b=AyNCQi9so9bcfCEL/61KemZwZe
	6yi4mvjQh4xGwWr/8kcN27xWHg/ry3AG2m+I7xGxJAWp8Jp/CZybacTxNHGr+bfcs9R987BBtynhx
	W6T/GzylpPIISze/6DhnDDQpUXODizom1ZPib6FmAxdfJzB5SEaRoLov/99m7DRJAldA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165160-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165160: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-ovmf-amd64:guest-start/debianhvm.repeat:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=2c3e83f92d93fbab071b8a96b8ab769b01902475
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 14:17:32 +0000

flight 165160 qemu-mainline real [real]
flight 165164 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165160/
http://logs.test-lab.xenproject.org/osstest/logs/165164/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-ovmf-amd64 20 guest-start/debianhvm.repeat fail pass in 165164-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     14 guest-start              fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                2c3e83f92d93fbab071b8a96b8ab769b01902475
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   11 days
Failing since        164967  2021-09-13 13:06:52 Z   10 days   19 attempts
Testing same since   165160  2021-09-23 01:54:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4691 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:27:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:27:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194322.346164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPhW-00057J-NY; Thu, 23 Sep 2021 14:27:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194322.346164; Thu, 23 Sep 2021 14:27:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPhW-00057C-Ka; Thu, 23 Sep 2021 14:27:50 +0000
Received: by outflank-mailman (input) for mailman id 194322;
 Thu, 23 Sep 2021 14:27:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTPhV-000576-1Y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:27:49 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6bb4ff1b-1c7a-11ec-ba48-12813bfff9fa;
 Thu, 23 Sep 2021 14:27:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bb4ff1b-1c7a-11ec-ba48-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632407266;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=x/gEY9gzTGZhm/wQT/kiNjp1mxgYWbHDqyrq2xP0rQ0=;
  b=SJASOTrx0/OEnKfRzVpGfH1Dfk4PnHrZJWnQtGCAiUwlS6GT5bOfbzfk
   L5fbkiv7EJmvThMFg5EpBwmIYvF+sUoge0M+7jT6YvqJH/HHC3VbBphv+
   fnXszsoz0qMuf2ny0VX6JBigsIG2eBcnyKuRIFWvUO2CmYYoYRcHyBrdd
   w=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: O/jMWOvezTj0FUwOymsiK7vUL7HT0kJHhAuuMOHTnN1fQjuzCDQXadvq+iFPqMWCEl6AVkO3Aw
 ZsDi/Oos1Wd/KvVGs3UmXIu+qNzKRLjLsySk11pvvqxwAFDLQEYWWAVCL2uMugX8CSjksfG9ts
 Qk1gK8Cq5lRL5p4byXHGD6Q2DkuGoNhrnZ8wafhyvvaZl5rtTFxXmoO0ID0QvyBSZELMcPywRw
 v4CocLvQjy/iH619UaXxGYBAibPNiDaSF5pv8SUkcjaVtGbLi39FhCac7U5SFs0JKx8478Xywp
 6AeJDjuIOfe3j0EVCiH7qRJI
X-SBRS: 5.1
X-MesageID: 53862938
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:IXzjb6mrqkDQBihFY2AH/Q7o5gxYIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xJOC2zUbPneYmGnfNkkbtuz80gG7MPQmoA2Tws+/ntnQyMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Qz2YHR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 JJdhIe+e1sMBLDFs8U/eR56HShGYYQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpr1p4UTK2OD
 yYfQX1IaBHsejBwBkk0Aqgzl/rvmXejWQQN/Tp5ooJoujOOnWSdyoPFL979atGMA8JPkS6wt
 m/Aumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3ntZoilCOwm0YCDUNSEC25/K+jyaDt8l3c
 hJOvHB09O5rqRLtHoKVswCETGCs+Q4tQNRiT8gA9Qim64j+5CSXD1BYZ2sUADA5j/MeSTsv3
 16PutrmAz1zrbGYIU6gGqeoQSCaYndOcDVcDcMQZU5cuYCy/d1q5v7aZos7SMaIYsvJ9SYcK
 txghBM3gakaxeUP3r+ylbwsq2Px/sWRJuLZCwO+Y45E0u+bTNL+D2BLwQKChRqlEGp/ZgPa1
 JTjs5LChN3i9bnXyESwrBwlRdlFHcqtPjzGmkJIFJI87Tmr8HPLVdkOu2slfx8xaZheKG6Bj
 KrvVeV5vsQ70JyCN/MfXm5MI55ykfiI+SrNDJg4keaikrAuLVTarUmClGab3nz3kVhErE3ME
 c3zTCpYNl5DUf4P5GPvH481iOZ3rghjlTK7bc2qlHyPjOvBDEN5vJ9YaTNimMhit/jayOgUm
 v4CX/a3J+J3CrajPXWHrdBJfTjn7xETXPjLliCeTcbaSiJOE2A9Ef7Bh7Qnfo1uhaNOkenUu
 Hq6XydlJJDX3yWvxdyiZi8xZbXxc4x4qH5nbyUgMUzxgyooYJq17bdZfJwyJOF1+OtmxP9yb
 v8EZ8TfXagfFmWZo2wQPcvnsYhvVBW3ngbSbSCrVycyIsx7TAvT9966Iga2rHsSDjC6vNcVq
 qG70l+JWoIKQglvVZ6EaP+mw16rk2IaneZ+AxnBLtVJIR2++4l2MS3hyPQwJphUexnEwzKb0
 SeQAAsZ+raR89NkroGRiPnd/YmzEuZ4Ek5LJEXh7O67ZXvA426u4Y5cS+LULzrTY3z5pfe5b
 uJPwvCibPBexARWs5BxGqpAxL4l44e9vKdTywlpESmZb1mvDb88cHCK0dMW6/9Iz75d/wC3R
 liO6p9RPrDQYJHpF1sYJQwEaOWf1K5LxmmOvKpteEiqtjVq+LenUFlJO0jegSNQG7J5LYc5z
 Lpzo8UR8QG+1kInP9vuYvq4LIhQwqjsi5kai6w=
IronPort-HdrOrdr: A9a23:3Nga4qzQ6Ip36wjykFA+KrPxv+skLtp133Aq2lEZdPULSKKlfp
 GV88jziyWZtN9wYhEdcdDpAtjnfZr5z+8J3WB3B8bfYOCGghrTEGgG1+rfKlLbakjDH4JmpM
 Ndmu1FeaLN5DtB/LbHCWuDYq4dKbC8mcjC74qurAYOPHJXguNbnnxE426gYzxLrWJ9dOME/f
 Snl616T23KQwVoUi33PAhIY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX222oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iCnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDw4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWAoqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocXTbqjVQGdgoBT+q3pYpxqdS32BXTq+/blkgS+pUoJjXfxn6ck7zE9HJFUcegN2w
 2LCNUwqFniJvVmGp6VP91xNPdfPFa9CC4kAFjiU2gPK5t3T04li6SHqondt9vaNaDh8vMJ6e
 L8uRVjxDYPR34=
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53862938"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bI2YVqn0Lu+5mKNGVhG5xyPiU95j85s3Hl6VihprBE9a8UBtt9+qn4ycJjIu7Cwaf67vUYBtE0zXQ/rnMIAaQKgf1P+0ik3Q5q2JFnA2pdRnmT7X8be3UwHDUJRY3BGhqNiSeTUbXxWkqAufm+w5fGqwjDcfMv1HQyV5AaRcolm6Vpb5wM0G9BZnejA1XhvdfyvEhnSPzXvHGwPJ4ocCRwO349X63e+plu1rkkFe5YFizrJe/ivWgRCyjaya+g7363Os3MMO0Meix4nr6y+NtlvnUVNX7idgSDM41HlROgOmO5kl2WI6bFKaUsYFFOGhrZP+nEbWNE7sz1Qmbs6iIQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=axzB9TFmmredPYZ3kjovrzEr0FcHwwllGcngRBrn7bs=;
 b=RzmJfZbDFRyt0l4kGBKDYXPF7yiyRZxuNuDrdq0INQMsJukrc4OZ54A3j3ylH9B7wqMwLn3qqTT2rXBkdbH5fQFoai7njNenYPVOiD3q12afSgHE9+h/1dF36Aa8w9EsXTH5ZUnxgs2E/RRWo2jb2FyXdTx+uu6No/Ee4TdT3riTF3Sqw5CyVojfEXb+fQ3Mm/QYVS2Y5xrnRJDSe8J9EiPiaSDk3Zdb7kM+5C1IIEneGKBIkVXsBfvu2NACf2AVPKXjJIHPtozwU/cMXsCnkW1yY2es6jDb1hDWot0OZFJiHxX4AQ2fZAVo8GVpa/l+zTPZPDgA7jTf+VkV4d79Lw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=axzB9TFmmredPYZ3kjovrzEr0FcHwwllGcngRBrn7bs=;
 b=vHzmndbyAPMZLWrS70EAYPjMsNi/4uWANdCKlBM2jcKIN+ggmT5vCZOF1We3qlvF4k/HLVVp9QzfROU2ISg2xItN0ZRz9SWrUu3BBrH8c5dcj0BqxAouTnKrkyem7cwJffpQIteXym4fZdko5KR26fcBysOhM9tGXkW4eeL4vLs=
Date: Thu, 23 Sep 2021 16:27:12 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 5/9] x86/PVH: actually show Dom0's register state from
 debug key '0'
Message-ID: <YUyOwBNNa6Go1yrW@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <4ae1a7ef-d6e8-75db-ddf5-987f10175196@suse.com>
 <YUtQPUHjZ8GnfeCE@MacBook-Air-de-Roger.local>
 <d58fdb48-0452-321a-5ded-d949740682c5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d58fdb48-0452-321a-5ded-d949740682c5@suse.com>
X-ClientProxiedBy: LO2P265CA0205.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9e::25) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b7f6d310-a131-4faa-eb88-08d97e9e3f44
X-MS-TrafficTypeDiagnostic: DM6PR03MB5321:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB5321020CD9A02A664B3F65918FA39@DM6PR03MB5321.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1BemFRzilzvcHGRUZh/vDy4Bau9/Rr/kTsQcVqWeaa4p5oLhcpdNmq3Kx1wNz3t4SICi+9h38O3Pg4f6bQ52cEUhLao0T1Ca5KgI39SylF2D6QzNlddJgqhcnzjxcoIGNEHY3dGYsorfxT1Dcremhmy3pRCQ1U59ldsrAu5WCJCsHxbLKtSZeTskatmbXYUYgZKFyhUMYtw/QlWxwMZYeiXeAQLTJIxRblh5N+yUVeYdP2xRHrWki7ywCfvqVxtH3PQAzzrh23GVH/YgNlr7qcXIBo28eRFrsRzoUrAqPggetYa1OxGTU7mcVimYDG4LQ+ETZ62JUV7cKA1ECBjh9H6ourNODWjG4TqS3ujU4BQtwEFTGjZUjyQsFQZJkZsIQAyviOqzMutz27cZr2b1B2p0PXOMuMsNEmyc5M9ykkuLNs3tVvR8ecfi2aatgci2vBAcancW9FZZcX0J3k0u+b+eA5SCbbtF2JPwuUjR3aD4Bq9yJAPbIV2HD+T07UE/AK2jXzqlA3jXO0DSGOSIRdDByOWCsRkcgqdpx7oZoZIH9O0JDF6UBvEVjfqgDzLX2bI2UiDwd9nv7JN1x2w2FI3ly+FdVmRwljOR9pmTG1AiotlHZLX29W94N7pC5gTVzOXS9oh5R2MG/T2lQW0sHg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(9686003)(66946007)(38100700002)(2906002)(6486002)(66556008)(66476007)(85182001)(8676002)(8936002)(956004)(508600001)(4326008)(83380400001)(54906003)(53546011)(316002)(6496006)(6666004)(26005)(5660300002)(6916009)(186003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QlN2N0xNcWRiZVFtL09heXJCbVpycjRhK3R6NnZMaFdLU0VyaHdqVmo0c0U0?=
 =?utf-8?B?eGhhRXZIWEFVczcxcU5rZlZXSmdmQTJhUGp4SEVtR1o5bVMzK0ZyZ1VQVDhI?=
 =?utf-8?B?OEp6ZWlZdFgrcW1JT2VhZ2lFZGFlYUNIanZWSHdiRVJ6b2NCZlcvS3dMa3NO?=
 =?utf-8?B?OTNQOFlWT1FPeWFIQk5lRVcxYkpsUG1jcmNmQkVCZytuQkRxeWtKdlNHQXNz?=
 =?utf-8?B?RndBME5KZXMxMUN4MjBTcTRoQldOdnNlQkNOTStyQjlKYjVDOS9QMFRqZHhB?=
 =?utf-8?B?U2hFVUNaK29qcU1PMjU1d3VhSVRnL25pRTVraVpqYW1pRTcxd2dnRFZsR2Js?=
 =?utf-8?B?eEU2cU1XbWsvY2w1cmpsL3ltMm8raHBJYmMxaHo1blpUMVBXdnNLcjJBc05K?=
 =?utf-8?B?SHlXUEd3MXZBLzF1SVk3L0psdXNOUXgvc1I3QSs1UC91UVVORHEvUGluVlRM?=
 =?utf-8?B?alI4c3NSUzJLTmtqRklKOTNoVmRwSUJENk5PNkc4QkNMb1FWRUhPL0x4SXVP?=
 =?utf-8?B?c09PUVlsc2lrWEJ1UjhBTVB1TDFrK3dta2N0dWMrMi9BczQxM3FKcmZOdXBw?=
 =?utf-8?B?czJFOUYycjREblFrTXRZQk8zTzBDd0o5L2hxZ21KS3FMQXM4aCtHYTBDU0Rt?=
 =?utf-8?B?U21rNTlBVWpiZlp5SGR6T2dmYmEzTGFOZVBPRzFLcEx2VEZ2OWdnWUsxZXJa?=
 =?utf-8?B?Z2JLVmNrMldCZExZdU01WEd5ekJFcTZBTE83R01Ub0NIN1ZXSTh3ZDArNmZv?=
 =?utf-8?B?OXp6WUQ5OWJyV2FRTFNpTUNpRGh4N090RE5EQmYyMitBb3NFdTRVdlkvck5C?=
 =?utf-8?B?aU1uT3JHczE0WjN4MFBvb3lWdlozOGtjV29SbDFvbzhmWmM1UGM4b3M1L1cr?=
 =?utf-8?B?dnhNOEIyU012RlZHQWNRNDFBUjhubnJqT3BZa1Z1elVtMnltbDVZRFJ6Qks5?=
 =?utf-8?B?VDVmQlY1RHFzQ1FSdGUwWGVDQUY5ZlkxcGduVHEvVE04djR2V21GS0FoanMy?=
 =?utf-8?B?QW03eGFDRUlicUVDRTF3bHdJeU4xTzRCaEF2SkwveFE3aDY1V05hK2FJUkJn?=
 =?utf-8?B?ZnRoWndHVXU3TWw4Ti9lUWtPc1hSSGFhc0pvaW5BaGYzWjZsOFkyK0ZzaThl?=
 =?utf-8?B?cEFLaHVsN1RmR3RuSlJ5eXhJZFEyVGRxK3B0OXJ3b3hRVXZsdm5kK3lxMVhY?=
 =?utf-8?B?WnFGd0pyVGFCcmIyY0VnbGVONW16NmltL3dxaG5HeXlwVGJtZGg1U1pPTGZw?=
 =?utf-8?B?NHVtWGRqdUJacldoYjV2TUlEcFpqdHdIM3dtRWJCWFFVWEVuZnd5d2pqTnZ2?=
 =?utf-8?B?dXBudkFUREdXOVI1c1RPTEQ4WllCNGE5SlRVNHBucWRPSUdBS1F0bVNnTWRW?=
 =?utf-8?B?R3o1eDFFVGZJRmc4ejA2UVoxNnhtV1RrdjVkL0VPbXVhc1RnczJEMElMTi9J?=
 =?utf-8?B?WlcvVDl3NmZVaUc3K05MSU1FVUdpSmQyTjBmYUpXOStPNWNHeWVBTmlpYmN1?=
 =?utf-8?B?djB2Wk9Bc0Vrd3oxa01iWTllWGJlSEJDdDh5d3liOUxDaldWZWJQYStXTG1P?=
 =?utf-8?B?bzd3K1Y5ZjVkSHZtc3JCa2Ryellqa2tyQk15cjRpOWttalJEZ2EyS0dDMEJW?=
 =?utf-8?B?YVNUQzNTS0FoZkM4MktKQTJYblhrU2E3bFM1WXE1MG45OSsvWkJnOUs3bjFS?=
 =?utf-8?B?UW5URG5aU2YxUUEvdVlBRWVISU4vOFZ2TnVSY25Id0Y2d2M5TTRFRzZUbW13?=
 =?utf-8?Q?dwM+PT0FudV8uNwe8t9L9tAqCX509iXsIzwXxi4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b7f6d310-a131-4faa-eb88-08d97e9e3f44
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:27:18.2871
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DvQpqLmbDxBJoCb5CPGOYEoUo1vIrdCUN/cSBElCT1oJIXOIDh/NQ8MbwiswlvTrtNiFGwl4VGDhPoPhReq+Sw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5321
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 12:21:42PM +0200, Jan Beulich wrote:
> On 22.09.2021 17:48, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:19:06AM +0200, Jan Beulich wrote:
> >> vcpu_show_registers() didn't do anything for HVM so far. Note though
> >> that some extra hackery is needed for VMX - see the code comment.
> >>
> >> Note further that the show_guest_stack() invocation is left alone here:
> >> While strictly speaking guest_kernel_mode() should be predicated by a
> >> PV / !HVM check, show_guest_stack() itself will bail immediately for
> >> HVM.
> >>
> >> While there and despite not being PVH-specific, take the opportunity and
> >> filter offline vCPU-s: There's not really any register state associated
> >> with them, so avoid spamming the log with useless information while
> >> still leaving an indication of the fact.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> I was pondering whether to also have the VMCS/VMCB dumped for every
> >> vCPU, to present full state. The downside is that for larger systems
> >> this would be a lot of output.
> > 
> > At least for Intel there's already a debug key to dump VMCS, so I'm
> > unsure it's worth dumping it here also, as a user can get the
> > information elsewhere (that's what I've always used to debug PVH
> > TBH).
> 
> I know there is a respective debug key. That dumps _all_ VMCSes, though,
> so might be quite verbose on a big system (where Dom0's output alone
> may already be quite verbose).
> 
> >> --- a/xen/arch/x86/x86_64/traps.c
> >> +++ b/xen/arch/x86/x86_64/traps.c
> >> @@ -49,6 +49,39 @@ static void read_registers(struct cpu_us
> >>      crs[7] = read_gs_shadow();
> >>  }
> >>  
> >> +static void get_hvm_registers(struct vcpu *v, struct cpu_user_regs *regs,
> >> +                              unsigned long crs[8])
> > 
> > Would this better be placed in hvm.c now that it's a HVM only
> > function?
> 
> I was asking myself this question, but decided that the placement here
> is perhaps at least no bigger of a problem than putting it there.
> Factors played into this:
> - the specifics of the usage of the crs[8] array,
> - the fact that the PV function also lives here, not under pv/,

I think both functions should live under hvm/ and pv/ respectively.
There's nothing x86_64 specific about them in order to guarantee a
placement here.

> - the desire to keep the function static.

Well, that's obviously not possible if it's moved to a different file.

> I can certainly be talked into moving the code, but I will want to see
> convincing arguments that none of the three items above (and possible
> other ones I may have missed) are really a problem then.

As said above, my preference would be to move those to pv/ and hvm/,
but I can also live with the current arrangement.

> >> @@ -159,24 +173,35 @@ void show_registers(const struct cpu_use
> >>  void vcpu_show_registers(const struct vcpu *v)
> >>  {
> >>      const struct cpu_user_regs *regs = &v->arch.user_regs;
> 
> Please note this in addition to my response below.
> 
> >> -    bool kernel = guest_kernel_mode(v, regs);
> >> +    struct cpu_user_regs aux_regs;
> >> +    enum context context;
> >>      unsigned long crs[8];
> >>  
> >> -    /* Only handle PV guests for now */
> >> -    if ( !is_pv_vcpu(v) )
> >> -        return;
> >> -
> >> -    crs[0] = v->arch.pv.ctrlreg[0];
> >> -    crs[2] = arch_get_cr2(v);
> >> -    crs[3] = pagetable_get_paddr(kernel ?
> >> -                                 v->arch.guest_table :
> >> -                                 v->arch.guest_table_user);
> >> -    crs[4] = v->arch.pv.ctrlreg[4];
> >> -    crs[5] = v->arch.pv.fs_base;
> >> -    crs[6 + !kernel] = v->arch.pv.gs_base_kernel;
> >> -    crs[7 - !kernel] = v->arch.pv.gs_base_user;
> >> +    if ( is_hvm_vcpu(v) )
> >> +    {
> >> +        aux_regs = *regs;
> >> +        get_hvm_registers(v->domain->vcpu[v->vcpu_id], &aux_regs, crs);
> > 
> > I wonder if you could load the values directly into v->arch.user_regs,
> > but maybe that would taint some other info already there. I certainly
> > haven't looked closely.
> 
> I had it that other way first, wondering whether altering the structure
> there might be safe. It felt wrong to fiddle with the live registers,
> and the "const" above than was the final bit that convinced me I should
> go the chosen route. Yet again - I can be talked into going the route
> you outline via convincing arguments. Don't forget that we e.g.
> deliberately poison the selector values in debug builds (see
> hvm_invalidate_regs_fields()) - that poisoning would get undermined if
> we wrote directly into the structure.

The vcpu is paused at this point, but I agree should not undermine the
poisoning. I assume those fields don't get filled because it's an
expensive operation and it doesn't get used that often.

Long term it might be helpful to have something akin to
guest_cpu_user_regs that could be used on paused remote vCPUs.

Anyway, I don't really have much other comments, so:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:29:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:29:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194327.346176 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPin-0005iP-2y; Thu, 23 Sep 2021 14:29:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194327.346176; Thu, 23 Sep 2021 14:29:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPim-0005iI-VQ; Thu, 23 Sep 2021 14:29:08 +0000
Received: by outflank-mailman (input) for mailman id 194327;
 Thu, 23 Sep 2021 14:29:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTPim-0005iA-7P
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:29:08 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5700156c-078e-4655-9f36-d967359d60cc;
 Thu, 23 Sep 2021 14:29:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5700156c-078e-4655-9f36-d967359d60cc
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632407346;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=fpJmtosQkJuDZnBHK4mfaF9DrubibZc9iSd5IWVmdFk=;
  b=DOZ/wLdJauGAnszjg7ooGBr1a1iuvw6f0CBUPPlav+uubOkYVqF/VHyo
   S6xwLKaAmJnckehq3GDNmt/9mJzKcPiwIdWz7MwAMmwn/Nah1nTvc6tka
   oGx9rG+NO/CsuVNUwbOT+z8uJE/hAcg4cgwQ6RNmKHD4n8IwT8xpAPi35
   A=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 7IfbeGX0JCuLGRorB6pmow2vVVhXJ/8YS/eRR3uTqL1xpK4VED2/Sf2fGlTMsJ0VBzZGKbjxMf
 IRcKDM+mjaAYG5QVIy5ps51mF8p6Km9bpZiuqhTxyQzOxLrMYZL46VjpsRXRJo/Ct+N5yiGgb9
 AB1pkg+utO8S1zm6twZDS5UM4QpSI7ExTSNltYKPvoxduIvdRwmWZHZr5X/MuxfSeOdMy3NeKu
 hdMt12abBVd+I7TYUli6iyqa7EIfLn7k3n+uzmDxQ4RvrU0WZGePYtnpnO+pNREGEqXPELoW4O
 9sdUTY3phovAz9RHYZO+6DZo
X-SBRS: 5.1
X-MesageID: 53466486
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kmtP4KvHueiG7jJgAKzZXRBDGufnVIhZMUV32f8akzHdYApBsoF/q
 tZmKTuGa/uNZGanc4h+a4+y8hsEsZ/UzNRqHVM/rihjHihA+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524PhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpltJbuZSF5LIn32/lEUzdzNWYuDKF/weqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sSQ6iCN
 5RxhTxHVDnMSA1qFFcrOa0bjN32lCPUKyFggQfAzUYwyzeKl1EguFT3C/LKfvSaSMMTmVyXz
 krW8mK8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECqF+Zy3EXCRYWfUCmuvT/gUm7M++zM
 GRNpHBo9/JrshX2EJ+tBHVUvUJooDYdUYR8Nb0ozDrO1/PUzS2/QTg1Qgd4PYlOWNANeRQm0
 VqAntXMDDNpsaGIRX/1yop4vQ9eKgBOcjRfP3FsoR8tpoC5+dBu0kunosNLTfbt5uAZDw0c1
 NxjQMIWvLwVkcdD/KGy51mvb9mE98WRE1ZdCuk6WAuYAuJFiGyNO9fABbvzt68owGOlor+p5
 iNsdy+2trxmMH11vHbRKNjh5Znwjxp/DBXSgER0A74q/Cm39niocOh4uW8lfhwyap5fJ2W4M
 Sc/XD+9ArcJZxNGioctP+qM5zkCl/C8RbwJqNiOBjaxXnSBXFDep3w/DaJh92vsjFItgckC1
 WSzK66R4YIhIf0/llKeHr5FuZdyn3xW7T6DFPjTkkX8uZLDNSH9dFvwGAbXBgzPxPjf+1u9H
 hc2H5bi9iizp8WkOXSIrd5PcwpaRZX5bLivw/Fqmie4ClMOMEkqCuPLwKNnfIpgnq9PkfzP8
 G37UUhdoGcTT1WdQelTQnw8Or7pQ7hlqnc3YX4lMVqygiBxaoez9qYPMZAweOB/puBkyPd1S
 dgDetmBXasTGmiWpWxFYMmvtpFmeTSqmRmKY3ivbg8gcsMyXAfO4NLlIFfirXFcEiqtuMIii
 LS8zQeHE4EbTgFvAZ+OOvKixl+8p1YHn+d2UxeaK9VfYhy0ooNrNzbwnrk8JMRVcUfPwT6T1
 gC3BxYEpLaS/99poYeR3a3d9tWnCepzGEZeDlL317fuOHmI5HenzK9BTP2MIWLXWlTr9fjwf
 u5S1fz9bqEKxQ4Yr4pmHr935qsi/N+z9aRCxwFpEXiXPVSmDrRsfiuP0cVV7/Afw7ZYvU29W
 16V+8kcMrKMYZu3HFkULQsjT+KCyfBLxWWCsaVreB33tH1t4b6KcUROJB3d2iVSIYx8PJ4h3
 ep86tUd7Bayi0ZyP9uL5syOG79g8pDUv30bi6wn
IronPort-HdrOrdr: A9a23:zzjD8al8takgczKmEU6GrXKrRYHpDfLW3DAbv31ZSRFFG/Fw9/
 rCoB3U73/JYVcqKRUdcLW7UpVoLkmyyXcY2+cs1NSZLWzbUQmTXeJfBOLZqlWNJ8SXzIVgPM
 xbAspD4bPLbGSTjazBkXSF+9RL+qj6zEh/792usEuETmtRGt9dBx8SMHf9LqXvLjM2fqbQEv
 Cnl6x6jgvlQ1s7ROKhCEIIWuDSzue77q4PMXY9dmcaABDlt0LR1ILH
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53466486"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YI1K0F2T0sC6McBTlBzRg0LeiT993zlVtn0sC/nQgqFL0SBRAzqt+8o8cBZ8hb80I7N2VK+Wujr8uFnrokE643l75+nd5QLvtJK8Q2M4+qD/499XhlQqxZ4jHMT3G0dT1DYRBdd8zIlrGSfXVUUfqNCJKBh7OcB3YBA1oeJGQG7Ap9YG9VZ4efOQwW9vHDK2TFXg/mMziBTUn1oJddXqI+6opqtYJ792Pqk2Q/T9DP/jfVclNPxzNibqK/TKHtXWYenlyk0PROtuuDukuNPrQ729m3scoEjSVpFMuXQBBEJl86ffDaQiGtlKx2j1AShVDq3qVWcy+tTGDnMdyoKOKA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3GGkmR/LRh2cfPblNWsv8n5RSfXbfLzVOUnqQuL1NSI=;
 b=Q/hf/3JryyKFPNahtJhoUtsdWGDQ8/RWp8SWzRkGLbvC8QWn90NXSp6zAaQuf0ZvMoPJ1WVdm1K+2nTnOFEY16ehCpjVqy1Q+bn6NrVCO82K0+pZf979PHlJC1xGo71PIW/PIlTZOmxEVmEfa6ZbPkxQbvg0nbFWp1JOdn6QTKAIPIsNCvfLScTDQSJbj9WXeXmwCieEc9YF1uX8EqbaMx7UYVc+ZdlotWt8IQDAbD00hGx//8dukS10lKvq9z7f3SIBXv10SmtMiudJs8uXR7xrIsNv9+pgxCerQq8lRI+gqRJiHDqC6EwKOlg15OesDlRqdWZKEhpxqYbZ7kB1dg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3GGkmR/LRh2cfPblNWsv8n5RSfXbfLzVOUnqQuL1NSI=;
 b=Hy2rYIIdt2tmML9MRMLxsunIPDuywkVbQZcNe8y66wErgMv4/5hY8lvvXgu91qMfJ003zyzQzBzNboknRAtWWhNat/NymggCuFSh4M/9j+DfU8U3iGt4mg0eVBEvAcQJm8DjD4ii1ROE0V65JL6CY0ewLmhI1YN/SrlmYQEKRf4=
Date: Thu, 23 Sep 2021 16:28:58 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 6/9] x86/HVM: convert hvm_virtual_to_linear_addr() to
 be remote-capable
Message-ID: <YUyPKjWdYzCsrBQc@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <3397b5ea-bc21-5a2c-ba5d-afc974a92df4@suse.com>
 <YUw2RLtI73QNjaPE@MacBook-Air-de-Roger.local>
 <46f90ac6-a4f0-d00a-2956-a0e4b351f698@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <46f90ac6-a4f0-d00a-2956-a0e4b351f698@suse.com>
X-ClientProxiedBy: LO4P123CA0138.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:193::17) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9d6ac4d9-a35a-4158-16ce-08d97e9e7e14
X-MS-TrafficTypeDiagnostic: DM5PR03MB3370:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM5PR03MB3370D494467D81A4156D338F8FA39@DM5PR03MB3370.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: KglShhpYKZuX8AwLvRN6msBPGGmPMLWwaMz296dZXX9riAkt3KXkQPNFtPH/gnrmRl+AJNkk2+qUOn1lO9Ooyrq5cC3VQdB1pIdKQDF2htWiUENFTFfzZl0gZj4pv4Owo9/Oml5d4kRAoPxnTwLcNFa6J1nbJT42OhQYENjTMbsCDcjnyddc3DrrsaA/Cy+9MgvTOMgLMIcaD2tB+xir1lwjrozGDyuqPTK5wyIVVz0dt3tc8IGj1Nxvz7OGdL/bbtBPcMs3baQ0ElOI6Phw6zKJj0U7q0F0uAXpGWVJnj1HONrfbLI40p1+Ds7IsGbjy1foVY1p0mH13JRNUAtLtxMfVj9xUzE8xd8xBUzU8cm7VS5QL43cffBXcC3LzjSFNi3sGnlFO7/wtsXbLDWI3Zu3OT/kuuytTKt8zIlGzKwzYT1Nexq3v1tKPHQOhcOVzbJav/s0AQKHVsHVtIgPFHKfPDB1zdEOV8k628CA7XW6BcltvlibT/Az8b0ZQ4w5D1E3kktmVz/C1q62BFjEkyj6+L2Cqa+dtHtNnuHAIYD7L6/UULdrJiuikNE/T9LNA4th1dtJVvwmKMvL5muGuYqd0mnJqZCG9gyZwaGvpMu4Fs8/EpyFHxnK1nbYpukfQmCDEn5Ya9Y4Y7QloSeVAw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(6496006)(508600001)(6916009)(6666004)(9686003)(85182001)(38100700002)(86362001)(8936002)(54906003)(956004)(66476007)(66946007)(66556008)(316002)(8676002)(26005)(186003)(6486002)(5660300002)(4326008)(53546011)(2906002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZVVqUTJlMDgybUk0VDlSdUtiL1hOd202bE1UK080TWdFY1V0ZHlNa3M4TUcw?=
 =?utf-8?B?T1VPcmswQWJOaXNYMlVRWmUxVnFVZ0l6YWtScnNkUmZ1WmZUNzlkMFlnOTE3?=
 =?utf-8?B?Zi9VRzFXYldYVTdpMjVOT3lDck13Y3F0Y1NFak8wb1ovcnU0VzNKR1NjTEFT?=
 =?utf-8?B?SVBFaUo5S2Z3UndkUjZZcnVHRzdRZkxlVmRmOVRoNWNtTXc2TTVBekx1WW9E?=
 =?utf-8?B?UUlrUmg1aElxOTM4UzFPNitvRXZSaDZaQWxLSUdEZFNxdzY5SGVmelQvWkxx?=
 =?utf-8?B?OHgwa3BhNkZ2eGUvRXhxV3lyZm92aU1oZVo0ZnJhUFZ3RTN3aGcvdUpINnEr?=
 =?utf-8?B?M2EwWFd4blBkWlh4Wm93ek5RcG1KdnU4QmVSTjJXY3pjNStUbCt4TTZ6TUtF?=
 =?utf-8?B?c0JkM2RjbmNsMFpna0RscU1OMGE3ekFiTnM1SEtJM21Na3A5UmV4M2hjeUxn?=
 =?utf-8?B?MjBvWWxJNWZUaVlCZTBHdUMvOW1PamRTNllZdVN6dkZIbER2ZDNRN0tlOGY2?=
 =?utf-8?B?VHhRdG5sSkt2SEh4ZUQrZFE1TXMzY2pOZ0dBcXY2Z0NmemVtVWdwQ2NIcHJ6?=
 =?utf-8?B?a25PRVdwYVFTZjlHaW1SR1hYekgwdy9wc2l6OGlhSWFIUzhIRENZU1JML2VH?=
 =?utf-8?B?YzNRU1pBbmprYWhMQk1KWFNtUWwxV2RoVzE5dmhLNzU3aklMM01vSW0rYzBz?=
 =?utf-8?B?RWtTNlA4Z3orNkhRdjlvU29GUEw4SmJDamQ3YllyQ25haTkvYlJYeDMvYlJP?=
 =?utf-8?B?ZFMybFNPNzNibXZBcEZTKzZYZ2RRUUZwSXFrS1hFbjZ6UVBGbk02OVNLVzFS?=
 =?utf-8?B?Mmk5L0FXNlRPd3RkR2FQR3pNQU5NY00rRld1cGlCYzM5ZmtBZnBaMURRMlR5?=
 =?utf-8?B?M3I1NmdOVXlLcm90TXlQM1VLUjdxQjNtb0ZRZzZETTgxNGhtdUl0bnFtZzFh?=
 =?utf-8?B?SUZQck9zd1NNZ01FWno0Z3drKzd5czRxM0x6QW9BUU0xdHdONWFPNHRFYTY1?=
 =?utf-8?B?bjVLaVRLanh0YUJ4bFc2TkVLV0tacGt2QjBvY280TXR4N3lpR2poZ21VVXpi?=
 =?utf-8?B?K29pci9PWklMQU9Rdk5LNU1pVWRuVUxiV2tsc1BWMHRERWFWbkZVMWN4UFgz?=
 =?utf-8?B?alA3UmxtQ3p6aVZ4cWNLUWtHYW9LUTVzRHJsNk9BTzJyaWplYVFXRHFOd0Qx?=
 =?utf-8?B?RGdHcVJNdnhwRHhhbmdtODA1MjNCZUt0Uk9kV1hyQUt3VVdhcGNCbFhpU3pE?=
 =?utf-8?B?SkFqVGg3T3FSRXBIcmFVY1c3M1hOZVFVbkxJWXpYdE1rZ1BPb0FEM1BaVVpL?=
 =?utf-8?B?U2RvcmdIdG40bWdERTIzWUxWVldnR0RxQU5USmNqVjY1N3g1eU1wVHhZRitS?=
 =?utf-8?B?dzVnL0dNdlkxY2tWejVsSUpaTzJySmtUNVdGdWNrT0FZY3RnaWFUb002dldj?=
 =?utf-8?B?MUhHWnNUYmRvNitoM2FOdWg0NkkwWUY1VGc5aDhycDE4aytpL004cGRFSXVw?=
 =?utf-8?B?VW1hL3JkRVFaelEzME5GeGp3T2ZydXIrelM3dWJOY1l2U0swQjY1R2lBREFw?=
 =?utf-8?B?NzlSbDlqUlN0dHZsdTFydThvZXVjTmFFaHdVbE92cDNib3VnUFVrTlRxQ1lI?=
 =?utf-8?B?ZWtsZCtOYkhlSkgvRy80VkVGZUsrUG5ncXFKRzZDeVVkZUxMSzlBMjNBWHlD?=
 =?utf-8?B?anRTQlJVZFVaSnV1RWFTeExjV1BGdFB4Z3pxSGdNOUJIY09LWkVSQWp1clVU?=
 =?utf-8?Q?swSFqoN4dP47DwK/QwCMja71DE3m5DABDLdKfPf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9d6ac4d9-a35a-4158-16ce-08d97e9e7e14
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:29:03.6745
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7nCqYkvcOhQSBXEbt64xRrL5RFM/yuQI0rcSMWrhz4jyTelWbURHTuKn8pxLSnZIprDVtB6tCxvUhmbo7yHm/g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3370
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 12:34:48PM +0200, Jan Beulich wrote:
> On 23.09.2021 10:09, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:19:37AM +0200, Jan Beulich wrote:
> >> --- a/xen/arch/x86/hvm/hvm.c
> >> +++ b/xen/arch/x86/hvm/hvm.c
> >> @@ -2526,7 +2526,8 @@ int hvm_set_cr4(unsigned long value, boo
> >>      return X86EMUL_OKAY;
> >>  }
> >>  
> >> -bool_t hvm_virtual_to_linear_addr(
> >> +bool hvm_vcpu_virtual_to_linear(
> >> +    struct vcpu *v,
> >>      enum x86_segment seg,
> >>      const struct segment_register *reg,
> >>      unsigned long offset,
> >> @@ -2535,8 +2536,9 @@ bool_t hvm_virtual_to_linear_addr(
> >>      const struct segment_register *active_cs,
> >>      unsigned long *linear_addr)
> >>  {
> >> -    const struct vcpu *curr = current;
> >>      unsigned long addr = offset, last_byte;
> >> +    const struct cpu_user_regs *regs = v == current ? guest_cpu_user_regs()
> >> +                                                    : &v->arch.user_regs;
> >>      bool_t okay = 0;
> > 
> > Since you change the function return type to bool, you should also
> > change the type of the returned variable IMO. It's just a two line
> > change.
> 
> Can do; I would have viewed this as an unrelated change: While the
> first change needed indeed is on an adjacent line (above), the other
> one isn't.
> 
> > Also is it worth adding some check that the remote vCPU is paused? Or
> > else you might get inconsistent results by using data that's stale  by
> > the time Xen acts on it.
> 
> I did ask myself the same question. I did conclude that even if the
> remote vCPU is paused at the time of the check, it may not be anymore
> immediately after, so the information returned might be stale anyway.
> I further looked at {hvm,vmx}_get_segment_register() to see what they
> did - nothing. It is only now that I've also checked
> svm_get_segment_register(), which - interestingly - has such a check.
> I will copy the ASSERT() used there.

Thanks, with that:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:36:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:36:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194332.346186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPpy-000787-S4; Thu, 23 Sep 2021 14:36:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194332.346186; Thu, 23 Sep 2021 14:36:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPpy-000780-Ou; Thu, 23 Sep 2021 14:36:34 +0000
Received: by outflank-mailman (input) for mailman id 194332;
 Thu, 23 Sep 2021 14:36:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTPpx-00077u-GJ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:36:33 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d9ecf389-9a48-46c0-b7a3-1e211e02ec1e;
 Thu, 23 Sep 2021 14:36:32 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id g41so27440712lfv.1
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 07:36:32 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id e1sm608964ljp.114.2021.09.23.07.36.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 07:36:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9ecf389-9a48-46c0-b7a3-1e211e02ec1e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=kcm7GQxPRLl4JaE3RrEZ0xk8j4QAy1OICAxW3VEdfUg=;
        b=pqP9KCkdBel26c0mkwZRwjljeX4ricLoRDihcrcJIxnpX81IJTp/H1Z3ePseMa8jI6
         +iZd5GqwMLywyMGFhbYbaTtCVdxHzZzym2W2aQrTZP86oDGEbJgDpNDGv5p8AyPiuBhZ
         LfUfzRsL0m4K3p0oL0wa/jaK5K2DyxMIq70lfp7imqdLPnDrwSgW2UgKyzJg74DcJWhG
         9pZuY3Ii5N6Rc8e21axgU1yAYS24IdxixukK3L+vho29JkQ0Iv8Gtuk7lHtoLHTCMWHo
         yPoamBnZrf4gVy4TfsQXQ9WZ8Z+rDM9ZgV9GTkemrT2EaGo1Ktd8IcAvLbIXmVL4LWyT
         AQfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=kcm7GQxPRLl4JaE3RrEZ0xk8j4QAy1OICAxW3VEdfUg=;
        b=W+Ke5BNcrPJ6omCzvor9m9gKPQ96ETK4gCO7a7G3rVCvDqe+E/1Nmqsqc6sn9iHVo0
         9rcQ+m9t33j7y7fG1V694FBB75oVAYr7UqhyQOfCua1hP69dprTtAONDKOUci8/cxM8i
         fQSLlFropzEQG61iuLHuCDwIH4b7ONxm+vz0Cs7pt2ojP0CWAnKvsynwmR39zEA8SFWu
         PHxk4FsOl/g2igv4LtGrD1Tu6KFnrIvwz0OFMUxJbWGvIlMwO7CCIxey2WcupWFNLHm4
         MFJ/9Ms1cuGkfEoahhucpoYxLcggANXmYpQhJDfs9gBcdHq0RwjSbYdrN2HKwQ5zTzNz
         uprA==
X-Gm-Message-State: AOAM5314QFB9wi8htzI0mtQXWW0Ktxqp7LA+TBiPMzGT2xGMV2Du/LfG
	F8SiBVOzuW2IR/4JsX58BeQ=
X-Google-Smtp-Source: ABdhPJzv4n6y2GmcfYIh4P4WiBxwTEj1dvgifhmjEXv/uipewKxSwhLCTAQtyI2AN5E5ajchZlqOSg==
X-Received: by 2002:a05:6512:e98:: with SMTP id bi24mr4301005lfb.497.1632407790744;
        Thu, 23 Sep 2021 07:36:30 -0700 (PDT)
Subject: Re: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
From: Oleksandr <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Julien Grall <jgrall@amazon.com>
References: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
Message-ID: <5c1b7d53-2a12-cff5-d3a7-284e44bdef27@gmail.com>
Date: Thu, 23 Sep 2021 17:36:29 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


Hello all.


Gentle reminder.


On 06.09.21 16:42, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> Allocate anonymous domheap pages as there is no strict need to
> account them to a particular domain.
>
> Since XSA-383 "xen/arm: Restrict the amount of memory that dom0less
> domU and dom0 can allocate" the dom0 cannot allocate memory outside
> of the pre-allocated region. This means if we try to allocate
> non-anonymous page to be accounted to dom0 we will get an
> over-allocation issue when assigning that page to the domain.
> The anonymous page, in turn, is not assigned to any domain.
>
> CC: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> ---
>   xen/arch/arm/tee/optee.c | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..83b4994 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -410,7 +410,7 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
>       if ( !shm_rpc )
>           return ERR_PTR(-ENOMEM);
>   
> -    shm_rpc->xen_arg_pg = alloc_domheap_page(current->domain, 0);
> +    shm_rpc->xen_arg_pg = alloc_domheap_page(NULL, 0);
>       if ( !shm_rpc->xen_arg_pg )
>       {
>           xfree(shm_rpc);
> @@ -774,7 +774,7 @@ static int translate_noncontig(struct optee_domain *ctx,
>        * - There is a plan to implement preemption in the code below, which
>        *   will allow use to increase default MAX_SHM_BUFFER_PG value.
>        */
> -    xen_pgs = alloc_domheap_pages(current->domain, order, 0);
> +    xen_pgs = alloc_domheap_pages(NULL, order, 0);
>       if ( !xen_pgs )
>           return -ENOMEM;
>   
> @@ -938,7 +938,7 @@ static bool copy_std_request(struct cpu_user_regs *regs,
>   
>       BUILD_BUG_ON(OPTEE_MSG_NONCONTIG_PAGE_SIZE > PAGE_SIZE);
>   
> -    call->xen_arg_pg = alloc_domheap_page(current->domain, 0);
> +    call->xen_arg_pg = alloc_domheap_page(NULL, 0);
>       if ( !call->xen_arg_pg )
>       {
>           set_user_reg(regs, 0, OPTEE_SMC_RETURN_ENOMEM);

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:43:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:43:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194340.346198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPwq-0000Uj-My; Thu, 23 Sep 2021 14:43:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194340.346198; Thu, 23 Sep 2021 14:43:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTPwq-0000Uc-Jv; Thu, 23 Sep 2021 14:43:40 +0000
Received: by outflank-mailman (input) for mailman id 194340;
 Thu, 23 Sep 2021 14:43:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTPwo-0000UD-O8
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:43:38 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id aedf81eb-4966-446c-a304-5ea23a3db700;
 Thu, 23 Sep 2021 14:43:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aedf81eb-4966-446c-a304-5ea23a3db700
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632408217;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=XJvEERLCN/XxEr4BePaaduZAx41IlMhjyB+i5cYk6AU=;
  b=KW0wN6a8UZR2qjWr3p/SLyIWPAekX+ZC+9SlnG6ZiRN3MgndX/B0zBsv
   3ibFqkWz8Buq/VkltrDpHQlq9mltx4HZUaCxX86FrHopFKZwE92AGW5PT
   UvAN75+8BpzHe6OocAGC94TSDDvtQCYTM1aq8LOxvgjRxRdPitMJfddON
   E=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: G11mm8YVrmt3AbN3jnhqJSiNH0WcFi1DiWudyZi5WJTUbqSxE6rlHFRAYebxiUVSYFCeNqWXzr
 9QpKyFxInjBv6YaQHHZJv1p1bzF7fxhI7M3YXyLfJk0uzL5mukAEeniVKMA43/R3jqT0qA2DGN
 XD1ZagwvcK0yOu+XCkWeGQs46nrlHBtD+MIA2zUq7B3vP4HqPLE+l+CpFchTjNWWiF2Kz//ijW
 TjIZfzt86fULGYVf9icFAcSj88mF2XzBGtXqYoiY9eNhMbhgY+LZ829/br9RibrRy7rsGZKH3B
 Dm8Wb41V/uoUxIUxl9/YNIPj
X-SBRS: 5.1
X-MesageID: 53865162
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:gmz11qPxArvmqeHvrR17kMFynXyQoLVcMsEvi/4bfWQNrUol12MHy
 WMeWm3SPKqPN2vxfdtwOtu29UNXvpHcyNIxQAto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Eg4w7dRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYoxCNj9Bq0
 Yxcib/qbQo5JPXBl94QXiANRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YvNrick5atHiIasUu216zCGfBvEjKXzGa/iRtYEFhWlq7ixINfGCY
 9ozMRw/UCvvcgVsGFYyJ6sZrd790xETdBUH8QnI9MLb+VP70whZwLXrdt3PdbSiVchT20qVu
 G/C12D4GQ0BcsySzyKf9XChjfOJmjn0MKoQHrCl8v9hgHWI23ceThYRUDOGTeKR0xDkHYgFc
 gpNp3Ro/fNaGFGXosfVehmooGWd5w8geIB0Hv0gsAyC8KfZ2lPMboQbdQJpZNsjvc4wYDUl0
 F6Vgt/kbQBSXK2ppWG1rejP8GnsUcQBBSpbP3ZVEFdcizX2iNxr1nryosBf/LlZZzEfMQr5x
 SyD5AM6jq8a5SLg//TmpQ2f695AS56gc+LU2uk1dj7+hu+aTNT8D2BN1bQ9xawbRGp+ZgPd1
 EXoY+DEsIgz4WilzURhutklErCz/OqiOzbBm1NpFJRJ323zoCT4JtgIu28kdRYB3iM4ldnBO
 hS7VeR5vsM7AZdXRfUvP9LZ5zoCl8AM6ugJptiLN4ETM/CdhSeM/T10ZF744oweuBJErE3LA
 r/CKZzEJS9DUcxPlWPqL89Age5D7n1vngv7GMGkpylLJJLDPRZ5v59eawDQBg34hYvZyDjoH
 yF3bZDXlEkPDLOkMkE6M+c7dDg3EJTyPrivw+R/fe+fOAt2XmYnDv7a27Q6fIJ52a9Sk4/1E
 ruVACe0EXLz2i/KLxukcHdmZO+9VJpztytjbyctIUypyz4oZoP2tPUTcJ4+fL8G8u1/zKErE
 6lZKpvYWvkfGC7a/zk9bIXmqNAwfhqcmg/TbTGuZyIyfsA8SlWRqMPkZAbm6AIHEjGz6Zklu
 7Sl2w6CGcgDSg1uAdz4cvWqy1/t73ERlPgrBxnDI8VJeVWq+49vcnSjgvgyKsAKCBPC2jrFi
 FrGXUZG/bHA+tZn/sPIiKaIq5aSP9F/RkcKTXPG6buWNDXB+jbxy4F3T+vVLyvWU3n5+fv+a
 LwNne38KvAOgH1Dr5F4T+Rw1as76tbi++1awwBjECmZZlinEOo9cHyP3M0JvaxR3L5J/wCxX
 xvXqNVdPLyIPuLjEUIQe1V5PrjSi6lMl2mA9+kxLWX7+DRzreiOXkhlNhWRjDBQceluO4Q/z
 OZ94MMb5mRTUPbx3gpqWsyMy1mxEw==
IronPort-HdrOrdr: A9a23:i4smHKsyZmt8Lg3B59aXTsPu7skDetV00zEX/kB9WHVpm5qj5q
 STdYcgtCMc6Qx/ZJhOo7y90cW7K080sKQFgrX5Xo3IYOCFgguVxehZhOPfKn/bak/DH4VmpM
 JdmsZFZeEZRDJB4/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53865162"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bNW5xPJnRLT6xxqSL/qUTc3aUur+3sDo1vUIDLg5KlH84cQNqE18UsGmGpWzXa0b9RvzyNX/BHra5/oVKLW0UBqW8clRKHllFFSKO8stACRY5V4iaVS2VOeo+bqmyU0P6PvWOQbMPVITm+mehU63uP4Z7gMN6GhDpTRdoSCEz/zTZcxOCGYHAzNUaEkqZGKoGBJYwNVoAcAHr+iYynElNxGo9Ihy/u2LaLIGu1Ex6xZ9gpJrAU5N0cDpNgLE6my7j5Scsnr+UAlvo2co8RJTmkrIjdMIYSLDBH/A9s9WaxWD6eIMm21TwNfxB0EUlNIAfy7GEvCPaG+0DzxD3q6SNQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ex+r5cO30xvtc57Sq7mNz5e6J4VNRSJqIfeh3v66rFU=;
 b=lsHftSTBKqFjGybhE+qElsLgdVzXFADDxoepgdzjnYrEdSYw6Luuwj/jKvOXdSGH53ljjY9hLoja+huMPgen1MJYjlZQ0FoT3kxTeJSbF98+u/EKc/gh3l889AmFGaTX7c9VxkaGvdq0WbqF090cTwZYb1Q4wWHZV29UUko9W31Uy+xrP1XRZtC2zB4nDpwoPtpurnNp/zT0G9VjFb/YeGhXMRAee58fgH6xEBWfgzoKO4gnQhUd+Ag4YPvCr+KuAe7H6HDpRonMhfWo94/1ooEkT5ZyGjC7taeG8/cqqS4AGtaufnTXWwZES2YBGWu/FOJLODh5tnFdKfq2Wm1CCA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ex+r5cO30xvtc57Sq7mNz5e6J4VNRSJqIfeh3v66rFU=;
 b=HKBStHBfAWiVv5RsScYIVe0fncI4Tyq+7UMTbSszlTDZ5O/U9s2VvnMCmH7P60krndJXyevf4EmTxVpWj7KJyv8IrM2QmlYNufJ/zUtXuxsUCn0MofXSIQ00Mkd0vSdMVzkX6nxQLXgBW8LaxPepyskZhygjzxmfotJt2tQZpe0=
Date: Thu, 23 Sep 2021 16:43:27 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 7/9] x86/PVH: actually show Dom0's stacks from debug
 key '0'
Message-ID: <YUySj7w0ru9xfvE6@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <ca129fa5-7165-a9ef-4e57-75c43a708960@suse.com>
 <YUxXcrMiPDiGkdw9@MacBook-Air-de-Roger.local>
 <3b292c10-e346-4af0-1575-acdb060f0c41@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <3b292c10-e346-4af0-1575-acdb060f0c41@suse.com>
X-ClientProxiedBy: LO4P123CA0337.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18c::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 23e76a7a-ce07-42fc-d24d-08d97ea083a5
X-MS-TrafficTypeDiagnostic: DM6PR03MB4969:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB49698CD1CC05E448A5D4BD948FA39@DM6PR03MB4969.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: qbfjwso4o/bJXNsEEyFLc1IvDIr6+WgGkQJ9ZntQzUWn263VrOd3BKisalDoO6GzSkjEV+VVW0yanISKaQ5+WPlCZpXtqN6NKKd3AOG2aai6NuA7c/BxhgIdVMSwTEzXNozbLe+haWwrmxHBFXly+F5s4E4jDUA1fkzJGslwfjwhuBLJ1vKW7wnOnWlo/dRsyFUt9bfR0C6W0ZSR1TxTyuZHSF/MmNB9AcPKqwRAkA/uX7qNUh4aOxQoWb+wiWKAY4x8h/FUrJ2+nPee7/8YrSmA+0Kh90osSezXMXQ9yToWLvAUTWo/NkZb18GYLT4AsJPrB7C1qA6A+rdKuR3FQV8R8+1VOWbwyfXts+sFPHkzz+SHgQZVoZP2elaCiC7gpIYyJ8QwQVJKVuk99BnLDr9eEvGXBXMBvNJ9n/hUDSsumxiaZpCbRMPYdEu/959VbM6oJ9UjGNSKY7CIXZR+xxa3q4ItEDZezOrbjReV++XxNgtpoyXqpbtoGJVFQYyM/IBZpIpR3Qx/svCct2oSTr7JfoV8rsIE3twl/H50h52XG6oY+PjUAtRxF2i+0wk7GIkQFyC1MSrMfap7b4HABnnB3/Dho5SpYTWC3MVBbcQ2idPNJ0xNJff4emXTVf16bXA4SmbdDxJIbHaCKwTtwg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(85182001)(6666004)(316002)(8676002)(86362001)(6486002)(9686003)(8936002)(2906002)(4326008)(66476007)(66946007)(66556008)(508600001)(6496006)(53546011)(186003)(5660300002)(54906003)(956004)(83380400001)(6916009)(26005)(38100700002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q0NxTk90V3lCN0I2aEFaUWpkM0Z0NUx1RVh6VUxIZS9NdE5TRktHbFVHM0VN?=
 =?utf-8?B?UnVuNzVkclI0NlVFY0tZVlh4bW54NUNGc09FRTBOOWdyMitwMWEraFZxU3Nq?=
 =?utf-8?B?cy9uNmw5QXBBS09OOUhLQm1VRFVKUWZ4WWs4bTUwT0V1VWtDR1RtNUxYN09D?=
 =?utf-8?B?bDRXaElKSGE4S3FJUUJEMVRyK2o5cVJrLzBEMytwZnhlM1hvemdyMnMzbVFX?=
 =?utf-8?B?Ykc5N0IrMUtFU3ZjSTkzMkgyM1Jtb2Jzdm9mNldGVU5rRExxbnFISERCVUVq?=
 =?utf-8?B?UDlpNGhzZGMvRG5WdHA4S29EVER3MXIwL3hUeHViTVg0b3hTWi9WWDZHbGFT?=
 =?utf-8?B?WDRHa29HYnI4SWc4R0drL3ZDVG4zSHJtaTFXL1gwSG93c1dlV1VTUTNtSzFo?=
 =?utf-8?B?bEw5dEx2azRMN25OejBvK0JYUzcwdEFBOFlETk81NnFDbURWMFhSMFJIN1Iy?=
 =?utf-8?B?VnFHdk41Z0QxWmdsYTJtanZaRW43WDJSN0U3MllSTmZtR2VGdFArcVFlbW9K?=
 =?utf-8?B?NUFtakZNcEVZREY2UER1dXNSTE5lSEwvb0ZVR1JQTkVJVjRkRHVSMExzcXBQ?=
 =?utf-8?B?R1VsME5UYzV4S2dlUG50Y0NCam82TVMzR2JmNTNRVDZ3b3A4OTBjdGlYb0Rx?=
 =?utf-8?B?WnF2V0d2SkZJSjc3Y2dyS2hFL3B0MHNXVnNtTVFFWlFjYWZOL3hld2FjanU0?=
 =?utf-8?B?Um9IWmVXVUZmVmEvMWFvS2Q1ZkxWRXM1Q2pyYVA0dEVkOTRGTk9DU3FuZ2FH?=
 =?utf-8?B?eHR5V0tIVWtNNGRBQnZTcERDdnNOWTdZQTdLR2twWmdKWlNzempWb2lOT1hO?=
 =?utf-8?B?VG4vRWl6VGhEczlLa25GOGIvUVRIdFFma0NCM2JTQ1RxRDhBNEtzMzdkUTcv?=
 =?utf-8?B?MlBPVEdUdkU3U3B1OEEwekxieWpaYmVsaVFtQWFDaXNxdXd5dzNuVTNHRUpv?=
 =?utf-8?B?NGsvamVmKzF6UWUzM293TkpDak9oSGZoaWRielA2ei9kR1lMY1lzU0NtazFt?=
 =?utf-8?B?U1B4RVk3UnFpZFU5QWI1NDJSdDhXUm9mTmtqSG9zU0tLWUhhakpoemNiZlp4?=
 =?utf-8?B?dXRKOTZleFJMR2NId2ZlN0MyaHRoWHpiTUVRMUZVZ1dCaEoyeTJNVngxOVhh?=
 =?utf-8?B?VjYrTWNGM1RORWVnY0hYVHFjUkdvOUhVN21odHk5L3NQTm9ETm1TbDBpdHpn?=
 =?utf-8?B?VTNteTR5WTVwbUZ5NWtlSmtRZHZFTVpmTFZzNlJGekVPak9KT3FtTCs4RGEv?=
 =?utf-8?B?TG1RSnJHZkJoNHpnZVhxRVhnQllCbUJaaHZ1d0tmWjFpbnlFZ3VVajRtbUlq?=
 =?utf-8?B?OVk3MXNqZWZTcFVhOFN3ZVhMdll6a25sU0tnbmJOOGFVb2tKamhCTWVtQXo5?=
 =?utf-8?B?U05FRVhMY2t4bGd1b2ozQmFaV1FjVlYyZEtMYlVub0tIQ2hIb2JGZG1Vdmxs?=
 =?utf-8?B?V3FzVFFGcjd4b2JtQmN1NDVsSzNqSmRvS0ZqM2doWGtWbXRkNUlHM3hzTTJ6?=
 =?utf-8?B?TXJpMnBSWGdibUFLUU9RUThWaHNlVDlCa2xvbFg0anZzTVhwSXdrWVFnYlM3?=
 =?utf-8?B?d3NjOGtzQVNLTFdnb2xlOFZXczB3djVxZ0RmeHV6S2N2aCtWdU1jWDVPL2hl?=
 =?utf-8?B?REMyUkpMeGJ1b2tEd1RNMldQY1hRdk5JWnU3dmpsa0k1VTlGejJlb05HU2Zm?=
 =?utf-8?B?VmZETWxKR1ZTWUxoRHdub0ZTamplbmFGaThKZVBCbUtiQ2NSMFZaOGVZQXBZ?=
 =?utf-8?Q?ge8RUbKlcywXdeZjs4+NLZPxQ257OWW243Ze4QC?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 23e76a7a-ce07-42fc-d24d-08d97ea083a5
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:43:32.1033
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xJsGuRLac5l4VXahZcwDuYjueQkbB9L3LU8uu2K7+JNw0ZF91pQdeS5XlalOv+Fiy8PERiNpnsMLONkp1T3zRg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4969
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 12:47:26PM +0200, Jan Beulich wrote:
> On 23.09.2021 12:31, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:20:00AM +0200, Jan Beulich wrote:
> >> --- a/xen/arch/x86/hvm/hvm.c
> >> +++ b/xen/arch/x86/hvm/hvm.c
> >> @@ -3408,6 +3408,15 @@ enum hvm_translation_result hvm_copy_fro
> >>                        PFEC_page_present | pfec, pfinfo);
> >>  }
> >>  
> >> +enum hvm_translation_result hvm_copy_from_vcpu_linear(
> >> +    void *buf, unsigned long addr, unsigned int size, struct vcpu *v,
> >> +    unsigned int pfec)
> > 
> > Even if your current use case doesn't need it, would it be worth
> > adding a pagefault_info_t parameter?
> 
> I'd prefer to add such parameters only once they become necessary.
> 
> >> --- a/xen/arch/x86/traps.c
> >> +++ b/xen/arch/x86/traps.c
> >> @@ -364,6 +364,71 @@ static void show_guest_stack(struct vcpu
> >>      printk("\n");
> >>  }
> >>  
> >> +static void show_hvm_stack(struct vcpu *v, const struct cpu_user_regs *regs)
> >> +{
> >> +#ifdef CONFIG_HVM
> >> +    unsigned long sp = regs->rsp, addr;
> >> +    unsigned int i, bytes, words_per_line, pfec = PFEC_page_present;
> >> +    struct segment_register ss, cs;
> >> +
> >> +    hvm_get_segment_register(v, x86_seg_ss, &ss);
> >> +    hvm_get_segment_register(v, x86_seg_cs, &cs);
> >> +
> >> +    if ( hvm_long_mode_active(v) && cs.l )
> >> +        i = 16, bytes = 8;
> >> +    else
> >> +    {
> >> +        sp = ss.db ? (uint32_t)sp : (uint16_t)sp;
> >> +        i = ss.db ? 8 : 4;
> >> +        bytes = cs.db ? 4 : 2;
> >> +    }
> >> +
> >> +    if ( bytes == 8 || (ss.db && !ss.base) )
> >> +        printk("Guest stack trace from sp=%0*lx:", i, sp);
> >> +    else
> >> +        printk("Guest stack trace from ss:sp=%04x:%0*lx:", ss.sel, i, sp);
> >> +
> >> +    if ( !hvm_vcpu_virtual_to_linear(v, x86_seg_ss, &ss, sp, bytes,
> >> +                                     hvm_access_read, &cs, &addr) )
> >> +    {
> >> +        printk(" Guest-inaccessible memory\n");
> >> +        return;
> >> +    }
> >> +
> >> +    if ( ss.dpl == 3 )
> >> +        pfec |= PFEC_user_mode;
> >> +
> >> +    words_per_line = stack_words_per_line * (sizeof(void *) / bytes);
> >> +    for ( i = 0; i < debug_stack_lines * words_per_line; )
> >> +    {
> >> +        unsigned long val = 0;
> >> +
> >> +        if ( (addr ^ (addr + bytes - 1)) & PAGE_SIZE )
> >> +            break;
> >> +
> >> +        if ( !(i++ % words_per_line) )
> >> +            printk("\n  ");
> >> +
> >> +        if ( hvm_copy_from_vcpu_linear(&val, addr, bytes, v,
> >> +                                       pfec) != HVMTRANS_okay )
> > 
> > I think I'm confused, but what about guests without paging enabled?
> > Don't you need to use hvm_copy_from_guest_phys (likely transformed
> > into hvm_copy_from_vcpu_phys)?
> 
> __hvm_copy() calls hvm_translate_get_page() telling it whether the
> input is a linear or physical address. hvm_translate_get_page() will
> use paging_gva_to_gfn() in this case. The HAP backing function
> changes when the guest {en,dis}ables paging, while shadow code deals
> with paging disabled by installing an identity mapping
> (d->arch.paging.shadow.unpaged_pagetable) which it would then end up
> (needlessly) walking.
> 
> It really is - afaict - intentional for callers to not have to deal
> with the special case.

I always forget that we change the paging_mode handlers when switching
between modes.

> >> @@ -663,14 +728,22 @@ void vcpu_show_execution_state(struct vc
> >>      }
> >>  #endif
> >>  
> >> -    /* Prevent interleaving of output. */
> >> -    flags = console_lock_recursive_irqsave();
> >> +    /*
> >> +     * Prevent interleaving of output if possible. For HVM we can't do so, as
> >> +     * the necessary P2M lookups involve locking, which has to occur with IRQs
> >> +     * enabled.
> >> +     */
> >> +    if ( !is_hvm_vcpu(v) )
> >> +        flags = console_lock_recursive_irqsave();
> >>  
> >>      vcpu_show_registers(v);
> >> -    if ( guest_kernel_mode(v, &v->arch.user_regs) )
> >> +    if ( is_hvm_vcpu(v) )
> >> +        show_hvm_stack(v, &v->arch.user_regs);
> > 
> > Would it make sense to unlock in show_hvm_stack, and thus keep the
> > printing of vcpu_show_registers locked even when in HVM context?
> 
> Perhaps not _in_, but before calling it, yet - why not.

Indeed, with that:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:47:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:47:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194345.346209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ0I-0001PI-8o; Thu, 23 Sep 2021 14:47:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194345.346209; Thu, 23 Sep 2021 14:47:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ0I-0001PB-4v; Thu, 23 Sep 2021 14:47:14 +0000
Received: by outflank-mailman (input) for mailman id 194345;
 Thu, 23 Sep 2021 14:47:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTQ0H-0001P5-Iw
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:47:13 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 22b821e0-1c7d-11ec-ba4d-12813bfff9fa;
 Thu, 23 Sep 2021 14:47:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22b821e0-1c7d-11ec-ba4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632408432;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=I1WD2uBaP9UhrpSJiu2bDrjKH0zg5LHi1uv+Q9MdDcc=;
  b=ImSEM4B9w89vR8EM/+Ou/Di2/oIZZ0MgbQxH9TlqLva3/OtHw7yhJghK
   dOrnN0ByZx+ns0B/Ixv5E1jyB44TBBIq36qtHTQE4YNFQmAzOv6YkA8Sz
   nP+rHfu/k1QUMMsELqcyE71zUOWK9yA0/cI0l5XoSvWfIXSr2rVSZDbRk
   M=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: B3AR+bnOGwQjmySAzpHcklR0xQAQrUN/O3/aazgL2TbBBL28QQRZ/ygrG9VyeTAa6r8MgW7106
 DXJ6lNUEe8Jes7pi7Kwsty02AgI9RY3T8urNKEKseXwZnrN782y37wyxuqmzct8c6W0TmvTuVf
 oCyVnwIZh2Ii+qNmXsQlYVneqh2FNmK4PV5j0q7aYiW0QQXAEC1o4WPCT5aeb5qtpW6mJpxX+W
 PJQJJIaDsLqGKhPbsm/QexOn+GikyoHoT4RqOsO8SHy19iAnBEOLiX4Yg5o+BpdvBbRas+ygSW
 xb8x+Z+GLv373w1ddlrlVU5T
X-SBRS: 5.1
X-MesageID: 53045020
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:oOP+5asUsKlo8hJM4UIFsdKNDufnVK1ZMUV32f8akzHdYApBsoF/q
 tZmKWuEaf+OYmrxe95wbY6y8k0Bv8SDn9E1TlY9rns8FShG+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524PhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npljc22cA0iLJfwu6c/FENBFiZGA6Eb0eqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY25sWQauGP
 ZJxhTxHYAWQYSBjMVUrK7l5u+ilmkDmcSxCpwfAzUYwyzeKl1EguFT3C/LWd8KLQ4NJn0+ej
 mPA42n9RBodMbS3yyeDtHSlhebNnCbycIMUCLC8sPVthTW7xGYeFRkXXluTuuSihwi1XNc3A
 1wZ/G8ioLY/8GSvT8LhRFuorXicpBkeVtFMVeog52mwJrH8uljDQDJeF3gYNYJg5JReqSEWO
 kGhp4LvPDZkooepajG/se2NqDSKByRIBDpXDcMbdjct797mqYA1qxvASNd/DaK45uHI9SHML
 yOi93dl2u9N5SIf/+DipwmW3m71znTcZlNtvm3qsnSZAhSVjWJPT7eh70TSpd1EJZyQJrVql
 ClZw5XGhAzi4JfkqcBsfAnvNO3yjxpmGGeF6bKKI3XH3272k5JEVdoMiAyS3G8zbq45lcbBO
 Sc/Qz956p5JJ2eNZqRqeY+3AMlC5fG+To6+DaiMNYQUM8kZmOq7EMdGPxX4M4fFyhRErE3CE
 c3DLZbE4YgyU8yLMwZat89CiOR2l0jSNEvYRIzhzgTP7FZtTCT9dFvxC3PXNrpRxPrd+G39q
 o8DX+PXm0Q3eLCvOUH/rN9MRW3m2FBmXPgaXeQMLbXdSuencUl8Y8LsLUQJIdc4wPgFyrmSr
 hlQmCZwkTLCuJEOEi3TAlhLY7LzR5dv63U9OC0nJ1Gz3HY/J42o6c8im1EfJ+NPGDVLwaEmQ
 v8bVd+HB/gTGD3L9y5ENcv2rZB4dQTtjgWLZnL3bD86dp9mZgrI5t67IVe/qHhQVnK65Zkkv
 rmt9gLHWp5fFQ5sO9nbNaC0xFSrsHlDxO8rBxnUIsNecVnH+ZRxL3Cjlec+JswBcE2RxjaT2
 wuMLw0foO3B/908/NXT3PjWpIa1CepuWEFdGjCDv7qxMCDb+EulwJNBD7nULWyMCjus9fz7N
 +tPzvz6PPkWp3pwstJxQ+Rx0KYzx9rzvLsGnA5qK2rGMgawAbR6L3jYgcQW7v9RxqVUsBedU
 16U/oUIIq2APc7oHQJDJAchaejfh/gYliOLsKYwKUT+oiR24KCGQQNZOBzV0H5RK758MYUEx
 +Y9uZFJt1zj20RyatvW3DpJ82msL2AbV/R1v54XN4bnlw43xwwQepfbECL3vMmCZtgk3pPG+
 dNIaH4uX4hh+3c=
IronPort-HdrOrdr: A9a23:SfJSiqAbZ5JHrynlHegksceALOsnbusQ8zAXPh9KJiC9I/b1qy
 nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHPlOkPQs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ
 0QD5SWa+eAfGSS7/yKmTVQeuxIqLLskNHK9JfjJjVWPHlXgslbnnlE422gYytLrWd9dP4E/M
 323Ls5m9PsQwVaUiz9bUN1K9TrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJqJmLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87CsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXkHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1XackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMW9yV0qp+1WH/ebcGkjaRny9Mw8/U42uonlrdUlCvgklLJd1pAZHyHo/I6M0r9
 gsfJ4YzY2n46ctHNVA7dw6MLmK41r2MFvx2VKpUCfa/Z48SgfwQr7Mkf8IDbKRCdE1JKVbou
 W2bLofjx9rR37T
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53045020"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EqlSdH62Qjh5sNdzkN9936gC8TJo4m5EJuFt+sI16l9/+EgscX+yyHZVjY4uvqLUo2q5CZaqCNQMa0C4zmtYwG7eVj1bLAayfawxBqb9a8KxRMH+X0HndK3mdwlJcGq8oW0N0dZXURvqIGM+xMisvz2QXbunogbOJOCcnhdqy90/26x5p1JOX4wWmeMEuiHAWqrHmkx5ybLzWimpr6SOtpb2lohTqRs71DX05FxKPZGt87ltaUlZSjOp/zaBGsFHVyW57T73BU/9V4+Mex06+xN6SRirRTyllCwLBVOFHUP+UWIOpOUzjgasSZLdEDKpCn0Y9yMC6TsDKg9GfkjtRg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mC3GrvuImFn6pBisvCaJpoOGz0i9QCI0i7xjXQmqFC4=;
 b=MWgJ5gfBYh7NX84LIuclejAj9jJlphi4HW5xu9y3UPZcoVDMmRc6FyQuAhDXiBrfIKAGx+vg3QjSpdXuF4Np7AXCfFdtpz8ZZgScALzhpYa1/7D1yV28BlGxI4drI7b356XNAlO8R46ud9PldKbfpHQH20+iX585GzUli6F72mGyAl52qSuQqSWheRD2oKiFBHt0kiUxGfXjGsp+L+mzdDFFj9lKF+0n/IF+ncRr/2Jd9DLvb3W53tKoWqEewJVsQVug9qbYa27qYmki1kH/xCiajgJazCEgTH0ihpzGBszMgSWrnD3gUG3HXPiPj4205TlrR0hq/LgL7PL09+21Rg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=mC3GrvuImFn6pBisvCaJpoOGz0i9QCI0i7xjXQmqFC4=;
 b=TTx3N7qksF/H+NL8ia8+9K+gSV5WJmimvI/3FLrACSaNhiR7azQVgx0KLkJxhqiPnR6CqXcNImfxV/G4ufa3VHG+BvfOQyLF9YR6oaa0jwUT2HvdDCKz5vldqBX3O0bWa9wsoPdJTmUP/qKOwC1toWUfhZYCpNYUPyIDXfqDuL8=
Date: Thu, 23 Sep 2021 16:46:52 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Kevin Tian <kevin.tian@intel.com>, Jun
 Nakajima <jun.nakajima@intel.com>
Subject: Re: [PATCH v3 8/9] x86/HVM: skip offline vCPU-s when dumping
 VMCBs/VMCSes
Message-ID: <YUyTXJacLZi0ZmTm@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <5cc8f7e0-399e-f6a5-bc84-823a6160a3f1@suse.com>
 <YUw5babRHdk6Jh8E@MacBook-Air-de-Roger.local>
 <b1c14b85-e65a-d563-bf51-0cb4483fde0b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b1c14b85-e65a-d563-bf51-0cb4483fde0b@suse.com>
X-ClientProxiedBy: LO4P123CA0377.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18e::22) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 02223062-c060-4ce4-b9cf-08d97ea0fe70
X-MS-TrafficTypeDiagnostic: DM6PR03MB4764:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4764A436760EB8E293555BD18FA39@DM6PR03MB4764.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: rYoIK6XZ/NEwFQ8MgG/RROV/qL88ikGYhagmts7KlNib6u5Q5J4guqRVZ4Qdb65FmZBTDYovuS/OcyHl+Yq8e19wPxIRhuT0tsnUB3ncAViEMPaQx++jNR9SBFjLaVFU3b2gAIv/cz/kpVvlLGbcxhAqLSfKr85obglR3SWym+T8MxS0lHOyuAm5k8peSqZBZr77sUAoIAVGPF42oh61TKRY6c4IMehxInkx0nUgsHeGQdpwPWY5UR+A9WoL+PtNJvX4Abcj1Ac7lLz9+Fr8kxE9ZTdGTVyLL8To2gUWqBmQ6viz5NIEXQFgzCEa5Z2Re8LzTmJYQjiH4uLF3EfimVvwTHbeqgXBGiECikIIVtZU058INfkdHNaxlKAecVeupK9cGaYsp4qNRDlv0MeNdFE3n0L+owZahoC6XxnFXAppaxRpaxRV/thbetLCjHySIvCoe/vF1QtPmtmzEWNwARXFEAwU5b5SVXFrbqpWnNG0eRJj4lWD+k//NiSXsDuyCnArpjqdbLXhjoiEw1zWDQ2Amc/x+GiZ8gYEQzYpFg2xeIECoGGQGArH+5qUkewLwSfgZb7SHK6edjXrt7uwC1RUoHKLU6Cwck/c9MbtCsCTuwtlhPFSXh64FGP/2Bk0KZrDeORrc+2MqktrdQ9cNQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(53546011)(6486002)(5660300002)(26005)(2906002)(956004)(9686003)(186003)(6496006)(6916009)(8676002)(66476007)(38100700002)(508600001)(83380400001)(66946007)(316002)(8936002)(85182001)(54906003)(86362001)(66556008)(4326008)(6666004);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WStOOTd1NXFPYTRGMU0xbnVXWDhNcmd4OHVGMFRTL1ZRcHMyM1hJUEp6UnUr?=
 =?utf-8?B?ekhwSnFSQWFNenFkaUFLMzM2U1dobHcrcmFIZ2lXdmkrVzB2Uy9DTmxkNzRu?=
 =?utf-8?B?aWhGcE9GMGxxTFN1emdFM1JreSsxeVI3NXFrN0RLYVFMYVU0S2pFMXk2bDQv?=
 =?utf-8?B?SjYzOXhvWHc4dmNQQThTOXp3dThMcnVRaUE0ZUhXeURjcW9uSXozRTlnTTVn?=
 =?utf-8?B?QmhyVS9kdElBdXlPdkN2MXlNQVRNOUJJSDVlQTlzUjFZeFdhNjYzSHpzVU5p?=
 =?utf-8?B?RUx3SE5jN2lzNEdQb2dOa3lONFVpM1FNaXZROWVtblVjVmNSQ1QxS1Rmb01n?=
 =?utf-8?B?T3ZDTjJxSnlmQ05yeEp6Rkxzd1dobWg5TzVWS3dFODhaY21FQ1J6aDRBdW8v?=
 =?utf-8?B?L3YzTFdmZDJLRk1reEtuVklQS1BxbjJvNnE3WXNraHpvZ2MxRDVYRFJiNWxi?=
 =?utf-8?B?RlRjcVp2Sm9jM2g0QXloWHMrc3lVSGRERGxFSkNDMkkxcGkxck4vQitpbDFr?=
 =?utf-8?B?ZkRBSDZhV2l5NmFaUDJtbVN2Z3EwYTIxeEdla2JPNlQyWkllbmJIQjhLanNQ?=
 =?utf-8?B?MmRTVklvaWlvdlNNNm5MS05EbEhNZ2R0d1JBU000RFdHYVF6ZkFRMGt6aDZQ?=
 =?utf-8?B?Q3FBQ0FVR0xwRXp4bklPK3hjN1RuVDRsTmtyek9aVTFwVTNFUG9JcVZFZVJi?=
 =?utf-8?B?ZDNzM2hkeWR4ZTB4eFU4dWVBMVlCT2NxUkFudHE5U1p5eGN5aER1MG1DRldi?=
 =?utf-8?B?MFN4WkVNS2t5eFJPZ3d5UHVoMGVBc3NhOWZKWVVEN2VrY0tDQjFrM2Zxc21x?=
 =?utf-8?B?bDU1SXZRTWkxT2NUSmNqcS9IRGxCNDNYSXgyQmQzdlMvS2laMGZJcDZsYjBv?=
 =?utf-8?B?TXJnYVFudVF3YXBBZ1dDb09aVlJLMENWdnFMSlpZRUd0MTUvYXMwbHVRZ0dT?=
 =?utf-8?B?MkFwcDNQcUk2bllYUXllRTRPbCtJOC9wdVFVL0p6cU9QSW93VUJkYzhpaVc1?=
 =?utf-8?B?R3BENlpGZGRsZTBPMk1nUExVTjNnU1lId3A5V2ViVzh6ZkdPblV3NzBuS1Vw?=
 =?utf-8?B?SVFmRkZRelZOWmUzMmd5NXV1dm53VkRuWldrL0h1Z2h2Nlk3ak56cW53dncr?=
 =?utf-8?B?NkxYZXRZSDEzeU95dmJ3UzdScWtXRHQ0eFFTUldOZk8vWndVaUJsWXR4Nk9Y?=
 =?utf-8?B?ZEFtQjRRZnZ6QVJyUnR0Z3NITlhESUFLT1hiYnE4cWVDMG55V1VqdFJhQVJp?=
 =?utf-8?B?Q3BxYXhnWTFJc280WE80MngwcG9KaHBKZ2U5aG1ub0F3QkNTdzV4enB3Z0hw?=
 =?utf-8?B?cUt3YTc5OW5lQWMwQmxHUXgyc0xya0tGdUVwU2ErZFVscWRDTXhKajRNS25o?=
 =?utf-8?B?aE5BdlFUdWx0NVFxeVFpTDNBQ0loVDlEcGpSTXhueER6RTBqSzUxN0tpbVgv?=
 =?utf-8?B?dnRRa3hXRkRpQzF3UEtzUFBXM1FSUlBKZmYrZUZ3ajVYZFlYN2ZLUksvVXVi?=
 =?utf-8?B?UnhxVmVWMSs3ekpaYzRlNjlkOXZxZytTWWtmT2QvODltdmc5czBOdUpJTFBk?=
 =?utf-8?B?RmlpV0hCQlpkYk1ueCtVYVEwK3J1ZUc1bUZsQVR6eXdMMEZEdGtQdll3Nzkw?=
 =?utf-8?B?SEptZ1I2QWcvMTdIc1VBQ2tia0I3ZlpPZ1kzdS9mM2drd24vNXFWYUljRWVM?=
 =?utf-8?B?VGhtdGlpOGFRTUE5OGl3ZG9jaDNGWTdMbTluZ3RRNFVpRzZlYXkvZkU0dE5N?=
 =?utf-8?Q?U3Jk/sXlH+8qzqUC+7g6vNOpHN7EyXsYqRSVYuS?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 02223062-c060-4ce4-b9cf-08d97ea0fe70
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:46:58.0802
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CdrMrAXle0IsTRutQtaEA83ejrN4TOW0u2sPBkVRyVQDlxuGOoq4GWPz8CaS+60Nw7/ZLpUzZYrj34T0btU1Kw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4764
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 01:27:30PM +0200, Jan Beulich wrote:
> On 23.09.2021 10:23, Roger Pau Monné wrote:
> > On Tue, Sep 21, 2021 at 09:20:32AM +0200, Jan Beulich wrote:
> >> There's not really any register state associated with offline vCPU-s, so
> >> avoid spamming the log with largely useless information while still
> >> leaving an indication of the fact.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.
> 
> > Is the state cleared when the vCPU is brought down? Or else it might
> > be interesting to print such state for debug purposes.
> 
> Hmm, if that's considered to be potentially useful, then we'd need a
> key different from VPF_down. HLT would leave state in place, and the
> INIT/SIPI sequence would clobber prior state also only on the 2nd
> step. v->is_initialised may be an option, but gets cleared by INIT,
> not SIPI. Any other ideas?

Well, I guess it's close enough. I also wonder whether we should do
something similar for PV, where it might even make more sense as you
can upload a vCPU state in that case when the vCPU is down.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:52:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:52:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194350.346220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ50-0002me-Sk; Thu, 23 Sep 2021 14:52:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194350.346220; Thu, 23 Sep 2021 14:52:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ50-0002mX-PE; Thu, 23 Sep 2021 14:52:06 +0000
Received: by outflank-mailman (input) for mailman id 194350;
 Thu, 23 Sep 2021 14:52:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RP5N=ON=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mTQ4z-0002mR-MB
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:52:05 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.46]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d0cde1d4-1c7d-11ec-ba4d-12813bfff9fa;
 Thu, 23 Sep 2021 14:52:04 +0000 (UTC)
Received: from AM5PR0201CA0022.eurprd02.prod.outlook.com
 (2603:10a6:203:3d::32) by DB6PR0802MB2566.eurprd08.prod.outlook.com
 (2603:10a6:4:a2::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 14:52:03 +0000
Received: from AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:3d:cafe::d6) by AM5PR0201CA0022.outlook.office365.com
 (2603:10a6:203:3d::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 14:52:03 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT042.mail.protection.outlook.com (10.152.17.168) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 14:52:02 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 23 Sep 2021 14:52:00 +0000
Received: from 0d8b64f886bf.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4AF9AD22-7D81-4C82-8B6B-942862198891.1; 
 Thu, 23 Sep 2021 14:51:46 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0d8b64f886bf.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 14:51:46 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6032.eurprd08.prod.outlook.com (2603:10a6:102:e4::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 14:51:44 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Thu, 23 Sep 2021
 14:51:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0cde1d4-1c7d-11ec-ba4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zUDPicpPpTARMQlSPA5/agCsr7gquYRqozuTjvJn4vY=;
 b=PmpoaKrB2HNGU0WoEauo8W/kFXFzTlZhu0dNrtNiBJ3CkqDLedvbXY/divvY7rBXZS0aN+Y7dpyRep2UIrP8r9BYsLHkPpYK+eg8KqG/xeUlkqzCa8dPbaIiLyNgO8e1z5JggsYV/ujGGcPq7PX3Mo11BH9IYt5+c3uOMChW20A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d309d9a9a1e8393c
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Xtyf9J6Inq/YlsZoJt6hT6mWEJTAuv95TzmQp29x9Bto1ppLDUAhZGSQaT9EzgK1uh+DI2a0ukWKqTtFsdc1XqaZBBsYxMy6HHqSaVNmBnmdIDrYz4QaLFXDgiUNfuFbh6/GH6pyc/FIXR1Sm+I407iNEku652GbJNXPnFh9w5png5FUgHKiqoYJjPf/zP9c2Glk93lQrVSxImEWecfL96+m2NnP8dBIVsHcj7XZcdYu/pDSYiBVvaocBXFw+NvvZm2Yit9ja2NkTsK/gsKMXi3UX1msHZbNeEo+ibznfCNpDcxQle2xIfD4XNRps2Q5Lr5yDLIqqvzJ+tcrlwDpYQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zUDPicpPpTARMQlSPA5/agCsr7gquYRqozuTjvJn4vY=;
 b=J6oB2bcC5RLfTX/Rkm7pM814/stXOGd67THSVESr0jdXziVdD7bLleJ2N0bVAIGn9qAlQi7iZcvH/o8OrcYpE75Dlayq9flKUN3Mqs1B2AcjxVUuib+HzO87NMXK/aOFvpEHXMmgjs4E1p8hyPteJ8xEMVRb3PQ/xrtnI2fCfbtfKYncbmAe/bTd+9AhsJH27wotD+8TMIZohB2FySp48b8otmC3MD4wlfqNIeUhkPA/g2+4NG2G/uuCsfm3eV6TDbD//LdcJm8SZRARZ7hZmCHaCJoMFhiSmwiDSNT7KKaAT1gkXF9kozUogv8nv36wg8F7Nx921z9wBOCGxU/7yw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zUDPicpPpTARMQlSPA5/agCsr7gquYRqozuTjvJn4vY=;
 b=PmpoaKrB2HNGU0WoEauo8W/kFXFzTlZhu0dNrtNiBJ3CkqDLedvbXY/divvY7rBXZS0aN+Y7dpyRep2UIrP8r9BYsLHkPpYK+eg8KqG/xeUlkqzCa8dPbaIiLyNgO8e1z5JggsYV/ujGGcPq7PX3Mo11BH9IYt5+c3uOMChW20A=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
Thread-Topic: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
Thread-Index: AQHXoyUczCdL3CcIR0eYq/JO8eFke6uxzsGA
Date: Thu, 23 Sep 2021 14:51:44 +0000
Message-ID: <FFA5F03B-93E3-40AD-B98E-82141788999D@arm.com>
References: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
In-Reply-To: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 462f082d-0e74-45ea-6c2f-08d97ea1b42e
x-ms-traffictypediagnostic: PA4PR08MB6032:|DB6PR0802MB2566:
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB256618CC80BF1DBDC18B7AFB9DA39@DB6PR0802MB2566.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 KPPog++FqUDZVEBQO2x9pK8kIB5Eg9a8NFOaVCrMS7OVEIaJKpHAUidqAMUjz823ClMw/33m/B79gi/dAcpUxoR27du4g4Cal4SYmVs7TrVhjcfYeXHBb/5rwXB6Hitpe93cxpG1wZswiRnsRTZF9TAfAFORzg6qzPuzwlJitCrF8SIdbTZVRNlnCNrqj1kEd1BSZlkqm3cvmWlipSn+xVZA3bKM85RktVyYWRcAfO3gLYDZ2y7cELTradDRkFmdM73IcUaDjt2cn7Iwk6/uDm2noEMwBz96GlTFZDGJi+LwYbBrjKWbPHPxRNvVyNFFyGAU95Z7N//+6/CGtBlsUOiZOYuPRLPiaMxMeNIunL7nFQUDff8ljkvdt3GHJFyl6/kt8YIc+7ApC+NEMJGuN6ckFUT5Ln8TjyCGGecyOPEpHCVLRhzBDMWyTWsskxMo2OtqP3rEYiZwuyf9HGF9Seo5ksRzY35WmwVhzUPyHoPvb3kIZYkeKstMYNw84w0crrU0c4EoQlOJI4R9WrNfKPytjtzE2NwnLqYnJSglEtRCKTVJEH9qT1SXjh4poGbf6NPx0m3m2b75uGNuLqX+X1DPczCFk7bD2oTJux6a/e1XHdIc46sl/v2Q8TkY/oAe5ujHNiMejcxPFoaYcdooGson8XjxZz7urGk11JUX6zSonJgUQc2w0j+DGnGZQFRwR5YQwmZD2f+A9qUHFp/RZ6jQ5zMaQEunpJUjSms61/b1vAIzje1fRitpqZE3hxGI
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(53546011)(6506007)(66556008)(64756008)(66446008)(26005)(8676002)(66946007)(4326008)(508600001)(6512007)(8936002)(38070700005)(186003)(6916009)(38100700002)(71200400001)(2906002)(316002)(54906003)(122000001)(83380400001)(5660300002)(36756003)(2616005)(33656002)(86362001)(66476007)(6486002)(76116006)(91956017)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7558E0D4BE902A44A12E7347C91A1A96@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6032
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	71a15ea1-ea24-41d8-7688-08d97ea1a93f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VkTKuxslo5GP3yZN8hHibN6wAPHbRWHDLPPnOCSXxaoeSmIbFR3NxRuQbKzhp9FnKET0Jq8SplsD71KsRCzACLlqcEbpIKSx8a2GxCKeuUw7Mn+1RoALXr4cLQVGuVnjz9LzqPyYpLmbSGZskreEJh5OFEphmvPo/TL11MSZgA/YhHwEmtAbDupyKSmkL9u6iR1ZxscgdcDYWH4rI7QtGbPn5ossLiOt6gPbpoKTEmfAexVWWD2IsqpMpsQ59/IIblbDi+rd9dXXDQmT8aDkxQ+rAnM64BO6gYnXIfFOSOIwu0SqUQDTDPIRICzKL2jQfvtph1QzD8758QGlJLQrvlt5QOu8riGYCTdD+nOFPqGE4pznn1AFcDCT211nSMsUfBRWe/0mlNDERfU9PzPOiXZglu7YRs/2i3PTtrWI8+Yewki2BXZw9l84fyBnR+QmOWnad8BM7BAWgdSNOxtquEaBe4cWuPRiAfnjsqEET+tm7aK4EcOxD2rtPm9ynDtLXWmou2tgyIV/1ZnpvocQC662lDrWoev2YgUd+QliCUFefgfjzLRmVvXUd6rHwCFRrkZbKFJFe+wZhraruczTO8VpbShpUans7CUhH+GgDdGpmD5EYzJXKBAL1xROwc9sY+4m8fVo0C5Sj1hpljLykQJmx9HUzLXdjL16BC9eydnH9b4rwGpEumu3eos/dMcZF0I0JLNQhdZsNPVAtccsKA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(186003)(86362001)(6506007)(36756003)(53546011)(33656002)(508600001)(36860700001)(336012)(2616005)(47076005)(8936002)(4326008)(82310400003)(83380400001)(54906003)(70206006)(70586007)(2906002)(356005)(6486002)(316002)(6512007)(81166007)(6862004)(5660300002)(8676002)(107886003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:52:02.6871
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 462f082d-0e74-45ea-6c2f-08d97ea1b42e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2566

Hi Oleksandr,

> On 6 Sep 2021, at 14:42, Oleksandr Tyshchenko <olekstysh@gmail.com> wrote=
:
>=20
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>=20
> Allocate anonymous domheap pages as there is no strict need to
> account them to a particular domain.
>=20
> Since XSA-383 "xen/arm: Restrict the amount of memory that dom0less
> domU and dom0 can allocate" the dom0 cannot allocate memory outside
> of the pre-allocated region. This means if we try to allocate
> non-anonymous page to be accounted to dom0 we will get an
> over-allocation issue when assigning that page to the domain.
> The anonymous page, in turn, is not assigned to any domain.
>=20
> CC: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> xen/arch/arm/tee/optee.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>=20
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..83b4994 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -410,7 +410,7 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struc=
t optee_domain *ctx,
>     if ( !shm_rpc )
>         return ERR_PTR(-ENOMEM);
>=20
> -    shm_rpc->xen_arg_pg =3D alloc_domheap_page(current->domain, 0);
> +    shm_rpc->xen_arg_pg =3D alloc_domheap_page(NULL, 0);
>     if ( !shm_rpc->xen_arg_pg )
>     {
>         xfree(shm_rpc);
> @@ -774,7 +774,7 @@ static int translate_noncontig(struct optee_domain *c=
tx,
>      * - There is a plan to implement preemption in the code below, which
>      *   will allow use to increase default MAX_SHM_BUFFER_PG value.
>      */
> -    xen_pgs =3D alloc_domheap_pages(current->domain, order, 0);
> +    xen_pgs =3D alloc_domheap_pages(NULL, order, 0);
>     if ( !xen_pgs )
>         return -ENOMEM;
>=20
> @@ -938,7 +938,7 @@ static bool copy_std_request(struct cpu_user_regs *re=
gs,
>=20
>     BUILD_BUG_ON(OPTEE_MSG_NONCONTIG_PAGE_SIZE > PAGE_SIZE);
>=20
> -    call->xen_arg_pg =3D alloc_domheap_page(current->domain, 0);
> +    call->xen_arg_pg =3D alloc_domheap_page(NULL, 0);
>     if ( !call->xen_arg_pg )
>     {
>         set_user_reg(regs, 0, OPTEE_SMC_RETURN_ENOMEM);
> --=20
> 2.7.4
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:53:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:53:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194357.346231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ6i-0003Sz-C3; Thu, 23 Sep 2021 14:53:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194357.346231; Thu, 23 Sep 2021 14:53:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ6i-0003Ss-80; Thu, 23 Sep 2021 14:53:52 +0000
Received: by outflank-mailman (input) for mailman id 194357;
 Thu, 23 Sep 2021 14:53:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQ6h-0003Sk-7r
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:53:51 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.41]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0f7864fe-1c7e-11ec-ba4d-12813bfff9fa;
 Thu, 23 Sep 2021 14:53:49 +0000 (UTC)
Received: from DB7PR05CA0055.eurprd05.prod.outlook.com (2603:10a6:10:2e::32)
 by VI1PR0802MB2478.eurprd08.prod.outlook.com (2603:10a6:800:bb::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 14:53:42 +0000
Received: from DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2e:cafe::e0) by DB7PR05CA0055.outlook.office365.com
 (2603:10a6:10:2e::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 14:53:42 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT053.mail.protection.outlook.com (10.152.21.119) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 14:53:42 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Thu, 23 Sep 2021 14:53:41 +0000
Received: from 35598c80bba4.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B25BBD46-064C-4A2A-99CD-181970BD9539.1; 
 Thu, 23 Sep 2021 14:53:30 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 35598c80bba4.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 14:53:30 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0802MB2385.eurprd08.prod.outlook.com (2603:10a6:203:9e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 14:53:28 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 14:53:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f7864fe-1c7e-11ec-ba4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ElhrFnB5ab2oZRXsVLBHtVJqO97uKi/ogzrBE4TU2+w=;
 b=rglkXaBGN/RukX8Yg7rZRaKWILbKTAaMIstx6ZH3nc2gh0w5Aj0rOmXg9bFvaf4ZXCdVuTM4+wxtT9VvnxTUUWqzvg8PSJyUexbtsRjwAdxTV/CRNulwez61060Qhif/0P7OUa+PN+bNYdDCE2IOkqiL72vWbL9oQ8j7PDxON+M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 7da530717aa57a53
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YZg3Q4o6eshRLEp1G5yI8Yf5v9vOsRSKHNX7NbJgH93A1RreL8HTWgvxwlzVVVyBvOAxECwib45tO6BvBy8MX3dwTrj4I3++gpJNMHf1S8+gjRrJJGSX63uiPHvRyMA9PtAQx6Y+kbvEJVlJY2WosQgPg91FIpYZlC2MHOl+PcuVbqMlbBA9eH/ckn5oF33MpQGYUKFSobpNURyyihHc3BINmFCgMn1IsD+wvNeLUMQVYV12ODd+cRjkphzN5v7dJ+NS4C8Bpt5Kv2OlAFB8FDd7xmCbWF5fF3blahNmMN+DiGPxXeUcd+dY+KnUfJ0s2kFpFsFgRaN+EqE/QbP6Rw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ElhrFnB5ab2oZRXsVLBHtVJqO97uKi/ogzrBE4TU2+w=;
 b=O0b89N6PrZnwjIartnjIGXbRZsIVQoP95iqfGIedjwggMSPM6jgWf97SdySqIvKFDMkNsTQezp5hW7Bt5P+kiGCiyWaVE5Sues39IOlOjoUZwxGMsrsPcZTeFuFUHbiIGVcxrWggb0T9j9hZ5x5rtnWh7x/Z5MsDHxUZP5U3g3fmO2+MR1SoCJkTFBLMrwfSx+waa4qtOK04s54l8KpL5YuCR1pKZxFr2J2YzxAZGDQLgf2LhUBBJRAAevlpBaPtUUrgoabVpNBhwLAxAdHe/ZQI8/6bz0TFE4EPkGRoFczvmqYv5zBddPSljzmRZbNtwEtDg4uwUdZQQzDsHglqKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ElhrFnB5ab2oZRXsVLBHtVJqO97uKi/ogzrBE4TU2+w=;
 b=rglkXaBGN/RukX8Yg7rZRaKWILbKTAaMIstx6ZH3nc2gh0w5Aj0rOmXg9bFvaf4ZXCdVuTM4+wxtT9VvnxTUUWqzvg8PSJyUexbtsRjwAdxTV/CRNulwez61060Qhif/0P7OUa+PN+bNYdDCE2IOkqiL72vWbL9oQ8j7PDxON+M=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Topic: [PATCH v2 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Index: AQHXr6Z0kesRmMgn3UWCiz/08WLEoauwvZ2AgAD4nYA=
Date: Thu, 23 Sep 2021 14:53:28 +0000
Message-ID: <7A85434E-D15E-4DBE-8FE4-403F1E9228F2@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <000832623dc7fb429db4b4517583f16affdba35b.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221657080.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221657080.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ca394ab5-2181-4226-5409-08d97ea1ef7f
x-ms-traffictypediagnostic: AM5PR0802MB2385:|VI1PR0802MB2478:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0802MB2478AF4E0A9E5E508DE5C99FFCA39@VI1PR0802MB2478.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 IRl3HUayaqf86YyQrZfUcFbTM17CkLzqpHU1celpBKn32bJb2s0eOq6kGqnhcuxb34hR4iZb0hh5X1Med8wzk9aXtedqq3gfr+38bEWYFJXDB5KYUOHjpEhBhGcbAQsqqy82TxdNy3IPcLP+4oTD/q4wILQ5/m22vZZ8xO0UquwtJjSZGAwxFCVKw1hOYAhLc8OHbfoM3PC1zsUCy8CpkucENYqhWW3eXk7yWUjZk8zEVZjxvcisGCd8zNtfCxLpjygYhEVT2D8DmrUv1DXZ+GazzFBI7YHJz4FVFHRwxe45uNaCxT9B2LTzV2FlD9AakR4Lpwa9jyr3QIcVkg9sdgl1kpcYfACZ/d/AvV3JzCWjZEBOGmNZHzU8k4SGp0GtMFcS7X52onLUM42ym7LHx+pkz7iriZ8x7JhaIrAV0rJ/3FLxc0g3+jNyPlPOlqbyCYkSTHBm0jCKbpYeJb4EwIobvQDu9N3dK05TWt6zNt8j4IqyttDHske4bez4JxV4Buuyn4Kk+5vqa0IiSStqwrTBxkG4UFKllToz8dZ6CZ5x82G2yYz93pPdJ1Fb+tccteJdSU3ZKW+GZXk2aiVt3Qeub8jhgfV4UW6uLEvRIHeb4EV+i449hg5OZUnr0Qp6hqUgBK9sPvC4LzhyP2gpTp6Rs0b7ok/cZcyQsq2m76m9YUhGrw38IXjMTzHXEpPS1/xT2kscuWy50MGJbZK2REhzWOkKX36yuzZyqnwrAJAgZXJvf2Nrv5KhOjT23Kis3IufdUj/H5h3WAiFqJto61yAW+9YsVepLSpHIF27ogJZG9BuDKLKbeBetlkf18ZnO8F6QqSeufdAsUKXICcFKQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(6512007)(83380400001)(38070700005)(86362001)(91956017)(6486002)(8936002)(122000001)(64756008)(38100700002)(5660300002)(2906002)(186003)(8676002)(6506007)(71200400001)(316002)(33656002)(76116006)(53546011)(66476007)(36756003)(2616005)(66446008)(6916009)(54906003)(66946007)(26005)(4326008)(66556008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <466D6902CD2425468EFE7624F66E4FAA@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2385
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bb032813-d932-48bc-6cd9-08d97ea1e70e
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JEDcKUPGvQ7Pgj+pozya6E1YOWqKpM84mDRlF5PGn3sosnTmwUyI42GKxCaZnlRW/c6rC95Dmc80d0Vw3m+f4kErTec1Z203l3pdi275XfHhQ887arc7FTrFpnp69sPpyYvoC3O3ymFJL9mqlwObHQjcgu/j/s+u045ErW8j8D+sBj6z/CHQJjbzSLFDpVviv56PQ+3iKL3x6JjibFQn4SpqP1L6/cmn6zhD+fnYmUeNaOwZkHnmVyF5pWMDkGBjyhlTNGv1xr5/Vxv/AJGamaHwubo2AmO2VO6b9FtdYz/5CcPt18flewcHxyvPMDlIvdWlCs67FIiNZdPAcH/UhHDF2T9Vovif3okUtNt+1BPv6pDJUTMGSeJcXHh7Lg6t9QmJ5frLFndRKaugMgMZ8FPm2yiOsh5i/nlTSUe4qXjkewrn66rTYRdnlgeYo2XV7FT4OB+rdiEfxCpYOfmvHFsyxIyOx3dlWasQe9W2/YOGay4BXwUE/KXj/5/n3jJZvtI5LXCjXN5pmpwMZr3t7aTFngZg6K6ZfxExUv+Gzppx1GMSkJCduwLAv/Y+lqAP7v7o9VjjiEoYMx60h13nh0Z0U7jMglQlU7h+USSIS8zcBpptt7eq4Q458ZqBiJxg0aCDrQxWj7VXxWiInwj42WfUszBbmwcMGUbxrOyQLwad5rNhwP9/CCfbw4f9ymLc0Il6Wo9S5pAR0H8Uxaw6Hx5+WTbBa87GEsbqmAoJwuHQICJzkvQqi3h+oZgufZdgZQPUbxPWKrun+J8fcNb0safHF/BzUug1Cn0VgtKgmVM=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(26005)(8676002)(70586007)(70206006)(82310400003)(8936002)(83380400001)(336012)(4326008)(2616005)(2906002)(6486002)(33656002)(36860700001)(186003)(6512007)(53546011)(6506007)(356005)(47076005)(54906003)(5660300002)(36756003)(107886003)(316002)(6862004)(81166007)(508600001)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 14:53:42.2509
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ca394ab5-2181-4226-5409-08d97ea1ef7f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2478

Hi Stefano,

> On 23 Sep 2021, at 1:03 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> pci_init(..) will be called during xen startup to initialize and probe
>> the PCI host-bridge driver.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2:
>> - ACPI init function to return int
>> - pci_segments_init() called before dt/acpi init
>> ---
>> xen/arch/arm/pci/pci.c       | 54 ++++++++++++++++++++++++++++++++++++
>> xen/include/asm-arm/device.h |  1 +
>> 2 files changed, 55 insertions(+)
>>=20
>> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
>> index a7a7bc3213..71fa532842 100644
>> --- a/xen/arch/arm/pci/pci.c
>> +++ b/xen/arch/arm/pci/pci.c
>> @@ -12,6 +12,10 @@
>>  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>  */
>>=20
>> +#include <xen/acpi.h>
>> +#include <xen/device_tree.h>
>> +#include <xen/errno.h>
>> +#include <xen/init.h>
>> #include <xen/pci.h>
>>=20
>> /*
>> @@ -22,6 +26,56 @@ int arch_pci_clean_pirqs(struct domain *d)
>>     return 0;
>> }
>>=20
>> +static int __init dt_pci_init(void)
>> +{
>> +    struct dt_device_node *np;
>> +    int rc;
>> +
>> +    dt_for_each_device_node(dt_host, np)
>> +    {
>> +        rc =3D device_init(np, DEVICE_PCI, NULL);
>> +        if( !rc )
>> +            continue;
>=20
> This is a NIT, so feel free to code it as you prefer, but I would write
> it as follows for simplicity:
>=20
>=20
> /* comment about EBADF and ENODEV */
> if ( !rc || rc =3D=3D -EBADF || rc =3D=3D -ENODEV )
>    continue;
> return rc;

Ack.=20
>=20
>=20
>> +        /*
>> +         * Ignore the following error codes:
>> +         *   - EBADF: Indicate the current is not an pci
>                                             ^ device ^a   ^ device

Ac.
>=20
>=20
>> +         *   - ENODEV: The pci device is not present or cannot be used =
by
>> +         *     Xen.
>> +         */
>> +        else if ( rc !=3D -EBADF && rc !=3D -ENODEV )
>> +        {
>> +            printk(XENLOG_ERR "No driver found in XEN or driver init er=
ror.\n");
>> +            return rc;
>> +        }
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +#ifdef CONFIG_ACPI
>> +static int __init acpi_pci_init(void)
>> +{
>> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
>> +    return 0;
>=20
> Should return ENOSYS or EOPNOTSUPP?

I think EOPNOTSUPP is right choice.

Regards,
Rahul

>=20
>=20
>> +}
>> +#else
>> +static inline int __init acpi_pci_init(void)
>=20
> Not sure I would inline it but OK either way
>=20
>=20
>> +{
>> +    return -EINVAL;
>> +}
>> +#endif
>> +
>> +static int __init pci_init(void)
>> +{
>> +    pci_segments_init();
>> +
>> +    if ( acpi_disabled )
>> +        return dt_pci_init();
>> +    else
>> +        return acpi_pci_init();
>> +}
>> +__initcall(pci_init);
>> +
>> /*
>>  * Local variables:
>>  * mode: C
>> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
>> index ee7cff2d44..5ecd5e7bd1 100644
>> --- a/xen/include/asm-arm/device.h
>> +++ b/xen/include/asm-arm/device.h
>> @@ -34,6 +34,7 @@ enum device_class
>>     DEVICE_SERIAL,
>>     DEVICE_IOMMU,
>>     DEVICE_GIC,
>> +    DEVICE_PCI,
>>     /* Use for error */
>>     DEVICE_UNKNOWN,
>> };
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:55:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:55:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194366.346246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ7r-000478-Ow; Thu, 23 Sep 2021 14:55:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194366.346246; Thu, 23 Sep 2021 14:55:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQ7r-000471-Lk; Thu, 23 Sep 2021 14:55:03 +0000
Received: by outflank-mailman (input) for mailman id 194366;
 Thu, 23 Sep 2021 14:55:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQ7r-00046v-6J
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:55:03 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3a5a4df4-1c7e-11ec-ba4d-12813bfff9fa;
 Thu, 23 Sep 2021 14:55:01 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id EC0512236C;
 Thu, 23 Sep 2021 14:55:00 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C73D913E76;
 Thu, 23 Sep 2021 14:55:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id wSNSL0SVTGFnfQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:55:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a5a4df4-1c7e-11ec-ba4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632408900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=my7VqY53A6e1Wd90ewQAn8Sl1sh3HuXoPlhSNUiEzmI=;
	b=EO78IeaGVeI9AbNU3/Oo2sIhF+U1+YlUVfZWAr10ZWTMqchVE+WF5xlgvH+WPmnxvW/29R
	5R0LpnckeK1fJMx5c621/D6rcNYqmo8VRF3RhEyfyJPaYtEUZaoYGmzGcPTjLxQlzNX86X
	ioKLPIS4JgZMHHq3J0YxGw+7hC85R28=
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <0b7afef6-1c46-ed74-ca83-f1e29f763f4a@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 6/9] xen/x86: generalize preferred console model from PV
 to PVH Dom0
Message-ID: <eac38453-a0d9-69ab-8fa2-35b3c55933de@suse.com>
Date: Thu, 23 Sep 2021 16:54:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <0b7afef6-1c46-ed74-ca83-f1e29f763f4a@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="91iDXi0U6EUiXy5e5GnMRq1VY3EqTjDTl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--91iDXi0U6EUiXy5e5GnMRq1VY3EqTjDTl
Content-Type: multipart/mixed; boundary="KJGvBtqHlnedeq33aAE3U0cqUhtZBlVt0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <eac38453-a0d9-69ab-8fa2-35b3c55933de@suse.com>
Subject: Re: [PATCH 6/9] xen/x86: generalize preferred console model from PV
 to PVH Dom0
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <0b7afef6-1c46-ed74-ca83-f1e29f763f4a@suse.com>
In-Reply-To: <0b7afef6-1c46-ed74-ca83-f1e29f763f4a@suse.com>

--KJGvBtqHlnedeq33aAE3U0cqUhtZBlVt0
Content-Type: multipart/mixed;
 boundary="------------B81120A55C794E6A11CAA6AE"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------B81120A55C794E6A11CAA6AE
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:10, Jan Beulich wrote:
> Without announcing hvc0 as preferred it won't get used as long as tty0
> gets registered earlier. This is particularly problematic with there no=
t
> being any screen output for PVH Dom0 when the screen is in graphics
> mode, as the necessary information doesn't get conveyed yet from the
> hypervisor.
>=20
> Follow PV's model, but be conservative and do this for Dom0 only for
> now.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>

> ---
> Prior to 418492ba40b2 ("x86/virt/xen: Use guest_late_init to detect Xen=

> PVH guest") x86_init.oem.arch_setup was already used by PVH, so I assum=
e
> the use of this hook is acceptable here.

Yes, I think so.

> Seeing that change, I wonder in how far setting xen_pvh to true only in=

> xen_hvm_guest_late_init() can really work: This hook, as its name says,=

> gets called pretty late; any decision taken earlier might have been
> wrong. One such wrong decision is what gets added here - preferred
> consoles won't be registered when taking that path. While adding a 2nd
> call there might work, aiui they would better be registered prior to
> parse_early_param(), i.e. before "earlyprintk=3D" gets evaluated.
>=20
> I also consider tying "detecting" PVH mode to the no-VGA and no-CMOS-RT=
C
> FADT flags as problematic looking forward: There may conceivably be
> "legacy free" HVM guests down the road, yet they shouldn't be mistaken
> for being PVH. Most of the XEN_X86_EMU_* controlled functionality would=

> seem unsuitable for the same reason; presence/absence of
> XENFEAT_hvm_pirqs (tied to XEN_X86_EMU_USE_PIRQ) might be sufficiently
> reliable an indicator. Question there is whether the separation
> introduced by Xen commit b96b50004804 ("x86: remove XENFEAT_hvm_pirqs
> for PVHv2 guests") came early enough in the process of enabling PVHv2.

Yes, it did. The boot path not using the PVH specific entry point was
enabled with Xen 4.11, while commit b96b50004804 was in 4.9.


Juergen

--------------B81120A55C794E6A11CAA6AE
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------B81120A55C794E6A11CAA6AE--

--KJGvBtqHlnedeq33aAE3U0cqUhtZBlVt0--

--91iDXi0U6EUiXy5e5GnMRq1VY3EqTjDTl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMlUQFAwAAAAAACgkQsN6d1ii/Ey8x
iAf/fgB9XcFj3SYVqzq+hQfPHb/7f8TzC1lVTGPiQA7e9d4ro//aIspKnnpzSujRBBA1vbdcNgqS
2g2xF8pJkE2idoYXUcIJbGr+/r+3mOjfqjQSCDC6OcoR1qBC8kTWChjjHw8Wufb8nqF7j4ilW30p
kjVKta6xvUN7hSw6KIvjRrejw8SnjkgeESP8kLKEm71uWuCbR7URli2MI3G2lDORfIlwoRyBfETY
Ti6vQNgYPU7Ol2i+syUPxhDdnfF96niNSKFyNqzETQMnteuoO+yxBwlAoejaP2zjIXDt2sHRkKvK
0WcvpWYxl3qY51WgYc1ivB7G2Bk84fAO553ibr4OiA==
=Xbvv
-----END PGP SIGNATURE-----

--91iDXi0U6EUiXy5e5GnMRq1VY3EqTjDTl--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 14:59:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 14:59:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194376.346257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQCI-00056p-Cb; Thu, 23 Sep 2021 14:59:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194376.346257; Thu, 23 Sep 2021 14:59:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQCI-00056i-82; Thu, 23 Sep 2021 14:59:38 +0000
Received: by outflank-mailman (input) for mailman id 194376;
 Thu, 23 Sep 2021 14:59:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQCH-00056c-Df
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 14:59:37 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id de178632-1c7e-11ec-ba4d-12813bfff9fa;
 Thu, 23 Sep 2021 14:59:36 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 936FC20294;
 Thu, 23 Sep 2021 14:59:35 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6CAED13E76;
 Thu, 23 Sep 2021 14:59:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id J8QkGVeWTGGZfwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 14:59:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de178632-1c7e-11ec-ba4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632409175; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=9HFropYBJ8YpRGx+QAcqDurcWRDcCojO5AfgHlSOxT4=;
	b=SbS4Z1xAOBDtDVBpudYlPGztHRFCTLtFO0doTbgtFuzS+B8lKXt3ayjgM0sW3WFunXsoCo
	RfEyplYTHeh5HrVUV/OtKtf+x2L5CSKiAUZYmB63t32f4vlHc4Z9cGoBXOcZIFXWKnhQiy
	5McbjcZAzqZgJRkboKOa6GQj0NABzzk=
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
Date: Thu, 23 Sep 2021 16:59:34 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="gTgklXBTE0AbhosPPX93qUzO32E1efRzU"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--gTgklXBTE0AbhosPPX93qUzO32E1efRzU
Content-Type: multipart/mixed; boundary="qNdT34BbLw9pHUlJhY8krWdCZq8J6erIV";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
In-Reply-To: <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>

--qNdT34BbLw9pHUlJhY8krWdCZq8J6erIV
Content-Type: multipart/mixed;
 boundary="------------E71AE326E9F0E8F3E1ECD495"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------E71AE326E9F0E8F3E1ECD495
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:11, Jan Beulich wrote:
> This was effectively lost while dropping PVHv1 code. Move the function
> and arrange for it to be called the same way as done in PV mode. Clearl=
y
> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
> check that was recently removed, as that's a PV-only feature.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>=20
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>   	return ((per_cpu(xen_vcpu, cpu) =3D=3D NULL) ? -ENODEV : 0);
>   }
>  =20
> +void __init xen_banner(void)
> +{
> +	unsigned version =3D HYPERVISOR_xen_version(XENVER_version, NULL);
> +	struct xen_extraversion extra;

Please add a blank line here.

> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
> +
> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);

Is this correct? I don't think the kernel needs to be paravirtualized
with PVH (at least not to the same extend as for PV).


Juergen

--------------E71AE326E9F0E8F3E1ECD495
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------E71AE326E9F0E8F3E1ECD495--

--qNdT34BbLw9pHUlJhY8krWdCZq8J6erIV--

--gTgklXBTE0AbhosPPX93qUzO32E1efRzU
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMllYFAwAAAAAACgkQsN6d1ii/Ey8U
sgf/U4ARXy/yLmreMwAjpcLLtIiwgLKoQ8NJaIlamaTGjEg9OyTPKrAp45kFsYa71XqXrvtis+Sl
xNQcCOCOa01qV9ECGDHE/1m3NVZeeg4hGbjLiV+zzqZxbifoqQQg8h0wBWfbWaecVtV5YcRfh7aS
WMe5FqL0VVfIPeERvD2xDogIQS0djxVlLX4R0q6zFF+su0UyGnDQjIzhze+wgRoefYAg4SqpSbRH
gkoy8L0tKxALwf7yWuhjMQeLk30rHZZi4whyNp+2Tl7oM0y34YCv+TSF1sPS5oWNYGpx1FCqz+6/
9depZ6bN9kAiWJAXCN6Xa2qaoGLj4DBWaDMR7AdVJg==
=t0Qw
-----END PGP SIGNATURE-----

--gTgklXBTE0AbhosPPX93qUzO32E1efRzU--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:02:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:02:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194383.346268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQEk-0006XC-Rd; Thu, 23 Sep 2021 15:02:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194383.346268; Thu, 23 Sep 2021 15:02:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQEk-0006X5-OM; Thu, 23 Sep 2021 15:02:10 +0000
Received: by outflank-mailman (input) for mailman id 194383;
 Thu, 23 Sep 2021 15:02:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQEj-0006Wz-UN
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:02:09 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e88feefc-3b36-46c9-ac03-1af8b74fc9ae;
 Thu, 23 Sep 2021 15:02:09 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6341A20294;
 Thu, 23 Sep 2021 15:02:08 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3BB8513E76;
 Thu, 23 Sep 2021 15:02:08 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id P1NwDfCWTGEwAgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 15:02:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e88feefc-3b36-46c9-ac03-1af8b74fc9ae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632409328; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=jxkpWB9pOzMT5Ky0Sf1nmXwSl8g4r5MQkHH+2F49T1M=;
	b=mE8lBu+QFbB7uuyU1VWNDGbUOeFjnYndLQKJS93e8ExPOI7s6PFDrYeX0e++T3zh7ir+5J
	w9XfIaTRg5vYVhv3zxwjoUvltMhJkELZk3jklIQ3oCzcRUeWZEqfuzSjbl3BrNoarh82Pj
	amyCumuRaHA2GJKtuU77GThKgddqwgU=
Subject: Re: [PATCH 8/9] x86/PVH: adjust function/data placement
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <cf3fd833-0379-aba5-421d-7f2588d2d881@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <42fc11c8-de4a-4f00-8bef-1f9268551f4b@suse.com>
Date: Thu, 23 Sep 2021 17:02:07 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <cf3fd833-0379-aba5-421d-7f2588d2d881@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="tdg01WsrPs5385VXsxmJfbLOaCgKlOHjq"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--tdg01WsrPs5385VXsxmJfbLOaCgKlOHjq
Content-Type: multipart/mixed; boundary="8U5rkdWgyq2Y57YzPy0pQ2feesNhNjJUi";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <42fc11c8-de4a-4f00-8bef-1f9268551f4b@suse.com>
Subject: Re: [PATCH 8/9] x86/PVH: adjust function/data placement
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <cf3fd833-0379-aba5-421d-7f2588d2d881@suse.com>
In-Reply-To: <cf3fd833-0379-aba5-421d-7f2588d2d881@suse.com>

--8U5rkdWgyq2Y57YzPy0pQ2feesNhNjJUi
Content-Type: multipart/mixed;
 boundary="------------DC890C5714C660458BFB6630"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------DC890C5714C660458BFB6630
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:12, Jan Beulich wrote:
> Two of the variables can live in .init.data, allowing the open-coded
> placing in .data to go away. Another "variable" is used to communicate =
a
> size value only to very early assembly code, which hence can be both
> const and live in .init.*. Additionally two functions were lacking
> __init annotations.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------DC890C5714C660458BFB6630
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------DC890C5714C660458BFB6630--

--8U5rkdWgyq2Y57YzPy0pQ2feesNhNjJUi--

--tdg01WsrPs5385VXsxmJfbLOaCgKlOHjq
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMlu8FAwAAAAAACgkQsN6d1ii/Ey/u
ogf/Xg3OFbD3rxakzq49yd7jHeyC1xaLcZB6zQrMJ2eoRNGzIu2+5UEnDZd5UjlX6oytv6lrNy/n
ieEhgybp0Pd+P1JlcQh2U0FKWve616nkQrZgSbhtwyJ3+50pSQChkbKvJtZeT4ri1xSfx4hV3bwY
irp0o7hmY4SRV1JPJ/fLVywtt5SXH3SSYbs+lh2Sui1P1B2OzeW205Z9o6cwAyJO1Oiwiqo/MbwH
8bjjqzD6Y2/pRZL9eokVUeJ6kvg+PuRz0VYLQmU/UcvD5jnWDpQq1K1hZix7oWeFfsusYRTBP1Pm
AtIRDABgiZbwJ/cqSAYzPtKZfChfxLvCnxg7V0G0Aw==
=Ps0F
-----END PGP SIGNATURE-----

--tdg01WsrPs5385VXsxmJfbLOaCgKlOHjq--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:04:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:04:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194388.346278 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQHK-0007AY-9N; Thu, 23 Sep 2021 15:04:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194388.346278; Thu, 23 Sep 2021 15:04:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQHK-0007AR-6R; Thu, 23 Sep 2021 15:04:50 +0000
Received: by outflank-mailman (input) for mailman id 194388;
 Thu, 23 Sep 2021 15:04:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQHJ-0007AL-3Q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:04:49 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 97f19d72-1c7f-11ec-ba50-12813bfff9fa;
 Thu, 23 Sep 2021 15:04:48 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 5BFBB1FD83;
 Thu, 23 Sep 2021 15:04:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2CE6613E82;
 Thu, 23 Sep 2021 15:04:47 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id CEW9CY+XTGGfAwAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 15:04:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97f19d72-1c7f-11ec-ba50-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632409487; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=NjQbEyoSx0YxSIwoKNcSM/Z7wVUqWHXkni/5eDVktMA=;
	b=N7S8vWIo5mF9BTTYwZWmiNvQBnfhtlZTf70WXpdZxxd/utSfu69qJTn3G3OPZ0qp2q13wE
	fKiRZL2YLO7AHKts3it/7RBLroF8g35p2VFNSlfyuM/rSolk0isKlCxNOFPL7ZvbZXqJQX
	GwB8c8uKOhcNJpiLXK5ghIX0kt1TlcM=
Subject: Re: [PATCH 9/9] xen/x86: adjust data placement
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <a7d9936d-2fa8-507e-5b0a-a4930eb457dd@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <97182850-ebdc-aac2-5265-a3cac9244979@suse.com>
Date: Thu, 23 Sep 2021 17:04:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <a7d9936d-2fa8-507e-5b0a-a4930eb457dd@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="0Sx9cgE53kdTwMqk7K9cToIUkiBuNibHj"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--0Sx9cgE53kdTwMqk7K9cToIUkiBuNibHj
Content-Type: multipart/mixed; boundary="H5gPcRA7OkalQqgpm7NNjNEglpEMbritD";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <97182850-ebdc-aac2-5265-a3cac9244979@suse.com>
Subject: Re: [PATCH 9/9] xen/x86: adjust data placement
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <a7d9936d-2fa8-507e-5b0a-a4930eb457dd@suse.com>
In-Reply-To: <a7d9936d-2fa8-507e-5b0a-a4930eb457dd@suse.com>

--H5gPcRA7OkalQqgpm7NNjNEglpEMbritD
Content-Type: multipart/mixed;
 boundary="------------8851BDAFF3F6C844886BE608"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8851BDAFF3F6C844886BE608
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 07.09.21 12:13, Jan Beulich wrote:
> Both xen_pvh and xen_start_flags get written just once aeryl during
> init. Using the respective annotation then allows the open-coded placin=
g
> in .data to go away.
>=20
> Additionally the former, like the latter, wants exporting, or else
> xen_pvh_domain() can't be used from modules.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I have to admit that it is completely unclear to me which form of
> exporting I should have used: xen_domain_type is GPL-only while
> xen_start_flags is not, yet both are used in similar ways, extending to=

> xen_pvh.

I'd use EXPORT_SYMBOL_GPL(xen_pvh).


Juergen

--------------8851BDAFF3F6C844886BE608
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8851BDAFF3F6C844886BE608--

--H5gPcRA7OkalQqgpm7NNjNEglpEMbritD--

--0Sx9cgE53kdTwMqk7K9cToIUkiBuNibHj
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMl44FAwAAAAAACgkQsN6d1ii/Ey+1
/gf/UqD3JRe6kpdcgdfU1RG/Pw01LoglcuTSAWRJ33pYK5kMLvjbqLD2PFNbaMvPdS7GL1ezLmBm
XVXrJ6LnyU2d6JmIO/vz8ThaVcdAyjS1q4VrUYCZd3GVw/KPTvYMQrmD2BoBhZlGO8B+1VSaCNwB
N9b2LonVm5wlSPGruIz36opNwo0lzpd+8QDQFDKq8dQ11IsuSx141isYzcyNTse6OMh8kDiq+3+4
GKWcKpWVPH0TeQM4dJvWL7AO2dP9HMLfImKaVs0KHUfNnCTY5eGkwX/bBCxvzpuGz3Zpj5ivUzJS
aLWBnHafyugQSyt7SYCU47EOG4koBfrRvCzlrPO56g==
=QB+E
-----END PGP SIGNATURE-----

--0Sx9cgE53kdTwMqk7K9cToIUkiBuNibHj--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:05:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:05:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194391.346290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQHf-0007e1-JH; Thu, 23 Sep 2021 15:05:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194391.346290; Thu, 23 Sep 2021 15:05:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQHf-0007du-Fg; Thu, 23 Sep 2021 15:05:11 +0000
Received: by outflank-mailman (input) for mailman id 194391;
 Thu, 23 Sep 2021 15:05:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQHd-0007cJ-VT
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:05:10 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.70]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ab828809-2483-4b72-842d-51463d976851;
 Thu, 23 Sep 2021 15:05:07 +0000 (UTC)
Received: from AM6P193CA0051.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:8e::28)
 by AM5PR0801MB1699.eurprd08.prod.outlook.com (2603:10a6:203:3b::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 15:05:04 +0000
Received: from AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8e:cafe::dd) by AM6P193CA0051.outlook.office365.com
 (2603:10a6:209:8e::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 15:05:04 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT058.mail.protection.outlook.com (10.152.17.48) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:05:04 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Thu, 23 Sep 2021 15:04:06 +0000
Received: from f55a7396d942.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7E245724-0687-488F-BDB8-42B1E2FC0054.1; 
 Thu, 23 Sep 2021 15:03:57 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f55a7396d942.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:03:57 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1620.eurprd08.prod.outlook.com (2603:10a6:203:3a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 15:03:55 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:03:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab828809-2483-4b72-842d-51463d976851
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zSxdhoMMbF/mqnw131qdf2hO7pXPwEWkw6Pj7c3lmLQ=;
 b=vXvlvjFn7eEqX1twUL21XLYCCHsOnYn1Hil7g5C17fWxGbcpU/ZSbd0tKRtmv0tlWWxGEUsdwE5XG0FpkGlmCCZ7unVE7VkbbIWictjzvVtK9BmT3K8eSuVy1FiKI0KNwLNFct16cU4knoKyKmgD1p8Yq+QrKy9JS+2677tAl+s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 257b36c61aa6ca08
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TAoQRy6S79UqOVqGo3BIg9eOm633gw6FY3RqT9f1WjQoEbTBHvBLl3K0X+FPQlbD/mmPBWjGK2dPZMbZL73oHxfy0a2RMuEvmzxKGJjxViVFNWKEn9DY1Z3a4IFkdlbIMSrvx2Vnt+LwL+8dorkHfsXbmY/9yN4BrTup2GtDhU0oX4vMjzLqtsm2hTJbWpY/tooSn7PUxYruojaEIKz6kvEIWpqCgVP6j4ngP8z4CrYgQO75FnQpcFqhqYzqtuFuc7jZhqseDTwQtbRmfpgQPLQ+J+EFWici1jl+7cMyC77kKKuuC7nYS5izyY4aY4LZNd6VujQTNSidpx5zbfPLXg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zSxdhoMMbF/mqnw131qdf2hO7pXPwEWkw6Pj7c3lmLQ=;
 b=fuieuE9r6pyen4oAC7+gvaOhHQy6M9BzKkIuWEAt3J4n207JYK8gHl1whBsXtjgb0Ebydhoqi74gp7Lfsb7+mijn2bi6i752LkLiU8yyEqSa8xNR5AG7eXFHhAMxD8HWRKeeHNJ+S3192gj+7aFv9AIMn5LJytP7120HlcIFt5hY44yP6DrRslgC7SDi3BREoOlCWfQ0dSsKT1ExY6mJVU2WuLOwR7C9VjI9SIRiQTXrR6n4O05YW2lzNX8KzlZKEI0cjbSh2z/gmXUk/UHzMsSgHoNHtDHP4lfEABG6/+o9cauvXL8R533PoWObMV+87+b/nHf09szwPyw1WpqNTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zSxdhoMMbF/mqnw131qdf2hO7pXPwEWkw6Pj7c3lmLQ=;
 b=vXvlvjFn7eEqX1twUL21XLYCCHsOnYn1Hil7g5C17fWxGbcpU/ZSbd0tKRtmv0tlWWxGEUsdwE5XG0FpkGlmCCZ7unVE7VkbbIWictjzvVtK9BmT3K8eSuVy1FiKI0KNwLNFct16cU4knoKyKmgD1p8Yq+QrKy9JS+2677tAl+s=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien
 Grall <julien@xen.org>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>
Subject: Re: [PATCH v2 10/17] xen/arm: Add cmdline boot option
 "pci-passthrough = <boolean>"
Thread-Topic: [PATCH v2 10/17] xen/arm: Add cmdline boot option
 "pci-passthrough = <boolean>"
Thread-Index: AQHXr6aLQPm07B/B+UG8SHm5ElV1K6uwwKwAgAD4egA=
Date: Thu, 23 Sep 2021 15:03:55 +0000
Message-ID: <F6A0717A-C188-4E15-9DD6-196BA58DADA7@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <9dcb9b3b6b6923db00d6e56da26a8503d5a4855a.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221708500.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221708500.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 6e10b345-a6bd-4a32-3b67-08d97ea385ec
x-ms-traffictypediagnostic: AM5PR0801MB1620:|AM5PR0801MB1699:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM5PR0801MB1699A52DDF8927453CA7907DFCA39@AM5PR0801MB1699.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Hf/6on+0XXqsdVxHTN2HrsKvQLdjo8kzWfaoNG9UZOh8GBRisNhv9d7RGWc8kmIloMz5Eix1LsZm5jjn+mC5vIGb/Gmm6pOed3w8Z5wYwAf5/CkoUdz6Pmv5M4SevYSZ+LV4SpjiBW8KnrutCry/T6hJY8ONITWn0rcC60TlwQA5EN8721aMMMUnlgsFpS/KmbNAQcrMhQFwe59zKi148fgwamDuZrlvNtrSA8IZjyYGV/qMCBRds9LZkN3Z0/gc83B6ybCiPPoX2VJ8xCy/ggqPKmYg9VOanwQq8osoQKH6AiQZQoT1zoQ62g/mnWtKLoeRoa6egMAW2OkYyUtWgElrv8l6Ut51J6HCzXRWbQscNC5jTDCggBkBWpL6Sw+Nck9OjD39kKI1beq15W4NK09m47DxDGFkYh40vruTFHD2ncmuU3IGEuNKsMAQqoLZqD89KiovVZto5EcwLzKZmISG0T8mnxfcvJdqdpHXYMnPgiOYVIEOQl408R/e4JpwRo5ByqHOwpsdYbPTwDE2up0ZadSIh9zZ/skNzAabYuvzDFh3nEnxNQ69XrDggQlyXGSnoEa/c1nYUqM4RxaN/HjbFWXBQSLA7oj7WqtcYZydj9IiLMCilFl5S77UxiqHKcA93LejoZJ782Uo6w/GVqDx8e16qwatLSnaPNIcqdD3fq/nhSkxF3gPGtg0oSReVP0CaSbB8rJEJ4Wenz+7K+yT5gDZ4/Mj5Wof9mW8Gfr7IZd39FC0qug8MDCAWSBm
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(83380400001)(66446008)(76116006)(316002)(5660300002)(66946007)(38070700005)(6916009)(8676002)(91956017)(66556008)(8936002)(36756003)(6486002)(86362001)(2906002)(7416002)(6506007)(53546011)(2616005)(4326008)(6512007)(38100700002)(122000001)(71200400001)(66476007)(186003)(26005)(54906003)(33656002)(508600001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <77DF9E54E6B45344B8237F370BAEE72D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1620
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	72324085-558c-492a-1c02-08d97ea35cf2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pk1stN4VxxxIZPTXTpOo4eC0lvfEC5ZQAJtNqclIfu8SOnw3hfBeOBNafj3GKKy+G7B3ZMNTLVbwO3SkhlVdAmf9y7eS8JNmXnkXRbqHQf8isgjjNFGpJ/z5GYwXdX6OnVTaZ2UyXCafUqtNsTr7/nzMvTU1ULR8+MDL8O8XpZIWjUlW0/EVDpg9Vec45/reOfAkKSWwCPNejuEGSNaKAH82hai7cu0iVqGeCjI+0htp456Y/w355bUUA0jNeY2wKIiFOn5OysZdbTWYJwW3v/F9n7GSfU8nbqI4cwx3SKXmbZgmHU5FtUqe33jdo9jrCudI271ixqxmrvQSsutUwVrc0MlE8+ErBVLBBgM3TwnE4WloTKWJOSRe6FI/QW9NeHW2SGpmtAn9bCunE0KNmliCN/hOOZMokGkWHfQ+p5EQ5xgqWegk2lYW89aBdBfCpLuK6VkfydjCF5KBSb3FMvbrdmp0TsrglrpwO3n7hYhPPaT2k/IO+5XO+IojG0334RXewRVFap3shHX7+iV/OtQhTf67sOTJtL79JAB+kiYQkPi3+w9R9UkRlfsxPs8Kqsrt4MX3k4ji24IF36Ki653lBRtdeAmg9Q62x4PacziSprs91clgVMNv5KzQNwpi7Zf9AhSZ8opROVh0MK7gQ3OUuKVolWPaCImqZVlvhVnRWF2QGxG0XaMgyqKVJPVmF1Jv+deZ5Zkxf+PX3eorIw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36756003)(83380400001)(26005)(356005)(186003)(4326008)(107886003)(70206006)(70586007)(6862004)(8936002)(86362001)(82310400003)(54906003)(508600001)(336012)(47076005)(316002)(81166007)(36860700001)(5660300002)(33656002)(2906002)(8676002)(6512007)(6486002)(2616005)(6506007)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:05:04.0720
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e10b345-a6bd-4a32-3b67-08d97ea385ec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1699

Hi Stefano,

> On 23 Sep 2021, at 1:14 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Add cmdline boot option "pci-passthrough =3D =3D <boolean>" to enable
>> disable the PCI passthrough support on ARM.
> ^ or disable
Ack.
>=20
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2:
>> - Add option in xen-command-line.pandoc
>> - Change pci option to pci-passthrough
>> - modify option from custom_param to boolean param
>> ---
>> docs/misc/xen-command-line.pandoc |  7 +++++++
>> xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
>> xen/common/physdev.c              |  6 ++++++
>> xen/include/asm-arm/pci.h         | 13 +++++++++++++
>> xen/include/asm-x86/pci.h         |  8 ++++++++
>> 5 files changed, 48 insertions(+)
>>=20
>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-l=
ine.pandoc
>> index b175645fde..c867f1cf58 100644
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -1783,6 +1783,13 @@ All numbers specified must be hexadecimal ones.
>>=20
>> This option can be specified more than once (up to 8 times at present).
>>=20
>> +### pci-passthrough (arm)
>> +> `=3D <boolean>`
>> +
>> +> Default: `false`
>> +
>> +Flag to enable or disable support for PCI passthrough
>> +
>> ### pcid (x86)
>>> `=3D <boolean> | xpti=3D<bool>`
>>=20
>> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
>> index 71fa532842..fe96a9b135 100644
>> --- a/xen/arch/arm/pci/pci.c
>> +++ b/xen/arch/arm/pci/pci.c
>> @@ -16,6 +16,7 @@
>> #include <xen/device_tree.h>
>> #include <xen/errno.h>
>> #include <xen/init.h>
>> +#include <xen/param.h>
>> #include <xen/pci.h>
>>=20
>> /*
>> @@ -65,8 +66,21 @@ static inline int __init acpi_pci_init(void)
>> }
>> #endif
>>=20
>> +/*
>> + * By default pci passthrough is disabled
>> + */
>> +bool_t __read_mostly pci_passthrough_enabled =3D 0;
>=20
> I think we are using bool rather than bool_t nowadays. Also could do =3D
> false.

Yes I missed that I will use bool.
>=20
>=20
>> +boolean_param("pci-passthrough", pci_passthrough_enabled);
>> +
>> static int __init pci_init(void)
>> {
>> +    /*
>> +     * Enable PCI passthrough when has been enabled explicitly
>> +     * (pci-passthrough=3Don)
>> +     */
>> +    if ( !pci_passthrough_enabled)
>=20
> missing space

Ack.=20
>=20
>=20
>> +        return 0;
>> +
>>     pci_segments_init();
>>=20
>>     if ( acpi_disabled )
>> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
>> index 8d44b20db8..7390d5d584 100644
>> --- a/xen/common/physdev.c
>> +++ b/xen/common/physdev.c
>> @@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(vo=
id) arg)
>>         struct pci_dev_info pdev_info;
>>         nodeid_t node;
>>=20
>> +        if ( !is_pci_passthrough_enabled() )
>> +            return -ENOSYS;
>> +
>>         ret =3D -EFAULT;
>>         if ( copy_from_guest(&add, arg, 1) !=3D 0 )
>>             break;
>> @@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(vo=
id) arg)
>>     case PHYSDEVOP_pci_device_remove: {
>>         struct physdev_pci_device dev;
>>=20
>> +        if ( !is_pci_passthrough_enabled() )
>> +            return -ENOSYS;
>> +
>>         ret =3D -EFAULT;
>>         if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
>>             break;
>> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
>> index 7dd9eb3dba..f2f86be9bc 100644
>> --- a/xen/include/asm-arm/pci.h
>> +++ b/xen/include/asm-arm/pci.h
>> @@ -19,14 +19,27 @@
>>=20
>> #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>>=20
>> +extern bool_t pci_passthrough_enabled;
>> +
>> /* Arch pci dev struct */
>> struct arch_pci_dev {
>>     struct device dev;
>> };
>>=20
>> +static always_inline bool is_pci_passthrough_enabled(void)
>> +{
>> +    return pci_passthrough_enabled;
>> +}
>> #else   /*!CONFIG_HAS_PCI*/
>>=20
>> +#define pci_passthrough_enabled (false)
>=20
> We don't need to define it, do we?

No need to define I will remove.
>=20
>> struct arch_pci_dev { };
>>=20
>> +static always_inline bool is_pci_passthrough_enabled(void)
>> +{
>> +    return false;
>> +}
>> +
>> #endif  /*!CONFIG_HAS_PCI*/
>> #endif /* __ARM_PCI_H__ */
>> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
>> index cc05045e9c..0e160c6d01 100644
>> --- a/xen/include/asm-x86/pci.h
>> +++ b/xen/include/asm-x86/pci.h
>> @@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigne=
d int *seg,
>> extern int pci_mmcfg_config_num;
>> extern struct acpi_mcfg_allocation *pci_mmcfg_config;
>>=20
>> +/*
>> + * Unlike ARM, PCI passthrough always enabled for x86.
>                                 ^ is

Ack.

Regards,
Rahul
>=20
>=20
>> + */
>> +static always_inline bool is_pci_passthrough_enabled(void)
>> +{
>> +    return true;
>> +}
>> +
>> #endif /* __X86_PCI_H__ */
>> --=20
>> 2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:08:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:08:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194400.346301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQL1-0000Mq-80; Thu, 23 Sep 2021 15:08:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194400.346301; Thu, 23 Sep 2021 15:08:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQL1-0000Mi-35; Thu, 23 Sep 2021 15:08:39 +0000
Received: by outflank-mailman (input) for mailman id 194400;
 Thu, 23 Sep 2021 15:08:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQL0-0000Mc-HD
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:08:38 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.88]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2022d2b0-1c80-11ec-ba51-12813bfff9fa;
 Thu, 23 Sep 2021 15:08:36 +0000 (UTC)
Received: from PR3P189CA0047.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:53::22)
 by GV1PR08MB7332.eurprd08.prod.outlook.com (2603:10a6:150:1f::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:08:33 +0000
Received: from VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:102:53:cafe::6c) by PR3P189CA0047.outlook.office365.com
 (2603:10a6:102:53::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Thu, 23 Sep 2021 15:08:33 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT036.mail.protection.outlook.com (10.152.19.204) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:08:32 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Thu, 23 Sep 2021 15:08:31 +0000
Received: from 784eda64e86b.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4B056137-0EFC-417C-9CAC-CDB6804BF0BB.1; 
 Thu, 23 Sep 2021 15:08:19 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 784eda64e86b.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:08:19 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6680.eurprd08.prod.outlook.com (2603:10a6:20b:397::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 15:08:18 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:08:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2022d2b0-1c80-11ec-ba51-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tmEKL5FOlIqrwgL6aclc6rgeZDwEPx8m8trK9+m/LyA=;
 b=4OKqnAPai/f95/zUJPb096hXkqa9RqDh9qHgIqSiMHHI+70oYvjFoj0Gnhtj7OYSq+fRNARFEf9JcH54xyqwP6LCZTrRUfZynMElILMPREC9YnytNgcxZAoNjWqiwQC4aBTdJnFj5LekkTQnRtuwTRkpEo4k7zrU5y75YNYOZxw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6cb50aff5fec0c07
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=J0LTEFx2SEMdDS5Q8+0jxjaGEPD/I5cV3Zrfh+j0jl64zgH4eH27M4MdhqNCm02r0EJdlSpgTNrL0poAgOm52L5ls/dqyRkpROU8+bHhmNHX5ZJmnG4HI9uVr3j1xh3yIMRuIZYHQHGHboAzGy6BNBU/52ivBLMWp+nQYDnsg2YQDjxoBFBtFdnmk3PabXOrez20Nf7weblnWn6n0El7B44hy/9kxbvvHknciE8M0OHhPbklPsy/RjSZreoq+AoIsjjc/+VEvFvEOoqHrqFTeaWa7yBZnWIOtgC++sD2eRbikFQ/b+RawLtrF9tWX+1LTjTPXk0PAYdl1u0/O7PTIw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tmEKL5FOlIqrwgL6aclc6rgeZDwEPx8m8trK9+m/LyA=;
 b=Dvp2H9cjuSwhfycRWG4cgqynjXckdJi8kTtpxAJ5Fv+EmhvJF13ksAprCJjh0/TjuX9WKdc5fsvrAF9o11zRE2lP0cg0y0/WUo5Pt8nhx5Yw0Lf6Xdyb1mqlI50dQo9YGA99rRUAF3qu0SiQgYZ/ffCyC6jNrOHckjrozIKcYKPjsg38IDutOpcCcEFnguojUU1dhe/JCfqHyIPYDkNDJQoNfO2HYKbqVJWNAEL4nUzHe4B04BEN+0QB5GfxGQjA7wYNmI3cCQqlgXd0KNuzBN9DZhRxbJzKNUOKG4g6MMzgMf7D16CmGrSxg4Ygwx9iFsnZU9l1OhBRis8sr6LRQg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tmEKL5FOlIqrwgL6aclc6rgeZDwEPx8m8trK9+m/LyA=;
 b=4OKqnAPai/f95/zUJPb096hXkqa9RqDh9qHgIqSiMHHI+70oYvjFoj0Gnhtj7OYSq+fRNARFEf9JcH54xyqwP6LCZTrRUfZynMElILMPREC9YnytNgcxZAoNjWqiwQC4aBTdJnFj5LekkTQnRtuwTRkpEo4k7zrU5y75YNYOZxw=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v2 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXr6abGeE+lk7OX02wltbxtcd8MKuw4W8AgADY8IA=
Date: Thu, 23 Sep 2021 15:08:18 +0000
Message-ID: <5BB27951-3083-47AE-A6C8-910741171741@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <4b2b04ab1dbc1a5c52bf54c399cdb6cf454183ea.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221911020.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221911020.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: e04ed871-b62d-47d4-6601-08d97ea40270
x-ms-traffictypediagnostic: AS8PR08MB6680:|GV1PR08MB7332:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<GV1PR08MB7332EC9BD18C51871B7CD8FDFCA39@GV1PR08MB7332.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:3276;OLM:3276;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +B517C0AaJqRfExs+7MGyoqzn7p5wqSGbTj2iVdBto3Y/i4HUF9Ezka92BuY8Tkr7nurKMX5axf89GBXXUe5mdrQIXX9sknwani+qIdXJbKIrM9pYsxek9NS1HKUD8lQsF55Rh4UfkAVr7yzbcOhuKUcIj3F/FeHe/HKjJLVi4/uBQgaVx0dOuJJvhkKlIyBVVjo+ctqYG+WOYXKWDqlv7p5ch4p1voS8AXJ/JBuFjA7DdBrkQfiyse/7u2jPk8beZVGmcXr1VMwtMXjJL/OsR+kTCCwSeNq2Cev7liV+k6wVDu+aBu0N99JlTL2MDUMryk3EDKkafXWI2xqDT9KYDeQ9U7LwKf/c3B2EQww4r6Kvja3G4EMleeOcF3xvmjFAwNWlygIvlfHdjQpLLCOSOlFgNULS/Yd3OI0Ra92992BqaC5HX4GpHSQCUkK5N6xFE6Jkl8/VcssWdrHwL5vILTx+KbOoBOAeVc6Ktwb/ALDliU3DhorDWr0HZJg9ezfS+Ak+KUeIBsS5skskrO4oo6OXmO6m2q3ZODMhDtE05moVW906OL1lKW1NJrok7BIT1c7sg57V05GrU9XHkN4gg/vME6y1RQgG6rhbl40vNXMxvgufCp5+Xx5efEC/YTVJy5TNtgmoB90ouwZqKmULh2p9hD1oURZ0WFdKlcGKtimQEIdecL9JI/JEvDc/l4hUrFg0aB78WmgTDIOKM0vps6CfQ6oz3f2q4eQrnnKIa2mquLRgTL4wTknxJEgGLynhcTYvTPVAFATyFfq4TqNKaOst1XbkCLmQWPcsbSsfwmYLzcnmif+LRxE5nZsOsfT4hHTd7mz+ObILXivW3wv+9IgwLesLBRSyt4qBJf2jsM=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(2616005)(53546011)(38100700002)(26005)(91956017)(76116006)(86362001)(4326008)(66946007)(122000001)(6506007)(8676002)(66556008)(508600001)(66476007)(54906003)(38070700005)(36756003)(8936002)(6916009)(71200400001)(33656002)(186003)(316002)(5660300002)(66446008)(2906002)(64756008)(6512007)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D675A0CC20CE0D4FB469154FB27A1442@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6680
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8b87f70c-08f7-4d77-d2ae-08d97ea3f9b9
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OUoT46ZGFGs7+aH4d4jzr06g77oaxYx5s0WQwSGV8uwCYbC+7PkwWfhKiPsqWqcJiWIxRrheZwMk8VAokBKhb+l9fcG+P6y953bCWJ7e6iiD0sLcOR0A0GsEhNqk1mcK0VPOwzfWk5o45HUZngI3umR2QS3AViiuo7wZx2dQYBxj2ImPz+XAFcutsa4fwcOJK2UlfCDI7pmphARqN/GRsbDy0yLAVzyMRNvhFqruRe0z4ltW2wnrosUrl6/xx1YdLlUWhxN2labhHoJVSSYXYc+m6f1KR1j9DzNx+ifO9vxCbAkev5If659NiYz5CP+mpGMpIkB04Bx17RFFuMWMW5gv8+Qqterxoa+6gwM9n14PYOnEy3adivcFmD0vI3uwhwcPdacs23gw/TeXkhxbJi0o4rMFnifavLxCC4ZTuSYCXPttgIvyL0wnZtGqbXSeNcsjRYpao9Z2QEkrilGuNbznAOnlyzv6vX3op+tzmv9PIv51NqsoWpwBCPJIlIJ1Wdo5PsBvCVy5Z1+UOAcHN5w1ngCTRRv66jnnLJSrSj1Hc7ffnSMUXYF3I93JdEjMpgWhtHZjjmB+Dq1S+QXl5VuJoCEG3XQV2668U5M1SjiWmCoZbuDG3CNWiFusn3zHPZBvVNjhGYPbf2giSHJs92IogQo3HbPxILI2gFfAEId9RobpTcogfL654nsq/stB6hT/6sIfjeuQPhOB8OGQifmudJYrHJeDgi05z2QjXw6JmKhlFP6O7leRGagfK8u5DofMKrzvFFFDujijonc6jV9KYFFod1X45io521TMRI691lugFstMRv5FSZhJ6CGb
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(186003)(6862004)(81166007)(6486002)(70586007)(47076005)(26005)(2906002)(33656002)(6512007)(107886003)(86362001)(36756003)(82310400003)(70206006)(2616005)(53546011)(356005)(8676002)(508600001)(316002)(336012)(6506007)(36860700001)(4326008)(54906003)(8936002)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:08:32.9213
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e04ed871-b62d-47d4-6601-08d97ea40270
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7332

Hi Stefano,

> On 23 Sep 2021, at 3:11 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>=20
>> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
>> space to the XEN memory.
>>=20
>> Patch helps to understand how the generic infrastructure for PCI
>> host-bridge discovery will be used for future references.
>>=20
>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com=
>
>> ---
>> Change on v2:
>> - Add more info in commit msg
>> ---
>> xen/arch/arm/pci/Makefile          |  1 +
>> xen/arch/arm/pci/pci-host-zynqmp.c | 58 ++++++++++++++++++++++++++++++
>> 2 files changed, 59 insertions(+)
>> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>>=20
>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
>> index e86f2b46fd..0572c68bba 100644
>> --- a/xen/arch/arm/pci/Makefile
>> +++ b/xen/arch/arm/pci/Makefile
>> @@ -4,3 +4,4 @@ obj-y +=3D pci.o
>> obj-y +=3D pci-host-generic.o
>> obj-y +=3D pci-host-common.o
>> obj-y +=3D ecam.o
>> +obj-y +=3D pci-host-zynqmp.o
>> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-h=
ost-zynqmp.c
>> new file mode 100644
>> index 0000000000..c27b4ea9f0
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
>> @@ -0,0 +1,58 @@
>> +/*
>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>> + * Based on xen/arch/arm/pci/pci-host-generic.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>=
.
>> + */
>> +
>> +#include <asm/device.h>
>> +#include <xen/pci.h>
>> +#include <asm/pci.h>
>> +
>> +int nwl_cfg_reg_index(struct dt_device_node *np)
>=20
> static
>=20

Ack.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:11:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:11:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194405.346312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQNG-0001hP-JJ; Thu, 23 Sep 2021 15:10:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194405.346312; Thu, 23 Sep 2021 15:10:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQNG-0001hI-G2; Thu, 23 Sep 2021 15:10:58 +0000
Received: by outflank-mailman (input) for mailman id 194405;
 Thu, 23 Sep 2021 15:10:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTQNF-0001hC-GH
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:10:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 72dc77d6-1c80-11ec-ba51-12813bfff9fa;
 Thu, 23 Sep 2021 15:10:55 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-rf9lCpNaMdeAnba_5yJhfQ-1; Thu, 23 Sep 2021 17:10:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6861.eurprd04.prod.outlook.com (2603:10a6:803:13c::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:10:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 15:10:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0163.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:10:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72dc77d6-1c80-11ec-ba51-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632409854;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=fceFkzy43CdmrEzXYcvTDoaKGsGNbxOgjHaMvMntd/k=;
	b=gly72MkKwo4XTDtWQ1mIrXRP+yaW0IpO0hzenaHoahdh7k9rIIyYu+Ruk7LBRzb4qECK5I
	fte/e386kR+aWhfsM3aTlJhDmDl4DQ+WhLHE50VaC91XkKHTRKR+bvpB3KXhT/07T9E2eN
	2uncuhr5AxtfROnKhquLh+HFKRz0/pc=
X-MC-Unique: rf9lCpNaMdeAnba_5yJhfQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WyfGoWL58uEeTV8LDE35QQjMQMBfzCx/QM774QvOzFq1609xi7qeTOKMVHUdZUUeF3F6WAWStC0aP2WPVBbt0RU7ryDuqny+a6KRunW5jb9GBEyBr7yEYC9jUeTJRrxdPJZtKSUtL9E3DPOT9EKuNR1SrkZTctL9cqIgRP+2JfTu4Ott6MK/oP1mWHmU5XPxmTgfIaf5CDN29B+9uNzZ75FFS+q67TOWg+jt9+XtmsfPh9pFAyl0AaK5n8y+3Utog6RirUWu6DGPqAsjBFACenS+teeIytAfCSc+Cclsyh9s9JhbjgoAqtCtJ93d5QFdGWIUAcF1eQmfUd7XqkQHwA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fceFkzy43CdmrEzXYcvTDoaKGsGNbxOgjHaMvMntd/k=;
 b=UlyUTdtkqXTQbk0l1ucvXLnF555uRfEFyN0mY+Q4pLOxp7JgutZ03Y7eNjbOFX/KfqAr2K0/ysl/qjUsQGmeDKuw4N/bFIyjSj2Vyg0l9VYnzbZl16rlZf0/PHDVwZOvr89Tuf+6WybmNT146JlcTvIWBCeHk6eSsfgiE6whYIlhYCb7uxVzyZbt+Tsc+6we6LI7zJak7aRg3xV8X6jtklDJILB8ALwUSZ9/oaVPFYujelV/xeyCAINZWpFlTVUa8ncNhGmYiRap4TjkGb56rOV/oPGla0XfKMrkuaQvb47PBUEd7IBSCXUJHpEXxIDmNFUvs68szh7uCFCbORihtw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
Date: Thu, 23 Sep 2021 17:10:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0163.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::31) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e0656f5f-4186-455c-0fdc-08d97ea4557a
X-MS-TrafficTypeDiagnostic: VI1PR04MB6861:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB686182E1C4C51339E27EC861B3A39@VI1PR04MB6861.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7jxfke8Z/LhqiGqxd9aTkpxMxV9FCKTzX5TPhDrRSwTiki42woWJw7By+bKmhbJ2AuD0XVN4p5ytIGFDL3MTAH7a2cpFNWK+rJuHReD4IHqcjK5zXuttTfGDLMDC82X9fHXdx4c7gkMv2z13qAdmf6cptvVRb6vI6mFUYJFsQwOMkbxU5zaQg5VdfFgOElsj1FVMbUoepCbHfgG64s4d56abj7G4vcdW2yfpuyWOPoLRpO7nW6dw4heZrBVTXhOVhAxoPc7IsM4EXje/YPybBTj3E9TbN3lnYUtLeTPq1bY/He9Bjs5FttFEDD9+zAv/bB0SvqbPI2DRg80nKRB7KSdiNhkLyVg6Y4+tUBPVe0LOUZyuN1G8U9qS7eEQm55ENSIrfitHChgpc7trdrVgCLlpbAPSgzk4lUlgBi8ETFHcs2MrafMeWg4ANC4rQZeZG1XqI3HZx++UqZZgoWFSb2YQFPAhGHVSEVCG/GN1yRaEdyQUWMui+wzEXLQ6KGdWVUSAjs7ekTBbvranXA5Q1d+hALCa859/7lP5Uq4Yeb88z+2H6nI0i8uR3tkRaKNLGEjwKyTDFuGUxs4THKkz7v2U7chzyacYeadB6tS2P/TyLapnETTCzgHYLKl0OmSesKHl9pAj76luzUjjKIDNcuY6Cs0eVWADre1lVAh1RaQmNN2IPezeDj8fp+mrse57rcYgyEQ99NKsOsT6xWPCP+o/vYecLDEcNZ2ByU+qUZkJSXKzttybZhBvIPYgw9MfXWRTIYsmb0uPDN65xtI5kzhXOg157s4QEPhVxUJI2oU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(26005)(6486002)(86362001)(36756003)(8676002)(38100700002)(83380400001)(5660300002)(2906002)(54906003)(37006003)(956004)(53546011)(4326008)(2616005)(508600001)(186003)(6636002)(16576012)(316002)(6862004)(8936002)(66946007)(66556008)(31686004)(66476007)(52103002)(158003001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?N0pzcnpJTGFiQ0x1WnJLNXZvWUI4d09pME1YZUdya3JoeVpoTlFaOFdhbTFH?=
 =?utf-8?B?K3VRQnBIcG9ySk5nZ0ZIR1l4YjUxeldESE1KL3owR2xhM0YyNEx1NWgwNkxz?=
 =?utf-8?B?UExNelg3TnNkMDdTSDBiWCtpNExBMjA5TkFZOG0zQkUrbFpGVTlJWVVseDVS?=
 =?utf-8?B?SEhXOVBiUnphdEZmWE1RVDR6Tk50VkZ3YnBtdXZ5R1Awa1cvQTZtb09tbzlu?=
 =?utf-8?B?SC9XYlVzZEI3RzNmYmh5Y2cwWnJmMkJZek9uUEFtelNraEV0dlFqRlpQRGZp?=
 =?utf-8?B?a3BVcUdxSnczQk5FZjRNVkhjeFRHdXpJMjlhTFF0Q2ZYVUdiNnEvSm1PWWk5?=
 =?utf-8?B?MDFwMmFMSytCR2V1YVBCODF4SEs1bm92eE4waW9HOG1RNHlzaUdheklubmhn?=
 =?utf-8?B?YmEzU1NHaVl4czh1VmMwbHg2NVdXMHpsZ0N2NFErT2g4eXBQQXdhazh5SEJC?=
 =?utf-8?B?bkFmNkxmYWZwSSs2dmp5THJWdU9xSzFnZjdvQzZmZjBETm1XeUhaRTIzV3Ey?=
 =?utf-8?B?SW9VVCtjbmdNdDgrNkl2a2hrUE95MzRqblRvK1NpV3BYc2dQVldZZFVVUTY2?=
 =?utf-8?B?UVV3WG8wUmpIWmFUMXNOKzhGVlUwL0xSeUp2bjhvaUhoUWx5UVJNdnZVc2tP?=
 =?utf-8?B?dlJUQW9WZG95ck1MT0lUaTV2MEZ4ZE9CVXVTeEJ4NkNpeE41b3ljWTBqa0VP?=
 =?utf-8?B?UW1xQWU4YWlTbjJpaEFsMkw3WGJTL0xOc0lCc2VxN1EvVmwvOVdJdXJsQk5m?=
 =?utf-8?B?YkwyNlJNUUtQU3hFODZVL0RTNG4vd2IraEZmWlhmaldSbFYyb2RqNnNsWDBP?=
 =?utf-8?B?QXhiZkFsWW4xMitXVkRFT3hzT2JGS3FDanJMZFp2Rkd1VmQwWG5LNmExNUxt?=
 =?utf-8?B?U25UdzRmK3Z1U3IzakgyYUxJZnRhUVFIeW5nczAyUWNsNDFpK3k2TW1tUUpX?=
 =?utf-8?B?OUNmQmI5dE9uTHhzZEdPeEFxcEFCNE9EU2FTUU1SallhYTRDWjY4aDdCRExa?=
 =?utf-8?B?dlIvc0p1NStpQ0hFQ1E3QWlaVEdCM3ljVUVETU5LTXU4YS9GRFRqM1E3K1lN?=
 =?utf-8?B?WFZkN1Nld2p6RTJMOFd2em5PMEVxMUtsMjlBQnlIbSsvb1U0UUsva2dIWDY5?=
 =?utf-8?B?Z0ZpRWZ6TG1yMzl1WmlvQ2FpY1RuZGZ4NklId1o4Vm5aRlFLa0dRZ2xncGlk?=
 =?utf-8?B?cDF6ampRQitmZ2tkZXdIVWUwemNjZGdvUFU1bmZHMUxSTCs3WHFIZFlBMlVx?=
 =?utf-8?B?dXNHREc1WVV5NFhmeDJTdE9Mbm9rcjBHQ0luYXh5QzFiUU9XV3FWbUFSZFVk?=
 =?utf-8?B?R0dnRzgyVzRNUTVlMDlFVU15NzZpKzk5Zy9aMzQ5VzJSSlQ3Nlk3VXFvaFZY?=
 =?utf-8?B?QWZYeS9Uc09CMmVYS0FpdnFBcVBFWDFJVlVZamtFbmtPWjdjbm9MRnlsL0pN?=
 =?utf-8?B?L0tweUdjRVZ2STkyVUdMY0hPUnliMjRRVFlQVFdXNnFVbWsyUjJLV0Jnb1pW?=
 =?utf-8?B?OUtiQ056L1JkODU5SXhaVG9Ccit2bmlRdGhuaWhTa3JvcTdTOHVGN1pLN3lM?=
 =?utf-8?B?ZzdJQ0lxOWYwUG1PdkpqQTNyandlZ0ZqQkUyN0doRFBvVUo0K01PcjM5aGlF?=
 =?utf-8?B?L0dqWERoTm45NVdheTAyT3ZhcysrWkFsQjFNVlgrU0NxcGtmaWlIY085TjBn?=
 =?utf-8?B?SDNiMXNZbVEzcE1SWkNabTYyWGo1L0t3clFINlpHZnNVR2ZwclM0YUhhQjhT?=
 =?utf-8?Q?tyQB0BhupmEvnOqf8pFnULEXXYUvOmMUt0kCJA5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e0656f5f-4186-455c-0fdc-08d97ea4557a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:10:52.4514
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NfPjBuDJUizHOroqFRAfdWxtmCoHN92MVcGThMqYOrL1aC1Lyk2wTNkuUWU0W8oSFq9I8t2GcjwFXmDeat70tQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6861

On 23.09.2021 16:59, Juergen Gross wrote:
> On 07.09.21 12:11, Jan Beulich wrote:
>> This was effectively lost while dropping PVHv1 code. Move the function
>> and arrange for it to be called the same way as done in PV mode. Clearly
>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
>> check that was recently removed, as that's a PV-only feature.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>
>> --- a/arch/x86/xen/enlighten.c
>> +++ b/arch/x86/xen/enlighten.c
>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>   	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
>>   }
>>   
>> +void __init xen_banner(void)
>> +{
>> +	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
>> +	struct xen_extraversion extra;
> 
> Please add a blank line here.

Oops.

>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>> +
>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
> 
> Is this correct? I don't think the kernel needs to be paravirtualized
> with PVH (at least not to the same extend as for PV).

What else do you suggest the message to say? Simply drop
"paravirtualized"? To some extent it is applicable imo, further
qualified by pv_info.name. And that's how it apparently was with
PVHv1.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:15:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:15:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194410.346323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRh-0002ON-67; Thu, 23 Sep 2021 15:15:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194410.346323; Thu, 23 Sep 2021 15:15:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRh-0002OG-2I; Thu, 23 Sep 2021 15:15:33 +0000
Received: by outflank-mailman (input) for mailman id 194410;
 Thu, 23 Sep 2021 15:15:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTQRf-0002OA-Te
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:15:31 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 16f8ed5e-1c81-11ec-ba51-12813bfff9fa;
 Thu, 23 Sep 2021 15:15:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16f8ed5e-1c81-11ec-ba51-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632410130;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=B6zG86aQYgEV8mV02SHeuMwLHz//ADy1sKLf6lK799Y=;
  b=Xy+Gt0ajzjAL6U0p7WFEpf9IULo8aVN6RjkzDOis6BwH1tElC5hKG2nQ
   ysMKIQtF+pFjEregTT78VWuDoJ/ftiukLjQpitvuj6wwsKj992I0IPT5d
   OGMjlocgC3N2H2ZhVzb0UVcmDGFr38jugqygGZHFkUb+Ud44q4lpr5bQq
   k=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 6RLjGJjVJNrfGEEGs76qjVkrq5UFAU5TMeW4wNVCgr2rP1BKeuHQhqyg6le8WNf9WWuyKjFhG7
 Fr0ooO9rmLwdbOWFI9YhIF3vRUdG1kDkZ7A1Q0leE3+NXTq/7j8UMkndhmVC72JwHf02xQYJ2B
 p+v96Zhx+ibyimhi2KxjspsLWFXGW6gbrL3Vp/rZ11KOyGBVfHrFf3HtwSB8jgNLVzG3vovLAT
 nBHWxzXMWA9w/2CiKsj5bYMOGTVvkk7GrEp4h7Iq5+IXo+MqaYoFWJdCmXWltD4wMeyRFM9TlW
 IiJ3x5tHE2nDFtQ305tjnc4c
X-SBRS: 5.1
X-MesageID: 55266241
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:y0hDb60ohCPiHj98WfbD5Q12kn2cJEfYwER7XKvMYLTBsI5bpzVSx
 zQYXziDOv2JM2v0Loh/a4rn9E9X7JWDxtRrTQBupC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbVh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhu+gtl
 dB1sqGJaSR4J6bHl7VMQx5qHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIICgmlq2pkm8fD2e
 ekgSmRKUUX8eUdiOAs1IcoBrcmumSyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1Fk0Ddq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PL+y++NugVaT7ncOExBQXly+ycRVkWbnBYgZc
 RZNvHNz8+5iryRHU+URQTWcmCLZuyYbZuNwMLwj4wqCm/vE5DSWUz1soiF6VPQqs8o/RDoP3
 1CPns/0CTEHjIB5WU5x5Z/P8mjvY3Z9wXsqIHZeFFpYv4CLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3mrESnQkSGcICuTrsoCNslgRiG2NPdXABb3nARBodt3xor6p5
 iRspiRmxLpSZaxhbQTUKAn3IJmn5uyeLBrXikN1Ep8q+lyFoiD4J9sLuGkleh04Y67onAMFh
 meJ4mu9A7cJYBOXgVJfOdrtW6zGM4C6fTgaahwkRoUXOcUgHON21CpveVSRzwjQfLsEy8kC1
 WOgWZ/0Vx4yUP0/pBLvHrt1+eJ7l0gWmDKILbimnkvP7FZrTCPMIVvzGADVNb5RAWLtiFi9z
 uuzwOPQkEgADL2iPHOHmWPRRHhTRUUG6VnNg5U/XsaIIxZ8GXFnDPnUwLg7fJdikbgTneDNl
 kxRkGcEoLYmrXGYewiMdF55b7bjAcR2oX4hZHR+Nle0wXkzJ42o6f5HJZcweLAm8s1lzOJ1E
 KZZK5nRXKwXR2SV4SkZYLn8sJdmKEahizWRMnf3ezM4ZZNhGVDEo4e2Ygv1+SASJSOrrs9i8
 aa43wbWTMNbFQRvBcrbcty1yFa1sSRPke5+RRKQcNJSZF/t4M5hLCmo1q07JMQFKBPiwDqG1
 lnJXUdE9LeV+4JsqYvHn6GJqYutAtBSJEsCEjmJ96uyOAnb4nGnnd1KXtGXcG2PT2jz4qijO
 7lYlqmuLP0dkV9WmINgCLI3n7km7t7iqrIGnARpGHLHMwaiBr96eyTU2MBOsutGx6NDuBvwU
 UWKo4EINbKMMcLjMVgQOAt6MbjTiaBKwmHfvaYvPUH3xC5r577WA0xdMi6FhDFZMLYoYpgux
 v0suZJO5gGy4vbw3g1qUsyAG7ywE0E9
IronPort-HdrOrdr: A9a23:myKTUahOgZdaV00RANq6ixx7ZXBQX0t13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3I+ergBEGBKUmskqKdxbNhR4tKOzOWxVdATbsSlrcKpgePJ8SQzJ8+6U
 4NSdkaNDS0NykHsS+Y2njILz9D+qj/zEnAv463pB0MPGIaG52IrT0JcjpzencGOjWubqBJcq
 Z0iPA3wwZJLh8sH7uG7zQ+LqX+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+qemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lsdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNzN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wqJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABKnhkjizytSKeGXLzEO9k/seDlHhiXV6UkZoJlB9Tpa+CRF9U1ws67USPF/lq
 352+pT5fdzpmJ/V9MIOA47e7rENoX6e2O7DIujGyWVKEg5AQO5l3fW2sR/2Aj4Qu1D8HMN8K
 6xJ2+w81RCIn7TNQ==
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="55266241"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ahzHFFMslkj85t5MqmPOEW/gVTBFK5IBdQKik0RB0is2TmhNvRYURppFQG17g01AUFwov9bL8H0ipjSps6LeJw8vqBX60j9tEFi8trKTAswvJm9/VpLS0/tDSMvUaAQBstXzPTT/sRPNXaHmcct3vgPmZp7hEpMVq5koN5fWM/arkIqC21t8XmYecoR37/c5bb8Jafkm+l7Zw6fDUaKyKnSQCSe4Tm5kNUt20jYZlgSv8aL0fZmYZMU4goF5s0kJnzBJcylmHO4W1Sd8zgtOXW5OxQrB3Jgnr8lD6JagG7JfzgE9hN8BlciG8IHjIKZw9oQxg+cDd5g8rFYIyUB8Nw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QhwNl3BWRz36UYDGvQyycjsgS94kBKFE4yuwg4hlGqI=;
 b=bpc03ETd1SJR1ElPvOskjQxvmNOgfgM0WgU4Q2OSHXarkS0FDQuZtfpelLghjRnqOb+g691/DNAit1n0KMj6KxyvO0fmjzR9TWVU+dznKQecbbgC0NhzyUI1ZcN1pfj+mgHwdiLDE2uLgSKXwUZsUwkBRH94fAo+nJkXrNRNWDR5JGvNXz2vp+RR5usB2IPmIdEhR4QeVA/JqP2CRf59aYgrPUwX9Mjoj39TN8+5Jp62Dtx+1ox5drWgLP4h1dpv3WKY+Ti3nknSgT4HWbU0XbiM9CaqZrju4ghHTxm8OOfhTI5/GI8nshCDxVR/9Nthjd8dcjcwsTUIWQ79hhEc8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QhwNl3BWRz36UYDGvQyycjsgS94kBKFE4yuwg4hlGqI=;
 b=gIFo2s7gHIltb0hi6L65RtPH77ImSAoLPyvoeuOEeJ25LJduBVdT7iQsabWVJCgEzyVdrJueLCtmAXJNlRRTWiyI7OKo2nxUAnYpBxWsXLjtCPCobZncYvPmVP5nxQyOs+4G/B32yVXhtdhmzfNvyVLUCSsFHzu8gc6387rssVA=
Date: Thu, 23 Sep 2021 17:15:22 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
Message-ID: <YUyaCrdAdpnn/o0K@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
 <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
 <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
 <33abd731-7c2e-8c9b-963a-fd446a21e87a@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <33abd731-7c2e-8c9b-963a-fd446a21e87a@suse.com>
X-ClientProxiedBy: LO2P265CA0414.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b54f447-1079-4fd7-c945-08d97ea4f9a8
X-MS-TrafficTypeDiagnostic: DM6PR03MB4474:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4474426E57B345067698EED88FA39@DM6PR03MB4474.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: K3gf4MkGRdq4js9WvXkZPKz81U5lTIcpHH7ZNKcGvfiTTMVqR2Jdhr/AtCXtKQra/61UBu0fQc0HNfEq9wnaYur1bnj0GHTQoQbBnhuPY93HKctFWrEK2IsbxFih+eu0whkMt8VYwOfsxmweqfn9RJ1fNAQt9mrecddvQ9zQYRIk1kdUecVlRsLVKZJI0BlOoJMfYu1aFQaNuMwc9l+4JAbMRvY4djmUmJ8xFOy1A3vmFVmiZmEDWkd5vgSZKd+eE1ivAd3OGDHHlxQWpJ7NJFMpkw9H+2XOItnpztvJJ/mgr/B57EkZKvU3Ao7Roakm0jUuqoTfjSi9h3GCeSA+znFY4B9gVGOd/AUF5YXXAKFtYOn8LsFeXN0lBRsyWfjhk8vu4cwMxMLMrGLUDvKX8VwHnmwu2URohgcM/BldjNAU6xNu8aYdaCMcj634RYXQmXctqLJUJ7Xdv1DMQI33DIoeqd3ufxs5fUDcau1fETDQpE2c6uT/luS5hd7unXYNSGyBODfU1v/MxcNzDxbDPHje1LnjOrJQwbAYLMeqNjGZ42lYkCQYF0imkNK1mvI3lbcSCnYY2ri3TH/KCDfhqSH3SeNzptVtNiP75QwhckdFnFiT7MYfH0JW9jEdX6EpYM45ThK5LpBTzn3SsMzKiw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(6666004)(66946007)(53546011)(5660300002)(107886003)(6496006)(66556008)(6486002)(186003)(26005)(85182001)(316002)(8936002)(4326008)(66476007)(956004)(9686003)(54906003)(508600001)(2906002)(38100700002)(86362001)(83380400001)(6916009);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NFVuWGRzSmJyV2MwOVdZZ1EwMHh3SUtrOWh1TlVXNUJGSTRRbFZ0YmZBa3ZS?=
 =?utf-8?B?YUJQWFYvRjB1dWNOVkc1NXYwWGpHVTZOZnBnd1M0amg1OVdseUdaVzZIUHN6?=
 =?utf-8?B?TUEvdUQwellHbTIvaUNWU1pldURBaDhMUm40WU05TzBCeXdDUUYyL0Y5L05G?=
 =?utf-8?B?Umo4cDlpNUFPN1Myb3pZblNCbGo5cFlUZGdVd080RlhOaTRiRTVPUVlteXdS?=
 =?utf-8?B?ckxSYzJKN0N5di94bWk4TjIzWkNhSUNaTU9kbHN2UXF1dDVKc0lrSVVlQjhn?=
 =?utf-8?B?TGkzVnZvSHIyaS9Gd243RzltdzNrQU5wWTlWRk85QnB4RXM3TndMUTM3ZTZq?=
 =?utf-8?B?bGJVM0J4U2kxd2tvSUp2M2poWFN0cFBTMWlJWkJLN3VGUkl5b0hpWTE3SnBm?=
 =?utf-8?B?Wno5S0lzWHQ4TzNVSlJKdkI1T3BPbzEzdnNHamNianBWSFI2QUM0NnVOVGlj?=
 =?utf-8?B?Z3dpMDRVYnFPVjgwZjNyN2tyMGVKRkNtZThZOUhUa0pMZ3kvTHlLRm5TeFRn?=
 =?utf-8?B?RTVOYTB6RTBXalZQRC9XeFR1dmpWWXRaMVhtVjBNTEY1M043UzAySFVVMHFS?=
 =?utf-8?B?azZxcUFuOHc0ODk2eExhZW5WYXVib2FySW8ybUdDM0ptYmRVLzFHWmNNblNw?=
 =?utf-8?B?ZkRoQ1V6cDg2RUtzMUpUTHgxSUY1UmxXUTJHU3RDQVZpNVBTNlFzVC9jNlBV?=
 =?utf-8?B?V1g3MjkwakFieDY1bG1rTTFvc3hydkZkMFEzTUhVN3FSNEZUZHNRYjRzeVFC?=
 =?utf-8?B?ODdYN0lJaUdGTnh1R3hjSzFvS3dzaDcrb1g5SmcxMUp6Ujh4MjBpSVBGRXcw?=
 =?utf-8?B?VkNFVGt1MkI5bWl5b1ZzOEhnaVcrRDdtdXB5TkU5U1N5ekNITkQ5RnVibzIz?=
 =?utf-8?B?MXBtcDVjMHVLNGFubWZwVXRGRksvRVdQaHF3T2FDN2tjMjBTeGVmcDAxVzJK?=
 =?utf-8?B?cWJGeVlrSUR2NSt3MmdiU2J5dVhKZHcrS083V0tTT3FPZy9hWVM1VVU4cXVS?=
 =?utf-8?B?Vys1bFhnMnZEMW5jYngxMndqRnl3dTNnWUFRTk1VZWFPZU14UDRXbmF6MjE3?=
 =?utf-8?B?YUJqQTRFVklJekF4T202V0pMbDJSd2JmdzUwQlIrWGk5cWhnTlZHbkYybTh2?=
 =?utf-8?B?MnVxbzhDREMwRUJMbFhPNGRXcHJGTHFuVDlDNE5LYVZkd2Z0MlVMWERFWSto?=
 =?utf-8?B?ODFPbHFBcHRhcElwakFvY1FuN3QwSHpmTFNzMzFEblAwV2gxdHp3NElvVDhI?=
 =?utf-8?B?UkQ2UVRDem4vMFJhSjk1T3FSWWNDWGlrYlZUN1RGUEFRcHh2RjFoVlYrc3RX?=
 =?utf-8?B?ZVZFSlJTTkRSS0MvZWN5N283WForZXlENzV2L2NQaXAzTEY2S0V6RmdOS3BE?=
 =?utf-8?B?RHFaUEJwQ2greDQ0ZEZPUUxQV2J2bFNQZlNKQ085cE1WQVV5RU5WdDZOM1Nq?=
 =?utf-8?B?UExBSjlqRnE4UkZXOHlHcU12dGJHNGtmV3NqMi9GREZBTEgxVnVWM2tzRTNE?=
 =?utf-8?B?a0R5SmRmbW03bFptVWpTRk5TSXNxVnQzdlpuellBdDcrL2xFL3hQZ2MyS3Mw?=
 =?utf-8?B?dU9mOGVtLzB6aWlORllpQmdUZG9qcGticzRmdnd0VWtHUzdFY21UNXN0NVZv?=
 =?utf-8?B?bzRrMlFrTFJHdVBySEwyQU9LbTZVUi95SHpjSXJIcFpwQk83dVBxbG1WQTNS?=
 =?utf-8?B?WlBhTmpRRDNLTVZlZ0o1Y3d5bmw5cHhzbG1XQThLbGFmTVZNYlpmZEtRbEht?=
 =?utf-8?Q?59tw3KkBVtTgDS00uow/RoFoiHx2d302PORGzBf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b54f447-1079-4fd7-c945-08d97ea4f9a8
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:15:28.0369
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hC4QJHzaZEW5Z7T354lh9rR/xg1vhCmK24cOAVozUTahRynWm6YsHP3Ep7TEDeaj34IqSqsSHrYKUHcTwY16dw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4474
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 02:15:25PM +0200, Jan Beulich wrote:
> On 23.09.2021 13:54, Roger Pau Monné wrote:
> > On Thu, Sep 23, 2021 at 01:32:52PM +0200, Jan Beulich wrote:
> >> On 23.09.2021 13:10, Roger Pau Monné wrote:
> >>> On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
> >>>> --- a/xen/arch/x86/mm/p2m.c
> >>>> +++ b/xen/arch/x86/mm/p2m.c
> >>>> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
> >>>>              return -EPERM;
> >>>>          }
> >>>>  
> >>>> +        /*
> >>>> +         * Gross bodge, to go away again rather sooner than later:
> >>>> +         *
> >>>> +         * For MMIO allow access permissions to accumulate, but only for Dom0.
> >>>> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
> >>>> +         * the way they specify "access", this will allow the ultimate result
> >>>> +         * to be independent of the sequence of operations.
> >>>
> >>> Wouldn't it be better to 'fix' those operations so that they can work
> >>> together?
> >>
> >> Yes, but then we should do this properly by removing all abuse of
> >> p2m_access_t.
> > 
> > I'm not sure I follow what that abuse is.
> 
> As was clarified, p2m_access_t should be solely used by e.g.
> introspection agents, who are then the entity responsible for
> resolving the resulting faults. Any other uses (to e.g. merely
> restrict permissions for other reasons) are really abuses.

But some p2m types don't really have a fixed access tied to them, for
example MMIO regions just inherit whatever is the default access for
the domain, which makes all this look slightly weird. If the access
should solely be used by introspection, then each type should have a
fixed access tied to it?

> That
> is, if e.g. a r/o direct-MMIO mapping is needed, this should not
> be expressed as (p2m_mmio_direct, p2m_access_r) tuple, but would
> require a distinct p2m_mmio_direct_ro type.

I guess we would then require a p2m_mmio_direct_ro,
p2m_mmio_direct_rwx and a p2m_mmio_direct_n at least, and ideally we
would need to differentiate the mandatory regions as present in ACPI
tables using yet different types.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:15:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194411.346334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRl-0002g0-Hx; Thu, 23 Sep 2021 15:15:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194411.346334; Thu, 23 Sep 2021 15:15:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRl-0002fr-Eu; Thu, 23 Sep 2021 15:15:37 +0000
Received: by outflank-mailman (input) for mailman id 194411;
 Thu, 23 Sep 2021 15:15:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQRk-0002OA-S8
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:15:37 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.86]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1929f9e2-1c81-11ec-ba51-12813bfff9fa;
 Thu, 23 Sep 2021 15:15:34 +0000 (UTC)
Received: from DB7PR03CA0077.eurprd03.prod.outlook.com (2603:10a6:10:72::18)
 by AM5PR0801MB1732.eurprd08.prod.outlook.com (2603:10a6:203:35::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Thu, 23 Sep
 2021 15:15:31 +0000
Received: from DB5EUR03FT033.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:72:cafe::e6) by DB7PR03CA0077.outlook.office365.com
 (2603:10a6:10:72::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 15:15:31 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT033.mail.protection.outlook.com (10.152.20.76) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:15:31 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 15:15:30 +0000
Received: from d15928c17eef.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 83D1DC5D-B044-4AE6-98CC-7CA82DE851C6.1; 
 Thu, 23 Sep 2021 15:15:18 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d15928c17eef.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:15:18 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4933.eurprd08.prod.outlook.com (2603:10a6:20b:e9::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:15:17 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:15:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1929f9e2-1c81-11ec-ba51-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bu6yH2OU1L8pGt0OBEYLrX9c4kaLijVOoTSHnKy0jo0=;
 b=rMmyeTC52PALgvE75fyHk0fA7ICOD+FgYNWJeGuUcWSsEVDmYbik3mn7RJ7qhDr615Mdi/W3K6LdCiIQoeQmVcQWAO4MN004m5fqXYgNUxtdpsB2ZRV+Zm1NJTxUyk9YYXsf46wmB4apdCrdOyY2FlvGw/kMbPTml02aJqS11hg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 98f3d76b6eb4bfd1
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UAEHBz2Bq0xGcIAfj9UlQYBAwSOdhoClVC9FSOorz+BE9+Gwp1lTq2NQT0F0lvmK0dB4p8xkWuO/BKGRxYRmejXz+WqT01QL9EmCx0gWoUjPuu1xWp7whtU09oT6veMOAoiE9KDguZDZGeJ1jJaxb6K91bIvZnfVgljHC3FLet/+JZfcOHi/tuB+wbwz8Gn0xtMNoaH9ydC4Zon6FFFYJaFVdAaUkfVr75tlH4LSUhoGvvXiN7N5+2PifxS/QmkIsGBSwimfNlGR7DCCQZ6BG/XxVsGvReFjoK+K9erR3UFJsQyQ9BPP06lQZqQvyAAgt4Gp8PaK9npj8VDwvMH5BA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Bu6yH2OU1L8pGt0OBEYLrX9c4kaLijVOoTSHnKy0jo0=;
 b=QX6J4XLyW/ShRYF1YNCyz21DKzSlKpQUZ3Q4b0u37dPrpTAs70qztLXjqcqHdFHcxZ5cQ9H5cxBwx7FHehNB0erk6PCtVcVULAEAdqY+oZTBOIbFsrD4dJWe3uFJJh5Hbe9eZ2eaOcDZgEyWaCcMD4n0tvGUhxAmAddJnGjhn7xHUFAT7JxW5UkOUBDd8e6T+JRODqgLRdVkdhZjrpr65NZ2oVhKTib3HLaFq6/tl9RBPqft+F/p9QjFOKnlF7HAWOo6zl6HG5/mQQZdZKmdGlDmZOvp0Je0r7cKeV757DKCOESQfaVjdUwlrSLl8nh1pSFfnbg2SXyLdj2HBxJMNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Bu6yH2OU1L8pGt0OBEYLrX9c4kaLijVOoTSHnKy0jo0=;
 b=rMmyeTC52PALgvE75fyHk0fA7ICOD+FgYNWJeGuUcWSsEVDmYbik3mn7RJ7qhDr615Mdi/W3K6LdCiIQoeQmVcQWAO4MN004m5fqXYgNUxtdpsB2ZRV+Zm1NJTxUyk9YYXsf46wmB4apdCrdOyY2FlvGw/kMbPTml02aJqS11hg=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Index: AQHXr6aktO5EpOLLjku5HahQRSHFEauw5LyAgADXlwA=
Date: Thu, 23 Sep 2021 15:15:17 +0000
Message-ID: <96F69005-D39F-4F6B-A684-7A4BCA275860@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 24b323bd-eaa3-4265-9663-08d97ea4fbd5
x-ms-traffictypediagnostic: AM6PR08MB4933:|AM5PR0801MB1732:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM5PR0801MB173227F653DD978CD23C83FEFCA39@AM5PR0801MB1732.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HKMEx/iL+arVCwdNvGwpQ07lsSfxTehw6yOdPOz1/srx8t8cazWios7yRQFgu4lqZHxifFdI7GP7UKvU8jZ2vibGu0n9NbQc5lX74SEkhOihntf7C7SdOMqJeTgIi0Nt+E3BiXNW8oo39S2Z1GIYs0Bx6IZDc76bSB3n8+5lKJj02Oc0k4UnoplheJOS/W/sJYFN5p2jYzUWqOC/73zKBJmy+P9ldETfV2xOMl4GLu63iE1AJ3tVqnnkvqdDJO5samBAuQFRFj5NaSPHm4gYoxdpPif2hoY9B+MjbOESl1rmTGZymiSSqE7l4u5UzxlU2eCqBCPHzNULMPkr3SgI7BXYq6RTAX0Of+9XXLg4Gj1O1X7SlT1wTGuXvCAzerR2ooB+5dkHoF6YlOp3Y+WPy9VYmoCSFgfPpWjsLjlL7lCxJDZBEfNfEItGrHVl1SZwWvqIeG49EI6KnY2phDkupXyJXsdSJDym8WNEU8ow+LcesprlA4GBf8rYLSCP2ZMokt+cal7X1rSptaiEBWI7S2jCGjz1mmB9AT+xjCUjwhEg2NoFIUJBJSziPNmgl9qVbMKjTBRwlhMSXbXLlUrluYN2t8/jw+Jlre4f1SFo01LEZ6XJeELwpI4HuGhPOt8Vlc2/mInrGOLNXmyxgdlyVIL8qFfg2syVSAekOENbaCTnKrS0rULKug6eBdPom3EmksYqHptRHh4IB92NRXiQ8jc8iEJmmVS7bF5AlXkQoY4s8dmkpH6X5NToBrLqRLo8
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(4326008)(53546011)(26005)(36756003)(86362001)(71200400001)(33656002)(6506007)(38070700005)(6916009)(54906003)(316002)(186003)(66476007)(508600001)(66556008)(83380400001)(38100700002)(2616005)(64756008)(66946007)(6486002)(8676002)(6512007)(66446008)(122000001)(5660300002)(91956017)(8936002)(76116006)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <28394010AEEA9843B2A93847D8DAB758@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4933
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT033.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	de500561-e58b-4fb7-9e25-08d97ea4f35d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	E/Oj6aUHNEo1cksKlpd5IjKlPg6a+mmgXk9LKZwKc8JdaXLMLAojlTnSUVPKLGCCitXhucTo/5uAUWdcuEyLN9gQ97lPs4Gdvhs75DXAXMGb+FPIMimOPjDjNLZYPUjWShPNDjvozV07SSkVywLqkqmLLNnmVfZbXBYH729ztoPO06SM9DSm8koDkz/JeTt8VVpayDz5+yzVvFlfkLVal7muIOV1Vznuy/tYRVD4asaKMNvscGM7Eanf02zVFYPdac73GyH/b2wtlm2U/YQ6soUW/6BeHSEHhRT9Vfjp8C27C7rdn9m+aRImlHdxoGROQYJ8k7K5ZbPW9+gugxYHELSHjJY7gRGr9hCEBNxn2Mh27EW6xhD95+aSqiOQeIFVYRS5zSlPuw6T22Xt3XZtxfsFFDHN8Fjo7DR7v2epMrrwlIEkViifZpJXn4qdC5TxKPtOTw3Gsh5YN+3O5c3Xvqfc81eo3jNpnYdhULMh5v40y99n3F0otND/0SDK63XHlImBh9R2dInVYdooUUakfOXXJCO4y1IwKJ/pUr205+f1vbdS+LO+CIPBKkfvrG4l77mZCLLDXxILz8n4Fx8gvJts/1zmPmWPaAJXxMdYox9R4Cn6B/FWyw0QvNYezIJSrqLWEjMel4IClLmQbjyIghGU9UxrSB1UC3WkLSsKNdcbeghFgZst64MkOYH2617iVJKlZxq+vVBZ8D4LmVZhXA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(81166007)(26005)(86362001)(508600001)(2616005)(83380400001)(82310400003)(70586007)(70206006)(33656002)(47076005)(6486002)(6506007)(36860700001)(6862004)(186003)(356005)(6512007)(5660300002)(8676002)(54906003)(336012)(8936002)(36756003)(2906002)(4326008)(53546011)(316002)(107886003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:15:31.4381
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 24b323bd-eaa3-4265-9663-08d97ea4fbd5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT033.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1732

Hi Stefano,

> On 23 Sep 2021, at 3:23 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> Subject should have xen/arm
>=20
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> Implement generic pci access functions to read/write the configuration
>> space.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2: Fixed comments=20
>> ---
>> xen/arch/arm/pci/pci-access.c      | 58 ++++++++++++++++++++++++++++++
>> xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
>> xen/include/asm-arm/pci.h          |  2 ++
>> 3 files changed, 79 insertions(+)
>>=20
>> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access=
.c
>> index 04fe9fbf92..45500cec2a 100644
>> --- a/xen/arch/arm/pci/pci-access.c
>> +++ b/xen/arch/arm/pci/pci-access.c
>> @@ -16,6 +16,7 @@
>> #include <asm/io.h>
>>=20
>> #define INVALID_VALUE (~0U)
>> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>>=20
>> int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbd=
f,
>>                             uint32_t reg, uint32_t len, uint32_t *value)
>> @@ -72,6 +73,63 @@ int pci_generic_config_write(struct pci_host_bridge *=
bridge, uint32_t sbdf,
>>     return 0;
>> }
>>=20
>> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>> +                                unsigned int len)
>> +{
>> +    uint32_t val =3D PCI_ERR_VALUE(len);
>> +
>=20
> No blank line
Ack.
>=20
>=20
>> +    struct pci_host_bridge *bridge =3D pci_find_host_bridge(sbdf.seg, s=
bdf.bus);
>> +
>> +    if ( unlikely(!bridge) )
>> +        return val;
>> +
>> +    if ( unlikely(!bridge->ops->read) )
>> +        return val;
>> +
>> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);
>=20
> The more I look at these casts the less I like them :-D
>=20
> One idea is to move the definition of pci_sbdf_t somewhere else
> entirely, for instance xen/include/xen/types.h, then we can use
> pci_sbdf_t everywhere
>=20
Ok. Let me try to get rid of this in the next version.
>=20
>> +    return val;
>> +}
>> +
>> +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
>> +                             unsigned int len, uint32_t val)
>> +{
>> +    struct pci_host_bridge *bridge =3D pci_find_host_bridge(sbdf.seg, s=
bdf.bus);
>> +
>> +    if ( unlikely(!bridge) )
>> +        return;
>> +
>> +    if ( unlikely(!bridge->ops->write) )
>> +        return;
>> +
>> +    bridge->ops->write(bridge, (uint32_t) sbdf.sbdf, reg, len, val);
>> +}
>> +
>> +/*
>> + * Wrappers for all PCI configuration access functions.
>> + */
>> +
>> +#define PCI_OP_WRITE(size, type)                            \
>> +    void pci_conf_write##size(pci_sbdf_t sbdf,              \
>> +                              unsigned int reg, type val)   \
>> +{                                                           \
>> +    pci_config_write(sbdf, reg, size / 8, val);             \
>> +}
>> +
>> +#define PCI_OP_READ(size, type)                             \
>> +    type pci_conf_read##size(pci_sbdf_t sbdf,               \
>> +                              unsigned int reg)             \
>> +{                                                           \
>> +    return pci_config_read(sbdf, reg, size / 8);            \
>> +}
>> +
>> +PCI_OP_READ(8, uint8_t)
>> +PCI_OP_READ(16, uint16_t)
>> +PCI_OP_READ(32, uint32_t)
>> +PCI_OP_WRITE(8, uint8_t)
>> +PCI_OP_WRITE(16, uint16_t)
>> +PCI_OP_WRITE(32, uint32_t)
>> +
>> /*
>>  * Local variables:
>>  * mode: C
>> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-h=
ost-common.c
>> index 4beec14f2f..3bdc336268 100644
>> --- a/xen/arch/arm/pci/pci-host-common.c
>> +++ b/xen/arch/arm/pci/pci-host-common.c
>> @@ -243,6 +243,25 @@ err_exit:
>>     return err;
>> }
>>=20
>> +/*
>> + * This function will lookup an hostbridge based on the segment and bus
>> + * number.
>> + */
>> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t =
bus)
>> +{
>> +    struct pci_host_bridge *bridge;
>> +
>> +    list_for_each_entry( bridge, &pci_host_bridges, node )
>> +    {
>> +        if ( bridge->segment !=3D segment )
>> +            continue;
>> +        if ( (bus < bridge->bus_start) || (bus > bridge->bus_end) )
>> +            continue;
>> +        return bridge;
>> +    }
>> +
>> +    return NULL;
>> +}
>> /*
>>  * Local variables:
>>  * mode: C
>> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
>> index 4b32c7088a..5406daecda 100644
>> --- a/xen/include/asm-arm/pci.h
>> +++ b/xen/include/asm-arm/pci.h
>> @@ -18,6 +18,7 @@
>> #ifdef CONFIG_HAS_PCI
>>=20
>> #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>> +#define PRI_pci "%04x:%02x:%02x.%u"
>=20
> This is added in this patch but it is unused here

Ack. I will remove this and will use %pp to print SBDF.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:15:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:15:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194412.346345 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRp-00030N-Rq; Thu, 23 Sep 2021 15:15:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194412.346345; Thu, 23 Sep 2021 15:15:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQRp-000308-OJ; Thu, 23 Sep 2021 15:15:41 +0000
Received: by outflank-mailman (input) for mailman id 194412;
 Thu, 23 Sep 2021 15:15:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQRo-0002yp-Ee
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:15:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6bc83fef-b7e5-4552-8dcb-d34fb63797be;
 Thu, 23 Sep 2021 15:15:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A87422213F;
 Thu, 23 Sep 2021 15:15:38 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 70AF013E6E;
 Thu, 23 Sep 2021 15:15:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id jxU6GRqaTGG1CQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 15:15:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bc83fef-b7e5-4552-8dcb-d34fb63797be
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632410138; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=qiutPdaDUEQ53OVxPbWUI57eHOHJmixSloo1uUtF/HQ=;
	b=b9q3+E8fNtxbq9TBJSjcVGQBGVgDFmEOCzxIbLRb2xZyHQyeykuPcz01tn6d7lMZTxdv5G
	otsWgjFiqMP8zmhQALXwc5D0vgk024l+hgpVZRsmCmrq1S4r6w/cP84VG959dTtWL1/dUe
	nbNhFNbN/Z/kERDW8Z8CCE4yvzTG510=
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
Date: Thu, 23 Sep 2021 17:15:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="eB5LX1qkXLLfbcP2eMLtht3bzL22XkEnp"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--eB5LX1qkXLLfbcP2eMLtht3bzL22XkEnp
Content-Type: multipart/mixed; boundary="mh2JNQMn8SCSb66PUXlECvaGtLWicgkYR";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
In-Reply-To: <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>

--mh2JNQMn8SCSb66PUXlECvaGtLWicgkYR
Content-Type: multipart/mixed;
 boundary="------------8709DC3FBC915280D9627895"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------8709DC3FBC915280D9627895
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 17:10, Jan Beulich wrote:
> On 23.09.2021 16:59, Juergen Gross wrote:
>> On 07.09.21 12:11, Jan Beulich wrote:
>>> This was effectively lost while dropping PVHv1 code. Move the functio=
n
>>> and arrange for it to be called the same way as done in PV mode. Clea=
rly
>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
>>> check that was recently removed, as that's a PV-only feature.
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> --- a/arch/x86/xen/enlighten.c
>>> +++ b/arch/x86/xen/enlighten.c
>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>    	return ((per_cpu(xen_vcpu, cpu) =3D=3D NULL) ? -ENODEV : 0);
>>>    }
>>>   =20
>>> +void __init xen_banner(void)
>>> +{
>>> +	unsigned version =3D HYPERVISOR_xen_version(XENVER_version, NULL);
>>> +	struct xen_extraversion extra;
>>
>> Please add a blank line here.
>=20
> Oops.
>=20
>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>> +
>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
>>
>> Is this correct? I don't think the kernel needs to be paravirtualized
>> with PVH (at least not to the same extend as for PV).
>=20
> What else do you suggest the message to say? Simply drop
> "paravirtualized"? To some extent it is applicable imo, further
> qualified by pv_info.name. And that's how it apparently was with
> PVHv1.

The string could be selected depending on CONFIG_XEN_PV.


Juergen

--------------8709DC3FBC915280D9627895
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------8709DC3FBC915280D9627895--

--mh2JNQMn8SCSb66PUXlECvaGtLWicgkYR--

--eB5LX1qkXLLfbcP2eMLtht3bzL22XkEnp
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMmhkFAwAAAAAACgkQsN6d1ii/Ey/Y
Mwf+LW2t72MHtJWCoGlUMA/77oMXOPnTIo1U3l2LEMoesswuyQqGBMtFNXbz91NUdqv6xF+0nAME
rgSXsWK289p+Mrsq9FedKXg9/LLjEsCOh6PRE7qK6SwkbmsO4CB6JnkWIrytprgn273Ngcs+LnIE
Izqe9aEjiCQYwVMnSL0orWuq8WTKrc4vSQz3puHaIrvakVjtfucVPQMjH3KIaM+I8+Fgm7v4QXb4
c3an2dhYGASYtjh7Sf1EMLDzymMU1QPIpo799valwpK0mATVL02z+qcUdpZ+ipStjhOZtYa8lPMH
kOrp1nkKen/VH6fK8iRq69A8reCM8qJowaDKS4KVRg==
=h7Jj
-----END PGP SIGNATURE-----

--eB5LX1qkXLLfbcP2eMLtht3bzL22XkEnp--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:18:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:18:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194427.346356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQUF-0004ZF-Ab; Thu, 23 Sep 2021 15:18:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194427.346356; Thu, 23 Sep 2021 15:18:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQUF-0004Z8-5p; Thu, 23 Sep 2021 15:18:11 +0000
Received: by outflank-mailman (input) for mailman id 194427;
 Thu, 23 Sep 2021 15:18:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQUD-0004Z2-FF
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:18:09 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.83]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b503e5b6-6f1d-4a3f-97c2-6bb786c58327;
 Thu, 23 Sep 2021 15:18:06 +0000 (UTC)
Received: from AM5PR0602CA0010.eurprd06.prod.outlook.com
 (2603:10a6:203:a3::20) by PR2PR08MB5225.eurprd08.prod.outlook.com
 (2603:10a6:101:1c::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 15:18:04 +0000
Received: from AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:a3:cafe::58) by AM5PR0602CA0010.outlook.office365.com
 (2603:10a6:203:a3::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 15:18:04 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT046.mail.protection.outlook.com (10.152.16.164) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:18:04 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 23 Sep 2021 15:18:03 +0000
Received: from debc9f0720eb.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B1035B8D-BCAA-49A0-A61D-E1D9B7EAA35B.1; 
 Thu, 23 Sep 2021 15:17:50 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id debc9f0720eb.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:17:50 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6630.eurprd08.prod.outlook.com (2603:10a6:20b:337::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Thu, 23 Sep
 2021 15:17:47 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:17:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b503e5b6-6f1d-4a3f-97c2-6bb786c58327
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G36TS6GIEuBLd1hXsVLcS/2Ztquu31Kr/R6Fo0WYLN8=;
 b=ceX/b4MxkCZuBl+lFNyvhRjvtcgG+i8fvfq7jGpsbOYAdywojoKVQc40eSGsqCY5zO3cTFUkCBLtl0k2MGe8EC86d4HDMcUTkZ426YI6HWgIiJ1UJBeKCNpQ5yuX9/OnvOGbnpca/lOr6hfV7rePrmHH6X61ejlc/o0PIuED7MQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a2a460d7dbd3db60
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZoLgDEGDWuphW87KTynSJRuKMyyWtg2xqBVh/0h2jjt8187zgP0jLWoMnVFCVDxA/gm+FUVgVyfq4d1oVH+LXWPfBAdW3LBbiqiAYowHdPMvRpEyyqF59+TbIUQwXvq+Sp2Z+lyO3qwMsUygRchKeaJPS5sjknj93SJd0izAsMutbwtsT9LcgTF786hHC9Rmi14uv0QIN8CSUQf5wCb0iiZbPDpx5whELknHSOMOaepBe0VGSHw0DRMVGD9Kot1vNwj3VkfGlkkyW1VCLTRAAwAg+HjP23kgpKe1VQz/zqQi3kPBjPLIm8t0aW4FWC8nf+YKabh20WFV8RIO4sKkVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=G36TS6GIEuBLd1hXsVLcS/2Ztquu31Kr/R6Fo0WYLN8=;
 b=P835xLS+TMtFxcsb+E6YrN0lyBz7PERrSBAczzeIuB8nAEniPkEiz+0zZmH1xGVWsoEp+iZ7i1CMZHDa7Y34Ik7RZFdFxQeUj64T5P/uQQAyLHZTeCr8dWe2MOY/CeK0ywDhEOI4H2zEmC+oBI+g1W02CzN70VuVVS83yChDyrvvWZjvOiGyEbWhnYt8vLNO6P+plWp1Ge7wckVa6rlJMIB6UxSe/d1HKsjxLFdElUq5BUocNOt5jFRfqI6r5JFWaTf5UWmq9ZhxTGlOGXoiRFhMEuep4iaBShxN6FFG1KFS1txV/CqQgDK2DcQThRP+kUuXflFDoZ0cORcmvBiJuQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G36TS6GIEuBLd1hXsVLcS/2Ztquu31Kr/R6Fo0WYLN8=;
 b=ceX/b4MxkCZuBl+lFNyvhRjvtcgG+i8fvfq7jGpsbOYAdywojoKVQc40eSGsqCY5zO3cTFUkCBLtl0k2MGe8EC86d4HDMcUTkZ426YI6HWgIiJ1UJBeKCNpQ5yuX9/OnvOGbnpca/lOr6hfV7rePrmHH6X61ejlc/o0PIuED7MQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Index: AQHXr6aktO5EpOLLjku5HahQRSHFEauw5LyAgABsh4CAAGvCAA==
Date: Thu, 23 Sep 2021 15:17:47 +0000
Message-ID: <5610FB39-51D5-43BB-880A-4899125DB788@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
 <33deb79b-54e3-c00e-653c-9c1191574b4a@xen.org>
In-Reply-To: <33deb79b-54e3-c00e-653c-9c1191574b4a@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: fa31dfd2-1a1e-47fa-28fa-08d97ea55700
x-ms-traffictypediagnostic: AS8PR08MB6630:|PR2PR08MB5225:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PR2PR08MB5225C671661E9794215E2848FCA39@PR2PR08MB5225.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 tQh7OT8HTcUI9FsSt2qRcTNothBKXY0FpQkLOY9vO5kK4rY33aZDdU6X/RQ1GtmhDeP0ltf649NhGkNBO+l7ptVF5R7Ipds1hL17HOAH2BDghttpDzqYElDwZNdZnZoye6mvyXX0WcJttUsELGCn3soW6C/olJPZ1ZzaeEeE0mGgwHZhSy4Uimvm+sKvUHnv7bwAnWmY7EEViLwvM5AcgVvkV6SiuB3jvNNOQso3IzUN49gEOZswhuQFubErGKj2ruvA91bpd+U2bGYKv8bJ9Imyv6MMdSQotKRQKATTuMk1dmKs0hOssp2jQtIHxh5E6R5XqXKLg3sbOT2mJmjA7Hf9NAA8SHlkqYU4QlxCYRvq7eKItfSrUbNeHMCDUJ1bK/KilFHyB1AZkMrUcXB4eOaztJ5C24kXpfc6fu669teVqDTZ/anrSxWrohiIkRAtDnUiCoHbN6pJsZjrs8L2KA0WbZTblIegOQdg2XwKcdpkblAPtxWlhG2ZZ8eL1sbOqzq10t9u3oV7yYsysyadAEEgqAZR2gyXOZbB/PYbqOoHt3Y3bBTUNeXpH2inOcpA2DmAHylm984XG6KuP0cKUvFtIW9NAsrBh0vX1VHDVTYVp5bAXGce8+P/ULosy5IJfH0+hwDy9q6IoyyBPVa54O1fmzDfBTrPF/Q0VxgHrpRIFKfBD9XBaJ4qHQ7dhRj8aM2h4Esr6xJXezKfwdeVIJX+ZYGwUXjJ65yqmPpgbWeIIPt6V8KFo+KQol2iPgtg
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4744005)(86362001)(6506007)(66446008)(2616005)(186003)(6512007)(2906002)(6486002)(53546011)(38070700005)(66556008)(36756003)(5660300002)(71200400001)(8936002)(54906003)(66476007)(508600001)(316002)(38100700002)(4326008)(33656002)(6916009)(64756008)(122000001)(91956017)(76116006)(26005)(8676002)(66946007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <846F4F9E641AE649A63380F52929C2B6@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6630
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8c1b0374-8375-493c-ce90-08d97ea54d2a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MdmLkAkrFC1aXl8DK0ykaYVdE79L3j9/AejMfhACUJfQNyiV3eLyCe0Fdj79KYhQPDvTXKovNamKZMHEP4qBIgAnViaA9Kk2q6Q59RleNDsyeehGKFymSWrJni/DGXsiAwTFxSIwz9q3a9w417tpEyGEQu8EcTQmvXLIzgD6aMExO2gfFHqca27Zqh7jmhKbP1LqhCBjzflEPMw3G0R4sfmi5gpflJeOmg6l4d/rX7LmolGM16yXJaxxWPHloJoq/XmEFhX8ilnBrROyRlhT6okXWDjJhgSJttQMnwIYWTRKUDwIRBOCMF33kLh1Vr0TVxJHp2rKj10c2iUbwnWZXEqrrx74jLmcCasCwhp2np3Pk6FMDzAvZih/1H7CeF6XwtoSZAezaYgCXraaSQeDz7VqtTLDv8dWCCVRJlvJv1XhiYvPMzlnCar4qNzg4zDW7qgRAejBxAfjYQ9g3Rz65C4pBzbAe4eP4+b7hDcI5zrdpAhJsx4aMRHdD9sz+mgK+QJDJd/jiwg5N61l0lH7a+ZLa9T0WoGviA7NWWyx1C2UvmyEmWrSBUbrQmrwyfUL44rh0JW7LpiqPrYZsDdyaQCfIIj2lgDwxOdJApe9/KiIUJQ40Aqu5mgl0toAU3Kb0gIWKXqDnFQ94v6abiGeLQg9WikFzwF9xc+09QMPm6n4A7ja3ZKS4ieG/CSmhso7HK5YbILB2TnL/oguB44+hg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2906002)(26005)(107886003)(186003)(4744005)(4326008)(6486002)(53546011)(356005)(8936002)(6506007)(8676002)(2616005)(6862004)(6512007)(316002)(70586007)(70206006)(54906003)(336012)(36860700001)(36756003)(5660300002)(33656002)(508600001)(81166007)(86362001)(82310400003)(47076005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:18:04.3243
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fa31dfd2-1a1e-47fa-28fa-08d97ea55700
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB5225

Hi Julien,

> On 23 Sep 2021, at 9:52 am, Julien Grall <julien@xen.org> wrote:
>=20
> Hi,
>=20
> On 23/09/2021 07:23, Stefano Stabellini wrote:
>>> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
>>> index 4b32c7088a..5406daecda 100644
>>> --- a/xen/include/asm-arm/pci.h
>>> +++ b/xen/include/asm-arm/pci.h
>>> @@ -18,6 +18,7 @@
>>>  #ifdef CONFIG_HAS_PCI
>>>    #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>>> +#define PRI_pci "%04x:%02x:%02x.%u"
>> This is added in this patch but it is unused here
>=20
> To add on this, this is technically not arch specific. So if this is nece=
ssary in a follow-up patch, then the definition should be moved to a generi=
c header.

I introduces this to print SDBF but I think it is better to use %pp to prin=
t SBDF. I will remove this in next version.

Regards,
Rahul
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:19:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:19:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194434.346367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQVY-0005FT-Pk; Thu, 23 Sep 2021 15:19:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194434.346367; Thu, 23 Sep 2021 15:19:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQVY-0005FM-Md; Thu, 23 Sep 2021 15:19:32 +0000
Received: by outflank-mailman (input) for mailman id 194434;
 Thu, 23 Sep 2021 15:19:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQVW-0005FA-R8
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:19:30 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.69]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 36a27d2a-66df-4bf4-8863-d8d7e125042a;
 Thu, 23 Sep 2021 15:19:28 +0000 (UTC)
Received: from AM4PR05CA0029.eurprd05.prod.outlook.com (2603:10a6:205::42) by
 AM6PR08MB4935.eurprd08.prod.outlook.com (2603:10a6:20b:d5::25) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 15:19:26 +0000
Received: from VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:205:0:cafe::7b) by AM4PR05CA0029.outlook.office365.com
 (2603:10a6:205::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 15:19:26 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT020.mail.protection.outlook.com (10.152.18.242) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:19:26 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 15:19:24 +0000
Received: from 7f3d68d77daa.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 CB340EBD-CA0A-499D-A8A4-83DA249EB9C2.1; 
 Thu, 23 Sep 2021 15:19:12 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7f3d68d77daa.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:19:12 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6792.eurprd08.prod.outlook.com (2603:10a6:20b:39b::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 15:19:11 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:19:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36a27d2a-66df-4bf4-8863-d8d7e125042a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CE6xfdgEbdAmnNOTa4mvdWHE85o6Ets6LCElyMVzqmI=;
 b=EvaYEBfTRw/VPr0aj1qumeSPtOHwvoj6iZeLTElmh5LBd79Qqv+PcqVx0ymAzwcwRqvXYkViaxrC8CZht72gm6voEmoYUBM3bja8g5P8ilcvqwJ8iT2PGbrGPZcGxLJlGDdLAsOz3FvCITjA4mlN+fVXXiwXnzH7YMrm8QWEGZo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: beb4aa114580152b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GwI/NQJ4EtjRxkkENbLLwO+Ekd8/EkAYBiIFWsG9nr3QdJQVEviupLb0TIMwzMglC+ufMuH0egpmr8QEHdr/fLXD1tbUKTOvENW52K+W56iPrsFNFTPUCLipCSGcNrlGiCMRaZErE7pbRVyKOuurehQPu6NjTahwLBx/ctV/0qMbX9+NnKkA5RX/257yqFDcGaepXIwOB7FMFh+WtB1rdW2Quhw2PxzyJvjf6nRi2p72JvOEttNLfIgs9gDw7QdYcWskYk5EgrPF9Bb1WkHUQxYUa6ROoRwVRr9O8Tb1XpTr2oE7+7RZPkc0V2q989nTq6wvYvE0xF3YibS9+h5B+w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CE6xfdgEbdAmnNOTa4mvdWHE85o6Ets6LCElyMVzqmI=;
 b=ixoCFCj4MrdtZCCcLMQ0bk1IIpUh8X3gTGCpYI9U9uS4nPfe5hDkw6YIRA5l6IRLwMrao15xp25hzjzQEAxHGbVSUAynMO7B18kxSeoK0RsByQ+WPnqJNYQ3E45sNDx1M9lAgi6SPP36tx4E5apcu3KNxAeIj/JhlqSSlJsnsEbiuVM+eUqZ4BhAbvLZUnujv11cTLADl7L8r34nflbI8xhF6U8nulbRQIjzIW2EP+S+IOffGXm7AIUQ1CPTmgI90z6J9XKoR2KtbLeStPrrJVFB0ck/7hBTbi7M1oZXBzMSj289EojiOAZZyK7SvmCKX3jTMipRRFRe/pvLx3sXMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CE6xfdgEbdAmnNOTa4mvdWHE85o6Ets6LCElyMVzqmI=;
 b=EvaYEBfTRw/VPr0aj1qumeSPtOHwvoj6iZeLTElmh5LBd79Qqv+PcqVx0ymAzwcwRqvXYkViaxrC8CZht72gm6voEmoYUBM3bja8g5P8ilcvqwJ8iT2PGbrGPZcGxLJlGDdLAsOz3FvCITjA4mlN+fVXXiwXnzH7YMrm8QWEGZo=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Topic: [PATCH v2 13/17] xen:arm: Implement pci access functions
Thread-Index: AQHXr6aktO5EpOLLjku5HahQRSHFEauw5LyAgABvjQCAAGkgAA==
Date: Thu, 23 Sep 2021 15:19:11 +0000
Message-ID: <FE809237-508B-430E-BD96-13AC88BB392B@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <f05b02c04803809bbe4ebd49ed4abb5a4656b010.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221913430.17979@sstabellini-ThinkPad-T480s>
 <bcb7c35b-cad1-8da1-8723-82f1f79d4f74@xen.org>
In-Reply-To: <bcb7c35b-cad1-8da1-8723-82f1f79d4f74@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: f21d33b3-fec9-4e01-71f9-08d97ea587e0
x-ms-traffictypediagnostic: AS8PR08MB6792:|AM6PR08MB4935:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB49358738DDB4D67001E26A96FCA39@AM6PR08MB4935.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 copAQoWmCUzwHanAMiTKXi/rq7xtLCNZRT2UYILgYnMTtE/eFPOHfXPonQbwFoJ2h6PW+hA/19sGOfMXrdZxuJkw1hALaDoe/4S9ti2DL29s5JYz3MDW1jU5bg7bS74yDK1+p6pjN6agqUN/Los54B+P0FsZ9UBvUbYzs/vngHj+5i/oMWu0abw9PrBliZ+MARFTgL0xpmzXjPgTQR3Uo/OGz7gJpPUAf7MTTOVararOZQhutDXFv64e4Gqc8HG+iVcCLjPolLBx9jYLOLtmk4+1jtS1l28JOg8sz5A4Lyp8hUL2YF7nVsO8gA58JEKEKDvjwCouusxEV0hSbWnmxfQNh3OzzImK6vq0suZILt89krzAKY6RGAmLwXtWDeDxASMtU+4ZnvblUJ+X2NysyN9OiJruaiZZx53/NKThx7C2idV2LfsEEDcI4sTeEW8Qe8n5MUvmZsM7OI+oLWAL0gQPja6K+v+O2gQKp+OiIwZz7Vh9SUFJ6avR7w+sWWgOLSzMRhq8L5VbN9O90Ld6P15sfKlUU7ARxB440nBMz/lJ6KDGxzS1IvjkpJ45bWGGG9lrzas/4V08izgi4WrX5LrI77ObvZ+gFKeMOrGU4hSeEZ3cTgZnv3AlhjPYqeNddyGut3hdqQ2add1GHtsM6C4OomlSzyjlk/W/Yk2zxVPvo1mlgyL7TfDnbrIYm4QpkubQaJNEvmEar+IeQEPieR0/d6D4LdwVrn/rHHRYt4wMsVrA/Bio9wgzH3PwGPcm
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(4326008)(66946007)(66476007)(91956017)(122000001)(186003)(2616005)(38100700002)(8936002)(8676002)(36756003)(54906003)(76116006)(64756008)(66556008)(26005)(2906002)(508600001)(6512007)(83380400001)(66446008)(38070700005)(5660300002)(86362001)(53546011)(316002)(6506007)(71200400001)(6486002)(33656002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F1C75946B535174BB86B87F7D1FB6FD1@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6792
Original-Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c170cc3a-aa0a-4058-4f14-08d97ea57ef9
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vuoLhwseRH4dELh47r+9Au9TbCbTXt+bO7jkUg5aleRVGspjPI00VPOuyNbSx/IkwHcmVn/IxIR1j228tfnC10CTeouk8P3PVu+rDD2n/6JNyuNiQN+UwEL8/ry79V6cYMpHZuXsf3jHg1G3tpw3FMjw4cqxHdufRAg0GnGcXpnkYm0Byp40CpOmTGoHPV+wEsSObSkcI6lM3zIw59fsIwlR0JGlmG5qZn86Nj94aId5Ah+twF+qF58E0KPoLr+XM85AivuUUNtPVaxtav3kAUwObRulXgQNK4r7+wz2/kZ48mgtL3VG3IQSOG7VJbxfXj0ugU9hUrE7RViJ8LULpB6/lFMkY5Qkc74RPeeDT4ZUgzX67DUH/t5/HDMDrJq9+LgZ7STbqdQFxV1IR9+ty5i2Ef0JfQgFsoBfJS5riT+OwW5z89zPGqlSkhrbKyoql7cANad6sa/C9LXF6RcXoRn+eXdAQmmTOHKrPpToZBSt93/PeGIwyXIE79zA3yNeGV8BsyUsDFLLQUrFKrdVS7m3sTHtNpGtJM+l6G+GDnRtIWLMSxS5H0rtRc2NEHVlT6XcBqXM5D6/5a3UwOIqTdPOE5muDHw2vW+eplfWQHN8nOjjmBl9jKYZcQo6cUeC2RJJbwcAT3Rp/7BMt9akkEww9TX+K8u0PDuz6qlsgEauFxJYV7Qx4hbPs19lwROs10iE/cgxE5i2Pj07BAueqA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(6486002)(6512007)(36756003)(2906002)(186003)(8936002)(508600001)(6506007)(53546011)(83380400001)(54906003)(26005)(6862004)(316002)(336012)(8676002)(81166007)(5660300002)(356005)(4326008)(70586007)(86362001)(70206006)(36860700001)(33656002)(2616005)(47076005)(107886003)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:19:26.2923
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f21d33b3-fec9-4e01-71f9-08d97ea587e0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4935

Hi Julien,

> On 23 Sep 2021, at 10:02 am, Julien Grall <julien@xen.org> wrote:
>=20
> Hi Stefano,
>=20
> On 23/09/2021 07:23, Stefano Stabellini wrote:
>> Subject should have xen/arm
>> On Wed, 22 Sep 2021, Rahul Singh wrote:
>>> Implement generic pci access functions to read/write the configuration
>>> space.
>>>=20
>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>> ---
>>> Change in v2: Fixed comments
>>> ---
>>>  xen/arch/arm/pci/pci-access.c      | 58 ++++++++++++++++++++++++++++++
>>>  xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
>>>  xen/include/asm-arm/pci.h          |  2 ++
>>>  3 files changed, 79 insertions(+)
>>>=20
>>> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-acces=
s.c
>>> index 04fe9fbf92..45500cec2a 100644
>>> --- a/xen/arch/arm/pci/pci-access.c
>>> +++ b/xen/arch/arm/pci/pci-access.c
>>> @@ -16,6 +16,7 @@
>>>  #include <asm/io.h>
>>>    #define INVALID_VALUE (~0U)
>>> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>>>    int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t=
 sbdf,
>>>                              uint32_t reg, uint32_t len, uint32_t *valu=
e)
>>> @@ -72,6 +73,63 @@ int pci_generic_config_write(struct pci_host_bridge =
*bridge, uint32_t sbdf,
>>>      return 0;
>>>  }
>>>  +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>>> +                                unsigned int len)
>>> +{
>>> +    uint32_t val =3D PCI_ERR_VALUE(len);
>>> +
>> No blank line
>>> +    struct pci_host_bridge *bridge =3D pci_find_host_bridge(sbdf.seg, =
sbdf.bus);
>>> +
>>> +    if ( unlikely(!bridge) )
>>> +        return val;
>>> +
>>> +    if ( unlikely(!bridge->ops->read) )
>>> +        return val;
>>> +
>>> +    bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val);
>> The more I look at these casts the less I like them :-D
>=20
> I really dislike them. This is kind of defeating the purpose of trying to=
 be more typesafe.
>=20
>> One idea is to move the definition of pci_sbdf_t somewhere else
>> entirely, for instance xen/include/xen/types.h, then we can use
>> pci_sbdf_t everywhere
>=20
> AFAIU, the problem is the prototype helpers are defined in asm/pci.h whic=
h is included by xen/pci.h before defining sbdf_t. Is it correct?
>=20
> If so there are two options:
>  1) define sbdf_t and then include asm/pci.h.
>  2) Name the union and then pre-declare it.
>=20
> Option 1 is probably nicer is we have more types in the future that are u=
sed by arch specific but defined in the common headers. We have a few place=
s that uses this approach.
>=20

Thanks for the pointer I will fix this in next version.

Regards,
Rahul
> Cheers,
>=20
> --=20
> Julien Grall



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:19:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:19:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194437.346378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQVq-0005ha-24; Thu, 23 Sep 2021 15:19:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194437.346378; Thu, 23 Sep 2021 15:19:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQVp-0005hT-VC; Thu, 23 Sep 2021 15:19:49 +0000
Received: by outflank-mailman (input) for mailman id 194437;
 Thu, 23 Sep 2021 15:19:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTQVp-0005bT-C9
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:19:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ad584d1c-1c81-11ec-ba52-12813bfff9fa;
 Thu, 23 Sep 2021 15:19:43 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-UF8XVssKPn2UierGQXgDSA-2; Thu, 23 Sep 2021 17:19:41 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:19:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 15:19:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR04CA0112.eurprd04.prod.outlook.com (2603:10a6:208:55::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Thu, 23 Sep 2021 15:19:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad584d1c-1c81-11ec-ba52-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632410382;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ePumqdto0fRB5Q6HDi/I4XD/7s1DkFeFdUnzVrFS644=;
	b=ewLPF+TMFUayRwgA7noFLvwt0vHae/4RClxgzSct7oan91UXkh5utzmO4MEbpe5IUkQ1x4
	P/OaEbPTmv+54N5EeJ3oVnS20g+E460Wxbr5TQX85Yc/yw+785ky3McotRlw9uVP7MTojG
	jwFeRcAi4U9QjJcwzuToqTxK+jsoSDM=
X-MC-Unique: UF8XVssKPn2UierGQXgDSA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GrmJxD9ZV/eppmkPovLbT+61w5cSlI9CfYjIe8ZdpdMou4Lob+KC+Eq3wzvxpUsqRGb/EPgs5PtNV1fJeh2WtSxhRf6pUmZ6C79V1bcTtePo2Bno/2fYWgv0JeSUj5f3buifI8CAi9Uc12PvfKXtszUHyHMRJ6o9x9jQ26S+qf4JiJKA2js+ll6zFt7oS9RBJ2seQKj6646Px1Ak2yxeXY2wLNvaKDffpIW+AKbygsro6kE1rNh8FDFmumJwcqkRL3OjtslcGL+0OCVx+XsUQea1kckPznts6PIm7TnDiqG0Ovya72D/XwokvPXiMIv0dCy+99c3ub781h+O9+jc0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ePumqdto0fRB5Q6HDi/I4XD/7s1DkFeFdUnzVrFS644=;
 b=e22TUJQrDPrjbdHQ5eeS6a7SlIJtmOlWU/nUWBc83+YDkuXMg+ebOLGu9bPlsr8r+7CS7PgQOyR9sVGVRUJG22SYm5GEJ21x6JLsA6XFsWr4xibhBiU6QzA9rvvSZKsUtTvujbl8divMVodzRirhUQt3uJScQUZY45wSBmrK9m1cDIXMtvVEUm9aaVwLuP51CdeyoNSEvVupxVGH4D3b7lygs6aqPV3hUPMNg/ivNzZc6tR7icQ4e7wO33iHQ/OTpsc5cJgihW/Q+81YXitYy0dIWpenBFto8Fo4UDq+wVvy9C0Ey9zQzCN5p94JJ/5Tw9yp+MBaF/e4JQ0Ohop51w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
Date: Thu, 23 Sep 2021 17:19:36 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR04CA0112.eurprd04.prod.outlook.com
 (2603:10a6:208:55::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a2b8563e-2070-43c7-3b39-08d97ea58e68
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68628E9EEFE1B77CA4BAA484B3A39@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Hnwhemz2GuRAS1JqvSq2mV/HlLdGzYR7CjDynwi/mU49jeDjUpwmpM4xYQbaU8Z3GF5q7wW5nC7zoDJNdGDQwLJ29BHjo/aSbeo+DxvhCZXoRhUMhOy4sYyaxr2+RyHx2U95j/reIssBhhp2gRakX2TXIHCABPBZj040okY8WMEMbfNJqc4ReLmiaSq4mukSmPa7Pb2vfIv8My2BIvqQzqA9aU24wRKpT+nle80evLP1++3Rf8ksej0RLQxa5c8J6A4hgflqoDvlxDXrc7wnrOFGSGxHPnvDWv7cJ1MkDcZRpBNxFr2PLRERNrBSMyzou0Q0OB4gTXpjPKRphhgYwOi7JNRX+YHpcZhSWA1RYIgqjVhBZOVJnul1YVf/HTu5peKYT5cDiKZWqzEmOgzSfNEs69iEXbt/SUIj5p0y9AsekOesKHa1AiHO5/ywyxutV74IdMV5CYcJZXdQQP0fmZAR/gA8iIOWjfd3WlaNkgy7jeJr5/o1eo2NNgwzn/Dto7+tG+J0/W8eSMnByLj/ZUBfwzgZ8mz+YAhGVbNQx4UfDGHyBu8ttq+gH+k2VPhfKWfjB54KvbbhbCau5t2xUnybYbcJ1TfyAbHPeGvZvUnfhxj4wU1iqVmyWd4qUpQWvJY6IjO/nVnW81b6qokCj3FmPQO9FdSP+GYeWtzxSeg5PvCkgCDueO+hJZowZjGxHuAsdCAXp2wQ8rH/FZ2Fz2frQmjRONiDeR9HdR+msYEBgthKpyDOQtSjOFDVYmMNZCQlTL66Ooxt4bbMx+BAq70yWKk8E2BEJ0tM/N37mDk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(37006003)(66556008)(31686004)(8936002)(38100700002)(2906002)(53546011)(31696002)(54906003)(6636002)(16576012)(66946007)(956004)(4326008)(6862004)(6486002)(86362001)(508600001)(36756003)(2616005)(66476007)(83380400001)(26005)(186003)(316002)(5660300002)(52103002)(158003001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SGJUSWNIRjNnWm5aUlVMS3hXVnFZYzUralBTVGY1SWYraGswcFUwZTQxOXJF?=
 =?utf-8?B?V205Zk15T053RjJHb1JkUjhvNUhBbmh4cnZrZzMxZHptV2h5Tmo4MzQ0akNW?=
 =?utf-8?B?L2JFVXdLeWo4U1NKT2dIa0t3Wll0MDhpTGx3QXBlc2hqQjMvMUdEM3FqV1A5?=
 =?utf-8?B?eEZUMXNYbEoxSmk1NWNVWnltNTNsWEJmR2IzZDh0Zjkyc1FWT1J1UTdHZW9D?=
 =?utf-8?B?dzJaVFo0Uy9WcktWbm1zVVVKdVN2dGxOMmZ3NyttTkVHdUsrVnl4SFJGaHZB?=
 =?utf-8?B?bldSNmt4Y2lPR1NYNjIzWXAyOUJqSkhQMzF2RUppMVo0ckFweFNhZmZiTjhZ?=
 =?utf-8?B?WG1lQTRwamNMQWFod1Q5Q09UcG9MS3pJVksySzVHbkdaMlJsbnhkRGZWekY2?=
 =?utf-8?B?SHlUQ0ROZ3AzdFpuSjFLUE1uL2lJTi9GdTdSQ3VKZEwvU1VXWFk0RHZ5cExt?=
 =?utf-8?B?ZkkxMmZESWs1YjhqL2dTRndTR2NtZjFQeUxSemxELzlGbGxCU0JFSmdkZmpR?=
 =?utf-8?B?UXR3WXowWjNpanlVcE9nVWF6aXhRMkpkbytGRDVtUnNXWGR4TE5KdnF4NUli?=
 =?utf-8?B?N3pIT3BYaWZORWJYZGtpR08ydmI0UWhOTDI1dGhWSThMM0JvKzAvNHZYdGtV?=
 =?utf-8?B?M1lTUStVYi9LQzB1aEJyajR6alFWR3MwdmxaZGJFdFlnZFROazJHcG56MUFu?=
 =?utf-8?B?VEF5MVhHemt0eEFoelRhY00xMm92OThqYmFkbkV2REVzc3VOSll1YnN2RHBn?=
 =?utf-8?B?K2p1b3F5d2IrZ004UFc0aEd3RXM2MVNROU5CaEZQS3lvQTV2MHlvUldVamJr?=
 =?utf-8?B?RmJOVXNiWFFXR0svNkFKZzB0RFVqOG5uZ2V6NXU1OTRRNzlGQ25kOTl1S3l6?=
 =?utf-8?B?WlQyUmVDRzZwRXpqbnU3aUNwVWxtbWhPUlBuVlpGaFZlbjBCZjRlNG84U1BL?=
 =?utf-8?B?bnZpakE5UThlNzlMbDc2SUZwVU4xdWlha24vcFFjdGpOVUZvRDlDSFJ2eTBo?=
 =?utf-8?B?Y0ErVWdPRTlYVm5RdVB3aFhEenRLVGFUQmEwTHBZM0IvNkQ5eVB0M1VpWUF2?=
 =?utf-8?B?UTNyNFFYTzRUMFJvN3Q1ay9OaHpVb0NjWWlNamVCZEszRkNkaXpsQVhaYUJI?=
 =?utf-8?B?b0dPL1AvaDN2VzZBblRhZ2VucEhab0ZKMzBMektFdzFIcnpHSUh3amRsUTZp?=
 =?utf-8?B?L3FyVjRlaVhDcHNYYzMzWGwxcEh6a2tRQ1dsbmNOQkdNaGdDam5BM0FYQzYw?=
 =?utf-8?B?K08zeXBuSzN2ZXI0RWhMblNNODkrclN6V2JnbXBZYmRZT1VpTTE2RE92UklJ?=
 =?utf-8?B?dE1pNHVWMENpcHppRC9McURnZk5aeDVxejNrSGQ2UHdYSEN0ZWpvMVFoRHJw?=
 =?utf-8?B?Rzd3UDhySnh4Um5JT05kNHcwVXdaSGs1TnlQaU9HeHY4NWNYVU1ZcURpa3Ru?=
 =?utf-8?B?bXhYWnEyeW5HSm9YNWpsMm1jck9ueFlMYjVtSURlQlRYUG13TXZkQS9DV2FO?=
 =?utf-8?B?ZG1yUFRZYjYxSjNZR3FLcG5wcFMwajlFdmlLWDV1ZTE2ZGg5d0VLdEx0Q3pW?=
 =?utf-8?B?c0hsQUZyK0cwaHo4d0tkdENoNWxZQnROYlFLc2VFM3E3ZXIxOGRYYXFON3BV?=
 =?utf-8?B?Ujh0eklBLzZNUnArVHp0UklobmNLa0prUEk3MHgvamNlVndZSytBMlQrTUZj?=
 =?utf-8?B?Q1RZL0pRRFJsdFBFQ2ZMRGhaUDYvMmlnempQQ0xCZnJ1ckpmODBZcDY1MzVp?=
 =?utf-8?Q?Q47NSnk/ihufygNf4wWv354jf4Fh59/KpZ/yi5+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a2b8563e-2070-43c7-3b39-08d97ea58e68
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:19:37.4384
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nt+Z4L/F4H+D62AlbFUV0oYkYkZWBLPj/hDF2/+GcoEj8CQTiT2EFEtuqc6a62yVEWzvFPwfCUT2F3cCSz4RKw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

On 23.09.2021 17:15, Juergen Gross wrote:
> On 23.09.21 17:10, Jan Beulich wrote:
>> On 23.09.2021 16:59, Juergen Gross wrote:
>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>> This was effectively lost while dropping PVHv1 code. Move the function
>>>> and arrange for it to be called the same way as done in PV mode. Clearly
>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
>>>> check that was recently removed, as that's a PV-only feature.
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>> --- a/arch/x86/xen/enlighten.c
>>>> +++ b/arch/x86/xen/enlighten.c
>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>    	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
>>>>    }
>>>>    
>>>> +void __init xen_banner(void)
>>>> +{
>>>> +	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
>>>> +	struct xen_extraversion extra;
>>>
>>> Please add a blank line here.
>>
>> Oops.
>>
>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>> +
>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
>>>
>>> Is this correct? I don't think the kernel needs to be paravirtualized
>>> with PVH (at least not to the same extend as for PV).
>>
>> What else do you suggest the message to say? Simply drop
>> "paravirtualized"? To some extent it is applicable imo, further
>> qualified by pv_info.name. And that's how it apparently was with
>> PVHv1.
> 
> The string could be selected depending on CONFIG_XEN_PV.

Hmm, now I'm confused: Doesn't this setting control whether the kernel
can run in PV mode? If so, that functionality being present should have
no effect on the functionality of the kernel when running in PVH mode.
So what you suggest would end up in misleading information imo.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:22:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:22:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194444.346389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQY0-0007Bq-Dj; Thu, 23 Sep 2021 15:22:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194444.346389; Thu, 23 Sep 2021 15:22:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQY0-0007Bj-9h; Thu, 23 Sep 2021 15:22:04 +0000
Received: by outflank-mailman (input) for mailman id 194444;
 Thu, 23 Sep 2021 15:22:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQXz-0007Bd-KK
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:22:03 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.50]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0072c7fd-1c82-11ec-ba52-12813bfff9fa;
 Thu, 23 Sep 2021 15:22:02 +0000 (UTC)
Received: from DB6P195CA0019.EURP195.PROD.OUTLOOK.COM (2603:10a6:4:cb::29) by
 PAXPR08MB6527.eurprd08.prod.outlook.com (2603:10a6:102:156::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:22:00 +0000
Received: from DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:cb:cafe::4a) by DB6P195CA0019.outlook.office365.com
 (2603:10a6:4:cb::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Thu, 23 Sep 2021 15:22:00 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT035.mail.protection.outlook.com (10.152.20.65) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:22:00 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 23 Sep 2021 15:22:00 +0000
Received: from c6faf7a0b3f7.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 C1A37528-9CFA-4F4C-8CAD-A1DC48B1ADA6.1; 
 Thu, 23 Sep 2021 15:21:48 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c6faf7a0b3f7.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:21:48 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0802MB2497.eurprd08.prod.outlook.com (2603:10a6:203:9f::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 15:21:43 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:21:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0072c7fd-1c82-11ec-ba52-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9r3/3zlgkI8hibVZupWUcz56kvYJylQd6uukQR2DlrY=;
 b=8J2of4c23oLIE5Y6Fw1u0T/VyrGPEYEpotFzAU7Vc7gwCMkJc66f9dq2tKF7+Q4khWy8+4jT2fwf/r4BdCXZeChbnA/YTt/HB1sL6kvJ0Ir0Rqxnx8rz4MVso6Z8wNumZYmH+KlKmY8O676aZlm2sYr6xZhQSzMZRUKDnziKRKw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5660dd6988b5a634
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j8UNjUhFliCzMYHfv7O759+cNTvrbfuYeG9o8tb0++r7TyR/vC3oJfZ7iohZrZw3oeyptrFVzEPljWHnQNNBCfmOrNVy5vL1pR+3PB7TsvEPgYazIXCkPjiLf92Bi05zhU1OFt7AwwR3U25MDlS3e12PUrrMZJzRS9Baq5v9aEQh0pS0XiQr7tf8FjqzCEowNM9YZk6X4WmjdU+zQKcuueSoXiS4e5h0R+dp5wzZV+hzNu5a9LrE0g9kfp8eo1eQnjMnD4cOhTc5PL9IW3bgkuIsj5v8df/mUw32O2Oite/kg7YtooXceBTIa9MGprCyjlN+oRkWEbsCkcUKiMR8bA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9r3/3zlgkI8hibVZupWUcz56kvYJylQd6uukQR2DlrY=;
 b=Xp1cETXRk9JPxwHwK6AnZY+NHPJzwTXiwlNmlbq6e3/Kk3xxIfLLtDPNGJlxOghs4Dlt4K8/sSdG5jfmIrwcFJ353IRgJIMtRWzzbcEL0zy7I14EYm9nmFkMiE/WAsQW9agt08PyMc3ipi9bUhCwf2YBqIfLRkN2EerXt/BPOHea99i+FUN9TWrI/3NAuLqnyfVq4FN3L5+9BzzS5BN1eutZUY5foJAIPc76UGEcjxPIRkHQ2e2D0gamLsmOdV5UzW4eMeT+RUK6SyzO883GSAENmghdRQOdOV48RO+xWOI2XBGrGtAbQIx9B2eXB5ZW+YmiX2ahJzNz84WgpdCQlg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9r3/3zlgkI8hibVZupWUcz56kvYJylQd6uukQR2DlrY=;
 b=8J2of4c23oLIE5Y6Fw1u0T/VyrGPEYEpotFzAU7Vc7gwCMkJc66f9dq2tKF7+Q4khWy8+4jT2fwf/r4BdCXZeChbnA/YTt/HB1sL6kvJ0Ir0Rqxnx8rz4MVso6Z8wNumZYmH+KlKmY8O676aZlm2sYr6xZhQSzMZRUKDnziKRKw=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 17/17] xen/arm: Add linux,pci-domain property for hwdom
 if not available.
Thread-Topic: [PATCH v2 17/17] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
Thread-Index: AQHXr6bBfRQPNKXni0uaclDek7ufJKuw7NmAgADRRQA=
Date: Thu, 23 Sep 2021 15:21:43 +0000
Message-ID: <D73B88A5-1D76-444E-9DCD-12522004280C@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <e1b8d878709dc8e93a3bb340f921a24976cebb57.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221948300.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221948300.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4b25f4d2-b377-4b6e-7f2a-08d97ea5e3d3
x-ms-traffictypediagnostic: AM5PR0802MB2497:|PAXPR08MB6527:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6527B854F640C5EC2FC53B23FCA39@PAXPR08MB6527.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 yCeIxKJvoCBsXkFJpX8b4VoSbsDD4iotV59vXoU+y/6KK+GnV7JWrHaZDZPT4+W2qFGwUuqx+wvbNq6Ij9FAsYL9hrxqBWPWoD8hGJY0+nfCJP5WBpp4YppHve/xp0BjZRpo6toPCtyfvOHDJCSQRm0cyZa+M+v39pOIixZv2VqZY7g8NfezVePjS3MgGax6EUhlITllZI+z22kgSiHjDESEyubwGtNnWauyQqgd/H/VHWIJvuR199d+yPj9hSgmq4i+etvTuN1QMAFli5M5zcWabaCHmzenD/RjTPvaVhnsSOofofYq4dexFFNiQFQ4tLW3Uqe4qYUOI6FrRuevoPpSK92IaxvRC262rOCUxKgw9+hCingPatU3TPKiB4MrZXjVw0iPwslj4LtZGE6x7TgHjd7A3fkPS+4K+HKF+r2boF4Hi6yb1iKPWAiPoLvcsjOR7K5J8A9ngLgxKPDXxtkprOOpcryoNVeFwLA1DjJiQGeoO/6oDEnujjAzAaJOr3mJxw1pPG1Vng3DCrBJ2seRVRHEiJ+oIZxA99CTKFZw4DF33mOWFkw8yVVf1R3Nb3ZcMt/BO0Pg2GLXQf5dtYfJ0VoahhGEFRwc/U/2+j3oaVEBz3HX13WB95z+NAEynBgtkqEj/7wXE9R+v7BRhhoYn6/LeckKRycAUpqs5mxIg/41rK6zAvZynx8yKM2PeRTHfYj6v5OarpPcjLNXL5v1dnA8IMM2H1nZr/SRGM8CvbyCKeKFWitPZ5GKWnRL
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(4326008)(508600001)(2616005)(83380400001)(26005)(8676002)(2906002)(186003)(6916009)(36756003)(38070700005)(38100700002)(71200400001)(6512007)(54906003)(316002)(86362001)(33656002)(66446008)(53546011)(5660300002)(8936002)(6486002)(6506007)(76116006)(66556008)(64756008)(66476007)(91956017)(66946007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D88DD35C69378748AE4CB12AEC9C7C8C@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2497
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	835e2544-17b3-4a36-035c-08d97ea5d9bd
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yEM7lEf5FeeUVLj/63hFoV3A3SmxhcxjJE21alokEiIc9K1ZSiVV2xQvHcBVvK9ACbUgFDm4YAAwU9rf8zOrPI8X34nBx0U+8VhV7eEuxX/AOMO1+rT4+qdJCwyOrAqBPz3Y+sNUjpzzH1pNLqwJUeVgVWv3PHNs4gr7TBE4lzn3GNxMO8zpgNCrm+zerKjsvWF1T+9YmwDAjCFia5JCs4L5Vw08k+nStCpW3QQO+HwIudDqhHtZ6Yl5ERqpRXHEMyvRveJjcCcfT0ZqowSo4weYiJzYWKUNzTMhl+gYoLx9P4HwhaS6mh05KYWf3nMu8rBr5HgsSKBVikLeFCEyUSfmBlZIjAtly2OGQVnnvezhVhwGQAGjErzYBsylxWDstGYIft7VLvJ6KS1RUJV7aioRRX3KVzvYC2fCBagAohG+CGMzH/4mjszMXlnDePS9beuTChh7+dxUG7ti3kwD5fz80xMkDVaY1KFwZyn6LJIaSGLANQ/n3eUIepPLuwyRLvnw0yE0TdQmPitL6avL8l3Ev7eHGMHxl1Ho35GtfoPszBU4ReGoZrnrqRYUUEPpnoVzLK2Fzsx1+TI1Xont478Yfw/WafRavX1F5RyHoUoyE7HtR9Sa6g7OdAI4grWIalMPGhKmEUlWIy15WcA9h5GIIGyMHfG2r1D8Dek47ZrUWeuibhYVKb3Z/8Nu/yepgMJuJWBdM6TfZa8r3recwQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2906002)(54906003)(107886003)(336012)(186003)(82310400003)(8936002)(6486002)(81166007)(356005)(4326008)(6506007)(47076005)(8676002)(53546011)(36860700001)(83380400001)(36756003)(6862004)(508600001)(70206006)(70586007)(316002)(5660300002)(86362001)(26005)(6512007)(2616005)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:22:00.6558
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b25f4d2-b377-4b6e-7f2a-08d97ea5e3d3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6527

Hi Stefano,

> On 23 Sep 2021, at 3:52 am, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> If the property is not present in the device tree node for host bridge,
>> XEN while creating the dtb for hwdom will create this property and
>> assigns the already allocated segment to the host bridge
>> so that XEN and linux will have the same segment for the host bridges.
>>=20
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> Change in v2:
>> - Add linux,pci-domain only when pci-passthrough command line option is =
enabeld
>> ---
>> xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
>> xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
>> xen/include/asm-arm/pci.h          |  9 +++++++++
>> 3 files changed, 46 insertions(+)
>>=20
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 5eb83b12a1..83ab0d52cc 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d=
, struct kernel_info *kinfo,
>>             return res;
>>     }
>>=20
>> +    if ( pci_passthrough_enabled && dt_device_type_is_equal(node, "pci"=
) )
>=20
> Please use the accessor function is_pci_passthrough_enabled
Ack.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:22:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:22:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194446.346400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQYE-0007bj-Ph; Thu, 23 Sep 2021 15:22:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194446.346400; Thu, 23 Sep 2021 15:22:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQYE-0007ba-Lu; Thu, 23 Sep 2021 15:22:18 +0000
Received: by outflank-mailman (input) for mailman id 194446;
 Thu, 23 Sep 2021 15:22:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTQYD-0007WT-9s
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:22:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1310704d-45fa-4451-8c65-8a8254a335f9;
 Thu, 23 Sep 2021 15:22:14 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-9-KJ7ZvzHLMkuqtin62IXlRw-1; Thu, 23 Sep 2021 17:22:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2957.eurprd04.prod.outlook.com (2603:10a6:802:4::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:22:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 15:22:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0001.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Thu, 23 Sep 2021 15:22:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1310704d-45fa-4451-8c65-8a8254a335f9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632410533;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=v7Pl3SWeBURLTZQxDHrHpm8C1w3I54d7TkktwoukbbU=;
	b=XmKY2Qt3cJFwDEldNeHx6DPFjiC+6ym7dtFtRpyIVDaGnrDxe22mB8Y0XxO+9Ayo1JDZF5
	pgd/k3jlS4zHJXOhromzYisg5qJOxJd6sRmoI4XR8+xmPwQ3K7QAeD2EZXT6E/kBMyiyJ+
	PEflQdvbM1G88bQYHDT3UQpQLZFoxf4=
X-MC-Unique: KJ7ZvzHLMkuqtin62IXlRw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=K61WgJnFEqxShCxcPAFIQom1g3D4L9mCeViYzeJfbxbCuOM1dmY/WnRhKW+VaVzcrWXOguCjI2Kgi5plNHzmRgM+BUu6ZqHvzePaKRbSipgmAaUyt/pdSjGve0UCxFvSZLx5MagnuODOl0i//NxlCQHOwzCrw5YeubzEQyfHpo60jhs0X7KLXP9d7AFWbbk+cFdYb7u3vTo8noRH8p/mk48f/ruILtcJLicEILSoSR95TcWmOXVuI+zvVCl9GR83OqS3HtTXJnHGg3lycoT1m3MnA2Jwsc/CrvqlW6STZP7UI3RHH5WUdoy6zHELVBEgDsr0ZqjfJHQNbg8PFSXh4g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=S8F5spUXCIFyMGVa/FcabhfBFjtAKXUGA3Y0aDXtnNU=;
 b=EKfctUrY1SXXDK7jHadv0VkeIkdYPOI0uHTfvLlIjNh7HgFq8dJhkl4Xu533gurSdQlrHyEXJit/SIh/3s6002edqFrmnhk81wHAThRW49iQu6BvGyZyh7PDT2KLi2HJo8ZGpQXaz6nLB8JhCaF8bh56kzr5A9SPwaH2vjBWPpbeyIaVKm7BMWWE52w5bjv10dbFDNgP3OqtWqkmtk4sCOalcIYSxmhXwHfYDlZMe9UyvmqM/U0fRbYDNRgRZyV/TlEpxMmoot3/47S8f5Ge6m/V7fLu/VSkOYxgYK4XNmYQqHOZBJaG0nsIHPGLyKrzApZgJv2CbeVYE67rBwf1Og==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
 <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
 <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
 <33abd731-7c2e-8c9b-963a-fd446a21e87a@suse.com>
 <YUyaCrdAdpnn/o0K@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <959850f7-1d06-7dee-b9fe-0d28d068b8e4@suse.com>
Date: Thu, 23 Sep 2021 17:22:08 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUyaCrdAdpnn/o0K@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P195CA0001.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7357a8e9-c829-4d9e-9f9e-08d97ea5e90f
X-MS-TrafficTypeDiagnostic: VI1PR04MB2957:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB29574DF8ABB2A7E85148DD17B3A39@VI1PR04MB2957.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VtDoDP+Dciln7mF5+2qU8w9L0uPG94m89EpKy6ZW9QrTObAkGV+4SyCTVGDH0oodhF6Y0kYNxN4LpHfUNAA1zekTUe85YQQXtoYe/nsZ23eBtNL0CdLtvUmErvS2TW4AjGYBQtOBPdVoEXKH3mFTtlmE0yeo27vYNUPeRM4gGfBplaZ/m7+cm5kUZI18jBXmaUl10mzDI8rXQSTyoTt6KQyok0MTYsKfLyOzlTChUM87vrIxLhnVCweS99p9RVCRi6MsA90elHKeq2QqmOby94c+UUsslIvJX9bwSXJUzNhvdSKB85XAq9GZnrRgYLLnwW4ona5SmG37NmM9LcYEwbnRtZMG4wxJ9s7A62EQqfkh/o8jw09N9Nt0vKL4PNmxKT5XZhOj8o26rylbWldUErYvpC0fLy6+IQgKEBK5iT76A0Zh/sFDH5s1vhvAGHEc8261gHTUxUYP/a7E2I83/FVEz0HGEZUY3TP04HDLDrUsNWVytD054VRKDvVWR5IFLtYvFKY4moaNmGLLJIM6gtyoxhp08U6uvQ6WBnGB1qY+a4luNFaGiOLpF+sd1M9FY/Tz+Kr/89yDsbJOuFePYHYtkg5vS+FV/IkPdODFLKKx7zBXME7Cz4t0BB21+aLZcfgQ0tHLL9Y7e+aSdnndAobOclagptfN+VTK7aEhlZCvHSPHz5wr3b6wxeY3Kwfd8cLZIXgbV1uJr/aPZKWMyxohq1AiUL5hkcQRjPfGgV4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(86362001)(54906003)(508600001)(38100700002)(26005)(186003)(53546011)(2616005)(956004)(16576012)(2906002)(6486002)(66556008)(36756003)(8936002)(66946007)(8676002)(6916009)(5660300002)(66476007)(83380400001)(31686004)(31696002)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?IttuWFCkOS3muAKFW1Oc7uT1HjltB0YZB21bSXK0M+lm5+gjmjKezvF9pu6V?=
 =?us-ascii?Q?YwyMJB5Nwr7Y/TpiggjIJo41QeIIhvDsvZgkmLxHqIFTXZ4HH+q/pvYNCj3D?=
 =?us-ascii?Q?bOqSM+thK34TDH1GDuZ/OUtl40G722Cd81FwdINzRi00ixCshT7P5w7CLcdC?=
 =?us-ascii?Q?vnpYcO7D8LtriAZH2cthJF9Xhv3ROySRKHS6lCyZibcdDB8UNXTlmw1G/kPE?=
 =?us-ascii?Q?mtyvZ1rVnULyadkdkYg5DJ2XCqwC4GNbWvupOoTo0nbb6dwGTPbzNbodGFfv?=
 =?us-ascii?Q?zHcs7dtPV8659PnPD6Fs9nbRqTyOnAE5S9j5rpeTk3K8MpLTxm4H2Bovkzm1?=
 =?us-ascii?Q?ugrpiujUxEVQTzqiBT/KJZyfb1xJdRM1uIH4ygbhaGYl3pNKUtXKG5G3aSuS?=
 =?us-ascii?Q?tPCzzSzJ8YbIltnQXjzvvyrGwcusQsecYsc2DvuvyPFBHuewN5iE5Vs8oBvH?=
 =?us-ascii?Q?R6cM7N0DhPJIxP7z/ZOrXBHLd2o9a/QnUPoWFA3m+6uNcAv0s4/oLOAAlt4L?=
 =?us-ascii?Q?I5OpZxJN5I5ls7H7lnfjtxv5sXLjJtuMXWa/LGFpiT4py9IhpteHNz2t5Y21?=
 =?us-ascii?Q?e69L1bhlFpfr5DyvVBv1NLJZDwC36qzBR5D0CN8j99P1xR3PI6hASmXRXFTS?=
 =?us-ascii?Q?yLTfnh9S6yzzz3cm+67Cu2TY3mpONh8wy+FLxC0Gb0q/hGrl7vCNUIqCBt5q?=
 =?us-ascii?Q?lNyKEWbWA9Tej5gioXp8/Ht6l1AgY6M0LwkBiu9Fp8zckYRWnL2kG5T82D/z?=
 =?us-ascii?Q?q+p8+qU2NvnMgfdI0/FQ9Yi1jzQXVJ9Lgp+Jqn4NKihSz4CkGScKfOaBXJ8m?=
 =?us-ascii?Q?PGnrDIrPeTMdBDKVHsRHzs5P37JY+3faqLE0hr917vSRSI1Y8BG4fCIEnj3q?=
 =?us-ascii?Q?FpQUYclCcaIlCzy5BPWyrAQpw0OuVJZvrgO+GwqnHsdZ970PvgyUnrxhucex?=
 =?us-ascii?Q?cnACG1hLDNIZbh7uuxf0U9FF7zDWQUzNeHuqVRkAzGwyRxLVn0Mqo00GZFT+?=
 =?us-ascii?Q?Yo33Q/tB608zE3YxYaqwTs6gEljsplbC3qW7h5NlBfJzD71RQOBh92keCItZ?=
 =?us-ascii?Q?QutyqKwbIQm9y4AXGnv/blp6y9FMJfKScOqkNMvzW8mEVo89XeKLIprtN8W+?=
 =?us-ascii?Q?ezl/DaY7jqoQINSPtjAEOZe+8qmXCTDfTc63s/LkKMwEVWZjVi+mU+7yA+qK?=
 =?us-ascii?Q?RSlLTsFFDav/4GzxupVxEqHo63x6QUjuVzvyh6dKR7Y66/sk6yp8eKySC9p2?=
 =?us-ascii?Q?beHG9enrMdrR7YQwOTpC0O23psvvAbSu7j+6qQZHLj5WMTMPYJFGFzn31msA?=
 =?us-ascii?Q?z7lAVRWeEevCNwiNzu5mw49X?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7357a8e9-c829-4d9e-9f9e-08d97ea5e90f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:22:09.5384
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JGBQPRXTEGh3BtQ/AJ2yk7mhV6DzP/f8ZOeGPyeGloW9WykqT/pxT/ObC8MFfN4ikQ9YSDuc5tIgfjqtgZvdKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2957

On 23.09.2021 17:15, Roger Pau Monn=C3=A9 wrote:
> On Thu, Sep 23, 2021 at 02:15:25PM +0200, Jan Beulich wrote:
>> On 23.09.2021 13:54, Roger Pau Monn=C3=A9 wrote:
>>> On Thu, Sep 23, 2021 at 01:32:52PM +0200, Jan Beulich wrote:
>>>> On 23.09.2021 13:10, Roger Pau Monn=C3=A9 wrote:
>>>>> On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
>>>>>> --- a/xen/arch/x86/mm/p2m.c
>>>>>> +++ b/xen/arch/x86/mm/p2m.c
>>>>>> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
>>>>>>              return -EPERM;
>>>>>>          }
>>>>>> =20
>>>>>> +        /*
>>>>>> +         * Gross bodge, to go away again rather sooner than later:
>>>>>> +         *
>>>>>> +         * For MMIO allow access permissions to accumulate, but onl=
y for Dom0.
>>>>>> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() =
differ in
>>>>>> +         * the way they specify "access", this will allow the ultim=
ate result
>>>>>> +         * to be independent of the sequence of operations.
>>>>>
>>>>> Wouldn't it be better to 'fix' those operations so that they can work
>>>>> together?
>>>>
>>>> Yes, but then we should do this properly by removing all abuse of
>>>> p2m_access_t.
>>>
>>> I'm not sure I follow what that abuse is.
>>
>> As was clarified, p2m_access_t should be solely used by e.g.
>> introspection agents, who are then the entity responsible for
>> resolving the resulting faults. Any other uses (to e.g. merely
>> restrict permissions for other reasons) are really abuses.
>=20
> But some p2m types don't really have a fixed access tied to them, for
> example MMIO regions just inherit whatever is the default access for
> the domain, which makes all this look slightly weird. If the access
> should solely be used by introspection, then each type should have a
> fixed access tied to it?

I think so, yes. Hence e.g. p2m_ram_ro and p2m_grant_map_r{w,o}.

>> That
>> is, if e.g. a r/o direct-MMIO mapping is needed, this should not
>> be expressed as (p2m_mmio_direct, p2m_access_r) tuple, but would
>> require a distinct p2m_mmio_direct_ro type.
>=20
> I guess we would then require a p2m_mmio_direct_ro,
> p2m_mmio_direct_rwx and a p2m_mmio_direct_n at least, and ideally we
> would need to differentiate the mandatory regions as present in ACPI
> tables using yet different types.

What would we need p2m_mmio_direct_n for? And what's the (present,
not future) reason for the x in p2m_mmio_direct_rwx?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:25:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:25:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194456.346411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQbP-0008VW-9f; Thu, 23 Sep 2021 15:25:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194456.346411; Thu, 23 Sep 2021 15:25:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQbP-0008VP-5h; Thu, 23 Sep 2021 15:25:35 +0000
Received: by outflank-mailman (input) for mailman id 194456;
 Thu, 23 Sep 2021 15:25:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=q2Jl=ON=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTQbN-0008VJ-49
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:25:33 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cfb6bf79-9ac5-4a67-a859-21cf5f5b4e41;
 Thu, 23 Sep 2021 15:25:32 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6F8A622259;
 Thu, 23 Sep 2021 15:25:31 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3927313E6E;
 Thu, 23 Sep 2021 15:25:31 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id GFygDGucTGGrDgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 23 Sep 2021 15:25:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfb6bf79-9ac5-4a67-a859-21cf5f5b4e41
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632410731; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=4ej12isJcowNNLOgeaywRRKDnV7BrcTMYPQN4LE2jtM=;
	b=nk/pt2Cd+40MLiSy77vU9N0l6Hgyxkt9XZ6YBuEUBjCL/+lRzsPQKnh24s+RsgPJX+ufyX
	5Zyvx/LsIrJW5w4gLm2dTqGpP01Tt+s6VnezQ6lNtSUTk+dfhmRR+BFczXPPmUiJPz8H05
	4LG0d1lTw2IBepayEdBxxf5/TQIkxXw=
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
Message-ID: <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
Date: Thu, 23 Sep 2021 17:25:30 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="PFXOQuIsPnpM36gnvqC8IX13s63RGmP6X"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--PFXOQuIsPnpM36gnvqC8IX13s63RGmP6X
Content-Type: multipart/mixed; boundary="mUFqiPeflUgtMo9r5bhO1fdNfDpEYCym1";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
In-Reply-To: <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>

--mUFqiPeflUgtMo9r5bhO1fdNfDpEYCym1
Content-Type: multipart/mixed;
 boundary="------------C640D5E0E67FA74A447A863B"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------C640D5E0E67FA74A447A863B
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 17:19, Jan Beulich wrote:
> On 23.09.2021 17:15, Juergen Gross wrote:
>> On 23.09.21 17:10, Jan Beulich wrote:
>>> On 23.09.2021 16:59, Juergen Gross wrote:
>>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>>> This was effectively lost while dropping PVHv1 code. Move the funct=
ion
>>>>> and arrange for it to be called the same way as done in PV mode. Cl=
early
>>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_a=
d
>>>>> check that was recently removed, as that's a PV-only feature.
>>>>>
>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>
>>>>> --- a/arch/x86/xen/enlighten.c
>>>>> +++ b/arch/x86/xen/enlighten.c
>>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>>     	return ((per_cpu(xen_vcpu, cpu) =3D=3D NULL) ? -ENODEV : 0);
>>>>>     }
>>>>>    =20
>>>>> +void __init xen_banner(void)
>>>>> +{
>>>>> +	unsigned version =3D HYPERVISOR_xen_version(XENVER_version, NULL)=
;
>>>>> +	struct xen_extraversion extra;
>>>>
>>>> Please add a blank line here.
>>>
>>> Oops.
>>>
>>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>>> +
>>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
>>>>
>>>> Is this correct? I don't think the kernel needs to be paravirtualize=
d
>>>> with PVH (at least not to the same extend as for PV).
>>>
>>> What else do you suggest the message to say? Simply drop
>>> "paravirtualized"? To some extent it is applicable imo, further
>>> qualified by pv_info.name. And that's how it apparently was with
>>> PVHv1.
>>
>> The string could be selected depending on CONFIG_XEN_PV.
>=20
> Hmm, now I'm confused: Doesn't this setting control whether the kernel
> can run in PV mode? If so, that functionality being present should have=

> no effect on the functionality of the kernel when running in PVH mode.
> So what you suggest would end up in misleading information imo.

Hmm, yes, I mixed "paravirtualized" with "capable to run
paravirtualized".

So the string should depend on xen_pv_domain().


Juergen

--------------C640D5E0E67FA74A447A863B
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------C640D5E0E67FA74A447A863B--

--mUFqiPeflUgtMo9r5bhO1fdNfDpEYCym1--

--PFXOQuIsPnpM36gnvqC8IX13s63RGmP6X
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFMnGoFAwAAAAAACgkQsN6d1ii/Ey9z
uwf8C88vGxy+hk85cObqee7HTXy4o0t1QPe02mx9fEDsYrOnesJBQwrBX3BlYbUDMXUpsMxqMOgi
jcUTJIExSYGXCKmp8OS9+8bBxlO7kk1T+G7qDDgUqcZW6WJHY8BekoKTfzvk9oS3eW49KPrJyEwM
1udbzsYB9VNNUAklxFJ5k7kVR6YHVQpAfq+2wC2+UPffHjgpZW9kmRs8lS5PcTONWQCSvWY9mbth
aAzP6/ImzElQmRGrtJ02N82o5e+T38hc53wZW3uvOQvdJb22iSH2uzw+uRmo9cS1ZHuMpwyX0akw
Xrr01pTaSeTTDIFatD6AnAy4toV9nz3xX9jvY9BjCw==
=PcsT
-----END PGP SIGNATURE-----

--PFXOQuIsPnpM36gnvqC8IX13s63RGmP6X--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:32:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:32:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194461.346421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQhi-0001la-0L; Thu, 23 Sep 2021 15:32:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194461.346421; Thu, 23 Sep 2021 15:32:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQhh-0001lT-Ta; Thu, 23 Sep 2021 15:32:05 +0000
Received: by outflank-mailman (input) for mailman id 194461;
 Thu, 23 Sep 2021 15:32:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Uzs+=ON=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTQhh-0001lN-6U
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:32:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01735a8f-fa7d-4cff-809b-e25f27363241;
 Thu, 23 Sep 2021 15:32:04 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-N8twr9TiNfeJGpqMDpf8KA-1; Thu, 23 Sep 2021 17:32:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6863.eurprd04.prod.outlook.com (2603:10a6:803:12f::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 15:31:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.013; Thu, 23 Sep 2021
 15:31:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0035.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:31:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01735a8f-fa7d-4cff-809b-e25f27363241
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632411123;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9kXcq69D0bXdw3pbsqlftrpFQJqP372BXXSOWoCFpws=;
	b=TU9Iv1eaZhMcYC5WwTMSC/BQouaVGVApbihssSihtV91Vf8avuw2CG2YP7mWXMeZHeeYf2
	LKitMPS82sDRiM4k8qYjj0NobiScNUCv9ytv07IcL2mWuJ5LeuuUpCL8K6kXgcPoRS4g9h
	CFHhtJEm/ErqVqF7AEaRqYani77YRQI=
X-MC-Unique: N8twr9TiNfeJGpqMDpf8KA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NJyMLl9S4u2ce2KlGq/i+WrRjis1fO+Ebw3fJtmyMLNfRMgd55uc2Np/KNSIYqZZIILSNIgaNt2PMuZTC4m1QPN4N4SLVipPxudOT3XSnHgXT2eHs/jJE9DB+zHtC2EYvMGyM/5zRy9VfUDjOWYq+IPwPSX84ZfflgvAXhreQOfTiw8KyJyUwINBObapOfYUnEmXAl5Nhyu8/ITavzaitFJTF3X01OpdfhCUu8bd4ERuFN9EqSDftMdu0TNCyCEXCf4pNT1xu2G/7VfXY+bHVAPxezOKO2lIU2bbBY4d2HnUP+kKz7aGhXjavu/Qxwcw5MacsJmPfSdsVEaWdXrp0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9kXcq69D0bXdw3pbsqlftrpFQJqP372BXXSOWoCFpws=;
 b=G3X6L+wGTWRXPIn9J2uT2xsjoujoDFJwbeQZw2E2tTHCkjjY4M3m24Ph4o72EnKX5zaGXHKi2aRljbV00PAtG1tkMBEzs6QsW7WlyAfPSkrarAkNJVZskxgveawTC12Qs7yrZPcjhKQwaYAlDPdXALvnQDha7BA7bNDDJ/y1b2YckvQyFRw9gNlRb5nMDV4mlGTcxHsYijKcmI2x1y0Zd+Oo1GbqTP3phiokMLLbKXhnoqjRcUjc/R4VZmP11tykLZ2BUqAOG3Gg5C8ySjkqfQKt9LLnFOiK6/z3sBfWAx7qqjcVy2tpZj1hUXAOSEOQbe+xrj+NCN9ovC/JKjBJwg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
Date: Thu, 23 Sep 2021 17:31:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0035.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 416953b3-d528-477b-5289-08d97ea74868
X-MS-TrafficTypeDiagnostic: VI1PR04MB6863:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB68637095E69AFD9C149EDAC3B3A39@VI1PR04MB6863.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Eola3OnvjqlASQMzAKxKZClOYH4l2bZejwtY4wYxupTU3tO1BvP+Mv3+yVqv0UHglhBgdIccSV69d6o0xD7B7IvNBkK22fbBLvLov+z5lDHNq05CURSxDJFJka0XsjiXoeOs9qm5joOn6tGzhm4OH02ZL52OGGGo8Lf1W+TIuno8pZTxyzR8qqoTvMmtPg48DfdNsDGgIsfagtDe7u8xetVYBhEZrgkWOujoQ38X+BZQdjUW4GcSZt3fpdqXo8pwdE6mNKfq73AfIrjfcocGw1EWAqaWf33ZiZZNBjZVorPzHPNCtGbYnQ+dPuXQW8ktaIB0yQmQRCcOGWF+o5QsLqkfa367j6m6RkClJJte4V0FvNkqMZ0DZHiRm673/v9f52akoBSbE0/1OZgisVqfWU1yu2y0MewhPUOiheREeOOtvNYbzbCx2wc93Mmym/AuCUC5vL5VCRxP5jimK1ZSXlPEAv3P/8aOESmOT4qtLAMDQxsinY3Dw2qxPmFxtlDHuGpFXuTMyEjeuxUt5naSc8EajqXbJZRcqZfO0CJ4ed5KzPlbrJZiu3H+qBHXHIfFD9O333HBVxZEB/nin9SPd3d+XTz24uI+5N2Nc57MgANPIa0VO7lc54EE+eHUJtW+UXkGYQqhya8OeJI1tR6XL0ec8qn/xHuE59CAITsU8gzDSERZMjQPSLNNYPZrqUAzEPQJqRDHETOuEzacMNdun7oRZZDby8wE6vKcY7I9VFwpqUUtvVs5WFq7NPHBkB9f3DOTw7XR3SVE2f1JFLJRUZ6ZCANUFYaBer0GzAjWnlA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(956004)(86362001)(16576012)(508600001)(2616005)(186003)(2906002)(83380400001)(8936002)(4326008)(38100700002)(37006003)(316002)(5660300002)(31686004)(66946007)(31696002)(53546011)(6486002)(36756003)(54906003)(66476007)(26005)(66556008)(6862004)(6636002)(52103002)(158003001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VmU4TnZOTE00NXNrdEQyM1NvZnovR3JTZjgwbEFQZWFSWTc3bXRsSzhidWlz?=
 =?utf-8?B?SWxZMDlaTGk1KzdsZFh5WXlMdjBIWGZOZEFjSk1MZ2g1NDd6bHdNRmJBV0s2?=
 =?utf-8?B?Rzlkc2JXSTk5aWY5TEMrbVN2RExGKzVIQ3hoeC9PSTFnRU9va0V0aXlnNHFZ?=
 =?utf-8?B?ajFkZ3hlMXd6MjhDNjZxWlFjaCt6SmNObEd5RXBzMFFYalUvdkdtbFc5aDNp?=
 =?utf-8?B?K01tbVEzclJ3SVBxS2dpRkJsc1lBOHNBRDhYU1FWR0J4enNSNnZoV0MwcW1q?=
 =?utf-8?B?Q1JQNEhpZ2lSdmtnbTl2eTBNaFZiMVRKbk81QklsOTJrY3o1UG43V1NPMGNS?=
 =?utf-8?B?VHEwZ1ZvWE1jb05xQ1JGK3ZBRWYwaUpwRmIyQ1AvSkR2d0hub0toS1JvMHlV?=
 =?utf-8?B?R2RRSCtjNHlBSzB4T0RGN0RydVpobnBHSTNFVGJBb0ZVYXFlT3ZMVVBBK21k?=
 =?utf-8?B?dVN3cytjdFp5RVpPMXIwZHB5ZkxSU3hFSWlDdmxRSVlKaVNWQ0NSMEMwc0xv?=
 =?utf-8?B?NldGMVNmNTlTd1lxUXVoeTBoYmc5SHFwM3ZoUmEzNGFLMWtjSWozaDlVc2J3?=
 =?utf-8?B?aC9GYnk0b0FKQm5PVWFmS2FIZnpqNnFqblh2VWdvbHhud2FTR3RQMUh6TTFR?=
 =?utf-8?B?UGs0dnY0NFJoTnVDdHBWa0dJQyt2a3JzOEJPTXNyRWpXVUF0cjB4K3NGMkVi?=
 =?utf-8?B?SFB3Zm8zOHBjaFFWczJUcFYzbUxLVEcwYUoxRXR0YkQxTkdtVHF4eWxROXFm?=
 =?utf-8?B?VHNaemFLSWlrNVk0SDRWRFN2MFdONThzMVJDTmxWNTdnUWZ4OFRZTzZlMDB2?=
 =?utf-8?B?MDN4a2tpelcwcVVFZEw3U0x3Y21Mcm52MHpWanA2YXlockYrcy9ZRlZmcGZu?=
 =?utf-8?B?N2FxdFp0aVNaZmI1NmdpUklYTmFSLzBranRLZGxPbS9KQk5Vb2d5T1pKN3pT?=
 =?utf-8?B?WktUcCs5bFZpZCtQcUFTNnc3alJqUnpaTnRvU0RKek5OQi9CTTFVMzEzQ24r?=
 =?utf-8?B?ZDFMR2hVbXNvaHcrRW1VSGNyc1h3M3pydllHVU5kZFRlNElWSEI1NktBUzhr?=
 =?utf-8?B?bTFCTFhuOExYbjJKL2VVNnMrM1NDZCtWbHg3V3pYVGhxQUNEd3F5SUV6UlIr?=
 =?utf-8?B?WFNrdFcvWmpXdVZzR0grZW5kUzJScjBZYnVoWW1HN1lmVmthR3lEektzZVlJ?=
 =?utf-8?B?bkNiMU1KQzFIMFNnQUMvWkt5c2VyeXI3eFV4Z0c1MEF3VldrenhGMGw5MWpK?=
 =?utf-8?B?RnU3eTFha3VRTjZaaFRqTE1UZEE0Z0xMSTg2UjBQR2VOdFJ4azl2ZFFhSFNO?=
 =?utf-8?B?SndveDl4Z2xmRGtFZXdlVDljdjNZSU9FUVFUc2lPS2hJSXpHZVJ5d1lhU0N4?=
 =?utf-8?B?OVluOTRmYUNhVFl3TUtUQ2ZvWTNSTVROTWxBRGpnbFN6USs0TzNqdDhnRVE5?=
 =?utf-8?B?Z0VMQ1lzb0RJaEp5Zmg3U0drVFpPVDRuZ2x6N0dlRW45YmJzUWswQitKTVNu?=
 =?utf-8?B?NWZOQTVQWVN0VnNLOTlxNVVTak1SSFNJSzF5OFlVM3RGUDROWGlWWi9TSkVB?=
 =?utf-8?B?LzBoVWl6VjAzWVgrYTJUUVdZUEEyeFdYZExDMS9YNFBhUnRVSC8zRlFvUlVQ?=
 =?utf-8?B?VHZsL2REN3lLR2lJaWt6SCtEUDNYbUx0RW5FZkVWU2NGbTlVSFRCcVVFRWZu?=
 =?utf-8?B?VWU2cEY5SjhjVFZ1ZGhqMEExVWt5Q0dYWmlZZGVUUUJra0NoaC81TWRKZUkr?=
 =?utf-8?Q?N2skWF7X/M1xfwoMvyk0TScRY+xX2COGtiQV7yD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 416953b3-d528-477b-5289-08d97ea74868
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:31:58.9939
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 188gB6kPrY2aEcMVPD2BV3o8uiFr/2HV5Kmev+AYknyl1GYtXsls50A7LqsuzoMnhCG+CdRbMWdmtIM8qGMwqw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6863

On 23.09.2021 17:25, Juergen Gross wrote:
> On 23.09.21 17:19, Jan Beulich wrote:
>> On 23.09.2021 17:15, Juergen Gross wrote:
>>> On 23.09.21 17:10, Jan Beulich wrote:
>>>> On 23.09.2021 16:59, Juergen Gross wrote:
>>>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>>>> This was effectively lost while dropping PVHv1 code. Move the function
>>>>>> and arrange for it to be called the same way as done in PV mode. Clearly
>>>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
>>>>>> check that was recently removed, as that's a PV-only feature.
>>>>>>
>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>
>>>>>> --- a/arch/x86/xen/enlighten.c
>>>>>> +++ b/arch/x86/xen/enlighten.c
>>>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>>>     	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
>>>>>>     }
>>>>>>     
>>>>>> +void __init xen_banner(void)
>>>>>> +{
>>>>>> +	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
>>>>>> +	struct xen_extraversion extra;
>>>>>
>>>>> Please add a blank line here.
>>>>
>>>> Oops.
>>>>
>>>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>>>> +
>>>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
>>>>>
>>>>> Is this correct? I don't think the kernel needs to be paravirtualized
>>>>> with PVH (at least not to the same extend as for PV).
>>>>
>>>> What else do you suggest the message to say? Simply drop
>>>> "paravirtualized"? To some extent it is applicable imo, further
>>>> qualified by pv_info.name. And that's how it apparently was with
>>>> PVHv1.
>>>
>>> The string could be selected depending on CONFIG_XEN_PV.
>>
>> Hmm, now I'm confused: Doesn't this setting control whether the kernel
>> can run in PV mode? If so, that functionality being present should have
>> no effect on the functionality of the kernel when running in PVH mode.
>> So what you suggest would end up in misleading information imo.
> 
> Hmm, yes, I mixed "paravirtualized" with "capable to run
> paravirtualized".
> 
> So the string should depend on xen_pv_domain().

But that's already expressed by pv_info.name then being "Xen PV".

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:32:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:32:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194466.346433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQiS-0002O6-EC; Thu, 23 Sep 2021 15:32:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194466.346433; Thu, 23 Sep 2021 15:32:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQiS-0002Nz-9x; Thu, 23 Sep 2021 15:32:52 +0000
Received: by outflank-mailman (input) for mailman id 194466;
 Thu, 23 Sep 2021 15:32:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qmWf=ON=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTQiQ-0002Np-Tz
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:32:51 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0b887012-d133-440d-8d37-48b6af99417a;
 Thu, 23 Sep 2021 15:32:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b887012-d133-440d-8d37-48b6af99417a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632411169;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=mRYhV/YxFFDWgN2paOw5bft5dY4LTyWLa0EQtr5zxJg=;
  b=QH0cazgI7wFm39SlCvzyHIEZgqIwRR2om950BVajZyiRE1vAe9l22N1d
   ak+zk6sbk81pi2oIxWAziNPUidmRkcHwLf8HjvfLjt/K1PzfA0c+Ea9rP
   8SyQ1Ygylc4p9s+0tsfkio8UoOf3KqXoTmGZSXbhfOQWNfezLTdcqEI+P
   E=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 6wUaDRBw9jS01BN8Po48pkde8pPtEGawwOJK7nIdLKya1KfWtZ3LFY/qTAsESwDGwo63G8PYAj
 yIVhQlCzyykIQmJI0W0UBf/nNeijQYwQ5QJzaXfvh7olR+TCVYrGZfl1ZFAt0/oDpMnp06JKhv
 xw/AKTjOJZYWWCnndwPl6zC4ZAx/jsg5ciRrAf7lM7QN850DIfXogHDszWrXDY3O51hq60WfTS
 lP3myj6Kq1DwNZXmsHolHkfCr7pelHzMWgeh/T+9rZeggMLdVUxhCQLvqplB8LOle+FZeiZOyr
 n7+W9p1RC6kTWkf8Jg3+kR4a
X-SBRS: 5.1
X-MesageID: 53448890
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KqBYBq3TlRf24cPxrfbD5Q12kn2cJEfYwER7XKvMYLTBsI5bpz0Hz
 GVODT3QO/qLYzD1copzOd638E5QvMKBn9dhTAY6pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbVh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhz/dI1
 fQdjMWKWAoFG7OdmuEhbxZ5DHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQEgGdg15gm8fD2Z
 uA4azlsagn6PTp+HwZMN801jPr2vyyqG9FfgA3M/vdmi4TJ9yRu1JD9PdyTfcaFLe1Fk0Ddq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PL+y++NugVaT7ncOExBQXly+ycRVkWbnBYgZc
 RZNvHNz8+5iryRHU+URQTXoh0eZn04lAuBBHsEf9lCPz5bo7CeGUz1soiF6VPQqs8o/RDoP3
 1CPns/0CTEHjIB5WU5x5Z/P8mjvY3Z9wXsqIHZeFFpYv4CLTJQb00qXJuuPBpJZmTEc9dvY+
 DmMsCF2rLEal8djO06TrA2f3mrESnQkSGcICuTrsoCNslgRiG2NPdXABb3nARBodt3xor6p5
 iRspiRmxLpSZaxhbQTUKAn3IJmn5uyeLBrXikN1Ep8q+lyFoiD4J9sLuGkleh04Y67onAMFh
 meJ4mu9A7cJYBOXgVJfOdrtW6zGM4C6fTgaahwkRoUXOcUgHON21CpveVSRzwjQfLsEy8kC1
 WOgWZ/0Vx4yUP0/pBLvHrt1+eJ7l0gWmDKILbimnkvP7FZrTCPMIVvzGADVNb5RAWLtiFi9z
 uuzwOPQkEgADL2iPHOHmWPRRHhTRUUG6VnNg5U/XsaIIxZ8GXFnDPnUwLg7fJdikbgTneDNl
 kxRkGcDoLYmrXGYewiMdF55b7bjAcR2oX4hZHR+Nle0wXkzJ42o6f5HJZcweLAm8s1lzOJ1E
 KZZK5nRXKwXR2SV4SkZYLn8sJdmKEahizWRMnf3ezM4ZZNhGVDEo4e2Ygv1+SASJSOrrs9i8
 aa43wbWTMNbFQRvBcrbcty1yFa1sSRPke5+RRKQcNJSZF/t4M5hLCmo1q07JMQFKBPiwDqG1
 lnJXUdE9LeV+4JsqYvHn6GJqYutAtBSJEsCEjmJ96uyOAnb4nGnnd1KXtGXcG2PT2jz4qijO
 7lYlqmuLP0dkV9WmINgCLI3n7km7t7iqrIGnARpGHLHMwaiBr96eyTU2MBOsutGx6NDuBvwU
 UWKo4EINbKMMcLjMVgQOAt6MbjTiaBKwmHfvaYvPUH3xC5r577WA0xdMi6FhDFZMLYoYpgux
 v0suZJO5gGy4vbw3g1qUsyAG7ywE0E9
IronPort-HdrOrdr: A9a23:/Wh1gK87Pv5+oL65Xgxuk+HIdr1zdoMgy1knxilNoENuHfBwxv
 rDoB1E73LJYW4qKQodcdDpAtjkfZquz+8O3WBxB8bsYOCCggWVxe5ZnPLfKlHbak7DH41mpO
 ldmspFeaXN5DFB5K6QimjZLz9K+qjizEncv5a5854bd3AMV0gP1XYaNi+rVmlNACVWD5swE5
 SRouBdoSC7RHgRZsOnQlEYQunqvbTw5dLbSC9DIyRixBiFjDuu5rK/OQOfxA0iXzRGxqpn2X
 TZkjb++r6ov5iAu1zhPi7ontZrcenau59+7f+3+48ow/LX+0CVjbFaKvi/VfYO0biSARgR4Y
 HxSlwbTrlOAjvqDx2ISF3Wqkjd+Qdr0mTlz1CAh3vlvIjWeBIWYvAx3r5xQ1/h8Ewns8h70K
 VXm0Sjl7QSIy/hsU3GloL1vzcDrDvpnZPnq59Ps5UXa/puVFdcwLZvg399CosPEi7h9YwrJu
 FyEcnX5fJbdk6tdXzCpGlox+qtUx0Ib2m7a1lHtcqP3zdMmndli0Me2cwEh38FsIkwUp9e+o
 3/Q+5VfZx1P4crhJhGdaw8qAqMexjwaAOJNHjXLUXsFakBNX6Io5nr4K8t7OXvfJAT1pM9lJ
 nITVsd7AcJCg7TINzL2IcO/gHGQW27UziowsZC54Jhsrm5QLbwKyWMRF0njsPlqfQCBc/QXe
 q1JfttcrLeBHqrHZwM0xz1WpFUJ3VbWMoJuswjU1bLuc7PIp2CjJ2TTB8SHsuaLd8AYBKLPp
 IuZkmBGCxw1DHdZpajummgZ5rEQD2Mwa5N
X-IronPort-AV: E=Sophos;i="5.85,316,1624334400"; 
   d="scan'208";a="53448890"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=O0GzuGB+4YLxoZRtukIlmPVu1ArzaH2pIAPm0kneKxwvAZ4JL4gnB1P5938JCy6X0pvSZZ6u92xamCgC4Z+scSNnuS0ZfwK9x9xL6MOS+5l9Tl8nozlQ+DyXx97t22kBXFZj1KfB04eLUWV7TAo8vVt2tS3MnLpo+tRABCr7WYoWAwF0uW8gbFvjYq1EHltWce3zMSx3L3EVp5dH8xlAT7qDY3ItHH7arpfI3AxMN/Bmr53Np4KW95Q2LXllK4waDjZe2OH/2Y8QSJNLfsjXBgvdfDA25nHZwCdLcdSXqWG2G99cMAXM/vFHsDgtUGknlnx0FdiFv3jyTMgD9AdHrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GNggsnDnIbmEeToVRGIezV+HEFw+Kkt3ctCENGZ+mME=;
 b=asW/NMPOKpzYvvcVqCjtcUhyY5ouvMipwpqK/MZZ9wy7GJ3WtUD1J9rytyC4i350bPgI8MvesQ/LMXI+wBd8yP+SIEWGi1XhZrB8WbQZmNLUWouyJ4MSW4AI0nAs8IdDQaeLg/raC7HKY/nDCl8lwDinjv2dGU5sBa7L5NB0J5vGJjOY/MaozcU1NB+iS+0DO5ei3hOMHWsl8MZJ+f4j2ldFGyxnSNBeoS9ckr8chxsHU4lEzKAYjMfZhSJctaxkQX/xWTk87GqW4yVQWsYjvkZFzmjeQv8bAjW00Lb2ySFmBMhHPGV6dWYhsa+6GgoGmcD8N55lcSBIshrS0ZC+rg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GNggsnDnIbmEeToVRGIezV+HEFw+Kkt3ctCENGZ+mME=;
 b=gOhQOERWUmUtZldYu7KQvKrCQCeoplWPKLLAfVkZlBOjZ6tf5gOf5IlpQGzPa9KMb6Dztwvcl8u9kTQgkJj+A5nlGpNdQFkgxvcYV4nEb8Lixv6xO3KIS/UKFSbhniAi+u04oa5TRoIovNXZyrnPa1rT8TppaLmJr/plBukT/ko=
Date: Thu, 23 Sep 2021 17:32:41 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>
Subject: Re: [PATCH v3 9/9] x86/P2M: relax permissions of PVH Dom0's MMIO
 entries
Message-ID: <YUyeGYFMdeiPFHV3@MacBook-Air-de-Roger.local>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <a141e1a5-0185-4923-a91e-68c06a4f78cf@suse.com>
 <YUxgsF47nrTn8WrS@MacBook-Air-de-Roger.local>
 <ec456bf7-df78-59f2-6c14-5bda165125c8@suse.com>
 <YUxrE/kx8wmftVIX@MacBook-Air-de-Roger.local>
 <33abd731-7c2e-8c9b-963a-fd446a21e87a@suse.com>
 <YUyaCrdAdpnn/o0K@MacBook-Air-de-Roger.local>
 <959850f7-1d06-7dee-b9fe-0d28d068b8e4@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <959850f7-1d06-7dee-b9fe-0d28d068b8e4@suse.com>
X-ClientProxiedBy: LO4P123CA0452.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1aa::7) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 42b2d216-d033-4edd-1960-08d97ea7645d
X-MS-TrafficTypeDiagnostic: DS7PR03MB5606:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5606F2A4B09FC1FD2C98C6D58FA39@DS7PR03MB5606.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: +E49BChgq53ZqetzIOHdaB5lDikflxelQ+Ujd9zEuNvS5W/RX0PrR3GYCwH6ggIidp2qQQ3V4wenJpO2DRpdjrNNFrm3DJm7MN7Tu6W+SgXjmKJqewoRP9LBqCD1ehN/Uhd5MF+85G/lHCPhy963Qvi0YsH7sgDDIMereCOueK9md/R1I6NCA+bdahX4wTkTgIjh9SXIna0A5AzLA6iPkTbtK7iz68Dy67N/aF6ZrzkmiimJyd5P1L/PEV8fiAVzvwYQJkhJgfMVTjExpmDcp7i8vcsLpoIJTeTRcfbCD00wZ+EAJ7mSekhWAlMTguqQtaoPix4fJwKedRLNE8nDNRyM2ieecEaRkeTAmO/AXhx+8VeeC8ybXLMAeklVc4bfxY8b5M/QxMHW5eNPt+6Sppsn+5UBa5pH370HmFnCV6+mnVo4cgiZov12tDPec0K2X+h9BrgxF7TIMu/F7NTzVgXCFEYxtwTdHWL04EmL0cXAlPb6b1DtF5ble3h7h7+dBuM+Qh8AAdtAA/+JWespEBLTC6xHmu9tjNk52SHt8D8WeyYdVOURySvT0Oweyqp34z/UyeupceYeWr3srvaHOyrLF7nfhzfSB3NwjxtP4WqMoiXyyRx4wijNeF1beIJKhBp6x6dC1Gl2ZAS5UXU7Sw==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(38100700002)(83380400001)(26005)(53546011)(8936002)(6666004)(54906003)(316002)(6916009)(85182001)(86362001)(6486002)(6496006)(9686003)(66556008)(508600001)(66476007)(186003)(107886003)(8676002)(956004)(4326008)(2906002)(66946007);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OTg0YWRiaTZMMjFrdkk1WnQ3UzlWWkR3ai9LbVZYeDNLYXhOdTcxNDVoODln?=
 =?utf-8?B?NGJHOTFNTEtnTm54ZWQ5Q2VFcXNSZEF6RlVrT01qa2swM2k0aDhVeU9jMnB1?=
 =?utf-8?B?K0dSZDd5OTB0Tkcxc0w5dXBsdC9mbzh3Z0RYMGRGR1lTbzJ0aEpkL0gzWHpB?=
 =?utf-8?B?MUZacHkwSnBUcUxNZE8rRGZsQTVSTGJCWTFKa3A4ckc2N00xYVBkajNHQzRm?=
 =?utf-8?B?MHlnaTl4MFBJNVNLTWp4V1dWdzJuQ05XRThwWFpEdDhhZlF4MXNseHpHaklv?=
 =?utf-8?B?VnR4Z0tQYVdtTlY3SkNwYVFzSlptOGZGd0doVWdwNXVXNGFOZ1o2VDRtNHZo?=
 =?utf-8?B?SHc0bi83MUttQjMzbWg3dktGalF1WjZidXhRbisvOTVrWFFNcUwySEthYWtX?=
 =?utf-8?B?S0swM1VaRXk0THE4cnF2QmtRYnNEbmc3UnlmNEsrcjc0UzFTUEI1UGZaSkdI?=
 =?utf-8?B?NUN4L0FlazZtNUp2d2p3U0czOTFWN0NvK2NIeFo4MHJRbUtpd09sZjY1a2Ez?=
 =?utf-8?B?Nmt4QkxQUHNkV25PRU4xMDVCUVByU0EwazcvWXptdXNJVkk0Q1JCWFZpWEJ5?=
 =?utf-8?B?VXBWMzF0RnY4R1RQRzJpM1pzZTRpYS81Smh5YkZDTHZSZVhhTnFsdlcxYVY3?=
 =?utf-8?B?RmpHWHFvWHlSenJHZWpQVnczUEx6Vy9xcWdtY3k4dVRTWHp2bURiT3V3bHpo?=
 =?utf-8?B?bUFKZnJzWXBHbXUvanRTcjhteHhmd0VWTlZneFN2RXNJVHg0N2ZtdVdrS0JE?=
 =?utf-8?B?UUY5aWdJT1ZJUER4bEJCYkxBRUMrUCtuOHpNMURKaGptM1A5OVRNL2dObWtM?=
 =?utf-8?B?T1lRL1BJS2hSdHpOam1mb0ZsY1gxcHMrbEVUZkxSaUZrSWgzNEZPODIySXpM?=
 =?utf-8?B?eVdUaExUaVJTSmk2NEV3MUUxWEdFVEdJYkxGb0J0ejZpZ1gvSmlJTnRJOFFy?=
 =?utf-8?B?eXlJOXQ4aWdQTC9qV0tQRG1DeG1OcC9BWEpjUHVheTl5VGxoSHJZdkQwRnlV?=
 =?utf-8?B?OUNxcHFUeWY0bWxFeE9XUmdJWVZFc0lzcUhHYit2a21nZmFCVXlyVjBpTVc0?=
 =?utf-8?B?cWY1RUFCWjVzeGwxNnZaZUFSelJNY2dQZUprdlMxZWdMUW9IWDN0TTR6OXdJ?=
 =?utf-8?B?QjFaZERFR1lvcTAwYjhkZ1luNFpvZ0Y2UUFqWHg3MlNlSUgxUmxiWHJhMUJ5?=
 =?utf-8?B?UXNML3FULzEwMERySDNsNW9Kd291WDJVdlZHcW5DT2R3VzlQRzdpWUJ4TVJr?=
 =?utf-8?B?K1N1WW9UYTJVOGtlSlZhcm1TdW9JNDE5bXB5RTZJWm43NkRENjB4dSttcHl3?=
 =?utf-8?B?SUJSeGsvQ0w2bDFvYTQ3ZTRuc3RKanZGZGVWdkM4cnNSU0d5VmZVOHgwL3g3?=
 =?utf-8?B?VDFNNU5vR0I1UWVLQ2RVQXFuQ3FsT1I5QkRadVNySEMvbXR4bmh3cVRTcW5X?=
 =?utf-8?B?dWpjak9LQ3hnZ0k0UXdtWmFXTkt1L2hxMmFrazdlNElHcWNJMlBHNk9ZQVB1?=
 =?utf-8?B?VnlSY1k3cTVFTVl4Z2xpMEtMQk5aUmwvTnFFZlM5UHc3YzJJcHlJK2xKVEFY?=
 =?utf-8?B?dzk4YzduT1JiQ21rQlhQcFgyQ285d0Fmd2pNUjBzaDJpZmJZdzYxNC9SdWpM?=
 =?utf-8?B?ajBnd24yLzlCTXAyVEVNd3liQU01c1dwR0lwNDNVKzNPR2tYRlVCZXgxbFJL?=
 =?utf-8?B?eXdTRHE1UVZwMi9YK3BrV3JWcHEzclpXNEtjdUR5eHVSSEpzMW40c3pTN0tm?=
 =?utf-8?Q?g0PBu34M2cXfN6t98FiRBRFLZ2rnbDtcZ/IeWQ0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 42b2d216-d033-4edd-1960-08d97ea7645d
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:32:46.0128
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DLuEn5+Dj5DPBZYZkmm4aSh+jS9mBAPvTYYxbk7e0l+m6nBahvxYJKEASLruPH0IA0NqAsCXBDDb7nyAcIUIFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5606
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 05:22:08PM +0200, Jan Beulich wrote:
> On 23.09.2021 17:15, Roger Pau Monné wrote:
> > On Thu, Sep 23, 2021 at 02:15:25PM +0200, Jan Beulich wrote:
> >> On 23.09.2021 13:54, Roger Pau Monné wrote:
> >>> On Thu, Sep 23, 2021 at 01:32:52PM +0200, Jan Beulich wrote:
> >>>> On 23.09.2021 13:10, Roger Pau Monné wrote:
> >>>>> On Tue, Sep 21, 2021 at 09:21:11AM +0200, Jan Beulich wrote:
> >>>>>> --- a/xen/arch/x86/mm/p2m.c
> >>>>>> +++ b/xen/arch/x86/mm/p2m.c
> >>>>>> @@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
> >>>>>>              return -EPERM;
> >>>>>>          }
> >>>>>>  
> >>>>>> +        /*
> >>>>>> +         * Gross bodge, to go away again rather sooner than later:
> >>>>>> +         *
> >>>>>> +         * For MMIO allow access permissions to accumulate, but only for Dom0.
> >>>>>> +         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
> >>>>>> +         * the way they specify "access", this will allow the ultimate result
> >>>>>> +         * to be independent of the sequence of operations.
> >>>>>
> >>>>> Wouldn't it be better to 'fix' those operations so that they can work
> >>>>> together?
> >>>>
> >>>> Yes, but then we should do this properly by removing all abuse of
> >>>> p2m_access_t.
> >>>
> >>> I'm not sure I follow what that abuse is.
> >>
> >> As was clarified, p2m_access_t should be solely used by e.g.
> >> introspection agents, who are then the entity responsible for
> >> resolving the resulting faults. Any other uses (to e.g. merely
> >> restrict permissions for other reasons) are really abuses.
> > 
> > But some p2m types don't really have a fixed access tied to them, for
> > example MMIO regions just inherit whatever is the default access for
> > the domain, which makes all this look slightly weird. If the access
> > should solely be used by introspection, then each type should have a
> > fixed access tied to it?
> 
> I think so, yes. Hence e.g. p2m_ram_ro and p2m_grant_map_r{w,o}.
> 
> >> That
> >> is, if e.g. a r/o direct-MMIO mapping is needed, this should not
> >> be expressed as (p2m_mmio_direct, p2m_access_r) tuple, but would
> >> require a distinct p2m_mmio_direct_ro type.
> > 
> > I guess we would then require a p2m_mmio_direct_ro,
> > p2m_mmio_direct_rwx and a p2m_mmio_direct_n at least, and ideally we
> > would need to differentiate the mandatory regions as present in ACPI
> > tables using yet different types.
> 
> What would we need p2m_mmio_direct_n for?

AMD can specify no access at all for certain regions on the ACPI
tables from what I've read on the manual (IW = IR = 0 in IVMD Flags).
AFAICT amd_iommu_reserve_domain_unity_map can already call
iommu_identity_mapping with access p2m_access_n and that would get
propagated into set_identity_p2m_entry.

> And what's the (present,
> not future) reason for the x in p2m_mmio_direct_rwx?

Mapped ROM BARs, but I'm also unsure we shouldn't just map MMIO with
execute permissions by default unless stated otherwise.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:35:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:35:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194471.346444 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQkw-00034d-Ra; Thu, 23 Sep 2021 15:35:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194471.346444; Thu, 23 Sep 2021 15:35:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQkw-00034W-OI; Thu, 23 Sep 2021 15:35:26 +0000
Received: by outflank-mailman (input) for mailman id 194471;
 Thu, 23 Sep 2021 15:35:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTQkv-00034Q-K9
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:35:25 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::611])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dac96d1b-d3ad-41c2-a838-5b68ca670f75;
 Thu, 23 Sep 2021 15:35:23 +0000 (UTC)
Received: from DB6PR07CA0188.eurprd07.prod.outlook.com (2603:10a6:6:42::18) by
 PR2PR08MB4843.eurprd08.prod.outlook.com (2603:10a6:101:24::12) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Thu, 23 Sep 2021 15:35:11 +0000
Received: from DB5EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:42:cafe::fd) by DB6PR07CA0188.outlook.office365.com
 (2603:10a6:6:42::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.8 via Frontend
 Transport; Thu, 23 Sep 2021 15:35:11 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT048.mail.protection.outlook.com (10.152.21.28) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 15:35:11 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 23 Sep 2021 15:34:58 +0000
Received: from 69973995d261.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1CC564E7-C643-47EA-8DA5-314F453E446C.1; 
 Thu, 23 Sep 2021 15:34:47 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 69973995d261.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 15:34:47 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4740.eurprd08.prod.outlook.com (2603:10a6:20b:ca::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 23 Sep
 2021 15:34:43 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 15:34:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dac96d1b-d3ad-41c2-a838-5b68ca670f75
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=A/03ZlYeiuZl+tcAoBuDABEvaxPmLRAJSWQsALpm6y4=;
 b=q6jNCo6vurpQF5Hh3QA3C4stN6TESfgAiKbjsh+w/H7VJ157ujhU8ZH20cQdgJ7KjasKuHB/BZmF3SXUpY99U+LdfNaKuutFpeDM1ZnHh9yVd9KclfhH7bYlQzESwP8StOj6SYGb57iILmJ2u3h204769OEJjA2N4fmSjmPOcrU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: a7e56fdf2677298a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=emTSOzV1l6TVCT5gteYxASzwuyXNSwKsZKColKkF2+tyM25tPYH5pyTmvDDPToLU4Jkv8oz8+36twqTEJPpUi2LWIIBv5ul2j2qphUFHqKIKTFt9gl/wswlcsorOLcLWIrGWshVpW6Z6IF7tyD9TYH4mtmEx0c/2EI4HhbDilKpz2Qz4f5OsObOgzhaO1ToNbk+EdUsitwTkZKWTOtU7kzt9+2PhaV00Gj3KIsi8oiW7BG7U24gFICDAOlSAGL3qqwj5azK9OarNTUX7rMjDw2UVUF7fx4JHP9KK42nCWvG3CTL8yIA1cRjBvAM9rKx0GporT7MnJ0Fx/O+3DnwgDg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=A/03ZlYeiuZl+tcAoBuDABEvaxPmLRAJSWQsALpm6y4=;
 b=ez+/AQPyLfKuign3ePRmZL9t8cOIsfL582jedsLRg92tUijmvU3DlQGDshyzt4rqBH8xfxfZQtQDBby3LLL+wc1RWd97X03gPhzlfhU/ueisZOgWmM5gXPOeLOSZjwi2WS4UJG1VkxdYJT3lyrTGe/EXjedW+sLv5ROQYb5J54pDIcSZP5CEAvgq8rCw+COdXMAIyaW6z/Wd/lJQuj98CxuJQHtUJ2vBxYb2vUVaex7Ndl/yJW00f76WWSRwyVY2jXN4KmTAHBaiI9IhF6419A1BY3qTw4V0d7p2Ane1ZshQwxts8Bk33XpMZSy+sECcb7ON7M7NjkEuRDPYQGIzLw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=A/03ZlYeiuZl+tcAoBuDABEvaxPmLRAJSWQsALpm6y4=;
 b=q6jNCo6vurpQF5Hh3QA3C4stN6TESfgAiKbjsh+w/H7VJ157ujhU8ZH20cQdgJ7KjasKuHB/BZmF3SXUpY99U+LdfNaKuutFpeDM1ZnHh9yVd9KclfhH7bYlQzESwP8StOj6SYGb57iILmJ2u3h204769OEJjA2N4fmSjmPOcrU=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>, Paul Durrant <paul@xen.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXr6at03qE3GlVrUSsAw4u98X5EKuw6bQAgADYDAA=
Date: Thu, 23 Sep 2021 15:34:43 +0000
Message-ID: <438E9A7D-65A3-444C-A24E-87E3A9F92234@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4596307d-529f-402c-5985-08d97ea7bb35
x-ms-traffictypediagnostic: AM6PR08MB4740:|PR2PR08MB4843:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PR2PR08MB484373D6E41D42D8806CD772FCA39@PR2PR08MB4843.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 zQWQsN9Zed3FN86ONedOUjsu2U8Ha5NW9Sud6ABYm6qQZdIXzKZG5M+HzeWEYoNhNNRMgPhkusPL6o2alMM1LpwYA0HWpozGmA5M+sgd6NjsrE4AGQa2HNZpO3WbQf4buUm5scAl6QgQtyoO5o8KNdVHCht+XKw6ACHPSzJPOEhhyYwwFgOaTYXojhfjKCBXKBgW+vZOmVmxgC+vkifSVJ114RaDnH2Ed1UtYaHmIKIJ41uoLn2WYJ8057VMj7KICkdcTiGNR1O17nZXaz66VU20SPahjEdRJuxT6oD47QMwLtH92Dl38AK5Tm1gaCr1KYOWiTpRb6i4rpIPM0Os4qBGJNsxmaV1QHb0tOgl/Yd6XFT/eHKHZ3P+uJXIpaic8h8hl4GmB2BOzDgNz/RmcDxbJ4JFK+lwqwFyTsYxdfB95UZYiGqMcfKT4k0v71aVeDV8jtF1tjvzMJQcLcfmhhAvzHSzEYFg2pm41cqicZ84RVFt18L4EPqZrGU5/mRLoxkQNR4WQeVpMeaNzE5v3eCcsRQGDSLt663F54UCKkfDqO2J/ux28aQKfA9saU9sWk+p8XZ3IjmiGpuThH8nStOfALMGShisPimL2S8ZYca99Fo1vNryUTBZ5hLEmUOkcjCZtvicZ2qZC+T09+zEtLL6J1SFEajQ+7JPRobrwa3CUSr3rvS/jFhOuT/3vZi5e4GsZHV41YrjIwcXqJmNMEIj1ZyMyfDCw8JRol98/Ljc7tWa5JUf6u/gYQnxUcWR7JARBnIfPQqQCGbZj6u1bw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(76116006)(91956017)(38070700005)(53546011)(6506007)(86362001)(316002)(26005)(6486002)(64756008)(66556008)(66476007)(36756003)(66446008)(66946007)(38100700002)(186003)(508600001)(33656002)(7416002)(54906003)(6916009)(8676002)(4326008)(122000001)(2906002)(83380400001)(71200400001)(5660300002)(6512007)(2616005)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <0CC7753ED3B4134E9F72A5C97E894270@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4740
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e2bdf5be-becc-42ff-b75f-08d97ea7aa9a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EfdZCxU5nrDuzhqd1AJQ7tbbhRWoa2DRs22BSk1a4aCrXG/QU5w+Tm+ZIYZK7Tkz/Gb+FOZQ8PgpryhUAaBkvWttrssEvmOGd+JyPt8VSvXFzKVfZWj/gEba5Hkzw0uv8Ig3bObNBJzLdOYjzAiSZFU0n42cSKjbocRea3Ka6znYbU6xOclmSgKta4np3I+xCQPGpzHp3ZS9oWg/YX0OoNNfwNdI8lCkHMqSIeCOzv3lolJuhWsrBhEKeR4Q3l4wPHFz1qt9iUbDM1mtc+yw7ab4Q7R/wkQQg4ev4SBzFqHT2dPA2HKA5SAH6JTNYEOp53FK4EjdAhe8/hoho2Hq783d3vy+2qEkAkfJkVEQdO2LqG5kaEfsD/ur6gCIaavxZj2ZzF8eXzqwdfazRF7st32VhA7WicbTZmJOsc3+qTdfiJt8UGPoHh+3pvCzDw+w/y+dxiRsA7tPMavpFKSGOc6nYl7StLl00VeNFocoZW++nNmqTrsUWqNhug4DNeul7eXjtN1/VD/N6qCw7FD+gzyRFwuxeiTv9pSjcEmM/e5gZG+ET41RCyBM3Z+lJldNOjxK9KyLPt+5wTpU1zoQ/+ldc5MIPS/3PjRyEA0TO82neLY4Lm2PmJUtV5lfzEwNoAI9uGvQky+BwLI54fH2ShkOikeaigmaADjDK7aYMYwP5Zqsve9O5xUeltdWS+ZFfT7JjZ9O6Gh6MF7cyisqa2fIBg6eLqZZnXI3yDGqb+U=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6486002)(6512007)(86362001)(356005)(36860700001)(26005)(36756003)(30864003)(5660300002)(2616005)(54906003)(53546011)(6506007)(336012)(2906002)(8676002)(83380400001)(107886003)(4326008)(508600001)(186003)(81166007)(6862004)(47076005)(8936002)(70586007)(70206006)(33656002)(82310400003)(316002)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:35:11.5029
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4596307d-529f-402c-5985-08d97ea7bb35
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4843

SGkgU3RlZmFubywNCg0KPiBPbiAyMyBTZXAgMjAyMSwgYXQgMzo0MSBhbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFdlZCwgMjIg
U2VwIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4gVGhlIGV4aXN0aW5nIFZQQ0kgc3VwcG9y
dCBhdmFpbGFibGUgZm9yIFg4NiBpcyBhZGFwdGVkIGZvciBBcm0uDQo+PiBXaGVuIHRoZSBkZXZp
Y2UgaXMgYWRkZWQgdG8gWEVOIHZpYSB0aGUgaHlwZXIgY2FsbA0KPj4g4oCcUEhZU0RFVk9QX3Bj
aV9kZXZpY2VfYWRk4oCdLCBWUENJIGhhbmRsZXIgZm9yIHRoZSBjb25maWcgc3BhY2UNCj4+IGFj
Y2VzcyBpcyBhZGRlZCB0byB0aGUgWGVuIHRvIGVtdWxhdGUgdGhlIFBDSSBkZXZpY2VzIGNvbmZp
ZyBzcGFjZS4NCj4+IA0KPj4gQSBNTUlPIHRyYXAgaGFuZGxlciBmb3IgdGhlIFBDSSBFQ0FNIHNw
YWNlIGlzIHJlZ2lzdGVyZWQgaW4gWEVODQo+PiBzbyB0aGF0IHdoZW4gZ3Vlc3QgaXMgdHJ5aW5n
IHRvIGFjY2VzcyB0aGUgUENJIGNvbmZpZyBzcGFjZSxYRU4NCj4+IHdpbGwgdHJhcCB0aGUgYWNj
ZXNzIGFuZCBlbXVsYXRlIHJlYWQvd3JpdGUgdXNpbmcgdGhlIFZQQ0kgYW5kDQo+PiBub3QgdGhl
IHJlYWwgUENJIGhhcmR3YXJlLg0KPj4gDQo+PiBGb3IgRG9tMGxlc3Mgc3lzdGVtcyBzY2FuX3Bj
aV9kZXZpY2VzKCkgd291bGQgYmUgdXNlZCB0byBkaXNjb3ZlciB0aGUNCj4+IFBDSSBkZXZpY2Ug
aW4gWEVOIGFuZCBWUENJIGhhbmRsZXIgd2lsbCBiZSBhZGRlZCBkdXJpbmcgWEVOIGJvb3RzLg0K
Pj4gDQo+PiBTaWduZWQtb2ZmLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2luZ2hAYXJtLmNvbT4N
Cj4+IC0tLQ0KPj4gQ2hhbmdlIGluIHYyOg0KPj4gLSBBZGQgbmV3IFhFTl9ET01DVExfQ0RGX3Zw
Y2kgZmxhZw0KPj4gLSBtb2RpZnkgaGFzX3ZwY2koKSB0byBpbmNsdWRlIFhFTl9ET01DVExfQ0RG
X3ZwY2kNCj4+IC0gZW5hYmxlIHZwY2kgc3VwcG9ydCB3aGVuIHBjaS1wYXNzdGhvdWdoIG9wdGlv
biBpcyBlbmFibGVkLg0KPj4gLS0tDQo+PiB4ZW4vYXJjaC9hcm0vTWFrZWZpbGUgICAgICAgICB8
ICAgMSArDQo+PiB4ZW4vYXJjaC9hcm0vZG9tYWluLmMgICAgICAgICB8ICAgNiArLQ0KPj4geGVu
L2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jICAgfCAgIDMgKw0KPj4geGVuL2FyY2gvYXJtL3ZwY2ku
YyAgICAgICAgICAgfCAxMDIgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4g
eGVuL2FyY2gvYXJtL3ZwY2kuaCAgICAgICAgICAgfCAgMzYgKysrKysrKysrKysrDQo+PiB4ZW4v
Y29tbW9uL2RvbWFpbi5jICAgICAgICAgICB8ICAgMiArLQ0KPj4geGVuL2RyaXZlcnMvcGFzc3Ro
cm91Z2gvcGNpLmMgfCAgMTIgKysrKw0KPj4geGVuL2luY2x1ZGUvYXNtLWFybS9kb21haW4uaCAg
fCAgIDggKystDQo+PiB4ZW4vaW5jbHVkZS9hc20teDg2L3BjaS5oICAgICB8ICAgMiAtDQo+PiB4
ZW4vaW5jbHVkZS9wdWJsaWMvYXJjaC1hcm0uaCB8ICAgNyArKysNCj4+IHhlbi9pbmNsdWRlL3B1
YmxpYy9kb21jdGwuaCAgIHwgICA0ICstDQo+PiB4ZW4vaW5jbHVkZS94ZW4vcGNpLmggICAgICAg
ICB8ICAgMiArDQo+PiAxMiBmaWxlcyBjaGFuZ2VkLCAxNzkgaW5zZXJ0aW9ucygrKSwgNiBkZWxl
dGlvbnMoLSkNCj4+IGNyZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiBj
cmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3ZwY2kuaA0KPj4gDQo+PiBkaWZmIC0tZ2l0
IGEveGVuL2FyY2gvYXJtL01ha2VmaWxlIGIveGVuL2FyY2gvYXJtL01ha2VmaWxlDQo+PiBpbmRl
eCA0NGQ3Y2M4MWZhLi5mYjljOTc2ZWEyIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL01h
a2VmaWxlDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4+IEBAIC03LDYgKzcsNyBA
QCBpZm5lcSAoJChDT05GSUdfTk9fUExBVCkseSkNCj4+IG9iai15ICs9IHBsYXRmb3Jtcy8NCj4+
IGVuZGlmDQo+PiBvYmotJChDT05GSUdfVEVFKSArPSB0ZWUvDQo+PiArb2JqLSQoQ09ORklHX0hB
U19WUENJKSArPSB2cGNpLm8NCj4+IA0KPj4gb2JqLSQoQ09ORklHX0hBU19BTFRFUk5BVElWRSkg
Kz0gYWx0ZXJuYXRpdmUubw0KPj4gb2JqLXkgKz0gYm9vdGZkdC5pbml0Lm8NCj4+IGRpZmYgLS1n
aXQgYS94ZW4vYXJjaC9hcm0vZG9tYWluLmMgYi94ZW4vYXJjaC9hcm0vZG9tYWluLmMNCj4+IGlu
ZGV4IDE5Yzc1NmFjM2QuLmY3ZWQxMzAwMjMgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0v
ZG9tYWluLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9kb21haW4uYw0KPj4gQEAgLTM5LDYgKzM5
LDcgQEANCj4+ICNpbmNsdWRlIDxhc20vdmdpYy5oPg0KPj4gI2luY2x1ZGUgPGFzbS92dGltZXIu
aD4NCj4+IA0KPj4gKyNpbmNsdWRlICJ2cGNpLmgiDQo+PiAjaW5jbHVkZSAidnVhcnQuaCINCj4+
IA0KPj4gREVGSU5FX1BFUl9DUFUoc3RydWN0IHZjcHUgKiwgY3Vycl92Y3B1KTsNCj4+IEBAIC02
MjMsNyArNjI0LDcgQEAgaW50IGFyY2hfc2FuaXRpc2VfZG9tYWluX2NvbmZpZyhzdHJ1Y3QgeGVu
X2RvbWN0bF9jcmVhdGVkb21haW4gKmNvbmZpZykNCj4+ICAgICB1bnNpZ25lZCBpbnQgbWF4X3Zj
cHVzOw0KPj4gDQo+PiAgICAgLyogSFZNIGFuZCBIQVAgbXVzdCBiZSBzZXQuIElPTU1VIG1heSBv
ciBtYXkgbm90IGJlICovDQo+PiAtICAgIGlmICggKGNvbmZpZy0+ZmxhZ3MgJiB+WEVOX0RPTUNU
TF9DREZfaW9tbXUpICE9DQo+PiArICAgIGlmICggKGNvbmZpZy0+ZmxhZ3MgJiB+WEVOX0RPTUNU
TF9DREZfaW9tbXUgJiB+WEVOX0RPTUNUTF9DREZfdnBjaSkgIT0NCj4+ICAgICAgICAgIChYRU5f
RE9NQ1RMX0NERl9odm0gfCBYRU5fRE9NQ1RMX0NERl9oYXApICkNCj4+ICAgICB7DQo+PiAgICAg
ICAgIGRwcmludGsoWEVOTE9HX0lORk8sICJVbnN1cHBvcnRlZCBjb25maWd1cmF0aW9uICUjeFxu
IiwNCj4+IEBAIC03NjcsNiArNzY4LDkgQEAgaW50IGFyY2hfZG9tYWluX2NyZWF0ZShzdHJ1Y3Qg
ZG9tYWluICpkLA0KPj4gICAgIGlmICggaXNfaGFyZHdhcmVfZG9tYWluKGQpICYmIChyYyA9IGRv
bWFpbl92dWFydF9pbml0KGQpKSApDQo+PiAgICAgICAgIGdvdG8gZmFpbDsNCj4+IA0KPj4gKyAg
ICBpZiAoIChyYyA9IGRvbWFpbl92cGNpX2luaXQoZCkpICE9IDAgKQ0KPj4gKyAgICAgICAgZ290
byBmYWlsOw0KPj4gKw0KPj4gICAgIHJldHVybiAwOw0KPj4gDQo+PiBmYWlsOg0KPj4gZGlmZiAt
LWdpdCBhL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYyBiL3hlbi9hcmNoL2FybS9kb21haW5f
YnVpbGQuYw0KPj4gaW5kZXggZDIzM2Q2MzRjMS4uNWViODNiMTJhMSAxMDA2NDQNCj4+IC0tLSBh
L3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL2RvbWFp
bl9idWlsZC5jDQo+PiBAQCAtMjc3Myw2ICsyNzczLDkgQEAgdm9pZCBfX2luaXQgY3JlYXRlX2Rv
bTAodm9pZCkNCj4+ICAgICBpZiAoIGlvbW11X2VuYWJsZWQgKQ0KPj4gICAgICAgICBkb20wX2Nm
Zy5mbGFncyB8PSBYRU5fRE9NQ1RMX0NERl9pb21tdTsNCj4+IA0KPj4gKyAgICBpZiAoIHBjaV9w
YXNzdGhyb3VnaF9lbmFibGVkICkNCj4gDQo+IHdlIHNob3VsZCB1c2UgdGhlIGFjY2Vzc29yIGZ1
bmN0aW9uIGluc3RlYWQgaXNfcGNpX3Bhc3N0aHJvdWdoX2VuYWJsZWQNCg0KQWNrLiANCj4gDQo+
IA0KPj4gKyAgICAgICAgZG9tMF9jZmcuZmxhZ3MgfD0gWEVOX0RPTUNUTF9DREZfdnBjaTsNCj4+
ICsNCj4+ICAgICBkb20wID0gZG9tYWluX2NyZWF0ZSgwLCAmZG9tMF9jZmcsIHRydWUpOw0KPj4g
ICAgIGlmICggSVNfRVJSKGRvbTApIHx8IChhbGxvY19kb20wX3ZjcHUwKGRvbTApID09IE5VTEwp
ICkNCj4+ICAgICAgICAgcGFuaWMoIkVycm9yIGNyZWF0aW5nIGRvbWFpbiAwXG4iKTsNCj4+IGRp
ZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vdnBjaS5jIGIveGVuL2FyY2gvYXJtL3ZwY2kuYw0KPj4g
bmV3IGZpbGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4IDAwMDAwMDAwMDAuLjc2YzEyYjkyODENCj4+
IC0tLSAvZGV2L251bGwNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS92cGNpLmMNCj4+IEBAIC0wLDAg
KzEsMTAyIEBADQo+PiArLyoNCj4+ICsgKiB4ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiArICoNCj4+
ICsgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUg
aXQgYW5kL29yIG1vZGlmeQ0KPj4gKyAqIGl0IHVuZGVyIHRoZSB0ZXJtcyBvZiB0aGUgR05VIEdl
bmVyYWwgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5DQo+PiArICogdGhlIEZyZWUgU29m
dHdhcmUgRm91bmRhdGlvbjsgZWl0aGVyIHZlcnNpb24gMiBvZiB0aGUgTGljZW5zZSwgb3INCj4+
ICsgKiAoYXQgeW91ciBvcHRpb24pIGFueSBsYXRlciB2ZXJzaW9uLg0KPj4gKyAqDQo+PiArICog
VGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1
c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3aXRob3V0IGV2ZW4gdGhl
IGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFCSUxJVFkgb3IgRklUTkVTUyBG
T1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiArICogR05VIEdlbmVyYWwgUHVi
bGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKi8NCj4+ICsjaW5jbHVkZSA8eGVu
L3NjaGVkLmg+DQo+PiArDQo+PiArI2luY2x1ZGUgPGFzbS9tbWlvLmg+DQo+PiArDQo+PiArI2Rl
ZmluZSBSRUdJU1RFUl9PRkZTRVQoYWRkcikgICggKGFkZHIpICYgMHgwMDAwMGZmZikNCj4+ICsN
Cj4+ICsvKiBEbyBzb21lIHNhbml0eSBjaGVja3MuICovDQo+PiArc3RhdGljIGJvb2wgdnBjaV9t
bWlvX2FjY2Vzc19hbGxvd2VkKHVuc2lnbmVkIGludCByZWcsIHVuc2lnbmVkIGludCBsZW4pDQo+
PiArew0KPj4gKyAgICAvKiBDaGVjayBhY2Nlc3Mgc2l6ZS4gKi8NCj4+ICsgICAgaWYgKCBsZW4g
PiA4ICkNCj4+ICsgICAgICAgIHJldHVybiBmYWxzZTsNCj4+ICsNCj4+ICsgICAgLyogQ2hlY2sg
dGhhdCBhY2Nlc3MgaXMgc2l6ZSBhbGlnbmVkLiAqLw0KPj4gKyAgICBpZiAoIChyZWcgJiAobGVu
IC0gMSkpICkNCj4+ICsgICAgICAgIHJldHVybiBmYWxzZTsNCj4+ICsNCj4+ICsgICAgcmV0dXJu
IHRydWU7DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRpYyBpbnQgdnBjaV9tbWlvX3JlYWQoc3RydWN0
IHZjcHUgKnYsIG1taW9faW5mb190ICppbmZvLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgcmVnaXN0ZXJfdCAqciwgdm9pZCAqcCkNCj4+ICt7DQo+PiArICAgIHVuc2lnbmVkIGludCBy
ZWc7DQo+PiArICAgIHBjaV9zYmRmX3Qgc2JkZjsNCj4+ICsgICAgdW5zaWduZWQgbG9uZyBkYXRh
ID0gfjBVTDsNCj4+ICsgICAgdW5zaWduZWQgaW50IHNpemUgPSAxVSA8PCBpbmZvLT5kYWJ0LnNp
emU7DQo+PiArDQo+PiArICAgIHNiZGYuc2JkZiA9IE1NQ0ZHX0JERihpbmZvLT5ncGEpOw0KPj4g
KyAgICByZWcgPSBSRUdJU1RFUl9PRkZTRVQoaW5mby0+Z3BhKTsNCj4+ICsNCj4+ICsgICAgaWYg
KCAhdnBjaV9tbWlvX2FjY2Vzc19hbGxvd2VkKHJlZywgc2l6ZSkgKQ0KPj4gKyAgICAgICAgcmV0
dXJuIDA7DQo+PiArDQo+PiArICAgIGRhdGEgPSB2cGNpX3JlYWQoc2JkZiwgcmVnLCBtaW4oNHUs
IHNpemUpKTsNCj4+ICsgICAgaWYgKCBzaXplID09IDggKQ0KPj4gKyAgICAgICAgZGF0YSB8PSAo
dWludDY0X3QpdnBjaV9yZWFkKHNiZGYsIHJlZyArIDQsIDQpIDw8IDMyOw0KPj4gKw0KPj4gKyAg
ICAqciA9IGRhdGE7DQo+PiArDQo+PiArICAgIHJldHVybiAxOw0KPj4gK30NCj4+ICsNCj4+ICtz
dGF0aWMgaW50IHZwY2lfbW1pb193cml0ZShzdHJ1Y3QgdmNwdSAqdiwgbW1pb19pbmZvX3QgKmlu
Zm8sDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVnaXN0ZXJfdCByLCB2b2lkICpw
KQ0KPj4gK3sNCj4+ICsgICAgdW5zaWduZWQgaW50IHJlZzsNCj4+ICsgICAgcGNpX3NiZGZfdCBz
YmRmOw0KPj4gKyAgICB1bnNpZ25lZCBsb25nIGRhdGEgPSByOw0KPj4gKyAgICB1bnNpZ25lZCBp
bnQgc2l6ZSA9IDFVIDw8IGluZm8tPmRhYnQuc2l6ZTsNCj4+ICsNCj4+ICsgICAgc2JkZi5zYmRm
ID0gTU1DRkdfQkRGKGluZm8tPmdwYSk7DQo+PiArICAgIHJlZyA9IFJFR0lTVEVSX09GRlNFVChp
bmZvLT5ncGEpOw0KPj4gKw0KPj4gKyAgICBpZiAoICF2cGNpX21taW9fYWNjZXNzX2FsbG93ZWQo
cmVnLCBzaXplKSApDQo+PiArICAgICAgICByZXR1cm4gMDsNCj4+ICsNCj4+ICsgICAgdnBjaV93
cml0ZShzYmRmLCByZWcsIG1pbig0dSwgc2l6ZSksIGRhdGEpOw0KPj4gKyAgICBpZiAoIHNpemUg
PT0gOCApDQo+PiArICAgICAgICB2cGNpX3dyaXRlKHNiZGYsIHJlZyArIDQsIDQsIGRhdGEgPj4g
MzIpOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gMTsNCj4+ICt9DQo+PiArDQo+PiArc3RhdGljIGNv
bnN0IHN0cnVjdCBtbWlvX2hhbmRsZXJfb3BzIHZwY2lfbW1pb19oYW5kbGVyID0gew0KPj4gKyAg
ICAucmVhZCAgPSB2cGNpX21taW9fcmVhZCwNCj4+ICsgICAgLndyaXRlID0gdnBjaV9tbWlvX3dy
aXRlLA0KPj4gK307DQo+PiArDQo+PiAraW50IGRvbWFpbl92cGNpX2luaXQoc3RydWN0IGRvbWFp
biAqZCkNCj4+ICt7DQo+PiArICAgIGlmICggIWhhc192cGNpKGQpICkNCj4+ICsgICAgICAgIHJl
dHVybiAwOw0KPj4gKw0KPj4gKyAgICByZWdpc3Rlcl9tbWlvX2hhbmRsZXIoZCwgJnZwY2lfbW1p
b19oYW5kbGVyLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgR1VFU1RfVlBDSV9FQ0FN
X0JBU0UsIEdVRVNUX1ZQQ0lfRUNBTV9TSVpFLCBOVUxMKTsNCj4+ICsNCj4+ICsgICAgcmV0dXJu
IDA7DQo+PiArfQ0KPj4gKw0KPj4gKy8qDQo+PiArICogTG9jYWwgdmFyaWFibGVzOg0KPj4gKyAq
IG1vZGU6IEMNCj4+ICsgKiBjLWZpbGUtc3R5bGU6ICJCU0QiDQo+PiArICogYy1iYXNpYy1vZmZz
ZXQ6IDQNCj4+ICsgKiBpbmRlbnQtdGFicy1tb2RlOiBuaWwNCj4+ICsgKiBFbmQ6DQo+PiArICov
DQo+PiArDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3ZwY2kuaCBiL3hlbi9hcmNoL2Fy
bS92cGNpLmgNCj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+PiBpbmRleCAwMDAwMDAwMDAwLi5k
OGE3YjBlM2U4DQo+PiAtLS0gL2Rldi9udWxsDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vdnBjaS5o
DQo+PiBAQCAtMCwwICsxLDM2IEBADQo+PiArLyoNCj4+ICsgKiB4ZW4vYXJjaC9hcm0vdnBjaS5o
DQo+PiArICoNCj4+ICsgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiBy
ZWRpc3RyaWJ1dGUgaXQgYW5kL29yIG1vZGlmeQ0KPj4gKyAqIGl0IHVuZGVyIHRoZSB0ZXJtcyBv
ZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5DQo+PiArICog
dGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbjsgZWl0aGVyIHZlcnNpb24gMiBvZiB0aGUgTGlj
ZW5zZSwgb3INCj4+ICsgKiAoYXQgeW91ciBvcHRpb24pIGFueSBsYXRlciB2ZXJzaW9uLg0KPj4g
KyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQg
aXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3aXRo
b3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFCSUxJVFkg
b3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiArICogR05V
IEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKi8NCj4+ICsN
Cj4+ICsjaWZuZGVmIF9fQVJDSF9BUk1fVlBDSV9IX18NCj4+ICsjZGVmaW5lIF9fQVJDSF9BUk1f
VlBDSV9IX18NCj4+ICsNCj4+ICsjaWZkZWYgQ09ORklHX0hBU19WUENJDQo+PiAraW50IGRvbWFp
bl92cGNpX2luaXQoc3RydWN0IGRvbWFpbiAqZCk7DQo+PiArI2Vsc2UNCj4+ICtzdGF0aWMgaW5s
aW5lIGludCBkb21haW5fdnBjaV9pbml0KHN0cnVjdCBkb21haW4gKmQpDQo+PiArew0KPj4gKyAg
ICByZXR1cm4gMDsNCj4+ICt9DQo+PiArI2VuZGlmDQo+PiArDQo+PiArI2VuZGlmIC8qIF9fQVJD
SF9BUk1fVlBDSV9IX18gKi8NCj4+ICsNCj4+ICsvKg0KPj4gKyAqIExvY2FsIHZhcmlhYmxlczoN
Cj4+ICsgKiBtb2RlOiBDDQo+PiArICogYy1maWxlLXN0eWxlOiAiQlNEIg0KPj4gKyAqIGMtYmFz
aWMtb2Zmc2V0OiA0DQo+PiArICogaW5kZW50LXRhYnMtbW9kZTogbmlsDQo+PiArICogRW5kOg0K
Pj4gKyAqLw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9jb21tb24vZG9tYWluLmMgYi94ZW4vY29tbW9u
L2RvbWFpbi5jDQo+PiBpbmRleCA2ZWU1ZDAzM2IwLi40MGQ2N2VjMzQyIDEwMDY0NA0KPj4gLS0t
IGEveGVuL2NvbW1vbi9kb21haW4uYw0KPj4gKysrIGIveGVuL2NvbW1vbi9kb21haW4uYw0KPj4g
QEAgLTQ4Myw3ICs0ODMsNyBAQCBzdGF0aWMgaW50IHNhbml0aXNlX2RvbWFpbl9jb25maWcoc3Ry
dWN0IHhlbl9kb21jdGxfY3JlYXRlZG9tYWluICpjb25maWcpDQo+PiAgICAgICAgICB+KFhFTl9E
T01DVExfQ0RGX2h2bSB8IFhFTl9ET01DVExfQ0RGX2hhcCB8DQo+PiAgICAgICAgICAgIFhFTl9E
T01DVExfQ0RGX3MzX2ludGVncml0eSB8IFhFTl9ET01DVExfQ0RGX29vc19vZmYgfA0KPj4gICAg
ICAgICAgICBYRU5fRE9NQ1RMX0NERl94c19kb21haW4gfCBYRU5fRE9NQ1RMX0NERl9pb21tdSB8
DQo+PiAtICAgICAgICAgICBYRU5fRE9NQ1RMX0NERl9uZXN0ZWRfdmlydCkgKQ0KPj4gKyAgICAg
ICAgICAgWEVOX0RPTUNUTF9DREZfbmVzdGVkX3ZpcnQgfCBYRU5fRE9NQ1RMX0NERl92cGNpKSAp
DQo+PiAgICAgew0KPj4gICAgICAgICBkcHJpbnRrKFhFTkxPR19JTkZPLCAiVW5rbm93biBDREYg
ZmxhZ3MgJSN4XG4iLCBjb25maWctPmZsYWdzKTsNCj4+ICAgICAgICAgcmV0dXJuIC1FSU5WQUw7
DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMgYi94ZW4vZHJp
dmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gaW5kZXggZDc3NGE2MTU0ZS4uNjMzZTg5YWMxMyAx
MDA2NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+PiArKysgYi94
ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gQEAgLTc2Nyw2ICs3NjcsMTggQEAgaW50
IHBjaV9hZGRfZGV2aWNlKHUxNiBzZWcsIHU4IGJ1cywgdTggZGV2Zm4sDQo+PiAgICAgZWxzZQ0K
Pj4gICAgICAgICBpb21tdV9lbmFibGVfZGV2aWNlKHBkZXYpOw0KPj4gDQo+PiArI2lmZGVmIENP
TkZJR19BUk0NCj4+ICsgICAgLyoNCj4+ICsgICAgICogT24gQVJNIFBDSSBkZXZpY2VzIGRpc2Nv
dmVyeSB3aWxsIGJlIGRvbmUgYnkgRG9tMC4gQWRkIHZwY2kgaGFuZGxlciB3aGVuDQo+PiArICAg
ICAqIERvbTAgaW5mb3JtIFhFTiB0byBhZGQgdGhlIFBDSSBkZXZpY2VzIGluIFhFTi4NCj4+ICsg
ICAgICovDQo+IA0KPiBJIHRha2UgdGhpcyBpcyBub3QgbmVlZGVkIG9uIHg4NiBiZWNhdXNlIGZv
ciB2cGNpIGlzIG9ubHkgdXNlZCBieSBEb20wDQo+IFBWSD8gIEl0IHdvdWxkIGJlIGdvb2QgdG8g
Y2xhcmlmeSBlaXRoZXIgd2F5Lg0KDQogSSBoYXZlIHRoZSBzYW1lIHVuZGVyc3RhbmRpbmcgbGV0
IHNvbWVvbmUgZWxzZSBjb25maXJtcy4NCj4gDQo+IA0KPj4gKyAgICByZXQgPSB2cGNpX2FkZF9o
YW5kbGVycyhwZGV2KTsNCj4+ICsgICAgaWYgKCByZXQgKSB7DQo+PiArICAgICAgICBwcmludGso
WEVOTE9HX0VSUiAic2V0dXAgb2YgdlBDSSBmb3IgZmFpbGVkOiAlZFxuIixyZXQpOw0KPj4gKyAg
ICAgICAgZ290byBvdXQ7DQo+PiArICAgIH0NCj4+ICsjZW5kaWYNCj4+ICsNCj4+ICAgICBwY2lf
ZW5hYmxlX2FjcyhwZGV2KTsNCj4+IA0KPj4gb3V0Og0KPj4gZGlmZiAtLWdpdCBhL3hlbi9pbmNs
dWRlL2FzbS1hcm0vZG9tYWluLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL2RvbWFpbi5oDQo+PiBp
bmRleCBjOTI3N2I1YzZkLi41ZmI0ZTI0YWRmIDEwMDY0NA0KPj4gLS0tIGEveGVuL2luY2x1ZGUv
YXNtLWFybS9kb21haW4uaA0KPj4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9kb21haW4uaA0K
Pj4gQEAgLTIsNiArMiw3IEBADQo+PiAjZGVmaW5lIF9fQVNNX0RPTUFJTl9IX18NCj4+IA0KPj4g
I2luY2x1ZGUgPHhlbi9jYWNoZS5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vbm9zcGVjLmg+DQo+PiAj
aW5jbHVkZSA8eGVuL3RpbWVyLmg+DQo+PiAjaW5jbHVkZSA8YXNtL3BhZ2UuaD4NCj4+ICNpbmNs
dWRlIDxhc20vcDJtLmg+DQo+PiBAQCAtMjYyLDcgKzI2MywxMiBAQCBzdGF0aWMgaW5saW5lIHZv
aWQgYXJjaF92Y3B1X2Jsb2NrKHN0cnVjdCB2Y3B1ICp2KSB7fQ0KPj4gDQo+PiAjZGVmaW5lIGFy
Y2hfdm1fYXNzaXN0X3ZhbGlkX21hc2soZCkgKDFVTCA8PCBWTUFTU1RfVFlQRV9ydW5zdGF0ZV91
cGRhdGVfZmxhZykNCj4+IA0KPj4gLSNkZWZpbmUgaGFzX3ZwY2koZCkgICAgKHsgKHZvaWQpKGQp
OyBmYWxzZTsgfSkNCj4+ICsvKg0KPj4gKyAqIEZvciBYODYgVlBDSSBpcyBlbmFibGVkIGFuZCB0
ZXN0ZWQgZm9yIFBWSCBET00wIG9ubHkgYnV0DQo+PiArICogZm9yIEFSTSB3ZSBlbmFibGUgc3Vw
cG9ydCBWUENJIGZvciBndWVzdCBkb21haW4gYWxzby4NCj4+ICsgKi8NCj4+ICsjZGVmaW5lIGhh
c192cGNpKGQpICgodm9pZCkoZCksIElTX0VOQUJMRUQoQ09ORklHX0hBU19WUENJKSAmJiBcDQo+
PiArICAgICAgICBldmFsdWF0ZV9ub3NwZWMoZC0+b3B0aW9ucyAmIFhFTl9ET01DVExfQ0RGX3Zw
Y2kpKQ0KPiANCj4gV2l0aCB0aGUgbmV3IG9wdGlvbnMgJiBYRU5fRE9NQ1RMX0NERl92cGNpIGNo
ZWNrIGluIHBsYWNlLCBkbyB3ZSBldmVuDQo+IG5lZWQgSVNfRU5BQkxFRChDT05GSUdfSEFTX1ZQ
Q0kpIGFueSBsb25nZXI/DQo+IA0KIFllcyB5b3UgYXJlIHJpZ2h0IG5vdCByZXF1aXJlZC4NCg0K
UmVnYXJkcywNClJhaHVs


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:36:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:36:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194479.346455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQld-0003hG-8e; Thu, 23 Sep 2021 15:36:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194479.346455; Thu, 23 Sep 2021 15:36:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQld-0003h9-5g; Thu, 23 Sep 2021 15:36:09 +0000
Received: by outflank-mailman (input) for mailman id 194479;
 Thu, 23 Sep 2021 15:36:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTQlb-0003h1-UI
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:36:07 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b8fe116c-aa07-43e5-8542-9a476c4eaec8;
 Thu, 23 Sep 2021 15:36:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1234761029;
 Thu, 23 Sep 2021 15:36:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8fe116c-aa07-43e5-8542-9a476c4eaec8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632411366;
	bh=aT5z/PuEXqsyUmbvog5JXhGE87ZZ8uE4fYmPcFjU0PI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=im/Xnqf41Ihr0HK717PQhrg6e3fT3HE+6OYKLHf54ZnRzkRi2+q1WnFqAJxaVpYaO
	 8Zhlfm9mIoAK/kza5hh3vjubvQeb6AdTfwgGAUiKltKpAOuvQVPKttpvpL6XPpsVIu
	 RRZTrbhW2rguiHO9tNpalcseoPNILWp6bahyTz5n8CCNucXLhWCtQYS5V1Dc62b2p5
	 LJj05thr+j13XpcIY6nyeku15YlTiepPRveB0KRqJXgMZuqxJ9v09kV1qVb7JldonL
	 EOp2almrYNzQ+drXi84tCN38AObEhKcj6XM/hltwNijgTLkYk2EDyHZtEfc+BSxRRs
	 bhuGo6tA8pQsg==
Date: Thu, 23 Sep 2021 08:36:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Rahul Singh <rahul.singh@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    "andre.przywara@arm.com" <andre.przywara@arm.com>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <e2634706-3198-3224-371c-3d13210a35c9@epam.com>
Message-ID: <alpine.DEB.2.21.2109230835110.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s> <e2634706-3198-3224-371c-3d13210a35c9@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1032229729-1632411330=:17979"
Content-ID: <alpine.DEB.2.21.2109230835470.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1032229729-1632411330=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109230835471.17979@sstabellini-ThinkPad-T480s>

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> Hi, Stefano!
> 
> [snip]
> 
> 
> >> +};
> >> +
> >> +/* Default ECAM ops */
> >> +extern const struct pci_ecam_ops pci_generic_ecam_ops;
> > If we use container_of and get rid of sysdata, I wonder if we get avoid
> > exporting pci_generic_ecam_ops.
> >
> >
> >> +int pci_host_common_probe(struct dt_device_node *dev, const void *data);
> > This should be static and not exported
> >
> >
> >> +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> +                            uint32_t reg, uint32_t len, uint32_t *value);
> > also this
> >
> >
> >> +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> >> +                            uint32_t reg, uint32_t len, uint32_t value);
> > also this
> >
> >
> >> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> >> +                               uint32_t sbdf, uint32_t where);
> > also this
> >
> >>   static always_inline bool is_pci_passthrough_enabled(void)
> >>   {
> >>       return pci_passthrough_enabled;
> >> -- 
> >> 2.17.1
> 
> All the above is still need to be exported as those are going to be used
> 
> by other bridge's implementations, see ZynqMP for instance later in the series.
> 
> I'll post a snippet from the future:
> 
> /* ECAM ops */
> const struct pci_ecam_ops nwl_pcie_ops = {
>      .bus_shift  = 20,
>      .cfg_reg_index = nwl_cfg_reg_index,
>      .pci_ops    = {
>          .map_bus                = pci_ecam_map_bus,
>          .read                   = pci_generic_config_read,
>          .write                  = pci_generic_config_write,
>          .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>      }
> };
> 
> DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
> .dt_match = nwl_pcie_dt_match,
> .init = pci_host_common_probe,
> DT_DEVICE_END

OK then
--8323329-1032229729-1632411330=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 15:48:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 15:48:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194487.346469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQxr-0005rY-CE; Thu, 23 Sep 2021 15:48:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194487.346469; Thu, 23 Sep 2021 15:48:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTQxr-0005rR-9G; Thu, 23 Sep 2021 15:48:47 +0000
Received: by outflank-mailman (input) for mailman id 194487;
 Thu, 23 Sep 2021 15:48:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTQxq-0005rL-N4
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 15:48:46 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bbd06948-1c85-11ec-ba58-12813bfff9fa;
 Thu, 23 Sep 2021 15:48:45 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4853B6008E;
 Thu, 23 Sep 2021 15:48:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbd06948-1c85-11ec-ba58-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632412124;
	bh=FI+LxKsIIc2jqDmoLIF3HH4uGPkcIv4DllCKMTlM1lA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=YrAL3rtByebZdH0B+5sxM2xIMXgYEkp8WRfTrIyZBbIHYt5FSnhvbEO5vzNCj6Mwl
	 0gmUX6PfilnDPVtJnbcWIZxGM6q3PMhNXWwH2IfHsVK9QozPiQO/AajcR90nH8jkwu
	 1lRoy1MX3bs6mLq0z83hDSbsYifaXNgGJwQB1QbOaGDaRUbVXZ7fYMmQHLhkiUuZBo
	 12Z5+daWlV/5E+9WLmdoxXbGELDQBoRnsnfExXr97G1XXVqoLxxK7e/L/vPyPY5iAk
	 5GiFytYxShi2LlFWOaxkpWZ3RnBD/W4aEZpg0S8uKcyIRGmLoaZMxOYTSj+nJOplY/
	 QzmjIsrPlOvjQ==
Date: Thu, 23 Sep 2021 08:48:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>
Subject: Re: [PATCH 2/2] xen-pciback: prepare for the split for stub and PV
In-Reply-To: <98e432f0-6be1-6394-1c06-ac5e726c708d@epam.com>
Message-ID: <alpine.DEB.2.21.2109230848290.17979@sstabellini-ThinkPad-T480s>
References: <20210922101422.2319240-1-andr2000@gmail.com> <20210922101422.2319240-2-andr2000@gmail.com> <alpine.DEB.2.21.2109221407350.17979@sstabellini-ThinkPad-T480s> <4552e4b6-21a8-4829-16b4-7cda8ba0c0d1@epam.com> <7880dee9-7372-5a25-db55-018f21e8b08c@suse.com>
 <98e432f0-6be1-6394-1c06-ac5e726c708d@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1873022447-1632412124=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1873022447-1632412124=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> On 23.09.21 12:05, Juergen Gross wrote:
> > On 23.09.21 11:02, Oleksandr Andrushchenko wrote:
> >>
> >> On 23.09.21 00:10, Stefano Stabellini wrote:
> >>> On Wed, 22 Sep 2021, Oleksandr Andrushchenko wrote:
> >>>> --- a/drivers/xen/xen-pciback/xenbus.c
> >>>> +++ b/drivers/xen/xen-pciback/xenbus.c
> >>>> @@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
> >>>>       int __init xen_pcibk_xenbus_register(void)
> >>>>    {
> >>>> +    if (!xen_pcibk_pv_support())
> >>>> +        return 0;
> >>> Is this truly enough to stop the PV backend from initializing? Have you
> >>> actually tested it to make sure? If it works, amazing! I am quite happy
> >>> about this approach :-)
> >>
> >> Well, I put some logs into the driver and saw nothing obvious pointing
> >>
> >> to any backend activities (probably this is also because I don't have any
> >>
> >> frontend). I see that the xenbus driver is not registered. In XenStore I see:
> >>
> >> root@dom0:~# xenstore-ls -f | grep pci
> >> /local/domain/0/backend/pci = ""
> >> /local/domain/0/backend/pci/2 = ""
> >> /local/domain/0/backend/pci/2/0 = ""
> >> /local/domain/0/backend/pci/2/0/frontend = "/local/domain/2/device/pci/0"
> >> /local/domain/0/backend/pci/2/0/frontend-id = "2"
> >> /local/domain/0/backend/pci/2/0/online = "1"
> >> /local/domain/0/backend/pci/2/0/state = "1"
> >> /local/domain/0/backend/pci/2/0/domain = "DomU"
> >> /local/domain/0/backend/pci/2/0/key-0 = "0000:03:00.0"
> >> /local/domain/0/backend/pci/2/0/dev-0 = "0000:03:00.0"
> >> /local/domain/0/backend/pci/2/0/opts-0 = "msitranslate=0,power_mgmt=0,permissive=0,rdm_policy=strict"
> >> /local/domain/0/backend/pci/2/0/state-0 = "1"
> >> /local/domain/0/backend/pci/2/0/num_devs = "1"
> >> /local/domain/2/device/pci = ""
> >> /local/domain/2/device/pci/0 = ""
> >> /local/domain/2/device/pci/0/backend = "/local/domain/0/backend/pci/2/0"
> >> /local/domain/2/device/pci/0/backend-id = "0"
> >> /local/domain/2/device/pci/0/state = "1"
> >> /libxl/pci = ""
> >> /libxl/pci/0000-03-00-0 = ""
> >> /libxl/pci/0000-03-00-0/domid = "2"
> >>
> >> But IIUIC these come from the toolstack
> >>
> >> @Juergen, do you know how to check if the backend is indeed not running
> >>
> >> or the above should be enough to prove?
> >
> > I don't see how the backend could be running without being registered
> > with xenbus. It won't receive any watches, so there is no way a
> > connection with a frontend could be established.
> 
> This is my understanding too, so the only change I've put in patch I removed
> 
> register/unregister. It seems this is just enough and the patch should be ok as is

Fantastic! Thanks for checking.
--8323329-1873022447-1632412124=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 16:02:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 16:02:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194492.346480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTRAv-0000SW-GD; Thu, 23 Sep 2021 16:02:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194492.346480; Thu, 23 Sep 2021 16:02:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTRAv-0000SP-C7; Thu, 23 Sep 2021 16:02:17 +0000
Received: by outflank-mailman (input) for mailman id 194492;
 Thu, 23 Sep 2021 16:02:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9gl1=ON=linuxfoundation.org=torvalds@srs-us1.protection.inumbo.net>)
 id 1mTRAt-0000SJ-TI
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 16:02:16 +0000
Received: from mail-ed1-x536.google.com (unknown [2a00:1450:4864:20::536])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 50eb9f99-289b-41fd-92a1-959b7e063bc5;
 Thu, 23 Sep 2021 16:02:14 +0000 (UTC)
Received: by mail-ed1-x536.google.com with SMTP id g8so25313702edt.7
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 09:02:14 -0700 (PDT)
Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com.
 [209.85.218.48])
 by smtp.gmail.com with ESMTPSA id n10sm3294322ejk.86.2021.09.23.09.02.13
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 09:02:13 -0700 (PDT)
Received: by mail-ej1-f48.google.com with SMTP id e21so225548ejz.12
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 09:02:13 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50eb9f99-289b-41fd-92a1-959b7e063bc5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linux-foundation.org; s=google;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=DLOxhnwH/JaVIhRbDcxcZPpiYGHLJbtd0CRIGAWnOsw=;
        b=QMXOiJ6n28J2emY/y7cY9+DD4h8Pn0KRvNWmJxYGi/2iw91Mhhsv5k4E/yi6SvFmH+
         cwsdxpHCcIaSp+pnaMQx9j8vfDg38b4ujoTb2lzS4jvwjrgaoWuJfcNbrdzMUshkRDfy
         wyxZpTZP4dsY0GDCSs5bbcT3wFiFu+kSB/Akk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=DLOxhnwH/JaVIhRbDcxcZPpiYGHLJbtd0CRIGAWnOsw=;
        b=Lr5dPBaeMQugsGOJeUC56HO7tuySBs0gHTXtHRTrSmB5OQcHW4aR24oeLRs4yEcGtO
         ZerSSkj/v4foZLmEFeOkbo4ZaYfT3dDKkmhkhDtKDjGhUQiJAbjzFnLhJ+bcx2EEdGxP
         GwN8WIt5c/w9B+96eRTEtJFyJRczwp/RimYLuepClQfXdomB/kNrUvpL7mxO+z4OJG1u
         Oi3SkNG2JOD+lNnfxqLEXlz0s7wgqGEqE/xaBfIpQdaY5HfEY5DuPefAGMrgwjK3DD9R
         Mz9DdfImh8e+eTwLHbILndG+3snkEsIqouju5rnrDx8CTfIqvEirC5l7iGsobZ2+tz4y
         3E4g==
X-Gm-Message-State: AOAM533J3FZzVsUC9ZcMcMOx+YlP1GJ7Vx3eu+n7f53EJxPxsBP+ziZp
	HPbI4M30vKWPW6cankK2sJHVMfMRFAct232pQbY=
X-Google-Smtp-Source: ABdhPJy1THLnoCM+GbaIhRuthYQzp/+4SGFDokVasBXXLI2ttoCWFCMYc4b2mAmYotUMRDMcIj0YOA==
X-Received: by 2002:a05:6402:3088:: with SMTP id de8mr6441783edb.76.1632412933833;
        Thu, 23 Sep 2021 09:02:13 -0700 (PDT)
X-Received: by 2002:a2e:3309:: with SMTP id d9mr5950451ljc.249.1632412922441;
 Thu, 23 Sep 2021 09:02:02 -0700 (PDT)
MIME-Version: 1.0
References: <20210923074335.12583-1-rppt@kernel.org>
In-Reply-To: <20210923074335.12583-1-rppt@kernel.org>
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Thu, 23 Sep 2021 09:01:46 -0700
X-Gmail-Original-Message-ID: <CAHk-=wiJB8H5pZz-AKaSJ7ViRtdxQGJT7eOByp8DJx2OwZSYwA@mail.gmail.com>
Message-ID: <CAHk-=wiJB8H5pZz-AKaSJ7ViRtdxQGJT7eOByp8DJx2OwZSYwA@mail.gmail.com>
Subject: Re: [PATCH 0/3] memblock: cleanup memblock_free interface
To: Mike Rapoport <rppt@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, devicetree <devicetree@vger.kernel.org>, 
	iommu <iommu@lists.linux-foundation.org>, kasan-dev <kasan-dev@googlegroups.com>, 
	KVM list <kvm@vger.kernel.org>, alpha <linux-alpha@vger.kernel.org>, 
	Linux ARM <linux-arm-kernel@lists.infradead.org>, 
	linux-efi <linux-efi@vger.kernel.org>, 
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, 
	"open list:BROADCOM NVRAM DRIVER" <linux-mips@vger.kernel.org>, Linux-MM <linux-mm@kvack.org>, 
	linux-riscv <linux-riscv@lists.infradead.org>, linux-s390 <linux-s390@vger.kernel.org>, 
	Linux-sh list <linux-sh@vger.kernel.org>, 
	"open list:SYNOPSYS ARC ARCHITECTURE" <linux-snps-arc@lists.infradead.org>, 
	linux-um <linux-um@lists.infradead.org>, linux-usb@vger.kernel.org, 
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, linux-sparc <sparclinux@vger.kernel.org>, 
	xen-devel@lists.xenproject.org, Mike Rapoport <rppt@linux.ibm.com>
Content-Type: text/plain; charset="UTF-8"

On Thu, Sep 23, 2021 at 12:43 AM Mike Rapoport <rppt@kernel.org> wrote:
>
> The core change is in the third patch that makes memblock_free() a
> counterpart of memblock_alloc() and adds memblock_phys_alloc() to be a

^^^^^^^^^^^^^^^^^^^
> counterpart of memblock_phys_alloc().

That should be 'memblock_phys_free()'

HOWEVER.

The real reason I'm replying is that this patch is horribly buggy, and
will cause subtle problems that are nasty to debug.

You need to be a LOT more careful.

>From a trivial check - exactly because I looked at doing it with a
script, and decided it's not so easy - I found cases like this:

-               memblock_free(__pa(paca_ptrs) + new_ptrs_size,
+               memblock_free(paca_ptrs + new_ptrs_size,

which is COMPLETELY wrong.

Why? Because now that addition is done as _pointer_ addition, not as
an integer addition, and the end result is something completely
different.

pcac_ptrs is of type 'struct paca_struct **', so when you add
new_ptrs_size to it, it will add it in terms of that many pointers,
not that many bytes.

You need to use some smarter scripting, or some way to validate it.

And no, making the scripting just replace '__pa(x)' with '(void *)(x)'
- which _would_ be mindless and get the same result - is not
acceptable either, because it avoids one of the big improvements from
using the right interface, namely having compiler type checking (and
saner code that people understand).

So NAK. No broken automated scripting patches.

               Linus


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 16:38:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 16:38:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194497.346490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTRkC-0004uA-By; Thu, 23 Sep 2021 16:38:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194497.346490; Thu, 23 Sep 2021 16:38:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTRkC-0004u3-91; Thu, 23 Sep 2021 16:38:44 +0000
Received: by outflank-mailman (input) for mailman id 194497;
 Thu, 23 Sep 2021 16:38:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTRkB-0004tx-GZ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 16:38:43 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5bcc798-1c8c-11ec-ba63-12813bfff9fa;
 Thu, 23 Sep 2021 16:38:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 83B1E6103C;
 Thu, 23 Sep 2021 16:38:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5bcc798-1c8c-11ec-ba63-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632415120;
	bh=7wJh0VSvQaXvpVS0CJyZ7ZvzqRATiL2kWVMSaWXJSTo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Rzgj9APkdFsCen3U5IvFLtA1ai90k3xiZUTjuysjNi4KgxiPlYGUzu/2EdKluZzkz
	 ZKPixqQONt/woiRTDDNswYF4BWnBPKZaCUdTPwHxbcIQ0H06FGDlXYbwFL1goRZfVz
	 odZkcRx05hlJ1PlvJXphC0rZa58t6o5RcqHha5yGREGqSs/7G3RS9zqDrQ9Dvs7J8i
	 2lcsZZanXoN3T2I4bCOfLL5mrQuY3mktZyQZwFOcEyBYTSY5Ha1retZbMTNYsQgbIo
	 2vFxflGg77j7CCJZMIrjCkV0HjcCobeY1bR8yaLgBZ76bp6ie9ZTZcT0eskN72NMQN
	 L2wmF+/qPMboA==
Date: Thu, 23 Sep 2021 09:38:39 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Henry Wang <Henry.Wang@arm.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <8665a10b-60b1-1551-0aab-e6725f4c6ff1@gmail.com>
Message-ID: <alpine.DEB.2.21.2109230931110.17979@sstabellini-ThinkPad-T480s>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com> <1631297924-8658-3-git-send-email-olekstysh@gmail.com> <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org> <08294c53-109a-8544-3a23-85e034d2992d@gmail.com> <cc5ee8cc-84ac-a27f-af99-ac0ba3ab8d68@gmail.com>
 <8665a10b-60b1-1551-0aab-e6725f4c6ff1@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-2026154566-1632415038=:17979"
Content-ID: <alpine.DEB.2.21.2109230937320.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2026154566-1632415038=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109230937321.17979@sstabellini-ThinkPad-T480s>

On Thu, 23 Sep 2021, Oleksandr wrote:
> On 18.09.21 19:59, Oleksandr wrote:
> > > > > +#define EXT_REGION_END 0x80003fffffffULL
> > > > > +
> > > > > +static int __init find_unallocated_memory(const struct kernel_info
> > > > > *kinfo,
> > > > > +                                          struct meminfo
> > > > > *ext_regions)
> > > > > +{
> > > > > +    const struct meminfo *assign_mem = &kinfo->mem;
> > > > > +    struct rangeset *unalloc_mem;
> > > > > +    paddr_t start, end;
> > > > > +    unsigned int i;
> > > > > +    int res;
> > > > 
> > > > We technically already know which range of memory is unused. This is
> > > > pretty much any region in the freelist of the page allocator. So how
> > > > about walking the freelist instead?
> > > 
> > > ok, I will investigate the page allocator code (right now I have no
> > > understanding of how to do that). BTW, I have just grepped "freelist"
> > > through the code and all page context related appearances are in x86 code
> > > only.
> > > 
> > > > 
> > > > The advantage is we don't need to worry about modifying the function
> > > > when adding new memory type.
> > > > 
> > > > One disavantage is this will not cover *all* the unused memory as this
> > > > is doing. But I think this is an acceptable downside.
> > 
> > I did some investigations and create test patch. Although, I am not 100%
> > sure this is exactly what you meant, but I will provide results anyway.
> > 
> > 1. Below the extended regions (unallocated memory, regions >=64MB )
> > calculated by my initial method (bootinfo.mem - kinfo->mem -
> > bootinfo.reserved_mem - kinfo->gnttab):
> > 
> > (XEN) Extended region 0: 0x48000000->0x54000000
> > (XEN) Extended region 1: 0x57000000->0x60000000
> > (XEN) Extended region 2: 0x70000000->0x78000000
> > (XEN) Extended region 3: 0x78200000->0xc0000000
> > (XEN) Extended region 4: 0x500000000->0x580000000
> > (XEN) Extended region 5: 0x600000000->0x680000000
> > (XEN) Extended region 6: 0x700000000->0x780000000
> > 
> > 2. Below the extended regions (unallocated memory, regions >=64MB)
> > calculated by new method (free memory in page allocator):
> > 
> > (XEN) Extended region 0: 0x48000000->0x54000000
> > (XEN) Extended region 1: 0x58000000->0x60000000
> > (XEN) Extended region 2: 0x70000000->0x78000000
> > (XEN) Extended region 3: 0x78200000->0x84000000
> > (XEN) Extended region 4: 0x86000000->0x8a000000
> > (XEN) Extended region 5: 0x8c200000->0xc0000000
> > (XEN) Extended region 6: 0x500000000->0x580000000
> > (XEN) Extended region 7: 0x600000000->0x680000000
> > (XEN) Extended region 8: 0x700000000->0x765e00000
> > 
> > Some thoughts regarding that.
> > 
> > 1. A few ranges below 4GB are absent in resulting extended regions. I
> > assume, this is because of the modules:
> > 
> > (XEN) Checking for initrd in /chosen
> > (XEN) Initrd 0000000084000040-0000000085effc48
> > (XEN) RAM: 0000000048000000 - 00000000bfffffff
> > (XEN) RAM: 0000000500000000 - 000000057fffffff
> > (XEN) RAM: 0000000600000000 - 000000067fffffff
> > (XEN) RAM: 0000000700000000 - 000000077fffffff
> > (XEN)
> > (XEN) MODULE[0]: 0000000078080000 - 00000000781d74c8 Xen
> > (XEN) MODULE[1]: 0000000057fe7000 - 0000000057ffd080 Device Tree
> > (XEN) MODULE[2]: 0000000084000040 - 0000000085effc48 Ramdisk
> > (XEN) MODULE[3]: 000000008a000000 - 000000008c000000 Kernel
> > (XEN) MODULE[4]: 000000008c000000 - 000000008c010000 XSM
> > (XEN)  RESVD[0]: 0000000084000040 - 0000000085effc48
> > (XEN)  RESVD[1]: 0000000054000000 - 0000000056ffffff
> > 
> > 2. Also, it worth mentioning that relatively large chunk (~417MB) of memory
> > above 4GB is absent (to be precise, at the end of last RAM bank), which I
> > assume, used for Xen internals.
> > We could really use it for extended regions.
> > Below free regions in the heap (for last RAM bank) just in case:
> > 
> > (XEN) heap[node=0][zone=23][order=5] 0x00000765ec0000-0x00000765ee0000
> > (XEN) heap[node=0][zone=23][order=6] 0x00000765e80000-0x00000765ec0000
> > (XEN) heap[node=0][zone=23][order=7] 0x00000765e00000-0x00000765e80000
> > (XEN) heap[node=0][zone=23][order=9] 0x00000765c00000-0x00000765e00000
> > (XEN) heap[node=0][zone=23][order=10] 0x00000765800000-0x00000765c00000
> > (XEN) heap[node=0][zone=23][order=11] 0x00000765000000-0x00000765800000
> > (XEN) heap[node=0][zone=23][order=12] 0x00000764000000-0x00000765000000
> > (XEN) heap[node=0][zone=23][order=14] 0x00000760000000-0x00000764000000
> > (XEN) heap[node=0][zone=23][order=17] 0x00000740000000-0x00000760000000
> > (XEN) heap[node=0][zone=23][order=18] 0x00000540000000-0x00000580000000
> > (XEN) heap[node=0][zone=23][order=18] 0x00000500000000-0x00000540000000
> > (XEN) heap[node=0][zone=23][order=18] 0x00000640000000-0x00000680000000
> > (XEN) heap[node=0][zone=23][order=18] 0x00000600000000-0x00000640000000
> > (XEN) heap[node=0][zone=23][order=18] 0x00000700000000-0x00000740000000
> > 
> > Yes, you already pointed out this disadvantage, so if it is an acceptable
> > downside, I am absolutely OK.
> > 
> > 
> > 3. Common code updates. There is a question how to properly make a
> > connection between common allocator internals and Arm's code for creating
> > DT. I didn’t come up with anything better
> > than creating for_each_avail_page() for invoking a callback with page and
> > its order.
> > 
> > **********
> > 
> > Below the proposed changes on top of the initial patch, would this be
> > acceptable in general?
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 523eb19..1e58fc5 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -753,16 +753,33 @@ static int __init add_ext_regions(unsigned long s,
> > unsigned long e, void *data)
> >      return 0;
> >  }
> > 
> > +static int __init add_unalloc_mem(struct page_info *page, unsigned int
> > order,
> > +                                  void *data)
> > +{
> > +    struct rangeset *unalloc_mem = data;
> > +    paddr_t start, end;
> > +    int res;
> > +
> > +    start = page_to_maddr(page);
> > +    end = start + pfn_to_paddr(1UL << order);
> > +    res = rangeset_add_range(unalloc_mem, start, end - 1);
> > +    if ( res )
> > +    {
> > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > +               start, end);
> > +        return res;
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> >  #define EXT_REGION_START   0x40000000ULL
> >  #define EXT_REGION_END     0x80003fffffffULL
> > 
> > -static int __init find_unallocated_memory(const struct kernel_info *kinfo,
> > -                                          struct meminfo *ext_regions)
> > +static int __init find_unallocated_memory(struct meminfo *ext_regions)
> >  {
> > -    const struct meminfo *assign_mem = &kinfo->mem;
> >      struct rangeset *unalloc_mem;
> >      paddr_t start, end;
> > -    unsigned int i;
> >      int res;
> > 
> >      dt_dprintk("Find unallocated memory for extended regions\n");
> > @@ -771,59 +788,9 @@ static int __init find_unallocated_memory(const struct
> > kernel_info *kinfo,
> >      if ( !unalloc_mem )
> >          return -ENOMEM;
> > 
> > -    /* Start with all available RAM */
> > -    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > -    {
> > -        start = bootinfo.mem.bank[i].start;
> > -        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> > -        res = rangeset_add_range(unalloc_mem, start, end - 1);
> > -        if ( res )
> > -        {
> > -            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > -                   start, end);
> > -            goto out;
> > -        }
> > -    }
> > -
> > -    /* Remove RAM assigned to Dom0 */
> > -    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > -    {
> > -        start = assign_mem->bank[i].start;
> > -        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> > -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > -        if ( res )
> > -        {
> > -            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > -                   start, end);
> > -            goto out;
> > -        }
> > -    }
> > -
> > -    /* Remove reserved-memory regions */
> > -    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > -    {
> > -        start = bootinfo.reserved_mem.bank[i].start;
> > -        end = bootinfo.reserved_mem.bank[i].start +
> > -            bootinfo.reserved_mem.bank[i].size;
> > -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > -        if ( res )
> > -        {
> > -            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > -                   start, end);
> > -            goto out;
> > -        }
> > -    }
> > -
> > -    /* Remove grant table region */
> > -    start = kinfo->gnttab_start;
> > -    end = kinfo->gnttab_start + kinfo->gnttab_size;
> > -    res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > +    res = for_each_avail_page(add_unalloc_mem, unalloc_mem);
> >      if ( res )
> > -    {
> > -        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > -               start, end);
> >          goto out;
> > -    }
> > 
> >      start = EXT_REGION_START;
> >      end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
> > @@ -840,8 +807,7 @@ out:
> >      return res;
> >  }
> > 
> > -static int __init find_memory_holes(const struct kernel_info *kinfo,
> > -                                    struct meminfo *ext_regions)
> > +static int __init find_memory_holes(struct meminfo *ext_regions)
> >  {
> >      struct dt_device_node *np;
> >      struct rangeset *mem_holes;
> > @@ -961,9 +927,9 @@ static int __init make_hypervisor_node(struct domain *d,
> >      else
> >      {
> >          if ( !is_iommu_enabled(d) )
> > -            res = find_unallocated_memory(kinfo, ext_regions);
> > +            res = find_unallocated_memory(ext_regions);
> >          else
> > -            res = find_memory_holes(kinfo, ext_regions);
> > +            res = find_memory_holes(ext_regions);
> > 
> >          if ( res )
> >              printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> > diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> > index 8fad139..7cd1020 100644
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -1572,6 +1572,40 @@ static int reserve_heap_page(struct page_info *pg)
> > 
> >  }
> > 
> > +/* TODO heap_lock? */
> > +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, void
> > *),
> > +                        void *data)
> > +{
> > +    unsigned int node, zone, order;
> > +    int ret;
> > +
> > +    for ( node = 0; node < MAX_NUMNODES; node++ )
> > +    {
> > +        if ( !avail[node] )
> > +            continue;
> > +
> > +        for ( zone = 0; zone < NR_ZONES; zone++ )
> > +        {
> > +            for ( order = 0; order <= MAX_ORDER; order++ )
> > +            {
> > +                struct page_info *head, *tmp;
> > +
> > +                if ( page_list_empty(&heap(node, zone, order)) )
> > +                    continue;
> > +
> > +                page_list_for_each_safe ( head, tmp, &heap(node, zone,
> > order) )
> > +                {
> > +                    ret = cb(head, order, data);
> > +                    if ( ret )
> > +                        return ret;
> > +                }
> > +            }
> > +        }
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> >  int offline_page(mfn_t mfn, int broken, uint32_t *status)
> >  {
> >      unsigned long old_info = 0;
> > diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> > index 667f9da..64dd3e2 100644
> > --- a/xen/include/xen/mm.h
> > +++ b/xen/include/xen/mm.h
> > @@ -123,6 +123,9 @@ unsigned int online_page(mfn_t mfn, uint32_t *status);
> >  int offline_page(mfn_t mfn, int broken, uint32_t *status);
> >  int query_page_offline(mfn_t mfn, uint32_t *status);
> > 
> > +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, void
> > *),
> > +                        void *data);
> > +
> >  void heap_init_late(void);
> > 
> >  int assign_pages(
> 
> 
> I am sorry, but may I please clarify regarding that? Whether we will go this
> new direction (free memory in page allocator) or leave things as they are
> (bootinfo.mem - kinfo->mem - bootinfo.reserved_mem - kinfo->gnttab). This is
> only one still unclear moment to me in current patch before preparing V3.

I think both approaches are fine. Your original approach leads to better
results in terms of extended regions but the difference is not drastic.
The original approach requires more code (bad) but probably less CPU
cycles (good).

Personally I am fine either way but as Julien was the one to provide
feedback on this it would be best to get his opinion.

But in the meantime I think it is OK to send a v3 so that we can review
the rest.
--8323329-2026154566-1632415038=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 16:59:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 16:59:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194504.346502 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTS4L-0007sK-AZ; Thu, 23 Sep 2021 16:59:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194504.346502; Thu, 23 Sep 2021 16:59:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTS4L-0007sD-7M; Thu, 23 Sep 2021 16:59:33 +0000
Received: by outflank-mailman (input) for mailman id 194504;
 Thu, 23 Sep 2021 16:59:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTS4J-0007s7-7u
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 16:59:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9dda70aa-1c8f-11ec-ba67-12813bfff9fa;
 Thu, 23 Sep 2021 16:59:30 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D16FF60F43;
 Thu, 23 Sep 2021 16:59:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9dda70aa-1c8f-11ec-ba67-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632416369;
	bh=1qSkwmGiBJ/LOLd7+ISnhDLvK4uxCuXn7rSgFt0fOaQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bGczcaJ3Vpz761wfat5RBG4kgrQAev60iow7cTNS3zhaSNh6yPkdt1yyGnkXIhgv3
	 VZeFWdb+XhCD0DUO3YI2pZGggQIgtyUgmT4rTXMiZVWBUL4BaUUIwmN3P6v/H/fZ4V
	 O4B/tXuOENgOKUExUqs0XRwhbw825H3+iQCH6R2EdHKplDhhLTkfbm4WvUJnj6fJMa
	 cyDKrvYPu4qRPWiLsQqhDM3P5YanlAuU0ExUJRG00tr3jBxdaK8qNuyXaA4X3+OJ87
	 puSTTaQS27QVpJOx2dkngNCNP25Wig7RV0ka+5ylIQvRDNObjh1yzTCvIq877Ma5/t
	 oybsANrq9WnfQ==
Date: Thu, 23 Sep 2021 09:59:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using
 EFI boot
In-Reply-To: <2C0E480C-DF1C-4AAD-89F5-99D52B016970@arm.com>
Message-ID: <alpine.DEB.2.21.2109230943510.17979@sstabellini-ThinkPad-T480s>
References: <20210922141341.42288-1-luca.fancellu@arm.com> <20210922141341.42288-3-luca.fancellu@arm.com> <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s> <2C0E480C-DF1C-4AAD-89F5-99D52B016970@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1527541655-1632416369=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1527541655-1632416369=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 23 Sep 2021, Luca Fancellu wrote:
> >> +/*
> >> + * Binaries will be translated into bootmodules, the maximum number for them is
> >> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> >> + */
> >> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> >> +static struct file __initdata dom0less_file;
> >> +static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
> >> +static unsigned int __initdata dom0less_modules_available =
> >> +                               MAX_DOM0LESS_MODULES;
> >> +static unsigned int __initdata dom0less_modules_idx;
> > 
> > This is a lot better!
> > 
> > We don't need both dom0less_modules_idx and dom0less_modules_available.
> > You can just do:
> > 
> > #define dom0less_modules_available (MAX_DOM0LESS_MODULES - dom0less_modules_idx)
> > static unsigned int __initdata dom0less_modules_idx;
> > 
> > But maybe we can even get rid of dom0less_modules_available entirely?
> > 
> > We can change the check at the beginning of allocate_dom0less_file to:
> > 
> > if ( dom0less_modules_idx == dom0less_modules_available )
> >    blexit
> > 
> > Would that work?
> 
> I thought about it but I think they need to stay, because dom0less_modules_available is the
> upper bound for the additional dom0less modules (it is decremented each time a dom0 module
> Is added), instead dom0less_modules_idx is the typical index for the array of dom0less modules.

[...]


> >> +    /*
> >> +     * Check if there is any space left for a domU module, the variable
> >> +     * dom0less_modules_available is updated each time we use read_file(...)
> >> +     * successfully.
> >> +     */
> >> +    if ( !dom0less_modules_available )
> >> +        blexit(L"No space left for domU modules");
> > 
> > This is the check that could be based on dom0less_modules_idx
> > 
> 
> The only way I see to have it based on dom0less_modules_idx will be to compare it
> to the amount of modules still available, that is not constant because it is dependent
> on how many dom0 modules are loaded, so still two variables needed.
> Don’t know if I’m missing something.

I think I understand where the confusion comes from. I am appending a
small patch to show what I had in mind. We are already accounting for
Xen and the DTB when declaring MAX_DOM0LESS_MODULES (MAX_MODULES - 2).
The other binaries are the Dom0 kernel and ramdisk, however, in my setup
they don't trigger a call to handle_dom0less_module_node because they
are compatible xen,linux-zimage and xen,linux-initrd.

However, the Dom0 kernel and ramdisk can be also compatible
multiboot,kernel and multiboot,ramdisk. If that is the case, then they
would indeed trigger a call to handle_dom0less_module_node.

I think that is not a good idea: a function called
handle_dom0less_module_node should only be called for dom0less modules
(domUs) and not dom0.

But from the memory consumption point of view, it would be better
actually to catch dom0 modules too as you intended. In that case we need to:

- add a check for xen,linux-zimage and xen,linux-initrd in
  handle_dom0less_module_node also

- rename handle_dom0less_domain_node, handle_dom0less_module_node,
  dom0less_file, dom0less_modules, dom0less_modules_idx to something
  else more generic


For instance they could be called:

handle_domain_node
handle_module_node
module_file
modules
modules_idx




diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index e2b007ece0..812d0bd607 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -22,8 +22,6 @@ typedef struct {
 #define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
 static struct file __initdata dom0less_file;
 static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
-static unsigned int __initdata dom0less_modules_available =
-                               MAX_DOM0LESS_MODULES;
 static unsigned int __initdata dom0less_modules_idx;
 
 #define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
@@ -592,14 +590,6 @@ static void __init efi_arch_handle_module(const struct file *file,
          * stop here.
          */
         blexit(L"Unknown module type");
-
-    /*
-     * dom0less_modules_available is decremented here because for each dom0
-     * file added, there will be an additional bootmodule, so the number
-     * of dom0less module files will be decremented because there is
-     * a maximum amount of bootmodules that can be loaded.
-     */
-    dom0less_modules_available--;
 }
 
 /*
@@ -643,7 +633,7 @@ static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
      * dom0less_modules_available is updated each time we use read_file(...)
      * successfully.
      */
-    if ( !dom0less_modules_available )
+    if ( dom0less_modules_idx == MAX_DOM0LESS_MODULES )
         blexit(L"No space left for domU modules");
 
     module_name.s = (char*) name;
--8323329-1527541655-1632416369=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 17:02:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 17:02:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194509.346513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTS6u-0000nx-Nt; Thu, 23 Sep 2021 17:02:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194509.346513; Thu, 23 Sep 2021 17:02:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTS6u-0000nq-Kg; Thu, 23 Sep 2021 17:02:12 +0000
Received: by outflank-mailman (input) for mailman id 194509;
 Thu, 23 Sep 2021 17:02:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTS6s-0000ng-Ke; Thu, 23 Sep 2021 17:02:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTS6s-0004tk-Fh; Thu, 23 Sep 2021 17:02:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTS6s-0006ay-2Y; Thu, 23 Sep 2021 17:02:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTS6s-0003oV-21; Thu, 23 Sep 2021 17:02:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DBPHKrMlDAS+Lr3iHpvNNiEJHGs9SNYth3xipW+LxZc=; b=j1OHtJkftcNdZ3/hHDW2TKO5gu
	/VHe0aDq1knYbVLpxGGWr2hXrLnyJH4PwJKb4g2WDKMLmhKxkyeE0/j10eyRp4Sx7xCnHMfSWGC89
	AsNtn+6EjLF/9z5ET5//Aoa6i7V9BbZQAWUw+R0+PmTlRZyTZGNSABLI5rSm4vjlPIAA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165161-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165161: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:guest-localmigrate:fail:allowable
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=604be1b333b1b66052ab9b0133f156890549a4f0
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 17:02:10 +0000

flight 165161 xen-unstable real [real]
flight 165167 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165161/
http://logs.test-lab.xenproject.org/osstest/logs/165167/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine      4 memdisk-try-append  fail pass in 165167-retest

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-freebsd11-amd64 17 guest-localmigrate fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  604be1b333b1b66052ab9b0133f156890549a4f0
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   12 days
Failing since        164951  2021-09-12 00:14:36 Z   11 days   23 attempts
Testing same since   165161  2021-09-23 03:54:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       fail    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 899 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 17:09:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 17:09:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194517.346526 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSDP-0001tJ-M3; Thu, 23 Sep 2021 17:08:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194517.346526; Thu, 23 Sep 2021 17:08:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSDP-0001tC-Iz; Thu, 23 Sep 2021 17:08:55 +0000
Received: by outflank-mailman (input) for mailman id 194517;
 Thu, 23 Sep 2021 17:08:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y7SS=ON=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTSDO-0001t4-1k
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 17:08:54 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe1e::61d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7d4352b4-3a26-4f50-b022-1eb66a45c48b;
 Thu, 23 Sep 2021 17:08:49 +0000 (UTC)
Received: from AM5PR0202CA0023.eurprd02.prod.outlook.com
 (2603:10a6:203:69::33) by DB8PR08MB5530.eurprd08.prod.outlook.com
 (2603:10a6:10:11f::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep
 2021 17:08:47 +0000
Received: from VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:69:cafe::8e) by AM5PR0202CA0023.outlook.office365.com
 (2603:10a6:203:69::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Thu, 23 Sep 2021 17:08:47 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT016.mail.protection.outlook.com (10.152.18.115) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 17:08:46 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Thu, 23 Sep 2021 17:08:44 +0000
Received: from 82d30b1a9384.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A02E5A4C-21A1-4D58-940B-CF9D6878611B.1; 
 Thu, 23 Sep 2021 17:08:33 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 82d30b1a9384.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 23 Sep 2021 17:08:33 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6583.eurprd08.prod.outlook.com (2603:10a6:20b:33f::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep
 2021 17:08:31 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.014; Thu, 23 Sep 2021
 17:08:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d4352b4-3a26-4f50-b022-1eb66a45c48b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KuOdNB7f5p23V0o+yeBa2L7aF0tohMacHfHwB8K9MHw=;
 b=Vgp3RTYOQZiwtT/4drmOkpAQtAxUT2xRQzQrd7xE8iB3LJAEHnIr/NPki6+uq+N7bEHPliSCmECJdV2FXHpEznbnpZAvEsUfEsvCZ545lZ/MRDuP7CGFCSxB+hau3rZ/4viih8mgldSM9zbwwN6yU4zSbzQhLP3/ik/d2FHu1mA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3f195421e89d97e7
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RLPEllDXCozI11MhuKPQDth8R7cKo840TLtQXvSJPsvuUugULqYkMgox4kKicd0raAIIiOp7XEs9RW8fmCzqgqgV8VFtRfNW9ofvZRvuexa4/Paqcurr9iPoB0q4BRyfVtbdr1AaZXV6sO2Hcu2xVvewQ8gvcrC307XQCcMsEPkZ/jcz7ZvjBwKB76Z+hzFqo9JNKXPDn034yZ/kaCR2BTLAJpa2aStSZs/VBuwgYNMQ2Rct0RKZUxaWqL1mIrEd7esdTQrLA7vrfufUmEi2PIuNE+JwPT9WUCniYE7HZSxf5bPWVda86Fa+MWofkmyXklIEX6zdZQR7K5J8gU/UWw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KuOdNB7f5p23V0o+yeBa2L7aF0tohMacHfHwB8K9MHw=;
 b=UmPVsQ1uCW6H2ZZuW3JA6EcEtTQx7hqjNittDGX6j+eL1U5LA/b+O3WJochvzlELGrJ7LwhJaSdECZSFedKnr5A5PxP0ga3IaZrmWa6V1BjRlyJFjN8msT84pdQLEcoBnohe4ZBcraaLu6062nuOwV+M/EEaexekrdPbyvDczSay3SUlAObBRzUCCpf15HNfIiHrJdC7Df9l1PyCxRdbYtxDKcSlRD2E4Pw0sqL0fObJ7MTQsmg/qd3S0Gh2prSpuunpECgivuyFOf9Fmfqv1Y+LKhUlvEFClCiidTzlY3WkRRMUQg/KjmXI34ysx1N0+70Z6sLX0516cRtNFh4qIQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=KuOdNB7f5p23V0o+yeBa2L7aF0tohMacHfHwB8K9MHw=;
 b=Vgp3RTYOQZiwtT/4drmOkpAQtAxUT2xRQzQrd7xE8iB3LJAEHnIr/NPki6+uq+N7bEHPliSCmECJdV2FXHpEznbnpZAvEsUfEsvCZ545lZ/MRDuP7CGFCSxB+hau3rZ/4viih8mgldSM9zbwwN6yU4zSbzQhLP3/ik/d2FHu1mA=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXr6aPsmXcRwLbA0y2fuTM/iBxWauw4OaAgAD7DQA=
Date: Thu, 23 Sep 2021 17:08:31 +0000
Message-ID: <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: be1f51e2-002f-41eb-b7dd-08d97eb4ce27
x-ms-traffictypediagnostic: AS8PR08MB6583:|DB8PR08MB5530:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB55302EBC3C116ACDEAE27541FCA39@DB8PR08MB5530.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:5797;OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 eSRdtnGiOO1a1Iib0I2X3XdC9fC3uIMFhKrDsUhN9xgjSlDW+Ta3P3nMGuydX7EYB9aJawU6w8Fij4Zzs1QX/wsOV5QoLg7EUaqiSEdI3kRoIYq8vSdl1FHiu2jN2LYUiDtaJqo36U5ptvVwc494X5VENQjRMK1jNkjHKVod6Npn7OunKKKk9RmXNas030j8dPTvY8H6DgQVjYDeZ2SE4mVTvbDDLHOI9W9pL5UNCCXIawCc5WHSO6L54XJIlSIKY+XpgnRCbdBhIFvNo+0Upa0OmqcclgatcmwLtIpUX6YcpkRaTwxgaBac9rMcfvJEYSeC1AfhTpdN5CGW8GcqtNRhqwJ5KUuON0e912FlvdkVhkW0UYR7Ll9BWXs485rJEFzz15ar1BXPTX7GJAgcyDyT78Y2GZUZl1b7qyemXo4fXEdIiQe6ItgF137l+j46+grKsipw0klxbzwv3IC0bIQJwEGWijFm8xXHNLamTQOSfvS/KfH4Jq0WPqWTE20yJbQrCl5kz+yCc3LGlzM4pThNWOJkvgewHDNP8/yqkEYdMucJBBdp/HF8VzdYHpGQkl/06c7mAfBxMb4FSaociMoybQF8qI9eQrKG/2Wd+SY7Yj/yymqUsek+ak7ZjqydiXI5CLrJfZly34Y5wsVdnJfpTBLPVxQxY5ftIyiL2M+yPenDPNlky5aaA9/wOz/rHEfOI5zBV00EjpnK9Y6i/qfJWC7oDVdYxJinFuXplKAUUbQEipqnLP1sUfmBDNbIWGAjUyc+CR1Mp2xB1vcpUP1Bazmwx/7fO6Eq5RxYxyjbXc8IiauuLOi2pecB2qRfasapN9OhOALskDO7ONjp3e+T5E/Qe/Xcyk1I/Vysm4W3RPoBVH/wdzCK3Sjx0tkZ/gonYTOsRP3oKxeFRhogAA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(36756003)(6512007)(71200400001)(8936002)(122000001)(83380400001)(2906002)(4326008)(6486002)(186003)(66946007)(66476007)(64756008)(38100700002)(6506007)(26005)(508600001)(8676002)(86362001)(966005)(53546011)(54906003)(316002)(6916009)(33656002)(30864003)(38070700005)(66556008)(5660300002)(91956017)(66446008)(76116006)(2616005)(2004002)(45980500001)(579004);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <99F9FA584904B648826048C51F9A327D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6583
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	728862c3-54c8-4f8d-c933-08d97eb4c507
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	21TKRWdlWlL0XShL2mKhxt2y3R/LjfPLD/6vhKlyCruM5EFADXfLS4QiiUjlslezOFXA3+cjzXdXro5b38kolIoCVFFOhe2mfBr6NhXFS9RMS6HhFk4zlX3rZeGisUP8EkcZ8Sud19p3gzrJTl0vUrr3POQYKucz3vf7qz/MGxOJVEbhRP3Ain2CakbwyRKtdZ/oxBeSV9CLq9v264w/JyeQPW8rGy02zSsIugciwo5H19kUMZflY4qjqoZ6b888Y5+1xf+K1vhnNYTDhfVfHHId3WvQaWUvWw2a2mHyfuGVlvj5CY6ixSVDyuc9foh5G8ifctMeSrEEUmI3h3SOthpZPXbuQIVZSSKlR7Sub8utASRxgmFAaZlGgsqr9+9DpoJGUun+n3MN9TIlnDU3JS9aHcLn4qpol0fJCeBCw9DdPAHdN0TatYjdS2gcn9/fHcUvzribCKvEFfHLQUZ83xX9oPRgle/Ix6quwB6JIHj9n91h6ivDMlxCQdMu0IkpBOrfyGFF23yjWJ/CLsvUqMtATAvBOP6uPcSQiWnv66BZefq1I+uT9V8Af4cUQfZcvnITFnx1C3Gp73kQui9JO0X/yDgIRKU9uSvllUJfcW2NA+HUAiqjmoJswCk1XAijK/cD60ll0B5u9xoqan6/gLMIRXR8p23DAGhJW/z4U+XcCIiIlBEGi0aaYTN+ij7OSkQmSzY5UBxody1QAh69fRTu0ARGclBwqx17+zybdj+CllfZ4deupR7gT9aEztVcqM3LfjsjdcffS6vb7G/2YXjcxp6HytySzzkpaiF7V7VYFfZQY9owbuTnpDCB79QEJSCFJ4WPP5H+fo7VSekffQkOhzb0Bm/8y1yg3z9uVqk=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(966005)(508600001)(2616005)(83380400001)(26005)(8676002)(6862004)(2906002)(186003)(36756003)(336012)(6512007)(54906003)(86362001)(316002)(33656002)(107886003)(81166007)(53546011)(5660300002)(30864003)(8936002)(356005)(6486002)(6506007)(47076005)(36860700001)(82310400003)(70586007)(70206006)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 17:08:46.6075
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: be1f51e2-002f-41eb-b7dd-08d97eb4ce27
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5530

SGkgU3RlZmFubywNCg0KPiBPbiAyMyBTZXAgMjAyMSwgYXQgMzowOSBhbSwgU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPiB3cm90ZToNCj4gDQo+IE9uIFdlZCwgMjIg
U2VwIDIwMjEsIFJhaHVsIFNpbmdoIHdyb3RlOg0KPj4gWEVOIGR1cmluZyBib290IHdpbGwgcmVh
ZCB0aGUgUENJIGRldmljZSB0cmVlIG5vZGUg4oCccmVn4oCdIHByb3BlcnR5DQo+PiBhbmQgd2ls
bCBtYXAgdGhlIFBDSSBjb25maWcgc3BhY2UgdG8gdGhlIFhFTiBtZW1vcnkuDQo+PiANCj4+IEFz
IG9mIG5vdyBvbmx5ICJwY2ktaG9zdC1lY2FtLWdlbmVyaWMiIGNvbXBhdGlibGUgYm9hcmQgaXMg
c3VwcG9ydGVkLg0KPj4gDQo+PiAibGludXgscGNpLWRvbWFpbiIgZGV2aWNlIHRyZWUgcHJvcGVy
dHkgYXNzaWducyBhIGZpeGVkIFBDSSBkb21haW4NCj4+IG51bWJlciB0byBhIGhvc3QgYnJpZGdl
LCBvdGhlcndpc2UgYW4gdW5zdGFibGUgKGFjcm9zcyBib290cykgdW5pcXVlDQo+PiBudW1iZXIg
d2lsbCBiZSBhc3NpZ25lZCBieSBMaW51eC4gWEVOIGFjY2VzcyB0aGUgUENJIGRldmljZXMgYmFz
ZWQgb24NCj4+IFNlZ21lbnQ6QnVzOkRldmljZTpGdW5jdGlvbi4gU2VnbWVudCBudW1iZXIgaW4g
WEVOIGlzIHNhbWUgYXMgZG9tYWluDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4g
QSBzZWdtZW50ICAgICAgICAgICAgIF4gdGhlICAgIF4gYQ0KDQpBY2suDQo+IA0KPiANCj4+IG51
bWJlciBpbiBMaW51eC5TZWdtZW50IG51bWJlciBhbmQgZG9tYWluIG51bWJlciBoYXMgdG8gYmUg
aW4gc3luYw0KPiAgICAgICAgICAgICAgICAgIF4g4oCcIOKAnA0KPiANCkFjay4NCj4+IHRvIGFj
Y2VzcyB0aGUgY29ycmVjdCBQQ0kgZGV2aWNlcy4NCj4+IA0KPj4gWEVOIHdpbGwgcmVhZCB0aGUg
4oCcbGludXgscGNpLWRvbWFpbuKAnSBwcm9wZXJ0eSBmcm9tIHRoZSBkZXZpY2UgdHJlZSBub2Rl
DQo+PiBhbmQgY29uZmlndXJlIHRoZSBob3N0IGJyaWRnZSBzZWdtZW50IG51bWJlciBhY2NvcmRp
bmdseS4gSWYgdGhpcw0KPj4gcHJvcGVydHkgaXMgbm90IGF2YWlsYWJsZSBYRU4gd2lsbCBhbGxv
Y2F0ZSB0aGUgdW5pcXVlIHNlZ21lbnQgbnVtYmVyDQo+PiB0byB0aGUgaG9zdCBicmlkZ2UuDQo+
PiANCj4+IGR0X3BjaV9idXNfZmluZF9kb21haW5fbnIoLi4pIGltcG9ydGVkIGZyb20gdGhlIExp
bnV4IHNvdXJjZSB0cmVlIHdpdGgNCj4+IHNsaWdodCBtb2RpZmljYXRpb24gYmFzZWQgb24gdGFn
IExpbnV4IHY1LjE0LjINCj4+IGNvbW1pdCBiYmRkM2RlMTQ0ZmMxNDJmMmY0Yjk4MzRjOTI0MWNj
NGU3ZjNkM2ZjLg0KPiANCj4gZHRfcGNpX2J1c19maW5kX2RvbWFpbl9uciBpcyBub3QgaW50cm9k
dWNlZCBieSB0aGlzIHBhdGNoIGFueSBsb25nZXINCg0KQWNrLg0KPiANCj4gDQo+PiBTaWduZWQt
b2ZmLWJ5OiBSYWh1bCBTaW5naCA8cmFodWwuc2luZ2hAYXJtLmNvbT4NCj4+IC0tLQ0KPj4gQ2hh
bmdlIGluIHYyOg0KPj4gLSBBZGQgbW9yZSBpbmZvIGluIGNvbW1pdCBtc2cNCj4+IC0gQWRkIGNh
bGxiYWNrIHRvIHBhcnNlIHJlZ2lzdGVyIGluZGV4Lg0KPj4gLSBNZXJnZSBwYXRjaCBwY2lfZWNh
bV9vcGVyYXRpb24gaW50byB0aGlzIHBhdGNoIHRvIGF2b2lkIGNvbmZ1c2lvbg0KPj4gLSBBZGQg
bmV3IHN0cnVjdCBpbiBzdHJ1Y3QgZGV2aWNlIGZvciBtYXRjaCB0YWJsZQ0KPj4gLS0tDQo+PiB4
ZW4vYXJjaC9hcm0vZGV2aWNlLmMgICAgICAgICAgICAgICB8ICAgMiArDQo+PiB4ZW4vYXJjaC9h
cm0vcGNpL01ha2VmaWxlICAgICAgICAgICB8ICAgNSArDQo+PiB4ZW4vYXJjaC9hcm0vcGNpL2Vj
YW0uYyAgICAgICAgICAgICB8ICA2MCArKysrKysrDQo+PiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1h
Y2Nlc3MuYyAgICAgICB8ICA4MyArKysrKysrKysNCj4+IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhv
c3QtY29tbW9uLmMgIHwgMjU0ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+IHhlbi9h
cmNoL2FybS9wY2kvcGNpLWhvc3QtZ2VuZXJpYy5jIHwgIDQyICsrKysrDQo+PiB4ZW4vaW5jbHVk
ZS9hc20tYXJtL2RldmljZS5oICAgICAgICB8ICAgMiArDQo+PiB4ZW4vaW5jbHVkZS9hc20tYXJt
L3BjaS5oICAgICAgICAgICB8ICA1OSArKysrKysrDQo+PiA4IGZpbGVzIGNoYW5nZWQsIDUwNyBp
bnNlcnRpb25zKCspDQo+PiBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3BjaS9lY2Ft
LmMNCj4+IGNyZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1hY2Nlc3MuYw0K
Pj4gY3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMN
Cj4+IGNyZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMu
Yw0KPj4gDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL2RldmljZS5jIGIveGVuL2FyY2gv
YXJtL2RldmljZS5jDQo+PiBpbmRleCA3MGNkNmMxYTE5Li4xOTdiYjNjNmU4IDEwMDY0NA0KPj4g
LS0tIGEveGVuL2FyY2gvYXJtL2RldmljZS5jDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vZGV2aWNl
LmMNCj4+IEBAIC00NCw2ICs0NCw4IEBAIGludCBfX2luaXQgZGV2aWNlX2luaXQoc3RydWN0IGR0
X2RldmljZV9ub2RlICpkZXYsIGVudW0gZGV2aWNlX2NsYXNzIGNsYXNzLA0KPj4gICAgICAgICB7
DQo+PiAgICAgICAgICAgICBBU1NFUlQoZGVzYy0+aW5pdCAhPSBOVUxMKTsNCj4+IA0KPj4gKyAg
ICAgICAgICAgIGRldi0+ZGV2Lm9mX21hdGNoX3RhYmxlID0gZGVzYy0+ZHRfbWF0Y2g7DQo+PiAr
DQo+PiAgICAgICAgICAgICByZXR1cm4gZGVzYy0+aW5pdChkZXYsIGRhdGEpOw0KPj4gICAgICAg
ICB9DQo+PiANCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL01ha2VmaWxlIGIveGVu
L2FyY2gvYXJtL3BjaS9NYWtlZmlsZQ0KPj4gaW5kZXggYTk4MDM1ZGY0Yy4uZTg2ZjJiNDZmZCAx
MDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+ICsrKyBiL3hlbi9h
cmNoL2FybS9wY2kvTWFrZWZpbGUNCj4+IEBAIC0xICsxLDYgQEANCj4+IG9iai15ICs9IHBjaS5v
DQo+PiArb2JqLXkgKz0gcGNpLWFjY2Vzcy5vDQo+PiArb2JqLXkgKz0gcGNpLm8NCj4gDQo+IEFk
ZGVkIHR3aWNlPw0KDQpBY2suIA0KPiANCj4gDQo+PiArb2JqLXkgKz0gcGNpLWhvc3QtZ2VuZXJp
Yy5vDQo+PiArb2JqLXkgKz0gcGNpLWhvc3QtY29tbW9uLm8NCj4+ICtvYmoteSArPSBlY2FtLm8N
Cj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyBiL3hlbi9hcmNoL2FybS9w
Y2kvZWNhbS5jDQo+PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPj4gaW5kZXggMDAwMDAwMDAwMC4u
OWI4OGIxY2VkYQ0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9l
Y2FtLmMNCj4+IEBAIC0wLDAgKzEsNjAgQEANCj4+ICsvKg0KPj4gKyAqIEJhc2VkIG9uIExpbnV4
IGRyaXZlcnMvcGNpL2VjYW0uYw0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGZyZWUg
c29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBp
dCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNp
b24gMiBhcw0KPj4gKyAqIHB1Ymxpc2hlZCBieSB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9u
Lg0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3Bl
IHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZ
OyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFC
SUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiAr
ICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKg0K
Pj4gKyAqIFlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFs
IFB1YmxpYyBMaWNlbnNlDQo+PiArICogYWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uICBJZiBub3Qs
IHNlZSA8aHR0cDovL3d3dy5nbnUub3JnL2xpY2Vuc2VzLz4uDQo+PiArICovDQo+PiArDQo+PiAr
I2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5jbHVkZSA8eGVuL3NjaGVkLmg+DQo+PiArDQo+
PiArLyoNCj4+ICsgKiBGdW5jdGlvbiB0byBpbXBsZW1lbnQgdGhlIHBjaV9vcHMgLT5tYXBfYnVz
IG1ldGhvZC4NCj4+ICsgKi8NCj4+ICt2b2lkIF9faW9tZW0gKnBjaV9lY2FtX21hcF9idXMoc3Ry
dWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1aW50MzJfdCBzYmRmLCB1aW50MzJfdCB3aGVyZSkNCj4+ICt7DQo+PiArICAgIGNv
bnN0IHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnID0gYnJpZGdlLT5jZmc7DQo+PiArICAg
IGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMgKm9wcyA9IGJyaWRnZS0+c3lzZGF0YTsNCj4+ICsg
ICAgdW5zaWduZWQgaW50IGRldmZuX3NoaWZ0ID0gb3BzLT5idXNfc2hpZnQgLSA4Ow0KPj4gKyAg
ICB2b2lkIF9faW9tZW0gKmJhc2U7DQo+PiArDQo+PiArICAgIHVuc2lnbmVkIGludCBidXNuID0g
UENJX0JVUyhzYmRmKTsNCj4+ICsNCj4+ICsgICAgaWYgKCBidXNuIDwgY2ZnLT5idXNuX3N0YXJ0
IHx8IGJ1c24gPiBjZmctPmJ1c25fZW5kICkNCj4+ICsgICAgICAgIHJldHVybiBOVUxMOw0KPj4g
Kw0KPj4gKyAgICBidXNuIC09IGNmZy0+YnVzbl9zdGFydDsNCj4+ICsgICAgYmFzZSA9IGNmZy0+
d2luICsgKGJ1c24gPDwgb3BzLT5idXNfc2hpZnQpOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gYmFz
ZSArIChQQ0lfREVWRk4yKHNiZGYpIDw8IGRldmZuX3NoaWZ0KSArIHdoZXJlOw0KPj4gK30NCj4+
ICsNCj4+ICsvKiBFQ0FNIG9wcyAqLw0KPj4gK2NvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMgcGNp
X2dlbmVyaWNfZWNhbV9vcHMgPSB7DQo+PiArICAgIC5idXNfc2hpZnQgID0gMjAsDQo+PiArICAg
IC5wY2lfb3BzICAgID0gew0KPj4gKyAgICAgICAgLm1hcF9idXMgICAgICAgICAgICAgICAgPSBw
Y2lfZWNhbV9tYXBfYnVzLA0KPj4gKyAgICAgICAgLnJlYWQgICAgICAgICAgICAgICAgICAgPSBw
Y2lfZ2VuZXJpY19jb25maWdfcmVhZCwNCj4+ICsgICAgICAgIC53cml0ZSAgICAgICAgICAgICAg
ICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3dyaXRlLA0KPj4gKyAgICB9DQo+PiArfTsNCj4+ICsN
Cj4+ICsvKg0KPj4gKyAqIExvY2FsIHZhcmlhYmxlczoNCj4+ICsgKiBtb2RlOiBDDQo+PiArICog
Yy1maWxlLXN0eWxlOiAiQlNEIg0KPj4gKyAqIGMtYmFzaWMtb2Zmc2V0OiA0DQo+PiArICogdGFi
LXdpZHRoOiA0DQo+PiArICogaW5kZW50LXRhYnMtbW9kZTogbmlsDQo+PiArICogRW5kOg0KPj4g
KyAqLw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWFjY2Vzcy5jIGIveGVu
L2FyY2gvYXJtL3BjaS9wY2ktYWNjZXNzLmMNCj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+PiBp
bmRleCAwMDAwMDAwMDAwLi4wNGZlOWZiZjkyDQo+PiAtLS0gL2Rldi9udWxsDQo+PiArKysgYi94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1hY2Nlc3MuYw0KPj4gQEAgLTAsMCArMSw4MyBAQA0KPj4gKy8q
DQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmli
dXRlIGl0IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdO
VSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24gMiBhcw0KPj4gKyAqIHB1Ymxpc2hlZCBi
eSB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLg0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9n
cmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+
PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQg
d2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFCSUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJU
SUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vu
c2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKg0KPj4gKyAqIFlvdSBzaG91bGQgaGF2ZSByZWNl
aXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlDQo+PiArICogYWxv
bmcgd2l0aCB0aGlzIHByb2dyYW0uICBJZiBub3QsIHNlZSA8aHR0cDovL3d3dy5nbnUub3JnL2xp
Y2Vuc2VzLz4uDQo+PiArICovDQo+PiArDQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsj
aW5jbHVkZSA8YXNtL2lvLmg+DQo+PiArDQo+PiArI2RlZmluZSBJTlZBTElEX1ZBTFVFICh+MFUp
DQo+PiArDQo+PiAraW50IHBjaV9nZW5lcmljX2NvbmZpZ19yZWFkKHN0cnVjdCBwY2lfaG9zdF9i
cmlkZ2UgKmJyaWRnZSwgdWludDMyX3Qgc2JkZiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdWludDMyX3QgcmVnLCB1aW50MzJfdCBsZW4sIHVpbnQzMl90ICp2YWx1ZSkNCj4+ICt7
DQo+PiArICAgIHZvaWQgX19pb21lbSAqYWRkciA9IGJyaWRnZS0+b3BzLT5tYXBfYnVzKGJyaWRn
ZSwgc2JkZiwgcmVnKTsNCj4+ICsNCj4+ICsgICAgaWYgKCAhYWRkciApDQo+PiArICAgIHsNCj4+
ICsgICAgICAgICp2YWx1ZSA9IElOVkFMSURfVkFMVUU7DQo+PiArICAgICAgICByZXR1cm4gLUVO
T0RFVjsNCj4+ICsgICAgfQ0KPj4gKw0KPj4gKyAgICBzd2l0Y2ggKCBsZW4gKQ0KPj4gKyAgICB7
DQo+PiArICAgIGNhc2UgMToNCj4+ICsgICAgICAgICp2YWx1ZSA9IHJlYWRiKGFkZHIpOw0KPj4g
KyAgICAgICAgYnJlYWs7DQo+PiArICAgIGNhc2UgMjoNCj4+ICsgICAgICAgICp2YWx1ZSA9IHJl
YWR3KGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7DQo+PiArICAgIGNhc2UgNDoNCj4+ICsgICAg
ICAgICp2YWx1ZSA9IHJlYWRsKGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7DQo+PiArICAgIGRl
ZmF1bHQ6DQo+PiArICAgICAgICBBU1NFUlRfVU5SRUFDSEFCTEUoKTsNCj4+ICsgICAgfQ0KPj4g
Kw0KPj4gKyAgICByZXR1cm4gMDsNCj4+ICt9DQo+PiArDQo+PiAraW50IHBjaV9nZW5lcmljX2Nv
bmZpZ193cml0ZShzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYs
DQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3Qg
bGVuLCB1aW50MzJfdCB2YWx1ZSkNCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXiBj
b2RlIHN0eWxlDQo+IA0KQWNrLiANCj4gDQo+PiArew0KPj4gKyAgICB2b2lkIF9faW9tZW0gKmFk
ZHIgPSBicmlkZ2UtPm9wcy0+bWFwX2J1cyhicmlkZ2UsIHNiZGYsIHJlZyk7DQo+PiArDQo+PiAr
ICAgIGlmICggIWFkZHIgKQ0KPj4gKyAgICAgICAgcmV0dXJuIC1FTk9ERVY7DQo+PiArDQo+PiAr
ICAgIHN3aXRjaCAoIGxlbiApDQo+PiArICAgIHsNCj4+ICsgICAgY2FzZSAxOg0KPj4gKyAgICAg
ICAgd3JpdGViKHZhbHVlLCBhZGRyKTsNCj4+ICsgICAgICAgIGJyZWFrOw0KPj4gKyAgICBjYXNl
IDI6DQo+PiArICAgICAgICB3cml0ZXcodmFsdWUsIGFkZHIpOw0KPj4gKyAgICAgICAgYnJlYWs7
DQo+PiArICAgIGNhc2UgNDoNCj4+ICsgICAgICAgIHdyaXRlbCh2YWx1ZSwgYWRkcik7DQo+PiAr
ICAgICAgICBicmVhazsNCj4+ICsgICAgZGVmYXVsdDoNCj4+ICsgICAgICAgIEFTU0VSVF9VTlJF
QUNIQUJMRSgpOw0KPj4gKyAgICB9DQo+PiArDQo+PiArICAgIHJldHVybiAwOw0KPj4gK30NCj4+
ICsNCj4+ICsvKg0KPj4gKyAqIExvY2FsIHZhcmlhYmxlczoNCj4+ICsgKiBtb2RlOiBDDQo+PiAr
ICogYy1maWxlLXN0eWxlOiAiQlNEIg0KPj4gKyAqIGMtYmFzaWMtb2Zmc2V0OiA0DQo+PiArICog
dGFiLXdpZHRoOiA0DQo+PiArICogaW5kZW50LXRhYnMtbW9kZTogbmlsDQo+PiArICogRW5kOg0K
Pj4gKyAqLw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMgYi94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+PiBuZXcgZmlsZSBtb2Rl
IDEwMDY0NA0KPj4gaW5kZXggMDAwMDAwMDAwMC4uNGJlZWMxNGYyZg0KPj4gLS0tIC9kZXYvbnVs
bA0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KPj4gQEAgLTAs
MCArMSwyNTQgQEANCj4+ICsvKg0KPj4gKyAqIEJhc2VkIG9uIExpbnV4IGRyaXZlcnMvcGNpL2Vj
YW0uYw0KPj4gKyAqIEJhc2VkIG9uIExpbnV4IGRyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpLWhv
c3QtY29tbW9uLmMNCj4+ICsgKiBCYXNlZCBvbiBMaW51eCBkcml2ZXJzL3BjaS9jb250cm9sbGVy
L3BjaS1ob3N0LWdlbmVyaWMuYw0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGZyZWUg
c29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBp
dCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNp
b24gMiBhcw0KPj4gKyAqIHB1Ymxpc2hlZCBieSB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9u
Lg0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3Bl
IHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZ
OyB3aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFC
SUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiAr
ICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKg0K
Pj4gKyAqIFlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFs
IFB1YmxpYyBMaWNlbnNlDQo+PiArICogYWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uICBJZiBub3Qs
IHNlZSA8aHR0cDovL3d3dy5nbnUub3JnL2xpY2Vuc2VzLz4uDQo+PiArICovDQo+PiArDQo+PiAr
I2luY2x1ZGUgPHhlbi9pbml0Lmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5j
bHVkZSA8eGVuL3J3bG9jay5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vc2NoZWQuaD4NCj4+ICsjaW5j
bHVkZSA8eGVuL3ZtYXAuaD4NCj4+ICsNCj4+ICsvKg0KPj4gKyAqIExpc3QgZm9yIGFsbCB0aGUg
cGNpIGhvc3QgYnJpZGdlcy4NCj4+ICsgKi8NCj4+ICsNCj4+ICtzdGF0aWMgTElTVF9IRUFEKHBj
aV9ob3N0X2JyaWRnZXMpOw0KPj4gKw0KPj4gK3N0YXRpYyBhdG9taWNfdCBkb21haW5fbnIgPSBB
VE9NSUNfSU5JVCgtMSk7DQo+PiArDQo+PiArc3RhdGljIGlubGluZSB2b2lkIF9faW9tZW0gKnBj
aV9yZW1hcF9jZmdzcGFjZShwYWRkcl90IHN0YXJ0LCBzaXplX3QgbGVuKQ0KPj4gK3sNCj4+ICsg
ICAgcmV0dXJuIGlvcmVtYXBfbm9jYWNoZShzdGFydCwgbGVuKTsNCj4+ICt9DQo+PiArDQo+PiAr
c3RhdGljIHZvaWQgcGNpX2VjYW1fZnJlZShzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZykN
Cj4+ICt7DQo+PiArICAgIGlmICggY2ZnLT53aW4gKQ0KPj4gKyAgICAgICAgaW91bm1hcChjZmct
Pndpbik7DQo+PiArDQo+PiArICAgIHhmcmVlKGNmZyk7DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRp
YyBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKiBfX2luaXQNCj4+ICtnZW5fcGNpX2luaXQoc3Ry
dWN0IGR0X2RldmljZV9ub2RlICpkZXYsIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMgKm9wcykN
Cj4+ICt7DQo+PiArICAgIGludCBlcnIsIGNmZ19yZWdfaWR4Ow0KPj4gKyAgICB1MzIgYnVzX3Jh
bmdlWzJdOw0KPj4gKyAgICBwYWRkcl90IGFkZHIsIHNpemU7DQo+PiArICAgIHN0cnVjdCBwY2lf
Y29uZmlnX3dpbmRvdyAqY2ZnOw0KPj4gKw0KPj4gKyAgICBjZmcgPSB4emFsbG9jKHN0cnVjdCBw
Y2lfY29uZmlnX3dpbmRvdyk7DQo+PiArICAgIGlmICggIWNmZyApDQo+PiArICAgICAgICByZXR1
cm4gTlVMTDsNCj4+ICsNCj4+ICsgICAgZXJyID0gZHRfcHJvcGVydHlfcmVhZF91MzJfYXJyYXko
ZGV2LCAiYnVzLXJhbmdlIiwgYnVzX3JhbmdlLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBBUlJBWV9TSVpFKGJ1c19yYW5nZSkpOw0KPj4gKyAgICBpZiAoIGVyciAp
IHsNCj4+ICsgICAgICAgIGNmZy0+YnVzbl9zdGFydCA9IDA7DQo+PiArICAgICAgICBjZmctPmJ1
c25fZW5kID0gMHhmZjsNCj4+ICsgICAgICAgIHByaW50ayhYRU5MT0dfSU5GTyAiJXM6IE5vIGJ1
cyByYW5nZSBmb3VuZCBmb3IgcGNpIGNvbnRyb2xsZXJcbiIsDQo+PiArICAgICAgICAgICAgICAg
ZHRfbm9kZV9mdWxsX25hbWUoZGV2KSk7DQo+PiArICAgIH0gZWxzZSB7DQo+PiArICAgICAgICBj
ZmctPmJ1c25fc3RhcnQgPSBidXNfcmFuZ2VbMF07DQo+PiArICAgICAgICBjZmctPmJ1c25fZW5k
ID0gYnVzX3JhbmdlWzFdOw0KPj4gKyAgICAgICAgaWYgKCBjZmctPmJ1c25fZW5kID4gY2ZnLT5i
dXNuX3N0YXJ0ICsgMHhmZiApDQo+PiArICAgICAgICAgICAgY2ZnLT5idXNuX2VuZCA9IGNmZy0+
YnVzbl9zdGFydCArIDB4ZmY7DQo+PiArICAgIH0NCj4+ICsNCj4+ICsgICAgaWYgKCBvcHMtPmNm
Z19yZWdfaW5kZXggKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBjZmdfcmVnX2lkeCA9IG9wcy0+
Y2ZnX3JlZ19pbmRleChkZXYpOw0KPj4gKyAgICAgICAgaWYgKCBjZmdfcmVnX2lkeCA8IDAgKQ0K
Pj4gKyAgICAgICAgICAgIGdvdG8gZXJyX2V4aXQ7DQo+PiArICAgIH0NCj4+ICsgICAgZWxzZQ0K
Pj4gKyAgICAgICAgY2ZnX3JlZ19pZHggPSAwOw0KPj4gKw0KPj4gKyAgICAvKiBQYXJzZSBvdXIg
UENJIGVjYW0gcmVnaXN0ZXIgYWRkcmVzcyAqLw0KPj4gKyAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0
X2FkZHJlc3MoZGV2LCBjZmdfcmVnX2lkeCwgJmFkZHIsICZzaXplKTsNCj4+ICsgICAgaWYgKCBl
cnIgKQ0KPj4gKyAgICAgICAgZ290byBlcnJfZXhpdDsNCj4+ICsNCj4+ICsgICAgY2ZnLT5waHlz
X2FkZHIgPSBhZGRyOw0KPj4gKyAgICBjZmctPnNpemUgPSBzaXplOw0KPj4gKw0KPj4gKyAgICAv
Kg0KPj4gKyAgICAgKiBPbiA2NC1iaXQgc3lzdGVtcywgd2UgZG8gYSBzaW5nbGUgaW9yZW1hcCBm
b3IgdGhlIHdob2xlIGNvbmZpZyBzcGFjZQ0KPj4gKyAgICAgKiBzaW5jZSB3ZSBoYXZlIGVub3Vn
aCB2aXJ0dWFsIGFkZHJlc3MgcmFuZ2UgYXZhaWxhYmxlLiAgT24gMzItYml0LCB3ZQ0KPj4gKyAg
ICAgKiBpb3JlbWFwIHRoZSBjb25maWcgc3BhY2UgZm9yIGVhY2ggYnVzIGluZGl2aWR1YWxseS4N
Cj4+ICsgICAgICogQXMgb2Ygbm93IG9ubHkgNjQtYml0IGlzIHN1cHBvcnRlZCAzMi1iaXQgaXMg
bm90IHN1cHBvcnRlZC4NCj4+ICsgICAgICoNCj4+ICsgICAgICogVE9ETzogRm9yIDMyLWJpdCBp
bXBsZW1lbnQgdGhlIGlvcmVtYXAvaW91bm1hcCBvZiBjb25maWcgc3BhY2UNCj4+ICsgICAgICog
ZHluYW1pY2FsbHkgZm9yIGVhY2ggcmVhZC93cml0ZSBjYWxsLg0KPj4gKyAgICAgKi8NCj4+ICsg
ICAgY2ZnLT53aW4gPSBwY2lfcmVtYXBfY2Znc3BhY2UoY2ZnLT5waHlzX2FkZHIsIGNmZy0+c2l6
ZSk7DQo+PiArICAgIGlmICggIWNmZy0+d2luICkNCj4+ICsgICAgICAgIGdvdG8gZXJyX2V4aXRf
cmVtYXA7DQo+PiArDQo+PiArICAgIHByaW50aygiRUNBTSBhdCBbbWVtIDB4JSJQUklwYWRkciIt
MHglIlBSSXBhZGRyIl0gZm9yIFtidXMgJXgtJXhdIFxuIiwNCj4+ICsgICAgICAgICAgICBjZmct
PnBoeXNfYWRkciwgY2ZnLT5waHlzX2FkZHIgKyBjZmctPnNpemUgLSAxLA0KPj4gKyAgICAgICAg
ICAgIGNmZy0+YnVzbl9zdGFydCwgY2ZnLT5idXNuX2VuZCk7DQo+PiArDQo+PiArICAgIGlmICgg
b3BzLT5pbml0ICkNCj4+ICsgICAgew0KPj4gKyAgICAgICAgZXJyID0gb3BzLT5pbml0KGNmZyk7
DQo+PiArICAgICAgICBpZiAoZXJyKQ0KPiANCj4gY29kZSBzdHlsZQ0KDQpBY2suIA0KPiANCj4g
DQo+PiArICAgICAgICAgICAgZ290byBlcnJfZXhpdDsNCj4+ICsgICAgfQ0KPiANCj4gVGhpcyBp
cyB1bm5lY2Vzc2FyeSBhdCB0aGUgbW9tZW50LCByaWdodD8gQ2FuIHdlIGdldCByaWQgb2Ygb3Bz
LT5pbml0ID8NCg0KTm8gdGhpcyBpcyByZXF1aXJlZCBmb3IgTjFTRFAgYm9hcmQuIFBsZWFzZSBj
aGVjayBiZWxvdyBwYXRjaC4NCmh0dHBzOi8vZ2l0bGFiLmNvbS9yYWhzaW5naC94ZW4taW50ZWdy
YXRpb24vLS9jb21taXQvNjM3OWJhNTc2NGRmMzNkNTc1NDcwODdjZmY0ZmZjMDc4ZGM1MTVkNQ0K
PiANCj4gDQo+PiArICAgIHJldHVybiBjZmc7DQo+PiArDQo+PiArZXJyX2V4aXRfcmVtYXA6DQo+
PiArICAgIHByaW50ayhYRU5MT0dfRVJSICJFQ0FNIGlvcmVtYXAgZmFpbGVkXG4iKTsNCj4gDQo+
IE5vIG5lZWQgZm9yIGVycl9leGl0X3JlbWFwIGFzIHBjaV9lY2FtX2ZyZWUgY2FuIHRha2UgY2Fy
ZSBvZiBib3RoIGNhc2VzLg0KDQpBY2suIA0KPiANCj4+ICtlcnJfZXhpdDoNCj4+ICsgICAgcGNp
X2VjYW1fZnJlZShjZmcpOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gTlVMTDsNCj4+ICt9DQo+PiAr
DQo+PiArc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqcGNpX2FsbG9jX2hvc3RfYnJpZGdlKHZvaWQp
DQo+PiArew0KPj4gKyAgICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UgPSB4emFsbG9j
KHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UpOw0KPj4gKw0KPj4gKyAgICBpZiAoICFicmlkZ2UgKQ0K
Pj4gKyAgICAgICAgcmV0dXJuIE5VTEw7DQo+PiArDQo+PiArICAgIElOSVRfTElTVF9IRUFEKCZi
cmlkZ2UtPm5vZGUpOw0KPj4gKyAgICBicmlkZ2UtPmJ1c19zdGFydCA9IFVJTlQ4X01BWDsNCj4+
ICsgICAgYnJpZGdlLT5idXNfZW5kID0gVUlOVDhfTUFYOw0KPj4gKw0KPj4gKyAgICByZXR1cm4g
YnJpZGdlOw0KPj4gK30NCj4+ICsNCj4+ICt2b2lkIHBjaV9hZGRfaG9zdF9icmlkZ2Uoc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlKQ0KPj4gK3sNCj4+ICsgICAgbGlzdF9hZGRfdGFpbCgm
YnJpZGdlLT5ub2RlLCAmcGNpX2hvc3RfYnJpZGdlcyk7DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRp
YyBpbnQgcGNpX2dldF9uZXdfZG9tYWluX25yKHZvaWQpDQo+PiArew0KPj4gKyAgICByZXR1cm4g
YXRvbWljX2luY19yZXR1cm4oJmRvbWFpbl9ucik7DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRpYyBp
bnQgcGNpX2J1c19maW5kX2RvbWFpbl9ucihzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldikNCj4+
ICt7DQo+PiArICAgIHN0YXRpYyBpbnQgdXNlX2R0X2RvbWFpbnMgPSAtMTsNCj4+ICsgICAgaW50
IGRvbWFpbjsNCj4+ICsNCj4+ICsgICAgZG9tYWluID0gZHRfZ2V0X3BjaV9kb21haW5fbnIoZGV2
KTsNCj4+ICsNCj4+ICsgICAgLyoNCj4+ICsgICAgICogQ2hlY2sgRFQgZG9tYWluIGFuZCB1c2Vf
ZHRfZG9tYWlucyB2YWx1ZXMuDQo+PiArICAgICAqDQo+PiArICAgICAqIElmIERUIGRvbWFpbiBw
cm9wZXJ0eSBpcyB2YWxpZCAoZG9tYWluID49IDApIGFuZA0KPj4gKyAgICAgKiB1c2VfZHRfZG9t
YWlucyAhPSAwLCB0aGUgRFQgYXNzaWdubWVudCBpcyB2YWxpZCBzaW5jZSB0aGlzIG1lYW5zDQo+
PiArICAgICAqIHdlIGhhdmUgbm90IHByZXZpb3VzbHkgYWxsb2NhdGVkIGEgZG9tYWluIG51bWJl
ciBieSB1c2luZw0KPj4gKyAgICAgKiBwY2lfZ2V0X25ld19kb21haW5fbnIoKTsgd2Ugc2hvdWxk
IGFsc28gdXBkYXRlIHVzZV9kdF9kb21haW5zIHRvDQo+PiArICAgICAqIDEsIHRvIGluZGljYXRl
IHRoYXQgd2UgaGF2ZSBqdXN0IGFzc2lnbmVkIGEgZG9tYWluIG51bWJlciBmcm9tDQo+PiArICAg
ICAqIERULg0KPj4gKyAgICAgKg0KPj4gKyAgICAgKiBJZiBEVCBkb21haW4gcHJvcGVydHkgdmFs
dWUgaXMgbm90IHZhbGlkIChpZSBkb21haW4gPCAwKSwgYW5kIHdlDQo+PiArICAgICAqIGhhdmUg
bm90IHByZXZpb3VzbHkgYXNzaWduZWQgYSBkb21haW4gbnVtYmVyIGZyb20gRFQNCj4+ICsgICAg
ICogKHVzZV9kdF9kb21haW5zICE9IDEpIHdlIHNob3VsZCBhc3NpZ24gYSBkb21haW4gbnVtYmVy
IGJ5DQo+PiArICAgICAqIHVzaW5nIHRoZToNCj4+ICsgICAgICoNCj4+ICsgICAgICogcGNpX2dl
dF9uZXdfZG9tYWluX25yKCkNCj4+ICsgICAgICoNCj4+ICsgICAgICogQVBJIGFuZCB1cGRhdGUg
dGhlIHVzZV9kdF9kb21haW5zIHZhbHVlIHRvIGtlZXAgdHJhY2sgb2YgbWV0aG9kIHdlDQo+PiAr
ICAgICAqIGFyZSB1c2luZyB0byBhc3NpZ24gZG9tYWluIG51bWJlcnMgKHVzZV9kdF9kb21haW5z
ID0gMCkuDQo+PiArICAgICAqDQo+PiArICAgICAqIEFsbCBvdGhlciBjb21iaW5hdGlvbnMgaW1w
bHkgd2UgaGF2ZSBhIHBsYXRmb3JtIHRoYXQgaXMgdHJ5aW5nDQo+PiArICAgICAqIHRvIG1peCBk
b21haW4gbnVtYmVycyBvYnRhaW5lZCBmcm9tIERUIGFuZCBwY2lfZ2V0X25ld19kb21haW5fbnIo
KSwNCj4+ICsgICAgICogd2hpY2ggaXMgYSByZWNpcGUgZm9yIGRvbWFpbiBtaXNoYW5kbGluZyBh
bmQgaXQgaXMgcHJldmVudGVkIGJ5DQo+PiArICAgICAqIGludmFsaWRhdGluZyB0aGUgZG9tYWlu
IHZhbHVlIChkb21haW4gPSAtMSkgYW5kIHByaW50aW5nIGENCj4+ICsgICAgICogY29ycmVzcG9u
ZGluZyBlcnJvci4NCj4+ICsgICAgICovDQo+PiArICAgIGlmICggZG9tYWluID49IDAgJiYgdXNl
X2R0X2RvbWFpbnMgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICB1c2VfZHRfZG9tYWlucyA9IDE7
DQo+PiArICAgIH0NCj4+ICsgICAgZWxzZSBpZiAoIGRvbWFpbiA8IDAgJiYgdXNlX2R0X2RvbWFp
bnMgIT0gMSApDQo+PiArICAgIHsNCj4+ICsgICAgICAgIHVzZV9kdF9kb21haW5zID0gMDsNCj4+
ICsgICAgICAgIGRvbWFpbiA9IHBjaV9nZXRfbmV3X2RvbWFpbl9ucigpOw0KPj4gKyAgICB9DQo+
PiArICAgIGVsc2UNCj4+ICsgICAgew0KPj4gKyAgICAgICAgZG9tYWluID0gLTE7DQo+PiArICAg
IH0NCj4+ICsNCj4+ICsgICAgcmV0dXJuIGRvbWFpbjsNCj4+ICt9DQo+PiArDQo+PiAraW50IHBj
aV9ob3N0X2NvbW1vbl9wcm9iZShzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwgY29uc3Qgdm9p
ZCAqZGF0YSkNCj4+ICt7DQo+PiArICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZTsN
Cj4+ICsgICAgc3RydWN0IHBjaV9jb25maWdfd2luZG93ICpjZmc7DQo+PiArICAgIHN0cnVjdCBw
Y2lfZWNhbV9vcHMgKm9wczsNCj4+ICsgICAgY29uc3Qgc3RydWN0IGR0X2RldmljZV9tYXRjaCAq
b2ZfaWQ7DQo+PiArICAgIGludCBlcnI7DQo+PiArDQo+PiArICAgIGlmICggZHRfZGV2aWNlX2Zv
cl9wYXNzdGhyb3VnaChkZXYpICkNCj4+ICsgICAgICAgIHJldHVybiAwOw0KPj4gKw0KPj4gKyAg
ICBvZl9pZCA9IGR0X21hdGNoX25vZGUoZGV2LT5kZXYub2ZfbWF0Y2hfdGFibGUsIGRldi0+ZGV2
Lm9mX25vZGUpOw0KPj4gKyAgICBvcHMgPSAoc3RydWN0IHBjaV9lY2FtX29wcyAqKSBvZl9pZC0+
ZGF0YTsNCj4gDQo+IERvIHdlIHJlYWxseSBuZWVkIGR0X21hdGNoX25vZGUgYW5kIGRldi0+ZGV2
Lm9mX21hdGNoX3RhYmxlIHRvIGdldA0KPiBkdF9kZXZpY2VfbWF0Y2guZGF0YT8NCj4gDQoNCj4g
ZGF0YSBpcyBwYXNzZWQgYXMgYSBwYXJhbWV0ZXIgdG8gcGNpX2hvc3RfY29tbW9uX3Byb2JlLCBp
c24ndCBpdCBlbm91Z2gNCj4gdG8gZG86DQo+IA0KPiBvcHMgPSAoc3RydWN0IHBjaV9lY2FtX29w
cyAqKSBkYXRhOw0KDQpBcyBvZiBub3cgbm90IHJlcXVpcmVkIGJ1dCBpbiBmdXR1cmUgd2UgbWln
aHQgbmVlZCBpdCBpZiB3ZSBpbXBsZW1lbnQgb3RoZXIgZWNhbSBzdXBwb3J0ZWQgYnJpZGdlDQoN
CnN0YXRpYyBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21hdGNoIGdlbl9wY2lfZHRfbWF0Y2hbXSA9
IHsgICAgICAgICAgICAgICAgICAgICAgDQogICAgeyAuY29tcGF0aWJsZSA9ICJwY2ktaG9zdC1l
Y2FtLWdlbmVyaWMiLCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIA0KICAgICAg
LmRhdGEgPSAgICAgICAmcGNpX2dlbmVyaWNfZWNhbV9vcHMgfSwNCg0KICAgIHsgLmNvbXBhdGli
bGUgPSAicGNpLWhvc3QtY2FtLWdlbmVyaWMiLA0KICAgICAgLmRhdGEgPSAmZ2VuX3BjaV9jZmdf
Y2FtX2J1c19vcHMgfSwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICANCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgDQogICAgeyB9LCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIA0KfTsNCj4gDQo+IA0KPj4g
KyAgICBicmlkZ2UgPSBwY2lfYWxsb2NfaG9zdF9icmlkZ2UoKTsNCj4+ICsgICAgaWYgKCAhYnJp
ZGdlICkNCj4+ICsgICAgICAgIHJldHVybiAtRU5PTUVNOw0KPj4gKw0KPj4gKyAgICAvKiBQYXJz
ZSBhbmQgbWFwIG91ciBDb25maWd1cmF0aW9uIFNwYWNlIHdpbmRvd3MgKi8NCj4+ICsgICAgY2Zn
ID0gZ2VuX3BjaV9pbml0KGRldiwgb3BzKTsNCj4+ICsgICAgaWYgKCAhY2ZnICkNCj4+ICsgICAg
ew0KPj4gKyAgICAgICAgZXJyID0gLUVOT01FTTsNCj4+ICsgICAgICAgIGdvdG8gZXJyX2V4aXQ7
DQo+PiArICAgIH0NCj4+ICsNCj4+ICsgICAgYnJpZGdlLT5kdF9ub2RlID0gZGV2Ow0KPj4gKyAg
ICBicmlkZ2UtPmNmZyA9IGNmZzsNCj4+ICsgICAgYnJpZGdlLT5zeXNkYXRhID0gb3BzOw0KPj4g
KyAgICBicmlkZ2UtPm9wcyA9ICZvcHMtPnBjaV9vcHM7DQo+PiArICAgIGJyaWRnZS0+YnVzX3N0
YXJ0ID0gY2ZnLT5idXNuX3N0YXJ0Ow0KPj4gKyAgICBicmlkZ2UtPmJ1c19lbmQgPSBjZmctPmJ1
c25fZW5kOw0KPj4gKw0KPj4gKyAgICBicmlkZ2UtPnNlZ21lbnQgPSBwY2lfYnVzX2ZpbmRfZG9t
YWluX25yKGRldik7DQo+PiArICAgIGlmICggYnJpZGdlLT5zZWdtZW50IDwgMCApDQo+PiArICAg
IHsNCj4+ICsgICAgICAgIHByaW50ayhYRU5MT0dfRVJSICJJbmNvbnNpc3RlbnQgXCJsaW51eCxw
Y2ktZG9tYWluXCIgcHJvcGVydHkgaW4gRFRcbiIpOw0KPj4gKyAgICAgICAgQlVHKCk7DQo+PiAr
ICAgIH0NCj4+ICsgICAgcGNpX2FkZF9ob3N0X2JyaWRnZShicmlkZ2UpOw0KPj4gKw0KPj4gKyAg
ICByZXR1cm4gMDsNCj4+ICsNCj4+ICtlcnJfZXhpdDoNCj4+ICsgICAgeGZyZWUoYnJpZGdlKTsN
Cj4+ICsNCj4+ICsgICAgcmV0dXJuIGVycjsNCj4+ICt9DQo+PiArDQo+PiArLyoNCj4+ICsgKiBM
b2NhbCB2YXJpYWJsZXM6DQo+PiArICogbW9kZTogQw0KPj4gKyAqIGMtZmlsZS1zdHlsZTogIkJT
RCINCj4+ICsgKiBjLWJhc2ljLW9mZnNldDogNA0KPj4gKyAqIHRhYi13aWR0aDogNA0KPj4gKyAq
IGluZGVudC10YWJzLW1vZGU6IG5pbA0KPj4gKyAqIEVuZDoNCj4+ICsgKi8NCj4+IGRpZmYgLS1n
aXQgYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWdlbmVyaWMuYyBiL3hlbi9hcmNoL2FybS9w
Y2kvcGNpLWhvc3QtZ2VuZXJpYy5jDQo+PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPj4gaW5kZXgg
MDAwMDAwMDAwMC4uNmIzMjg4ZDZmMw0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIveGVuL2Fy
Y2gvYXJtL3BjaS9wY2ktaG9zdC1nZW5lcmljLmMNCj4+IEBAIC0wLDAgKzEsNDIgQEANCj4+ICsv
Kg0KPj4gKyAqIEJhc2VkIG9uIExpbnV4IGRyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpLWhvc3Qt
Y29tbW9uLmMNCj4+ICsgKiBCYXNlZCBvbiBMaW51eCBkcml2ZXJzL3BjaS9jb250cm9sbGVyL3Bj
aS1ob3N0LWdlbmVyaWMuYw0KPj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29m
dHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0IGFuZC9vciBtb2RpZnkNCj4+ICsgKiBpdCB1
bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24g
MiBhcw0KPj4gKyAqIHB1Ymxpc2hlZCBieSB0aGUgRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLg0K
Pj4gKyAqDQo+PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRo
YXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+PiArICogYnV0IFdJVEhPVVQgQU5ZIFdBUlJBTlRZOyB3
aXRob3V0IGV2ZW4gdGhlIGltcGxpZWQgd2FycmFudHkgb2YNCj4+ICsgKiBNRVJDSEFOVEFCSUxJ
VFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuICBTZWUgdGhlDQo+PiArICog
R05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4+ICsgKg0KPj4g
KyAqIFlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBHZW5lcmFsIFB1
YmxpYyBMaWNlbnNlDQo+PiArICogYWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uICBJZiBub3QsIHNl
ZSA8aHR0cDovL3d3dy5nbnUub3JnL2xpY2Vuc2VzLz4uDQo+PiArICovDQo+PiArDQo+PiArI2lu
Y2x1ZGUgPGFzbS9kZXZpY2UuaD4NCj4+ICsjaW5jbHVkZSA8eGVuL3BjaS5oPg0KPj4gKyNpbmNs
dWRlIDxhc20vcGNpLmg+DQo+PiArDQo+PiArc3RhdGljIGNvbnN0IHN0cnVjdCBkdF9kZXZpY2Vf
bWF0Y2ggZ2VuX3BjaV9kdF9tYXRjaFtdID0gew0KPj4gKyAgICB7IC5jb21wYXRpYmxlID0gInBj
aS1ob3N0LWVjYW0tZ2VuZXJpYyIsDQo+PiArICAgICAgLmRhdGEgPSAgICAgICAmcGNpX2dlbmVy
aWNfZWNhbV9vcHMgfSwNCj4+ICsNCj4+ICsgICAgeyB9LA0KPj4gK307DQo+PiArDQo+PiArRFRf
REVWSUNFX1NUQVJUKHBjaV9nZW4sICJQQ0kgSE9TVCBHRU5FUklDIiwgREVWSUNFX1BDSSkNCj4+
ICsuZHRfbWF0Y2ggPSBnZW5fcGNpX2R0X21hdGNoLA0KPj4gKy5pbml0ID0gcGNpX2hvc3RfY29t
bW9uX3Byb2JlLA0KPj4gK0RUX0RFVklDRV9FTkQNCj4+ICsNCj4+ICsvKg0KPj4gKyAqIExvY2Fs
IHZhcmlhYmxlczoNCj4+ICsgKiBtb2RlOiBDDQo+PiArICogYy1maWxlLXN0eWxlOiAiQlNEIg0K
Pj4gKyAqIGMtYmFzaWMtb2Zmc2V0OiA0DQo+PiArICogdGFiLXdpZHRoOiA0DQo+PiArICogaW5k
ZW50LXRhYnMtbW9kZTogbmlsDQo+PiArICogRW5kOg0KPj4gKyAqLw0KPj4gZGlmZiAtLWdpdCBh
L3hlbi9pbmNsdWRlL2FzbS1hcm0vZGV2aWNlLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL2Rldmlj
ZS5oDQo+PiBpbmRleCA1ZWNkNWU3YmQxLi41ODIxMTljMzFlIDEwMDY0NA0KPj4gLS0tIGEveGVu
L2luY2x1ZGUvYXNtLWFybS9kZXZpY2UuaA0KPj4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9k
ZXZpY2UuaA0KPj4gQEAgLTE2LDYgKzE2LDggQEAgc3RydWN0IGRldmljZQ0KPj4gICAgIGVudW0g
ZGV2aWNlX3R5cGUgdHlwZTsNCj4+ICNpZmRlZiBDT05GSUdfSEFTX0RFVklDRV9UUkVFDQo+PiAg
ICAgc3RydWN0IGR0X2RldmljZV9ub2RlICpvZl9ub2RlOyAvKiBVc2VkIGJ5IGRyaXZlcnMgaW1w
b3J0ZWQgZnJvbSBMaW51eCAqLw0KPj4gKyAgICAvKiBVc2VkIGJ5IGRyaXZlcnMgaW1wb3J0ZWQg
ZnJvbSBMaW51eCAqLw0KPj4gKyAgICBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21hdGNoICpvZl9t
YXRjaF90YWJsZTsNCj4+ICNlbmRpZg0KPj4gICAgIHN0cnVjdCBkZXZfYXJjaGRhdGEgYXJjaGRh
dGE7DQo+PiAgICAgc3RydWN0IGlvbW11X2Z3c3BlYyAqaW9tbXVfZndzcGVjOyAvKiBwZXItZGV2
aWNlIElPTU1VIGluc3RhbmNlIGRhdGEgKi8NCj4+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9h
c20tYXJtL3BjaS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KPj4gaW5kZXggZjJmODZi
ZTliYy4uNGIzMmM3MDg4YSAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNp
LmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+IEBAIC0yNiw2ICsyNiw2
NSBAQCBzdHJ1Y3QgYXJjaF9wY2lfZGV2IHsNCj4+ICAgICBzdHJ1Y3QgZGV2aWNlIGRldjsNCj4+
IH07DQo+PiANCj4+ICsvKg0KPj4gKyAqIHN0cnVjdCB0byBob2xkIHRoZSBtYXBwaW5ncyBvZiBh
IGNvbmZpZyBzcGFjZSB3aW5kb3cuIFRoaXMNCj4+ICsgKiBpcyBleHBlY3RlZCB0byBiZSB1c2Vk
IGFzIHN5c2RhdGEgZm9yIFBDSSBjb250cm9sbGVycyB0aGF0DQo+PiArICogdXNlIEVDQU0uDQo+
PiArICovDQo+PiArc3RydWN0IHBjaV9jb25maWdfd2luZG93IHsNCj4+ICsgICAgcGFkZHJfdCAg
ICAgICAgIHBoeXNfYWRkcjsNCj4+ICsgICAgcGFkZHJfdCAgICAgICAgIHNpemU7DQo+PiArICAg
IHVpbnQ4X3QgICAgICAgICBidXNuX3N0YXJ0Ow0KPj4gKyAgICB1aW50OF90ICAgICAgICAgYnVz
bl9lbmQ7DQo+PiArICAgIHZvaWQgX19pb21lbSAgICAqd2luOw0KPj4gK307DQo+PiArDQo+PiAr
LyoNCj4+ICsgKiBzdHJ1Y3QgdG8gaG9sZCBwY2kgaG9zdCBicmlkZ2UgaW5mb3JtYXRpb24NCj4+
ICsgKiBmb3IgYSBQQ0kgY29udHJvbGxlci4NCj4+ICsgKi8NCj4+ICtzdHJ1Y3QgcGNpX2hvc3Rf
YnJpZGdlIHsNCj4+ICsgICAgc3RydWN0IGR0X2RldmljZV9ub2RlICpkdF9ub2RlOyAgLyogUG9p
bnRlciB0byB0aGUgYXNzb2NpYXRlZCBEVCBub2RlICovDQo+PiArICAgIHN0cnVjdCBsaXN0X2hl
YWQgbm9kZTsgICAgICAgICAgIC8qIE5vZGUgaW4gbGlzdCBvZiBob3N0IGJyaWRnZXMgKi8NCj4+
ICsgICAgdWludDE2X3Qgc2VnbWVudDsgICAgICAgICAgICAgICAgLyogU2VnbWVudCBudW1iZXIg
Ki8NCj4+ICsgICAgdWludDhfdCBidXNfc3RhcnQ7ICAgICAgICAgICAgICAgLyogQnVzIHN0YXJ0
IG9mIHRoaXMgYnJpZGdlLiAqLw0KPj4gKyAgICB1aW50OF90IGJ1c19lbmQ7ICAgICAgICAgICAg
ICAgICAvKiBCdXMgZW5kIG9mIHRoaXMgYnJpZGdlLiAqLw0KPiANCj4gYnVzX3N0YXJ0IGFuZCBi
dXNfZW5kIGFyZSBib3RoIGhlcmUgYW5kIGFsc28gdW5kZXIgcGNpX2NvbmZpZ193aW5kb3cuDQo+
IFNob3VsZCB3ZSByZW1vdmUgdGhlbSBmcm9tIGhlcmUgKGlmIG5vdCwgdGhlbiBjYW4gd2UgcmVt
b3ZlZCB0aGVtIGZyb20NCj4gc3RydWN0IHBjaV9jb25maWdfd2luZG93KT8NCg0KWWVzIEkgd2ls
bCByZW1vdmUgdGhpcy4NCj4gDQo+IA0KPj4gKyAgICBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cq
IGNmZzsgICAvKiBQb2ludGVyIHRvIHRoZSBicmlkZ2UgY29uZmlnIHdpbmRvdyAqLw0KPj4gKyAg
ICB2b2lkICpzeXNkYXRhOyAgICAgICAgICAgICAgICAgICAvKiBQb2ludGVyIHRvIHRoZSBjb25m
aWcgc3BhY2Ugd2luZG93Ki8NCj4+ICsgICAgY29uc3Qgc3RydWN0IHBjaV9vcHMgKm9wczsNCj4g
DQo+IEl0IGxvb2tzIGxpa2Ugc3lzZGF0YSBpcyB1bm5lY2Vzc2FyeSBiZWNhdXNlIHdlIGNhbiBn
ZXQgdGhlIHJpZ2h0DQo+IHBvaW50ZXIgZnJvbSBvcHMsIGdpdmVuIHRoYXQgb3BzIGlzIHBvaW50
aW5nIHRvIGEgbWVtYmVyIG9mIHRoZSBzdHJ1Y3QNCj4gcG9pbnQgYnkgc3lzZGF0YS4gSW4gb3Ro
ZXIgd29yZHMsIGlmIHlvdSB1c2UgY29udGFpbmVyX29mKG9wcywgc3RydWN0DQo+IHBjaV9lY2Ft
X29wcywgcGNpX29wcykgaXQgc2hvdWxkIHdvcms/DQo+IA0KWWVzIG1ha2Ugc2Vuc2UgSSB3aWxs
IHJlbW92ZSB0aGlzLg0KDQo+IA0KPj4gK307DQo+PiArDQo+PiArc3RydWN0IHBjaV9vcHMgew0K
Pj4gKyAgICB2b2lkIF9faW9tZW0gKigqbWFwX2J1cykoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAq
YnJpZGdlLCB1aW50MzJfdCBzYmRmLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dWludDMyX3Qgb2Zmc2V0KTsNCj4+ICsgICAgaW50ICgqcmVhZCkoc3RydWN0IHBjaV9ob3N0X2Jy
aWRnZSAqYnJpZGdlLCB1aW50MzJfdCBzYmRmLA0KPj4gKyAgICAgICAgICAgICAgICB1aW50MzJf
dCByZWcsIHVpbnQzMl90IGxlbiwgdWludDMyX3QgKnZhbHVlKTsNCj4+ICsgICAgaW50ICgqd3Jp
dGUpKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwgdWludDMyX3Qgc2JkZiwNCj4+ICsg
ICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3QgbGVuLCB1aW50MzJfdCB2YWx1
ZSk7DQo+PiArfTsNCj4+ICsNCj4+ICsvKg0KPj4gKyAqIHN0cnVjdCB0byBob2xkIHBjaSBvcHMg
YW5kIGJ1cyBzaGlmdCBvZiB0aGUgY29uZmlnIHdpbmRvdw0KPj4gKyAqIGZvciBhIFBDSSBjb250
cm9sbGVyLg0KPj4gKyAqLw0KPj4gK3N0cnVjdCBwY2lfZWNhbV9vcHMgew0KPj4gKyAgICB1bnNp
Z25lZCBpbnQgICAgICAgICAgICBidXNfc2hpZnQ7DQo+PiArICAgIHN0cnVjdCBwY2lfb3BzICAg
ICAgICAgIHBjaV9vcHM7DQo+PiArICAgIGludCAoKmNmZ19yZWdfaW5kZXgpKHN0cnVjdCBkdF9k
ZXZpY2Vfbm9kZSAqZGV2KTsNCj4+ICsgICAgaW50ICgqaW5pdCkoc3RydWN0IHBjaV9jb25maWdf
d2luZG93ICopOw0KPiANCj4gaW5pdCBpcyB1bnVzZWQsIGNhbiB3ZSBnZXQgcmlkIG9mIGl0Pw0K
PiANCj4gDQo+PiArfTsNCj4+ICsNCj4+ICsvKiBEZWZhdWx0IEVDQU0gb3BzICovDQo+PiArZXh0
ZXJuIGNvbnN0IHN0cnVjdCBwY2lfZWNhbV9vcHMgcGNpX2dlbmVyaWNfZWNhbV9vcHM7DQo+IA0K
PiBJZiB3ZSB1c2UgY29udGFpbmVyX29mIGFuZCBnZXQgcmlkIG9mIHN5c2RhdGEsIEkgd29uZGVy
IGlmIHdlIGdldCBhdm9pZA0KPiBleHBvcnRpbmcgcGNpX2dlbmVyaWNfZWNhbV9vcHMuDQo+IA0K
PiANCj4+ICtpbnQgcGNpX2hvc3RfY29tbW9uX3Byb2JlKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAq
ZGV2LCBjb25zdCB2b2lkICpkYXRhKTsNCj4gDQo+IFRoaXMgc2hvdWxkIGJlIHN0YXRpYyBhbmQg
bm90IGV4cG9ydGVkDQo+IA0KV2UgcmVxdWlyZWQgdGhpcyBuZWVkIHRvIGJlIGV4cG9ydGVkIGFz
IHN1Z2dlc3RlZCBieSBPbGVrc2FuZHIuDQo+IA0KPj4gK2ludCBwY2lfZ2VuZXJpY19jb25maWdf
cmVhZChzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYsDQo+PiAr
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3QgbGVuLCB1
aW50MzJfdCAqdmFsdWUpOw0KPiANCj4gYWxzbyB0aGlzDQo+IA0KPiANCj4+ICtpbnQgcGNpX2dl
bmVyaWNfY29uZmlnX3dyaXRlKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwgdWludDMy
X3Qgc2JkZiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgcmVnLCB1
aW50MzJfdCBsZW4sIHVpbnQzMl90IHZhbHVlKTsNCj4gDQo+IGFsc28gdGhpcw0KPiANCj4gDQo+
PiArdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2Ug
KmJyaWRnZSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3Qgc2Jk
ZiwgdWludDMyX3Qgd2hlcmUpOw0KPiANCj4gYWxzbyB0aGlzDQo+IA0KPj4gc3RhdGljIGFsd2F5
c19pbmxpbmUgYm9vbCBpc19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCh2b2lkKQ0KPj4gew0KPj4g
ICAgIHJldHVybiBwY2lfcGFzc3Rocm91Z2hfZW5hYmxlZDsNCj4+IC0tIA0KPj4gMi4xNy4xDQoN
ClJlZ2FyZHMsDQpSYWh1bA0KDQo=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 17:14:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 17:14:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194524.346538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSIp-0003LL-Ef; Thu, 23 Sep 2021 17:14:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194524.346538; Thu, 23 Sep 2021 17:14:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSIp-0003LE-BG; Thu, 23 Sep 2021 17:14:31 +0000
Received: by outflank-mailman (input) for mailman id 194524;
 Thu, 23 Sep 2021 17:14:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTSIn-0003L4-Ol; Thu, 23 Sep 2021 17:14:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTSIn-00057R-JU; Thu, 23 Sep 2021 17:14:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTSIn-0007GX-6l; Thu, 23 Sep 2021 17:14:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTSIn-0000qM-6G; Thu, 23 Sep 2021 17:14:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9rwkIae5egJPcBMvrMn/u3CIGl0KXejc7SFy7oop6sA=; b=J4NKn9Jk3Eg+I/IgvV4OnB6ncd
	HG7W5coexuX3oIWbwgDjsswUlj+Z+kKrTVZ2oq6EZVQZHkajpWW8RakcotNg6QLE+a+8qnUUcwkzo
	XxTIxnS4i4hkBmME6QlIPeQxqDMUZMZ6xjaQksyF+X6z6YB0Cz5irUDMaD2dH9c2h/wU=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165163-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165163: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=58e2cf5d794616b84f591d4d1276c8953278ce24
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 17:14:29 +0000

flight 165163 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165163/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                58e2cf5d794616b84f591d4d1276c8953278ce24
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  418 days
Failing since        152366  2020-08-01 20:49:34 Z  417 days  730 attempts
Testing same since   165163  2021-09-23 07:54:43 Z    0 days    1 attempts

------------------------------------------------------------
7324 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263977 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 17:44:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 17:44:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194530.346551 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSlv-0007PS-SS; Thu, 23 Sep 2021 17:44:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194530.346551; Thu, 23 Sep 2021 17:44:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTSlv-0007PL-PY; Thu, 23 Sep 2021 17:44:35 +0000
Received: by outflank-mailman (input) for mailman id 194530;
 Thu, 23 Sep 2021 17:44:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTSlu-0007PF-Qu
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 17:44:34 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 593cd0ad-cd1a-4b8a-a5cc-d7116ffeadee;
 Thu, 23 Sep 2021 17:44:32 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id i25so29499090lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 10:44:32 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id 9sm513691lfp.129.2021.09.23.10.44.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 10:44:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 593cd0ad-cd1a-4b8a-a5cc-d7116ffeadee
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=/TedQXzo+rAZLoCGqN7JBHlekw4ScOZMoMKHvF3rAPo=;
        b=LMyrSpoNYv8F2MAY4H9az4Pixvp4jDdBVajpfAMd6MAVYUyTEtHV/RIhVy0YgGr9SZ
         qZMdaDCXkBIS77i+JKl+NZKgRRZVCHJSTH3LbpfXD6if7eaYsSBrRL1BHRVIYy3sTSks
         2i9V2khnMTiN3+HupGjNFDNA+YxrdSs7D9gs9y91fv/+SViE4YSL0q43r87Zh8GX1fFE
         3GRhXcWfqPX67yO2n/ELHAW1CKyDuxTSVKjtKPfYPTDfY4dRdsTVl40Bov1B9nqjS+7e
         QGTFSUMrUgKQbAbWpRctJxwWwqFZ+axAA+Wetel5eR1djvpuXrhlU1mJD1Fy8ig9GrLG
         LKgg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=/TedQXzo+rAZLoCGqN7JBHlekw4ScOZMoMKHvF3rAPo=;
        b=5ao0u5le8iPoiE39hOuVZcZBbsIC0Y3UcUGUh8rvHzOCTzVkrV0u1cAQe+ZzsI3hs6
         qhfQ3rDy+ojWImwrTDNHder8jInQd7qGyS3dA7+IxCR5bcVfzJ6oxgTfie5bJ0EMWDkD
         d3Qh/YfxNl1dd68iQpM5B3BWjJeQytV5MBrIOaUNzz9Hsow3v4lPzzuuWeauLtOrAJW4
         kSMrW13uMhRWcj21bZDAZaw8T3ZtWJ6PYo4ZJTkvGrzu7JxMvMJD0wzKlWwxU+Bqdu9W
         JjipI81Ca/Xh2MiRyYDX5FZSxWAv2f7okqHXgriZFNAD5D1u3Nq5y4TagNvzX533DKW0
         28rQ==
X-Gm-Message-State: AOAM531i939aH2CEYZH7Cc/XVe0wURyUVNin38cKJshmx9d9N9JXpg2h
	9MiGS4VZIlsdO7mV6ssE2xc=
X-Google-Smtp-Source: ABdhPJwhbl7y9LTxQHm0Z3DzsGmUsl2SaCPyZH/gO1rACGJhbgwtZuPHxQjqhNnhoSzJ0oz+GroJrg==
X-Received: by 2002:ac2:4f02:: with SMTP id k2mr5517622lfr.266.1632419071561;
        Thu, 23 Sep 2021 10:44:31 -0700 (PDT)
Subject: Re: [PATCH V2 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Henry Wang <Henry.Wang@arm.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>
References: <1631297924-8658-1-git-send-email-olekstysh@gmail.com>
 <1631297924-8658-3-git-send-email-olekstysh@gmail.com>
 <0a72559e-5742-dc33-1c8f-5903c50b27be@xen.org>
 <08294c53-109a-8544-3a23-85e034d2992d@gmail.com>
 <cc5ee8cc-84ac-a27f-af99-ac0ba3ab8d68@gmail.com>
 <8665a10b-60b1-1551-0aab-e6725f4c6ff1@gmail.com>
 <alpine.DEB.2.21.2109230931110.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com>
Date: Thu, 23 Sep 2021 20:44:30 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109230931110.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 23.09.21 19:38, Stefano Stabellini wrote:

Hi Stefano

> On Thu, 23 Sep 2021, Oleksandr wrote:
>> On 18.09.21 19:59, Oleksandr wrote:
>>>>>> +#define EXT_REGION_END 0x80003fffffffULL
>>>>>> +
>>>>>> +static int __init find_unallocated_memory(const struct kernel_info
>>>>>> *kinfo,
>>>>>> +                                          struct meminfo
>>>>>> *ext_regions)
>>>>>> +{
>>>>>> +    const struct meminfo *assign_mem = &kinfo->mem;
>>>>>> +    struct rangeset *unalloc_mem;
>>>>>> +    paddr_t start, end;
>>>>>> +    unsigned int i;
>>>>>> +    int res;
>>>>> We technically already know which range of memory is unused. This is
>>>>> pretty much any region in the freelist of the page allocator. So how
>>>>> about walking the freelist instead?
>>>> ok, I will investigate the page allocator code (right now I have no
>>>> understanding of how to do that). BTW, I have just grepped "freelist"
>>>> through the code and all page context related appearances are in x86 code
>>>> only.
>>>>
>>>>> The advantage is we don't need to worry about modifying the function
>>>>> when adding new memory type.
>>>>>
>>>>> One disavantage is this will not cover *all* the unused memory as this
>>>>> is doing. But I think this is an acceptable downside.
>>> I did some investigations and create test patch. Although, I am not 100%
>>> sure this is exactly what you meant, but I will provide results anyway.
>>>
>>> 1. Below the extended regions (unallocated memory, regions >=64MB )
>>> calculated by my initial method (bootinfo.mem - kinfo->mem -
>>> bootinfo.reserved_mem - kinfo->gnttab):
>>>
>>> (XEN) Extended region 0: 0x48000000->0x54000000
>>> (XEN) Extended region 1: 0x57000000->0x60000000
>>> (XEN) Extended region 2: 0x70000000->0x78000000
>>> (XEN) Extended region 3: 0x78200000->0xc0000000
>>> (XEN) Extended region 4: 0x500000000->0x580000000
>>> (XEN) Extended region 5: 0x600000000->0x680000000
>>> (XEN) Extended region 6: 0x700000000->0x780000000
>>>
>>> 2. Below the extended regions (unallocated memory, regions >=64MB)
>>> calculated by new method (free memory in page allocator):
>>>
>>> (XEN) Extended region 0: 0x48000000->0x54000000
>>> (XEN) Extended region 1: 0x58000000->0x60000000
>>> (XEN) Extended region 2: 0x70000000->0x78000000
>>> (XEN) Extended region 3: 0x78200000->0x84000000
>>> (XEN) Extended region 4: 0x86000000->0x8a000000
>>> (XEN) Extended region 5: 0x8c200000->0xc0000000
>>> (XEN) Extended region 6: 0x500000000->0x580000000
>>> (XEN) Extended region 7: 0x600000000->0x680000000
>>> (XEN) Extended region 8: 0x700000000->0x765e00000
>>>
>>> Some thoughts regarding that.
>>>
>>> 1. A few ranges below 4GB are absent in resulting extended regions. I
>>> assume, this is because of the modules:
>>>
>>> (XEN) Checking for initrd in /chosen
>>> (XEN) Initrd 0000000084000040-0000000085effc48
>>> (XEN) RAM: 0000000048000000 - 00000000bfffffff
>>> (XEN) RAM: 0000000500000000 - 000000057fffffff
>>> (XEN) RAM: 0000000600000000 - 000000067fffffff
>>> (XEN) RAM: 0000000700000000 - 000000077fffffff
>>> (XEN)
>>> (XEN) MODULE[0]: 0000000078080000 - 00000000781d74c8 Xen
>>> (XEN) MODULE[1]: 0000000057fe7000 - 0000000057ffd080 Device Tree
>>> (XEN) MODULE[2]: 0000000084000040 - 0000000085effc48 Ramdisk
>>> (XEN) MODULE[3]: 000000008a000000 - 000000008c000000 Kernel
>>> (XEN) MODULE[4]: 000000008c000000 - 000000008c010000 XSM
>>> (XEN)  RESVD[0]: 0000000084000040 - 0000000085effc48
>>> (XEN)  RESVD[1]: 0000000054000000 - 0000000056ffffff
>>>
>>> 2. Also, it worth mentioning that relatively large chunk (~417MB) of memory
>>> above 4GB is absent (to be precise, at the end of last RAM bank), which I
>>> assume, used for Xen internals.
>>> We could really use it for extended regions.
>>> Below free regions in the heap (for last RAM bank) just in case:
>>>
>>> (XEN) heap[node=0][zone=23][order=5] 0x00000765ec0000-0x00000765ee0000
>>> (XEN) heap[node=0][zone=23][order=6] 0x00000765e80000-0x00000765ec0000
>>> (XEN) heap[node=0][zone=23][order=7] 0x00000765e00000-0x00000765e80000
>>> (XEN) heap[node=0][zone=23][order=9] 0x00000765c00000-0x00000765e00000
>>> (XEN) heap[node=0][zone=23][order=10] 0x00000765800000-0x00000765c00000
>>> (XEN) heap[node=0][zone=23][order=11] 0x00000765000000-0x00000765800000
>>> (XEN) heap[node=0][zone=23][order=12] 0x00000764000000-0x00000765000000
>>> (XEN) heap[node=0][zone=23][order=14] 0x00000760000000-0x00000764000000
>>> (XEN) heap[node=0][zone=23][order=17] 0x00000740000000-0x00000760000000
>>> (XEN) heap[node=0][zone=23][order=18] 0x00000540000000-0x00000580000000
>>> (XEN) heap[node=0][zone=23][order=18] 0x00000500000000-0x00000540000000
>>> (XEN) heap[node=0][zone=23][order=18] 0x00000640000000-0x00000680000000
>>> (XEN) heap[node=0][zone=23][order=18] 0x00000600000000-0x00000640000000
>>> (XEN) heap[node=0][zone=23][order=18] 0x00000700000000-0x00000740000000
>>>
>>> Yes, you already pointed out this disadvantage, so if it is an acceptable
>>> downside, I am absolutely OK.
>>>
>>>
>>> 3. Common code updates. There is a question how to properly make a
>>> connection between common allocator internals and Arm's code for creating
>>> DT. I didn’t come up with anything better
>>> than creating for_each_avail_page() for invoking a callback with page and
>>> its order.
>>>
>>> **********
>>>
>>> Below the proposed changes on top of the initial patch, would this be
>>> acceptable in general?
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 523eb19..1e58fc5 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -753,16 +753,33 @@ static int __init add_ext_regions(unsigned long s,
>>> unsigned long e, void *data)
>>>       return 0;
>>>   }
>>>
>>> +static int __init add_unalloc_mem(struct page_info *page, unsigned int
>>> order,
>>> +                                  void *data)
>>> +{
>>> +    struct rangeset *unalloc_mem = data;
>>> +    paddr_t start, end;
>>> +    int res;
>>> +
>>> +    start = page_to_maddr(page);
>>> +    end = start + pfn_to_paddr(1UL << order);
>>> +    res = rangeset_add_range(unalloc_mem, start, end - 1);
>>> +    if ( res )
>>> +    {
>>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>> +               start, end);
>>> +        return res;
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>   #define EXT_REGION_START   0x40000000ULL
>>>   #define EXT_REGION_END     0x80003fffffffULL
>>>
>>> -static int __init find_unallocated_memory(const struct kernel_info *kinfo,
>>> -                                          struct meminfo *ext_regions)
>>> +static int __init find_unallocated_memory(struct meminfo *ext_regions)
>>>   {
>>> -    const struct meminfo *assign_mem = &kinfo->mem;
>>>       struct rangeset *unalloc_mem;
>>>       paddr_t start, end;
>>> -    unsigned int i;
>>>       int res;
>>>
>>>       dt_dprintk("Find unallocated memory for extended regions\n");
>>> @@ -771,59 +788,9 @@ static int __init find_unallocated_memory(const struct
>>> kernel_info *kinfo,
>>>       if ( !unalloc_mem )
>>>           return -ENOMEM;
>>>
>>> -    /* Start with all available RAM */
>>> -    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>>> -    {
>>> -        start = bootinfo.mem.bank[i].start;
>>> -        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
>>> -        res = rangeset_add_range(unalloc_mem, start, end - 1);
>>> -        if ( res )
>>> -        {
>>> -            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>>> -                   start, end);
>>> -            goto out;
>>> -        }
>>> -    }
>>> -
>>> -    /* Remove RAM assigned to Dom0 */
>>> -    for ( i = 0; i < assign_mem->nr_banks; i++ )
>>> -    {
>>> -        start = assign_mem->bank[i].start;
>>> -        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
>>> -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
>>> -        if ( res )
>>> -        {
>>> -            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>> -                   start, end);
>>> -            goto out;
>>> -        }
>>> -    }
>>> -
>>> -    /* Remove reserved-memory regions */
>>> -    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>>> -    {
>>> -        start = bootinfo.reserved_mem.bank[i].start;
>>> -        end = bootinfo.reserved_mem.bank[i].start +
>>> -            bootinfo.reserved_mem.bank[i].size;
>>> -        res = rangeset_remove_range(unalloc_mem, start, end - 1);
>>> -        if ( res )
>>> -        {
>>> -            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>> -                   start, end);
>>> -            goto out;
>>> -        }
>>> -    }
>>> -
>>> -    /* Remove grant table region */
>>> -    start = kinfo->gnttab_start;
>>> -    end = kinfo->gnttab_start + kinfo->gnttab_size;
>>> -    res = rangeset_remove_range(unalloc_mem, start, end - 1);
>>> +    res = for_each_avail_page(add_unalloc_mem, unalloc_mem);
>>>       if ( res )
>>> -    {
>>> -        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>>> -               start, end);
>>>           goto out;
>>> -    }
>>>
>>>       start = EXT_REGION_START;
>>>       end = min((1ULL << p2m_ipa_bits) - 1, EXT_REGION_END);
>>> @@ -840,8 +807,7 @@ out:
>>>       return res;
>>>   }
>>>
>>> -static int __init find_memory_holes(const struct kernel_info *kinfo,
>>> -                                    struct meminfo *ext_regions)
>>> +static int __init find_memory_holes(struct meminfo *ext_regions)
>>>   {
>>>       struct dt_device_node *np;
>>>       struct rangeset *mem_holes;
>>> @@ -961,9 +927,9 @@ static int __init make_hypervisor_node(struct domain *d,
>>>       else
>>>       {
>>>           if ( !is_iommu_enabled(d) )
>>> -            res = find_unallocated_memory(kinfo, ext_regions);
>>> +            res = find_unallocated_memory(ext_regions);
>>>           else
>>> -            res = find_memory_holes(kinfo, ext_regions);
>>> +            res = find_memory_holes(ext_regions);
>>>
>>>           if ( res )
>>>               printk(XENLOG_WARNING "Failed to allocate extended regions\n");
>>> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
>>> index 8fad139..7cd1020 100644
>>> --- a/xen/common/page_alloc.c
>>> +++ b/xen/common/page_alloc.c
>>> @@ -1572,6 +1572,40 @@ static int reserve_heap_page(struct page_info *pg)
>>>
>>>   }
>>>
>>> +/* TODO heap_lock? */
>>> +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, void
>>> *),
>>> +                        void *data)
>>> +{
>>> +    unsigned int node, zone, order;
>>> +    int ret;
>>> +
>>> +    for ( node = 0; node < MAX_NUMNODES; node++ )
>>> +    {
>>> +        if ( !avail[node] )
>>> +            continue;
>>> +
>>> +        for ( zone = 0; zone < NR_ZONES; zone++ )
>>> +        {
>>> +            for ( order = 0; order <= MAX_ORDER; order++ )
>>> +            {
>>> +                struct page_info *head, *tmp;
>>> +
>>> +                if ( page_list_empty(&heap(node, zone, order)) )
>>> +                    continue;
>>> +
>>> +                page_list_for_each_safe ( head, tmp, &heap(node, zone,
>>> order) )
>>> +                {
>>> +                    ret = cb(head, order, data);
>>> +                    if ( ret )
>>> +                        return ret;
>>> +                }
>>> +            }
>>> +        }
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>   int offline_page(mfn_t mfn, int broken, uint32_t *status)
>>>   {
>>>       unsigned long old_info = 0;
>>> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
>>> index 667f9da..64dd3e2 100644
>>> --- a/xen/include/xen/mm.h
>>> +++ b/xen/include/xen/mm.h
>>> @@ -123,6 +123,9 @@ unsigned int online_page(mfn_t mfn, uint32_t *status);
>>>   int offline_page(mfn_t mfn, int broken, uint32_t *status);
>>>   int query_page_offline(mfn_t mfn, uint32_t *status);
>>>
>>> +int for_each_avail_page(int (*cb)(struct page_info *, unsigned int, void
>>> *),
>>> +                        void *data);
>>> +
>>>   void heap_init_late(void);
>>>
>>>   int assign_pages(
>>
>> I am sorry, but may I please clarify regarding that? Whether we will go this
>> new direction (free memory in page allocator) or leave things as they are
>> (bootinfo.mem - kinfo->mem - bootinfo.reserved_mem - kinfo->gnttab). This is
>> only one still unclear moment to me in current patch before preparing V3.
> I think both approaches are fine. Your original approach leads to better
> results in terms of extended regions but the difference is not drastic.
> The original approach requires more code (bad) but probably less CPU
> cycles (good).
>
> Personally I am fine either way but as Julien was the one to provide
> feedback on this it would be best to get his opinion.
>
> But in the meantime I think it is OK to send a v3 so that we can review
> the rest.


OK, thank you for the clarification.

I am also fine either way, I just wanted to know which one to pick. 
Anyway, I think I will be able to make updates later on.


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 19:12:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 19:12:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194538.346562 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTU8i-0004mS-6L; Thu, 23 Sep 2021 19:12:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194538.346562; Thu, 23 Sep 2021 19:12:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTU8i-0004mL-38; Thu, 23 Sep 2021 19:12:12 +0000
Received: by outflank-mailman (input) for mailman id 194538;
 Thu, 23 Sep 2021 19:12:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTU8h-0004m4-KQ
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 19:12:11 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 74bb6cc1-9920-4ccb-be29-23b5b192f525;
 Thu, 23 Sep 2021 19:12:10 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B2C916109E;
 Thu, 23 Sep 2021 19:12:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74bb6cc1-9920-4ccb-be29-23b5b192f525
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632424329;
	bh=ItYHWUdHtysSe2603vDDSqLA2H/jNVUQL0qmiwt3PCE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=aJz/1vSMd270C2jiAWo1h8dQ1OpDCTtsnxnIdx/lJx5ikX0Wj1evayiiiXJFxREo9
	 K9NIOR7PiLQFPEKrN3C460/IjdgUil9e2oUD1qOko8+ICBWMmyAu/u3NnApNmB63MX
	 nU8SaaXh0AJvqwbCllpznFrEJmovlfr48L/urs7fVe7A0G5tgPJ/AXCgA3ss6fK3Dv
	 zHlow+JukHQfZZUhzCZZM9cDAdSJbYkRsT3scafI7f62fl7Xdjtox5GQrU2yK/8l3n
	 GbSLuecm/5tDpk2fXSGfpViaOIwDxpfV0VtVAB3AMe6t7CfuJs4J4a2mntS5i9orP6
	 D+315Ns7IMJ7A==
Date: Thu, 23 Sep 2021 12:12:08 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Andre Przywara <Andre.Przywara@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
Message-ID: <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s> <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Rahul Singh wrote:
> >> +            goto err_exit;
> >> +    }
> > 
> > This is unnecessary at the moment, right? Can we get rid of ops->init ?
> 
> No this is required for N1SDP board. Please check below patch.
> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d57547087cff4ffc078dc515d5

OK


> >> +int pci_host_common_probe(struct dt_device_node *dev, const void *data)
> >> +{
> >> +    struct pci_host_bridge *bridge;
> >> +    struct pci_config_window *cfg;
> >> +    struct pci_ecam_ops *ops;
> >> +    const struct dt_device_match *of_id;
> >> +    int err;
> >> +
> >> +    if ( dt_device_for_passthrough(dev) )
> >> +        return 0;
> >> +
> >> +    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
> >> +    ops = (struct pci_ecam_ops *) of_id->data;
> > 
> > Do we really need dt_match_node and dev->dev.of_match_table to get
> > dt_device_match.data?
> > 
> 
> > data is passed as a parameter to pci_host_common_probe, isn't it enough
> > to do:
> > 
> > ops = (struct pci_ecam_ops *) data;
> 
> As of now not required but in future we might need it if we implement other ecam supported bridge
> 
> static const struct dt_device_match gen_pci_dt_match[] = {                      
>     { .compatible = "pci-host-ecam-generic",                                    
>       .data =       &pci_generic_ecam_ops },
> 
>     { .compatible = "pci-host-cam-generic",
>       .data = &gen_pci_cfg_cam_bus_ops },                                 
>                                                                                 
>     { },                                                                        
> };

Even if we add another ECAM-supported bridge, the following:

ops = (struct pci_ecam_ops *) data;

could still work, right? The probe function will directly receive as
parameter the .data pointer. You shouldn't need the indirection via
dt_match_node?

If you are worried about gen_pci_cfg_cam_bus_ops not being a struct
pci_ecam_ops: that problem can also be solved by making
gen_pci_cfg_cam_bus_ops a struct containinig a struct pci_ecam_ops.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 19:32:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 19:32:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194542.346574 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUSc-0007vI-Ss; Thu, 23 Sep 2021 19:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194542.346574; Thu, 23 Sep 2021 19:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUSc-0007vB-Q0; Thu, 23 Sep 2021 19:32:46 +0000
Received: by outflank-mailman (input) for mailman id 194542;
 Thu, 23 Sep 2021 19:32:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTUSb-0007v5-8C
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 19:32:45 +0000
Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6b253121-5e78-4b5f-8504-b6aadc5b442d;
 Thu, 23 Sep 2021 19:32:41 +0000 (UTC)
Received: by mail-lf1-x136.google.com with SMTP id t10so30381613lfd.8
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 12:32:41 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id j20sm533476lfu.165.2021.09.23.12.32.39
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 23 Sep 2021 12:32:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b253121-5e78-4b5f-8504-b6aadc5b442d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=YAbpIis/w2297iCt96qCCK9VooOBw3jgsoMVgYXZUlg=;
        b=kXteXJAQkAWnrwuh0v5E0/z78PvNyKqnUKJXtX0ThusA2CeKGcRRchibwmE0BFP8BT
         8E8ApmKXPTgp5oJW21hcgh4ccqMSqWxdw2WaBxQjCgCPoHybRLeCiGrpfaO92M7xDNfF
         U8kdizRTqd499BdTHPsiNUh2ZW+aRRVjs8l1KcszlP7CG7exbapR5gahGqstt59/yUaZ
         bSWba8MJ9lBzB6dF1jxGvJ3qIYeKp/bW7QW4uCwDCSazeGcvbsxYEp4aDF+MD2dZ4IzX
         UEdsWvh4XRlbJV5k+gOdJX4XCjAQGIKEswos8CyixQs1zEgy0Mno1/SOfXovF6KoaHqn
         Gsdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=YAbpIis/w2297iCt96qCCK9VooOBw3jgsoMVgYXZUlg=;
        b=zuqBFZZt0UuPPUFS6zlnT+z+1utC2yin5FnmPF7GUamlVduXygT1TteVo2Rns59R9R
         6yTZSEgKNJnrZR/JHTt24eUWvcLshKYjW2qMNytYJBWrkvYdsGpIcTBOIysdjjMsHHER
         /rpXxXuhcRGuB0um0vrz7rhLSKxICEnb/Xbw6wzuphbZOhP1X9S7600DxDeebzoIQO8b
         0omQ4+GL7CqRCJkIZlmsp/gKFVHt2ZVLrpvp/tMeW+XH+3fAvJkuSfY7auxC3eTTiEen
         Y6XtbdNnxbmU8QyNkDyfYiSbjSVSE7USmztSojVju5FQKazzatKXC2VxamBrLZTm7Me2
         HitQ==
X-Gm-Message-State: AOAM531s6nz9sV1qxtF9UColCkeGULmcJJqpUoIV0NwbRKJhaJAdWIFy
	42OJNbSXLRwZZIBnA0qKoZrmU0du2wU=
X-Google-Smtp-Source: ABdhPJyPdqFK5+HPrwDsuMMrjU45KjGVwhqAtDx3tQme4ZYf2+dGluAZYMibjMAUq90mLc5BFAiQnw==
X-Received: by 2002:a05:651c:2de:: with SMTP id f30mr7402848ljo.129.1632425560093;
        Thu, 23 Sep 2021 12:32:40 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [RFC PATCH V3] xen/gnttab: Store frame GFN in struct page_info on Arm
Date: Thu, 23 Sep 2021 22:32:31 +0300
Message-Id: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Rework Arm implementation to store grant table frame GFN
in struct page_info directly instead of keeping it in
standalone status/shared arrays.

To cover 64-bit/40-bit IPA on Arm64/Arm32 we need the space
to hold 52-bit/28-bit + extra bit value respectively. In order
to not grow the size of struct page_info borrow the required
amount of bits from type_info's count portion which current
context won't suffer (currently only 1 bit is used on Arm).
Please note, to minimize code changes and avoid introducing
an extra #ifdef-s to the header, we keep the same amount of
bits on both subarches, although the count portion on Arm64
could be wider, so we waste some bits here.

Introduce corresponding PGT_* constructs and access macros.
Update existing gnttab macros to deal with GFN value according
to new location. Also update the use of count portion on Arm
in share_xen_page_with_guest().

Update the P2M code to clean said GFN portion when putting
a reference on the grant table page in p2m_put_l3_page().
The added check is based on the assumption that grant table page
is the xen_heap page and its entry has p2m_ram_rw type, which
is correct. However, this check is not entirely precise and we
might end up clearing the GFN portion for other xen_heap pages
with the same p2m_type. But, this action is considered as
harmless, since only grant table pages really use that portion.

And for everything to work correctly introduce arch-specific
macros to be called from alloc_xenheap_pages()/free_xenheap_pages()
which purposes on Arm are to clear the portion before use and
make sure the portion is cleared after use, on x86 these are
just stubs.

This patch is intended to fix the potential issue on Arm
which might happen when remapping grant-table frame.
A guest (or the toolstack) will unmap the grant-table frame
using XENMEM_remove_physmap. This is a generic hypercall,
so on x86, we are relying on the fact the M2P entry will
be cleared on removal. For architecture without the M2P,
the GFN would still be present in the grant frame/status
array. So on the next call to map the page, we will end up to
request the P2M to remove whatever mapping was the given GFN.
This could well be another mapping.

Besides that, this patch simplifies arch code on Arm by
removing arrays and corresponding management code and
as the result gnttab_init_arch/gnttab_destroy_arch helpers
and struct grant_table_arch become useless and can be
dropped globally.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
You can find the related discussions at:
https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/

! Please note, there is still unresolved locking question here for which
I failed to find a suitable solution. So, it is still an RFC !

According to the internal conversation:
Now the GFN field in the struct page_info is accessed from
gnttab_set_frame_gfn() in the grant table code and from page_set_frame_gfn()
in the P2M code (the former uses the latter).

We need to prevent the concurrent access to this field. But, we cannot grab
the grant lock from the P2M code because we will introduce a lock inversion.
The page_set_frame_gfn() will be called from the P2M code with the p2m lock held
and then acquire the grant table lock. The gnttab_map_frame() will do the inverse.

Changes RFC1 -> RFC2:
 - update patch description
 - add/update comments in code
 - clarify check in p2m_put_l3_page()
 - introduce arch_alloc_xenheap_page() and arch_free_xenheap_page()
   and drop page_arch_init()
 - add ASSERT to gnttab_shared_page() and gnttab_status_page()
 - rework changes to Arm's struct page_info: do not split type_info,
   allocate GFN portion by reducing count portion, create corresponding
   PGT_* construct, etc
 - update page_get_frame_gfn() and page_set_frame_gfn()
 - update the use of count portion on Arm
 - drop the leading underscore in the macro parameter names

Changes RFC2 -> RFC3:
 - update patch description
 - drop PGT_count_base and MASK_INSR() in share_xen_page_with_guest()
 - update alloc_xenheap_page() and free_xenheap_page() for SEPARATE_XENHEAP
   case (Arm32)
 - provide an extra bit for GFN portion, to get PGT_INVALID_FRAME_GFN
   one bit more than the maximum number of physical address bits on Arm32
---
 xen/arch/arm/mm.c                 |  8 ++++--
 xen/arch/arm/p2m.c                | 21 ++++++++++++---
 xen/common/grant_table.c          |  9 -------
 xen/common/page_alloc.c           | 20 +++++++++++++-
 xen/include/asm-arm/grant_table.h | 57 +++++++++++++++------------------------
 xen/include/asm-arm/mm.h          | 35 +++++++++++++++++++++---
 xen/include/asm-x86/grant_table.h |  5 ----
 xen/include/asm-x86/mm.h          |  4 +++
 8 files changed, 99 insertions(+), 60 deletions(-)

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index eea926d..b1e42e5 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1376,14 +1376,18 @@ unsigned long domain_get_maximum_gpfn(struct domain *d)
 void share_xen_page_with_guest(struct page_info *page, struct domain *d,
                                enum XENSHARE_flags flags)
 {
+    unsigned long type_info;
+
     if ( page_get_owner(page) == d )
         return;
 
     spin_lock(&d->page_alloc_lock);
 
     /* The incremented type count pins as writable or read-only. */
-    page->u.inuse.type_info =
-        (flags == SHARE_ro ? PGT_none : PGT_writable_page) | 1;
+    type_info = page->u.inuse.type_info & ~(PGT_type_mask | PGT_count_mask);
+    page->u.inuse.type_info = type_info |
+        (flags == SHARE_ro ? PGT_none : PGT_writable_page) |
+        MASK_INSR(1, PGT_count_mask);
 
     page_set_owner(page, d);
     smp_wmb(); /* install valid domain ptr before updating refcnt. */
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 8b20b43..7a8d92d 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -718,8 +718,10 @@ static int p2m_mem_access_radix_set(struct p2m_domain *p2m, gfn_t gfn,
  * TODO: Handle superpages, for now we only take special references for leaf
  * pages (specifically foreign ones, which can't be super mapped today).
  */
-static void p2m_put_l3_page(const lpae_t pte)
+static void p2m_put_l3_page(struct p2m_domain *p2m, const lpae_t pte)
 {
+    mfn_t mfn = lpae_get_mfn(pte);
+
     ASSERT(p2m_is_valid(pte));
 
     /*
@@ -731,11 +733,22 @@ static void p2m_put_l3_page(const lpae_t pte)
      */
     if ( p2m_is_foreign(pte.p2m.type) )
     {
-        mfn_t mfn = lpae_get_mfn(pte);
-
         ASSERT(mfn_valid(mfn));
         put_page(mfn_to_page(mfn));
     }
+
+#ifdef CONFIG_GRANT_TABLE
+    /*
+     * Check whether we deal with grant table page. As the grant table page
+     * is xen_heap page and its entry has known p2m type, detect it and mark
+     * the stored GFN as invalid. Although this check is not precise and we
+     * might end up updating this for other xen_heap pages, this action is
+     * harmless to these pages since only grant table pages have this field
+     * in use. So, at worst, unnecessary action might be performed.
+     */
+    if ( (pte.p2m.type == p2m_ram_rw) && is_xen_heap_mfn(mfn) )
+        page_set_frame_gfn(mfn_to_page(mfn), INVALID_GFN);
+#endif
 }
 
 /* Free lpae sub-tree behind an entry */
@@ -768,7 +781,7 @@ static void p2m_free_entry(struct p2m_domain *p2m,
         p2m->stats.mappings[level]--;
         /* Nothing to do if the entry is a super-page. */
         if ( level == 3 )
-            p2m_put_l3_page(entry);
+            p2m_put_l3_page(p2m, entry);
         return;
     }
 
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index fe1fc11..08fc827 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -93,8 +93,6 @@ struct grant_table {
 
     /* Domain to which this struct grant_table belongs. */
     const struct domain *domain;
-
-    struct grant_table_arch arch;
 };
 
 unsigned int __read_mostly opt_max_grant_frames = 64;
@@ -1981,14 +1979,9 @@ int grant_table_init(struct domain *d, int max_grant_frames,
 
     grant_write_lock(gt);
 
-    ret = gnttab_init_arch(gt);
-    if ( ret )
-        goto unlock;
-
     /* gnttab_grow_table() allocates a min number of frames, so 0 is okay. */
     ret = gnttab_grow_table(d, 0);
 
- unlock:
     grant_write_unlock(gt);
 
  out:
@@ -3894,8 +3887,6 @@ grant_table_destroy(
     if ( t == NULL )
         return;
 
-    gnttab_destroy_arch(t);
-
     for ( i = 0; i < nr_grant_frames(t); i++ )
         free_xenheap_page(t->shared_raw[i]);
     xfree(t->shared_raw);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 5801358..aafd847 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2161,6 +2161,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe)
 void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 {
     struct page_info *pg;
+    unsigned int i;
 
     ASSERT(!in_irq());
 
@@ -2169,6 +2170,9 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
     if ( unlikely(pg == NULL) )
         return NULL;
 
+    for ( i = 0; i < (1u << order); i++ )
+        arch_alloc_xenheap_page(&pg[i]);
+
     memguard_unguard_range(page_to_virt(pg), 1 << (order + PAGE_SHIFT));
 
     return page_to_virt(pg);
@@ -2177,14 +2181,22 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
 
 void free_xenheap_pages(void *v, unsigned int order)
 {
+    struct page_info *pg;
+    unsigned int i;
+
     ASSERT(!in_irq());
 
     if ( v == NULL )
         return;
 
+    pg = virt_to_page(v);
+
     memguard_guard_range(v, 1 << (order + PAGE_SHIFT));
 
-    free_heap_pages(virt_to_page(v), order, false);
+    for ( i = 0; i < (1u << order); i++ )
+        arch_free_xenheap_page(&pg[i]);
+
+    free_heap_pages(pg, order, false);
 }
 
 #else  /* !CONFIG_SEPARATE_XENHEAP */
@@ -2220,7 +2232,10 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags)
         return NULL;
 
     for ( i = 0; i < (1u << order); i++ )
+    {
         pg[i].count_info |= PGC_xen_heap;
+        arch_alloc_xenheap_page(&pg[i]);
+    }
 
     return page_to_virt(pg);
 }
@@ -2238,7 +2253,10 @@ void free_xenheap_pages(void *v, unsigned int order)
     pg = virt_to_page(v);
 
     for ( i = 0; i < (1u << order); i++ )
+    {
         pg[i].count_info &= ~PGC_xen_heap;
+        arch_free_xenheap_page(&pg[i]);
+    }
 
     free_heap_pages(pg, order, true);
 }
diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h
index 0ce77f9..479339d 100644
--- a/xen/include/asm-arm/grant_table.h
+++ b/xen/include/asm-arm/grant_table.h
@@ -11,11 +11,6 @@
 #define INITIAL_NR_GRANT_FRAMES 1U
 #define GNTTAB_MAX_VERSION 1
 
-struct grant_table_arch {
-    gfn_t *shared_gfn;
-    gfn_t *status_gfn;
-};
-
 static inline void gnttab_clear_flags(struct domain *d,
                                       unsigned int mask, uint16_t *addr)
 {
@@ -46,35 +41,11 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
 #define gnttab_dom0_frames()                                             \
     min_t(unsigned int, opt_max_grant_frames, PFN_DOWN(_etext - _stext))
 
-#define gnttab_init_arch(gt)                                             \
-({                                                                       \
-    unsigned int ngf_ = (gt)->max_grant_frames;                          \
-    unsigned int nsf_ = grant_to_status_frames(ngf_);                    \
-                                                                         \
-    (gt)->arch.shared_gfn = xmalloc_array(gfn_t, ngf_);                  \
-    (gt)->arch.status_gfn = xmalloc_array(gfn_t, nsf_);                  \
-    if ( (gt)->arch.shared_gfn && (gt)->arch.status_gfn )                \
-    {                                                                    \
-        while ( ngf_-- )                                                 \
-            (gt)->arch.shared_gfn[ngf_] = INVALID_GFN;                   \
-        while ( nsf_-- )                                                 \
-            (gt)->arch.status_gfn[nsf_] = INVALID_GFN;                   \
-    }                                                                    \
-    else                                                                 \
-        gnttab_destroy_arch(gt);                                         \
-    (gt)->arch.shared_gfn ? 0 : -ENOMEM;                                 \
-})
-
-#define gnttab_destroy_arch(gt)                                          \
-    do {                                                                 \
-        XFREE((gt)->arch.shared_gfn);                                    \
-        XFREE((gt)->arch.status_gfn);                                    \
-    } while ( 0 )
-
 #define gnttab_set_frame_gfn(gt, st, idx, gfn)                           \
     do {                                                                 \
-        ((st) ? (gt)->arch.status_gfn : (gt)->arch.shared_gfn)[idx] =    \
-            (gfn);                                                       \
+        struct page_info *pg_ = (st) ? gnttab_status_page(gt, idx)       \
+                                     : gnttab_shared_page(gt, idx);      \
+        page_set_frame_gfn(pg_, gfn);                                    \
     } while ( 0 )
 
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
@@ -82,11 +53,25 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn,
         : gnttab_shared_gfn(NULL, gt, idx);                              \
 })
 
-#define gnttab_shared_gfn(d, t, i)                                       \
-    (((i) >= nr_grant_frames(t)) ? INVALID_GFN : (t)->arch.shared_gfn[i])
+#define gnttab_shared_page(t, i) ({                                      \
+    ASSERT((t)->shared_raw[i]);                                          \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->shared_raw[i])));                \
+})
+
+#define gnttab_status_page(t, i) ({                                      \
+    ASSERT((t)->status[i]);                                              \
+    mfn_to_page(_mfn(__virt_to_mfn((t)->status[i])));                    \
+})
 
-#define gnttab_status_gfn(d, t, i)                                       \
-    (((i) >= nr_status_frames(t)) ? INVALID_GFN : (t)->arch.status_gfn[i])
+#define gnttab_shared_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_shared_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
+
+#define gnttab_status_gfn(d, t, i) ({                                    \
+    struct page_info *pg_ = gnttab_status_page(t, i);                    \
+    page_get_frame_gfn(pg_);                                             \
+})
 
 #define gnttab_need_iommu_mapping(d)                    \
     (is_domain_direct_mapped(d) && is_iommu_enabled(d))
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index 7b5e7b7..a00c5f5 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -98,9 +98,17 @@ struct page_info
 #define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
 #define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
 
- /* Count of uses of this frame as its current type. */
-#define PGT_count_width   PG_shift(2)
-#define PGT_count_mask    ((1UL<<PGT_count_width)-1)
+ /* 2-bit count of uses of this frame as its current type. */
+#define PGT_count_mask    PG_mask(3, 3)
+
+/*
+ * Stored in bits [28:0] or [60:0] GFN if page is used for grant table frame.
+ * This only valid for the xenheap pages.
+ */
+#define PGT_gfn_width     PG_shift(3)
+#define PGT_gfn_mask      ((1UL<<PGT_gfn_width)-1)
+
+#define PGT_INVALID_FRAME_GFN   _gfn(PGT_gfn_mask)
 
  /* Cleared when the owning guest 'frees' this page. */
 #define _PGC_allocated    PG_shift(1)
@@ -166,6 +174,27 @@ extern unsigned long xenheap_base_pdx;
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+#define page_get_frame_gfn(p) ({                                \
+    gfn_t gfn_ = _gfn((p)->u.inuse.type_info & PGT_gfn_mask);   \
+    gfn_eq(gfn_, PGT_INVALID_FRAME_GFN) ? INVALID_GFN : gfn_;   \
+})
+
+#define page_set_frame_gfn(p, gfn) ({                           \
+    gfn_t gfn_ = gfn_eq(gfn, INVALID_GFN) ?                     \
+                 PGT_INVALID_FRAME_GFN : gfn;                   \
+    (p)->u.inuse.type_info &= ~PGT_gfn_mask;                    \
+    (p)->u.inuse.type_info |= gfn_x(gfn_);                      \
+})
+
+/*
+ * As the struct page_info representing the xen_heap page can contain
+ * the grant table frame GFN on Arm we need to clear it beforehand and
+ * make sure it is not still set when freeing a page.
+ */
+#define arch_alloc_xenheap_page(p)   page_set_frame_gfn(p, INVALID_GFN)
+#define arch_free_xenheap_page(p) \
+    BUG_ON(!gfn_eq(page_get_frame_gfn(p), INVALID_GFN))
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 /* PDX of the first page in the frame table. */
 extern unsigned long frametable_base_pdx;
diff --git a/xen/include/asm-x86/grant_table.h b/xen/include/asm-x86/grant_table.h
index 84e3296..0eb018f 100644
--- a/xen/include/asm-x86/grant_table.h
+++ b/xen/include/asm-x86/grant_table.h
@@ -14,9 +14,6 @@
 
 #define INITIAL_NR_GRANT_FRAMES 1U
 
-struct grant_table_arch {
-};
-
 static inline int create_grant_host_mapping(uint64_t addr, mfn_t frame,
                                             unsigned int flags,
                                             unsigned int cache_flags)
@@ -35,8 +32,6 @@ static inline int replace_grant_host_mapping(uint64_t addr, mfn_t frame,
     return replace_grant_pv_mapping(addr, frame, new_addr, flags);
 }
 
-#define gnttab_init_arch(gt) 0
-#define gnttab_destroy_arch(gt) do {} while ( 0 )
 #define gnttab_set_frame_gfn(gt, st, idx, gfn) do {} while ( 0 )
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
     mfn_t mfn_ = (st) ? gnttab_status_mfn(gt, idx)                       \
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index cb90527..04d8704 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -327,6 +327,10 @@ struct page_info
 
 #define maddr_get_owner(ma)   (page_get_owner(maddr_to_page((ma))))
 
+/* No arch-specific actions are needed for the xen_heap page */
+#define arch_alloc_xenheap_page(p)   do {} while ( 0 )
+#define arch_free_xenheap_page(p)    do {} while ( 0 )
+
 #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
 extern unsigned long max_page;
 extern unsigned long total_pages;
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 19:46:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 19:46:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194547.346585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUfd-0001HY-4S; Thu, 23 Sep 2021 19:46:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194547.346585; Thu, 23 Sep 2021 19:46:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUfd-0001HR-0R; Thu, 23 Sep 2021 19:46:13 +0000
Received: by outflank-mailman (input) for mailman id 194547;
 Thu, 23 Sep 2021 19:46:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CYt9=ON=linux.ibm.com=rppt@srs-us1.protection.inumbo.net>)
 id 1mTUfb-0001HL-Kj
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 19:46:11 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.156.1])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e655d3c6-1ca6-11ec-ba89-12813bfff9fa;
 Thu, 23 Sep 2021 19:46:10 +0000 (UTC)
Received: from pps.filterd (m0098393.ppops.net [127.0.0.1])
 by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NJW36f031236; 
 Thu, 23 Sep 2021 15:45:59 -0400
Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com
 [149.81.74.107])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3b8wkuupat-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 15:45:58 -0400
Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1])
 by ppma03fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18NJRuDe003688;
 Thu, 23 Sep 2021 19:45:56 GMT
Received: from b06cxnps3075.portsmouth.uk.ibm.com
 (d06relay10.portsmouth.uk.ibm.com [9.149.109.195])
 by ppma03fra.de.ibm.com with ESMTP id 3b7q6kd587-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 19:45:56 +0000
Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com
 [9.149.105.58])
 by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18NJjrxv44106172
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 23 Sep 2021 19:45:53 GMT
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id A19734C04A;
 Thu, 23 Sep 2021 19:45:53 +0000 (GMT)
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 903DC4C05A;
 Thu, 23 Sep 2021 19:45:51 +0000 (GMT)
Received: from linux.ibm.com (unknown [9.145.159.121])
 by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS;
 Thu, 23 Sep 2021 19:45:51 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e655d3c6-1ca6-11ec-ba89-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc :
 subject : message-id : references : mime-version : content-type :
 in-reply-to; s=pp1; bh=OlfxwL5SxCQJGsEygNnP+Bd9S5CycoibyIwI1T9qbFQ=;
 b=o2g9xUdg4iWokOSpifGukNkZK7jrG3PCnBFvuz4C7oP+gzjHDglARWvYTHBDYKWgrmUN
 nywTanPgUpTOTd5/xMfhi1L+mJwrCvVoKxej3UCLp5sG7C25GVynwAmr4q0nxjw5T5YF
 XfSS+haqO/MAAWW7EoLHEph+pIjpH8koxcOuzA2lkqyOwZ+Lt0jX5schDBGsmjxT7nCn
 nj0KpCKZ1MAIxT1dHKe+ArroeI/VY7R10b19vbUGLmbRaxdWWrC+tHtvcRU2gpGyiZee
 fQz965XLLMlLTgvc9SCxBhiIocufoeoMuCL9s+OrHLUbKOqWsHzC5EoSrMfU3XKh1xK9 Bg== 
Date: Thu, 23 Sep 2021 22:45:49 +0300
From: Mike Rapoport <rppt@linux.ibm.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Rapoport <rppt@kernel.org>, Andrew Morton <akpm@linux-foundation.org>,
        devicetree <devicetree@vger.kernel.org>,
        iommu <iommu@lists.linux-foundation.org>,
        kasan-dev <kasan-dev@googlegroups.com>, KVM list <kvm@vger.kernel.org>,
        alpha <linux-alpha@vger.kernel.org>,
        Linux ARM <linux-arm-kernel@lists.infradead.org>,
        linux-efi <linux-efi@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        "open list:BROADCOM NVRAM DRIVER" <linux-mips@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        linux-riscv <linux-riscv@lists.infradead.org>,
        linux-s390 <linux-s390@vger.kernel.org>,
        Linux-sh list <linux-sh@vger.kernel.org>,
        "open list:SYNOPSYS ARC ARCHITECTURE" <linux-snps-arc@lists.infradead.org>,
        linux-um <linux-um@lists.infradead.org>, linux-usb@vger.kernel.org,
        linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
        linux-sparc <sparclinux@vger.kernel.org>,
        xen-devel@lists.xenproject.org
Subject: Re: [PATCH 0/3] memblock: cleanup memblock_free interface
Message-ID: <YUzZberbgZE+7HEo@linux.ibm.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <CAHk-=wiJB8H5pZz-AKaSJ7ViRtdxQGJT7eOByp8DJx2OwZSYwA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAHk-=wiJB8H5pZz-AKaSJ7ViRtdxQGJT7eOByp8DJx2OwZSYwA@mail.gmail.com>
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: NmcKtlYbna1qS7iyZ4wr-YnA7UP6mwwQ
X-Proofpoint-ORIG-GUID: NmcKtlYbna1qS7iyZ4wr-YnA7UP6mwwQ
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_06,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 lowpriorityscore=0 mlxscore=0 malwarescore=0 spamscore=0
 priorityscore=1501 bulkscore=0 clxscore=1015 suspectscore=0
 mlxlogscore=691 adultscore=0 phishscore=0 classifier=spam adjust=0
 reason=mlx scancount=1 engine=8.12.0-2109200000
 definitions=main-2109230115

Hi Linus,

On Thu, Sep 23, 2021 at 09:01:46AM -0700, Linus Torvalds wrote:
> On Thu, Sep 23, 2021 at 12:43 AM Mike Rapoport <rppt@kernel.org> wrote:
> >
> You need to be a LOT more careful.
> 
> From a trivial check - exactly because I looked at doing it with a
> script, and decided it's not so easy - I found cases like this:
> 
> -               memblock_free(__pa(paca_ptrs) + new_ptrs_size,
> +               memblock_free(paca_ptrs + new_ptrs_size,
> 
> which is COMPLETELY wrong.

I did use a coccinelle script that's slightly more robust that a sed you've
sent, but then I did a manual review, hence the two small patches with
fixes. Indeed I missed this one, so to be on the safe side I'll rename only
the obvious cases where coccinelle can be used reliably and leave all the
rest as it's now. If somebody cares enough they can update it later.
 
> And no, making the scripting just replace '__pa(x)' with '(void *)(x)'

These were actually manual and they are required for variables that
used as virtual addresses but have unsigned long type, like e.g.
initrd_start. So it's either __pa(x) or (void *).

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 19:48:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 19:48:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194553.346596 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUhO-0002Fn-In; Thu, 23 Sep 2021 19:48:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194553.346596; Thu, 23 Sep 2021 19:48:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUhO-0002Fg-Ew; Thu, 23 Sep 2021 19:48:02 +0000
Received: by outflank-mailman (input) for mailman id 194553;
 Thu, 23 Sep 2021 19:48:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTUhN-0002Fa-9l
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 19:48:01 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7ad8b244-4bd1-41b7-b359-1407ac9800ea;
 Thu, 23 Sep 2021 19:48:00 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1CDF360F6F;
 Thu, 23 Sep 2021 19:47:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ad8b244-4bd1-41b7-b359-1407ac9800ea
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632426479;
	bh=OeWc14n9eipH1NnZQkPVKzOesAfMuGylT40OBED6fnI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RHH6bQOPihZRnwDCQoYpuBzKCTvOYtNwF12A6tTkkaZI+OnweE3Aa0ovbwXkx3ySQ
	 d7Tpn0knfTv8BcroVAStQr/okxA1LL15V4LndkFfAf+tETDLmxycdEkTUEcRqJMaBj
	 XduUuUI7F1nMfBGQ1RUJ9fRBoix+6gXrPiji0dQeI+mokdYTJ8Gn+1iCHGeeyUqwC3
	 IUV1lj5TN2TRH0W9KAESVcUkOn2MbB478Xp5MfDdQnF0tNN2aZIB/P05i7JX8OrHIL
	 vOXmDC6+flu3RqIBKnoOYCs04E868thQhVNWWFzXJDIsusfFytfCx8CbHDPsGqc21x
	 u84HlbqSdXufA==
Date: Thu, 23 Sep 2021 12:47:58 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org, 
    sstabellini@kernel.org, jbeulich@suse.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and
 PV
In-Reply-To: <20210923095345.185489-1-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109231217050.17979@sstabellini-ThinkPad-T480s>
References: <20210923095345.185489-1-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> Currently PCI backend implements multiple functionalities at a time.
> To name a few:
> 1. It is used as a database for assignable PCI devices, e.g. xl
>    pci-assignable-{add|remove|list} manipulates that list. So, whenever
>    the toolstack needs to know which PCI devices can be passed through
>    it reads that from the relevant sysfs entries of the pciback.
> 2. It is used to hold the unbound PCI devices list, e.g. when passing
>    through a PCI device it needs to be unbound from the relevant device
>    driver and bound to pciback (strictly speaking it is not required
>    that the device is bound to pciback, but pciback is again used as a
>    database of the passed through PCI devices, so we can re-bind the
>    devices back to their original drivers when guest domain shuts down)
> 3. Device reset for the devices being passed through
> 4. Para-virtualized use-cases support
> 
> The para-virtualized part of the driver is not always needed as some
> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
> model for PCI device passthrough. For such use-cases make the very
> first step in splitting the xen-pciback driver into two parts: extended
> PCI stub and PCI PV backend drivers. At the moment x86 platform will
> continue using CONFIG_XEN_PCIDEV_BACKEND for the fully featured backend
> driver and new platforms may build a driver with limited functionality
> (no PV) by enabling CONFIG_XEN_PCIDEV_STUB.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> ---
> Changes since v2:
>  - swap the patch order
> New in v2
> ---
>  drivers/xen/Kconfig               | 24 ++++++++++++++++++++++++
>  drivers/xen/Makefile              |  2 +-
>  drivers/xen/xen-pciback/Makefile  |  1 +
>  drivers/xen/xen-pciback/pciback.h |  5 +++++
>  drivers/xen/xen-pciback/xenbus.c  |  6 +++++-
>  5 files changed, 36 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
> index a37eb52fb401..6e92c6be19f1 100644
> --- a/drivers/xen/Kconfig
> +++ b/drivers/xen/Kconfig
> @@ -180,10 +180,34 @@ config SWIOTLB_XEN
>  	select DMA_OPS
>  	select SWIOTLB
>  
> +config XEN_PCI_STUB
> +	bool
> +
> +config XEN_PCIDEV_STUB
> +	tristate "Xen PCI-device stub driver"
> +	depends on PCI && !X86 && XEN
> +	depends on XEN_BACKEND
> +	select XEN_PCI_STUB
> +	default m
> +	help
> +	  The PCI device stub driver provides limited version of the PCI
> +	  device backend driver without para-virtualized support for guests.
> +	  If you select this to be a module, you will need to make sure no
> +	  other driver has bound to the device(s) you want to make visible to
> +	  other guests.
> +
> +	  The "hide" parameter (only applicable if backend driver is compiled
> +	  into the kernel) allows you to bind the PCI devices to this module
> +	  from the default device drivers. The argument is the list of PCI BDFs:
> +	  xen-pciback.hide=(03:00.0)(04:00.0)
> +
> +	  If in doubt, say m.
> +

I get this build error on ARM:

drivers/xen/xen-pciback/pci_stub.c:22:10: fatal error: asm/xen/pci.h: No such file or directory
 #include <asm/xen/pci.h>
          ^~~~~~~~~~~~~~~
compilation terminated.
scripts/Makefile.build:277: recipe for target 'drivers/xen/xen-pciback/pci_stub.o' failed


I imagine that the second patch is required for the build on ARM. I
suggest to move the definition of XEN_PCIDEV_STUB to the second patch.
(keep XEN_PCI_STUB and xen_pcibk_pv_support here.)

Alternatively, just remove "tristate "Xen PCI-device stub driver"" from
here so that XEN_PCIDEV_STUB becomes non-selectable. You can add the
tristate property in patch #2.



>  config XEN_PCIDEV_BACKEND
>  	tristate "Xen PCI-device backend driver"
>  	depends on PCI && X86 && XEN
>  	depends on XEN_BACKEND
> +	select XEN_PCI_STUB
>  	default m
>  	help
>  	  The PCI device backend driver allows the kernel to export arbitrary
> diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
> index 3434593455b2..5aae66e638a7 100644
> --- a/drivers/xen/Makefile
> +++ b/drivers/xen/Makefile
> @@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
>  obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
>  obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
>  obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
> -obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
> +obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
>  obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
>  obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
>  obj-$(CONFIG_XEN_EFI)			+= efi.o
> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
> index e8d981d43235..e2cb376444a6 100644
> --- a/drivers/xen/xen-pciback/Makefile
> +++ b/drivers/xen/xen-pciback/Makefile
> @@ -1,5 +1,6 @@
>  # SPDX-License-Identifier: GPL-2.0
>  obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
>  
>  xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
>  xen-pciback-y += conf_space.o conf_space_header.o \
> diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
> index 95e28ee48d52..9a64196e831d 100644
> --- a/drivers/xen/xen-pciback/pciback.h
> +++ b/drivers/xen/xen-pciback/pciback.h
> @@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
>  				    struct pci_dev *dev);
>  void pcistub_put_pci_dev(struct pci_dev *dev);
>  
> +static inline bool xen_pcibk_pv_support(void)
> +{
> +	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
> +}
> +
>  /* Ensure a device is turned off or reset */
>  void xen_pcibk_reset_device(struct pci_dev *pdev);
>  
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index c09c7ebd6968..f8ba2903a3ff 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
>  
>  int __init xen_pcibk_xenbus_register(void)
>  {
> +	if (!xen_pcibk_pv_support())
> +		return 0;
> +
>  	xen_pcibk_backend = &xen_pcibk_vpci_backend;
>  	if (passthrough)
>  		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
> @@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
>  
>  void __exit xen_pcibk_xenbus_unregister(void)
>  {
> -	xenbus_unregister_driver(&xen_pcibk_driver);
> +	if (xen_pcibk_pv_support())
> +		xenbus_unregister_driver(&xen_pcibk_driver);
>  }
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:00:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:00:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194559.346606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUte-0004r0-NA; Thu, 23 Sep 2021 20:00:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194559.346606; Thu, 23 Sep 2021 20:00:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTUte-0004qt-KI; Thu, 23 Sep 2021 20:00:42 +0000
Received: by outflank-mailman (input) for mailman id 194559;
 Thu, 23 Sep 2021 20:00:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTUte-0004qW-2m
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:00:42 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ed324d26-1ca8-11ec-ba89-12813bfff9fa;
 Thu, 23 Sep 2021 20:00:40 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6BD1760F6D;
 Thu, 23 Sep 2021 20:00:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed324d26-1ca8-11ec-ba89-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632427239;
	bh=ds92p2VMyOLjwhV8WhWyt/jh2tUUrZrBeNUkmH+/IbU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ry0mam8DUkXj+3s5eAc/3Fd3TKQfxaQsg8HYBYEYHMRquyizJCqssbGOWGWkP2EIS
	 ogXEH3qHou7nVcDXMzmixXXwcizrawx9wmhw5WFAr2M0ikjaBlcqbz1X5E2h/UTP1G
	 DRLyHeGRyAGeL3BoJc78wXBiSFu8AnWU/h/LfgFV19TZRjiHlnk5gJFJP7iLnhTYGP
	 n21Bz4Tiy3F0nODSEDDo0NyaqFJZKaPl27RMHCzm6EBtqTnD4GLBt8lLxrNuleS+N9
	 ediC35mHvG8vMVLrdv9pghTUul1Uzz11MfeNqNeyta0lzAvuGcaHzuXVoppWwic0QR
	 i/ooFwxhqP3Dg==
Date: Thu, 23 Sep 2021 13:00:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
    boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org, 
    sstabellini@kernel.org, jbeulich@suse.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: Re: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
In-Reply-To: <20210923095345.185489-2-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s>
References: <20210923095345.185489-1-andr2000@gmail.com> <20210923095345.185489-2-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-36133199-1632427239=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-36133199-1632427239=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Xen-pciback driver was designed to be built for x86 only. But it
> can also be used by other architectures, e.g. Arm.
> Re-structure the driver in a way that it can be built for other
> platforms as well.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

The patch looks good to me. Only one thing: on ARM32 I get:

drivers/xen/xen-pciback/conf_space_header.c: In function ‘bar_init’:
drivers/xen/xen-pciback/conf_space_header.c:239:34: warning: right shift count >= width of type [-Wshift-count-overflow]
    bar->val = res[pos - 1].start >> 32;
                                  ^~
drivers/xen/xen-pciback/conf_space_header.c:240:49: warning: right shift count >= width of type [-Wshift-count-overflow]
    bar->len_val = -resource_size(&res[pos - 1]) >> 32;
 
 
resource_size_t is defined as phys_addr_t and it can be 32bit on arm32.


One fix is to surround:

		if (pos && (res[pos - 1].flags & IORESOURCE_MEM_64)) {
			bar->val = res[pos - 1].start >> 32;
			bar->len_val = -resource_size(&res[pos - 1]) >> 32;
			return bar;
		}

with #ifdef PHYS_ADDR_T_64BIT



> ---
> Since v2:
>  - swap the patch order
> Since v1:
>  - Do not move pci_xen_initial_domain as it is x86 specific
> ---
>  arch/x86/include/asm/xen/pci.h     | 18 +------
>  arch/x86/pci/xen.c                 | 74 +----------------------------
>  drivers/xen/pci.c                  | 75 ++++++++++++++++++++++++++++++
>  drivers/xen/xen-pciback/pci_stub.c |  3 +-
>  drivers/xen/xen-pciback/xenbus.c   |  2 +-
>  include/xen/pci.h                  | 28 +++++++++++
>  6 files changed, 108 insertions(+), 92 deletions(-)
>  create mode 100644 include/xen/pci.h
> 
> diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
> index 3506d8c598c1..2889f091f459 100644
> --- a/arch/x86/include/asm/xen/pci.h
> +++ b/arch/x86/include/asm/xen/pci.h
> @@ -16,26 +16,10 @@ static inline int pci_xen_hvm_init(void)
>  #endif
>  #if defined(CONFIG_XEN_DOM0)
>  int __init pci_xen_initial_domain(void);
> -int xen_find_device_domain_owner(struct pci_dev *dev);
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> -int xen_unregister_device_domain_owner(struct pci_dev *dev);
>  #else
>  static inline int __init pci_xen_initial_domain(void)
>  {
> -	return -1;
> -}
> -static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> -}
> -static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> -						   uint16_t domain)
> -{
> -	return -1;
> -}
> -static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	return -1;
> +       return -1;
>  }
>  #endif
>  
> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
> index 3d41a09c2c14..4a45b0bf9ae4 100644
> --- a/arch/x86/pci/xen.c
> +++ b/arch/x86/pci/xen.c
> @@ -23,6 +23,7 @@
>  
>  #include <xen/features.h>
>  #include <xen/events.h>
> +#include <xen/pci.h>
>  #include <asm/xen/pci.h>
>  #include <asm/xen/cpuid.h>
>  #include <asm/apic.h>
> @@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
>  	}
>  	return 0;
>  }
> -
> -struct xen_device_domain_owner {
> -	domid_t domain;
> -	struct pci_dev *dev;
> -	struct list_head list;
> -};
> -
> -static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> -static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> -
> -static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	list_for_each_entry(owner, &dev_domain_list, list) {
> -		if (owner->dev == dev)
> -			return owner;
> -	}
> -	return NULL;
> -}
> -
> -int xen_find_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -	int domain = -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (owner)
> -		domain = owner->domain;
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return domain;
> -}
> -EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> -
> -int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> -	if (!owner)
> -		return -ENODEV;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	if (find_device(dev)) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		kfree(owner);
> -		return -EEXIST;
> -	}
> -	owner->domain = domain;
> -	owner->dev = dev;
> -	list_add_tail(&owner->list, &dev_domain_list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> -
> -int xen_unregister_device_domain_owner(struct pci_dev *dev)
> -{
> -	struct xen_device_domain_owner *owner;
> -
> -	spin_lock(&dev_domain_list_spinlock);
> -	owner = find_device(dev);
> -	if (!owner) {
> -		spin_unlock(&dev_domain_list_spinlock);
> -		return -ENODEV;
> -	}
> -	list_del(&owner->list);
> -	spin_unlock(&dev_domain_list_spinlock);
> -	kfree(owner);
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
>  #endif
> diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
> index 224df03ce42e..fc8c1249d49f 100644
> --- a/drivers/xen/pci.c
> +++ b/drivers/xen/pci.c
> @@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
>  	return 0;
>  }
>  #endif
> +
> +#ifdef CONFIG_XEN_DOM0
> +struct xen_device_domain_owner {
> +	domid_t domain;
> +	struct pci_dev *dev;
> +	struct list_head list;
> +};
> +
> +static DEFINE_SPINLOCK(dev_domain_list_spinlock);
> +static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
> +
> +static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	list_for_each_entry(owner, &dev_domain_list, list) {
> +		if (owner->dev == dev)
> +			return owner;
> +	}
> +	return NULL;
> +}
> +
> +int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +	int domain = -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (owner)
> +		domain = owner->domain;
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return domain;
> +}
> +EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
> +
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
> +	if (!owner)
> +		return -ENODEV;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	if (find_device(dev)) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		kfree(owner);
> +		return -EEXIST;
> +	}
> +	owner->domain = domain;
> +	owner->dev = dev;
> +	list_add_tail(&owner->list, &dev_domain_list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
> +
> +int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	struct xen_device_domain_owner *owner;
> +
> +	spin_lock(&dev_domain_list_spinlock);
> +	owner = find_device(dev);
> +	if (!owner) {
> +		spin_unlock(&dev_domain_list_spinlock);
> +		return -ENODEV;
> +	}
> +	list_del(&owner->list);
> +	spin_unlock(&dev_domain_list_spinlock);
> +	kfree(owner);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
> +#endif
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index f8e4faa96ad6..bba527620507 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -19,7 +19,8 @@
>  #include <linux/sched.h>
>  #include <linux/atomic.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
> +#include <xen/xen.h>
>  #include <asm/xen/hypervisor.h>
>  #include <xen/interface/physdev.h>
>  #include "pciback.h"
> diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
> index f8ba2903a3ff..bde63ef677b8 100644
> --- a/drivers/xen/xen-pciback/xenbus.c
> +++ b/drivers/xen/xen-pciback/xenbus.c
> @@ -14,7 +14,7 @@
>  #include <linux/workqueue.h>
>  #include <xen/xenbus.h>
>  #include <xen/events.h>
> -#include <asm/xen/pci.h>
> +#include <xen/pci.h>
>  #include "pciback.h"
>  
>  #define INVALID_EVTCHN_IRQ  (-1)
> diff --git a/include/xen/pci.h b/include/xen/pci.h
> new file mode 100644
> index 000000000000..b8337cf85fd1
> --- /dev/null
> +++ b/include/xen/pci.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __XEN_PCI_H__
> +#define __XEN_PCI_H__
> +
> +#if defined(CONFIG_XEN_DOM0)
> +int xen_find_device_domain_owner(struct pci_dev *dev);
> +int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
> +int xen_unregister_device_domain_owner(struct pci_dev *dev);
> +#else
> +static inline int xen_find_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_register_device_domain_owner(struct pci_dev *dev,
> +						   uint16_t domain)
> +{
> +	return -1;
> +}
> +
> +static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +#endif
> +
> +#endif
> -- 
> 2.25.1
> 
--8323329-36133199-1632427239=:17979--


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:08:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:08:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194564.346618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTV1G-0005rU-Gu; Thu, 23 Sep 2021 20:08:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194564.346618; Thu, 23 Sep 2021 20:08:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTV1G-0005rN-DH; Thu, 23 Sep 2021 20:08:34 +0000
Received: by outflank-mailman (input) for mailman id 194564;
 Thu, 23 Sep 2021 20:08:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTV1F-0005qh-23
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:08:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 66ffd947-a319-4c32-9222-bfdc7c0a8aa0;
 Thu, 23 Sep 2021 20:08:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2428A61241;
 Thu, 23 Sep 2021 20:08:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 66ffd947-a319-4c32-9222-bfdc7c0a8aa0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632427711;
	bh=MNyvKaFgFCUPXsfRzD3Uni2c6kX2doji4UyUo8l97yw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=V26BPQGxZaB4QNujdfM1ze5ozILa0oAsLzjB1tEYa7pIR/SPmPy39Of3Q4ZFDiF4n
	 B/x4OoVpDCt5QoZox7h1/06YszQoG21MiQXigGCN3XqyjhZypBw6HTzHp31Dq0sYkw
	 5/sKMiyi2PTQWAGBUkGegUSgaQ0s1Mac3PZlpjTsHBm/GMfBKLab2qdvV88r+Bo9p6
	 uPCFPEeZRFCKhz9Je0jEZxP/+qyWCH26dvuDZFuZ4p7HkOjgtrP3D75o23DFSQBHQl
	 keviyVMxxLmJH9Xc84nnn47GAojNYKDT56KV1RNxQGRVAtEx+Lq7PBs3MOQlar4YYa
	 uiaV/2+w3fqTw==
Date: Thu, 23 Sep 2021 13:08:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <volodymyr_babchuk@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
In-Reply-To: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109231308200.17979@sstabellini-ThinkPad-T480s>
References: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 6 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Allocate anonymous domheap pages as there is no strict need to
> account them to a particular domain.
> 
> Since XSA-383 "xen/arm: Restrict the amount of memory that dom0less
> domU and dom0 can allocate" the dom0 cannot allocate memory outside
> of the pre-allocated region. This means if we try to allocate
> non-anonymous page to be accounted to dom0 we will get an
> over-allocation issue when assigning that page to the domain.
> The anonymous page, in turn, is not assigned to any domain.
> 
> CC: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

Only one question, which is more architectural: given that these pages
are "unlimited", could the guest exploit the interface somehow to force
Xen to allocate an very high number of anonymous pages?

E.g. could a domain call OPTEE_SMC_RPC_FUNC_ALLOC in a loop to force Xen
to exaust all memory pages?



> ---
>  xen/arch/arm/tee/optee.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..83b4994 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -410,7 +410,7 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
>      if ( !shm_rpc )
>          return ERR_PTR(-ENOMEM);
>  
> -    shm_rpc->xen_arg_pg = alloc_domheap_page(current->domain, 0);
> +    shm_rpc->xen_arg_pg = alloc_domheap_page(NULL, 0);
>      if ( !shm_rpc->xen_arg_pg )
>      {
>          xfree(shm_rpc);
> @@ -774,7 +774,7 @@ static int translate_noncontig(struct optee_domain *ctx,
>       * - There is a plan to implement preemption in the code below, which
>       *   will allow use to increase default MAX_SHM_BUFFER_PG value.
>       */
> -    xen_pgs = alloc_domheap_pages(current->domain, order, 0);
> +    xen_pgs = alloc_domheap_pages(NULL, order, 0);
>      if ( !xen_pgs )
>          return -ENOMEM;
>  
> @@ -938,7 +938,7 @@ static bool copy_std_request(struct cpu_user_regs *regs,
>  
>      BUILD_BUG_ON(OPTEE_MSG_NONCONTIG_PAGE_SIZE > PAGE_SIZE);
>  
> -    call->xen_arg_pg = alloc_domheap_page(current->domain, 0);
> +    call->xen_arg_pg = alloc_domheap_page(NULL, 0);
>      if ( !call->xen_arg_pg )
>      {
>          set_user_reg(regs, 0, OPTEE_SMC_RETURN_ENOMEM);
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:41:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:41:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194569.346628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVX8-0002X2-3e; Thu, 23 Sep 2021 20:41:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194569.346628; Thu, 23 Sep 2021 20:41:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVX8-0002Wv-0m; Thu, 23 Sep 2021 20:41:30 +0000
Received: by outflank-mailman (input) for mailman id 194569;
 Thu, 23 Sep 2021 20:41:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTVX6-0002Wp-H1
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:41:28 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0aae2d68-de8e-45e1-ac18-e42cb2b32305;
 Thu, 23 Sep 2021 20:41:26 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A96FC61211;
 Thu, 23 Sep 2021 20:41:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0aae2d68-de8e-45e1-ac18-e42cb2b32305
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632429686;
	bh=wTV1KAEXYbefzMMk3fnmO2m4Yo9Q8JtJj7B2h/s0uI0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iNIwu/eesVzNCUxtkPqncYe+C8zfA8SCzjs2uZERrPPXtzuthtFlmtDkW5ZCiOx68
	 S1CXQmh4l863Edf99jXCnAu1pEDQqXU0MAnmppSelApUfLF/U5eAMJS3n7560PTTh8
	 lhLocPoxM7FXpGM7a3wfXuchFIECt1v2s3AB7ZZM4pZx37pVWOYxWLHKsP0X3mKHMV
	 FoS1XFEQIenM7rMmtbT7YY+wDi3ecqQIXR3Yfb+jhRNMlCxEUed2wrd0yWFuZ4SorE
	 WFA3t9z3XHtte/Lek0Qph/7CzCHDOpQ3my4TGfzOnWK5hyG+5ME3lLRZRvI2+BfPTl
	 2sfHXJvgHMG+g==
Date: Thu, 23 Sep 2021 13:41:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    andre.przywara@arm.com, Ian Jackson <iwj@xenproject.org>, 
    Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 16/17] arm/libxl: Emulated PCI device tree node in
 libxl
In-Reply-To: <5c66821e2333f6b27cb20a345eaf01a141a6c63b.1632307952.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109231339070.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <5c66821e2333f6b27cb20a345eaf01a141a6c63b.1632307952.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 22 Sep 2021, Rahul Singh wrote:
> libxl will create an emulated PCI device tree node in the device tree to
> enable the guest OS to discover the virtual PCI during guest boot.
> Emulated PCI device tree node will only be created when there is any
> device assigned to guest.
> 
> A new area has been reserved in the arm guest physical map at
> which the VPCI bus is declared in the device tree (reg and ranges
> parameters of the node).
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v2:
> - enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain.
> ---
>  tools/include/libxl.h            |   6 ++
>  tools/libs/light/libxl_arm.c     | 105 +++++++++++++++++++++++++++++++
>  tools/libs/light/libxl_create.c  |   3 +
>  tools/libs/light/libxl_types.idl |   1 +
>  tools/xl/xl_parse.c              |   2 +
>  xen/include/public/arch-arm.h    |  10 +++
>  6 files changed, 127 insertions(+)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d698..3362073b21 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -358,6 +358,12 @@
>   */
>  #define LIBXL_HAVE_BUILDINFO_ARM_VUART 1
>  
> +/*
> + * LIBXL_HAVE_BUILDINFO_ARM_VPCI indicates that the toolstack supports virtual
> + * PCI for ARM.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARM_VPCI 1
> +
>  /*
>   * LIBXL_HAVE_BUILDINFO_GRANT_LIMITS indicates that libxl_domain_build_info
>   * has the max_grant_frames and max_maptrack_frames fields.
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e00..52f1ddce48 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
>      return fdt_property(fdt, "reg", regs, sizeof(regs));
>  }
>  
> +static int fdt_property_values(libxl__gc *gc, void *fdt,
> +        const char *name, unsigned num_cells, ...)
> +{
> +    uint32_t prop[num_cells];
> +    be32 *cells = &prop[0];
> +    int i;
> +    va_list ap;
> +    uint32_t arg;
> +
> +    va_start(ap, num_cells);
> +    for (i = 0 ; i < num_cells; i++) {
> +        arg = va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, name, prop, sizeof(prop));
> +}
> +
> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
> +                                    unsigned addr_cells,
> +                                    unsigned size_cells,
> +                                    unsigned num_regs, ...)
> +{
> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
> +    be32 *cells = &regs[0];
> +    int i;
> +    va_list ap;
> +    uint64_t arg;
> +
> +    va_start(ap, num_regs);
> +    for (i = 0 ; i < num_regs; i++) {
> +        /* Set the memory bit field */
> +        arg = va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +
> +        /* Set the vpci bus address */
> +        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, addr_cells , arg);
> +
> +        /* Set the cpu bus address where vpci address is mapped */
> +        set_cell(&cells, addr_cells, arg);
> +
> +        /* Set the vpci size requested */
> +        arg = size_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, size_cells, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
> +}
> +
>  static int make_root_properties(libxl__gc *gc,
>                                  const libxl_version_info *vers,
>                                  void *fdt)
> @@ -668,6 +720,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
>      return 0;
>  }
>  
> +static int make_vpci_node(libxl__gc *gc, void *fdt,
> +        const struct arch_info *ainfo,
> +        struct xc_dom_image *dom)
> +{
> +    int res;
> +    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
> +    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
> +    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "device_type", "pci");
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
> +    if (res) return res;
> +
> +    res = fdt_property_values(gc, fdt, "bus-range", 2, 0, 255);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 3);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", 2);
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "status", "okay");
> +    if (res) return res;
> +
> +    res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +        GUEST_ROOT_SIZE_CELLS, 2,
> +        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE,
> +        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
> +        GUEST_VPCI_PREFETCH_MEM_SIZE);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
>  static const struct arch_info *get_arch_info(libxl__gc *gc,
>                                               const struct xc_dom_image *dom)
>  {
> @@ -971,6 +1070,9 @@ next_resize:
>          if (info->tee == LIBXL_TEE_TYPE_OPTEE)
>              FDT( make_optee_node(gc, fdt) );
>  
> +        if (libxl_defbool_val(info->arch_arm.vpci))
> +            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
> +
>          if (pfdt)
>              FDT( copy_partial_fdt(gc, fdt, pfdt) );
>  
> @@ -1189,6 +1291,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
>      /* ACPI is disabled by default */
>      libxl_defbool_setdefault(&b_info->acpi, false);
>  
> +    /* VPCI is disabled by default */
> +    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
> +
>      if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
>          return;
>  
> diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
> index e356b2106d..529e475489 100644
> --- a/tools/libs/light/libxl_create.c
> +++ b/tools/libs/light/libxl_create.c
> @@ -632,6 +632,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
>          if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
>              create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
>  
> +        if ( libxl_defbool_val(b_info->arch_arm.vpci) )
> +            create.flags |= XEN_DOMCTL_CDF_vpci;
> +
>          /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
>          libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
>  
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 3f9fff653a..78b1ddf0b8 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>  
>      ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                 ("vuart", libxl_vuart_type),
> +                               ("vpci", libxl_defbool),
>                                ])),
>      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                                ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd5..ffafbeffb4 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>          }
>          if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
>              libxl_defbool_set(&b_info->u.pv.e820_host, true);
> +        if (d_config->num_pcidevs)
> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>      }
>  
>      if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 727541a321..85692eca6c 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -431,6 +431,11 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
>  #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
>  
> +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
> +#define GUEST_VPCI_ADDR_TYPE_MEM            xen_mk_ullong(0x02000000)
> +#define GUEST_VPCI_MEM_ADDR                 xen_mk_ullong(0x22001000)

Please make the address 2MB aligned


> +#define GUEST_VPCI_MEM_SIZE                 xen_mk_ullong(0x10000000)
> +
>  /*
>   * 16MB == 4096 pages reserved for guest to use as a region to map its
>   * grant table in.
> @@ -446,6 +451,11 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_RAM0_BASE   xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1GB */
>  #define GUEST_RAM0_SIZE   xen_mk_ullong(0xc0000000)
>  
> +/* 4GB @ 4GB Prefetch Memory for VPCI */
> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM   xen_mk_ullong(0x42000000)
> +#define GUEST_VPCI_PREFETCH_MEM_ADDR        xen_mk_ullong(0x100000000)
> +#define GUEST_VPCI_PREFETCH_MEM_SIZE        xen_mk_ullong(0x100000000)
> +
>  #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ 8GB */
>  #define GUEST_RAM1_SIZE   xen_mk_ullong(0xfe00000000)
>  
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:54:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:54:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194575.346639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVje-0004Ng-Dy; Thu, 23 Sep 2021 20:54:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194575.346639; Thu, 23 Sep 2021 20:54:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVje-0004NZ-Aw; Thu, 23 Sep 2021 20:54:26 +0000
Received: by outflank-mailman (input) for mailman id 194575;
 Thu, 23 Sep 2021 20:54:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTVjc-0004NT-5Y
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:54:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6da82258-1cb0-11ec-ba8c-12813bfff9fa;
 Thu, 23 Sep 2021 20:54:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9B92A60F6F;
 Thu, 23 Sep 2021 20:54:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6da82258-1cb0-11ec-ba8c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632430461;
	bh=MZ1Cpxe/yyRtHke5E1DT0bNyKEHTkkdv7JaRRzbiH90=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=todhNFywr1KDbI8odMjNjGGQnFmH5hoz7CziyDk91GC/rqC3ENZqFPS1oItmQQuZ4
	 GhefOFCoSEwebMfJi8JwyhLIhlOnLDpUpmgTGnBRRTnDQYN8U97r4seCWma7kmePKs
	 rPvqKyBOE+HNTAiIJi2tKZhec0bcYk8rMnZUonv9MMPyeJoZsEpKOUdmhAkVGlxGYH
	 Fplul9Bmbf941n9sqHPhmwFsh0L6hcAclk4vhV9cHoaUb5Z+MuD3AhTfAkQKVfwXgg
	 7eC/fCPbSQbBvQnF9nLLaSk/9q/sqMJpkPOyJvu9C2uE8Wqqxp06gQsdVUxuA9ZPPh
	 HAa4edvx1jR0A==
Date: Thu, 23 Sep 2021 13:54:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien@xen.org>
cc: Hongda Deng <Hongda.Deng@arm.com>, xen-devel@lists.xenproject.org, 
    sstabellini@kernel.org, Bertrand.Marquis@arm.com, Wei.Chen@arm.com
Subject: Re: [PATCH] xen/arm: vgic to ignore GICD ICPENRn registers access
In-Reply-To: <0b0ede18-b944-8693-dede-616c3386e965@xen.org>
Message-ID: <alpine.DEB.2.21.2109231348200.17979@sstabellini-ThinkPad-T480s>
References: <20210923061429.16361-1-Hongda.Deng@arm.com> <0b0ede18-b944-8693-dede-616c3386e965@xen.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Julien Grall wrote:
> Hi,
> 
> On 23/09/2021 11:14, Hongda Deng wrote:
> > Currently, Xen will return IO unhandled when guests access GICD ICPENRn
> > registers. This will raise a data abort inside guest. For Linux Guest,
> > these virtual registers will not be accessed. But for Zephyr, in its
> > GIC initilization code, these virtual registers will be accessed. And
> > zephyr guest will get an IO dataabort in initilization stage and enter
> 
> s/dataabort/data abort/
> s/initilization/initialization/
> 
> > fatal error. Emulating ICPENDR is not easy with the existing vGIC, so
> > we currently ignore these virtual registers access and print a message
> > about whether they are already pending instead of returning unhandled.
> > More details can be found at [1].
> > 
> > [1] https://lists.xenproject.org/archives/html/xen-devel/2021-09/
> > msg00744.html
> > 
> > Signed-off-by: Hongda Deng <hongda.deng@arm.com>
> > ---
> >   xen/arch/arm/vgic-v2.c     | 10 +++++++---
> >   xen/arch/arm/vgic-v3.c     | 29 +++++++++++++++++------------
> >   xen/arch/arm/vgic.c        | 37 +++++++++++++++++++++++++++++++++++++
> >   xen/include/asm-arm/vgic.h |  2 ++
> >   4 files changed, 63 insertions(+), 15 deletions(-)
> > 
> > diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> > index b2da886adc..644c62757c 100644
> > --- a/xen/arch/arm/vgic-v2.c
> > +++ b/xen/arch/arm/vgic-v2.c
> > @@ -481,10 +481,14 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v,
> > mmio_info_t *info,
> >         case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
> >           if ( dabt.size != DABT_WORD ) goto bad_width;
> > +        rank = vgic_rank_offset(v, 1, gicd_reg - GICD_ICPENDR, DABT_WORD);
> > +        if ( rank == NULL ) goto write_ignore;
> 
> 
> 
> > +
> >           printk(XENLOG_G_ERR
> > -               "%pv: vGICD: unhandled word write %#"PRIregister" to
> > ICPENDR%d\n",
> > -               v, r, gicd_reg - GICD_ICPENDR);
> > -        return 0;
> > +               "%pv: vGICD: unhandled word write %#"PRIregister" to
> > ICPENDR%d, and current pending state is: %s\n",
> > +               v, r, gicd_reg - GICD_ICPENDR,
> > +               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");
> 
> Each register contain the information for multiple pending interrupts. So it
> is a bit confusing to say whether the state is on/off. Instead, it would be
> better to state which interrupt is pending.
> 
> Also, I would rather avoid printing a message if there are no interrupts
> pending because there are no issues if this is happening.
> 
> > +        goto write_ignore_32;
> >         case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
> >           if ( dabt.size != DABT_WORD ) goto bad_width;
> > diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> > index cb5a70c42e..c94e33ff4f 100644
> > --- a/xen/arch/arm/vgic-v3.c
> > +++ b/xen/arch/arm/vgic-v3.c
> > @@ -817,10 +817,14 @@ static int __vgic_v3_distr_common_mmio_write(const
> > char *name, struct vcpu *v,
> >         case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
> >           if ( dabt.size != DABT_WORD ) goto bad_width;
> > +        rank = vgic_rank_offset(v, 1, reg - GICD_ICPENDR, DABT_WORD);
> > +        if ( rank == NULL ) goto write_ignore;
> > +
> >           printk(XENLOG_G_ERR
> > -               "%pv: %s: unhandled word write %#"PRIregister" to
> > ICPENDR%d\n",
> > -               v, name, r, reg - GICD_ICPENDR);
> > -        return 0;
> > +               "%pv: %s: unhandled word write %#"PRIregister" to ICPENDR%d,
> > and current pending state is: %s\n",
> > +               v, name, r, reg - GICD_ICPENDR,
> > +               vgic_get_irqs_pending(v, r, rank->index) ? "on" : "off");
> > +        goto write_ignore_32;
> >         case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
> >           if ( dabt.size != DABT_WORD ) goto bad_width;
> > @@ -978,19 +982,20 @@ static int vgic_v3_rdistr_sgi_mmio_write(struct vcpu
> > *v, mmio_info_t *info,
> >       case VREG32(GICR_ICFGR1):
> >       case VRANGE32(GICR_IPRIORITYR0, GICR_IPRIORITYR7):
> >       case VREG32(GICR_ISPENDR0):
> > -         /*
> > -          * Above registers offset are common with GICD.
> > -          * So handle common with GICD handling
> > -          */
> > +        /*
> > +         * Above registers offset are common with GICD.
> > +         * So handle common with GICD handling
> > +         */
> >           return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
> >                                                    info, gicr_reg, r);
> >         case VREG32(GICR_ICPENDR0):
> > -        if ( dabt.size != DABT_WORD ) goto bad_width;
> > -        printk(XENLOG_G_ERR
> > -               "%pv: vGICR: SGI: unhandled word write %#"PRIregister" to
> > ICPENDR0\n",
> > -               v, r);
> > -        return 0;
> > +        /*
> > +         * Above registers offset are common with GICD.
> > +         * So handle common with GICD handling
> > +         */
> > +        return __vgic_v3_distr_common_mmio_write("vGICR: SGI", v,
> > +                                                 info, gicr_reg, r);
> >         case VREG32(GICR_IGRPMODR0):
> >           /* We do not implement security extensions for guests, write
> > ignore */
> > diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> > index 8f9400a519..29a1aa5056 100644
> > --- a/xen/arch/arm/vgic.c
> > +++ b/xen/arch/arm/vgic.c
> > @@ -470,6 +470,43 @@ void vgic_set_irqs_pending(struct vcpu *v, uint32_t r,
> > unsigned int rank)
> >       }
> >   }
> >   +bool vgic_get_irqs_pending(struct vcpu *v, uint32_t r, unsigned int rank)
> > +{
> > +    const unsigned long mask = r;
> > +    unsigned int i;
> > +    /* The first rank is always per-vCPU */
> > +    bool private = rank == 0;
> > +    bool is_pending = false;
> > +
> > +    /* LPIs status will never be retrieved via this function */
> > +    ASSERT(!is_lpi(32 * rank + 31));
> > +
> > +    for_each_set_bit( i, &mask, 32 )
> > +    {
> > +        unsigned int irq = i + 32 * rank;
> > +
> > +        if ( !private )
> 
> It is not clear to me why you not handling PPIs/SGIs and ...
> 
> > +        {
> > +            struct pending_irq *p = spi_to_pending(v->domain, irq);
> > +
> > +            if ( p->desc != NULL )
> 
> ... emulated SPIs (e.g. PL011).
> 
> > +            {
> > +                unsigned long flags;
> > +
> > +                spin_lock_irqsave(&p->desc->lock, flags);
> > +                is_pending = gic_read_pending_state(p->desc);
> > +                spin_unlock_irqrestore(&p->desc->lock, flags);
> 
> What you are reading here is the pending state from the HW. This is not the
> same as the pending state from the VM PoV. In fact, in the most common case,
> the interrupt will be pending from the VM PoV, but simply active from the HW
> PoV (it is deactivated once the interrupt has been handled by the guest).
> 
> I think what you want to check is whether the flag GIC_IRQ_GUEST_QUEUED is set
> in p->status (Stefano ?).
 
Yeah, that's right. In fact, there is no need for checking the hardware
registers. You can just go through the inflight_irqs list and print all
of them (the list is sync on hyp entry on the cpu you are running on,
not the others of course).


> This is technically still a bit racy as Xen may still think the interrupt is
> pending while the it may be actually active in the guest. AFAIK, the other way
> around (i.e. not pending in Xen but pending in the guest) cannot happen.
> 
> Anyway, this is just a message, so it is still better than crashing :).

+1


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:55:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:55:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194579.346650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVkZ-0004wn-P1; Thu, 23 Sep 2021 20:55:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194579.346650; Thu, 23 Sep 2021 20:55:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVkZ-0004wg-Lg; Thu, 23 Sep 2021 20:55:23 +0000
Received: by outflank-mailman (input) for mailman id 194579;
 Thu, 23 Sep 2021 20:55:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTVkY-0004wO-EY; Thu, 23 Sep 2021 20:55:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTVkY-0001Ys-8D; Thu, 23 Sep 2021 20:55:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTVkX-0000dP-Tj; Thu, 23 Sep 2021 20:55:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTVkX-0005D6-T5; Thu, 23 Sep 2021 20:55:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=16pJlZh/Q0P9rBQQ+Zqj0XgqB6Gi1/Zfx1hJQ4n1I0s=; b=6+aVKtCmphi2jbX515enPnjqtV
	UwXj8Si9Ac8jNKpJbKjqTDSv7T/2fXE4QwDyO/qr6zyvY8TpanMYZREAtiOgkfhmjDVA3I/qpQUB7
	rUXkKRdMB25FyHGjmisRdMMrc21e/ewIIdLyeF18KvLyYWcIuW0kLK/22ZZda5hH/NOc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165165-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165165: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start:fail:heisenbug
    qemu-mainline:test-amd64-i386-xl-qemuu-ovmf-amd64:guest-start/debianhvm.repeat:fail:heisenbug
    qemu-mainline:test-arm64-arm64-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=2c3e83f92d93fbab071b8a96b8ab769b01902475
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 23 Sep 2021 20:55:21 +0000

flight 165165 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165165/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds     14 guest-start      fail in 165160 pass in 165165
 test-amd64-i386-xl-qemuu-ovmf-amd64 20 guest-start/debianhvm.repeat fail in 165160 pass in 165165
 test-arm64-arm64-xl-vhd      17 guest-start/debian.repeat  fail pass in 165160

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                2c3e83f92d93fbab071b8a96b8ab769b01902475
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   12 days
Failing since        164967  2021-09-13 13:06:52 Z   10 days   20 attempts
Testing same since   165160  2021-09-23 01:54:09 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4691 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:57:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:57:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194586.346664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVn1-0005zD-C3; Thu, 23 Sep 2021 20:57:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194586.346664; Thu, 23 Sep 2021 20:57:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVn1-0005z6-8v; Thu, 23 Sep 2021 20:57:55 +0000
Received: by outflank-mailman (input) for mailman id 194586;
 Thu, 23 Sep 2021 20:57:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ODo4=ON=epam.com=prvs=99007947d0=volodymyr_babchuk@srs-us1.protection.inumbo.net>)
 id 1mTVmz-0005z0-V3
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:57:54 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eaf1b7e2-1cb0-11ec-ba8c-12813bfff9fa;
 Thu, 23 Sep 2021 20:57:52 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NJ3W4P006666; 
 Thu, 23 Sep 2021 20:57:48 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b8tbnsj6m-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 23 Sep 2021 20:57:47 +0000
Received: from PAXPR03MB7594.eurprd03.prod.outlook.com (2603:10a6:102:201::14)
 by PAXPR03MB7700.eurprd03.prod.outlook.com (2603:10a6:102:206::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Thu, 23 Sep
 2021 20:57:46 +0000
Received: from PAXPR03MB7594.eurprd03.prod.outlook.com
 ([fe80::24db:d530:e190:518b]) by PAXPR03MB7594.eurprd03.prod.outlook.com
 ([fe80::24db:d530:e190:518b%5]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021
 20:57:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eaf1b7e2-1cb0-11ec-ba8c-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NXCymN6coiIwm/KNaWGODnzmA/OjGooOMRbOKRU2uzLnavdeBsPIvhsf+c8d7Lr6h9N2tAhRPGAmR8TvV8UCAC7vag4vPcBcyNWSEnv57pz3XfiN1ee7vdIEP64MltFYeXaObAxfSVaGXAyX51urXxB+tcU1O9Qpul7QsLvlpSI4zQ7UoM4YBx9FZhuU/e+Ydfx9nSaLvYEg1LcaQgNep3UaXUWbviVmJ7OpJraKCLgt13LZSIYvYydCuWWYiYZpSK1iPryeXJhRXWHfvJwfZHoIphUnoIGYnQHqSY+/hYg0zrmhwMkOsVEGQ2MasGx02wltdnsY8DlnBFio45WBhw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=33YY/lzY/yKZw2ugpJeA/6jj2dtPeDssOR+MbSG/TjU=;
 b=S0LhuYmqkuxEA1GpwI2uykHWAuIMFAE7coutXM3DMcdtVz7ra8hzxWXZ4odqVKOzCYE65zJY31v5USPKcTmYm7Ri5PTULbjMYvZbRHWi401GdpXr833Tcwpbph+wokbxQ5GtB5bGGwaH/UlDjSaSnq6e1PXaTJMrAp6k/UNWpD6sQUSkq+cULbjg2rljnDTu0W2yoWaNSHq6/KI4PD++VVUgIEKrpnitfK2LPhQXq7almm9FTH5U+xnJK+JtvJQYj58qwpzPg0YLr8VTMsSK1szk5P24NUslTW77A2Ggqi3jb65vAahQkdiEJcEeh+AVsUqNcMVo5TJG8ddwA3hYTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=33YY/lzY/yKZw2ugpJeA/6jj2dtPeDssOR+MbSG/TjU=;
 b=3PqFSxevHSLvgesYLrUra+/5k+VPaVVPLA+X+jGC/Wu4SnpL5Ds9mXzUM5Y4lybILgiay8C0V5NDovrDcoO/shvizxPAEPCXxK+g1bcphSxu0k9kGa5o0bM1BLyY5DD+QMEmLRlm0Nlms4jEx0vI5ecl4gpdjNqSlMZSUSwzwAhlP6sP5NG8YgKjWEfl4VcB2hXflV+TbtcHxuT+koHCnQZrwCPdfVjSMkrN3YLmgBp7onU/h1Ii1Cr/Hqsy4gHsQBZy7Cuv6k747//f+LVi1V/sXTJqTY2WJ+8fkp3CVM0u2lL5o5neL75DZ2IXuN5gALerkI2jkCoDkWe2PhT/+A==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Oleksandr Tyshchenko <olekstysh@gmail.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Julien Grall <julien@xen.org>, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
Thread-Topic: [PATCH] xen/arm: optee: Allocate anonymous domheap pages
Thread-Index: AQHXoyUNSkY32SPluU6N5DMbFmm656uyJ0MAgAAJFoA=
Date: Thu, 23 Sep 2021 20:57:45 +0000
Message-ID: <87bl4jrovq.fsf@epam.com>
References: <1630935741-17088-1-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109231308200.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231308200.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: mu4e 1.6.5; emacs 27.2
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: fa924a4c-bb33-4751-8487-08d97ed4cb6e
x-ms-traffictypediagnostic: PAXPR03MB7700:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <PAXPR03MB77001B3D3646241774583DECE6A39@PAXPR03MB7700.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 DJGYX3Q/kpgUqRN7/8AFVdrtHbGNq6BnItyo5NNwtsmsRuYdnSeYCQugSTCfkjJ8dNw1m9pWQeptlCfvCNM0tXCXDg/7GRYBVShph8lbWT77vrtPB7ShfGN80vlYFSFWj6ZVNET5pLNXJ1caav3keELDcNtJNuladw+GHEJpkCispOSMHmmT8Htcg6fs6oIipmihYJscoc3n4TKKv6FjKkLLejOq8E6WFzG4YUeq9d9WySbH1bkh/7AQ1hUKNxHcB1OIl2hjy0PWozVVEN7nPHUNvOULb+9LDNcbIV/FN+39T/9LnhmY3A9ZpoyQpIPgnbkqDmtMO0bUQT4xIQHncvDPmvi/yld+6aK+It8khHXJqFfAAvOazzf+4BMFwT06IBaxP4yQOg45B7t0s28VvNGt/GNLRftdZ+1ZRdNdupHFv18ATCenYLLIbNf2kTjR4J0sILWLH/AXg8uF3x+I95+ET6pimQX4Izem6nrALbmzBZ5V/pSATE3GnaxVqZBy9WjwFrQzbqZYcPe2SDNgldarRw5L4At6t9qaunJPJNu1Naid0FePElsbwc0/pRqJDWnRUtKP6NBpfNV9U/B354IFrqfszeZSgBZTuzsKVdNA96gau1iVRLLM7LdBlGieutlEz5eypGjN/MvoAQxGF4ZcFOlfKjRmxOa6SsfiS9atrlVR3HKf/qIm9K2zf6hVz7FK27e+PZ11K6OhdDIkdw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR03MB7594.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(6916009)(508600001)(64756008)(4326008)(83380400001)(6506007)(91956017)(122000001)(38100700002)(8936002)(38070700005)(76116006)(2616005)(6486002)(8676002)(5660300002)(71200400001)(55236004)(66556008)(66476007)(66946007)(6512007)(66446008)(54906003)(186003)(86362001)(26005)(36756003)(2906002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?QVaiunLvcgeKju6E0TXHrF0eUsMoCO7X6Cht+4hHjxRo5cDnFdEJgdABxQ?=
 =?iso-8859-1?Q?0lHTIfYbaRvqEB31E6Vk2xcozRFZl5vVQZnc3MCeOpZISOSGdn1mq2SPZf?=
 =?iso-8859-1?Q?Sd+2I2zrwJj1wrqLPZ77USRm3QnxDQoH/fAK5VDHWyyTRfctB6Zy+6eepz?=
 =?iso-8859-1?Q?BLjICww4QZHaYk4zg/TbY7BYXCc/7k6BVVseNLgMtSfxpZTKXMTrLUT6uj?=
 =?iso-8859-1?Q?1nNgTsNyO+SD9fVaz0OkxQ2FTEtAQvrfi9aZ5cbPDmZL6ZoeQp/TDjkb6I?=
 =?iso-8859-1?Q?AvbD6hfVyDmpns1c2AkdULL4im1X1bW4tRCajYAzb4OOmJLFkWWUcbKVv6?=
 =?iso-8859-1?Q?+U/zPLz/S5ODcfn/blfsfOdFK/bXqh2Q4oxC2IHw6ETd4UszERD1D9e8KL?=
 =?iso-8859-1?Q?dTWBWwNNHPaywjZD/mlQyoCFNjWJK0Q4IrCjtlsjavWn2Ebc5wmclq5S8N?=
 =?iso-8859-1?Q?K1gHQUJ8hdQJJAEVjPBiiaHmRrY38Kqh3Vl3I4a0BLBSmW27HGcc/wiaC6?=
 =?iso-8859-1?Q?tqvH8+ihxHspr8OYcCZ+spQtakw15qEYTc5b6NAjSI+Tveoc1TYu22uX/i?=
 =?iso-8859-1?Q?OEmzW2Vd2/juJaFMSUATMAz/ftWoBUbhIWrmBKbJbfinL35rTm4slGxzjF?=
 =?iso-8859-1?Q?aU3nZ33P4imujJVB2rugfhIqltKelNSQsmBgSQJUq1vJhoqt5On/IGuS62?=
 =?iso-8859-1?Q?FrXeMWVzIejOJOKbPbeTxdb3tHr1k62imgkWVC8JByoU5gwnpJPLFwwg0n?=
 =?iso-8859-1?Q?k1qpbHloPrknDMl+Jd43svkwra61KW1pdZcXbTWvXLuczwplh3AkuUNVWK?=
 =?iso-8859-1?Q?MRL1zMqnTftxBmBjuc5AAfOOaCGPEQdbQcnA7mzUJ+7fD8s8N2vVF6Cwto?=
 =?iso-8859-1?Q?qxcsYfae+3R/58VDTfa8HREhRh8j5tknann9LJhka0G760ejP0+ades0Fb?=
 =?iso-8859-1?Q?3yA9Od3ivuVmZ+py2DFS0v3rFR6Gr+cM8J/v5GaWaVFwhPejpKpKoO5EZu?=
 =?iso-8859-1?Q?NPBKyIauc1v9VYizSBFpCtJ9oIYnuyyeXfuOB4UUpPqqAz5a5Hvnfx2Qzh?=
 =?iso-8859-1?Q?fFCt7wZo28IeIoBgR1o6ZADDiGrHJJ/rwVcsdWuVzA4dftXyUHoWjsRfWJ?=
 =?iso-8859-1?Q?8FjANnKBKDn1uT0pxm6/O90QTITlPSrBUWJJmFC0PvuhwPgrr9ejNtkySh?=
 =?iso-8859-1?Q?wpGaojl0cnzvpk/ZNKj5YXu74QkWyybvXagtjnaylFIneGAOAW5HEoBI94?=
 =?iso-8859-1?Q?ByI6NEnZk6683yCoMVwtOqOSggDqrg32QTLqWCKXWHyOksrGtU//Cc2YNP?=
 =?iso-8859-1?Q?Hrilho9orcgEX8vAcW+YWcvbPNKSbb4vtJD4njHJbFIuYV5v7stbJkTOn8?=
 =?iso-8859-1?Q?EIG9nWcPj7?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PAXPR03MB7594.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fa924a4c-bb33-4751-8487-08d97ed4cb6e
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 20:57:45.9665
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3zMsnTJ+6+cQvI9a3Vai/cmghYST7+PQBKIK6VQqnafKZN6hHGnaFUb2I3FWEdmgj5AWmvi8Iy/W72EQqFVaXntkgNU4LGEphnfjHV2RMq8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7700
X-Proofpoint-GUID: 7fb6tsuwvivTm5ggAeiswI-hPLjbrlFn
X-Proofpoint-ORIG-GUID: 7fb6tsuwvivTm5ggAeiswI-hPLjbrlFn
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-23_06,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=999
 impostorscore=0 suspectscore=0 priorityscore=1501 spamscore=0 adultscore=0
 clxscore=1011 mlxscore=0 phishscore=0 lowpriorityscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109200000
 definitions=main-2109230120


Hi Stefano,

Stefano Stabellini <sstabellini@kernel.org> writes:

> On Mon, 6 Sep 2021, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>=20
>> Allocate anonymous domheap pages as there is no strict need to
>> account them to a particular domain.
>>=20
>> Since XSA-383 "xen/arm: Restrict the amount of memory that dom0less
>> domU and dom0 can allocate" the dom0 cannot allocate memory outside
>> of the pre-allocated region. This means if we try to allocate
>> non-anonymous page to be accounted to dom0 we will get an
>> over-allocation issue when assigning that page to the domain.
>> The anonymous page, in turn, is not assigned to any domain.
>>=20
>> CC: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>
> Only one question, which is more architectural: given that these pages
> are "unlimited", could the guest exploit the interface somehow to force
> Xen to allocate an very high number of anonymous pages?
>
> E.g. could a domain call OPTEE_SMC_RPC_FUNC_ALLOC in a loop to force Xen
> to exaust all memory pages?

Generally, OP-TEE mediator tracks all resources allocated and imposes
limits on them.

OPTEE_SMC_RPC_FUNC_ALLOC case is a bit different, because it is issued
not by domain, but by OP-TEE itself. As OP-TEE is more trusted piece of
system we allow it to request as many buffers as it wants. Also, we know
that OP-TEE asks only for one such buffer per every standard call. And
number of simultaneous calls is limited by number of OP-TEE threads,
which is quite low: typically only two.

--=20
Volodymyr Babchuk at EPAM=


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 20:59:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 20:59:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194591.346676 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVoi-0006bz-Nb; Thu, 23 Sep 2021 20:59:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194591.346676; Thu, 23 Sep 2021 20:59:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTVoi-0006bs-Ke; Thu, 23 Sep 2021 20:59:40 +0000
Received: by outflank-mailman (input) for mailman id 194591;
 Thu, 23 Sep 2021 20:59:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TVYu=ON=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mTVoh-0006bm-29
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 20:59:39 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d683472e-d474-4231-aeb0-6911f734e160;
 Thu, 23 Sep 2021 20:59:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d683472e-d474-4231-aeb0-6911f734e160
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632430777;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=16Ef2jl80sOM5zhrzx/z6n+coMQrh2Uz54KD8xBEkPY=;
  b=eDOUFxyF/0sNTSnpkgxI+vhjd0deYf5doGwiDE1NMQABoavNSk0QSWoa
   R1Gt7ndGyQKitIWkcy2NqQQog6BiS5OJeOONOZmpXx/2NwF71ZJstqIuN
   w/qCWZqgcNCscShhxI6FmsuD/KRvlhSVyeP2H3pkE7DkwnwdinQP0ZhVL
   g=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: kfEWHmL5/7mwAxt12gGwBwI3BWL3ne4EBCOHBmVHObFZ5poTg+MdfrY6Y+uSbThsPVcLXJxGAd
 6gJezyA5+3rwlvg590IXI6JiDUdw5hBLX78x9hvcu8cgYg8cjU4uuGHzlKxDLMmuP+pEn/mAl7
 AZiQ0P2/hq41913btySXFwvcGHFoUMUcSBM2wrRrKuomSDx5q40PlFg2JnMQ303aZ0lw0wPlv4
 Pi9smCM4UWeFOfoAPUEPSDDUJbOFooY91TUNAkjX9w5iKqtlQoSHXyG4z43BryPea6DjW6fEra
 08vKLa7QyaUbMSr41wKbLHsY
X-SBRS: 5.1
X-MesageID: 53083837
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:SnN8k64KSKgLovPOyZyUPgxRtFHBchMFZxGqfqrLsTDasY5as4F+v
 jQaX27UOKzea2fwedskO42/80gE6sOGmIVqSwNorCo2Hi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2NUw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z5
 8hDtp2CRw0TYYLJhelFWh10A3xZBPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALBc/nJo4A/FpnyinUF60OSpHfWaTao9Rf2V/cg+gTRq6OP
 JpCNlKDajzpRSEIFAwxBKsau8z4qV/1LQZY+FG88P9fD2/7k1UqjemF3MDuUtuOSMZOl0CUv
 FXa7n/5CREXMt+Y4Ted+3fqjejK9QvgQ54bHrC88v9sgXWQy3YVBRlQUkG0ydG+jkO9QM5OM
 Ew89S8nrKx0/0uuJvHtUhv9rHOasxo0X9tLD/Z8+AyL0rDT4QuSGi4DVDEpQN47sM47QxQ62
 1nPmMnmbRRturCITXOW9p+PsCi/fyMSKAcqeissXQYDpd75r+kbsBXLSdpyFb+vuff8Ezrw3
 jOioTA3gvMYistj/6Sk/3jXjjS0vJ/LQwUpoALNUQqYAhhRPdD/IdbysB6CsKgGfN3xokS9U
 GYstNWQ7+E8A5u3ywOIXehVFbWF96uiCWiJ6bJwJKUJ+zOo8n+lWIlf5jBiOUtkWvo5lS/Vj
 Fz74lwKucUNVJe+ReouOdvgVphzpUT1PYm9Dpjpgsxyjo+dneNt1BpnY1KZl0vpmVIl+U3UE
 cbGKZv0ZZr25KIO8dZXewv/+eNwrszd7TmKLXwe8/hB+eDFDJJyYe1ZWGZilshjsMu5TPz9q
 r6zzfeixRRFS/HZaSLK64MVJl1iBSFlXsqt8JQPLbDZc1sO9IQd5xn5m+hJl2tNxfg9qws11
 ivlBh8wJKTX3xUr1jlmmlg8MeiyDP6TXFowPDA2PEbA5pTQSdzH0UvrTLNuJeNP3LU6lZZcF
 qBZE+3dUqUnYmmWoFw1MMiixLGOgTz23GpiyQL+O2NhF3OhLiSUkuLZkvzHrnJTUXbn6ZVk8
 9VNFGrzGPI+euirN+6PANqHxFKtp3kN3uV0WkrDON5If0vwtoNtLkTMYjUfeqng8D3PmWmX0
 Ri4GxAdqbWfqoM56oCR16uFs52oA611GU8DRzvX6rO/NC/7+Gu/wNAfDLbULG6FDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6AUnu1ZtAK6V
 1ih4N5fPbnVasrpHERIfFgub/iZ1OFSkT7XtKxnLEL/7S5x3byGTUQNYEXc1H0DdON4adp3z
 /0gtcga7x2EpiAratvW3DpJ82msL2AbV/l1vJ8tH4K22BEgzUtPYMKAB3auso2PcdhFLmIjP
 iSQ2PjZn71Zy0fPLygzGHzK0bYPjJgCokkXnloLJlDPkdvZnP4nmhZW9G1vHAhSyxxG1cN1O
 3RqaBIpdfnfoW8wiZgRRX2oFiFAGAadqx74xFY+nWHES1WlCz7WJ2onNOfRpE0U/gqwpNSAE
 G10HIo9bQvXQQ==
IronPort-HdrOrdr: A9a23:BMWM6KwUqstIo8Q4VABeKrPxu+skLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjkfZquz+8L3WB3B8bfYOCGghrUEGgG1+XfKlLbalXDH4JmpM
 Bdmu1FeafN5DtB/LbHCWuDYq8dKbC8mcjC74eurAYfcegpUdAF0+4QMHfrLqQcfnghOXNWLu
 v/2iMKnUvaRZxBBf7LeEXtEtKz6+HjpdbDW1orFhQn4A6BgXeB76P7KQGR2lM7XylUybkv3G
 DZm0ihj5/T/c2T+1v57Sv+/p5WkNzuxp9qA9GNsNEcLnHJhhyzbIpsdrWetHQeof2p6nwtjN
 7Qyi1QcPhb2jf0RCWYsBHt0w7v3HIH7GLj80aRhT/ZrcnwVFsBeoF8rLMcViGcx1srvdl63q
 4O9XmerYBrARTJmzm4z8TUVjlx/3DE4kYKoKo2tThyQIEeYLheocg050VOCqoNGyr89cQODP
 RuNsfB//xbGGnqL0wxhlMfheBEY05DWitvGiM5y4uoOnlt7TFEJnIjtY4idixqzuN6d3FGj9
 60epiA2os+F/P/wMpGdZA8qPCMexnwqCT3QSuvyGTcZdM60k322urKCZUOlauXkc8zvdYPcK
 qoaiIviYd1QTO3NfGz
X-IronPort-AV: E=Sophos;i="5.85,317,1624334400"; 
   d="scan'208";a="53083837"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fxQinuiTKGoB4ZGWEWVzXfuQ5XrtakgHAWZiJvox3JZ9/PCRj4tcV5haall2nheFhkqxFS8oyEDCOMAer5pPH7Qp4jrXHqwZ/QwP5z+miE61mmXC7axzViTCm1Es52lBX+KUoDpjuKFKt9V865tmu4TMz6jlQ5BDNkbhHE7oJwN23kDYqUMATxwRyCq4unIqmM0bjt+mc6YVAR0DgnlnH+iUTzSlWeg/d3YuXNifCvJP5GaCyQ94nZfr/BA5hQzmHCemIhqzDD+FOsDiZuotxzLYh//olThRTw2tJUziqEmWnqovu1bUNfaeUvTQWxBnfpSMXf7paByN9OGcJf37SA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=16Ef2jl80sOM5zhrzx/z6n+coMQrh2Uz54KD8xBEkPY=;
 b=eLuzP/VefyjfW4p0g1AG8pCtBZOJxkqLLxgCQKKKNvDtLcv15qR9w5fr9zV5PaWyh/oAwS3+NSi9RcMWhSNnj0E+pFEy8RSLsj8BrrXKmpIVg3de6nEXXUI+UrdXdV0FBLrnUQ7hAgdi77HyNjv3a2sETOnVnkfURcQfJngfNqh1Hcyyb1ztdhGKRq81G7Yo9081+qt9z5M9FFGhteI0lOHvbDga+32dBOCwNgkiGTmrZVxRBQ0sZdO9zFiIqWpBiuHi1Pgl2exs1Wg7uyKU7ksasRBWPH2LHFVCPa2j+WONf/3uou5OlPpHnnzvJ0PMfLkd0OE7SIqFSKuRkZmzww==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=16Ef2jl80sOM5zhrzx/z6n+coMQrh2Uz54KD8xBEkPY=;
 b=cGTe1l1O3IyG9BsMsaajuQTeIE6rnMLicYROQQqLKRlXarISXjhCRYj+EVf+TDPUrX5LuwYTdNl4k+mvNE4KY5t3JNaPwh4LTl1Am0XSBtbHCLwjCn9j6vWnTaki9cevuLV+4gH3+lDEXKLmUX4qOcx4urrjIeVZlRroLXxa5m8=
To: Oleksandr Tyshchenko <olekstysh@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in struct
 page_info on Arm
Message-ID: <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
Date: Thu, 23 Sep 2021 21:59:26 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO3P123CA0005.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:ba::10) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5f4b9f1f-b6ed-4884-020c-08d97ed50bd8
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5567:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5567D2BD200959963F1FE1B6BAA39@SJ0PR03MB5567.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: K5I1Bky2EQmnvG8ke4V7pTPMTEkxCW6DYQhnmBJV9BM9lGoNDGMPeWjKGKgPc22JADwqybJa2xSY4O3sLXs+diyF+Y17EZAAvG3/xC7gZPFYa059PYdYlSVTn0ZZhmPnaOyXDehyFKRBb+SwkcdbEOhdA/0bqr+UluW8y8qT+TXm/h0cisH+iqunu158QVe6fO7VCZlFU3Blc5AlwQrijLcIXq45/UpOFATPtwrLEiP0s7pwC/PPvRt8e0aA7xAT4vwJtF+t9cxu1cWTBN8EK0ix2Mrkf7A6uFlZz9f2t8pZP4GnVGZ3jDwG8Q3pOlYd08E0fRd9qBsHWo9rGKU8bPovuvYZ+4DuYtV4Lm5fCVFrA//UOJQ4Y9eBSO3OtQ+xEKye8lPDpZRHXpXKP2Y2j4LH81AWRKXnuv27NpuQwlaZcSnKy9dX2L3S9TJwchHHflsQZ6lHIbsiCLarM+jy4wjQMtqmrkzMchV1KBTy0jpPRub4Q8bVuEIEyNgvM5bHRNYvcBQ6SuyQIST8ifFEUyI3ckedo8HJRsyMfxoxdHjXE+uAsiwnvJMyYU9QmFihPxUYFhS2x0R9mC3S66h8zZOeQx1s1kFdRT4H+S+jV9EwBqs7Ot70wfZEfLVl9Kbtgg3orODEmHwpXTtEKMbyejYNDFIZGpUpKZtyIZEtGwMHKpGZXO7C2r8PqDFk3XRTvsfv1uy82he+11plUZ14MklvNirD/9jyKKPuW8R+JdsEFRMYGX8uKqUk45p55D/L7oAz+Fl7antyt/A+uX1R9Wb+hw59hGBLzjcEjU1erH4Gexu3LU0/ncZSxv9+LvGHh6cYODM0rmECHPE36+PrM85xzHI6NtdOutIANkc2bGw=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31696002)(16576012)(54906003)(38100700002)(956004)(31686004)(508600001)(316002)(2616005)(26005)(966005)(2906002)(8936002)(5660300002)(83380400001)(55236004)(186003)(86362001)(66556008)(6486002)(4326008)(107886003)(36756003)(53546011)(8676002)(66476007)(6666004)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V1dOd0NsR0FUYzFVVmhtU2MyU0gzY0ZWMURFL1h6cVU4cUdLUG5BcEVhaS92?=
 =?utf-8?B?SUI4WVozcllIL0NZMGorS1lMN2dkYjJwNWdRL2FYd2R4cHJWVnNGdUNCeC9L?=
 =?utf-8?B?Wk9SLzZwS1RhaWVZNEtoR0NwM2VLdGt2SDlVTmpCU1NrN2loSjkvMWswQTBR?=
 =?utf-8?B?M3ZpWXE1Y2lZUUxyclRVUzBQdElzU2FRRjRiQXFJK1JiVExGSDhHOFlabHE1?=
 =?utf-8?B?WFR5SVBjcENmcEtPVWY0UTJYbDVMcWFZcFQyM2pudkF2a0IyWHJhRUc3cTQ5?=
 =?utf-8?B?YSs1bVNmclRsWk5odmdEVUZXQ3NyR3RKRHFpSmMxVFRtUFRvNUUxaTMySDds?=
 =?utf-8?B?bU03Y2VEM0VwQytSalRXbUtkTFBtYVhPWVVIRkdCQmRoUGZIelNxOUhZU0JX?=
 =?utf-8?B?M2NyNVZuYWp1cFAzNkVZNlFLQWoxRlE1Q0pOOGF6eUFUTllnK1NnM2FvRHBw?=
 =?utf-8?B?ZFduMEJYZXBFbHA0TnVNYXNQbFI2UHAyOFpZRkYvc2t3amw4YW1NQm4xZFRq?=
 =?utf-8?B?RDNsS09Qcm5xU3ZJbjhhQitHb2FzdTNQbFpyazkwa1FGRkJvMnhHQjNEaXhE?=
 =?utf-8?B?cTUveDBYREJHbmllUUUwOWxEeEV4Nm0xMTRPOWh0K3BwQU1EL1ZTSnNCdndx?=
 =?utf-8?B?cEV1V2x2enNMalpQT2NMVmM4d0RGcE50VzJRRUhickhkTmc4dWZvc3RaVnVw?=
 =?utf-8?B?L3IzdkhqNGZRNnVSdEdhN1h2U0xKZTNIQ1pYZzBoR3ZvUmFSMmt0aGlwNVdZ?=
 =?utf-8?B?cTBuNUxoNk5xOUFzSk9SWFJXdTlQOHhZRCtYWW1ZcWo1cG5hbm9VZnkzZEl6?=
 =?utf-8?B?d2E0TzI5STl6UUhCZnp1WHA2YzV0eEkvTjVMbjIwMmsxYU1HWmEwR2FSdStO?=
 =?utf-8?B?Y25seTZ4WVVRanAxc3NmczI5MUpBbm1XVGRTLzhDUlVDS2ZmTHJZU2pTeDFz?=
 =?utf-8?B?R3VZTitONzFHN3IrZWZkeXk4UWt6N2I1aXVJd0hmcWVENmEyNjRLTGxSQjgv?=
 =?utf-8?B?MmE1cGhyd3R6U21qSmRwMU9vRUhSbjYzUUoyL0Jtd0xhUkY2SHliWVpoV0p1?=
 =?utf-8?B?M0t3VWRzTGgrSHdJakxZaXRFMTcvaVk4RDZWZXR6cWFZY2Z1akU1RW1uMjJK?=
 =?utf-8?B?ejRBU1h0M0hiR3dvWlIvQ0RsOVN2UFNHMVRuRkExR3RhTlhHdlhoNEplWUdY?=
 =?utf-8?B?NFl6QnR5Y0RIeUN5WlR6UWN4Q0ZvM3lOUFFJSkw4Y1JmV01CWkxXdnViditW?=
 =?utf-8?B?LzdkTUwzeU1ld29WL3lkTXRtd1pDZmpPNVRCYWZyR0k1dEpmZVpGeFkwdzh6?=
 =?utf-8?B?Zndnd0hidEgySjZkUUhpMTBUQm12TUQxdHBMWjNtaktUZjQzRFM3bGpDcTQ2?=
 =?utf-8?B?VFdpSmowOVFWbjBqbXhtYUFmN3MwbU9PdFV3Qkpkc2hWVlpNMFhteVA1NWtU?=
 =?utf-8?B?OUZZU3VXc2dKSE8yZ2g0U0VVL2Z3RDhoYUUvUytJdzcwbFpUbm1PbiszbGlx?=
 =?utf-8?B?NFJlR1Z2Mk55Qm1YVENjdE1YNHFxWEhxeko4amozM3VhQUNTek1XN0RDaWY0?=
 =?utf-8?B?bFFqRkVWcVBQbU1ObmRaZmE0U1VBQUpBUjV3cHZMc2lncmNlYTkvWFVoaktj?=
 =?utf-8?B?MXhDY0libUZzSHk0eGhvWHZ2UVNyOTdLbGxjV09YNEJ2ZGlraVJUOUZ5K2lz?=
 =?utf-8?B?VUh6RjFzZy9WWnpoTUdiaXRpdFpHZDhPcStPa0I4VjA4TnJUUXR6dlVETFJU?=
 =?utf-8?Q?GphddSmDYU8MmJKfHNYMXYYxzOwBetOJ5cDYe7S?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5f4b9f1f-b6ed-4884-020c-08d97ed50bd8
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 20:59:34.2814
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 45yjeUFkk8nTPP1WCQSnhO3jFCr6sF/9IfDUt0rXAanO8vASTcCKu8GVaIjoEzsM129CypUQQHPntzj0w7yxDxYSdDr/jsyGngp6VAz9v7w=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5567
X-OriginatorOrg: citrix.com

On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> You can find the related discussions at:
> https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xe=
n.org/
> https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-oleks=
tysh@gmail.com/
> https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-oleks=
tysh@gmail.com/
>
> ! Please note, there is still unresolved locking question here for which
> I failed to find a suitable solution. So, it is still an RFC !

Just FYI, I thought I'd share some of the plans for ABI v2.=C2=A0 Obviously
these plans are future work and don't solve the current problem.

Guests mapping Xen pages is backwards.=C2=A0 There are reasons why it was
used for x86 PV guests, but the entire interface should have been design
differently for x86 HVM.

In particular, Xen should be mapping guest RAM, rather than the guest
manipulating the 2nd stage tables to map Xen RAM.=C2=A0 Amongst other thing=
s,
its far far lower overhead.


A much better design is one where the grant table looks like an MMIO
device.=C2=A0 The domain builder decides the ABI (v1 vs v2 - none of this
dynamic switch at runtime nonsense), and picks a block of guest physical
addresses, which are registered with Xen.=C2=A0 This forms the grant table,
status table (v2 only), and holes to map into.=C2=A0 Details can be conveye=
d
via ACPI table or DT, as applicable

Xen maps the RAM (which is now accounted to the guest, an improvement
over today) forming the grant and status tables, and grant map/unmap
hypercalls simplify massively to just {src domid, gref, flags} =3D> slot,
which also solves the "is the same grant mapped elsewhere?" problem.

There is never a need for HVM guests to map the same grant twice (as it
controls the first stage tables and can create whatever alias it
desires), but there's also no need to allow the guest to pick where the
mapping occurs.=C2=A0 This vastly simplifies both the Xen and guest kernel
implementations.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 21:38:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 21:38:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194597.346686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTWQK-0003PV-LF; Thu, 23 Sep 2021 21:38:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194597.346686; Thu, 23 Sep 2021 21:38:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTWQK-0003PO-I6; Thu, 23 Sep 2021 21:38:32 +0000
Received: by outflank-mailman (input) for mailman id 194597;
 Thu, 23 Sep 2021 21:38:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTWQJ-0003OX-7q
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 21:38:31 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 490fe06e-ce2c-4b0f-b9a9-9d8e847a960d;
 Thu, 23 Sep 2021 21:38:29 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id i25so32055073lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 14:38:29 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id h13sm559447lfl.205.2021.09.23.14.38.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 14:38:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 490fe06e-ce2c-4b0f-b9a9-9d8e847a960d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=bT1O7aoUpv2ZOCFLemyoGNNPhvAYsQhrb5a1N4i9tCo=;
        b=nn+jFQUwNcEybu/mxRKn4Rp94Vz54efjQq3YHit2A65BFFTeLulu2OuqnwxbINpwSr
         7KyOCzh7nHgFADRnvKy59gTQlP3emr3fZ0O/OCV/kDao4RLhoIL0zWSJzZADbHqQNszE
         V/gEGCcd8VpV/DFn6wBZQYorOfIjzvWIiNxoGbStP6MFClQGqIgfwlLyHWaJJabLF61S
         bn4aJxekge4Z0HGW2yLw0FWbdEzntxgdeDvZHhyuPwsTaEOor7TOy59tC1BD9K36ak4t
         tItbn12tiHCxVSBL8L9EuITAefsFvid7m61HwX4e85V/3fvRKQWOYH5/5p2g2Du2upZd
         qDRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=bT1O7aoUpv2ZOCFLemyoGNNPhvAYsQhrb5a1N4i9tCo=;
        b=E6MbU97zIPlp9L+OavLo5mYBd3Eat3wxUaSeQQy9dvXPlr+Ozfu07kK7KzrxMUFyxV
         fcbF6d4yjS7tR2+QleM6//GDxdKDrIeqjR1NTdA/RPehFMAGqPlYftJQgjdTwDf7bFw/
         xB4JIiwekvPt5Eu5BuuxrYQcRRLkBSdeJgGgv4qf91mmsrJMXNpsHdBuwzIt5x66/fQ4
         /iCLe6w1zRRHuoH+1+fY17kOxIDeHNeCEMa/gEodre3LBrqP6UYgidCpuheWPvcAati9
         7vP2WYYBJ3ns/AxJJVSe6hdjsF5WYToIfOQu3PSKGl5WboAGO3n/h+nLc08CR20VBpEA
         mDGQ==
X-Gm-Message-State: AOAM531XWAnSt6Uu0qvMDaYkXSJH1TvuhZYHGpjKZgmgYJQ4YgKbIUXk
	9juTYdrxu3S6SbWeCCkQ1yc=
X-Google-Smtp-Source: ABdhPJxxWGo1dhU0aXwhW8MRLzyH3KEkUV3tr8VWWTuAsiuhT3rQ/BjwjoGnr9f6Rlo5e4DIoBf2tg==
X-Received: by 2002:a05:6512:39cd:: with SMTP id k13mr6244469lfu.289.1632433108696;
        Thu, 23 Sep 2021 14:38:28 -0700 (PDT)
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in struct
 page_info on Arm
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
 <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <1f825e6f-e93f-9332-077d-3fabca718156@gmail.com>
Date: Fri, 24 Sep 2021 00:38:27 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 23.09.21 23:59, Andrew Cooper wrote:

Hi Andrew.

> On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> You can find the related discussions at:
>> https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
>> https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
>> https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/
>>
>> ! Please note, there is still unresolved locking question here for which
>> I failed to find a suitable solution. So, it is still an RFC !
> Just FYI, I thought I'd share some of the plans for ABI v2.  Obviously
> these plans are future work and don't solve the current problem.
>
> Guests mapping Xen pages is backwards.  There are reasons why it was
> used for x86 PV guests, but the entire interface should have been design
> differently for x86 HVM.
>
> In particular, Xen should be mapping guest RAM, rather than the guest
> manipulating the 2nd stage tables to map Xen RAM.  Amongst other things,
> its far far lower overhead.
>
>
> A much better design is one where the grant table looks like an MMIO
> device.  The domain builder decides the ABI (v1 vs v2 - none of this
> dynamic switch at runtime nonsense), and picks a block of guest physical
> addresses, which are registered with Xen.  This forms the grant table,
> status table (v2 only), and holes to map into.  Details can be conveyed
> via ACPI table or DT, as applicable
>
> Xen maps the RAM (which is now accounted to the guest, an improvement
> over today) forming the grant and status tables, and grant map/unmap
> hypercalls simplify massively to just {src domid, gref, flags} => slot,
> which also solves the "is the same grant mapped elsewhere?" problem.
>
> There is never a need for HVM guests to map the same grant twice (as it
> controls the first stage tables and can create whatever alias it
> desires), but there's also no need to allow the guest to pick where the
> mapping occurs.  This vastly simplifies both the Xen and guest kernel
> implementations.

That's interesting. Thank you for detailed explanation. I think, I got 
the high level idea.


>
> ~Andrew
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 22:49:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 22:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194610.346698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWk-0003m9-0l; Thu, 23 Sep 2021 22:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194610.346698; Thu, 23 Sep 2021 22:49:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWj-0003m2-U7; Thu, 23 Sep 2021 22:49:13 +0000
Received: by outflank-mailman (input) for mailman id 194610;
 Thu, 23 Sep 2021 22:49:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTXWi-0003lw-Gb
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 22:49:12 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ebc77fb9-83f0-482b-8db8-80accfeb9db4;
 Thu, 23 Sep 2021 22:49:11 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id u18so31251863lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 15:49:11 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id q5sm230091lfd.17.2021.09.23.15.49.08
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 23 Sep 2021 15:49:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebc77fb9-83f0-482b-8db8-80accfeb9db4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=f+iHNZX30F+c9l8Tgf0I+ajS/K8LFWnAgou/URb3Z4Y=;
        b=N+1Lufe5p7WVXW7HUUXmUURqqG23MVOCB7LCzIOqorOjwy9lHld9ZQs2pkMmQSKm2W
         k5ThThDnlm1+53cyvcd9Dt0WNNfO4kcMLXj/76gwyVYu7xFWOp3WdtKcKxaNdyukWuLj
         WTR8NekI2+4n7yctj/BVce/BvqD3Wxe/MajVSBUjM+7PkU4//DanoRTFvgmLScELGSQl
         EKRuCOq7fNHJ1EWj979JHeuAJvN8+IcV/lJoAccpSr/mFRG9i3wRmkBTcSwUfvPdKvwP
         s9fGUYtYKuX9rK/CwOEAYPpyo+su8cysQAvyFiYJZy5v+ILEw3E4ZLFDC065qjr4GSmv
         hbEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=f+iHNZX30F+c9l8Tgf0I+ajS/K8LFWnAgou/URb3Z4Y=;
        b=3auqtE2HQC/a1HtIJf2+K3UuuIV7bx+VjGg344KWiCmtcVsPHK1tAs2hgdPvjGxVVY
         4O3bS3MVE6Rp8NrxerA56rxDvglUc7n1atF6f5TRtzn78moZ0rgBDFUdFBOwUViL2JH6
         C8n5hfdujEPXllU26tjkVKtgBxF5O5NavcQrltcLDoMI7IMGOtvdSr6zaYFgxVQATmVC
         XQGrnoagVynFruXaB32DQijUJyylwwnyOuBFLU9Oo1IZ2xvB5drfgJmF2MNRr3idrgTo
         ghWsAkxBeUUgLmVCAO2z6BP+xAog1s946e/JAL34poXtp9x0/edFgKpdZCpt+ptPPjnb
         YZUw==
X-Gm-Message-State: AOAM530JkMdQYnCFYu4yQ5DT+zfefMbZaS6RBV6fd6zmXM9fpLoTEUuH
	oq7MaebNeI7YvH0qGxanH9pHYe3jYE4=
X-Google-Smtp-Source: ABdhPJygKD6RtIKZ2Y06/Ezq1izyho6x3eI+8nE5H7YHNvcqf13XKhchOtCUYGWwyt8Cc034TSUEjA==
X-Received: by 2002:ac2:5681:: with SMTP id 1mr6277604lfr.471.1632437349830;
        Thu, 23 Sep 2021 15:49:09 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>
Subject: [PATCH V3 0/3] Add handling of extended regions (safe ranges) on Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
Date: Fri, 24 Sep 2021 01:48:51 +0300
Message-Id: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

You can find an initial discussion at [1],[2] and [3].

The extended region (safe range) is a region of guest physical address space
which is unused and could be safely used to create grant/foreign mappings instead
of wasting real RAM pages from the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and advertised
to it via "reg" property under hypervisor node in the guest device-tree
(the indexes for extended regions are 1...N).
No device tree bindings update is needed, guest infers the presense of extended
regions from the number of regions in "reg" property.
New compatible/property will be needed (but only after this patch [4] or alternative
goes in) to indicate that "region 0 is safe to use". Until this patch is merged it is
not safe to use extended regions for the grant table space.

The extended regions are calculated differently for direct mapped
Dom0 (with and without IOMMU) and non-direct mapped DomUs.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain currently.
- The ACPI case is not covered.

Xen patch series is also available at [5]. The corresponding Linux patch series is at [6] for now
(last 4 patches).

Tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with updated virtio-disk backend [7]
running in Dom0 (256MB RAM) and DomD (2GB RAM). In both cases the backend pre-maps DomU memory
which is 3GB. All foreign memory gets mapped into extended regions (so the amount of RAM in
the backend domain is not reduced). No issues were observed.

[1] https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
[2] https://lore.kernel.org/xen-devel/1631034578-12598-1-git-send-email-olekstysh@gmail.com/
[3] https://lore.kernel.org/xen-devel/1631297924-8658-1-git-send-email-olekstysh@gmail.com/
[4] https://lore.kernel.org/xen-devel/1632425551-18910-1-git-send-email-olekstysh@gmail.com/
[5] https://github.com/otyshchenko1/xen/commits/map_opt_ml4
[6] https://github.com/otyshchenko1/linux/commits/map_opt_ml3
[7] https://github.com/otyshchenko1/virtio-disk/commits/map_opt_next

Oleksandr Tyshchenko (3):
  xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
  xen/arm: Add handling of extended regions for Dom0
  libxl/arm: Add handling of extended regions for DomU

 docs/misc/xen-command-line.pandoc |   7 +
 tools/include/libxl.h             |   7 +
 tools/libs/light/libxl.c          |   2 +
 tools/libs/light/libxl_arm.c      |  70 +++++++++-
 tools/libs/light/libxl_types.idl  |   2 +
 xen/arch/arm/domain_build.c       | 280 +++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/sysctl.c             |   2 +
 xen/arch/x86/sysctl.c             |   2 +
 xen/include/public/arch-arm.h     |   3 +
 xen/include/public/sysctl.h       |   4 +-
 10 files changed, 370 insertions(+), 9 deletions(-)

-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 22:49:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 22:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194612.346720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWu-0004Ml-Hz; Thu, 23 Sep 2021 22:49:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194612.346720; Thu, 23 Sep 2021 22:49:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWu-0004Mc-Da; Thu, 23 Sep 2021 22:49:24 +0000
Received: by outflank-mailman (input) for mailman id 194612;
 Thu, 23 Sep 2021 22:49:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTXWs-0003lw-A6
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 22:49:22 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 58cc834b-4318-4c72-8ca9-65c42e9cea80;
 Thu, 23 Sep 2021 22:49:12 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id u18so31252076lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 15:49:12 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id q5sm230091lfd.17.2021.09.23.15.49.10
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 23 Sep 2021 15:49:11 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58cc834b-4318-4c72-8ca9-65c42e9cea80
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=rDIaq2zIrPogpfXrurzqgzFbIZpi6DRuEYogRKmWMm0=;
        b=dLv7npLjP8uJTP2IyctwBKqjaIitWlDkbYgkYLkb1/Sc/+vF8HpovJ7nCgGZgK9hMC
         8E80aNUeUFnRcrwGoVcId2iZFCOPdcp68BYdkrYfnZ1XtRzCusS04518eaGb+VHErbfR
         CfHW9MyjFOh/NEaWEYHxMlc0KBcLu9VdXQBedMww97DPwKuQKvvAouG9yvL2492H9787
         yrcW6Ea7WDtCwafEk4f3Rrc8ALeMAbTeIrGrEOmZPmD16WXv/6qH415Kh6a110e6CnVT
         I0RmnjoTg4btPRNTtZs84Wh+noIFor2/vRXI31q4xpyHPU9Inz4hga2/UYVgdCXRCL0p
         6pcQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=rDIaq2zIrPogpfXrurzqgzFbIZpi6DRuEYogRKmWMm0=;
        b=cPWwW3GXnnbLevJ37zEKRxPpi4bijEjMUTaw/UdkIA0UeIKKPDI6vssb31Wdoq2PQS
         t9/1vFYPFV7owHJhms+byGgI5r3PKmWv/yuxFKavk/dgld15ilS5OW9BeBuw9DwuaOGa
         fY98MoOiSYXQ/oK89rJnNoUvKFpnnrzJuOpH/Nvd+9UTQQFwFsaRPMB+CGGZhArbWmnd
         jB5P2Y0FBWNaQ3WfRvXKNjhziRI4HogotRV1Cu8965WdmTNzmi+zbXOudVjAyA1A72Vx
         JDRroHrlTE37nnrInDq0AAy/Nzn8nJdsY7YQEcQ1FiPQNrb4jhmhhcR4tE9T2uglMFYC
         Vsvw==
X-Gm-Message-State: AOAM532Birc9ZcM5qdtonzptNBJ7bNxg6dHwxAyO6R6oN9UKUFa9zBBe
	qyiQEXoHxtnYNhQsF+BB0oFMyuUeuQU=
X-Google-Smtp-Source: ABdhPJxy5ThjF9sKx20tt0R6AYiEQmDq94Nfet1V5taXeyG5QHJoFEGJbDk0IEgOkGY5indJ35tg4A==
X-Received: by 2002:a2e:a261:: with SMTP id k1mr7628694ljm.509.1632437351535;
        Thu, 23 Sep 2021 15:49:11 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH V3 2/3] xen/arm: Add handling of extended regions for Dom0
Date: Fri, 24 Sep 2021 01:48:53 +0300
Message-Id: <1632437334-12015-3-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain
  currently.
- The ACPI case is not covered.

***

As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
the algorithm to choose extended regions for it is different
in comparison with the algorithm for non-direct mapped DomU.
What is more, that extended regions should be chosen differently
whether IOMMU is enabled or not.

Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
holes found in host device-tree if otherwise. Make sure that
extended regions are 2MB-aligned and located within maximum possible
addressable physical memory range. The minimum size of extended
region is 64MB. The maximum number of extended regions is 128,
which is an artificial limit to minimize code changes (we reuse
struct meminfo to describe extended regions, so there are an array
field for 128 elements).

It worth mentioning that unallocated memory solution (when the IOMMU
is disabled) will work safely until Dom0 is able to allocate memory
outside of the original range.

Also introduce command line option to be able to globally enable or
disable support for extended regions for Dom0 (enabled by default).

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

---
Please note, we need to decide which approach to use in find_unallocated_memory(),
you can find details at:
https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/

Changes RFC -> V2:
   - update patch description
   - drop uneeded "extended-region" DT property

Changes V2 -> V3:
   - update patch description
   - add comment for "size" calculation in add_ext_regions()
   - clarify "end" calculation in find_unallocated_memory() and
     find_memory_holes()
   - only pick up regions with size >= 64MB
   - allocate reg dynamically instead of keeping on the stack in
     make_hypervisor_node()
   - do not show warning for 32-bit domain
   - drop Linux specific limits EXT_REGION_*
   - also cover "ranges" property in find_memory_holes()
   - add command line arg to enable/disable extended region support
---
 docs/misc/xen-command-line.pandoc |   7 +
 xen/arch/arm/domain_build.c       | 280 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 284 insertions(+), 3 deletions(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 177e656..3bb8eb7 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1081,6 +1081,13 @@ hardware domain is architecture dependent.
 Note that specifying zero as domU value means zero, while for dom0 it means
 to use the default.
 
+### ext_regions (Arm)
+> `= <boolean>`
+
+> Default : `true`
+
+Flag to globally enable or disable support for extended regions for dom0.
+
 ### flask
 > `= permissive | enforcing | late | disabled`
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d63..81997d5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -34,6 +34,10 @@
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
 
+/* If true, the extended regions support is enabled for dom0 */
+static bool __initdata opt_ext_regions = true;
+boolean_param("ext_regions", opt_ext_regions);
+
 static u64 __initdata dom0_mem;
 static bool __initdata dom0_mem_set;
 
@@ -886,6 +890,233 @@ static int __init make_memory_node(const struct domain *d,
     return res;
 }
 
+static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
+{
+    struct meminfo *ext_regions = data;
+    paddr_t start, size;
+
+    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
+        return 0;
+
+    /* Both start and size of the extended region should be 2MB aligned */
+    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
+    if ( start > e )
+        return 0;
+
+    /*
+     * e is actually "end-1" because it is called by rangeset functions
+     * which are inclusive of the last address.
+     */
+    e += 1;
+    size = (e - start) & ~(SZ_2M - 1);
+    if ( size < MB(64) )
+        return 0;
+
+    ext_regions->bank[ext_regions->nr_banks].start = start;
+    ext_regions->bank[ext_regions->nr_banks].size = size;
+    ext_regions->nr_banks++;
+
+    return 0;
+}
+
+static int __init find_unallocated_memory(const struct kernel_info *kinfo,
+                                          struct meminfo *ext_regions)
+{
+    const struct meminfo *assign_mem = &kinfo->mem;
+    struct rangeset *unalloc_mem;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find unallocated memory for extended regions\n");
+
+    unalloc_mem = rangeset_new(NULL, NULL, 0);
+    if ( !unalloc_mem )
+        return -ENOMEM;
+
+    /* Start with all available RAM */
+    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
+    {
+        start = bootinfo.mem.bank[i].start;
+        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
+        res = rangeset_add_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove RAM assigned to Dom0 */
+    for ( i = 0; i < assign_mem->nr_banks; i++ )
+    {
+        start = assign_mem->bank[i].start;
+        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
+        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove reserved-memory regions */
+    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
+    {
+        start = bootinfo.reserved_mem.bank[i].start;
+        end = bootinfo.reserved_mem.bank[i].start +
+            bootinfo.reserved_mem.bank[i].size;
+        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove grant table region */
+    start = kinfo->gnttab_start;
+    end = kinfo->gnttab_start + kinfo->gnttab_size;
+    res = rangeset_remove_range(unalloc_mem, start, end - 1);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_report_ranges(unalloc_mem, start, end,
+                                 add_ext_regions, ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(unalloc_mem);
+
+    return res;
+}
+
+static int __init find_memory_holes(const struct kernel_info *kinfo,
+                                    struct meminfo *ext_regions)
+{
+    struct dt_device_node *np;
+    struct rangeset *mem_holes;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find memory holes for extended regions\n");
+
+    mem_holes = rangeset_new(NULL, NULL, 0);
+    if ( !mem_holes )
+        return -ENOMEM;
+
+    /* Start with maximum possible addressable physical memory range */
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_add_range(mem_holes, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    /*
+     * Remove regions described by "reg" and "ranges" properties where
+     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
+     */
+    dt_for_each_device_node( dt_host, np )
+    {
+        unsigned int naddr;
+        u64 addr, size;
+
+        naddr = dt_number_of_address(np);
+
+        for ( i = 0; i < naddr; i++ )
+        {
+            res = dt_device_get_address(np, i, &addr, &size);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                       i, dt_node_full_name(np));
+                goto out;
+            }
+
+            start = addr & PAGE_MASK;
+            end = PAGE_ALIGN(addr + size);
+            res = rangeset_remove_range(mem_holes, start, end - 1);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                       start, end);
+                goto out;
+            }
+        }
+
+        if ( dt_device_type_is_equal(np, "pci" ) )
+        {
+            unsigned int range_size, nr_ranges;
+            int na, ns, pna;
+            const __be32 *ranges;
+            u32 len;
+
+            /*
+             * Looking for non-empty ranges property which in the context
+             * of the PCI host bridge device node describes the BARs for
+             * the PCI devices.
+             */
+            ranges = dt_get_property(np, "ranges", &len);
+            if ( !ranges || !len )
+                continue;
+
+            pna = dt_n_addr_cells(np);
+            na = dt_child_n_addr_cells(np);
+            ns = dt_child_n_size_cells(np);
+            range_size = pna + na + ns;
+            nr_ranges = len / sizeof(__be32) / range_size;
+
+            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
+            {
+                /* Skip the child address and get the parent (CPU) address */
+                addr = dt_read_number(ranges + na, pna);
+                size = dt_read_number(ranges + na + pna, ns);
+
+                start = addr & PAGE_MASK;
+                end = PAGE_ALIGN(addr + size);
+                res = rangeset_remove_range(mem_holes, start, end - 1);
+                if ( res )
+                {
+                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                           start, end);
+                    goto out;
+                }
+            }
+        }
+    }
+
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_report_ranges(mem_holes, start, end,
+                                 add_ext_regions,  ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(mem_holes);
+
+    return res;
+}
+
 static int __init make_hypervisor_node(struct domain *d,
                                        const struct kernel_info *kinfo,
                                        int addrcells, int sizecells)
@@ -893,11 +1124,12 @@ static int __init make_hypervisor_node(struct domain *d,
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
         "xen,xen";
-    __be32 reg[4];
+    __be32 *reg, *cells;
     gic_interrupt_t intr;
-    __be32 *cells;
     int res;
     void *fdt = kinfo->fdt;
+    struct meminfo *ext_regions;
+    unsigned int i;
 
     dt_dprintk("Create hypervisor node\n");
 
@@ -919,12 +1151,54 @@ static int __init make_hypervisor_node(struct domain *d,
     if ( res )
         return res;
 
+    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
+    if ( !reg )
+        return -ENOMEM;
+
+    ext_regions = xzalloc(struct meminfo);
+    if ( !ext_regions )
+    {
+        xfree(reg);
+        return -ENOMEM;
+    }
+
+    if ( !opt_ext_regions )
+        printk(XENLOG_DEBUG "The extended regions support is disabled\n");
+    else if ( is_32bit_domain(d) )
+        printk(XENLOG_DEBUG "The extended regions are only supported for 64-bit guest currently\n");
+    else
+    {
+        if ( !is_iommu_enabled(d) )
+            res = find_unallocated_memory(kinfo, ext_regions);
+        else
+            res = find_memory_holes(kinfo, ext_regions);
+
+        if ( res )
+            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
+    }
+
     /* reg 0 is grant table space */
     cells = &reg[0];
     dt_child_set_range(&cells, addrcells, sizecells,
                        kinfo->gnttab_start, kinfo->gnttab_size);
+    /* reg 1...N are extended regions */
+    for ( i = 0; i < ext_regions->nr_banks; i++ )
+    {
+        u64 start = ext_regions->bank[i].start;
+        u64 size = ext_regions->bank[i].size;
+
+        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
+                   i, start, start + size);
+
+        dt_child_set_range(&cells, addrcells, sizecells, start, size);
+    }
+
     res = fdt_property(fdt, "reg", reg,
-                       dt_cells_to_size(addrcells + sizecells));
+                       dt_cells_to_size(addrcells + sizecells) *
+                       (ext_regions->nr_banks + 1));
+    xfree(ext_regions);
+    xfree(reg);
+
     if ( res )
         return res;
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 22:49:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 22:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194611.346709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWp-000433-9N; Thu, 23 Sep 2021 22:49:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194611.346709; Thu, 23 Sep 2021 22:49:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWp-00042w-5J; Thu, 23 Sep 2021 22:49:19 +0000
Received: by outflank-mailman (input) for mailman id 194611;
 Thu, 23 Sep 2021 22:49:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTXWn-0003lw-9w
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 22:49:17 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb12a67f-e05c-43f5-a993-c375bc7a66ed;
 Thu, 23 Sep 2021 22:49:11 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id e15so32412173lfr.10
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 15:49:11 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id q5sm230091lfd.17.2021.09.23.15.49.09
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 23 Sep 2021 15:49:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb12a67f-e05c-43f5-a993-c375bc7a66ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=gE4WiiHg7wgQhJBYuF1By1N/OYigu0Zgyl1iqewwPA4=;
        b=CGTqXrpguxWBi8yf5FdCxBlzplGYJCWq+3fMmJjG4jX+Cu9rc4FsBC1ODT84K9B108
         aiON/rWGiimiCVqtZI4SbZO9CIuVlZvsjgpeSutGUWam4Es8ISC3a/D+v55zc5kamS2f
         CLbhqYk/5hiQGPmrjZw5jhv26yTavAHqg1kub0fK/vLD0cFhk+5PBArCdT9aSahQAz0e
         s0hEY+KlNs7eN8Q19u+geu7uJcmhWG4GywHir8eRlsik6Adh0loF+ffgpcUo8kxSIcKW
         UcPPGPctqrMDSdPS+49Rd0HklgcmErMqvLHlqknZZw4SiJUxI+SugxOILfdCQXfLkgtD
         8P4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=gE4WiiHg7wgQhJBYuF1By1N/OYigu0Zgyl1iqewwPA4=;
        b=vTaqxJRLQ91H0A5NSuWpQFfex6OSw+g/G9iSx7JCuyOKtr6vfZ/2YdIuHvQcXh/Skb
         JN4Zex+vyoN4ZTtc9IuZuJHi0YE8XAG/SsbplwP/xTEfFO7/mJg0pw3A3MlHAjRWLd58
         flGjZ40PUO+LlooNV0USSjZjKO2yV6oTwLe7wQk1Ri9rcW1KAX3L4kRWJIb2qtKiVFDT
         Uq89LcFdLqgN2BPhMzNylsMBgBqdl49NazYJ1fokIphX+ohVIu5hKvqIxWW0Ao6GzG0L
         H/P9SrI3x4iMiJHLlYL3hwKVgov1CK66PgSeZeZAok1lxMcIKZtb2HTP4vYqJTiCGxqL
         0m9g==
X-Gm-Message-State: AOAM533tcbo7vDtuYJQ+2aXxnCiIiuDH1qdQgtgdhJyPkpeOJl3A3ACa
	Kww23grhkukvgrFLJ1TUOaPJ8mIjDBQ=
X-Google-Smtp-Source: ABdhPJytMCsSFFy2ebsKDlLcbHLToD8FlJYQ9mPGIj7ZUU+MPnj+f/ofxOewNKiryd4fC4+mbn6+KQ==
X-Received: by 2002:a05:6512:2210:: with SMTP id h16mr6675559lfu.598.1632437350734;
        Thu, 23 Sep 2021 15:49:10 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH V3 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
Date: Fri, 24 Sep 2021 01:48:52 +0300
Message-Id: <1632437334-12015-2-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

We need to pass info about maximum supported guest address
space size to the toolstack on Arm in order to properly
calculate the base and size of the extended region (safe range)
for the guest. The extended region is unused address space which
could be safely used by domain for foreign/grant mappings on Arm.
The extended region itself will be handled by the subsequents
patch.

Use p2m_ipa_bits variable on Arm, the x86 equivalent is
hap_paddr_bits.

As we change the size of structure bump the interface version.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, that review comments for the RFC version [1] haven't been addressed yet.
It is not forgotten, some clarification is needed. It will be addressed for the next version.

[1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/

Changes RFC -> V2:
   - update patch subject/description
   - replace arch-specific sub-struct with common gpaddr_bits
     field and update code to reflect that

Changes V2 -> V3:
   - make the field uint8_t and add uint8_t pad[7] after
   - remove leading blanks in libxl.h
---
 tools/include/libxl.h            | 7 +++++++
 tools/libs/light/libxl.c         | 2 ++
 tools/libs/light/libxl_types.idl | 2 ++
 xen/arch/arm/sysctl.c            | 2 ++
 xen/arch/x86/sysctl.c            | 2 ++
 xen/include/public/sysctl.h      | 4 +++-
 6 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d..63f9550 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -856,6 +856,13 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
 
 /*
+ * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
+ *
+ * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
+ */
+#define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
+
+/*
  * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1
  *
  * If this is defined, libxl_dominfo will contain a MemKB type field called
diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c
index 204eb0b..c86624f 100644
--- a/tools/libs/light/libxl.c
+++ b/tools/libs/light/libxl.c
@@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
     physinfo->cap_vmtrace =
         !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace);
 
+    physinfo->gpaddr_bits = xcphysinfo.gpaddr_bits;
+
     GC_FREE;
     return 0;
 }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff6..bf27fe6 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -1061,6 +1061,8 @@ libxl_physinfo = Struct("physinfo", [
     ("cap_shadow", bool),
     ("cap_iommu_hap_pt_share", bool),
     ("cap_vmtrace", bool),
+
+    ("gpaddr_bits", uint8),
     ], dir=DIR_OUT)
 
 libxl_connectorinfo = Struct("connectorinfo", [
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index f87944e..91dca4f 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -15,6 +15,8 @@
 void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
 {
     pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
+
+    pi->gpaddr_bits = p2m_ipa_bits;
 }
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a1..7b14865 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
     if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow;
+
+    pi->gpaddr_bits = hap_paddr_bits;
 }
 
 long arch_do_sysctl(
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 039ccf8..0450a78 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -35,7 +35,7 @@
 #include "domctl.h"
 #include "physdev.h"
 
-#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014
 
 /*
  * Read console content from Xen buffer ring.
@@ -120,6 +120,8 @@ struct xen_sysctl_physinfo {
     uint64_aligned_t outstanding_pages;
     uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
     uint32_t hw_cap[8];
+    uint8_t gpaddr_bits;
+    uint8_t pad[7];
 };
 
 /*
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 22:49:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 22:49:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194613.346731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWy-0004k8-VS; Thu, 23 Sep 2021 22:49:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194613.346731; Thu, 23 Sep 2021 22:49:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXWy-0004k1-Ru; Thu, 23 Sep 2021 22:49:28 +0000
Received: by outflank-mailman (input) for mailman id 194613;
 Thu, 23 Sep 2021 22:49:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTXWx-0003lw-AF
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 22:49:27 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 92a87372-4cd6-4740-8d81-20ee3df470ab;
 Thu, 23 Sep 2021 22:49:13 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id u8so31596482lff.9
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 15:49:13 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id q5sm230091lfd.17.2021.09.23.15.49.11
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 23 Sep 2021 15:49:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92a87372-4cd6-4740-8d81-20ee3df470ab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=jmLs0mTyD34ofMZzmrSjgVJC5wsYcw101jP+QWhYOH8=;
        b=YyI9IJzQrby7TiZ4wbGnS276SPS/+3GW73X0VIaNNHyW0IINUfUfR5fdfPdHIYmHdE
         Pi1iKR/eM0K2mVgfUSNNDCQXXcQcemn1EGUgCQbA2+G0SM2Yh6fGCKtDPg8QPZMjssJZ
         6EgKVxXgFrfoDP4stA4bjDId3/fGB4+Yqa+sDnfnP4i2NStC9oKG+L7jmp9aMUXO2JrL
         vkfI/J/op5MBTn0YKUCA5T01hfmRQae8Zr2+81vM4OoQjOyV7yEqStWkVhgq1IbpONcN
         q4mVERIpB0tSQR0YIYHS20mvae5ao7uElHuKIelHhPUrzUsTr7IjrAr7C58yvUt6eGyb
         Dtig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=jmLs0mTyD34ofMZzmrSjgVJC5wsYcw101jP+QWhYOH8=;
        b=rvsnhC2gef0mO0lJmXVQ+e/Y/CExtykuMg6zH38So+7xZNkQwTAAP55a1HZBbvpMIk
         UVkyvvBaRMeTQOg+Dd5tuT7qBM2GQxZyrE3RrthnokmT7c3vO+/9ogJswIuu3D3q1VTt
         qLkYcvcbns9E+K925kQ/A72XsNMN7Xi7nPtVk2BYX1JxMODNytNmiPCFhsr6l2tSrOQS
         t4kyw0g/vROfKljqVdJ+4j5dnsHj6JjxB1f5KPaKNotoMPeBRZSQK10jgViwotz7dubi
         Kc1ikg7+hoRx6lyz8sX2qY5jGtDvBfkYlzIDVI5tPGVMb8cLKebPp1Yh/1J9UWjeMhlc
         d6Kg==
X-Gm-Message-State: AOAM530BHfAtvojUnPDrFqXmN6/0rGVIOBMdhLlhjIaXbtOXNzZ+bMej
	CFdxGRU+tjDVPfyijWMPFonmka4pOyU=
X-Google-Smtp-Source: ABdhPJzJ4gywKT2GPnBQZNU6Z/TJttX8q3HxdBvVA2xr556u3bBX1MO96LSCIMBzAnreZJrKPrii7A==
X-Received: by 2002:ac2:5fe7:: with SMTP id s7mr6504634lfg.666.1632437352352;
        Thu, 23 Sep 2021 15:49:12 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH V3 3/3] libxl/arm: Add handling of extended regions for DomU
Date: Fri, 24 Sep 2021 01:48:54 +0300
Message-Id: <1632437334-12015-4-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain
  currently.
- The ACPI case is not covered.

***

The algorithm to choose extended regions for non-direct mapped
DomU is simpler in comparison with the algorithm for direct mapped
Dom0. As we have a lot of unused space above 4GB, provide single
2MB-aligned region from the second RAM bank taking into the account
the maximum supported guest address space size and the amount of
memory assigned to the guest. The maximum size of the region is 128GB.
The minimum size is 64MB.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

---
Changes RFC -> V2:
   - update patch description
   - drop uneeded "extended-region" DT property
   - clear reg array in finalise_ext_region() and add a TODO

Changes V2 -> V3:
   - update patch description, comments in code
   - only pick up regions with size >= 64MB
   - move the region calculation to make_hypervisor_node() and drop
     finalise_ext_region()
   - extend the list of arguments for make_hypervisor_node()
   - do not show warning for 32-bit domain
   - change the region alignment from 1GB to 2MB
   - move EXT_REGION_SIZE to public/arch-arm.h
---
 tools/libs/light/libxl_arm.c  | 70 +++++++++++++++++++++++++++++++++++++++----
 xen/include/public/arch-arm.h |  3 ++
 2 files changed, 68 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6..a67b68e 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -598,9 +598,17 @@ static int make_timer_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+#define ALIGN_UP_TO_2MB(x)   (((x) + MB(2) - 1) & (~(MB(2) - 1)))
+
 static int make_hypervisor_node(libxl__gc *gc, void *fdt,
-                                const libxl_version_info *vers)
+                                const libxl_version_info *vers,
+                                const libxl_domain_build_info *b_info,
+                                const struct xc_dom_image *dom)
 {
+    uint64_t region_size = 0, region_base, ramsize, bank1size,
+        bank1end_align, bank1end_max;
+    uint8_t gpaddr_bits;
+    libxl_physinfo physinfo;
     int res;
     gic_interrupt intr;
 
@@ -615,9 +623,61 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
                               "xen,xen");
     if (res) return res;
 
-    /* reg 0 is grant table space */
-    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
+        LOG(DEBUG, "The extended regions are only supported for 64-bit guest currently");
+        goto out;
+    }
+
+    res = libxl_get_physinfo(CTX, &physinfo);
+    assert(!res);
+
+    gpaddr_bits = physinfo.gpaddr_bits;
+    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
+
+    /*
+     * Try to allocate single 2MB-aligned extended region from the second RAM
+     * bank (above 4GB) taking into the account the maximum supported guest
+     * address space size and the amount of memory assigned to the guest.
+     * As the guest memory layout is not populated yet we cannot rely on
+     * dom->rambank_size[1], so calculate the actual size of the second bank
+     * using "max_memkb" value.
+     */
+    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
+    ramsize = b_info->max_memkb * 1024;
+    if (ramsize <= GUEST_RAM0_SIZE)
+        bank1size = 0;
+    else
+        bank1size = ramsize - GUEST_RAM0_SIZE;
+    bank1end_align = GUEST_RAM1_BASE + ALIGN_UP_TO_2MB(bank1size);
+
+    if (bank1end_max <= bank1end_align) {
+        LOG(WARN, "The extended region cannot be allocated, not enough space");
+        goto out;
+    }
+
+    if (bank1end_max - bank1end_align > GUEST_EXT_REGION_MAX_SIZE) {
+        region_base = bank1end_max - GUEST_EXT_REGION_MAX_SIZE;
+        region_size = GUEST_EXT_REGION_MAX_SIZE;
+    } else {
+        region_base = bank1end_align;
+        region_size = bank1end_max - bank1end_align;
+    }
+
+out:
+    /*
+     * The region 0 for grant table space must be always present. If we managed
+     * to allocate the extended region then insert it as region 1.
+     */
+    if (region_size >= GUEST_EXT_REGION_MIN_SIZE) {
+        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
+            region_base, region_base + region_size);
+
+        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                                2, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE,
+                                region_base, region_size);
+    } else
+        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                                1, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
     if (res) return res;
 
     /*
@@ -963,7 +1023,7 @@ next_resize:
         }
 
         FDT( make_timer_node(gc, fdt, ainfo, state->clock_frequency) );
-        FDT( make_hypervisor_node(gc, fdt, vers) );
+        FDT( make_hypervisor_node(gc, fdt, vers, info, dom) );
 
         if (info->arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART)
             FDT( make_vpl011_uart_node(gc, fdt, ainfo, dom) );
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 6b5a5f8..df59933 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -449,6 +449,9 @@ typedef uint64_t xen_callback_t;
 #define GUEST_RAM_BANK_BASES   { GUEST_RAM0_BASE, GUEST_RAM1_BASE }
 #define GUEST_RAM_BANK_SIZES   { GUEST_RAM0_SIZE, GUEST_RAM1_SIZE }
 
+#define GUEST_EXT_REGION_MAX_SIZE   xen_mk_ullong(0x2000000000) /* 128GB */
+#define GUEST_EXT_REGION_MIN_SIZE   xen_mk_ullong(0x0004000000) /* 64MB */
+
 /* Current supported guest VCPUs */
 #define GUEST_MAX_VCPUS 128
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 22:52:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 22:52:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194627.346741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXa3-0006vv-F8; Thu, 23 Sep 2021 22:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194627.346741; Thu, 23 Sep 2021 22:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTXa3-0006vo-CB; Thu, 23 Sep 2021 22:52:39 +0000
Received: by outflank-mailman (input) for mailman id 194627;
 Thu, 23 Sep 2021 22:52:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ES3T=ON=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mTXa1-0006vb-Ix
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 22:52:37 +0000
Received: from mail-lf1-x133.google.com (unknown [2a00:1450:4864:20::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cebb96f1-935e-4a1c-a4f8-eb3baed19555;
 Thu, 23 Sep 2021 22:52:36 +0000 (UTC)
Received: by mail-lf1-x133.google.com with SMTP id x27so32458280lfu.5
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 15:52:36 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id l13sm363733lfk.211.2021.09.23.15.52.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 23 Sep 2021 15:52:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cebb96f1-935e-4a1c-a4f8-eb3baed19555
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=Rj7wStZWi35kb7uWr6bnSQfsxjooFhry4SVMfYi22tM=;
        b=ICz05OGAfH39Vk/ef/1HUaz+Dz3i9X3A556sgUc9XQqG+k2MZBZcEzDPXM7OB/9SKL
         5wXPIWHbwe38Z2ddK9VvrExNZrWNVsllyxBqMcS1sNFjp4vWyDxvhY0Jfdwb6bbk3vO4
         lUzNXX+KCYhIce+ISnMMie56sHYiRAg+pF5PbroAuMN8GKTvDHGNF0xTTveovzUxhZQR
         w7TwvpRF1WvA4gZdU8UU/8AA2CMBGRKfSq1C+Akw0rIy7wjOpZTig24Uki0v17+6TEUa
         a8p7Frg+bjRom1vcPq4RIfW7xgaLPWMauHRvzsUrjCiFXp+vmipJvB24kYYpI1wM6+KC
         3sHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=Rj7wStZWi35kb7uWr6bnSQfsxjooFhry4SVMfYi22tM=;
        b=MxPVsjHXNQlRN4DFG/6DrF48AGzJkKyUUtPDSBTh18cSao254fgF7ENEOVdbneh67p
         MfWk+sDdklQrkYi3Mh4x3S6Dh1Go4p1hqWNjHcZ/U4LIGPaJStB6rFj6hHzyVzlBv4jk
         Gh71UeEonMRTif3uH2MEOkIgNhPsdpC4lNwRa3TQPxmisK1Ewrj7XbBBH6XeZrJDXyIA
         LT8YeBWp3qXqhNsZd+Gb6kFk/LJeewtbUsDUtHWwa6HW2+jzWjVq7gFlOFC1UWsqQuEk
         shMMpWKJoAJAFYEX5njMfjh+/KcqW9QaU+pZHWbYGQWP3HFhicoNEH8uGOX26sSrL6Df
         d8sw==
X-Gm-Message-State: AOAM5331I2gcc9r2mApoqRyl30jbqh0rBlqu13fXjvVWUDZ1ym1KhUGv
	YGA+wrivJHtfhBqSa8UsLKw=
X-Google-Smtp-Source: ABdhPJwQ68hL26Z+5d7YHw5p610VS+FJNXOU3mRNAsV2mw2LQLtS9fcPVzfnxhvjDtyuALtcXkiPeg==
X-Received: by 2002:a05:651c:110f:: with SMTP id d15mr7659767ljo.99.1632437555568;
        Thu, 23 Sep 2021 15:52:35 -0700 (PDT)
Subject: Re: [RFC PATCH 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
From: Oleksandr <olekstysh@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>, Wei Chen <Wei.Chen@arm.com>,
 Henry Wang <Henry.Wang@arm.com>
References: <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
 <1631034578-12598-2-git-send-email-olekstysh@gmail.com>
 <973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com>
 <0994251e-5e02-009a-851d-4e4ee760fc66@gmail.com>
Message-ID: <6a2a183d-c9d8-df2a-41aa-b25283fab197@gmail.com>
Date: Fri, 24 Sep 2021 01:52:34 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <0994251e-5e02-009a-851d-4e4ee760fc66@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


Hi Andrew.


On 08.09.21 00:28, Oleksandr wrote:
>
> On 07.09.21 20:35, Andrew Cooper wrote:
>
> Hi Andrew
>
>> On 07/09/2021 18:09, Oleksandr Tyshchenko wrote:
>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>
>>> We need to pass info about maximum supported guest address
>>> space size to the toolstack on Arm in order to properly
>>> calculate the base and size of the extended region (safe range)
>>> for the guest. The extended region is unused address space which
>>> could be safely used by domain for foreign/grant mappings on Arm.
>>> The extended region itself will be handled by the subsequents
>>> patch.
>>>
>>> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
>>> hap_paddr_bits.
>>>
>>> As we change the size of structure bump the interface version.
>>>
>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> So while I think this is a suitable way forward, you're painting
>> yourself into a corner WRT migration.
>>
>> On x86, the correct value is d->arch.cpuid->extd.maxphysaddr and this
>> value is under toolstack control, not Xen control.  In particular, it
>> needs to be min(hostA, hostB) to make migration safe, and yes - this is
>> currently a hole in x86's migration logic that will cause large VMs to
>> explode.
>>
>> The same will be true on ARM as/when you gain migration support.
>
> Oh, I admit, I didn't keep in mind migration support while creating 
> this patch.
>
>
>>
>> I think this would be better as a domctl.  On ARM, it can reference
>> p2m_ipa_bits for now along with a /* TODO - make per-domain for
>> migration support */, while on x86 it can take the appropriate value
>> (which will soon actually be safe in migration scenarios).
>
> OK, could you please clarify, did you mean to introduce new domctl 
> (something like get_maxphysaddr) or reuse some existing?


May I please ask for clarification here ^ and ...


>
>
> And ...
>
>>
>> However, that does change the ordering requirements in the toolstack -
>> this hypercall would need to be made after the domain is created, and
>> has been levelled, and before its main memory layout is decided.
>
> ... I am not sure I totally understand the ordering requirements in 
> the toolstack.
>
> On Arm this information (gpaddr_bits) should be obtained by the time 
> we call libxl__arch_domain_finalise_hw_description()
> where we actually calculate extended region and insert it in domain's 
> device-tree. At that time, the domain memory is already populated, so 
> it's layout is known.
> Please see the last patch of this series, which demonstrates the usage:
>
> https://lore.kernel.org/xen-devel/1631034578-12598-4-git-send-email-olekstysh@gmail.com/ 
>
>
>
>> Alternatively, the abstraction could be hidden in libxl itself in arch
>> specific code, with x86 referring to the local cpu policy (as libxl has
>> the copy it is/has worked on), and ARM making a hypercall.  This does
>> make the ordering more obvious.
>
> May I please ask what would be the preferred option to move forward? I 
> am fine with both proposed options, with a little preference for the 
> former (common domctl, abstraction is hidden in Xen itself in arch 
> specific code). It is highly appreciated if we could choose the option 
> which would satisfy all parties, this would save me some time.


... here ^ ?


I have just pushed third version [1] of this series without these being 
addressed. I decided to push V3 to let the reviewers to focus on the 
main aspects, but the way how the information in question is passed to 
the toolstack also needs to be clarified and addressed.


[1] 
https://lore.kernel.org/xen-devel/1632437334-12015-1-git-send-email-olekstysh@gmail.com/


[snip]


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Thu Sep 23 23:45:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 23:45:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194637.346753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYPE-00053C-Fq; Thu, 23 Sep 2021 23:45:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194637.346753; Thu, 23 Sep 2021 23:45:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYPE-000535-CE; Thu, 23 Sep 2021 23:45:32 +0000
Received: by outflank-mailman (input) for mailman id 194637;
 Thu, 23 Sep 2021 23:45:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTYPD-00052z-3X
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 23:45:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0b3f7f2f-4c46-4171-87e7-ecaf100f97e0;
 Thu, 23 Sep 2021 23:45:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A1DE610FD;
 Thu, 23 Sep 2021 23:45:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b3f7f2f-4c46-4171-87e7-ecaf100f97e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632440729;
	bh=jOSq/3ovecOYuSTTpHKQ/vVhAqVUUrdiwVGeYsonii4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RuoA0uDz67utqpVotYZPg6ZDNuYqNc7MFWYPI8b4Ne/y5mrn60usoNUcSu8UEHOl0
	 Y+O28pMUOTDyUCSaGj2B728Ubm77i1coPGBDjRbTqRXDmGQ1TMakXFD8+ivtWAriXm
	 GfN1JwjLFd/nvi/JJarCAQ5Cggn9Im5mozKNVGHCU+GF13KQcSkJaxvSBrVY9VRPKD
	 ccMc/GuVUIsScAnjDt/QHxZqg6QBhIY9LCq0DOeBAOeL2nn3GIKbCJHno4Ezwp/spB
	 yhPqvXI/t3FpPE8N6UiB8m3aqoxhb51laVwMyvhd+xhAL6rdG2vYbJDEG2mptQrgyG
	 wiCQZuR+P3+gA==
Date: Thu, 23 Sep 2021 16:45:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
In-Reply-To: <20210923120236.3692135-3-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231639110.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-3-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Current NUMA nodes number is a hardcode configuration. This
> configuration is difficult for an administrator to change
> unless changing the code.
> 
> So in this patch, we introduce this new Kconfig option for
> administrators to change NUMA nodes number conveniently.
> Also considering that not all architectures support NUMA,
> this Kconfig option only can be visible on NUMA enabled
> architectures. Non-NUMA supported architectures can still
> use 1 as MAX_NUMNODES.

This is OK but I think you should also mention in the commit message
that you are taking the opportunity to remove NODES_SHIFT because it is
currently unused.

With that:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/Kconfig           | 11 +++++++++++
>  xen/include/asm-x86/numa.h |  2 --
>  xen/include/xen/numa.h     | 10 +++++-----
>  3 files changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
> index f16eb0df43..8a20da67ed 100644
> --- a/xen/arch/Kconfig
> +++ b/xen/arch/Kconfig
> @@ -17,3 +17,14 @@ config NR_CPUS
>  	  For CPU cores which support Simultaneous Multi-Threading or similar
>  	  technologies, this the number of logical threads which Xen will
>  	  support.
> +
> +config NR_NUMA_NODES
> +	int "Maximum number of NUMA nodes supported"
> +	range 1 4095
> +	default "64"
> +	depends on NUMA
> +	help
> +	  Controls the build-time size of various arrays and bitmaps
> +	  associated with multiple-nodes management. It is the upper bound of
> +	  the number of NUMA nodes the scheduler, memory allocation and other
> +	  NUMA-aware components can handle.
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index bada2c0bb9..3cf26c2def 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -3,8 +3,6 @@
>  
>  #include <xen/cpumask.h>
>  
> -#define NODES_SHIFT 6
> -
>  typedef u8 nodeid_t;
>  
>  extern int srat_rev;
> diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
> index 7aef1a88dc..52950a3150 100644
> --- a/xen/include/xen/numa.h
> +++ b/xen/include/xen/numa.h
> @@ -3,14 +3,14 @@
>  
>  #include <asm/numa.h>
>  
> -#ifndef NODES_SHIFT
> -#define NODES_SHIFT     0
> -#endif
> -
>  #define NUMA_NO_NODE     0xFF
>  #define NUMA_NO_DISTANCE 0xFF
>  
> -#define MAX_NUMNODES    (1 << NODES_SHIFT)
> +#ifdef CONFIG_NR_NUMA_NODES
> +#define MAX_NUMNODES CONFIG_NR_NUMA_NODES
> +#else
> +#define MAX_NUMNODES    1
> +#endif
>  
>  #define vcpu_to_node(v) (cpu_to_node((v)->processor))
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 23 23:55:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 23 Sep 2021 23:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194641.346764 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYZ8-0006nj-EB; Thu, 23 Sep 2021 23:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194641.346764; Thu, 23 Sep 2021 23:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYZ8-0006nc-Aw; Thu, 23 Sep 2021 23:55:46 +0000
Received: by outflank-mailman (input) for mailman id 194641;
 Thu, 23 Sep 2021 23:55:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/SRu=ON=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTYZ6-0006nD-4D
 for xen-devel@lists.xenproject.org; Thu, 23 Sep 2021 23:55:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c25fec72-1cc9-11ec-ba94-12813bfff9fa;
 Thu, 23 Sep 2021 23:55:42 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1E0A561042;
 Thu, 23 Sep 2021 23:55:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c25fec72-1cc9-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632441341;
	bh=g1d6plp9n6z37Cp90XkrvmZhdlPUWQ9JIkXZG3vlcYI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=sx8JoZhUxXXZtzSfvbzx3o7mmUY2oov2Vh33uRK8HGepEZERxaNekkGehEaWklz8U
	 v76PnrV0MKS9JNyiiUJYmVMUxw3AaZxa3Zv8OY16T3n7CPrzqiZbk5NGundWVI9nx3
	 h4TDIJC9AeDLRvSsrdVC7g38f2qR36nqaxV2CctXTrmMGJAmHRUVWvsoIqc1C5X8i2
	 ABN72Vmns9t7s6141hDVdIAq248azPbHVlvg55r2sq1EuYn75IafJhVWvD3hSlrKNF
	 NRB6nZcZm5+cYBNbuRT64UHU4IOIcViNktFP16cjZdFn9uxl6DP6owqVjtc4uc/aDq
	 SW//apUH7mvAw==
Date: Thu, 23 Sep 2021 16:55:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 04/37] xen: introduce an arch helper for default dma zone
 status
In-Reply-To: <20210923120236.3692135-5-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231652340.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-5-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> In current code, when Xen is running in a multiple nodes NUMA
> system, it will set dma_bitsize in end_boot_allocator to reserve
> some low address memory for DMA.
> 
> There are some x86 implications in current implementation. Becuase
                                    ^ the                    ^Because

> on x86, memory starts from 0. On a multiple nodes NUMA system, if
> a single node contains the majority or all of the DMA memory. x86
                                                              ^,

> prefer to give out memory from non-local allocations rather than
> exhausting the DMA memory ranges. Hence x86 use dma_bitsize to set
> aside some largely arbitrary amount memory for DMA memory ranges.
                                     ^ of memory

> The allocations from these memory ranges would happen only after
> exhausting all other nodes' memory.
> 
> But the implications are not shared across all architectures. For
> example, Arm doesn't have these implications. So in this patch, we
> introduce an arch_have_default_dmazone helper for arch to determine
> that it need to set dma_bitsize for reserve DMA allocations or not.
          ^ needs

> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c        | 5 +++++
>  xen/common/page_alloc.c    | 2 +-
>  xen/include/asm-arm/numa.h | 5 +++++
>  xen/include/asm-x86/numa.h | 1 +
>  4 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index ce79ee44ce..1fabbe8281 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -371,6 +371,11 @@ unsigned int __init arch_get_dma_bitsize(void)
>                   + PAGE_SHIFT, 32);
>  }
>  
> +unsigned int arch_have_default_dmazone(void)

Can this function return bool?
Also, can it be a static inline?


> +{
> +    return ( num_online_nodes() > 1 ) ? 1 : 0;
> +}
> +
>  static void dump_numa(unsigned char key)
>  {
>      s_time_t now = NOW();
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 5801358b4b..80916205e5 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1889,7 +1889,7 @@ void __init end_boot_allocator(void)
>      }
>      nr_bootmem_regions = 0;
>  
> -    if ( !dma_bitsize && (num_online_nodes() > 1) )
> +    if ( !dma_bitsize && arch_have_default_dmazone() )
>          dma_bitsize = arch_get_dma_bitsize();
>  
>      printk("Domain heap initialised");
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 31a6de4e23..9d5739542d 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +static inline unsigned int arch_have_default_dmazone(void)
> +{
> +    return 0;
> +}
> +
>  #endif /* __ARCH_ARM_NUMA_H */
>  /*
>   * Local variables:
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 3cf26c2def..8060cbf3f4 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -78,5 +78,6 @@ extern int valid_numa_range(u64 start, u64 end, nodeid_t node);
>  void srat_parse_regions(u64 addr);
>  extern u8 __node_distance(nodeid_t a, nodeid_t b);
>  unsigned int arch_get_dma_bitsize(void);
> +unsigned int arch_have_default_dmazone(void);
>  
>  #endif
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:05:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:05:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194645.346775 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYiI-0000jk-IT; Fri, 24 Sep 2021 00:05:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194645.346775; Fri, 24 Sep 2021 00:05:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYiI-0000jd-F5; Fri, 24 Sep 2021 00:05:14 +0000
Received: by outflank-mailman (input) for mailman id 194645;
 Fri, 24 Sep 2021 00:05:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTYiH-0000jX-3o
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:05:13 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e0ee273f-5349-477d-9146-9fd5b785ebd3;
 Fri, 24 Sep 2021 00:05:12 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 78BE3611B0;
 Fri, 24 Sep 2021 00:05:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0ee273f-5349-477d-9146-9fd5b785ebd3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632441911;
	bh=3O0Urk1mfIZ8pE3vYT71E1dtO5x3OpOa/42t+S1iesg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=iuPC2H92OE9IyHIipMFmGd9lklFb0im44mPDdbL0948LhTnfMVbPKrDmXyJG/7PtR
	 /JtxALnXHujZctHS4L3PaKUwaxVN6xcDLt4WT8Vl86Rd1Jf3LXCsW6OVZ0ZWyV1ZD3
	 umU2JuLzE0YfdG9/a7dOnDNL7/kvolPzbrz0zccvnQHFJmEBwiKvUFspoMSlvE0Ssq
	 KhvlKkIPDEv02ycLVf7g4fwlEaXuQ/2EhwvbwW61Tept4a9ExXxY9ckSlSQtG9lSTd
	 yZuHYmR4uGqZ7UU6ygbqs2gAHMYU2GRRZIGUK49WAc7AN5sYtFzDxsFPzte88yqKun
	 6XvA3xMI5XPLg==
Date: Thu, 23 Sep 2021 17:05:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 06/37] xen/arm: use !CONFIG_NUMA to keep fake NUMA API
In-Reply-To: <20210923120236.3692135-7-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231703070.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-7-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> We have introduced CONFIG_NUMA in previous patch. And this
                                   ^ a

> option is enabled only on x86 in current stage. In a follow
                                ^ at the

> up patch, we will enable this option for Arm. But we still
> want users can disable the CONFIG_NUMA through Kconfig. In
             ^ to be able to disable CONFIG_NUMA via Kconfig.


> this case, keep current fake NUMA API, will make Arm code
                 ^ the

> still can work with NUMA aware memory allocation and scheduler.
        ^ able to work

> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>

With the small grammar fixes:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/include/asm-arm/numa.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 9d5739542d..8f1c67e3eb 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -5,6 +5,8 @@
>  
>  typedef u8 nodeid_t;
>  
> +#ifndef CONFIG_NUMA
> +
>  /* Fake one node for now. See also node_online_map. */
>  #define cpu_to_node(cpu) 0
>  #define node_to_cpumask(node)   (cpu_online_map)
> @@ -25,6 +27,8 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +#endif
> +
>  static inline unsigned int arch_have_default_dmazone(void)
>  {
>      return 0;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:12:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:12:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194651.346785 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYom-0002VP-Dj; Fri, 24 Sep 2021 00:11:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194651.346785; Fri, 24 Sep 2021 00:11:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYom-0002VI-As; Fri, 24 Sep 2021 00:11:56 +0000
Received: by outflank-mailman (input) for mailman id 194651;
 Fri, 24 Sep 2021 00:11:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTYol-0002VC-3o
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:11:55 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae61e749-e991-4902-89be-9a61b163a5c1;
 Fri, 24 Sep 2021 00:11:53 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B2828604DA;
 Fri, 24 Sep 2021 00:11:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae61e749-e991-4902-89be-9a61b163a5c1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632442312;
	bh=eT1VwySAiX18zGiiz79v6G8srB6NMvzccaZplvStdOk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Nbzq0h4An/L+4urSr5nqiqZZhiKTU/Lw7Dnvb28ZC4hTOFZlcur0hjTJii3XI+Ini
	 QoMJw0ZlMG/GjDoLEbEP7CgcP4c0sfXibU4MduxAOVreI/zYc1s8FFprRDyORyU7R7
	 Oc+tfv7ZkGqf4AN7uG8q1+ztRDTOQ7O8DS1LCLZvXjgsg+IGsy4cr254sxbiBUSI+s
	 qVpl0ZsNmLlvGH9n8kMQ0D6u2HFuKjKcZ3WfHsSxNtwBhuhIdFWfY5JBsDGf2HBzZH
	 DRbtpwpLGf8eLf8QuWvL4QIjERHzMkU7cfp14CXDr5TrAjFzHjJkA/u+AIn+X5jpaB
	 wQ7KsS0TePInQ==
Date: Thu, 23 Sep 2021 17:11:51 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node
 structure
In-Reply-To: <20210923120236.3692135-8-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231709360.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-8-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> NUMA node structure "struct node" is using u64 as node memory
> range. In order to make other architectures can reuse this
> NUMA node relative code, we replace the u64 to paddr_t. And
> use pfn_to_paddr and paddr_to_pfn to replace explicit shift
> operations. The relate PRIx64 in print messages have been
> replaced by PRIpaddr at the same time.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c        | 32 +++++++++++++++++---------------
>  xen/arch/x86/srat.c        | 26 +++++++++++++-------------
>  xen/include/asm-x86/numa.h |  8 ++++----
>  3 files changed, 34 insertions(+), 32 deletions(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 1fabbe8281..6337bbdf31 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -165,12 +165,12 @@ int __init compute_hash_shift(struct node *nodes, int numnodes,
>      return shift;
>  }
>  /* initialize NODE_DATA given nodeid and start/end */
> -void __init setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end)
> -{ 
> +void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end)
> +{
>      unsigned long start_pfn, end_pfn;
>  
> -    start_pfn = start >> PAGE_SHIFT;
> -    end_pfn = end >> PAGE_SHIFT;
> +    start_pfn = paddr_to_pfn(start);
> +    end_pfn = paddr_to_pfn(end);
>  
>      NODE_DATA(nodeid)->node_start_pfn = start_pfn;
>      NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn;
> @@ -201,11 +201,12 @@ void __init numa_init_array(void)
>  static int numa_fake __initdata = 0;
>  
>  /* Numa emulation */
> -static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
> +static int __init numa_emulation(unsigned long start_pfn,
> +                                 unsigned long end_pfn)

Why not changing numa_emulation to take paddr_t too?


>  {
>      int i;
>      struct node nodes[MAX_NUMNODES];
> -    u64 sz = ((end_pfn - start_pfn)<<PAGE_SHIFT) / numa_fake;
> +    u64 sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake;
>  
>      /* Kludge needed for the hash function */
>      if ( hweight64(sz) > 1 )
> @@ -221,9 +222,9 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
>      memset(&nodes,0,sizeof(nodes));
>      for ( i = 0; i < numa_fake; i++ )
>      {
> -        nodes[i].start = (start_pfn<<PAGE_SHIFT) + i*sz;
> +        nodes[i].start = pfn_to_paddr(start_pfn) + i*sz;
>          if ( i == numa_fake - 1 )
> -            sz = (end_pfn<<PAGE_SHIFT) - nodes[i].start;
> +            sz = pfn_to_paddr(end_pfn) - nodes[i].start;
>          nodes[i].end = nodes[i].start + sz;
>          printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n",
>                 i,
> @@ -249,24 +250,26 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
>  void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)

same here


>  { 
>      int i;
> +    paddr_t start, end;
>  
>  #ifdef CONFIG_NUMA_EMU
>      if ( numa_fake && !numa_emulation(start_pfn, end_pfn) )
>          return;
>  #endif
>  
> +    start = pfn_to_paddr(start_pfn);
> +    end = pfn_to_paddr(end_pfn);
> +
>  #ifdef CONFIG_ACPI_NUMA
> -    if ( !numa_off && !acpi_scan_nodes((u64)start_pfn << PAGE_SHIFT,
> -         (u64)end_pfn << PAGE_SHIFT) )
> +    if ( !numa_off && !acpi_scan_nodes(start, end) )
>          return;
>  #endif
>  
>      printk(KERN_INFO "%s\n",
>             numa_off ? "NUMA turned off" : "No NUMA configuration found");
>  
> -    printk(KERN_INFO "Faking a node at %016"PRIx64"-%016"PRIx64"\n",
> -           (u64)start_pfn << PAGE_SHIFT,
> -           (u64)end_pfn << PAGE_SHIFT);
> +    printk(KERN_INFO "Faking a node at %016"PRIpaddr"-%016"PRIpaddr"\n",
> +           start, end);
>      /* setup dummy node covering all memory */
>      memnode_shift = BITS_PER_LONG - 1;
>      memnodemap = _memnodemap;
> @@ -279,8 +282,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
>      for ( i = 0; i < nr_cpu_ids; i++ )
>          numa_set_node(i, 0);
>      cpumask_copy(&node_to_cpumask[0], cpumask_of(0));
> -    setup_node_bootmem(0, (u64)start_pfn << PAGE_SHIFT,
> -                    (u64)end_pfn << PAGE_SHIFT);
> +    setup_node_bootmem(0, start, end);
>  }
>  
>  void numa_add_cpu(int cpu)
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 6b77b98201..7d20d7f222 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -104,7 +104,7 @@ nodeid_t setup_node(unsigned pxm)
>  	return node;
>  }
>  
> -int valid_numa_range(u64 start, u64 end, nodeid_t node)
> +int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
>  {
>  	int i;
>  
> @@ -119,7 +119,7 @@ int valid_numa_range(u64 start, u64 end, nodeid_t node)
>  	return 0;
>  }
>  
> -static __init int conflicting_memblks(u64 start, u64 end)
> +static __init int conflicting_memblks(paddr_t start, paddr_t end)
>  {
>  	int i;
>  
> @@ -135,7 +135,7 @@ static __init int conflicting_memblks(u64 start, u64 end)
>  	return -1;
>  }
>  
> -static __init void cutoff_node(int i, u64 start, u64 end)
> +static __init void cutoff_node(int i, paddr_t start, paddr_t end)
>  {
>  	struct node *nd = &nodes[i];
>  	if (nd->start < start) {
> @@ -275,7 +275,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
>  void __init
>  acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  {
> -	u64 start, end;
> +	paddr_t start, end;
>  	unsigned pxm;
>  	nodeid_t node;
>  	int i;
> @@ -318,7 +318,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  		bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
>  		                !test_bit(i, memblk_hotplug);
>  
> -		printk("%sSRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with itself (%"PRIx64"-%"PRIx64")\n",
> +		printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
>  		       mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
>  		       node_memblk_range[i].start, node_memblk_range[i].end);
>  		if (mismatch) {
> @@ -327,7 +327,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  		}
>  	} else {
>  		printk(KERN_ERR
> -		       "SRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with PXM %u (%"PRIx64"-%"PRIx64")\n",
> +		       "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
>  		       pxm, start, end, node_to_pxm(memblk_nodeid[i]),
>  		       node_memblk_range[i].start, node_memblk_range[i].end);
>  		bad_srat();
> @@ -346,7 +346,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  				nd->end = end;
>  		}
>  	}
> -	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIx64"-%"PRIx64"%s\n",
> +	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
>  	       node, pxm, start, end,
>  	       ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
>  
> @@ -369,7 +369,7 @@ static int __init nodes_cover_memory(void)
>  
>  	for (i = 0; i < e820.nr_map; i++) {
>  		int j, found;
> -		unsigned long long start, end;
> +		paddr_t start, end;
>  
>  		if (e820.map[i].type != E820_RAM) {
>  			continue;
> @@ -396,7 +396,7 @@ static int __init nodes_cover_memory(void)
>  
>  		if (start < end) {
>  			printk(KERN_ERR "SRAT: No PXM for e820 range: "
> -				"%016Lx - %016Lx\n", start, end);
> +				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
>  			return 0;
>  		}
>  	}
> @@ -432,7 +432,7 @@ static int __init srat_parse_region(struct acpi_subtable_header *header,
>  	return 0;
>  }
>  
> -void __init srat_parse_regions(u64 addr)
> +void __init srat_parse_regions(paddr_t addr)
>  {
>  	u64 mask;
>  	unsigned int i;
> @@ -441,7 +441,7 @@ void __init srat_parse_regions(u64 addr)
>  	    acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat))
>  		return;
>  
> -	srat_region_mask = pdx_init_mask(addr);
> +	srat_region_mask = pdx_init_mask((u64)addr);
>  	acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
>  			      srat_parse_region, 0);
>  
> @@ -457,7 +457,7 @@ void __init srat_parse_regions(u64 addr)
>  }
>  
>  /* Use the information discovered above to actually set up the nodes. */
> -int __init acpi_scan_nodes(u64 start, u64 end)
> +int __init acpi_scan_nodes(paddr_t start, paddr_t end)
>  {
>  	int i;
>  	nodemask_t all_nodes_parsed;
> @@ -489,7 +489,7 @@ int __init acpi_scan_nodes(u64 start, u64 end)
>  	/* Finally register nodes */
>  	for_each_node_mask(i, all_nodes_parsed)
>  	{
> -		u64 size = nodes[i].end - nodes[i].start;
> +		paddr_t size = nodes[i].end - nodes[i].start;
>  		if ( size == 0 )
>  			printk(KERN_WARNING "SRAT: Node %u has no memory. "
>  			       "BIOS Bug or mis-configured hardware?\n", i);
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 8060cbf3f4..50cfd8e7ef 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -16,7 +16,7 @@ extern cpumask_t     node_to_cpumask[];
>  #define node_to_cpumask(node)    (node_to_cpumask[node])
>  
>  struct node { 
> -	u64 start,end; 
> +	paddr_t start,end;
>  };
>  
>  extern int compute_hash_shift(struct node *nodes, int numnodes,
> @@ -36,7 +36,7 @@ extern void numa_set_node(int cpu, nodeid_t node);
>  extern nodeid_t setup_node(unsigned int pxm);
>  extern void srat_detect_node(int cpu);
>  
> -extern void setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end);
> +extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end);
>  extern nodeid_t apicid_to_node[];
>  extern void init_cpu_to_node(void);
>  
> @@ -73,9 +73,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
>  #define node_end_pfn(nid)       (NODE_DATA(nid)->node_start_pfn + \
>  				 NODE_DATA(nid)->node_spanned_pages)
>  
> -extern int valid_numa_range(u64 start, u64 end, nodeid_t node);
> +extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
>  
> -void srat_parse_regions(u64 addr);
> +void srat_parse_regions(paddr_t addr);
>  extern u8 __node_distance(nodeid_t a, nodeid_t b);
>  unsigned int arch_get_dma_bitsize(void);
>  unsigned int arch_have_default_dmazone(void);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:13:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:13:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194655.346796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYqd-000384-Qi; Fri, 24 Sep 2021 00:13:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194655.346796; Fri, 24 Sep 2021 00:13:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTYqd-00037x-No; Fri, 24 Sep 2021 00:13:51 +0000
Received: by outflank-mailman (input) for mailman id 194655;
 Fri, 24 Sep 2021 00:13:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTYqc-00037p-7J
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:13:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4a2b7818-1ccc-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:13:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E6656604DA;
 Fri, 24 Sep 2021 00:13:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a2b7818-1ccc-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632442428;
	bh=AClTcpI6SS4SJZM02TDSA/EjB0ezS8k/vLR0zqvwqRU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ubl69uLnOmRxDTGQmwtLXp0cqOBO83r8QigFnSR6v0SypAcNp/clbkCbYsvBe+iGA
	 MIGASeRYM24jaux6bBtJfLrNPbODZeBjPsx71YEO9AzuzaKlJ3oJY8+1gfazoCeRoL
	 3D+e2Erq5xWz5Pl5IgNSAwCuxiKYGRE8yv7ML9LN3dM+LvFTzcuAfsawKVS30sYr5M
	 k1pj1c4N6+aywHhHitR397kYk8UvBWv3fnuBGVmQMsiG1o+2j+q6Qd5ndARUwIOr2w
	 kNrHnOZLu3J+VqV/hNVyQv7Z/dAqjshIudLLsFDwpF5smt23+wZ9vl2x7S+Dwk8YWR
	 0qMPJm6H7m3Fw==
Date: Thu, 23 Sep 2021 17:13:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Wei Chen <wei.chen@arm.com>, xen-devel@lists.xenproject.org, 
    julien@xen.org, Bertrand.Marquis@arm.com, jbeulich@suse.com, 
    andrew.cooper3@citrix.com, roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node
 structure
In-Reply-To: <alpine.DEB.2.21.2109231709360.17979@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109231712550.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-8-wei.chen@arm.com> <alpine.DEB.2.21.2109231709360.17979@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

You forgot to add the x86 maintainers in CC to all the patches touching
x86 code in this series. Adding them now but you should probably resend.


On Thu, 23 Sep 2021, Stefano Stabellini wrote:
> On Thu, 23 Sep 2021, Wei Chen wrote:
> > NUMA node structure "struct node" is using u64 as node memory
> > range. In order to make other architectures can reuse this
> > NUMA node relative code, we replace the u64 to paddr_t. And
> > use pfn_to_paddr and paddr_to_pfn to replace explicit shift
> > operations. The relate PRIx64 in print messages have been
> > replaced by PRIpaddr at the same time.
> > 
> > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > ---
> >  xen/arch/x86/numa.c        | 32 +++++++++++++++++---------------
> >  xen/arch/x86/srat.c        | 26 +++++++++++++-------------
> >  xen/include/asm-x86/numa.h |  8 ++++----
> >  3 files changed, 34 insertions(+), 32 deletions(-)
> > 
> > diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> > index 1fabbe8281..6337bbdf31 100644
> > --- a/xen/arch/x86/numa.c
> > +++ b/xen/arch/x86/numa.c
> > @@ -165,12 +165,12 @@ int __init compute_hash_shift(struct node *nodes, int numnodes,
> >      return shift;
> >  }
> >  /* initialize NODE_DATA given nodeid and start/end */
> > -void __init setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end)
> > -{ 
> > +void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end)
> > +{
> >      unsigned long start_pfn, end_pfn;
> >  
> > -    start_pfn = start >> PAGE_SHIFT;
> > -    end_pfn = end >> PAGE_SHIFT;
> > +    start_pfn = paddr_to_pfn(start);
> > +    end_pfn = paddr_to_pfn(end);
> >  
> >      NODE_DATA(nodeid)->node_start_pfn = start_pfn;
> >      NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn;
> > @@ -201,11 +201,12 @@ void __init numa_init_array(void)
> >  static int numa_fake __initdata = 0;
> >  
> >  /* Numa emulation */
> > -static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
> > +static int __init numa_emulation(unsigned long start_pfn,
> > +                                 unsigned long end_pfn)
> 
> Why not changing numa_emulation to take paddr_t too?
> 
> 
> >  {
> >      int i;
> >      struct node nodes[MAX_NUMNODES];
> > -    u64 sz = ((end_pfn - start_pfn)<<PAGE_SHIFT) / numa_fake;
> > +    u64 sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake;
> >  
> >      /* Kludge needed for the hash function */
> >      if ( hweight64(sz) > 1 )
> > @@ -221,9 +222,9 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
> >      memset(&nodes,0,sizeof(nodes));
> >      for ( i = 0; i < numa_fake; i++ )
> >      {
> > -        nodes[i].start = (start_pfn<<PAGE_SHIFT) + i*sz;
> > +        nodes[i].start = pfn_to_paddr(start_pfn) + i*sz;
> >          if ( i == numa_fake - 1 )
> > -            sz = (end_pfn<<PAGE_SHIFT) - nodes[i].start;
> > +            sz = pfn_to_paddr(end_pfn) - nodes[i].start;
> >          nodes[i].end = nodes[i].start + sz;
> >          printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n",
> >                 i,
> > @@ -249,24 +250,26 @@ static int __init numa_emulation(u64 start_pfn, u64 end_pfn)
> >  void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
> 
> same here
> 
> 
> >  { 
> >      int i;
> > +    paddr_t start, end;
> >  
> >  #ifdef CONFIG_NUMA_EMU
> >      if ( numa_fake && !numa_emulation(start_pfn, end_pfn) )
> >          return;
> >  #endif
> >  
> > +    start = pfn_to_paddr(start_pfn);
> > +    end = pfn_to_paddr(end_pfn);
> > +
> >  #ifdef CONFIG_ACPI_NUMA
> > -    if ( !numa_off && !acpi_scan_nodes((u64)start_pfn << PAGE_SHIFT,
> > -         (u64)end_pfn << PAGE_SHIFT) )
> > +    if ( !numa_off && !acpi_scan_nodes(start, end) )
> >          return;
> >  #endif
> >  
> >      printk(KERN_INFO "%s\n",
> >             numa_off ? "NUMA turned off" : "No NUMA configuration found");
> >  
> > -    printk(KERN_INFO "Faking a node at %016"PRIx64"-%016"PRIx64"\n",
> > -           (u64)start_pfn << PAGE_SHIFT,
> > -           (u64)end_pfn << PAGE_SHIFT);
> > +    printk(KERN_INFO "Faking a node at %016"PRIpaddr"-%016"PRIpaddr"\n",
> > +           start, end);
> >      /* setup dummy node covering all memory */
> >      memnode_shift = BITS_PER_LONG - 1;
> >      memnodemap = _memnodemap;
> > @@ -279,8 +282,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
> >      for ( i = 0; i < nr_cpu_ids; i++ )
> >          numa_set_node(i, 0);
> >      cpumask_copy(&node_to_cpumask[0], cpumask_of(0));
> > -    setup_node_bootmem(0, (u64)start_pfn << PAGE_SHIFT,
> > -                    (u64)end_pfn << PAGE_SHIFT);
> > +    setup_node_bootmem(0, start, end);
> >  }
> >  
> >  void numa_add_cpu(int cpu)
> > diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> > index 6b77b98201..7d20d7f222 100644
> > --- a/xen/arch/x86/srat.c
> > +++ b/xen/arch/x86/srat.c
> > @@ -104,7 +104,7 @@ nodeid_t setup_node(unsigned pxm)
> >  	return node;
> >  }
> >  
> > -int valid_numa_range(u64 start, u64 end, nodeid_t node)
> > +int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
> >  {
> >  	int i;
> >  
> > @@ -119,7 +119,7 @@ int valid_numa_range(u64 start, u64 end, nodeid_t node)
> >  	return 0;
> >  }
> >  
> > -static __init int conflicting_memblks(u64 start, u64 end)
> > +static __init int conflicting_memblks(paddr_t start, paddr_t end)
> >  {
> >  	int i;
> >  
> > @@ -135,7 +135,7 @@ static __init int conflicting_memblks(u64 start, u64 end)
> >  	return -1;
> >  }
> >  
> > -static __init void cutoff_node(int i, u64 start, u64 end)
> > +static __init void cutoff_node(int i, paddr_t start, paddr_t end)
> >  {
> >  	struct node *nd = &nodes[i];
> >  	if (nd->start < start) {
> > @@ -275,7 +275,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
> >  void __init
> >  acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> >  {
> > -	u64 start, end;
> > +	paddr_t start, end;
> >  	unsigned pxm;
> >  	nodeid_t node;
> >  	int i;
> > @@ -318,7 +318,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> >  		bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
> >  		                !test_bit(i, memblk_hotplug);
> >  
> > -		printk("%sSRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with itself (%"PRIx64"-%"PRIx64")\n",
> > +		printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
> >  		       mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
> >  		       node_memblk_range[i].start, node_memblk_range[i].end);
> >  		if (mismatch) {
> > @@ -327,7 +327,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> >  		}
> >  	} else {
> >  		printk(KERN_ERR
> > -		       "SRAT: PXM %u (%"PRIx64"-%"PRIx64") overlaps with PXM %u (%"PRIx64"-%"PRIx64")\n",
> > +		       "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
> >  		       pxm, start, end, node_to_pxm(memblk_nodeid[i]),
> >  		       node_memblk_range[i].start, node_memblk_range[i].end);
> >  		bad_srat();
> > @@ -346,7 +346,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> >  				nd->end = end;
> >  		}
> >  	}
> > -	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIx64"-%"PRIx64"%s\n",
> > +	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> >  	       node, pxm, start, end,
> >  	       ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
> >  
> > @@ -369,7 +369,7 @@ static int __init nodes_cover_memory(void)
> >  
> >  	for (i = 0; i < e820.nr_map; i++) {
> >  		int j, found;
> > -		unsigned long long start, end;
> > +		paddr_t start, end;
> >  
> >  		if (e820.map[i].type != E820_RAM) {
> >  			continue;
> > @@ -396,7 +396,7 @@ static int __init nodes_cover_memory(void)
> >  
> >  		if (start < end) {
> >  			printk(KERN_ERR "SRAT: No PXM for e820 range: "
> > -				"%016Lx - %016Lx\n", start, end);
> > +				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
> >  			return 0;
> >  		}
> >  	}
> > @@ -432,7 +432,7 @@ static int __init srat_parse_region(struct acpi_subtable_header *header,
> >  	return 0;
> >  }
> >  
> > -void __init srat_parse_regions(u64 addr)
> > +void __init srat_parse_regions(paddr_t addr)
> >  {
> >  	u64 mask;
> >  	unsigned int i;
> > @@ -441,7 +441,7 @@ void __init srat_parse_regions(u64 addr)
> >  	    acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat))
> >  		return;
> >  
> > -	srat_region_mask = pdx_init_mask(addr);
> > +	srat_region_mask = pdx_init_mask((u64)addr);
> >  	acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
> >  			      srat_parse_region, 0);
> >  
> > @@ -457,7 +457,7 @@ void __init srat_parse_regions(u64 addr)
> >  }
> >  
> >  /* Use the information discovered above to actually set up the nodes. */
> > -int __init acpi_scan_nodes(u64 start, u64 end)
> > +int __init acpi_scan_nodes(paddr_t start, paddr_t end)
> >  {
> >  	int i;
> >  	nodemask_t all_nodes_parsed;
> > @@ -489,7 +489,7 @@ int __init acpi_scan_nodes(u64 start, u64 end)
> >  	/* Finally register nodes */
> >  	for_each_node_mask(i, all_nodes_parsed)
> >  	{
> > -		u64 size = nodes[i].end - nodes[i].start;
> > +		paddr_t size = nodes[i].end - nodes[i].start;
> >  		if ( size == 0 )
> >  			printk(KERN_WARNING "SRAT: Node %u has no memory. "
> >  			       "BIOS Bug or mis-configured hardware?\n", i);
> > diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> > index 8060cbf3f4..50cfd8e7ef 100644
> > --- a/xen/include/asm-x86/numa.h
> > +++ b/xen/include/asm-x86/numa.h
> > @@ -16,7 +16,7 @@ extern cpumask_t     node_to_cpumask[];
> >  #define node_to_cpumask(node)    (node_to_cpumask[node])
> >  
> >  struct node { 
> > -	u64 start,end; 
> > +	paddr_t start,end;
> >  };
> >  
> >  extern int compute_hash_shift(struct node *nodes, int numnodes,
> > @@ -36,7 +36,7 @@ extern void numa_set_node(int cpu, nodeid_t node);
> >  extern nodeid_t setup_node(unsigned int pxm);
> >  extern void srat_detect_node(int cpu);
> >  
> > -extern void setup_node_bootmem(nodeid_t nodeid, u64 start, u64 end);
> > +extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end);
> >  extern nodeid_t apicid_to_node[];
> >  extern void init_cpu_to_node(void);
> >  
> > @@ -73,9 +73,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
> >  #define node_end_pfn(nid)       (NODE_DATA(nid)->node_start_pfn + \
> >  				 NODE_DATA(nid)->node_spanned_pages)
> >  
> > -extern int valid_numa_range(u64 start, u64 end, nodeid_t node);
> > +extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
> >  
> > -void srat_parse_regions(u64 addr);
> > +void srat_parse_regions(paddr_t addr);
> >  extern u8 __node_distance(nodeid_t a, nodeid_t b);
> >  unsigned int arch_get_dma_bitsize(void);
> >  unsigned int arch_have_default_dmazone(void);
> > -- 
> > 2.25.1
> > 
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:25:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:25:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194660.346808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ24-0004u0-VU; Fri, 24 Sep 2021 00:25:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194660.346808; Fri, 24 Sep 2021 00:25:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ24-0004tt-Qk; Fri, 24 Sep 2021 00:25:40 +0000
Received: by outflank-mailman (input) for mailman id 194660;
 Fri, 24 Sep 2021 00:25:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZ23-0004tm-BZ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:25:39 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea2abf0c-4384-4dcc-abff-ea063efa0753;
 Fri, 24 Sep 2021 00:25:38 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 56D3361152;
 Fri, 24 Sep 2021 00:25:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea2abf0c-4384-4dcc-abff-ea063efa0753
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632443137;
	bh=n3TpWy2M3UYxLodJhEJsyb/cJ4+bpfaty6xukw3+XH4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tGwcMt+2tDymh1U1NFAoXpbNzyCJLzGukcC+7VMBKbKF/pYQdERZovd5/j0HYA1nk
	 780k63psgGRWgeXJOFC5gZF821KrUit10XyZlZHu0FhotpF3sZy2sDRMBvYQqqgQkW
	 9Q8Ou/kRpjKsUoHd8v0zh6ifwsM1OH7rXimDXKT/sVitORqoCLb2EwwWMoCiavapnq
	 oe8DtFC5AWdE93M1EfC4xXggbqPMOz8epzHMjmnqeyRvDMBj5HSvm1fUOblR7G8RsZ
	 IAKGSAmgDGDIDF0pc7EZCWDIKo6UJkLs5Hogu4b5F7HSNNON92jTY1fUzxdUaQWv3/
	 lDRzb2sTBjEVw==
Date: Thu, 23 Sep 2021 17:25:36 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <20210923120236.3692135-9-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

CC'ing x86 maintainers

On Thu, 23 Sep 2021, Wei Chen wrote:
> One NUMA node may contain several memory blocks. In current Xen
> code, Xen will maintain a node memory range for each node to cover
> all its memory blocks. But here comes the problem, in the gap of
> one node's two memory blocks, if there are some memory blocks don't
> belong to this node (remote memory blocks). This node's memory range
> will be expanded to cover these remote memory blocks.
> 
> One node's memory range contains othe nodes' memory, this is obviously
> not very reasonable. This means current NUMA code only can support
> node has continous memory blocks. However, on a physical machine, the
> addresses of multiple nodes can be interleaved.
> 
> So in this patch, we add code to detect discontinous memory blocks
> for one node. NUMA initializtion will be failed and error messages
> will be printed when Xen detect such hardware configuration.

At least on ARM, it is not just memory that can be interleaved, but also
MMIO regions. For instance:

node0 bank0 0-0x1000000
MMIO 0x1000000-0x1002000
Hole 0x1002000-0x2000000
node0 bank1 0x2000000-0x3000000

So I am not familiar with the SRAT format, but I think on ARM the check
would look different: we would just look for multiple memory ranges
under a device_type = "memory" node of a NUMA node in device tree.



> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/srat.c | 36 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
> 
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 7d20d7f222..2f08fa4660 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -271,6 +271,36 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
>  		       pxm, pa->apic_id, node);
>  }
>  
> +/*
> + * Check to see if there are other nodes within this node's range.
> + * We just need to check full contains situation. Because overlaps
> + * have been checked before by conflicting_memblks.
> + */
> +static bool __init is_node_memory_continuous(nodeid_t nid,
> +    paddr_t start, paddr_t end)
> +{
> +	nodeid_t i;
> +
> +	struct node *nd = &nodes[nid];
> +	for_each_node_mask(i, memory_nodes_parsed)
> +	{
> +		/* Skip itself */
> +		if (i == nid)
> +			continue;
> +
> +		nd = &nodes[i];
> +		if (start < nd->start && nd->end < end)
> +		{
> +			printk(KERN_ERR
> +			       "NODE %u: (%"PRIpaddr"-%"PRIpaddr") intertwine with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
> +			       nid, start, end, i, nd->start, nd->end);
> +			return false;
> +		}
> +	}
> +
> +	return true;
> +}
> +
>  /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
>  void __init
>  acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> @@ -344,6 +374,12 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  				nd->start = start;
>  			if (nd->end < end)
>  				nd->end = end;
> +
> +			/* Check whether this range contains memory for other nodes */
> +			if (!is_node_memory_continuous(node, nd->start, nd->end)) {
> +				bad_srat();
> +				return;
> +			}
>  		}
>  	}
>  	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:29:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:29:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194665.346819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ5U-0005cE-H2; Fri, 24 Sep 2021 00:29:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194665.346819; Fri, 24 Sep 2021 00:29:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ5U-0005c7-ED; Fri, 24 Sep 2021 00:29:12 +0000
Received: by outflank-mailman (input) for mailman id 194665;
 Fri, 24 Sep 2021 00:29:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZ5T-0005c0-SE
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:29:11 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 772e1a43-bce3-47df-945d-2b7b4497395e;
 Fri, 24 Sep 2021 00:29:11 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2BC9460E05;
 Fri, 24 Sep 2021 00:29:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 772e1a43-bce3-47df-945d-2b7b4497395e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632443350;
	bh=yHeTVy0YzXfruYnPoeScn+anwdCo/tra59HjVLEhVgo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WcczVfSxpQJCEmxyNdaQqG2dkpQWAzF2rQjdFo36JlV29gnRSO9LeNojRcok739a9
	 0598gyWjxupGxlvKlL+wET93p+saP23MfdQyEh2qIT88T9iYWHc9L2oVKrE2GsIFWA
	 qvTYIqShLGbEtrSc5ecmXt3xKvzzJKOdA9F9A29WWavK83h5TacmVjC7wYS3ql8P+9
	 mZvdu1IokGk9K/uJ8HCVUQwRhrdtqSx48k/rA4ks40aK9n+DFZnuL26X2nAAICezuO
	 XMZy0yFRg/ovDo4dP63QIUhCxyv3eie/of3jXe796/JxD6sRwSw4xWR3OM/mQHr0w4
	 g5Cqq/eUqN1zg==
Date: Thu, 23 Sep 2021 17:29:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 09/37] xen/x86: introduce two helpers to access memory
 hotplug end
In-Reply-To: <20210923120236.3692135-10-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231727560.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-10-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers

On Thu, 23 Sep 2021, Wei Chen wrote:
> x86 provides a mem_hotplug to maintain the end of memory hotplug
                            ^ variable

> end address. This variable can be accessed out of mm.c. We want
> some code out of mm.c can be reused by other architectures without
                       ^ so that it can be reused

> memory hotplug ability. So in this patch, we introduce these two
> helpers to replace mem_hotplug direct access. This will give the
> ability to stub these two API.
                            ^ APIs


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/include/asm-x86/mm.h | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
> index cb90527499..af2fc4b0cd 100644
> --- a/xen/include/asm-x86/mm.h
> +++ b/xen/include/asm-x86/mm.h
> @@ -475,6 +475,16 @@ static inline int get_page_and_type(struct page_info *page,
>  
>  extern paddr_t mem_hotplug;
>  
> +static inline void mem_hotplug_update_boundary(paddr_t end)
> +{
> +    mem_hotplug = end;
> +}
> +
> +static inline paddr_t mem_hotplug_boundary(void)
> +{
> +    return mem_hotplug;
> +}
> +
>  /******************************************************************************
>   * With shadow pagetables, the different kinds of address start
>   * to get get confusing.
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:32:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:32:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194669.346830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ8C-0006wk-VD; Fri, 24 Sep 2021 00:32:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194669.346830; Fri, 24 Sep 2021 00:32:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZ8C-0006wd-RZ; Fri, 24 Sep 2021 00:32:00 +0000
Received: by outflank-mailman (input) for mailman id 194669;
 Fri, 24 Sep 2021 00:31:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZ8B-0006wX-F9
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:31:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9676547b-99b9-4c62-b74f-155eed083eec;
 Fri, 24 Sep 2021 00:31:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B1D6F60E52;
 Fri, 24 Sep 2021 00:31:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9676547b-99b9-4c62-b74f-155eed083eec
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632443518;
	bh=glEVFsYw8Rwx27l7Pr4lyUMAEVW7nXCYix2FGmkWSQ8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=I4F6QhPv6s69OOXkc4LoZ/moFXp3J9bBM8DqallY8YZNfIe4Yjldqzp53+M0B5kJ5
	 pVvqWnog6VnqNSYK5zHYX74BFIcSujOrdWaQY/YkZv/vdQUjSRkPxdMRX8U/HOrJyi
	 zK4aR5HzV7BLyBPwEU5pssQvfgrFT/A8pCC3IyLpygO7btDYsbu21cvOUQgVAx1prN
	 oFQUL387JDJ5t2Ie8sRfHsSdNJshLmMaUBYCI69WiS/PlOjCu8NP3N7WU7zlqaCCOQ
	 PaNaPM5Zez1rqBCyrU1DEMOpMnuMHpMd9aBgOV7CYclaboCZi90iSLy4+3v9Pafwpu
	 r3eLo2btlRWyg==
Date: Thu, 23 Sep 2021 17:31:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 10/37] xen/x86: use helpers to access/update
 mem_hotplug
In-Reply-To: <20210923120236.3692135-11-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231730580.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-11-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> We want to abstract code from acpi_numa_memory_affinity_init.
> But mem_hotplug is coupled with x86. In this patch, we use
> helpers to repace mem_hotplug direct accessing. This will
             ^ replace

> allow most code can be common.
                  ^ to be

I think this patch could be merged with the previous patch


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/srat.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 2f08fa4660..3334ede7a5 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -391,8 +391,8 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  	memblk_nodeid[num_node_memblks] = node;
>  	if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) {
>  		__set_bit(num_node_memblks, memblk_hotplug);
> -		if (end > mem_hotplug)
> -			mem_hotplug = end;
> +		if (end > mem_hotplug_boundary())
> +			mem_hotplug_update_boundary(end);
>  	}
>  	num_node_memblks++;
>  }
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:38:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:38:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194675.346841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZEV-0008GF-L9; Fri, 24 Sep 2021 00:38:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194675.346841; Fri, 24 Sep 2021 00:38:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZEV-0008G8-Hk; Fri, 24 Sep 2021 00:38:31 +0000
Received: by outflank-mailman (input) for mailman id 194675;
 Fri, 24 Sep 2021 00:38:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZEU-0008G2-9f
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:38:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bc5f1b80-1ccf-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:38:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E555261211;
 Fri, 24 Sep 2021 00:38:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc5f1b80-1ccf-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632443908;
	bh=n9vFEGEt55OzmkCdWiV+es2O9vy67O1ekPUxnH4h1WA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=m85knBa+igHsH9WbFTNmLtWtAITrSxMROpmx4urQPFOQAy3qSVkyGoBB9ENJijnY7
	 7zwBO5UQngGk3TU6zFSEO3+oKycAPyGZsziGtbP8TZiuP9Oe2giTJo5zWQ3M0ixry4
	 tLh/XoZbc0724wb1BGdAjsbgkN3ANfGz/8WnvWGER4xJC8w8+8J5d3Wnaauvm5ueXo
	 2HhbJxF7vjz17qOYNQVj/sfvibF/loUTe51Ci+i4PXy86th+z+cEBq4FPxnWo0Vqzh
	 RstIXz4FZzFTtW2lZk64AOWLWHJxA1AXPO1qSwEr9lYXmLu1mc7uhh50StZRnuL5qu
	 Z6qyvIBQcRIUw==
Date: Thu, 23 Sep 2021 17:38:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 11/37] xen/x86: abstract neutral code from
 acpi_numa_memory_affinity_init
In-Reply-To: <20210923120236.3692135-12-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231732150.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-12-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> There is some code in acpi_numa_memory_affinity_init to update node
> memory range and update node_memblk_range array. This code is not
> ACPI specific, it can be shared by other NUMA implementation, like
> device tree based NUMA implementation.
> 
> So in this patch, we abstract this memory range and blocks relative
> code to a new function. This will avoid exporting static variables
> like node_memblk_range. And the PXM in neutral code print messages
> have been replaced by NODE, as PXM is ACPI specific.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/srat.c        | 131 +++++++++++++++++++++----------------
>  xen/include/asm-x86/numa.h |   3 +
>  2 files changed, 77 insertions(+), 57 deletions(-)
> 
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 3334ede7a5..18bc6b19bb 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -104,6 +104,14 @@ nodeid_t setup_node(unsigned pxm)
>  	return node;
>  }
>  
> +bool __init numa_memblks_available(void)
> +{
> +	if (num_node_memblks < NR_NODE_MEMBLKS)
> +		return true;
> +
> +	return false;
> +}
> +
>  int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node)
>  {
>  	int i;
> @@ -301,69 +309,35 @@ static bool __init is_node_memory_continuous(nodeid_t nid,
>  	return true;
>  }
>  
> -/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
> -void __init
> -acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> +/* Neutral NUMA memory affinity init function for ACPI and DT */
> +int __init numa_update_node_memblks(nodeid_t node,
> +		paddr_t start, paddr_t size, bool hotplug)
>  {
> -	paddr_t start, end;
> -	unsigned pxm;
> -	nodeid_t node;
> +	paddr_t end = start + size;
>  	int i;
>  
> -	if (srat_disabled())
> -		return;
> -	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
> -		bad_srat();
> -		return;
> -	}
> -	if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
> -		return;
> -
> -	start = ma->base_address;
> -	end = start + ma->length;
> -	/* Supplement the heuristics in l1tf_calculations(). */
> -	l1tf_safe_maddr = max(l1tf_safe_maddr, ROUNDUP(end, PAGE_SIZE));
> -
> -	if (num_node_memblks >= NR_NODE_MEMBLKS)
> -	{
> -		dprintk(XENLOG_WARNING,
> -                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
> -		bad_srat();
> -		return;
> -	}
> -
> -	pxm = ma->proximity_domain;
> -	if (srat_rev < 2)
> -		pxm &= 0xff;
> -	node = setup_node(pxm);
> -	if (node == NUMA_NO_NODE) {
> -		bad_srat();
> -		return;
> -	}
> -	/* It is fine to add this area to the nodes data it will be used later*/
> +	/* It is fine to add this area to the nodes data it will be used later */
>  	i = conflicting_memblks(start, end);
>  	if (i < 0)
>  		/* everything fine */;
>  	else if (memblk_nodeid[i] == node) {
> -		bool mismatch = !(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) !=
> -		                !test_bit(i, memblk_hotplug);
> +		bool mismatch = !hotplug != !test_bit(i, memblk_hotplug);
>  
> -		printk("%sSRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
> -		       mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end,
> +		printk("%sSRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with itself (%"PRIpaddr"-%"PRIpaddr")\n",
> +		       mismatch ? KERN_ERR : KERN_WARNING, node, start, end,
>  		       node_memblk_range[i].start, node_memblk_range[i].end);
>  		if (mismatch) {
> -			bad_srat();
> -			return;
> +			return -1;
>  		}
>  	} else {
>  		printk(KERN_ERR
> -		       "SRAT: PXM %u (%"PRIpaddr"-%"PRIpaddr") overlaps with PXM %u (%"PRIpaddr"-%"PRIpaddr")\n",
> -		       pxm, start, end, node_to_pxm(memblk_nodeid[i]),
> +		       "SRAT: NODE %u (%"PRIpaddr"-%"PRIpaddr") overlaps with NODE %u (%"PRIpaddr"-%"PRIpaddr")\n",
> +		       node, start, end, memblk_nodeid[i],
>  		       node_memblk_range[i].start, node_memblk_range[i].end);
> -		bad_srat();
> -		return;
> +		return -1;
>  	}
> -	if (!(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)) {
> +
> +	if (!hotplug) {
>  		struct node *nd = &nodes[node];
>  
>  		if (!node_test_and_set(node, memory_nodes_parsed)) {
> @@ -375,26 +349,69 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  			if (nd->end < end)
>  				nd->end = end;
>  
> -			/* Check whether this range contains memory for other nodes */
> -			if (!is_node_memory_continuous(node, nd->start, nd->end)) {
> -				bad_srat();
> -				return;
> -			}
> +			if (!is_node_memory_continuous(node, nd->start, nd->end))
> +				return -1;
>  		}
>  	}
> -	printk(KERN_INFO "SRAT: Node %u PXM %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> -	       node, pxm, start, end,
> -	       ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : "");
> +
> +	printk(KERN_INFO "SRAT: Node %u %"PRIpaddr"-%"PRIpaddr"%s\n",
> +	       node, start, end, hotplug ? " (hotplug)" : "");
>  
>  	node_memblk_range[num_node_memblks].start = start;
>  	node_memblk_range[num_node_memblks].end = end;
>  	memblk_nodeid[num_node_memblks] = node;
> -	if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) {
> +	if (hotplug) {
>  		__set_bit(num_node_memblks, memblk_hotplug);
>  		if (end > mem_hotplug_boundary())
>  			mem_hotplug_update_boundary(end);
>  	}
>  	num_node_memblks++;
> +
> +	return 0;
> +}
> +
> +/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
> +void __init
> +acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
> +{
> +	unsigned pxm;
> +	nodeid_t node;
> +	int ret;
> +
> +	if (srat_disabled())
> +		return;
> +	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) {
> +		bad_srat();
> +		return;
> +	}
> +	if (!(ma->flags & ACPI_SRAT_MEM_ENABLED))
> +		return;
> +
> +	/* Supplement the heuristics in l1tf_calculations(). */
> +	l1tf_safe_maddr = max(l1tf_safe_maddr,
> +			ROUNDUP((ma->base_address + ma->length), PAGE_SIZE));
> +
> +	if (!numa_memblks_available())
> +	{
> +		dprintk(XENLOG_WARNING,
> +                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
> +		bad_srat();
> +		return;
> +	}
> +
> +	pxm = ma->proximity_domain;
> +	if (srat_rev < 2)
> +		pxm &= 0xff;
> +	node = setup_node(pxm);
> +	if (node == NUMA_NO_NODE) {
> +		bad_srat();
> +		return;
> +	}
> +
> +	ret = numa_update_node_memblks(node, ma->base_address, ma->length,
> +					ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE);
> +	if (ret != 0)
> +		bad_srat();
>  }
>  
>  /* Sanity check to catch more bad SRATs (they are amazingly common).
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 50cfd8e7ef..5772a70665 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -74,6 +74,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr)
>  				 NODE_DATA(nid)->node_spanned_pages)
>  
>  extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
> +extern bool numa_memblks_available(void);
> +extern int numa_update_node_memblks(nodeid_t node,
> +		paddr_t start, paddr_t size, bool hotplug);
>  
>  void srat_parse_regions(paddr_t addr);
>  extern u8 __node_distance(nodeid_t a, nodeid_t b);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:39:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:39:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194679.346852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZFK-0000Qk-VJ; Fri, 24 Sep 2021 00:39:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194679.346852; Fri, 24 Sep 2021 00:39:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZFK-0000Qd-R9; Fri, 24 Sep 2021 00:39:22 +0000
Received: by outflank-mailman (input) for mailman id 194679;
 Fri, 24 Sep 2021 00:39:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZFJ-0000QP-Cu
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:39:21 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id db18646e-1ccf-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:39:20 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9960061211;
 Fri, 24 Sep 2021 00:39:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db18646e-1ccf-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632443959;
	bh=eMtkefiv+ueAHUaY0PD++Te6w5032p/saixDgl6EPuU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Ykxv//gr3ubWcTEqRFvuvOTv+AtciCSFnmve7tgh2mexgEUMOEfr5/O3bPrB7rRJh
	 RW0IOGpbUS7FlPRe3x0Y8ca7rXWpp02bJEG50lxTAW1Q77v7nu3lv3EN97cpbJt7BD
	 2Ifmj8grszILHU5GYPUB5ZBpTtUODSXXqhf7XOirCw9vA83ByC/14C+G5sCBBE4Tlj
	 HKuXkmurEuxrC3fvBSjetP9okno7W7KwYLvW0pvOBSmldBD323581sGDq9MoXeMnZO
	 W+IOaibN7aUHudQGl+sijE17w2BcbhcxIpIddBMPH5fIynGnIFzbfQ0r9AK0nFaNC8
	 pztttjj/ztYAg==
Date: Thu, 23 Sep 2021 17:39:19 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 12/37] xen/x86: decouple nodes_cover_memory from E820
 map
In-Reply-To: <20210923120236.3692135-13-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231739050.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-13-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> We will reuse nodes_cover_memory for Arm to check its bootmem
> info. So we introduce two arch helpers to get memory map's
> entry number and specified entry's range:
>     arch_get_memory_bank_number
>     arch_get_memory_bank_range
> 
> Depends above two helpers, we make nodes_cover_memory become
> architecture independent. And the only change from an x86
> perspective is the additional checks:
>   !start || !end
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c        | 18 ++++++++++++++++++
>  xen/arch/x86/srat.c        | 11 ++++-------
>  xen/include/asm-x86/numa.h |  3 +++
>  3 files changed, 25 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 6337bbdf31..6bc4ade411 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -378,6 +378,24 @@ unsigned int arch_have_default_dmazone(void)
>      return ( num_online_nodes() > 1 ) ? 1 : 0;
>  }
>  
> +uint32_t __init arch_meminfo_get_nr_bank(void)
> +{
> +	return e820.nr_map;
> +}
> +
> +int __init arch_meminfo_get_ram_bank_range(uint32_t bank,
> +	paddr_t *start, paddr_t *end)
> +{
> +	if (e820.map[bank].type != E820_RAM || !start || !end) {
> +		return -1;
> +	}
> +
> +	*start = e820.map[bank].addr;
> +	*end = e820.map[bank].addr + e820.map[bank].size;
> +
> +	return 0;
> +}
> +
>  static void dump_numa(unsigned char key)
>  {
>      s_time_t now = NOW();
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 18bc6b19bb..aa07a7e975 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -419,17 +419,14 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma)
>  static int __init nodes_cover_memory(void)
>  {
>  	int i;
> +	uint32_t nr_banks = arch_meminfo_get_nr_bank();
>  
> -	for (i = 0; i < e820.nr_map; i++) {
> +	for (i = 0; i < nr_banks; i++) {
>  		int j, found;
>  		paddr_t start, end;
>  
> -		if (e820.map[i].type != E820_RAM) {
> +		if (arch_meminfo_get_ram_bank_range(i, &start, &end))
>  			continue;
> -		}
> -
> -		start = e820.map[i].addr;
> -		end = e820.map[i].addr + e820.map[i].size;
>  
>  		do {
>  			found = 0;
> @@ -448,7 +445,7 @@ static int __init nodes_cover_memory(void)
>  		} while (found && start < end);
>  
>  		if (start < end) {
> -			printk(KERN_ERR "SRAT: No PXM for e820 range: "
> +			printk(KERN_ERR "SRAT: No NODE for memory map range: "
>  				"%"PRIpaddr" - %"PRIpaddr"\n", start, end);
>  			return 0;
>  		}
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 5772a70665..78e044a390 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -82,5 +82,8 @@ void srat_parse_regions(paddr_t addr);
>  extern u8 __node_distance(nodeid_t a, nodeid_t b);
>  unsigned int arch_get_dma_bitsize(void);
>  unsigned int arch_have_default_dmazone(void);
> +extern uint32_t arch_meminfo_get_nr_bank(void);
> +extern int arch_meminfo_get_ram_bank_range(uint32_t bank,
> +    paddr_t *start, paddr_t *end);
>  
>  #endif
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:40:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:40:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194684.346862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGK-0001lR-82; Fri, 24 Sep 2021 00:40:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194684.346862; Fri, 24 Sep 2021 00:40:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGK-0001lK-53; Fri, 24 Sep 2021 00:40:24 +0000
Received: by outflank-mailman (input) for mailman id 194684;
 Fri, 24 Sep 2021 00:40:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZGI-0001kJ-RP
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:40:22 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fd93dfc8-1ccf-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:40:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5D94960F3A;
 Fri, 24 Sep 2021 00:40:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd93dfc8-1ccf-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632444017;
	bh=sEK9rVy9pf6fi5/98YAuKPliIj45WL/ianbNclHQWSw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hCAdwaESoSMKLH7XsGzIrjSF9rZGZyWklOempQImtfuiYDoa4351AdRdnb3K17Fwy
	 nEQQbU1h6mbmCSyPTn+Eb40CKG3Zj3cIPSkxzA6v8BMLtZHeb9UjtGyJdJWhpgyjp9
	 Lyyyc3KRoJUBT0U8ty/VEzbhV2RmJTPzLu14teNPuFP3VikRLxs/Wxyp0j5XqnB1Aj
	 c7gMguSYEGW1wk9slaFf06JZTQLncak4C/O6T6/Xl+hxDJLTKpGq+fnoxqMU0CoePx
	 Jx31V98mG05WXurgZ/IB6ozlGtkR5KsNDCDwRjjLXj+l1yahJcHz6Y2O6qzHgJ++OU
	 PzIQW4fOFlvFA==
Date: Thu, 23 Sep 2021 17:40:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 13/37] xen/x86: decouple processor_nodes_parsed from acpi
 numa functions
In-Reply-To: <20210923120236.3692135-14-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231740070.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-14-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> Xen is using processor_nodes_parsed to record parsed processor nodes
> from ACPI table or other firmware provided resource table. This
> variable is used in ACPI numa functions directly. In follow-up
> patchs, neutral NUMA code will be abstracted and move to other files.
> So in this patch, we introduce numa_set_processor_nodes_parsed helper
> to decouple processor_nodes_parsed from acpi numa functions.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/srat.c        | 9 +++++++--
>  xen/include/asm-x86/numa.h | 1 +
>  2 files changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index aa07a7e975..9276a52138 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -104,6 +104,11 @@ nodeid_t setup_node(unsigned pxm)
>  	return node;
>  }
>  
> +void  __init numa_set_processor_nodes_parsed(nodeid_t node)
> +{
> +	node_set(node, processor_nodes_parsed);
> +}
> +
>  bool __init numa_memblks_available(void)
>  {
>  	if (num_node_memblks < NR_NODE_MEMBLKS)
> @@ -236,7 +241,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa)
>  	}
>  
>  	apicid_to_node[pa->apic_id] = node;
> -	node_set(node, processor_nodes_parsed);
> +	numa_set_processor_nodes_parsed(node);
>  	acpi_numa = 1;
>  
>  	if (opt_acpi_verbose)
> @@ -271,7 +276,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
>  		return;
>  	}
>  	apicid_to_node[pa->apic_id] = node;
> -	node_set(node, processor_nodes_parsed);
> +	numa_set_processor_nodes_parsed(node);
>  	acpi_numa = 1;
>  
>  	if (opt_acpi_verbose)
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 78e044a390..295f875a51 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -77,6 +77,7 @@ extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node);
>  extern bool numa_memblks_available(void);
>  extern int numa_update_node_memblks(nodeid_t node,
>  		paddr_t start, paddr_t size, bool hotplug);
> +extern void numa_set_processor_nodes_parsed(nodeid_t node);
>  
>  void srat_parse_regions(paddr_t addr);
>  extern u8 __node_distance(nodeid_t a, nodeid_t b);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:40:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:40:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194687.346873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGb-0002Em-GB; Fri, 24 Sep 2021 00:40:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194687.346873; Fri, 24 Sep 2021 00:40:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGb-0002Ef-DH; Fri, 24 Sep 2021 00:40:41 +0000
Received: by outflank-mailman (input) for mailman id 194687;
 Fri, 24 Sep 2021 00:40:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZGa-00024Z-4z
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:40:40 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d197e321-b655-4bf5-ba12-363477841df3;
 Fri, 24 Sep 2021 00:40:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C30C761211;
 Fri, 24 Sep 2021 00:40:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d197e321-b655-4bf5-ba12-363477841df3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632444035;
	bh=4VAdznDK5uMsDeS8RiIXv+6D21mqHwaZX6t0QAUGsq8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=J+TeqtBTERWXz2CYuEgLYUdEFU2WIv4rlesECJQoIwRNiGKfuISfNytrHTaYaccjv
	 RioWxOkaW5wrDM76jLNWMcX2jdlqi3z1Xd8QZYT4EccyIW3tYtoZAkoMlntB4KPp0I
	 v+DAVOk6sDG8BfaZkl60croMjWaIGiYara6g90jKtMIVfaoX7orwnYvBzL47S9P2i5
	 Cot7oivoopDv3W/X4hhJmN/SZlRMCvMOdiaqpFegsMFlzH6PgLeFUS824iLnmmMOae
	 qIEjzxXUqEKwZF3moPWvBCBgvI0zWyUeqGm05xl/MhfwL0SauCo1KS6R+JEpFnyaZi
	 +6HZLhQFVMikw==
Date: Thu, 23 Sep 2021 17:40:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 14/37] xen/x86: use name fw_numa to replace acpi_numa
In-Reply-To: <20210923120236.3692135-15-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231740220.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-15-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> Xen is using acpi_numa as a switch for ACPI based NUMA. We want to
> use this switch logic for other firmware based NUMA implementation,
> like device tree based NUMA in follow-up patches. As Xen will never
> use both ACPI and device tree based NUMA at runtime. So I rename
> acpi_numa to a more generic name fw_name. This will also allow to
> have the code mostly common.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c        |  6 +++---
>  xen/arch/x86/setup.c       |  2 +-
>  xen/arch/x86/srat.c        | 10 +++++-----
>  xen/include/asm-x86/acpi.h |  2 +-
>  4 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 6bc4ade411..2ef385ae3f 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -51,11 +51,11 @@ cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly;
>  nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
>  
>  bool numa_off;
> -s8 acpi_numa = 0;
> +s8 fw_numa = 0;
>  
>  int srat_disabled(void)
>  {
> -    return numa_off || acpi_numa < 0;
> +    return numa_off || fw_numa < 0;
>  }
>  
>  /*
> @@ -315,7 +315,7 @@ static __init int numa_setup(const char *opt)
>      else if ( !strncmp(opt,"noacpi",6) )
>      {
>          numa_off = false;
> -        acpi_numa = -1;
> +        fw_numa = -1;
>      }
>  #endif
>      else
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index b101565f14..1a2093b554 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -313,7 +313,7 @@ void srat_detect_node(int cpu)
>      node_set_online(node);
>      numa_set_node(cpu, node);
>  
> -    if ( opt_cpu_info && acpi_numa > 0 )
> +    if ( opt_cpu_info && fw_numa > 0 )
>          printk("CPU %d APIC %d -> Node %d\n", cpu, apicid, node);
>  }
>  
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 9276a52138..4921830f94 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -167,7 +167,7 @@ static __init void bad_srat(void)
>  {
>  	int i;
>  	printk(KERN_ERR "SRAT: SRAT not used.\n");
> -	acpi_numa = -1;
> +	fw_numa = -1;
>  	for (i = 0; i < MAX_LOCAL_APIC; i++)
>  		apicid_to_node[i] = NUMA_NO_NODE;
>  	for (i = 0; i < ARRAY_SIZE(pxm2node); i++)
> @@ -242,7 +242,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa)
>  
>  	apicid_to_node[pa->apic_id] = node;
>  	numa_set_processor_nodes_parsed(node);
> -	acpi_numa = 1;
> +	fw_numa = 1;
>  
>  	if (opt_acpi_verbose)
>  		printk(KERN_INFO "SRAT: PXM %u -> APIC %08x -> Node %u\n",
> @@ -277,7 +277,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa)
>  	}
>  	apicid_to_node[pa->apic_id] = node;
>  	numa_set_processor_nodes_parsed(node);
> -	acpi_numa = 1;
> +	fw_numa = 1;
>  
>  	if (opt_acpi_verbose)
>  		printk(KERN_INFO "SRAT: PXM %u -> APIC %02x -> Node %u\n",
> @@ -492,7 +492,7 @@ void __init srat_parse_regions(paddr_t addr)
>  	u64 mask;
>  	unsigned int i;
>  
> -	if (acpi_disabled || acpi_numa < 0 ||
> +	if (acpi_disabled || fw_numa < 0 ||
>  	    acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat))
>  		return;
>  
> @@ -521,7 +521,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end)
>  	for (i = 0; i < MAX_NUMNODES; i++)
>  		cutoff_node(i, start, end);
>  
> -	if (acpi_numa <= 0)
> +	if (fw_numa <= 0)
>  		return -1;
>  
>  	if (!nodes_cover_memory()) {
> diff --git a/xen/include/asm-x86/acpi.h b/xen/include/asm-x86/acpi.h
> index 7032f3a001..83be71fec3 100644
> --- a/xen/include/asm-x86/acpi.h
> +++ b/xen/include/asm-x86/acpi.h
> @@ -101,7 +101,7 @@ extern unsigned long acpi_wakeup_address;
>  
>  #define ARCH_HAS_POWER_INIT	1
>  
> -extern s8 acpi_numa;
> +extern s8 fw_numa;
>  extern int acpi_scan_nodes(u64 start, u64 end);
>  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:40:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:40:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194689.346885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGl-0002gk-U3; Fri, 24 Sep 2021 00:40:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194689.346885; Fri, 24 Sep 2021 00:40:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGl-0002gU-R0; Fri, 24 Sep 2021 00:40:51 +0000
Received: by outflank-mailman (input) for mailman id 194689;
 Fri, 24 Sep 2021 00:40:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZGk-0002bg-5W
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:40:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3619d34e-7c90-43e6-ae6f-21e23c2bb8c4;
 Fri, 24 Sep 2021 00:40:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9D31461211;
 Fri, 24 Sep 2021 00:40:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3619d34e-7c90-43e6-ae6f-21e23c2bb8c4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632444047;
	bh=KlzvEBlrAj3KfMIRLLDTGuTGCbdiigud5SjmjPEGICQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=eCBrXR515xDmsgPiJGarco3YaQT2yyM1XU865B9VCwyXb/Hbw5X1YIkR5WJE/HbaT
	 sU7JcJ+M3/JxBBl3hCkoUjMubvNZ3saMyTcfHYu8x//AkTVJimssj/6XYHW9Jw4862
	 7PVUzfEIqTh0pylNiRVpjiLTq/sKrbD8dZ/HPtCxihdRWJa5/Dq4YDMRmgRVcLxuhG
	 YlYS/DCYeTMJjoJu7yY4gVJ+XqygQ6gzOK6co2YZ+B/H/jZhKEgVG3kEGGsjFMMMXv
	 bJW6oISPh2ysfC2NnOkot8pM/AcnP/jG9soOCQY/HgmCU5Srvz6td/rr83IDsZ/Rbo
	 B2oOWDADTF7VQ==
Date: Thu, 23 Sep 2021 17:40:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 15/37] xen/x86: rename acpi_scan_nodes to
 numa_scan_nodes
In-Reply-To: <20210923120236.3692135-16-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231740400.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-16-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> The most code in acpi_scan_nodes can be reused by other NUMA
> implementation. Rename acpi_scan_nodes to a more generic name
> numa_scan_nodes, and replace BIOS to Firmware in print message,
> as BIOS is x86 specific name.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c        | 2 +-
>  xen/arch/x86/srat.c        | 4 ++--
>  xen/include/asm-x86/acpi.h | 2 +-
>  3 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 2ef385ae3f..8a4710df39 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -261,7 +261,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
>      end = pfn_to_paddr(end_pfn);
>  
>  #ifdef CONFIG_ACPI_NUMA
> -    if ( !numa_off && !acpi_scan_nodes(start, end) )
> +    if ( !numa_off && !numa_scan_nodes(start, end) )
>          return;
>  #endif
>  
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 4921830f94..0b8b0b0c95 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -512,7 +512,7 @@ void __init srat_parse_regions(paddr_t addr)
>  }
>  
>  /* Use the information discovered above to actually set up the nodes. */
> -int __init acpi_scan_nodes(paddr_t start, paddr_t end)
> +int __init numa_scan_nodes(paddr_t start, paddr_t end)
>  {
>  	int i;
>  	nodemask_t all_nodes_parsed;
> @@ -547,7 +547,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end)
>  		paddr_t size = nodes[i].end - nodes[i].start;
>  		if ( size == 0 )
>  			printk(KERN_WARNING "SRAT: Node %u has no memory. "
> -			       "BIOS Bug or mis-configured hardware?\n", i);
> +			       "Firmware Bug or mis-configured hardware?\n", i);
>  
>  		setup_node_bootmem(i, nodes[i].start, nodes[i].end);
>  	}
> diff --git a/xen/include/asm-x86/acpi.h b/xen/include/asm-x86/acpi.h
> index 83be71fec3..2add971072 100644
> --- a/xen/include/asm-x86/acpi.h
> +++ b/xen/include/asm-x86/acpi.h
> @@ -102,7 +102,7 @@ extern unsigned long acpi_wakeup_address;
>  #define ARCH_HAS_POWER_INIT	1
>  
>  extern s8 fw_numa;
> -extern int acpi_scan_nodes(u64 start, u64 end);
> +extern int numa_scan_nodes(u64 start, u64 end);
>  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
>  
>  extern struct acpi_sleep_info acpi_sinfo;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:41:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:41:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194692.346896 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGy-0003HB-7q; Fri, 24 Sep 2021 00:41:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194692.346896; Fri, 24 Sep 2021 00:41:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZGy-0003Gq-2f; Fri, 24 Sep 2021 00:41:04 +0000
Received: by outflank-mailman (input) for mailman id 194692;
 Fri, 24 Sep 2021 00:41:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZGw-0003Dl-5N
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:41:02 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 16d508e1-1cd0-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:41:01 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8D4E060F3A;
 Fri, 24 Sep 2021 00:41:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16d508e1-1cd0-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632444060;
	bh=d9sS+FOKFjc8Geb60S3J1WOFkjNcNJU0QtUT+GNO7+4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bR7q9ee+DAk96HIm7UxieidBjbKV7oBiQEQj5lwTEFTWeR2oQRnwANkB+Rp/9CmsW
	 P8p3R0LktQH9eu41Q8zo3GaXeCOPN95QvLJ11Nhhq8NmBaCY66dpITXhIkTGUWU19y
	 rdSoDtKQhs5DwbWvBSw2uLw1NfNrYBbMPYLBzDQ7wbMlXfSKClulm8RwofiQPt+Zte
	 jATZ8rrWPaadil3+oygvHOQBxiytLjzpud1AADcHt8+C+mbz+5GyIDMxBcJdDibrxp
	 ZFLGUsIkL6wupwNshdifDJPQD8H5SJMmPXiAliNN1+0IW9yo7fC+9XDGswTs6LpkwQ
	 Jl+ggTX5stG2Q==
Date: Thu, 23 Sep 2021 17:41:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 16/37] xen/x86: export srat_bad to external
In-Reply-To: <20210923120236.3692135-17-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231740510.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-17-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> srat_bad is used when NUMA initialization code scan SRAT failed.
> It will turn fw_numa to disabled status. Its implementation depends
> on NUMA implementation. We want every NUMA implementation to provide
> this function for common initialization code.
> 
> In this patch, we export srat_bad to external. This will allow to
> have the code mostly common.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/srat.c        | 2 +-
>  xen/include/asm-x86/numa.h | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
> index 0b8b0b0c95..94bd5b34da 100644
> --- a/xen/arch/x86/srat.c
> +++ b/xen/arch/x86/srat.c
> @@ -163,7 +163,7 @@ static __init void cutoff_node(int i, paddr_t start, paddr_t end)
>  	}
>  }
>  
> -static __init void bad_srat(void)
> +__init void bad_srat(void)
>  {
>  	int i;
>  	printk(KERN_ERR "SRAT: SRAT not used.\n");
> diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
> index 295f875a51..a5690a7098 100644
> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -32,6 +32,7 @@ extern bool numa_off;
>  
>  
>  extern int srat_disabled(void);
> +extern void bad_srat(void);
>  extern void numa_set_node(int cpu, nodeid_t node);
>  extern nodeid_t setup_node(unsigned int pxm);
>  extern void srat_detect_node(int cpu);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 00:41:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 00:41:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194700.346907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZHF-00042P-FR; Fri, 24 Sep 2021 00:41:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194700.346907; Fri, 24 Sep 2021 00:41:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZHF-00042G-CX; Fri, 24 Sep 2021 00:41:21 +0000
Received: by outflank-mailman (input) for mailman id 194700;
 Fri, 24 Sep 2021 00:41:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZHE-0003v3-US
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 00:41:20 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 21e828e9-1cd0-11ec-ba94-12813bfff9fa;
 Fri, 24 Sep 2021 00:41:20 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6ADC061211;
 Fri, 24 Sep 2021 00:41:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21e828e9-1cd0-11ec-ba94-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632444079;
	bh=4wOFVZiPcSpldLBMYtIEqdq2wNSrUafgFrAxx9/05G4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=V3Ms6Kz2VBOeqOeynJ3+bg3MUoR0OJAdlkUu0Y/OO8d4TVE6tKj38d7FduD/cOQH4
	 /KvzltuCXQ0iS4BYvs16S4n+9bk7GT7FXZ3jegtpBAamLNt8KMhW2wG0WQEMC3VcLM
	 O/ETOrGoYG1E2SNzCH3Y4/uThq0Sw/Xe60039+67uWI/laWKWO4VccvpwsFjIqcaCa
	 /jehIWtJIukoEWpicdPbM1ac9wM/15qtUBVm94JfYdfo1ZGDZMiBDlNyqY6KTNqEUm
	 NEMPEk9YCLzkWCrmNTXeT+8kV1ivBnpqTdaFMFer5B5fuVCeGnf0i0N9ctkLYL/8AV
	 fUNf3E1sc/Zjg==
Date: Thu, 23 Sep 2021 17:41:18 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 17/37] xen/x86: use CONFIG_NUMA to gate numa_scan_nodes
In-Reply-To: <20210923120236.3692135-18-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231741100.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-18-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers


On Thu, 23 Sep 2021, Wei Chen wrote:
> As we have turned numa_scan_nodes to neutral function. If we
> still use CONFIG_ACPI_NUMA in numa_initmem_init to gate
> numa_scan_nodes that doesn't make sense. As CONFIG_ACPI_NUMA
> will be selected by CONFIG_NUMA for x86. So in this patch,
> we replace CONFIG_ACPI_NUMA by CONFIG_NUMA.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/x86/numa.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 8a4710df39..509d2738c0 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -260,7 +260,7 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn)
>      start = pfn_to_paddr(start_pfn);
>      end = pfn_to_paddr(end_pfn);
>  
> -#ifdef CONFIG_ACPI_NUMA
> +#ifdef CONFIG_NUMA
>      if ( !numa_off && !numa_scan_nodes(start, end) )
>          return;
>  #endif
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:07:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:07:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194705.346918 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZgc-0005sy-Jk; Fri, 24 Sep 2021 01:07:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194705.346918; Fri, 24 Sep 2021 01:07:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZgc-0005sr-Gm; Fri, 24 Sep 2021 01:07:34 +0000
Received: by outflank-mailman (input) for mailman id 194705;
 Fri, 24 Sep 2021 01:07:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RRwQ=OO=citrix.com=igor.druzhinin@srs-us1.protection.inumbo.net>)
 id 1mTZgb-0005sl-DC
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:07:33 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc624788-1f63-49e5-ad7f-3b40609bd6b7;
 Fri, 24 Sep 2021 01:07:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc624788-1f63-49e5-ad7f-3b40609bd6b7
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632445650;
  h=from:to:cc:subject:date:message-id:mime-version;
  bh=AQPasy10jnAnd6QRIBKkT16f+WQmWbLReT1xEJ+hZHo=;
  b=YBSKvVoahNmyOO4Xeb1poBLavlE3+YxENSAkoZ44fgzB9CaMx+8mC5Km
   oP3wKlPrBiDMH3zdUC5Wl9MCXoCqTFIzbta9tp9ElH4aXxySKcG6tEj+W
   0A4URqdvoVecGCCcocGdGpD5tv4FF6ASK/golN5KSRftDp2D72bAsW7jg
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 52exaaOPYHh4O0trf5qSy0hCRAMunmEEbdVijJD+xLOGSdpJi/GNIq9SvZBBZ7OuXzZ40jBgzN
 Jm9E6yOjdXdkyQtaHpCqYqZx0zUE5cZv4a+HYlM0zm/VBJSxoahyKRQRdgrfS0y2abrspFawkt
 rJppBBJafpjcUVHbp33QmUKVJWfu10p+9jVaQpi1Pov3tyyq5WaAak10a0J0ESZeoBXtyXinjF
 FEYftvDH4+NY52aHhPiQFiV4AY4z1xnc1R7ixnrbO48gvE9yYEUgAHRZN/NW6lj5DDo5X4Ex7X
 Mj0=
X-SBRS: 5.1
X-MesageID: 53916454
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:yCvSk61yved+6545ovbD5dFxkn2cJEfYwER7XKvMYLTBsI5bp2AHz
 WVOWD+PMqyPNzahctxxaoy/8kNX6pTTxtYxSQc4pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbVh2eaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhutNy1
 Y5ptpmMQBZxZpHXncRFWUlXOnQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Eh8skNo/nJp4NunBmzDfxBvc6W5HTBa7N4Le02R9t3ZEXQ6qDO
 6L1bxJjYAifbAJIBG0uBas8zcuWn0XVSSZh/Qf9Sa0fvDGIkV0ZPKLWGMLcZ9iiVchT2EGCq
 Qru/W70HxUbP9y30iee/zSngeqntTP2XsceGaO18tZugUaP3SoDBRsOT1y5rPKlzEmkVLpix
 1c8o3R06/JorQryE4e7D0bQTGO4UgA0ftlTDrYe6wSxw7uMuzqCA0hcRG5OQYlz3CMpfgAC2
 liMltLvIDVgtryJVH6QnoupQSOO1Ts9djBZOHVaJecRy5y6+thi006WJjp2OPPt1rXI9SfML
 ydmRcTUr44ai9ICn46/9ErO695HjsmUFlNtjuk7s2TM0++YWGJHT9D0gbQ4xawZRGp8crVnl
 CJf8yR5xLpSZaxhbATXHI0w8EiBvp5pygH0j191BIUG/D+w4XOldo04yGggfx01aJxZJmK1P
 BW7VeZtCHl7ZiDCgUhfOd7ZNijX5fK4SYSNug78NLKinaSdhCfYpXozNCZ8LkjmkVQ2kLFXB
 HtoWZzEMJruMow+lGDeb75EidcDn3lirUuOFcGT50n2itK2OS/KIYrpxXPTN4jVGovf+16Lm
 zueXuPXoyhivBrWOHKKrtJDcQ5QfRDWx/ne8qRqSwJKGSI+cElJNhMb6epJlyFNz/wL/gsU1
 hlRgnNl9Wc=
IronPort-HdrOrdr: A9a23:+JPzVq0xhA5PjgWuUHnqlQqjBKQkLtp133Aq2lEZdPRUGvb3qy
 nOpoV96faaslYssR0b9exoW5PwJU80l6QFgrX5VI3KNGKN1VdARLsSiLcKqAeAJ8SRzIFgPN
 9bAspDNOE=
X-IronPort-AV: E=Sophos;i="5.85,318,1624334400"; 
   d="scan'208";a="53916454"
From: Igor Druzhinin <igor.druzhinin@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: <jbeulich@suse.com>, <paul@xen.org>, Igor Druzhinin
	<igor.druzhinin@citrix.com>
Subject: [PATCH] pci: fix handling of PCI bridges with subordinate bus number 0xff
Date: Fri, 24 Sep 2021 02:06:59 +0100
Message-ID: <1632445619-9211-1-git-send-email-igor.druzhinin@citrix.com>
X-Mailer: git-send-email 2.7.4
MIME-Version: 1.0
Content-Type: text/plain

Bus number 0xff is valid according to the PCI spec. Using u8 typed sub_bus
and assigning 0xff to it will result in the following loop getting stuck.

    for ( ; sec_bus <= sub_bus; sec_bus++ ) {...}

Just change its type to u16 the same way that is already handled in
dmar_scope_add_buses().

Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
---
 xen/drivers/passthrough/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index fc4fa2e..48b415c 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -364,7 +364,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     switch ( pdev->type = pdev_type(pseg->nr, bus, devfn) )
     {
         u16 cap;
-        u8 sec_bus, sub_bus;
+        u16 sec_bus, sub_bus;
 
         case DEV_TYPE_PCIe2PCI_BRIDGE:
         case DEV_TYPE_LEGACY_PCI_BRIDGE:
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:15:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:15:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194710.346929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZoJ-0007JR-DW; Fri, 24 Sep 2021 01:15:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194710.346929; Fri, 24 Sep 2021 01:15:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZoJ-0007JK-AR; Fri, 24 Sep 2021 01:15:31 +0000
Received: by outflank-mailman (input) for mailman id 194710;
 Fri, 24 Sep 2021 01:15:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZoI-0007JE-66
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:15:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e798c5e4-1cd4-11ec-ba97-12813bfff9fa;
 Fri, 24 Sep 2021 01:15:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F08D761211;
 Fri, 24 Sep 2021 01:15:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e798c5e4-1cd4-11ec-ba97-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632446128;
	bh=xaEcrMGc5UTMc7WSH9XwcG1s3jyZgAFA8/pBaWvTTQQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tK2bUrVsCTSH/XVTANrXVVvy4oYYn/nCR6q8KNYImfS17eG+k2MNldNJjj/a5A1tA
	 RKiunT3WBaOSaf7exzZwAwFAnFzgvVD9fwelbwi4IKSo0E2dDz5xWACSAREGzEDTdV
	 BHYtF8rEwwUpwCYCG1+F3kcuqPGmJE2umL6Hn77C4wG/BDWU1B6af0M2OGwZVoDtFu
	 IyQAnKLTIzj7jzpV5jQe/SBSpnAGbHSX9W6Gtjvow1bj0oBiJYR1RrHuu3ESYnNZaS
	 +x4vlqujMS0DoChTTuYldLM+CTshWSKlU8mzUjiz/Cxw4piW4cYFFXsUMu3EwsZ5xm
	 GNOYnpMilkEaw==
Date: Thu, 23 Sep 2021 18:15:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
In-Reply-To: <20210923120236.3692135-21-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-21-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Some architectures do not support EFI, but EFI API will be used
> in some common features. Instead of spreading #ifdef ARCH, we
> introduce this Kconfig option to give Xen the ability of stubing
> EFI API for non-EFI supported architectures.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/Kconfig  |  1 +
>  xen/arch/arm/Makefile |  2 +-
>  xen/arch/x86/Kconfig  |  1 +
>  xen/common/Kconfig    | 11 +++++++++++
>  xen/include/xen/efi.h |  4 ++++
>  5 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index ecfa6822e4..865ad83a89 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -6,6 +6,7 @@ config ARM_64
>  	def_bool y
>  	depends on !ARM_32
>  	select 64BIT
> +	select EFI
>  	select HAS_FAST_MULTIPLY
>  
>  config ARM
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b4..ae4efbf76e 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -1,6 +1,6 @@
>  obj-$(CONFIG_ARM_32) += arm32/
>  obj-$(CONFIG_ARM_64) += arm64/
> -obj-$(CONFIG_ARM_64) += efi/
> +obj-$(CONFIG_EFI) += efi/
>  obj-$(CONFIG_ACPI) += acpi/
>  ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y += platforms/
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 28d13b9705..b9ed187f6b 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -10,6 +10,7 @@ config X86
>  	select ALTERNATIVE_CALL
>  	select ARCH_SUPPORTS_INT128
>  	select CORE_PARKING
> +	select EFI
>  	select HAS_ALTERNATIVE
>  	select HAS_COMPAT
>  	select HAS_CPUFREQ
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 9ebb1c239b..f998746a1a 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -11,6 +11,16 @@ config COMPAT
>  config CORE_PARKING
>  	bool
>  
> +config EFI
> +	bool

Without the title the option is not user-selectable (or de-selectable).
So the help message below can never be seen.

Either add a title, e.g.:

bool "EFI support"

Or fully make the option a silent option by removing the help text.



> +	---help---
> +      This option provides support for runtime services provided
> +      by UEFI firmware (such as non-volatile variables, realtime
> +      clock, and platform reset). A UEFI stub is also provided to
> +      allow the kernel to be booted as an EFI application. This
> +      is only useful for kernels that may run on systems that have
> +      UEFI firmware.
> +
>  config GRANT_TABLE
>  	bool "Grant table support" if EXPERT
>  	default y
> @@ -196,6 +206,7 @@ config KEXEC
>  
>  config EFI_SET_VIRTUAL_ADDRESS_MAP
>      bool "EFI: call SetVirtualAddressMap()" if EXPERT
> +    depends on EFI
>      ---help---
>        Call EFI SetVirtualAddressMap() runtime service to setup memory map for
>        further runtime services. According to UEFI spec, it isn't strictly
> diff --git a/xen/include/xen/efi.h b/xen/include/xen/efi.h
> index 94a7e547f9..661a48286a 100644
> --- a/xen/include/xen/efi.h
> +++ b/xen/include/xen/efi.h
> @@ -25,6 +25,8 @@ extern struct efi efi;
>  
>  #ifndef __ASSEMBLY__
>  
> +#ifdef CONFIG_EFI
> +
>  union xenpf_efi_info;
>  union compat_pf_efi_info;
>  
> @@ -45,6 +47,8 @@ int efi_runtime_call(struct xenpf_efi_runtime_call *);
>  int efi_compat_get_info(uint32_t idx, union compat_pf_efi_info *);
>  int efi_compat_runtime_call(struct compat_pf_efi_runtime_call *);
>  
> +#endif /* CONFIG_EFI*/
> +
>  #endif /* !__ASSEMBLY__ */
>  
>  #endif /* __XEN_EFI_H__ */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:23:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:23:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194714.346940 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZvt-0000dg-61; Fri, 24 Sep 2021 01:23:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194714.346940; Fri, 24 Sep 2021 01:23:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZvt-0000dZ-35; Fri, 24 Sep 2021 01:23:21 +0000
Received: by outflank-mailman (input) for mailman id 194714;
 Fri, 24 Sep 2021 01:23:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTZvr-0000dT-Gi
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:23:19 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ff4dcb0c-1cd5-11ec-ba97-12813bfff9fa;
 Fri, 24 Sep 2021 01:23:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5F64F60D42;
 Fri, 24 Sep 2021 01:23:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff4dcb0c-1cd5-11ec-ba97-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632446597;
	bh=v7XjX7PamS/qhqjnlrVsOMjD1o/G9sQUnxpd/xsgtcQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=l4hYsbsGLPQvcbP2YRVPZ9CvXuJ7myq3NeySsNEOaXp/65cyptyBmMqHljO0uX4/P
	 GJIehVEUU6sl6VDanZ6t7PuFW44T78mKwx4mejKni+/QvI5jkcOssGz9X+oEOGJwua
	 1aH7F2HvbsQ2rxyyfbz3lKsawlfTEaWuzIWi5vZXdE78vYpCBhEQA5CkJyYkI1sOnn
	 x3av2AdiWKyVyuJsAoHoC2HVCAOaISbfEvPHEXpMmECAYI4+2oe2GXd3QMfpyXsTGQ
	 dHCH78Ye7V5zugptDhIObj4NLnzqoxdGrR3JeXTg5lV/MGqfK6tNO5nneF3/INjLOh
	 xjJk9gFl6vsLA==
Date: Thu, 23 Sep 2021 18:23:16 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 21/37] xen/arm: Keep memory nodes in dtb for NUMA when
 boot from EFI
In-Reply-To: <20210923120236.3692135-22-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231822090.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-22-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> EFI can get memory map from EFI system table. But EFI system
> table doesn't contain memory NUMA information, EFI depends on
> ACPI SRAT or device tree memory node to parse memory blocks'
> NUMA mapping.
> 
> But in current code, when Xen is booting from EFI, it will
> delete all memory nodes in device tree. So in UEFI + DTB
> boot, we don't have numa-node-id for memory blocks any more.
> 
> So in this patch, we will keep memory nodes in device tree for
> NUMA code to parse memory numa-node-id later.
> 
> As a side effect, if we still parse boot memory information in
> early_scan_node, bootmem.info will calculate memory ranges in
> memory nodes twice. So we have to prevent early_scan_node to
> parse memory nodes in EFI boot.
> 
> As EFI APIs only can be used in Arm64, so we introduced a stub
> API for non-EFI supported Arm32. This will prevent

This last sentence is incomplete.

But aside from that, this patch looks good to me.


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/bootfdt.c      |  8 +++++++-
>  xen/arch/arm/efi/efi-boot.h | 25 -------------------------
>  xen/include/xen/efi.h       |  7 +++++++
>  3 files changed, 14 insertions(+), 26 deletions(-)
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index afaa0e249b..6bc5a465ec 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -11,6 +11,7 @@
>  #include <xen/lib.h>
>  #include <xen/kernel.h>
>  #include <xen/init.h>
> +#include <xen/efi.h>
>  #include <xen/device_tree.h>
>  #include <xen/libfdt/libfdt.h>
>  #include <xen/sort.h>
> @@ -370,7 +371,12 @@ static int __init early_scan_node(const void *fdt,
>  {
>      int rc = 0;
>  
> -    if ( device_tree_node_matches(fdt, node, "memory") )
> +    /*
> +     * If Xen has been booted via UEFI, the memory banks will already
> +     * be populated. So we should skip the parsing.
> +     */
> +    if ( !efi_enabled(EFI_BOOT) &&
> +         device_tree_node_matches(fdt, node, "memory"))
>          rc = process_memory_node(fdt, node, name, depth,
>                                   address_cells, size_cells, &bootinfo.mem);
>      else if ( depth == 1 && !dt_node_cmp(name, "reserved-memory") )
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..d0a9987fa4 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -197,33 +197,8 @@ EFI_STATUS __init fdt_add_uefi_nodes(EFI_SYSTEM_TABLE *sys_table,
>      int status;
>      u32 fdt_val32;
>      u64 fdt_val64;
> -    int prev;
>      int num_rsv;
>  
> -    /*
> -     * Delete any memory nodes present.  The EFI memory map is the only
> -     * memory description provided to Xen.
> -     */
> -    prev = 0;
> -    for (;;)
> -    {
> -        const char *type;
> -        int len;
> -
> -        node = fdt_next_node(fdt, prev, NULL);
> -        if ( node < 0 )
> -            break;
> -
> -        type = fdt_getprop(fdt, node, "device_type", &len);
> -        if ( type && strncmp(type, "memory", len) == 0 )
> -        {
> -            fdt_del_node(fdt, node);
> -            continue;
> -        }
> -
> -        prev = node;
> -    }
> -
>     /*
>      * Delete all memory reserve map entries. When booting via UEFI,
>      * kernel will use the UEFI memory map to find reserved regions.
> diff --git a/xen/include/xen/efi.h b/xen/include/xen/efi.h
> index 661a48286a..b52a4678e9 100644
> --- a/xen/include/xen/efi.h
> +++ b/xen/include/xen/efi.h
> @@ -47,6 +47,13 @@ int efi_runtime_call(struct xenpf_efi_runtime_call *);
>  int efi_compat_get_info(uint32_t idx, union compat_pf_efi_info *);
>  int efi_compat_runtime_call(struct compat_pf_efi_runtime_call *);
>  
> +#else
> +
> +static inline bool efi_enabled(unsigned int feature)
> +{
> +    return false;
> +}
> +
>  #endif /* CONFIG_EFI*/
>  
>  #endif /* !__ASSEMBLY__ */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:24:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:24:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194719.346950 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZxJ-0001Jd-LX; Fri, 24 Sep 2021 01:24:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194719.346950; Fri, 24 Sep 2021 01:24:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTZxJ-0001JW-IV; Fri, 24 Sep 2021 01:24:49 +0000
Received: by outflank-mailman (input) for mailman id 194719;
 Fri, 24 Sep 2021 01:24:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTZxH-0001JQ-OW
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:24:47 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.52]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 51fcd975-3009-4e55-9998-58e079cc999c;
 Fri, 24 Sep 2021 01:24:45 +0000 (UTC)
Received: from AM5PR0202CA0007.eurprd02.prod.outlook.com
 (2603:10a6:203:69::17) by HE1PR0802MB2202.eurprd08.prod.outlook.com
 (2603:10a6:3:c2::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.15; Fri, 24 Sep
 2021 01:24:41 +0000
Received: from AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:69:cafe::41) by AM5PR0202CA0007.outlook.office365.com
 (2603:10a6:203:69::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 01:24:41 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT011.mail.protection.outlook.com (10.152.16.152) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 01:24:39 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Fri, 24 Sep 2021 01:24:37 +0000
Received: from aae5a6f8dc66.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 92C2AE0F-D581-4276-B3DE-1E2469D7BB87.1; 
 Fri, 24 Sep 2021 01:24:26 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id aae5a6f8dc66.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 01:24:26 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB4203.eurprd08.prod.outlook.com (2603:10a6:10:ac::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 01:24:22 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.013; Fri, 24 Sep 2021
 01:24:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51fcd975-3009-4e55-9998-58e079cc999c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AZHIHgGJeRuFiAiVk5L7f10ap5pNVfmoZ7c97Z+Icns=;
 b=TmxCvEkP309xQrPPh/2Gdo9tN9gb+JLIBNMkOtzdjlU0qE6vA29Bt7Qyp7g09iou2FZuyLnsklTkrOVTrL83+P107wbs+akxAdfwnUEXkDdzrAfA0XenZM9lJyGZWlFDqqonwDGAsfSXmPRDbBfPyni32VhUIO7ViAhgt4yPW5s=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DmnfV+L+rjdwRqXHWNLUFWM//+3SrzLX4GW3KiH8pbowHs6pVbQRu0ndJqYkK2RqQzK7gzIkYlCF91lKRMTAe+I9hpenlt/bNEKo3P0ADF/lXu6ICtUXNruUlYz3YcsCgTVmMUCpeHDzAJd5HmMokCCsCv8/oNpd3lMFOCIu9t89WR69aKV9q9/wmvojhEUuVRMIhSbc+sBGscFU/Cg4UDcOLsYBlyjrogt81/2l5/ddBXng00TZJD/QZNQjh+NaEXsfzzomQY2FVHXctEqlBUX53iF8uD7OEuAafgEz3GoDI+IpGC7T7amO6d8H1s/nM12kZ/IETB95hWxx8VCrGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AZHIHgGJeRuFiAiVk5L7f10ap5pNVfmoZ7c97Z+Icns=;
 b=DJGyzNsnO0O1OKZmVKzCmxsUZUBqXK3lQZiKXI4xSijZFLiyGOpNYR0l7Ecm/6LeaKCXOtM93g/4/7GPNDguUrsgtXD+0Ap3TUu2kQFLHwSGeQSVj5MeD8nIOinOc+5B6oWhnYWjP+Fz6Pl6YHkuZ2qoMgGUFiG4o788387kuf6sgnUligWQ5MWGIUk1PDGmuAkOB0pttQbp9ZQ2p+MC7ipL/zdR9KYyMM4sEbjs640lrcSfOu0hRrxBik8jP5NDWjy5Rah28hIBB7tpRO+WTAMsDy46whZyiFxDIHdFIjuGfAmOGUG2Qax9dE6e2fUpevJKHKz5BEkjSrs4078y/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AZHIHgGJeRuFiAiVk5L7f10ap5pNVfmoZ7c97Z+Icns=;
 b=TmxCvEkP309xQrPPh/2Gdo9tN9gb+JLIBNMkOtzdjlU0qE6vA29Bt7Qyp7g09iou2FZuyLnsklTkrOVTrL83+P107wbs+akxAdfwnUEXkDdzrAfA0XenZM9lJyGZWlFDqqonwDGAsfSXmPRDbBfPyni32VhUIO7ViAhgt4yPW5s=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
Thread-Topic: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
Thread-Index: AQHXsHMKuuhjNuZvvk2uWABlJOqp6quySUYAgAAbfeA=
Date: Fri, 24 Sep 2021 01:24:22 +0000
Message-ID:
 <DB9PR08MB68573AC630C8D97EFAB487E19EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-3-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231639110.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231639110.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 998BFE72FC411F41B63267CCCA11578E.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4c703b0e-c55c-4f74-c6f7-08d97efa13eb
x-ms-traffictypediagnostic: DB8PR08MB4203:|HE1PR0802MB2202:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB22026F5A25B2925055E5FC349EA49@HE1PR0802MB2202.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:4502;OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 6i0Ed15mUWeHGcQClg/p3ZzkUFXWhzFNt/dNFDZeWGQe+D4AIjZ++hSGC/eHOVpsjIUUFRJVEnEdz5Hhk5q/oREPA0Sjep+HLum+ejM/CxvpSKEma90PSpX/I/WyVBA3QpG27GTf2apv1psDpJe4m+xgUHJjbG76ndGL+Hq8484PAnR59RmupJYr/X1mCk2ZrTRJr6/zERi0ongN5zaC1qXHAE27NUvCc+2lKUV3yv29ZDBGwksojH2sheR0zgxgWsZyQ9zeOTxqdjorn3u2lPfYi7mh9DSQRXbGNbJgGOpbAkoowmwVx0nhhyoUMMEWzr2SjL+lHZVZVfuv1blhFA4KM5I28evE3cmI8q05ErwC4MoY5QA/cNgnPYpNimOpUzFUrGc8Fma8fay7PfAlynbznaaqbP7+BXIbawHgOekNIy4j4Ff20Zr/MIRDVCazD6heXrJArz3uAG3N6g7NR7eTf1ja3beFBH+4vXkQSs9/qzA5vco/++leEn2gSpxX4pzAl52SLoQ07KgD+VHIaMcgGJQvhRv+cTJN8zt7dcf5C77Xz/OUBuc+ZfRiBx6hFxp9YUqRfmpQfCjX2TEAzAZbWpG0A8BTj914EaLYAHWeHwy4eLqtAdT1KFM6HpdyDP7t3hZUN5wsxLHabTzLj/RIXlS7Iuw6Gij+G9iU7RF3O0n9phd2JvX3MITQNAKeus1wia/JGBJI4p3+bn1K2g==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(6916009)(186003)(71200400001)(66946007)(9686003)(38100700002)(316002)(66446008)(4326008)(33656002)(64756008)(66556008)(8936002)(66476007)(2906002)(26005)(38070700005)(55016002)(86362001)(7696005)(52536014)(53546011)(83380400001)(122000001)(54906003)(5660300002)(76116006)(6506007)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4203
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	892debf5-97e7-47c6-1985-08d97efa0a0d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CD72RJy/DfA/TbMkw99PnLT4kq2Xof3rlnGOpLx6J64HCoKkg5j79pcwPHoVY7VFKapCI038TMR27qp6VnicEMwsSANKBIP72vRMpGzzo4KfFxE6+xBMpj5FfzbddxtQkBUatPE/1cILUek4rbDAhl35dpoQEJWxyJIEKmofJD45nODX5tKe68zBNRd5jUnKe+RVytlqc1weU54abtsgqy+3ulYXQsypzxzmt7TAgDVNMU3+dALbZ7F/ayXUJG+GhjYdUQFKqbg4K+nWFhUJBm8S1/QRAhLQqWXkrs4j2u70Nv3mU6iz67buo7LUpPEpcpHxIq8Ms5DoMQvnf4TsH2e+1S83Sx49N7/3iaFlP8yek4+xMA++QQXBN87rMu+uSRXexAQaAuMK2QYXDcHbRxFfTfGMRrpqjFMB34JKBf7SDHEsYvn91R9v6hnbGCHW3a4qiTHo+KMCuuagQEDJYGgZI9Fuuc471eUHe75kA1yqSRL1oLLGTfjRBd04b46DwSxGPOFjxlTlzaJr9ELY1a6+1ccfCmw7308kD3hdB15A4KT3n+m2xpiWElcdldHuPYQP69pqDBrLzvu3q4sbUjflZDXFs0yT19cN8P4+hbpHIYNgrzlA0pAJxHqdc0avZA8qRGUpdOcNh13VSIj+q8hDyc0Ep8fMp4HFnKIJ18su2nSSClUAiWgJQTjywnazDivsjCjzfwKc9IQpU/BzUg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2906002)(83380400001)(7696005)(55016002)(33656002)(86362001)(26005)(508600001)(336012)(186003)(6506007)(53546011)(9686003)(36860700001)(47076005)(5660300002)(356005)(54906003)(70206006)(8676002)(8936002)(70586007)(4326008)(52536014)(316002)(81166007)(82310400003)(6862004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 01:24:39.0188
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4c703b0e-c55c-4f74-c6f7-08d97efa13eb
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2202

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgNzo0NQ0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4t
ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGll
bkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMDIvMzddIHhlbjogaW50cm9kdWNlIGEgS2NvbmZpZyBv
cHRpb24gdG8gY29uZmlndXJlDQo+IE5VTUEgbm9kZXMgbnVtYmVyDQo+IA0KPiBPbiBUaHUsIDIz
IFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiBDdXJyZW50IE5VTUEgbm9kZXMgbnVtYmVy
IGlzIGEgaGFyZGNvZGUgY29uZmlndXJhdGlvbi4gVGhpcw0KPiA+IGNvbmZpZ3VyYXRpb24gaXMg
ZGlmZmljdWx0IGZvciBhbiBhZG1pbmlzdHJhdG9yIHRvIGNoYW5nZQ0KPiA+IHVubGVzcyBjaGFu
Z2luZyB0aGUgY29kZS4NCj4gPg0KPiA+IFNvIGluIHRoaXMgcGF0Y2gsIHdlIGludHJvZHVjZSB0
aGlzIG5ldyBLY29uZmlnIG9wdGlvbiBmb3INCj4gPiBhZG1pbmlzdHJhdG9ycyB0byBjaGFuZ2Ug
TlVNQSBub2RlcyBudW1iZXIgY29udmVuaWVudGx5Lg0KPiA+IEFsc28gY29uc2lkZXJpbmcgdGhh
dCBub3QgYWxsIGFyY2hpdGVjdHVyZXMgc3VwcG9ydCBOVU1BLA0KPiA+IHRoaXMgS2NvbmZpZyBv
cHRpb24gb25seSBjYW4gYmUgdmlzaWJsZSBvbiBOVU1BIGVuYWJsZWQNCj4gPiBhcmNoaXRlY3R1
cmVzLiBOb24tTlVNQSBzdXBwb3J0ZWQgYXJjaGl0ZWN0dXJlcyBjYW4gc3RpbGwNCj4gPiB1c2Ug
MSBhcyBNQVhfTlVNTk9ERVMuDQo+IA0KPiBUaGlzIGlzIE9LIGJ1dCBJIHRoaW5rIHlvdSBzaG91
bGQgYWxzbyBtZW50aW9uIGluIHRoZSBjb21taXQgbWVzc2FnZQ0KPiB0aGF0IHlvdSBhcmUgdGFr
aW5nIHRoZSBvcHBvcnR1bml0eSB0byByZW1vdmUgTk9ERVNfU0hJRlQgYmVjYXVzZSBpdCBpcw0K
PiBjdXJyZW50bHkgdW51c2VkLg0KPiANCj4gV2l0aCB0aGF0Og0KPiANCj4gUmV2aWV3ZWQtYnk6
IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gDQo+IA0KDQpU
aGFua3MsIEkgd2lsbCB1cGRhdGUgaXQgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+ID4gU2lnbmVkLW9m
Zi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4gLS0tDQo+ID4gIHhlbi9hcmNo
L0tjb25maWcgICAgICAgICAgIHwgMTEgKysrKysrKysrKysNCj4gPiAgeGVuL2luY2x1ZGUvYXNt
LXg4Ni9udW1hLmggfCAgMiAtLQ0KPiA+ICB4ZW4vaW5jbHVkZS94ZW4vbnVtYS5oICAgICB8IDEw
ICsrKysrLS0tLS0NCj4gPiAgMyBmaWxlcyBjaGFuZ2VkLCAxNiBpbnNlcnRpb25zKCspLCA3IGRl
bGV0aW9ucygtKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL0tjb25maWcgYi94ZW4v
YXJjaC9LY29uZmlnDQo+ID4gaW5kZXggZjE2ZWIwZGY0My4uOGEyMGRhNjdlZCAxMDA2NDQNCj4g
PiAtLS0gYS94ZW4vYXJjaC9LY29uZmlnDQo+ID4gKysrIGIveGVuL2FyY2gvS2NvbmZpZw0KPiA+
IEBAIC0xNywzICsxNywxNCBAQCBjb25maWcgTlJfQ1BVUw0KPiA+ICAJICBGb3IgQ1BVIGNvcmVz
IHdoaWNoIHN1cHBvcnQgU2ltdWx0YW5lb3VzIE11bHRpLVRocmVhZGluZyBvcg0KPiBzaW1pbGFy
DQo+ID4gIAkgIHRlY2hub2xvZ2llcywgdGhpcyB0aGUgbnVtYmVyIG9mIGxvZ2ljYWwgdGhyZWFk
cyB3aGljaCBYZW4gd2lsbA0KPiA+ICAJICBzdXBwb3J0Lg0KPiA+ICsNCj4gPiArY29uZmlnIE5S
X05VTUFfTk9ERVMNCj4gPiArCWludCAiTWF4aW11bSBudW1iZXIgb2YgTlVNQSBub2RlcyBzdXBw
b3J0ZWQiDQo+ID4gKwlyYW5nZSAxIDQwOTUNCj4gPiArCWRlZmF1bHQgIjY0Ig0KPiA+ICsJZGVw
ZW5kcyBvbiBOVU1BDQo+ID4gKwloZWxwDQo+ID4gKwkgIENvbnRyb2xzIHRoZSBidWlsZC10aW1l
IHNpemUgb2YgdmFyaW91cyBhcnJheXMgYW5kIGJpdG1hcHMNCj4gPiArCSAgYXNzb2NpYXRlZCB3
aXRoIG11bHRpcGxlLW5vZGVzIG1hbmFnZW1lbnQuIEl0IGlzIHRoZSB1cHBlciBib3VuZA0KPiBv
Zg0KPiA+ICsJICB0aGUgbnVtYmVyIG9mIE5VTUEgbm9kZXMgdGhlIHNjaGVkdWxlciwgbWVtb3J5
IGFsbG9jYXRpb24gYW5kDQo+IG90aGVyDQo+ID4gKwkgIE5VTUEtYXdhcmUgY29tcG9uZW50cyBj
YW4gaGFuZGxlLg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9hc20teDg2L251bWEuaCBi
L3hlbi9pbmNsdWRlL2FzbS14ODYvbnVtYS5oDQo+ID4gaW5kZXggYmFkYTJjMGJiOS4uM2NmMjZj
MmRlZiAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L251bWEuaA0KPiA+ICsr
KyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvbnVtYS5oDQo+ID4gQEAgLTMsOCArMyw2IEBADQo+ID4N
Cj4gPiAgI2luY2x1ZGUgPHhlbi9jcHVtYXNrLmg+DQo+ID4NCj4gPiAtI2RlZmluZSBOT0RFU19T
SElGVCA2DQo+ID4gLQ0KPiA+ICB0eXBlZGVmIHU4IG5vZGVpZF90Ow0KPiA+DQo+ID4gIGV4dGVy
biBpbnQgc3JhdF9yZXY7DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL3hlbi9udW1hLmgg
Yi94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+ID4gaW5kZXggN2FlZjFhODhkYy4uNTI5NTBhMzE1
MCAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+ID4gKysrIGIveGVu
L2luY2x1ZGUveGVuL251bWEuaA0KPiA+IEBAIC0zLDE0ICszLDE0IEBADQo+ID4NCj4gPiAgI2lu
Y2x1ZGUgPGFzbS9udW1hLmg+DQo+ID4NCj4gPiAtI2lmbmRlZiBOT0RFU19TSElGVA0KPiA+IC0j
ZGVmaW5lIE5PREVTX1NISUZUICAgICAwDQo+ID4gLSNlbmRpZg0KPiA+IC0NCj4gPiAgI2RlZmlu
ZSBOVU1BX05PX05PREUgICAgIDB4RkYNCj4gPiAgI2RlZmluZSBOVU1BX05PX0RJU1RBTkNFIDB4
RkYNCj4gPg0KPiA+IC0jZGVmaW5lIE1BWF9OVU1OT0RFUyAgICAoMSA8PCBOT0RFU19TSElGVCkN
Cj4gPiArI2lmZGVmIENPTkZJR19OUl9OVU1BX05PREVTDQo+ID4gKyNkZWZpbmUgTUFYX05VTU5P
REVTIENPTkZJR19OUl9OVU1BX05PREVTDQo+ID4gKyNlbHNlDQo+ID4gKyNkZWZpbmUgTUFYX05V
TU5PREVTICAgIDENCj4gPiArI2VuZGlmDQo+ID4NCj4gPiAgI2RlZmluZSB2Y3B1X3RvX25vZGUo
dikgKGNwdV90b19ub2RlKCh2KS0+cHJvY2Vzc29yKSkNCj4gPg0KPiA+IC0tDQo+ID4gMi4yNS4x
DQo+ID4NCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:34:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:34:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194724.346961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTa6k-0002oC-Ki; Fri, 24 Sep 2021 01:34:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194724.346961; Fri, 24 Sep 2021 01:34:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTa6k-0002o5-Hc; Fri, 24 Sep 2021 01:34:34 +0000
Received: by outflank-mailman (input) for mailman id 194724;
 Fri, 24 Sep 2021 01:34:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTa6j-0002nz-Lp
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:34:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7ba2ad18-6209-4b54-b2ca-c184f914a455;
 Fri, 24 Sep 2021 01:34:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E0CB561211;
 Fri, 24 Sep 2021 01:34:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ba2ad18-6209-4b54-b2ca-c184f914a455
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632447272;
	bh=FQN+nMr8/U87vCJiePijUNnGt4KoccfGqtNT+UZamTU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XX6zcmJ1vetRjtZFkj2n/AQNIgB8cvQ7zPougum05LvEvKuZ9vXAo+yy0IDenFUrH
	 BxJzxAq0o2MyxSZnWSQUqOItdDzRvCLFGdG3+6i892+/O2l9ltA8ySGf208Z6ESBNx
	 WmLfGq4qxwDhGUgGQeQMgLVi9hJIKA4fmDSTBQjWRhQhYY6D+IpLBwHrQq5g8v/MvT
	 xzSw6QNJEQ3pXFnkh6OyQ+RUUGp1jn2tiva37bSjZ0Xw2ivGr68ue/4cDTcM/BySLz
	 dPkVTexNoceZI5dbYmNzIHbA8rGRkeZdzcyzNTLaDueMzYxLUaF0v/hz4thg8rH5bw
	 ly/wRmvN+kA4A==
Date: Thu, 23 Sep 2021 18:34:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
In-Reply-To: <20210923120236.3692135-23-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> As a memory range described in device tree cannot be split across
> multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS in
> arch header.

This statement is true but what is the goal of this patch? Is it to
reduce code size and memory consumption?

I am asking because NR_MEM_BANKS is 128 and
NR_NODE_MEMBLKS=2*MAX_NUMNODES which is 64 by default so again
NR_NODE_MEMBLKS is 128 before this patch.

In other words, this patch alone doesn't make any difference; at least
doesn't make any difference unless CONFIG_NR_NUMA_NODES is increased.

So, is the goal to reduce memory usage when CONFIG_NR_NUMA_NODES is
higher than 64?


> And keep default NR_NODE_MEMBLKS in common header
> for those architectures NUMA is disabled.

This last sentence is not accurate: on x86 NUMA is enabled and
NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h (there is no
x86 definition of it)


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/include/asm-arm/numa.h | 8 +++++++-
>  xen/include/xen/numa.h     | 2 ++
>  2 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 8f1c67e3eb..21569e634b 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -3,9 +3,15 @@
>  
>  #include <xen/mm.h>
>  
> +#include <asm/setup.h>
> +
>  typedef u8 nodeid_t;
>  
> -#ifndef CONFIG_NUMA
> +#ifdef CONFIG_NUMA
> +
> +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> +
> +#else
>  
>  /* Fake one node for now. See also node_online_map. */
>  #define cpu_to_node(cpu) 0
> diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
> index 1978e2be1b..1731e1cc6b 100644
> --- a/xen/include/xen/numa.h
> +++ b/xen/include/xen/numa.h
> @@ -12,7 +12,9 @@
>  #define MAX_NUMNODES    1
>  #endif
>  
> +#ifndef NR_NODE_MEMBLKS
>  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> +#endif
>  
>  #define vcpu_to_node(v) (cpu_to_node((v)->processor))
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:46:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:46:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194732.346973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaIc-0004tG-Pu; Fri, 24 Sep 2021 01:46:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194732.346973; Fri, 24 Sep 2021 01:46:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaIc-0004t9-Md; Fri, 24 Sep 2021 01:46:50 +0000
Received: by outflank-mailman (input) for mailman id 194732;
 Fri, 24 Sep 2021 01:46:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTaIb-0004t3-BM
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:46:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 213c8d99-62c5-40dd-9b10-afd71bbf6e99;
 Fri, 24 Sep 2021 01:46:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5B08D60E54;
 Fri, 24 Sep 2021 01:46:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 213c8d99-62c5-40dd-9b10-afd71bbf6e99
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632448007;
	bh=Q4mXo7T9EFe41mvEVyeat151K0Zw1kVacXSGaj5zNYI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OcEjokzaSIkm3CtKitKDV5K7bPGDKQAMg3RuBJ16FspBalFK0pw5xjQfGxoatMsZD
	 WxehN4KESDY5Bff9tnrgdtSybokfoOMsWqQ4P8Ns11AJhOgES5nemp49G1OTjGBRPd
	 64iuDysHwRNItRKDXEofdhDQ/WU1MDlPEsxuvrTAizrpv1kwkP5JHhuqR4icJOAuZq
	 VwFsbt21ttq01o68zdaAkCoacPfuMKMS8XQTjmn8Bj0aCESf7W2qSKAxcMEJoNPTdT
	 4iVG+3OrAl1cF5xDLqpsUH4+MD/RiCsvRTifj1mMFCwMDB0N20mYEkSeJTR113Jq07
	 IyZVpkg0RvjkA==
Date: Thu, 23 Sep 2021 18:46:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
In-Reply-To: <20210923120236.3692135-24-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231843530.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-24-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> We will parse NUMA nodes distances from device tree or ACPI
> table. So we need a matrix to record the distances between
> any two nodes we parsed. Accordingly, we provide this
> node_set_distance API for device tree or ACPI table parsers
> to set the distance for any two nodes in this patch.
> When NUMA initialization failed, __node_distance will return
> NUMA_REMOTE_DISTANCE, this will help us avoid doing rollback
> for distance maxtrix when NUMA initialization failed.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/Makefile      |  1 +
>  xen/arch/arm/numa.c        | 69 ++++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/numa.h | 13 +++++++
>  3 files changed, 83 insertions(+)
>  create mode 100644 xen/arch/arm/numa.c
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index ae4efbf76e..41ca311b6b 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -35,6 +35,7 @@ obj-$(CONFIG_LIVEPATCH) += livepatch.o
>  obj-y += mem_access.o
>  obj-y += mm.o
>  obj-y += monitor.o
> +obj-$(CONFIG_NUMA) += numa.o
>  obj-y += p2m.o
>  obj-y += percpu.o
>  obj-y += platform.o
> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> new file mode 100644
> index 0000000000..3f08870d69
> --- /dev/null
> +++ b/xen/arch/arm/numa.c
> @@ -0,0 +1,69 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Arm Architecture support layer for NUMA.
> + *
> + * Copyright (C) 2021 Arm Ltd
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +#include <xen/init.h>
> +#include <xen/numa.h>
> +
> +static uint8_t __read_mostly
> +node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
> +    { 0 }
> +};
> +
> +void __init numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance)
> +{
> +    if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
> +    {
> +        printk(KERN_WARNING
> +               "NUMA: invalid nodes: from=%"PRIu8" to=%"PRIu8" MAX=%"PRIu8"\n",
> +               from, to, MAX_NUMNODES);
> +        return;
> +    }
> +
> +    /* NUMA defines 0xff as an unreachable node and 0-9 are undefined */
> +    if ( distance >= NUMA_NO_DISTANCE ||
> +        (distance >= NUMA_DISTANCE_UDF_MIN &&
> +         distance <= NUMA_DISTANCE_UDF_MAX) ||
> +        (from == to && distance != NUMA_LOCAL_DISTANCE) )
> +    {
> +        printk(KERN_WARNING
> +               "NUMA: invalid distance: from=%"PRIu8" to=%"PRIu8" distance=%"PRIu32"\n",
> +               from, to, distance);
> +        return;
> +    }
> +
> +    node_distance_map[from][to] = distance;
> +}
> +
> +uint8_t __node_distance(nodeid_t from, nodeid_t to)
> +{
> +    /* When NUMA is off, any distance will be treated as remote. */
> +    if ( srat_disabled() )

Given that this is ARM specific code and specific to ACPI, I don't think
we should have any call to something called "srat_disabled".

I suggest to either rename srat_disabled to numa_distance_disabled.

Other than that, this patch looks OK to me.


> +        return NUMA_REMOTE_DISTANCE;
> +
> +    /*
> +     * Check whether the nodes are in the matrix range.
> +     * When any node is out of range, except from and to nodes are the
> +     * same, we treat them as unreachable (return 0xFF)
> +     */
> +    if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
> +        return from == to ? NUMA_LOCAL_DISTANCE : NUMA_NO_DISTANCE;
> +
> +    return node_distance_map[from][to];
> +}
> +EXPORT_SYMBOL(__node_distance);
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 21569e634b..758eafeb05 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -9,8 +9,21 @@ typedef u8 nodeid_t;
>  
>  #ifdef CONFIG_NUMA
>  
> +/*
> + * In ACPI spec, 0-9 are the reserved values for node distance,
> + * 10 indicates local node distance, 20 indicates remote node
> + * distance. Set node distance map in device tree will follow
> + * the ACPI's definition.
> + */
> +#define NUMA_DISTANCE_UDF_MIN   0
> +#define NUMA_DISTANCE_UDF_MAX   9
> +#define NUMA_LOCAL_DISTANCE     10
> +#define NUMA_REMOTE_DISTANCE    20
> +
>  #define NR_NODE_MEMBLKS NR_MEM_BANKS
>  
> +extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
> +
>  #else
>  
>  /* Fake one node for now. See also node_online_map. */
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 01:50:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 01:50:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194737.346984 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaMY-0006bK-Ax; Fri, 24 Sep 2021 01:50:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194737.346984; Fri, 24 Sep 2021 01:50:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaMY-0006bD-7P; Fri, 24 Sep 2021 01:50:54 +0000
Received: by outflank-mailman (input) for mailman id 194737;
 Fri, 24 Sep 2021 01:50:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTaMW-0006b6-VL
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 01:50:53 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.82]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d8c91eb0-1cd9-11ec-ba97-12813bfff9fa;
 Fri, 24 Sep 2021 01:50:51 +0000 (UTC)
Received: from AM6PR0202CA0070.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::47) by AM6PR08MB4519.eurprd08.prod.outlook.com
 (2603:10a6:20b:74::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 01:50:42 +0000
Received: from VE1EUR03FT046.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:3a:cafe::11) by AM6PR0202CA0070.outlook.office365.com
 (2603:10a6:20b:3a::47) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Fri, 24 Sep 2021 01:50:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT046.mail.protection.outlook.com (10.152.19.226) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 01:50:41 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Fri, 24 Sep 2021 01:50:41 +0000
Received: from 78c69ab509bc.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 D6E32DF7-C28D-4F9B-A04A-FD16B412C41B.1; 
 Fri, 24 Sep 2021 01:50:30 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 78c69ab509bc.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 01:50:30 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB6091.eurprd08.prod.outlook.com (2603:10a6:10:1f4::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 01:50:27 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.013; Fri, 24 Sep 2021
 01:50:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8c91eb0-1cd9-11ec-ba97-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LAJTHXmx2Vg3cZkJVMmD+DJrwl2pCfGVpxRJ5AcTaaA=;
 b=0i/iF7FkpTAwBH269U7vR/uDf7g+eEdICjMcet33ErhorIDjY2khYLPKmCeYfqWqJmRiuSx59CRwjtppBzBaSdpJcPgcyL/DvFw7XsapTLVtYOuxX55PwS0ri+z8q/FL4C9DuCpDEoikHQL4RJaDQ2cWMXEcfCVpuIskogXC83w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dvpGZhy9AQRA8f4HO1smWWmnKkmGanVHH2IqfK+GeRERDwBQWqv2J7LBiFFcJMwxpaAUOd8FenMN84UxJouWpQmhnuI+vZUoVoFWSHA2xcCJbjTKFQ6WIA3VxpgJWaw37SG7WHPpUzl7NGUoYNGLUclbaw1m6UVx+mOdp3kDIuBF8DC0pAU/kEkVJSs38wgw3vQ2MR9QLi2NinIIFmOib3TGFr5gXWwJEutUzNY6Lyv9+w7MR3uqWp7/xILd/LS57w7RcrVBKHBao3FHd9lcyEx9dPrGWIMDzNDiW9lLt97sMWL0qkJIr5xYOOudpQYl4Nqwv2facs2S5PnXvv2i6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LAJTHXmx2Vg3cZkJVMmD+DJrwl2pCfGVpxRJ5AcTaaA=;
 b=VqZqOoTU3F5magoW4NBiMGLeqTEgNDet6i5fKKROQblQ4HVbq7DRAI1muF+kYnBEd5xU6SlqHO/Eld3ya1D4OGKHwkg/s+Wr09avq99Kxrf+m0+lqWWtXgTdsU6vYHysPIhOY2BbezxY3QNsDUQRuB23Lm2pmpYwwf/ZDBkjkQb1/RxjuLQu/1WkuZA9L5HdN9c7lYDMKUBpNgofGtXlu2jQCJbsmv9xzyybawLck0PuH7fA3TJBpemz01zh74/KpYdqsCnaZkhbzLb8nvc2BoqjgUFOU8t/arjHdDWLwUs/DR5JQTGXf35vH4irFlVj7tnmpJbvmzOIgtBR+htNCA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LAJTHXmx2Vg3cZkJVMmD+DJrwl2pCfGVpxRJ5AcTaaA=;
 b=0i/iF7FkpTAwBH269U7vR/uDf7g+eEdICjMcet33ErhorIDjY2khYLPKmCeYfqWqJmRiuSx59CRwjtppBzBaSdpJcPgcyL/DvFw7XsapTLVtYOuxX55PwS0ri+z8q/FL4C9DuCpDEoikHQL4RJaDQ2cWMXEcfCVpuIskogXC83w=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 04/37] xen: introduce an arch helper for default dma zone
 status
Thread-Topic: [PATCH 04/37] xen: introduce an arch helper for default dma zone
 status
Thread-Index: AQHXsHMGx8xGhif/FUCDSa9NS/fSLquyTCAAgAAfquA=
Date: Fri, 24 Sep 2021 01:50:26 +0000
Message-ID:
 <DB9PR08MB6857017D740D8408A2C30CB39EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-5-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231652340.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231652340.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 79EB525EA391BE4182C09CC50462CB94.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ff0d81f0-f066-40a4-dbb2-08d97efdb74f
x-ms-traffictypediagnostic: DBBPR08MB6091:|AM6PR08MB4519:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4519349540103D92CEB2B0CA9EA49@AM6PR08MB4519.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Yl/yCqNk6YPoIZ4ARbmcLUmmDleGtEtP3D2atw1JMy5BRRPHBFwXugXzsCQLTXmogOhVBGwi79W+yOCxro7G9hy3x/dagulBgv4p3rmS0TBkw7PX+kM7XsRhMn+0sBpokEtDzaiY51RVi7YwW2pdwSYJZjByk8mp1QuuRTO0P01VDr6o5ZDKp7+iuB+Z2OVKbCFiz4/aoOjYntI1ZMxG5RNLL1gs1afLj78whkJIRtXEnz+nO8BWo7Cfpbk6hLqVvFA6OENC7koqcJVYVxLb0jCP6y2k8ZT+zryfJ6NMOZyL1RKvNLiyamyeKaRWyjUuImo0WriGzV/qAogc812zXtVR9wlODzEWjnYKu9jn3HzRAjfupz6N3ZTS5U18SdtoGC9jSEarhihXaod5sqS92GyuxsxnxEY3zU45J1oPWKgHWFa4ZNQMn68ZE6JpjcbFXOj7No0cHI5IVTOT1g/Jg8wVru25cZ9Ru7MJfZawELf6Jt/E8wtYe/2fbXHTFujXVJj/FsCAsl+cQCVZdNUuaVS37mojVrZ8hvtq7LejCcHkGOCd1CC2mVyea2U8Co40cU0rvbdxvzXs+hMVoTovZ2PInnmvKhQHKDsxI0j235wWs2VfaViDksVhSVKb8Sm7fwK3pdYwAY/58p4fLz2IeaUXK2/emf+0k+6mfH4gPaO9yTm+JApMHGix1HoqgVIBW9ik8ngX8irv4r+hf4/8ww==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(6506007)(53546011)(54906003)(66556008)(83380400001)(8676002)(9686003)(66446008)(7696005)(4326008)(86362001)(2906002)(38070700005)(64756008)(316002)(8936002)(186003)(66946007)(66476007)(71200400001)(55016002)(52536014)(33656002)(76116006)(122000001)(26005)(508600001)(38100700002)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6091
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d02deeda-2261-4467-cb21-08d97efdae98
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OmW231Hsuo0YBrCAI3WB7WrpReSr9OuEy/EgHe3M5qHrgd4la4sgO7y0HRbbeRuGI8y2Hu6ouK8irXdBdWAfJDHI815Q97GNpVMnj8h1Cz73XIuql7EJCKZkT7zQvuHK5OrwWOY2Phlhl9JCuMEtbVTYfpNn0XXisfiwxLOmtNRZu7xsO3F/6qCwtiMpfOwcowrvySJszIPD/2Afs+fX9ncYM2nlyoBMV4NfLyhBgv+ONLzSyRLsJKgzAOu2hC6STsBgjZ15YtRttaE7qmBtYegOQfq73IyODyjp6fhX6gV08QJgIDVrwQim5uLmlU/WW5nRw+2YZJvcn3M7IJyNSGjatr800Nv0Eh3b7VJG2RpZGe+rIcx3e53zgYmlw9QhfagB/wzRlqBbuP5qkTJfhXJlO8ozFrvGLu6zH8BWNVAFu4VC8EjoZTBrO/oS2xY8DVit2GBV7j+iIOnmRKt+5Jue6ozA8LMJfAFXVk6swp59TC4miqX+xe1ZEaikhKn+LTx9+HLbXPYGYZU3mrEFlAm3La8MEk0OamViWauGnOXt7RJF12RfI0f0fG7RFNAkWCPU2zXp+nlBvIbDA9YrmE3WdZsDjbtN8zbwsA4wTYbuvtH4Y7UujREjaVa4wbuEGMP4gXG6xkccX0kxDM6+QEMw9zDvGvr881dc+UmmvBfqqnAha2e9SRd8iWhwbuYvOccUV1doJ9K/h7829eMW7w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2906002)(52536014)(86362001)(5660300002)(316002)(54906003)(508600001)(4326008)(6862004)(33656002)(47076005)(36860700001)(82310400003)(55016002)(83380400001)(9686003)(81166007)(356005)(26005)(8676002)(8936002)(53546011)(6506007)(70206006)(70586007)(186003)(336012)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 01:50:41.5476
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ff0d81f0-f066-40a4-dbb2-08d97efdb74f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT046.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4519

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgNzo1Ng0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4t
ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGll
bkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMDQvMzddIHhlbjogaW50cm9kdWNlIGFuIGFyY2ggaGVs
cGVyIGZvciBkZWZhdWx0IGRtYQ0KPiB6b25lIHN0YXR1cw0KPiANCj4gT24gVGh1LCAyMyBTZXAg
MjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gSW4gY3VycmVudCBjb2RlLCB3aGVuIFhlbiBpcyBy
dW5uaW5nIGluIGEgbXVsdGlwbGUgbm9kZXMgTlVNQQ0KPiA+IHN5c3RlbSwgaXQgd2lsbCBzZXQg
ZG1hX2JpdHNpemUgaW4gZW5kX2Jvb3RfYWxsb2NhdG9yIHRvIHJlc2VydmUNCj4gPiBzb21lIGxv
dyBhZGRyZXNzIG1lbW9yeSBmb3IgRE1BLg0KPiA+DQo+ID4gVGhlcmUgYXJlIHNvbWUgeDg2IGlt
cGxpY2F0aW9ucyBpbiBjdXJyZW50IGltcGxlbWVudGF0aW9uLiBCZWN1YXNlDQo+ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIF4gdGhlICAgICAgICAgICAgICAgICAgICBeQmVj
YXVzZQ0KPiANCj4gPiBvbiB4ODYsIG1lbW9yeSBzdGFydHMgZnJvbSAwLiBPbiBhIG11bHRpcGxl
IG5vZGVzIE5VTUEgc3lzdGVtLCBpZg0KPiA+IGEgc2luZ2xlIG5vZGUgY29udGFpbnMgdGhlIG1h
am9yaXR5IG9yIGFsbCBvZiB0aGUgRE1BIG1lbW9yeS4geDg2DQo+ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXiwNCj4gDQo+ID4g
cHJlZmVyIHRvIGdpdmUgb3V0IG1lbW9yeSBmcm9tIG5vbi1sb2NhbCBhbGxvY2F0aW9ucyByYXRo
ZXIgdGhhbg0KPiA+IGV4aGF1c3RpbmcgdGhlIERNQSBtZW1vcnkgcmFuZ2VzLiBIZW5jZSB4ODYg
dXNlIGRtYV9iaXRzaXplIHRvIHNldA0KPiA+IGFzaWRlIHNvbWUgbGFyZ2VseSBhcmJpdHJhcnkg
YW1vdW50IG1lbW9yeSBmb3IgRE1BIG1lbW9yeSByYW5nZXMuDQo+ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICBeIG9mIG1lbW9yeQ0KPiANCj4gPiBUaGUgYWxsb2NhdGlvbnMg
ZnJvbSB0aGVzZSBtZW1vcnkgcmFuZ2VzIHdvdWxkIGhhcHBlbiBvbmx5IGFmdGVyDQo+ID4gZXho
YXVzdGluZyBhbGwgb3RoZXIgbm9kZXMnIG1lbW9yeS4NCj4gPg0KPiA+IEJ1dCB0aGUgaW1wbGlj
YXRpb25zIGFyZSBub3Qgc2hhcmVkIGFjcm9zcyBhbGwgYXJjaGl0ZWN0dXJlcy4gRm9yDQo+ID4g
ZXhhbXBsZSwgQXJtIGRvZXNuJ3QgaGF2ZSB0aGVzZSBpbXBsaWNhdGlvbnMuIFNvIGluIHRoaXMg
cGF0Y2gsIHdlDQo+ID4gaW50cm9kdWNlIGFuIGFyY2hfaGF2ZV9kZWZhdWx0X2RtYXpvbmUgaGVs
cGVyIGZvciBhcmNoIHRvIGRldGVybWluZQ0KPiA+IHRoYXQgaXQgbmVlZCB0byBzZXQgZG1hX2Jp
dHNpemUgZm9yIHJlc2VydmUgRE1BIGFsbG9jYXRpb25zIG9yIG5vdC4NCj4gICAgICAgICAgIF4g
bmVlZHMNCj4gDQoNCkkgd2lsbCBmaXggYWJvdmUgdHlwb3MgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+
ID4NCj4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAt
LS0NCj4gPiAgeGVuL2FyY2gveDg2L251bWEuYyAgICAgICAgfCA1ICsrKysrDQo+ID4gIHhlbi9j
b21tb24vcGFnZV9hbGxvYy5jICAgIHwgMiArLQ0KPiA+ICB4ZW4vaW5jbHVkZS9hc20tYXJtL251
bWEuaCB8IDUgKysrKysNCj4gPiAgeGVuL2luY2x1ZGUvYXNtLXg4Ni9udW1hLmggfCAxICsNCj4g
PiAgNCBmaWxlcyBjaGFuZ2VkLCAxMiBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pDQo+ID4N
Cj4gPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gveDg2L251bWEuYyBiL3hlbi9hcmNoL3g4Ni9udW1h
LmMNCj4gPiBpbmRleCBjZTc5ZWU0NGNlLi4xZmFiYmU4MjgxIDEwMDY0NA0KPiA+IC0tLSBhL3hl
bi9hcmNoL3g4Ni9udW1hLmMNCj4gPiArKysgYi94ZW4vYXJjaC94ODYvbnVtYS5jDQo+ID4gQEAg
LTM3MSw2ICszNzEsMTEgQEAgdW5zaWduZWQgaW50IF9faW5pdCBhcmNoX2dldF9kbWFfYml0c2l6
ZSh2b2lkKQ0KPiA+ICAgICAgICAgICAgICAgICAgICsgUEFHRV9TSElGVCwgMzIpOw0KPiA+ICB9
DQo+ID4NCj4gPiArdW5zaWduZWQgaW50IGFyY2hfaGF2ZV9kZWZhdWx0X2RtYXpvbmUodm9pZCkN
Cj4gDQo+IENhbiB0aGlzIGZ1bmN0aW9uIHJldHVybiBib29sPw0KPiBBbHNvLCBjYW4gaXQgYmUg
YSBzdGF0aWMgaW5saW5lPw0KPiANCg0KWWVzLCBib29sIHdvdWxkIGJlIGJldHRlci4gSSB3aWxs
IHBsYWNlIGEgc3RhdGljIGlubGluZSBpbiBhc20vbnVtYS5oLg0KQmVjYXVzZSBhcm0gd2lsbCBo
YXZlIGFub3RoZXIgc3RhdGljIGlubGluZSBpbXBsZW1lbnRhdGlvbi4NCg0KPiANCj4gPiArew0K
PiA+ICsgICAgcmV0dXJuICggbnVtX29ubGluZV9ub2RlcygpID4gMSApID8gMSA6IDA7DQo+ID4g
K30NCj4gPiArDQo+ID4gIHN0YXRpYyB2b2lkIGR1bXBfbnVtYSh1bnNpZ25lZCBjaGFyIGtleSkN
Cj4gPiAgew0KPiA+ICAgICAgc190aW1lX3Qgbm93ID0gTk9XKCk7DQo+ID4gZGlmZiAtLWdpdCBh
L3hlbi9jb21tb24vcGFnZV9hbGxvYy5jIGIveGVuL2NvbW1vbi9wYWdlX2FsbG9jLmMNCj4gPiBp
bmRleCA1ODAxMzU4YjRiLi44MDkxNjIwNWU1IDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9jb21tb24v
cGFnZV9hbGxvYy5jDQo+ID4gKysrIGIveGVuL2NvbW1vbi9wYWdlX2FsbG9jLmMNCj4gPiBAQCAt
MTg4OSw3ICsxODg5LDcgQEAgdm9pZCBfX2luaXQgZW5kX2Jvb3RfYWxsb2NhdG9yKHZvaWQpDQo+
ID4gICAgICB9DQo+ID4gICAgICBucl9ib290bWVtX3JlZ2lvbnMgPSAwOw0KPiA+DQo+ID4gLSAg
ICBpZiAoICFkbWFfYml0c2l6ZSAmJiAobnVtX29ubGluZV9ub2RlcygpID4gMSkgKQ0KPiA+ICsg
ICAgaWYgKCAhZG1hX2JpdHNpemUgJiYgYXJjaF9oYXZlX2RlZmF1bHRfZG1hem9uZSgpICkNCj4g
PiAgICAgICAgICBkbWFfYml0c2l6ZSA9IGFyY2hfZ2V0X2RtYV9iaXRzaXplKCk7DQo+ID4NCj4g
PiAgICAgIHByaW50aygiRG9tYWluIGhlYXAgaW5pdGlhbGlzZWQiKTsNCj4gPiBkaWZmIC0tZ2l0
IGEveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEu
aA0KPiA+IGluZGV4IDMxYTZkZTRlMjMuLjlkNTczOTU0MmQgMTAwNjQ0DQo+ID4gLS0tIGEveGVu
L2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL251
bWEuaA0KPiA+IEBAIC0yNSw2ICsyNSwxMSBAQCBleHRlcm4gbWZuX3QgZmlyc3RfdmFsaWRfbWZu
Ow0KPiA+ICAjZGVmaW5lIG5vZGVfc3RhcnRfcGZuKG5pZCkgKG1mbl94KGZpcnN0X3ZhbGlkX21m
bikpDQo+ID4gICNkZWZpbmUgX19ub2RlX2Rpc3RhbmNlKGEsIGIpICgyMCkNCj4gPg0KPiA+ICtz
dGF0aWMgaW5saW5lIHVuc2lnbmVkIGludCBhcmNoX2hhdmVfZGVmYXVsdF9kbWF6b25lKHZvaWQp
DQo+ID4gK3sNCj4gPiArICAgIHJldHVybiAwOw0KPiA+ICt9DQo+ID4gKw0KPiA+ICAjZW5kaWYg
LyogX19BUkNIX0FSTV9OVU1BX0ggKi8NCj4gPiAgLyoNCj4gPiAgICogTG9jYWwgdmFyaWFibGVz
Og0KPiA+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9hc20teDg2L251bWEuaCBiL3hlbi9pbmNs
dWRlL2FzbS14ODYvbnVtYS5oDQo+ID4gaW5kZXggM2NmMjZjMmRlZi4uODA2MGNiZjNmNCAxMDA2
NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L251bWEuaA0KPiA+ICsrKyBiL3hlbi9p
bmNsdWRlL2FzbS14ODYvbnVtYS5oDQo+ID4gQEAgLTc4LDUgKzc4LDYgQEAgZXh0ZXJuIGludCB2
YWxpZF9udW1hX3JhbmdlKHU2NCBzdGFydCwgdTY0IGVuZCwNCj4gbm9kZWlkX3Qgbm9kZSk7DQo+
ID4gIHZvaWQgc3JhdF9wYXJzZV9yZWdpb25zKHU2NCBhZGRyKTsNCj4gPiAgZXh0ZXJuIHU4IF9f
bm9kZV9kaXN0YW5jZShub2RlaWRfdCBhLCBub2RlaWRfdCBiKTsNCj4gPiAgdW5zaWduZWQgaW50
IGFyY2hfZ2V0X2RtYV9iaXRzaXplKHZvaWQpOw0KPiA+ICt1bnNpZ25lZCBpbnQgYXJjaF9oYXZl
X2RlZmF1bHRfZG1hem9uZSh2b2lkKTsNCj4gPg0KPiA+ICAjZW5kaWYNCj4gPiAtLQ0KPiA+IDIu
MjUuMQ0KPiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:06:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:06:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194745.346995 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTabf-0000Sc-SC; Fri, 24 Sep 2021 02:06:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194745.346995; Fri, 24 Sep 2021 02:06:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTabf-0000SV-OS; Fri, 24 Sep 2021 02:06:31 +0000
Received: by outflank-mailman (input) for mailman id 194745;
 Fri, 24 Sep 2021 02:06:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTabe-0000SP-GZ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:06:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 07d8ffa2-1cdc-11ec-ba98-12813bfff9fa;
 Fri, 24 Sep 2021 02:06:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A48BA61108;
 Fri, 24 Sep 2021 02:06:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07d8ffa2-1cdc-11ec-ba98-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632449188;
	bh=g08zLt9Gkbbfyj1h5ZtLbvf52EBAgqKephgTJlG3dd4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KiBCSUOJa6YEJmeG7qxp9usHG4tWV3jbmYGmzO/5LplVxJzaMn7+nDfIKhylqMlR2
	 x97+q5b6zUcmew+9N/PTV0C+jdKb6MEcR+uJKsw83ODwLkglVhjd/rPPoTzUfo2sy6
	 /375Xz0qyHgOXbmQunA4dCFQoBP0z89rsNgwGyfNSFsCQaGSxvlD6Na2+u/bjwpu3P
	 B06uuXMZEaDKoCFhAA3Wwc1vHdb7yB9bEdo/LQeR3JxClvJhh04bx+s96r+c1tWZjG
	 s5IsNk9yMnINAT7eRtjjlFkGC/8JcWjh7boSVsleKIcQqWd6OYZLY4Cs1EqLTGWE+k
	 hVQH8kprRNIuw==
Date: Thu, 23 Sep 2021 19:06:27 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 24/37] xen/arm: implement two arch helpers to get memory
 map info
In-Reply-To: <20210923120236.3692135-25-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231905180.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-25-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> These two helpers are architecture APIs that are required by
> nodes_cover_memory.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/numa.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> index 3f08870d69..3755b01ef4 100644
> --- a/xen/arch/arm/numa.c
> +++ b/xen/arch/arm/numa.c
> @@ -67,3 +67,17 @@ uint8_t __node_distance(nodeid_t from, nodeid_t to)
>      return node_distance_map[from][to];
>  }
>  EXPORT_SYMBOL(__node_distance);
> +
> +uint32_t __init arch_meminfo_get_nr_bank(void)
> +{
> +	return bootinfo.mem.nr_banks;
> +}
> +
> +int __init arch_meminfo_get_ram_bank_range(uint32_t bank,
> +	paddr_t *start, paddr_t *end)
> +{
> +	*start = bootinfo.mem.bank[bank].start;
> +	*end = bootinfo.mem.bank[bank].start + bootinfo.mem.bank[bank].size;
> +
> +	return 0;
> +}

The rest of the file is indented using spaces, while this patch is using
tabs.

Also, given the implementation, it looks like
arch_meminfo_get_ram_bank_range should either return void or bool.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:09:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:09:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194750.347006 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaec-0001Rq-AQ; Fri, 24 Sep 2021 02:09:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194750.347006; Fri, 24 Sep 2021 02:09:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaec-0001Rj-6k; Fri, 24 Sep 2021 02:09:34 +0000
Received: by outflank-mailman (input) for mailman id 194750;
 Fri, 24 Sep 2021 02:09:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTaeb-0001Rd-1g
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:09:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7f0eb23e-c6bf-4278-8105-caf39900c3fe;
 Fri, 24 Sep 2021 02:09:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 21A6E60F6F;
 Fri, 24 Sep 2021 02:09:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f0eb23e-c6bf-4278-8105-caf39900c3fe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632449371;
	bh=4jZPYpku3wEyw6MbydFbpZitaGb3nPMzTYyK/u6mYqE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=M88FS1ww3E6oPisAhKI6Qo/04ZW6UYWgzpw5eIz24IUkn54BvYA38BBPYuktDiMhk
	 RiAZO1HZVzVxGoWHTCIP496DrucCLGYa8x/cTdmlPwJiTR79MZiAKxiEuD76g++exJ
	 8M3C5KpyW0z+MybcZZDQcw1dYbYyZby1qPMnevmerd8w4jPFIZ6UIv+dhu8tj+z7j1
	 pe9wklR5EZCbs8juAOZ9fE8/okGgBRwLgXVvHMHp3dQVloUjgzOihaS/4/+0/NvLAz
	 fUlSloj7GQyW7YXjjr+1eE0Il94HxZGhO5B/TcfYST37sHR4AvZf4wfz/7w1+v4o/c
	 Ab7LXnjhJhkTQ==
Date: Thu, 23 Sep 2021 19:09:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA
 initialization
In-Reply-To: <20210923120236.3692135-26-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-26-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> NUMA initialization will parse information from firmware provided
> static resource affinity table (ACPI SRAT or DTB). bad_srat if a
> function that will be used when initialization code encounters
> some unexcepted errors.
> 
> In this patch, we introduce Arm version bad_srat for NUMA common
> initialization code to invoke it.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/numa.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> index 3755b01ef4..5209d3de4d 100644
> --- a/xen/arch/arm/numa.c
> +++ b/xen/arch/arm/numa.c
> @@ -18,6 +18,7 @@
>   *
>   */
>  #include <xen/init.h>
> +#include <xen/nodemask.h>
>  #include <xen/numa.h>
>  
>  static uint8_t __read_mostly
> @@ -25,6 +26,12 @@ node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
>      { 0 }
>  };
>  
> +__init void bad_srat(void)
> +{
> +    printk(KERN_ERR "NUMA: Firmware SRAT table not used.\n");
> +    fw_numa = -1;
> +}

I realize that the series keeps the "srat" terminology everywhere on DT
too. I wonder if it is worth replacing srat with something like
"numa_distance" everywhere as appropriate. I am adding the x86
maintainers for an opinion.

If you guys prefer to keep srat (if nothing else, it is concise), I am
also OK with keeping srat although it is not technically accurate.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:26:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:26:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194756.347017 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaui-00040f-QY; Fri, 24 Sep 2021 02:26:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194756.347017; Fri, 24 Sep 2021 02:26:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTaui-00040Y-M3; Fri, 24 Sep 2021 02:26:12 +0000
Received: by outflank-mailman (input) for mailman id 194756;
 Fri, 24 Sep 2021 02:26:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTaug-00040S-T6
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:26:10 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id df9f45c2-5f75-49a3-87ff-da5a14eefe23;
 Fri, 24 Sep 2021 02:26:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AF881611B0;
 Fri, 24 Sep 2021 02:26:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df9f45c2-5f75-49a3-87ff-da5a14eefe23
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632450366;
	bh=KWC+WCTnZuSzJ6ckG9uzgvOq4MOCfQ78yz2ehPfHGWU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JxxJEXGBCaVkkFa6NyI25wB0QnZGocgD4FmOOpwgC/yNEUccj/UpF+FJ8IXG7QIbS
	 or3FaJX3MxMogmIMYgGcFcc/BcWJN6mifTRsi9o91yYIrG1uhIDgcEzDXzR46xkHro
	 citbT6l03unDA7GSKwadFszl64Lbqy/gF/04Gd7HIjs0af66bkTxmBML2+uINUh/Tx
	 gsnCivsW1yN6Y5mTGqzUwGnDyNHjj6LbPK3zSUN3Bf+MCNnRYe8l6dihJzLb0fV/BG
	 QMsMG3XbLh3qKPYzXER3pCiUYFW97z9SflRqeipJusz6KCMUG+VfXpvSRJO1hXK3bK
	 y4Pwl9efDO/gA==
Date: Thu, 23 Sep 2021 19:26:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 26/37] xen/arm: build NUMA cpu_to_node map in
 dt_smp_init_cpus
In-Reply-To: <20210923120236.3692135-27-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231919070.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-27-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> NUMA implementation has a cpu_to_node array to store CPU to NODE
> map. Xen is using CPU logical ID in runtime components, so we
> use CPU logical ID as CPU index in cpu_to_node.
> 
> In device tree case, cpu_logical_map is created in dt_smp_init_cpus.
> So, when NUMA is enabled, dt_smp_init_cpus will fetch CPU NUMA id
> at the same time for cpu_to_node.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/smpboot.c     | 37 ++++++++++++++++++++++++++++++++++++-
>  xen/include/asm-arm/numa.h |  5 +++++
>  2 files changed, 41 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 60c0e82fc5..6e3cc8d3cc 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -121,7 +121,12 @@ static void __init dt_smp_init_cpus(void)
>      {
>          [0 ... NR_CPUS - 1] = MPIDR_INVALID
>      };
> +    static nodeid_t node_map[NR_CPUS] __initdata =
> +    {
> +        [0 ... NR_CPUS - 1] = NUMA_NO_NODE
> +    };
>      bool bootcpu_valid = false;
> +    uint32_t nid = 0;
>      int rc;
>  
>      mpidr = system_cpuinfo.mpidr.bits & MPIDR_HWID_MASK;
> @@ -172,6 +177,28 @@ static void __init dt_smp_init_cpus(void)
>              continue;
>          }
>  
> +        if ( IS_ENABLED(CONFIG_NUMA) )
> +        {
> +            /*
> +             * When CONFIG_NUMA is set, try to fetch numa infomation
> +             * from CPU dts node, otherwise the nid is always 0.
> +             */
> +            if ( !dt_property_read_u32(cpu, "numa-node-id", &nid) )
> +            {
> +                printk(XENLOG_WARNING
> +                       "cpu[%d] dts path: %s: doesn't have numa information!\n",
                               ^ %u


> +                       cpuidx, dt_node_full_name(cpu));

I think that this message shouldn't be a warning: CONFIG_NUMA is a
compile time option. Anybody that enables CONFIG_NUMA in the Xen build
will get this warning printed out at boot time if Xen is booting on a
regular non-NUMA machine, right?

The warning should only be printed if NUMA is actively enabled, e.g.
there is a distance-map but the cpus don't have numa-node-id.



> +                /*
> +                 * During the early stage of NUMA initialization, when Xen
> +                 * found any CPU dts node doesn't have numa-node-id info, the
> +                 * NUMA will be treated as off, all CPU will be set to a FAKE
> +                 * node 0. So if we get numa-node-id failed here, we should
> +                 * set nid to 0.
> +                 */
> +                nid = 0;
> +            }
> +        }
> +
>          /*
>           * 8 MSBs must be set to 0 in the DT since the reg property
>           * defines the MPIDR[23:0]
> @@ -231,9 +258,12 @@ static void __init dt_smp_init_cpus(void)
>          {
>              printk("cpu%d init failed (hwid %"PRIregister"): %d\n", i, hwid, rc);
>              tmp_map[i] = MPIDR_INVALID;
> +            node_map[i] = NUMA_NO_NODE;
>          }
> -        else
> +        else {
>              tmp_map[i] = hwid;
> +            node_map[i] = nid;
> +        }
>      }
>  
>      if ( !bootcpu_valid )
> @@ -249,6 +279,11 @@ static void __init dt_smp_init_cpus(void)
>              continue;
>          cpumask_set_cpu(i, &cpu_possible_map);
>          cpu_logical_map(i) = tmp_map[i];
> +
> +        nid = node_map[i];
> +        if ( nid >= MAX_NUMNODES )
> +            nid = 0;
> +        numa_set_node(i, nid);
>      }
>  }
>  
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 758eafeb05..8a4ad379e0 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -46,6 +46,11 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +static inline void numa_set_node(int cpu, nodeid_t node)
> +{
> +
> +}
> +
>  #endif
>  
>  static inline unsigned int arch_have_default_dmazone(void)
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:31:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:31:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194761.347028 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTazT-0005Nl-Eq; Fri, 24 Sep 2021 02:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194761.347028; Fri, 24 Sep 2021 02:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTazT-0005Ne-9q; Fri, 24 Sep 2021 02:31:07 +0000
Received: by outflank-mailman (input) for mailman id 194761;
 Fri, 24 Sep 2021 02:31:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CKs0=OO=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mTazS-0005NY-1r
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:31:06 +0000
Received: from mail-ed1-x530.google.com (unknown [2a00:1450:4864:20::530])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 13febcb0-fdcc-45dc-b845-9d830e41fcef;
 Fri, 24 Sep 2021 02:31:04 +0000 (UTC)
Received: by mail-ed1-x530.google.com with SMTP id x7so15787149edd.6
 for <xen-devel@lists.xenproject.org>; Thu, 23 Sep 2021 19:31:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13febcb0-fdcc-45dc-b845-9d830e41fcef
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=oo/mxOc5qkAqjzAwodZ8n1OU7XMmawmdUNiILrtwiEU=;
        b=LCQGKQSdQDlCakC56rdLLOUjGn9QXDJwVwC6THANMSM+btqwpgD/1jjMC4cgcrWdCY
         hEJhYr/5aLTIQ5vtJZTZt7YFHncNdz/dwHltphjBI2Rvkyw3C5zv3SHCppQS3na9ZzrN
         d21a+0wwhaUbjGRODZe7D9HsMqFtfroKgkfIFk38DYk6PJ5WXiGZfsjCmy1SUCMQlx0C
         xAPESpITJN8kGumfKfpqp4B1n3ARYFNqSWD2iljRKhQWuQU34YBwyDskvO4pHL40nDMO
         GrDSl+Z7AZ4HVoyNVGuEocjXGJIu3i/APDsjVlGIg84FBFKO7/V8BFsSLpBY6xQV6/tu
         MRTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=oo/mxOc5qkAqjzAwodZ8n1OU7XMmawmdUNiILrtwiEU=;
        b=BKFLx2An+wbb62Kd4/Nu888j1pGyFM35m3DQYiGRAPh6Y5atIAoz7exVCL0kPPMId8
         zZCc0VWejue5axS04Yf/xnaSKs5RkoOlxlOMA6IvG1l6YI0tDmxfIlTEfShq+aAcKALU
         LDTMtHrhoJNFi2lNYWVsxMXeN1ctVr1d3W1AIF5CwPa/fBkwElkHD9AudAMIGoHr7hr9
         AYP6+5BxrO/MFPtzk9KY3dfInfxhgGU9Wb6n5Y6NT3V8ZqQD6Dw2kxslqOdqHAy8OgSq
         keyOBR3b17eHkApXuNte5D8xhCOBMnOCFGq8lIPvx2/Py9xdCA3TLRWu+QqThBzUDYD7
         aYtw==
X-Gm-Message-State: AOAM533i1sFUAf+lpIqRnAHb6XY8UTBmI77JAWrZcyCYditcyi0WcALc
	Scwr8Vv1PAnzetmd/y4eBT9frEW60bxdFySCoQ8=
X-Google-Smtp-Source: ABdhPJw7ddLpHt2d3wLo4dmeWJDd/4bQSLi/zmGsnM+cDdeYeWkxEUQP6H0Qxuyd13pKdHq+M8HIXZVyElW42ZAd600=
X-Received: by 2002:a17:906:2887:: with SMTP id o7mr8723264ejd.425.1632450663372;
 Thu, 23 Sep 2021 19:31:03 -0700 (PDT)
MIME-Version: 1.0
References: <20210922082123.54374-1-roger.pau@citrix.com> <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local> <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
 <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
In-Reply-To: <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Fri, 24 Sep 2021 07:30:51 +0500
Message-ID: <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, 
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD <anthony.perard@citrix.com>, 
	Juergen Gross <jgross@suse.com>, Christian Lindig <christian.lindig@citrix.com>, 
	David Scott <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Type: multipart/alternative; boundary="000000000000e75a5e05ccb489d0"

--000000000000e75a5e05ccb489d0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Roger,

On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=A9, <roger.pau@citrix.com> wro=
te:

> On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
> > Hi Roger,
> >
> > On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:
> > > On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
> > > >
> > > >
> > > > On 22/09/2021 13:21, Roger Pau Monne wrote:
> > > > > Hello,
> > > >
> > > > Hi Roger,
> > > >
> > > > > First patch on the series is a trivial change to xenconsoled in
> order to
> > > > > use xenforeignmemory stable library in order to map the shared
> console
> > > > > ring instead of the unstable libxc interface. It's reviewed and
> ready to
> > > > > go in.
> > > > >
> > > > > Patches 2 and 3 allow setting the host wide command line `gnttab`
> option
> > > > > on a per domain basis. That means selecting the max allowed grant
> table
> > > > > version and whether transitive grants are allowed.
> > > > >
> > > > > The last 3 patches attempt to implement support for creating gues=
ts
> > > > > without a grant table. This requires some changes to xenstored in
> order
> > > > > to partially support guests without a valid ring interface, as th=
e
> lack
> > > > > of grant table will prevent C xenstored from mapping the shared
> ring.
> > > > > Note this is not an issue for Ocaml xenstored, as it still uses t=
he
> > > > > foreign memory interface to map the shared ring, and thus won't
> notice
> > > > > the lack of grant table support on the domain.
> > > >
> > > > I find a bit odd that the Xenstore support is conditional to whethe=
r
> grant
> > > > table is available. Are you expecting domains with no grant table t=
o
> have no
> > > > PV drivers (including PV shutdown)?
> > >
> > > I don't really expect much, as having guests without grant table is a
> > > developer option right now, if someone wants to make use of them for
> > > any reason it would need some thought.
> > >
> > > The other option would be my first proposal to restore foreign mappin=
g
> > > of the xenstore ring on that case:
> > >
> > >
> https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix=
.com/
> > >
> > > But it's also arguable that a guest not having a grant table should
> > > also likely prevent foreign mapping attempts. Plus such foreign
> > > mapping won't work from stubdomains.
> >
> > There is another option: extend the acquire hypercall to allow xenstore=
d
> > domain to map the xenstore interface. This would require more work, but
> at
> > least it would avoid the interesting dependency on the grant table.
>
> Xen isn't aware of the shared xenstore ring page currently, so that
> would mean introducing more knowledge to the hypervisor that what's
> strictly required IMO, as Xen has no business in knowing such details.
>

Well Xen already knows the page for HVM/PVH because the guest retrieve it
through an HMV param.

We only miss (?) the PV part.


> The grant table slot used by the xenstore shared page is just an
> agreement at the toolstack level, but not known to the hypervisor so
> far.
>

Right, we need to find a different way to provide/map the shared page if
the grant table is not present.

To me the acquire hypercall is the best way to resolve it as Xen knows
whether the domain will run Xenstored (at least we used to have a flag) and
we can do the permission control easily.

Do you have another alternative?


> > >
> > > I'm fine with dropping those patches if they turn out to be too
> > > controversial, I think it's an interesting option to be able to
> > > disable the grant table, but I don't have a full picture of how that
> > > could be used in practice. Maybe others have and would be willing to
> > > pick this up.
> >
> > I think the current approach is probably OK as a developper option.
> However,
> > we should at least document in the option that disabling the grant-tabl=
e
> > will also disable Xenstore (anything else?) support when using C
> Xenstored.
> >
> > It might also be worth to clearly state in the doc that this is only
> > intended for developer use and not supported.
>
> Sure, adding it to xl.cfg man page is likely the best place. Will do
> when updating the patches.
>
> Thanks, Roger.
>

--000000000000e75a5e05ccb489d0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div>Hi Roger,<br><br><div class=3D"gmail_quote"><div dir=
=3D"ltr" class=3D"gmail_attr">On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=
=A9, &lt;<a href=3D"mailto:roger.pau@citrix.com">roger.pau@citrix.com</a>&g=
t; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Sep 23, 2021 at 0=
1:47:37PM +0500, Julien Grall wrote:<br>
&gt; Hi Roger,<br>
&gt; <br>
&gt; On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:<br>
&gt; &gt; On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; On 22/09/2021 13:21, Roger Pau Monne wrote:<br>
&gt; &gt; &gt; &gt; Hello,<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; Hi Roger,<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; First patch on the series is a trivial change to xencon=
soled in order to<br>
&gt; &gt; &gt; &gt; use xenforeignmemory stable library in order to map the=
 shared console<br>
&gt; &gt; &gt; &gt; ring instead of the unstable libxc interface. It&#39;s =
reviewed and ready to<br>
&gt; &gt; &gt; &gt; go in.<br>
&gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; Patches 2 and 3 allow setting the host wide command lin=
e `gnttab` option<br>
&gt; &gt; &gt; &gt; on a per domain basis. That means selecting the max all=
owed grant table<br>
&gt; &gt; &gt; &gt; version and whether transitive grants are allowed.<br>
&gt; &gt; &gt; &gt; <br>
&gt; &gt; &gt; &gt; The last 3 patches attempt to implement support for cre=
ating guests<br>
&gt; &gt; &gt; &gt; without a grant table. This requires some changes to xe=
nstored in order<br>
&gt; &gt; &gt; &gt; to partially support guests without a valid ring interf=
ace, as the lack<br>
&gt; &gt; &gt; &gt; of grant table will prevent C xenstored from mapping th=
e shared ring.<br>
&gt; &gt; &gt; &gt; Note this is not an issue for Ocaml xenstored, as it st=
ill uses the<br>
&gt; &gt; &gt; &gt; foreign memory interface to map the shared ring, and th=
us won&#39;t notice<br>
&gt; &gt; &gt; &gt; the lack of grant table support on the domain.<br>
&gt; &gt; &gt; <br>
&gt; &gt; &gt; I find a bit odd that the Xenstore support is conditional to=
 whether grant<br>
&gt; &gt; &gt; table is available. Are you expecting domains with no grant =
table to have no<br>
&gt; &gt; &gt; PV drivers (including PV shutdown)?<br>
&gt; &gt; <br>
&gt; &gt; I don&#39;t really expect much, as having guests without grant ta=
ble is a<br>
&gt; &gt; developer option right now, if someone wants to make use of them =
for<br>
&gt; &gt; any reason it would need some thought.<br>
&gt; &gt; <br>
&gt; &gt; The other option would be my first proposal to restore foreign ma=
pping<br>
&gt; &gt; of the xenstore ring on that case:<br>
&gt; &gt; <br>
&gt; &gt; <a href=3D"https://lore.kernel.org/xen-devel/20210917154625.89315=
-6-roger.pau@citrix.com/" rel=3D"noreferrer noreferrer" target=3D"_blank">h=
ttps://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.co=
m/</a><br>
&gt; &gt; <br>
&gt; &gt; But it&#39;s also arguable that a guest not having a grant table =
should<br>
&gt; &gt; also likely prevent foreign mapping attempts. Plus such foreign<b=
r>
&gt; &gt; mapping won&#39;t work from stubdomains.<br>
&gt; <br>
&gt; There is another option: extend the acquire hypercall to allow xenstor=
ed<br>
&gt; domain to map the xenstore interface. This would require more work, bu=
t at<br>
&gt; least it would avoid the interesting dependency on the grant table.<br=
>
<br>
Xen isn&#39;t aware of the shared xenstore ring page currently, so that<br>
would mean introducing more knowledge to the hypervisor that what&#39;s<br>
strictly required IMO, as Xen has no business in knowing such details.<br><=
/blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">Well =
Xen already knows the page for HVM/PVH because the guest retrieve it throug=
h an HMV param.</div><div dir=3D"auto"><br></div><div dir=3D"auto">We only =
miss (?) the PV part.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><d=
iv class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:=
0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The grant table slot used by the xenstore shared page is just an<br>
agreement at the toolstack level, but not known to the hypervisor so<br>
far.<br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"au=
to">Right, we need to find a different way to provide/map the shared page i=
f the grant table is not present.</div><div dir=3D"auto"><br></div><div dir=
=3D"auto">To me the acquire hypercall is the best way to resolve it as Xen =
knows whether the domain will run Xenstored (at least we used to have a fla=
g) and we can do the permission control easily.</div><div dir=3D"auto"><br>=
</div><div dir=3D"auto">Do you have another alternative?</div><div dir=3D"a=
uto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa=
dding-left:1ex">
<br>
&gt; &gt; <br>
&gt; &gt; I&#39;m fine with dropping those patches if they turn out to be t=
oo<br>
&gt; &gt; controversial, I think it&#39;s an interesting option to be able =
to<br>
&gt; &gt; disable the grant table, but I don&#39;t have a full picture of h=
ow that<br>
&gt; &gt; could be used in practice. Maybe others have and would be willing=
 to<br>
&gt; &gt; pick this up.<br>
&gt; <br>
&gt; I think the current approach is probably OK as a developper option. Ho=
wever,<br>
&gt; we should at least document in the option that disabling the grant-tab=
le<br>
&gt; will also disable Xenstore (anything else?) support when using C Xenst=
ored.<br>
&gt; <br>
&gt; It might also be worth to clearly state in the doc that this is only<b=
r>
&gt; intended for developer use and not supported.<br>
<br>
Sure, adding it to xl.cfg man page is likely the best place. Will do<br>
when updating the patches.<br>
<br>
Thanks, Roger.<br>
</blockquote></div></div></div>

--000000000000e75a5e05ccb489d0--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:33:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:33:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194766.347039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTb1r-0006MO-Qv; Fri, 24 Sep 2021 02:33:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194766.347039; Fri, 24 Sep 2021 02:33:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTb1r-0006MH-N3; Fri, 24 Sep 2021 02:33:35 +0000
Received: by outflank-mailman (input) for mailman id 194766;
 Fri, 24 Sep 2021 02:33:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTb1q-0006MB-Ht
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:33:34 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cf9c4168-1cdf-11ec-ba99-12813bfff9fa;
 Fri, 24 Sep 2021 02:33:33 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 577C16103D;
 Fri, 24 Sep 2021 02:33:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf9c4168-1cdf-11ec-ba99-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632450812;
	bh=we3haw8iI8cP0tiAzQV1NlBx/lAfQZZaO+9kZSUQW00=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pnXwc/F6r6PFUvg11ctXL/gbPeZwZrHeCtt2O4yhQRxYAdVumVw9r52p/q18yCQTy
	 lqr5YHQgntOOiAYiabeiLp/DED3WXrFs3Nxvmzfk8uJ4eCLyi79+oy1eW3YDVJXBbC
	 YkFtL5dxkEQrC8dc0yDtvk6DqPzMsNGKeR2OUH5RkoMD4aHEpZxLElY6EhKmbXtjEA
	 QlaAyJ6ZwrxMBcHBYuebzfKrO6nLcL1osy3zNnHhqaCImPWQnitOCjikDxaEQWnKee
	 M+uHKtDCtLy0nLjzy7wtaUeLIX0a/waHNvLk6PMV24F0HgXv01bODFCrmpqSy14hhQ
	 fFn3v8y9tzy6w==
Date: Thu, 23 Sep 2021 19:33:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 28/37] xen/arm: stub memory hotplug access helpers for
 Arm
In-Reply-To: <20210923120236.3692135-29-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231932250.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-29-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Common code in NUMA need these two helpers to access/update
> memory hotplug end address. Arm has not support memory hotplug
> yet. So we stub these two helpers in this patch to make NUMA
> common code happy.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/include/asm-arm/mm.h | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
> index 7b5e7b7f69..fc9433165d 100644
> --- a/xen/include/asm-arm/mm.h
> +++ b/xen/include/asm-arm/mm.h
> @@ -362,6 +362,16 @@ void clear_and_clean_page(struct page_info *page);
>  
>  unsigned int arch_get_dma_bitsize(void);
>  
> +static inline void mem_hotplug_update_boundary(paddr_t end)
> +{
> +
> +}
> +
> +static inline paddr_t mem_hotplug_boundary(void)
> +{
> +    return 0;
> +}

Why zero? Could it be INVALID_PADDR ?


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 02:45:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 02:45:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194772.347050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbCu-0008De-VR; Fri, 24 Sep 2021 02:45:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194772.347050; Fri, 24 Sep 2021 02:45:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbCu-0008DX-SA; Fri, 24 Sep 2021 02:45:00 +0000
Received: by outflank-mailman (input) for mailman id 194772;
 Fri, 24 Sep 2021 02:44:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbCt-0008DR-8f
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 02:44:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 67c7a4c2-1ce1-11ec-ba99-12813bfff9fa;
 Fri, 24 Sep 2021 02:44:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 19669610C9;
 Fri, 24 Sep 2021 02:44:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67c7a4c2-1ce1-11ec-ba99-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632451497;
	bh=DHPIrhRky9S/lVBCYIdFlkJ1ph5Q83A2LaJS2PruxFM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=PRShDCR3z0V80XN/6RHhCTL1WEl8igLrx1Dr7fyulx4BdIfzf2dIlLcHerhmfWSsp
	 KWCX2ZemeQvNsmMVB0HdLd9TZT+APuzz4fBPltvdiAbwADvGD+zPMh1KL6/Rr5ACam
	 0T2tWA4m1ZnGIkbv8U24CTDmxCIsZ6i0QUZdubT5WIDAT42GzyZ2oG6ylTG9vLMXDn
	 n/1xZ/7BU8z6lgd4lkHxP3V7q7xQhSgJzfiuzb/UJ/l5C9kaYmCMqgbpX4GpoPPdzv
	 VtCYnEbz3uE6vOlag/yQG4Ql7zU7pRn8rXLwuhxSUXekyFxOckq4pg27PJLyXw5QzG
	 FSHHnFsDaVJwA==
Date: Thu, 23 Sep 2021 19:44:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 29/37] xen/arm: introduce a helper to parse device tree
 processor node
In-Reply-To: <20210923120236.3692135-30-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231940110.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-30-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Processor NUMA ID information is stored in device tree's processor
> node as "numa-node-id". We need a new helper to parse this ID from
> processor node. If we get this ID from processor node, this ID's
> validity still need to be checked. Once we got a invalid NUMA ID
> from any processor node, the device tree will be marked as NUMA
> information invalid.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/Makefile           |  1 +
>  xen/arch/arm/numa_device_tree.c | 58 +++++++++++++++++++++++++++++++++
>  2 files changed, 59 insertions(+)
>  create mode 100644 xen/arch/arm/numa_device_tree.c
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 41ca311b6b..c50df2c25d 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -36,6 +36,7 @@ obj-y += mem_access.o
>  obj-y += mm.o
>  obj-y += monitor.o
>  obj-$(CONFIG_NUMA) += numa.o
> +obj-$(CONFIG_DEVICE_TREE_NUMA) += numa_device_tree.o
>  obj-y += p2m.o
>  obj-y += percpu.o
>  obj-y += platform.o
> diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
> new file mode 100644
> index 0000000000..2428fbae0b
> --- /dev/null
> +++ b/xen/arch/arm/numa_device_tree.c
> @@ -0,0 +1,58 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Arm Architecture support layer for NUMA.
> + *
> + * Copyright (C) 2021 Arm Ltd
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +#include <xen/init.h>
> +#include <xen/nodemask.h>
> +#include <xen/numa.h>
> +#include <xen/libfdt/libfdt.h>
> +#include <xen/device_tree.h>
> +
> +/* Callback for device tree processor affinity */
> +static int __init fdt_numa_processor_affinity_init(nodeid_t node)
> +{
> +    if ( srat_disabled() )
> +        return -EINVAL;

fdt_numa_processor_affinity_init is called by fdt_parse_numa_cpu_node
which is already parsing NUMA related info. Should this srat_disabled
check be moved to fdt_parse_numa_cpu_node?


> +    else if ( node == NUMA_NO_NODE || node >= MAX_NUMNODES )
> +    {
> +        bad_srat();
> +        return -EINVAL;
> +	}
> +
> +    numa_set_processor_nodes_parsed(node);
> +    fw_numa = 1;
> +
> +    printk(KERN_INFO "DT: NUMA node %"PRIu7" processor parsed\n", node);
> +
> +    return 0;
> +}
> +
> +/* Parse CPU NUMA node info */
> +static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
> +{
> +    uint32_t nid;
> +
> +    nid = device_tree_get_u32(fdt, node, "numa-node-id", MAX_NUMNODES);
> +    if ( nid >= MAX_NUMNODES )
> +    {
> +        printk(XENLOG_ERR "Node id %u exceeds maximum value\n", nid);
                                      ^ PRIu32


> +        return -EINVAL;
> +    }
> +
> +    return fdt_numa_processor_affinity_init(nid);
> +}
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:01:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:01:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194778.347060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbSe-0002RW-Cg; Fri, 24 Sep 2021 03:01:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194778.347060; Fri, 24 Sep 2021 03:01:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbSe-0002RP-9P; Fri, 24 Sep 2021 03:01:16 +0000
Received: by outflank-mailman (input) for mailman id 194778;
 Fri, 24 Sep 2021 03:01:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTbSc-0002RI-Mu
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:01:14 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea4e916a-6ea3-4361-a41b-42f6368532d3;
 Fri, 24 Sep 2021 03:01:11 +0000 (UTC)
Received: from DB8PR03CA0035.eurprd03.prod.outlook.com (2603:10a6:10:be::48)
 by AM6PR08MB3784.eurprd08.prod.outlook.com (2603:10a6:20b:85::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Fri, 24 Sep
 2021 03:01:08 +0000
Received: from DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:be:cafe::98) by DB8PR03CA0035.outlook.office365.com
 (2603:10a6:10:be::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 03:01:08 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT057.mail.protection.outlook.com (10.152.20.235) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 03:01:08 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Fri, 24 Sep 2021 03:01:07 +0000
Received: from 0b08c5453336.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 59A818B7-7E46-4B57-9589-4CC849481D94.1; 
 Fri, 24 Sep 2021 03:00:55 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0b08c5453336.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 03:00:55 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6378.eurprd08.prod.outlook.com (2603:10a6:10:257::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 24 Sep
 2021 03:00:54 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.013; Fri, 24 Sep 2021
 03:00:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea4e916a-6ea3-4361-a41b-42f6368532d3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rx47xGyb8kjvKgRFqMEqgcWVhvdNKjWci5ON7KC10Zk=;
 b=qM1OGXCRAbAhxx2+ibUU3v5LCXdxmgL2yrG2EEx8I+dJMEYM3eqhZIXADnN0SW+1jcszHq9ZXFe5nXjl+KZAnLtiUs1ZHFLs5JEwb8eSBF7MoPFYFdvvK9ZPTRdGKSNEuyfZUrvKFqlvyVx5XBDK3zWbujlZ30RMBeqbycp7wlg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BDqX40Ajrwj7Owe52xEHhiB1KNPxI7fFO5QMmFqCtHngFLek5iwD3ymVdaKApE6B69DufYCXJYsmb2Z2s+k015zLKeEufWF5YDtWFSdq1TUduwG2cV5GB7u1iAghg7QjMAMSGsnUXpJl6uPZOEqdwzp7+EZSXEAFgKzRdZZGU26NM49nZQJ17h9WW3QsBpL9TvetgWKixtHSfMDvhYQ1ZxzbOTxE1koph7gb+iT5GOlmamznQOBNU3xbhSkitNVp1WIZG1LHd5GqM9/2+DoKiKNxcUZI/bOhaYxs9RmYaCfiIPSpFIPMuLsvl4B8LwPTBVQ0gwSYz1tcKz23KWdqyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Rx47xGyb8kjvKgRFqMEqgcWVhvdNKjWci5ON7KC10Zk=;
 b=QkMYAhl3rhcUrDPuolwGneTXHTIyiL6NF+4tnnr+UupyIWzF0mZ5+gJRi/H4E01ZSiQosECTTw10mgJXiQKZA5EqOvpVhLceSXS8D0iNhHNdMzluo3r9ZXKe9fSJiVEi/8vl+4uPNEVUEOfBNj5ZiKKH12NqqUGhPK1CnRPlhppwAEDwcrOukb4Yhr+FxT32SOOn3fQxM8CW+z/7Kdcj13XDeDYmqL60UkIKnWW3mFlxRymYrOJMwLPR/NQB7WF6IppZr98cma93bAinKJo6sKgD93mWpFEOroMBhNnteHNwLpxKzxEW7ttWFKuZrabsnGU+jAni8d6grz1Lmyg+NA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Rx47xGyb8kjvKgRFqMEqgcWVhvdNKjWci5ON7KC10Zk=;
 b=qM1OGXCRAbAhxx2+ibUU3v5LCXdxmgL2yrG2EEx8I+dJMEYM3eqhZIXADnN0SW+1jcszHq9ZXFe5nXjl+KZAnLtiUs1ZHFLs5JEwb8eSBF7MoPFYFdvvK9ZPTRdGKSNEuyfZUrvKFqlvyVx5XBDK3zWbujlZ30RMBeqbycp7wlg=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node
 structure
Thread-Topic: [PATCH 07/37] xen/x86: use paddr_t for addresses in NUMA node
 structure
Thread-Index: AQHXsHMRNGZeupZbtEeYycYXK8HrbKuyUKWAgAAAioCAAB88gA==
Date: Fri, 24 Sep 2021 03:00:54 +0000
Message-ID:
 <DB9PR08MB68572DE787D203438F15DE269EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-8-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231709360.17979@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109231712550.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231712550.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 13C80DD69F8C534AB5FCEA8B08120F95.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: dad65c51-020f-4f83-8f53-08d97f078ec3
x-ms-traffictypediagnostic: DB9PR08MB6378:|AM6PR08MB3784:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB378444164BE44F185F6434E89EA49@AM6PR08MB3784.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2582;OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 xzSxwxKWX5nz3Z1HexenaGur/ILHRIyHrrB0fNeu4DVX4t+Z1oxoIgCE4+9NFQ81QJInwXfwNwSinpPsPeddxG4u/fll8BQepbF8HSBORTjQfmySxlhKlhmBITvo3VstdtYZGMYzFeAlNqEj42C4UVEtcuLixlMkZzMdOOlRYcarQ2Zx9ZoIiL8L3JXdu325srpP891P7keTGmJ1uZyOuWj1eYRr1cH59LE/R7TMcdtccsRN5+24qMIgBYWMk+jPd93I6nqj9vJsaBLXshOAQ41Zi/RKj+kC1r0yDBZoIBQAWn8OrqpvyLShBmoeeq52lOQkGvLE5rWT1iaDbsNoc17tcM1mG5h/mMiN1lpQO3ThoXSc8IR96oXKYZds0wZ/GSPuVm4p/6cEGnQFZB6fjoCUob5ASbl/zMteP7jAmLZ9qckY5Gvf3ECuGaEzLSXHGaCcL6NMirTfunL21u3kDVKQ80HEeCtLDywh3O7yihsbrexmoGZ0h37IL3gtgWKy/3SCT1x0QZcSYe9f4ta6xXBN6RbF14BPkQydJERiJVz/ScrcdyyuuIrgTl/PoH6Q8w+gUr+uiYsMxBfuk5kqXocWiPCgWR6jK9iBQ22UFLRuwrQO/2yZ9rNWO5Cu5925I7oQ9mFoAenktn+9TP6Pc+eg4YE42VwzKMM67KIIDSfXeI6c418bL9R0VVBQZBCo+NAFPTHMhmCr2qlcAhpTcg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66446008)(66476007)(83380400001)(53546011)(6916009)(2906002)(122000001)(316002)(76116006)(64756008)(30864003)(508600001)(66556008)(7696005)(33656002)(8936002)(54906003)(55016002)(38100700002)(86362001)(5660300002)(4326008)(38070700005)(52536014)(6506007)(186003)(66946007)(9686003)(71200400001)(8676002)(26005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6378
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9da63b67-7aca-4991-feb0-08d97f07863a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dRqCTtScb/4EqgInf/1xuvD2dAo1cXlviAwLRijHVEKUEwL0q6dRt3heKy0WXnJ/glkaNdsnru2p4S+mI7mBi1h31c26YEnu+dhGgY0K62TUPzAD7rK5mGB9XFmyNEEsxtYihtSQoXfFF7R4SJ1nAJ8lk8iwe5oUGR3NEX6I6PiYh100zBuuGqyyd3J9aryRehhOlJgU27n/ybftycIJUYksayTvd6A0V0EHHdK8nGFp8gcRt2blJj0L8259jAFzKc1k4kqLAx3rjeXMU4q7/P+cqu4hbkWL1qZLxjtHx08MIbFdcgRPOL16yIR3Am75F9GMafKxA7d7rThINmUFLOu7Ij7nJvMhqHWeGD7ytcybXLBcLrMQ+iXMj/5v+j420FZrqL/YXTAM+8hELHNVjGWIIvQzWli2FOa03o/67izM1kNgsw9prdqUTqdmOC2IG4DcpphWlRQnIaOh6DpL4QtsA4p2IQnonUhVM9hcoleLGvJOOskq86Bl0f8gc7f/10pd1wjaCdzl8PU03GZ6l6oPHOwcEbtQhM5NbV/FocLfqtfppJ/k8iHMLwsuA+Jp4KkFwYARLcHMDP3QmN/PjOjK7SGb+stsaEy145s9/DivhMW514+ipKYoxLLKzNwrqVtrzbQVW0n94XDe3qt0zMs55vlSjOn6upQxvy+YBDAcIFr027E0+Uk2aAF+tKcr8JwyD4ZrZF4fhdG+pTdYMA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(2906002)(336012)(9686003)(54906003)(508600001)(26005)(8936002)(7696005)(6862004)(70206006)(55016002)(186003)(47076005)(70586007)(8676002)(33656002)(52536014)(30864003)(82310400003)(86362001)(83380400001)(36860700001)(5660300002)(81166007)(6506007)(356005)(316002)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 03:01:08.6235
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dad65c51-020f-4f83-8f53-08d97f078ec3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3784

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDg6MTQN
Cj4gVG86IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gQ2M6
IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPjsgeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qu
b3JnOw0KPiBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVp
c0Bhcm0uY29tPjsNCj4gamJldWxpY2hAc3VzZS5jb207IGFuZHJldy5jb29wZXIzQGNpdHJpeC5j
b207IHJvZ2VyLnBhdUBjaXRyaXguY29tOw0KPiB3bEB4ZW4ub3JnDQo+IFN1YmplY3Q6IFJlOiBb
UEFUQ0ggMDcvMzddIHhlbi94ODY6IHVzZSBwYWRkcl90IGZvciBhZGRyZXNzZXMgaW4gTlVNQSBu
b2RlDQo+IHN0cnVjdHVyZQ0KPiANCj4gWW91IGZvcmdvdCB0byBhZGQgdGhlIHg4NiBtYWludGFp
bmVycyBpbiBDQyB0byBhbGwgdGhlIHBhdGNoZXMgdG91Y2hpbmcNCj4geDg2IGNvZGUgaW4gdGhp
cyBzZXJpZXMuIEFkZGluZyB0aGVtIG5vdyBidXQgeW91IHNob3VsZCBwcm9iYWJseSByZXNlbmQu
DQo+DQoNCkkgYW0gdmVyeSBzb3JyeSBhYm91dCBpdC4gSSByZWFsaXplZCB0aGUgcHJvYmxlbSB3
aGVuIEkgcHJlc3NlZCBFbnRlci4NCkkgaGFkIHdhbnRlZCB0byByZXBvc3QgaXQgYXQgdGhhdCB0
aW1lLCBidXQgSSBkaWRuJ3Qga25vdyB3aGV0aGVyIHRoZXNlDQpwYXRjaGVzIHdpbGwgYmUgdHVy
bmVkIGludG8gc3BhbW1pbmcuLi4gDQoNCkkgd2lsbCByZXNlbmQgdGhpcyBzZXJpZXMgQVNBUCB3
aXRoIHNvbWUgY2hhbmdlcyB0byBhZGRyZXNzIHlvdXIgY29tbWVudHMuDQoNCj4gDQo+IE9uIFRo
dSwgMjMgU2VwIDIwMjEsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4gPiBPbiBUaHUsIDIz
IFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+IE5VTUEgbm9kZSBzdHJ1Y3R1cmUgInN0
cnVjdCBub2RlIiBpcyB1c2luZyB1NjQgYXMgbm9kZSBtZW1vcnkNCj4gPiA+IHJhbmdlLiBJbiBv
cmRlciB0byBtYWtlIG90aGVyIGFyY2hpdGVjdHVyZXMgY2FuIHJldXNlIHRoaXMNCj4gPiA+IE5V
TUEgbm9kZSByZWxhdGl2ZSBjb2RlLCB3ZSByZXBsYWNlIHRoZSB1NjQgdG8gcGFkZHJfdC4gQW5k
DQo+ID4gPiB1c2UgcGZuX3RvX3BhZGRyIGFuZCBwYWRkcl90b19wZm4gdG8gcmVwbGFjZSBleHBs
aWNpdCBzaGlmdA0KPiA+ID4gb3BlcmF0aW9ucy4gVGhlIHJlbGF0ZSBQUkl4NjQgaW4gcHJpbnQg
bWVzc2FnZXMgaGF2ZSBiZWVuDQo+ID4gPiByZXBsYWNlZCBieSBQUklwYWRkciBhdCB0aGUgc2Ft
ZSB0aW1lLg0KPiA+ID4NCj4gPiA+IFNpZ25lZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBh
cm0uY29tPg0KPiA+ID4gLS0tDQo+ID4gPiAgeGVuL2FyY2gveDg2L251bWEuYyAgICAgICAgfCAz
MiArKysrKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0tLQ0KPiA+ID4gIHhlbi9hcmNoL3g4Ni9z
cmF0LmMgICAgICAgIHwgMjYgKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0NCj4gPiA+ICB4ZW4v
aW5jbHVkZS9hc20teDg2L251bWEuaCB8ICA4ICsrKystLS0tDQo+ID4gPiAgMyBmaWxlcyBjaGFu
Z2VkLCAzNCBpbnNlcnRpb25zKCspLCAzMiBkZWxldGlvbnMoLSkNCj4gPiA+DQo+ID4gPiBkaWZm
IC0tZ2l0IGEveGVuL2FyY2gveDg2L251bWEuYyBiL3hlbi9hcmNoL3g4Ni9udW1hLmMNCj4gPiA+
IGluZGV4IDFmYWJiZTgyODEuLjYzMzdiYmRmMzEgMTAwNjQ0DQo+ID4gPiAtLS0gYS94ZW4vYXJj
aC94ODYvbnVtYS5jDQo+ID4gPiArKysgYi94ZW4vYXJjaC94ODYvbnVtYS5jDQo+ID4gPiBAQCAt
MTY1LDEyICsxNjUsMTIgQEAgaW50IF9faW5pdCBjb21wdXRlX2hhc2hfc2hpZnQoc3RydWN0IG5v
ZGUgKm5vZGVzLA0KPiBpbnQgbnVtbm9kZXMsDQo+ID4gPiAgICAgIHJldHVybiBzaGlmdDsNCj4g
PiA+ICB9DQo+ID4gPiAgLyogaW5pdGlhbGl6ZSBOT0RFX0RBVEEgZ2l2ZW4gbm9kZWlkIGFuZCBz
dGFydC9lbmQgKi8NCj4gPiA+IC12b2lkIF9faW5pdCBzZXR1cF9ub2RlX2Jvb3RtZW0obm9kZWlk
X3Qgbm9kZWlkLCB1NjQgc3RhcnQsIHU2NCBlbmQpDQo+ID4gPiAtew0KPiA+ID4gK3ZvaWQgX19p
bml0IHNldHVwX25vZGVfYm9vdG1lbShub2RlaWRfdCBub2RlaWQsIHBhZGRyX3Qgc3RhcnQsDQo+
IHBhZGRyX3QgZW5kKQ0KPiA+ID4gK3sNCj4gPiA+ICAgICAgdW5zaWduZWQgbG9uZyBzdGFydF9w
Zm4sIGVuZF9wZm47DQo+ID4gPg0KPiA+ID4gLSAgICBzdGFydF9wZm4gPSBzdGFydCA+PiBQQUdF
X1NISUZUOw0KPiA+ID4gLSAgICBlbmRfcGZuID0gZW5kID4+IFBBR0VfU0hJRlQ7DQo+ID4gPiAr
ICAgIHN0YXJ0X3BmbiA9IHBhZGRyX3RvX3BmbihzdGFydCk7DQo+ID4gPiArICAgIGVuZF9wZm4g
PSBwYWRkcl90b19wZm4oZW5kKTsNCj4gPiA+DQo+ID4gPiAgICAgIE5PREVfREFUQShub2RlaWQp
LT5ub2RlX3N0YXJ0X3BmbiA9IHN0YXJ0X3BmbjsNCj4gPiA+ICAgICAgTk9ERV9EQVRBKG5vZGVp
ZCktPm5vZGVfc3Bhbm5lZF9wYWdlcyA9IGVuZF9wZm4gLSBzdGFydF9wZm47DQo+ID4gPiBAQCAt
MjAxLDExICsyMDEsMTIgQEAgdm9pZCBfX2luaXQgbnVtYV9pbml0X2FycmF5KHZvaWQpDQo+ID4g
PiAgc3RhdGljIGludCBudW1hX2Zha2UgX19pbml0ZGF0YSA9IDA7DQo+ID4gPg0KPiA+ID4gIC8q
IE51bWEgZW11bGF0aW9uICovDQo+ID4gPiAtc3RhdGljIGludCBfX2luaXQgbnVtYV9lbXVsYXRp
b24odTY0IHN0YXJ0X3BmbiwgdTY0IGVuZF9wZm4pDQo+ID4gPiArc3RhdGljIGludCBfX2luaXQg
bnVtYV9lbXVsYXRpb24odW5zaWduZWQgbG9uZyBzdGFydF9wZm4sDQo+ID4gPiArICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBlbmRfcGZuKQ0KPiA+DQo+ID4g
V2h5IG5vdCBjaGFuZ2luZyBudW1hX2VtdWxhdGlvbiB0byB0YWtlIHBhZGRyX3QgdG9vPw0KPiA+
DQoNCm51bWFfZW11bGF0aW9uIHBhcmFtZXRlciBpcyBwZm4sIGl0J3Mgbm90IGFkZHJlc3MuIEkg
aGF2ZSBkaXNjdXNzZWQNCndpdGggSnVsaWVuIGluIFJGQyBhYm91dCBwZm4uIEhlIHN1Z2dlc3Rl
ZCB0byB1c2UgbWZuX3Qgb3IgdW5zaWduZWQNCmxvbmcgZm9yIHBmbi4gQ29tcGFyaW5nIHRvIG1m
bl90LCB1c2UgdW5zaWduZWQgbG9uZyBicmluZ3MgbGVzcw0KY2hhbmdlcy4NCg0KPiA+DQo+ID4g
PiAgew0KPiA+ID4gICAgICBpbnQgaTsNCj4gPiA+ICAgICAgc3RydWN0IG5vZGUgbm9kZXNbTUFY
X05VTU5PREVTXTsNCj4gPiA+IC0gICAgdTY0IHN6ID0gKChlbmRfcGZuIC0gc3RhcnRfcGZuKTw8
UEFHRV9TSElGVCkgLyBudW1hX2Zha2U7DQo+ID4gPiArICAgIHU2NCBzeiA9IHBmbl90b19wYWRk
cihlbmRfcGZuIC0gc3RhcnRfcGZuKSAvIG51bWFfZmFrZTsNCj4gPiA+DQo+ID4gPiAgICAgIC8q
IEtsdWRnZSBuZWVkZWQgZm9yIHRoZSBoYXNoIGZ1bmN0aW9uICovDQo+ID4gPiAgICAgIGlmICgg
aHdlaWdodDY0KHN6KSA+IDEgKQ0KPiA+ID4gQEAgLTIyMSw5ICsyMjIsOSBAQCBzdGF0aWMgaW50
IF9faW5pdCBudW1hX2VtdWxhdGlvbih1NjQgc3RhcnRfcGZuLA0KPiB1NjQgZW5kX3BmbikNCj4g
PiA+ICAgICAgbWVtc2V0KCZub2RlcywwLHNpemVvZihub2RlcykpOw0KPiA+ID4gICAgICBmb3Ig
KCBpID0gMDsgaSA8IG51bWFfZmFrZTsgaSsrICkNCj4gPiA+ICAgICAgew0KPiA+ID4gLSAgICAg
ICAgbm9kZXNbaV0uc3RhcnQgPSAoc3RhcnRfcGZuPDxQQUdFX1NISUZUKSArIGkqc3o7DQo+ID4g
PiArICAgICAgICBub2Rlc1tpXS5zdGFydCA9IHBmbl90b19wYWRkcihzdGFydF9wZm4pICsgaSpz
ejsNCj4gPiA+ICAgICAgICAgIGlmICggaSA9PSBudW1hX2Zha2UgLSAxICkNCj4gPiA+IC0gICAg
ICAgICAgICBzeiA9IChlbmRfcGZuPDxQQUdFX1NISUZUKSAtIG5vZGVzW2ldLnN0YXJ0Ow0KPiA+
ID4gKyAgICAgICAgICAgIHN6ID0gcGZuX3RvX3BhZGRyKGVuZF9wZm4pIC0gbm9kZXNbaV0uc3Rh
cnQ7DQo+ID4gPiAgICAgICAgICBub2Rlc1tpXS5lbmQgPSBub2Rlc1tpXS5zdGFydCArIHN6Ow0K
PiA+ID4gICAgICAgICAgcHJpbnRrKEtFUk5fSU5GTyAiRmFraW5nIG5vZGUgJWQgYXQgJSJQUkl4
NjQiLSUiUFJJeDY0Ig0KPiAoJSJQUkl1NjQiTUIpXG4iLA0KPiA+ID4gICAgICAgICAgICAgICAg
IGksDQo+ID4gPiBAQCAtMjQ5LDI0ICsyNTAsMjYgQEAgc3RhdGljIGludCBfX2luaXQgbnVtYV9l
bXVsYXRpb24odTY0IHN0YXJ0X3BmbiwNCj4gdTY0IGVuZF9wZm4pDQo+ID4gPiAgdm9pZCBfX2lu
aXQgbnVtYV9pbml0bWVtX2luaXQodW5zaWduZWQgbG9uZyBzdGFydF9wZm4sIHVuc2lnbmVkIGxv
bmcNCj4gZW5kX3BmbikNCj4gPg0KPiA+IHNhbWUgaGVyZQ0KPiA+DQo+ID4NCj4gPiA+ICB7DQo+
ID4gPiAgICAgIGludCBpOw0KPiA+ID4gKyAgICBwYWRkcl90IHN0YXJ0LCBlbmQ7DQo+ID4gPg0K
PiA+ID4gICNpZmRlZiBDT05GSUdfTlVNQV9FTVUNCj4gPiA+ICAgICAgaWYgKCBudW1hX2Zha2Ug
JiYgIW51bWFfZW11bGF0aW9uKHN0YXJ0X3BmbiwgZW5kX3BmbikgKQ0KPiA+ID4gICAgICAgICAg
cmV0dXJuOw0KPiA+ID4gICNlbmRpZg0KPiA+ID4NCj4gPiA+ICsgICAgc3RhcnQgPSBwZm5fdG9f
cGFkZHIoc3RhcnRfcGZuKTsNCj4gPiA+ICsgICAgZW5kID0gcGZuX3RvX3BhZGRyKGVuZF9wZm4p
Ow0KPiA+ID4gKw0KPiA+ID4gICNpZmRlZiBDT05GSUdfQUNQSV9OVU1BDQo+ID4gPiAtICAgIGlm
ICggIW51bWFfb2ZmICYmICFhY3BpX3NjYW5fbm9kZXMoKHU2NClzdGFydF9wZm4gPDwgUEFHRV9T
SElGVCwNCj4gPiA+IC0gICAgICAgICAodTY0KWVuZF9wZm4gPDwgUEFHRV9TSElGVCkgKQ0KPiA+
ID4gKyAgICBpZiAoICFudW1hX29mZiAmJiAhYWNwaV9zY2FuX25vZGVzKHN0YXJ0LCBlbmQpICkN
Cj4gPiA+ICAgICAgICAgIHJldHVybjsNCj4gPiA+ICAjZW5kaWYNCj4gPiA+DQo+ID4gPiAgICAg
IHByaW50ayhLRVJOX0lORk8gIiVzXG4iLA0KPiA+ID4gICAgICAgICAgICAgbnVtYV9vZmYgPyAi
TlVNQSB0dXJuZWQgb2ZmIiA6ICJObyBOVU1BIGNvbmZpZ3VyYXRpb24NCj4gZm91bmQiKTsNCj4g
PiA+DQo+ID4gPiAtICAgIHByaW50ayhLRVJOX0lORk8gIkZha2luZyBhIG5vZGUgYXQgJTAxNiJQ
Ukl4NjQiLSUwMTYiUFJJeDY0IlxuIiwNCj4gPiA+IC0gICAgICAgICAgICh1NjQpc3RhcnRfcGZu
IDw8IFBBR0VfU0hJRlQsDQo+ID4gPiAtICAgICAgICAgICAodTY0KWVuZF9wZm4gPDwgUEFHRV9T
SElGVCk7DQo+ID4gPiArICAgIHByaW50ayhLRVJOX0lORk8gIkZha2luZyBhIG5vZGUgYXQgJTAx
NiJQUklwYWRkciItDQo+ICUwMTYiUFJJcGFkZHIiXG4iLA0KPiA+ID4gKyAgICAgICAgICAgc3Rh
cnQsIGVuZCk7DQo+ID4gPiAgICAgIC8qIHNldHVwIGR1bW15IG5vZGUgY292ZXJpbmcgYWxsIG1l
bW9yeSAqLw0KPiA+ID4gICAgICBtZW1ub2RlX3NoaWZ0ID0gQklUU19QRVJfTE9ORyAtIDE7DQo+
ID4gPiAgICAgIG1lbW5vZGVtYXAgPSBfbWVtbm9kZW1hcDsNCj4gPiA+IEBAIC0yNzksOCArMjgy
LDcgQEAgdm9pZCBfX2luaXQgbnVtYV9pbml0bWVtX2luaXQodW5zaWduZWQgbG9uZw0KPiBzdGFy
dF9wZm4sIHVuc2lnbmVkIGxvbmcgZW5kX3BmbikNCj4gPiA+ICAgICAgZm9yICggaSA9IDA7IGkg
PCBucl9jcHVfaWRzOyBpKysgKQ0KPiA+ID4gICAgICAgICAgbnVtYV9zZXRfbm9kZShpLCAwKTsN
Cj4gPiA+ICAgICAgY3B1bWFza19jb3B5KCZub2RlX3RvX2NwdW1hc2tbMF0sIGNwdW1hc2tfb2Yo
MCkpOw0KPiA+ID4gLSAgICBzZXR1cF9ub2RlX2Jvb3RtZW0oMCwgKHU2NClzdGFydF9wZm4gPDwg
UEFHRV9TSElGVCwNCj4gPiA+IC0gICAgICAgICAgICAgICAgICAgICh1NjQpZW5kX3BmbiA8PCBQ
QUdFX1NISUZUKTsNCj4gPiA+ICsgICAgc2V0dXBfbm9kZV9ib290bWVtKDAsIHN0YXJ0LCBlbmQp
Ow0KPiA+ID4gIH0NCj4gPiA+DQo+ID4gPiAgdm9pZCBudW1hX2FkZF9jcHUoaW50IGNwdSkNCj4g
PiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC94ODYvc3JhdC5jIGIveGVuL2FyY2gveDg2L3NyYXQu
Yw0KPiA+ID4gaW5kZXggNmI3N2I5ODIwMS4uN2QyMGQ3ZjIyMiAxMDA2NDQNCj4gPiA+IC0tLSBh
L3hlbi9hcmNoL3g4Ni9zcmF0LmMNCj4gPiA+ICsrKyBiL3hlbi9hcmNoL3g4Ni9zcmF0LmMNCj4g
PiA+IEBAIC0xMDQsNyArMTA0LDcgQEAgbm9kZWlkX3Qgc2V0dXBfbm9kZSh1bnNpZ25lZCBweG0p
DQo+ID4gPiAgCXJldHVybiBub2RlOw0KPiA+ID4gIH0NCj4gPiA+DQo+ID4gPiAtaW50IHZhbGlk
X251bWFfcmFuZ2UodTY0IHN0YXJ0LCB1NjQgZW5kLCBub2RlaWRfdCBub2RlKQ0KPiA+ID4gK2lu
dCB2YWxpZF9udW1hX3JhbmdlKHBhZGRyX3Qgc3RhcnQsIHBhZGRyX3QgZW5kLCBub2RlaWRfdCBu
b2RlKQ0KPiA+ID4gIHsNCj4gPiA+ICAJaW50IGk7DQo+ID4gPg0KPiA+ID4gQEAgLTExOSw3ICsx
MTksNyBAQCBpbnQgdmFsaWRfbnVtYV9yYW5nZSh1NjQgc3RhcnQsIHU2NCBlbmQsIG5vZGVpZF90
DQo+IG5vZGUpDQo+ID4gPiAgCXJldHVybiAwOw0KPiA+ID4gIH0NCj4gPiA+DQo+ID4gPiAtc3Rh
dGljIF9faW5pdCBpbnQgY29uZmxpY3RpbmdfbWVtYmxrcyh1NjQgc3RhcnQsIHU2NCBlbmQpDQo+
ID4gPiArc3RhdGljIF9faW5pdCBpbnQgY29uZmxpY3RpbmdfbWVtYmxrcyhwYWRkcl90IHN0YXJ0
LCBwYWRkcl90IGVuZCkNCj4gPiA+ICB7DQo+ID4gPiAgCWludCBpOw0KPiA+ID4NCj4gPiA+IEBA
IC0xMzUsNyArMTM1LDcgQEAgc3RhdGljIF9faW5pdCBpbnQgY29uZmxpY3RpbmdfbWVtYmxrcyh1
NjQgc3RhcnQsDQo+IHU2NCBlbmQpDQo+ID4gPiAgCXJldHVybiAtMTsNCj4gPiA+ICB9DQo+ID4g
Pg0KPiA+ID4gLXN0YXRpYyBfX2luaXQgdm9pZCBjdXRvZmZfbm9kZShpbnQgaSwgdTY0IHN0YXJ0
LCB1NjQgZW5kKQ0KPiA+ID4gK3N0YXRpYyBfX2luaXQgdm9pZCBjdXRvZmZfbm9kZShpbnQgaSwg
cGFkZHJfdCBzdGFydCwgcGFkZHJfdCBlbmQpDQo+ID4gPiAgew0KPiA+ID4gIAlzdHJ1Y3Qgbm9k
ZSAqbmQgPSAmbm9kZXNbaV07DQo+ID4gPiAgCWlmIChuZC0+c3RhcnQgPCBzdGFydCkgew0KPiA+
ID4gQEAgLTI3NSw3ICsyNzUsNyBAQCBhY3BpX251bWFfcHJvY2Vzc29yX2FmZmluaXR5X2luaXQo
Y29uc3Qgc3RydWN0DQo+IGFjcGlfc3JhdF9jcHVfYWZmaW5pdHkgKnBhKQ0KPiA+ID4gIHZvaWQg
X19pbml0DQo+ID4gPiAgYWNwaV9udW1hX21lbW9yeV9hZmZpbml0eV9pbml0KGNvbnN0IHN0cnVj
dCBhY3BpX3NyYXRfbWVtX2FmZmluaXR5DQo+ICptYSkNCj4gPiA+ICB7DQo+ID4gPiAtCXU2NCBz
dGFydCwgZW5kOw0KPiA+ID4gKwlwYWRkcl90IHN0YXJ0LCBlbmQ7DQo+ID4gPiAgCXVuc2lnbmVk
IHB4bTsNCj4gPiA+ICAJbm9kZWlkX3Qgbm9kZTsNCj4gPiA+ICAJaW50IGk7DQo+ID4gPiBAQCAt
MzE4LDcgKzMxOCw3IEBAIGFjcGlfbnVtYV9tZW1vcnlfYWZmaW5pdHlfaW5pdChjb25zdCBzdHJ1
Y3QNCj4gYWNwaV9zcmF0X21lbV9hZmZpbml0eSAqbWEpDQo+ID4gPiAgCQlib29sIG1pc21hdGNo
ID0gIShtYS0+ZmxhZ3MgJiBBQ1BJX1NSQVRfTUVNX0hPVF9QTFVHR0FCTEUpICE9DQo+ID4gPiAg
CQkgICAgICAgICAgICAgICAgIXRlc3RfYml0KGksIG1lbWJsa19ob3RwbHVnKTsNCj4gPiA+DQo+
ID4gPiAtCQlwcmludGsoIiVzU1JBVDogUFhNICV1ICglIlBSSXg2NCItJSJQUkl4NjQiKSBvdmVy
bGFwcyB3aXRoDQo+IGl0c2VsZiAoJSJQUkl4NjQiLSUiUFJJeDY0IilcbiIsDQo+ID4gPiArCQlw
cmludGsoIiVzU1JBVDogUFhNICV1ICglIlBSSXBhZGRyIi0lIlBSSXBhZGRyIikgb3ZlcmxhcHMg
d2l0aA0KPiBpdHNlbGYgKCUiUFJJcGFkZHIiLSUiUFJJcGFkZHIiKVxuIiwNCj4gPiA+ICAJCSAg
ICAgICBtaXNtYXRjaCA/IEtFUk5fRVJSIDogS0VSTl9XQVJOSU5HLCBweG0sIHN0YXJ0LCBlbmQs
DQo+ID4gPiAgCQkgICAgICAgbm9kZV9tZW1ibGtfcmFuZ2VbaV0uc3RhcnQsIG5vZGVfbWVtYmxr
X3JhbmdlW2ldLmVuZCk7DQo+ID4gPiAgCQlpZiAobWlzbWF0Y2gpIHsNCj4gPiA+IEBAIC0zMjcs
NyArMzI3LDcgQEAgYWNwaV9udW1hX21lbW9yeV9hZmZpbml0eV9pbml0KGNvbnN0IHN0cnVjdA0K
PiBhY3BpX3NyYXRfbWVtX2FmZmluaXR5ICptYSkNCj4gPiA+ICAJCX0NCj4gPiA+ICAJfSBlbHNl
IHsNCj4gPiA+ICAJCXByaW50ayhLRVJOX0VSUg0KPiA+ID4gLQkJICAgICAgICJTUkFUOiBQWE0g
JXUgKCUiUFJJeDY0Ii0lIlBSSXg2NCIpIG92ZXJsYXBzIHdpdGgNCj4gUFhNICV1ICglIlBSSXg2
NCItJSJQUkl4NjQiKVxuIiwNCj4gPiA+ICsJCSAgICAgICAiU1JBVDogUFhNICV1ICglIlBSSXBh
ZGRyIi0lIlBSSXBhZGRyIikgb3ZlcmxhcHMgd2l0aA0KPiBQWE0gJXUgKCUiUFJJcGFkZHIiLSUi
UFJJcGFkZHIiKVxuIiwNCj4gPiA+ICAJCSAgICAgICBweG0sIHN0YXJ0LCBlbmQsIG5vZGVfdG9f
cHhtKG1lbWJsa19ub2RlaWRbaV0pLA0KPiA+ID4gIAkJICAgICAgIG5vZGVfbWVtYmxrX3Jhbmdl
W2ldLnN0YXJ0LCBub2RlX21lbWJsa19yYW5nZVtpXS5lbmQpOw0KPiA+ID4gIAkJYmFkX3NyYXQo
KTsNCj4gPiA+IEBAIC0zNDYsNyArMzQ2LDcgQEAgYWNwaV9udW1hX21lbW9yeV9hZmZpbml0eV9p
bml0KGNvbnN0IHN0cnVjdA0KPiBhY3BpX3NyYXRfbWVtX2FmZmluaXR5ICptYSkNCj4gPiA+ICAJ
CQkJbmQtPmVuZCA9IGVuZDsNCj4gPiA+ICAJCX0NCj4gPiA+ICAJfQ0KPiA+ID4gLQlwcmludGso
S0VSTl9JTkZPICJTUkFUOiBOb2RlICV1IFBYTSAldSAlIlBSSXg2NCItJSJQUkl4NjQiJXNcbiIs
DQo+ID4gPiArCXByaW50ayhLRVJOX0lORk8gIlNSQVQ6IE5vZGUgJXUgUFhNICV1ICUiUFJJcGFk
ZHIiLSUiUFJJcGFkZHIiJXNcbiIsDQo+ID4gPiAgCSAgICAgICBub2RlLCBweG0sIHN0YXJ0LCBl
bmQsDQo+ID4gPiAgCSAgICAgICBtYS0+ZmxhZ3MgJiBBQ1BJX1NSQVRfTUVNX0hPVF9QTFVHR0FC
TEUgPyAiIChob3RwbHVnKSIgOiAiIik7DQo+ID4gPg0KPiA+ID4gQEAgLTM2OSw3ICszNjksNyBA
QCBzdGF0aWMgaW50IF9faW5pdCBub2Rlc19jb3Zlcl9tZW1vcnkodm9pZCkNCj4gPiA+DQo+ID4g
PiAgCWZvciAoaSA9IDA7IGkgPCBlODIwLm5yX21hcDsgaSsrKSB7DQo+ID4gPiAgCQlpbnQgaiwg
Zm91bmQ7DQo+ID4gPiAtCQl1bnNpZ25lZCBsb25nIGxvbmcgc3RhcnQsIGVuZDsNCj4gPiA+ICsJ
CXBhZGRyX3Qgc3RhcnQsIGVuZDsNCj4gPiA+DQo+ID4gPiAgCQlpZiAoZTgyMC5tYXBbaV0udHlw
ZSAhPSBFODIwX1JBTSkgew0KPiA+ID4gIAkJCWNvbnRpbnVlOw0KPiA+ID4gQEAgLTM5Niw3ICsz
OTYsNyBAQCBzdGF0aWMgaW50IF9faW5pdCBub2Rlc19jb3Zlcl9tZW1vcnkodm9pZCkNCj4gPiA+
DQo+ID4gPiAgCQlpZiAoc3RhcnQgPCBlbmQpIHsNCj4gPiA+ICAJCQlwcmludGsoS0VSTl9FUlIg
IlNSQVQ6IE5vIFBYTSBmb3IgZTgyMCByYW5nZTogIg0KPiA+ID4gLQkJCQkiJTAxNkx4IC0gJTAx
Nkx4XG4iLCBzdGFydCwgZW5kKTsNCj4gPiA+ICsJCQkJIiUiUFJJcGFkZHIiIC0gJSJQUklwYWRk
ciJcbiIsIHN0YXJ0LCBlbmQpOw0KPiA+ID4gIAkJCXJldHVybiAwOw0KPiA+ID4gIAkJfQ0KPiA+
ID4gIAl9DQo+ID4gPiBAQCAtNDMyLDcgKzQzMiw3IEBAIHN0YXRpYyBpbnQgX19pbml0IHNyYXRf
cGFyc2VfcmVnaW9uKHN0cnVjdA0KPiBhY3BpX3N1YnRhYmxlX2hlYWRlciAqaGVhZGVyLA0KPiA+
ID4gIAlyZXR1cm4gMDsNCj4gPiA+ICB9DQo+ID4gPg0KPiA+ID4gLXZvaWQgX19pbml0IHNyYXRf
cGFyc2VfcmVnaW9ucyh1NjQgYWRkcikNCj4gPiA+ICt2b2lkIF9faW5pdCBzcmF0X3BhcnNlX3Jl
Z2lvbnMocGFkZHJfdCBhZGRyKQ0KPiA+ID4gIHsNCj4gPiA+ICAJdTY0IG1hc2s7DQo+ID4gPiAg
CXVuc2lnbmVkIGludCBpOw0KPiA+ID4gQEAgLTQ0MSw3ICs0NDEsNyBAQCB2b2lkIF9faW5pdCBz
cmF0X3BhcnNlX3JlZ2lvbnModTY0IGFkZHIpDQo+ID4gPiAgCSAgICBhY3BpX3RhYmxlX3BhcnNl
KEFDUElfU0lHX1NSQVQsIGFjcGlfcGFyc2Vfc3JhdCkpDQo+ID4gPiAgCQlyZXR1cm47DQo+ID4g
Pg0KPiA+ID4gLQlzcmF0X3JlZ2lvbl9tYXNrID0gcGR4X2luaXRfbWFzayhhZGRyKTsNCj4gPiA+
ICsJc3JhdF9yZWdpb25fbWFzayA9IHBkeF9pbml0X21hc2soKHU2NClhZGRyKTsNCj4gPiA+ICAJ
YWNwaV90YWJsZV9wYXJzZV9zcmF0KEFDUElfU1JBVF9UWVBFX01FTU9SWV9BRkZJTklUWSwNCj4g
PiA+ICAJCQkgICAgICBzcmF0X3BhcnNlX3JlZ2lvbiwgMCk7DQo+ID4gPg0KPiA+ID4gQEAgLTQ1
Nyw3ICs0NTcsNyBAQCB2b2lkIF9faW5pdCBzcmF0X3BhcnNlX3JlZ2lvbnModTY0IGFkZHIpDQo+
ID4gPiAgfQ0KPiA+ID4NCj4gPiA+ICAvKiBVc2UgdGhlIGluZm9ybWF0aW9uIGRpc2NvdmVyZWQg
YWJvdmUgdG8gYWN0dWFsbHkgc2V0IHVwIHRoZSBub2Rlcy4NCj4gKi8NCj4gPiA+IC1pbnQgX19p
bml0IGFjcGlfc2Nhbl9ub2Rlcyh1NjQgc3RhcnQsIHU2NCBlbmQpDQo+ID4gPiAraW50IF9faW5p
dCBhY3BpX3NjYW5fbm9kZXMocGFkZHJfdCBzdGFydCwgcGFkZHJfdCBlbmQpDQo+ID4gPiAgew0K
PiA+ID4gIAlpbnQgaTsNCj4gPiA+ICAJbm9kZW1hc2tfdCBhbGxfbm9kZXNfcGFyc2VkOw0KPiA+
ID4gQEAgLTQ4OSw3ICs0ODksNyBAQCBpbnQgX19pbml0IGFjcGlfc2Nhbl9ub2Rlcyh1NjQgc3Rh
cnQsIHU2NCBlbmQpDQo+ID4gPiAgCS8qIEZpbmFsbHkgcmVnaXN0ZXIgbm9kZXMgKi8NCj4gPiA+
ICAJZm9yX2VhY2hfbm9kZV9tYXNrKGksIGFsbF9ub2Rlc19wYXJzZWQpDQo+ID4gPiAgCXsNCj4g
PiA+IC0JCXU2NCBzaXplID0gbm9kZXNbaV0uZW5kIC0gbm9kZXNbaV0uc3RhcnQ7DQo+ID4gPiAr
CQlwYWRkcl90IHNpemUgPSBub2Rlc1tpXS5lbmQgLSBub2Rlc1tpXS5zdGFydDsNCj4gPiA+ICAJ
CWlmICggc2l6ZSA9PSAwICkNCj4gPiA+ICAJCQlwcmludGsoS0VSTl9XQVJOSU5HICJTUkFUOiBO
b2RlICV1IGhhcyBubyBtZW1vcnkuICINCj4gPiA+ICAJCQkgICAgICAgIkJJT1MgQnVnIG9yIG1p
cy1jb25maWd1cmVkIGhhcmR3YXJlP1xuIiwgaSk7DQo+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2lu
Y2x1ZGUvYXNtLXg4Ni9udW1hLmggYi94ZW4vaW5jbHVkZS9hc20teDg2L251bWEuaA0KPiA+ID4g
aW5kZXggODA2MGNiZjNmNC4uNTBjZmQ4ZTdlZiAxMDA2NDQNCj4gPiA+IC0tLSBhL3hlbi9pbmNs
dWRlL2FzbS14ODYvbnVtYS5oDQo+ID4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20teDg2L251bWEu
aA0KPiA+ID4gQEAgLTE2LDcgKzE2LDcgQEAgZXh0ZXJuIGNwdW1hc2tfdCAgICAgbm9kZV90b19j
cHVtYXNrW107DQo+ID4gPiAgI2RlZmluZSBub2RlX3RvX2NwdW1hc2sobm9kZSkgICAgKG5vZGVf
dG9fY3B1bWFza1tub2RlXSkNCj4gPiA+DQo+ID4gPiAgc3RydWN0IG5vZGUgew0KPiA+ID4gLQl1
NjQgc3RhcnQsZW5kOw0KPiA+ID4gKwlwYWRkcl90IHN0YXJ0LGVuZDsNCj4gPiA+ICB9Ow0KPiA+
ID4NCj4gPiA+ICBleHRlcm4gaW50IGNvbXB1dGVfaGFzaF9zaGlmdChzdHJ1Y3Qgbm9kZSAqbm9k
ZXMsIGludCBudW1ub2RlcywNCj4gPiA+IEBAIC0zNiw3ICszNiw3IEBAIGV4dGVybiB2b2lkIG51
bWFfc2V0X25vZGUoaW50IGNwdSwgbm9kZWlkX3Qgbm9kZSk7DQo+ID4gPiAgZXh0ZXJuIG5vZGVp
ZF90IHNldHVwX25vZGUodW5zaWduZWQgaW50IHB4bSk7DQo+ID4gPiAgZXh0ZXJuIHZvaWQgc3Jh
dF9kZXRlY3Rfbm9kZShpbnQgY3B1KTsNCj4gPiA+DQo+ID4gPiAtZXh0ZXJuIHZvaWQgc2V0dXBf
bm9kZV9ib290bWVtKG5vZGVpZF90IG5vZGVpZCwgdTY0IHN0YXJ0LCB1NjQgZW5kKTsNCj4gPiA+
ICtleHRlcm4gdm9pZCBzZXR1cF9ub2RlX2Jvb3RtZW0obm9kZWlkX3Qgbm9kZWlkLCBwYWRkcl90
IHN0YXJ0LA0KPiBwYWRkcl90IGVuZCk7DQo+ID4gPiAgZXh0ZXJuIG5vZGVpZF90IGFwaWNpZF90
b19ub2RlW107DQo+ID4gPiAgZXh0ZXJuIHZvaWQgaW5pdF9jcHVfdG9fbm9kZSh2b2lkKTsNCj4g
PiA+DQo+ID4gPiBAQCAtNzMsOSArNzMsOSBAQCBzdGF0aWMgaW5saW5lIF9fYXR0cmlidXRlX18o
KHB1cmUpKSBub2RlaWRfdA0KPiBwaHlzX3RvX25pZChwYWRkcl90IGFkZHIpDQo+ID4gPiAgI2Rl
ZmluZSBub2RlX2VuZF9wZm4obmlkKSAgICAgICAoTk9ERV9EQVRBKG5pZCktPm5vZGVfc3RhcnRf
cGZuICsgXA0KPiA+ID4gIAkJCQkgTk9ERV9EQVRBKG5pZCktPm5vZGVfc3Bhbm5lZF9wYWdlcykN
Cj4gPiA+DQo+ID4gPiAtZXh0ZXJuIGludCB2YWxpZF9udW1hX3JhbmdlKHU2NCBzdGFydCwgdTY0
IGVuZCwgbm9kZWlkX3Qgbm9kZSk7DQo+ID4gPiArZXh0ZXJuIGludCB2YWxpZF9udW1hX3Jhbmdl
KHBhZGRyX3Qgc3RhcnQsIHBhZGRyX3QgZW5kLCBub2RlaWRfdA0KPiBub2RlKTsNCj4gPiA+DQo+
ID4gPiAtdm9pZCBzcmF0X3BhcnNlX3JlZ2lvbnModTY0IGFkZHIpOw0KPiA+ID4gK3ZvaWQgc3Jh
dF9wYXJzZV9yZWdpb25zKHBhZGRyX3QgYWRkcik7DQo+ID4gPiAgZXh0ZXJuIHU4IF9fbm9kZV9k
aXN0YW5jZShub2RlaWRfdCBhLCBub2RlaWRfdCBiKTsNCj4gPiA+ICB1bnNpZ25lZCBpbnQgYXJj
aF9nZXRfZG1hX2JpdHNpemUodm9pZCk7DQo+ID4gPiAgdW5zaWduZWQgaW50IGFyY2hfaGF2ZV9k
ZWZhdWx0X2RtYXpvbmUodm9pZCk7DQo+ID4gPiAtLQ0KPiA+ID4gMi4yNS4xDQo+ID4gPg0KPiA+
DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:05:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:05:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194785.347072 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbWR-0003Sz-2E; Fri, 24 Sep 2021 03:05:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194785.347072; Fri, 24 Sep 2021 03:05:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbWQ-0003Ss-Tz; Fri, 24 Sep 2021 03:05:10 +0000
Received: by outflank-mailman (input) for mailman id 194785;
 Fri, 24 Sep 2021 03:05:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbWO-0003Sl-SJ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:05:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 38bc3e24-1ce4-11ec-ba9b-12813bfff9fa;
 Fri, 24 Sep 2021 03:05:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A8BC160EE4;
 Fri, 24 Sep 2021 03:05:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38bc3e24-1ce4-11ec-ba9b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632452706;
	bh=/eA7/u9DaTW6tNWxh5lPmgqSrcwL+FYI1a0bOogpyOM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=cxbYxoMB1iplY7Kz+9HKEjO5N72QFskIAAluWg1zC7IMv9wTOULt8kPHZkl/ETZ9w
	 OHpRnaleRIL+9kj8a944tcEkgIkw84c7PsWwwPl9Z2WOLlS1VVjLge0owiqu16EcmO
	 POuYZaXBJJ86RH1vV+sx2RmZzA8S50ZOIqJR3yEjHwHXoDnqWlMEaaTZV0jxphWATC
	 2rP5WoSQe2DJszKUgEmSfKH6rzUjz8VuhRKdQuYm5Uv6MGzghpvafXYXTRK23zQ8zt
	 c5FDX+GfpzL5gT2LX1ws+ezx1Y9H538eZZUGDBeDegHHANOgB6P/i31KnXMHmO9clG
	 /1UnlBE0hRpcg==
Date: Thu, 23 Sep 2021 20:05:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 31/37] xen/arm: introduce a helper to parse device tree
 NUMA distance map
In-Reply-To: <20210923120236.3692135-32-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231958250.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-32-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> A NUMA aware device tree will provide a "distance-map" node to
> describe distance between any two nodes. This patch introduce a
> new helper to parse this distance map.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/numa_device_tree.c | 106 ++++++++++++++++++++++++++++++++
>  1 file changed, 106 insertions(+)
> 
> diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
> index 7918a397fa..e7fa84df4c 100644
> --- a/xen/arch/arm/numa_device_tree.c
> +++ b/xen/arch/arm/numa_device_tree.c
> @@ -136,3 +136,109 @@ static int __init fdt_parse_numa_memory_node(const void *fdt, int node,
>  
>      return 0;
>  }
> +
> +
> +/* Parse NUMA distance map v1 */
> +static int __init fdt_parse_numa_distance_map_v1(const void *fdt, int node)
> +{
> +    const struct fdt_property *prop;
> +    const __be32 *matrix;
> +    uint32_t entry_count;
> +    int len, i;
> +
> +    printk(XENLOG_INFO "NUMA: parsing numa-distance-map\n");
> +
> +    prop = fdt_get_property(fdt, node, "distance-matrix", &len);
> +    if ( !prop )
> +    {
> +        printk(XENLOG_WARNING
> +               "NUMA: No distance-matrix property in distance-map\n");

I haven't seen where this is called from yet but make sure to print an
error here only if NUMA info is actually expected and required, not on
regular non-NUMA boots on non-NUMA machines.


> +        return -EINVAL;
> +    }
> +
> +    if ( len % sizeof(uint32_t) != 0 )
> +    {
> +        printk(XENLOG_WARNING
> +               "distance-matrix in node is not a multiple of u32\n");
> +        return -EINVAL;
> +    }
> +
> +    entry_count = len / sizeof(uint32_t);
> +    if ( entry_count == 0 )
> +    {
> +        printk(XENLOG_WARNING "NUMA: Invalid distance-matrix\n");
> +
> +        return -EINVAL;
> +    }
> +
> +    matrix = (const __be32 *)prop->data;
> +    for ( i = 0; i + 2 < entry_count; i += 3 )
> +    {
> +        uint32_t from, to, distance, opposite;
> +
> +        from = dt_next_cell(1, &matrix);
> +        to = dt_next_cell(1, &matrix);
> +        distance = dt_next_cell(1, &matrix);
> +        if ( (from == to && distance != NUMA_LOCAL_DISTANCE) ||
> +            (from != to && distance <= NUMA_LOCAL_DISTANCE) )
> +        {
> +            printk(XENLOG_WARNING
> +                   "NUMA: Invalid distance: NODE#%u->NODE#%u:%u\n",
> +                   from, to, distance);
> +            return -EINVAL;
> +        }
> +
> +        printk(XENLOG_INFO "NUMA: distance: NODE#%u->NODE#%u:%u\n",
> +               from, to, distance);
> +
> +        /* Get opposite way distance */
> +        opposite = __node_distance(from, to);

This is not checking for the opposite node distance but...


> +        if ( opposite == 0 )
> +        {
> +            /* Bi-directions are not set, set both */
> +            numa_set_distance(from, to, distance);
> +            numa_set_distance(to, from, distance);

...since you set both directions here at once then it is OK. You are
checking if this direction has already been set which is correct I
think. But the comment "Get opposite way distance" and the variable name
"opposite" are wrong.


> +        }
> +        else
> +        {
> +            /*
> +             * Opposite way distance has been set to a different value.
> +             * It may be a firmware device tree bug?
> +             */
> +            if ( opposite != distance )
> +            {
> +                /*
> +                 * In device tree NUMA distance-matrix binding:
> +                 * https://www.kernel.org/doc/Documentation/devicetree/bindings/numa.txt
> +                 * There is a notes mentions:
> +                 * "Each entry represents distance from first node to
> +                 *  second node. The distances are equal in either
> +                 *  direction."
> +                 *
> +                 * That means device tree doesn't permit this case.
> +                 * But in ACPI spec, it cares to specifically permit this
> +                 * case:
> +                 * "Except for the relative distance from a System Locality
> +                 *  to itself, each relative distance is stored twice in the
> +                 *  matrix. This provides the capability to describe the
> +                 *  scenario where the relative distances for the two
> +                 *  directions between System Localities is different."
> +                 *
> +                 * That means a real machine allows such NUMA configuration.
> +                 * So, place a WARNING here to notice system administrators,
> +                 * is it the specail case that they hijack the device tree
> +                 * to support their rare machines?
> +                 */
> +                printk(XENLOG_WARNING
> +                       "Un-matched bi-direction! NODE#%u->NODE#%u:%u, NODE#%u->NODE#%u:%u\n",
> +                       from, to, distance, to, from, opposite);

PRIu32


> +            }
> +
> +            /* Opposite way distance has been set, just set this way */
> +            numa_set_distance(from, to, distance);
> +        }
> +    }
> +
> +    return 0;
> +}
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:05:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:05:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194787.347082 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbWa-0003mn-92; Fri, 24 Sep 2021 03:05:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194787.347082; Fri, 24 Sep 2021 03:05:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbWa-0003mg-6B; Fri, 24 Sep 2021 03:05:20 +0000
Received: by outflank-mailman (input) for mailman id 194787;
 Fri, 24 Sep 2021 03:05:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbWZ-0003m4-D6
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:05:19 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3f4306e2-1ce4-11ec-ba9b-12813bfff9fa;
 Fri, 24 Sep 2021 03:05:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B69C561100;
 Fri, 24 Sep 2021 03:05:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f4306e2-1ce4-11ec-ba9b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632452717;
	bh=zPiaRw7Cz/4I+LwHkh6RTKYtKX7KOesVkVjlk0A4+cw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EnKN3lalzWoCdXne81kvbfgkUTKoJmqtwspMo1y9x96Zifq5dQo8VgAwS0joKmZgr
	 VT0x/KZ6jaoyQD+Yhip83BTEKjmXk+t2Z+ApY2kZO6n9J34PI3qQcJsdl8MIWV8ZCj
	 EU7W8j9ayS/DVVQ8TAvQle5yeM9AcfW1u0NINZiIbv5HMoQimi25+7UZlQPqt4h/PU
	 xVe9Trv1LIgYiNyJAsuaDpnvQTZ+9adSrVd2JxkyOweRfibGcJr+3SunRH1VSegjgq
	 Kv5pUn3fwyCmYkzV1AIVvdjjtYBZecsMrF7UL6hIpiBBlj4GchPulq0D8aZ3lpHejE
	 /QGyaVqqyd6FA==
Date: Thu, 23 Sep 2021 20:05:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 30/37] xen/arm: introduce a helper to parse device tree
 memory node
In-Reply-To: <20210923120236.3692135-31-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109231945190.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-31-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Memory blocks' NUMA ID information is stored in device tree's
> memory nodes as "numa-node-id". We need a new helper to parse
> and verify this ID from memory nodes.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>

There are tabs for indentation in this patch, we use spaces.


> ---
>  xen/arch/arm/numa_device_tree.c | 80 +++++++++++++++++++++++++++++++++
>  1 file changed, 80 insertions(+)
> 
> diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
> index 2428fbae0b..7918a397fa 100644
> --- a/xen/arch/arm/numa_device_tree.c
> +++ b/xen/arch/arm/numa_device_tree.c
> @@ -42,6 +42,35 @@ static int __init fdt_numa_processor_affinity_init(nodeid_t node)
>      return 0;
>  }
>  
> +/* Callback for parsing of the memory regions affinity */
> +static int __init fdt_numa_memory_affinity_init(nodeid_t node,
> +                                paddr_t start, paddr_t size)

Please align the parameters


> +{
> +    int ret;
> +
> +    if ( srat_disabled() )
> +    {
> +        return -EINVAL;
> +    }
> +
> +	if ( !numa_memblks_available() )
> +	{
> +		dprintk(XENLOG_WARNING,
> +                "Too many numa entry, try bigger NR_NODE_MEMBLKS \n");
> +		bad_srat();
> +		return -EINVAL;
> +	}
> +
> +	ret = numa_update_node_memblks(node, start, size, false);
> +	if ( ret != 0 )
> +	{
> +		bad_srat();
> +	    return -EINVAL;
> +	}
> +
> +    return 0;
> +}

Aside from spaces/tabs, this is a lot better!


>  /* Parse CPU NUMA node info */
>  static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
>  {
> @@ -56,3 +85,54 @@ static int __init fdt_parse_numa_cpu_node(const void *fdt, int node)
>  
>      return fdt_numa_processor_affinity_init(nid);
>  }
> +
> +/* Parse memory node NUMA info */
> +static int __init fdt_parse_numa_memory_node(const void *fdt, int node,
> +    const char *name, uint32_t addr_cells, uint32_t size_cells)

Please align the parameters


> +{
> +    uint32_t nid;
> +    int ret = 0, len;
> +    paddr_t addr, size;
> +    const struct fdt_property *prop;
> +    uint32_t idx, ranges;
> +    const __be32 *addresses;
> +
> +    nid = device_tree_get_u32(fdt, node, "numa-node-id", MAX_NUMNODES);
> +    if ( nid >= MAX_NUMNODES )
> +    {
> +        printk(XENLOG_WARNING "Node id %u exceeds maximum value\n", nid);
> +        return -EINVAL;
> +    }
> +
> +    prop = fdt_get_property(fdt, node, "reg", &len);
> +    if ( !prop )
> +    {
> +        printk(XENLOG_WARNING
> +               "fdt: node `%s': missing `reg' property\n", name);
> +        return -EINVAL;
> +    }
> +
> +    addresses = (const __be32 *)prop->data;
> +    ranges = len / (sizeof(__be32)* (addr_cells + size_cells));
> +    for ( idx = 0; idx < ranges; idx++ )
> +    {
> +        device_tree_get_reg(&addresses, addr_cells, size_cells, &addr, &size);
> +        /* Skip zero size ranges */
> +        if ( !size )
> +            continue;
> +
> +        ret = fdt_numa_memory_affinity_init(nid, addr, size);
> +        if ( ret ) {
> +            return -EINVAL;
> +        }
> +    }

I take it would be difficult to parse numa-node-id and call
fdt_numa_memory_affinity_init from
xen/arch/arm/bootfdt.c:device_tree_get_meminfo. Is it because
device_tree_get_meminfo is called too early?


> +    if ( idx == 0 )
> +    {
> +        printk(XENLOG_ERR
> +               "bad property in memory node, idx=%d ret=%d\n", idx, ret);
> +        return -EINVAL;
> +    }
> +
> +    return 0;
> +}
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:17:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:17:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194794.347094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbhj-0005rG-Bd; Fri, 24 Sep 2021 03:16:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194794.347094; Fri, 24 Sep 2021 03:16:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbhj-0005r9-8n; Fri, 24 Sep 2021 03:16:51 +0000
Received: by outflank-mailman (input) for mailman id 194794;
 Fri, 24 Sep 2021 03:16:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbhi-0005r3-GM
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:16:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id daf8d1ce-1ce5-11ec-ba9e-12813bfff9fa;
 Fri, 24 Sep 2021 03:16:49 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5A60660F44;
 Fri, 24 Sep 2021 03:16:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: daf8d1ce-1ce5-11ec-ba9e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632453408;
	bh=ltAcXF8jHCCBFR25qG2ATZFsI5V5NFZLpl2oTefcOCY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=C8V3hPJbB4qCRJ5bmSu2RuhnpmiAHQ3Jc10IObpcxKQe+ng8P6CFHHdVf3YE8g9X/
	 ZT4RFGM2kXnMlGEh6I3MEKbL4lOwQUoGrsXFcUFU9RJfN1gHXqDP6GRfCpBicf5HqW
	 cocyMF0byPiwQZIXx/p2NOFktoRN2/Oxkzs9IDH9ZKDeHaQXAHJMGUB+XOVI9ZCWwc
	 g5pGxz6NwP/b9XsR3aek7STtVATgAcgFLErgiHFdgDyTgZihFnDURAIHhtwXmoFeZh
	 ER4yIX3kWWb8u9Ma+mOOiYJSwQK8fOyCVVJQXwIeTzh3c6rgo/WQt1h7m7qCaCCdMh
	 aS4CgN+NZLfeg==
Date: Thu, 23 Sep 2021 20:16:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from
 device tree
In-Reply-To: <20210923120236.3692135-33-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109232007160.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-33-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> In this API, we scan whole device tree to parse CPU node id, memory
          ^ function   ^ the whole

> node id and distance-map. Though early_scan_node will invoke has a
> handler to process memory nodes. If we want to parse memory node id
> in this handler, we have to embeded NUMA parse code in this handler.
                              ^ embed

> But we still need to scan whole device tree to find CPU NUMA id and
> distance-map. In this case, we include memory NUMA id parse in this
> API too. Another benefit is that we have a unique entry for device
  ^ function

> tree NUMA data parse.

Ah, that's the explanation I was asking for earlier!


> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/numa_device_tree.c | 30 ++++++++++++++++++++++++++++++
>  xen/include/asm-arm/numa.h      |  1 +
>  2 files changed, 31 insertions(+)
> 
> diff --git a/xen/arch/arm/numa_device_tree.c b/xen/arch/arm/numa_device_tree.c
> index e7fa84df4c..6a3fed0002 100644
> --- a/xen/arch/arm/numa_device_tree.c
> +++ b/xen/arch/arm/numa_device_tree.c
> @@ -242,3 +242,33 @@ static int __init fdt_parse_numa_distance_map_v1(const void *fdt, int node)
>  
>      return 0;
>  }
> +
> +static int __init fdt_scan_numa_nodes(const void *fdt,
> +                int node, const char *uname, int depth,
> +                u32 address_cells, u32 size_cells, void *data)

Please align parameters


> +{
> +    int len, ret = 0;
> +    const void *prop;
> +
> +    prop = fdt_getprop(fdt, node, "device_type", &len);
> +    if (prop)

code style


> +    {
> +        len += 1;
> +        if ( memcmp(prop, "cpu", len) == 0 )
> +            ret = fdt_parse_numa_cpu_node(fdt, node);
> +        else if ( memcmp(prop, "memory", len) == 0 )
> +            ret = fdt_parse_numa_memory_node(fdt, node, uname,
> +                                address_cells, size_cells);

I realize that with the inclusion of '\0' in the check, the usage of
memcmp should be safe, but I would prefer if we used strncmp instead.


> +    }
> +    else if ( fdt_node_check_compatible(fdt, node,
> +                                "numa-distance-map-v1") == 0 )
> +        ret = fdt_parse_numa_distance_map_v1(fdt, node);
> +
> +    return ret;
> +}
> +
> +/* Initialize NUMA from device tree */
> +int __init numa_device_tree_init(const void *fdt)
> +{
> +    return device_tree_for_each_node(fdt, 0, fdt_scan_numa_nodes, NULL);
> +}
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 7675012cb7..f46e8e2935 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -23,6 +23,7 @@ typedef u8 nodeid_t;
>  #define NR_NODE_MEMBLKS NR_MEM_BANKS
>  
>  extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
> +extern int numa_device_tree_init(const void *fdt);
>  
>  #else
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:19:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:19:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194798.347105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbkD-0006r4-Pf; Fri, 24 Sep 2021 03:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194798.347105; Fri, 24 Sep 2021 03:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbkD-0006qx-MC; Fri, 24 Sep 2021 03:19:25 +0000
Received: by outflank-mailman (input) for mailman id 194798;
 Fri, 24 Sep 2021 03:19:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbkC-0006qr-62
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:19:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 36e9d6fe-1ce6-11ec-ba9e-12813bfff9fa;
 Fri, 24 Sep 2021 03:19:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AB31161211;
 Fri, 24 Sep 2021 03:19:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36e9d6fe-1ce6-11ec-ba9e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632453562;
	bh=798yw/5WSiC7yw35GxgVS2qF173BV6xPb14iN2pYQKw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pUR2YtwzZPJpnqaUzEenRow7j02u4ykhB329rbUuxR72NxIkFv9op6OFMsIQ71OMt
	 YPUe/GsnElgKz2Z8A5VFvd0c7Hu/6dpwBWwjy8e+tSrYT7Pf+SC9CddWxuTcFstak4
	 9Ub0CStlk8FXM7RFfp6OxpWoijv2LNb2J7Ms4ExkNtV1B4pAn+GqlDCUpqoM4ZMmoy
	 IPGMQt0orWgYuySWSF9armc2LuaY6izaiBGAnsqIe3cUA4vVkYYImCbDSPjrc46MwM
	 jc+qkb+TUXrPl65C/Af4MW4PbbI+0TjKCAcqRVJz9nNPQZ3ScnQlMlycXy/uVTqhZA
	 XLJy4jra9xrDw==
Date: Thu, 23 Sep 2021 20:19:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 33/37] xen/arm: keep guest still be NUMA unware
In-Reply-To: <20210923120236.3692135-34-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109232018070.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-34-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> The NUMA information provided in the host Device-Tree
> are only for Xen. For dom0, we want to hide them as they
> may be different (for now, dom0 is still not aware of NUMA)
> The CPU and memory nodes are recreated from scratch for the
> domain. So we already skip the "numa-node-id" property for
> these two types of nodes.
> 
> However, some devices like PCIe may have "numa-node-id"
> property too. We have to skip them as well.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/domain_build.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d634c1..6e94922238 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -737,6 +737,10 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                  continue;
>          }
>  
> +        /* Guest is numa unaware in current stage */

I would say: "Dom0 is currently NUMA unaware"

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> +        if ( dt_property_name_is_equal(prop, "numa-node-id") )
> +            continue;
> +
>          res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>  
>          if ( res )
> @@ -1607,6 +1611,8 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
>          DT_MATCH_TYPE("memory"),
>          /* The memory mapped timer is not supported by Xen. */
>          DT_MATCH_COMPATIBLE("arm,armv7-timer-mem"),
> +        /* Numa info doesn't need to be exposed to Domain-0 */
> +        DT_MATCH_COMPATIBLE("numa-distance-map-v1"),
>          { /* sentinel */ },
>      };
>      static const struct dt_device_match timer_matches[] __initconst =
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:28:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:28:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194806.347116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbsu-0000CX-M8; Fri, 24 Sep 2021 03:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194806.347116; Fri, 24 Sep 2021 03:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbsu-0000CQ-Ha; Fri, 24 Sep 2021 03:28:24 +0000
Received: by outflank-mailman (input) for mailman id 194806;
 Fri, 24 Sep 2021 03:28:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbst-0000CK-2S
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:28:23 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 77f992c8-1ce7-11ec-baa2-12813bfff9fa;
 Fri, 24 Sep 2021 03:28:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0DC6B61164;
 Fri, 24 Sep 2021 03:28:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77f992c8-1ce7-11ec-baa2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632454101;
	bh=Sn/tzhQHKN7SLExWX1gi6S586dhysD4Nzf2jsLV9Un0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=S0eS8354qu+tj5IttsByGX8x+GjgdsrrKhZym+iPDobwHaCMl1ecfEIcZhxSW20AM
	 BOmYSbdNab8oDsuBgtKyrgBWzChbkx+svxXsKOTIt5mnynr+eucn81RifPeGy9vL4L
	 JHV/5RraFAOfZRLtRSM4g+q/qKppeM5F73774GxmqPvpbbWg3ZCBb1BxSxHRY6eRLI
	 hnZbKuR93Mo4iRDo+2PO2HH3wA2C2pjd2Y12IQsmU7KlkLP5gqI5oWG+WdKutY93AK
	 uNuRZ2z+IKpNmbVOhgrE2c5DxNsoGg+b27nGctEksbSNCmULn1+2MYYFygC0JsQMw5
	 etiqiCJMxXLIQ==
Date: Thu, 23 Sep 2021 20:28:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 34/37] xen/arm: enable device tree based NUMA in system
 init
In-Reply-To: <20210923120236.3692135-35-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109232021270.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-35-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> In this patch, we can start to create NUMA system that is
> based on device tree.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/numa.c        | 55 ++++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/setup.c       |  7 +++++
>  xen/include/asm-arm/numa.h |  6 +++++
>  3 files changed, 68 insertions(+)
> 
> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> index 7f05299b76..d7a3d32d4b 100644
> --- a/xen/arch/arm/numa.c
> +++ b/xen/arch/arm/numa.c
> @@ -18,8 +18,10 @@
>   *
>   */
>  #include <xen/init.h>
> +#include <xen/device_tree.h>
>  #include <xen/nodemask.h>
>  #include <xen/numa.h>
> +#include <xen/pfn.h>
>  
>  static uint8_t __read_mostly
>  node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
> @@ -85,6 +87,59 @@ uint8_t __node_distance(nodeid_t from, nodeid_t to)
>  }
>  EXPORT_SYMBOL(__node_distance);
>  
> +void __init numa_init(bool acpi_off)
> +{
> +    uint32_t idx;
> +    paddr_t ram_start = ~0;

INVALID_PADDR


> +    paddr_t ram_size = 0;
> +    paddr_t ram_end = 0;
> +
> +    /* NUMA has been turned off through Xen parameters */
> +    if ( numa_off )
> +        goto mem_init;
> +
> +    /* Initialize NUMA from device tree when system is not ACPI booted */
> +    if ( acpi_off )
> +    {
> +        int ret = numa_device_tree_init(device_tree_flattened);
> +        if ( ret )
> +        {
> +            printk(XENLOG_WARNING
> +                   "Init NUMA from device tree failed, ret=%d\n", ret);

As I mentioned in other patches we need to distinguish between two
cases:

1) NUMA initialization failed because no NUMA information has been found
2) NUMA initialization failed because wrong/inconsistent NUMA info has
   been found

In case of 1), we print nothing. Maybe a single XENLOG_DEBUG message.
In case of 2), all the warnings are good to print.


In this case, if ret != 0 because of 2), then it is fine to print this
warning. But it looks like could be that ret is -EINVAL simply because a
CPU node doesn't have numa-node-id, which is a normal condition for
non-NUMA machines.


> +            numa_off = true;
> +        }
> +    }
> +    else
> +    {
> +        /* We don't support NUMA for ACPI boot currently */
> +        printk(XENLOG_WARNING
> +               "ACPI NUMA has not been supported yet, NUMA off!\n");
> +        numa_off = true;
> +    }
> +
> +mem_init:
> +    /*
> +     * Find the minimal and maximum address of RAM, NUMA will
> +     * build a memory to node mapping table for the whole range.
> +     */
> +    ram_start = bootinfo.mem.bank[0].start;
> +    ram_size  = bootinfo.mem.bank[0].size;
> +    ram_end   = ram_start + ram_size;
> +    for ( idx = 1 ; idx < bootinfo.mem.nr_banks; idx++ )
> +    {
> +        paddr_t bank_start = bootinfo.mem.bank[idx].start;
> +        paddr_t bank_size = bootinfo.mem.bank[idx].size;
> +        paddr_t bank_end = bank_start + bank_size;
> +
> +        ram_size  = ram_size + bank_size;
> +        ram_start = min(ram_start, bank_start);
> +        ram_end   = max(ram_end, bank_end);
> +    }
> +
> +    numa_initmem_init(PFN_UP(ram_start), PFN_DOWN(ram_end));
> +    return;

No need for return


> +}
> +
>  uint32_t __init arch_meminfo_get_nr_bank(void)
>  {
>  	return bootinfo.mem.nr_banks;
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 1f0fbc95b5..6097850682 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -905,6 +905,13 @@ void __init start_xen(unsigned long boot_phys_offset,
>      /* Parse the ACPI tables for possible boot-time configuration */
>      acpi_boot_table_init();
>  
> +    /*
> +     * Try to initialize NUMA system, if failed, the system will
> +     * fallback to uniform system which means system has only 1
> +     * NUMA node.
> +     */
> +    numa_init(acpi_disabled);
> +
>      end_boot_allocator();
>  
>      /*
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index f46e8e2935..5b03dde87f 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -24,6 +24,7 @@ typedef u8 nodeid_t;
>  
>  extern void numa_set_distance(nodeid_t from, nodeid_t to, uint32_t distance);
>  extern int numa_device_tree_init(const void *fdt);
> +extern void numa_init(bool acpi_off);
>  
>  #else
>  
> @@ -47,6 +48,11 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +static inline void numa_init(bool acpi_off)
> +{
> +
> +}
> +
>  static inline void numa_add_cpu(int cpu)
>  {
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:31:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:31:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194813.347127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbw5-0001dD-A9; Fri, 24 Sep 2021 03:31:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194813.347127; Fri, 24 Sep 2021 03:31:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTbw5-0001d6-64; Fri, 24 Sep 2021 03:31:41 +0000
Received: by outflank-mailman (input) for mailman id 194813;
 Fri, 24 Sep 2021 03:31:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTbw3-0001d0-QS
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 03:31:39 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b45617d-3942-436f-b594-d91a3dd76e3f;
 Fri, 24 Sep 2021 03:31:38 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 09A6860F6F;
 Fri, 24 Sep 2021 03:31:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b45617d-3942-436f-b594-d91a3dd76e3f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632454298;
	bh=dAdDLoaWbNDuoSkjnKGL+RhFO5NZr9gkXUVwjNilcLQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jRuaJu6boPmN8vOfdd/KG3oFejou9jOCQAAowC3ovFGSMEtnk6FGSvBjohmGmW1HY
	 Vs8nbrLsarK4YHZZS8AJS3yhhUNlrgwhIdCpvRPLmBUKznk4m8THgjDkWtHvoa0Afp
	 E73GeFm36ei1so5W4LJa82ct/UG+R7gC3SfqKPuIbYlviu7/vAURmuezWKwFYxjVTY
	 dV16s7lsWH5LdV2y5urxJuLyomIQvyEHdpq/cbqCg4FHwoiLvak7XavcrBlNLjswJ8
	 u6UMvZciBnn223a/RApyvFVwQOtT5N9ubiuI313aJqfHP/8/KS6t3g5SkuKI/J3q5W
	 43yFNaeRAeIyA==
Date: Thu, 23 Sep 2021 20:31:36 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <wei.chen@arm.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, julien@xen.org, 
    Bertrand.Marquis@arm.com
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
In-Reply-To: <20210923120236.3692135-37-wei.chen@arm.com>
Message-ID: <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-37-wei.chen@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Wei Chen wrote:
> Arm platforms support both ACPI and device tree. We don't
> want users to select device tree NUMA or ACPI NUMA manually.
> We hope usrs can just enable NUMA for Arm, and device tree
          ^ users

> NUMA and ACPI NUMA can be selected depends on device tree
> feature and ACPI feature status automatically. In this case,
> these two kinds of NUMA support code can be co-exist in one
> Xen binary. Xen can check feature flags to decide using
> device tree or ACPI as NUMA based firmware.
> 
> So in this patch, we introduce a generic option:
> CONFIG_ARM_NUMA for user to enable NUMA for Arm.
                      ^ users

> And one CONFIG_DEVICE_TREE_NUMA option for ARM_NUMA
> to select when HAS_DEVICE_TREE option is enabled.
> Once when ACPI NUMA for Arm is supported, ACPI_NUMA
> can be selected here too.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>
> ---
>  xen/arch/arm/Kconfig | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 865ad83a89..ded94ebd37 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -34,6 +34,17 @@ config ACPI
>  	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>  	  an alternative to device tree on ARM64.
>  
> + config DEVICE_TREE_NUMA
> +	def_bool n
> +	select NUMA
> +
> +config ARM_NUMA
> +	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
> +	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE

Should it be: depends on HAS_DEVICE_TREE ?
(And eventually depends on HAS_DEVICE_TREE || ACPI)


> +	---help---
> +
> +	  Enable Non-Uniform Memory Access (NUMA) for Arm architecutres
                                                      ^ architectures


> +
>  config GICV3
>  	bool "GICv3 driver"
>  	depends on ARM_64 && !NEW_VGIC
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:40:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:40:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194819.347138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTc4U-00033t-5L; Fri, 24 Sep 2021 03:40:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194819.347138; Fri, 24 Sep 2021 03:40:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTc4U-00033m-0S; Fri, 24 Sep 2021 03:40:22 +0000
Received: by outflank-mailman (input) for mailman id 194819;
 Fri, 24 Sep 2021 03:40:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTc4S-00033c-Ps; Fri, 24 Sep 2021 03:40:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTc4S-0007Ob-HL; Fri, 24 Sep 2021 03:40:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTc4S-0004jP-4u; Fri, 24 Sep 2021 03:40:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTc4S-0002Ap-48; Fri, 24 Sep 2021 03:40:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ziOD/HH4ztMnN1DM82L9jLF0P6HiSo25QW4uYxJW0aE=; b=qK/vJIQX6mTB0F3RxHvSjl9P1J
	1j0LBz41TAIRUgLBM042MHXk9j+aG0PDRIioRInXfr3yKBCm6r0Rzq7J3Q9hOC0rYvD8ajNibVhVL
	WvrUvNQi+nKjj5fPbkoZIXB3EMQvtp2uPwMXxtVqJ8i0d57BKKLbDV+BatBNw0TYphuw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165168-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165168: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=604be1b333b1b66052ab9b0133f156890549a4f0
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 03:40:20 +0000

flight 165168 xen-unstable real [real]
flight 165172 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165168/
http://logs.test-lab.xenproject.org/osstest/logs/165172/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  604be1b333b1b66052ab9b0133f156890549a4f0
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   13 days
Failing since        164951  2021-09-12 00:14:36 Z   12 days   24 attempts
Testing same since   165161  2021-09-23 03:54:29 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 899 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:46:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:46:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194824.347151 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcAF-00045d-Qc; Fri, 24 Sep 2021 03:46:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194824.347151; Fri, 24 Sep 2021 03:46:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcAF-00045W-Nd; Fri, 24 Sep 2021 03:46:19 +0000
Received: by outflank-mailman (input) for mailman id 194824;
 Fri, 24 Sep 2021 03:46:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcAE-00045L-Mp; Fri, 24 Sep 2021 03:46:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcAE-0007UP-D6; Fri, 24 Sep 2021 03:46:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcAD-0004uW-VY; Fri, 24 Sep 2021 03:46:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcAD-0006Qr-V3; Fri, 24 Sep 2021 03:46:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=spwDjb3wljFjes/H+y+aLWaimmSrThkqXKWq8zLkQLM=; b=YiSvksllC/3iPiKhW5I25dWeb7
	wMQ969MKYez5cl2rPCcJ81fGzKut6W/dXZITIoBt1MuO8ILkq5nCtelF1Q2quD14H9oHIRSNEEERt
	fQbcK/4a51LMMqi4XiIU5XSGIELZZ3OffzccG/jj/MOqd+r1fUW7XDlyXk1jupXJjRVA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165170-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165170: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=7ea7f9c07757b9445c24b23acf4c2e8e60b30b7e
X-Osstest-Versions-That:
    ovmf=79019c7a42287e8591ec98487db0e849384a74e1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 03:46:17 +0000

flight 165170 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165170/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 7ea7f9c07757b9445c24b23acf4c2e8e60b30b7e
baseline version:
 ovmf                 79019c7a42287e8591ec98487db0e849384a74e1

Last test of basis   165155  2021-09-22 12:40:08 Z    1 days
Testing same since   165170  2021-09-23 18:41:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jeff Brasen <jbrasen@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   79019c7a42..7ea7f9c077  7ea7f9c07757b9445c24b23acf4c2e8e60b30b7e -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 03:50:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 03:50:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194832.347167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcEh-0005tn-MY; Fri, 24 Sep 2021 03:50:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194832.347167; Fri, 24 Sep 2021 03:50:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcEh-0005tg-H7; Fri, 24 Sep 2021 03:50:55 +0000
Received: by outflank-mailman (input) for mailman id 194832;
 Fri, 24 Sep 2021 03:50:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcEg-0005tW-Ra; Fri, 24 Sep 2021 03:50:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcEg-0007Z7-OD; Fri, 24 Sep 2021 03:50:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcEg-00051q-En; Fri, 24 Sep 2021 03:50:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTcEg-0008B8-EJ; Fri, 24 Sep 2021 03:50:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LN1D9VwlKlyBqC3MzYpFUslOKryxXQr9/gEkgjK2toA=; b=crkecqOJc8xnzfB6v1peo+f5nX
	eYLJ9uDRyGIdb/p8vyCNNPXv4mkX5EDHHK3nHwUey09dn6unFTVFTJJygPHQTv8R71uVJULHjfsdF
	DmfsM8nS8soFh9yZz61P+ThzZSlD66hL7T90O9EQ0lIlNqDKiVwuTfBvA8Mdp6pVRKdA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165169-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165169: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=58e2cf5d794616b84f591d4d1276c8953278ce24
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 03:50:54 +0000

flight 165169 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165169/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                58e2cf5d794616b84f591d4d1276c8953278ce24
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  419 days
Failing since        152366  2020-08-01 20:49:34 Z  418 days  731 attempts
Testing same since   165163  2021-09-23 07:54:43 Z    0 days    2 attempts

------------------------------------------------------------
7324 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2263977 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:22:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:22:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194851.347179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcj3-0001p2-RB; Fri, 24 Sep 2021 04:22:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194851.347179; Fri, 24 Sep 2021 04:22:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcj3-0001ov-OI; Fri, 24 Sep 2021 04:22:17 +0000
Received: by outflank-mailman (input) for mailman id 194851;
 Fri, 24 Sep 2021 04:22:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcj2-0001op-BF
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:22:16 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.73]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42a149e4-f3d1-4a43-862d-6328430a4f54;
 Fri, 24 Sep 2021 04:22:13 +0000 (UTC)
Received: from AM6PR10CA0020.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:89::33)
 by AS8PR08MB6390.eurprd08.prod.outlook.com (2603:10a6:20b:31b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:22:12 +0000
Received: from AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:89:cafe::e6) by AM6PR10CA0020.outlook.office365.com
 (2603:10a6:209:89::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:22:12 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT063.mail.protection.outlook.com (10.152.16.226) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:22:11 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Fri, 24 Sep 2021 04:22:10 +0000
Received: from ba69f07cbf5d.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A811D178-BB77-4615-B5D3-BF1A83F15D30.1; 
 Fri, 24 Sep 2021 04:21:59 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ba69f07cbf5d.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:21:59 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PR2PR08MB4668.eurprd08.prod.outlook.com (2603:10a6:101:1e::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:21:51 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:21:51 +0000
Received: from [10.169.190.22] (203.126.0.113) by
 SG2PR04CA0156.apcprd04.prod.outlook.com (2603:1096:4::18) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:21:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42a149e4-f3d1-4a43-862d-6328430a4f54
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=70BDfoH8abFUJ4M2MbLLGe5j0d8DOpxenlSyI3vyyvs=;
 b=Ig43jKYQwXTYjHau/4p+BpK1E71DmCSodBQSeVu/7+/hkh0vef2jLEZZ49vI907OMgA1S1UgCVqtAFToyAH7gs3SbUxQQ2cqTAgMNK0zN7nmv7aFAxuj7XUi6We2xoW8NigAhd6EsR45/uyAtJ8Rp3tBxewY4cN04f2l/8hFx4Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 18c2473c1d1b7452
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IrHjkxOIP9NzLUAeor0glvLZIm4eq+OjwjunGFAnvGhjAdbrGtvd+rfWog76SK2dkFbZ3jQyQvVbIIEeB89GXb0qS52TuDp2NISou0Ga+4OoOVRVRtxLZeoBYeeADj31TCrmXRYzCUhhLTl4EAM8KEbMhrjjvfvG/1726/xyVg+7Nvk5UJ3QRxMK6o+U1pv3LyksHC1RDO6Erv09KsPO6tRdFH46CemS6J5gwsrV0ixu9f7qKUQGxzuK5jOV0RxJxwjJmIC9FJBRvriHq3b2/jjvejjyShgcYsMj9vhUNrV2bSpLYzYLX5pU/r3sLMYK1Y+a3mEGXamTJXH2wyXwOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=70BDfoH8abFUJ4M2MbLLGe5j0d8DOpxenlSyI3vyyvs=;
 b=C6BRpxlsDcbMg9FRXZMXIOdCaytzB1YRNyJYz/zCShJlYJwGA9RCxR0Oq8dTNnR2LbMQQO9hKdsgZbMgg5JGSCbjHzKaHE7tKC6AE3NukiJc0LFWlwT3O5PBWM3XjhwzxSj3rmWEQKrSZk0zz/xxgStLtYjGbqpfmDokRA5XAclXKwpXF4VRx/NlfMBldsb7X7BqV64Bgmty1X9cjzRbjFQ6hHCKt0HDBUYxWTm2m5CN0wF3/ACagZ8Ck3yC17iR3UXb6GJOgmL6xIW+VjxaIb+ASHnVC9GPGmsiaLMwmpxXh7UPyt0iBAQgq22FuF6sdkcqpZfa9NL/Zj/QXbqMJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=70BDfoH8abFUJ4M2MbLLGe5j0d8DOpxenlSyI3vyyvs=;
 b=Ig43jKYQwXTYjHau/4p+BpK1E71DmCSodBQSeVu/7+/hkh0vef2jLEZZ49vI907OMgA1S1UgCVqtAFToyAH7gs3SbUxQQ2cqTAgMNK0zN7nmv7aFAxuj7XUi6We2xoW8NigAhd6EsR45/uyAtJ8Rp3tBxewY4cN04f2l/8hFx4Q=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Message-ID: <d4a91b84-5a71-e554-8bac-850946bd5b28@arm.com>
Date: Fri, 24 Sep 2021 12:21:43 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Subject: Re: [PATCH 09/37] xen/x86: introduce two helpers to access memory
 hotplug end
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com,
 jbeulich@suse.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 wl@xen.org
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-10-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231727560.17979@sstabellini-ThinkPad-T480s>
From: Wei Chen <Wei.Chen@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109231727560.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: SG2PR04CA0156.apcprd04.prod.outlook.com (2603:1096:4::18)
 To PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 61d8cc8b-e784-448e-faa5-08d97f12e18d
X-MS-TrafficTypeDiagnostic: PR2PR08MB4668:|AS8PR08MB6390:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB63906E3018818C45727841D69EA49@AS8PR08MB6390.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 bKYYsccn17M1ULiCa86iVP9qsWr/Klh7uDE5nealHjyy3nnAmLY0jZU/vwlMltUCL0nkfaidRyJAVYgHENY8LRcANtawxme2wEo1Og6xlWLpxw51vKsljYl/Ph/TlHxkDYW4v7ni6KJcBpPwOgCrvo249aUm9SNCmJUZQBh8EHU5Kse1TtNpGsq60/weisiQrplv4g86sie1kV1ls5UPSWpGSPYPol21S0370BlUkdWtTZGa6+sBRNJZX3JHP2/LbZYzk2TYWxJpfMfKTX33tTef1ykaNZ6PELc8Q948auciZigy66ZHTE7Gs0Lzi6l6gxX748X/w2OSpW8BPstQDjXm3TrUIDmzO6KWzU/1Lv5aWfqvc68usG7nKQ/gj/4LYJ8puzob35u3Ht20klqYc3DEbUmG6oYDMvNqH8MZtRixwYY/BIPtKX4lWFTkhZ3KJR/fIGPsUaXOiOo8SbU2K8Z9KfLhFo4b9Hd7y7qN8vm8V51Of1CTmqt53e6xNDpYwM7PRuo+9uBnQPHn7ATI3tW6+aY/ql5cSTLV6eRAUlsq8MP4MQ72UMTfWePQYBGSA6DYOWoghV7t7ohCCQ7wmOAsHvqKJNM8w5eijPxk7S505b3GqPdi/nZirzDp66X8FFrr7u15ENreSbZdaT/6zIEzsHUZJS/bykc7MNQw9VgkPLmDxm9pSCkaQqEkbAVOrZnW/IDGbaObJ66ArRsAFS80SqNfx0gPjxfeOClYB3M=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(186003)(31696002)(316002)(16576012)(53546011)(66946007)(26005)(66476007)(66556008)(2906002)(6666004)(31686004)(4326008)(8936002)(6916009)(5660300002)(38100700002)(86362001)(956004)(2616005)(8676002)(6486002)(508600001)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4668
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	75e81188-7b3b-411e-a791-08d97f12d4fd
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mkMY7qvB9p0nIHB6Y+F9as1OzrU6U1+j3D/LhPslcXNEX3gy3NHG7yuCOAoJu2qPJN27fXBA5qYPvvEfr4euoD5y+jGdPGGIKR1lfqdEGhJms14gwmO3XOJ86KaQTTQk4ZyEH5yQ6JZfNs5ByG1agbuJZok0IgAD9EwCS+ejitHlLhZw/RhGrF9MKzE6sMhBtbdUviUZLF65cOA1jaiHZIlxRMT133X/YHX508eqnQi8Qo7UEftXid7WrBkJMSF5PzfA3+ueyX7teeRhYe1UFqK8dKK1WdDUTKGUWZkqauPW8yM5zRkoCj86WvuNms4cQC/oADKZohi4BPTv9WjVEBio9kqrb1+mvU7N1wZyA7/8iVksLgpDOvzxtLYPkbyvTu20a8KTlgL9ZMJxFI7HbiapEfde6lvH7w+P82TvKcrZ7zdpqw8YW8VkRc3o3aGaJjqGC92khahEVs0fF20Z6zvCrba08O9eKw3SgIqtl+jpZZJJCebbNenjm4g+H/pt2xBqYfflQF6kx7eN5diIHrftR4zbeYqtrffS24sJ0+9HKvZOsv25tbw0boMgvDUV9CofMwqmol7iVTQAz46SOFvwd7St8DQri8bfatnFH3XTU6mMT8hPyZ4D+lYQtgsFJrb1NeXdZsDqb9UJBUcSacx31k+LNf1VjmD2jgDip2eMVqA4F18VsTawvRCEdxbx5z3jN+qOL+Ammt+yXq+2I116/a7VPWZRH1qJlOhOlAs=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6862004)(316002)(70586007)(508600001)(2616005)(8936002)(6486002)(31686004)(8676002)(6666004)(26005)(82310400003)(70206006)(83380400001)(4326008)(16576012)(31696002)(5660300002)(186003)(36756003)(81166007)(86362001)(47076005)(53546011)(356005)(36860700001)(956004)(336012)(2906002)(43740500002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:22:11.9363
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 61d8cc8b-e784-448e-faa5-08d97f12e18d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6390



On 2021/9/24 8:29, Stefano Stabellini wrote:
> +x86 maintainers
> 
> On Thu, 23 Sep 2021, Wei Chen wrote:
>> x86 provides a mem_hotplug to maintain the end of memory hotplug
>                              ^ variable
> 
>> end address. This variable can be accessed out of mm.c. We want
>> some code out of mm.c can be reused by other architectures without
>                         ^ so that it can be reused
> 
>> memory hotplug ability. So in this patch, we introduce these two
>> helpers to replace mem_hotplug direct access. This will give the
>> ability to stub these two API.
>                              ^ APIs
> 
> 

OK

>> Signed-off-by: Wei Chen <wei.chen@arm.com>
>> ---
>>   xen/include/asm-x86/mm.h | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
>> index cb90527499..af2fc4b0cd 100644
>> --- a/xen/include/asm-x86/mm.h
>> +++ b/xen/include/asm-x86/mm.h
>> @@ -475,6 +475,16 @@ static inline int get_page_and_type(struct page_info *page,
>>   
>>   extern paddr_t mem_hotplug;
>>   
>> +static inline void mem_hotplug_update_boundary(paddr_t end)
>> +{
>> +    mem_hotplug = end;
>> +}
>> +
>> +static inline paddr_t mem_hotplug_boundary(void)
>> +{
>> +    return mem_hotplug;
>> +}
>> +
>>   /******************************************************************************
>>    * With shadow pagetables, the different kinds of address start
>>    * to get get confusing.
>> -- 
>> 2.25.1
>>


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:26:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:26:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194857.347190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcmu-0002XA-Gb; Fri, 24 Sep 2021 04:26:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194857.347190; Fri, 24 Sep 2021 04:26:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcmu-0002X3-DY; Fri, 24 Sep 2021 04:26:16 +0000
Received: by outflank-mailman (input) for mailman id 194857;
 Fri, 24 Sep 2021 04:26:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcmt-0002Ww-Ax
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:26:15 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.58]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f43721eb-92e2-48e3-946b-4ef2e123e4e1;
 Fri, 24 Sep 2021 04:26:12 +0000 (UTC)
Received: from AS8PR04CA0145.eurprd04.prod.outlook.com (2603:10a6:20b:127::30)
 by AM8PR08MB6418.eurprd08.prod.outlook.com (2603:10a6:20b:360::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 24 Sep
 2021 04:26:11 +0000
Received: from AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:127:cafe::f9) by AS8PR04CA0145.outlook.office365.com
 (2603:10a6:20b:127::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:26:10 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT023.mail.protection.outlook.com (10.152.16.169) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:26:10 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Fri, 24 Sep 2021 04:26:08 +0000
Received: from f2fe4660b582.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 CE90A968-2BC4-4C43-9503-244342010EBE.1; 
 Fri, 24 Sep 2021 04:25:58 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f2fe4660b582.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:25:58 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PR3PR08MB5771.eurprd08.prod.outlook.com (2603:10a6:102:83::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 04:25:56 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:25:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f43721eb-92e2-48e3-946b-4ef2e123e4e1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Sw1iiTAo9BPTzZ9AjtznHmOlnpi7/koDvJb5TugMpXM=;
 b=QqchGc6Cqr3WlMbmHESYi35a3BydOm5LWiJe2qLxcq4h1oLGKsxED6Jej0jnqZ1OCSL3Q/qyXJniueSOimSlqDkQeai2rRbvc3V31g2/Zfbeaye2xmIKK45+A/hAF4AZRIGCEgJE5WRiidlg9YDWbXWOcMY5qbiEvHEd+VATlfg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LcnIC6rHwvL7yHMaTROXXqiRIKRX1p+iJjRq1xIAtV1qjS3+NhlbzMTxBS1NnQrOJVl4mksHGh/C3/uCwDse8ayKIikjMYHYFIfIhqOpFkhBnmZEp8m6XBRpfIY2EWw1tn0mjBNWceX0cyxImU5EEM5nLLCoY8JNgfx0jXP1owVe9xWLbDFJLB52/NTI2I788DK+aB972NbVDz9ciLa7wDo7vhwxv8y7ARhXVZNQQil1cklv4R6fRSMM0CrC+jWWhRIlmWjtvXUmZ2IpQUEDxP0QSjqKnSDMjYknF1ACM7xdcbwUoeKgTrUG8jqzElarJrgi5pvYvIQB5bkECdigug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Sw1iiTAo9BPTzZ9AjtznHmOlnpi7/koDvJb5TugMpXM=;
 b=VsXyYjZ34TQe0OCxtkuYl3lELZb6VtAA8knFe3g2f483N75neLptLddPkiRoiw/dq/DIN7/Z00I7TS2+x9hRe3rrixC08hkOpR6Uvzuuica2Nh2aIQwbTrbgYWJd8oIYYbQ0w0jdMhPdJEIVGpW0LdAyvM3crshQbvvnL0vZH0lzh7Jb4kEedL2SOQIWFbIcxwSox7ORJuuHqrqZG/s+3gBUr1hr39JWjoMQxpKvvArxJlZDkpjtxgySqwhsIzHK6knLHHvIi3diwAe/UFUSRBKiz09SjOycBgwHJSOW7ZIGb5mcAfktCDOWiLCYL+yJFXgT4YkNW4p8jaO/7SPIDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Sw1iiTAo9BPTzZ9AjtznHmOlnpi7/koDvJb5TugMpXM=;
 b=QqchGc6Cqr3WlMbmHESYi35a3BydOm5LWiJe2qLxcq4h1oLGKsxED6Jej0jnqZ1OCSL3Q/qyXJniueSOimSlqDkQeai2rRbvc3V31g2/Zfbeaye2xmIKK45+A/hAF4AZRIGCEgJE5WRiidlg9YDWbXWOcMY5qbiEvHEd+VATlfg=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 26/37] xen/arm: build NUMA cpu_to_node map in
 dt_smp_init_cpus
Thread-Topic: [PATCH 26/37] xen/arm: build NUMA cpu_to_node map in
 dt_smp_init_cpus
Thread-Index: AQHXsHMutbV6xKNOcke5Q+luAXEwQquydicAgAAg9TA=
Date: Fri, 24 Sep 2021 04:25:55 +0000
Message-ID:
 <PAXPR08MB6864920A4072FC029623BBC29EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-27-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231919070.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231919070.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: E7D9C557CCAAA44FBCAC19C270D50842.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 11ec1d55-6fde-42b3-245b-08d97f136fb7
x-ms-traffictypediagnostic: PR3PR08MB5771:|AM8PR08MB6418:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM8PR08MB64183DB80577274BE59579589EA49@AM8PR08MB6418.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:404;OLM:404;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 oG1uiJjXGdAHmOuORlRNYeIRJIW8YLyqtELIcZmVHrWWr/7Nn5McaOf3GANQ5XXs0Y1JHQsS3YQ0u61mhuPLCdybXBWEMiFAdqfc8LAFEV173diBrY5m8vl0P0PVbFnEertSwtfkZxUGflRtPwvfHsg/gSymz1Nhv4AD2zNlev6cqOOrCMcztwjre7Ep2XKCnvH/D9m7z4osoc83/Hn4vNJmnvjzRdEyYinHq0I/zqDFRtyH9/o5dczCPQcmYNTF/hze8ST4bXI/oYCL/gIhmhEV5abPNUiieNELGJvX8Vref+sudzE2JZ+po6wTX21ee3ozA+iYKxC+AtaB7X48ZzNoz+Tn6WPiWiqqcQ8w9oNRbWOaNQThS/f0+Xzt1Vc0m9ncRy44PEBqlyzebMZ86T02kad8VSlsy/Is6I6oiBqUJ1OS0xtZsCTQrmIsCNRN2Zuy54RO9wDgWQiN+2p0CTnZJqj9RQJthSlBDMajHfkqVEs4OpR3oxb2DSHwOwA5pi1cHkoXGRye196ZVdIw0a79DUuDoNqwp3GGzEMzpZgLr1ScQTmOtHQfp0aor61pmLtfE/bxzRXOHi1HBLxuybMesw2fPbXPRqSkfWacyi7X5emklMYrvxeN6PSxns9tmJBv1SLULqzkkqNIZWqwwNnmYgAiwomZp6/X9fQL5PCyqYh0LaxmZrXJPtiqzacvuAnX0c4lk+/HVlnCTUsyDg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(4326008)(122000001)(8936002)(55016002)(38070700005)(52536014)(9686003)(6506007)(53546011)(7696005)(71200400001)(186003)(66476007)(66946007)(6916009)(508600001)(66446008)(64756008)(38100700002)(83380400001)(76116006)(2906002)(5660300002)(33656002)(26005)(66556008)(316002)(54906003)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5771
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	84a9baf2-5995-472e-bdaf-08d97f136717
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SH6fc/JRWpPahBRKii/McN3ez23zfAVn+uSCQ7+OBDT5kHLEzfDRnJ3oC3XODho3iGsgQTUkZN7CBZVlEmo2Be3Z2xRRAxYrsxFYPKhFT9MSFehGB+fbVgiPoZQkQLm4XR46DrLPkX3tMWPD3EdSbCYyMadcFnzLy+G9eNzpkZpReoNcFeDHq8Au4/fa2VdOL9hMY56zD2UGcrIuraJ2TP00I0cUF0knySaS4r/KIne55lUqWWu9aXgjxlmurHyGvQkmlIs8WDe2QwNPWgdYCmqhvkdreWlBsrW0lxNb/rnxihBGNOgvvr9eu/mvIzBIeqIDjvdTlEo+zKKWOlGTcOtZI1f97NhYwxQUiHG3vegq7yov1Mdpa5qj4N8Yleu9yayV7B6P0MaJd6Uh8//laORMfh6j3YjFlLoUYzQAOySTItwoAOqcvB++KboU6NACvylOb+j9ZHwVrkvyfOpen2JYKo4vUkUy0lxFxlFkbVowTEHy0EjH+NMrqtk2JRUY2z/uomth0s5MrvUY03eBzGhj2Vw6S7Ro0mYYk66tkUMTGryZX10hXTK1TAp1d15AUYZtM+ZOtLoaiKhIYV/qEWj2vPb/EvbS1YKCDpcd8Ijp1JPu6X7C+UvnVJ1qlTBjwd5E4bllMe80F5hZK5A7vCPsKcOXV2O6pwhpGb0ZCHauD1KKYlK8+VIEsdFoTcvgQbW7SsDItlPnvecPioLZtQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8676002)(7696005)(4326008)(316002)(55016002)(5660300002)(6506007)(36860700001)(26005)(6862004)(86362001)(70206006)(47076005)(70586007)(186003)(54906003)(508600001)(9686003)(8936002)(52536014)(83380400001)(336012)(33656002)(82310400003)(2906002)(356005)(81166007)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:26:10.4285
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 11ec1d55-6fde-42b3-245b-08d97f136fb7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6418

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTA6MjYNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
Pg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIDI2LzM3XSB4ZW4vYXJtOiBidWlsZCBOVU1BIGNwdV90
b19ub2RlIG1hcCBpbg0KPiBkdF9zbXBfaW5pdF9jcHVzDQo+IA0KPiBPbiBUaHUsIDIzIFNlcCAy
MDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiBOVU1BIGltcGxlbWVudGF0aW9uIGhhcyBhIGNwdV90
b19ub2RlIGFycmF5IHRvIHN0b3JlIENQVSB0byBOT0RFDQo+ID4gbWFwLiBYZW4gaXMgdXNpbmcg
Q1BVIGxvZ2ljYWwgSUQgaW4gcnVudGltZSBjb21wb25lbnRzLCBzbyB3ZQ0KPiA+IHVzZSBDUFUg
bG9naWNhbCBJRCBhcyBDUFUgaW5kZXggaW4gY3B1X3RvX25vZGUuDQo+ID4NCj4gPiBJbiBkZXZp
Y2UgdHJlZSBjYXNlLCBjcHVfbG9naWNhbF9tYXAgaXMgY3JlYXRlZCBpbiBkdF9zbXBfaW5pdF9j
cHVzLg0KPiA+IFNvLCB3aGVuIE5VTUEgaXMgZW5hYmxlZCwgZHRfc21wX2luaXRfY3B1cyB3aWxs
IGZldGNoIENQVSBOVU1BIGlkDQo+ID4gYXQgdGhlIHNhbWUgdGltZSBmb3IgY3B1X3RvX25vZGUu
DQo+ID4NCj4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4g
PiAtLS0NCj4gPiAgeGVuL2FyY2gvYXJtL3NtcGJvb3QuYyAgICAgfCAzNyArKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKystDQo+ID4gIHhlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5o
IHwgIDUgKysrKysNCj4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA0MSBpbnNlcnRpb25zKCspLCAxIGRl
bGV0aW9uKC0pDQo+ID4NCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3NtcGJvb3QuYyBi
L3hlbi9hcmNoL2FybS9zbXBib290LmMNCj4gPiBpbmRleCA2MGMwZTgyZmM1Li42ZTNjYzhkM2Nj
IDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9hcmNoL2FybS9zbXBib290LmMNCj4gPiArKysgYi94ZW4v
YXJjaC9hcm0vc21wYm9vdC5jDQo+ID4gQEAgLTEyMSw3ICsxMjEsMTIgQEAgc3RhdGljIHZvaWQg
X19pbml0IGR0X3NtcF9pbml0X2NwdXModm9pZCkNCj4gPiAgICAgIHsNCj4gPiAgICAgICAgICBb
MCAuLi4gTlJfQ1BVUyAtIDFdID0gTVBJRFJfSU5WQUxJRA0KPiA+ICAgICAgfTsNCj4gPiArICAg
IHN0YXRpYyBub2RlaWRfdCBub2RlX21hcFtOUl9DUFVTXSBfX2luaXRkYXRhID0NCj4gPiArICAg
IHsNCj4gPiArICAgICAgICBbMCAuLi4gTlJfQ1BVUyAtIDFdID0gTlVNQV9OT19OT0RFDQo+ID4g
KyAgICB9Ow0KPiA+ICAgICAgYm9vbCBib290Y3B1X3ZhbGlkID0gZmFsc2U7DQo+ID4gKyAgICB1
aW50MzJfdCBuaWQgPSAwOw0KPiA+ICAgICAgaW50IHJjOw0KPiA+DQo+ID4gICAgICBtcGlkciA9
IHN5c3RlbV9jcHVpbmZvLm1waWRyLmJpdHMgJiBNUElEUl9IV0lEX01BU0s7DQo+ID4gQEAgLTE3
Miw2ICsxNzcsMjggQEAgc3RhdGljIHZvaWQgX19pbml0IGR0X3NtcF9pbml0X2NwdXModm9pZCkN
Cj4gPiAgICAgICAgICAgICAgY29udGludWU7DQo+ID4gICAgICAgICAgfQ0KPiA+DQo+ID4gKyAg
ICAgICAgaWYgKCBJU19FTkFCTEVEKENPTkZJR19OVU1BKSApDQo+ID4gKyAgICAgICAgew0KPiA+
ICsgICAgICAgICAgICAvKg0KPiA+ICsgICAgICAgICAgICAgKiBXaGVuIENPTkZJR19OVU1BIGlz
IHNldCwgdHJ5IHRvIGZldGNoIG51bWEgaW5mb21hdGlvbg0KPiA+ICsgICAgICAgICAgICAgKiBm
cm9tIENQVSBkdHMgbm9kZSwgb3RoZXJ3aXNlIHRoZSBuaWQgaXMgYWx3YXlzIDAuDQo+ID4gKyAg
ICAgICAgICAgICAqLw0KPiA+ICsgICAgICAgICAgICBpZiAoICFkdF9wcm9wZXJ0eV9yZWFkX3Uz
MihjcHUsICJudW1hLW5vZGUtaWQiLCAmbmlkKSApDQo+ID4gKyAgICAgICAgICAgIHsNCj4gPiAr
ICAgICAgICAgICAgICAgIHByaW50ayhYRU5MT0dfV0FSTklORw0KPiA+ICsgICAgICAgICAgICAg
ICAgICAgICAgICJjcHVbJWRdIGR0cyBwYXRoOiAlczogZG9lc24ndCBoYXZlIG51bWENCj4gaW5m
b3JtYXRpb24hXG4iLA0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXiAldQ0KPiAN
Cj4gDQo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgY3B1aWR4LCBkdF9ub2RlX2Z1bGxfbmFt
ZShjcHUpKTsNCj4gDQo+IEkgdGhpbmsgdGhhdCB0aGlzIG1lc3NhZ2Ugc2hvdWxkbid0IGJlIGEg
d2FybmluZzogQ09ORklHX05VTUEgaXMgYQ0KPiBjb21waWxlIHRpbWUgb3B0aW9uLiBBbnlib2R5
IHRoYXQgZW5hYmxlcyBDT05GSUdfTlVNQSBpbiB0aGUgWGVuIGJ1aWxkDQo+IHdpbGwgZ2V0IHRo
aXMgd2FybmluZyBwcmludGVkIG91dCBhdCBib290IHRpbWUgaWYgWGVuIGlzIGJvb3Rpbmcgb24g
YQ0KPiByZWd1bGFyIG5vbi1OVU1BIG1hY2hpbmUsIHJpZ2h0Pw0KPiANCj4gVGhlIHdhcm5pbmcg
c2hvdWxkIG9ubHkgYmUgcHJpbnRlZCBpZiBOVU1BIGlzIGFjdGl2ZWx5IGVuYWJsZWQsIGUuZy4N
Cj4gdGhlcmUgaXMgYSBkaXN0YW5jZS1tYXAgYnV0IHRoZSBjcHVzIGRvbid0IGhhdmUgbnVtYS1u
b2RlLWlkLg0KPiANCj4gDQoNClllcywgdGhpcyBtZXNzYWdlIHdvdWxkIGJlIHVuZXhwZWN0ZWQg
b24gYSByZWd1bGFyIG5vbi1OVU1BIG1hY2hpbmUuDQpJIHdpbGwgc29tZSBjaGVjayBjb25kaXRp
b24gdG8gcHJpbnQgdGhpcyBtZXNzYWdlLg0KDQo+IA0KPiA+ICsgICAgICAgICAgICAgICAgLyoN
Cj4gPiArICAgICAgICAgICAgICAgICAqIER1cmluZyB0aGUgZWFybHkgc3RhZ2Ugb2YgTlVNQSBp
bml0aWFsaXphdGlvbiwgd2hlbg0KPiBYZW4NCj4gPiArICAgICAgICAgICAgICAgICAqIGZvdW5k
IGFueSBDUFUgZHRzIG5vZGUgZG9lc24ndCBoYXZlIG51bWEtbm9kZS1pZA0KPiBpbmZvLCB0aGUN
Cj4gPiArICAgICAgICAgICAgICAgICAqIE5VTUEgd2lsbCBiZSB0cmVhdGVkIGFzIG9mZiwgYWxs
IENQVSB3aWxsIGJlIHNldCB0bw0KPiBhIEZBS0UNCj4gPiArICAgICAgICAgICAgICAgICAqIG5v
ZGUgMC4gU28gaWYgd2UgZ2V0IG51bWEtbm9kZS1pZCBmYWlsZWQgaGVyZSwgd2UNCj4gc2hvdWxk
DQo+ID4gKyAgICAgICAgICAgICAgICAgKiBzZXQgbmlkIHRvIDAuDQo+ID4gKyAgICAgICAgICAg
ICAgICAgKi8NCj4gPiArICAgICAgICAgICAgICAgIG5pZCA9IDA7DQo+ID4gKyAgICAgICAgICAg
IH0NCj4gPiArICAgICAgICB9DQo+ID4gKw0KPiA+ICAgICAgICAgIC8qDQo+ID4gICAgICAgICAg
ICogOCBNU0JzIG11c3QgYmUgc2V0IHRvIDAgaW4gdGhlIERUIHNpbmNlIHRoZSByZWcgcHJvcGVy
dHkNCj4gPiAgICAgICAgICAgKiBkZWZpbmVzIHRoZSBNUElEUlsyMzowXQ0KPiA+IEBAIC0yMzEs
OSArMjU4LDEyIEBAIHN0YXRpYyB2b2lkIF9faW5pdCBkdF9zbXBfaW5pdF9jcHVzKHZvaWQpDQo+
ID4gICAgICAgICAgew0KPiA+ICAgICAgICAgICAgICBwcmludGsoImNwdSVkIGluaXQgZmFpbGVk
IChod2lkICUiUFJJcmVnaXN0ZXIiKTogJWRcbiIsIGksDQo+IGh3aWQsIHJjKTsNCj4gPiAgICAg
ICAgICAgICAgdG1wX21hcFtpXSA9IE1QSURSX0lOVkFMSUQ7DQo+ID4gKyAgICAgICAgICAgIG5v
ZGVfbWFwW2ldID0gTlVNQV9OT19OT0RFOw0KPiA+ICAgICAgICAgIH0NCj4gPiAtICAgICAgICBl
bHNlDQo+ID4gKyAgICAgICAgZWxzZSB7DQo+ID4gICAgICAgICAgICAgIHRtcF9tYXBbaV0gPSBo
d2lkOw0KPiA+ICsgICAgICAgICAgICBub2RlX21hcFtpXSA9IG5pZDsNCj4gPiArICAgICAgICB9
DQo+ID4gICAgICB9DQo+ID4NCj4gPiAgICAgIGlmICggIWJvb3RjcHVfdmFsaWQgKQ0KPiA+IEBA
IC0yNDksNiArMjc5LDExIEBAIHN0YXRpYyB2b2lkIF9faW5pdCBkdF9zbXBfaW5pdF9jcHVzKHZv
aWQpDQo+ID4gICAgICAgICAgICAgIGNvbnRpbnVlOw0KPiA+ICAgICAgICAgIGNwdW1hc2tfc2V0
X2NwdShpLCAmY3B1X3Bvc3NpYmxlX21hcCk7DQo+ID4gICAgICAgICAgY3B1X2xvZ2ljYWxfbWFw
KGkpID0gdG1wX21hcFtpXTsNCj4gPiArDQo+ID4gKyAgICAgICAgbmlkID0gbm9kZV9tYXBbaV07
DQo+ID4gKyAgICAgICAgaWYgKCBuaWQgPj0gTUFYX05VTU5PREVTICkNCj4gPiArICAgICAgICAg
ICAgbmlkID0gMDsNCj4gPiArICAgICAgICBudW1hX3NldF9ub2RlKGksIG5pZCk7DQo+ID4gICAg
ICB9DQo+ID4gIH0NCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9hc20tYXJtL251
bWEuaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gaW5kZXggNzU4ZWFmZWIwNS4u
OGE0YWQzNzllMCAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0K
PiA+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gQEAgLTQ2LDYgKzQ2LDEx
IEBAIGV4dGVybiBtZm5fdCBmaXJzdF92YWxpZF9tZm47DQo+ID4gICNkZWZpbmUgbm9kZV9zdGFy
dF9wZm4obmlkKSAobWZuX3goZmlyc3RfdmFsaWRfbWZuKSkNCj4gPiAgI2RlZmluZSBfX25vZGVf
ZGlzdGFuY2UoYSwgYikgKDIwKQ0KPiA+DQo+ID4gK3N0YXRpYyBpbmxpbmUgdm9pZCBudW1hX3Nl
dF9ub2RlKGludCBjcHUsIG5vZGVpZF90IG5vZGUpDQo+ID4gK3sNCj4gPiArDQo+ID4gK30NCj4g
PiArDQo+ID4gICNlbmRpZg0KPiA+DQo+ID4gIHN0YXRpYyBpbmxpbmUgdW5zaWduZWQgaW50IGFy
Y2hfaGF2ZV9kZWZhdWx0X2RtYXpvbmUodm9pZCkNCj4gPiAtLQ0KPiA+IDIuMjUuMQ0KPiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:27:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194862.347201 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcnw-00036b-Qm; Fri, 24 Sep 2021 04:27:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194862.347201; Fri, 24 Sep 2021 04:27:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcnw-00036U-Nu; Fri, 24 Sep 2021 04:27:20 +0000
Received: by outflank-mailman (input) for mailman id 194862;
 Fri, 24 Sep 2021 04:27:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcnv-00036L-EV
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:27:19 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.59]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d60924b7-ed67-4866-a9dd-e0c9660937e9;
 Fri, 24 Sep 2021 04:27:17 +0000 (UTC)
Received: from AM6P194CA0021.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:90::34)
 by AM5PR0802MB2466.eurprd08.prod.outlook.com (2603:10a6:203:97::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 04:27:08 +0000
Received: from AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:90:cafe::14) by AM6P194CA0021.outlook.office365.com
 (2603:10a6:209:90::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:27:08 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT054.mail.protection.outlook.com (10.152.16.212) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:27:07 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Fri, 24 Sep 2021 04:27:06 +0000
Received: from d76a2de587e8.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 AAF7B539-C802-4615-924A-ECF049DE4CBB.1; 
 Fri, 24 Sep 2021 04:26:56 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d76a2de587e8.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:26:56 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PR3PR08MB5771.eurprd08.prod.outlook.com (2603:10a6:102:83::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 04:26:54 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:26:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d60924b7-ed67-4866-a9dd-e0c9660937e9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Dcpt/BRXQhfJCq59NIwGW+HzCC6vr4mLckkyoqQvrf8=;
 b=uCoUS4NHe7FMmMqD6YTSYe2BF9AZHLxbbu6fPKI+bqUM9GVZ8ZXkh07MRB3T9X/qWcreTSj13upGRvu2LbLsUcl+XO94Jb+AgFsuHzBRl8XLyUCtZb0SuXibjOnDUTIB1jYHe6iz8pdP01PD3bhPldZuY+sJOb6uBbnUQ9bYqRk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WJfx8Oat6p19Im90572ZLuD5sOt9DTbsLSntMEekZl0Du9BYpXefcmlFXR9fx2qCyqZAxvx8dWr8haX50CuExGhxNMnoJCfoSGVIp4R6SdanVy+Jl2lrDy4/r6/okOcW1QSPuEPa2lBrIQkYhUpmgzz8MeK4k2V7xC/YIrDHg1aj4qaoqCdiufI8QtRS2yCaQYILat8ucLXxl4kyWjeijk6xphwonr8mJ1UTSSn1qd5xqf3Kq84ijTDrh4Hl9drzn/uLii7SiJ/FPuTOscW8PCUhYh7k3h4q4p14/7lDuAQzcYQAZYPu5mD+Hi4e/nTQGZlnwHMbw6QfHZmO5/T2zA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Dcpt/BRXQhfJCq59NIwGW+HzCC6vr4mLckkyoqQvrf8=;
 b=FcCmDihdg1u7mrkD8Pv0PEaSOJXRpE0zJqhFchB+pD85mQYto1HUNPqz2VUuXvZxH0S8DYuWZR6AghehCe1ePy8fgKFrUUEH/ig339354HdwmoYtLS/ZNTtNtA8qw6rsfI1tZ33D09xCXq+UqJuUS9iHonXAhoB4GWHgxR9i19u+a+Fge0vz7Ic5yyZ9Yadb9kD17Q/mpfynQ1L5l+TJQR7NK7GpdT7nKzHC9eaRbWyfJtp9EIP8SJcwm9asrF29unM8J0Qz84rp/VFrf4AyiQ3DbRRTElB5tHj/08s7o8+vxexZCkDJUVktTD+yMRt2jzPh4YC4qzahGMEl00OGfg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Dcpt/BRXQhfJCq59NIwGW+HzCC6vr4mLckkyoqQvrf8=;
 b=uCoUS4NHe7FMmMqD6YTSYe2BF9AZHLxbbu6fPKI+bqUM9GVZ8ZXkh07MRB3T9X/qWcreTSj13upGRvu2LbLsUcl+XO94Jb+AgFsuHzBRl8XLyUCtZb0SuXibjOnDUTIB1jYHe6iz8pdP01PD3bhPldZuY+sJOb6uBbnUQ9bYqRk=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 28/37] xen/arm: stub memory hotplug access helpers for Arm
Thread-Topic: [PATCH 28/37] xen/arm: stub memory hotplug access helpers for
 Arm
Thread-Index: AQHXsHM7ZH4VKSj50US+Svlpv1AyWquyeDqAgAAfkdA=
Date: Fri, 24 Sep 2021 04:26:54 +0000
Message-ID:
 <PAXPR08MB68643E4949FCAB1CC84CFC2F9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-29-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231932250.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231932250.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6B5F14B452839145BBC41D28DD33E36B.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a9d628c1-9fc9-4cae-3aa6-08d97f1391ec
x-ms-traffictypediagnostic: PR3PR08MB5771:|AM5PR0802MB2466:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM5PR0802MB2466E4739EC21617604E16FB9EA49@AM5PR0802MB2466.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:5797;OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YFZXq/NEyGrCAithY5Q3CwFzBsbkP+Pz5de6kfp+Pz9ERxCz0XcKG1fMqJ8a93tRiJeEcTo1mIzV2v0imnGVxAC+ahT3n8GACSToUT+o1uu0noD1BctgVZiLHdFz0i7i6SWDp8oOOmYXsaSDdyV8NbJKnbYXc4l9qfRy2YqAMtWzgOJlvTMBjR6GDSnieramYGcvx9xRNgFkzH2g4swmCfTdywD8XbtKTcuhRTomKZwlAX3/nhzII9hh4vUG/DeUYo7YPRTZHieYYzuIkA64QvgIoZMZd1pDrRQ5nc1GzDJiBQ9jwHYco7nmssycvCnsNCJjwbe6hPudoi2NrXR1yKkG2YhEkt83um5s5iSvqk+C1nmQavSo27hrjJKEC2q3tnhatzl+cZxxSh9Ft51E0u2vSrv5/UCKhwjj2QUELZpgQmKKZ8NleepZTM24BrpxvlahtFlWKygEGxKisenJ8W948Yf/TGr2qX0UuZyPKr8cWgaH9C5QbwWKgpaZBN2IVpFL8K0LuQABiax5lewZRd+fzMENSZhmmiDHk7xV/3c4eGsmnJ2GW2Uc17KWMbSKXQcaGmTgfpBzxqxLoqldTo+aHSF0NbVZS1tiDu9OV/vbl2avQI9D85D1WGfAm5ggxPbZ780T8/Xuhpybmcm7STqpvMiw09xJJJEz2CxPq4mtGF6sWGNVbrY/S7Lo6sIgRzx1nNEvXavrrKdospj0Ag==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(4326008)(122000001)(8936002)(55016002)(38070700005)(52536014)(9686003)(6506007)(53546011)(7696005)(71200400001)(186003)(66476007)(66946007)(6916009)(508600001)(66446008)(64756008)(38100700002)(83380400001)(76116006)(2906002)(5660300002)(33656002)(26005)(66556008)(316002)(54906003)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5771
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	9c78b180-37c9-4776-9241-08d97f138a03
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YRr/f37WvpJ/xYHTbjpOFxnvnAZ1HrNaNc0jJTVClBx5kNEadOtZeL03wR3JtD9xQGCfgp4REObP5pNnRLtJuxHUNs38rYfzdP8yZSwUidKy11sEb26gu996Fjq6HF9e6ZYRee9XyVgHoMaOlQC05j3AeygpbhDWuhraJodegNeODOpp1zuMuvsCJEEnZB6/6jqme11ht2xyb0Ae5jMHKsYjaZnGd3tam8ll/D3W/+UZShcNMQg0YkrCNliNed0eKT5S8YN2qUF+2Fv8TYYuX1TeXWBrX4usXC44To8a/xOd8vNIIA/hRM3JGfdR/enTVen/jXBlV/SQZx8vTM+3zzwc++36e6uEJ48p/NsLuYOjMx6jyGtOEE4R8st6nX6FqfBiat8+5MG3F9Ex2El/OzjXZYpNOCCk8Cp5l/KV6O1P1zgu/UyXH3auIrd5an9fzVmrR6Zt1uQf8OKB6uuNdvbOC7YoTfX6yXxpxPLEraD/Yr1YR4teBtezi/194bRGgjkW0zjvqatVGm6me8GMaA/b0OkdoC8q+zbZVrHYQzYRBkKl5psSE62dZjZnlKqeUuEk6/q0nzKCFbq/ziL2G/6nVC9Cat9bmzn/AlZO9KCTOxvGPgHgli8s9kL+VQwRGJMfZ8DIamIvTWe4OMvyuyu5jr14qoNT98hBRhjxccRueRrWQpYkQBqMyvx//0ocsRQEkkkBY43C4avFFUSrCw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(9686003)(33656002)(36860700001)(8676002)(70586007)(186003)(70206006)(5660300002)(336012)(8936002)(86362001)(55016002)(26005)(83380400001)(81166007)(4326008)(47076005)(7696005)(6862004)(2906002)(82310400003)(54906003)(316002)(6506007)(53546011)(356005)(508600001)(52536014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:27:07.8270
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a9d628c1-9fc9-4cae-3aa6-08d97f1391ec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT054.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2466

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDEwOjM0
DQo+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0
cy54ZW5wcm9qZWN0Lm9yZzsgc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVuQHhlbi5vcmc7
DQo+IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gU3ViamVj
dDogUmU6IFtQQVRDSCAyOC8zN10geGVuL2FybTogc3R1YiBtZW1vcnkgaG90cGx1ZyBhY2Nlc3Mg
aGVscGVycyBmb3INCj4gQXJtDQo+IA0KPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkgQ2hlbiB3
cm90ZToNCj4gPiBDb21tb24gY29kZSBpbiBOVU1BIG5lZWQgdGhlc2UgdHdvIGhlbHBlcnMgdG8g
YWNjZXNzL3VwZGF0ZQ0KPiA+IG1lbW9yeSBob3RwbHVnIGVuZCBhZGRyZXNzLiBBcm0gaGFzIG5v
dCBzdXBwb3J0IG1lbW9yeSBob3RwbHVnDQo+ID4geWV0LiBTbyB3ZSBzdHViIHRoZXNlIHR3byBo
ZWxwZXJzIGluIHRoaXMgcGF0Y2ggdG8gbWFrZSBOVU1BDQo+ID4gY29tbW9uIGNvZGUgaGFwcHku
DQo+ID4NCj4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4g
PiAtLS0NCj4gPiAgeGVuL2luY2x1ZGUvYXNtLWFybS9tbS5oIHwgMTAgKysrKysrKysrKw0KPiA+
ICAxIGZpbGUgY2hhbmdlZCwgMTAgaW5zZXJ0aW9ucygrKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBh
L3hlbi9pbmNsdWRlL2FzbS1hcm0vbW0uaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vbW0uaA0KPiA+
IGluZGV4IDdiNWU3YjdmNjkuLmZjOTQzMzE2NWQgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2luY2x1
ZGUvYXNtLWFybS9tbS5oDQo+ID4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9tbS5oDQo+ID4g
QEAgLTM2Miw2ICszNjIsMTYgQEAgdm9pZCBjbGVhcl9hbmRfY2xlYW5fcGFnZShzdHJ1Y3QgcGFn
ZV9pbmZvICpwYWdlKTsNCj4gPg0KPiA+ICB1bnNpZ25lZCBpbnQgYXJjaF9nZXRfZG1hX2JpdHNp
emUodm9pZCk7DQo+ID4NCj4gPiArc3RhdGljIGlubGluZSB2b2lkIG1lbV9ob3RwbHVnX3VwZGF0
ZV9ib3VuZGFyeShwYWRkcl90IGVuZCkNCj4gPiArew0KPiA+ICsNCj4gPiArfQ0KPiA+ICsNCj4g
PiArc3RhdGljIGlubGluZSBwYWRkcl90IG1lbV9ob3RwbHVnX2JvdW5kYXJ5KHZvaWQpDQo+ID4g
K3sNCj4gPiArICAgIHJldHVybiAwOw0KPiA+ICt9DQo+IA0KPiBXaHkgemVybz8gQ291bGQgaXQg
YmUgSU5WQUxJRF9QQUREUiA/DQoNClllcywgSU5WQUxJRF9QQUREUiBpcyBiZXR0ZXIuDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:29:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:29:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194870.347212 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcpo-0004Me-Dh; Fri, 24 Sep 2021 04:29:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194870.347212; Fri, 24 Sep 2021 04:29:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcpo-0004MX-AY; Fri, 24 Sep 2021 04:29:16 +0000
Received: by outflank-mailman (input) for mailman id 194870;
 Fri, 24 Sep 2021 04:29:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcpn-0004MR-34
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:29:15 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5446baf4-4f36-4bab-a0a0-ae538cb50bde;
 Fri, 24 Sep 2021 04:29:12 +0000 (UTC)
Received: from PR0P264CA0196.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::16)
 by VE1PR08MB4766.eurprd08.prod.outlook.com (2603:10a6:802:a9::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 24 Sep
 2021 04:29:10 +0000
Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:100:1f:cafe::24) by PR0P264CA0196.outlook.office365.com
 (2603:10a6:100:1f::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:29:09 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:29:09 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Fri, 24 Sep 2021 04:29:08 +0000
Received: from 5aecf7542546.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0D925695-8B5E-4D6B-AEC4-205470F0A028.1; 
 Fri, 24 Sep 2021 04:28:58 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5aecf7542546.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:28:58 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PR3PR08MB5771.eurprd08.prod.outlook.com (2603:10a6:102:83::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 04:28:56 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:28:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5446baf4-4f36-4bab-a0a0-ae538cb50bde
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Zs6oHfWBEU2Z7IZde4JkiH/HT8DfYr+zVd0B+TsJwyo=;
 b=n2mEaDSZB05RYq/cGVdS8f1NZZhOe5eTduP73g3dUR2K8oKzmtX+1r2ZNAJPhmXWCrHxu0NPDS+PTesYqjlUPscT+i/S/FY0d8/6h4w6jS5c2jYHKqC5rzp/RRyIj25HvoOUlV1xeMI7FSDtvkrzKGxvNjdYz9Hwvg6MzEW7qbI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hEgI5zS7sM6fjcyyLWmvkyOSJbFA6UCc3F+wiB/MJNukOkpqkELNxiKBvkEpmxo8DpQK62GBkb1wzxKPqB9irHP4349HXxaeTi2fcaSCy15kdqLM8cG9E5i3n/JOMjxySw7H/xWPF1Wfmm2PW7kdzf1yA5cM3Ua/WY9MYuknJnpg8c7d6aZmLzX+O8OZdqk+ho2KnUt6ngbU/jIkXg7DWmo/4MeQUuL+IgtEdB1JZvb7fUqsHRHzQ+Cdi9sWziFxCoJosd5xul2OWkN0O40+934lsmLD4s5uH8HH9I2SlsTu/8xEhTxJ4mbtiIuIb9AvT1Gm6hsSfEJ8kEZ5s1MtIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Zs6oHfWBEU2Z7IZde4JkiH/HT8DfYr+zVd0B+TsJwyo=;
 b=kOzmx8fWdXYiHh9XcsD6tRmWevlgUbVzNszd/LSEGtMhq58lvgQGXq58Io+44dpJJKjK4AForI8BMAUx78rntbGfVzMyuEtGvnJKdOaPLsSQj2kcCAVL2oIyOVxTrNcjpZLm5aVgucHzQlGLu6Ez3UNqEtA7Cxo7IMDfzb3ycl2Oi8TwpJAzDu2v+YwpDEtXY3clPzLesaf4sg3tXQDkpeOSKDnWztxgSRr6i0uthg1pQU7J0ghyxRoPd+CPtstJbQYvyE2GOquja3OLCSYu9WCybPsNHay58s0RXNFDctDflv97DXV09DPdsCaP2SbU9ZMBk6EfoWqWa8GeSFUxtA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Zs6oHfWBEU2Z7IZde4JkiH/HT8DfYr+zVd0B+TsJwyo=;
 b=n2mEaDSZB05RYq/cGVdS8f1NZZhOe5eTduP73g3dUR2K8oKzmtX+1r2ZNAJPhmXWCrHxu0NPDS+PTesYqjlUPscT+i/S/FY0d8/6h4w6jS5c2jYHKqC5rzp/RRyIj25HvoOUlV1xeMI7FSDtvkrzKGxvNjdYz9Hwvg6MzEW7qbI=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Topic: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Index: AQHXsHMRZjZ8eZQrbkW8SEq2x0f00KuyVH0AgABDnEA=
Date: Fri, 24 Sep 2021 04:28:56 +0000
Message-ID:
 <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-9-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6F0A5B5F03ED554A8D509BC09F619A37.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b6d918df-5af5-4493-c17a-08d97f13da94
x-ms-traffictypediagnostic: PR3PR08MB5771:|VE1PR08MB4766:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB476607CCA25FCF4A508A69E39EA49@VE1PR08MB4766.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AFBGnggK6u1qWPSs7pzy72+o07Fg/ilxqV8xmK+c/k3TT+iyY+tY+QtQZLuC7lJ+JGsh6Vh9OLzZZ1QixXY8M8rz5hHWMCVc/gYnpXQ5ZiJ9yCduOJgHTvIkls93CuVyW7kk8UDXsQFM77tjPD8Lmnu7L3es6VTa+GMqPEmeb5SkDZhySE4WSkVOaDgdz0CaR3l6kOFCDOemTkZk7/YtOhHqYxbQohg+jHBJxYGOw6M6rxMigJALSiT8b36hXk5Gbzag6Eib/32VwtKCurrbZZg4+K4zqDpIFQARkT0fn1cbIkiBP7tAyh9jkR1fFj2n9zXHo1CgWsk6hctCWRnUUZzCo879vI8dq8Oeg2gPWkHy92ENfKcCziAR4Qdnki3vhR1CdtcSUsv8iNX5fSFHXqy9WahoZhd54cL4uX9cE3Iku5aW/8XHessB214UmhntH/rIRyYjwswPRB1rWeZbEPnjhXfO5B8Tf/LWZFQyjgr6IQkuMOhzrcTpygczM3NfaDvu0rXtwEI9cTsQqzJy0I5GEM5IuwSmIK2bItTD4DVMiw54OnWQgSmNtO3ByX6BasllNu+rs2oYp49FEgZ37R02cxKL77lHZ781l1PBczK1gScQyJT+Rtr1RNWcbgfad89wASx4WSzwoI+xtpANe9Bf3Q+oOR/4GQBmdrdIQl5irMzAH0jqrtS8wnlIjSoizcLv773TlnmEhqwkS+6VTw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(4326008)(122000001)(8936002)(55016002)(38070700005)(52536014)(9686003)(6506007)(53546011)(7696005)(71200400001)(186003)(66476007)(66946007)(6916009)(508600001)(66446008)(64756008)(38100700002)(83380400001)(76116006)(2906002)(5660300002)(33656002)(26005)(66556008)(316002)(54906003)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5771
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ef41bee6-d68f-4028-a801-08d97f13d2aa
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DFuCruuKdSlGLoBvHYKZcR0U2pVoFxdnbMJSqqMzEK6vHtukiy7mGDKuu2OuGbEq2saihNhit9iFscbnrxzZq6gMfotlHcAwrV2OJzjR0ctmpFJ5jfqUVLoswLxExUZwqJ5k4Ttb9E8yCv56lJ+JIst3VOZPfFKu8N+0/hG76wclfNCvS6wRNYmOYRrPiLEb3vg3cEBPiigEq600glaeVqHVxBH8HZ0zvm7lYYMq4hVpSnG7qNFSlaScVieXhoDJmaqMPaplp4XVYexVeJhhqz/3o5O/R0B+ozg31AAlVIHB9AcyfNm0yJsE8Jj3RWcNd/IypQm7yUCB6+XpMUABaqpOYrThSqUurdhneKPHY8XP1545ei/0URD3nP+KEmXMmzgR+ZDeshoiRhpwunxhoiNYSI6GN1c4ebuFTCxxHrdxBbHrh2ptdk4cLgAv9ev68thw/HZEKsMvjBAtuf8lgGszpQXbICfWB/rSepl1aBxc8KHSFc+3WY55eSHr02gYFOaHijMmZU5zj0ANsjuC8CCwZxE3FYs9c+psCXkorvWY7PRMnROg4/iZq1he0ZMo8W4B1cdut2p76kY6tsrh/eWbqxs+MnuLt1AM4vpllTREllRtz59SudetEdWH2sHqp60FaLHVPtz3ut85aJ/1n15slyVtgERyXWkI9/FDtaFR5Twb+jozY5haEX3LTDZzRMfH/pGw8ps4QeeLreniIw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(336012)(86362001)(186003)(70586007)(8936002)(83380400001)(54906003)(8676002)(70206006)(7696005)(36860700001)(55016002)(52536014)(4326008)(33656002)(316002)(6506007)(356005)(47076005)(53546011)(5660300002)(81166007)(9686003)(2906002)(6862004)(82310400003)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:29:09.6855
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b6d918df-5af5-4493-c17a-08d97f13da94
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4766

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDg6MjYN
Cj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVuLWRldmVsQGxpc3Rz
LnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxpZW5AeGVuLm9yZzsN
Cj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hA
c3VzZS5jb207DQo+IGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb207IHJvZ2VyLnBhdUBjaXRyaXgu
Y29tOyB3bEB4ZW4ub3JnDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMDgvMzddIHhlbi94ODY6IGFk
ZCBkZXRlY3Rpb24gb2YgZGlzY29udGlub3VzIG5vZGUNCj4gbWVtb3J5IHJhbmdlDQo+IA0KPiBD
QydpbmcgeDg2IG1haW50YWluZXJzDQo+IA0KPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkgQ2hl
biB3cm90ZToNCj4gPiBPbmUgTlVNQSBub2RlIG1heSBjb250YWluIHNldmVyYWwgbWVtb3J5IGJs
b2Nrcy4gSW4gY3VycmVudCBYZW4NCj4gPiBjb2RlLCBYZW4gd2lsbCBtYWludGFpbiBhIG5vZGUg
bWVtb3J5IHJhbmdlIGZvciBlYWNoIG5vZGUgdG8gY292ZXINCj4gPiBhbGwgaXRzIG1lbW9yeSBi
bG9ja3MuIEJ1dCBoZXJlIGNvbWVzIHRoZSBwcm9ibGVtLCBpbiB0aGUgZ2FwIG9mDQo+ID4gb25l
IG5vZGUncyB0d28gbWVtb3J5IGJsb2NrcywgaWYgdGhlcmUgYXJlIHNvbWUgbWVtb3J5IGJsb2Nr
cyBkb24ndA0KPiA+IGJlbG9uZyB0byB0aGlzIG5vZGUgKHJlbW90ZSBtZW1vcnkgYmxvY2tzKS4g
VGhpcyBub2RlJ3MgbWVtb3J5IHJhbmdlDQo+ID4gd2lsbCBiZSBleHBhbmRlZCB0byBjb3ZlciB0
aGVzZSByZW1vdGUgbWVtb3J5IGJsb2Nrcy4NCj4gPg0KPiA+IE9uZSBub2RlJ3MgbWVtb3J5IHJh
bmdlIGNvbnRhaW5zIG90aGUgbm9kZXMnIG1lbW9yeSwgdGhpcyBpcyBvYnZpb3VzbHkNCj4gPiBu
b3QgdmVyeSByZWFzb25hYmxlLiBUaGlzIG1lYW5zIGN1cnJlbnQgTlVNQSBjb2RlIG9ubHkgY2Fu
IHN1cHBvcnQNCj4gPiBub2RlIGhhcyBjb250aW5vdXMgbWVtb3J5IGJsb2Nrcy4gSG93ZXZlciwg
b24gYSBwaHlzaWNhbCBtYWNoaW5lLCB0aGUNCj4gPiBhZGRyZXNzZXMgb2YgbXVsdGlwbGUgbm9k
ZXMgY2FuIGJlIGludGVybGVhdmVkLg0KPiA+DQo+ID4gU28gaW4gdGhpcyBwYXRjaCwgd2UgYWRk
IGNvZGUgdG8gZGV0ZWN0IGRpc2NvbnRpbm91cyBtZW1vcnkgYmxvY2tzDQo+ID4gZm9yIG9uZSBu
b2RlLiBOVU1BIGluaXRpYWxpenRpb24gd2lsbCBiZSBmYWlsZWQgYW5kIGVycm9yIG1lc3NhZ2Vz
DQo+ID4gd2lsbCBiZSBwcmludGVkIHdoZW4gWGVuIGRldGVjdCBzdWNoIGhhcmR3YXJlIGNvbmZp
Z3VyYXRpb24uDQo+IA0KPiBBdCBsZWFzdCBvbiBBUk0sIGl0IGlzIG5vdCBqdXN0IG1lbW9yeSB0
aGF0IGNhbiBiZSBpbnRlcmxlYXZlZCwgYnV0IGFsc28NCj4gTU1JTyByZWdpb25zLiBGb3IgaW5z
dGFuY2U6DQo+IA0KPiBub2RlMCBiYW5rMCAwLTB4MTAwMDAwMA0KPiBNTUlPIDB4MTAwMDAwMC0w
eDEwMDIwMDANCj4gSG9sZSAweDEwMDIwMDAtMHgyMDAwMDAwDQo+IG5vZGUwIGJhbmsxIDB4MjAw
MDAwMC0weDMwMDAwMDANCj4gDQo+IFNvIEkgYW0gbm90IGZhbWlsaWFyIHdpdGggdGhlIFNSQVQg
Zm9ybWF0LCBidXQgSSB0aGluayBvbiBBUk0gdGhlIGNoZWNrDQo+IHdvdWxkIGxvb2sgZGlmZmVy
ZW50OiB3ZSB3b3VsZCBqdXN0IGxvb2sgZm9yIG11bHRpcGxlIG1lbW9yeSByYW5nZXMNCj4gdW5k
ZXIgYSBkZXZpY2VfdHlwZSA9ICJtZW1vcnkiIG5vZGUgb2YgYSBOVU1BIG5vZGUgaW4gZGV2aWNl
IHRyZWUuDQo+IA0KPiANCg0KU2hvdWxkIEkgbmVlZCB0byBpbmNsdWRlL3JlZmluZSBhYm92ZSBt
ZXNzYWdlIHRvIGNvbW1pdCBsb2c/DQoNCj4gDQo+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4g
PHdlaS5jaGVuQGFybS5jb20+DQo+ID4gLS0tDQo+ID4gIHhlbi9hcmNoL3g4Ni9zcmF0LmMgfCAz
NiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4gPiAgMSBmaWxlIGNoYW5n
ZWQsIDM2IGluc2VydGlvbnMoKykNCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC94ODYv
c3JhdC5jIGIveGVuL2FyY2gveDg2L3NyYXQuYw0KPiA+IGluZGV4IDdkMjBkN2YyMjIuLjJmMDhm
YTQ2NjAgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gveDg2L3NyYXQuYw0KPiA+ICsrKyBiL3hl
bi9hcmNoL3g4Ni9zcmF0LmMNCj4gPiBAQCAtMjcxLDYgKzI3MSwzNiBAQCBhY3BpX251bWFfcHJv
Y2Vzc29yX2FmZmluaXR5X2luaXQoY29uc3Qgc3RydWN0DQo+IGFjcGlfc3JhdF9jcHVfYWZmaW5p
dHkgKnBhKQ0KPiA+ICAJCSAgICAgICBweG0sIHBhLT5hcGljX2lkLCBub2RlKTsNCj4gPiAgfQ0K
PiA+DQo+ID4gKy8qDQo+ID4gKyAqIENoZWNrIHRvIHNlZSBpZiB0aGVyZSBhcmUgb3RoZXIgbm9k
ZXMgd2l0aGluIHRoaXMgbm9kZSdzIHJhbmdlLg0KPiA+ICsgKiBXZSBqdXN0IG5lZWQgdG8gY2hl
Y2sgZnVsbCBjb250YWlucyBzaXR1YXRpb24uIEJlY2F1c2Ugb3ZlcmxhcHMNCj4gPiArICogaGF2
ZSBiZWVuIGNoZWNrZWQgYmVmb3JlIGJ5IGNvbmZsaWN0aW5nX21lbWJsa3MuDQo+ID4gKyAqLw0K
PiA+ICtzdGF0aWMgYm9vbCBfX2luaXQgaXNfbm9kZV9tZW1vcnlfY29udGludW91cyhub2RlaWRf
dCBuaWQsDQo+ID4gKyAgICBwYWRkcl90IHN0YXJ0LCBwYWRkcl90IGVuZCkNCj4gPiArew0KPiA+
ICsJbm9kZWlkX3QgaTsNCj4gPiArDQo+ID4gKwlzdHJ1Y3Qgbm9kZSAqbmQgPSAmbm9kZXNbbmlk
XTsNCj4gPiArCWZvcl9lYWNoX25vZGVfbWFzayhpLCBtZW1vcnlfbm9kZXNfcGFyc2VkKQ0KPiA+
ICsJew0KPiA+ICsJCS8qIFNraXAgaXRzZWxmICovDQo+ID4gKwkJaWYgKGkgPT0gbmlkKQ0KPiA+
ICsJCQljb250aW51ZTsNCj4gPiArDQo+ID4gKwkJbmQgPSAmbm9kZXNbaV07DQo+ID4gKwkJaWYg
KHN0YXJ0IDwgbmQtPnN0YXJ0ICYmIG5kLT5lbmQgPCBlbmQpDQo+ID4gKwkJew0KPiA+ICsJCQlw
cmludGsoS0VSTl9FUlINCj4gPiArCQkJICAgICAgICJOT0RFICV1OiAoJSJQUklwYWRkciItJSJQ
UklwYWRkciIpIGludGVydHdpbmUNCj4gd2l0aCBOT0RFICV1ICglIlBSSXBhZGRyIi0lIlBSSXBh
ZGRyIilcbiIsDQo+ID4gKwkJCSAgICAgICBuaWQsIHN0YXJ0LCBlbmQsIGksIG5kLT5zdGFydCwg
bmQtPmVuZCk7DQo+ID4gKwkJCXJldHVybiBmYWxzZTsNCj4gPiArCQl9DQo+ID4gKwl9DQo+ID4g
Kw0KPiA+ICsJcmV0dXJuIHRydWU7DQo+ID4gK30NCj4gPiArDQo+ID4gIC8qIENhbGxiYWNrIGZv
ciBwYXJzaW5nIG9mIHRoZSBQcm94aW1pdHkgRG9tYWluIDwtPiBNZW1vcnkgQXJlYQ0KPiBtYXBw
aW5ncyAqLw0KPiA+ICB2b2lkIF9faW5pdA0KPiA+ICBhY3BpX251bWFfbWVtb3J5X2FmZmluaXR5
X2luaXQoY29uc3Qgc3RydWN0IGFjcGlfc3JhdF9tZW1fYWZmaW5pdHkgKm1hKQ0KPiA+IEBAIC0z
NDQsNiArMzc0LDEyIEBAIGFjcGlfbnVtYV9tZW1vcnlfYWZmaW5pdHlfaW5pdChjb25zdCBzdHJ1
Y3QNCj4gYWNwaV9zcmF0X21lbV9hZmZpbml0eSAqbWEpDQo+ID4gIAkJCQluZC0+c3RhcnQgPSBz
dGFydDsNCj4gPiAgCQkJaWYgKG5kLT5lbmQgPCBlbmQpDQo+ID4gIAkJCQluZC0+ZW5kID0gZW5k
Ow0KPiA+ICsNCj4gPiArCQkJLyogQ2hlY2sgd2hldGhlciB0aGlzIHJhbmdlIGNvbnRhaW5zIG1l
bW9yeSBmb3Igb3RoZXINCj4gbm9kZXMgKi8NCj4gPiArCQkJaWYgKCFpc19ub2RlX21lbW9yeV9j
b250aW51b3VzKG5vZGUsIG5kLT5zdGFydCwgbmQtPmVuZCkpDQo+IHsNCj4gPiArCQkJCWJhZF9z
cmF0KCk7DQo+ID4gKwkJCQlyZXR1cm47DQo+ID4gKwkJCX0NCj4gPiAgCQl9DQo+ID4gIAl9DQo+
ID4gIAlwcmludGsoS0VSTl9JTkZPICJTUkFUOiBOb2RlICV1IFBYTSAldSAlIlBSSXBhZGRyIi0l
IlBSSXBhZGRyIiVzXG4iLA0KPiA+IC0tDQo+ID4gMi4yNS4xDQo+ID4NCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:30:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:30:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194874.347224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcqY-00052S-Nu; Fri, 24 Sep 2021 04:30:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194874.347224; Fri, 24 Sep 2021 04:30:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcqY-00051u-Jq; Fri, 24 Sep 2021 04:30:02 +0000
Received: by outflank-mailman (input) for mailman id 194874;
 Fri, 24 Sep 2021 04:30:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcqX-0004ux-97
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:30:01 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.40]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 13cd8dfa-1cf0-11ec-baa7-12813bfff9fa;
 Fri, 24 Sep 2021 04:29:59 +0000 (UTC)
Received: from AM5PR0301CA0025.eurprd03.prod.outlook.com
 (2603:10a6:206:14::38) by AM7PR08MB5479.eurprd08.prod.outlook.com
 (2603:10a6:20b:104::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:29:46 +0000
Received: from AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:14:cafe::86) by AM5PR0301CA0025.outlook.office365.com
 (2603:10a6:206:14::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:29:46 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT055.mail.protection.outlook.com (10.152.17.214) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:29:46 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Fri, 24 Sep 2021 04:29:43 +0000
Received: from 02980e9bd565.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F156BAF8-02A0-447F-BB18-DB787B447883.1; 
 Fri, 24 Sep 2021 04:29:33 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 02980e9bd565.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:29:33 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PAXPR08MB6477.eurprd08.prod.outlook.com (2603:10a6:102:12d::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:29:31 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:29:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13cd8dfa-1cf0-11ec-baa7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v3/But/Psy4Wob6vpKoPsMIkVP5NEP9GBQsgC6TY/NY=;
 b=2Ew83vGHVygIT7i3KIj0MQSMasmvK+Q2le+UYVmMqN8o92SnBZobNORYFWhLoIjZqrt/hCdlifaRsMdSIvfhMG2kkR/sldaSXCKYgjwtF4gbj+Hl0HMbQrppnhyQXNw3bPKyWnHu5EfBtPGr9tpzPlvHrgVsNpLXGqRda6W7pwM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IJzuc2vJMhaSmWrAyt1qRCOfmJIAsEyqDZmMvBfUmqV0s4PCT4PERPBQadcHQtcepdJEnOOMJIP8wsbKEGECyNOJS/T76oYKCQohb/WofAv+2voslHwPTyZWYLDe/85oj1ZS07QJarEIEQ9APezpfdeL9YCDK0ebhDwMYJww/eqdES+XSd3xQoy95D7yNCi7r4BybUZ+39DM6hoON1uwJD6EQstwG2KwEiyn9m2LjaJfxTIcv+EGH5upyLLxKtL5+AZVLCXFunMH0B7Mb88pmHScgw2gQf/kifRhmKMrbVkWBQmwT7TEJ0c466KctDoT8JQNTvtncYOOJfV18qZlpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=v3/But/Psy4Wob6vpKoPsMIkVP5NEP9GBQsgC6TY/NY=;
 b=iD302siRU0eH1MbT92gPTsdVNZFLYjTey6zP5f+xYvm0VdTZNk0t9cHDuLdeZuGqauSh5WYdshfBa+pTJoN8cBNV3wPAdBFyjIo9tvd5ZIun9iRoBdzTEoxaQXGsPrNTjQVrvueVDYKl2c9EQNPEJIYXSBlTKfxeDO1QrLdwctCPNajCPrivcYKTzVC0uDbkovrEl88+AQXkhJ2hwdYn1/zxuE9LClViNvWFCFTzz+8WBeBeF+kg2Zbr0VaQrF/2x7YlptGIfEht/rqe994fybZmRKc3xkAauEHy4YrJfo3Q1fCzyWwunFyybchSwYhmHqWIo1YmMz96PJ6ykIIjJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v3/But/Psy4Wob6vpKoPsMIkVP5NEP9GBQsgC6TY/NY=;
 b=2Ew83vGHVygIT7i3KIj0MQSMasmvK+Q2le+UYVmMqN8o92SnBZobNORYFWhLoIjZqrt/hCdlifaRsMdSIvfhMG2kkR/sldaSXCKYgjwtF4gbj+Hl0HMbQrppnhyQXNw3bPKyWnHu5EfBtPGr9tpzPlvHrgVsNpLXGqRda6W7pwM=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 10/37] xen/x86: use helpers to access/update mem_hotplug
Thread-Topic: [PATCH 10/37] xen/x86: use helpers to access/update mem_hotplug
Thread-Index: AQHXsHMVa+NLaHkUKUKV++B1/dWC4quyVkOAgABCPlA=
Date: Fri, 24 Sep 2021 04:29:31 +0000
Message-ID:
 <PAXPR08MB686434EC55F63DA87FAFEC4D9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-11-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231730580.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231730580.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: D6EFDF62BC87D44985BF2C264832F7EB.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 03e64f84-aceb-43a6-1fd2-08d97f13f048
x-ms-traffictypediagnostic: PAXPR08MB6477:|AM7PR08MB5479:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM7PR08MB5479261AF26400FAFE0A50869EA49@AM7PR08MB5479.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:4125;OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 JgYHosk3pGnCOzHYvqvGNi2C58KU+RQgxbphy0Z2kKXkrPlFk4mi3CnRMpRwvkLpgazDJbuJA/ftDMkRMMfZP//6k8g0oRFtAEaGgsIWl3kyx4L0XMkxlAnxjVLGWCwDYXk/AVSGvIkxS3PK0dWKBMj2q2Q3NtbOKabk3lLEKid+77KiFuulK1pkqFhA+CcDIH8jZ53lHLqjgk7fLv8Nqjeh5orV9HOxVkCloOPY1PkZZEvqoWb67CvW/dNLpcjN/Yo9ExQ5quXuGfr5AT5BZU32Dm9DOE9plwWZdyobU0Gda7Uw5Z8eEXQq1WliTZDWaJ8v85kXWgk/x8HAiwV0LcpfAo8O0WyKYv8Y0X/U238C1xXGwlso13SEs79//zT9CUPt5IZS1CAQgEn+VKhN6Yvawg6E6afX8ll/NTdAIayTPT6H+Dj/t3Papel5ssLKsH0Skj4XxhIIP2dwcWhT4eMgPp6nFJIh6hTeUe+6gLgHumygJyfd1ljeXnGIHOS5DNsJAWU2Btz/FhL4JcxV++2JcDNhHm2aeLTF7fPM4AmZ8AjOFxPf1OS7nJLh/h9F59ofGjrJvl7YUDFH/HW1N7l8EHMYaXucwnFTiCmvwmRP6WhhnpNLNSn/kg9AIdblb2syds8c3OWPOurRTF8R4Z2tWSR8ZJa8UVRa0wH4cN/PceTtcWF2dEOOJIgxIZDQ5O1IncQJ5RcpRp1EJ4gpjQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(66556008)(66476007)(8676002)(186003)(76116006)(38100700002)(66446008)(316002)(8936002)(86362001)(55016002)(66946007)(508600001)(54906003)(15650500001)(83380400001)(52536014)(6916009)(122000001)(7696005)(4326008)(71200400001)(38070700005)(26005)(9686003)(33656002)(6506007)(5660300002)(2906002)(53546011);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6477
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c70609cc-d4a2-4afe-ff29-08d97f13e790
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	BKZ99xN3Su6SRw+2PZNcUN2mfyAda8Tb8X2uWBWKhlxvkufA0M3bXX30E/AVvdplsKWHBrTAV7NEG7IwG0EBHaMkGp2gh1n7DtRNjOPO+n/zai0pcQIysneegFbqiUIXp/T+1PEAxu3O5eufoZtMBeL8PmPmTiD/Qby5o1bdNtRsc06DkFdZjy+IQXOTUM02HfBkQs2p7vq/Xf1j0PKRiPu66y1K5CS2IHTInWGUhhXVN6f/b2lBI3QdspzIxOOPS+9+j07j3aT+KLLhnmaHVbQwXhLoHPEScKwpWuKnxTwIJA4Z2B4YDJMlvD8GzdkdvSWXCJNSgXPgnN+eoS1OYpJmvqKlTPDM3SabKBgAYKD3hG7YrGMdyyOcbNwXkHhs3GssY8knP5K0D2xReEY590uL188Eoa22Xxiy9DcWvnrWEcasOaeWW6H0H1VUd+Laf5AFtDtSKUoWJlr69gzYh+tPhgeFvOhByAXTN03w+qb8daulZaCOO3uv0/+WOJsnaMyNWa62f3ZP1ptCUyyCKdqT7QFQk4LQxSnvGQ12A3DJJ7nx98wuRSDATKBmE+93UdYGJ6ylwQoMBglqdUqL8p1HtSHV5f9J+wg8+jG5+DxPcnMnWdkovln5SoI99egr+PiLoNe6NvZrABnCKcrzX04wWF3CN4qKo2Vg3uMMxg4KjrNGK7t+gG6zCHIIX58XHuh4cxiwLClhMCwLzve+xg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(33656002)(36860700001)(336012)(5660300002)(8936002)(82310400003)(8676002)(6506007)(15650500001)(186003)(52536014)(47076005)(53546011)(26005)(70586007)(70206006)(86362001)(83380400001)(81166007)(2906002)(508600001)(7696005)(356005)(4326008)(316002)(55016002)(54906003)(9686003)(6862004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:29:46.1401
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 03e64f84-aceb-43a6-1fd2-08d97f13f048
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT055.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5479

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogWGVuLWRldmVsIDx4ZW4t
ZGV2ZWwtYm91bmNlc0BsaXN0cy54ZW5wcm9qZWN0Lm9yZz4gT24gQmVoYWxmIE9mDQo+IFN0ZWZh
bm8gU3RhYmVsbGluaQ0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDg6MzINCj4gVG86IFdlaSBDaGVu
IDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3Jn
OyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxpZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFy
cXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hAc3VzZS5jb207DQo+IGFu
ZHJldy5jb29wZXIzQGNpdHJpeC5jb207IHJvZ2VyLnBhdUBjaXRyaXguY29tOyB3bEB4ZW4ub3Jn
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMTAvMzddIHhlbi94ODY6IHVzZSBoZWxwZXJzIHRvIGFj
Y2Vzcy91cGRhdGUNCj4gbWVtX2hvdHBsdWcNCj4gDQo+ICt4ODYgbWFpbnRhaW5lcnMNCj4gDQo+
IA0KPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiBXZSB3YW50IHRv
IGFic3RyYWN0IGNvZGUgZnJvbSBhY3BpX251bWFfbWVtb3J5X2FmZmluaXR5X2luaXQuDQo+ID4g
QnV0IG1lbV9ob3RwbHVnIGlzIGNvdXBsZWQgd2l0aCB4ODYuIEluIHRoaXMgcGF0Y2gsIHdlIHVz
ZQ0KPiA+IGhlbHBlcnMgdG8gcmVwYWNlIG1lbV9ob3RwbHVnIGRpcmVjdCBhY2Nlc3NpbmcuIFRo
aXMgd2lsbA0KPiAgICAgICAgICAgICAgXiByZXBsYWNlDQo+IA0KPiA+IGFsbG93IG1vc3QgY29k
ZSBjYW4gYmUgY29tbW9uLg0KPiAgICAgICAgICAgICAgICAgICBeIHRvIGJlDQo+IA0KPiBJIHRo
aW5rIHRoaXMgcGF0Y2ggY291bGQgYmUgbWVyZ2VkIHdpdGggdGhlIHByZXZpb3VzIHBhdGNoDQo+
IA0KDQpPaywgSSB3aWxsIGRvIGl0LCBhbmQgZml4IGFib3ZlIHR5cG9zDQoNCj4gDQo+ID4gU2ln
bmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4gLS0tDQo+ID4gIHhl
bi9hcmNoL3g4Ni9zcmF0LmMgfCA0ICsrLS0NCj4gPiAgMSBmaWxlIGNoYW5nZWQsIDIgaW5zZXJ0
aW9ucygrKSwgMiBkZWxldGlvbnMoLSkNCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC94
ODYvc3JhdC5jIGIveGVuL2FyY2gveDg2L3NyYXQuYw0KPiA+IGluZGV4IDJmMDhmYTQ2NjAuLjMz
MzRlZGU3YTUgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gveDg2L3NyYXQuYw0KPiA+ICsrKyBi
L3hlbi9hcmNoL3g4Ni9zcmF0LmMNCj4gPiBAQCAtMzkxLDggKzM5MSw4IEBAIGFjcGlfbnVtYV9t
ZW1vcnlfYWZmaW5pdHlfaW5pdChjb25zdCBzdHJ1Y3QNCj4gYWNwaV9zcmF0X21lbV9hZmZpbml0
eSAqbWEpDQo+ID4gIAltZW1ibGtfbm9kZWlkW251bV9ub2RlX21lbWJsa3NdID0gbm9kZTsNCj4g
PiAgCWlmIChtYS0+ZmxhZ3MgJiBBQ1BJX1NSQVRfTUVNX0hPVF9QTFVHR0FCTEUpIHsNCj4gPiAg
CQlfX3NldF9iaXQobnVtX25vZGVfbWVtYmxrcywgbWVtYmxrX2hvdHBsdWcpOw0KPiA+IC0JCWlm
IChlbmQgPiBtZW1faG90cGx1ZykNCj4gPiAtCQkJbWVtX2hvdHBsdWcgPSBlbmQ7DQo+ID4gKwkJ
aWYgKGVuZCA+IG1lbV9ob3RwbHVnX2JvdW5kYXJ5KCkpDQo+ID4gKwkJCW1lbV9ob3RwbHVnX3Vw
ZGF0ZV9ib3VuZGFyeShlbmQpOw0KPiA+ICAJfQ0KPiA+ICAJbnVtX25vZGVfbWVtYmxrcysrOw0K
PiA+ICB9DQo+ID4gLS0NCj4gPiAyLjI1LjENCj4gPg0KDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:35:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:35:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194880.347235 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcvX-0006Kv-A1; Fri, 24 Sep 2021 04:35:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194880.347235; Fri, 24 Sep 2021 04:35:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcvX-0006Ko-6t; Fri, 24 Sep 2021 04:35:11 +0000
Received: by outflank-mailman (input) for mailman id 194880;
 Fri, 24 Sep 2021 04:35:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcvW-0006Kh-HA
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:35:10 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.57]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bfa9576b-e26c-477f-a6ba-a69718dbcbb4;
 Fri, 24 Sep 2021 04:35:08 +0000 (UTC)
Received: from DB6PR0201CA0003.eurprd02.prod.outlook.com (2603:10a6:4:3f::13)
 by AM0PR08MB4161.eurprd08.prod.outlook.com (2603:10a6:208:134::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 04:35:03 +0000
Received: from DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:3f:cafe::b5) by DB6PR0201CA0003.outlook.office365.com
 (2603:10a6:4:3f::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:35:03 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT053.mail.protection.outlook.com (10.152.21.119) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:35:02 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Fri, 24 Sep 2021 04:35:01 +0000
Received: from 2269cab4da01.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0CB02BDC-C2A7-4F28-8FFC-88242D302714.1; 
 Fri, 24 Sep 2021 04:34:49 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2269cab4da01.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:34:49 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com (2603:10a6:102:132::14)
 by PAXPR08MB6493.eurprd08.prod.outlook.com (2603:10a6:102:15a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:34:26 +0000
Received: from PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be]) by PAXPR08MB6864.eurprd08.prod.outlook.com
 ([fe80::14c4:e1c6:bc6d:77be%6]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:34:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfa9576b-e26c-477f-a6ba-a69718dbcbb4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kbxhEwLcYaFANGdHa6f541Zk8Gg8706fzHfZtEIl7r4=;
 b=bbX5SKM0njKyc1VypDWCAtw4xmvjsy/TWkPQvu3BExo12cx6XKOkFy3XAgPceydRLyGFr8G0WwHXJHN9WDK+kEeYFKVe290YaQw3zLqju6ENOZm5F6nnnr3HPmOh/qCx45DRFDFJjwnvDUk1/eES0n0+TzXgs8hSuZxbePShgSY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Oq+cm7MyXGl3aR90QcBxZHqUWOj4B9VMgPNnjFa6D9UlDhTfIl4aIi4g2iVWsVQz6f03afd8NwRsnyuWdZ7LdNstKNZHqv6HcPNJcqBY0L4EArjhCPl54buqAmXyrJkq8mY6W4B61oSb+e9rFL6NOAQTbYkjaH1d66WqLq8Gak5O9hkLo7yRWLnjuyJehogHKm3hRqUgLxCyKN8OSoK1mQyB1Poo4osQMtNGFnCDWgJ2ik1uxTpSUqyGB034dRSwGwqIRb2oDqLxlcN61oejQcZ0Bh9fGcKl2Chuxt5hECxXU8LmJYWqnXrB4+5li6mTqea1zeaZdFdo8bPWzpec/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kbxhEwLcYaFANGdHa6f541Zk8Gg8706fzHfZtEIl7r4=;
 b=Yf/0OsRJV1z4RadadI+pEkYC7zmokTB6jqv0UPAKbT+IMhnl2s7cdpty02UEOSUCXs2Tkt/TGoLv9Xw6JoK4Lls+1UgdLJ/lnSCgUCbKOrO+guQgeKNsmZ/DptJoNq38kripM7/ORsxrFdst+Ft3ojcEXh1XevDFTtvWeNJuekT4S4AbrMs5dwbS3UCfcvm6+LU2bOlkUXpczhGYHU/7EYDwm1oCk3ddTg9JHrbSZJ2dYPdrDUQEUfrWZtEl4t86jGUnY74y7RRVgm7gCnqV/Dvnexea1CwJN9dW7z83vC0HKfSIiBVrr+4/Wzjg+wcJBVWzIvJfMGbwn/SpmbXlXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kbxhEwLcYaFANGdHa6f541Zk8Gg8706fzHfZtEIl7r4=;
 b=bbX5SKM0njKyc1VypDWCAtw4xmvjsy/TWkPQvu3BExo12cx6XKOkFy3XAgPceydRLyGFr8G0WwHXJHN9WDK+kEeYFKVe290YaQw3zLqju6ENOZm5F6nnnr3HPmOh/qCx45DRFDFJjwnvDUk1/eES0n0+TzXgs8hSuZxbePShgSY=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Topic: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Index: AQHXsHMtR+l18+MmpE6rl/RjSXTBgquyYmkAgAA2xQA=
Date: Fri, 24 Sep 2021 04:34:26 +0000
Message-ID:
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 81D3082093BE4F4C9C52CF9AD266759E.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 1a5503b8-64e6-4ea5-82b9-08d97f14ac9b
x-ms-traffictypediagnostic: PAXPR08MB6493:|AM0PR08MB4161:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB4161743AB1F73A0A75CB5A0E9EA49@AM0PR08MB4161.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 6VyYTR4qB5THRHBcua1V8Ppsjq+6xS7zuXdwgeDSDUMwVV9MTLQaQnBoMBMVsKhru2H0m62I+yyKg5CHiEzXUKMUDSS4DzBzmTOjKqjSgOPS6vH0VERD94SYZuFeq0P0b8G6H4Kb1n5f2V3WRVrlCG873VorQas0wLue+ceA438DAvwNKXHumnRJEN1daepfkGjebNd1NiQWrC28sgw/Ib8nA0DTkiTE4zKwzVj+Agq5PccvTVshf8v1Y/C0juLZKhpB7mUsy6llKdPEJuF38jBKjp2KgLBCqk6jyvXHQJNIG3D3mx5kH2pVv/nYGCZuc87AYJGSrgYaOQVYHSKlqLQKPiID4gLu5iq/wnOmwALt4Y4I2dwhMFOAD958oTt8zxMqSyo35g8TyskjlwPBeCf1fiVX8wht7yOQ8mCNIZZn8P6xH76ATi4uFQ67+kMQ6bDJWUcgVAzOm/DSGcgAdEB41cMeDSP7diXJOEPYFOYEcmnNQqmbyQbPB5xxMeLT/b+ftryalMyWNLWCRb9L5ELzDzb38C2zkW46bdX9nWNWdd1PbPscSOpipHSyvbe0WD178VKVlIr5NoIHgcTcHan057gGvunfQy4nHF+zF4imd92ySEfZPOzbhH8fHXjhVoAmsOjCMYBvFaDC4Rzp75eQFO1X5MOznOiPCrb9+zbFj6Ult9SAYYyRTA/MqM6rDumhzrGBkRUIelqM7YkHOA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6864.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(52536014)(6916009)(76116006)(38100700002)(2906002)(54906003)(26005)(7696005)(66446008)(122000001)(66556008)(66476007)(5660300002)(66946007)(64756008)(316002)(38070700005)(53546011)(6506007)(83380400001)(55016002)(186003)(71200400001)(4326008)(86362001)(8676002)(9686003)(33656002)(8936002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6493
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	feac45ae-4107-4dc3-8667-08d97f149756
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lA8ChsIPtNdPjPX42a5ktL2w4xj3UHIZsUmIgBATdTNju/rusgWhbF3zNXqOyI+JZNGWzRcnGJj0uixViNpfi9ktBQy49nsyao9qhagvyGiHqC0KMIpRbNGgku9S9PjUO8FI/Woj0qthVEAPeKcqC5DvSJB1iCka0jZZhxuCBPCLQ/JxtFLrB1lRkulGMP3NTBim4KykWEL/1A96zaI4GAw/HlwFPIrshCNeqnYHqauUD7D++L25QM8O4xY/4zQ8KAgwdI4W+D9+2mkqprigK1bKkWIr7j5SFbUdvL2/R0E9t9CZQZIlAf6VgXyZeRR2Vbvsiv6NNoX408aQve4KMgZbYh+h6VVUNNPy29FMEB8wav5yvkwzntrq/0WLQO2f9meZ8+GpobFs+T3MDFfS/V18/+zPulPJvp5xluxcQw+Ktu9nHXGL/SU7rbhj056qTKQ6wp0XPYn0JaAGDxrkFiEaPdv3Ox0w4Qvn1Ymgy6Y6VwlPUhb6rp1c+UKhbimzXM5JbN0odGyKkt3JLCU20O9DHnG1sWf0hvaZgNqB0M9CgCbpFloEBknPrJRSo4V91NiAwgipyLd/vDQvyVaKQhjSaMWu7GyL2K15baJs2hUsD1iZ74GVr4N/AHuyoE7UPPSnjH52GUz8EB5z/vZAeadKp/5jMwf5l+LKD7s3XbVDYMoMRavePiGz6TDKPNnTS+8HTfd9M0k3x1zHD+EN+g==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(8936002)(54906003)(55016002)(2906002)(33656002)(6506007)(336012)(7696005)(53546011)(36860700001)(82310400003)(52536014)(9686003)(47076005)(316002)(4326008)(186003)(508600001)(26005)(356005)(83380400001)(86362001)(81166007)(70206006)(70586007)(6862004)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:35:02.1532
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a5503b8-64e6-4ea5-82b9-08d97f14ac9b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4161

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgOToxNQ0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4t
ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGll
bkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjAvMzddIHhlbjogaW50cm9kdWNlIENPTkZJR19FRkkg
dG8gc3R1YiBBUEkgZm9yIG5vbi0NCj4gRUZJIGFyY2hpdGVjdHVyZQ0KPiANCj4gT24gVGh1LCAy
MyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gU29tZSBhcmNoaXRlY3R1cmVzIGRvIG5v
dCBzdXBwb3J0IEVGSSwgYnV0IEVGSSBBUEkgd2lsbCBiZSB1c2VkDQo+ID4gaW4gc29tZSBjb21t
b24gZmVhdHVyZXMuIEluc3RlYWQgb2Ygc3ByZWFkaW5nICNpZmRlZiBBUkNILCB3ZQ0KPiA+IGlu
dHJvZHVjZSB0aGlzIEtjb25maWcgb3B0aW9uIHRvIGdpdmUgWGVuIHRoZSBhYmlsaXR5IG9mIHN0
dWJpbmcNCj4gPiBFRkkgQVBJIGZvciBub24tRUZJIHN1cHBvcnRlZCBhcmNoaXRlY3R1cmVzLg0K
PiA+DQo+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4g
LS0tDQo+ID4gIHhlbi9hcmNoL2FybS9LY29uZmlnICB8ICAxICsNCj4gPiAgeGVuL2FyY2gvYXJt
L01ha2VmaWxlIHwgIDIgKy0NCj4gPiAgeGVuL2FyY2gveDg2L0tjb25maWcgIHwgIDEgKw0KPiA+
ICB4ZW4vY29tbW9uL0tjb25maWcgICAgfCAxMSArKysrKysrKysrKw0KPiA+ICB4ZW4vaW5jbHVk
ZS94ZW4vZWZpLmggfCAgNCArKysrDQo+ID4gIDUgZmlsZXMgY2hhbmdlZCwgMTggaW5zZXJ0aW9u
cygrKSwgMSBkZWxldGlvbigtKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9L
Y29uZmlnIGIveGVuL2FyY2gvYXJtL0tjb25maWcNCj4gPiBpbmRleCBlY2ZhNjgyMmU0Li44NjVh
ZDgzYTg5IDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9hcmNoL2FybS9LY29uZmlnDQo+ID4gKysrIGIv
eGVuL2FyY2gvYXJtL0tjb25maWcNCj4gPiBAQCAtNiw2ICs2LDcgQEAgY29uZmlnIEFSTV82NA0K
PiA+ICAJZGVmX2Jvb2wgeQ0KPiA+ICAJZGVwZW5kcyBvbiAhQVJNXzMyDQo+ID4gIAlzZWxlY3Qg
NjRCSVQNCj4gPiArCXNlbGVjdCBFRkkNCj4gPiAgCXNlbGVjdCBIQVNfRkFTVF9NVUxUSVBMWQ0K
PiA+DQo+ID4gIGNvbmZpZyBBUk0NCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL01ha2Vm
aWxlIGIveGVuL2FyY2gvYXJtL01ha2VmaWxlDQo+ID4gaW5kZXggM2QzYjk3YjViNC4uYWU0ZWZi
Zjc2ZSAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4gPiArKysgYi94
ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4gPiBAQCAtMSw2ICsxLDYgQEANCj4gPiAgb2JqLSQoQ09O
RklHX0FSTV8zMikgKz0gYXJtMzIvDQo+ID4gIG9iai0kKENPTkZJR19BUk1fNjQpICs9IGFybTY0
Lw0KPiA+IC1vYmotJChDT05GSUdfQVJNXzY0KSArPSBlZmkvDQo+ID4gK29iai0kKENPTkZJR19F
RkkpICs9IGVmaS8NCj4gPiAgb2JqLSQoQ09ORklHX0FDUEkpICs9IGFjcGkvDQo+ID4gIGlmbmVx
ICgkKENPTkZJR19OT19QTEFUKSx5KQ0KPiA+ICBvYmoteSArPSBwbGF0Zm9ybXMvDQo+ID4gZGlm
ZiAtLWdpdCBhL3hlbi9hcmNoL3g4Ni9LY29uZmlnIGIveGVuL2FyY2gveDg2L0tjb25maWcNCj4g
PiBpbmRleCAyOGQxM2I5NzA1Li5iOWVkMTg3ZjZiIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9hcmNo
L3g4Ni9LY29uZmlnDQo+ID4gKysrIGIveGVuL2FyY2gveDg2L0tjb25maWcNCj4gPiBAQCAtMTAs
NiArMTAsNyBAQCBjb25maWcgWDg2DQo+ID4gIAlzZWxlY3QgQUxURVJOQVRJVkVfQ0FMTA0KPiA+
ICAJc2VsZWN0IEFSQ0hfU1VQUE9SVFNfSU5UMTI4DQo+ID4gIAlzZWxlY3QgQ09SRV9QQVJLSU5H
DQo+ID4gKwlzZWxlY3QgRUZJDQo+ID4gIAlzZWxlY3QgSEFTX0FMVEVSTkFUSVZFDQo+ID4gIAlz
ZWxlY3QgSEFTX0NPTVBBVA0KPiA+ICAJc2VsZWN0IEhBU19DUFVGUkVRDQo+ID4gZGlmZiAtLWdp
dCBhL3hlbi9jb21tb24vS2NvbmZpZyBiL3hlbi9jb21tb24vS2NvbmZpZw0KPiA+IGluZGV4IDll
YmIxYzIzOWIuLmY5OTg3NDZhMWEgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2NvbW1vbi9LY29uZmln
DQo+ID4gKysrIGIveGVuL2NvbW1vbi9LY29uZmlnDQo+ID4gQEAgLTExLDYgKzExLDE2IEBAIGNv
bmZpZyBDT01QQVQNCj4gPiAgY29uZmlnIENPUkVfUEFSS0lORw0KPiA+ICAJYm9vbA0KPiA+DQo+
ID4gK2NvbmZpZyBFRkkNCj4gPiArCWJvb2wNCj4gDQo+IFdpdGhvdXQgdGhlIHRpdGxlIHRoZSBv
cHRpb24gaXMgbm90IHVzZXItc2VsZWN0YWJsZSAob3IgZGUtc2VsZWN0YWJsZSkuDQo+IFNvIHRo
ZSBoZWxwIG1lc3NhZ2UgYmVsb3cgY2FuIG5ldmVyIGJlIHNlZW4uDQo+IA0KPiBFaXRoZXIgYWRk
IGEgdGl0bGUsIGUuZy46DQo+IA0KPiBib29sICJFRkkgc3VwcG9ydCINCj4gDQo+IE9yIGZ1bGx5
IG1ha2UgdGhlIG9wdGlvbiBhIHNpbGVudCBvcHRpb24gYnkgcmVtb3ZpbmcgdGhlIGhlbHAgdGV4
dC4NCj4gDQo+IA0KDQpPSywgaW4gY3VycmVudCBYZW4gY29kZSwgRUZJIGlzIHVuY29uZGl0aW9u
YWxseSBjb21waWxlZC4gQmVmb3JlDQp3ZSBjaGFuZ2UgcmVsYXRlZCBjb2RlLCBJIHByZWZlciB0
byByZW1vdmUgdGhlIGhlbHAgdGV4dC4NCg0KPiANCj4gPiArCS0tLWhlbHAtLS0NCj4gPiArICAg
ICAgVGhpcyBvcHRpb24gcHJvdmlkZXMgc3VwcG9ydCBmb3IgcnVudGltZSBzZXJ2aWNlcyBwcm92
aWRlZA0KPiA+ICsgICAgICBieSBVRUZJIGZpcm13YXJlIChzdWNoIGFzIG5vbi12b2xhdGlsZSB2
YXJpYWJsZXMsIHJlYWx0aW1lDQo+ID4gKyAgICAgIGNsb2NrLCBhbmQgcGxhdGZvcm0gcmVzZXQp
LiBBIFVFRkkgc3R1YiBpcyBhbHNvIHByb3ZpZGVkIHRvDQo+ID4gKyAgICAgIGFsbG93IHRoZSBr
ZXJuZWwgdG8gYmUgYm9vdGVkIGFzIGFuIEVGSSBhcHBsaWNhdGlvbi4gVGhpcw0KPiA+ICsgICAg
ICBpcyBvbmx5IHVzZWZ1bCBmb3Iga2VybmVscyB0aGF0IG1heSBydW4gb24gc3lzdGVtcyB0aGF0
IGhhdmUNCj4gPiArICAgICAgVUVGSSBmaXJtd2FyZS4NCj4gPiArDQo+ID4gIGNvbmZpZyBHUkFO
VF9UQUJMRQ0KPiA+ICAJYm9vbCAiR3JhbnQgdGFibGUgc3VwcG9ydCIgaWYgRVhQRVJUDQo+ID4g
IAlkZWZhdWx0IHkNCj4gPiBAQCAtMTk2LDYgKzIwNiw3IEBAIGNvbmZpZyBLRVhFQw0KPiA+DQo+
ID4gIGNvbmZpZyBFRklfU0VUX1ZJUlRVQUxfQUREUkVTU19NQVANCj4gPiAgICAgIGJvb2wgIkVG
STogY2FsbCBTZXRWaXJ0dWFsQWRkcmVzc01hcCgpIiBpZiBFWFBFUlQNCj4gPiArICAgIGRlcGVu
ZHMgb24gRUZJDQo+ID4gICAgICAtLS1oZWxwLS0tDQo+ID4gICAgICAgIENhbGwgRUZJIFNldFZp
cnR1YWxBZGRyZXNzTWFwKCkgcnVudGltZSBzZXJ2aWNlIHRvIHNldHVwIG1lbW9yeQ0KPiBtYXAg
Zm9yDQo+ID4gICAgICAgIGZ1cnRoZXIgcnVudGltZSBzZXJ2aWNlcy4gQWNjb3JkaW5nIHRvIFVF
Rkkgc3BlYywgaXQgaXNuJ3QNCj4gc3RyaWN0bHkNCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1
ZGUveGVuL2VmaS5oIGIveGVuL2luY2x1ZGUveGVuL2VmaS5oDQo+ID4gaW5kZXggOTRhN2U1NDdm
OS4uNjYxYTQ4Mjg2YSAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS94ZW4vZWZpLmgNCj4g
PiArKysgYi94ZW4vaW5jbHVkZS94ZW4vZWZpLmgNCj4gPiBAQCAtMjUsNiArMjUsOCBAQCBleHRl
cm4gc3RydWN0IGVmaSBlZmk7DQo+ID4NCj4gPiAgI2lmbmRlZiBfX0FTU0VNQkxZX18NCj4gPg0K
PiA+ICsjaWZkZWYgQ09ORklHX0VGSQ0KPiA+ICsNCj4gPiAgdW5pb24geGVucGZfZWZpX2luZm87
DQo+ID4gIHVuaW9uIGNvbXBhdF9wZl9lZmlfaW5mbzsNCj4gPg0KPiA+IEBAIC00NSw2ICs0Nyw4
IEBAIGludCBlZmlfcnVudGltZV9jYWxsKHN0cnVjdCB4ZW5wZl9lZmlfcnVudGltZV9jYWxsICop
Ow0KPiA+ICBpbnQgZWZpX2NvbXBhdF9nZXRfaW5mbyh1aW50MzJfdCBpZHgsIHVuaW9uIGNvbXBh
dF9wZl9lZmlfaW5mbyAqKTsNCj4gPiAgaW50IGVmaV9jb21wYXRfcnVudGltZV9jYWxsKHN0cnVj
dCBjb21wYXRfcGZfZWZpX3J1bnRpbWVfY2FsbCAqKTsNCj4gPg0KPiA+ICsjZW5kaWYgLyogQ09O
RklHX0VGSSovDQo+ID4gKw0KPiA+ICAjZW5kaWYgLyogIV9fQVNTRU1CTFlfXyAqLw0KPiA+DQo+
ID4gICNlbmRpZiAvKiBfX1hFTl9FRklfSF9fICovDQo+ID4gLS0NCj4gPiAyLjI1LjENCj4gPg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:36:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:36:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194887.347245 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcx8-00070D-Ow; Fri, 24 Sep 2021 04:36:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194887.347245; Fri, 24 Sep 2021 04:36:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTcx8-000706-Lv; Fri, 24 Sep 2021 04:36:50 +0000
Received: by outflank-mailman (input) for mailman id 194887;
 Fri, 24 Sep 2021 04:36:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTcx7-0006zs-72
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:36:49 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::621])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f5770aee-7044-4e82-bf57-dc7579dfddd1;
 Fri, 24 Sep 2021 04:36:47 +0000 (UTC)
Received: from AS9PR06CA0269.eurprd06.prod.outlook.com (2603:10a6:20b:45f::32)
 by PR2PR08MB4681.eurprd08.prod.outlook.com (2603:10a6:101:20::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:36:37 +0000
Received: from VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::a1) by AS9PR06CA0269.outlook.office365.com
 (2603:10a6:20b:45f::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:36:37 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT004.mail.protection.outlook.com (10.152.18.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:36:37 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Fri, 24 Sep 2021 04:36:36 +0000
Received: from a2a297f5982e.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 9C50575E-169D-440D-8ED5-037014C52036.1; 
 Fri, 24 Sep 2021 04:36:26 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a2a297f5982e.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:36:26 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5622.eurprd08.prod.outlook.com (2603:10a6:10:1af::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:36:23 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:36:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5770aee-7044-4e82-bf57-dc7579dfddd1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tVeTKmFk2chsP5nTlMY3FXuEgOnCDrECEp7oEM9EoLA=;
 b=INBqV41BacFudr+LACwxwgQrRYmvZqZUQHsSjjDb7WVuYSsu+7fe8MTvEFn8gXZWNLvyDMjqgjwk7Ewd3Ti/b4bw0lyClFUVjrTUw5dlgVqEUe/3Ozjsgqj756oSEi7lcdbJp7KIXvTmfyH/j6WbbMk1dFSpLeCpzKV8LIYrQaU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n0YaAbO4+wLePv7LipsZqiyGWC5zHj40dnARG6uKWiJb72M+QO6XLopW4UOh/rOJl1cboIiAynhY5pO4W/h3KPyo7VAHCxyK06BrkO0nV4BSoCliwEHuMylf9PB4VxcuyDyRKkBbc6dTYhE26Wt/O5FEIbWG2FWc283FFRfZeqmDaobGQRmjRewBFFnftxoqhQey6xWqBHEHkglTPZrNXABRs9/CcZJotpaYf/yVURF0MjF07Zts/8Djb5lDaNnjxedNb7q152iCQYMqo84mWTt+jwCFj3QcucWyM/Jy5Jy7rGkik8XNHqBZHRVj/lS39mYeRiPKetYe/csNf98saQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tVeTKmFk2chsP5nTlMY3FXuEgOnCDrECEp7oEM9EoLA=;
 b=jSCLauedtxenkTU4xL6LdPcLdQgVkPp0RXk6kJpY0Y5AAPjojRbHwi+iUzsR1Hh2/Mhg/1EZKZd67hdX6KChm3Zs9kljWDLWoQ+mOBmjUbbMLldmTP7sDH/wDM9Q8B5j6JD12zoUSkDnwkubgjEOnldMQUQCKiRCA6idCHZ/OUgIjORHeFbm5d2k4RjcxsA8MrBXZFpLMYhfc3WrWQR0yK1YAjTUpixK52kOOojU2++y6M3k4+mStSlm8dpo5PAIffnBW3CEitHo6gDSbL2J4KyjNY1t3ck9UDYRUseFF7PhW0bHcORalWFnu9qYvrqCkhNdvDGJ/WWLaWJ5ZKNuTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tVeTKmFk2chsP5nTlMY3FXuEgOnCDrECEp7oEM9EoLA=;
 b=INBqV41BacFudr+LACwxwgQrRYmvZqZUQHsSjjDb7WVuYSsu+7fe8MTvEFn8gXZWNLvyDMjqgjwk7Ewd3Ti/b4bw0lyClFUVjrTUw5dlgVqEUe/3Ozjsgqj756oSEi7lcdbJp7KIXvTmfyH/j6WbbMk1dFSpLeCpzKV8LIYrQaU=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 21/37] xen/arm: Keep memory nodes in dtb for NUMA when
 boot from EFI
Thread-Topic: [PATCH 21/37] xen/arm: Keep memory nodes in dtb for NUMA when
 boot from EFI
Thread-Index: AQHXsHMrd58EkQofR0OqZuXsqMtGrKuyZJkAgAA1glA=
Date: Fri, 24 Sep 2021 04:36:23 +0000
Message-ID:
 <DB9PR08MB685714E62F7FAC1E2E137FE59EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-22-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231822090.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231822090.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 42CA5B9B816F1941868DFDE0F90CA2FC.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 0a8b42f1-f1b4-48c1-e63c-08d97f14e54c
x-ms-traffictypediagnostic: DBAPR08MB5622:|PR2PR08MB4681:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PR2PR08MB4681FC28D450B356DD4459499EA49@PR2PR08MB4681.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 BsnWqRxFEKXP7GWTUWS+4cP6bPrzZj2LDWs522yrjoEJwnFvmWmLEGcYAP3WCzi+k7H8F8sEW6QJsTYZfMxJEJmVosug1UcLGtFBx9cQad+qrCmBBu+F2wBC5I70svzroBuILdGsDOCquBE6OIQ3dhaE5oNAEGRgqdONi2RVZUGfRw6TP46gZTIxTZ7gLokrBBxKHv5d9B4AxcBcT1+SubFLNUFFfGTnnLolLvY10Zf6yYi48Lx6LZGLgRdxpLuFEKRyejvEwkYv/nzWzNPLkeMQnUmH7+em+31nmlFYtQFv0EugLEk0jFluLtbY9UCd7uvKuH63kXMoDZorw13yw//+Blqr2urxn04EreJMUn+qdAv9gs7iwB+fFjsxBkmJpFFTHTIx9BEuLFzs5ShE1c8LRljAzpz9QTh5an6+hUvr/v2ePnxRpETvsx2EJhdy8y3lkzwMsm4aZb54ttVeUcraj7Hhpzx/32l78UaASy0IcRrXnLRXO7XyL49PuNIbKwEczj0zA+gdrNGS3PzKknOZS7C42BsGdmaoe3rgu/As90UyMCkXSs6nfSLW16zBDhjVvypEdIrmHMKqmTwnKGKivXFq8BM4j77kODr/Qc2hjgGCluCpsH4ecC4pK8I4hlpBdQpJTuxsXQxkOoQwTQrMsqeYJoje3tkhSjRRXH/1h7oyQQwLoFl87Utyt3heD4eMHRCz0JTTUqHOqbPXEw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6506007)(8936002)(26005)(53546011)(7696005)(55016002)(38070700005)(5660300002)(66446008)(76116006)(64756008)(66946007)(66476007)(66556008)(186003)(52536014)(8676002)(83380400001)(9686003)(33656002)(122000001)(6916009)(508600001)(71200400001)(38100700002)(86362001)(4326008)(54906003)(2906002)(316002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5622
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	02e277ab-fda3-418c-f7fc-08d97f14dd17
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1axZoOUNCREROq/nmQ6vUlF8+PXv0gJJR1kR3eGCcV6wHhTX5oCDE8psIjkeu4Qw6sLNUNJ4WuPwswQtphee65xGxPSGRe5mpcPUHFSfnGp75CjF+8Y1iHTyD9lzHmzVr680njitNqfCd04km9FlZZU+DlOuvoae5YMt+frMkNwlTAqF7lghcf5Ixd6eDLc4h4843VIgwxbMjTaUKqle+FD50E3/3mHjuxUcneCmx8uGwlYZirngPWZdSp3KauBHZEM3NDjUODtHWXV0t1ihXUwC9bz5jtG1o1ly6KQlYKHK21603tLuT5mNDKl9Hu3M3JPQlh5KRb5Fghyq9pNvyzEu4vl7ZaPqChdMqII2fzt/dbTCVGHwywQ9VZeUmwiIDyE5wgjLnTgQHmyKp9mvoJ2h7GPTupdc9tpa9WOOHIb/lTOrLy4N8I6HAQMDwCKpWUywZzJuZoE1yKuketIfT9ORq8o16SVWRZtEiU6BmwbZlgkMd4BE91oPznxvXI80dMsXkQTILgB9+kZ/G98V94lv8Tsdrf6NRvXhdXIG7kXBTGQYDDO6jpXQl5T7dxbJfzpMyPjmUkKIpe49G5dUn5QA116crP31BBbd/QRq+JIuCABlYxzJnSz64+7UMf3Q+vuZskufIXDW5KpxgS6/+iMHyNWr0MtIYURcuUKJNF+aXtwH2O/C+VRbNbxO1nM94a7EcWW+BmbgRwHTo4ssfg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(82310400003)(8676002)(83380400001)(336012)(7696005)(5660300002)(316002)(70206006)(70586007)(508600001)(53546011)(26005)(54906003)(33656002)(36860700001)(2906002)(9686003)(6862004)(47076005)(52536014)(55016002)(186003)(8936002)(356005)(81166007)(86362001)(4326008)(6506007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:36:37.1349
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a8b42f1-f1b4-48c1-e63c-08d97f14e54c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4681

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDogMjAyMcTqOdTCMjTI1SA5OjIzDQo+
IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0cy54
ZW5wcm9qZWN0Lm9yZzsgc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVuQHhlbi5vcmc7DQo+
IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gU3ViamVjdDog
UmU6IFtQQVRDSCAyMS8zN10geGVuL2FybTogS2VlcCBtZW1vcnkgbm9kZXMgaW4gZHRiIGZvciBO
VU1BIHdoZW4NCj4gYm9vdCBmcm9tIEVGSQ0KPiANCj4gT24gVGh1LCAyMyBTZXAgMjAyMSwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gRUZJIGNhbiBnZXQgbWVtb3J5IG1hcCBmcm9tIEVGSSBzeXN0ZW0g
dGFibGUuIEJ1dCBFRkkgc3lzdGVtDQo+ID4gdGFibGUgZG9lc24ndCBjb250YWluIG1lbW9yeSBO
VU1BIGluZm9ybWF0aW9uLCBFRkkgZGVwZW5kcyBvbg0KPiA+IEFDUEkgU1JBVCBvciBkZXZpY2Ug
dHJlZSBtZW1vcnkgbm9kZSB0byBwYXJzZSBtZW1vcnkgYmxvY2tzJw0KPiA+IE5VTUEgbWFwcGlu
Zy4NCj4gPg0KPiA+IEJ1dCBpbiBjdXJyZW50IGNvZGUsIHdoZW4gWGVuIGlzIGJvb3RpbmcgZnJv
bSBFRkksIGl0IHdpbGwNCj4gPiBkZWxldGUgYWxsIG1lbW9yeSBub2RlcyBpbiBkZXZpY2UgdHJl
ZS4gU28gaW4gVUVGSSArIERUQg0KPiA+IGJvb3QsIHdlIGRvbid0IGhhdmUgbnVtYS1ub2RlLWlk
IGZvciBtZW1vcnkgYmxvY2tzIGFueSBtb3JlLg0KPiA+DQo+ID4gU28gaW4gdGhpcyBwYXRjaCwg
d2Ugd2lsbCBrZWVwIG1lbW9yeSBub2RlcyBpbiBkZXZpY2UgdHJlZSBmb3INCj4gPiBOVU1BIGNv
ZGUgdG8gcGFyc2UgbWVtb3J5IG51bWEtbm9kZS1pZCBsYXRlci4NCj4gPg0KPiA+IEFzIGEgc2lk
ZSBlZmZlY3QsIGlmIHdlIHN0aWxsIHBhcnNlIGJvb3QgbWVtb3J5IGluZm9ybWF0aW9uIGluDQo+
ID4gZWFybHlfc2Nhbl9ub2RlLCBib290bWVtLmluZm8gd2lsbCBjYWxjdWxhdGUgbWVtb3J5IHJh
bmdlcyBpbg0KPiA+IG1lbW9yeSBub2RlcyB0d2ljZS4gU28gd2UgaGF2ZSB0byBwcmV2ZW50IGVh
cmx5X3NjYW5fbm9kZSB0bw0KPiA+IHBhcnNlIG1lbW9yeSBub2RlcyBpbiBFRkkgYm9vdC4NCj4g
Pg0KPiA+IEFzIEVGSSBBUElzIG9ubHkgY2FuIGJlIHVzZWQgaW4gQXJtNjQsIHNvIHdlIGludHJv
ZHVjZWQgYSBzdHViDQo+ID4gQVBJIGZvciBub24tRUZJIHN1cHBvcnRlZCBBcm0zMi4gVGhpcyB3
aWxsIHByZXZlbnQNCj4gDQo+IFRoaXMgbGFzdCBzZW50ZW5jZSBpcyBpbmNvbXBsZXRlLg0KPiAN
Cj4gQnV0IGFzaWRlIGZyb20gdGhhdCwgdGhpcyBwYXRjaCBsb29rcyBnb29kIHRvIG1lLg0KPiAN
Cg0KQWgsIGl0IHRydW5jYXRlZCBieSBhY2NpZGVudC4gSSB3aWxsIGZpeCBpdC4NCg0KPiANCj4g
PiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAtLS0NCj4g
PiAgeGVuL2FyY2gvYXJtL2Jvb3RmZHQuYyAgICAgIHwgIDggKysrKysrKy0NCj4gPiAgeGVuL2Fy
Y2gvYXJtL2VmaS9lZmktYm9vdC5oIHwgMjUgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiA+
ICB4ZW4vaW5jbHVkZS94ZW4vZWZpLmggICAgICAgfCAgNyArKysrKysrDQo+ID4gIDMgZmlsZXMg
Y2hhbmdlZCwgMTQgaW5zZXJ0aW9ucygrKSwgMjYgZGVsZXRpb25zKC0pDQo+ID4NCj4gPiBkaWZm
IC0tZ2l0IGEveGVuL2FyY2gvYXJtL2Jvb3RmZHQuYyBiL3hlbi9hcmNoL2FybS9ib290ZmR0LmMN
Cj4gPiBpbmRleCBhZmFhMGUyNDliLi42YmM1YTQ2NWVjIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9h
cmNoL2FybS9ib290ZmR0LmMNCj4gPiArKysgYi94ZW4vYXJjaC9hcm0vYm9vdGZkdC5jDQo+ID4g
QEAgLTExLDYgKzExLDcgQEANCj4gPiAgI2luY2x1ZGUgPHhlbi9saWIuaD4NCj4gPiAgI2luY2x1
ZGUgPHhlbi9rZXJuZWwuaD4NCj4gPiAgI2luY2x1ZGUgPHhlbi9pbml0Lmg+DQo+ID4gKyNpbmNs
dWRlIDx4ZW4vZWZpLmg+DQo+ID4gICNpbmNsdWRlIDx4ZW4vZGV2aWNlX3RyZWUuaD4NCj4gPiAg
I2luY2x1ZGUgPHhlbi9saWJmZHQvbGliZmR0Lmg+DQo+ID4gICNpbmNsdWRlIDx4ZW4vc29ydC5o
Pg0KPiA+IEBAIC0zNzAsNyArMzcxLDEyIEBAIHN0YXRpYyBpbnQgX19pbml0IGVhcmx5X3NjYW5f
bm9kZShjb25zdCB2b2lkICpmZHQsDQo+ID4gIHsNCj4gPiAgICAgIGludCByYyA9IDA7DQo+ID4N
Cj4gPiAtICAgIGlmICggZGV2aWNlX3RyZWVfbm9kZV9tYXRjaGVzKGZkdCwgbm9kZSwgIm1lbW9y
eSIpICkNCj4gPiArICAgIC8qDQo+ID4gKyAgICAgKiBJZiBYZW4gaGFzIGJlZW4gYm9vdGVkIHZp
YSBVRUZJLCB0aGUgbWVtb3J5IGJhbmtzIHdpbGwgYWxyZWFkeQ0KPiA+ICsgICAgICogYmUgcG9w
dWxhdGVkLiBTbyB3ZSBzaG91bGQgc2tpcCB0aGUgcGFyc2luZy4NCj4gPiArICAgICAqLw0KPiA+
ICsgICAgaWYgKCAhZWZpX2VuYWJsZWQoRUZJX0JPT1QpICYmDQo+ID4gKyAgICAgICAgIGRldmlj
ZV90cmVlX25vZGVfbWF0Y2hlcyhmZHQsIG5vZGUsICJtZW1vcnkiKSkNCj4gPiAgICAgICAgICBy
YyA9IHByb2Nlc3NfbWVtb3J5X25vZGUoZmR0LCBub2RlLCBuYW1lLCBkZXB0aCwNCj4gPiAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWRkcmVzc19jZWxscywgc2l6ZV9jZWxscywN
Cj4gJmJvb3RpbmZvLm1lbSk7DQo+ID4gICAgICBlbHNlIGlmICggZGVwdGggPT0gMSAmJiAhZHRf
bm9kZV9jbXAobmFtZSwgInJlc2VydmVkLW1lbW9yeSIpICkNCj4gPiBkaWZmIC0tZ2l0IGEveGVu
L2FyY2gvYXJtL2VmaS9lZmktYm9vdC5oIGIveGVuL2FyY2gvYXJtL2VmaS9lZmktYm9vdC5oDQo+
ID4gaW5kZXggY2Y5YzM3MTUzZi4uZDBhOTk4N2ZhNCAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vYXJj
aC9hcm0vZWZpL2VmaS1ib290LmgNCj4gPiArKysgYi94ZW4vYXJjaC9hcm0vZWZpL2VmaS1ib290
LmgNCj4gPiBAQCAtMTk3LDMzICsxOTcsOCBAQCBFRklfU1RBVFVTIF9faW5pdA0KPiBmZHRfYWRk
X3VlZmlfbm9kZXMoRUZJX1NZU1RFTV9UQUJMRSAqc3lzX3RhYmxlLA0KPiA+ICAgICAgaW50IHN0
YXR1czsNCj4gPiAgICAgIHUzMiBmZHRfdmFsMzI7DQo+ID4gICAgICB1NjQgZmR0X3ZhbDY0Ow0K
PiA+IC0gICAgaW50IHByZXY7DQo+ID4gICAgICBpbnQgbnVtX3JzdjsNCj4gPg0KPiA+IC0gICAg
LyoNCj4gPiAtICAgICAqIERlbGV0ZSBhbnkgbWVtb3J5IG5vZGVzIHByZXNlbnQuICBUaGUgRUZJ
IG1lbW9yeSBtYXAgaXMgdGhlIG9ubHkNCj4gPiAtICAgICAqIG1lbW9yeSBkZXNjcmlwdGlvbiBw
cm92aWRlZCB0byBYZW4uDQo+ID4gLSAgICAgKi8NCj4gPiAtICAgIHByZXYgPSAwOw0KPiA+IC0g
ICAgZm9yICg7OykNCj4gPiAtICAgIHsNCj4gPiAtICAgICAgICBjb25zdCBjaGFyICp0eXBlOw0K
PiA+IC0gICAgICAgIGludCBsZW47DQo+ID4gLQ0KPiA+IC0gICAgICAgIG5vZGUgPSBmZHRfbmV4
dF9ub2RlKGZkdCwgcHJldiwgTlVMTCk7DQo+ID4gLSAgICAgICAgaWYgKCBub2RlIDwgMCApDQo+
ID4gLSAgICAgICAgICAgIGJyZWFrOw0KPiA+IC0NCj4gPiAtICAgICAgICB0eXBlID0gZmR0X2dl
dHByb3AoZmR0LCBub2RlLCAiZGV2aWNlX3R5cGUiLCAmbGVuKTsNCj4gPiAtICAgICAgICBpZiAo
IHR5cGUgJiYgc3RybmNtcCh0eXBlLCAibWVtb3J5IiwgbGVuKSA9PSAwICkNCj4gPiAtICAgICAg
ICB7DQo+ID4gLSAgICAgICAgICAgIGZkdF9kZWxfbm9kZShmZHQsIG5vZGUpOw0KPiA+IC0gICAg
ICAgICAgICBjb250aW51ZTsNCj4gPiAtICAgICAgICB9DQo+ID4gLQ0KPiA+IC0gICAgICAgIHBy
ZXYgPSBub2RlOw0KPiA+IC0gICAgfQ0KPiA+IC0NCj4gPiAgICAgLyoNCj4gPiAgICAgICogRGVs
ZXRlIGFsbCBtZW1vcnkgcmVzZXJ2ZSBtYXAgZW50cmllcy4gV2hlbiBib290aW5nIHZpYSBVRUZJ
LA0KPiA+ICAgICAgKiBrZXJuZWwgd2lsbCB1c2UgdGhlIFVFRkkgbWVtb3J5IG1hcCB0byBmaW5k
IHJlc2VydmVkIHJlZ2lvbnMuDQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL3hlbi9lZmku
aCBiL3hlbi9pbmNsdWRlL3hlbi9lZmkuaA0KPiA+IGluZGV4IDY2MWE0ODI4NmEuLmI1MmE0Njc4
ZTkgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2luY2x1ZGUveGVuL2VmaS5oDQo+ID4gKysrIGIveGVu
L2luY2x1ZGUveGVuL2VmaS5oDQo+ID4gQEAgLTQ3LDYgKzQ3LDEzIEBAIGludCBlZmlfcnVudGlt
ZV9jYWxsKHN0cnVjdCB4ZW5wZl9lZmlfcnVudGltZV9jYWxsICopOw0KPiA+ICBpbnQgZWZpX2Nv
bXBhdF9nZXRfaW5mbyh1aW50MzJfdCBpZHgsIHVuaW9uIGNvbXBhdF9wZl9lZmlfaW5mbyAqKTsN
Cj4gPiAgaW50IGVmaV9jb21wYXRfcnVudGltZV9jYWxsKHN0cnVjdCBjb21wYXRfcGZfZWZpX3J1
bnRpbWVfY2FsbCAqKTsNCj4gPg0KPiA+ICsjZWxzZQ0KPiA+ICsNCj4gPiArc3RhdGljIGlubGlu
ZSBib29sIGVmaV9lbmFibGVkKHVuc2lnbmVkIGludCBmZWF0dXJlKQ0KPiA+ICt7DQo+ID4gKyAg
ICByZXR1cm4gZmFsc2U7DQo+ID4gK30NCj4gPiArDQo+ID4gICNlbmRpZiAvKiBDT05GSUdfRUZJ
Ki8NCj4gPg0KPiA+ICAjZW5kaWYgLyogIV9fQVNTRU1CTFlfXyAqLw0KPiA+IC0tDQo+ID4gMi4y
NS4xDQo+ID4NCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:42:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:42:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194892.347257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd2D-0000Le-E1; Fri, 24 Sep 2021 04:42:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194892.347257; Fri, 24 Sep 2021 04:42:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd2D-0000LX-9z; Fri, 24 Sep 2021 04:42:05 +0000
Received: by outflank-mailman (input) for mailman id 194892;
 Fri, 24 Sep 2021 04:42:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTd2B-0000LO-WF
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:42:04 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.78]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b3265748-3213-4fdb-a17f-71d97c1dc665;
 Fri, 24 Sep 2021 04:42:01 +0000 (UTC)
Received: from DU2PR04CA0318.eurprd04.prod.outlook.com (2603:10a6:10:2b5::23)
 by PAXPR08MB6480.eurprd08.prod.outlook.com (2603:10a6:102:155::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 04:41:52 +0000
Received: from DB5EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::1c) by DU2PR04CA0318.outlook.office365.com
 (2603:10a6:10:2b5::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16 via Frontend
 Transport; Fri, 24 Sep 2021 04:41:52 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT014.mail.protection.outlook.com (10.152.20.102) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:41:51 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Fri, 24 Sep 2021 04:41:51 +0000
Received: from 6e7a80343b7f.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 34509C8B-E875-4177-9924-3DA8DAC35113.1; 
 Fri, 24 Sep 2021 04:41:40 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6e7a80343b7f.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:41:40 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5622.eurprd08.prod.outlook.com (2603:10a6:10:1af::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:41:39 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:41:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3265748-3213-4fdb-a17f-71d97c1dc665
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZBlq6h6M1zHBD87kL+Y1HK06DiDaAcZlMN2lyMlHGZw=;
 b=K0+ugwzNSXstWTvSqXMfNZgTkZNQI5jnmg+T0XAnfLI9MiYRrQhRH5XBlimYtypHJGDAA5KC9mWKxZXpqTEayeFFnWVH6sREvz4aNNlQABOjklo2Dr3Oqa2/E+39Uj0/m/DnPYzcJnphOalq5VDDrc8VFTImZ9oCX25a6rGbO7o=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XxvUooFYYlhGKdCq4a5YJx2sWVNsRC/PbkQCF6bMWCiW5NMhuEjO/uFql00Z+FWTJHWQdoODuZw220DAMEmu+Wk0+jFD+2Ig3dKsM69xELeWt0/pyzRgOf6mV0+gWEB61Gk5+ET+JuZlYeeQn7D0+anagGWxF6p7zTn1MjQco9FGHgiwTkJ73Yi5TjHVVG06DpftiP1SzMF/sABBdUmXBW89xozp7tUcrsYTJmAkTfkR2U9+2f5vKdnaIktLi6OQcTSYP7f3ehp6tMp3uN0MpYh4/s3oB+G2nJyGX+1/VxmsDpMr97jfveedXp6/+xoTcOy74YqW+o1aHs/ljkKgaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZBlq6h6M1zHBD87kL+Y1HK06DiDaAcZlMN2lyMlHGZw=;
 b=J1WdAIG8vGVb6cPiRcDsi+ZNC8NC0EZ/LQMwiNxYrXgNtJwz/RTzr6LjTDtHtMBul14O0WbQNjnVbHEP1pf8pJPtvI9QYs7RkLL13BCWievy1t+Y9v5M51etrcePxELcIgzpWHeStdGQPzGUO9rlSB0kAGio3pbXFb+zbMxByuDwwdR9P+zQ3kNtZoYOK8rf0ThdJ1uSjJ3A6MMLQ0yZ9rC4Wbn37gQcRK70rzXDcRATaYMeIUAUlWaSaOjaQ9rXy9GN3/oJfhX1Eyp4K9PGRqm4JAUo08/5dPnIOHB43w/bvo14bLk0rzBrS4KMej3YzS5SeWZHIL9Z6gYC4z8TDQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZBlq6h6M1zHBD87kL+Y1HK06DiDaAcZlMN2lyMlHGZw=;
 b=K0+ugwzNSXstWTvSqXMfNZgTkZNQI5jnmg+T0XAnfLI9MiYRrQhRH5XBlimYtypHJGDAA5KC9mWKxZXpqTEayeFFnWVH6sREvz4aNNlQABOjklo2Dr3Oqa2/E+39Uj0/m/DnPYzcJnphOalq5VDDrc8VFTImZ9oCX25a6rGbO7o=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
Thread-Topic: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
Thread-Index: AQHXsHMu1eZPzfhk6EKVEGWM7eNLZ6uyayoAgAAvnmA=
Date: Fri, 24 Sep 2021 04:41:39 +0000
Message-ID:
 <DB9PR08MB68571BAA4C8B07BB2061C8579EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-24-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231843530.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231843530.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 25FDE39980836D4D8F82B971B36E8D2C.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: c8c4d86e-2ce6-4d44-b7e7-08d97f15a0dd
x-ms-traffictypediagnostic: DBAPR08MB5622:|PAXPR08MB6480:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6480ABF83D78E1C15631AA4E9EA49@PAXPR08MB6480.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 cuZVj02NP1nWRPECQqEIbq6r16CT7mNYXYTPof5yzaQNy9hgHHF/jiLWbqOkZMaU8O1u8Ts49UQUmvoPhxR3ef9peGAdlDwejQvQxddPJrktpiq4gXLdKO97wTUI7FfN+WxWL4bUbOfe9CkZoOgz2zwqQgz+bVto+EhcRWxw7OA02W4nHky0H57/9v8YOun/cevXdd1YWNodwAf0tquhNbzw5bVvuT5T9NvfqK0XGu/d7W/4DZ5Ou/HvVRDzif6WKcurwg8arU4ZE1YQ1/bS2p4B6ZdO9NQ99ROjcZyDfMLTf2PJ4Qcg2yBH3CqcuUGdXQChgRlj5318zmSR13v8gyk6Toh2ZEaPoReHkYK7RLN79s+1I2YK4whk6dAwlwnu+ZfflDr/t3hc6rIbU9Nrj+dt2h6Pzvrb0q3kmQw8LqPDYDxHdRhGQSDSrovGqX0wbA6pdz3rvOZ6NENbFpGGagp+Ggve+Z/Cp81PoW5uk9Cv+heXksl8jlPs53KM8eozfSoZa3Mf2QPYdFDzCrHXQnfsjnOFx6oGobCTEPDgGGKvr8HSBCIUCigOO8xqnkmzAkqjxZcqsHz1hgQH7G405bRJQL1zMA4P+BwOGGHly7zsooYz7N9TQUnXKQMQ/wPENRTpRXdOsek/tgJw1On60ihHH15o1LSryCg8kzfgGAn+gAMPh/k2EO4R4OwTNv1LKN08ZG1oXEOqgztG1Q3EjfUf0eBx6vyrtOykIj5YuScvbYqhxgfLviBpd5FPSseXjs4oNE5R+9BlGDjhIuVMOb2ABoRKY4qTvyUgRoCj7Glg3tX/YLCpZRspvjy05Keg
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(38100700002)(71200400001)(33656002)(6916009)(122000001)(54906003)(2906002)(316002)(4326008)(86362001)(5660300002)(38070700005)(6506007)(8936002)(55016002)(26005)(53546011)(7696005)(186003)(52536014)(8676002)(83380400001)(9686003)(76116006)(64756008)(66946007)(66476007)(66446008)(66556008)(2004002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5622
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ddc63845-7688-40a8-445f-08d97f159967
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ohGiDzwECEWnBOreC4e0/j1VdZQVI55uj/eEQlBE3j6ERDPLGO1Dpjejd1eOWS1o5jMDnVFcLxQbUx2vMFbqGXqDFUwF0IFFgPMxDhCKMm0ofHv3ySXcXWqDIjw4OWzJwAdbElvMhzWdnxt6K+mj7ZUC4gXouBnhWxiu2Mk4/bxhrSBWFpItoigPg0O2cZDcwPwlYFgcHAifgBmPgg3L44BX1yRBsuMaqqCcv6ZEarQtqej9tbfGDTyWPxF26lTYBNr1Ong8eQ08Pz0vBDDdpzixVoRNmDwRKyZsk4McdYMm6LsH82jTQlxE+Iaa7df1jfcn0UHjxcFSdW3eTSiofZKdTC4b6x5cZJNPWvutxMtyZpv21pZiOvCTn7NevXsdClb5yYMJH50wP9iyoF8/RGS8+CZdml/qRHluZ459Fdb7Sp+MH6P/yQbkNmyGYBrwV/tjiZXsNcRdshNK21jw3SvJsAkz2fIu6dYeG0TCl+qeF/s2b/BGONFJDldr9coGG0Wdf74HuAgRH91R7k+i9pqVOQ6rjPBdASYUKsqjwCR+ayOqkSUgUynuFqTawPJDQR5Vc59LDwekRDiqFe5+GamKtwWvwoBEep0F/N/oa/kkqYRA3WV2wJbJ6yHiNiDygyUPc0s6h3j3MivNf2TinAs7KsF/K6AFWKUlDOb4B/PAFr0ntNiznZXLWoxJpniVwFkmp6XnamwXwx9Gdm1ECQdbuDfpwco4bmxJcnApn/OgNjqbB+3sqeIq5VxBgEY7xdHYY9zaAdamrz59/NVcToHY54AHhB/pIJ883kcwuxLV3HwWI6GMSO/PLayHzzsV
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(83380400001)(47076005)(2906002)(81166007)(26005)(186003)(5660300002)(8936002)(4326008)(6506007)(33656002)(508600001)(53546011)(70586007)(336012)(8676002)(52536014)(82310400003)(70206006)(316002)(7696005)(6862004)(9686003)(55016002)(356005)(86362001)(36860700001)(54906003)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:41:51.9463
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c8c4d86e-2ce6-4d44-b7e7-08d97f15a0dd
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6480

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgOTo0Nw0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4t
ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGll
bkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjMvMzddIHhlbi9hcm06IGltcGxlbWVudCBub2RlIGRp
c3RhbmNlIGhlbHBlcnMgZm9yDQo+IEFybQ0KPiANCj4gT24gVGh1LCAyMyBTZXAgMjAyMSwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gV2Ugd2lsbCBwYXJzZSBOVU1BIG5vZGVzIGRpc3RhbmNlcyBmcm9t
IGRldmljZSB0cmVlIG9yIEFDUEkNCj4gPiB0YWJsZS4gU28gd2UgbmVlZCBhIG1hdHJpeCB0byBy
ZWNvcmQgdGhlIGRpc3RhbmNlcyBiZXR3ZWVuDQo+ID4gYW55IHR3byBub2RlcyB3ZSBwYXJzZWQu
IEFjY29yZGluZ2x5LCB3ZSBwcm92aWRlIHRoaXMNCj4gPiBub2RlX3NldF9kaXN0YW5jZSBBUEkg
Zm9yIGRldmljZSB0cmVlIG9yIEFDUEkgdGFibGUgcGFyc2Vycw0KPiA+IHRvIHNldCB0aGUgZGlz
dGFuY2UgZm9yIGFueSB0d28gbm9kZXMgaW4gdGhpcyBwYXRjaC4NCj4gPiBXaGVuIE5VTUEgaW5p
dGlhbGl6YXRpb24gZmFpbGVkLCBfX25vZGVfZGlzdGFuY2Ugd2lsbCByZXR1cm4NCj4gPiBOVU1B
X1JFTU9URV9ESVNUQU5DRSwgdGhpcyB3aWxsIGhlbHAgdXMgYXZvaWQgZG9pbmcgcm9sbGJhY2sN
Cj4gPiBmb3IgZGlzdGFuY2UgbWF4dHJpeCB3aGVuIE5VTUEgaW5pdGlhbGl6YXRpb24gZmFpbGVk
Lg0KPiA+DQo+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+
ID4gLS0tDQo+ID4gIHhlbi9hcmNoL2FybS9NYWtlZmlsZSAgICAgIHwgIDEgKw0KPiA+ICB4ZW4v
YXJjaC9hcm0vbnVtYS5jICAgICAgICB8IDY5ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrDQo+ID4gIHhlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIHwgMTMgKysrKysrKw0K
PiA+ICAzIGZpbGVzIGNoYW5nZWQsIDgzIGluc2VydGlvbnMoKykNCj4gPiAgY3JlYXRlIG1vZGUg
MTAwNjQ0IHhlbi9hcmNoL2FybS9udW1hLmMNCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJj
aC9hcm0vTWFrZWZpbGUgYi94ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4gPiBpbmRleCBhZTRlZmJm
NzZlLi40MWNhMzExYjZiIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9hcmNoL2FybS9NYWtlZmlsZQ0K
PiA+ICsrKyBiL3hlbi9hcmNoL2FybS9NYWtlZmlsZQ0KPiA+IEBAIC0zNSw2ICszNSw3IEBAIG9i
ai0kKENPTkZJR19MSVZFUEFUQ0gpICs9IGxpdmVwYXRjaC5vDQo+ID4gIG9iai15ICs9IG1lbV9h
Y2Nlc3Mubw0KPiA+ICBvYmoteSArPSBtbS5vDQo+ID4gIG9iai15ICs9IG1vbml0b3Iubw0KPiA+
ICtvYmotJChDT05GSUdfTlVNQSkgKz0gbnVtYS5vDQo+ID4gIG9iai15ICs9IHAybS5vDQo+ID4g
IG9iai15ICs9IHBlcmNwdS5vDQo+ID4gIG9iai15ICs9IHBsYXRmb3JtLm8NCj4gPiBkaWZmIC0t
Z2l0IGEveGVuL2FyY2gvYXJtL251bWEuYyBiL3hlbi9hcmNoL2FybS9udW1hLmMNCj4gPiBuZXcg
ZmlsZSBtb2RlIDEwMDY0NA0KPiA+IGluZGV4IDAwMDAwMDAwMDAuLjNmMDg4NzBkNjkNCj4gPiAt
LS0gL2Rldi9udWxsDQo+ID4gKysrIGIveGVuL2FyY2gvYXJtL251bWEuYw0KPiA+IEBAIC0wLDAg
KzEsNjkgQEANCj4gPiArLy8gU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjANCj4gPiAr
LyoNCj4gPiArICogQXJtIEFyY2hpdGVjdHVyZSBzdXBwb3J0IGxheWVyIGZvciBOVU1BLg0KPiA+
ICsgKg0KPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMjEgQXJtIEx0ZA0KPiA+ICsgKg0KPiA+ICsg
KiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgaXQg
YW5kL29yIG1vZGlmeQ0KPiA+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5l
cmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24gMiBhcw0KPiA+ICsgKiBwdWJsaXNoZWQgYnkgdGhl
IEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbi4NCj4gPiArICoNCj4gPiArICogVGhpcyBwcm9ncmFt
IGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+ID4g
KyAqIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0aG91dCBldmVuIHRoZSBpbXBsaWVkIHdh
cnJhbnR5IG9mDQo+ID4gKyAqIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBBIFBBUlRJ
Q1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUNCj4gPiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vu
c2UgZm9yIG1vcmUgZGV0YWlscy4NCj4gPiArICoNCj4gPiArICogWW91IHNob3VsZCBoYXZlIHJl
Y2VpdmVkIGEgY29weSBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UNCj4gPiArICog
YWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uIElmIG5vdCwgc2VlIDxodHRwOi8vd3d3LmdudS5vcmcv
bGljZW5zZXMvPi4NCj4gPiArICoNCj4gPiArICovDQo+ID4gKyNpbmNsdWRlIDx4ZW4vaW5pdC5o
Pg0KPiA+ICsjaW5jbHVkZSA8eGVuL251bWEuaD4NCj4gPiArDQo+ID4gK3N0YXRpYyB1aW50OF90
IF9fcmVhZF9tb3N0bHkNCj4gPiArbm9kZV9kaXN0YW5jZV9tYXBbTUFYX05VTU5PREVTXVtNQVhf
TlVNTk9ERVNdID0gew0KPiA+ICsgICAgeyAwIH0NCj4gPiArfTsNCj4gPiArDQo+ID4gK3ZvaWQg
X19pbml0IG51bWFfc2V0X2Rpc3RhbmNlKG5vZGVpZF90IGZyb20sIG5vZGVpZF90IHRvLCB1aW50
MzJfdA0KPiBkaXN0YW5jZSkNCj4gPiArew0KPiA+ICsgICAgaWYgKCBmcm9tID49IE1BWF9OVU1O
T0RFUyB8fCB0byA+PSBNQVhfTlVNTk9ERVMgKQ0KPiA+ICsgICAgew0KPiA+ICsgICAgICAgIHBy
aW50ayhLRVJOX1dBUk5JTkcNCj4gPiArICAgICAgICAgICAgICAgIk5VTUE6IGludmFsaWQgbm9k
ZXM6IGZyb209JSJQUkl1OCIgdG89JSJQUkl1OCINCj4gTUFYPSUiUFJJdTgiXG4iLA0KPiA+ICsg
ICAgICAgICAgICAgICBmcm9tLCB0bywgTUFYX05VTU5PREVTKTsNCj4gPiArICAgICAgICByZXR1
cm47DQo+ID4gKyAgICB9DQo+ID4gKw0KPiA+ICsgICAgLyogTlVNQSBkZWZpbmVzIDB4ZmYgYXMg
YW4gdW5yZWFjaGFibGUgbm9kZSBhbmQgMC05IGFyZSB1bmRlZmluZWQNCj4gKi8NCj4gPiArICAg
IGlmICggZGlzdGFuY2UgPj0gTlVNQV9OT19ESVNUQU5DRSB8fA0KPiA+ICsgICAgICAgIChkaXN0
YW5jZSA+PSBOVU1BX0RJU1RBTkNFX1VERl9NSU4gJiYNCj4gPiArICAgICAgICAgZGlzdGFuY2Ug
PD0gTlVNQV9ESVNUQU5DRV9VREZfTUFYKSB8fA0KPiA+ICsgICAgICAgIChmcm9tID09IHRvICYm
IGRpc3RhbmNlICE9IE5VTUFfTE9DQUxfRElTVEFOQ0UpICkNCj4gPiArICAgIHsNCj4gPiArICAg
ICAgICBwcmludGsoS0VSTl9XQVJOSU5HDQo+ID4gKyAgICAgICAgICAgICAgICJOVU1BOiBpbnZh
bGlkIGRpc3RhbmNlOiBmcm9tPSUiUFJJdTgiIHRvPSUiUFJJdTgiDQo+IGRpc3RhbmNlPSUiUFJJ
dTMyIlxuIiwNCj4gPiArICAgICAgICAgICAgICAgZnJvbSwgdG8sIGRpc3RhbmNlKTsNCj4gPiAr
ICAgICAgICByZXR1cm47DQo+ID4gKyAgICB9DQo+ID4gKw0KPiA+ICsgICAgbm9kZV9kaXN0YW5j
ZV9tYXBbZnJvbV1bdG9dID0gZGlzdGFuY2U7DQo+ID4gK30NCj4gPiArDQo+ID4gK3VpbnQ4X3Qg
X19ub2RlX2Rpc3RhbmNlKG5vZGVpZF90IGZyb20sIG5vZGVpZF90IHRvKQ0KPiA+ICt7DQo+ID4g
KyAgICAvKiBXaGVuIE5VTUEgaXMgb2ZmLCBhbnkgZGlzdGFuY2Ugd2lsbCBiZSB0cmVhdGVkIGFz
IHJlbW90ZS4gKi8NCj4gPiArICAgIGlmICggc3JhdF9kaXNhYmxlZCgpICkNCj4gDQo+IEdpdmVu
IHRoYXQgdGhpcyBpcyBBUk0gc3BlY2lmaWMgY29kZSBhbmQgc3BlY2lmaWMgdG8gQUNQSSwgSSBk
b24ndCB0aGluaw0KPiB3ZSBzaG91bGQgaGF2ZSBhbnkgY2FsbCB0byBzb21ldGhpbmcgY2FsbGVk
ICJzcmF0X2Rpc2FibGVkIi4NCj4gDQo+IEkgc3VnZ2VzdCB0byBlaXRoZXIgcmVuYW1lIHNyYXRf
ZGlzYWJsZWQgdG8gbnVtYV9kaXN0YW5jZV9kaXNhYmxlZC4NCj4gDQo+IE90aGVyIHRoYW4gdGhh
dCwgdGhpcyBwYXRjaCBsb29rcyBPSyB0byBtZS4NCj4gDQoNCnNyYXQgc3RhbmRzIGZvciBzdGF0
aWMgcmVzb3VyY2UgYWZmaW5pdHkgdGFibGUsIEkgdGhpbmsgZHRiIGFsc28gY2FuIGJlDQp0cmVh
dGVkIGFzIGEgc3RhdGljIHJlc291cmNlIGFmZmluaXR5IHRhYmxlLiBTbyBJIGtlZXAgU1JBVCBp
biB0aGlzIHBhdGNoDQphbmQgb3RoZXIgcGF0Y2hlcy4gSSBoYXZlIHNlZW4geW91ciBjb21tZW50
IGluIHBhdGNoIzI1LiBCZWZvcmUgeDg2IG1haW50YWluZXJzDQpnaXZlIGFueSBmZWVkYmFjaywg
Y2FuIHdlIHN0aWxsIGtlZXAgc3JhdCBoZXJlPw0KDQo+IA0KPiA+ICsgICAgICAgIHJldHVybiBO
VU1BX1JFTU9URV9ESVNUQU5DRTsNCj4gPiArDQo+ID4gKyAgICAvKg0KPiA+ICsgICAgICogQ2hl
Y2sgd2hldGhlciB0aGUgbm9kZXMgYXJlIGluIHRoZSBtYXRyaXggcmFuZ2UuDQo+ID4gKyAgICAg
KiBXaGVuIGFueSBub2RlIGlzIG91dCBvZiByYW5nZSwgZXhjZXB0IGZyb20gYW5kIHRvIG5vZGVz
IGFyZSB0aGUNCj4gPiArICAgICAqIHNhbWUsIHdlIHRyZWF0IHRoZW0gYXMgdW5yZWFjaGFibGUg
KHJldHVybiAweEZGKQ0KPiA+ICsgICAgICovDQo+ID4gKyAgICBpZiAoIGZyb20gPj0gTUFYX05V
TU5PREVTIHx8IHRvID49IE1BWF9OVU1OT0RFUyApDQo+ID4gKyAgICAgICAgcmV0dXJuIGZyb20g
PT0gdG8gPyBOVU1BX0xPQ0FMX0RJU1RBTkNFIDogTlVNQV9OT19ESVNUQU5DRTsNCj4gPiArDQo+
ID4gKyAgICByZXR1cm4gbm9kZV9kaXN0YW5jZV9tYXBbZnJvbV1bdG9dOw0KPiA+ICt9DQo+ID4g
K0VYUE9SVF9TWU1CT0woX19ub2RlX2Rpc3RhbmNlKTsNCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2lu
Y2x1ZGUvYXNtLWFybS9udW1hLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+IGlu
ZGV4IDIxNTY5ZTYzNGIuLjc1OGVhZmViMDUgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2luY2x1ZGUv
YXNtLWFybS9udW1hLmgNCj4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+
IEBAIC05LDggKzksMjEgQEAgdHlwZWRlZiB1OCBub2RlaWRfdDsNCj4gPg0KPiA+ICAjaWZkZWYg
Q09ORklHX05VTUENCj4gPg0KPiA+ICsvKg0KPiA+ICsgKiBJbiBBQ1BJIHNwZWMsIDAtOSBhcmUg
dGhlIHJlc2VydmVkIHZhbHVlcyBmb3Igbm9kZSBkaXN0YW5jZSwNCj4gPiArICogMTAgaW5kaWNh
dGVzIGxvY2FsIG5vZGUgZGlzdGFuY2UsIDIwIGluZGljYXRlcyByZW1vdGUgbm9kZQ0KPiA+ICsg
KiBkaXN0YW5jZS4gU2V0IG5vZGUgZGlzdGFuY2UgbWFwIGluIGRldmljZSB0cmVlIHdpbGwgZm9s
bG93DQo+ID4gKyAqIHRoZSBBQ1BJJ3MgZGVmaW5pdGlvbi4NCj4gPiArICovDQo+ID4gKyNkZWZp
bmUgTlVNQV9ESVNUQU5DRV9VREZfTUlOICAgMA0KPiA+ICsjZGVmaW5lIE5VTUFfRElTVEFOQ0Vf
VURGX01BWCAgIDkNCj4gPiArI2RlZmluZSBOVU1BX0xPQ0FMX0RJU1RBTkNFICAgICAxMA0KPiA+
ICsjZGVmaW5lIE5VTUFfUkVNT1RFX0RJU1RBTkNFICAgIDIwDQo+ID4gKw0KPiA+ICAjZGVmaW5l
IE5SX05PREVfTUVNQkxLUyBOUl9NRU1fQkFOS1MNCj4gPg0KPiA+ICtleHRlcm4gdm9pZCBudW1h
X3NldF9kaXN0YW5jZShub2RlaWRfdCBmcm9tLCBub2RlaWRfdCB0bywgdWludDMyX3QNCj4gZGlz
dGFuY2UpOw0KPiA+ICsNCj4gPiAgI2Vsc2UNCj4gPg0KPiA+ICAvKiBGYWtlIG9uZSBub2RlIGZv
ciBub3cuIFNlZSBhbHNvIG5vZGVfb25saW5lX21hcC4gKi8NCj4gPiAtLQ0KPiA+IDIuMjUuMQ0K
PiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:43:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:43:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194899.347268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd3B-0000yV-T0; Fri, 24 Sep 2021 04:43:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194899.347268; Fri, 24 Sep 2021 04:43:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd3B-0000yO-OZ; Fri, 24 Sep 2021 04:43:05 +0000
Received: by outflank-mailman (input) for mailman id 194899;
 Fri, 24 Sep 2021 04:43:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTd3A-0000yA-PE
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:43:04 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe02::602])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9e6246c1-dab2-47f6-8b44-e04de3ba071c;
 Fri, 24 Sep 2021 04:43:03 +0000 (UTC)
Received: from DB7PR02CA0020.eurprd02.prod.outlook.com (2603:10a6:10:52::33)
 by AM6PR08MB3445.eurprd08.prod.outlook.com (2603:10a6:20b:43::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 24 Sep
 2021 04:43:01 +0000
Received: from DB5EUR03FT041.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:52:cafe::1b) by DB7PR02CA0020.outlook.office365.com
 (2603:10a6:10:52::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 04:43:01 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT041.mail.protection.outlook.com (10.152.21.4) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:43:01 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Fri, 24 Sep 2021 04:43:00 +0000
Received: from b18ee790f46c.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E81CFE4F-46D0-41EB-887A-9F7C157F981F.1; 
 Fri, 24 Sep 2021 04:42:50 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b18ee790f46c.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:42:50 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5622.eurprd08.prod.outlook.com (2603:10a6:10:1af::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:42:49 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:42:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e6246c1-dab2-47f6-8b44-e04de3ba071c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=00S8jcIrVFBN1uvTM84xghtlh7RjwXB42v2XnvIPF7c=;
 b=sSYyRMYy2rgj/viXEr6D5ecS7CgsZT49QaE/fl94zdjSIufpEa5NuObXt2im6Z3mkWwbEOHak66WS9L5iibJWvlykwiYOubWLGXgBjWl+1zNsylLNWp9i7mc7zNzrfp4dDfyDHV+rA8BUCy5ZTNydYrjtHR+QrG4lPShIWkneBk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iAjnapZeGk/eal9QCSPtSol+JN/cpq+xf4emWUKkSJs0eyhjkFG7er8EOkGxQqklQ3/YoURdCKVq4lOPVAANatOL7et9Qnz2Jeunj43T2W3OD4RPo9UID2ejg4H53Uwkq1eHLkiF1v/dd0VY8UzppI/DRQxSCyUgpyaFrsScHHk8aXSzOtCREhgFJg6H75h1mdYnhOU8vx1e2/0/iuCMuKb8RQZ6yfWd7P2UuVTk5MT5PTZWIlUYAcunpoF0OapOW5LoJZbpJqLX0+4K2GcIhom8Uq4CB8oIbIPYa7dmybftRuT5svsH0j55eD3kKDpJV4qn/SPl1tNaD73d0z/lig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=00S8jcIrVFBN1uvTM84xghtlh7RjwXB42v2XnvIPF7c=;
 b=F7QI23EDhSjlKUSspSI3qJGVxzzbJWSITbXr5Pmjr3B6aurbVGCwqQSvB9/MrZc+r8kjkPeYIppO0ga/jJWDRoPD6PuZjPQyWxAfOIKoNfDZ0zN+Nx1UhozH4pxh/Y8sBVZ74aX/V1Mm4j3ps8CBnQUr86qQAY8VEdX2ku7bpWChYVBg2vy/PNOSj7oacaMb7wBVhsCpy37NkBtKKQK9u/PXUNSB9rQwCPY80bZNPLjGvPbE9wAwxKkjLV0Yd2NXLNsycAIjRd/yxptHTBBj1CMqCuEUKT5+FQd84uHUe3HS3ZT0bEjSvXkRVeRa2KgRrQ+EoKFmceP6SDEcKF1vag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=00S8jcIrVFBN1uvTM84xghtlh7RjwXB42v2XnvIPF7c=;
 b=sSYyRMYy2rgj/viXEr6D5ecS7CgsZT49QaE/fl94zdjSIufpEa5NuObXt2im6Z3mkWwbEOHak66WS9L5iibJWvlykwiYOubWLGXgBjWl+1zNsylLNWp9i7mc7zNzrfp4dDfyDHV+rA8BUCy5ZTNydYrjtHR+QrG4lPShIWkneBk=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 24/37] xen/arm: implement two arch helpers to get memory
 map info
Thread-Topic: [PATCH 24/37] xen/arm: implement two arch helpers to get memory
 map info
Thread-Index: AQHXsHM4IX7m67p9UE2HNfweaKhzBauycKqAgAArlfA=
Date: Fri, 24 Sep 2021 04:42:48 +0000
Message-ID:
 <DB9PR08MB68578A23A896A06B788EC5929EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-25-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231905180.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231905180.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6B792F6B4C29E443A934A2AFFB43AB3A.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4e8291dd-9121-440e-06a4-08d97f15ca18
x-ms-traffictypediagnostic: DBAPR08MB5622:|AM6PR08MB3445:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB344557BE16A67180DA5F0DE19EA49@AM6PR08MB3445.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YXS6r6jU5rzz3sAG/zyu/xwFTezm9k/MciRh3lW9zrRoPV2/IdoaNDec7Rowl3YiXo+P6D/brmLSnWZjyx/sfZdErRVIsXer+PXM2AcWRJDZA/uNg4AP4ldBfltKt4jzSzRtAf0KA1NXrnlFFZL0cloFBI0cJuAw+LnUHcvW53HWN7+62zu7fJK8JZnEZWSawf46HH8A6GbhX3E0MzWVoaC4Q3Pd1373NikB0zC9b0Aod6gaNlLGcC061jnc/HG01rBmYDA6HdkLmHwgxnwnWnz6ivpJrnssx44ZltxaBYimzSe3n2Mjtud62glUZtiG/6z5o/Hwaw+aVU3w7uy+CVCEs78ZK98Y3Lz9tais/D4FGHb2Nhyr1Y6LbiSHL0Jz+8jTPqnCkZ4KmDTtaMnlpup3oAojDrsQJ0Em2rAgulgh+imBLE3gu2cUkdOuQyXjEyfrkbpqFj3+VGPYKmqGbtmXC69nhRYvushbZLM6TVJAWRNAkV2r7EXhjvkOMWgOKhTo7SbdpMBelNl9mQbB7PZZv6atRjoF7YmNBf+Dnz3LgEiDt+8ZPsdh5gZxnFWk4iPcMVXHmjmUdSz13rqGUI1gEDom/v/nKNrat0elpHKFxrlW/tb8esw+G0HQUGCFyxttH3L3QNhlshs5k1dy66tlj9b+wMFei+Zt017PZSxVwSKcSqn4pUXw78ECrNT26woujilGwyd5IFi95PxxDw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(38100700002)(71200400001)(33656002)(6916009)(122000001)(54906003)(2906002)(316002)(4326008)(86362001)(5660300002)(38070700005)(6506007)(8936002)(55016002)(26005)(53546011)(7696005)(186003)(52536014)(8676002)(83380400001)(9686003)(76116006)(64756008)(66946007)(66476007)(66446008)(66556008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5622
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bf7cb980-1b4d-43d0-3d13-08d97f15c2e0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9/YBvCU6oKHuMS8r213u2wFioeAEXtqkjz1JmjVFHjzOSjBJyQWqT43H7VG4wa7Mws2yhHDAvHVrVLuSbaXQ3ccKEYttXt85IVg9JPaPZ4NOT7XmakneLvAwX2ZrvDGYax32Am2JspnzK4ltszmPsUkK7+rD8aF0MT8ohY+2XKVvozxo3qGvOHY5kWAHyuRA22pN360K8labJN8pAj+b6O1tnasl07MPS1XBpLS6igX4Z9LeMofk5LeWIddUeEuZUsVUXda9JBKFYErpHMDsNn00meItMMQLQ2h661omwPhI1XJv+iX7sIKfcD649NNsghBGbDgNJRDKtDGiQhu00dbF7rpTbDd/GBwXKQrqbxzJWE9SpXFeMAfD1m9LbmEJFOhZeFh/+QLjbHvLNElPEofcPdjocAzfPkwuoch4tS75Vt7Lp306vsYAqN5B4qK32WrO3pEj11R65+cVcTvLyvMjAkchPDyCQwfk3Tf6Crub5iRJfpFSIfCw+kpfJRpO5kj2xSrnq2xFlkeIUvrQvlETPAAuKsGt/WVtuMnASwH5OKNzRtz+u3LLHFN2g57m7XVlgIndQUMtu1/jg7aak/J6BgVeNswxWBvDZ1WAZm5IYTDkZP5F0U4JXDLvQ/ndaXbuwL0b6m3U5BoJV27XBZVlrIQzfNZPKat4JC/lV4Bn6f5544H+4K3V8H80oGS6nuQwws3zDhZb8cGcuNYhYw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(508600001)(52536014)(356005)(86362001)(5660300002)(82310400003)(81166007)(4326008)(70206006)(336012)(33656002)(7696005)(47076005)(53546011)(2906002)(55016002)(8676002)(6862004)(316002)(54906003)(8936002)(6506007)(9686003)(70586007)(36860700001)(186003)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:43:01.1168
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e8291dd-9121-440e-06a4-08d97f15ca18
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3445

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDEwOjA2
DQo+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0
cy54ZW5wcm9qZWN0Lm9yZzsgc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVuQHhlbi5vcmc7
DQo+IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gU3ViamVj
dDogUmU6IFtQQVRDSCAyNC8zN10geGVuL2FybTogaW1wbGVtZW50IHR3byBhcmNoIGhlbHBlcnMg
dG8gZ2V0DQo+IG1lbW9yeSBtYXAgaW5mbw0KPiANCj4gT24gVGh1LCAyMyBTZXAgMjAyMSwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gVGhlc2UgdHdvIGhlbHBlcnMgYXJlIGFyY2hpdGVjdHVyZSBBUElz
IHRoYXQgYXJlIHJlcXVpcmVkIGJ5DQo+ID4gbm9kZXNfY292ZXJfbWVtb3J5Lg0KPiA+DQo+ID4g
U2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4gLS0tDQo+ID4g
IHhlbi9hcmNoL2FybS9udW1hLmMgfCAxNCArKysrKysrKysrKysrKw0KPiA+ICAxIGZpbGUgY2hh
bmdlZCwgMTQgaW5zZXJ0aW9ucygrKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2Fy
bS9udW1hLmMgYi94ZW4vYXJjaC9hcm0vbnVtYS5jDQo+ID4gaW5kZXggM2YwODg3MGQ2OS4uMzc1
NWIwMWVmNCAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vYXJjaC9hcm0vbnVtYS5jDQo+ID4gKysrIGIv
eGVuL2FyY2gvYXJtL251bWEuYw0KPiA+IEBAIC02NywzICs2NywxNyBAQCB1aW50OF90IF9fbm9k
ZV9kaXN0YW5jZShub2RlaWRfdCBmcm9tLCBub2RlaWRfdCB0bykNCj4gPiAgICAgIHJldHVybiBu
b2RlX2Rpc3RhbmNlX21hcFtmcm9tXVt0b107DQo+ID4gIH0NCj4gPiAgRVhQT1JUX1NZTUJPTChf
X25vZGVfZGlzdGFuY2UpOw0KPiA+ICsNCj4gPiArdWludDMyX3QgX19pbml0IGFyY2hfbWVtaW5m
b19nZXRfbnJfYmFuayh2b2lkKQ0KPiA+ICt7DQo+ID4gKwlyZXR1cm4gYm9vdGluZm8ubWVtLm5y
X2JhbmtzOw0KPiA+ICt9DQo+ID4gKw0KPiA+ICtpbnQgX19pbml0IGFyY2hfbWVtaW5mb19nZXRf
cmFtX2JhbmtfcmFuZ2UodWludDMyX3QgYmFuaywNCj4gPiArCXBhZGRyX3QgKnN0YXJ0LCBwYWRk
cl90ICplbmQpDQo+ID4gK3sNCj4gPiArCSpzdGFydCA9IGJvb3RpbmZvLm1lbS5iYW5rW2Jhbmtd
LnN0YXJ0Ow0KPiA+ICsJKmVuZCA9IGJvb3RpbmZvLm1lbS5iYW5rW2JhbmtdLnN0YXJ0ICsgYm9v
dGluZm8ubWVtLmJhbmtbYmFua10uc2l6ZTsNCj4gPiArDQo+ID4gKwlyZXR1cm4gMDsNCj4gPiAr
fQ0KPiANCj4gVGhlIHJlc3Qgb2YgdGhlIGZpbGUgaXMgaW5kZW50ZWQgdXNpbmcgc3BhY2VzLCB3
aGlsZSB0aGlzIHBhdGNoIGlzIHVzaW5nDQo+IHRhYnMuDQo+IA0KPiBBbHNvLCBnaXZlbiB0aGUg
aW1wbGVtZW50YXRpb24sIGl0IGxvb2tzIGxpa2UNCj4gYXJjaF9tZW1pbmZvX2dldF9yYW1fYmFu
a19yYW5nZSBzaG91bGQgZWl0aGVyIHJldHVybiB2b2lkIG9yIGJvb2wuDQoNCkkgd2lsbCBmaXgg
dGhlbSBpbiBuZXh0IHZlcnNpb24uDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:45:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:45:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194904.347279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd5b-0001dx-Ad; Fri, 24 Sep 2021 04:45:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194904.347279; Fri, 24 Sep 2021 04:45:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd5b-0001dq-6u; Fri, 24 Sep 2021 04:45:35 +0000
Received: by outflank-mailman (input) for mailman id 194904;
 Fri, 24 Sep 2021 04:45:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTd5Z-0001dk-Mi
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:45:33 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe06::613])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4341efa4-38af-4279-96b5-7aa00c0e67c1;
 Fri, 24 Sep 2021 04:45:32 +0000 (UTC)
Received: from DB6P192CA0011.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::21) by
 PAXPR08MB7202.eurprd08.prod.outlook.com (2603:10a6:102:209::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Fri, 24 Sep
 2021 04:45:30 +0000
Received: from DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:b8:cafe::bc) by DB6P192CA0011.outlook.office365.com
 (2603:10a6:4:b8::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Fri, 24 Sep 2021 04:45:30 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT044.mail.protection.outlook.com (10.152.21.167) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:45:30 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Fri, 24 Sep 2021 04:45:29 +0000
Received: from 4c11f6bd24ce.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5FB78F6D-2F6B-4B97-BF8B-8C6BFC6E2691.1; 
 Fri, 24 Sep 2021 04:45:18 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4c11f6bd24ce.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:45:18 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5622.eurprd08.prod.outlook.com (2603:10a6:10:1af::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:45:16 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:45:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4341efa4-38af-4279-96b5-7aa00c0e67c1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g8T94Mbto6pvFVI8qCVmcdrb3oV9mjt0Zfo4bQrqJgE=;
 b=CWkwGlYsOE+bO9xLddx/IUpjdvC9ct+jSZ3Ml9rvYd8JfUtyOijkypeDozWddwCsx6SE0Ha8vJvQ7KFE65ERm7JChVvR1z2RCk0iOu+/o3EmkK+CACRRU1x/2rd2OchDWANelEICvWPD6YRYWr0yAmfQ3o9u7Y62392gMssH5XM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k6+i9oUP8O8Jju3+e6qee7qyVDvex42xByVQRRQK2PIN/YZddk4pyZtv9kY3STJuA/QfOoBfEBokPwzdDPTMXku7s02mv7QqmA25JEQCZuVIT67w8amWE1TCx8cZx19xZp63leEQZlGuPJ+4UNEAU96MmogfPNSEm55JTRE6KwakTNXh1oG/Ys/jbMoY9p8XK6SYDahDuOH2qzssaI7YtS5S4jI8bVXXNB8MezcsqFVFcmwe5EcDW/CYRRujSzll1iAjM/ArGZEu7I7tFjuL1Wroz5Y4WurC5QixVgpPmgsqG5yhaAnR0q9Agkfqk4AOsvrf0EnC2pJrE+oG+Qbm3Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g8T94Mbto6pvFVI8qCVmcdrb3oV9mjt0Zfo4bQrqJgE=;
 b=aFXmYD2WqXbcJwqBAHcecCCF7KBK6J6upwUn1yFwC2mCVLlaUU8OV/lLpwyEq3Tih17Abx57YJxBpjqCkpqx5X5uegxuZHarRfW7lRVYqVeM0m7FOZjEMSU6XZ7uI5eA5V5fneuFYfs0WeqMOOMJaOaGvPO4QIztjqPIk+MuyUBl2KAnD2AAwsFEOYCKmyXi0I0OvP0JKK6pGnEQIVb3gYlEnrrVYSZAxUgUf4g72NTo8vNQlS04Jj/4SITb57aTpgLy+BlkMR3OzH2OLmchz478WLGNAHCrIQzR7l98ZC2ZWydNtz7W/AQ/2/LxqLt+p5Hp7reNDd+bK8u9JKPPHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g8T94Mbto6pvFVI8qCVmcdrb3oV9mjt0Zfo4bQrqJgE=;
 b=CWkwGlYsOE+bO9xLddx/IUpjdvC9ct+jSZ3Ml9rvYd8JfUtyOijkypeDozWddwCsx6SE0Ha8vJvQ7KFE65ERm7JChVvR1z2RCk0iOu+/o3EmkK+CACRRU1x/2rd2OchDWANelEICvWPD6YRYWr0yAmfQ3o9u7Y62392gMssH5XM=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA
 initialization
Thread-Topic: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA
 initialization
Thread-Index: AQHXsHMq3VjOt9gtW0yi3uRIsAbNq6uycYQAgAAq5HA=
Date: Fri, 24 Sep 2021 04:45:16 +0000
Message-ID:
 <DB9PR08MB68570E6EA33BF7C6E9E23CBD9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-26-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 5FFABA7B4929D349990F917B7E6D4545.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: db282b1d-e18e-4496-f539-08d97f1622ea
x-ms-traffictypediagnostic: DBAPR08MB5622:|PAXPR08MB7202:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB7202671E0288E84D512266279EA49@PAXPR08MB7202.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 NKHx34q+Laa25z0FSM1R169E0y1tnL2r+paMiUDzCdcEw9PxWw5V1X0ZXyC1J4VLk3O3KKiE1MWnGsKTHiwwr2xqXJji5Q/N12E/9hGfOUBjpSHGpUdvOqDmC6auDZQ9YXH/wq0aR4S76+c/8Cj2DNqzHx8ZnQBOIQZqqP2qzFYwqUydrVVHgCDpEMlG8V9KB9HjUzUvgi4nBJgF0vA5sY5/jV4HTbPfMJOmO5MwyFFDMA5/MkIe0j7E+R1sOy+6lq7mlGlG9a1oTjY+y9nWJcHg/POO9eJflHkfvJJasm3vPBUT7ovPTSEveTJKeIMZ7wtdw7jkCH2WF9ylUWQPvs5xUIbsUsrkmUrvQPa5chBwEOGn8syJxYzcsSijqQbVD2s62H5tsgG4yAXU+YsPFIDpZiP9AMDRaIzXwM4yyQX7fsef+p4qMui+pV+jrIz36ns89SYIp7CYRqsngfusp0UiboWZKWRz4iwZxQrjEBvChpZ3YYqEoWx1ASfbDC2f1HmVx9L8jInMY/ci0LBVh/tu6LAHSSerVy5uFyZL906gg7IqcnJh0nRNhtgeLnSLXYNeCBZAsGxLHgl11Q+x39zZC+tmU5ercZSajGXO5nSsT+9JPTN2HLRqXeh39UHt8B1Jt2FssfJODEo/vZFfYs0MBr552Ft44rRyJ4UdME3+aLkIFH08ncB/AA0BNmuK2Z6rAbQAD89iqTk17i2BKA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(38100700002)(71200400001)(33656002)(6916009)(122000001)(54906003)(2906002)(316002)(4326008)(86362001)(5660300002)(38070700005)(6506007)(8936002)(55016002)(26005)(53546011)(7696005)(186003)(52536014)(8676002)(83380400001)(9686003)(76116006)(64756008)(66946007)(66476007)(66446008)(66556008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5622
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6fabb5a0-9473-4d53-f97b-08d97f161ae2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LKDR3FuE6uhC7YjXZhLZWA0P55mCkbgvxuYGXejS2UfpdrRaaHSBa9qQPksf3lYdH3WdZDkukntJli3YyhGlp3uux/WjTQeJ1RPWSxElEXdePHUewISIZ6jdkY+OMypcIzeoTlb0CSksAq8ti5lfKSS/6bUyH1wPQnVK6gRSjim8m74tDe1vt+5G0cO1QFaQoe5io4HEhowpoz5myIn2BsuD+W7UjAsiyJre3h9Ziry7XQHDErwhOJRHkDzwvMZ6zmCqbOhodoZz5Tozj1UbxvVrqBJDU71arWAIqdAyD8F+MKUOJcPHWGJLgurNAFHE0pyUPeEnhJzIQpRTG/uV39g6BX7DNNbnXu2VlP7mdWhnh/UzEDykGfd9eFHy9N4zPrJI0d52ieRVDk2cHVK5bx4xBCIhqPRezXbbNdAAdApXQFEvN1DC0PMGLMGy6C5TZmYHKNojogzBxnVMjNq0XJm4jCOrhUa3ATmdlWVwmfwSWHuhdNERxa8qYB0+I93h/enjoCvWbsngDAzyYh04ophCmk0t9PpJ0MaBFLMGUniV3OFymdP0iICqoMzRMaZk/LSBAmv277gNqYlsxzPH2v47TR5lGj7tfkuPLgkDAXNK8T7Fl4kfyw3Xo2HLchO4hDQXT3xkGoG757rgMIHbv4YMsxD94BLsKyIlEpYldSR52ffHUh8J4YoDyrYTRFYBWHvE7QbGCoGSyukrMa63zw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6862004)(316002)(82310400003)(54906003)(8936002)(7696005)(47076005)(81166007)(336012)(4326008)(9686003)(508600001)(8676002)(36860700001)(26005)(53546011)(55016002)(6506007)(186003)(70206006)(52536014)(33656002)(70586007)(83380400001)(356005)(2906002)(86362001)(5660300002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:45:30.1397
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: db282b1d-e18e-4496-f539-08d97f1622ea
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7202

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTA6MTANCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
PjsgamJldWxpY2hAc3VzZS5jb207DQo+IGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb207IHJvZ2Vy
LnBhdUBjaXRyaXguY29tOyB3bEB4ZW4ub3JnDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjUvMzdd
IHhlbi9hcm06IGltcGxlbWVudCBiYWRfc3JhdCBmb3IgQXJtIE5VTUENCj4gaW5pdGlhbGl6YXRp
b24NCj4gDQo+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IE5VTUEg
aW5pdGlhbGl6YXRpb24gd2lsbCBwYXJzZSBpbmZvcm1hdGlvbiBmcm9tIGZpcm13YXJlIHByb3Zp
ZGVkDQo+ID4gc3RhdGljIHJlc291cmNlIGFmZmluaXR5IHRhYmxlIChBQ1BJIFNSQVQgb3IgRFRC
KS4gYmFkX3NyYXQgaWYgYQ0KPiA+IGZ1bmN0aW9uIHRoYXQgd2lsbCBiZSB1c2VkIHdoZW4gaW5p
dGlhbGl6YXRpb24gY29kZSBlbmNvdW50ZXJzDQo+ID4gc29tZSB1bmV4Y2VwdGVkIGVycm9ycy4N
Cj4gPg0KPiA+IEluIHRoaXMgcGF0Y2gsIHdlIGludHJvZHVjZSBBcm0gdmVyc2lvbiBiYWRfc3Jh
dCBmb3IgTlVNQSBjb21tb24NCj4gPiBpbml0aWFsaXphdGlvbiBjb2RlIHRvIGludm9rZSBpdC4N
Cj4gPg0KPiA+IFNpZ25lZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29tPg0KPiA+
IC0tLQ0KPiA+ICB4ZW4vYXJjaC9hcm0vbnVtYS5jIHwgNyArKysrKysrDQo+ID4gIDEgZmlsZSBj
aGFuZ2VkLCA3IGluc2VydGlvbnMoKykNCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9h
cm0vbnVtYS5jIGIveGVuL2FyY2gvYXJtL251bWEuYw0KPiA+IGluZGV4IDM3NTViMDFlZjQuLjUy
MDlkM2RlNGQgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gvYXJtL251bWEuYw0KPiA+ICsrKyBi
L3hlbi9hcmNoL2FybS9udW1hLmMNCj4gPiBAQCAtMTgsNiArMTgsNyBAQA0KPiA+ICAgKg0KPiA+
ICAgKi8NCj4gPiAgI2luY2x1ZGUgPHhlbi9pbml0Lmg+DQo+ID4gKyNpbmNsdWRlIDx4ZW4vbm9k
ZW1hc2suaD4NCj4gPiAgI2luY2x1ZGUgPHhlbi9udW1hLmg+DQo+ID4NCj4gPiAgc3RhdGljIHVp
bnQ4X3QgX19yZWFkX21vc3RseQ0KPiA+IEBAIC0yNSw2ICsyNiwxMiBAQCBub2RlX2Rpc3RhbmNl
X21hcFtNQVhfTlVNTk9ERVNdW01BWF9OVU1OT0RFU10gPSB7DQo+ID4gICAgICB7IDAgfQ0KPiA+
ICB9Ow0KPiA+DQo+ID4gK19faW5pdCB2b2lkIGJhZF9zcmF0KHZvaWQpDQo+ID4gK3sNCj4gPiAr
ICAgIHByaW50ayhLRVJOX0VSUiAiTlVNQTogRmlybXdhcmUgU1JBVCB0YWJsZSBub3QgdXNlZC5c
biIpOw0KPiA+ICsgICAgZndfbnVtYSA9IC0xOw0KPiA+ICt9DQo+IA0KPiBJIHJlYWxpemUgdGhh
dCB0aGUgc2VyaWVzIGtlZXBzIHRoZSAic3JhdCIgdGVybWlub2xvZ3kgZXZlcnl3aGVyZSBvbiBE
VA0KPiB0b28uIEkgd29uZGVyIGlmIGl0IGlzIHdvcnRoIHJlcGxhY2luZyBzcmF0IHdpdGggc29t
ZXRoaW5nIGxpa2UNCj4gIm51bWFfZGlzdGFuY2UiIGV2ZXJ5d2hlcmUgYXMgYXBwcm9wcmlhdGUu
IEkgYW0gYWRkaW5nIHRoZSB4ODYNCj4gbWFpbnRhaW5lcnMgZm9yIGFuIG9waW5pb24uDQo+IA0K
PiBJZiB5b3UgZ3V5cyBwcmVmZXIgdG8ga2VlcCBzcmF0IChpZiBub3RoaW5nIGVsc2UsIGl0IGlz
IGNvbmNpc2UpLCBJIGFtDQo+IGFsc28gT0sgd2l0aCBrZWVwaW5nIHNyYXQgYWx0aG91Z2ggaXQg
aXMgbm90IHRlY2huaWNhbGx5IGFjY3VyYXRlLg0KDQpJIGhhdmUgcGxhY2VkIHNvbWUgY29tbWVu
dHMgaW4gcGF0Y2gjMjMgZm9yIHNyYXQuDQpJIHdvdWxkbid0IGxpa2UgdG8gcmVwbGFjZSBzcmF0
IGJ5IG51bWFfZGlzdGFuY2UuIGJlY2F1c2Ugc3JhdCBub3Qgb25seQ0KY29udGFpbnMgZGlzdGFu
Y2UsIGFuZCBpdCBhbHNvIGluY2x1ZGVzIHNvbWUgYWZmaW5pdHkgaW5mb3JtYXRpb24gZm9yIENQ
VQ0Kb3Igb3RoZXIgZGV2aWNlcy4NCg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 04:46:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 04:46:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194909.347289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd6Q-0002Cz-JD; Fri, 24 Sep 2021 04:46:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194909.347289; Fri, 24 Sep 2021 04:46:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTd6Q-0002Cs-GJ; Fri, 24 Sep 2021 04:46:26 +0000
Received: by outflank-mailman (input) for mailman id 194909;
 Fri, 24 Sep 2021 04:46:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTd6P-0002Ci-EP
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 04:46:25 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.74]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5de2cc84-1cf2-11ec-baa8-12813bfff9fa;
 Fri, 24 Sep 2021 04:46:24 +0000 (UTC)
Received: from AM6PR10CA0003.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:89::16)
 by AM0PR08MB5025.eurprd08.prod.outlook.com (2603:10a6:208:15e::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Fri, 24 Sep
 2021 04:46:22 +0000
Received: from AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:89:cafe::63) by AM6PR10CA0003.outlook.office365.com
 (2603:10a6:209:89::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 04:46:22 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT056.mail.protection.outlook.com (10.152.17.224) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 04:46:21 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Fri, 24 Sep 2021 04:46:20 +0000
Received: from ed3464f4dcd4.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A6075550-3637-4EED-8A6F-1B3EB3CA1173.1; 
 Fri, 24 Sep 2021 04:46:09 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ed3464f4dcd4.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 04:46:09 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6505.eurprd08.prod.outlook.com (2603:10a6:10:23e::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 04:46:07 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 04:46:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5de2cc84-1cf2-11ec-baa8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HayqveibyC0u7zRGugoPlW+XlwdFa78evUZyH0bKnaI=;
 b=WTNfbb5JPGmQCdqf+Xt2zST7Co/yfg+v0KNXTvHEi8cn+ewBO24hkjwWhgiUgA/VU21lhAvrtJSQoGykjnCCQZdYnfpoxq1QWv9hFScf4P9a9r2nxGD4RcpKah3g8cORm1yaiwhSNVB/N72iT2qowgaf5ohkQfAP7mJf/7SOJEA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N+xgJtQ+PlLC4BvFNjbnRB3g62fpsZoOEF7wML7BBEQRyY20Ufqbn55CmbK5JPAk4/LdexiwWQULwWzEadFAIYiiaRq1yxFhi1h7R5WZYFnOT1y9BG+JzFz0M/7KBkp3QvafK5mxkutK4/DUtuB29JW1kLh4lAqOOdJWtkTap9ky7LpQb3hcd8clS+f1AF6V4qHOfWvvlzuQTnsrK/4LtkxQDY9d7egiDyww6xriV8QJigfOahM4W5LXtbEH88xfKpmPy/mG+5HZrjhWM42gyeeKUc4Fe7HvHOW/VnQDNBLFK13U2pg2RuFTPc2aHyUpzw/sVu/DchNk3ZWmJmprnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HayqveibyC0u7zRGugoPlW+XlwdFa78evUZyH0bKnaI=;
 b=IfIs68GX5bkZ1NmSehJ68o8N4Qx/1sb570xo2KePaykFKkDFXs6id8dCSIlHBsSMHaUfua1cOzz9FyZ/mGZWHsNMgDZRI1GyEvWBj1CJH36WN8Rw+zJr4I+UL7tyIn7IBIpwVdSFb5SS9rUA/VhZm+9MY8V+cVvWRSzELuU5Uaalqz9OpSe2DGQgDxLD0yHXzhyET1fATQO0xuv3b2Cq/tRFsU414eO8aRwzP9G1SfisF+z/v15JmGnDq5POY3ngYDIxzOAEYdemfN8r4tv4l+/OGu0bKCIOX90C4esdHmfm76GKhI3IVdsjdNSDTsV/+dQ18HpWXJJ8z3kmoTcPYA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HayqveibyC0u7zRGugoPlW+XlwdFa78evUZyH0bKnaI=;
 b=WTNfbb5JPGmQCdqf+Xt2zST7Co/yfg+v0KNXTvHEi8cn+ewBO24hkjwWhgiUgA/VU21lhAvrtJSQoGykjnCCQZdYnfpoxq1QWv9hFScf4P9a9r2nxGD4RcpKah3g8cORm1yaiwhSNVB/N72iT2qowgaf5ohkQfAP7mJf/7SOJEA=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 29/37] xen/arm: introduce a helper to parse device tree
 processor node
Thread-Topic: [PATCH 29/37] xen/arm: introduce a helper to parse device tree
 processor node
Thread-Index: AQHXsHMy8BVdIW2w9kGY+s/rVXmgtKuye2oAgAAhuZA=
Date: Fri, 24 Sep 2021 04:46:07 +0000
Message-ID:
 <DB9PR08MB6857B805811A0A010C4706B89EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-30-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231940110.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231940110.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 5552BC78E3C6B64C98A2D794C2EE5CE7.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 20478977-87bd-4d9a-9af0-08d97f1641d6
x-ms-traffictypediagnostic: DB9PR08MB6505:|AM0PR08MB5025:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB50257AF5F50F0B6FF02BF66A9EA49@AM0PR08MB5025.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 mDBlIQQPHQTDpjWsalKvOny82wmtLvjhpk0uioD13qxBjUWS7d+WZTzQ3vJa4VLFOjuM/gsD/QeoDlGKzkfFwCOmbRSn/VMzqlSi9ytFKaP+j0Ku0/lGWLpHhvV2Z1NelRZplc5DwC+isStKMh7oXsZmwnj2P7uwfNypgxXEfpbXmj98pqryf2zlLNvoRaAXsB1mVudXV1NiLBDL0/RYFmtArjDVnlOywRngtqqsmgVBhMRGQTEsmM6Xo8UEyOomqt55/ShKsJD7jBe8+TuJDXcUrvgRm7O+1wibGyl5sXuS9611bGUxcM3rewN2d6e8IL5vtuf4wlIg/vvAxef2ZYRzit8iskjQv4Zm0gXV5Mt323iVoOpF2cXu0ZyzB+zC3pEajg2e1o6i4bpT1GOO3+OV233Zxu7gtit6+C/pMtUKH96or5nceWwOhcdfb3R9eOhS3lAhCQA4iA/KzoFkyBWxZZV8bq9OvMKEAZ+o/QdM4wP/ucmuneGh7UzEOqaSBqZvgwpQjNTIOObPPr7vMF9p8eJiZHkuKqzT2hAvVvjwoI+B8/ZJZeLr5FxknHi3tX6N8tnlPwuA18c1PFamTZ316HWO9dXeGziJxTTfY1b3Z5FsrRw8i62/sze4V4/TtJ5kUOgDc6HqXLk/xCzTcqh3NWsYjz4R3qoAn/j2QMK+w2uFTrKu3cm6hQW0IsC3asqRErlJu+pIkC5Jr0BKji+YGzBDCRGPNaNb9266FClYVlZAAZLucjjBlWhOeUg3sqLlAVmEhWXo/y5JUVzjnjmGtICiiofekZyIE/gzNDLQy3nC6/iZ+Z6IvQ1SxtrR
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52536014)(8936002)(122000001)(33656002)(5660300002)(83380400001)(316002)(38100700002)(4326008)(9686003)(8676002)(55016002)(66446008)(64756008)(54906003)(66556008)(76116006)(6506007)(2906002)(66476007)(66946007)(186003)(53546011)(86362001)(71200400001)(26005)(508600001)(38070700005)(6916009)(7696005)(2004002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6505
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d6f0f171-03ff-46c4-2745-08d97f16392d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tA884DfCAEYwI+MKsuCrkGrdqx0sF51Z6Z0Llg3flrtyjTS/VsVVtSLN//kT+B3HvQNJDgvsySNyJeTZnFaAPa1xOGx/5by2qZk4GMx24dh6SbjFM6hSgDp6FkreutpVE9Cm9cQGcWzTHt7Y78QeGTZoZCTi82CZMoapppN3GAPRwd55eFUXHjYdkyYUcLOTPLZ/W2X0xUapJaraEFAu9hV+aV5cW6miWGiRar/OQKm8DgPWvKlRXuRgaqYR9kMEIAyDQsMttUzj0/VDGGUyttAqRjzyf91I86je3E87hk1xioPuXHxpIBoTe0jQ9rs6etIDn5c+TAze/cf227T6oG/3N3t3JfUhxx5VhRQKd4PAET+XLQ+QezNPUxeJFbJRpp1Y866xeTCd6zq+x3P2kvmfxA8qgs+aOTZ44YmALir8zP+hqEe4O34aOxui6L5hqqpy/41WPttbJy24Em+Z+2hOSSs7mMGnAdkZrmi5rA2VVXW+GwnE2BhO7xToz97xkgidv6YiUFYQlc+ShHA8v1vjsYMonXtdPuD5XlXTA2P9DzteHO3zQ8jQNumzXCY6pPUvA/7RA9DjQ+xF0wc4+pIe//cMUaoopC35ZbnajV5Z19pP7pT1jU8POWXejPsQc7xphK4G0ZH2M8PSGHoRYRvep8R6IGbE0O4gKGAWvAm5dK/LVFJkIEq6Df3Rp7HHN1ahV1y6CoMy1EjqXNUVij6Ww2OsGi1BmmXYwB6XTR/5em2nil3Fze8CbmG/gKT/sEbXSvtED6mB0pXqrtakHS89QtADS5WUZwOSDrmYROXlHofh/NBRMuJxTOtLDjbY
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(82310400003)(316002)(70206006)(83380400001)(6862004)(86362001)(26005)(186003)(55016002)(8676002)(356005)(8936002)(33656002)(70586007)(47076005)(53546011)(52536014)(9686003)(81166007)(36860700001)(54906003)(6506007)(336012)(7696005)(4326008)(508600001)(2906002)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 04:46:21.9657
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 20478977-87bd-4d9a-9af0-08d97f1641d6
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5025

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDogMjAyMcTqOdTCMjTI1SAxMDo0NQ0K
PiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4tZGV2ZWxAbGlzdHMu
eGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGllbkB4ZW4ub3JnOw0K
PiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+IFN1YmplY3Q6
IFJlOiBbUEFUQ0ggMjkvMzddIHhlbi9hcm06IGludHJvZHVjZSBhIGhlbHBlciB0byBwYXJzZSBk
ZXZpY2UNCj4gdHJlZSBwcm9jZXNzb3Igbm9kZQ0KPiANCj4gT24gVGh1LCAyMyBTZXAgMjAyMSwg
V2VpIENoZW4gd3JvdGU6DQo+ID4gUHJvY2Vzc29yIE5VTUEgSUQgaW5mb3JtYXRpb24gaXMgc3Rv
cmVkIGluIGRldmljZSB0cmVlJ3MgcHJvY2Vzc29yDQo+ID4gbm9kZSBhcyAibnVtYS1ub2RlLWlk
Ii4gV2UgbmVlZCBhIG5ldyBoZWxwZXIgdG8gcGFyc2UgdGhpcyBJRCBmcm9tDQo+ID4gcHJvY2Vz
c29yIG5vZGUuIElmIHdlIGdldCB0aGlzIElEIGZyb20gcHJvY2Vzc29yIG5vZGUsIHRoaXMgSUQn
cw0KPiA+IHZhbGlkaXR5IHN0aWxsIG5lZWQgdG8gYmUgY2hlY2tlZC4gT25jZSB3ZSBnb3QgYSBp
bnZhbGlkIE5VTUEgSUQNCj4gPiBmcm9tIGFueSBwcm9jZXNzb3Igbm9kZSwgdGhlIGRldmljZSB0
cmVlIHdpbGwgYmUgbWFya2VkIGFzIE5VTUENCj4gPiBpbmZvcm1hdGlvbiBpbnZhbGlkLg0KPiA+
DQo+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4gLS0t
DQo+ID4gIHhlbi9hcmNoL2FybS9NYWtlZmlsZSAgICAgICAgICAgfCAgMSArDQo+ID4gIHhlbi9h
cmNoL2FybS9udW1hX2RldmljZV90cmVlLmMgfCA1OCArKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysNCj4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA1OSBpbnNlcnRpb25zKCspDQo+ID4gIGNy
ZWF0ZSBtb2RlIDEwMDY0NCB4ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+ID4NCj4g
PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL01ha2VmaWxlIGIveGVuL2FyY2gvYXJtL01ha2Vm
aWxlDQo+ID4gaW5kZXggNDFjYTMxMWI2Yi4uYzUwZGYyYzI1ZCAxMDA2NDQNCj4gPiAtLS0gYS94
ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4gPiArKysgYi94ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4g
PiBAQCAtMzYsNiArMzYsNyBAQCBvYmoteSArPSBtZW1fYWNjZXNzLm8NCj4gPiAgb2JqLXkgKz0g
bW0ubw0KPiA+ICBvYmoteSArPSBtb25pdG9yLm8NCj4gPiAgb2JqLSQoQ09ORklHX05VTUEpICs9
IG51bWEubw0KPiA+ICtvYmotJChDT05GSUdfREVWSUNFX1RSRUVfTlVNQSkgKz0gbnVtYV9kZXZp
Y2VfdHJlZS5vDQo+ID4gIG9iai15ICs9IHAybS5vDQo+ID4gIG9iai15ICs9IHBlcmNwdS5vDQo+
ID4gIG9iai15ICs9IHBsYXRmb3JtLm8NCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL251
bWFfZGV2aWNlX3RyZWUuYw0KPiBiL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4g
PiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPiA+IGluZGV4IDAwMDAwMDAwMDAuLjI0MjhmYmFlMGIN
Cj4gPiAtLS0gL2Rldi9udWxsDQo+ID4gKysrIGIveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3Ry
ZWUuYw0KPiA+IEBAIC0wLDAgKzEsNTggQEANCj4gPiArLy8gU1BEWC1MaWNlbnNlLUlkZW50aWZp
ZXI6IEdQTC0yLjANCj4gPiArLyoNCj4gPiArICogQXJtIEFyY2hpdGVjdHVyZSBzdXBwb3J0IGxh
eWVyIGZvciBOVU1BLg0KPiA+ICsgKg0KPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMjEgQXJtIEx0
ZA0KPiA+ICsgKg0KPiA+ICsgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNh
biByZWRpc3RyaWJ1dGUgaXQgYW5kL29yIG1vZGlmeQ0KPiA+ICsgKiBpdCB1bmRlciB0aGUgdGVy
bXMgb2YgdGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24gMiBhcw0KPiA+ICsg
KiBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbi4NCj4gPiArICoNCj4g
PiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRoYXQgaXQgd2ls
bCBiZSB1c2VmdWwsDQo+ID4gKyAqIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsgd2l0aG91dCBl
dmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9mDQo+ID4gKyAqIE1FUkNIQU5UQUJJTElUWSBvciBG
SVRORVNTIEZPUiBBIFBBUlRJQ1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUNCj4gPiArICogR05VIEdl
bmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCj4gPiArICoNCj4gPiArICog
WW91IHNob3VsZCBoYXZlIHJlY2VpdmVkIGEgY29weSBvZiB0aGUgR05VIEdlbmVyYWwgUHVibGlj
IExpY2Vuc2UNCj4gPiArICogYWxvbmcgd2l0aCB0aGlzIHByb2dyYW0uIElmIG5vdCwgc2VlIDxo
dHRwOi8vd3d3LmdudS5vcmcvbGljZW5zZXMvPi4NCj4gPiArICoNCj4gPiArICovDQo+ID4gKyNp
bmNsdWRlIDx4ZW4vaW5pdC5oPg0KPiA+ICsjaW5jbHVkZSA8eGVuL25vZGVtYXNrLmg+DQo+ID4g
KyNpbmNsdWRlIDx4ZW4vbnVtYS5oPg0KPiA+ICsjaW5jbHVkZSA8eGVuL2xpYmZkdC9saWJmZHQu
aD4NCj4gPiArI2luY2x1ZGUgPHhlbi9kZXZpY2VfdHJlZS5oPg0KPiA+ICsNCj4gPiArLyogQ2Fs
bGJhY2sgZm9yIGRldmljZSB0cmVlIHByb2Nlc3NvciBhZmZpbml0eSAqLw0KPiA+ICtzdGF0aWMg
aW50IF9faW5pdCBmZHRfbnVtYV9wcm9jZXNzb3JfYWZmaW5pdHlfaW5pdChub2RlaWRfdCBub2Rl
KQ0KPiA+ICt7DQo+ID4gKyAgICBpZiAoIHNyYXRfZGlzYWJsZWQoKSApDQo+ID4gKyAgICAgICAg
cmV0dXJuIC1FSU5WQUw7DQo+IA0KPiBmZHRfbnVtYV9wcm9jZXNzb3JfYWZmaW5pdHlfaW5pdCBp
cyBjYWxsZWQgYnkgZmR0X3BhcnNlX251bWFfY3B1X25vZGUNCj4gd2hpY2ggaXMgYWxyZWFkeSBw
YXJzaW5nIE5VTUEgcmVsYXRlZCBpbmZvLiBTaG91bGQgdGhpcyBzcmF0X2Rpc2FibGVkDQo+IGNo
ZWNrIGJlIG1vdmVkIHRvIGZkdF9wYXJzZV9udW1hX2NwdV9ub2RlPw0KPiANCg0KQWgsIHllcywg
aXQncyBhIGdvb2Qgc3VnZ2VzdGlvbiwgSSB3aWxsIGFkZHJlc3MgaXQgaW4gbmV4dCB2ZXJzaW9u
Lg0KDQo+IA0KPiA+ICsgICAgZWxzZSBpZiAoIG5vZGUgPT0gTlVNQV9OT19OT0RFIHx8IG5vZGUg
Pj0gTUFYX05VTU5PREVTICkNCj4gPiArICAgIHsNCj4gPiArICAgICAgICBiYWRfc3JhdCgpOw0K
PiA+ICsgICAgICAgIHJldHVybiAtRUlOVkFMOw0KPiA+ICsJfQ0KPiA+ICsNCj4gPiArICAgIG51
bWFfc2V0X3Byb2Nlc3Nvcl9ub2Rlc19wYXJzZWQobm9kZSk7DQo+ID4gKyAgICBmd19udW1hID0g
MTsNCj4gPiArDQo+ID4gKyAgICBwcmludGsoS0VSTl9JTkZPICJEVDogTlVNQSBub2RlICUiUFJJ
dTciIHByb2Nlc3NvciBwYXJzZWRcbiIsIG5vZGUpOw0KPiA+ICsNCj4gPiArICAgIHJldHVybiAw
Ow0KPiA+ICt9DQo+ID4gKw0KPiA+ICsvKiBQYXJzZSBDUFUgTlVNQSBub2RlIGluZm8gKi8NCj4g
PiArc3RhdGljIGludCBfX2luaXQgZmR0X3BhcnNlX251bWFfY3B1X25vZGUoY29uc3Qgdm9pZCAq
ZmR0LCBpbnQgbm9kZSkNCj4gPiArew0KPiA+ICsgICAgdWludDMyX3QgbmlkOw0KPiA+ICsNCj4g
PiArICAgIG5pZCA9IGRldmljZV90cmVlX2dldF91MzIoZmR0LCBub2RlLCAibnVtYS1ub2RlLWlk
IiwgTUFYX05VTU5PREVTKTsNCj4gPiArICAgIGlmICggbmlkID49IE1BWF9OVU1OT0RFUyApDQo+
ID4gKyAgICB7DQo+ID4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIk5vZGUgaWQgJXUgZXhj
ZWVkcyBtYXhpbXVtIHZhbHVlXG4iLCBuaWQpOw0KPiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIF4gUFJJdTMyDQo+IA0KPiANCj4gPiArICAgICAgICByZXR1cm4gLUVJTlZB
TDsNCj4gPiArICAgIH0NCj4gPiArDQo+ID4gKyAgICByZXR1cm4gZmR0X251bWFfcHJvY2Vzc29y
X2FmZmluaXR5X2luaXQobmlkKTsNCj4gPiArfQ0KPiA+IC0tDQo+ID4gMi4yNS4xDQo+ID4NCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 05:24:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 05:24:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194917.347300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTdgu-00086S-J8; Fri, 24 Sep 2021 05:24:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194917.347300; Fri, 24 Sep 2021 05:24:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTdgu-00086L-GA; Fri, 24 Sep 2021 05:24:08 +0000
Received: by outflank-mailman (input) for mailman id 194917;
 Fri, 24 Sep 2021 05:24:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTdgs-00086F-VM
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 05:24:07 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.51]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f9efbe5d-f110-4d99-92f8-9545335aab91;
 Fri, 24 Sep 2021 05:24:04 +0000 (UTC)
Received: from AS8P250CA0003.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:330::8)
 by DB8PR08MB5018.eurprd08.prod.outlook.com (2603:10a6:10:e2::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Fri, 24 Sep
 2021 05:23:59 +0000
Received: from VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:330:cafe::6a) by AS8P250CA0003.outlook.office365.com
 (2603:10a6:20b:330::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 05:23:59 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT020.mail.protection.outlook.com (10.152.18.242) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 05:23:59 +0000
Received: ("Tessian outbound b324a1dbd9be:v103");
 Fri, 24 Sep 2021 05:23:57 +0000
Received: from d7225c28f6e2.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 68100AED-80F2-4B76-8C58-05D7F8FADA34.1; 
 Fri, 24 Sep 2021 05:23:46 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d7225c28f6e2.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 05:23:46 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB6060.eurprd08.prod.outlook.com (2603:10a6:10:20a::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 05:23:45 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 05:23:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9efbe5d-f110-4d99-92f8-9545335aab91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oSd0/RzzIp9LN58ByGpKpfys2qCamPVwSVxllXFsM3M=;
 b=17BRHa24PzzPBrJ/rH86FzkXiTeJNhgsWfSG0SExP2Uo1yVfKrvOgBVwSs29yfAFCnSQNR8ta29aZdqE29zbqiT90zAXgKuO3Nixu2p+OwIhv87XmKdXyIpw+quWZ8lyIdKvsL+ivgfZ3f5eQK+CUBXW+mJMmUGVjAJDJIHV5hw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LQo9ExXWJdHidBiwBhGIEajZtFD+aN6QROBWWAC+AhlQlCOJUikQoRqoux8bvkD1qzE/jvbJeGj05uJ3glOYNkvxwCXJ3V7my8e2WdpjtmMhhklGshwei3EltpaIuYt9X5c/pXLISeMp2s3fyxm4Jet+/plt15bo8Yik+utBkPKJ15kTz+TgKBdJRVkz6IbR5RNcrsm5y3bvwFKtikiJxrsJLUJFFluDtLk71BFdBe0u/v3NfgJjXBbtFxkCngebv0wEaieYcanNsJiCTBG7Fwfhl1GuDKDfdDHQAFwQ3je8rFb/c3AJ4whuUrzs1vS9ufrOH4mMlnEncZ950bavOA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oSd0/RzzIp9LN58ByGpKpfys2qCamPVwSVxllXFsM3M=;
 b=hIyW+GkQWCyJDDm1G7kMJR5HWRR2ySq8jUibG+A49MUJAd8wZHDyp9eJYIZ5Umlb/2DRH4c5UN3eoa60eTM13Ar7Nq/42/0SNsuVpYEKtXofnVnDq/ZF9vwwzg5hNPDwE2FPSMpwqHUIoM9sxHI49im3ZPNht3DzpRTC6xR63BnTfLYhgCRkK79FDEBJIwOIYxiAwFAQy/7tOiUDohcDo49cVC0G9XfOdLELXxNyEI3i0h4m+FkI8PZoOU6mN7z7KbqEp2l5BoHU9fZkPbqKR+MnH1DyO4albw3DSlIJeTr0yTwsVcHHyzdMQwIBqYUXXhk0ldFS6bUSSlxpmG6emQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oSd0/RzzIp9LN58ByGpKpfys2qCamPVwSVxllXFsM3M=;
 b=17BRHa24PzzPBrJ/rH86FzkXiTeJNhgsWfSG0SExP2Uo1yVfKrvOgBVwSs29yfAFCnSQNR8ta29aZdqE29zbqiT90zAXgKuO3Nixu2p+OwIhv87XmKdXyIpw+quWZ8lyIdKvsL+ivgfZ3f5eQK+CUBXW+mJMmUGVjAJDJIHV5hw=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 31/37] xen/arm: introduce a helper to parse device tree
 NUMA distance map
Thread-Topic: [PATCH 31/37] xen/arm: introduce a helper to parse device tree
 NUMA distance map
Thread-Index: AQHXsHM2Uv3zsz8Z7USP26PICjaLN6uygQuAgAAi0qA=
Date: Fri, 24 Sep 2021 05:23:45 +0000
Message-ID:
 <DB9PR08MB6857DE08F7A14D887CD6DBC99EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-32-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231958250.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231958250.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: E6777E74B2606742881527366C1376D5.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 61230c63-2902-4ba8-4ec1-08d97f1b8340
x-ms-traffictypediagnostic: DBBPR08MB6060:|DB8PR08MB5018:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB5018DA36D42CE0D4E275A8F69EA49@DB8PR08MB5018.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 DnahBi0biaiuLDEHUy8RJ9oiJcEkBFuQNl4v2egAnRCtdZmoO0TpKF/+m6XYsnpgTcjcGHsgpV8GGfMpxS83/3JBtVieN1BbjC/FmUiLRDn/Et/+kGSSYA24kLFPv85a8NQ64zoHTmfTUed/YmQJ98PGYVuYO01X0SEODRcEG1XN42Kb5vx6qubKD5Iq+USJFcm8lBvLmZ/9fAqagXBAROHN5HXce049pnKlnMqBIugRBMaYNl5rqT9+2yiH0rEsGkL76ipyQkNbhMvzNnmLdV+wpnLntQTzgesgtP/pXE+7WdmW83NIxcbb+daOkERxGQWpL7u0jRekM+6j8eW2h9RREn6cj7KLmDfnFIHUIBJ2kILBjFQYWjo1OvgH3GjxHtfxmv6mc1Lv3aoBdPePdYTQrLElzxfDMV1kNmCnn43furJjEp0mI8r7xPhq/uVD0TERsTxPDwcOolEgC1MY9646JbhvuylPF0FAidD8LFqKhRhSwMCkO30dr4/v2eGUR1mMRH+ZAPEGLb6o4vT13Tilg92xmImb90Ws+ZlmzTHud3RpGYSESA+ijdNqpRUIbBEKiBA/4TMcn2GT3WG3YNpVA4H5NvIYS+huyK2aLCOIloKrx6ZpYzbnUs8GW2vj0PadQ6owrFs1BSLKoQa1ykm13Ifaw2+ULQB7NScR+Nwa/YNwY4+2v+diaVUBwNG59gTdHLP80XAFcioysRGq/6wF9fmAc29Z0TgtcUNr1yD/A7Z0VeB9JI29/rHEwxHCjbIViS1L7wycxqJgwf1MWh11JB3E84xj5vA/Q4t6r/Q=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(52536014)(53546011)(508600001)(7696005)(33656002)(54906003)(8676002)(6916009)(186003)(38100700002)(122000001)(38070700005)(76116006)(6506007)(966005)(4326008)(83380400001)(2906002)(64756008)(316002)(66556008)(71200400001)(66446008)(8936002)(66476007)(55016002)(5660300002)(66946007)(86362001)(9686003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6060
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f21e6947-088c-4f7e-5c6a-08d97f1b7ae0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	deYq4XvPI+5giYDb9EPH1FA++bhWyaX1nmFjI9B7/Mlaxv8ut25+lpk4h1b9UBybW1z4y7XzTSp6M1vdrrOtXaIND9gHIxhNxhdjKrfnSpTMrKDMrj57FXjpJeuy9Tar+Ak/mPdWfyzxIq6uG+pivL1dqt5Q3Hx54fNXMc6S/NoT6cyfkhSRlkrcpXeZ+4LZIGN2iZaaiW76eprNKU54+K+lteY9n5YsXi+Ne5f/O5tflJ3p44m3i4R80YHXR2089p7gsBjcWOJrInV+expr1IA/icotbjOLJ81caLnLO7hdga/l+NRCrGsAbmDshaqW4YB3HoMS/0yTDq8RbwDKnyyILkGMDj7Y7c5B8YbKnoRNgvBGucpIW488c+8LJrOhaURtpJtE9fx6Xpl/Z17PErxS/9LvJv7s3YJMWfS7xu4UPlHRUlQecYKSOpCi5XBB9FtT7bUeUq6oMJCHQx4AhZKKyjbXjtN6G078f1TRqlHzjnpDdDcZ9ZhNz5kFbe67PK4hJfp5fc0Eq4Dwr0pXht0SAjqgraL50q+M1vNPDOQHT/S0PuCc0rEBzj2bpPI54FDMta9S8daPSPeNr3BDhimRYM3yC9LCP2O9zG2Q7xrq2Y0Fb2etOsnyhy+fiX9P9TYPjZlbCSSSCEpLNBNVT2O+76yy85aaOxzL1vqpwPpFF5+nmXB2CiA+f19yv/vWzHZlfd4bJImWlYDKlIuXO3eCyB0++uw6RuoYEJpJwCmbeneY4cyyWoApnFoNUcrhXE367TC191uVjj7WZE7LO600JdyxMC61SR7G3sIeDuQ=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(33656002)(54906003)(55016002)(70206006)(36860700001)(7696005)(356005)(508600001)(6862004)(52536014)(8936002)(2906002)(26005)(81166007)(8676002)(47076005)(186003)(336012)(966005)(83380400001)(316002)(86362001)(53546011)(70586007)(5660300002)(9686003)(6506007)(82310400003)(4326008);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 05:23:59.1170
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 61230c63-2902-4ba8-4ec1-08d97f1b8340
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT020.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5018

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTE6MDUNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
Pg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIDMxLzM3XSB4ZW4vYXJtOiBpbnRyb2R1Y2UgYSBoZWxw
ZXIgdG8gcGFyc2UgZGV2aWNlDQo+IHRyZWUgTlVNQSBkaXN0YW5jZSBtYXANCj4gDQo+IE9uIFRo
dSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IEEgTlVNQSBhd2FyZSBkZXZpY2Ug
dHJlZSB3aWxsIHByb3ZpZGUgYSAiZGlzdGFuY2UtbWFwIiBub2RlIHRvDQo+ID4gZGVzY3JpYmUg
ZGlzdGFuY2UgYmV0d2VlbiBhbnkgdHdvIG5vZGVzLiBUaGlzIHBhdGNoIGludHJvZHVjZSBhDQo+
ID4gbmV3IGhlbHBlciB0byBwYXJzZSB0aGlzIGRpc3RhbmNlIG1hcC4NCj4gPg0KPiA+IFNpZ25l
ZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29tPg0KPiA+IC0tLQ0KPiA+ICB4ZW4v
YXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jIHwgMTA2ICsrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrDQo+ID4gIDEgZmlsZSBjaGFuZ2VkLCAxMDYgaW5zZXJ0aW9ucygrKQ0KPiA+DQo+
ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gYi94ZW4v
YXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+ID4gaW5kZXggNzkxOGEzOTdmYS4uZTdmYTg0
ZGY0YyAxMDA2NDQNCj4gPiAtLS0gYS94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+
ID4gKysrIGIveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+IEBAIC0xMzYsMyAr
MTM2LDEwOSBAQCBzdGF0aWMgaW50IF9faW5pdCBmZHRfcGFyc2VfbnVtYV9tZW1vcnlfbm9kZShj
b25zdA0KPiB2b2lkICpmZHQsIGludCBub2RlLA0KPiA+DQo+ID4gICAgICByZXR1cm4gMDsNCj4g
PiAgfQ0KPiA+ICsNCj4gPiArDQo+ID4gKy8qIFBhcnNlIE5VTUEgZGlzdGFuY2UgbWFwIHYxICov
DQo+ID4gK3N0YXRpYyBpbnQgX19pbml0IGZkdF9wYXJzZV9udW1hX2Rpc3RhbmNlX21hcF92MShj
b25zdCB2b2lkICpmZHQsIGludA0KPiBub2RlKQ0KPiA+ICt7DQo+ID4gKyAgICBjb25zdCBzdHJ1
Y3QgZmR0X3Byb3BlcnR5ICpwcm9wOw0KPiA+ICsgICAgY29uc3QgX19iZTMyICptYXRyaXg7DQo+
ID4gKyAgICB1aW50MzJfdCBlbnRyeV9jb3VudDsNCj4gPiArICAgIGludCBsZW4sIGk7DQo+ID4g
Kw0KPiA+ICsgICAgcHJpbnRrKFhFTkxPR19JTkZPICJOVU1BOiBwYXJzaW5nIG51bWEtZGlzdGFu
Y2UtbWFwXG4iKTsNCj4gPiArDQo+ID4gKyAgICBwcm9wID0gZmR0X2dldF9wcm9wZXJ0eShmZHQs
IG5vZGUsICJkaXN0YW5jZS1tYXRyaXgiLCAmbGVuKTsNCj4gPiArICAgIGlmICggIXByb3AgKQ0K
PiA+ICsgICAgew0KPiA+ICsgICAgICAgIHByaW50ayhYRU5MT0dfV0FSTklORw0KPiA+ICsgICAg
ICAgICAgICAgICAiTlVNQTogTm8gZGlzdGFuY2UtbWF0cml4IHByb3BlcnR5IGluIGRpc3RhbmNl
LW1hcFxuIik7DQo+IA0KPiBJIGhhdmVuJ3Qgc2VlbiB3aGVyZSB0aGlzIGlzIGNhbGxlZCBmcm9t
IHlldCBidXQgbWFrZSBzdXJlIHRvIHByaW50IGFuDQo+IGVycm9yIGhlcmUgb25seSBpZiBOVU1B
IGluZm8gaXMgYWN0dWFsbHkgZXhwZWN0ZWQgYW5kIHJlcXVpcmVkLCBub3Qgb24NCj4gcmVndWxh
ciBub24tTlVNQSBib290cyBvbiBub24tTlVNQSBtYWNoaW5lcy4NCj4gDQoNCk9ubHkgdXNlcnMg
ZW5hYmxlIE5VTUEgb3B0aW9uIGFuZCBudW1hX29mZiBpcyBmYWxzZSwgdGhlbiBYZW4gY2FuIHJ1
biBpbnRvDQp0aGlzIGZ1bmN0aW9uICh0aGlzIGNoZWNrIGlzIGluIG51bWFfaW5pdCkuIFNvIG5v
bi1OVU1BIG1hY2hpbmVzIHdpbGwgbm90DQpyZWFjaCBoZXJlLg0KDQo+IA0KPiA+ICsgICAgICAg
IHJldHVybiAtRUlOVkFMOw0KPiA+ICsgICAgfQ0KPiA+ICsNCj4gPiArICAgIGlmICggbGVuICUg
c2l6ZW9mKHVpbnQzMl90KSAhPSAwICkNCj4gPiArICAgIHsNCj4gPiArICAgICAgICBwcmludGso
WEVOTE9HX1dBUk5JTkcNCj4gPiArICAgICAgICAgICAgICAgImRpc3RhbmNlLW1hdHJpeCBpbiBu
b2RlIGlzIG5vdCBhIG11bHRpcGxlIG9mIHUzMlxuIik7DQo+ID4gKyAgICAgICAgcmV0dXJuIC1F
SU5WQUw7DQo+ID4gKyAgICB9DQo+ID4gKw0KPiA+ICsgICAgZW50cnlfY291bnQgPSBsZW4gLyBz
aXplb2YodWludDMyX3QpOw0KPiA+ICsgICAgaWYgKCBlbnRyeV9jb3VudCA9PSAwICkNCj4gPiAr
ICAgIHsNCj4gPiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcgIk5VTUE6IEludmFsaWQg
ZGlzdGFuY2UtbWF0cml4XG4iKTsNCj4gPiArDQo+ID4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7
DQo+ID4gKyAgICB9DQo+ID4gKw0KPiA+ICsgICAgbWF0cml4ID0gKGNvbnN0IF9fYmUzMiAqKXBy
b3AtPmRhdGE7DQo+ID4gKyAgICBmb3IgKCBpID0gMDsgaSArIDIgPCBlbnRyeV9jb3VudDsgaSAr
PSAzICkNCj4gPiArICAgIHsNCj4gPiArICAgICAgICB1aW50MzJfdCBmcm9tLCB0bywgZGlzdGFu
Y2UsIG9wcG9zaXRlOw0KPiA+ICsNCj4gPiArICAgICAgICBmcm9tID0gZHRfbmV4dF9jZWxsKDEs
ICZtYXRyaXgpOw0KPiA+ICsgICAgICAgIHRvID0gZHRfbmV4dF9jZWxsKDEsICZtYXRyaXgpOw0K
PiA+ICsgICAgICAgIGRpc3RhbmNlID0gZHRfbmV4dF9jZWxsKDEsICZtYXRyaXgpOw0KPiA+ICsg
ICAgICAgIGlmICggKGZyb20gPT0gdG8gJiYgZGlzdGFuY2UgIT0gTlVNQV9MT0NBTF9ESVNUQU5D
RSkgfHwNCj4gPiArICAgICAgICAgICAgKGZyb20gIT0gdG8gJiYgZGlzdGFuY2UgPD0gTlVNQV9M
T0NBTF9ESVNUQU5DRSkgKQ0KPiA+ICsgICAgICAgIHsNCj4gPiArICAgICAgICAgICAgcHJpbnRr
KFhFTkxPR19XQVJOSU5HDQo+ID4gKyAgICAgICAgICAgICAgICAgICAiTlVNQTogSW52YWxpZCBk
aXN0YW5jZTogTk9ERSMldS0+Tk9ERSMldToldVxuIiwNCj4gPiArICAgICAgICAgICAgICAgICAg
IGZyb20sIHRvLCBkaXN0YW5jZSk7DQo+ID4gKyAgICAgICAgICAgIHJldHVybiAtRUlOVkFMOw0K
PiA+ICsgICAgICAgIH0NCj4gPiArDQo+ID4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19JTkZPICJO
VU1BOiBkaXN0YW5jZTogTk9ERSMldS0+Tk9ERSMldToldVxuIiwNCj4gPiArICAgICAgICAgICAg
ICAgZnJvbSwgdG8sIGRpc3RhbmNlKTsNCj4gPiArDQo+ID4gKyAgICAgICAgLyogR2V0IG9wcG9z
aXRlIHdheSBkaXN0YW5jZSAqLw0KPiA+ICsgICAgICAgIG9wcG9zaXRlID0gX19ub2RlX2Rpc3Rh
bmNlKGZyb20sIHRvKTsNCj4gDQo+IFRoaXMgaXMgbm90IGNoZWNraW5nIGZvciB0aGUgb3Bwb3Np
dGUgbm9kZSBkaXN0YW5jZSBidXQuLi4NCj4gDQoNCkFoLCB5ZXMsIGl0J3MgYSBtaXN0YWtlLiBJ
dCBzaG91bGQgYmUgX19ub2RlX2Rpc3RhbmNlKHRvLCBmcm9tKTsNCj4gDQo+ID4gKyAgICAgICAg
aWYgKCBvcHBvc2l0ZSA9PSAwICkNCj4gPiArICAgICAgICB7DQo+ID4gKyAgICAgICAgICAgIC8q
IEJpLWRpcmVjdGlvbnMgYXJlIG5vdCBzZXQsIHNldCBib3RoICovDQo+ID4gKyAgICAgICAgICAg
IG51bWFfc2V0X2Rpc3RhbmNlKGZyb20sIHRvLCBkaXN0YW5jZSk7DQo+ID4gKyAgICAgICAgICAg
IG51bWFfc2V0X2Rpc3RhbmNlKHRvLCBmcm9tLCBkaXN0YW5jZSk7DQo+IA0KPiAuLi5zaW5jZSB5
b3Ugc2V0IGJvdGggZGlyZWN0aW9ucyBoZXJlIGF0IG9uY2UgdGhlbiBpdCBpcyBPSy4gWW91IGFy
ZQ0KPiBjaGVja2luZyBpZiB0aGlzIGRpcmVjdGlvbiBoYXMgYWxyZWFkeSBiZWVuIHNldCB3aGlj
aCBpcyBjb3JyZWN0IEkNCj4gdGhpbmsuIEJ1dCB0aGUgY29tbWVudCAiR2V0IG9wcG9zaXRlIHdh
eSBkaXN0YW5jZSIgYW5kIHRoZSB2YXJpYWJsZSBuYW1lDQo+ICJvcHBvc2l0ZSIgYXJlIHdyb25n
Lg0KPiANCg0KTXkgYWJvdmUgbWlzdGFrZSBtYWtlIHRoaXMgbWlzLXVuZGVyc3RhbmRpbmc6DQpJ
IHdhbnQgdG8gY2hlY2sgdGhlIG9wcG9zaXRlIHdheSBkaXN0YW5jZSBpcyBzZXQgb3Igbm90Lg0K
SWYgb3Bwb3NpdGUgd2F5IGRpc3RhbmNlIGlzIG5vdCBzZXQsIEkgd2lsbCBzZXQgYm90aCB3YXkg
aGVyZS4NCg0KU28gSSB3aWxsIGNoYW5nZSAiIG9wcG9zaXRlID0gX19ub2RlX2Rpc3RhbmNlKGZy
b20sIHRvKTsiIHRvDQoiIG9wcG9zaXRlID0gX19ub2RlX2Rpc3RhbmNlKHRvLCBmcm9tKTsiLiBB
bmQga2VlcCB0aGUgY29tbWVudC4NCkhvdyBkbyB5b3UgdGhpbmsgYWJvdXQgaXQ/DQoNCj4gDQo+
ID4gKyAgICAgICAgfQ0KPiA+ICsgICAgICAgIGVsc2UNCj4gPiArICAgICAgICB7DQo+ID4gKyAg
ICAgICAgICAgIC8qDQo+ID4gKyAgICAgICAgICAgICAqIE9wcG9zaXRlIHdheSBkaXN0YW5jZSBo
YXMgYmVlbiBzZXQgdG8gYSBkaWZmZXJlbnQgdmFsdWUuDQo+ID4gKyAgICAgICAgICAgICAqIEl0
IG1heSBiZSBhIGZpcm13YXJlIGRldmljZSB0cmVlIGJ1Zz8NCj4gPiArICAgICAgICAgICAgICov
DQo+ID4gKyAgICAgICAgICAgIGlmICggb3Bwb3NpdGUgIT0gZGlzdGFuY2UgKQ0KPiA+ICsgICAg
ICAgICAgICB7DQo+ID4gKyAgICAgICAgICAgICAgICAvKg0KPiA+ICsgICAgICAgICAgICAgICAg
ICogSW4gZGV2aWNlIHRyZWUgTlVNQSBkaXN0YW5jZS1tYXRyaXggYmluZGluZzoNCj4gPiArICAg
ICAgICAgICAgICAgICAqDQo+IGh0dHBzOi8vd3d3Lmtlcm5lbC5vcmcvZG9jL0RvY3VtZW50YXRp
b24vZGV2aWNldHJlZS9iaW5kaW5ncy9udW1hLnR4dA0KPiA+ICsgICAgICAgICAgICAgICAgICog
VGhlcmUgaXMgYSBub3RlcyBtZW50aW9uczoNCj4gPiArICAgICAgICAgICAgICAgICAqICJFYWNo
IGVudHJ5IHJlcHJlc2VudHMgZGlzdGFuY2UgZnJvbSBmaXJzdCBub2RlIHRvDQo+ID4gKyAgICAg
ICAgICAgICAgICAgKiAgc2Vjb25kIG5vZGUuIFRoZSBkaXN0YW5jZXMgYXJlIGVxdWFsIGluIGVp
dGhlcg0KPiA+ICsgICAgICAgICAgICAgICAgICogIGRpcmVjdGlvbi4iDQo+ID4gKyAgICAgICAg
ICAgICAgICAgKg0KPiA+ICsgICAgICAgICAgICAgICAgICogVGhhdCBtZWFucyBkZXZpY2UgdHJl
ZSBkb2Vzbid0IHBlcm1pdCB0aGlzIGNhc2UuDQo+ID4gKyAgICAgICAgICAgICAgICAgKiBCdXQg
aW4gQUNQSSBzcGVjLCBpdCBjYXJlcyB0byBzcGVjaWZpY2FsbHkgcGVybWl0DQo+IHRoaXMNCj4g
PiArICAgICAgICAgICAgICAgICAqIGNhc2U6DQo+ID4gKyAgICAgICAgICAgICAgICAgKiAiRXhj
ZXB0IGZvciB0aGUgcmVsYXRpdmUgZGlzdGFuY2UgZnJvbSBhIFN5c3RlbQ0KPiBMb2NhbGl0eQ0K
PiA+ICsgICAgICAgICAgICAgICAgICogIHRvIGl0c2VsZiwgZWFjaCByZWxhdGl2ZSBkaXN0YW5j
ZSBpcyBzdG9yZWQgdHdpY2UNCj4gaW4gdGhlDQo+ID4gKyAgICAgICAgICAgICAgICAgKiAgbWF0
cml4LiBUaGlzIHByb3ZpZGVzIHRoZSBjYXBhYmlsaXR5IHRvIGRlc2NyaWJlDQo+IHRoZQ0KPiA+
ICsgICAgICAgICAgICAgICAgICogIHNjZW5hcmlvIHdoZXJlIHRoZSByZWxhdGl2ZSBkaXN0YW5j
ZXMgZm9yIHRoZSB0d28NCj4gPiArICAgICAgICAgICAgICAgICAqICBkaXJlY3Rpb25zIGJldHdl
ZW4gU3lzdGVtIExvY2FsaXRpZXMgaXMgZGlmZmVyZW50LiINCj4gPiArICAgICAgICAgICAgICAg
ICAqDQo+ID4gKyAgICAgICAgICAgICAgICAgKiBUaGF0IG1lYW5zIGEgcmVhbCBtYWNoaW5lIGFs
bG93cyBzdWNoIE5VTUENCj4gY29uZmlndXJhdGlvbi4NCj4gPiArICAgICAgICAgICAgICAgICAq
IFNvLCBwbGFjZSBhIFdBUk5JTkcgaGVyZSB0byBub3RpY2Ugc3lzdGVtDQo+IGFkbWluaXN0cmF0
b3JzLA0KPiA+ICsgICAgICAgICAgICAgICAgICogaXMgaXQgdGhlIHNwZWNhaWwgY2FzZSB0aGF0
IHRoZXkgaGlqYWNrIHRoZSBkZXZpY2UNCj4gdHJlZQ0KPiA+ICsgICAgICAgICAgICAgICAgICog
dG8gc3VwcG9ydCB0aGVpciByYXJlIG1hY2hpbmVzPw0KPiA+ICsgICAgICAgICAgICAgICAgICov
DQo+ID4gKyAgICAgICAgICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcNCj4gPiArICAgICAg
ICAgICAgICAgICAgICAgICAiVW4tbWF0Y2hlZCBiaS1kaXJlY3Rpb24hIE5PREUjJXUtPk5PREUj
JXU6JXUsDQo+IE5PREUjJXUtPk5PREUjJXU6JXVcbiIsDQo+ID4gKyAgICAgICAgICAgICAgICAg
ICAgICAgZnJvbSwgdG8sIGRpc3RhbmNlLCB0bywgZnJvbSwgb3Bwb3NpdGUpOw0KPiANCj4gUFJJ
dTMyDQoNClllcy4NCg0KPiANCj4gDQo+ID4gKyAgICAgICAgICAgIH0NCj4gPiArDQo+ID4gKyAg
ICAgICAgICAgIC8qIE9wcG9zaXRlIHdheSBkaXN0YW5jZSBoYXMgYmVlbiBzZXQsIGp1c3Qgc2V0
IHRoaXMgd2F5ICovDQo+ID4gKyAgICAgICAgICAgIG51bWFfc2V0X2Rpc3RhbmNlKGZyb20sIHRv
LCBkaXN0YW5jZSk7DQo+ID4gKyAgICAgICAgfQ0KPiA+ICsgICAgfQ0KPiA+ICsNCj4gPiArICAg
IHJldHVybiAwOw0KPiA+ICt9DQo+ID4gLS0NCj4gPiAyLjI1LjENCj4gPg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 05:32:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 05:32:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194923.347312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTdpI-0001T3-G3; Fri, 24 Sep 2021 05:32:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194923.347312; Fri, 24 Sep 2021 05:32:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTdpI-0001Sw-Cv; Fri, 24 Sep 2021 05:32:48 +0000
Received: by outflank-mailman (input) for mailman id 194923;
 Fri, 24 Sep 2021 05:32:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GRL2=OO=linux.ibm.com=rppt@srs-us1.protection.inumbo.net>)
 id 1mTdpG-0001Sq-Nz
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 05:32:46 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.158.5])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d842d71e-1cf8-11ec-baa9-12813bfff9fa;
 Fri, 24 Sep 2021 05:32:44 +0000 (UTC)
Received: from pps.filterd (m0098419.ppops.net [127.0.0.1])
 by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18O4TxLE003727; 
 Fri, 24 Sep 2021 01:32:30 -0400
Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com
 [169.51.49.98])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3b97px199p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 01:32:30 -0400
Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1])
 by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18O5WBU3010032;
 Fri, 24 Sep 2021 05:32:28 GMT
Received: from b06avi18878370.portsmouth.uk.ibm.com
 (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194])
 by ppma03ams.nl.ibm.com with ESMTP id 3b93gb1yvs-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 05:32:28 +0000
Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com
 [9.149.105.58])
 by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP
 id 18O5RWr150528586
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Fri, 24 Sep 2021 05:27:32 GMT
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id A77234C04E;
 Fri, 24 Sep 2021 05:32:25 +0000 (GMT)
Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 26CB04C050;
 Fri, 24 Sep 2021 05:32:23 +0000 (GMT)
Received: from linux.ibm.com (unknown [9.145.159.121])
 by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS;
 Fri, 24 Sep 2021 05:32:23 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d842d71e-1cf8-11ec-baa9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc :
 subject : message-id : references : content-type : in-reply-to :
 content-transfer-encoding : mime-version; s=pp1;
 bh=LcclFexTC3W4CGWQ05F30N7TENmKQs3CWJF8lIJhAyA=;
 b=fM5r0pOd7rJd2mnURChsSfJfJrX/C+3BofObflhTYGzVXv/HEJS9Aw8BgIC//ggGIR4j
 CZ5JxdcP59VcALFL5IzVij2oY4zwlRyWWbJs8bYZFXoi5eLF6aEHqAyx0sRU0twQi+Nx
 uTzT35Ud5rz3mMLTSHcDte6njyO8gAKbwGxAXdnivxmbV3hoFkXGse9K5ImTnY3FP/yx
 Dcg6IMAClTzZSYBturZR2pFZSHk8drui/RqjiPVFtbOoDuEmvxDYqAtOWfweNAxqZrTZ
 HQI3Eeh+Tf3jj5APaqHV6+/E8KJV71sFQyUh51zWXPgx91EVjartcf89ESTdpFpWq0zU zQ== 
Date: Fri, 24 Sep 2021 08:32:21 +0300
From: Mike Rapoport <rppt@linux.ibm.com>
To: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Mike Rapoport <rppt@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        devicetree@vger.kernel.org, linux-efi@vger.kernel.org,
        kvm@vger.kernel.org, linux-s390@vger.kernel.org,
        linux-sh@vger.kernel.org, linux-um@lists.infradead.org,
        linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com,
        linux-mips@vger.kernel.org, linux-mm@kvack.org,
        iommu@lists.linux-foundation.org, linux-usb@vger.kernel.org,
        linux-alpha@vger.kernel.org, sparclinux@vger.kernel.org,
        xen-devel@lists.xenproject.org,
        Andrew Morton <akpm@linux-foundation.org>,
        linux-snps-arc@lists.infradead.org, linuxppc-dev@lists.ozlabs.org,
        linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 3/3] memblock: cleanup memblock_free interface
Message-ID: <YU1i5YyldfS1HH0j@linux.ibm.com>
References: <20210923074335.12583-1-rppt@kernel.org>
 <20210923074335.12583-4-rppt@kernel.org>
 <1101e3c7-fcb7-a632-8e22-47f4a01ea02e@csgroup.eu>
 <YUxsgN/uolhn1Ok+@linux.ibm.com>
 <96e3da9f-70ff-e5c0-ef2e-cf0b636e5695@csgroup.eu>
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <96e3da9f-70ff-e5c0-ef2e-cf0b636e5695@csgroup.eu>
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: EYYWPQtbK60SNKLXoEtnrAemGAQvCJn5
X-Proofpoint-GUID: EYYWPQtbK60SNKLXoEtnrAemGAQvCJn5
Content-Transfer-Encoding: 8bit
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-24_01,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 impostorscore=0 mlxscore=0 lowpriorityscore=0 bulkscore=0 spamscore=0
 clxscore=1015 mlxlogscore=856 adultscore=0 phishscore=0 suspectscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109240031

On Thu, Sep 23, 2021 at 03:54:46PM +0200, Christophe Leroy wrote:
> 
> Le 23/09/2021  14:01, Mike Rapoport a crit:
> > On Thu, Sep 23, 2021 at 11:47:48AM +0200, Christophe Leroy wrote:
> > > 
> > > 
> > > Le 23/09/2021  09:43, Mike Rapoport a crit:
> > > > From: Mike Rapoport <rppt@linux.ibm.com>
> > > > 
> > > > For ages memblock_free() interface dealt with physical addresses even
> > > > despite the existence of memblock_alloc_xx() functions that return a
> > > > virtual pointer.
> > > > 
> > > > Introduce memblock_phys_free() for freeing physical ranges and repurpose
> > > > memblock_free() to free virtual pointers to make the following pairing
> > > > abundantly clear:
> > > > 
> > > > 	int memblock_phys_free(phys_addr_t base, phys_addr_t size);
> > > > 	phys_addr_t memblock_phys_alloc(phys_addr_t base, phys_addr_t size);
> > > > 
> > > > 	void *memblock_alloc(phys_addr_t size, phys_addr_t align);
> > > > 	void memblock_free(void *ptr, size_t size);
> > > > 
> > > > Replace intermediate memblock_free_ptr() with memblock_free() and drop
> > > > unnecessary aliases memblock_free_early() and memblock_free_early_nid().
> > > > 
> > > > Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> > > > Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
> > > > ---
> > > 
> > > > diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
> > > > index 1a04e5bdf655..37826d8c4f74 100644
> > > > --- a/arch/s390/kernel/smp.c
> > > > +++ b/arch/s390/kernel/smp.c
> > > > @@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
> > > >    			/* Get the CPU registers */
> > > >    			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
> > > >    	}
> > > > -	memblock_free(page, PAGE_SIZE);
> > > > +	memblock_phys_free(page, PAGE_SIZE);
> > > >    	diag_amode31_ops.diag308_reset();
> > > >    	pcpu_set_smt(0);
> > > >    }
> > > > @@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
> > > >    	/* Add CPUs present at boot */
> > > >    	__smp_rescan_cpus(info, true);
> > > > -	memblock_free_early((unsigned long)info, sizeof(*info));
> > > > +	memblock_free(info, sizeof(*info));
> > > >    }
> > > >    /*
> > > 
> > > I'm a bit lost. IIUC memblock_free_early() and memblock_free() where
> > > identical.
> > 
> > Yes, they were, but all calls to memblock_free_early() were using
> > __pa(vaddr) because they had a virtual address at hand.
> 
> I'm still not following. In the above memblock_free_early() was taking
> (unsigned long)info . Was it a bug ? 

Not really because s390 has pa == va:

https://elixir.bootlin.com/linux/latest/source/arch/s390/include/asm/page.h#L169


-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 05:38:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 05:38:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194928.347323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTduT-0002Vo-4L; Fri, 24 Sep 2021 05:38:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194928.347323; Fri, 24 Sep 2021 05:38:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTduT-0002Vh-19; Fri, 24 Sep 2021 05:38:09 +0000
Received: by outflank-mailman (input) for mailman id 194928;
 Fri, 24 Sep 2021 05:38:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Coby=OO=epam.com=prvs=9901c588ba=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTduR-0002Vb-0w
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 05:38:07 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 976e6392-1cf9-11ec-baa9-12813bfff9fa;
 Fri, 24 Sep 2021 05:38:05 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18O3wOWd022021;
 Fri, 24 Sep 2021 05:38:04 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b93g78f5p-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 05:38:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6322.eurprd03.prod.outlook.com (2603:10a6:20b:15e::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 05:37:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 05:37:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 976e6392-1cf9-11ec-baa9-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=i3sDiaUboNg+WxdsGbk9SHJe6cSCbu/df5nlFYt4p63U0jyY8jCzM3fbshevsfq29yjFMiPWZlxaPRdP3CbLyp6xstkCPA2l0XgVFVgW7kXfmOkiDvjFcDP9PjIPurUgqSPsN/YRy7AFqMX7YlPWXKm5osUM86D7Vpv4cO+jJd+Q5XoXvZ0h4Z/O28LjGeglxkGzMPg4g6vgRJw+I/Q4nH6lIvnyH2nCB/VbeaxEQ/2his+oR7dxlk/DQtBoaIYjUla+QZSP0FGI7QvPTwa/lR1KXKhzqYZ5ZNYAdEAXclMazQ5DYgHNLQOeZ2zh+1EmmTjwUdWdFvWOOz7wg6WU9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6Fd3E1E5kG9ArBNNIfiV0R2o1x38+reHDTauKWc6XF0=;
 b=DhybKxS+fqtQzji0e/NaVOb8Rd8/uIZB1aqzsCFMJBnKcqmdCfCz7tAI7Ch52LJB2E2bFmg+9mwmLogMYL8pq5SdvIEYZBGwMGcnGWGBIinxgbvTByoegltBYOZlWTQU+uqDwAqTQHTJE7Hj4QJttqVLYssEl4pg+WGhm/Z26VBazicGtB6/sJ2nnmRjWJQsDqeSCoxkYab/KwvKRnr4D510Xm1PzDtrYIrJUP+trKZVVsULgZvgGbRNfg3iuQQ3oxpxU/UD6cOnxZIjWnSFOTAVgnGFiuxR7hg5oEvRqCW924QtLz/JvdB6IQ2Jw3mID2Yw0hdrriD22KrnXoRNVQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6Fd3E1E5kG9ArBNNIfiV0R2o1x38+reHDTauKWc6XF0=;
 b=CafRroWl+pWb5K+Big+feEUQ+rDfkfsATTOhFvCCzR25gnUWzM5WxYXBUZh0SnBGZVGVuY4oB/FBtk6Dr3f7a9K5sAiRrKCxqEtNzNKjkXPDJfrFXnPWDciMfGg5QaQE2KUM0sfzjHS2J0xxbCyvoYOaLC9jRlLGwpcSJHQztiHk1r/7yXmtXFpx3ljXmUrdkq1vhRvZTbTXl7FFnnKP088dJ5o7xDh2KIEET4EGYFW2huLj+tylYEVXIoW9/BdE4WhDnaMFtOdtwxq4Rk01tnhnQFfL+LrlRphgR/gCJjYSY72nF7d7dptokGkaASq8WvY5s1CWGGsShCN5ouEhiw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com"
	<jgross@suse.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>
Subject: Re: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v3 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: AQHXsGDshY1QGXQ13Ui5WskBVnVMCauyBw8AgACk2YA=
Date: Fri, 24 Sep 2021 05:37:59 +0000
Message-ID: <9fa3bd45-6605-7a77-e0b7-3fb91f093eff@epam.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <alpine.DEB.2.21.2109231217050.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231217050.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a71aeaae-7311-495b-44f1-08d97f1d783e
x-ms-traffictypediagnostic: AM0PR03MB6322:
x-microsoft-antispam-prvs: 
 <AM0PR03MB6322513C3D69F5778A24BCBDE7A49@AM0PR03MB6322.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Rq7Fc1MfZbHfCgRRRc8FfZuKwqXUhvkFy+T4PED7Wdvnk8F9frxSaFz8OuOklZIqDNWV5V8nXJZLSDEhR2SW3FoP8QQPFbOwKYcT+bau2s/nvpHlh9ClPfEK5tIir4AgMAjLaic57twM8G0nsuyOF9+dpRNjOdStaCUaXOhQ36zSJayiXBaRX6VGqTnFmM4N1mW86Fo4Yt0f1WmsEJzoE+9swhad2ZJoXaJ1zIp0lvr1yxUAFNqJIpvlg8fNVZOt3VIcWDdCU6LdhaSJgIXJmokxzdISU9cOuYs/R25qYH7gzwga4RD5YsWVRPriHhp6fMhT9UWKWr420FxDgZWCQj4uOI10JFGlXcbtETWsl66Rz80oqcmv41tmCDknoSbJwNHqSgC1miOcnUixv2naTlW9Agi6R6Zp5RNBOfEUErrEp9a4wihRZDcQ3OkIpaBbLKaMnLTH1NHhZofB1I/1M/o8oFrAjoej0BNB02Bku3rPND/EpNElQ/fvcFJfQovhaVdnWI7kzEnwcMRCyNi+IKvmjC7GiySUduaLk1iRP0LcXVGmd6Ngo62NJt2RDQwTx2DivQB3BkzwHbbgpJAiZiW7cdOQW1ofV3MxdWOKKD7ZEuvnkCGcfCke7KNCJOtNLqZEFyVrkeq6d1f1UIZks++6nAGBwLyKGUbzHGsnIXVzNggKM+Uocx+m7qZksvYAaPae509L7UnuKMg8QBxdVymiv1TFXBZMTttHpT0lwcvovk4uEKdn9GCK9gRSOQnZ
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(55236004)(54906003)(71200400001)(122000001)(186003)(83380400001)(6512007)(31696002)(66446008)(64756008)(38100700002)(86362001)(508600001)(38070700005)(66556008)(66946007)(36756003)(66476007)(5660300002)(8936002)(2906002)(26005)(76116006)(91956017)(6486002)(8676002)(31686004)(53546011)(2616005)(316002)(6506007)(4326008)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?TU9ya2hmU3R6QU5XdVpqT3ljMEI4TlhPQStwWGd5SlZTYlBaQUNTS3lvZ2t5?=
 =?utf-8?B?V0ZORElKQTFWcDZmNERxYVRjRnVtWmJnamNZVG5hV1BPYjlmTWhCSWxyV3Zu?=
 =?utf-8?B?RGJVYm5YditycFZGT1puVUNwYUtPci9mNW02R2dnekppWGw4anlYUHJnTUl3?=
 =?utf-8?B?VE83aVNxRUx0UDhtNXdoMmI2UXA5a29XVm5lUWFRdjU0cml4UUxSdHNKV1ho?=
 =?utf-8?B?citKc0d1b0pWSDF0dzczSUdJejhBNGhuM3gvY2FocmtVMDJaZGw2SGI5UEZj?=
 =?utf-8?B?NnFRNVpmTVFCSlZwNVBSNUxxZjUrN1pCdSt3aERCangrVFdMRElEZUtlc1Rs?=
 =?utf-8?B?RW9xVy9oUzlBWi9aQU1ad2JnbndlUTJvbTA0M0U2aFBES0lXTXVucytUN2p2?=
 =?utf-8?B?ZDlsdFRIRFh5SENVbkI2K3N1VW1WTnZTTXVJL2ZodXlyZkdBcFdGU3M3NURh?=
 =?utf-8?B?SDlBeXpXRVNpTW1yOU9IbTRHdEhDakJGeXVNbXpGbGd1UkVEUHRQWEtMTGZV?=
 =?utf-8?B?OTNudm03VmYvci9TcE8zaTZIMkFFVUd5ZjBvYytOcSs4OGk0UFQ0dFArRkM1?=
 =?utf-8?B?QUdqQUc2Y0VOOFNOTWtqdmdRMTQxOERwQXZteXNHNGF5SGNpLzdzMnI4cFB6?=
 =?utf-8?B?OW9QWDlzZDNTQ3Y2NW1XZTVOemdMVnZkK0N3RktHd01sTDhIenZHL3oxSlNy?=
 =?utf-8?B?SzROVGN5UDk4WmVaUzJWOXRUaEVmN0xWa0NIZUJMeGdHcCtteTN3eUp2OEQv?=
 =?utf-8?B?MTBpajdaSDBJdjA2a2hXTXQ3bU15OXFld3hyN2NPUlF1UzR3SEhYVGYyY3dD?=
 =?utf-8?B?Z0FEOUJ3Zmg5NlJ1WHZPbDUrVWNBRG53QVFISE5EQnNENGNpQ29aUGpyL3VE?=
 =?utf-8?B?Rm9STDBFMDl5akZPVWZhT3JBOUpqZWMrdWllZUVXZm1YbUE2RDUxM1pSRnc1?=
 =?utf-8?B?aHVTZ0Jhdkk1aXNBbmY3djhTMlljcUhYNVBPRU9YSEZWalpwYmVQd1NOSjE5?=
 =?utf-8?B?US94aUhZTm5wcjdqNnBFaVVURFdGZVJhTUNETDRSR3Voams0bmdKVytTV01J?=
 =?utf-8?B?K0d3a2hHQm5nY2tsQ2hVRVBKQnZRWWtNeXExVVNZcjdwVkdVelBraTh3ZXZN?=
 =?utf-8?B?QkppS2RTaytha2JWcVdBSGZjakVLQlRFMXJ6dk9peEhIZE14emNUWFVJcDFF?=
 =?utf-8?B?Mi9XczJRaisxK29zY0VhcFVzT3MyZXFTK21lWEpBVUplTzFOVXBldTZXU2h1?=
 =?utf-8?B?WE1ucWZsUDJxay9IUWlmNHBNeXREaXdhMTZzOU9hdzdrWVBJcVVFektBTlBW?=
 =?utf-8?B?YjN6b2piRXpZanhRN1k0WERJd1l3Ynh3ZnU3ZXd0OVU0MG5Jai90NXlRaWtX?=
 =?utf-8?B?ODFLV0FOL2JJTE9XajNtYjdIOGk4b1JYREVUeHV5NmFQdGxtRFdlUTFWb1lw?=
 =?utf-8?B?TE4vZGJKdEpNUno5NG5FNjBhMHl6NDVlamJ6dDlYOUFwUmk0WXRYTy9UbjB1?=
 =?utf-8?B?OENSUWtOc2d1aUVVTEUvUmFnSnJseVpJYkdtRHNoNUZOTzRBblNNWk9Gd016?=
 =?utf-8?B?V01EaTFLdWFrMy96cXFCcy9qZm9GS2ZvaDZTR2dOSFBXTTdLZ0F3Vk14RTN3?=
 =?utf-8?B?OXArS3F0cm4vNndvZm84Zmx3TWxWREpsVmdpbmdUZGhLaHFBMVhJTXZPSC9k?=
 =?utf-8?B?cmd5NURtTTl3bVFGTmpUQmdWaS8wcENXaHR0SytQcWFTQk5ZeDhtV3NHeGd6?=
 =?utf-8?B?OVVUVGVraXk3UkVydFJTa0kxUUUzZWc5MVphT1EveG5ocDhnUGlGTjVQbjkx?=
 =?utf-8?B?MExHek53amxhRUZEVllldz09?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <149BF2E93DE3A540A631D03426217848@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a71aeaae-7311-495b-44f1-08d97f1d783e
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 05:37:59.6975
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: PcJuUaYVv6WTtlxDjI9ttXide0zBdEC/m6N/PPqXGyMa7UKv6BpzOabeleGm2WcGez8J3K0OIh4bZ9Ozm06Ej2EbajKBH56hMTo7+v6aCiKTZL5YJRUCwYsgnkLbRQAM
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6322
X-Proofpoint-GUID: VPe82lMRurUDbbIx5lU38aspdAM_CR9H
X-Proofpoint-ORIG-GUID: VPe82lMRurUDbbIx5lU38aspdAM_CR9H
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-24_01,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 spamscore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0 phishscore=0
 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 clxscore=1015
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109240032

DQpPbiAyMy4wOS4yMSAyMjo0NywgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEN1cnJlbnRs
eSBQQ0kgYmFja2VuZCBpbXBsZW1lbnRzIG11bHRpcGxlIGZ1bmN0aW9uYWxpdGllcyBhdCBhIHRp
bWUuDQo+PiBUbyBuYW1lIGEgZmV3Og0KPj4gMS4gSXQgaXMgdXNlZCBhcyBhIGRhdGFiYXNlIGZv
ciBhc3NpZ25hYmxlIFBDSSBkZXZpY2VzLCBlLmcuIHhsDQo+PiAgICAgcGNpLWFzc2lnbmFibGUt
e2FkZHxyZW1vdmV8bGlzdH0gbWFuaXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywgd2hlbmV2ZXINCj4+
ICAgICB0aGUgdG9vbHN0YWNrIG5lZWRzIHRvIGtub3cgd2hpY2ggUENJIGRldmljZXMgY2FuIGJl
IHBhc3NlZCB0aHJvdWdoDQo+PiAgICAgaXQgcmVhZHMgdGhhdCBmcm9tIHRoZSByZWxldmFudCBz
eXNmcyBlbnRyaWVzIG9mIHRoZSBwY2liYWNrLg0KPj4gMi4gSXQgaXMgdXNlZCB0byBob2xkIHRo
ZSB1bmJvdW5kIFBDSSBkZXZpY2VzIGxpc3QsIGUuZy4gd2hlbiBwYXNzaW5nDQo+PiAgICAgdGhy
b3VnaCBhIFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFu
dCBkZXZpY2UNCj4+ICAgICBkcml2ZXIgYW5kIGJvdW5kIHRvIHBjaWJhY2sgKHN0cmljdGx5IHNw
ZWFraW5nIGl0IGlzIG5vdCByZXF1aXJlZA0KPj4gICAgIHRoYXQgdGhlIGRldmljZSBpcyBib3Vu
ZCB0byBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1c2VkIGFzIGENCj4+ICAgICBkYXRh
YmFzZSBvZiB0aGUgcGFzc2VkIHRocm91Z2ggUENJIGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5k
IHRoZQ0KPj4gICAgIGRldmljZXMgYmFjayB0byB0aGVpciBvcmlnaW5hbCBkcml2ZXJzIHdoZW4g
Z3Vlc3QgZG9tYWluIHNodXRzIGRvd24pDQo+PiAzLiBEZXZpY2UgcmVzZXQgZm9yIHRoZSBkZXZp
Y2VzIGJlaW5nIHBhc3NlZCB0aHJvdWdoDQo+PiA0LiBQYXJhLXZpcnR1YWxpemVkIHVzZS1jYXNl
cyBzdXBwb3J0DQo+Pg0KPj4gVGhlIHBhcmEtdmlydHVhbGl6ZWQgcGFydCBvZiB0aGUgZHJpdmVy
IGlzIG5vdCBhbHdheXMgbmVlZGVkIGFzIHNvbWUNCj4+IGFyY2hpdGVjdHVyZXMsIGUuZy4gQXJt
IG9yIHg4NiBQVkggRG9tMCwgYXJlIG5vdCB1c2luZyBiYWNrZW5kLWZyb250ZW5kDQo+PiBtb2Rl
bCBmb3IgUENJIGRldmljZSBwYXNzdGhyb3VnaC4gRm9yIHN1Y2ggdXNlLWNhc2VzIG1ha2UgdGhl
IHZlcnkNCj4+IGZpcnN0IHN0ZXAgaW4gc3BsaXR0aW5nIHRoZSB4ZW4tcGNpYmFjayBkcml2ZXIg
aW50byB0d28gcGFydHM6IGV4dGVuZGVkDQo+PiBQQ0kgc3R1YiBhbmQgUENJIFBWIGJhY2tlbmQg
ZHJpdmVycy4gQXQgdGhlIG1vbWVudCB4ODYgcGxhdGZvcm0gd2lsbA0KPj4gY29udGludWUgdXNp
bmcgQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCBmb3IgdGhlIGZ1bGx5IGZlYXR1cmVkIGJhY2tl
bmQNCj4+IGRyaXZlciBhbmQgbmV3IHBsYXRmb3JtcyBtYXkgYnVpbGQgYSBkcml2ZXIgd2l0aCBs
aW1pdGVkIGZ1bmN0aW9uYWxpdHkNCj4+IChubyBQVikgYnkgZW5hYmxpbmcgQ09ORklHX1hFTl9Q
Q0lERVZfU1RVQi4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pg0KPj4gLS0tDQo+PiBDaGFu
Z2VzIHNpbmNlIHYyOg0KPj4gICAtIHN3YXAgdGhlIHBhdGNoIG9yZGVyDQo+PiBOZXcgaW4gdjIN
Cj4+IC0tLQ0KPj4gICBkcml2ZXJzL3hlbi9LY29uZmlnICAgICAgICAgICAgICAgfCAyNCArKysr
KysrKysrKysrKysrKysrKysrKysNCj4+ICAgZHJpdmVycy94ZW4vTWFrZWZpbGUgICAgICAgICAg
ICAgIHwgIDIgKy0NCj4+ICAgZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUgIHwgIDEg
Kw0KPj4gICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2liYWNrLmggfCAgNSArKysrKw0KPj4g
ICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYyAgfCAgNiArKysrKy0NCj4+ICAgNSBm
aWxlcyBjaGFuZ2VkLCAzNiBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQ0KPj4NCj4+IGRp
ZmYgLS1naXQgYS9kcml2ZXJzL3hlbi9LY29uZmlnIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPj4g
aW5kZXggYTM3ZWI1MmZiNDAxLi42ZTkyYzZiZTE5ZjEgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJz
L3hlbi9LY29uZmlnDQo+PiArKysgYi9kcml2ZXJzL3hlbi9LY29uZmlnDQo+PiBAQCAtMTgwLDEw
ICsxODAsMzQgQEAgY29uZmlnIFNXSU9UTEJfWEVODQo+PiAgIAlzZWxlY3QgRE1BX09QUw0KPj4g
ICAJc2VsZWN0IFNXSU9UTEINCj4+ICAgDQo+PiArY29uZmlnIFhFTl9QQ0lfU1RVQg0KPj4gKwli
b29sDQo+PiArDQo+PiArY29uZmlnIFhFTl9QQ0lERVZfU1RVQg0KPj4gKwl0cmlzdGF0ZSAiWGVu
IFBDSS1kZXZpY2Ugc3R1YiBkcml2ZXIiDQo+PiArCWRlcGVuZHMgb24gUENJICYmICFYODYgJiYg
WEVODQo+PiArCWRlcGVuZHMgb24gWEVOX0JBQ0tFTkQNCj4+ICsJc2VsZWN0IFhFTl9QQ0lfU1RV
Qg0KPj4gKwlkZWZhdWx0IG0NCj4+ICsJaGVscA0KPj4gKwkgIFRoZSBQQ0kgZGV2aWNlIHN0dWIg
ZHJpdmVyIHByb3ZpZGVzIGxpbWl0ZWQgdmVyc2lvbiBvZiB0aGUgUENJDQo+PiArCSAgZGV2aWNl
IGJhY2tlbmQgZHJpdmVyIHdpdGhvdXQgcGFyYS12aXJ0dWFsaXplZCBzdXBwb3J0IGZvciBndWVz
dHMuDQo+PiArCSAgSWYgeW91IHNlbGVjdCB0aGlzIHRvIGJlIGEgbW9kdWxlLCB5b3Ugd2lsbCBu
ZWVkIHRvIG1ha2Ugc3VyZSBubw0KPj4gKwkgIG90aGVyIGRyaXZlciBoYXMgYm91bmQgdG8gdGhl
IGRldmljZShzKSB5b3Ugd2FudCB0byBtYWtlIHZpc2libGUgdG8NCj4+ICsJICBvdGhlciBndWVz
dHMuDQo+PiArDQo+PiArCSAgVGhlICJoaWRlIiBwYXJhbWV0ZXIgKG9ubHkgYXBwbGljYWJsZSBp
ZiBiYWNrZW5kIGRyaXZlciBpcyBjb21waWxlZA0KPj4gKwkgIGludG8gdGhlIGtlcm5lbCkgYWxs
b3dzIHlvdSB0byBiaW5kIHRoZSBQQ0kgZGV2aWNlcyB0byB0aGlzIG1vZHVsZQ0KPj4gKwkgIGZy
b20gdGhlIGRlZmF1bHQgZGV2aWNlIGRyaXZlcnMuIFRoZSBhcmd1bWVudCBpcyB0aGUgbGlzdCBv
ZiBQQ0kgQkRGczoNCj4+ICsJICB4ZW4tcGNpYmFjay5oaWRlPSgwMzowMC4wKSgwNDowMC4wKQ0K
Pj4gKw0KPj4gKwkgIElmIGluIGRvdWJ0LCBzYXkgbS4NCj4+ICsNCj4gSSBnZXQgdGhpcyBidWls
ZCBlcnJvciBvbiBBUk06DQo+DQo+IGRyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmM6
MjI6MTA6IGZhdGFsIGVycm9yOiBhc20veGVuL3BjaS5oOiBObyBzdWNoIGZpbGUgb3IgZGlyZWN0
b3J5DQo+ICAgI2luY2x1ZGUgPGFzbS94ZW4vcGNpLmg+DQo+ICAgICAgICAgICAgXn5+fn5+fn5+
fn5+fn5+DQo+IGNvbXBpbGF0aW9uIHRlcm1pbmF0ZWQuDQo+IHNjcmlwdHMvTWFrZWZpbGUuYnVp
bGQ6Mjc3OiByZWNpcGUgZm9yIHRhcmdldCAnZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0
dWIubycgZmFpbGVkDQo+DQo+DQo+IEkgaW1hZ2luZSB0aGF0IHRoZSBzZWNvbmQgcGF0Y2ggaXMg
cmVxdWlyZWQgZm9yIHRoZSBidWlsZCBvbiBBUk0uDQpBaCwgeWVzLiBUaGF0IGlzIHdoeSBpdCB3
YXMgdGhlIGZpcnN0DQo+ICAgSQ0KPiBzdWdnZXN0IHRvIG1vdmUgdGhlIGRlZmluaXRpb24gb2Yg
WEVOX1BDSURFVl9TVFVCIHRvIHRoZSBzZWNvbmQgcGF0Y2guDQo+IChrZWVwIFhFTl9QQ0lfU1RV
QiBhbmQgeGVuX3BjaWJrX3B2X3N1cHBvcnQgaGVyZS4pDQpJJ2xsIGdvIHdpdGggdGhpcw0KPg0K
PiBBbHRlcm5hdGl2ZWx5LCBqdXN0IHJlbW92ZSAidHJpc3RhdGUgIlhlbiBQQ0ktZGV2aWNlIHN0
dWIgZHJpdmVyIiIgZnJvbQ0KPiBoZXJlIHNvIHRoYXQgWEVOX1BDSURFVl9TVFVCIGJlY29tZXMg
bm9uLXNlbGVjdGFibGUuIFlvdSBjYW4gYWRkIHRoZQ0KPiB0cmlzdGF0ZSBwcm9wZXJ0eSBpbiBw
YXRjaCAjMi4NCkl0IGxvb2tzIGxpa2UgaGFsZi1iYWtlZC4uLg0KPg0KPg0KPg0KPj4gICBjb25m
aWcgWEVOX1BDSURFVl9CQUNLRU5EDQo+PiAgIAl0cmlzdGF0ZSAiWGVuIFBDSS1kZXZpY2UgYmFj
a2VuZCBkcml2ZXIiDQo+PiAgIAlkZXBlbmRzIG9uIFBDSSAmJiBYODYgJiYgWEVODQo+PiAgIAlk
ZXBlbmRzIG9uIFhFTl9CQUNLRU5EDQo+PiArCXNlbGVjdCBYRU5fUENJX1NUVUINCj4+ICAgCWRl
ZmF1bHQgbQ0KPj4gICAJaGVscA0KPj4gICAJICBUaGUgUENJIGRldmljZSBiYWNrZW5kIGRyaXZl
ciBhbGxvd3MgdGhlIGtlcm5lbCB0byBleHBvcnQgYXJiaXRyYXJ5DQo+PiBkaWZmIC0tZ2l0IGEv
ZHJpdmVycy94ZW4vTWFrZWZpbGUgYi9kcml2ZXJzL3hlbi9NYWtlZmlsZQ0KPj4gaW5kZXggMzQz
NDU5MzQ1NWIyLi41YWFlNjZlNjM4YTcgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi9NYWtl
ZmlsZQ0KPj4gKysrIGIvZHJpdmVycy94ZW4vTWFrZWZpbGUNCj4+IEBAIC0yNCw3ICsyNCw3IEBA
IG9iai0kKENPTkZJR19YRU5fU1lTX0hZUEVSVklTT1IpCSs9IHN5cy1oeXBlcnZpc29yLm8NCj4+
ICAgb2JqLSQoQ09ORklHX1hFTl9QVkhWTV9HVUVTVCkJCSs9IHBsYXRmb3JtLXBjaS5vDQo+PiAg
IG9iai0kKENPTkZJR19TV0lPVExCX1hFTikJCSs9IHN3aW90bGIteGVuLm8NCj4+ICAgb2JqLSQo
Q09ORklHX1hFTl9NQ0VfTE9HKQkJKz0gbWNlbG9nLm8NCj4+IC1vYmotJChDT05GSUdfWEVOX1BD
SURFVl9CQUNLRU5EKQkrPSB4ZW4tcGNpYmFjay8NCj4+ICtvYmotJChDT05GSUdfWEVOX1BDSV9T
VFVCKQkgICAgICAgICs9IHhlbi1wY2liYWNrLw0KPj4gICBvYmotJChDT05GSUdfWEVOX1BSSVZD
TUQpCQkrPSB4ZW4tcHJpdmNtZC5vDQo+PiAgIG9iai0kKENPTkZJR19YRU5fQUNQSV9QUk9DRVNT
T1IpCSs9IHhlbi1hY3BpLXByb2Nlc3Nvci5vDQo+PiAgIG9iai0kKENPTkZJR19YRU5fRUZJKQkJ
CSs9IGVmaS5vDQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZp
bGUgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4gaW5kZXggZThkOTgxZDQz
MjM1Li5lMmNiMzc2NDQ0YTYgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9NYWtlZmlsZQ0KPj4gKysrIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4+
IEBAIC0xLDUgKzEsNiBAQA0KPj4gICAjIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4w
DQo+PiAgIG9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNrLm8N
Cj4+ICtvYmotJChDT05GSUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+PiAg
IA0KPj4gICB4ZW4tcGNpYmFjay15IDo9IHBjaV9zdHViLm8gcGNpYmFja19vcHMubyB4ZW5idXMu
bw0KPj4gICB4ZW4tcGNpYmFjay15ICs9IGNvbmZfc3BhY2UubyBjb25mX3NwYWNlX2hlYWRlci5v
IFwNCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2liYWNrLmggYi9k
cml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2liYWNrLmgNCj4+IGluZGV4IDk1ZTI4ZWU0OGQ1Mi4u
OWE2NDE5NmU4MzFkIDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNp
YmFjay5oDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2liYWNrLmgNCj4+IEBA
IC03MSw2ICs3MSwxMSBAQCBzdHJ1Y3QgcGNpX2RldiAqcGNpc3R1Yl9nZXRfcGNpX2RldihzdHJ1
Y3QgeGVuX3BjaWJrX2RldmljZSAqcGRldiwNCj4+ICAgCQkJCSAgICBzdHJ1Y3QgcGNpX2RldiAq
ZGV2KTsNCj4+ICAgdm9pZCBwY2lzdHViX3B1dF9wY2lfZGV2KHN0cnVjdCBwY2lfZGV2ICpkZXYp
Ow0KPj4gICANCj4+ICtzdGF0aWMgaW5saW5lIGJvb2wgeGVuX3BjaWJrX3B2X3N1cHBvcnQodm9p
ZCkNCj4+ICt7DQo+PiArCXJldHVybiBJU19FTkFCTEVEKENPTkZJR19YRU5fUENJREVWX0JBQ0tF
TkQpOw0KPj4gK30NCj4+ICsNCj4+ICAgLyogRW5zdXJlIGEgZGV2aWNlIGlzIHR1cm5lZCBvZmYg
b3IgcmVzZXQgKi8NCj4+ICAgdm9pZCB4ZW5fcGNpYmtfcmVzZXRfZGV2aWNlKHN0cnVjdCBwY2lf
ZGV2ICpwZGV2KTsNCj4+ICAgDQo+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2sveGVuYnVzLmMgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gaW5kZXgg
YzA5YzdlYmQ2OTY4Li5mOGJhMjkwM2EzZmYgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gKysrIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVu
YnVzLmMNCj4+IEBAIC03NDMsNiArNzQzLDkgQEAgY29uc3Qgc3RydWN0IHhlbl9wY2lia19iYWNr
ZW5kICpfX3JlYWRfbW9zdGx5IHhlbl9wY2lia19iYWNrZW5kOw0KPj4gICANCj4+ICAgaW50IF9f
aW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+PiAgIHsNCj4+ICsJaWYgKCF4
ZW5fcGNpYmtfcHZfc3VwcG9ydCgpKQ0KPj4gKwkJcmV0dXJuIDA7DQo+PiArDQo+PiAgIAl4ZW5f
cGNpYmtfYmFja2VuZCA9ICZ4ZW5fcGNpYmtfdnBjaV9iYWNrZW5kOw0KPj4gICAJaWYgKHBhc3N0
aHJvdWdoKQ0KPj4gICAJCXhlbl9wY2lia19iYWNrZW5kID0gJnhlbl9wY2lia19wYXNzdGhyb3Vn
aF9iYWNrZW5kOw0KPj4gQEAgLTc1Miw1ICs3NTUsNiBAQCBpbnQgX19pbml0IHhlbl9wY2lia194
ZW5idXNfcmVnaXN0ZXIodm9pZCkNCj4+ICAgDQo+PiAgIHZvaWQgX19leGl0IHhlbl9wY2lia194
ZW5idXNfdW5yZWdpc3Rlcih2b2lkKQ0KPj4gICB7DQo+PiAtCXhlbmJ1c191bnJlZ2lzdGVyX2Ry
aXZlcigmeGVuX3BjaWJrX2RyaXZlcik7DQo+PiArCWlmICh4ZW5fcGNpYmtfcHZfc3VwcG9ydCgp
KQ0KPj4gKwkJeGVuYnVzX3VucmVnaXN0ZXJfZHJpdmVyKCZ4ZW5fcGNpYmtfZHJpdmVyKTsNCj4+
ICAgfQ0KPj4gLS0gDQo+PiAyLjI1LjENCj4+


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 05:46:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 05:46:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194936.347334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTe2V-000407-4N; Fri, 24 Sep 2021 05:46:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194936.347334; Fri, 24 Sep 2021 05:46:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTe2U-000400-Vy; Fri, 24 Sep 2021 05:46:26 +0000
Received: by outflank-mailman (input) for mailman id 194936;
 Fri, 24 Sep 2021 05:46:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Coby=OO=epam.com=prvs=9901c588ba=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTe2T-0003zu-NU
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 05:46:25 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 60ec728a-8fb1-4c68-b210-5aa8e684f043;
 Fri, 24 Sep 2021 05:46:24 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18O5h71n021037; 
 Fri, 24 Sep 2021 05:46:22 GMT
Received: from eur02-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2057.outbound.protection.outlook.com [104.47.6.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3b93ew8fnt-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 05:46:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 05:46:17 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 05:46:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60ec728a-8fb1-4c68-b210-5aa8e684f043
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=StAjYoRgSRCuCWSaTtSkTSsKfTcsZ3xwB9GUBdNnwIb6n40pNGA2s6DNx+W8iSZQ2rILQyxJVR9bAdgg1fDNGs5UovC6W7RVRdZWh8WwjkYpdOeebfuPXLc2Tz0bWSujYQGR5fYKKlNfzgzLDISRJmYqqIElyCqSqjYJiYATTbMmCm8ni1hPA0hOwHYH244U04RiD2lRb99i2hqoVFY6ByRvq1k17my9jq7pgp5WPfyN6Ns13K7MiN9L59wOcydR02zCKSJabFKwON0eNXjxBWKR3lrMJrOHXN9pITYNzscrjNQ1lDt2H/4zRN10U2RlJOe/AaGgwkbd32qrQh1lzw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Jpe7EPQrnWyIZKlD1EhC2b9CiuDNDW5A1m+zCwTvWTA=;
 b=Oumx0NFEp13U+eu1pVjaDydZAAjf4iXT0nYzo0T1CFAJ16A5xODRGm948Bs8YNIjC9ICS0S74IUluaTt501IHjO4XgGP2HfySKUqzqTtPxAkpmMl1aaZCDkHpWPod+jeVPtU5R9J1d4YCAqOBrmll9gZlLS1wQwizMWWVLmGHHujxWqhtP2k+CzRiqX7Z+49xmrfwK2NooRCNcxHFXjJrldK1rrsiosp2F1teBxJKorgn+dFtngLcn1p5VZdnuDKyhoAOzGdyW52iNHKV0q75/GJJI5PoCO6wwD1G8hEEzZg95WyqH/Y28ISzlXSsD76s+IpfqOhMNa9DcHS7mdPTQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Jpe7EPQrnWyIZKlD1EhC2b9CiuDNDW5A1m+zCwTvWTA=;
 b=QQ5q+aNGE9BrL7sSHi7OzTuWgrGEWnCyd/OL0pBZT7P02MTWVKKy5qnY5ogzV6MytTbUOPC5iAom0FVi/bM5svEm/Dj0dGpM+0HqMH+zQnN3C5ypcG/VE7xNL+nxqiNKEJGiBZIARSCTisJAJZqOyRwpAqx+8UmoMTrkD50j4/ohBLxFWGOJsuLDI7UaH4nApYVBno+vnoZpWG890xn5p3IN0tWujZksAcMbAqoKRjL7OqsumNM+9/i4GN9fKW7osX6evDgDZ3GoCZJ1lOhI/cUrTzmEVxU3xBd59zHER2D2ac81LhAETPHQ7jwmPGYzfETR0mJd2PE788fCTer2pQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com"
	<jgross@suse.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>,
        Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Topic: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Index: AQHXsGDrkewW5NG+ZkqXauJ2P2aIyauyCpkAgACjoYA=
Date: Fri, 24 Sep 2021 05:46:17 +0000
Message-ID: <f62a1e2c-4253-c998-c206-6bb0681a84fb@epam.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <20210923095345.185489-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 81f8a3e1-5c7e-4654-4a7c-08d97f1ea0fd
x-ms-traffictypediagnostic: AM0PR03MB6324:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB632464C25E505BA7024890ECE7A49@AM0PR03MB6324.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7691;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 EduD81sEApTcpqS7L/1WG6HOW9HR6eyJZwOdBJfexhT7aTqMNCdhCzOYSQ9KNLDzWTykouw6Ge/WWWYpJ+ZiiJQpqmve5j9pd68nf/2T1CkTOHMRHRvLS3LVUng9s6hG1Ef/FNwLD3jUnZUk51H1KvaKlU58mWkzVe8E4hidHOi59nmpybrsXmfuXckGJMUqo4GDO/Jwf/3ZlYF4cBzfcMAM9+X4Wmg2AlhE8T1QMR745C/zREXrLpKgbYtt32yHjEPuNIeSFlGfHXEWQydbB+LgiTd+Xwir6IC+mNYNB8+9Gdbdo0TFoZxWejl+OTZB+drm5fJxjR6fn9JnzH1F7o3bjVBiEZEYEsvdWODMGKJXWw2hPfXtiskz7HlBNU9RJPgjNR1JDz0ng7DzfnhkqZT8q42FW5RxiDuQSUrnzn/871bFMz0hwAkwa3Dm7rws0eM26koaqpLkKveYvCyhz1Hk6RogenEB9GAnYaCI8dLLEWpmJCwj68ynQV8jP6AImkGyc8y9BABOZZZFxrN3KtRsV8q890RZqXS0zlsWv564XzhICZ2oheHclOfF9kIPvzYMCXnO6AaYWXJK8xJao4cPPpGiYiizTYA91HUxoryWTDlarBmlAC19dkbrOvOXVlwXF3rahOWSnwzlcbhWXc9ZdaBsmjxBE/09Zw78rmNYqZIjufiB0hWg+aS5TA2Ayd8NULbh3q2RJYxVEMwqk9CZCAG73g2VE76f05d1Ly4tmdCNPq7RoQu7Wjlvp60p
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(186003)(107886003)(83380400001)(26005)(6512007)(6506007)(86362001)(53546011)(36756003)(122000001)(38100700002)(31696002)(5660300002)(54906003)(55236004)(2616005)(6486002)(8936002)(508600001)(2906002)(66476007)(110136005)(316002)(4326008)(66446008)(91956017)(71200400001)(38070700005)(66946007)(66556008)(64756008)(31686004)(76116006)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aUFEUWpxcXZFUGxsb2JXQk9VbzNSallqcEZIeGVTOVoyVHBlNEhwTnMxR25v?=
 =?utf-8?B?Tm9PZTJQYVBlQ1N5YnovOXExMmZ6R2FiM2ZvcFBZL2hkdTVVblR5RzlzcnV3?=
 =?utf-8?B?UjVWNFEwRTBFV0ZvMEEvaFo5V2lVY204Tm1XTEJweGRjcTY5TzhuWGpLOHhB?=
 =?utf-8?B?Z1ZEZXJxV0ZuQ2JUMG14UVMwUHNrRzFGbUtWRlpaSXdJWW9sSUNsRlN0S3FF?=
 =?utf-8?B?MVYrTXpiVk9KUzdnQjRBNHE2azJzcEYyNWJQQ21NYUlnOWZXVEcxL045Z2I3?=
 =?utf-8?B?ZWZaN01DZTdYaVVGVDlJMUlTTkRXRUNqeGRWWmcxVU1oaGdIUGMvbXo1ZU43?=
 =?utf-8?B?N0tMNHg5SUJrNHp2d3ZVYmxtUjEySHdZV1ZtclVPUTdxSDRvQ2lFeUdtbVNY?=
 =?utf-8?B?UG9HWWtISU9wTW5qQWRPbzZPc21xdEJoZWpzWnhwSEtCYmRFNmFsaVJlcC9Q?=
 =?utf-8?B?SFRuS3JMei9TaVhBQmhDdUV1MEQwTEgxMyszK01RciszRDk5dmI0MmVMUCtZ?=
 =?utf-8?B?cUMyZ0NSUFNhcnRsTExadkt1b1dHejZaVzBSUmw2azVtdWZIRUlHdWRCbDN0?=
 =?utf-8?B?eUlhK2dmVTRibDBENWRlVzB6RU5yQURCbW5lTlI4bzJ3SE1BakRFTzg4Z1RT?=
 =?utf-8?B?eCtCWG5oOUw2K2lHc0Q3TnhxdTlzeWxhVndORDQ3elp4dVRIcTF6NC9JRlAv?=
 =?utf-8?B?M2t2eGhnU05FVFg5Z2U2RGxvZE1wZVU2NTYyVU50WDRQVXNrWmRibWordmMv?=
 =?utf-8?B?K2VaR28wU0UwcEVOcDJOajVlNmhFTDRlemo0Q3FXdjJ4OFhLeEJnTFArNjBG?=
 =?utf-8?B?VmRrWGE1S1d0ZnNKd2JhN1NUTWdLTE4rd1ZZSVB1eFZpdXB5cWFFU1lsb3Fy?=
 =?utf-8?B?N1pDbjAzcU1KSCt3RDJoVURaS0VKU0NGQzR1OEdPRkJNbVptMjRYN1MrK3l5?=
 =?utf-8?B?QTZrTVpxRWU5MllGRWpaV3pEWG0zd0lvOGsweXJYcXlERDdoODQvQ05LKzMv?=
 =?utf-8?B?TDcvOWdYRndnbEtDTXdBYXI2YWRVTW02VGZMRDNhcmJSbW13OVVjdFR2cTc2?=
 =?utf-8?B?NmJWSXBTM08zcGxpNUdXRTN1Y1gzME5QbUVEY21LMGxMQys3d3RNb0ZsbllW?=
 =?utf-8?B?YnI0cTBCU3gzT0QwS1JaWWx0ZTl1emM2M0dTZWMvVFVPVmNnQkpJL0R0TExP?=
 =?utf-8?B?ZXZVMFRqaUtveG5HMVVUM0V1eVo4YmkyVHZjOGZJUm43dGRubWVpSWFoRU1T?=
 =?utf-8?B?MTZWVndEZS9JcWdWeWtnUWJ6cUZEeHpmYzJoUGZqMVhiZ0RYcGVzenZRR1hJ?=
 =?utf-8?B?S3RHZHdtQTZwN1VQNXI2RHUydEZpZnAyK0lDZy9iUDE4OHJ0cnFwbkcvMXNt?=
 =?utf-8?B?Nm5VdWtqZ3VhREZKcXFINjdKSll3SHcyUnJHRFFWYW05WUhiSWdlMEVGUmhS?=
 =?utf-8?B?WFRNeE5iWmdrY1gxb0R0RUIyb1NRbkVTdjBXQ2I3QmxTNjd5SThwSzQ3Vkwy?=
 =?utf-8?B?blB4aDFZTDlLc2RLZzZUUzFRM3NXU1hlbmtMT01Vd0VKUm1CWEVHS2lHT3BY?=
 =?utf-8?B?NU95dHJ2L2phVS9pYk1pUzZLa0VwRmJtYkF0Q2xyOERtdC9Jck5JSGxLY0N1?=
 =?utf-8?B?OVVFZ1dya0YwbXQxWmo3dDVBb2NMNk9Kd284OWJHQWcxd3FNcENLbUphZUZ2?=
 =?utf-8?B?eDZERzViR2tBRmNCV1FzR3A4WHFQcVI2bHRqeG44K0xuZ1g1aHRzdDBUV2dq?=
 =?utf-8?B?VlRrZ211VWQzZGFKbUM1TGhNR3M0S1M3ekM4aUhVQTA1dUk5ZytES1kwcWhv?=
 =?utf-8?B?RlB2eDVKdmF4N056Rllsdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <523ED06AE8C3D54A9AE42A40864A1494@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 81f8a3e1-5c7e-4654-4a7c-08d97f1ea0fd
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 05:46:17.5988
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Q8Fwd9cw6xHMSj/b0UFnrK990fLNxtM4DMCEvpHmT0CB+B5PtIfGYQSrgUlvXxxseABljwIJKqhkHQGdnPShib+E6cOS6lImRCw6YL1/WJV/XO/bjIaHZ7H4saVPcSJ2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6324
X-Proofpoint-GUID: HrShrPtJuHMBhGLoiONxBA_6CJ5DtMOt
X-Proofpoint-ORIG-GUID: HrShrPtJuHMBhGLoiONxBA_6CJ5DtMOt
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-24_01,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 impostorscore=0
 clxscore=1015 bulkscore=0 adultscore=0 spamscore=0 phishscore=0 mlxscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109240033

DQpPbiAyMy4wOS4yMSAyMzowMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBYZW4tcGNpYmFjayBkcml2ZXIgd2FzIGRlc2lnbmVkIHRvIGJlIGJ1aWx0IGZvciB4
ODYgb25seS4gQnV0IGl0DQo+PiBjYW4gYWxzbyBiZSB1c2VkIGJ5IG90aGVyIGFyY2hpdGVjdHVy
ZXMsIGUuZy4gQXJtLg0KPj4gUmUtc3RydWN0dXJlIHRoZSBkcml2ZXIgaW4gYSB3YXkgdGhhdCBp
dCBjYW4gYmUgYnVpbHQgZm9yIG90aGVyDQo+PiBwbGF0Zm9ybXMgYXMgd2VsbC4NCj4+DQo+PiBT
aWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hj
aGVua29AZXBhbS5jb20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBBbmFzdGFzaWlhIEx1a2lhbmVua28g
PGFuYXN0YXNpaWFfbHVraWFuZW5rb0BlcGFtLmNvbT4NCj4gVGhlIHBhdGNoIGxvb2tzIGdvb2Qg
dG8gbWUuIE9ubHkgb25lIHRoaW5nOiBvbiBBUk0zMiBJIGdldDoNCldFIGRvIG5vdCB5ZXQgc3Vw
cG9ydCBYZW4gUENJIHBhc3N0aHJvdWdoIGZvciBBUk0zMg0KPg0KPiBkcml2ZXJzL3hlbi94ZW4t
cGNpYmFjay9jb25mX3NwYWNlX2hlYWRlci5jOiBJbiBmdW5jdGlvbiDigJhiYXJfaW5pdOKAmToN
Cj4gZHJpdmVycy94ZW4veGVuLXBjaWJhY2svY29uZl9zcGFjZV9oZWFkZXIuYzoyMzk6MzQ6IHdh
cm5pbmc6IHJpZ2h0IHNoaWZ0IGNvdW50ID49IHdpZHRoIG9mIHR5cGUgWy1Xc2hpZnQtY291bnQt
b3ZlcmZsb3ddDQo+ICAgICAgYmFyLT52YWwgPSByZXNbcG9zIC0gMV0uc3RhcnQgPj4gMzI7DQo+
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4NCj4gZHJpdmVycy94ZW4veGVu
LXBjaWJhY2svY29uZl9zcGFjZV9oZWFkZXIuYzoyNDA6NDk6IHdhcm5pbmc6IHJpZ2h0IHNoaWZ0
IGNvdW50ID49IHdpZHRoIG9mIHR5cGUgWy1Xc2hpZnQtY291bnQtb3ZlcmZsb3ddDQo+ICAgICAg
YmFyLT5sZW5fdmFsID0gLXJlc291cmNlX3NpemUoJnJlc1twb3MgLSAxXSkgPj4gMzI7DQo+ICAg
DQo+ICAgDQo+IHJlc291cmNlX3NpemVfdCBpcyBkZWZpbmVkIGFzIHBoeXNfYWRkcl90IGFuZCBp
dCBjYW4gYmUgMzJiaXQgb24gYXJtMzIuDQo+DQo+DQo+IE9uZSBmaXggaXMgdG8gc3Vycm91bmQ6
DQo+DQo+IAkJaWYgKHBvcyAmJiAocmVzW3BvcyAtIDFdLmZsYWdzICYgSU9SRVNPVVJDRV9NRU1f
NjQpKSB7DQo+IAkJCWJhci0+dmFsID0gcmVzW3BvcyAtIDFdLnN0YXJ0ID4+IDMyOw0KPiAJCQli
YXItPmxlbl92YWwgPSAtcmVzb3VyY2Vfc2l6ZSgmcmVzW3BvcyAtIDFdKSA+PiAzMjsNCj4gCQkJ
cmV0dXJuIGJhcjsNCj4gCQl9DQo+DQo+IHdpdGggI2lmZGVmIFBIWVNfQUREUl9UXzY0QklUDQo+
DQpUaGlzIG1pZ2h0IG5vdCBiZSBjb3JyZWN0LiBXZSBhcmUgZGVhbGluZyBoZXJlIHdpdGggYSA2
NC1iaXQgQkFSIG9uIGEgMzItYml0IE9TLg0KDQpJIHRoaW5rIHRoYXQgdGhpcyBjYW4gc3RpbGwg
YmUgdmFsaWQgdXNlLWNhc2UgaWYgQkFSNjQuaGkgPT0gMC4gU28sIG5vdCBzdXJlDQoNCndlIGNh
biBqdXN0IHNraXAgaXQgd2l0aCBpZmRlZi4NCg0KSW5zdGVhZCwgdG8gYmUgb24gdGhlIHNhZmUg
c2lkZSwgd2UgY2FuIGhhdmU6DQoNCmNvbmZpZyBYRU5fUENJREVWX1NUVUINCiDCoMKgwqDCoMKg
wqAgdHJpc3RhdGUgIlhlbiBQQ0ktZGV2aWNlIHN0dWIgZHJpdmVyIg0KIMKgwqDCoMKgwqDCoCBk
ZXBlbmRzIG9uIFBDSSAmJiBBUk02NCAmJiBYRU4NCmUuZy4gb25seSBhbGxvdyBidWlsZGluZyB0
aGUgInN0dWIiIGZvciBBUk02NCBmb3Igbm93Lg0KDQoNCj4NCj4+IC0tLQ0KPj4gU2luY2UgdjI6
DQo+PiAgIC0gc3dhcCB0aGUgcGF0Y2ggb3JkZXINCj4+IFNpbmNlIHYxOg0KPj4gICAtIERvIG5v
dCBtb3ZlIHBjaV94ZW5faW5pdGlhbF9kb21haW4gYXMgaXQgaXMgeDg2IHNwZWNpZmljDQo+PiAt
LS0NCj4+ICAgYXJjaC94ODYvaW5jbHVkZS9hc20veGVuL3BjaS5oICAgICB8IDE4ICstLS0tLS0N
Cj4+ICAgYXJjaC94ODYvcGNpL3hlbi5jICAgICAgICAgICAgICAgICB8IDc0ICstLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tDQo+PiAgIGRyaXZlcnMveGVuL3BjaS5jICAgICAgICAgICAgICAg
ICAgfCA3NSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+ICAgZHJpdmVycy94ZW4v
eGVuLXBjaWJhY2svcGNpX3N0dWIuYyB8ICAzICstDQo+PiAgIGRyaXZlcnMveGVuL3hlbi1wY2li
YWNrL3hlbmJ1cy5jICAgfCAgMiArLQ0KPj4gICBpbmNsdWRlL3hlbi9wY2kuaCAgICAgICAgICAg
ICAgICAgIHwgMjggKysrKysrKysrKysNCj4+ICAgNiBmaWxlcyBjaGFuZ2VkLCAxMDggaW5zZXJ0
aW9ucygrKSwgOTIgZGVsZXRpb25zKC0pDQo+PiAgIGNyZWF0ZSBtb2RlIDEwMDY0NCBpbmNsdWRl
L3hlbi9wY2kuaA0KPj4NCj4+IGRpZmYgLS1naXQgYS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4v
cGNpLmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgNCj4+IGluZGV4IDM1MDZkOGM1
OThjMS4uMjg4OWYwOTFmNDU5IDEwMDY0NA0KPj4gLS0tIGEvYXJjaC94ODYvaW5jbHVkZS9hc20v
eGVuL3BjaS5oDQo+PiArKysgYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vcGNpLmgNCj4+IEBA
IC0xNiwyNiArMTYsMTAgQEAgc3RhdGljIGlubGluZSBpbnQgcGNpX3hlbl9odm1faW5pdCh2b2lk
KQ0KPj4gICAjZW5kaWYNCj4+ICAgI2lmIGRlZmluZWQoQ09ORklHX1hFTl9ET00wKQ0KPj4gICBp
bnQgX19pbml0IHBjaV94ZW5faW5pdGlhbF9kb21haW4odm9pZCk7DQo+PiAtaW50IHhlbl9maW5k
X2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAtaW50IHhlbl9y
ZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsIHVpbnQxNl90
IGRvbWFpbik7DQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3Ry
dWN0IHBjaV9kZXYgKmRldik7DQo+PiAgICNlbHNlDQo+PiAgIHN0YXRpYyBpbmxpbmUgaW50IF9f
aW5pdCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpDQo+PiAgIHsNCj4+IC0JcmV0dXJuIC0x
Ow0KPj4gLX0NCj4+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293
bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLQlyZXR1cm4gLTE7DQo+PiAtfQ0K
Pj4gLXN0YXRpYyBpbmxpbmUgaW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0
cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiAtCQkJCQkJICAgdWludDE2X3QgZG9tYWluKQ0KPj4gLXsN
Cj4+IC0JcmV0dXJuIC0xOw0KPj4gLX0NCj4+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fdW5yZWdp
c3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAtew0KPj4g
LQlyZXR1cm4gLTE7DQo+PiArICAgICAgIHJldHVybiAtMTsNCj4+ICAgfQ0KPj4gICAjZW5kaWYN
Cj4+ICAgDQo+PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYvcGNpL3hlbi5jIGIvYXJjaC94ODYvcGNp
L3hlbi5jDQo+PiBpbmRleCAzZDQxYTA5YzJjMTQuLjRhNDViMGJmOWFlNCAxMDA2NDQNCj4+IC0t
LSBhL2FyY2gveDg2L3BjaS94ZW4uYw0KPj4gKysrIGIvYXJjaC94ODYvcGNpL3hlbi5jDQo+PiBA
QCAtMjMsNiArMjMsNyBAQA0KPj4gICANCj4+ICAgI2luY2x1ZGUgPHhlbi9mZWF0dXJlcy5oPg0K
Pj4gICAjaW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+
PiAgICNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL3hlbi9jcHVp
ZC5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL2FwaWMuaD4NCj4+IEBAIC01ODMsNzcgKzU4NCw0IEBA
IGludCBfX2luaXQgcGNpX3hlbl9pbml0aWFsX2RvbWFpbih2b2lkKQ0KPj4gICAJfQ0KPj4gICAJ
cmV0dXJuIDA7DQo+PiAgIH0NCj4+IC0NCj4+IC1zdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3du
ZXIgew0KPj4gLQlkb21pZF90IGRvbWFpbjsNCj4+IC0Jc3RydWN0IHBjaV9kZXYgKmRldjsNCj4+
IC0Jc3RydWN0IGxpc3RfaGVhZCBsaXN0Ow0KPj4gLX07DQo+PiAtDQo+PiAtc3RhdGljIERFRklO
RV9TUElOTE9DSyhkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLXN0YXRpYyBzdHJ1Y3Qg
bGlzdF9oZWFkIGRldl9kb21haW5fbGlzdCA9IExJU1RfSEVBRF9JTklUKGRldl9kb21haW5fbGlz
dCk7DQo+PiAtDQo+PiAtc3RhdGljIHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqZmlu
ZF9kZXZpY2Uoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXN0cnVjdCB4ZW5fZGV2
aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+PiAtDQo+PiAtCWxpc3RfZm9yX2VhY2hfZW50cnko
b3duZXIsICZkZXZfZG9tYWluX2xpc3QsIGxpc3QpIHsNCj4+IC0JCWlmIChvd25lci0+ZGV2ID09
IGRldikNCj4+IC0JCQlyZXR1cm4gb3duZXI7DQo+PiAtCX0NCj4+IC0JcmV0dXJuIE5VTEw7DQo+
PiAtfQ0KPj4gLQ0KPj4gLWludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBw
Y2lfZGV2ICpkZXYpDQo+PiAtew0KPj4gLQlzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIg
Km93bmVyOw0KPj4gLQlpbnQgZG9tYWluID0gLUVOT0RFVjsNCj4+IC0NCj4+IC0Jc3Bpbl9sb2Nr
KCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQlvd25lciA9IGZpbmRfZGV2aWNlKGRl
dik7DQo+PiAtCWlmIChvd25lcikNCj4+IC0JCWRvbWFpbiA9IG93bmVyLT5kb21haW47DQo+PiAt
CXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQlyZXR1cm4gZG9t
YWluOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fZmluZF9kZXZpY2VfZG9tYWlu
X293bmVyKTsNCj4+IC0NCj4+IC1pbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIo
c3RydWN0IHBjaV9kZXYgKmRldiwgdWludDE2X3QgZG9tYWluKQ0KPj4gLXsNCj4+IC0Jc3RydWN0
IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4+IC0NCj4+IC0Jb3duZXIgPSBremFs
bG9jKHNpemVvZihzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIpLCBHRlBfS0VSTkVMKTsN
Cj4+IC0JaWYgKCFvd25lcikNCj4+IC0JCXJldHVybiAtRU5PREVWOw0KPj4gLQ0KPj4gLQlzcGlu
X2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtCWlmIChmaW5kX2RldmljZShk
ZXYpKSB7DQo+PiAtCQlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+
IC0JCWtmcmVlKG93bmVyKTsNCj4+IC0JCXJldHVybiAtRUVYSVNUOw0KPj4gLQl9DQo+PiAtCW93
bmVyLT5kb21haW4gPSBkb21haW47DQo+PiAtCW93bmVyLT5kZXYgPSBkZXY7DQo+PiAtCWxpc3Rf
YWRkX3RhaWwoJm93bmVyLT5saXN0LCAmZGV2X2RvbWFpbl9saXN0KTsNCj4+IC0Jc3Bpbl91bmxv
Y2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAtCXJldHVybiAwOw0KPj4gLX0NCj4+
IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fcmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcik7DQo+
PiAtDQo+PiAtaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBj
aV9kZXYgKmRldikNCj4+IC17DQo+PiAtCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAq
b3duZXI7DQo+PiAtDQo+PiAtCXNwaW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsN
Cj4+IC0Jb3duZXIgPSBmaW5kX2RldmljZShkZXYpOw0KPj4gLQlpZiAoIW93bmVyKSB7DQo+PiAt
CQlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+IC0JCXJldHVybiAt
RU5PREVWOw0KPj4gLQl9DQo+PiAtCWxpc3RfZGVsKCZvd25lci0+bGlzdCk7DQo+PiAtCXNwaW5f
dW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4gLQlrZnJlZShvd25lcik7DQo+
PiAtCXJldHVybiAwOw0KPj4gLX0NCj4+IC1FWFBPUlRfU1lNQk9MX0dQTCh4ZW5fdW5yZWdpc3Rl
cl9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4+ICAgI2VuZGlmDQo+PiBkaWZmIC0tZ2l0IGEvZHJp
dmVycy94ZW4vcGNpLmMgYi9kcml2ZXJzL3hlbi9wY2kuYw0KPj4gaW5kZXggMjI0ZGYwM2NlNDJl
Li5mYzhjMTI0OWQ0OWYgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi9wY2kuYw0KPj4gKysr
IGIvZHJpdmVycy94ZW4vcGNpLmMNCj4+IEBAIC0yNTQsMyArMjU0LDc4IEBAIHN0YXRpYyBpbnQg
eGVuX21jZmdfbGF0ZSh2b2lkKQ0KPj4gICAJcmV0dXJuIDA7DQo+PiAgIH0NCj4+ICAgI2VuZGlm
DQo+PiArDQo+PiArI2lmZGVmIENPTkZJR19YRU5fRE9NMA0KPj4gK3N0cnVjdCB4ZW5fZGV2aWNl
X2RvbWFpbl9vd25lciB7DQo+PiArCWRvbWlkX3QgZG9tYWluOw0KPj4gKwlzdHJ1Y3QgcGNpX2Rl
diAqZGV2Ow0KPj4gKwlzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7DQo+PiArfTsNCj4+ICsNCj4+ICtz
dGF0aWMgREVGSU5FX1NQSU5MT0NLKGRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArc3Rh
dGljIHN0cnVjdCBsaXN0X2hlYWQgZGV2X2RvbWFpbl9saXN0ID0gTElTVF9IRUFEX0lOSVQoZGV2
X2RvbWFpbl9saXN0KTsNCj4+ICsNCj4+ICtzdGF0aWMgc3RydWN0IHhlbl9kZXZpY2VfZG9tYWlu
X293bmVyICpmaW5kX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gK3sNCj4+ICsJc3Ry
dWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4+ICsNCj4+ICsJbGlzdF9mb3Jf
ZWFjaF9lbnRyeShvd25lciwgJmRldl9kb21haW5fbGlzdCwgbGlzdCkgew0KPj4gKwkJaWYgKG93
bmVyLT5kZXYgPT0gZGV2KQ0KPj4gKwkJCXJldHVybiBvd25lcjsNCj4+ICsJfQ0KPj4gKwlyZXR1
cm4gTlVMTDsNCj4+ICt9DQo+PiArDQo+PiAraW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3du
ZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+ICt7DQo+PiArCXN0cnVjdCB4ZW5fZGV2aWNlX2Rv
bWFpbl9vd25lciAqb3duZXI7DQo+PiArCWludCBkb21haW4gPSAtRU5PREVWOw0KPj4gKw0KPj4g
KwlzcGluX2xvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArCW93bmVyID0gZmlu
ZF9kZXZpY2UoZGV2KTsNCj4+ICsJaWYgKG93bmVyKQ0KPj4gKwkJZG9tYWluID0gb3duZXItPmRv
bWFpbjsNCj4+ICsJc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiAr
CXJldHVybiBkb21haW47DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl9maW5kX2Rl
dmljZV9kb21haW5fb3duZXIpOw0KPj4gKw0KPj4gK2ludCB4ZW5fcmVnaXN0ZXJfZGV2aWNlX2Rv
bWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2LCB1aW50MTZfdCBkb21haW4pDQo+PiArew0K
Pj4gKwlzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPj4gKw0KPj4gKwlv
d25lciA9IGt6YWxsb2Moc2l6ZW9mKHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciksIEdG
UF9LRVJORUwpOw0KPj4gKwlpZiAoIW93bmVyKQ0KPj4gKwkJcmV0dXJuIC1FTk9ERVY7DQo+PiAr
DQo+PiArCXNwaW5fbG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJaWYgKGZp
bmRfZGV2aWNlKGRldikpIHsNCj4+ICsJCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3Bp
bmxvY2spOw0KPj4gKwkJa2ZyZWUob3duZXIpOw0KPj4gKwkJcmV0dXJuIC1FRVhJU1Q7DQo+PiAr
CX0NCj4+ICsJb3duZXItPmRvbWFpbiA9IGRvbWFpbjsNCj4+ICsJb3duZXItPmRldiA9IGRldjsN
Cj4+ICsJbGlzdF9hZGRfdGFpbCgmb3duZXItPmxpc3QsICZkZXZfZG9tYWluX2xpc3QpOw0KPj4g
KwlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4+ICsJcmV0dXJuIDA7
DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWlu
X293bmVyKTsNCj4+ICsNCj4+ICtpbnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25l
cihzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gK3sNCj4+ICsJc3RydWN0IHhlbl9kZXZpY2VfZG9t
YWluX293bmVyICpvd25lcjsNCj4+ICsNCj4+ICsJc3Bpbl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rf
c3BpbmxvY2spOw0KPj4gKwlvd25lciA9IGZpbmRfZGV2aWNlKGRldik7DQo+PiArCWlmICghb3du
ZXIpIHsNCj4+ICsJCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPj4g
KwkJcmV0dXJuIC1FTk9ERVY7DQo+PiArCX0NCj4+ICsJbGlzdF9kZWwoJm93bmVyLT5saXN0KTsN
Cj4+ICsJc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+PiArCWtmcmVl
KG93bmVyKTsNCj4+ICsJcmV0dXJuIDA7DQo+PiArfQ0KPj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhl
bl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIpOw0KPj4gKyNlbmRpZg0KPj4gZGlmZiAt
LWdpdCBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMgYi9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay9wY2lfc3R1Yi5jDQo+PiBpbmRleCBmOGU0ZmFhOTZhZDYuLmJiYTUyNzYyMDUw
NyAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMNCj4+
ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMNCj4+IEBAIC0xOSw3ICsx
OSw4IEBADQo+PiAgICNpbmNsdWRlIDxsaW51eC9zY2hlZC5oPg0KPj4gICAjaW5jbHVkZSA8bGlu
dXgvYXRvbWljLmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4vZXZlbnRzLmg+DQo+PiAtI2luY2x1ZGUg
PGFzbS94ZW4vcGNpLmg+DQo+PiArI2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4+ICsjaW5jbHVkZSA8
eGVuL3hlbi5oPg0KPj4gICAjaW5jbHVkZSA8YXNtL3hlbi9oeXBlcnZpc29yLmg+DQo+PiAgICNp
bmNsdWRlIDx4ZW4vaW50ZXJmYWNlL3BoeXNkZXYuaD4NCj4+ICAgI2luY2x1ZGUgInBjaWJhY2su
aCINCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYyBiL2Ry
aXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1cy5jDQo+PiBpbmRleCBmOGJhMjkwM2EzZmYuLmJk
ZTYzZWY2NzdiOCAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL3hlbmJ1
cy5jDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5idXMuYw0KPj4gQEAgLTE0
LDcgKzE0LDcgQEANCj4+ICAgI2luY2x1ZGUgPGxpbnV4L3dvcmtxdWV1ZS5oPg0KPj4gICAjaW5j
bHVkZSA8eGVuL3hlbmJ1cy5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4gLSNp
bmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPj4gKyNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiAgICNp
bmNsdWRlICJwY2liYWNrLmgiDQo+PiAgIA0KPj4gICAjZGVmaW5lIElOVkFMSURfRVZUQ0hOX0lS
USAgKC0xKQ0KPj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUveGVuL3BjaS5oIGIvaW5jbHVkZS94ZW4v
cGNpLmgNCj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+PiBpbmRleCAwMDAwMDAwMDAwMDAuLmI4
MzM3Y2Y4NWZkMQ0KPj4gLS0tIC9kZXYvbnVsbA0KPj4gKysrIGIvaW5jbHVkZS94ZW4vcGNpLmgN
Cj4+IEBAIC0wLDAgKzEsMjggQEANCj4+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BM
LTIuMCAqLw0KPj4gKw0KPj4gKyNpZm5kZWYgX19YRU5fUENJX0hfXw0KPj4gKyNkZWZpbmUgX19Y
RU5fUENJX0hfXw0KPj4gKw0KPj4gKyNpZiBkZWZpbmVkKENPTkZJR19YRU5fRE9NMCkNCj4+ICtp
bnQgeGVuX2ZpbmRfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2KTsNCj4+
ICtpbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRl
diwgdWludDE2X3QgZG9tYWluKTsNCj4+ICtpbnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFp
bl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2KTsNCj4+ICsjZWxzZQ0KPj4gK3N0YXRpYyBpbmxp
bmUgaW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikN
Cj4+ICt7DQo+PiArCXJldHVybiAtMTsNCj4+ICt9DQo+PiArDQo+PiArc3RhdGljIGlubGluZSBp
bnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwN
Cj4+ICsJCQkJCQkgICB1aW50MTZfdCBkb21haW4pDQo+PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+
PiArfQ0KPj4gKw0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IHhlbl91bnJlZ2lzdGVyX2RldmljZV9k
b21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4+ICt7DQo+PiArCXJldHVybiAtMTsN
Cj4+ICt9DQo+PiArI2VuZGlmDQo+PiArDQo+PiArI2VuZGlmDQo+PiAtLSANCj4+IDIuMjUuMQ==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 06:22:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 06:22:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194943.347345 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTeao-00016G-PT; Fri, 24 Sep 2021 06:21:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194943.347345; Fri, 24 Sep 2021 06:21:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTeao-000169-M4; Fri, 24 Sep 2021 06:21:54 +0000
Received: by outflank-mailman (input) for mailman id 194943;
 Fri, 24 Sep 2021 06:21:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTean-000163-Bv
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 06:21:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b42ec4ef-1cff-11ec-baaa-12813bfff9fa;
 Fri, 24 Sep 2021 06:21:51 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-9-Mw_5A7hPOby7RWzliVmJvQ-1; Fri, 24 Sep 2021 08:21:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2605.eurprd04.prod.outlook.com (2603:10a6:800:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Fri, 24 Sep
 2021 06:21:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 06:21:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0069.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 06:21:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b42ec4ef-1cff-11ec-baaa-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632464511;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KZrB8F6ggnte03TqI3CxjcNTLp1b3ryilcCbfzjkUgA=;
	b=NpJcbwO8ABaMPmn7lnIF3dkcO3sif5q8Tb7EkQEEBqNUZfmr5ReUh0N9OOe4aRv5vynS1T
	gUX1aQG89sa1NJXTqkqoPbAseyTsHWX1LQ6Rh9SpHvjdBrnPU4+4qG09M4qPkZw6o60Xtk
	/K7pYlOde+b1SyEE/8AOXcuAnzX966o=
X-MC-Unique: Mw_5A7hPOby7RWzliVmJvQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UJmFKRb6VQ/3OAfJ3Lu+0jhfnXaIYX0lt7p+4cG4i8IU6clAZI+KyONjVKXmhjx22+ySlKDMQBLEh1QglqogZEtLYq3H3Ju5u8RY0PiBtpAtZwyoQyMKY9UGCGikaB1c80LXBAj8XfiiJXV9h9pQrxd2vKz/qSbczkTcNOumYXHsnoB8HdwIjt+6/3cVvp4xkK09SSwCFU1ky/4K7LAuZ4G7XMyhrhuSCnz/+3v36pFlrTmHQ1jsnyLzTEwyBDQ1tgLKmB1la5xuRL5u0ikokP0xwvkvARgnRf7udJ8nKDdCVVFbIRKnDQH0Lf8Lka3TKHkNkz37fJryWDdiQMh6mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7YEc9SYiPPjqILBDuFB6z1o1QMRBbd1GcrnJgtX+fik=;
 b=KHjAAHEesRJXyIq5+zU1KWib3S1EB4CXFGLdWAR4qg09Cj3gbJ6xJDzPCFfhTyPgq/w7y+Evy2HdsrIL6B2fqn/ugdHtVajVVVKikd9S11dzbJaGH/k6RRjwCTiZGoALMhulzJNoBWo4SaOdY2qX7/kS0s5TgWJB9I+fT5n/3ki9wLgKbsl3enZccrwfZt46pHY15zjLNjpa1mUdMOkPfuqmoS2RwvAUBOurmXf950j+qMN0KDv2O9c+FBNSOMymbAEZkoDNjz6uPqfS3bWSZzt6SjWUI1EktCsjWSUz129FgnIpieV3rx5TgHcySbVEfy0BST//cpMBXkaQ3y/MQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: Julien Grall <julien.grall.oss@gmail.com>
CC: xen-devel <xen-devel@lists.xenproject.org>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
 <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
 <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
 <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d3a314ae-a2bb-58da-b98e-bc03a049bc1d@suse.com>
Date: Fri, 24 Sep 2021 08:21:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR0P281CA0069.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a9b55a3-cd1a-48aa-fe8b-08d97f239684
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26051C7EEB74B0601BEC9E3BB3A49@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IsNUcYDEw8tvL8q25p7u+hRHhYMbBvlIKwyr5WI/CUZ/3QSsr/PbbedwyKNKOv/RzMJdZHmOUcTFqFRgEwBP48NFk373hktTtdt/rsE7RD1oVz/T5rTrbURakDYCmr0PBg15LRkoFcL+EIUrwbOzCqhTkbPt5ET5YuxZqdG/CRd4Nk/FoVt8j+XoCLkDQzqhdVKLmlF5M6tOPAzhMK3KmK/azTPOro7CdnQVDU92VUlZERIKYcQFLK5SUL0p2zlRDd8U99kUUwgkjo9gUKNwiqcm0FUDCrn9lZEm923kDJoxhZ9y/x4rIse6HljbjkFoQCkxtNwZWkA/orBKRUf4wmpSryT+ctx+9IMzAtbtw7OBx+xJiBkqHan5CphHmCYqngkq8EBQfirYo929/vOISbRg0gPWYlkV7s7O+CWpqjBa11BijceedHE19esdf0IANC3944TxZr5jYuvR1QR4XLJaK8adlCHlu1n73jCDvlcrxqOhFaxZ+XlbOJmcfjfDkbSnR/Tj6wsXP+vFIsxRX/+Q91W2qDef46fdI3VCBw0JWzfSvxQtRA41c9zbaO/KkZDB7sH1cre+KEWhF8uehevRKYBhOIV5UYt9WIHQBan1PBmu/LYq27H0NeUc0m34AXGh6wmYfH2vZbR93s9VyRIzrtMDOaqpBLefhK7EkxOPl/JuOS2fxFmWTD6aaBEGFSFCKVFjJ12GJjLfk8Yt5NmbJrgnWkZm5BR2SUtPCAuEXAwIYZiA5caoo86v9KY4inlxKxm851jmIb9Fs4VB2jqk/y1hIPX75xjGjtYYzakSHsXUSVZfvcsjW4UmEY/j
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(6486002)(5660300002)(83380400001)(6916009)(26005)(8936002)(36756003)(7416002)(38100700002)(53546011)(54906003)(31696002)(16576012)(316002)(2906002)(8676002)(31686004)(66556008)(2616005)(186003)(66476007)(508600001)(66946007)(4326008)(956004)(966005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?bK5kJKqX3WS5VTbiZI2dT56NhzkBCtVJNvzE9wplqSNmo5AJPbCKjlUxYXgg?=
 =?us-ascii?Q?mMauf1sfm0pJLoh6CkEw44paJH1zLmKCUIB3SdNgEz8UKlkh7TYc1sZuzhgy?=
 =?us-ascii?Q?qR1//UkkhR6vrn7XswiOE8aw1LXV1kkFhjODO13BliVGD/EAkrAF8jlE76bm?=
 =?us-ascii?Q?C1IikaS9JfeqS8/N9ZQPA3W7/O0fyGyjgha7iwYvxpiK9sn5ZTbzO2HVaooQ?=
 =?us-ascii?Q?HWmZUhirkVDCkAIeY2LnM+3lJK3QBQnkPv/MH5oAxKRToe0BYcUZb3u18mRX?=
 =?us-ascii?Q?6Ae3xSCXxF3BncbDhCEeqXUOVdEHTpvroLR1uS3QzP0TkN7eSfTSs0TyCkzH?=
 =?us-ascii?Q?dPz9OE/3H3XSTOrshD+oEGNz5GuqBWXF93L9RSA7bAHuHN6SUZK4bj115kk2?=
 =?us-ascii?Q?dYGomdQ8L8u9R9u/Uj1TgpzdcoJUarBNt1StXa0YbSpEvIdsq+98CjVKkQkH?=
 =?us-ascii?Q?2rCgUQuprCQ0LE8ibBbd5eEAvJu5pc5HnxIPqxaIEIwbHAgtFXZAL1a/WpW3?=
 =?us-ascii?Q?JbrLU0cdosJfkwQnPvxMYZczhq5MmeW2qsiMkddoB9AhJomD9o3rf+BicNGL?=
 =?us-ascii?Q?nv34Tc9rIToavg834PtQeWTlGjcTSOgB3tn0VjDsIBq8d6gaTV/BY0jco/Yi?=
 =?us-ascii?Q?+2ufG3K90pTJmj22RqkMfWF75NM+D9dJ9fzl+Y7KRdRI+gATAd9FFtu2tH8Z?=
 =?us-ascii?Q?R5MFiQDhMsc9hqkP3v+qW0IQU+AYY81ThoBRIbYwudJCJUpclSGlWyCy/rOl?=
 =?us-ascii?Q?fwBa9kDIH1V05/JKbF1PJVxkRNUUm9uLDCDi5AyQEOIUR3FdNO8N9pIcqr8e?=
 =?us-ascii?Q?U8FiwXU+hsT5qkx0TjXv3f/hrtVUBAqcBJPv1UhftF29q2Uqhp7tEvtSLeA0?=
 =?us-ascii?Q?O23wjZ+DhjLLpQhpmJxtMKv/dx5GhiKh6kBaPDNZvdBVsa1wmiEjq8aC/hUj?=
 =?us-ascii?Q?cxEW5hUpDFH/dLxwESKRuYfS7mvpb8xF22lidHZiSzij6pOWkbxxGaEKHHQw?=
 =?us-ascii?Q?XmT7x9MzrFppHXOS4lSMnK6rFkajhNLnIVon6oYvsIZTpDEDl3cq8gAxIlhS?=
 =?us-ascii?Q?kZZhI/kmF5URPI0uP3Y2UwKorcwo/4SnZqyhA0nYGKMrvSQdqWQPIRlCgNuN?=
 =?us-ascii?Q?C44VifdZTevtbqAuF8XJgoSc5c83BfXpwJ54QZUfZyp0MHOjwippYM3T1DrR?=
 =?us-ascii?Q?TEasCDBrIYPTZOEVGzT+sgn4T0e0wfaHHvxIhuFg2+1O9Gf+ec8aiCprZ7GR?=
 =?us-ascii?Q?aJWJl4GQeHt0Rrx0VEYWL8+83FFx4pO9AImDE52QU/vAGuyawzeQteEwt60l?=
 =?us-ascii?Q?flIaWFSOpI0+Vfu0Mo7GAtj7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a9b55a3-cd1a-48aa-fe8b-08d97f239684
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 06:21:47.7336
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WzOfaURf+fyG6k1GOJdIxnZK0n7zeBkoCv1XV/m/E91DxqXl4WlBc8CFFgph7x5AUL4P+OXR9IyyPCi7AnANVA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 24.09.2021 04:30, Julien Grall wrote:
> Hi Roger,
>=20
> On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=A9, <roger.pau@citrix.com> w=
rote:
>=20
>> On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
>>> Hi Roger,
>>>
>>> On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:
>>>> On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
>>>>>
>>>>>
>>>>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>>>>> Hello,
>>>>>
>>>>> Hi Roger,
>>>>>
>>>>>> First patch on the series is a trivial change to xenconsoled in
>> order to
>>>>>> use xenforeignmemory stable library in order to map the shared
>> console
>>>>>> ring instead of the unstable libxc interface. It's reviewed and
>> ready to
>>>>>> go in.
>>>>>>
>>>>>> Patches 2 and 3 allow setting the host wide command line `gnttab`
>> option
>>>>>> on a per domain basis. That means selecting the max allowed grant
>> table
>>>>>> version and whether transitive grants are allowed.
>>>>>>
>>>>>> The last 3 patches attempt to implement support for creating guests
>>>>>> without a grant table. This requires some changes to xenstored in
>> order
>>>>>> to partially support guests without a valid ring interface, as the
>> lack
>>>>>> of grant table will prevent C xenstored from mapping the shared
>> ring.
>>>>>> Note this is not an issue for Ocaml xenstored, as it still uses the
>>>>>> foreign memory interface to map the shared ring, and thus won't
>> notice
>>>>>> the lack of grant table support on the domain.
>>>>>
>>>>> I find a bit odd that the Xenstore support is conditional to whether
>> grant
>>>>> table is available. Are you expecting domains with no grant table to
>> have no
>>>>> PV drivers (including PV shutdown)?
>>>>
>>>> I don't really expect much, as having guests without grant table is a
>>>> developer option right now, if someone wants to make use of them for
>>>> any reason it would need some thought.
>>>>
>>>> The other option would be my first proposal to restore foreign mapping
>>>> of the xenstore ring on that case:
>>>>
>>>>
>> https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citri=
x.com/
>>>>
>>>> But it's also arguable that a guest not having a grant table should
>>>> also likely prevent foreign mapping attempts. Plus such foreign
>>>> mapping won't work from stubdomains.
>>>
>>> There is another option: extend the acquire hypercall to allow xenstore=
d
>>> domain to map the xenstore interface. This would require more work, but
>> at
>>> least it would avoid the interesting dependency on the grant table.
>>
>> Xen isn't aware of the shared xenstore ring page currently, so that
>> would mean introducing more knowledge to the hypervisor that what's
>> strictly required IMO, as Xen has no business in knowing such details.
>>
>=20
> Well Xen already knows the page for HVM/PVH because the guest retrieve it
> through an HMV param.

To be honest using this in such a way would feel like an abuse / layering
violation to me.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 06:38:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 06:38:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194950.347355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTeql-00034i-Av; Fri, 24 Sep 2021 06:38:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194950.347355; Fri, 24 Sep 2021 06:38:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTeql-00034b-83; Fri, 24 Sep 2021 06:38:23 +0000
Received: by outflank-mailman (input) for mailman id 194950;
 Fri, 24 Sep 2021 06:38:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTeqk-00034U-AK
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 06:38:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 58a4c971-d3e1-4211-9ad1-a9b492028043;
 Fri, 24 Sep 2021 06:38:21 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-LMzXYZvKM1WhegGBGOgIAw-1; Fri, 24 Sep 2021 08:38:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 06:38:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 06:38:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0025.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 06:38:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58a4c971-d3e1-4211-9ad1-a9b492028043
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632465500;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=yeXPBeXfD92VrJgIPKVAT3xAa1YNY34ei/eCe2HJlTM=;
	b=Qd+Qfl8B84s1JT7QcG7HSBlpF08DATImJ5L7n2QkheDfzY6fdmV5y9t7BrY9H9L6ixPxjh
	YGSVxxoz18Vh61bv5euR+uPbnrooKd2MGU94yAnpSo0DmPdCkgCPuggg5QSTfQFkyJokNU
	TDISJrmh3aQt2SviHXCa6xmFiotuyPw=
X-MC-Unique: LMzXYZvKM1WhegGBGOgIAw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dzoYykAEq0N2VnSwePffQr3pnaWiSh3sjqS599GtkTk1npASbuCAJC2UhjDNoAk3R543MCrrQjwowd/eJYjVW7payCb8vRv3FjZDCCER+9v2+vsmy4hoUkG4uAtIs6VQbh+s1Yhz99lsHtI3dgQeQkenAok1h9PCX3EAvJWZVSFzolkj+We2/Off9NVZwaa8MPnZIEPbtUlyP4R2n8QkWbFXA1DXktnLfKSx/7E4zuXRJqIDG9AIl78wcd7VUCRVPPqRfBh3HsL7h1q68aUpSyPDpDOL0slWJnGEtVixzP6+yOqkEQS6Krf5qJWP9Rs51FvjOeNWlYSDyd1aTQs26w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=yeXPBeXfD92VrJgIPKVAT3xAa1YNY34ei/eCe2HJlTM=;
 b=MqwUjqBvY9W8Cmmo5KspKVcAVXy8H9XLAaQT9hU2D2NrS7sE0J96UeRXgeMm0h13dO35x6HiwA9UO8n/hp1AlTrC85Pvj7LgtwIpOiNpvnU5ESXHMWWw6CBtS/gfVQqJBMebrc+eUnyyovJbz63AbZKUdvruKv47TWsYnb+emAVeMwQY3SRBd8mhGnTXxc6l/DJfQdq4qlQFSK0jHMNYr0nwkybbqgfOOTLsz1yucMTUWC+dxsPETw4UfssKEhFvikYqWxaOEp7fHuljZJsmH79YytKC1DNmKX/bo3UkUyWwx7GDSDfjKt544qvH89t/uRSZff+LWFTE2PcErp2Kgw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] pci: fix handling of PCI bridges with subordinate bus
 number 0xff
To: Igor Druzhinin <igor.druzhinin@citrix.com>
Cc: paul@xen.org, xen-devel@lists.xenproject.org
References: <1632445619-9211-1-git-send-email-igor.druzhinin@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <80a1c6d3-8b02-8052-e69c-73d8afbd1e43@suse.com>
Date: Fri, 24 Sep 2021 08:38:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1632445619-9211-1-git-send-email-igor.druzhinin@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0025.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e85ae663-3b88-410b-2f3d-08d97f25e49f
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2334A45B40F0D3E2B3EE401DB3A49@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LTO2X3N46BdV6cLGNEMLGWjuaH6CgpPdyPnzWwzQpKiPdHrMcgvcGQElIh3tDistQqpbRScdNlarsZMkCaniJH6qYvPIuUioJ9dhEq8Ke9Oz6MTHmcifheHpdXanrcjmc1VyQemwV3nmHuMNRcnoYTXgkjG1/1e9QKOU/u+e3ipDyDN3NtYhASsOE/tqPO/Jucyk/57l/zL5seRFFPb6GhcSwLnznyzJ2Q1QetHgnm+S1oLLrkGf0Xo1cVHsQsz6EHsctkkpcd5NwEXmHDGhUFX5nQlGyWGLGeIopP9TAk6NkrMV4WUzG5acmIrD23Rnuci7GrYsR7vBMg4/FvUxvD0baWMDjRLXr0hDGSFM109M1rbpn/IV3tASXnsfkvfONrbjg1zyZ0UYEatI3VbFA20PJrb45e/Q51qfkfRNVUojNnn0OyjgZrRem7d9/B/p9/2qONE2KzDkFRbnAvVtvzC9NEruRdRwvxWbsm7dfTSlpOzLLzkH3HoaxgV4DhAyiyceI1xFga/RFeC7F7cfy4Bseg+hLSgYO8af2FqRwN9yquKJIRUjFKd6JVEdqoRD/cZXS15VRDyzfjrFma4ZLlS5JO0MJNW2eUua0XFNdd6MWg8MCTkuDIHMRimxqkoPyeImnLkbjQACDjr3g9JDrYsLGc6yHF/WV+zwlwG7Uu4AnxeAKX0upH+j3WzQRn5p6GIVvQ53U33iSsisKUsmuBjxmH5w+F+PZyP+2IbHfcE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(4326008)(66476007)(8936002)(31686004)(2906002)(4744005)(38100700002)(66556008)(8676002)(6486002)(316002)(16576012)(66946007)(53546011)(86362001)(186003)(26005)(36756003)(956004)(2616005)(6916009)(508600001)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SXlFcnFDczNYZDBKM2dROUVzdEZLSk56cTNvUGFCWWZZczVDMktvS3h2VEdT?=
 =?utf-8?B?U1pzTDhxalBDQ0g4ay95QVNvS2FyeVQzZ0h5V2NPNEIyeSs0aUZiSzJzalNk?=
 =?utf-8?B?REdWTFkxSzNjejVlTlM3cXFxdmFkcSswUFNKb0N5US9GUEFQUy9lRDJoeVdW?=
 =?utf-8?B?K3dudEY2WkFUNHZNVCtqUy9sTjhYMWtSYXVVd3htTmU5dXJmM25scUNKWW5i?=
 =?utf-8?B?MTJQU21tdjFudEJxV0FGTldmOEJ5SWFIR2RqSDJXUnZlTG5qaU9YOFF6Vkgr?=
 =?utf-8?B?bTcwamJUWVQvUkU0aFdxbHpDMzBsYU5DSGZFWi9DdjJVWms1TWV1SUQvQlcw?=
 =?utf-8?B?ZVh3Q242S2JXOTEyY2RRZUZ2WlUvRXlqdElpbkpjRkRvZnQrNVhlVjhPWFRB?=
 =?utf-8?B?dmlocVVoLzVwM1NNem9IUVRzeWkzS1FxUEpYTWRJcFFuT3NiTFBoS2wzejZG?=
 =?utf-8?B?OTFoeWtFU2hFV295SXU5OUxSMDVkNzlTdVV4WVh1WnlOdHloM0k1RU9RaHlr?=
 =?utf-8?B?dHQvenJXRlNEL09BM0pjQWRoVGFiM2pUNm1FZm9VWUJ2L1llVmRhRHVxMVVi?=
 =?utf-8?B?NWNIdzdSR3BzUkY1QnNSdy9JTUZuQU9mTzNJZW9LbkNnNFdreXA0Z21CYkpT?=
 =?utf-8?B?R3lsaHQ3Q2YzTk1sS0VNUTE0NVlSME9tYnVvK0xYQjZXS0dwUmhpdzB2bmNH?=
 =?utf-8?B?ZUFmVjE1Nk85Sk82L2FsbzRxMStaZnl4anBSb1hhZHNVNERvdWpNZkhYSXBl?=
 =?utf-8?B?OXJlaGI0Y0xGSXdMazVNMkRRWHpHblZxSUNZUTB2WkhQRllaUm5Ea0phUUhz?=
 =?utf-8?B?b3JEWTBXcUE0UndCc0xSREVVaHY4WjdaTUJ3dHd0TVRxMlVmM040TWxxZWgw?=
 =?utf-8?B?TFU1TEV6eVRCMzhTTHpRdGtnZEJhT0xNTUxDRDhndkZHdzF5TnFqbTJKSVhp?=
 =?utf-8?B?ZDgySlM1clBUOU1Fb2hrYlVYVUJCdzNobXZKZjA2bW9oNHRjeFBRdlZ1N3lH?=
 =?utf-8?B?Vnp5VlMxWjJSSkcxcVZnQ3h3Qll3ekNwc1NEdXJPM2tkL2tlYnRhelVaU3k4?=
 =?utf-8?B?dFlsUXR6WkdXemhpTGh1NmNqSTRPM2FtTUM4a1dhMjJUbnNEbUNGOHJqVUxn?=
 =?utf-8?B?UjhnTUJuLzd1TzZxTFZjMVdURjdvYU9aSTlnclJzS3pOeGxLcUdJTkJnQjdZ?=
 =?utf-8?B?NDQxeEF1bmx3bXlMa1ZJbFJCSTdhblNMaER1RVQ4OWJORVFXckN2Tm1QY2Rm?=
 =?utf-8?B?KzI0ZDUxMml0djhhZG9JaW50MXRKK2c2TkNYWmU1ZkdsS1pwZmcyWWJGRXdy?=
 =?utf-8?B?amk1WjQvaXU1OGxrZzFmcTB4ZWZBTlBGNjg3T3dlOWYrVkJ3MlU4R2FwZmZM?=
 =?utf-8?B?am5ybitRSmVCaXRDZnpaSWFNWStRbU9tMks5OWVseTVsS3hZamNpVWJLSGR4?=
 =?utf-8?B?aWN3ZytZYWZpR2tsMW92QUVQS1p3Nm4wQXhIT2RRUi9yeWJMN0EyZ2EwRWxY?=
 =?utf-8?B?TDg0UFMxVDJCWHNIaFhsK1g4WHFrQjJaTDVYd1FNNkcrOGYrcXNTQ00vckth?=
 =?utf-8?B?dXVpNWoxMjFyNC9ENE5JTkFTSit5a3FQVHdWZ1Z1a25hM3U4aVJiYVZobjRX?=
 =?utf-8?B?TTdxb1dXR09GZk5mOGN3ZW1KbVJlQ25sQmJUT0V3ZVQ2bGtnc0RiUnI3NHoy?=
 =?utf-8?B?WWdCaWd6OVU0d3NNeU5NM0c2STBVL3k5d1h2ZjQ5dTRSK3RXZDhLS2xyNjNp?=
 =?utf-8?Q?3pAuhM54UOIm02XIMZXU/gKaUS53g3mfzHRaQmL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e85ae663-3b88-410b-2f3d-08d97f25e49f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 06:38:17.7120
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: t702Ru4cKXF7EjZQWB6lFZAnRKa5iH7ehzefuhzMnATUuzSogmz92rlBNW0RfchzOT5/QBl3W/9NbqUQIdCCqw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

On 24.09.2021 03:06, Igor Druzhinin wrote:
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -364,7 +364,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>      switch ( pdev->type = pdev_type(pseg->nr, bus, devfn) )
>      {
>          u16 cap;
> -        u8 sec_bus, sub_bus;
> +        u16 sec_bus, sub_bus;
>  
>          case DEV_TYPE_PCIe2PCI_BRIDGE:
>          case DEV_TYPE_LEGACY_PCI_BRIDGE:

First of all you want to also address the same issue in free_pdev()
then. Further, since we're switching away from u16, uint16_t would
be the legitimate replacement. Plus, since cap is then of the same
type, fold all three variable declarations into a single line.
Finally, as per ./CODING_STYLE, fixed width types should be used
only where strictly needed. I can't see a reason for any of these
to be other than "unsigned int".

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:21:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:21:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194955.347367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfWJ-0001Mx-Lm; Fri, 24 Sep 2021 07:21:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194955.347367; Fri, 24 Sep 2021 07:21:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfWJ-0001Mq-It; Fri, 24 Sep 2021 07:21:19 +0000
Received: by outflank-mailman (input) for mailman id 194955;
 Fri, 24 Sep 2021 07:21:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Coby=OO=epam.com=prvs=9901c588ba=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTfWI-0001Mk-4G
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:21:18 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ca920d28-96cc-467c-9ba4-aae1f7936072;
 Fri, 24 Sep 2021 07:21:16 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18O6hU4w020274; 
 Fri, 24 Sep 2021 07:21:11 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2052.outbound.protection.outlook.com [104.47.9.52])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b942g8pa6-3
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 07:21:11 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7347.eurprd03.prod.outlook.com (2603:10a6:20b:268::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 07:21:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:21:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca920d28-96cc-467c-9ba4-aae1f7936072
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=myE/3hL2rcKMSkhM1IvCaRvkSW9gM5JNLNcizk9QT3cx0NfA35dHtsF/ucWhEPS6a+u/LAb5GVf3TcbV0iOxPPQHkVtuVWGUM0HSnz+kEHA50NdZbxvR9zvWLZcOJ4jsLVf1RFSMAgCqXyUYHB1JI1KQBNE8LeoCqw5d0sslFLaACw0GLE9JaS0Q4mT5bpTVxrHNuE1k6UzTBlIq+DZV0iR03rToXkgI7xfh0ABQmTjaq8Vd5yKyi+ywCZEGhokWsurjZe57VNcooP0E1z0jFQN14DuofjigV8M9a2JXcHErCxFAUxumI4BuA0H96QhAZcQwtPswklAA0UldRrmaHg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5u1zeojJ5FGa2r+z5GcHg6hT6Nv3ZQvaRfrp8iLeq48=;
 b=B6NUC/aKzKSPZ1bp4rQ7e9Dqx7h5x55/R4tsfPWZZI4fPtY1haAl6z8Ql5JM8N1I0+P2H09oEswkTpMBvR9IIbL9SU/BIkkDAePVPOhzKnCgz9vveHwtmOKTPJp1nMP6PWTP1rWwCQO8ATx9paIuFbPyqmdzvaA5G11XSWecDw9T3YU2NMWTTFQMNuRmQBKHpBoqgiYc4UmIYDf0DyIvuwSKcTNllDiVyM67TEHZH+17k0Exx8m1e79OfQSXc25aTeizIfiaNe7P5eL2iRKGdDtLc0Y5Rl+dLYnt3Zega3yEjX6nkgV5kSKeSZdB2S9eNqvEkal+AcAOtdbruNJrtg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5u1zeojJ5FGa2r+z5GcHg6hT6Nv3ZQvaRfrp8iLeq48=;
 b=j5zMvm40oVoHUuoqEU74kbGA27DNRsLrft5rP5pgc0PYxaiBsMSDdM1P/3itL7PcWaInNXHP/ftGjrH5XTuOsoUw39TqbhJCHxRlYYI8hzOYhs+5wAambYYa2/Vheb7CYbGFXCcb8cpL5DTiT2e6Mojm6axIcGxgcMsGy6akE+9CzoGNPCHsob8Ryr8CmvuSOjFySYUcGPF6gZzLtXBKr1CCS1qHcp3KsusFE0U4Bf0mq9vLo+f6CDbgDPIe+O1wFtmR1M97B8kI+bvCWR5Rzb2Wr49b7JTfhpczd02xBqgzm/MXkGoUkYlpczcclCO8SRNEr75Tacul2VN2YMhOLw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Rahul Singh <rahul.singh@arm.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Andre Przywara <Andre.Przywara@arm.com>,
        Julien
 Grall <julien@xen.org>,
        Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
        Andrew Cooper <andrew.cooper3@citrix.com>,
        George Dunlap
	<george.dunlap@citrix.com>,
        Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>,
        Wei Liu <wl@xen.org>, Paul Durrant <paul@xen.org>,
        =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXsRS8cqOzQsPJjEGeH6Ub0BW3mw==
Date: Fri, 24 Sep 2021 07:21:05 +0000
Message-ID: <089415bc-71b4-79c0-0028-aac6a75e744f@epam.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
 <438E9A7D-65A3-444C-A24E-87E3A9F92234@arm.com>
In-Reply-To: <438E9A7D-65A3-444C-A24E-87E3A9F92234@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: fec98dcf-d894-4696-ca1e-08d97f2bdf1c
x-ms-traffictypediagnostic: AM9PR03MB7347:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB734783FDA8BE07757C367F7EE7A49@AM9PR03MB7347.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7691;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 JF7ehJt0DbCq9SJBRojgSBtoqV1axcR9+KpMVK/beGO9Lu3ubpSCEB/EGKfIeqELICz1aztjWyy63DQ/kp5QuorOzDAl0z0JB9bFM5j0O8L4Vkc4rpHRAkKKWF1vZIRkCrIgKEgAsNQvo4ranjM+C5B31zj3na3tBKAL4tuaU+CO/GLXghVQYeSer85uZngxRYN85HOErCu3wl27ivwnxVlPI5fWmXg5RVyr3yEjgp/aocBRfGWVcEz3FqkfAMABqoui72MAyz/WploAjHB6T9IjDOr51yJBmNkHDhalTx6RxuuWyZ/UVwoKoh3JIy+c1P3Pg2v0VFpJ7UqhlykkcwkbPkzJtptUwSSLMV6wux6CPWo7aqbL1j/i0MIwRmXfeD6ebRG2J1eTDoaoiska5wMpddUSJv3hZd5h2tTOutOk5l58DrOInC4cGswj2IBXaBu28E55WRz679X2vl3wj4GKxmGydtkvjnGSgFetdYG7qJnWgGByf5eBKG/4gBH0VuOeeeiI8lrzgSqnAkQVFzT30b1dWsOWp19Orwc3zuVMsnhxja6Vaw90b66pygq3y7QEbLRx8kJFKmRpggW5lr70dGWhyCnrMM1HfSp63xsrdmGotKJcef+oV6LnRqquOF2hHZmiWKzlw9dtKQRgiKxzQCsDVwvSruGxTfZLUl6bjDxtLU9OkMvonmlI9c05NB/S/L/7aiHGlFBm0ZG8eIeunGvvrDJuRcdPd+4YB35+ogm2ww+FgmNJiS82VHZ1
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(55236004)(6506007)(76116006)(66946007)(66476007)(64756008)(186003)(91956017)(83380400001)(38100700002)(66556008)(66446008)(4744005)(5660300002)(38070700005)(7416002)(508600001)(2616005)(6486002)(86362001)(31686004)(122000001)(36756003)(31696002)(71200400001)(4326008)(2906002)(316002)(6512007)(54906003)(110136005)(8676002)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?L2RPTEhGeVhQM3QzdjV5TUVSdXdyN2dnR1JhYU1odHBCbkZXc0EyL3piYjdO?=
 =?utf-8?B?VEJ3SEhSUU5lSnVvaGR6Z2JPK1h4MjFkeU9WcENuREU4NUpkUkhUQW4ra1NF?=
 =?utf-8?B?RWpmVXVjcXExaFVIVXlQeTBkRkhrQVhTSGkySHFjS0prV2IvbUNmRE84K0VQ?=
 =?utf-8?B?T1d3Zng3VnR6c1lQMlZ5TGZZWVpNRERHczZwS3JPdWp3M016Z2hlanZ3cFdG?=
 =?utf-8?B?SnV4TkpWMGp0cGxlQzVRR3RJOHo2N25vTnlqNFpCY25jNlRWSElBS01BaGUy?=
 =?utf-8?B?N2xnRTQ2MHNVWkdENXlTbzN1Y0FLdHlLUmJVT2tpMG5oVk1sc01iZ3Rab3JF?=
 =?utf-8?B?OGRMdTFSRGxYb0tXdEtEMU9lZnhoRDgzM1ZzOEFqSzBKZXBYZXVSZzFKSGdm?=
 =?utf-8?B?WExmRDJtU2I2bU9lT2JoanJpODdYSTZqNnd6RlVmNlZGNjRSTFFtOFBLaXNG?=
 =?utf-8?B?ckp1YTI5NWNtajhxZFB1ZXY0NVBaOGY5SlN6bWljb294Zi92YURMUXFWS3lH?=
 =?utf-8?B?dGlhVk01Q3lEU3ltVnluTGN3UmlJN1puOWd4QTBHWkNndDRDRGFRKzRCK1Vw?=
 =?utf-8?B?YUdqWS9ZNThSOGxieG9MSUNnTENEOFd2eVk1YWZ4eWdGRk44dzFGUUhRNWRO?=
 =?utf-8?B?eU5Ub0xEZDdUNnFXVWMwNmJVR0tJRlVFN3JUZ1dEMHYzcTJXM0dDdDgzRnZN?=
 =?utf-8?B?cWtKVFAvU0p0cmhGWFQ0M1dUbGFibU8vSjZnclBHV3BHbWNBY28xbkZRRlNq?=
 =?utf-8?B?V2pyOHlMQ0ZMR0EwTEk1ek1lMUxxUFh3bHJWczd4MmVlTDVMQmljWUJTa0tw?=
 =?utf-8?B?dXc2emVVYlphbU0vSm5abGplT3B5aTVZaStmZTU4cGpZRjRlSGNmT00zVXVq?=
 =?utf-8?B?L0YxMWtsS3dXU2N3dWxjbFJ3TG5HUFArdnNmTkM1YXdSSVloNlgzQzZMSlA1?=
 =?utf-8?B?V0lDOTVxWmNaNDg1RFo2RzR5WWhjVklQTld4WlErSEVESUVFRTF0cXhNWUZ5?=
 =?utf-8?B?d2V0cEhsVU1ya09BcVhEdDNZM3hialdXL29LUmJqNm95YW5RVGdEdEY4TFlU?=
 =?utf-8?B?T2VoZlBvYTJqc2MwMDFMWmV6b3J5eHkrK0lZTlkzeDZVKzgwelZMVUZXNDZl?=
 =?utf-8?B?ejdpTXJGMXE3UEJRQlovVGVIMkdCQ2F1T2JROVg3a1VleDhYVXFvUE55dWRy?=
 =?utf-8?B?d0trRkdpQ1grcnhxbkUxeThSNnQyNGljTVVuZDkzcGZBQnlhN1U5QkRGS3dO?=
 =?utf-8?B?MmRzdU5oOVM0d0twL1ZMVGpVd0toaVlPTjN2bW01YzhRRUtiWFJSalVxbS93?=
 =?utf-8?B?NGNjNEVoM0tSTEN3ZkNibHFNejZvYmVqcVdvdnpoR2podlhWQUxaV2JIUlJa?=
 =?utf-8?B?ZjZUM3pjRVdQOS8zVklTNnJBTWpjeDljQVVIWWJuZGs4U0pXYU5EZi8rUGJC?=
 =?utf-8?B?TU5QeUFjRnpKSDFDZGwyQWc5UUhMU1phL0M3Y0hLMHNoV3ZodHAvRGVDRzVI?=
 =?utf-8?B?T21aazh4bFRZT3liNnAzdUNhR2tVVElTNlZ5Q2lQSUQ5b1NkY21EeGJHa2E3?=
 =?utf-8?B?c3BYY0E1RTZqckJIYkYwSjV3RGx2Zml0NzhpZzQyckdtMERPUGVWTUR3WWRw?=
 =?utf-8?B?eCtUZ0YrWHFleEk2RUZ3Ny9td2pKT1ZEZFIwZkx6dXllY3VlTjJwVm43RzlY?=
 =?utf-8?B?Y0FsT1daQ0VSVXBnSFEwMFZSK3E1WnlQZUJOcWNORkhZZ1RFZ0h3MHl2SmNL?=
 =?utf-8?B?LzRsdWc4c3dVbFVudGtmTVBrMTVXUmQ2VWlwNjNlREV5T0taTnpEOU14Wk1u?=
 =?utf-8?B?K09OTFZBckhTN0VsTVlKUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <8C6F5150524ECD489FA4F9C864BD5FB6@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fec98dcf-d894-4696-ca1e-08d97f2bdf1c
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 07:21:05.2605
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: taYtc1Nr/Uf+PQDyGPBn6/nKtpu7LdJrtFplsJ+y+CvLxpZAN1Ve6M4TRjSqatAOJdir6ugUI/TJN/qerHf2hK/NmqZ3RzGqtmr60LJhXg+/BskB9bwTmAc6a9aJP5mN
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7347
X-Proofpoint-ORIG-GUID: zW2iYfPy16wjOXf05k7fn3oLyDZCnWrn
X-Proofpoint-GUID: zW2iYfPy16wjOXf05k7fn3oLyDZCnWrn
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-24_02,2021-09-23_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 lowpriorityscore=0 adultscore=0 bulkscore=0 impostorscore=0 mlxscore=0
 suspectscore=0 spamscore=0 malwarescore=0 phishscore=0 clxscore=1011
 mlxlogscore=783 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109240042

W3NuaXBdDQoNCg0KPj4+IGluZGV4IGQ3NzRhNjE1NGUuLjYzM2U4OWFjMTMgMTAwNjQ0DQo+Pj4g
LS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+PiArKysgYi94ZW4vZHJpdmVy
cy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+IEBAIC03NjcsNiArNzY3LDE4IEBAIGludCBwY2lfYWRk
X2RldmljZSh1MTYgc2VnLCB1OCBidXMsIHU4IGRldmZuLA0KPj4+ICAgICAgZWxzZQ0KPj4+ICAg
ICAgICAgIGlvbW11X2VuYWJsZV9kZXZpY2UocGRldik7DQo+Pj4NCj4+PiArI2lmZGVmIENPTkZJ
R19BUk0NCj4+PiArICAgIC8qDQo+Pj4gKyAgICAgKiBPbiBBUk0gUENJIGRldmljZXMgZGlzY292
ZXJ5IHdpbGwgYmUgZG9uZSBieSBEb20wLiBBZGQgdnBjaSBoYW5kbGVyIHdoZW4NCj4+PiArICAg
ICAqIERvbTAgaW5mb3JtIFhFTiB0byBhZGQgdGhlIFBDSSBkZXZpY2VzIGluIFhFTi4NCj4+PiAr
ICAgICAqLw0KPj4gSSB0YWtlIHRoaXMgaXMgbm90IG5lZWRlZCBvbiB4ODYgYmVjYXVzZSBmb3Ig
dnBjaSBpcyBvbmx5IHVzZWQgYnkgRG9tMA0KPj4gUFZIPyAgSXQgd291bGQgYmUgZ29vZCB0byBj
bGFyaWZ5IGVpdGhlciB3YXkuDQo+ICAgSSBoYXZlIHRoZSBzYW1lIHVuZGVyc3RhbmRpbmcgbGV0
IHNvbWVvbmUgZWxzZSBjb25maXJtcy4NClRoaXMgaXMgbXkgdW5kZXJzdGFuZGluZyB0b28=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:30:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:30:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194963.347378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTff1-0003BA-I9; Fri, 24 Sep 2021 07:30:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194963.347378; Fri, 24 Sep 2021 07:30:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTff1-0003B3-Ea; Fri, 24 Sep 2021 07:30:19 +0000
Received: by outflank-mailman (input) for mailman id 194963;
 Fri, 24 Sep 2021 07:30:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CKs0=OO=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mTff0-0003Ax-MU
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:30:18 +0000
Received: from mail-ed1-x529.google.com (unknown [2a00:1450:4864:20::529])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2029eedd-bd8a-4673-ade1-7261e8fec038;
 Fri, 24 Sep 2021 07:30:16 +0000 (UTC)
Received: by mail-ed1-x529.google.com with SMTP id x7so18069805edd.6
 for <xen-devel@lists.xenproject.org>; Fri, 24 Sep 2021 00:30:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2029eedd-bd8a-4673-ade1-7261e8fec038
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=R+8UTVFE+U8I1rxxBt0QNZ0Q3ZF9T77NQxVWTO7O2tk=;
        b=Kbogy6wFwhR5I9me9bGnHOeo/qHTQBi18rCkTv1kK55jeY8MeAoGfdMgj5T39tI4y7
         6wqfEgJnz3fUZusKH5paIB5DX7Pa37Sl+CnIjz159DGWJ35SnOrAUXLp7hOiHh6OMnIN
         AlRFYmm7966VH4ai5E/zd67okgbYpOaRJOAn5+W/YxdOAfD5FjdvfTRlPvwqEbm8HSDK
         Sev8nsG0D5Er/Qgz4i3ePm/9xs3b6wdA+ck4ZVDMOYZfynb1/nCw3zAv4xCfrmkleX6m
         0dzo7RF1cBqOBy1N0NS9uGwcSAHm74sl5UHgVm5Xcahf+r9LJaA2K8IJtLW085u4ps8N
         tWIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=R+8UTVFE+U8I1rxxBt0QNZ0Q3ZF9T77NQxVWTO7O2tk=;
        b=fAZar7VGJtLXbHr4O1L495jK4y75pbrM3MSdRPqA8QI4ojHVmJPH1H+dA4612ca4oa
         VEHaMqRKk0dAa+N1ncbsVEgUfS1pRupQGTtV3zNnPQPmHlA9SI7dCunO7AoLlAw6MmiU
         b06OkQIbdAW7nfTVLM9CnR5tT9W17XWNYFLp9H2lFdVn+dF7RE/LRdydWz7i4U5GE5v1
         /EVZ0n7O0y3AIyGtbU7SyQtUGF1ybGlNm/dCxpZI42K1oqDKDngtyCcLysETEh530TvN
         wtqEhPqRA8b5OwHOF7uIxjH0cYZSqMwiIRUuAoLXwfUZetPEa0txT7tYvkzMEn/GFaD1
         DaHA==
X-Gm-Message-State: AOAM531se+xNSo9IEEI8YCnL74tavFXYtIaaTsk9Z3oNDxP36JakkLwu
	Nbm8LhEUHb6x4Rp1jkiaunNq5XddWZ9Rd2QXz/A=
X-Google-Smtp-Source: ABdhPJx34xPauPGB8H4Lh541nZ81+tZgUtHo9oCDQXkMUrmivUqZjs3qoJrtMz0cMI3wcH2DJSvzr04lOrivYwjVxls=
X-Received: by 2002:a05:6402:2cd:: with SMTP id b13mr3377863edx.267.1632468615719;
 Fri, 24 Sep 2021 00:30:15 -0700 (PDT)
MIME-Version: 1.0
References: <20210922082123.54374-1-roger.pau@citrix.com> <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local> <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
 <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local> <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
 <d3a314ae-a2bb-58da-b98e-bc03a049bc1d@suse.com>
In-Reply-To: <d3a314ae-a2bb-58da-b98e-bc03a049bc1d@suse.com>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Fri, 24 Sep 2021 12:30:04 +0500
Message-ID: <CAJ=z9a2gub_Dn6P3T2H05kwB6TUHf-y=rBvxcXpYaiGNrAXqyw@mail.gmail.com>
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, 
	Wei Liu <wl@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, 
	Christian Lindig <christian.lindig@citrix.com>, David Scott <dave@recoil.org>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000f26cf705ccb8b726"

--000000000000f26cf705ccb8b726
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 24 Sep 2021, 11:21 Jan Beulich, <jbeulich@suse.com> wrote:

> On 24.09.2021 04:30, Julien Grall wrote:
> > Hi Roger,
> >
> > On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=A9, <roger.pau@citrix.com>
> wrote:
> >
> >> On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
> >>> Hi Roger,
> >>>
> >>> On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:
> >>>> On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
> >>>>>
> >>>>>
> >>>>> On 22/09/2021 13:21, Roger Pau Monne wrote:
> >>>>>> Hello,
> >>>>>
> >>>>> Hi Roger,
> >>>>>
> >>>>>> First patch on the series is a trivial change to xenconsoled in
> >> order to
> >>>>>> use xenforeignmemory stable library in order to map the shared
> >> console
> >>>>>> ring instead of the unstable libxc interface. It's reviewed and
> >> ready to
> >>>>>> go in.
> >>>>>>
> >>>>>> Patches 2 and 3 allow setting the host wide command line `gnttab`
> >> option
> >>>>>> on a per domain basis. That means selecting the max allowed grant
> >> table
> >>>>>> version and whether transitive grants are allowed.
> >>>>>>
> >>>>>> The last 3 patches attempt to implement support for creating guest=
s
> >>>>>> without a grant table. This requires some changes to xenstored in
> >> order
> >>>>>> to partially support guests without a valid ring interface, as the
> >> lack
> >>>>>> of grant table will prevent C xenstored from mapping the shared
> >> ring.
> >>>>>> Note this is not an issue for Ocaml xenstored, as it still uses th=
e
> >>>>>> foreign memory interface to map the shared ring, and thus won't
> >> notice
> >>>>>> the lack of grant table support on the domain.
> >>>>>
> >>>>> I find a bit odd that the Xenstore support is conditional to whethe=
r
> >> grant
> >>>>> table is available. Are you expecting domains with no grant table t=
o
> >> have no
> >>>>> PV drivers (including PV shutdown)?
> >>>>
> >>>> I don't really expect much, as having guests without grant table is =
a
> >>>> developer option right now, if someone wants to make use of them for
> >>>> any reason it would need some thought.
> >>>>
> >>>> The other option would be my first proposal to restore foreign mappi=
ng
> >>>> of the xenstore ring on that case:
> >>>>
> >>>>
> >>
> https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix=
.com/
> >>>>
> >>>> But it's also arguable that a guest not having a grant table should
> >>>> also likely prevent foreign mapping attempts. Plus such foreign
> >>>> mapping won't work from stubdomains.
> >>>
> >>> There is another option: extend the acquire hypercall to allow
> xenstored
> >>> domain to map the xenstore interface. This would require more work, b=
ut
> >> at
> >>> least it would avoid the interesting dependency on the grant table.
> >>
> >> Xen isn't aware of the shared xenstore ring page currently, so that
> >> would mean introducing more knowledge to the hypervisor that what's
> >> strictly required IMO, as Xen has no business in knowing such details.
> >>
> >
> > Well Xen already knows the page for HVM/PVH because the guest retrieve =
it
> > through an HMV param.
>
> To be honest using this in such a way would feel like an abuse / layering
> violation to me.
>

I can see how it can be seen like this. Do you have a better suggestion to
be able to map mapping without the foreign mapping interface and the grant
table?

Jan
>
>

--000000000000f26cf705ccb8b726
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Fri, 24 Sep 2021, 11:21 Jan Beulich, &lt;<a href=3D=
"mailto:jbeulich@suse.com">jbeulich@suse.com</a>&gt; wrote:<br></div><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex">On 24.09.2021 04:30, Julien Grall wrote:<br>
&gt; Hi Roger,<br>
&gt; <br>
&gt; On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=A9, &lt;<a href=3D"mailto=
:roger.pau@citrix.com" target=3D"_blank" rel=3D"noreferrer">roger.pau@citri=
x.com</a>&gt; wrote:<br>
&gt; <br>
&gt;&gt; On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:<br>
&gt;&gt;&gt; Hi Roger,<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:<br>
&gt;&gt;&gt;&gt; On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wro=
te:<br>
&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt; On 22/09/2021 13:21, Roger Pau Monne wrote:<br>
&gt;&gt;&gt;&gt;&gt;&gt; Hello,<br>
&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt; Hi Roger,<br>
&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt;&gt; First patch on the series is a trivial change to x=
enconsoled in<br>
&gt;&gt; order to<br>
&gt;&gt;&gt;&gt;&gt;&gt; use xenforeignmemory stable library in order to ma=
p the shared<br>
&gt;&gt; console<br>
&gt;&gt;&gt;&gt;&gt;&gt; ring instead of the unstable libxc interface. It&#=
39;s reviewed and<br>
&gt;&gt; ready to<br>
&gt;&gt;&gt;&gt;&gt;&gt; go in.<br>
&gt;&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt;&gt; Patches 2 and 3 allow setting the host wide comman=
d line `gnttab`<br>
&gt;&gt; option<br>
&gt;&gt;&gt;&gt;&gt;&gt; on a per domain basis. That means selecting the ma=
x allowed grant<br>
&gt;&gt; table<br>
&gt;&gt;&gt;&gt;&gt;&gt; version and whether transitive grants are allowed.=
<br>
&gt;&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt;&gt; The last 3 patches attempt to implement support fo=
r creating guests<br>
&gt;&gt;&gt;&gt;&gt;&gt; without a grant table. This requires some changes =
to xenstored in<br>
&gt;&gt; order<br>
&gt;&gt;&gt;&gt;&gt;&gt; to partially support guests without a valid ring i=
nterface, as the<br>
&gt;&gt; lack<br>
&gt;&gt;&gt;&gt;&gt;&gt; of grant table will prevent C xenstored from mappi=
ng the shared<br>
&gt;&gt; ring.<br>
&gt;&gt;&gt;&gt;&gt;&gt; Note this is not an issue for Ocaml xenstored, as =
it still uses the<br>
&gt;&gt;&gt;&gt;&gt;&gt; foreign memory interface to map the shared ring, a=
nd thus won&#39;t<br>
&gt;&gt; notice<br>
&gt;&gt;&gt;&gt;&gt;&gt; the lack of grant table support on the domain.<br>
&gt;&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;&gt; I find a bit odd that the Xenstore support is conditio=
nal to whether<br>
&gt;&gt; grant<br>
&gt;&gt;&gt;&gt;&gt; table is available. Are you expecting domains with no =
grant table to<br>
&gt;&gt; have no<br>
&gt;&gt;&gt;&gt;&gt; PV drivers (including PV shutdown)?<br>
&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt; I don&#39;t really expect much, as having guests without g=
rant table is a<br>
&gt;&gt;&gt;&gt; developer option right now, if someone wants to make use o=
f them for<br>
&gt;&gt;&gt;&gt; any reason it would need some thought.<br>
&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt; The other option would be my first proposal to restore for=
eign mapping<br>
&gt;&gt;&gt;&gt; of the xenstore ring on that case:<br>
&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt;<br>
&gt;&gt; <a href=3D"https://lore.kernel.org/xen-devel/20210917154625.89315-=
6-roger.pau@citrix.com/" rel=3D"noreferrer noreferrer" target=3D"_blank">ht=
tps://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.com=
/</a><br>
&gt;&gt;&gt;&gt;<br>
&gt;&gt;&gt;&gt; But it&#39;s also arguable that a guest not having a grant=
 table should<br>
&gt;&gt;&gt;&gt; also likely prevent foreign mapping attempts. Plus such fo=
reign<br>
&gt;&gt;&gt;&gt; mapping won&#39;t work from stubdomains.<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; There is another option: extend the acquire hypercall to allow=
 xenstored<br>
&gt;&gt;&gt; domain to map the xenstore interface. This would require more =
work, but<br>
&gt;&gt; at<br>
&gt;&gt;&gt; least it would avoid the interesting dependency on the grant t=
able.<br>
&gt;&gt;<br>
&gt;&gt; Xen isn&#39;t aware of the shared xenstore ring page currently, so=
 that<br>
&gt;&gt; would mean introducing more knowledge to the hypervisor that what&=
#39;s<br>
&gt;&gt; strictly required IMO, as Xen has no business in knowing such deta=
ils.<br>
&gt;&gt;<br>
&gt; <br>
&gt; Well Xen already knows the page for HVM/PVH because the guest retrieve=
 it<br>
&gt; through an HMV param.<br>
<br>
To be honest using this in such a way would feel like an abuse / layering<b=
r>
violation to me.<br></blockquote></div></div><div dir=3D"auto"><br></div><d=
iv dir=3D"auto">I can see how it can be seen like this. Do you have a bette=
r suggestion to be able to map mapping without the foreign mapping interfac=
e and the grant table?</div><div dir=3D"auto"><br></div><div dir=3D"auto"><=
div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin=
:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Jan<br>
<br>
</blockquote></div></div></div>

--000000000000f26cf705ccb8b726--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:37:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:37:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194970.347389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfm3-0003vO-ED; Fri, 24 Sep 2021 07:37:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194970.347389; Fri, 24 Sep 2021 07:37:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfm3-0003vH-B8; Fri, 24 Sep 2021 07:37:35 +0000
Received: by outflank-mailman (input) for mailman id 194970;
 Fri, 24 Sep 2021 07:37:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTfm2-0003vB-Bu
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:37:34 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4715872a-1d0a-11ec-baad-12813bfff9fa;
 Fri, 24 Sep 2021 07:37:32 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2053.outbound.protection.outlook.com [104.47.12.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-fedGMrEWM1-uD9vl32jFeQ-1;
 Fri, 24 Sep 2021 09:37:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2336.eurprd04.prod.outlook.com (2603:10a6:800:27::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 24 Sep
 2021 07:37:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:37:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FRYP281CA0018.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Fri, 24 Sep 2021 07:37:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4715872a-1d0a-11ec-baad-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632469051;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kzFNDGGnURHa/AMypY4IXW+8W1K9iX4ncP8x+aBrxjs=;
	b=RI6a56l/rD5qOSKwD5Yc0rCvTjc6E9lnGyrQ4Mv6vS73fucQ09pQoGLFMMSV0spcYIZZVN
	BGRI8Dloi9xGHBhYvx0E78HJFMY5hWM9bWg6qHtY2+i7IKjGPdNuKq+V/4H0Rpaac3AR/w
	MdPEygpu1dWHvIVVsDbXp+cLoziq3g4=
X-MC-Unique: fedGMrEWM1-uD9vl32jFeQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=J9vor3GkLNun6PMy0wBOo4kMeDI9IdMe/gK55SN3ACJmdLOQ4yBVaLSfgv6+F0czbE7nW28UBMYMGWybEQ8KZfEv6G3zLHIbCbGQEIPBCWXc7tLoIJZCFGE2KIo13nY8h1T0EcwlyiLwhyMninEOMcbNlK8idaoSR9zb24VwNNQEdrddf3IjiMGxMLke9R0ZVbdCCWifsdHIaqhrrumYpauSAERRUFNSBskDNLyiMCTzFHPFVGQjujbfa9cW0yPKrUppxIEP5a3tWuIXAU+rd89sjEBs/vWEdhZeqYvpz1477qTsDSQjVrppO+N/MwNRHrMT5kDXgG/F2KeG2Cviuw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kzFNDGGnURHa/AMypY4IXW+8W1K9iX4ncP8x+aBrxjs=;
 b=PfsbrPUqi0rfQ1Hbfxqyqzku+MNFqKaWKtoXLb1Ep1G0u/36Id8GcbRjGniQQLTemJs0IB5adu1HBTqgh/QVji985FIbAGg5n1fV2CjoMQzHG94BQFl3gVZrsqsWxPIX2RIs1eQU9yUQGhzCRrZjn3ATeTOOEMEf1HMqmEfpcD9SpIAL1GpwnGoxe/XZ80NWZfLUByGXl1pykQ5NN5khXMCm2hh9Pi9jBi+a1qPzigH53GXa9M2VNJATIKNXWrGM2feI6iLQnMSBZmMo3g0xQsLNYzlMA/mes1YpEIEvvOsic4bcCHtY+6YU6ammgxfUAvmf/h9PR2n4REEvHB5RrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 andre.przywara@arm.com, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Paul Durrant <paul@xen.org>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7f4ef734-3218-1f4d-7fd7-445820ad9f79@suse.com>
Date: Fri, 24 Sep 2021 09:37:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109221926410.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FRYP281CA0018.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::28)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7e8c1872-cc18-4176-5077-08d97f2e28e5
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2336:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2336A648730A282E868F98F9B3A49@VI1PR0401MB2336.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	n4qhCLzrpCxIoL2+TanNaU71Esa7ntP3xv6JcIBzNfnKsya6H4wCKRQEOcoqi9XB4bzry72pycriL6kftk3oCYdB2Sc5R8TAGmN4PT00ExlMUiPBl6IVwqH/mYnrt+fyZrPL37Umj3bxIFLQiJb/LfPltrIvz43XjcY6KsaBkWGh+Z+kuXQNLKgKQgPdI+NrwU+04ExTvHKFp6fBb5lm+6otufbXQL7AorPeI8T+MwDPRrOTh3Z7wucZ13NIZUEOvuBP+3ZuAjNxFH0s2ovUZ2Hc7t4Askrn7QxyLw0Z4R/7cebwDwXMLHK/X2/PaS+m1p4cxihb1t4eJCVacDajTuqOrT5oZYkMhcaT3tWlvsmRRNWS01Nm6p8HsmJ7g0E2uzngKB2e6Hsh0aeFxLP/ywDWL2Ph4KWoFgcX7B3Ywj4NqWxcJ57s7SnUUVQOpJchQgGddRpAlk0DDHQD+lbVkcMLk671iaZq5fhOQxcybVbAGSoRajU9JfjaLEQ6kggQ0gZLvyyWCnyCQH3XyZguHXSWdlttFKtBMW2G/k7n9mHP9t/zjRZ43iO1U4+9FlbCvQQvz9PDKHgBpnOCdHFwn4zMlvg20A5N+LSMNOY6Xczby/hcVSJ/UaX8rfD+YfO9jjo/op0ChzT2MsaaOfbmlwtvHgqeur3TuWa16R4Sh+nuTCE7LfaR7UxRmeqn+5CMb20IuyS9n+I7JEZjSL6IXcdaYIyw791nR40jnFHVGL8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(110136005)(2616005)(5660300002)(4326008)(66556008)(54906003)(316002)(8936002)(83380400001)(86362001)(8676002)(956004)(508600001)(66946007)(7416002)(38100700002)(31696002)(6486002)(53546011)(186003)(26005)(36756003)(31686004)(66476007)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eDBnZjV6SlBMRVpDKzNkSnhQSnRSMWJGWnZRU0R1MlRCTmdUazF5cCtZbXBt?=
 =?utf-8?B?OExLdkRyYWo4ZDdTbDB5M29JTnB0ZHg4Rmw5S0pqaDlidHBFMVRVOTczeHlW?=
 =?utf-8?B?Qy81Zk5ycHBMdzVnSEU5azNveXFLcTNyVWdHRk1abE45eWVyM2R3UWVHL09z?=
 =?utf-8?B?TVVxZU9TZ25aaUpBNHUwczU0aG93aWhPMkZaYWlXRklPMHliTGdJSGJHckdH?=
 =?utf-8?B?dzNyN0cyaWJlVTVUc1pYb21vcDA4dS9lUWhzWHlCTmw1c09VczNhM2NQY1Ft?=
 =?utf-8?B?WHFHUlltYnozYTkxbk5kM0J3OWU4aEJYWTU4VjI2Q3FmKzdMZERIblBYUE02?=
 =?utf-8?B?Q3h3V3dQbEYvcTN3Y2QySzErL2JQMzFac1d6YnBEWlNldk5WaVR0MkxESmk3?=
 =?utf-8?B?RmJmRjQyNlBhOHZ0bXFKWllsTVhOTTdBRHhDeTJBOEhDdEtiTUJZdzlzR0Er?=
 =?utf-8?B?dzN3ZlBpd2RlcE5oVi9KRFY1ZWFtMlA4dG5SdFphKzR1TUpsN1JWcEsrRG1m?=
 =?utf-8?B?Mjg0Y2tFYWNrcGhNYjVubXlDUERudDlqR2ZkcGEwcXFDa3hROVZrTlg4ODVC?=
 =?utf-8?B?L0E1aTY5NVhoRjdXSzh6dVRQRWVxRnZqSFMxSHA1MWdsdUhUU1M4TFlhV1Iy?=
 =?utf-8?B?cUE5L3d1Q01TRGJsWHR6c1Y0a3d0Zk1hYlJoSHdMSlduZTVsTlF6YW83Q3lD?=
 =?utf-8?B?Nk9MYXRmSnJZaTZIK3dOcURaMFZIQ2VvOWRpbG5wTGtFcFlmNzliRDU2OGNk?=
 =?utf-8?B?Rm1rcmtHZ0VlaVJobGt0TlhCcGFIRHEwbnhIcjJZUzNvMWxvTmdkWHp4MnhJ?=
 =?utf-8?B?QzRhMnB1VGc1RXB0MzFiYVNWMFdZeTV2bTdaZDlCY0liM1Z0RTg1MGdHU0NV?=
 =?utf-8?B?UG9UL3IxL0hGNVpHYk1Lbk9UMGVzeUZTb0UwbnlOZ2twRHF1d1BGV050Q3R3?=
 =?utf-8?B?ZUpicEc4bXJjMURWYW4vOUNNRndPTzNRZDRuMnV2KzFKNHVVbitxUytxTnl6?=
 =?utf-8?B?emw5WER4VEtSK1RFWDg2d1JxRXdtM25ReFg5ZTFoTThvdDQ0NjlVVG9nakVU?=
 =?utf-8?B?am1DTHF6S1RBZzB0NmxGbkkzMDhPdWUxU09EMVJxWGZySmZoWE43ZWlpaXZM?=
 =?utf-8?B?cXdWUWZSTmhET0s1Uk43c3d1OUNIMTI5TXgvNlc2eS9tZVoxMXppTC82MGFI?=
 =?utf-8?B?T2tHWmV5bEZQL0g0VzBoQ0xaL0p5MjNUMk1yMWdvcFhuMiswd2w5cExvejF0?=
 =?utf-8?B?RGk3b2Y0Y0FIRk9nZ3cvejgyWmVYUnhNMkhWUS9YVkRZbFNuc2hWM1dtOHRE?=
 =?utf-8?B?ejQyenJRZ05Sc042QUtpbWloeVpqbjZwNlpUd3FGdll0bUY0dEdBb2ZZNWNO?=
 =?utf-8?B?Vm4zYVR6SnMwL3loWTNPR0YwcmpXVnpKTUtnZXVSL1IyWGR6bDVEMWtYSjdI?=
 =?utf-8?B?ZHJTZTk0MHZGQXdnRjk3Z2xBUEdNamEwcVVlWHRFUW1BSHRKaEwvdUd4LzdL?=
 =?utf-8?B?QTRvNUZlK1NoY2RoMHN6Yk9BcXlJU243MkcyczRQcTJjenZMbHIzaHYzR0Zj?=
 =?utf-8?B?aWQyV0tPQ2J6MjZvejNreXAwcExWNyt3clVmSnRyYmpwcXpSL3owcmpxbzRZ?=
 =?utf-8?B?M0Rwc1drU3gwbUorZVgyQWlXM2I5MCsydmlyemVSOVhoNXJpS1owWHZxd0FW?=
 =?utf-8?B?WlZzS0dxSEhYbWwzL01oUjJXSGlUNUxCN29jbDhMYlQ1V1RMSk1xaldUSWJL?=
 =?utf-8?Q?2KdlUCE9i+Ye9xxVR7o0YPDoj4oCzSKa0M5WxY8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e8c1872-cc18-4176-5077-08d97f2e28e5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:37:28.2545
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qJDTH0XyaSjGvBRi0+/ArsmjRGkJbXGCS0svxnXtetPjYxx2BGB91MKVjduHIMxtR7hfwtmer6QFybitkrOljQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2336

On 23.09.2021 04:41, Stefano Stabellini wrote:
> On Wed, 22 Sep 2021, Rahul Singh wrote:
>> --- a/xen/drivers/passthrough/pci.c
>> +++ b/xen/drivers/passthrough/pci.c
>> @@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>>      else
>>          iommu_enable_device(pdev);
>>  
>> +#ifdef CONFIG_ARM
>> +    /*
>> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
>> +     * Dom0 inform XEN to add the PCI devices in XEN.
>> +     */
> 
> I take this is not needed on x86 because for vpci is only used by Dom0
> PVH?  It would be good to clarify either way.

I'd prefer if x86'es present vPCI restriction to Dom0 would not be
baked into any new code additions, if avoidable. Furthermore I'm
not convinced there's a connection here. Instead the question is
purely how Xen learns of PCI devices, which is information I surely
expect to never come from a DomU.

Just recently we've discussed the hotplug aspect on x86: While Xen
scans the bus during boot, it has to rely on Dom0 for hotplugged
devices. Hence Dom0 even there needs to notify Xen of new devices
(see "x86/PVH: permit more physdevop-s to be used by Dom0"). So
there will need to be a 2nd place to add handlers for x86 as well.
Which means preferably the change here would cover x86, too. Which
in turn means re-registration of handlers needs somehow dealing
with (either by making the call here conditional, or by making the
function itself spot the case, or by lower layer functions being
tolerant of finding a handler already in place).

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:45:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:45:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194975.347400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTft3-0005Kp-6J; Fri, 24 Sep 2021 07:44:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194975.347400; Fri, 24 Sep 2021 07:44:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTft3-0005Ki-3H; Fri, 24 Sep 2021 07:44:49 +0000
Received: by outflank-mailman (input) for mailman id 194975;
 Fri, 24 Sep 2021 07:44:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTft1-0005Kc-OK
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:44:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 489dd967-1d0b-11ec-baad-12813bfff9fa;
 Fri, 24 Sep 2021 07:44:46 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2058.outbound.protection.outlook.com [104.47.12.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-aHLWf5b9OcmtMVFSng3zDA-1;
 Fri, 24 Sep 2021 09:44:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7037.eurprd04.prod.outlook.com (2603:10a6:800:125::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 07:44:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:44:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Fri, 24 Sep 2021 07:44:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 489dd967-1d0b-11ec-baad-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632469485;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=bY4915iIBcBMnu2PBv7qqO8mlIYYQFdth88fwkacaFQ=;
	b=W+ZpUUPWqc6a+tPizLAcMEYdBh/+EvSwg0/DxBYdJGhbkgxJqkUfT/mBUV0VnTROSw1Bw5
	+btKEJeh3DD+8FD+ozwToN6H/BcvcjV057A9bDdsE/6zSue4XTsADh4hLrGVvKA6GAP3QQ
	yzoCt1jHriEa46q89TxAkaB9R6Xr9XY=
X-MC-Unique: aHLWf5b9OcmtMVFSng3zDA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kdnfHjkf2DXJsdaeTOB+SvNYjmHtDAzZ+Tq/PD2AldnjauzzXlr+m4noIWQEPYHFL0/6QtH4EXkkX3PE0YVz+osKRrKqIRk4y3NMaQHI6bjECAb9Q6fjwKBYSugXjRBJaLrc9O637uNPXWVtNLTjJgfRSHzW9jsETbLlXGY8GgaGQvY82UZfihR98ASutbxga6Nl+OboFbJ38D2aT3ZEr8jp+CnYMQwF5UQm/JaVvNTUddtsYlkmIeISLsOA3UAuhD54U0xfLXpoCtATo5ycGRpzE8PnNA5Tv7xP6aFm0nfXpCMPAhKZJIVJXLdWk0ja8yj9fkQXyknNPo/hbg4Xgg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bY4915iIBcBMnu2PBv7qqO8mlIYYQFdth88fwkacaFQ=;
 b=ESK+Nkd4Cyo7vI487aHccAyrI+LCD0miU2YaKbpb6DvmeRWJvH4yn3XwU+1lUJiNsGry0BxTJjoi1RHKIlwin58nQQI3DRB/byTIJfPiI26ns82kOf0ngPobkvYLhkyamfF35iq7v7zAssHTXz0TzUat302igOR1DiE965Gkh49N1A37LcqGZojjuL+6GIpKMA0saNjyLECsauZqs/cj2/O2L0sRDuePrN9seuebUKFbhfk8PSRSFOcdZcGcVkIGnNKpC9g62nOAmwOtN9LmgeJv6j9h073arb+baGIr2zJ9R6E0MBa+ECEJIkfBMpV0ItKULx84Ki69CCVWml3/iw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
To: Rahul Singh <rahul.singh@arm.com>
Cc: bertrand.marquis@arm.com, andre.przywara@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1632307952.git.rahul.singh@arm.com>
 <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <fe23eeb0-c31f-0525-dcda-f7cf548ea0cf@suse.com>
Date: Fri, 24 Sep 2021 09:44:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0047.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 301a8b15-7b5e-4e13-8f06-08d97f2f2bae
X-MS-TrafficTypeDiagnostic: VI1PR04MB7037:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70371C30E5A04787B4A057CAB3A49@VI1PR04MB7037.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CFmHxB5F7g6HzyPhEKlWx0dmz0XyHIC6idcgT6csarKvrTyLcmilZE4pfpp9STIVcTNf1GwoQWSomKFNcG0yMJAMHZL3a5jZFIqQfkExlplUwYcWL1X7AsDkRcvUEdFJ6wDpEJ0uK+d03mZSjvqzGluomTD47vSoq9E6w6UOyVPWVTyridGNcStoexTCIlQ8L1sG1OIgxeVuHD9PGRLMEjlO8Ypk5q7sRG9J6jqUVCMuJHz0HMOlXl2KDiBWkwFSoJGqfFRqLL+HkPHvL526IkX1D9PKieKN/zCipPCPKuCaCOgWGyetoV8wCtkEnelif3K4VbJq7u77ywX4UpwES0GvRZt/c7ndfQy5FzmOV7vxDJYcO77K9AGVM9piecXKpOGd0zMlxSD6saJNYhNdnaz6vuifM9SHFoZwlyeroV3KNk8DG6G2fBmYrUlRWyEv57pr9baekePIERMJrHklAnOsm4unvGA46CEFv+26h74ojDqtzc6eWhzoCrpYdlswFFNTcn9FZhrEpuLFg6Rnnh1lPGWlQDbeoxXq32l8k9wlpot3mdOwAevEBqqGNcOVljT0CwPn+LhutgAysRHcbIeTuMKVUBnMFf5/7JfGen/J8SsgtDhAO4FeYahkiCCROjgD66lR8Npdy99h0ymVwcptfTq5R9v4i6lYS3A3JK9gIfmw3uKSHgGWl1lX9pyV+jGc2F3+rxeCWY2jC1q+H7aNn04N5MkCTzXhllLHDai+MCcX6DdEpIjcCwDLiNoa
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(6486002)(7416002)(36756003)(86362001)(66556008)(8676002)(956004)(54906003)(66946007)(31686004)(5660300002)(6916009)(16576012)(2616005)(2906002)(508600001)(38100700002)(53546011)(8936002)(186003)(31696002)(316002)(66476007)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ak9jNEdhNk4vZFFVWEFaUzVuNFdwN21NdHc0a2hEVmYvbmE0TUg1djFUV0d6?=
 =?utf-8?B?N1NSNnVFeEhsR2dlRDBxWW1WNmVzR0swYWdXU2lJZjhlSDhmTk9KejU4M1Rm?=
 =?utf-8?B?ZTFvanlkbVZDcFZ5L0ZKT3hFWFdUMXpxMWpRZjRRdUpGY3ZCTW5OY2E1VEFH?=
 =?utf-8?B?dWgxdm9aem9mdGNUZjdFSlJJL21xa29ycGVUdjVxSEl6ekV4bjMrMkhGV2dn?=
 =?utf-8?B?TkVtL2FGeXZmSFdyOWh0QzJjYno5WTlxRUFVbkxzNXlrajg2Yk1RY2FmWEF1?=
 =?utf-8?B?Y1A2ckxZT0dGa2F6TStIaExKL2tpcUlnc1ZQUVBtcjlhaEViL0tQTENnTkVm?=
 =?utf-8?B?dW8xUDJCaXFxajA2ZjlpdjBlQlNhcUFyek1RL2lpcEJPQlo4Wjc0a0hqd2Rn?=
 =?utf-8?B?MC8vbDNGcjM5Y09QUXBKclFQUGFKMGxNWDV2WEV2SHFMbTRsczYwMkJTQ2xV?=
 =?utf-8?B?aGgrbWFoOFBranh0c3NqRXhYWmRIV3g0ZjlKZzRLcjNpdmRzLzFvL0JIdzBw?=
 =?utf-8?B?b29aNUx0eEt3aGEwUDh6VHZrTFUycjdwMUFIRURWQmRqa0ZUazkrRmVyRUw3?=
 =?utf-8?B?VEQzcU45d0o0MjQxeldKSWhqVmhhMTVxMEszemF0R3F3anVjMVNRZlhYNUU3?=
 =?utf-8?B?U1FScEN1ZzRmV3RIL0E5cmgvYXdocG85bDBFd2xLbjZMelVDQzRWR2lYMjdh?=
 =?utf-8?B?bkdabnNvbzR4VnVVMUpOcnJBclRvLytsc0tZUldGMVVPUFVTanJvSGNrWTM3?=
 =?utf-8?B?NnpSRDVYVndoUmFnSVhyRlVkQXJHc29VdkFBdDV3YVh4TCsyODhBZGVGZ0g5?=
 =?utf-8?B?T1ErVmVsZ1F6eGptMEpWSUFtWnpnSTZNeSsrR0M5Q2FjeTNWQ2VPVi8zQXoz?=
 =?utf-8?B?UnRZWEpCRU9JMkg1dUtaOHhZREtodFFEbE9wSVl3TTMwOGUxV0kvK3BWb3ZX?=
 =?utf-8?B?QUlGcDRIMTE4WHZkY0NxTktoMFoyTldGY2tmRE85ZnRUcERVRklqWmpmWGFv?=
 =?utf-8?B?NEt1ejRBVlUxYnNpWmE4dTEwV2RHL2E3OXRTUDJQQ0dDNXQrQlpBbW1XTnNU?=
 =?utf-8?B?RHBnNFZMWHJaeUJ2OC8vaFpnc3pmMzNwSFk4VTJjdzk1U0Q4Q0hDQjVzK3Vz?=
 =?utf-8?B?MUgyTmhRTEhTVUFVcmNBQlk3cW5hQ3N3NnJEZXdPVE16eHBKY0lRYVkyQkg2?=
 =?utf-8?B?cDU1OXAxSExNcHJLaGFqZjZ6NkpWWU5nVWtqTWtteVlYVGU2WjNUTVFPTlZ4?=
 =?utf-8?B?SFNnWDFsMmZIRVdIa1BPQ2NhS3R4QjhXcFBIdHB3QmsvdnE2VENBdFBPRnNH?=
 =?utf-8?B?bFFaWjY1WmFnV2RDYzEveVRNaVd4M1RURzIyK0haYjEwckljZTcwOStpR2oy?=
 =?utf-8?B?MXdLZUJON1hBU2hkUUsrMWtoNmliWXpRSjZVT0pSNXd4K2MrUm1wcDFhd0wv?=
 =?utf-8?B?S2hOdlVKanRHNFE5b1FNNzhRLzdLeVp6a0tWQkxHVDlZV0ZUc0ZHUWd6bkVC?=
 =?utf-8?B?VzliTXBBYkp5cmVjRU8wdDRNd0tTRk45U1d5SzErQ2NIdmlzN0xWdUxhQ0Y4?=
 =?utf-8?B?bENiRU1oRXNKLzhLK3lxLzExdlMrczhORUVJZ2xzRDVhM2lpcDF0KzlWUFQ4?=
 =?utf-8?B?Z04vaENwemJvZjBJUEl3WHgrVzkxZkxIRkRUSnFzSWVwbWJZREtpdlJSQWFK?=
 =?utf-8?B?RXltV1ZXUmpCb09rb2xXRjM4U1dIaXl0K1FvMUdsVS9SRG0zbENWOFVBbGJC?=
 =?utf-8?Q?iHyyJzfHmD35bybtp6PEwoXkgayhTVN4Qa7uRfc?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 301a8b15-7b5e-4e13-8f06-08d97f2f2bae
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:44:42.3456
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6kqDozmzhNoxbCyMpLtqkofLy4d1YTd5XV6nhhzSqJCHOb+NYJbTvhF4HXx965ZUJvGxohzyaKY0JQJTw4Cpgw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7037

On 22.09.2021 13:35, Rahul Singh wrote:
> @@ -623,7 +624,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>      unsigned int max_vcpus;
>  
>      /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
> +    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci) !=
>           (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>      {
>          dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",

While you accept the new flag here and ...

> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>           ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>             XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>             XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> -           XEN_DOMCTL_CDF_nested_virt) )
> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
>      {
>          dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>          return -EINVAL;

... here, you need to somehow reject it on x86, until DomU support
there gets added (unless I have misunderstood things and you're
aiming at enabing that support for x86 here at the same time). I
cannot spot existing code which would take care of such a newly
added flag.

> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -6,8 +6,6 @@
>  #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
>  #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
>  
> -#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)

While there was a reason for the padding blank after the first
opening parentheses here, ...

> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -41,6 +41,8 @@
>  #define PCI_SBDF3(s,b,df) \
>      ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
>  
> +#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)

... that blank ends up bogus here.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:47:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:47:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194980.347411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfvD-00060I-Ko; Fri, 24 Sep 2021 07:47:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194980.347411; Fri, 24 Sep 2021 07:47:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfvD-000603-HC; Fri, 24 Sep 2021 07:47:03 +0000
Received: by outflank-mailman (input) for mailman id 194980;
 Fri, 24 Sep 2021 07:47:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTfvB-0005z8-Rr
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:47:02 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d1e50fdf-72a6-4ef5-9eda-43ef928678e0;
 Fri, 24 Sep 2021 07:47:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1e50fdf-72a6-4ef5-9eda-43ef928678e0
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632469619;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=pTlBDqZdKY4LeD58fEyRgJYHin+WlU3LZwcyGbFp3TE=;
  b=WC3Xgaec/eLhhPJ2u4um6xpCu55boRAuye0zqouu7wMhCWxJBvxsVLgz
   3GZlmN9WM8Dqq653Q2lfpY64liYppd3U/kuuuriCbuUicyi6uvd3ud4pG
   sGjgAdmxhuSaD3gqL8lY7EEa4G5/MNFRBs7xryAlzq+o56H1idMtDVkn8
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: ysZPMm5LZL8YMZSypebsTMT5tLJt2ekAXAsbGEqn96Av9Sa+Tr48HiUyjlYWcQlYNogK/2Fd3d
 /zrIWYv2HBEs/GDG9jJ7kvuPPVCHRDVLAPYm27x1igt28jPAgOlm/mM567WN0ZV7+k6fcWIskR
 BG6txSQLeoM3yxdgpaTvXjhYMz3tJUmBU6GNKPSCedro4kpekTH5B7hg/dFqRWYAfFzaWdJJ0K
 F6VcALjVHm/hfPJ2jkMhDdQfg7r/mltGH3bQVIEeO5fzDNl4oJmI6Y8E4YazCO++90AFtuK9hu
 AllwU+X7f21sPz4Uk5HNSirc
X-SBRS: 5.1
X-MesageID: 53536726
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:dBBWY63s8t9I7YHPMPbD5bd3kn2cJEfYwER7XKvMYLTBsI5bpzZWy
 WUcCGuCPfiLN2v0fdB+O46ypEkCscXUyN5hHVc+pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywbJh2+aEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhlI972
 uRSst+McjgWAYKVmqMmeiRUHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1EnMMsIdOtJIoCknph0SvYHbAtRpWrr6DiuIQFh2lv3J4m8fD2f
 pI/OSQ+VQT5YTZqOG80OMsFkd/xmSyqG9FfgA3M/vdmi4TJ9yRz36LqK8H9YcGRSINemUPwj
 nnd423zDxUeNdqe4TmI6HShgqnIhyyTcLwVELq05/t7mmq5z2YYCAAVfVajqPz/gUm7M/pTI
 lIZ0jAjpq8z8AqsVNaVdx+yrWOAvxUcc8FNCOB84waIooLE7gDcCmUaQzppbN09qNRwVTEsz
 kWOnd7iGXpoqrL9YXCA8raZq3W9IyERKSkFfjQsQg4M4t2lq4Y25jrQSv5zHajzicf6cRnx3
 DKivCU4n68Uj8MAy+O851+vvt63jsGXFEhvvFyRBz/7qFMiDGK4W2C2wVOE5+RxM4HacmK+g
 0crwsrZ18kzAZ7YwURhX94xNL2u4v+ENhjVjlhuA4Qt+lyRxpKzQWxDyGohfxozbK7obResO
 RWK6F4Nvfe/KVP3NfcfXm6nNyg9IUEM//zeX/bIZ5JlZpFrfWdrFwk/OBbNhwgBfKUq+JzT2
 Kt3k+7wUR726ow9lVJaotvxN5dxnUjSIkuJGfjGI+yPi+b2WZJsYe5t3KGyRu449riYhw7e7
 sxSMcCHoz0GDrakPnONrNFOcQxQRZTeOXwQg5YMHgJkClA6cFzN9teLme9xE2Cbt/49ehj0E
 oGVBRYDlQuXaYzvIgSWcHFzAI4Drr4lxU/XyRcEZA7ys1B6ON7HxP5GK/MfIOl2nMQ+nKUcZ
 6RUJK297gFnF22vF8I1NsKm8uSPtX2D2GqzAsZSSGJkIsE8F1CXoI+Mk8mG3HBmMxdbfPAW+
 tWI/gjaXYACV0JlCsPXY+io1FS/oT4Wn+caYqcCCoA7lJzE/Nc4JirvoOUwJs1QexzPyiHDj
 1SdAAsCpPmLqIgwqYGbiaeBpoavMu1/AksFQDWLsefobXHXrji53ItNcOeUZjSBBmn6z7qvO
 LdOxPbmPfxZwFsT69hgE6xmxL4V7sf0o+MI1RxtGXjGNgz5Cr5pLnSc89NIs6lBmu1QtQesA
 xrd8dhGI7SZfsjiFQdJdgYia+2C09ASmyXTsqtpcBmruncv8ePeA0tIPhSKhChMF5dPMdsok
 bU7pcobyw2jkR52YNyIuT9ZqjaXJXsaXqR56pxDWN33ihAmw01paIDHDnOk+4mGbthBPxV4I
 jKQg6ae1b1QylCbLig2HHnJm+FcmY4PqFZBy1pbfwaFnd/Mh/kW2hxN8GtoElQJn0sfi+8ja
 HJ2M0BVJLmV+2Y6jcdOaGmgBgVdCUDL4Uf20VYIyDXUQkTAurYh94Hh1TJhJHwkzl8=
IronPort-HdrOrdr: A9a23:9v97I6wZYCRX5RNPHYDuKrPxveskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjmfZr5z+8O3WB3B8beYOCGghrSEGgG1+XfKlLbak/DH4JmpM
 Jdmu1FeaHN5DtB/LfHCWuDYq8dKbC8mcjC74eurAYccegpUdAZ0+4QMHfkLqQcfnghOXNWLu
 v52iIRzADQBkj/I/7LS0UtbqzmnZnmhZjmaRkJC1oO7xSPtyqh7PrfHwKD1hkTfjtTyfN6mF
 K13DDR1+GGibWW2xXc32jc49B/n8bg8MJKAIiphtIOIjvhpw60bMBKWqGEvhoyvOazgWxa3O
 XkklMFBYBe+nnRdma6rV/E3BTh6i8n7zvYxVqRkRLY0ITEbQN/L/AEqZNScxPf5UZllsp7yr
 h302WQsIcSJQ/cnQzmjuK4Fy1Cpw6Rmz4PgOQTh3tQXc81c7lKt7ES+0tTDdMpAD/60oY6C+
 NjZfuspcq+SWnqLUwxg1MfheBFBh8Ib1O7qwk5y4KoOgFt7TNEJxBy/r1Zop8CnKhNAqWsqd
 60dJiBOdl1P7srhJlGdZU8qP2MexrwqCL3QRGvyGvcZdQ60lL22tXKCeYOlauXkKJh9upEpH
 2GaiIAiVIP
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="53536726"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EL0WcxrfLJRmJPDoxpX6GwHnKYX8Lyf7EgkRNb0OhrpubbRyfLLXDLkMkXcwH/tU9PuzdoFmQ2KQy1FxFTu3RO5CQ2szhd08olKCfslk9OVcNdFXNJmWG6ielfRCzaWWRaMeUfqexE+67o/pgbEw89HlBcKIzPIK+nIWZwXZjbSS31wKVuBaeqYX5cK/kuLuv4iVZhMSA3fyQOL1f1yLRgz5n5dET7+BSvNd+63pkEoe0cR5+MxgHUqhg89JPr8ctZGA5D9JNnJgpT2l5N7juWNzreJhAj8lAYmwVQvOX/9lPS3sYOmpgReSR7gU6XYDyscpXKDdX/khz9uWfKxngQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dDWIilNdVyaucOEu4Us4hr6C1PWIhQ6ULkkUa/Vip1U=;
 b=R6KU7tE+kR7HuPFpejSRxJMuw2qRpIeEuY4BXNitVO+xC8n2kot9I3eX4t1oi4VZJUtEGXlLLFKz1/ZeOsXvb0q1/kN9kDO9Q44X/UgVocKSieYBcN1mg5q33wJ9CwW2lz1Gzcw2e7J/bogTQdJbQlcq57ivqYXlGg10jnB6ErRMHJeEGq/CsfeaoaYw3IsJ3EBkGw7vdcZfwzvizixRyIJCrnUsfiX5HebM9SXIDQkNuausBrWIS1viWr1WR3FWvhOszpMJ/9vDWoaHUnTuDem9HkLzN6sVCDLnawvoBoral8Wvm1EHKkLnPBWUU3EuC0F6/jn12TPQS9Uc4B6Izg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dDWIilNdVyaucOEu4Us4hr6C1PWIhQ6ULkkUa/Vip1U=;
 b=B23gtK5nKppQcSFUqQkc9YXN3WSGvtU7j+vZXr4sJEmDrPGIkS93TTPWEFb0o8awe+zuaF8BnFpdKUkrHOFI7H45GvBqNTlXZq71wwuhgMhEQ+bZ19emaE2PNwvoE8RF+aQWddZYYPsusTgK2QOTwy+TSBUxbWU3ODCLMKGJuqU=
Date: Fri, 24 Sep 2021 09:46:44 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien.grall.oss@gmail.com>
CC: xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
	Christian Lindig <christian.lindig@citrix.com>, David Scott
	<dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
Message-ID: <YU2CZMvlyGJvqJ3z@MacBook-Air-de-Roger.local>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
 <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
 <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
 <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
X-ClientProxiedBy: LO2P265CA0502.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:13b::9) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b5d0688a-e44c-4e0e-d82f-08d97f2f77b1
X-MS-TrafficTypeDiagnostic: DM6PR03MB4602:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB460240F10193AEF943A8DD2F8FA49@DM6PR03MB4602.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: cXABK+oPks2ZNZSx/qBoGvtHT/LjyTkZO4F2tmPTAGth88+FTaXDXFNZbVGtfgKxKzMwlTwq7B9EdtcAVm9R2zJXtaAfCAOaG4ZD6eDE469l0eLYCTfm/LXvmOZIk9aAYO9FDt7CCQjSZBfJRr7T3cfM7oJuiBBIAyvDXcwHGRLbx5w5JDiSDzcuj2+07ea1Uo8UNeK3E7HuhWxUvB53Qe0s1kP2HzAuEnahkJnyuvsc1Z8vuCEVeCGnt3Qj2ogQW8ak9YXXoH/CKV+ZPH4WngcBqZS/miGqrkIzuJ8q8lgdLaazFIGi61T+dqFWFrGUin7y0iexZlpY5Rz4W+bzbe/JokMAxN+8H8c4NmRF9lAAcHpRYh7aGCAMTiNVgTu0FLjMBAYytMUWPZZJhC4PXIUy8EC+RN64N8jpXAn/RKVfnmiRshm8+sg49RJFOuONGi91sB1Q0RO1YMm//9lCpnXtgtLjJBBokKWBT/IqYZuflJAxQcs8o72+Yq4dludfYt3jR03G3WAA2HVWxJR99kJxSVj5H5vzFaSq6LIqTlX3bYo200RVhiDhGozVC9HPlEI8kIrMp1/XbGMuCx2X0pVrpLiOhqSilMwhPXVmy29mD7QZqE3xEeTYoJr1RNy0hnGalmJud6uPU7oKgfM13ax/agmX/1fVNK+XbOWODbfE1fIFnBEIjfT/3hluqnkqrMAI2Juc7BnWWOazexlPAAV/H4FuQzTP9jN+YT41xH0=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(2906002)(6916009)(966005)(6666004)(5660300002)(54906003)(38100700002)(8936002)(508600001)(956004)(9686003)(8676002)(6496006)(6486002)(85182001)(53546011)(316002)(26005)(4326008)(66946007)(66476007)(66556008)(186003)(86362001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cThvbkNlOGF3OWJ5TjBqTUhtSXN4Q3AwYm1EbVZHZ0U0ZzlTdTBXK3d1Zld6?=
 =?utf-8?B?dGlTR0NuNS9pSEZBOWtHTWJMN1psM2VKdkE4TWp5WVhBVlYrdG5OMmo2Sko0?=
 =?utf-8?B?bTFlNW9qSzdIWnhoOTh5TWNHWGpNZk1ob21tbE52MkJDajdFZ3FxODFDcU9m?=
 =?utf-8?B?UHI1YzAxemQ0Mzh5VVdrTHFIQVNjRTJ6amVRWU0wbmJab0NCa0ltUXA0SzNE?=
 =?utf-8?B?TUZFaFNNaUw0NVQ2c1lSMk43SU5saXJ1WWtWS2d6dnN6QWlBSW1GUzBZeGJq?=
 =?utf-8?B?enZ3cFdZWkJiYzE2RDVIZ2FpY3ZOU0IzYTFOdEZ6ZGVRNzNRSXNZRUIzQStX?=
 =?utf-8?B?VFFWQ3dXSGMzWTJsUEhZcXp5aXpmQ1VwQ1ZPMElrK2FxZ2N4aTNYWk1wMWYz?=
 =?utf-8?B?MjF5aWJEYW9ibndxOWszRkZ6YzhPU2FCY09oeWtwK0QvaHRoaXRiWmFoQVBw?=
 =?utf-8?B?Y2N3WlFPQkVvUHhkcVVlcFJNOHNtbTZoRkU1QnVVV1Q0K2pWaWJxQ3JHcG9q?=
 =?utf-8?B?K2Qyd25UWnRlQWFPS2V1Lyt5K3VyYnkyWDdWTFB6UWRhcjdyY3YwalpiS2Yx?=
 =?utf-8?B?aUMrd2VYOVlQU1U1S2JWNFJnNFhCaml1WURMNXJmeFBMd3BCSmIvcUg5Vmh1?=
 =?utf-8?B?VHRnbWFuejlmZGd2djI3VTBkOHhZZXp5eXFaTHJJckhERW9STUJYQndBQ1JL?=
 =?utf-8?B?UHFwdmFQY1NUc2c5WkltRVRIQ0FEZ0J2RGpNZkxUYy92dnN2SkxHM0kwaUhP?=
 =?utf-8?B?MkdDNCtyZDloNEVRSUdVY0hUVUJ5dldodFg5M2owNlFMZWptVXBJVllnLzlk?=
 =?utf-8?B?c1JmVHM0TWthRUs0VWNVS1RiVWR5SEJwTStvRmlscldTczZMQkZTR0pHdW9X?=
 =?utf-8?B?R0c2TytlSjhEMXFxcDM4ZGlIRCtCay9GU09KS2wwcDRtR3M1RGlVSlZka2N6?=
 =?utf-8?B?b3BhdnhEY3k3b1ZDdFhqRytWQjBVRzk0Ty9qaFNIVlZBR3JzZGhzZE9UbGtW?=
 =?utf-8?B?eHdSQ21IWm9JM3ZQZnM3QVhWYjAwOW9CMUx4RFN2cWt2aCthSFRqcEhaSUJv?=
 =?utf-8?B?QThaQWIwY21qcXRqcWVyeUltOUpia0NjWkZ5SU5SWjZkVnlId3VvdWp2d3RN?=
 =?utf-8?B?VTNyQ0tlL0ZJVnVUcng1V1Nsd3d1Rk9DTzJDNHN1Mzl2dkdwZjh6akI4aHZ5?=
 =?utf-8?B?MFZwN2xPV0h6V3VvUS9XQkV4N2JsSWt5OHJuQmtiVk9SUU53OU9ENHUzSDZx?=
 =?utf-8?B?MEVUUVRKQWhLd1ZvRXZJdGt2MFY2bHNkM1o1cXNrT1QxeGd2RVJSTXJmSGsv?=
 =?utf-8?B?WnR5cXdNV08zNFJ0WE9ORGoyd0c1dkdIdVVKSW9YMkVEaFkwdHZ6akwyZG9Q?=
 =?utf-8?B?ZjBmVE16ejYvNDY1WnIvK3Q1Q25sVGpYOTJnSDhIV3lYYjQzemZyaU5CMCto?=
 =?utf-8?B?TDEzWkhBeEplQVJnNmhNblQ1aFJBOTB1ZmowTGRTZ1I5dS9rQ2FlV2dYZU4v?=
 =?utf-8?B?Y2JwZFZkV2FVZ3RxYkgwR29LWU5CV3N3dHl4NktTSGxod3pzMnBSUEJRSGV1?=
 =?utf-8?B?M2ZZVEdaNDZOT0lNcXJTSWgyT1JrT1FGVWdvNE1nRGFDSkJQbW1yOG5tek4y?=
 =?utf-8?B?eGRLd1pxQytWRlFyaUV2ZkY3clpIb3ZTQ3RmR3JseTV2bDJCWkZyNVBLVHY1?=
 =?utf-8?B?SGRPZXZQMTkxSVhaYktKQjM1Y3JVSTFNREFKWXg0aTVKVHFjcHR5dTBxNldH?=
 =?utf-8?Q?OS37+jpcatmCrWnVMSqz1J6OqiOz5JP71/h0BCM?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b5d0688a-e44c-4e0e-d82f-08d97f2f77b1
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:46:50.0917
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tiGcIXyei/vR33g3T8V3+BhoZsMKCbRIwPITLn0H6ezOJccTTzhWsfGsdO6JtLf/lbQLDFREDEUyWip78F5vvQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4602
X-OriginatorOrg: citrix.com

On Fri, Sep 24, 2021 at 07:30:51AM +0500, Julien Grall wrote:
> Hi Roger,
> 
> On Thu, 23 Sep 2021, 16:20 Roger Pau Monné, <roger.pau@citrix.com> wrote:
> 
> > On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
> > > Hi Roger,
> > >
> > > On 22/09/2021 14:39, Roger Pau Monné wrote:
> > > > On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
> > > > >
> > > > >
> > > > > On 22/09/2021 13:21, Roger Pau Monne wrote:
> > > > > > Hello,
> > > > >
> > > > > Hi Roger,
> > > > >
> > > > > > First patch on the series is a trivial change to xenconsoled in
> > order to
> > > > > > use xenforeignmemory stable library in order to map the shared
> > console
> > > > > > ring instead of the unstable libxc interface. It's reviewed and
> > ready to
> > > > > > go in.
> > > > > >
> > > > > > Patches 2 and 3 allow setting the host wide command line `gnttab`
> > option
> > > > > > on a per domain basis. That means selecting the max allowed grant
> > table
> > > > > > version and whether transitive grants are allowed.
> > > > > >
> > > > > > The last 3 patches attempt to implement support for creating guests
> > > > > > without a grant table. This requires some changes to xenstored in
> > order
> > > > > > to partially support guests without a valid ring interface, as the
> > lack
> > > > > > of grant table will prevent C xenstored from mapping the shared
> > ring.
> > > > > > Note this is not an issue for Ocaml xenstored, as it still uses the
> > > > > > foreign memory interface to map the shared ring, and thus won't
> > notice
> > > > > > the lack of grant table support on the domain.
> > > > >
> > > > > I find a bit odd that the Xenstore support is conditional to whether
> > grant
> > > > > table is available. Are you expecting domains with no grant table to
> > have no
> > > > > PV drivers (including PV shutdown)?
> > > >
> > > > I don't really expect much, as having guests without grant table is a
> > > > developer option right now, if someone wants to make use of them for
> > > > any reason it would need some thought.
> > > >
> > > > The other option would be my first proposal to restore foreign mapping
> > > > of the xenstore ring on that case:
> > > >
> > > >
> > https://lore.kernel.org/xen-devel/20210917154625.89315-6-roger.pau@citrix.com/
> > > >
> > > > But it's also arguable that a guest not having a grant table should
> > > > also likely prevent foreign mapping attempts. Plus such foreign
> > > > mapping won't work from stubdomains.
> > >
> > > There is another option: extend the acquire hypercall to allow xenstored
> > > domain to map the xenstore interface. This would require more work, but
> > at
> > > least it would avoid the interesting dependency on the grant table.
> >
> > Xen isn't aware of the shared xenstore ring page currently, so that
> > would mean introducing more knowledge to the hypervisor that what's
> > strictly required IMO, as Xen has no business in knowing such details.
> >
> 
> Well Xen already knows the page for HVM/PVH because the guest retrieve it
> through an HMV param.

HVM params has always been a bit weird IMO, as some are known and used
by Xen (like the identity page tables) while others (like the xenstore
or console ones) are merely used as a way to pass information from
the toolstack into the domain.

> We only miss (?) the PV part.

Right - ideally we should use the same mechanism for PV and HVM, which
would rule out he piggyback on HVM params.

> 
> > The grant table slot used by the xenstore shared page is just an
> > agreement at the toolstack level, but not known to the hypervisor so
> > far.
> >
> 
> Right, we need to find a different way to provide/map the shared page if
> the grant table is not present.
> 
> To me the acquire hypercall is the best way to resolve it as Xen knows
> whether the domain will run Xenstored (at least we used to have a flag) and
> we can do the permission control easily.
> 
> Do you have another alternative?

As said before, I didn't give much through about a practical use case
for this. My main focus where patches 1 and 2 (which sadly seem to be
shadowed by this no grant-table option) in order to have more fine
grained control over the grant table support on a per-domain basis.
The no grant-table mode was mostly the cherry on the cake.

I could see people using no grant-table also likely wanting
no-xenstore and no-console as to prevent mappings from other domains
into it's private memory altogether. Then using Argo or a similar no
memory sharing mechanism in order to interact with other entities. So
I think we shouldn't over engineer this xenstore usage without grant
tables, as it might not even be relevant as a use-case itself.

Using the acquire hypercall could be a solution, but I think we would
also need to introduce a new hypercall then in order for user space to
be able to tell Xen about resource areas, so that the xenstore address
can be provided to Xen without (ab)using HVM params.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:49:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:49:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194987.347422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfxZ-000728-6g; Fri, 24 Sep 2021 07:49:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194987.347422; Fri, 24 Sep 2021 07:49:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTfxZ-000721-2n; Fri, 24 Sep 2021 07:49:29 +0000
Received: by outflank-mailman (input) for mailman id 194987;
 Fri, 24 Sep 2021 07:49:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTfxY-00071v-DR
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:49:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8262cc89-d26e-4d6b-8bf7-2ecf592eff9f;
 Fri, 24 Sep 2021 07:49:27 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-Nu6-gG0JOfWQ6jNbHziwIA-1; Fri, 24 Sep 2021 09:49:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5167.eurprd04.prod.outlook.com (2603:10a6:803:5b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 07:49:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:49:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0039.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 07:49:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8262cc89-d26e-4d6b-8bf7-2ecf592eff9f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632469766;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=p8uCJz+MYkeuLo/efHyUcBQn4gVf/Twqd4VF2Bc6Q0U=;
	b=WfC+blyvELkp8k2OhIEp7rf5zy/MxJb9IjIn90Z8j77TmAwUFudG3/ax7fNCC5bnPBgxIs
	EKo5JWO97EUpgLQB/f4XKYZyKr+/JMlLA2lgE6CkqlZyZQEKoSwiWnBxVSnJPASBcCcj7V
	e35pr1+bKBQPEq6LZ/yPSqHQE77xDZI=
X-MC-Unique: Nu6-gG0JOfWQ6jNbHziwIA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DMoLHdhLLsuO/VvRCaUelXIBTTWXIerob24J3+9es9I85Vnu76j13M8fTvtpKbzoS9ut6onhwzRhJq52JQN9hlpVbogOSTh+p+Y2InrbCICy0mGQl3l+bshpGUHAzm6yJimkPh+mxi2xu4bDuQTsg3tN+Wy3Jxqb39DsAPVTMzV/4sClltDhWXsnjDb5M8SeezFjnqu6h31b47WdDti8nbnTUqaz1VxPY960tN8ieCDwcEKWDMbrX6bJq//8CkjYsCl+nqoCmff5WjtUM0XXlSpwv/M/dJA2ya6pE3A0Ku3gU9TljBoXu/U2PHgUFlyo6gY9y5d/i052KOjTc5mbEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=T07ergxdZj5ReZ6Vn90qMN6EcVeBJ/TX1/I/NHLmnFc=;
 b=aZnl/eY094kJuPsrwvW9MJwc5cgGxhC97WPsDkLQUUa4tFHejJcPkILmeaX6moJnjM9YGy4EDuD3TsqtIhO5Ar5KPwUjnj3R28gWlaslmLRY9YTKRklMC5X3JKHStT0cmHtGjXS+bJlvxkhWPdQh2wQmw1Z5MIJBUqMl3pEAP5ZbmJFYhwOHnQQJH0fxDsc6c8oo8PxxgEuRVQu/ZFazGeebnhPHgorPO3cuOet+4iq/V8n9VAOIEqJqUwOgNHVPpjgvyQV2Cu9swaZCK6vAZVXwMeUQUh8DgDp9TRvl2GOfCRHfWSDIy91T5tTifpJ0o/jebWQ2V7xuE39H4Zitmg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 0/6] gnttab: add per-domain controls
To: Julien Grall <julien.grall.oss@gmail.com>
CC: xen-devel <xen-devel@lists.xenproject.org>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20210922082123.54374-1-roger.pau@citrix.com>
 <69208166-2356-cc8a-4e78-2ce8d6cddcac@xen.org>
 <YUr525A3B0cs2rBo@MacBook-Air-de-Roger.local>
 <b93a13e3-2ad9-aac4-7f87-b290aa4c3953@xen.org>
 <YUxi2DeEXhsnUTi7@MacBook-Air-de-Roger.local>
 <CAJ=z9a1MxDKg3pB0Qq2JvAfDJ6LHsSoYmg-m2Xotxi_EB1jb1g@mail.gmail.com>
 <d3a314ae-a2bb-58da-b98e-bc03a049bc1d@suse.com>
 <CAJ=z9a2gub_Dn6P3T2H05kwB6TUHf-y=rBvxcXpYaiGNrAXqyw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <be9beaea-54ee-3f13-073b-fa069a0dc81a@suse.com>
Date: Fri, 24 Sep 2021 09:49:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CAJ=z9a2gub_Dn6P3T2H05kwB6TUHf-y=rBvxcXpYaiGNrAXqyw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P193CA0039.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae305402-d6a9-4425-29f6-08d97f2fd28f
X-MS-TrafficTypeDiagnostic: VI1PR04MB5167:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5167A2890BD217CA996EB8ECB3A49@VI1PR04MB5167.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UtC9vk1ALeqZFoLxWE5XVwkmlbYLJS9qhhmQhldMl3/trhmV3qj3mmWPxIVzQQ8x71sgNbiON7GOs2Jl/YXPr5rgq/PDzJ4TDclhDozpHhGutCx6haTg9/avsNJkaguOdwt+1bqkR7ygGahTCYsYZZ8jQ3mIhpC0Jrlnvc+yKJ5LLOmgAZd9WWxO/Ul6i7JG6ZYOsz8y+PjOF7nqiDaaELDSChgoDUNo/uANmtHV3VC1WxC6vAyQ48yy5uZPWUS6FvJozJJ0Q6oWWwfdoSZ2bDy8JrXgVdW0c/1QBbKWSYNdU3wmy9YcPeBFnpPf2WS+k03uFpf2jHI3lnxcSF3EiuxY8Gdpjwwc8Pt3bRuZGMY2QQx0V3W/lerrNLKYzYIqZl3ayVVE+A5TkGaJ9f94Tfd4lJjx7JtuLbbWHA0AxZwbKXYsTtRAAWQ19Q/jPhycXOFm6377+V1Xjob15JOJU+j1Bi33r46hkR0twEj6On/w5O1J6EdchYFBp0Uo3vqvr7YXWm1AgrUpZM3yNfzbaletk1K6yy/pWsrxp0jqoXl2W/Ymplrr4/E4+amwtPrCXA/Uooo8k3pIOYuIC1Jqd4lm2QuTQ8zWNOf60RlHYB2OZyhGP2YOPSb/Kj+0s9YrZM2uSu33mru1nhh6HKMLAStbX5Hx/JQHT5Yv1XdtwLFBONhlXiK43bEpSWcVMYBDmuGKwdBMnSZ6iL/Ht7sjEXAXm6/bbEobVw0pFaxjRQo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(316002)(956004)(2616005)(7416002)(6486002)(186003)(16576012)(6916009)(31686004)(66556008)(66476007)(66946007)(5660300002)(26005)(31696002)(54906003)(8676002)(38100700002)(2906002)(36756003)(8936002)(4326008)(53546011)(86362001)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?zlW4X4REhhKiNE/MevjsvoC6/TK08qr9Ltr8frMiH0tqQxQ1QWqdH+HwN8u7?=
 =?us-ascii?Q?Eb3u+Y1zkus183hNdzNpIIcvQm2VPFLUSrs0DvPPeOVX+3fDBf6hZejaAJOe?=
 =?us-ascii?Q?0GgO+Wrspky3VhyesJKtorSQG6pt7Up310suTLZEDALumXXqEw+DDm1mjlYo?=
 =?us-ascii?Q?XjDOISbJXMBHyPFymVK99FUNF2HVznN3zBuPPLyYFrV+BxLObB0Rzd4wAqkM?=
 =?us-ascii?Q?eLIS7hzLxCfU+aq7BYXvrPrnlYKW1ubLasYNHnsy3HOwn/FePk8kOhRuFDTr?=
 =?us-ascii?Q?6mw3n8piZwG423o+/q2oV3+t5kODicQ/4Xm3hapkJVIfXmKAcu7wHy0ZDeey?=
 =?us-ascii?Q?IBIQc1/gcJ2DcPpJgs08esynTlTLNziSXiaClgm3vLCdBJtG7G48uzwhhNFp?=
 =?us-ascii?Q?4O4fbV/VYlrNa55WU/yKhT73iB29NatRSOw1gpIRgowizhTWkYy9Y9qND0Bw?=
 =?us-ascii?Q?e3IB1CCVUbzZKYGEOKYWIhu/IanugZ/ezHSzn9QshfWw0egUGbW1WGEb+haB?=
 =?us-ascii?Q?dFU34OV6G9DMZ0RVkVXd95T3kB/u8gJ9C0d6BoJm9laC2xPcOFjj2lp1yWL6?=
 =?us-ascii?Q?rl8tTXTImQNmPa4psNdGG5NQ86bRlC1HXDvg0Wp7q/KIT8m0qz6hkBJWV9yp?=
 =?us-ascii?Q?B0IK88D2X+5JOW7I1C2f0kvFndaqxOOQtd+xD2jVYtiEbyXTY5SVynST/qer?=
 =?us-ascii?Q?ZGKKJHnW8lyhOf8mebFsMSQ63/zMTKLs7kfNBzhf0QXmzHVsyRLGXeR0MT4U?=
 =?us-ascii?Q?+6bOTnjZImnAYPJ2VHoloVh7mrXbI0u4nwkNjAygzTjCPJO2C5VpQi+mEmR8?=
 =?us-ascii?Q?ApWFcCB/wVYFow3rn8eLy7Tt/AHO2Esw+AH5bXNNKqTP6S3J7Ij9ZuUaG8Qx?=
 =?us-ascii?Q?gLmj/VkUZzgW1ern9efay3TuU++lN4u+adGKLszVUgLNU/raMusTNcz76sn3?=
 =?us-ascii?Q?i+qyoVvdY73hjiFmb3S6QMHkbQyyFXPqxenlUuiwHN3tG8LWS2Ggdpo+kpVj?=
 =?us-ascii?Q?DE6+iC31iGKXGwApj185i8BG6DYIOMlBMVIwraZwtCoqVouydl49w4tSN7uZ?=
 =?us-ascii?Q?Ovv+XGrdzjmoTrIVFk88v4Kl3nnVR5249RQPx9dw8ImeHHZk7akLxcniRdP0?=
 =?us-ascii?Q?w+aJoV55rdcdBk7Gftfikas5uRExmDWCTuiDTdVlhzMVquEObWbzKTZUskRX?=
 =?us-ascii?Q?kfVUsYte1gEvvE8gEdC0GReuXfdVnH36gT86GoP+LO5fI5Wq06vCujabTlJo?=
 =?us-ascii?Q?r6qrj9/7owBEPhI1+w9aaiamiY3j0+LkaIRRujIs9STrMsMV6yTrEOZY2/Kz?=
 =?us-ascii?Q?l3hIyT426xrl7tiLLCgxUTgy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae305402-d6a9-4425-29f6-08d97f2fd28f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:49:22.3563
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zxxrXON+o8YMtMMd40xonpLCpGrsVJb+XQ7uCI+UDeRK3seVLXk1KFkmEdctoug7LfyV3tR/mdAeq5eRULeMSQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5167

On 24.09.2021 09:30, Julien Grall wrote:
> On Fri, 24 Sep 2021, 11:21 Jan Beulich, <jbeulich@suse.com> wrote:
>> On 24.09.2021 04:30, Julien Grall wrote:
>>> On Thu, 23 Sep 2021, 16:20 Roger Pau Monn=C3=A9, <roger.pau@citrix.com>
>> wrote:
>>>> On Thu, Sep 23, 2021 at 01:47:37PM +0500, Julien Grall wrote:
>>>>> On 22/09/2021 14:39, Roger Pau Monn=C3=A9 wrote:
>>>>>> On Wed, Sep 22, 2021 at 01:57:02PM +0500, Julien Grall wrote:
>>>>>>> On 22/09/2021 13:21, Roger Pau Monne wrote:
>>>>>> But it's also arguable that a guest not having a grant table should
>>>>>> also likely prevent foreign mapping attempts. Plus such foreign
>>>>>> mapping won't work from stubdomains.
>>>>>
>>>>> There is another option: extend the acquire hypercall to allow
>> xenstored
>>>>> domain to map the xenstore interface. This would require more work, b=
ut
>>>> at
>>>>> least it would avoid the interesting dependency on the grant table.
>>>>
>>>> Xen isn't aware of the shared xenstore ring page currently, so that
>>>> would mean introducing more knowledge to the hypervisor that what's
>>>> strictly required IMO, as Xen has no business in knowing such details.
>>>>
>>>
>>> Well Xen already knows the page for HVM/PVH because the guest retrieve =
it
>>> through an HMV param.
>>
>> To be honest using this in such a way would feel like an abuse / layerin=
g
>> violation to me.
>>
>=20
> I can see how it can be seen like this. Do you have a better suggestion t=
o
> be able to map mapping without the foreign mapping interface and the gran=
t
> table?

Well, as was mentioned, PV would need covering anyway. And I think just
like with grants the guest should consent with such foreign mappings
outside of the "can map everything anyway" category. Hence I think if
such a capability is indeed needed/wanted, it ought to be the guest to
announce this page to Xen.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:54:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:54:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194992.347432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg2V-0008P9-QV; Fri, 24 Sep 2021 07:54:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194992.347432; Fri, 24 Sep 2021 07:54:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg2V-0008P2-Nc; Fri, 24 Sep 2021 07:54:35 +0000
Received: by outflank-mailman (input) for mailman id 194992;
 Fri, 24 Sep 2021 07:54:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTg2U-0008Ow-BZ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:54:34 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.47]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a5d1489d-1d0c-11ec-baad-12813bfff9fa;
 Fri, 24 Sep 2021 07:54:32 +0000 (UTC)
Received: from AM7PR03CA0021.eurprd03.prod.outlook.com (2603:10a6:20b:130::31)
 by HE1PR0802MB2219.eurprd08.prod.outlook.com (2603:10a6:3:c3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.16; Fri, 24 Sep
 2021 07:54:30 +0000
Received: from AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:130:cafe::d7) by AM7PR03CA0021.outlook.office365.com
 (2603:10a6:20b:130::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 07:54:30 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT012.mail.protection.outlook.com (10.152.16.161) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 07:54:30 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Fri, 24 Sep 2021 07:54:29 +0000
Received: from d775d31bf7b9.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4C65841A-226C-48D2-BCBF-6C281A901D03.1; 
 Fri, 24 Sep 2021 07:54:18 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d775d31bf7b9.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 07:54:18 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB5466.eurprd08.prod.outlook.com (2603:10a6:10:114::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.16; Fri, 24 Sep
 2021 07:54:16 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:54:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5d1489d-1d0c-11ec-baad-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gzJBlzNEV/yqoCzrv32maG35E4hu99vpWuR7s7TJBNM=;
 b=YhkgS6AFW2sqcA+obLEjGz88/HoOTwPmfsaUV+0IIINfOsPv9pukcalo4b1ZnzohkzhJqf73SU9HTz32gRz9tvOO1356S51PZlg/3o3eO4OWEnFVe6LcZIKp964eH5L2wwQJ9+l4fCO8e9xTeoynDLyzO3RfB7CmvlAjXkvZ4AA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=a0PEYmk0Osw6ZHS8oWTHu/XzQ73Bvk/HGDUykCsYvjEo7mfCGDOqxAxe/aAMo60fbPTGy+WvVe9cRR3LTO67LmHDTk48jen6k1Of0ZZ9dKHOZQhupEEUVEVAF/giAWdhXQGP77JBDpIM9st5S6dx1MHlWvKWUP9Gw6E56rHAPQzQinYQruDSKcxxgNv1Kb6dsARG3sbtfe/ou/sbbd/TJAfkkZcv7QLWH//29F2PBIb6YJQaxAqyLLeMd6a3LeTjU1tcKSQndSuQDo+VORFWRurrkMidYlY55HJHckaRUwwzTS+tMCoYAS1GZHkR+tUJj4WDz/Nhv/6fohftpHhjnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gzJBlzNEV/yqoCzrv32maG35E4hu99vpWuR7s7TJBNM=;
 b=R7AjvHHazirLh2lqDF0rGYue3bYsf33CBLiXvpGVbECZcze8skGGr4GUpacZaEcz3MhxfG9gVprOiasGcs+3pjLs+ELi6ExW/CydMIZW4sW74yggB4INTmRnsmJ1Oh3Jlb6XHbxXBMOyMF0T4tr9aDSo239HXCD2xIKLhSmYaHAGcwC7EJFFbH0J9Nb+uAewa8hwZhrR5Shy4PjKfgG4IhhZ4uOL17e0n/V8nt7+Tqs/4zSHdTx6M/+8Sy2e3zTj78+16xTFGlYU5/pSdwddZ1Xnjj5GRBlZHOzHGs5X2mKuCUuJ86iHtWrp/8OWz5eDwmZ6bvKR4L2r31tP6RM1gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gzJBlzNEV/yqoCzrv32maG35E4hu99vpWuR7s7TJBNM=;
 b=YhkgS6AFW2sqcA+obLEjGz88/HoOTwPmfsaUV+0IIINfOsPv9pukcalo4b1ZnzohkzhJqf73SU9HTz32gRz9tvOO1356S51PZlg/3o3eO4OWEnFVe6LcZIKp964eH5L2wwQJ9+l4fCO8e9xTeoynDLyzO3RfB7CmvlAjXkvZ4AA=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 30/37] xen/arm: introduce a helper to parse device tree
 memory node
Thread-Topic: [PATCH 30/37] xen/arm: introduce a helper to parse device tree
 memory node
Thread-Index: AQHXsHMzd7zwv2PLeUu2UelHuD4H8quygRqAgABQWwA=
Date: Fri, 24 Sep 2021 07:54:16 +0000
Message-ID:
 <DB9PR08MB6857CC06CB5190A770EC60079EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-31-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231945190.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231945190.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 03FD5B1AFF8BA44A82D1F6F29BE196B6.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ef23b600-2409-4453-b229-08d97f308a5f
x-ms-traffictypediagnostic: DB8PR08MB5466:|HE1PR0802MB2219:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB2219E67EE272839E7879A6F09EA49@HE1PR0802MB2219.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 x7u89GBbELKlLdn2K0DudQGdUd4c1HqDkIanbvM/XzWxxKIy0URzlwJC3pc7Zi+RKGMnrsylWYUb19sexi9eLbhxtIX/8bZy8bFBEQpLqBqCPZmCmlqmWwLsKfQ1VIuXrbqssnm8/qy3AFA3s3AnHppIVUGmDQGVn8PhJ9Kzc7/PBgE33im41bKGfTLOTiWA1gzRlpqsvbl3u8XTqj9YfnAmPz2SF55I0NODSUFecW5oMo/lMgMxZbqzWmOvrhWlkMF5CyX0gF6Pi/ssgYG1Rfg/yci8/q+7V3cT2lylh7D5BV8C0oyEgJctXppJ6AhVwVpnfafixGpcLcxW+1gj4CQHnZDr5u2DGn2FOQ0FzGPcwoBIqp2tsqPbPuP8BXerYmU9BR6zgYkfhl9SXlo6qkdly3lvSNlDSBGfZCrzzSwb5V4XS53NI53ceHDnikKyC/HuermHVIDmlZ4PeVLO1IFFNNuu3GNuuU1XeoYjc1qJGCcKT3zfhQZAAwMD1OadICzytrCMgtSoEO2NBt3NPT3buVlXUGCKcXBAV6JafTaTt3ladbXKty5V/nG57yFiA/tZRSisSA3IHskPlZ/wdSLifQ5zlFTNSRe1oNw2ZiqC2/lB3clSQ8MDLqlihXi3/VvdzfIMBr3dR29RNRzVKI1NrPUgZx2NUEuxncSJsgPSsH9LPzChjxtEGSBFQxZp/fQ+tXos0i9wQCNHEeTjnw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6506007)(7696005)(4326008)(5660300002)(186003)(53546011)(122000001)(71200400001)(83380400001)(26005)(55016002)(54906003)(8936002)(9686003)(64756008)(38100700002)(33656002)(316002)(66476007)(86362001)(66446008)(66556008)(6916009)(2906002)(8676002)(52536014)(76116006)(66946007)(508600001)(38070700005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5466
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e83bdb3a-7da7-4610-2869-08d97f30821f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Y0FOVLps+Fps4cnyAOXq5fgncookznOW/BiYHjAU2t5rDoJmBcnKLg1uPteb7vHkD3BnuTFsnbOhkIylS3klD4wS+etTVVX54Y//3PMKvX7sHzIYwUcl8rv+KgOkiMs2Q9EaFi3jsKxVKjQQiBibqcAhyDXsBASeYVeKOs6Sx/EDwNh6RRtZ/gHcjuIGKPZM8AFGct31diaerxoc50CtYYI8n9TIbOLWBoiHXtZG3U4VUURrlLOiy2mZos6Xzhf9XxvUPvEPr1AcUCFmXL5RFd9E/6Qv3eLDK0euoOk8XTHzYBVVlJd9WH0+LHZqNxu5y3ShhdTC940UWR6Vt1Y1m/76ToyaUln7NcmmRKjWJwYYDXgGHirnrIUUj2vC77Y/fhNM9vnXD17Wfe19AKbjvs/7vDLYKtf4Uf6BKfmFRJ66ZRJurZCJqk8QuGFPemCBgdU6dbeQ3wCEuqgoNzBasv//ZIFdSTE7an9PrAA5xHWS8qo9jypj6DJLZpL51cxMrrCgFuEfIbRxlFOd5eYXY+RoLxPpEneI+uwE00qR0kUkRaDXOxWtqLqSOEw2PvOdLkwwjD68QiYVRSF6gxhqmeK98czufCSTcNBoXLJHHzmTtu6bwUn3CMHyaT3MlbhlKwgQ/vKdj5/tzlSCWvhR7T/Mtz1DNqQMrxgeLK1vQfs/66hgtBK1C5rrmSRVj64sAlDIbvKJmWB0nGZJNJfzKQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(83380400001)(7696005)(26005)(2906002)(5660300002)(52536014)(8936002)(53546011)(36860700001)(6506007)(82310400003)(336012)(8676002)(186003)(54906003)(33656002)(356005)(47076005)(55016002)(81166007)(4326008)(70586007)(70206006)(508600001)(316002)(6862004)(9686003)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:54:30.5501
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ef23b600-2409-4453-b229-08d97f308a5f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2219

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTE6MDUNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
Pg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIDMwLzM3XSB4ZW4vYXJtOiBpbnRyb2R1Y2UgYSBoZWxw
ZXIgdG8gcGFyc2UgZGV2aWNlDQo+IHRyZWUgbWVtb3J5IG5vZGUNCj4gDQo+IE9uIFRodSwgMjMg
U2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IE1lbW9yeSBibG9ja3MnIE5VTUEgSUQgaW5m
b3JtYXRpb24gaXMgc3RvcmVkIGluIGRldmljZSB0cmVlJ3MNCj4gPiBtZW1vcnkgbm9kZXMgYXMg
Im51bWEtbm9kZS1pZCIuIFdlIG5lZWQgYSBuZXcgaGVscGVyIHRvIHBhcnNlDQo+ID4gYW5kIHZl
cmlmeSB0aGlzIElEIGZyb20gbWVtb3J5IG5vZGVzLg0KPiA+DQo+ID4gU2lnbmVkLW9mZi1ieTog
V2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+IA0KPiBUaGVyZSBhcmUgdGFicyBmb3IgaW5k
ZW50YXRpb24gaW4gdGhpcyBwYXRjaCwgd2UgdXNlIHNwYWNlcy4NCj4gDQoNCk9LDQoNCj4gDQo+
ID4gLS0tDQo+ID4gIHhlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMgfCA4MCArKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysNCj4gPiAgMSBmaWxlIGNoYW5nZWQsIDgwIGluc2Vy
dGlvbnMoKykNCj4gPg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2Vf
dHJlZS5jDQo+IGIveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+IGluZGV4IDI0
MjhmYmFlMGIuLjc5MThhMzk3ZmEgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gvYXJtL251bWFf
ZGV2aWNlX3RyZWUuYw0KPiA+ICsrKyBiL3hlbi9hcmNoL2FybS9udW1hX2RldmljZV90cmVlLmMN
Cj4gPiBAQCAtNDIsNiArNDIsMzUgQEAgc3RhdGljIGludCBfX2luaXQNCj4gZmR0X251bWFfcHJv
Y2Vzc29yX2FmZmluaXR5X2luaXQobm9kZWlkX3Qgbm9kZSkNCj4gPiAgICAgIHJldHVybiAwOw0K
PiA+ICB9DQo+ID4NCj4gPiArLyogQ2FsbGJhY2sgZm9yIHBhcnNpbmcgb2YgdGhlIG1lbW9yeSBy
ZWdpb25zIGFmZmluaXR5ICovDQo+ID4gK3N0YXRpYyBpbnQgX19pbml0IGZkdF9udW1hX21lbW9y
eV9hZmZpbml0eV9pbml0KG5vZGVpZF90IG5vZGUsDQo+ID4gKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgcGFkZHJfdCBzdGFydCwgcGFkZHJfdCBzaXplKQ0KPiANCj4gUGxlYXNlIGFs
aWduIHRoZSBwYXJhbWV0ZXJzDQo+IA0KDQpPSw0KDQo+IA0KPiA+ICt7DQo+ID4gKyAgICBpbnQg
cmV0Ow0KPiA+ICsNCj4gPiArICAgIGlmICggc3JhdF9kaXNhYmxlZCgpICkNCj4gPiArICAgIHsN
Cj4gPiArICAgICAgICByZXR1cm4gLUVJTlZBTDsNCj4gPiArICAgIH0NCj4gPiArDQo+ID4gKwlp
ZiAoICFudW1hX21lbWJsa3NfYXZhaWxhYmxlKCkgKQ0KPiA+ICsJew0KPiA+ICsJCWRwcmludGso
WEVOTE9HX1dBUk5JTkcsDQo+ID4gKyAgICAgICAgICAgICAgICAiVG9vIG1hbnkgbnVtYSBlbnRy
eSwgdHJ5IGJpZ2dlciBOUl9OT0RFX01FTUJMS1MgXG4iKTsNCj4gPiArCQliYWRfc3JhdCgpOw0K
PiA+ICsJCXJldHVybiAtRUlOVkFMOw0KPiA+ICsJfQ0KPiA+ICsNCj4gPiArCXJldCA9IG51bWFf
dXBkYXRlX25vZGVfbWVtYmxrcyhub2RlLCBzdGFydCwgc2l6ZSwgZmFsc2UpOw0KPiA+ICsJaWYg
KCByZXQgIT0gMCApDQo+ID4gKwl7DQo+ID4gKwkJYmFkX3NyYXQoKTsNCj4gPiArCSAgICByZXR1
cm4gLUVJTlZBTDsNCj4gPiArCX0NCj4gPiArDQo+ID4gKyAgICByZXR1cm4gMDsNCj4gPiArfQ0K
PiANCj4gQXNpZGUgZnJvbSBzcGFjZXMvdGFicywgdGhpcyBpcyBhIGxvdCBiZXR0ZXIhDQo+IA0K
DQpvaw0KDQo+IA0KPiA+ICAvKiBQYXJzZSBDUFUgTlVNQSBub2RlIGluZm8gKi8NCj4gPiAgc3Rh
dGljIGludCBfX2luaXQgZmR0X3BhcnNlX251bWFfY3B1X25vZGUoY29uc3Qgdm9pZCAqZmR0LCBp
bnQgbm9kZSkNCj4gPiAgew0KPiA+IEBAIC01NiwzICs4NSw1NCBAQCBzdGF0aWMgaW50IF9faW5p
dCBmZHRfcGFyc2VfbnVtYV9jcHVfbm9kZShjb25zdCB2b2lkDQo+ICpmZHQsIGludCBub2RlKQ0K
PiA+DQo+ID4gICAgICByZXR1cm4gZmR0X251bWFfcHJvY2Vzc29yX2FmZmluaXR5X2luaXQobmlk
KTsNCj4gPiAgfQ0KPiA+ICsNCj4gPiArLyogUGFyc2UgbWVtb3J5IG5vZGUgTlVNQSBpbmZvICov
DQo+ID4gK3N0YXRpYyBpbnQgX19pbml0IGZkdF9wYXJzZV9udW1hX21lbW9yeV9ub2RlKGNvbnN0
IHZvaWQgKmZkdCwgaW50IG5vZGUsDQo+ID4gKyAgICBjb25zdCBjaGFyICpuYW1lLCB1aW50MzJf
dCBhZGRyX2NlbGxzLCB1aW50MzJfdCBzaXplX2NlbGxzKQ0KPiANCj4gUGxlYXNlIGFsaWduIHRo
ZSBwYXJhbWV0ZXJzDQo+IA0KDQpvaw0KDQo+IA0KPiA+ICt7DQo+ID4gKyAgICB1aW50MzJfdCBu
aWQ7DQo+ID4gKyAgICBpbnQgcmV0ID0gMCwgbGVuOw0KPiA+ICsgICAgcGFkZHJfdCBhZGRyLCBz
aXplOw0KPiA+ICsgICAgY29uc3Qgc3RydWN0IGZkdF9wcm9wZXJ0eSAqcHJvcDsNCj4gPiArICAg
IHVpbnQzMl90IGlkeCwgcmFuZ2VzOw0KPiA+ICsgICAgY29uc3QgX19iZTMyICphZGRyZXNzZXM7
DQo+ID4gKw0KPiA+ICsgICAgbmlkID0gZGV2aWNlX3RyZWVfZ2V0X3UzMihmZHQsIG5vZGUsICJu
dW1hLW5vZGUtaWQiLCBNQVhfTlVNTk9ERVMpOw0KPiA+ICsgICAgaWYgKCBuaWQgPj0gTUFYX05V
TU5PREVTICkNCj4gPiArICAgIHsNCj4gPiArICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcg
Ik5vZGUgaWQgJXUgZXhjZWVkcyBtYXhpbXVtIHZhbHVlXG4iLA0KPiBuaWQpOw0KPiA+ICsgICAg
ICAgIHJldHVybiAtRUlOVkFMOw0KPiA+ICsgICAgfQ0KPiA+ICsNCj4gPiArICAgIHByb3AgPSBm
ZHRfZ2V0X3Byb3BlcnR5KGZkdCwgbm9kZSwgInJlZyIsICZsZW4pOw0KPiA+ICsgICAgaWYgKCAh
cHJvcCApDQo+ID4gKyAgICB7DQo+ID4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19XQVJOSU5HDQo+
ID4gKyAgICAgICAgICAgICAgICJmZHQ6IG5vZGUgYCVzJzogbWlzc2luZyBgcmVnJyBwcm9wZXJ0
eVxuIiwgbmFtZSk7DQo+ID4gKyAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gKyAgICB9DQo+
ID4gKw0KPiA+ICsgICAgYWRkcmVzc2VzID0gKGNvbnN0IF9fYmUzMiAqKXByb3AtPmRhdGE7DQo+
ID4gKyAgICByYW5nZXMgPSBsZW4gLyAoc2l6ZW9mKF9fYmUzMikqIChhZGRyX2NlbGxzICsgc2l6
ZV9jZWxscykpOw0KPiA+ICsgICAgZm9yICggaWR4ID0gMDsgaWR4IDwgcmFuZ2VzOyBpZHgrKyAp
DQo+ID4gKyAgICB7DQo+ID4gKyAgICAgICAgZGV2aWNlX3RyZWVfZ2V0X3JlZygmYWRkcmVzc2Vz
LCBhZGRyX2NlbGxzLCBzaXplX2NlbGxzLCAmYWRkciwNCj4gJnNpemUpOw0KPiA+ICsgICAgICAg
IC8qIFNraXAgemVybyBzaXplIHJhbmdlcyAqLw0KPiA+ICsgICAgICAgIGlmICggIXNpemUgKQ0K
PiA+ICsgICAgICAgICAgICBjb250aW51ZTsNCj4gPiArDQo+ID4gKyAgICAgICAgcmV0ID0gZmR0
X251bWFfbWVtb3J5X2FmZmluaXR5X2luaXQobmlkLCBhZGRyLCBzaXplKTsNCj4gPiArICAgICAg
ICBpZiAoIHJldCApIHsNCj4gPiArICAgICAgICAgICAgcmV0dXJuIC1FSU5WQUw7DQo+ID4gKyAg
ICAgICAgfQ0KPiA+ICsgICAgfQ0KPiANCj4gSSB0YWtlIGl0IHdvdWxkIGJlIGRpZmZpY3VsdCB0
byBwYXJzZSBudW1hLW5vZGUtaWQgYW5kIGNhbGwNCj4gZmR0X251bWFfbWVtb3J5X2FmZmluaXR5
X2luaXQgZnJvbQ0KPiB4ZW4vYXJjaC9hcm0vYm9vdGZkdC5jOmRldmljZV90cmVlX2dldF9tZW1p
bmZvLiBJcyBpdCBiZWNhdXNlDQo+IGRldmljZV90cmVlX2dldF9tZW1pbmZvIGlzIGNhbGxlZCB0
b28gZWFybHk/DQo+IA0KDQpXaGVuIEkgd2FzIGNvbXBvc2luZyB0aGlzIHBhdGNoLCBwZW5ueSdz
IHBhdGNoIGhhZG4ndCBiZWVuIG1lcmdlZC4NCkkgd2lsbCBsb29rIGludG8gaXQuDQoNCj4gDQo+
ID4gKyAgICBpZiAoIGlkeCA9PSAwICkNCj4gPiArICAgIHsNCj4gPiArICAgICAgICBwcmludGso
WEVOTE9HX0VSUg0KPiA+ICsgICAgICAgICAgICAgICAiYmFkIHByb3BlcnR5IGluIG1lbW9yeSBu
b2RlLCBpZHg9JWQgcmV0PSVkXG4iLCBpZHgsDQo+IHJldCk7DQo+ID4gKyAgICAgICAgcmV0dXJu
IC1FSU5WQUw7DQo+ID4gKyAgICB9DQo+ID4gKw0KPiA+ICsgICAgcmV0dXJuIDA7DQo+ID4gK30N
Cj4gPiAtLQ0KPiA+IDIuMjUuMQ0KPiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:58:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:58:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194997.347444 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg6P-00011T-Ce; Fri, 24 Sep 2021 07:58:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194997.347444; Fri, 24 Sep 2021 07:58:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg6P-00011M-9R; Fri, 24 Sep 2021 07:58:37 +0000
Received: by outflank-mailman (input) for mailman id 194997;
 Fri, 24 Sep 2021 07:58:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTg6O-00011G-59
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:58:36 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.47]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 36dd8904-1d0d-11ec-baad-12813bfff9fa;
 Fri, 24 Sep 2021 07:58:34 +0000 (UTC)
Received: from DB6PR07CA0006.eurprd07.prod.outlook.com (2603:10a6:6:2d::16) by
 DB6PR0802MB2229.eurprd08.prod.outlook.com (2603:10a6:4:86::10) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Fri, 24 Sep 2021 07:58:31 +0000
Received: from DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2d:cafe::80) by DB6PR07CA0006.outlook.office365.com
 (2603:10a6:6:2d::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.9 via Frontend
 Transport; Fri, 24 Sep 2021 07:58:31 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT035.mail.protection.outlook.com (10.152.20.65) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 07:58:31 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Fri, 24 Sep 2021 07:58:30 +0000
Received: from 7a150549a02d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 57D04B7E-F877-42D8-A7A3-B0B4719A2269.1; 
 Fri, 24 Sep 2021 07:58:21 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7a150549a02d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 07:58:21 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB7PR08MB3404.eurprd08.prod.outlook.com (2603:10a6:10:4c::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 07:58:16 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:58:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36dd8904-1d0d-11ec-baad-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LPOKGE2c7b+0EaNxWhyU8qdtwe1wc73sG+KhY52GqZM=;
 b=Smb/S1q9RhToQ2kmCk9E/k/CNACHeHEx3j//PLZmzOwpUbV1QvMAcMQrNSgLc5FAubymptVNQADgnsekt2gzb8h6cjfuEqpSk37jijYujshMpqeyl4OcN5XCJOnHCU7J8KPiD0GNQMwXueYqvZ8ItIIil3ajcDZOne2kAk4rwSk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lUft3LcFVvbOtORL2WTEPsXlnRNWnPEEptHlIPGSJ645ReprVLcFNIYcUTvtCCpcrRDGA5oQbZKRK2THgSrmAowyY1tZHENSp2q15Qbnqjn/knL7FlcT/6bZG0ZI8pWyyrti6RjRvrpXkNZ6lZcUz6Nh/45AiRQuUVuDo+chTheLNuqKJWxeDas8pa7Q+88o13dl6fptGEBj85nAn7pi6czGUSRNntPVvXM4rYZwLC1wJO0X1M+/sOv8Hc3OUTJrwcRyKAUWr9i2Q7tFkNNI64AUiW8jVPGwd7CL7KIDD2KCXNT43TKvLYHC4VXnTWJ7jT10gMNpr6XHNW0WaL+Ozw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LPOKGE2c7b+0EaNxWhyU8qdtwe1wc73sG+KhY52GqZM=;
 b=DaYy+RMH08D6CDeg+3SynvK7dqaj7XZIevbydUe+SSqE5L+/5KV3lLD7NHWo9PyMIAJzY/stVgW1MdWPkOoHMA4UjcauPGS2S+dnx2nuy9lm8m0xKtj75KHYKL7QirGzurMj1toi8g4JXB5bQhfHXMKe+IrzZdSHHb0JzoAUNuM2mXAp6Q/65NwLqJHrMr2IRmgDQYo+EoVqaBZt8zLUA5QQ7HyIrfwNB0A5bH2DSapO3Js9Tjj394mvMh2LBtvr5RHLCW8PJWWJmKRgPpPB4IxjFRrwhOdvqJYRa1tLxgbWy+XjscxkYjT0srRPed5jQFSoBnw5mpepi4YPdKlB6g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LPOKGE2c7b+0EaNxWhyU8qdtwe1wc73sG+KhY52GqZM=;
 b=Smb/S1q9RhToQ2kmCk9E/k/CNACHeHEx3j//PLZmzOwpUbV1QvMAcMQrNSgLc5FAubymptVNQADgnsekt2gzb8h6cjfuEqpSk37jijYujshMpqeyl4OcN5XCJOnHCU7J8KPiD0GNQMwXueYqvZ8ItIIil3ajcDZOne2kAk4rwSk=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from
 device tree
Thread-Topic: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from
 device tree
Thread-Index: AQHXsHM2z1sQ26jXekKStLWrlrd1WKuyhFCAgABOShA=
Date: Fri, 24 Sep 2021 07:58:15 +0000
Message-ID:
 <DB9PR08MB6857E6B6BDE4D55F7D468DDF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-33-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232007160.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109232007160.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 0365E1471FCB5243A4E4F359521A6FD0.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 74837a7c-3057-47ae-d484-08d97f3119f0
x-ms-traffictypediagnostic: DB7PR08MB3404:|DB6PR0802MB2229:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB2229E1643C880E4EC92DD8B49EA49@DB6PR0802MB2229.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 RqLipg38+tX5rHVftjfPMcwn7C4LU3svZS4B1U1aOqAmtE/NovKX2j7SpnbKASlo0UFyYF8ubdp1d2e8poNjpk3sQyi5cm34T3vW7PtfbAz4F98hY/JRYNsiWxWvbGCM/XP0gyqS1yGsU3hmKc8gavQMf+yet2SEwF9yKSzIIOaxeOIgrbYRov70k+cAHpeiv8TPfwGq++cnwxAleSsQ6rON8ypp1CNwpfgIqrdgnFUadSiGYXVSRMHRJPtMCIRfdEIbsWLqMCJ6993XeUgO5MjOCo0kD9dCHgc1D9t+VkUPru2ReV3ReVcbm6F+K3HukxdXZZ5fhRTHTFitLc/QLGst6S2xibRGOQwtRf4OL8ytpa0OjxHIoDf0Z1K2dYPYIs7kZo4zBISFdJ7IOqBsHLMuoUwZUKovhNDXE5Oo4B+fPUCbeZ9cbZz+x3bite5g9Xmne/VLjZkZT0u0QQzNo+8RLYRbYMd6ZANZiQZX8v+cG1mvp2BIWUwQ5SLmK1MDYx7I8ri/lgCGJb6KSq/VU559rOFsVNWFgvta6wkJJrm35W7kAFMpJFjOhHV+UI3umJYOQuK79zKqHEzEXwxIu8Y9y22G3qOhj6O/XX+ODMw6acihRbCUJaGLY5qJ4BQz1YsR1B5+nos49UMpV5mftxvy7jwz4v78rtTF9r8fma4nnL4b55533i743i3YtOvH7jUMm6XHsIcV0Yi4zYRHfA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(86362001)(83380400001)(66476007)(64756008)(66556008)(508600001)(38100700002)(26005)(8676002)(66946007)(122000001)(71200400001)(38070700005)(6506007)(76116006)(33656002)(66446008)(5660300002)(55016002)(8936002)(54906003)(316002)(4326008)(186003)(2906002)(7696005)(52536014)(53546011)(9686003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3404
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	baa974ae-af3d-41fd-cd6e-08d97f3110bd
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wHVYtJx/U4yI91cjHt/3xvA51u63bEFB1F4apiGjeABvP3Vv2fLTFGK+utgIdRtyxqXGOLFMDfP/vir/ASINWfwUz+oFp2HmpP3bSRHTpbATHb590fQVa5Ga3XUKeP+f2GcwXuXut/EnBclQ7rOg26traVM/uXLUcKgKZQJ1LrRyDMnJBr3uNjpvR7E7dNhla6E2qLG5mq5Z4bf4Bz6bMCkjD2+RewC/meOXB4JOLcHiCZ/M4zRCpXnjwwGIcoA+njxtTfjdmk4XBGvNxsKszKvMzawF1tMRoYjjFR6iR+tZqz3JUY1/OfcHJx6m/ACqnFT/Wlfs7qHboQ0u/aZjuclUyJOifR/RxfRMkjzTTgwoPLaxBCIqyc9HZklGmbSo4NT2VmRorL3bOHRiL/Cr93pNEQyqamyp6MNkdVhtc11kpAzIHZF5ahl4ZwhuvOl0HG7ZCXXaizVIiiURg71wbjHeP2u+/uhHr9wMMxOtlV4Pm+UANIs8j0YxjgM7fVDufLEoq5lBOgCHGgH3HyUpxqVE90LpYwxlP7IwwJ8eVFi3JMMrq4BpctZ7B4lS2LQyqUsBYh8up9Ggczbf8LOYSApch0+ZrSBaBI8GLH1gGEhvwCxAFCJgHqfTY+Vr40wECR4XCTaFd7BU4ByULzloyKVBVoynktXK106//PJ88CWEiSXwXhE/aBy7g0HB1Ar3ry+tPlp2WdaasV0GBqQGmQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(52536014)(8676002)(8936002)(83380400001)(81166007)(36860700001)(54906003)(6862004)(9686003)(508600001)(47076005)(4326008)(55016002)(86362001)(33656002)(26005)(70586007)(70206006)(336012)(186003)(5660300002)(2906002)(6506007)(7696005)(316002)(53546011)(356005)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:58:31.4904
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 74837a7c-3057-47ae-d484-08d97f3119f0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT035.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2229

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTE6MTcNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
Pg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIDMyLzM3XSB4ZW4vYXJtOiB1bmlmaWVkIGVudHJ5IHRv
IHBhcnNlIGFsbCBOVU1BIGRhdGENCj4gZnJvbSBkZXZpY2UgdHJlZQ0KPiANCj4gT24gVGh1LCAy
MyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gSW4gdGhpcyBBUEksIHdlIHNjYW4gd2hv
bGUgZGV2aWNlIHRyZWUgdG8gcGFyc2UgQ1BVIG5vZGUgaWQsIG1lbW9yeQ0KPiAgICAgICAgICAg
XiBmdW5jdGlvbiAgIF4gdGhlIHdob2xlDQo+IA0KPiA+IG5vZGUgaWQgYW5kIGRpc3RhbmNlLW1h
cC4gVGhvdWdoIGVhcmx5X3NjYW5fbm9kZSB3aWxsIGludm9rZSBoYXMgYQ0KPiA+IGhhbmRsZXIg
dG8gcHJvY2VzcyBtZW1vcnkgbm9kZXMuIElmIHdlIHdhbnQgdG8gcGFyc2UgbWVtb3J5IG5vZGUg
aWQNCj4gPiBpbiB0aGlzIGhhbmRsZXIsIHdlIGhhdmUgdG8gZW1iZWRlZCBOVU1BIHBhcnNlIGNv
ZGUgaW4gdGhpcyBoYW5kbGVyLg0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeIGVt
YmVkDQo+IA0KPiA+IEJ1dCB3ZSBzdGlsbCBuZWVkIHRvIHNjYW4gd2hvbGUgZGV2aWNlIHRyZWUg
dG8gZmluZCBDUFUgTlVNQSBpZCBhbmQNCj4gPiBkaXN0YW5jZS1tYXAuIEluIHRoaXMgY2FzZSwg
d2UgaW5jbHVkZSBtZW1vcnkgTlVNQSBpZCBwYXJzZSBpbiB0aGlzDQo+ID4gQVBJIHRvby4gQW5v
dGhlciBiZW5lZml0IGlzIHRoYXQgd2UgaGF2ZSBhIHVuaXF1ZSBlbnRyeSBmb3IgZGV2aWNlDQo+
ICAgXiBmdW5jdGlvbg0KPiANCj4gPiB0cmVlIE5VTUEgZGF0YSBwYXJzZS4NCj4gDQo+IEFoLCB0
aGF0J3MgdGhlIGV4cGxhbmF0aW9uIEkgd2FzIGFza2luZyBmb3IgZWFybGllciENCj4gDQoNClRo
ZSBxdWVzdGlvbiBhYm91dCBkZXZpY2VfdHJlZV9nZXRfbWVtaW5mbz8NCg0KPiANCj4gPiBTaWdu
ZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAtLS0NCj4gPiAgeGVu
L2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYyB8IDMwICsrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKw0KPiA+ICB4ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaCAgICAgIHwgIDEgKw0KPiA+
ICAyIGZpbGVzIGNoYW5nZWQsIDMxIGluc2VydGlvbnMoKykNCj4gPg0KPiA+IGRpZmYgLS1naXQg
YS94ZW4vYXJjaC9hcm0vbnVtYV9kZXZpY2VfdHJlZS5jDQo+IGIveGVuL2FyY2gvYXJtL251bWFf
ZGV2aWNlX3RyZWUuYw0KPiA+IGluZGV4IGU3ZmE4NGRmNGMuLjZhM2ZlZDAwMDIgMTAwNjQ0DQo+
ID4gLS0tIGEveGVuL2FyY2gvYXJtL251bWFfZGV2aWNlX3RyZWUuYw0KPiA+ICsrKyBiL3hlbi9h
cmNoL2FybS9udW1hX2RldmljZV90cmVlLmMNCj4gPiBAQCAtMjQyLDMgKzI0MiwzMyBAQCBzdGF0
aWMgaW50IF9faW5pdA0KPiBmZHRfcGFyc2VfbnVtYV9kaXN0YW5jZV9tYXBfdjEoY29uc3Qgdm9p
ZCAqZmR0LCBpbnQgbm9kZSkNCj4gPg0KPiA+ICAgICAgcmV0dXJuIDA7DQo+ID4gIH0NCj4gPiAr
DQo+ID4gK3N0YXRpYyBpbnQgX19pbml0IGZkdF9zY2FuX251bWFfbm9kZXMoY29uc3Qgdm9pZCAq
ZmR0LA0KPiA+ICsgICAgICAgICAgICAgICAgaW50IG5vZGUsIGNvbnN0IGNoYXIgKnVuYW1lLCBp
bnQgZGVwdGgsDQo+ID4gKyAgICAgICAgICAgICAgICB1MzIgYWRkcmVzc19jZWxscywgdTMyIHNp
emVfY2VsbHMsIHZvaWQgKmRhdGEpDQo+IA0KPiBQbGVhc2UgYWxpZ24gcGFyYW1ldGVycw0KPiAN
Cg0KT0sNCg0KPiANCj4gPiArew0KPiA+ICsgICAgaW50IGxlbiwgcmV0ID0gMDsNCj4gPiArICAg
IGNvbnN0IHZvaWQgKnByb3A7DQo+ID4gKw0KPiA+ICsgICAgcHJvcCA9IGZkdF9nZXRwcm9wKGZk
dCwgbm9kZSwgImRldmljZV90eXBlIiwgJmxlbik7DQo+ID4gKyAgICBpZiAocHJvcCkNCj4gDQo+
IGNvZGUgc3R5bGUNCj4gDQoNCk9LDQoNCj4gDQo+ID4gKyAgICB7DQo+ID4gKyAgICAgICAgbGVu
ICs9IDE7DQo+ID4gKyAgICAgICAgaWYgKCBtZW1jbXAocHJvcCwgImNwdSIsIGxlbikgPT0gMCAp
DQo+ID4gKyAgICAgICAgICAgIHJldCA9IGZkdF9wYXJzZV9udW1hX2NwdV9ub2RlKGZkdCwgbm9k
ZSk7DQo+ID4gKyAgICAgICAgZWxzZSBpZiAoIG1lbWNtcChwcm9wLCAibWVtb3J5IiwgbGVuKSA9
PSAwICkNCj4gPiArICAgICAgICAgICAgcmV0ID0gZmR0X3BhcnNlX251bWFfbWVtb3J5X25vZGUo
ZmR0LCBub2RlLCB1bmFtZSwNCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBh
ZGRyZXNzX2NlbGxzLCBzaXplX2NlbGxzKTsNCj4gDQo+IEkgcmVhbGl6ZSB0aGF0IHdpdGggdGhl
IGluY2x1c2lvbiBvZiAnXDAnIGluIHRoZSBjaGVjaywgdGhlIHVzYWdlIG9mDQo+IG1lbWNtcCBz
aG91bGQgYmUgc2FmZSwgYnV0IEkgd291bGQgcHJlZmVyIGlmIHdlIHVzZWQgc3RybmNtcCBpbnN0
ZWFkLg0KPiANCg0KT2ssIEkgd2lsbCB1c2Ugc3RybmNtcCBpbiBuZXh0IHZlcnNpb24uDQoNCj4g
DQo+ID4gKyAgICB9DQo+ID4gKyAgICBlbHNlIGlmICggZmR0X25vZGVfY2hlY2tfY29tcGF0aWJs
ZShmZHQsIG5vZGUsDQo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIm51bWEt
ZGlzdGFuY2UtbWFwLXYxIikgPT0gMCApDQo+ID4gKyAgICAgICAgcmV0ID0gZmR0X3BhcnNlX251
bWFfZGlzdGFuY2VfbWFwX3YxKGZkdCwgbm9kZSk7DQo+ID4gKw0KPiA+ICsgICAgcmV0dXJuIHJl
dDsNCj4gPiArfQ0KPiA+ICsNCj4gPiArLyogSW5pdGlhbGl6ZSBOVU1BIGZyb20gZGV2aWNlIHRy
ZWUgKi8NCj4gPiAraW50IF9faW5pdCBudW1hX2RldmljZV90cmVlX2luaXQoY29uc3Qgdm9pZCAq
ZmR0KQ0KPiA+ICt7DQo+ID4gKyAgICByZXR1cm4gZGV2aWNlX3RyZWVfZm9yX2VhY2hfbm9kZShm
ZHQsIDAsIGZkdF9zY2FuX251bWFfbm9kZXMsIE5VTEwpOw0KPiA+ICt9DQo+ID4gZGlmZiAtLWdp
dCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9udW1h
LmgNCj4gPiBpbmRleCA3Njc1MDEyY2I3Li5mNDZlOGUyOTM1IDEwMDY0NA0KPiA+IC0tLSBhL3hl
bi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9u
dW1hLmgNCj4gPiBAQCAtMjMsNiArMjMsNyBAQCB0eXBlZGVmIHU4IG5vZGVpZF90Ow0KPiA+ICAj
ZGVmaW5lIE5SX05PREVfTUVNQkxLUyBOUl9NRU1fQkFOS1MNCj4gPg0KPiA+ICBleHRlcm4gdm9p
ZCBudW1hX3NldF9kaXN0YW5jZShub2RlaWRfdCBmcm9tLCBub2RlaWRfdCB0bywgdWludDMyX3QN
Cj4gZGlzdGFuY2UpOw0KPiA+ICtleHRlcm4gaW50IG51bWFfZGV2aWNlX3RyZWVfaW5pdChjb25z
dCB2b2lkICpmZHQpOw0KPiA+DQo+ID4gICNlbHNlDQo+ID4NCj4gPiAtLQ0KPiA+IDIuMjUuMQ0K
PiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 07:58:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 07:58:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.194998.347455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg6S-0001J5-NJ; Fri, 24 Sep 2021 07:58:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 194998.347455; Fri, 24 Sep 2021 07:58:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTg6S-0001Iy-Jv; Fri, 24 Sep 2021 07:58:40 +0000
Received: by outflank-mailman (input) for mailman id 194998;
 Fri, 24 Sep 2021 07:58:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTg6R-0001IB-0M
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 07:58:39 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 103d2369-facb-4a15-a104-53ce3810adc6;
 Fri, 24 Sep 2021 07:58:38 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2056.outbound.protection.outlook.com [104.47.1.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-6-6pRkRmivNsSowsppTC6T3Q-1;
 Fri, 24 Sep 2021 09:58:36 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3295.eurprd04.prod.outlook.com (2603:10a6:802:f::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 07:58:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 07:58:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0024.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Fri, 24 Sep 2021 07:58:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 103d2369-facb-4a15-a104-53ce3810adc6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632470317;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pK+Y+/VwjWDQFpf/t3v0s+e1dmBUQYS/g1XInHQtIWI=;
	b=KOoZY2z0EKgYzDEnjrOZW22dr6DbR8EbvmoVklXUMqpJv+E6Ur+7h1JmZjtX9w5ozfmEQQ
	UQedrVbWCqjWFSfh5iJglMg5GeJhyqBYfNzZE1xctQ4msMGSexVR70rIwEt6Xz+hcVEd3r
	l50EjRMy4jriEH1u0QMPe70mYizFjbo=
X-MC-Unique: 6pRkRmivNsSowsppTC6T3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZtKmmCUOaFx60qajl4bcBdj9HK0zWATu/OGsTmF0kt0RVZxKCThCF8uSEwliTgyiTP+dbwxZIXdlDV8Y2za9oGZlLEpLPvciV2ajw9ACWkLoZkecaVrKGbQJ7PnoFoBWfDMQtnGypYi2kiJLNK2mR8PTsgLHTMcdUynPmuWhbTKx3ZjqEVTohcC4znVBeTb/Ns0vs+cwn6jX3kni8OD+7NE/FxNEH8sr0J1tQI54/Ue43uTc1ipj2u0vDXEyCYURd/fsFUroI5YWQcVHyGH934v1osLnjAJ8gtFoFik75VyCI1A4YcPKC3oA2jiuooqWI6dg3dyOSEJnq+8SoheZxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ndh2/WkbZIVBkTWBYx/Nsdae6UJSf66kgIISO2ix5KQ=;
 b=H8Ez8OyoDl3NAleGMBeux8Nd8Yuc008erM5Yqu8ST9mru/6SbAGmPGjlFqI6S0kp4ZMPdUT3qaf2/KGcUYQxSYr/qFVedzsPv63kr8yEuY3grQTE45TcM054B9005RZ46IixfyO0Uloid7MdxQ/2bo37HPjbe3f9WLIobREGea4NYCbxVEVsUhYJkZIBEm7PfRrlJEO+s2M2kSD15wIRIctUzq5CEcvbnpNbXopoZQaUFtqYabgP1NgU1F6y4DLrTVlUfXTIE2tzcCC1doY8eZEc37MljoIUDdKzwTAp/EsjJPfH73A7m75cWVXxa2FtGkHKHIS1uq4Ef57qODSIuw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
To: Wei Chen <Wei.Chen@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "julien@xen.org" <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
Date: Fri, 24 Sep 2021 09:58:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=gbk
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P192CA0024.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 511ea431-5174-4c62-1a5d-08d97f311aeb
X-MS-TrafficTypeDiagnostic: VI1PR04MB3295:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB32958D42449EF77F63FB25E7B3A49@VI1PR04MB3295.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dFefmsN6ZyBRFFDXMjHjQ9ALnwmLLV0iOJZxqLUaQAAj+lBQaKFuPiXxMP+JN4GsDeVhn+eP4eJXiKge9Kty9aZN9JhlBZfjpkFv6jzQyjf9nnJPTYWXOrlWMNjkKI99Uypu1nUr2SDrm6mOGExmgCF0P653HqpHCxwOqtA+eEsOR6OiRRwtoxwbLHaGW/7Jq/4rtuJNO3KhGDn8jFeEleih3OVzdD07+thf575Oc2MDBZCjORAM9itZZyRsAHAOQ0joffpyp6867ZwIjPKGw5kA93sxSer2lUdRrrpoCu0ZD/k9bShV9Ao+QWoCYezGpeapcYEac96/wJRoMG6zAAaINrlyZYeecnibf1MbqElEvqUj80XHCehIY3h6qU4WpGamu88iyJkF+klXggT4XWUApLHi77DnvV3Vs2Q750yXJLYt4i0Xk7Qjppn7GzsN5HrOriZ4y0Q6uhyPKQTYh6Uu3oruYriVVgjKCDqqu3ywxlcTWnz13SbwbV2Kw2qiopsZRKr5rXKPa/yDbJ8fh5sHiVpSawN6bpHjanYU7WMjinIMSAmX3oO5c0AErGgn8XRFDsR5u4aJKFCUlfr5vukRSJ5BrVC0JfVnaNVNvUXhkqwh0Yom/7dNFhbdNc1K6Kc3xDHOLWqhQGiC3d09z48qRZbIEtruWyxxODJ/0QUsJT8kfVpjgyUAuaD/QideFFvlBIffLYcvmHmqFziADyaiFNhACAzVIrcH62h7/5M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(16576012)(316002)(5660300002)(8936002)(66476007)(2906002)(83380400001)(4326008)(66946007)(86362001)(36756003)(6486002)(508600001)(31686004)(26005)(53546011)(956004)(2616005)(38100700002)(8676002)(31696002)(4744005)(54906003)(186003)(6916009)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?jMfrNL2jckKkxJ6gasHEpSNIaK6h1A/ptgCKEyCu0rdtRqDSmEwsn130OaTo?=
 =?us-ascii?Q?V0p5kViNiJN3GbQ4yfIuT70rCnbF+kf+X75a4bQkDGdJCvTk+RaPAFkyNHEF?=
 =?us-ascii?Q?q3Mogx0V5BmBNPGJiygTlQhl/l+JKLrQyQK8ja4VDe9X5aKyjLaaWiJ1PgmW?=
 =?us-ascii?Q?MhOg2KI/cRr2S1V0wUbBfi5MfO09YIImBfrgDSQy9bE7WiZhWoZ14JwSpV4V?=
 =?us-ascii?Q?qQ0DMpGfcxPYwfjtZG8BCKpbGnFQDyyk4MK1Gsx1A3P8VJl4wix++x2F/wW0?=
 =?us-ascii?Q?0f8cdf54AyHqqhlbSHbxlhiVlOGlOgkp0aKF4DvfiSZ1qkz9WFpQSJgdH298?=
 =?us-ascii?Q?IR0pi/TZiROkdcKXyAWxwsKn3UkE0FG36jYs1n5SEd1Hk/HLG0spoXNvfvm0?=
 =?us-ascii?Q?MWWWUgxXeDrc1463OY4jF1WWqD0gQqguRKUzIlRLz8Ce6BaVTw0AtRkyQ4wb?=
 =?us-ascii?Q?2aAEfqd/voAEziY+oyKvUZjJ0r9qx+jP38q2T7EcgJqxiKAa4OzBwkp64ore?=
 =?us-ascii?Q?2y31ZbBxXQlelGNuzhzGwP4cFWhs3agM9FfOYdYlaN2Y+zIpu/qzFvex4jty?=
 =?us-ascii?Q?cQVABG5aBNY/a7bdef9bA6cijbT2aj4fbS2vaG6FHPBLqM4gCD/Wf8c1SUr2?=
 =?us-ascii?Q?y+rfmJZyX0Fz3IxwxI51KvfH9IfblrnQEI+1GDaUxsIdhCDUbYlUUKAeaqwt?=
 =?us-ascii?Q?pGF97DPdHELhwIiMhHwO7tc6rMCyOSXiYKlPlKBMQg+4zxrXtMOjtl/nL6wg?=
 =?us-ascii?Q?GGtKRxsOinTwHtlkiTEzMraFp2AnX7b7dj0SuGEaFczj6VATuCWM3rrLvn9Z?=
 =?us-ascii?Q?TljLylnsl2nUuYDJQsR3MPLXiMPwMALgIerlohzGGkIHd7d1ZuCBJANI9XTg?=
 =?us-ascii?Q?3hHFv4X9G2tbsDv03xcHoVUoq4loUBw9J3Zc2QBa0y16bM0nbxx/IfrPDyho?=
 =?us-ascii?Q?75vf2kXxPwQw0g3XN765SufTSPxnCrGxj5b6xrkFDSz/+1Utrz8EbC8p+D2c?=
 =?us-ascii?Q?PYZWfOBQH88ffNUIiItOxncg9gjNINhO3/QQdqyUyf8ig5v/nNRJ0PievGy1?=
 =?us-ascii?Q?VNarPuFPGnzwbZJphBlZU3sR4lsWkhW4xFvfTGX12i+ic5igYQg1JqV4LcyP?=
 =?us-ascii?Q?2gatgPPEpDEod80HdBNs1sMAHFJOJ5/nFH5As72HSJQAc3QxUnNUgeyb1Uxk?=
 =?us-ascii?Q?bQ0LTEBH1mPW/KhlBisQSLTtUObi6sIboA46wn6DpTbJO/aXjDGjprU+HxbY?=
 =?us-ascii?Q?kf450BypGOUS6j/NUUVKc3U/nRCRiZmMSoJZgrHUBkMVJmWTI7WDZ8PSjEzx?=
 =?us-ascii?Q?RR/Ijb1VV7qfGiz0Ct5hKJ+x?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 511ea431-5174-4c62-1a5d-08d97f311aeb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 07:58:33.4674
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Uzw6eMSV+IuxQX9TYEegC7fQoJM2QOL2Lf44cfAWA3mH9vW1rUkrzhHA3D+jfiiox2OJf+NBNI0Ycvg69P9pEA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3295

On 24.09.2021 06:34, Wei Chen wrote:
>> From: Stefano Stabellini <sstabellini@kernel.org>
>> Sent: 2021=C4=EA9=D4=C224=C8=D5 9:15
>>
>> On Thu, 23 Sep 2021, Wei Chen wrote:
>>> --- a/xen/common/Kconfig
>>> +++ b/xen/common/Kconfig
>>> @@ -11,6 +11,16 @@ config COMPAT
>>>  config CORE_PARKING
>>>  	bool
>>>
>>> +config EFI
>>> +	bool
>>
>> Without the title the option is not user-selectable (or de-selectable).
>> So the help message below can never be seen.
>>
>> Either add a title, e.g.:
>>
>> bool "EFI support"
>>
>> Or fully make the option a silent option by removing the help text.
>=20
> OK, in current Xen code, EFI is unconditionally compiled. Before
> we change related code, I prefer to remove the help text.

But that's not true: At least on x86 EFI gets compiled depending on
tool chain capabilities. Ultimately we may indeed want a user
selectable option here, but until then I'm afraid having this option
at all may be misleading on x86.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 08:08:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 08:08:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195013.347465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgFc-0003gE-SZ; Fri, 24 Sep 2021 08:08:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195013.347465; Fri, 24 Sep 2021 08:08:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgFc-0003g7-PS; Fri, 24 Sep 2021 08:08:08 +0000
Received: by outflank-mailman (input) for mailman id 195013;
 Fri, 24 Sep 2021 08:08:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTgFb-0003g0-7P
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 08:08:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bf3f3c68-6a39-4cb1-9f96-af490b964b6c;
 Fri, 24 Sep 2021 08:08:06 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-l1EHB0llMpyKhJGKMg4Bjg-1; Fri, 24 Sep 2021 10:08:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5168.eurprd04.prod.outlook.com (2603:10a6:803:56::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Fri, 24 Sep
 2021 08:08:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 08:08:01 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0122.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 08:08:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf3f3c68-6a39-4cb1-9f96-af490b964b6c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632470885;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=F6No7pRe5XjDcTkRgjdWd+8yzZrBfcMAd46s/hA9XNE=;
	b=NuZO9Y6hyMG7Ne++eiziby2y6TajlnpzdGJDjAdMOVfsRx7S/D6G2mizvn0/oocr3rpS7h
	zJpC7+RhYuJa6vDM+BSxyHSDGm5Gq/AQgCJx0eMFYeB6WeMbvCyA9/W0jOiASmGWKp+W0/
	+//g9kpM1g7d2GCAxNzA2lxyYUUadRQ=
X-MC-Unique: l1EHB0llMpyKhJGKMg4Bjg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PTprzOliNT8eaxHUPP9XYT/lOIws7ZSdDiDQJ/Tchqs6r37a0chTmmqo9JxkkqwYT0hhkktbxD3aBtZDUYJKRJvaO46G791i6Ag8+gn+e9CBE5iw1+yhqjpJ07XkdAS/ACY1krSqiWHXef0EqJKJGGbH87jtSnTr1Pm1dcqRuVSOAPxikVf0SGZm++NH43uTaT3jUjk0M5Fwy5uRrX0X1zkEkHoN1yTKcp8sIU19A2AJfMqWD2jd5LoCZHGT5aaz7rE0lzyllBmDiUl+6hUsN4w6+gsMHNPS9H3hg9OkEqd5mSqAPMBmtsr9G2F1FMMFTI7Ymmd3sKii/o0vm52j3g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=F6No7pRe5XjDcTkRgjdWd+8yzZrBfcMAd46s/hA9XNE=;
 b=bL1MBn+w5gQUv58bzgMEXlsWMloMEbFPxhisnDddHxBHzNli5DwnUBNWgykCY35t3Co29xVKE7MlyTUKMeKNyeBR6GgQOq5KIwu+ZiWprFNuenGLJvkpGg/aiJrJ/CY79Cp5ehJn3EZQLmW/F0cSYJjnEzegU83c0VIxOcDlDouEkYG4Glwl4sGiIEp13v1D/LF3Fc2nlwpR3aLosShsRFev8Y/e8ISN7aFqWdGdunCIwmOsjYwSJtXdQ9b5GFYfNoBwn1DNQQLANoXeJb8pT2KKrLbmnhBTvn0jTuNiBfyoniHmT1MDbgk2MnsdN1c/VYp1rGV+e9Cnjlxa4gAWLQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA
 initialization
To: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <wei.chen@arm.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, wl@xen.org
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-26-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <d6973324-52e2-1c78-4f57-929d3af07a80@suse.com>
Date: Fri, 24 Sep 2021 10:07:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0122.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9d6e22f1-d52a-42c8-c5a2-08d97f326d68
X-MS-TrafficTypeDiagnostic: VI1PR04MB5168:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5168439786B2D16EC4514FD4B3A49@VI1PR04MB5168.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IxTsph81lmTl/c3MBqxVQ8TpB5vHsM3Un36kTpAsx2ued5k+uQC7p7TW8imzfOBe8RRyOi8iU13kwVNvevjFa5YCT8iGok0HprJRW/ccDEprxAOpXtHa6qlsytJ6YKcOBeNekVdezxYwzObPkp/TBrqZL52iiAfE3rtsXjlylII2S3ysotS0rptpWlU64aXQ1Ra6aEKARZR8PGJPs4rMKj27fgY5iqHc7/OE26yw+5md+8SF9rMIvox9kEm39he+iubcUH0bcEDwOwcbmO2LZxiwLOcl0CmD5SF8No6cECke6T8C9qYWvHg2d+JMgR8sLKpy+7ibVzsM0u3tUlh7+RRr8YpBZFXaw83g0MMMfc4zmQcIKPoKqvQxeAlHGV3UzxAPkqrpMFtz/JzSX+Pgwd9I9C9fLE9qPsz9/yTOwOyJtJrN/kbX7whXhm/Ye8A2SKdpxxyj8sizrUrKRdLYiZoaIG5K5fWCDXEPXQZEmH150r0CzntybIO9fj/ZIP1fAlH0IvR50tck5MNyIjz0+uWQw9LGOtBDxj7mAq4W6oHBTObYX5Fu5bsivttemdYF1F0aPxl/QjF78UX1GOlijsdcM9LFH2S2nw2pOKAo9rUhCqnFYzBUpbiaa21aGI9L64p1DsTW/omS6XQ3H2UdUjhK9N3qP4eEmtavaDRRMhYr0nd88Jq7yytonWxtlDF1ZmPjEnyiKqSg1/oSQndK/zVMYH2sDB1/lmUceoKgd1w=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66476007)(2906002)(53546011)(8676002)(956004)(8936002)(66946007)(31686004)(110136005)(316002)(36756003)(4326008)(2616005)(5660300002)(16576012)(31696002)(38100700002)(6486002)(86362001)(26005)(186003)(83380400001)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?amVFNjFYTklSdTRtM3FUS2VPOFdETGZaY281UTk3Z05hdmowanBEbTdSNGVj?=
 =?utf-8?B?TEpJSFQ1dytiUHhyYmh1ekpOQkdBOTgwSlRielVSci9qdHloQXQxbXhiNmlo?=
 =?utf-8?B?OXdINGsra3B5a3QvTk0zUUc3dm1VRnEvaVk5V3RDSmRzWXRnOHlDTnIzLzVC?=
 =?utf-8?B?TnorODVOLzFEWWVubkNtQ1k3aGpMMnZEck5VS24zZ0VCbHdsRUhncnhpNUNU?=
 =?utf-8?B?Yk5CYlZ4d1F6TS8vdGFQYWlLaXkvd3dIQWpUaEVONWw5N01JYlhWTnFqQ05W?=
 =?utf-8?B?UkpCNTEzMFQ2Y1VXMS9pYU9xSEluZjhuMlN3Uk1kRnRmZTJWSitzNEhGOUJH?=
 =?utf-8?B?aGZOK3kvSmxHRG5UWmNOZW5UU2t6U2ZmM09EakpYVHJjc01UbWhoNjlmTm90?=
 =?utf-8?B?aUNVMk4wd3k4RWpuZjRmRmV3dlZYMnZnVzNVR2pldkZsdDNxa21wSkp3MWc2?=
 =?utf-8?B?UVhUMGUzZVgrcG93bVlvN1Q1UzUwc2syNE1RQ3k5Rm44SVdmN3d6dWVuMnFJ?=
 =?utf-8?B?ZFBkN1U2YUZoM09XM2hiTnRidG9HSnZEMEsvNFkrWTNOZ0oxWW1KaEpOR3dv?=
 =?utf-8?B?M1I1SHNmcFJKQzlyT2pEc08xRVV5WlJIVURSVCtzb3pEM0psWW1GYVNWRnZT?=
 =?utf-8?B?K1FOSWMybWRwNXVsM3hnekdiTUU4RG9Yc2VERkp1N3Q3MDNZOEkxVlozUkVk?=
 =?utf-8?B?QWh1YUlXalVQRFJ3NHViYng0ZC91U2p5MHlKbWEzMFluV3pjZXNoazgwUnRK?=
 =?utf-8?B?NmRvTFVvNE44N1l6aS9Hbi9LSVZ1dFpvYTg1bkxWckN5UWozOWJqdFBNcTRn?=
 =?utf-8?B?Mythek4vTnlIOGFkYkUxQWFRM3dzOUNEN3lhUlRlSDh2THpVVFNVNmg0ODQ2?=
 =?utf-8?B?TzErcGJwcDYvcjZvZ2UxaFNjcXE5ZG1MWXJSREo4YTR3TFZ4S29OTU5SNFFD?=
 =?utf-8?B?UnJsUUpFcDB4amVvelB1UEJ2dkF0NlE3SjdkdURWekJpb3hGM3FnbGh4QnhZ?=
 =?utf-8?B?L256VnRsUTY1c0ZOMEtmVEZyMFE1c2JFQzJKL0h3WXZmQ2VoL1p6WUZyMUQ0?=
 =?utf-8?B?Y1FKTGp3MDU4V0FSMnY5eExuam5xamMzc2xkVHpjRHF2Vk9VMFR1M2xMem1s?=
 =?utf-8?B?NkNJUFVRcVB1d3BjMWJVYlQyUmI0TUxGbmNnTDU0QlRqRjZ5N1lTUmVKTjA3?=
 =?utf-8?B?c1VBMGZVUUMxK3BOK3h2VnJMM1dWL2prczhqYlRMZkFydjcycVovMG9HZ3Fw?=
 =?utf-8?B?WFVoMEFNdzA2MWpsUHBQNmh6MkwzWEp3WDRFT1B2SXJudjVudWkzWGFOUUtX?=
 =?utf-8?B?VHVpY3ZNbkpLd3FmTE05WCt3TFBXdzQzblBNYVZDaFZLQ2RUUUtFRlZ3eWFp?=
 =?utf-8?B?Yi8wemVKVzZLcmlvRU9reGdMVGptYWU1SkFCdVhqaDRQZHVPU0lJTnplb3JC?=
 =?utf-8?B?MHd1ZXovVEc2YXlzaHo1c09iZmdWZ01SR2VuZWJUbW9zYWs2Q0JzbzdVQjZV?=
 =?utf-8?B?WDZKSlhlekQvK2kvbm9pSFBRckUycGZCVGxUL2JNOExvTGUzRjRHWTJIVGdH?=
 =?utf-8?B?VTBnSDR1cXUzWUpQS2lNRmR0MWpoc002SzBGeThzNU42M3EvZkJQd2paQVBs?=
 =?utf-8?B?cHBUSHY0Ymo1b2pzNDIvUVMzUHh4S1IxSjg0eTBKVzhqTENHMjdQNU1QV3Fv?=
 =?utf-8?B?Z1BwdDM5clNQZmhlVGdMdWxZck8rTkJWSi9EYlg5UmJ1Umg5b0JydHlxakIy?=
 =?utf-8?Q?RZVelcWzQZIBex/l8vINMOoFdtoU+MAR+BOPwhk?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9d6e22f1-d52a-42c8-c5a2-08d97f326d68
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 08:08:01.1600
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3xyIT/+ngRMeZxvPTZ3G7NmVEZJtwx/8x8vURDUnGYt2oKmREWe1tX0RzvymvE4qq08vrermweoiLVq2kp0l2g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5168

On 24.09.2021 04:09, Stefano Stabellini wrote:
> On Thu, 23 Sep 2021, Wei Chen wrote:
>> NUMA initialization will parse information from firmware provided
>> static resource affinity table (ACPI SRAT or DTB). bad_srat if a
>> function that will be used when initialization code encounters
>> some unexcepted errors.
>>
>> In this patch, we introduce Arm version bad_srat for NUMA common
>> initialization code to invoke it.
>>
>> Signed-off-by: Wei Chen <wei.chen@arm.com>
>> ---
>>  xen/arch/arm/numa.c | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
>> index 3755b01ef4..5209d3de4d 100644
>> --- a/xen/arch/arm/numa.c
>> +++ b/xen/arch/arm/numa.c
>> @@ -18,6 +18,7 @@
>>   *
>>   */
>>  #include <xen/init.h>
>> +#include <xen/nodemask.h>
>>  #include <xen/numa.h>
>>  
>>  static uint8_t __read_mostly
>> @@ -25,6 +26,12 @@ node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
>>      { 0 }
>>  };
>>  
>> +__init void bad_srat(void)
>> +{
>> +    printk(KERN_ERR "NUMA: Firmware SRAT table not used.\n");
>> +    fw_numa = -1;
>> +}
> 
> I realize that the series keeps the "srat" terminology everywhere on DT
> too. I wonder if it is worth replacing srat with something like
> "numa_distance" everywhere as appropriate. I am adding the x86
> maintainers for an opinion.
> 
> If you guys prefer to keep srat (if nothing else, it is concise), I am
> also OK with keeping srat although it is not technically accurate.

I think we want to tell apart both things: Where we truly talk about
the firmware's SRAT table, keeping that name is fine. But I suppose
there no "Firmware SRAT table" (as in the log message above) when
using DT? If so, at the very least in log messages SRAT shouldn't be
mentioned. Perhaps even functions serving both an ACPI and a DT
purpose would better not use "srat" in their names (but I'm not as
fussed about it there.)

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 08:42:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 08:42:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195024.347477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgmQ-0000bf-II; Fri, 24 Sep 2021 08:42:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195024.347477; Fri, 24 Sep 2021 08:42:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgmQ-0000bY-F1; Fri, 24 Sep 2021 08:42:02 +0000
Received: by outflank-mailman (input) for mailman id 195024;
 Fri, 24 Sep 2021 08:42:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTgmO-0000bS-OA
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 08:42:00 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bdc87bb7-617e-404a-8ae4-57beb9f8ba0c;
 Fri, 24 Sep 2021 08:41:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdc87bb7-617e-404a-8ae4-57beb9f8ba0c
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632472919;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=F/B0cVyUVT5RpzjBNiaxSAVTbNHy4nCj5kV36uuCphc=;
  b=Cp1FfYxjKNSogYzhRM1VgYxucjcjMrEAUiiEHk0/br87/6bZhJFwBvAq
   Yj4pfmL0zfXsE6F3vSUPgRsoCWC9ffzJeFOnXXgND+ZY6egYqvEJn4ug8
   g5yRoJvt5EUJMmyRP4nR5830mnkI/zAIETj8UqNnT3KeU5lA0fDOtOA0a
   E=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: frdSI581sgvoyUHsLsyqKL2zJ65Ajsw4Rt0Qp50jVdfsJhnjbUFZs/zDMkpjyO2j5wrDsEtNS8
 99vhdQIaY77ht5MjwT6wdPDRfjOuEj8CWkvbKntucthv+igIQJR2ZfuKG/gQzC4t/sPqanus8b
 kDdQ1+sr9C1K/Nrez15SIlzKtqAmBE57zZJElquX8LOmKWGmPKP7WfHj1PXmpH/qb/+bL9r3eJ
 9vy7A6uCFw0Uk0hgD7WSCof82+DXSsOwn8ZOWcc2leAOmTVnqpmlnTRNXYZP6J2hQodrXsbwlt
 ogwvTS+rlDbcD0U5VkJHBrY0
X-SBRS: 5.1
X-MesageID: 53516225
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Yks8ZKmskRMcRhrO57bcVv/o5gzjIERdPkR7XQ2eYbSJt1+Wr1Gzt
 xIeWWGAMv6KZzGmfIh1PN6x9RhUuJXWnNFiTgFprStkRiMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Q02YHR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 MRw5KCSbis1B7zBu+MkXQYCMC5RMbITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gpr25kQR6yDD
 yYfQRNgazv6aiZKBnoeBZt5tcjyrSWuUTIN/Tp5ooJoujOOnWSdyoPFMcXXUsyHQ91PmUSVr
 X6A+H72ajkkM9iYxSuA42ibrObFliPmW6ofDLS9sPVthTW72Wg7GBAQE1yhrpGRiESzRtZeI
 Ew84Tc1oO4580nDZtvgWxy1plaUsxhaXMBfe8Uq5QfIxqfK7gKxAmkfUiUHeNEgrNUxRzEhy
 hmOhdyBLTtrvb6OVWOG9p+bqDqzPW4eKmpqTTQfUQIP7t3noYcyphHCVNBuFOiylNKdMTb0z
 jORpS4ynYIPnNUL3KW2+1PAqz+0r52PRQkwjjg7RUr8sFk/PtT8IdX1tx6Ltp6sMbp1UHGQg
 Sk4ue+y8NkfALKKrSKSGsYCM722sqPt3CLnvXZjGJwo9jKI8nGlfJxN7DwWGHqFIvroaheyP
 xSO4105CIt7eSLwNP4qP9vZ59ECkPC4fekJQMw4eTanjnJZTwaB4CgmTkqZxWmFfKMEwPxnZ
 M/znSpBCx8n5UVbINieG7t1PVwDnHlWKYbvqXbTlU/PPV22PiL9dFv9GAHSBt3VFYvdyOkvz
 /5RNtGR1zJUW/Dkby/c/OY7dA5RdyRhX82p9pALLIZvxzaK/kl7UJc9Jpt7J+RYc1l9zL+Ur
 hlRpGcCoLYAuZE3AVrTMS0yAF8edZ1+sWg6LUQR0aWAgRAejXKUxP5HLfMfJOB/nMQ6lKIcZ
 6RVKq2oX6UUIhyaqmt1UHUIhNE7HPhdrVnVZHTNjflWV8MIejElDfe+JVawqHlSUHDn3Sb8y
 pX5vj7mrVM4b10KJO7dae61zkP3un4YmeloWFDPLMUVc0LpmLWG4QSo5hPuC81TexjF2BWA0
 AOaXUURqeXX+tdn+9jVn6GU6YyuFrImTEZdGmDa65ewNDXboTX/kdMRDr7QcGCPTn7w9YWje
 f5Rk6P2PsoYkQsYqIF7Cbtqk/4zvoO9u79Aww14N3zXdFD3WKh4K3yL0JAX5K1Az7NUoyWsX
 UeL9oUIMLmFIpq9QlUQOBAkfqKI0vRNwmve6vE8IUPb4i5r/eXYDRUObkfU0CEEdel7KoIox
 +slqfU60Q3nh0p4KMuCgwBV63+Ici4KXZI4u8xIG4TskAcqlA1POMSOFi/s7ZiTQNxQKU12c
 CSMja/PirkAlErPd30/SSrE0eZH3MlcvRlLyBkJJkiTm8qDjfgyhUUD/TMyRwVT7xNGz+MsZ
 TQ7axwrff2Dr2VymcxOf2GwAAUQVhSW93v4x0YNiGCEHVKjUXbALTFlNOuAlKzDH7mwotSPE
 Gmk9VvY
IronPort-HdrOrdr: A9a23:QIKye64AIJcneT2TowPXwUuBI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc0AxhIE3I6urwQ5VoIEmsv6KdhLN+AV7MZniBhILFFvAA0WKA+UysJ8SdzJ8l6U
 4IScEXY7eQbWSS5fyKpzVQeOxQpeVvhZrY4ts2uE0dKT2CBZsQjTtRO0K+KAlbVQNGDZ02GN
 63/cxcvQetfnwRc4CSGmQFd/KrnayGqLvWJTo9QzI34giHij2lrJTgFQKD4xsYWzRThZ8/7G
 n+lRDj7KnLiYD19vac7R6c031loqqg9jJxPr3OtiHTEESvtu+cXvUhZ1RFhkFxnAjg0idvrD
 CGmWZbAy060QKtQojym2qq5+Co6kdT11byjVCfmnftusr/WXYzDNdAn5tQdl/D51Mnp8wU6t
 M844u1jesiMfr7plWL2zEIbWAbqmOk5X451eIDhX1WVoUTLLdXsIwE5UtQVJMNBjjz5owrGP
 RnSJi03ocfTXqKK3TC+mV/yt2lWXo+Wh+AX0gZo8SQlzxbhmpwwUcUzNEW2n0A6JU+QZ9Z4P
 msCNUgqJheCssNKa5tDuYIRsW6TmTLXBLXKWqXZU/qEakWUki93qIfII9Flt1CXaZ4h6fatK
 6xLm+whFRCCH4GU/f+o6Gj2iq9MVmAYQ==
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="53516225"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OuEMvmzLM8km5KGc1RV2XxnqsSzkvc0HmVn8KIceW8Onpo7ocglDRtAG+v4tFOYy6L72/vfOu0/j5d/a7UPgZiUtM/d0q03myT9xapXM+CGUNGPKDzn4x2UCcrttfQmGBVkdl6O2dw4tFl2toXCeoxjs+fBfiD4qC9wafwn3BmA6G+4KoNAiqg85LBret2kTnS2Lc3gZWNCQoiUub/FDG3guUFMOX/V1DUfFlcliPP680Xd+Fwm9Bt7gX1CaLyF0rSfW1IjC78dyaqsw2cQsZWs1llEJgdZr0BqkTMDLUyNr4rYmtsv8EkqpQe8Pg4BUkuz6C1CGi5+7b8U8b6U43A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3QfVlt2un1DDm8HNlvp7qY/nnB/5eJ5J7RpyeAoGVS8=;
 b=DBOlumFH264fjITZG2BOPIA83CpCUaax4hcjNmQXVm29yrpQGxTbPUNOaFoQUjvaqQHGLCQCQh2GR6izzOUJ6tuw6GGmiUuN//JBRGZPg8xP7k687S2nzknD7hJV3F5DXqxn21ocdO4zH70/Kd1ya/WdjmvlaHD2IzSF2AMq0kSmKRgtsPO1LWyzSPZg1bBaSuGUy/phOCZxd0+seXiluy4IYx5DbfCBlJ2rg7rncqgrySCWoHqyVpBYsbgUf4QmlUrCdblEWlP7wZ2ZdZudBVUCmLfl9Ccjn5HD7slfMosyHAcZR1P/MRwCIHm2v9A5iLOYuXK992sH48nCI69oyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3QfVlt2un1DDm8HNlvp7qY/nnB/5eJ5J7RpyeAoGVS8=;
 b=b3BGXGM/fgRLx6Y2m5Netjvb1TjaL9mnioXjuDjbx63er395GBBEnOGkqmHbAJz33Nbf7IA1aOxaSbkCIHmK9SVZqM3nJXJ3vnFXQHDuSoCDK13Sm15UnpjwjZUIe9QteW5XC6IJFnw9l8eDr4yiUquTFh54bnU0cJEv83p0CpY=
Date: Fri, 24 Sep 2021 10:41:51 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Oleksandr Tyshchenko <olekstysh@gmail.com>,
	<xen-devel@lists.xenproject.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, George Dunlap <george.dunlap@citrix.com>, "Ian
 Jackson" <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in
 struct page_info on Arm
Message-ID: <YU2PT4rUts8KljKe@MacBook-Air-de-Roger.local>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
 <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
X-ClientProxiedBy: LO4P123CA0222.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a6::11) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 334a77eb-6421-4a13-c052-08d97f372a01
X-MS-TrafficTypeDiagnostic: DM4PR03MB6094:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM4PR03MB609442F79FCB9E546AD37E568FA49@DM4PR03MB6094.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: fdxJfjsLyfEktOrD8ZwrHcvtAfc1h6l/xoavR4HqIvUQaNFLc0iS5soaGyBMSew6VCIU5CSxgA5Asz+AG9ONf+DDjWhByG+AycNhH0UXMNL+3Isabv4aimo6Q2wozLDagN2LPEt4Rom0gIdFZl5bq92iqkIF/Z7X8dM5DHSK0TfNg47qDnYOQHmCVLE6woQKXNMw6icyrjEhQ84nJvs+prAWW/ykv5UcwMBhfxXjv6eV9PxLFjFZMVUJSsOhoRv3u2UrMcdGNYQGT/udr5Ki5wkhNHwxOPxpTSv9W4/r/izci4JgDp+EuEiaKzLSQGk63SeUX3izYN/YMF69ocnFnq3ApYp+DWZJtECkykNApbC1rZ1T2l/y+XQXcp6c1ZNRKi60WApknuk6LQhsQPeaX4lC2fuMek/stJuZrnG6QS2pAa0VSSbpb96GNxGO0gBDqeykPLzRxGZy13x8F5iKcgwa9uefIgclR4tLKtSgA26JVtl0A40kjiuBLZXeL8tKpYk77WQaAxIyxkKkuI3XIS/zxr4YxwX+v0QkwGZlMuiZKzmWaDMmFL83id04+hXTvSH8VVHadEYPNQ/NngduW+oVg0qB77W2Fxft+Iov00MStA1LogEeAMg13heijrckP2WT1OWC0XvjRs4w5LaKH9BlbcusvHJXpREtRoidb2BQYelV6snv4i9+pMpsp/zyjfR3IbBJ4B3dgDUeHLKRCJgR/vpQhbgn3jiwol44+kqCbIoHYaCZ8ddux7jOTmOgH/c1qLWCZrgURnh8t5F6tA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(8936002)(956004)(5660300002)(6496006)(6862004)(38100700002)(66476007)(508600001)(66946007)(26005)(8676002)(66556008)(53546011)(966005)(6666004)(86362001)(6486002)(2906002)(83380400001)(4326008)(6636002)(85182001)(9686003)(316002)(54906003);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UWwwNEZuWkw1N2o4bWxzTVdjVWpsZ0t2YW9GSUlTRElCMmk2WThtQ3FSYkY0?=
 =?utf-8?B?ME9Cclo0bFcyNGFTcXdha1FYakZqeWR3enNXblpWRGlncmx0VmR6RGxFVUFU?=
 =?utf-8?B?VG1zSHJUQ1ZnZTZ3aE5KZUt1YUsxbGtmRU8ya0gvQmVORjhncEVWNEdHN3lP?=
 =?utf-8?B?NGNPTzdnOU1LZXUvSmRhb1VQRkU5ck5BeGptV0hWVmpaZ1UwMnFnYmVEbGJV?=
 =?utf-8?B?dEhLeUhXalpySndYcGE1ZHlpNXhUZjFHUW8yMWM0L3RWekRaVnZabXN0VGJI?=
 =?utf-8?B?bVp0WFJNOHk5R1lHM0ZidHZTTjdNM3VzeWlBcHRHU04yQTJzdk5EZVFZWThP?=
 =?utf-8?B?MHlDckMxTU51TVRFUzBtb0YyZVBBZXNGTkFhUWdaN3Z3UjVmVGczODlsNHZM?=
 =?utf-8?B?UVRpajVJZTZrd2xQUGlvTkxSblhwVk1XZ1I4cFdLaytpWk9NRmRXN01TNE4v?=
 =?utf-8?B?M1BuYlU5Z1F0TThJSFNjclRpUUJQU0RGZ2xpSzBxc3B0R0lVWHpxKzM4dXd1?=
 =?utf-8?B?ZEVJcWlxTzZ4cGZPNnlwbTNoeVJHWllHODFwbkNtK1o3d0ZkWFg5eGNTemJh?=
 =?utf-8?B?TkVQTHdVNlNFTXczMTlMUUxXWGpIUzVVK3p4ajk3UjB1eFFHQmJLZGc5NmhN?=
 =?utf-8?B?K2dYWHFOQVltRHQxc2dxdVpIL2ZvcmZUUEIvemdKK3ozSi9DZk1qTHo4cm1H?=
 =?utf-8?B?QUtsb21NbVB5VmhqMFZLQUZjMi9HWmF1MG14c3p6NDlZenlMYm93YWZ4dGtw?=
 =?utf-8?B?SUh5bXBTd1pPRFd2ak5MZ3dlUUErcmZJTUZqVDVhWVQrSXo0S3NiMkpKSGxM?=
 =?utf-8?B?b2tnK0tweGNLd0xVRE1xcW1tLyszN3VZUkRXU0dJd0RBaVFxTDNtcEw3N2FH?=
 =?utf-8?B?Y2hKeDMvNVhxbHV3a0dTTTk2N2xaUXY0Z2hHWmJJQUtBWDJ2QUFqMXYvTkpk?=
 =?utf-8?B?ZHdDRi9UbU9XZDhGQkxndEN0c3FvbnFHM1pXZlAzWExLZSs0dVYwbWNZUDF4?=
 =?utf-8?B?N1YvQlpXOWNWUnM5blBXL1dFL1JxK0lBRzRZbmtwcTBBTnpuYUEvTktwYWdw?=
 =?utf-8?B?cG1hNS9WeVhWVTNGR2tIMWs5bS91WUx6VGR6WWF2dDdjMTFja1FjaXY4M0JY?=
 =?utf-8?B?cDNMNjM1MnpWR0F4Rmc2dHlVQjBBSWZCNWpRL0g3RGhrQThFcUFQK3lOclRY?=
 =?utf-8?B?ZERxcGlsMnhkenVjZzBqTU1ZZzh1SXpXeUZuNGYyK1lnQTNvVjVqMGpNUlBD?=
 =?utf-8?B?cmlFY2xFcXA5SHNGbG1KaDNUWXM2ZlhoaHQyT25IeFNKUGNnTEd4QkxwajI0?=
 =?utf-8?B?OE0wQWpkM2hMNzFjY0tQWSsvbXZFMXBES2xWSCtxdnRWOHI0Y0llSWQ3bDk5?=
 =?utf-8?B?ZFdFQllUNFBCZWpyMk1MaWpmVDlydU5JS01TSFIwMVJVeFJGMyt4aUs0U1dO?=
 =?utf-8?B?MXlMdXo3UzJiLzhWaTdIdS9ZM3gwaUR6cEdkUkRYejMrVkNPVGRCMGtQbEV4?=
 =?utf-8?B?RmMxb0xtNFZTV3cyZ2djRzA2bTJtaGpIZVJLbkhGRXNPSUJjemFadkRBLzZ2?=
 =?utf-8?B?MjhGSmhWK1cwRGRWUmM5WnpDYjJKMXM3eXc1WkFWQkt3V01rb3QzWGVtRjM1?=
 =?utf-8?B?bHNuZHpqa0dDV2RhZUozZ3VwU2lSUkVUQ1R6S3BYTE1wSzhGV1NsR1pZamhI?=
 =?utf-8?B?N3NKQmVhK2ZsRUpCVUZiR3J0c2lKU0Uwc3BUTWRQb1BKVkxvUGRGYnBMd1RK?=
 =?utf-8?Q?OdqYdTHONCVcjleUiieE7rSiNRR6JAmeChT3v4f?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 334a77eb-6421-4a13-c052-08d97f372a01
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 08:41:55.6532
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GJ1brtyKVI+esUhwc0ygZ0I5UQ/MnPiYwC+5YLp96x72Z5G5TiLJUBeMQLLKOVGWCI7NdzNDRGvmenbRDJYN8w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR03MB6094
X-OriginatorOrg: citrix.com

On Thu, Sep 23, 2021 at 09:59:26PM +0100, Andrew Cooper wrote:
> On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
> > Suggested-by: Julien Grall <jgrall@amazon.com>
> > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > ---
> > You can find the related discussions at:
> > https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
> > https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
> > https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/
> >
> > ! Please note, there is still unresolved locking question here for which
> > I failed to find a suitable solution. So, it is still an RFC !
> 
> Just FYI, I thought I'd share some of the plans for ABI v2.  Obviously
> these plans are future work and don't solve the current problem.
> 
> Guests mapping Xen pages is backwards.  There are reasons why it was
> used for x86 PV guests, but the entire interface should have been design
> differently for x86 HVM.
> 
> In particular, Xen should be mapping guest RAM, rather than the guest
> manipulating the 2nd stage tables to map Xen RAM.  Amongst other things,
> its far far lower overhead.
> 
> 
> A much better design is one where the grant table looks like an MMIO
> device.  The domain builder decides the ABI (v1 vs v2 - none of this
> dynamic switch at runtime nonsense), and picks a block of guest physical
> addresses, which are registered with Xen.  This forms the grant table,
> status table (v2 only), and holes to map into.

I think this could be problematic for identity mapped Arm dom0, as
IIRC in that case grants are mapped so that gfn == mfn in order to
account for the lack of an IOMMU. You could use a bounce buffer, but
that would introduce a big performance penalty.

Other question is whether we want/need to keep such mode going
forward.

Regards, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 08:56:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 08:56:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195031.347488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgzv-0002V9-Sr; Fri, 24 Sep 2021 08:55:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195031.347488; Fri, 24 Sep 2021 08:55:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTgzv-0002V2-PG; Fri, 24 Sep 2021 08:55:59 +0000
Received: by outflank-mailman (input) for mailman id 195031;
 Fri, 24 Sep 2021 08:55:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTgzu-0002Uw-FA
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 08:55:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3abc449a-1d15-11ec-bab0-12813bfff9fa;
 Fri, 24 Sep 2021 08:55:56 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6-6bHg7BIhPGSQ0mNpn5SSDg-1; Fri, 24 Sep 2021 10:55:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3773.eurprd04.prod.outlook.com (2603:10a6:803:1c::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 08:55:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 08:55:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0051.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:55::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 08:55:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3abc449a-1d15-11ec-bab0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632473755;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oVQQTPXxci+0Cg1D5M+tpUegLfCxCcaSTPoNvSXUPpw=;
	b=l7m4Z9XuYF914ttfRiI2uHMTMD2h++KGldNt+sjfhyvDOk9XGfSaTJmr/QZ31Ndy3AnIIh
	15zf7l9HjojSbqeg3mbP9coOuP8zklxRG5hlOnIz6Y2pr4ad6SrunCwIBcngQc23PqoUM0
	KP73v9uLgyR97rgdATUv0IBeMTsyeeo=
X-MC-Unique: 6bHg7BIhPGSQ0mNpn5SSDg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kVc/JlSz+BOOldySCKMvYXOOHFIx9j3tzq1cK/rJf8110VMtJGb9+KF7ho5PC8/lSa3pDKKCKDqFTxxx8hdw4vuQHsX+83bL8+kB+y96UmC8mPKGJqqkEpb7xnSL2GfiPy5CPYktRByJ6ODy8k4XqUH2Aga3bab4hr/O3Q2ArUf4ZWlZmMvydIwCFd5XDpyTzs3Z8OmSfccHQcnBtVR8AHffUyKqru0sJG24SrMS7hBa7WOLmMvY58jDwj59WlbpcM9AW3LwdmDmFiI2aH24cLQ4igJS7HKjZl6JnPs+RqJ1+iHt4U09GoHZqfwDI/toiCfndbkZbxiFtGi3xPXj4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oVQQTPXxci+0Cg1D5M+tpUegLfCxCcaSTPoNvSXUPpw=;
 b=ZkkAjew49PnH3QLtRAHKDissVwDe/tGqdAvQqouTVggKMNGN6xVysbq5whiE/DXEp3KSNg3p7E7wKORly7sHBHnPM3yKkQU6Favxv9ikp5VNYM0kskC9TgLpF6547Ufkj2I2cLv1xm3LlNFl5XXbx9xT2g1mCfD3iAk+cNcnItrQlVzZNpt4eynIsv5A17EOaJMlkfTa9dmnhwN2gb1rEVduR9mazlsuW9KeC89W2Ut4JjaXGywrlBGihVvayK/8MSlmao2t918cCXNASWB8CBguFIXyXuFki6xER9yZGdI+n89kqAjJJGU3G6YkVUqS6NwgAQJ5VS7ZHMsLSMZMRQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
To: Wei Chen <wei.chen@arm.com>
Cc: Bertrand.Marquis@arm.com, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org, julien@xen.org
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-3-wei.chen@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9bec4214-5f50-9ea5-0212-52d248b63ca6@suse.com>
Date: Fri, 24 Sep 2021 10:55:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923120236.3692135-3-wei.chen@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0051.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:55::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3bbca9cb-fb5a-402d-cd4e-08d97f391ced
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3773:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB37732F22C84A90D56253F7E1B3A49@VI1PR0402MB3773.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NhPIuKa9eGml2mDnEdRp0mP/0jV2CsaypgsvjGVts3lgO/3qXPJU8SSpOSdxOMcWkjs7EaStK3FogsuoFa9VtxqxkxPN9KbrHH6yIQYOWuzKwr4HDJZF2r0ykNghSczO425Mlzq0n8ZxEpdgH37HQXZr4YeSINh9lJOYCvC+6UovDOhHR0QtrqZKt1lHKgiLbGe0rErHUaFtjzvX8O1mEiFlGR+PpYh3goO+qY2jE/malmA+hKcg3drlzh8kjwEN+CNep9A+N5O3koV/VmxqLuQQxsD/vF/dgZ2HEkCpRRrZKnIrkSt9poi9J+zM6HmPBQMprUOngDm7G78eZlEHiWyJ7S4oXWHqoPn7cI5ScrodojONg+J/gRdR7QXZNmfXPkZAMOmCPuoJIsYwhiCG7TCyuTjABFNvD1bbLWMJVeEMAGmp8RKRCY0kLeE1qQD5g4FMfZge8gjtN0LAOkRjdlnUC3q6cgjwBZL72TrHZjcNQi2SnryTx/rE5wdwfLEnJFao7ClwbAjwXmNq2dJ040GxpEGl8wnSboBLlURJouiAYLyrrdTJIPpzWH33KW2Hg6GQesgUVtG9mYhlbFzLBv/9+A1aXaXixSX/51GM3zLnstcvZZrnf9pD5R8HmC3t0mxF6tOjQkQP/SvRSod4tfy2I4dltDth9LvUIWvX3WrcVR2u7NAXTjXO0lOr7ZzcHCnhPlYFUpwTFIPD9QnKOgdI9zKZhKa6iyp3QggXGcA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(53546011)(2616005)(38100700002)(956004)(26005)(31696002)(86362001)(2906002)(6916009)(5660300002)(36756003)(83380400001)(31686004)(316002)(16576012)(8936002)(66556008)(8676002)(508600001)(66476007)(4326008)(66946007)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L1lTK004L0RkckVRTUd4YVNiSHcwa3dmY0NoQWY1Um94a1hpaTU0SnlFVEw2?=
 =?utf-8?B?WWh6VGw3azBzVXpueWlleXlUK3V1Vi9LL2tkYnR3M1FNdHo1UjRiYk1uRGl3?=
 =?utf-8?B?UzFKVWtTUTZFbEtlK2h6bFdTaEtBbFdjbm1uend4ajFLekN4TFNDajlLM0RB?=
 =?utf-8?B?S2N6T0NFTklJM2ZaS1BUL3o2ekIyb2IxRGFOZXN4MlhnSkpUWStGK1h0M2lK?=
 =?utf-8?B?K0UvdXpPVzBFRGJ0VE5YaURxbVVlckh0NWFkVjdteC9lK0VhbDZlZTc4QjJ6?=
 =?utf-8?B?U1FoYWY4RmM0ek1VUDNCVGdsQmFldHFLRGkzaWVZb2pBOVJuSEVpUTFTbTNV?=
 =?utf-8?B?RFJCbUpzNFZDZ1JQbTVKQ0s1djFEcEJQazhiL2p5ZEplWnNRWGczT1doL2RK?=
 =?utf-8?B?UTJDcW96Rkk0cUFQUHhxcXJSZllBbTRuOTBXY2U0TTYvQjUzRXFPR3kvQ1By?=
 =?utf-8?B?UXlmZ09XM0tIVHo0YUZBc0RtazFlWEgzcDVXTzYvcEVnR2NNTjhEakVFcmlE?=
 =?utf-8?B?R0t5Q296czkrTEhjajRjOUdqTVJEcGxIeFV1VEdWbkFIY1NpOEpjQjJmY0Rt?=
 =?utf-8?B?NlUydmRTa0N5VWtNbWpUZlpGaDRDL093alNGcTAxeXRwNm9BZEZLL0lWZnU3?=
 =?utf-8?B?Z3RaMit1NS9wS2RwTU9JclBXYTl4OW84ZVB2MCthV29Pa1YrY2hpMnUvWVll?=
 =?utf-8?B?SGhyMTJ6bmxxeUt4UE5tRVlUY3dyVVM1U1F4WDJLV0hXd3M2NTJOSmsyU3Nj?=
 =?utf-8?B?MHZiRFh3TmVER1g2WkpvOUpXb25PN2p3UkRIWlprdEhxd01kYXBFMEc2ZmRM?=
 =?utf-8?B?T2VOMEc3VlA4dDc0a3NHelExazkyb0hVUFVPSkFlNmR2UFpEODFkOWh1cWxD?=
 =?utf-8?B?ZEYySU55QVQrclU5eFdEcHNPWFg4dlcyZXR3NE9OckZ1MWU3dlZENjdwQW1k?=
 =?utf-8?B?L2RheDlYY3FvVzZEVElzYUV1dndXVyt3OWdxZnN5U0lHdEU2U29SR2NhWThO?=
 =?utf-8?B?MUpadGlVcHZMT0QyZy9EWS85Y2NOZk8rVnRyL0lnVmFhaVZCT3Q4THROVzJ3?=
 =?utf-8?B?SWN0ZGtqdVM3bDNDV3I2M3I1aGxkNGIvL01ObFFTQVZsWXZBZTFTNlA5akxE?=
 =?utf-8?B?UmtDTDgrMUtsWDNJb1pja2lRV29uak8zUytpTkdLLy9yU3pXb3Q1MTkxalo3?=
 =?utf-8?B?NEZMRHBEZCtYWFR4ck4zR2h4dXo5MVo5Z1FKeFVIeFBjQnlVeGlFNjI4dmRo?=
 =?utf-8?B?Nll1MFpaeVJkSzhrZExSRFYyQmRJNmdmVzZOT09PamwxdG1Fc0IrSnBpQmRz?=
 =?utf-8?B?VTRBck1iVHBwWkgyN2FkUlliQTNMTzZSUjhqeEZqMDQ1d2FKbnZKcG96NVR2?=
 =?utf-8?B?YldvT1ZUWVQ4TXk2ZUJIWGZsWmNTUFNnT21QRVRQZ09SZFRuVFQ4RXZEa05Y?=
 =?utf-8?B?YktpNEN6Mm9EMXM2R0YybSs5Y3dvMVdvQ290N2Z4VU9IUURYM1JYUnZ0dmFm?=
 =?utf-8?B?cTB1UDlmUEtkZ2gzWk12b3JPMDFyNmpXMkRjSHZKbWVlM00vakFFQnlrVzJ0?=
 =?utf-8?B?MmcyMFQ3emR0TGh3RUsyWUFDRkxHMjZRdEhGNEExd0ZHK0RsU0JRRzB6a3RI?=
 =?utf-8?B?bWtWd1hnWEg0SmNwMWFKQSt1SGFNbkZPU21wUDNTa043aHFrS2hNMGtaMkRV?=
 =?utf-8?B?VnA5UlJFT1c2YlRKcytuTjk1Ylc4bGtWZWdGM3FGTGVRNW9jbjZka1oyZ0Va?=
 =?utf-8?Q?SRDDc4zHbd8RbvlTJh/pM2K/tUI0W/PjwX5nTrz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3bbca9cb-fb5a-402d-cd4e-08d97f391ced
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 08:55:52.6944
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QAN4tFWXLZ6KIrHkIAOBdl77lRxDY4mDT82ioO3VDSYMIk8B4LJOoqeh8CbND2Jwfx9VHye//RtfpgeNsEbT+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3773

On 23.09.2021 14:02, Wei Chen wrote:
> Current NUMA nodes number is a hardcode configuration. This
> configuration is difficult for an administrator to change
> unless changing the code.
> 
> So in this patch, we introduce this new Kconfig option for
> administrators to change NUMA nodes number conveniently.
> Also considering that not all architectures support NUMA,
> this Kconfig option only can be visible on NUMA enabled
> architectures. Non-NUMA supported architectures can still
> use 1 as MAX_NUMNODES.

Do you really mean administrators here? To me command line options
are for administrators, but build decisions are usually taken by
build managers of distros.

> --- a/xen/arch/Kconfig
> +++ b/xen/arch/Kconfig
> @@ -17,3 +17,14 @@ config NR_CPUS
>  	  For CPU cores which support Simultaneous Multi-Threading or similar
>  	  technologies, this the number of logical threads which Xen will
>  	  support.
> +
> +config NR_NUMA_NODES
> +	int "Maximum number of NUMA nodes supported"
> +	range 1 4095

How was this upper bound established? Seeing 4095 is the limit of the
number of CPUs, do we really expect a CPU per node on such huge
systems? And did you check that whichever involved data types and
structures are actually suitable? I'm thinking e.g. of things like ...

> --- a/xen/include/asm-x86/numa.h
> +++ b/xen/include/asm-x86/numa.h
> @@ -3,8 +3,6 @@
>  
>  #include <xen/cpumask.h>
>  
> -#define NODES_SHIFT 6
> -
>  typedef u8 nodeid_t;

... this.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 08:57:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 08:57:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195036.347499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTh1M-0003QS-7J; Fri, 24 Sep 2021 08:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195036.347499; Fri, 24 Sep 2021 08:57:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTh1M-0003QL-4G; Fri, 24 Sep 2021 08:57:28 +0000
Received: by outflank-mailman (input) for mailman id 195036;
 Fri, 24 Sep 2021 08:57:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTh1L-0003QD-Cu
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 08:57:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 706235dc-1d15-11ec-bab0-12813bfff9fa;
 Fri, 24 Sep 2021 08:57:26 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2057.outbound.protection.outlook.com [104.47.2.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-Cs9DX4kzMJ-iShH5feWU1w-1;
 Fri, 24 Sep 2021 10:57:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5743.eurprd04.prod.outlook.com (2603:10a6:803:e0::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 08:57:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 08:57:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0003.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Fri, 24 Sep 2021 08:57:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 706235dc-1d15-11ec-bab0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632473845;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O0+qqISdYcBc46T1s3IsLnxBbWiKFt4J3XldhYnLsKI=;
	b=ntptDiWFMB9l5xGu3UuTTgu97O/ABiP1p7dTyohNJsEHmdVnyd/GahQv0cPMZcvLQl3EJ6
	PP1BFqBkK8h/jGxZsTM1YGrdTROdlm7wqiHG6Z+FMiBOIsGYt7EkRj+BKwxvkHXkAc6lcW
	iUWI5PzFQmi/xQDnr0hY2P0m497OJPY=
X-MC-Unique: Cs9DX4kzMJ-iShH5feWU1w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oAIm6kD10/5aOGVDuD3dKBGMNJ0ycHk5y0zefr9QjnWpC5qu5TKz0aDhmDAcObeNkHmMMTY+ajtDQUxx5z0oQA3n7/qJr9ulcUTazAf4Dozo/NoTp4Qm09+wQTCG1GW+bEf+Wyp6n2/1XTLxls8e/1IZfJJ5PkDaG/VQbOrDnCh2bYwr/SLJ7WiAL65RqWXZqd5uSiLHVXArBG+ewvkg7CY4LHMtxLqkPvikUqZWSSkZb3G31WbckK9ZD9vXJmLWD0wGUC99jkv1kktx+yQHLPczoh5SLQo7lb+QSzLj/HR8WcCUCHDpc4Hzwn+HtBcPBSdVcY5pP8XLoyKnoUfK6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=O0+qqISdYcBc46T1s3IsLnxBbWiKFt4J3XldhYnLsKI=;
 b=iH/wi8Z0Z2PVXcXnPilvPWtJTyThhRsz/9514a0euJ9TVyylyFG/NIyK+rAjyl77c8n7K7jpsjaZNR7FDNM8fbAAjAiUHfP1oX+JUgpV36p/cKusaetFSFOgzhh2E06PLFPYnfkDt5Ac4U8hYWETjs09oo8QOZ8APfVnQHVqUXGUWRSC04OjDpp3oITHeoBJqsvYmFGNa+YtVVRvMY9A52sN4GO8dX8jxj4gHwGtSq0HbgzUxT95X8Df894utIgYt6iQLbJerSJSUcCtyN7ab3wPsCH2hL4GtofvdYF6spptyNllv6b1M6MOg2d4XUgGMYmU8KXfDNqRiWUGwUzOhA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 03/37] xen/x86: Initialize memnodemapsize while faking
 NUMA node
To: Wei Chen <wei.chen@arm.com>
Cc: Bertrand.Marquis@arm.com, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org, julien@xen.org
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-4-wei.chen@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <27150d2e-b269-c1ef-0e73-bdce1975dcb4@suse.com>
Date: Fri, 24 Sep 2021 10:57:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923120236.3692135-4-wei.chen@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0003.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fd59f896-4906-4e0f-2847-08d97f395251
X-MS-TrafficTypeDiagnostic: VI1PR04MB5743:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB57437D5D7B87B87E61D11597B3A49@VI1PR04MB5743.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OuJVXPLteKl+GKj1K8GrkAVrvIadLMhdp/loNjN+vpgiNHXdSYg3FH7sjhixEpfVsXk3HSoHzWhrX+rIDOjSnZF8pNwy1pY38jlY/P+NjRXmn+ZnX10trlGMXiL1fpdqF6/5+mf6yV7FBrjbLaZy7SExJMCZirG7W6AugzcBBxHN9Z7yZ6yCmBKl7yv9Jsr1E6z3dF2qgqOez0ctoKYtogCI80aQOLSMreVNFnAYh5QReQCfoX4xERmadKhX6UFSwhTz+rosHpVCvitx1YKoroNtGJgm0MdreT5t3Yk9gSDIa0MPcba/E0KnQkrR/r/lhAzV+T661n8fjR0gSsbPnrfEVnOts1QOSS0CwG0KTHY+1hmqGZJoNiHSMYeez1ogQ5pj7wK5U4r/LKkDMoJoSpvQ2R5D2UKCMNRShnQcPgrGSOp6FJWb8i7+OjGDbTpDf2uRrgto5ZPC2xS50ZSJbMndJEPrQr1fwCXncqyfmQoJ4wxXBEOT2kQmkkmuPP8nA9p1FjKOQ2a4rPMfRIWpbGZvRiQ+pCi1rT8lvM+CsURetEaUQZ2BS2lCE9wQyi5OvC9uNSrmvl1LvHgorOIuR2KYkD1Q0VBkOIluNNFlS/WwzlYkdBhfkiD0cFK6A/fwzW1V08rr6RZn4dJdER39gwa/uTmZsJTttfqMq66DHjiAwAAaseHEatQeZbJYeSD9D9egE2ggziuzQ4DSwej07EiH9TOOisK6aCmR7qIhh5qn7JbGOVTV1d34E7+gHP5b
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(508600001)(2616005)(956004)(26005)(8676002)(38100700002)(31686004)(186003)(6916009)(36756003)(2906002)(16576012)(86362001)(316002)(8936002)(31696002)(5660300002)(6486002)(53546011)(4744005)(66476007)(66946007)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eloxK2F5K05SUWdUUjNQQ2EyUTh4alQ0TTZidWNkYW9ER2krZU9iMFk5TDEz?=
 =?utf-8?B?bXZnTmNyRDBnZ0hIRGVDZld0emgwcUZpVktJRFRsTitSQXFtdmdJcFZsdEdk?=
 =?utf-8?B?QW1JdEVEVXFNbnRrVnZwSUtOazJFQ0VkQnFRQy9oTmdUZFQ0S0UyK2JxRG9o?=
 =?utf-8?B?UlpNcHRWaWNWbmZKVUQ3aTB2VENaNURhblpZRHVhTFBhTXhYd3FycW0vZGhP?=
 =?utf-8?B?dDJZZnVEYnB0QzVRU2IxTDAvbitpblMwODBEMnNMWkJCaTV4cTk4RkhDdkJi?=
 =?utf-8?B?Y3Y0WFV3Y1RSWVlIWkVRWkovNkZHV0VkS2NLUmVtbnRnQ0lFeEpGd0x4U1pI?=
 =?utf-8?B?V09pd1pGVlB6T2Z5SE54eWpUZllZY0dHUG01VHZtTDNySjVueFlwTXFBcWZz?=
 =?utf-8?B?dUgrSmpGYWFpK05WVjFMWWhYUTdpS090Z3RON1NpdHoyWEJ1QmRxd2xvb3Zi?=
 =?utf-8?B?WS9ieW1LSTFFMjZTaDJEOVF1SHFOc3dLZUE4UytoUjVHUXVXZFExeCtzVU92?=
 =?utf-8?B?a0VnNVhhL0lkN0pnQ1FQVlRjcXQwQmNPKzNFTnZUbkJTWHFFaTEyUXhCZVkx?=
 =?utf-8?B?NGNPV0hZMU1MM2tadTlxeDhzQnBGK2V5UkJVSmhoUkpvQzJDMENVallOQlBl?=
 =?utf-8?B?ZXlBcHZZam9Md0kyQUZVd0tQVXlaUnpXU2dwYnBTcEM0czJuUlhlZ2ZqZ3BJ?=
 =?utf-8?B?akcrLzFvVTgvZzJsM1BGRjJ1SVBScU13NWlxOTlFWnRhaFdnZFFWVWNZZjJX?=
 =?utf-8?B?OTY2eklKU3B2NXlVUmYydmpCNkRBK3M5elRMcXp6MjBYaWpRWUhETXRVaktJ?=
 =?utf-8?B?MDltU09lYmxaRkM5S3UwSUVNbU1IY3BsWStUS0NXL2dvVE5CeXA2dE02N05z?=
 =?utf-8?B?emhkYzRnR3JOcFVSc0NrN3NCUkU3cERxWDdvUk1FV0tyWTNqZHZZbmdxUnNN?=
 =?utf-8?B?OXJ6Wm95NnB4K2IyUlQxVkMxNTQ1eGhXaVcwV3oyejA3dWE1RzNqWlFzZFdP?=
 =?utf-8?B?TWQyWHFSVUk3UlN0NmVMcUFBSGJ2MC9ZZmdYRnhPYmhOS0VqdWsxWDlCOVlr?=
 =?utf-8?B?SFdEOUJKUDlJcjhoOUFWeXIwWmxCc28yOU9zKzVXQnQrY0M3QmNGdVZxVzZh?=
 =?utf-8?B?NzR4NVVRTEs4UE5NK3Yvb2k3YkxDWEtqWitCWDk1MVBTN1RSZVF4aVY5dW5h?=
 =?utf-8?B?WXFRREZZbWNVU3QrUnhGK3BBTGlMc0RnbEhtL1NpNEpuZlk3OGtzTFI4QVRB?=
 =?utf-8?B?b0lXYnplRUVqalZFWmNIWVNDYUNmR2N0NWl3QzVlc2pGVWVKZ0xuMEV3T2I3?=
 =?utf-8?B?bVQ0UUhXMGt3cnRPT01YYjFZT3JKUE9zZ0U5anhCbU16TVJlVkplRlVRUU54?=
 =?utf-8?B?ZWtaRDJybWFtcTJPM25kMytWZmpteThCYkJXNjBnMlJPRHEvU3YreTV3TWRm?=
 =?utf-8?B?dkJOMVR3Qytvb0tLbGk5dVhtRUZ6TTZydVBya0hLWXpTYThQVnYzcnl1THg0?=
 =?utf-8?B?a1RCRnc2clR4VE1FRG1rRHE5NVRZRlZyR1ordE5xamJnajdqSlp1UDc4Zld6?=
 =?utf-8?B?bG5hL2ZlaGdqTnZ5T3h5ZmtsNk1SN0NNRUVFeU9PdStaUGs0Q3VHZFAyNHll?=
 =?utf-8?B?NUZkSHlHMGRDdTRtWnMvOXNvR3Eza0VKcUdVd08rSjdtTk1GOWI5U3h3QTk4?=
 =?utf-8?B?NURoa1U1RGhKa3lmcU04N3pTQ2ZiY2FjVy9mb1BxWVA2c2Z5Y0U3UVB6WDlz?=
 =?utf-8?Q?hlqu+94/ZNBRgTGuPBvPy2gkDS3m4sVEzQdInej?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fd59f896-4906-4e0f-2847-08d97f395251
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 08:57:22.1918
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Hn8VAUP8uuZe0RvmQXBbPrWe3HXjME4MLp+NKVVt3yk2SgZ4HoPkVnVGdfOCo8qVPlTLw2qlD2O9oZHk96Uvcg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5743

On 23.09.2021 14:02, Wei Chen wrote:
> When system turns NUMA off or system lacks of NUMA support,
> Xen will fake a NUMA node to make system works as a single
> node NUMA system.
> 
> In this case the memory node map doesn't need to be allocated
> from boot pages, it will use the _memnodemap directly. But
> memnodemapsize hasn't been set. Xen should assert in phys_to_nid.
> Because x86 was using an empty macro "VIRTUAL_BUG_ON" to replace
> SSERT, this bug will not be triggered on x86.

Somehow and A got lost here, which I'll add back while committing.

> Actually, Xen will only use 1 slot of memnodemap in this case.
> So we set memnodemap[0] to 0 and memnodemapsize to 1 in this
> patch to fix it.
> 
> Signed-off-by: Wei Chen <wei.chen@arm.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:10:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:10:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195045.347510 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThDz-00061r-F7; Fri, 24 Sep 2021 09:10:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195045.347510; Fri, 24 Sep 2021 09:10:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThDz-00061k-C0; Fri, 24 Sep 2021 09:10:31 +0000
Received: by outflank-mailman (input) for mailman id 195045;
 Fri, 24 Sep 2021 09:10:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RRwQ=OO=citrix.com=igor.druzhinin@srs-us1.protection.inumbo.net>)
 id 1mThDy-00061e-3i
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:10:30 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 42617344-1d17-11ec-bab1-12813bfff9fa;
 Fri, 24 Sep 2021 09:10:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42617344-1d17-11ec-bab1-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632474627;
  h=from:to:cc:subject:date:message-id:mime-version;
  bh=flt40ZSuy8Pfpgn6sTAAkDoqhWcDyGnzCI4H5hH2DIs=;
  b=Lg5JsFRbEJGWFH+pcvswaNIF/Jogzg4SNc2gFXYTzJyqhfJHd6uxeQby
   PERTPXOcwPakFqirinxpFy2WfeJXqsKD6oWsJGttGV+wX9ahcL6LZQ8ZC
   VShWdQJ0jy1guIzEFuWoX7SmexgbGameKX7zYg77nEuC8515XA51lMoUj
   s=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: Oln3FXpqFEwxf91ovtV68pqbs9UGCDCbxGbwh9T1JDi2Q5VYNzzFNWKUClKfHHfuVp5jRyXBlU
 JOQUTuz92tpFOzM1AzVYzHw/V0Db8i3PxcX1JH76BBDX0vEDOu4xkMAh3+TBB7ARP0mxHPzvEv
 /vumyUYgY/QE2qoX+AM4bU1l0Xsyhbf0ezfITnoM2R0qmXDJJi23pzTUpxTJARbZEultUDfjyN
 PZ96OhZAsU10WQ8zE03YRDn5WP3HD0D1yt1jGXgjIKTQQsKHv3PBaYZnvedICZr3n6m3LnM2O7
 ajXEShY85AnNTT3WdO2DGpJx
X-SBRS: 5.1
X-MesageID: 55337888
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:JCPae6nWl2dbiEd8f9ZNu63o5gxPIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIeCz+AaaqJYWT2KNgja4Sy9kkBuJSAn4VlTwRpq3hhEyMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Q02YHR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 NpGi4PzZDgUBIOPhN0hAxkHFA9vZoQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBMjkO5lZtmt93zHYC/cgaZvCX7/L9ZlT2zJYasVmQKuFP
 JVJOWoHgBLoPCUWC28QT68Fk/6ZgUmlcDZg933Oqv9ii4TU5FMoi+W8WDbPQfSSTNhRtlaVo
 CTB5WuRKhMQOcGbyDGF2mmxneKJliT+MKoCGbv9+vN0jVm7wm0IFAZQRVa9ueO+iEO1R5RYM
 UN8x8Y1hfFsrgrxFIC7BkDm5i7f1vIBZzZOO+Ix1h7O55Xp2SjaGlE4Hj4eb/I37+ZjEFTGy
 WS1t9/uADVutpicRnSc6qqYoFuOBMQFEYMRTXRbFlZfvbEPtKl210iWH4s/ScZZm/WoQWmY/
 tyckMQpa1z/Z+Yw3qOn9Bjshzu2r/AlpSZku12KBgpJAu5/DbNJhrBEC3CHsZ6sz67DFzFtW
 UTofeDEt4ji6rnXyESwrB0lRu3B2hp8GGS0baRTN5cg7S+x3HWoYJpd5jpzTG8wbJ1eJWWzP
 RCO5V8LjHO2AJdMRfUsC25WI552pZUM6Py/DqyEBjawSsIZmPC7ENFGOhfLgjGFfLkEmqAjI
 5aLGftA/l5AYZmLOAGeHr9HuZdyn3hW7TqKGfjTkkT2uZLDNSX9YepUbzOzghURsfrsTPP9q
 I0EaaNnCnx3DYXDX8Ug2dRIcA9WfSRlX8CeRg4+XrfrHzeK0VoJU5f5qY7NsaQ890iMvuuXr
 Hy7RGFCz1/z2S/OJQmQMygxY7LzR5dv63k8OHV0b1qv3nEiZ6ep7bseKMRrLeV2qrQ7wK4mV
 eQBduWBHu9LFmbN9QMCYMSvt4dlbhmq216DZnL3fDglcpd8bAXV4du4LBD3/SwDA3Pv58szq
 rGtzC3BRp8HS1gwBcracqv3nViwoWIciKR5WE6Reotff0Dl8Y5LLS3tj6Bof5FQeEubnjbDj
 lSYGxYVo+XJsrQZytiRiPDWtZqtHst/AlFeQzvR44GpOHSI5WGk24JBDrqFJGiPSGPu9ay+T
 uxJ1PWgYuYflVNHvocgQbZmyaUyu4nmq7NAl1k2GXzKaxKgC696I2nA1s5K7/UfyrhcsAqwe
 0SO5tgFZunZZJK7SAYcdFg/c+CO9fAIgT2Dv/06LXLz6DJz4LfaA15ZOAOBiXAFIbZ4WG//L
 TzNZCLCB9SDtycX
IronPort-HdrOrdr: A9a23:W/AIfqDxvfEdkbTlHemo55DYdb4zR+YMi2TDsHoBLiC9E/bo8/
 xG+c5x6faaslossR0b9uxoW5PhfZq/z/BICOAqVN/JMTUO01HIEKhSqafk3j38C2nf24dmpM
 JdmnFFeb7N5I5B/KTH3DU=
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="55337888"
From: Igor Druzhinin <igor.druzhinin@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: <jbeulich@suse.com>, <paul@xen.org>, Igor Druzhinin
	<igor.druzhinin@citrix.com>
Subject: [PATCH v2] pci: fix handling of PCI bridges with subordinate bus number 0xff
Date: Fri, 24 Sep 2021 10:10:12 +0100
Message-ID: <1632474612-9940-1-git-send-email-igor.druzhinin@citrix.com>
X-Mailer: git-send-email 2.7.4
MIME-Version: 1.0
Content-Type: text/plain

Bus number 0xff is valid according to the PCI spec. Using u8 typed sub_bus
and assigning 0xff to it will result in the following loop getting stuck.

    for ( ; sec_bus <= sub_bus; sec_bus++ ) {...}

Just change its type to unsigned int similarly to what is already done in
dmar_scope_add_buses().

Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
---
v2:
- fix free_pdev() as well
- style fixes
---
 xen/drivers/passthrough/pci.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index fc4fa2e..d65cda8 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -363,8 +363,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     /* update bus2bridge */
     switch ( pdev->type = pdev_type(pseg->nr, bus, devfn) )
     {
-        u16 cap;
-        u8 sec_bus, sub_bus;
+        unsigned int cap, sec_bus, sub_bus;
 
         case DEV_TYPE_PCIe2PCI_BRIDGE:
         case DEV_TYPE_LEGACY_PCI_BRIDGE:
@@ -431,7 +430,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev)
     /* update bus2bridge */
     switch ( pdev->type )
     {
-        uint8_t sec_bus, sub_bus;
+        unsigned int sec_bus, sub_bus;
 
         case DEV_TYPE_PCIe2PCI_BRIDGE:
         case DEV_TYPE_LEGACY_PCI_BRIDGE:
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:28:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:28:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195056.347528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThVC-0008KU-0b; Fri, 24 Sep 2021 09:28:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195056.347528; Fri, 24 Sep 2021 09:28:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThVB-0008KN-Tt; Fri, 24 Sep 2021 09:28:17 +0000
Received: by outflank-mailman (input) for mailman id 195056;
 Fri, 24 Sep 2021 09:28:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThVA-0008KD-K3; Fri, 24 Sep 2021 09:28:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThVA-0005xt-3u; Fri, 24 Sep 2021 09:28:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThV9-0006x8-Nd; Fri, 24 Sep 2021 09:28:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mThV9-0001DJ-N7; Fri, 24 Sep 2021 09:28:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XCwaJN2IkdvgXZlQu//cqb62yOXlwDAv1hnrNTKfcUQ=; b=nMv39jxDTPbUFnp46Ual3s7MM2
	Ipyh+D6VobIWZgNWYlPFN73jxsgmkmDwqzz83N+ARP5RwFMIZz5GKFvbgXiBfl61BxOHYA4dXPZxZ
	NRpca+iriay28Pe777dUO/75jsU9mIfKdwM9/mAzSqUQxTvldsitIe1eGN9FxSgUyDKA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165171-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165171: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=2c3e83f92d93fbab071b8a96b8ab769b01902475
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 09:28:15 +0000

flight 165171 qemu-mainline real [real]
flight 165178 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165171/
http://logs.test-lab.xenproject.org/osstest/logs/165178/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                2c3e83f92d93fbab071b8a96b8ab769b01902475
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   12 days
Failing since        164967  2021-09-13 13:06:52 Z   10 days   21 attempts
Testing same since   165160  2021-09-23 01:54:09 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4691 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:39:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:39:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195065.347543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThgB-0001nl-AN; Fri, 24 Sep 2021 09:39:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195065.347543; Fri, 24 Sep 2021 09:39:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThgB-0001ne-7F; Fri, 24 Sep 2021 09:39:39 +0000
Received: by outflank-mailman (input) for mailman id 195065;
 Fri, 24 Sep 2021 09:39:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gqZh=OO=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mThgA-0001nY-Av
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:39:38 +0000
Received: from mail-wr1-x42a.google.com (unknown [2a00:1450:4864:20::42a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 010ed95d-f68b-4f90-983d-4912a67057b2;
 Fri, 24 Sep 2021 09:39:37 +0000 (UTC)
Received: by mail-wr1-x42a.google.com with SMTP id i23so25587473wrb.2
 for <xen-devel@lists.xenproject.org>; Fri, 24 Sep 2021 02:39:37 -0700 (PDT)
Received: from x1w.. (118.red-83-35-24.dynamicip.rima-tde.net. [83.35.24.118])
 by smtp.gmail.com with ESMTPSA id
 c185sm8256519wma.8.2021.09.24.02.39.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 Sep 2021 02:39:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 010ed95d-f68b-4f90-983d-4912a67057b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=I61ARzsVO4/KzAGYtcJMZn32sXSEJVm6F+jp7uLlVMM=;
        b=bfJDRUtHlsDizMGatZPk963VYYfAkBfTjsNmiwj/ymhx0kRc6VMLElf3zBqxnARhDn
         lgmgbrEhIGHtgHu/8x+tntzdV2SngC2qWF2GAwfvOtMhG+V42rxSWucJiP7xIrIGReXz
         l5t9jVP65mPF1GmicTiZGk4oLjys78iV6BtOhMY43Z6OA7grVc9JwzvqRzjqVuuxwUkC
         i1rYTGO8A5FNd3Xng6BsOrIPE1h+jG56FVJbFE/XhtVgUXtuAxE+HmINDUE2vF+hCr/y
         7CIfaAlsit+sLqNVs49NVF94QACZCvY/8QbD4Llhgxgoqiw1AsyONwYLBrR6BOTi/Z/H
         7hpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=I61ARzsVO4/KzAGYtcJMZn32sXSEJVm6F+jp7uLlVMM=;
        b=M7lugumHkhZfDGtaKlcUfzmC7ap7BhxGMMj2hz3CFLvP3nBzA8JxpQeC0xT1NSfTYq
         PJajpvLz87D7uTxH4wafv6SZL6tTkQYnyB9KFGRlmo1vPB86iECK5HkB7vPj1YqfYT3x
         PT35d3LBWIFfAZTboZZ9Jt6CDo2OmkT9JN1qLdhZoF1Ynqe5qHtEOIs4YHwrow7W3mik
         bh8kfRGnKJkZTGwiQk85DXZMrHe5A34qlD0Eud9stmc3C3dYd7ElxZvX3fIr2qT1Dgax
         b+LAsdmxzlNI0xYjfJCQWAxsU4lka9QeJUD0M6mKgsFfrYuhAmTMUhecxVBsHgVpf2qS
         6w/w==
X-Gm-Message-State: AOAM53249hKo6V9hY7ZSvMssZxW/WZ3nzjQAZgqPu2B6jqbrpSCAdHJa
	UX3K14VNiD2JBmnHFx1smEI=
X-Google-Smtp-Source: ABdhPJzv1Bde65pNl0VvYERfNs+TTxQ5croc4mEh0wp0bxdkKK3MHRcKkC10fZSBqwi13saNHJwzfg==
X-Received: by 2002:a7b:c459:: with SMTP id l25mr1077703wmi.60.1632476376611;
        Fri, 24 Sep 2021 02:39:36 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Paul Durrant <paul@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH v6 10/40] accel/xen: Implement AccelOpsClass::has_work()
Date: Fri, 24 Sep 2021 11:38:17 +0200
Message-Id: <20210924093847.1014331-11-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210924093847.1014331-1-f4bug@amsat.org>
References: <20210924093847.1014331-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since there is no specific Xen handling for cpu_has_work() in
cpu_thread_is_idle(), implement Xen has_work() handler as a
simple 'return false' code.

Acked-by: Paul Durrant <paul@xen.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 accel/xen/xen-all.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index 69aa7d018b2..fe5a37fa2e6 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -215,11 +215,17 @@ static const TypeInfo xen_accel_type = {
     .class_init = xen_accel_class_init,
 };
 
+static bool xen_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void xen_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
     ops->create_vcpu_thread = dummy_start_vcpu_thread;
+    ops->has_work = xen_cpu_has_work;
 }
 
 static const TypeInfo xen_accel_ops_type = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:39:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:39:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195066.347554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThgI-00026R-I5; Fri, 24 Sep 2021 09:39:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195066.347554; Fri, 24 Sep 2021 09:39:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThgI-00026K-F2; Fri, 24 Sep 2021 09:39:46 +0000
Received: by outflank-mailman (input) for mailman id 195066;
 Fri, 24 Sep 2021 09:39:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThgH-00025k-GO
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:39:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 58f8a16e-1d1b-11ec-bab8-12813bfff9fa;
 Fri, 24 Sep 2021 09:39:44 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-sDFkuKTQPfKHbNhopJ82oA-1; Fri, 24 Sep 2021 11:39:42 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 09:39:40 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:39:40 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0054.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Fri, 24 Sep 2021 09:39:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58f8a16e-1d1b-11ec-bab8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476383;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=sAh4qxCYW2BkeZSUGmjdagQLfFE+wJDwsdXfXjxI/CY=;
	b=AMAk4YVJyJN16+V/PeBPWnhYo7oSW8xBL9XiSSBcBVbL2V9LIEHGiUVh2Oni09YUVKaaGu
	hzYdb1l55YUJCXB+afZAUarqRE6JdtaHqBN/BbxPZgJwcq8MtYfsRfgUvhTK41OEGMCJgW
	pdHPjd3fnlLHQYRlMa1ct5AseW315OI=
X-MC-Unique: sDFkuKTQPfKHbNhopJ82oA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nVK3FASUw85h6qK8WZk/uv0Y9e8+D4ac3zipZ28Gjj13aPWlhKWTxvIIOkwJSntuU4YDbWD8jJuWgMjrfH5OlvCzhlG3FKFS/un3Zf/SyXssNp3cDbQKXvCO54CIr9w8EfnE8VSfiNVJviGCyR+OIhqJdo41BihWKK0KXud/2c6qEGebIb6VsOYNblz5hH/4yKpptZEVZCcX94eF8j5TCK8P2ZrhM9pqBKkAiNWCvdR8Ws4hJV74xI4kl8rRrjOvXHRkAZwEF/CZyvuabSF8k1dFLoP9R7nUHkVhb529UtseKgND5T3NmM3oSBG6ao8vN6nMS9JZGLtlwXkm6zQdxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sAh4qxCYW2BkeZSUGmjdagQLfFE+wJDwsdXfXjxI/CY=;
 b=ciw3/yDfpd8ZR31w9h0em2oW1DTy5GzO4pgW6YLrohKmio2D2bQODcUvutnv7SJPHXy/T9U/Ix3rizn4ufwbPpTC7ESTZrivsgNUvZYYiPUGDWL/XwmDhLIlyHW1QgD8WiEId0C/mV42SXnuWTXkOWXPZa3DVV+rLsCxw3JMq0G0viXMOPPdIzM/kIJyCtZz3O8wO6i8PDu4yBjhYYtGUeXG+5BKeVJZtRb8c7ly5JxaqizZxt8UBsEKbZPsRKGH70RMf5nOo4Ydwh/L0pt/p25oGQ7dnrjACMh/3VBJS0w5Sk6msBh2lixj96UuYr+VVCo861apbeZeQw7HW4JKUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 00/18] IOMMU: superpage support when not sharing pagetables
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Message-ID: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Date: Fri, 24 Sep 2021 11:39:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0054.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40504fd2-599a-429c-ac23-08d97f3f3b28
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23344F5D079B49F80249FBC6B3A49@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZL02RNxQ0IkV3a83cKEJ3BKvyEQKJU0cTEpCIADaNBRMwHKdTMPuAh66VfNUc2TmHxl3i0zfI5mZIgVD5PmWvxTsBOJcYNVn10VWXE8fieOuyc4GItW3439quMamNdYCLnGCtvS+9Hvb9JNTRUSev3tmFf/vtrACvdBIEYRldJn6d0qa+AExN0saj596low/NSuk7OI5lHZCmp7eisI8L5FqcbmCGKwQU4FNwp+OpyuHHkTqtWYZdo86Qm3+AdxI3gnp6Qsn12jMwr4Kh/JQssHcplukV3MCksvs9i5E0/Ivo7CDt93i/UxFi1L4Ve5Gitr7RsUVKvY3/9Uo/+MePIuZzJWft8wU0vshnPuvS0fST/OD67k8LHNscCchQ0VaTWy/icIEAjrUj5qI7C8X+Ubtx17+UUUbijz6Hw2t9O6V2HXOvbvwFnvtorJAnGR3INYrcy/Cmoe19Gd7SthI+b22EKWbOzddwSO0FK5Ml867yU8wIUTR4QDfHfe40dxTKQXj/ueJZnnOlc0JNjUIxmQAdQbt50Ns/tcnJFJ+ZiqcezsNCgnm9xVplcod2QsfN+cJUuRm/JW1FH9TFt5/SppQVelUaYYlx7SDT+TLWZh0X4LMJBbjY0JKToaTSyb7N0q197gyi2wjSxrKYYV6H3mAfciDLQ4Myai5UYCplXpyEOf44JhikjVUCTr1tG6BgMLJmWl++BZ/9e0wd/0mjev4fACZS49Ryj4BSld2J9FxP470D5XGIRL3AQNrXJxiwtMQDUDY7pXPZ7SlWzDJNRwD/m1JvO7BL+gzTO01IFjb5CHEsAsa4FlKGhSdLLiI
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(54906003)(4326008)(66476007)(2906002)(31686004)(83380400001)(38100700002)(6486002)(16576012)(66556008)(316002)(8676002)(66946007)(8936002)(86362001)(186003)(26005)(36756003)(956004)(2616005)(6916009)(966005)(508600001)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?U3ZHWTJvL1dPaE5Ub2VucHVIZkJnVTJTS2NiV3dTV09QTGVKUkFrOWRwbUs2?=
 =?utf-8?B?Z3ZqWCsxMksyeExvc2xHS2JCWW9FS0dRdzQxcHJWbHgzcERlaExMOUtlZWc3?=
 =?utf-8?B?SVJhRnRuREJheUQ2WGdKOThjYWFqcHNRNkdoQlN1U0lEcFJ2V041Rnk1aGJE?=
 =?utf-8?B?RS9GM1l0R09ZOXByT3QxcVRvSDdnblBQcjFtdW0wRTVMWjRTaXBpd0xFZ0ZR?=
 =?utf-8?B?RlorTEZIUiswOVdxcmZxTkhNajUyVDVDa0V6V0JnbTM4SVBvSm9ub2JpbzI2?=
 =?utf-8?B?QnJtZU9qM1VVNXAyZzkyVW9qWlI1L0hRMmNIYVRkMTQxTWNRZGJwREo2Qy9p?=
 =?utf-8?B?bmh2Z3dObXA1Y0s0TmxZRFA5VmRHb3ZzMm5GckpCL2dhQ3hyeTNsSnRWR3hD?=
 =?utf-8?B?em5sYnZrYWVrM2k4NW9VV1A4RU5JYWIwc2VjR0pyM0s5RHQrYnAxWThFa3Bm?=
 =?utf-8?B?SERhdlZGNUg5bXZRQnVZRE9iVTlWOXdmMWZMMFFhN2QyRlZkUUZkajBpV0pZ?=
 =?utf-8?B?eFNoT0F2Sno3eGxVdndmSTFRbTNEUEluVDNBYjdqWjZBem1rKzdzZmNuNkZ5?=
 =?utf-8?B?eURiZTJtdlEzZ1JSY0JaR0J2cWdrODV1TjRnbytpOGxpbGNPaTlLUmFFaXFB?=
 =?utf-8?B?SkQzWGRHUFRkbDQ5OFRhUEFKWmNZWlU1bkFzYmZ4R1JuSzdwUWFSTjlMMnpz?=
 =?utf-8?B?ZEkrMm80bFVmRjlhRTVqOFFwcXNXb2NtajliWjFzT0FPMHphTjZrNUZVVkZS?=
 =?utf-8?B?WTQ4ZUJCbTR5b2daT1g0RlFiSStHajJ6eWFuY3ZyN1hwOWh5MVZITXBpaHR5?=
 =?utf-8?B?VkRHcm0xbS9QMUI1VWQwV2ZuY3RSYTBPbGFNa0grMlQxalZmbUFMbzNWTkI0?=
 =?utf-8?B?Z1N0MTZPV2UycXlUMFVMa1NiVTZ2NXVSZ011emVQenI3NEVBbDQzOVgwZko0?=
 =?utf-8?B?dEdSdzNuTVl1OG5lWGRLazlKVXkrQmFRam81bHdyQ1V4d2pEWmxYWWRZYjZp?=
 =?utf-8?B?TjBuYllCZUh5NzYzUzQ5SElDekhsbnRCOHZDTDh6SS9LeFRsZXN0ZjU5d2U4?=
 =?utf-8?B?WmFua1J2clFCSXNqa1kyNUNXMTNZZG5CQkxnU3JWcjFEQ0dzSjdzM3k4dEFx?=
 =?utf-8?B?a2I1YjNXdmw4dHI2MHZhMGxDSHRVSFpVVjBIcGVLbkY0cU9Tc1hYN1g3aXVr?=
 =?utf-8?B?RnJBa09VYlhRRS9VSk96TmdpbHBxdXlMMmp4dVhRaDFOVDBlYTUzVnd1ZzhD?=
 =?utf-8?B?VGNxRGZZMDVJbHhBSFdCVWJDSmhzWnR1VVh5UUNBL2VRS2Z0UlA5T3pUd2c3?=
 =?utf-8?B?dWQyZHZHeUl5U0FkOE42cEZZNTZiZ3BVRGZVUHBYRUd4UGI5SUFvbEtQb21P?=
 =?utf-8?B?R2hJeEJaRkZOSmdkQ0VSeFZGWXZoUDdzODRudE5SaTJxdDJ5WkVJWTNOeXlN?=
 =?utf-8?B?VktBOVpXZytEd0FpYXcrZXRzMHFkWityQjZGUmN3bkR6OCtETitsSGFhNlla?=
 =?utf-8?B?TzdlSVZhSnF3aEtpY0pKbHptL2hZQXhra1h1VUNCMXJIZEN5OHBFK1hYcFdV?=
 =?utf-8?B?enVOQkdvNjlJbm1sa2dveGtqM0VkRGE3bEwvOU1zcnF2NTV4SEZ6WlZLS3Z2?=
 =?utf-8?B?YkREY0h4Q1ZLTU1vT21iMlBTd3M2eXpwVGFkUDVUNXlncnhKWER1dnFWbTVE?=
 =?utf-8?B?WnZ3THNqWkRvVHJEQWNTaTN2MzVhZnlHaXFYaU9nRHdaREU0OU15S0M5RmFZ?=
 =?utf-8?Q?e6OUxkLUrWfQ2JGBF9npvyUwWbNEM45vLNIo9Va?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40504fd2-599a-429c-ac23-08d97f3f3b28
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:39:40.4712
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fFt/n/+njX4tJ/S7hTOwsmMi93knT/Re6zOF6CvQHe2pR0+1vHOox4ndm4+LM9JRYWjwy6Gaashj00JtrZ9vcQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

For a long time we've been rather inefficient with IOMMU page table
management when not sharing page tables, i.e. in particular for PV (and
further specifically also for PV Dom0) and AMD (where nowadays we never
share page tables). While up to about 2.5 years ago AMD code had logic
to un-shatter page mappings, that logic was ripped out for being buggy
(XSA-275 plus follow-on).

This series enables use of large pages in AMD and Intel (VT-d) code;
Arm is presently not in need of any enabling as pagetables are always
shared there. It also augments PV Dom0 creation with suitable explicit
IOMMU mapping calls to facilitate use of large pages there without
getting into the business of un-shattering page mappings just yet.
Depending on the amount of memory handed to Dom0 this improves booting
time (latency until Dom0 actually starts) quite a bit; subsequent
shattering of some of the large pages may of course consume some of the
saved time.

Known fallout has been spelled out here:
https://lists.xen.org/archives/html/xen-devel/2021-08/msg00781.html

I'm inclined to say "of course" there are also a few seemingly unrelated
changes included here, which I just came to consider necessary or at
least desirable (in part for having been in need of adjustment for a
long time) along the way. Some of these changes are likely independent
of the bulk of the work here, and hence may be fine to go in ahead of
earlier patches.

While, as said above, un-shattering of mappings isn't an immediate goal,
teh last few patches now at least arrange for freeing page tables which
have ended up all empty. This also introduces the underlying support to
then un-shatter large pages (potentially re-usable elsewhere as well),
but that's not part of this v2 of the series.

01: AMD/IOMMU: have callers specify the target level for page table walks
02: VT-d: have callers specify the target level for page table walks
03: IOMMU: have vendor code announce supported page sizes
04: IOMMU: add order parameter to ->{,un}map_page() hooks
05: IOMMU: have iommu_{,un}map() split requests into largest possible chunks
06: IOMMU/x86: restrict IO-APIC mappings for PV Dom0
07: IOMMU/x86: perform PV Dom0 mappings in batches
08: IOMMU/x86: support freeing of pagetables
09: AMD/IOMMU: drop stray TLB flush
10: AMD/IOMMU: walk trees upon page fault
11: AMD/IOMMU: return old PTE from {set,clear}_iommu_pte_present()
12: AMD/IOMMU: allow use of superpage mappings
13: VT-d: allow use of superpage mappings
14: IOMMU: fold flush-all hook into "flush one"
15: IOMMU/x86: prefill newly allocate page tables
16: x86: introduce helper for recording degree of contiguity in page tables
17: AMD/IOMMU: free all-empty page tables
18: VT-d: free all-empty page tables

While not directly related (except that making this mode work properly
here was a fair part of the overall work), at this occasion I'd also
like to renew my proposal to make "iommu=dom0-strict" the default going
forward. It already is not only the default, but the only possible mode
for PVH Dom0.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:41:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:41:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195075.347565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThhq-00042g-Tn; Fri, 24 Sep 2021 09:41:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195075.347565; Fri, 24 Sep 2021 09:41:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThhq-00042Z-QS; Fri, 24 Sep 2021 09:41:22 +0000
Received: by outflank-mailman (input) for mailman id 195075;
 Fri, 24 Sep 2021 09:41:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThhp-00042O-8a
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:41:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c803563d-5602-4e49-962c-0fa09b52f13a;
 Fri, 24 Sep 2021 09:41:20 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2174.outbound.protection.outlook.com [104.47.17.174])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-9-CHzz_hOYPzqzag2p5z2rRQ-1; Fri, 24 Sep 2021 11:41:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Fri, 24 Sep
 2021 09:41:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:41:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0013.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:41:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c803563d-5602-4e49-962c-0fa09b52f13a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476479;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=j5i+Xnl5ocLbmeDxy8GlyB1nTmg2e1vt95em7FKfvfs=;
	b=R6JxTTjjQXJLzTFMFbpM9m7TIJ0oi/w+xnYZmRbHvoyKGYpfeBZbcrAyyhluj5I7DOkTzF
	nVEoGD4TNSnL6C4dpeox0CabXJ0a3gaiSlX1IEFfO4g7FZVb19nO8LAGzqCWix6tVerHAc
	4yTYjG+x8g99MOKXliTFaerZk3u41h8=
X-MC-Unique: CHzz_hOYPzqzag2p5z2rRQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WfmE2NI6yWZPw4gn7WuXfXNyIJc/OrADxL2ggFPVYzrZdGyyS6REf6jaUM94VfEE+id6jl10QrBppg9ZJL+voGdBLHd789motWYqjKSOERu2jw70Fj3lzWvDapsx0rTGH+ENC5fT2XnlnpkDdEjiQLtImlTV3rZhO8eLUV5dfP6cxlt+Xl8pD7XZx4KOagDfkfXNASUB+WsSSx7jjkE1WSnNuHdLNzDN/W7mKXALn6SQ/khG9R5UzrZZyns5gaeXDt8eEcCBKHqS4Ly38AvSr7e4gmXV6znZbF/uUXyIuZ/GkxRM/ziVHeIQef0fnz1OMC5tDBv2Fb0Qeak/Y3YzFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=j5i+Xnl5ocLbmeDxy8GlyB1nTmg2e1vt95em7FKfvfs=;
 b=C7is/+7mb5nIeHUkqOtnIxlBqns7FefGH2HzpEUkB5ZN+9JkCdkjMstXwrU55Jfe/b7xXhBFNrFrdGJ0zbxU787YSxE9+0LRG/8V94cr9ib7aFaXoEsd5tCf41GgPRQaQ6ilcvFhgJ0fmeyAbBuTGb6E12GVgohGgPVI/uW0gZvoPXunKgrTrT+2nSGzs+xipHEt7dXKxaGjxhowQwsLxh4CAeXP4w+SPki+d6X1uLUIBobzp0a6CLsElCPD0FIEJU9EjqgNYa9/bR3elHaGDyjJgO/2dw2Ozx/U3k5NP55Hbd3KNzcaISrKfKQBZVDlk3p28ZWVkC9sS8zF4gwrzA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 01/18] AMD/IOMMU: have callers specify the target level for
 page table walks
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <57abd488-1fa1-8eae-66b9-196f32f7f4be@suse.com>
Date: Fri, 24 Sep 2021 11:41:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0013.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 41438e88-62c3-4c5c-ccd3-08d97f3f7462
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2335CD96E25FC771F080EC56B3A49@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	robe0bU+hsEqZ/FUcX7R7k6vN7TMlO7Wvb0Qx48qQ3ISg+a9ctXgzC16VTFBkWr2KjCA8TsslEDEkY5In/v8K3naVxu2U+6a85mh4DTKJbPhTcjM99xNBT8mgoH63+RLLrsNf3HvPj6MhZIyyDPuSQr7bvN4CFxO+SNkXhGoPDvueyBp/FV+3qvXWHrsu/Rihc/AB1lPQFqmlAsrxWWYGnEqvpAW09Gw/bo82WuuAdxvHKJrIETq5nTJ5glWZ1fRNEGcZRr0e4QSZUePvKXarBOEusE5cw8hpJbWcug955ooWoVhDbpaFnm0lwxzpPmlTHr6x6YgkUd9BLFAFf/JGfKJybYpnowbWErH1ZHkgPfxeKDWIAxVZ7+FqDgLkR0fEYacWYfdodMOLdosj1IDRblWUZS5Ju5XmwD+XwrRxI32xqrkUB5VIr4B7Q4O4/yUkhDY+7pqUvVi3Ot8+OUGx1nSRPsFdGhDMaf0gsXL6b2kyYXYamqEyjCOiy0Y1zavyOzQ69QcOAc6bbMAHjrbilQvevyaPoLeqW9ic5OxiG2wj8RYXyPYlTeN6kAmiYge/G8afHqMN1IswIcl/Y+8LJ5NE2nkwtEMm5MaLCI3sn8pvm6SIAmuCWwPEt2Bo7VL918HA8pn4pvhhfmwf0Z6FXGiKkmlUQRBNTG8AX0KMLkGjtEj7GWXWqhjBOs5HMiZkXsNdJ93qxg5thgRQWwfEgs6L0TgumGahSW434JR1UuSSxhnublBJVVXHbigGgdn
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(16576012)(54906003)(66946007)(66556008)(2906002)(66476007)(36756003)(186003)(83380400001)(316002)(5660300002)(8676002)(26005)(38100700002)(86362001)(2616005)(6916009)(956004)(31696002)(31686004)(4326008)(8936002)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?elByVUdZR0hjZ1RGb3ZBQmlMSThGQ2hzaGdNYzhNdDF2bUtqNVNIandlSk5m?=
 =?utf-8?B?WTJjSFlhRTJGWUgzN0V6MzdsMzFka21MSTlrYXkvY3BhM2RYZ2llSHcyME1z?=
 =?utf-8?B?cDlNdS85V20za3Q4bXE2YXVKVm9DaUUzejB4bEx3UmZMSFpKU2FCa0VCSUhj?=
 =?utf-8?B?SWpuSmk5UFhwYmtBRDBkQ1hBSEhFVnNZZHZhOVlySGJqZTFMeExWSkU2NWFx?=
 =?utf-8?B?YUxWQzhMUnZYN2NmSUFrQlNodm4ycWRnWmN3eXV6ZEZtY0loQkhKTnQwUnpP?=
 =?utf-8?B?WDZrZnViMDJzWUlBTFVsdzBVeDNQRjZDc05ocEdwY0dsZFFlUEE3M0hza2ll?=
 =?utf-8?B?U3VUbWpoM3RKSkdxZStIRkl4R3pXRE14b25nOHBScGQzclVtM3I4UXlYbEVV?=
 =?utf-8?B?dmhxWGlaaEtIQlhTeVJZUHVuamJJNzRRM0hJVlRZYmpHRUpBVXR4TVBVVzZQ?=
 =?utf-8?B?aVZKRzJSRDZrc2xmV2piY041Z1NwcG5sOTVzV3lSUTY1VWtBRmVESnJ6YXVE?=
 =?utf-8?B?ek1WZ2VvZ0ttL3d2ekdPRW1KZDZITU5wd2VHbHNGWktiaWo1cXBZVWR0Tkln?=
 =?utf-8?B?eDNpTEVzc3o4UkUwZ2c5SXdMNXJFenBFdm40Nk1tbkVEUU1YM1VyMmNlZ2Jn?=
 =?utf-8?B?NzhVWUd6OStKOFppbHBkUjkxdTNSd0ZWMStzd1RvdDBzUm9Pa29qdE5uZW91?=
 =?utf-8?B?RWRuVXJVYUJoKzBXMHdOT2VsZGU2YVhGZnZ6ZWNNL0RVL1lrRTZFdGVubFZq?=
 =?utf-8?B?ODJRWEZEVlZxZExzS2R2WFJ5MWFDaXpmUXNrS3lWRHpTVzZMUS83R3d5YXE2?=
 =?utf-8?B?UG9JT1RpcEllVW9KaDVRbXVPUG80U0ZzM21qZ05UTkRmaXBjM0p0V3ZuSGh4?=
 =?utf-8?B?cWxCYlRMMDhteHNJMHlrRzVUMG50NFIrSC84UkY4aWphemJuUCsrRERUMU8v?=
 =?utf-8?B?OXpLMnFyTWxjc0pFY2lqVSszcU5wdW9wbkxjQis4R1FOK0R1enlqb0R2R1A2?=
 =?utf-8?B?WllEYmkxdGpkM01IQVVzU1lyMnlUazVnTEhkdmpmUkFoU0ZHVlhSTGlrcDZy?=
 =?utf-8?B?b3ZVRUZLbVgxV3BpVlk3QldHMGFnaE9nditmZ3NnMWsvblN1dnhLODRlVnRl?=
 =?utf-8?B?R21lL090R3JPSXlMN1E2Q1RZVFhrTnllUUY4SGJqZk1QWGF4eFlkRmxXeGZs?=
 =?utf-8?B?UFgxNWJFRHIzUFFhcldNNEFKQjRZTXdvT2xzRkt3NGExSjA0TmkzNHpuQnpJ?=
 =?utf-8?B?dFJOaC9iWHNIdzBYckV6ay84VWt5SDhNVzRsc2RBV1pjcWdwY21EU1RaZDAy?=
 =?utf-8?B?R0NGUmxCUzBsbEpMa2RkV0NFN21Yd01QR3hucWowVnhSVnMwTitXT05LYmlv?=
 =?utf-8?B?M3k3NjhvRUhtTXpRQk5xTi85YWl1R294Nkh5M1dzS2ppUWd6MnlvY1cwU3F4?=
 =?utf-8?B?MTNwNWNVWDdzSnFzaEZQb2Q0WHlFUjVOSXZLVC82OTh0RVpBWnQ4cFN3dUNM?=
 =?utf-8?B?Y1hZbU1ZcTNwbHlaSGFJM1V1c0NST2tMeVE4QThiVnpIZlFEeDJNNFZyempy?=
 =?utf-8?B?b29GWnJ1VFZTUC9lOUpPNHkrZEQzTFNFNzZhN2w1QlNsaGN5M0RFdk40TXB1?=
 =?utf-8?B?L2szaVJXem1ZdDBNbXZuVTNhcWdaSTJ4a3VIUUtnUldzaFYwOHhqTmlxMHpo?=
 =?utf-8?B?U2NFY2pGNTRaSXpVNFl2dlRtZmtoa0lJeEJvNU02UzJwRFFWYkd0MEVZdGRV?=
 =?utf-8?Q?PtVfsuzxuWJMcKs4l4AaTlDSbuCwwPdCrppRPJr?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 41438e88-62c3-4c5c-ccd3-08d97f3f7462
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:41:16.3074
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Qtq42MmFRRpN/+37XxX8+jZ/zvdGnNyleRQg3uglA8dPecz6GPbZCd162iIXLAO6uuGvNfjUnOvqW1eulixIxQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

In order to be able to insert/remove super-pages we need to allow
callers of the walking function to specify at which point to stop the
walk. (For now at least gcc will instantiate just a variant of the
function with the parameter eliminated, so effectively no change to
generated code as far as the parameter addition goes.)

Instead of merely adjusting a BUG_ON() condition, convert it into an
error return - there's no reason to crash the entire host in that case.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -178,7 +178,8 @@ void __init iommu_dte_add_device_entry(s
  * page tables.
  */
 static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
-                              unsigned long *pt_mfn, bool map)
+                              unsigned int target, unsigned long *pt_mfn,
+                              bool map)
 {
     union amd_iommu_pte *pde, *next_table_vaddr;
     unsigned long  next_table_mfn;
@@ -189,7 +190,8 @@ static int iommu_pde_from_dfn(struct dom
     table = hd->arch.amd.root_table;
     level = hd->arch.amd.paging_mode;
 
-    BUG_ON( table == NULL || level < 1 || level > 6 );
+    if ( !table || target < 1 || level < target || level > 6 )
+        return 1;
 
     /*
      * A frame number past what the current page tables can represent can't
@@ -200,7 +202,7 @@ static int iommu_pde_from_dfn(struct dom
 
     next_table_mfn = mfn_x(page_to_mfn(table));
 
-    while ( level > 1 )
+    while ( level > target )
     {
         unsigned int next_level = level - 1;
 
@@ -307,7 +309,7 @@ int amd_iommu_map_page(struct domain *d,
         return rc;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, true) || !pt_mfn )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, true) || !pt_mfn )
     {
         spin_unlock(&hd->arch.mapping_lock);
         AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
@@ -340,7 +342,7 @@ int amd_iommu_unmap_page(struct domain *
         return 0;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), &pt_mfn, false) )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, false) )
     {
         spin_unlock(&hd->arch.mapping_lock);
         AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:42:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:42:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195080.347576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThio-0004cq-8W; Fri, 24 Sep 2021 09:42:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195080.347576; Fri, 24 Sep 2021 09:42:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThio-0004cj-4Z; Fri, 24 Sep 2021 09:42:22 +0000
Received: by outflank-mailman (input) for mailman id 195080;
 Fri, 24 Sep 2021 09:42:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThin-0004cX-27
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:42:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b7ab1362-8d84-4907-b3e2-5df732f872ae;
 Fri, 24 Sep 2021 09:42:19 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-17-adVlg-waNZuWIr99FQbj3w-1; Fri, 24 Sep 2021 11:42:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6303.eurprd04.prod.outlook.com (2603:10a6:803:f2::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 09:42:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:42:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0016.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 09:42:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7ab1362-8d84-4907-b3e2-5df732f872ae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476538;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ou8PXKvNNIJKUClYCy2gG2WGmmb4DyDLE4jV2AzEnuQ=;
	b=OrZxJseX0+E4r229LqhBdLkqTa3MB+W074+N3I2gf/P357toeI0PKnZ3bn+E0cAus+1iZ9
	Z4k+8J4PM211AxlcC7RLQ086tDML4Lq0yEMgcZ8rbxnIoljr+JXeGuI+NGjb9eKCOoM8bg
	XUBeBS2ULywibK8LNzNsuPCOZsrqDss=
X-MC-Unique: adVlg-waNZuWIr99FQbj3w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=djRsVeta72DPaEfTt00mxBkIIPCRZKNBpOmJGd3nrxXW7qspNeNTomO3yqDIIiyZ/bFxvownlPdFz2QQ4EhLbEhAHT37qxw9rvf8yidg+dCMaFwEZIMbGak+3vyl8KryTt7JNXwzdkl9Kp4tC/ChQmAZTwk1SmxVFEwFP6HN3TDq6J2sIft6foFyoOIj4O26ek63Pd6urpQtDXwVXwZtHx2Oz8aUlhHLDQno//6Z8RuKAMaGomFiIWvUc2K2k93k3AD1N6Bnbg+laN/XaNMRNnglMmHPrdXiFkY7dMOk2C4uVgCKtO5rRSKiqJiIGO9295Y4rgbBLkF2ChCFmPELCQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ou8PXKvNNIJKUClYCy2gG2WGmmb4DyDLE4jV2AzEnuQ=;
 b=TEixJleToN/Daz6GQY3BHpTEYsAq5oJgtOLbFNWigxsBob9IIUSZDbAxFQD2kxANWKdkFmUBFJ5qVHLu8WJCN4Mg7nAHYQTGvHQVkU25D7Xo6tDk4QxRdZLK7aUqZZF4Hrqy0CksDu65Rv/tc83E5hFmBSZlSUpMTn7pMDjNtOAPZl+EVFyToL4xzvVc0txwicH5To7WeBNnJ6thyqJOENdBei1WWkbNg9bgVa9iBfCmD3CxB7RT9VQ53LG1UMOJPvbXNhe3LjGpTOwoUDXvywmbBLVhiruLP9lQZal3QrqJt2BrZgS2JIOxaSKNnuUN+l4+dZNxQO7MSzo2fW3BFA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 02/18] VT-d: have callers specify the target level for page
 table walks
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <6d01a401-ac53-d4ca-e0ac-165d045341af@suse.com>
Date: Fri, 24 Sep 2021 11:42:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0016.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ace5a24a-179e-4422-3ebd-08d97f3f9768
X-MS-TrafficTypeDiagnostic: VI1PR04MB6303:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB63036BF1FC722D86C8D91CB4B3A49@VI1PR04MB6303.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TIb5mSjHuRzyJrl/cdytFCcSl24glRtpJZQ4mMvpZ9sESNpL18GGp8IYR9Mg4CyUKNRsEtoB0wnTqRgsogwhNeAJn3QYewltUegMhGYYEfYpr9rFLUdO7qnRGDLBuCw9sYVbenRnd7La0KHqg6GfVP5sHK+bjNPpnFI4eAvwVPNUfyAGUYcSohj5x3vjqwN3DwzRAlYkTgdZQodugWepiAzHdPuGnEVU6mHQZylPhawmD4KyvBtE+aEQdscp2Svu7kH6N5tuWSdv6evFSSr9IpLdqLXHfaDG9GfP4QNilrUSpirdmOtIvISUG4eeHa6Pr72vzqu6aiabdtqAF3nfhxQpri4S/rEJh4OonGRNVRzW3twELDkhManJ+ZLobr6IVjXIr6bwI+CUzd2ZD07NWrHyiIdI6gM7kh1mEC08z7flugsjV5qtBPGZZGbYZZ1hgdzUAWQkOG+PvxotPg2mOWaiQWaudEaNdCdrEl5Fqe0gF+JavE6dbAc97ujPvtzbz+dUNZVE2hSXLTbVya70a7RceHbd34KwQvXcSCfVzZCdet0eEGAGHLpMd/Rwv9aKuGjbCOAkFHLkvnM+/cBJqO2RpHj6RqtshnFz8hHVNYN/Nd4gG9Jqqr7VzZUw2U8hH3RX00g6LnlYe5CDNI7lNP67UULKLf+LbbF0J2Y6/7/jbPrg88OuNzfYuuxekBpyIj5oDDAqvFvwPDI7biux9hFkxGLLXUZtkNs3neFWRf8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(2616005)(38100700002)(956004)(86362001)(83380400001)(508600001)(66476007)(66556008)(31696002)(31686004)(36756003)(8676002)(16576012)(66946007)(5660300002)(54906003)(8936002)(2906002)(4326008)(186003)(316002)(6486002)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MUZrQ1ZvZndZdWNvRlg3U01qMW4rQzZudEdVV05wRU5hWFU4UzltdGhoUi9Q?=
 =?utf-8?B?Um9PYTJxUmM5TzRFaFF5SmJsZ0hiOW1xb0cvdjFOSWYyTzd5T1VyVGpwaXB4?=
 =?utf-8?B?UmpNZFFUdUI1cGUzakc3Zk81R2drQUU0MXIvY0JmMmtVVTRhRFZ3c2x0R1BP?=
 =?utf-8?B?dmtXWmp0eGZCNU5aeTRiMDFGNWYzTFF5SU04MmFwZjdodlA2NEVlSkc1Rkti?=
 =?utf-8?B?NnUwRy8wRTNGSHF2Qm1BUlFQT0xMRmRkUUhxSit2MnAxVjAwZEh1R29jdDZi?=
 =?utf-8?B?NzZPbnlLWHVyTm5FcHJSK3R6bzMxeTBsczAvZTZLNkdZZTRnQTJEd2xUVllr?=
 =?utf-8?B?TkVZMVZIMlpLWXUzbXgzbURZeHN5VFRET082QmZFNUJoRENIQi8rT2FMZXFh?=
 =?utf-8?B?R3lBK2hGU1BYVXVkYytFM01XUS81eTR2cDJqNk5OTmh5c1lLRmZ3bFo1dm9C?=
 =?utf-8?B?R1FKUDdibmpMNnVLbUpCS3NuMVAxQ3JBYkhDNzNrVThBMXBQMkFVM2VzUnJz?=
 =?utf-8?B?dG9UMTM0eC9PQWhocWRuemNhK1l2aVRjWUF2dHdsRVpxT2hqb29HbnlKQ0Va?=
 =?utf-8?B?MjFJZnJoYkF6M3FzRmJhdU1wOGVyaHRIWW02ZEg2WmN5REZVMlJqSFhFYmx6?=
 =?utf-8?B?Znk4V0FFOHlRVUpHeHlWamVRWUl5eU5DbFN5VjJJUXZrT0dLelBFeEdrcFpz?=
 =?utf-8?B?TEJZU2tqUkU3aGpuSnBjR1cwQkVha3RITFVxNVRzZkU1S1F5TWhrWGdLMnd6?=
 =?utf-8?B?UDdPLy9yMEtjMzRLbXpEREhiaGVHMUt3SjRjWkRMQ2RIQ21zVStmSkJnVXg5?=
 =?utf-8?B?RFpVdEdGUHFPUjFRaTM4MDlIQzIvNG8rUnI5OGVvUytFangzYVh5ckVKRmQv?=
 =?utf-8?B?Z2RFeDdnVmw1ODBYZlJrY1Y2ZEMxaHNTS3BSY1luQXNENFRQWjkzT0w4SU11?=
 =?utf-8?B?TGJzdi8wL0VCdkVKVWZuZ1piTGQ4eFFRV1ZCVjV6UmhFai9jTWgrRk04bWg2?=
 =?utf-8?B?ZDFFRGNmQVdrMUw5MXR4ejVQa0F3Znk4aFFoV3dlRjJkZEdOTUhaTHo4dWU2?=
 =?utf-8?B?cEZ6aFVhK0UxVEI0bks2ZkJzZVVoVDVSQTVkNjljTzYzSVNZV0x4NkVlazR3?=
 =?utf-8?B?M3pCc1BtVm52NDJka0xDakF6M0sxYzRadjVTaFdpbi8zaVRwODdFaUN2b1dT?=
 =?utf-8?B?MFlpSEQxeTkwSUxsaVp0Rk5MaUxNay93eVM5VXM5b2krdnhqd3B0U0Q4TUNT?=
 =?utf-8?B?TDVRcnVEWGFROHR2WHFYenRwUFVVOFBYdjViUmlQSGhHSFVCeTJIUmtqWUUr?=
 =?utf-8?B?SXkyZzFRKy9vRlo4c1d4M0JKaGU2ZzB2bjJwRXBaYnlhQWJ0QjV3T1diM3Ux?=
 =?utf-8?B?bFFIRm9CZm4wMVdSWkFOaXNhQ1BBWlEvZW0wY25IRlVrbHRZRWJLbXhCb29O?=
 =?utf-8?B?VGVpSzFXZkVZMFZrRkhCN21tN1NWU0RkMFFuN2RGOXFCVEtFY0hzM2JuMHJ0?=
 =?utf-8?B?R3RTT2p2VGtWRGJ6aDQyVm0zdkNhb2lVSVE0b0pkcUZVNWNNN3JBOTNNT3Zh?=
 =?utf-8?B?WXh2aW1EcHR5OEludE1ZRmgwOFQ2ams2bHozeVJkMjA1UWZaNDV2b29hdkt2?=
 =?utf-8?B?SnFiYmltYVA3T2lRQWxyRzdHOVlMNjB3V3hxQUZkRjM1Rm5aQndEWUVzMGsw?=
 =?utf-8?B?YkIzVFhjWDd5Y25RTlFGUndXS05qQ1V3YTI2WUxNcnZSYlM0a3IraklTWlZN?=
 =?utf-8?Q?yEFOkRG1+PE2I2okVLSowuyYgAjzpStenFVx1Dn?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ace5a24a-179e-4422-3ebd-08d97f3f9768
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:42:15.0903
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GtPHs46Daeci0R1qTwIx05CMvFrxMvdaSePclm2v/EZ/zWNRNBLbSbCQzSKuAruDrqieDQFQr3XosvOiIBr3bg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6303

In order to be able to insert/remove super-pages we need to allow
callers of the walking function to specify at which point to stop the
walk.

For intel_iommu_lookup_page() integrate the last level access into
the main walking function.

dma_pte_clear_one() gets only partly adjusted for now: Error handling
and order parameter get put in place, but the order parameter remains
ignored (just like intel_iommu_map_page()'s order part of the flags).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I have to admit that I don't understand why domain_pgd_maddr() wants to
populate all page table levels for DFN 0.

I was actually wondering whether it wouldn't make sense to integrate
dma_pte_clear_one() into its only caller intel_iommu_unmap_page(), for
better symmetry with intel_iommu_map_page().
---
v2: Fix build.

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -264,63 +264,116 @@ static u64 bus_to_context_maddr(struct v
     return maddr;
 }
 
-static u64 addr_to_dma_page_maddr(struct domain *domain, u64 addr, int alloc)
+/*
+ * This function walks (and if requested allocates) page tables to the
+ * designated target level. It returns
+ * - 0 when a non-present entry was encountered and no allocation was
+ *   requested,
+ * - a small positive value (the level, i.e. below PAGE_SIZE) upon allocation
+ *   failure,
+ * - for target > 0 the address of the page table holding the leaf PTE for
+ *   the requested address,
+ * - for target == 0 the full PTE.
+ */
+static uint64_t addr_to_dma_page_maddr(struct domain *domain, daddr_t addr,
+                                       unsigned int target,
+                                       unsigned int *flush_flags, bool alloc)
 {
     struct domain_iommu *hd = dom_iommu(domain);
     int addr_width = agaw_to_width(hd->arch.vtd.agaw);
     struct dma_pte *parent, *pte = NULL;
-    int level = agaw_to_level(hd->arch.vtd.agaw);
-    int offset;
+    unsigned int level = agaw_to_level(hd->arch.vtd.agaw), offset;
     u64 pte_maddr = 0;
 
     addr &= (((u64)1) << addr_width) - 1;
     ASSERT(spin_is_locked(&hd->arch.mapping_lock));
+    ASSERT(target || !alloc);
+
     if ( !hd->arch.vtd.pgd_maddr )
     {
         struct page_info *pg;
 
-        if ( !alloc || !(pg = iommu_alloc_pgtable(domain)) )
+        if ( !alloc )
+            goto out;
+
+        pte_maddr = level;
+        if ( !(pg = iommu_alloc_pgtable(domain)) )
             goto out;
 
         hd->arch.vtd.pgd_maddr = page_to_maddr(pg);
     }
 
-    parent = (struct dma_pte *)map_vtd_domain_page(hd->arch.vtd.pgd_maddr);
-    while ( level > 1 )
+    pte_maddr = hd->arch.vtd.pgd_maddr;
+    parent = map_vtd_domain_page(pte_maddr);
+    while ( level > target )
     {
         offset = address_level_offset(addr, level);
         pte = &parent[offset];
 
         pte_maddr = dma_pte_addr(*pte);
-        if ( !pte_maddr )
+        if ( !dma_pte_present(*pte) || (level > 1 && dma_pte_superpage(*pte)) )
         {
             struct page_info *pg;
+            /*
+             * Higher level tables always set r/w, last level page table
+             * controls read/write.
+             */
+            struct dma_pte new_pte = { DMA_PTE_PROT };
 
             if ( !alloc )
-                break;
+            {
+                pte_maddr = 0;
+                if ( !dma_pte_present(*pte) )
+                    break;
+
+                /*
+                 * When the leaf entry was requested, pass back the full PTE,
+                 * with the address adjusted to account for the residual of
+                 * the walk.
+                 */
+                pte_maddr = pte->val +
+                    (addr & ((1UL << level_to_offset_bits(level)) - 1) &
+                     PAGE_MASK);
+                if ( !target )
+                    break;
+            }
 
+            pte_maddr = level - 1;
             pg = iommu_alloc_pgtable(domain);
             if ( !pg )
                 break;
 
             pte_maddr = page_to_maddr(pg);
-            dma_set_pte_addr(*pte, pte_maddr);
+            dma_set_pte_addr(new_pte, pte_maddr);
 
-            /*
-             * high level table always sets r/w, last level
-             * page table control read/write
-             */
-            dma_set_pte_readable(*pte);
-            dma_set_pte_writable(*pte);
+            if ( dma_pte_present(*pte) )
+            {
+                struct dma_pte *split = map_vtd_domain_page(pte_maddr);
+                unsigned long inc = 1UL << level_to_offset_bits(level - 1);
+
+                split[0].val = pte->val;
+                if ( inc == PAGE_SIZE )
+                    split[0].val &= ~DMA_PTE_SP;
+
+                for ( offset = 1; offset < PTE_NUM; ++offset )
+                    split[offset].val = split[offset - 1].val + inc;
+
+                iommu_sync_cache(split, PAGE_SIZE);
+                unmap_vtd_domain_page(split);
+
+                if ( flush_flags )
+                    *flush_flags |= IOMMU_FLUSHF_modified;
+            }
+
+            write_atomic(&pte->val, new_pte.val);
             iommu_sync_cache(pte, sizeof(struct dma_pte));
         }
 
-        if ( level == 2 )
+        if ( --level == target )
             break;
 
         unmap_vtd_domain_page(parent);
         parent = map_vtd_domain_page(pte_maddr);
-        level--;
     }
 
     unmap_vtd_domain_page(parent);
@@ -346,7 +399,7 @@ static uint64_t domain_pgd_maddr(struct
     if ( !hd->arch.vtd.pgd_maddr )
     {
         /* Ensure we have pagetables allocated down to leaf PTE. */
-        addr_to_dma_page_maddr(d, 0, 1);
+        addr_to_dma_page_maddr(d, 0, 1, NULL, true);
 
         if ( !hd->arch.vtd.pgd_maddr )
             return 0;
@@ -691,8 +744,9 @@ static int __must_check iommu_flush_iotl
 }
 
 /* clear one page's page table */
-static void dma_pte_clear_one(struct domain *domain, uint64_t addr,
-                              unsigned int *flush_flags)
+static int dma_pte_clear_one(struct domain *domain, daddr_t addr,
+                             unsigned int order,
+                             unsigned int *flush_flags)
 {
     struct domain_iommu *hd = dom_iommu(domain);
     struct dma_pte *page = NULL, *pte = NULL;
@@ -700,11 +754,11 @@ static void dma_pte_clear_one(struct dom
 
     spin_lock(&hd->arch.mapping_lock);
     /* get last level pte */
-    pg_maddr = addr_to_dma_page_maddr(domain, addr, 0);
-    if ( pg_maddr == 0 )
+    pg_maddr = addr_to_dma_page_maddr(domain, addr, 1, flush_flags, false);
+    if ( pg_maddr < PAGE_SIZE )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        return;
+        return pg_maddr ? -ENOMEM : 0;
     }
 
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
@@ -714,7 +768,7 @@ static void dma_pte_clear_one(struct dom
     {
         spin_unlock(&hd->arch.mapping_lock);
         unmap_vtd_domain_page(page);
-        return;
+        return 0;
     }
 
     dma_clear_pte(*pte);
@@ -724,6 +778,8 @@ static void dma_pte_clear_one(struct dom
     iommu_sync_cache(pte, sizeof(struct dma_pte));
 
     unmap_vtd_domain_page(page);
+
+    return 0;
 }
 
 static int iommu_set_root_entry(struct vtd_iommu *iommu)
@@ -1836,8 +1892,9 @@ static int __must_check intel_iommu_map_
         return 0;
     }
 
-    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1);
-    if ( !pg_maddr )
+    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1, flush_flags,
+                                      true);
+    if ( pg_maddr < PAGE_SIZE )
     {
         spin_unlock(&hd->arch.mapping_lock);
         return -ENOMEM;
@@ -1887,17 +1944,14 @@ static int __must_check intel_iommu_unma
     if ( iommu_hwdom_passthrough && is_hardware_domain(d) )
         return 0;
 
-    dma_pte_clear_one(d, dfn_to_daddr(dfn), flush_flags);
-
-    return 0;
+    return dma_pte_clear_one(d, dfn_to_daddr(dfn), 0, flush_flags);
 }
 
 static int intel_iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
                                    unsigned int *flags)
 {
     struct domain_iommu *hd = dom_iommu(d);
-    struct dma_pte *page, val;
-    u64 pg_maddr;
+    uint64_t val;
 
     /*
      * If VT-d shares EPT page table or if the domain is the hardware
@@ -1909,25 +1963,16 @@ static int intel_iommu_lookup_page(struc
 
     spin_lock(&hd->arch.mapping_lock);
 
-    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 0);
-    if ( !pg_maddr )
-    {
-        spin_unlock(&hd->arch.mapping_lock);
-        return -ENOENT;
-    }
-
-    page = map_vtd_domain_page(pg_maddr);
-    val = page[dfn_x(dfn) & LEVEL_MASK];
+    val = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 0, NULL, false);
 
-    unmap_vtd_domain_page(page);
     spin_unlock(&hd->arch.mapping_lock);
 
-    if ( !dma_pte_present(val) )
+    if ( val < PAGE_SIZE )
         return -ENOENT;
 
-    *mfn = maddr_to_mfn(dma_pte_addr(val));
-    *flags = dma_pte_read(val) ? IOMMUF_readable : 0;
-    *flags |= dma_pte_write(val) ? IOMMUF_writable : 0;
+    *mfn = maddr_to_mfn(val);
+    *flags = val & DMA_PTE_READ ? IOMMUF_readable : 0;
+    *flags |= val & DMA_PTE_WRITE ? IOMMUF_writable : 0;
 
     return 0;
 }



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:44:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:44:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195087.347587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThkT-0005LZ-NX; Fri, 24 Sep 2021 09:44:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195087.347587; Fri, 24 Sep 2021 09:44:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThkT-0005LS-KP; Fri, 24 Sep 2021 09:44:05 +0000
Received: by outflank-mailman (input) for mailman id 195087;
 Fri, 24 Sep 2021 09:44:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThkT-0005LM-Ar
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:44:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f33e3a7c-1d1b-11ec-bab8-12813bfff9fa;
 Fri, 24 Sep 2021 09:44:03 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2054.outbound.protection.outlook.com [104.47.10.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-m4FxfhpNP7yuh-9v9GSWQw-1; Fri, 24 Sep 2021 11:44:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6303.eurprd04.prod.outlook.com (2603:10a6:803:f2::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 09:43:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:43:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0146.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:43:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f33e3a7c-1d1b-11ec-bab8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476642;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Gc5Okgblxwjveskkdg4CB+2OMoPev7pqS00OgVJvY3k=;
	b=bLE8jdOjvwwqVH6x+EgjvCE4YBpcKd+Xg05XRvHxKenEfOzeoUOqWPOhB3DuvJfLDhublL
	jEdJVH7HZeCbqXM9Lhm32CgvuBEyUH4RKZnOpPYYA4PQWJjXkL0Bw6+g5RUvt4ojQmfMFo
	MKRrdbSDVVJ8AfP0dEnFS127L5zH9so=
X-MC-Unique: m4FxfhpNP7yuh-9v9GSWQw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=S8DjcxGPrgKvFECwwES6GBxG2VpJRSo6wDP4/e7f7LW4hg6wcRqlIuBwELAVx8fnurKaPidnBRrZu38GEXymFspr9tXXyQZSV+gmfAxUpLzLwg7CPVF+aRYsUYXujcO3uiVeTQCnKsuERSJpmlCY1nIBaw5UdtRigSreMQLvbfJDwr5/X+CiVszHU1B7sXax6zyq+LamZq/+ie4JLos3mT9ooDofL4CJvR2MlXCUw2GW2UeZB9wDGBHK8pAsTOoGT+9NwjwnqvcPsVyUcsP+xf9nAoNEK2gpZJzmccGnXfi8MCfQ8eJI+pJY7zRWDhNBw/tu+jgi8YyMJzBcvJypLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Gc5Okgblxwjveskkdg4CB+2OMoPev7pqS00OgVJvY3k=;
 b=JQ+H0OCA3zZWEtEGWTNHSM+GslqVpqhDon/zYmR//C1/zkIogay0wzSRTLzGrKQnHyGwC5OkpuRbOCTpyy7ZCvOHJYnOII//JJlViLIJvLHKvTYFmPYDP5pJ5bb0YJayi9Gacbh1HjUa3mAkzJsOpiNOB7XygL7jCTY+8cRFhN0tqYlS8d1Vacz6zvjj4x46TA6Qu+nzCHQtFA8Dj34Vc8dwG/62YQjEWHuuhzeGmAqE/0vHRGQrFTiossj/e9NIQMtZsXpXFI/3RugGNhE4uCrvgHjJFktEJZoiDW96yK/hNog77dnHYTm3z5YtzPOpApRpfjBNaNJyq6Vh9lGRIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 03/18] IOMMU: have vendor code announce supported page
 sizes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <Rahul.Singh@arm.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <3f0adf5b-cc2a-1b72-8376-baad574fa824@suse.com>
Date: Fri, 24 Sep 2021 11:43:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0146.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d662cd64-4a70-4b29-aa39-08d97f3fd56f
X-MS-TrafficTypeDiagnostic: VI1PR04MB6303:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6303D9FCC359D2C78AB78EB9B3A49@VI1PR04MB6303.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pOb51MJ4veuL+KJH99uO1P4N/k2uUAZCTp9+XiUtQ0/+DRC5y8mPin0fFfXc5Foanq1FRXaiN1j8D4UZMCgncw4gzamBSGiizEjfsWHMtIfHsTJl0yvSuY13Kkx3mF/3LVZE1GcebAuNbTmA3VOsPJ4Z9on25t+Eu+G/zxPcQ//78dDSrFPv9EEcwd7jpJ2CK75BLX861saaD9m0xB4L8nBqz+QZw0xEX+dva0Bev6gJ8JVW7KOLGPIyNMYErBRph8zEQrIrfPw2OtN7P9+5GDjGJXvTNOPx6qlHRemBzp3bSpQSZV50g+t0RKBEd84JDQG8kp1LFAcu5RhIjbPz+AoRpbOwnqqStpBq6DXNHS81ofcKyJ1EqtX4DCS2e1+Bw1ClflHq40YvHBKQ+bBKfPcBfV20hmc130A6pHhH0KeX6g7rX6ehnoF4t0Z4Pg/CAP956oyB2LW5Wo9hB8w3dadVoKWll+0Dj7avrx+2HnRfFxHfNtOyIJmZ0n5GC71yNd1weOUQNNq3A9f5suoFtKyX6Sty7hlFLOougpFjIwEyVjGsBKIORcp20J/HPAv/4nhlI0RQA1CJDFiao4zVnIZgUD+LgIT+hnxBfEC+KyKKyal1mraki08E3aThh5U7FDiNTFLivTnPMJyQKBDa2kepjGAAj2rGx2QNDTNDIEmaW5zoPhB6yVnXZiLjrccCDv68pEGOcHLlrUV5SGiHeor6MdeOZGcnMpPDixqoLZ94uq7sl8/ikq+ZC6v74BWZ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(2616005)(38100700002)(956004)(86362001)(83380400001)(508600001)(66476007)(66556008)(31696002)(31686004)(36756003)(8676002)(16576012)(66946007)(5660300002)(54906003)(8936002)(2906002)(4326008)(186003)(316002)(6486002)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QzlFaUhpR25uT2ViblJTRmIrYzFMMHBPUkErQ3ZtRmtUK1h5S2FMeUUrOEdG?=
 =?utf-8?B?THQ1S1A1N3lIMkZZa0Y2RG95QTNQY1U3R21oUTV4MkwwWVNCMDF5Nkc1dWFy?=
 =?utf-8?B?NG9sa2luSzhKMERMaWFteC8zdjJJRmxjNkZXWnJzUXVFVnBscGN4d2tuRTJa?=
 =?utf-8?B?aUVIWkVIZW0wR0p6Nkh2R0JLbzRNbElRV2VwcCtlOW1SMDNiUGkzWnFRbjZp?=
 =?utf-8?B?SnNabDlpaVRoeitLTFZtOFpvRmlxZXkwV1ZiQ0RqZnFSajUyQ2VaSXNDZFpK?=
 =?utf-8?B?eUQ2cHRWYnZOTm9mcDNwOFo1UzlHRWlpL1Y1KzhUNU5lb1o1cHdEbDQ5OFo4?=
 =?utf-8?B?cDBaMiswRTRPTitmSGJwNUpJdGlKUE5pOGlSSEhXNEhUQ1BnMGNDNDhyaG9M?=
 =?utf-8?B?NzllOEpqSGJ4TEpoV2duSkNmcEYzZEg4RjJ4TnVSeWY3UDFJVTVyTlFkdS82?=
 =?utf-8?B?VTNOMXZTU3hEYWh6YXhNY0JBWlY5dkYxR3BINjdQR1lZYW5HakUza05rUXNw?=
 =?utf-8?B?Mys1ZE43UTQ2dUEzQ0krenIxYXVYWFdOcnZyUUxEejhOdVVjTlVJWXBpSytJ?=
 =?utf-8?B?NXlyWnZpZTQrMVlwVU5JNlB6blFmaE03RWlYN2hqTmxMWlZvcDl1OEtiT2lV?=
 =?utf-8?B?RUdlZ1Y2QUtPZFJWSjNHcEVOYTQ3dHMxYmVZeUNNN2M2bzNGenhSbHloSzFm?=
 =?utf-8?B?ajMwQk9CdnYvdm5nVVlpZjBOSHR4aUJydTN3ZW5DcXByUFBFZnJ0VDRWemFZ?=
 =?utf-8?B?QTRjd09YZ1FEQUJZYUxmMTJka3l5anRKRTJTYzRLN0F6TVhoU3BjR3ZZSkwz?=
 =?utf-8?B?Y2hyVlYxby9BZVlQMGs2SFpxeURDQ1Qya0M2WFI3ZXVNM2FURTU1N2pxRGV5?=
 =?utf-8?B?eGxndnBwdDVGYSt4S05ROHY5aFZJQTZ5U1RNUVlRalZSTkRMc2FVYTZDZWlq?=
 =?utf-8?B?YjM1eUVQWGJDMnd1bHFuVFNNd0tyS0EzMG4xdi9JUytxK3AxRjFJcjhQUHds?=
 =?utf-8?B?aXZsLzFUTk5GQi9LMUt3aW55NzRDZlZEMnpILzUwOG1uWUFoWEpDVlZ6TkNY?=
 =?utf-8?B?MWdyZStpbndSKyt6Vjd5N0RkQ1RpYkthYTd4ZFRzWmZMSmRTaDNla0RZTG1F?=
 =?utf-8?B?bExQa1BGZCtqRlI4c1Q3amVORStBZllPVVUvUEZLS2F0MzZPVkhPYU9FS2Zy?=
 =?utf-8?B?UE5DSmZUYXJKMmdhWXFqZkxZWEVDL3ZibWZhWXZhM0dlaHc5VU5QaENjMWRn?=
 =?utf-8?B?RG94dHB0SXBnN2R4Rkxkc0k5THFVMVNvME1LbkFQRnNQK3ljWVlJVlJxQmll?=
 =?utf-8?B?RHkxQXNyV0U0N0RSalArd1FnKzQ0ZU1PbU92QWJ5S3hrTHhRcCszNHUvRGox?=
 =?utf-8?B?dkZycE81cmw2cS9ydG1QbEh0cmNaT1dhODltZ3gvSVZPRmQvZW9xNGFBanZK?=
 =?utf-8?B?Sk55Mk1nOXJZR29TMEMrV1pKU0pEd0c5WlF6Ni9qYkhXNGRZWGVRcnppdzVk?=
 =?utf-8?B?NkU0UTdtTEZ5UCtaRlpoNHNiNDNZaHhBaEVjRDFITFpzMFRqeGZMVFJqbzR0?=
 =?utf-8?B?Y2NtbVNrRVFHWVg1aFVnOUZkUmNYUEFHNHZrbm1scHlveU04N1ROOXVoNHhY?=
 =?utf-8?B?dEZOTEQvVW9WVW5jUFRZbEQzRHl4cDBIR0I4TUJZek5JRExBcnQ2bzUwcjQv?=
 =?utf-8?B?bGtOeXdqYVByMUlOTTZRWUZsUHJyM1FhN1RUTlNCWWhkd3FyR01aL3prNjl0?=
 =?utf-8?Q?PKtfgQvHGNNR0EchOCVbBs8ekJ6ipNQ/u4qByEu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d662cd64-4a70-4b29-aa39-08d97f3fd56f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:43:59.2606
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sPkvg9i3Nsn3OPsUFQtdsdfRK/BjMvxJLgOrmR54+YY2DcGHRrfvS4ZJ2KZJLPcruIyzDU6qgPKvYZygjMgZOA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6303

Generic code will use this information to determine what order values
can legitimately be passed to the ->{,un}map_page() hooks. For now all
ops structures simply get to announce 4k mappings (as base page size),
and there is (and always has been) an assumption that this matches the
CPU's MMU base page size (eventually we will want to permit IOMMUs with
a base page size smaller than the CPU MMU's).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>

--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -629,6 +629,7 @@ static void amd_dump_page_tables(struct
 }
 
 static const struct iommu_ops __initconstrel _iommu_ops = {
+    .page_sizes = PAGE_SIZE_4K,
     .init = amd_iommu_domain_init,
     .hwdom_init = amd_iommu_hwdom_init,
     .quarantine_init = amd_iommu_quarantine_init,
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -1298,6 +1298,7 @@ static void ipmmu_iommu_domain_teardown(
 
 static const struct iommu_ops ipmmu_iommu_ops =
 {
+    .page_sizes      = PAGE_SIZE_4K,
     .init            = ipmmu_iommu_domain_init,
     .hwdom_init      = ipmmu_iommu_hwdom_init,
     .teardown        = ipmmu_iommu_domain_teardown,
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -2873,6 +2873,7 @@ static void arm_smmu_iommu_domain_teardo
 }
 
 static const struct iommu_ops arm_smmu_iommu_ops = {
+    .page_sizes = PAGE_SIZE_4K,
     .init = arm_smmu_iommu_domain_init,
     .hwdom_init = arm_smmu_iommu_hwdom_init,
     .add_device = arm_smmu_dt_add_device_generic,
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -3426,7 +3426,8 @@ static void arm_smmu_iommu_xen_domain_te
 }
 
 static const struct iommu_ops arm_smmu_iommu_ops = {
-	.init		= arm_smmu_iommu_xen_domain_init,
+	.page_sizes		= PAGE_SIZE_4K,
+	.init			= arm_smmu_iommu_xen_domain_init,
 	.hwdom_init		= arm_smmu_iommu_hwdom_init,
 	.teardown		= arm_smmu_iommu_xen_domain_teardown,
 	.iotlb_flush		= arm_smmu_iotlb_flush,
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -470,7 +470,17 @@ int __init iommu_setup(void)
 
     if ( iommu_enable )
     {
+        const struct iommu_ops *ops = NULL;
+
         rc = iommu_hardware_setup();
+        if ( !rc )
+            ops = iommu_get_ops();
+        if ( ops && (ops->page_sizes & -ops->page_sizes) != PAGE_SIZE )
+        {
+            printk(XENLOG_ERR "IOMMU: page size mask %lx unsupported\n",
+                   ops->page_sizes);
+            rc = ops->page_sizes ? -EPERM : -ENODATA;
+        }
         iommu_enabled = (rc == 0);
     }
 
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2806,6 +2806,7 @@ static int __init intel_iommu_quarantine
 }
 
 static struct iommu_ops __initdata vtd_ops = {
+    .page_sizes = PAGE_SIZE_4K,
     .init = intel_iommu_domain_init,
     .hwdom_init = intel_iommu_hwdom_init,
     .quarantine_init = intel_iommu_quarantine_init,
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -231,6 +231,7 @@ struct page_info;
 typedef int iommu_grdm_t(xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt);
 
 struct iommu_ops {
+    unsigned long page_sizes;
     int (*init)(struct domain *d);
     void (*hwdom_init)(struct domain *d);
     int (*quarantine_init)(struct domain *d);



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:45:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:45:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195092.347598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThlK-0005ut-1i; Fri, 24 Sep 2021 09:44:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195092.347598; Fri, 24 Sep 2021 09:44:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThlJ-0005um-UX; Fri, 24 Sep 2021 09:44:57 +0000
Received: by outflank-mailman (input) for mailman id 195092;
 Fri, 24 Sep 2021 09:44:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThlJ-0005ug-8h
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:44:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 12c6fd84-1d1c-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:44:56 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-AJt0MlORMzeDdOz-RyxTTA-1; Fri, 24 Sep 2021 11:44:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5328.eurprd04.prod.outlook.com (2603:10a6:803:59::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Fri, 24 Sep
 2021 09:44:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:44:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0164.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 09:44:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12c6fd84-1d1c-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476695;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ShVz0SZZ0jMQznxGF5xuq3Xcm5rK5MB8JC2kaB72Tbs=;
	b=Pbt1RcKdYzEPmDfqpXKoix1uDc6BKtWOl9En0GiXK9+q3Kr9/Sc+vzCuMuVXCfUd7hZwVE
	Oxns+/qVdm0YR2cJOxx5ftGZmf0oQqTgTFpqJdg3vPRlTJ4f+hdBo9GrqU9NDEJyARTisk
	uh4VM9BaVNraz6AVgz5U6w4sLT0jsRw=
X-MC-Unique: AJt0MlORMzeDdOz-RyxTTA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PiBv4x0FpwScyZIwciISMKrUc0tdO1gTXqrnQUq8rP51yK616rpam4T+00IaLG0tHxVxg4cD/30YnrgtqfG/0+6++Xp8k+XfuVD9bJ2XCv7hF28LaxysCSK5RQzdTP9DUQb03iN2L+/E/ONCtXkEqPGYUBzyYAlgdhC0pHj8YRrbk0KKaNrKCLoRdAsyb/+lO20F5MQ6sRVeFxHHQQ7jP9UW3r5u98a2BoEyOYdWsjrAAqYMol7ZXIbtosuC35+LEwyFdHE/8a6bysu2toazbna82D8lQh5pjLrSqi9YqbUPw77Gv9vN2hZMQe90t3UAD8Dp+pW5lykFbb0COn4JYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ShVz0SZZ0jMQznxGF5xuq3Xcm5rK5MB8JC2kaB72Tbs=;
 b=oL0KxlXr8cwOEU/dIviiDjGR/F1VkUZV5qRvELxd8jlDrtL3+9UJ07VXWfZlaboMToFWx4oSkf/+BT97X+dRwvXgyvbLGA3dy7axPLMMU0UH8NfGFEte1UjTzc+8kCOlaJpgK9QRgQE0rtfh5pLy/GfEWHnxOP08494q1N1+/wIudv8FKs2YPt4V+vRgCoR+Ajq/xG5Oi+/tBneWIsSkKUuAfwHBvCgJcCQ78jAgbHTr3vDpBzcEAnv5gKefn1w2gz3HgSd88n/Oc18IAnkvYMQvMeVRYY9N3us7kPtXZpbxLHr1LJ2nZumzIwiGs+tx7TJc8FLYZWCeHT+btMIkSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 04/18] IOMMU: add order parameter to ->{,un}map_page()
 hooks
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <7500296c-588e-983e-35ac-39048a72cb47@suse.com>
Date: Fri, 24 Sep 2021 11:44:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0164.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4f4f7fa0-0a54-4d93-fc70-08d97f3ff49e
X-MS-TrafficTypeDiagnostic: VI1PR04MB5328:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB53289AF470D13E39BEF6D037B3A49@VI1PR04MB5328.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	r9F3ZsZjYKW9onHGkymSwG4oPK3busWGaFxn6bGdMM/bTMFcoZ1upz7+cmEOrJofVNGbVka4/X/ynzgaYMjYcJWjDU73vwP5my87EBQJoyM/pMHrs8BwbE7Yu9/LC5f1RHCCaeygK0cHXLhxBER471P3rCslbYLbKMadADWU6JjXr9S2SvYlZHGsT6u7TAJhg2Sf945CDI8SIIyX/zTri275y7Qfteb6CSrjqg87ykWqUuiUexHk4JBVX/9+Qas/XpQHAbqL9IHGLraZ0zGk0+oPEyfG1je5nuEVOJe+ByxLoMut1zzW4v7qgzYlr+u3me4Tikg0wwvshv/dhH19hgNECxX866l02pQyoXigjZX1eERweVx3GBzAuRFxXi6eT0vEW0PIfJXPAlgT0M8ajTJv0j/nmj+Kag5wUqhgz1/UJhreaoHlgln/SV5mpAVbjqV5S8clthgtp82+PR1V8qU9PHWwdLf/Vc6OpG8AuOD3sqcX/dY7W96lhZ85iMrsAJkLglKbUJKpPnyOLevA4qTQzOgZDWk575y/sca+haBaHU+IynkOWBJWyOORUWmEYCamB91XV0omauISl39lbZtq6pDKuz2R0swMJGq1VUJXnHJXH+wNiZIQoWj2ca+KmwXlhCdt8Wq2iriaPmdTyDfDJCsJVvrMGGmfC95j0jbC9nEmeecyyastQgIIRcfV7akENl6YbGVhUIocR5PYfIeUZRCu2CHDFAOo8tMHX1s=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(83380400001)(31686004)(31696002)(38100700002)(186003)(16576012)(36756003)(6486002)(316002)(54906003)(2616005)(508600001)(66556008)(2906002)(26005)(6916009)(66946007)(66476007)(8936002)(86362001)(5660300002)(956004)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SkFWQ0VjSENRdzBZRXVYSEE2K05IRUMrVTBicnplVGRBSkhiMDN5Sm9ZUEE3?=
 =?utf-8?B?eC9ic0s1Z3V4T3cyNXlTTCtrem95bkhia08vMkl4Z3JQWHkwN0xKbWFpWGN2?=
 =?utf-8?B?cTJYam5HT2pxNHJMNTByL2dPSkFHaWF1RTRjQ3A2Yk44UjFUWTN2OWkrekhC?=
 =?utf-8?B?VG5sTmVobmVzL1l5NDJPQjBBWTEzTTN6WU13cFVuK3FIQ3RLRCtaNUZGTkdL?=
 =?utf-8?B?WlRONnFaVnJINU81NXQwL0kyb3k3K2d1YUVTSnlxM0NGVjR3TFJzc0pRQWVK?=
 =?utf-8?B?SnhSd0M5aUdXSkVGWGhPRUQ5aGlPNU01L0R3SmJhRlloYzV6TzRNcW8ySyt3?=
 =?utf-8?B?QUNEZFMweXJDb2E0WFRRTEMvMk91a3hpclNhaWUya2JRQStBVm5adkloYjJh?=
 =?utf-8?B?c3AyRmVwd0x2MXkzQSsvdThNK3hNVGlKckhwVWg4VGZ2R2s4aXlIb0swYlRm?=
 =?utf-8?B?ZldJQzlmRlhpRENhUzFGTW9uV3FPQ2M3Snh3VlBRTFVGaTk3MFl0NlNaT2dY?=
 =?utf-8?B?ZUwwZndEQVpzT0tndGNtTytCODN1Tk9EcHFvRHViMythaHZ1bXRJaDkrM016?=
 =?utf-8?B?emJaZ0hxRmNQN2FzOFpxZ1d5S1FRblhJOXR0a3JUQktqQnVrdEtIVjd0RWV3?=
 =?utf-8?B?VnUzOWVYRUxzdmk3TU5kT1grN3FtUGhITXRnVHRJbWdPNEpJSnR2K1dQeEhr?=
 =?utf-8?B?WjdrclVUMWZQLzNseCt3WVJNTVNFemhUcGxMTmRub2Y3Y2d6Z1Nod0xDWUJm?=
 =?utf-8?B?M1NtWGMxaE5nMHIxZE5yaEhVamRBY2EzTVFIQi9mK3o0anlkeHNpT3R6b1NK?=
 =?utf-8?B?WVpNSG9PSnhFbjFWTUhRRjl0blp1cVdjdGliSzdlbTFQYUc4K0xGdm52NUd3?=
 =?utf-8?B?ZXJzbXZvY29pbzhmZUltWFIwVjZid2V6QmdUUGVVVFE3VlhRRnIyKzdTTkds?=
 =?utf-8?B?VXVyNndxcGM2UjhYa2V3cE5mbER2NEk5bmkrOE00Tmp2dDJ3Wk1tM05Wb1R4?=
 =?utf-8?B?Uk9zTW10dW5ESVZsSi9tVEVCeUo1TmtqaWcvWjl0aStrYnJaS0R4MFVZOHkw?=
 =?utf-8?B?aW9NK1NMNUFCNXBWUllSWHpMd0VFK3lxMlJaZEVtdGlmcnBvbTA1MVpNS0JV?=
 =?utf-8?B?dEZVeldpSDZ2dVozR0JGbU5tUS9qOXB0ejc3eEhER0lkUkgzZm1NSEJwWTVC?=
 =?utf-8?B?S3RxSlFhcCtTRkd0YVFmb3gxNjhNaXZxN2NsNWVZdkNZTWd0T3JtV2tRSGNt?=
 =?utf-8?B?SlI2cXpFRGROeHVORGNBQnVTNE1KTTVwMDBUS0J2ZmJrZktWNjQvTVdaTmxU?=
 =?utf-8?B?aEhheUJBWEN1WkdHeW9DZ0Z5NVhRY2RzbmtBeTZrNTEzNVhwZWN0Y3dib0py?=
 =?utf-8?B?SnhteldBVlRTNVJKWlNUQWlRUTlLN0pQNXg3dWdRQ3hkeUcxa2R6WG1ybnFH?=
 =?utf-8?B?ckhKb1FwT2FHS2dJMU5MeGdBcTNkdEtFa1ZvWWVoVjZYcHFFUUtGN2hWTGds?=
 =?utf-8?B?R2NpcWNWSGFBc3FyTjRzOXRFTU4zbVpVUzZxK1hqZHlYc0JMUE96OE1RTWl1?=
 =?utf-8?B?eW5LaGlnQW9CeEtrR3c0c1dMRFhBL1F2SEl1T3QveFp5TmRSVFYvdUJWREY5?=
 =?utf-8?B?dVcxR1dmVm9sNlRGU2ZDRE1oTStSaVV1ZG1XNU1ZUlV2ejQ2cVRxd3gya2Zw?=
 =?utf-8?B?L2dFdWVLYURtb3dSNUR0dkllY3NXbUc3ZHc5c1VUTWdSL3VwOHN4RU1IVndj?=
 =?utf-8?Q?4vgijiCeIiOHQxVMbmEZyHainON0JIIEqrWTCwE?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f4f7fa0-0a54-4d93-fc70-08d97f3ff49e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:44:51.5681
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TeOl3674EEPBpKWegswLU0A0dzD1aZyexiltd+kcw+P5o3y0GBqjS2+PLJgrfBvYBuv6d9WKJdFL3A1nCsAkBQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5328

Or really, in the case of ->map_page(), accommodate it in the existing
"flags" parameter. All call sites will pass 0 for now.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
---
v2: Re-base over change earlier in the series.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -230,6 +230,7 @@ int __must_check amd_iommu_map_page(stru
                                     mfn_t mfn, unsigned int flags,
                                     unsigned int *flush_flags);
 int __must_check amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int order,
                                       unsigned int *flush_flags);
 int __must_check amd_iommu_alloc_root(struct domain *d);
 int amd_iommu_reserve_domain_unity_map(struct domain *domain,
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -328,7 +328,7 @@ int amd_iommu_map_page(struct domain *d,
     return 0;
 }
 
-int amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
+int amd_iommu_unmap_page(struct domain *d, dfn_t dfn, unsigned int order,
                          unsigned int *flush_flags)
 {
     unsigned long pt_mfn = 0;
--- a/xen/drivers/passthrough/arm/iommu_helpers.c
+++ b/xen/drivers/passthrough/arm/iommu_helpers.c
@@ -57,11 +57,13 @@ int __must_check arm_iommu_map_page(stru
      * The function guest_physmap_add_entry replaces the current mapping
      * if there is already one...
      */
-    return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0, t);
+    return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)),
+                                   IOMMUF_order(flags), t);
 }
 
 /* Should only be used if P2M Table is shared between the CPU and the IOMMU. */
 int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int order,
                                       unsigned int *flush_flags)
 {
     /*
@@ -71,7 +73,8 @@ int __must_check arm_iommu_unmap_page(st
     if ( !is_domain_direct_mapped(d) )
         return -EINVAL;
 
-    return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0);
+    return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)),
+                                     order);
 }
 
 /*
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -271,6 +271,8 @@ int iommu_map(struct domain *d, dfn_t df
     if ( !is_iommu_enabled(d) )
         return 0;
 
+    ASSERT(!IOMMUF_order(flags));
+
     for ( i = 0; i < page_count; i++ )
     {
         rc = iommu_call(hd->platform_ops, map_page, d, dfn_add(dfn, i),
@@ -288,7 +290,7 @@ int iommu_map(struct domain *d, dfn_t df
         while ( i-- )
             /* if statement to satisfy __must_check */
             if ( iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i),
-                            flush_flags) )
+                            0, flush_flags) )
                 continue;
 
         if ( !is_hardware_domain(d) )
@@ -333,7 +335,7 @@ int iommu_unmap(struct domain *d, dfn_t
     for ( i = 0; i < page_count; i++ )
     {
         int err = iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i),
-                             flush_flags);
+                             0, flush_flags);
 
         if ( likely(!err) )
             continue;
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1934,6 +1934,7 @@ static int __must_check intel_iommu_map_
 }
 
 static int __must_check intel_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                               unsigned int order,
                                                unsigned int *flush_flags)
 {
     /* Do nothing if VT-d shares EPT page table */
@@ -1944,7 +1945,7 @@ static int __must_check intel_iommu_unma
     if ( iommu_hwdom_passthrough && is_hardware_domain(d) )
         return 0;
 
-    return dma_pte_clear_one(d, dfn_to_daddr(dfn), 0, flush_flags);
+    return dma_pte_clear_one(d, dfn_to_daddr(dfn), order, flush_flags);
 }
 
 static int intel_iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
--- a/xen/include/asm-arm/iommu.h
+++ b/xen/include/asm-arm/iommu.h
@@ -31,6 +31,7 @@ int __must_check arm_iommu_map_page(stru
                                     unsigned int flags,
                                     unsigned int *flush_flags);
 int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn,
+                                      unsigned int order,
                                       unsigned int *flush_flags);
 
 #endif /* __ARCH_ARM_IOMMU_H__ */
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -127,9 +127,10 @@ void arch_iommu_hwdom_init(struct domain
  * The following flags are passed to map operations and passed by lookup
  * operations.
  */
-#define _IOMMUF_readable 0
+#define IOMMUF_order(n)  ((n) & 0x3f)
+#define _IOMMUF_readable 6
 #define IOMMUF_readable  (1u<<_IOMMUF_readable)
-#define _IOMMUF_writable 1
+#define _IOMMUF_writable 7
 #define IOMMUF_writable  (1u<<_IOMMUF_writable)
 
 /*
@@ -255,6 +256,7 @@ struct iommu_ops {
                                  unsigned int flags,
                                  unsigned int *flush_flags);
     int __must_check (*unmap_page)(struct domain *d, dfn_t dfn,
+                                   unsigned int order,
                                    unsigned int *flush_flags);
     int __must_check (*lookup_page)(struct domain *d, dfn_t dfn, mfn_t *mfn,
                                     unsigned int *flags);



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:46:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:46:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195097.347609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThmO-0006WY-BT; Fri, 24 Sep 2021 09:46:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195097.347609; Fri, 24 Sep 2021 09:46:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThmO-0006WR-8Z; Fri, 24 Sep 2021 09:46:04 +0000
Received: by outflank-mailman (input) for mailman id 195097;
 Fri, 24 Sep 2021 09:46:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThmM-0006WJ-Oi
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:46:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 39f2bbb5-1d1c-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:46:01 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-59ajN07nOGOgxzxYJAVcBg-1;
 Fri, 24 Sep 2021 11:45:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:45:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:45:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0028.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:45:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39f2bbb5-1d1c-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476760;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=s/xqY7vvGGkXpN8hylBtpj3jGQh5qHksSw0RnRhvYDs=;
	b=nOn+jCd+yqX5EKGTQj2C9wEqaD3ySdJh52qjDu9JBUl2nNdrsf3YT3FhQiwll5JQbj9ddm
	8vmL1CSuniV9WDVJuUbnWyIW6jjQlywJa2CiKLe2Kd23TqpBUjB0BtxRLBW6HB0rXeBVoZ
	n5R2RCnT5zQJWplfFs757nI4KKzQWhg=
X-MC-Unique: 59ajN07nOGOgxzxYJAVcBg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dNc2RIpnVnaqZHNOG2WnMYVTe89i6ol1WPN0xGE3jKUjfrIXQak94pNocn/1SosWtrJ8dcVuczMHDS0CJk2YhI95gs+Gh+UIo4i4477B7qtb6iT7s3pxQdquoOTDhsJtsp5Tvup0NQM4BjVlevU49r1DRDxySKiJtAsRpHwVsZ3Pi8Y0/2ppjIewf//W94z7jhFlZzqmGMPB/DsPGzErA5B1M3FBJBf8wbxMj9hEZT/l0GRlxiJxWzaGI63UhD5LoITGGWc/y0efi65e32Tsy9+bxlOeb+gx9zTj9cLcnAH1iutCQk60FwDObHSsYlmPICrY30PiDHocFDJjP/n7IA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=s/xqY7vvGGkXpN8hylBtpj3jGQh5qHksSw0RnRhvYDs=;
 b=QzCTRbv6cMqoEZYHQEZ9SWxXbyhf+A1bRCL9ABdK2dF8LWUU3OPGfl1FNo53XKJUdiPT9fz8t7y4cZd7ElmJKD+qDMnAQGwWd0WtkJ3YV8i0gm9I0Yovd5yFNXNNq7nBOoENXq8dAsOLiqD+WgcfrKICT+m1eYQheK68QaSJe/Bdccl2jaw2vd/THNyG+p7W2coASuVLNeYuvJ8eaudJ4ZZDvM83PRBFzEeT6Gz2XrJ/ndQMCMGSqPbsbvSg/NfxyPrkxcc4aM2MSEqaVZXmcQiQY1s5wnfHg8HtKfVCSjMtbRW+5vWqm5HqKeD1EFMdfyU2ZnTWSN768dDcdUalsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 05/18] IOMMU: have iommu_{,un}map() split requests into
 largest possible chunks
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <134785fb-8ac1-50f0-de75-e0d6fe22f711@suse.com>
Date: Fri, 24 Sep 2021 11:45:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0028.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6bee5668-25ff-40b2-1e4e-08d97f401c46
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383E6CC4CDC61DF1FB193B0B3A49@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1468;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yo0pDFLvICmiv31hJub9ecyxA+bFBlDU40VRVGYb7N41qolLU6/m2UjCnfjIkZKQ4vzwcbn6TsGabGGEb0gr8IQuAiHmw8kq+yBb/Bup4VhT5M0SJGaucSCUXghLuz9g0mRiMYJU63py4hglNop8bVMo+ovVb0A+XycLkbvcONKh9Wvy5LgzVSqmRBYauNSZs0jkYBGb/Z2sfhXHWnmLEzCJql9KXfCgZQopUHSlxnVZPzAGp7z6sqKBqiXk/kvMBmtrB+UYJ/8lkSURjb4qHxAqCAC4wRe92v5/f9QTh5mnq7LjB+0uoXnfwGlBs7cTjp8w42lW0egEb8kdDGwKH/dqaNicRhi8SVTHjy5FvvhugJw7p3VmgEQ0GCz3aQYav77fV1Fa+F2xkiT2FeEJhcpNvbGno7by0gQWF/R4gMWJL32K+N/PPKzlvejVd0ZMvfPhADhwcEmNMWImNYbkRYOJ2iP5a30pxtMzg8TUY4noJktx+Rx2laai85oNHzm/dxZLPOQmdwMDYwPoSSZyXh3anwv5ND+1wZ86RWdP+JjmFjlZmisKmQ9KyhvU4Hrp5tVoSVcsiMT3ddtw/XF0rAleNSh/0g/nltkuAx9dcz9/jruImqaovNzGnh8/BTSM5QMMWz+TDvJKHPP1S3vr3zdbvknSP7B3ieUZQAEiTtcjk7rx7aqW/+UgYFtUIjXkDU2pOMlcMqx8oyu8gUkXkkzgXIsuWqn3NYytaR0fjpQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(508600001)(31686004)(8676002)(16576012)(38100700002)(83380400001)(54906003)(26005)(36756003)(66946007)(31696002)(2906002)(186003)(66476007)(2616005)(66556008)(86362001)(956004)(6916009)(8936002)(316002)(4326008)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TGNUSWZMKzFicjJOMXhjQWhJWDkxbVdEMGY4bGNtMWRrQ1JxVVlDOXFQdEx1?=
 =?utf-8?B?UmNqMjlCQy9HemFvS2luOWtIL2wyYWdXN3ZYTFpFLy9Kdkg3SDBxVmU0dU1a?=
 =?utf-8?B?Nys1M0RLNjRhUWFqenlDNUt4amh4WVU2MTA0YjBUZ01GL2htSXRBUGpuUHFn?=
 =?utf-8?B?dFhWRVhrRkgybHhHbHR0N0lPYWJlT3prcEtINThhUUhkbkphVlNKNEp6MDBn?=
 =?utf-8?B?OVZXZ29TcWY0TkdJTWJHSTgxN1l1Zyt5Q1RoVnFBa1RpOWFSR1RBUkxycXdJ?=
 =?utf-8?B?VHdqQnBpRFZWSE9McUl0QnFibUo4TGREWjFuZ1VCWDZ3NUFBWXQvZ2pmSkNv?=
 =?utf-8?B?VUNGRzcwMTM3OElQam1vNFpPVVdSUW5LTXVBc3NEYUdLRFZ4TXdmcTBYNW9x?=
 =?utf-8?B?L0cvb1ZsWXBFWEYzWi8yTWF0M0x1UVJBWHliWU85SmpoNnFiZWQvY040Z1ZX?=
 =?utf-8?B?Q1ppZHhRYUd5WmZld1BKRVpRNWJLdzkzK3o1Vk04UEJKWkVBWHhoV0Z0VXJ3?=
 =?utf-8?B?cytUbEwyK0lJQWxpbmlQbFV3YytWS1R1N3lpNDFrcC9HY2RSdzdkQlJZVGJI?=
 =?utf-8?B?OEJ0dG9rU0NGcnBsSnIwUlllS1d4ekJmdExtZVQrdlhoK2hVYnFQTHdXeG1p?=
 =?utf-8?B?UkxHajZhUlRHdmVOdGRaeU14MHN5NSsxYXBDTE8yUGhDOTNyWnRtUVhMOGd0?=
 =?utf-8?B?QXRLWEJtcHJmSzBxdG9aMER5QjhYQXpvRVpJWlVoMGlrWFN3OS9DK2F2dkVK?=
 =?utf-8?B?QlpFNXpacXEwVFc3cDZ4N3hob3lxanNqS0lCVVRpcGJ2UjRxV1hpY3VJNE1B?=
 =?utf-8?B?M1p3YlNOaCtDbFNhc1lJVXMwQ1B1QTNITG5DeUpwOEhRdTJwbUxvSU9nVncw?=
 =?utf-8?B?clYzdnM2MHBxbzYxbTBEc0U3dzdSY2dwaGZuTHJMOEZOWHdKcmJlZG5ZZFBT?=
 =?utf-8?B?dXRUR05VVGkzVXZwSUlaOXFPNlJUOTlueER2Y3lTU3cvTjJKdEdSSEk1YUdm?=
 =?utf-8?B?NURFRWhKNHVvN3kvMXJMYWlMdTFtZnhGemdZMFVoYy9ReEJOZnhtZGluTUtG?=
 =?utf-8?B?SVR3TkplTGh3MTM5TmQybVFENlZRd1NWaXpPT0xCbmJ5QXp1QVAwUEJ2Z3I3?=
 =?utf-8?B?dXVkcXo5QXhjdEJ5aEdDV2R2MEJlYVFQdHp4T2Z2bUpFcFV0MkNwWjEyTVJk?=
 =?utf-8?B?VVhDMVk2TmpiV1pBaFFPL3JqQ0NKYmsyQzlTZ3FFMnB0YTV3Ym9EeUhOK0VZ?=
 =?utf-8?B?NEFlNlZTSUpMdmNLdFJoVDVUZGhFenB5VGF0ZnlaWWs0Yy9pbmlJL2pEa1g2?=
 =?utf-8?B?c214YWdrNjh4cWVoTTNyVmlWVEZIRk1DdWs4enlhUC96dzJTanNGdlJVTC9t?=
 =?utf-8?B?SmJyTGp4eVJ5U2RaWUwwdDh4ZHNrYWM0Wkd1dVVITnU0UjJaYUJhYW5MeEx1?=
 =?utf-8?B?bVZrNGdKWkxaKzZ3SW1jWjI0bml4ak1JRWl5akRRUEc2Z21BM2JsakhnT091?=
 =?utf-8?B?WUZvUE55RERFelovNnpVdm80SUFuTU81OEkrdUZnUVFsd2t6L2JXNTZ4K3Y4?=
 =?utf-8?B?eGVmYnVKY3cyQmxaTTlMWGRBSkM2eWxRSUFZekk4TjE0K250VlkwKzlPUi9Z?=
 =?utf-8?B?ZnJwNTFYQnJ1VHY2NlVWMEtyVTBUWTlyS1lSMGd5RWhnK1ZoZEpjWEhZTHZh?=
 =?utf-8?B?S1FZV3N2NktISVBpWEhXeUEwdTZCdXh0ZERLTzh2MGdrMzRabnJmdHdVOVIz?=
 =?utf-8?Q?EdniEpeja+Y27b6FQee1ujltuFhnZY82FEWJ0jY?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6bee5668-25ff-40b2-1e4e-08d97f401c46
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:45:57.9876
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aF/11JB0CpfQDFHdLGJYfxGBwxlEPSpV3e77QoVf3Tkwyzf6tqZTggYotoKUkTsSMv3xKYaeqnP2RKR4Zs+VaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

Introduce a helper function to determine the largest possible mapping
that allows covering a request (or the next part of it that is left to
be processed).

In order to not add yet more recurring dfn_add() / mfn_add() to the two
callers of the new helper, also introduce local variables holding the
values presently operated on.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -260,12 +260,38 @@ void iommu_domain_destroy(struct domain
     arch_iommu_domain_destroy(d);
 }
 
-int iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn,
+static unsigned int mapping_order(const struct domain_iommu *hd,
+                                  dfn_t dfn, mfn_t mfn, unsigned long nr)
+{
+    unsigned long res = dfn_x(dfn) | mfn_x(mfn);
+    unsigned long sizes = hd->platform_ops->page_sizes;
+    unsigned int bit = find_first_set_bit(sizes), order = 0;
+
+    ASSERT(bit == PAGE_SHIFT);
+
+    while ( (sizes = (sizes >> bit) & ~1) )
+    {
+        unsigned long mask;
+
+        bit = find_first_set_bit(sizes);
+        mask = (1UL << bit) - 1;
+        if ( nr <= mask || (res & mask) )
+            break;
+        order += bit;
+        nr >>= bit;
+        res >>= bit;
+    }
+
+    return order;
+}
+
+int iommu_map(struct domain *d, dfn_t dfn0, mfn_t mfn0,
               unsigned long page_count, unsigned int flags,
               unsigned int *flush_flags)
 {
     const struct domain_iommu *hd = dom_iommu(d);
     unsigned long i;
+    unsigned int order;
     int rc = 0;
 
     if ( !is_iommu_enabled(d) )
@@ -273,10 +299,16 @@ int iommu_map(struct domain *d, dfn_t df
 
     ASSERT(!IOMMUF_order(flags));
 
-    for ( i = 0; i < page_count; i++ )
+    for ( i = 0; i < page_count; i += 1UL << order )
     {
-        rc = iommu_call(hd->platform_ops, map_page, d, dfn_add(dfn, i),
-                        mfn_add(mfn, i), flags, flush_flags);
+        dfn_t dfn = dfn_add(dfn0, i);
+        mfn_t mfn = mfn_add(mfn0, i);
+        unsigned long j;
+
+        order = mapping_order(hd, dfn, mfn, page_count - i);
+
+        rc = iommu_call(hd->platform_ops, map_page, d, dfn, mfn,
+                        flags | IOMMUF_order(order), flush_flags);
 
         if ( likely(!rc) )
             continue;
@@ -284,14 +316,18 @@ int iommu_map(struct domain *d, dfn_t df
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
                    "d%d: IOMMU mapping dfn %"PRI_dfn" to mfn %"PRI_mfn" failed: %d\n",
-                   d->domain_id, dfn_x(dfn_add(dfn, i)),
-                   mfn_x(mfn_add(mfn, i)), rc);
+                   d->domain_id, dfn_x(dfn), mfn_x(mfn), rc);
+
+        for ( j = 0; j < i; j += 1UL << order )
+        {
+            dfn = dfn_add(dfn0, j);
+            order = mapping_order(hd, dfn, _mfn(0), i - j);
 
-        while ( i-- )
             /* if statement to satisfy __must_check */
-            if ( iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i),
-                            0, flush_flags) )
+            if ( iommu_call(hd->platform_ops, unmap_page, d, dfn, order,
+                            flush_flags) )
                 continue;
+        }
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
@@ -322,20 +358,25 @@ int iommu_legacy_map(struct domain *d, d
     return rc;
 }
 
-int iommu_unmap(struct domain *d, dfn_t dfn, unsigned long page_count,
+int iommu_unmap(struct domain *d, dfn_t dfn0, unsigned long page_count,
                 unsigned int *flush_flags)
 {
     const struct domain_iommu *hd = dom_iommu(d);
     unsigned long i;
+    unsigned int order;
     int rc = 0;
 
     if ( !is_iommu_enabled(d) )
         return 0;
 
-    for ( i = 0; i < page_count; i++ )
+    for ( i = 0; i < page_count; i += 1UL << order )
     {
-        int err = iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i),
-                             0, flush_flags);
+        dfn_t dfn = dfn_add(dfn0, i);
+        int err;
+
+        order = mapping_order(hd, dfn, _mfn(0), page_count - i);
+        err = iommu_call(hd->platform_ops, unmap_page, d, dfn,
+                         order, flush_flags);
 
         if ( likely(!err) )
             continue;
@@ -343,7 +384,7 @@ int iommu_unmap(struct domain *d, dfn_t
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
                    "d%d: IOMMU unmapping dfn %"PRI_dfn" failed: %d\n",
-                   d->domain_id, dfn_x(dfn_add(dfn, i)), err);
+                   d->domain_id, dfn_x(dfn), err);
 
         if ( !rc )
             rc = err;



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:47:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:47:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195103.347620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThnL-0007Ql-PF; Fri, 24 Sep 2021 09:47:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195103.347620; Fri, 24 Sep 2021 09:47:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThnL-0007Qe-M6; Fri, 24 Sep 2021 09:47:03 +0000
Received: by outflank-mailman (input) for mailman id 195103;
 Fri, 24 Sep 2021 09:47:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThnK-0007QG-2w
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:47:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b352b52-7d7b-493e-acfc-50dd02446495;
 Fri, 24 Sep 2021 09:47:01 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2050.outbound.protection.outlook.com [104.47.1.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-2haBhCC6OTyNDQtOH42hIQ-1;
 Fri, 24 Sep 2021 11:46:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:46:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:46:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0041.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:46:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b352b52-7d7b-493e-acfc-50dd02446495
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476820;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DOSbrpk9/qGJMr59gRB1vlnOjAUciScStVGaTjpeGYw=;
	b=g1JHjEThSty2Dew853P1LWg2/hWkZXRuMkcspy2mJYrwM+HYpYav9DbOl2Rg5G1Ldgw/Cg
	d25G5L2XCUjOti4NhgcArGBkCYw5xe3bAz6nkgpojE1x9662kNs2URc00KBMRnp1ExFTBo
	TZJn77SsqbJj8E+RhETDn5TEzpIU360=
X-MC-Unique: 2haBhCC6OTyNDQtOH42hIQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FZa79uOnGtugO6xxV8FnTvOdPgOqbVgBWyz9UWvJH5jETbpPKVbBKapFuDIPZcKAfBYBBrK3ZRDM+5zfSR7oj4h70fxQoKZImfN+klSQswlJ+xQ0RSSFR4J3CqwWPuKFos19SefY9F/jCVmJLUzCisrEI9lXEyySF3cvEjtQhbN+uqTSikQyrFj1keoc2fXXpAxLSA2MTtrMyOmL/D0qdZ0RYXHLJDQxYupzaL+QKn6N6jooabeHa7obMrAH+/c2RX7Xwosw9bDTTRJrb8QWT6Yb7rsggIgC5fQSBZA//Ca0flp5e9nTpU0nY24MdXZMPaWqEny1jwbey1+TZbCprQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DOSbrpk9/qGJMr59gRB1vlnOjAUciScStVGaTjpeGYw=;
 b=LoOwsR3GnwqvnuOjj7JZGNfrqoID/QkJ2O/qW4WlUT16kK6OP29T9J9E5cIOo29bLzUcmwzQrMic6IJxQHthHA+zeG70C63KuV4CJEPCSdim3MfkusW0hPIDzeGKyPVKtUzaX2r2GN7eGXzQ+NGZZJQTJsJpycMYO5LY9zrJ50fQVeSmLJFzYHQRKODP1IxFyPlj7yBphUXiE5q//Cj1wcFdgqLa6u+jn4eBGKsXvlwtaSOtKlV4Fqve2vIf4Dc5PDesbdY/mEWMRvkGjwDLFAZXhwgENY2J3BUahzu+wj3I/+kTGtChja2N4Kkq0DDyRtH2jEUx94cbtF/weJj6Aw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 06/18] IOMMU/x86: restrict IO-APIC mappings for PV Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <f66b6403-13dd-cb05-7411-20efd7364b08@suse.com>
Date: Fri, 24 Sep 2021 11:46:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0041.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3ac48827-7f92-469d-dbe7-08d97f40402d
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383866C0D352EF8B42E20F3B3A49@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	MLQCRiUl3XKfvd42bzVy5+IPs8pb0i0aWkNaOpkkFKOduQPo+u8sPkHT0KiViRrO1MvrJ+XGb0HzdwfI7KgAyOwOiwm4/IfGrE4LwA5f7RkkMYfzM3Luj75FezNoeVv5zrxt8D6rzXRWpf4wCI8pDObeDMUb5x11kTuriDQD6EOGNTCqsU+Ap48uTGaH2Kq8AHOd3WuuC54ss0U/zitPmDup25GzOGr9zFrjUu9K1FavAkWwYuphSXUeJ2yKrxopL3iSxk71eu0r0N4OH3nNq1318nd6C7vJYryYpGRCIjIrVWfSRNuVamql83WVmCCdtCMlIycR+7AtVGJdC8LiE1wwEPDRLuDiG1xRq4rekPYrKaOhnjwfDspvpKwvq8QDwDAhJ1rpaKnUDKLJ3kcnmNFV3I6AQWohGrkoOYYnuPV/UGDZkGiS7aPDtsa7lNqEM4itVBT2b2i8lbjGLfm6ohKZFkPxHw4gz2i9cQ/RwG9BhemYoZaQqci8ckgU1wxXAIlV1i4HTAgu+h0eU5liL+pGvnf0P2HRxz3s5KVuzKWAbuqPhlRzB/p++arwSDKbl81AfouTNc4wxW//7TII/EqlZG+A3G1f3WXkt2Glg3JfuozrQOL74aQryrB3sgEuFh+j8iRarWg9d8Kbw3N7WDFvtxXvtckYQcr3i1aMwi6J0wBgMb02I8D89P3g34rQ2Y/VYo5unSdf+txYtLpClOnHguBtVu53WfKBOjgbkKg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(508600001)(31686004)(8676002)(16576012)(38100700002)(83380400001)(54906003)(26005)(36756003)(66946007)(31696002)(2906002)(186003)(66476007)(2616005)(66556008)(86362001)(956004)(6916009)(8936002)(316002)(4326008)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?R0hJa1kwQU9PWmk5bHI5YkZsaW1Ra1p0M1J3dlJ5b0lGWEFkVTErV09DNHVY?=
 =?utf-8?B?TFRHcVI5N1dPc1VYQ0lOcXY5eStkYm1KMUh6STM0N3VDNnY1UkNLT0U5N2hZ?=
 =?utf-8?B?cVBoRS8wQ0JpZldKTW8vSzZZTVJ2K2hPa2FRcXhveUc0Mkk1enhEVHBQSkZn?=
 =?utf-8?B?OGJDNTNFcDR1ODk5eEtXNDBJV05IZ2w1MktZY1NITit1VVdZZ2lJeTRKNDN2?=
 =?utf-8?B?cFgwNWFUUUlSSGJ6RmtIQmsxZXA0UzV3Q0Y1TkdXaFpaVnBqMDV5blFDTDlD?=
 =?utf-8?B?ZkdqWmkyTEdKcmVmUFBrenlrNWh0TnpPc0hvd3FnMXJUZG5kdDNDMEpmaHY1?=
 =?utf-8?B?aER4ZkxKOTREZXN0aWdmNDhJM3BDaXNUQlNhR0dCelJKemJOR3BXN1dXOHJV?=
 =?utf-8?B?L2tjbjRJY1pWNU4wWnMycnJUOEozWlIyNDlxUG5iQXNOcE5Gc0p6M1E2YzYr?=
 =?utf-8?B?aVlIS1A5YXZ1clBuRDBueEFhK3doeHRnV3UySUxUSTRvU1pHWnRkeE1EM0U2?=
 =?utf-8?B?d2VNQWdUdWxtSjNKd202QmNzMzQzc09rcHFscEhOZ0REK2RWSUtOdmVRRElE?=
 =?utf-8?B?VlpXL01OYmVNbkx0MEVBSERhRlVYN3RmZU1MdzZ3Ni9oc0hKbFBnZ0VkVzdr?=
 =?utf-8?B?V0tDczB1engzZEFJZXo5ZG9JUjc3Z3dRa3Q3RC8vYzBaUmxzRTA5SzFMUktL?=
 =?utf-8?B?aHNYYjlZRXZJZHhKaTJIZHpaMmd0NERxYmFKZGd3SXg1NE9Nb3BzblcvcU9S?=
 =?utf-8?B?R3U3bVUyWGpuNTBmc3NaQmhHaERKbWhhaTRQdExMVWRuK21CeTFMUEJaVkYr?=
 =?utf-8?B?QjhaejVqQzlGOGZVTGNXZFNucmY0K3BRdmEvRlh3MCtvODE5NFYyT0puS053?=
 =?utf-8?B?QW44RFlUdmFBcS9kOUhkL1BYWnBJTnFYbGd0RWxlMXNDNFBYd2NZS2tPSnJZ?=
 =?utf-8?B?aHJlcWtaRHJGU1B0cGN5YVJ0bk93RVJsekxYcnJTeVFWL1N4eWJOcG5RZ1Va?=
 =?utf-8?B?NW5SMm1pMlppdFErRWVWcHFjRDdnVHd4a2VKbzhrUlRSSUJOa2VtektkZUNJ?=
 =?utf-8?B?Si9CZmtyZ3pqUU9kSDZWMHhreHB4bHhFT28xUlVveVhxUTBZakVNUUo5d3RY?=
 =?utf-8?B?VzNBRVlMK0s4VTdwb0lVcHpPMVpxZDdXNllSRTM0TXBNSTRNY3BEUkFuUWZv?=
 =?utf-8?B?TFRZcVE2YlRsZEFtbGhvblA2MWE4MU1kdW5FYnB1Qm9Gb3JuTkZ1eGVrb2t4?=
 =?utf-8?B?alVaclVReEg5SlFjQTI3ZWtPSGhYZDI2d0pJSDdEcFE0UE41RHlFcTVzVXpC?=
 =?utf-8?B?QnZnK1A4T3VRUk5tc3BwSFJkWFZOTkRLRElEZWtHdnlJZ3VsaUo2SWQvZkx6?=
 =?utf-8?B?VjhYbHFORFVRRHV0SE9GVE1MRTlwTi8xVi9JYTRYZjRzcDlzWi9aTytsSlQv?=
 =?utf-8?B?ckFXcjE4emJxdC9rSnJ4M01TdFA0eXFKK3E3S2JBWlgxb3VvR0t0cnEzZFll?=
 =?utf-8?B?VXROOGp6OXZuM1hmS2xLckRQbXlSa25zT01FeGsyanBqT2pmVWh1diszWHc1?=
 =?utf-8?B?aXlobTJmT1E2SjNLK3RWbFAwU2J4R0E4cnJhS1EzK0pHVWVKQVFJa2VIQk50?=
 =?utf-8?B?TTFLMzZVcktSaU9lcjBrKzlBelZQeXZYdEtZR1NjQnZDVUVaMldRQVpzdEY0?=
 =?utf-8?B?L3Y2cUREbnB1SWx0TzUzYXJUVGw0T2swQUdKMEtyTnFieHQ5Q1FKR2wzMnJK?=
 =?utf-8?Q?YFRPNOqdxeOqZ1Nf1/qX0j4fJTXwmGu7VM0ThPz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3ac48827-7f92-469d-dbe7-08d97f40402d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:46:58.2066
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0gwnhEXx0zUCGaaw4yV3SPpOwbBnprqS+lTMZ8oq8dkkUDaYe9NgUZSrobX93HQzAzr+jXNIRTsc6hSyeI3yMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

While already the case for PVH, there's no reason to treat PV
differently here, though of course the addresses get taken from another
source in this case. Except that, to match CPU side mappings, by default
we permit r/o ones. This then also means we now deal consistently with
IO-APICs whose MMIO is or is not covered by E820 reserved regions.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
[integrated] v1: Integrate into series.
[standalone] v2: Keep IOMMU mappings in sync with CPU ones.

--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -253,12 +253,12 @@ void iommu_identity_map_teardown(struct
     }
 }
 
-static bool __hwdom_init hwdom_iommu_map(const struct domain *d,
-                                         unsigned long pfn,
-                                         unsigned long max_pfn)
+static unsigned int __hwdom_init hwdom_iommu_map(const struct domain *d,
+                                                 unsigned long pfn,
+                                                 unsigned long max_pfn)
 {
     mfn_t mfn = _mfn(pfn);
-    unsigned int i, type;
+    unsigned int i, type, perms = IOMMUF_readable | IOMMUF_writable;
 
     /*
      * Set up 1:1 mapping for dom0. Default to include only conventional RAM
@@ -267,44 +267,60 @@ static bool __hwdom_init hwdom_iommu_map
      * that fall in unusable ranges for PV Dom0.
      */
     if ( (pfn > max_pfn && !mfn_valid(mfn)) || xen_in_range(pfn) )
-        return false;
+        return 0;
 
     switch ( type = page_get_ram_type(mfn) )
     {
     case RAM_TYPE_UNUSABLE:
-        return false;
+        return 0;
 
     case RAM_TYPE_CONVENTIONAL:
         if ( iommu_hwdom_strict )
-            return false;
+            return 0;
         break;
 
     default:
         if ( type & RAM_TYPE_RESERVED )
         {
             if ( !iommu_hwdom_inclusive && !iommu_hwdom_reserved )
-                return false;
+                perms = 0;
         }
-        else if ( is_hvm_domain(d) || !iommu_hwdom_inclusive || pfn > max_pfn )
-            return false;
+        else if ( is_hvm_domain(d) )
+            return 0;
+        else if ( !iommu_hwdom_inclusive || pfn > max_pfn )
+            perms = 0;
     }
 
     /* Check that it doesn't overlap with the Interrupt Address Range. */
     if ( pfn >= 0xfee00 && pfn <= 0xfeeff )
-        return false;
+        return 0;
     /* ... or the IO-APIC */
-    for ( i = 0; has_vioapic(d) && i < d->arch.hvm.nr_vioapics; i++ )
-        if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) )
-            return false;
+    if ( has_vioapic(d) )
+    {
+        for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ )
+            if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) )
+                return 0;
+    }
+    else if ( is_pv_domain(d) )
+    {
+        /*
+         * Be consistent with CPU mappings: Dom0 is permitted to establish r/o
+         * ones there, so it should also have such established for IOMMUs.
+         */
+        for ( i = 0; i < nr_ioapics; i++ )
+            if ( pfn == PFN_DOWN(mp_ioapics[i].mpc_apicaddr) )
+                return rangeset_contains_singleton(mmio_ro_ranges, pfn)
+                       ? IOMMUF_readable : 0;
+    }
     /*
      * ... or the PCIe MCFG regions.
      * TODO: runtime added MMCFG regions are not checked to make sure they
      * don't overlap with already mapped regions, thus preventing trapping.
      */
     if ( has_vpci(d) && vpci_is_mmcfg_address(d, pfn_to_paddr(pfn)) )
-        return false;
+        return 0;
 
-    return true;
+    return perms;
 }
 
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
@@ -346,15 +362,19 @@ void __hwdom_init arch_iommu_hwdom_init(
     for ( ; i < top; i++ )
     {
         unsigned long pfn = pdx_to_pfn(i);
+        unsigned int perms = hwdom_iommu_map(d, pfn, max_pfn);
         int rc;
 
-        if ( !hwdom_iommu_map(d, pfn, max_pfn) )
+        if ( !perms )
             rc = 0;
         else if ( paging_mode_translate(d) )
-            rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
+            rc = set_identity_p2m_entry(d, pfn,
+                                        perms & IOMMUF_writable ? p2m_access_rw
+                                                                : p2m_access_r,
+                                        0);
         else
             rc = iommu_map(d, _dfn(pfn), _mfn(pfn), 1ul << PAGE_ORDER_4K,
-                           IOMMUF_readable | IOMMUF_writable, &flush_flags);
+                           perms, &flush_flags);
 
         if ( rc )
             printk(XENLOG_WARNING "%pd: identity %smapping of %lx failed: %d\n",



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:47:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:47:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195110.347631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTho6-00088H-3g; Fri, 24 Sep 2021 09:47:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195110.347631; Fri, 24 Sep 2021 09:47:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTho6-00088A-0S; Fri, 24 Sep 2021 09:47:50 +0000
Received: by outflank-mailman (input) for mailman id 195110;
 Fri, 24 Sep 2021 09:47:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTho4-00087y-EL
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:47:48 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 56b188d0-db91-4d15-b6ad-625cb9512862;
 Fri, 24 Sep 2021 09:47:46 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-kIPxbBSXPkuyq9C-KpSDfQ-1; Fri, 24 Sep 2021 11:47:44 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5470.eurprd04.prod.outlook.com (2603:10a6:803:d6::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 09:47:42 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:47:42 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0173.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1c::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.14 via Frontend Transport; Fri, 24 Sep 2021 09:47:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56b188d0-db91-4d15-b6ad-625cb9512862
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476865;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oyX0YzgcM98sgcoXD/7eRYbXR7J1MZ8Fv/UNIJmQ4Us=;
	b=bZevIhY4fsTtsOAlmoWzxY2125m59BhXLgZqW6JrPJDYCRQaryFIahPsOoy/MXCHahhW5o
	E8/s6JFmDlXPPmYHKb9I5ZkHfQMXGhU1e5ZVnoIQDBnbJX+HMuNm2vRH+g/296LYHfPfUX
	C/GE5CQ9QutyTTbZmSNI2nVevF9Ruvs=
X-MC-Unique: kIPxbBSXPkuyq9C-KpSDfQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HU1c2h/q6Te64nCMhuZjMQKFgpkp507DkIyNYQ3+/X/pOvVkgooczSTN1Q8zIWceP8I45aQBGOCytGCB2c4tmxR2W4BSPZf/Cz7A/8Y7iIe1V8hNH8u174lQtjVLxFq5iEDirKU8j2Tr1/PAk+QWvt3L8Q6BWlv8A96oO4+9DDO/Lj7wjXAGAEaspqLur4fIAv2pjBZJtOKn+FbbRUygwlSd1X3/iVeHs8uLovq09LX0wUZKRsHiaRhG7PexVuCQhqIjhF3QJnqjDRBE3sl9/kJUDk7QUucURzfd6zB4bZqLqbfbFxxJaLMO2+qDPToLSQmZ8/G3EMPpW6afUtdljA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oyX0YzgcM98sgcoXD/7eRYbXR7J1MZ8Fv/UNIJmQ4Us=;
 b=L6L/Vti8FWFeV+eAWkuC1YSOtOkoQYILXmWoX7dzlSUSfiumXXwlakI6n98nLTc+pKJLjsBVBAF2ZalAc9iTGWkvfjwlxxOYR8GNUabzbJj2tNHl6IHLqIVcPPqhI2Pdaa/bmlldQo2oq6eMLINcgYO6h/BYPmRHwiqkzIMxGI+XuxWT9KDAHzqkZTLywDKdp5C7uoCTtp582xsjViEGbCFaFCwD72jZpB5N2An07XkNhvVIep2rrhQ23kl+pcEO1reRXznlJKL2N2/8hg4nqVG1GK8Ixhvl2ouOB/w7dXCd8RB6f4itMliKh3pJoonqskvb/OEyvffem0QV935FSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 07/18] IOMMU/x86: perform PV Dom0 mappings in batches
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <cc183d61-f4ed-1cf5-185b-0673c9927892@suse.com>
Date: Fri, 24 Sep 2021 11:47:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0173.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1c::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3359d961-8103-4945-fb6b-08d97f405a78
X-MS-TrafficTypeDiagnostic: VI1PR04MB5470:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB54706C786B90078EEC987FDEB3A49@VI1PR04MB5470.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:669;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fmEq7vsEOaF0T0a3PEyqQrIxRDvtouCx1pDXmmZm4BFuaUZeFm0kqt4ugo1ZZprFe6cU7tveuuhF951h7QF/GiPG1BaMaJNrjq+WapevubTmjPK3QGRrLj3qs1Vd/LbVs0XdxcU8kf/NJ0MegS64DYNTzkYUOoNgzzFTTTuAWZM5R2ZmKXrLgQaC0hiJhAC/j7VJwQ2jQ255w+B1+Xjo+mIjqgp7YhYqClfoTnaUL8ofiguvxMBrDaSHgQGLQNtBdu8WhC8awJrA24/NpoAUGC6q+rm8d7KPLVZo0mFzwnyd/EP4VdDp+9EL0VCGJg/Jz33+DzeMCLc97BCeTDGWKa6JsLqTUpH21YIa09rjWGHXcSA4wJr77NHfX2J1vcGWxukdOSc0Bc/LY7lcppDtXBGK1uHHLLC0ElFSEws0RtL9RcFQd/uYhhfLGvppKgjINfVdhQjmfvoFzo1W+NxtS4tP+FVODsPCi4V/ax+UUfhgvDFqhg9C3eN8yRyiYQbkKyHEWM9lV5pJUzUjyM8243Cwvzvz+30plgiXw0pCwF/TPDtXa2OoUFUcwY/KZx2X6xxNpS6GJ8SSn2FchIqJqujVKu8NYhuHsKX+Y2pU/nLYkmFBysAU4iOx1N95Ji2Rzofks0Q61/4rAszDDt1L490cUcUrVy4tXSzkgEi6ci9fDomTxX5bZwcwpoqmreRs5Dz5bD4Af6aBE6Pp4k+x50filgGke5z8TPbEKyDlYLs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(186003)(66556008)(66476007)(26005)(2906002)(8676002)(31686004)(86362001)(6486002)(66946007)(8936002)(2616005)(956004)(83380400001)(31696002)(508600001)(38100700002)(5660300002)(16576012)(54906003)(4326008)(316002)(30864003)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YnNzbkF0aS9FMFQ5RGI5SFBlZklKeGNFN2FRcW11TTRxTWlmQmU0bG5qTlBq?=
 =?utf-8?B?RFlYTnJHTW51K05oNEtOMGFkbkxETDYrWGJtR280QnBtbFFCVk9KNXdNamZF?=
 =?utf-8?B?V2xXK0xuZFhVTXNtckZsSmFqbERYQXUrQVdwcUtia1BETHRudEpXNDF2RlFL?=
 =?utf-8?B?RUlqSFpXbVFtWUpCQ2ozdXRram82bjNYZjJwU3hmR2pjTXVwOGxoK0l5Sm0v?=
 =?utf-8?B?Uk1kUHdjZThrMWNWbkNwUWMwUlROSE1ZZDlsWEt1SzlMYmdDcksyTFJwR3lT?=
 =?utf-8?B?MVJhWnB2VmtrZkR0dW1NT3pLTzZQdGVCeU90M2kzckJIL1AwV3BDSkFCUTV2?=
 =?utf-8?B?NXpSUWcrcW9jbzlsOXluWlo0aTEvaTVJZWJwZTVnTXdHOWlqOXBHUkZEb0p6?=
 =?utf-8?B?Y2tnZ2lRMFRjVXAvSUhlV3VjQ3FmL3RuK1g2eWQvYkhmUUsrSG50UFVrd0l5?=
 =?utf-8?B?NER0TWlMSjVLV1llZEZxQzNoRTZ1eDV2T2x6a3BETmNlSEFyY20yMDdpdHN5?=
 =?utf-8?B?eUwyOExiVkJkYTJNaFVZNWtsS2pzZXA4M0QxU1g2MU5idzBCcmt4TFJBRW42?=
 =?utf-8?B?Vk9pc3BJTzNWSmpndjZNelVQbW0yS0duS0pkaXZjSnVHYURmOHloR2dGZ1JW?=
 =?utf-8?B?ZWk1THBUTXRabmRCT2lUeWs5dGFybkR1dUNaSEt3TU9KdkRFT1M5YkZsejBw?=
 =?utf-8?B?TW9EeHpYdWlLdWloVXVJdTZTWEdGSHRzTUxtcUFxRll1UHU5VzloVHRSckFC?=
 =?utf-8?B?cjcvdVJrYXBlVWMvL3c2Z0xPS0E1bzJwdHhzclNSOE5ZSGt1cVQ3bm9GU1h3?=
 =?utf-8?B?a1l2bThjVzNqNkErV0VIK0IveVNlNHdIa0VVTjZ1SXFZOVFSQ1gwL2h0NjBZ?=
 =?utf-8?B?N2xzYXFPQzJGMkVuTldmNXRKSVZYemNzYlcyRjV0YlpWaUp5a2ErM3VhY25a?=
 =?utf-8?B?U2hIcUErc1NLK1RtdGhVSXZ0RGZ0d0ROdHdMdEVuS3RVVy9pV1RoY05TUnRk?=
 =?utf-8?B?UGZ6cXNxUUtQd1c2MlIydHFIblRqVmlzeDhSVG5jdDFmSHNGYjdHNEpDNmFG?=
 =?utf-8?B?ZnFveVIrY2x5dGRtS3hCWEFId1VOVGpkZnNLYnlvQnQyUTNBUVVWL0dvVzdU?=
 =?utf-8?B?b2ZzYVk1YTM0R1B0ZXVqYnFwcUNpdXZDSjdlczFGZCsvVVNTeVd1Q3Z4Nmo5?=
 =?utf-8?B?SDliMFg4QXR2cDBKUDRzUTVtSm5QcWQ2eCt5TVB6T1UyZVRjR1pKbnFIY0Qw?=
 =?utf-8?B?c0dTTFdvanFEQkN2OW0wWEVPbWpHYTRJbkg2TmlRUDNMUnhLSGlFNlZtd0dL?=
 =?utf-8?B?Z0xpQWlrMytqTjF3cHVxVkI3U2xPdWhLT2JlUE82dlVGNjQ0WmppYVcxdGp6?=
 =?utf-8?B?cGtCSDBMR2ZmTGFWU2J3dTRiYXJFWXJiNnp4TUQ5Z0xNd3BKSVVwWlprbE50?=
 =?utf-8?B?RVJOVXVuT0N2MlF4MmYxOFJ0N2E2Y0JRWnJZZDVoVWcwUERBUWtYeVpISGJS?=
 =?utf-8?B?L0MzeHMrdnoyVmloYzVISVRVU09vT2JBQWVEZlkwNkp4YTNkZ2lzWGpBa1Uw?=
 =?utf-8?B?OWhjQ1NyNXVWS2ZxUFQySXRmMUU4cHdiUHhZWHJsaHl2c21pcDlaQWF6dXRE?=
 =?utf-8?B?V25YN1piZGNhT1JLTHFyT2FyNmZnVFBmQ0c5RGtxaSs1VEN0Z3JvdkFodEhQ?=
 =?utf-8?B?VEY1bWhWeWRkTUJWbllxVk1IcG1uWHdqYy8rZ3ZIMVBPanppTHZHS1lWZ0xk?=
 =?utf-8?Q?jW0GJaqWJAqq5+rJ4UVpC4BVLcK+fxw5efC71mj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3359d961-8103-4945-fb6b-08d97f405a78
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:47:42.3487
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7UvDYijL80joRxvqUXbDRrCFgBXI/RAWVyZyTTcJ/4obGxS6pVuqTJRaHMcDr9uTi3X/hCHD/KrFMQPrfT6rGg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5470

For large page mappings to be easily usable (i.e. in particular without
un-shattering of smaller page mappings) and for mapping operations to
then also be more efficient, pass batches of Dom0 memory to iommu_map().
In dom0_construct_pv() and its helpers (covering strict mode) this
additionally requires establishing the type of those pages (albeit with
zero type references).

The earlier establishing of PGT_writable_page | PGT_validated requires
the existing places where this gets done (through get_page_and_type())
to be updated: For pages which actually have a mapping, the type
refcount needs to be 1.

There is actually a related bug that gets fixed here as a side effect:
Typically the last L1 table would get marked as such only after
get_page_and_type(..., PGT_writable_page). While this is fine as far as
refcounting goes, the page did remain mapped in the IOMMU in this case
(when "iommu=dom0-strict").

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Subsequently p2m_add_identity_entry() may want to also gain an order
parameter, for arch_iommu_hwdom_init() to use. While this only affects
non-RAM regions, systems typically have 2-16Mb of reserved space
immediately below 4Gb, which hence could be mapped more efficiently.

The installing of zero-ref writable types has in fact shown (observed
while putting together the change) that despite the intention by the
XSA-288 changes (affecting DomU-s only) for Dom0 a number of
sufficiently ordinary pages (at the very least initrd and P2M ones as
well as pages that are part of the initial allocation but not part of
the initial mapping) still have been starting out as PGT_none, meaning
that they would have gained IOMMU mappings only the first time these
pages would get mapped writably.

I didn't think I need to address the bug mentioned in the description in
a separate (prereq) patch, but if others disagree I could certainly
break out that part (needing to first use iommu_legacy_unmap() then).

Note that 4k P2M pages don't get (pre-)mapped in setup_pv_physmap():
They'll end up mapped via the later get_page_and_type().

As to the way these refs get installed: I've chosen to avoid the more
expensive {get,put}_page_and_type(), putting in place the intended type
directly. I guess I could be convinced to avoid this bypassing of the
actual logic; I merely think it's unnecessarily expensive.

--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -106,11 +106,26 @@ static __init void mark_pv_pt_pages_rdon
     unmap_domain_page(pl3e);
 }
 
+/*
+ * For IOMMU mappings done while building Dom0 the type of the pages needs to
+ * match (for _get_page_type() to unmap upon type change). Set the pages to
+ * writable with no type ref. NB: This is benign when !need_iommu_pt_sync(d).
+ */
+static void __init make_pages_writable(struct page_info *page, unsigned long nr)
+{
+    for ( ; nr--; ++page )
+    {
+        ASSERT(!page->u.inuse.type_info);
+        page->u.inuse.type_info = PGT_writable_page | PGT_validated;
+    }
+}
+
 static __init void setup_pv_physmap(struct domain *d, unsigned long pgtbl_pfn,
                                     unsigned long v_start, unsigned long v_end,
                                     unsigned long vphysmap_start,
                                     unsigned long vphysmap_end,
-                                    unsigned long nr_pages)
+                                    unsigned long nr_pages,
+                                    unsigned int *flush_flags)
 {
     struct page_info *page = NULL;
     l4_pgentry_t *pl4e, *l4start = map_domain_page(_mfn(pgtbl_pfn));
@@ -123,6 +138,8 @@ static __init void setup_pv_physmap(stru
 
     while ( vphysmap_start < vphysmap_end )
     {
+        int rc = 0;
+
         if ( domain_tot_pages(d) +
              ((round_pgup(vphysmap_end) - vphysmap_start) >> PAGE_SHIFT) +
              3 > nr_pages )
@@ -176,7 +193,22 @@ static __init void setup_pv_physmap(stru
                                              L3_PAGETABLE_SHIFT - PAGE_SHIFT,
                                              MEMF_no_scrub)) != NULL )
             {
-                *pl3e = l3e_from_page(page, L1_PROT|_PAGE_DIRTY|_PAGE_PSE);
+                mfn_t mfn = page_to_mfn(page);
+
+                if ( need_iommu_pt_sync(d) )
+                    rc = iommu_map(d, _dfn(mfn_x(mfn)), mfn,
+                                   SUPERPAGE_PAGES * SUPERPAGE_PAGES,
+                                   IOMMUF_readable | IOMMUF_writable,
+                                   flush_flags);
+                if ( !rc )
+                    make_pages_writable(page,
+                                        SUPERPAGE_PAGES * SUPERPAGE_PAGES);
+                else
+                    printk(XENLOG_ERR
+                           "pre-mapping P2M 1G-MFN %lx into IOMMU failed: %d\n",
+                           mfn_x(mfn), rc);
+
+                *pl3e = l3e_from_mfn(mfn, L1_PROT|_PAGE_DIRTY|_PAGE_PSE);
                 vphysmap_start += 1UL << L3_PAGETABLE_SHIFT;
                 continue;
             }
@@ -202,7 +234,20 @@ static __init void setup_pv_physmap(stru
                                              L2_PAGETABLE_SHIFT - PAGE_SHIFT,
                                              MEMF_no_scrub)) != NULL )
             {
-                *pl2e = l2e_from_page(page, L1_PROT|_PAGE_DIRTY|_PAGE_PSE);
+                mfn_t mfn = page_to_mfn(page);
+
+                if ( need_iommu_pt_sync(d) )
+                    rc = iommu_map(d, _dfn(mfn_x(mfn)), mfn, SUPERPAGE_PAGES,
+                                   IOMMUF_readable | IOMMUF_writable,
+                                   flush_flags);
+                if ( !rc )
+                    make_pages_writable(page, SUPERPAGE_PAGES);
+                else
+                    printk(XENLOG_ERR
+                           "pre-mapping P2M 2M-MFN %lx into IOMMU failed: %d\n",
+                           mfn_x(mfn), rc);
+
+                *pl2e = l2e_from_mfn(mfn, L1_PROT|_PAGE_DIRTY|_PAGE_PSE);
                 vphysmap_start += 1UL << L2_PAGETABLE_SHIFT;
                 continue;
             }
@@ -310,6 +355,7 @@ int __init dom0_construct_pv(struct doma
     unsigned long initrd_pfn = -1, initrd_mfn = 0;
     unsigned long count;
     struct page_info *page = NULL;
+    unsigned int flush_flags = 0;
     start_info_t *si;
     struct vcpu *v = d->vcpu[0];
     void *image_base = bootstrap_map(image);
@@ -572,6 +618,18 @@ int __init dom0_construct_pv(struct doma
                     BUG();
         }
         initrd->mod_end = 0;
+
+        count = PFN_UP(initrd_len);
+
+        if ( need_iommu_pt_sync(d) )
+            rc = iommu_map(d, _dfn(initrd_mfn), _mfn(initrd_mfn), count,
+                           IOMMUF_readable | IOMMUF_writable, &flush_flags);
+        if ( !rc )
+            make_pages_writable(mfn_to_page(_mfn(initrd_mfn)), count);
+        else
+            printk(XENLOG_ERR
+                   "pre-mapping initrd (MFN %lx) into IOMMU failed: %d\n",
+                   initrd_mfn, rc);
     }
 
     printk("PHYSICAL MEMORY ARRANGEMENT:\n"
@@ -605,6 +663,22 @@ int __init dom0_construct_pv(struct doma
 
     process_pending_softirqs();
 
+    /*
+     * We map the full range here and then punch a hole for page tables via
+     * iommu_unmap() further down, once they have got marked as such.
+     */
+    if ( need_iommu_pt_sync(d) )
+        rc = iommu_map(d, _dfn(alloc_spfn), _mfn(alloc_spfn),
+                       alloc_epfn - alloc_spfn,
+                       IOMMUF_readable | IOMMUF_writable, &flush_flags);
+    if ( !rc )
+        make_pages_writable(mfn_to_page(_mfn(alloc_spfn)),
+                            alloc_epfn - alloc_spfn);
+    else
+        printk(XENLOG_ERR
+               "pre-mapping MFNs [%lx,%lx) into IOMMU failed: %d\n",
+               alloc_spfn, alloc_epfn, rc);
+
     mpt_alloc = (vpt_start - v_start) + pfn_to_paddr(alloc_spfn);
     if ( vinitrd_start )
         mpt_alloc -= PAGE_ALIGN(initrd_len);
@@ -689,7 +763,8 @@ int __init dom0_construct_pv(struct doma
         l1tab++;
 
         page = mfn_to_page(_mfn(mfn));
-        if ( !page->u.inuse.type_info &&
+        if ( (!page->u.inuse.type_info ||
+              page->u.inuse.type_info == (PGT_writable_page | PGT_validated)) &&
              !get_page_and_type(page, d, PGT_writable_page) )
             BUG();
     }
@@ -720,6 +795,17 @@ int __init dom0_construct_pv(struct doma
     /* Pages that are part of page tables must be read only. */
     mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages);
 
+    /*
+     * This needs to come after all potentially excess
+     * get_page_and_type(..., PGT_writable_page) invocations; see the loop a
+     * few lines further up, where the effect of calling that function in an
+     * earlier loop iteration may get overwritten by a later one.
+     */
+    if ( need_iommu_pt_sync(d) &&
+         iommu_unmap(d, _dfn(PFN_DOWN(mpt_alloc) - nr_pt_pages), nr_pt_pages,
+                     &flush_flags) )
+        BUG();
+
     /* Mask all upcalls... */
     for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ )
         shared_info(d, vcpu_info[i].evtchn_upcall_mask) = 1;
@@ -793,7 +879,7 @@ int __init dom0_construct_pv(struct doma
     {
         pfn = pagetable_get_pfn(v->arch.guest_table);
         setup_pv_physmap(d, pfn, v_start, v_end, vphysmap_start, vphysmap_end,
-                         nr_pages);
+                         nr_pages, &flush_flags);
     }
 
     /* Write the phys->machine and machine->phys table entries. */
@@ -824,7 +910,9 @@ int __init dom0_construct_pv(struct doma
         if ( get_gpfn_from_mfn(mfn) >= count )
         {
             BUG_ON(compat);
-            if ( !page->u.inuse.type_info &&
+            if ( (!page->u.inuse.type_info ||
+                  page->u.inuse.type_info == (PGT_writable_page |
+                                              PGT_validated)) &&
                  !get_page_and_type(page, d, PGT_writable_page) )
                 BUG();
 
@@ -840,22 +928,41 @@ int __init dom0_construct_pv(struct doma
 #endif
     while ( pfn < nr_pages )
     {
-        if ( (page = alloc_chunk(d, nr_pages - domain_tot_pages(d))) == NULL )
+        count = domain_tot_pages(d);
+        if ( (page = alloc_chunk(d, nr_pages - count)) == NULL )
             panic("Not enough RAM for DOM0 reservation\n");
+        mfn = mfn_x(page_to_mfn(page));
+
+        if ( need_iommu_pt_sync(d) )
+        {
+            rc = iommu_map(d, _dfn(mfn), _mfn(mfn), domain_tot_pages(d) - count,
+                           IOMMUF_readable | IOMMUF_writable, &flush_flags);
+            if ( rc )
+                printk(XENLOG_ERR
+                       "pre-mapping MFN %lx (PFN %lx) into IOMMU failed: %d\n",
+                       mfn, pfn, rc);
+        }
+
         while ( pfn < domain_tot_pages(d) )
         {
-            mfn = mfn_x(page_to_mfn(page));
+            if ( !rc )
+                make_pages_writable(page, 1);
+
 #ifndef NDEBUG
 #define pfn (nr_pages - 1 - (pfn - (alloc_epfn - alloc_spfn)))
 #endif
             dom0_update_physmap(compat, pfn, mfn, vphysmap_start);
 #undef pfn
-            page++; pfn++;
+            page++; mfn++; pfn++;
             if ( !(pfn & 0xfffff) )
                 process_pending_softirqs();
         }
     }
 
+    /* Use while() to avoid compiler warning. */
+    while ( iommu_iotlb_flush_all(d, flush_flags) )
+        break;
+
     if ( initrd_len != 0 )
     {
         si->mod_start = vinitrd_start ?: initrd_pfn;
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -325,8 +325,8 @@ static unsigned int __hwdom_init hwdom_i
 
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 {
-    unsigned long i, top, max_pfn;
-    unsigned int flush_flags = 0;
+    unsigned long i, top, max_pfn, start, count;
+    unsigned int flush_flags = 0, start_perms = 0;
 
     BUG_ON(!is_hardware_domain(d));
 
@@ -357,9 +357,9 @@ void __hwdom_init arch_iommu_hwdom_init(
      * First Mb will get mapped in one go by pvh_populate_p2m(). Avoid
      * setting up potentially conflicting mappings here.
      */
-    i = paging_mode_translate(d) ? PFN_DOWN(MB(1)) : 0;
+    start = paging_mode_translate(d) ? PFN_DOWN(MB(1)) : 0;
 
-    for ( ; i < top; i++ )
+    for ( i = start, count = 0; i < top; )
     {
         unsigned long pfn = pdx_to_pfn(i);
         unsigned int perms = hwdom_iommu_map(d, pfn, max_pfn);
@@ -372,16 +372,30 @@ void __hwdom_init arch_iommu_hwdom_init(
                                         perms & IOMMUF_writable ? p2m_access_rw
                                                                 : p2m_access_r,
                                         0);
+        else if ( pfn != start + count || perms != start_perms )
+        {
+        commit:
+            rc = iommu_map(d, _dfn(start), _mfn(start), count,
+                           start_perms, &flush_flags);
+            SWAP(start, pfn);
+            start_perms = perms;
+            count = 1;
+        }
         else
-            rc = iommu_map(d, _dfn(pfn), _mfn(pfn), 1ul << PAGE_ORDER_4K,
-                           perms, &flush_flags);
+        {
+            ++count;
+            rc = 0;
+        }
 
         if ( rc )
             printk(XENLOG_WARNING "%pd: identity %smapping of %lx failed: %d\n",
                    d, !paging_mode_translate(d) ? "IOMMU " : "", pfn, rc);
 
-        if (!(i & 0xfffff))
+        if ( !(++i & 0xfffff) )
             process_pending_softirqs();
+
+        if ( i == top && count )
+            goto commit;
     }
 
     /* Use if to avoid compiler warning */



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:48:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:48:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195117.347641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThoi-0000NJ-G8; Fri, 24 Sep 2021 09:48:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195117.347641; Fri, 24 Sep 2021 09:48:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThoi-0000NC-DD; Fri, 24 Sep 2021 09:48:28 +0000
Received: by outflank-mailman (input) for mailman id 195117;
 Fri, 24 Sep 2021 09:48:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThoh-0000Mx-Cn
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:48:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9034335e-1d1c-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:48:26 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-As5PKxn4PvSSFiflEF4zwQ-1; Fri, 24 Sep 2021 11:48:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:48:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:48:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0029.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:48:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9034335e-1d1c-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476905;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=151rl1mGGzZn83HuceugMX8Iyq5H3jwKYMBAqhoHteE=;
	b=lSVyiqs+8ghGfNwz4Q+1WzhguZ90YH903I5yv76w85ORSo2hvH6vt49lqHBRNbBGU4OS+Y
	bo/z59uZqWxSyOPkqTvJ1zjxkVKoduyFygGC+V3Yi4rt0rAOfP539wDLW1uu+DScG+aDjz
	XZjq1djoJCdUhzJ3SMigGNwXVgGhEu8=
X-MC-Unique: As5PKxn4PvSSFiflEF4zwQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WJe3Ed7vXfSsY9jLwApv6GWY9JcwvFKdo5gXdBn7+vlj1GgNEKBY+4HtRfp8Mf+P/X7Yhw7CWWa8W8ucjEFWE1KBox9qTmCr5g+sdXFo+4CyF0Oo5et8P+hLdgc3i2rt7tSROt8gPARSLo6dx/0RXzeUhcSRkckErwWq/e+uQ9ar3iS3T612YWn0PSV5vYevStNZct+CrZo9EY+rJx+9OzuZEgjGkB0nO5a0h7A+sywg9JFcINh00GjBkKTR2C1MKCEsBLUbcIjxEhPJflhz/LriDcqIZD1gUyrpl2RRIyDjSGZYMYcqBP1aDqnqXxBrBZEv9N1n+sNX2HL30rfWuQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=151rl1mGGzZn83HuceugMX8Iyq5H3jwKYMBAqhoHteE=;
 b=Flz7lE5uXkmVdwABDcL8Qc2JFrMX57bzmt0R2EsR0qglV517fogxDcmsLt8focPsp+oUjnUU2SeUxofC1DEcmsrOlc3EOFtOCLI+1PVIk/grT1cSH4JM4uSdbljwcYNNBLa9XUIqk+tpHT5cCyd2+AeIH7f206RdEL8SVz8YlPriJ1kMBgXlW8lOxlfvQVgyWIEJxRNH8tvCqXsku2xz4Ewq33Sfh/g+yn9tyqGwtVQziFImkuuhpAYj1ZBBzkzY+Hridqkm+ujsHFNPGoJq96dXrD7/A0LLEITcQNO1Snf1iEj97352UXvgsTLA16H24zQv9ztPJFNYB3/+H8L2rg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 08/18] IOMMU/x86: support freeing of pagetables
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <c12fccbf-82f4-1259-f69d-a6ad8d78ea15@suse.com>
Date: Fri, 24 Sep 2021 11:48:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0029.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7c925c99-869b-44cc-d280-08d97f40732f
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383D3F380362AED5E2D87D0B3A49@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1284;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UoTjbOIIz7dQ/dbSoFlRrQkf4wXyncZyCIjeAq1Ir4b1ieLk8hDbar30qp3snUKZbALNgdEWcjJmePwdR/g1TaLjlw18JIPU7lME5/GFlKiRyfRKye4ZMxgCButMTVFuSIuWUKYDFdqIYvxZ7d+2K/Z4bQJOqnKKgL8m8K5k3q46Rlf/1wY8Og3Sf9sUbVfb5UQEbv21P1aBFLOO13Y0KGrA75Yq362rl2mXg4rsz4CTC9A8x7goEHtT4AYepTjpsm9qDxeIn54ou9N2CEYS3tYCXs2R37uyli9WhehzGO64QO/lSQFmc3lJ0LkTNYak5ehz6IjZbI7HSjkWYsYMuMd/4pVAbj+6j2iXwVbmyhFz9ifKui57Lr183VCKu7SCswmHev92+AWQwWhoSINAxStNFpn4vfevXr1cylGt4d43hMqdMb/U6YVLOVfKPRxMEQy+4Bd0go8+1MbgjxFgc7azsToAMjp8TEvohyJxv15yFwtuDPRcDtybLaqPqDte7ZT1PL4mgccW8y3nYGsplC91qTFAGRajbesNRa26g/oRIQBGL085MdRgZQgdpIBWKOto7Va+jdMdpNjCRVoLURzkQulirwUIFR1nY1rwApWJTgPs7/t7PAvVHVul6yV1JZ5hYODnsjFvWUmyNbICANn6BKXi8JTvsIpiYcLG7tsJH7T85dJLqrpSlLrq8oGOkIKuxQUMGs6Jq8/tf6qbUnIxFmHqtxDfeD7rnJ43RaYOkvcRT70a7vSpDAx+UqQdrefKquQoWhQT3ldikF6hYef8NroEeEcBa5dHSI5DVXv+qQ+3HciZ2fSWXsbCiyuC
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(508600001)(31686004)(8676002)(16576012)(38100700002)(83380400001)(54906003)(26005)(36756003)(66946007)(31696002)(2906002)(186003)(66476007)(2616005)(66556008)(86362001)(956004)(6916009)(8936002)(316002)(4326008)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OUE2d05HK08zYkNJUUw4a1BYQURXV3RSQkptM2pBWU0rQWt4eW4wMG50K0lC?=
 =?utf-8?B?TkVrblhnK255TU5PKzhHUDZZU2JJS1JueGFDMC94SHhBNDVwMWdKUnlzRU1o?=
 =?utf-8?B?cXpGazFGT0cvWHFJUGJPN0Qyei9kcWNmZjN4blptbDlLT2lCaEJUZkJOSk1S?=
 =?utf-8?B?dEZaMklsbWEvZTlsbFRUUWZtbHlaSERFbG1mRmFRMVR4R3NtNFhHQTc2VlNX?=
 =?utf-8?B?VnIxY2J1cENoQ0tjLzhmbTYxVWpUYnhvYWpIWDNDMWEwY2xLVFJBWWxRZ1hl?=
 =?utf-8?B?eXVLZU52VGk0b1VtNy9oUURYd0FGazcwTnhzN2FzZW01TmI0b25QWmhZWHZI?=
 =?utf-8?B?ZGFkcEdEeGdGSUxSWXg4OW9RMkhuTHlsS01KNllNOFU2d09YdjF4d1NnMjdO?=
 =?utf-8?B?YkhwY3BiOVBCVk9sR0Q1SWJ5ZkpsTjlrOTFZN1ZJZ1lzLzF2VE9SWDZVSU1C?=
 =?utf-8?B?Z1JNdUZtSzUxN05SSzRFY0RjUi9QL2ZNOTVlRG90dDRtejBVS2I3T0pTL0xa?=
 =?utf-8?B?K1N0VlM2aEo0OWpsUThVMTkvelFUS08wc0xWdXFTTmNUTnB2dnQzbjhkS3hY?=
 =?utf-8?B?a2hEa3RuQWVIZitDWWpjU3BTb0w4WStEZTRRellodWpMMzl2TFZ6bUU5WWh5?=
 =?utf-8?B?M1JEWE9PT3dMdjNnWENsTXNUZTNXRU1DaEpWTjF3dG1zVEJXU04wRDFnRjJC?=
 =?utf-8?B?RHZFdFlzZE9Camc4dEc1YTVsV1RsTm40QXJ5VTF5ZnNVVWo0c09yK2FJNFZw?=
 =?utf-8?B?cGJEY280SlBvSnlpUllOejBVeUxTelhTeE1oMGU4QnVjdS90MThyYitIRTdq?=
 =?utf-8?B?ZEJIeGVNdTl5bDRtV3JENGMrdVQydzFpNEhKVDNIRGNFMEVlVmpucVJVazBI?=
 =?utf-8?B?amk2cFRPTzgxMUNpbnFZRWhNQXVXczRFeHVVNjgveVFEQ24xUjF3OFdpSjc1?=
 =?utf-8?B?RWJOZDI0ZWVObGorbTNEeENtZ2pkLzNLejNjQW56MUVZUWhPR1R1V3djY0tM?=
 =?utf-8?B?dVJqWk5TaHJXbUQvdllIKzFxZEtDZHV1RmF2V2swNDljWXZRYTBwYVlKVUdL?=
 =?utf-8?B?SEhYS1p3cThScVJ2QWxRV1pTRUV1ZjMwa1IvcVdYcnpudjAzYUcwUjB6OGRr?=
 =?utf-8?B?emlIYmFzbTV6eG85bVNNOEtWZTBOZytXOGpKZEJzWDNWa0w2VlhyYlRncHpx?=
 =?utf-8?B?cldlNzNpdzhqY0oxbVVZOXRaMFFVWTUzOFAvcTVmV2g5MkM2Rk9iakZKRWkr?=
 =?utf-8?B?OXp3YVBvT3lrMUs4Q1gzQ01wQ21DTitBUkxBS0Q5S2ZSdTNuWjZET3FkZDV2?=
 =?utf-8?B?YzVzRHlwTGdoRnF1VVl4MFdHZU5EMG9DVnp0dXBIVHRabjdvR3FNcjZ2RVB3?=
 =?utf-8?B?dGdscjZqSUowSlV5alZKOFVNQTJnTEpkMjBIcUtZYTk2QUlHV2ZZSFB6blBq?=
 =?utf-8?B?dXFEdmVpbWRNUEtLek9OajJRNlYzekhPR1lPaUM0dlJpY3lkY1ZGY25tak0y?=
 =?utf-8?B?a2tXM2s1ZUs3MjZjR0E2bTlOMlhMWmMxeXlWMVBsNThka3VjVEU0L0JxejQr?=
 =?utf-8?B?ZEgzVGVkdDdJOHZKQ3kvcEZua2ZHU0ZzSHIxZ08xQm9yZUprUHl5QXpUdXVi?=
 =?utf-8?B?Rm9HZnVGZzFtR1VISGNrRzRuUmVrMVFOUTJPZGtVc1VBTWdrWXJuTzErYlZ2?=
 =?utf-8?B?NlFRTDdocGNaY0hmTkZKZ1F5aWhYNHVlZ1gwcW5pSklXdHh1S1FRQlEvQmw4?=
 =?utf-8?Q?YHSA9giPmNOhzj62C0kWcKMclvhR2k7N1iStJAQ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c925c99-869b-44cc-d280-08d97f40732f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:48:23.7903
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gxqJ3sjyiF3LvzhDjXITQDQjX3bVXyPdSskhWGKLxAIxRGIoCKTjrj8lpVCPKchXTBuMslIPosBDkB/is+Vnkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

For vendor specific code to support superpages we need to be able to
deal with a superpage mapping replacing an intermediate page table (or
hierarchy thereof). Consequently an iommu_alloc_pgtable() counterpart is
needed to free individual page tables while a domain is still alive.
Since the freeing needs to be deferred until after a suitable IOTLB
flush was performed, released page tables get queued for processing by a
tasklet.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I was considering whether to use a softirq-taklet instead. This would
have the benefit of avoiding extra scheduling operations, but come with
the risk of the freeing happening prematurely because of a
process_pending_softirqs() somewhere.

--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -12,6 +12,7 @@
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/cpu.h>
 #include <xen/sched.h>
 #include <xen/iommu.h>
 #include <xen/paging.h>
@@ -463,6 +464,85 @@ struct page_info *iommu_alloc_pgtable(st
     return pg;
 }
 
+/*
+ * Intermediate page tables which get replaced by large pages may only be
+ * freed after a suitable IOTLB flush. Hence such pages get queued on a
+ * per-CPU list, with a per-CPU tasklet processing the list on the assumption
+ * that the necessary IOTLB flush will have occurred by the time tasklets get
+ * to run. (List and tasklet being per-CPU has the benefit of accesses not
+ * requiring any locking.)
+ */
+static DEFINE_PER_CPU(struct page_list_head, free_pgt_list);
+static DEFINE_PER_CPU(struct tasklet, free_pgt_tasklet);
+
+static void free_queued_pgtables(void *arg)
+{
+    struct page_list_head *list = arg;
+    struct page_info *pg;
+
+    while ( (pg = page_list_remove_head(list)) )
+        free_domheap_page(pg);
+}
+
+void iommu_queue_free_pgtable(struct domain *d, struct page_info *pg)
+{
+    struct domain_iommu *hd = dom_iommu(d);
+    unsigned int cpu = smp_processor_id();
+
+    spin_lock(&hd->arch.pgtables.lock);
+    page_list_del(pg, &hd->arch.pgtables.list);
+    spin_unlock(&hd->arch.pgtables.lock);
+
+    page_list_add_tail(pg, &per_cpu(free_pgt_list, cpu));
+
+    tasklet_schedule(&per_cpu(free_pgt_tasklet, cpu));
+}
+
+static int cpu_callback(
+    struct notifier_block *nfb, unsigned long action, void *hcpu)
+{
+    unsigned int cpu = (unsigned long)hcpu;
+    struct page_list_head *list = &per_cpu(free_pgt_list, cpu);
+    struct tasklet *tasklet = &per_cpu(free_pgt_tasklet, cpu);
+
+    switch ( action )
+    {
+    case CPU_DOWN_PREPARE:
+        tasklet_kill(tasklet);
+        break;
+
+    case CPU_DEAD:
+        page_list_splice(list, &this_cpu(free_pgt_list));
+        INIT_PAGE_LIST_HEAD(list);
+        tasklet_schedule(&this_cpu(free_pgt_tasklet));
+        break;
+
+    case CPU_UP_PREPARE:
+    case CPU_DOWN_FAILED:
+        tasklet_init(tasklet, free_queued_pgtables, list);
+        break;
+    }
+
+    return NOTIFY_DONE;
+}
+
+static struct notifier_block cpu_nfb = {
+    .notifier_call = cpu_callback,
+};
+
+static int __init bsp_init(void)
+{
+    if ( iommu_enabled )
+    {
+        cpu_callback(&cpu_nfb, CPU_UP_PREPARE,
+                     (void *)(unsigned long)smp_processor_id());
+        register_cpu_notifier(&cpu_nfb);
+    }
+
+    return 0;
+}
+presmp_initcall(bsp_init);
+
 bool arch_iommu_use_permitted(const struct domain *d)
 {
     /*
--- a/xen/include/asm-x86/iommu.h
+++ b/xen/include/asm-x86/iommu.h
@@ -143,6 +143,7 @@ int pi_update_irte(const struct pi_desc
 
 int __must_check iommu_free_pgtables(struct domain *d);
 struct page_info *__must_check iommu_alloc_pgtable(struct domain *d);
+void iommu_queue_free_pgtable(struct domain *d, struct page_info *pg);
 
 #endif /* !__ARCH_X86_IOMMU_H__ */
 /*



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:49:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:49:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195122.347653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThpI-0000zO-Qw; Fri, 24 Sep 2021 09:49:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195122.347653; Fri, 24 Sep 2021 09:49:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThpI-0000zH-MS; Fri, 24 Sep 2021 09:49:04 +0000
Received: by outflank-mailman (input) for mailman id 195122;
 Fri, 24 Sep 2021 09:49:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThpH-0000z6-Bk
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:49:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 34e2868e-edc8-4611-bdb0-d85d1b12a56b;
 Fri, 24 Sep 2021 09:49:02 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2053.outbound.protection.outlook.com [104.47.1.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-knYvFKPcPPuzKwAHNz8SYQ-1; Fri, 24 Sep 2021 11:49:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4383.eurprd04.prod.outlook.com (2603:10a6:803:6b::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:48:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:48:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0015.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 09:48:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34e2868e-edc8-4611-bdb0-d85d1b12a56b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632476941;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YHr+CH5Pz0j1yX5jp8hrrxKCetSkHo6HQeq3jEu7QTM=;
	b=H4RlaSok+NWd2mnH+/4nQsDa/EFGgNFFrXIOZdp2C/Lmv1fYmEHwxpCQaHwFkY1NaHY79D
	UL9l4T3eUqjSDBXM7GNsBkhGf7XZDG8XoFvYNzYncGdU83PcigB2o3WQdxWK73jQsHKQ8n
	UXzyrbTAioAKBOkU38ijnIe5gDZEmOU=
X-MC-Unique: knYvFKPcPPuzKwAHNz8SYQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ie9SxD5ecB6XO2f1xNO4xbXITdouMtirBdeNhfNejrfO0wZT56BB7wBZHFgcUx5w+PZlHVsXU0aoptHQ6hmVpK+D/HqvzIlBtrKZofgJP/DmHFGhhGhO0+oB656ZZw1xs4hJH8hLVbR+/XY7gusME6BVyl/L54FvI+UCsGWR0phxk66yrYx0DEBbjIw0If4GJQR/xhCnwBCnMvrZ7O6gF+SegO94sYLMO2xJU16WHspIoqQ09l6ajsWXWzC/ApnFm9do5EWfyWoCz/nRsrUgBSys3iIXP8PMGVkndhyj/1sY8mkQ1r7aH25AKQJ8kyHrps9MXJxsse7Kz98Y9FYaKQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YHr+CH5Pz0j1yX5jp8hrrxKCetSkHo6HQeq3jEu7QTM=;
 b=I0doe435a5+KyFpwAFi4r8rGn1Vtu5OXJ+CNmm0Iph50qCR3M+i9kYb/rQ4xd+Qt+AOFE/eSfrcomMBPDysMMEsFe5d+NQBLxoMb7eKcCh3r4ADV14PzciH0ydXGQCA/J2mzK9PGouyAHLZOBv5miLlji8GNH8We2YNBFKZwStfq22uhgMFsMPgo6TVwHr6g7Pc+XjvXocgW/goBpmDJl7IeYESowhmZlw7AKZMW63Zfa8WtF0qg5oaaciwhxxo6HcMNF6HP2Xo//lHhHTcSHjReDxNL3OKpL+9AFWNw/7HYjBy9oxdPLNjaQPbn9PIBIPHIIUp6gNBX6903gM7JsA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 09/18] AMD/IOMMU: drop stray TLB flush
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <6d825fc7-d600-6759-5bc2-ab8b17e4033b@suse.com>
Date: Fri, 24 Sep 2021 11:48:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0015.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 699ce415-eff7-45cd-7d45-08d97f408877
X-MS-TrafficTypeDiagnostic: VI1PR04MB4383:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4383D1F8BF2D3E7335DF15F2B3A49@VI1PR04MB4383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ui0APsBmqrSL6p0QicK29x2jrKvGyP4f2042V+nx86DTiOHQnHwQ+4zPqfxBWrRqgjdFwN3MlFrt1RXHLuPBY2hYCce1ki6Ytiz23wvYwAsyijq9Ftp1IEOVev3twqRxR3RHakhuJ+J7rahVzu9APyDrKmK+pJd8bY2w63HvA7VdZ3YlZKHvdQ3m7ZsukspF+9PMgOHgn7a2HrNCkZRIY3f/Ao+OpXNZoupqHw+aTZLNjYdcJ/K7DDoHNqsE5JtE4rYFezXSilPWsYoSzrz9keSkkmqmxsU4A7U8YR9f5ty1+9D9k3wD2v7pNOpJuxfVfOMbZgRA28q4PlAxR/ui7j64CCpKwXmKeWY4enBdo7P/bY0pv1VK72mdlFNmmqoNW8erPfn2lgwXyEqw4u7SrMblWNnnjr7tmww98Nw2D+PNfHg/BjUkUW2QkiZnJTnQ0dNpYQ1YZ3MiqKeN/rXr/SQWTb6ffsdiicfg8tzKGRu6GAmtmKay937AbTjmHOaH2Ex3u7ZHoMZsV97qFgE7ra532b35FQCR8JFF6j6YYvYGZjfc4E5kne0fvc9qn69ngQQRvE2Xc9PrYBknGvRpufC59nFzHuCcsz5VRKdtD4wHNNrcojLKejo8FoqKaSh++mMebuyaDtKuj8zpx8HxlJ+oTv32m2qNuC1uW8n0cKoutasKl3a9Av9xIXyRGKNjXVvHlBOzeVjQBUdbTfGzeZJnjPVb83AyAtjLb6dCDdg=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(508600001)(31686004)(8676002)(16576012)(38100700002)(83380400001)(54906003)(26005)(36756003)(66946007)(31696002)(2906002)(186003)(66476007)(2616005)(66556008)(86362001)(956004)(6916009)(8936002)(316002)(4326008)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bnduOG9EMzVyWTFlclF5Y3V4amVLb3hkVkk5b2dOZzZ0YVRXZzZ4a2FKRThQ?=
 =?utf-8?B?SDFudVgyVlRpYmI1Zm84RllnZmtGUHE1Qk51RHgxZWdCL28vL0M4V2xJUCtl?=
 =?utf-8?B?MVBFK21FQXd0RXRqSzg4eTduU1p3a2lveTA1bnEyY1U1M2ZmZlpXUDdmVHFD?=
 =?utf-8?B?enBlc1FJNVBnZ3gzVjg1cTFmRTJJZTFGNVJDZC9ZM2dpdkN0Z1pSNGtyTmht?=
 =?utf-8?B?akFkSkJWMExQbnRRd0x4eEhKa3dNWllYeEZYY0tGbndyMHlwL3pDdGZ5UHEx?=
 =?utf-8?B?MHFTMjdsd3Zza05NZS9sSTM5aGFDTnFTWENJR3lBbHVKS3Q0aHo2OVhZblNI?=
 =?utf-8?B?U1RGcDdJWnh6dzlyNTliYVJjRWxNclh3aDFXN09DSmxqRlkzQWpIRnMxMW1G?=
 =?utf-8?B?TjNRbTFUUG9SNlo2UkNEZ2pldXFJRlFOMmN4ZlB5cGJ1eDhNclA1c1ExZFdp?=
 =?utf-8?B?RlcySExBdGNMdVEraXRpSjd5MThCczJKTW5CZE5ST2FvbEZVaVdkZ0Jtc2dx?=
 =?utf-8?B?SmRROWlqaWdqVEEzTXlneEZyMlZ2eEk1M3paK3dXdDFjbVM5YlJ1QTZZSWxi?=
 =?utf-8?B?VUtiaE5YVWhKdzVsajlnajFsSlVReXdxVm9kek1XeGpQdUxYNVBtV3FYcjJ6?=
 =?utf-8?B?R3ZMVS9pYTV2eEpuOEdnREM0anRibFliU3drS1VzRm42clI1R09KMG9ydkJC?=
 =?utf-8?B?YlNxVXl6TTcvbVE5TEJmMXYvUVFyYi9UTmhUMkVvV0RLMHFaZ0ZiTk1DaTd6?=
 =?utf-8?B?R3lDN09lcXdsSlBqYkpTWnB0b2dPVlg3K2lacDZzSm1WU0dMVlVVS1FObjJR?=
 =?utf-8?B?M3BETDNqOXltekZXVWdBZXB4VnkwNmtPV2d3blJ0ejNYSzJJMkMyaWRzTnNT?=
 =?utf-8?B?bU1IdGpybzhxeEFXeU12ZmQvM0h4L0hzekY0YTRmYlpMYjJIS3Q3Tnlnb0hP?=
 =?utf-8?B?aXBXWmxsZDlZNFRCa2c1N3lKWTBpMmlMZC9lcSs1REpNdDNGZU1ma1VwTG1q?=
 =?utf-8?B?WVRFQkxkN1dPYWNqQnhncVVzUUo4ZEVjQXJ2RTlQZHI3N2RLYzBTLzFLeTF6?=
 =?utf-8?B?ZnFZT1NMNGRoZFpQNGZjNm1WUGZyU0w4QXdKbkMxN1g5ZnBxemJxaU8zc0x1?=
 =?utf-8?B?WndDQnZFWVp4Nkl1Mm1pS0lQeUhyRVZCdEpxNUloUWd0UExKeEtwNFhocWJV?=
 =?utf-8?B?NG5ETnUwVXRLS3psaUhVNXZaZGZRNEorSnlCMGM4WC9ta0FZWjFQNXowdkdK?=
 =?utf-8?B?L2RJaEN5YWs0cmZJZzlhREs2K1hHamN1RkFZMFB2aHp6NFpNVEdtMlVkOHZ2?=
 =?utf-8?B?aXR0RlhOREowQjFZOVgyb2xtU2ZtUTlGTE9CMWNWOWxJU0VubkVPc1pDOTgz?=
 =?utf-8?B?bEpFYWx1OWd0QXBlN2ZHRStyQUlicmJFOU1xbUsybmxlLzIwTEhxQ296UmNO?=
 =?utf-8?B?TXVsVWkrSlFRN0dYUmQvUmhuUWEzdkwrSllzTTh4dG9BcnhmRUxtL2lqOCtV?=
 =?utf-8?B?M3BGUk9YVHBpMUZzK1crM3d6N2laVUpVNUtuYjVYRS9OamVWTlRZVGtWNCtF?=
 =?utf-8?B?ZEN6d1hOMlpWcFl0QlgwcisrcUhqVytId1hDTUdNeTJJMFlPVWY0ZHBUbUFx?=
 =?utf-8?B?a1RubFBYNTFTeXRHUWVldGtsTEtsQ1pSN21vT0xUY3FnZS9saDZwelFidmgz?=
 =?utf-8?B?WExLTjRwRHZNSjE0QkVDdzQ2aWQyem8zdE8yTDFFUTQ2K0Rwd3J1WXA3V0d0?=
 =?utf-8?Q?Ksb6LR+DK2EttLnmbFJ4vuKCZUaYAfJympo1g95?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 699ce415-eff7-45cd-7d45-08d97f408877
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:48:59.5421
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yG8po0PC8Zdu0M6Wyc4c1WwPes3P4NUP2qR5LlShPUixxjY3NPlySaduZK5ceaSCrbfnpCrs8S9ai+4csLUDog==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4383

I think this flush was overlooked when flushing was moved out of the
core (un)mapping functions. The flush the caller is required to invoke
anyway will satisfy the needs resulting from the splitting of a
superpage.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -179,7 +179,7 @@ void __init iommu_dte_add_device_entry(s
  */
 static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
                               unsigned int target, unsigned long *pt_mfn,
-                              bool map)
+                              unsigned int *flush_flags, bool map)
 {
     union amd_iommu_pte *pde, *next_table_vaddr;
     unsigned long  next_table_mfn;
@@ -237,7 +237,7 @@ static int iommu_pde_from_dfn(struct dom
             set_iommu_pde_present(pde, next_table_mfn, next_level, true,
                                   true);
 
-            amd_iommu_flush_all_pages(d);
+            *flush_flags |= IOMMU_FLUSHF_modified;
         }
 
         /* Install lower level page table for non-present entries */
@@ -309,7 +309,8 @@ int amd_iommu_map_page(struct domain *d,
         return rc;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, true) || !pt_mfn )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, true) ||
+         !pt_mfn )
     {
         spin_unlock(&hd->arch.mapping_lock);
         AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
@@ -342,7 +343,7 @@ int amd_iommu_unmap_page(struct domain *
         return 0;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, false) )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, false) )
     {
         spin_unlock(&hd->arch.mapping_lock);
         AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:49:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:49:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195126.347664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThpi-0001ZO-2Z; Fri, 24 Sep 2021 09:49:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195126.347664; Fri, 24 Sep 2021 09:49:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThph-0001ZH-Vh; Fri, 24 Sep 2021 09:49:29 +0000
Received: by outflank-mailman (input) for mailman id 195126;
 Fri, 24 Sep 2021 09:49:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mThpg-0001Yt-9f
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:49:28 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 09f3a1f3-f912-4d6f-bbe5-e1fcbf8a522d;
 Fri, 24 Sep 2021 09:49:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09f3a1f3-f912-4d6f-bbe5-e1fcbf8a522d
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632476967;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=rbwANCvizQntMTQB13f95Hp0vttb28C69lgrf0KIdRc=;
  b=hZ8efAP9tfhQEltgvJgYYj+6K8Dl44iii+SHA4M8rzx1C9Ski/BbRRpB
   88qp8Yjs/Xm4au/CsX1qejTxngGWFsQXzPSj80lH3tEvKOZuCqlurfWn6
   y6tOJASowzyp/+6iyGT+2JvypRGa/SfaSZjm5yi3pyK+w/zfhHOebJgo9
   A=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 9bB0sOGC8/U61e/kpQPqlWaJTZs3EgXixKSKbP1h/cvKKVUyRT6c9jf4JEue/HZvv3jqf8cSyj
 mW5vfkYM/BweM13D9ea5ZoHUz139sCfwbp3tiDCn4SdTncCKbjjFZDDfXagJO5NhkygL89gxu4
 W8gEmGMfqzNeqdK06Mk2I+pzbJjXgZ7A9X5EvtDIYYGHwZKXUCCOIw93x1+BcsfcevOL1Xvvwa
 Tl+y5XWcDQhwLVy73WRXASL9AUBjTM8gRnXO5hMUpBk0ax4kurJy4rwMcJCYzis4ElpIVI++Hg
 6ANb/W0giWFMG0InaeQE85s2
X-SBRS: 5.1
X-MesageID: 53122381
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:d84Lz6N9qeiaszfvrR1PkMFynXyQoLVcMsEvi/4bfWQNrUp2gjFUz
 2cdWzvSO/eKamukfo0lPIrl/R9QsZDXmoRiSAto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Eg/w7dRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYozPOoN1vy
 vd9jKS1eTkTB7Ljku8YcBYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YvNrick5atHiIasUu216zCGfBvEjKXzGa/iRvIUCh2dp7ixINdD3f
 +sQUiB/VxWebD5sKGgrEpAhls790xETdBUH8QnI9MLb+VP7zRNz+KjgNsLPfd6HTtkTmVyXz
 krZ+0zpDxdcM8aQoRKV/3TpiuLRkCfTXIMJCKb+5vNsmEeUxGEYFFsRT1TTifuzh1O6WtlfA
 1cJ4Sdopq83nHFHVfGkAUf++iTd+EdBBZwASIXW9T1h1ILSpDmUGGomfgd5bewYjvYpajYh0
 3yGyoaB6SNUjJWZTneU97GxpDy0ODQIIWJqWRLoXTfp8PG4/9pu1EynosJLVffv14yoRWiYL
 yWi8XBm74j/m/LnwElSEbrvuDuqupGBZQo8/Ay/somNv14hOdbNi2BF7zHmARd8wGSxEgXpU
 JsswZH2AAUy4XelznflrAIlRu3B2hp9GGeA6WOD5rF4n9hXx1atfJpL/BZ1L1pzP8APdFfBO
 RGI418LvsIMYSf2N8ebhr5d7exwlsAM8vy/CpjpgidmOMAtJGdrAgk0DaJv44wduBd1yvxuU
 XtqWc2tEWwbGcxaIMmeHY8gPUsQ7nlmnwv7HMmjpzz+iOb2TCPFGN8tbQrVBshkvfzsnekg2
 4sGXyd8404EC7OWj+i+2dN7EG3m2lBhXs2p95wLKLTTSuekcUl4Y8LsLXoaU9UNt4xel/vS/
 2H7XUldyVHlgmbAJxnMYXdmAI4Dl74kxZ7iFSBzb1uuxVY5ZoOjsPUWe5ctJOF1/+1/1/9kC
 fICfpzYUPhITz3G/RUbbIX889M+JEj621rWMnr3eiU7cr5hWxfNpo3ucDzw+XRcFSGwr8Y//
 eGtj1uJXZoZSg1+J8/Kc/bznUiptH0QlbsqDUvFK9VeYmv2941uJ3Cjh/M7OZhUex7C2iGbx
 0CdBhJB/bvBpIo88d/og6GYrtj2T7siTxQCR2SCtOS4LyjX+Gan0LRsaufQcGCPTn7w9YWje
 f5Rk6P2PsoYkQsYqIF7Cbtqk/4zvoO9u79Aww14N3zXdFD3WKh4K3yL0MQT5K1AwrhV5Vm/V
 k6Vo4QIPLyIPIXuEUILJRpjZeOGjKlGlj7X5PUzAUP7+C4oo+bXDRQMZ0GB2H5HMb94EII52
 uNw6scZ5ju2hgcuLtvb3Dtf8H6BLyBYXqgq3n3A7FQHVub/Jol+XKHh
IronPort-HdrOrdr: A9a23:Cc+IfKyxuR4omU8qADo2KrPxveskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9wYhAdcdDpAtjmfZr5z+8O3WB3B8beYOCGghrSEGgG1+XfKlLbak/DH4JmpM
 Jdmu1FeaHN5DtB/LfHCWuDYq8dKbC8mcjC74eurAYccegpUdAZ0+4QMHfkLqQcfnghOXNWLu
 v52iIRzADQBkj/I/7LS0UtbqzmnZnmhZjmaRkJC1oO7xSPtyqh7PrfHwKD1hkTfjtTyfN6mF
 K13DDR1+GGibWW2xXc32jc49B/n8bg8MJKAIiphtIOIjvhpw60bMBKWqGEvhoyvOazgWxa3O
 XkklMFBYBe+nnRdma6rV/E3BTh6i8n7zvYxVqRkRLY0ITEbQN/L/AEqZNScxPf5UZllsp7yr
 h302WQsIcSJQ/cnQzmjuK4Fy1Cpw6Rmz4PgOQTh3tQXc81c7lKt7ES+0tTDdMpAD/60oY6C+
 NjZfuspcq+SWnqLUwxg1MfheBFBh8Ib1O7qwk5y4KoOgFt7TNEJxBy/r1Zop8CnKhNAqWsqd
 60dJiBOdl1P7srhJlGdZU8qP2MexrwqCL3QRGvyGvcZdQ60lL22tXKCeYOlauXkKJh9upEpH
 2GaiIAiVIP
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="53122381"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=S1DpJ0UYQll5O40jQoweMFmhR7K7J4Sw9h8lDYhnxKYMevakd0F8ZkPHfhwytOngzFfnvQCEwfn+TxKK8jZkKFhzQfHM/Sws+hL8Ndjso1iOZGTCmiRu51eRKBsrSHvnEvqfGUVjS3wLyx7zaRo/R4XWx8cPCOdEa9MOtFGlTIWrg+4E36GqOHJpAtI//PxGuTGH0H/ToIFDYSZXR+JTIob0gXeaop+VJCvX7PIXe8wK7CBxoPQhxlBXzijPfYuhko89I/tp5v0IhpOKoHm0AcawW1VDA7UT5XXzGmUe/v4ZGTDRu/FouNBuDCYwaBzE06ioSZ1i1NbMQ9Cay/3e+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V0DzXFN+t1jwWMbnCdqzLLN0qwlkJwPhGT2ZBTZkP8U=;
 b=Np+8i1KcS+wsVeh83yjMxFW954MTY8ZL8SNggxPT9cnrUhdd75bHpsm7RIRa01ssJGHCTmdOmSPy20ZaTxk5efTSEJjoFLJABfd3+w3UD3YtvhR2EybJo1FyglgRDj5Oj7f0my7QvzIuzjVqTYtFqdhFnKMfwpIbK6cZzepCgryWbVdTwlsbjxW/BsHA1i37ktKpNgDYyzfVw5XPF/bR71Cy0s2FYF9GVMZOT7KronSFZ5GtEE7FVBOoLPhF2skkfpxtO2JmHKJyHF2NICU5jNZNaK+4xGehrXRw2491E9eqHXb/ggLSegJ5eHfmhUGMWpnqv1BncPXyTHxXIRYsXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=V0DzXFN+t1jwWMbnCdqzLLN0qwlkJwPhGT2ZBTZkP8U=;
 b=ZAlMQyDn8UpRhzuDex3xw3ko1fVkcRS0BrbVJmjYyK3vwmTygFrKwmt7QG3oSVs4EsgMTCXcDR625qkbN7iPARF3SNoASCGYKgGzDM4onWAnKU5kz9kWOgRWP156PBBMYoa66aDb1+lAZ7JDvpXmZ4jKRo3vbYcUhuaEUssJ8/U=
Date: Fri, 24 Sep 2021 11:49:18 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Kevin Stefanov <kevin.stefanov@citrix.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>
Subject: Re: [PATCH v3] tools/libxl: Correctly align the ACPI tables
Message-ID: <YU2fHhA+SeE8lgVi@MacBook-Air-de-Roger.local>
References: <20210915143000.36353-1-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210915143000.36353-1-kevin.stefanov@citrix.com>
X-ClientProxiedBy: LO2P265CA0343.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:d::19) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ac561020-7412-4a5d-b9a8-08d97f40973b
X-MS-TrafficTypeDiagnostic: DM6PR03MB4971:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB4971F6C316C8B468B900DA998FA49@DM6PR03MB4971.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: R+K5faII3vJ/b03t64srAjrCh8rzJk+NkheKQLdrfOfCRKaBngqpo+9HeruxsfepG+6oe/AlqIMn8DD8BT+ydi3+Eovzh4u0CJ5WRKklKIyFx5Dn/zuYN/UKq7kTE7OfKPj6wmlCvY2c/2h7p7Fz3se9eDZeJAe6Yrc+/dTCtwQLvWaYBDnuagNJ4FtAlpYKNOQ5s66q8I2FQflIQq1aZLvR9eMIUNTe6XuIb17J7D6O0TNElXta7K7crO7UEGOrET1ydm4X/MnJSYClIf0bYOBgda4LegC8H8uhGmgiddgy+Ky9Ki65pNzys2ifOWBo2CZLjKOjTRDMubgLV1wdOEwNz8Rjxyy5rRwitOSmBeF8mBDsnthXPn/ZUHtMgx53czzyaZtXiH0fBNCud10nrLI5wzvDDWc5WQbD1adimTG7YPiAi/d9dv4kxmMkIlUxYeJ/Yk9AX/wqBhlE0tXAV3jLOWrx3sWgebg/KMRr/qMpkVJOVRAn9wCX7ov4BmGAPrU2SOMOGbxbBypsqSR1u22DxMSexZyfrqOtjtpziAj+d5hvzj55ySLbMq/VV0YkbFZDay9sVeBBSxutnUNouGvFG1MbCle99cM/SPo555lV0fqFxYDQty2okyKE8dyJukvB42YEgVhqE0BNKM9f4AnBQGFFc8FWVd5Z1d2Z8gWB5/otmN0hJMFKpKOhhfQTzHocQUFWITarCIAjAOGLJg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(186003)(956004)(5660300002)(6486002)(83380400001)(54906003)(2906002)(26005)(107886003)(9686003)(6496006)(6862004)(66946007)(6636002)(66476007)(66556008)(4326008)(38100700002)(6666004)(8676002)(316002)(508600001)(8936002)(85182001)(67856001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R3V1eGNMQklwVUF0RlVFMkloVWgwaTdPU0VZcFp2d0hncWVYTjV5c0xSTlY5?=
 =?utf-8?B?WjhMVnF3dUxLNlQ2bVY4UkdFRE1UdWNYKzl4RkFCL00zdlhTZ3Zoclh3ZkF6?=
 =?utf-8?B?UVVLOWNuZDAzRm1mRE15R3RMNi92U2p4YTFWQ0tPbVNiYllkRFIxUm1TMk4v?=
 =?utf-8?B?K25IYlp1RFgwR1Mwb0JMSERPSTJ6ejRQTUptcGdnUFY5blhQQk9kNDFJY3RZ?=
 =?utf-8?B?VWJvQk9QS0xCTHo4UXd0VDhENituN2t1cnhlQ05MOW5kN0tseS8rTGk4OHpY?=
 =?utf-8?B?OVJyLzd6YTUwOFMvWXh1OTQ1YjA5aFd3eU5rYTJmcGVkQmRrUzlqTENTWFlV?=
 =?utf-8?B?SzZkbDFRa1VzdWVjYTE3MlpLMkVKc1JHZGduNzM4eGoydVU4N28yU0crMXpY?=
 =?utf-8?B?R21XQUtXSFpSeDRmOEpmVjZwc0t4STNGWTE3T0xWVGVuTS92a084d0psSkxU?=
 =?utf-8?B?dXVreFpFaUhrdjh6NnJvNisrNU1zUHVDbmxOd2E4dGpzQ0g2dkZIL29UR2tM?=
 =?utf-8?B?ZlVSV2k2M2c1YXpOajJjeWpyb2hKOE1zbzdsR1pyMjduQ01FNU9lOHRCVUxv?=
 =?utf-8?B?cGhkb3M2TUw1WnN3L3czbkJHcjFYeUt6dDVLTXVkaWNGTXkzenBWcGpaU3Mr?=
 =?utf-8?B?NVZmbXdEeEhBcjc5azdpSTNzREErbkpGZ3pTUUZxZC9EYS9zN3M0akZRNHhi?=
 =?utf-8?B?eWNjZ3EyK1dwdmlJdmdGRGV0NmFyTWNZTWdxYThBOGU4M3RxbzBEUnFSNlZC?=
 =?utf-8?B?WEd2RktMOWpDTzFYK2hDYXhwbERpRGRhVWZHWTBBRkdqRmR1dkVWdGhleEdB?=
 =?utf-8?B?WGFYdzNsUjltbjhPczFVSEFzM1lDaGlWeDVEamRSSEtJaEFFT0tpbStGNXY3?=
 =?utf-8?B?RHFOWFFSTFNUZFExakgxbEFNM3h6QXJ0Tjk0eTBHN2Q0djRtajdKeHlpVS9x?=
 =?utf-8?B?Tlk2eTVmRjVqMlJsZitPaW5abndQbzdrVWNLSyt2TmczdkdxYjc4dUdoTUVi?=
 =?utf-8?B?SFlLcmdBVS9UVGc5SjJIa0RXakQ2elJlTTNOTzdhY054THRnZzF0OXA5TjI1?=
 =?utf-8?B?V1JYYVFaQVBDcncvc0xjTW5sRk1EdnJhejFpdGI0bFc2T25hQlRRb0V0MjRl?=
 =?utf-8?B?YnZ5cU1pSGlndHRrWFJhbVZXY25XZXA5L0xFc1FyT280MTZPaTh2K0FFTkZw?=
 =?utf-8?B?L0U0YUlWV3k2T0ZDRk1vcHc5UitDN0FYanYvbXhQZXhMNjdYOExsRGFYZnc1?=
 =?utf-8?B?bTM5QXFNMHZtNUV3T2s2OHluWDhMcHlGaW4yVnpMTHUxcUNRdW1VTy85YWNR?=
 =?utf-8?B?Rm5jOXprTGZUY094YXh5U3Mwc1NzWlVMQm9xTHRtMWhZTVJrNXpjZjZWdTJn?=
 =?utf-8?B?dWcxSXZGMUNTQ2dXMUdkTkR3WXUzeHljeUc0ajdwcEt0NU5sek5wcUtSTlo2?=
 =?utf-8?B?K2crNTl0cmpmbmd1SlUzWVpOQ2ZxRDVOSGg0MUJ1ckp4cWxqTWVjczBZRmNP?=
 =?utf-8?B?VStrRWZxbUZGRUJDT2lGRzgxd0R4VHJGK3lWQ09PazZNT3F5clpTWm9QQmNJ?=
 =?utf-8?B?UnlWY1pNRFVFM0VHL0VVbjB4K3UzQ3FJa0o0N0grSS9JK2p5a2pLQUt5cGpU?=
 =?utf-8?B?RHYzN0hiZHlWVERyM0NHSW02Vkt2aXpnRnpaNWVOS2IzWTR6T29RaGI3RVFI?=
 =?utf-8?B?bTVWSkRvUW96TVBrODV3SG8wcCtjNzM5dnF2aHEydVJKemI5cUMvekdWUk5I?=
 =?utf-8?Q?hrKYNzQC4O4xrhartSxwQmdOcBZYPnlmw+yxA+E?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ac561020-7412-4a5d-b9a8-08d97f40973b
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:49:24.3644
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Zsg+XGTZRWJiKMQwgULQiajF9toN9Y7NThvXrrN87J+afKC0smMR4PGBrLzCunVKQEz800E7XbnAU82t7Nr+hg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4971
X-OriginatorOrg: citrix.com

On Wed, Sep 15, 2021 at 03:30:00PM +0100, Kevin Stefanov wrote:
> The memory allocator currently calculates alignment in libxl's virtual
> address space, rather than guest physical address space. This results
> in the FACS being commonly misaligned.
> 
> Furthermore, the allocator has several other bugs.
> 
> The opencoded align-up calculation is currently susceptible to a bug
> that occurs in the corner case that the buffer is already aligned to
> begin with. In that case, an align-sized memory hole is introduced.
> 
> The while loop is dead logic because its effects are entirely and
> unconditionally overwritten immediately after it.
> 
> Rework the memory allocator to align in guest physical address space
> instead of libxl's virtual memory and improve the calculation, drop
> errant extra page in allocated buffer for ACPI tables, and give some
> of the variables better names/types.
> 
> Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:51:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:51:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195134.347675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThrW-00032b-Jg; Fri, 24 Sep 2021 09:51:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195134.347675; Fri, 24 Sep 2021 09:51:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThrW-00032U-FP; Fri, 24 Sep 2021 09:51:22 +0000
Received: by outflank-mailman (input) for mailman id 195134;
 Fri, 24 Sep 2021 09:51:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThrW-00032O-4E
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:51:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cb5b3791-959e-421c-ad86-365dbbe10941;
 Fri, 24 Sep 2021 09:51:21 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-G3Ndlw9nNqes4sh3LDKR_A-1; Fri, 24 Sep 2021 11:51:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2446.eurprd04.prod.outlook.com (2603:10a6:800:4e::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 09:51:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:51:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0076.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Fri, 24 Sep 2021 09:51:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb5b3791-959e-421c-ad86-365dbbe10941
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477080;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cN0XMYBu453TsEQycVn7JwMY6NZVxUsfyHNqqJKG/hA=;
	b=Idcapd28bJJSymNVxTDmifj9CKDoqSD1PqEl7GsKxWMziYtnqMp4ga/SKPAm96VZWSWlwT
	UTHCHGc5ya3CqQ+NUuD0PX2z2u4H5XvDL6cYxGpvRp3f7e1JID0+ivep23nRFC244zvg47
	12E/zP6Z7g6b20AE+edVMrTwcU/H0LI=
X-MC-Unique: G3Ndlw9nNqes4sh3LDKR_A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hjg14M4DWctNYQBNaBYJ2EPyWzRT0DAkpdbAk9NUwmKpBNTxX76gOkiG6TjejTfb2LMTtpmCuhriiy2MpCtB1ppfTUwUBWvjrb1UPTBmG+eNz760uLC8VNMODivoUDQWi9f9Rz2WIXV0x9VUVzXjmmcQA1e0pO9mG+9eT5Nv3xIbhRuQMbAUHpcjtqnhlxdcDy7BG9Ag9j0I37I2sAzz/HbK6QZp65ygzz6ueAKsYJ9ehCjlCyqUTY7lIho/tL+kmJP8yHrzTkAgP3xklsgFuDrg/7FFPZl+LeL3jJ0M5Q+wdqEmOg+QpXZDGvG6llVK1U+pLEiw9exlyViAXcGIVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cN0XMYBu453TsEQycVn7JwMY6NZVxUsfyHNqqJKG/hA=;
 b=gGd4jvbmvJ6mNlrkbHZVO7DWwhOIIuyY3oSfQQRkVh6tQPUjjCA4OQihcFjawXfSXQZylys/2LjcexE6irNoN60Iyf/xFGGbD13JBw7ygCvTFHB8VBBMNVawrCe072B10p6Owb5tz3vsTM2xoI/hrU0fsR29W5LNPTIqrXSprN6I2XQfjRizN+qsmqivG3XP/VI61qeiR9xpx6vTXmZpUPL9DOHr9VzAmuiWMIUboZ9I7+V+6I5VaSYqtzBr0MB9mtBPQxg/xYWzS+tpiHTS0GgVnwEw6gSLSaL1T24DTu6v1JJt1CMH02mU94+GisANRMcCiCIPpsdY+dy4Ka5Z2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 10/18] AMD/IOMMU: walk trees upon page fault
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <5d4a4cd8-ffb0-951a-c86d-98f659ab8d0b@suse.com>
Date: Fri, 24 Sep 2021 11:51:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0076.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d8386776-8669-4586-927a-08d97f40da44
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2446:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB244686214157309FD7E00932B3A49@VI1PR0401MB2446.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:989;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WRUZZmzOy58uOkgUYVtNIpTJSr/p6heiQQbsHlu8VZHcPPwStp/4HHb1L043TkmEawKrbze4N08bDTZRbQlVq8rQ7B38JzrfXRojYyFnqLqBeF5rp1lYVBYMPs2tnGdbzounO/OINZ6nJ4Pgp6DwIpczBA+YvRaFotV0uGigIIcwVekk15CmSyqzz1yZAnUZ2Vxrj6NUI6YmPAazKIi0X+82V1sq9vKLubz9rlBTSPPw7ApDETmM1Qb2+ioVN4w/4RylEJqcST1FoWH/a8E9C9h9QQbfwk20OPFOLUOkqZQKvKrF22zqyel413d//Lif0zbpOT5sE1oAR+6/KheGNmr6VNFNkmHLIvLVFq6XUz+/Z3iba++0MCtzwNPD8uwzm9l9bQG9i822868CFMpzXFqvMq/N/17LiNJNqn05yLJRo641W7zzENLn+BLCFCCPp2KyaHPDMEvHh54Wqu0CvrKMrQkLxBXWt7WyXSfo7WJJybICrCnxAUAduQVqf5+FfuGv+WxpHL0FTe1dDhqpLxm/ykdtPwkYOoGLhU1uwCSCHrue77alvP/l9XPx3hGI/bhYGen4ubEhfFYXlLs+oogYZIAt2yAQTCNeOpExqoeriWa1BKD3ziFfCoCRRPrBZy8TOuUczGkqvBUFobC0rIkhOHPMrepBIUKrKOQ/Kr4vjCKQhi+iJS6H1kK/o47LWq8GH/DTX7Tr4tYZHfTtruEqDJmNUOhIPXX+qUT+r4U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(956004)(4326008)(5660300002)(16576012)(6486002)(66946007)(186003)(2616005)(54906003)(6916009)(86362001)(8676002)(38100700002)(66476007)(36756003)(66556008)(31696002)(83380400001)(2906002)(26005)(31686004)(316002)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?N1d4UFlITU5WaFR1dUFXZXB4YkdHTmtuV2p2bHE5N2pkdDlPRzk0Y1h0MGxh?=
 =?utf-8?B?M09hUXRhakVOUFRsd2I4V0NndGVMWnR1dldnUm9MWnhPRWwvUVZ2amNKM3Zm?=
 =?utf-8?B?SHd6MlAxcHN4alF2ZEUzcEMydVB2Y091QURENGpjMkxxb0p6M2hMa0hLakJI?=
 =?utf-8?B?cE5yOFlCcmxSQWhqMXczeHcrVVdvMVNYQVNDM2hnbnRkVE5jQnJkeGQwTHlQ?=
 =?utf-8?B?ZlpaL3JhNnRDai9SK3o3Vk9sRXY0VVZHQ0RGTFdIOGtQUU8rSUxOSnliakph?=
 =?utf-8?B?MnVpVmVuM1JkQnBMS1RXM2haYzJsbHFJUUtRTlJ4VEIraytUTDNjWm54eHR4?=
 =?utf-8?B?QnZlWnlhdVkzWHFEZmx4amZKbXJoVDdqS1ZNWWhFSUg2dGVlVGVLK1labWx4?=
 =?utf-8?B?cXNzVlJSMElOVWpaTFFmL1J3ZWdkeXhNR3JBYTliaXRubTVlSVBoOGQyWXo3?=
 =?utf-8?B?ZHlMT2s3cUR3UnFOaVl4YnhoQUc4OEtjckEzMmtkZmZMbVZzaldKTW5vWUNX?=
 =?utf-8?B?YkFCSEl1d3BJMTdDT2o3UE9hU0lNKzM5Q2UrdTJjNnZaNFdLSmxzS3RQcTFj?=
 =?utf-8?B?TFFVVHJHay9tWVR2ZzJ6T0xDaTVvSW9TbEwzT2ducTl3SnlNWVNGek5nZjFL?=
 =?utf-8?B?VnI3bFNOd1J3ejVPQmNKMWZkbGFoOU5zczZVRS81aGVSajFKcTZmRWtzcFBQ?=
 =?utf-8?B?RkVUZmVqdW14SHluSjRUbG9SeHRUYU1kNlBtekM4enJkRUxYdWF3TzZ0NVB4?=
 =?utf-8?B?MllrcVpQbENDWUdmVFpoSVNQRXRXcGFEbHR4MHpta3JPbk5IRWs1SlZna1Vo?=
 =?utf-8?B?T0Q0OFJYV1FPR1VmRHFxZUhwWktsOWlibjZNczMxWXhkR3A5RFBzeWdHNlZ2?=
 =?utf-8?B?SkVuMStzWVJSTk5xdnkwS0Y0NWJBbld3b1pSWi9ydmJYNmJ0RHFMTTNVbEVk?=
 =?utf-8?B?ZmtCNEswZDF5YUpXZW4rUEVjRGk4NlJIa3VXYWJtSXZHWjVGMFMxN2Nmbm9Y?=
 =?utf-8?B?YnlVS2VEWjBhQ2x2a3YyaWhjZ3RLZ0lyY25wekMrcmRRTVlsWG1YU3JWVnBm?=
 =?utf-8?B?WkVyNUlrUkVuWTdwZ051ZTlMRTl3c0FRQUdPa29yYkd2L1BIVVNHWWY3b1Bp?=
 =?utf-8?B?K2ZLejRtd05FYStWZ0poemtYR1FsM09IS0NKaHNkL0lNM3g0ZVh1T0ZNUEtI?=
 =?utf-8?B?cXJXZzZqNG1JYlpXNVFyS3NidGE5UG5SaGQ2eFFPNU5GSjRLWVJONWNoR3Zq?=
 =?utf-8?B?aWVmNWQ4QWRhMzhUak9KV1AweGdQVHRmaGloYlhLMWtHc0VHcG82dkRJanlu?=
 =?utf-8?B?ak9iMHVJaTBGYUxrMEtqay9NWHNVZkx0bHBJK2NXcEZrQWxWTjRoMVdxbnJ4?=
 =?utf-8?B?a0tCSW81SU5ZSVY1WTczMHQ1YTJ5bDh3MU5XWmtXQ0t4THBrYkxhbC9TSW8z?=
 =?utf-8?B?T0NUMGlOU1dMSTFFblZqa2V3SnE4Z24xZmZqM3RkdU5pSm5WQW40V1o4eVZq?=
 =?utf-8?B?ZzdEVUFKN1FuZVVPWFJ0TGNYRWVLSlBOd20rckd0Nk55RFpBajlsc2Z3aE5B?=
 =?utf-8?B?T3dkMk5Ndlg4dkhaRXpja1hHTzNhME5RZmduN25nSjVEcGM0QzEzVUlzZUh3?=
 =?utf-8?B?SmJ5cVE0RCtSZ2dxMHVMNzBmYzdEMm9WMTRJa2ZaNnREWnhpRERsd0poM1g0?=
 =?utf-8?B?RG5oSGdYK0pYLys5ZFo0VmZHWHIrZDFlTXhrdzB2dkExMXFZQTdoWGQvMWNL?=
 =?utf-8?Q?wSD1ynKy4VZ9Kz5rzxCtggwa/EmnHWl9O+06Bon?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d8386776-8669-4586-927a-08d97f40da44
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:51:16.8176
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BwWmJ+NKXfNkaddDeoANaeE3eW+yshrPdeBiYKgQCKlUntGkhpe6gxZZXD3XPGAO5KInLU3hdk+FU7j11w/JNw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2446

This is to aid diagnosing issues and largely matches VT-d's behavior.
Since I'm adding permissions output here as well, take the opportunity
and also add their displaying to amd_dump_page_table_level().

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -243,6 +243,8 @@ int __must_check amd_iommu_flush_iotlb_p
                                              unsigned long page_count,
                                              unsigned int flush_flags);
 int __must_check amd_iommu_flush_iotlb_all(struct domain *d);
+void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id,
+                             dfn_t dfn);
 
 /* device table functions */
 int get_dma_requestor_id(uint16_t seg, uint16_t bdf);
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -573,6 +573,9 @@ static void parse_event_log_entry(struct
                (flags & 0x002) ? " NX" : "",
                (flags & 0x001) ? " GN" : "");
 
+        if ( iommu_verbose )
+            amd_iommu_print_entries(iommu, device_id, daddr_to_dfn(addr));
+
         for ( bdf = 0; bdf < ivrs_bdf_entries; bdf++ )
             if ( get_dma_requestor_id(iommu->seg, bdf) == device_id )
                 pci_check_disable_device(iommu->seg, PCI_BUS(bdf),
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -363,6 +363,50 @@ int amd_iommu_unmap_page(struct domain *
     return 0;
 }
 
+void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id,
+                             dfn_t dfn)
+{
+    mfn_t pt_mfn;
+    unsigned int level;
+    const struct amd_iommu_dte *dt = iommu->dev_table.buffer;
+
+    if ( !dt[dev_id].tv )
+    {
+        printk("%pp: no root\n", &PCI_SBDF2(iommu->seg, dev_id));
+        return;
+    }
+
+    pt_mfn = _mfn(dt[dev_id].pt_root);
+    level = dt[dev_id].paging_mode;
+    printk("%pp root @ %"PRI_mfn" (%u levels) dfn=%"PRI_dfn"\n",
+           &PCI_SBDF2(iommu->seg, dev_id), mfn_x(pt_mfn), level, dfn_x(dfn));
+
+    while ( level )
+    {
+        const union amd_iommu_pte *pt = map_domain_page(pt_mfn);
+        unsigned int idx = pfn_to_pde_idx(dfn_x(dfn), level);
+        union amd_iommu_pte pte = pt[idx];
+
+        unmap_domain_page(pt);
+
+        printk("  L%u[%03x] = %"PRIx64" %c%c\n", level, idx, pte.raw,
+               pte.pr ? pte.ir ? 'r' : '-' : 'n',
+               pte.pr ? pte.iw ? 'w' : '-' : 'p');
+
+        if ( !pte.pr )
+            break;
+
+        if ( pte.next_level >= level )
+        {
+            printk("  L%u[%03x]: next: %u\n", level, idx, pte.next_level);
+            break;
+        }
+
+        pt_mfn = _mfn(pte.mfn);
+        level = pte.next_level;
+    }
+}
+
 static unsigned long flush_count(unsigned long dfn, unsigned long page_count,
                                  unsigned int order)
 {
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -607,10 +607,11 @@ static void amd_dump_page_table_level(st
                 mfn_to_page(_mfn(pde->mfn)), pde->next_level,
                 address, indent + 1);
         else
-            printk("%*sdfn: %08lx  mfn: %08lx\n",
+            printk("%*sdfn: %08lx  mfn: %08lx  %c%c\n",
                    indent, "",
                    (unsigned long)PFN_DOWN(address),
-                   (unsigned long)PFN_DOWN(pfn_to_paddr(pde->mfn)));
+                   (unsigned long)PFN_DOWN(pfn_to_paddr(pde->mfn)),
+                   pde->ir ? 'r' : '-', pde->iw ? 'w' : '-');
     }
 
     unmap_domain_page(table_vaddr);



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:51:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:51:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195137.347685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThrv-0003YU-Qa; Fri, 24 Sep 2021 09:51:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195137.347685; Fri, 24 Sep 2021 09:51:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThrv-0003YM-Nl; Fri, 24 Sep 2021 09:51:47 +0000
Received: by outflank-mailman (input) for mailman id 195137;
 Fri, 24 Sep 2021 09:51:46 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThru-0003Uf-K6
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:51:46 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 069d156a-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:51:45 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2056.outbound.protection.outlook.com [104.47.12.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-e_Q9zH3ROgOp0QTUyJUQ1w-1; Fri, 24 Sep 2021 11:51:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2446.eurprd04.prod.outlook.com (2603:10a6:800:4e::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 09:51:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:51:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 09:51:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 069d156a-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477104;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=QnyJfGA//OwEpfHeLio8RibL7MiCX+yG5T2utpneOsU=;
	b=J61IVrvQSEPDXeQStfQ9dhPRWc9QFNwIOXcVosX4+w9nJ+sjgFg01MaMHX4NbtItrK+QWn
	H7un91cqGKIfcgvo7md3I2BBkSzsQaDMZF7By4u9pMUr2WIP19LUPlWJunQPyikWR95AjI
	3NGy0xHg69HuHvh+GwJjC1Iv3nxqHy4=
X-MC-Unique: e_Q9zH3ROgOp0QTUyJUQ1w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bkUnLuMKY5cejw1hi8vSj/drZOWPMkwDuy+R7RHBiY58f7nZZuP9SfRb7RA26O4cxNy+3/g8sl/fo4AV8/Mc/zdQKa0N/uypQq3kC4DvF/11xrvO+eS1CvVTcJQna/gfVOsx/y5SBo6pVVAh5ONzPfSibMUPi5GV4pF/XqR7Qp7VzVoZnnpb5bC+Cc//lhwnba62fk1Jf0bnLS1D5et4h/1c72IzpzXx9XzdF40et2TFQlkFWdtUFjN+sZluTF40erLk5jWTKXnmYvpLwGRMckfyfj+zK5NK32Oyjac7A161ItzRVq+orGEJwGj19Yc7zt48HFYgWlZXzQ1eUgcL4Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QnyJfGA//OwEpfHeLio8RibL7MiCX+yG5T2utpneOsU=;
 b=a9Vly0idnV/kxTbee7tSV/ocW0qYSpN0r4nhV9FrhNtGDELNVkM9/rY8mIs0GNQLR7DlyPxAWWC0HZIgLMS1FiqP6z6NkzvYloMq8crm4x067RhYfjoI5lgOLfKkyNbW7VDaUfpvsuzXG8D93nECP+GjouWPZBV16zUspDsL5wcs4P21HrRjoU/jEzCfBJUhxQsALBYBVy7c1Wff6Gv31KqZnNexpbVFcJa+uA8+SAGKfH1Q8EQdtHEsGiuEfJF9wrrGwhfgbpG5OhL3I/APXaLGJK1NNw7Q9NRcxMsj4CxhIgmotLdfaOBZWTGNRHZvu49lxiWZLBhzP1b3bSFk8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 11/18] AMD/IOMMU: return old PTE from
 {set,clear}_iommu_pte_present()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <33819694-2787-4bd6-1830-6b4eb81cb4a1@suse.com>
Date: Fri, 24 Sep 2021 11:51:40 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 38bf7131-378d-40b3-c1d6-08d97f40e937
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2446:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2446ECD9FE2262CB99D3BF1BB3A49@VI1PR0401MB2446.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	DecMR9iCG+UI9Tw/ozLV3/S484/KUAtvflLntrE2LElkMl3UUYmz18+/kHOUn/Wbt39pkkznu4R9exBavylKjzKVuzLN2TQmXfj9Yvk+gI3xqPUlrGUGkWwLEJ8dfy6LK/QCSzZYKXL9QvVV1nscE0KAMC7JGxIIZuZnImiAg+rWkQhJ+bOC2kP/qP2MdWTFXYUqep37UJ1KbaSKSqqcD1dD9g8od5AvuGtGXEe/81mRlvzYYC0ADKi0wOwhjeE3YSAYpMYTxevVBKeqrg9fsrclF+49Vt9HlueZ638yVelYqpMbDNbz8vOPa1leKVS47Lwbxb0qMQX5BCvAkP+nxBIIMf8dB3UyIEVnv66UYxR1Kzf/WaFpGTWsvhngnaHA60n6ohOwSf5cYFmvUOCBiiLMpUMoGfG07DGdmgZsL8yTRs7Cd8T+g0ISEAuKoUqGpECOsjFGSyeZJh+q/gqKSy1iAakrwqvlGLcN/UJEpTL2vTRDATwDp8SXwIyoccU4+3Wra1ked1/0cecR9T1q8vaF0NkyiB+VPlgL2S69NhgAsIBujypz6SWdAc7DORdCoQkpEVslHcRswMGezq31KuZiPrUMFK3LHThGyck8yT45M6cL63qYA+GIPohyBIiToTAuA3mQM1HMe5rZSVO3tKzo2xgN44uday/iB7JfqKYoZ/08nrQW85cxXHPR+k354JTeO+BiiNMcPcdzPsCmDo6mhyWiF10J4RefDc/o+ic=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(956004)(4326008)(5660300002)(16576012)(6486002)(66946007)(186003)(2616005)(54906003)(6916009)(86362001)(8676002)(38100700002)(66476007)(36756003)(66556008)(31696002)(83380400001)(2906002)(26005)(31686004)(316002)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L1hiUzNuTVJCcGNVbFpjZC9iUnlWMjRxd3pGTzF4ajNvU1JCbHUybkVOeWE1?=
 =?utf-8?B?V05mR2lSM3RmUjJuZlZNbnVTSHlvSTdiNFd6MnowSndmTjJTTjRtL2Jrb2di?=
 =?utf-8?B?K1ArOGlUVXQ4bVhVdVl6QVVacXJwTGw0TDFXaFJjcGhuUEgwMldZTHNJZ0hw?=
 =?utf-8?B?UW1qUGlMR0hNZExvTGpBYXRJa0JCL3ZtM3RBK2lQellxTkZKMmlXb2RmTFFo?=
 =?utf-8?B?d3RDSlNTcUJkN1Z6c1hLaEdwT0lyMnV3QThodWdaMFA5NGNCQnpmdllHd1I4?=
 =?utf-8?B?S0pmbEhyd3BGdEdMSmhVcFEzZlFsczlKOGJPNTB1c0ZtN3VvOHdnOXhEY3B2?=
 =?utf-8?B?SU9yV0NPUnE0QVcxeU96K0lIUUhkN0hZaFJjTy9zclVJV2wyYno1MWtqZCtU?=
 =?utf-8?B?bFhINk9ieU1kSWdmZGloUndIU0oxY0tPU3JJVUJYdjJkNkE5dWtkVWUvUzJx?=
 =?utf-8?B?OUFZN3hYYzduT1cxWTRFdGdHSmFHQVliMGM1azMwaWROaU1ZTEdlQjJGT1g2?=
 =?utf-8?B?aThiOGhmdlF5NG5WK0haMHRFaUZUUk5FMnoyOEo2Zk52REJEMmVTNEs0bXU2?=
 =?utf-8?B?V1hLaHRjMjZQeHd1RXc0bHp1K2FQOXd4QUJYZWQ4RDBHNjlMa3dsYVc1N0lm?=
 =?utf-8?B?T0o1cVVOcUhXOEIwOUNSUmxvQmp4YUQ4ZTFSM1o3NkVNeWlkVjlGbU1WS1FI?=
 =?utf-8?B?NHdKVE5GNUF5U2RvbFM5dHJxVERRRmgvZ3FZd2M2YzVGMTdDcG0zbGY4aGRz?=
 =?utf-8?B?RFhuU1drS0JpV2xndTN3S0NjZy9XdHVmTW5TREhxUUtPWHA5VlhRRFZya0lO?=
 =?utf-8?B?WVFZVGE2NnNIZUtrdUdyQUx0TjFIWkRlS0cxVzQ5U3RzZzF6RlpnaGtGa0V0?=
 =?utf-8?B?NysybUMwc3N6T3Iwd0hJT2JpWWVFS0RieEFVcnlhcDRtQTF1azJOM0NadFhC?=
 =?utf-8?B?Ym4weUl3R0tmMmpENTVYUXR6V29aQ3Rscll4TlhPQWZXMGhhTEVZT2hFMEIz?=
 =?utf-8?B?MERIZElZYlI0ZGpZYklTRlNIdkd5ZHdrOWR4R2JGMjIrbHR4Nmp6ZTVSRVVr?=
 =?utf-8?B?NlIyd1pJSENyZm1KWkhqQ25MWmhMTHh2ZDVBRE1Vblo0bkp1a1JqZ0EzeUtm?=
 =?utf-8?B?TUlWcEQ3Q1dCckNoWXd5bm5nTXpjWUhZTDZzWHZSSkU1cVJTdWRTUFFvRy80?=
 =?utf-8?B?My9NN0M5VzhqMzNlWFlkYVg2TGQyUGxHemYySlp4SVBBWkVCQm05R3V6c1R1?=
 =?utf-8?B?Y1RxMWtFRlR1aWVDTGcwK0lOazZCZXJZU1B3Zm1uS1ZhaCsrZmkyNWxsOGdT?=
 =?utf-8?B?SkhWdU81L1Z1RVd4cyswUHNLOUtNREFIRzFXS1J3TW4rYUVMeHN1eWt6bitW?=
 =?utf-8?B?R2hkdzBHYm9xYS9OQnduY2o1Q08wMVdVTzYxbVU1VVkvTHVKa3JyMWN4cTUv?=
 =?utf-8?B?Zlg5Z1BDcStudTZ4elhpY1ZaRmV3NDlkUlZyTEZJNUhaR1R2VkZyTEhvNmNU?=
 =?utf-8?B?cWRMbThsK1BmNUhncUtBT2s2WUZmaitWQkl0ZXlvRTVnekZkaWVXb2lEU3RT?=
 =?utf-8?B?bFhZMkZ5cGFUTmlXQzl0bzd0RTMzSE1ZMVVCTU9rUDhLeTFwc05XcUUrUlV2?=
 =?utf-8?B?cGh3VFNaV2dLWkNDblByc2pQNkxGTHBGbXY5eSs0SHRpU2ljc1UyZm4xV1FY?=
 =?utf-8?B?eHhrbEFXUzd1QW91aVE4UHJSOUs1MWkvdXEwRlMxcmlRWWlVYTlEbXJONHA2?=
 =?utf-8?Q?W11Uo+h+bfEcMVJtvhEXs+9ZBKjrl5EA/W8/7wJ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 38bf7131-378d-40b3-c1d6-08d97f40e937
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:51:41.8105
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vBwb93oGjtzvmKFAiPqS72Nzpo96yMVT51Eyi5LcOuOc5wZcxKqj973Mx1pMmigNrwaDm+hC68crmdgwZhILog==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2446

In order to free intermediate page tables when replacing smaller
mappings by a single larger one callers will need to know the full PTE.
Flush indicators can be derived from this in the callers (and outside
the locked regions). First split set_iommu_pte_present() from
set_iommu_ptes_present(): Only the former needs to return the old PTE,
while the latter (like also set_iommu_pde_present()) doesn't even need
to return flush indicators. Then change return types/values and callers
accordingly.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -31,30 +31,28 @@ static unsigned int pfn_to_pde_idx(unsig
     return idx;
 }
 
-static unsigned int clear_iommu_pte_present(unsigned long l1_mfn,
-                                            unsigned long dfn)
+static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn,
+                                                   unsigned long dfn)
 {
-    union amd_iommu_pte *table, *pte;
-    unsigned int flush_flags;
+    union amd_iommu_pte *table, *pte, old;
 
     table = map_domain_page(_mfn(l1_mfn));
     pte = &table[pfn_to_pde_idx(dfn, 1)];
+    old = *pte;
 
-    flush_flags = pte->pr ? IOMMU_FLUSHF_modified : 0;
     write_atomic(&pte->raw, 0);
 
     unmap_domain_page(table);
 
-    return flush_flags;
+    return old;
 }
 
-static unsigned int set_iommu_pde_present(union amd_iommu_pte *pte,
-                                          unsigned long next_mfn,
-                                          unsigned int next_level, bool iw,
-                                          bool ir)
+static void set_iommu_pde_present(union amd_iommu_pte *pte,
+                                  unsigned long next_mfn,
+                                  unsigned int next_level,
+                                  bool iw, bool ir)
 {
-    union amd_iommu_pte new = {}, old;
-    unsigned int flush_flags = IOMMU_FLUSHF_added;
+    union amd_iommu_pte new = {};
 
     /*
      * FC bit should be enabled in PTE, this helps to solve potential
@@ -68,28 +66,42 @@ static unsigned int set_iommu_pde_presen
     new.next_level = next_level;
     new.pr = true;
 
-    old.raw = read_atomic(&pte->raw);
-    old.ign0 = 0;
-    old.ign1 = 0;
-    old.ign2 = 0;
+    write_atomic(&pte->raw, new.raw);
+}
 
-    if ( old.pr && old.raw != new.raw )
-        flush_flags |= IOMMU_FLUSHF_modified;
+static union amd_iommu_pte set_iommu_pte_present(unsigned long pt_mfn,
+                                                 unsigned long dfn,
+                                                 unsigned long next_mfn,
+                                                 unsigned int level,
+                                                 bool iw, bool ir)
+{
+    union amd_iommu_pte *table, *pde, old;
 
-    write_atomic(&pte->raw, new.raw);
+    table = map_domain_page(_mfn(pt_mfn));
+    pde = &table[pfn_to_pde_idx(dfn, level)];
+
+    old = *pde;
+    if ( !old.pr || old.next_level ||
+         old.mfn != next_mfn ||
+         old.iw != iw || old.ir != ir )
+        set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
+    else
+        old.pr = false; /* signal "no change" to the caller */
 
-    return flush_flags;
+    unmap_domain_page(table);
+
+    return old;
 }
 
-static unsigned int set_iommu_ptes_present(unsigned long pt_mfn,
-                                           unsigned long dfn,
-                                           unsigned long next_mfn,
-                                           unsigned int nr_ptes,
-                                           unsigned int pde_level,
-                                           bool iw, bool ir)
+static void set_iommu_ptes_present(unsigned long pt_mfn,
+                                   unsigned long dfn,
+                                   unsigned long next_mfn,
+                                   unsigned int nr_ptes,
+                                   unsigned int pde_level,
+                                   bool iw, bool ir)
 {
     union amd_iommu_pte *table, *pde;
-    unsigned int page_sz, flush_flags = 0;
+    unsigned int page_sz;
 
     table = map_domain_page(_mfn(pt_mfn));
     pde = &table[pfn_to_pde_idx(dfn, pde_level)];
@@ -98,20 +110,18 @@ static unsigned int set_iommu_ptes_prese
     if ( (void *)(pde + nr_ptes) > (void *)table + PAGE_SIZE )
     {
         ASSERT_UNREACHABLE();
-        return 0;
+        return;
     }
 
     while ( nr_ptes-- )
     {
-        flush_flags |= set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
+        set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
 
         ++pde;
         next_mfn += page_sz;
     }
 
     unmap_domain_page(table);
-
-    return flush_flags;
 }
 
 void amd_iommu_set_root_page_table(struct amd_iommu_dte *dte,
@@ -284,6 +294,7 @@ int amd_iommu_map_page(struct domain *d,
     struct domain_iommu *hd = dom_iommu(d);
     int rc;
     unsigned long pt_mfn = 0;
+    union amd_iommu_pte old;
 
     spin_lock(&hd->arch.mapping_lock);
 
@@ -320,12 +331,16 @@ int amd_iommu_map_page(struct domain *d,
     }
 
     /* Install 4k mapping */
-    *flush_flags |= set_iommu_ptes_present(pt_mfn, dfn_x(dfn), mfn_x(mfn),
-                                           1, 1, (flags & IOMMUF_writable),
-                                           (flags & IOMMUF_readable));
+    old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), 1,
+                                (flags & IOMMUF_writable),
+                                (flags & IOMMUF_readable));
 
     spin_unlock(&hd->arch.mapping_lock);
 
+    *flush_flags |= IOMMU_FLUSHF_added;
+    if ( old.pr )
+        *flush_flags |= IOMMU_FLUSHF_modified;
+
     return 0;
 }
 
@@ -334,6 +349,7 @@ int amd_iommu_unmap_page(struct domain *
 {
     unsigned long pt_mfn = 0;
     struct domain_iommu *hd = dom_iommu(d);
+    union amd_iommu_pte old = {};
 
     spin_lock(&hd->arch.mapping_lock);
 
@@ -355,11 +371,14 @@ int amd_iommu_unmap_page(struct domain *
     if ( pt_mfn )
     {
         /* Mark PTE as 'page not present'. */
-        *flush_flags |= clear_iommu_pte_present(pt_mfn, dfn_x(dfn));
+        old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn));
     }
 
     spin_unlock(&hd->arch.mapping_lock);
 
+    if ( old.pr )
+        *flush_flags |= IOMMU_FLUSHF_modified;
+
     return 0;
 }
 



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:52:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:52:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195144.347697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThsU-00049T-4G; Fri, 24 Sep 2021 09:52:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195144.347697; Fri, 24 Sep 2021 09:52:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThsU-00049M-1A; Fri, 24 Sep 2021 09:52:22 +0000
Received: by outflank-mailman (input) for mailman id 195144;
 Fri, 24 Sep 2021 09:52:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThsT-00049B-1v
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:52:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1b5fe4a0-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:52:20 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-rp95dWVnOrSOvTMBGECgbQ-1; Fri, 24 Sep 2021 11:52:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2446.eurprd04.prod.outlook.com (2603:10a6:800:4e::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 09:52:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:52:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Fri, 24 Sep 2021 09:52:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b5fe4a0-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477139;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oST90GrSEuah3sHHpzoExF0h0tTGYRPH7JQTtrWQFzA=;
	b=lKMsaJc6tSnA6HMssLBmjxejkKnBAEYEzzHz6YsVTNFz3SadxtYL17JJaxzx3s43mcrNBq
	qFrwaba/yzdP1WSgeoAi9TaaJ7+z+k5vaxV1KiRjwel+tqY07irs1/O1BncMyDsRLiOA4Y
	IZ/AbQc/ZJ6zwYv1UcwT+3bqjZTtY88=
X-MC-Unique: rp95dWVnOrSOvTMBGECgbQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X4GPuwzlsFl9x7jmEt9rZ3agxzqa+iKMQmOKPWhD6ygP7v4h8mqBw+QRMLAi7hkmlPCBpA1XFoSXhteuTWkBApCUBEKsSxJFYuREcvh6Wvx91XhGqeHzDZMMI3Fph+3ASCrgonTJM9fYqgUh6w0lhF2XSb0yWo1sbnocq+LR27k2FlP/RjGgKE1darIHzL7ON+MK4XZGpTAagmSbE7Dlj2HNu5xki4TaygFDcwlBNDB7oxKhBlaChiySJPcZj6fbqdRy0Dd3A8SEFyX39oxs9Wmn4pYiDnNGQ2UpbK1E/E74fDBsTDZBt2qCclKIIh4D077CFEI0Zif4ehAhqVOx2w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oST90GrSEuah3sHHpzoExF0h0tTGYRPH7JQTtrWQFzA=;
 b=XDZ6Ivb/mzdCULOa/2mug9/D2bBLlgsjsqV5lhQKGxDKJVdN2QG6gLaW7HGhOCS6IBHbjmNrLeYa6a3ntWp30/LGMQCywl5gXykbqkjE5KdppEAgG6kxUSa8sDXRUiHUfxKOTe6VtAjKUT150FESQsaIxqNKpsXPdq9/wYJO2vff73cie2/9NPbJDFbtYKdYDmHRst/bMp6ukCme9Pk+64hrn40qCgHFkT1yIiNe8AQcs+yJ06/BfnRU83QT7JENQZV6TamX7YM8WwaAzhHpUlF1kcWxPTMsK5xypu3ex8z+LZlBDhlZAXKRmL6LO9YYo84E7J3JQvtGcjmVx4opog==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 12/18] AMD/IOMMU: allow use of superpage mappings
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <cc93398d-982a-edbc-4ddd-b5459cef8f9a@suse.com>
Date: Fri, 24 Sep 2021 11:52:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0023.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a89f903f-cb29-40e6-b50f-08d97f40fd59
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2446:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB24468DF29910A86232D20DD5B3A49@VI1PR0401MB2446.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	T1NSOBLFumlTC1xyJ8gqsFeoolks4xg3OAoXVOD5bGTIcrepoW+G4+4x1ZVv2wB04i73lbfphY3JZ58JlBhET/4wEZRUnLRcLCLrHepfH57frmIBX5bgcz7UcObaYB11ikgqHiWwVIMh/f1L95vVZLLgGCn0EXNIzbCh2qt/Dm7hym4lVznojHfG7rfpUEIiIvhZprJwUTvm4ZGZlLdSiiLgG6WuQ6lwZY8bruRwXnKmZOKcOUSuAu1l23/ONYhwv7GhywmCPN2xw+ksKIkZOCHgkFF0Qurm/DjR1HWUYkKEXL4wNqN0RNPx9lmO4RfhW6NLlhjo+VFs2PsFjx9tj/ThClMP+HRYPp//bqtcuEviuXnkOS1c+fyQIop4WvMC49KGABhkLzImxxPTnabjDDVkIiIYDmYsjz8KXBxY4FyhilwSLDH/BZ6XjtPolC5hhWU8Ra1Lh1YnPTZBqxpqogaPVJOhPxO/kHvtO7vgFGw+O+HYKMXOvrT7n/7nbzG/SJxrhibipodJpjzHSlKhgvF59GtjK+BjptU7S36Z7H+Qcz6zNs3/sbXAuspVbQ5z3bCdeukp110P0577atZkHD+nSRWhBccUOvXPemKHqUC5ZilkmoK/otV0fMeTgXtNWpM6AFoOId3SsqRDYnmZwUpB9IVusIzrnng4r5B/xbzCrdwnshN/j0Oj/IfZG2MKgEQK14x0R1PBdafOtViP0jfwLAN9mLe6SqScRacU/pQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(956004)(4326008)(5660300002)(16576012)(6486002)(66946007)(186003)(2616005)(54906003)(6916009)(86362001)(8676002)(38100700002)(66476007)(36756003)(66556008)(31696002)(83380400001)(2906002)(26005)(31686004)(316002)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M0dzUmJwekxyeWVTRnZDWE8rN3h6NEFDZ1poWXVoeDQ1ZkhNMitseTZxUjFw?=
 =?utf-8?B?UGhCZVNnc2RuMngxbEJzeGNRbDhrLzBRbGk0WkdZeUhsSlRXYkJJaEhIRVc4?=
 =?utf-8?B?RFo4TlBTN09VK0wvRFVJdWhWUFdRdUtnWEZaMUl4elRvYnFoRkZOaExZNXp6?=
 =?utf-8?B?WTRtRGVieElpWElIQkZzMWRQaGwvTGVTTFI4WVVwd2JvdXRIWmFEZ0JEQW9s?=
 =?utf-8?B?a1JDSHlGY3Yyd0NiRmRDcVA5ODJrUlBSTnlSUFBXb2dndXlWTXNBOUloVkgw?=
 =?utf-8?B?UThXMnZHd2NDUjc2SkR0Y1hmd3ZvdEgyZGRsNDl3YUZDRDRyWDF0anhCU3hF?=
 =?utf-8?B?UXQvc0I3ejBIYmJtSlZldml1WEhCbHFlQ0xQM3hjcEtNQzg3UlZ2bVRDczJY?=
 =?utf-8?B?UHkrODNNS2NScWV0YmFWUnI1K3YxL1NueHN1S01YMU1BTEhNTnRpWkNXYXFy?=
 =?utf-8?B?ZVVMQ0RmYzFyMnh3dWpSWnY2RTZOdk1BN3V5OXRDQjNWMlFRT1FkanZOYTZi?=
 =?utf-8?B?TkNucEhJdHk4Mzc5ZmlISW9HOURZWjVBc0tEN3M3dnkwdHkrOEhTWk9IblhX?=
 =?utf-8?B?WFVybmdzWnRVQVdFU2lMZktpRXZpc1pWR21EQmRjRi9hSXg5djhyeFZES3Iy?=
 =?utf-8?B?UDZ1TWpzcXNyTldTQUlOZU01RW5uQUVabko4Uit1eEZmbkRNUzM5WVdVck00?=
 =?utf-8?B?Y3c1NWpxVlZScE9hYWQzSlcvcUJaYk5TUStiNmxOMUFMcVRkL05weWk3NjJ2?=
 =?utf-8?B?a3QwaEZycWFRMGFRc1p1N2ZwK3V6cGNjdDBGeGE0WC8yVWhGZXBmUDZDK2Fs?=
 =?utf-8?B?elU4Y2s2WS9PbjMxNkJsc2RGaFFVVy8xS2RYVzZ0NnR6NzVTMklEYkduREZI?=
 =?utf-8?B?eTBPSEJOR1FKbEhEcURkcVNoaUkxT0JtcXRMaHdYTFhRYmxwVExxOGdwMG52?=
 =?utf-8?B?Uk9RaXRDbkhJVk5UTE5yMDZpeGdDZGZkVmZwdW5mOEI4NktQeXh4RWRscjA5?=
 =?utf-8?B?QXFMSkdLSjFnbTZ4eWNFcVRDWHRnMEFyUUtzaDUxbmorRVNnNG5vU2dubEox?=
 =?utf-8?B?NUUxTXFmdXl3VmREQVJiS3hjcG9CWUlMUzBEUEtkeThSa095YlhNR2NWOWVT?=
 =?utf-8?B?UGxPVGNiOWJGYm9lYzFOWHp6WVllYVlEc1Mvb3BhSUVFMFZLVWNoY2ZmeXlN?=
 =?utf-8?B?L1k0a0h0YzJxNjl5UzlXdHo0dWVFY3orOFFYc3k2ekhRWk5SSHIwUGYrRC96?=
 =?utf-8?B?VXVHcndnVFo5VThpS3VCSEVYN0lUaVFDWkQrcnlzY2ZCVmJTL3FTZnozdStE?=
 =?utf-8?B?c3UyUWZ3bzh6aVlORUNwTmh5ZXZlZHlSdmtNVmxsaHkxVHZ5ZUwxYkc4SEZO?=
 =?utf-8?B?K0ZpNzA0SVFQenEvNnVycEhaVm90UndzY3lPejdPOHdiUEQxNW1UbiszVjIw?=
 =?utf-8?B?Uk9nS1pHakRNbzJKSWlJUFdXUUZHWG10bDc5dzhKeEZ5cEFCYndwMit0YkFv?=
 =?utf-8?B?eTJ3S0pTdHp2ZjlyWFpLSjhXLzYrK3dJbHBJOVFoMmZTTE1FTDFvQ1cvZmJF?=
 =?utf-8?B?aUsvTmhLTktzNHVPTHpUaWJZcGdoZWN0V3dBMXlYR1VDOWZ4Z3l5amsrekc0?=
 =?utf-8?B?em5rdHRYQ052THFFS0hvRkRVOXFiVHNCdnEvNTJQS29QclhEMk9QUDRZa1Fj?=
 =?utf-8?B?TitZRDBvVFcxdXlMRUpjYXdsVzdEL1dwYzVXZExqaC9TSjZ0S2liZDBRdUVG?=
 =?utf-8?Q?n4fpGwHE8OX/z2+Ubdt8aNxlnd5xaxQpG1tYevo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a89f903f-cb29-40e6-b50f-08d97f40fd59
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:52:15.6354
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vtqpuih0nbd3ev1MRJB+9xT4LCKFyc1v8tGozTtCRf7WKTtd/WmLESutc2OsOiPDieyJYGt6+YdIiLd82TwftA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2446

No separate feature flags exist which would control availability of
these; the only restriction is HATS (establishing the maximum number of
page table levels in general), and even that has a lower bound of 4.
Thus we can unconditionally announce 2M, 1G, and 512G mappings. (Via
non-default page sizes the implementation in principle permits arbitrary
size mappings, but these require multiple identical leaf PTEs to be
written, which isn't all that different from having to write multiple
consecutive PTEs with increasing frame numbers. IMO that's therefore
beneficial only on hardware where suitable TLBs exist; I'm unaware of
such hardware.)

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm not fully sure about allowing 512G mappings: The scheduling-for-
freeing of intermediate page tables can take quite a while when
replacing a tree of 4k mappings by a single 512G one. Plus (or otoh)
there's no present code path via which 512G chunks of memory could be
allocated (and hence mapped) anyway.

--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -32,12 +32,13 @@ static unsigned int pfn_to_pde_idx(unsig
 }
 
 static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn,
-                                                   unsigned long dfn)
+                                                   unsigned long dfn,
+                                                   unsigned int level)
 {
     union amd_iommu_pte *table, *pte, old;
 
     table = map_domain_page(_mfn(l1_mfn));
-    pte = &table[pfn_to_pde_idx(dfn, 1)];
+    pte = &table[pfn_to_pde_idx(dfn, level)];
     old = *pte;
 
     write_atomic(&pte->raw, 0);
@@ -288,10 +289,31 @@ static int iommu_pde_from_dfn(struct dom
     return 0;
 }
 
+static void queue_free_pt(struct domain *d, mfn_t mfn, unsigned int next_level)
+{
+    if ( next_level > 1 )
+    {
+        union amd_iommu_pte *pt = map_domain_page(mfn);
+        unsigned int i;
+
+        for ( i = 0; i < PTE_PER_TABLE_SIZE; ++i )
+            if ( pt[i].pr && pt[i].next_level )
+            {
+                ASSERT(pt[i].next_level < next_level);
+                queue_free_pt(d, _mfn(pt[i].mfn), pt[i].next_level);
+            }
+
+        unmap_domain_page(pt);
+    }
+
+    iommu_queue_free_pgtable(d, mfn_to_page(mfn));
+}
+
 int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
                        unsigned int flags, unsigned int *flush_flags)
 {
     struct domain_iommu *hd = dom_iommu(d);
+    unsigned int level = (IOMMUF_order(flags) / PTE_PER_TABLE_SHIFT) + 1;
     int rc;
     unsigned long pt_mfn = 0;
     union amd_iommu_pte old;
@@ -320,7 +342,7 @@ int amd_iommu_map_page(struct domain *d,
         return rc;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, true) ||
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, flush_flags, true) ||
          !pt_mfn )
     {
         spin_unlock(&hd->arch.mapping_lock);
@@ -330,8 +352,8 @@ int amd_iommu_map_page(struct domain *d,
         return -EFAULT;
     }
 
-    /* Install 4k mapping */
-    old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), 1,
+    /* Install mapping */
+    old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), level,
                                 (flags & IOMMUF_writable),
                                 (flags & IOMMUF_readable));
 
@@ -339,8 +361,13 @@ int amd_iommu_map_page(struct domain *d,
 
     *flush_flags |= IOMMU_FLUSHF_added;
     if ( old.pr )
+    {
         *flush_flags |= IOMMU_FLUSHF_modified;
 
+        if ( level > 1 && old.next_level )
+            queue_free_pt(d, _mfn(old.mfn), old.next_level);
+    }
+
     return 0;
 }
 
@@ -349,6 +376,7 @@ int amd_iommu_unmap_page(struct domain *
 {
     unsigned long pt_mfn = 0;
     struct domain_iommu *hd = dom_iommu(d);
+    unsigned int level = (order / PTE_PER_TABLE_SHIFT) + 1;
     union amd_iommu_pte old = {};
 
     spin_lock(&hd->arch.mapping_lock);
@@ -359,7 +387,7 @@ int amd_iommu_unmap_page(struct domain *
         return 0;
     }
 
-    if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, false) )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, flush_flags, false) )
     {
         spin_unlock(&hd->arch.mapping_lock);
         AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
@@ -371,14 +399,19 @@ int amd_iommu_unmap_page(struct domain *
     if ( pt_mfn )
     {
         /* Mark PTE as 'page not present'. */
-        old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn));
+        old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level);
     }
 
     spin_unlock(&hd->arch.mapping_lock);
 
     if ( old.pr )
+    {
         *flush_flags |= IOMMU_FLUSHF_modified;
 
+        if ( level > 1 && old.next_level )
+            queue_free_pt(d, _mfn(old.mfn), old.next_level);
+    }
+
     return 0;
 }
 
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -630,7 +630,7 @@ static void amd_dump_page_tables(struct
 }
 
 static const struct iommu_ops __initconstrel _iommu_ops = {
-    .page_sizes = PAGE_SIZE_4K,
+    .page_sizes = PAGE_SIZE_4K | PAGE_SIZE_2M | PAGE_SIZE_1G | PAGE_SIZE_512G,
     .init = amd_iommu_domain_init,
     .hwdom_init = amd_iommu_hwdom_init,
     .quarantine_init = amd_iommu_quarantine_init,
--- a/xen/include/xen/page-defs.h
+++ b/xen/include/xen/page-defs.h
@@ -21,4 +21,19 @@
 #define PAGE_MASK_64K               PAGE_MASK_GRAN(64K)
 #define PAGE_ALIGN_64K(addr)        PAGE_ALIGN_GRAN(64K, addr)
 
+#define PAGE_SHIFT_2M               21
+#define PAGE_SIZE_2M                PAGE_SIZE_GRAN(2M)
+#define PAGE_MASK_2M                PAGE_MASK_GRAN(2M)
+#define PAGE_ALIGN_2M(addr)         PAGE_ALIGN_GRAN(2M, addr)
+
+#define PAGE_SHIFT_1G               30
+#define PAGE_SIZE_1G                PAGE_SIZE_GRAN(1G)
+#define PAGE_MASK_1G                PAGE_MASK_GRAN(1G)
+#define PAGE_ALIGN_1G(addr)         PAGE_ALIGN_GRAN(1G, addr)
+
+#define PAGE_SHIFT_512G             39
+#define PAGE_SIZE_512G              PAGE_SIZE_GRAN(512G)
+#define PAGE_MASK_512G              PAGE_MASK_GRAN(512G)
+#define PAGE_ALIGN_512G(addr)       PAGE_ALIGN_GRAN(512G, addr)
+
 #endif /* __XEN_PAGE_DEFS_H__ */



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:52:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:52:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195150.347708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThsq-0004kL-HG; Fri, 24 Sep 2021 09:52:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195150.347708; Fri, 24 Sep 2021 09:52:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThsq-0004kB-Df; Fri, 24 Sep 2021 09:52:44 +0000
Received: by outflank-mailman (input) for mailman id 195150;
 Fri, 24 Sep 2021 09:52:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mThso-0004iI-GB
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:52:42 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.72]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 279b250e-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:52:40 +0000 (UTC)
Received: from AM6P194CA0015.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:90::28)
 by DBAPR08MB5591.eurprd08.prod.outlook.com (2603:10a6:10:1ae::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Fri, 24 Sep
 2021 09:52:38 +0000
Received: from AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:90:cafe::7c) by AM6P194CA0015.outlook.office365.com
 (2603:10a6:209:90::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 09:52:38 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT056.mail.protection.outlook.com (10.152.17.224) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:52:37 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Fri, 24 Sep 2021 09:52:37 +0000
Received: from ed9a0c2bc179.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6A16684F-4741-488E-974B-82E59013C166.1; 
 Fri, 24 Sep 2021 09:52:27 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ed9a0c2bc179.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 09:52:27 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 09:52:25 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:52:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 279b250e-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L2CgbX7qgYVadUbPgfACKmpkBOnaL4nN85Fl11zIlLY=;
 b=APXRYfkztN0/mxf1Xr5a5Hy6iCUFKDPnmQcds/RTIESNMiRhV9X9jqKhqWxSYGxDjLWQZIYHWWKzM12l4izkiu4fHP6JeI4z/WOXA8racYCL2t0FyrWhstlVq7M1727dJS1fSuaT78yG3rtoQkGi1g/GxUfAfSoxtHDLhfa3bTo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LwNoiH5ugdh+G9Lnq4EXS8Q6X2YaSPW50ksZzZlU0xHZq/wolceynjuQ0TNs5QZOQlB8jL3Jb4bPTFVaxaHiVI638Z7qMrm4Cug5lae9oFW1uqgh3DzlFV2ppDYNjgg7K+yXya33pdplKKyyMSgrEzf/7CjFy5AibJ2pG2HqLURtTNHtALK2bK3ZbvihSx+JVet9h4jy/NF2GOCtyQKdq82Bq0XqT+/7blS/olu3fz/Fi3SdWC1HBWJyhUkHtqL2Ly7c5cxlHEzHdupgWnDaCpWaAsaev8QmybcdJ6v8FJvcWyAfRUmgAjCPyAmAyGRZ+oN/HxjpULRjSAmpYvaQug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=L2CgbX7qgYVadUbPgfACKmpkBOnaL4nN85Fl11zIlLY=;
 b=gcKdxM5xC65XY6pRR7DsFHyN6cyi269tNxSABmIeXL13wxOv02F8D2X4wHHWWgiGyueDjzYGFC/kfV40i0/AIopBoJxLt1dQOuFv5MC+5C8v30R3KBVruH7oqQVynPIlg0VKkitZjVc/LkhYBt3BCiFwKiAf131Ci9ugg27tVVIGKhFFBNC/Zt2YWY5WqKVOFOSfFFI/puJsXwdWCp0iwGnWKj9pQz+CcZtexAaUYnUS55yuNRgLY2gEIDciysAryPCfqC1bQbu9sT/vPqVYW80bRs9JQpP9LuwdDlwR+x/FJ2u2TXum71zxm1n2GTo9kr9jQeziqAyxDJ7uwy6a/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=L2CgbX7qgYVadUbPgfACKmpkBOnaL4nN85Fl11zIlLY=;
 b=APXRYfkztN0/mxf1Xr5a5Hy6iCUFKDPnmQcds/RTIESNMiRhV9X9jqKhqWxSYGxDjLWQZIYHWWKzM12l4izkiu4fHP6JeI4z/WOXA8racYCL2t0FyrWhstlVq7M1727dJS1fSuaT78yG3rtoQkGi1g/GxUfAfSoxtHDLhfa3bTo=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 34/37] xen/arm: enable device tree based NUMA in system
 init
Thread-Topic: [PATCH 34/37] xen/arm: enable device tree based NUMA in system
 init
Thread-Index: AQHXsHM8PJZnWMzeQUKgL4OJKEMnLKuyh4oAgABLdGA=
Date: Fri, 24 Sep 2021 09:52:25 +0000
Message-ID:
 <DB9PR08MB6857ACA0C3C2FD59EC6FF5559EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-35-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232021270.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109232021270.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: F2E682EDD10EDE479AD17DE1B3B33138.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 394df3af-3ade-402d-fdd9-08d97f410ac9
x-ms-traffictypediagnostic: DBBPR08MB4646:|DBAPR08MB5591:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBAPR08MB559175F87394DB702050A9B99EA49@DBAPR08MB5591.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:3968;OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 S+aVtYXtRxRM+r82EHuzYzoDJn5YW1UjhOR629kQC68faFGCzn84YMv7LWY4iBEbu9qWTAYb65cqGL+416MaIULHSah4lICNQCkZB788yepEpnPX7j1JsvdVfFmFZGjfv/Wip/nn6Ip8KnN4bQmOOwYUbdIQwXEwguRR8jnAbS5k+qlTwC8usWxl0OJLT3Bz9FnSrx7/DBZ3xJSmPIjnhhNvQyAodb7e4pxgBnHnXr2xSJJ8uQ4BzclHFM9G9ZJv8Kw6/U0GkZ+oVuXOi/44g5nmm1o8+S+ThKlDL4kVbDvVa3bJTubZIqcmZRyjja1I4AJd4LF8CRSsgMbtDQuQM9i3HMmNgPTaRpDycTQRIbOVY0prV/r8LdabU2bvbJdyHH2LOwKXlbxRZ+BmdQ5k2Hg1vEXORgW+nLG5cPbXBMG8t/Q+pj3i9BEzhvOy5Er+r4vm/KZGMgVqEx9XNF4mcbwyshWIfqB1C7YzFaij+TdUgi+wW83bwRIyuFwJP7jPUReJR35l+JA2zmpHuNjzkvH5JjcWy9TNppbMIBNAh9GYBxuAoj9dO1zTeviys2CADPcicL8qoynxqQhpUtCDei/yNxoBcJWkFyq2bhnvA58ScWVnkFVTOl2Dd0hCzmmovg9CRj7MPRZwh9B80VDoelgMRSuhipdezXCtVVkt014PNQ2xUpENy64abeSud/NaRi+8JAtUW6BCsk0a2azY3A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(53546011)(6506007)(86362001)(66446008)(7696005)(186003)(66946007)(4326008)(26005)(76116006)(66556008)(64756008)(316002)(55016002)(8676002)(66476007)(5660300002)(9686003)(38070700005)(83380400001)(6916009)(71200400001)(2906002)(33656002)(508600001)(122000001)(52536014)(8936002)(54906003)(38100700002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4646
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	04fa6e02-dd0a-4ae3-d0bb-08d97f41033d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NgoS0dT7mcSdafJJh0q3NHtUUIHapnchsPpBVP1f4LSYhsfHJhjtnc5NtsWMwVZ50S0g7Ki3mZ4JptDkOtueCwfBaivgoeuZuoK1FEDoHtF36lR2eFQcyOntYn2Hu7qzrDM04rpsmD6TXmNV2s3KDidGJjLtITnKh47r9+p8eJLRm6hKgn/Eqoc0PNOdpfW0P6FncaqSBSzL81c1WW6pu0eah2zenCv9HpV9m8ATidZ640K3rsTTGIvzFkl9ytW2ykBk8AgzhfFh9PVYuFhEpn2H4FdyVvlSQLRtk9Eu+Mi6Mk6i2iKLW9XR6NiTD9sExtSzMWKFofnjMCUIe9DSSNI7TZUOl8uYFX+6qq+Tq7tSnMdF2xXVV2QccAxKQV+zlp6JoaXko024ejZrtoXvrJW6/LyBnCotwws5jd49nYeC38kjUF5x0/gCJ/DL1QHWovJv2n2rhTEryPRfmVJvDc0DY86+RInePEWsoOB6dQ516wj3PnW/C4NSYMfJ9lTpS1A8rB5BStW/61T3Hl9i09C18kAgs4T+f+gnCJaLSOZIgHbl6ROlUHNX8IvOLRUBJRyha/I9MaX8amu6cgUOZsumOlcFshrAbdC17LMkrU53bAhjHsH/VwIdE5p4wqKQTs8zfgHFJ9XNpIBCWOZNNiEYeidEpN6GG3rTRQm4RsNwKrikV7JrEiZpk4Q5GXkivIKr3FXEkILFdzfBFHjHuA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(9686003)(4326008)(2906002)(336012)(508600001)(356005)(8936002)(54906003)(26005)(81166007)(55016002)(7696005)(47076005)(186003)(6862004)(70206006)(8676002)(70586007)(82310400003)(33656002)(52536014)(5660300002)(86362001)(316002)(83380400001)(36860700001)(6506007)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:52:37.9497
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 394df3af-3ade-402d-fdd9-08d97f410ac9
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT056.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5591

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgMTE6MjgNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzogeGVu
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxp
ZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29t
Pg0KPiBTdWJqZWN0OiBSZTogW1BBVENIIDM0LzM3XSB4ZW4vYXJtOiBlbmFibGUgZGV2aWNlIHRy
ZWUgYmFzZWQgTlVNQSBpbg0KPiBzeXN0ZW0gaW5pdA0KPiANCj4gT24gVGh1LCAyMyBTZXAgMjAy
MSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gSW4gdGhpcyBwYXRjaCwgd2UgY2FuIHN0YXJ0IHRvIGNy
ZWF0ZSBOVU1BIHN5c3RlbSB0aGF0IGlzDQo+ID4gYmFzZWQgb24gZGV2aWNlIHRyZWUuDQo+ID4N
Cj4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAtLS0N
Cj4gPiAgeGVuL2FyY2gvYXJtL251bWEuYyAgICAgICAgfCA1NSArKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKw0KPiA+ICB4ZW4vYXJjaC9hcm0vc2V0dXAuYyAgICAgICB8ICA3
ICsrKysrDQo+ID4gIHhlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIHwgIDYgKysrKysNCj4gPiAg
MyBmaWxlcyBjaGFuZ2VkLCA2OCBpbnNlcnRpb25zKCspDQo+ID4NCj4gPiBkaWZmIC0tZ2l0IGEv
eGVuL2FyY2gvYXJtL251bWEuYyBiL3hlbi9hcmNoL2FybS9udW1hLmMNCj4gPiBpbmRleCA3ZjA1
Mjk5Yjc2Li5kN2EzZDMyZDRiIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9hcmNoL2FybS9udW1hLmMN
Cj4gPiArKysgYi94ZW4vYXJjaC9hcm0vbnVtYS5jDQo+ID4gQEAgLTE4LDggKzE4LDEwIEBADQo+
ID4gICAqDQo+ID4gICAqLw0KPiA+ICAjaW5jbHVkZSA8eGVuL2luaXQuaD4NCj4gPiArI2luY2x1
ZGUgPHhlbi9kZXZpY2VfdHJlZS5oPg0KPiA+ICAjaW5jbHVkZSA8eGVuL25vZGVtYXNrLmg+DQo+
ID4gICNpbmNsdWRlIDx4ZW4vbnVtYS5oPg0KPiA+ICsjaW5jbHVkZSA8eGVuL3Bmbi5oPg0KPiA+
DQo+ID4gIHN0YXRpYyB1aW50OF90IF9fcmVhZF9tb3N0bHkNCj4gPiAgbm9kZV9kaXN0YW5jZV9t
YXBbTUFYX05VTU5PREVTXVtNQVhfTlVNTk9ERVNdID0gew0KPiA+IEBAIC04NSw2ICs4Nyw1OSBA
QCB1aW50OF90IF9fbm9kZV9kaXN0YW5jZShub2RlaWRfdCBmcm9tLCBub2RlaWRfdCB0bykNCj4g
PiAgfQ0KPiA+ICBFWFBPUlRfU1lNQk9MKF9fbm9kZV9kaXN0YW5jZSk7DQo+ID4NCj4gPiArdm9p
ZCBfX2luaXQgbnVtYV9pbml0KGJvb2wgYWNwaV9vZmYpDQo+ID4gK3sNCj4gPiArICAgIHVpbnQz
Ml90IGlkeDsNCj4gPiArICAgIHBhZGRyX3QgcmFtX3N0YXJ0ID0gfjA7DQo+IA0KPiBJTlZBTElE
X1BBRERSDQo+IA0KDQpPaCwgeWVzDQoNCj4gDQo+ID4gKyAgICBwYWRkcl90IHJhbV9zaXplID0g
MDsNCj4gPiArICAgIHBhZGRyX3QgcmFtX2VuZCA9IDA7DQo+ID4gKw0KPiA+ICsgICAgLyogTlVN
QSBoYXMgYmVlbiB0dXJuZWQgb2ZmIHRocm91Z2ggWGVuIHBhcmFtZXRlcnMgKi8NCj4gPiArICAg
IGlmICggbnVtYV9vZmYgKQ0KPiA+ICsgICAgICAgIGdvdG8gbWVtX2luaXQ7DQo+ID4gKw0KPiA+
ICsgICAgLyogSW5pdGlhbGl6ZSBOVU1BIGZyb20gZGV2aWNlIHRyZWUgd2hlbiBzeXN0ZW0gaXMg
bm90IEFDUEkgYm9vdGVkDQo+ICovDQo+ID4gKyAgICBpZiAoIGFjcGlfb2ZmICkNCj4gPiArICAg
IHsNCj4gPiArICAgICAgICBpbnQgcmV0ID0gbnVtYV9kZXZpY2VfdHJlZV9pbml0KGRldmljZV90
cmVlX2ZsYXR0ZW5lZCk7DQo+ID4gKyAgICAgICAgaWYgKCByZXQgKQ0KPiA+ICsgICAgICAgIHsN
Cj4gPiArICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19XQVJOSU5HDQo+ID4gKyAgICAgICAgICAg
ICAgICAgICAiSW5pdCBOVU1BIGZyb20gZGV2aWNlIHRyZWUgZmFpbGVkLCByZXQ9JWRcbiIsIHJl
dCk7DQo+IA0KPiBBcyBJIG1lbnRpb25lZCBpbiBvdGhlciBwYXRjaGVzIHdlIG5lZWQgdG8gZGlz
dGluZ3Vpc2ggYmV0d2VlbiB0d28NCj4gY2FzZXM6DQo+IA0KPiAxKSBOVU1BIGluaXRpYWxpemF0
aW9uIGZhaWxlZCBiZWNhdXNlIG5vIE5VTUEgaW5mb3JtYXRpb24gaGFzIGJlZW4gZm91bmQNCj4g
MikgTlVNQSBpbml0aWFsaXphdGlvbiBmYWlsZWQgYmVjYXVzZSB3cm9uZy9pbmNvbnNpc3RlbnQg
TlVNQSBpbmZvIGhhcw0KPiAgICBiZWVuIGZvdW5kDQo+IA0KPiBJbiBjYXNlIG9mIDEpLCB3ZSBw
cmludCBub3RoaW5nLiBNYXliZSBhIHNpbmdsZSBYRU5MT0dfREVCVUcgbWVzc2FnZS4NCj4gSW4g
Y2FzZSBvZiAyKSwgYWxsIHRoZSB3YXJuaW5ncyBhcmUgZ29vZCB0byBwcmludC4NCj4gDQo+IA0K
PiBJbiB0aGlzIGNhc2UsIGlmIHJldCAhPSAwIGJlY2F1c2Ugb2YgMiksIHRoZW4gaXQgaXMgZmlu
ZSB0byBwcmludCB0aGlzDQo+IHdhcm5pbmcuIEJ1dCBpdCBsb29rcyBsaWtlIGNvdWxkIGJlIHRo
YXQgcmV0IGlzIC1FSU5WQUwgc2ltcGx5IGJlY2F1c2UgYQ0KPiBDUFUgbm9kZSBkb2Vzbid0IGhh
dmUgbnVtYS1ub2RlLWlkLCB3aGljaCBpcyBhIG5vcm1hbCBjb25kaXRpb24gZm9yDQo+IG5vbi1O
VU1BIG1hY2hpbmVzLg0KPiANCg0KWWVzLCB3ZSBzaG91bGQgaGF2ZSB0byBkaXN0aW5ndWlzaCB0
aGVzZSB0d28gY2FzZXMuIEkgd2lsbCB0cnkgdG8gYWRkcmVzcw0KaXQgaW4gbmV4dCB2ZXJzaW9u
Lg0KDQo+IA0KPiA+ICsgICAgICAgICAgICBudW1hX29mZiA9IHRydWU7DQo+ID4gKyAgICAgICAg
fQ0KPiA+ICsgICAgfQ0KPiA+ICsgICAgZWxzZQ0KPiA+ICsgICAgew0KPiA+ICsgICAgICAgIC8q
IFdlIGRvbid0IHN1cHBvcnQgTlVNQSBmb3IgQUNQSSBib290IGN1cnJlbnRseSAqLw0KPiA+ICsg
ICAgICAgIHByaW50ayhYRU5MT0dfV0FSTklORw0KPiA+ICsgICAgICAgICAgICAgICAiQUNQSSBO
VU1BIGhhcyBub3QgYmVlbiBzdXBwb3J0ZWQgeWV0LCBOVU1BIG9mZiFcbiIpOw0KPiA+ICsgICAg
ICAgIG51bWFfb2ZmID0gdHJ1ZTsNCj4gPiArICAgIH0NCj4gPiArDQo+ID4gK21lbV9pbml0Og0K
PiA+ICsgICAgLyoNCj4gPiArICAgICAqIEZpbmQgdGhlIG1pbmltYWwgYW5kIG1heGltdW0gYWRk
cmVzcyBvZiBSQU0sIE5VTUEgd2lsbA0KPiA+ICsgICAgICogYnVpbGQgYSBtZW1vcnkgdG8gbm9k
ZSBtYXBwaW5nIHRhYmxlIGZvciB0aGUgd2hvbGUgcmFuZ2UuDQo+ID4gKyAgICAgKi8NCj4gPiAr
ICAgIHJhbV9zdGFydCA9IGJvb3RpbmZvLm1lbS5iYW5rWzBdLnN0YXJ0Ow0KPiA+ICsgICAgcmFt
X3NpemUgID0gYm9vdGluZm8ubWVtLmJhbmtbMF0uc2l6ZTsNCj4gPiArICAgIHJhbV9lbmQgICA9
IHJhbV9zdGFydCArIHJhbV9zaXplOw0KPiA+ICsgICAgZm9yICggaWR4ID0gMSA7IGlkeCA8IGJv
b3RpbmZvLm1lbS5ucl9iYW5rczsgaWR4KysgKQ0KPiA+ICsgICAgew0KPiA+ICsgICAgICAgIHBh
ZGRyX3QgYmFua19zdGFydCA9IGJvb3RpbmZvLm1lbS5iYW5rW2lkeF0uc3RhcnQ7DQo+ID4gKyAg
ICAgICAgcGFkZHJfdCBiYW5rX3NpemUgPSBib290aW5mby5tZW0uYmFua1tpZHhdLnNpemU7DQo+
ID4gKyAgICAgICAgcGFkZHJfdCBiYW5rX2VuZCA9IGJhbmtfc3RhcnQgKyBiYW5rX3NpemU7DQo+
ID4gKw0KPiA+ICsgICAgICAgIHJhbV9zaXplICA9IHJhbV9zaXplICsgYmFua19zaXplOw0KPiA+
ICsgICAgICAgIHJhbV9zdGFydCA9IG1pbihyYW1fc3RhcnQsIGJhbmtfc3RhcnQpOw0KPiA+ICsg
ICAgICAgIHJhbV9lbmQgICA9IG1heChyYW1fZW5kLCBiYW5rX2VuZCk7DQo+ID4gKyAgICB9DQo+
ID4gKw0KPiA+ICsgICAgbnVtYV9pbml0bWVtX2luaXQoUEZOX1VQKHJhbV9zdGFydCksIFBGTl9E
T1dOKHJhbV9lbmQpKTsNCj4gPiArICAgIHJldHVybjsNCj4gDQo+IE5vIG5lZWQgZm9yIHJldHVy
bg0KPiANCg0KT2ssIEkgd2lsbCByZW1vdmUgaXQuDQoNCj4gDQo+ID4gK30NCj4gPiArDQo+ID4g
IHVpbnQzMl90IF9faW5pdCBhcmNoX21lbWluZm9fZ2V0X25yX2Jhbmsodm9pZCkNCj4gPiAgew0K
PiA+ICAJcmV0dXJuIGJvb3RpbmZvLm1lbS5ucl9iYW5rczsNCj4gPiBkaWZmIC0tZ2l0IGEveGVu
L2FyY2gvYXJtL3NldHVwLmMgYi94ZW4vYXJjaC9hcm0vc2V0dXAuYw0KPiA+IGluZGV4IDFmMGZi
Yzk1YjUuLjYwOTc4NTA2ODIgMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gvYXJtL3NldHVwLmMN
Cj4gPiArKysgYi94ZW4vYXJjaC9hcm0vc2V0dXAuYw0KPiA+IEBAIC05MDUsNiArOTA1LDEzIEBA
IHZvaWQgX19pbml0IHN0YXJ0X3hlbih1bnNpZ25lZCBsb25nDQo+IGJvb3RfcGh5c19vZmZzZXQs
DQo+ID4gICAgICAvKiBQYXJzZSB0aGUgQUNQSSB0YWJsZXMgZm9yIHBvc3NpYmxlIGJvb3QtdGlt
ZSBjb25maWd1cmF0aW9uICovDQo+ID4gICAgICBhY3BpX2Jvb3RfdGFibGVfaW5pdCgpOw0KPiA+
DQo+ID4gKyAgICAvKg0KPiA+ICsgICAgICogVHJ5IHRvIGluaXRpYWxpemUgTlVNQSBzeXN0ZW0s
IGlmIGZhaWxlZCwgdGhlIHN5c3RlbSB3aWxsDQo+ID4gKyAgICAgKiBmYWxsYmFjayB0byB1bmlm
b3JtIHN5c3RlbSB3aGljaCBtZWFucyBzeXN0ZW0gaGFzIG9ubHkgMQ0KPiA+ICsgICAgICogTlVN
QSBub2RlLg0KPiA+ICsgICAgICovDQo+ID4gKyAgICBudW1hX2luaXQoYWNwaV9kaXNhYmxlZCk7
DQo+ID4gKw0KPiA+ICAgICAgZW5kX2Jvb3RfYWxsb2NhdG9yKCk7DQo+ID4NCj4gPiAgICAgIC8q
DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIGIveGVuL2luY2x1
ZGUvYXNtLWFybS9udW1hLmgNCj4gPiBpbmRleCBmNDZlOGUyOTM1Li41YjAzZGRlODdmIDEwMDY0
NA0KPiA+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gKysrIGIveGVuL2lu
Y2x1ZGUvYXNtLWFybS9udW1hLmgNCj4gPiBAQCAtMjQsNiArMjQsNyBAQCB0eXBlZGVmIHU4IG5v
ZGVpZF90Ow0KPiA+DQo+ID4gIGV4dGVybiB2b2lkIG51bWFfc2V0X2Rpc3RhbmNlKG5vZGVpZF90
IGZyb20sIG5vZGVpZF90IHRvLCB1aW50MzJfdA0KPiBkaXN0YW5jZSk7DQo+ID4gIGV4dGVybiBp
bnQgbnVtYV9kZXZpY2VfdHJlZV9pbml0KGNvbnN0IHZvaWQgKmZkdCk7DQo+ID4gK2V4dGVybiB2
b2lkIG51bWFfaW5pdChib29sIGFjcGlfb2ZmKTsNCj4gPg0KPiA+ICAjZWxzZQ0KPiA+DQo+ID4g
QEAgLTQ3LDYgKzQ4LDExIEBAIGV4dGVybiBtZm5fdCBmaXJzdF92YWxpZF9tZm47DQo+ID4gICNk
ZWZpbmUgbm9kZV9zdGFydF9wZm4obmlkKSAobWZuX3goZmlyc3RfdmFsaWRfbWZuKSkNCj4gPiAg
I2RlZmluZSBfX25vZGVfZGlzdGFuY2UoYSwgYikgKDIwKQ0KPiA+DQo+ID4gK3N0YXRpYyBpbmxp
bmUgdm9pZCBudW1hX2luaXQoYm9vbCBhY3BpX29mZikNCj4gPiArew0KPiA+ICsNCj4gPiArfQ0K
PiA+ICsNCj4gPiAgc3RhdGljIGlubGluZSB2b2lkIG51bWFfYWRkX2NwdShpbnQgY3B1KQ0KPiA+
ICB7DQo+ID4NCj4gPiAtLQ0KPiA+IDIuMjUuMQ0KPiA+DQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195153.347719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTht0-00058o-Qe; Fri, 24 Sep 2021 09:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195153.347719; Fri, 24 Sep 2021 09:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTht0-00058d-N4; Fri, 24 Sep 2021 09:52:54 +0000
Received: by outflank-mailman (input) for mailman id 195153;
 Fri, 24 Sep 2021 09:52:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTht0-00055r-2l
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:52:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 15deca61-bb3c-4e11-88ce-c749ce2b69cf;
 Fri, 24 Sep 2021 09:52:53 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-hP4Sf-beMd-BWGlFNp7Dkw-1; Fri, 24 Sep 2021 11:52:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:52:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:52:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0041.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Fri, 24 Sep 2021 09:52:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15deca61-bb3c-4e11-88ce-c749ce2b69cf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477172;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uq1n+wvXInUj+R2+IA3D3xi4j2nX4n5cE5fQ+7WfxoE=;
	b=CF+ses7i1RsjwTWWkCthgB3rdZ/r6Afji7fEdkggg0XgN54RNOA0/hPjLfKJCuoK8O5/QF
	juHC6ilWDipVeentkHodEibM6WjkgPOAik5YIb/A6bRBPBth1ckLWs8QfY32YOopyeAYXI
	24yQrpAXjSg6tED98RuQvNnfZnwWJsw=
X-MC-Unique: hP4Sf-beMd-BWGlFNp7Dkw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P6CtNP403gb3t0MQX3b37wqIEKlcmvdbvi38upK8z3RfZzLfF/fVYxTDPuqiAWo+X980a0Noolt7jlIuj7dPGmvlwjqAq2xFlERoYMcr918z9X4wSzf/Dnl6hVBdZqCh3xT1bRgMziFZdwxVTC4Z9A9AOTp46WtY13/Z7xmQ9cMjeLmu13azhE8Dasvn6mWzRq0vDkCoJfIA9JO6ixC9azAR1J/5+bsK0GrEcMnmp6SrgUACTc5yywtyMCXWh6feocB3RodJcWJDpICb4F0TH2kJDAR5Ke9loGjIC4c/b1jlphx4LcxgHM6qqbT2RocUq0kJCoY5he/r2rC8navJ2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uq1n+wvXInUj+R2+IA3D3xi4j2nX4n5cE5fQ+7WfxoE=;
 b=KJZzLHlBd8kYOJk7b10XLOohyPTGzrFH5bx/dvPiql/gmHeaYK6GNXvsF+hZZ0j6OKhOFybw/YVA2eTufDZHQ+dYXuT7b8YTqk5mCseJqlz6U2KDMoclQghOlOdoZuQx0Rn51IvYyLqypdMD0S7J6SC13WOMRV4JKF6nkbMLZTIfv5s2xT1n+MCmXrgpf/fgkju6Yb4cFfAxR/dHvYOY0FBGcTipg2B2XGU76cW3/uSFaYfwHKZhXwb35KNchQg0727xGH+GHEJ2kE4xpRSmmmyw/pdDCXww37RxLkbLPWlCcBFuLQtKdU/BoA9OAZTaaay9Tp20+TQ079WPRutRjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 13/18] VT-d: allow use of superpage mappings
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <df4bfaa9-e708-929d-3c2c-68833dcc2d2f@suse.com>
Date: Fri, 24 Sep 2021 11:52:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0041.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d1f2f27e-083b-460e-d624-08d97f411155
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB715092BE9E27CC454C2F7D13B3A49@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lj87Ew5+NFKb/5zRCHsBh2/z0f1+SJh+UTNbTKRM7jTaD1L90zUXFTV1I2ydqY6LEYBUVoltI4uY40+FWPE8VGUR33o+i2bYl424SH4eIgAWl0zFVsOz8rZReoNo1Ww1XYfqshhRWVtxQNDh1mp39pPFHI8J8qfpb/8XOCJ5OxXMt60YfBUbl17zZ7CpGd93IkFRm4BX9+uUKqjPECJtgW/NYDKaJKJqJJPhtVwbAwrI+0MJCmJxY8rod17ZwObDGGNuG+Jt4+g8dJPJJSTuVJHNybvNq1KzGi3J6rRO1p2h1zgAKg8WchN6/88yV/+wLE4+WKL9NIUe6Yk/cVeEtaAjUtYhkHbxv5EBNAC9or0vim7FwQfV/StrVtIWPLHHlT3FMQqbPko6RXzt3OzSx31tLekzBBgAS3TirYLjqW/kZejaJ7wvEdUdZzg01fXQaTYyGdS0ColRvhdQJc2oJp+6rs4mCs3ekmVI98UGZ4s3GyCa2VsOs9+9p5f/yFqyFjPr58i/wEeiP40cFO7TNjzv5CLvWdZqdXoVyCZrQ0D02iWTsXr6CKN2xRp7fUUSkGlcdM7a1HkQEO4eBE/oZe3NgAsLcoxGcGUhi+wsiFCXD6QU3CHfe0Y2ElM8G2tmmkP0MPdssslGG61q3kDfyJY5UNuIl7/N6eBNGd4WPc9HOOk8SMzZMLECzpscxYgZZXhKtJXqnLxFtCk+eG2mlzj8A3NKOOykbiP7e3pfB3c=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(8936002)(5660300002)(2616005)(508600001)(31686004)(956004)(38100700002)(66476007)(66946007)(26005)(8676002)(66556008)(6916009)(2906002)(36756003)(86362001)(6486002)(16576012)(31696002)(83380400001)(4326008)(316002)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Um9JWGtnemJZRndGVXRRelRjcTZPeHhlS0VXY2tTdURjUWN1NkIwT3JhZU4y?=
 =?utf-8?B?QkxLWFF5RnpuZE52bzZSZVlqeERtenhBSGNaNlVHRTBHeHBwckFRdnp3MWVL?=
 =?utf-8?B?WG1PWFlvTXIzWGRRY2NnSzZSSmVZZVB1cEFYYXIwLy9PR3JYeHpXMGNFVlYx?=
 =?utf-8?B?NGxxSldWcG9nY3J4ZGM4WDJtVzYrUmkwN0RZRUUyOCt6TVJLVUZtR2tiVm9O?=
 =?utf-8?B?aEprb2NwV28wMUs3WCtXMjJXVE9DbWJoand1MThOVjVhbzRhN2ZXeTUzNVQr?=
 =?utf-8?B?M0hHNzl5RGN1ZXFiWG50bXBhUG9CQkdMeUZCaVRMV09YdUtrRnptQm1NM0tU?=
 =?utf-8?B?L2E2UjVCS1J5UE5UUWVhb3VyVUMyN3d5Vjh1TVA0cC9IU0pBU0xqWkFMTXR4?=
 =?utf-8?B?Sy9PeHhLbi9rNVZDUVdoc3MxOTVsVXh3ekFvcXZyM3Fuazc5Y2UwMUJDTnJC?=
 =?utf-8?B?VGZSTHJick80eitIUzEvWWJFZjc0bllrTzdBcHBwNHRlN25MMFBjalNzemJs?=
 =?utf-8?B?Tm9wT2syRWZRY3FxY1p6bXpzZTJrVUVTOUJ3OThSVnVHdlY2QXN6ZjJCV0Qv?=
 =?utf-8?B?Zm4wMmJmdjU4UEZPWmF4OXJpenFzaHZLalZVNW42TDM4K3M4RTkwaUZUb0pr?=
 =?utf-8?B?aEdkUm9Gei9GMERwbkMyaHJjYU1mMGh5Y2twaHQycmJJcjRhTTdYNEZWZ3VE?=
 =?utf-8?B?QThTR2dzdEVnSWJXTWhNWTdMK0JUeHUvMzRvbUY2TlViQzViMVEydXpDbGhq?=
 =?utf-8?B?ek5PR2J2ZWg2czhUeWlVSHFrTlA3RDR6aUlwMmVsUHlOQWd2bzFBTkxoRW5F?=
 =?utf-8?B?QnFiQmlLaEpxUW5vaURFUkJGTUl5eUVoNVJIV2VidHhtVlJUNTl3ZFNUUm5t?=
 =?utf-8?B?TlBmUWcyMUZUYS9oQ2UvNStBVGdEVjZBWTBrVmcxL2VreXh2WE9CWnJ5Y1JS?=
 =?utf-8?B?ZGhJOVRxdCtlWmMwY0MydXEyZlZjMGpOZFdzVDIxMktHRUtEdnN3SXRtVDRx?=
 =?utf-8?B?cDBXTVNEQlpjQmF5OVlOdnBjKzFZS01tVllsRnB3TlFtTEJBQmh0SmNldXoy?=
 =?utf-8?B?bmRvRWxRRlgwNkxkUFl2UGI3OENLYVJ6MkxuNkVYT2QvUkYrV2JMV1B4aElV?=
 =?utf-8?B?SkdNVzRicFF3TkRKV2h2WXMzWWQ0VENsNXZMUUgxNGUyNHFWK244RUJGK0dv?=
 =?utf-8?B?RG9JYkxPeW5pazB0V05wczFRQjJ0Wk52YkdxN0xZeXBiMG1wSlV4d0dMc0R4?=
 =?utf-8?B?Y0tRNWlXY2N3MXRCekx1QTB2ajlYT0NDQ3ZpeFNTczd2dkZzdFZpME9VakV6?=
 =?utf-8?B?dWR1WGNYMll6WlhiYnlpbHdQRTIrL0E0M054U3orUmJpTmcyMUJqcmcrb0Rk?=
 =?utf-8?B?L0hQcU9KdjNxTFNYSlVmYnpRWnFMMktRdnJiaWF5STY0RnQyRDZjUEQrdTJo?=
 =?utf-8?B?eFJJOUtEaGh5RXB1UzdnOWdBMXVGdFp5R1Z6RVZvZHlIZHJpbFpVMVo1RFZE?=
 =?utf-8?B?NUdZcVVlSi9BYXZUU0thZm9Wek14L2tTQnhkUW8vNFZKaVY4MGd1UGxWbDEr?=
 =?utf-8?B?VFhBTSs4YUk5UFFHRTlQZXRkZWh0V0tkL2tNdEdoVVpBTDVJRVlUaHV5RzFM?=
 =?utf-8?B?ZjdEVytkT2NpWjZlbUtYQ3IvNzNKTUtmTmk2YmFzZjNkS0prUDREd2tGZmRo?=
 =?utf-8?B?cHVBK04vKzV4VWFLOGM4QkwvWGgyYUVxRkJVYWlmeitvZWRZNXRzRTgrUzlq?=
 =?utf-8?Q?jYDRTBy/Ue4rrmhLquAKL8DRe9pJOJWWfLDg0n5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d1f2f27e-083b-460e-d624-08d97f411155
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:52:49.1245
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rwXwJ/lgjokpY4GE2GUligQaF6vc3OtxUQ7NJujFQPkEaam4oFoouUWzbwk4qUyUfz9EZ16e80Zn0NR7FpMEbQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

... depending on feature availability (and absence of quirks).

Also make the page table dumping function aware of superpages.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -743,18 +743,37 @@ static int __must_check iommu_flush_iotl
     return iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
 }
 
+static void queue_free_pt(struct domain *d, mfn_t mfn, unsigned int next_level)
+{
+    if ( next_level > 1 )
+    {
+        struct dma_pte *pt = map_domain_page(mfn);
+        unsigned int i;
+
+        for ( i = 0; i < PTE_NUM; ++i )
+            if ( dma_pte_present(pt[i]) && !dma_pte_superpage(pt[i]) )
+                queue_free_pt(d, maddr_to_mfn(dma_pte_addr(pt[i])),
+                              next_level - 1);
+
+        unmap_domain_page(pt);
+    }
+
+    iommu_queue_free_pgtable(d, mfn_to_page(mfn));
+}
+
 /* clear one page's page table */
 static int dma_pte_clear_one(struct domain *domain, daddr_t addr,
                              unsigned int order,
                              unsigned int *flush_flags)
 {
     struct domain_iommu *hd = dom_iommu(domain);
-    struct dma_pte *page = NULL, *pte = NULL;
+    struct dma_pte *page = NULL, *pte = NULL, old;
     u64 pg_maddr;
+    unsigned int level = (order / LEVEL_STRIDE) + 1;
 
     spin_lock(&hd->arch.mapping_lock);
-    /* get last level pte */
-    pg_maddr = addr_to_dma_page_maddr(domain, addr, 1, flush_flags, false);
+    /* get target level pte */
+    pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, false);
     if ( pg_maddr < PAGE_SIZE )
     {
         spin_unlock(&hd->arch.mapping_lock);
@@ -762,7 +781,7 @@ static int dma_pte_clear_one(struct doma
     }
 
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
-    pte = page + address_level_offset(addr, 1);
+    pte = &page[address_level_offset(addr, level)];
 
     if ( !dma_pte_present(*pte) )
     {
@@ -771,14 +790,19 @@ static int dma_pte_clear_one(struct doma
         return 0;
     }
 
+    old = *pte;
     dma_clear_pte(*pte);
-    *flush_flags |= IOMMU_FLUSHF_modified;
 
     spin_unlock(&hd->arch.mapping_lock);
     iommu_sync_cache(pte, sizeof(struct dma_pte));
 
     unmap_vtd_domain_page(page);
 
+    *flush_flags |= IOMMU_FLUSHF_modified;
+
+    if ( level > 1 && !dma_pte_superpage(old) )
+        queue_free_pt(domain, maddr_to_mfn(dma_pte_addr(old)), level - 1);
+
     return 0;
 }
 
@@ -1868,6 +1892,7 @@ static int __must_check intel_iommu_map_
     struct domain_iommu *hd = dom_iommu(d);
     struct dma_pte *page, *pte, old, new = {};
     u64 pg_maddr;
+    unsigned int level = (IOMMUF_order(flags) / LEVEL_STRIDE) + 1;
     int rc = 0;
 
     /* Do nothing if VT-d shares EPT page table */
@@ -1892,7 +1917,7 @@ static int __must_check intel_iommu_map_
         return 0;
     }
 
-    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1, flush_flags,
+    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), level, flush_flags,
                                       true);
     if ( pg_maddr < PAGE_SIZE )
     {
@@ -1901,13 +1926,15 @@ static int __must_check intel_iommu_map_
     }
 
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
-    pte = &page[dfn_x(dfn) & LEVEL_MASK];
+    pte = &page[address_level_offset(dfn_to_daddr(dfn), level)];
     old = *pte;
 
     dma_set_pte_addr(new, mfn_to_maddr(mfn));
     dma_set_pte_prot(new,
                      ((flags & IOMMUF_readable) ? DMA_PTE_READ  : 0) |
                      ((flags & IOMMUF_writable) ? DMA_PTE_WRITE : 0));
+    if ( IOMMUF_order(flags) )
+        dma_set_pte_superpage(new);
 
     /* Set the SNP on leaf page table if Snoop Control available */
     if ( iommu_snoop )
@@ -1928,8 +1955,13 @@ static int __must_check intel_iommu_map_
 
     *flush_flags |= IOMMU_FLUSHF_added;
     if ( dma_pte_present(old) )
+    {
         *flush_flags |= IOMMU_FLUSHF_modified;
 
+        if ( level > 1 && !dma_pte_superpage(old) )
+            queue_free_pt(d, maddr_to_mfn(dma_pte_addr(old)), level - 1);
+    }
+
     return rc;
 }
 
@@ -2286,6 +2318,7 @@ static int __init vtd_setup(void)
 {
     struct acpi_drhd_unit *drhd;
     struct vtd_iommu *iommu;
+    unsigned int large_sizes = PAGE_SIZE_2M | PAGE_SIZE_1G;
     int ret;
     bool reg_inval_supported = true;
 
@@ -2328,6 +2361,11 @@ static int __init vtd_setup(void)
                cap_sps_2mb(iommu->cap) ? ", 2MB" : "",
                cap_sps_1gb(iommu->cap) ? ", 1GB" : "");
 
+        if ( !cap_sps_2mb(iommu->cap) )
+            large_sizes &= ~PAGE_SIZE_2M;
+        if ( !cap_sps_1gb(iommu->cap) )
+            large_sizes &= ~PAGE_SIZE_1G;
+
 #ifndef iommu_snoop
         if ( iommu_snoop && !ecap_snp_ctl(iommu->ecap) )
             iommu_snoop = false;
@@ -2399,6 +2437,9 @@ static int __init vtd_setup(void)
     if ( ret )
         goto error;
 
+    ASSERT(iommu_ops.page_sizes & PAGE_SIZE_4K);
+    iommu_ops.page_sizes |= large_sizes;
+
     register_keyhandler('V', vtd_dump_iommu_info, "dump iommu info", 1);
 
     return 0;
@@ -2712,7 +2753,7 @@ static void vtd_dump_page_table_level(pa
             continue;
 
         address = gpa + offset_level_address(i, level);
-        if ( next_level >= 1 ) 
+        if ( next_level && !dma_pte_superpage(*pte) )
             vtd_dump_page_table_level(dma_pte_addr(*pte), next_level,
                                       address, indent + 1);
         else



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:54:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:54:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195163.347729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThuA-0006AP-Ah; Fri, 24 Sep 2021 09:54:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195163.347729; Fri, 24 Sep 2021 09:54:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThuA-0006AI-7K; Fri, 24 Sep 2021 09:54:06 +0000
Received: by outflank-mailman (input) for mailman id 195163;
 Fri, 24 Sep 2021 09:54:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThu9-0006A4-Gt
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:54:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5960d94e-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:54:04 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2059.outbound.protection.outlook.com [104.47.10.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-HQz8LnudOXeUL3TlzMVZVQ-1; Fri, 24 Sep 2021 11:54:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:54:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:54:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0060.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Fri, 24 Sep 2021 09:53:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5960d94e-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477243;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XkEZg476K/l+ml14jRUFl4XZ13rCjdSGL7S/KURFmd8=;
	b=nY9BVqRH/JTXQ+m+PQKaQvTYRDJUMg7in0c10AXtcpdn2qml9l6IGT4nYJixmGcoqQZAzp
	IwinRZWEzvtpnf/yRv/ILZW71BvRl1P1pTC3frz9AbqO/VSPlMiF8X6nSYWGEHDjH8wf6Q
	ZKJaN672W0z3Fu7IohdacAEpsU+B4AU=
X-MC-Unique: HQz8LnudOXeUL3TlzMVZVQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=i3Szt+/nQYzoi/BIjPZ3nCcFbN+hcj55Dw806BEJ2tVe1kQmHB5Z7A/+9F7RAqS7LQQ8IFWffAeEv9wVAl7zSJ1DuqTNoYJd3unt55ayciIsVpyAazSq1siaR4ilosC9Tm8AcenCgk5RqoEWtJqxDu0WQsQnto8R+z4bNHtMjqyIKZedtMcQZKvEDAWWm20OzQrOCcPXCRH4mqqpH5L6ur2vYBiGBI5uvzhxELpRor2i/J+Fe3L+xzP5pdD/O2VWeAkYVU2oByBNv7s4UUB041ktcNwN2A/xSjKlM8RJgzChLVBC1efrRmy5x9q0KxVDq8n6m+oJaX84s6RyCOCjmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XkEZg476K/l+ml14jRUFl4XZ13rCjdSGL7S/KURFmd8=;
 b=H8mB2x4XbKOFoA3HnecGkcELRNNN4x6LVA3OGieucG+Ub25Cb7RvuOcJcJti4ZkzndeKPbqYo5VbMHbSpx7mDzatSfMP/5A7PlKyJgG5O7FnUhVM4CU012a3nSUUView+F5U2Lcb0lg2CxuJEstp5fhnmzuWP8oVLcSMXFhxg6u493ineACvaH75IA2W3HKRp1D6fHE7eRJSVWOOHuk1veibWEGoBw9C3XzaIPfvWqHK7qRW/pOiiqEgtlmxqDsjjvJGHGsu5sEixEuJG5wE1xfxpqjfJSDRL9FUHI5TBPEVrYjQdwbXltPowzO44/0iS8jpje7zPvFtCSklohcOmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 14/18] IOMMU: fold flush-all hook into "flush one"
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <Rahul.Singh@arm.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <e40ee980-9151-101a-1484-b1710aaeafb0@suse.com>
Date: Fri, 24 Sep 2021 11:53:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0060.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c6380498-4a4e-4697-3c47-08d97f413bb2
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71506C14C1ABB14A8BB17453B3A49@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	63TupDoMFKOxoa99SbE+BbE3cjZdgrbtHknmoeQkBztCaGZxipsoF002sCuLr6isl1fQD94blRpLrRpktPN3F0cv9UeVEbw3zmzFofs+QbrHdu4vETyjNtD2gX2jVIVFoj1iU7ZVB0bq7VMkzd3RMQe4LgSjHt/kSso+e7LjMYINhAzlrTc2SbtP0FSS4UCPtSdq/WTofhGjVIHy2RepuG2cJwBkvK0o1WMHkssP2iPwLJnQZa2+qlVjVrwYpnY70OKvycebvR5KDsJxSb6DtmlbOKduYcPPxCnty9TKfR1mkPBmd1WIX3stHKJMRN1UzlkHM4BcLdWLTFNFaXKAtccxClO9DSfc1xueQaAFxZQVBuRp12/T7FcwpgQ5j86W8FTUATRVSwROIR54K4w0NBIfY4rv2jhBP9k2wESjUGFhlHvQSPnbXBc30LeFcDeBzbCDsKtQ0D3/kuyJXrPYHFLtVR8hOXGADVwdTzkNU7nynebqZSlnphYSgeIe2GPX5kTw4aWE1NZKTyYey67tKhtR+hQ+VwJT58uT/0rS/NWKjFLSiUOWXUyifxLeoUJr7Xoaxe7i6c0hh11qeSn8gcl7Uyiq3iqxLmFsWR7+ZWF6GLTPelEEQPSfD9y3SfW06Vzb8hYLM3eIOD/s/90GAw34lszTk5l0hz8b7nCUpFaszmIjjQvF60YJBX+ycI33wnZDk5mntSfvi2YL+9emluYhBC83eRHYL8oyDFwPhVY=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(8936002)(5660300002)(2616005)(508600001)(31686004)(956004)(38100700002)(66476007)(66946007)(26005)(8676002)(66556008)(6916009)(2906002)(36756003)(86362001)(6486002)(16576012)(31696002)(83380400001)(4326008)(316002)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MUttSFF1RW11NnA0M0dockJxVUkwMy9yYUk3YXg5dytqeGtGZnF1bG1nd1Nk?=
 =?utf-8?B?SmVwRW9lR0ZCV3hjOExCeWVLbFZnVkhRSytTV21UUlZtSGp1akhHRUh4MHpZ?=
 =?utf-8?B?OGZQTlVBSFZra3VpSFdlcERxUHIxNlV1SG45WWErU09FanhFeWwva3ExSURP?=
 =?utf-8?B?eE14aVZtZDlGekNTbklkdWIxMEZHSC8rNEpFelM2cUt2V1owcG9VU2FwQUtK?=
 =?utf-8?B?WjlBWHo5Y2hrYkd5azRaaVU4bnFWT0lSczNteWRRUXRkRE50THZ0eUx2M3ZQ?=
 =?utf-8?B?ZC9rLzdCWGVWQXIyTm5YTy9nVW91MU9QYWo3MzRzRmNIbHZiOE1LRGFvckJK?=
 =?utf-8?B?dVBOc25mQUVHb2pZTHlESzAxc0tsZC9FTG1FTWFHMmE1VURuUlRkTGU0aGF4?=
 =?utf-8?B?M3hJcDNpTlM0SWtLVmM1aXdoTlNpVUtCcSswQmswd1M2dGFRVkV4cktjOWFh?=
 =?utf-8?B?S1JYalVja1M1REZMZW1vUGZmL3BHOUszaUlhczRiSVBGdUxGL2s0UHlkYlhm?=
 =?utf-8?B?eCtrTEdFeU4yWWJuSFN4b1VOVlhGeUY5c0ZGVUwxWjhuRFlicjBtYmh1VUFu?=
 =?utf-8?B?K2hpckdob2dCRi9jWklCa0pXUVJtbFIxYlM1cFBLcVRQYnRVN2VmYTgzL3dr?=
 =?utf-8?B?QzZVN2dsMU5UU3ByZ3owV2FQaUpqQmJUc0NNTElLbERMRUlsRFBKekdCVXN6?=
 =?utf-8?B?RmZoMWJWM1o3UGZyaUcxYmtCSURXVXBjclh5cGl0VjV5NXdHb0JCSS9YZjk2?=
 =?utf-8?B?V3oyVzVhd29YcjBOUXRXdUMrVk92VE1zTHh2WjR5SkROU1RxK2V2YTNCMWp2?=
 =?utf-8?B?cTBpVDIrbmFzeFQ3RXA2M2Zxa2FsNHl6clJuUmt2V2U3bEFJQnNoN0JFanhz?=
 =?utf-8?B?RXBrNlR3ejlhaFFaMVhmTS9TeHdMbUowdjYwd1NVQUpMMDM0WlJmRTVWZDgr?=
 =?utf-8?B?N3pJV3pkZDkza1dSdmdNbUFZNWNpYlRxZ29OejM4TTF4dXp2N1BNU3R3WE5i?=
 =?utf-8?B?VDhOY25LcXh5aFArTE9ocnZ5cHRBblhWaVlKWkpvUmZJRjRTSk1TeFY4bERL?=
 =?utf-8?B?S3NzT0laYjBud2t1a0NvNFI0b2hjaGVGUjJyUm0zdXVMcGV4dTkrNE10Uysx?=
 =?utf-8?B?ZGtlUVdlOTdqVmtwTXhQQmpJdFpIS2cxS2doVHJpVHBYNXNoOTA2dG5ocDQy?=
 =?utf-8?B?ZjZJWHN5c0YwUU5VS0FsQnRFOW9NZFNzZzAvTEtxK1grRFZTMkJ4UG9QM29z?=
 =?utf-8?B?ajMvVmpLQjl3WWNvczUydkEwVGRRcGRCSUx4Y0ZsZDcydkFMSXA5ZEIxcWhw?=
 =?utf-8?B?T2FOYkY1SytPL0ZldkZNK01pQ0lxc0dod3hmYkZQWXJUMkhneVlsRmRBdG5D?=
 =?utf-8?B?a2JtcjRnZjB5NDB3U3JPWjg4eUMrWjFNMFBaYytwMlZvNjRUM25sNkRyOHo5?=
 =?utf-8?B?S0pRMUE3dTY5MC9wa0tXcDQxWWZuelAwMHhOZitwSzR1SzhPcitNRDc3NUVM?=
 =?utf-8?B?TGF3cGt6WXBFTlp5NTFCSW1EUGs5em1iKzlHSXlFcWc0L29lTzRHOWxKbFls?=
 =?utf-8?B?QVd0TkwvQk9hWkpEakdTUVYwdDM5cFR1cEt1elYzVGpOaUJMMC91N1pHbG1y?=
 =?utf-8?B?L0NjTlNDN0FIT1R5dTdaL29lS1VxZnF2SUpuQld2eTNSd2dGRi8rQWMveWdj?=
 =?utf-8?B?NDJ0dEx4aTRpbkJIbS9iK1UyVldUOFJMalZoK0tqWmZENDZhTFM4SkY3YnQ2?=
 =?utf-8?Q?0cVFeIyxlts1awDo+gua+2XpRk/q+Z0WIHSykQy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c6380498-4a4e-4697-3c47-08d97f413bb2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:54:00.1853
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FUqsrO/XBh4/c+LL6ef8fNj7GrLGp5c4qSbVvdJx2QxTJeZwCnG3SvRx8bXdmIdwzRh+4URu76jz0Sq/2oj08w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

Having a separate flush-all hook has always been puzzling me some. We
will want to be able to force a full flush via accumulated flush flags
from the map/unmap functions. Introduce a respective new flag and fold
all flush handling to use the single remaining hook.

Note that because of the respective comments in SMMU and IPMMU-VMSA
code, I've folded the two prior hook functions into one. For SMMU-v3,
which lacks a comment towards incapable hardware, I've left both
functions in place on the assumption that selective and full flushes
will eventually want separating.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: What we really are going to need is for the map/unmap functions to
     specify that a wider region needs flushing than just the one
     covered by the present set of (un)maps. This may still be less than
     a full flush, but at least as a first step it seemed better to me
     to keep things simple and go the flush-all route.
---
v2: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -242,7 +242,6 @@ int amd_iommu_get_reserved_device_memory
 int __must_check amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
                                              unsigned long page_count,
                                              unsigned int flush_flags);
-int __must_check amd_iommu_flush_iotlb_all(struct domain *d);
 void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id,
                              dfn_t dfn);
 
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -475,15 +475,18 @@ int amd_iommu_flush_iotlb_pages(struct d
 {
     unsigned long dfn_l = dfn_x(dfn);
 
-    ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
-    ASSERT(flush_flags);
+    if ( !(flush_flags & IOMMU_FLUSHF_all) )
+    {
+        ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
+        ASSERT(flush_flags);
+    }
 
     /* Unless a PTE was modified, no flush is required */
     if ( !(flush_flags & IOMMU_FLUSHF_modified) )
         return 0;
 
-    /* If the range wraps then just flush everything */
-    if ( dfn_l + page_count < dfn_l )
+    /* If so requested or if the range wraps then just flush everything. */
+    if ( (flush_flags & IOMMU_FLUSHF_all) || dfn_l + page_count < dfn_l )
     {
         amd_iommu_flush_all_pages(d);
         return 0;
@@ -508,13 +511,6 @@ int amd_iommu_flush_iotlb_pages(struct d
 
     return 0;
 }
-
-int amd_iommu_flush_iotlb_all(struct domain *d)
-{
-    amd_iommu_flush_all_pages(d);
-
-    return 0;
-}
 
 int amd_iommu_reserve_domain_unity_map(struct domain *d,
                                        const struct ivrs_unity_map *map,
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -642,7 +642,6 @@ static const struct iommu_ops __initcons
     .map_page = amd_iommu_map_page,
     .unmap_page = amd_iommu_unmap_page,
     .iotlb_flush = amd_iommu_flush_iotlb_pages,
-    .iotlb_flush_all = amd_iommu_flush_iotlb_all,
     .reassign_device = reassign_device,
     .get_device_group_id = amd_iommu_group_id,
     .enable_x2apic = iov_enable_xt,
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -930,13 +930,19 @@ out:
 }
 
 /* Xen IOMMU ops */
-static int __must_check ipmmu_iotlb_flush_all(struct domain *d)
+static int __must_check ipmmu_iotlb_flush(struct domain *d, dfn_t dfn,
+                                          unsigned long page_count,
+                                          unsigned int flush_flags)
 {
     struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
 
+    ASSERT(flush_flags);
+
     if ( !xen_domain || !xen_domain->root_domain )
         return 0;
 
+    /* The hardware doesn't support selective TLB flush. */
+
     spin_lock(&xen_domain->lock);
     ipmmu_tlb_invalidate(xen_domain->root_domain);
     spin_unlock(&xen_domain->lock);
@@ -944,16 +950,6 @@ static int __must_check ipmmu_iotlb_flus
     return 0;
 }
 
-static int __must_check ipmmu_iotlb_flush(struct domain *d, dfn_t dfn,
-                                          unsigned long page_count,
-                                          unsigned int flush_flags)
-{
-    ASSERT(flush_flags);
-
-    /* The hardware doesn't support selective TLB flush. */
-    return ipmmu_iotlb_flush_all(d);
-}
-
 static struct ipmmu_vmsa_domain *ipmmu_get_cache_domain(struct domain *d,
                                                         struct device *dev)
 {
@@ -1303,7 +1299,6 @@ static const struct iommu_ops ipmmu_iomm
     .hwdom_init      = ipmmu_iommu_hwdom_init,
     .teardown        = ipmmu_iommu_domain_teardown,
     .iotlb_flush     = ipmmu_iotlb_flush,
-    .iotlb_flush_all = ipmmu_iotlb_flush_all,
     .assign_device   = ipmmu_assign_device,
     .reassign_device = ipmmu_reassign_device,
     .map_page        = arm_iommu_map_page,
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -2649,11 +2649,17 @@ static int force_stage = 2;
  */
 static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK;
 
-static int __must_check arm_smmu_iotlb_flush_all(struct domain *d)
+static int __must_check arm_smmu_iotlb_flush(struct domain *d, dfn_t dfn,
+					     unsigned long page_count,
+					     unsigned int flush_flags)
 {
 	struct arm_smmu_xen_domain *smmu_domain = dom_iommu(d)->arch.priv;
 	struct iommu_domain *cfg;
 
+	ASSERT(flush_flags);
+
+	/* ARM SMMU v1 doesn't have flush by VMA and VMID */
+
 	spin_lock(&smmu_domain->lock);
 	list_for_each_entry(cfg, &smmu_domain->contexts, list) {
 		/*
@@ -2670,16 +2676,6 @@ static int __must_check arm_smmu_iotlb_f
 	return 0;
 }
 
-static int __must_check arm_smmu_iotlb_flush(struct domain *d, dfn_t dfn,
-					     unsigned long page_count,
-					     unsigned int flush_flags)
-{
-	ASSERT(flush_flags);
-
-	/* ARM SMMU v1 doesn't have flush by VMA and VMID */
-	return arm_smmu_iotlb_flush_all(d);
-}
-
 static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 						struct device *dev)
 {
@@ -2879,7 +2875,6 @@ static const struct iommu_ops arm_smmu_i
     .add_device = arm_smmu_dt_add_device_generic,
     .teardown = arm_smmu_iommu_domain_teardown,
     .iotlb_flush = arm_smmu_iotlb_flush,
-    .iotlb_flush_all = arm_smmu_iotlb_flush_all,
     .assign_device = arm_smmu_assign_dev,
     .reassign_device = arm_smmu_reassign_dev,
     .map_page = arm_iommu_map_page,
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -3431,7 +3431,6 @@ static const struct iommu_ops arm_smmu_i
 	.hwdom_init		= arm_smmu_iommu_hwdom_init,
 	.teardown		= arm_smmu_iommu_xen_domain_teardown,
 	.iotlb_flush		= arm_smmu_iotlb_flush,
-	.iotlb_flush_all	= arm_smmu_iotlb_flush_all,
 	.assign_device		= arm_smmu_assign_dev,
 	.reassign_device	= arm_smmu_reassign_dev,
 	.map_page		= arm_iommu_map_page,
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -463,15 +463,12 @@ int iommu_iotlb_flush_all(struct domain
     const struct domain_iommu *hd = dom_iommu(d);
     int rc;
 
-    if ( !is_iommu_enabled(d) || !hd->platform_ops->iotlb_flush_all ||
+    if ( !is_iommu_enabled(d) || !hd->platform_ops->iotlb_flush ||
          !flush_flags )
         return 0;
 
-    /*
-     * The operation does a full flush so we don't need to pass the
-     * flush_flags in.
-     */
-    rc = iommu_call(hd->platform_ops, iotlb_flush_all, d);
+    rc = iommu_call(hd->platform_ops, iotlb_flush, d, INVALID_DFN, 0,
+                    flush_flags | IOMMU_FLUSHF_all);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -731,18 +731,21 @@ static int __must_check iommu_flush_iotl
                                                 unsigned long page_count,
                                                 unsigned int flush_flags)
 {
-    ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
-    ASSERT(flush_flags);
+    if ( flush_flags & IOMMU_FLUSHF_all )
+    {
+        dfn = INVALID_DFN;
+        page_count = 0;
+    }
+    else
+    {
+        ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
+        ASSERT(flush_flags);
+    }
 
     return iommu_flush_iotlb(d, dfn, flush_flags & IOMMU_FLUSHF_modified,
                              page_count);
 }
 
-static int __must_check iommu_flush_iotlb_all(struct domain *d)
-{
-    return iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
-}
-
 static void queue_free_pt(struct domain *d, mfn_t mfn, unsigned int next_level)
 {
     if ( next_level > 1 )
@@ -2841,7 +2844,7 @@ static int __init intel_iommu_quarantine
     spin_unlock(&hd->arch.mapping_lock);
 
     if ( !rc )
-        rc = iommu_flush_iotlb_all(d);
+        rc = iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
 
     /* Pages may be leaked in failure case */
     return rc;
@@ -2874,7 +2877,6 @@ static struct iommu_ops __initdata vtd_o
     .resume = vtd_resume,
     .crash_shutdown = vtd_crash_shutdown,
     .iotlb_flush = iommu_flush_iotlb_pages,
-    .iotlb_flush_all = iommu_flush_iotlb_all,
     .get_reserved_device_memory = intel_iommu_get_reserved_device_memory,
     .dump_page_tables = vtd_dump_page_tables,
 };
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -147,9 +147,11 @@ enum
 {
     _IOMMU_FLUSHF_added,
     _IOMMU_FLUSHF_modified,
+    _IOMMU_FLUSHF_all,
 };
 #define IOMMU_FLUSHF_added (1u << _IOMMU_FLUSHF_added)
 #define IOMMU_FLUSHF_modified (1u << _IOMMU_FLUSHF_modified)
+#define IOMMU_FLUSHF_all (1u << _IOMMU_FLUSHF_all)
 
 int __must_check iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn,
                            unsigned long page_count, unsigned int flags,
@@ -282,7 +284,6 @@ struct iommu_ops {
     int __must_check (*iotlb_flush)(struct domain *d, dfn_t dfn,
                                     unsigned long page_count,
                                     unsigned int flush_flags);
-    int __must_check (*iotlb_flush_all)(struct domain *d);
     int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
     void (*dump_page_tables)(struct domain *d);
 



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:55:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:55:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195168.347741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThv8-0006lj-Ks; Fri, 24 Sep 2021 09:55:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195168.347741; Fri, 24 Sep 2021 09:55:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThv8-0006lc-HK; Fri, 24 Sep 2021 09:55:06 +0000
Received: by outflank-mailman (input) for mailman id 195168;
 Fri, 24 Sep 2021 09:55:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThv7-0006lS-HA
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:55:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7d369976-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:55:04 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-9FElwOWLNf6AuWWuEBVJcw-1; Fri, 24 Sep 2021 11:55:02 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:54:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:54:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0064.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 09:54:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d369976-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477303;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2Xjl2PorN1TigRAzYAHXhOxQtJC1FCMn+69GJeBFkFk=;
	b=gvmMyyq95yPoaWyYTsQy88rYU9RIbbU4ghEzdgZoTbPmxeK7SFiiDKSzoJWXlr520Ba7eJ
	V1d/I2fIW3pogPWuRBc8zddXmain4lNn0UZbv5ATVlaI9XjifRh6410MjpyjZTRd5BuXrn
	kMNPADMVpwQbjcpAoc4E9KCAE69u2ls=
X-MC-Unique: 9FElwOWLNf6AuWWuEBVJcw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ioK3fdrWVXG1xc4MwrXPL/jnz5GUkgYaB/0K6oeJuuak+76NQpjd+pWKcgrMdcBQV/qq4wJiWtYaXLy9or2tZKVSboaXBGFjVAXLEASPYnbkroq77Gy6VHVslwblZOxW0aHldlymuV20aBtzMb1N29EhRnih/c/Tt6Yei+qqLucJo017q8oSUGEU0f+bAJUYSjP4IZETHh2miyNduC4yM6Mptf5+71TZ/NSelD4lAJnjEMaQm9FVdc/4XwuhQkugOwJ49qofNMICXAABqQj9S+OS+WJ6Eex5xZ5F8D1Vr6eYZLbUEiXcYwCYnMuxshyAVQ1hPQmvsrF+5Kpvey1C/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2Xjl2PorN1TigRAzYAHXhOxQtJC1FCMn+69GJeBFkFk=;
 b=B+HkSGiowL/ncV++IFp0XHTZq0v3DPKawl3OD9TJGuddA0sZXRlTPo3frS2Ivvvgw9rgxXdxIU2QbvauZNd8v2Sot9VPUD6sajcdeJtyUI+79+zrqAA6dUhLfmSSzDMjneZDRcJZo2aayhUQMP42fA2xZPdLG9HC+ywSBFH9YWOzpyoq5547S/QT1OsDJ2n5LsOQjptqB6VHb2BoMs0R9p3xu6Txb3PqFsxqfr6xurBr3bBs/H85tXPxKzwGDYzb/9zmllkZOafi6PU9BivsD2Aau1kjvpAgNvxbQzBGs4UbFYM/ut4k4HRbwGySXiwhXrJG/hz0PYJuiN5UaZtlyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 15/18] IOMMU/x86: prefill newly allocate page tables
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <2656844d-47cc-70c3-d7ce-7d83967d576e@suse.com>
Date: Fri, 24 Sep 2021 11:54:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0064.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::8) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fd918ac0-7c5c-46a5-2530-08d97f415f0c
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71507FB3DDBAAF8B8FD05E82B3A49@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1468;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1+UFfH+THGlmBkk6yAKeH267tMxKAzscCarzLzcpubUqEX/758DicTtpHV0pqT0jwBdQ2FaxOJsqu9hMKVgJJe+X7+nJr7qNE5O4Gc/SR0K8cvKA1AauObl4at9eFVbcdbRh7Iw5FNOpx3+VAOulTJBhRi3c2UNENNZ8Cx0zSPTZjlCDScR0rc0b+nc4qQRAI/f5iKigkvQAnft6XN1/CGZxVhRpsRZN4HYkpV4LTxVyKWrc0NCw3IiW8h6rXL2utjcAKgqAGkxb0ngHxEJzOEgk5AtrpypZwOfwaSeUBpO4aRJp/hLUW9vZkmKsyOHEdKDCbpXb2gdOzuvrUGvOJhvUpxLfiQqx431tt/TVuP5JVQ97PHp4B2rSvzqXcmITQwydnt5hVXIIszu8QUSXnakRnYFBQg9sJiZkeHkBGdZSM3FMF33U+GTAdAe2TwrjPCvXhQL0OZu0VxkAWJjMgTgvxYRGm3GbBSHALlhjWlaIb1BDS4dyoBXcGDdyX9UYijuF7eTJ+cGuMQ6YNzRZwxW3f5/dfm59sX2Iir7Kq7mZUe5C0oBwTnsozSsni5MxDj+z+/mwxMElMCVUOBFsjYZwA7lautA3VVbWIiWReFeX0GMj/6f1jP2BNhZJi+58iZFuZHEtuV6LW9rJd03fVI8oIA+bCLqk0lEEM95VuSMFhWXgOef6ciENQojPwbkuMnN7BVXoUF7UWNkavm7qyL5a6agj5n92tGGI0Ob+1k4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(8936002)(5660300002)(2616005)(508600001)(31686004)(956004)(38100700002)(66476007)(66946007)(26005)(8676002)(66556008)(6916009)(2906002)(36756003)(86362001)(6486002)(16576012)(31696002)(83380400001)(4326008)(316002)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YUpuaDNtLyt3dkkwN2w1bElkeitiUmhtMHJEVkRucFB4SVdGbWdMRkVHNS9J?=
 =?utf-8?B?SnFzWmQrTWFqMnk2M1l4US9pTGczUnl5RlhKUVlRT0FLOE44QVltMjA0NFFQ?=
 =?utf-8?B?MjVEOCtJS3VSdXBIOVBCSjlPU1pwVUZ4dVZjSm9Wa2k5amxtT3lpcCtGV0Rm?=
 =?utf-8?B?Tk9PbjZsZ1RWVnhUOWJOLzY5R09NVkV2Wk95emhzakVhTGFpQWhFK1QxaWZi?=
 =?utf-8?B?OVpwbkJ1Wmh1eU0wa3VQLzl3UWkySlRxZXpTdGlkVU1GK2NQdEZoczQzMi9z?=
 =?utf-8?B?RUEzblcvRDc0YTJGL0ZXOGZ0ZGFDU2NLZDgwdXByelRrcEVBT1pvTVJDR0tC?=
 =?utf-8?B?QllDOHptcXdPOWJ2aDl5ZmpDaUtydEdCY0dQeUpOV0pzR3R1aUNBbVJwQitj?=
 =?utf-8?B?ZXdYTGhXY2w4UGNMVFBJZldPTWN3WjRCUTZRd2orTDMzdHBpa2h5WjFuREo1?=
 =?utf-8?B?eStpazlBYUNpWFZEcnVhZ3duRWVLQ1JYUUQ4TjZVYzBYRStBSEVSeDNWUk9k?=
 =?utf-8?B?Vi9DZ1ZMVXpnakdRaDdCK3YzN2pVR0xJN1Fkb1ZvbEIrOTRWckE3Q25EbU9o?=
 =?utf-8?B?MWtnMjA2SElBRUdQNVhzZSt3NlVxUWlwVmh2VUVVUmNnS1RlQWN2bGJxdStH?=
 =?utf-8?B?VkxRVmo4Z1FKMkNzb3FDeURueGRnVGpBeGRtU25oTEZvSFUxWTNnMkNFSzdG?=
 =?utf-8?B?OFEvS2RzMStCVDlUR1hZNjZpUnRwa1N2TVYyVlpJQ1pGSEVkSFcrSXJZY1F1?=
 =?utf-8?B?NGl1MGdpVFVnUlRHbzZnUXB3SmRVRlJOcHVDMldhUUVZMUhPakZLUFlnU01h?=
 =?utf-8?B?ZTFYenpzcCtUZHptR2UvVm5WbWtwUkhyZHhoZnJyQkplWjN3MTBuWmtTYVpL?=
 =?utf-8?B?SFBUOTZGSE9vcjFQbmJ2LytyOUc4bFd0a3diNHMwZHZ0TlhNZUhFMmRDeGdo?=
 =?utf-8?B?c2FUaE4yR3phSGs2cHA3YXl1MDAzYnhFcXVRSm1TNXRGa05DNlhDVzlUNVZJ?=
 =?utf-8?B?MjhITStTN2J4aXhObEpaMm05R0FvS3lPb0hyT0p5WGhvbkxkdytXaU9TWSto?=
 =?utf-8?B?REhQbTZTQUxhNTY2WkVURXlHS1UyNEdEMllKUmwxbTNPbzZpaVVzOWU3Slpt?=
 =?utf-8?B?TWZoSFpvTnh4elNPQkN1QSs1UWltb0ozRFp4VVc0WUhmQ3dqODNOcGpnKzE2?=
 =?utf-8?B?ZFRqVWRKT2FkUzhWbWt2VUZGMTJwOGZEMGc4bXkvajZXTVBOOURLYVgwME5L?=
 =?utf-8?B?c3pITi9ZUUJFZUlaLytLN1RpU1RPRWhqTEF0UTlzUS9Xc25zQklUMzJ3L096?=
 =?utf-8?B?UWp3SDNwUU4vR3I5YUNYTnd6M1JIZ3g2eFFLUG5QVCtVaUQxK3o4SzdKZG1z?=
 =?utf-8?B?V1VNZ2ZWampVZ2pVUWhtRFMxVG5IU0xsUWhDdWU5VnlDN2tUZWEyZnNZQUZY?=
 =?utf-8?B?Z2dQUDQ5MFBxM0Zsb21LUXB2ZHRDQ21XQTFrbmdnbGF2UkN2dUpiWngvTkQ5?=
 =?utf-8?B?R3VYeEdqclRqdkUxSjhvUzRCMG9JS0N2RklpNVUzVnNsTCtDZDRNTHduaDIx?=
 =?utf-8?B?bTJiU0ZSQUdOMG5ncElpM29WbFJoQ052cDBDdUQ2bXBsa3k0QjFKWk5zUWtJ?=
 =?utf-8?B?aU05aVlER3IvZ00rRGdYajlnOHVNcEl5T0xIM2JWelVZQnhSL1dGQTlxektD?=
 =?utf-8?B?THFhUXZCVHA3UXl3VUQrb1ZNL2lWL0hpVWNxT25XUU8wT29EaDNzUTFHZDhW?=
 =?utf-8?Q?i4jmI1wCOdv5toMJGTvD2ouV9oLUYCml68UqQ0u?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fd918ac0-7c5c-46a5-2530-08d97f415f0c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:54:59.6557
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: z4DSXSFrGMxQ/1fhA/3UdQZmSII+bRvG+AyZ4ZH2oNswy+xyDAJb5KV3vYWTaP65P8VtLRknhFlNuOsQwuRvcw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

Page table are used for two purposes after allocation: They either start
out all empty, or they get filled to replace a superpage. Subsequently,
to replace all empty or fully contiguous page tables, contiguous sub-
regions will be recorded within individual page tables. Install the
initial set of markers immediately after allocation. Make sure to retain
these markers when further populating a page table in preparation for it
to replace a superpage.

The markers are simply 4-bit fields holding the order value of
contiguous entries. To demonstrate this, if a page table had just 16
entries, this would be the initial (fully contiguous) set of markers:

index  0 1 2 3 4 5 6 7 8 9 A B C D E F
marker 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0

"Contiguous" here means not only present entries with successively
increasing MFNs, each one suitably aligned for its slot, but also a
respective number of all non-present entries.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
An alternative to the ASSERT()s added to set_iommu_ptes_present() would
be to make the function less general-purpose; it's used in a single
place only after all (i.e. it might as well be folded into its only
caller).
---
v2: New.

--- a/xen/drivers/passthrough/amd/iommu-defs.h
+++ b/xen/drivers/passthrough/amd/iommu-defs.h
@@ -445,6 +445,8 @@ union amd_iommu_x2apic_control {
 #define IOMMU_PAGE_TABLE_U32_PER_ENTRY	(IOMMU_PAGE_TABLE_ENTRY_SIZE / 4)
 #define IOMMU_PAGE_TABLE_ALIGNMENT	4096
 
+#define IOMMU_PTE_CONTIG_MASK           0x1e /* The ign0 field below. */
+
 union amd_iommu_pte {
     uint64_t raw;
     struct {
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -116,7 +116,19 @@ static void set_iommu_ptes_present(unsig
 
     while ( nr_ptes-- )
     {
-        set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
+        ASSERT(!pde->next_level);
+        ASSERT(!pde->u);
+
+        if ( pde > table )
+            ASSERT(pde->ign0 == find_first_set_bit(pde - table));
+        else
+            ASSERT(pde->ign0 == PAGE_SHIFT - 3);
+
+        pde->iw = iw;
+        pde->ir = ir;
+        pde->fc = true; /* See set_iommu_pde_present(). */
+        pde->mfn = next_mfn;
+        pde->pr = true;
 
         ++pde;
         next_mfn += page_sz;
@@ -232,7 +244,7 @@ static int iommu_pde_from_dfn(struct dom
             mfn = next_table_mfn;
 
             /* allocate lower level page table */
-            table = iommu_alloc_pgtable(d);
+            table = iommu_alloc_pgtable(d, IOMMU_PTE_CONTIG_MASK);
             if ( table == NULL )
             {
                 AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n");
@@ -262,7 +274,7 @@ static int iommu_pde_from_dfn(struct dom
 
             if ( next_table_mfn == 0 )
             {
-                table = iommu_alloc_pgtable(d);
+                table = iommu_alloc_pgtable(d, IOMMU_PTE_CONTIG_MASK);
                 if ( table == NULL )
                 {
                     AMD_IOMMU_DEBUG("Cannot allocate I/O page table\n");
@@ -648,7 +660,7 @@ int __init amd_iommu_quarantine_init(str
 
     spin_lock(&hd->arch.mapping_lock);
 
-    hd->arch.amd.root_table = iommu_alloc_pgtable(d);
+    hd->arch.amd.root_table = iommu_alloc_pgtable(d, 0);
     if ( !hd->arch.amd.root_table )
         goto out;
 
@@ -663,7 +675,7 @@ int __init amd_iommu_quarantine_init(str
          * page table pages, and the resulting allocations are always
          * zeroed.
          */
-        pg = iommu_alloc_pgtable(d);
+        pg = iommu_alloc_pgtable(d, 0);
         if ( !pg )
             break;
 
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -238,7 +238,7 @@ int amd_iommu_alloc_root(struct domain *
 
     if ( unlikely(!hd->arch.amd.root_table) )
     {
-        hd->arch.amd.root_table = iommu_alloc_pgtable(d);
+        hd->arch.amd.root_table = iommu_alloc_pgtable(d, 0);
         if ( !hd->arch.amd.root_table )
             return -ENOMEM;
     }
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -297,7 +297,7 @@ static uint64_t addr_to_dma_page_maddr(s
             goto out;
 
         pte_maddr = level;
-        if ( !(pg = iommu_alloc_pgtable(domain)) )
+        if ( !(pg = iommu_alloc_pgtable(domain, 0)) )
             goto out;
 
         hd->arch.vtd.pgd_maddr = page_to_maddr(pg);
@@ -339,7 +339,7 @@ static uint64_t addr_to_dma_page_maddr(s
             }
 
             pte_maddr = level - 1;
-            pg = iommu_alloc_pgtable(domain);
+            pg = iommu_alloc_pgtable(domain, DMA_PTE_CONTIG_MASK);
             if ( !pg )
                 break;
 
@@ -351,12 +351,13 @@ static uint64_t addr_to_dma_page_maddr(s
                 struct dma_pte *split = map_vtd_domain_page(pte_maddr);
                 unsigned long inc = 1UL << level_to_offset_bits(level - 1);
 
-                split[0].val = pte->val;
+                split[0].val |= pte->val & ~DMA_PTE_CONTIG_MASK;
                 if ( inc == PAGE_SIZE )
                     split[0].val &= ~DMA_PTE_SP;
 
                 for ( offset = 1; offset < PTE_NUM; ++offset )
-                    split[offset].val = split[offset - 1].val + inc;
+                    split[offset].val |=
+                        (split[offset - 1].val & ~DMA_PTE_CONTIG_MASK) + inc;
 
                 iommu_sync_cache(split, PAGE_SIZE);
                 unmap_vtd_domain_page(split);
@@ -1943,7 +1944,7 @@ static int __must_check intel_iommu_map_
     if ( iommu_snoop )
         dma_set_pte_snp(new);
 
-    if ( old.val == new.val )
+    if ( !((old.val ^ new.val) & ~DMA_PTE_CONTIG_MASK) )
     {
         spin_unlock(&hd->arch.mapping_lock);
         unmap_vtd_domain_page(page);
@@ -2798,7 +2799,7 @@ static int __init intel_iommu_quarantine
         goto out;
     }
 
-    pg = iommu_alloc_pgtable(d);
+    pg = iommu_alloc_pgtable(d, 0);
 
     rc = -ENOMEM;
     if ( !pg )
@@ -2817,7 +2818,7 @@ static int __init intel_iommu_quarantine
          * page table pages, and the resulting allocations are always
          * zeroed.
          */
-        pg = iommu_alloc_pgtable(d);
+        pg = iommu_alloc_pgtable(d, 0);
 
         if ( !pg )
             goto out;
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -265,6 +265,7 @@ struct dma_pte {
 #define DMA_PTE_PROT (DMA_PTE_READ | DMA_PTE_WRITE)
 #define DMA_PTE_SP   (1 << 7)
 #define DMA_PTE_SNP  (1 << 11)
+#define DMA_PTE_CONTIG_MASK  (0xfull << PADDR_BITS)
 #define dma_clear_pte(p)    do {(p).val = 0;} while(0)
 #define dma_set_pte_readable(p) do {(p).val |= DMA_PTE_READ;} while(0)
 #define dma_set_pte_writable(p) do {(p).val |= DMA_PTE_WRITE;} while(0)
@@ -278,7 +279,7 @@ struct dma_pte {
 #define dma_pte_write(p) (dma_pte_prot(p) & DMA_PTE_WRITE)
 #define dma_pte_addr(p) ((p).val & PADDR_MASK & PAGE_MASK_4K)
 #define dma_set_pte_addr(p, addr) do {\
-            (p).val |= ((addr) & PAGE_MASK_4K); } while (0)
+            (p).val |= ((addr) & PADDR_MASK & PAGE_MASK_4K); } while (0)
 #define dma_pte_present(p) (((p).val & DMA_PTE_PROT) != 0)
 #define dma_pte_superpage(p) (((p).val & DMA_PTE_SP) != 0)
 
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -433,12 +433,12 @@ int iommu_free_pgtables(struct domain *d
     return 0;
 }
 
-struct page_info *iommu_alloc_pgtable(struct domain *d)
+struct page_info *iommu_alloc_pgtable(struct domain *d, uint64_t contig_mask)
 {
     struct domain_iommu *hd = dom_iommu(d);
     unsigned int memflags = 0;
     struct page_info *pg;
-    void *p;
+    uint64_t *p;
 
 #ifdef CONFIG_NUMA
     if ( hd->node != NUMA_NO_NODE )
@@ -450,7 +450,28 @@ struct page_info *iommu_alloc_pgtable(st
         return NULL;
 
     p = __map_domain_page(pg);
-    clear_page(p);
+
+    if ( contig_mask )
+    {
+        unsigned int i, shift = find_first_set_bit(contig_mask);
+
+        ASSERT(((PAGE_SHIFT - 3) & (contig_mask >> shift)) == PAGE_SHIFT - 3);
+
+        p[0] = (PAGE_SHIFT - 3ull) << shift;
+        p[1] = 0;
+        p[2] = 1ull << shift;
+        p[3] = 0;
+
+        for ( i = 4; i < PAGE_SIZE / 8; i += 4 )
+        {
+            p[i + 0] = (find_first_set_bit(i) + 0ull) << shift;
+            p[i + 1] = 0;
+            p[i + 2] = 1ull << shift;
+            p[i + 3] = 0;
+        }
+    }
+    else
+        clear_page(p);
 
     if ( hd->platform_ops->sync_cache )
         iommu_vcall(hd->platform_ops, sync_cache, p, PAGE_SIZE);
--- a/xen/include/asm-x86/iommu.h
+++ b/xen/include/asm-x86/iommu.h
@@ -142,7 +142,8 @@ int pi_update_irte(const struct pi_desc
 })
 
 int __must_check iommu_free_pgtables(struct domain *d);
-struct page_info *__must_check iommu_alloc_pgtable(struct domain *d);
+struct page_info *__must_check iommu_alloc_pgtable(struct domain *d,
+                                                   uint64_t contig_mask);
 void iommu_queue_free_pgtable(struct domain *d, struct page_info *pg);
 
 #endif /* !__ARCH_X86_IOMMU_H__ */



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:55:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:55:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195175.347752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThvf-0007Od-1x; Fri, 24 Sep 2021 09:55:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195175.347752; Fri, 24 Sep 2021 09:55:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThve-0007OW-V6; Fri, 24 Sep 2021 09:55:38 +0000
Received: by outflank-mailman (input) for mailman id 195175;
 Fri, 24 Sep 2021 09:55:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThvc-0007OH-Vq
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:55:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9004f430-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:55:35 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-13-Rz5mZz5XPKqiIPn6C0XiNg-1; Fri, 24 Sep 2021 11:55:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:55:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:55:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0217.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1e::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 09:55:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9004f430-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477334;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=D2c15IMrfJ+Hh7ordOiYcIR+2DVc7QDDbgHC4O7HA/g=;
	b=Of1loTSoRrJPTrwKq04oH2nGUBwxwhFM0ipKRL2heHTQIuyhx9D/T9PPlbZyn9yhx9FnqN
	uB5NPTvx1driPEEq6iq6UFFhuKXDegoiIlyGXFVwgvxUdvCI6/sjQktL+77xF2nWpvn52d
	xim7JpEc84Khy/8jvB5UlP7gJ/RuSFA=
X-MC-Unique: Rz5mZz5XPKqiIPn6C0XiNg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BxaTx3GFrtAKwrJKnIR+Ah53uUqoEyJGQ5behSh9i9e4hBnp8oGDlTjd8pw8JPGPPTs43bzeLjTv/UAGG88E5RwrYXWF4RFejOCz1vOoKzskK+YF7uESLalNevaOYgfJJ4CJubPlndizV3GFrtW26DqjXYIoKA/J7JdvWUNhSuKFjnmJaXfz1/o/+ZiRFEhmSK5AAfNbb16vk8PcrQreCjYPJgKCmrgrmxbziLmGlIwSC7pZ/5AWFBvozJ9BFPy3AE4gZXI6ahesLDYDNZnQsgqqGpcPze9DO/d8Z/G+owbkqDT8CzMTD+HCB3YWQ8mBKO5Z189MsUaN04M00xYUSw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=D2c15IMrfJ+Hh7ordOiYcIR+2DVc7QDDbgHC4O7HA/g=;
 b=c3jKAKZpEMwp+ShSGw5fBGNYs9YMRiT1izHBcdKEQuUp/e4ZM949XdJogxWOlAhoXZMFCfLKAxGjjr01BGnqeXFwVTQJ9GSSiBEh4hWFB+FGLhtKCmJanr5tUv5pw61m+lID3dVcce8YURpNi27ldKWSFa5E8cY5KaVZB3OT6+dzxr0O3QM+HFxyw7tQqrTuQ5L0ZnVLBtNbMUiAA7Qv5Rb1gNVxPaDyk1cIg5srIyucIDUWIxYe6550fHyOVrWDfXhS222q3TtcbIlaHlFjhqlH7b50rBcvaZSYYOzC4A2VnHq0yuBW0gYTgR8E217pVimAXvoD7hAbBsVT4S2nhw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 16/18] x86: introduce helper for recording degree of
 contiguity in page tables
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <aab0b88b-7643-cc08-756b-0684f93be257@suse.com>
Date: Fri, 24 Sep 2021 11:55:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0217.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1e::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 104740b7-38b6-4852-81a1-08d97f417296
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB715029FD3ED2AD8F0DBDECFDB3A49@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zQQ9DEqa2ac9nHxTqMsIrcm6Hw2+HBxew1sZ/tXGKKRt+G/cyHRhzJD0NYc0xExF2xUY1dknXJnq0ewFOIZVT9VLYR8ADJ1eKwJmPR8QznCyFRkpqJa962MutLSKA60xeTb9JbRMaClGGiYBukIrAfULFRPuYSWrQU2+sM8WZPyrnvMlhi2RuB76mitmxJQuyv4BnKrk2PiMkn71kcHvfCVNtsr/BYvXcIoigdF8Hgr3S9qmo7fx8/uUug1TXjLuzQF5dsZFSYsUo1vB/9QuD9yafLUIxs9pcYFmSJo7+nU+S7aBcossOD7ylbIaGgJ+3Q2xLOGHauavls4nPNcYSfzJjzxFifuMDocrcDp9W0c3RIwszTGZBoKe8ge3GqHadwtMyWHYZiafDy4a5ubATOfPOjlra/8S5cvsXR5nj+jlPfPNNQcytFuEwxB1s5AFha/c5bPLzXpkLgKP992WxsouEyfCS43dRALDDFHGtetqXrqHgMEIJa/oxDhQhFzvhbmKQJwxXMRTvXX1v6tlU24fi5g5OVkpOkZZH+ytQHNw1RVHfE8qRAUL2c9bd+kv/dg3Q6J7/6nqAAmgA2vXZHOZAQPCusZBvfckMTxs7F/HBXDNCG0tqC3hoHPsb8QRTbIHY8NVLqbJA5wvuE2X5E7mRZcPoLmEpUuEjcg6Afg9/O+MwGpZ9W/WSyiFHxSFaGqpjOTiolFBrmvxL8xvV4uaxUJZHkOrkwcinnQgOXw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(8936002)(5660300002)(2616005)(508600001)(31686004)(956004)(38100700002)(66476007)(66946007)(26005)(8676002)(66556008)(6916009)(2906002)(36756003)(86362001)(6486002)(16576012)(31696002)(83380400001)(4326008)(316002)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cVhTNStJYmJIUVVQVWZLcnFmc21PMktYMktnZmxIM2Rsd0dLZVc3cTNQQUdl?=
 =?utf-8?B?Qm83djNKdzdRZzZyQ3c3NElYTWdWbzAxTEVWRGlmMnRFdkJ3RENwckpIVVhp?=
 =?utf-8?B?WjFEcXVnRnFBeE9rMDJKMURrTjVTc1F0UWdBbUxRdzc1Y3VTQi92QnE2UjNu?=
 =?utf-8?B?c2FkS0Q0L2x4Y2h4cTYwU05oL1dnZEVsZEEwVzJidStCdXBXY0lBRTkyZ3J6?=
 =?utf-8?B?c0wwOVIwSXhZWWlZbEtOZ0hyVTJqR3pOSnc5NW5kaEgrNHZjazFGZnhxeDdG?=
 =?utf-8?B?Sjg1SFZXMTdxYkNGS3FvNXBKRUJmL3NacGJaaWRHQk1lT3d6Y3hmMENaWElp?=
 =?utf-8?B?cDZiMG1PK0N0R1FYZms2MFphc0ZNWng2SEhQT1RIYlhCNTd5ZktDWlFsZklF?=
 =?utf-8?B?ZkYwYVRtRjdqZWxjdkZmNW1UdDZaaGRDNUlzdTdVdDR4K1B1RGx1K0x6c3Jq?=
 =?utf-8?B?VjZFaFY0QXozYjVtWHdkQ01ZVDdpb3ZFSHU3UUtxSG5vVlJpMXhDNWpBZXNG?=
 =?utf-8?B?U3NCbDVGcVpORmluWWNrTk5XS0VWOVlCOFRUWDRZc0c1ZlRpUGNieVJ4L3BC?=
 =?utf-8?B?cEh3cDFUeThEdFcyVjFibGh4SEErM2RMTG45U2RWU2V0THl4N3ZGVGZoZlc0?=
 =?utf-8?B?M1Nrd3U2YnRmdXpaYytsMTJwemp1YzJpdDVaN2VYSjAreGpacWZDdDQwckdC?=
 =?utf-8?B?dXdGdnZWN1Q1NVRSbFV5a3cxVVFobXRybjVCK0ZnWTB5M0RLdnBqSTFDNjhw?=
 =?utf-8?B?d2x6OTFYZ2RFTXNOR0NrNWlIYWJZTFRML1o4NDM0bVFTU1crc0hWV2J5SUZy?=
 =?utf-8?B?UGhJRGg2MFFmL0YzYkUwakpIUVhOTnVhblBLMVI2eHB4RXJnaTk5eWN5amgy?=
 =?utf-8?B?T1AxdWFMUHNXcmVKUzJtNHNqcmRDVkVVN3hqb1N3MEp3MC9YSVlML0E0RTJy?=
 =?utf-8?B?MVl4NkRJTVRwUVZpOUVJZ3VXOVErWXB5eksvWkt4Y2VlKy9BaWpvTks5TTFo?=
 =?utf-8?B?bmoyWUdjZlBYV1lyZ2lsY3poeGNidUxtRTRmWkFpNTd3S1NuRXEyTEFlRXQ4?=
 =?utf-8?B?cVZYck5qMXViK3J0TDhQS2ZNZmpDbyt2Zmo5aGVGZVZxY2xqNlh2Z0UrNGN2?=
 =?utf-8?B?MGk1TEVMUkNVQ0NwbDZKWXo2RDRyclBWUVhuVmpqblVaOTZyY2xCbjVaR2do?=
 =?utf-8?B?ZkhHOFl1Qll0Y0x6Nll2OVFBZHBtVzZMZGtZNUV6UGhWUm1oZlFVRWh4dy82?=
 =?utf-8?B?eGNjejVqSTJUSTFQMFlOMGk0LzJoWTRlN1Jna1dCU2VHMWR4TFl2bTVTZ0p3?=
 =?utf-8?B?V2ZMMXE4dlNkcTkyTDR1K3B4eWl6WnBIaHhya1kvUWVWU05HRVR2M05BWVJG?=
 =?utf-8?B?YjRoMVRaWnpOTEhMSWVoUld1cTVDTUZJdVI0VXRNNEF2cFNyeXRpcmRxWGpk?=
 =?utf-8?B?NVh3VUJpNWd1NUZoNnN4UmFDWjNua2Q3ZGZyY2tpNmZSTGlmVG1QNDBsQWJX?=
 =?utf-8?B?SDMvWUI1a2tWV1FYeDJvcFJGVEVIbm5JQUNjOUxGL3BzeGlJMmdzZFlBODB2?=
 =?utf-8?B?SVlReVpxcVlsSGtxdDJPekRLRU0ya1VOYnNJRTJSakdsa1V4cWpzWFJYblBn?=
 =?utf-8?B?WEdPOEQrQW1JclBpb1JTakc5cWxSK2JIaVdsSGx0LzVzbjQrV0haeEcrejBp?=
 =?utf-8?B?ZGxPK2x2RGtxVXRiSlFObUFLR0lCOWRZUlRVWWhVUzRoSlFyTlMvMXRwNEtK?=
 =?utf-8?Q?2vPsq2kdjeDTDnYZE9O9KsWHaZ9FkhSL7oR0UUu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 104740b7-38b6-4852-81a1-08d97f417296
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:55:32.2813
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: la014zidY3Bd4wGrF+6awJep5bkaH4TaLcgf8knf3IT3lmuQhX/fBOmN/+Wf3k5mm/ZZWup71NowDFHBoTJ4FA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

This is a re-usable helper (kind of a template) which gets introduced
without users so that the individual subsequent patches introducing such
users can get committed independently of one another.

See the comment at the top of the new file. To demonstrate the effect,
if a page table had just 16 entries, this would be the set of markers
for a page table with fully contiguous mappings:

index  0 1 2 3 4 5 6 7 8 9 A B C D E F
marker 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0

"Contiguous" here means not only present entries with successively
increasing MFNs, each one suitably aligned for its slot, but also a
respective number of all non-present entries.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- /dev/null
+++ b/xen/include/asm-x86/contig-marker.h
@@ -0,0 +1,105 @@
+#ifndef __ASM_X86_CONTIG_MARKER_H
+#define __ASM_X86_CONTIG_MARKER_H
+
+/*
+ * Short of having function templates in C, the function defined below is
+ * intended to be used by multiple parties interested in recording the
+ * degree of contiguity in mappings by a single page table.
+ *
+ * Scheme: Every entry records the order of contiguous successive entries,
+ * up to the maximum order covered by that entry (which is the number of
+ * clear low bits in its index, with entry 0 being the exception using
+ * the base-2 logarithm of the number of entries in a single page table).
+ * While a few entries need touching upon update, knowing whether the
+ * table is fully contiguous (and can hence be replaced by a higher level
+ * leaf entry) is then possible by simply looking at entry 0's marker.
+ *
+ * Prereqs:
+ * - CONTIG_MASK needs to be #define-d, to a value having at least 4
+ *   contiguous bits (ignored by hardware), before including this file,
+ * - page tables to be passed here need to be initialized with correct
+ *   markers.
+ */
+
+#include <xen/bitops.h>
+#include <xen/lib.h>
+#include <xen/page-size.h>
+
+/* This is the same for all anticipated users, so doesn't need passing in. */
+#define CONTIG_LEVEL_SHIFT 9
+#define CONTIG_NR          (1 << CONTIG_LEVEL_SHIFT)
+
+#define GET_MARKER(e) MASK_EXTR(e, CONTIG_MASK)
+#define SET_MARKER(e, m) \
+    ((void)(e = ((e) & ~CONTIG_MASK) | MASK_INSR(m, CONTIG_MASK)))
+
+enum PTE_kind {
+    PTE_kind_null,
+    PTE_kind_leaf,
+    PTE_kind_table,
+};
+
+static bool update_contig_markers(uint64_t *pt, unsigned int idx,
+                                  unsigned int level, enum PTE_kind kind)
+{
+    unsigned int b, i = idx;
+    unsigned int shift = (level - 1) * CONTIG_LEVEL_SHIFT + PAGE_SHIFT;
+
+    ASSERT(idx < CONTIG_NR);
+    ASSERT(!(pt[idx] & CONTIG_MASK));
+
+    /* Step 1: Reduce markers in lower numbered entries. */
+    while ( i )
+    {
+        b = find_first_set_bit(i);
+        i &= ~(1U << b);
+        if ( GET_MARKER(pt[i]) > b )
+            SET_MARKER(pt[i], b);
+    }
+
+    /* An intermediate table is never contiguous with anything. */
+    if ( kind == PTE_kind_table )
+        return false;
+
+    /*
+     * Present entries need in sync index and address to be a candidate
+     * for being contiguous: What we're after is whether ultimately the
+     * intermediate table can be replaced by a superpage.
+     */
+    if ( kind != PTE_kind_null &&
+         idx != ((pt[idx] >> shift) & (CONTIG_NR - 1)) )
+        return false;
+
+    /* Step 2: Check higher numbered entries for contiguity. */
+    for ( b = 0; b < CONTIG_LEVEL_SHIFT && !(idx & (1U << b)); ++b )
+    {
+        i = idx | (1U << b);
+        if ( (kind == PTE_kind_leaf
+              ? ((pt[i] ^ pt[idx]) & ~CONTIG_MASK) != (1ULL << (b + shift))
+              : pt[i] & ~CONTIG_MASK) ||
+             GET_MARKER(pt[i]) != b )
+            break;
+    }
+
+    /* Step 3: Update markers in this and lower numbered entries. */
+    for ( ; SET_MARKER(pt[idx], b), b < CONTIG_LEVEL_SHIFT; ++b )
+    {
+        i = idx ^ (1U << b);
+        if ( (kind == PTE_kind_leaf
+              ? ((pt[i] ^ pt[idx]) & ~CONTIG_MASK) != (1ULL << (b + shift))
+              : pt[i] & ~CONTIG_MASK) ||
+             GET_MARKER(pt[i]) != b )
+            break;
+        idx &= ~(1U << b);
+    }
+
+    return b == CONTIG_LEVEL_SHIFT;
+}
+
+#undef SET_MARKER
+#undef GET_MARKER
+#undef CONTIG_NR
+#undef CONTIG_LEVEL_SHIFT
+#undef CONTIG_MASK
+
+#endif /* __ASM_X86_CONTIG_MARKER_H */



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:55:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:55:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195178.347762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThvr-0007oL-9d; Fri, 24 Sep 2021 09:55:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195178.347762; Fri, 24 Sep 2021 09:55:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThvr-0007o9-6j; Fri, 24 Sep 2021 09:55:51 +0000
Received: by outflank-mailman (input) for mailman id 195178;
 Fri, 24 Sep 2021 09:55:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThvq-0007lG-LX; Fri, 24 Sep 2021 09:55:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThvq-0006VS-9O; Fri, 24 Sep 2021 09:55:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mThvp-0008VI-Vl; Fri, 24 Sep 2021 09:55:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mThvp-0005KM-VE; Fri, 24 Sep 2021 09:55:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cAPrTvJun3ICDTKaI/ZSUzLM369LXqlrypYyPSxdUE8=; b=bbLyFho4xqnA+T/oPuGIV+zoly
	IRfHfrISQKkY8QMRo0yqlZ9RS1n2Wi+Ij0jjctHY8lrmEpFCAXOe6miAl8U+BJRShemnqLxqmFis6
	Su8nMI+qUWrBy/LUTTjrqnTrur/jS2+Xc5crB7j+zeovjve1uZ7nIN8V1jSZ6P5Iph+o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165173-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [seabios test] 165173: tolerable FAIL - PUSHED
X-Osstest-Failures:
    seabios:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    seabios:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    seabios:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    seabios:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    seabios:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    seabios:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    seabios:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    seabios=64f37cc530f144e53c190c9e8209a51b58fd5c43
X-Osstest-Versions-That:
    seabios=54082c81d96028ba8c76fbe6784085cf1df76b20
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 09:55:49 +0000

flight 165173 seabios real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165173/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 163203
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163203
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 163203
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 163203
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 163203
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass

version targeted for testing:
 seabios              64f37cc530f144e53c190c9e8209a51b58fd5c43
baseline version:
 seabios              54082c81d96028ba8c76fbe6784085cf1df76b20

Last test of basis   163203  2021-06-30 21:10:04 Z   85 days
Testing same since   165173  2021-09-24 03:09:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/seabios.git
   54082c8..64f37cc  64f37cc530f144e53c190c9e8209a51b58fd5c43 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:56:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:56:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195182.347776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThw4-0008OU-JE; Fri, 24 Sep 2021 09:56:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195182.347776; Fri, 24 Sep 2021 09:56:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThw4-0008ON-Fs; Fri, 24 Sep 2021 09:56:04 +0000
Received: by outflank-mailman (input) for mailman id 195182;
 Fri, 24 Sep 2021 09:56:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThw3-0008MN-1h
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:56:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9fb259ae-1d1d-11ec-bab9-12813bfff9fa;
 Fri, 24 Sep 2021 09:56:02 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2058.outbound.protection.outlook.com [104.47.9.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-Zt7p6K7bOJ6cFH9_m3En3Q-1; Fri, 24 Sep 2021 11:55:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7040.eurprd04.prod.outlook.com (2603:10a6:800:121::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 09:55:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:55:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0062.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.6 via Frontend Transport; Fri, 24 Sep 2021 09:55:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9fb259ae-1d1d-11ec-bab9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477361;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+Ote1TPd9Kh2XKCxDVFlIWjOJeJk0V39fRCOYujD0nE=;
	b=SwY+3xWjW2cTRjegXPgxgKsnL9lWR1UYC90UVACMLBUhy6dcsucksPki9Ap5z0/cn/9Lkm
	LDiXGzbwRUtZKf/UTHqxgJGxyrw9tFv7THWY3kcSk0E4s//V/NBpoRFxMVrMN3LEeAftOe
	0xvaQxCJgiWkRWi2FSWumZ8ranYMTTk=
X-MC-Unique: Zt7p6K7bOJ6cFH9_m3En3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Velw2ue/po/y0uyNoYLXAVczhFp1edbNoR6+iRESXltNhjrRtxjUEsQCL+kf0ds8luMqbhDL5RHJdW13aMvUSyUKNCKhs5S0OWjxNiwYf+WeYDAibEr00KPE5Y1O468rKBjLIDO9VtVUzSZPANxWNHoo+tbdJygXUXTa1kXPmXRZTtkSqEABYfuum7aLJ24wlU/Rsgf56z2cUL4kNaJFx3Y8uxP7mLGQGJxYwtcNWiJaJMfmWcWBN4hBpbqSijobkNa1qqZ7XAC/Ft+I4nqB2HnnNEVUDe6Pt0QxcDQfFV9KJ52suVJ9UQxLkh9wQk/Jc2sM/nUnV0IwN52PpZ4Jcw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+Ote1TPd9Kh2XKCxDVFlIWjOJeJk0V39fRCOYujD0nE=;
 b=M5dOqTnME0e4T0ErNxdxZgQdr89pdfihEzmL1JqN2SA6r6mRvZ0qQPC/LEcJ+50dtXb5Er0QUcXZ+FrrZLtLII0yabdQ6eYiLHY1geeinnC2WyLHZus6f+vpIXWzJaEIxmjG86PSPKPctbOkHUQQNMkidXa3yrg57c3L4yn2kS7ODI1ONRM65tqxVr7Ey4e7xIsCXY1yJn6KNuOrqWq9gScadmWUgfMPgSVXtapobMAxuaVRaMj6J2buZ7djDmp5QhSHxuBlzhcbqFEeQFFw6BjHKCc/SWZPHsbH+UjVM+jhBK88n2++vwsAchcWmLZoRMGClzJ44FdIdihpuSw6TA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 17/18] AMD/IOMMU: free all-empty page tables
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <ef31b6f4-09f5-cded-a235-5dce3c5399ce@suse.com>
Date: Fri, 24 Sep 2021 11:55:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0062.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e75dc628-f4e8-455e-2399-08d97f418265
X-MS-TrafficTypeDiagnostic: VI1PR04MB7040:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7040E2FA4A6C69B2EA047628B3A49@VI1PR04MB7040.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cDHMBVAcUywx9qicKDYwVW//qYc1RIWCGYh4ljcKo+5eJn31ejM8MSf/LZN/oOpjE+Kb/pOLt+l7GQah5RA4KZRfLwIKCxDNO8+jrjEiSAcdQxfh5KXQJwcFppbR1u9qK8Gmd/Ti+s+/lNV+qThjiLyHNrSelYT8G5jy38S9ONJfYtKGCyIoedfpwaQJEiUZCp7k3EHBEe3kNENLB55RLtBw6loh2OIELWXOEscIs+Z1H9p8hV/Ao9z2gLsZG6zBHFFSW5H2i4KpRR9b17IFpagH9Su8BdZtIvTBE6M86aicCv9WhtwkKEjkV7jaxUG78MMF3qt+IljLFb+9qu4ksVfQOQBlZjFaILuzsv/J52Av8OwerH7jZ8uHnzJ6+XFXs4F3O2HCsiNo22JaoXHQSzt/wlXpqgLmmgFn3/8gCqAXvuP5pn4ZjtlfVejMrIW93Kqc/0kwWY4thcFIrXYxEO5+2BS/L3rjt3bvDSaKlZLwHatuNR6ajvivuPzdhYm1UXjrM9DPvA7JW0oCCS8n3ZYkVZdMWwE+yvp1M8ufkAxXKH+zAc1NxZiTYDV/HtTT4jxWvN5c0aCPxnfTvAiP9SlwzQsF54/oVMz+RY+uLximPd8JeQimy7C9lnuiiMlheSfeqfz6iE1MxHKMfmjUVNhm1UQK3V8IPSMDSFHYP9ovqknMOMiRsqfFmxUt7/WQ969wULSaRcrs5h9pnQC/wlzKTpXZhLXTtP81rFkbcas=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(86362001)(83380400001)(6916009)(508600001)(66476007)(66556008)(31696002)(38100700002)(31686004)(26005)(8676002)(36756003)(66946007)(16576012)(8936002)(54906003)(5660300002)(316002)(4326008)(186003)(6486002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MGhCL3dyM2JnMHhkd0hoT2xIWVI2UVZrUFU3akloVGpveDJCYUpRdDh1dENJ?=
 =?utf-8?B?cXdFano3TGUzcmZ0d3VET2trRjRHdDdFeXNGTlRYSzZzVW8xd25DMUoxZVlZ?=
 =?utf-8?B?Nk5tUk5OandtK3pqZ0thcjZ4SE5nQUpkRGxlNk1ZbWxadXVRbDNFcVVlQVlu?=
 =?utf-8?B?SHdaSERPYTN6MnBnKzB3ZmM0VmIvbU85dHc2eWhzYm1WWU9XQ3JIVE9Rc2NG?=
 =?utf-8?B?SVF0TGZWNUI2MmJRRjJyRGdCNGtKNE5OUmNFMXFJYTluYU9WVTZkdjdQdUNv?=
 =?utf-8?B?U0V0eklYdUtETXRlaDhoUHZ5Q2ZFbi81U1Z1N2dIL3pKUDd0ampNQTJNN0tq?=
 =?utf-8?B?Ym93YWVodzdpWWllQzZWWWJJVjVOK3dQMno1RWhhWGppVVZlN0RqcXAwNno0?=
 =?utf-8?B?QmluLzd2UXJMaWFvUWFkZEZkRGxLeGJrK2QxaXNnSDYyVGFwYUJEbUZ4TUFn?=
 =?utf-8?B?bmVLc3V5STg1OUhaMWYyNVpFMW9SVkYyKzFBMDBLR0FZYlZ0eis0bG1vaDRk?=
 =?utf-8?B?Q3RFMHFOU0hwbTZrMk56Tmg3S0dXUVdRYzFsb3Vzcy9nUjdBc3RWQmV4K0FY?=
 =?utf-8?B?NmV5UFFvemRrbm1BWFlEVFp0SFdHVGQxTWE1WHJMc0M3cGFSa0NkTnFaR0Qy?=
 =?utf-8?B?SlU5ek9XTURINDl1cU9jY0VqR1NoSnRFbE40OXczdk9JR3dFYi81VE5jZzBV?=
 =?utf-8?B?MlZ1MHdOakY2eVBFZjYwVnVUNVpCUnFsc2xtV2FENUhMdXhxUG83OUV0TnJq?=
 =?utf-8?B?NHhzWlltbU5ObTFYQ0JjOFdQNWhCY0lkUkI4SDB6M0l6TDRzM2JBNHkrL0xS?=
 =?utf-8?B?eXlhVlo3Mnk4cExOVHdvMEc1MGwvVlJvYWlxMUpRSGMzNlhjK3ZrRG83WmVH?=
 =?utf-8?B?NHJwVGNGZXVPeE9Pc2Z0eW43YkN1SmJFZlFjOEVwN2hZOWFzZEpXZXBidXI2?=
 =?utf-8?B?UDZkV2RRa1lCUjdobmZqVjFaUTVtVjdjQTQ3eVlsK2FzcGNQSmpDU1Q5UklU?=
 =?utf-8?B?ZENIV3B6RWE0ak5GbXllNlc4eG1udVNjaENqMy9ncXc5TjY4cHN3Um1LYjFs?=
 =?utf-8?B?bjkwL0ZEWVVtYm1BYVhaaTVoTEFaM3ExaFJOR0N4bjlkNjJRYUQraTNKSkNx?=
 =?utf-8?B?THpWemtRcXh0bWxwZVY1cFFNdG9DamVLTFJmUm51ZDMvZTBEdUZxbWxQcnk3?=
 =?utf-8?B?d3g2blJYcm9zYWpzbGNRenJtSXBwVUgrdkRVTmswZ0Y5RU5PV1dkaDJjWDlC?=
 =?utf-8?B?QWZGOGhyRy80ZnFPbzdSZXd6ZzlSSEV3MkVkR0NpNGtwNjIvUExtOFc0R3Zl?=
 =?utf-8?B?Y1h2b0doeFJkb3p2Nm1jTkV1Q2JMbXVBcmc3L2doQURZNk81cHNHOS9yMWs4?=
 =?utf-8?B?a09aR3V4UG9NR3F2cjBCdWNrRUFad25ycTZTZ2tJNzJkK28zeTJvNks0NVFn?=
 =?utf-8?B?c0w4N2JOVCt3bVkwUUpWbnZ4ZHBzNjBwbTJ5NVo4Q2JUazh1OFZROXUrdWhW?=
 =?utf-8?B?eEtqMzFObVFzKzJ3WUpwcHZhNXJ3c0V6T1p6NW9LR09JMWRYb3BENFJyVjRa?=
 =?utf-8?B?V285TW9UTFlUenVlU3p0NTd4M3NtZlFRM0tPaHh6NXVCYjVaaFp5UkZDd2k1?=
 =?utf-8?B?OXduVUp6bmkzOUpkdGxTRTloSU5EUUUyREdXcjBVVXpTRmtuM1RLeHRJNm1o?=
 =?utf-8?B?QmdwUnBrOW9aOXhQSlpxSG5sOVdkdXZKT3NSRmFYNVQ2RlcyNDdmOStXWFRp?=
 =?utf-8?Q?1wg7C9QRJxytV/go8DkK2h5ZJTfOtyslu2BwUg0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e75dc628-f4e8-455e-2399-08d97f418265
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:55:58.7973
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zD/qbOFOETGAYa+ORXpvwOdg2Fl1FSJCgS9qv17d27nCJ9gdf56sQO2ebNnUpRdaBAn09fbaoYK2BdbFo+vbnw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7040

When a page table ends up with no present entries left, it can be
replaced by a non-present entry at the next higher level. The page table
itself can then be scheduled for freeing.

Note that while its output isn't used there yet, update_contig_markers()
right away needs to be called in all places where entries get updated,
not just the one where entries get cleared.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -21,6 +21,9 @@
 
 #include "iommu.h"
 
+#define CONTIG_MASK IOMMU_PTE_CONTIG_MASK
+#include <asm/contig-marker.h>
+
 /* Given pfn and page table level, return pde index */
 static unsigned int pfn_to_pde_idx(unsigned long pfn, unsigned int level)
 {
@@ -33,16 +36,20 @@ static unsigned int pfn_to_pde_idx(unsig
 
 static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn,
                                                    unsigned long dfn,
-                                                   unsigned int level)
+                                                   unsigned int level,
+                                                   bool *free)
 {
     union amd_iommu_pte *table, *pte, old;
+    unsigned int idx = pfn_to_pde_idx(dfn, level);
 
     table = map_domain_page(_mfn(l1_mfn));
-    pte = &table[pfn_to_pde_idx(dfn, level)];
+    pte = &table[idx];
     old = *pte;
 
     write_atomic(&pte->raw, 0);
 
+    *free = update_contig_markers(&table->raw, idx, level, PTE_kind_null);
+
     unmap_domain_page(table);
 
     return old;
@@ -85,7 +92,11 @@ static union amd_iommu_pte set_iommu_pte
     if ( !old.pr || old.next_level ||
          old.mfn != next_mfn ||
          old.iw != iw || old.ir != ir )
+    {
         set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
+        update_contig_markers(&table->raw, pfn_to_pde_idx(dfn, level), level,
+                              PTE_kind_leaf);
+    }
     else
         old.pr = false; /* signal "no change" to the caller */
 
@@ -259,6 +270,9 @@ static int iommu_pde_from_dfn(struct dom
             smp_wmb();
             set_iommu_pde_present(pde, next_table_mfn, next_level, true,
                                   true);
+            update_contig_markers(&next_table_vaddr->raw,
+                                  pfn_to_pde_idx(dfn, level),
+                                  level, PTE_kind_table);
 
             *flush_flags |= IOMMU_FLUSHF_modified;
         }
@@ -284,6 +298,9 @@ static int iommu_pde_from_dfn(struct dom
                 next_table_mfn = mfn_x(page_to_mfn(table));
                 set_iommu_pde_present(pde, next_table_mfn, next_level, true,
                                       true);
+                update_contig_markers(&next_table_vaddr->raw,
+                                      pfn_to_pde_idx(dfn, level),
+                                      level, PTE_kind_table);
             }
             else /* should never reach here */
             {
@@ -410,8 +427,25 @@ int amd_iommu_unmap_page(struct domain *
 
     if ( pt_mfn )
     {
+        bool free;
+        unsigned int pt_lvl = level;
+
         /* Mark PTE as 'page not present'. */
-        old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level);
+        old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free);
+
+        while ( unlikely(free) && ++pt_lvl < hd->arch.amd.paging_mode )
+        {
+            struct page_info *pg = mfn_to_page(_mfn(pt_mfn));
+
+            if ( iommu_pde_from_dfn(d, dfn_x(dfn), pt_lvl, &pt_mfn,
+                                    flush_flags, false) )
+                BUG();
+            BUG_ON(!pt_mfn);
+
+            clear_iommu_pte_present(pt_mfn, dfn_x(dfn), pt_lvl, &free);
+            *flush_flags |= IOMMU_FLUSHF_all;
+            iommu_queue_free_pgtable(d, pg);
+        }
     }
 
     spin_unlock(&hd->arch.mapping_lock);



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:56:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:56:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195195.347788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThwi-0000oU-1C; Fri, 24 Sep 2021 09:56:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195195.347788; Fri, 24 Sep 2021 09:56:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThwh-0000oN-Tq; Fri, 24 Sep 2021 09:56:43 +0000
Received: by outflank-mailman (input) for mailman id 195195;
 Fri, 24 Sep 2021 09:56:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThwg-0000o3-0D
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:56:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f73c821-0d12-4fc5-ad16-e97a87849c94;
 Fri, 24 Sep 2021 09:56:41 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-Qghzi9XzNM62bBAmM4P2Fw-1; Fri, 24 Sep 2021 11:56:39 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7040.eurprd04.prod.outlook.com (2603:10a6:800:121::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 09:56:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:56:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0045.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Fri, 24 Sep 2021 09:56:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f73c821-0d12-4fc5-ad16-e97a87849c94
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477400;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=W8EuFDCQk/F12Hxw1IKCpr1YasMuKbjeGSquhoEqDIo=;
	b=Kb9qclod/bXPJOGN9AUEH1qu/OH/Y60f5EbKZ9hEE1e20GQ1ibPr6Fh9fKZOt1/W7yi/WN
	xk7LiNa0RWdDKoVx3mwqZBwChsygR94A6v5HP0/j20Uh8RM9tfe0obOpJEPDjevdjpEvWT
	b+LHSk/hU/fHxWj0dBfEd/Eegqq2XQY=
X-MC-Unique: Qghzi9XzNM62bBAmM4P2Fw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nmYP87IF0gTRoMWJqdJG8l1+Pw2TmMH/UMU2zrUDRbdQHxpBEinfxM2V9VHf+Y8478Uk3bF2TT1kpT1bHLgGFY/M1yS77yGkcHoyn/dmClfZIg097nmri6fEX7j9MVWKenmAZe3AxdtFMqCYbzvE/emlY7rAfUCzKiDyzYG3sim5AUWhOc8RUq0/lT9v6tRJc20VOn9Ltag2npiNO+9LNkEgWiqOWPhjC012Wu9K+4osOoLR586HQ2LWXvs7KaB1Qy7/IzjCF8xW/o5KLIllJgEfQLxzR4YQlH1KxY1QBiUpbPbZN1rygz5BYJFZnM/r77yM3+vAfCFyfe+yRlq0Ww==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=W8EuFDCQk/F12Hxw1IKCpr1YasMuKbjeGSquhoEqDIo=;
 b=DaXdRmcn2YLOqvnHnSy+8Rnd7VpDKkD/lbwSbT+Dc+U+1nxWaW4MV7TTpsESOdp03z19QGfgbtRvFzc1ImSH+J/EPuckfrIZCZ1Hx6R3PEg/xSWc1qzgpkLcIIn4JpDETc7Y+VUzA0296Hzw0WceUZrqGE75HnXLCNWs1F6HkQta8cZ4ZF4avHaRXjQgHwgP1Ox8gL2YveVNihRYoEl8A9KlguN23Qourbffvat94nMbXrXdPh7oDKv39sTlM9jKSSCLHHZnH5k2QDzegw74qUdsfyIeKQFz4hXZTOTOtvbpcDBj5mbvLHXmAlKC9OYBP6NsiGFTG97Ma2Jn9U6ssw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 18/18] VT-d: free all-empty page tables
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
 Kevin Tian <kevin.tian@intel.com>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Message-ID: <4d3f1748-4964-77d7-77f6-db58a9eef55f@suse.com>
Date: Fri, 24 Sep 2021 11:56:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0045.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d85ed0d-14c6-48cb-9d94-08d97f4199b3
X-MS-TrafficTypeDiagnostic: VI1PR04MB7040:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7040A1E05306CFF35C9F8F5BB3A49@VI1PR04MB7040.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Uhqe0b9VOaRCzcINyBhF/rVhMXD6Tm+iqZ2DHO/Mdq8h3sMC9hieG308uerWrF/3j+tIMXuvOoVZygD5Nm8qPJmKZ5BFWXyuHpPeo3uESQx6XVAe0Q+P+ouQzJ1B8zkabpKBBJaaMPKEsl+1HQ5GCYovr7axBXvdkLCVVdy8n5TTW18t0EXkhfjCpfftwXaiBLpL0mXgHXrWXTg+DsmWQ/q/qiON3l4vqxj12+ILtU79MgpxUrrpSP1kPPFZHvyoWKzGpHXQ1pB/5oqlsQVdA2Miz57DxPF2Ck6Lz/SXWJX4HDqjQkcMvyvvWUkFA4t8Lc+f3fXDVt8D6HHwLUJ4R/rcw7P2J5H+mhOM/1W4cpJbQu5DYIjv4Wkw63SPjg2JjglV/G24p51o5G4p2LGBqocW5+n0p5bzM3OtqfK4jQksuxC5FFgdQFnsjSbz951bviadAaN6hk5NqP1IJXeAGWwZI2y3SBa9Mn5IV8Wy5ky3DkM05jjywNzf6ezgssQqiOBPbG+oMcqIphYIELexvokF7WO43oLQU+Krr1/WXNQq/AI8wbU23PMYGKtmeFNq4Zzg3wx6ZrsWi7eCvoFpb+KaZVI/bnhByJFb3rrlum83WdeUt2G7agKcRih5ASDglRoJr0fV0X7JSSJFrzwBLyvShZNxF17BDMHYRT6iSqE6lxalZdJNOkLMVF5PEBlK8nwBlNmRBZ1wJN8bpbfUKkrjIMQU5ya+LeXUMk3R8AA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(86362001)(83380400001)(6916009)(508600001)(66476007)(66556008)(31696002)(38100700002)(31686004)(26005)(8676002)(36756003)(6666004)(66946007)(16576012)(8936002)(54906003)(5660300002)(316002)(4326008)(186003)(6486002)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZFFscm5xbVZZMGkrblFFVkE4eFgxcTNVSDlXdjZsY0dMSHZKYXU3QVdJeU5F?=
 =?utf-8?B?STk5aUQrVXNKVWE1MHJrZENwUExWWXNrSVhkQ2htU1JNYzJmcXJYcGxGZytP?=
 =?utf-8?B?WUxnZExOQkMrNlNuOStHZzlVZjgyWWhiOU5uMnI5TzBuNWlLVS9ZaUVNdkdI?=
 =?utf-8?B?Mjl3dkgzVTFHa3BjSzVId3d1YXViNndjK2c5bHFWWlFTeDFhbE0rcmVoNkFq?=
 =?utf-8?B?bUtuTWhCK2dFYUNpU3dFNTZsTldVZHQvSS95eVMxUEd6WHc1VlA1NnRSbGJr?=
 =?utf-8?B?RUdUZmJOMTd2MFhOQTJxWkdNWXVaUEdvWmNHVy95ZWlWa045Smtmdk1XVWRl?=
 =?utf-8?B?YndvaVdOa3NEUHJJN2ZEM3NGKzVvSnVGUWZwcWNvTDNndlhQQlFWQTQyTC9j?=
 =?utf-8?B?U09xaFk1d2VtTXE1K0tPTWo2TUlUWVdKZWNrSmlYSGtHR3k1V2t1NFlGRmVH?=
 =?utf-8?B?NFZSL0ZnQklZcDNjZ0pRYmkyeHlLSkdVQ09NRzBCZjdScXJzOWxXSXFvY2Ra?=
 =?utf-8?B?NG9uQWx0emFNZG1FcjVnT1REakJwaytmTXptSzlmell6SjJSR0pLd0hsUGhZ?=
 =?utf-8?B?Q2pTU1d6amJDdHJISUdJZHh4Q0V5Rk1FUmR2c1RpSnJ4cG16VGNSWCsxV1dE?=
 =?utf-8?B?WjVRdXJlTW55cWE1R2d2Zjk0UUI4clByemZyVnJ1NjUzQzFGanRzSHh2OEFH?=
 =?utf-8?B?V3V1NjlqVjUvbWluV1lPdmFocTNuUHhIYUlYL0pPVGVya1FVaS94TVBZMGJl?=
 =?utf-8?B?VmVqWElkdU8vdVZqUkkzZWRvd29XL21Ickl5eTgxYUtBdFIxblNoOHl2RnV6?=
 =?utf-8?B?a01KY0tkQ1k2SUFsWFBRalByTTZsUnJveXArL0hBSmV0NGs2Q29JRTd3QlA4?=
 =?utf-8?B?YzBRSmdVNkpleUhjTWd6SENicjAvd2ErMUR6TG9oWmthK2M0cmdnZlpTaXBj?=
 =?utf-8?B?NU5xTkcwbCtmNEZ3VjBlOEQ5OFRxbWhOSFBuYlZudE1hWVdYdjFNTm1RTXVF?=
 =?utf-8?B?eUF4V2hBTEp0V2hHTHZ0cEZqRGtLcEJONFN4YWtDL1h1cVdXZVUzWkg2R1Nk?=
 =?utf-8?B?QmFtMmZtc09DMXVidDQwcWdGTUJ4MDNZbGlmbHZwanpqV0FOU3ZuNXpkdjVV?=
 =?utf-8?B?ZEdlWERNOTY2eTgrMDl6UDI1NlM1UjRxYUxxeTBhL3Z6T0dWVktuMi8vWTNq?=
 =?utf-8?B?YjVIR0QwT0tVUVZzMnRlNUM5WTFlSmJ6N3FJMnB5WGE0SmJKNnVGYVo2UG1D?=
 =?utf-8?B?UTZnTXdoUko4WGxUaXd1Nk5KbXBiNHNYQmgwZE1McEVVbEFKdEpBT2FLa0RP?=
 =?utf-8?B?Wi9QUlp0clgwcE1KWkY1bGJmY09hYytZL0psVmNZd3RPVkVOaWQ1cW9UTFFC?=
 =?utf-8?B?TmZsRVpaVHBwOWFUdDhNa0JNTWRuT2MyWWtuSnk3VW4ydlh3cTkrSlRxNnB6?=
 =?utf-8?B?YmpUME9Pb0QrS3QvUEFwa0tqMHA1MURLdVQySEt5WlJkc3d1cm9mK1FibGZZ?=
 =?utf-8?B?TEorZitYTmZ2d2Jhdi9tZHRVQ01GQ2hDMGpVVUxZSHJVV3U0bTJ1ZndjK2ds?=
 =?utf-8?B?MVBONGkyTE9yWjM5N2hrdSticUR5TU4ycHI1SjYxNzFhOElGUXIvY2kvYkFK?=
 =?utf-8?B?MjdKNXZXUncxbDBEQzcvNE9UTEpIeFhxdE5FUHU4WDBkcEJkbzg2NFRpOFI1?=
 =?utf-8?B?VFNiVTBnOFVqaU5QZW5TeG1FaTlJa0orOGRJc0dGL1UxekdnK2Zja2NORWJI?=
 =?utf-8?Q?20FKG0bWaGFytLZ3ddRlxwrQ2tA+qRzYgrcjmRA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d85ed0d-14c6-48cb-9d94-08d97f4199b3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:56:37.9022
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fnhDj8i1QWTHWrN0t0N897BTOUMmdsPq4+bgW8bODIuoOXsSlqQa46TxUcSTKg74E1CwPZuoNwd3HX10PU1G+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7040

When a page table ends up with no present entries left, it can be
replaced by a non-present entry at the next higher level. The page table
itself can then be scheduled for freeing.

Note that while its output isn't used there yet, update_contig_markers()
right away needs to be called in all places where entries get updated,
not just the one where entries get cleared.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -42,6 +42,9 @@
 #include "vtd.h"
 #include "../ats.h"
 
+#define CONTIG_MASK DMA_PTE_CONTIG_MASK
+#include <asm/contig-marker.h>
+
 /* dom_io is used as a sentinel for quarantined devices */
 #define QUARANTINE_SKIP(d) ((d) == dom_io && !dom_iommu(d)->arch.vtd.pgd_maddr)
 
@@ -368,6 +371,9 @@ static uint64_t addr_to_dma_page_maddr(s
 
             write_atomic(&pte->val, new_pte.val);
             iommu_sync_cache(pte, sizeof(struct dma_pte));
+            update_contig_markers(&parent->val,
+                                  address_level_offset(addr, level),
+                                  level, PTE_kind_table);
         }
 
         if ( --level == target )
@@ -773,7 +779,7 @@ static int dma_pte_clear_one(struct doma
     struct domain_iommu *hd = dom_iommu(domain);
     struct dma_pte *page = NULL, *pte = NULL, old;
     u64 pg_maddr;
-    unsigned int level = (order / LEVEL_STRIDE) + 1;
+    unsigned int level = (order / LEVEL_STRIDE) + 1, pt_lvl = level;
 
     spin_lock(&hd->arch.mapping_lock);
     /* get target level pte */
@@ -796,9 +802,31 @@ static int dma_pte_clear_one(struct doma
 
     old = *pte;
     dma_clear_pte(*pte);
+    iommu_sync_cache(pte, sizeof(*pte));
+
+    while ( update_contig_markers(&page->val,
+                                  address_level_offset(addr, pt_lvl),
+                                  pt_lvl, PTE_kind_null) &&
+            ++pt_lvl < agaw_to_level(hd->arch.vtd.agaw) )
+    {
+        struct page_info *pg = maddr_to_page(pg_maddr);
+
+        unmap_vtd_domain_page(page);
+
+        pg_maddr = addr_to_dma_page_maddr(domain, addr, pt_lvl, flush_flags,
+                                          false);
+        BUG_ON(pg_maddr < PAGE_SIZE);
+
+        page = map_vtd_domain_page(pg_maddr);
+        pte = &page[address_level_offset(addr, pt_lvl)];
+        dma_clear_pte(*pte);
+        iommu_sync_cache(pte, sizeof(*pte));
+
+        *flush_flags |= IOMMU_FLUSHF_all;
+        iommu_queue_free_pgtable(domain, pg);
+    }
 
     spin_unlock(&hd->arch.mapping_lock);
-    iommu_sync_cache(pte, sizeof(struct dma_pte));
 
     unmap_vtd_domain_page(page);
 
@@ -1952,8 +1980,11 @@ static int __must_check intel_iommu_map_
     }
 
     *pte = new;
-
     iommu_sync_cache(pte, sizeof(struct dma_pte));
+    update_contig_markers(&page->val,
+                          address_level_offset(dfn_to_daddr(dfn), level),
+                          level, PTE_kind_leaf);
+
     spin_unlock(&hd->arch.mapping_lock);
     unmap_vtd_domain_page(page);
 



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 09:59:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 09:59:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195200.347799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThzC-0001vW-EP; Fri, 24 Sep 2021 09:59:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195200.347799; Fri, 24 Sep 2021 09:59:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mThzC-0001vP-BK; Fri, 24 Sep 2021 09:59:18 +0000
Received: by outflank-mailman (input) for mailman id 195200;
 Fri, 24 Sep 2021 09:59:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mThzA-0001vD-Nw
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 09:59:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78f18b1f-8cae-4a57-9fa7-8b80fc127153;
 Fri, 24 Sep 2021 09:59:16 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2173.outbound.protection.outlook.com [104.47.17.173])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-Tz40-cw2O-u5kW-x1X3tCg-1; Fri, 24 Sep 2021 11:59:13 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5901.eurprd04.prod.outlook.com (2603:10a6:803:e9::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 09:59:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 09:59:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1PR01CA0029.eurprd01.prod.exchangelabs.com (2603:10a6:102::42) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 09:59:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78f18b1f-8cae-4a57-9fa7-8b80fc127153
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632477555;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=5WR5TsPKpIUkmQ4IQU9GK9qAtnS1zO3umVFMyOynuwM=;
	b=UGKDFHZWK/pC2GpoceEe6I3vC43S0d+qZ2L3ZnKidtV/dd4vnPxnFB7tg73KABjgVa+cyM
	zdmxGaepZhr57i9qI0AU7kqScyShj9GpKulMFOMQ36cAnpPuDObxNz/Ux/i0WY3rEhuT6H
	S8TXEW+on5k94U5Zy1bpYOxlu/qt54o=
X-MC-Unique: Tz40-cw2O-u5kW-x1X3tCg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZYhDv6x1ajlroHEjGPfUkuhpCMT1sUMurmdSEy32PwQ8Evs3/vfUwIovPhJm1/wZKrlQCfuSpA72PMRhTbm6jek0FZIQb8sJTnSRZvvfQvq9lHQY+LDXYedUyoTEY2qonOdP8EJWpDHiQexvnFiXq00sHEx8Qep5VLzaeS7C4I/6iJCy6Yl3ViziFhd+YnUbPPWV6AwX9k0RN/CObmH5lveZTfed1ZKffEkauoaEFd09wK9xm3zC8iJSpf2eQxf5hw2Tv4MnAQzpkXjZLBaZXxst3EQ91UU+4dnTL6dPBWvbR5XB6MxCPkgqMvPenaKgVcAfAOwnzXX8RQ1TTVBYwQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5WR5TsPKpIUkmQ4IQU9GK9qAtnS1zO3umVFMyOynuwM=;
 b=Kzm/Lh0NzSHUIzpnSTZeaUBJNo6Xe/+zhjwxNm5/e1o7AAESgRC7VYnq0M0TZojSPLsnSBBPLVRtT4BdX+7+zDlPvuS4c9than9buxCXvHWXVwyDnkZD21NtLJatDHauVlxDdjQ5v9MZz3qckfUPaFkQ+Ez1c+n6cF2AvXpj/KPV9Ty8wR/8dZWB8Z8TbWK7bbHehJTCSk8x/9TRijmsuWF5kXZrZj51YSb2DNogqfPDOKHdDMhC2+YdVD0pwSszSFqXzokE3MJAoQeKfB51AYcqTe4EgYHnrBFSM/lRtudU3VuGtvt18gvziHkM37HX4C2igEMosui1b2goB0wHug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2] pci: fix handling of PCI bridges with subordinate bus
 number 0xff
To: Igor Druzhinin <igor.druzhinin@citrix.com>
Cc: paul@xen.org, xen-devel@lists.xenproject.org
References: <1632474612-9940-1-git-send-email-igor.druzhinin@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0b008762-e483-5d94-e380-1b37adb36396@suse.com>
Date: Fri, 24 Sep 2021 11:59:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1632474612-9940-1-git-send-email-igor.druzhinin@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR1PR01CA0029.eurprd01.prod.exchangelabs.com
 (2603:10a6:102::42) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ddc86da-70c2-44cf-6f81-08d97f41f3ee
X-MS-TrafficTypeDiagnostic: VI1PR04MB5901:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB59015D27E7839D17FACE28F1B3A49@VI1PR04MB5901.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eSQdU4FCp+hRxcHPLcQWiHVnPKzu6S4OHNnALadJAcxHwry0//fH3F0wYaEIxsPOq4Ywei7j+LJhxUCtDASfRngYDkIp1TsM9THXuiZ1OiJ8Jx+dTKfBXxK9mveTnMCJdy9UCNVhecqm6LhpZYmWlr0wQXknblI/Am7E/3R3GoAFOBHIHmOHekseghVhySF8kVC8CYmBfPveZua9/sYBAG5zETVuxIxRaMp5CXQvnIMV0adx51kb/nlw+jxM8dlwuUkfh4n0olSa9QZd0lHwCqywI3eELT22G4RmimP/3/AlNPlr+G048esdpQEsYKvuBfrZL/oqTaT4nhKTh8r8ZDhmr7Us1kn3KiRzvq322gSy8s9T7pJxmlCcrNfJycmj/xyMd48xXKSs19/X5vk9AvuBzie7C29GHY1dt5iEnwh/g8wo8B+f0WeTPN3zpQ1acrDoZRjhqVmMHbHDrupFMSRLaoDhFJDx0Hjf1NBIKR/o3t7V2uFkO8hycd4A55JL+OqysKoo1Nvzh/oZfEbPD7Rm8AEfHb0Hd6NKv2FkML0CR8BUvlkmGk0WGM+3JrL52tw9NHjs6LFymgoh+liK9iGuIduKrT8Tc2gwS71xcPi1E9gU7bC2BUL35QQ1byQhXay7USKFAvjK0yx+QMEr3T5b9iOhCOSOrkrJjo85yBWoa6Dd4XQq8FSLA8nsKsJbgQJhqfqn72i1gLGMAX7WMA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(31686004)(4744005)(6486002)(53546011)(31696002)(956004)(8936002)(186003)(2616005)(4326008)(8676002)(86362001)(508600001)(6916009)(2906002)(36756003)(38100700002)(316002)(66556008)(66476007)(16576012)(5660300002)(26005)(66946007)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aWUxQVB6aVZKVVM0SExlZStHcjlkdVh3WGpTQXQ0T0hmNFZjenN3M1Baa2FF?=
 =?utf-8?B?R3BmbFplczhYVEhGTWN6ekN1dUpsamFEaDRPRlJwTFhjWVF6L2NScEdDLzEy?=
 =?utf-8?B?czdrTlczaXh4TEhTemF3RUVKbXBpUFFVdWxuS3kwVncyY1FRb3V5U2YwQW01?=
 =?utf-8?B?c3NrdldXS3E1bmsrNlFtcjhJNGF2UzltcjJxU1J3K2JyMTUxRUdXR0VmajhQ?=
 =?utf-8?B?SVdRdW1pN3NXalJyeC9zaGlZNWtFQUJwcFJYSUNodlhIa0JGRmRJSCtsNm1i?=
 =?utf-8?B?VnVTN1R0ODVXVTNKdmtuOVlVYzM2UlU3bmZMYTVoVWdVWHVERUJWVHFwOElS?=
 =?utf-8?B?Uk1qa3Vtd0gzMUlaQjh1amVyR1pmbkY0Ui84Q2x2b0EwaVIzTXFuZUpYd0Qz?=
 =?utf-8?B?bXFLUDJldnVYWEo4MEVEYXp6S21STnJsRjhNSTBiUU5QOEIvTXlRUUc2WXF1?=
 =?utf-8?B?SzByR3YvYUp5aTVocjMzSmVwK2Q1S24yaDlFTTIvN3JiMWgyN3ZtNUtmWTRz?=
 =?utf-8?B?ckNBQTJJTDJCQzJWY2RtdnVHMTVvUXRrbFY2dHhzYURUMU1QYVUzMkJIdHM5?=
 =?utf-8?B?NkZMWGVVZnY3cCtCeFc4Z3NxTmxBeks0aUtsUUFvMXhRNW9TYXQ2K2ZSdWhI?=
 =?utf-8?B?UXUvY0JsSlRPcWI5WC95YUQ2VS9BdUZybnBBRndLOHROQ05EMDVFdHBkV2k2?=
 =?utf-8?B?WmZoMEtJVElGbWVEU2M0c0dOZTVRL0ZDQzBVdldYdkJlRiswOVhBVUozYi94?=
 =?utf-8?B?dmJ3MlhxNFFtcjhtNHlGN205R2U0Mk9FUVhkc0toa01xRDA1WUVoVk42T1RL?=
 =?utf-8?B?K1VmM1BEU3Q4MHh2Y3RJTS9KOXBNaEdBSnpKRmNyTXZ3elR6QnR4OUpha3Qy?=
 =?utf-8?B?TzBkZGFlcU00eHpQWVVMT3VtQngwckhMYSt0TGlia1ZxVURTQmZWbGhTVDZj?=
 =?utf-8?B?Ykx5OVFZYXFzNCt3T2JtalhrczV4Si9QYUFRNHMvdTA5cVVQNHJhejJUTGNa?=
 =?utf-8?B?UkRtSDRCMStjY0VWQnF6RTRhbTJBNCtBa3ExUlIyUUFUd2ovRUhIMkdSSDY5?=
 =?utf-8?B?NlY4NjVhSk1GM0ZheWNON0o0QkNZYmk1S0N1eG96UjBOVTUvdGRydXNsQXk3?=
 =?utf-8?B?OXFWZjI0QmxtaHg3VHMybXRraWR6QllFai8xWGlWR2dRWXdnMXpnc0NqaUdm?=
 =?utf-8?B?K0xoOUR5RldGRHhKdUpvUmtRVkwxV0k5eXJOWVA1SGZGbE5JWERoTnFsa0ls?=
 =?utf-8?B?T2dTcXQrMlIyYUd4dFplbGw1UUpjSTd5U0tSZDlxU1VvOEZueExQQ0tqcXlO?=
 =?utf-8?B?SEQ4Y0JSdCt2MkhqeWVjWUVJWVcyK3NaQ21lTVhZbmt1d2R1S295d1AvZGZ3?=
 =?utf-8?B?Y2p2SUcvZTV4MmV0NFpIWkRFYmR3S2JKQlIvajJZZzF1eDkyYVZvN1ZUc09P?=
 =?utf-8?B?QVgwc2Z4WW9VZ0tKT042ZVg2TUp5ZDljRCtSU094cUFHNlc1YkhSRm82aXNl?=
 =?utf-8?B?WFBJTGt6Z0VLelZTQkRsU2VYZ0RzUUVkbFNKck5BZTFuUFZkUlh2RElmeUJq?=
 =?utf-8?B?SGExdEF1eHF6OXZrbzFZa0VtM1JLbXlTMHFLMCtpZENubjIvL0FSM1JpY0FB?=
 =?utf-8?B?ZitEdTVFQytrY0NKVUJSc0FreFZkYUlwNUplTitxV2p1cU9ONzIzY0FnbTFN?=
 =?utf-8?B?SFdvZzQ4RTAzK1BycGZwdnAvNVN6WFNmT25Rd2RtZS83MnVTWnZ3RWFxa09l?=
 =?utf-8?Q?RFRyZqQoBwQiaKPcdF2QS6gO7mPQ6mJwh6adPGv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ddc86da-70c2-44cf-6f81-08d97f41f3ee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 09:59:09.2866
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XFXBZtxMAfXUJU8RyPJZG8W9Ic4rmdHYdlKkKvwH7BPOZD7DmVrHkn1nxowYY4EDW8R+qA05Tv03+AhxnBZjfw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5901

On 24.09.2021 11:10, Igor Druzhinin wrote:
> Bus number 0xff is valid according to the PCI spec. Using u8 typed sub_bus
> and assigning 0xff to it will result in the following loop getting stuck.
> 
>     for ( ; sec_bus <= sub_bus; sec_bus++ ) {...}
> 
> Just change its type to unsigned int similarly to what is already done in
> dmar_scope_add_buses().
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:08:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:08:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195206.347810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTi8G-0003nQ-Be; Fri, 24 Sep 2021 10:08:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195206.347810; Fri, 24 Sep 2021 10:08:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTi8G-0003nJ-8W; Fri, 24 Sep 2021 10:08:40 +0000
Received: by outflank-mailman (input) for mailman id 195206;
 Fri, 24 Sep 2021 10:08:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTi8F-0003nD-1d
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:08:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTi8E-0006rQ-VN
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:08:38 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTi8E-0003Tq-UG
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:08:38 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mTi8A-0000Kb-65; Fri, 24 Sep 2021 11:08:34 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=e8wGDe8rC7sCsy0rJFdm+OqA/1xuDaDsGZd5O2Y9hUA=; b=vnCa+SkGUx+UzH7rWJniyNnChf
	LoO/4YHP389ztwm9GG91uLhn/Z3baDr0RafjGW9UxSPtwy10IN5JAXmRjEKl9wGngL11C2LvBMySA
	W/tyvfR4B7Tv5xHD1FB9oyqAnozacda0rX9JsrTAgZReebxaguIxA7FyuYB5i5HrF2O4=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Message-ID: <24909.41889.936464.136607@mariner.uk.xensource.com>
Date: Fri, 24 Sep 2021 11:08:33 +0100
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Kevin Stefanov <kevin.stefanov@citrix.com>,
    Xen-devel <xen-devel@lists.xenproject.org>,
    Jan Beulich <jbeulich@suse.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Ian Jackson <iwj@xenproject.org>,
    Wei Liu <wl@xen.org>,
    Anthony PERARD <anthony.perard@citrix.com>
Subject: Re: [PATCH v3] tools/libxl: Correctly align the ACPI tables
In-Reply-To: <YU2fHhA+SeE8lgVi@MacBook-Air-de-Roger.local>
References: <20210915143000.36353-1-kevin.stefanov@citrix.com>
	<YU2fHhA+SeE8lgVi@MacBook-Air-de-Roger.local>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Roger Pau Monn writes ("Re: [PATCH v3] tools/libxl: Correctly align the ACPI tables"):
> On Wed, Sep 15, 2021 at 03:30:00PM +0100, Kevin Stefanov wrote:
> > Fixes: 14c0d328da2b ("libxl/acpi: Build ACPI tables for HVMlite guests")
> > Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
> > Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Roger Pau Monn <roger.pau@citrix.com>

Thanks to both of you.

Acked-by: Ian Jackson <iwj@xenproject.org>

and pushed.

Ian.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:13:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:13:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195212.347825 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiDC-0005Dk-13; Fri, 24 Sep 2021 10:13:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195212.347825; Fri, 24 Sep 2021 10:13:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiDB-0005Dd-TQ; Fri, 24 Sep 2021 10:13:45 +0000
Received: by outflank-mailman (input) for mailman id 195212;
 Fri, 24 Sep 2021 10:13:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiDB-0005DX-8Y
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:13:45 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0c::608])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9d40249b-1d79-463a-bcbf-53b2e97c97c2;
 Fri, 24 Sep 2021 10:13:42 +0000 (UTC)
Received: from AM5PR04CA0007.eurprd04.prod.outlook.com (2603:10a6:206:1::20)
 by AM0PR08MB5489.eurprd08.prod.outlook.com (2603:10a6:208:18e::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 10:13:41 +0000
Received: from VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:1:cafe::82) by AM5PR04CA0007.outlook.office365.com
 (2603:10a6:206:1::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 10:13:40 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT028.mail.protection.outlook.com (10.152.18.88) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:13:40 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Fri, 24 Sep 2021 10:13:39 +0000
Received: from e68fd4dda50b.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7427E9C3-1591-43A1-A2FC-948BAC14583F.1; 
 Fri, 24 Sep 2021 10:13:27 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e68fd4dda50b.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:13:27 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB7PR08MB3803.eurprd08.prod.outlook.com (2603:10a6:10:7f::30) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 10:13:25 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:13:25 +0000
Received: from [10.169.190.22] (203.126.0.113) by
 SG2P153CA0023.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 10:13:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d40249b-1d79-463a-bcbf-53b2e97c97c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g0p843kXw5D0c+OyBjvTrcjNQBSNRMvYTz/gjTVOgj8=;
 b=5StRR+AcSvc/nHSgJuc2vLkciSGDG0tC409w2o5eR9eF+Qj8Xiwnsz3xuEmjOoVQ/DpgRdBR6PUNmjtkNlgvcs8KPlxSog0nC7Ffp/wvBF03SRHnUDMX2ZwDzXDztKtXKsJh2bk64XG1WTwG6bImvXqOXkyWVPGYJGv9KL+BMO0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 05ee377574518d19
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fhpabmuyDqh4ZifmhBLFKBmOblX+StqJluU61IZdVkQYizFuEzhmK+Jueopjj9HDegg0i1Pw63rVJx0mCz6ebtO3/BIEELA6693HHyeQ0dC1A3Jckb5KoSwI4y4uJM8PPUUZPozsF8MaOsqTjcsaocOP/MzqHxpqrWP16IqIXccwe3hBpQav5YcxelAlI+O6EtoSy+TsHo5RkKqZnd7qWEmLRvQZst0QVMSPBrqTfF+VD3XiXPIT7qZDUHc7+fZgye7EuVD0bwLxGYYlIt5e7Oc97ijPCHqD7wOdi51uentYmmX7lQieSUD2TZtJZ1S9c/L0/EKs1vu3dDzw9OB6og==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g0p843kXw5D0c+OyBjvTrcjNQBSNRMvYTz/gjTVOgj8=;
 b=ItUY7mK6QMxy5R2mF87i6GXlZB8/7uYeRqb8CV6NveaNMkuyLvWcTHpEDXxdSsSZexwHX4VyslKy5hjpaTfWkaVcGI+/Tpqx8GwFI5BNv/3KkJMus9Sht8G6N+ByZ7n+/EW0j/DfurBFvYMRA21GBhvYIjoMbS/pIL+OHDI2gqu1FbFDw9efJOtohKCpOtm/s3n52Fvu5eWoN9x6G4p/A0+xWwPL3y8fhjgPFx7Vd2MMrZwxrcv0yJdsReaiJtrjfz4vkvFk7sAHanY8FrPCgp0cncbcDHkmWt6EWKssYae5kk2EgjwoPbnZMmlufw+Cgi8wzWLLPXJFquwVHE656Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g0p843kXw5D0c+OyBjvTrcjNQBSNRMvYTz/gjTVOgj8=;
 b=5StRR+AcSvc/nHSgJuc2vLkciSGDG0tC409w2o5eR9eF+Qj8Xiwnsz3xuEmjOoVQ/DpgRdBR6PUNmjtkNlgvcs8KPlxSog0nC7Ffp/wvBF03SRHnUDMX2ZwDzXDztKtXKsJh2bk64XG1WTwG6bImvXqOXkyWVPGYJGv9KL+BMO0=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Message-ID: <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
Date: Fri, 24 Sep 2021 18:13:18 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
From: Wei Chen <Wei.Chen@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: SG2P153CA0023.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::10)
 To DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 66170ae2-774a-423f-1bee-08d97f43fb6b
X-MS-TrafficTypeDiagnostic: DB7PR08MB3803:|AM0PR08MB5489:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB5489E467FC99A8038DB689D99EA49@AM0PR08MB5489.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 IwyJVpDxOF52/0PQu7+wmWsKZPFiKbMOmlg1A0v8hxwE6y1/5xPIiM9Lm7hrlfgS4VFLu8nVS5asRLFTEIwWNaFhHztapM9AhqJ6Wp6Z4fcC5MM96EKrxjBwYUo9LEz6xi38WJI1D8FUQ9N9/JgI0uCi8lGgLPk5Ljbavrcia60YWiz7eCV3GamxvdAvo37iockfCP3TPnuUppKk5uK3pYbXUVpDNNvrcTwB28w/0kRZ5i/ImumbsMcKvQ0mH1ZsNe59NIv+C2QxbwwzJBhzXx7/+YqDPy77HJk4i0zQ0O7fkSN6i3MiCP/V99K/qmyRrp+B6dvQ84k1/jXkWw0HNZtuUanWA/f68iaFuVnr27hsXt4Yv0CXuYiwJu5qbtVfGlLtLT6Ettwzintls3biYLmw+r5d0DsIemf0J7Arha/3YCxhIV4FMqLuydRd3wkT6fb+TtOPnr+i1/u1wAN9tiChog7LJ6vZxzC2b33yA7GoF9x4Hf1J8cR8th4suj+AdxwLzVom5+cNywNvbAbV91xpf5tutuvJa3ONdfIqF4+MJ1+Vb1wT+C5/LifAT/VRSGXkRft4N+Qh/7G01a45ImMRhtZSj7v2f8nw+uUoy/20BsziWN45EZL2S3oku5+8qYiV1NeUZ8zR3J03i9lkxjaHD+QdNNL5Oapq3HVX2nZQZLBex13neeI2ZpHDiU/EOO966oQanPm69A+UFmQ1Lz1iMLRqyaATnlHJRih2vkLbe0XyrCZ10Kc4br5dqFzHAM+lRxX/VEoW+OyLiaj/Y8uE40ne59Y0o6G/D+77siK0noVg/towNM2KJZkY3tOR
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(53546011)(5660300002)(31696002)(36756003)(8676002)(966005)(508600001)(6666004)(86362001)(6486002)(2616005)(4326008)(8936002)(66946007)(38100700002)(186003)(26005)(956004)(83380400001)(31686004)(2906002)(6916009)(316002)(66556008)(66476007)(16576012)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3803
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5ea29d8d-bd6f-40d1-b94c-08d97f43f201
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pGt1AEVxq2EEodfy+aII2QSduMuFhMpcWzj5uKK6JY1GveXQGCUcUm6us/ZIWJZ5uK6CHhmecEehLDZAqf9mWBFdqfgu8xj0LswzLOcj/K/IgyXgIawLVs7Y2Ud07YSrHZeqvqu5MY7iWkpZp+veW1ZkSxTOEQi1a2isPHy5XvJoxmIhYSJT8TgRRk5hK1sXS5MhK+akk/qTodIBy1IKtuM1ld5uFwj89YTBRiiY5e6k/B+XpfaRQe3Z/Fu1v3hXr8RenUT62cE4egemHCyyFdVAJF+vEALK3t5vcWJJCYwqz1TDeMonUv7fOhTlOhpvLkz8UX+jX8NOVA5i6IVKMs/Nsp0VSRYA7m4HGxRT/48mvBmD9KNAkE5ue3uKjsDSEEoMO6Xq0ncwQ4FwzLCiZuZYrZFCPI4Q5He8BoRn3mYcpmQ1ofdqZtwkvYJ0glNZTW81YFFFXb6ZJArccc9/JrxWHTm+UftycyH3/wJplBykD2ItO89HvuIleGlJhAXl6xhDlawdq9yPz5a14Mf5QygXBKTH9A0eN4scIV3YWSgZbtsxCsU6vR35fISQv23qKSx1tmEB7zxxTVb+LGEhadwWBLiWzHrJ+rfha8RTgYTAJiCljChKRy67PGVy7XHTQcvCU0OZpnBWec8fRdSsa2Aczk1TqdCojIV4ZwcIUZEdj4hOzW5KIFiJbNnI6iJ42WKvdmNvU1v+UcjdJKHVQ2Q59BB5NU5qz51uTKJdZqxKy0J8EarJtLhReNFHJKFj8JzK57DnKCozAsnRLSz2c9O7NqYfDxaENA98mGjtEw4B7s+311ZsfzbNL3wbb4Gq
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(186003)(70586007)(966005)(81166007)(8936002)(36860700001)(70206006)(47076005)(36756003)(6486002)(316002)(4326008)(31696002)(6666004)(26005)(356005)(2906002)(16576012)(2616005)(53546011)(86362001)(6862004)(83380400001)(336012)(31686004)(956004)(5660300002)(8676002)(82310400003)(43740500002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:13:40.6222
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 66170ae2-774a-423f-1bee-08d97f43fb6b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT028.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5489

Hi Stefano,

On 2021/9/24 11:31, Stefano Stabellini wrote:
> On Thu, 23 Sep 2021, Wei Chen wrote:
>> Arm platforms support both ACPI and device tree. We don't
>> want users to select device tree NUMA or ACPI NUMA manually.
>> We hope usrs can just enable NUMA for Arm, and device tree
>            ^ users
> 
>> NUMA and ACPI NUMA can be selected depends on device tree
>> feature and ACPI feature status automatically. In this case,
>> these two kinds of NUMA support code can be co-exist in one
>> Xen binary. Xen can check feature flags to decide using
>> device tree or ACPI as NUMA based firmware.
>>
>> So in this patch, we introduce a generic option:
>> CONFIG_ARM_NUMA for user to enable NUMA for Arm.
>                        ^ users
>

OK

>> And one CONFIG_DEVICE_TREE_NUMA option for ARM_NUMA
>> to select when HAS_DEVICE_TREE option is enabled.
>> Once when ACPI NUMA for Arm is supported, ACPI_NUMA
>> can be selected here too.
>>
>> Signed-off-by: Wei Chen <wei.chen@arm.com>
>> ---
>>   xen/arch/arm/Kconfig | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 865ad83a89..ded94ebd37 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -34,6 +34,17 @@ config ACPI
>>   	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>>   	  an alternative to device tree on ARM64.
>>   
>> + config DEVICE_TREE_NUMA
>> +	def_bool n
>> +	select NUMA
>> +
>> +config ARM_NUMA
>> +	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
>> +	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> 
> Should it be: depends on HAS_DEVICE_TREE ?
> (And eventually depends on HAS_DEVICE_TREE || ACPI)
> 

As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
option can be selected by users. And depends on has_device_tree
or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.

If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
does it become a loop dependency?

https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html
> 
>> +	---help---
>> +
>> +	  Enable Non-Uniform Memory Access (NUMA) for Arm architecutres
>                                                        ^ architectures
> 
> 
>> +
>>   config GICV3
>>   	bool "GICv3 driver"
>>   	depends on ARM_64 && !NEW_VGIC
>> -- 
>> 2.25.1
>>


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:21:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:21:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195219.347836 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiKu-00072Z-Vj; Fri, 24 Sep 2021 10:21:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195219.347836; Fri, 24 Sep 2021 10:21:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiKu-00072S-RK; Fri, 24 Sep 2021 10:21:44 +0000
Received: by outflank-mailman (input) for mailman id 195219;
 Fri, 24 Sep 2021 10:21:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiKt-00072M-N6
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:21:43 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.73]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 35fe0e78-1d21-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:21:42 +0000 (UTC)
Received: from AM5PR0101CA0019.eurprd01.prod.exchangelabs.com
 (2603:10a6:206:16::32) by AM0PR08MB3892.eurprd08.prod.outlook.com
 (2603:10a6:208:106::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 10:21:40 +0000
Received: from AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:16:cafe::7a) by AM5PR0101CA0019.outlook.office365.com
 (2603:10a6:206:16::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 10:21:40 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT051.mail.protection.outlook.com (10.152.16.246) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:21:39 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Fri, 24 Sep 2021 10:21:38 +0000
Received: from 51299a89b255.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0C17EFCB-20D8-450C-93DF-AF900EA873BB.1; 
 Fri, 24 Sep 2021 10:21:21 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 51299a89b255.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:21:21 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB6060.eurprd08.prod.outlook.com (2603:10a6:10:20a::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:21:19 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:21:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35fe0e78-1d21-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zKZgEUAaQkZ8hVm0rcV+n6jvTYurK20EOZky1CvKFdc=;
 b=JMS17EXlEgPaLJ4Sfkbig9HCwWHpxnSAtG/zroCIiP6oVIfuCG7G7G4WK+kKvHhgeY2dTqi0N906HYg2HqD9ppd+E8izi7PH4a9W5gW21HqodNRSKwkHuh+R6Fsr0UYmjiHVMKRtylp+872yiH7nPezkkJLszAnt9JHtHhGaikQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e4drZaJ53Z3VWnX3uxLNKuyy9ySw+LMJjKeSulZOkuj/n6FjQq9zE8H7NpeFJ7lsycl7U11MGxc/7j3bw/9qv+H3avYXHLaxFefzBJLxwh/aXx+yIs4UxkxfCYbWkbK7hO/gs/AHQPFWXlLzQ8AByyny0Wae7vkV6uEwDYyzF6ZetR7vBBTTLZ6cJesRO1NEudVaqFHTXUbcfdjaW8nJqHpOWKqpJ3RuR0N296xm5gTTDAH4HaSinc0bgf84xXG//kC8OUcLEueLQ2sXF6ZvF73orda2A35gL7/8kFJt/q8dcwCaJueFpCx2wsxrnxb2x7TUyKbHAth3lJhoQ+hcgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zKZgEUAaQkZ8hVm0rcV+n6jvTYurK20EOZky1CvKFdc=;
 b=MCaArhi/C9m+fSezfFyXsLF1kukx5/k+xTj2OFqspOZuqMWmByw+ksoqef6kWCvLv5h5GaOkiZtV5tP0sLY25RlwNjSL9jxyudQqcsDniYJ6ShN4FKQuumPFaCL3OL0dQgRNjNv3OZeQ++J5Nm0oAsy+8U8uZF0MUD1yV2elJ0Vkt6MHULixz7P28EWTqza4Xf3A3LxSFysDU+ZSQ7geMbzmhCSdEenMr9NX+VTTr5qlbTIKSLtSw5rSlnk7OSHtiEJcp6qZqQU3sxv9qfYMJzVVpMfnaSHxDfVHlEXUOeS3Gwh+w0Bx98UdFDv7Fhhd12mcPp8s7twUEO/vPjFPyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zKZgEUAaQkZ8hVm0rcV+n6jvTYurK20EOZky1CvKFdc=;
 b=JMS17EXlEgPaLJ4Sfkbig9HCwWHpxnSAtG/zroCIiP6oVIfuCG7G7G4WK+kKvHhgeY2dTqi0N906HYg2HqD9ppd+E8izi7PH4a9W5gW21HqodNRSKwkHuh+R6Fsr0UYmjiHVMKRtylp+872yiH7nPezkkJLszAnt9JHtHhGaikQ=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 06/37] xen/arm: use !CONFIG_NUMA to keep fake NUMA API
Thread-Topic: [PATCH 06/37] xen/arm: use !CONFIG_NUMA to keep fake NUMA API
Thread-Index: AQHXsHMNLLzNj13Mj0W39ByxhAsuC6uyTsaAgACsBgA=
Date: Fri, 24 Sep 2021 10:21:19 +0000
Message-ID:
 <DB9PR08MB6857A2291DBD9B3EC250CE059EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-7-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231703070.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231703070.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6B5006D9D23C044EBF0ABB2B80B1EABE.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: d6dca47c-f9d9-4cc3-8e50-08d97f4518e9
x-ms-traffictypediagnostic: DBBPR08MB6060:|AM0PR08MB3892:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB38928FE973BD68790EC1F55A9EA49@AM0PR08MB3892.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:6108;OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 0XiCNKa/6ijcuVY9Bf5KTTHodAmdPICHPW7/QmtPjnnA/lq/8KYcwL335XcdqBFMlGMrcJvXyRBtzolk3AmFCmIQ+QlSv1h3wXLqPGkN0toke8MJ4zMirblFmpIIyqS1X6lQnt8ZwTMqdPySBdiG8XcPGusi0wUqPuuZ7pLu2ee3mdh6ZG2n56djcnqBs7AocN8Cul7GjhIa7WnBhi8afBaRWJ5xdkikBtK6YQ4nTDhvkV5YWJYWkAd44zNecRZcCS3V7JltBIB1lQ50exqVB2P1b/eenkmoPRIePGBsOoFFh7feBjIAU8rjhUYofklwkcINcjWzdDz9/TTmEPr0e8WRhq77xvVbr0d8m56anMViGHGlO6zM5jb/q4BM2jOucKvoU8Ii1Tp19UJwy5DLLyttdky90bMnMp63W570ncUWUR/GtQeFuiupX2DbMdeJdUsv2RXTx5M7ALOY7qGMM8O9xVvZVZBgBNZAHDvKGAi9Cf/8K8E03CAD1IPBzt9iccNhoZTiVvlNcXrrhs7mjbRzwR7HjNomUvzcTL8v31j/o6DlQkVsv/4BHnf+tRVxg3GB3rvnHDapJ3naTzJl+6emSgbXC5d78034PXTxU3TOoXPailLSCwZSx50FLlqRr4KuR+DIqlkdiLP7zKy+VuSUfbxPh+gDNm1Vwxhwdm/hpPei9s5DYesBew7enPLX8cDpaafUtyqeaBa9M7/NZQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(52536014)(53546011)(7696005)(6916009)(54906003)(508600001)(8676002)(122000001)(38100700002)(38070700005)(33656002)(4326008)(2906002)(83380400001)(6506007)(186003)(55016002)(8936002)(66556008)(71200400001)(66446008)(66476007)(64756008)(316002)(5660300002)(66946007)(9686003)(86362001)(76116006);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6060
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0bfcb2d2-6f45-424d-c85c-08d97f450ce1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	p6MsYrL2oxh+UITGF+vhvSKDSccxflYE9rtCz34CO/0TwgR0MLKbtAaX71x3Nny3tgNUX2vtNb7ki+86laZ+T7XcSiqRp9K5LWHdb964OxFERZA+yjFhz8tNKswho7r08O9JF4NRTdHxliA1Rai5zpwOkpkw37HpmOCf27z7wZCfNlBCU8p+IIRufSV3ESGL2Siw3bn13vCi6Pc+sNzDWd4WwrYAOrZnKaDVQRlc0DSUiOsBgjcGFNXw9957abUDC5wdQHjUtuuKeJGAn0U1EhebUuRy64v2HtWiYsrS7vNLGimWyFIbi16S1WfhnJPqmmj8oEE/NXr9sNNI6RxSlMd8Sh9+revbGd427L2JDik3Wvw8euzb4Uu8vUkYy6NZ97pTodf4tt+W0exQyuEWcVE6H8B/BSxnkJyzT7pr362NX16X4Wyc7QtuZZT/53AZHdEqQ0JlZEzEgjUo/RiYg35GYmM/m+17/j5N5RDi/PG76ea+IzsNFDMwvaP1KUOgvFod6+eCpNlnqC3ib5Lw7BxiIr+MtD3i3nE9whol6FcVuySxzbenMn1rgotgoyd/nq/7w/fZcdHMFkHnm+hp5TLMg6jJt7hXDUzz1pkn3mDsl4yPXJw1vF0ffkxUd3iEGphe8zIA971DIARu/rzc03oHuqVH4j5XmOnK020XuM9gc4P9//QNdtzxucbbiACpMz1isHhLQxafJBl7ZOPGOA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(81166007)(26005)(4326008)(33656002)(2906002)(55016002)(70206006)(336012)(6862004)(8676002)(86362001)(54906003)(53546011)(36860700001)(52536014)(186003)(8936002)(356005)(82310400003)(6506007)(508600001)(9686003)(5660300002)(316002)(70586007)(47076005)(83380400001)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:21:39.6516
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d6dca47c-f9d9-4cc3-8e50-08d97f4518e9
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3892

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDogMjAyMcTqOdTCMjTI1SA4OjA1DQo+
IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0cy54
ZW5wcm9qZWN0Lm9yZzsgc3N0YWJlbGxpbmlAa2VybmVsLm9yZzsganVsaWVuQHhlbi5vcmc7DQo+
IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gU3ViamVjdDog
UmU6IFtQQVRDSCAwNi8zN10geGVuL2FybTogdXNlICFDT05GSUdfTlVNQSB0byBrZWVwIGZha2Ug
TlVNQSBBUEkNCj4gDQo+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+
IFdlIGhhdmUgaW50cm9kdWNlZCBDT05GSUdfTlVNQSBpbiBwcmV2aW91cyBwYXRjaC4gQW5kIHRo
aXMNCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeIGENCj4gDQo+ID4gb3B0
aW9uIGlzIGVuYWJsZWQgb25seSBvbiB4ODYgaW4gY3VycmVudCBzdGFnZS4gSW4gYSBmb2xsb3cN
Cj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBeIGF0IHRoZQ0KPiANCj4gPiB1cCBw
YXRjaCwgd2Ugd2lsbCBlbmFibGUgdGhpcyBvcHRpb24gZm9yIEFybS4gQnV0IHdlIHN0aWxsDQo+
ID4gd2FudCB1c2VycyBjYW4gZGlzYWJsZSB0aGUgQ09ORklHX05VTUEgdGhyb3VnaCBLY29uZmln
LiBJbg0KPiAgICAgICAgICAgICAgXiB0byBiZSBhYmxlIHRvIGRpc2FibGUgQ09ORklHX05VTUEg
dmlhIEtjb25maWcuDQo+IA0KPiANCj4gPiB0aGlzIGNhc2UsIGtlZXAgY3VycmVudCBmYWtlIE5V
TUEgQVBJLCB3aWxsIG1ha2UgQXJtIGNvZGUNCj4gICAgICAgICAgICAgICAgICBeIHRoZQ0KPiAN
Cj4gPiBzdGlsbCBjYW4gd29yayB3aXRoIE5VTUEgYXdhcmUgbWVtb3J5IGFsbG9jYXRpb24gYW5k
IHNjaGVkdWxlci4NCj4gICAgICAgICBeIGFibGUgdG8gd29yaw0KPiANCj4gPg0KPiA+IFNpZ25l
ZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29tPg0KPiANCj4gV2l0aCB0aGUgc21h
bGwgZ3JhbW1hciBmaXhlczoNCj4gDQo+IFJldmlld2VkLWJ5OiBTdGVmYW5vIFN0YWJlbGxpbmkg
PHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IA0KPiANCg0KVGhhbmtzLCBJIHdpbGwgZml4IHRo
ZW0gaW4gbmV4dCB2ZXJzaW9uLg0KDQo+ID4gLS0tDQo+ID4gIHhlbi9pbmNsdWRlL2FzbS1hcm0v
bnVtYS5oIHwgNCArKysrDQo+ID4gIDEgZmlsZSBjaGFuZ2VkLCA0IGluc2VydGlvbnMoKykNCj4g
Pg0KPiA+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaCBiL3hlbi9pbmNs
dWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gaW5kZXggOWQ1NzM5NTQyZC4uOGYxYzY3ZTNlYiAxMDA2
NDQNCj4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+ICsrKyBiL3hlbi9p
bmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gQEAgLTUsNiArNSw4IEBADQo+ID4NCj4gPiAgdHlw
ZWRlZiB1OCBub2RlaWRfdDsNCj4gPg0KPiA+ICsjaWZuZGVmIENPTkZJR19OVU1BDQo+ID4gKw0K
PiA+ICAvKiBGYWtlIG9uZSBub2RlIGZvciBub3cuIFNlZSBhbHNvIG5vZGVfb25saW5lX21hcC4g
Ki8NCj4gPiAgI2RlZmluZSBjcHVfdG9fbm9kZShjcHUpIDANCj4gPiAgI2RlZmluZSBub2RlX3Rv
X2NwdW1hc2sobm9kZSkgICAoY3B1X29ubGluZV9tYXApDQo+ID4gQEAgLTI1LDYgKzI3LDggQEAg
ZXh0ZXJuIG1mbl90IGZpcnN0X3ZhbGlkX21mbjsNCj4gPiAgI2RlZmluZSBub2RlX3N0YXJ0X3Bm
bihuaWQpIChtZm5feChmaXJzdF92YWxpZF9tZm4pKQ0KPiA+ICAjZGVmaW5lIF9fbm9kZV9kaXN0
YW5jZShhLCBiKSAoMjApDQo+ID4NCj4gPiArI2VuZGlmDQo+ID4gKw0KPiA+ICBzdGF0aWMgaW5s
aW5lIHVuc2lnbmVkIGludCBhcmNoX2hhdmVfZGVmYXVsdF9kbWF6b25lKHZvaWQpDQo+ID4gIHsN
Cj4gPiAgICAgIHJldHVybiAwOw0KPiA+IC0tDQo+ID4gMi4yNS4xDQo+ID4NCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:23:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:23:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195224.347846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiMY-0007dG-9p; Fri, 24 Sep 2021 10:23:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195224.347846; Fri, 24 Sep 2021 10:23:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiMY-0007d9-6c; Fri, 24 Sep 2021 10:23:26 +0000
Received: by outflank-mailman (input) for mailman id 195224;
 Fri, 24 Sep 2021 10:23:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiMW-0007d1-BN
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:23:24 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.76]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8dfa70e6-fd26-463d-9999-78a6c18f15d7;
 Fri, 24 Sep 2021 10:23:22 +0000 (UTC)
Received: from AM6PR08CA0030.eurprd08.prod.outlook.com (2603:10a6:20b:c0::18)
 by PAXPR08MB7280.eurprd08.prod.outlook.com (2603:10a6:102:217::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:23:18 +0000
Received: from AM5EUR03FT022.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:c0:cafe::3) by AM6PR08CA0030.outlook.office365.com
 (2603:10a6:20b:c0::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Fri, 24 Sep 2021 10:23:18 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT022.mail.protection.outlook.com (10.152.16.79) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:23:18 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Fri, 24 Sep 2021 10:23:18 +0000
Received: from 95fe3d34ccf0.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 ECFA96C9-52B8-483A-86B7-48EE3F725349.1; 
 Fri, 24 Sep 2021 10:23:07 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 95fe3d34ccf0.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:23:07 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB5898.eurprd08.prod.outlook.com (2603:10a6:10:20c::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:23:06 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:23:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8dfa70e6-fd26-463d-9999-78a6c18f15d7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=f7Ys0AlnPPVo+qE1SrJrFiZ+dpPJsmWm14CcFfyelW4=;
 b=NJ80COTnjLweLiDc9KMHhSQmlUSZ1PaWTmutBySCc2xOX/qzch9cfVfGOGlG3apOYFCZ07ubLLo+7At3k4qo5douRYppkxQpiWxe8ov73RZEOeIMjYzk+JB6XBctSUUw/jfCfbcub+2p+kjHsWNFBx15CLwtcL27xHrTP+9olrs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wl9yX3EswmhQ1GQzHRvkVl4gdAssJ+7rGNnJZS9s4h9WxySmsc9TrcpjOOYwfXAMNINR/lBLGKA+QOrYANs4G+IeGz3AzL9NWOm7EkQn+6hgrawovVg2CBU3nCvyd3fkOVCspfEFpor8jeNQqA3Bt+VlX71aXf4nSFmpFLu0qN5M4xYWPD2pPE0qZafPc52gCXiMp8ExoEQNEoTQPoqVPmCn806N65BuqC5dv+mC8/D60OlJvtHqDIxbypvOsU+uGLh0l8vHA2sw/fcgDlPC5m8qtWnT8x4sSnah3nokOAz70CKs3ZUyhm0AtYLxnkBeg6ehCz8lHzK2+ezg45HjAA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=f7Ys0AlnPPVo+qE1SrJrFiZ+dpPJsmWm14CcFfyelW4=;
 b=Yudf/yQqINIueVuj77ENZwjIMDEAvOXYqpDFWaR6Y2XhAnrxLlwNes9Nf5wm1+sh+UTtxO1/4GvaS98Gxajcw83rRabTe7xnjk9HtsHm9fYXKypjk6R57uRJzF5Cn/PmrlU/Idt5aMDluYqcaGaJrHbY0pWTO7ZCtQujyodu6mjSbz2FRvUC5rzYsvAEHUWXPW6CH8+aFpuNp0q9+Gk0OrVX9Uu0BjzpKkG0RCKBQirWxWdnNCsW/ZvwA1newFy8oIOptBOGtj6nq1gTHNaq4dNYD05CyXXmurYjM6dFG09Lw7ZogL59OEwpOtRVOJaliLlKrON0H4rTOAGsTccTpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=f7Ys0AlnPPVo+qE1SrJrFiZ+dpPJsmWm14CcFfyelW4=;
 b=NJ80COTnjLweLiDc9KMHhSQmlUSZ1PaWTmutBySCc2xOX/qzch9cfVfGOGlG3apOYFCZ07ubLLo+7At3k4qo5douRYppkxQpiWxe8ov73RZEOeIMjYzk+JB6XBctSUUw/jfCfbcub+2p+kjHsWNFBx15CLwtcL27xHrTP+9olrs=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 33/37] xen/arm: keep guest still be NUMA unware
Thread-Topic: [PATCH 33/37] xen/arm: keep guest still be NUMA unware
Thread-Index: AQHXsHNOacjZjYh6d0ebH0gUeyb/WauyhQiAgAB2NnA=
Date: Fri, 24 Sep 2021 10:23:05 +0000
Message-ID:
 <DB9PR08MB6857A249AB6D5CD5B8FB62649EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-34-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232018070.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109232018070.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 7E134BADA0B60D4CBD9569D282241422.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 1dd9c1a9-0ffe-4439-3b46-08d97f4553f4
x-ms-traffictypediagnostic: DBBPR08MB5898:|PAXPR08MB7280:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB72807991BB5B4D3FE0150AE09EA49@PAXPR08MB7280.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ++qj+Ie7ORBMAqkA7ppMa1/uv+xeuis2HPQAp540XBwgpNfp4GqjYcQC0/OxsKC5NxDLWHvac9BpIU/i9WlgOLzK2tbMf3ZuDWeYetQgFfPQwbuBq+HWMzoLiEpI/KP45fW1jMaYh08HojBEm0wEeg1Cv6mf6IhjTv9ZYLzquRuONYoSHLQ7glXSnP7rS7DZ3i4S8SkJDdrFDUMM5nqtzHzh4glX2ZbrxBw88ZxwU7zkC3WQ2aG0ZwOirrZz+obizJ47LpO5fV1+2lz1Ge4cXrPwi31ss+bkK7St9Ya0YP6ofZOhHcbP/rydpYn9B6HwGWxWo2+pEDiV3XQGLlfXZEPJcycVzl3aD0aPw3peUlKF7I9LAaGtoUqw53VYoqCvfcwc4DuPLHg5dJg6EFVoYzAnAfW4q4BRs/YqKoX4ue5y/TVhejkDpkdbzTnK3oahmfTGoiRkDgbakMYnZplRPgmhFaTnajsLA77CROLY+tO0trbUkwUZfoTJRPxD7nGMV8FH8iTC4ZkHpPL/09GNuI4DlupOyzjN8k4b2qJxUVLFcVXCyoUlAHNMLsA3X+SWi+FJKkIlwDKjAD2eBnCX30SSCiu9N+7fbBz8sDfUnBHsm/IXTU4bmD2hhkImH7guuC/pXptmH14snX9TDPr25mpQ1nAts4YWHsYTRjhmuZDYpwzM/n6Cosj9bbU5cOHVSYwemXZ4snW2ixjO+nhugw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(8676002)(66556008)(186003)(66476007)(76116006)(66446008)(38100700002)(8936002)(316002)(86362001)(55016002)(508600001)(66946007)(54906003)(83380400001)(52536014)(6916009)(122000001)(7696005)(71200400001)(4326008)(38070700005)(6506007)(33656002)(2906002)(26005)(9686003)(53546011)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5898
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT022.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	143c24ed-7264-4d09-7803-08d97f454c74
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1E2YYqQpC5LdP8ENJvWpBPeuNGZ0aqE4g2MebPCwBvZg+m0kXF5UUyZQfuvAENmo4F5UymGzsptvdGv2lJgLaAfEa/4Rk5i98IB1iXrgrVqy8ZACGVzsTqCM5IIPTy9zFQFVf1yXi7BKt8qg/yA2mX4S04TycP2Dr95V/WNd35EnV/Bl2pSnKsHDoKuONBr0TVuFtF0CRxA/8TdsMjlo5NxJCG5stn8lcQbIRBSve1smXBsmO+T7U5REY9Pt9QRtudiB1mnnsj3UHFs1DnowA7RvrxtLpXNRn78YvDjYvz2AsrhqCyRH6ydfSQLwDoHLGjSLixGX/OGWy2mLjm6ksQnzlZiNFivLYLCuO7oHqEotTY3zYSwFwbYaFfZQJulDZMRQzzVSWu2hZ9+adlE+eA7Ej5cihO1Bu8P4K1gi/JFoW1sDsXgBtnn/cvmwOIWHS6YKEDhUtDZ92HRvZdIklXwq7AcsnAO+tcYln/uwl0n4L2syHR/IrnDDHh8TAxXRlJJSv7Q3B8pIYiUkBW5RPGiEjQwqltMDinbC+eiG9OaOH/4fQAdBvXMi8FyIfbQRPpH/9khHP975pq8wprDkU3NKoIIU4vCjMz7Yhne5zUi6jNIP/dbr9KUtdTto6OSkpZCOfdUOtysGC13piPIHiilud4TPpXt7stXC33n5KrRJWVFNnSfukkITPnODs7joJadFXnKx2ZquWA/lq1qLCA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(70206006)(7696005)(8936002)(36860700001)(316002)(81166007)(508600001)(70586007)(82310400003)(52536014)(9686003)(26005)(4326008)(336012)(186003)(33656002)(86362001)(8676002)(6862004)(55016002)(6506007)(53546011)(5660300002)(356005)(2906002)(54906003)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:23:18.6942
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1dd9c1a9-0ffe-4439-3b46-08d97f4553f4
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT022.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7280

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDogMjAyMcTqOdTCMjTI1SAxMToxOQ0K
PiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4tZGV2ZWxAbGlzdHMu
eGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGllbkB4ZW4ub3JnOw0K
PiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+IFN1YmplY3Q6
IFJlOiBbUEFUQ0ggMzMvMzddIHhlbi9hcm06IGtlZXAgZ3Vlc3Qgc3RpbGwgYmUgTlVNQSB1bndh
cmUNCj4gDQo+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IFRoZSBO
VU1BIGluZm9ybWF0aW9uIHByb3ZpZGVkIGluIHRoZSBob3N0IERldmljZS1UcmVlDQo+ID4gYXJl
IG9ubHkgZm9yIFhlbi4gRm9yIGRvbTAsIHdlIHdhbnQgdG8gaGlkZSB0aGVtIGFzIHRoZXkNCj4g
PiBtYXkgYmUgZGlmZmVyZW50IChmb3Igbm93LCBkb20wIGlzIHN0aWxsIG5vdCBhd2FyZSBvZiBO
VU1BKQ0KPiA+IFRoZSBDUFUgYW5kIG1lbW9yeSBub2RlcyBhcmUgcmVjcmVhdGVkIGZyb20gc2Ny
YXRjaCBmb3IgdGhlDQo+ID4gZG9tYWluLiBTbyB3ZSBhbHJlYWR5IHNraXAgdGhlICJudW1hLW5v
ZGUtaWQiIHByb3BlcnR5IGZvcg0KPiA+IHRoZXNlIHR3byB0eXBlcyBvZiBub2Rlcy4NCj4gPg0K
PiA+IEhvd2V2ZXIsIHNvbWUgZGV2aWNlcyBsaWtlIFBDSWUgbWF5IGhhdmUgIm51bWEtbm9kZS1p
ZCINCj4gPiBwcm9wZXJ0eSB0b28uIFdlIGhhdmUgdG8gc2tpcCB0aGVtIGFzIHdlbGwuDQo+ID4N
Cj4gPiBTaWduZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAtLS0N
Cj4gPiAgeGVuL2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jIHwgNiArKysrKysNCj4gPiAgMSBmaWxl
IGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNo
L2FybS9kb21haW5fYnVpbGQuYyBiL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KPiA+IGlu
ZGV4IGQyMzNkNjM0YzEuLjZlOTQ5MjIyMzggMTAwNjQ0DQo+ID4gLS0tIGEveGVuL2FyY2gvYXJt
L2RvbWFpbl9idWlsZC5jDQo+ID4gKysrIGIveGVuL2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jDQo+
ID4gQEAgLTczNyw2ICs3MzcsMTAgQEAgc3RhdGljIGludCBfX2luaXQgd3JpdGVfcHJvcGVydGll
cyhzdHJ1Y3QgZG9tYWluICpkLA0KPiBzdHJ1Y3Qga2VybmVsX2luZm8gKmtpbmZvLA0KPiA+ICAg
ICAgICAgICAgICAgICAgY29udGludWU7DQo+ID4gICAgICAgICAgfQ0KPiA+DQo+ID4gKyAgICAg
ICAgLyogR3Vlc3QgaXMgbnVtYSB1bmF3YXJlIGluIGN1cnJlbnQgc3RhZ2UgKi8NCj4gDQo+IEkg
d291bGQgc2F5OiAiRG9tMCBpcyBjdXJyZW50bHkgTlVNQSB1bmF3YXJlIg0KPiANCj4gUmV2aWV3
ZWQtYnk6IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gDQoN
Ckkgd2lsbCB1cGRhdGUgdGhlIGNvZGUgY29tbWVudCBpbiBuZXh0IHZlcnNpb24uDQpUaGFua3Mh
DQoNCj4gDQo+ID4gKyAgICAgICAgaWYgKCBkdF9wcm9wZXJ0eV9uYW1lX2lzX2VxdWFsKHByb3As
ICJudW1hLW5vZGUtaWQiKSApDQo+ID4gKyAgICAgICAgICAgIGNvbnRpbnVlOw0KPiA+ICsNCj4g
PiAgICAgICAgICByZXMgPSBmZHRfcHJvcGVydHkoa2luZm8tPmZkdCwgcHJvcC0+bmFtZSwgcHJv
cF9kYXRhLCBwcm9wX2xlbik7DQo+ID4NCj4gPiAgICAgICAgICBpZiAoIHJlcyApDQo+ID4gQEAg
LTE2MDcsNiArMTYxMSw4IEBAIHN0YXRpYyBpbnQgX19pbml0IGhhbmRsZV9ub2RlKHN0cnVjdCBk
b21haW4gKmQsDQo+IHN0cnVjdCBrZXJuZWxfaW5mbyAqa2luZm8sDQo+ID4gICAgICAgICAgRFRf
TUFUQ0hfVFlQRSgibWVtb3J5IiksDQo+ID4gICAgICAgICAgLyogVGhlIG1lbW9yeSBtYXBwZWQg
dGltZXIgaXMgbm90IHN1cHBvcnRlZCBieSBYZW4uICovDQo+ID4gICAgICAgICAgRFRfTUFUQ0hf
Q09NUEFUSUJMRSgiYXJtLGFybXY3LXRpbWVyLW1lbSIpLA0KPiA+ICsgICAgICAgIC8qIE51bWEg
aW5mbyBkb2Vzbid0IG5lZWQgdG8gYmUgZXhwb3NlZCB0byBEb21haW4tMCAqLw0KPiA+ICsgICAg
ICAgIERUX01BVENIX0NPTVBBVElCTEUoIm51bWEtZGlzdGFuY2UtbWFwLXYxIiksDQo+ID4gICAg
ICAgICAgeyAvKiBzZW50aW5lbCAqLyB9LA0KPiA+ICAgICAgfTsNCj4gPiAgICAgIHN0YXRpYyBj
b25zdCBzdHJ1Y3QgZHRfZGV2aWNlX21hdGNoIHRpbWVyX21hdGNoZXNbXSBfX2luaXRjb25zdCA9
DQo+ID4gLS0NCj4gPiAyLjI1LjENCj4gPg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:25:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:25:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195229.347858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiOx-0008Hz-OE; Fri, 24 Sep 2021 10:25:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195229.347858; Fri, 24 Sep 2021 10:25:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiOx-0008Hs-L3; Fri, 24 Sep 2021 10:25:55 +0000
Received: by outflank-mailman (input) for mailman id 195229;
 Fri, 24 Sep 2021 10:25:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTiOx-0008Hm-7n
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:25:55 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc2492dc-1d21-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:25:54 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2052.outbound.protection.outlook.com [104.47.2.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-12ZFw3nFMUO77ZMEnZ1e3Q-1;
 Fri, 24 Sep 2021 12:25:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7149.eurprd04.prod.outlook.com (2603:10a6:800:12e::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 10:25:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:25:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.7 via Frontend Transport; Fri, 24 Sep 2021 10:25:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc2492dc-1d21-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632479153;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IWDsTB/XrnmD4UkTyPlFg9JMBb+0jAMIBX8dFfssLRM=;
	b=bAKHHpEz0bpj+wPQOEz7yN9hUCqfF2b5DynkV3uuwiDyCXbdF1tlrz7soR/dIvyEzB8xeJ
	p+jXtFq1AcO+ZemktVyG1WHyooHTMmaMwhdAafiJEv4HnhQzfBspkESl5VyeLKVCn0NiIa
	A5OJZ1brlSk+KYET9Dai+Mol1qQ7wUk=
X-MC-Unique: 12ZFw3nFMUO77ZMEnZ1e3Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Jvay3FowmX6zIIQXRijjnvdekZZwWmyPDRuxd71BeyV49CviMPNzA6M3RqNfpUpn1qciznz6NEyvhMriQeyb26T8rg3DaTxVTkSTugKW8dJ9ouZqHql4nEUjYfg41faESFrCasRr9L3BVcJE/xz23SGrOOZpXYUrJyN8jw3s2iTpQZDoJkyyCW1EwwhUCUOlAd7ICDnf4DwhgcaIb5QKtRVDMbRBU5Ww3Rbp7XjCtIkPOo1+pXBkMx4WipDoZRpf5w6Y6KlLVMtDq7eof7R9b77VZxKt3qlzL+q5Z4ALQwk1edQYrwlZ74XSZeOXn9tPHiicubcH0ZP5XuKMs6wGnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IWDsTB/XrnmD4UkTyPlFg9JMBb+0jAMIBX8dFfssLRM=;
 b=bjzRERp5LUFZnlA0j4w/3woOICe0Tr7gUk9f+30z2v89uOT6g6PYMG1791V2PRKJ7bpg7FgyY7yrAW0O0JufxriuOOymlbopxCDmF4bz5eyhpEe75BRZLnhtJpljbVzLENrA6raVuo7efB6PITvsI2tUSuUSM87uFHkrlWvzrygVBISvQI64oe2XkOT+mZhOgUadM3Ky6/qTY0YldHeNKr3YqztdOwsiNL1kFvJUVNpmvvvdk56VOfmr8vv5fAb4vSzb5aTkH98QLKQvcG1We/D/knTBNRQpYuQhT38g3fJHpLVSnd1nEreFBthJ02SReLxF4mMr8fU8dr3w54DjTw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
To: Wei Chen <wei.chen@arm.com>
Cc: Bertrand.Marquis@arm.com, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org, julien@xen.org
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a3d3db98-420a-79a2-97a7-d332aee5b531@suse.com>
Date: Fri, 24 Sep 2021 12:25:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923120236.3692135-37-wei.chen@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0037.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f61de166-ab00-41cd-9b12-08d97f45aea1
X-MS-TrafficTypeDiagnostic: VI1PR04MB7149:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7149B0049AFFF390AC36B412B3A49@VI1PR04MB7149.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oYLod0DoG8A7NMlhcZGXhGfMNwlZmPTd8vfLWgGXWrxK5z3NR13rpp2zThdW+9vuQP5TGAY3pnbtnulO14y+VVjIG6camXo+Vri+ywsYqXQBSSVq9jgnljn2gyzYss7q6PRe6JP/vBs4DPyUJpsx93Pe4RMWHOvWvowiYszc6q9dQ7gmUKTyhgoXQI78XfXoqtQaY1CHh6EMZkcnfIAu5HH9IE3MoGtBVl/JcRLZTruorJjVDaziGqrjv/P2bemIJBVDBKntULR6DW4X4PTrJDJpdFzN7zOpd+3wd5TQWBlZAimpQyshqBE7KxZw6asf41gDih8ypyYkaBJ+troCUWqmRmjs1kfj6Vl6U82J2TWvAw5/cHonbLt8C8Cm3Ofg+g7B4mHEpaKUtoHHZsPLPyFmqLWVZdhHgeNgVu4NUcrEMQUbHb9Z7Aw6wCScCGA+YV/UOIMz8SbgwzVQiZaVkWolmq4j9Eyx8GjxpTkYQvEy/fQOn36PCtfON869UO41KSVW3sjPaiS9MYum4bbcASLi3yovplS+eZM1WKzkG3YF+stYGupCHlBxvAJq1BNfU9RYx3/B6mW+ZEG4zrd3Ze6uUVobNIaH9iFL22ejw5kTcrTXEYm36Y1HZfdocR8Tc1ijZhgYzp0afBhgnNFuJhELR0QFpDvBt867F+oroBxku6+aPGcOJbdg+OGNpl6YRCnTionH7oa+koqf7wuf6MocDQNsThf1Uaukz6MsKV8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(38100700002)(186003)(8936002)(4326008)(66556008)(66476007)(16576012)(31686004)(26005)(956004)(83380400001)(6916009)(316002)(2906002)(2616005)(4744005)(53546011)(31696002)(36756003)(5660300002)(86362001)(6486002)(508600001)(6666004)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDVKbWVXekJaUEwwMDlJellrWXdkNmRZblY4ZWw3aXl0djhhSmtxVXBmZXBJ?=
 =?utf-8?B?Rk9XUEpOLzA1YzdHZzRwajFxUUtIYlhva2pRdUk0RUZVQThHRlZMM0RHOElT?=
 =?utf-8?B?RkFkRTl2TzM3WDVreWFkcGE3Q0ptNTdqZDJGei9zc3Q3R0JYZU5QUkJOdjNk?=
 =?utf-8?B?dWpSbGU3dTJyRVZFdEtYdnR3cU1uNEhUN2dKeE02M3J3dGVieW1heW5YM0pL?=
 =?utf-8?B?RDVPdjZHeXgwS1p1YTBMdXdrdkVYYjc4YmJteE9tRWFFRG10LzRlL0E1TlZR?=
 =?utf-8?B?NWdaeUFmMzZXRzlia2tXN1drdUtCbnhYeGt5dXZXNWc4YlJuaWg2RVZIdkxp?=
 =?utf-8?B?RUtPMmk2VWszMVduR0tvNkdGdzdLWXJUUk9kbUcweGtSSnlNWEsyMFEyVG1K?=
 =?utf-8?B?M2pyV3N1NlAxalFxMzRVSVRqZFM1UkZTYUZ5bmNUWjBGUk9RQWRhSE0wQXdo?=
 =?utf-8?B?STBHOUZkVm1qUDhBbU96NVRrVjE1anRRYWcyUmUreUFqY2dTNGJubFhBRG1j?=
 =?utf-8?B?TGdJdFNpV1dlczBlMkRkUnNwaERHVnlNS0tUNHAvUDZWU3Jhd3FTVWtZQ2VH?=
 =?utf-8?B?MTBlSzNlQzQ0SS91UUZwbmg2eEFtcCtCbktXeWFPVWJWVmpBK1FKcXFmNjJY?=
 =?utf-8?B?SUEyN283WDZrZEYzTlFuUVFzUWxYOVdXQ2tiVFQreml5eGJsVkFneEx1S3dQ?=
 =?utf-8?B?WkR5c3FleE9ZL04rQTZRWnFyUnRrTWx1eVloOVVtVFFjRkdjanV3dmhPcnoy?=
 =?utf-8?B?czY4OUk5czRzSDc4NGg4ODBHZ1IxUUgvaFhIZ0pQc21VYjMrc1RQdnl6TG5G?=
 =?utf-8?B?LytOUUV1VUxsTmdxbi9IZUdSem5Damg0Y1NEeHJ5Mm5VR1gzOHl5Um5IcmpF?=
 =?utf-8?B?MFFOVFZGbUtZZ0lYd0FlT0wzallJZ0ZaMy83VFFWQ2gxNjRqS2hNcUxLLzJT?=
 =?utf-8?B?cW9XTmZEWkFud3UwM0plYlBWZmtuY0ZnZHVnaW5ZUy9mMFlHeG0zd1c4WjhY?=
 =?utf-8?B?eWI3Z1l1TGVPVGttaFowVEFKT1c5Q29YYklaY0M2Y1lMWlNEZHV6bStpME53?=
 =?utf-8?B?eHJXNnBtR0hINnRTSEdjcXMvNVd1eERHOWh6WWt3S3pvUENCNWlRVmNhV3VZ?=
 =?utf-8?B?ZDBVNVBNT2lQTDBRMDlIWFB0ZitMWXBoeUJKMk9wSEVIVlppM25QbWJJM3ly?=
 =?utf-8?B?QlcxRHdkdE1vckNOdUxITnVFUHNyS0YzYzZlTy90NFU2c0hJTjcxekRHa2Yr?=
 =?utf-8?B?Y3U2dG4xTUF6dmlNVUw2dHJDNTRXOHR4ZTdrdkVoQXdERTByRWJJcGJFd2N5?=
 =?utf-8?B?d25XZldBSi9BODFxYytQOUhEaFFpd1Rzb3VROUF5dStrMHMzc3hCb1dxQlNO?=
 =?utf-8?B?RkJhMGx6L0JTS0RJNHB1dDZsdUNtQVkrMHhtVXE3SmpTbjNDOHpiRmFPczcy?=
 =?utf-8?B?RytlV0lhZXVKK0JQR1BJZUtHQWpwbERqdll1YS9MRy81U0QxVkJmVFRDRWk2?=
 =?utf-8?B?K0RpZXQvelEwQWt2bksvMkhSQzlRTW9SblArVXhIUTQwUkk0cUsxeU53NElX?=
 =?utf-8?B?SzBlcE5jOEQzU1AvbWFnVUZFYXNEenFnbTZIVVZaZForNWhrem9qWGxhMUFV?=
 =?utf-8?B?d3FiT3pUUFF0ajRhb1ZPeUVmQ3NlQ0Y0aEtzMndleVVia1htenVKblNRdDJQ?=
 =?utf-8?B?bHAwTHJBTmp5dW42NnBudVZ0dVBacmhvd3RPendIMlZ2NUVjSm02YTA2V1hR?=
 =?utf-8?Q?DLHUmIO7j5YEcjhDBpYoul9bNdaHhNg1TlWqOf4?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f61de166-ab00-41cd-9b12-08d97f45aea1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:25:51.0302
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JUU5vXpi778KfAeKCp5LYIhpX/9wrlERXAF3cfhHq+BA/09iqOGbglYAitJPXluDh2K3nBAq4Jjrcotst+m2Tw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7149

On 23.09.2021 14:02, Wei Chen wrote:
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -34,6 +34,17 @@ config ACPI
>  	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>  	  an alternative to device tree on ARM64.
>  
> + config DEVICE_TREE_NUMA
> +	def_bool n
> +	select NUMA

Two nits here: There's a stray blank on the first line, and you
appear to mean just "bool", not "def_bool n" (there's no point
in having defaults for select-only options).

> +config ARM_NUMA
> +	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if UNSUPPORTED
> +	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> +	---help---

And another nit here: We try to move away from "---help---", which
is no longer supported by Linux'es newer kconfig. Please use just
"help" in new code.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:31:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:31:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195237.347869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiUE-0001N8-FE; Fri, 24 Sep 2021 10:31:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195237.347869; Fri, 24 Sep 2021 10:31:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiUE-0001N1-CC; Fri, 24 Sep 2021 10:31:22 +0000
Received: by outflank-mailman (input) for mailman id 195237;
 Fri, 24 Sep 2021 10:31:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiUD-0001Mv-Ek
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:31:21 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::61d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f5a407a3-fc88-4bf0-8bb3-9a061a8f1cce;
 Fri, 24 Sep 2021 10:31:19 +0000 (UTC)
Received: from AM6P195CA0071.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:87::48)
 by VI1PR08MB3743.eurprd08.prod.outlook.com (2603:10a6:803:b5::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 24 Sep
 2021 10:31:17 +0000
Received: from AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:87:cafe::10) by AM6P195CA0071.outlook.office365.com
 (2603:10a6:209:87::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 10:31:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT012.mail.protection.outlook.com (10.152.16.161) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:31:17 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Fri, 24 Sep 2021 10:31:15 +0000
Received: from 0f1836c68c29.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6B168DCE-86AF-4034-9BA1-8078932968B2.1; 
 Fri, 24 Sep 2021 10:31:04 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0f1836c68c29.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:31:04 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB4572.eurprd08.prod.outlook.com (2603:10a6:10:b0::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:31:03 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:31:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5a407a3-fc88-4bf0-8bb3-9a061a8f1cce
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DNqoZ/UOsh2oKbYQi+6YlA+8uaNRduIatnzyl5VMi+o=;
 b=zol9dOqqdWCvKjYVYEj4aNR6B5Dtxz2eCKeroNyUskoG2L0pG9pvvxmxkn5OsYWilpD3wRmTtFou6PHoziIb6Z94fQL7C9p/bkLKARxbLlY/37MkVgFd3H3ygJaLLWmFjboSWLLoHaLsKwGdqcqi2rbntLnJGHEE/Qx9QHhzDPo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ty8STzCIhOTToQA73yRr23B4SKIY23S9XnbxUzwWs0u8RFAld3lBnA4C45nqafDLM/UPrV89qWm+qXUApwM4kT0r0IYk+wUZvqxPyMqJvZ87YSbr6TdJTkvMOf2PRzzvSBszJqrV9PavBiYtBfelCSuOHCszhsa+233YCua2I9qjInSEDzu0J9KAh7hGFgBxfkG99LlLTgwuZuNYBJ+F2fLdH33E5cwxQAGXKfpFnA683cXolPnAbdMUQd9YC1oIvGAAyNSuo7kguZrldhWc20W/8X+trneDrGcExH8+bsxWbllV+dg5IdaMPXEbY9OOUfWv4bvOPHv8AKt1bHocsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DNqoZ/UOsh2oKbYQi+6YlA+8uaNRduIatnzyl5VMi+o=;
 b=NtIbFlRMeWqS+UDS8wWVONBEAFaf4KCyN//6nGWF4gOYnFm1FTwKzRcFALyk9yshrnCpNxOHbOCoUiTtZiIH//5SkabHJ5gEP/v58ouRaUeL4Xaa5vjYlntqvAOc3v3OxFaMx3lkT5X98nNHwnr0wNdwOhbwukAYIZRKMLf2WTK5g4wTAZHM92Ditbo1pizO+JhUPq/5+GDqXjdKy6gX+5zIxoguiM458BFZIJdLTaMVhgseqZoE13Acp5sGX2U0JSVQGDXuWZEfhUmAF+QtUyoSChLpNPpYonWSOLgJNudF94SrEcQ9JNrVBZZNhyC3qSf9XM5E52XT38wnfWWL+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DNqoZ/UOsh2oKbYQi+6YlA+8uaNRduIatnzyl5VMi+o=;
 b=zol9dOqqdWCvKjYVYEj4aNR6B5Dtxz2eCKeroNyUskoG2L0pG9pvvxmxkn5OsYWilpD3wRmTtFou6PHoziIb6Z94fQL7C9p/bkLKARxbLlY/37MkVgFd3H3ygJaLLWmFjboSWLLoHaLsKwGdqcqi2rbntLnJGHEE/Qx9QHhzDPo=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Topic: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Index: AQHXsHMtR+l18+MmpE6rl/RjSXTBgquyYmkAgAA2xQCAADnbAIAAKWhQ
Date: Fri, 24 Sep 2021 10:31:03 +0000
Message-ID:
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
In-Reply-To: <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 5A6E1B38E53A0845A2F880F3D13691E8.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 9515af94-1fb2-45f1-b919-08d97f467122
x-ms-traffictypediagnostic: DB8PR08MB4572:|VI1PR08MB3743:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB37434F79274B134132B63BA09EA49@VI1PR08MB3743.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 ORUtDS7f6u1eFh9NNHozNAP/So6gEZgsaoyr5Pzqf38ukZ4QM4qVsNc4+gziSb3Xd0Boj8ha0uQ2XB7VeQ68mQGqyup7NCWZJjyvwzHep+Rr3ugFRRsRf356lP4EZlzNSVlHaOtAHsJzayt5HmVHO9+bssi46fm+a5tybznLjX4LPZzi+P3Df+M3OjPybmHROoqq0Ho9T/JdHAMiXc3a1vfEy/MhsjM1rkLVIjfIM+ZZHo/g9Cz6DuCzXT4TxX3+fkwE8HCZc8kY/tBgPY3MCzUy/f9QXofJB7kb910co9DMfABj1wz7qSNoezf9/wJU2jTXW0t5d/HDCdwgFwyQ08dvp3LhlLPmZJeO52JVqcxzoqXth+1u/x9vXgMV4fnDUNqJaWzBpST0x+b4Ki07eqyhkJ7TLpMgwtSGDuc6v9ef8ZZudm/VuvMz1OVZ162aKfAkIoHk4ybA9wpFlS2vcRfcg/xgDx4KQWZSe6m+9Wm7icUb2e7CPN1L60a+M+kpULHnw8+nPJp09Tswjpm+6m2hYJsNFLPsF9iB3UtOQfbBQDvLaAZZYPR6j0Ku0vpKVI+ZmFS5g/8H0mBjPGiBu2+SlJWypi5RKwMSbdqNTk22DgQHxvscYpuM5sfoc3Yh2eu60KsnmyP4VN1N0s8xGdEKxVLK1jQEBz5WPVjAmtd9HYYae6astEzJZplSq45aSPDSUzmdtIPLGEBIGyMu+w==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55016002)(5660300002)(316002)(38100700002)(508600001)(66946007)(66556008)(33656002)(6916009)(8936002)(2906002)(66446008)(52536014)(53546011)(86362001)(76116006)(9686003)(83380400001)(54906003)(6506007)(66476007)(186003)(38070700005)(71200400001)(4326008)(26005)(7696005)(122000001)(8676002)(64756008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4572
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e47acb85-86c8-47fb-d5e8-08d97f4668f7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9DCqyivwfR76XUNgFCtS2Uuvm3H68FRzRP2TVbIg3x5nt23s62MjHYYaheRVR6B0jKqLhVxytdUSMF9ArZYBKUiHjJwiZfvT8UNSXPDJkWS8u/0C1Eu6OJvn/YWZuTXi4LHazesjlZbermGpu7fwqCOFueHHHFqOvXqTYQHZpw094M8nuZST7k6RflqftcMsOa42ym68uTX3wbYdwvFGiWYiQTwc8rGvG47RkVvZQfvzlui0h5Gmr2mLUWgwqfMCiFeCb+GeaXZSqRWaUZgQqC7y3f+DLq+8Tsl8cAUnPrINW0ePMfU455eu+TDeSHmEEt2kCQrcbFFe5XWBKBRQ7N3vCJn7cZjLVNw9MlJUzC/TS5j3Gzk0lOjW3zfBD0OzamlJtuxU9VZFegIhYiWXraltbjdonfpDJGOvGey7B5pOHnYIqSNdTtwgfWue7xtaurF8/bBmEHqI1IAaqdIi+FPFzE6zoycScS15b6P/06VYcJbm0FbIL1ccsNp7qmXAVMjNzhyoIhCKOv4x6Vf9V0KG1UiDEelJ4rSN3SS7CFOr8R0HU0en14mHSPBTzLQ3lbuvxSo8ZOjMFzAM5eCEtqaK43scfsc/6GJC4SHSapJE1TkOhQMFpPHTnIRnXcxOf+9asfsIBN4V5qmr9TYNh8qiySccvbLGyazavfvno5arkgrrLkooca+sifK9l45JizhmpVHEa9kz0GtMBh9UWQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(26005)(36860700001)(2906002)(52536014)(55016002)(6506007)(6862004)(5660300002)(186003)(9686003)(8936002)(107886003)(82310400003)(8676002)(53546011)(86362001)(70586007)(7696005)(316002)(70206006)(47076005)(83380400001)(4326008)(508600001)(33656002)(336012)(54906003)(356005)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:31:17.1555
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9515af94-1fb2-45f1-b919-08d97f467122
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3743

SGkgSmFuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gU2VudDogMjAyMcTqOdTCMjTI1SAxNTo1OQ0KPiBU
bzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVu
cHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+IDxCZXJ0cmFu
ZC5NYXJxdWlzQGFybS5jb20+OyBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5l
bC5vcmc+DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjAvMzddIHhlbjogaW50cm9kdWNlIENPTkZJ
R19FRkkgdG8gc3R1YiBBUEkgZm9yIG5vbi0NCj4gRUZJIGFyY2hpdGVjdHVyZQ0KPiANCj4gT24g
MjQuMDkuMjAyMSAwNjozNCwgV2VpIENoZW4gd3JvdGU6DQo+ID4+IEZyb206IFN0ZWZhbm8gU3Rh
YmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPj4gU2VudDogMjAyMcTqOdTCMjTI
1SA5OjE1DQo+ID4+DQo+ID4+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0K
PiA+Pj4gLS0tIGEveGVuL2NvbW1vbi9LY29uZmlnDQo+ID4+PiArKysgYi94ZW4vY29tbW9uL0tj
b25maWcNCj4gPj4+IEBAIC0xMSw2ICsxMSwxNiBAQCBjb25maWcgQ09NUEFUDQo+ID4+PiAgY29u
ZmlnIENPUkVfUEFSS0lORw0KPiA+Pj4gIAlib29sDQo+ID4+Pg0KPiA+Pj4gK2NvbmZpZyBFRkkN
Cj4gPj4+ICsJYm9vbA0KPiA+Pg0KPiA+PiBXaXRob3V0IHRoZSB0aXRsZSB0aGUgb3B0aW9uIGlz
IG5vdCB1c2VyLXNlbGVjdGFibGUgKG9yIGRlLXNlbGVjdGFibGUpLg0KPiA+PiBTbyB0aGUgaGVs
cCBtZXNzYWdlIGJlbG93IGNhbiBuZXZlciBiZSBzZWVuLg0KPiA+Pg0KPiA+PiBFaXRoZXIgYWRk
IGEgdGl0bGUsIGUuZy46DQo+ID4+DQo+ID4+IGJvb2wgIkVGSSBzdXBwb3J0Ig0KPiA+Pg0KPiA+
PiBPciBmdWxseSBtYWtlIHRoZSBvcHRpb24gYSBzaWxlbnQgb3B0aW9uIGJ5IHJlbW92aW5nIHRo
ZSBoZWxwIHRleHQuDQo+ID4NCj4gPiBPSywgaW4gY3VycmVudCBYZW4gY29kZSwgRUZJIGlzIHVu
Y29uZGl0aW9uYWxseSBjb21waWxlZC4gQmVmb3JlDQo+ID4gd2UgY2hhbmdlIHJlbGF0ZWQgY29k
ZSwgSSBwcmVmZXIgdG8gcmVtb3ZlIHRoZSBoZWxwIHRleHQuDQo+IA0KPiBCdXQgdGhhdCdzIG5v
dCB0cnVlOiBBdCBsZWFzdCBvbiB4ODYgRUZJIGdldHMgY29tcGlsZWQgZGVwZW5kaW5nIG9uDQo+
IHRvb2wgY2hhaW4gY2FwYWJpbGl0aWVzLiBVbHRpbWF0ZWx5IHdlIG1heSBpbmRlZWQgd2FudCBh
IHVzZXINCj4gc2VsZWN0YWJsZSBvcHRpb24gaGVyZSwgYnV0IHVudGlsIHRoZW4gSSdtIGFmcmFp
ZCBoYXZpbmcgdGhpcyBvcHRpb24NCj4gYXQgYWxsIG1heSBiZSBtaXNsZWFkaW5nIG9uIHg4Ni4N
Cj4gDQoNCkkgY2hlY2sgdGhlIGJ1aWxkIHNjcmlwdHMsIHllcywgeW91J3JlIHJpZ2h0LiBGb3Ig
eDg2LCBFRkkgaXMgbm90IGENCnNlbGVjdGFibGUgb3B0aW9uIGluIEtjb25maWcuIEkgYWdyZWUg
d2l0aCB5b3UsIHdlIGNhbid0IHVzZSBLY29uZmlnDQpzeXN0ZW0gdG8gZGVjaWRlIHRvIGVuYWJs
ZSBFRkkgYnVpbGQgZm9yIHg4NiBvciBub3QuDQoNClNvIGhvdyBhYm91dCB3ZSBqdXN0IHVzZSB0
aGlzIEVGSSBvcHRpb24gZm9yIEFybSBvbmx5PyBCZWNhdXNlIG9uIEFybSwNCndlIGRvIG5vdCBo
YXZlIHN1Y2ggdG9vbGNoYWluIGRlcGVuZGVuY3kuDQoNCj4gSmFuDQoNCg==


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:34:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:34:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195245.347880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiWm-00024m-U7; Fri, 24 Sep 2021 10:34:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195245.347880; Fri, 24 Sep 2021 10:34:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiWm-00024f-Q3; Fri, 24 Sep 2021 10:34:00 +0000
Received: by outflank-mailman (input) for mailman id 195245;
 Fri, 24 Sep 2021 10:33:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiWl-00023q-06
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:33:59 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.50]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ec739dfd-1d22-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:33:58 +0000 (UTC)
Received: from AM5PR0701CA0063.eurprd07.prod.outlook.com (2603:10a6:203:2::25)
 by AS8PR08MB7047.eurprd08.prod.outlook.com (2603:10a6:20b:347::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:33:50 +0000
Received: from VE1EUR03FT049.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:2:cafe::28) by AM5PR0701CA0063.outlook.office365.com
 (2603:10a6:203:2::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Fri, 24 Sep 2021 10:33:50 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT049.mail.protection.outlook.com (10.152.19.216) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:33:49 +0000
Received: ("Tessian outbound b324a1dbd9be:v103");
 Fri, 24 Sep 2021 10:33:45 +0000
Received: from 1c83914c6937.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BD9AD9D4-FCC2-4B9A-A60A-1ABCD5D31AC6.1; 
 Fri, 24 Sep 2021 10:33:34 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1c83914c6937.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:33:34 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6959.eurprd08.prod.outlook.com (2603:10a6:10:2bf::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:33:32 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:33:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec739dfd-1d22-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NrlQQVZM2whjIHVadx7f0f/C77QMIN1PZzvyKX895Sg=;
 b=+GprZigTHLRYYoFRBqETpbyO36KImJXu5mUTNM5EFxkoqQ+QX+09eRKucK4UlG0RMBDyq8U41moTArTKdR6snG4fs7PfYDGozlsEeV6yJnU7W+albjicfXePnh/tjGg1SzSE6jXK93N0PZ+w6FlAJFM7Y+ypjVi4yU+6dgqXj6Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=B805XAQ03J4Pp3AOwMHMaOZIMPxjs44QiOkCy3AqOvx+40z4ehpHrr09hq7ICNvMmXQG+0ELSUHSUK0PVczuCxJC6XzXLQmRWA8atq76JTezwXXtKnvGvFJDbw95Wh9tJyZgXeRj5UK2mdCQDs8LWXXnra8GjVKGGE7iVVVP6CYGzOoaLrHteJfx+GoQTjLo7QHmAAGK3mW+96d4VwPpKcpaWlLrqDhGxu3fMTccnmdekj7lBO8KdCOsfGTWgAe3Co3GXWT/kFdMPqafebqw8gwcHPmFaebGW1zpi9AQzhwohCk0AjRg4ZWdJstdmd67ZvY5LKff3BqVhEUj/cR3WQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NrlQQVZM2whjIHVadx7f0f/C77QMIN1PZzvyKX895Sg=;
 b=J1UiPMHekcWbdEXABqBq2ygvS/B8OmmJwQ0M5BJa5dCMkhJ6/Ejpqo+hwoJJfaHc2gLf/gr6Y5IeCczQB2ZM0uDFhAILat0JPnAdWw2+8jTbLBigp2pMNsbUkkVGGcaG6AF+CZYaIgGEqMvaKCvfYgSyGsXhyZpj8mMLwsbzOsJIYi9V4w+wRvmhZ7E0Vch/9pfjX7ddHLRZTjgS+WNw0TthVWnbrXUDVolQQHCHFgT6wIK43D+vhqf0bFwy0RqqKsjKAyJ/2rSu+HiQ+6BhaW8lql1+tZ1PNSKNRV/fGBZojVFcY3S0q6qxODnVEn5rVZog+tT/9U0fPxGcKj9l5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NrlQQVZM2whjIHVadx7f0f/C77QMIN1PZzvyKX895Sg=;
 b=+GprZigTHLRYYoFRBqETpbyO36KImJXu5mUTNM5EFxkoqQ+QX+09eRKucK4UlG0RMBDyq8U41moTArTKdR6snG4fs7PfYDGozlsEeV6yJnU7W+albjicfXePnh/tjGg1SzSE6jXK93N0PZ+w6FlAJFM7Y+ypjVi4yU+6dgqXj6Q=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>, "julien@xen.org"
	<julien@xen.org>
Subject: RE: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
Thread-Topic: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
Thread-Index: AQHXsHMKuuhjNuZvvk2uWABlJOqp6quy4w2AgAAazgA=
Date: Fri, 24 Sep 2021 10:33:31 +0000
Message-ID:
 <DB9PR08MB6857E123564C930D7BC1E2BF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-3-wei.chen@arm.com>
 <9bec4214-5f50-9ea5-0212-52d248b63ca6@suse.com>
In-Reply-To: <9bec4214-5f50-9ea5-0212-52d248b63ca6@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 439BC191470FD1479D88350AB35075D3.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 8970f3e1-eb0c-4fde-56b2-08d97f46cc2f
x-ms-traffictypediagnostic: DB9PR08MB6959:|AS8PR08MB7047:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB7047CE069F4715DB1143EA929EA49@AS8PR08MB7047.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Q7BexfQiy2eSahceS3/lePYT7wHscCkdraL5arT1UvcevuSZ8/lFw0cq6Y2lwOT/3IatZf0v9G8UbM+lFjrAR9JJJVgQGqsenLqYYyTJR79mltiQBwFlvnTOAV4xRSrzxVvtIICdSFepzDhtPtKhZeQroEzdEE9ZXx3fwAgdUXl0M/nmP0Vuap6h58/uXf52QW0LqEfynDdnYSrGjU0av9ewefTIVt41hGll5gNP+IsMpAO0XZFXKVmaQe8KTJI9XFPpRfdpzkcc1eEnkJ/aPTnWGyrwZdvSf5nlZ/BAuOyZpIgWOsUhUrdQhoDQVE4cPX/z9E24J7lk6EijHp2J1f5L8d5+ZdKUcLPjrUtYbpD/TZC1f9e5LSeB6kxJQ/ZyxpA4JRJ8+J/WEv0+R1gZP7mMRlGPDf9xLnObgiFVav1Ohm7pq2YftlL1EtVRB2prW4lISsmNP/jaDoJhKSYaRhqkNKVpcWzgh7PqhkODTQ3+QvmMThPCi+1lHffIOQmbTYKXgT14h7z7UVnkEkUBh5lzqYP4EYsB8MRKpgD4dCYpzyyMtBKSZ9npypA3LGluiNMRFlcudi93tj3LxsZp7+VmQPVkpRDtfjhDWqAvKDMW6+fR5JnCcOam5xQmMixeIgybZ0kQhB/T1YVl7ehcYnar6qzJ2c4m0kZ2QP+JH0iTrBO3hUBzA53ye0jvEBs475HftZcCQ6rNYjSv6rJS6A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(4326008)(54906003)(66446008)(71200400001)(66476007)(66556008)(8676002)(186003)(5660300002)(66946007)(9686003)(316002)(33656002)(8936002)(76116006)(55016002)(508600001)(86362001)(6916009)(38070700005)(26005)(122000001)(53546011)(52536014)(83380400001)(6506007)(7696005)(38100700002)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6959
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	51ea5b40-3288-4e7b-ae97-08d97f46c1a8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SR6krs5aFaXipFny0Ma33KOc27TGS/GXCCi4y8bFIHHBlDyhi/YGbgLT1aOSfTzPTGpixXAvmikEw5n5R/o43mVyY4lbYDIeqZiKejCSN13Ip8vYgzysbSRGbt8CtEayCav8+mJQvSGTB7d9MBA8/bW5qBs6119rF5uOfhH0VtKNCRt/GmdY1kx+ihjx2f7zKAKTh9DfgydrO/9UQU4MMZoqm6FS/9kfRNbYM4r4iVbO0IVqYniIwUS6DEq9qfYSHONgQBkVKLU8CdlRPwwHeU2/W222ubehaNVMXE1e8LBMna5jfDx896NPQzafPSw15YQxbnDsX3CVhMsF70okRY1TxkAixSyZ2jYNHlg9Bc++X250Vc+7sxg+EbPTThRwhGotFSSv9qpEXKONFhREqkgBBmm6haDknHWFQ2+gp1Hc/fwdPFlu3l1m2/phFPzS3k0bi/XDi2UrVHqzJPgJ5o9XkwvuUA6QL7EFJBIBd0ze7cby+Zdc/mW8TLTj4cXlOfp1uK5DKLJxrIdfLblIltNGlEtKWi0DHMOKWYzgRG4gaSrC+hluOgExUjEwlW4f/V+BWU+AIRcQ9+Tx9IARgCvHBWtSVu8sQ3ZWHCAK5nGje+JaLnOaGEeHO1v6c8Mp4lLVlvhjnCokvouG9RdPnYrLexV4IsF+dIGrby5B0aZ2lriyaAqWwEMOxZwyZXQZ+LEh5P/Jr6ebZTL7+w/QRw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(55016002)(53546011)(82310400003)(70586007)(83380400001)(86362001)(2906002)(186003)(8676002)(33656002)(8936002)(36860700001)(356005)(81166007)(70206006)(6506007)(47076005)(7696005)(54906003)(5660300002)(26005)(9686003)(52536014)(6862004)(336012)(4326008)(508600001)(316002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:33:49.8679
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8970f3e1-eb0c-4fde-56b2-08d97f46cc2f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT049.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7047

SGkgSmFuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEphbiBCZXVs
aWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSAxNjo1Ng0K
PiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBCZXJ0cmFuZCBNYXJxdWlz
IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyB4ZW4tDQo+IGRldmVsQGxpc3RzLnhlbnByb2pl
Y3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxpZW5AeGVuLm9yZw0KPiBTdWJqZWN0
OiBSZTogW1BBVENIIDAyLzM3XSB4ZW46IGludHJvZHVjZSBhIEtjb25maWcgb3B0aW9uIHRvIGNv
bmZpZ3VyZQ0KPiBOVU1BIG5vZGVzIG51bWJlcg0KPiANCj4gT24gMjMuMDkuMjAyMSAxNDowMiwg
V2VpIENoZW4gd3JvdGU6DQo+ID4gQ3VycmVudCBOVU1BIG5vZGVzIG51bWJlciBpcyBhIGhhcmRj
b2RlIGNvbmZpZ3VyYXRpb24uIFRoaXMNCj4gPiBjb25maWd1cmF0aW9uIGlzIGRpZmZpY3VsdCBm
b3IgYW4gYWRtaW5pc3RyYXRvciB0byBjaGFuZ2UNCj4gPiB1bmxlc3MgY2hhbmdpbmcgdGhlIGNv
ZGUuDQo+ID4NCj4gPiBTbyBpbiB0aGlzIHBhdGNoLCB3ZSBpbnRyb2R1Y2UgdGhpcyBuZXcgS2Nv
bmZpZyBvcHRpb24gZm9yDQo+ID4gYWRtaW5pc3RyYXRvcnMgdG8gY2hhbmdlIE5VTUEgbm9kZXMg
bnVtYmVyIGNvbnZlbmllbnRseS4NCj4gPiBBbHNvIGNvbnNpZGVyaW5nIHRoYXQgbm90IGFsbCBh
cmNoaXRlY3R1cmVzIHN1cHBvcnQgTlVNQSwNCj4gPiB0aGlzIEtjb25maWcgb3B0aW9uIG9ubHkg
Y2FuIGJlIHZpc2libGUgb24gTlVNQSBlbmFibGVkDQo+ID4gYXJjaGl0ZWN0dXJlcy4gTm9uLU5V
TUEgc3VwcG9ydGVkIGFyY2hpdGVjdHVyZXMgY2FuIHN0aWxsDQo+ID4gdXNlIDEgYXMgTUFYX05V
TU5PREVTLg0KPiANCj4gRG8geW91IHJlYWxseSBtZWFuIGFkbWluaXN0cmF0b3JzIGhlcmU/IFRv
IG1lIGNvbW1hbmQgbGluZSBvcHRpb25zDQo+IGFyZSBmb3IgYWRtaW5pc3RyYXRvcnMsIGJ1dCBi
dWlsZCBkZWNpc2lvbnMgYXJlIHVzdWFsbHkgdGFrZW4gYnkNCj4gYnVpbGQgbWFuYWdlcnMgb2Yg
ZGlzdHJvcy4NCj4gDQo+ID4gLS0tIGEveGVuL2FyY2gvS2NvbmZpZw0KPiA+ICsrKyBiL3hlbi9h
cmNoL0tjb25maWcNCj4gPiBAQCAtMTcsMyArMTcsMTQgQEAgY29uZmlnIE5SX0NQVVMNCj4gPiAg
CSAgRm9yIENQVSBjb3JlcyB3aGljaCBzdXBwb3J0IFNpbXVsdGFuZW91cyBNdWx0aS1UaHJlYWRp
bmcgb3INCj4gc2ltaWxhcg0KPiA+ICAJICB0ZWNobm9sb2dpZXMsIHRoaXMgdGhlIG51bWJlciBv
ZiBsb2dpY2FsIHRocmVhZHMgd2hpY2ggWGVuIHdpbGwNCj4gPiAgCSAgc3VwcG9ydC4NCj4gPiAr
DQo+ID4gK2NvbmZpZyBOUl9OVU1BX05PREVTDQo+ID4gKwlpbnQgIk1heGltdW0gbnVtYmVyIG9m
IE5VTUEgbm9kZXMgc3VwcG9ydGVkIg0KPiA+ICsJcmFuZ2UgMSA0MDk1DQo+IA0KPiBIb3cgd2Fz
IHRoaXMgdXBwZXIgYm91bmQgZXN0YWJsaXNoZWQ/IFNlZWluZyA0MDk1IGlzIHRoZSBsaW1pdCBv
ZiB0aGUNCj4gbnVtYmVyIG9mIENQVXMsIGRvIHdlIHJlYWxseSBleHBlY3QgYSBDUFUgcGVyIG5v
ZGUgb24gc3VjaCBodWdlDQo+IHN5c3RlbXM/IEFuZCBkaWQgeW91IGNoZWNrIHRoYXQgd2hpY2hl
dmVyIGludm9sdmVkIGRhdGEgdHlwZXMgYW5kDQo+IHN0cnVjdHVyZXMgYXJlIGFjdHVhbGx5IHN1
aXRhYmxlPyBJJ20gdGhpbmtpbmcgZS5nLiBvZiB0aGluZ3MgbGlrZSAuLi4NCj4gDQo+ID4gLS0t
IGEveGVuL2luY2x1ZGUvYXNtLXg4Ni9udW1hLmgNCj4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20t
eDg2L251bWEuaA0KPiA+IEBAIC0zLDggKzMsNiBAQA0KPiA+DQo+ID4gICNpbmNsdWRlIDx4ZW4v
Y3B1bWFzay5oPg0KPiA+DQo+ID4gLSNkZWZpbmUgTk9ERVNfU0hJRlQgNg0KPiA+IC0NCj4gPiAg
dHlwZWRlZiB1OCBub2RlaWRfdDsNCj4gDQo+IC4uLiB0aGlzLg0KPiANCg0KeW91J3JlIHJpZ2h0
LCB3ZSB1c2UgdTggYXMgbm9kZWlkX3QuIDQwOTUgZm9yIG5vZGUgbnVtYmVyIGluIHRoaXMgb3B0
aW9uDQppcyBub3QgcmVhc29uYWJsZS4gTWF5YmUgYSAyNTUgdXBwZXIgYm91bmQgaXMgZ29vZD8N
Cg0KPiBKYW4NCg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:34:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:34:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195251.347891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiXT-0002dI-61; Fri, 24 Sep 2021 10:34:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195251.347891; Fri, 24 Sep 2021 10:34:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTiXT-0002dB-30; Fri, 24 Sep 2021 10:34:43 +0000
Received: by outflank-mailman (input) for mailman id 195251;
 Fri, 24 Sep 2021 10:34:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTiXR-0002cw-Au
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:34:41 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.54]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 03a3808c-1d23-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:34:37 +0000 (UTC)
Received: from FR3P281CA0027.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::14)
 by DB7PR08MB3884.eurprd08.prod.outlook.com (2603:10a6:10:78::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 10:34:30 +0000
Received: from VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:d10:1c:cafe::9f) by FR3P281CA0027.outlook.office365.com
 (2603:10a6:d10:1c::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.8 via Frontend
 Transport; Fri, 24 Sep 2021 10:34:30 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT037.mail.protection.outlook.com (10.152.19.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:34:30 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Fri, 24 Sep 2021 10:34:29 +0000
Received: from f86800bb0033.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E00E082B-5228-449D-BAE0-0BF837903CA5.1; 
 Fri, 24 Sep 2021 10:34:16 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f86800bb0033.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:34:16 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6959.eurprd08.prod.outlook.com (2603:10a6:10:2bf::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:34:15 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:34:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03a3808c-1d23-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fkc19n3/f09++KXFEYT/F+y3BGSBmy0DKrhefV9sAgE=;
 b=gv35uVTsTArrPo3XZujH4UFkHbO9DVOXS5PnLRMOUcmEM6QsxSzRb28IhNDtgGFZQv2mGJz2koeg/w/B9fOGNFwnbMXJ2c1XoIGt+hvveQYJACcmoYDKxSprqe5Zt6m6kaYQ2+QDRY+u8ezZ2msGKRAfnLRY6MvO0LM8Efq6X+M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GnqMGs2MOW4FFM04zVLJUJzqrzfBv4hn3fU2/ZeSnLaW/FiJAqRZLBJoTjdHcqp/sJdudkfjo8POFdWl9ZArk6CEjIbnfU7LEzSn2mAYOukI45AZPhaDFqqUrRxkqrIVX9qDxQo8gI8oTIuh8ipV7sYftnDTkqxli+hzPaBbx9AXQBOODAtQRSttf+Tvkg1bpeyOSxX2OfcZM48lkLMBIapMD8sUgvrTasSN6exlTBbHrykB8Bg935YySWpdFSwFaBvtMp5jkbA8noG9JNYcqLruChyvGxkrwrMsqx8TtaYAlMDq2q21kTPuTRVc8hb42eloSqr9HEr37uXMhVN12g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fkc19n3/f09++KXFEYT/F+y3BGSBmy0DKrhefV9sAgE=;
 b=W1jsXSV9ETXwzRvYU2GIw8Tt0ciZRPvOWXDMAdFn8YsDJVzW8m//mw1IGYXNllducxoE163xgqhSy+72RFWCMkkLePSBhAZMlWAiYemz9hN7FUs48nFqPk3cEsb0Y7AeDnRMKtA/Q7tfnm33H/bjkX2GsIJ8TA9IOjVZBLuID/wT18qGo7zQiN6G+s+7QDXUpu68CT3aPURmMzDsh11gZyOg7uUuJxV2H9SJh4yaigLXSwnhF054IDdk9lnbNzxZGUZdMWYuDNlBB87Si3ZMyrl1Z+ZEvI1WcS9D1iraaIvX+mJi0S1QWt8RjrvqooIE42dr6kbxy/ciKCpuRmQX3g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fkc19n3/f09++KXFEYT/F+y3BGSBmy0DKrhefV9sAgE=;
 b=gv35uVTsTArrPo3XZujH4UFkHbO9DVOXS5PnLRMOUcmEM6QsxSzRb28IhNDtgGFZQv2mGJz2koeg/w/B9fOGNFwnbMXJ2c1XoIGt+hvveQYJACcmoYDKxSprqe5Zt6m6kaYQ2+QDRY+u8ezZ2msGKRAfnLRY6MvO0LM8Efq6X+M=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>, "julien@xen.org"
	<julien@xen.org>
Subject: RE: [PATCH 03/37] xen/x86: Initialize memnodemapsize while faking
 NUMA node
Thread-Topic: [PATCH 03/37] xen/x86: Initialize memnodemapsize while faking
 NUMA node
Thread-Index: AQHXsHMIJLEC2+fmVEC3A6RnJgdqV6uy43iAgAAa/qA=
Date: Fri, 24 Sep 2021 10:34:15 +0000
Message-ID:
 <DB9PR08MB685755F25447F054B5EE199F9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-4-wei.chen@arm.com>
 <27150d2e-b269-c1ef-0e73-bdce1975dcb4@suse.com>
In-Reply-To: <27150d2e-b269-c1ef-0e73-bdce1975dcb4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: A3DD0F2CA4E6434AA2313A233FFD2C33.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3ade7974-4478-468d-6b6d-08d97f46e489
x-ms-traffictypediagnostic: DB9PR08MB6959:|DB7PR08MB3884:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB38847B1A4862D207B5C8FCD99EA49@DB7PR08MB3884.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 c4hTNuzWmQi/vZDAjDw0jHOdxPEETtBfPkH44laDhk4LWNXkFU6/GIAqN9u4+IAFZ9M4hQPM57UP3I7S0OEs16ph2N506+CKeUr4gHU99O7rq9oyS4OplISAR7izVW4WbYigPzErRKtR3edZbZ8o5XJaBrc+TPeSaUhh+63mIAhqtpCv9SBUZl3wOChGHgEd3lAYDmXM8EnPAQEK9XuT3Tq4kL9qHMUf5+sAMIFPi5Fp4oyNw7jJ6pbuAYFNx3ZRHTtZJrN0DtX+zCWpdnc3hswk+Jv8WWTU9kjI4MwQtg1fYEI9PUruyWtBsJARsm2sMJCzFc0h+aLN4aHTBr1i+4DYtJgA3ZjBNm6MbiVAWDV2FNza+s2Thw174nEkXI7Q3sYPGkSAVfVbWEdQc+99ROcJeWMGI4Ds1wO/kGNE1taXUcHir9/E9X8lv0Ns3EEoHqO2o1jVsEllgiSmnqnqgNbsMX8sqwSBfywX4pM9O404bhPTphv40iI6D78hkGVKKUgIAe+U03yGllu0ifWaG1mabiL0CTnkoCw4WxqCB7fIu7GFaV8l8IDzGIcoubME9va88oeELOMsoJpyxMHp8M7mDUzbwlorbL28ET88bE+LJ+zwrxULAKj0hoseaSSUpT+b50+crD1TcxX9a0240dEK7SBvNK7Yj/obuM6gQTKU5U1a9JdeuLZZJsf3FRVqK7V7bKMSdbTaQQYLYSbtnw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(64756008)(4326008)(54906003)(66446008)(71200400001)(66476007)(66556008)(8676002)(186003)(5660300002)(66946007)(9686003)(316002)(33656002)(8936002)(76116006)(55016002)(508600001)(86362001)(6916009)(38070700005)(26005)(122000001)(53546011)(52536014)(83380400001)(6506007)(7696005)(38100700002)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6959
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e567fd6b-5f04-41df-b975-08d97f46db7c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VAaP51Ffv8gJsrEd5cp+F9dPvUhv5s/PSyV9rfrbBHRP/+RLCHTcIqxyCznyFmZ1polJOaEgrYVbcW4aOY11YljTSjNlTEaV+8bWwkpp9sxz/S77rTQufhkZTr+srKRqE/JmHJPLtKr6dsEbQH7+1+XI49SCz3HAlZOsEY5pTwWcFVljieRkcM2CKX50UHOFjC2CjKEPuTROx/eBonhLVYSwnHcvC+X9qE0ZwCAnBSfwKL8rfYlpMLllUepBU8Z7q79mEchFfpOohnTGkbaD5SlAJVopsrlT4cPom3tRx/bpo6qp29TJpRTY9887U1jKYiv2KzRdoXlmlcWaGLj8B91cZFGG3/p+bwFBU+VuC1YfYT1OdnziCnzKT7b8eXOpZQ84zxmVKaoNlC0YPiu4w89uGs1LDnIUs+phVjRQ7v9WJLybjoVm9H5t+g7UQr6OiY/mHxJ1oYzRGa5awV5zTHP8amjpdJLvswA1ZYF2DxUUOhkLlmj2hdUPOkQjsM9tzUPVTl1giSZIq6mYtLqbgpXwyMF7WmKmz9akxcjC6+eEzJ09gD/A48NNph6ZJ10ExrmMkQyP6qrsJLU/q8UD5nwocvxK9ch1LlPktWCqkLL1jC1MYLwGOQ/fidQYfqDp7d84r15SWd6rA+3u6YknIc/4atqC3KJ4nHh9pkQ9jUHKCwesa5t2SIRa9IaA8EgeZ3qXeXvKM9y8oDShthgbcg==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(5660300002)(47076005)(36860700001)(508600001)(53546011)(316002)(6506007)(2906002)(83380400001)(81166007)(356005)(9686003)(55016002)(33656002)(7696005)(70206006)(52536014)(186003)(4326008)(6862004)(8676002)(54906003)(70586007)(26005)(8936002)(336012)(82310400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:34:30.7235
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3ade7974-4478-468d-6b6d-08d97f46e489
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3884

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogSmFuIEJldWxpY2ggPGpi
ZXVsaWNoQHN1c2UuY29tPg0KPiBTZW50OiAyMDIx5bm0OeaciDI05pelIDE2OjU3DQo+IFRvOiBX
ZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRy
YW5kLk1hcnF1aXNAYXJtLmNvbT47IHhlbi0NCj4gZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7
IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGllbkB4ZW4ub3JnDQo+IFN1YmplY3Q6IFJlOiBb
UEFUQ0ggMDMvMzddIHhlbi94ODY6IEluaXRpYWxpemUgbWVtbm9kZW1hcHNpemUgd2hpbGUgZmFr
aW5nDQo+IE5VTUEgbm9kZQ0KPiANCj4gT24gMjMuMDkuMjAyMSAxNDowMiwgV2VpIENoZW4gd3Jv
dGU6DQo+ID4gV2hlbiBzeXN0ZW0gdHVybnMgTlVNQSBvZmYgb3Igc3lzdGVtIGxhY2tzIG9mIE5V
TUEgc3VwcG9ydCwNCj4gPiBYZW4gd2lsbCBmYWtlIGEgTlVNQSBub2RlIHRvIG1ha2Ugc3lzdGVt
IHdvcmtzIGFzIGEgc2luZ2xlDQo+ID4gbm9kZSBOVU1BIHN5c3RlbS4NCj4gPg0KPiA+IEluIHRo
aXMgY2FzZSB0aGUgbWVtb3J5IG5vZGUgbWFwIGRvZXNuJ3QgbmVlZCB0byBiZSBhbGxvY2F0ZWQN
Cj4gPiBmcm9tIGJvb3QgcGFnZXMsIGl0IHdpbGwgdXNlIHRoZSBfbWVtbm9kZW1hcCBkaXJlY3Rs
eS4gQnV0DQo+ID4gbWVtbm9kZW1hcHNpemUgaGFzbid0IGJlZW4gc2V0LiBYZW4gc2hvdWxkIGFz
c2VydCBpbiBwaHlzX3RvX25pZC4NCj4gPiBCZWNhdXNlIHg4NiB3YXMgdXNpbmcgYW4gZW1wdHkg
bWFjcm8gIlZJUlRVQUxfQlVHX09OIiB0byByZXBsYWNlDQo+ID4gU1NFUlQsIHRoaXMgYnVnIHdp
bGwgbm90IGJlIHRyaWdnZXJlZCBvbiB4ODYuDQo+IA0KPiBTb21laG93IGFuZCBBIGdvdCBsb3N0
IGhlcmUsIHdoaWNoIEknbGwgYWRkIGJhY2sgd2hpbGUgY29tbWl0dGluZy4NCj4gDQoNClRoYW5r
cyENCg0KPiA+IEFjdHVhbGx5LCBYZW4gd2lsbCBvbmx5IHVzZSAxIHNsb3Qgb2YgbWVtbm9kZW1h
cCBpbiB0aGlzIGNhc2UuDQo+ID4gU28gd2Ugc2V0IG1lbW5vZGVtYXBbMF0gdG8gMCBhbmQgbWVt
bm9kZW1hcHNpemUgdG8gMSBpbiB0aGlzDQo+ID4gcGF0Y2ggdG8gZml4IGl0Lg0KPiA+DQo+ID4g
U2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+IA0KPiBBY2tlZC1i
eTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KDQo=


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:37:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:37:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195277.347901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTia1-00040B-MW; Fri, 24 Sep 2021 10:37:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195277.347901; Fri, 24 Sep 2021 10:37:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTia1-000404-JX; Fri, 24 Sep 2021 10:37:21 +0000
Received: by outflank-mailman (input) for mailman id 195277;
 Fri, 24 Sep 2021 10:37:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OJ83=OO=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mTia0-0003zy-8R
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:37:20 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.78]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 64441348-1d23-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:37:19 +0000 (UTC)
Received: from DU2PR04CA0337.eurprd04.prod.outlook.com (2603:10a6:10:2b4::35)
 by VE1PR08MB5598.eurprd08.prod.outlook.com (2603:10a6:800:1a2::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 10:37:17 +0000
Received: from DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b4:cafe::5) by DU2PR04CA0337.outlook.office365.com
 (2603:10a6:10:2b4::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Fri, 24 Sep 2021 10:37:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT058.mail.protection.outlook.com (10.152.20.255) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:37:17 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Fri, 24 Sep 2021 10:37:17 +0000
Received: from 9cc27ec6097c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F98A5DFB-0504-4CC3-A116-712A8CE4C00D.1; 
 Fri, 24 Sep 2021 10:37:05 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9cc27ec6097c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:37:05 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5559.eurprd08.prod.outlook.com (2603:10a6:10:1a2::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 10:37:05 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:37:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64441348-1d23-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=B0elbpzxYorW87yCM8WsQtWw5QXTwy1oDV/rL89rzRo=;
 b=OA7QbL01mk/vXpiZeyFFrH6lecgBnsFuLLCG6KbUxcF9+3G6Pgelb/s9Ea53omlSqRuiepwd3zbYrRrIHa2iTAW4iyc163l7Ebuno8m0fvme6pYs+zdZENyhpjir0pNAFQqMysq4alP2MbqycsTSzuezTKAfXoonapoo7ISSnJg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E4Tc9A0JHxgKbbwUY5x36opJBfk65B0dNg6vTKMNcNgUYwOKK2G97wknBXw7b8Y9izsjJrb4quXZEuyCj5Utvq5v2rLKZx2HGoSapMWhXo82In7wAPx8/QoN2m51NCpBcZlKPxLPQ7f0y1dLEMriXL303ebwVMEc10wtIW5oDMd/lzlfj5bNkCTzwM/b8P/HNn70uOgvF5I9mJTlH5UW9JZ/4wbu6NXvajcTSc+9j/qbWOKcrDsKv6zaGwLdZvLmzJlD6OKMF4VUEf/bXXroMao4QvxWSvBm+hjfKIZ3nFCGj+y2bPiNdMD7uT4dOjznnB+6nCT1KBpnjwPOYPSoyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=B0elbpzxYorW87yCM8WsQtWw5QXTwy1oDV/rL89rzRo=;
 b=CC+Kr1IZvrfCt4f01sd1yrALPz6IUtRtHKxTD80Pe+zwjCIPGxiFt6PnoXedDEedLw8ukPVAxgNNxyaMkZvowcBoaRlode2/ZhuxbdGPfTy0t7bh4dmNnmMlcAMcZgKBYtWhp8kXJAoQSSFAu9HdA7NCxMy3GLxG7F8GnSuM7pkq/Yvicavu0rqRKBvlxbAYBlP9/D8Ck6u31H4rHybtSAgklFnbTFlUv3kLjAcVR9/1IGMww8YxFqEBT9IGICRRq13jDJysMDIcsVWwLhGhKgBZDns94vfP8ThCiCErwaoGz1q8w7Ss0qZrvWJ93ONP42jK3erX6U4JqderYij9XA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=B0elbpzxYorW87yCM8WsQtWw5QXTwy1oDV/rL89rzRo=;
 b=OA7QbL01mk/vXpiZeyFFrH6lecgBnsFuLLCG6KbUxcF9+3G6Pgelb/s9Ea53omlSqRuiepwd3zbYrRrIHa2iTAW4iyc163l7Ebuno8m0fvme6pYs+zdZENyhpjir0pNAFQqMysq4alP2MbqycsTSzuezTKAfXoonapoo7ISSnJg=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>, "julien@xen.org"
	<julien@xen.org>
Subject: RE: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
Thread-Topic: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
Thread-Index: AQHXsHM4CZyvsR6BnEO/IHEJe48Inquy/CcAgAADDZA=
Date: Fri, 24 Sep 2021 10:37:04 +0000
Message-ID:
 <DB9PR08MB6857B7C19C333E03397F762A9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
 <a3d3db98-420a-79a2-97a7-d332aee5b531@suse.com>
In-Reply-To: <a3d3db98-420a-79a2-97a7-d332aee5b531@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6B42B8AECBE203488B7B327700F11857.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 028caed2-35ac-4560-b41a-08d97f4747da
x-ms-traffictypediagnostic: DBAPR08MB5559:|VE1PR08MB5598:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB55982DBA16EA00E173E3DFD69EA49@VE1PR08MB5598.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 VCTTitALY0+iS9YbEHbLoQvf8XnhkeA2fX+Ru5vguRd89I3dQkjp9xsWaqaofaWvd6Q1Pj+yB10dQbL+KBD8os27frqLhzR+ueuprViPVazdte9HlqibsaAGR0hYZ/SlanBBmgDlcjXWBIXFrAIsU9RgM3ejAFpT7Ek4VM3HJM+lqBZU7tkbP+g0KMFsB7owcHRJnC8sI6JFPh3v1J1aL3wVmdz5Kp3vFncI2jibYA3q98AQ82fkTiQplWgy+AWlR7tzafEAHHZpDLS7h9YhJkfF96TnWWtVf7Ko6QOFIZxZqExEYcS+1aIC9s+9jqdm/wqbDTd7QQBvns14MhY3ZlAkvtwrTnULi2+p8ePAgOgq47mTVIm6bOxa3Nnf21YcyreKZh2LD3tbBznfqcSpVsckI7vhl4Xar6AjQTuojWVwqfcMKT9nknbKYuGwjbAEFXokMKXOr9Bkx6iUrtc1KelvF3KNcHfke2T3eFIOsvZlG4BUc1eVKW/59GrsjAUdW2SjJy+xHVM6qbOTDUjUmygLgS1pBN9Fb0/MKgI2eZ8EKgyVtLvDRpCbwLTL4F+Lvfbzdaym4BvaPZylj860KI9RJM6iia226WpoWLWgMVAKuZLg0qoz7oaysoOm3yAOFXohMPc3DT3/3FM5yMzTGR9jFDwj1ZlcW4yI0TZubSgUjTI7ru+/tt6Gic4AnLSWn4J7xvtFP5CbuUJD5mucTQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(4326008)(38070700005)(508600001)(66476007)(83380400001)(26005)(8676002)(186003)(2906002)(38100700002)(71200400001)(6916009)(6506007)(54906003)(7696005)(8936002)(55016002)(86362001)(316002)(33656002)(52536014)(66446008)(53546011)(5660300002)(9686003)(66556008)(66946007)(76116006)(64756008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5559
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f5d78897-f733-490a-9437-08d97f47407a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	doqRvpmUzhxM4Qq3Q1Sz8IT0/iFv15/1hYN16dTBImseUGoAvT6rEgCPHNkaADRU+8JPbq7ysY9BaDhH35zKDV8tKMj9MNa6+pfBM9HdPwJku/w35Gp/KQXn+6I3omm61kCmfzix2G+4pkswsj5qnT5hnYFtg/R58yyQPkoIDt9yfjOuKVE2bSx4VcPlhTBCPsxBOKD5RRmDsW1KDVZnsIeYM5cBTu0IwwJRLKPLTDsEN9V5Ii8ic13Y1E9gdw8D81ae8HOv3xjtoiyjhIUJ5tcg2XS3N4V+VHXvNouEO/31ZbQXLje1UcvGOCLsB/M22l1l3CQXN83L4I4/lJWTqpIrkhhWhxWjF3IZQyiK39ZG93xXaSk/lUcmYpsg8NG2GIPayM4QPtjyH3PE3OIcpqpr+ZGWGMGNC8KHm8cO+CC5HkAUdBigWAnYCN4kwsI7vkCIb77JrwqnNrhXaYTDXlEFeoS0uqwUcj70odI+Ly22rpcy2AdDhnELRb9rVHfFDECcSjFiTgJXmR1lC++5eH4be7ThlWmb5gVzn5LgHcmE2PLx11yVDw72ehBir5cXbxgkGmHe4yneqnfjzY10VYirU2Bo9a09tZivSVdVHtduG2GJoeqSy1gHxvauYbAMIsuWbmrrK4+Ooa6RDtsoskOKZ21VtrmaKi52Nm9HmiI6o5hnvpNtWD+Uz1JjNZ+Sj2r997cOPBL4A2tyr0D0JQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(53546011)(81166007)(6506007)(47076005)(7696005)(70586007)(33656002)(508600001)(4326008)(6862004)(52536014)(9686003)(8676002)(70206006)(336012)(5660300002)(26005)(82310400003)(186003)(356005)(316002)(54906003)(83380400001)(2906002)(86362001)(36860700001)(8936002)(55016002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:37:17.4486
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 028caed2-35ac-4560-b41a-08d97f4747da
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5598

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEphbiBCZXVsaWNoIDxqYmV1
bGljaEBzdXNlLmNvbT4NCj4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSAxODoyNg0KPiBUbzogV2Vp
IENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFu
ZC5NYXJxdWlzQGFybS5jb20+OyB4ZW4tDQo+IGRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBz
c3RhYmVsbGluaUBrZXJuZWwub3JnOyBqdWxpZW5AeGVuLm9yZw0KPiBTdWJqZWN0OiBSZTogW1BB
VENIIDM2LzM3XSB4ZW4vYXJtOiBQcm92aWRlIEtjb25maWcgb3B0aW9ucyBmb3IgQXJtIHRvDQo+
IGVuYWJsZSBOVU1BDQo+IA0KPiBPbiAyMy4wOS4yMDIxIDE0OjAyLCBXZWkgQ2hlbiB3cm90ZToN
Cj4gPiAtLS0gYS94ZW4vYXJjaC9hcm0vS2NvbmZpZw0KPiA+ICsrKyBiL3hlbi9hcmNoL2FybS9L
Y29uZmlnDQo+ID4gQEAgLTM0LDYgKzM0LDE3IEBAIGNvbmZpZyBBQ1BJDQo+ID4gIAkgIEFkdmFu
Y2VkIENvbmZpZ3VyYXRpb24gYW5kIFBvd2VyIEludGVyZmFjZSAoQUNQSSkgc3VwcG9ydCBmb3Ig
WGVuDQo+IGlzDQo+ID4gIAkgIGFuIGFsdGVybmF0aXZlIHRvIGRldmljZSB0cmVlIG9uIEFSTTY0
Lg0KPiA+DQo+ID4gKyBjb25maWcgREVWSUNFX1RSRUVfTlVNQQ0KPiA+ICsJZGVmX2Jvb2wgbg0K
PiA+ICsJc2VsZWN0IE5VTUENCj4gDQo+IFR3byBuaXRzIGhlcmU6IFRoZXJlJ3MgYSBzdHJheSBi
bGFuayBvbiB0aGUgZmlyc3QgbGluZSwgYW5kIHlvdQ0KPiBhcHBlYXIgdG8gbWVhbiBqdXN0ICJi
b29sIiwgbm90ICJkZWZfYm9vbCBuIiAodGhlcmUncyBubyBwb2ludA0KPiBpbiBoYXZpbmcgZGVm
YXVsdHMgZm9yIHNlbGVjdC1vbmx5IG9wdGlvbnMpLg0KPiANCg0KT2sNCg0KPiA+ICtjb25maWcg
QVJNX05VTUENCj4gPiArCWJvb2wgIkFybSBOVU1BIChOb24tVW5pZm9ybSBNZW1vcnkgQWNjZXNz
KSBTdXBwb3J0IChVTlNVUFBPUlRFRCkiIGlmDQo+IFVOU1VQUE9SVEVEDQo+ID4gKwlzZWxlY3Qg
REVWSUNFX1RSRUVfTlVNQSBpZiBIQVNfREVWSUNFX1RSRUUNCj4gPiArCS0tLWhlbHAtLS0NCj4g
DQo+IEFuZCBhbm90aGVyIG5pdCBoZXJlOiBXZSB0cnkgdG8gbW92ZSBhd2F5IGZyb20gIi0tLWhl
bHAtLS0iLCB3aGljaA0KPiBpcyBubyBsb25nZXIgc3VwcG9ydGVkIGJ5IExpbnV4J2VzIG5ld2Vy
IGtjb25maWcuIFBsZWFzZSB1c2UganVzdA0KPiAiaGVscCIgaW4gbmV3IGNvZGUuDQo+IA0KDQpU
aGFua3MsIEkgd2lsbCBkbyBpdC4NCg0KPiBKYW4NCg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:48:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:48:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195284.347913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTikO-0005nL-MD; Fri, 24 Sep 2021 10:48:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195284.347913; Fri, 24 Sep 2021 10:48:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTikO-0005nE-J0; Fri, 24 Sep 2021 10:48:04 +0000
Received: by outflank-mailman (input) for mailman id 195284;
 Fri, 24 Sep 2021 10:48:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTikN-0005n8-SH
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:48:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e3a8691c-1d24-11ec-babb-12813bfff9fa;
 Fri, 24 Sep 2021 10:48:02 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-QN_VGcWWN5-PJbMsV60Iqg-1; Fri, 24 Sep 2021 12:48:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5742.eurprd04.prod.outlook.com (2603:10a6:803:e5::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 10:47:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:47:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR1P264CA0017.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:47:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3a8691c-1d24-11ec-babb-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632480481;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=VYb8pBlbVWtqo+n4g/3NRPt9nGgrYy6WaPLLK4SXB4A=;
	b=cyQKXdQq6GNhBPoIDuUDnTdDh5C8WdQdURBvbAV+zdx2tBhWiB8I374FsJ0Vc5F3lj1r0u
	EU8AuvUvkuSX+yJTQZBDiUbX+oBOpdrSUrz4awOV4RgfZjAtdLHSnMr6V4M2lYQlESZMtA
	N8mAGJnIyPEGPQQ3BO19+GPnPZ/TbD8=
X-MC-Unique: QN_VGcWWN5-PJbMsV60Iqg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DF+EyuRTUtRmdOebdCVfwMl+S2oy4chaSOfp/bZkpNXRkDIkfryxz/EtOdFOU/AoWtaltJ2OQ3YpL6PHcgdrZdkmhNwRIpo/uWYeNEeKBYok1dB5o2e0z75aVIX5l2qWDeZj7ULKk1d58ohQffVTQo96D3Tufcj8RQUfi3kS0rpuPONmrUgp2mC95IqRHLV5vzzLb6LIHJsJ0g9IsFknqUWr4f0cT5NEU4CcaGQfFV1LqdSPFUcbGeO3bFhB+5Rg6f/ni8M/yEiZ+i4XCKcWtAjk8jd41u6xMvEf2Gtl0rS3wmcvkz5LIGY5D+TUjmK5OdfNT0oYMNJ+fZFMCJv5iw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=KnO8Eali880/it8jdrwlJFiG2EqxzerrSYMMNelIrX4=;
 b=HpE7yRmy3WY37gQf3ECywwfcaK8drMhGplijhp/kkpXvI83ipwOBZkXvGHME1OsmhnlTtxUFY6KKPXwaZP8YC5cI1WrNOJr9vVAqtsaqlPEOybd5KwPHhoACMRsHL+xt0C1AH0Q2SKw07/libQL1QXWNJjXptaylGmBcYlbr4vz4J8uBreholF5ear7WaOLP2F0fz76+7OTyWgZz+K2f8ePU51Leb9u/hivRxUTeim/Xrrnk9Vs5aI0BSvwIEzoP6aTggXUayX9dsskchdg6Bhn2zBQ9fC8S4LNoLJ4W//2ZulDc5cIWdovK/37N0xZqrtJI0ci9rcLEP3ugMClbJg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 02/37] xen: introduce a Kconfig option to configure NUMA
 nodes number
To: Wei Chen <Wei.Chen@arm.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "julien@xen.org" <julien@xen.org>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-3-wei.chen@arm.com>
 <9bec4214-5f50-9ea5-0212-52d248b63ca6@suse.com>
 <DB9PR08MB6857E123564C930D7BC1E2BF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <18e9ea20-657e-bc54-5f8e-f07b2d5f09e4@suse.com>
Date: Fri, 24 Sep 2021 12:47:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <DB9PR08MB6857E123564C930D7BC1E2BF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR1P264CA0017.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:102:19e::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: aaa4bd41-bfce-4115-b6cc-08d97f48c5f6
X-MS-TrafficTypeDiagnostic: VI1PR04MB5742:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5742D7C9DC2F4A21E39B5061B3A49@VI1PR04MB5742.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tbj9HzSLuysTuT3yWjynFskc/6bdvq5gjLQUnJ6rpAJbMIO9TMngnvLlauyvfvyQb1LqgjafgUF6jgSvcVDRqUXeUObhbdah7iYRP0rSfngEDd+vKpHNlVN8JHyLPcR0ij1Cm6G5zSqaHz+w9UdeoPVOMY6IUx91pQd3IkXIsfuzvccHbqJQITYUn1976q+fYHjh6MxFxbXqKWW8pQbL/6S7Nj+4KjpKRniAd1uiIoredRhfjU0XxOzM6ka9BkI548eYyJIN54ylxToh9QesEwDJJjD3ab2J5uW2tYFFpGZyTrKz/GQMqEBEZN6IQq3BOS+8LF7tLxgRNwQZTAuciEFae3oOKSOEh2+1EMnBqtAhdLgpML/pbzLxuPzMWxzCB/2qptC+PmMwen1X1SsYp9ptmBq49yxFNHDh/FPgWo43qaJsG2N1VLNiMLzll1e2rTWZ/UsPSvLq0uJmmsNSWdutPxPMuDnzEEXbPqFfnxgD5mPUYo8eiPSKAft7sNs+UaiY35XvNrvoZzDEn768o1UTUtB+mhWrAGKYkgizIU3NZbEuIz/V62d9ZPMPxTI4b1S6hxzgr7VlrOo1lxICx4Ad0x9NsQ6PfeOc8yWyoNTo9lVer7GQHYtaPrYALki3NouHR8HnApC14V+WSQEgqc6slPoTDsQ/y5CQ/26caQyjsbNlFhvgCG5OUbZrChp18zx19HJSDuZBGOrqDMNtyQyT6ubQEb3BKar2sbEzX5c=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66476007)(31686004)(66556008)(6916009)(53546011)(8676002)(38100700002)(956004)(86362001)(26005)(66946007)(83380400001)(54906003)(186003)(316002)(6486002)(2906002)(5660300002)(508600001)(36756003)(4326008)(31696002)(16576012)(8936002)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?F6D0AWGqA6R6SZn+4bB/R1yrnJso8bljH4GInYdgRnIP25QHSRqbAYWOa+Cf?=
 =?us-ascii?Q?YRLlOdLbK5HSMlOxiPY5t+O+00WKu32x153q1Ud/oQm5tZYTZGxchMrI+UVg?=
 =?us-ascii?Q?LuCwjIKcejAKjMRK2wU6hTzG4PZPGJjPR9EiprKlzDeRWvD5dTX0/DrchhAW?=
 =?us-ascii?Q?GF5Oh+JQm92nsm3GIDPC6pxc5zWoyv4zwnAPLUZfTD58PW4wwUNgNERZQCTX?=
 =?us-ascii?Q?Q8jPlxLbb4zBpvdI+L6yniK5soSaTL2VHra0JKtvHs66hGWwx2/qfNAm1nR9?=
 =?us-ascii?Q?J0tUf7+UwmtvLSrmls9N1s0BCxRnCAMFVGmYEliIDAtHnuOpl3/R28RgfNDp?=
 =?us-ascii?Q?5urSU0kplLUCkXOHPi0HAk4adE2K4fUHQBjB1TWACyaUEUfeL2uKQ+ceGJHL?=
 =?us-ascii?Q?7y+W6NQyIp4MWbm5/PSBS3ruui/qi35tLyia8uw55EXodM6Wfufrl4uhOnEj?=
 =?us-ascii?Q?hkXsYk4Sk/eUKBzvxuuJv1JxIy18k2hxO5xn5/XLW3Cm09X6UUp0XzrjJx8l?=
 =?us-ascii?Q?nzJcKUA6O0iv/s0TkOPhNiq6Gxa3EnqvmsApau76HXGBATY+bnFxgpPNvKCs?=
 =?us-ascii?Q?cdj+Ql/B9RRCOBkdFd3b4huFXaQxbZ/6QfqLMcBitkkCelOwUKNmF9PwvWqp?=
 =?us-ascii?Q?7Y1vYwkc6SWG+w5pFsTL9JWOGol+OEY/w6nH0eWAdilIyES7cvi0iCPtahRb?=
 =?us-ascii?Q?4izazXJfFiWHfoIV6Zn8Km+0wwcvFj84XRwMJDTJ1Be9FrB6GG8D1dAKKGQ+?=
 =?us-ascii?Q?cSQoc0gyx/3iQTyWo9PccqJ+36GUOhyzFy5xS0fxPKkJKmlNMKLjN0r2xFyw?=
 =?us-ascii?Q?r4mEmFh9v24hLsY7EQ9wp+LSuemS+jjIzMbXlopGm+Vzr0fwK4Yz0kVp1dky?=
 =?us-ascii?Q?VxZYwxCJaIc3e8Nkx8CDDG1DY0uhh1ijbH+xGUV0YAJzx4zGvI7WF8zcQua+?=
 =?us-ascii?Q?ZQwKcLGFa8w2bld7yujOTxFRtbbBqQUaGqXaU+MhORqcS0wGGeZ5OtKtDJJM?=
 =?us-ascii?Q?1xhZCBW/XVxNaQQF1oj8WCnTGHIOETjUUEv551l7ezHp5C1ZMKLzdovFDdZ0?=
 =?us-ascii?Q?H2it//GVSE5wxUfEe0gbR4EQ90yEyoIMjHGWD+PQ5tNAIGoeCuvk1bufmMCU?=
 =?us-ascii?Q?qzUux0KxpHwz3v3Cg965klHmlN9tJ7kT1IkssaTPHjRP1ZqBWObhNl1RlW5H?=
 =?us-ascii?Q?sebdCbj2Vr4rwz7dd5eT8r6im6T4QZ/9d3TUD+ZMd0mT89S5u4J2iMqvXNOB?=
 =?us-ascii?Q?Vqch2uoi10pWNlBPRcOwwXv7WHuqWpYCK8svwfjMM3ZqCWFuPsHDulXBAz9a?=
 =?us-ascii?Q?B5C/KJPn1giJWreUInCDJzp7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aaa4bd41-bfce-4115-b6cc-08d97f48c5f6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:47:59.0885
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FyTqVgvC0MDTrg7c7KckWaVbx3yNvWQ1CnDRpKVt3PquI+yWR8bzIFxuniUfS5aINx4hLUikyy8RWnZQOBbeJg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5742

On 24.09.2021 12:33, Wei Chen wrote:
>> From: Jan Beulich <jbeulich@suse.com>
>> Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 16:56
>>
>> On 23.09.2021 14:02, Wei Chen wrote:
>>> --- a/xen/arch/Kconfig
>>> +++ b/xen/arch/Kconfig
>>> @@ -17,3 +17,14 @@ config NR_CPUS
>>>  	  For CPU cores which support Simultaneous Multi-Threading or
>> similar
>>>  	  technologies, this the number of logical threads which Xen will
>>>  	  support.
>>> +
>>> +config NR_NUMA_NODES
>>> +	int "Maximum number of NUMA nodes supported"
>>> +	range 1 4095
>>
>> How was this upper bound established? Seeing 4095 is the limit of the
>> number of CPUs, do we really expect a CPU per node on such huge
>> systems? And did you check that whichever involved data types and
>> structures are actually suitable? I'm thinking e.g. of things like ...
>>
>>> --- a/xen/include/asm-x86/numa.h
>>> +++ b/xen/include/asm-x86/numa.h
>>> @@ -3,8 +3,6 @@
>>>
>>>  #include <xen/cpumask.h>
>>>
>>> -#define NODES_SHIFT 6
>>> -
>>>  typedef u8 nodeid_t;
>>
>> ... this.
>>
>=20
> you're right, we use u8 as nodeid_t. 4095 for node number in this option
> is not reasonable. Maybe a 255 upper bound is good?

I think it is, yes, but you will want to properly check.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:49:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:49:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195289.347923 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTill-0006Ou-0v; Fri, 24 Sep 2021 10:49:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195289.347923; Fri, 24 Sep 2021 10:49:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTilk-0006On-U5; Fri, 24 Sep 2021 10:49:28 +0000
Received: by outflank-mailman (input) for mailman id 195289;
 Fri, 24 Sep 2021 10:49:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTilj-0006OO-8g
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:49:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fb26324d-7afb-4c28-987c-3d1365314a4b;
 Fri, 24 Sep 2021 10:49:26 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2055.outbound.protection.outlook.com [104.47.9.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-RAtbxrpTOZShSo288wCwsg-1; Fri, 24 Sep 2021 12:49:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4350.eurprd04.prod.outlook.com (2603:10a6:803:3e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 10:49:23 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:49:23 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0025.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 10:49:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb26324d-7afb-4c28-987c-3d1365314a4b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632480565;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1q3puiliYdZoVqJouatsAVDuwCBWPLUfp26THS3CwNw=;
	b=ZtE6pLIV49NWSJjjKNqv4dfKPcsPD5c6+T3RbP90BmLuTsLaEsmCi8Ctk6BWjkkDovSFb3
	woM/4qo034Tvs0Mnl38RbNxou4MNgP/ojej02BEQ/9gG5tXYjdDXwdr36Iy9jmd9wFXiZH
	YtF14ennXui0GvcyXRd+k2RNx1b2sQU=
X-MC-Unique: RAtbxrpTOZShSo288wCwsg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JZD7ROh4zTHI5cwZjHKVwArR1L+UIRyJ8XJNwdm5KHvz4dXTN9bKU0Ns9qfmqnVj4LY24KI+yKKjZMfTRj36vN37upAK2NAvL4V4aWedKHPKFdaIqR0Py33LWO2WWtuM8Phw8pwsaCYrj4JudujB2ngY/4hoW0i/G3D6UZ35JgpsWC5gMx2PdP9teyo2GW87WEqTskX28AKa7n9Ez42K6NB4WFCkoL8qFpizRImg85C5ZSJ2wvOBkzHVdT0iNRwEgdLhpvoIONvcE3iJDOGvHrW08bP/9bpFcSFOeACTdrKjg+XqjoB7cMupRr9wa5qnFE5BJkW35mwjAZXGohLpDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pLRBaqlOdRpvHdVYmUYhZb1e4XBqWWZzrnIW7aIUZMI=;
 b=ZNUmKxkArVJWMp1wYfnDc2RVNI4T2GE35j0ZdeSEDP6oQ6bxh4SQaNLAqX7pdXluLX8gfPRWpY8uScvIkH3yamTtPgZqF7BD7qx8wWKoEt/Cg1QvEiXjtIH5O7cckAAOcS8V7NVmmtkoHzrW+QGHtI9L8W7lJEODfwE10LaNmt0w4+5wXwqXn5Ap+8nv7/ZlbFumcYePGZ/cyAhELFPv8OYp4n2XAOI5/qthGIG3IE88annifiYAOHkiJUdVYah9IebBsjsT9HhrJQRJL+vA/SWBA6cKQuaJ9wf5T+kauDeSQupRlB53dos8shZyNjo8vi3d4yrIUD9W0TVxflOKUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
To: Wei Chen <Wei.Chen@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "julien@xen.org" <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
Date: Fri, 24 Sep 2021 12:49:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
Content-Type: text/plain; charset=gbk
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0025.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1c::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 11c3c23c-a282-4bfa-7b0d-08d97f48f899
X-MS-TrafficTypeDiagnostic: VI1PR04MB4350:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435083335944CA2FA629699CB3A49@VI1PR04MB4350.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oE9z13yfAeEWfKOFJ1wYufROMKSRduIrKt1VbS/3DxX8zRf+9AYSGWvVvpG9ILf15U2Wal9KPKRmFzYE5LWpJTl7UNLa7FlDF53H3UcDWIr372l9dpck5fH3n98k21zrVp6TGIjtjYumYqlz5QpoA1wpPlkE6pQHKqoGeL0k/emWrlOpagfbtze7ZgCc/5hIEsZo/yWgdfFA/Df/PVlJuA28MTVJVTIzaX9MjUV8HkoxkSf1WWevylGTbfR3Cwo36opiAy7w7ruXwTSeIfBcfjI5PD+JVzdxl5s66rjwmgSUpFO+Cu7R17lHc8vQcdAJwWJsJC5NbS0nQmhsVv4Zxlhb2AzuKXR7Y48v1e+1OCdSz74n9rThXTozN9TKrMG5i5sdL9a3M8LouxFC9UYx7QNGtKOSpwtmoZEgiqrBtgzkoSJ9X8PARZcHZYALEldu82u2t/kvnCVkcCzrMc4C7jqmHov6tUh8rQDL0m2WIA2kJPeyWBXi7+7fDDUUYVA4H3LmeQpzInqCMjZu6FVyZ+8dQTyX2NqRvZuvMPYeYfjcihTPIf9EdDbxpYoOTk9JQYsyxaRv7g40+7x9zDKpLgoFJS9991PDnGUA/m1R6/rcZ2/ySwBHw7VLe60Ztkj7FJOCzM+fzfZjozb30EXmaxFfH0200yPfCWdNmGaUoaGtcHgNCIprjDylyugLvDOfj4OKgHZnA2zydbItph6XEtwwUfXoni70TXdwJf/oAcM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(83380400001)(38100700002)(66946007)(186003)(8676002)(8936002)(6486002)(53546011)(66476007)(2616005)(31696002)(16576012)(66556008)(6916009)(54906003)(36756003)(31686004)(86362001)(5660300002)(316002)(508600001)(2906002)(4326008)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?fiO9HKP0S0n4Zk/bZNnKfJS2jz+fwuksxxx92RH6LfoOs2PDW99hbW08z9Z8?=
 =?us-ascii?Q?2PLN4RCIkI22mFSheZzzVUJ4cLXHfi1BjCWe4AsO3VfqYNNUt0P9ZnSuztwt?=
 =?us-ascii?Q?KucbiPpqIADlbK/jKt2SF+ju4vA5gq8oUMas+fZLrPz5c7c1nfUgDgFwnTdr?=
 =?us-ascii?Q?/+umsaekBIrgmkxsl4OCYIkUdAH9EkV6aBvjliOYALrd7WCCvCqciRQ8dY+l?=
 =?us-ascii?Q?7pN2FrbcaHpJxTH8GtAzMiseRfxEBKK1Z7MCJDjI08qnZP6yiSr2ZltT7ZBp?=
 =?us-ascii?Q?dgaiFrvY3rCFIToYuB3A17Zfc+u/9YzBHT/3MNyqDRtLCOpL7TdMly7hbzgA?=
 =?us-ascii?Q?9MZMsT3RxB/W/wnllW3w40I1ZitHjtacsfC8P3zUEUK29YpuRNIqG4dk1X+l?=
 =?us-ascii?Q?bnDw/JS3RHAjjWlEw2guB0sLU1FuBrTalQv+FviKHMLKM/BuJ4s59Z57iHKL?=
 =?us-ascii?Q?wopUGCWG8muQmu+AAbJfUxvoGkQnAzpp5PU6dnO1xzpLfMh5i0VQ1b9T1X6x?=
 =?us-ascii?Q?QJMrRhklYLVZVCP/2PYVD1EjQt1+EsKHHgNtK1AmweNiDZ5dTHgrB3GDCZP8?=
 =?us-ascii?Q?XRSpmqrRlPYNTHezjDZ3nRrL99xyoXRvQLeYo9ILVhU9rsZ4RQTSUMx4xnwf?=
 =?us-ascii?Q?pyX+McL19MpxnvYzXkDltudVu+08+bMYlVad/IunHNKjJzoWIe0qmqWR8zVb?=
 =?us-ascii?Q?sCP23NdTi3i9nIx5P1FkFy/4TVMshAl7pZQA21TGJaH/Q2Ie1XXpDdLHEgIK?=
 =?us-ascii?Q?RzhG8jBwCg1JMj2WxNhShZBBI5RXRZnLw67eggC7uKGGlRshTeMv03d/JZgc?=
 =?us-ascii?Q?P286XlE8wsIRTw0/S8SfCKzcFErmgeOMNKnzv33Rpsl7eTHTE1nRqq4PrkFl?=
 =?us-ascii?Q?tFkG7K9v3cHnwqR7GBY/cOfTU3vhHiWfB3/3O3uSPoHP3hBKdpQH5TPtDWlm?=
 =?us-ascii?Q?31qWi+dJd4Q2Zvpr/IW037cKUBfM6mGHLL3A1DBmv1pIpGSUGuVLfYx4QSdj?=
 =?us-ascii?Q?15lj/y3IiGZPi+on5oUhbG2aC/c86+kmUST9iTdFcgDNcG3OzGNFDitHBYaY?=
 =?us-ascii?Q?XMkOoAZoHoqK0CfFVCZOo0E8NUanJjN0RIHOQhAvf9oumWewiAOdfEDELtql?=
 =?us-ascii?Q?SSRggymPzz5wBBzAZlvGOvvIfn0e9QijG3zOwf8lDXEYdNhOKLwXuRZmqzAI?=
 =?us-ascii?Q?xj5Rspn4AMt8h/2oxBCz/CjSujNxeRhsDDq4325d88pu3Rs2KCWGnFcaN3Zl?=
 =?us-ascii?Q?XZrcZDtgwcT6FZVf6fiPKc+r+tgzDYO1QyEUUVRrxWIzLEFUwXFzHyJcSFIb?=
 =?us-ascii?Q?JE1aCKlxHEqXCo9aFIZF6yxZ?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 11c3c23c-a282-4bfa-7b0d-08d97f48f899
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:49:23.5896
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4/3+Zb1kzN6Vu0AL+UlQCPlmQXIwrUBuCJ8fZOAN/ZPd7T57obtngg7La42TdPXT0lQ91gpZh6CPQ9p5akO1rQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4350

On 24.09.2021 12:31, Wei Chen wrote:
>> From: Jan Beulich <jbeulich@suse.com>
>> Sent: 2021=C4=EA9=D4=C224=C8=D5 15:59
>>
>> On 24.09.2021 06:34, Wei Chen wrote:
>>>> From: Stefano Stabellini <sstabellini@kernel.org>
>>>> Sent: 2021=C4=EA9=D4=C224=C8=D5 9:15
>>>>
>>>> On Thu, 23 Sep 2021, Wei Chen wrote:
>>>>> --- a/xen/common/Kconfig
>>>>> +++ b/xen/common/Kconfig
>>>>> @@ -11,6 +11,16 @@ config COMPAT
>>>>>  config CORE_PARKING
>>>>>  	bool
>>>>>
>>>>> +config EFI
>>>>> +	bool
>>>>
>>>> Without the title the option is not user-selectable (or de-selectable)=
.
>>>> So the help message below can never be seen.
>>>>
>>>> Either add a title, e.g.:
>>>>
>>>> bool "EFI support"
>>>>
>>>> Or fully make the option a silent option by removing the help text.
>>>
>>> OK, in current Xen code, EFI is unconditionally compiled. Before
>>> we change related code, I prefer to remove the help text.
>>
>> But that's not true: At least on x86 EFI gets compiled depending on
>> tool chain capabilities. Ultimately we may indeed want a user
>> selectable option here, but until then I'm afraid having this option
>> at all may be misleading on x86.
>>
>=20
> I check the build scripts, yes, you're right. For x86, EFI is not a
> selectable option in Kconfig. I agree with you, we can't use Kconfig
> system to decide to enable EFI build for x86 or not.
>=20
> So how about we just use this EFI option for Arm only? Because on Arm,
> we do not have such toolchain dependency.

To be honest - don't know. That's because I don't know what you want
to use the option for subsequently.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:52:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:52:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195296.347935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTioY-0007or-Ik; Fri, 24 Sep 2021 10:52:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195296.347935; Fri, 24 Sep 2021 10:52:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTioY-0007ok-Ff; Fri, 24 Sep 2021 10:52:22 +0000
Received: by outflank-mailman (input) for mailman id 195296;
 Fri, 24 Sep 2021 10:52:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Mfdr=OO=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mTioW-0007oe-H2
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:52:20 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.48]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89e8340a-01c5-4a1b-88fd-e6c5d7a1339a;
 Fri, 24 Sep 2021 10:52:18 +0000 (UTC)
Received: from AM6P192CA0081.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::22)
 by AM9PR08MB5874.eurprd08.prod.outlook.com (2603:10a6:20b:281::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 10:52:15 +0000
Received: from VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:8d:cafe::f4) by AM6P192CA0081.outlook.office365.com
 (2603:10a6:209:8d::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 10:52:15 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT051.mail.protection.outlook.com (10.152.19.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:52:14 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Fri, 24 Sep 2021 10:52:12 +0000
Received: from a65bb6ca84ca.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 608D8706-5811-496B-92CA-7F1113CE724B.1; 
 Fri, 24 Sep 2021 10:52:01 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a65bb6ca84ca.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 10:52:01 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6128.eurprd08.prod.outlook.com (2603:10a6:102:f2::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 10:51:59 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 10:51:59 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0188.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 10:51:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89e8340a-01c5-4a1b-88fd-e6c5d7a1339a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vCmURqAOGUZi1Us3emkbyTPWGvgqPY5SYraxsyRAEO0=;
 b=2vMSkrqW9d7yohjkkkcrN+7z3BUbkoOu6q23+COGe3daFiSNonIr5Zzr/9ysnx+heiNeYpIvuqr/cmCBuu3C6Ky2Bv/WQ2OzSkSYyGNPM8G5y31jrFrpIIyIfAY3QnWf798rQOdtMVknOC2/O1xOeryU0f2pSi/wqbdhLBfh1JU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 85284076dcb9a8a2
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aPPQ3in+84MHNukL6Bvs9cDSu5heFLO4V0v8XBnnJjA4fp2zeGX1pH9vbTcmpGmcYj5T3eJVqTgI2nhZps0gmQklAsHl77jvfXSvvlU7hU2I2CdoCU42OrlIzzGfb8u5GhNQC9NZwGD+XpQVAbzNP9eoxMpp9nVnfSw9/uDmaOio8dz9plVvdyfCphzHWBRgrHX1q/XqvJmv34s9qeTNL+Z26oejOFzBrkyrtU8e3jkF/u4t7r2WKC9AjIGLLBOZy4WnuHNQgceVsrH3Dfhhy5mbkpG4bndZJdXxg4vkFOlPlcnetVNyKouPotBQjNQeQl7zU19Lei4MXsyo0ytQRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vCmURqAOGUZi1Us3emkbyTPWGvgqPY5SYraxsyRAEO0=;
 b=HuVeKb3elfy84Q+MjNXs3fvjOlIzHLIXDjnhRkK1jvm9h8rHRCMwGZoI5VlvLbGE20aoZfyB4/36Cwyr3Nk/f3R8SqIZ69emsvdYGE761BH4xNS4cqDuXXUyfL6DQbajJsifUSZvY81siA5KHkPf0247b298lugcko5CKyX0rwGmtvt4knFb/ZCOgITt7ziBdhllPailO0uXeC/IgmP7BkPjMzhjohakhV2aZZSFrFSsa5gsLucOF9xpFvwGf6bzdlgB6kcf+8enqlPPEmKKYyMa7GaDcTybwztSXEBantVSpoltayhuwRRhfb/RRtmfpjf8smjZj5xX6o6SszbZyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vCmURqAOGUZi1Us3emkbyTPWGvgqPY5SYraxsyRAEO0=;
 b=2vMSkrqW9d7yohjkkkcrN+7z3BUbkoOu6q23+COGe3daFiSNonIr5Zzr/9ysnx+heiNeYpIvuqr/cmCBuu3C6Ky2Bv/WQ2OzSkSYyGNPM8G5y31jrFrpIIyIfAY3QnWf798rQOdtMVknOC2/O1xOeryU0f2pSi/wqbdhLBfh1JU=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <alpine.DEB.2.21.2109230943510.17979@sstabellini-ThinkPad-T480s>
Date: Fri, 24 Sep 2021 11:51:51 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>,
 =?utf-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <6A929631-AA44-4ECF-AACB-16C0110393F6@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>
 <20210922141341.42288-3-luca.fancellu@arm.com>
 <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s>
 <2C0E480C-DF1C-4AAD-89F5-99D52B016970@arm.com>
 <alpine.DEB.2.21.2109230943510.17979@sstabellini-ThinkPad-T480s>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0188.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a::32) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8fb9fe46-6e22-41f0-d7e5-08d97f495ed0
X-MS-TrafficTypeDiagnostic: PA4PR08MB6128:|AM9PR08MB5874:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB58749BE6B9D9028FA086C77FE4A49@AM9PR08MB5874.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 XVFje0UnoBC+OdEx4u1AjtNcplC/+FwoSRpu+HNs6vtibnPMqZu4DUrq7UdZwnGparcXHvvcPGxND+GupxPkJj8ZGCOGtgItJ1T9iErwSRgpGut0A+qHe9gGVy5+Pfgo9cdNPLP1ARQCju8jYZbL94iFeTpV1Dj84zQWbaMBCnz5VNYZG63UYGGZpuM2Ck5Sasf8bqQYC66c5AiPs9j7/tIUntxzCsKNVCEOkC1MTVM4L1SHyhcQqx9JQTsi6y373kocY9lWejtNaOYx/DB9kFdQldn7QSVdbr4Sxd7wY5cEst3KOlD6P/HO46mqS8eIy9AL5USnRnJL9DjDLnAq8kvV2XNCDa53eE0WcNaQCqWPuHfc2EJz4aLCg1Iu8muVhlLVmflkLJaeP72Br8qqIDkQ5gYEBzkIDNfXyKI1yyDAj91QXlqTs7Q2EWaASnbQbFBvIW5g1pX/29fNArvp/MGXJ7vLr6PXsh6zo1I0jhYBkX8HadvsBEfxQHRtthpQ9l8shQEMMigbyU2GFduyH2Zd8ewQPUfrNJO1clwj/eh/pTpABXWjXnvgXtY6+fuZ0IWRJQ6Y/qQc5ZIJsRJB8WDBLMyE1ZbasaJd+e+WVdf3/IXB1gVaMeGfnzh6PbtlEUU/6ciPCjkR8vjW3FubF9Qwsd1ESTVQlMnpGB7zAzCRGFNwuKFZ69J4z8H077AQVi8cw90jdJX1x3UgRDaTy6BoJsysgnI6bNPiITihtiz97FJcOwVEcUm1FksyR2wy34wZczeIrxV6DaaAG2ObvA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(2906002)(4326008)(2616005)(86362001)(6916009)(316002)(52116002)(66946007)(508600001)(956004)(6512007)(44832011)(54906003)(8936002)(7416002)(66556008)(33656002)(66476007)(38350700002)(38100700002)(26005)(5660300002)(36756003)(53546011)(8676002)(6506007)(6486002)(6666004)(186003)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6128
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0dc4f034-5cb1-4913-2d22-08d97f495565
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VRG1Tkpw4netxyaFU95TAkPbgCTal/yDn2AohDhgQ04YK+EmXyT0qRljnLripAUQgGuG3XGL59dingg9eOx9d3PKYjRXZk0SLLfaNyyBMJ3yKpHRILjU5cfQ3lZ3ywQ2ppVq6hhSQVwoZKiyPXDNdEHLaTvFjl0rV7j/nk/iFZPEg1A3XD5aRWfoGcOY3JPu4dhBhM2olhPH92mpAmhFHS57d7qM1V1uf0ersuOdDuyTaW9iUO3/4xLj6gsE6WkmOjGQ3cxaXjLYU6NE3189O9p1tk7WsoLv0mGAuCypK7YkDYxabttGZjoH0epPlXisvAQO5HayY3Yp69Qm6W+a8h7S9GM2pAegQ58HSwwwfhXFhNCsB4pHVN/B2L+x4D6rBTi6X9Xa/ngKTkOYjVnTq091wMOAoaA6ZSF/ztccdOcBVDH0Bl2pwC86ZbDKd+ea+rD9PoXZuTv/KHFlTAYTYzlowaY+rk6I+VFW1N8zRZXZwX2splVxdHZMAWIvUPmTECWzwq8U0j00zoT1+unL4QMSGofVBQmQUlL/b1LQVdrgHZSb51hqAUuT2TEAzBvfwtDG1mC4/24oziE7240Ko5PFmI2pCJUmVOzS/dg7bAZjHEYNWtrJcgH6tOlmMGswjL9klQzwV6lve9HNAqMIym+Jb2Wjq4dNMeXB7j994wFgZUNHNsiYBkjCETqCrtTaJaNsnERgTJ+VyCWA5ZcdfTw2+3d2qWcuMJtyjwq02pg=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(336012)(8936002)(82310400003)(86362001)(6486002)(36756003)(33656002)(70586007)(36860700001)(2906002)(6862004)(70206006)(2616005)(956004)(6512007)(316002)(81166007)(4326008)(54906003)(47076005)(8676002)(44832011)(53546011)(83380400001)(6506007)(26005)(186003)(508600001)(107886003)(356005)(6666004)(5660300002)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:52:14.8631
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fb9fe46-6e22-41f0-d7e5-08d97f495ed0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB5874



> On 23 Sep 2021, at 17:59, Stefano Stabellini <sstabellini@kernel.org> wro=
te:
>=20
> On Thu, 23 Sep 2021, Luca Fancellu wrote:
>>>> +/*
>>>> + * Binaries will be translated into bootmodules, the maximum number f=
or them is
>>>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen =
DTB
>>>> + */
>>>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
>>>> +static struct file __initdata dom0less_file;
>>>> +static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_=
MODULES];
>>>> +static unsigned int __initdata dom0less_modules_available =3D
>>>> +                               MAX_DOM0LESS_MODULES;
>>>> +static unsigned int __initdata dom0less_modules_idx;
>>>=20
>>> This is a lot better!
>>>=20
>>> We don't need both dom0less_modules_idx and dom0less_modules_available.
>>> You can just do:
>>>=20
>>> #define dom0less_modules_available (MAX_DOM0LESS_MODULES - dom0less_mod=
ules_idx)
>>> static unsigned int __initdata dom0less_modules_idx;
>>>=20
>>> But maybe we can even get rid of dom0less_modules_available entirely?
>>>=20
>>> We can change the check at the beginning of allocate_dom0less_file to:
>>>=20
>>> if ( dom0less_modules_idx =3D=3D dom0less_modules_available )
>>>   blexit
>>>=20
>>> Would that work?
>>=20
>> I thought about it but I think they need to stay, because dom0less_modul=
es_available is the
>> upper bound for the additional dom0less modules (it is decremented each =
time a dom0 module
>> Is added), instead dom0less_modules_idx is the typical index for the arr=
ay of dom0less modules.
>=20
> [...]
>=20
>=20
>>>> +    /*
>>>> +     * Check if there is any space left for a domU module, the variab=
le
>>>> +     * dom0less_modules_available is updated each time we use read_fi=
le(...)
>>>> +     * successfully.
>>>> +     */
>>>> +    if ( !dom0less_modules_available )
>>>> +        blexit(L"No space left for domU modules");
>>>=20
>>> This is the check that could be based on dom0less_modules_idx
>>>=20
>>=20
>> The only way I see to have it based on dom0less_modules_idx will be to c=
ompare it
>> to the amount of modules still available, that is not constant because i=
t is dependent
>> on how many dom0 modules are loaded, so still two variables needed.
>> Don=E2=80=99t know if I=E2=80=99m missing something.
>=20
> I think I understand where the confusion comes from. I am appending a
> small patch to show what I had in mind. We are already accounting for
> Xen and the DTB when declaring MAX_DOM0LESS_MODULES (MAX_MODULES - 2).
> The other binaries are the Dom0 kernel and ramdisk, however, in my setup
> they don't trigger a call to handle_dom0less_module_node because they
> are compatible xen,linux-zimage and xen,linux-initrd.
>=20
> However, the Dom0 kernel and ramdisk can be also compatible
> multiboot,kernel and multiboot,ramdisk. If that is the case, then they
> would indeed trigger a call to handle_dom0less_module_node.
>=20
> I think that is not a good idea: a function called
> handle_dom0less_module_node should only be called for dom0less modules
> (domUs) and not dom0.
>=20
> But from the memory consumption point of view, it would be better
> actually to catch dom0 modules too as you intended. In that case we need =
to:
>=20
> - add a check for xen,linux-zimage and xen,linux-initrd in
>  handle_dom0less_module_node also
>=20
> - rename handle_dom0less_domain_node, handle_dom0less_module_node,
>  dom0less_file, dom0less_modules, dom0less_modules_idx to something
>  else more generic
>=20
>=20
> For instance they could be called:
>=20
> handle_domain_node
> handle_module_node
> module_file
> modules
> modules_idx
>=20
>=20
>=20
>=20
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index e2b007ece0..812d0bd607 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -22,8 +22,6 @@ typedef struct {
> #define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> static struct file __initdata dom0less_file;
> static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODU=
LES];
> -static unsigned int __initdata dom0less_modules_available =3D
> -                               MAX_DOM0LESS_MODULES;
> static unsigned int __initdata dom0less_modules_idx;
>=20
> #define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
> @@ -592,14 +590,6 @@ static void __init efi_arch_handle_module(const stru=
ct file *file,
>          * stop here.
>          */
>         blexit(L"Unknown module type");
> -
> -    /*
> -     * dom0less_modules_available is decremented here because for each d=
om0
> -     * file added, there will be an additional bootmodule, so the number
> -     * of dom0less module files will be decremented because there is
> -     * a maximum amount of bootmodules that can be loaded.
> -     */
> -    dom0less_modules_available--;
> }
>=20
> /*
> @@ -643,7 +633,7 @@ static unsigned int __init allocate_dom0less_file(EFI=
_FILE_HANDLE dir_handle,
>      * dom0less_modules_available is updated each time we use read_file(.=
..)
>      * successfully.
>      */
> -    if ( !dom0less_modules_available )
> +    if ( dom0less_modules_idx =3D=3D MAX_DOM0LESS_MODULES )
>         blexit(L"No space left for domU modules");
>=20
>     module_name.s =3D (char*) name;

Hi Stefano,

Yes I understand what you mean, unfortunately it can=E2=80=99t be done, I w=
ill explain why in details
because the UEFI code is very tricky in the way it was written.

Dom0 modules and xsm-policy are handled in boot.c by read_section(=E2=80=A6=
) or read_file(=E2=80=A6) and
both call handle_file_info(=E2=80=A6) that inside calls efi_arch_handle_mod=
ule(=E2=80=A6).
Dom0less modules (xen,domain child modules) are handled in efi-boot.h by ha=
ndle_dom0less_module_node(=E2=80=A6)
that may call allocate_dom0less_file(=E2=80=A6) and (to reuse code) calls r=
ead_file(=E2=80=A6).

So there can be maximum (MAX_MODULES-2) boot modules because at start in st=
art_xen(=E2=80=A6)
we add Xen and the DTB as boot modules.

This amount is to be shared among dom0 modules (kernel, ramdisk), xsm-polic=
y and domU
modules (kernel, ramdisk, dtb).

The thing is that we don=E2=80=99t know how many dom0 modules will be speci=
fied and also for the xsm-policy.

In your code example, let=E2=80=99s say I have MAX_DOM0LESS_MODULES=3D(32-2=
) so 30 modules available,
If I declare a Dom0 kernel and 30 DomU kernels, it will pass the check, but=
 on boot I think it will ignore
the exceeding modules.

For that reason we need to keep track of how many =E2=80=9Cslots=E2=80=9D a=
re available, so in my code every time the
read_file(=E2=80=A6)/read_section(=E2=80=A6) is called, the dom0less_module=
s_available is decremented.

If we want to get rid of one variable, we can just assume the dom0 modules =
and xsm-policy will be always
loaded and we can set MAX_DOM0LESS_MODULES=3D(MAX_MODULES - 2 - 3) where 3 =
is dom0 kernel,
dom0 ramdisk and xsm-policy. If the user doesn=E2=80=99t load any of them, =
we just lost 3 slots.

Another solution can be keeping just the dom0less_modules_available and eve=
ry time loop backward in the array
starting from that position and stopping when we have a dom0less_module_nam=
e.name =3D=3D NULL so we
know we have an available slot or we reach below zero and we know there is =
no space. But I think it=E2=80=99s not
worthy.

So if you really want to have only one variable, I will remove space from M=
AX_DOM0LESS_MODULES and
I will push it in v3.

Cheers,
Luca










From xen-devel-bounces@lists.xenproject.org Fri Sep 24 10:59:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 10:59:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195304.347946 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTivF-0000Rp-Av; Fri, 24 Sep 2021 10:59:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195304.347946; Fri, 24 Sep 2021 10:59:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTivF-0000Ri-7j; Fri, 24 Sep 2021 10:59:17 +0000
Received: by outflank-mailman (input) for mailman id 195304;
 Fri, 24 Sep 2021 10:59:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTivD-0000Rc-RV
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 10:59:16 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 745d2c8a-1d26-11ec-babe-12813bfff9fa;
 Fri, 24 Sep 2021 10:59:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 745d2c8a-1d26-11ec-babe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632481154;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=FlwNEC52HeN95IM3kjyUNPqzSxw77wc9w4VwtcGC+pA=;
  b=YWwQxxnxmzlYQlgJR8DwJA02UzUOtgPEcyMBmPxylTDfO6GbM1icDDX5
   I7vVdRIn+hQqTGEEWmHE8hG9qU0Bp7nCd7MO9CFIQLy3N/lMSP5ytcg/Z
   bFVIlRUDJrXbliIOOdXgFzeVsQHdKfpdwU5IlyviL+8dcbpXLTYwSQ+N7
   o=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: TyOn3jP3+h8p18ls447RQWQkuy0t6ljSkRfr1JlZscvEID/oWuXdZnPQwBpiNbAS1R2Kr4TX6u
 bVq0l3H5/hr3V5QHaQ2VS2cSpxjFvpyiZV8hgYD1kI8m8O4mAc3YIusIuRQJv87Rcg1Nb2rp44
 SoEKsMQqSPKE3wzCJwFHPLXDgFTMYKCKvHimyzY2IU6TMaON7PFg8FuiKpQS2ytjpGm6fLRId1
 lJD6OAcj1l+RHZZed1/xvxU/08mNzmbiQyiom3Ny2Zn8sG9utAWsnqXPW1aK3wjOt1o9Fq3sGS
 j9KM0R4Fs7B/mzMgkQ/8tbXN
X-SBRS: 5.1
X-MesageID: 53126583
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:n82gU68TAONO6zREhsmZDrUDWHmTJUtcMsCJ2f8bNWPcYEJGY0x3z
 jBMUG7Qb/uMYzemLtAiOti0pktVscCExoRrHVdppXw8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j3tYx6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhey
 9hs5LWUcz1yFa7Mv+E0Ax1GTRthaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFh25q2J0eRp4yY
 eIFYxd2QUnqZSQWOw4OVLE7nO2StGnWJmgwRFW9+vNsvjm7IBZK+KDkLd79atGMA8JPkS6wt
 m/Aumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3ntZoilCOwm0YCDUNSEC25/K+jyaDt8l3c
 hJOvHB09O5rqRLtHoKVswCETGCskR0Reot5N9cAsyqOyo3RyladGDY+UWsUADA5j/MeSTsv3
 16PutrmAz1zrbGYIU6gGqeoQSCaYndOcDVcDcMQZU5cuYCy/d1q5v7aZos7SMaIYsvJ9SYcK
 txghBM3gakaxeUP3r+ylbwsq2Px/sWRJuLZCwO+Y45E0u+bTNL5D2BLwQKChRqlEGp/ZgPa1
 JTjs5LChN3i9bnXyESwrBwlRdlFHcqtPjzGmkJIFJI87Tmr8HPLVdkOu2sifh00bZtYJWWBj
 KrvVeV5vsQ70JyCN/MfXm5MI55ykfiI+SrNDJg4keaikrAuLVTarUmClGab3nz3kVhErE3ME
 c3zTCpYNl5DUf4P5GPvH481iOZ3rghjlTK7bc2qlHyPjOvBDEN5vJ9YaTNimMhit/jayOgUm
 v4CX/a3J+J3CrajPXWHrdBJfTjn7xETXPjLliCeTcbaSiJOE2A9Ef7Bh7Qnfo1uhaNOkenUu
 Hq6XydlJJDX2RUr8C2GNSJubq3BR5F6oS5pNCAgJw/wiXMifZyu/OEUcJ5uJesr8+lqzPhVS
 fgZeprfXqQTG2qfozlNP4PgqIFCdQiwgV7cNSSSfzViLYVrQBbE+4G4c1K3pjUOFCe+qeA3v
 6akilHAWZMGSgk7VJTWZfujwkmfp38YnO4uDULELsMKIBfn8ZRwKjy3hfgyepleJRLGzzqc9
 gCXHRZH+rWd/95rqIHE3PnWoZ2oHu1yGlthM1PatbvmZzPH+meDwJNbVLradz7qS26pqr6pY
 v9Yzq+gPaRfzkpKqYd1D51i0bk6u4n0v7ZfwwlpQCfLYlCsBu8yK3WKx5AS5KhEx7sfsgqqQ
 EOfvNJdPOzRas/iFVcQIisjb/iCiq5IymWDs6xtLRWo/jJz8ZqGTV5WbkuFhyFqJbdoNJ8on
 LU6s8kM5g3j0hcnP75qVMyPG7hg+pDYb5gaiw==
IronPort-HdrOrdr: A9a23:nQzN+65+GXxnguSJHQPXwVmBI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwX5VoZUmsj6KdhrNhQItKPTOWw1dASbsN0WKM+UyDJ8STzJ856U
 4kSdkDNDSSNykKsS+Z2njALz9I+rDum8rJ9ISuv0uFDzsaE52Ihz0JdDpzeXcGIjWua6BJcK
 Z1saF81kWdkDksH4mGL0hAe9KGi8zAlZrgbxJDLxk76DOWhTftzLLhCRCX0joXTjsKmN4ZgC
 f4uj28wp/mn+Cwyxfa2WOWx5NKmOH5wt8GIMCXkMAaJhjllw7tToV8XL+puiwzvYiUmRoXue
 iJhy1lE9V46nvXcG3wiRzx2zP42DJr0HPmwU/wuwqqneXJABYBT+ZRj4NQdRXUr2A6ustn7a
 5N12WF87JKEBLphk3Glpj1fiAvsnDxjWspkOYVgXAae5AZcqVtoYsW+14QOIscHRj99JssHI
 BVfYDhDc5tABGnhk3izyxSKITGZAV2Iv7GeDlNhiWt6UkUoJgjpHFog/D2nR87hdsAotd/lq
 L52gkBrsA7ciYsV9MOOA42e7rANoX8e2O+DIusGyWTKEgmAQOHl3el2sR+2AmVEKZ4u6fa3q
 6xCW9liQ==
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="53126583"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JGu5e44a/HA3S6UT4ywYX8gvbV5YlV3tF3V+W8RdVuRPMOrQnNlLlpx2+8jmmHYS2cR+06TcSy9Iq1uvp577EFrg+3SSHOzo4rnzhSkVBBCROnJT7snT/KvXqrvweOCawmG+a/NglYLm41qA0QrTc5lRj6fVxtwW51j5k6g5UD+uS0z816M3FkKSlaVghzyJhhRkOg9bLeaC/iZD3ijVYv4yscN00ZP/tFCXE5a9/H0uErVlecHeuv4rKquApLWLTSAhTWSznZWPfxauae+MWtwjLG7rPNyyJbFoBiCyegM+fzP/jVSs37A2W3skh/GYon5GHtF8APhnEnxNCE7rKQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CLNmb92P0u42Xt8YjmX35du58AFKhSE3OO/HBV0QoXM=;
 b=RzCnb8czC/LKifAIoeWv5lGfLvb3SlWXCuSow6qst31q691bxj2FSbOYPMJX8HxiBsme9hazEPmQP+yNPAc0lEPl0yWkuEUFKiVew8GNZE+AfyB63JI9Z7HPrLp+9pOmIo/zPVmrF2s8RuebPzvp+UDKHfJWJfZ/anynDtGIwKRVpWgjEoLDQ37zOYWmXffILSf+AcoOh57WBYQTIwY0O9CTLZKpeGCcoP7jN+m9azVGE5L1tkzZGxGZL4gC8LTIMEALNaaH9bXkxQOy0N9spSTkxqD/dyp0ZhddykBaGQX8YnC1qWGGccc1t9m9TqkT7HfP1ny7N9zVychoxss01A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CLNmb92P0u42Xt8YjmX35du58AFKhSE3OO/HBV0QoXM=;
 b=glFg20pk+Ted/9NrSpbTfdKuYFlJFCKVLFeHlsxLZI9b2c90Gik1bFdddYEnIzQDeDlESFNhSiKdM9ox2fjdHp3G37uqbmQK/x/y4YWyXsZWZVHxsarS0z7YvnJKCJ5hWMjW9qD+8MfAhPOF7RVORWm41ukey3TW9JYMzuQzFL0=
Date: Fri, 24 Sep 2021 12:58:37 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew
 Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v2 01/18] AMD/IOMMU: have callers specify the target
 level for page table walks
Message-ID: <YU2vXU3UffSQ/iQX@MacBook-Air-de-Roger.local>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
 <57abd488-1fa1-8eae-66b9-196f32f7f4be@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <57abd488-1fa1-8eae-66b9-196f32f7f4be@suse.com>
X-ClientProxiedBy: LO2P265CA0409.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::13) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 16726403-a766-4491-8324-08d97f4a4650
X-MS-TrafficTypeDiagnostic: DM6PR03MB4841:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB48413071098E162063D9B4928FA49@DM6PR03MB4841.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: t75TIxo1YRrmX22yGU5+cQUC3zPV+T7Pj8CbBNl5FzTfqcqQeHxldUmpT36394ZELJ2lOz4idJYDc+wRDCDXPYsabqJCiyddYhsu+diuCudyocKZ5EB4oTrlCU4NKJyQ2tD67/gY5NIl+PU/tbdmf7Pu8wuPqVTgY7WPN8xKxMG5sWHGQXib0x18oW3f6JFNYeAdA7+Gm4vZZQrtx4XphxVaVR4/cP56mmBQ2Sgtwv+BPifPc+uB3CUgF4RQw6S2zVYkEGG9SjgBhQ7F2BnfmBJ9IUCkcMueMyAaQy8PKSvu9mdmtOeQvhaYO3952L0mGjMbquKD05wKP/4o9/I52//PV65toz64Q8EFE2mWxZG7rkF1w200/TtUzudN3YMbbwu0ywKl/kD2pqA15bqbfk7U+akSnSQ2UPIPUh+5cOeyhAtG9rPHicmawuVv5w5DPkylRgAtvEkyH5ryviZ/zT6p/isP4nM3axxK/yBOaoz3J3UZFnCrf+htw/NOKifF+Blp11avdxpj0K5+0w3hZiSQPXxhzl+Qn4HgNMez69S4TPahM3/wozsdrJ/JEq/Do9MctV/KxV61wd2ddskMRi4qfIDwhpnLg0S41dHuYMZmkg9Yer4+AqhEnJnNbwW6q6MUJ3OW8iW8LQQLU/0/ww==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(66476007)(6486002)(66946007)(66556008)(508600001)(9686003)(83380400001)(316002)(186003)(6496006)(38100700002)(54906003)(2906002)(5660300002)(85182001)(86362001)(956004)(6916009)(4326008)(8936002)(6666004)(8676002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cXU2aEpQa0dNRDVzc1dOUHowL3k2d2xhOE9TWDdxcG9CbDBQeitzaWhCcFFS?=
 =?utf-8?B?Y0RPRlhnSTZBdVRIcDZWU3I5cTlYVWtGV0Fxb0ZBdWsvbnBoMGJxUnJGejlS?=
 =?utf-8?B?K2NpczdiVTdMNWNrcjVONXVrWjlMOHJZTGJ2UHdSTWVvMHBqU0JxK2dJNGt4?=
 =?utf-8?B?VS9OQytEUmo5a2htWlhESnZ5NUNUS1p3YzNjZ3Uxc2JRZkFQLytkK0VKaWFL?=
 =?utf-8?B?YjNXME9uNThMamhlQmRLSXY2RzExamlmN25yUTFZVDNJOGUvVm4xVXkrcHVv?=
 =?utf-8?B?Z1didEJXTnRjWW9KbFU4MzZveGczS1ZWUmRzbkRkcG9Bc0g3UWs3YWxkTjhu?=
 =?utf-8?B?N1UwMEFJcGVLbmhhZUZlVTVYMUQ1NVY5cUE2YjA5RVBhbXRGWExXZVpLUjdr?=
 =?utf-8?B?eDl2OS9ERFVWRmRLTnlESmlMSVQ3TnpuTHdCaG9PNkp1N3B3VXEvSGFpVW4y?=
 =?utf-8?B?eVFUSWEyTGFzZGhSVlBMdUw2dUZqTUhWQmwvdDRpTWRXOE8yMHllWjZ5N2pC?=
 =?utf-8?B?Y0FDQXcwUFlsRTJDdzRGVXZMK2ZIQ1lKdzZaTzU0eTZTNmVCSlZYRzdtVlpS?=
 =?utf-8?B?VWZuMVJjWUJGdUhhYnFTbStUNll2OVJ3MFhrN0tWcDZNY2NlN2Fxblpqa2dE?=
 =?utf-8?B?aDlFTzk0d04ySWpLalVRd0Z4VlAyaHdEZkpnTGhBY0Qwdm92Nm9iMlRRWGdo?=
 =?utf-8?B?TCtxZU5GNXFNZW1ndzVrbnFCZnI2WjZXSlUvNzdmekM1WnEwcENLUFpYQy9Z?=
 =?utf-8?B?QlB0NU9Gd21CVE9oZ3hXa2NQU2RLcXJMVDBYREVnNDBxSTFTSStNbEJEOHBh?=
 =?utf-8?B?ZUdwMUhFb3NEWkl5L2dDK0JMYmlxVlFkUnpLdkJzd0VWU2ljVTZOeVFMWnli?=
 =?utf-8?B?VlYvNUVyT0cvWnU1bGl2VzFUSlpYTUV6NytmQkRnUTJqcklkeE9IaXhPNFk5?=
 =?utf-8?B?SnppSVl0UTlPMWxXU2lYRGdmcE1JM2tSR3FBRDBJSmFMY1NGQlVQSmRPOHBY?=
 =?utf-8?B?ZVpicmpjcHk5NlRXdVJlU0ppQ1lLZGhRckJ3YVdKdVczUG9sT0RmRmVWTFpP?=
 =?utf-8?B?WjdjbmllbTh3VmVMT1VId1VhUXZ5c2h1Qk5CQXhpcEtROWVTU3o0N2F0ZUZV?=
 =?utf-8?B?ZWRvOGU3OVNvRXhKMEtGQVhndHdKUitqVDRBYzU1R2IzY2ZzL0VKaVFsbWhz?=
 =?utf-8?B?eUZtWjdyQlVjT3F3VU5qeHp0a3Vkek9RYngzQ09xdWVkbzUvcTFaWldKYWx2?=
 =?utf-8?B?emdXSmlHRURHdDVNRDVweUJlVVB4TjNMajVzK0E3YW5TaWtGcDRHYUdjZno5?=
 =?utf-8?B?bXZOaTJoeVlrWFJaYUZ0MUpvU2d5OTFzVkdGZHR4cnc1dms2bjZMeWxGTWxi?=
 =?utf-8?B?clppUFl3OXpFRG1BUHRWWlUremVnMTRlb2xmRThZV2FVN0dQTWRKQmw2V1Nz?=
 =?utf-8?B?RU1yU2t1cUhJMUVyU2J5aGcycUkwc3NiSS82QzhwTG4vNTZSK0puVWZRQnQw?=
 =?utf-8?B?UWF5cTlwNjEwdjJvc1hIRTVYN1BXSVB5OFYzUk9uNjlqVlp4OXRBemp5UEoy?=
 =?utf-8?B?amloNE15MEdoRmU0eEdXb0t4cUhTbUVyUVBQSDQrTGlzY242V20rVWpwNXRJ?=
 =?utf-8?B?c0pQS253TDBRcG00cWh0Si9yUVF6NnBXUDdGT3dQVXl4THlJbUZqM1Y3NEwz?=
 =?utf-8?B?Wm0zb283aUFjNXdHS0FiZ2VEYm13dzBwWE5sS3p2UjB1S1ZOejBJeUtCVGUy?=
 =?utf-8?Q?6KSKc4ERmh70QjtCptVMU2BVD+N/hW87rOXFWvR?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 16726403-a766-4491-8324-08d97f4a4650
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 10:58:43.5775
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Q+KDbdT6tMWlTKo8uDmIV0sUMca4c0SVjSqqZ9O1FyVJRRKeutg/9e957h1VcaOrVYR2jZ4Z4/rN9MCYFRBD3A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4841
X-OriginatorOrg: citrix.com

On Fri, Sep 24, 2021 at 11:41:14AM +0200, Jan Beulich wrote:
> In order to be able to insert/remove super-pages we need to allow
> callers of the walking function to specify at which point to stop the
> walk. (For now at least gcc will instantiate just a variant of the
> function with the parameter eliminated, so effectively no change to
> generated code as far as the parameter addition goes.)
> 
> Instead of merely adjusting a BUG_ON() condition, convert it into an
> error return - there's no reason to crash the entire host in that case.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/xen/drivers/passthrough/amd/iommu_map.c
> +++ b/xen/drivers/passthrough/amd/iommu_map.c
> @@ -178,7 +178,8 @@ void __init iommu_dte_add_device_entry(s
>   * page tables.
>   */
>  static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
> -                              unsigned long *pt_mfn, bool map)
> +                              unsigned int target, unsigned long *pt_mfn,
> +                              bool map)
>  {
>      union amd_iommu_pte *pde, *next_table_vaddr;
>      unsigned long  next_table_mfn;
> @@ -189,7 +190,8 @@ static int iommu_pde_from_dfn(struct dom
>      table = hd->arch.amd.root_table;
>      level = hd->arch.amd.paging_mode;
>  
> -    BUG_ON( table == NULL || level < 1 || level > 6 );
> +    if ( !table || target < 1 || level < target || level > 6 )
> +        return 1;

I would consider adding an ASSERT_UNREACHABLE here, since there should
be no callers passing those parameters, and we shouldn't be
introducing new ones. Unless you believe there could be valid callers
passing level < target parameter.

>  
>      /*
>       * A frame number past what the current page tables can represent can't
> @@ -200,7 +202,7 @@ static int iommu_pde_from_dfn(struct dom
>  
>      next_table_mfn = mfn_x(page_to_mfn(table));
>  
> -    while ( level > 1 )
> +    while ( level > target )
>      {
>          unsigned int next_level = level - 1;

There's a comment at the bottom of iommu_pde_from_dfn that needs to be
adjusted to no longer explicitly mention level 1.

With that adjusted:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

FWIW, I always get confused with AMD and shadow code using level 1 to
denote the smaller page size level while Intel uses 0.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 11:05:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 11:05:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195311.347957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTj1Y-0001vZ-5y; Fri, 24 Sep 2021 11:05:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195311.347957; Fri, 24 Sep 2021 11:05:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTj1Y-0001vS-2q; Fri, 24 Sep 2021 11:05:48 +0000
Received: by outflank-mailman (input) for mailman id 195311;
 Fri, 24 Sep 2021 11:05:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cs4H=OO=citrix.com=Kevin.Stefanov@srs-us1.protection.inumbo.net>)
 id 1mTj1W-0001vM-U3
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 11:05:46 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5d6c40f0-1d27-11ec-bac0-12813bfff9fa;
 Fri, 24 Sep 2021 11:05:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d6c40f0-1d27-11ec-bac0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632481545;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=S/lyUAQUW4OkBE8yiINlYYf99gt7ImS4tR3CM7RQMgM=;
  b=TCXJuL5sgBJ/wwJjTOfwve0XnEykZpLlRrr6I0+Ym42Gfq7yGETXJZkY
   aSVh5v+Epuz6OGP46JKi7QYesUUFMmRV3cyDtrPbuJX8vo2toM3JzxErE
   jgVLbu/cvEob0ftC4AGX4UF5VLKNpO2rF2R79/6mpnnOVzU30E/k5GI9d
   Q=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: EIXpkoCH/Ri/lzEy6/7Z4GlyUaSrEh6IkhLDlVqAkqCtlMuKYkrvnz8TpOaKn44nggwoNDxZH7
 aWKsGqsK68LUNslZDPakfT2+J+Q79/GnuCHjTk/uGpVyeUQ9/1anXrOV+xg8jMyobF6WCMZCje
 zPLWlQUXGnjsS4yp9+YfzeFsMFHMMg+ZzFxjj+LwOjhKIk0nKHlLgnqP4xcBp2daRoeFyenGQb
 6K/sqmsN6PL0cTMbH2z6tF97bG086HoRe9ls4mGwCS544NMkGpJYxYXKOrD6kJ9mFiHL4Rj6mk
 gqE25Q1QwX3PwQcTgn8qyiMJ
X-SBRS: 5.1
X-MesageID: 53524765
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:0iXL/6s+e2T67fBZ92HNR3WTg+fnVJlZMUV32f8akzHdYApBsoF/q
 tZmKW7SOKmNYjP8ftB+aIq/oBsAvJXTzoM2GgRprH9gRSwU+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524ThWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NplsqS+WDwGOKvwwe09WF58SAdhFo1k5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DH44Bu3cm9i3UA/8gRo7rSKTW/95Imjw3g6iiGN6DP
 JdDNWUxMXwsZTVMJEVIUbkHxtvroXKmKBBH+FSyp404tj27IAtZj+G2bYu9lsaxbdpRtlaVo
 CTB5WuRKhAFNvSPxDyd6HWui+TT2yThV+o6BLC+s/JnnlCX7mgSEwENE0u2p+GjjUyzUM4ZL
 FYbkhfCtoBrqhbtFIOkGUTl/jjU5XbwRua8DcU90A+c8YXS5TrAD0gPCRoaTsMNlO0pEGlCO
 kCyo/vlAjlmsbuwQH2b96uJoT7aBRX5PVPudgdfElBYuYeLTJUby0uVF4c+TPfdYsjdRGmoq
 w1muhTSkFn6YSQj7Ky94VmPqDalvJGhouUdt1iPAzrNAu+UYueYi22UBbrzsaoowGWxFADpU
 J04dy62trtm4XalznDlfQn1NOv1j8tpyRWF6bKVI7Ev9i6251modp1K7Td1KS9Ba5hfI2OzO
 hKM5F8Nvve/2UdGi4ctOOpd7OxwkcDd+SnNDKiIPrKinLArHON4wM2eTRHJhD28+KTduao+J
 Y2aYa6R4YUyU8xaIM6Nb75Fi9cDn3lmrUuKHMyT50n3gNK2OS/OIZ9YYQTmUwzMxP7dyOkj2
 40EbJXiJtQ2eLCWXxQ7BqZJdg1VcShnW8qmwyGVH8baSjdb9KgaI6e56dscl0ZNxsy5T8/Eo
 SOwXFF20l36iSGVIAmGcCk7OrjuQYx+vTQwOil1ZQSk3H0qYICO6qYDdsRoIel7pbI7lfMkH
 eMYf8igA+hUTmiV8ToqcpSg/pdpcw6mhFzSMnP9MiQ/ZZNpWyfA5sTgIln07CALAyfu7Zk+r
 rSs2xn1W50GQwg+Xs/aZOj2lwG6vGQHmfI0VEzNe4EBdELp+YlsCirwkv5ofJ1cdUSdnmOXj
 l/EDw0ZqO/Bp54O3OPI3a3U/Z20F+ZeH1ZBGzWJ57iBKiSHrHGoxpVNUbjUcGmFBn/04qire
 c5c0+r4bK8chF9PvodxT+RrwKY564e9rrNW1F05TnDCblDtAbJ8OHiWm8JIs/QVlLNevAK3X
 GOJ+8VbZurVaJ+0TgZJKVp3dPmH2NEVhiLWvKY8L0jN7SNq+KaKDBdJNB6WhS0BdLZ4PevJG
 wv6VBL6P+BnticXDw==
IronPort-HdrOrdr: A9a23:lji7Kq8kmCLZ7ACUje9uk+DWI+orL9Y04lQ7vn2YSXRuE/Bw8P
 re+8jztCWE8Qr5N0tQ+uxoVJPufZq+z+8Q3WByB8bBYOCOggLBR+sOgbcKqweQfREWndQ86U
 4PScZD4aXLfD1Hsfo=
X-IronPort-AV: E=Sophos;i="5.85,319,1624334400"; 
   d="scan'208";a="53524765"
From: Kevin Stefanov <kevin.stefanov@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Kevin Stefanov <kevin.stefanov@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/libxl: Remove page_size and page_shift from struct libxl_acpi_ctxt
Date: Fri, 24 Sep 2021 12:05:00 +0100
Message-ID: <20210924110500.25412-1-kevin.stefanov@citrix.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

As a result of recent work, two members of struct libxl_acpi_ctxt were
left with only one user. Thus, it becomes illogical for them to be
members of the struct at all.

Drop the two struct members and instead let the only function using
them have them as local variables.

Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <iwj@xenproject.org>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libs/light/libxl_x86_acpi.c | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 57a6b63790..68902e7809 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -25,9 +25,6 @@
 struct libxl_acpi_ctxt {
     struct acpi_ctxt c;
 
-    unsigned int page_size;
-    unsigned int page_shift;
-
     /* Memory allocator */
     unsigned long guest_start;
     unsigned long guest_curr;
@@ -159,12 +156,13 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     struct acpi_config config = {0};
     struct libxl_acpi_ctxt libxl_ctxt;
     int rc = 0, acpi_pages_num;
+    unsigned int page_size, page_shift;
 
     if (b_info->type != LIBXL_DOMAIN_TYPE_PVH)
         goto out;
 
-    libxl_ctxt.page_size = XC_DOM_PAGE_SIZE(dom);
-    libxl_ctxt.page_shift =  XC_DOM_PAGE_SHIFT(dom);
+    page_size = XC_DOM_PAGE_SIZE(dom);
+    page_shift = XC_DOM_PAGE_SHIFT(dom);
 
     libxl_ctxt.c.mem_ops.alloc = mem_alloc;
     libxl_ctxt.c.mem_ops.v2p = virt_to_phys;
@@ -176,20 +174,19 @@ int libxl__dom_load_acpi(libxl__gc *gc,
         goto out;
     }
 
-    config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
-    config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
+    config.rsdp = (unsigned long)libxl__malloc(gc, page_size);
+    config.infop = (unsigned long)libxl__malloc(gc, page_size);
     /* Pages to hold ACPI tables */
-    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES *
-                                   libxl_ctxt.page_size);
+    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES * page_size);
 
     /*
      * Set up allocator memory.
      * Start next to acpi_info page to avoid fracturing e820.
      */
     libxl_ctxt.guest_start = libxl_ctxt.guest_curr = libxl_ctxt.guest_end =
-        ACPI_INFO_PHYSICAL_ADDRESS + libxl_ctxt.page_size;
+        ACPI_INFO_PHYSICAL_ADDRESS + page_size;
 
-    libxl_ctxt.guest_end += NUM_ACPI_PAGES * libxl_ctxt.page_size;
+    libxl_ctxt.guest_end += NUM_ACPI_PAGES * page_size;
 
     /* Build the tables. */
     rc = acpi_build_tables(&libxl_ctxt.c, &config);
@@ -199,8 +196,8 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     }
 
     /* Calculate how many pages are needed for the tables. */
-    acpi_pages_num = (ALIGN(libxl_ctxt.guest_curr, libxl_ctxt.page_size) -
-                      libxl_ctxt.guest_start) >> libxl_ctxt.page_shift;
+    acpi_pages_num = (ALIGN(libxl_ctxt.guest_curr, page_size) -
+                      libxl_ctxt.guest_start) >> page_shift;
 
     dom->acpi_modules[0].data = (void *)config.rsdp;
     dom->acpi_modules[0].length = 64;
@@ -212,7 +209,7 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     if (strcmp(xc_dom_guest_os(dom), "linux") ||
         xc_dom_feature_get(dom, XENFEAT_linux_rsdp_unrestricted))
         dom->acpi_modules[0].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS +
-            (1 + acpi_pages_num) * libxl_ctxt.page_size;
+            (1 + acpi_pages_num) * page_size;
     else
         dom->acpi_modules[0].guest_addr_out = 0x100000 - 64;
 
@@ -221,9 +218,9 @@ int libxl__dom_load_acpi(libxl__gc *gc,
     dom->acpi_modules[1].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS;
 
     dom->acpi_modules[2].data = libxl_ctxt.buf;
-    dom->acpi_modules[2].length = acpi_pages_num  << libxl_ctxt.page_shift;
+    dom->acpi_modules[2].length = acpi_pages_num << page_shift;
     dom->acpi_modules[2].guest_addr_out = ACPI_INFO_PHYSICAL_ADDRESS +
-        libxl_ctxt.page_size;
+        page_size;
 
 out:
     return rc;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 11:31:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 11:31:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195319.347967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjQo-00060i-6j; Fri, 24 Sep 2021 11:31:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195319.347967; Fri, 24 Sep 2021 11:31:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjQo-00060b-3W; Fri, 24 Sep 2021 11:31:54 +0000
Received: by outflank-mailman (input) for mailman id 195319;
 Fri, 24 Sep 2021 11:31:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTjQm-00060C-Di
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 11:31:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0d47ac62-699f-47d6-aa13-97e98398c8bd;
 Fri, 24 Sep 2021 11:31:51 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-BEwFl4_4OHaRNGT3NKMDDA-1; Fri, 24 Sep 2021 13:31:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 11:31:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 11:31:46 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0198.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1f::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 11:31:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d47ac62-699f-47d6-aa13-97e98398c8bd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632483110;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=IirCkd4B5oXhSrzUbNH52RfizmlgngvG0g3h4esKx6E=;
	b=n8mxWnHH7BpaAejfg5kQJRzGu0gBcwV+3RhZy2CneOV3oOz8gB0OUvLKmM++JF/NBI11vD
	SrbtV1e1nwiCnDD1ISngfwRxk8JafyulmAnhd9yOeMHmyHCUd4UUFk8Na4qdhxWXT89Z1v
	0DZuK3HiVmV0v8ifMbHwvaSmMkJSaoo=
X-MC-Unique: BEwFl4_4OHaRNGT3NKMDDA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iobeUcx6eC3oayB70qhdNpEo3HJZ991VhZVmatqSbSSRURPSN2srMR87a7DTvzIkIpCvVOjPKc/dcdk9rS1enq8JrlXKtjfRt1XQ4lIxKf9Rk0dWMSpS+2QISt7NAPgeQ1VtERl+0iegpTLuUBwAKQCf4bbeIbocmexietvMai/r7JDrkBNMdqRqT+0aZTs/WTIfJ5S4pC/WfnMnEf4JeUlhNqZZ2rzCpTe6JXcZdVVrxA4EGPDCBh1fZ7qZuoR3598Mq6diREqE1CYLJJgH+jqodeP0VpUgvwa5A5b5s/lwAz6B8oLtnDNckCl+q/a2OyrkcHQ/w7BjBa7zqpwwiA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IirCkd4B5oXhSrzUbNH52RfizmlgngvG0g3h4esKx6E=;
 b=c4A1nnU7FxoBcdY8lb8HZrmLdONKglRmcSna9wALqOXAqW5AN2iwM2GOVnrMqUae6DgreBcUZkPcmVc9HZOoiI1WkLneSwcRBgtHqlYfPSjZY6inFIMzRavWjhws/G5iZ+QzVY4JsrufMQRmWSv6AuJRep1gBYh/gAi+b8JnGpQ6DfoW7aZ+0L+FSjVXT50nfzYDdjeDtlgRxPwocMR/ZKlWMTEn8qTZwHiWPmJQktbzd0iCXZ9RmZ0u2FXofvwGK9f3Bq7FddZoEqlUNsIh3NOOlltPDNrV7H2ePJ2D0u2IwPyljdsMMLP8lMUnb3dphHfXu/Vyovf0YMRoSEDKvQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Tim Deegan <tim@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: sh_unshadow_for_p2m_change() vs p2m_set_entry()
Message-ID: <ae8b6cf1-d1ae-8a55-7b88-7ef24db0c554@suse.com>
Date: Fri, 24 Sep 2021 13:31:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0198.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1f::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fb1083d1-907a-43d1-fa95-08d97f4ee468
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2606F90EB00CCA81602A241BB3A49@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8CY7/98DhsJMx49nB3ZJNhKlZNTQ+N1Mi9oTQ4gb7eWxoP1klbAqS2pqQUw+6FWdZOru1kBU4WgCbQPwl/t4wloOFCUZCALfBGBcwd16DarTyC9lT0N8mtRqP7oCSoYfnNa6+7CbKWCd/X6Z+MWbKAmv/mBB+PYIOubyHfkr5CrqcnAkXjZzWm6g7AnSJ4BUsKxUZ0GVoasWS1AHzO3x4D5ZI2B10FOnp/W4zzur2z5TjT451xCseQwbvl9Ni7LfrRTediGpZD/pCK8T/crizr10iNFwIvStbiSBhXvao5aITcBi2CG8iP/iQ1NaqBReT36zo+IzkKBqSX4bEhotMmm8RiuHOWu6bQCWVSCdnGX0DmMZCxTQRU371Ba/V7whHZYlqljmDvz0fvKX7gLKvnonJK4ylAZRwcccrJD58qFcRluIf1hJDWzDgGaoCGUPCYQDdpl2rtcacJ9SQHsdaSkB1gtCHEJorT+HLGdQwpwo9N9zQIam+iR5lnwQsuzehl6sTHfJpjM5SxjPPhW+Cw5ffjcoozt2tGmmjtpfsv2EJjTaUqjWZhQs/Ite5Eay0dIsXx5eK+pps2LATN99/k8Gz5UiSZoQeATFa/E9MkseTVhyq79AjYg7oGVo7P4VPvyybMvGk7RK8zWAe0xHOJGGBrgAg4SXtv1OiT9jEUtYsUdAABYVh+o2oY3RZGCVke3AzRtwg193RpQo2iCXd8w3MkGB2UBKgIe4BgMRb3xV+OB5tZGYbzUL88V1EJiA
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(86362001)(31696002)(6486002)(2616005)(508600001)(6916009)(956004)(38100700002)(8936002)(2906002)(5660300002)(83380400001)(4326008)(26005)(66946007)(36756003)(66556008)(66476007)(186003)(316002)(31686004)(16576012)(70780200001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L09vZTV5QTd3ZHRnU0VYR0w0eHVUWkRld3YrQUNsazJXeVZJUzlNU0dOc0VX?=
 =?utf-8?B?Nm1NenB4V01nZ3JWMi8rREszaXNWY3B0QzhvSVJHWHBkUGFrSFBET3YvaUdl?=
 =?utf-8?B?LzBHZi9XNkI3TEl3YWZYaXVmY3VmYzBIU0JHWnIyV3FmaWNNTFZKdCtiSkE5?=
 =?utf-8?B?c0ZwU1ZNT2VGeHNPQUpMZzM3V2Zwc0NFUDU3MTlIRE44aVM4Y29tRzhJZjVs?=
 =?utf-8?B?R0RuQUVvME1tWjFrUk1OTDZQd1RMbUJJUWFaODk2OHg0YUtxdnZ4M1dzZzY2?=
 =?utf-8?B?aFk0bGJuMzZMZkVhK2xmdVRHQ0lYNlNrQjRqSXJHaDAyMzYvQnR3ZGhuZ1RR?=
 =?utf-8?B?a0FWRVVTZG5lN05TVDlPamQzUUt5WmFqV2k4Y2pyQWF5eDRQQlN3UDdjYUVL?=
 =?utf-8?B?SExrSTIxcGxiODhmMVE3dDNDb0g1OFhObm42NWdwelk4Y2NQcUxoOEdhcXV1?=
 =?utf-8?B?TVV6TkJETjlQdU1wM3VneDQ1d2VxRTNsVDNNT3hOVEFiYURyUTRLK1IzYWNL?=
 =?utf-8?B?RkRIY1ZrM1hISDh0bTBSRGRmeTRWeDV2VGpXY01XOEV4VGhBVkhjYTJoVjc0?=
 =?utf-8?B?VHl2YWlFRG1MQmpSaXRNQW1xbjVzTEtRckFwU2RIQlRlMTJONlV4OFBLVjVI?=
 =?utf-8?B?U1hhcXBBbFpTWXVEeWIyZGFrTSsvY3BRVkhJK0pkWGUzcEQvZHFZaEkyTXFQ?=
 =?utf-8?B?dTZrWGxicFBNbndqRzRmdkZqbks2YXM1M05GSWdjdTBabXp5U29sUXFQYk9E?=
 =?utf-8?B?S1cxOFYyelkrVXlaWDdoNDdVc1FaTHQzOXc1d2xRSmwza0tDVFZpckpUSzlL?=
 =?utf-8?B?UnVhcy9xQnFjRTAybWZ3NXpQYXV4ejFCQVNtdjU2Z1RiOS9MU1A1RWZXYnV1?=
 =?utf-8?B?REgxSnJCVnFoNE1sd0g5cW5hUnFtN0RvQ1o0YVBTTFFia1hmaHBmaFMxTjYv?=
 =?utf-8?B?R29UeGN0Um9PbFgwTGtnWGZWdE1HazRqSUZvNUp5K20wUjNtakFENzVUN1FV?=
 =?utf-8?B?QzFWUGVYQkRpVnVKdzV1MjRncVkrcGVqOTdqOTZFVWJQeFZmSzZ6Sm4rWGRo?=
 =?utf-8?B?UjZwNTFSWHBFRkVOaFFjSUpjWGNRTGZ5ZWwyY25sQ0liUEtaaXBia0xxZFZt?=
 =?utf-8?B?dllIajd2a1YzRjZ4VFd3dklaUElPQWNWdDhJZXR0MXE5clNhcnJEYlkyekFw?=
 =?utf-8?B?SUl4Z3czV0w1bnZ5MXBCRzhobGF3Zmc3bHNqZW5RZlRSU3h4WlZaMVR2MU1i?=
 =?utf-8?B?SkpualFoNklJRTlNNWhJNk1oUW1NR2xlaldjWnJQNlREUlRLTVJhQ2RscmZv?=
 =?utf-8?B?UTBzSVNFQTBQem9qNTE2UGYyM2taVWNOODNaejVEV2pFYzBJczJhYm1JdUE2?=
 =?utf-8?B?YmRBbkNRU0pOTWdhd0lOdmYwbGttTEI0ZTVWU00vZHJFUmpFM09WaHRvODhU?=
 =?utf-8?B?LzNybjg0ZXZhdHkvaVliaFcvM3lMcWtaSmdvRkRlamc0anoxamlmTGhCZW02?=
 =?utf-8?B?N1JXU1BoNzdEY3FGZGJKRkdyeFRlb2NNK29tMGphTHc1NUV1YTRCQk10MjZw?=
 =?utf-8?B?NHpZQWUvSUFZU0hPUVhwcEs2ZmNESE5GTGs3b01YcHN3eVpQZU5WZUVUQXp6?=
 =?utf-8?B?TFdmSUxHWFFhT0Z6YkFWSDRyWWxUSkZleEQzOGliV2FsZ1BQMDlueThxajlC?=
 =?utf-8?B?NVJkeFBZT0FDWTlzZmdpMTFJTzZjN2VZRDRveDN5TkdPQlFDS1gxV0I5aDJE?=
 =?utf-8?Q?QpbQ0AeGzMAPmjK33xq/9r4fsdsdNJSDlzf0Gpo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fb1083d1-907a-43d1-fa95-08d97f4ee468
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 11:31:46.7166
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PKWlEoX2KczHtEz7tu0X7iFHPbhTv1JCG98p+DUTfva/1o9ES579FGtKHGUFJNBODCryW9dHjimNFIkTySzCOw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

Tim,

I'm afraid you're still my best guess to hopefully get an insight
on issues like this one.

While doing IOMMU superpage work I was, just in the background,
considering in how far the superpage re-coalescing to be used there
couldn't be re-used for P2M / EPT / NPT. Which got me to think about
shadow mode's using of p2m-pt.c: That's purely software use of the
tables in that case, isn't it? In which case hardware support for
superpages shouldn't matter at all.

The only place where I could spot that P2M superpages would actually
make a difference to shadow code was sh_unshadow_for_p2m_change().
That one appears to have been dealing with 2M pages (more below)
already at the time of 0ca1669871f8a ("P2M: check whether hap mode
is enabled before using 2mb pages"), so I wonder what "potential
errors when hap is disabled" this commit's description might be
talking about. (Really, if it's software use of the tables only, in
principle even 512Gb superpages could be made use of there. But of
course sh_unshadow_for_p2m_change() wouldn't really like this, just
like it doesn't like 1Gb pages. So that's purely a theoretical
consideration.)

As to sh_unshadow_for_p2m_change()'s readiness for at least 2Mb
pages: The 4k page handling there differs from the 2M one primarily
in the p2m type checks: "p2m_is_valid(...) || p2m_is_grant(...)"
vs "p2m_is_valid(...)" plus later "!p2m_is_ram(...)", the first
three acting on the type taken from the old PTE, while the latter
acts on the type in the new (split) PTEs. Shouldn't the exact same
checks be used in both cases (less the p2m_is_grant() check which
can't be true for superpages)? IOW isn't !p2m_is_ram() at least
superfluous (and perhaps further redundant with the subsequent
!mfn_eq(l1e_get_mfn(npte[i]), omfn))? Instead I'm missing an
entry-is-present check, without which l1e_get_mfn(npte[i]) looks
risky at best. Or is p2m_is_ram() considered a sufficient
replacement, making assumptions on the behavior of a lot of other
code?

The 2M logic also first checks _PAGE_PRESENT (and _PAGE_PSE), while
the 4k logic appears to infer that the old page was present from
p2m_is_{valid,grant}().

And isn't this 2M page handling code (because of the commit pointed
at above) dead right now anyway? If not, where would P2M superpages
come from?

Thanks much,
Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 11:38:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 11:38:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195325.347979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjWk-000714-Rz; Fri, 24 Sep 2021 11:38:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195325.347979; Fri, 24 Sep 2021 11:38:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjWk-00070x-Oz; Fri, 24 Sep 2021 11:38:02 +0000
Received: by outflank-mailman (input) for mailman id 195325;
 Fri, 24 Sep 2021 11:38:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTjWi-00070r-Sj
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 11:38:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 87dce733-ed67-4177-b44a-08542309bee1;
 Fri, 24 Sep 2021 11:37:59 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-VgvvKEGoMpWqXG4iDbmBgg-1; Fri, 24 Sep 2021 13:37:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5742.eurprd04.prod.outlook.com (2603:10a6:803:e5::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 11:37:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 11:37:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 11:37:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87dce733-ed67-4177-b44a-08542309bee1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632483478;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=x2sBTeOOjlsr88v/K89sm3OS2YnDx3P3vKYIUokmi48=;
	b=fWeDJ4bF3q1mzDeQzTlnlH4OvNJZTkfA6+17cSqpcFJJS4LyNBmsr2IsX/16uxSI6A4t3Y
	AEIjgM1NYTKxEcGtcwBzhsSilvIUhhdgs1Sh5Oi4wqQ5jGQfpDQzv4X17+BfUrPI6M77wj
	+mE4syPCBXoPz4xhswdmkmuim69gsJU=
X-MC-Unique: VgvvKEGoMpWqXG4iDbmBgg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SqwJ0VsO+PwTff6owfUfDq9QyRBJrcmRMIzH3SCgOOxBOqiPNz8khhmsLkxi7y5txZL5xcRDj0e/fyu+8beYInWZfzg8+zuCNj0w2zsUa32mIlpYyv/Faz6OkR6HWjYI6mqvEOl6LEqZTGYQs1xKMWYW8vuQZSVAxfD1DOWr1Jj6AhtsqpsHOljgBMZ7MWn4mzHLHrCdUzHyUD3VMf790CjpUioYSZS2Gno7CiF/asI9q6Ov5crbclQI+flvTnYdpr0qiAKVhMLx/wy4aLJgroRM+k7pBrNMYK4PW59ifHFzlL3LCzrmss4mGVgoptOU+yhGHdoc1W+gLGLXqSyFuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=x2sBTeOOjlsr88v/K89sm3OS2YnDx3P3vKYIUokmi48=;
 b=ajS+qonkuLHlE5bK93nQ0K3Kv6JB/npJoujYIhQBKz5nnJ8e9Zsv2Y+r31f1L+p3IGzXcLlP1D+6jxYdg84XF2LOdy/1vlQHsmz23i/9Ks6JrIWZQ363Xr97zcLqrQXbKMHChzJj1DeEIEg8M0bTepMRcwxYGzpOHEthDxWZ652LughN3cCcpps6OKU3UjopVi5XaMgK0uSSD/scICBYnXou0sXheKnq8YhQIt2qasw6nnf9fEpNgMrycSY6jMt7XBqBztxh13TpxiNs26gseUivu92HkiqrXGxSTzga86kmsaoWgaZZUhDMUBoKMoVjzTYmhGZ379R+3GTGD9f5bA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] tools/libxl: Remove page_size and page_shift from struct
 libxl_acpi_ctxt
To: Kevin Stefanov <kevin.stefanov@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210924110500.25412-1-kevin.stefanov@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e3d86499-3c1e-6cac-9225-f64b101c5478@suse.com>
Date: Fri, 24 Sep 2021 13:37:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210924110500.25412-1-kevin.stefanov@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0089.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 521ff169-e0da-4135-6ae8-08d97f4fc08f
X-MS-TrafficTypeDiagnostic: VI1PR04MB5742:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5742D55108E9960368F8FD8DB3A49@VI1PR04MB5742.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	v21MI/IJPJEvoCML1DAZUrw7BXTMZM8gGi7T+4qh2BXhZN92BKt751WMY/IL9JQQ8BWvrpQriT66NQnOgkWL6TxMqY3XkhRVNADN13nRLa+tOnNOdN/7gDrCV0Stxq47jGfQ/WdgZtrUFYfeUP1ovddSxWp8kncEZHC+RnolauurjyzK8OjbzjQmULT1DwiX3mUG9HLLsZl6qkpSX7sKax4ZSJYctQl06fv5i2b7Evd+vmmn1nolc+vdMA5ykRM0eziGPXPHnx7++fZgNebD4QDikaalXZ/B9zE+ZC6LPQSeXlHgDmSM0gWgFHIHDOjvMTcFrbJu7vQUB5M8utLntC7nesEsKS9C6W/nrfiGFf+88dtwQWGe2Zb/M95/iEKBNO35RZMLxjxjxj/QDaYK2f6uWP6oGfarhaU5UGxgdWS9Js9K3XoMOZPP7j86W4w4/oswvJfvhJKn+8lynL+J6xhDX3U4dLbiYwd2EyOFrAPXIR8HnWFgbldnyDzTk84KBh6lnJJvKFK/K1wGaaMB19qoO+0mxDGIMpljvzZl0UkeSNzt2e7Brpj1LMA/OG/EQcYtTVdqR9h0FVfzX4Nms9upEmaf52c3hN051QzrDiH1IHRHBNWcwMWoOi9LKGhJTXMVzPYpxTubu8VcbvgaJQl2Ja1ajkgpTb0w93jGyjOU9C8/b4Gs0Us1/qvrH9LqIJVQ6JPfqSuFluiZte4f3aA1nq0Z0y1DcNIbfhHOExI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31686004)(66476007)(66556008)(6916009)(53546011)(8676002)(38100700002)(956004)(86362001)(26005)(66946007)(54906003)(186003)(316002)(6486002)(2906002)(5660300002)(508600001)(36756003)(4326008)(31696002)(16576012)(8936002)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SkR6Umh3anFkZVB6U2xaTjdma3hTS2kzNlhPUlVGSjFYTnBRZTBsbzBpTHVJ?=
 =?utf-8?B?UGpGa2Q2ZGV1UlNjTHQzVHpvNXpNVTFrcTFQenRDWER3WCtSSU1WaVpxM0FZ?=
 =?utf-8?B?RlZET05SaGZDTTRERDRsZFA5a1NDaTEvMjZCWnVzVFhQazZNWnlWbjM2Vm02?=
 =?utf-8?B?SlZJendXdmNFSEMzRkdGZVR6N0VZMGE2dHFBTExZcE0yY1JndWhmNno1QThX?=
 =?utf-8?B?TkIvbGhBandheW54YjdNVm9FVUxLNkZFRS9KdENBMFBSMUZ5TytEYjFIYkhW?=
 =?utf-8?B?eE16VjVQYlJnUmFUMnY1aGhIT3poTFpTbzd1ZGVHR0dHamU2c3E3UEx4ZDdU?=
 =?utf-8?B?ZzRRNWpYRCsxUVhhNmp4ODFRTHFsN09ZUGNxb3BVdE8ybXFYK2ZORkJGL3ZX?=
 =?utf-8?B?dnQyb1BGMlVPQzdpeTVHU0FvU09McUhZZU83SFNmc3FjcGxiSmdZZnYwU2dR?=
 =?utf-8?B?UW1iT0lTbGZxcDBORllJVmw0Und5cERUUVJDSlhQK2ZxUytzcmt6ZXJFd243?=
 =?utf-8?B?TVZDT2dtSjljT3VkZVBFMlVSdVU2MVhjTEtZU3A4N095R3BvQjFBY2s3bk1N?=
 =?utf-8?B?T2c2elY4c2hITHZRTEdKNVlwSUpTYlFOaU5YbXpubCtndUxzN25uTDd4d3Fv?=
 =?utf-8?B?ZkpXa2ZYTXdHUDlxZnlnbE0vN2NUWmlEVC9zVFBoSTNtUldEMlBqZkdSakNO?=
 =?utf-8?B?bGt0TlVxalkvRHNuV290ZWQwZDlQQ25NZk1Qbk4zZUFSUVBkUTJSdEZpU1M5?=
 =?utf-8?B?MUQzNmZsQW5xVVl4bVZleWNGeE5DWnptbjlSeEo0clhNUWhoLzlkN0tFd0Zz?=
 =?utf-8?B?NG9BR2dHSkpNS3BRdWVCbHdDTzJ2OVkrMmR6TjZ6REV0UCtiaVVLd0E2dEF1?=
 =?utf-8?B?eHU0QTMvaEsvUmxvSi9NVHBRN0d4Q1lIOStYNEM3MkkyRzhpdFdlWGNOWExL?=
 =?utf-8?B?U0xGcE9LcFBuajVEWnpHMFhZSmZBTHFDOUVObUhPMzFsNXM3VUM5LyttS3Jj?=
 =?utf-8?B?REFkWkEwOUZabE8wMU9NNDVBcXpvVDNqRWxzZUx5dWJmTWxmZ0pGVzBTcTh0?=
 =?utf-8?B?V3U5c3VnczJqZSsrdnUzbmpoUldabHBHeEo5U3M2bm1SQzdiZG13bWZWcWRU?=
 =?utf-8?B?dUw3RWorcDBmdXpMS2M3MEQ5ZTF1OHVpMXVHM3BNNXdrSGkwSVF5WGZzSEdn?=
 =?utf-8?B?VjFRSjZrZzZnMUVYeS9JOGszM0o0WkNjaTJBYzY0SWU2Ly9iVkNkbWpKUjRv?=
 =?utf-8?B?VTVhSGFZL3phN3o5dVNFMHBsMzMzT2tZSnFkUEZXVzNuTjMxYWVUbVBiNjlz?=
 =?utf-8?B?bVpzOHdleVNTU3hhNEk5TDYwQU1qK0w1MElPRXdoYTdqM2x0NzkybkhZZW0w?=
 =?utf-8?B?a1o5b2hxanFXYUZUSGxZU1VXUFFMUU5RQUl4NmxWM0lwSHNNTTREZzUyaElI?=
 =?utf-8?B?STlGUnBUOTh2RGhnaVlLY1Bubzc3QVQ4ZEkzU1V3TjFDZml5N25DeHlsZElR?=
 =?utf-8?B?Mm52M1Zic0FrSDI3dlFmY1l4cG45eDNqWVZhWHpRWXlWY05KZjJIeVhQS1U1?=
 =?utf-8?B?ZHJtaXBxMGk0NkFkcGM2WEk5KzJhYURnaXZobDM4WU1USzdGZlJkZ25odUNR?=
 =?utf-8?B?N0x2ZTVrWTZTeXlEMm5aK3ltZWlIZ2EzRFl5cVdXdEhBeEt3VFBFOTdCTk4v?=
 =?utf-8?B?UnBzeXpkVmJ1T0lpekwxTU5IM3ZhMks5NVRCKzRoWVBPK0lWVnNrMmlTTE56?=
 =?utf-8?Q?vPP4uWj/WyLpYW5bzCcpZi8AsnwzDgFpBQvxsP5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 521ff169-e0da-4135-6ae8-08d97f4fc08f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 11:37:56.0753
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: o7Q+jGnzXgYO6WX2QqUwvRA5PARQ45tEM1s/FwvpF3YsDkxuLKy7i1hToUF4s1hT09aw9UptTq44ZKv/QT4hmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5742

On 24.09.2021 13:05, Kevin Stefanov wrote:
> As a result of recent work, two members of struct libxl_acpi_ctxt were
> left with only one user. Thus, it becomes illogical for them to be
> members of the struct at all.
> 
> Drop the two struct members and instead let the only function using
> them have them as local variables.
> 
> Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

I would like to suggest though to consider ...

> @@ -176,20 +174,19 @@ int libxl__dom_load_acpi(libxl__gc *gc,
>          goto out;
>      }
>  
> -    config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
> -    config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
> +    config.rsdp = (unsigned long)libxl__malloc(gc, page_size);
> +    config.infop = (unsigned long)libxl__malloc(gc, page_size);
>      /* Pages to hold ACPI tables */
> -    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES *
> -                                   libxl_ctxt.page_size);
> +    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES * page_size);

... using page_shift to replace all multiplications like the one here
at this occasion.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 11:38:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 11:38:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195330.347989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjXB-0007ZT-8b; Fri, 24 Sep 2021 11:38:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195330.347989; Fri, 24 Sep 2021 11:38:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjXB-0007ZM-5W; Fri, 24 Sep 2021 11:38:29 +0000
Received: by outflank-mailman (input) for mailman id 195330;
 Fri, 24 Sep 2021 11:38:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Coby=OO=epam.com=prvs=9901c588ba=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mTjXA-0007ZA-Il
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 11:38:28 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eedd39fa-1d2b-11ec-bac2-12813bfff9fa;
 Fri, 24 Sep 2021 11:38:27 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18OAO96Z019791; 
 Fri, 24 Sep 2021 11:38:26 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 by mx0a-0039f301.pphosted.com with ESMTP id 3b9cvy8e3q-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 24 Sep 2021 11:38:25 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5939.eurprd03.prod.outlook.com (2603:10a6:208:15e::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 11:38:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 11:38:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eedd39fa-1d2b-11ec-bac2-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SpNYp0N5ejI+ACQ5gdG3WqDY116mkchXXi4mtqmP9YDrDfabvatVnRAqd3g+2CfvvaG3gzdz0hPS8GMuOTG5ZNReMo9llErhWYIEZ1zwQ60IFN9UjmauO/WabCqmNvbzta5rx7sWwrqAvrqs3JSeuZK6sdGfBai5Hnk8zlrH64DWKITipDCTvhJP61vBu6h5JTwZOmuImXHA8VIBfCHxiKrOzidAWo6HkzgA4VE9kBhHoI3HyXx/B83BH55tMYbG5ZJZLP5kMM9BpeREdh0HP7MsPB8PwfHFVMDNu4AZyfHlUCXaiXRET1z4zU6OtR/erUvrWM00P7tqKYyWqCNlDA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IOmFWBEPT++RtLjxPBdbE8hk0xmS1z+BauBEJuz3d40=;
 b=WSOmVbdeOk3sSmGkYwWheHNEgNiHRrqUSYL93YubqiNiAbERae8cx2NOXNbhqzOk/ExikaE0v0AOiHEcPIEplUq1hylXqFBkHXPviTQWp2YXh/cOl3TYynusdpsVXbWI3KIqno1SiXVX/OHFaPBSuTu+w5PInMDheIdDSC+FS8Y2w964EIrhIP7AlkJ5PjyHE2d3B5q0BpHWrwlTf2SNvPuQsej1LCXq9D+hO7Hj7Ac63NvnD2uNgS8QztdiTAf8BR209KEp2MrC+JV2RnxBwVYU7ssEpmdOeQwS8VVJRaF8wqo8M4MD0kG371Jiql78dw1g7HDhRruYFgEE5zJQKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IOmFWBEPT++RtLjxPBdbE8hk0xmS1z+BauBEJuz3d40=;
 b=2JrtRI2eV5kn0yKO98C3iLKblqps0BDZSeX/2c3Zp3g3MEqWtQhiuB63E2PGRsu011cVrw/iG5CIw3L9Gsbj1Qyf5Uo0ut31eL1Ow+GIq1+F0Cr7L/JJgIFV1OSSTWS0n3sDVnvWFYvtZ0HrolVovfS3N1yN2TGg90ZMWNhTRnPA4Rshc2D8u3jDErTAeljrP8WRjj+cMCOhtVdttmI58G7M8MgGfDpY3z+eUngxfgNgxcxojSHdr89dh2HGtpAeFa6HdHpAiZqPWZ2KSnQN4nnpXY1e7ulPVzbXyNkHZd2GrBagYp2DxcuXnM7eXdvOPr2pFI/8Bizj8GBsX3+1mA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        "jgross@suse.com"
	<jgross@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia
 Lukianenko <Anastasiia_Lukianenko@epam.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Topic: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Index: AQHXsGDrkewW5NG+ZkqXauJ2P2aIyauyCpkAgACjoYCAAGJfAA==
Date: Fri, 24 Sep 2021 11:38:22 +0000
Message-ID: <7310d23e-4193-3f4c-06da-606b30e73f24@epam.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <20210923095345.185489-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s>
 <f62a1e2c-4253-c998-c206-6bb0681a84fb@epam.com>
In-Reply-To: <f62a1e2c-4253-c998-c206-6bb0681a84fb@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: fc37a11a-0a6b-4031-db13-08d97f4fd08b
x-ms-traffictypediagnostic: AM0PR03MB5939:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB5939AE8DC12BA434BA13BEEEE7A49@AM0PR03MB5939.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3968;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 TBSm1sZMWUHxVNKI1dhZnri7HvMHKM7Xpp2ekkCt2CSZR+Sx8tz4ztnYg6FhuwL1uBQFHEehYh1TSgpEhfzfcMMDWeBpqqV2+bDQ3qpidvbiXSsZI+ZTDgp/YcbZ7N/9iNgVxV4b6+aPnd75Eh8G4wq52vVPt2aETg+5HKEmgCm4dwrWQsSosKwstUKP309KbFaOCIWXHmXkcgut8xaWyAq4nu0ZWfOoDMTsAfDKA2Cr60rSEj7SN3O+t183kHSStl44MkyG1pNLpGdtZnNxv2bfV7ig5nicWIXeXAMWoY7DrFVzotMAxi+BZ5dRaQ01fdpJFgnDYJUNf/SlDNYlXT4Oi1AcCbxeTsGGp3gJ0RT4Hfp5fbzWII1/bKo0vz1AheAf9A/2HNYXaZZCRSwjfnzBsROfOHy4KVnYx69rP2fuXbSB18GjkB7pIIVuZfXSsrJdnq1ndfrO1RfbyPzMxyP5A1KLV0DW6+ePg2tRe3sxqCPavkSeDU+2HxQ2lmAfUFB4BAVL29uoTv6Yr49h/U3Mab6p5V+9vgdnm2FgSTIdlbw7b3nxpLONiUkGstn/yMQbbJAxng6Iw6HtMC6EeXMD+v8riHG5WLpK6nTL9oFEbcHHdB1xSTOPQJG5AK0gpsw7NmRirSLs/LbOoqS0Y7D9/O1O8U82sGNiypFif7D4xgnVK1HaK4WpWZXwr8KTqpIvGb9BxIvz/OJ7Z0fky692j62X5qf81LARTHL7GAZiO4IiN4uq68IVx2oDhHsy
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55236004)(6506007)(31686004)(66946007)(91956017)(64756008)(53546011)(66556008)(83380400001)(26005)(54906003)(36756003)(8936002)(6486002)(4326008)(71200400001)(2906002)(110136005)(66476007)(66446008)(76116006)(38070700005)(6512007)(316002)(122000001)(508600001)(5660300002)(2616005)(186003)(8676002)(86362001)(38100700002)(31696002)(107886003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aEZkWDByK2RZR2tHdXpvNWdiVU5aemIzMHNKWWRTd2pqeS9OQU14RW5QWERX?=
 =?utf-8?B?WGg1bnBMdSs2Z3VxcDVHeTJLTkNmZEJtMitoQlgyQkd2WVVpcmJzV1RIOSsy?=
 =?utf-8?B?LzN0VDNkaWM5ckNqK0pWNHZ0U2tDeEc5RUYzWUhFOUZGVzRFY0tsNVF2MXlW?=
 =?utf-8?B?SkxYZlJSUmI0bzdyblZ4TE9YdlRjZ1NnTlJoamNITCtycnc4VU8xU0llZVow?=
 =?utf-8?B?TWhNby9jMDFoeFl0Rm5FNnFlZGlGZmh4OXdCbUxBRnpzWVJvT1BhL1hHcE1j?=
 =?utf-8?B?cS9rZ2VhUmdxSmttZEFkSVV1S25tM1dVYTNCRDg2NHBSRytKcG16MnBjck5O?=
 =?utf-8?B?Zm1jZkp0eUxYb1N1ODJiL2lpVkx5ZTViNmxSb3JwajBBWGRuR2ZDK0lza0NF?=
 =?utf-8?B?OStsaHh3NUpwNW9TdU1jU3ZnZ0pOZDJ6dnBsdkJ3MXhScDVxKzh5eEZxYnlv?=
 =?utf-8?B?RTdhb0Z1YStOYmdVdE8rRUhMVTg1eVNlMElGTUNkQkxxNnYvcml0VFRIV2w0?=
 =?utf-8?B?WkVrWGE2cHo0RzQycGpnMXp3RFJhQlhzY1F2aWJMdmJwSnA3LzVTRzJaZm9N?=
 =?utf-8?B?aktwOTlXOFZwVjZJQ3MyVzh3bGdDLzRmb3laczdVSWIzLzhNTkJHZFRrRlRG?=
 =?utf-8?B?ZU5Uem1YZitZZ20wM2IyelJzVUhQaTVyMXBzeDR1TWt5WmtNcVFUM1ZTUmlY?=
 =?utf-8?B?RXVFN2lmUm9HdWhLb3Y4cERocGNuSTNGbjNyb0FLQWExTnhjcXZ0cVZYcldu?=
 =?utf-8?B?THpkSi9TTkxzenlMZ21wSDV3T0o2L2xOSDFoMm5iRSttQTBwb3c5dWVBWVo2?=
 =?utf-8?B?ZzZRZmVKZDMwWTZsc1ROdkhZeU5HOWpRMXhWT1Fwd2NNSVNESmVibXFVNW5s?=
 =?utf-8?B?K0NMMHVCMC9KdkYwV1BrYVBsQ2ZtaHBMQ0dWZEpXdkIzdDVHSFViUXl6RmJ5?=
 =?utf-8?B?WGR3aVlqbnVwTXB3VWdza2lTdDRQeEZkOUFsOU13U1BJS0Q0U2RoYUQ5VE9S?=
 =?utf-8?B?SHZwK0hrUmlmTy96UlIxL0NaNHZqTkN5QlBOK0JMZWNnMUtoT0FPZGJ2cU14?=
 =?utf-8?B?MThCU3VVL3oyTHMwbHlHRFRHck45Ni9ab3Q5eFpvL0gvRTR4TnhUc0xVYTI5?=
 =?utf-8?B?VFNHMG1BU0NmcUpTRTBacEZEcC9WUE5DQS9vdTFoZmpOcU5NdzFSK2Fja0sx?=
 =?utf-8?B?c0pFWTdKc3NPL1ZHS0xqNjdtQnNOZjlNbkxyK0s4VHpWZGdIOHpGeGxzZ0p4?=
 =?utf-8?B?TlFyaUkxaWp4dFdrOTZQMVVjU1RicnJGelE0T3Nwb0N2MElycmMzTEtBUHhD?=
 =?utf-8?B?ZjQyaEhicG5LMWFicFppN1BnKzBhMEg2cmJ0MUZvWEpad0JENjVOZGhXK1Ro?=
 =?utf-8?B?VUk5b05OTjExSVI3aWUrWmh0UzdIMk1TbzFQK0QrSlBLekloZ2tOYzJNcHN2?=
 =?utf-8?B?ZlRoamFWdHRabC9BQzBmZ0hYK2haQTcxRWNib2JQU3NRR2hIWDYwTWtUbTU4?=
 =?utf-8?B?MVNjT1pJRTdBZExPRmYraUx0bmhzTUxocXZlTjdFcjhGcThoYWxLUVZITmtU?=
 =?utf-8?B?ekJDa1RES2ZQVnlHclJpU1RkNnVYOVRjcGlaS2Z1cmhMdXJNREE4NUtnWm1w?=
 =?utf-8?B?VXBJRmEyMG1MMHZYeEx4bUVyYzVJVEVMVFRhVDU2UEJoMlNhUHZ0bUdTSG9w?=
 =?utf-8?B?QVRQWFpZUDB5WVZZSDA2dWZBTUNLSkNpQVpTbjgrOUNyK1FleE9Hblk2a3lX?=
 =?utf-8?B?UXc3M3k4aGNTSFVveVF2R3JUTnFlV3RHeGVsOUJ2MDl0MW1XdnJ2UExZVSto?=
 =?utf-8?B?emhPdVRCYjJOV3RxdHdkdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <00A3167252623A43BD1F8B19045ACD31@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fc37a11a-0a6b-4031-db13-08d97f4fd08b
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 11:38:22.6252
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: bL/p2nhSSxXpH4BdYKSZNMnWwKRVPXPQJU3oXP+1UKaIw7mCxEHFzv2tI6fLuLh71Vn+KZWt8XjItCnZr2TD992mL6afAHeq2xSalcwJ3IH+/7QylnzNnDa1ZD4GCHhA
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5939
X-Proofpoint-GUID: LeL34fTg2KwNHEOwQTfpsYIONbGK00FP
X-Proofpoint-ORIG-GUID: LeL34fTg2KwNHEOwQTfpsYIONbGK00FP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-24_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0
 priorityscore=1501 mlxscore=0 spamscore=0 impostorscore=0 bulkscore=0
 phishscore=0 adultscore=0 lowpriorityscore=0 mlxlogscore=999
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109240069

DQpPbiAyNC4wOS4yMSAwODo0NiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+IE9u
IDIzLjA5LjIxIDIzOjAwLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+PiBPbiBUaHUsIDIz
IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+PiBGcm9tOiBPbGVr
c2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+
Pj4NCj4+PiBYZW4tcGNpYmFjayBkcml2ZXIgd2FzIGRlc2lnbmVkIHRvIGJlIGJ1aWx0IGZvciB4
ODYgb25seS4gQnV0IGl0DQo+Pj4gY2FuIGFsc28gYmUgdXNlZCBieSBvdGhlciBhcmNoaXRlY3R1
cmVzLCBlLmcuIEFybS4NCj4+PiBSZS1zdHJ1Y3R1cmUgdGhlIGRyaXZlciBpbiBhIHdheSB0aGF0
IGl0IGNhbiBiZSBidWlsdCBmb3Igb3RoZXINCj4+PiBwbGF0Zm9ybXMgYXMgd2VsbC4NCj4+Pg0K
Pj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5k
cnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+PiBTaWduZWQtb2ZmLWJ5OiBBbmFzdGFzaWlhIEx1a2lh
bmVua28gPGFuYXN0YXNpaWFfbHVraWFuZW5rb0BlcGFtLmNvbT4NCj4+IFRoZSBwYXRjaCBsb29r
cyBnb29kIHRvIG1lLiBPbmx5IG9uZSB0aGluZzogb24gQVJNMzIgSSBnZXQ6DQo+IFdFIGRvIG5v
dCB5ZXQgc3VwcG9ydCBYZW4gUENJIHBhc3N0aHJvdWdoIGZvciBBUk0zMg0KPj4gZHJpdmVycy94
ZW4veGVuLXBjaWJhY2svY29uZl9zcGFjZV9oZWFkZXIuYzogSW4gZnVuY3Rpb24g4oCYYmFyX2lu
aXTigJk6DQo+PiBkcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9jb25mX3NwYWNlX2hlYWRlci5jOjIz
OTozNDogd2FybmluZzogcmlnaHQgc2hpZnQgY291bnQgPj0gd2lkdGggb2YgdHlwZSBbLVdzaGlm
dC1jb3VudC1vdmVyZmxvd10NCj4+ICAgICAgIGJhci0+dmFsID0gcmVzW3BvcyAtIDFdLnN0YXJ0
ID4+IDMyOw0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn4NCj4+IGRy
aXZlcnMveGVuL3hlbi1wY2liYWNrL2NvbmZfc3BhY2VfaGVhZGVyLmM6MjQwOjQ5OiB3YXJuaW5n
OiByaWdodCBzaGlmdCBjb3VudCA+PSB3aWR0aCBvZiB0eXBlIFstV3NoaWZ0LWNvdW50LW92ZXJm
bG93XQ0KPj4gICAgICAgYmFyLT5sZW5fdmFsID0gLXJlc291cmNlX3NpemUoJnJlc1twb3MgLSAx
XSkgPj4gMzI7DQo+PiAgICANCj4+ICAgIA0KPj4gcmVzb3VyY2Vfc2l6ZV90IGlzIGRlZmluZWQg
YXMgcGh5c19hZGRyX3QgYW5kIGl0IGNhbiBiZSAzMmJpdCBvbiBhcm0zMi4NCj4+DQo+Pg0KPj4g
T25lIGZpeCBpcyB0byBzdXJyb3VuZDoNCj4+DQo+PiAJCWlmIChwb3MgJiYgKHJlc1twb3MgLSAx
XS5mbGFncyAmIElPUkVTT1VSQ0VfTUVNXzY0KSkgew0KPj4gCQkJYmFyLT52YWwgPSByZXNbcG9z
IC0gMV0uc3RhcnQgPj4gMzI7DQo+PiAJCQliYXItPmxlbl92YWwgPSAtcmVzb3VyY2Vfc2l6ZSgm
cmVzW3BvcyAtIDFdKSA+PiAzMjsNCj4+IAkJCXJldHVybiBiYXI7DQo+PiAJCX0NCj4+DQo+PiB3
aXRoICNpZmRlZiBQSFlTX0FERFJfVF82NEJJVA0KPj4NCj4gVGhpcyBtaWdodCBub3QgYmUgY29y
cmVjdC4gV2UgYXJlIGRlYWxpbmcgaGVyZSB3aXRoIGEgNjQtYml0IEJBUiBvbiBhIDMyLWJpdCBP
Uy4NCj4NCj4gSSB0aGluayB0aGF0IHRoaXMgY2FuIHN0aWxsIGJlIHZhbGlkIHVzZS1jYXNlIGlm
IEJBUjY0LmhpID09IDAuIFNvLCBub3Qgc3VyZQ0KPg0KPiB3ZSBjYW4ganVzdCBza2lwIGl0IHdp
dGggaWZkZWYuDQo+DQo+IEluc3RlYWQsIHRvIGJlIG9uIHRoZSBzYWZlIHNpZGUsIHdlIGNhbiBo
YXZlOg0KPg0KPiBjb25maWcgWEVOX1BDSURFVl9TVFVCDQo+ICAgwqDCoMKgwqDCoMKgIHRyaXN0
YXRlICJYZW4gUENJLWRldmljZSBzdHViIGRyaXZlciINCj4gICDCoMKgwqDCoMKgwqAgZGVwZW5k
cyBvbiBQQ0kgJiYgQVJNNjQgJiYgWEVODQo+IGUuZy4gb25seSBhbGxvdyBidWlsZGluZyB0aGUg
InN0dWIiIGZvciBBUk02NCBmb3Igbm93Lg0KDQpPci4uLiB0aGVyZSBhcmUgY291cGxlIG9mIHBs
YWNlcyBpbiB0aGUga2VybmVsIHdoZXJlIFBDSSBkZWFscyB3aXRoIHRoZSAzMiBiaXQgc2hpZnQg
YXM6DQoNCmRyaXZlcnMvcGNpL3NldHVwLXJlcy5jOjEwODrCoMKgwqAgwqDCoMKgIG5ldyA9IHJl
Z2lvbi5zdGFydCA+PiAxNiA+PiAxNjsNCmRyaXZlcnMvcGNpL2lvdi5jOjk0OTrCoMKgwqAgwqDC
oMKgIG5ldyA9IHJlZ2lvbi5zdGFydCA+PiAxNiA+PiAxNjsNCg0KY29tbWl0IGNmN2JlZTVhMGJm
MjcwYTRlYWNlMGJlMzkzMjlkNmFjMDEzNmNjNDcNCkRhdGU6wqDCoCBTdW4gQXVnIDcgMTM6NDk6
NTkgKjIwMDUqICswNDAwDQoNCltzbmlwXQ0KDQogwqDCoMKgIEFsc28gbWFrZSBzdXJlIHRvIHdy
aXRlIGhpZ2ggYml0cyAtIHVzZSAieCA+PiAxNiA+PiAxNiIgKHJhdGhlciB0aGFuIHRoZQ0KIMKg
wqDCoCBzaW1wbGVyICI+PiAzMiIpIHRvIGF2b2lkIHdhcm5pbmdzIG9uIDMyLWJpdCBhcmNoaXRl
Y3R1cmVzIHdoZXJlIHdlJ3JlDQogwqDCoMKgIG5vdCBnb2luZyB0byBoYXZlIGFueSBoaWdoIGJp
dHMuDQoNClRoaXMgbWlnaHQgbm90IGJlKD8pIGltbWVkaWF0ZWx5IGNvcnJlY3QgaW4gY2FzZSBv
ZiBMUEFFIHRob3VnaCwgZS5nLg0KDQo2NC1iaXQgQkFSIG1heSB0b2xlcmF0ZSA0MC1iaXQgYWRk
cmVzcyBpbiBzb21lIHVzZS1jYXNlcz8NCg0K


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 12:03:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 12:03:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195344.348001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjuo-00036p-Dq; Fri, 24 Sep 2021 12:02:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195344.348001; Fri, 24 Sep 2021 12:02:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTjuo-00036i-Ap; Fri, 24 Sep 2021 12:02:54 +0000
Received: by outflank-mailman (input) for mailman id 195344;
 Fri, 24 Sep 2021 12:02:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTjum-00036J-MV
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:02:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e4bbf1d8-11ed-42c9-8442-d58e8d54389d;
 Fri, 24 Sep 2021 12:02:51 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2054.outbound.protection.outlook.com [104.47.0.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-62rC-_ISPda0KYu9AXNEyQ-1; Fri, 24 Sep 2021 14:02:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4192.eurprd04.prod.outlook.com (2603:10a6:803:4c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 12:02:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 12:02:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 12:02:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4bbf1d8-11ed-42c9-8442-d58e8d54389d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632484970;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Jda2F2n00mJxCmwc2V9HhThsClvDfmqRmCUN1I/E/PE=;
	b=OlaHQsTj9qeAZ7nNbsdnM59Tmhtrb6uFREyfCIsiEzLGthyA2Mm1TMTA51np0xEd0Z9a5B
	bXZkPcFrNQF/pcZtIATahlaORDHjYG4Xhh8jHPN9K19dboJmBZk+Bvrds59Az3TuosYO9j
	lpnR3vjFLvNK2U5+yFxihTd1lb39ciY=
X-MC-Unique: 62rC-_ISPda0KYu9AXNEyQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kN0WhsvxHeKGOu05cCwFSYp6HJQ3kdfS8g8TiiabnP4SEXCNSKpYmaAR/pX+3VQz7PfEagciEvcvlS2E2XClM3+xiyacm60xiOrp5w3l4xi7NnhfhQJsOFcnSZ4Cc3V7psZDi6WJYdEAJctdyvWxCIyyfj5t0/VizJehqkZ77ZvfYnbjm8q8ggLzQQRIrhqpJwdhchYBoXbBEh3tdUtmZwJ8fudNIP8q1CIaHHMFXWP2ksXlUv0gYlc3/tYj8MKLWE4Hwd4wck5nmZByHAYRxIZeOxac5ereSkACJl3q2Y66+e4NGkAssOgKVHG84QoES5kukRk2EGTZVbT50LjHmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tZ2JJ0qf38l3SeHPkotDBqFhOnZimuvD4duLlByCEDk=;
 b=J0P2LaSyyhgf2lAHD6ItuVqRutRly1JpDbVqQ+I0LEHswLmG8zOn1+lWZ0gQCP2Vk0Yq2dLtHio3mG1/e2mHR80QG2poNRBkxA782cLhpb/YvZ8F0W9Ho3HUCkRWiydXBEgG5oKPr7iPdslAkrsBjRtgk0BOtGHxlsUezDCHr3ZMnn6ugr/GcQ5LnnDuXfnJ95XcYsI7IJ9ZTSlp/pHXmNE6mIKWRr+KmwwcKoM522ufk2/F79YutACKZt6iOrjMc3cdjToveY+Ct4myunQHE858W78uW4hef4AseHlCtHlh752bn+d3dYF6XKruPL/XY2+bnWjYn4wISrnA0oho4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 01/18] AMD/IOMMU: have callers specify the target level
 for page table walks
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
 <57abd488-1fa1-8eae-66b9-196f32f7f4be@suse.com>
 <YU2vXU3UffSQ/iQX@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <eadb98f9-f398-aa55-641d-d4b29c1cddac@suse.com>
Date: Fri, 24 Sep 2021 14:02:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YU2vXU3UffSQ/iQX@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR0P281CA0070.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:49::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 51f3b0c6-18b6-47c0-621a-08d97f533a00
X-MS-TrafficTypeDiagnostic: VI1PR04MB4192:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB41926109B65E330FE805F8F2B3A49@VI1PR04MB4192.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	AVH1hjZpbFbVu8gpZ9GcmjKBIi8Ki8ZlrgZNXKoAa80F7DVkhB8lUssYfjUMjNwp8AUunUS98WdBEF2nrLj7bMJKOJYkcEXpqFh7n5s6M/RUYYeGyHMdCtubMJ8o0mBU9xuzNZq5NGGkriSKPCFOK7LSEf7+R8BhjXYKq3ZB7Wsyuu9GGQ342jnHP71QRAzMdP7ii76fg8k2NtCgXKv/AuPbczahEoqD9EN5TDp/Trr4eMvvX/MI1DFkhNV7uzhr4q/jXZBleu+O2/QU5PfJlos29ggr4CvR8wQnWaq+qC53/mVJEgXOg4jupWNHE2ewlUZUQk9SUBBjxVQ80ojoyYe9KqORyXy+4IF6MS5yj5hbYGAU7wtZDbhC7zvdxKRhxksCYM1nocqF4ixGeHMy34HStuYmueyC6SXgnuhBOHBohdVdd8h2kaHOWh3e+qWMdEy0UixVlj+hP+hnim2xwclhbCZQIPfvVEQj9phELvGsi8XdK5DTXjiSVg6uzQgp6AzO/yyP3NqrPgZgF+WgvOi8Wr7uxZMiQlpVDHI86kGpU6fyofjyHWTVHwUl/jRzGfVAvhUCOtxF9CUx0k9hlxtAzgBuD10mp2tQO5ZnIDVWGvbM3+sIk7y9Eh0nSS4mRnDTM96qOeNDr6fxuyfyc9jgfr5yBdoV4wg2+rpr/Zn4ed/qV9VjKQf0ull8UTCwrWH8Zzc5xibeAl11vgdKW/FkObRfCwmXBe0y0sbMnPw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(2616005)(66946007)(86362001)(8936002)(316002)(956004)(54906003)(31696002)(8676002)(66476007)(53546011)(6916009)(6486002)(186003)(4326008)(31686004)(26005)(508600001)(2906002)(66556008)(5660300002)(16576012)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?QOlmqlWM1Y/Ser9t45VlW2ZXuFgiZguovRZGgmXbLnn8D7hLc1+PfNmqRJJ0?=
 =?us-ascii?Q?p0mvYUmsgJCUanPqZyWjRip9fp3DOFKxgucuhXJcQv1N+LXbXv1UFC1ZjNtm?=
 =?us-ascii?Q?OW3h2bpgJa0+HojB292r8uWwvB3fWLF4Gj/xqsVarjZl1iJjbJ3RgiYmVadJ?=
 =?us-ascii?Q?7lvzuQd+nQ/HjRIka7sxnfWOxFrz3iLMCSwGL4ZsRnB8HK6bXOUIcU4EKjiq?=
 =?us-ascii?Q?raRjv+/LFDMDO5LILAqyQXaW0BcU+ALtIZRqsSkZtS0njLPIEb+SYia47ROo?=
 =?us-ascii?Q?Fz7dKfyJE/3yL203quTbDt9gVel1jOKlxkNd3bp4Q0c/lIDqH8Nw0WoicxBj?=
 =?us-ascii?Q?dx0m4qz1fRxpqT2RDhf06Yeud0na9eZAaRcdm9uvCsmvF2BCE0M9r55RE0qw?=
 =?us-ascii?Q?Gy3C2WoMXVe23GHxyFP4BIpV12zLSb5dqo5RSx4NjknfU8aQpXJXVDWw2Kqj?=
 =?us-ascii?Q?dnuTeXQlkfnmCs/yrbms3iUDZldxIciKEn3j444LnTKumR6Z2vjCT4tl2Zdd?=
 =?us-ascii?Q?URVo7NQTNRzOxLtVcXs2draJY9412YH+CHctbhOhw8/lwRCFnv98zFx5AUVM?=
 =?us-ascii?Q?aR6n+sWIaTV3qLak66iHwShqbBjivnhiqHFRrRs+iiZtEvZDDdu/okZl4/BE?=
 =?us-ascii?Q?hqbzJ0lqcbHks+T4wcwPDJYjV5Q4PjU7ibdzQaGLQRSha2Ffp1Z4fDXSGbQy?=
 =?us-ascii?Q?hnQEl0cFoyN3EvqF4R+eJDbNjimXKUj1BVc9zysKNFefdiPtNDHEQISv5jau?=
 =?us-ascii?Q?W/LPbOBdNIsFpuM46oHw5JlO90JL5u7j2pzOX8qdz+tPpjJuKXsV833pjs9D?=
 =?us-ascii?Q?hdKKkTbr8yhl+KvpaV6BaEyNS4Z313zD01qK2urWy5x9rOlX4ApfJtCcSRe8?=
 =?us-ascii?Q?DtiQJOT+fTT9x4RfHSqtsQGaZ0gBM3dyZFNSvuVAAHiMIwy8q8DHA5xZuB1R?=
 =?us-ascii?Q?cI17XkctZ2m9X1ZF4y05kiCHi8dx1S7bb865G/nAvr6QUD3Dpvulo+yeZqIa?=
 =?us-ascii?Q?fdd7X/4MaXMJ+6I8eFuQYNBnc3luLPHcric5yf9Hy9an33I1HNScOp/gP2qX?=
 =?us-ascii?Q?4ckJY7lAJecU4SqAcNklJuxafOYLt6nwsrVBoQAvxeHFivCftR5EvEDCiVWX?=
 =?us-ascii?Q?QvXablI5UBmXRYu1vog2BNv7NX+p/Mtr0LN5L5Z/dyDpsA+Vf8k/meeXaF+8?=
 =?us-ascii?Q?4k7G/waEB85Me2I8knQn2yBTfBDglKVc5H+BRcIGci4lEkqPtuAh+SRWM1Yv?=
 =?us-ascii?Q?aNpRPxzB+tDPNH3LqGEWJuM5lyupwbi5R6jKMXU6So2xmrcd2JIT1YcI8Ew4?=
 =?us-ascii?Q?N6yJQY1FKVzBJw/k97Ogkq8h?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 51f3b0c6-18b6-47c0-621a-08d97f533a00
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 12:02:48.3181
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0eT8JB4p31sVrqewNdxRoQIFeRfs4w3nNpYvIpQGPvgC+PcfWGm/HSPzRezlgEB3ZeiG39BRgon7bypEpYMa9Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4192

On 24.09.2021 12:58, Roger Pau Monn=C3=A9 wrote:
> On Fri, Sep 24, 2021 at 11:41:14AM +0200, Jan Beulich wrote:
>> --- a/xen/drivers/passthrough/amd/iommu_map.c
>> +++ b/xen/drivers/passthrough/amd/iommu_map.c
>> @@ -178,7 +178,8 @@ void __init iommu_dte_add_device_entry(s
>>   * page tables.
>>   */
>>  static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
>> -                              unsigned long *pt_mfn, bool map)
>> +                              unsigned int target, unsigned long *pt_mf=
n,
>> +                              bool map)
>>  {
>>      union amd_iommu_pte *pde, *next_table_vaddr;
>>      unsigned long  next_table_mfn;
>> @@ -189,7 +190,8 @@ static int iommu_pde_from_dfn(struct dom
>>      table =3D hd->arch.amd.root_table;
>>      level =3D hd->arch.amd.paging_mode;
>> =20
>> -    BUG_ON( table =3D=3D NULL || level < 1 || level > 6 );
>> +    if ( !table || target < 1 || level < target || level > 6 )
>> +        return 1;
>=20
> I would consider adding an ASSERT_UNREACHABLE here, since there should
> be no callers passing those parameters, and we shouldn't be
> introducing new ones. Unless you believe there could be valid callers
> passing level < target parameter.

Ah yes - added.

>> @@ -200,7 +202,7 @@ static int iommu_pde_from_dfn(struct dom
>> =20
>>      next_table_mfn =3D mfn_x(page_to_mfn(table));
>> =20
>> -    while ( level > 1 )
>> +    while ( level > target )
>>      {
>>          unsigned int next_level =3D level - 1;
>=20
> There's a comment at the bottom of iommu_pde_from_dfn that needs to be
> adjusted to no longer explicitly mention level 1.

Oh, thanks for noticing. I recall spotting that comment as in
need of updating before starting any of this work. And then I
forgot ...

> With that adjusted:
>=20
> Reviewed-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Thanks.

> FWIW, I always get confused with AMD and shadow code using level 1 to
> denote the smaller page size level while Intel uses 0.

Wait - with "Intel" you mean just EPT here, don't you? VT-d
code is using 1-based numbering again from all I can tell.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 12:12:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 12:12:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195351.348012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTk3y-0004st-Bc; Fri, 24 Sep 2021 12:12:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195351.348012; Fri, 24 Sep 2021 12:12:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTk3y-0004sm-8d; Fri, 24 Sep 2021 12:12:22 +0000
Received: by outflank-mailman (input) for mailman id 195351;
 Fri, 24 Sep 2021 12:12:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTk3w-0004sg-Se
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:12:20 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTk3w-0000ZA-Pz
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:12:20 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mTk3w-0008Se-PA
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:12:20 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mTk3m-0000dH-Tv; Fri, 24 Sep 2021 13:12:11 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=2S9JgcJNkwk46B8y3Gw1osQ9Eb/Yh4Ozw+h2VWPDH2g=; b=ezcFIq8qdkj7EbR3r1rrfSMD4/
	UlIZZfU2Mzeb+kjUSDW/6HsovYWvZerPSdTMXUfcYx5qbuDl4xYKgSFEkN+5aiwTN2gzaAF42ApsO
	wDnaKsU9E2Wj/UlOTRBKTedLwfLkGrzKV/jwM0EaZoZvPlAQ0R76OuSIrJwgFEQv21IE=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24909.49306.649660.76404@mariner.uk.xensource.com>
Date: Fri, 24 Sep 2021 13:12:10 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Kevin Stefanov <kevin.stefanov@citrix.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    Anthony PERARD <anthony.perard@citrix.com>,
    Xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH] tools/libxl: Remove page_size and page_shift from struct
 libxl_acpi_ctxt
In-Reply-To: <e3d86499-3c1e-6cac-9225-f64b101c5478@suse.com>
References: <20210924110500.25412-1-kevin.stefanov@citrix.com>
	<e3d86499-3c1e-6cac-9225-f64b101c5478@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [PATCH] tools/libxl: Remove page_size and page_shift from struct libxl_acpi_ctxt"):
> On 24.09.2021 13:05, Kevin Stefanov wrote:
> > As a result of recent work, two members of struct libxl_acpi_ctxt were
> > left with only one user. Thus, it becomes illogical for them to be
> > members of the struct at all.
> > 
> > Drop the two struct members and instead let the only function using
> > them have them as local variables.
> > 
> > Signed-off-by: Kevin Stefanov <kevin.stefanov@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Ian Jackson <iwj@xenproject.org>

> I would like to suggest though to consider ...
> 
> > @@ -176,20 +174,19 @@ int libxl__dom_load_acpi(libxl__gc *gc,
> >          goto out;
> >      }
> >  
> > -    config.rsdp = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
> > -    config.infop = (unsigned long)libxl__malloc(gc, libxl_ctxt.page_size);
> > +    config.rsdp = (unsigned long)libxl__malloc(gc, page_size);
> > +    config.infop = (unsigned long)libxl__malloc(gc, page_size);
> >      /* Pages to hold ACPI tables */
> > -    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES *
> > -                                   libxl_ctxt.page_size);
> > +    libxl_ctxt.buf = libxl__malloc(gc, NUM_ACPI_PAGES * page_size);
> 
> ... using page_shift to replace all multiplications like the one here
> at this occasion.

I don't have an opinion about this; my tools ack can stand if this
change is made and reviewed.

Ian.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 12:18:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 12:18:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195357.348023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTkA7-0005ud-1v; Fri, 24 Sep 2021 12:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195357.348023; Fri, 24 Sep 2021 12:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTkA6-0005uW-V6; Fri, 24 Sep 2021 12:18:42 +0000
Received: by outflank-mailman (input) for mailman id 195357;
 Fri, 24 Sep 2021 12:18:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTkA4-0005uQ-Vq
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:18:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 70e866ae-532c-482c-a1e1-67ac0b533209;
 Fri, 24 Sep 2021 12:18:39 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-hgUXlle2P_CVRsO8_m7p1A-1; Fri, 24 Sep 2021 14:18:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2606.eurprd04.prod.outlook.com (2603:10a6:800:51::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 12:18:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 12:18:35 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2PR09CA0020.eurprd09.prod.outlook.com (2603:10a6:101:16::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 12:18:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70e866ae-532c-482c-a1e1-67ac0b533209
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632485919;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=l4svGse+imCW3XeBYeXhFLL5vLox02qw2i6TxB/lCOk=;
	b=L1vDqf6iVlMRqdbtr1iWxLQ03C/HfmnohbvSXZvDbezf9/aklpQ9V9hAob4rro+ZXvu1OD
	anmQNszLvvNwo5d6cC//KIftnSg3ZuP40WQ73EqG8B/g/ZTBNERUdaptHESK2TVASb4SG0
	yeQlRFWZu1jxzRrzVRHIcJ2Bjyln65o=
X-MC-Unique: hgUXlle2P_CVRsO8_m7p1A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WJ4KGtpoVPVH1m7pem8BsA0O7iEkwqwdarLRBwPpJ3jq6kcK7GAZM9tV30saBe8lRymZD/8jAvYhUmwESvcNN2IlRWvHu2QDZlPpuJAf3VwxSN/YPmKwDVfDQ++Fb4gMgwHdFJvZ+/4P7oTFsayUJpKH51ahI5/EGa2PHzx1AnIabvOpjdB15pMkxUn7o++bKjSui0MqH7yxgvw4oBezMyRElIYq7T+boIv/Peo7W0wDaK0Yidgm0pyWo23U3eFB1vP3zgOQaWM0YaCk69c61ndWyy/hZYy1uPhVyZ0031/Q3fokEbP17fMwSCnmCS+YmaIztOTbdQOFvh1f+V33/A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nU9oC7z5BXKFkeO02V59+X3J20q93nT+VQB78TQv6mg=;
 b=OHS8bvKC8HYIRUbaidkdtcY0fX3OoBfsB6dpI9Yjy0p6exg6D2RbJoTA9FSOQ43ccaR+kC2VjHyJOIfs0JckvoLVQxmT6jnwI3Jw8lAmEC9CbBqCY/Cd5orARTTkuUW2nahRXJpoS+zcOtnLwc8pDzrBnDVuh5GJ97hPkJSHwxGK2SXRlmN8a7GOz2nzZQLk6JK4TncqUVY/9/l+O24yBTgb9vxxbPZvqJSa49EOEQDxCUuYTcezbbmgVz97r+61OfpcKR+uqzyxNgUeN7/9Xtw4+3i+lWEdHkPgxVz/qrfsRSs5Xh0k1K6EbH846lA9UaWzjtPFhWcQlb7uz4dGMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 3/9] x86/PVH: permit more physdevop-s to be used by
 Dom0
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <78c84053-3d79-b36d-be03-08b3b3a3d26c@suse.com>
 <YUs8Jq4mei9+DDCU@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <986cba30-f920-854a-2ac7-70d3e16de5ab@suse.com>
Date: Fri, 24 Sep 2021 14:18:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUs8Jq4mei9+DDCU@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR2PR09CA0020.eurprd09.prod.outlook.com
 (2603:10a6:101:16::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6a96b52a-3d85-4695-f579-08d97f556ecf
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2606:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB26060C9412569283901EB940B3A49@VI1PR0401MB2606.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hR/dG76/AB+ntvhlfdSSy6EIZboiDmUN0/nad8HvKhHwqMgLMQ7Egpi//ApAh5EIwItDae1T/zBbmwRPeKDnlLGyVtJL0Z2i6Bc/kuoFslGxo9FiehstM0iP+/3ZcBLVdf0+Z3jHGVa9XT5ynGr/jQwgiK10do4ksZ8oHNtf4XFOLLWrm+T3Je6yHjnAH4RKIzhpEKk26eOZocRyixiR10Jdjc72mn82W7QL5Lu3As7NZVPG+VAUGTwFHgN/nnyAvlFWs+P2Mnuu3O1gVA9lE/PinRrlsUwcxN0w8OzT19t5tvU8HvgYG0C+CB61Xn0+Yx9cl9+Yw/3id2ulJ0obkELNCer1Dx94JhrG5ZkPY4qDGInz8UlPHlCB24dLAW1TZSTucRKgArSf203Veit6ueuh8GixDPVE7aMtIeWSeMxqZ+vSEvZSDdRwMS1SK+LdVjZafXYdy4rpCqlhwFMPGUEwPClw63Jlug9q6CIPIrbaJKeA7roLlgXugLKnqmrkcZWWOoCUiYqGhHfYim3SnKkxvptSdG+QdAneAwrZJJeTbcIOVYE8pMxS2WANTRMXtPXjlaAuMF2IE5kTZtvlbYgJKLU0oSCxqw4RTKU6IdFoIbJY6dUs5QYVNPJVAdOp/LWAGou7KyoA8eVclr5lZqwM1DzClB1o6bU1qTILHw7YxH1rHTdLo17DgpMAm81ovC9+bXHyzsTds9K9AdsRP2C+Or2jwAOxcvK7LQveuSo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(66946007)(36756003)(5660300002)(2906002)(4326008)(83380400001)(316002)(31686004)(16576012)(53546011)(54906003)(66556008)(66476007)(186003)(6486002)(8676002)(86362001)(31696002)(2616005)(508600001)(6916009)(956004)(8936002)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?LzbkYhB4pIkwIuKRu/gyP6ceajrgdlAXcoNUAZfT9g6TZvmu7xif37qU82uD?=
 =?us-ascii?Q?LZWEBH3m4osRtuI+E/iyORQn8c/FTNMpLgld4owZLfBmS6L1SYtq5rdKlWcH?=
 =?us-ascii?Q?0RuqIWBcnevV1jrhqdGUwvOsk1Kg9CJOuisS5QB9AeNIyIGxPH4eRwY4hiG9?=
 =?us-ascii?Q?CFUi2FpdMSkJohEIDCMCNYGkqxMimS4MSENeUs/500rfQ+fCGzPUl8kNxIHn?=
 =?us-ascii?Q?Tb8SErkVj4t5C1mgN4Ei7Jju1BOEDorn/yrxKPKkf/T3Iv+fR8VC1YlY/gzK?=
 =?us-ascii?Q?WIowlUCJ2Qm+oIVYVU+tRcnZ3yDtjPeMpHJoaN/dG3Tl64QGST8BEU+Qzh9B?=
 =?us-ascii?Q?alBghjE3xTqoAZ4GoMuZW3ZfPi7Ypw4zzhKqLNMntqZKY4cF+XCc/HMc2lIV?=
 =?us-ascii?Q?0YtucLmUhqYeGBFTo99Ta11tTtMOjzlaeJeltQ6v86A4U7CA7CsIcwE5O8n8?=
 =?us-ascii?Q?Lf0HjnpciN0RlvkUs3WT6M2lAKZv8EtxSfECHywb9PpZI9HDZtGaTODFfMu0?=
 =?us-ascii?Q?2WDuqOI+XQ09ah4KAjaot69GTnpqIztBBr6VFTMT7q053l5IxnnzPH1/tu2p?=
 =?us-ascii?Q?q31YmUw+dCMxLzYoz+z9iBlB7y6eQxpvDgisvRBKS1UDVJQhP7yfy1Il9P8x?=
 =?us-ascii?Q?9HcbGo6rduMhR92A7PMfOBfDlv+5fabw0Hy7h9wQNgjfPPHcuK8cAuQdRC+T?=
 =?us-ascii?Q?I5PNclrCTu3QHNTd5NFFojVYgg/z2ycrjA/5/xy0rHjDw/2StXNn6nFiUYTq?=
 =?us-ascii?Q?fieKOj/dCspUJRikq1ggQwtHhkv8z+Rrv0LTT1gUW5hHuts1uovLXIk+D+r0?=
 =?us-ascii?Q?wseYY47OtWn5uxhwX+gaHZrFLMHbvvMPgB5pBUUI6UPMD9rDotY2Fka/hnvM?=
 =?us-ascii?Q?9WrEJE77el/e2NDu6OPXvfshP65gSRHDKEvuuA1qzux0BOey7ImSyAcy5DVQ?=
 =?us-ascii?Q?lYcpHmlJwNxGcpriqe8gbwM7Z0B0BQm/d75XaVyf65EOhULPbcb68JcLgNwk?=
 =?us-ascii?Q?WrSu4fqilZKd6rVO8Ik6+1ltrjYd38wajnu2nqQ+ugXlUgoBNW144myic7SU?=
 =?us-ascii?Q?lAWJewgD3OTUJLPnPwfdp5bCqDTo+RkU2pc4a5PC7kKJbA03UQc8CFBXo6Nt?=
 =?us-ascii?Q?P6AZSpVIEFL1SXznYtCKTGTdZ+X0PvknbB0RvKjt4Esmu//YlZ/gVYU3ZNo6?=
 =?us-ascii?Q?LC7PCAA0n+RGQLlBEa9SzvLC+mKSmtlNBNPJWe6NaDCkjscvkAn7XLpM+yoK?=
 =?us-ascii?Q?xxguRmNtpAQoA/o7ttRy/b2m55kYPcSiMkMm3al4TJ0+LpbjZwwbHwDsflGx?=
 =?us-ascii?Q?d+gj6WOwEWHxlrWC/J68UEW+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a96b52a-3d85-4695-f579-08d97f556ecf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 12:18:35.8925
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vyZN+C+Zvtjy8NW7tBdcirT7E/0b+f6daMhINnJrsIfg7wcegTCtpvdb+UFK3WvqxeUPuYcvM+39M1QZ4lfnLQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2606

On 22.09.2021 16:22, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:17:37AM +0200, Jan Beulich wrote:
>> Certain notifications of Dom0 to Xen are independent of the mode Dom0 is
>> running in. Permit further PCI related ones (only their modern forms).
>> Also include the USB2 debug port operation at this occasion.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I'm uncertain about the has_vpci() part of the check: I would think
>> is_hardware_domain() is both sufficient and concise. Without vPCI a PVH
>> Dom0 won't see any PCI devices in the first place (and hence would
>> effectively be non-functioning). Dropping this would in particular make
>> PHYSDEVOP_dbgp_op better fit in the mix.
>> ---
>> v3: New.
>>
>> --- a/xen/arch/x86/hvm/hypercall.c
>> +++ b/xen/arch/x86/hvm/hypercall.c
>> @@ -94,6 +94,12 @@ static long hvm_physdev_op(int cmd, XEN_
>>          break;
>> =20
>>      case PHYSDEVOP_pci_mmcfg_reserved:
>> +    case PHYSDEVOP_pci_device_add:
>> +    case PHYSDEVOP_pci_device_remove:
>> +    case PHYSDEVOP_restore_msi_ext:
>=20
> Hm, I'm slightly unsure we need the restore operation. Wouldn't it be
> better to just reset all device state on suspend and then let dom0
> restore it's state as it does on native?

Hmm - Linux (even after my patch separating XEN_DOM0 from XEN_PV)
only issues this call when running in PV mode, so from that
perspective leaving it out would be okay. (Otherwise, i.e. if we
decide to permit its use, I guess we would better also permit
PHYSDEVOP_restore_msi. Somehow I had managed to not spot that.)
However, ...

> Maybe there's some wrinkle that prevents that from working properly.

... Xen might be using MSI for the serial console, and I'm not sure
this interrupt would get properly re-setup.

>> +    case PHYSDEVOP_dbgp_op:
>> +    case PHYSDEVOP_prepare_msix:
>> +    case PHYSDEVOP_release_msix:
>=20
> Albeit I think those two operations won't strictly conflict with vPCI
> usage (as they require no MSIX entries to be activ) I still wonder
> whether we will end up needing them on a PVH dom0. They are used by
> pciback and it's not yet clear how we will end up using pciback on a
> PVH dom0, hence I would prefer if we could leave them out until
> strictly required.

Even without a clear plan towards pciback, do you have any idea how
their function could sensibly be replaced in the PVH case? If there
is at least a rough idea, I'd be fine leaving them out here.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 12:54:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 12:54:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195365.348034 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTkiv-0002Pt-Rg; Fri, 24 Sep 2021 12:54:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195365.348034; Fri, 24 Sep 2021 12:54:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTkiv-0002Pm-O5; Fri, 24 Sep 2021 12:54:41 +0000
Received: by outflank-mailman (input) for mailman id 195365;
 Fri, 24 Sep 2021 12:54:40 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wplY=OO=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mTkiu-0002Pg-89
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 12:54:40 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.48]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 93729e38-1d36-11ec-bace-12813bfff9fa;
 Fri, 24 Sep 2021 12:54:38 +0000 (UTC)
Received: from AM5PR0101CA0035.eurprd01.prod.exchangelabs.com
 (2603:10a6:206:16::48) by AM6PR08MB4456.eurprd08.prod.outlook.com
 (2603:10a6:20b:b3::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.17; Fri, 24 Sep
 2021 12:54:36 +0000
Received: from AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:16:cafe::e6) by AM5PR0101CA0035.outlook.office365.com
 (2603:10a6:206:16::48) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 12:54:36 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT026.mail.protection.outlook.com (10.152.16.155) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 12:54:36 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Fri, 24 Sep 2021 12:54:34 +0000
Received: from 9550938d897d.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7CDF5944-6C1B-41B4-A4F6-F38CC7166581.1; 
 Fri, 24 Sep 2021 12:54:22 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9550938d897d.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 12:54:22 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4658.eurprd08.prod.outlook.com (2603:10a6:20b:c4::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 12:54:21 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 12:54:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93729e38-1d36-11ec-bace-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9GtB33dIgD5zIx9S6HYS1ULvpV3qdRk8yqcQyYGnGnE=;
 b=FMPQeA0ywoIXjBbQWBMwum0M7PySWK32WRKkJvEvrlOhmUTCU0Pvsumpu1jYMYj+4T7+Tl4RZBghz4HNE1vjC0fhAfkWHO5taqFkGSSFCqB8PjzksqLuk2oXyUNvgdpdfbgDAHxIz1t5r8txBkv62vd28q4/WmYi7iDw/8qcjaQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2e0f3c23e4870163
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vy7eFj6SLavyhKy202AVtE5bMnrwmil8CcKdECzEuFrIjDUfECk+ivKuu/YiRuE4KOzHScJ4mIG5yopNlZ7Lb+R6dXrOgaQuXCQOCbK98UvFg5bfE7N/9LZS08dP5bsIHqw9nQjeRsGBzy61+mBsBKRZZ/g++c8lJpLXiKYLkC/qxE6vwNSjCKQzTYToI9ONfx3DKhwWQKY4lPdK1BB2BD89MFA2d2wqAksp0ORUisStgF+MdVYg7NqKLBxwyYAxvSl0l8fa4CWPorrxllheJIRpFyjVaYSnl3awsrPXODD5x/6iYl99ofjwdPyFDsrpTy2Qgiq4l3KLMuZ+PZcwHQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9GtB33dIgD5zIx9S6HYS1ULvpV3qdRk8yqcQyYGnGnE=;
 b=UfNf/Zetqy4qo0Hd5W/NScP78uZyAtuW4IEoyXivJNtwHosFjThTigNUMHeT+w/0YEdrS2NLwr60mxjdXEWTtHSfwvUwbbHwCQKQJk+X7xqwb3CX1m8Bn5rQUzAe/nG6YBeFZMyy0prLkM5Ht7Iv5Dn1lVOtY1Gm73TVNxtTth6Ba5esqJAtDF+KpgDPkQRvtsk7Ynovb37rcBM2bWVHjD0c1gxLGPDqqT7EQ3O7nbr4JxGWUXv2wD+F+NfWD6TytB78ZLl8YP4CsGUMNqcAxtW/C9ml7nYL2whHIcRTQAZhuVSNxgG4HmpR8o2NMOI8j5y303IlYiuWDvVZD7u9vQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9GtB33dIgD5zIx9S6HYS1ULvpV3qdRk8yqcQyYGnGnE=;
 b=FMPQeA0ywoIXjBbQWBMwum0M7PySWK32WRKkJvEvrlOhmUTCU0Pvsumpu1jYMYj+4T7+Tl4RZBghz4HNE1vjC0fhAfkWHO5taqFkGSSFCqB8PjzksqLuk2oXyUNvgdpdfbgDAHxIz1t5r8txBkv62vd28q4/WmYi7iDw/8qcjaQ=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXr6aPsmXcRwLbA0y2fuTM/iBxWauw4OaAgAD7DQCAACKOAIABKMWA
Date: Fri, 24 Sep 2021 12:54:20 +0000
Message-ID: <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
 <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
 <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 122b33e6-29f5-48a0-9335-08d97f5a7678
x-ms-traffictypediagnostic: AM6PR08MB4658:|AM6PR08MB4456:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB44566D84E1FE6E19A453413CFCA49@AM6PR08MB4456.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:4941;OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GVz+ea4oIy4WyeldOF0X1DypdYt6NCpfec4FSCqG3t4q/odvqF8nW8mvt9UFBb9YHkRRaytxHBDUdknTNvBeW0atkOt51fYS+96CyYHqhQuLbil3sJUG5SlGDbNYNwhjjSJsRmB2wvnekkQFf2U+/D7O4HbXtyRx5AUCEMxEAkqZk+vpMtFGTZce8pEeg3G6PiT9qmdNj1meqwmHnIxSu1qQM+Y3HAvRJsxreSKKB/JTnMZpLeErIpFK4vdAeUEC3Jd1yOJBixHf4ZAfoGAuMryG9b0gLcr6c1n2GXbYHDqWB5VcSVxOaadkk9PslCYnBT9Td0eELzK9+3R75/2AcLmRCliY4nITDsrQ+RhM9QqUyrf6hXI3hfE23rLDZ809+n5fwW1eFsreo3ddF8fiGAOczjpjXnfKrbHkgUdQxT5FG+s35WXpoBg/8fRMiw2bBq9qwvTq5lQD9adZx9gkv3aONlVz2k7icMJp6TsGOH2CH4VV5nL1mrSwoOYWYC4ddIbfgn7xfcgf+a72zatKstQQYPvHEiExfyfesWP2qzIlLxee0/zTGGSJlvT0kPCFwjRuzoDbvdcq7+u2KSXv3QsXyMTppyGkWqxIdtWQIlLNo/j5i0mzN78k4rvnd7xQLG9aP3KmU8Bld+Vmd3QcNMC70xpA/NTEo6G56+Y80mqv9Z4gEOg+8aaPPgy6LYyT9UZcgUeYQ3RRDIyOMAI6MNx3Niz5AVJdr6a2ynCNVvdA3b7QP005nbL0z+ID5hnUuL0eQPuNhq4/lbEM2Rxf5eUhDBIzrYMQoI4m13cBdgRJHrR8f75PedXbWVB8O13vYG++qTIbZCYlSIPhMVaw6A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(8936002)(33656002)(86362001)(54906003)(6506007)(91956017)(66946007)(36756003)(66476007)(64756008)(66446008)(53546011)(5660300002)(6486002)(6512007)(2616005)(8676002)(26005)(66556008)(966005)(76116006)(122000001)(508600001)(4326008)(38070700005)(38100700002)(71200400001)(6916009)(2906002)(186003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8E0741153CA8824BA9561541E2F0371A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4658
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	59b4fb72-af19-4e36-d539-08d97f5a6d7b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fBmx3ZnKk3qkAfkhNRycpCRHKeM4NIuKxkGPejgyC6D4Y2KR3583l/g0WuY4/4boGdDrJQWwNLV4pLicmfVVnN8wDOXfiPhsFT4LBpNvM4GGdgdvSPAfiLLNyqHZ81r6s2K4zJGRhN3u6ivOr18IXsMl5vI614nIakqWb2gbKLOPOd6idh8ESMHoRfTifQFhmCX/t6BViVLQ2MzO357Hj3bC65VpOVUT4kGKm7wC9za/xaANtQgf12LH9dycpOtEeUSOz+olAjz383UNbWSRgIZUc9YclDHHRwPq6rjhAikOkwknNmYJozqpBc6W+81D9KbdLqzTmfip1lqUP1mP+u9or6gJBeM+NzG6NadV3dLj5CVCFCC6DHy0Q+f8NZBbpc5Fd2csLdDQPty8Wa/Hie03GclZ5By/Na3gJib+7D4iykoeUUszEoyzrPJL9i3/s5wq362LAekE6vLukvpX/A9BDuGkN3r7ClOvpn4h3fmoYoE0Vcy3/OzVojHdrPfATKKWAp1/h+o21xPxPQ/m3QSDpnO6f/lfYDLY8xsr37DH0COyjzCZGFUtILv1if1r8o+wpbexHVqnExljlmRhWYEwLAB8gAuAY31lJ4LjQ2dAF+BnLB3trycD8uln5/VKvruFT+jytZEMbaOsoDdaoIIeDDpaV76eKuVrqtav3sUOjU4aCbRdStquwkGH68vdeiIQPWXhYqy/uH8StKhnLfTOcVhfdBD/FWqnOdW+MOvkJru+vmIrH4nNXLnhNpb5tL8oUJQ/rsgT7keueb7b1wJhQI/X3FXWd9Kv8ylc+lo=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6506007)(8936002)(47076005)(4326008)(81166007)(36756003)(336012)(6486002)(53546011)(186003)(316002)(356005)(5660300002)(8676002)(26005)(508600001)(70586007)(2906002)(86362001)(54906003)(6512007)(70206006)(33656002)(36860700001)(6862004)(966005)(82310400003)(107886003)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 12:54:36.0383
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 122b33e6-29f5-48a0-9335-08d97f5a7678
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4456

Hi Stefano,

> On 23 Sep 2021, at 8:12 pm, Stefano Stabellini <sstabellini@kernel.org> w=
rote:
>=20
> On Thu, 23 Sep 2021, Rahul Singh wrote:
>>>> +            goto err_exit;
>>>> +    }
>>>=20
>>> This is unnecessary at the moment, right? Can we get rid of ops->init ?
>>=20
>> No this is required for N1SDP board. Please check below patch.
>> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d5754=
7087cff4ffc078dc515d5
>=20
> OK
>=20
>=20
>>>> +int pci_host_common_probe(struct dt_device_node *dev, const void *dat=
a)
>>>> +{
>>>> +    struct pci_host_bridge *bridge;
>>>> +    struct pci_config_window *cfg;
>>>> +    struct pci_ecam_ops *ops;
>>>> +    const struct dt_device_match *of_id;
>>>> +    int err;
>>>> +
>>>> +    if ( dt_device_for_passthrough(dev) )
>>>> +        return 0;
>>>> +
>>>> +    of_id =3D dt_match_node(dev->dev.of_match_table, dev->dev.of_node=
);
>>>> +    ops =3D (struct pci_ecam_ops *) of_id->data;
>>>=20
>>> Do we really need dt_match_node and dev->dev.of_match_table to get
>>> dt_device_match.data?
>>>=20
>>=20
>>> data is passed as a parameter to pci_host_common_probe, isn't it enough
>>> to do:
>>>=20
>>> ops =3D (struct pci_ecam_ops *) data;
>>=20
>> As of now not required but in future we might need it if we implement ot=
her ecam supported bridge
>>=20
>> static const struct dt_device_match gen_pci_dt_match[] =3D {            =
         =20
>>    { .compatible =3D "pci-host-ecam-generic",                           =
        =20
>>      .data =3D       &pci_generic_ecam_ops },
>>=20
>>    { .compatible =3D "pci-host-cam-generic",
>>      .data =3D &gen_pci_cfg_cam_bus_ops },                              =
  =20
>>=20
>>    { },                                                                 =
      =20
>> };
>=20
> Even if we add another ECAM-supported bridge, the following:
>=20
> ops =3D (struct pci_ecam_ops *) data;
>=20
> could still work, right? The probe function will directly receive as
> parameter the .data pointer. You shouldn't need the indirection via
> dt_match_node?

As per my understanding probe function will not get .data pointer.Probe dat=
a argument is NULL in most of the cases in XEN
Please have a look once dt_pci_init() -> device_init(..) call flow implemen=
tation.

Regards,
Rahul
=20
>=20
> If you are worried about gen_pci_cfg_cam_bus_ops not being a struct
> pci_ecam_ops: that problem can also be solved by making
> gen_pci_cfg_cam_bus_ops a struct containinig a struct pci_ecam_ops.



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 13:15:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 13:15:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195374.348044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTl2S-0005Sy-IC; Fri, 24 Sep 2021 13:14:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195374.348044; Fri, 24 Sep 2021 13:14:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTl2S-0005Sr-FI; Fri, 24 Sep 2021 13:14:52 +0000
Received: by outflank-mailman (input) for mailman id 195374;
 Fri, 24 Sep 2021 13:14:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTl2R-0005Sh-9N; Fri, 24 Sep 2021 13:14:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTl2Q-0001fk-MI; Fri, 24 Sep 2021 13:14:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTl2Q-0001t5-9T; Fri, 24 Sep 2021 13:14:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTl2Q-0000n5-8y; Fri, 24 Sep 2021 13:14:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3a1fmERS6o+52gszaJ0oRC0b1a9shplAWYgNYxwnY8Q=; b=bZ3usL2YwZNKAD8b5Ezi+tLRCB
	F/R4TdWHN88lCd8NF/kf9CWQviZtt/ZmCOTAYM03iVXog/KgcIrhJQGY793KI1ukD7ZngGZ1O2Yi3
	NkU6v3j/NlNupam3qk3UTmOKSHlfzWGsZ6MijJx54oggWLKQnytN05OYZqgRuFuzLGtA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165180-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165180: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=728998f6f2b7e1420e771236efec65cbf6143b7b
X-Osstest-Versions-That:
    xen=604be1b333b1b66052ab9b0133f156890549a4f0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 13:14:50 +0000

flight 165180 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165180/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  728998f6f2b7e1420e771236efec65cbf6143b7b
baseline version:
 xen                  604be1b333b1b66052ab9b0133f156890549a4f0

Last test of basis   165157  2021-09-22 15:01:38 Z    1 days
Testing same since   165180  2021-09-24 10:01:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   604be1b333..728998f6f2  728998f6f2b7e1420e771236efec65cbf6143b7b -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 13:21:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 13:21:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195387.348059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTl8Y-0007D3-AM; Fri, 24 Sep 2021 13:21:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195387.348059; Fri, 24 Sep 2021 13:21:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTl8Y-0007Cw-6g; Fri, 24 Sep 2021 13:21:10 +0000
Received: by outflank-mailman (input) for mailman id 195387;
 Fri, 24 Sep 2021 13:21:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTl8W-0007Cq-KR
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 13:21:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 46609bf0-1d3a-11ec-bad2-12813bfff9fa;
 Fri, 24 Sep 2021 13:21:07 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2050.outbound.protection.outlook.com [104.47.1.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-FncHXj3yOk6ymIOP23IG3w-1; Fri, 24 Sep 2021 15:21:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4846.eurprd04.prod.outlook.com (2603:10a6:803:5b::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 13:21:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 13:21:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0059.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Fri, 24 Sep 2021 13:21:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46609bf0-1d3a-11ec-bad2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632489666;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=fHgElt4qryXv37AnjsR2C/8raSqMWtIc6kDlGJKdS64=;
	b=YPelb7Hxv4vj1nMiHSowuO9ny0X5gWD1hHJkF7K1gartbOxO9hGBZkCkYnszzsYBueI53T
	nd0/0xRsawZplH6aadFcD2Z5oLWkpubrfUD9Z1VbkE6qT1YG5rIAgPIZu8TqziHxYVCrEx
	UCVRZuABHP8lGKenVIVWPMXJQVloLNE=
X-MC-Unique: FncHXj3yOk6ymIOP23IG3w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DKULslxvj4CBOp419bNWYTikUaUkY3MH5w8QOaKbikqEGHyM6SnwOeqHVOhw0+TqB4Ehcg35ZGo8WCNaNF9piPScW/cltxWaXX6QqlCxTTWdK7SIqBWsI3bpYZXsZ41+kAPeXcefpBVudZTFnu6KD1/j2J7QD+mEfvyMp2/vAYMUm5OTatVZk2bZoaCFM3M+aVKSyk44WtB0aVFjpEFZ54Z/NUr0g2T5uml0binvcnaK3spflwqe0s2/olYlTZxu8V74UIh/7IIQllHrQG9Z/wMR77ZDZlgwp2qEmZwzTMQtpsdM/07K8nbuRkyKK7xrm0Vw4HPseK93+w3h/306Bw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=fHgElt4qryXv37AnjsR2C/8raSqMWtIc6kDlGJKdS64=;
 b=XfS1BC66p8onJc3u5+0JWaBSPzMwXK0RVukSuXoMT8KY4yDUTGQ57BIFvQU4TffSgBGKhLIRF6ri43x0vED/rjHX/V/oLRRf8jES0i2VZ4uRoPMEQPbZjsI8MYxAIg5XZFu+FAZSJxRBImE0r2bX4SSnAfC62GI9AlMsu/dNJjFFdtqQIhw1Ynt2n0E8ujpUNdscDN+yrW1QO3yTNZY5Kb+1gtYykQ8MFe0uL32C75D2Np2ZKn/DzKE8XpDaAg8WRw+3kgjqfsxkenSRMzOHQ4xO2J36Indtvu9J9rr+nGW1W6s+UMKjKmhLm54l1/P1q9qtP8axd4rbftGne3tm1A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2.1 13/12] xen/trace: Introduce new API
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920192939.16483-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1eb8dc32-8e52-f165-b691-4b04de059318@suse.com>
Date: Fri, 24 Sep 2021 15:21:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920192939.16483-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0059.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bac5440b-55bc-45f7-968e-08d97f5e2921
X-MS-TrafficTypeDiagnostic: VI1PR04MB4846:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB484654F89BDE2D0122B56317B3A49@VI1PR04MB4846.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zNJbHSJfHrmBfPiJg2Qjv1YqXwZ7hfHc6eHv9M+zsZZ3Al6aaGYoipmfzW78SvxB+QK7lJ2BH2COUKPy8kqzJauueikQdqFQb2vBgkBQI0BIGmVA6PAon9sLRt0F5Nlr9mO52GSymrbI5zgPsdQT2ZOHFtcXGRe040Ah1/YE0RV6NOJ0C2sFSmDDbU+2LQiB7lAIxsjliumBxHoAAPXk0sn/3EyB9815qe/ZaiE8VEVzuUba+Tm8yES6moiByvhTa4mDzBBSeTYMi6hEOQgMUCxv4Q4pBIEDvjgMk6snksduFamNayxJzkVEixUN2RRVPH9V5mWd4EKh75QYeHdPbqlb5YcCjbYbJImLRcjToNcm1d0x3v/4xgIqT4leILpuqQFbVwLlwi69cJ4VwFUrC4sv3jjZW3DB2Au721V9tioN6VLoEXQHi/BlW7EZbSuKdWjs99T3imxIh9djTGmnWXOjZV2lAPCQ99isPvTP2qWd17cEyouZm6yaml3NpQrUYSY8xBR0b2Ots4OoqcdcUN6ZfhsSoymBDLbUixxPjChznaJsqlIMe1KiHAVl1JtZbLQQUTDrKwWY80vtoSUALm3azo9GMvDG59Z/gRKWv0b6ofyyeLTQtEY8QWaVeJwNCSr6Beg2gMR/eZwVl55zd71zlPzC7CbGNl/CoZTirdK4JJKFjn5v/XOjiRc0WmS7EV7qGN/dW2AEpwP8fOkvRsdzVOJ4uvsr/KXTRgjUha0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(956004)(2616005)(86362001)(54906003)(38100700002)(31686004)(2906002)(316002)(83380400001)(186003)(6486002)(53546011)(6666004)(36756003)(508600001)(26005)(66946007)(66556008)(31696002)(8936002)(5660300002)(8676002)(6916009)(16576012)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MTR3aXc5bnNDSlZDMkxvQnVqZlp5anlPWmhoNXo0eFF5T1VUWk5HOUcyNElh?=
 =?utf-8?B?L29mdENJS0xCblFyTEV0T25pNFNXYnR2aytITzFCNEN5T2VzT1UrRDdQVzlw?=
 =?utf-8?B?UDN6Y1pNOCt6TEY3MEY3YXk4cUM5bXY3a3NHa3JWWVdKTTN5ME1EZFVKUVN4?=
 =?utf-8?B?UlRGSnJ2OTJkU0xmZmlqZG8zcXU4d3piam1PS2dncEM2b25ETjBacW02Rm82?=
 =?utf-8?B?SGgxSmNPM29KNksreWRNZDVaUk0zZDVPYnB3Rnk5UGxwZzhOdEI4dTVGK1Mz?=
 =?utf-8?B?N2g0WGZIbFBCSlAwbGNrc1FSdExNLzg3Q21ya3czNXBTM1plZ1lMNzR3OEo4?=
 =?utf-8?B?L0I3amF5TUtycy92clFJNkh6VjczRDFxWlduS0RXVzRoNk5CMUo2dU80UnQ3?=
 =?utf-8?B?VnBPNmtKeUhMUFhDMjZlYkFlK1Y4eFRIT2dQbnRBWTdpR3A3RWxoMTB5bGhS?=
 =?utf-8?B?VTJoOFVPbWQvc3ArdDAwdmV5ZkIyczFIM3ZWQVFzWUpBekFFRWEwOE55RWVB?=
 =?utf-8?B?aEt4TmQ3QnBTbS93V2FpWGNKVGtUQTBNYUkraVRTbkd4bVJqRlBHeG9QNlJD?=
 =?utf-8?B?c2d4dmVyV2lVRHRzNGpxK2xQWGlib2FzREV4eEZ0VEh0cE5MV2Z0RUh3WElC?=
 =?utf-8?B?Q1ZydHVmVEo2TDJNUHp2dkI1N0FTeWJWLzRYM21vTGU3SThkN2t5eHIxK3JN?=
 =?utf-8?B?NFM3ZjZieEt1T2JhU1JFNU4xZFVjenNKbXFNQlNZNE1RMnlqYjR2Y254Um00?=
 =?utf-8?B?Ym0rQnpIWHBUd1BORlNhSTNiaVlhT3lCSUFDTTVibTlaV3BBOHM1Yld1VVlQ?=
 =?utf-8?B?Qy83bmJFTklvbGpPN3JPdmdNcnFoT3p5N2VSRkhTM1I0R2hiNjRkVWZLNElF?=
 =?utf-8?B?TXVtRmZoR2hPRDg3ZHE4Qkx1Rmo3c1lZM1ZaR0d6YjNUd0VWWE1ZMHUvWkwz?=
 =?utf-8?B?aXhvZDAzRHpvRVJxUlA4VjBKbXJKa0thM2l4K0RFaDArRGFYMjBRZzhmUDNJ?=
 =?utf-8?B?RmpIazR3VmZDVG8rSFFlMHNXSlEvRW8zcnRtNG15QjRxNW5ZVHZGRHF2ckhW?=
 =?utf-8?B?WldIY1hFWlI0U3hHRjV1Zml2cXhvOW1PODJ3bTdsYS93K2R0TTg2ZkxEMmRN?=
 =?utf-8?B?Uk9mUUJRMVBFeElMaHdBYVpKUzIrTllYOHFqaEo3cHowcDk3N2VYcUxpdmhr?=
 =?utf-8?B?bXROVnc3MkVLbGhmNkRpMGVJTUMyOUlXVEJyanVqM0lsdHNrNkJNc2kxL05I?=
 =?utf-8?B?NStqcEtKdFhiVitEeVJJZVRqSWFHdFRHK3d6cjFTd2svREVNWU1PVEZ1eUJL?=
 =?utf-8?B?SGY2TlRNMkFzdW91QVBoV1Z2SHptNDMxWk51OWJJM0xHS1g1UzNrYXV0L0ht?=
 =?utf-8?B?RFRhZUxTU3Q4bjRtWis3QjBxM2RGQ0ZsemhENTErSEZvT1ZpYW9icFdiUXhv?=
 =?utf-8?B?STViaEdGTVNMME5iMzBrTDNmVmp6Skw1Ti8wVldjclVBRGJIQitSMlhzay84?=
 =?utf-8?B?MXlVcTdWZmxGbmhDMkxvVlJablpGNkRMVlJsaSt2YitqRllBWlEwRkVvcUpl?=
 =?utf-8?B?M005V0p1VmY4WGoxMTFIOVdOMWhtT1VwbGpPTlQxMCsvWXZ3eUhET2dFYXls?=
 =?utf-8?B?VzNHdU41UUtWLzZRRDA4QkQyVWJXMTJuRHlYbTdXTTVycnpCRlczOHRiSENK?=
 =?utf-8?B?SnVuK0ZFSmliWDUvc0RsNWJMY1FwcjR4VlVMT3NoM2pGb0VwV1haRzJlZ1pS?=
 =?utf-8?Q?JBe7nX+kHioVLDyvlP7DAX6AjfVHW9GfR6cLexX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bac5440b-55bc-45f7-968e-08d97f5e2921
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 13:21:04.4975
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: O+I6nyyD6puD/CRxoSV8aKlvpzyVCj3NIy3OWi6NHUOzx8LAY7tfOEyxC88sqpy9f4hRCqAz1yg8KYG3xUhhQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4846

On 20.09.2021 21:29, Andrew Cooper wrote:
> --- a/xen/include/xen/trace.h
> +++ b/xen/include/xen/trace.h
> @@ -74,6 +74,30 @@ static inline void __trace_hypercall(uint32_t event, unsigned long op,
>                                       const xen_ulong_t *args) {}
>  #endif /* CONFIG_TRACEBUFFER */
>  
> +/*
> + * Create a trace record, packaging up to 7 additional parameters into a
> + * uint32_t array.
> + */
> +#define TRACE_INTERNAL(_e, _c, ...)                                     \
> +    do {                                                                \
> +        if ( unlikely(tb_init_done) )                                   \
> +        {                                                               \
> +            uint32_t _d[] = { __VA_ARGS__ };                            \
> +            BUILD_BUG_ON(ARRAY_SIZE(_d) > TRACE_EXTRA_MAX);             \
> +            __trace_var(_e, _c, sizeof(_d), sizeof(_d) ? _d : NULL);    \
> +        }                                                               \
> +    } while ( 0 )

I know we sort of disagree on this aspect, but I would really like
to understand what you (and others) think the leading underscores
are good for in macro parameter names. And if those went away, I'd
like to ask that the local variable also become e.g. d_, like we
have started doing elsewhere.

> +/* Split a uint64_t into two adjacent uint32_t's for a trace record. */
> +#define TRACE_PARAM64(p)    (uint32_t)(p), ((p) >> 32)

You don't have a leading underscore here, for example.

> +/* Create a trace record with time included. */
> +#define TRACE_TIME(_e, ...) TRACE_INTERNAL(_e, true,  ##__VA_ARGS__)
> +
> +/* Create a trace record with no time included. */
> +#define TRACE(_e, ...)      TRACE_INTERNAL(_e, false, ##__VA_ARGS__)

Is , ## __VA_ARGS__ really doing what you expect? So far it has been
my understanding that the special behavior concatenating with a
comma only applies to the GNU form of variable macro arguments, e.g.

#define TRACE(_e, args...)      TRACE_INTERNAL(_e, false, ## args)

As a minor aspect (nit) - iirc it was you who had been asking me in a
few cases to treat ## like a normal binary operator when considering
style, requesting me to have a blank on each side of it.

> +
> +

Nit: Please can you avoid introducing double blank lines?

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 13:32:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 13:32:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195394.348069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlIw-0000ZO-9R; Fri, 24 Sep 2021 13:31:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195394.348069; Fri, 24 Sep 2021 13:31:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlIw-0000ZH-6Y; Fri, 24 Sep 2021 13:31:54 +0000
Received: by outflank-mailman (input) for mailman id 195394;
 Fri, 24 Sep 2021 13:31:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTlIu-0000Ys-Ql
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 13:31:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f9c28bf0-fb40-44e4-90d8-abdbab65d47b;
 Fri, 24 Sep 2021 13:31:51 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2054.outbound.protection.outlook.com [104.47.9.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-F9I1osO7O3CTDIhUI8FDVQ-1; Fri, 24 Sep 2021 15:31:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4943.eurprd04.prod.outlook.com (2603:10a6:803:51::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 13:31:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 13:31:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0078.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Fri, 24 Sep 2021 13:31:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9c28bf0-fb40-44e4-90d8-abdbab65d47b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632490311;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wcPWF3Ey9MNZejsP0eOYbzs9p+6nxvxxT3zaAA/DYik=;
	b=UUmJ7J+ie3Rr3yq5Z9+JOmZJvAOM7eXVU93ld8TSG4SyLMFvwhSC9oE9sHn3ToMKdhY0+p
	et5Yy7X7wd3LX2KXgKnLbMPON5mI/Ie9WaWge17AyeToJ5COgjkQzHI3vfhMhocQwgyafb
	fGf3VSmvkmvsFl/9S3Z2w8v0aH0p8kA=
X-MC-Unique: F9I1osO7O3CTDIhUI8FDVQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dRBh+LHNf/otLKI/ih3o9SnjwsMXnAKw7lAHVi6s4ecdXF54myEp1zJF8qHJpMLOFjMxwKGwsYxlSSGtd9DLCMkAvpwPd4t9U3wvi1NA7mBU1e2DK3ehVWvAW8tmbtePXhBZ1UcZzwORsFrTe/BY4pbJK9FRTm6JGGpmllfl3ajy1ISDS3iXHs9CwnedpToJznP5VjL6+FXGON8sRINOwNfP5FKvnNjylfco0ZO3Hj20ENcrw0HjIhHgOugcRVzs9cH1GJC/Oe1oDH0wuPJraiPPz3EoZoFTlrKFFZlYUgtm17Rq77+745gRl4uYs9aOStRoQrWV8SuubAiOIYZANA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wcPWF3Ey9MNZejsP0eOYbzs9p+6nxvxxT3zaAA/DYik=;
 b=M9fQjPLdYYlnndjB/B+jbzRshzB3jngw4YQsQFHKB/BgIGCmTxVrDxmSs9aH3xaR2x/NaDmjMlIlt8QnjZm65CKzqsg+Vk8+qp6wvYDPeS37W9M085ee1iMQRPmkX5lOdJ0y0ME5k2EbgOUT4TkEswogPwHgu7iH1AtLIqkLtZSnxuYMm5u/wbg9s8iSz+6WhTLfwNmR2jIFefG+AgD5VW45iUAS/ni26buW6rti5gkkrAkJQdLA2ysE2r5ZDTR2bQhMwgu3lz3o3EQFL1ibod1IsRyGBT/gUAY0+B0B1StkFF5RmMpf9TdGfhxlHwvstgxIAGYyUmGwDNZHRNDuUA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2.1 15/12] xen/trace: Drop old trace macros
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920193302.20275-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8681fd36-2217-90f5-89cf-648caa47ce93@suse.com>
Date: Fri, 24 Sep 2021 15:31:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920193302.20275-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0078.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6bed2296-9b79-49ed-ae74-08d97f5fa888
X-MS-TrafficTypeDiagnostic: VI1PR04MB4943:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4943CCEE18324B350AE99708B3A49@VI1PR04MB4943.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1360;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	i0e+Gr8t7bD6Wd07TePdzf2inG6R+dvE86qOqCXQW2SF642vSwpTsb2gdHNPYw6VyxHMRqr/sG9LbMCRwNjRgl8S1ImDKS12xc/teJ1wsm9lpH9MQZHr9C5yv8/qaP15W6fEvU1nRxz+VKMZ40r52UynzvX5r6a22kMaQLeq6G5ht2Zn52ddkfaqX8Xv1eiHT+kjChZj/UWd44CooiIHCS4apNh7dz8w7TEnij9Icf6N+lTBWCk1Do4oZYQ3iM013keKQU92SwBfDN3jIfPAXuI/1Hre05esAM1NYkhta4OauIYQwEYV5Tfn2vMxu2BkFwd6zyyLXjj0l0EdbPB0msi5pHTmc03On2PP786hbOKHMj/Yah1XI5q05U5v+Xlb0F/p+pzDqS1oMffpIXXsL8h3N7Lh6Jk0Y1/hqCOxKjQIOE0uJZgIQtpBHdTCl7ShOoqoU/GupAGrskacF92qOLTOfoNBq7SF4axDiMe6Qf6L6mYA73rlErBMsvgObYRwn+/WNTbssXOivwh4V1v24xXXH/8JPfvf3JhGMMx35znfLdIJM/TroBQ39qLhi1J0nR4quaMc7q2Co41Iqiw92dPvhjk9W2vBE2tUfLgIswfwO/66rE+KjH60S+BVBUpD3AXLzv8asQYME8rfFH+Eov/vT7RjDa+sasWhfpDWt/dbzgr0jilPvD6L7apgnZRxcM3nJ1DS0LWago5YzjymNjRc/KN14Hqodb9CCZ6mTL0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(66946007)(66556008)(66476007)(36756003)(31696002)(6486002)(83380400001)(54906003)(86362001)(16576012)(8936002)(316002)(8676002)(2906002)(6916009)(31686004)(956004)(53546011)(5660300002)(2616005)(508600001)(26005)(38100700002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OE1yaExqd2paY2Q3ak8rR00yY1puYmlSUGNhQStoclF5VXpSZEZvY2xIcndP?=
 =?utf-8?B?RUtpaFRWNjI0WjFNLzdySlAyYTE2YkNDc2ZVRmZkMjVJZ1k2bjJHRHp0Vmd1?=
 =?utf-8?B?UVFROEp6bVFHWXc2emczeHBqR1pSb3RWRTBqbWdnQ2NhWEIxUWZPOHFjbStz?=
 =?utf-8?B?VFNBMXNmTFdMcW1aeE9zQkl2MTRCRTdXZEFiSjFNTmVxb0N4ZlFxcFhhUVg2?=
 =?utf-8?B?cDNWOE9QeWNGQUdQTEJ3ZmNPZS9LY1RaN1diSkpnTFBCcitVNkl4VTl5UGNk?=
 =?utf-8?B?WXJ4OUhMMnVPeW44K0JxT0U3SjZHVjgyZE1vMkh2QlE0U0x1eERuRnI4S3NI?=
 =?utf-8?B?OWlsdE5KYlA1eXZVYUQ1blhudzhwalhmYnNicjRDbkVtaVZOOXFtcEZuUU93?=
 =?utf-8?B?alNsUkVyazlUZ3NGcHhoMzJ5aHl3emRKMjFEN3o0Ym1EWmtHWlJIZXdnemRl?=
 =?utf-8?B?VWY3dnVOaWRvdHkrc0FYODV5QS85a0JNM1F5UEFKNXpQZGtxV2NnUkZFSzYy?=
 =?utf-8?B?K1NkaWZQdGU0emZWQzZ4cTllaVozeVUxdWxqenNIN3NGaW5XRkFGY1Roek05?=
 =?utf-8?B?OWtHZkRVNFNncWRFV1E3SjVBaitNMVhNbnhRTFdTUHQ4bVVIbXZkdkJkQ2kr?=
 =?utf-8?B?dVM1dWpkR3llei91VVFMZWhEQWV6N2x2V3RaRjNIMDFLd09yU09sSGtUeVp4?=
 =?utf-8?B?NS9jS2w0S01jVUh4M2YwRmtDVzU4QXVpVWlsMG9aWVBqUC8wMzhVRjlTR3lK?=
 =?utf-8?B?by93TFJteTVublBNRXdiQ0RSL3VSTzlmK1k0ZVJzMFZrdlE4c1lKTFFrRFds?=
 =?utf-8?B?dTNCWldaWnUrM1NZT3I2S2JlMWtCRkpDVjZOcHMxcjBtaUZ3S1Q2WFlpVWEw?=
 =?utf-8?B?N0JsZmUveHFNVlVUTWdCeGU1anJnb0ZpMTlzejhhL1A1RmZYeEVuTGpPVVVT?=
 =?utf-8?B?VENXbGJrTG5KZzNYV29SOFI0Qy9VM0JFVWlqWlc5SHBIaWJFa0tjdzlJNVls?=
 =?utf-8?B?RlRobGhmSitORllXaVVaLzdXWnNqSkU4c01Lck1rN1V3KzUvNTd2eHJMZllO?=
 =?utf-8?B?U2pwT3E0bHpsQ3FGd0gvNEF1RWl3NUxnZnUzUXdpQkF6TVhTbGF2OW4vdlFr?=
 =?utf-8?B?em1lUHJLU2ZKdXhTVmIwVXV1NlZvRGFYVnlNeFNINlAzem45VmdOYkJtQXJG?=
 =?utf-8?B?LzZyd1EyZ2NKaGFjdHhtOXhSZUpjYkVVMkk2TER6TCtLeEFwWHBROFNsN25J?=
 =?utf-8?B?ZDdjNmc0dnY0c2syL1BaVzlZaXlFUmhORnU5OGJ4WVlVUVJTaUNUelI2NFNx?=
 =?utf-8?B?VUZ4Q3lXWitxOXY1alBWRVNralYyRU5EQk0wK1NKNFJSYWdza1E2TEI3cndB?=
 =?utf-8?B?MTIyeVhPQkdVWkg3dzZhM0ZidTN2d05majVXRjFYMWxzL0FEQmh2Qnl4VGs4?=
 =?utf-8?B?Z05rYUx1SkdvK2wwU0F4WWtVajJTOE90UkV1emRPWHJzOHNmNlVQTXZXdExP?=
 =?utf-8?B?UnUyUFYrTEJ0S3BabkVpZVpiVDIzNFZ2bUFydlRJUDlISmhLSkcwcTU4cklF?=
 =?utf-8?B?N2k3bUhBbGdsbXpBKzVaQTIrT3l6SlJ1enF6NXk2dU5GemxFQkVucWQ5Y0hl?=
 =?utf-8?B?LytVUE1LeTROL2pQdTdHSE5RNW9VOENwVmhueWlNTEp6R3NTOGNSMlh6K3Fi?=
 =?utf-8?B?RGExVnBlWjFCMjZsSFJKTXU5Rk83ZjJyeHVlOHV2aVVsSGpWZ0RaeVl2ZHNR?=
 =?utf-8?Q?VZ75o+AppmNlPGjrZsoPUIas2bsRRWraF2Cr9sW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6bed2296-9b79-49ed-ae74-08d97f5fa888
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 13:31:47.7075
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e7bEUW/JCUuc9Ls6C3pXDyKPcBT6td1chXLaBirOELDeXQK3IlXGIwOLUdxt53Q0+LmfxNrRvHPHDX8Zn58Szw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4943

On 20.09.2021 21:33, Andrew Cooper wrote:
> With all users updated to the new API, drop the old API.  This includes all of
> asm/hvm/trace.h, which allows us to drop some includes.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
albeit I'd like to note that ...

> --- a/xen/include/asm-x86/hvm/trace.h
> +++ /dev/null
> @@ -1,114 +0,0 @@
> -#ifndef __ASM_X86_HVM_TRACE_H__
> -#define __ASM_X86_HVM_TRACE_H__
> -
> -#include <xen/trace.h>
> -
> -#define DEFAULT_HVM_TRACE_ON  1
> -#define DEFAULT_HVM_TRACE_OFF 0
> -
> -#define DEFAULT_HVM_VMSWITCH   DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_PF         DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_INJECT     DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_IO         DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_REGACCESS  DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_MISC       DEFAULT_HVM_TRACE_ON
> -#define DEFAULT_HVM_INTR       DEFAULT_HVM_TRACE_ON

... least the part up to here as potentially useful to limit trace
output. Afaics there's no replacement in the new model, as you
invoke the base tracing macros now directly.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 13:34:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 13:34:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195400.348080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlLs-0001DG-Na; Fri, 24 Sep 2021 13:34:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195400.348080; Fri, 24 Sep 2021 13:34:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlLs-0001D9-KI; Fri, 24 Sep 2021 13:34:56 +0000
Received: by outflank-mailman (input) for mailman id 195400;
 Fri, 24 Sep 2021 13:34:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTlLs-0001D0-2H
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 13:34:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 336a549e-1d3c-11ec-bad2-12813bfff9fa;
 Fri, 24 Sep 2021 13:34:54 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-_Y7-zfS1O-2kTtxSM_KuiQ-1; Fri, 24 Sep 2021 15:34:52 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 13:34:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 13:34:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P195CA0011.EURP195.PROD.OUTLOOK.COM (2603:10a6:102:b6::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 13:34:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 336a549e-1d3c-11ec-bad2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632490493;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vm0NoYzt0htDoXnmvDOfpIHeSYoMkw+l5gL+SKv2DPs=;
	b=msSaBZfdv5JzapX6Xwysk1fxUp1wzRSUFNFNlwrzSUKKSyP/gu/ImAYx1M1YLDqsWQDo39
	aFDOUCml4M+vit10JsXI0gn9rnOFK8dATXohd8PN8H4V7Vv+Tp67ddjd2fghoguV8Vp0Go
	v27TglNanQBd1wV2eNGuvm6C5q5Aw6A=
X-MC-Unique: _Y7-zfS1O-2kTtxSM_KuiQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RfAqsoJlrXPNN2UTA67MkPT0lxWpaePFegM6BNJGFnYYdRJc1HcK6Jq42kRAKXaRPZ8vmP/5fdp6SKpur3+sPEK2wguz3Xsg/knBxNqseGYenLySz42GlYJWKdCFh5uAOwuOEP9NgKcYnWfmHexU4QSYPENxlHKMJWTN8z/d2g2CHmOC72cLYdEaaVqSLSyiUgk75XG7MT7A+40HkOnGAboYyGwqT3yKnZ3ICSRM2PhnEhfOCXctTYwYGJJfLYU1DzsCIoLkdS3/2FKMaI1ZWXmAU/YEwtyGEVMDq6omxeInNuy70k55E2JIOCGxe4UF0//XPFZA2e15wz2Ho4p/yQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vm0NoYzt0htDoXnmvDOfpIHeSYoMkw+l5gL+SKv2DPs=;
 b=fMxzWPbRbgrrjPLRG0QPPc4LJ2tcbdLfT8vGQ8f0czuKrPQH+MUHRWG2irryzlN5HS649K9Gutm0DDfNvYUR6uHokQmIqLdnI0+w6ZHsijNFQGDaXPg2R8I0NlMwvwLUczGCdBbw48hPx5S3TFy7s+KI5kLpoFR5qu8Nouf/Bh95l1i0r/2rDps9uaJK8ojJo3MLcAVIQnzREph9xmCEMe4F8oh70cHczjiuR27p5J/OLpvAgkFalycXYcG5DNSE4cTe11Xzv24ykSIAr8xW2Hmh/8En1pnx2JOwBS0fJcbheSl51hFfrcFVQS5VPy6SHw2WAA20TCDv8jLLbLLObQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2.1 14/12] xen: Switch to new TRACE() API
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
 <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920193234.19409-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <34bf6a80-d306-87c5-8bb0-6fa1f189ea6a@suse.com>
Date: Fri, 24 Sep 2021 15:34:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210920193234.19409-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P195CA0011.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:102:b6::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4a1fff6a-8592-4075-cefb-08d97f60161c
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024A31B362397BB0423A9B6B3A49@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	e2lnrFOSl8IsMzBvNAYZ+6CYmWZxRUlx5fCUmVrqZMdpWR/2ZGBdYa8DpLpoJj9dfFG+NntV4bK0L8HBbC3g2hg/nZ/UBtY+mnLmupe19ENlI6eNJ5REdwassNCKZIH/D+roy19rnGzb1PgDJjR8QqaiHh6rKM5bVhm3gjN/trm6m0hh3kiMiZXUb9+57K87klDUPOvaLQaRXUmW1ztkTHfJ73Z8G08Q/2hCCd+WTKkI1uQ27I+SDsLtA3XUCrt3aFJ0FvHwqTmUE2InwBYVMtkOMNsbr4WAqH0QcaCimj5/pHW+wXs9W0ZQJxmCyYwsU5+duGOE14LpvR8hPMF96Buc0KyNg8gBWfa5DoE9aidaDb+Hae3iHThjrplEKG14PXQKPRGJwBgNpf4v+9jpbX3bs4+GnqGJ7oZHSxjlmoElYgCm0Zcsp2iwSknEMOWMiDhJKSJoEa0cRsMjz/uTWXGeMgMDLTmlW7A3fRNtuK0AQeF73OU+kUi8aa8eW07t9jzYV6F03BGg7Al1+e03tCnqx76fCdPFeahzfxPA4gpSop1lILMvqOlXDJaChmw9WsCarkBvgXymo4n6fbegzHqn7gkcJKH5r36uPt0MTtgCqVvgMb83c3Frwhmk87aq9Btk3E6mu9bXOfJws53E2YrhyV3w09tggcqDt3wCUvLjzfrbCn0nFdeSzKvqNzW/yXXof1nvql444BE8UVocwKuvX1xVNkc9VD7Qy+ijvkA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(54906003)(31696002)(36756003)(5660300002)(83380400001)(6916009)(53546011)(16576012)(31686004)(86362001)(8936002)(8676002)(508600001)(26005)(186003)(66556008)(66476007)(4326008)(66946007)(6486002)(316002)(38100700002)(956004)(2616005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Mi9yc1lqckdBWlBlaTRGVFArK2Myd1o1YktRSkRTMG80bW5mNTcxRFlyTnBP?=
 =?utf-8?B?ZEc4RVVnbWJnSUVSOWNGdWtQZ21pWmdIbVFnMTNjVUFPVnlhUmdHdXVTN01M?=
 =?utf-8?B?Q01qdllhRGMzUWpieWpTNmxvbldxZkM1dllDUXdVNStzbTh0UzFtc3VWN1Y1?=
 =?utf-8?B?bUtsYzU5UWtRZjRsRkZveDdzcUJMcnJFOHRESEZ1cHRrY2hNbUxnbUVzWGVR?=
 =?utf-8?B?QVI3WHRHQkJsaDMyQmdlYldxbkRCQXZPdnZSSzAzZFY3NzEyVmZibk1rOXYv?=
 =?utf-8?B?M1BMaDVycnFIUmt2MnA3KzF3Yk5Td0EzRkp4KytiUkwrV3ZESzFRUjNGUklj?=
 =?utf-8?B?UHBEZTQrMDZEcTFMYWI2ZU82WXhYWTRjWGpBbUR2WHdUNUJlN25XU0xreXVv?=
 =?utf-8?B?OWF6a0ZNRC9HQlZWYmh6OVFaRjl0eXduSk83Ymh6RlQyeVoyZzAxajZVdWQr?=
 =?utf-8?B?ejBiMGgzc3BXNVN4anR6VDBQWmVxMTdCZzhnem42dW5WRy8xV0pJOTVSYmUr?=
 =?utf-8?B?WUVKa1RvU2p3b09tV0NmRkIzcXorR0NJTEt3L29lNWsycmZFSldlT2pjZnNt?=
 =?utf-8?B?eDIyVGZES2Zwajc4TlJleWgvblV4Y2o5ekdyK1dNZG9qWjZLcDkwam5ONndT?=
 =?utf-8?B?Y0NLTUhEbklrR0l6ekpiNmVYVTE1OEpEOVVJbGkyTlg4QURrbnRyTzVlWTRD?=
 =?utf-8?B?L3cxT0g1aThBckdtbGUwY3NnRUs0d0VjZWVWVmxCSStyOHNRbGF3Yit6WUZG?=
 =?utf-8?B?b2UzbzhpWkRpNjR4ZXpiQ3QwZkpQQ2RvS2tHTzZiSnRkcGZSb0lmT1o3MElX?=
 =?utf-8?B?UDlrMDlsN3VVUkl5b1ZSaTc1YjZnWTlzbWwzOW1reTZkbGxybHJQV0c2QkJr?=
 =?utf-8?B?WjQvaW8wUmZ0S1N4YVRsQWVEaStGa3BTb3BNcUpOeCtMKzMvK09keEliZksy?=
 =?utf-8?B?ZnZMSDhMRk1jTFY4VEI5a3hhZHR6RXR0TlhJR1NuS0ZSZ1kvUWFFUzRZNDNv?=
 =?utf-8?B?NldsanQ5RFg2bDJVZ08wc1grRGJ0RlZvOTZ5WDhldFMvdExWRXRGdzBtNW03?=
 =?utf-8?B?YVJIbmx4c1crbTllRTkvV1k4Z1QyVXRmSkdsN0orcDh6YXdxTi9lbzk5N09h?=
 =?utf-8?B?c3oxdGNPVHl2RWsrT2lYZko3cEZwSVhaMXY3VUI0dk05Sjk3TmRESlBDc2pM?=
 =?utf-8?B?V0VZaW5DZzBud1NxUnA2RDVGR3laK2tmNjlDTmhzN0l1a3cvUnlFcU5wUGdy?=
 =?utf-8?B?WHUwZmxpYmZ1YWJaN3MxTzhWWmJiREpUVEpmRHNkNDByZWZmQTd5ajN3cHg4?=
 =?utf-8?B?eFFLWmVtWUFMSUtaOW12SG5OSnJFNEpqZXVqWkhBbGI4QjRJVHRZMkdTaEFG?=
 =?utf-8?B?VW1XcFZTRURaQmd1M1A1cCtGZHdrY1BFOGlyRElseUVtWkNTNTVZcUg2OHJU?=
 =?utf-8?B?dWdRS2xGaGYzb29NNm1ValRNcUlIK09nc0dXbkhDbVNCMkhMdk5wSkdGaC9H?=
 =?utf-8?B?RjdHOTgwWGdPUVA3RFc2U1FqTHk0RVlQUGJtS3ZUS0lQVDJhcm90TFdJVC91?=
 =?utf-8?B?ZlM2T2JVaS9ZbnlJVzVSU28yY1NsaElBRnBoN0VUZklSTU5qQ3dISmtyV2Nw?=
 =?utf-8?B?MHdDemRaM2tnRnBpcC9jRVk3Z0ZPWWw5Rkg3UDZESVp6Q0dvc2phSkxaalYx?=
 =?utf-8?B?bWZ0KzliRjZSRzNwN1B5UnhVeTVJeUkwcE5UYVk4R2FJc2daM3JhV1dHVEV6?=
 =?utf-8?Q?eMXlHU91Ls0hnWTECpf7yvz0gfZp2qsaJkwA3U2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a1fff6a-8592-4075-cefb-08d97f60161c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 13:34:51.5229
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zV4qeOuqOV0KsCF7FdoWtd9OGRwLA8SdDH7E4MjlOmvxKO/oUzGJ8RfE8bAmjJMk+k2+6Bp420J+qyZlXtf/xw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 20.09.2021 21:32, Andrew Cooper wrote:
> (Almost) no functional change.
> 
> irq_move_cleanup_interrupt() changes two smp_processor_id() calls to the 'me'
> local variable which manifests as a minor code improvement.  All other
> differences in the compiled binary are to do with line numbers changing.
> 
> Some conversion notes:
>  * HVMTRACE_LONG_[234]D() and TRACE_2_LONG_[234]D() were latently buggy.  They
>    blindly discard extra parameters, but luckily no users are impacted.  They
>    are also obfuscated wrappers, depending on exactly one or two parameters
>    being TRC_PAR_LONG() to compile successfully.
>  * HVMTRACE_LONG_1D() behaves unlike its named companions, and takes exactly
>    one 64bit parameter which it splits manually.  It's one user,
>    vmx_cr_access()'s LMSW path, is gets adjusted to use TRACE_PARAM64().
>  * TRACE_?D() and TRACE_2_LONG_*() change to TRACE_TIME() as cycles is always.
>  * HVMTRACE_ND() is opencoded for VMENTRY/VMEXIT records to include cycles.
>    These are converted to TRACE_TIME(), with the old modifier parameter
>    expressed as an OR at the callsite.  One callsite, svm_vmenter_helper() had
>    a nested tb_init_done check, which is dropped.  (The optimiser also spotted
>    this, which is why it doesn't manifest as a binary difference.)
>  * All HVMTRACE_?D() change to TRACE() as cycles is explicitly skipped.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

> I'm in two minds as to whether to split this up by subsystem or not.  It is
> 95% x86, and isn't a massive patch.

Either way looks fine to me in this case; splitting might allow parts
to go in before you've managed to get acks from all relevant people.
If anything I might have preferred seeing e.g. all the HVM*() macros
getting replaced and dropped at the same time, rather than the
dropping (combined with others) getting split off.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:02:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:02:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195411.348091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlmK-0005Sc-Ve; Fri, 24 Sep 2021 14:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195411.348091; Fri, 24 Sep 2021 14:02:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTlmK-0005SV-Sn; Fri, 24 Sep 2021 14:02:16 +0000
Received: by outflank-mailman (input) for mailman id 195411;
 Fri, 24 Sep 2021 14:02:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wINm=OO=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mTlmJ-0005SP-SB
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 14:02:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 04dcecc8-1d40-11ec-bad8-12813bfff9fa;
 Fri, 24 Sep 2021 14:02:14 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2053.outbound.protection.outlook.com [104.47.5.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-9dRITHmvPAqyJVSyQL42bQ-1; Fri, 24 Sep 2021 16:02:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6670.eurprd04.prod.outlook.com (2603:10a6:803:120::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 14:02:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 14:02:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0041.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Fri, 24 Sep 2021 14:02:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04dcecc8-1d40-11ec-bad8-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632492133;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aQsxUjV1AEHF730NqCI/i760oUoFSMLxw6RtHJFXWJY=;
	b=Hfd70CiNurDVTrJ2ah67EgrZgKH/fjYPAhHUA/w/JQ1GWHaB6V6SSXnIGoLk7X6Kijk7M5
	/LIwQ4F0og2rWVSSUBFGGKoXoAOANJLfI0PQYssxk36MWp7ITgF99ZHC/08h8ulXF6Vu9F
	4vFo7CAbbtRFp9i4LtpoFay9nMWrFQ4=
X-MC-Unique: 9dRITHmvPAqyJVSyQL42bQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HNSUJTScYY8FaZBHbABUxAVY00jAH8rscEi4FuQ7gCYLhWmfwrjJF4+WG0P7mqeUGhQMSaadYtxzb3OHzb6I0uGD/IjVyUcldaL5Koz04As6cIn8FCbV8asB1wHbm9aoo21vr7qLcDgnrZdxwuQGs1jzBq2Yacf3BTCmprvYINdBuzLyiNAoykFNfE9/34EQ836/WwuThMA9eQ446staS1RX5viE+3KnJQ9V50eYcrvwthjOfAdx22OG7AoxxfMF9Y9xdNvqOAjY355GhmrMKrXedvArMwEFICNcqgJGpqoCSprfQuutqu1QSzixtsAHvDF9uUp+kexiARxbY1XAaA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aQsxUjV1AEHF730NqCI/i760oUoFSMLxw6RtHJFXWJY=;
 b=Jq724m4Cho2hSd7AkN53EF6RfxQArXzgA4uwnCVyfIjsSQDjCBk1w6c91eDUZzzi5qdbX62lF0Cu6X3DqK6B8ZAYrDv3ztn7zTpYfP6Q6wGdmqZxkvGrLddY9y4jArvg0Q9f2uz81P0ZuWuux+G4LYvurKNhAqjunY2rtAlyW2IpQVndvMK1S2VAOmUuNHs4D9xRcjnDSw/GtkOxzo1SzdoaPJ35Jbb8xD5Ao+eJI+kUidiFKJlHD7GiNRblhaG8OmeYSu+2jyf1lQmaoHXti9Jmx+f+Oxspm6WxgREIf1T0b8ZIvaV1yuTbczBu0XCt/T/JQBPAEh3mm7IfyDMdbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20210922141341.42288-1-luca.fancellu@arm.com>
 <20210922141341.42288-3-luca.fancellu@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1318fbb2-f7ae-e7e8-0e3a-9dcef69fb430@suse.com>
Date: Fri, 24 Sep 2021 16:02:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210922141341.42288-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0041.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4a::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 99289bae-cfd1-4171-1072-08d97f63e66a
X-MS-TrafficTypeDiagnostic: VE1PR04MB6670:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB66706F4EB969C62FA21836ECB3A49@VE1PR04MB6670.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2150;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cFPQMDVi47icNofIErhESw39KvIfs2cm97U6/i6XW7zyBQELlx24aoOXx/TzBss2x1ZMvs0Pg1oNM+Jy5nf01WlMTmwd1ZteYOup9zD3YasG+kOR91hUfl5BX3Ext7bgzL7GUl0RlMS6eOEg7XtBcqoEQUoAHLAgpM7/mhQ+D4mScHbFEDmZhB4y4ciM3cTv6Y8nqO1XMFkbL1FeS5O3tfus1rzC1BlScYksuZi+1iUQcmMvu3ylY4zf1eZsrFAtxg3IR8H6fSmQ71QuCruAlC0mIizeYzu9hS52+COYbBTBOX+HHUyQrGknZOWLu4d+wO33ubeaGMQsoLx7lF2Nv2TMFz3Vg0jKdOeBTopBpy8lODpIt3SAqeRhKRJXXejDHGqhdjDGyU38l+KGffYk6tMskZrZ3OKtVVksM4Vcnrhl4N9Sh3tl3KqB6I/YCvi7IAWp7kb1AuVn2MAnKdTd0oqrAeAYz3d62nggupAuCpp48mYZBPHLyEnfTQ5jo1HJmP9daHYE0Ray+wGQTznqv2qe735ZFDmK/TblJwp60Yl1Bm6nxcF/M3h1s+idNq53rR8H8wuQVJnsQQL81kqEn4wHydM1JZjU4y629+mWc1SJTZR1IlopwOKZK09OnM6VetuhTAOtzM9ooDa+rS4FpbR6PBq5d7i0sRxXs1i8aOnXcC1/mgb3Vs18ytkwC15l9UIfTkxyq1M8ILebd3wtbSnp8/E3/YQ5xYhjXPGLb4cxNO4AL/k6OJ3NPdmnKBDA
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(4326008)(66476007)(66556008)(31696002)(36756003)(8676002)(6486002)(54906003)(86362001)(16576012)(8936002)(316002)(83380400001)(956004)(6916009)(31686004)(7416002)(5660300002)(2616005)(2906002)(53546011)(508600001)(26005)(38100700002)(186003)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UklpbkxOWk1oR3BaOTR2SXR0L0xwUmlTeUF3RThtZWxwUmQyditYTW1Yc3JN?=
 =?utf-8?B?K3NyUWNILzhpcFFtZStuNkN5YkNNb3FGQzlGOXQ2S0xWZEtqSjY3cUJNVm1U?=
 =?utf-8?B?SWZOeG4wcURad3owNjFmMUZYdEI0VEtGOGJsTHZQUlY2SWlhUHByVFRXOXVJ?=
 =?utf-8?B?RmxwQkZhdlhIZDY2T0tScHlBMXBHY0ZISmFUb09IcnF0MUVEdjA4enlJUU9I?=
 =?utf-8?B?cU9WWnFkR3VWRUgzWUpXR25NRWhyMmlCTDFBeXVWMm8zbjRVQy9RSVgzbVQ0?=
 =?utf-8?B?Sit2T1dnRjF0OTVwZVFRaWk3SHJPR285enNBZVJ1RnJycWVSUUhLZkJCQ0FV?=
 =?utf-8?B?ZWxEMURzb0JzbHRud2hPT3A3WHM1eWhZVGpOV2w1TTBxMUh2YTdBdG5FSlZ6?=
 =?utf-8?B?ZCtMWFBrU1NxVzR1dnJnWUhHYWtWd2hveWFoaEVQQmJSelpEYVV4OUtwYlpZ?=
 =?utf-8?B?NE45ZnU1U2lnSVlJUzJTQW85V2s3T2hmNVV1eVd6SHhWY1QrdDNJM0IwRjdh?=
 =?utf-8?B?VWtBN096UUFnc3ZmMFBpcms4VmtMZi9xQXkrMzBBRTJZY3FkSnQ4QmMybmkz?=
 =?utf-8?B?MUxRT2RCN3EwdkZySVJ1MURYV0xISVZqYXIyK3lEWlpGM2Nld2I0cEhvdVVl?=
 =?utf-8?B?NW12c29iekFLdzRzb1ZHQ2F4dWVwc1dBS2FZVDlQWVRWbzZXaHFxSG1JNlJE?=
 =?utf-8?B?SjNud012Umd0VDd2cG1tc2wrYUkweURQRVpaOEpCaTd2RytWQWhzYmx4VmNt?=
 =?utf-8?B?N0luNVYwdTVRVmpqNm05aFZySUZBTEFRaUlrTEYwVVFOcU1FN1d1ZjlZYzVp?=
 =?utf-8?B?Tkp2OTgySzczUTNkaHVudnBxQkFDbG1zOG9WQWJvTzJWSFM5RFNOcE16eFN2?=
 =?utf-8?B?ZDkwRmN1dnphWThWby9SUHdxOVBoenYwMkpWZXE2QnREb3JrRWYzeHd1S1BE?=
 =?utf-8?B?eWtXbW5vQWxDbGNxUkJPbzJDTWJ3MUdMbjVCTU1SS05YVVNmdWRVd2J3Nkwy?=
 =?utf-8?B?TjBHRjg0cWxtdzFSeHFZQm1qa0V0RlRWNG14NFNJUjczWS9QUzJzY2g5empz?=
 =?utf-8?B?VURXQjFlUWZDSU1vTHBEcGp2VWhTb1FEclZyWUh4c2o0T3h0OWd0SzRIMzln?=
 =?utf-8?B?U1FFaW5RTGQ5cHNmS0xTM2dGNFdpK1NMOWp2cTgrdThBOUVBYysxRTRnQm9O?=
 =?utf-8?B?b3ZjTW1JOHFvZXpyemtrZHRSeGtxNTNlUUdLbTd0ZEFuOVo5bzRFVStTb1d2?=
 =?utf-8?B?bVllcjFSZ0dNM25RaUc4d1RZQ0Flb1RyU0FRbmlXWUh2MUhXM3VrT3VHWFVp?=
 =?utf-8?B?dU1BdzdUUTBMRzI3VzZSYjRmS0tlVFNiTUNXSVoxR2h5NTd3NVNxQUZUY2tN?=
 =?utf-8?B?elM1em9uOXVmd3hrdWszK3ptb1ZxZGlEUlRNaVFWby9vUnMxbmtPOS9ubzdG?=
 =?utf-8?B?c244QkozQlA5Qnc3Y3hRNDE5QklmTndCNnZhQmdmYys4ZWZuYnV4OFd1Rjd5?=
 =?utf-8?B?aC9UODU0eTNXQzZHaHBZTExBWlg0UDh1OVdWMjdxODNWQnRGVEdyZ2ZPanF2?=
 =?utf-8?B?Zk82cjBBUTYrTmU4WnRaT3BiWGpjNWhzWGxtVVJBa04xR0RreXRzYStOTHlR?=
 =?utf-8?B?REpCQUtSRDY2aE5FbGwzamt1cXhFZTNHdllUTE1wNHZIeDNSUnBuRzdiajhV?=
 =?utf-8?B?cEl2MWRIalVHMnNJTnlJdHRENGlGaVRtcUlaKzQyUjU3NHh0ZUJLUHE3bHdT?=
 =?utf-8?Q?qXTwIV1hoqBfk3ujf8NKgRqjOEK9JeEPYn1TDgf?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 99289bae-cfd1-4171-1072-08d97f63e66a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 14:02:09.5065
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xddhm/NQ66pVIA9ZTvsVJCQZziYRVTZuGOT03PM6uUXPBpyFmWlAVjVY/nNLzwYB5OtUyvm13RtxF7p7gtkZXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6670

On 22.09.2021 16:13, Luca Fancellu wrote:
> +static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> +                                                  const char *name,
> +                                                  unsigned int name_len)
> +{
> +    dom0less_module_name* file_name;
> +    union string module_name;
> +    unsigned int ret_idx;
> +
> +    /*
> +     * Check if there is any space left for a domU module, the variable
> +     * dom0less_modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !dom0less_modules_available )
> +        blexit(L"No space left for domU modules");
> +
> +    module_name.s = (char*) name;

Unfortunately there are too many style issues in these Arm additions to
really enumerate; I'd like to ask that you go through yourself with
./CODING_STYLE, surrounding code, and review comments on earlier patches
of yours in mind. This cast stands out, though: I'm pretty sure you were
told before that casts are often dangerous and hence should be avoided
whenever (easily) possible. There was a prior case where union string
was used in a similar way, not all that long ago. Hence why it now has
a "const char *" member. (That's still somewhat risky, but imo way
better than a cast.)

> @@ -1361,12 +1360,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !efi_arch_check_dom0less_boot(dir_handle) && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");
> +
> +    dir_handle->Close(dir_handle);

So far I was under the impression that handles and alike need closing
before calling Exit(), to prevent resource leaks. While I will admit
that likely there are more (pre-existing) affected paths, I think that
- if that understanding of mine is correct - it would be nice to avoid
adding yet more instances.

Jan



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:36:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:36:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195420.348103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmIk-0001Cs-L7; Fri, 24 Sep 2021 14:35:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195420.348103; Fri, 24 Sep 2021 14:35:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmIk-0001Cl-Hw; Fri, 24 Sep 2021 14:35:46 +0000
Received: by outflank-mailman (input) for mailman id 195420;
 Fri, 24 Sep 2021 14:35:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mTmIj-0001Cf-4H
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 14:35:45 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b22de1aa-1d44-11ec-bae2-12813bfff9fa;
 Fri, 24 Sep 2021 14:35:43 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-lqQHz9IiPIef5FUVpSO7Cg-1; Fri, 24 Sep 2021 16:35:40 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB6930.eurprd04.prod.outlook.com (2603:10a6:208:186::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 14:35:37 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 14:35:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b22de1aa-1d44-11ec-bae2-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632494142;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=gwEnhWffpKikd6V7ogwpbryzoiQJzAI8CfaIrrWolWU=;
	b=loEd6vpuUwDhwdXx4WCFlZsGZKmbhzTpgnXAtWD7wKylmKrCsucXkOOetTp0apKZrx0BBX
	Ff+OTvt/G07onbfGHN4fopy4urE7GJ7ClQnaUNvRO1DPKM6cRW77Da7A3kNQq1EqdF64z+
	dBJqIpar3gy+kvEKIUBiQA4mJLXJHa8=
X-MC-Unique: lqQHz9IiPIef5FUVpSO7Cg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hCNqIGr6p6gL9osUJl95idQYvXMFMsb8M6mMnNPfNgWfNQIhOLPI1jglOREMximPWDi8dvseuWsLp11i8/fGg/8v+PwIgVq8piTw9etfoYl/XvDAoya+VtiUOm/Q8CzRpi4mb3rR/y0QZKqv5NwbMJCs/4IHAHpdomotuGU24mJmTtrBc/m19Rdueyu7MPs5ZzpRupLfrzOsbTRCG3PHInzXjNgcolYLPGH8buS+AqvPVDg+XARz78KbNs2m3Z9qFSm2iyJ0jvsHVKJ5tqmF5OU1/7uTOPEbnaQhWKibR/50v6sDX2xcZF+mLjVq0zn8q9aTjfRBVz4xtXPExLqUcA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gwEnhWffpKikd6V7ogwpbryzoiQJzAI8CfaIrrWolWU=;
 b=CEzyCVPsV1263893sqW8UUAznLGHTk819LJAXLM2sO6OwzdI8Lcdi+bOs0vCqYwH5Rk3IPKL+1WFS9D9rQyJJ/BV/pbP4NgNh1d8w5t2pOzPVajgZNfX52pIthKSxp/FItKhPsCa10L280af/oAOzCgsoS04U1jPHStDJUd7scrAjca/9y0l+vy/Ds8HQQrjYIRqViBHA8IUBH9plajixRi/WRseBy2uXjbeArj9bY1eIZ8cTomWXvPn0FiBpOfatZWlCUKH7JPd2YrW4wPaU6TI1meY+Bvt4tTW1X6V4NVQ2GXBSKfuc4OZTeqpGtuA7qctPvY9wze8E3ZisjLaXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: Jan Beulich <JBeulich@suse.com>, "andrew.cooper3@citrix.com"
	<andrew.cooper3@citrix.com>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "iwj@xenproject.org" <iwj@xenproject.org>,
	"wl@xen.org" <wl@xen.org>, "mengxu@cis.upenn.edu" <mengxu@cis.upenn.edu>
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Topic: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Index: AQHXrkSLbMGUvlBXkkK2FQ1vuFXyFauuDj2AgAC3ywCAANyhAIAAY+SAgAM/yQA=
Date: Fri, 24 Sep 2021 14:35:37 +0000
Message-ID: <aa5151723de33c6c871fc98432d720ae3317711e.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-2-andrew.cooper3@citrix.com>
	 <731e3474-77bc-b11b-41ca-5dff57831264@suse.com>
	 <171e6f01-63cc-e453-7268-1bf2ec6fe7cf@citrix.com>
	 <69770706-85f6-6509-b5c4-bda61dfdf994@suse.com>
	 <84b780c0-7177-d1ba-fabd-9a6258985f9d@citrix.com>
In-Reply-To: <84b780c0-7177-d1ba-fabd-9a6258985f9d@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b4f7b562-f152-440e-61cc-08d97f689367
x-ms-traffictypediagnostic: AM0PR04MB6930:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB6930631A6C9DC97DA9973B75C5A49@AM0PR04MB6930.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 Q7wL6fmPIyKetq7t6QqVLOobGmns2tsbNNw6URk1SPCwN3ZyiHSYyiKI/l+bYCIL0+wOhOMQooU+GMufv+VqP1WZj3L2vxBmclCKCY7YYovE16XiFX6Vg1IgJjLYKF119Msdyfr0429WhUHnSw97XEYeOjidN+2K6L7MIlkGpF7B2SX/zcf7HAVVoKTM0HvVnTGaxa1QG1Kyk9HfZm01ayECeeX9ugbwsWfsR8S3HFqlsDXbvPPt4IX3WF6m964KVlAZ/cdF01QSHnWMYgCpesLFxgw/9PKsFNfTNcI6uAJe8pWnZ7NleRg3I/KxnHJu+UT2krYSZP2nOCo+wASYo950ze2viYHyK1IWre3IdO55H7X1hK/1ldMlSF1moZuvxhuvs6VAP9JJZ2GsoJi9vtR4bgQ173o5EsZ22hu3OWf1b7+F0Rw15fmWb5aP4r0ptTfmqmRZ1uTo23NCKFD1xWGk8iokyx/Z0+KQu3U1H9QoBMmUMZvAe6Kh3IiFT2nSljq0YqrKTfJHw2xKG95dWSNOsYBeK0JkOlrT8xXE0jdK3/4qqPKHG1kaKpdRVKw3p2fMQF24akp3JsOrZAegtyfT//5ffUMtUuvafBXnctcAH1pRyvOhz9g0RISQmbsWUF+4HTCF25mO9QuR9NO2jKCTjHDWXAlMOt2Wslb2AQYfuBOLE0qUPNCT8N2DsWeQwSO1HztGZqREnbkgNqKTaS3phC7KGJWKIt8vaum5eyBraV8gW0sil8WWoJmP6nPOPlB6AIs2ZFqpDWZHVL2UUzcGjYjMi5YpwAW60wg47YFt9DbSprjpxBYYJ2/EWiie
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(86362001)(54906003)(38070700005)(2906002)(99936003)(8936002)(71200400001)(316002)(966005)(186003)(4744005)(8676002)(53546011)(6512007)(6506007)(122000001)(26005)(2616005)(110136005)(38100700002)(66446008)(4326008)(91956017)(66946007)(66556008)(64756008)(66476007)(76116006)(66616009)(5660300002)(6486002)(36756003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?MElTcnIydG9PbG1GbERsOFAyeW9naVdqaEZIczF4UjgvOUljVTlRTlRnMW1h?=
 =?utf-8?B?WTN4bVhoOVZtQ0JqNEREN3RGSHh2ZU9hOXJzYkxudlJLSERzQnNzM2UwZ3F1?=
 =?utf-8?B?REZRYjNTVnUraWkyR3VkRFBIbHFHR3BkbkpDZXdjVEhGcWZxbXJBMFZ6VjUx?=
 =?utf-8?B?OFdMVHkxdmpaZklMYTRLLzlmSXFqbzZlZXBvYUdib1RBT1hhdm9scmhRbnlG?=
 =?utf-8?B?MVNic21zanhCblhBQmxyNGZDcS9mbkdUaHdxNTRaRGFxWFhRREVqdFZwWXFU?=
 =?utf-8?B?UWNZcVVacm00RkY4Q25QalBVRWNpUkFuOUp6RWJSK2dURE41UDZBbThjemZ5?=
 =?utf-8?B?ZG9qMisyc1dDOTRtajNMa1hsa1BZZzl6TEpMRkFDNnZBcFBYeVpha0tHeFBk?=
 =?utf-8?B?ZFUxNkJ0TVBqNis2KytUTFErQk03aGdleGJzSk9lSGwweGxsc2M4WnFtWnJX?=
 =?utf-8?B?Qngra01lNldUenUwMDNFc3BROXY5MmYza3phZHJlUFBjeVY5NTlyWlhWSkcx?=
 =?utf-8?B?Ym1Qemh5QXBjUitVZlNhOVNpU0xqdFdaaENhL0pLMTgyVm05UFU1K0M3R0hl?=
 =?utf-8?B?U1RDR3N4NnFheEJ3MGhNS3JyWENCa3FpZmFxcEorb0g2MWRvL1oxWmRkeHBL?=
 =?utf-8?B?OUM0akUyODROakZIQU1FODhXUGFDK24vTlZjbkRZUEl1eVlscVNVNUJkTk5K?=
 =?utf-8?B?cnN5WVE0T1FvS0dyTGxiVTNFOXBubE1VeS9JRXVYWWxXSTZHeDJhaGphQVY4?=
 =?utf-8?B?dHZ1WUx4blEwRnhCVTRIYWR5T1ZLRXhUR01oNnhSaytvMlM0MVpWM1NsWHNX?=
 =?utf-8?B?enRhN1dmWG9iY0ZTcmJLTVJTWmlSTlhtbmZpN3VjVDFNb1c5WUtVYjc0bHAw?=
 =?utf-8?B?ZWNSd3ZBeTBGVVh0MnFjbS9Gc3dYOWtua3NZMkhaWUlXUW9QRHFKYXpORlJK?=
 =?utf-8?B?RHIvajd3RUErSGUzcWlGSWd3MFV0N0MrNDVhMGxYNWJOVGVZR0JmaEszdnBq?=
 =?utf-8?B?MFVHdXp6em5QbkxoSkNiK3lUZnJmN0QyZlczclRNdTVBd0NnclhWLzNmcUdk?=
 =?utf-8?B?c2Z3NE1VK2p2djVWaTNRSFFxMWwxeDFjcVJqTjYvYmZlNXVESVdYR096RUYr?=
 =?utf-8?B?WXJTbVVNeFJ2bjl6RDRDQnAwMkhOVllDVjdFZk1aMzU4MnB3Umg2czVNcnRG?=
 =?utf-8?B?YS9HL1M1aHFmSUQ2eXYvYlZ0U21xdnVBajg1Z0o3Y1JOSGNrbm8vUGJBRlFQ?=
 =?utf-8?B?am1uL1B6elVucXVVUWNmbmJRQnZUSDFwbHl5Sy9WaDF6WW9sQkJpMERqODdZ?=
 =?utf-8?B?dEMyUWdaV01mM2ZRdWxxeEVFWEtNbW81R0dJUFV1bXlwbDZKeWdxNUlIYytF?=
 =?utf-8?B?ZEZCR0tGaUx5c0IzS0RLQThMQk9WdHYwZFoyU04rQzl0SE8yWUVNQ2REZk5R?=
 =?utf-8?B?bitIWHVpR1RBbGtWbTI4SVBiUUVVNWwvZzVHd0J3LzVpazBIMjVHYTNVUFFn?=
 =?utf-8?B?MDFWa2VFZGMvVlc3Zk5wdzdzbkprMklIZEd5Y0J5dHJXZHY4RkZyZDV6Sk1i?=
 =?utf-8?B?Q24rSE54cnBZazlMSC9pd3A4YU1VNEo5bElleGdOeGt6QWRoQ3RYUEFPdlV4?=
 =?utf-8?B?bENXOGFiYTdLZW8zaWMxRi8ra2RmNnVRVUVnekw1c3NMUmhHWDUvclJZWnFn?=
 =?utf-8?B?R1FHQzRYTE1ydW42dmVRMVYyRVpWcjVIbVl3T0tqVnlHMFdMUDNrRFNsZDJH?=
 =?utf-8?Q?B2bRUajTpwBdbsPOmyG2tatiViUUm5m4gMsre1Z?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-7oF2mRctqEMgy14sO+Rw"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b4f7b562-f152-440e-61cc-08d97f689367
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 14:35:37.5238
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ROv0vNOpTo38O7dTu52mg2p9tlMtVE8Qq8fiaAsJLbZd2tSZvEMvgjPi69eFIWYA2rCKbbt/L0YYOzYj0bnSkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6930

--=-7oF2mRctqEMgy14sO+Rw
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 2021-09-22 at 13:58 +0100, Andrew Cooper wrote:
> On 22/09/2021 08:01, Jan Beulich wrote:
>=20
> >=20
> > Agreed. Whether the truncation is an issue in practice is
> > questionable,
> > as I wouldn't expect budget to be consumed in multiple-second
> > individual
> > steps. But I didn't check whether this scheduler might allow a vCPU
> > to
> > run for this long all in one go.
>=20
> I expect it's marginal too.=C2=A0=C2=A0
>
It is indeed.

> Honestly, its not a bug I care to fix right
> about now.=C2=A0 I could leave a /* TODO: truncation? */ in place so
> whomever
> encounters weird behaviour from this trace record has a bit more help
> of
> where to look?
>=20
Sure, that's fine for me.

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-7oF2mRctqEMgy14sO+Rw
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFN4jgACgkQFkJ4iaW4
c+5xHg//TWZKxoGYFy4Ile0SpjrjVtjS3wr27iEx0CeIbYcTS0+w0o2NsaQSc0dm
8tactDRsrXV4VI8zxbobHejgxGeTWnXMIX8imRewkS0GilRdDKDTBcsGqDi9e6V2
0LjfjSNSpCjZe2+ICgf8F/JvWuOwCbsrQ2bypIXgBrs7oJIb2tujkelyNXGmDPEu
NlrWO9nAfhc2rSeCWrNW91Bir30+ymz3EgFLnVMz3NJG8LLnwCzkacgCn+ABctf1
+2TdW9i5BfkbdzTs0ON9/cHy7rj9BYep2cTbDt+Y7YRsYc2RoJFNuMDtWwTCXwUg
9bn1WsS9EdQsQU3kW2jHN4pT1K/rC8F1PzcStF8yDVWXas3tNysgDvJVOMOuk1pV
19dJ7QINrLo5iy1/NHMuhl3gmiOzNJv1oRILrcxqhaCKm4XV7oKiL17q3Xdw8EZ3
uPJ4ttfTkmjaduzYuS+MO3Z2RzF6iDLNhPvLnoDknfQBNrguK8fDJ/HFEju7EC5n
oq/lYCfNfTHvVX/yl/NlAKBZdw/NoVwHWbxtA+/VvPaBJ2UMrT3h5Drhz9V6k0zy
WsMZS0byrrxwR3EYZER2UuY2zMQ9zHUQqngFtHfaUjp92nitMumgPt705UyJZ3wJ
G6BfKf3mKuH0CeUGxrEPYTxndpuBXqAltZSjPM9A538jOyD5hnA=
=l/Gi
-----END PGP SIGNATURE-----

--=-7oF2mRctqEMgy14sO+Rw--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:45:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:45:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195426.348114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmSE-0002yJ-Ji; Fri, 24 Sep 2021 14:45:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195426.348114; Fri, 24 Sep 2021 14:45:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmSE-0002yC-Fi; Fri, 24 Sep 2021 14:45:34 +0000
Received: by outflank-mailman (input) for mailman id 195426;
 Fri, 24 Sep 2021 14:45:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTmSC-0002y6-B9
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 14:45:32 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9a73b4ed-7b95-4807-946e-b48f7cc2af13;
 Fri, 24 Sep 2021 14:45:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a73b4ed-7b95-4807-946e-b48f7cc2af13
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632494730;
  h=date:from:to:cc:subject:message-id:references:
   in-reply-to:mime-version;
  bh=AJ5YqHnRCrIpw6uryiaWZ6V6v3mtHei2MaDKjURtxGA=;
  b=aqEMr0tye7lGyngd4tw4pnpO8QCDikEd70OLUUSzdgjRpQU8iJmVgvFo
   Ws3PwxPeS9NHQWKRxHdfWYfseu23NAyUnb7obckeaBvqRUWckTXzsHBf2
   /spwj5U0VVnwlAJ4KbOGEMY/8koM+lPBqywHQRp0CoWsbikIclaBI3gkP
   s=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: ZcmUGWPcInpnDb5NUTQf0gL/MaC0R3+gLOrMSIs6rHQK05QjoI4aysboCC2jk4Q6GA9CnTYQZ+
 3YYv3PU2tnUE2TPN3zdGdefVvN1R4JPpqabd+uKIWc8ywPgWgeGupNRn0/5JT6oJvILngC+Y65
 SnFYm4krubZG0FIxh+VwTEjZ/ss2aB9e65mZeH/HDgRGAiGimjnrjX5t9CNfiWieRcxdbFWdYm
 7JY61tZml2lnoZAdCcp6ctLBCRlJkfIcm6f5QMBPQgNgY+iMAE3hix0nyCqsDItUzES1upa+n4
 loHbsP91JdeA/3bmzc1CDIKx
X-SBRS: 5.1
X-MesageID: 53541331
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:7M/Y9q+TZQLkypaVWYdBDrUDcXmTJUtcMsCJ2f8bNWPcYEJGY0x3x
 mZLWWnSOP+JazeheopyYIq08hlV65bUyddlTAJlqig8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j3tYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhy9
 PxEjd+Ccj0sJ4TPvLwWWRlTFgZhaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguw5K8bmJsUHs2xIxjDFF/c2B5vERs0m4PcFh2ps35sVRJ4yY
 eIaaxxdYFPxTyZPK1MMJZgOzMazhSjwJmgwRFW9+vNsvjm7IBZK+KDkLd79atGMA8JPkS6wp
 W/Y/mK/HhATMvSezyaI9jSngeqntSHxVZ8WFba43uV3m1DVzWsWYDUGWF3+rfSnh0qWX9NEN
 1dS6icotbI19kGgUp/6RRLQnZKflkdCAZwKSbR8sVzTjPqPi+qEOoQaZj19QdUe9/0Nfzkz/
 1ytx9jjBTdql6LAHBpx6YyohT+1PCEUK0oLaikFURYJ7rHfnW0jsv7cZo08S/Pv0LUZDRm1m
 mrT/XVi3t3/mOZWj/3TwLzRv967SnElpCYO7wPLVyqO6gpjbeZJjKT5tACGtZ6swGufJ2RtX
 UToeeDCsIji7rnXzURhpdnh+pnzvJ5p1xWG3TZS82EJrWjFxpJaVdk4DMtCDEloKN0YXjTif
 VXevwhcjLcKYiDwMfUtPt3sV5hwpUQFKTgDfqqPBjapSsItHDJrAQk0PRLAt4wTuBJEfV4D1
 WezLp/3UCdy5VVPxzuqXeYNuYLHNQhkrV4/savTlkz9uZLHPSb9Ye5cbDOmM7BohIvZ8V692
 4sOaKO3J+B3DbSWjt//qtVIczjn7BETWPjLliCgXrTYf1U6SD1/Ua+5LHFIU9UNopm5X9zgp
 xmVckRZ1ED+lTvALwCLYWpkc7ThQdB0qndTAMDmFQzzs5T6SYrwvqoZabUterwrqL5qwfJuF
 qFXcMScGPVfDD/A/m1FP5X6qYVjcjWthB6PYHX5MGRuIcY4Slyb4MLgcyvu6DIKUni9u/whr
 uDyzQjcW5cCGVhvVZ6EdPK1wlqtlnEBg+YuDVDQK9xedRy0oohnIiD8lNEtJMQIJUmRzzeWz
 V/OUxwZufPMs8k+99yQ3fKIqIKgEu1fGEtGHjaEsebqZHeCpmf6mN1OSueFezzZRVjYwqT6a
 LUH1ez4Pd0GgE1O79h2HYF0wP9s/NDovbJbkFhpRS2Zc1SxB7p8CXCaxs0T5LZVz7pUtAbqC
 EKC/t5WZeeANM//SQNDIQMkaqKI1O0OmymU5vMweR2o6Chy9buBcENTIxjT13ANcOoraNsok
 bU7pcobyw2jkR57YN+Jgxdd+3mIMnFdAb4ssYsXAdOzhwcmor2YjUcw1sMiDEmzVuhx
IronPort-HdrOrdr: A9a23:ymFZAagHz88wOmsiu6zXedEsOXBQX0513DAbv31ZSRFFG/FwyP
 rBoB1L73DJYWgqNE3I+erhBEGBKUmsk6KdxbNhQItKPTOWwldASbsC0WKM+UyEJ8STzJ846U
 4kSdkDNDSSNykKsS+Z2njBLz9I+rDum8rE9ISurQYccegpUdAa0+4QMHfkLqQcfng+OXNWLu
 v62iIRzADQBkj/I/7LSkXsGIP41qn2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB2+ffv7ChPaHb3WLtB1B5vzke+D3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,320,1624334400"; 
   d="scan'208";a="53541331"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ESjf9uV81iq9sG523Ttlli6B43SRoVYawd0tTwjkzEcRPTPSi8BInrjqa543TTNsxhfVqxvGdrS1XRGiQX4Nnj7gQPkv+9NEwieiYzo0Z1l5WWffj2AERIsq9y4rGAcTnHhbCTgxTe5ZGT2yHHYc6/DXeObCZcgPsnBH1gAclmzt/pOeg9q4yFZfjRXknTnSQe+nZI+sD0F87g+3RRbQJp9PSHg33vVXqoGqMEyKtT47zLLdC6BGA7/XhTH9Z08JjGiPJbl+hIxnvP0gb1O9VIYazj8K5/k/2IbOBzOOzL4q58IlZAQA90fW1AbGwuZuU2sbERq83UH0QULbA0CUyQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dDQWpOYS7KFKCF6NT5OFbYwfuoN6iff1ZjYEVf9qt2s=;
 b=F9+660ItLvAM4WcUuClnAHbv/NDbGOY7BKqfX7HI1KgTAAoE9pKAaE9/gxR3BBjCFYnQUYdM46nbatAUcWzaGTBYeMcNbPzGZCyCuMnAN97pnw069XgXpgnNwsKnNHVfVqdhP0ZDk56Kk4oOf6ej6Aoifvk4tVnvewLOf8TGeYRM9wRAIh+xFJmvJZMTgaaXHOCDkL2n0qHXAI6ztmE71rgK/clK9Hz+y3OZO5z7oelHqfXuJeyxJ5CEJxjumGzACz5YggyLyhBfAhEzjy7xWlHBE9oZjNzsFsQC/PbnH+XvEEgix/8nPV2jb0uy+SESoareGYt49mm6amD7SeNcng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dDQWpOYS7KFKCF6NT5OFbYwfuoN6iff1ZjYEVf9qt2s=;
 b=MGfSce8VPN8SbXqzr548EcZUxR1ZYnBShbAdF9GQQpU+XO3Lt56G7E6AVJ41qj8E7ywDAG+ez44ZUOxGZOC2EpOJ1tAzKMI7bUenS+U4ud1Tq/CZm5gOAp+P9oB5z7TVbCX83OAIzzTXLSsBCp7ZsOr7fpOdgbI/g2tNKWpyeWk=
Date: Fri, 24 Sep 2021 16:45:13 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>, Kevin Tian
	<kevin.tian@intel.com>
Subject: Re: [PATCH v2 02/18] VT-d: have callers specify the target level for
 page table walks
Message-ID: <YU3kedkf/mSsGcpD@MacBook-Air-de-Roger.local>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
 <6d01a401-ac53-d4ca-e0ac-165d045341af@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <6d01a401-ac53-d4ca-e0ac-165d045341af@suse.com>
X-ClientProxiedBy: LO4P265CA0028.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2ae::20) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3af6e8ed-0385-40a6-0bee-08d97f69edc9
X-MS-TrafficTypeDiagnostic: DM6PR03MB5322:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DM6PR03MB5322D00597944380D4B668898FA49@DM6PR03MB5322.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: UEfZwjSEwgxI5sZXe3r4LWEFBCU4WqcjvMOj/5A8LVEI2y9wrX7x84uYVRarbkycvvumsHrWoI/zmTIVNliEmpH5PHJSWgc9hbRnDUox3OxsKpSm6DnkVjslCAfXlmTg6G3zPoqzMU1OsEump4+QD4aPzi5j7ay/aczeKdGv97YIT5TKShWVEJrgOyd2njhMVoskW96C8Am3fqBvimVVtRYUfAKko0rHnSsAWrSo1Y5fYrYXJD/K3Ps8fwRzCRKNCjjmnKMtFYtbhbIHHLYlKECIVPbTlPimqoNKuXez6Y6qSBLse3IxgYFhwveBCWe/2GQnJx6Q8AezJ3LX7QLsW8S8NsFY57yianXPcBB3RlKjwfwVokzFH5JkiyjrSP6RnCV2AS1eP7QHC4Sy29LVeGq+hlElt/Hy0N4kbgtLD03FfBXVd2IcN+uu+K6XXAz9xeX/EmjgvKcEjIYvLRpN9LHGdsq2RW0RMHhqz6q5s6rH+u2WGRV7Q93/iZ5jaE/NGXl/npRW1L4b1gEamel9FTtNl+VjQFprzMYVQIeURL8P3YQ3eQtEWH1GF3gqWgZ5BxooobxAe5VUrbG3nN/LsqtGVoItXJ2Vzk2VeNSzUGXSKSA8t5fp+UbidZLnJacVMeYoOfP0HHLzlPyJg1BLCA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(38100700002)(83380400001)(66556008)(8676002)(316002)(508600001)(85182001)(86362001)(4326008)(9686003)(66946007)(66476007)(6496006)(26005)(186003)(2906002)(956004)(6486002)(6916009)(6666004)(54906003)(5660300002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dFdDWElNUUtwTDdBVnFRYS9oMkJSZDNQSnMvZmI2cTNVTEpKcnpWRisxaDRk?=
 =?utf-8?B?ZHArRmZyMjcvd0VVYmdJdkNtMStDQTNXTXFyUzdJanVMRFBnTGNXcjY3QXdI?=
 =?utf-8?B?NU9GZWpFSm4rR0I4Rm94MzZXeFRvbjdWcnVGVy9PcXdvbkljTGpsYllzczd6?=
 =?utf-8?B?NnVybGJ0TXNjS1RGZ2NoQ3YwY2p2TVJNVkJzTkYxZGRmTy9mSkJpZVErd0FU?=
 =?utf-8?B?Zndzb2h3YzUraXp4dGlXMkNWVTh0NlE3U3lDRXRLY21XVnFMQk9rWlQ3SEty?=
 =?utf-8?B?dFp1RzRqaCtNOGlkWXgvWHYra2hsZ044MWIvMExJTUltUHp1RGxGdkxRcTZC?=
 =?utf-8?B?ODBaL21MbndUaWJZcCtnYmowVXdSSE04VTBMbUUrMGV6NnI0ZE9QSDJFWWlO?=
 =?utf-8?B?ZWtMaktZYjZiK1VnMFdKWk9Pc3F3M1FncTZPdTlDUEgrZHIzV0ZLSCttVWlt?=
 =?utf-8?B?dkZlZ0l1cnJXdTBRYk8vNVBla0dyYmFrWGFVRVJGRHZ3RUkvOXpqb3JWdmVu?=
 =?utf-8?B?cTJrUHgzTVFMWVBuS1ZSMHJISm0zcHRmcWFqNWNtcXk2SVQ3SXVQQi93NE1E?=
 =?utf-8?B?eUp2R0JZOVIrbGhPQWNOM2c3MUxFRGp0SkZBMzN5WkVJc3I3cWFOUWUyMHFX?=
 =?utf-8?B?cTRWR2pMeENJVW9GWS9jZ1VoUmZwVWZaTysyNjU4Z0x3WVdRQmhkTkRYeEc2?=
 =?utf-8?B?MU1KTG56Uy9kSVFQRGU1TDlJbDBDVHIzM0EzN212WmlJdmhlanpqdEFBanhG?=
 =?utf-8?B?c21FeW9OTkdmQ1I3azdSUjRGcDRBVWg5STJhMDBZdElENHFEMEFNc0pZWUhM?=
 =?utf-8?B?c0FLak9pNDBGQSt0WStUZzE4YVJqWjJZblRTS1lreC9TNkVrK1l5NXhoaEhq?=
 =?utf-8?B?Y0QyQ2EzUWpWVTFEdlZ1WUFrWU5nUWdmYVBLTHkzM3J5UjNidjExWTlVNCsz?=
 =?utf-8?B?L3Fodm1DWFAweXU4N05pTXVHTzlCT0E2Z0QyOWVNd1BrWnIyNE1pVno4K3BM?=
 =?utf-8?B?azczcHhFZkNVeXJZdEJlUWlrRk1UWFJqeHVHd3p4cUxuVExsa0o5Q2x6OU5L?=
 =?utf-8?B?dzB3ZGxCdzY5YnBYLzFhc1JyMm1hMVdQVUZCTVRobXNrOU9HT1JMZ0xvcFJW?=
 =?utf-8?B?dFBsM1VJODdScms4NG9keGQ3eHAyWFl0cndZWjFWZitLUEhGNlZIWGt2a083?=
 =?utf-8?B?emhmaGdaRTd6Ris1THBTR2V2TklvNkVXeUJDckV4dzZBdWJnNitXQnRUK1lz?=
 =?utf-8?B?alJxRjhCOVVuSFZhQTdwU2xLSllIRGhpdHZsRko4YW9EOU1BZ3JmV09zQ2t6?=
 =?utf-8?B?bG5BN2ZYS0hFYjhjdnZGaXhkaEhPTUJ6V1hyTzM4TXVyYmFQUUI5RnZEejB3?=
 =?utf-8?B?K09QdmFQVVNpUlAxWjlRbStzMGgydE5zV2RlY0k0SThSQnoyZy9QQTN4bFJX?=
 =?utf-8?B?TzBUdzh2K29IK3FMUXYrTHB3bi9UZGNaaWpPdEFMQTZSMlJ4bEVIRS8rUzlL?=
 =?utf-8?B?bTJGS2FQaUZ4YXB2dHZLTE52ZnM4NUxJa1h4bENWSzFYOWFTYXQraVQyTmpQ?=
 =?utf-8?B?a0ZEZVhKV1F2cUFFVmc2QmhvY25xWXVsbHhPNnRDWEhqRDBWVHovV29nTFhO?=
 =?utf-8?B?MUxmdFloeEprU204K2Q0WWJLSjg5bnhMQm1tSmdXQjZPTE1PamNrblNaWUFm?=
 =?utf-8?B?TmpoclBjVm9xREV0RmplemdSQUhzcVROUFMyejFlU3BicHdQVVJIdEVnOWdk?=
 =?utf-8?Q?59XHDe3Skhu/JlZsgTwjwRjkMQAmOreyfkdB6Xn?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3af6e8ed-0385-40a6-0bee-08d97f69edc9
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 14:45:18.8619
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gPSTnePVwbTWtrZ+Fh7vz+HT48M3DusVy5R7GPXDrV8iMMQAMtQ+VJf5jpU6fhh4Cb4DKTbDXFmeD+SAw7+x7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5322
X-OriginatorOrg: citrix.com

On Fri, Sep 24, 2021 at 11:42:13AM +0200, Jan Beulich wrote:
> In order to be able to insert/remove super-pages we need to allow
> callers of the walking function to specify at which point to stop the
> walk.
> 
> For intel_iommu_lookup_page() integrate the last level access into
> the main walking function.
> 
> dma_pte_clear_one() gets only partly adjusted for now: Error handling
> and order parameter get put in place, but the order parameter remains
> ignored (just like intel_iommu_map_page()'s order part of the flags).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I have to admit that I don't understand why domain_pgd_maddr() wants to
> populate all page table levels for DFN 0.

I think it would be enough to create up to the level requested by the
caller?

Seems like a lazy way to always assert that the level requested by the
caller would be present.

> 
> I was actually wondering whether it wouldn't make sense to integrate
> dma_pte_clear_one() into its only caller intel_iommu_unmap_page(), for
> better symmetry with intel_iommu_map_page().
> ---
> v2: Fix build.
> 
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -264,63 +264,116 @@ static u64 bus_to_context_maddr(struct v
>      return maddr;
>  }
>  
> -static u64 addr_to_dma_page_maddr(struct domain *domain, u64 addr, int alloc)
> +/*
> + * This function walks (and if requested allocates) page tables to the
> + * designated target level. It returns
> + * - 0 when a non-present entry was encountered and no allocation was
> + *   requested,
> + * - a small positive value (the level, i.e. below PAGE_SIZE) upon allocation
> + *   failure,
> + * - for target > 0 the address of the page table holding the leaf PTE for
                          ^ physical

I think it's clearer, as the return type could be ambiguous.

> + *   the requested address,
> + * - for target == 0 the full PTE.

Could this create confusion if for example one PTE maps physical page
0? As the caller when getting back a full PTE with address 0 and some of
the low bits set could interpret the result as an error.

I think we already had this discussion on other occasions, but I would
rather add a parameter to be used as a return placeholder (ie: a
*dma_pte maybe?) and use the function return value just for errors
because IMO it's clearer, but I know you don't usually like this
approach, so I'm not going to insist.

> + */
> +static uint64_t addr_to_dma_page_maddr(struct domain *domain, daddr_t addr,
> +                                       unsigned int target,
> +                                       unsigned int *flush_flags, bool alloc)
>  {
>      struct domain_iommu *hd = dom_iommu(domain);
>      int addr_width = agaw_to_width(hd->arch.vtd.agaw);
>      struct dma_pte *parent, *pte = NULL;
> -    int level = agaw_to_level(hd->arch.vtd.agaw);
> -    int offset;
> +    unsigned int level = agaw_to_level(hd->arch.vtd.agaw), offset;
>      u64 pte_maddr = 0;
>  
>      addr &= (((u64)1) << addr_width) - 1;
>      ASSERT(spin_is_locked(&hd->arch.mapping_lock));
> +    ASSERT(target || !alloc);

Might be better to use an if with ASSERT_UNREACHABLE and return an
error? (ie: level itself?)

> +
>      if ( !hd->arch.vtd.pgd_maddr )
>      {
>          struct page_info *pg;
>  
> -        if ( !alloc || !(pg = iommu_alloc_pgtable(domain)) )
> +        if ( !alloc )
> +            goto out;
> +
> +        pte_maddr = level;
> +        if ( !(pg = iommu_alloc_pgtable(domain)) )
>              goto out;
>  
>          hd->arch.vtd.pgd_maddr = page_to_maddr(pg);
>      }
>  
> -    parent = (struct dma_pte *)map_vtd_domain_page(hd->arch.vtd.pgd_maddr);
> -    while ( level > 1 )
> +    pte_maddr = hd->arch.vtd.pgd_maddr;
> +    parent = map_vtd_domain_page(pte_maddr);
> +    while ( level > target )
>      {
>          offset = address_level_offset(addr, level);
>          pte = &parent[offset];
>  
>          pte_maddr = dma_pte_addr(*pte);
> -        if ( !pte_maddr )
> +        if ( !dma_pte_present(*pte) || (level > 1 && dma_pte_superpage(*pte)) )
>          {
>              struct page_info *pg;
> +            /*
> +             * Higher level tables always set r/w, last level page table
> +             * controls read/write.
> +             */
> +            struct dma_pte new_pte = { DMA_PTE_PROT };
>  
>              if ( !alloc )
> -                break;
> +            {
> +                pte_maddr = 0;
> +                if ( !dma_pte_present(*pte) )
> +                    break;
> +
> +                /*
> +                 * When the leaf entry was requested, pass back the full PTE,
> +                 * with the address adjusted to account for the residual of
> +                 * the walk.
> +                 */
> +                pte_maddr = pte->val +

Wouldn't it be better to use dma_pte_addr(*pte) rather than accessing
pte->val, and then you could drop the PAGE_MASK?

Or is the addr parameter not guaranteed to be page aligned?

> +                    (addr & ((1UL << level_to_offset_bits(level)) - 1) &
> +                     PAGE_MASK);
> +                if ( !target )
> +                    break;

I'm confused by the conditional break here, why would you calculate
pte_maddr unconditionally to get overwritten just the line below if
target != 0?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:51:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:51:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195433.348125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmY0-0004kl-7l; Fri, 24 Sep 2021 14:51:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195433.348125; Fri, 24 Sep 2021 14:51:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmY0-0004ke-4q; Fri, 24 Sep 2021 14:51:32 +0000
Received: by outflank-mailman (input) for mailman id 195433;
 Fri, 24 Sep 2021 14:51:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mTmXy-0004kY-K4
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 14:51:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b9e4161c-1754-41a6-a8a5-ef51a49bebab;
 Fri, 24 Sep 2021 14:51:29 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-nI-OD5Q2Ol6IEAX7tEF2Xg-1; Fri, 24 Sep 2021 16:51:27 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM9PR04MB7700.eurprd04.prod.outlook.com (2603:10a6:20b:2db::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Fri, 24 Sep
 2021 14:51:25 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 14:51:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9e4161c-1754-41a6-a8a5-ef51a49bebab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632495088;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=0ne2rZslJgOXXz3fWsZIIWBdbnwrHNB68NV7aproGxE=;
	b=BkVI4kEw4EnWit9EpxllLY3LK/OI+3Ay5vFLOq2TmDbhohhWYmgQPDtdgpPy5iGzpGfsN6
	26O+/NXh14DkICvE6d+grTO8tpN4pcdSlR6aNxVaJCnO0nD8v6YB9h6mVuPPBe3vkU+QsQ
	Wd142MpLp8l6asbL5XzJw2wfWAFa5Vs=
X-MC-Unique: nI-OD5Q2Ol6IEAX7tEF2Xg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gUsUM9nx+uYoiJn1rLc5AwG2dd01PZ6JldoGGVL2lBP3/ewTz5luAcTwNAlXhjq0jESFC9Ga+npBabDvm/slXeH3JJKV/01RWQx++V1kxDvzZsQSpx6cSes8Q5It39w09LKkIa4iD3Iv6+AqKEWTP/KjaL88ZPlmhl06qQTd6NSZ3rwUIYN9zNkERQoeRARwbqnjFyQsIOvC2wgn2KEVfXXex20CKQTPsCg6o1NxPIrjVsMe5YEJB6pqgEsRN/OoegTW9YVakEWYW837g9uksqONxCFj5QkzJIHNWY9mOq9JlvDHhIOw9GTwC2oPWIJUSc9K+OPH4jXd3xtcCIMAGA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0ne2rZslJgOXXz3fWsZIIWBdbnwrHNB68NV7aproGxE=;
 b=BVFuzv/S/Fy20on8iswfiKiVXYv8RAHP1cNqyaXMxpLIEDh4mbAFnghasIzaBxgUgl2Z3kNrjH5w+Ya5pe5DOycuLANPGdD3534nxxJ0eE/F+FBnrffygjCgmrQZW3diGz8xQVI7DHzUBgvaAZyz7cj/g0Mfdt8N91z4O0gqeDep3D/gKMoqT8+7L7LRZl2sfsocjJrW6V97Q/iI4f0K4tPBfRFCk5wLmI2f6yUuhdLpGbhBcbxviaP2vBxTPGHSzZjjbC33G0c2y4X5hIQ2EGMM0htMJOqjFARDu10pTgElFN8hpqnD3sJwHGbLfRZ0O2ya0TiAEemqdUeVg/C+Lg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>,
	"mengxu@cis.upenn.edu" <mengxu@cis.upenn.edu>
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Topic: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Index: AQHXrkSLbMGUvlBXkkK2FQ1vuFXyFauzSsKA
Date: Fri, 24 Sep 2021 14:51:25 +0000
Message-ID: <da06dfe42adad13650755650518a232dd41ac46e.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-2-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-2-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: eb51e165-532e-4f23-c050-08d97f6ac8a9
x-ms-traffictypediagnostic: AM9PR04MB7700:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM9PR04MB7700C61C8740999592F9BC99C5A49@AM9PR04MB7700.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 tiQWRd6+4T/5DQk8qdWbhklcI06KMt+6M+SynfBaMBQMAKGMFgYZXrKSbdjAPcAF/Htf8boDp2+7akjHYTOEDZz10pvygQzk5kYL/Y93bZivw+pnvntNI/PATZ52428uGi6KZJY2NYICDI0p5q/gp2tXsTJ9bh+lv92pnLGVRULTmDa4sq193jO38vAlj3Dd9yWfzMUdSwzFx/0yDnBCOYMfBaPqPiKWlzAYtZ6GqmG2uWLE6QH4G2MHgkFS4XcdS4pOO3OoT+CSzsPfCd4I5cYMQW+RmhVgCXfihfCmNgM6o0V778/Uxt38IBBl0es9mkPr9t+FATcy1B1TvjBqBvly08puGXrwE96Cn9vukghk4NC9eDipImt5AqJVR1LowBdPXrigJEE4D4WddwSzmqwngCBFEJf+kGFxSZZknc1YPeHckyFSOSWGppwxBdQC60Eei5L25Q8BKaHxEhW+Koi4HGMyzg46wyXTM/dpavTkzVFRAOL2I//xjlHGERlMorWUTfYx3MlG7H6CYlKS/ckrUk3D6N3UNFL5ZeVN+D89tvaeOeIZPLjMgPoXExlvsIOzazXHt4t7d8vtKQNKrelo6UUlxbcisVyohp2KoGO0BOipBcFDaEbf7KV+ngwvXktosYMLmBOsN6yDP+hBH/ASlTGq3zs5rT/7i+G5oSccAYnRhSeDLWhfLnHsVUbE7Om14zhYu5soVhXpL/C/kwjpNmLAli9lnzZrd8BVbwc5SCbC0veThgiP636tcIoPdbnBBdrptJhICRIZrhDpI4Y/SMVuEUoyFOQFxnx+X9jud1Ht2EZPIkB/tzfvpi4glJR2/6at8H74k7LCrC5ssw==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(110136005)(86362001)(122000001)(83380400001)(2906002)(91956017)(2616005)(38070700005)(26005)(8676002)(5660300002)(54906003)(66476007)(99936003)(76116006)(6506007)(186003)(6512007)(71200400001)(508600001)(66946007)(66616009)(66556008)(6486002)(64756008)(36756003)(316002)(66446008)(4326008)(8936002)(38100700002)(966005);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?K3ZoeXd3UDFwMFduMjNqei9YdXlLZHhwMjFKQTE0bEFQQ21xY0hsRWZRMUhJ?=
 =?utf-8?B?aU5zNVJPY1lNdnpwWG1oSGQ1T1o5MktwRzFDZmlwTG9RNE41QUh1TTVKajds?=
 =?utf-8?B?S1JHVE9YWEtSdnhJSk0xWUtHV25LWTBKdUM2UDZQRTJDbHE3MDh1YW85RWQr?=
 =?utf-8?B?OUdpb3Njc0p6Ym9DblRMcDdsTnhUR1AxbWJ5cmhCSFJQZlBuMksyNEF4VkxO?=
 =?utf-8?B?QWp3LzhqcHBFWWlOa0ZOVFhMaGg0QXFaL2d5WElDTjlyZHBQODlWUmVmQ3hI?=
 =?utf-8?B?UXZEUWtxMWdaNk84aW9nWFdzUjNaVGJMYllpYktmbnhPSENqMTk2TzJSTFBp?=
 =?utf-8?B?c3J4azZHRENTNGV0VFlFSmhwd2txa1VqVzJ6ZjR3NmI0c2JlRWFSdHZ4cU5j?=
 =?utf-8?B?WVo0MDRzWnF3OGQzSFg5Qlk3ek0zUExxQ1dxWk5hTzhQN05HZSszcUw4YzJD?=
 =?utf-8?B?ei9VcUJBNzdlWUxKSGJPWXV6amFkTUg5dVJESmhMSFo3bW1JNXA4T1gzL0NF?=
 =?utf-8?B?aldWeEFRUXdMaUIwMmpTSmJsQS91ZzVFQkxpN3pGd0JnOG5ZM2V0VVRnUjY1?=
 =?utf-8?B?VUtaRlUzeUd0ckwrNHpmaW5vRmJmK01sYUJOY0pKRjJRRHlKalBZMGcvdGhp?=
 =?utf-8?B?WjU3Zy9sRWVzbmdmUy9DLzA1M3lFQ1RNa213U1Z6Ykt0WGwzMURiVjhWakpi?=
 =?utf-8?B?M0tLa0Uxa0Rub3VzUGdqamd3NTJmaS9QU3d5U3NEWG1sb0Z5cEsxL3dKV3Uy?=
 =?utf-8?B?UkkxNHIraGtEenNhSzRCWHBOS3FnZXAzK0FlT1Iwa3NqbmJXQ2RNY3RqL1Br?=
 =?utf-8?B?OE1MZVplbHhLaTV2YkRiUEhOUFBkc0FvUU53eXc4U3BqeVZpS0xacDJvS2Fh?=
 =?utf-8?B?Y2hwdHprSG5NU3p5RDVSWjVtOGVnRzNDWHVXU2IrTWRSUDAyQVRWYUx4VTl1?=
 =?utf-8?B?T3p6WnNhN0xmQ2dXekc0RjgyZW04TGtoTDlldDBMNmhRZEJHTVlSVVdRTytk?=
 =?utf-8?B?djU2eFZLaS84VHJ1UDFIUjVlLzQvcUdYbkZWRHYxT3ZxTFMvaU8vcm5XYVZB?=
 =?utf-8?B?TFBEcnRCQWNsWjQzZHRQYzl3SFlER0N4L21XUXlHVTZzL2QrTCs2WXJveWdl?=
 =?utf-8?B?MTZqYzJXdkN5aFVHUkFsaDFveUx3Ri9QUnpFVmI2UGlzSHdDR0FFd21xd0Fr?=
 =?utf-8?B?M1NqQUEzcXg5WnlLUVVBbmNlek9FUGNUVVhMSWNHeWtYWVlQa2dOV0RpcnBa?=
 =?utf-8?B?ZXJrU3VVeE9IdFo5d3V1VXlvMk9FQ3Jqb3MvN29VQ09ET2pHRjNDTWxneXRv?=
 =?utf-8?B?cExlUk1tRGpldlRTSFBWaDR0NzdxMjVLYXpydG1yTWhaYTZRM0tYZWV6M2M5?=
 =?utf-8?B?TTR1M0sraUZSQ0xwSmZlVUxNK3lyUEQ4YytDSElzWjZMNUNkWXBmV2JHRitz?=
 =?utf-8?B?L1pxVzV4d2k1Wm9CN0JkNGNQbVkvb2N6NjExV3dONHgxa1ZkekpSenJaRXhE?=
 =?utf-8?B?TE1kajVtZGVMNUZiZkhnTHp5ZG4wS0pxUWR5dEMyV0tqcmZQK2p2dkQ0N2pN?=
 =?utf-8?B?Q2Y1L3dIWTFOMUdCZGdDVTRETmlxSXpSeGdLd2R0YWZTRU5WMkFMRWxndm9r?=
 =?utf-8?B?b2JNY3c3bDIwTGFvdnVFK1p0bU5HeVFvZ1NJT0svRnNpN08yUHY3VU56WmJE?=
 =?utf-8?B?RTdyS09Tdnl3NjdwVEM1UzBDc05MdThKZFZxK0lYdkdvTVpvZUhFS3oyTHF5?=
 =?utf-8?Q?3zbsunYrHFlzrpJKhSjMy1OQCk0JWDbDHDSw2Bd?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-aVHzvgoRXIEmFtazal28"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eb51e165-532e-4f23-c050-08d97f6ac8a9
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 14:51:25.8655
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: G7abEomiAhj9mCdC74yL2sHmcHPBOjb6+f/KpuMIa3OuCKO5sn7XE1aHm3AC8wgpphXX3keAucvPq6zgcy+6zg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB7700

--=-aVHzvgoRXIEmFtazal28
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:

> There is one buggy race record, TRC_RTDS_BUDGET_BURN.=C2=A0 As it must
> remain
> __packed (as cur_budget is misaligned), change bool has_extratime to
> uint32_t
> to compensate.
>=20
Mmm... maybe my understanding of data alignment inside structs is a bit
lacking, but what the actual issue here, and what would we need to do
to fix it (where, by fix, I mean us being able to get rid of the
`__packed`)?

If rearranging fields is not enough, we can think about making
priority_level and has_extratime smaller, or even combining them in
just one field and decode the information in xentrace.

Of course, I can send a patch for that myself, even as a followup of
this series when it's in, as soon as we agree about the best way
forward.

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-aVHzvgoRXIEmFtazal28
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFN5e0ACgkQFkJ4iaW4
c+6q/Q/8DwA4DlewXTHS29SDjjqKfIyqDwSFAvRLJRqbLX4o8MFy4t/j2+rxgs09
1xw3uNBLxWpc6BtD490++TUraYo3YSY9Xw3fVYmStWNh1jRl3cOrl2g6H+neswcM
XwP+rz3sN9J1Yz5RR/GbUTlygDdhxD14k6c/sQwMuzZNqNIRx9MuG+YegOzN/UqL
jGkjhd8NC1Pa5XYv7L4aOo9ETlCmd1cjNYPihJZNjZsBHdcGxjsu12Yu43RogIda
2HCNgIlz0ncc8AU+XJzEY0VmilWTVzVrPkNKeFf+53HBElZ0+87tGP3Epf4/Orik
FzOmlDAvNyzzeblsvrmwOgBjxKckouPd4m+QqPn15ZVgs5fzOoC5Sq5cwr5HBBMO
tLfddq4IuDo0Qe7CaeuQtmjWXhqhyk32erBC0+dpYMDM4tt7ba6acU53unRP5G2a
DCZoeDwUF7z2BCvMAwoKsu/w0C6XeIhMPCYvhXjbo4Dv2qrFD7I/C56T8bdHN2c0
EK7rzavkY9L2lo4XLWOMW4Mdg2JZjmWiRC/7ol+cdyINoZ2B51H//+1a+432IwLu
VSOmtxGFwiOSOcMIcwXRdboXtzfuTHl4h84j4MS+22/ELH4/sc2t+FL+ADbIy0p9
G6WDt9qUazC347p/YTrP/yXLwGNBBWqyFLnHxPY31lJrv28ui9E=
=itj6
-----END PGP SIGNATURE-----

--=-aVHzvgoRXIEmFtazal28--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:52:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:52:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195439.348135 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmZ2-0005Jn-I6; Fri, 24 Sep 2021 14:52:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195439.348135; Fri, 24 Sep 2021 14:52:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmZ2-0005Jg-FD; Fri, 24 Sep 2021 14:52:36 +0000
Received: by outflank-mailman (input) for mailman id 195439;
 Fri, 24 Sep 2021 14:52:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTmZ1-0005Ja-VX
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 14:52:35 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTmYy-0003OG-Aa; Fri, 24 Sep 2021 14:52:32 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTmYx-0008Jt-Vo; Fri, 24 Sep 2021 14:52:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=tw9TyCjQi/+QYd+Zot2/jpZ1pPlgz5c29aIla0z6OrI=; b=yJSTN/PvQhe2oDfDnqbtH/iiSb
	qeNd5UqwVUsvkvN5OayGxBtCJTjKLXWXVbn8s/cqdtVul8+lccIw7yFjicvYx7xY9uJ7jOVdhNlJ+
	RitO0AUiH3zB5NzITU4pfXft6ss2ZVzXH/1IFU6qugj5lLIzsGetFWesltiv7IGWj7dI=;
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in struct
 page_info on Arm
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>,
 xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
 <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
 <YU2PT4rUts8KljKe@MacBook-Air-de-Roger.local>
From: Julien Grall <julien@xen.org>
Message-ID: <04400e18-dde2-4b90-4056-f56c5d7937af@xen.org>
Date: Fri, 24 Sep 2021 19:52:24 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YU2PT4rUts8KljKe@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 24/09/2021 13:41, Roger Pau Monné wrote:
> On Thu, Sep 23, 2021 at 09:59:26PM +0100, Andrew Cooper wrote:
>> On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>> ---
>>> You can find the related discussions at:
>>> https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
>>> https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
>>> https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/
>>>
>>> ! Please note, there is still unresolved locking question here for which
>>> I failed to find a suitable solution. So, it is still an RFC !
>>
>> Just FYI, I thought I'd share some of the plans for ABI v2.  Obviously
>> these plans are future work and don't solve the current problem.
>>
>> Guests mapping Xen pages is backwards.  There are reasons why it was
>> used for x86 PV guests, but the entire interface should have been design
>> differently for x86 HVM.
>>
>> In particular, Xen should be mapping guest RAM, rather than the guest
>> manipulating the 2nd stage tables to map Xen RAM.  Amongst other things,
>> its far far lower overhead.
>>
>>
>> A much better design is one where the grant table looks like an MMIO
>> device.  The domain builder decides the ABI (v1 vs v2 - none of this
>> dynamic switch at runtime nonsense), and picks a block of guest physical
>> addresses, which are registered with Xen.  This forms the grant table,
>> status table (v2 only), and holes to map into.
> 
> I think this could be problematic for identity mapped Arm dom0, as
> IIRC in that case grants are mapped so that gfn == mfn in order to
> account for the lack of an IOMMU. You could use a bounce buffer, but
> that would introduce a big performance penalty.

Or you could find a hole that is outside of the RAM regions. This is not 
trivial but not impossible (see [1]).

> 
> Other question is whether we want/need to keep such mode going
> forward.

I am assunming by "such mode" you mean "identity mapped". If so, then I 
am afraid this is not going to disappear on Arm at least. There are 
still out there many platforms without IOMMUs or devices which are not 
protected (the GPU is a common one).

Furthermore, Arm just sent a series to introduce identity mapping for 
domUs as well (see [2]).

[1] <1631034578-12598-1-git-send-email-olekstysh@gmail.com>
[2] <20210923031115.1429719-1-penny.zheng@arm.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 14:55:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 14:55:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195446.348150 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmcH-00061F-61; Fri, 24 Sep 2021 14:55:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195446.348150; Fri, 24 Sep 2021 14:55:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmcH-000618-1Q; Fri, 24 Sep 2021 14:55:57 +0000
Received: by outflank-mailman (input) for mailman id 195446;
 Fri, 24 Sep 2021 14:55:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTmcF-00060s-Rf; Fri, 24 Sep 2021 14:55:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTmcF-0003Sh-Mt; Fri, 24 Sep 2021 14:55:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTmcF-0006iH-8H; Fri, 24 Sep 2021 14:55:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTmcF-0000lr-7n; Fri, 24 Sep 2021 14:55:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=IxZj+TWPN0h/sYBQjnIifhXJal0bQDOW15bh56yaujc=; b=eE3oX6dHzSQcXAODqc2G5JUy4P
	cjHnDXoJn6yPXnxqn8NVfyXF4XlFzuZMSw6nCkqzA+Gx/nbSeKKqIqcbgxRy49EWGVM2FianHxJCP
	zxPDeo7YIEu4xyhCJpaGyciNniBNshdT+CSeRghCuDMYzOsNvc5ubDMlDzMi9xUjaX0A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165174-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165174: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-i386-prev:xen-build:fail:regression
    xen-unstable:build-amd64-prev:xen-build:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-i386-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=604be1b333b1b66052ab9b0133f156890549a4f0
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 14:55:55 +0000

flight 165174 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165174/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-prev               6 xen-build                fail REGR. vs. 164945
 build-amd64-prev              6 xen-build                fail REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-migrupgrade  1 build-check(1)               blocked  n/a
 test-amd64-i386-migrupgrade   1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  604be1b333b1b66052ab9b0133f156890549a4f0
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   13 days
Failing since        164951  2021-09-12 00:14:36 Z   12 days   25 attempts
Testing same since   165161  2021-09-23 03:54:29 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             fail    
 build-i386-prev                                              fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 blocked 
 test-amd64-i386-migrupgrade                                  blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 899 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:04:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:04:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195456.348163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkY-0007so-2v; Fri, 24 Sep 2021 15:04:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195456.348163; Fri, 24 Sep 2021 15:04:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkX-0007sh-WE; Fri, 24 Sep 2021 15:04:30 +0000
Received: by outflank-mailman (input) for mailman id 195456;
 Fri, 24 Sep 2021 15:04:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VJv2=OO=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTmkW-0007sV-6u
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:04:28 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5e813c8-1d48-11ec-bae9-12813bfff9fa;
 Fri, 24 Sep 2021 15:04:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 891232243C;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 62B2513C0A;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 25YzFvroTWHvWwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 24 Sep 2021 15:04:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5e813c8-1d48-11ec-bae9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632495866; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=HQBiBcdcCvlRqPJN3Rt7DD3yXpK+oxbfPfUN/MgrAX4=;
	b=kHf9WiHLhYIbQZXBGb5fIfXQ80vYkOCHNwpoD3yXIsVKPAZ5t5Qj++D5huwubmewKKqrEK
	3BJ7GxsRjRiRJePOmXVyLTpwmHAnCNJfvjUVsDseVfrQ7sco2BgHXnKnirVfdjC57QdK7p
	RpJJj+5gjjGlT/RYmdxbOQOCue1j8No=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH 0/2] include/public: update usbif.h
Date: Fri, 24 Sep 2021 17:04:22 +0200
Message-Id: <20210924150424.10047-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add some missing defines and documentation to the pvUSB header file.

Juergen Gross (2):
  include/public: add possible status values to usbif.h
  include/public: add better interface description to usbif.h

 xen/include/public/io/usbif.h | 171 ++++++++++++++++++++++++++++++++++
 1 file changed, 171 insertions(+)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:04:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:04:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195457.348167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkY-0007wH-BW; Fri, 24 Sep 2021 15:04:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195457.348167; Fri, 24 Sep 2021 15:04:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkY-0007uw-7J; Fri, 24 Sep 2021 15:04:30 +0000
Received: by outflank-mailman (input) for mailman id 195457;
 Fri, 24 Sep 2021 15:04:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VJv2=OO=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTmkW-0007sb-TJ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:04:28 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4c592ba0-ab74-4fa4-a5ce-f1c8b8b2f7bd;
 Fri, 24 Sep 2021 15:04:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C76A11FDF0;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AD9A513C0A;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id aCIzKfroTWHvWwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 24 Sep 2021 15:04:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c592ba0-ab74-4fa4-a5ce-f1c8b8b2f7bd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632495866; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=W/uBl5hwPg6EF4w9vueW8esS9XeIP9DMgCbyFhO8PuM=;
	b=URdFAOWDv5ckSLDLrO9skMzBIBJMioRnDM3ijBK+eChFiEhwWphtVJIyvOw/uMeVwOFIdE
	9LjTesF+mcWF5GoEijY/ghzHaiV9MfDHypwraJqBBbHsaEeSAu4YGey/pI6ZrRm9d6WTHK
	N72wbDX9nmSQFAqVEzSjovvnu0yadrA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH 2/2] include/public: add better interface description to usbif.h
Date: Fri, 24 Sep 2021 17:04:24 +0200
Message-Id: <20210924150424.10047-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210924150424.10047-1-jgross@suse.com>
References: <20210924150424.10047-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The PV USB protocol is poorly described. Add a more detailed
description to the usbif.h header file.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/include/public/io/usbif.h | 164 ++++++++++++++++++++++++++++++++++
 1 file changed, 164 insertions(+)

diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
index fbd6f953f8..10ff2ded58 100644
--- a/xen/include/public/io/usbif.h
+++ b/xen/include/public/io/usbif.h
@@ -32,6 +32,34 @@
 #include "../grant_table.h"
 
 /*
+ * Detailed Interface Description
+ * ==============================
+ * The pvUSB interface is using a split driver design: a frontend driver in
+ * the guest and a backend driver in a driver domain (normally dom0) having
+ * access to the physical USB device(s) being passed to the guest.
+ *
+ * The frontend and backend drivers use XenStore to initiate the connection
+ * between them, the I/O activity is handled via two shared ring pages and an
+ * event channel. As the interface between frontend and backend is at the USB
+ * host connector level, multiple (up to 31) physical USB devices can be
+ * handled by a single connection.
+ *
+ * The Xen pvUSB device name is "qusb", so the frontend's XenStore entries are
+ * to be found under "device/qusb", while the backend's XenStore entries are
+ * under "backend/<guest-dom-id>/qusb".
+ *
+ * When a new pvUSB connection is established, the frontend needs to setup the
+ * two shared ring pages for communication and the event channel. The ring
+ * pages need to be made available to the backend via the grant table
+ * interface.
+ *
+ * One of the shared ring pages is used by the backend to inform the frontend
+ * about USB device plug events (device to be added or removed). This is the
+ * "conn-ring".
+ *
+ * The other ring page is used for USB I/O communication (requests and
+ * responses). This is the "urb-ring".
+ *
  * Feature and Parameter Negotiation
  * =================================
  * The two halves of a Xen pvUSB driver utilize nodes within the XenStore to
@@ -99,6 +127,142 @@
  *      The machine ABI rules governing the format of all ring request and
  *      response structures.
  *
+ * Protocol Description
+ * ====================
+ *
+ *-------------------------- USB device plug events --------------------------
+ *
+ * USB device plug events are send via the "conn-ring" shared page. As only
+ * events are being sent, the respective requests from the frontend to the
+ * backend are just dummy ones.
+ * The events sent to the frontend have the following layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |    portnum     |     speed      | 4
+ * +----------------+----------------+----------------+----------------+
+ *   id - uint16_t, event id (taken from the actual frontend dummy request)
+ *   portnum - uint8_t, port number (1 ... 31)
+ *   speed - uint8_t, device USBIF_SPEED_*, USBIF_SPEED_NONE == unplug
+ *
+ * The dummy request:
+ *         0                1        octet
+ * +----------------+----------------+
+ * |               id                | 2
+ * +----------------+----------------+
+ *   id - uint16_t, guest supplied value (no need for being unique)
+ *
+ *-------------------------- USB I/O request ---------------------------------
+ *
+ * A single USB I/O request on the "urb-ring" has the following layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |         nr_buffer_segs          | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                               pipe                                | 8
+ * +----------------+----------------+----------------+----------------+
+ * |         transfer_flags          |          buffer_length          | 12
+ * +----------------+----------------+----------------+----------------+
+ * |                       request type specific                       | 16
+ * |                               data                                | 20
+ * +----------------+----------------+----------------+----------------+
+ * |                              seg[0]                               | 24
+ * |                               data                                | 28
+ * +----------------+----------------+----------------+----------------+
+ * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
+ * +----------------+----------------+----------------+----------------+
+ * |             seg[USBIF_MAX_SEGMENTS_PER_REQUEST - 1]               | 144
+ * |                               data                                | 148
+ * +----------------+----------------+----------------+----------------+
+ * Bit field bit number 0 is always least significant bit, undefined bits must
+ * be zero.
+ *   id - uint16_t, guest supplied value
+ *   nr_buffer_segs - uint16_t, number of segment entries in seg[] array
+ *   pipe - uint32_t, bit field with multiple information:
+ *     bits 0-4: port request to send to
+ *     bit 5: unlink request with specified id (cancel I/O) if set (see below)
+ *     bit 7: direction (1 = read from device)
+ *     bits 8-14: device number on port
+ *     bits 15-18: endpoint of device
+ *     bits 30-31: request type: 00 = isochronous, 01 = interrupt,
+ *                               10 = control, 11 = bulk
+ *   transfer_flags - uint16_t, bit field with processing flags:
+ *     bit 0: less data than specified allowed
+ *   buffer_length - uint16_t, total length of data
+ *   request type specific data - 8 bytes, see below
+ *   seg[] - array with 8 byte elements, see below
+ *
+ * Request type specific data for isochronous request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |            interval             |           start_frame           | 4
+ * +----------------+----------------+----------------+----------------+
+ * |       number_of_packets         |       nr_frame_desc_segs        | 8
+ * +----------------+----------------+----------------+----------------+
+ *   interval - uint16_t, time interval in msecs between frames
+ *   start_frame - uint16_t, start frame number
+ *   number_of_packets - uint16_t, number of packets to transfer
+ *   nr_frame_desc_segs - uint16_t number of seg[] frame descriptors elements
+ *
+ * Request type specific data for interrupt request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |            interval             |                0                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *   interval - uint16_t, time in msecs until interruption
+ *
+ * Request type specific data for control request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                      data of setup packet                         | 4
+ * |                                                                   | 8
+ * +----------------+----------------+----------------+----------------+
+ *
+ * Request type specific data for bulk request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 4
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *
+ * Request type specific data for unlink request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |           unlink_id             |                0                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *   unlink_id - uint16_t, request id of request to terminate
+ *
+ * seg[] array element layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                               gref                                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |             offset              |             length              | 8
+ * +----------------+----------------+----------------+----------------+
+ *   gref - uint32_t, grant reference of buffer page
+ *   offset - uint16_t, offset of buffer start in page
+ *   length - uint16_t, length of buffer in page
+ *
+ *-------------------------- USB I/O response --------------------------------
+ *
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |          start_frame            | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                              status                               | 8
+ * +----------------+----------------+----------------+----------------+
+ * |                          actual_length                            | 12
+ * +----------------+----------------+----------------+----------------+
+ * |                           error_count                             | 16
+ * +----------------+----------------+----------------+----------------+
+ *   id - uint16_t, id of the request this response belongs to
+ *   start_frame - uint16_t, start_frame this response (iso requests only)
+ *   status - int32_t, USBIF_STATUS_* (non-iso requests)
+ *   actual_length - uint32_t, actual size of data transferred
+ *   error_count - uint32_t, number of errors (iso requests)
  */
 
 enum usb_spec_version {
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:04:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:04:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195458.348186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkc-0008RR-Ka; Fri, 24 Sep 2021 15:04:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195458.348186; Fri, 24 Sep 2021 15:04:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmkc-0008RI-HP; Fri, 24 Sep 2021 15:04:34 +0000
Received: by outflank-mailman (input) for mailman id 195458;
 Fri, 24 Sep 2021 15:04:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VJv2=OO=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mTmkb-0007sV-5K
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:04:33 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b5f7e3e8-1d48-11ec-bae9-12813bfff9fa;
 Fri, 24 Sep 2021 15:04:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A634D22443;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8DA5E13EF2;
 Fri, 24 Sep 2021 15:04:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id uApJIfroTWHvWwAAMHmgww
 (envelope-from <jgross@suse.com>); Fri, 24 Sep 2021 15:04:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5f7e3e8-1d48-11ec-bae9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632495866; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=tphERlob2bA1MgckZcUA+BdGjUEA0arBUyhlGjAd/Jk=;
	b=EGRoMJqGtU71p4HR0OUXjRB4s/I7Ei7S1JupQsmisywq22LeNLFz+qSPtk0OwuN/I7i7kG
	6rchKZIharlauG9ztdDEWUyY9IiWXo9jnOKZaECL6oKXFjFS9C+ioEoANBnhnzJiDt4h4r
	Ail97ujF6VXLYHCJU5qEvONddANrsQA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH 1/2] include/public: add possible status values to usbif.h
Date: Fri, 24 Sep 2021 17:04:23 +0200
Message-Id: <20210924150424.10047-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210924150424.10047-1-jgross@suse.com>
References: <20210924150424.10047-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The interface definition of PV USB devices is lacking the specification
of possible values of the status filed in a response. Those are
negative errno values as used in Linux, so they might differ in other
OS's. Specify them via appropriate defines.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/include/public/io/usbif.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
index c6a58639d6..fbd6f953f8 100644
--- a/xen/include/public/io/usbif.h
+++ b/xen/include/public/io/usbif.h
@@ -221,6 +221,13 @@ struct usbif_urb_response {
 	uint16_t id; /* request id */
 	uint16_t start_frame;  /* start frame (ISO) */
 	int32_t status; /* status (non-ISO) */
+#define USBIF_STATUS_OK		0
+#define USBIF_STATUS_NODEV	-19
+#define USBIF_STATUS_INVAL	-22
+#define USBIF_STATUS_STALL	-32
+#define USBIF_STATUS_IOERROR	-71
+#define USBIF_STATUS_BABBLE	-75
+#define USBIF_STATUS_SHUTDOWN	-108
 	int32_t actual_length; /* actual transfer length */
 	int32_t error_count; /* number of ISO errors */
 };
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:16:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:16:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195476.348200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmvv-0002Sj-OZ; Fri, 24 Sep 2021 15:16:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195476.348200; Fri, 24 Sep 2021 15:16:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTmvv-0002Sc-LY; Fri, 24 Sep 2021 15:16:15 +0000
Received: by outflank-mailman (input) for mailman id 195476;
 Fri, 24 Sep 2021 15:16:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mTmvu-0002SS-H0
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:16:14 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id af7ea26f-acd1-4698-92f0-3e0ddb40643c;
 Fri, 24 Sep 2021 15:16:13 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-llJG86DGPoW3Jd4SZsxV4g-1; Fri, 24 Sep 2021 17:16:10 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM8PR04MB7474.eurprd04.prod.outlook.com (2603:10a6:20b:1d9::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Fri, 24 Sep
 2021 15:16:09 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 15:16:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af7ea26f-acd1-4698-92f0-3e0ddb40643c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632496572;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=RWqq9mAcIYwMkvDTrO88ltRzUibHN1PcXj89F8puzRM=;
	b=g4tBmr/6RzQdA1T/nAGPfr34UReyVwTvBzFwnhbREwFRXsjR9cbHywCtBoU4A52X6KIwho
	krIz2M5s70kKKw580xFCXlCuVWm/gPRUdgCv41KQecSp+o2ZFYPU54oJbRoLxQvWbn/EoC
	5aakb0ri8jdRp/hs/y1I+h36zfrjYH0=
X-MC-Unique: llJG86DGPoW3Jd4SZsxV4g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TNSZskXnU5WZu7dUDps60oM1rpbeo0ubPPkqu3+D7NMYQYAekP9qZW3xSNvAUl1gswXKGjYlnM3SyfzJLe9cKlw2E2qhfZ06H5uORXVqFFvpaQPJ9Ke2SeV0Hrvu68ks9YK3kqd1fdT84HpUjRQY1zbfHz09m1AgvNc1i7xNukWWryT6EhQhGURLCjl0NTNJQhzm4nBrMCGUANm8zNSb0QY/bihtRhGukbElIVup7wAlgJbUwkZhF3PQpY9QYOms/EAeqpFunEoTAHaEKE0MEEDIduqwV6vqhzbvz0ziUcNWWrII0hSOQVexIPPZ4u8fl73zq4AoSj0W+2JRthpfQA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=RWqq9mAcIYwMkvDTrO88ltRzUibHN1PcXj89F8puzRM=;
 b=mrSRr78jtI/LLWJQhYqRJP/3+xDiteCsvOb5K51qg2eLmmNyNbZSB0zgzp8jSBUQ0CXB/Nmk/FrSCjibIYV9oaviYc+qNCoyhOi7In3LjnEKxwLXpsqlLVeGG2Vg9TixjKQghMBUvfS54tQoQBEeIXy8wyMgcO9CCRTZmJsG3YdI4ve7LnpilAznSVEVuAkmPfXFTgjMVybNwegSTqdKeSA9sBT+85XbTOm9xFWRTXnR+k6xCNlIId6dkZ7yBJpfwctqFc0j0IqVvhu4CqEp5ehB05VOKmT3LbFZk4gC5BjaZRJRIvgpDfgzScQ7WI8VLkdqjU04nTp/kvlBYvjAiw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: Jan Beulich <JBeulich@suse.com>, "andrew.cooper3@citrix.com"
	<andrew.cooper3@citrix.com>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, "iwj@xenproject.org" <iwj@xenproject.org>,
	"wl@xen.org" <wl@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 09/12] xen/trace: Minor code cleanup
Thread-Topic: [PATCH v2 09/12] xen/trace: Minor code cleanup
Thread-Index: AQHXrkSNUXB03Bf2kEahk+mMoXgWnquuVDIAgAT9eAA=
Date: Fri, 24 Sep 2021 15:16:08 +0000
Message-ID: <8caf579ed621fa477038f7a7b7a5ce43c8b75abd.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-10-andrew.cooper3@citrix.com>
	 <c34d236d-d749-5d33-207a-94fef482577f@suse.com>
In-Reply-To: <c34d236d-d749-5d33-207a-94fef482577f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c4e33af4-6d92-49dd-10b5-08d97f6e3cb0
x-ms-traffictypediagnostic: AM8PR04MB7474:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM8PR04MB74744498FD52629C169F180FC5A49@AM8PR04MB7474.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 pQatuKB8uutMIt+Egw2Kn7sNCjJODpmIBxIubEYyM6YR+WgVEsP7nKmI35qH+XItS1bjFUb0DJB7s8cd8WKCYtsOYEiUqWvzIjYsHnLuQjSJtUzjvZGBh6rufsMlk0bremM5RcppCCcyyhKCdDLikijRk3DPLVF01unaSVEhQTCYsmHChqtK6BxzdlL9TTMsMbOB1s5ZpnTm0GDK77Y5qU48N9EVuvMpKaJO4JGkikPxDiU7ClrgmaVXrPvFTiYNyh6RCFqIGZphywfIi/15t/7V1IUYJ1VfZl5Kox7yEXt9nKP9MQUw05xSxfNuIjsSeLxTqX3GxHvrmkIqDJOYo5M0wC1yDNQPhjYdtItdCa7w92q63m7ttYgAczGqosd024u7zW2mi2kM/nUizaOk0mRb0pdG+EAxztW9ERc/Gyf+WTiOyK/DjNL278J3ysjv0mVJpqFFkQHm6qtZbH7OGfUU9d+wTB8MbriJVOOVpC5qULxTxe6+goYWb4Z89g4hJatx+ABmCpTDEYD6vIeLcGRSWdfeaFDyOWP22B4epyi6xC0O0lH40/u8TgLhVf/XfWRafvU8UKPckc6BH++KrrHgIOD8u9wkus3sWRCt0n2b+uTQgbkHqBDpv0u93IkjyDUtZyDtStoqvJ7asquOcStunqUAa7aKnN3AnQNcRjbLW/zyfq0v7U4yo+5nvuheUM0pGuP94mpNWNtGrxY4phLQHBgnznkI3cIqNCNz4CB2sJJKkpP+0JUAAWyXJ2DOjd1Mmyp3dZMkEbFK2lx2bJhZQv718NkRk5JIybYww6uGuE4c8fwpqxOit3tVPCUPDnSwAVYBau8/hiav8sdRMA==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(5660300002)(122000001)(38100700002)(66556008)(66616009)(76116006)(66446008)(2906002)(99936003)(36756003)(316002)(91956017)(64756008)(66476007)(66946007)(26005)(8936002)(54906003)(8676002)(86362001)(4326008)(110136005)(966005)(6512007)(186003)(53546011)(38070700005)(508600001)(71200400001)(83380400001)(2616005)(6506007);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?ZlhTdkYwYnFzNTI1U0FlYlViTDFQTW4wclpJaUgzd2NXeTVRTHZNT2JvYm9u?=
 =?utf-8?B?S002T200U0QxTHNPL0w2V0pSM3FpRmpzeWJ0WVRGWDB5M2hXcVhMd1M3MG9B?=
 =?utf-8?B?aFp2N25iU25OcVN2cGNSWlhsYUcxRWtJQ09JWmxtNUd6VW40VC8wRENqL0lB?=
 =?utf-8?B?eks2aC84dEdJYit0ZVM1VmxZbTVGZEdoeStTUUk5QjdURXN1aFV5THRaazVB?=
 =?utf-8?B?NkdSS0FGZ2haaVBRUUhWR1AwSGZaZ0FVZ1B3R25aQ3FZRmIzTGVmM2plQ1Nr?=
 =?utf-8?B?ak1iTDBGUnZGb3FSc3VxeVVIeEVBditUYTZLWkhrL2Q3SnZVZEN2WlRJS2JT?=
 =?utf-8?B?ZkZQcnk1bVNUaW52ZjJNRjg0QjRUeDB3S3RhZlRKMWRuZUNlZURleUVYTW00?=
 =?utf-8?B?VUgyWXJqQ2VHK2hVV0M0YzVaWWdoVW05TWNtR0IvWFpHdW9oUlhvTHJqeVNW?=
 =?utf-8?B?UnZxNnhKZE1mUWJ1Y0w1bEFiaytXb3JXQzhpWTAvNHJ4WHQ3ellLa0dKbCtK?=
 =?utf-8?B?cmM2T09zWGRBTlRuemszS0YwQU5TWWNsZ011L20ycmVWV3dZcXZGSUtEeDFK?=
 =?utf-8?B?ckxrZ2lxSTdHZXRLTXNuSHVxemZhQWpPOTJodzJnWXdaeXVFRWx5VkR6UE5o?=
 =?utf-8?B?UThRWE1uQTltaUZ1eUtxZG43cVVQYWN0K0VsQnFJR3VoTnQyV1NqNG1aMDlk?=
 =?utf-8?B?U1ZHZnFCSjBxVEx6MTh1RTdCZWZtRkx0NWhuYjk1Vk9NaEt5NmtCb1lkTE5N?=
 =?utf-8?B?a0UvNytTTU5PdHZZWjhwZGNZbUJYNmQvdWJVL3pnV1lEVS9xUUx6NFRSTzZm?=
 =?utf-8?B?WUNOV3UxUUlDcHNhY1g4dlRoYXJXdHBHQ2UvVXpYRDFKbXlNSlNhQmpMbWFK?=
 =?utf-8?B?ZFBYUFJ0Z0dTcnk2ZzJLZlErenh2RTFhT3lHeWdnTHpPeFNjNUVaTTY3Y2JJ?=
 =?utf-8?B?aUpiNmovMmJQMk9oR05tNDIwWkxuRUh2RXhKMDVLZGFjTS9LSlkwUzVQRU5N?=
 =?utf-8?B?VlQ5SjVxd1BYQTZMRUVBczI3S1hxY09ucEo3Q2dCQlpHWjYwdU9JQjJNQlhv?=
 =?utf-8?B?NGRhZE5mQUE5a0ZJVmx1UEZvTGJjbmttdHRPVDVUVFZvcWpRTlVRS1lvL2VH?=
 =?utf-8?B?ZjBvSGMrZEJuUHRzbVVFMnc2OEFqeXFLbU51MTdUM3FleS8vWXUrTWpWc254?=
 =?utf-8?B?czR1ZitDYjd5Q0xKdytpZDlxTVZzNkZRL1c0aElnZHBWSUMrYm5KamN5czVZ?=
 =?utf-8?B?ZThjM3FwTjZiakdCam8xTUE3Sk1lbGFVTWphV2k5K09FakYzaG4zRlIzZ2Jv?=
 =?utf-8?B?aWVBalg1amJUajhMamphaUM0Y3Jxc0pHRU84Rys5eC9qZzNBNEJ6UkkxNU9p?=
 =?utf-8?B?SXF2c3d6dTBZZnZNZDJhQStaSVFTbmN6K20vV3UrcGJiaGRtQnJZNk5jdS90?=
 =?utf-8?B?Qnoxd1NhcWJqTEdqSk1SRlgyRTdHeTJ2OUlJeERsRko5MGIzVkhRYjFyTzU3?=
 =?utf-8?B?ZXBDejU1dWtYMlFFM0FOUnUvVkE0ZVBuK21ZUFdpUHdhUzB1U052blZSdmdv?=
 =?utf-8?B?cDZCRER0NGVqWVM3UE03Z2JQbzJ1NGU4QUI3Ky95Y2FTUVNkVC9vY0h6aHVs?=
 =?utf-8?B?SFkrczkvaHNzVC9tWFJubElWMjE3RVNBY2JxT2p5YUpNeFJ4MXlwMTV0b2w0?=
 =?utf-8?B?dXJObTBJRmtjbzh2bS80RGFlOE9yTktHWHJvNHQ2M0lodVQ5VkUweEYrYjZw?=
 =?utf-8?Q?s9JHLDOeXVbDJa3dm7x9Hvp89n2lUPtmxKHEPf0?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-zCOrHJAO1yPZS0GupZLf"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c4e33af4-6d92-49dd-10b5-08d97f6e3cb0
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 15:16:08.9468
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 44ip2EWx8OfJpjjrdNo1yZ3N/HKvgvepNzZrSPrlLnvbi7HA87fwYID6r2i8pDIcceYFtQ3z4CsNvGDvIDB7nA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR04MB7474

--=-zCOrHJAO1yPZS0GupZLf
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2021-09-21 at 13:03 +0200, Jan Beulich wrote:
> On 20.09.2021 19:25, Andrew Cooper wrote:
> >=20
> > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>=20
> Like for v1: Largely
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

> One remark:
>=20
> > @@ -717,9 +713,6 @@ void __trace_var(u32 event, bool_t cycles,
> > unsigned int extra,
> > =C2=A0=C2=A0=C2=A0=C2=A0 if ( !cpumask_test_cpu(smp_processor_id(), &tb=
_cpu_mask) )
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return;
> > =C2=A0
> > -=C2=A0=C2=A0=C2=A0 /* Read tb_init_done /before/ t_bufs. */
> > -=C2=A0=C2=A0=C2=A0 smp_rmb();
> > -
> > =C2=A0=C2=A0=C2=A0=C2=A0 spin_lock_irqsave(&this_cpu(t_lock), flags);
> > =C2=A0
> > =C2=A0=C2=A0=C2=A0=C2=A0 buf =3D this_cpu(t_bufs);
>=20
> I wonder whether the comment wouldn't be helpful to move down here,
> in of course a slightly edited form (going from /before/ to /after/).
>=20
FWIW, I agree with this (but the R-o-b: stands no matter whether it's
done or not).

Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-zCOrHJAO1yPZS0GupZLf
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIyBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFN67gACgkQFkJ4iaW4
c+7o9Q/3czW58sr7O1UMSQybVu2HK4m1H6Zd339rTRaEhS9+zSnTfr7NRoyFF1Wn
C9aFL1fFWeO7NohumUiqjOsTrWiIDQdQ4fl16jl+KzRmuz8dqS73IGSQL9MtXK1G
C9I0XiOyI2LIy2ripC9bnjWbZRi3kr8aZCEqV8vo+0gApqwR42X57b0/+pwqt0qf
yuny/uMpBHfsOK1mlfBZMPbuUTu+N1ijCUNjR0HADOm2u+zqCtt11zeYOVIntZeP
unwAWHepnJDfUehhXG0EDXfq9zHQ8LhfSm1cDzr021SyX+R4W1pAEhn4kXhLYsPj
pbDyQZbJN9ykfENacieNPOKfi383ALoJDVcBp2re42ZqqlVNj5nbWS0vR9CI0Kdo
CZx2CMZQ5uAL/lToISNDpsnqieahkErs1y60qh6FTz0ZT7IYSq8RGX8Cbt5U16ZB
1ajRPUhDZBYnov+wjDwcJYGCdAZyA28DOuQm7jhtOysvwUiGk3CODWMrtR5zorLx
TUJm3YPg7haGyff2TakYNtVJqY4Q1TamRbpdXQzu+fmd0q/LO/bPjSAcun+Df51Z
Y24b4oifc50Upnn01THPSI3BapHtigNCKPOBYBxRtxa++X89vITnOJvInQBYUO2u
5d9WdxMzMJcAu7ZsKID2dxR4ojmaerN4qtSwcMgmAjk1zUCJzg==
=R5xu
-----END PGP SIGNATURE-----

--=-zCOrHJAO1yPZS0GupZLf--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:29:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:29:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195488.348211 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTn8X-0004h8-32; Fri, 24 Sep 2021 15:29:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195488.348211; Fri, 24 Sep 2021 15:29:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTn8W-0004h1-VL; Fri, 24 Sep 2021 15:29:16 +0000
Received: by outflank-mailman (input) for mailman id 195488;
 Fri, 24 Sep 2021 15:29:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Mfdr=OO=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mTn8V-0004gv-NZ
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:29:15 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.46]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2be79636-1d4c-11ec-baec-12813bfff9fa;
 Fri, 24 Sep 2021 15:29:13 +0000 (UTC)
Received: from AM5PR0402CA0006.eurprd04.prod.outlook.com
 (2603:10a6:203:90::16) by VE1PR08MB4957.eurprd08.prod.outlook.com
 (2603:10a6:803:109::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 15:29:10 +0000
Received: from VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:90:cafe::32) by AM5PR0402CA0006.outlook.office365.com
 (2603:10a6:203:90::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Fri, 24 Sep 2021 15:29:10 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT013.mail.protection.outlook.com (10.152.19.37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Fri, 24 Sep 2021 15:29:10 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Fri, 24 Sep 2021 15:29:08 +0000
Received: from e8eec4cfd867.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E5865BF7-0584-450A-9A88-EF04751FDBE5.1; 
 Fri, 24 Sep 2021 15:28:59 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e8eec4cfd867.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Fri, 24 Sep 2021 15:28:59 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6256.eurprd08.prod.outlook.com (2603:10a6:102:e6::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 15:28:54 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 15:28:54 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO3P123CA0001.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:ba::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Fri, 24 Sep 2021 15:28:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2be79636-1d4c-11ec-baec-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p4xggSuehu3Z4kQU35gQC/BEPRVfobQILqP/luDi4qg=;
 b=BdSVqwHEl/a8aStwG6wZdAx9k7gNARKatjfyrFrSx8HR5jTvmbf7jSWNgUGfqfRFU3z6TWTLdFnSMpRvOlF5Y+nvOr3F9b9tyZTVDBMlC8iqoOgPMR9GUodyy7gmc6Mc1UowchP1UFjYM2rPj7E5nhfKQaxVXabk8pKWDLxEUW8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: b399fbb56b578120
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lKmjoxHnhfJyt6UwfWRF1hTPHmEgGJLQ8mFuFJtGa95lV1aY3+YpbeMsTK6LfrpruRjKF7kKw43czoBSmt537rZjqGv6CEx0w3tdVjk1QL10oFDau7tFOFXFf/sCLZePcP+TNzDstcy28hxbCV19xCO/DKGiT+3fFDJJlbbazlWBnTouC4INOjM3kClXHtlAjIfMj/2JUzKx6gOsitO3YYkSOAQcqJkLQbPoROJE0Zn9x9m+ISBTgkB/t+rhWcgpY372bLH9V+rTViCGDoBY8WjYZZALDArMvnPhw1tXEUf05+0+PRM6p01l/Sel/3YeQIl3YwMLRDGiBhYzyUnF7g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=p4xggSuehu3Z4kQU35gQC/BEPRVfobQILqP/luDi4qg=;
 b=dxKk/RaP7r/jkgvzTFRoZD+wCbxCWxxloXv2CdkjBVgXw2n8PmVHoUVJ6OQ4w1q9BO8FVTdDiCKJQxPC+5Opw9gBpX3NYS1BgTH4em1Dgb2hUJSLj0KR66MYqkr3YVKep5c3j5foH+GS1O746IDdnBG+CNHNfaxGozczlQkiJ4tJRgPVKIjf0sDkMVzruesQFHkpmxeiEZ7mdeA+WlB44UtySE9AzLtQQnIPIgqpg0/gBpd2O+eocvXEanlDHpeVL46qCOmiNVMMVhJXO2HkMcGQoCwQGwKg1BZgAFgAiS2kKzG5r+jgjMT03TLT+6EJ4VmaupyMD+xOLoiCNun4pA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p4xggSuehu3Z4kQU35gQC/BEPRVfobQILqP/luDi4qg=;
 b=BdSVqwHEl/a8aStwG6wZdAx9k7gNARKatjfyrFrSx8HR5jTvmbf7jSWNgUGfqfRFU3z6TWTLdFnSMpRvOlF5Y+nvOr3F9b9tyZTVDBMlC8iqoOgPMR9GUodyy7gmc6Mc1UowchP1UFjYM2rPj7E5nhfKQaxVXabk8pKWDLxEUW8=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1318fbb2-f7ae-e7e8-0e3a-9dcef69fb430@suse.com>
Date: Fri, 24 Sep 2021 16:28:47 +0100
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>,
 =?utf-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <A2BF4A2F-03A6-439E-AE69-636E8E4F405F@arm.com>
References: <20210922141341.42288-1-luca.fancellu@arm.com>
 <20210922141341.42288-3-luca.fancellu@arm.com>
 <1318fbb2-f7ae-e7e8-0e3a-9dcef69fb430@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO3P123CA0001.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:ba::6) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 020aded2-5708-4877-8717-08d97f700e3b
X-MS-TrafficTypeDiagnostic: PA4PR08MB6256:|VE1PR08MB4957:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR08MB49574E7D6375C43A5881C20AE4A49@VE1PR08MB4957.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2733;OLM:2733;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HeIZ2yOhdDZc822Mwcmep+TeK12ZZb88q5Yycn9byLeo52FKsBbb2Lo+2XW/pvAhODjcky12UYjD+ZrisOEc6PS0riwP9j+BwIqt4vk62LDJqmRFvV9DImm7mTjbTYI+q/DIMWTOgbcUH2OVXOLAydN4tl74xW1T88AUVhweywdYVcWOHjgP347Hv6RRTXi/AZ+NXO7cCKTm3yC5Z7D5ihavAYeMag8F7oOWqM9nak8vuYxin858lZYfXzTg5wxJJ+ssGu/90mOs2kqr0xBCfA1zhiIngjKu9UveFSxeiczBJLUXlMhNud8MW/imSrEhTb/KZ1snRpV+9hEBGuezI5B6b2hnBA110/NmY/HVR5bzIk74Tv8DEHuwr+y2YedrI1jFYSk8xako2OoE8usKKBNiElY2oY4bdXM4s3DIxK6Uo8XmXX35BZwhAAIn3D02x1Bbg4IObHcT5lgRnBd4Axr8op2NqZ2Uf8qxFoNboZM1y3Q1jNOrnj5Y06b1MmPkH5rCJQQwebE+nIfnG8da0E2cI6Z5m/CIJkcpHzEJjy0AtMzlxk11xtY3JBlWyd81/fR00/hqSzvALI8hyhP7Gs8+U2PE9O2JjjwlSGap91vhI6YfUdZFyvzQ43yjMPerKdOUTnQiDX1pIwPJnMw+FX6uVnxh0+uSmvReRyKtDB0qpYsdA8flkWsVgRTzDBO2BFnQ9xFk6v/9tEb1ZMWKb8fDgAT3W/ftQrBXP++B/XGBHHPnJhLj0rnSMmGhEHdDsV1pDj4vdhpiMyvuNTkIgw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(52116002)(7416002)(54906003)(6506007)(66476007)(66556008)(38350700002)(44832011)(4326008)(6486002)(6666004)(26005)(36756003)(8676002)(83380400001)(316002)(508600001)(38100700002)(66946007)(5660300002)(86362001)(956004)(53546011)(33656002)(2616005)(6916009)(6512007)(8936002)(2906002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6256
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	cf10bfb2-f5ce-4d8e-3a94-08d97f700495
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0/VM9O/P00YKPoXdeiF6GwJU3NSwQCv9x7/5GL5hB9KujxTByeYZ41qQcYSNNtRTVMlJmdTWFGd4VO0ELGdLf89OpJWYzZ7zLxl0r91D/TAOUSL520LIdVSVYyEQ7PktVO3fQfHLGjI8eHNk4OufGgiqEDf4bkyNBbbXoP+qpqjcK1bw1necxflnzyAPoOBJUCoXJi8ueHv8CkuwrWbJ2hbppkJNeCafv34c818TqAN70OKh3BIoyDb1VKCDyIRqEv93c0MfkrAxae8pW5ls8oTxfT3ZzrIzSKCcpUT3S7BUixsDImxitYFIm3ukFyawz50kxlHfM7UzW6iLmjny51KCpMbpFHxk13ghY1W1656qXBTe4aPRU+BfjC60eOzQGbPsonCBlZBNDJOg8wdZMm1ltibuxotGY3EVQG+XyJaTboaXVqwVl/+S7Zl945+3TsYykmKw5/dwIcf25e07kG6jG9eGz/1UK24ayuk2ScBA7Mkq9vPBXGz5zQZwuNC4shbo4xpFOXN2Cynrthl7BLPx2w6mKJ59AYeoBRMLqr0WG4/LfCAGGt+szGwhQb/LH03TKpJPwNcQp8OKO/lehdxABbTVJ3bmYgMrfzMqQTkFYX3CQRJYof0woK/d8HW3rIAS8W+7dGWWjUwwHmsDXjK7JL4XR3lwzAoQsKnp5wVXI/MtfgVE/eiK5bEjjt1xXRZ+ckU7Pl9G6ohBZb6oZDmBZAbooDzTP6Yg4uxfDzg=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(26005)(956004)(36756003)(508600001)(316002)(8936002)(6512007)(81166007)(5660300002)(44832011)(47076005)(6862004)(336012)(70586007)(54906003)(356005)(53546011)(86362001)(2616005)(6506007)(8676002)(4326008)(2906002)(186003)(6486002)(6666004)(83380400001)(70206006)(36860700001)(82310400003)(33656002)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 15:29:10.0423
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 020aded2-5708-4877-8717-08d97f700e3b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4957



> On 24 Sep 2021, at 15:02, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 22.09.2021 16:13, Luca Fancellu wrote:
>> +static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_h=
andle,
>> +                                                  const char *name,
>> +                                                  unsigned int name_len=
)
>> +{
>> +    dom0less_module_name* file_name;
>> +    union string module_name;
>> +    unsigned int ret_idx;
>> +
>> +    /*
>> +     * Check if there is any space left for a domU module, the variable
>> +     * dom0less_modules_available is updated each time we use read_file=
(...)
>> +     * successfully.
>> +     */
>> +    if ( !dom0less_modules_available )
>> +        blexit(L"No space left for domU modules");
>> +
>> +    module_name.s =3D (char*) name;
>=20
> Unfortunately there are too many style issues in these Arm additions to
> really enumerate; I'd like to ask that you go through yourself with
> ./CODING_STYLE, surrounding code, and review comments on earlier patches
> of yours in mind. This cast stands out, though: I'm pretty sure you were
> told before that casts are often dangerous and hence should be avoided
> whenever (easily) possible. There was a prior case where union string
> was used in a similar way, not all that long ago. Hence why it now has
> a "const char *" member. (That's still somewhat risky, but imo way
> better than a cast.)

Hi Jan,

Yes I will use the .cs member, I will have also a better look on the patch =
to
find the style issues.

>=20
>> @@ -1361,12 +1360,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>>         cfg.addr =3D 0;
>>=20
>> -        dir_handle->Close(dir_handle);
>> -
>>         if ( gop && !base_video )
>>             gop_mode =3D efi_find_gop_mode(gop, cols, rows, depth);
>>     }
>>=20
>> +    /*
>> +     * Check if a proper configuration is provided to start Xen:
>> +     *  - Dom0 specified (minimum required)
>> +     *  - Dom0 and DomU(s) specified
>> +     *  - DomU(s) specified
>> +     */
>> +    if ( !efi_arch_check_dom0less_boot(dir_handle) && !kernel.addr )
>> +        blexit(L"No Dom0 kernel image specified.");
>> +
>> +    dir_handle->Close(dir_handle);
>=20
> So far I was under the impression that handles and alike need closing
> before calling Exit(), to prevent resource leaks. While I will admit
> that likely there are more (pre-existing) affected paths, I think that
> - if that understanding of mine is correct - it would be nice to avoid
> adding yet more instances.

Ok sure, I will close the handle before the blexit.

Cheers,
Luca

>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:30:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:30:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195494.348221 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTn9Q-0005yG-Bt; Fri, 24 Sep 2021 15:30:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195494.348221; Fri, 24 Sep 2021 15:30:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTn9Q-0005y9-8q; Fri, 24 Sep 2021 15:30:12 +0000
Received: by outflank-mailman (input) for mailman id 195494;
 Fri, 24 Sep 2021 15:30:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mTn9O-0005xw-KY
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 15:30:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 881846e6-a919-4662-a658-ba7f17234707;
 Fri, 24 Sep 2021 15:30:09 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2052.outbound.protection.outlook.com [104.47.5.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-1-E8LQTbL8No2h6mhgrTfSqg-1;
 Fri, 24 Sep 2021 17:30:07 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 15:30:05 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 15:30:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 881846e6-a919-4662-a658-ba7f17234707
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632497408;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=QIrnMBnIvxtySEdOc2GunU4qTCLynbDICb1WQvw3Inw=;
	b=dWdQiTCFBhdgS9VmwGvihATopi4zHpThZ36xTMc+eFlzw2Ag0qQDWo1Nj42kSQYewzQDIa
	kEKJNHp6wL57vwpVWbszdJpCPv17viUu3zru1w5dtoFTAnrqBWcvirsWfCr8X4SN64WRLF
	goLnW9N38uEhSyz0AOLr6Yvn9m0zQNY=
X-MC-Unique: E8LQTbL8No2h6mhgrTfSqg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n+IcIT7iTZt48Ho8ak/k/1m7fwe/eKK6iUlfFpw2Z1hf6Pi99ELh6kR3rASGweFYsjQ25vuakeQYVxgMNP2sBoJhF+e8Iz8z0pFVAADaz/d9h5VInVKC8bzEpF4SJOpVgOfGUidjxKrPF5AO70x7Iu4JEs+gTNRl14UBWrn3y9qFVyvfgvX+Y1RkDU1Q18NwDQw/5c8i2AcrQZ1yZzT63coZyXO1kNnEBnTPDMv3sIoHiybz3woND/IyPbez9eSKDqTtj2wwI9hUnopZzrAapDhpAs62D3rwFI3JaDf4z0/XZib4lb5B2BbsvN+wyGGpf8poKh/uarSUJakL5wEMhg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QIrnMBnIvxtySEdOc2GunU4qTCLynbDICb1WQvw3Inw=;
 b=jyZwAr0K320Y15CKXC3X/Z/QOdTAHLOZCYSvNQnArZ4H1U1bndurwMy62wQRMAbkEXUHggrxWB8WuLptdAH7Qzg2OkDQao6Du61YPTygpZNHsskhlBLf+E6cdDpseD5LtLUXW1nqg+sB6fhybWihJEaRBHiLGgT9v7Y44KJnOHQcHfCcmzcCQLuxrjBZTKfhilYpz4qqySPuPQk3lrQLQS64S543cmgFKcNfEVlGDi83uJuMDN6HXMwQW94NculsUeimeBuTiG6lCpN2apIPd0CKb/yfB6jF54vEPcIue7ecxWccSmCpEWMVGmJWCG9adHfPuA0HeKcdo1sj1NX+QQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>
Subject: Re: [PATCH v2.1 14/12] xen: Switch to new TRACE() API
Thread-Topic: [PATCH v2.1 14/12] xen: Switch to new TRACE() API
Thread-Index: AQHXsVkMcY6jWQHaHkuxnpRFfdaE/Q==
Date: Fri, 24 Sep 2021 15:30:05 +0000
Message-ID: <6136f9d77c3aea601a409bddb28a8f81f5709e3a.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920193234.19409-1-andrew.cooper3@citrix.com>
In-Reply-To: <20210920193234.19409-1-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2f3aeef8-b1e0-48c9-3d3d-08d97f702f3a
x-ms-traffictypediagnostic: AM0PR04MB5826:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB58268D9A1F8BD8FDE97BD773C5A49@AM0PR04MB5826.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 DNTwUI5QYF7BcqCHTCsmsNShu7uwkqlE4OJcoJDnzuP3tpQam9eMjyUZxxCGJaBb/TwJO6w8S5IdSwWKn2Plzj8bo6h2WR74PhPJY/PCkflm2xmM9Obx+dt99O4H9ShJAXnQcYNJ2oFzBoJF7dlsf4uXWfJillrvgeHUMj0q51Aef0DHxJm6LU7oRbiGrwu0BWHry9S68Ka5PvB6VrXr7BEhL9Vdn70Vc8RR2l0j/cAi7U8FIXcYVGehYhly3hXsTEsCht5KfQTl7CbcInKIT/8qFT1OOHzDlULnZz5dF6e3AEj3H6fPMSZQo3nYdWHtTrj29y2hkFDbOAY9Ye4eHIqAzdvxaxo0L4pjwXY1gzarn/DmXN0WP4UOMdKPOa5htY9O1yUC7FCuOWzdt5lhjuPmnATlBBHM8gAWUWZnYCEratpK16NRsxjsJsnHAvFc/drN9nKdeKSOS4NIAHURe13DJWZ5yMj3CoQm8FhrkOpS7fQNGJDFZs2RNY6qRUnEpOPAH3b97e9gYnt494IZFjao0Wc3aFOWf7WBlsnF1Nuaka+T2LcFJ2OrzLgV549gQMmwPfyXjH+TTPKFUdeNoYFO3ZVAXaxC/NQLc6BZYxASDw+4OZQmAhetXNwFm2ctAw034r/5NJsmsaD1Z0QJyWO6qRaK2kaxVX7eeqzDO25pl/1a3bOf7Ip14INmXEec/LGxdc3RwJQCE59ZdDZ9/flUx7GGNWl5llRATjYCRmCsdGrP5DsrVHrndXSgemRC2ewjPCfEZWQU3YblkhFxE59/WE2csGi4oDdnyl4qtsPOO1ZpaS4hOq48Ut5AnZJnbh+VL2LpvbJ4wDSMwrNRHA==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(99936003)(66946007)(5660300002)(186003)(4326008)(66616009)(8936002)(6486002)(86362001)(508600001)(76116006)(2906002)(64756008)(66556008)(110136005)(6506007)(66476007)(38070700005)(316002)(66446008)(91956017)(8676002)(36756003)(54906003)(122000001)(2616005)(71200400001)(966005)(6512007)(38100700002)(83380400001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?NDBvUDR0Nk54RVdURjgxRkFRbm9DZVN5dDRNT3lFZlkrZnl1dVdQa3RCbzZF?=
 =?utf-8?B?eWV4ZTlMeDl4UVI3bUQ4b1lDK1BaZFBLb3kwQkFVSmtxSDM3RlBDM1MwZlhQ?=
 =?utf-8?B?QlgzcGxGcG9JaGpGbXl6ZzdwdS9HNzFpdGx2N2NrdTR3YVhHSGM4YnJIUkJo?=
 =?utf-8?B?eXdSSlJuNXZPUWp1VUNPTUpjUlhpMmE5UVFxNGxRVEo0d0llUHBnenhraGVs?=
 =?utf-8?B?eGVNRWpxYVFKTHltM2MrNXFwaVErYTdiK0kweDczV2p5WTM3MndXVjh0eFdk?=
 =?utf-8?B?R3psbDlzOWRCQllKcVBENCtOekVodUQ1TEhWV1J6ZVlTZEY3SkpjckIwM2ZI?=
 =?utf-8?B?c21XdFhTa2JzM3N1a0htSDlSNkp3WmpwdjFueWU4Z1RBdG1ZOVdxOVRlNjdO?=
 =?utf-8?B?by95d2gxMkhIL3hCWDhnYXY1ZWEwb0VtSDVQOEwvQU1qRjlUMHc5dEJVSmtp?=
 =?utf-8?B?UlVnWlYvNFo3TEtjaVBNRnU1WXg2SkdKZjJvNVdhc1NmWWVpU1ZLblZrRzlQ?=
 =?utf-8?B?R2ZZR2xqM2MyZzZMb0FONElsaTQvYjZiMDYrdEJ3ZTg3Slp1ZUtYS2lmMWlE?=
 =?utf-8?B?L2tHYis0TkhuUkhVcExpVnpLMldsK2RmSTEzT3gydi8rdkNJOFpmdkU5WXNM?=
 =?utf-8?B?d3RWTHB2UjhERTJ2WmNlcHBadHN0bUhKbzNLaW9zRlVtaFk3Mi9COHpVVHlN?=
 =?utf-8?B?enJ1Y3JHRlYvZmpFVCsxUGR3Z2kzNkpBVDRHODQwR1liN24rTFQ5SXFUQ0FM?=
 =?utf-8?B?YStic2JoV21aR3BRZzREQ2FsTTFhK2RDTDAxU2Z6QWxpeHVKZkx3WDZKK1NJ?=
 =?utf-8?B?TkE2Q3ZaN25DSGF4TUd4S1FmT2IvMjEzYnJQVXlFcHMrcXpKd2lvTDIyR0hU?=
 =?utf-8?B?dVJoRzFhYVRwei9EVXVVWFNJVDAybHI4L28zNEFsUzJZUTljSG5HcHQ5Y2FR?=
 =?utf-8?B?ZkFHZ256M0grc2lZZTM3RGs2cnFrbGZtVFduYXN5VXZwV09vTDQxRTdxS3ZQ?=
 =?utf-8?B?K24rTUFwZmZjUEREU0NGb1pBOHVLRndrWWt3ck02WkR3bWJOMVZqSkMvbE9U?=
 =?utf-8?B?TlpobXlGSnkvSUo4WUtPTmZiMjVzM09FMmVoRkJCTUtoblJGTWxuOWlpVlJh?=
 =?utf-8?B?M05HMHZxTEo4cTJmeUpwa0lsdjNCSjRGN1FzL1hZaFhTSmNoa2cxc1BHT2dW?=
 =?utf-8?B?M3RvRHVZdHQxREhoaGw2aUFVTEJsSUxXU2tVTllUVGRpUlczWkJQUGJ0UWlk?=
 =?utf-8?B?TlRSK0RYVXRSb3BwWm1EWU8rZ1JIbXRQZlNpNGlRM2xUWkk5WGpjRXhZeGJ3?=
 =?utf-8?B?dnNvRHRJOXc0TVF6TWNVQXJNTnVCcWN0T0QyUHhzU1owMnlEM0hIcjl0Y2s5?=
 =?utf-8?B?RllqWjc5U0ZLWWJVbk9wM0ZQbTBtd0N5ZitmRU1wUVFXalRlQ3NzbHdYQk9R?=
 =?utf-8?B?Rk5RODUza3JRd0R0QlVJNXpNeDFWSldpSzR5UHFla2RTOEp6T0RoMGYrL3Ux?=
 =?utf-8?B?c1VjZkoxMWc0RVJncklvcnlMNGlNYlk3UG9rL200VVRWUU9ZdTV6aTV5eFRS?=
 =?utf-8?B?VkNyaFU4S0VMd3ROK2I2RElXSzZqdmNUOXZNQmRGMmxxbXFFTXgvTzd5aFZJ?=
 =?utf-8?B?ZCtpZFlOa2JrNFB1TU1mZGp1S1BqS2NSRXdQdjU2dllMQ204ZVVHalE5R3B1?=
 =?utf-8?B?ZWpEdkNQNXJ3MytyN21SbGpNelFnaGg1ekdid3BBTEkxRllCbUhCWkZzY2NV?=
 =?utf-8?Q?0WW/WyeyzAe+Nnav3zHg0ivWQS+zS64yNVJTQ1E?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-s25DmJGB9spnQ39XkZXG"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f3aeef8-b1e0-48c9-3d3d-08d97f702f3a
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 15:30:05.4426
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4fEHOsF3mCMf9PpzWzjLDq27KaBwUaPe225xmNilL/DrUXoECdaHeS8BWY2VMqzY/hUpryBhbFe6KqJMBq6mIQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB5826

--=-s25DmJGB9spnQ39XkZXG
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 20:32 +0100, Andrew Cooper wrote:
> (Almost) no functional change.
>=20
> irq_move_cleanup_interrupt() changes two smp_processor_id() calls to
> the 'me'
> local variable which manifests as a minor code improvement.=C2=A0 All
> other
> differences in the compiled binary are to do with line numbers
> changing.
>=20
> Some conversion notes:
> =C2=A0* HVMTRACE_LONG_[234]D() and TRACE_2_LONG_[234]D() were latently
> buggy.=C2=A0 They
> =C2=A0=C2=A0 blindly discard extra parameters, but luckily no users are
> impacted.=C2=A0 They
> =C2=A0=C2=A0 are also obfuscated wrappers, depending on exactly one or tw=
o
> parameters
> =C2=A0=C2=A0 being TRC_PAR_LONG() to compile successfully.
> =C2=A0* HVMTRACE_LONG_1D() behaves unlike its named companions, and takes
> exactly
> =C2=A0=C2=A0 one 64bit parameter which it splits manually.=C2=A0 It's one=
 user,
> =C2=A0=C2=A0 vmx_cr_access()'s LMSW path, is gets adjusted to use
> TRACE_PARAM64().
> =C2=A0* TRACE_?D() and TRACE_2_LONG_*() change to TRACE_TIME() as cycles
> is always.
>
Was this supposed to be "as cycles is always 1", or something like
that? (But maybe it's fine and it is me. I'm no native speaker after
all...)

In any case...

> =C2=A0* HVMTRACE_ND() is opencoded for VMENTRY/VMEXIT records to include
> cycles.
> =C2=A0=C2=A0 These are converted to TRACE_TIME(), with the old modifier
> parameter
> =C2=A0=C2=A0 expressed as an OR at the callsite.=C2=A0 One callsite,
> svm_vmenter_helper() had
> =C2=A0=C2=A0 a nested tb_init_done check, which is dropped.=C2=A0 (The op=
timiser
> also spotted
> =C2=A0=C2=A0 this, which is why it doesn't manifest as a binary differenc=
e.)
> =C2=A0* All HVMTRACE_?D() change to TRACE() as cycles is explicitly
> skipped.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-s25DmJGB9spnQ39XkZXG
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFN7vwACgkQFkJ4iaW4
c+5Dfg/7BeS3v7YzuzWmh2o/w1VLaaiX38VvZ/KcG1X8pBIjKnEbk7v894wUDnyg
oxMxJ3eN2Rmb+QR9AIRTCfms0gfPQfkF5mZpAb/z778sLn+ifARaxkQ7bi5LKDtP
How+m1YCl0FRXEGxheC9c/m5+6d4mZGtK2SjiANXsbyAiXWTHFaAJTm7L51Ru69t
BTGkwQC28++KpahoPE4OY+WxA89mKeFCgpZaJap4TethuxEmLZKFcdQKYOC+fRu+
0+Wa8QydnwtbTVwaHXzFxfpckT33mh2E0vGWJ5GM7dvbr9KzBmHCqAxL+/0gK3g4
1HWaY3I2qbUhjrVeP2czhKRkas4nqO7SHLcsRXKaEDzwBRgeRYUlTTyzV7HJSuQ3
6QhNhRB4shwqoXZ4xVpAWl1K/g2YxDY4s3OpibgcYKS4zymn9e7p6YMWkOC0loVP
MjfXMwheLIYJjXokmZcD2UHOC0CAiLQfQ8wCGxzulWPupTIG3HNYqtUDz5xaa4Ch
dSjIZiIH4AsxHyezG4awn7BtcUW+cYOqKhMJ2JkAA6d6DPLJSlYmVyUKEVeRVvOd
S7g6/Dsd8glHKxRiLdcyuhvVu9UjJUsM69yh4eG9MePNImGnqcJyUN9+E6HKL3LF
zyCu4VlVsJ+rLmb1MSvbqmO6xIgPzidCwDHbWYLHOT70Oz4FUr0=
=VHUo
-----END PGP SIGNATURE-----

--=-s25DmJGB9spnQ39XkZXG--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 15:55:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 15:55:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195502.348232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTnXl-0000bY-Fx; Fri, 24 Sep 2021 15:55:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195502.348232; Fri, 24 Sep 2021 15:55:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTnXl-0000bR-D6; Fri, 24 Sep 2021 15:55:21 +0000
Received: by outflank-mailman (input) for mailman id 195502;
 Fri, 24 Sep 2021 15:55:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnXj-0000bH-Ur; Fri, 24 Sep 2021 15:55:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnXj-0004SU-MQ; Fri, 24 Sep 2021 15:55:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnXj-0000oI-EO; Fri, 24 Sep 2021 15:55:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnXj-0003J9-Du; Fri, 24 Sep 2021 15:55:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KdTQVxKwPBteNtqr652iRXsPOVdIWVu6htTmBZNUG+M=; b=V7ojLSxEw71fXUorvu63XUMuLr
	E3THUa+1DPk+7hcO9ZuLEng/c+MX0gVwxEnTfZadNT5p0+MkRGE4Q541tDPlh99p3cRPUydsjozeQ
	n8eZ5pGmj66vLYhQBsjvbO5KPZJeauQgHUDLIUudRJQxNiBbbJMe5ONyFWeOYG2O4qCo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165175-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165175: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=d60915b7516c87ec49ad579a1cb8ff9226d85928
X-Osstest-Versions-That:
    ovmf=7ea7f9c07757b9445c24b23acf4c2e8e60b30b7e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 15:55:19 +0000

flight 165175 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165175/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 d60915b7516c87ec49ad579a1cb8ff9226d85928
baseline version:
 ovmf                 7ea7f9c07757b9445c24b23acf4c2e8e60b30b7e

Last test of basis   165170  2021-09-23 18:41:23 Z    0 days
Testing same since   165175  2021-09-24 03:46:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Zhiguang Liu <zhiguang.liu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   7ea7f9c077..d60915b751  d60915b7516c87ec49ad579a1cb8ff9226d85928 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:06:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:06:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195511.348247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTni1-0002zM-Kp; Fri, 24 Sep 2021 16:05:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195511.348247; Fri, 24 Sep 2021 16:05:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTni1-0002zF-Hd; Fri, 24 Sep 2021 16:05:57 +0000
Received: by outflank-mailman (input) for mailman id 195511;
 Fri, 24 Sep 2021 16:05:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnhz-0002z5-VM; Fri, 24 Sep 2021 16:05:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnhz-0005C4-LK; Fri, 24 Sep 2021 16:05:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnhz-00018s-AP; Fri, 24 Sep 2021 16:05:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTnhz-0001Oi-9n; Fri, 24 Sep 2021 16:05:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hCaHDX7C7c1rKGfCTPncsbpmiGy8G71aJJpX+4DbLHk=; b=VG3XPOUj6DLmoUPYLHsncMZVKq
	2OqXk4I+UrOIPCtY3iVX9uJmssPCC1YB6ag+xqPd7pcQchy0fXi39m1sFJlQ5p2cZC6CWj4dOZdwl
	Ddet3GrxO4dNEav88N5KUmiBiMZCEwDuJDUSLe0T3AYXAQxTdc6GWk4vvswJAgKsfyCk=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165177-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165177: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=e0e0bf662896240f2293eb60e23d2570f110fa5a
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 16:05:55 +0000

flight 165177 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165177/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              e0e0bf662896240f2293eb60e23d2570f110fa5a
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  441 days
Failing since        151818  2020-07-11 04:18:52 Z  440 days  431 attempts
Testing same since   165177  2021-09-24 04:20:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 75556 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:10:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:10:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195519.348261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTnmc-0004hr-AP; Fri, 24 Sep 2021 16:10:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195519.348261; Fri, 24 Sep 2021 16:10:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTnmc-0004hk-61; Fri, 24 Sep 2021 16:10:42 +0000
Received: by outflank-mailman (input) for mailman id 195519;
 Fri, 24 Sep 2021 16:10:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N27y=OO=citrix.com=roger.pau@srs-us1.protection.inumbo.net>)
 id 1mTnma-0004he-4z
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:10:40 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f1985d62-59ed-46cf-8685-1cc2a19a2c1c;
 Fri, 24 Sep 2021 16:10:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1985d62-59ed-46cf-8685-1cc2a19a2c1c
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632499838;
  h=date:from:to:cc:subject:message-id:references:
   content-transfer-encoding:in-reply-to:mime-version;
  bh=MeSee60f3BLx3E2EiiB1a9aLCb4HxN/cClOSIv/zLyA=;
  b=VFXLhOQY+B6vqPZwBdiCzJMgY0CwA4KZ6bGjaSglYI22P5ohsCLS/BdW
   C0a8EnOGFbFPCMWIA2HkAeEi2cWLZd66Q1Ilhq4N+WwnCSdiuS3GaJZvs
   TJUUFLHf7Dy05kdB+EvitoR6dJce8sU9c3OpKYQGtMYV39LOO6z6VYYUG
   M=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: cdY/KAXvb36ZmXszRwkZeYs/4tc44ICszkczQE0nNDdY/pb+Q0i3LEF9X3OTeiZTmgjVCVg8WK
 vldKa7wVkx4U34vjHoruSWZYBeRWc7+/dEWVGM6/1FPqh3aBAVkksCUEgPwsWybF83bfiLAvdP
 pE4LHchTgzC3fwbWAO3qF8CZ3bsW9C+Qyxy02euj5Rw3uefRCkZMwE7e1OY4UL9LoKVFRdAWQM
 6B4h/fgF/YyRZ+BSxjxs9ODwbTsaJZe2uLb5gesvH4f8xslBMaRR8Y7X3khyIhcxgjEhlsLPFp
 kHyZMTmASBw4l9ZTEp9TmDpS
X-SBRS: 5.1
X-MesageID: 55373547
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:OpyUj6t1cnOjUGNtDEt/HUzV5+fnVDRYMUV32f8akzHdYApBsoF/q
 tZmKWGEOK7ZMGPxLd9+bYvn9BhT78XQy9EwTFBkrSAxHy4S+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524ThWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpllraOcz4WDvH32+kvCV5jDxhjJJwF9+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DJoQQt2sm1TjEJf0nXYrCU+PB4towMDIY250VQq2DO
 5VxhTxHTS7eRyARZAssV58FzOixnGHOTmNakQfAzUYwyzeKl1EguFT3C/LUZd6iVchThlyfp
 G/N4yL+GB5yHOKYzT2J43e9nNjllCnwWJ8RPLCg//ssi1qWrkQMDDUGWF39puO24ma8Ud9CL
 00f+gI1sLM/skesS7HVRAakqXSJuhodXdt4EOAg7gyJjK3O7G6xAWwJTSVdeccmnMAzTD0uk
 FSOmrvBByFrsbCTYWKQ8PGTtzzaES8RIGwZbCkIVzwZ8sLjq4E+iBHIZtt7GavzhdrwcRngz
 jbPoCUgirE7ic8Qy7797V3BmyirpJXCUkgy/Aq/Y46+xlonPsj/PdXusAWFq6YbRGqEcrWfl
 FsupfKn09BSNtKApnbKe/k0AaiYpPnQZVUwnmVTN5Um8j2s/VuqcoZR/CxyKS9VDyoURdP6S
 BSM4l4Luve/KFPvNPUuMt/tUqzG2IC9TYyNaxzCUjZZjnGdnie89SdybAa722n3mSDAeoluZ
 M/GKa5A4ZscYJmLLQZapc9BjdfHJQhknAs/oKwXKTz9itJyg1bPFd843KOmNLxR0U99iFy9H
 yxjH8WL0Q5Dd+b1fzPa94UeRXhTcyNhWcqs9pMKKrLZSuaDJI3HI6WKqV/GU9Y595m5a8+Sp
 i3tMqOm4ACXaYL7xfWiNSk4NeKHsWdXpnMnJy08VWtEKFB5CbtDGJw3LsNtFZF+rbQL5actE
 5EtJpXRatwSG2+v02lMMvHAQHlKKU3DafSmZHH+PlDSvvdIGmT0xzMTVlG0qXZSXnbp75tWT
 n/J/lqzfKfvjj9KVa7+QPmu00mwrT4anudzVFHPOd5dZAPn940CFsA7pqVfzxgkJUqRyz2E+
 RyRBBtE9+DBr5VsqIvChLyerpfvGOx7RxIIE27e5LewFC/b4mv8ntMQDLfWJWjQBDHu5aGvR
 eRJ1PWgYvcJq0lH7thnGLFxwKNgu9a2/+1Gzh5pFWngZkiwDu8yOWGP2MRC7/UfxrJQtQasd
 FiI/91WZeeANM//SQZDLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIeIsYo0/w
 OontMoH0CCFi0InYoSckyRZ12WQNXhcAa8pgY4XXd3wgQ0xx1AcPZGFUn3q4IuCYslnO1UxJ
 mPGn7LLgrlRyxaQc3c3EnSRj+NRiY5X5UJPxV4GYV+IhsDElrk82xgIqWY7SQFczxNm1eNvO
 zc0ax0pdPvWpzo41tJeW22MGh1aAEzL80P8/FIFiWnFQhT6TWfKNmA8Zb6A8U1xH7iwpdSHE
 GV0EFrYbAs=
IronPort-HdrOrdr: A9a23:OsWp2qoyaDW/njmefzxnDXEaV5u6L9V00zEX/kB9WHVpm5Oj+P
 xGzc526farslsssREb+OxpOMG7MBThHLpOkPMs1NCZLXTbUQqTXfpfBO7ZrQEIdBeOlNK1uZ
 0QFpSWTeeAcWSS7vyKkTVQcexQueVvmZrA7Yy1rwYPPHFXguNbnn9E426gYzNLrWJ9dPwE/f
 Snl656T23KQwVpUi33PAhJY8Hz4/nw0L72ax8PABAqrCGIkDOT8bb/VzyVxA0XXT9jyaortT
 GtqX232oyT99WAjjPM3W7a6Jpb3PPn19t4HcSJzuwYMC/lhAqEbJloH5eCoDc2iuey70tCqq
 iBnz4Qe+BIr1/BdGC8phXgnyHmzTYV8nfnjWSVhHPyyPaJDA4SOo5kv8Z0YxHZ400vsJVXy6
 RQxV+UsJJREFfpgDn9z8KgbWArqmOE5V4Z1cIDhX1WVoUTLJVLq5YEwU9TGJAcWArn9YEcFv
 V0Bs203ocWTbqjVQGagoBT+q3oYpxqdS32BnTq+/blnQS+pUoJjHfxn6ck7zA9HJFUcegM2w
 2LCNUvqFniJvVmGZ6VP91xM/dfPFa9Ny4kAFjiUmgPK5t3Tk4li6SHq4ndt9vaMqDh8vMJ6e
 P8uRVjxDcPR34=
X-IronPort-AV: E=Sophos;i="5.85,320,1624334400"; 
   d="scan'208";a="55373547"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AyaZYfWqBEjCHMFJC3/Hs5wAdk4CKDaAYEcysPbMu7Am+PnaYHXCgEWMjkVnxl9KXCDPCVY75b2FT8sUsYZwf+748hHRKN+bCeY180QFL8b/Wq+yyVW+IjdA8PCtEKwOhnQ7YVi/Q+IOKgaVeBClM/KzBDqbcCRZs+896m6AS9Q9fkATZ9ubpkHYK6bLBfEr5uIc8pjQ6dd1QGLtPh4ArnOkq42S2LDxnvvIc4PH2KVau47ZcVbSFKBO8nCvwqjR9h+3/MygeOGsvyiiZDULrFKTdopiOqNIv9tcqvmmH/DWPommXKxE1XhF+L+SWi3avTXlRltrLK1vJ74qy3nr6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=O8nMueq38I2p+lvETxLMhIuc2Rdsh+z6aiNM5xAQUpE=;
 b=Y+pOPCmr5CQVBsJgVtHEQPEMysjRUDFgs025dC7iz19VYohXJLusX9465i3iHGNbma444Ce0F1rwO9bRHbzENhlzhLblWO1wVOiuz9L/3WaCwEiOkCgAVye5YA6EUPchSH3X48PJNDP6EQqgP/4zaI3eGcHIbNCG1jYAA6TlPkGhIOmV2Dv1JchuCOto6cci/qN2jxil+Q0qL/FUd3JV3bFmr8Z68yw+IRGmAfkPiAsx16eU2x6vqbuzJAlg7Q4uUfbxlmO1X0Z5qd65lMTh8jvVO26ekmuA2dP8ND3qJwNqSIhvakMqMeiKEJFdsTMONytFPgRyPjlDS58N4sDzhQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O8nMueq38I2p+lvETxLMhIuc2Rdsh+z6aiNM5xAQUpE=;
 b=vn5iYFbs2sr+PP3OSsquHSxERxuhzBCnbkwjY84mutjPvxXV6C+YES355Bl30p1gsJj0dOC9N3DTcc4Rq5Hafndl1zYDLyv8YoIdlYdIPuWrKODNoshrxm+y4baCzllbUOt2BCvwizKZ6LFnQxduRFO37mG6QMJTljJbSCexeVg=
Date: Fri, 24 Sep 2021 18:10:30 +0200
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Oleksandr Tyshchenko
	<olekstysh@gmail.com>, <xen-devel@lists.xenproject.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in
 struct page_info on Arm
Message-ID: <YU34dsl4cSCBbfrk@MacBook-Air-de-Roger.local>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
 <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
 <YU2PT4rUts8KljKe@MacBook-Air-de-Roger.local>
 <04400e18-dde2-4b90-4056-f56c5d7937af@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <04400e18-dde2-4b90-4056-f56c5d7937af@xen.org>
X-ClientProxiedBy: LO2P265CA0126.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9f::18) To DS7PR03MB5608.namprd03.prod.outlook.com
 (2603:10b6:5:2c9::18)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f361e42c-f2e5-496f-99cc-08d97f75d7e4
X-MS-TrafficTypeDiagnostic: DS7PR03MB5608:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <DS7PR03MB5608C12985EDBC525B43A2B58FA49@DS7PR03MB5608.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PAcArE8NoewHrhdkAOnYtd6FDS8q7ym/R77POPL5pqDbu5y29VKOV1O/ZIdf3jIs7XAK+siFwPpXeaBlduuIxDzdFtAaOY9eRELaj2yCJx6g/+Gy1KrkUIzfBKnpCcF/lFsMCziSvQcZViHc1kIE/9SRi8kDvrE6M2c6p/CpKiT1FSC0R9ft/tvk/uKunybqNAsFMHXBtfxbFIOmkRkNYk51/htqpGVu1nZ8INh5EL13nf2+W3Gf+Z7Dw0z/ptOkJq3jHfgNTq5ddodJkioYWsW5S/oS19ooWYu6znGC1FwmOFzVJNDwXTmKXnrEoPhdyrzgskRPc+9JytysYFyqJJyj/3XYsKRW0syTtYKiv6QU6UhmaXVDJQA6Nv0IauYgYyIaLTdoEzB0GGqerOPNSjh/d4bw2nS9En4yCMTBheJNLViQVTsCFlLEsRfu/I9vJJ8t70iLTAl7gsXsIUuhANnHbgcrTiWgAV6W3Lfb85EB11G5Yk14/VTHqS9wae1SPbKQMVW1zgAwlA1W7X7rBkTuOf5RrNYjviS1Z2/wpLHJ32qQc3CiqsIeuwGOBiC2GaRvEjhSEC4VtnFvLZGJZSe/9XQUfRKv2pOin6XGafxRxexeWtAY49b53aXDzDz//KR2mgx1bmRTYi8GI3oE/hwiiDkKSgAMglagy5O+fcC3556FCFHDGhyebxXr5WJ6BPttJt3gYwaA4rXiXRwxyfVNZq83GUt770xBXsMmuoJqIQl5/ooIob1lwZwJl31IZeBcO88/pyBxOdhYwdMAYg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(956004)(53546011)(6666004)(85182001)(26005)(5660300002)(186003)(8676002)(6496006)(83380400001)(66556008)(66476007)(66946007)(508600001)(86362001)(966005)(2906002)(9686003)(316002)(54906003)(4326008)(6916009)(6486002)(8936002)(38100700002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OWdHRml6Ui9VVGlkNDFDZ29XNFc5S09rRHRyTm1LTFk3V05tRWEyN1JtYnJP?=
 =?utf-8?B?a2gvWUkzd1hoYjQ0OTFwekl5bVNRUkNvSnJWOXNzV2lOSXI3NFkyc0R1a3BX?=
 =?utf-8?B?SVQ3QUoyT0RiMXVBaEdOZ3kzOHBESi9vMTVuRjNEKzhIWVZvOCs3UW9JWHlh?=
 =?utf-8?B?bnkyckRLOUFDUWdyTG9oVGM5cWU4Ykc5MUpWNENISHZQM1NHSXRWT2RSaUhP?=
 =?utf-8?B?SGpXd29zUHVKbTN0WjJWa2VXbVZMbVUyMUxUQWpveU02K2RDSTZNWGZiaFBD?=
 =?utf-8?B?SlVZWnBvbUlENThSV00zc0dnMzhvbEh2c3ZJcFlDbGlRQjQ4UVVOb2tJTUUy?=
 =?utf-8?B?dEc5TjlYMVp0SmNTRHAwREpRYU44K2h4S2d0dXJYOGptcGFxV2M0L0pPTjAw?=
 =?utf-8?B?Z00zbzRscDBncThpS2czOVhadmhWVXJ1UGtjZHM1N1dlWDgvSWpncWZ2Y3pa?=
 =?utf-8?B?cXJLL1NyblE4aE9KWkhSMlh4RjJnTm84YjBZK2VJTWM1QUxhdy9JME4wMGNq?=
 =?utf-8?B?RzloTFR5MnJKL0NvMi9mTGd5UXhiUDFlS2x4cEt0SFZ2aS8yN2JQRm1KY05N?=
 =?utf-8?B?VkdBZVB4Y2xTUVBQWHdqNklCcDdiL1MxK3J2ZWVKZVFaa1dLdkJUOWdZZWRn?=
 =?utf-8?B?ZlllZjkzN1pyeUNMQjVGaHR2WVQ4S2hEbW1sNThpT09TRFNaWWg0eG05THgx?=
 =?utf-8?B?MHd5NzhOMTFoL2llY1hWdVd5Yy9pZGlVb1BoQXc1MmZ4WHh1dStLUzlLaG5p?=
 =?utf-8?B?TVlQTUk4bnpkNnlxS0FZMFcxSlVBMHIvYUtMSkZ0elQ2NkMyU3Q2ZzRETFBS?=
 =?utf-8?B?UmYwVmFmenE1T3ZQU1d4VXJQelVZcUszS3EyaGt4UkwyUVgxZTdmMFBzc3ZJ?=
 =?utf-8?B?ditGU3BqVmd5MEJZWXMxdU10R2M0UVJBcDRhVTRaRkVBZ0M5SnZWb045dEw0?=
 =?utf-8?B?Tm03RWhENngvWVhHTTlFSWh6MGpBY3FsZ0hFd3VzQWdHaC8rb0c4NEREeFU1?=
 =?utf-8?B?WnpMbzllNlh6djFCNDVNNjBPK3BSVTBWOENRMkkwSnEzcDg0Z2c2SWFZRDF5?=
 =?utf-8?B?dXRET1dOdWVzRHkrTFNmZGNrbnk0UWU0cmZQQ3dyMkZLWVNGK0hBWUU5cGdr?=
 =?utf-8?B?VlpVTm1oeU8xRUh3NjIycTd0dlo5NXR3cjlSRVBYYnhNYkJkYnZrbDk0MjV1?=
 =?utf-8?B?alBVZzVVZmRjQ29FL1hIT2FIUWRkZnpiSnJOUDFDQ3dsOXg4ckkrQktnMTVM?=
 =?utf-8?B?dXY2M05TZUZ2VkNPWWxCYnNhYjk1Nyt4WjkrNkx2c2tOcXVya1J3cXBFem9Q?=
 =?utf-8?B?eHZpOStPQW5sSWNpUWlnbjY2YnBteGdzT2JILzJYeTFrMUM0Q1prQVVYTFY4?=
 =?utf-8?B?N0xMbnl3TlhwbnhyUmcvbXg5MUoyZTRiWGxsQS9CelBJQ041b3RhNDk3Skdq?=
 =?utf-8?B?MHJWQ3BWYjlGTVV2R0xpYmNCaWh4VDA1cU90NTFuZzYwd0c3c29EdWU4UlFj?=
 =?utf-8?B?bDZvUkkyYm5vdkhST0RYdU9nQXZxd2hCRG5IVEJMZEcxdDhhWTZVMjJXQ21a?=
 =?utf-8?B?Z1kwSjBJY0VCMXN4dExHWkUvb2tWb2x6a2VlMCtvZXJyRnZIa2t5d3NETzdU?=
 =?utf-8?B?c2swQ0ZValNXSmMvcDhNd2Jpa1IwQm5FVi9SVURNdEpaQzhYSEpwZGtLNHgx?=
 =?utf-8?B?WEdwTjhZL1l0M211WjVuQzU3Zk1sRU0zTHZVMEI3L1lkSTV2WUNYbUlEWFZ2?=
 =?utf-8?Q?AJG3ZTTXpSToHqlsZDl1b/1OjBLi4YrBKm0GtFQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f361e42c-f2e5-496f-99cc-08d97f75d7e4
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2021 16:10:36.1382
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BrhBm+9ifDGaZIeifABgcfpFLrYqTwkETPI6FvYHUrSL1RCj65Qj6KgInGwPBvI8nJmZBsa6AQqRlC89Xe8ABw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR03MB5608
X-OriginatorOrg: citrix.com

On Fri, Sep 24, 2021 at 07:52:24PM +0500, Julien Grall wrote:
> Hi Roger,
> 
> On 24/09/2021 13:41, Roger Pau Monné wrote:
> > On Thu, Sep 23, 2021 at 09:59:26PM +0100, Andrew Cooper wrote:
> > > On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
> > > > Suggested-by: Julien Grall <jgrall@amazon.com>
> > > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > > ---
> > > > You can find the related discussions at:
> > > > https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
> > > > https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
> > > > https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/
> > > > 
> > > > ! Please note, there is still unresolved locking question here for which
> > > > I failed to find a suitable solution. So, it is still an RFC !
> > > 
> > > Just FYI, I thought I'd share some of the plans for ABI v2.  Obviously
> > > these plans are future work and don't solve the current problem.
> > > 
> > > Guests mapping Xen pages is backwards.  There are reasons why it was
> > > used for x86 PV guests, but the entire interface should have been design
> > > differently for x86 HVM.
> > > 
> > > In particular, Xen should be mapping guest RAM, rather than the guest
> > > manipulating the 2nd stage tables to map Xen RAM.  Amongst other things,
> > > its far far lower overhead.
> > > 
> > > 
> > > A much better design is one where the grant table looks like an MMIO
> > > device.  The domain builder decides the ABI (v1 vs v2 - none of this
> > > dynamic switch at runtime nonsense), and picks a block of guest physical
> > > addresses, which are registered with Xen.  This forms the grant table,
> > > status table (v2 only), and holes to map into.
> > 
> > I think this could be problematic for identity mapped Arm dom0, as
> > IIRC in that case grants are mapped so that gfn == mfn in order to
> > account for the lack of an IOMMU. You could use a bounce buffer, but
> > that would introduce a big performance penalty.
> 
> Or you could find a hole that is outside of the RAM regions. This is not
> trivial but not impossible (see [1]).

I certainly not familiar with the Arm identity map.

If you map them at random areas (so no longer identity mapped), how do
you pass the addresses to the physical devices for DMA operations? I
assume there must be some kind of translation then that converts from
gfn to mfn in order to cope with the lack of an IOMMU, and because
dom0 doesn't know the mfn of the grant reference in order to map it at
the same gfn.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:30:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:30:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195528.348272 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTo5E-00071l-1a; Fri, 24 Sep 2021 16:29:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195528.348272; Fri, 24 Sep 2021 16:29:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTo5D-00071e-Uf; Fri, 24 Sep 2021 16:29:55 +0000
Received: by outflank-mailman (input) for mailman id 195528;
 Fri, 24 Sep 2021 16:29:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ux8n=OO=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mTo5C-00071Y-PI
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:29:55 +0000
Received: from mail-wr1-x436.google.com (unknown [2a00:1450:4864:20::436])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 53515554-81fd-4d5c-904d-a7458f354e79;
 Fri, 24 Sep 2021 16:29:52 +0000 (UTC)
Received: by mail-wr1-x436.google.com with SMTP id w29so29088837wra.8
 for <xen-devel@lists.xenproject.org>; Fri, 24 Sep 2021 09:29:52 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id u5sm9395511wrg.57.2021.09.24.09.29.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 Sep 2021 09:29:50 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id 9722A1FF96;
 Fri, 24 Sep 2021 17:29:49 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53515554-81fd-4d5c-904d-a7458f354e79
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=user-agent:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=pK+e0gcZSWyBBOoTvejTaB+/zyvm0r9wyq1cvfaHXU4=;
        b=vQhxZeohU4exu/jwGyxm7tJJGZaGq1dMuhdkp2PAuU72UjELINp3Cv4kuFTRFkA8o+
         MP+WJ/BviYn3eA29rR0ZVCb8VWjFHhlHPMt6TjUg6zYqK9iPRjDYmpTzGKv0oJIfJHF7
         gBeeTPFv3sol0GldPG/WvzaC2TUVCtxXRl5xYTPuUxY5Mq2QlzQP34qAQYCvJSN1yVod
         IFSzLBfxlPMjFvI9twk5BAmYSKMXPwWOOfpBlCnS/szGEX6Kyhq0YzYFQvu5fnKGQtYI
         0MmZerE0WI4mSbebsZ6mPWwBlrEqbSUw57oZ+2Q4jD+v7aTL3/yRB2y3fgEF4buArC97
         sEUw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:user-agent:from:to:cc:subject:date:message-id
         :mime-version:content-transfer-encoding;
        bh=pK+e0gcZSWyBBOoTvejTaB+/zyvm0r9wyq1cvfaHXU4=;
        b=jvjz98WrUmg4+iJb1FF4rH3mMVtFt5aUH56GMWYc72l3cFDazdGjFR8GQczlKvckgt
         KrTdusWwB+gOe7CGxvvrpweM5AHfLdFjqHI6Dcp4I6X65Pwiks+fbM8k8OFrq8/pRBng
         WG8ln2MWuqEsegBS/0ue5O0I3YDb8Iq4baGajQ1kTbxfAQlq5yuVGg5qZNdxx8UaFzD+
         d+xqS8WV20B7vab1fJUQzvIxFe15Qe4Ncw7yEg8GN5oOYXGGTiLooCsTWAsgjWRTqHO/
         K3xiWUJDeJ46EayThDD+AorAtg1g73B1I37Il0tXbvp/nGL1C3E0+fiJsq0nZYxg2rzp
         wqiA==
X-Gm-Message-State: AOAM530f9LKbSK635zj5KNfdxNT5rKgnC5T+tMXEbF9fy6bt7YXGQWQp
	4wNusrQlsGsV0+wwYK8EXVWUHw==
X-Google-Smtp-Source: ABdhPJyr5rsOWUnFfKbQyu5WNqvZfGkfv2j2QIG/rgh9kYWQpaVzAPa8kgY70OTZ+bBraZdbrxBdSg==
X-Received: by 2002:a7b:c191:: with SMTP id y17mr3108178wmi.122.1632500991281;
        Fri, 24 Sep 2021 09:29:51 -0700 (PDT)
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Stratos Mailing List <stratos-dev@op-lists.linaro.org>
Cc: Mike Holmes <mike.holmes@linaro.org>,
 Mathieu Poirier <mathieu.poirier@linaro.org>,
 Viresh Kumar <viresh.kumar@linaro.org>,
 Peter Griffin <peter.griffin@linaro.org>,
 xen-devel@lists.xenproject.org, wl@xen.org,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Doug Goldstein <cardoe@cardoe.com>,
 Oleksandr Tyshchenko <olekstysh@gmail.com>,
 "Rust-VMM Mailing List" <rust-vmm@lists.opendev.org>,
 Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>, David
 Woodhouse <dwmw2@infradead.org>
Subject: Xen Rust VirtIO demos work breakdown for Project Stratos
Date: Fri, 24 Sep 2021 17:02:46 +0100
Message-ID: <87pmsylywy.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Hi,

The following is a breakdown (as best I can figure) of the work needed
to demonstrate VirtIO backends in Rust on the Xen hypervisor. It
requires work across a number of projects but notably core rust and virtio
enabling in the Xen project (building on the work EPAM has already done)
and the start of enabling rust-vmm crate to work with Xen.

The first demo is a fairly simple toy to exercise the direct hypercall
approach for a unikernel backend. On it's own it isn't super impressive
but hopefully serves as a proof of concept for the idea of having
backends running in a single exception level where latency will be
important.

The second is a much more ambitious bridge between Xen and vhost-user to
allow for re-use of the existing vhost-user backends with the bridge
acting as a proxy for what would usually be a full VMM in the type-2
hypervisor case. With that in mind the rust-vmm work is only aimed at
doing the device emulation and doesn't address the larger question of
how type-1 hypervisors can be integrated into the rust-vmm hypervisor
model.

A quick note about the estimates. They are exceedingly rough guesses
plucked out of the air and I would be grateful for feedback from the
appropriate domain experts on if I'm being overly optimistic or
pessimistic.

The links to the Stratos JIRA should be at least read accessible to all
although they contain the same information as the attached document
(albeit with nicer PNG renderings of my ASCII art ;-). There is a
Stratos sync-up call next Thursday:

  https://calendar.google.com/event?action=3DTEMPLATE&tmeid=3DMWpidm5lbzM5N=
jlydnAxdWxvc2s4aGI0ZGpfMjAyMTA5MzBUMTUwMDAwWiBjX2o3bmdpMW84cmxvZmtwZWQ0cjVj=
aDk4bXZnQGc&tmsrc=3Dc_j7ngi1o8rlofkped4r5ch98mvg%40group.calendar.google.com

and I'm sure there will also be discussion in the various projects
(hence the wide CC list). The Stratos calls are open to anyone who wants
to attend and we welcome feedback from all who are interested.

So on with the work breakdown:

                    =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81
                     STRATOS PLANNING FOR 21 TO 22

                              Alex Benn=C3=A9e
                    =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=
=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=
=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=
=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81


Table of Contents
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80

1. Xen Rust Bindings ([STR-51])
.. 1. Upstream an "official" rust crate for Xen ([STR-52])
.. 2. Basic Hypervisor Interactions hypercalls ([STR-53])
.. 3. [#10] Access to XenStore service ([STR-54])
.. 4. VirtIO support hypercalls ([STR-55])
2. Xen Hypervisor Support for Stratos ([STR-56])
.. 1. Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
.. 2. Tweaks to tooling to launch VirtIO guests
3. rust-vmm support for Xen VirtIO ([STR-59])
.. 1. Make vm-memory Xen aware ([STR-60])
.. 2. Xen IO notification and IRQ injections ([STR-61])
4. Stratos Demos
.. 1. Rust based stubdomain monitor ([STR-62])
.. 2. Xen aware vhost-user master ([STR-63])





1 Xen Rust Bindings ([STR-51])
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=
=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90

  There exists a [placeholder repository] with the start of a set of
  x86_64 bindings for Xen and a very basic hello world uni-kernel
  example. This forms the basis of the initial Xen Rust work and will be
  available as a [xen-sys crate] via cargo.


[STR-51] <https://linaro.atlassian.net/browse/STR-51>

[placeholder repository] <https://gitlab.com/cardoe/oxerun.git>

[xen-sys crate] <https://crates.io/crates/xen-sys>

1.1 Upstream an "official" rust crate for Xen ([STR-52])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  To start with we will want an upstream location for future work to be
  based upon. The intention is the crate is independent of the version
  of Xen it runs on (above the baseline version chosen). This will
  entail:

  =E2=80=A2 =E2=98=90 agreeing with upstream the name/location for the sour=
ce
  =E2=80=A2 =E2=98=90 documenting the rules for the "stable" hypercall ABI
  =E2=80=A2 =E2=98=90 establish an internal interface to elide between ioct=
l mediated
    and direct hypercalls
  =E2=80=A2 =E2=98=90 ensure the crate is multi-arch and has feature parity=
 for arm64

  As such we expect the implementation to be standalone, i.e. not
  wrapping the existing Xen libraries for mediation. There should be a
  close (1-to-1) mapping between the interfaces in the crate and the
  eventual hypercall made to the hypervisor.

  Estimate: 4w (elapsed likely longer due to discussion)


[STR-52] <https://linaro.atlassian.net/browse/STR-52>


1.2 Basic Hypervisor Interactions hypercalls ([STR-53])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  These are the bare minimum hypercalls implemented as both ioctl and
  direct calls. These allow for a very basic binary to:

  =E2=80=A2 =E2=98=90 console_io - output IO via the Xen console
  =E2=80=A2 =E2=98=90 domctl stub - basic stub for domain control (differen=
t API?)
  =E2=80=A2 =E2=98=90 sysctl stub - basic stub for system control (differen=
t API?)

  The idea would be this provides enough hypercall interface to query
  the list of domains and output their status via the xen console. There
  is an open question about if the domctl and sysctl hypercalls are way
  to go.

  Estimate: 6w


[STR-53] <https://linaro.atlassian.net/browse/STR-53>


1.3 [#10] Access to XenStore service ([STR-54])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  This is a shared configuration storage space accessed via either Unix
  sockets (on dom0) or via the Xenbus. This is used to access
  configuration information for the domain.

  Is this needed for a backend though? Can everything just be passed
  direct on the command line?

  Estimate: 4w


[STR-54] <https://linaro.atlassian.net/browse/STR-54>


1.4 VirtIO support hypercalls ([STR-55])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  These are the hypercalls that need to be implemented to support a
  VirtIO backend. This includes the ability to map another guests memory
  into the current domains address space, register to receive IOREQ
  events when the guest knocks at the doorbell and inject kicks into the
  guest. The hypercalls we need to support would be:

  =E2=80=A2 =E2=98=90 dmop - device model ops (*_ioreq_server, setirq, nr_v=
pus)
  =E2=80=A2 =E2=98=90 foreignmemory - map and unmap guest memory

  The DMOP space is larger than what we need for an IOREQ backend so
  I've based it just on what arch/arm/dm.c exports which is the subset
  introduced for EPAM's virtio work.

  Estimate: 12w


[STR-55] <https://linaro.atlassian.net/browse/STR-55>


2 Xen Hypervisor Support for Stratos ([STR-56])
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=
=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=
=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90

  These tasks include tasks needed to support the various different
  deployments of Stratos components in Xen.


[STR-56] <https://linaro.atlassian.net/browse/STR-56>

2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  Currently the foreign memory mapping support only works for dom0 due
  to reference counting issues. If we are to support backends running in
  their own domains this will need to get fixed.

  Estimate: 8w


[STR-57] <https://linaro.atlassian.net/browse/STR-57>


2.2 Tweaks to tooling to launch VirtIO guests
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80

  There might not be too much to do here. The EPAM work already did
  something similar for their PoC for virtio-block. Essentially we need
  to ensure:
  =E2=80=A2 =E2=98=90 DT bindings are passed to the guest for virtio-mmio d=
evice
    discovery
  =E2=80=A2 =E2=98=90 Our rust backend can be instantiated before the domU =
is launched

  This currently assumes the tools and the backend are running in dom0.

  Estimate: 4w


3 rust-vmm support for Xen VirtIO ([STR-59])
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=
=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=
=90=E2=95=90=E2=95=90

  This encompasses the tasks required to get a vhost-user server up and
  running while interfacing to the Xen hypervisor. This will require the
  xen-sys.rs crate for the actual interface to the hypervisor.

  We need to work out how a Xen configuration option would be passed to
  the various bits of rust-vmm when something is being built.


[STR-59] <https://linaro.atlassian.net/browse/STR-59>

3.1 Make vm-memory Xen aware ([STR-60])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80

  The vm-memory crate is the root crate for abstracting access to the
  guests memory. It currently has multiple configuration builds to
  handle difference between mmap on Windows and Unix. Although mmap
  isn't directly exposed the public interfaces support a mmap like
  interface. We would need to:

  =E2=80=A2 =E2=98=90 work out how to expose foreign memory via the vm-memo=
ry mechanism

  I'm not sure if this just means implementing the GuestMemory trait for
  a GuestMemoryXen or if we need to present a mmap like interface.

  Estimate: 8w


[STR-60] <https://linaro.atlassian.net/browse/STR-60>


3.2 Xen IO notification and IRQ injections ([STR-61])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80

  The KVM world provides for ioeventfd (notifications) and irqfd
  (injection) to signal asynchronously between the guest and the
  backend. As far a I can tell this is currently handled inside the
  various VMMs which assume a KVM backend.

  While the vhost-user slave code doesn't see the
  register_ioevent/register_irqfd events it does deal with EventFDs
  throughout the code. Perhaps the best approach here would be to create
  a IOREQ crate that can create EventFD descriptors which can then be
  passed to the slaves to use for notification and injection.

  Otherwise there might be an argument for a new crate that can
  encapsulate this behaviour for both KVM/ioeventd and Xen/IOREQ setups?

  Estimate: 8w?


[STR-61] <https://linaro.atlassian.net/browse/STR-61>


4 Stratos Demos
=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=
=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90

  These tasks cover the creation of demos that brig together all the
  previous bits of work to demonstrate a new area of capability that has
  been opened up by Stratos work.


4.1 Rust based stubdomain monitor ([STR-62])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80

  This is a basic demo that is a proof of concept for a unikernel style
  backend written in pure Rust. This work would be a useful precursor
  for things such as the RTOS Dom0 on a safety island ([STR-11]) or as a
  carrier for the virtio-scmi backend.

  The monitor program will periodically poll the state of the other
  domains and echo their status to the Xen console.

  Estimate: 4w

#+name: stub-domain-example
#+begin_src ditaa :cmdline -o :file stub_domain_example.png
                      Dom0                      |        DomU       |      =
DomStub=20=20=20
                                                |                   |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                :  /-------------\  :=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                |  |cPNK         |  |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                |  |             |  |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                |  |             |  |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
        /------------------------------------\  |  |   GuestOS   |  |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
        |cPNK                                |  |  |             |  |=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
  EL0   |   Dom0 Userspace (xl tools, QEMU)  |  |  |             |  |  /---=
------------\
        |                                    |  |  |             |  |  |cYE=
L           |
        \------------------------------------/  |  |             |  |  |   =
            |
        +------------------------------------+  |  |             |  |  | Ru=
st Monitor  |
  EL1   |cA1B        Dom0 Kernel             |  |  |             |  |  |   =
            |
        +------------------------------------+  |  \-------------/  |  \---=
------------/
  -------------------------------------------------------------------------=
------=3D------------------
        +------------------------------------------------------------------=
-------------------+
  EL2   |cC02                              Xen Hypervisor                  =
                   |
        +------------------------------------------------------------------=
-------------------+
#+end_src

[STR-62] <https://linaro.atlassian.net/browse/STR-62>

[STR-11] <https://linaro.atlassian.net/browse/STR-11>


4.2 Xen aware vhost-user master ([STR-63])
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80

  Usually the master side of a vhost-user system is embedded directly in
  the VMM itself. However in a Xen deployment their is no overarching
  VMM but a series of utility programs that query the hypervisor
  directly. The Xen tooling is also responsible for setting up any
  support processes that are responsible for emulating HW for the guest.

  The task aims to bridge the gap between Xen's normal HW emulation path
  (ioreq) and VirtIO's userspace device emulation (vhost-user). The
  process would be started with some information on where the
  virtio-mmio address space is and what the slave binary will be. It
  will then:

  =E2=80=A2 map the guest into Dom0 userspace and attach to a MemFD
  =E2=80=A2 register the appropriate memory regions as IOREQ regions with X=
en
  =E2=80=A2 create EventFD channels for the virtio kick notifications (one =
each
    way)
  =E2=80=A2 spawn the vhost-user slave process and mediate the notification=
s and
    kicks between the slave and Xen itself

#+name: xen-vhost-user-master
#+begin_src ditaa :cmdline -o :file xen_vhost_user_master.png

                          Dom0                                            D=
omU=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
                                                          |=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
  +-------------------+            +-------------------+  |
  |                   |----------->|                   |  |
  |    vhost-user     | vhost-user |    vhost-user     |  :  /-------------=
-----------------------\
  |      slave        |  protocol  |      master       |  |  |             =
                       |
  |    (existing)     |<-----------|      (rust)       |  |  |             =
                       |
  +-------------------+            +-------------------+  |  |             =
                       |
           ^                           ^   |       ^      |  |             =
Guest Userspace        |
           |                           |   |       |      |  |             =
                       |
           |                           |   | IOREQ |      |  |             =
                       |=20=20=20=20=20=20=20
           |                           |   |       |      |  |             =
                       |=20=20=20=20=20=20=20
           v                           v   V       |      |  \-------------=
-----------------------/=20=20=20=20=20=20=20
   +---------------------------------------------------+  |  +-------------=
-----------------------+
   |       ^                           ^   | ioctl ^   |  |  |             =
                       |
   |       |   iofd/irqfd eventFD      |   |       |   |  |  |             =
 Guest Kernel          |
   |       +---------------------------+   |       |   |  |  | +-----------=
--+                    |
   |                                       |       |   |  |  | | virtio-dev=
  |                    |
   |                       Host Kernel     V       |   |  |  | +-----------=
--+                    |
   +---------------------------------------------------+  |  +-------------=
-----------------------+
                                           |       ^      |      |         =
^=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20
                                           | hyper |             |         =
|=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20
      ----------------------=3D------------- | -=3D--- | ----=3D------ | --=
---=3D- | --------=3D------------------=20=20
                                           |  call |        Trap |         =
| IRQ=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20
                                           V       |             V         =
|=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20
            +--------------------------------------------------------------=
-----------------------+=20=20=20=20=20=20=20
            |                              |       ^             |         =
^                      |=20=20=20=20=20=20=20
            |                              |       +-------------+         =
|                      |=20=20=20=20=20=20=20
      EL2   |      Xen Hypervisor          |                               =
|                      |=20=20=20=20=20=20=20
            |                              +-------------------------------=
+                      |=20=20=20=20=20=20=20
            |                                                              =
                       |=20=20=20=20=20=20=20
            +--------------------------------------------------------------=
-----------------------+=20=20=20=20=20=20=20

#+end_src

[STR-63] <https://linaro.atlassian.net/browse/STR-63>

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:50:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:50:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195534.348282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToOz-0002H8-Pi; Fri, 24 Sep 2021 16:50:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195534.348282; Fri, 24 Sep 2021 16:50:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToOz-0002H1-Mh; Fri, 24 Sep 2021 16:50:21 +0000
Received: by outflank-mailman (input) for mailman id 195534;
 Fri, 24 Sep 2021 16:50:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mToOy-0002Gv-R3
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:50:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7ff7cb78-1d57-11ec-baff-12813bfff9fa;
 Fri, 24 Sep 2021 16:50:19 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2054.outbound.protection.outlook.com [104.47.4.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-4-PHeMdsOjNle11dol2LZ0og-1;
 Fri, 24 Sep 2021 18:50:17 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB7089.eurprd04.prod.outlook.com (2603:10a6:208:1a1::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 16:50:15 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 16:50:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ff7cb78-1d57-11ec-baff-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632502218;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=9A0+pnQRcAewGwpyyb5SKYkn/HCvTCcn0tztU1NU3PA=;
	b=Ln++xIen5dfyKUVnHXR7OMHQbeML6uknPiwbZpQoq8X5dmEGIttGhE3IgWzyLS613NP+dH
	4tRBRgvWcRh3/GX1W3WhosrO8rHUn6rnLmGJntNcrOt5Zj38KqZ58WPionV56/APU9Uueg
	EvaPTeN+muu3jf2akPHU0NAnjApEAQQ=
X-MC-Unique: PHeMdsOjNle11dol2LZ0og-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jSuHP3GBaE1fg98lgJURIpB8431AkMQutWepoB4Lb3PS1R145eSRnkRSZeCO9xFEW33zxNA1chdSuCY8dKYpxBZGgokIeKQh/dTYwZTeZq+WMi8kZU2ijjAilXuz/i/SRraBPtqH4LlM++BvaKzkqUlheWgW2lrj3z0XNlFKUZnw0ViZjbyhvXofM+Tphdj1Zep7YPolfcWuDHLqMR9Tx9igtX1QP01xc4VpPfKip1W2qvMW5VcN0YE3Egg5As6ohvbVbSNY4h11uvApTjNiI6FKo8loI3/E/Bn4U2qYT7N6WtIqysEkuxCGT8+91verBOv4hZveG6Sv0B0tG9cR/A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9A0+pnQRcAewGwpyyb5SKYkn/HCvTCcn0tztU1NU3PA=;
 b=O3BK7iBvsZoodbctepiqUigwSp5FsY9d8nyy97qpIt/TTeDG5acDEccqrWigEUU6gN6vXrdODYhKwdRb8029bPhFpdJCDWMhGdRUVYnyqbqGHs8zHf2+L/zibQOcUg413h5eYfVOHqCrIZYQiytvYLe7VYakIVBhtp+dWzboog0AbyhNiS04qCJiAUcFbKvmy+ZCMgZ5rkpbqvH/7T6Cc7l6yUcprCK31NVTMy4zh1vpZu6A33B/LVwnuJozqAfCA4GBVMU4A7yS0XLmFcbFgX6buF3MRBr/eqm6k9pno0yg3I93AZOEjAX2SY/OLj6P5TY/sgGMD7K1+axoQQA+sA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>
Subject: Re: [PATCH v2 02/12] xen/memory: Remove tail padding from TRC_MEM_*
 records
Thread-Topic: [PATCH v2 02/12] xen/memory: Remove tail padding from TRC_MEM_*
 records
Thread-Index: AQHXrkSKbARHP/AzFEumZlsoMq5Uf6uza/QA
Date: Fri, 24 Sep 2021 16:50:15 +0000
Message-ID: <dbecc633dda9f0efb4b458a204affad44dfaa177.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-3-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-3-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 70ebaf77-b529-47eb-5d96-08d97f7b6223
x-ms-traffictypediagnostic: AM0PR04MB7089:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB70895722889E15F3D62F66ECC5A49@AM0PR04MB7089.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 nLDBB+DKJJOLo76yhW+EheCK6Fw94AkdsdNipiHPIl0vijyu1Urh4FMceNOTnxR1eK6v5P+Gp5xS57WDgbQ73LZAnae9Fb7aeVANJsSiI3KrnYfoTYvXimYAB+J4rlOCtN4SqoEOuzraVeg0nYWvFQfWzsH3qr6D9r573xioqOeS0mLidvz1l9CFxq2NNFc5nE+wmm2lU+V2RjrE3a5fxwqhRmKBLsgIqas5Ty/4yGkLBIPHTqqkwxBNwzqRF3sytKitiskb0FR2z+UkkDWVobWhQkMm97hv83f0sNBgRoU1jSmHJr6iwuABGSfvemXrqzrqk5XqM4XnSkhptJ8GO+TrfaMn2DPPbn9KdnwoaiUBWrhsp6y5nIm9v2gOsHnwmUVjmOp7GibJn1mq/NPOy/+ZCytubfRp6gqI4d/YQA3g297oFnyQPeDO9K95Isi61HXgpxYrCTYZMqzYfwK0ZF0DVv7DzL7fDSlxkhnc9CEQQxzaCKaOTDRlVWUh+iTiAz+wJBzVXEmL3x/WJfFiBZ5YplnB0Zqptfl9t2Utfomfc63zbNB70dJofDnc4OhloDDf9RmzCPMawR9W4ssMMGQBY1JCJpnlRDqZPkye/rTfOjpsH4K/PEETpA+G6gzkO5KWfPTWAFLQLgYxBpI2GEOUiaCe/b3GQfzMWdWJEFnQtwAX9WiDYv9yLPyssNVQptlsvUZ4xY65mtG9CafGvHp6d8vwH+PEI+qF8CN+M3J4YOIy9tNkUoIqox+FMQV6XIsjgtazfCx3pHOiEiScxYsV+PNDMNLQZJEXu5L5RhYP+VuiEvsInoDm29aP41TB4KjjmDrw6a0CVwFOrZJRjw==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(6512007)(5660300002)(99936003)(86362001)(110136005)(54906003)(508600001)(2616005)(8936002)(71200400001)(4326008)(76116006)(91956017)(66446008)(26005)(38070700005)(64756008)(66476007)(6486002)(6506007)(66556008)(66616009)(186003)(66946007)(966005)(83380400001)(2906002)(122000001)(36756003)(8676002)(38100700002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?VEl5QndLNGRVQ0JyOU9mOGpqbXRPWkowcThqMEVqdTB0Skc0MGc5YTdCN2FK?=
 =?utf-8?B?WEJJSkRnY3U4dE1ZejRWQWFOMGYySmZQYWo4cGozMU1vbHJBREdpZXJ6ZDR6?=
 =?utf-8?B?Wmh0RVJkN014dmpQdFUxZ28xcUFHbzRqNkx1L0VZUVZPM0V6NUtUc1BrdUdM?=
 =?utf-8?B?VUZFNXl4aVVwazg1R0E0eDlQS29EandTQWd6ZDgyS1lOM2Qrc1JzMHpiS3Qx?=
 =?utf-8?B?bVdMMzQ3ejFKQ2xNdEZRNHZlNEtIeDNDMWxNNURXNkdJVkEyYVYvL1VWbTY1?=
 =?utf-8?B?d0lqbHJZT1NNdzBMbm1oSFAyNUQ3aHdYazV0YURzdHYzeFZpTERIQ0FHcEt4?=
 =?utf-8?B?dENPM1ZtSjYyV3ljaTBFckFyNTFtTGFUalRzUE8rWGRQRXZaMmVCVllNMmIr?=
 =?utf-8?B?MmpnNkNVa0dCYWtqelAxSFVwNmJnU2x3MUFZT1NjM3BhT1NLdWtmbzE3NDdC?=
 =?utf-8?B?Qm5FNkRJbVdnQU5tZ2tqM20xcmRybll2VHlEZXAxZzF5aTZaMmk4dlFENnIz?=
 =?utf-8?B?OHhxTWVzeHJFbFBvNWpSQXJmUWhpQmk2WHIxNytkT1hvNWhqVmxTcUtqMkVQ?=
 =?utf-8?B?ejk5cU1hSjBVODNORjQ2TEJDYVA1V3VjeHVibEIrWEl3M2dCOXdhSkFIR0V1?=
 =?utf-8?B?SmdjWVdEY3FzZE5rWW14OGxBalVibmh5VkJxNGRIOWlJdHhXcjdKcDREaXIv?=
 =?utf-8?B?TWVUK1Q3UTltOHhnQW5BUWI2ZTQ1Ym9xaytta3JHUmVBOERkNWU0VWRTWkNF?=
 =?utf-8?B?V3ZlTUdMeWs2QlpLM29NelBNRXVjUHpHZElyWGtIeFRkWTFsZ2NGZTYxeVpt?=
 =?utf-8?B?QkFBNFZYMGRCTkxwRG9naHJEQVRDTW9ZdmhrNDAyUmFhdUVFN1orT3dPUDlL?=
 =?utf-8?B?SmVvempSQk4rMmQxaXkzcEhGVjMzOEF5UFUyeDdlWEIwR0FNU1U5dDR0ajhs?=
 =?utf-8?B?UHBqNDRGeFFpWHRBZnB1UTVjUGdIUWRXajI4OUhGVVpKQ0JldkxwWTdHZzRq?=
 =?utf-8?B?cUtoNDkrSkVzcU9UMHJ6L1Y0MEpySjVWZCtFLzVvc1I2SkZCRU8zQ01uOW84?=
 =?utf-8?B?K214WExzZGZyZ0FOcmwyTCtlM2FuRGlTU0tqRy9qeDJFTjlEWXZ0dUdyajVR?=
 =?utf-8?B?TGIrMVA5QzcxeUxxQTRmdlIrNXhheEJSY1pjMCtUbDBxU0RiTUtPUlF4S2s0?=
 =?utf-8?B?ZnRHUjdkN2o2M0UvNkNWZmMwWTJBcjhaMm0xWHhhVW15YXFpbFNyM2JMc2hz?=
 =?utf-8?B?NlpWdDhaVWNJVEVxc3ZuMVpELzNSbmpldDJVcmRtL2ZzTkFFR1RSdjMzcnJ4?=
 =?utf-8?B?Y3pOd3hueEpCdFAvL3dNNzRUbkpGV2hSQnMvckMyVE1pY0tlWW56OHJJYnQz?=
 =?utf-8?B?ZHIxenJUTTU0MThDSExkdzZXSFF1Q1A0YnlvcTIvYUNIci9SZ0xrVDZGUTVF?=
 =?utf-8?B?TSs1WUgzQzhyamFCQUhHcTdFc29Qb1RsZWUvcU04OFJENjBmVStUYXVscmVQ?=
 =?utf-8?B?NFplaVUrUGRHRTZwdHNPYlVpVk1VdDhQaXNiWFlBQmNNVzRtaFFYRWZZODlp?=
 =?utf-8?B?ZS9JYThIbGFkM0w0VkxrdXMrQ042T1NYeTQxNERQVEtzQzE5TVJvZDJmanNz?=
 =?utf-8?B?YVo0emlSandtUUtFZjJPQWpOeFZkc0QyTS9jUUpqZHhUSTEyVDYvU2ZaallP?=
 =?utf-8?B?Vjd5eThIejR3ZEhORXE0Q3dtUGE0c3NhNVVsS1FCRlVIOWFueUVicFRrdm1a?=
 =?utf-8?Q?yt7hNTNDeh/hXDL063+NOPyZe9QUKkII0pM2M6g?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-yreP9YeHWsEKtcDML9fj"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 70ebaf77-b529-47eb-5d96-08d97f7b6223
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 16:50:15.3036
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: M1AmenVmVXD6jXNcX9LLzfEklWUUaIqxtXyoB2kGCcWOBsOCeenvslKFy0Mhr7jHsJMqv7QIIQGLf5SCuEJ8uw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB7089

--=-yreP9YeHWsEKtcDML9fj
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> Four TRC_MEM_* records supply custom structures with tail padding,
> leaking
> stack rubble into the trace buffer.=C2=A0 Three of the records were fine
> in 32-bit
> builds of Xen, due to the relaxed alignment of 64-bit integers, but
> POD_SUPERPAGE_SPLITER was broken right from the outset.
>=20
> We could pack the datastructures to remove the padding, but
> xentrace_format
> has no way of rendering the upper half of a 16-bit field.=C2=A0 Instead,
> expand all
> 16-bit fields to 32-bit.
>=20
> For POD_SUPERPAGE_SPLINTER, introduce an order field as it is
> relevant
> information, and to match DECREASE_RESERVATION, and so it doesn't
> require a
> __packed attribute to drop tail padding.
>=20
> Update xenalyze's structures to match, and introduce xentrace_format
> rendering
> which was absent previously.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-yreP9YeHWsEKtcDML9fj
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFOAcYACgkQFkJ4iaW4
c+79zw//TlLQ7V82XrFwJwJOqVTscw0tkTv07UzpF0CB27W8Jtwjxz9ZonCG67zp
h9ZDnuCtutmLZ0U/NSa1HShDgthj2H+wZbICKMhseE/MTngCai8pzuVT+dg4dPVX
Bbvqpy7ZiSKxXUxF9mX5G/XOK2b55ofbD94g0Z57ZeSeb1UR42crLMhGFOdzgLWa
LHTN5eB9Gu9p+SlM40QlKqFylnUYuYiG9u9ap0gNqpnKbwP+qGdp0lWp9VfxkkIG
gKgNUKAWMCA2u69EjToX4vBeFaNRbRKg+FtYIBXBiA+FkQ0Bj27sdMAnCJdXPJAJ
PTBhbF1FZBCNiiBuWHVtx4zJGIc5jinMFxbsd8imR2LL/esR5ipH8tgz1Ej2zXI6
WJp6UJRl6aExU+7WFDZSy7tZVxKxQ7kWAf+erbj6Wf0KE6qHxbOrurVdCJ5hPisR
BUwyfSjXLALj82CpxarEd6EMiCL+yx/HpXM7wVme4FZPSwCH1DMj9N8pet0Puva6
N+qu8L6pfBj5gUJkDNM8/k7nGjTHF6nF6Rm8HDBcVXMlr5YMMlDSuZoiACzjzVsX
iJouM+PzR4cY4fMyJr9v0RRYzYdvmZljjobSz5G99KgaDszTf8WeGY7tDB1cz8iH
gkWhMX2+nB9FMBqW1EeP7ufXFoGwx+ZUxXuD6xYByPICbALrVaw=
=2bPf
-----END PGP SIGNATURE-----

--=-yreP9YeHWsEKtcDML9fj--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:54:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:54:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195542.348294 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToTG-0002zX-GX; Fri, 24 Sep 2021 16:54:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195542.348294; Fri, 24 Sep 2021 16:54:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToTG-0002zQ-CB; Fri, 24 Sep 2021 16:54:46 +0000
Received: by outflank-mailman (input) for mailman id 195542;
 Fri, 24 Sep 2021 16:54:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mToTE-0002zK-LT
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:54:44 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1d643266-1d58-11ec-baff-12813bfff9fa;
 Fri, 24 Sep 2021 16:54:43 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2051.outbound.protection.outlook.com [104.47.4.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-34-xTvMKznCMKCnZGEA3gj4_g-1; Fri, 24 Sep 2021 18:54:41 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB7089.eurprd04.prod.outlook.com (2603:10a6:208:1a1::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Fri, 24 Sep
 2021 16:54:40 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 16:54:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d643266-1d58-11ec-baff-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632502482;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=WcRgIvLdOeeOVAbGhqlrNrO/FmyiaDsbtnAlH9WiAH4=;
	b=biZ3QFnMqnpN+BLuoMyf2QZbjsrci5InG1XDQtORHNj2WYvRCxUsKmbXJqA90KhdUjphlB
	t6i9i8d90Yzx5JFHfz8q4+eWXC2RxHJHoh0idPEDzUsiOsiKaJahyxbJfMw+D1G9hK6OlE
	z7YzhRg33r2lipscnj2o6+zS1SvqEAA=
X-MC-Unique: xTvMKznCMKCnZGEA3gj4_g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mmXbb742Rk3c6nDITJMth10BMvdB4Q+V00WxdSjTtv811IfCJLk9E77H57P0Wr5+N+A25KA7nvAnhqdQkgGLGYLiLGRkUGGydBT3xGA+beALRSFit7kXwCrY5z3v3Ol02LCosyKqTsXNja+C5wHi66u8ulTQfebkeycUN8o3gBb23v2ZMxl8G2i8Q/RSJDtKrM9amIfOFyXMPoZiHPKnSsi/k1KT3dFGAuUdXMxheyKlPEx4ie4IkymxRfz8MN65KNt6J2ufHRMaTq6u4RZ5FZHFFOJG2p9L/KISPtqGBJdciaWmEaUmun8dhIpGP4gRLyCM6npP68Xm2Nc73TL9wQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WcRgIvLdOeeOVAbGhqlrNrO/FmyiaDsbtnAlH9WiAH4=;
 b=KqHCRld9RQtzuN/GwmU/2PHKMCHc/4rmvlg9ZcD10rzDbs/uJcjTJHr0/ssdPvcKwBYkusTMhOgKw3rTO4/XBrGWwK2NUfwJKMStycpnqVCZDFAADENvxN5d1d5SZTfXggOR5HISvjcSFpkRswz4NvLTl0z4D57EIMx4dsmdITu/oQzQDjUwy20tabq5UJ1pRpg2VZYoCJ+5bRG+3/cOrmqmpBlxWo1OLWtCcy59trUcursVg7IYXiKthddcenybM0bvqdFx/XMlUjGAckH7hbeGVPgYbmgnAa86kUeTmn8aYcuoWqgbQr2qWmDILz/5bS8UjDJG2z22q/vCVEK7FQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>
Subject: Re: [PATCH v2 03/12] xen/credit2: Remove tail padding from
 TRC_CSCHED2_* records
Thread-Topic: [PATCH v2 03/12] xen/credit2: Remove tail padding from
 TRC_CSCHED2_* records
Thread-Index: AQHXrkSMLCUBSY8ld06BUZqwnuzywKuzbTCA
Date: Fri, 24 Sep 2021 16:54:40 +0000
Message-ID: <8ad0beb748c2fb5b6e232d725ec6cf7ff7f945f2.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-4-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-4-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 93702dd1-f762-4744-3531-08d97f7c0043
x-ms-traffictypediagnostic: AM0PR04MB7089:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB7089B3D6806613CB05865E43C5A49@AM0PR04MB7089.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3513;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 w9Go1MvWPi2EFVuF2CmniUI116hNYqozhLI5E0uGBmBVx2RDk5feuwqVmQqtfC467IZZf6jGFEGvXEjR6XOO80d/f5bqEFJdZYH2KQWnQ6ue4w2stMxsyva7tBmFvs9/RZK9Q/eNhz4U8wnfkjpNSE+PKCk9GqElQKr+Oa1toZ42s7tdlxjn2xW5yEyX0w6yQ7ZgxqMv/WM9PIQqdlknzGamX+AP6/f4ghUlmp1BOmSJRvUK33bGFy/wvpMtG361gtTFKax+6CG1q6uQ4iAcjjPDYIonKuzPVcV9edVI41Tm3Z4jewerH7sq2ADk//usRwTC4wlooeBC/lJrYtFe3/GM/Pd5g95bWelKqeouDT0ZDc/ZlvUJJ4JAhBEcePRPwbRlHTg6hp0q2OcLCLhJV/ArVPu2oy/zPFWvJNt+XnyqU1jnI/5fQqDmHpHRhOG5sDwmGfAB8FRFLDzMfpjbnAxsuog95qUfP+gc49dB2u5sXy0+sVs8t6UDvs5TUcfNYBiwOx9Aoz6LwxYSYkmQz2mVCWqq3w4NWmBbWdGDmaopjuaNDVKkPcsI6gkRUl7uqN7N/R6sKJ8fdy1vVMmAE21wx1ApjYYXkmviqZOm9YOt2fLgJqGY4orbMkcETwwWJL4GCAOA4+BVewaaBH8uk7Vkp1Ut/f0qfTB4bgkKGfSp7CM9SDCP7nTZecGnBi2ln4J1BxGBcxkBfWg08SGo1jlb+k24Tf+bkpL4xbvxh3QKL7HR1dALek2m/7cxoT3/HQ8mWcUlgZPx49DXK6lVHVbwvMrCp0bLuDGxtNIn6mL8NLBdhtygqnrNq5ZrLyUF7i1zh5kTOFI11hS7hX6LoA==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(6512007)(5660300002)(99936003)(86362001)(110136005)(54906003)(508600001)(2616005)(8936002)(71200400001)(4326008)(4744005)(76116006)(91956017)(66446008)(26005)(38070700005)(64756008)(66476007)(6486002)(6506007)(66556008)(66616009)(186003)(66946007)(966005)(2906002)(122000001)(36756003)(8676002)(38100700002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?djEwVzdsc2tPaSs4S1hQN2FZRnM0RWZTV3RkdWRURGhrQm95SzdXQnVNTVBZ?=
 =?utf-8?B?aHJPbGJrZnVSRFg3cHJDVktYMjI3L25LdkEwQ2x1Q29rTDI0R0ZDOWVJdHF6?=
 =?utf-8?B?L3hzWFBRdXlackNHZVZJa3lWOEZGVzkyeGNxdnNmcjBqdnVIUDZEcmxVRWFT?=
 =?utf-8?B?S2pPZ1IvVjdiZUJHc0loL2drNkhUTUM3ZVFhZHZXaWU3NU43Yk1wVC9CTUhs?=
 =?utf-8?B?allyMEtDcDU3WmhEenBRR3lvRFk1ZGJlQWY2T2VnREFjSXdMMFVITDM4ZzBn?=
 =?utf-8?B?OXBQWTZBRUZjeG1raWpIVFFxbVpGT0NIWjFTQWpsNWo5L3cwV1RjQlN2SGFC?=
 =?utf-8?B?Vnd1eDNYNkl3U2xqZHZOTWxMdWp6ZUxkK1YzS2E2R0R4M21RT0MrRzhZb2ZG?=
 =?utf-8?B?T3JNYlpaQW40VVlIVm5GOS94Yzd1Q0xISnNTd2E1ZU9sb0pjejE2UjlKclZZ?=
 =?utf-8?B?SC84SCtLaWFXVExaQnhrMFNMcUY4bzQ2bXlGN3V3Ty9ubmJLTzlTdCtDQ2Vp?=
 =?utf-8?B?T05LbDJRTnpTY0E0dTAwYjZXNFYxVTZpYUZwNmpWaCtaWTNVWXl4M0JQMzll?=
 =?utf-8?B?UHZCWUIwTmt6Z3Q1VzNZWjJhMUJtcWtyN0pmYllhSExIVjFUVjl1ZVB5eU9k?=
 =?utf-8?B?ajk1WlduYXJPU1dkS3JjV0ZXYUVKczhoTmszN0tuZVFxQjY5OU5WNjlnQ2pU?=
 =?utf-8?B?YTJOZmxxbVhrMVJMVzczU2JqZU4yTE1lS1FLUDB4bnZvem5OY3cxRTBreHZO?=
 =?utf-8?B?UUNEenNuUmpVRkRwSTg2SEp2elJweFRUQ040TmxkTlRKdlo2K1B3OGlqNTg1?=
 =?utf-8?B?RXo0YXhJdmpFL1BxTWhVUmVpZXg2eTNBcVMrZjJzMGIxdjRnSHVzSjRIUW9L?=
 =?utf-8?B?bzJKNzl3U21CY1ovYnAxVzNnNlE2aG91QmlPdDNVTjdFaHJpMnNoSTI3TFNX?=
 =?utf-8?B?OUZaN0IrWG5ibFZRRTJUM0Rpckg0T28vaUpZZVN0L2ZvdGxPZm1GZHdCSEhO?=
 =?utf-8?B?Mjhscjg5NC9sZjBnTTJyVVQxZGRmTUJFOE5HbFVkQnREbDJpbHJsaDZKU2JU?=
 =?utf-8?B?dnhkRk90U0h4VkRKcHhGeGg1UlVybkpZdG04OUdvVm96M0ttRlpPTDlqQjhP?=
 =?utf-8?B?L0UyMHN0bW5hNTlEWWdsVnFnR0o1YkpvR3Evc1dUNzU2S2VJQlh5T0d1RmdZ?=
 =?utf-8?B?WGVVaUI0ZkFrUUpCM0FUSG9YQjFaWUUrNzNvWXd0Yks1SDQxZS84QXROSTBC?=
 =?utf-8?B?WUN6eDJjVEowbHhuWDZGdFFQWUQza0QwN2ZKdjZYWjM5VjZrZ01xS2Mvak1q?=
 =?utf-8?B?dUt4eHlJanhRWmNrTnd6OUs2N2lyY1gwa3JxQ3k3bDZkekdBZjZMcDdnLzli?=
 =?utf-8?B?K094dFhOYnBUQTgvMjNGNTh2cXk1TWplaXBIOCtsVEMyekdlamd5Y3dGTW5N?=
 =?utf-8?B?WVVZZHh2Q2xxbW1rR0VISlBjZ2Z6VGhPekk1dXFMcVVrMXA4NG4zdS9xZFpS?=
 =?utf-8?B?QlpqRG0wOVVyQk96RnZEcXk1cldNWFpBVVdRTlRIaVU3ZVBpb0tWcGt1d2Iz?=
 =?utf-8?B?K21CRWt2RTBhUFZCVm0wUmlyZFAxcVkrdWJUaXA1UlFDWlJIYjhFT1Mwcjli?=
 =?utf-8?B?Uk5uZFVRSjB1cm5jY0ZoUzB3dE8yNlFHMW8vU0gwdGxHdTgyNy9mQ3o0MGVT?=
 =?utf-8?B?cndUdVhta21WcjIwcldCc3hSanZxNkM1YnVFSVMyVm53NXZQaEwrZlFkOTZH?=
 =?utf-8?Q?JA2spt/FVtpkNJ2rUtms8YuOjxdZtIdVtfSMdsN?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-UE3ECcQOdSMTkwtNCqBC"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 93702dd1-f762-4744-3531-08d97f7c0043
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 16:54:40.5843
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fTY/GwZSbUYWic4W/mmS6zwAeBgIgJGnPpVCLiozMUe68REXBrrPGKDce6pUteQZSWBTRfdA+LSCGqMtqwHDjA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB7089

--=-UE3ECcQOdSMTkwtNCqBC
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> All three of these records have tail padding, leaking stack rubble
> into the
> trace buffer.=C2=A0 Introduce an explicit _pad field and have the compile=
r
> zero the
> padding automatically.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-UE3ECcQOdSMTkwtNCqBC
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFOAtAACgkQFkJ4iaW4
c+77zw/8CJBHAQPQgIBeRjIlRDHfjhSA3bDGpEyq/UuXRDLRWhdLgt2Y/zB+9nID
uno7vmIokm2ZnIZAIb1D9eYS9W/SbVtPCa6+bpoDKUBchnqw7j5aFJARYXlpZOwl
/yONPn10bgK3weDNIAL0jde+NUrPLUL8rF5jc6ouMv3qxR6tdazkq3mJBAHQOwby
eWBvQyiZaIvv8VdOfE+fjp18KhYeXT4heQAVDyaHEAlfEnOUb7T2CuBnMjIvSik+
5S/8L5K8irW0vJD/eTt9Tq2y2IVtzwXJBHJQG42o5nSCcvmc4VViKE4Wo4guGl+P
94ZHZWh1UNvqVZMXdYCjTx4iAs55R7+PXTYYBvUVB+PS29aKMaiaYc2rZMxnI90Y
JxpLpKG9Yu9jX0dlhZzWUZmxkYKJifj9YffIlLLfW5sPyMEarfAoB3PpjRoiXsmw
BhnhSGWkzVuf2A8XfEqo7HnIJ/cqmjHwpRa5n5/myljI8oMTkBpQNdvFi8uj5R/T
76bl1BNGV5glxdZxcnTena+JPpuBjSddlbb1EZyDNo/EcKqMRaCg1RJGEUVnQCNS
jzOjrnpBQa13XS/kIHGdDMY431k2XlVAac0v20v2wWTbdY9z47b6+OKewSOhQoEc
ULc84joCCat5xR+9ORy8/7ZSqh+r5UGbVkC1ZK32Qe99ctSU690=
=+oz5
-----END PGP SIGNATURE-----

--=-UE3ECcQOdSMTkwtNCqBC--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:55:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:55:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195548.348305 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToUA-0003Z2-Pu; Fri, 24 Sep 2021 16:55:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195548.348305; Fri, 24 Sep 2021 16:55:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToUA-0003Yv-Mm; Fri, 24 Sep 2021 16:55:42 +0000
Received: by outflank-mailman (input) for mailman id 195548;
 Fri, 24 Sep 2021 16:55:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mToUA-0003Yn-9T
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:55:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 41227806-ac2e-4c80-a937-c24d58ad0643;
 Fri, 24 Sep 2021 16:55:40 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2053.outbound.protection.outlook.com [104.47.5.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-iXIvZgvxNbKbOhCFbRyftA-1; Fri, 24 Sep 2021 18:55:38 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB4852.eurprd04.prod.outlook.com (2603:10a6:208:c4::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 16:55:36 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 16:55:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 41227806-ac2e-4c80-a937-c24d58ad0643
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632502539;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=TFPkAcyRbiMboSMP0wOapdLx/LTk8YFY2obi434qYKI=;
	b=Zeie5XjYuidQPAK3HAU/jqcewstJOu+t1MK9EOp0xIru1sViv3LA2Dm+zonGH+g7gVqe2g
	Wc99ILHHaO6QNJj2eSJVU9S5at70tEvVFwDNYJOb9ocD/Q4zYun/8nDl1N2uUQJxY560sJ
	aEaNkaLZ0LAkhGJcR5nX5gBUffN0Qa4=
X-MC-Unique: iXIvZgvxNbKbOhCFbRyftA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=a6RAaO7ZEgoxo0hl08BF15vCYjXN5IvFlVLewCUpp42JDynfwuYiUx67H/Y8BESWvS3brMF22iSVX6gUoKdB+SZ9irJTYZvs0ZR2yC2h0yt82L5mKsLC69kaOAhekUTM9e1n3xDzUSZecJXzkMD7JAK2WzkV2FFKTfHGaGblM2eMuNUDFcXCZ4SFyRmDf/DT95kfk0SX8njFspREcm0gsISZKxTqs119OFO7JC8MCTDKBLVXhK8t8Splzkr1FfLSc4WhqN6Cu6y9/iR+/Ch44AwXWHmpjnSHGGgTfm1eNomvMwU2wVmAQhb7kb3q1bIimP6pBIzTs4mjkXa5JWn1TA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TFPkAcyRbiMboSMP0wOapdLx/LTk8YFY2obi434qYKI=;
 b=hDqNG6LYWCYPPI4nW2niD+DZJoU9x1NckaxFNDpF+Bf3E2bHRmxvr+CA7vT5vP2HVcGK8rYSYnUmgJdnGCsovbk7ungov/k1RwhKw5fEMKxLc1CW2pMrfgaFB3wZZt7Le/wZwYyHELKSywIm0GvVj0i1GHAoc5/uDi3sCrkYav82j46czAZD9y8qG8ClIc1y8oZcZ/cfAGttFqXrQnV7W0oPgHRmz54J2rNTgrtROPlfiSi4q9V/9FEDWcIDdOFNq9YZ/OteEYEWPCHR7ZE6w8NlS/AzeAUPa+BdPQzo7U0g87PyxyMusY44GsJsDjowCdy4F6vnyaYc70H1W/rNKA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>
Subject: Re: [PATCH v2 06/12] xen/credit2: Clean up trace handling
Thread-Topic: [PATCH v2 06/12] xen/credit2: Clean up trace handling
Thread-Index: AQHXrkSN6XYb+hOIeEOo8+4FDChVo6uzbXOA
Date: Fri, 24 Sep 2021 16:55:36 +0000
Message-ID: <c35eb0695c5d10fb7883424442b09da5497db87e.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-7-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-7-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 82f146b9-f018-49bb-bc82-08d97f7c21ac
x-ms-traffictypediagnostic: AM0PR04MB4852:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB485203ADE737ADCDCF07028CC5A49@AM0PR04MB4852.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5516;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 aFrAfKMvsQ2wFy0hlgCXki/6bnsn3HANvmapJexAuUsNZ+H+GNpx9nWjlTdLx8XLz8oFcFjIn0VbmTTHdCHwbLdHnVQu+OSBftUwaKMHzc9Jfu4xOrALlln8MrSIwaiQQf4i/d9EbocDj17IxhOFXnyaFFAlK6HtkRaGa6WkJQOl4avU9kHnRRsPK+qEskD6G3i7DBpp+58MrrE2CbXMa+JZgoI10+nREgPWW1El8EJOJIz5LXVh3plQlyQkdJZyyEhosz6i7jj/06fzGJSxqOCyP9Jb7ImbRpt6JKi6Rj4ykR7xx+6VnyJ/4FVKJUDfweoR68gVj7axstDWH0BFIonOiSDFCr9Lut6qnqDqKbhvGhjT7hDOxrSKRJuGnt2ap7KdkWmFa5XuL/TPzkNV5NJWkSJ8i8us3xhAWMFX2Lx4YBeQjsxvhIDgFpxwOjjfiEQb9DgJzkcfZf5vYb874gqBCWocJaUi4CmXR5tQ/O30awjd29g2E4agFcWL7znUpQn8C7LsSlweWpI01pAxezb2bjFcPFPSmSJXw1J8kfwOQdvS+w+Ti0n+3uOo7KTbMuaHLMsFbRVfy7Box4UQZ1WBEBMGyMv0dkUUiZtgGpNla1BltYE0jveYKlS5I0hbzQ/qRLmgLQnzoFEF2XHH4lHc/bIoEo6mPpgNCQvtPMhpqJ176SCn5u8CDO536BNI22AvNpCHLPRbP2d+nr5tecR1yc5B7c78Kce4Du870gn05Tn66cI+u3mzFKAGecsYXIJnYzQIRftRWmqLVhBOTF6fDekteJBYbSkTqf4rLdQ7Gu/BgyPWiDVqiRRm+pY1lggJn5RiSnmYWcOpy5VR5A==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(966005)(91956017)(5660300002)(38100700002)(66946007)(316002)(508600001)(2906002)(110136005)(6512007)(8936002)(66476007)(86362001)(76116006)(2616005)(99936003)(66446008)(36756003)(66556008)(6506007)(71200400001)(186003)(38070700005)(4744005)(26005)(6486002)(4326008)(64756008)(122000001)(8676002)(66616009)(54906003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?S2Q2UVZFMVRtZ3NWRlczMXgvU296VmJlN2E0Z2UyZmhDbzVGQ2FIOXN2d3Jh?=
 =?utf-8?B?b0pLYUVWZlQ2dkxNWFIxcGZKSjVHMjloVDBqNFZ1V2ZIT1hObmdpNnlPTTN4?=
 =?utf-8?B?eFQySDlrV1pid3Q5WnBuYWpzU3ovdVN0MWM4S3VGcThGbmJNTDBQQ0RIZ3h3?=
 =?utf-8?B?czF6Tnl5ZkNCdWpPWERsZkxvSFQwZ1ZkMWdBakhONzdUbFluQU53WTRqd2tY?=
 =?utf-8?B?QmY2VisyeC82bjIwa1REZWVNK0RRdFNMbk5uTVNLYkNTM2czcmJDdlYxWHZa?=
 =?utf-8?B?dTVEdE42M1BLMUVrOEJXYlpNZlNEN3NLYzlINWtWVVN3dHhEZ2ZSNjdrQkkx?=
 =?utf-8?B?WlRZOHhZSXIzQXEzajdoK3A1LzlsVG5qK1ZCUVRZd09xeEdMczZyUkZGbmNp?=
 =?utf-8?B?UXZmaGVYdlVsZC8vMEtFRlZ0UG9iRUJXQm90djd5bzV6Uzg2SU5JSzkzTjht?=
 =?utf-8?B?TTZkRTd0UlVGekdkYnZCSFJCcjlTTUVUUEUrcUowS05PRE1wTG1qQlBHSkZa?=
 =?utf-8?B?cEtEQkpGV0F4ZnNaNnRYeFhxdi9UWTlUZHQ5SVgyQytDTDhKN0VEQ0VXbXJM?=
 =?utf-8?B?VDNJT2M3WFFiMVJaZlU5eWh0ZVNDSkdDZ0JZZUNwZFo3RVVsWGp4blA1WWNW?=
 =?utf-8?B?SW9wU2MrbkltVG5UMHZISUl1RysvTVJlWE9XU04raXdSamNpR1UyZlU3czlH?=
 =?utf-8?B?US9ORGx1REdPZ1lFM3lqcVEvN3hjYldhL2hENTZiTENsYVZtYjlXK0RPOW1m?=
 =?utf-8?B?a25lSmJ5eGhCcHUxQng3czRITHlSRUNxYWE4bFljNmFKdkhmMnE4T05Meit3?=
 =?utf-8?B?TlFOSUlOY1FoRkN2a2dTV0R3YVl4aVRiYndyZUdBT2RYSkZPOUh1YTRNZTk1?=
 =?utf-8?B?QU5ENE5oT2FqbmdVbSsyR1lqT2RWR0xCTERFbC9lRGJTenlPbFRMT3RMdFBt?=
 =?utf-8?B?VjZPU05takJJd0dVTytsRWtEVVJ5bkI1Z0NXcTRtemMyY3VNblIxc2pFS2xt?=
 =?utf-8?B?cUh3QmpCclhXWCthaFEwOFZuY2JtQ1owd2ozUXloWHROOEhuL2V6K1NyTWJR?=
 =?utf-8?B?Y2hURnlwU0wzWDZFbDVoTGQ2WGRwLzJlZk1RcHRRZVEvdW9aM09UdTdleUwy?=
 =?utf-8?B?aWc5cEJRL0tIRTFpR2kyY1JabHllS3N4ampIWm9YUEpxYUtna1NqUnM2UWNm?=
 =?utf-8?B?OWVveGVlUnR6RTkvMXRYaDBhanVLL0MrODk5N0h2SVNaaHY3TEFiVTMxTkth?=
 =?utf-8?B?WkkwdmxneFpjak5Jdis2UEg2aXdURGVERFN3QVVjYmF2dFJwd0hSQSt2ME1n?=
 =?utf-8?B?dWV1UWlWbldmMi9ySEw5YjJ0ejlaL2s2cGNlR2tJZGZJcnFCamZiTlMxeEJk?=
 =?utf-8?B?aHdBa2VCRjVrSFJ4enY0bmpEOHBjUWlsam4rWDYrWlcrWURqZ1lXdEpwUzhC?=
 =?utf-8?B?akFHWHlXYmN6dHpHVS80bHYrREdjc1MrbllLckV5TFVzNUpwOVFnVmdyYlln?=
 =?utf-8?B?aUZybEJDWERzZTdDK0NFZnhNUnlKMFBmQ2YyWWRoMnlaR1VaTnJISG4zRG1o?=
 =?utf-8?B?TjQ3TXZRL1dGQ2RnT09ESkpTcndWYXlGVktvZlYyeThKeTZjdml3YW9mem5B?=
 =?utf-8?B?SXR5SWUxZDFiMFVZSWk0YnZwRlpqQW5rWVY3Qk41MlVpc0NoU21RaXZnaDMv?=
 =?utf-8?B?SXpvYTE0bnhqSmtNZ2l1cEtZcGhzblp3U25EckJWMHpYakhYTlgyR1FjZ3ZZ?=
 =?utf-8?Q?Jf6w9+x9Y0CGpWrLrQ0C7dv+KRnn6wvj2Qq+zLJ?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-cbsApmocyeJ2b5RoHR7B"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 82f146b9-f018-49bb-bc82-08d97f7c21ac
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 16:55:36.3428
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: n5Y/JRYACIS77odMmJqmF5rkVYwTpk+0/IaiBQW8hk5eUtG2AW/rUC1Wl2FAf85XK1+snDO8yZ5Su7V6Kk8Oag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4852

--=-cbsApmocyeJ2b5RoHR7B
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> There is no need for bitfields anywhere - use more sensible types.=C2=A0
> There is
> also no need to cast 'd' to (unsigned char *) before passing it to a
> function
> taking void *.
>=20
> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-cbsApmocyeJ2b5RoHR7B
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFOAwcACgkQFkJ4iaW4
c+5VoxAAm5IRhGDKOLM0ld3nYjiY8MLk11R0ESt+5Nyar76kiRYs+1IRPMCLXuuv
sfg+dayyN1xxpKaq2MD/9gO7eUoQRO7znlAz0F8uLBnqZfsUuEuzqovCNsAJ27tj
GvozaqXvWc1G/ET8PxBeugXCZ58Ta8G3pUFtYePSTquBp6oGU0BBYpWH/QbxbPB2
fa34c+qJMzhblvSJR9U2FFY9eT6EGaHHJSd5np+i9/oSkzB8TE0I/HoCHf8SZ8lT
4NV2rVEwRTkZnLZj+UHPm0rtByNp9MXty2l5AUVmSWdDKoySk+9irEUsdN2k1cCA
Jw4l0G7cbXSwjMRD0tnb4UTs1JPtzZDRVvJP26DOeKnco3paM3e7a8Lb+Y45LrQS
MMKyi8+Lmo1bueRdEHe6SSsc5pRT+z2PGVOyrE39szwEOrWDLPDSOKFTOEIJiwV5
eNnB6QyT1901gbU+VX8UQg3exrIKM3TzqI3ucOHxkaojRYlmT/oOncntUBix6Bgp
czY6+cuHe5e1qKfoEHBYlQx6Uh92knMdgbd+f0dKkU3CE+jA5BEwL/Czd963OFFy
uTsjKmIsQKQAu9/jlxOxQ2EL+StZwTS0Bpgllbf0b5ZJb/MtRcJ2KqlZXC2lRnZl
vmmHf7g2mmSwbAo4b5U9F8rHk3I33ZF1wtRjq5GvzFtCnRmgydM=
=m7Dn
-----END PGP SIGNATURE-----

--=-cbsApmocyeJ2b5RoHR7B--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:56:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:56:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195554.348316 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToUy-00047p-4G; Fri, 24 Sep 2021 16:56:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195554.348316; Fri, 24 Sep 2021 16:56:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToUy-00047i-0E; Fri, 24 Sep 2021 16:56:32 +0000
Received: by outflank-mailman (input) for mailman id 195554;
 Fri, 24 Sep 2021 16:56:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mToUx-00047c-Fc
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:56:31 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cabdd5f3-4bf4-425b-804e-60f03347a173;
 Fri, 24 Sep 2021 16:56:30 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2056.outbound.protection.outlook.com [104.47.5.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-KNbwxSlQN32gDs059HQpFw-1; Fri, 24 Sep 2021 18:56:28 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB4852.eurprd04.prod.outlook.com (2603:10a6:208:c4::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 16:56:27 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 16:56:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cabdd5f3-4bf4-425b-804e-60f03347a173
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632502589;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=CIAOIVl1ibqmSuT0l70VRYoVc3b+GFNXFAe7MZWjNRU=;
	b=l3lbmovS0Qfa/zfAwZCydpeLn4FjFgCDIU76gVUhAAHnOttFeK35xpSJoRDKVQo2JGvjvp
	FU3xCDLqAOa6DXg0tC5mOBh/r7aXA9L7y0UIyPkRe3Au6294eMKI1kf+Bqan3aQICmaXM1
	OIVYuQfZe7M/xmiPBDpe/ogRFtQTrVk=
X-MC-Unique: KNbwxSlQN32gDs059HQpFw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IPH65MB8kPCGmwVlofl5bFavmFJGx4sKSeVqjHKZ735w1kM/uYZai4rCMnWr8RixSy3HGb/10slULU21OZasS2teqax1pSj8hFgaCUeMIgG/9YWbtiATCGvbkiEqczurIEiAakX/pW3NPap0IvOGvDrgJf0D5vyGr0Npsi8FFBNRYK1Ir+npSQ3hOo6eClwpndvrji/MG48i0GHjPKTOXOW7XnGiDYYitWnXkvPzAX8BHJE2hn5R+Z/QGtND0ytiWzmwWrXnpF9G2OBLlcBF4RzvbMh7CXCW7Qs2ckeAAYmVWgF88MjZXha3HhTGxLlVP9aebSDZAkL/0y1gRMArog==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CIAOIVl1ibqmSuT0l70VRYoVc3b+GFNXFAe7MZWjNRU=;
 b=i6JmSgqZjumSq2iC3Bo8vU6y3CzD1qa+YA9GHPBJH1FJOqJURV1NAG0w3YAnnNrLoGJJi4hT/KgoX0t1MRFxaEfCNw57JqOVMPGVp/6XFIFgFBB1ZDGLwRcJ8aH7TzmH27bm0/x7Zd+LGvPlsBXJewgGln559S0tPi5vI/xjVBfqZ+DfxjvTAiQkL9ZUJ/NZXQrmfKLtIgWBUw8lMTOK/07QEVlYQ54ftPE/U7AGfRjUn+4ddy8k/itkyA3z+0PmBLEO4NkNizGbV3AO4mWIX+ZYIYJfA2gkj59eLGhKH3qkqUFdv5nTqpqmAt/CDD8wYLSRc+RTFXKA4kQfyLUThg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Jan Beulich <JBeulich@suse.com>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>,
	"mengxu@cis.upenn.edu" <mengxu@cis.upenn.edu>
Subject: Re: [PATCH v2 07/12] xen/rt: Clean up trace handling
Thread-Topic: [PATCH v2 07/12] xen/rt: Clean up trace handling
Thread-Index: AQHXrkSMaq9oWJXIp0iPAISC71kjzKuzba6A
Date: Fri, 24 Sep 2021 16:56:26 +0000
Message-ID: <7efbdf8d22de4ffcc71a86f2f742ac63f335e07a.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-8-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-8-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8d5613a0-750e-49a6-aff5-08d97f7c3f97
x-ms-traffictypediagnostic: AM0PR04MB4852:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB4852E9818CA171F7FEA7ED7EC5A49@AM0PR04MB4852.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:1201;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 1BBzMFQVoa4XI/xTETe72OFHEGvkJ2cA7Yo99SwzWQ3uuBN1vYcM1F6I8vABvcgX2bt8fGhdNvBWqZfBp4Ag7ioNrGxXHDq+zEmNtg0hwupMMMq+TLsVBg+wO4Xo5UV50iDdubGwWJUfItRFTSUK/BeIiY195/6AxbLOdtZKQmPLt2Qtr7uP6AyovExgufHOoAZXzxh0Unbp6QmqVO9HSNsjtqhYHGtcD2zMpQrYZeYo7cZaaZ5AFajA3vrYnvO56TuAj9F/uozRy3RQhdF5hWU8OJj1S4Le8hnV1lGWMUU+8VHEDcoXI3tKVK88nL/TWXHSKLbObkxCRi4a4Z9/A3NsASsfHyDzf0vO8a96hroTSJInyHaNUVJWsHFL6OyIXEJM/Ssp8u8/K6J0geaLU7v92fgEtsk0TUZW3/ZbLiEGZxBLfU7kdSpvgqqXqNGqaXkRX2u6HkLN9tPiDEjVFAfr3nam5r6qzVoOkdkLjYAyGipYUfRqwMXOWVzxxenKC7lMWE+ulPh2iiIBjli/mXNIKk6T0YyywlsHmsIIvk8qJP+TnVPFQuJL3fT3iF6B0tPsByWE4W/x8xfDqARtNFt+cFRwnPmHRoSgnfEG7x9qEiP7CF/lyfMN0qhL8BKWMe2r/ujUUjz/LzXS3AGDT2+xh/rU/9npm4ElPhUKU8Dos6ebqw2OgasyyqdjyrKJjI2kYGtnsyR16E4TTX8N27l0yNH0dz43XWcJjA5Q7DVETFb0tamPHqQBbDiQ0jxXtXBtfE8Nh8ZDKq36WfxoKV2Sg5R5CLh/x2LPy/wK59LNqfjQG3DUBrkkcH8lwvK9gv9f+QNuG8BqsY7hVpzv3w==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(966005)(91956017)(5660300002)(38100700002)(66946007)(316002)(508600001)(2906002)(110136005)(6512007)(8936002)(66476007)(86362001)(76116006)(2616005)(99936003)(66446008)(36756003)(66556008)(6506007)(71200400001)(186003)(38070700005)(4744005)(26005)(6486002)(4326008)(64756008)(122000001)(8676002)(66616009)(54906003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?RlVwUHNjL05aTFd0MUxmc3U5Q05UQ1pqbjY4TitlNEMzLzNZS0VvOUFCSmRI?=
 =?utf-8?B?QmI0dVVzcGpTMC9jdTJyNFk0RmtGUW5HNERnMCtzV3RVaXk1WGdqcDhGOXcw?=
 =?utf-8?B?VDBsR3JZaGxxRG5kVDJQU3UzTm5IQmtTLzd4QXNTeUVNdG94UFMyd1A3dWlu?=
 =?utf-8?B?eklsSy9BemN1MjM2WFZ4cW1yL0xYMzVCclRnbjVHUDRQZ09JWlJreEdsbEZF?=
 =?utf-8?B?ZHBLbUVBamsvTGxrM3RpNWhWcEdIZ0t1a3BNSVd5elR5cjczWkN4MjhNNVE5?=
 =?utf-8?B?TVYrby9iWXlyYmpTQ0NHK29yaUhSUE13dnNkSGIySlVwa0p3bUpXTThWeUlt?=
 =?utf-8?B?dlVXcjdSOVRraXhjUVRKT2VhVVBUUlFvN3docmpuTksyQnJ5bFVISHBoVU95?=
 =?utf-8?B?aERib2k1d0lqSE5uckh5SWFVU2JONFZPL2F0dGlBU0JPdzZUYncvMXVSYkNl?=
 =?utf-8?B?d2JIMk5tQ201bGVyWjNWb0tSVTQzVjVVcU11Y29CZFZWZFYxQ0l0RXlUeG13?=
 =?utf-8?B?RVZpeVJwTzdyVFFtTEpvOGRQdXQ0ZW94VWE1dS9ZemxoMkY3OU9na2srVFll?=
 =?utf-8?B?N3N5aDlBK1pMMGQ2Sk9hb2k4bTB3NEZBdkFjcUR3R2lvUk1mVVlxT3Z0QzN1?=
 =?utf-8?B?WkRLKzd1N0FPMXBRQXBicHBMRHdOODN6clVFTDk4L1I1VExyd0txNWV2TTh3?=
 =?utf-8?B?eHhJZi9GTTdOdk1yL3JyZm1vNE90TVdGTjBPZjhsc1pkWkdpWHB4TEpWM3Nk?=
 =?utf-8?B?d3lONkdBVER6NVpjTnNMYVZUVE9kNGlHMUMzL0lSQmVrU2NiQWN1M2kvbmVC?=
 =?utf-8?B?SGwyWS9EOG5TK01CSmR3d2RSNCs2SkJEK3Vpbm1tRG83L3FuNUZKZUJqZlRL?=
 =?utf-8?B?N1VvODNUZHFkRGpGcGExcDhIN3ZhcEhFRkpPWld5anFzVHlaQ3duQk9Fd1A5?=
 =?utf-8?B?S0YvOUthZHlsNVNLZjFzUXYwd2FJVmhXZTgzT0FoYXZYUWt6MkNmRDlHYzBa?=
 =?utf-8?B?d0tWNi9JTmptT3d6MVhaWFBEOXRLVm5UMnlBZ24yWTM3a1JhbTVWaG5xd1Bx?=
 =?utf-8?B?NWhxTlk1QnFrazZ2bElVMEZ0eGlzcDh2RlU1WWJOVU9iVUpvOC9LVm9UdThN?=
 =?utf-8?B?TVF5aDJXLzU0QXJ5L3NuditrUkRMUjJkdy9ZTnpFL1laeS9jdGlIS0hwY3hK?=
 =?utf-8?B?Y3l4WEZTa01YUDV3eTBrRElTYkdNajBtWUdxT1k2SG5UOXpxQ055M2E3T215?=
 =?utf-8?B?a0pmSDNFa2lyQm8yU0Fta05ZRkZTWVVhN3BYTW5GaEJYRXNvbWh2TGpYZEFw?=
 =?utf-8?B?OFdMQURLMk5oQmtIK1lnUi9KYWpVOU1wWG5nNEtKRDF1enBFeDRFUk1nZ1FU?=
 =?utf-8?B?c3JRWnNmRUVFeVdEenNvSUlCSDAvWG5mSjZFczU3SWgrWitPN3k2UVRIM05G?=
 =?utf-8?B?alJ3NnFudHV3YlpKaEYvM2RMUGhBNFRITzNJVENJaXVMeHROYTd2VHRuVith?=
 =?utf-8?B?dW1TKzRGNVlKc0FzOVBmd2piQmU5M05PSjg0YnhZZWJBMThYNi9SRnM5cUZY?=
 =?utf-8?B?S0pRV0NBb1FBcU1VTkNSZEU2OXpQSi9ZTXRFMTBFMmFsL3c3bHdQemVrSm1G?=
 =?utf-8?B?a2RFZE5RZ0pDcG51eWI2SzA4enFIVmxRcjRvbkFDdzNzL2FPcmc0Yk81M25F?=
 =?utf-8?B?cFNkMXg3N3VUT3ZEOG04eHpGR2VueG5QY1NEZ0dtbTh5ay8vL2hYVGdzYUFw?=
 =?utf-8?Q?DaxzYkgcTS86/iQjRu3OFJ7eDLuAGeU0Fywud/H?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-Doqs+l0Pe63MATy+SOIy"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8d5613a0-750e-49a6-aff5-08d97f7c3f97
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 16:56:26.7814
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: n6ymTEo514KxhyKdwhRYUUQZtdb4227cnr976uZutzrCiksoq7D1XbwrPpyiryUeQNeIVts0WqK9M9fcqxPssw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4852

--=-Doqs+l0Pe63MATy+SOIy
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> Most uses of bitfields and __packed are unnecessary.=C2=A0 There is also
> no need to
> cast 'd' to (unsigned char *) before passing it to a function taking
> void *.
>=20
> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-Doqs+l0Pe63MATy+SOIy
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFOAzoACgkQFkJ4iaW4
c+6E9A//Ymjk8w16+d6jo9lk/XJFJiJmSJjZLJ7S6BZlkc7IiIrHOgJLhiuC821X
y4il47O0Ezqj40HnzH8upM0dGe5J42bXZVWlMw9fqiRtfJt1r6nDYP2k/O3VceLu
jG2B5dS5IqXjOHkpLuH6ke7Rx5wv3nWmJee6jARC58iIaT9rl4lFSNL+/5VP+swM
AoFRGg3mGFZP1ZcJcce7zcf40xsh29LOXlDit88aGZpspZ86r96P8Mn78kMemqu2
5zIW1RZUzMGA2ei4ulNbTbF5brfPSyyCPgTRMTNuTdToy+kQnVwuaxfO8voBm8Gu
wREF439lkT6s5P1G6hJxpiWhHZB0B9vKYc1QxuCLjJWK5w93eyE2WUZUfiCmP74b
ygjoJp7z2lXWDbmH/J+J8MoSADKFvZyvhpatMolv9z4Vr6Ob3vD2pmJx5nEhUJ+r
SGPgxneSDNLQ3EJaUHSYfMpsTXcmqqcCTmyHCPeDlSZeZNrkgHfkmPBvMAPqkoiJ
qP//VsVDYAFY11KHQU75GJXPjg7Kq77GYzRHbDQ2N/vC6lJp+ZcQ0bfcb+AjJqN1
QYi7lf2R7wH128eGZnZgClA70HLHcoJp+91jndYTuvdNyix0BfloD7I1RMCOTECH
/2/Rr6zYUzHrmyhjc9RFyKYA248I1gayfy+sjbkuyTtovw2dg/w=
=J5BY
-----END PGP SIGNATURE-----

--=-Doqs+l0Pe63MATy+SOIy--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 16:57:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 16:57:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195562.348327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToVs-00059w-HE; Fri, 24 Sep 2021 16:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195562.348327; Fri, 24 Sep 2021 16:57:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mToVs-00059p-EB; Fri, 24 Sep 2021 16:57:28 +0000
Received: by outflank-mailman (input) for mailman id 195562;
 Fri, 24 Sep 2021 16:57:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3weo=OO=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mToVq-00059b-PM
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 16:57:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb504482-6e8f-4d3e-85df-d39b96453610;
 Fri, 24 Sep 2021 16:57:26 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2050.outbound.protection.outlook.com [104.47.5.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-JAobXZVBPwuJQn4PsboeRQ-1; Fri, 24 Sep 2021 18:57:23 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM0PR04MB4852.eurprd04.prod.outlook.com (2603:10a6:208:c4::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Fri, 24 Sep
 2021 16:57:22 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.018; Fri, 24 Sep 2021
 16:57:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb504482-6e8f-4d3e-85df-d39b96453610
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632502645;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=oIoAN0UX4ceFgmqtgrx4FF5mhK3rydaz373p31fVaf0=;
	b=M4tiOz5a+MaDuX1pyx7gIQALQHLLbPaQ5W8bDjp+bERaE70o1R+f+3ISkwSVEy4Ij1DYkM
	gcGKw3ibfaj56O3IlUQ7PvxXbHvJ9CoIJLrOo9tCXDae4YMkzNN2MfS5f5TWsdDg5SiSeJ
	1TfjnRwuJwWVCd8gDOAzoWUa2cbIejs=
X-MC-Unique: JAobXZVBPwuJQn4PsboeRQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b9547WxQnAm9OZ0xM4sfoTiugVhy6YXvxh2/XIew5DigC4zufYkhGgNKnUof4CfPHlKRMghjqVUY1G6Gm/XYk0/EOn1nwWfA/bG3qH8/yOlgbMXlzEQE1wCkYVTkGuiGuzFg0nAaYqyCBEoUfqVEDkc+2bXHZLyVLkz7yrUMh9nyGCpz/uOyQnviMvZtz2Q+wJlEdDpEjZwrpxxuib0x6YzKeXdZHHN/hKWOfPl10EFyRXRpmwEn709wfaejiAJmF0ApVGcA1cpt90p1ehQx1tvC3PxWioIqUGe68pScF/55MzZTNcUdQqy2mtqZpcyzoUVo0Dh11Ex7DPzyhegsHg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oIoAN0UX4ceFgmqtgrx4FF5mhK3rydaz373p31fVaf0=;
 b=XuQJSjt9/Jhd012vi85/jnlt5Nmww04xsAZ0nxibjd7aLWS6MKZqRhoTiJ50xNSmT/Si7gMb5sidHzK3MVdmkPcNbUcjRBCRv0SkxcCr/VyC200zJiOvIK6YDZD146RFA/lfpWdu+jbXtwlKiBsGZmZIoauF9Zh8XQ/NQoY7cF9dCBMX4V5ZPt57oeEBF0TcTMsyFGxPe/urC4lo5xW06TqKk7UKYmu26/W2fZ8Ybl0bxtNNWgMrxvj4FM4GT3jbv20JmjX+KTmOqtOgdDAk5YXTI0qEvv8nNQsdE/Vlg6PxOF3JxK20364j/K6scdX9VGHmKohRoMjLA0PJPXNH8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>, "George.Dunlap@eu.citrix.com"
	<George.Dunlap@eu.citrix.com>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, Juergen Gross <JGross@suse.com>, Jan Beulich
	<JBeulich@suse.com>, "iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org"
	<wl@xen.org>
Subject: Re: [PATCH v2 08/12] xen/sched: Clean up trace handling
Thread-Topic: [PATCH v2 08/12] xen/sched: Clean up trace handling
Thread-Index: AQHXrkSPAFgdWM+pi0KvFKeP2OUjvKuzbfGA
Date: Fri, 24 Sep 2021 16:57:22 +0000
Message-ID: <8037af821237cea64218ed2a04afbcf40f122fcd.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-9-andrew.cooper3@citrix.com>
In-Reply-To: <20210920172529.24932-9-andrew.cooper3@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 513694dd-8921-4daa-9836-08d97f7c60a1
x-ms-traffictypediagnostic: AM0PR04MB4852:
x-ld-processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM0PR04MB48528E531D9F1F9E845F1E8CC5A49@AM0PR04MB4852.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5516;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 EGkcb7UHx7hBe3E0Xyqjt89uIRbGmMOZCDvDk/1y341UmJkgDFktxlCsL6bbdiSRvCaP4/rGdksGPM+ziR0/L/32pokqq25V93TvdeKQ4e2mMJPlyepdiDmfENn/3SUHq0t6yGEIzPGKqciEjAuQGTt1SWGC0iQgMhOl/oxnOeG4naMCzwbsKQrgUjPDifogncNRFqMCHEQxYOvi12bjtg8L5yJLD0D5cnv6R3QV1EGRXU8I6T/EeeX3vxTKitqWvBoEAiISpGnkuAMebUIMlA0wyR7XUD9AkAaNjpg5ToRBP7rCY+FIrugU3YbK7nNUWFO6tBNVsQgSjhtQQ8QZCwa/USnCMtar7ALrNCfK6zli9ZuH1q5XXNGK3B8e02n7pDC1v8nTCNQYh5BUqunOma3sMCW23J6nOfbwvSIGbzda3fDSy53KniMt0OQPV0MA7Yqw24MzxxNa1FuNB4W3v9zaRtXll3jvSuKnDNXE0Xjezok4TqMAh8v4cuMzFYhn1wuU/KHvB4fsnv96+evzPeD/3cJnkZuSXPeJB7Xw2slCX4U1H4mA4UT2wWBsKyF10Kgyd/Y6YgIZ8sGMHJlpy/vvfs9KyrkU10OXDZcUAHcz3inK8qA5S0S+JpajjQVhLnY80R3jxEhwfMVb5kV38Oo6Iemkz3WSraq2r3wcBozR4Ze+QpidqzDMpybb7JPu8Rt/Pba5dEHR0xOJdYYy5t4H35AfTvMBbRZGulQ7xo3adex20C32wiSWBQwoBYdnYKVjPv3kCi0SHfpIjX+FzDB2ZBGFtEzp+RkS+mzdfBkMj/XLu5vs0bg1ffXLxMRif3X5Gc3G2BKezGTfDkolBw==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(71200400001)(186003)(38070700005)(4744005)(66446008)(36756003)(66556008)(6506007)(64756008)(122000001)(4326008)(66616009)(54906003)(8676002)(26005)(6486002)(508600001)(38100700002)(316002)(66946007)(966005)(91956017)(5660300002)(86362001)(76116006)(99936003)(2616005)(110136005)(6512007)(2906002)(66476007)(8936002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?dnAyWGhEKzB5YkU1ZlVaaU5seHJaeXM3UkRlY0ZWNDN2MkpBdHlNMldIWWpn?=
 =?utf-8?B?dExwS1c4YXkzNUFqL3pTcFJBRXdkc2ZjNzZIWnJ3ZkFETERlcms2K2lTdERu?=
 =?utf-8?B?c2JoM21qVFlkM3J0dGFhK25NTjAyWDhyaHBYQVZzOW1YdEhMSFJnTUdLbTVH?=
 =?utf-8?B?dGY2dFBBSWQ5UTFUeEQ2bjJwcXF1Wmt1V2R1MldBRTdXTDFsYi9YUXNySlIy?=
 =?utf-8?B?UTRsZlpmcUxTVXpsdE1BQ1dJbmFZVzdCbk5Mazg5dFpmZkE3VWx1WC8vdStX?=
 =?utf-8?B?Rk9aNEd1YzFESTc2NDlFMEpCby9sWFRGS3ZaeHN3MFdIYU55RVpUMnd4eG8v?=
 =?utf-8?B?ZHNibm90QW5tbkplOVA5MVFFOFZibStxUngzT3NmQ08xdldZNDl6K3NXUjlY?=
 =?utf-8?B?VU5EbThyMkJNN2lrQlplSG5UeFM2Vmt3NUNrNEJFWjAyZTRiR3F5SmRHeWpY?=
 =?utf-8?B?bllWM2hMMk5EdmQ2ejlvMXVPOHJUQjh0VkdEdVFkdG13bWpCQ0J4YVFtTDVa?=
 =?utf-8?B?R2lFZk83dE1ZZDhnUEkwemUxcE1Qc2dKVFVUY0dHTlN6Z21lT2dnVUV5NWR5?=
 =?utf-8?B?KzVPVXZxdWZZa3FuenhNQXJoVXZoVjc5VEpiWkhtU3E0b253blBiR3dHVjUv?=
 =?utf-8?B?d2VDbDZWWkpmVEpFRUR3bzBDWUs1YnRwcXVtYlZpV0lwcHBDbmVTZHZId29y?=
 =?utf-8?B?MUxxRGlzeUQvQ29GT3ZmVXZaSEVSb3VIdndQU1JmTnFLUXB2VkZ4ZTVtWHds?=
 =?utf-8?B?VHd3K0kyZGQ3alIyd2VadjBMMVUxKzZOYmw0M0lGT1NRNCtYNytZdm13NjQw?=
 =?utf-8?B?RkN1ZlRMamllb3lUL2pEUkZrbUo4Z01WM290TkJFa1hjSXo2SEl0cmZQdWh1?=
 =?utf-8?B?L1RqRVo3eGlIemNCSUZpaWh5Vk5xdmdEVHlKUGF6S01ad2JIdXExU2xuMHoz?=
 =?utf-8?B?VHhQVlg0WVQxYytQMlM4MytDOVZRdnJ0akVySllSaVpGTlcvbkRrWTM0ZGdF?=
 =?utf-8?B?SGdrWm5mTVd4VXdhY005cW1nRFJiV3BNWE95RXFtRDN5MU9YQ2hxOFhwUFRC?=
 =?utf-8?B?RFlVSHdTSEduaDR5Z2lvVW44QlRDczhMZ3c0TXF0c2VhU2JjMDkvb09aVGt1?=
 =?utf-8?B?SHhZaDJxRy9pSXVLSUh5elhqQWIvQ0ZRTnhYblowckxtVk52RmRsdXZWNEd4?=
 =?utf-8?B?MDJOREdRTjNvb2JtdFE1c211Y0ZkWUVselphWDIxRHFuckpHMEZZQkZMRUZK?=
 =?utf-8?B?NWtNT3paUjJkbk5mR1hTalorVGZ5cW5obG5pdStIckhNVnYwKzlOVTg2aXNS?=
 =?utf-8?B?OVJ0aTBaRUpxY1A4L1BmZjgvSVBqL0xpL3lWcW8rczFOL09HYlZDWlg2bDFw?=
 =?utf-8?B?MkxmeGZtVGdlWWl2YjlkUmFRekFyc1lWVTVuZTRKV3JuY0t4TW1scXpvUVo2?=
 =?utf-8?B?a3FNeFlrNENjNUgrdHplOWpFeENTSk04bCtMRFBvbkFwWGZlbjMwOVhFZzUz?=
 =?utf-8?B?L1lBMnRMMTdzZFV0UWVaaExINW9OVll1YjNmU0VLZ0lDUDJVVkU3YlVOYWxo?=
 =?utf-8?B?TXozUU80aFFPcWsvU2paTGloMTY5czBEZGV1ZElMY2pIVFdXRGdnNExLT2Va?=
 =?utf-8?B?azhLWEwyNDBzb2g3NU5zVlFVbmloTm4rUG1oYk11SEtVd2Uya0N1VlN6anRN?=
 =?utf-8?B?bHR6SDJlNVpxNG9VNXh0anZ0WXNwMUcxcnNGcDhNK052VGxaU1pvN1Zxamth?=
 =?utf-8?Q?WatXOzR16OJ0NeD8/D/r87a9LvL3ESwaABZtmk9?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-oqgLAJZhJ+E3VrEJ99bD"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 513694dd-8921-4daa-9836-08d97f7c60a1
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2021 16:57:22.2651
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4ZPHBVGY24SFU/0/y2xA9+MX4poZZlfL3VBARQWBOeXI9+6jcWM227+V0XI2bPFstVWp0Pxb1em4O91gKCSaUA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4852

--=-oqgLAJZhJ+E3VrEJ99bD
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> There is no need for bitfields anywhere - use more sensible types.=C2=A0
> There is
> also no need to cast 'd' to (unsigned char *) before passing it to a
> function
> taking void *.
>=20
> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>

Thanks and Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-oqgLAJZhJ+E3VrEJ99bD
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFOA3EACgkQFkJ4iaW4
c+4qFQ//RMiv73DW67mCYDoeGyRA+j3F1OgjrtmMVdP4Ez7oeHBlW3+YH3AYO3Mm
6ANyiqTduhqdrZmfilVlLcasL9wi7HvPo5KVjuFFiTqZhfVHb97n66y0Pizqb1bG
6pohBWmbx2jUsE6mcGMp/wdxmBxH5Cy6CgeMbwHsGy2RBiRCviakAMI4d3OTPeiP
NfCAShhPpXmWz2G9dHQ6s1w5D22785OgPzN8LD2JYRPPShqqZbGkQOyc5MebDmRI
YR+JJg3a+nmQwtig5rASlsAzqxKuv4gRIT0LCJh778Fa0jXr//7zgLPkx46nVjyX
0hYKHCnUiHpBAfnRj/zgUSG+a97ksuETVKKTOHL4FExSXGzCO8RaDSLY2z1CZkES
/J89Um/Zm/EHBVhUcKnPguHCh253IO3do0uIWG880k72jC52jSWkMBbv+8+L9k78
nbet97QwlKRoVAY/Xd5kfQYfyup7oN7/TTJhiLD3D7I/6x9hR7GUCtgIphequYnW
Io2lNwzxnAh6PhWAfqENQiSfSokVJeaBNH2dWcthbww8ePdWj34fBA1pLUY4Z2Pp
dZxmL+K+b7QHv0HX71lcDqEqpZ5BNJkoPQMM2e9XQHtHXcSUwv9IqbyslpFo1h7A
T0I8woWIxYNCBB5UQk0y51HEtWvuRgfvrMsJOQRNe71/DDnQLNs=
=aA5s
-----END PGP SIGNATURE-----

--=-oqgLAJZhJ+E3VrEJ99bD--



From xen-devel-bounces@lists.xenproject.org Fri Sep 24 17:48:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 17:48:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195582.348337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTpIz-0003M6-7S; Fri, 24 Sep 2021 17:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195582.348337; Fri, 24 Sep 2021 17:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTpIz-0003Lz-3Y; Fri, 24 Sep 2021 17:48:13 +0000
Received: by outflank-mailman (input) for mailman id 195582;
 Fri, 24 Sep 2021 17:48:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpIx-0003Lp-U5; Fri, 24 Sep 2021 17:48:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpIx-0006wj-ID; Fri, 24 Sep 2021 17:48:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpIx-0006U8-69; Fri, 24 Sep 2021 17:48:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpIx-00012Q-5g; Fri, 24 Sep 2021 17:48:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=cPEYRdQ0wShhX96KvIYl147rmlLmd3ubgHVBkEPlVWQ=; b=DtIXn1l161we3fo9nZnMMoqU3p
	kR0mRuBkdsesAc6zpf/mGV8OIL0eiNX3KaG5ZhtE4qeao7IZx9cveS/43l66BomKBh0+sUGFftArd
	dE8c6TOwLOgPcSuJSFDxqNX9bfM0vEBFjPyw3gfgUeff30IuIv3VAdX7IL9Vo7GTGZa0=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165176-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165176: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f9e36107ec70445fbdc2562ba5b60c0a7ed57c20
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 17:48:11 +0000

flight 165176 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165176/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                f9e36107ec70445fbdc2562ba5b60c0a7ed57c20
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  419 days
Failing since        152366  2020-08-01 20:49:34 Z  418 days  732 attempts
Testing same since   165176  2021-09-24 03:53:47 Z    0 days    1 attempts

------------------------------------------------------------
7328 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2266271 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 17:51:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 17:51:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195591.348352 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTpMM-0004n6-Td; Fri, 24 Sep 2021 17:51:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195591.348352; Fri, 24 Sep 2021 17:51:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTpMM-0004mz-Qg; Fri, 24 Sep 2021 17:51:42 +0000
Received: by outflank-mailman (input) for mailman id 195591;
 Fri, 24 Sep 2021 17:51:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpML-0004mp-Dz; Fri, 24 Sep 2021 17:51:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpML-0006zQ-8Z; Fri, 24 Sep 2021 17:51:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpML-0006in-2E; Fri, 24 Sep 2021 17:51:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTpML-0002Ot-1n; Fri, 24 Sep 2021 17:51:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=F3wX8W8ztKG7Y1Qo4m0zpjn8ea+k5UNSUwmdunrDQkc=; b=o1wqJrslCTWyu0Qpj83pr8eqyF
	NpX0YKQbwjX1a7BA8QZLsRNUxrT68V2B2SzqZ4CUSOAhMRTMzB9Mf8fYQQkaKiGaAHeXSOwx1W5RT
	LH4Rx3lmB1qnkfatixyxyOvSPVtcLtUrAeY04JnxEpyTvuiVWJIcDmeW4EDGVuZm+ETg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165181-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165181: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=728998f6f2b7e1420e771236efec65cbf6143b7b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 17:51:41 +0000

flight 165181 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165181/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  728998f6f2b7e1420e771236efec65cbf6143b7b

Last test of basis   165180  2021-09-24 10:01:36 Z    0 days
Testing same since   165181  2021-09-24 14:03:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ian Jackson <iwj@xenproject.org>
  Kevin Stefanov <kevin.stefanov@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   728998f6f2..dd6c062a7a  dd6c062a7a4abdb662c18af03d1396325969d155 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 19:34:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 19:34:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195600.348365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTqxG-0000Ot-E1; Fri, 24 Sep 2021 19:33:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195600.348365; Fri, 24 Sep 2021 19:33:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTqxG-0000Om-B8; Fri, 24 Sep 2021 19:33:54 +0000
Received: by outflank-mailman (input) for mailman id 195600;
 Fri, 24 Sep 2021 19:33:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTqxF-0000Og-8F
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:33:53 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b54de261-7f0e-487f-8cea-2c8eee6516b0;
 Fri, 24 Sep 2021 19:33:52 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1E35661164;
 Fri, 24 Sep 2021 19:33:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b54de261-7f0e-487f-8cea-2c8eee6516b0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632512031;
	bh=3ikA21jUfX+27vnyX9raytkHozwFnL1Vzbqobe6Vmvg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HSZNcDrcACykuSPV9IpsIPnvlFgAPUipXZ+BrtzpIuFEaoe2mQjKb8abORy/k9XUW
	 OrOCUkuHJwcUYkQo0Nu4O7rgX1rKgCbS810G88TioUjxF33OFxaK8ndEuak8ywAUuY
	 v5EkyPJRZ+OPynH79QWgjseZ8sdwSqb+eZIxdMchcYWLdmo7RP4pyPY1N2v2LfNi4k
	 s52rXfJCT7LyHb8ls0hnRTzYXYGc2ilSLRz/+AmcKpF+mURIwawqOzy5in8bHpm86Z
	 ZfY/sYSRgwfQpmkagqSXkRsynJGt8OHhVL8LOAJdAjGCwOktnI6kwvO1WQUdPuZEvC
	 JIdHJAwErevWw==
Date: Fri, 24 Sep 2021 12:33:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <wei.chen@arm.com>, 
    xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com, 
    andrew.cooper3@citrix.com, roger.pau@citrix.com, wl@xen.org
Subject: Re: [PATCH 25/37] xen/arm: implement bad_srat for Arm NUMA
 initialization
In-Reply-To: <d6973324-52e2-1c78-4f57-929d3af07a80@suse.com>
Message-ID: <alpine.DEB.2.21.2109241231460.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-26-wei.chen@arm.com> <alpine.DEB.2.21.2109231906570.17979@sstabellini-ThinkPad-T480s> <d6973324-52e2-1c78-4f57-929d3af07a80@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Jan Beulich wrote:
> On 24.09.2021 04:09, Stefano Stabellini wrote:
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> >> NUMA initialization will parse information from firmware provided
> >> static resource affinity table (ACPI SRAT or DTB). bad_srat if a
> >> function that will be used when initialization code encounters
> >> some unexcepted errors.
> >>
> >> In this patch, we introduce Arm version bad_srat for NUMA common
> >> initialization code to invoke it.
> >>
> >> Signed-off-by: Wei Chen <wei.chen@arm.com>
> >> ---
> >>  xen/arch/arm/numa.c | 7 +++++++
> >>  1 file changed, 7 insertions(+)
> >>
> >> diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> >> index 3755b01ef4..5209d3de4d 100644
> >> --- a/xen/arch/arm/numa.c
> >> +++ b/xen/arch/arm/numa.c
> >> @@ -18,6 +18,7 @@
> >>   *
> >>   */
> >>  #include <xen/init.h>
> >> +#include <xen/nodemask.h>
> >>  #include <xen/numa.h>
> >>  
> >>  static uint8_t __read_mostly
> >> @@ -25,6 +26,12 @@ node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
> >>      { 0 }
> >>  };
> >>  
> >> +__init void bad_srat(void)
> >> +{
> >> +    printk(KERN_ERR "NUMA: Firmware SRAT table not used.\n");
> >> +    fw_numa = -1;
> >> +}
> > 
> > I realize that the series keeps the "srat" terminology everywhere on DT
> > too. I wonder if it is worth replacing srat with something like
> > "numa_distance" everywhere as appropriate. I am adding the x86
> > maintainers for an opinion.
> > 
> > If you guys prefer to keep srat (if nothing else, it is concise), I am
> > also OK with keeping srat although it is not technically accurate.
> 
> I think we want to tell apart both things: Where we truly talk about
> the firmware's SRAT table, keeping that name is fine. But I suppose
> there no "Firmware SRAT table" (as in the log message above) when
> using DT?

No. FYI this is the DT binding:
https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/numa.txt

The interesting bit is the "distance-map"


> If so, at the very least in log messages SRAT shouldn't be
> mentioned. Perhaps even functions serving both an ACPI and a DT
> purpose would better not use "srat" in their names (but I'm not as
> fussed about it there.)

I agree 100% with what you wrote.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 19:36:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 19:36:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195606.348376 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTqzY-000108-R6; Fri, 24 Sep 2021 19:36:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195606.348376; Fri, 24 Sep 2021 19:36:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTqzY-000101-O2; Fri, 24 Sep 2021 19:36:16 +0000
Received: by outflank-mailman (input) for mailman id 195606;
 Fri, 24 Sep 2021 19:36:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTqzX-0000zs-QT
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:36:15 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 53ad9104-a96a-4f1b-88a6-2de41d0346fe;
 Fri, 24 Sep 2021 19:36:14 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id EC71560F9C;
 Fri, 24 Sep 2021 19:36:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53ad9104-a96a-4f1b-88a6-2de41d0346fe
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632512174;
	bh=pSTOQ28ChJIeFpsB9MN0yrrxhbt2p0i8GULiGjbziCI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gTbiAeySPcJxFw6eeyPjK9TlyD2rQG8VOpuLLrnn2u/zRVTHIXy7Dgap4DoGa0+em
	 GUacClFqlMWhkjeSeCdtrRT3snsUOiSaE4SGzrDEfgiWLgIA34oUW+pJsHeXPmKQ+y
	 F6gMRL6NAkAyc1pZ5EY5EpoJi+CkAo8tHRzU8zoHyAm5S45E7pcNFs5L4UILqX6lj1
	 y8SsvntPJLy98hCHuwBbaELmsRfrfx8ACPZ8WF97Sk2YymxzLGRS1C5tbVJ+4aE3yO
	 52yHhULrRn6ODJRn1cWFhZytVDB9NjZOb+KfRZTvdiEhZ6Kf9abJh8UbfseB5eIZ/z
	 bXdE0MqxRF+kw==
Date: Fri, 24 Sep 2021 12:36:13 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
In-Reply-To: <DB9PR08MB68571BAA4C8B07BB2061C8579EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109241233590.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-24-wei.chen@arm.com> <alpine.DEB.2.21.2109231843530.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB68571BAA4C8B07BB2061C8579EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-59369348-1632512174=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-59369348-1632512174=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月24日 9:47
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.org;
> > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > Subject: Re: [PATCH 23/37] xen/arm: implement node distance helpers for
> > Arm
> > 
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > We will parse NUMA nodes distances from device tree or ACPI
> > > table. So we need a matrix to record the distances between
> > > any two nodes we parsed. Accordingly, we provide this
> > > node_set_distance API for device tree or ACPI table parsers
> > > to set the distance for any two nodes in this patch.
> > > When NUMA initialization failed, __node_distance will return
> > > NUMA_REMOTE_DISTANCE, this will help us avoid doing rollback
> > > for distance maxtrix when NUMA initialization failed.
> > >
> > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > ---
> > >  xen/arch/arm/Makefile      |  1 +
> > >  xen/arch/arm/numa.c        | 69 ++++++++++++++++++++++++++++++++++++++
> > >  xen/include/asm-arm/numa.h | 13 +++++++
> > >  3 files changed, 83 insertions(+)
> > >  create mode 100644 xen/arch/arm/numa.c
> > >
> > > diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> > > index ae4efbf76e..41ca311b6b 100644
> > > --- a/xen/arch/arm/Makefile
> > > +++ b/xen/arch/arm/Makefile
> > > @@ -35,6 +35,7 @@ obj-$(CONFIG_LIVEPATCH) += livepatch.o
> > >  obj-y += mem_access.o
> > >  obj-y += mm.o
> > >  obj-y += monitor.o
> > > +obj-$(CONFIG_NUMA) += numa.o
> > >  obj-y += p2m.o
> > >  obj-y += percpu.o
> > >  obj-y += platform.o
> > > diff --git a/xen/arch/arm/numa.c b/xen/arch/arm/numa.c
> > > new file mode 100644
> > > index 0000000000..3f08870d69
> > > --- /dev/null
> > > +++ b/xen/arch/arm/numa.c
> > > @@ -0,0 +1,69 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * Arm Architecture support layer for NUMA.
> > > + *
> > > + * Copyright (C) 2021 Arm Ltd
> > > + *
> > > + * This program is free software; you can redistribute it and/or modify
> > > + * it under the terms of the GNU General Public License version 2 as
> > > + * published by the Free Software Foundation.
> > > + *
> > > + * This program is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > > + * GNU General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU General Public License
> > > + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > > + *
> > > + */
> > > +#include <xen/init.h>
> > > +#include <xen/numa.h>
> > > +
> > > +static uint8_t __read_mostly
> > > +node_distance_map[MAX_NUMNODES][MAX_NUMNODES] = {
> > > +    { 0 }
> > > +};
> > > +
> > > +void __init numa_set_distance(nodeid_t from, nodeid_t to, uint32_t
> > distance)
> > > +{
> > > +    if ( from >= MAX_NUMNODES || to >= MAX_NUMNODES )
> > > +    {
> > > +        printk(KERN_WARNING
> > > +               "NUMA: invalid nodes: from=%"PRIu8" to=%"PRIu8"
> > MAX=%"PRIu8"\n",
> > > +               from, to, MAX_NUMNODES);
> > > +        return;
> > > +    }
> > > +
> > > +    /* NUMA defines 0xff as an unreachable node and 0-9 are undefined
> > */
> > > +    if ( distance >= NUMA_NO_DISTANCE ||
> > > +        (distance >= NUMA_DISTANCE_UDF_MIN &&
> > > +         distance <= NUMA_DISTANCE_UDF_MAX) ||
> > > +        (from == to && distance != NUMA_LOCAL_DISTANCE) )
> > > +    {
> > > +        printk(KERN_WARNING
> > > +               "NUMA: invalid distance: from=%"PRIu8" to=%"PRIu8"
> > distance=%"PRIu32"\n",
> > > +               from, to, distance);
> > > +        return;
> > > +    }
> > > +
> > > +    node_distance_map[from][to] = distance;
> > > +}
> > > +
> > > +uint8_t __node_distance(nodeid_t from, nodeid_t to)
> > > +{
> > > +    /* When NUMA is off, any distance will be treated as remote. */
> > > +    if ( srat_disabled() )
> > 
> > Given that this is ARM specific code and specific to ACPI, I don't think
> > we should have any call to something called "srat_disabled".
> > 
> > I suggest to either rename srat_disabled to numa_distance_disabled.
> > 
> > Other than that, this patch looks OK to me.
> > 
> 
> srat stands for static resource affinity table, I think dtb also can be
> treated as a static resource affinity table. So I keep SRAT in this patch
> and other patches. I have seen your comment in patch#25. Before x86 maintainers
> give any feedback, can we still keep srat here?

Jan and I replied in the other thread. I think that in warning messages
"SRAT" should not be mentioned when booting from DT. Ideally functions
names and variables should be renamed too when shared between ACPI and
DT but it is less critical, and it is fine if you don't do that in the
next version.
--8323329-59369348-1632512174=:17979--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 19:39:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 19:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195616.348388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTr2k-00020e-Az; Fri, 24 Sep 2021 19:39:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195616.348388; Fri, 24 Sep 2021 19:39:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTr2k-00020X-77; Fri, 24 Sep 2021 19:39:34 +0000
Received: by outflank-mailman (input) for mailman id 195616;
 Fri, 24 Sep 2021 19:39:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTr2i-00020R-Gg
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:39:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b15ded08-77a2-423a-a907-3f01ababa5f6;
 Fri, 24 Sep 2021 19:39:31 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9B440611C8;
 Fri, 24 Sep 2021 19:39:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b15ded08-77a2-423a-a907-3f01ababa5f6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632512370;
	bh=YNQBVARhL5zGPlsY8c6JbZE4vqiRLj/dGpAmkEmi8rE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NriPQmHT0yFKfq9IjBUU981hc6ogqQM0e+kZAhb/Mq5M3PX97mKmZCysbHKN3xqy9
	 8JYa4e1K5zxhMzpVz6p+rb+lIOF20YiJo1I1eS0KW4ecWGNYQ5kV3EnK0VlViRfVos
	 bD+yOnu+RxJep3n/ibtLUblFPFauMZ9THl2sCBCincb+zUCx4XnMzl+I5BjnRYUvZ+
	 mMFWYiZxITzFxjt7ha+29dOrY4UMuMUznDdBAPpzW/9O4jvPiC6lV7f+H0syLMM215
	 RqbSqBWWrLXusgm+zPlYuhUPh4TzTmRYRjuFS3I+XwXmFr6YY4Or77g/POruGBfwJp
	 /2xhS8K1BlKhQ==
Date: Fri, 24 Sep 2021 12:39:29 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
In-Reply-To: <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
Message-ID: <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-37-wei.chen@arm.com> <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s> <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Wei Chen wrote:
> Hi Stefano,
> 
> On 2021/9/24 11:31, Stefano Stabellini wrote:
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > Arm platforms support both ACPI and device tree. We don't
> > > want users to select device tree NUMA or ACPI NUMA manually.
> > > We hope usrs can just enable NUMA for Arm, and device tree
> >            ^ users
> > 
> > > NUMA and ACPI NUMA can be selected depends on device tree
> > > feature and ACPI feature status automatically. In this case,
> > > these two kinds of NUMA support code can be co-exist in one
> > > Xen binary. Xen can check feature flags to decide using
> > > device tree or ACPI as NUMA based firmware.
> > > 
> > > So in this patch, we introduce a generic option:
> > > CONFIG_ARM_NUMA for user to enable NUMA for Arm.
> >                        ^ users
> > 
> 
> OK
> 
> > > And one CONFIG_DEVICE_TREE_NUMA option for ARM_NUMA
> > > to select when HAS_DEVICE_TREE option is enabled.
> > > Once when ACPI NUMA for Arm is supported, ACPI_NUMA
> > > can be selected here too.
> > > 
> > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > ---
> > >   xen/arch/arm/Kconfig | 11 +++++++++++
> > >   1 file changed, 11 insertions(+)
> > > 
> > > diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> > > index 865ad83a89..ded94ebd37 100644
> > > --- a/xen/arch/arm/Kconfig
> > > +++ b/xen/arch/arm/Kconfig
> > > @@ -34,6 +34,17 @@ config ACPI
> > >   	  Advanced Configuration and Power Interface (ACPI) support for Xen is
> > >   	  an alternative to device tree on ARM64.
> > >   + config DEVICE_TREE_NUMA
> > > +	def_bool n
> > > +	select NUMA
> > > +
> > > +config ARM_NUMA
> > > +	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if
> > > UNSUPPORTED
> > > +	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> > 
> > Should it be: depends on HAS_DEVICE_TREE ?
> > (And eventually depends on HAS_DEVICE_TREE || ACPI)
> > 
> 
> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
> option can be selected by users. And depends on has_device_tree
> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
> 
> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
> does it become a loop dependency?
> 
> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html

OK, I am fine with that. I was just trying to catch the case where a
user selects "ARM_NUMA" but actually neither ACPI nor HAS_DEVICE_TREE
are selected so nothing happens. I was trying to make it clear that
ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACPI
because otherwise it is not going to work.

That said, I don't think this is important because HAS_DEVICE_TREE
cannot be unselected. So if we cannot find a way to express the
dependency, I think it is fine to keep the patch as is.


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 19:42:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 19:42:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195626.348399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTr5O-0003MF-T6; Fri, 24 Sep 2021 19:42:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195626.348399; Fri, 24 Sep 2021 19:42:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTr5O-0003M8-Q8; Fri, 24 Sep 2021 19:42:18 +0000
Received: by outflank-mailman (input) for mailman id 195626;
 Fri, 24 Sep 2021 19:42:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTr5N-0003M2-Ai
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:42:17 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8595dd97-1d6f-11ec-bb18-12813bfff9fa;
 Fri, 24 Sep 2021 19:42:16 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AA0D961050;
 Fri, 24 Sep 2021 19:42:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8595dd97-1d6f-11ec-bb18-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632512535;
	bh=E4zyvvlfShrWcO7DtJukosxn9xc084kCbr2MI8bzreI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ml1eqjyZqGrBv1twWNvzhVMjKJBGjSoLX6/sb/0iIUMCJmGtR1/1G5zqSYAMb/M9/
	 TMz5JxujUAF90EOJXMVB2OFIrK1DGj++LicU68wUwEWd4OF+jGOSSs+MfDcjJVapZX
	 hiw7jPCGA8oxJ1UfeBPIL1QuScU3ScrjVkigfu7/aiMEq0P3R8HXlsM3MMmy87l0GR
	 OeelQE+mC3cfsaEQuaQG4QPN3Qm/84OuBCiphw+qeh+ojCg5zvrvZrMlUba5YHKbOw
	 zcnUwJVdcIDQZmLWtWa+3d+R7TBEMq8v+aLz5CsSBnxLv6mSkBSK0gqCYx9Eid7BEr
	 i7uYag1scoKng==
Date: Fri, 24 Sep 2021 12:42:15 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data from
 device tree
In-Reply-To: <DB9PR08MB6857E6B6BDE4D55F7D468DDF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109241239450.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-33-wei.chen@arm.com> <alpine.DEB.2.21.2109232007160.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB6857E6B6BDE4D55F7D468DDF9EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-2011522536-1632512535=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2011522536-1632512535=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月24日 11:17
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.org;
> > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > Subject: Re: [PATCH 32/37] xen/arm: unified entry to parse all NUMA data
> > from device tree
> > 
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > In this API, we scan whole device tree to parse CPU node id, memory
> >           ^ function   ^ the whole
> > 
> > > node id and distance-map. Though early_scan_node will invoke has a
> > > handler to process memory nodes. If we want to parse memory node id
> > > in this handler, we have to embeded NUMA parse code in this handler.
> >                               ^ embed
> > 
> > > But we still need to scan whole device tree to find CPU NUMA id and
> > > distance-map. In this case, we include memory NUMA id parse in this
> > > API too. Another benefit is that we have a unique entry for device
> >   ^ function
> > 
> > > tree NUMA data parse.
> > 
> > Ah, that's the explanation I was asking for earlier!
> > 
> 
> The question about device_tree_get_meminfo?

Yes, it would be nice to reuse process_memory_node if we can, but I
understand if we cannot.
--8323329-2011522536-1632512535=:17979--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 19:52:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 19:52:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195633.348410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTrFZ-00058H-TV; Fri, 24 Sep 2021 19:52:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195633.348410; Fri, 24 Sep 2021 19:52:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTrFZ-00058A-QK; Fri, 24 Sep 2021 19:52:49 +0000
Received: by outflank-mailman (input) for mailman id 195633;
 Fri, 24 Sep 2021 19:52:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTrFZ-000584-1T
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:52:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fd890765-1d70-11ec-bb18-12813bfff9fa;
 Fri, 24 Sep 2021 19:52:47 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A255B6103D;
 Fri, 24 Sep 2021 19:52:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd890765-1d70-11ec-bb18-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632513166;
	bh=HYq8vef2T/Mz3Uqsl0mASy2K0c2VYjZ9WfF4zQs9xFw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=pqIhoB81gqaQ6sw1OdEROKxCCVxyql56qbRkTbbEKL50dO5FOpw9pPpl/FDRTx081
	 T8p0EVaRoVUaHM0NBuoWcnQXEmLZgOFKBEtVB0d76YmV0WlWORsDJszHKqKToLAjuW
	 qanxx45vJ/BKtAV8kmbcbdOO3pLcqxGGncUe57g+x0+nBlOZj5h338xzCHRWjMbXEv
	 dzE+h7qboDtMw5bpzOXuxVvMd9ch+sm3dK55vO7WO7EavUX1dDr6e7uIqWKwqzvhMf
	 wJq6yHLqaYb7zfY3cAPfoiAIJoBDdvV88vH1fXwELaxep1twQClvFsiNIJDSs6ACca
	 4r6GbCutjT2Ng==
Date: Fri, 24 Sep 2021 12:52:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    "jbeulich@suse.com" <jbeulich@suse.com>, 
    "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com> <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-640249418-1632513166=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-640249418-1632513166=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月24日 8:26
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.org;
> > Bertrand Marquis <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > andrew.cooper3@citrix.com; roger.pau@citrix.com; wl@xen.org
> > Subject: Re: [PATCH 08/37] xen/x86: add detection of discontinous node
> > memory range
> > 
> > CC'ing x86 maintainers
> > 
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > One NUMA node may contain several memory blocks. In current Xen
> > > code, Xen will maintain a node memory range for each node to cover
> > > all its memory blocks. But here comes the problem, in the gap of
> > > one node's two memory blocks, if there are some memory blocks don't
> > > belong to this node (remote memory blocks). This node's memory range
> > > will be expanded to cover these remote memory blocks.
> > >
> > > One node's memory range contains othe nodes' memory, this is obviously
> > > not very reasonable. This means current NUMA code only can support
> > > node has continous memory blocks. However, on a physical machine, the
> > > addresses of multiple nodes can be interleaved.
> > >
> > > So in this patch, we add code to detect discontinous memory blocks
> > > for one node. NUMA initializtion will be failed and error messages
> > > will be printed when Xen detect such hardware configuration.
> > 
> > At least on ARM, it is not just memory that can be interleaved, but also
> > MMIO regions. For instance:
> > 
> > node0 bank0 0-0x1000000
> > MMIO 0x1000000-0x1002000
> > Hole 0x1002000-0x2000000
> > node0 bank1 0x2000000-0x3000000
> > 
> > So I am not familiar with the SRAT format, but I think on ARM the check
> > would look different: we would just look for multiple memory ranges
> > under a device_type = "memory" node of a NUMA node in device tree.
> > 
> > 
> 
> Should I need to include/refine above message to commit log?

Let me ask you a question first.

With the NUMA implementation of this patch series, can we deal with
cases where each node has multiple memory banks, not interleaved?
An an example:

node0: 0x0        - 0x10000000
MMIO : 0x10000000 - 0x20000000
node0: 0x20000000 - 0x30000000
MMIO : 0x30000000 - 0x50000000
node1: 0x50000000 - 0x60000000
MMIO : 0x60000000 - 0x80000000
node2: 0x80000000 - 0x90000000


I assume we can deal with this case simply by setting node0 memory to
0x0-0x30000000 even if there is actually something else, a device, that
doesn't belong to node0 in between the two node0 banks?

Is it only other nodes' memory interleaved that cause issues? In other
words, only the following is a problematic scenario?

node0: 0x0        - 0x10000000
MMIO : 0x10000000 - 0x20000000
node1: 0x20000000 - 0x30000000
MMIO : 0x30000000 - 0x50000000
node0: 0x50000000 - 0x60000000

Because node1 is in between the two ranges of node0?


I am asking these questions because it is certainly possible to have
multiple memory ranges for each NUMA node in device tree, either by
specifying multiple ranges with a single "reg" property, or by
specifying multiple memory nodes with the same numa-node-id.
--8323329-640249418-1632513166=:17979--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 20:04:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 20:04:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195641.348421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTrQo-000755-3h; Fri, 24 Sep 2021 20:04:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195641.348421; Fri, 24 Sep 2021 20:04:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTrQo-00074y-0e; Fri, 24 Sep 2021 20:04:26 +0000
Received: by outflank-mailman (input) for mailman id 195641;
 Fri, 24 Sep 2021 20:04:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTrQm-00074s-FW
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 20:04:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 54c40272-acab-4974-963a-765555693b00;
 Fri, 24 Sep 2021 20:04:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DC2B61107;
 Fri, 24 Sep 2021 20:04:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54c40272-acab-4974-963a-765555693b00
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632513862;
	bh=5g6F24eAkfUrluLlmCOMkaLhZyk84IZGlaFRfiC0LOE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=lmDBZ9oIyS3njcFNZoPha7UcobA4aS2FQkVoccD3LUilnqmsyJrUfNqkmWR8yOB3t
	 8gbmOYvaF43D55psInmqaApM29ibqVjCF3q0iWMhNWZejhTBf16LlT8XDlYP/fnHMX
	 Dt5XOweabDGODl/B+KyIhLcaXT+S08vFjV90vk2cmsrtWNdgwXvWuviwLz/6byzySc
	 BeG8mbo13ZAdxU/T9llrJzg8G4nbQgevNASLL/ys8+F0tTqo6BRrzQP0D9Q5NZqahq
	 SaQeX5Hw8NIojx1AZbVFK0OufMlA62FfoHWWQ6n3vY5YqOehQqRG4GQEEnOAz6Uo7J
	 NFCZmDQbtjK2w==
Date: Fri, 24 Sep 2021 13:04:21 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "jgross@suse.com" <jgross@suse.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, "jbeulich@suse.com" <jbeulich@suse.com>, 
    Anastasiia Lukianenko <Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
In-Reply-To: <7310d23e-4193-3f4c-06da-606b30e73f24@epam.com>
Message-ID: <alpine.DEB.2.21.2109241258190.17979@sstabellini-ThinkPad-T480s>
References: <20210923095345.185489-1-andr2000@gmail.com> <20210923095345.185489-2-andr2000@gmail.com> <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s> <f62a1e2c-4253-c998-c206-6bb0681a84fb@epam.com>
 <7310d23e-4193-3f4c-06da-606b30e73f24@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-2142721594-1632513512=:17979"
Content-ID: <alpine.DEB.2.21.2109241258500.17979@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2142721594-1632513512=:17979
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109241258501.17979@sstabellini-ThinkPad-T480s>

On Fri, 24 Sep 2021, Oleksandr Andrushchenko wrote:
> On 24.09.21 08:46, Oleksandr Andrushchenko wrote:
> > On 23.09.21 23:00, Stefano Stabellini wrote:
> >> On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> >>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>>
> >>> Xen-pciback driver was designed to be built for x86 only. But it
> >>> can also be used by other architectures, e.g. Arm.
> >>> Re-structure the driver in a way that it can be built for other
> >>> platforms as well.
> >>>
> >>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
> >> The patch looks good to me. Only one thing: on ARM32 I get:
> > WE do not yet support Xen PCI passthrough for ARM32

Keep in mind that it is possible to run ARM32 guests on an ARM64
hypervisor.


> >> drivers/xen/xen-pciback/conf_space_header.c: In function ‘bar_init’:
> >> drivers/xen/xen-pciback/conf_space_header.c:239:34: warning: right shift count >= width of type [-Wshift-count-overflow]
> >>       bar->val = res[pos - 1].start >> 32;
> >>                                     ^~
> >> drivers/xen/xen-pciback/conf_space_header.c:240:49: warning: right shift count >= width of type [-Wshift-count-overflow]
> >>       bar->len_val = -resource_size(&res[pos - 1]) >> 32;
> >>    
> >>    
> >> resource_size_t is defined as phys_addr_t and it can be 32bit on arm32.
> >>
> >>
> >> One fix is to surround:
> >>
> >> 		if (pos && (res[pos - 1].flags & IORESOURCE_MEM_64)) {
> >> 			bar->val = res[pos - 1].start >> 32;
> >> 			bar->len_val = -resource_size(&res[pos - 1]) >> 32;
> >> 			return bar;
> >> 		}
> >>
> >> with #ifdef PHYS_ADDR_T_64BIT
> >>
> > This might not be correct. We are dealing here with a 64-bit BAR on a 32-bit OS.
> >
> > I think that this can still be valid use-case if BAR64.hi == 0. So, not sure
> >
> > we can just skip it with ifdef.
> >
> > Instead, to be on the safe side, we can have:
> >
> > config XEN_PCIDEV_STUB
> >          tristate "Xen PCI-device stub driver"
> >          depends on PCI && ARM64 && XEN
> > e.g. only allow building the "stub" for ARM64 for now.

This is a pretty drastic solution. I would be OK with it but I prefer
the solution below >> 16 >> 16.


> Or... there are couple of places in the kernel where PCI deals with the 32 bit shift as:
> 
> drivers/pci/setup-res.c:108:        new = region.start >> 16 >> 16;
> drivers/pci/iov.c:949:        new = region.start >> 16 >> 16;
> 
> commit cf7bee5a0bf270a4eace0be39329d6ac0136cc47
> Date:   Sun Aug 7 13:49:59 *2005* +0400
> 
> [snip]
> 
>      Also make sure to write high bits - use "x >> 16 >> 16" (rather than the
>      simpler ">> 32") to avoid warnings on 32-bit architectures where we're
>      not going to have any high bits.

I think this is the best option


> This might not be(?) immediately correct in case of LPAE though, e.g.
> 
> 64-bit BAR may tolerate 40-bit address in some use-cases?

It is correct for LPAE too, it is just that with LPAE it would be
unnecessary.

--8323329-2142721594-1632513512=:17979--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 20:45:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 20:45:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195649.348432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTs4K-0003zf-Un; Fri, 24 Sep 2021 20:45:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195649.348432; Fri, 24 Sep 2021 20:45:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTs4K-0003zY-Re; Fri, 24 Sep 2021 20:45:16 +0000
Received: by outflank-mailman (input) for mailman id 195649;
 Fri, 24 Sep 2021 20:45:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTs4J-0003zS-7R
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 20:45:15 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 50f128e4-1d78-11ec-bb1d-12813bfff9fa;
 Fri, 24 Sep 2021 20:45:13 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6973961241;
 Fri, 24 Sep 2021 20:45:12 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50f128e4-1d78-11ec-bb1d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632516313;
	bh=KyAVkwIvSD0HPZVFUG8s3/STS6juQwGiAJR/VIKsUdA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dEETuFulqpJBIvmXE7x8dQ0IoPWpqW7R0/IRkjZGQVvmEne/9JqKsL+sZXX+VAmZf
	 IIm7HImBP2rIB2b0EjPaSaKY1M0KTQBoo0DvwX2CaTs9o/s8dSs2K8Mjr6lFgYYTOD
	 wL/MBgiAfaoKRpwa7qCFcMonLT91oK9EmiTlq2uafMopQZBnGkmH1L2RkccIzSWrYg
	 jmZF/ElE0WmRByiGQdzxgbejnlGXpjgnNEZldfbNgoSrvz7kl/J8BVGq7AFPVZ1x5L
	 p3P4/wwwVPUqVgRXNU8Ra3fRpXUP0rajxfFCb0d36s/AdX7tQ/ZCrhJa/L8TNHWY1S
	 WQMENkuIEr6ng==
Date: Fri, 24 Sep 2021 13:45:10 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v2 2/2] arm/efi: Use dom0less configuration when using
 EFI boot
In-Reply-To: <6A929631-AA44-4ECF-AACB-16C0110393F6@arm.com>
Message-ID: <alpine.DEB.2.21.2109241334430.17979@sstabellini-ThinkPad-T480s>
References: <20210922141341.42288-1-luca.fancellu@arm.com> <20210922141341.42288-3-luca.fancellu@arm.com> <alpine.DEB.2.21.2109221430210.17979@sstabellini-ThinkPad-T480s> <2C0E480C-DF1C-4AAD-89F5-99D52B016970@arm.com> <alpine.DEB.2.21.2109230943510.17979@sstabellini-ThinkPad-T480s>
 <6A929631-AA44-4ECF-AACB-16C0110393F6@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-156143097-1632516312=:17979"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-156143097-1632516312=:17979
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 24 Sep 2021, Luca Fancellu wrote:
> > On 23 Sep 2021, at 17:59, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 23 Sep 2021, Luca Fancellu wrote:
> >>>> +/*
> >>>> + * Binaries will be translated into bootmodules, the maximum number for them is
> >>>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> >>>> + */
> >>>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> >>>> +static struct file __initdata dom0less_file;
> >>>> +static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
> >>>> +static unsigned int __initdata dom0less_modules_available =
> >>>> +                               MAX_DOM0LESS_MODULES;
> >>>> +static unsigned int __initdata dom0less_modules_idx;
> >>> 
> >>> This is a lot better!
> >>> 
> >>> We don't need both dom0less_modules_idx and dom0less_modules_available.
> >>> You can just do:
> >>> 
> >>> #define dom0less_modules_available (MAX_DOM0LESS_MODULES - dom0less_modules_idx)
> >>> static unsigned int __initdata dom0less_modules_idx;
> >>> 
> >>> But maybe we can even get rid of dom0less_modules_available entirely?
> >>> 
> >>> We can change the check at the beginning of allocate_dom0less_file to:
> >>> 
> >>> if ( dom0less_modules_idx == dom0less_modules_available )
> >>>   blexit
> >>> 
> >>> Would that work?
> >> 
> >> I thought about it but I think they need to stay, because dom0less_modules_available is the
> >> upper bound for the additional dom0less modules (it is decremented each time a dom0 module
> >> Is added), instead dom0less_modules_idx is the typical index for the array of dom0less modules.
> > 
> > [...]
> > 
> > 
> >>>> +    /*
> >>>> +     * Check if there is any space left for a domU module, the variable
> >>>> +     * dom0less_modules_available is updated each time we use read_file(...)
> >>>> +     * successfully.
> >>>> +     */
> >>>> +    if ( !dom0less_modules_available )
> >>>> +        blexit(L"No space left for domU modules");
> >>> 
> >>> This is the check that could be based on dom0less_modules_idx
> >>> 
> >> 
> >> The only way I see to have it based on dom0less_modules_idx will be to compare it
> >> to the amount of modules still available, that is not constant because it is dependent
> >> on how many dom0 modules are loaded, so still two variables needed.
> >> Don’t know if I’m missing something.
> > 
> > I think I understand where the confusion comes from. I am appending a
> > small patch to show what I had in mind. We are already accounting for
> > Xen and the DTB when declaring MAX_DOM0LESS_MODULES (MAX_MODULES - 2).
> > The other binaries are the Dom0 kernel and ramdisk, however, in my setup
> > they don't trigger a call to handle_dom0less_module_node because they
> > are compatible xen,linux-zimage and xen,linux-initrd.
> > 
> > However, the Dom0 kernel and ramdisk can be also compatible
> > multiboot,kernel and multiboot,ramdisk. If that is the case, then they
> > would indeed trigger a call to handle_dom0less_module_node.
> > 
> > I think that is not a good idea: a function called
> > handle_dom0less_module_node should only be called for dom0less modules
> > (domUs) and not dom0.

I can see that I misread the code yesterday: Dom0 modules don't go
through handle_dom0less_module_node thanks to the xen,domain check in
efi_arch_check_dom0less_boot.


> > But from the memory consumption point of view, it would be better
> > actually to catch dom0 modules too as you intended. In that case we need to:
> > 
> > - add a check for xen,linux-zimage and xen,linux-initrd in
> >  handle_dom0less_module_node also
> > 
> > - rename handle_dom0less_domain_node, handle_dom0less_module_node,
> >  dom0less_file, dom0less_modules, dom0less_modules_idx to something
> >  else more generic
> > 
> > 
> > For instance they could be called:
> > 
> > handle_domain_node
> > handle_module_node
> > module_file
> > modules
> > modules_idx
> > 
> > 
> > 
> > 
> > diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> > index e2b007ece0..812d0bd607 100644
> > --- a/xen/arch/arm/efi/efi-boot.h
> > +++ b/xen/arch/arm/efi/efi-boot.h
> > @@ -22,8 +22,6 @@ typedef struct {
> > #define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> > static struct file __initdata dom0less_file;
> > static dom0less_module_name __initdata dom0less_modules[MAX_DOM0LESS_MODULES];
> > -static unsigned int __initdata dom0less_modules_available =
> > -                               MAX_DOM0LESS_MODULES;
> > static unsigned int __initdata dom0less_modules_idx;
> > 
> > #define ERROR_DOM0LESS_FILE_NOT_FOUND (-1)
> > @@ -592,14 +590,6 @@ static void __init efi_arch_handle_module(const struct file *file,
> >          * stop here.
> >          */
> >         blexit(L"Unknown module type");
> > -
> > -    /*
> > -     * dom0less_modules_available is decremented here because for each dom0
> > -     * file added, there will be an additional bootmodule, so the number
> > -     * of dom0less module files will be decremented because there is
> > -     * a maximum amount of bootmodules that can be loaded.
> > -     */
> > -    dom0less_modules_available--;
> > }
> > 
> > /*
> > @@ -643,7 +633,7 @@ static unsigned int __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
> >      * dom0less_modules_available is updated each time we use read_file(...)
> >      * successfully.
> >      */
> > -    if ( !dom0less_modules_available )
> > +    if ( dom0less_modules_idx == MAX_DOM0LESS_MODULES )
> >         blexit(L"No space left for domU modules");
> > 
> >     module_name.s = (char*) name;
> 
> Hi Stefano,
> 
> Yes I understand what you mean, unfortunately it can’t be done, I will explain why in details
> because the UEFI code is very tricky in the way it was written.
> 
> Dom0 modules and xsm-policy are handled in boot.c by read_section(…) or read_file(…) and
> both call handle_file_info(…) that inside calls efi_arch_handle_module(…).
> Dom0less modules (xen,domain child modules) are handled in efi-boot.h by handle_dom0less_module_node(…)
> that may call allocate_dom0less_file(…) and (to reuse code) calls read_file(…).
> 
> So there can be maximum (MAX_MODULES-2) boot modules because at start in start_xen(…)
> we add Xen and the DTB as boot modules.
> 
> This amount is to be shared among dom0 modules (kernel, ramdisk), xsm-policy and domU
> modules (kernel, ramdisk, dtb).
> 
> The thing is that we don’t know how many dom0 modules will be specified and also for the xsm-policy.
> 
> In your code example, let’s say I have MAX_DOM0LESS_MODULES=(32-2) so 30 modules available,
> If I declare a Dom0 kernel and 30 DomU kernels, it will pass the check, but on boot I think it will ignore
> the exceeding modules.
> 
> For that reason we need to keep track of how many “slots” are available, so in my code every time the
> read_file(…)/read_section(…) is called, the dom0less_modules_available is decremented.
> 
> If we want to get rid of one variable, we can just assume the dom0 modules and xsm-policy will be always
> loaded and we can set MAX_DOM0LESS_MODULES=(MAX_MODULES - 2 - 3) where 3 is dom0 kernel,
> dom0 ramdisk and xsm-policy. If the user doesn’t load any of them, we just lost 3 slots.
> 
> Another solution can be keeping just the dom0less_modules_available and every time loop backward in the array
> starting from that position and stopping when we have a dom0less_module_name.name == NULL so we
> know we have an available slot or we reach below zero and we know there is no space. But I think it’s not
> worthy.
> 
> So if you really want to have only one variable, I will remove space from MAX_DOM0LESS_MODULES and
> I will push it in v3.
 
You are right, let's just keep dom0less_modules_available, thanks for
the explanation.

One suggestion for future improvement (this patch series is OK as is)
would be to extend get_dom0less_file_index to also be able to search for
the dom0 kernel and ramdisk modules so that if a dom0less kernel or
ramdisk has the same filename as the dom0 kernel or ramdisk, the file
doesn't need to be loaded twice. It could be a follow-up patch after
this series gets committed.
--8323329-156143097-1632516312=:17979--


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 21:42:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 21:42:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195655.348442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTsxI-0003Cz-VE; Fri, 24 Sep 2021 21:42:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195655.348442; Fri, 24 Sep 2021 21:42:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTsxI-0003Cs-SD; Fri, 24 Sep 2021 21:42:04 +0000
Received: by outflank-mailman (input) for mailman id 195655;
 Fri, 24 Sep 2021 21:42:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTsxH-0003Cj-OM
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 21:42:03 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ed99cc6e-4b01-4baa-8dec-55881c98fb26;
 Fri, 24 Sep 2021 21:42:02 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AE7FB60F6D;
 Fri, 24 Sep 2021 21:42:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed99cc6e-4b01-4baa-8dec-55881c98fb26
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632519722;
	bh=LalBGFaYkCzugsFxqYC4kt+8k7oBL8tjeSNC7JGy6Qc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=d7SXTVa8wFJI75FLZGK7pWHQsIWZfXdRURxn+G9YGd945pUm85HUaWQoa6l8SLf65
	 AMlA9v9yzCUKLyV9VqNDoP1ndJfph1I5NpJOn2h4JCu022JP2qQ93zNls+Feq59TKk
	 jGHlurImXTMJXadQna8HXkXXrztDOrDBdpHGftJ9jDJisBAPeIHAo/cKlgpizh4fFB
	 yhAU1A4BUuNXI07yG02I+qV0W+QBXyCWfpLZLQpoxda3q6vp+2mjnLTrvYM3WHT+T2
	 mUW4LE9HYaLacN8NQKNw10LWNugtS445vWt7SKQ50Y9vHLAC5tR8PQru1zLPz/ufNv
	 XIhCgOrXv8LhQ==
Date: Fri, 24 Sep 2021 14:42:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <Rahul.Singh@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Andre Przywara <Andre.Przywara@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com>
Message-ID: <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s> <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
 <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s> <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Rahul Singh wrote:
> Hi Stefano,
> 
> > On 23 Sep 2021, at 8:12 pm, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Thu, 23 Sep 2021, Rahul Singh wrote:
> >>>> +            goto err_exit;
> >>>> +    }
> >>> 
> >>> This is unnecessary at the moment, right? Can we get rid of ops->init ?
> >> 
> >> No this is required for N1SDP board. Please check below patch.
> >> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d57547087cff4ffc078dc515d5
> > 
> > OK
> > 
> > 
> >>>> +int pci_host_common_probe(struct dt_device_node *dev, const void *data)
> >>>> +{
> >>>> +    struct pci_host_bridge *bridge;
> >>>> +    struct pci_config_window *cfg;
> >>>> +    struct pci_ecam_ops *ops;
> >>>> +    const struct dt_device_match *of_id;
> >>>> +    int err;
> >>>> +
> >>>> +    if ( dt_device_for_passthrough(dev) )
> >>>> +        return 0;
> >>>> +
> >>>> +    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
> >>>> +    ops = (struct pci_ecam_ops *) of_id->data;
> >>> 
> >>> Do we really need dt_match_node and dev->dev.of_match_table to get
> >>> dt_device_match.data?
> >>> 
> >> 
> >>> data is passed as a parameter to pci_host_common_probe, isn't it enough
> >>> to do:
> >>> 
> >>> ops = (struct pci_ecam_ops *) data;
> >> 
> >> As of now not required but in future we might need it if we implement other ecam supported bridge
> >> 
> >> static const struct dt_device_match gen_pci_dt_match[] = {                      
> >>    { .compatible = "pci-host-ecam-generic",                                    
> >>      .data =       &pci_generic_ecam_ops },
> >> 
> >>    { .compatible = "pci-host-cam-generic",
> >>      .data = &gen_pci_cfg_cam_bus_ops },                                 
> >> 
> >>    { },                                                                        
> >> };
> > 
> > Even if we add another ECAM-supported bridge, the following:
> > 
> > ops = (struct pci_ecam_ops *) data;
> > 
> > could still work, right? The probe function will directly receive as
> > parameter the .data pointer. You shouldn't need the indirection via
> > dt_match_node?
> 
> As per my understanding probe function will not get .data pointer.Probe data argument is NULL in most of the cases in XEN
> Please have a look once dt_pci_init() -> device_init(..) call flow implementation.

You are right. Looking at the code, nobody is currently using
dt_device_match.data and it is clear why: it is not passed to the
device_desc.init function at all. As it is today, it is basically
useless.

And there is only one case where device_init has a non-NULL data
parameter and it is in xen/drivers/char/arm-uart.c. All the others are
not even using the data parameter of device_init.

I think we need to change device_init so that dt_device_match.data can
be useful. Sorry for the scope-creep but I think we should do the
following:

- do not add of_match_table to struct device

- add one more parameter to device_desc.init:
  int (*init)(struct dt_device_node *dev, struct device_desc *desc, const void *data);

- change device_init to call desc->init with the right parameters:
  desc->init(dev, desc, data);

This way pci_host_common_probe is just going to get a desc directly as
parameter. I think it would make a lot more sense from an interface
perspective. It does require a change in all the DT_DEVICE_START.init
functions adding a struct device_desc *desc parameter, but it should be
a mechanical change.

Alternatively we could just change device_init to pass
device_desc.dt_match.data when the data parameter is NULL but it feels
like a hack.


What do you think?


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 22:12:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 22:12:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195661.348454 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTtQp-0007LB-Co; Fri, 24 Sep 2021 22:12:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195661.348454; Fri, 24 Sep 2021 22:12:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTtQp-0007L4-9b; Fri, 24 Sep 2021 22:12:35 +0000
Received: by outflank-mailman (input) for mailman id 195661;
 Fri, 24 Sep 2021 22:12:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTtQn-0007Ku-W8; Fri, 24 Sep 2021 22:12:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTtQn-00033O-Qi; Fri, 24 Sep 2021 22:12:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTtQn-0002rM-E1; Fri, 24 Sep 2021 22:12:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTtQn-0007oT-DK; Fri, 24 Sep 2021 22:12:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=koKciXIZj/jDJkL3M87Dt88K+Y/apYBobqjm8baE+50=; b=lGYqiQ/bJYYRvI7bDPTmjPUAHh
	r7jpZAbSS2yetk9QfxpUzPkxfgaM8h3vztManAMoqpfAi79SwtvjFOprj7hIveGf2wQmOeLX+Li1b
	bMzq3cK89ywPGZcP1LtrJLxiIRe7Zgh0bqoVGJg96GQcORcgfaJXepFNH/M7a/LDKqew=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165179-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165179: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate:fail:heisenbug
    qemu-mainline:test-amd64-i386-xl-shadow:guest-localmigrate/x10:fail:heisenbug
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=2c3e83f92d93fbab071b8a96b8ab769b01902475
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 24 Sep 2021 22:12:33 +0000

flight 165179 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165179/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-rtds     18 guest-localmigrate         fail pass in 165171
 test-amd64-i386-xl-shadow    20 guest-localmigrate/x10     fail pass in 165171

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds  20 guest-localmigrate/x10 fail in 165171 like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                2c3e83f92d93fbab071b8a96b8ab769b01902475
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   13 days
Failing since        164967  2021-09-13 13:06:52 Z   11 days   22 attempts
Testing same since   165160  2021-09-23 01:54:09 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4691 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 22:40:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 22:40:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195669.348468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTtrH-0002N4-RE; Fri, 24 Sep 2021 22:39:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195669.348468; Fri, 24 Sep 2021 22:39:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTtrH-0002Mx-Nu; Fri, 24 Sep 2021 22:39:55 +0000
Received: by outflank-mailman (input) for mailman id 195669;
 Fri, 24 Sep 2021 22:39:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTtrG-0002Mr-V9
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 22:39:55 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e651b01c-a224-4c3c-b7ab-5d05bf76eaa0;
 Fri, 24 Sep 2021 22:39:53 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id F423C60F70;
 Fri, 24 Sep 2021 22:39:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e651b01c-a224-4c3c-b7ab-5d05bf76eaa0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632523192;
	bh=6sxCqSuPSoiJCj3MKMHr6805a4ttamfAKzcvO6paPS4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kZEq7WWRUEK5rwcZHVSGxyrVb3Ondym8sUJXIgY1HABVdjBFgRLxFKeVB62hEEopO
	 /M07+hXZyy/YP6cAnmIHJ1VUrFgzXjwKXCvcGq5l5FY848inKAdzdMlaYlani0wFvy
	 avXsMoYgqFeL/bd4bgZa9Rx1QVildXPEN8dgKDHSxBFVEK5wQO4Jo7QWt8VeHuLajm
	 yURZ6iYhesF3c2G753PwCUCes4MDL7ePP1MOZVGQN9LJU/0IndyTZ/UJ+wR/rLnZ2I
	 6Boh3poKcxhWAxdSZoV3V7ugOaQO1UWD0dQ7IlVe7IpqdEtd5JVVg/C2biJM0hc8ZU
	 AnZmm+xxFAy1w==
Date: Fri, 24 Sep 2021 15:39:51 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH V3 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <1632437334-12015-3-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241508460.17979@sstabellini-ThinkPad-T480s>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com> <1632437334-12015-3-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain
>   currently.
> - The ACPI case is not covered.
> 
> ***
> 
> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> the algorithm to choose extended regions for it is different
> in comparison with the algorithm for non-direct mapped DomU.
> What is more, that extended regions should be chosen differently
> whether IOMMU is enabled or not.
> 
> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> holes found in host device-tree if otherwise. Make sure that
> extended regions are 2MB-aligned and located within maximum possible
> addressable physical memory range. The minimum size of extended
> region is 64MB. The maximum number of extended regions is 128,
> which is an artificial limit to minimize code changes (we reuse
> struct meminfo to describe extended regions, so there are an array
> field for 128 elements).
> 
> It worth mentioning that unallocated memory solution (when the IOMMU
> is disabled) will work safely until Dom0 is able to allocate memory
> outside of the original range.
> 
> Also introduce command line option to be able to globally enable or
> disable support for extended regions for Dom0 (enabled by default).
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> Please note, we need to decide which approach to use in find_unallocated_memory(),
> you can find details at:
> https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/
> 
> Changes RFC -> V2:
>    - update patch description
>    - drop uneeded "extended-region" DT property
> 
> Changes V2 -> V3:
>    - update patch description
>    - add comment for "size" calculation in add_ext_regions()
>    - clarify "end" calculation in find_unallocated_memory() and
>      find_memory_holes()
>    - only pick up regions with size >= 64MB
>    - allocate reg dynamically instead of keeping on the stack in
>      make_hypervisor_node()
>    - do not show warning for 32-bit domain
>    - drop Linux specific limits EXT_REGION_*
>    - also cover "ranges" property in find_memory_holes()
>    - add command line arg to enable/disable extended region support
> ---
>  docs/misc/xen-command-line.pandoc |   7 +
>  xen/arch/arm/domain_build.c       | 280 +++++++++++++++++++++++++++++++++++++-
>  2 files changed, 284 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 177e656..3bb8eb7 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1081,6 +1081,13 @@ hardware domain is architecture dependent.
>  Note that specifying zero as domU value means zero, while for dom0 it means
>  to use the default.
>  
> +### ext_regions (Arm)
> +> `= <boolean>`
> +
> +> Default : `true`
> +
> +Flag to globally enable or disable support for extended regions for dom0.

I'd say:

Flag to enable or disable extended regions for Dom0.

Extended regions are ranges of unused address space exposed to Dom0 as
"safe to use" for special memory mappings. Disable if your board device
tree is incomplete.


>  ### flask
>  > `= permissive | enforcing | late | disabled`
>  
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d63..81997d5 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -34,6 +34,10 @@
>  static unsigned int __initdata opt_dom0_max_vcpus;
>  integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>  
> +/* If true, the extended regions support is enabled for dom0 */
> +static bool __initdata opt_ext_regions = true;
> +boolean_param("ext_regions", opt_ext_regions);
> +
>  static u64 __initdata dom0_mem;
>  static bool __initdata dom0_mem_set;
>  
> @@ -886,6 +890,233 @@ static int __init make_memory_node(const struct domain *d,
>      return res;
>  }
>  
> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
> +{
> +    struct meminfo *ext_regions = data;
> +    paddr_t start, size;
> +
> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> +        return 0;
> +
> +    /* Both start and size of the extended region should be 2MB aligned */
> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> +    if ( start > e )
> +        return 0;
> +
> +    /*
> +     * e is actually "end-1" because it is called by rangeset functions
> +     * which are inclusive of the last address.
> +     */
> +    e += 1;
> +    size = (e - start) & ~(SZ_2M - 1);
> +    if ( size < MB(64) )
> +        return 0;
> +
> +    ext_regions->bank[ext_regions->nr_banks].start = start;
> +    ext_regions->bank[ext_regions->nr_banks].size = size;
> +    ext_regions->nr_banks++;
> +
> +    return 0;
> +}

This is a lot better


> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
> +                                          struct meminfo *ext_regions)
> +{
> +    const struct meminfo *assign_mem = &kinfo->mem;
> +    struct rangeset *unalloc_mem;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find unallocated memory for extended regions\n");
> +
> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> +    if ( !unalloc_mem )
> +        return -ENOMEM;
> +
> +    /* Start with all available RAM */
> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.mem.bank[i].start;
> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> +        res = rangeset_add_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove RAM assigned to Dom0 */
> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> +    {
> +        start = assign_mem->bank[i].start;
> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove reserved-memory regions */
> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.reserved_mem.bank[i].start;
> +        end = bootinfo.reserved_mem.bank[i].start +
> +            bootinfo.reserved_mem.bank[i].size;
> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove grant table region */
> +    start = kinfo->gnttab_start;
> +    end = kinfo->gnttab_start + kinfo->gnttab_size;
> +    res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_report_ranges(unalloc_mem, start, end,
> +                                 add_ext_regions, ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(unalloc_mem);
> +
> +    return res;
> +}
> +
> +static int __init find_memory_holes(const struct kernel_info *kinfo,
> +                                    struct meminfo *ext_regions)
> +{
> +    struct dt_device_node *np;
> +    struct rangeset *mem_holes;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find memory holes for extended regions\n");
> +
> +    mem_holes = rangeset_new(NULL, NULL, 0);
> +    if ( !mem_holes )
> +        return -ENOMEM;
> +
> +    /* Start with maximum possible addressable physical memory range */
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_add_range(mem_holes, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    /*
> +     * Remove regions described by "reg" and "ranges" properties where
> +     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
> +     */
> +    dt_for_each_device_node( dt_host, np )
> +    {
> +        unsigned int naddr;
> +        u64 addr, size;
> +
> +        naddr = dt_number_of_address(np);
> +
> +        for ( i = 0; i < naddr; i++ )
> +        {
> +            res = dt_device_get_address(np, i, &addr, &size);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> +                       i, dt_node_full_name(np));
> +                goto out;
> +            }
> +
> +            start = addr & PAGE_MASK;

PAGE_ALIGN(addr)


> +            end = PAGE_ALIGN(addr + size);
> +            res = rangeset_remove_range(mem_holes, start, end - 1);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                       start, end);
> +                goto out;
> +            }
> +        }
> +
> +        if ( dt_device_type_is_equal(np, "pci" ) )
> +        {
> +            unsigned int range_size, nr_ranges;
> +            int na, ns, pna;
> +            const __be32 *ranges;
> +            u32 len;
> +
> +            /*
> +             * Looking for non-empty ranges property which in the context
> +             * of the PCI host bridge device node describes the BARs for
> +             * the PCI devices.

I'd say that "it describes the PCI host bridge aperture"


> +             */
> +            ranges = dt_get_property(np, "ranges", &len);
> +            if ( !ranges || !len )
> +                continue;
> +
> +            pna = dt_n_addr_cells(np);
> +            na = dt_child_n_addr_cells(np);
> +            ns = dt_child_n_size_cells(np);
> +
> +            range_size = pna + na + ns;
> +
> +            nr_ranges = len / sizeof(__be32) / range_size;
> +
> +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
> +            {
> +                /* Skip the child address and get the parent (CPU) address */
> +                addr = dt_read_number(ranges + na, pna);
> +
> +                size = dt_read_number(ranges + na + pna, ns);

Parsing the PCI ranges property is never intuitive, but everything
checks out as far as I can tell


> +                start = addr & PAGE_MASK;

PAGE_ALIGN(start)


> +                end = PAGE_ALIGN(addr + size);
> +                res = rangeset_remove_range(mem_holes, start, end - 1);
> +                if ( res )
> +                {
> +                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                           start, end);
> +                    goto out;
> +                }
> +            }
> +        }
> +    }
> +
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_report_ranges(mem_holes, start, end,
> +                                 add_ext_regions,  ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(mem_holes);
> +
> +    return res;
> +}
> +
>  static int __init make_hypervisor_node(struct domain *d,
>                                         const struct kernel_info *kinfo,
>                                         int addrcells, int sizecells)
> @@ -893,11 +1124,12 @@ static int __init make_hypervisor_node(struct domain *d,
>      const char compat[] =
>          "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>          "xen,xen";
> -    __be32 reg[4];
> +    __be32 *reg, *cells;
>      gic_interrupt_t intr;
> -    __be32 *cells;
>      int res;
>      void *fdt = kinfo->fdt;
> +    struct meminfo *ext_regions;
> +    unsigned int i;
>  
>      dt_dprintk("Create hypervisor node\n");
>  
> @@ -919,12 +1151,54 @@ static int __init make_hypervisor_node(struct domain *d,
>      if ( res )
>          return res;
>  
> +    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
> +    if ( !reg )
> +        return -ENOMEM;

Instead of (NR_MEM_BANKS + 1) * 4, shouldn't it be:

(ext_regions->nr_banks + 1) * (addrcells + sizecells)

?

xzalloc_array allocates a number of elements, not a number of bytes.


> +
> +    ext_regions = xzalloc(struct meminfo);
> +    if ( !ext_regions )
> +    {
> +        xfree(reg);
> +        return -ENOMEM;
> +    }
> +
> +    if ( !opt_ext_regions )
> +        printk(XENLOG_DEBUG "The extended regions support is disabled\n");
> +    else if ( is_32bit_domain(d) )
> +        printk(XENLOG_DEBUG "The extended regions are only supported for 64-bit guest currently\n");

These checks should be before the memory allocations


> +    else
> +    {
> +        if ( !is_iommu_enabled(d) )
> +            res = find_unallocated_memory(kinfo, ext_regions);
> +        else
> +            res = find_memory_holes(kinfo, ext_regions);
> +
> +        if ( res )
> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> +    }
> +
>      /* reg 0 is grant table space */
>      cells = &reg[0];
>      dt_child_set_range(&cells, addrcells, sizecells,
>                         kinfo->gnttab_start, kinfo->gnttab_size);
> +    /* reg 1...N are extended regions */
> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
> +    {
> +        u64 start = ext_regions->bank[i].start;
> +        u64 size = ext_regions->bank[i].size;
> +
> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
> +                   i, start, start + size);
> +
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> +    }
> +
>      res = fdt_property(fdt, "reg", reg,
> -                       dt_cells_to_size(addrcells + sizecells));
> +                       dt_cells_to_size(addrcells + sizecells) *
> +                       (ext_regions->nr_banks + 1));
> +    xfree(ext_regions);
> +    xfree(reg);
> +
>      if ( res )
>          return res;
>  
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:15:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:15:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195678.348479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuP8-0007HN-Hq; Fri, 24 Sep 2021 23:14:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195678.348479; Fri, 24 Sep 2021 23:14:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuP8-0007HG-Em; Fri, 24 Sep 2021 23:14:54 +0000
Received: by outflank-mailman (input) for mailman id 195678;
 Fri, 24 Sep 2021 23:14:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTuP7-0007HA-5Q
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:14:53 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 381f411a-1d8d-11ec-bb27-12813bfff9fa;
 Fri, 24 Sep 2021 23:14:51 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7385E61250;
 Fri, 24 Sep 2021 23:14:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 381f411a-1d8d-11ec-bb27-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632525290;
	bh=CWMIq8TaRZSwls8j6/ZsjbJCxxFHzW2UK3jD1MhVyQ4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=fv0xl2DmwVtUO+5MekrXMjHXaJwSRTkVksPm4oyvI7f/uAq6MgHCzH1B8rm/Yw0fF
	 crb58dRIF7fMmFKMJMlXwTPf7F0fkXnt+epe6co7FC3P907GijbbkaVL1QBz9TAt0S
	 /TgD5oiCBCZl//Ikvu+uhHrn9dAUuXzsH1ueG90+BWL1aJv3k11RhPq4Ate9h24Jde
	 kUAPaup0oBpASPJKgPTzKbPcvqEU8+kG31B8ZZEhg1fnWvCkIfu+GevE9rZtfahVma
	 /GDfMjZjoNLVxywVe5k4atzvwO99eAoOacl6/KuZ/WD2+PrUqp7se378f8KczJHkvY
	 YEZcQQoeGQXhw==
Date: Fri, 24 Sep 2021 16:14:49 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH V3 3/3] libxl/arm: Add handling of extended regions for
 DomU
In-Reply-To: <1632437334-12015-4-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241614300.17979@sstabellini-ThinkPad-T480s>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com> <1632437334-12015-4-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain
>   currently.
> - The ACPI case is not covered.
> 
> ***
> 
> The algorithm to choose extended regions for non-direct mapped
> DomU is simpler in comparison with the algorithm for direct mapped
> Dom0. As we have a lot of unused space above 4GB, provide single
> 2MB-aligned region from the second RAM bank taking into the account
> the maximum supported guest address space size and the amount of
> memory assigned to the guest. The maximum size of the region is 128GB.
> The minimum size is 64MB.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes RFC -> V2:
>    - update patch description
>    - drop uneeded "extended-region" DT property
>    - clear reg array in finalise_ext_region() and add a TODO
> 
> Changes V2 -> V3:
>    - update patch description, comments in code
>    - only pick up regions with size >= 64MB
>    - move the region calculation to make_hypervisor_node() and drop
>      finalise_ext_region()
>    - extend the list of arguments for make_hypervisor_node()
>    - do not show warning for 32-bit domain
>    - change the region alignment from 1GB to 2MB
>    - move EXT_REGION_SIZE to public/arch-arm.h
> ---
>  tools/libs/light/libxl_arm.c  | 70 +++++++++++++++++++++++++++++++++++++++----
>  xen/include/public/arch-arm.h |  3 ++
>  2 files changed, 68 insertions(+), 5 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6..a67b68e 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -598,9 +598,17 @@ static int make_timer_node(libxl__gc *gc, void *fdt,
>      return 0;
>  }
>  
> +#define ALIGN_UP_TO_2MB(x)   (((x) + MB(2) - 1) & (~(MB(2) - 1)))
> +
>  static int make_hypervisor_node(libxl__gc *gc, void *fdt,
> -                                const libxl_version_info *vers)
> +                                const libxl_version_info *vers,
> +                                const libxl_domain_build_info *b_info,
> +                                const struct xc_dom_image *dom)
>  {
> +    uint64_t region_size = 0, region_base, ramsize, bank1size,
> +        bank1end_align, bank1end_max;
> +    uint8_t gpaddr_bits;
> +    libxl_physinfo physinfo;
>      int res;
>      gic_interrupt intr;
>  
> @@ -615,9 +623,61 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
>                                "xen,xen");
>      if (res) return res;
>  
> -    /* reg 0 is grant table space */
> -    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> -                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
> +    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
> +        LOG(DEBUG, "The extended regions are only supported for 64-bit guest currently");
> +        goto out;
> +    }
> +
> +    res = libxl_get_physinfo(CTX, &physinfo);
> +    assert(!res);
> +
> +    gpaddr_bits = physinfo.gpaddr_bits;
> +    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
> +
> +    /*
> +     * Try to allocate single 2MB-aligned extended region from the second RAM
> +     * bank (above 4GB) taking into the account the maximum supported guest
> +     * address space size and the amount of memory assigned to the guest.
> +     * As the guest memory layout is not populated yet we cannot rely on
> +     * dom->rambank_size[1], so calculate the actual size of the second bank
> +     * using "max_memkb" value.
> +     */
> +    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
> +    ramsize = b_info->max_memkb * 1024;
> +    if (ramsize <= GUEST_RAM0_SIZE)
> +        bank1size = 0;
> +    else
> +        bank1size = ramsize - GUEST_RAM0_SIZE;
> +    bank1end_align = GUEST_RAM1_BASE + ALIGN_UP_TO_2MB(bank1size);
> +
> +    if (bank1end_max <= bank1end_align) {
> +        LOG(WARN, "The extended region cannot be allocated, not enough space");
> +        goto out;
> +    }
> +
> +    if (bank1end_max - bank1end_align > GUEST_EXT_REGION_MAX_SIZE) {
> +        region_base = bank1end_max - GUEST_EXT_REGION_MAX_SIZE;
> +        region_size = GUEST_EXT_REGION_MAX_SIZE;
> +    } else {
> +        region_base = bank1end_align;
> +        region_size = bank1end_max - bank1end_align;
> +    }
> +
> +out:
> +    /*
> +     * The region 0 for grant table space must be always present. If we managed
> +     * to allocate the extended region then insert it as region 1.
> +     */
> +    if (region_size >= GUEST_EXT_REGION_MIN_SIZE) {
> +        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
> +            region_base, region_base + region_size);
> +
> +        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> +                                2, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE,
> +                                region_base, region_size);
> +    } else
> +        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
> +                                1, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
>      if (res) return res;
>  
>      /*
> @@ -963,7 +1023,7 @@ next_resize:
>          }
>  
>          FDT( make_timer_node(gc, fdt, ainfo, state->clock_frequency) );
> -        FDT( make_hypervisor_node(gc, fdt, vers) );
> +        FDT( make_hypervisor_node(gc, fdt, vers, info, dom) );
>  
>          if (info->arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART)
>              FDT( make_vpl011_uart_node(gc, fdt, ainfo, dom) );
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 6b5a5f8..df59933 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -449,6 +449,9 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_RAM_BANK_BASES   { GUEST_RAM0_BASE, GUEST_RAM1_BASE }
>  #define GUEST_RAM_BANK_SIZES   { GUEST_RAM0_SIZE, GUEST_RAM1_SIZE }
>  
> +#define GUEST_EXT_REGION_MAX_SIZE   xen_mk_ullong(0x2000000000) /* 128GB */
> +#define GUEST_EXT_REGION_MIN_SIZE   xen_mk_ullong(0x0004000000) /* 64MB */
> +
>  /* Current supported guest VCPUs */
>  #define GUEST_MAX_VCPUS 128
>  
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:27:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:27:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195685.348490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuap-0000eA-J5; Fri, 24 Sep 2021 23:26:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195685.348490; Fri, 24 Sep 2021 23:26:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuap-0000e3-G1; Fri, 24 Sep 2021 23:26:59 +0000
Received: by outflank-mailman (input) for mailman id 195685;
 Fri, 24 Sep 2021 23:26:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTuan-0000dx-R8
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:26:57 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a7276548-9b35-48be-8456-800c17153e66;
 Fri, 24 Sep 2021 23:26:56 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 602406103C;
 Fri, 24 Sep 2021 23:26:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7276548-9b35-48be-8456-800c17153e66
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632526015;
	bh=KIAc7SuFXpXIunAGF1lk0wPoQygJ7DlfFaorUwIMSvg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=SRCZzqEiiiY6dumjGCmCVwZFUPI1GBUAU8rGqN+kMCknzvvnWxwxxs27F6pysqnQr
	 eghF0WbaZJaek8abeoThdys7JRSC8lxM/oQK6MLIqTMLZpyhe1uB0tPAIzQbBgnf5H
	 dOcPNnUt9CV3t+f4Jdd84w4H0kDHm/H5/IcKionojq9TUNbUHkjRuLrVXNv79ZxDIu
	 ajEK0S2Xv9iPXP9ZiPp55oxxwTxo3d5cV3QfpPfbQv/Kg6L8ofmO6iPjVnPv779h90
	 8GSCDf2mH4j0UttaG+ML1JufGrfV6fssxGxycHLIxq/qeSFn4kcsGUpDudnqkYp6E9
	 kwigbZcXvEaQQ==
Date: Fri, 24 Sep 2021 16:26:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Rahul Singh <Rahul.Singh@arm.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Andre Przywara <Andre.Przywara@arm.com>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109241619270.17979@sstabellini-ThinkPad-T480s>
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com> <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s> <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
 <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s> <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com> <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 24 Sep 2021, Stefano Stabellini wrote:
> On Fri, 24 Sep 2021, Rahul Singh wrote:
> > Hi Stefano,
> > 
> > > On 23 Sep 2021, at 8:12 pm, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > > 
> > > On Thu, 23 Sep 2021, Rahul Singh wrote:
> > >>>> +            goto err_exit;
> > >>>> +    }
> > >>> 
> > >>> This is unnecessary at the moment, right? Can we get rid of ops->init ?
> > >> 
> > >> No this is required for N1SDP board. Please check below patch.
> > >> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d57547087cff4ffc078dc515d5
> > > 
> > > OK
> > > 
> > > 
> > >>>> +int pci_host_common_probe(struct dt_device_node *dev, const void *data)
> > >>>> +{
> > >>>> +    struct pci_host_bridge *bridge;
> > >>>> +    struct pci_config_window *cfg;
> > >>>> +    struct pci_ecam_ops *ops;
> > >>>> +    const struct dt_device_match *of_id;
> > >>>> +    int err;
> > >>>> +
> > >>>> +    if ( dt_device_for_passthrough(dev) )
> > >>>> +        return 0;
> > >>>> +
> > >>>> +    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
> > >>>> +    ops = (struct pci_ecam_ops *) of_id->data;
> > >>> 
> > >>> Do we really need dt_match_node and dev->dev.of_match_table to get
> > >>> dt_device_match.data?
> > >>> 
> > >> 
> > >>> data is passed as a parameter to pci_host_common_probe, isn't it enough
> > >>> to do:
> > >>> 
> > >>> ops = (struct pci_ecam_ops *) data;
> > >> 
> > >> As of now not required but in future we might need it if we implement other ecam supported bridge
> > >> 
> > >> static const struct dt_device_match gen_pci_dt_match[] = {                      
> > >>    { .compatible = "pci-host-ecam-generic",                                    
> > >>      .data =       &pci_generic_ecam_ops },
> > >> 
> > >>    { .compatible = "pci-host-cam-generic",
> > >>      .data = &gen_pci_cfg_cam_bus_ops },                                 
> > >> 
> > >>    { },                                                                        
> > >> };
> > > 
> > > Even if we add another ECAM-supported bridge, the following:
> > > 
> > > ops = (struct pci_ecam_ops *) data;
> > > 
> > > could still work, right? The probe function will directly receive as
> > > parameter the .data pointer. You shouldn't need the indirection via
> > > dt_match_node?
> > 
> > As per my understanding probe function will not get .data pointer.Probe data argument is NULL in most of the cases in XEN
> > Please have a look once dt_pci_init() -> device_init(..) call flow implementation.
> 
> You are right. Looking at the code, nobody is currently using
> dt_device_match.data and it is clear why: it is not passed to the
> device_desc.init function at all. As it is today, it is basically
> useless.
> 
> And there is only one case where device_init has a non-NULL data
> parameter and it is in xen/drivers/char/arm-uart.c. All the others are
> not even using the data parameter of device_init.
> 
> I think we need to change device_init so that dt_device_match.data can
> be useful. Sorry for the scope-creep but I think we should do the
> following:
> 
> - do not add of_match_table to struct device
> 
> - add one more parameter to device_desc.init:
>   int (*init)(struct dt_device_node *dev, struct device_desc *desc, const void *data);
> 
> - change device_init to call desc->init with the right parameters:
>   desc->init(dev, desc, data);
> 
> This way pci_host_common_probe is just going to get a desc directly as
> parameter. I think it would make a lot more sense from an interface
> perspective. It does require a change in all the DT_DEVICE_START.init
> functions adding a struct device_desc *desc parameter, but it should be
> a mechanical change.
> 
> Alternatively we could just change device_init to pass
> device_desc.dt_match.data when the data parameter is NULL but it feels
> like a hack.
> 
> 
> What do you think?
 

Another idea that doesn't require a device_desc.init change and also
doesn't require a change to struct device is the following:


diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index a88f20175e..1aa0ef4c1e 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -205,8 +205,7 @@ int pci_host_common_probe(struct dt_device_node *dev, const void *data)
     if ( dt_device_for_passthrough(dev) )
         return 0;
 
-    of_id = dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
-    ops = (struct pci_ecam_ops *) of_id->data;
+    ops = (struct pci_ecam_ops *) data;
 
     bridge = pci_alloc_host_bridge();
     if ( !bridge )
diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
index 6b3288d6f3..66fb843f49 100644
--- a/xen/arch/arm/pci/pci-host-generic.c
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -20,15 +20,19 @@
 #include <asm/pci.h>
 
 static const struct dt_device_match gen_pci_dt_match[] = {
-    { .compatible = "pci-host-ecam-generic",
-      .data =       &pci_generic_ecam_ops },
-
+    { .compatible = "pci-host-ecam-generic" },
     { },
 };
 
+static int pci_host_generic_probe(struct dt_device_node *dev,
+                                         const void *data)
+{
+    return pci_host_common_probe(dev, &pci_generic_ecam_ops);
+}
+
 DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
 .dt_match = gen_pci_dt_match,
-.init = pci_host_common_probe,
+.init = pci_host_generic_probe,
 DT_DEVICE_END
 
 /*


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:39:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:39:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195694.348501 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTun9-0002rc-ST; Fri, 24 Sep 2021 23:39:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195694.348501; Fri, 24 Sep 2021 23:39:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTun9-0002rV-OA; Fri, 24 Sep 2021 23:39:43 +0000
Received: by outflank-mailman (input) for mailman id 195694;
 Fri, 24 Sep 2021 23:39:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTun8-0002rP-WC
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:39:43 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b092362d-1d90-11ec-bb28-12813bfff9fa;
 Fri, 24 Sep 2021 23:39:42 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 01B3E61212;
 Fri, 24 Sep 2021 23:39:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b092362d-1d90-11ec-bb28-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632526781;
	bh=ncldt2P4bIC1q0WncDFHmYhS3OigG1VYxOsQVRu860c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GP6CsIqmUCRQzhbBD6eY7TSIOgEdziqRs+kPsMf1CwlauVFZ8Wbz55JvCANGlb9cK
	 im7cyAdNILqHOWoERJ4A1TJZG1+CLEcezGSjEP3v/15uhMKmeNsXuuab5L6QuEkT8h
	 5L0mcPffEvm0yxr2n7xhYnYntSOpligALZpgrCgbpP+YaIYkTfz/D4aS7ErLXmBbEq
	 xvg7Z5X30fQbVHf62G1xhYH2GKapaWpuIaF10A7ISYCZa0AZ3L0wRJwB+aBcTKbOSd
	 0e5JAeO/7gn0CTQnWHTH4LQWMN+eGBqp6+9M6zvIDREy/OES6zzJSKk1sjooJ5NLMS
	 DL6CWD2Nj4CBg==
Date: Fri, 24 Sep 2021 16:39:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 01/11] xen/arm: Fix dev_is_dt macro definition
In-Reply-To: <20210923125438.234162-2-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241639230.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-2-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> This macro is not currently used, but still has an error in it:
> a missing parenthesis. Fix this, so the macro is properly defined.
> 
> Fixes: 6c5d3075d97e ("xen/arm: Introduce a generic way to describe device")
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> New in v2
> ---
>  xen/include/asm-arm/device.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index 582119c31ee0..64aaa2641b7f 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -29,7 +29,7 @@ typedef struct device device_t;
>  
>  /* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
>  #define dev_is_pci(dev) ((void)(dev), 0)
> -#define dev_is_dt(dev)  ((dev->type == DEV_DT)
> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
>  
>  enum device_class
>  {
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:45:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:45:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195700.348511 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTusQ-0004F0-Ej; Fri, 24 Sep 2021 23:45:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195700.348511; Fri, 24 Sep 2021 23:45:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTusQ-0004Et-Bu; Fri, 24 Sep 2021 23:45:10 +0000
Received: by outflank-mailman (input) for mailman id 195700;
 Fri, 24 Sep 2021 23:45:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTusP-0004En-Oo
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:45:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21594702-b5bb-4bde-a12f-01b07b41f5d6;
 Fri, 24 Sep 2021 23:45:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9D52D61241;
 Fri, 24 Sep 2021 23:45:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21594702-b5bb-4bde-a12f-01b07b41f5d6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632527108;
	bh=1/4uNqIKIYjFRwLoc6ioYZaaOj/bA3bYP21j38ZPscc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rVXaVb6n4tOgPbXUqRilqePXkwq7ed0IgVVfUm0RmR1+ff95xFGUUQ/EamtYHy1RP
	 JdjEUtO/cAJ6Q/bsoKJKA3KHmfTUwCoq9QKD72/gTBDmCLGJ7eCQSug7VFqDXXqJbi
	 OnYJz+WemUjE1YBfu/F4AslOm6WNBk2J/OvcRLZq1rZYMbrC/6ts7njGVKm6qYA0Xm
	 iLnEUikdaG4JThvTWAxzhhnmtFI0CSjN/E0+8lpjqh2VAnmxJl0GaFGxIZ2lvy3R7Q
	 epBpU5KqR+pMxpIV+WNH6P3d8mBlY7Qk2LZCBbyOhVLDdK/5acEgInGYZ/NQaDA0i5
	 BZf9/NYwfKyyA==
Date: Fri, 24 Sep 2021 16:45:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 02/11] xen/arm: Add new device type for PCI
In-Reply-To: <20210923125438.234162-3-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241644570.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-3-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add new device type (DEV_PCI) to distinguish PCI devices from platform
> DT devices, so some drivers, like IOMMU, can handle PCI devices
> differently.
> 
> Also add a helper which is when given a struct device returns the
> corresponding struct pci_dev which this device is a part of.
> 
> Because of the header cross-dependencies, e.g. we need both
> struct pci_dev and struct arch_pci_dev at the same time, this cannot be
> done with an inline. Macro can be implemented, but looks scary:
> 
>  #define dev_to_pci_dev(dev) container_of((container_of((dev), \
>                         struct arch_pci_dev, dev), struct pci_dev, arch)
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Since v1:
>  - Folded new device type (DEV_PCI) into this patch.
> ---
>  xen/arch/arm/pci/pci.c       | 10 ++++++++++
>  xen/include/asm-arm/device.h |  4 ++--
>  xen/include/asm-arm/pci.h    |  7 +++++++
>  3 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index bb15edbccc90..e0420d0d86c1 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -27,6 +27,16 @@ int arch_pci_clean_pirqs(struct domain *d)
>      return 0;
>  }
>  
> +struct pci_dev *dev_to_pci(struct device *dev)
> +{
> +    struct arch_pci_dev *arch_dev;
> +
> +    ASSERT(dev->type == DEV_PCI);
> +
> +    arch_dev = container_of((dev), struct arch_pci_dev, dev);
> +    return container_of(arch_dev, struct pci_dev, arch);
> +}
> +
>  static int __init dt_pci_init(void)
>  {
>      struct dt_device_node *np;
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index 64aaa2641b7f..12de217b36b9 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -4,6 +4,7 @@
>  enum device_type
>  {
>      DEV_DT,
> +    DEV_PCI,
>  };
>  
>  struct dev_archdata {
> @@ -27,8 +28,7 @@ typedef struct device device_t;
>  
>  #include <xen/device_tree.h>
>  
> -/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
> -#define dev_is_pci(dev) ((void)(dev), 0)
> +#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
>  #define dev_is_dt(dev)  ((dev)->type == DEV_DT)
>  
>  enum device_class
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index d2728a098a11..9e366ae67e83 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -27,6 +27,13 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/*
> + * Because of the header cross-dependencies, e.g. we need both
> + * struct pci_dev and struct arch_pci_dev at the same time, this cannot be
> + * done with an inline here. Macro can be implemented, but looks scary.
> + */
> +struct pci_dev *dev_to_pci(struct device *dev);
> +
>  /* Arch-specific MSI data for vPCI. */
>  struct vpci_arch_msi {
>  };
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:48:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:48:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195706.348523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuvH-0005C7-UM; Fri, 24 Sep 2021 23:48:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195706.348523; Fri, 24 Sep 2021 23:48:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuvH-0005C0-Qm; Fri, 24 Sep 2021 23:48:07 +0000
Received: by outflank-mailman (input) for mailman id 195706;
 Fri, 24 Sep 2021 23:48:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTuvG-0005Bu-8z
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:48:06 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id dc11bab1-1d91-11ec-bb28-12813bfff9fa;
 Fri, 24 Sep 2021 23:48:05 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5746C61212;
 Fri, 24 Sep 2021 23:48:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc11bab1-1d91-11ec-bb28-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632527284;
	bh=g9p3Ou62KpnsjpUew8yvdKULg04zqDYk7HMoNoeDW3M=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DZ7W84FbVoriI4A4725qQ1bnz9b8/DJ5X8AiYBh3cnJIiNG9ci0YU8q3vOdw70+V+
	 A9kUKJQihBGQ2+s0tyOXALjisj3/5sZPwoZVr5We9ewhkBMSfL/nQZsbuS8nLXNEpt
	 WNy5mQ8+4nMQmBSVBLJASrj2yMeVDWqLuM5FcY/Cd+Cb3iolrzsYIVoUlG/+gvnsBn
	 40J5rIhwIyJz8nJ6VIgSH5PASU8OiA8ky9VsDzB8A/QMHw0hR/4nodHqNsvtwEmg85
	 O8n6YcrodH9dYKupp6Y+f2TctI0E5F/r48IylA5qrE2ERu9wB7Bxi7wf4oxKnQijef
	 FpniECVRSgcXw==
Date: Fri, 24 Sep 2021 16:48:03 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 03/11] xen/arm: Introduce pci_find_host_bridge_node
 helper
In-Reply-To: <20210923125438.234162-4-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241647510.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-4-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Get host bridge node given a PCI device attached to it.
> 
> This helper will be re-used for adding PCI devices by the subsequent
> patches.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/arch/arm/pci/pci-host-common.c | 17 +++++++++++++++++
>  xen/include/asm-arm/pci.h          |  1 +
>  2 files changed, 18 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index a88f20175ea9..1567b6e2956c 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -283,6 +283,23 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
>      return -EINVAL;
>  }
>  
> +/*
> + * Get host bridge node given a device attached to it.
> + */
> +struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct pci_dev *pdev = dev_to_pci(dev);
> +
> +    bridge = pci_find_host_bridge(pdev->seg, pdev->bus);
> +    if ( unlikely(!bridge) )
> +    {
> +        printk(XENLOG_ERR "Unable to find PCI bridge for "PRI_pci"\n",
> +               pdev->seg, pdev->bus, pdev->sbdf.dev, pdev->sbdf.fn);
> +        return NULL;
> +    }
> +    return bridge->dt_node;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 9e366ae67e83..5b100556225e 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -103,6 +103,7 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>  struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
>  int pci_get_host_bridge_segment(const struct dt_device_node *node,
>                                  uint16_t *segment);
> +struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
>  
>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:51:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:51:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195712.348533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuyW-0006YG-DL; Fri, 24 Sep 2021 23:51:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195712.348533; Fri, 24 Sep 2021 23:51:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTuyW-0006Y9-AK; Fri, 24 Sep 2021 23:51:28 +0000
Received: by outflank-mailman (input) for mailman id 195712;
 Fri, 24 Sep 2021 23:51:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTuyU-0006Y3-M6
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:51:26 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 81b02602-c84c-472e-8247-e0e2a2d6a9ea;
 Fri, 24 Sep 2021 23:51:25 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D43D161250;
 Fri, 24 Sep 2021 23:51:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81b02602-c84c-472e-8247-e0e2a2d6a9ea
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632527485;
	bh=Ack5/UqmxYXHPfI3KCJ5he7bOUsz5tmrXD9+cyyP6tg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NInlFZupMSFB4HG+w5qSVF5rQ4onuU73MBj6w5rFPjvyE4jzXkkaUwntDMVowlvOC
	 h8F+5UcZCfi3KD+uWQoFrWL1Il94BCebrsw9wiybtcRfGJFhP4IHiJgBPRGL+20TBW
	 Im2IYifGbjiTS0sYvw18EeXBgmfsfqTymg40h6DEPRoYPgg/JMPh54WFR30TPakkRo
	 Zodw4MTDX9sB2zhuSZ3aQEXvmjJiBtnA+causraesMyCM6m/7YM8XJuBiC0j8TBCoA
	 REwhUrqwaMEhc7nV21szgkGkA2whu2DDtK4Osh2PTKLZBjrz7AseMJ4zzAAfRMAYQD
	 RHXmExOjU3oaw==
Date: Fri, 24 Sep 2021 16:51:23 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 04/11] xen/device-tree: Make dt_find_node_by_phandle
 global
In-Reply-To: <20210923125438.234162-5-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241649000.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-5-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Make dt_find_node_by_phandle globally visible, so it can be re-used by
> other frameworks.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/common/device_tree.c      | 2 +-
>  xen/include/xen/device_tree.h | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index ea93da1725f6..4aae281e89bf 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -1047,7 +1047,7 @@ int dt_for_each_range(const struct dt_device_node *dev,
>   *
>   * Returns a node pointer.
>   */
> -static struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
> +struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
>  {
>      struct dt_device_node *np;
>  
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 9069040ef7f7..3334048d3bb5 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -850,6 +850,8 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
>   */
>  int dt_get_pci_domain_nr(struct dt_device_node *node);
>  
> +struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle);
> +
>  #ifdef CONFIG_DEVICE_TREE_DEBUG
>  #define dt_dprintk(fmt, args...)  \
>      printk(XENLOG_DEBUG fmt, ## args)
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:54:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:54:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195718.348545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv1J-0007C9-Sj; Fri, 24 Sep 2021 23:54:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195718.348545; Fri, 24 Sep 2021 23:54:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv1J-0007C2-Od; Fri, 24 Sep 2021 23:54:21 +0000
Received: by outflank-mailman (input) for mailman id 195718;
 Fri, 24 Sep 2021 23:54:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTv1I-0007Bw-8E
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:54:20 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bb9708de-1d92-11ec-bb29-12813bfff9fa;
 Fri, 24 Sep 2021 23:54:19 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 66C0160E75;
 Fri, 24 Sep 2021 23:54:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb9708de-1d92-11ec-bb29-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632527658;
	bh=TRAlP3p69GeUfeKyiaMW8KGao8NRc+06mZvJs8GApSo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EtWc/wN8cYfZL2s1zzsy0R5YUguLdcySb5a4xXHSZPSzCGOYcg6SI3M/UVtJkAr+w
	 1OJmUwjuC3UKkWtasHjC7w+aBzEOShiyE/gym66pRvGCeWnK79ZuLWai4UOzfOVDRv
	 6uMUyvX+1L1qyk0U7Igc6LiCjzN6ej/icljIP25oKB7Axmr9MHZnhh8y27liOQ3d6v
	 H8X0NFam3at4jgOnyV+xRkl0XU4i+mSJ1PF0fIDm+KPemG+EayNWPlAVYaf3cP2C+A
	 P4Wxp93q8ZcKgF9LpbdZFwmFiJOklJTW5piHDyGosYn+rkeCBhzKqTejtKdrschRTv
	 EUOEr4jk69nRg==
Date: Fri, 24 Sep 2021 16:54:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating
 one
In-Reply-To: <20210923125438.234162-6-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241653510.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-6-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> While adding a PCI device mark it as such, so other frameworks
> can distinguish it form DT devices.
                     ^ from

> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Since v1:
>  - Moved the assignment from iommu_add_device to alloc_pdev
> ---
>  xen/drivers/passthrough/pci.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index 633e89ac1311..fc3469bc12dc 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>      *((u8*) &pdev->bus) = bus;
>      *((u8*) &pdev->devfn) = devfn;
>      pdev->domain = NULL;
> +#ifdef CONFIG_ARM
> +    pci_to_dev(pdev)->type = DEV_PCI;
> +#endif
>  
>      rc = pdev_msi_init(pdev);
>      if ( rc )
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:57:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:57:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195725.348555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv44-00089d-8v; Fri, 24 Sep 2021 23:57:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195725.348555; Fri, 24 Sep 2021 23:57:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv44-00089W-5v; Fri, 24 Sep 2021 23:57:12 +0000
Received: by outflank-mailman (input) for mailman id 195725;
 Fri, 24 Sep 2021 23:57:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UhVx=OO=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTv42-00089P-U9
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:57:10 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 21432ca8-1d93-11ec-bb2c-12813bfff9fa;
 Fri, 24 Sep 2021 23:57:10 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E664660FDC;
 Fri, 24 Sep 2021 23:57:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21432ca8-1d93-11ec-bb2c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632527829;
	bh=ilQoD3kE5tQqzSiX3Ozc9q439DsoUQqqLwiQIgVnB5M=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Ea7GqnfPPFx7h/uBvjdpbEAM5CMSDZBCIHId5vUYsc+w/okdwy9zEU9yXzb4L5lQm
	 /Duf5JgFee8jRQa9WLudXdhVSIvGjH1rPcpbCUDlSU7JGozeO2Zp05Cd0xMW0rnous
	 GZhvgHibmCPhbruBO7WnZGggdjHBSu6QLkOzL+5dO8srHeB7+6W2Q3+Yk9QMAcyc6a
	 FUMYUL4wlpRiXxMaEDPtpbJ6AaYmdVvO9u5zyBRL/6KJEKVDCouKZgAbo4GroYD8ic
	 YlGEgomLe/QDoTucd2T9WnfcOFIj3/s9mwe7e3bp1bTTcwCPZy8J+7SXuFJQvt/qwm
	 8FQZVkYtaqlgw==
Date: Fri, 24 Sep 2021 16:57:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com
Subject: Re: [PATCH v2 06/11] xen/domain: Call pci_release_devices() when
 releasing domain resources
In-Reply-To: <20210923125438.234162-7-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241656430.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-7-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> This is the very same that we already do for DT devices. Moreover, x86
> already calls pci_release_devices().
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Since v1:
>  - re-wording in the commit message
> ---
>  xen/arch/arm/domain.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index f7ed130023d5..854e8fed0393 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -985,7 +985,8 @@ static int relinquish_memory(struct domain *d, struct page_list_head *list)
>   * function which may return -ERESTART.
>   */
>  enum {
> -    PROG_tee = 1,
> +    PROG_pci = 1,
> +    PROG_tee,
>      PROG_xen,
>      PROG_page,
>      PROG_mapping,
> @@ -1022,6 +1023,12 @@ int domain_relinquish_resources(struct domain *d)
>  #ifdef CONFIG_IOREQ_SERVER
>          ioreq_server_destroy_all(d);
>  #endif
> +#ifdef CONFIG_HAS_PCI
> +    PROGRESS(pci):
> +        ret = pci_release_devices(d);
> +        if ( ret )
> +            return ret;
> +#endif
>  
>      PROGRESS(tee):
>          ret = tee_relinquish_resources(d);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Sep 24 23:59:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 24 Sep 2021 23:59:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195730.348567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv6J-0000NA-MC; Fri, 24 Sep 2021 23:59:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195730.348567; Fri, 24 Sep 2021 23:59:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv6J-0000N3-Io; Fri, 24 Sep 2021 23:59:31 +0000
Received: by outflank-mailman (input) for mailman id 195730;
 Fri, 24 Sep 2021 23:59:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=03IS=OO=invisiblethingslab.com=marmarek@srs-us1.protection.inumbo.net>)
 id 1mTv6I-0000Mv-Gm
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 23:59:30 +0000
Received: from new1-smtp.messagingengine.com (unknown [66.111.4.221])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f726a939-066d-47d4-b114-0ea74bd941e5;
 Fri, 24 Sep 2021 23:59:29 +0000 (UTC)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailnew.nyi.internal (Postfix) with ESMTP id 4251A580E9D;
 Fri, 24 Sep 2021 19:59:29 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute3.internal (MEProxy); Fri, 24 Sep 2021 19:59:29 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 24 Sep 2021 19:59:26 -0400 (EDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f726a939-066d-47d4-b114-0ea74bd941e5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:content-type:date:from:in-reply-to
	:message-id:mime-version:references:subject:to:x-me-proxy
	:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=D3+aV5
	pdnNgiVBKhSk9fIgGRiazPjnt4ThMKaQG4k6Y=; b=SpyX4SqT3VyudFwA/TLXma
	xC6kmBiBViHTKXLD6n2o9iABbnceZyRegDybfOXiazh3LawiM2lPS2oxcgqyaYTH
	akHqSgmeG6AVcR96mxERYS2DlbpOrkg0YLwGposQIm4u04sk4uEMBkmS3HcYrP9G
	ZYAT5c670xxVbSi0SGq+DrfOma/RD0rFWFOpvuRDjja+yqb12SNxLIx9bpVwG7h/
	gaII18IwQKs8PwOnjVnzVPgs0moy1B3+kjdxMO7WYWwUeSzrPqjxEjKfC7qsmAyS
	cqgSGHb1dnI+ANP8FrHqGtKw+aJwM+LRjbJ+Wcbv4qkzhl/zb6NEadwvZY96GM/Q
	==
X-ME-Sender: <xms:YGZOYSY9873HBPvfzRw_c-fmno2y3AZFzqYvjhAxKHd_NOa-iXi5-A>
    <xme:YGZOYVZJfwSo12W8Jy9r_Mv1yQj9c6MARwAxAZKqEo9SwfkLJs-tpgGD-_siLGkJH
    z9vujofXMmHkA>
X-ME-Received: <xmr:YGZOYc_VvWynAvgFu3kstJImC6Mro2NFT9n21AROOQRoLbBfHcIpj5IB3phD61eNhNL122eb-PEH1-bb0Z1NZVOOyTBGujfi>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudejvddgvdekucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeegudfg
    hfegfeeigfegtdetgefghfekgfeihfduhefhleeuvddvvdetiedvudeltdenucffohhmrg
    hinheprghtlhgrshhsihgrnhdrnhgvthdpgigvnhhprhhojhgvtghtrdhorhhgnecuvehl
    uhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvg
    hksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:YGZOYUpvo7zhNFwpLBgm1J9Ps-U-CeeZbEts9bZU9mVxU22St44i_w>
    <xmx:YGZOYdqiDihTu2dpOsRxCqbIkElRP1kknFVT3hgQx-d5E4icgjVKug>
    <xmx:YGZOYSTPaeKkwav6FQaPJhCh__vo-MD2RXc0urhdVknKYOXW6vodmg>
    <xmx:YWZOYYCVNzoM6m7c_ew43nGtiGsDvZ7cFQnSzfiiePWnX7k1577plQ>
Date: Sat, 25 Sep 2021 01:59:23 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
Cc: Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	Mike Holmes <mike.holmes@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Peter Griffin <peter.griffin@linaro.org>,
	xen-devel@lists.xenproject.org, wl@xen.org,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>,
	Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Rust-VMM Mailing List <rust-vmm@lists.opendev.org>,
	Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>,
	David Woodhouse <dwmw2@infradead.org>
Subject: Re: Xen Rust VirtIO demos work breakdown for Project Stratos
Message-ID: <YU5mW396S04IsCBr@mail-itl>
References: <87pmsylywy.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="AHnRrpNF9QyKNyZE"
Content-Disposition: inline
In-Reply-To: <87pmsylywy.fsf@linaro.org>


--AHnRrpNF9QyKNyZE
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sat, 25 Sep 2021 01:59:23 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
Cc: Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
	Mike Holmes <mike.holmes@linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Peter Griffin <peter.griffin@linaro.org>,
	xen-devel@lists.xenproject.org, wl@xen.org,
	Artem Mygaiev <Artem_Mygaiev@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Doug Goldstein <cardoe@cardoe.com>,
	Oleksandr Tyshchenko <olekstysh@gmail.com>,
	Rust-VMM Mailing List <rust-vmm@lists.opendev.org>,
	Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>,
	David Woodhouse <dwmw2@infradead.org>
Subject: Re: Xen Rust VirtIO demos work breakdown for Project Stratos

On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrote:
> Hi,

Hi,

> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
> =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80
>=20
>   Currently the foreign memory mapping support only works for dom0 due
>   to reference counting issues. If we are to support backends running in
>   their own domains this will need to get fixed.
>=20
>   Estimate: 8w
>=20
>=20
> [STR-57] <https://linaro.atlassian.net/browse/STR-57>

I'm pretty sure it was discussed before, but I can't find relevant
(part of) thread right now: does your model assumes the backend (running
outside of dom0) will gain ability to map (or access in other way)
_arbitrary_ memory page of a frontend domain? Or worse: any domain?
That is a significant regression in terms of security model Xen
provides. It would give the backend domain _a lot more_ control over the
system that it normally has with Xen PV drivers - negating significant
part of security benefits of using driver domains.

So, does the above require frontend agreeing (explicitly or implicitly)
for accessing specific pages by the backend? There were several
approaches to that discussed, including using grant tables (as PV
drivers do), vIOMMU(?), or even drastically different model with no
shared memory at all (Argo). Can you clarify which (if any) approach
your attempt of VirtIO on Xen will use?

A more general idea: can we collect info on various VirtIO on Xen
approaches (since there is more than one) in a single place, including:
 - key characteristics, differences
 - who is involved
 - status
 - links to relevant threads, maybe

I'd propose to revive https://wiki.xenproject.org/wiki/Virtio_On_Xen

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--AHnRrpNF9QyKNyZE
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmFOZlsACgkQ24/THMrX
1yznCgf+PxWtKZFIkMoii7nrqq1zsVOOz1nD0JyDJ6nL66bRTK2L2IH4kZkZaCZF
hSwGSjPU2ulafdtyDwNbMkezDqREjfQ2QfNwZ9s/d/J7i+kujFlWrbNyf8EOvADy
BFTafXlE6wv0BwLHu5qNx9QrJ1rhK1pFFqWXr8+bgTlkyJ1XdYg6QCbK8+Z2lGFD
IMhLIfUOkzYJzCjSgQrDUh0GIejgEs5XtZGJkJOg+5oto3sGk941Ypf1f0137HE+
KQEi8PB949kdDq6AiSQ6WohylBu4crSYqDjKSqB8mdmC06JWEzDumfPsx18TU/TM
JWvdlcY04hj37zcN+MEmA1La+B/Rbw==
=hcgf
-----END PGP SIGNATURE-----

--AHnRrpNF9QyKNyZE--


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 00:00:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 00:00:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195736.348578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv6q-0001t6-VX; Sat, 25 Sep 2021 00:00:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195736.348578; Sat, 25 Sep 2021 00:00:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTv6q-0001sm-Rl; Sat, 25 Sep 2021 00:00:04 +0000
Received: by outflank-mailman (input) for mailman id 195736;
 Sat, 25 Sep 2021 00:00:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTv6p-0001e9-DK
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 00:00:03 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 87bab42e-1d93-11ec-bb2c-12813bfff9fa;
 Sat, 25 Sep 2021 00:00:02 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B156060FDC;
 Sat, 25 Sep 2021 00:00:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87bab42e-1d93-11ec-bb2c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632528001;
	bh=EKfV/yiH0BiCtSeksLFvBycTxpbIEpQuEjnB3dB/SZc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=k/0c6mrmVD+UX/KMUwkIADgnVwtolbmTF08NMHErewJvHipX5BV+3pzfMmv7A01LJ
	 m/W3X2A4g5y8gHfiP/lLYKHw7+Yoc7fjOEYC7ixBniAlfFsgvWnWdmYF2WCx0kH5T3
	 iuc7XPtXzaqBMB3vVXXzZ3+J8W3N5lzkMKMHi8JQL9j6/iJrsmturyffV1l5YDRARj
	 dbQ3V+Hmvoox7dWn5Sx7BscQ7LwVRq3GBMnSNnbgoZ50JBzQ6c1F87NfIJB+DtynuR
	 dxS+iEhTFb/5sjo16dB0MImaUzE8rk1q83/l6brHoyfby+LJnk+06/32JyHAYrs7tz
	 N3laaS8fJO49Q==
Date: Fri, 24 Sep 2021 17:00:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Juergen Gross <jgross@suse.com>, 
    jbeulich@suse.com, roger.pau@citrix.com, andrew.cooper3@citrix.com
Subject: Re: [PATCH v2 07/11] libxl: Allow removing PCI devices for all types
 of domains
In-Reply-To: <20210923125438.234162-8-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241658350.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-8-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+x86 maintainers

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> The PCI device remove path may now be used by PVH on ARM, so the
> assert is no longer valid.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Ian Jackson <iwj@xenproject.org>
> Cc: Juergen Gross <jgross@suse.com>
>
> ---
>  tools/libs/light/libxl_pci.c | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> index 1a1c2630803b..59f3686fc85e 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1947,8 +1947,6 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
>              goto out_fail;
>          }
>      } else {
> -        assert(type == LIBXL_DOMAIN_TYPE_PV);

This is fine for ARM, but is it OK from an x86 point of view considering
the PVH implications?


>          char *sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", pci->domain,
>                                       pci->bus, pci->dev, pci->func);
>          FILE *f = fopen(sysfs_path, "r");
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 00:06:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 00:06:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195743.348589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvCj-00031u-Mm; Sat, 25 Sep 2021 00:06:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195743.348589; Sat, 25 Sep 2021 00:06:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvCj-00031n-JT; Sat, 25 Sep 2021 00:06:09 +0000
Received: by outflank-mailman (input) for mailman id 195743;
 Sat, 25 Sep 2021 00:06:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTvCi-00031h-Tp
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 00:06:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1f88de43-d7bf-4c9b-a069-aa7bc68edc69;
 Sat, 25 Sep 2021 00:06:07 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B1FB060FDC;
 Sat, 25 Sep 2021 00:06:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f88de43-d7bf-4c9b-a069-aa7bc68edc69
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632528367;
	bh=8Kr3nfOHfayj52scmf+IKB3qa8sEIJ3rRqab05Wiji0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=AUe6sH52GD2gU751uaTSiRa7XFw2VeJW+PdwmUzlLddvXRH0nayiADPFkzCaQWAUE
	 2OiNlOMOwhZaFqgI999M0mxTF5pwVbWFZICR2YuRJKjLl2iI+ki2IkW3tSDm+p7PdV
	 lz3NrfnNtpsOXf/Gf83nC6ZUcsN5cRDJu47fw/MfY78+K1BZ1pXHB+tQiv5B1LCFY0
	 AW+vnnKSlybOmuzG0gU88kcBE5982sEeoLdtlyEdNFQ+CHbjn/REPltKWD+RTElz9o
	 G1Zu+aBTFTxOQfbc+9stYWijofG7Z4DwreP9awOCwkzSb8la2RxtsYxYkf5SAfldJT
	 mDGMAvHizrkWQ==
Date: Fri, 24 Sep 2021 17:06:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Juergen Gross <jgross@suse.com>, 
    jbeulich@suse.com, roger.pau@citrix.com, andrew.cooper3@citrix.com
Subject: Re: [PATCH v2 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
In-Reply-To: <20210923125438.234162-9-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241704050.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-9-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Arm's PCI passthrough implementation doesn't support legacy interrupts,
> but MSI/MSI-X. This can be the case for other platforms too.
> For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
> it to the CFLAGS and compile the relevant code in the toolstack only if
> applicable.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Cc: Ian Jackson <iwj@xenproject.org>
> Cc: Juergen Gross <jgross@suse.com>
>
> ---
> Since v1:
>  - Minimized #idefery by introducing pci_supp_legacy_irq function
>    for relevant checks
> ---
>  tools/libs/light/Makefile    |  4 ++++
>  tools/libs/light/libxl_pci.c | 13 +++++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
> index 7d8c51d49242..bd3f6be2a183 100644
> --- a/tools/libs/light/Makefile
> +++ b/tools/libs/light/Makefile
> @@ -46,6 +46,10 @@ CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
>  	-Wno-declaration-after-statement -Wformat-nonliteral
>  CFLAGS += -I.
>  
> +ifeq ($(CONFIG_X86),y)
> +CFLAGS += -DCONFIG_PCI_SUPP_LEGACY_IRQ
> +endif

This patch is a lot better than the previous version, thanks!

I think the usage of pci_supp_legacy_irq below is good and we can't do
better than that.

I wonder if there is a better way than the above to export
CONFIG_PCI_SUPP_LEGACY_IRQ. Suggestions?


>  SRCS-$(CONFIG_X86) += libxl_cpuid.c
>  SRCS-$(CONFIG_X86) += libxl_x86.c
>  SRCS-$(CONFIG_X86) += libxl_psr.c
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> index 59f3686fc85e..4c2d7aeefbb2 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1364,6 +1364,15 @@ static void pci_add_timeout(libxl__egc *egc, libxl__ev_time *ev,
>      pci_add_dm_done(egc, pas, rc);
>  }
>  
> +static bool pci_supp_legacy_irq(void)
> +{
> +#ifdef CONFIG_PCI_SUPP_LEGACY_IRQ
> +    return true;
> +#else
> +    return false;
> +#endif
> +}
> +
>  static void pci_add_dm_done(libxl__egc *egc,
>                              pci_add_state *pas,
>                              int rc)
> @@ -1434,6 +1443,8 @@ static void pci_add_dm_done(libxl__egc *egc,
>          }
>      }
>      fclose(f);
> +    if (!pci_supp_legacy_irq())
> +        goto out_no_irq;
>      sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                  pci->bus, pci->dev, pci->func);
>      f = fopen(sysfs_path, "r");
> @@ -1983,6 +1994,8 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
>          }
>          fclose(f);
>  skip1:
> +        if (!pci_supp_legacy_irq())
> +            goto skip_irq;
>          sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                 pci->bus, pci->dev, pci->func);
>          f = fopen(sysfs_path, "r");
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 00:18:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 00:18:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195753.348600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvOl-0005Br-Qz; Sat, 25 Sep 2021 00:18:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195753.348600; Sat, 25 Sep 2021 00:18:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvOl-0005Bk-O1; Sat, 25 Sep 2021 00:18:35 +0000
Received: by outflank-mailman (input) for mailman id 195753;
 Sat, 25 Sep 2021 00:18:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTvOk-0005Be-5v
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 00:18:34 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1dcb2564-1d96-11ec-bb2c-12813bfff9fa;
 Sat, 25 Sep 2021 00:18:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9D512610C9;
 Sat, 25 Sep 2021 00:18:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1dcb2564-1d96-11ec-bb2c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632529112;
	bh=9Dcp95dj4NZN6nzRAgoBLdmQxsw4wAIBVUfXvr5Deec=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HSKJAY90h6Lo1QG5Oi2y7syJoaWiTAAOMLB1Xj+gaYeMxyh+iQdiq9cAPpgHj6QP0
	 FmwEKJmJSE2ht4ESOa1nfiCsXUkyAHuouogQZahBfAfDs/m/BNcbDNlF+Wh6SYM0dI
	 zTYKcM0uYLiA2tHaNZWXiHQMM9qPjV9CMy4EQxTUkMxVhMe6WV7E5d8ZXXo19DVok0
	 UfdKF0YrX5yKmXsP18t7i65JjPZaaFo5W3kAYft/SzY1L04NQY0ymyobsOaw2bXR6m
	 SVpGCSoFWcCJXX9GXBl9uU0+ViPXvNj85yKOFkzlIrg/k3s+PlLMGsHK7AX1aaelK8
	 9jqGr+zRq9Bvw==
Date: Fri, 24 Sep 2021 17:18:30 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
In-Reply-To: <20210923125438.234162-10-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241713160.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-10-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> In order for vPCI to work it needs to maintain guest and hardware
> domain's views on the configuration space. For example, BARs and
                 ^ of

> COMMAND registers require emulation for guests and the guest view
> on the registers needs to be in sync with the real contents of the
  ^ of

> relevant registers. For that ECAM address space needs to also be
> trapped for the hardware domain, so we need to implement PCI host
> bridge specific callbacks to properly setup MMIO handlers for those
> ranges depending on particular host bridge implementation.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

The patch looks pretty good, only a couple of minor comments below


> ---
> Since v1:
>  - Dynamically calculate the number of MMIO handlers required for vPCI
>    and update the total number accordingly
>  - s/clb/cb
>  - Do not introduce a new callback for MMIO handler setup
> ---
>  xen/arch/arm/domain.c              |  2 ++
>  xen/arch/arm/pci/pci-host-common.c | 28 +++++++++++++++++++++++++
>  xen/arch/arm/vpci.c                | 33 ++++++++++++++++++++++++++++++
>  xen/arch/arm/vpci.h                |  6 ++++++
>  xen/include/asm-arm/pci.h          |  7 +++++++
>  5 files changed, 76 insertions(+)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 854e8fed0393..c7b25bc70439 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -733,6 +733,8 @@ int arch_domain_create(struct domain *d,
>      if ( (rc = domain_vgic_register(d, &count)) != 0 )
>          goto fail;
>  
> +    count += domain_vpci_get_num_mmio_handlers(d);
> +
>      if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 )
>          goto fail;
>  
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 1567b6e2956c..155f2a2743af 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -300,6 +300,34 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
>      }
>      return bridge->dt_node;
>  }
> +
> +int pci_host_iterate_bridges(struct domain *d,
> +                             int (*cb)(struct domain *d,
> +                                       struct pci_host_bridge *bridge))
> +{
> +    struct pci_host_bridge *bridge;
> +    int err;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        err = cb(d, bridge);
> +        if ( err )
> +            return err;
> +    }
> +    return 0;
> +}
> +
> +int pci_host_get_num_bridges(void)
> +{
> +    struct pci_host_bridge *bridge;
> +    int count = 0;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +        count++;
> +
> +    return count;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 76c12b92814f..14947e975d69 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -80,17 +80,50 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
>      .write = vpci_mmio_write,
>  };
>  
> +static int vpci_setup_mmio_handler(struct domain *d,
> +                                   struct pci_host_bridge *bridge)
> +{
> +    struct pci_config_window *cfg = bridge->cfg;
> +
> +    register_mmio_handler(d, &vpci_mmio_handler,
> +                          cfg->phys_addr, cfg->size, NULL);
> +    return 0;
> +}
> +
>  int domain_vpci_init(struct domain *d)
>  {
>      if ( !has_vpci(d) )
>          return 0;
>  
> +    if ( is_hardware_domain(d) )
> +        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
> +
> +    /* Guest domains use what is programmed in their device tree. */
>      register_mmio_handler(d, &vpci_mmio_handler,
>                            GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
>  
>      return 0;
>  }
>  
> +int domain_vpci_get_num_mmio_handlers(struct domain *d)
> +{
> +    int count = 0;
> +
> +    if ( is_hardware_domain(d) )
> +        /* For each PCI host bridge's configuration space. */
> +        count += pci_host_get_num_bridges();

NIT: why += instead of = ?


> +    else
> +        /*
> +         * VPCI_MSIX_MEM_NUM handlers for MSI-X tables per each PCI device
> +         * being passed through. Maximum number of supported devices
> +         * is 32 as virtual bus topology emulates the devices as embedded
> +         * endpoints.
> +         * +1 for a single emulated host bridge's configuration space. */
> +        count = VPCI_MSIX_MEM_NUM * 32 + 1;
> +
> +    return count;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> index d8a7b0e3e802..27a2b069abd2 100644
> --- a/xen/arch/arm/vpci.h
> +++ b/xen/arch/arm/vpci.h
> @@ -17,11 +17,17 @@
>  
>  #ifdef CONFIG_HAS_VPCI
>  int domain_vpci_init(struct domain *d);
> +int domain_vpci_get_num_mmio_handlers(struct domain *d);
>  #else
>  static inline int domain_vpci_init(struct domain *d)
>  {
>      return 0;
>  }
> +
> +static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
> +{
> +    return 0;
> +}
>  #endif
>  
>  #endif /* __ARCH_ARM_VPCI_H__ */
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 5b100556225e..7618f0b6725b 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -15,6 +15,8 @@
>  #ifndef __ARM_PCI_H__
>  #define __ARM_PCI_H__
>  
> +#include <asm/mmio.h>

This should be no longer required.


>  #ifdef CONFIG_HAS_PCI
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> @@ -109,6 +111,11 @@ static always_inline bool is_pci_passthrough_enabled(void)
>  {
>      return !!pci_passthrough_enabled;
>  }
> +
> +int pci_host_iterate_bridges(struct domain *d,
> +                             int (*clb)(struct domain *d,
> +                                        struct pci_host_bridge *bridge));
> +int pci_host_get_num_bridges(void);
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  #define pci_passthrough_enabled (false)



From xen-devel-bounces@lists.xenproject.org Sat Sep 25 00:44:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 00:44:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195759.348610 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvnk-0000UG-U1; Sat, 25 Sep 2021 00:44:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195759.348610; Sat, 25 Sep 2021 00:44:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTvnk-0000U9-R6; Sat, 25 Sep 2021 00:44:24 +0000
Received: by outflank-mailman (input) for mailman id 195759;
 Sat, 25 Sep 2021 00:44:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTvnj-0000U3-Lv
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 00:44:23 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b9422cba-1d99-11ec-bb2d-12813bfff9fa;
 Sat, 25 Sep 2021 00:44:22 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 096A06103C;
 Sat, 25 Sep 2021 00:44:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9422cba-1d99-11ec-bb2d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632530661;
	bh=O1EQP2c2OI+3X3OdV5SNnC1A2WhVwNq51UxFehtZ2zE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jDu0O6sz261Ro83OJNEeKdcYAt8S3JyKzXAZkKb1YzMdJQLTAg6Gt7J1TnHE9Gkgq
	 k/WYb0MWegkU6Mlgb3yiWBGeBgC0qWjVqjkBsCj9LTDy8ibLvso3kYzXM1SqfpMHvd
	 dFOl5pqXZEcH0rI9UbHdTjq2X9ebMcfsmRWvpp84ntZ0HcBjP2keXPlQ3BSf5Y0LwK
	 5W3Ly5Eo/F1rnWfacO3D8Og/03TbxkzSFIgtMQlBQy0lFviKwo2c6NVBlKGW83yzwR
	 pCbG6SvAIBXsof683ljmBnR8zY2yhQv3VUA41cDFx2Sx7TovdZgZRi/pKdojrvzsg7
	 jYK5uMPGVxk3g==
Date: Fri, 24 Sep 2021 17:44:20 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
In-Reply-To: <20210923125438.234162-11-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-11-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> PCI host bridges are special devices in terms of implementing PCI
> passthrough. According to [1] the current implementation depends on
> Domain-0 to perform the initialization of the relevant PCI host
> bridge hardware and perform PCI device enumeration. In order to
> achieve that one of the required changes is to not map all the memory
> ranges in map_range_to_domain as we traverse the device tree on startup
> and perform some additional checks if the range needs to be mapped to
> Domain-0.
> 
> The generic PCI host controller device tree binding says [2]:
> - ranges: As described in IEEE Std 1275-1994, but must provide
>           at least a definition of non-prefetchable memory. One
>           or both of prefetchable Memory and IO Space may also
>           be provided.
> 
> - reg   : The Configuration Space base address and size, as accessed
>           from the parent bus.  The base address corresponds to
>           the first bus in the "bus-range" property.  If no
>           "bus-range" is specified, this will be bus 0 (the default).
> 
> >From the above none of the memory ranges from the "ranges" property
> needs to be mapped to Domain-0 at startup as MMIO mapping is going to
> be handled dynamically by vPCI as we assign PCI devices, e.g. each
> device assigned to Domain-0/guest will have its MMIOs mapped/unmapped
> as needed by Xen.
> 
> The "reg" property covers not only ECAM space, but may also have other
> then the configuration memory ranges described, for example [3]:
> - reg: Should contain rc_dbi, config registers location and length.
> - reg-names: Must include the following entries:
>    "rc_dbi": controller configuration registers;
>    "config": PCIe configuration space registers.
> 
> This patch makes it possible to not map all the ranges from the
> "ranges" property and also ECAM from the "reg". All the rest from the
> "reg" property still needs to be mapped to Domain-0, so the PCI
> host bridge remains functional in Domain-0.
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00777.html
> [2] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt
> [3] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - Added better description of why and what needs to be mapped into
>    Domain-0's p2m and what doesn't
>  - Do not do any mappings for PCI devices while traversing the DT
>  - Walk all the bridges and make required mappings in one go
> ---
>  xen/arch/arm/domain_build.c        | 38 +++++++++++++++--------
>  xen/arch/arm/pci/ecam.c            | 14 +++++++++
>  xen/arch/arm/pci/pci-host-common.c | 48 ++++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
>  xen/include/asm-arm/pci.h          |  9 ++++++
>  xen/include/asm-arm/setup.h        | 13 ++++++++
>  6 files changed, 111 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 83ab0d52cce9..e72c1b881cae 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -10,7 +10,6 @@
>  #include <asm/regs.h>
>  #include <xen/errno.h>
>  #include <xen/err.h>
> -#include <xen/device_tree.h>
>  #include <xen/libfdt/libfdt.h>
>  #include <xen/guest_access.h>
>  #include <xen/iocap.h>
> @@ -47,12 +46,6 @@ static int __init parse_dom0_mem(const char *s)
>  }
>  custom_param("dom0_mem", parse_dom0_mem);
>  
> -struct map_range_data
> -{
> -    struct domain *d;
> -    p2m_type_t p2mt;
> -};
> -
>  /* Override macros from asm/page.h to make them work with mfn_t */
>  #undef virt_to_mfn
>  #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
> @@ -1388,9 +1381,8 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
>      return 0;
>  }
>  
> -static int __init map_range_to_domain(const struct dt_device_node *dev,
> -                                      u64 addr, u64 len,
> -                                      void *data)
> +int __init map_range_to_domain(const struct dt_device_node *dev,
> +                               u64 addr, u64 len, void *data)
>  {
>      struct map_range_data *mr_data = data;
>      struct domain *d = mr_data->d;
> @@ -1417,6 +1409,13 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
>          }
>      }
>  
> +#ifdef CONFIG_HAS_PCI
> +    if ( is_pci_passthrough_enabled() &&
> +         (device_get_class(dev) == DEVICE_PCI) &&
> +         !mr_data->map_pci_bridge )
> +        need_mapping = false;
> +#endif

With the change I suggested below turning map_pci_bridge into
skip_mapping, then this check could go away if we just set need_mapping
as follows:

bool need_mapping = !dt_device_for_passthrough(dev) &&
                    !mr_data->skip_mapping;


>      if ( need_mapping )
>      {
>          res = map_regions_p2mt(d,
> @@ -1450,7 +1449,11 @@ static int __init map_device_children(struct domain *d,
>                                        const struct dt_device_node *dev,
>                                        p2m_type_t p2mt)
>  {
> -    struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
> +    struct map_range_data mr_data = {
> +        .d = d,
> +        .p2mt = p2mt,
> +        .map_pci_bridge = false
> +    };
>      int ret;
>  
>      if ( dt_device_type_is_equal(dev, "pci") )
> @@ -1582,7 +1585,11 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
>      /* Give permission and map MMIOs */
>      for ( i = 0; i < naddr; i++ )
>      {
> -        struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
> +        struct map_range_data mr_data = {
> +            .d = d,
> +            .p2mt = p2mt,
> +            .map_pci_bridge = false
> +        };
>          res = dt_device_get_address(dev, i, &addr, &size);
>          if ( res )
>          {
> @@ -2754,7 +2761,14 @@ static int __init construct_dom0(struct domain *d)
>          return rc;
>  
>      if ( acpi_disabled )
> +    {
>          rc = prepare_dtb_hwdom(d, &kinfo);
> +#ifdef CONFIG_HAS_PCI
> +        if ( rc < 0 )
> +            return rc;

This doesn't look great :-)

I would move the call to pci_host_bridge_mappings() below just before
construct_domain.


> +        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
> +#endif
> +    }
>      else
>          rc = prepare_acpi(d, &kinfo);
>  
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> index 9b88b1cedaa2..eae177f2cbc2 100644
> --- a/xen/arch/arm/pci/ecam.c
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -39,6 +39,19 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>      return base + (PCI_DEVFN2(sbdf) << devfn_shift) + where;
>  }
>  
> +bool pci_ecam_need_p2m_mapping(struct domain *d,
> +                               struct pci_host_bridge *bridge,
> +                               uint64_t addr)
> +{
> +    struct pci_config_window *cfg = bridge->cfg;
> +
> +    /*
> +     * We do not want ECAM address space to be mapped in Domain-0's p2m,
> +     * so we can trap access to it.
> +     */
> +    return cfg->phys_addr != addr;
> +}
> +
>  /* ECAM ops */
>  const struct pci_ecam_ops pci_generic_ecam_ops = {
>      .bus_shift  = 20,
> @@ -46,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
>          .map_bus                = pci_ecam_map_bus,
>          .read                   = pci_generic_config_read,
>          .write                  = pci_generic_config_write,
> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>      }
>  };
>  
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 155f2a2743af..f350826ea26b 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -18,6 +18,7 @@
>  
>  #include <xen/init.h>
>  #include <xen/pci.h>
> +#include <asm/setup.h>
>  #include <xen/rwlock.h>
>  #include <xen/sched.h>
>  #include <xen/vmap.h>
> @@ -328,6 +329,53 @@ int pci_host_get_num_bridges(void)
>      return count;
>  }
>  
> +int __init pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct map_range_data mr_data = {
> +        .d = d,
> +        .p2mt = p2mt,
> +        .map_pci_bridge = true
> +    };
> +
> +    /*
> +     * For each PCI host bridge we need to only map those ranges
> +     * which are used by Domain-0 to properly initialize the bridge,
> +     * e.g. we do not want to map ECAM configuration space which lives in
> +     * "reg" or "assigned-addresses" device tree property.
> +     * Neither we want to map any of the MMIO ranges found in the "ranges"
> +     * device tree property.
> +     */
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        const struct dt_device_node *dev = bridge->dt_node;
> +        int i;

i should be unsigned int


> +        for ( i = 0; i < dt_number_of_address(dev); i++ )
> +        {
> +            uint64_t addr, size;
> +            int err;
> +
> +            err = dt_device_get_address(dev, i, &addr, &size);
> +            if ( err )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",

Maybe rephrase it to:

    Unable to retrieve address range index=%u for %s


> +                       i, dt_node_full_name(dev));
> +                return err;
> +            }
> +
> +            if ( bridge->ops->need_p2m_mapping(d, bridge, addr) )

The current implementation of need_p2m_mapping filters out ECAM and
nothing else. Just double-checking here: is there anything else we
should filter out? Looking at the device tree pci node for ZynqMP:

			reg = <0x0 0xfd0e0000 0x0 0x1000 0x0 0xfd480000 0x0 0x1000 0x80 0x0 0x0 0x1000000>;
			reg-names = "breg", "pcireg", "cfg";

We are filtering out cfg, but do we need both "breg" and "pcireg" here?

If not, do we need another function like .cfg_reg_index to know what we
actually need to map?


> +            {
> +                err = map_range_to_domain(dev, addr, size, &mr_data);
> +                if ( err )
> +                    return err;
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> index c27b4ea9f02f..adbe3627871f 100644
> --- a/xen/arch/arm/pci/pci-host-zynqmp.c
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -33,6 +33,7 @@ const struct pci_ecam_ops nwl_pcie_ops = {
>          .map_bus                = pci_ecam_map_bus,
>          .read                   = pci_generic_config_read,
>          .write                  = pci_generic_config_write,
> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
>      }
>  };
>  
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 7618f0b6725b..b81f66e813ef 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -19,6 +19,8 @@
>  
>  #ifdef CONFIG_HAS_PCI
>  
> +#include <asm/p2m.h>
> +
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>  #define PRI_pci "%04x:%02x:%02x.%u"
>  
> @@ -79,6 +81,9 @@ struct pci_ops {
>                  uint32_t reg, uint32_t len, uint32_t *value);
>      int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
>                   uint32_t reg, uint32_t len, uint32_t value);
> +    bool (*need_p2m_mapping)(struct domain *d,
> +                             struct pci_host_bridge *bridge,
> +                             uint64_t addr);

I would call this function: need_p2m_hwdom_mapping


>  };
>  
>  /*
> @@ -102,6 +107,9 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
>                              uint32_t reg, uint32_t len, uint32_t value);
>  void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                 uint32_t sbdf, uint32_t where);
> +bool pci_ecam_need_p2m_mapping(struct domain *d,
> +                               struct pci_host_bridge *bridge,
> +                               uint64_t addr);
>  struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
>  int pci_get_host_bridge_segment(const struct dt_device_node *node,
>                                  uint16_t *segment);
> @@ -116,6 +124,7 @@ int pci_host_iterate_bridges(struct domain *d,
>                               int (*clb)(struct domain *d,
>                                          struct pci_host_bridge *bridge));
>  int pci_host_get_num_bridges(void);
> +int pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt);
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  #define pci_passthrough_enabled (false)
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index 95da0b7ab9cd..21863dd2bc58 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -2,6 +2,8 @@
>  #define __ARM_SETUP_H_
>  
>  #include <public/version.h>
> +#include <asm/p2m.h>
> +#include <xen/device_tree.h>
>  
>  #define MIN_FDT_ALIGN 8
>  #define MAX_FDT_SIZE SZ_2M
> @@ -77,6 +79,14 @@ struct bootinfo {
>  #endif
>  };
>  
> +struct map_range_data
> +{
> +    struct domain *d;
> +    p2m_type_t p2mt;
> +    /* Set if mappings for PCI host bridges must not be skipped. */
> +    bool map_pci_bridge;

To make this more generally applicable, I would call the new property:

  bool skip_mapping;

and it could apply to any class of devices. All current users would set
it to false except for pci_host_bridge_mappings.


> +};
>
>  extern struct bootinfo bootinfo;
>  
>  extern domid_t max_init_domid;
> @@ -124,6 +134,9 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
>  u32 device_tree_get_u32(const void *fdt, int node,
>                          const char *prop_name, u32 dflt);
>  
> +int map_range_to_domain(const struct dt_device_node *dev,
> +                        u64 addr, u64 len, void *data);
> +
>  #endif
>  /*
>   * Local variables:
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 01:11:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 01:11:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195766.348622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwDp-0002wX-8u; Sat, 25 Sep 2021 01:11:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195766.348622; Sat, 25 Sep 2021 01:11:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwDp-0002wQ-4y; Sat, 25 Sep 2021 01:11:21 +0000
Received: by outflank-mailman (input) for mailman id 195766;
 Sat, 25 Sep 2021 01:11:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTwDo-0002wG-DY; Sat, 25 Sep 2021 01:11:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTwDo-0004UY-1U; Sat, 25 Sep 2021 01:11:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTwDn-0003s6-Ml; Sat, 25 Sep 2021 01:11:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTwDn-0002aW-MK; Sat, 25 Sep 2021 01:11:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PtO9QWGcsfMzz4ISizvKNDT0N5FT3+b6+uKq5b9uM+w=; b=fQ/qEYZ8pw8n1g6E8YlBzIQJHV
	Bczoa/zjcZvDduJ8dt55n8Ju7AYyw/Osdi2lKAmba7Vkp+FyUOjA1XCvvL4qd81aQKRNznKERzF9P
	VXD5meuV+HBjn1s4iLRDSaM+ggO1gYayJhxflKqgG7KHFiE1F5/Sv5qzmW2Dzk8GB3PM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165183-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165183: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-amd64-prev:xen-build:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:guest-start/debianhvm.repeat:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-i386-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=728998f6f2b7e1420e771236efec65cbf6143b7b
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 01:11:19 +0000

flight 165183 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165183/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-prev              6 xen-build                fail REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 20 guest-start/debianhvm.repeat fail REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-migrupgrade  1 build-check(1)               blocked  n/a
 test-amd64-i386-migrupgrade   1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  728998f6f2b7e1420e771236efec65cbf6143b7b
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   14 days
Failing since        164951  2021-09-12 00:14:36 Z   13 days   26 attempts
Testing same since   165183  2021-09-24 15:07:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             fail    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 blocked 
 test-amd64-i386-migrupgrade                                  blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 937 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 01:21:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 01:21:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195773.348638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwN3-0004jA-8O; Sat, 25 Sep 2021 01:20:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195773.348638; Sat, 25 Sep 2021 01:20:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwN3-0004j3-5V; Sat, 25 Sep 2021 01:20:53 +0000
Received: by outflank-mailman (input) for mailman id 195773;
 Sat, 25 Sep 2021 01:20:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mTwN1-0004iv-OY
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 01:20:51 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e753d7f2-4f2f-4fc8-8d66-7dc340e4fd90;
 Sat, 25 Sep 2021 01:20:50 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7F38260F41;
 Sat, 25 Sep 2021 01:20:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e753d7f2-4f2f-4fc8-8d66-7dc340e4fd90
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632532849;
	bh=KsxaLz/v5KTZhH/PmH9Oaxq7suytbzRWHf6TANu2sak=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=euZvsQVzwDkoOxTSzw48e762D/4UBW7JpziXpkTCX8VpxZkSSkQ28NGT+nCrX5bA5
	 +OT87RwWZ1YbmQahPp4/28le+3ejgrIEopoZni0YesU3iOkamkmfRC3ms5iDArOTo6
	 xj5uLZb8JMpGwbJHF4Ax8WHuEP9OlTOi/jaLwfwP9utXsNwGMvlpleA6y1dCQB1sgc
	 M879mLioWeZcrBPzrZykmTsgS4QkFKMPFf97iAqvDx0wIxH1fgEVD9R6/t4nkQhmqg
	 QwA85qwn8p+BOy3Om6ZQlEpFW04tcAQ1WGi2J/SL8zsf6Vb0dw0Bzy9nEq9UCYDIlQ
	 mHdbZa++/Lg9w==
Date: Fri, 24 Sep 2021 18:20:48 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <andr2000@gmail.com>
cc: xen-devel@lists.xenproject.org, julien@xen.org, sstabellini@kernel.org, 
    oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, 
    Artem_Mygaiev@epam.com, roger.pau@citrix.com, bertrand.marquis@arm.com, 
    rahul.singh@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    jbeulich@suse.com, andrew.cooper3@citrix.com
Subject: Re: [PATCH v2 11/11] xen/arm: Process pending vPCI map/unmap
 operations
In-Reply-To: <20210923125438.234162-12-andr2000@gmail.com>
Message-ID: <alpine.DEB.2.21.2109241816370.17979@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-12-andr2000@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

+ x86 maintainers

On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> vPCI may map and unmap PCI device memory (BARs) being passed through which
> may take a lot of time. For this those operations may be deferred to be
> performed later, so that they can be safely preempted.
> Run the corresponding vPCI code while switching a vCPU.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

>From an ARM point of view, I think the code change is OK.

Only one note: it would be good to add to the commit message a short
list of relevant TODOs which in particular affect this patch.

Something like:

Please be aware that there are a few outstanding TODOs affecting this
code path, see xen/drivers/vpci/header.c:map_range and
xen/drivers/vpci/header.c:vpci_process_pending.


> ---
> Since v1:
>  - Moved the check for pending vpci work from the common IOREQ code
>    to hvm_do_resume on x86
>  - Re-worked the code for Arm to ensure we don't miss pending vPCI work
> ---
>  xen/arch/arm/traps.c   | 13 +++++++++++++
>  xen/arch/x86/hvm/hvm.c |  6 ++++++
>  xen/common/ioreq.c     |  9 ---------
>  3 files changed, 19 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 219ab3c3fbde..b246f51086e3 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -34,6 +34,7 @@
>  #include <xen/symbols.h>
>  #include <xen/version.h>
>  #include <xen/virtual_region.h>
> +#include <xen/vpci.h>
>  
>  #include <public/sched.h>
>  #include <public/xen.h>
> @@ -2304,6 +2305,18 @@ static bool check_for_vcpu_work(void)
>      }
>  #endif
>  
> +    if ( has_vpci(v->domain) )
> +    {
> +        bool pending;
> +
> +        local_irq_enable();
> +        pending = vpci_process_pending(v);
> +        local_irq_disable();
> +
> +        if ( pending )
> +            return true;
> +    }
> +
>      if ( likely(!v->arch.need_flush_to_ram) )
>          return false;
>  
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 7b48a1b925bb..d32f5d572941 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -549,6 +549,12 @@ void hvm_do_resume(struct vcpu *v)
>      if ( !vcpu_ioreq_handle_completion(v) )
>          return;
>  
> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )
> +    {
> +        raise_softirq(SCHEDULE_SOFTIRQ);
> +        return;
> +    }
> +
>      if ( unlikely(v->arch.vm_event) )
>          hvm_vm_event_do_resume(v);
>  
> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
> index d732dc045df9..689d256544c8 100644
> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -25,9 +25,7 @@
>  #include <xen/lib.h>
>  #include <xen/paging.h>
>  #include <xen/sched.h>
> -#include <xen/softirq.h>
>  #include <xen/trace.h>
> -#include <xen/vpci.h>
>  
>  #include <asm/guest_atomics.h>
>  #include <asm/ioreq.h>
> @@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
>  
>  bool vcpu_ioreq_handle_completion(struct vcpu *v)
>  {
> -    struct domain *d = v->domain;
>      struct vcpu_io *vio = &v->io;
>      struct ioreq_server *s;
>      struct ioreq_vcpu *sv;
>      enum vio_completion completion;
>      bool res = true;
>  
> -    if ( has_vpci(d) && vpci_process_pending(v) )
> -    {
> -        raise_softirq(SCHEDULE_SOFTIRQ);
> -        return false;
> -    }
> -
>      while ( (sv = get_pending_vcpu(v, &s)) != NULL )
>          if ( !wait_for_io(sv, get_ioreq(s, v)) )
>              return false;
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 01:49:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 01:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195779.348650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwoA-00087A-Ee; Sat, 25 Sep 2021 01:48:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195779.348650; Sat, 25 Sep 2021 01:48:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTwoA-000873-BT; Sat, 25 Sep 2021 01:48:54 +0000
Received: by outflank-mailman (input) for mailman id 195779;
 Sat, 25 Sep 2021 01:48:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1mTwo9-00086x-49
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 01:48:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTwo6-000569-9E; Sat, 25 Sep 2021 01:48:50 +0000
Received: from [202.153.84.92] (helo=a483e7b01a66.ant.amazon.com)
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1mTwo5-0007B1-T5; Sat, 25 Sep 2021 01:48:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	MIME-Version:Date:Message-ID:From:References:Cc:To:Subject;
	bh=HMJ52tmpOQVCg3/stX7Qdi9VPmCsKC5Ix2Wvi6AMRpM=; b=IHBD4H9t7qPao+RyPvKbzKQTGo
	jq9f/lvG6vqOpqhrmaKnNVHpEqS1tqCs9BPSsQGvMrey7otHurcN3PvEO3lt1TFSsmwMr516yjinN
	2VPtNpZzvzY/R0Dfy/uccPqywiEr9XzvUx6yiesbxAxf1/7SBAeQzKJPOGGn5bwy3JlE=;
Subject: Re: [future abi] [RFC PATCH V3] xen/gnttab: Store frame GFN in struct
 page_info on Arm
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Oleksandr Tyshchenko <olekstysh@gmail.com>, xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
References: <1632425551-18910-1-git-send-email-olekstysh@gmail.com>
 <de92ab24-a9f0-8f3b-e721-e0465e4dc065@citrix.com>
 <YU2PT4rUts8KljKe@MacBook-Air-de-Roger.local>
 <04400e18-dde2-4b90-4056-f56c5d7937af@xen.org>
 <YU34dsl4cSCBbfrk@MacBook-Air-de-Roger.local>
From: Julien Grall <julien@xen.org>
Message-ID: <547aff1d-d9dd-c7a5-bfeb-fb6aaa011051@xen.org>
Date: Sat, 25 Sep 2021 06:48:43 +0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YU34dsl4cSCBbfrk@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 8bit

Hi Roger,

On 24/09/2021 21:10, Roger Pau Monné wrote:
> On Fri, Sep 24, 2021 at 07:52:24PM +0500, Julien Grall wrote:
>> Hi Roger,
>>
>> On 24/09/2021 13:41, Roger Pau Monné wrote:
>>> On Thu, Sep 23, 2021 at 09:59:26PM +0100, Andrew Cooper wrote:
>>>> On 23/09/2021 20:32, Oleksandr Tyshchenko wrote:
>>>>> Suggested-by: Julien Grall <jgrall@amazon.com>
>>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>> ---
>>>>> You can find the related discussions at:
>>>>> https://lore.kernel.org/xen-devel/93d0df14-2c8a-c2e3-8c51-54412190171c@xen.org/
>>>>> https://lore.kernel.org/xen-devel/1628890077-12545-1-git-send-email-olekstysh@gmail.com/
>>>>> https://lore.kernel.org/xen-devel/1631652245-30746-1-git-send-email-olekstysh@gmail.com/
>>>>>
>>>>> ! Please note, there is still unresolved locking question here for which
>>>>> I failed to find a suitable solution. So, it is still an RFC !
>>>>
>>>> Just FYI, I thought I'd share some of the plans for ABI v2.  Obviously
>>>> these plans are future work and don't solve the current problem.
>>>>
>>>> Guests mapping Xen pages is backwards.  There are reasons why it was
>>>> used for x86 PV guests, but the entire interface should have been design
>>>> differently for x86 HVM.
>>>>
>>>> In particular, Xen should be mapping guest RAM, rather than the guest
>>>> manipulating the 2nd stage tables to map Xen RAM.  Amongst other things,
>>>> its far far lower overhead.
>>>>
>>>>
>>>> A much better design is one where the grant table looks like an MMIO
>>>> device.  The domain builder decides the ABI (v1 vs v2 - none of this
>>>> dynamic switch at runtime nonsense), and picks a block of guest physical
>>>> addresses, which are registered with Xen.  This forms the grant table,
>>>> status table (v2 only), and holes to map into.
>>>
>>> I think this could be problematic for identity mapped Arm dom0, as
>>> IIRC in that case grants are mapped so that gfn == mfn in order to
>>> account for the lack of an IOMMU. You could use a bounce buffer, but
>>> that would introduce a big performance penalty.
>>
>> Or you could find a hole that is outside of the RAM regions. This is not
>> trivial but not impossible (see [1]).
> 
> I certainly not familiar with the Arm identity map.
> 
> If you map them at random areas (so no longer identity mapped), how do
> you pass the addresses to the physical devices for DMA operations? I
> assume there must be some kind of translation then that converts from
> gfn to mfn in order to cope with the lack of an IOMMU, 

For grant mapping, the hypercall will return the machine address in 
dev_bus_addr. Dom0, will keep the conversion dom0 GFN <-> MFN for later 
use in the swiotlb.

For foreign mapping, AFAICT, we are expecting them to bounce everytime. 
But DMA into a foreign mapping should be rarer.

> and because
> dom0 doesn't know the mfn of the grant reference in order to map it at
> the same gfn.

IIRC, we tried an approach where the grant mapping would be direct 
mapped in dom0. However, this was an issue on arm32 because Debian was 
(is?) using short descriptor page tables. This didn't allow dom0 to 
cover all the mappings and therefore some mappings would not be accessible.

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 04:37:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 04:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195624.348666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzR1-0004Dq-3U; Sat, 25 Sep 2021 04:37:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195624.348666; Sat, 25 Sep 2021 04:37:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzR0-0004CW-Tc; Sat, 25 Sep 2021 04:37:10 +0000
Received: by outflank-mailman (input) for mailman id 195624;
 Fri, 24 Sep 2021 19:40:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MekI=OO=gmail.com=this.is.a0lson@srs-us1.protection.inumbo.net>)
 id 1mTr3n-0003J8-TX
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:40:39 +0000
Received: from mail-qt1-x833.google.com (unknown [2607:f8b0:4864:20::833])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 18a82e6f-e431-48f1-80ac-8f7083d24bae;
 Fri, 24 Sep 2021 19:40:36 +0000 (UTC)
Received: by mail-qt1-x833.google.com with SMTP id r1so10488410qta.12
 for <xen-devel@lists.xenproject.org>; Fri, 24 Sep 2021 12:40:36 -0700 (PDT)
Received: from localhost.localdomain ([24.214.236.228])
 by smtp.gmail.com with ESMTPSA id s10sm6957971qko.134.2021.09.24.12.40.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 Sep 2021 12:40:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18a82e6f-e431-48f1-80ac-8f7083d24bae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=9SjNRKk5NXPDvp5QWjMJFRC3W8sz2kqMr2/m/QyAXos=;
        b=YLnmy4MhxIOeh27w+2iaWtlqaOtNA/wco23lILFwUvPrqfxEVZy0SNBQbX/VHfRymH
         s4tTtNwIH8Yxq57mEcNQPanObde3berAZwimM28vsyBXtozOeKOlb+jKsBlO2n4541Fg
         0gO1qdBTraBk1fkjF8tBuLvUalMAJlD2yAXEg6UF0H96E0rV2dKz/OWI4HWIHIGh+1CL
         kwbzxUWI/ItfJxvij1nJMoTNb+juSXueFTRsjcEA89zbbIxbIzKlbKY7o4/hdU246Fb+
         keMjIEm+3tR+Vgp4+Y2C/xV8WfEhOMa5DhBkWajEu9vH71zayBeDSV0gF9Tq7L1bHkH8
         bJmw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=9SjNRKk5NXPDvp5QWjMJFRC3W8sz2kqMr2/m/QyAXos=;
        b=Eo4pvajqSzrjJjVvByxT7hvFTI9cVpWK014DlfIcHqAGTKg+xs6tNJeSH4uCOXys0Y
         kCQbeJnsCq11oDehMNDGroPsGQie4Zg7d6ejSZpSz1f+QsQiD65Ml3sHak7oTEbeg7v0
         RUOmvQuvi30rNmMBY4LgACIxyvOUJoFdwWxxeENJDwHj4CPtOV608oyd5LNd8V3M6k88
         p6G8EFlUOuNSduCsQ1t8i3q0NP6bDQ1/b9Fm60v0T4aAxmZWDuIr6aOW4NvDZw91NrUN
         ZFU8fY+zatYqyLWMRU2IoGBhBdmP3CXx1Ci6vXsEgFaaxAK2ZsaWANlUV/UwkK59IOXZ
         YP7A==
X-Gm-Message-State: AOAM532GuLoFF9GMUfUApX5UWUo9MDDorimsvJHwzbuPfsxeNushz4kF
	B3sYmdzPYU4RbyH7HKxzRSaQC6HY4nqIaA==
X-Google-Smtp-Source: ABdhPJwaUDVJHuEB0YctSzsdaH5+ImJpcYhgb38pEzB5Qxbt6mMGgA1kElXucw2fyZqsS7ykji2mVg==
X-Received: by 2002:ac8:7143:: with SMTP id h3mr6216840qtp.242.1632512435684;
        Fri, 24 Sep 2021 12:40:35 -0700 (PDT)
From: Alex Olson <this.is.a0lson@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Alex Olson <this.is.a0lson@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Ian Jackson <iwj@xenproject.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Alex Olson <alex.olson@starlab.io>
Subject: [PATCH 1/1] x86: centralize default APIC id definition
Date: Fri, 24 Sep 2021 14:39:55 -0500
Message-Id: <85b59046315b8a84afa8538aacdea92b19200faa.1632512149.git.this.is.a0lson@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <cover.1632512149.git.this.is.a0lson@gmail.com>
References: <cover.1632512149.git.this.is.a0lson@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Inspired by an earlier attempt by Chao Gao <chao.gao@intel.com>,
this revision aims to put the hypervisor in control of x86 APIC identifier
definition instead of hard-coding a formula in multiple places
(libxl, hvmloader, hypervisor).

This is intended as a first step toward exposing/altering CPU topology
seen by guests.

Changes:

- Add field to vlapic for holding default ID (on reset)

- add HVMOP_get_vcpu_topology_id hypercall so libxl (for PVH domains)
  can access APIC ids needed for ACPI table definition prior to domain start.

- For HVM guests, hvmloader now also uses the same hypercall.

- Make CPUID code use vlapic ID instead of hard-coded formula
  for runtime reporting to guests

Signed-off-by: Alex Olson <alex.olson@starlab.io>
---
 tools/firmware/hvmloader/Makefile      |  2 +-
 tools/firmware/hvmloader/config.h      |  1 -
 tools/firmware/hvmloader/hvmloader.c   |  3 +-
 tools/firmware/hvmloader/mp_tables.c   |  3 +-
 tools/firmware/hvmloader/smp.c         |  3 +-
 tools/firmware/hvmloader/topology.c    | 54 ++++++++++++++++++++++++++
 tools/firmware/hvmloader/topology.h    | 33 ++++++++++++++++
 tools/firmware/hvmloader/util.c        |  6 ++-
 tools/include/xenctrl.h                |  6 +++
 tools/libacpi/build.c                  |  4 +-
 tools/libacpi/libacpi.h                |  3 +-
 tools/libs/ctrl/xc_domain.c            | 27 +++++++++++++
 tools/libs/light/libxl_x86_acpi.c      |  9 ++++-
 xen/arch/x86/cpuid.c                   | 14 +++++--
 xen/arch/x86/hvm/hvm.c                 | 36 ++++++++++++++++-
 xen/arch/x86/hvm/vlapic.c              | 18 ++++++---
 xen/include/asm-x86/hvm/vlapic.h       |  4 +-
 xen/include/public/arch-x86/hvm/save.h |  1 +
 xen/include/public/hvm/hvm_op.h        | 17 ++++++++
 19 files changed, 222 insertions(+), 22 deletions(-)
 create mode 100644 tools/firmware/hvmloader/topology.c
 create mode 100644 tools/firmware/hvmloader/topology.h

diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile
index e980ce7c5f..158f62b4e6 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -29,7 +29,7 @@ CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__
 
 OBJS  = hvmloader.o mp_tables.o util.o smbios.o 
-OBJS += smp.o cacheattr.o xenbus.o vnuma.o
+OBJS += smp.o cacheattr.o xenbus.o vnuma.o topology.o
 OBJS += e820.o pci.o pir.o ctype.o
 OBJS += hvm_param.o
 OBJS += ovmf.o seabios.o
diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h
index 844120bc87..91d73c9086 100644
--- a/tools/firmware/hvmloader/config.h
+++ b/tools/firmware/hvmloader/config.h
@@ -50,7 +50,6 @@ extern uint8_t ioapic_version;
 #define IOAPIC_ID           0x01
 
 #define LAPIC_BASE_ADDRESS  0xfee00000
-#define LAPIC_ID(vcpu_id)   ((vcpu_id) * 2)
 
 #define PCI_ISA_DEVFN       0x08    /* dev 1, fn 0 */
 #define PCI_ISA_IRQ_MASK    0x0c20U /* ISA IRQs 5,10,11 are PCI connected */
diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c
index c58841e5b5..250e6779b1 100644
--- a/tools/firmware/hvmloader/hvmloader.c
+++ b/tools/firmware/hvmloader/hvmloader.c
@@ -25,6 +25,7 @@
 #include "pci_regs.h"
 #include "apic_regs.h"
 #include "vnuma.h"
+#include "topology.h"
 #include <acpi2_0.h>
 #include <xen/version.h>
 #include <xen/hvm/params.h>
@@ -225,7 +226,7 @@ static void apic_setup(void)
 
     /* 8259A ExtInts are delivered through IOAPIC pin 0 (Virtual Wire Mode). */
     ioapic_write(0x10, APIC_DM_EXTINT);
-    ioapic_write(0x11, SET_APIC_ID(LAPIC_ID(0)));
+    ioapic_write(0x11, SET_APIC_ID(get_topology_id(0)));
 }
 
 struct bios_info {
diff --git a/tools/firmware/hvmloader/mp_tables.c b/tools/firmware/hvmloader/mp_tables.c
index d207ecbf00..562961ca0b 100644
--- a/tools/firmware/hvmloader/mp_tables.c
+++ b/tools/firmware/hvmloader/mp_tables.c
@@ -29,6 +29,7 @@
 
 #include <stdint.h>
 #include "config.h"
+#include "topology.h"
 
 /* number of non-processor MP table entries */
 #define NR_NONPROC_ENTRIES     18
@@ -199,7 +200,7 @@ static void fill_mp_config_table(struct mp_config_table *mpct, int length)
 static void fill_mp_proc_entry(struct mp_proc_entry *mppe, int vcpu_id)
 {
     mppe->type = ENTRY_TYPE_PROCESSOR;
-    mppe->lapic_id = LAPIC_ID(vcpu_id);
+    mppe->lapic_id = get_topology_id(vcpu_id);
     mppe->lapic_version = 0x11;
     mppe->cpu_flags = CPU_FLAG_ENABLED;
     if ( vcpu_id == 0 )
diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c
index 082b17f138..5091ff1f1f 100644
--- a/tools/firmware/hvmloader/smp.c
+++ b/tools/firmware/hvmloader/smp.c
@@ -22,6 +22,7 @@
 #include "util.h"
 #include "config.h"
 #include "apic_regs.h"
+#include "topology.h"
 
 #define AP_BOOT_EIP 0x1000
 extern char ap_boot_start[], ap_boot_end[];
@@ -86,7 +87,7 @@ static void lapic_wait_ready(void)
 
 static void boot_cpu(unsigned int cpu)
 {
-    unsigned int icr2 = SET_APIC_DEST_FIELD(LAPIC_ID(cpu));
+    unsigned int icr2 = SET_APIC_DEST_FIELD(get_topology_id(cpu));
 
     /* Initialise shared variables. */
     ap_cpuid = cpu;
diff --git a/tools/firmware/hvmloader/topology.c b/tools/firmware/hvmloader/topology.c
new file mode 100644
index 0000000000..f92bbb8b7a
--- /dev/null
+++ b/tools/firmware/hvmloader/topology.c
@@ -0,0 +1,54 @@
+/*
+ * topology.c: obtain vCPU topology from hypervisor
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "util.h"
+#include "hypercall.h"
+#include "topology.h"
+#include <xen/hvm/hvm_op.h>
+
+uint32_t get_topology_id(unsigned vcpu_id)
+{
+    int rc;
+    struct xen_vcpu_topology_id tid =
+        { .domid = DOMID_SELF, .vcpu_id = vcpu_id };
+
+    rc = hypercall_hvm_op(HVMOP_get_vcpu_topology_id, &tid);
+    if ( rc < 0 )
+    {
+        printf("Failed to retrieve cpu topology for vcpu=%u, rc = %d\n", vcpu_id, rc);
+        return 0xdeadbeef;
+    }
+    return tid.topology_id;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/firmware/hvmloader/topology.h b/tools/firmware/hvmloader/topology.h
new file mode 100644
index 0000000000..6e37051556
--- /dev/null
+++ b/tools/firmware/hvmloader/topology.h
@@ -0,0 +1,33 @@
+/******************************************************************************
+ * topology.h
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+#ifndef __HVMLOADER_TOPOLOGY_H_
+#define __HVMLOADER_TOPOLOGY_H_
+
+uint32_t get_topology_id(unsigned vcpu_id);
+
+#endif /* __HVMLOADER_TOPOLOGY_H__ */
diff --git a/tools/firmware/hvmloader/util.c b/tools/firmware/hvmloader/util.c
index 581b35e5cf..a5fe15d198 100644
--- a/tools/firmware/hvmloader/util.c
+++ b/tools/firmware/hvmloader/util.c
@@ -22,6 +22,7 @@
 #include "hypercall.h"
 #include "ctype.h"
 #include "vnuma.h"
+#include "topology.h"
 #include <acpi2_0.h>
 #include <libacpi.h>
 #include <stdint.h>
@@ -910,9 +911,9 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt,
     /* ACPI builder currently doesn't free memory so this is just a stub */
 }
 
-static uint32_t acpi_lapic_id(unsigned cpu)
+static uint32_t acpi_lapic_id(unsigned cpu, void *arg)
 {
-    return LAPIC_ID(cpu);
+    return get_topology_id(cpu);
 }
 
 void hvmloader_acpi_build_tables(struct acpi_config *config,
@@ -943,6 +944,7 @@ void hvmloader_acpi_build_tables(struct acpi_config *config,
 
     config->lapic_base_address = LAPIC_BASE_ADDRESS;
     config->lapic_id = acpi_lapic_id;
+    config->lapic_id_arg = NULL;
     config->ioapic_base_address = ioapic_base_address;
     config->ioapic_id = IOAPIC_ID;
     config->pci_isa_irq_mask = PCI_ISA_IRQ_MASK; 
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index a3063998e0..c15a2bb0c8 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1262,6 +1262,12 @@ int xc_domain_set_memory_map(xc_interface *xch,
 int xc_get_machine_memory_map(xc_interface *xch,
                               struct e820entry entries[],
                               uint32_t max_entries);
+
+int xc_get_vcpu_topology_id(xc_interface *xch,
+                            uint32_t domid,
+                            unsigned int vcpu_id,
+                            uint32_t *topology_id);
+
 #endif
 
 int xc_reserved_device_memory_map(xc_interface *xch,
diff --git a/tools/libacpi/build.c b/tools/libacpi/build.c
index fe2db66a62..aa5e851172 100644
--- a/tools/libacpi/build.c
+++ b/tools/libacpi/build.c
@@ -156,7 +156,7 @@ static struct acpi_20_madt *construct_madt(struct acpi_ctxt *ctxt,
         lapic->length  = sizeof(*lapic);
         /* Processor ID must match processor-object IDs in the DSDT. */
         lapic->acpi_processor_id = i;
-        lapic->apic_id = config->lapic_id(i);
+        lapic->apic_id = config->lapic_id(i, config->lapic_id_arg);
         lapic->flags = (test_bit(i, hvminfo->vcpu_online)
                         ? ACPI_LOCAL_APIC_ENABLED : 0);
         lapic++;
@@ -244,7 +244,7 @@ static struct acpi_20_srat *construct_srat(struct acpi_ctxt *ctxt,
         processor->type     = ACPI_PROCESSOR_AFFINITY;
         processor->length   = sizeof(*processor);
         processor->domain   = config->numa.vcpu_to_vnode[i];
-        processor->apic_id  = config->lapic_id(i);
+        processor->apic_id  = config->lapic_id(i, config->lapic_id_arg);
         processor->flags    = ACPI_LOCAL_APIC_AFFIN_ENABLED;
         processor++;
     }
diff --git a/tools/libacpi/libacpi.h b/tools/libacpi/libacpi.h
index a2efd23b0b..c66d1f0ea1 100644
--- a/tools/libacpi/libacpi.h
+++ b/tools/libacpi/libacpi.h
@@ -91,7 +91,8 @@ struct acpi_config {
     unsigned long rsdp;
 
     /* x86-specific parameters */
-    uint32_t (*lapic_id)(unsigned cpu);
+    uint32_t (*lapic_id)(unsigned cpu, void *arg);
+    void     *lapic_id_arg;
     uint32_t lapic_base_address;
     uint32_t ioapic_base_address;
     uint16_t pci_isa_irq_mask;
diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index 23322b70b5..087196130c 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -1506,6 +1506,33 @@ int xc_get_hvm_param(xc_interface *handle, uint32_t dom, int param, unsigned lon
     return 0;
 }
 
+
+#if defined (__i386__) || defined (__x86_64__)
+int xc_get_vcpu_topology_id(xc_interface *xch,
+                            uint32_t dom,
+                            unsigned int vcpu_id,
+                            uint32_t *topology_id)
+{
+    DECLARE_HYPERCALL_BUFFER(xen_vcpu_topology_id_t, arg);
+    int rc;
+
+    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
+    if ( arg == NULL )
+        return -1;
+
+    arg->domid = dom;
+    arg->vcpu_id = vcpu_id;
+    arg->topology_id = 0xdeadbeef;
+
+    rc = xencall2(xch->xcall, __HYPERVISOR_hvm_op,
+                  HVMOP_get_vcpu_topology_id,
+                  HYPERCALL_BUFFER_AS_ARG(arg));
+    *topology_id = arg->topology_id;
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+#endif
+
 int xc_domain_setdebugging(xc_interface *xch,
                            uint32_t domid,
                            unsigned int enable)
diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 3eca1c7a9f..8e471f222f 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -79,9 +79,13 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt,
 {
 }
 
-static uint32_t acpi_lapic_id(unsigned cpu)
+static uint32_t acpi_lapic_id(unsigned cpu, void *arg)
 {
-    return cpu * 2;
+    struct xc_dom_image *dom = (struct xc_dom_image *)arg;
+    uint32_t ret = 0xdeadbeef;
+    int rc;
+    rc = xc_get_vcpu_topology_id(dom->xch, dom->guest_domid, cpu, &ret);
+    return ret;
 }
 
 static int init_acpi_config(libxl__gc *gc, 
@@ -149,6 +153,7 @@ static int init_acpi_config(libxl__gc *gc,
 
     config->lapic_base_address = LAPIC_BASE_ADDRESS;
     config->lapic_id = acpi_lapic_id;
+    config->lapic_id_arg = dom;
     config->acpi_revision = 5;
 
     rc = 0;
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 2079a30ae4..cbed233726 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -867,8 +867,10 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
     case 0x1:
         /* TODO: Rework topology logic. */
         res->b &= 0x00ffffffu;
-        if ( is_hvm_domain(d) )
-            res->b |= (v->vcpu_id * 2) << 24;
+
+#ifdef CONFIG_HVM
+        res->b |= vlapic_get_default_id(v) << 24;
+#endif
 
         /* TODO: Rework vPMU control in terms of toolstack choices. */
         if ( vpmu_available(v) &&
@@ -1049,7 +1051,13 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf,
             *(uint8_t *)&res->c = subleaf;
 
             /* Fix the x2APIC identifier. */
-            res->d = v->vcpu_id * 2;
+#ifdef CONFIG_HVM
+            res->d = vlapic_get_default_id(v);
+#endif
+        }
+        else
+        {
+            *res = EMPTY_LEAF;
         }
         break;
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7b48a1b925..e93ca8187d 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1555,7 +1555,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
         goto fail1;
 
     /* NB: vlapic_init must be called before hvm_funcs.vcpu_initialise */
-    rc = vlapic_init(v);
+    rc = vlapic_init(v, v->vcpu_id * 2);
     if ( rc != 0 ) /* teardown: vlapic_destroy */
         goto fail2;
 
@@ -5084,6 +5084,40 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
         rc = current->hcall_compat ? compat_altp2m_op(arg) : do_altp2m_op(arg);
         break;
 
+    case HVMOP_get_vcpu_topology_id:
+    {
+        struct domain *d;
+        struct xen_vcpu_topology_id tid;
+
+        if ( copy_from_guest(&tid, arg, 1) )
+            return -EFAULT;
+
+        if (tid.domid != DOMID_SELF && !is_hardware_domain(current->domain))
+            return -EPERM;
+
+        if ( (d = rcu_lock_domain_by_any_id(tid.domid)) == NULL )
+            return -ESRCH;
+
+        if ( !is_hvm_domain(d))
+        {
+            rc = -EOPNOTSUPP;
+            goto get_cpu_topology_failed;
+        }
+
+        if ( tid.vcpu_id >= d->max_vcpus )
+        {
+            rc = -EINVAL;
+            goto get_cpu_topology_failed;
+        }
+        tid.topology_id = vlapic_get_default_id(d->vcpu[tid.vcpu_id]);
+
+        rc = copy_to_guest(arg, &tid, 1) ? -EFAULT : 0;
+
+ get_cpu_topology_failed:
+        rcu_unlock_domain(d);
+        break;
+    }
+
     default:
     {
         gdprintk(XENLOG_DEBUG, "Bad HVM op %ld.\n", op);
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 5e21fb4937..e65b1e8e94 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1068,12 +1068,18 @@ static const struct hvm_mmio_ops vlapic_mmio_ops = {
     .write = vlapic_mmio_write,
 };
 
+uint32_t vlapic_get_default_id(const struct vcpu *v)
+{
+    const struct vlapic *vlapic = vcpu_vlapic(v);
+    return vlapic->hw.default_id;
+}
+
 static void set_x2apic_id(struct vlapic *vlapic)
 {
-    u32 id = vlapic_vcpu(vlapic)->vcpu_id;
+    u32 id = vlapic->hw.default_id;
     u32 ldr = ((id & ~0xf) << 12) | (1 << (id & 0xf));
 
-    vlapic_set_reg(vlapic, APIC_ID, id * 2);
+    vlapic_set_reg(vlapic, APIC_ID, id);
     vlapic_set_reg(vlapic, APIC_LDR, ldr);
 }
 
@@ -1440,7 +1446,7 @@ void vlapic_reset(struct vlapic *vlapic)
     if ( v->vcpu_id == 0 )
         vlapic->hw.apic_base_msr |= APIC_BASE_BSP;
 
-    vlapic_set_reg(vlapic, APIC_ID, (v->vcpu_id * 2) << 24);
+    vlapic_set_reg(vlapic, APIC_ID, vlapic->hw.default_id << 24);
     vlapic_do_init(vlapic);
 }
 
@@ -1508,7 +1514,7 @@ static void lapic_load_fixup(struct vlapic *vlapic)
          * here, but can be dropped as soon as it is found to conflict with
          * other (future) changes.
          */
-        if ( GET_xAPIC_ID(id) != vlapic_vcpu(vlapic)->vcpu_id * 2 ||
+        if ( GET_xAPIC_ID(id) != vlapic->hw.default_id ||
              id != SET_xAPIC_ID(GET_xAPIC_ID(id)) )
             printk(XENLOG_G_WARNING "%pv: bogus APIC ID %#x loaded\n",
                    vlapic_vcpu(vlapic), id);
@@ -1596,12 +1602,14 @@ HVM_REGISTER_SAVE_RESTORE(LAPIC, lapic_save_hidden,
 HVM_REGISTER_SAVE_RESTORE(LAPIC_REGS, lapic_save_regs,
                           lapic_load_regs, 1, HVMSR_PER_VCPU);
 
-int vlapic_init(struct vcpu *v)
+int vlapic_init(struct vcpu *v, uint32_t apic_id)
 {
     struct vlapic *vlapic = vcpu_vlapic(v);
 
     HVM_DBG_LOG(DBG_LEVEL_VLAPIC, "%d", v->vcpu_id);
 
+    vlapic->hw.default_id = apic_id;
+
     if ( !has_vlapic(v->domain) )
     {
         vlapic->hw.disabled = VLAPIC_HW_DISABLED;
diff --git a/xen/include/asm-x86/hvm/vlapic.h b/xen/include/asm-x86/hvm/vlapic.h
index 8f908928c3..06c9c5cd7d 100644
--- a/xen/include/asm-x86/hvm/vlapic.h
+++ b/xen/include/asm-x86/hvm/vlapic.h
@@ -116,7 +116,7 @@ void vlapic_set_irq(struct vlapic *vlapic, uint8_t vec, uint8_t trig);
 int vlapic_has_pending_irq(struct vcpu *v);
 int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool_t force_ack);
 
-int  vlapic_init(struct vcpu *v);
+int  vlapic_init(struct vcpu *v, uint32_t apic_id);
 void vlapic_destroy(struct vcpu *v);
 
 void vlapic_reset(struct vlapic *vlapic);
@@ -154,4 +154,6 @@ static inline void vlapic_sync_pir_to_irr(struct vcpu *v)
         alternative_vcall(hvm_funcs.sync_pir_to_irr, v);
 }
 
+uint32_t vlapic_get_default_id(const struct vcpu *v);
+
 #endif /* __ASM_X86_HVM_VLAPIC_H__ */
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 773a380bc2..f960257dbf 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -411,6 +411,7 @@ struct hvm_hw_lapic {
     uint32_t             disabled; /* VLAPIC_xx_DISABLED */
     uint32_t             timer_divisor;
     uint64_t             tdt_msr;
+    uint32_t             default_id;
 };
 
 DECLARE_HVM_SAVE_TYPE(LAPIC, 5, struct hvm_hw_lapic);
diff --git a/xen/include/public/hvm/hvm_op.h b/xen/include/public/hvm/hvm_op.h
index 870ec52060..bf7872a4d1 100644
--- a/xen/include/public/hvm/hvm_op.h
+++ b/xen/include/public/hvm/hvm_op.h
@@ -183,6 +183,23 @@ struct xen_hvm_get_mem_type {
 typedef struct xen_hvm_get_mem_type xen_hvm_get_mem_type_t;
 DEFINE_XEN_GUEST_HANDLE(xen_hvm_get_mem_type_t);
 
+/*
+ * HVMOP_get_cpu_topology is used by guest to acquire vcpu topology from
+ * hypervisor.
+ */
+#define HVMOP_get_vcpu_topology_id     16
+
+struct xen_vcpu_topology_id {
+    /* IN */
+    domid_t domid;
+    uint32_t vcpu_id;
+
+    /* OUT */
+    uint32_t topology_id;
+};
+typedef struct xen_vcpu_topology_id xen_vcpu_topology_id_t;
+DEFINE_XEN_GUEST_HANDLE(xen_vcpu_topology_id_t);
+
 /* Following tools-only interfaces may change in future. */
 #if defined(__XEN__) || defined(__XEN_TOOLS__)
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Sat Sep 25 04:37:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 04:37:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195622.348661 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzR0-0004Ag-PO; Sat, 25 Sep 2021 04:37:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195622.348661; Sat, 25 Sep 2021 04:37:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzR0-0004AZ-MQ; Sat, 25 Sep 2021 04:37:10 +0000
Received: by outflank-mailman (input) for mailman id 195622;
 Fri, 24 Sep 2021 19:40:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MekI=OO=gmail.com=this.is.a0lson@srs-us1.protection.inumbo.net>)
 id 1mTr3Y-0003IB-Vf
 for xen-devel@lists.xenproject.org; Fri, 24 Sep 2021 19:40:25 +0000
Received: from mail-qv1-xf32.google.com (unknown [2607:f8b0:4864:20::f32])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2f4ebd2d-6a11-4205-aac2-f93601508c98;
 Fri, 24 Sep 2021 19:40:24 +0000 (UTC)
Received: by mail-qv1-xf32.google.com with SMTP id jo30so6980984qvb.3
 for <xen-devel@lists.xenproject.org>; Fri, 24 Sep 2021 12:40:24 -0700 (PDT)
Received: from localhost.localdomain ([24.214.236.228])
 by smtp.gmail.com with ESMTPSA id s10sm6957971qko.134.2021.09.24.12.40.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 24 Sep 2021 12:40:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f4ebd2d-6a11-4205-aac2-f93601508c98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=ChVaZv1jb3kHb49X6DQZKb1FYMKbVWrD9eIvacxmHqU=;
        b=Z76KdCIRM/kNv5rQJjuvfa8w+r/kjnJqsBl6lNhJqpgNKEllB2PMLCVtnf9Xg3d9yQ
         aY9qvJa0yXvKh2xE5WEjYDnmcz6ualhNczi4RMgSbmuvi1/NB6zjIWt/sGl7a6NCNqNq
         zJGZ3wWpgthqLTNlX2hRps+2646HXz+8sP9nYj3EjPiF7dLz0A0W8Ap9Pj8m9ETxExP4
         Ewc7WTwL5bTq+EAychBWf/OEaH5zSCew82hcqmbW2YRsJD8rlgbgV6SSK4+iCtopjN4T
         9ZZP/PyBRMPQ5+45PG20C99jXQpWjNVpfpVLypL5c5buSHWYnHuDlR0ca8EO7ftQkEPi
         Q9+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=ChVaZv1jb3kHb49X6DQZKb1FYMKbVWrD9eIvacxmHqU=;
        b=3raujs5zfblQF8nBRjALEzfTF8hsSgLbXn0e30qjVIT20u+XEcXdKoBODfBrGFcUas
         2EbPRdYpNN2wueliH5fdHNOSDUIiZ9554g5e1kHU87Y2VlV5vgQ4DRTLpV+F1Me3neqz
         Y8SqNArexHBz6CxbinChdK0W4UT+ghFqGsH6PJWHSnCxgZ8JeX9gwK7nQ+JdAuCLpSpe
         SxvgMdBfMqEVaiJ5J3m/EoYts9rsxzITSMM1PUKcDfFhwolc5Q6zvJbC6v9CYEgjcb0p
         N8zcnUsUItqlQ/YRBcnRvJ7IRzZBNc49O1ijZ76ktLuWV8c+yDySBLoNdE2yXwRhBGFJ
         Dq0g==
X-Gm-Message-State: AOAM530GwpKgJz4EmOuLAIEU83PQhE1ciig0Ih3z24tYjtPdJ+l6VyMK
	4pc/k6jRiRbldLMN6EjV7uJukhUin4Uv1Q==
X-Google-Smtp-Source: ABdhPJyz/2eLETM/dg1zwLYUM7sH88/cKFfkgBOAD9IXz8HIxSWRicPrSyibPbN72DecR+WVZpqeNQ==
X-Received: by 2002:a0c:b3c9:: with SMTP id b9mr11688934qvf.40.1632512423894;
        Fri, 24 Sep 2021 12:40:23 -0700 (PDT)
From: Alex Olson <this.is.a0lson@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Alex Olson <this.is.a0lson@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Ian Jackson <iwj@xenproject.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 0/1] x86: centralize default APIC id definition
Date: Fri, 24 Sep 2021 14:39:54 -0500
Message-Id: <cover.1632512149.git.this.is.a0lson@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

I am interested in making the x86 topology seen by guests more flexible.
This patch keeps the original functionality but allows the APIC identifier
seen by guests for each vCPU to be altered more easily in future revisions.

Since the same mapping of vcpu_id to vlapic id is currently preserved,
the existing adjustments for 'logical processors per package' are left unchanged.

Alex Olson (1):
  x86: centralize default APIC id definition

 tools/firmware/hvmloader/Makefile      |  2 +-
 tools/firmware/hvmloader/config.h      |  1 -
 tools/firmware/hvmloader/hvmloader.c   |  3 +-
 tools/firmware/hvmloader/mp_tables.c   |  3 +-
 tools/firmware/hvmloader/smp.c         |  3 +-
 tools/firmware/hvmloader/topology.c    | 54 ++++++++++++++++++++++++++
 tools/firmware/hvmloader/topology.h    | 33 ++++++++++++++++
 tools/firmware/hvmloader/util.c        |  6 ++-
 tools/include/xenctrl.h                |  6 +++
 tools/libacpi/build.c                  |  4 +-
 tools/libacpi/libacpi.h                |  3 +-
 tools/libs/ctrl/xc_domain.c            | 27 +++++++++++++
 tools/libs/light/libxl_x86_acpi.c      |  9 ++++-
 xen/arch/x86/cpuid.c                   | 14 +++++--
 xen/arch/x86/hvm/hvm.c                 | 36 ++++++++++++++++-
 xen/arch/x86/hvm/vlapic.c              | 18 ++++++---
 xen/include/asm-x86/hvm/vlapic.h       |  4 +-
 xen/include/public/arch-x86/hvm/save.h |  1 +
 xen/include/public/hvm/hvm_op.h        | 17 ++++++++
 19 files changed, 222 insertions(+), 22 deletions(-)
 create mode 100644 tools/firmware/hvmloader/topology.c
 create mode 100644 tools/firmware/hvmloader/topology.h

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Sat Sep 25 04:41:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 04:41:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195800.348683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzVb-0006CG-PS; Sat, 25 Sep 2021 04:41:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195800.348683; Sat, 25 Sep 2021 04:41:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mTzVb-0006C9-Lq; Sat, 25 Sep 2021 04:41:55 +0000
Received: by outflank-mailman (input) for mailman id 195800;
 Sat, 25 Sep 2021 04:41:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTzVb-0006Bz-55; Sat, 25 Sep 2021 04:41:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTzVa-0008VP-RU; Sat, 25 Sep 2021 04:41:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mTzVa-0005Sz-GI; Sat, 25 Sep 2021 04:41:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mTzVa-00012S-FQ; Sat, 25 Sep 2021 04:41:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=a9cWSpVkBzm8y/7sCRuAikYK+SiCrk2WmGEHKRWYcK4=; b=4NyeO3oEY19ebCGDHTlxXhS8w/
	setKbNvk9bgCZN2QM1dz+hGeWG8/Zc+v33/kmjt2oW/pJvNofcbUV19kiu4AcP2LMyTvRaJT480c3
	m8cQUNiCe9rR4i5ex+NhtSnfH61ZgnG0gYx7hZJBviiMwYz+KyRjIX7Jm2xZGkctCgTQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165185-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165185: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-arm64-arm64-xl:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-xsm:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-credit2:debian-fixup:fail:regression
    linux-linus:test-arm64-arm64-xl-thunderx:debian-fixup:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4c4f0c2bf3415a5fe7512d17d841bc1f8020b2cb
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 04:41:54 +0000

flight 165185 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165185/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-arm64-arm64-xl          13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm      13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  13 debian-fixup             fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 13 debian-fixup             fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4c4f0c2bf3415a5fe7512d17d841bc1f8020b2cb
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  420 days
Failing since        152366  2020-08-01 20:49:34 Z  419 days  733 attempts
Testing same since   165185  2021-09-24 18:12:09 Z    0 days    1 attempts

------------------------------------------------------------
7328 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          fail    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 fail    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2266771 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 11:06:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 11:06:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195862.348706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU5VF-00024j-IP; Sat, 25 Sep 2021 11:05:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195862.348706; Sat, 25 Sep 2021 11:05:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU5VF-00024c-FQ; Sat, 25 Sep 2021 11:05:57 +0000
Received: by outflank-mailman (input) for mailman id 195862;
 Sat, 25 Sep 2021 11:05:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU5VE-00024S-3u; Sat, 25 Sep 2021 11:05:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU5VD-0007ON-Tc; Sat, 25 Sep 2021 11:05:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU5VD-0001mS-Hg; Sat, 25 Sep 2021 11:05:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mU5VD-0007Vv-H6; Sat, 25 Sep 2021 11:05:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nhalkRrDZqBqDmo9fivm3bTnYQRhXHicD4O2Z7zy0jY=; b=kE3A2t9MWDMoh0mJg4FFpjt5QH
	fqQb6h1Htfm5O++BsxC1wgKHnKwqbtjhiP9RQRy1xexa0rmRSSPOcpGVoa4QCcLSaeW1PVnd9QYih
	ZnPJO2mZOK9BjUWenQZp2/qtlv8c686cs13UDIdHOGuN8yFIBxFnhK5IE/29yiaLQVgA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165186-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165186: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=e749ea24791e40d22fc9d21e248a30d69dd847db
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 11:05:55 +0000

flight 165186 qemu-mainline real [real]
flight 165192 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165186/
http://logs.test-lab.xenproject.org/osstest/logs/165192/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     18 guest-localmigrate       fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                e749ea24791e40d22fc9d21e248a30d69dd847db
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   13 days
Failing since        164967  2021-09-13 13:06:52 Z   11 days   23 attempts
Testing same since   165186  2021-09-24 22:39:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 4916 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 14:07:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 14:07:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195876.348720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8KT-0007ID-IX; Sat, 25 Sep 2021 14:07:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195876.348720; Sat, 25 Sep 2021 14:07:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8KT-0007I6-Ds; Sat, 25 Sep 2021 14:07:01 +0000
Received: by outflank-mailman (input) for mailman id 195876;
 Sat, 25 Sep 2021 14:07:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8KS-0006zv-6h; Sat, 25 Sep 2021 14:07:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8KS-00024B-1u; Sat, 25 Sep 2021 14:07:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8KR-0000cg-NF; Sat, 25 Sep 2021 14:06:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8KR-0001UR-Mp; Sat, 25 Sep 2021 14:06:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aRB0mYzWjXJvtlrt0M2K61BQQ0AMNuNkfEEhgSKgEwQ=; b=AZ2jLpYBSXg/gBaHe+JIA1X33T
	svIyvaozU5LznekLsvRlKfIROJkQAjXNdmqgoC19M9oNn164r2FNrl8moBp7M8vJjjhF9M4eN9LgG
	9KaqPgq9bFCEJQKRcsrSUC46M6flD4WqMe3PdDwpxdacdP3d0oUQ0ZvBN9YmJCFrfs8o=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165189-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165189: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=7c9832027874f08518442725385cda5992c5f643
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 14:06:59 +0000

flight 165189 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165189/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              7c9832027874f08518442725385cda5992c5f643
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  442 days
Failing since        151818  2020-07-11 04:18:52 Z  441 days  432 attempts
Testing same since   165189  2021-09-25 04:20:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 75905 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 14:31:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 14:31:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195882.348733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8i1-0002ZO-DG; Sat, 25 Sep 2021 14:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195882.348733; Sat, 25 Sep 2021 14:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8i1-0002ZH-AK; Sat, 25 Sep 2021 14:31:21 +0000
Received: by outflank-mailman (input) for mailman id 195882;
 Sat, 25 Sep 2021 14:31:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8hz-0002Z7-Pr; Sat, 25 Sep 2021 14:31:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8hz-0002Sy-K9; Sat, 25 Sep 2021 14:31:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8hz-0001ye-6s; Sat, 25 Sep 2021 14:31:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mU8hz-00085E-6S; Sat, 25 Sep 2021 14:31:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xhXHcTE5KKybPVnBWg51iLUk9APkgm6Sk3L3FZHp7aM=; b=eHnH5tV0kmafHU8jE8YgbS/0fU
	HLfTy/7140uWaPTYSHrnOb1ym4DCwBQCtI+igWCm1CFDSD/5d+r9mSjrxOAKAjSf0wrBS8jfvZORh
	q3IFSyscKSxzGDR/BNqq0GyRbn8Wq3/nGqHu+PgS6Go6HVR3sLM1cGMZfB6oZbiPjtNI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165188-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165188: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:build-amd64-prev:xen-build:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-i386-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 14:31:19 +0000

flight 165188 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165188/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-prev              6 xen-build                fail REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-migrupgrade  1 build-check(1)               blocked  n/a
 test-amd64-i386-migrupgrade   1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   14 days
Failing since        164951  2021-09-12 00:14:36 Z   13 days   27 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             fail    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 blocked 
 test-amd64-i386-migrupgrade                                  blocked 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 14:31:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 14:31:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195884.348747 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8i9-0002sR-QW; Sat, 25 Sep 2021 14:31:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195884.348747; Sat, 25 Sep 2021 14:31:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU8i9-0002sI-NJ; Sat, 25 Sep 2021 14:31:29 +0000
Received: by outflank-mailman (input) for mailman id 195884;
 Sat, 25 Sep 2021 14:31:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nlhD=OP=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mU8i8-0002rk-O2
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 14:31:28 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 442a34d0-1e0d-11ec-bb87-12813bfff9fa;
 Sat, 25 Sep 2021 14:31:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9BDAD1FE78;
 Sat, 25 Sep 2021 14:31:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 756E913398;
 Sat, 25 Sep 2021 14:31:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 3cZVG74yT2EtEwAAMHmgww
 (envelope-from <jgross@suse.com>); Sat, 25 Sep 2021 14:31:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 442a34d0-1e0d-11ec-bb87-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632580286; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Wpvi9Btcg02A0UWx/KC6Hh6eVUMEVHaSr7Cmxo2893M=;
	b=VwYoSAw+lzQPAiJURGqOKsrbOlP0YIeCHCp0NI2uaSeIJun3pGIIaknmfLI9b6L9J0FXuV
	vlrPIovlybqiEIQZNzNUcDuIVkFla6eWNHDe3DdYFgfwqnLqGRtCZIhnxjRNSxfhCPy9LR
	z/8EwUltrFeOjWe630C4a8r0B2bi8AM=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	boris.ostrovsky@oracle.com
Subject: [GIT PULL] xen: branch for v5.15-rc3
Date: Sat, 25 Sep 2021 16:31:26 +0200
Message-Id: <20210925143126.26439-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc3-tag

xen: branch for v5.15-rc3

It contains some minor cleanups and fixes of some theoretical bugs, as
well as a fix of a bug introduced in 5.15-rc1.

Thanks.

Juergen

 arch/x86/Kconfig                       |  1 -
 arch/x86/include/asm/xen/swiotlb-xen.h |  6 +-----
 arch/x86/xen/enlighten_pv.c            | 15 +++++++++------
 arch/x86/xen/pci-swiotlb-xen.c         |  4 ++--
 arch/x86/xen/smp_pv.c                  |  4 ----
 drivers/pci/Kconfig                    |  2 +-
 drivers/xen/Kconfig                    |  1 +
 drivers/xen/balloon.c                  |  4 ++--
 drivers/xen/gntdev.c                   |  8 ++++++++
 drivers/xen/swiotlb-xen.c              |  7 ++++---
 include/xen/xen-ops.h                  | 12 ------------
 11 files changed, 28 insertions(+), 36 deletions(-)

Jan Beulich (7):
      xen/x86: drop redundant zeroing from cpu_initialize_context()
      Xen/gntdev: don't ignore kernel unmapping error
      swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
      PCI: only build xen-pcifront in PV-enabled environments
      xen/pci-swiotlb: reduce visibility of symbols
      swiotlb-xen: this is PV-only on x86
      xen/x86: fix PV trap handling on secondary processors

Juergen Gross (1):
      xen/balloon: fix balloon kthread freezing


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 14:52:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 14:52:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195895.348759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU925-00068S-Ir; Sat, 25 Sep 2021 14:52:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195895.348759; Sat, 25 Sep 2021 14:52:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mU925-00068L-F5; Sat, 25 Sep 2021 14:52:05 +0000
Received: by outflank-mailman (input) for mailman id 195895;
 Sat, 25 Sep 2021 14:52:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lVfW=OP=gmail.com=philippe.mathieu.daude@srs-us1.protection.inumbo.net>)
 id 1mU924-00068C-6Y
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 14:52:04 +0000
Received: from mail-wr1-x429.google.com (unknown [2a00:1450:4864:20::429])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 61fdaef3-827c-4cd1-85b7-902ff9a5ac01;
 Sat, 25 Sep 2021 14:52:03 +0000 (UTC)
Received: by mail-wr1-x429.google.com with SMTP id w29so36384886wra.8
 for <xen-devel@lists.xenproject.org>; Sat, 25 Sep 2021 07:52:03 -0700 (PDT)
Received: from x1w.. (118.red-83-35-24.dynamicip.rima-tde.net. [83.35.24.118])
 by smtp.gmail.com with ESMTPSA id
 20sm16920479wme.46.2021.09.25.07.52.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 25 Sep 2021 07:52:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 61fdaef3-827c-4cd1-85b7-902ff9a5ac01
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=I61ARzsVO4/KzAGYtcJMZn32sXSEJVm6F+jp7uLlVMM=;
        b=G+GmU72zjpBNcQjaJkUxR7Hm9Y3LE8KiQt/SI3NeS6OLKULhwLAVtEcL4ZDR0Jag3I
         9Iqu7RniWGbPuunBaeHSb4uaOW1HdC2SIyhodow0oCFW0XZoRHzN/zJPKDXAUsrTJYbM
         o0tDzvkIczPzHnDbHc0znNPAZVEXTt+qBGmRo4pyGYCa3HQJ83meRkO9SVxoKlo9UeJ1
         nIqgFF+VvYvmGPzkpUox87plIThC0qe8NZE+3Q2d5FskCrFpydmZtE6Xb6AqInYZwrDa
         YbNDzLTFdcU1ZqZ6fDRMfqsaJAvDTGh4DXeDd8S8Htmyi/O/u1OIGLRJSsN5WaOPH7nW
         CM7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=I61ARzsVO4/KzAGYtcJMZn32sXSEJVm6F+jp7uLlVMM=;
        b=vpm4cmKuEvDdDmg1kuVZCviZtheNJwTXkpbc+2fDO+Im3qkle+dytNAKZCDHDvJlTb
         64QB+HavPUmiSSEfcwn+QDvd3QXs+EO9ppjxAVfiWggupMWZXOCvnpH0LZJrecQvFUhJ
         aa7TnYI2wRMo6bv/fwpSCeg1TDE8aCpPlNKdGD18i0K2Dm1WWEaVT570yRP86mNLVlHA
         Ev5CmzDoLpjKrhNf12ZrxHi2MEypg3xxJsfuTem1Uq+Nko8K1iO3zzr5kbd37f8UtElN
         gK7aKpC2KcEwEMCKpW+VquqWomR3TKZkzTlhCW/jVHXaw3DSGuWm4UVK5zAflMvBHdbZ
         AzOQ==
X-Gm-Message-State: AOAM533lSP+zVBvYtXxzYt3lGOcgv4ts3GaspoDuTtp5uU3enwnFrGOZ
	4UkEdqjb1YbmZ0P7HI2JumA=
X-Google-Smtp-Source: ABdhPJzkjnLgo6lRvy/ft690imOHBkl2yBKGxN4EX4PM++nl4LReYCioSoopkYZjVGMaG0ouAce+jg==
X-Received: by 2002:a5d:6741:: with SMTP id l1mr17319095wrw.289.1632581522279;
        Sat, 25 Sep 2021 07:52:02 -0700 (PDT)
Sender: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philippe.mathieu.daude@gmail.com>
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
	=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Paul Durrant <paul@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH v7 09/40] accel/xen: Implement AccelOpsClass::has_work()
Date: Sat, 25 Sep 2021 16:50:47 +0200
Message-Id: <20210925145118.1361230-10-f4bug@amsat.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210925145118.1361230-1-f4bug@amsat.org>
References: <20210925145118.1361230-1-f4bug@amsat.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since there is no specific Xen handling for cpu_has_work() in
cpu_thread_is_idle(), implement Xen has_work() handler as a
simple 'return false' code.

Acked-by: Paul Durrant <paul@xen.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 accel/xen/xen-all.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index 69aa7d018b2..fe5a37fa2e6 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -215,11 +215,17 @@ static const TypeInfo xen_accel_type = {
     .class_init = xen_accel_class_init,
 };
 
+static bool xen_cpu_has_work(CPUState *cpu)
+{
+    return false;
+}
+
 static void xen_accel_ops_class_init(ObjectClass *oc, void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
     ops->create_vcpu_thread = dummy_start_vcpu_thread;
+    ops->has_work = xen_cpu_has_work;
 }
 
 static const TypeInfo xen_accel_ops_type = {
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Sat Sep 25 16:42:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 16:42:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195908.348769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUAkY-00038m-M4; Sat, 25 Sep 2021 16:42:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195908.348769; Sat, 25 Sep 2021 16:42:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUAkY-00038f-J9; Sat, 25 Sep 2021 16:42:06 +0000
Received: by outflank-mailman (input) for mailman id 195908;
 Sat, 25 Sep 2021 16:42:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUAkX-00038V-4x; Sat, 25 Sep 2021 16:42:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUAkW-0005Aj-RR; Sat, 25 Sep 2021 16:42:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUAkW-0008WE-FY; Sat, 25 Sep 2021 16:42:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUAkW-0004ro-Bj; Sat, 25 Sep 2021 16:42:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iCZWA9HdJGfydsUeSWJTIkmsNwDog6jQZePadE3dONI=; b=EqCxqrn/KaXzFkKr1KhTaQ5Q0U
	ZuMGNb5HOnq9++ZhcL2xwKPN/EZtFmuCGeKx3vsp7dZYs0Erk5B4oDs5EZxu06zbfjQeeJ3WvxTDZ
	9lUIEk23+1U4FiJp2BEDOnrQZru3Bacv2PQlOJ8gFzahN1UKibT/L/VJPBSF4hr1tGNI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165190-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165190: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-saverestore:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7d42e98182586f57f376406d033f05fe135edb75
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 16:42:04 +0000

flight 165190 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165190/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 16 guest-saverestore   fail baseline untested
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7d42e98182586f57f376406d033f05fe135edb75
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  420 days
Failing since        152366  2020-08-01 20:49:34 Z  419 days  734 attempts
Testing same since   165190  2021-09-25 04:44:43 Z    0 days    1 attempts

------------------------------------------------------------
7328 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2267184 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 18:20:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 18:20:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195919.348783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUCHw-0006u6-QM; Sat, 25 Sep 2021 18:20:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195919.348783; Sat, 25 Sep 2021 18:20:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUCHw-0006tz-NQ; Sat, 25 Sep 2021 18:20:40 +0000
Received: by outflank-mailman (input) for mailman id 195919;
 Sat, 25 Sep 2021 18:20:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ycym=OP=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mUCHu-0006tt-Md
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 18:20:38 +0000
Received: from mail-wr1-x430.google.com (unknown [2a00:1450:4864:20::430])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b3c9dd29-a3e9-456a-9ba3-88e4176f9acd;
 Sat, 25 Sep 2021 18:20:35 +0000 (UTC)
Received: by mail-wr1-x430.google.com with SMTP id d21so37527824wra.12
 for <xen-devel@lists.xenproject.org>; Sat, 25 Sep 2021 11:20:35 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id w23sm655925wmi.6.2021.09.25.11.20.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sat, 25 Sep 2021 11:20:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3c9dd29-a3e9-456a-9ba3-88e4176f9acd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=UFeanbXfO9Pnbtl1PTmUTkde5BUEOwUcxyA/6fxgacQ=;
        b=afazfJGTXMo45XFpqwLR2qoN5QjWQ38Qc3kL+4/YAehQ3DERBriUop1t+SK75UFUmJ
         BqlpigQrjeEjBwfB4+VG7Cbax3mxg+iMga1WaNRqlxBx8uv4ZDm5UbXkkIuaF440w+ON
         +jtNSahau4BWn0qCyFZ3zNlpe+hYuY/hiWcZyOn9gaG+bXwzwRpN6dRGeCGrZyJKUUlp
         fZ4JmgEZ4/It7+5mMVXsBis/gCvUi62rbl0wiSzZiFYeiZlybxcDBz0PGrUAPbQvNzee
         9KgMUIN5XJjDxcs7ZCe1P/FOzMQjt+csh6hbRN7C/4yzk2r6YS2sRUlt9lA/gzF2dUCT
         QRjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=UFeanbXfO9Pnbtl1PTmUTkde5BUEOwUcxyA/6fxgacQ=;
        b=y4LWK99nepJ+ezG0bWaWNS3OyEu8xBr97UfuEfVuGwdouDBoD4BDn0yURWD98T4uc8
         2pZFl3JI1vJJZbvru+c8p4PCtk3mh735Jvd7k87jUddhAplXZwrHO+6sxJUTAOHmcp+G
         UUEOfZS8E/k1JGG5ybQnPFeJXP0TM1NrcIGaFZMJq3I9aBj1yTyHcz6aQHYq8OfKY+OA
         wwyBQBlOe4oGiRUCUYOda8VXSBeVIxgVthqc5q539ZdFtmlGfE7LSPLYWrjRkPPi5EiL
         fZncP+UycfzeYiZHOgV0DZHk89vGZSE/ZZm1ZyGIDd3G2wBd95kWi0jRyeZxrLJ8b5v/
         cz8Q==
X-Gm-Message-State: AOAM5331k2GMSiMQMWXV8nxDqHxENs1ou+r0daJFfBKg44Qd3n81s9Ob
	G3zux9eNSW8bYHwoK4cYfd8=
X-Google-Smtp-Source: ABdhPJztRenbuaTJT0yIdIEfFlNlkveo8My0WD81j6she2ZZMB45B3MiaEckNy276JWE1FVerG9VTw==
X-Received: by 2002:a5d:4579:: with SMTP id a25mr18715744wrc.222.1632594034864;
        Sat, 25 Sep 2021 11:20:34 -0700 (PDT)
Subject: Re: [PATCH V3 2/3] xen/arm: Add handling of extended regions for Dom0
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
 <1632437334-12015-3-git-send-email-olekstysh@gmail.com>
 <alpine.DEB.2.21.2109241508460.17979@sstabellini-ThinkPad-T480s>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <61516584-a2c9-16e1-b935-f7151bb0b530@gmail.com>
Date: Sat, 25 Sep 2021 21:20:33 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.2109241508460.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 25.09.21 01:39, Stefano Stabellini wrote:

Hi Stefano

> On Fri, 24 Sep 2021, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The extended region (safe range) is a region of guest physical
>> address space which is unused and could be safely used to create
>> grant/foreign mappings instead of wasting real RAM pages from
>> the domain memory for establishing these mappings.
>>
>> The extended regions are chosen at the domain creation time and
>> advertised to it via "reg" property under hypervisor node in
>> the guest device-tree. As region 0 is reserved for grant table
>> space (always present), the indexes for extended regions are 1...N.
>> If extended regions could not be allocated for some reason,
>> Xen doesn't fail and behaves as usual, so only inserts region 0.
>>
>> Please note the following limitations:
>> - The extended region feature is only supported for 64-bit domain
>>    currently.
>> - The ACPI case is not covered.
>>
>> ***
>>
>> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
>> the algorithm to choose extended regions for it is different
>> in comparison with the algorithm for non-direct mapped DomU.
>> What is more, that extended regions should be chosen differently
>> whether IOMMU is enabled or not.
>>
>> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
>> holes found in host device-tree if otherwise. Make sure that
>> extended regions are 2MB-aligned and located within maximum possible
>> addressable physical memory range. The minimum size of extended
>> region is 64MB. The maximum number of extended regions is 128,
>> which is an artificial limit to minimize code changes (we reuse
>> struct meminfo to describe extended regions, so there are an array
>> field for 128 elements).
>>
>> It worth mentioning that unallocated memory solution (when the IOMMU
>> is disabled) will work safely until Dom0 is able to allocate memory
>> outside of the original range.
>>
>> Also introduce command line option to be able to globally enable or
>> disable support for extended regions for Dom0 (enabled by default).
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> Please note, we need to decide which approach to use in find_unallocated_memory(),
>> you can find details at:
>> https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/
>>
>> Changes RFC -> V2:
>>     - update patch description
>>     - drop uneeded "extended-region" DT property
>>
>> Changes V2 -> V3:
>>     - update patch description
>>     - add comment for "size" calculation in add_ext_regions()
>>     - clarify "end" calculation in find_unallocated_memory() and
>>       find_memory_holes()
>>     - only pick up regions with size >= 64MB
>>     - allocate reg dynamically instead of keeping on the stack in
>>       make_hypervisor_node()
>>     - do not show warning for 32-bit domain
>>     - drop Linux specific limits EXT_REGION_*
>>     - also cover "ranges" property in find_memory_holes()
>>     - add command line arg to enable/disable extended region support
>> ---
>>   docs/misc/xen-command-line.pandoc |   7 +
>>   xen/arch/arm/domain_build.c       | 280 +++++++++++++++++++++++++++++++++++++-
>>   2 files changed, 284 insertions(+), 3 deletions(-)
>>
>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>> index 177e656..3bb8eb7 100644
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -1081,6 +1081,13 @@ hardware domain is architecture dependent.
>>   Note that specifying zero as domU value means zero, while for dom0 it means
>>   to use the default.
>>   
>> +### ext_regions (Arm)
>> +> `= <boolean>`
>> +
>> +> Default : `true`
>> +
>> +Flag to globally enable or disable support for extended regions for dom0.
> I'd say:
>
> Flag to enable or disable extended regions for Dom0.
>
> Extended regions are ranges of unused address space exposed to Dom0 as
> "safe to use" for special memory mappings. Disable if your board device
> tree is incomplete.


ok, will update


>
>
>>   ### flask
>>   > `= permissive | enforcing | late | disabled`
>>   
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index d233d63..81997d5 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -34,6 +34,10 @@
>>   static unsigned int __initdata opt_dom0_max_vcpus;
>>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>>   
>> +/* If true, the extended regions support is enabled for dom0 */
>> +static bool __initdata opt_ext_regions = true;
>> +boolean_param("ext_regions", opt_ext_regions);
>> +
>>   static u64 __initdata dom0_mem;
>>   static bool __initdata dom0_mem_set;
>>   
>> @@ -886,6 +890,233 @@ static int __init make_memory_node(const struct domain *d,
>>       return res;
>>   }
>>   
>> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
>> +{
>> +    struct meminfo *ext_regions = data;
>> +    paddr_t start, size;
>> +
>> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
>> +        return 0;
>> +
>> +    /* Both start and size of the extended region should be 2MB aligned */
>> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
>> +    if ( start > e )
>> +        return 0;
>> +
>> +    /*
>> +     * e is actually "end-1" because it is called by rangeset functions
>> +     * which are inclusive of the last address.
>> +     */
>> +    e += 1;
>> +    size = (e - start) & ~(SZ_2M - 1);
>> +    if ( size < MB(64) )
>> +        return 0;
>> +
>> +    ext_regions->bank[ext_regions->nr_banks].start = start;
>> +    ext_regions->bank[ext_regions->nr_banks].size = size;
>> +    ext_regions->nr_banks++;
>> +
>> +    return 0;
>> +}
> This is a lot better

Great!


>
>
>> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
>> +                                          struct meminfo *ext_regions)
>> +{
>> +    const struct meminfo *assign_mem = &kinfo->mem;
>> +    struct rangeset *unalloc_mem;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>> +
>> +    dt_dprintk("Find unallocated memory for extended regions\n");
>> +
>> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
>> +    if ( !unalloc_mem )
>> +        return -ENOMEM;
>> +
>> +    /* Start with all available RAM */
>> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.mem.bank[i].start;
>> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
>> +        res = rangeset_add_range(unalloc_mem, start, end - 1);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove RAM assigned to Dom0 */
>> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
>> +    {
>> +        start = assign_mem->bank[i].start;
>> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
>> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove reserved-memory regions */
>> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
>> +    {
>> +        start = bootinfo.reserved_mem.bank[i].start;
>> +        end = bootinfo.reserved_mem.bank[i].start +
>> +            bootinfo.reserved_mem.bank[i].size;
>> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                   start, end);
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    /* Remove grant table region */
>> +    start = kinfo->gnttab_start;
>> +    end = kinfo->gnttab_start + kinfo->gnttab_size;
>> +    res = rangeset_remove_range(unalloc_mem, start, end - 1);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    start = 0;
>> +    end = (1ULL << p2m_ipa_bits) - 1;
>> +    res = rangeset_report_ranges(unalloc_mem, start, end,
>> +                                 add_ext_regions, ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(unalloc_mem);
>> +
>> +    return res;
>> +}
>> +
>> +static int __init find_memory_holes(const struct kernel_info *kinfo,
>> +                                    struct meminfo *ext_regions)
>> +{
>> +    struct dt_device_node *np;
>> +    struct rangeset *mem_holes;
>> +    paddr_t start, end;
>> +    unsigned int i;
>> +    int res;
>> +
>> +    dt_dprintk("Find memory holes for extended regions\n");
>> +
>> +    mem_holes = rangeset_new(NULL, NULL, 0);
>> +    if ( !mem_holes )
>> +        return -ENOMEM;
>> +
>> +    /* Start with maximum possible addressable physical memory range */
>> +    start = 0;
>> +    end = (1ULL << p2m_ipa_bits) - 1;
>> +    res = rangeset_add_range(mem_holes, start, end);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
>> +               start, end);
>> +        goto out;
>> +    }
>> +
>> +    /*
>> +     * Remove regions described by "reg" and "ranges" properties where
>> +     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
>> +     */
>> +    dt_for_each_device_node( dt_host, np )
>> +    {
>> +        unsigned int naddr;
>> +        u64 addr, size;
>> +
>> +        naddr = dt_number_of_address(np);
>> +
>> +        for ( i = 0; i < naddr; i++ )
>> +        {
>> +            res = dt_device_get_address(np, i, &addr, &size);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
>> +                       i, dt_node_full_name(np));
>> +                goto out;
>> +            }
>> +
>> +            start = addr & PAGE_MASK;
> PAGE_ALIGN(addr)

Let's imagine we have reg = <0x0 0xee080200 0x0 0x700>;
So using PAGE_ALIGN(0xee080200) we will get a result aligned up - 
0xee081000, but this is not what we want here. But, the end should be 
aligned up.


>
>
>> +            end = PAGE_ALIGN(addr + size);
>> +            res = rangeset_remove_range(mem_holes, start, end - 1);
>> +            if ( res )
>> +            {
>> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                       start, end);
>> +                goto out;
>> +            }
>> +        }
>> +
>> +        if ( dt_device_type_is_equal(np, "pci" ) )
>> +        {
>> +            unsigned int range_size, nr_ranges;
>> +            int na, ns, pna;
>> +            const __be32 *ranges;
>> +            u32 len;
>> +
>> +            /*
>> +             * Looking for non-empty ranges property which in the context
>> +             * of the PCI host bridge device node describes the BARs for
>> +             * the PCI devices.
> I'd say that "it describes the PCI host bridge aperture"

ok, will update


>
>
>> +             */
>> +            ranges = dt_get_property(np, "ranges", &len);
>> +            if ( !ranges || !len )
>> +                continue;
>> +
>> +            pna = dt_n_addr_cells(np);
>> +            na = dt_child_n_addr_cells(np);
>> +            ns = dt_child_n_size_cells(np);
>> +
>> +            range_size = pna + na + ns;
>> +
>> +            nr_ranges = len / sizeof(__be32) / range_size;
>> +
>> +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
>> +            {
>> +                /* Skip the child address and get the parent (CPU) address */
>> +                addr = dt_read_number(ranges + na, pna);
>> +
>> +                size = dt_read_number(ranges + na + pna, ns);
> Parsing the PCI ranges property is never intuitive, but everything
> checks out as far as I can tell
>
>
>> +                start = addr & PAGE_MASK;
> PAGE_ALIGN(start)

same here


>
>
>> +                end = PAGE_ALIGN(addr + size);
>> +                res = rangeset_remove_range(mem_holes, start, end - 1);
>> +                if ( res )
>> +                {
>> +                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
>> +                           start, end);
>> +                    goto out;
>> +                }
>> +            }
>> +        }
>> +    }
>> +
>> +    start = 0;
>> +    end = (1ULL << p2m_ipa_bits) - 1;
>> +    res = rangeset_report_ranges(mem_holes, start, end,
>> +                                 add_ext_regions,  ext_regions);
>> +    if ( res )
>> +        ext_regions->nr_banks = 0;
>> +    else if ( !ext_regions->nr_banks )
>> +        res = -ENOENT;
>> +
>> +out:
>> +    rangeset_destroy(mem_holes);
>> +
>> +    return res;
>> +}
>> +
>>   static int __init make_hypervisor_node(struct domain *d,
>>                                          const struct kernel_info *kinfo,
>>                                          int addrcells, int sizecells)
>> @@ -893,11 +1124,12 @@ static int __init make_hypervisor_node(struct domain *d,
>>       const char compat[] =
>>           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>>           "xen,xen";
>> -    __be32 reg[4];
>> +    __be32 *reg, *cells;
>>       gic_interrupt_t intr;
>> -    __be32 *cells;
>>       int res;
>>       void *fdt = kinfo->fdt;
>> +    struct meminfo *ext_regions;
>> +    unsigned int i;
>>   
>>       dt_dprintk("Create hypervisor node\n");
>>   
>> @@ -919,12 +1151,54 @@ static int __init make_hypervisor_node(struct domain *d,
>>       if ( res )
>>           return res;
>>   
>> +    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
>> +    if ( !reg )
>> +        return -ENOMEM;
> Instead of (NR_MEM_BANKS + 1) * 4, shouldn't it be:
>
> (ext_regions->nr_banks + 1) * (addrcells + sizecells)
>
> ?

I think, yes ... But, with other modifications. Please see below


>
> xzalloc_array allocates a number of elements, not a number of bytes.
>
>
>> +
>> +    ext_regions = xzalloc(struct meminfo);
>> +    if ( !ext_regions )
>> +    {
>> +        xfree(reg);
>> +        return -ENOMEM;
>> +    }
>> +
>> +    if ( !opt_ext_regions )
>> +        printk(XENLOG_DEBUG "The extended regions support is disabled\n");
>> +    else if ( is_32bit_domain(d) )
>> +        printk(XENLOG_DEBUG "The extended regions are only supported for 64-bit guest currently\n");
> These checks should be before the memory allocations

Good point! Indeed there is no sense of whole "ext_regions" allocations 
if we are not going to handle extended regions. Also there is no need
to allocate "reg" array with maximum possible elements in advance 
(NR_MEM_BANKS + 1) * 4, we can allocate it afterwards when we clearly 
know how many elements we really need
(nr_ext_regions + 1) * 4, as you suggested above.


Below the changes to this function on top of current patch:

@@ -1128,8 +1127,8 @@ static int __init make_hypervisor_node(struct 
domain *d,
      gic_interrupt_t intr;
      int res;
      void *fdt = kinfo->fdt;
-    struct meminfo *ext_regions;
-    unsigned int i;
+    struct meminfo *ext_regions = NULL;
+    unsigned int i, nr_ext_regions;

      dt_dprintk("Create hypervisor node\n");

@@ -1151,23 +1150,22 @@ static int __init make_hypervisor_node(struct 
domain *d,
      if ( res )
          return res;

-    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
-    if ( !reg )
-        return -ENOMEM;
-
-    ext_regions = xzalloc(struct meminfo);
-    if ( !ext_regions )
-    {
-        xfree(reg);
-        return -ENOMEM;
-    }
-
      if ( !opt_ext_regions )
+    {
          printk(XENLOG_DEBUG "The extended regions support is disabled\n");
+        nr_ext_regions = 0;
+    }
      else if ( is_32bit_domain(d) )
+    {
          printk(XENLOG_DEBUG "The extended regions are only supported 
for 64-bit guest currently\n");
+        nr_ext_regions = 0;
+    }
      else
      {
+        ext_regions = xzalloc(struct meminfo);
+        if ( !ext_regions )
+            return -ENOMEM;
+
          if ( !is_iommu_enabled(d) )
              res = find_unallocated_memory(kinfo, ext_regions);
          else
@@ -1175,6 +1173,14 @@ static int __init make_hypervisor_node(struct 
domain *d,

          if ( res )
              printk(XENLOG_WARNING "Failed to allocate extended 
regions\n");
+        nr_ext_regions = ext_regions->nr_banks;
+    }
+
+    reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + 
sizecells));
+    if ( !reg )
+    {
+        xfree(ext_regions);
+        return -ENOMEM;
      }

      /* reg 0 is grant table space */
@@ -1182,7 +1188,7 @@ static int __init make_hypervisor_node(struct 
domain *d,
      dt_child_set_range(&cells, addrcells, sizecells,
                         kinfo->gnttab_start, kinfo->gnttab_size);
      /* reg 1...N are extended regions */
-    for ( i = 0; i < ext_regions->nr_banks; i++ )
+    for ( i = 0; i < nr_ext_regions; i++ )
      {
          u64 start = ext_regions->bank[i].start;
          u64 size = ext_regions->bank[i].size;
@@ -1195,7 +1201,7 @@ static int __init make_hypervisor_node(struct 
domain *d,

      res = fdt_property(fdt, "reg", reg,
                         dt_cells_to_size(addrcells + sizecells) *
-                       (ext_regions->nr_banks + 1));
+                       (nr_ext_regions + 1));
      xfree(ext_regions);
      xfree(reg);

(END)



>
>
>> +    else
>> +    {
>> +        if ( !is_iommu_enabled(d) )
>> +            res = find_unallocated_memory(kinfo, ext_regions);
>> +        else
>> +            res = find_memory_holes(kinfo, ext_regions);
>> +
>> +        if ( res )
>> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
>> +    }
>> +
>>       /* reg 0 is grant table space */
>>       cells = &reg[0];
>>       dt_child_set_range(&cells, addrcells, sizecells,
>>                          kinfo->gnttab_start, kinfo->gnttab_size);
>> +    /* reg 1...N are extended regions */
>> +    for ( i = 0; i < ext_regions->nr_banks; i++ )
>> +    {
>> +        u64 start = ext_regions->bank[i].start;
>> +        u64 size = ext_regions->bank[i].size;
>> +
>> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
>> +                   i, start, start + size);
>> +
>> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
>> +    }
>> +
>>       res = fdt_property(fdt, "reg", reg,
>> -                       dt_cells_to_size(addrcells + sizecells));
>> +                       dt_cells_to_size(addrcells + sizecells) *
>> +                       (ext_regions->nr_banks + 1));
>> +    xfree(ext_regions);
>> +    xfree(reg);
>> +
>>       if ( res )
>>           return res;
>>   
>> -- 
>> 2.7.4
>>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Sat Sep 25 19:08:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 19:08:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195927.348795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUD1f-00049O-Kj; Sat, 25 Sep 2021 19:07:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195927.348795; Sat, 25 Sep 2021 19:07:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUD1f-00049H-Hc; Sat, 25 Sep 2021 19:07:55 +0000
Received: by outflank-mailman (input) for mailman id 195927;
 Sat, 25 Sep 2021 19:07:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ba80=OP=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUD1d-00049B-Te
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 19:07:53 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e1f44aa3-1809-4e30-a1cb-c69b3bc0277d;
 Sat, 25 Sep 2021 19:07:51 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 846C160F21;
 Sat, 25 Sep 2021 19:07:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1f44aa3-1809-4e30-a1cb-c69b3bc0277d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632596870;
	bh=kd34QygbIleP1H0yNt/SatOlYdJs1iwAx5gw06RZmb0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tukiF5tooSFozgkaHO/H+xacT2B+K0N1UhKmIG0hAQxGJy0HGOHkFlUxmLURx/8/n
	 kV24x7Dm1i2AzUZq81G/+BbCnY4VfabyCupFu0zo+kFbwr/h5b6qkhqh30US2CmTki
	 +NZ3RXbppTVUtyZmiinfKVKGf5GvBHp/rYjbFTero97ISHSeDEUx0VXeYVprj+GgRO
	 ePaHbkJZ5sfJ7bhEDUYYTWzGq5lqQE8NzZm6e+G7zRcxmTjf+Pi5TG6NBKxQt5tuLo
	 11aDm5+Agz3wfbLD4Cmm761Pgrm6eyxtjVtmlS8UtCGa/ukH4saF5HDOmzy+XhaPdb
	 SMLH0XlJ+W4gw==
Date: Sat, 25 Sep 2021 12:07:41 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr <olekstysh@gmail.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH V3 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <61516584-a2c9-16e1-b935-f7151bb0b530@gmail.com>
Message-ID: <alpine.DEB.2.21.2109251201500.29224@sstabellini-ThinkPad-T480s>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com> <1632437334-12015-3-git-send-email-olekstysh@gmail.com> <alpine.DEB.2.21.2109241508460.17979@sstabellini-ThinkPad-T480s> <61516584-a2c9-16e1-b935-f7151bb0b530@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-731668745-1632596870=:29224"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-731668745-1632596870=:29224
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sat, 25 Sep 2021, Oleksandr wrote:
> On 25.09.21 01:39, Stefano Stabellini wrote:
> > On Fri, 24 Sep 2021, Oleksandr Tyshchenko wrote:
> > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > 
> > > The extended region (safe range) is a region of guest physical
> > > address space which is unused and could be safely used to create
> > > grant/foreign mappings instead of wasting real RAM pages from
> > > the domain memory for establishing these mappings.
> > > 
> > > The extended regions are chosen at the domain creation time and
> > > advertised to it via "reg" property under hypervisor node in
> > > the guest device-tree. As region 0 is reserved for grant table
> > > space (always present), the indexes for extended regions are 1...N.
> > > If extended regions could not be allocated for some reason,
> > > Xen doesn't fail and behaves as usual, so only inserts region 0.
> > > 
> > > Please note the following limitations:
> > > - The extended region feature is only supported for 64-bit domain
> > >    currently.
> > > - The ACPI case is not covered.
> > > 
> > > ***
> > > 
> > > As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> > > the algorithm to choose extended regions for it is different
> > > in comparison with the algorithm for non-direct mapped DomU.
> > > What is more, that extended regions should be chosen differently
> > > whether IOMMU is enabled or not.
> > > 
> > > Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> > > holes found in host device-tree if otherwise. Make sure that
> > > extended regions are 2MB-aligned and located within maximum possible
> > > addressable physical memory range. The minimum size of extended
> > > region is 64MB. The maximum number of extended regions is 128,
> > > which is an artificial limit to minimize code changes (we reuse
> > > struct meminfo to describe extended regions, so there are an array
> > > field for 128 elements).
> > > 
> > > It worth mentioning that unallocated memory solution (when the IOMMU
> > > is disabled) will work safely until Dom0 is able to allocate memory
> > > outside of the original range.
> > > 
> > > Also introduce command line option to be able to globally enable or
> > > disable support for extended regions for Dom0 (enabled by default).
> > > 
> > > Suggested-by: Julien Grall <jgrall@amazon.com>
> > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > ---
> > > Please note, we need to decide which approach to use in
> > > find_unallocated_memory(),
> > > you can find details at:
> > > https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/
> > > 
> > > Changes RFC -> V2:
> > >     - update patch description
> > >     - drop uneeded "extended-region" DT property
> > > 
> > > Changes V2 -> V3:
> > >     - update patch description
> > >     - add comment for "size" calculation in add_ext_regions()
> > >     - clarify "end" calculation in find_unallocated_memory() and
> > >       find_memory_holes()
> > >     - only pick up regions with size >= 64MB
> > >     - allocate reg dynamically instead of keeping on the stack in
> > >       make_hypervisor_node()
> > >     - do not show warning for 32-bit domain
> > >     - drop Linux specific limits EXT_REGION_*
> > >     - also cover "ranges" property in find_memory_holes()
> > >     - add command line arg to enable/disable extended region support
> > > ---
> > >   docs/misc/xen-command-line.pandoc |   7 +
> > >   xen/arch/arm/domain_build.c       | 280
> > > +++++++++++++++++++++++++++++++++++++-
> > >   2 files changed, 284 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/docs/misc/xen-command-line.pandoc
> > > b/docs/misc/xen-command-line.pandoc
> > > index 177e656..3bb8eb7 100644
> > > --- a/docs/misc/xen-command-line.pandoc
> > > +++ b/docs/misc/xen-command-line.pandoc
> > > @@ -1081,6 +1081,13 @@ hardware domain is architecture dependent.
> > >   Note that specifying zero as domU value means zero, while for dom0 it
> > > means
> > >   to use the default.
> > >   +### ext_regions (Arm)
> > > +> `= <boolean>`
> > > +
> > > +> Default : `true`
> > > +
> > > +Flag to globally enable or disable support for extended regions for dom0.
> > I'd say:
> > 
> > Flag to enable or disable extended regions for Dom0.
> > 
> > Extended regions are ranges of unused address space exposed to Dom0 as
> > "safe to use" for special memory mappings. Disable if your board device
> > tree is incomplete.
> 
> 
> ok, will update
> 
> 
> > 
> > 
> > >   ### flask
> > >   > `= permissive | enforcing | late | disabled`
> > >   diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > index d233d63..81997d5 100644
> > > --- a/xen/arch/arm/domain_build.c
> > > +++ b/xen/arch/arm/domain_build.c
> > > @@ -34,6 +34,10 @@
> > >   static unsigned int __initdata opt_dom0_max_vcpus;
> > >   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
> > >   +/* If true, the extended regions support is enabled for dom0 */
> > > +static bool __initdata opt_ext_regions = true;
> > > +boolean_param("ext_regions", opt_ext_regions);
> > > +
> > >   static u64 __initdata dom0_mem;
> > >   static bool __initdata dom0_mem_set;
> > >   @@ -886,6 +890,233 @@ static int __init make_memory_node(const struct
> > > domain *d,
> > >       return res;
> > >   }
> > >   +static int __init add_ext_regions(unsigned long s, unsigned long e,
> > > void *data)
> > > +{
> > > +    struct meminfo *ext_regions = data;
> > > +    paddr_t start, size;
> > > +
> > > +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> > > +        return 0;
> > > +
> > > +    /* Both start and size of the extended region should be 2MB aligned
> > > */
> > > +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> > > +    if ( start > e )
> > > +        return 0;
> > > +
> > > +    /*
> > > +     * e is actually "end-1" because it is called by rangeset functions
> > > +     * which are inclusive of the last address.
> > > +     */
> > > +    e += 1;
> > > +    size = (e - start) & ~(SZ_2M - 1);
> > > +    if ( size < MB(64) )
> > > +        return 0;
> > > +
> > > +    ext_regions->bank[ext_regions->nr_banks].start = start;
> > > +    ext_regions->bank[ext_regions->nr_banks].size = size;
> > > +    ext_regions->nr_banks++;
> > > +
> > > +    return 0;
> > > +}
> > This is a lot better
> 
> Great!
> 
> 
> > 
> > 
> > > +static int __init find_unallocated_memory(const struct kernel_info
> > > *kinfo,
> > > +                                          struct meminfo *ext_regions)
> > > +{
> > > +    const struct meminfo *assign_mem = &kinfo->mem;
> > > +    struct rangeset *unalloc_mem;
> > > +    paddr_t start, end;
> > > +    unsigned int i;
> > > +    int res;
> > > +
> > > +    dt_dprintk("Find unallocated memory for extended regions\n");
> > > +
> > > +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> > > +    if ( !unalloc_mem )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with all available RAM */
> > > +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.mem.bank[i].start;
> > > +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> > > +        res = rangeset_add_range(unalloc_mem, start, end - 1);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove RAM assigned to Dom0 */
> > > +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> > > +    {
> > > +        start = assign_mem->bank[i].start;
> > > +        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove reserved-memory regions */
> > > +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> > > +    {
> > > +        start = bootinfo.reserved_mem.bank[i].start;
> > > +        end = bootinfo.reserved_mem.bank[i].start +
> > > +            bootinfo.reserved_mem.bank[i].size;
> > > +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > > +        if ( res )
> > > +        {
> > > +            printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                   start, end);
> > > +            goto out;
> > > +        }
> > > +    }
> > > +
> > > +    /* Remove grant table region */
> > > +    start = kinfo->gnttab_start;
> > > +    end = kinfo->gnttab_start + kinfo->gnttab_size;
> > > +    res = rangeset_remove_range(unalloc_mem, start, end - 1);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    start = 0;
> > > +    end = (1ULL << p2m_ipa_bits) - 1;
> > > +    res = rangeset_report_ranges(unalloc_mem, start, end,
> > > +                                 add_ext_regions, ext_regions);
> > > +    if ( res )
> > > +        ext_regions->nr_banks = 0;
> > > +    else if ( !ext_regions->nr_banks )
> > > +        res = -ENOENT;
> > > +
> > > +out:
> > > +    rangeset_destroy(unalloc_mem);
> > > +
> > > +    return res;
> > > +}
> > > +
> > > +static int __init find_memory_holes(const struct kernel_info *kinfo,
> > > +                                    struct meminfo *ext_regions)
> > > +{
> > > +    struct dt_device_node *np;
> > > +    struct rangeset *mem_holes;
> > > +    paddr_t start, end;
> > > +    unsigned int i;
> > > +    int res;
> > > +
> > > +    dt_dprintk("Find memory holes for extended regions\n");
> > > +
> > > +    mem_holes = rangeset_new(NULL, NULL, 0);
> > > +    if ( !mem_holes )
> > > +        return -ENOMEM;
> > > +
> > > +    /* Start with maximum possible addressable physical memory range */
> > > +    start = 0;
> > > +    end = (1ULL << p2m_ipa_bits) - 1;
> > > +    res = rangeset_add_range(mem_holes, start, end);
> > > +    if ( res )
> > > +    {
> > > +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> > > +               start, end);
> > > +        goto out;
> > > +    }
> > > +
> > > +    /*
> > > +     * Remove regions described by "reg" and "ranges" properties where
> > > +     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
> > > +     */
> > > +    dt_for_each_device_node( dt_host, np )
> > > +    {
> > > +        unsigned int naddr;
> > > +        u64 addr, size;
> > > +
> > > +        naddr = dt_number_of_address(np);
> > > +
> > > +        for ( i = 0; i < naddr; i++ )
> > > +        {
> > > +            res = dt_device_get_address(np, i, &addr, &size);
> > > +            if ( res )
> > > +            {
> > > +                printk(XENLOG_ERR "Unable to retrieve address %u for
> > > %s\n",
> > > +                       i, dt_node_full_name(np));
> > > +                goto out;
> > > +            }
> > > +
> > > +            start = addr & PAGE_MASK;
> > PAGE_ALIGN(addr)
> 
> Let's imagine we have reg = <0x0 0xee080200 0x0 0x700>;
> So using PAGE_ALIGN(0xee080200) we will get a result aligned up - 0xee081000,
> but this is not what we want here. But, the end should be aligned up.

You are right, we want to be conservative here, So PAGE_ALIGN is fine
for end, but for start we need "& PAGE_MASK".


 
> > > +            end = PAGE_ALIGN(addr + size);
> > > +            res = rangeset_remove_range(mem_holes, start, end - 1);
> > > +            if ( res )
> > > +            {
> > > +                printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                       start, end);
> > > +                goto out;
> > > +            }
> > > +        }
> > > +
> > > +        if ( dt_device_type_is_equal(np, "pci" ) )
> > > +        {
> > > +            unsigned int range_size, nr_ranges;
> > > +            int na, ns, pna;
> > > +            const __be32 *ranges;
> > > +            u32 len;
> > > +
> > > +            /*
> > > +             * Looking for non-empty ranges property which in the context
> > > +             * of the PCI host bridge device node describes the BARs for
> > > +             * the PCI devices.
> > I'd say that "it describes the PCI host bridge aperture"
> 
> ok, will update
> 
> 
> > 
> > 
> > > +             */
> > > +            ranges = dt_get_property(np, "ranges", &len);
> > > +            if ( !ranges || !len )
> > > +                continue;
> > > +
> > > +            pna = dt_n_addr_cells(np);
> > > +            na = dt_child_n_addr_cells(np);
> > > +            ns = dt_child_n_size_cells(np);
> > > +
> > > +            range_size = pna + na + ns;
> > > +
> > > +            nr_ranges = len / sizeof(__be32) / range_size;
> > > +
> > > +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
> > > +            {
> > > +                /* Skip the child address and get the parent (CPU)
> > > address */
> > > +                addr = dt_read_number(ranges + na, pna);
> > > +
> > > +                size = dt_read_number(ranges + na + pna, ns);
> > Parsing the PCI ranges property is never intuitive, but everything
> > checks out as far as I can tell
> > 
> > 
> > > +                start = addr & PAGE_MASK;
> > PAGE_ALIGN(start)
> 
> same here
> 
> 
> > 
> > 
> > > +                end = PAGE_ALIGN(addr + size);
> > > +                res = rangeset_remove_range(mem_holes, start, end - 1);
> > > +                if ( res )
> > > +                {
> > > +                    printk(XENLOG_ERR "Failed to remove:
> > > %#"PRIx64"->%#"PRIx64"\n",
> > > +                           start, end);
> > > +                    goto out;
> > > +                }
> > > +            }
> > > +        }
> > > +    }
> > > +
> > > +    start = 0;
> > > +    end = (1ULL << p2m_ipa_bits) - 1;
> > > +    res = rangeset_report_ranges(mem_holes, start, end,
> > > +                                 add_ext_regions,  ext_regions);
> > > +    if ( res )
> > > +        ext_regions->nr_banks = 0;
> > > +    else if ( !ext_regions->nr_banks )
> > > +        res = -ENOENT;
> > > +
> > > +out:
> > > +    rangeset_destroy(mem_holes);
> > > +
> > > +    return res;
> > > +}
> > > +
> > >   static int __init make_hypervisor_node(struct domain *d,
> > >                                          const struct kernel_info *kinfo,
> > >                                          int addrcells, int sizecells)
> > > @@ -893,11 +1124,12 @@ static int __init make_hypervisor_node(struct
> > > domain *d,
> > >       const char compat[] =
> > >           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
> > >           "xen,xen";
> > > -    __be32 reg[4];
> > > +    __be32 *reg, *cells;
> > >       gic_interrupt_t intr;
> > > -    __be32 *cells;
> > >       int res;
> > >       void *fdt = kinfo->fdt;
> > > +    struct meminfo *ext_regions;
> > > +    unsigned int i;
> > >         dt_dprintk("Create hypervisor node\n");
> > >   @@ -919,12 +1151,54 @@ static int __init make_hypervisor_node(struct
> > > domain *d,
> > >       if ( res )
> > >           return res;
> > >   +    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
> > > +    if ( !reg )
> > > +        return -ENOMEM;
> > Instead of (NR_MEM_BANKS + 1) * 4, shouldn't it be:
> > 
> > (ext_regions->nr_banks + 1) * (addrcells + sizecells)
> > 
> > ?
> 
> I think, yes ... But, with other modifications. Please see below
> 
> 
> > 
> > xzalloc_array allocates a number of elements, not a number of bytes.
> > 
> > 
> > > +
> > > +    ext_regions = xzalloc(struct meminfo);
> > > +    if ( !ext_regions )
> > > +    {
> > > +        xfree(reg);
> > > +        return -ENOMEM;
> > > +    }
> > > +
> > > +    if ( !opt_ext_regions )
> > > +        printk(XENLOG_DEBUG "The extended regions support is
> > > disabled\n");
> > > +    else if ( is_32bit_domain(d) )
> > > +        printk(XENLOG_DEBUG "The extended regions are only supported for
> > > 64-bit guest currently\n");
> > These checks should be before the memory allocations
> 
> Good point! Indeed there is no sense of whole "ext_regions" allocations if we
> are not going to handle extended regions. Also there is no need
> to allocate "reg" array with maximum possible elements in advance
> (NR_MEM_BANKS + 1) * 4, we can allocate it afterwards when we clearly know how
> many elements we really need
> (nr_ext_regions + 1) * 4, as you suggested above.
> 
> 
> Below the changes to this function on top of current patch:
> 
> @@ -1128,8 +1127,8 @@ static int __init make_hypervisor_node(struct domain *d,
>      gic_interrupt_t intr;
>      int res;
>      void *fdt = kinfo->fdt;
> -    struct meminfo *ext_regions;
> -    unsigned int i;
> +    struct meminfo *ext_regions = NULL;
> +    unsigned int i, nr_ext_regions;
> 
>      dt_dprintk("Create hypervisor node\n");
> 
> @@ -1151,23 +1150,22 @@ static int __init make_hypervisor_node(struct domain
> *d,
>      if ( res )
>          return res;
> 
> -    reg = xzalloc_array(__be32, (NR_MEM_BANKS + 1) * 4);
> -    if ( !reg )
> -        return -ENOMEM;
> -
> -    ext_regions = xzalloc(struct meminfo);
> -    if ( !ext_regions )
> -    {
> -        xfree(reg);
> -        return -ENOMEM;
> -    }
> -
>      if ( !opt_ext_regions )
> +    {
>          printk(XENLOG_DEBUG "The extended regions support is disabled\n");
> +        nr_ext_regions = 0;
> +    }
>      else if ( is_32bit_domain(d) )
> +    {
>          printk(XENLOG_DEBUG "The extended regions are only supported for
> 64-bit guest currently\n");
> +        nr_ext_regions = 0;
> +    }
>      else
>      {
> +        ext_regions = xzalloc(struct meminfo);
> +        if ( !ext_regions )
> +            return -ENOMEM;
> +
>          if ( !is_iommu_enabled(d) )
>              res = find_unallocated_memory(kinfo, ext_regions);
>          else
> @@ -1175,6 +1173,14 @@ static int __init make_hypervisor_node(struct domain
> *d,
> 
>          if ( res )
>              printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> +        nr_ext_regions = ext_regions->nr_banks;
> +    }
> +
> +    reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells +
> sizecells));
> +    if ( !reg )
> +    {
> +        xfree(ext_regions);
> +        return -ENOMEM;
>      }
> 
>      /* reg 0 is grant table space */
> @@ -1182,7 +1188,7 @@ static int __init make_hypervisor_node(struct domain *d,
>      dt_child_set_range(&cells, addrcells, sizecells,
>                         kinfo->gnttab_start, kinfo->gnttab_size);
>      /* reg 1...N are extended regions */
> -    for ( i = 0; i < ext_regions->nr_banks; i++ )
> +    for ( i = 0; i < nr_ext_regions; i++ )
>      {
>          u64 start = ext_regions->bank[i].start;
>          u64 size = ext_regions->bank[i].size;
> @@ -1195,7 +1201,7 @@ static int __init make_hypervisor_node(struct domain *d,
> 
>      res = fdt_property(fdt, "reg", reg,
>                         dt_cells_to_size(addrcells + sizecells) *
> -                       (ext_regions->nr_banks + 1));
> +                       (nr_ext_regions + 1));
>      xfree(ext_regions);
>      xfree(reg);
> 

Yep, that's what I meant, thanks
--8323329-731668745-1632596870=:29224--


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 22:04:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 22:04:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195940.348806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUFlx-0008MA-61; Sat, 25 Sep 2021 22:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195940.348806; Sat, 25 Sep 2021 22:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUFlx-0008M3-2y; Sat, 25 Sep 2021 22:03:53 +0000
Received: by outflank-mailman (input) for mailman id 195940;
 Sat, 25 Sep 2021 22:03:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUFlv-0008Lt-KE; Sat, 25 Sep 2021 22:03:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUFlv-0002Bs-Bm; Sat, 25 Sep 2021 22:03:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUFlv-0000MV-0v; Sat, 25 Sep 2021 22:03:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUFlv-0004gh-0J; Sat, 25 Sep 2021 22:03:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LkYUGGcADvgVMhcz3GVNAsKh7FgyeMmOHIpRV3AudWw=; b=vTv3++nYNgkZWZqsrj1QEdAhqF
	i5D2YZGzLNFeDEWeHN9ALGTsPEnh1aoEaIClBB7Sm/hNZE82A5aE5NsTKd2Dctq4u7NwkX1/WyE2k
	V1P45hGgrZXUtNN5cwrWMbcBk7Y2Fckt65Yr2d+wiRIIi9EmZfFi5Qo7IAOW3L4Ib6mg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165193-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165193: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=11a11998460ed84d9a127c025f50f7234e5a483f
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 22:03:51 +0000

flight 165193 qemu-mainline real [real]
flight 165196 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165193/
http://logs.test-lab.xenproject.org/osstest/logs/165196/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                11a11998460ed84d9a127c025f50f7234e5a483f
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   14 days
Failing since        164967  2021-09-13 13:06:52 Z   12 days   24 attempts
Testing same since   165193  2021-09-25 11:09:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5088 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 23:06:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 23:06:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195951.348819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUGk0-0007WY-6i; Sat, 25 Sep 2021 23:05:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195951.348819; Sat, 25 Sep 2021 23:05:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUGk0-0007WR-3M; Sat, 25 Sep 2021 23:05:56 +0000
Received: by outflank-mailman (input) for mailman id 195951;
 Sat, 25 Sep 2021 23:05:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5GXT=OP=kernel.org=pr-tracker-bot@srs-us1.protection.inumbo.net>)
 id 1mUGjy-0007WL-Bb
 for xen-devel@lists.xenproject.org; Sat, 25 Sep 2021 23:05:54 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 21c853b6-1e55-11ec-bbc9-12813bfff9fa;
 Sat, 25 Sep 2021 23:05:53 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPS id 83ADE60F3A;
 Sat, 25 Sep 2021 23:05:52 +0000 (UTC)
Received: from pdx-korg-docbuild-2.ci.codeaurora.org (localhost.localdomain
 [127.0.0.1])
 by pdx-korg-docbuild-2.ci.codeaurora.org (Postfix) with ESMTP id 7D760600E8;
 Sat, 25 Sep 2021 23:05:52 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21c853b6-1e55-11ec-bbc9-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632611152;
	bh=2FUsp8B+t6o61u2VSDNEqSK3jIOVD0jJY/RPX02w5jE=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=fYaXaYxa4RYC98MpdROIbMFMmT54RXOSVDnFIvVpMo7aT9z/xPHmyPcTjkyT+mKj0
	 fmhfpEe+Rx8WyiZ/VrA+JCxoS8R38+pfzKTNzfthLg37cx1Zie0lGDEMw8QtcZQNsq
	 868TTlkbRVCFZ7/e8ggyEEWl4luFmbfA1cy36fpwPynWSP/LQXJuMkI1ohNwThPO8g
	 7iIs708l1RHMt9YwiXKkx70z5kTImALP47LKKROnsL1H8FrBpw/OSGRg+LESQuWyLB
	 PLXKrapCpkDkFp13gLKM8/JuoygobCe0ulpLsL43QSCR3ClDkAbjvi4SIeE8sDbTU+
	 A59+7/DftYtOw==
Subject: Re: [GIT PULL] xen: branch for v5.15-rc3
From: pr-tracker-bot@kernel.org
In-Reply-To: <20210925143126.26439-1-jgross@suse.com>
References: <20210925143126.26439-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20210925143126.26439-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc3-tag
X-PR-Tracked-Commit-Id: 0594c58161b6e0f3da8efa9c6e3d4ba52b652717
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: 5739844347518a0f4c327ae79e73fb101d864726
Message-Id: <163261115250.2532.5770040613699339771.pr-tracker-bot@kernel.org>
Date: Sat, 25 Sep 2021 23:05:52 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, boris.ostrovsky@oracle.com

The pull request you sent on Sat, 25 Sep 2021 16:31:26 +0200:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc3-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/5739844347518a0f4c327ae79e73fb101d864726

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Sat Sep 25 23:31:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 25 Sep 2021 23:31:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195957.348831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUH8f-000350-7t; Sat, 25 Sep 2021 23:31:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195957.348831; Sat, 25 Sep 2021 23:31:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUH8f-00034t-4O; Sat, 25 Sep 2021 23:31:25 +0000
Received: by outflank-mailman (input) for mailman id 195957;
 Sat, 25 Sep 2021 23:31:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUH8d-00034j-T9; Sat, 25 Sep 2021 23:31:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUH8d-0003aF-LR; Sat, 25 Sep 2021 23:31:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUH8d-0004Bg-C9; Sat, 25 Sep 2021 23:31:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUH8d-0000bo-Bd; Sat, 25 Sep 2021 23:31:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=K1yGBRHvLP6D7AG9EEVogZzJYYDXPsYFz+K/qYE6w28=; b=X4XcTp+ZgMyxiLk3r1TJqJ2bdj
	m752bha7ZHfg6utXqmAED7EQ4uE15XrA50+X1s3xCMB+Vj5+6gEUy5Q3QH96gH/WRIuvflMp55mE+
	Z0nPYr8D3U2vVUrY4iurxxpS7YOF80GXeX15JcPOUh/5NXx6IYFH00/zk+KwRAbV++jo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165194-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165194: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:build-amd64-prev:xen-build:fail:regression
    xen-unstable:test-arm64-arm64-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-i386-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-migrupgrade:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 25 Sep 2021 23:31:23 +0000

flight 165194 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165194/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945
 build-amd64-prev              6 xen-build      fail in 165188 REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-vhd      17 guest-start/debian.repeat  fail pass in 165188

Tests which did not succeed, but are not blocking:
 test-amd64-i386-migrupgrade   1 build-check(1)           blocked in 165188 n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)           blocked in 165188 n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   15 days
Failing since        164951  2021-09-12 00:14:36 Z   13 days   28 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 02:46:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 02:46:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.195982.348845 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUKBI-0007jL-Mq; Sun, 26 Sep 2021 02:46:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 195982.348845; Sun, 26 Sep 2021 02:46:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUKBI-0007jB-H0; Sun, 26 Sep 2021 02:46:20 +0000
Received: by outflank-mailman (input) for mailman id 195982;
 Sun, 26 Sep 2021 02:46:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUKBH-0007j1-Bx; Sun, 26 Sep 2021 02:46:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUKBG-0005cw-Vr; Sun, 26 Sep 2021 02:46:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUKBG-00041a-IW; Sun, 26 Sep 2021 02:46:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUKBG-0005kp-I1; Sun, 26 Sep 2021 02:46:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hRQhgIAaQY8oYLTZthpv6VI8x9UBa/uzHxKiZTXpeVw=; b=w2XrJYcKyCDtInWYKsFy0+/CK4
	8rHWyeV/q1HOXEImXdKauxCl0e9nYwX5R+aQ2sn+2Gu9Y42L4eGLoawmt1H9dj/6oxywPJW2nR0Gb
	F0374VrSYeMRLRZ5iDx2On0xUYv5UoCMGwVDB19UPyvdRTOHjSgOnahNJfvRvsKWaS2A=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165195-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165195: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvshim:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-credit2:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-shadow:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-libvirt:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-libvirt-pair:guest-migrate/src_host/dst_host:fail:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:guest-saverestore:fail:regression
    linux-linus:test-amd64-coresched-amd64-xl:guest-saverestore:fail:regression
    linux-linus:test-amd64-amd64-xl-xsm:guest-saverestore:fail:regression
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:heisenbug
    linux-linus:test-amd64-amd64-examine:memdisk-try-append:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-rtds:guest-saverestore:fail:allowable
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-localmigrate/x10:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-saverestore:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7d42e98182586f57f376406d033f05fe135edb75
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 02:46:18 +0000

flight 165195 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165195/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvshim   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-amd 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-dom0pvh-xl-intel 17 guest-saverestore   fail REGR. vs. 152332
 test-amd64-amd64-xl          17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt-xsm 17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-credit1  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-multivcpu 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-intel 17 guest-saverestore     fail REGR. vs. 152332
 test-amd64-amd64-xl-credit2  17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-xl-shadow   17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-libvirt     17 guest-saverestore        fail REGR. vs. 152332
 test-amd64-amd64-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-libvirt-pair 26 guest-migrate/src_host/dst_host fail REGR. vs. 152332
 test-amd64-amd64-xl-pvhv2-amd 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-coresched-amd64-xl 17 guest-saverestore       fail REGR. vs. 152332
 test-amd64-amd64-xl-xsm      17 guest-saverestore        fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail in 165190 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 165190 pass in 165195
 test-armhf-armhf-xl-credit1  14 guest-start                fail pass in 165190
 test-amd64-amd64-examine      4 memdisk-try-append         fail pass in 165190

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     17 guest-saverestore        fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 19 guest-localmigrate/x10 fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-freebsd12-amd64 16 guest-saverestore fail in 165190 baseline untested
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail in 165190 baseline untested
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165190 like 152332
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 165190 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 165190 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7d42e98182586f57f376406d033f05fe135edb75
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  421 days
Failing since        152366  2020-08-01 20:49:34 Z  420 days  735 attempts
Testing same since   165190  2021-09-25 04:44:43 Z    0 days    2 attempts

------------------------------------------------------------
7328 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-coresched-amd64-xl                                fail    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      fail    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                fail    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  fail    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     fail    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                fail    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        fail    
 test-amd64-amd64-libvirt-pair                                fail    
 test-amd64-amd64-xl-pvshim                                   fail    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2267184 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 08:10:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 08:10:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196012.348858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUPF5-00050a-Hc; Sun, 26 Sep 2021 08:10:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196012.348858; Sun, 26 Sep 2021 08:10:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUPF5-00050T-Ec; Sun, 26 Sep 2021 08:10:35 +0000
Received: by outflank-mailman (input) for mailman id 196012;
 Sun, 26 Sep 2021 08:10:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUPF3-00050J-C2; Sun, 26 Sep 2021 08:10:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUPF3-0003UX-6N; Sun, 26 Sep 2021 08:10:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUPF2-0005bs-QL; Sun, 26 Sep 2021 08:10:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUPF2-0001kr-Pq; Sun, 26 Sep 2021 08:10:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mpxsew5NAqbFYUyUjFNnzxVdoBJprT902pb+7RHIQMc=; b=IYEPvOoVdMPkevP7ub0WDM85GT
	0hDw81ZTGK9EgoI/Q58St3MLsWNZ4WoQyjjDSgwnprGoapYF0b3erXAxlZ99ALFlfM8AAEWdWkehO
	utJx2FRAKbCMSXapNyoAWGUnw+vnYuIMEzUyArDVx+ZKulm/5SU4isLlBZmDtZnPHa2Q=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165197-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165197: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=11a11998460ed84d9a127c025f50f7234e5a483f
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 08:10:32 +0000

flight 165197 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165197/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds     14 guest-start                fail pass in 165193

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165193 REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 165193 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 165193 never pass
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                11a11998460ed84d9a127c025f50f7234e5a483f
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   14 days
Failing since        164967  2021-09-13 13:06:52 Z   12 days   25 attempts
Testing same since   165193  2021-09-25 11:09:10 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5088 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 10:02:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 10:02:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196036.348872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUQym-0001J4-Hm; Sun, 26 Sep 2021 10:01:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196036.348872; Sun, 26 Sep 2021 10:01:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUQym-0001Ix-Ev; Sun, 26 Sep 2021 10:01:52 +0000
Received: by outflank-mailman (input) for mailman id 196036;
 Sun, 26 Sep 2021 10:01:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUQyl-0001In-18; Sun, 26 Sep 2021 10:01:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUQyk-0005Ms-Tb; Sun, 26 Sep 2021 10:01:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUQyk-0002t0-K7; Sun, 26 Sep 2021 10:01:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUQyk-0001o4-Ja; Sun, 26 Sep 2021 10:01:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rZgBktrPpQRrWpns+E3nYmlZAdRm/Unj1fMABuB0kCU=; b=Y5JjvgcxnoaaAbGN/SabZqmQ0s
	j3EWDvBB62wz2c4D/mRhe2GwNUHH6+720BxiPdhpupMkasZeIT5uHQx4JMx6SOw/ZDAVU+/kbxevl
	hdXaBjcXacoQ3P7ut/eIVAQNcNlawgUQ3v7FiIziyhzG+YO/q9zFGwdlMyFSZPy3/Eeo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165203-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-coverity test] 165203: all pass - PUSHED
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=4e467d390eaf6736377766adae0166711866fac3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 10:01:50 +0000

flight 165203 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165203/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  4e467d390eaf6736377766adae0166711866fac3

Last test of basis   165150  2021-09-22 09:30:48 Z    4 days
Testing same since   165203  2021-09-26 09:19:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 coverity-amd64                                               pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4e467d390e..dd6c062a7a  dd6c062a7a4abdb662c18af03d1396325969d155 -> coverity-tested/smoke


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 10:12:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 10:12:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196045.348887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUR90-00032E-I4; Sun, 26 Sep 2021 10:12:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196045.348887; Sun, 26 Sep 2021 10:12:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUR90-000327-EX; Sun, 26 Sep 2021 10:12:26 +0000
Received: by outflank-mailman (input) for mailman id 196045;
 Sun, 26 Sep 2021 10:12:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ODXE=OQ=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUR8z-000321-Ap
 for xen-devel@lists.xenproject.org; Sun, 26 Sep 2021 10:12:25 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.53]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a5634ffb-a464-4df4-b4bd-dc5929300532;
 Sun, 26 Sep 2021 10:12:21 +0000 (UTC)
Received: from FR3P281CA0009.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::8) by
 DBBPR08MB4821.eurprd08.prod.outlook.com (2603:10a6:10:d5::22) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4523.18; Sun, 26 Sep 2021 10:12:08 +0000
Received: from VE1EUR03FT029.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:d10:1d:cafe::5) by FR3P281CA0009.outlook.office365.com
 (2603:10a6:d10:1d::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Sun, 26 Sep 2021 10:12:08 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT029.mail.protection.outlook.com (10.152.18.107) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Sun, 26 Sep 2021 10:12:07 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Sun, 26 Sep 2021 10:12:06 +0000
Received: from 5caf25f83aa4.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 321481F2-BD88-45DC-A39E-165477F68436.1; 
 Sun, 26 Sep 2021 10:11:54 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5caf25f83aa4.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Sun, 26 Sep 2021 10:11:54 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB4569.eurprd08.prod.outlook.com (2603:10a6:10:b2::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Sun, 26 Sep
 2021 10:11:52 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Sun, 26 Sep 2021
 10:11:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5634ffb-a464-4df4-b4bd-dc5929300532
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2CHNZMW4X8CflWPGMqk3n/4C28ulPxVyey+BzhzUUyI=;
 b=SsWqcw5+ljmFcFAuh6bWC8WftMVyQ3BARfJUdACrFycJP4/TfWdBLPcERsZMWIKXATDA4Q2uWQhvbhnDVhv/DDXrzVM3qKQT5hmfsfmB2AN1/daDnYW3SXBZv1G/Z7x8eWe/fzGS5U8aNrnMdnTTPBGQkFPtSdOuqcCgXpMi1X0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=evIq/tcA7CSHkkKc0NfjdrZx73WWXBu8Z48ztGYnXNlCGQFSsyRq1cM6qtZEJuwR26IvZAsY1E2ohn+xJIFFUqq6s3Di9UWc8I66dsGciUc2LWZtzza6K5H8YesZy5SgQdEAVrJWWYtLS58bpWY/3SqmvAp6MgVsk8eyBby+EzBMXPATWbEZ0xiMI4WfOQb34pfRAPJi8d0fCgUQ0z67YrhPvDVit5ADN0AURd/wnvj+l6jE3BfNDoq+8jSoTjH7NrIWeMaxT1bofb+ScbJTY/a+XDJYbhSYUUB0XWkMvC70j4ZWq5X9lqJ1pEBgTnhnuYlZdwSzf8jp9DzuoIqKkg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2CHNZMW4X8CflWPGMqk3n/4C28ulPxVyey+BzhzUUyI=;
 b=GU4q8ByKuQ5htQCThQzmF9s/NJJrpcHAxsURY4wFg60VizOUtfGpz3nHni/6A7cNBW1lQgwaPun1Ese5aL+l6iP+wyyzfA+HQREh6cRj+nG4bbz1UN6y/nUVO/WrbJq46wv3JN6Gv8fMjm20rE/NhEL3FafYtTyEW0p07B1UvzXUaBPKgiTJFh+sD85ljv9jG1ZWyuAH3Ktu4LyoiSTD2o+CQOkC7cHtTOqHDZpp6EEQkZnlfcTUsp5S9fSO/Atr8/JIEFb1Y+fDqOP0CZ3Ptmawaq6vr89E5r0Wlmrd88KFZVVw2esLuTpGB18Ntwd5Cde4VTiOicEw81rN7fT1Gw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2CHNZMW4X8CflWPGMqk3n/4C28ulPxVyey+BzhzUUyI=;
 b=SsWqcw5+ljmFcFAuh6bWC8WftMVyQ3BARfJUdACrFycJP4/TfWdBLPcERsZMWIKXATDA4Q2uWQhvbhnDVhv/DDXrzVM3qKQT5hmfsfmB2AN1/daDnYW3SXBZv1G/Z7x8eWe/fzGS5U8aNrnMdnTTPBGQkFPtSdOuqcCgXpMi1X0=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Topic: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Index: AQHXsHMRZjZ8eZQrbkW8SEq2x0f00KuyVH0AgABDnECAAQJ+AIACfOAg
Date: Sun, 26 Sep 2021 10:11:51 +0000
Message-ID:
 <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-9-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: B603888FD9C0314FBBF36B8F636AF508.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: be1c4e6f-30c8-4c2b-295b-08d980d618f1
x-ms-traffictypediagnostic: DB8PR08MB4569:|DBBPR08MB4821:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB4821A63551ED8A9073AC8AE49EA69@DBBPR08MB4821.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 u5gj0Xy/o5NWiz9a1QkbxlzwxBb6mocsFyIoCVGUNZ9DcOQZVo18sD/VIL/VWVSqiwU5862tlERi2XNXZiR/+VXr39X0q7ob9Vg/Mn3dUt/i8ipreBKH33Wg0zgFl5+AwRd9Env4iIw1RYXO+TzcH+y6AoJZQfwB/w/Oeph2K2hq6QjkKImvaBFtIxmlRObT18CrIiAh50bUl9vo8VrJ6kbQx6qQA7wBuTS++LupFJMgQNngTcBid+Ceoz4Mohye/9G3FFSMFQJ8blcC8TgKwJ27iL/pKoX1+y1wMRhrhLZzMfxU93zD+84xmF8PvjwAxXdt2wI8tdVODg37pESkKjJ9dl41fnVoZPn9kg58fNFU8lN2szLUpYLPCdgAd7TG/w5iLS8P5bIJfTAVvZpnisYYlQnypXZetgLw1XoEuR+OLFq9TGuU9XzjfmfuW8zYkDi/+8z+IqYhyRTcnu72LP71vsu69OpLbn7MPvAFvgJZ5VVJPDg3gXdMEsXEeKSLvQaPav6tZphWzkfz6wu2QutUmsa17d3PABvPC6PqXx1jdhviNPWZbDC21AcQSbwbjidKMVTwGZBMQzgoCuoMWsJsPtwj7mFkkSQWsiTbHb6pgPN31Jjcik4ZdfASJs9VOWBK4/V36icYorhbCyNM2ONWuj7SMB9h6WPHSfuGv7MGNHOK7exJUjscGRrCWhXIIrB3CnJjQuKKUQSFwxhzhA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(9686003)(122000001)(4326008)(71200400001)(33656002)(186003)(8676002)(38100700002)(6916009)(83380400001)(5660300002)(508600001)(26005)(76116006)(8936002)(2906002)(7696005)(86362001)(64756008)(52536014)(66556008)(66946007)(66476007)(53546011)(6506007)(66446008)(54906003)(55016002)(38070700005)(316002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4569
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c0d94cb2-fa12-4f71-5bd6-08d980d60f87
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+L/uXENvpxdUr1hjYDbvNdf4q4ETGqI9RTwbkozH2Ye3Uy5Uv5Zix7Rv/I/FZI+EW1OEbvB60XILzUmtDJSrnk5lhDbLw5LrTCO4eAXQFU9XNG8Cit31KST7MTKGmzVc3Y+OeczIUHFUm8E7lrjRm+jJfglwdc1DdfyVtulMWqZBQWhGMboXooMly/4cltEVCt3uIfim0CU/LtQWA/DXAP5Vqdo/6hqS/AY3/K98LSdvhjwPPAAOb/rqnwDTTJuBpDFa23j/zTqJGEISSfbWjHLtLDJt0LPLYiU+kmuARSiKeI+8e6ajR/FNA8YMH/VVg550gosAcmgASFp7UfV7ZB/32KyHvsKGURJ6uGeRYE3xXEWX+DsF23RCc5JPsj9IkSDqUi2RGYtpYnUCDBndnSMvAMFKjvVN/IneLbU/JLFz6eGsm3PzXiwXo0Dgc6HJWEf6N1fQiLp1B8PUouGfuy2LipT9m3PjWle1Uu3Ozoxi4qITJNO/CB4sW2SQDBsAiVtxonca/KEk7z3iEAWn43y2p9jvbqpWkZBS1aFdBQ0f95j+KZBYJG+8L987FDP8gBaM6k4uujYCizMUjupCNlUkstS2mbEOFiOjoykZHRPKi0TwUFdu8dBi/wm+PBT3GrX9C0JFoNNki/6VqCMw5qd45vhmEOIGxb8Xqbe117wZArztSl0DOS7OUG1sR8kfXR/HCRG7a8NCUSOt3RIhIA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(55016002)(7696005)(356005)(54906003)(83380400001)(52536014)(4326008)(8936002)(508600001)(47076005)(6862004)(53546011)(316002)(5660300002)(81166007)(6506007)(186003)(36860700001)(82310400003)(33656002)(70206006)(26005)(70586007)(86362001)(336012)(2906002)(9686003)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2021 10:12:07.8051
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: be1c4e6f-30c8-4c2b-295b-08d980d618f1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4821

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMjXml6UgMzo1Mw0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBT
dGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tDQo+IGRldmVs
QGxpc3RzLnhlbnByb2plY3Qub3JnOyBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcw0K
PiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hAc3VzZS5jb207IGFuZHJldy5j
b29wZXIzQGNpdHJpeC5jb207DQo+IHJvZ2VyLnBhdUBjaXRyaXguY29tOyB3bEB4ZW4ub3JnDQo+
IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMDgvMzddIHhlbi94ODY6IGFkZCBkZXRlY3Rpb24gb2YgZGlz
Y29udGlub3VzIG5vZGUNCj4gbWVtb3J5IHJhbmdlDQo+IA0KPiBPbiBGcmksIDI0IFNlcCAyMDIx
LCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4g
PiBGcm9tOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4g
PiBTZW50OiAyMDIx5bm0OeaciDI05pelIDg6MjYNCj4gPiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNo
ZW5AYXJtLmNvbT4NCj4gPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNz
dGFiZWxsaW5pQGtlcm5lbC5vcmc7DQo+IGp1bGllbkB4ZW4ub3JnOw0KPiA+ID4gQmVydHJhbmQg
TWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hAc3VzZS5jb207DQo+
ID4gPiBhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tOyByb2dlci5wYXVAY2l0cml4LmNvbTsgd2xA
eGVuLm9yZw0KPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCAwOC8zN10geGVuL3g4NjogYWRkIGRl
dGVjdGlvbiBvZiBkaXNjb250aW5vdXMgbm9kZQ0KPiA+ID4gbWVtb3J5IHJhbmdlDQo+ID4gPg0K
PiA+ID4gQ0MnaW5nIHg4NiBtYWludGFpbmVycw0KPiA+ID4NCj4gPiA+IE9uIFRodSwgMjMgU2Vw
IDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiBPbmUgTlVNQSBub2RlIG1heSBjb250YWlu
IHNldmVyYWwgbWVtb3J5IGJsb2Nrcy4gSW4gY3VycmVudCBYZW4NCj4gPiA+ID4gY29kZSwgWGVu
IHdpbGwgbWFpbnRhaW4gYSBub2RlIG1lbW9yeSByYW5nZSBmb3IgZWFjaCBub2RlIHRvIGNvdmVy
DQo+ID4gPiA+IGFsbCBpdHMgbWVtb3J5IGJsb2Nrcy4gQnV0IGhlcmUgY29tZXMgdGhlIHByb2Js
ZW0sIGluIHRoZSBnYXAgb2YNCj4gPiA+ID4gb25lIG5vZGUncyB0d28gbWVtb3J5IGJsb2Nrcywg
aWYgdGhlcmUgYXJlIHNvbWUgbWVtb3J5IGJsb2NrcyBkb24ndA0KPiA+ID4gPiBiZWxvbmcgdG8g
dGhpcyBub2RlIChyZW1vdGUgbWVtb3J5IGJsb2NrcykuIFRoaXMgbm9kZSdzIG1lbW9yeSByYW5n
ZQ0KPiA+ID4gPiB3aWxsIGJlIGV4cGFuZGVkIHRvIGNvdmVyIHRoZXNlIHJlbW90ZSBtZW1vcnkg
YmxvY2tzLg0KPiA+ID4gPg0KPiA+ID4gPiBPbmUgbm9kZSdzIG1lbW9yeSByYW5nZSBjb250YWlu
cyBvdGhlIG5vZGVzJyBtZW1vcnksIHRoaXMgaXMNCj4gb2J2aW91c2x5DQo+ID4gPiA+IG5vdCB2
ZXJ5IHJlYXNvbmFibGUuIFRoaXMgbWVhbnMgY3VycmVudCBOVU1BIGNvZGUgb25seSBjYW4gc3Vw
cG9ydA0KPiA+ID4gPiBub2RlIGhhcyBjb250aW5vdXMgbWVtb3J5IGJsb2Nrcy4gSG93ZXZlciwg
b24gYSBwaHlzaWNhbCBtYWNoaW5lLA0KPiB0aGUNCj4gPiA+ID4gYWRkcmVzc2VzIG9mIG11bHRp
cGxlIG5vZGVzIGNhbiBiZSBpbnRlcmxlYXZlZC4NCj4gPiA+ID4NCj4gPiA+ID4gU28gaW4gdGhp
cyBwYXRjaCwgd2UgYWRkIGNvZGUgdG8gZGV0ZWN0IGRpc2NvbnRpbm91cyBtZW1vcnkgYmxvY2tz
DQo+ID4gPiA+IGZvciBvbmUgbm9kZS4gTlVNQSBpbml0aWFsaXp0aW9uIHdpbGwgYmUgZmFpbGVk
IGFuZCBlcnJvciBtZXNzYWdlcw0KPiA+ID4gPiB3aWxsIGJlIHByaW50ZWQgd2hlbiBYZW4gZGV0
ZWN0IHN1Y2ggaGFyZHdhcmUgY29uZmlndXJhdGlvbi4NCj4gPiA+DQo+ID4gPiBBdCBsZWFzdCBv
biBBUk0sIGl0IGlzIG5vdCBqdXN0IG1lbW9yeSB0aGF0IGNhbiBiZSBpbnRlcmxlYXZlZCwgYnV0
DQo+IGFsc28NCj4gPiA+IE1NSU8gcmVnaW9ucy4gRm9yIGluc3RhbmNlOg0KPiA+ID4NCj4gPiA+
IG5vZGUwIGJhbmswIDAtMHgxMDAwMDAwDQo+ID4gPiBNTUlPIDB4MTAwMDAwMC0weDEwMDIwMDAN
Cj4gPiA+IEhvbGUgMHgxMDAyMDAwLTB4MjAwMDAwMA0KPiA+ID4gbm9kZTAgYmFuazEgMHgyMDAw
MDAwLTB4MzAwMDAwMA0KPiA+ID4NCj4gPiA+IFNvIEkgYW0gbm90IGZhbWlsaWFyIHdpdGggdGhl
IFNSQVQgZm9ybWF0LCBidXQgSSB0aGluayBvbiBBUk0gdGhlDQo+IGNoZWNrDQo+ID4gPiB3b3Vs
ZCBsb29rIGRpZmZlcmVudDogd2Ugd291bGQganVzdCBsb29rIGZvciBtdWx0aXBsZSBtZW1vcnkg
cmFuZ2VzDQo+ID4gPiB1bmRlciBhIGRldmljZV90eXBlID0gIm1lbW9yeSIgbm9kZSBvZiBhIE5V
TUEgbm9kZSBpbiBkZXZpY2UgdHJlZS4NCj4gPiA+DQo+ID4gPg0KPiA+DQo+ID4gU2hvdWxkIEkg
bmVlZCB0byBpbmNsdWRlL3JlZmluZSBhYm92ZSBtZXNzYWdlIHRvIGNvbW1pdCBsb2c/DQo+IA0K
PiBMZXQgbWUgYXNrIHlvdSBhIHF1ZXN0aW9uIGZpcnN0Lg0KPiANCj4gV2l0aCB0aGUgTlVNQSBp
bXBsZW1lbnRhdGlvbiBvZiB0aGlzIHBhdGNoIHNlcmllcywgY2FuIHdlIGRlYWwgd2l0aA0KPiBj
YXNlcyB3aGVyZSBlYWNoIG5vZGUgaGFzIG11bHRpcGxlIG1lbW9yeSBiYW5rcywgbm90IGludGVy
bGVhdmVkPw0KDQpZZXMuDQoNCj4gQW4gYW4gZXhhbXBsZToNCj4gDQo+IG5vZGUwOiAweDAgICAg
ICAgIC0gMHgxMDAwMDAwMA0KPiBNTUlPIDogMHgxMDAwMDAwMCAtIDB4MjAwMDAwMDANCj4gbm9k
ZTA6IDB4MjAwMDAwMDAgLSAweDMwMDAwMDAwDQo+IE1NSU8gOiAweDMwMDAwMDAwIC0gMHg1MDAw
MDAwMA0KPiBub2RlMTogMHg1MDAwMDAwMCAtIDB4NjAwMDAwMDANCj4gTU1JTyA6IDB4NjAwMDAw
MDAgLSAweDgwMDAwMDAwDQo+IG5vZGUyOiAweDgwMDAwMDAwIC0gMHg5MDAwMDAwMA0KPiANCj4g
DQo+IEkgYXNzdW1lIHdlIGNhbiBkZWFsIHdpdGggdGhpcyBjYXNlIHNpbXBseSBieSBzZXR0aW5n
IG5vZGUwIG1lbW9yeSB0bw0KPiAweDAtMHgzMDAwMDAwMCBldmVuIGlmIHRoZXJlIGlzIGFjdHVh
bGx5IHNvbWV0aGluZyBlbHNlLCBhIGRldmljZSwgdGhhdA0KPiBkb2Vzbid0IGJlbG9uZyB0byBu
b2RlMCBpbiBiZXR3ZWVuIHRoZSB0d28gbm9kZTAgYmFua3M/DQoNCldoaWxlIHRoaXMgY29uZmln
dXJhdGlvbiBpcyByYXJlIGluIFNvQyBkZXNpZ24sIGJ1dCBpdCBpcyBub3QgaW1wb3NzaWJsZS4g
DQoNCj4gDQo+IElzIGl0IG9ubHkgb3RoZXIgbm9kZXMnIG1lbW9yeSBpbnRlcmxlYXZlZCB0aGF0
IGNhdXNlIGlzc3Vlcz8gSW4gb3RoZXINCj4gd29yZHMsIG9ubHkgdGhlIGZvbGxvd2luZyBpcyBh
IHByb2JsZW1hdGljIHNjZW5hcmlvPw0KPiANCj4gbm9kZTA6IDB4MCAgICAgICAgLSAweDEwMDAw
MDAwDQo+IE1NSU8gOiAweDEwMDAwMDAwIC0gMHgyMDAwMDAwMA0KPiBub2RlMTogMHgyMDAwMDAw
MCAtIDB4MzAwMDAwMDANCj4gTU1JTyA6IDB4MzAwMDAwMDAgLSAweDUwMDAwMDAwDQo+IG5vZGUw
OiAweDUwMDAwMDAwIC0gMHg2MDAwMDAwMA0KPiANCj4gQmVjYXVzZSBub2RlMSBpcyBpbiBiZXR3
ZWVuIHRoZSB0d28gcmFuZ2VzIG9mIG5vZGUwPw0KPiANCg0KQnV0IG9ubHkgZGV2aWNlX3R5cGU9
Im1lbW9yeSIgY2FuIGJlIGFkZGVkIHRvIGFsbG9jYXRpb24uDQpGb3IgbW1pbyB0aGVyZSBhcmUg
dHdvIGNhc2VzOg0KMS4gbW1pbyBkb2Vzbid0IGhhdmUgTlVNQSBpZCBwcm9wZXJ0eS4NCjIuIG1t
aW8gaGFzIE5VTUEgaWQgcHJvcGVydHksIGp1c3QgbGlrZSBzb21lIFBDSWUgY29udHJvbGxlcnMu
DQogICBCdXQgd2UgZG9u4oCZdCBuZWVkIHRvIGhhbmRsZSB0aGVzZSBraW5kcyBvZiBNTUlPIGRl
dmljZXMNCiAgIGluIG1lbW9yeSBibG9jayBwYXJzaW5nLiBCZWNhdXNlIHdlIGRvbid0IG5lZWQg
dG8gYWxsb2NhdGUNCiAgIG1lbW9yeSBmcm9tIHRoZXNlIG1taW8gcmFuZ2VzLiBBbmQgZm9yIGFj
Y2Vzc2luZywgd2UgbmVlZA0KICAgYSBOVU1BLWF3YXJlIFBDSWUgY29udHJvbGxlciBkcml2ZXIg
b3IgYSBnZW5lcmljIE5VTUEtYXdhcmUNCiAgIE1NSU8gYWNjZXNzaW5nIEFQSXMuDQoNCj4gDQo+
IEkgYW0gYXNraW5nIHRoZXNlIHF1ZXN0aW9ucyBiZWNhdXNlIGl0IGlzIGNlcnRhaW5seSBwb3Nz
aWJsZSB0byBoYXZlDQo+IG11bHRpcGxlIG1lbW9yeSByYW5nZXMgZm9yIGVhY2ggTlVNQSBub2Rl
IGluIGRldmljZSB0cmVlLCBlaXRoZXIgYnkNCj4gc3BlY2lmeWluZyBtdWx0aXBsZSByYW5nZXMg
d2l0aCBhIHNpbmdsZSAicmVnIiBwcm9wZXJ0eSwgb3IgYnkNCj4gc3BlY2lmeWluZyBtdWx0aXBs
ZSBtZW1vcnkgbm9kZXMgd2l0aCB0aGUgc2FtZSBudW1hLW5vZGUtaWQuDQoNCg0K


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 10:15:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 10:15:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196052.348898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURBs-0003jz-56; Sun, 26 Sep 2021 10:15:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196052.348898; Sun, 26 Sep 2021 10:15:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURBs-0003js-0l; Sun, 26 Sep 2021 10:15:24 +0000
Received: by outflank-mailman (input) for mailman id 196052;
 Sun, 26 Sep 2021 10:15:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ODXE=OQ=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mURBq-0003jU-Dg
 for xen-devel@lists.xenproject.org; Sun, 26 Sep 2021 10:15:22 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 84e73a0c-a8f2-4fec-82dc-7ba967fc7fed;
 Sun, 26 Sep 2021 10:15:19 +0000 (UTC)
Received: from AS9PR06CA0162.eurprd06.prod.outlook.com (2603:10a6:20b:45c::31)
 by DB6PR0801MB1942.eurprd08.prod.outlook.com (2603:10a6:4:75::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Sun, 26 Sep
 2021 10:15:16 +0000
Received: from AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45c:cafe::f0) by AS9PR06CA0162.outlook.office365.com
 (2603:10a6:20b:45c::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Sun, 26 Sep 2021 10:15:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT010.mail.protection.outlook.com (10.152.16.134) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Sun, 26 Sep 2021 10:15:16 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Sun, 26 Sep 2021 10:15:14 +0000
Received: from d607cf8b24d8.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4F088FCA-A6D7-468F-BD34-F179FD51BFFA.1; 
 Sun, 26 Sep 2021 10:15:04 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d607cf8b24d8.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Sun, 26 Sep 2021 10:15:04 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB4569.eurprd08.prod.outlook.com (2603:10a6:10:b2::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Sun, 26 Sep
 2021 10:15:02 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Sun, 26 Sep 2021
 10:15:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84e73a0c-a8f2-4fec-82dc-7ba967fc7fed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EU3RDnxrdTOUAgDVbK7ldLobqqH5q0xYX1mf8P19fkk=;
 b=WRt5FNbFFCHUlWbhOe7PC/VSPM4VMrQiG9XedMRQnEFxOss09Rm29Bep6H22gjIucqBrxD3Od6AbkZjUbLQfnNPt6wvt/hM1Ou7FD0mJvW0MCGMTENQFl8aRzqr0xxk0m1g9FA/OvIBqiqDUO6HLM/8fFZJFjKAyq5tckhZMHl4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GV3Zx5QNSuPkCEt6egP1Xribt9Yvd1OHGqtyPgslRBkg+S6+CvNA30/rsapaKSKsD4nAObbSqy4Kby9zWSQUTpJ/hBHl/LRp9SgxI2M+1SwnB/3aYtN7I4Bsu/F9KXQjc00Ys8i/Po6RxNLeJjpcoUBvL32Q/nfwnyiTdGeTMcIUdAqNoGnGHCZTsTcQtJ+M3Z7XYQePUteMV5CwUYqIOCEs4miBIDuZsUtYaYNJ4fmnMK/NvXqrwI3l+lcniFNIVwKm0rqDevzPhU9MCgz53/3Vpp1XrjIx5av2pSZMUgT+0CcGn6KBG4/kkcAQ+UP3NpgFR9L73i2OLweRuF97Rg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EU3RDnxrdTOUAgDVbK7ldLobqqH5q0xYX1mf8P19fkk=;
 b=iHU5Y+XxJxqCzlI0qd5G3p1qbh+RlufHgw38tHvrZGuENCX2xQOlpxO9PQaBLsSIgB+TVAV7xU1vkxG3ThKb9h+buwz/2gQemjHZv1lFBExeIRoB1bSqmJmxPFhJ4gvsB6ypdbpXXpG+DA1K4/0XjYjqiLr6ZYt4V9xnDy1Wnu9cKZDM58qYfQsdsMKpduJnkVQx0cT+17D+3LEfZD86yCZgEKQ1bkFAlW8SsTelkddZS8KDPlwUCed8dWrSvqwgpws/A8wXDoAO4EencEhGXgZmxubL9U9ncEqWmhK5SMQU6d2gfymPD2bcttBKkP0WR7ys1hTcr2+Vyzf5nqfTqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EU3RDnxrdTOUAgDVbK7ldLobqqH5q0xYX1mf8P19fkk=;
 b=WRt5FNbFFCHUlWbhOe7PC/VSPM4VMrQiG9XedMRQnEFxOss09Rm29Bep6H22gjIucqBrxD3Od6AbkZjUbLQfnNPt6wvt/hM1Ou7FD0mJvW0MCGMTENQFl8aRzqr0xxk0m1g9FA/OvIBqiqDUO6HLM/8fFZJFjKAyq5tckhZMHl4=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
Thread-Topic: [PATCH 23/37] xen/arm: implement node distance helpers for Arm
Thread-Index: AQHXsHMu1eZPzfhk6EKVEGWM7eNLZ6uyayoAgAAvnmCAAPsvgIACh1gw
Date: Sun, 26 Sep 2021 10:15:02 +0000
Message-ID:
 <DB9PR08MB68578B994BD167A105BACE939EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-24-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231843530.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB68571BAA4C8B07BB2061C8579EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241233590.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241233590.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: ECA1477DF674ED479E9407B7F83C3E1C.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3e5e4d77-075b-49a8-9280-08d980d68929
x-ms-traffictypediagnostic: DB8PR08MB4569:|DB6PR0801MB1942:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0801MB194204DE817EBCA2A925FA0B9EA69@DB6PR0801MB1942.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 6GGQZfwBiDsGKkZh3ZATinUcf7lSZy7VVXf5i9GyiMKChXduVggryxTUng5iHghmNuSZqSNkgyrSQUQlG1cWC8LjFq+wnqX0OdE8ndHaIu81udg4r/nlNRCQgRIQQjHabET7ZIRo8NsoUieoquJMkiVc1BGRtRDnquH2jdD5575+GEWkiD+PlqybzkJuGHRz2mmtHJKIy0McSs0oRddXYlpjj5yeFbT+eFvgfL9F4AmmAUW4pOJwGDCld9griPyQz56GfDsf7MSu3upIUsyX68HSWihzuRdEjz79oEMLXPqWKQSXZ3XwF7pRx2g+NNyL4rKasKmjIyFfHxzLKIevU57T7Q0sDqoLJaL1tRBlIapasGeka3KtgFTeHKDqMrIaBSlTnNHcWFnfAHWknOenlfwYR34rjEeo8cgIBcOl7kiV58o2jvHziYZkSaWGRAuM7d39pUZynWqO6i9/QSNPuFp36RkDx/IERvi43XFpnREePC328hQ08hsHncZ7pWUXBLDFI/aIkT615VKFTPzyZuGMzRn5Ixbvp6QQJgqOhHCbjId6E0CR2HwNNBT+3agJmJC2oVQhpq7aynLwODnU0vLXU/EgHRpjh6hijQ18l/Id+Nnt4UkfTdn2C7+ZheSSYAtdfe7A9xpWbVsMU8EXrALLtar2nI/Nsi1KmVVAfOkJ/a7LAt1nbTQvEjs5J34aauyJFCpWN/72A1csp0KEoCrsttRQ+LTNWOXbrH8MU3e0TmP6oZA86AhTfs2MTdawl9iUslz+kNDR6IxWR7CvWc4pWl3XNHmpJZgeqdKw4KYQ5R0VJo3xcVH3mKcT2nt0
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(9686003)(122000001)(4326008)(71200400001)(33656002)(186003)(8676002)(38100700002)(6916009)(83380400001)(5660300002)(508600001)(26005)(76116006)(8936002)(2906002)(7696005)(86362001)(64756008)(52536014)(66556008)(66946007)(66476007)(53546011)(6506007)(66446008)(54906003)(55016002)(38070700005)(316002)(2004002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4569
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	623be939-30f4-4383-f6a4-08d980d680d9
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HGFg+yyqK/iahVHYRdjkLJf7mxHYNEmFODYNAbz6aHP9QPS4RTuBH6wfgUEEnDvHACENH2eVM63KzPP7ts3J2MLwCvUM8uG88+LLwyz/96zIoyY6eH1QZZxC/o8QFn9jyrcmeLoH7WEGuuAfZWNxd6wE63kq3PmjTh7EwSfZk2Z5cV8GKId1ZW/BWBsd07xXdrj72wp5D5p31T5+SuNXU1KRh9Fp3yHqTpyfmd91+1kgjKMczmC5tacEQ92QwP474BhgDRp6XRV3dnNcHcyNMcom0FBpjbsdDqayUwgkemCHWyr7fW8g4nJxFcGGpWREcIDLoGPVBXXrckm/22MynVS6jYUCdFv3syo84j/fpcY5ToipdZUoAsCsbX1u8zE3zW+kyRueX7YUET72mC0ItyMpGUfFJvNVC9bhd/IuMFn7LnfuFCcf4eH7KH4OiaAo9bIdegEMG+SNF3+m5TvyZ0VE37jmc3QTnXDeT4tZlOY62tefYufNh+s5Rd4vRlK4WblnViobC8s2sszt/JujJiCa8W/SgJyU6e4cNhgbaJxUXAruVnw2KWZcoMDDCEiXktZX3LQoMVNRZAt1G5S5Gvw6Ss+YUFiKuYANMPb+NsQUaukY+cgta0wAUWcrfHbFCIb7casxvEv3qYaEHtlPI4bGMrLB+hxPj4qWI+tZekMUAA0yh88DJDjTHLa5H5IY3XELtRAg9aFvkIHUe6kKZMuAWn6dbd1Qw0g8IMPZd6R1vC4CiCOIFzX6J7bS77tsFKPeYmpxkkirCse6pSUY4Lt+CtW6KpIzCKAlSxdwsGuokSqQu3hnb4jekB0RvxZd
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70586007)(70206006)(336012)(26005)(52536014)(6862004)(36860700001)(81166007)(316002)(86362001)(508600001)(54906003)(2906002)(33656002)(83380400001)(9686003)(186003)(53546011)(55016002)(8936002)(6506007)(47076005)(8676002)(5660300002)(356005)(82310400003)(4326008)(7696005)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2021 10:15:16.1375
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3e5e4d77-075b-49a8-9280-08d980d68929
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1942

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDogMjAyMeW5tDnmnIgyNeaXpSAzOjM2
DQo+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IFN0ZWZhbm8gU3RhYmVs
bGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0NCj4gZGV2ZWxAbGlzdHMueGVucHJv
amVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+IDxCZXJ0cmFuZC5N
YXJxdWlzQGFybS5jb20+DQo+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMjMvMzddIHhlbi9hcm06IGlt
cGxlbWVudCBub2RlIGRpc3RhbmNlIGhlbHBlcnMgZm9yDQo+IEFybQ0KPiANCj4gT24gRnJpLCAy
NCBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut
LS0tLQ0KPiA+ID4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwu
b3JnPg0KPiA+ID4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSA5OjQ3DQo+ID4gPiBUbzogV2VpIENo
ZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiBDYzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2pl
Y3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOw0KPiBqdWxpZW5AeGVuLm9yZzsNCj4gPiA+
IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gPiA+IFN1Ympl
Y3Q6IFJlOiBbUEFUQ0ggMjMvMzddIHhlbi9hcm06IGltcGxlbWVudCBub2RlIGRpc3RhbmNlIGhl
bHBlcnMNCj4gZm9yDQo+ID4gPiBBcm0NCj4gPiA+DQo+ID4gPiBPbiBUaHUsIDIzIFNlcCAyMDIx
LCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gV2Ugd2lsbCBwYXJzZSBOVU1BIG5vZGVzIGRpc3Rh
bmNlcyBmcm9tIGRldmljZSB0cmVlIG9yIEFDUEkNCj4gPiA+ID4gdGFibGUuIFNvIHdlIG5lZWQg
YSBtYXRyaXggdG8gcmVjb3JkIHRoZSBkaXN0YW5jZXMgYmV0d2Vlbg0KPiA+ID4gPiBhbnkgdHdv
IG5vZGVzIHdlIHBhcnNlZC4gQWNjb3JkaW5nbHksIHdlIHByb3ZpZGUgdGhpcw0KPiA+ID4gPiBu
b2RlX3NldF9kaXN0YW5jZSBBUEkgZm9yIGRldmljZSB0cmVlIG9yIEFDUEkgdGFibGUgcGFyc2Vy
cw0KPiA+ID4gPiB0byBzZXQgdGhlIGRpc3RhbmNlIGZvciBhbnkgdHdvIG5vZGVzIGluIHRoaXMg
cGF0Y2guDQo+ID4gPiA+IFdoZW4gTlVNQSBpbml0aWFsaXphdGlvbiBmYWlsZWQsIF9fbm9kZV9k
aXN0YW5jZSB3aWxsIHJldHVybg0KPiA+ID4gPiBOVU1BX1JFTU9URV9ESVNUQU5DRSwgdGhpcyB3
aWxsIGhlbHAgdXMgYXZvaWQgZG9pbmcgcm9sbGJhY2sNCj4gPiA+ID4gZm9yIGRpc3RhbmNlIG1h
eHRyaXggd2hlbiBOVU1BIGluaXRpYWxpemF0aW9uIGZhaWxlZC4NCj4gPiA+ID4NCj4gPiA+ID4g
U2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+ID4gPiA+IC0tLQ0K
PiA+ID4gPiAgeGVuL2FyY2gvYXJtL01ha2VmaWxlICAgICAgfCAgMSArDQo+ID4gPiA+ICB4ZW4v
YXJjaC9hcm0vbnVtYS5jICAgICAgICB8IDY5DQo+ICsrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrDQo+ID4gPiA+ICB4ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaCB8IDEzICsr
KysrKysNCj4gPiA+ID4gIDMgZmlsZXMgY2hhbmdlZCwgODMgaW5zZXJ0aW9ucygrKQ0KPiA+ID4g
PiAgY3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL2FybS9udW1hLmMNCj4gPiA+ID4NCj4gPiA+
ID4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9NYWtlZmlsZSBiL3hlbi9hcmNoL2FybS9NYWtl
ZmlsZQ0KPiA+ID4gPiBpbmRleCBhZTRlZmJmNzZlLi40MWNhMzExYjZiIDEwMDY0NA0KPiA+ID4g
PiAtLS0gYS94ZW4vYXJjaC9hcm0vTWFrZWZpbGUNCj4gPiA+ID4gKysrIGIveGVuL2FyY2gvYXJt
L01ha2VmaWxlDQo+ID4gPiA+IEBAIC0zNSw2ICszNSw3IEBAIG9iai0kKENPTkZJR19MSVZFUEFU
Q0gpICs9IGxpdmVwYXRjaC5vDQo+ID4gPiA+ICBvYmoteSArPSBtZW1fYWNjZXNzLm8NCj4gPiA+
ID4gIG9iai15ICs9IG1tLm8NCj4gPiA+ID4gIG9iai15ICs9IG1vbml0b3Iubw0KPiA+ID4gPiAr
b2JqLSQoQ09ORklHX05VTUEpICs9IG51bWEubw0KPiA+ID4gPiAgb2JqLXkgKz0gcDJtLm8NCj4g
PiA+ID4gIG9iai15ICs9IHBlcmNwdS5vDQo+ID4gPiA+ICBvYmoteSArPSBwbGF0Zm9ybS5vDQo+
ID4gPiA+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vbnVtYS5jIGIveGVuL2FyY2gvYXJtL251
bWEuYw0KPiA+ID4gPiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPiA+ID4gPiBpbmRleCAwMDAwMDAw
MDAwLi4zZjA4ODcwZDY5DQo+ID4gPiA+IC0tLSAvZGV2L251bGwNCj4gPiA+ID4gKysrIGIveGVu
L2FyY2gvYXJtL251bWEuYw0KPiA+ID4gPiBAQCAtMCwwICsxLDY5IEBADQo+ID4gPiA+ICsvLyBT
UERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMA0KPiA+ID4gPiArLyoNCj4gPiA+ID4gKyAq
IEFybSBBcmNoaXRlY3R1cmUgc3VwcG9ydCBsYXllciBmb3IgTlVNQS4NCj4gPiA+ID4gKyAqDQo+
ID4gPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMjEgQXJtIEx0ZA0KPiA+ID4gPiArICoNCj4gPiA+
ID4gKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0
ZSBpdCBhbmQvb3INCj4gbW9kaWZ5DQo+ID4gPiA+ICsgKiBpdCB1bmRlciB0aGUgdGVybXMgb2Yg
dGhlIEdOVSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlIHZlcnNpb24gMg0KPiBhcw0KPiA+ID4gPiAr
ICogcHVibGlzaGVkIGJ5IHRoZSBGcmVlIFNvZnR3YXJlIEZvdW5kYXRpb24uDQo+ID4gPiA+ICsg
Kg0KPiA+ID4gPiArICogVGhpcyBwcm9ncmFtIGlzIGRpc3RyaWJ1dGVkIGluIHRoZSBob3BlIHRo
YXQgaXQgd2lsbCBiZSB1c2VmdWwsDQo+ID4gPiA+ICsgKiBidXQgV0lUSE9VVCBBTlkgV0FSUkFO
VFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGllZCB3YXJyYW50eSBvZg0KPiA+ID4gPiArICogTUVS
Q0hBTlRBQklMSVRZIG9yIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRo
ZQ0KPiA+ID4gPiArICogR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWls
cy4NCj4gPiA+ID4gKyAqDQo+ID4gPiA+ICsgKiBZb3Ugc2hvdWxkIGhhdmUgcmVjZWl2ZWQgYSBj
b3B5IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMNCj4gTGljZW5zZQ0KPiA+ID4gPiArICogYWxv
bmcgd2l0aCB0aGlzIHByb2dyYW0uIElmIG5vdCwgc2VlDQo+IDxodHRwOi8vd3d3LmdudS5vcmcv
bGljZW5zZXMvPi4NCj4gPiA+ID4gKyAqDQo+ID4gPiA+ICsgKi8NCj4gPiA+ID4gKyNpbmNsdWRl
IDx4ZW4vaW5pdC5oPg0KPiA+ID4gPiArI2luY2x1ZGUgPHhlbi9udW1hLmg+DQo+ID4gPiA+ICsN
Cj4gPiA+ID4gK3N0YXRpYyB1aW50OF90IF9fcmVhZF9tb3N0bHkNCj4gPiA+ID4gK25vZGVfZGlz
dGFuY2VfbWFwW01BWF9OVU1OT0RFU11bTUFYX05VTU5PREVTXSA9IHsNCj4gPiA+ID4gKyAgICB7
IDAgfQ0KPiA+ID4gPiArfTsNCj4gPiA+ID4gKw0KPiA+ID4gPiArdm9pZCBfX2luaXQgbnVtYV9z
ZXRfZGlzdGFuY2Uobm9kZWlkX3QgZnJvbSwgbm9kZWlkX3QgdG8sIHVpbnQzMl90DQo+ID4gPiBk
aXN0YW5jZSkNCj4gPiA+ID4gK3sNCj4gPiA+ID4gKyAgICBpZiAoIGZyb20gPj0gTUFYX05VTU5P
REVTIHx8IHRvID49IE1BWF9OVU1OT0RFUyApDQo+ID4gPiA+ICsgICAgew0KPiA+ID4gPiArICAg
ICAgICBwcmludGsoS0VSTl9XQVJOSU5HDQo+ID4gPiA+ICsgICAgICAgICAgICAgICAiTlVNQTog
aW52YWxpZCBub2RlczogZnJvbT0lIlBSSXU4IiB0bz0lIlBSSXU4Ig0KPiA+ID4gTUFYPSUiUFJJ
dTgiXG4iLA0KPiA+ID4gPiArICAgICAgICAgICAgICAgZnJvbSwgdG8sIE1BWF9OVU1OT0RFUyk7
DQo+ID4gPiA+ICsgICAgICAgIHJldHVybjsNCj4gPiA+ID4gKyAgICB9DQo+ID4gPiA+ICsNCj4g
PiA+ID4gKyAgICAvKiBOVU1BIGRlZmluZXMgMHhmZiBhcyBhbiB1bnJlYWNoYWJsZSBub2RlIGFu
ZCAwLTkgYXJlDQo+IHVuZGVmaW5lZA0KPiA+ID4gKi8NCj4gPiA+ID4gKyAgICBpZiAoIGRpc3Rh
bmNlID49IE5VTUFfTk9fRElTVEFOQ0UgfHwNCj4gPiA+ID4gKyAgICAgICAgKGRpc3RhbmNlID49
IE5VTUFfRElTVEFOQ0VfVURGX01JTiAmJg0KPiA+ID4gPiArICAgICAgICAgZGlzdGFuY2UgPD0g
TlVNQV9ESVNUQU5DRV9VREZfTUFYKSB8fA0KPiA+ID4gPiArICAgICAgICAoZnJvbSA9PSB0byAm
JiBkaXN0YW5jZSAhPSBOVU1BX0xPQ0FMX0RJU1RBTkNFKSApDQo+ID4gPiA+ICsgICAgew0KPiA+
ID4gPiArICAgICAgICBwcmludGsoS0VSTl9XQVJOSU5HDQo+ID4gPiA+ICsgICAgICAgICAgICAg
ICAiTlVNQTogaW52YWxpZCBkaXN0YW5jZTogZnJvbT0lIlBSSXU4IiB0bz0lIlBSSXU4Ig0KPiA+
ID4gZGlzdGFuY2U9JSJQUkl1MzIiXG4iLA0KPiA+ID4gPiArICAgICAgICAgICAgICAgZnJvbSwg
dG8sIGRpc3RhbmNlKTsNCj4gPiA+ID4gKyAgICAgICAgcmV0dXJuOw0KPiA+ID4gPiArICAgIH0N
Cj4gPiA+ID4gKw0KPiA+ID4gPiArICAgIG5vZGVfZGlzdGFuY2VfbWFwW2Zyb21dW3RvXSA9IGRp
c3RhbmNlOw0KPiA+ID4gPiArfQ0KPiA+ID4gPiArDQo+ID4gPiA+ICt1aW50OF90IF9fbm9kZV9k
aXN0YW5jZShub2RlaWRfdCBmcm9tLCBub2RlaWRfdCB0bykNCj4gPiA+ID4gK3sNCj4gPiA+ID4g
KyAgICAvKiBXaGVuIE5VTUEgaXMgb2ZmLCBhbnkgZGlzdGFuY2Ugd2lsbCBiZSB0cmVhdGVkIGFz
IHJlbW90ZS4gKi8NCj4gPiA+ID4gKyAgICBpZiAoIHNyYXRfZGlzYWJsZWQoKSApDQo+ID4gPg0K
PiA+ID4gR2l2ZW4gdGhhdCB0aGlzIGlzIEFSTSBzcGVjaWZpYyBjb2RlIGFuZCBzcGVjaWZpYyB0
byBBQ1BJLCBJIGRvbid0DQo+IHRoaW5rDQo+ID4gPiB3ZSBzaG91bGQgaGF2ZSBhbnkgY2FsbCB0
byBzb21ldGhpbmcgY2FsbGVkICJzcmF0X2Rpc2FibGVkIi4NCj4gPiA+DQo+ID4gPiBJIHN1Z2dl
c3QgdG8gZWl0aGVyIHJlbmFtZSBzcmF0X2Rpc2FibGVkIHRvIG51bWFfZGlzdGFuY2VfZGlzYWJs
ZWQuDQo+ID4gPg0KPiA+ID4gT3RoZXIgdGhhbiB0aGF0LCB0aGlzIHBhdGNoIGxvb2tzIE9LIHRv
IG1lLg0KPiA+ID4NCj4gPg0KPiA+IHNyYXQgc3RhbmRzIGZvciBzdGF0aWMgcmVzb3VyY2UgYWZm
aW5pdHkgdGFibGUsIEkgdGhpbmsgZHRiIGFsc28gY2FuIGJlDQo+ID4gdHJlYXRlZCBhcyBhIHN0
YXRpYyByZXNvdXJjZSBhZmZpbml0eSB0YWJsZS4gU28gSSBrZWVwIFNSQVQgaW4gdGhpcw0KPiBw
YXRjaA0KPiA+IGFuZCBvdGhlciBwYXRjaGVzLiBJIGhhdmUgc2VlbiB5b3VyIGNvbW1lbnQgaW4g
cGF0Y2gjMjUuIEJlZm9yZSB4ODYNCj4gbWFpbnRhaW5lcnMNCj4gPiBnaXZlIGFueSBmZWVkYmFj
aywgY2FuIHdlIHN0aWxsIGtlZXAgc3JhdCBoZXJlPw0KPiANCj4gSmFuIGFuZCBJIHJlcGxpZWQg
aW4gdGhlIG90aGVyIHRocmVhZC4gSSB0aGluayB0aGF0IGluIHdhcm5pbmcgbWVzc2FnZXMNCj4g
IlNSQVQiIHNob3VsZCBub3QgYmUgbWVudGlvbmVkIHdoZW4gYm9vdGluZyBmcm9tIERULiBJZGVh
bGx5IGZ1bmN0aW9ucw0KPiBuYW1lcyBhbmQgdmFyaWFibGVzIHNob3VsZCBiZSByZW5hbWVkIHRv
byB3aGVuIHNoYXJlZCBiZXR3ZWVuIEFDUEkgYW5kDQo+IERUIGJ1dCBpdCBpcyBsZXNzIGNyaXRp
Y2FsLCBhbmQgaXQgaXMgZmluZSBpZiB5b3UgZG9uJ3QgZG8gdGhhdCBpbiB0aGUNCj4gbmV4dCB2
ZXJzaW9uLg0KDQpUaGFua3MuIEknbGwgbGVhdmUgaXQgYXMgaXQgaXMsIGlmIEkgZG8gbm90IGhh
dmUgYSBiZXR0ZXIgbmFtZS4NCg==


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 10:25:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 10:25:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196061.348909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURLg-0005Sj-3M; Sun, 26 Sep 2021 10:25:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196061.348909; Sun, 26 Sep 2021 10:25:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURLg-0005Sc-05; Sun, 26 Sep 2021 10:25:32 +0000
Received: by outflank-mailman (input) for mailman id 196061;
 Sun, 26 Sep 2021 10:25:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ODXE=OQ=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mURLe-0005SW-N3
 for xen-devel@lists.xenproject.org; Sun, 26 Sep 2021 10:25:30 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.70]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 82dad72b-8b05-4805-98a4-8c6ad714aea4;
 Sun, 26 Sep 2021 10:25:28 +0000 (UTC)
Received: from DB3PR06CA0026.eurprd06.prod.outlook.com (2603:10a6:8:1::39) by
 AM6PR08MB4422.eurprd08.prod.outlook.com (2603:10a6:20b:be::13) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.18; Sun, 26 Sep 2021 10:25:26 +0000
Received: from DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8:1:cafe::48) by DB3PR06CA0026.outlook.office365.com
 (2603:10a6:8:1::39) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Sun, 26 Sep 2021 10:25:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT057.mail.protection.outlook.com (10.152.20.235) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Sun, 26 Sep 2021 10:25:26 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Sun, 26 Sep 2021 10:25:25 +0000
Received: from e34a45c9b2b0.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 DA91558F-6C9E-4A19-923C-CAFC5879EC0B.1; 
 Sun, 26 Sep 2021 10:25:13 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e34a45c9b2b0.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Sun, 26 Sep 2021 10:25:13 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB6PR0801MB1623.eurprd08.prod.outlook.com (2603:10a6:4:3b::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.20; Sun, 26 Sep
 2021 10:25:10 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Sun, 26 Sep 2021
 10:25:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82dad72b-8b05-4805-98a4-8c6ad714aea4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=79FInNMSLaherc7NshKSNL14rlvm3gvZlBDeqKAzniM=;
 b=AebD2hzzcvTxUVr3fll+TsGA+sbkHgOxUAiSD0ZkUxNxxlCE7z3CgV0faWyrTQiUpzqfrbo4+/kXVSdKEfvX8vnJd5+o9LgPWNBZ1tTheqE6BatF2o5D7+uG7+YIsr7U/hj4WKSU8+WbRuS5CpbV24TafoTPQ+uh5iYWWBnQEiU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PEdCQ/d7ENQlI69RI81Y3IVugb0qRZbiIsXKV8SAIGUHy4oh8ItCGbLzq+SJ1IOXvUC61fDOOarzKR/eiD1/YMcL1pqymU947PBl9wiXo35zp2dTIHSsVRvRFfMedtMn3hcjjfwy+wyskubKwYvbAj8Dey6Yv9a96/r+DeAbr8/ZZ3/dLJZPVYusHaUduDIF02aiBGIG5WmpNxml066ByehQhVySFMfhLZuHmYVM0iJOatdFu4hCOp9UUOrMCrR34bnAO8d8qTyTfgEVIsyaNUjpsskI7+ZKGbjpDODMpl0X1BGkvejcqDDKRESqO8hFNWD1ap9UWf+dyxNW9L8VUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=79FInNMSLaherc7NshKSNL14rlvm3gvZlBDeqKAzniM=;
 b=IBBMoGaW+QU6R806yZ/sUgJlMlTK3rgqGOwaEg17NLci5Lnatq3+2oCf574t94QbzmpEC8vddipkgUV71FaLIzBmyaDysdNViLUk025doWwzZ2XMMKH/B208o8gwz3/cuMBNhmbaS3c/3bKap7ypP0R2grJZH3w8c8RDKjETiUFTYarj1iLcDUsADj9QGWBqVC/kYIzd54AeCsqKmhiVYjZkR5rxEyXrQ6WYAgifG7VIpAV6sr4FHvOYGB9wgpDG13oIuzqFgNMGCagFAjhReu60ymd4ff2gdX4C0SbSXIxrO1hZTVuyNhNo1VV5zPb8yrPRET9WKoUbBpHvIC+Kiw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=79FInNMSLaherc7NshKSNL14rlvm3gvZlBDeqKAzniM=;
 b=AebD2hzzcvTxUVr3fll+TsGA+sbkHgOxUAiSD0ZkUxNxxlCE7z3CgV0faWyrTQiUpzqfrbo4+/kXVSdKEfvX8vnJd5+o9LgPWNBZ1tTheqE6BatF2o5D7+uG7+YIsr7U/hj4WKSU8+WbRuS5CpbV24TafoTPQ+uh5iYWWBnQEiU=
From: Wei Chen <Wei.Chen@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Topic: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Index:
 AQHXsHMtR+l18+MmpE6rl/RjSXTBgquyYmkAgAA2xQCAADnbAIAAKWhQgAAGUwCAAxtPwA==
Date: Sun, 26 Sep 2021 10:25:10 +0000
Message-ID:
 <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
In-Reply-To: <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 0BCB7650B8379645A2B386A0009AA27D.0
x-checkrecipientchecked: true
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ae0de3af-cc24-423d-88b7-08d980d7f4bf
x-ms-traffictypediagnostic: DB6PR0801MB1623:|AM6PR08MB4422:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4422EC8B427A0EA235431B239EA69@AM6PR08MB4422.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 p4ROah0PDcr1MryGF8rtOy8LHnXNqVW0jEdoBoxi1LRFMYuFTC6Y4E9j5CDIqZXJdetORsV+GMZ542yGjEmhzkBn292xO1MGhTZeSQOAGLfDDGcOSOoL55/9ByU3eLrjP3hM5bzhiX18wIpk8zJO48dQhQH7Wf9uvghjfD4L0clVbunr/BTXUcZ2TNkIqhYvrNX+7Q0Ok3Ymm56p93/6YQyaOj4eVdo0ye3DMxd/ipvIiw94yOihWUixbq+4oTgxIDsoq8QI2Q34QNqdv0qtbOyB4MReIIxpR5cAMGQK9Fmw6Tqpxb2b+eUcw4EJO5GiY08vrS8xBcHps7O2JF1sBmwidkh4PllKFk/5tkYmyygoRHyTHoicXh5Uo7wnL78xS/wOjY/1+OdATrIAF0YEWfVJ5egWuqktzOemLq6/khMKb+hbC5qtSmfUBM8RjRi7QFMeo3wFSRe7oXxUG9bqtcbr4ocgRDB592qELa86TyqBqsEdVm4Y5yPmIWneoSfLDsoTXFgC4aLgXt2xxPf/een/OqisUIF0EuD/A7eD9rn9wKyljQi+lGhyfkZO85IXTyQmrN3h92lcSdgtMWyDiMf9B55qX27Cc5btTpZ6KSNbhm8xrG9f+vlxgAc2zyHZ1BJotuHka2QXzwq9XE6KtzHF5F978iQ4u7Pk2v0oADA/mvMyDvKZqMp9+MUAhSWLb0ZtzsgABjyf9NYaOAgdNXEDpHZs3b0oWjDxKBJ5qLxlTQlzR0SlWpBLLXMJluJbXWlubUU+/tJD/eVJPFvkZVtckr4GK8cIyrkOlLtNEBs=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(52536014)(55016002)(6506007)(53546011)(33656002)(7696005)(86362001)(76116006)(316002)(66476007)(4326008)(66446008)(83380400001)(2906002)(508600001)(54906003)(64756008)(66946007)(66556008)(71200400001)(9686003)(6916009)(38070700005)(186003)(8676002)(122000001)(8936002)(38100700002)(966005)(26005);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1623
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8a3058b8-99bc-4f72-94d1-08d980d7eb94
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZToLLbwpmrTMAYv/errTNUGVl7c61dHijqZDtNhBZSFDR3O7W7eINMb68NzkFZozhaQDnh/tITjWxkD91Q1qbNbkz4pOtmhfMuSVLqnrsYV8+ssFKDrgTWJa4qBOJloeKs/tz0ap6Jz0wIE1sbD7L4THjNwUGXtZ07W2jP3eVVfB32t2R7w/JP0FVF1zbDIAI956TuW0Mx7nrDerVcev67ZGvKARaNpII+AUq+KRECllIWdtVtBmkmSzc/66UGmeQZFWc+1LdQneamEP8Aq4NTjTmIzSCl1hRczcCPRkOybMZmsDjwKUCMwINvmX6ZAxUJxpBjGcmtxPf+BYhUYZkFoj+engp4GIT+NpWts+Sae9an/a/5cYWV0czuaZlmR2pk7kSOWjPlb8jeyhAd7P/Cw7dDWjhHYZzLcoPA6GIrK5b+E2xjQWO21SlQUh+cwZtoPX6c1doR4a0Z5/M+TKeAmqwWiDOsBIDRvQX2NpAktwj7U8OgK8IgFLqZkxK2dPgEyvv/8fGjcmKK0T2B1TWGLuT4+6sR0Rsh/F3Bitdcnc6OpODffQQigUGW5RA6VL2cAPtVaMi47p0nyasDSB5zllOvDs4eHk9/r50xFaeSdcMd8IekeKYJd7wtEtzMyzZdVcJ2/i3QWVSD3xz72pGC0TKz5TJvNWBY7HYhj8FySekWb+eZB2GzQk0XHMim1q8MEQqphVsmFAibDB0FcMmEP0JRrOQz4q12deB8U0xaQvyHIfpII4VFttk4zTLugvUcX3Dz5SoNjVlxahFLyel2MdzSzru1p/wwXDiHvPkf8=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(36860700001)(316002)(8936002)(4326008)(336012)(9686003)(7696005)(966005)(52536014)(33656002)(82310400003)(2906002)(107886003)(47076005)(186003)(55016002)(356005)(83380400001)(70586007)(53546011)(26005)(508600001)(81166007)(6506007)(70206006)(6862004)(8676002)(54906003)(5660300002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2021 10:25:26.2092
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ae0de3af-cc24-423d-88b7-08d980d7f4bf
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4422

SGkgSmFuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFhlbi1kZXZl
bCA8eGVuLWRldmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFsZiBPZiBK
YW4NCj4gQmV1bGljaA0KPiBTZW50OiAyMDIxxOo51MIyNMjVIDE4OjQ5DQo+IFRvOiBXZWkgQ2hl
biA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gQ2M6IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9y
ZzsganVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMNCj4gPEJlcnRyYW5kLk1hcnF1aXNA
YXJtLmNvbT47IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4g
U3ViamVjdDogUmU6IFtQQVRDSCAyMC8zN10geGVuOiBpbnRyb2R1Y2UgQ09ORklHX0VGSSB0byBz
dHViIEFQSSBmb3Igbm9uLQ0KPiBFRkkgYXJjaGl0ZWN0dXJlDQo+IA0KPiBPbiAyNC4wOS4yMDIx
IDEyOjMxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPj4gRnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNo
QHN1c2UuY29tPg0KPiA+PiBTZW50OiAyMDIxxOo51MIyNMjVIDE1OjU5DQo+ID4+DQo+ID4+IE9u
IDI0LjA5LjIwMjEgMDY6MzQsIFdlaSBDaGVuIHdyb3RlOg0KPiA+Pj4+IEZyb206IFN0ZWZhbm8g
U3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPj4+PiBTZW50OiAyMDIxxOo5
1MIyNMjVIDk6MTUNCj4gPj4+Pg0KPiA+Pj4+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVu
IHdyb3RlOg0KPiA+Pj4+PiAtLS0gYS94ZW4vY29tbW9uL0tjb25maWcNCj4gPj4+Pj4gKysrIGIv
eGVuL2NvbW1vbi9LY29uZmlnDQo+ID4+Pj4+IEBAIC0xMSw2ICsxMSwxNiBAQCBjb25maWcgQ09N
UEFUDQo+ID4+Pj4+ICBjb25maWcgQ09SRV9QQVJLSU5HDQo+ID4+Pj4+ICAJYm9vbA0KPiA+Pj4+
Pg0KPiA+Pj4+PiArY29uZmlnIEVGSQ0KPiA+Pj4+PiArCWJvb2wNCj4gPj4+Pg0KPiA+Pj4+IFdp
dGhvdXQgdGhlIHRpdGxlIHRoZSBvcHRpb24gaXMgbm90IHVzZXItc2VsZWN0YWJsZSAob3IgZGUt
DQo+IHNlbGVjdGFibGUpLg0KPiA+Pj4+IFNvIHRoZSBoZWxwIG1lc3NhZ2UgYmVsb3cgY2FuIG5l
dmVyIGJlIHNlZW4uDQo+ID4+Pj4NCj4gPj4+PiBFaXRoZXIgYWRkIGEgdGl0bGUsIGUuZy46DQo+
ID4+Pj4NCj4gPj4+PiBib29sICJFRkkgc3VwcG9ydCINCj4gPj4+Pg0KPiA+Pj4+IE9yIGZ1bGx5
IG1ha2UgdGhlIG9wdGlvbiBhIHNpbGVudCBvcHRpb24gYnkgcmVtb3ZpbmcgdGhlIGhlbHAgdGV4
dC4NCj4gPj4+DQo+ID4+PiBPSywgaW4gY3VycmVudCBYZW4gY29kZSwgRUZJIGlzIHVuY29uZGl0
aW9uYWxseSBjb21waWxlZC4gQmVmb3JlDQo+ID4+PiB3ZSBjaGFuZ2UgcmVsYXRlZCBjb2RlLCBJ
IHByZWZlciB0byByZW1vdmUgdGhlIGhlbHAgdGV4dC4NCj4gPj4NCj4gPj4gQnV0IHRoYXQncyBu
b3QgdHJ1ZTogQXQgbGVhc3Qgb24geDg2IEVGSSBnZXRzIGNvbXBpbGVkIGRlcGVuZGluZyBvbg0K
PiA+PiB0b29sIGNoYWluIGNhcGFiaWxpdGllcy4gVWx0aW1hdGVseSB3ZSBtYXkgaW5kZWVkIHdh
bnQgYSB1c2VyDQo+ID4+IHNlbGVjdGFibGUgb3B0aW9uIGhlcmUsIGJ1dCB1bnRpbCB0aGVuIEkn
bSBhZnJhaWQgaGF2aW5nIHRoaXMgb3B0aW9uDQo+ID4+IGF0IGFsbCBtYXkgYmUgbWlzbGVhZGlu
ZyBvbiB4ODYuDQo+ID4+DQo+ID4NCj4gPiBJIGNoZWNrIHRoZSBidWlsZCBzY3JpcHRzLCB5ZXMs
IHlvdSdyZSByaWdodC4gRm9yIHg4NiwgRUZJIGlzIG5vdCBhDQo+ID4gc2VsZWN0YWJsZSBvcHRp
b24gaW4gS2NvbmZpZy4gSSBhZ3JlZSB3aXRoIHlvdSwgd2UgY2FuJ3QgdXNlIEtjb25maWcNCj4g
PiBzeXN0ZW0gdG8gZGVjaWRlIHRvIGVuYWJsZSBFRkkgYnVpbGQgZm9yIHg4NiBvciBub3QuDQo+
ID4NCj4gPiBTbyBob3cgYWJvdXQgd2UganVzdCB1c2UgdGhpcyBFRkkgb3B0aW9uIGZvciBBcm0g
b25seT8gQmVjYXVzZSBvbiBBcm0sDQo+ID4gd2UgZG8gbm90IGhhdmUgc3VjaCB0b29sY2hhaW4g
ZGVwZW5kZW5jeS4NCj4gDQo+IFRvIGJlIGhvbmVzdCAtIGRvbid0IGtub3cuIFRoYXQncyBiZWNh
dXNlIEkgZG9uJ3Qga25vdyB3aGF0IHlvdSB3YW50DQo+IHRvIHVzZSB0aGUgb3B0aW9uIGZvciBz
dWJzZXF1ZW50bHkuDQo+IA0KDQpJbiBsYXN0IHZlcnNpb24sIEkgaGFkIGludHJvZHVjZWQgYW4g
YXJjaC1oZWxwZXIgdG8gc3R1YiBFRklfQk9PVA0KaW4gQXJtJ3MgY29tbW9uIGNvZGUgZm9yIEFy
bTMyLiBCZWNhdXNlIEFybTMyIGRvZXNuJ3Qgc3VwcG9ydCBFRkkuDQpTbyBKdWxpZW4gc3VnZ2Vz
dGVkIG1lIHRvIGludHJvZHVjZSBhIENPTkZJR19FRkkgb3B0aW9uIGZvciBub24tRUZJDQpzdXBw
b3J0ZWQgYXJjaGl0ZWN0dXJlcyB0byBzdHViIGluIEVGSSBsYXllci4NCg0KWzFdIGh0dHBzOi8v
bGlzdHMueGVucHJvamVjdC5vcmcvYXJjaGl2ZXMvaHRtbC94ZW4tZGV2ZWwvMjAyMS0wOC9tc2cw
MDgwOC5odG1sDQoNCj4gSmFuDQo+IA0KDQo=


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 11:02:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 11:02:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196078.348919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURvT-00029W-Vk; Sun, 26 Sep 2021 11:02:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196078.348919; Sun, 26 Sep 2021 11:02:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mURvT-00029P-So; Sun, 26 Sep 2021 11:02:31 +0000
Received: by outflank-mailman (input) for mailman id 196078;
 Sun, 26 Sep 2021 11:02:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mURvS-00029F-Bj; Sun, 26 Sep 2021 11:02:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mURvS-0006Ly-62; Sun, 26 Sep 2021 11:02:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mURvR-0005Gn-P1; Sun, 26 Sep 2021 11:02:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mURvR-0000V5-Ob; Sun, 26 Sep 2021 11:02:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8UIjU8CMr33USWypv6/6UdCl3PJWSqsvk2+D1zmWAog=; b=6YS6c6TFUcHuorlR9ayQRObb65
	SABtcvASocej8i9d26U7/wyNACqnRlwNsDCz4e9xxOoZUBB/mU3xktjMKPo8sAFsqM/LELUCaEVDb
	4NdcN4VTYFSyN6k8kDzBobM49ObRj9LRS5ekBSpGrXg+9zzzwhhn/ZKDqvwbfPlfwKIo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165200-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165200: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=46b4606ba23498d3d0e66b53e498eb3d5d592586
X-Osstest-Versions-That:
    ovmf=d60915b7516c87ec49ad579a1cb8ff9226d85928
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 11:02:29 +0000

flight 165200 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165200/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 46b4606ba23498d3d0e66b53e498eb3d5d592586
baseline version:
 ovmf                 d60915b7516c87ec49ad579a1cb8ff9226d85928

Last test of basis   165175  2021-09-24 03:46:51 Z    2 days
Testing same since   165200  2021-09-26 03:10:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  xueshengfeng <xueshengfeng@byosoft.com.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   d60915b751..46b4606ba2  46b4606ba23498d3d0e66b53e498eb3d5d592586 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 12:15:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 12:15:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196110.348934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUT3f-0002GL-Cn; Sun, 26 Sep 2021 12:15:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196110.348934; Sun, 26 Sep 2021 12:15:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUT3f-0002GE-9O; Sun, 26 Sep 2021 12:15:03 +0000
Received: by outflank-mailman (input) for mailman id 196110;
 Sun, 26 Sep 2021 12:15:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUT3e-0002G4-22; Sun, 26 Sep 2021 12:15:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUT3d-0007Wx-Q8; Sun, 26 Sep 2021 12:15:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUT3d-0000Ny-GP; Sun, 26 Sep 2021 12:15:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUT3d-00020M-Fw; Sun, 26 Sep 2021 12:15:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ktOG8QzHrVurBLPBf3LMGhgdKDqTH751vosHo0f/kuo=; b=pbOwZqTg0Dcdg0ZKF0vS8+YOVi
	UG9AIsw531KCNyQ0N4yidwCjKZUho7g4hrl2kfJI9fnMT1E76rkuJHdt96m6q/XnvdB13BnRX4P21
	UmF8UqUOcoH5ihIrdCPm/AqjPuGEragLI8hKMjoYjajr2MmzLO77AsQCdUKv1cCxl6+k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165198-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165198: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 12:15:01 +0000

flight 165198 xen-unstable real [real]
flight 165204 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165198/
http://logs.test-lab.xenproject.org/osstest/logs/165204/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-armhf-armhf-xl-rtds     18 guest-start/debian.repeat    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   15 days
Failing since        164951  2021-09-12 00:14:36 Z   14 days   29 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 13:13:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 13:13:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196123.348948 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUTy6-00011P-Nz; Sun, 26 Sep 2021 13:13:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196123.348948; Sun, 26 Sep 2021 13:13:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUTy6-00011I-Kg; Sun, 26 Sep 2021 13:13:22 +0000
Received: by outflank-mailman (input) for mailman id 196123;
 Sun, 26 Sep 2021 13:13:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ODXE=OQ=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUTy5-00011C-9A
 for xen-devel@lists.xenproject.org; Sun, 26 Sep 2021 13:13:21 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.82]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a05762b7-a0b2-4b88-90d7-78d11a945d36;
 Sun, 26 Sep 2021 13:13:18 +0000 (UTC)
Received: from DB6PR07CA0017.eurprd07.prod.outlook.com (2603:10a6:6:2d::27) by
 DB8PR08MB5516.eurprd08.prod.outlook.com (2603:10a6:10:f9::11) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15; Sun, 26 Sep 2021 13:13:16 +0000
Received: from DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2d:cafe::8) by DB6PR07CA0017.outlook.office365.com
 (2603:10a6:6:2d::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.8 via Frontend
 Transport; Sun, 26 Sep 2021 13:13:16 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT008.mail.protection.outlook.com (10.152.20.98) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Sun, 26 Sep 2021 13:13:16 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Sun, 26 Sep 2021 13:13:15 +0000
Received: from ea46bd69b4b9.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 683343D1-C111-4DCF-AA44-B128E6AA00CB.1; 
 Sun, 26 Sep 2021 13:13:04 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ea46bd69b4b9.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Sun, 26 Sep 2021 13:13:04 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB6155.eurprd08.prod.outlook.com (2603:10a6:10:1f5::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Sun, 26 Sep
 2021 13:13:01 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Sun, 26 Sep 2021
 13:13:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a05762b7-a0b2-4b88-90d7-78d11a945d36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ovA0XASrJjq008VECFLU/ddTx6Ar8xjVWsVwT5yfezg=;
 b=SdWHW/tObGtYBlcdm2e5pLprvuZ0PaRtw0McvC0rpU64FbzSCQWUZXI3kruFXbjO5/ZkieJtTLAX70kGpx9KT8xlETRPPnE8Ok1JdJleezsWGFF4X3i2x+lBUJIFX33Bbl8ZkgdyiWIf+CuXzzkeAurtqFjRAt5Pk8zDmFkpz4M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Sd5sY18UMc1pL2ewjelIS9L9DQaMtioRMkzCzK5o6KeLx1UwxjAPSxFieQ/eBU9YG6uVmqu3gnM494zkc66Mq9qcikp/z3q3h0S8jXJFec0a2seoIoDahyhDtQ05HW4WFfZyhhuWUHwQhd1dAgEFD9KHh5kz3VCrHZO5QKu6cZ6ZQdii/4SXSgejRBmVBGeyPPa5476gJRvvP6IU9f+DnvRNdWXhOumVq+ERCbqXfKYP5FbpVkLM0DhnmfQ4XghII2wjEAh8RlHkklvmMLkAc0ok+loHqcSfWZxZ/VfPIv12h9DTUxkSKWznNXlWDg7ACLo8twytlfx/corYtz7FXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ovA0XASrJjq008VECFLU/ddTx6Ar8xjVWsVwT5yfezg=;
 b=RV3C2ujYTdA/VTwWnYRRflYxOKe2ejQISOSndefEbDM7SERRyQiThDvpohbmVPEp6IOqb2v+VBNCn6kBoYmeL6rhQAYVFROFueUhC6//on/zOhFtJztCn4pvUKv42jzJlZHOf+Tf/6/ShFG8bWH1RSC9mzjs+isCrLexPeyBiO1vHnsSBrBjPmJgZjrAdBFHXUBljY4Uq1CpPu0AG6pIjelXCnQN0WK0ryA3ymunbObNZnf0rhYBGHrOj7ZMIZYJVrp2RhuuQJgKsLv+onwKQftD7vjI0q04t45XzqFeEVjHlMnBj2UoTKLS7Ai24x3oVeGq03qHLlKRHmvEmPUmpQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ovA0XASrJjq008VECFLU/ddTx6Ar8xjVWsVwT5yfezg=;
 b=SdWHW/tObGtYBlcdm2e5pLprvuZ0PaRtw0McvC0rpU64FbzSCQWUZXI3kruFXbjO5/ZkieJtTLAX70kGpx9KT8xlETRPPnE8Ok1JdJleezsWGFF4X3i2x+lBUJIFX33Bbl8ZkgdyiWIf+CuXzzkeAurtqFjRAt5Pk8zDmFkpz4M=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index: AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CA=
Date: Sun, 26 Sep 2021 13:13:01 +0000
Message-ID:
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: B8845E157B8DDA4D96BC41621B9A2BA7.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: ee920c2c-7447-4ed6-e2da-08d980ef6714
x-ms-traffictypediagnostic: DBBPR08MB6155:|DB8PR08MB5516:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB55166F7B0F5F8BD4A053A69F9EA69@DB8PR08MB5516.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 IiPz2rxj5Zn79Hj9T0sf3IH2mcWX7kFUQ/99VqGaIvF7Nq1JYEeTPjoGaWQZ/u9dL5E5yWGZh9WAYHZ3IDFYJu7883/VNuNnVATDT1Yv7ykYd8S3ngSVDdIe/CVxqV6GyA2obZ/by2nTnDGRjzqRFD24ZCe8/wUC3v6AYyvzO3Gn825SIgD6zkRe7fwgcCp1vnrDROl5BzX74zz+mox03m06UhxW2RGzGqNnd20mVRQk7oAyWf0bPISYtkNX2AKA3NwJqBp0W2r29qDsgSl7FGRScMyiNZ99Z1TR558xB/Td84jJrPbW65aFJkwIUQ6aPJnKqgDWPKszjj04vgNEShCLDuaHlXEKs38CcgJE6BDKfVmkiPIIdK5LozD1ZVfSySjr6GJt6tPhMM8z22L02Jj5VepFSSx1URb27fDlnfqNB5f9YWsOYnjTkxYsiaPYNlK7k6vIE88A8vhRTCueQ5JCfUHqFKkbaFHLekl26wwApUsJU9PdeQWvMQ+9qrryiTwqN/rvqsE5uIgpG7XeGUrLmFRMIa1J/Fra+dMlLYSxejjabWsQPBVvnB8WSbocfubHX6Qj/dTtZ4uuepfLrGYAh87IS+8xslvVnwnp6pdO7Df6+vwaG28MkLYcE3Gdba4bqXkhUz/7g5j50DRh+C25eHjHlXaTzCR/YtF21heAk9B8iNmQiuFDG3wzOTfgjIe8Y07f0Rz+m5ni60muuDO746sLNqUb7J8EZeqqD08lFWMITdtbuT1CIQNq1+/gVf50fZCspuJt7DwUWPn4Ujk+e5qiLbaRA/G+HED6WEc=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(966005)(38070700005)(9686003)(71200400001)(66476007)(86362001)(26005)(4326008)(66946007)(52536014)(83380400001)(6506007)(53546011)(55016002)(8676002)(66446008)(66556008)(122000001)(64756008)(186003)(54906003)(316002)(33656002)(7696005)(8936002)(38100700002)(6916009)(508600001)(2906002)(76116006);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6155
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b2d4c209-f5d8-4974-4715-08d980ef5e5d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2fMP9rGXyziCG3hpk3u/kYGKGdNLmCaTbaQmlNVfYoltH4rMehShcr8PFkh64DDMxCKXA4BcBvGaAIPvde6DGsyvD/0O2naZKeqacqmPrSAp3lGPf8vbU2PaqJpFhjshz9Xg1Ity6MOtPNeEm20ky1Xwa65JGD61K+njwLm3m965qeE3pxY3cf6XIifa6eKmdceC3hC9y81TMsgBHN+PA8GvmJUJU7bHOKwHUZyc9zPNM8s7XrOeyeVLKG/R2U3yb6HAhn2hVYv0nKjIP66UX1rrDLtu1jNxTJJuhuNt5mqS3UNMwSGRWosn7T//uyDiBp9R4FDlTFQWdYh5Vz8PtSoN1alzpaBo1l/W2JXNt6mFbdblFMVAd60b13iW3GP79SQ7N8tVJrD2fk9P3JbZ8AOoFX17VNL1GD+O3GNI6hFmQrtcUnjfXBcfau3M0Fxd2tGwG8/Om3jA0KQ2JFqdAzkISxmolC+Ljj/Bp7MmmDmKJ680Y1z5diUI81ACuL/GklhuY0099BvcKITL0LVyac4776Ulx3Z9eWNMetIF2EoO1aeLHgp2gsuKo58rgvN7K+cpNlTpJW/32YQFAlE9Vj1iyE1Fa12tDIONWnCoexwjG1y/iMGjId+pShuKCmweJ5atcHLzd2QjoPoCUFqSj5fXXVxzv4hwGiNvnIhlpHtGxIs3vG1R7EW4YAYB+ms05D5jR6BWzD4DvTXEI+cPNtgGqURb8a5nL5R7+sWm0ClPKtCYfu+8BXrFLBfJ6cSCaFBuxXYZUvgaOoN/TrnqewU8wUv/B012fLjM9q6yWBI=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(47076005)(8936002)(186003)(8676002)(336012)(54906003)(316002)(36860700001)(6506007)(6862004)(53546011)(83380400001)(4326008)(81166007)(2906002)(5660300002)(70586007)(966005)(33656002)(55016002)(70206006)(9686003)(82310400003)(52536014)(508600001)(26005)(356005)(7696005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2021 13:13:16.4516
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ee920c2c-7447-4ed6-e2da-08d980ef6714
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5516

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI0yNUgOTozNQ0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiB4ZW4t
ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7IGp1bGll
bkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9NRU1fQkFOS1Mg
dG8gb3ZlcnJpZGUgZGVmYXVsdA0KPiBOUl9OT0RFX01FTUJMS1MNCj4gDQo+IE9uIFRodSwgMjMg
U2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+IEFzIGEgbWVtb3J5IHJhbmdlIGRlc2NyaWJl
ZCBpbiBkZXZpY2UgdHJlZSBjYW5ub3QgYmUgc3BsaXQgYWNyb3NzDQo+ID4gbXVsdGlwbGUgbm9k
ZXMuIFNvIHdlIGRlZmluZSBOUl9OT0RFX01FTUJMS1MgYXMgTlJfTUVNX0JBTktTIGluDQo+ID4g
YXJjaCBoZWFkZXIuDQo+IA0KPiBUaGlzIHN0YXRlbWVudCBpcyB0cnVlIGJ1dCB3aGF0IGlzIHRo
ZSBnb2FsIG9mIHRoaXMgcGF0Y2g/IElzIGl0IHRvDQo+IHJlZHVjZSBjb2RlIHNpemUgYW5kIG1l
bW9yeSBjb25zdW1wdGlvbj8NCj4gDQoNCk5vLCB3aGVuIEp1bGllbiBhbmQgSSBkaXNjdXNzZWQg
dGhpcyBpbiBsYXN0IHZlcnNpb25bMV0sIHdlIGhhZG4ndCB0aG91Z2h0DQpzbyBkZWVwbHkuIFdl
IGp1c3QgdGhvdWdodCBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQgaW4gRFQgY2Fubm90IGJlIHNw
bGl0DQphY3Jvc3MgbXVsdGlwbGUgbm9kZXMuIFNvIE5SX01FTV9CQU5LUyBzaG91bGQgYmUgZXF1
YWwgdG8gTlJfTUVNX0JBTktTLg0KDQpodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hp
dmVzL2h0bWwveGVuLWRldmVsLzIwMjEtMDgvbXNnMDA5NzQuaHRtbCANCg0KPiBJIGFtIGFza2lu
ZyBiZWNhdXNlIE5SX01FTV9CQU5LUyBpcyAxMjggYW5kDQo+IE5SX05PREVfTUVNQkxLUz0yKk1B
WF9OVU1OT0RFUyB3aGljaCBpcyA2NCBieSBkZWZhdWx0IHNvIGFnYWluDQo+IE5SX05PREVfTUVN
QkxLUyBpcyAxMjggYmVmb3JlIHRoaXMgcGF0Y2guDQo+IA0KPiBJbiBvdGhlciB3b3JkcywgdGhp
cyBwYXRjaCBhbG9uZSBkb2Vzbid0IG1ha2UgYW55IGRpZmZlcmVuY2U7IGF0IGxlYXN0DQo+IGRv
ZXNuJ3QgbWFrZSBhbnkgZGlmZmVyZW5jZSB1bmxlc3MgQ09ORklHX05SX05VTUFfTk9ERVMgaXMg
aW5jcmVhc2VkLg0KPiANCj4gU28sIGlzIHRoZSBnb2FsIHRvIHJlZHVjZSBtZW1vcnkgdXNhZ2Ug
d2hlbiBDT05GSUdfTlJfTlVNQV9OT0RFUyBpcw0KPiBoaWdoZXIgdGhhbiA2ND8NCj4gDQoNCkkg
YWxzbyB0aG91Z2h0IGFib3V0IHRoaXMgcHJvYmxlbSB3aGVuIEkgd2FzIHdyaXRpbmcgdGhpcyBw
YXRjaC4NCkNPTkZJR19OUl9OVU1BX05PREVTIGlzIGluY3JlYXNpbmcsIGJ1dCBOUl9NRU1fQkFO
S1MgaXMgYSBmaXhlZA0KdmFsdWUsIHRoZW4gTlJfTUVNX0JBTktTIGNhbiBiZSBzbWFsbGVyIHRo
YW4gQ09ORklHX05SX05VTUFfTk9ERVMNCmF0IG9uZSBwb2ludC4NCg0KQnV0IEkgYWdyZWUgd2l0
aCBKdWxpZW4ncyBzdWdnZXN0aW9uLCBOUl9NRU1fQkFOS1MgYW5kIE5SX05PREVfTUVNQkxLUw0K
bXVzdCBiZSBhd2FyZSBvZiBlYWNoIG90aGVyLiBJIGhhZCB0aG91Z2h0IHRvIGFkZCBzb21lIEFT
U0VSVCBjaGVjaywNCmJ1dCBJIGRvbid0IGtub3cgaG93IHRvIGRvIGl0IGJldHRlci4gU28gSSBw
b3N0IHRoaXMgcGF0Y2ggZm9yIG1vcmUNCnN1Z2dlc3Rpb24uDQoNCj4gDQo+ID4gQW5kIGtlZXAg
ZGVmYXVsdCBOUl9OT0RFX01FTUJMS1MgaW4gY29tbW9uIGhlYWRlcg0KPiA+IGZvciB0aG9zZSBh
cmNoaXRlY3R1cmVzIE5VTUEgaXMgZGlzYWJsZWQuDQo+IA0KPiBUaGlzIGxhc3Qgc2VudGVuY2Ug
aXMgbm90IGFjY3VyYXRlOiBvbiB4ODYgTlVNQSBpcyBlbmFibGVkIGFuZA0KPiBOUl9OT0RFX01F
TUJMS1MgaXMgc3RpbGwgZGVmaW5lZCBpbiB4ZW4vaW5jbHVkZS94ZW4vbnVtYS5oICh0aGVyZSBp
cyBubw0KPiB4ODYgZGVmaW5pdGlvbiBvZiBpdCkNCj4gDQoNClllcy4NCg0KPiANCj4gPiBTaWdu
ZWQtb2ZmLWJ5OiBXZWkgQ2hlbiA8d2VpLmNoZW5AYXJtLmNvbT4NCj4gPiAtLS0NCj4gPiAgeGVu
L2luY2x1ZGUvYXNtLWFybS9udW1hLmggfCA4ICsrKysrKystDQo+ID4gIHhlbi9pbmNsdWRlL3hl
bi9udW1hLmggICAgIHwgMiArKw0KPiA+ICAyIGZpbGVzIGNoYW5nZWQsIDkgaW5zZXJ0aW9ucygr
KSwgMSBkZWxldGlvbigtKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1h
cm0vbnVtYS5oIGIveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4gPiBpbmRleCA4ZjFjNjdl
M2ViLi4yMTU2OWU2MzRiIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVt
YS5oDQo+ID4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4gPiBAQCAtMyw5ICsz
LDE1IEBADQo+ID4NCj4gPiAgI2luY2x1ZGUgPHhlbi9tbS5oPg0KPiA+DQo+ID4gKyNpbmNsdWRl
IDxhc20vc2V0dXAuaD4NCj4gPiArDQo+ID4gIHR5cGVkZWYgdTggbm9kZWlkX3Q7DQo+ID4NCj4g
PiAtI2lmbmRlZiBDT05GSUdfTlVNQQ0KPiA+ICsjaWZkZWYgQ09ORklHX05VTUENCj4gPiArDQo+
ID4gKyNkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIE5SX01FTV9CQU5LUw0KPiA+ICsNCj4gPiArI2Vs
c2UNCj4gPg0KPiA+ICAvKiBGYWtlIG9uZSBub2RlIGZvciBub3cuIFNlZSBhbHNvIG5vZGVfb25s
aW5lX21hcC4gKi8NCj4gPiAgI2RlZmluZSBjcHVfdG9fbm9kZShjcHUpIDANCj4gPiBkaWZmIC0t
Z2l0IGEveGVuL2luY2x1ZGUveGVuL251bWEuaCBiL3hlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4g
PiBpbmRleCAxOTc4ZTJiZTFiLi4xNzMxZTFjYzZiIDEwMDY0NA0KPiA+IC0tLSBhL3hlbi9pbmNs
dWRlL3hlbi9udW1hLmgNCj4gPiArKysgYi94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+ID4gQEAg
LTEyLDcgKzEyLDkgQEANCj4gPiAgI2RlZmluZSBNQVhfTlVNTk9ERVMgICAgMQ0KPiA+ICAjZW5k
aWYNCj4gPg0KPiA+ICsjaWZuZGVmIE5SX05PREVfTUVNQkxLUw0KPiA+ICAjZGVmaW5lIE5SX05P
REVfTUVNQkxLUyAoTUFYX05VTU5PREVTKjIpDQo+ID4gKyNlbmRpZg0KPiA+DQo+ID4gICNkZWZp
bmUgdmNwdV90b19ub2RlKHYpIChjcHVfdG9fbm9kZSgodiktPnByb2Nlc3NvcikpDQo+ID4NCj4g
PiAtLQ0KPiA+IDIuMjUuMQ0KPiA+DQo=


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 14:46:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 14:46:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196152.348958 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUVQ2-0003Fs-Lr; Sun, 26 Sep 2021 14:46:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196152.348958; Sun, 26 Sep 2021 14:46:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUVQ2-0003Fl-Iw; Sun, 26 Sep 2021 14:46:18 +0000
Received: by outflank-mailman (input) for mailman id 196152;
 Sun, 26 Sep 2021 14:46:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUVQ1-0003Fb-Sd; Sun, 26 Sep 2021 14:46:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUVQ1-0001b6-OV; Sun, 26 Sep 2021 14:46:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUVQ1-0007Pi-DJ; Sun, 26 Sep 2021 14:46:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUVQ1-0006MF-B9; Sun, 26 Sep 2021 14:46:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yGaHnzLVS4oIuKkIVklTH+XGLXsjhOI4xnoO6Y0UsOI=; b=m3WFUDvTGaZtNuv9914lMpfUDq
	uoZFYVoK+jxJqSZ9DI5DrRHA3TGD+WsbKp5NsIaHmZ9ATR0a/3WumrJXP0DEjvWhQUklXhz32JCbu
	tizVMM1qwkmDCnJKSSTn0CjGxqoLgdybn6RGjHQNk4c3+CAjfN03SVEsA0nEwkPJ7wVI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165201-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165201: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=49d74124a55fac3b2ed9574b7642887c1da49f0b
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 14:46:17 +0000

flight 165201 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165201/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              49d74124a55fac3b2ed9574b7642887c1da49f0b
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  443 days
Failing since        151818  2020-07-11 04:18:52 Z  442 days  433 attempts
Testing same since   165201  2021-09-26 04:20:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 75999 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 15:39:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 15:39:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196166.348973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUWF4-0001CU-Qe; Sun, 26 Sep 2021 15:39:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196166.348973; Sun, 26 Sep 2021 15:39:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUWF4-0001CN-Ml; Sun, 26 Sep 2021 15:39:02 +0000
Received: by outflank-mailman (input) for mailman id 196166;
 Sun, 26 Sep 2021 15:39:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUWF3-0001CD-B9; Sun, 26 Sep 2021 15:39:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUWF3-0002SU-4m; Sun, 26 Sep 2021 15:39:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUWF2-0001M3-Nq; Sun, 26 Sep 2021 15:39:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUWF2-0006Ms-NI; Sun, 26 Sep 2021 15:39:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=R7Ynqc5YNbRHzKHFMheBzLzqQ/WJbIwrrSfDkXi/cbU=; b=l+JPHzrA0iu01/oIbnMZ567PA4
	uPvJmHmAl8WhLTU/JAfWJBCKmx+fSc0g9EwyGQb1527VMI3g5pRuVWJBBUipnCUBktMJ6bRhndGHT
	ZZP7BfkqpFxVl79wOyI/951u5wAfXb2EEK+vGi1rtQeyHr24/KCf63Yi10Ltuf0zZd/U=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165199-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165199: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-saverestore.2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a3b397b4fffb799d25658defafd962f0fb3e9fe0
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 15:39:00 +0000

flight 165199 linux-linus real [real]
flight 165207 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165199/
http://logs.test-lab.xenproject.org/osstest/logs/165207/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 18 guest-saverestore.2 fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a3b397b4fffb799d25658defafd962f0fb3e9fe0
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  421 days
Failing since        152366  2020-08-01 20:49:34 Z  420 days  736 attempts
Testing same since   165199  2021-09-26 02:50:44 Z    0 days    1 attempts

------------------------------------------------------------
7339 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2270952 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 21:49:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 21:49:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196199.348987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUc16-0000Yz-Ez; Sun, 26 Sep 2021 21:49:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196199.348987; Sun, 26 Sep 2021 21:49:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUc16-0000Ys-Ay; Sun, 26 Sep 2021 21:49:00 +0000
Received: by outflank-mailman (input) for mailman id 196199;
 Sun, 26 Sep 2021 21:48:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUc15-0000Yi-Np; Sun, 26 Sep 2021 21:48:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUc15-0000kt-HZ; Sun, 26 Sep 2021 21:48:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUc15-0005If-3U; Sun, 26 Sep 2021 21:48:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUc15-0008LM-2v; Sun, 26 Sep 2021 21:48:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ruqo90XoK1NhTEiwOGI9vXw1GE6xfmHW2w5W3t7rN20=; b=zL8Q28j7aV2dA9IrKxVaqRIaHN
	lYGNpSPffn5ZVT/ZNwL7F6bE5Q1Qb01T4sjAPqslsd9sP+7qog+lrquRNkSn5u5nHWX6AheJSnB48
	aZaYfdc8sves0OoCL3xZxPBdzqWBP1Derlc9bVsuoiF7sAmGw7va8PHNUKGxYzAKjh9s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165202-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165202: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=11a11998460ed84d9a127c025f50f7234e5a483f
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 26 Sep 2021 21:48:59 +0000

flight 165202 qemu-mainline real [real]
flight 165209 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165202/
http://logs.test-lab.xenproject.org/osstest/logs/165209/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                11a11998460ed84d9a127c025f50f7234e5a483f
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   15 days
Failing since        164967  2021-09-13 13:06:52 Z   13 days   26 attempts
Testing same since   165193  2021-09-25 11:09:10 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5088 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Sep 26 22:54:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 26 Sep 2021 22:54:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196214.349001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUd1t-0008Qy-4V; Sun, 26 Sep 2021 22:53:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196214.349001; Sun, 26 Sep 2021 22:53:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUd1t-0008Qr-0e; Sun, 26 Sep 2021 22:53:53 +0000
Received: by outflank-mailman (input) for mailman id 196214;
 Sun, 26 Sep 2021 22:53:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YEj3=OQ=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mUd1r-0008Ql-LM
 for xen-devel@lists.xenproject.org; Sun, 26 Sep 2021 22:53:51 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9d0e5166-1f1c-11ec-bc2e-12813bfff9fa;
 Sun, 26 Sep 2021 22:53:50 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 18QMrfNX098393
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Sun, 26 Sep 2021 18:53:47 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 18QMrfqD098392;
 Sun, 26 Sep 2021 15:53:41 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d0e5166-1f1c-11ec-bc2e-12813bfff9fa
Date: Sun, 26 Sep 2021 15:53:41 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YVD59QVbmdVwzYQI@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
 <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Wed, Sep 15, 2021 at 08:05:05AM +0200, Jan Beulich wrote:
> On 15.09.2021 04:40, Elliott Mitchell wrote:
> > On Tue, Sep 07, 2021 at 05:57:10PM +0200, Jan Beulich wrote:
> >> On 07.09.2021 17:03, Elliott Mitchell wrote:
> >>>  Could be this system is in an
> >>> intergenerational hole, and some spot in the PVH/HVM code makes an
> >>> assumption of the presence of NPT guarantees presence of an operational
> >>> IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
> >>> code, some portion of the IOMMU code might be checking for presence of
> >>> NPT instead of presence of IOMMU.
> >>
> >> This is all very speculative; I consider what you suspect not very likely,
> >> but also not entirely impossible. This is not the least because for a
> >> long time we've been running without shared page tables on AMD.
> >>
> >> I'm afraid without technical data and without knowing how to repro, I
> >> don't see a way forward here.
> > 
> > Downtimes are very expensive even for lower-end servers.  Plus there is
> > the issue the system wasn't meant for development and thus never had
> > appropriate setup done.
> > 
> > Experimentation with a system of similar age suggested another candidate.
> > System has a conventional BIOS.  Might some dependancies on the presence
> > of UEFI snuck into the NPT code?
> 
> I can't think of any such, but as all of this is very nebulous I can't
> really rule out anything.

Getting everything right to recreate is rather inexact.  Having an
equivalent of `sysctl` to turn on the serial console while running might
be handy...

Luckily get things together and...

(XEN) mm locking order violation: 48 > 16
(XEN) Xen BUG at mm-locks.h:82
(XEN) ----[ Xen-4.14.3  x86_64  debug=n   Not tainted ]----
(XEN) CPU:    2
(XEN) RIP:    e008:[<ffff82d0402e8be0>] arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
(XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d1v0)
(XEN) rax: ffff83080b2f106c   rbx: ffff83081da0f2d0   rcx: 0000000000000000
(XEN) rdx: ffff83080b27ffff   rsi: 000000000000000a   rdi: ffff82d040469738
(XEN) rbp: ffff82d040580688   rsp: ffff83080b27f8b0   r8:  0000000000000002
(XEN) r9:  0000000000008000   r10: ffff82d04058f381   r11: ffff82d040375100
(XEN) r12: ffff82d040580688   r13: ffff83080b27ffff   r14: ffff83081ddf6000
(XEN) r15: 00000000004f8c00   cr0: 000000008005003b   cr4: 00000000000406e0
(XEN) cr3: 000000081dee6000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0010   gs: 0010   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d0402e8be0> (arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260):
(XEN)  e3 0c 00 e8 30 7f f6 ff <0f> 0b 66 0f 1f 44 00 00 42 8b 34 20 48 8d 3d 8d
(XEN) Xen stack trace from rsp=ffff83080b27f8b0:
(XEN)    ffff83081ddf67c8 ffff83081ddf6810 ffff82d040580688 000000081ddf4067
(XEN)    0000000000000001 ffff82d0402ec51c ffff83081ddf6000 0000000000000000
(XEN)    ffff82d0402e0528 ffff83081da0f010 ffff83081dde9000 ffff830800000002
(XEN)    ffff83081ddf6690 0000000000000001 ffff83081dde9000 ffff83081ddf5000
(XEN)    0000000000000000 ffff83081da0f010 ffff83081da0f010 00000000004f8c00
(XEN)    ffff82d0402f009a 0000000000000067 0000000100000000 ffff83080b27fa00
(XEN)    ffff83081dde9000 000000000081ddf4 ffff83081ddf4000 ffff83081da0f010
(XEN)    0000000000000000 0000000000000006 0000000000000000 0000000000000000
(XEN)    ffff83080b27f9f0 ffff82d0402f1097 0000000000000001 0000000000000000
(XEN)    ffffffffffffffff ffff83081ddf6000 ffff83080b27fa00 0000000400000000
(XEN)    0000000000000000 0000000000000000 ffff83081dde9000 0000000000000000
(XEN)    0000000000000000 ffffffffffffffff 0000000000000001 0000000000000001
(XEN)    0000000000000000 ffff83081ddf6000 0000000000000000 ffff82d0402ea0a6
(XEN)    ffffffffffffffff ffff83081da0f010 0000000700000006 ffff8304f8c00000
(XEN)    ffff83081da0f010 0000000000000000 ffff83080b27fba0 ffff83080b27fc98
(XEN)    0000000000000000 ffff82d0402f4ecd ffff83080b27fac8 ffff83080b27fb20
(XEN)    ffff83081ddf6000 ffff83080b27fae0 0000000100000000 0000000000000007
(XEN)    0000000000000002 ffff83081ca88018 ffff830800000000 0000000000000012
(XEN)    ffff82d0402f023f ffff82d0402f02ed 00000000000fa400 ffff82d0402f00ed
(XEN)    ffff83080b27fb38 ffff82d0402e03da 00000000004f8c00 ffff83081dff1e90
(XEN) Xen call trace:
(XEN)    [<ffff82d0402e8be0>] R arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
(XEN)    [<ffff82d0402ec51c>] S p2m_flush_nestedp2m+0x1c/0x30
(XEN)    [<ffff82d0402e0528>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x378/0x490
(XEN)    [<ffff82d0402f009a>] S arch/x86/mm/p2m-pt.c#p2m_next_level.constprop.10+0x24a/0x2e0
(XEN)    [<ffff82d0402f1097>] S arch/x86/mm/p2m-pt.c#p2m_pt_set_entry+0x3c7/0x7b0
(XEN)    [<ffff82d0402ea0a6>] S p2m_set_entry+0xa6/0x130
(XEN)    [<ffff82d0402f4ecd>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check+0x1cd/0x440
(XEN)    [<ffff82d0402f023f>] S arch/x86/mm/p2m-pt.c#do_recalc+0x10f/0x470
(XEN)    [<ffff82d0402f02ed>] S arch/x86/mm/p2m-pt.c#do_recalc+0x1bd/0x470
(XEN)    [<ffff82d0402f00ed>] S arch/x86/mm/p2m-pt.c#p2m_next_level.constprop.10+0x29d/0x2e0
(XEN)    [<ffff82d0402e03da>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x22a/0x490
(XEN)    [<ffff82d0402f0fe2>] S arch/x86/mm/p2m-pt.c#p2m_pt_set_entry+0x312/0x7b0
(XEN)    [<ffff82d0402f0c4e>] S arch/x86/mm/p2m-pt.c#p2m_pt_get_entry+0x3fe/0x480
(XEN)    [<ffff82d0402f59aa>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check_superpage+0x17a/0x600
(XEN)    [<ffff82d0402f5ba0>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check_superpage+0x370/0x600
(XEN)    [<ffff82d0402f7c78>] S p2m_pod_demand_populate+0x6b8/0xa90
(XEN)    [<ffff82d0402f0aa6>] S arch/x86/mm/p2m-pt.c#p2m_pt_get_entry+0x256/0x480
(XEN)    [<ffff82d0402e9a1f>] S __get_gfn_type_access+0x6f/0x130
(XEN)    [<ffff82d0402ab12b>] S hvm_hap_nested_page_fault+0xeb/0x760
(XEN)    [<ffff82d04028c87e>] S svm_asm_do_resume+0x12e/0x164
(XEN)    [<ffff82d04028c87e>] S svm_asm_do_resume+0x12e/0x164

The stack trace goes further, but I suspect the rest would be overkill.
That seems to readily qualify as "Xen bug".


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Mon Sep 27 00:56:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 00:56:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196235.349012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUewQ-0005fe-6L; Mon, 27 Sep 2021 00:56:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196235.349012; Mon, 27 Sep 2021 00:56:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUewQ-0005fX-1o; Mon, 27 Sep 2021 00:56:22 +0000
Received: by outflank-mailman (input) for mailman id 196235;
 Mon, 27 Sep 2021 00:56:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUewO-0005fN-CI; Mon, 27 Sep 2021 00:56:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUewO-0004QC-41; Mon, 27 Sep 2021 00:56:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUewN-0006W4-Rt; Mon, 27 Sep 2021 00:56:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUewN-00056f-RJ; Mon, 27 Sep 2021 00:56:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ux0Vd4t2W+Ljy16e9krDSXm6thCeVXWEcxO7JlX/7rw=; b=R78eBbRSk2hURu9fjX0H86+fXb
	O+l8iZ99Izj49zs6RIM4DJDxzYUbH8xBloRM7wdU/Xx006i/TS4Qh+D8dCmMd/GOLLbyK9wGfE3Y4
	6/9k9Y/5n/+sbhcEqXmwrcBjmApvn/vog45ev5Uqx2n52xSJvY69KBJLfDwxoRtzE0oQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165205-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165205: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 00:56:19 +0000

flight 165205 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165205/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   8 xen-boot                   fail pass in 165198
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 165198
 test-armhf-armhf-xl-rtds     14 guest-start                fail pass in 165198

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 165198 like 164945
 test-arm64-arm64-xl-credit2 15 migrate-support-check fail in 165198 never pass
 test-arm64-arm64-xl-credit2 16 saverestore-support-check fail in 165198 never pass
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 165198 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 165198 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   16 days
Failing since        164951  2021-09-12 00:14:36 Z   15 days   30 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  fail    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 02:31:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 02:31:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196251.349026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUgQG-00073X-9q; Mon, 27 Sep 2021 02:31:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196251.349026; Mon, 27 Sep 2021 02:31:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUgQG-00073P-4q; Mon, 27 Sep 2021 02:31:16 +0000
Received: by outflank-mailman (input) for mailman id 196251;
 Mon, 27 Sep 2021 02:31:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUgQE-00073E-L5; Mon, 27 Sep 2021 02:31:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUgQE-0004F9-Bm; Mon, 27 Sep 2021 02:31:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUgQE-0002Hh-3F; Mon, 27 Sep 2021 02:31:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUgQE-0007U0-2Z; Mon, 27 Sep 2021 02:31:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=h/IFySZY+AR7tPO4A1r+fAfic4/gHi/37JQh7zhv1fo=; b=nelrSPNR9FfJSpBliGMvUmp3R8
	qJ3hH3WergRwHM7p4ezLY4tLHP37UnzBodMuqdA/8eJE8afFtz1yxJsB8CPJ1lUNGh+sfJwAtrJC9
	6/WQ1wbJD4bOr2LiopoeGcVFb4KuU1miEBVSkAiESIlXMZcYeKCiCOuUipull2hFkV7s=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165206-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 165206: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    linux-5.4:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e74e2950a0d6f800858e54860d7124c86e494f62
X-Osstest-Versions-That:
    linux=07e5f23d3fa6ca98457d1a2177a735fcc65923c2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 02:31:14 +0000

flight 165206 linux-5.4 real [real]
flight 165212 linux-5.4 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165206/
http://logs.test-lab.xenproject.org/osstest/logs/165212/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail pass in 165212-retest
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail pass in 165212-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 165212 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 165212 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 165152
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 165152
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 165152
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 165152
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 165152
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 165152
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 165152
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 165152
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 165152
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 165152
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 165152
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 165152
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                e74e2950a0d6f800858e54860d7124c86e494f62
baseline version:
 linux                07e5f23d3fa6ca98457d1a2177a735fcc65923c2

Last test of basis   165152  2021-09-22 10:44:14 Z    4 days
Testing same since   165206  2021-09-26 12:42:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Sverdlin <alexander.sverdlin@nokia.com>
  Alexander Sverdlin <alexander.sverdlin@nokia.com>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Anand Jain <anand.jain@oracle.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrey Vagin <avagin@gmail.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Anton Ivanov <anton.ivanov@cambridgegreys.com>
  Arnd Bergmann <arnd@arndb.de>
  Bjorn Helgaas <bhelgaas@google.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Cyrill Gorcunov <gorcunov@gmail.com>
  Dan Carpenter <dan.carpenter@oracle.com>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  dann frazier <dann.frazier@canonical.com>
  Dave Jiang <dave.jiang@intel.com>
  David Hildenbrand <david@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Dominique Martinet <asmadeus@codewreck.org>
  Florian Fainelli <f.fainelli@gmail.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Grzegorz Jaszczyk <jaz@semihalf.com>
  Guenter Roeck <linux@roeck-us.net>
  Helge Deller <deller@gmx.de>
  Ilya Dryomov <idryomov@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Jens Axboe <axboe@kernel.dk>
  Johannes Berg <johannes.berg@intel.com>
  John Johansen <john.johansen@canonical.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jongsung Kim <neidhard.kim@lge.com>
  Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
  Li Jinlin <lijinlin3@huawei.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
  Lukas Bulwahn <lukas.bulwahn@gmail.com>
  Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>
  Nanyong Sun <sunnanyong@huawei.com>
  Nathan Chancellor <nathan@kernel.org>
  nick black <dankamongmen@gmail.com>
  Nitesh Narayan Lal <nitesh@redhat.com>
  Nobuhiro Iwamatsu (CIP) <nobuhiro1.iwamatsu@toshiba.co.jp>
  Pali Rohár <pali@kernel.org>
  Paolo Bonzini <pbonzini@redhat.com>
  Pavel Skripkin <paskripkin@gmail.com>
  Peter Zijlstra <peterz@infradead.org>
  Petr Oros <poros@redhat.com>
  Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
  Radim Krčmář <rkrcmar@redhat.com>
  Richard Weinberger <richard@nod.at>
  Russell King <rmk+kernel@armlinux.org.uk>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Sasha Levin <sashal@kernel.org>
  Shuah Khan <skhan@linuxfoundation.org>
  Su Yue <l@damenly.su>
  Thierry Reding <thierry.reding@gmail.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tuan Phan <tuanphan@os.amperecomputing.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vasily Gorbik <gor@linux.ibm.com>
  Vinod Koul <vkoul@kernel.org>
  Will Deacon <will@kernel.org>
  Xie Yongji <xieyongji@bytedance.com>
  Yu-Tung Chang <mtwget@gmail.com>
  Zhen Lei <thunder.leizhen@huawei.com>
  Zou Wei <zou_wei@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   07e5f23d3fa6..e74e2950a0d6  e74e2950a0d6f800858e54860d7124c86e494f62 -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 03:13:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 03:13:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196263.349040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUh4r-000427-F9; Mon, 27 Sep 2021 03:13:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196263.349040; Mon, 27 Sep 2021 03:13:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUh4r-000420-CD; Mon, 27 Sep 2021 03:13:13 +0000
Received: by outflank-mailman (input) for mailman id 196263;
 Mon, 27 Sep 2021 03:13:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUh4q-00041u-4V
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 03:13:12 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d7fc67c6-1f40-11ec-bc3d-12813bfff9fa;
 Mon, 27 Sep 2021 03:13:10 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id EF28E60F94;
 Mon, 27 Sep 2021 03:13:09 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7fc67c6-1f40-11ec-bc3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632712390;
	bh=lW7cNz9qpneRbTbro/qNVhThXLLuWAbKMniU9v7wSac=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=BP0IYWiHAkyGTRpFO5dih2CjQEE5OYS0GosDdoggCfjBR0I8zrpSuX7SJKSqOYBJY
	 HCSMcPEIYIOEhjsgwY6ZHGwilSl9VR6Yh34P9RaJipqyLecAHb5BOeehXApIalf+PM
	 6N2ZxZgkjT4vdKAyZmzPRN38C70slrVB9ODBXTmd5OwpzxEFDchJteGSMz/lNfDPby
	 POcy2WHeVUSowq7REdg79LEdcp7buIPdmDhVrK7tPI5LbWv2pTjj1PsD7ssHnVuvh9
	 lUA3ovileOn55UfqrQKORzq+2gIZegSKGXDU4Sx/GTUSgbO5hi54CBDvw6t9dlXez0
	 /jMc+pvVJkb3w==
Date: Sun, 26 Sep 2021 20:13:09 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    "jbeulich@suse.com" <jbeulich@suse.com>, 
    "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com> <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1278767783-1632712390=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1278767783-1632712390=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sun, 26 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月25日 3:53
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>; jbeulich@suse.com; andrew.cooper3@citrix.com;
> > roger.pau@citrix.com; wl@xen.org
> > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node
> > memory range
> > 
> > On Fri, 24 Sep 2021, Wei Chen wrote:
> > > > -----Original Message-----
> > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > Sent: 2021年9月24日 8:26
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > julien@xen.org;
> > > > Bertrand Marquis <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > > > andrew.cooper3@citrix.com; roger.pau@citrix.com; wl@xen.org
> > > > Subject: Re: [PATCH 08/37] xen/x86: add detection of discontinous node
> > > > memory range
> > > >
> > > > CC'ing x86 maintainers
> > > >
> > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > One NUMA node may contain several memory blocks. In current Xen
> > > > > code, Xen will maintain a node memory range for each node to cover
> > > > > all its memory blocks. But here comes the problem, in the gap of
> > > > > one node's two memory blocks, if there are some memory blocks don't
> > > > > belong to this node (remote memory blocks). This node's memory range
> > > > > will be expanded to cover these remote memory blocks.
> > > > >
> > > > > One node's memory range contains othe nodes' memory, this is
> > obviously
> > > > > not very reasonable. This means current NUMA code only can support
> > > > > node has continous memory blocks. However, on a physical machine,
> > the
> > > > > addresses of multiple nodes can be interleaved.
> > > > >
> > > > > So in this patch, we add code to detect discontinous memory blocks
> > > > > for one node. NUMA initializtion will be failed and error messages
> > > > > will be printed when Xen detect such hardware configuration.
> > > >
> > > > At least on ARM, it is not just memory that can be interleaved, but
> > also
> > > > MMIO regions. For instance:
> > > >
> > > > node0 bank0 0-0x1000000
> > > > MMIO 0x1000000-0x1002000
> > > > Hole 0x1002000-0x2000000
> > > > node0 bank1 0x2000000-0x3000000
> > > >
> > > > So I am not familiar with the SRAT format, but I think on ARM the
> > check
> > > > would look different: we would just look for multiple memory ranges
> > > > under a device_type = "memory" node of a NUMA node in device tree.
> > > >
> > > >
> > >
> > > Should I need to include/refine above message to commit log?
> > 
> > Let me ask you a question first.
> > 
> > With the NUMA implementation of this patch series, can we deal with
> > cases where each node has multiple memory banks, not interleaved?
> 
> Yes.
> 
> > An an example:
> > 
> > node0: 0x0        - 0x10000000
> > MMIO : 0x10000000 - 0x20000000
> > node0: 0x20000000 - 0x30000000
> > MMIO : 0x30000000 - 0x50000000
> > node1: 0x50000000 - 0x60000000
> > MMIO : 0x60000000 - 0x80000000
> > node2: 0x80000000 - 0x90000000
> > 
> > 
> > I assume we can deal with this case simply by setting node0 memory to
> > 0x0-0x30000000 even if there is actually something else, a device, that
> > doesn't belong to node0 in between the two node0 banks?
> 
> While this configuration is rare in SoC design, but it is not impossible. 

Definitely, I have seen it before.


> > Is it only other nodes' memory interleaved that cause issues? In other
> > words, only the following is a problematic scenario?
> > 
> > node0: 0x0        - 0x10000000
> > MMIO : 0x10000000 - 0x20000000
> > node1: 0x20000000 - 0x30000000
> > MMIO : 0x30000000 - 0x50000000
> > node0: 0x50000000 - 0x60000000
> > 
> > Because node1 is in between the two ranges of node0?
> > 
> 
> But only device_type="memory" can be added to allocation.
> For mmio there are two cases:
> 1. mmio doesn't have NUMA id property.
> 2. mmio has NUMA id property, just like some PCIe controllers.
>    But we don’t need to handle these kinds of MMIO devices
>    in memory block parsing. Because we don't need to allocate
>    memory from these mmio ranges. And for accessing, we need
>    a NUMA-aware PCIe controller driver or a generic NUMA-aware
>    MMIO accessing APIs.

Yes, I am not too worried about devices with a NUMA id property because
they are less common and this series doesn't handle them at all, right?
I imagine they would be treated like any other device without NUMA
awareness.

I am thinking about the case where the memory of each NUMA node is made
of multiple banks. I understand that this patch adds an explicit check
for cases where these banks are interleaving, however there are many
other cases where NUMA memory nodes are *not* interleaving but they are
still made of multiple discontinuous banks, like in the two example
above.

My question is whether this patch series in its current form can handle
the two cases above correctly. If so, I am wondering how it works given
that we only have a single "start" and "size" parameter per node.

On the other hand if this series cannot handle the two cases above, my
question is whether it would fail explicitly or not. The new
check is_node_memory_continuous doesn't seem to be able to catch them.


> > I am asking these questions because it is certainly possible to have
> > multiple memory ranges for each NUMA node in device tree, either by
> > specifying multiple ranges with a single "reg" property, or by
> > specifying multiple memory nodes with the same numa-node-id.
> 
> 
> 
--8323329-1278767783-1632712390=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 03:25:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 03:25:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196281.349051 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUhGz-0005kO-Ix; Mon, 27 Sep 2021 03:25:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196281.349051; Mon, 27 Sep 2021 03:25:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUhGz-0005kH-Fy; Mon, 27 Sep 2021 03:25:45 +0000
Received: by outflank-mailman (input) for mailman id 196281;
 Mon, 27 Sep 2021 03:25:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUhGy-0005kB-9k
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 03:25:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 988b2f6c-1f42-11ec-bc3d-12813bfff9fa;
 Mon, 27 Sep 2021 03:25:43 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 825CF60FED;
 Mon, 27 Sep 2021 03:25:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 988b2f6c-1f42-11ec-bc3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632713142;
	bh=6ZAemB+n9XgH6FGRuRLlgBbacK01h5ysUjv5fAYyDi0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=S5dK702KQ89G+93xNqPSAPNJgOqRz7Q9qBLCthmpUXT550pZHUr3NmM5rSuDWeyKD
	 1pPypqSMPaJhxgk6Eig4wH7iWqllJ451o6lByfuZl7BxuuabwvlNxoIUaWOLDP3m+C
	 5KMd7csnsIlDPXkuaStFjCmgvaEBPbergNyNx7n4fb4vccFFtWH1VlwlyhpfvnTAlY
	 KF66xXIKYkJm3KUyjpz1mdxmvmletp/KhPeDZ562TjpfESNp70d1ac2Dw0e23UcGc3
	 grlfFaW0t/S/hiYut8ZBZrqWf/clZ2xg49NF2vrSfU1NJ+ejjHxI5swT7ian2sAqzs
	 JaLFsZbY5znYQ==
Date: Sun, 26 Sep 2021 20:25:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
In-Reply-To: <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com> <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1750427606-1632713142=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1750427606-1632713142=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sun, 26 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月24日 9:35
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org; julien@xen.org;
> > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
> > NR_NODE_MEMBLKS
> > 
> > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > As a memory range described in device tree cannot be split across
> > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS in
> > > arch header.
> > 
> > This statement is true but what is the goal of this patch? Is it to
> > reduce code size and memory consumption?
> > 
> 
> No, when Julien and I discussed this in last version[1], we hadn't thought
> so deeply. We just thought a memory range described in DT cannot be split
> across multiple nodes. So NR_MEM_BANKS should be equal to NR_MEM_BANKS.
> 
> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00974.html 
> 
> > I am asking because NR_MEM_BANKS is 128 and
> > NR_NODE_MEMBLKS=2*MAX_NUMNODES which is 64 by default so again
> > NR_NODE_MEMBLKS is 128 before this patch.
> > 
> > In other words, this patch alone doesn't make any difference; at least
> > doesn't make any difference unless CONFIG_NR_NUMA_NODES is increased.
> > 
> > So, is the goal to reduce memory usage when CONFIG_NR_NUMA_NODES is
> > higher than 64?
> > 
> 
> I also thought about this problem when I was writing this patch.
> CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
> value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NODES
> at one point.
> 
> But I agree with Julien's suggestion, NR_MEM_BANKS and NR_NODE_MEMBLKS
> must be aware of each other. I had thought to add some ASSERT check,
> but I don't know how to do it better. So I post this patch for more
> suggestion.

OK. In that case I'd say to get rid of the previous definition of
NR_NODE_MEMBLKS as it is probably not necessary, see below.



> > 
> > > And keep default NR_NODE_MEMBLKS in common header
> > > for those architectures NUMA is disabled.
> > 
> > This last sentence is not accurate: on x86 NUMA is enabled and
> > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h (there is no
> > x86 definition of it)
> > 
> 
> Yes.
> 
> > 
> > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > ---
> > >  xen/include/asm-arm/numa.h | 8 +++++++-
> > >  xen/include/xen/numa.h     | 2 ++
> > >  2 files changed, 9 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> > > index 8f1c67e3eb..21569e634b 100644
> > > --- a/xen/include/asm-arm/numa.h
> > > +++ b/xen/include/asm-arm/numa.h
> > > @@ -3,9 +3,15 @@
> > >
> > >  #include <xen/mm.h>
> > >
> > > +#include <asm/setup.h>
> > > +
> > >  typedef u8 nodeid_t;
> > >
> > > -#ifndef CONFIG_NUMA
> > > +#ifdef CONFIG_NUMA
> > > +
> > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> > > +
> > > +#else
> > >
> > >  /* Fake one node for now. See also node_online_map. */
> > >  #define cpu_to_node(cpu) 0
> > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
> > > index 1978e2be1b..1731e1cc6b 100644
> > > --- a/xen/include/xen/numa.h
> > > +++ b/xen/include/xen/numa.h
> > > @@ -12,7 +12,9 @@
> > >  #define MAX_NUMNODES    1
> > >  #endif
> > >
> > > +#ifndef NR_NODE_MEMBLKS
> > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > > +#endif

This one we can remove it completely right?
--8323329-1750427606-1632713142=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 04:18:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 04:18:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196291.349061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUi6F-0003Uc-Gv; Mon, 27 Sep 2021 04:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196291.349061; Mon, 27 Sep 2021 04:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUi6F-0003UV-Dj; Mon, 27 Sep 2021 04:18:43 +0000
Received: by outflank-mailman (input) for mailman id 196291;
 Mon, 27 Sep 2021 04:18:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUi6D-0003UP-Ty
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 04:18:42 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.79]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fdba7454-1f49-11ec-bc3d-12813bfff9fa;
 Mon, 27 Sep 2021 04:18:39 +0000 (UTC)
Received: from DB6P191CA0021.EURP191.PROD.OUTLOOK.COM (2603:10a6:6:28::31) by
 AM6PR08MB4934.eurprd08.prod.outlook.com (2603:10a6:20b:ee::21) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13; Mon, 27 Sep 2021 04:18:37 +0000
Received: from DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:28:cafe::83) by DB6P191CA0021.outlook.office365.com
 (2603:10a6:6:28::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Mon, 27 Sep 2021 04:18:37 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT060.mail.protection.outlook.com (10.152.21.231) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 04:18:37 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Mon, 27 Sep 2021 04:18:37 +0000
Received: from 67f31e76db22.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B3BA62E1-41A9-4149-9F6E-66551AB95E8F.1; 
 Mon, 27 Sep 2021 04:18:26 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 67f31e76db22.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 04:18:26 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB8PR08MB4201.eurprd08.prod.outlook.com (2603:10a6:10:a3::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.17; Mon, 27 Sep
 2021 04:18:25 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 04:18:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdba7454-1f49-11ec-bc3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tquU0d1vXwH12rmXy34ZgK+kFXw1Cfp0uPLj9pvs/hY=;
 b=t543UH3rlE2JVfPcUeaIFoLSIPSkW6fIBhyF/I/hmgu60yfWbrUUfBCyHOaGnUmefT9njQbB6qKlbkDv0nR6ocMv2hpu4hreMkELuI+Buhl9ImrxEfCdi1vp2iGN8axvLE3VeMvZWe0R4FLYFAa80lN08fobIR8BmN5dIC04VnE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=C3gxvZio5YRVtXEJV7cMApmBP/hRPB4INy80LtQelKFCE1lohIrUSpVcvqqA0WZWzwwDCrpHmPSS9fJ/oB8nwdtXhMrTROXB1a8WXoJJNk+sumQXUQNmX1w/aY7vrroHi/9Ivm7oHFF69g5Wa/0rOiLxoOnNc4gFuQ3eLQFbvSOzl9zNdEHF1cVo5tS+e65+vbgJyrTgSwhkRu6shCiWhXHUji2t627NX1v+LJJFeZ00y0bcjERmbcU7O6lDnv0bAJfzhANGDVejTGRBwGwtVacSHo8emUvSJiUNWUgOYWTb5rfQ8peEoJmrj9w+BYzod4c0KMefCeeEa6ep8y9MXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tquU0d1vXwH12rmXy34ZgK+kFXw1Cfp0uPLj9pvs/hY=;
 b=PJyk0ryUOg/6VEmSTqrnKQHczty2o4i9LCz1PX6C0rC2VtpcW52Tv23dy4Oik7vezbTTprK8mY6Pftzv3mpFxcKF1G8TC1a683qsv4hk97g2PUnvHEsjq/gIKyHTTu0EEGxqOwwQbY7f4SRKfC7j2OwHEnDhEiMPDagm2MOzXUXcyASHxk56AVSbdIuDYwmBsu2tlRjYjdahrin8K98PFYwU3YSaWITZzhH5D6G2j0qj/oaxfx9tQtZi3Jo8g6rCO8WEQ8s86A8qf8E6xjuEF3zW7+0CjReRKbCbsPwlEtKR86ZbyiznOpwyClqe8/dCXA1UuzpfB+UUSO5n25HwGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tquU0d1vXwH12rmXy34ZgK+kFXw1Cfp0uPLj9pvs/hY=;
 b=t543UH3rlE2JVfPcUeaIFoLSIPSkW6fIBhyF/I/hmgu60yfWbrUUfBCyHOaGnUmefT9njQbB6qKlbkDv0nR6ocMv2hpu4hreMkELuI+Buhl9ImrxEfCdi1vp2iGN8axvLE3VeMvZWe0R4FLYFAa80lN08fobIR8BmN5dIC04VnE=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index: AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CCAAP08AIAADO8Q
Date: Mon, 27 Sep 2021 04:18:25 +0000
Message-ID:
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 7375F5EB05475F4D9A040A78B413529A.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 9c9a6d00-a506-423e-c7a2-08d9816de0f0
x-ms-traffictypediagnostic: DB8PR08MB4201:|AM6PR08MB4934:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4934365BED878AA0D24568419EA79@AM6PR08MB4934.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 tC3vusr0K/nwl8+5wEnJNIF5tuH8h6n5e9MDb3MqeY9NXPxG1At2xin0BqHOwhTisGeLaZoGbr23LnkF9Ah5BwLz87cXWwwy59L+uC5664YGMzh/iGfR0OqBwa+k/jKw5FlcTDH+IRmoFWzvrdj8XZzXsMc0WrR63TvQBQMhFL6qj+Nao072RSkd5XGN4yvkBxs8xm4pGq6zzva+c8cqbUet+gXfWUT2KAegu+IAlFgiL7Oh/iRIdU9sVqPFMwc6lBcDCBYucSoVhfS8WO3JpUVHcUZwNFu+eTtDqnTXAC5NHKUOQWeLxAW8UEcUQC4K89rTeaNFCQoR+eT1PGAG+ql1wTPsrNw6APOaG9PavU7xfKL+5lbnsgZ1BEk1c0xAMlFJqHR4bayHs7Ghs71IO/tFXp8CkbC23hu7ebmkam4bAKmZthToScgkGkKLzZAn1tBKl+ctgnGh7lgp84IB0xE70rZmV5nG7dCGMNWPMMPF3uWKeCxwP0UZaQbmqg4cJufmp96vvXS2KIn89Sh6ZKnlFClU8MZVRkwBeTT2aNgwbPl1HZEpb6Qk0TO91oM5bJFemnWZ0Cz1s5Bxp/L8wNjcuCXotPW9Eoncn9mW0qPWO7FpyNrYxHh5Du2Mmu7GfDKtccNFz+mcBdbgxNh4opzdNBKUcWaH0ZZN2wnHn5yM6WKqZAtP0ScqKd6y7PxdHiiUryTLVwjsQVfbcP1//2FseLvMW4ajsVlrXvXdB70vOboCTin9wtp2gHuS8xeb4JSlC42PqwSrOIuBsyN8pjyOJyJGTcWKebl0yVP8tW8=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(4326008)(966005)(83380400001)(33656002)(5660300002)(76116006)(71200400001)(6916009)(66446008)(64756008)(66556008)(66476007)(66946007)(38070700005)(316002)(6506007)(7696005)(26005)(8936002)(53546011)(2906002)(86362001)(186003)(9686003)(55016002)(52536014)(54906003)(38100700002)(122000001)(8676002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4201
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6c69f404-b6ec-43f3-77d8-08d9816dd9d2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kvZYEt+pTbu7MOHInPQnIAp58oMdThgiLgx2DTpp6Y0DwedgGeJnQ8Y7aYnDQPv/2zh9MpXqblUbSrTFbXu4plaK3p1GRSw1jyRpcME8zr+1G0E+cMvQT5MU0+YvFCpNUNmEjpIA5hFurZHL5yKTKTjHYy8zzXtFbGwpE3Gy8U1OJgqZjF4d1r2dKM04eIjM7aaDIobDd+D80TQ0DP1sxJ4FRC2DsLPudYHdmUN2927OZVlFsYzxdgtBijMgaQabhFwyP5BIgpRWYLfUZ/kQfjQ7P6ruxk0rlC1V0udstAtK99d+eSCDjpo8JKmAcDMZLz4iOyYvApKejpWmmLQ0IW73k2wOdDyu7Om3PzkPYNLQ+xq5TVNNOrG07zVwWzR0QL1u8c6OpA35YS0akmIfxdXGwctrXDPzIqBrM0fVcUfsdWczVn4av8dtsFENkfp2kbgAfP5bDlZGiKj544P0P+ONXTj5stBRZt77VvANQpzgmxMPI4g5gP5Hr/Kbrh27fO+MSa2+rajMoA5TxBweJgM5LnitCnE/V2II5capNqVl482fuQhzyFaZBwc2z01jRshMJlkaw0ArdALqf8j3yCJc9MNeXuKOPtRtohI7s+K8iGs5DY+RECsLRVnAvqHSPULE3n1RFdcZ1Q6QNRBJk2CnEDB0xDbZGAUUsJfaWYq1llpr8tiAbzd2R0gf3Iq0HZWX/PrA+C29I15UDYSOVFNh/oBsVWyJ3nNSfLmINdsw0J2raPxcu8GjNq+smjlOkJ6tdb/nPcSbH9RIBC0QPXLBSYJOfNQQrWKamuwefkc=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(316002)(186003)(26005)(86362001)(33656002)(7696005)(54906003)(336012)(9686003)(6862004)(82310400003)(70586007)(966005)(55016002)(53546011)(4326008)(70206006)(47076005)(8936002)(508600001)(356005)(6506007)(8676002)(81166007)(36860700001)(5660300002)(2906002)(83380400001)(52536014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 04:18:37.4850
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c9a6d00-a506-423e-c7a2-08d9816de0f0
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4934

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMjfml6UgMTE6MjYNCj4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiBDYzog
U3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgeGVuLQ0KPiBkZXZl
bEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzsganVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMN
Cj4gPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gU3ViamVjdDogUkU6IFtQQVRDSCAyMi8z
N10geGVuL2FybTogdXNlIE5SX01FTV9CQU5LUyB0byBvdmVycmlkZSBkZWZhdWx0DQo+IE5SX05P
REVfTUVNQkxLUw0KPiANCj4gT24gU3VuLCAyNiBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+
ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4gRnJvbTogU3RlZmFubyBTdGFi
ZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4gU2VudDogMjAyMeW5tDnmnIgy
NOaXpSA5OjM1DQo+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+ID4gPiBD
YzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3Jn
Ow0KPiBqdWxpZW5AeGVuLm9yZzsNCj4gPiA+IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1h
cnF1aXNAYXJtLmNvbT4NCj4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06
IHVzZSBOUl9NRU1fQkFOS1MgdG8gb3ZlcnJpZGUNCj4gZGVmYXVsdA0KPiA+ID4gTlJfTk9ERV9N
RU1CTEtTDQo+ID4gPg0KPiA+ID4gT24gVGh1LCAyMyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6
DQo+ID4gPiA+IEFzIGEgbWVtb3J5IHJhbmdlIGRlc2NyaWJlZCBpbiBkZXZpY2UgdHJlZSBjYW5u
b3QgYmUgc3BsaXQgYWNyb3NzDQo+ID4gPiA+IG11bHRpcGxlIG5vZGVzLiBTbyB3ZSBkZWZpbmUg
TlJfTk9ERV9NRU1CTEtTIGFzIE5SX01FTV9CQU5LUyBpbg0KPiA+ID4gPiBhcmNoIGhlYWRlci4N
Cj4gPiA+DQo+ID4gPiBUaGlzIHN0YXRlbWVudCBpcyB0cnVlIGJ1dCB3aGF0IGlzIHRoZSBnb2Fs
IG9mIHRoaXMgcGF0Y2g/IElzIGl0IHRvDQo+ID4gPiByZWR1Y2UgY29kZSBzaXplIGFuZCBtZW1v
cnkgY29uc3VtcHRpb24/DQo+ID4gPg0KPiA+DQo+ID4gTm8sIHdoZW4gSnVsaWVuIGFuZCBJIGRp
c2N1c3NlZCB0aGlzIGluIGxhc3QgdmVyc2lvblsxXSwgd2UgaGFkbid0DQo+IHRob3VnaHQNCj4g
PiBzbyBkZWVwbHkuIFdlIGp1c3QgdGhvdWdodCBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQgaW4g
RFQgY2Fubm90IGJlDQo+IHNwbGl0DQo+ID4gYWNyb3NzIG11bHRpcGxlIG5vZGVzLiBTbyBOUl9N
RU1fQkFOS1Mgc2hvdWxkIGJlIGVxdWFsIHRvIE5SX01FTV9CQU5LUy4NCj4gPg0KPiA+IGh0dHBz
Oi8vbGlzdHMueGVucHJvamVjdC5vcmcvYXJjaGl2ZXMvaHRtbC94ZW4tZGV2ZWwvMjAyMS0NCj4g
MDgvbXNnMDA5NzQuaHRtbA0KPiA+DQo+ID4gPiBJIGFtIGFza2luZyBiZWNhdXNlIE5SX01FTV9C
QU5LUyBpcyAxMjggYW5kDQo+ID4gPiBOUl9OT0RFX01FTUJMS1M9MipNQVhfTlVNTk9ERVMgd2hp
Y2ggaXMgNjQgYnkgZGVmYXVsdCBzbyBhZ2Fpbg0KPiA+ID4gTlJfTk9ERV9NRU1CTEtTIGlzIDEy
OCBiZWZvcmUgdGhpcyBwYXRjaC4NCj4gPiA+DQo+ID4gPiBJbiBvdGhlciB3b3JkcywgdGhpcyBw
YXRjaCBhbG9uZSBkb2Vzbid0IG1ha2UgYW55IGRpZmZlcmVuY2U7IGF0IGxlYXN0DQo+ID4gPiBk
b2Vzbid0IG1ha2UgYW55IGRpZmZlcmVuY2UgdW5sZXNzIENPTkZJR19OUl9OVU1BX05PREVTIGlz
IGluY3JlYXNlZC4NCj4gPiA+DQo+ID4gPiBTbywgaXMgdGhlIGdvYWwgdG8gcmVkdWNlIG1lbW9y
eSB1c2FnZSB3aGVuIENPTkZJR19OUl9OVU1BX05PREVTIGlzDQo+ID4gPiBoaWdoZXIgdGhhbiA2
ND8NCj4gPiA+DQo+ID4NCj4gPiBJIGFsc28gdGhvdWdodCBhYm91dCB0aGlzIHByb2JsZW0gd2hl
biBJIHdhcyB3cml0aW5nIHRoaXMgcGF0Y2guDQo+ID4gQ09ORklHX05SX05VTUFfTk9ERVMgaXMg
aW5jcmVhc2luZywgYnV0IE5SX01FTV9CQU5LUyBpcyBhIGZpeGVkDQo+ID4gdmFsdWUsIHRoZW4g
TlJfTUVNX0JBTktTIGNhbiBiZSBzbWFsbGVyIHRoYW4gQ09ORklHX05SX05VTUFfTk9ERVMNCj4g
PiBhdCBvbmUgcG9pbnQuDQo+ID4NCj4gPiBCdXQgSSBhZ3JlZSB3aXRoIEp1bGllbidzIHN1Z2dl
c3Rpb24sIE5SX01FTV9CQU5LUyBhbmQgTlJfTk9ERV9NRU1CTEtTDQo+ID4gbXVzdCBiZSBhd2Fy
ZSBvZiBlYWNoIG90aGVyLiBJIGhhZCB0aG91Z2h0IHRvIGFkZCBzb21lIEFTU0VSVCBjaGVjaywN
Cj4gPiBidXQgSSBkb24ndCBrbm93IGhvdyB0byBkbyBpdCBiZXR0ZXIuIFNvIEkgcG9zdCB0aGlz
IHBhdGNoIGZvciBtb3JlDQo+ID4gc3VnZ2VzdGlvbi4NCj4gDQo+IE9LLiBJbiB0aGF0IGNhc2Ug
SSdkIHNheSB0byBnZXQgcmlkIG9mIHRoZSBwcmV2aW91cyBkZWZpbml0aW9uIG9mDQo+IE5SX05P
REVfTUVNQkxLUyBhcyBpdCBpcyBwcm9iYWJseSBub3QgbmVjZXNzYXJ5LCBzZWUgYmVsb3cuDQo+
IA0KPiANCj4gDQo+ID4gPg0KPiA+ID4gPiBBbmQga2VlcCBkZWZhdWx0IE5SX05PREVfTUVNQkxL
UyBpbiBjb21tb24gaGVhZGVyDQo+ID4gPiA+IGZvciB0aG9zZSBhcmNoaXRlY3R1cmVzIE5VTUEg
aXMgZGlzYWJsZWQuDQo+ID4gPg0KPiA+ID4gVGhpcyBsYXN0IHNlbnRlbmNlIGlzIG5vdCBhY2N1
cmF0ZTogb24geDg2IE5VTUEgaXMgZW5hYmxlZCBhbmQNCj4gPiA+IE5SX05PREVfTUVNQkxLUyBp
cyBzdGlsbCBkZWZpbmVkIGluIHhlbi9pbmNsdWRlL3hlbi9udW1hLmggKHRoZXJlIGlzDQo+IG5v
DQo+ID4gPiB4ODYgZGVmaW5pdGlvbiBvZiBpdCkNCj4gPiA+DQo+ID4NCj4gPiBZZXMuDQo+ID4N
Cj4gPiA+DQo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29t
Pg0KPiA+ID4gPiAtLS0NCj4gPiA+ID4gIHhlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIHwgOCAr
KysrKysrLQ0KPiA+ID4gPiAgeGVuL2luY2x1ZGUveGVuL251bWEuaCAgICAgfCAyICsrDQo+ID4g
PiA+ICAyIGZpbGVzIGNoYW5nZWQsIDkgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KPiA+
ID4gPg0KPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmggYi94
ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+ID4gPiBpbmRleCA4ZjFjNjdlM2ViLi4yMTU2
OWU2MzRiIDEwMDY0NA0KPiA+ID4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0K
PiA+ID4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+ID4gPiBAQCAtMyw5
ICszLDE1IEBADQo+ID4gPiA+DQo+ID4gPiA+ICAjaW5jbHVkZSA8eGVuL21tLmg+DQo+ID4gPiA+
DQo+ID4gPiA+ICsjaW5jbHVkZSA8YXNtL3NldHVwLmg+DQo+ID4gPiA+ICsNCj4gPiA+ID4gIHR5
cGVkZWYgdTggbm9kZWlkX3Q7DQo+ID4gPiA+DQo+ID4gPiA+IC0jaWZuZGVmIENPTkZJR19OVU1B
DQo+ID4gPiA+ICsjaWZkZWYgQ09ORklHX05VTUENCj4gPiA+ID4gKw0KPiA+ID4gPiArI2RlZmlu
ZSBOUl9OT0RFX01FTUJMS1MgTlJfTUVNX0JBTktTDQo+ID4gPiA+ICsNCj4gPiA+ID4gKyNlbHNl
DQo+ID4gPiA+DQo+ID4gPiA+ICAvKiBGYWtlIG9uZSBub2RlIGZvciBub3cuIFNlZSBhbHNvIG5v
ZGVfb25saW5lX21hcC4gKi8NCj4gPiA+ID4gICNkZWZpbmUgY3B1X3RvX25vZGUoY3B1KSAwDQo+
ID4gPiA+IGRpZmYgLS1naXQgYS94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oIGIveGVuL2luY2x1ZGUv
eGVuL251bWEuaA0KPiA+ID4gPiBpbmRleCAxOTc4ZTJiZTFiLi4xNzMxZTFjYzZiIDEwMDY0NA0K
PiA+ID4gPiAtLS0gYS94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+ID4gPiA+ICsrKyBiL3hlbi9p
bmNsdWRlL3hlbi9udW1hLmgNCj4gPiA+ID4gQEAgLTEyLDcgKzEyLDkgQEANCj4gPiA+ID4gICNk
ZWZpbmUgTUFYX05VTU5PREVTICAgIDENCj4gPiA+ID4gICNlbmRpZg0KPiA+ID4gPg0KPiA+ID4g
PiArI2lmbmRlZiBOUl9OT0RFX01FTUJMS1MNCj4gPiA+ID4gICNkZWZpbmUgTlJfTk9ERV9NRU1C
TEtTIChNQVhfTlVNTk9ERVMqMikNCj4gPiA+ID4gKyNlbmRpZg0KPiANCj4gVGhpcyBvbmUgd2Ug
Y2FuIHJlbW92ZSBpdCBjb21wbGV0ZWx5IHJpZ2h0Pw0KDQpIb3cgYWJvdXQgZGVmaW5lIE5SX01F
TV9CQU5LUyB0bzoNCiNpZmRlZiBDT05GSUdfTlJfTlVNQV9OT0RFUw0KI2RlZmluZSBOUl9NRU1f
QkFOS1MgKENPTkZJR19OUl9OVU1BX05PREVTICogMikNCiNlbHNlDQojZGVmaW5lIE5SX01FTV9C
QU5LUyAxMjgNCiNlbmRpZg0KZm9yIGJvdGggeDg2IGFuZCBBcm0uIEZvciB0aG9zZSBhcmNoaXRl
Y3R1cmVzIGRvIG5vdCBzdXBwb3J0IG9yIGVuYWJsZQ0KTlVNQSwgdGhleSBjYW4gc3RpbGwgdXNl
ICJOUl9NRU1fQkFOS1MgMTI4Ii4gQW5kIHJlcGxhY2UgYWxsIE5SX05PREVfTUVNQkxLUw0KaW4g
TlVNQSBjb2RlIHRvIE5SX01FTV9CQU5LUyB0byByZW1vdmUgTlJfTk9ERV9NRU1CTEtTIGNvbXBs
ZXRlbHkuDQpJbiB0aGlzIGNhc2UsIE5SX01FTV9CQU5LUyBjYW4gYmUgYXdhcmUgb2YgdGhlIGNo
YW5nZXMgb2YgQ09ORklHX05SX05VTUFfTk9ERVMuDQoNCg0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 04:23:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 04:23:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196299.349073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUiAc-0004uH-9i; Mon, 27 Sep 2021 04:23:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196299.349073; Mon, 27 Sep 2021 04:23:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUiAc-0004uA-5J; Mon, 27 Sep 2021 04:23:14 +0000
Received: by outflank-mailman (input) for mailman id 196299;
 Mon, 27 Sep 2021 04:23:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUiAa-0004tx-Ut; Mon, 27 Sep 2021 04:23:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUiAa-0006lL-Dq; Mon, 27 Sep 2021 04:23:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUiAa-0008Ft-0y; Mon, 27 Sep 2021 04:23:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUiAa-0000xb-0M; Mon, 27 Sep 2021 04:23:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Lk0gXHvcABGMcf7r38nGDH9X+2orQ+HFtxP9qzvMKfQ=; b=p2PUIcZZla8N3nNJLa/vKc0JVO
	5aZVhiW0vx5ggfZZs60K2cTGQExVi/Lhq/4peoCVzkOgdQPGOvW4DF8GplG9zsWfKU6PadnizLG+W
	ox+8ZMIVMRGwexPw86bDTU01RPxQvf2rfvqYOYXSUHviCmRHc2WXuchpn65xTiPiReTA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165208-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165208: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a3b397b4fffb799d25658defafd962f0fb3e9fe0
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 04:23:12 +0000

flight 165208 linux-linus real [real]
flight 165213 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165208/
http://logs.test-lab.xenproject.org/osstest/logs/165213/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a3b397b4fffb799d25658defafd962f0fb3e9fe0
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  422 days
Failing since        152366  2020-08-01 20:49:34 Z  421 days  737 attempts
Testing same since   165199  2021-09-26 02:50:44 Z    1 days    2 attempts

------------------------------------------------------------
7339 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2270952 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 04:59:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 04:59:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196308.349086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUijv-0000mV-VA; Mon, 27 Sep 2021 04:59:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196308.349086; Mon, 27 Sep 2021 04:59:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUijv-0000mO-S8; Mon, 27 Sep 2021 04:59:43 +0000
Received: by outflank-mailman (input) for mailman id 196308;
 Mon, 27 Sep 2021 04:59:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUiju-0000mI-Ff
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 04:59:42 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b900a602-1f4f-11ec-bc3d-12813bfff9fa;
 Mon, 27 Sep 2021 04:59:41 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 667CE6113A;
 Mon, 27 Sep 2021 04:59:40 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b900a602-1f4f-11ec-bc3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632718780;
	bh=dmimcsQBlHFvXBdlXV0yfmz4bzwPaYAThfZ1hy3CUzk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UKCRTpwKD3Viim1hdu57c15cb/Shyq3nom9DpgfS4p9fp/UmlXn2ZItYoh+cytsQ2
	 jwzz6c27SAuimLY+WvuAq/PcSDL2QF+8jUmvh/2+1rklDI6BPq8nhSk2KYuUlLbcUZ
	 0bLXTysvGx0dg1rIq5FDkVCovE28nNP9hP6Od1WkS/d7XPYAa0HG4HVbW3DoAI7V91
	 +848HbZKwfi7PT8BtIntkoBBNzXxolFrk+EWHHwxnAyLMyjgcFIhyOhqMuZgzwOHeb
	 oyg+L9HbA9yAZOEvzZJlW8pCaq9SsapFAaKxBivkG+0D0c3sfdEbZ5XNmBvkl+ixjx
	 4mzbA1d1ACsHg==
Date: Sun, 26 Sep 2021 21:59:40 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, jbeulich@suse.com, 
    roger.pau@citrix.com, andrew.cooper3@citrix.com
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
In-Reply-To: <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com> <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s> <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1296240553-1632717348=:5022"
Content-ID: <alpine.DEB.2.21.2109262135590.5022@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1296240553-1632717348=:5022
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109262135591.5022@sstabellini-ThinkPad-T480s>

+x86 maintainers

On Mon, 27 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月27日 11:26
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>
> > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
> > NR_NODE_MEMBLKS
> > 
> > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > -----Original Message-----
> > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > Sent: 2021年9月24日 9:35
> > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > julien@xen.org;
> > > > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> > default
> > > > NR_NODE_MEMBLKS
> > > >
> > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > As a memory range described in device tree cannot be split across
> > > > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS in
> > > > > arch header.
> > > >
> > > > This statement is true but what is the goal of this patch? Is it to
> > > > reduce code size and memory consumption?
> > > >
> > >
> > > No, when Julien and I discussed this in last version[1], we hadn't
> > thought
> > > so deeply. We just thought a memory range described in DT cannot be
> > split
> > > across multiple nodes. So NR_MEM_BANKS should be equal to NR_MEM_BANKS.
> > >
> > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > 08/msg00974.html
> > >
> > > > I am asking because NR_MEM_BANKS is 128 and
> > > > NR_NODE_MEMBLKS=2*MAX_NUMNODES which is 64 by default so again
> > > > NR_NODE_MEMBLKS is 128 before this patch.
> > > >
> > > > In other words, this patch alone doesn't make any difference; at least
> > > > doesn't make any difference unless CONFIG_NR_NUMA_NODES is increased.
> > > >
> > > > So, is the goal to reduce memory usage when CONFIG_NR_NUMA_NODES is
> > > > higher than 64?
> > > >
> > >
> > > I also thought about this problem when I was writing this patch.
> > > CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
> > > value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NODES
> > > at one point.
> > >
> > > But I agree with Julien's suggestion, NR_MEM_BANKS and NR_NODE_MEMBLKS
> > > must be aware of each other. I had thought to add some ASSERT check,
> > > but I don't know how to do it better. So I post this patch for more
> > > suggestion.
> > 
> > OK. In that case I'd say to get rid of the previous definition of
> > NR_NODE_MEMBLKS as it is probably not necessary, see below.
> > 
> > 
> > 
> > > >
> > > > > And keep default NR_NODE_MEMBLKS in common header
> > > > > for those architectures NUMA is disabled.
> > > >
> > > > This last sentence is not accurate: on x86 NUMA is enabled and
> > > > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h (there is
> > no
> > > > x86 definition of it)
> > > >
> > >
> > > Yes.
> > >
> > > >
> > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > ---
> > > > >  xen/include/asm-arm/numa.h | 8 +++++++-
> > > > >  xen/include/xen/numa.h     | 2 ++
> > > > >  2 files changed, 9 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> > > > > index 8f1c67e3eb..21569e634b 100644
> > > > > --- a/xen/include/asm-arm/numa.h
> > > > > +++ b/xen/include/asm-arm/numa.h
> > > > > @@ -3,9 +3,15 @@
> > > > >
> > > > >  #include <xen/mm.h>
> > > > >
> > > > > +#include <asm/setup.h>
> > > > > +
> > > > >  typedef u8 nodeid_t;
> > > > >
> > > > > -#ifndef CONFIG_NUMA
> > > > > +#ifdef CONFIG_NUMA
> > > > > +
> > > > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> > > > > +
> > > > > +#else
> > > > >
> > > > >  /* Fake one node for now. See also node_online_map. */
> > > > >  #define cpu_to_node(cpu) 0
> > > > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
> > > > > index 1978e2be1b..1731e1cc6b 100644
> > > > > --- a/xen/include/xen/numa.h
> > > > > +++ b/xen/include/xen/numa.h
> > > > > @@ -12,7 +12,9 @@
> > > > >  #define MAX_NUMNODES    1
> > > > >  #endif
> > > > >
> > > > > +#ifndef NR_NODE_MEMBLKS
> > > > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > > > > +#endif
> > 
> > This one we can remove it completely right?
> 
> How about define NR_MEM_BANKS to:
> #ifdef CONFIG_NR_NUMA_NODES
> #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)
> #else
> #define NR_MEM_BANKS 128
> #endif
> for both x86 and Arm. For those architectures do not support or enable
> NUMA, they can still use "NR_MEM_BANKS 128". And replace all NR_NODE_MEMBLKS
> in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.
> In this case, NR_MEM_BANKS can be aware of the changes of CONFIG_NR_NUMA_NODES.

x86 doesn't have NR_MEM_BANKS as far as I can tell. I guess you also
meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?

But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES because
there can be many memory banks for each numa node, certainly more than
2. The existing definition on x86:

#define NR_NODE_MEMBLKS (MAX_NUMNODES*2)

Doesn't make a lot of sense to me. Was it just an arbitrary limit for
the lack of a better way to set a maximum?


On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be related.
In fact, what's the difference?

NR_MEM_BANKS is the max number of memory banks (with or without
numa-node-id).

NR_NODE_MEMBLKS is the max number of memory banks with NUMA support
(with numa-node-id)?

They are basically the same thing. On ARM I would just do:

#define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))


And maybe the definition could be common with x86 if we define
NR_MEM_BANKS to 128 on x86 too.
--8323329-1296240553-1632717348=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 05:05:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 05:05:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196314.349098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUipB-0002Tr-JD; Mon, 27 Sep 2021 05:05:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196314.349098; Mon, 27 Sep 2021 05:05:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUipB-0002Tk-Fs; Mon, 27 Sep 2021 05:05:09 +0000
Received: by outflank-mailman (input) for mailman id 196314;
 Mon, 27 Sep 2021 05:05:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUipA-0002Te-7L
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 05:05:08 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 43984053-ad26-46e7-9bfe-3f9bb271f388;
 Mon, 27 Sep 2021 05:05:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3B0B461186;
 Mon, 27 Sep 2021 05:05:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43984053-ad26-46e7-9bfe-3f9bb271f388
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632719106;
	bh=YxjL18L7J/nUxzaioAUHwuE1udmqAzgXu2YMU786K8s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=opB9hHPoTvK7Jbhk+UUNEu2GXKAgm8RgqAG5BYtib2DZDX7hbW1QbN1385RdOkoZj
	 0OEZxrR6PjCm5xqy/Zb9mRxKczHyU5yOt1SrMYRg3BH2u97Z9PSJzjxk5c4G8NY4p/
	 7MMLUAAyjTt+sKy3O67QRmMV98oPnZeeQ70n9mPUXFTosy/jNum4dHt/kn2HPOTD4V
	 q0++tMI6PLlqr9ke/hNjZYZBkEh3vhQEvBSesNdccsEMaUd5s9R3aYMq+QtlkaOkIG
	 izzYYLoyLK8GlBJj5VqKJ4x0Z7KxsgZR7rMbVlFoE8BmxkdwV9B77E8t0afg5AmNdc
	 qyxXnu9Lv/hHw==
Date: Sun, 26 Sep 2021 22:05:06 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Stefano Stabellini <sstabellini@kernel.org>
cc: Wei Chen <Wei.Chen@arm.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    "jbeulich@suse.com" <jbeulich@suse.com>, 
    "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s>
Message-ID: <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com> <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1993594964-1632719106=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1993594964-1632719106=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Sun, 26 Sep 2021, Stefano Stabellini wrote:
> On Sun, 26 Sep 2021, Wei Chen wrote:
> > > -----Original Message-----
> > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > Sent: 2021年9月25日 3:53
> > > To: Wei Chen <Wei.Chen@arm.com>
> > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > <Bertrand.Marquis@arm.com>; jbeulich@suse.com; andrew.cooper3@citrix.com;
> > > roger.pau@citrix.com; wl@xen.org
> > > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node
> > > memory range
> > > 
> > > On Fri, 24 Sep 2021, Wei Chen wrote:
> > > > > -----Original Message-----
> > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > Sent: 2021年9月24日 8:26
> > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > julien@xen.org;
> > > > > Bertrand Marquis <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > > > > andrew.cooper3@citrix.com; roger.pau@citrix.com; wl@xen.org
> > > > > Subject: Re: [PATCH 08/37] xen/x86: add detection of discontinous node
> > > > > memory range
> > > > >
> > > > > CC'ing x86 maintainers
> > > > >
> > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > One NUMA node may contain several memory blocks. In current Xen
> > > > > > code, Xen will maintain a node memory range for each node to cover
> > > > > > all its memory blocks. But here comes the problem, in the gap of
> > > > > > one node's two memory blocks, if there are some memory blocks don't
> > > > > > belong to this node (remote memory blocks). This node's memory range
> > > > > > will be expanded to cover these remote memory blocks.
> > > > > >
> > > > > > One node's memory range contains othe nodes' memory, this is
> > > obviously
> > > > > > not very reasonable. This means current NUMA code only can support
> > > > > > node has continous memory blocks. However, on a physical machine,
> > > the
> > > > > > addresses of multiple nodes can be interleaved.
> > > > > >
> > > > > > So in this patch, we add code to detect discontinous memory blocks
> > > > > > for one node. NUMA initializtion will be failed and error messages
> > > > > > will be printed when Xen detect such hardware configuration.
> > > > >
> > > > > At least on ARM, it is not just memory that can be interleaved, but
> > > also
> > > > > MMIO regions. For instance:
> > > > >
> > > > > node0 bank0 0-0x1000000
> > > > > MMIO 0x1000000-0x1002000
> > > > > Hole 0x1002000-0x2000000
> > > > > node0 bank1 0x2000000-0x3000000
> > > > >
> > > > > So I am not familiar with the SRAT format, but I think on ARM the
> > > check
> > > > > would look different: we would just look for multiple memory ranges
> > > > > under a device_type = "memory" node of a NUMA node in device tree.
> > > > >
> > > > >
> > > >
> > > > Should I need to include/refine above message to commit log?
> > > 
> > > Let me ask you a question first.
> > > 
> > > With the NUMA implementation of this patch series, can we deal with
> > > cases where each node has multiple memory banks, not interleaved?
> > 
> > Yes.
> > 
> > > An an example:
> > > 
> > > node0: 0x0        - 0x10000000
> > > MMIO : 0x10000000 - 0x20000000
> > > node0: 0x20000000 - 0x30000000
> > > MMIO : 0x30000000 - 0x50000000
> > > node1: 0x50000000 - 0x60000000
> > > MMIO : 0x60000000 - 0x80000000
> > > node2: 0x80000000 - 0x90000000
> > > 
> > > 
> > > I assume we can deal with this case simply by setting node0 memory to
> > > 0x0-0x30000000 even if there is actually something else, a device, that
> > > doesn't belong to node0 in between the two node0 banks?
> > 
> > While this configuration is rare in SoC design, but it is not impossible. 
> 
> Definitely, I have seen it before.
> 
> 
> > > Is it only other nodes' memory interleaved that cause issues? In other
> > > words, only the following is a problematic scenario?
> > > 
> > > node0: 0x0        - 0x10000000
> > > MMIO : 0x10000000 - 0x20000000
> > > node1: 0x20000000 - 0x30000000
> > > MMIO : 0x30000000 - 0x50000000
> > > node0: 0x50000000 - 0x60000000
> > > 
> > > Because node1 is in between the two ranges of node0?
> > > 
> > 
> > But only device_type="memory" can be added to allocation.
> > For mmio there are two cases:
> > 1. mmio doesn't have NUMA id property.
> > 2. mmio has NUMA id property, just like some PCIe controllers.
> >    But we don’t need to handle these kinds of MMIO devices
> >    in memory block parsing. Because we don't need to allocate
> >    memory from these mmio ranges. And for accessing, we need
> >    a NUMA-aware PCIe controller driver or a generic NUMA-aware
> >    MMIO accessing APIs.
> 
> Yes, I am not too worried about devices with a NUMA id property because
> they are less common and this series doesn't handle them at all, right?
> I imagine they would be treated like any other device without NUMA
> awareness.
> 
> I am thinking about the case where the memory of each NUMA node is made
> of multiple banks. I understand that this patch adds an explicit check
> for cases where these banks are interleaving, however there are many
> other cases where NUMA memory nodes are *not* interleaving but they are
> still made of multiple discontinuous banks, like in the two example
> above.
> 
> My question is whether this patch series in its current form can handle
> the two cases above correctly. If so, I am wondering how it works given
> that we only have a single "start" and "size" parameter per node.
> 
> On the other hand if this series cannot handle the two cases above, my
> question is whether it would fail explicitly or not. The new
> check is_node_memory_continuous doesn't seem to be able to catch them.


Looking at numa_update_node_memblks, it is clear that the code is meant
to increase the range of each numa node to cover even MMIO regions in
between memory banks. Also see the comment at the top of the file:

 * Assumes all memory regions belonging to a single proximity domain
 * are in one chunk. Holes between them will be included in the node.

So if there are multiple banks for each node, start and end are
stretched to cover the holes between them, and it works as long as
memory banks of different NUMA nodes don't interleave.

I would appreciate if you could add an in-code comment to explain this
on top of numa_update_node_memblk.

Have you had a chance to test this? If not it would be fantastic if you
could give it a quick test to make sure it works as intended: for
instance by creating multiple memory banks for each NUMA node by
splitting an real bank into two smaller banks with a hole in between in
device tree, just for the sake of testing.
--8323329-1993594964-1632719106=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 05:13:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 05:13:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196322.349109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUixa-0004Cw-Ir; Mon, 27 Sep 2021 05:13:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196322.349109; Mon, 27 Sep 2021 05:13:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUixa-0004Cp-FU; Mon, 27 Sep 2021 05:13:50 +0000
Received: by outflank-mailman (input) for mailman id 196322;
 Mon, 27 Sep 2021 05:13:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUixZ-0004Cj-Oa
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 05:13:49 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b2416232-1f51-11ec-bc3d-12813bfff9fa;
 Mon, 27 Sep 2021 05:13:48 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18R3wS3B002968;
 Mon, 27 Sep 2021 05:13:47 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2058.outbound.protection.outlook.com [104.47.12.58])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bat9013ep-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 05:13:46 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5939.eurprd03.prod.outlook.com (2603:10a6:208:15e::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 05:13:44 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 05:13:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2416232-1f51-11ec-bc3d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n5EiGtdaFQRcAnwiMxVokVCXYW7jOac1pWmuQvhh9jrRWXRMtTRxm7UXOsLxxfyJ85S2KGEYN1+5AGF+s2kQxDaNfWEttAr2YHzU/WtR0nkssLkpoVOTxiVafmA3/p+yCdU/gbNsACQP94oUGaNXgOTuY+G0gnbxab4fzFoNO0uCquKDh8waxD+Pga2DK5Tyf3lh9zUUIPR8r1shgmHQe3evHiA5BVtODfj5qptgOUpkS3xviVVgeizNIxMz8u4c9ArxC9GFi0dzMJru+4doxCUaF7+NLxqOFB+w9HPNdve+WPVnHmIkxUXNamCMNI9ERYZ9bnCr86zr7U/ItGgZrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0/znJzAAQQJ+c4hgRTIU8PEFLUy1MPlz0bgZAqDxVe4=;
 b=Yu2Xr+PiTjwk6skyW4OMk5pygP6ZBov+FiPsz5B3C0dxAXPJUoEBVtmxIUo+hwz2ZjnUHp6rmpHitifACL2mxB6bD+85lpT12/ynl+S4Z3p+jWoubsWGPwqwxvVVj5d8qf54eLulbx3nx7VonZ6cNP0oILRDN3BtGIVErhY+E2IuG5TN3Ukyx401yC2lBy42XEWMdUQ6ggMGT7GwcFwpfSEGfiTsi3Ewi5IPUxluAdWxkPq0GlC7puPNpLNnb8+Y4IrIDEro+tDVXKBhs9bjfQHztqQY/Gi7A83JznYPzT7+x5KL9fT5XsdEYX2FGR3AGfMRx+Z70ZDCibyWfZBBXA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0/znJzAAQQJ+c4hgRTIU8PEFLUy1MPlz0bgZAqDxVe4=;
 b=uQjvvzhNcSM0Vy+LEhCQBF9O3tl9gv9WqQslWXuk/ndQWhsRQMbQJqofWllDpogVae5MijTpdQSck0kXIhv6YhE3a758/VErQnoJ2/oAoxKFwN6L2FBFOPPWmq1yHNT4r4avkLrqbHL9f5F9vU0yLgcHVbZjgu+gtS7wvDfM9lCnXAAA5iOhnya2gC4HICANidNK6W8VltPIWwp27oRrKDH+OJrt/SdxtVC4eg/GuND+apeXZQb8tpZmVgHE3iGm8rlgcK/7AE1j8A0hTRFcBBC5xh54OiL5Zylx84t4n51n4YojE+JOsRE/FLYGdj43zbhlult/OUcxlt7joOqkQg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "jgross@suse.com" <jgross@suse.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        "boris.ostrovsky@oracle.com"
	<boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Anastasiia Lukianenko
	<Anastasiia_Lukianenko@epam.com>
Subject: Re: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Topic: [PATCH v3 2/2] xen-pciback: allow compiling on other archs than
 x86
Thread-Index: AQHXsGDrkewW5NG+ZkqXauJ2P2aIyauyCpkAgACjoYCAAGJfAIAAjV+AgAO+J4A=
Date: Mon, 27 Sep 2021 05:13:43 +0000
Message-ID: <f0c88ffe-bcb2-be78-876c-4cd0dd550498@epam.com>
References: <20210923095345.185489-1-andr2000@gmail.com>
 <20210923095345.185489-2-andr2000@gmail.com>
 <alpine.DEB.2.21.2109231252270.17979@sstabellini-ThinkPad-T480s>
 <f62a1e2c-4253-c998-c206-6bb0681a84fb@epam.com>
 <7310d23e-4193-3f4c-06da-606b30e73f24@epam.com>
 <alpine.DEB.2.21.2109241258190.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241258190.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6195a360-570d-4157-c45a-08d9817593db
x-ms-traffictypediagnostic: AM0PR03MB5939:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB5939FAD0FAD9E41451E8EE9AE7A79@AM0PR03MB5939.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 qE8N3wI+WTQqzaeBG2Ndg06vGNZ+zdOFU62Pl1KR7bLxI7zsf6LkkiifeNyhsQwVSv3VzCT25t/ReeZMNb03PGPBhdO7QZbelWGPwsYdO0S9eyJeYNWoT+Unw3XB3n4DnSNu7HuGhocmtcUlWKo8bapDqLVceS1dDWxMK9RgqPAJHQVXeJ8R/xjknlJiPcP7p1ym56AtaGKCBqt/zZRPxrwViwkV58HdgNY+YEWmcrRzUgEO7pQe0rt72MwyhtZd/m9O0u5FshnsHRvSi3aCbwokFF0SAQtvo8LbbhDABaFkUcWkN8Ln+G34DsMWEWoAtctHL9StasTGLlv3ynbElZCIdlB5IuZG3I9b69eSTf/Ep69BzJT9L/lxXSqn/glp4xka9CYrg8dWyWor9AZSzyg0gxJnGlZTD40BFj80ddl7bfOV4IS5Jna+WeO7bmvtZHhSSLAL1UfcwpUCGmnu/dOp/1o404OwRchSgdOiLY9+o7gOPLIX7zbBMYvxnKB3i/Rkv996z0tpDuVClfRwrqxvvr8wvQ87rpbUnsgDy1uHWcuhLAr1xZcPJb91YSu+6YWXPjhPNkRjDo60XU2fePjzw58rtxGMlBcszLaKU1C4DpLKWtIWW4yC/yu91PEaAu3+FRySwAT52/xSAFmpOPJ8WVuvR9Dl/y85I1cdWZ5677j78267VN5RDc1EyrBopU/C+3t+RQQxcyR+Mdxo7l8d9y/YOrIz/5RwbOlCv7rH+vYrl68Hr1XB7URu5JvT
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(6512007)(64756008)(66446008)(316002)(86362001)(76116006)(36756003)(91956017)(6486002)(66946007)(31696002)(6506007)(107886003)(83380400001)(66556008)(122000001)(66476007)(8676002)(4326008)(38070700005)(8936002)(38100700002)(31686004)(71200400001)(186003)(26005)(55236004)(53546011)(6916009)(2616005)(508600001)(2906002)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dEFxVmJEUXVud0dNZGdmaG45cW9xV0ZxdnowQ09BQTc5bytqNHVlMHpjenMz?=
 =?utf-8?B?VEVDWFVhSk9RWGVqKzNSV0dVL1JLRXBYV1hiQ1pqRWNpRHBZZXY4eThPa1BH?=
 =?utf-8?B?VVEvK1lrZDdndjBJZVZvKzJVMFowWXhMZWFTTjVjSkpZTDdkQnFndFgxNXRh?=
 =?utf-8?B?cGhqc3MzYlhaUW1ic2N6ZHVqYk04VEpRL20vNUc0ZkZaUWNCMXNncStLWWJ1?=
 =?utf-8?B?V280L3ExTVAxUTRxWitRSUdkVmovdkdoVzlIMTlLeWpsdWNKMStocHcxWmRz?=
 =?utf-8?B?OUpCZlhEVDVydXhzbWovSzRXWHduWVIrbG03WnJ1d3hwMlh1aGZlMC9nQ0gr?=
 =?utf-8?B?Y3dDSmNDV0dOU01SOWxNRmIrUmF6UUVXdTJjeHdUeFJFbDhRT1ZWUkVlV1Vs?=
 =?utf-8?B?MlVBLzE4UnNpQWxZbWVHcyt6WEFpdi9oeWlITEJMRWRndGU0dENpd29GRjZL?=
 =?utf-8?B?a2pFUlpUR3puZkFrdkJvZWdhLzRBYmFXUzQ3T1l5QnZrWjhkdUxTR3JHUTlF?=
 =?utf-8?B?VXlVd0FnN2tEUGhpSXB1dVprdUs1enEyOHAwZ2sycGI5UnRhazZ6VUdGTUla?=
 =?utf-8?B?SUZVci9WZkRRbkJ5QzU3Z1FHQVFnWUVLSCs2OXk1dWtucCtwMjJWL0NyQ2R5?=
 =?utf-8?B?YUliTy9mUTZBMmhVVU9BVHlGZGJ6NDQ0bjdjckliangwTzlpN2Zzb05Bd1hJ?=
 =?utf-8?B?WXNvcitkanpnZDUvMC9NeU5BZEpvditzNTZoRjEySEkxNDNueEkweW5BNUJG?=
 =?utf-8?B?eThBZzQ5VWQ4UU1SM0ZUZ2g5L2M3cUQxWlRRMTFYZ29kN3BZQ0FodUZ5VHpq?=
 =?utf-8?B?N01wZnpMcGQ1bERheW9CRlpLL3VsQWEvMEY2WUdSRXJWYUxNMEhkM1p2dW9n?=
 =?utf-8?B?QUpQdnh3d2N1MVRtZG5Oa3pOODhOZDBBcm5OUmtnNktON1hTTWFzLzJPR0Rw?=
 =?utf-8?B?MldnbjU0VW8wOE9UN201d0lXelp3TGlSMEFsaDQ0TTd5QU11UnRxeTBLdUpX?=
 =?utf-8?B?VHVsVDBZV045S2RIVFZhUU12elkrc2Q0Ukk2YmU1WnJTUjBKZGZHU3Q0S29s?=
 =?utf-8?B?T29JdHNlUmpKbGVSc3RpQnNJZ0NCaHBMRkFJOVUzWS9JQU5oa3ZQUHBDVUtP?=
 =?utf-8?B?blBBWjRuK2c2QnJTajQ2Q3ZSMGJWaGNFWUxzakJVREcxZXpNdU9FR255ZG1w?=
 =?utf-8?B?LzczbGpTZmJ6c2lNR0E0NjlOR2xVMk1GTzZKL3l0OUM2b1BhZjljcTdnQ2ht?=
 =?utf-8?B?OGgvQXVRWmlLZ01JVXNDd0pUb0Q0bE5vWVJySHZLK2pNU1RoSlBhUFM0d1Ev?=
 =?utf-8?B?b3g2UjQ2R05JMm5iRXFaakxzVUk5VVM1WlprK2lkV3BCREM4ZDdBYVN5OGlM?=
 =?utf-8?B?N2dhSkN5UG9vcU9QSldwbFRrcWlncmtENGRYYWFvUjlxcGNnQS81aG9MNWll?=
 =?utf-8?B?dytOMERJb2VKYi82MzVzb3IvQ08yZ1FEcE1oaEdvZWgxUjBlVGFteEx5cjlq?=
 =?utf-8?B?TVNLRnRuZ0FwUW9oY0hSYmZSeUlORXNPZzFBcElrU2tNbnNvblQzUi8yb3d1?=
 =?utf-8?B?aXo1aFozNzN4SW9tcEF6ODhXVU1BTldQNWMzOE83Rm12aExhMERtMjNBR3FE?=
 =?utf-8?B?ZHg0cjJyakN0WU9ZN3hkYkt5cG1hS2hPalVyY0dRZUhrVlZWMXVsWTV3OU9h?=
 =?utf-8?B?WjU5S2xlM2xNNEFzeW5XQzJoZVB1eStHWHlzNmxBdFoyWENOODZveW1ZRTdm?=
 =?utf-8?B?NjBmQXlXcjZPdVBnMDdrRSt0dUhtMDE5dmNVaGw0RnkwdDJ2Y0RkUDgvL09I?=
 =?utf-8?B?UXQxem55TzVsaG9Fb0Fhdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <604D9059A0AF1B4AB0418BE6B8D5C0A3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6195a360-570d-4157-c45a-08d9817593db
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 05:13:43.9892
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: LsQrzfuHTsJw13emF8YbKq2lEWYTRfMYcAWMUKAUpwqgz0v469/BaOZjvGgOzIgMjB4cIEz2ASS0SihlRFe2WYHSXKHqy26IrEFDCzzUN0taa/9qddq9IikPwIHe6hj9
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5939
X-Proofpoint-GUID: C9ZA00CNSrL44cOe5qJIUZjOehJhk5jk
X-Proofpoint-ORIG-GUID: C9ZA00CNSrL44cOe5qJIUZjOehJhk5jk
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_01,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0
 spamscore=0 adultscore=0 bulkscore=0 suspectscore=0 clxscore=1015
 priorityscore=1501 impostorscore=0 mlxlogscore=999 malwarescore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270034

DQpPbiAyNC4wOS4yMSAyMzowNCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBGcmks
IDI0IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IE9uIDI0LjA5
LjIxIDA4OjQ2LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+PiBPbiAyMy4wOS4y
MSAyMzowMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPj4+PiBPbiBUaHUsIDIzIFNlcCAy
MDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4+IEZyb206IE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+
DQo+Pj4+PiBYZW4tcGNpYmFjayBkcml2ZXIgd2FzIGRlc2lnbmVkIHRvIGJlIGJ1aWx0IGZvciB4
ODYgb25seS4gQnV0IGl0DQo+Pj4+PiBjYW4gYWxzbyBiZSB1c2VkIGJ5IG90aGVyIGFyY2hpdGVj
dHVyZXMsIGUuZy4gQXJtLg0KPj4+Pj4gUmUtc3RydWN0dXJlIHRoZSBkcml2ZXIgaW4gYSB3YXkg
dGhhdCBpdCBjYW4gYmUgYnVpbHQgZm9yIG90aGVyDQo+Pj4+PiBwbGF0Zm9ybXMgYXMgd2VsbC4N
Cj4+Pj4+DQo+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xl
a3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBBbmFz
dGFzaWlhIEx1a2lhbmVua28gPGFuYXN0YXNpaWFfbHVraWFuZW5rb0BlcGFtLmNvbT4NCj4+Pj4g
VGhlIHBhdGNoIGxvb2tzIGdvb2QgdG8gbWUuIE9ubHkgb25lIHRoaW5nOiBvbiBBUk0zMiBJIGdl
dDoNCj4+PiBXRSBkbyBub3QgeWV0IHN1cHBvcnQgWGVuIFBDSSBwYXNzdGhyb3VnaCBmb3IgQVJN
MzINCj4gS2VlcCBpbiBtaW5kIHRoYXQgaXQgaXMgcG9zc2libGUgdG8gcnVuIEFSTTMyIGd1ZXN0
cyBvbiBhbiBBUk02NA0KPiBoeXBlcnZpc29yLg0KPg0KPg0KPj4+PiBkcml2ZXJzL3hlbi94ZW4t
cGNpYmFjay9jb25mX3NwYWNlX2hlYWRlci5jOiBJbiBmdW5jdGlvbiDigJhiYXJfaW5pdOKAmToN
Cj4+Pj4gZHJpdmVycy94ZW4veGVuLXBjaWJhY2svY29uZl9zcGFjZV9oZWFkZXIuYzoyMzk6MzQ6
IHdhcm5pbmc6IHJpZ2h0IHNoaWZ0IGNvdW50ID49IHdpZHRoIG9mIHR5cGUgWy1Xc2hpZnQtY291
bnQtb3ZlcmZsb3ddDQo+Pj4+ICAgICAgICBiYXItPnZhbCA9IHJlc1twb3MgLSAxXS5zdGFydCA+
PiAzMjsNCj4+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+DQo+Pj4+
IGRyaXZlcnMveGVuL3hlbi1wY2liYWNrL2NvbmZfc3BhY2VfaGVhZGVyLmM6MjQwOjQ5OiB3YXJu
aW5nOiByaWdodCBzaGlmdCBjb3VudCA+PSB3aWR0aCBvZiB0eXBlIFstV3NoaWZ0LWNvdW50LW92
ZXJmbG93XQ0KPj4+PiAgICAgICAgYmFyLT5sZW5fdmFsID0gLXJlc291cmNlX3NpemUoJnJlc1tw
b3MgLSAxXSkgPj4gMzI7DQo+Pj4+ICAgICANCj4+Pj4gICAgIA0KPj4+PiByZXNvdXJjZV9zaXpl
X3QgaXMgZGVmaW5lZCBhcyBwaHlzX2FkZHJfdCBhbmQgaXQgY2FuIGJlIDMyYml0IG9uIGFybTMy
Lg0KPj4+Pg0KPj4+Pg0KPj4+PiBPbmUgZml4IGlzIHRvIHN1cnJvdW5kOg0KPj4+Pg0KPj4+PiAJ
CWlmIChwb3MgJiYgKHJlc1twb3MgLSAxXS5mbGFncyAmIElPUkVTT1VSQ0VfTUVNXzY0KSkgew0K
Pj4+PiAJCQliYXItPnZhbCA9IHJlc1twb3MgLSAxXS5zdGFydCA+PiAzMjsNCj4+Pj4gCQkJYmFy
LT5sZW5fdmFsID0gLXJlc291cmNlX3NpemUoJnJlc1twb3MgLSAxXSkgPj4gMzI7DQo+Pj4+IAkJ
CXJldHVybiBiYXI7DQo+Pj4+IAkJfQ0KPj4+Pg0KPj4+PiB3aXRoICNpZmRlZiBQSFlTX0FERFJf
VF82NEJJVA0KPj4+Pg0KPj4+IFRoaXMgbWlnaHQgbm90IGJlIGNvcnJlY3QuIFdlIGFyZSBkZWFs
aW5nIGhlcmUgd2l0aCBhIDY0LWJpdCBCQVIgb24gYSAzMi1iaXQgT1MuDQo+Pj4NCj4+PiBJIHRo
aW5rIHRoYXQgdGhpcyBjYW4gc3RpbGwgYmUgdmFsaWQgdXNlLWNhc2UgaWYgQkFSNjQuaGkgPT0g
MC4gU28sIG5vdCBzdXJlDQo+Pj4NCj4+PiB3ZSBjYW4ganVzdCBza2lwIGl0IHdpdGggaWZkZWYu
DQo+Pj4NCj4+PiBJbnN0ZWFkLCB0byBiZSBvbiB0aGUgc2FmZSBzaWRlLCB3ZSBjYW4gaGF2ZToN
Cj4+Pg0KPj4+IGNvbmZpZyBYRU5fUENJREVWX1NUVUINCj4+PiAgICDCoMKgwqDCoMKgwqAgdHJp
c3RhdGUgIlhlbiBQQ0ktZGV2aWNlIHN0dWIgZHJpdmVyIg0KPj4+ICAgIMKgwqDCoMKgwqDCoCBk
ZXBlbmRzIG9uIFBDSSAmJiBBUk02NCAmJiBYRU4NCj4+PiBlLmcuIG9ubHkgYWxsb3cgYnVpbGRp
bmcgdGhlICJzdHViIiBmb3IgQVJNNjQgZm9yIG5vdy4NCj4gVGhpcyBpcyBhIHByZXR0eSBkcmFz
dGljIHNvbHV0aW9uLiBJIHdvdWxkIGJlIE9LIHdpdGggaXQgYnV0IEkgcHJlZmVyDQo+IHRoZSBz
b2x1dGlvbiBiZWxvdyA+PiAxNiA+PiAxNi4NCj4NCj4NCj4+IE9yLi4uIHRoZXJlIGFyZSBjb3Vw
bGUgb2YgcGxhY2VzIGluIHRoZSBrZXJuZWwgd2hlcmUgUENJIGRlYWxzIHdpdGggdGhlIDMyIGJp
dCBzaGlmdCBhczoNCj4+DQo+PiBkcml2ZXJzL3BjaS9zZXR1cC1yZXMuYzoxMDg6wqDCoMKgIMKg
wqDCoCBuZXcgPSByZWdpb24uc3RhcnQgPj4gMTYgPj4gMTY7DQo+PiBkcml2ZXJzL3BjaS9pb3Yu
Yzo5NDk6wqDCoMKgIMKgwqDCoCBuZXcgPSByZWdpb24uc3RhcnQgPj4gMTYgPj4gMTY7DQo+Pg0K
Pj4gY29tbWl0IGNmN2JlZTVhMGJmMjcwYTRlYWNlMGJlMzkzMjlkNmFjMDEzNmNjNDcNCj4+IERh
dGU6wqDCoCBTdW4gQXVnIDcgMTM6NDk6NTkgKjIwMDUqICswNDAwDQo+Pg0KPj4gW3NuaXBdDQo+
Pg0KPj4gICDCoMKgwqAgQWxzbyBtYWtlIHN1cmUgdG8gd3JpdGUgaGlnaCBiaXRzIC0gdXNlICJ4
ID4+IDE2ID4+IDE2IiAocmF0aGVyIHRoYW4gdGhlDQo+PiAgIMKgwqDCoCBzaW1wbGVyICI+PiAz
MiIpIHRvIGF2b2lkIHdhcm5pbmdzIG9uIDMyLWJpdCBhcmNoaXRlY3R1cmVzIHdoZXJlIHdlJ3Jl
DQo+PiAgIMKgwqDCoCBub3QgZ29pbmcgdG8gaGF2ZSBhbnkgaGlnaCBiaXRzLg0KPiBJIHRoaW5r
IHRoaXMgaXMgdGhlIGJlc3Qgb3B0aW9uDQoNCk9rLCBzbyBmb3IgYm90aCBwYXRjaGVzOg0KDQox
LiBJJ2xsIG1vdmUgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiBpbnRvIHRoZSBzZWNvbmQgcGF0Y2gN
Cg0KMi4gSSdsbCB1c2UgPj4gMTYgPj4gMTYgdG8gZml4IDMyLWJpdCBhcmNoaXRlY3R1cmVzDQoN
Cg0KPg0KPg0KPj4gVGhpcyBtaWdodCBub3QgYmUoPykgaW1tZWRpYXRlbHkgY29ycmVjdCBpbiBj
YXNlIG9mIExQQUUgdGhvdWdoLCBlLmcuDQo+Pg0KPj4gNjQtYml0IEJBUiBtYXkgdG9sZXJhdGUg
NDAtYml0IGFkZHJlc3MgaW4gc29tZSB1c2UtY2FzZXM/DQo+IEl0IGlzIGNvcnJlY3QgZm9yIExQ
QUUgdG9vLCBpdCBpcyBqdXN0IHRoYXQgd2l0aCBMUEFFIGl0IHdvdWxkIGJlDQo+IHVubmVjZXNz
YXJ5Lg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 06:26:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 06:26:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196330.349120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUk5Y-00042p-WA; Mon, 27 Sep 2021 06:26:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196330.349120; Mon, 27 Sep 2021 06:26:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUk5Y-00042i-RR; Mon, 27 Sep 2021 06:26:08 +0000
Received: by outflank-mailman (input) for mailman id 196330;
 Mon, 27 Sep 2021 06:26:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Pp17=OR=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mUk5X-00042c-Gd
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 06:26:07 +0000
Received: from mail-ed1-x533.google.com (unknown [2a00:1450:4864:20::533])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d8ec3c02-ddf9-4bee-a139-4a13784933f3;
 Mon, 27 Sep 2021 06:26:04 +0000 (UTC)
Received: by mail-ed1-x533.google.com with SMTP id l8so17070407edw.2
 for <xen-devel@lists.xenproject.org>; Sun, 26 Sep 2021 23:26:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8ec3c02-ddf9-4bee-a139-4a13784933f3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=8UBUw9sPwa0hR2+hB7kSZJHo+WETZYtPt/5Hy4WHarQ=;
        b=ptPjq7BnviIrtwj0bdVa5z5CuE4ik5C9udM9UTUqGnrWz5RuNxktXGT4e4ZBjNsTQ2
         iJl6BHhqunFwQRi8xgChCrO87JkbG4WetDOI8qG4aaBgBiCs22BdY6C8ng+qKUzE2BD1
         XQXrn5uFaaY0mVpIvrnwBXIRMs4zq6F41ybo/0fhnEI+8VuYkgwFSaM5e+Wx3oHGSISP
         Wnptu/KVvkg130K4vJWazxW2BCx3YNyK7iTHSLiiHNok4BWUNgPV5m7+2bmbClYuOudr
         46y+Wm+CeSCakwNA0gYgQ0qHnj7OQ5symky/eAm74R6JP/zmQ/r/7+qnL9hv+JuHA32Q
         9CGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=8UBUw9sPwa0hR2+hB7kSZJHo+WETZYtPt/5Hy4WHarQ=;
        b=RwyyOORu5yG4Khxh8fw67nC4/XA5wJ9NmqxylhY7SBIxLik2iUhzhVzLkRK5ljehL2
         MeJjopZRmO9uRfMee2GUsvZStHSNCe+FLCic9QTWNy/OMaRr32kZAo403HueD2cjBYJS
         geUYerj1txhKS/oA+Kigyc0koNasvayLfKH4fZD6Ao1RVwjCZeLRe4/BDXWCvx+fRrom
         ZuXdUOPg/jmLzseJJj6Z+K1m2o47/z1Ae3uFWMr2liaMto7J+IUoGtBKJ1vdDEfeL/v/
         KeMFZjwQS84v8+X5+47kTWIQszdvkOfCulVbAZRT7A9oem/cVtq6caPjKq8C8iM/tjvZ
         oVBw==
X-Gm-Message-State: AOAM531wDo7qKL0kwGpB2vbMMaEnIA1Bjc+G0P6rKZP+Mz5hyJsXr36g
	t6/ysEX55SUEn1u1nraKjty1+bQ3HVsYV8WLxTY2+bJ6E1w=
X-Google-Smtp-Source: ABdhPJyYOBMxkQcYIEgH8JksXbWHr8E7wZWMhtRI+6/Kg3gzGShvg6E7oxv0hInMBnAxoeeXLHpf77Tn61sUMBhHypY=
X-Received: by 2002:a17:907:2639:: with SMTP id aq25mr26193232ejc.138.1632723963481;
 Sun, 26 Sep 2021 23:26:03 -0700 (PDT)
MIME-Version: 1.0
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Mon, 27 Sep 2021 10:25:52 +0400
Message-ID: <CAJ=z9a1iWE=Ezwyv5jJH_jFhwLpahbVgD10j+jb_q_UPzGyKug@mail.gmail.com>
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Wei Chen <Wei.Chen@arm.com>, xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000dc13f905ccf42ba1"

--000000000000dc13f905ccf42ba1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 27 Sep 2021, 06:59 Stefano Stabellini, <sstabellini@kernel.org>
wrote:

> +x86 maintainers
>
> On Mon, 27 Sep 2021, Wei Chen wrote:
> > > -----Original Message-----
> > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26
> > > To: Wei Chen <Wei.Chen@arm.com>
> > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > <Bertrand.Marquis@arm.com>
> > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> default
> > > NR_NODE_MEMBLKS
> > >
> > > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > > -----Original Message-----
> > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35
> > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > julien@xen.org;
> > > > > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> > > default
> > > > > NR_NODE_MEMBLKS
> > > > >
> > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > As a memory range described in device tree cannot be split acro=
ss
> > > > > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS in
> > > > > > arch header.
> > > > >
> > > > > This statement is true but what is the goal of this patch? Is it =
to
> > > > > reduce code size and memory consumption?
> > > > >
> > > >
> > > > No, when Julien and I discussed this in last version[1], we hadn't
> > > thought
> > > > so deeply. We just thought a memory range described in DT cannot be
> > > split
> > > > across multiple nodes. So NR_MEM_BANKS should be equal to
> NR_MEM_BANKS.
> > > >
> > > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > > 08/msg00974.html
> > > >
> > > > > I am asking because NR_MEM_BANKS is 128 and
> > > > > NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 by default so again
> > > > > NR_NODE_MEMBLKS is 128 before this patch.
> > > > >
> > > > > In other words, this patch alone doesn't make any difference; at
> least
> > > > > doesn't make any difference unless CONFIG_NR_NUMA_NODES is
> increased.
> > > > >
> > > > > So, is the goal to reduce memory usage when CONFIG_NR_NUMA_NODES =
is
> > > > > higher than 64?
> > > > >
> > > >
> > > > I also thought about this problem when I was writing this patch.
> > > > CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
> > > > value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NODES
> > > > at one point.
> > > >
> > > > But I agree with Julien's suggestion, NR_MEM_BANKS and
> NR_NODE_MEMBLKS
> > > > must be aware of each other. I had thought to add some ASSERT check=
,
> > > > but I don't know how to do it better. So I post this patch for more
> > > > suggestion.
> > >
> > > OK. In that case I'd say to get rid of the previous definition of
> > > NR_NODE_MEMBLKS as it is probably not necessary, see below.
> > >
> > >
> > >
> > > > >
> > > > > > And keep default NR_NODE_MEMBLKS in common header
> > > > > > for those architectures NUMA is disabled.
> > > > >
> > > > > This last sentence is not accurate: on x86 NUMA is enabled and
> > > > > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h (there
> is
> > > no
> > > > > x86 definition of it)
> > > > >
> > > >
> > > > Yes.
> > > >
> > > > >
> > > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > > ---
> > > > > >  xen/include/asm-arm/numa.h | 8 +++++++-
> > > > > >  xen/include/xen/numa.h     | 2 ++
> > > > > >  2 files changed, 9 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/xen/include/asm-arm/numa.h
> b/xen/include/asm-arm/numa.h
> > > > > > index 8f1c67e3eb..21569e634b 100644
> > > > > > --- a/xen/include/asm-arm/numa.h
> > > > > > +++ b/xen/include/asm-arm/numa.h
> > > > > > @@ -3,9 +3,15 @@
> > > > > >
> > > > > >  #include <xen/mm.h>
> > > > > >
> > > > > > +#include <asm/setup.h>
> > > > > > +
> > > > > >  typedef u8 nodeid_t;
> > > > > >
> > > > > > -#ifndef CONFIG_NUMA
> > > > > > +#ifdef CONFIG_NUMA
> > > > > > +
> > > > > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> > > > > > +
> > > > > > +#else
> > > > > >
> > > > > >  /* Fake one node for now. See also node_online_map. */
> > > > > >  #define cpu_to_node(cpu) 0
> > > > > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
> > > > > > index 1978e2be1b..1731e1cc6b 100644
> > > > > > --- a/xen/include/xen/numa.h
> > > > > > +++ b/xen/include/xen/numa.h
> > > > > > @@ -12,7 +12,9 @@
> > > > > >  #define MAX_NUMNODES    1
> > > > > >  #endif
> > > > > >
> > > > > > +#ifndef NR_NODE_MEMBLKS
> > > > > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > > > > > +#endif
> > >
> > > This one we can remove it completely right?
> >
> > How about define NR_MEM_BANKS to:
> > #ifdef CONFIG_NR_NUMA_NODES
> > #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)
> > #else
> > #define NR_MEM_BANKS 128
> > #endif
> > for both x86 and Arm. For those architectures do not support or enable
> > NUMA, they can still use "NR_MEM_BANKS 128". And replace all
> NR_NODE_MEMBLKS
> > in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.
> > In this case, NR_MEM_BANKS can be aware of the changes of
> CONFIG_NR_NUMA_NODES.
>
> x86 doesn't have NR_MEM_BANKS as far as I can tell. I guess you also
> meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?
>
> But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES because
> there can be many memory banks for each numa node, certainly more than
> 2. The existing definition on x86:
>
> #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
>
> Doesn't make a lot of sense to me. Was it just an arbitrary limit for
> the lack of a better way to set a maximum?
>
>
> On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be related.
> In fact, what's the difference?
>
> NR_MEM_BANKS is the max number of memory banks (with or without
> numa-node-id).
>
> NR_NODE_MEMBLKS is the max number of memory banks with NUMA support
> (with numa-node-id)?
>
> They are basically the same thing. On ARM I would just do:
>
> #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))
>

As you wrote above, the second part of the MAX is totally arbitrary. In
fact, it is very likely than if you have more than 64 nodes, you may need a
lot more than 2 regions per node.

So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to
NR_MEM_BANKS so it can be used by common code.

--000000000000dc13f905ccf42ba1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Mon, 27 Sep 2021, 06:59 Stefano Stabellini, &lt;<a =
href=3D"mailto:sstabellini@kernel.org">sstabellini@kernel.org</a>&gt; wrote=
:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor=
der-left:1px #ccc solid;padding-left:1ex">+x86 maintainers<br>
<br>
On Mon, 27 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; -----Original Message-----<br>
&gt; &gt; From: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel=
.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;<b=
r>
&gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26<br>
&gt; &gt; To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@arm.com" target=3D"_b=
lank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; Cc: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.o=
rg" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;; xe=
n-<br>
&gt; &gt; <a href=3D"mailto:devel@lists.xenproject.org" target=3D"_blank" r=
el=3D"noreferrer">devel@lists.xenproject.org</a>; <a href=3D"mailto:julien@=
xen.org" target=3D"_blank" rel=3D"noreferrer">julien@xen.org</a>; Bertrand =
Marquis<br>
&gt; &gt; &lt;<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"_blank"=
 rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;<br>
&gt; &gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override =
default<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; <br>
&gt; &gt; On Sun, 26 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; -----Original Message-----<br>
&gt; &gt; &gt; &gt; From: Stefano Stabellini &lt;<a href=3D"mailto:sstabell=
ini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org=
</a>&gt;<br>
&gt; &gt; &gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35<br>
&gt; &gt; &gt; &gt; To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@arm.com" ta=
rget=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; Cc: <a href=3D"mailto:xen-devel@lists.xenproject.org" t=
arget=3D"_blank" rel=3D"noreferrer">xen-devel@lists.xenproject.org</a>; <a =
href=3D"mailto:sstabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer"=
>sstabellini@kernel.org</a>;<br>
&gt; &gt; <a href=3D"mailto:julien@xen.org" target=3D"_blank" rel=3D"norefe=
rrer">julien@xen.org</a>;<br>
&gt; &gt; &gt; &gt; Bertrand Marquis &lt;<a href=3D"mailto:Bertrand.Marquis=
@arm.com" target=3D"_blank" rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>=
&gt;<br>
&gt; &gt; &gt; &gt; Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to=
 override<br>
&gt; &gt; default<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; On Thu, 23 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; &gt; As a memory range described in device tree cannot =
be split across<br>
&gt; &gt; &gt; &gt; &gt; multiple nodes. So we define NR_NODE_MEMBLKS as NR=
_MEM_BANKS in<br>
&gt; &gt; &gt; &gt; &gt; arch header.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; This statement is true but what is the goal of this pat=
ch? Is it to<br>
&gt; &gt; &gt; &gt; reduce code size and memory consumption?<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; No, when Julien and I discussed this in last version[1], we =
hadn&#39;t<br>
&gt; &gt; thought<br>
&gt; &gt; &gt; so deeply. We just thought a memory range described in DT ca=
nnot be<br>
&gt; &gt; split<br>
&gt; &gt; &gt; across multiple nodes. So NR_MEM_BANKS should be equal to NR=
_MEM_BANKS.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-de=
vel/2021-" rel=3D"noreferrer noreferrer" target=3D"_blank">https://lists.xe=
nproject.org/archives/html/xen-devel/2021-</a><br>
&gt; &gt; 08/msg00974.html<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; I am asking because NR_MEM_BANKS is 128 and<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 by default=
 so again<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is 128 before this patch.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; In other words, this patch alone doesn&#39;t make any d=
ifference; at least<br>
&gt; &gt; &gt; &gt; doesn&#39;t make any difference unless CONFIG_NR_NUMA_N=
ODES is increased.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; So, is the goal to reduce memory usage when CONFIG_NR_N=
UMA_NODES is<br>
&gt; &gt; &gt; &gt; higher than 64?<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; I also thought about this problem when I was writing this pa=
tch.<br>
&gt; &gt; &gt; CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fi=
xed<br>
&gt; &gt; &gt; value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_=
NODES<br>
&gt; &gt; &gt; at one point.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; But I agree with Julien&#39;s suggestion, NR_MEM_BANKS and N=
R_NODE_MEMBLKS<br>
&gt; &gt; &gt; must be aware of each other. I had thought to add some ASSER=
T check,<br>
&gt; &gt; &gt; but I don&#39;t know how to do it better. So I post this pat=
ch for more<br>
&gt; &gt; &gt; suggestion.<br>
&gt; &gt; <br>
&gt; &gt; OK. In that case I&#39;d say to get rid of the previous definitio=
n of<br>
&gt; &gt; NR_NODE_MEMBLKS as it is probably not necessary, see below.<br>
&gt; &gt; <br>
&gt; &gt; <br>
&gt; &gt; <br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; And keep default NR_NODE_MEMBLKS in common header<=
br>
&gt; &gt; &gt; &gt; &gt; for those architectures NUMA is disabled.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; This last sentence is not accurate: on x86 NUMA is enab=
led and<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is still defined in xen/include/xen/num=
a.h (there is<br>
&gt; &gt; no<br>
&gt; &gt; &gt; &gt; x86 definition of it)<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Yes.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; Signed-off-by: Wei Chen &lt;<a href=3D"mailto:wei.=
chen@arm.com" target=3D"_blank" rel=3D"noreferrer">wei.chen@arm.com</a>&gt;=
<br>
&gt; &gt; &gt; &gt; &gt; ---<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/asm-arm/numa.h | 8 +++++++-<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/xen/numa.h=C2=A0 =C2=A0 =C2=A0| =
2 ++<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 2 files changed, 9 insertions(+), 1 deletion=
(-)<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/asm-arm/numa.h b/xen/incl=
ude/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; index 8f1c67e3eb..21569e634b 100644<br>
&gt; &gt; &gt; &gt; &gt; --- a/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; +++ b/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; @@ -3,9 +3,15 @@<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 #include &lt;xen/mm.h&gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; +#include &lt;asm/setup.h&gt;<br>
&gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 typedef u8 nodeid_t;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; -#ifndef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; +#ifdef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; +#define NR_NODE_MEMBLKS NR_MEM_BANKS<br>
&gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; +#else<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 /* Fake one node for now. See also node_onli=
ne_map. */<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 #define cpu_to_node(cpu) 0<br>
&gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/xen/numa.h b/xen/include/=
xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; index 1978e2be1b..1731e1cc6b 100644<br>
&gt; &gt; &gt; &gt; &gt; --- a/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; +++ b/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; @@ -12,7 +12,9 @@<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 #define MAX_NUMNODES=C2=A0 =C2=A0 1<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 #endif<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; +#ifndef NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt;=C2=A0 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)<br>
&gt; &gt; &gt; &gt; &gt; +#endif<br>
&gt; &gt; <br>
&gt; &gt; This one we can remove it completely right?<br>
&gt; <br>
&gt; How about define NR_MEM_BANKS to:<br>
&gt; #ifdef CONFIG_NR_NUMA_NODES<br>
&gt; #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)<br>
&gt; #else<br>
&gt; #define NR_MEM_BANKS 128<br>
&gt; #endif<br>
&gt; for both x86 and Arm. For those architectures do not support or enable=
<br>
&gt; NUMA, they can still use &quot;NR_MEM_BANKS 128&quot;. And replace all=
 NR_NODE_MEMBLKS<br>
&gt; in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.<br>
&gt; In this case, NR_MEM_BANKS can be aware of the changes of CONFIG_NR_NU=
MA_NODES.<br>
<br>
x86 doesn&#39;t have NR_MEM_BANKS as far as I can tell. I guess you also<br=
>
meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?<br>
<br>
But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES because<br=
>
there can be many memory banks for each numa node, certainly more than<br>
2. The existing definition on x86:<br>
<br>
#define NR_NODE_MEMBLKS (MAX_NUMNODES*2)<br>
<br>
Doesn&#39;t make a lot of sense to me. Was it just an arbitrary limit for<b=
r>
the lack of a better way to set a maximum?<br>
<br>
<br>
On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be related.<br>
In fact, what&#39;s the difference?<br>
<br>
NR_MEM_BANKS is the max number of memory banks (with or without<br>
numa-node-id).<br>
<br>
NR_NODE_MEMBLKS is the max number of memory banks with NUMA support<br>
(with numa-node-id)?<br>
<br>
They are basically the same thing. On ARM I would just do:<br>
<br>
#define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))<br></=
blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">As you=
 wrote above, the second part of the MAX is totally arbitrary. In fact, it =
is very likely than if you have more than 64 nodes, you may need a lot more=
 than 2 regions per node.</div><div dir=3D"auto"><br></div><div dir=3D"auto=
">So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to NR_MEM_BA=
NKS so it can be used by common code.</div><div dir=3D"auto"></div><div dir=
=3D"auto"><br></div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div=
><div dir=3D"auto"></div></div>

--000000000000dc13f905ccf42ba1--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 06:46:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 06:46:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196338.349131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkPO-0006z7-PH; Mon, 27 Sep 2021 06:46:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196338.349131; Mon, 27 Sep 2021 06:46:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkPO-0006z0-LP; Mon, 27 Sep 2021 06:46:38 +0000
Received: by outflank-mailman (input) for mailman id 196338;
 Mon, 27 Sep 2021 06:46:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUkPM-0006ya-LX
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 06:46:36 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.62]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5293489b-af9d-4e21-8186-7dc547bdb355;
 Mon, 27 Sep 2021 06:46:32 +0000 (UTC)
Received: from AM6P194CA0057.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::34)
 by VI1PR08MB3231.eurprd08.prod.outlook.com (2603:10a6:803:4a::12) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 06:46:29 +0000
Received: from AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:84:cafe::37) by AM6P194CA0057.outlook.office365.com
 (2603:10a6:209:84::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 06:46:29 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT021.mail.protection.outlook.com (10.152.16.105) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 06:46:28 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Mon, 27 Sep 2021 06:46:25 +0000
Received: from cedbc914dd42.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A580B835-FE01-40A4-9381-AAF4AA14B66C.1; 
 Mon, 27 Sep 2021 06:46:14 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cedbc914dd42.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 06:46:14 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4489.eurprd08.prod.outlook.com (2603:10a6:10:cf::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 06:46:12 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 06:46:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5293489b-af9d-4e21-8186-7dc547bdb355
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CTrJnL0+GuKNeC5npFGaYN3AtHDG8HtXaFbXkHhc5VI=;
 b=etdvZeLONS+J5SAtlgla3z7VLqJrdwmReK90qUFl9tN0lgUVJgnM1llSG8dqSlj7FSlGSO/GMSfL31kxbpkObeJFJQXWwoMuVGkqAKEU0LRXsDq7ggjRB9NMcobUanMFa0bVwcOpmkoKib6o3njQjuYGezzudRLYynwLJm/9Q8I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lmn0yNEB+KlAY5Ypgc1FrAo+O6o7mgTAoSF9DpBHpNyaeg7eOpPl2a/oVAfHG2Yuj6GMOscp4z7nxUon/ORo5zWRAZxbFuYtl2qLIZ+6kAV8SBN0pMyR5ajgxv5ju6bD/uruJKDBzToTW8rEYA/c564mINrcgfjGFW/BkbQiEHR7xAPaMUVNMgzNAWIBm03iNKo/LY7mXpW2lAz/WLjwX02tMB3ZftxqCuBtUILX+9UuUYJJ2Ozb5gfzm+qCxMSML1ttFI4KS0xtJaZ7VaDVRimnntrKuEcCT7olR0odBQGVdrEx+hf6gFuiruIZi5zXwzvFdB9TkxJkdvkZddpAtw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CTrJnL0+GuKNeC5npFGaYN3AtHDG8HtXaFbXkHhc5VI=;
 b=ZanImUDsQxaHWxtUgYM/wlgg8H151CTrM4j7lWXo6A+l2H5ECRN+ZCfNt/ptdItxuEcHsWFE71RKYbbhW4u4UpCzxsrz3EWn0YUuwVlY18rcgV39namiNB1n124gh1Pwb+M8J3qywTPQZhhnzEvwBjBAB+Fw3wP+MzI17bp1KajNAxBz2WtLST2n/net5JLEn+s1Srb0GxuQyArEVI/gOR7yZb2MXmJ8Iot/Dz1EqC/CPfujCgn9aMVVCAV4+NQYpYX7jbC4UIMVA5p5lr/xZWqNAnBSYAT5Z3X5imY8Tl95IsBmVjhjb0N1aU02qZ3BJq8kS1rNKb7UnXkk8ot8nQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CTrJnL0+GuKNeC5npFGaYN3AtHDG8HtXaFbXkHhc5VI=;
 b=etdvZeLONS+J5SAtlgla3z7VLqJrdwmReK90qUFl9tN0lgUVJgnM1llSG8dqSlj7FSlGSO/GMSfL31kxbpkObeJFJQXWwoMuVGkqAKEU0LRXsDq7ggjRB9NMcobUanMFa0bVwcOpmkoKib6o3njQjuYGezzudRLYynwLJm/9Q8I=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "andrew.cooper3@citrix.com"
	<andrew.cooper3@citrix.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index:
 AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CCAAP08AIAADO8QgAANUgCAABvh0A==
Date: Mon, 27 Sep 2021 06:46:11 +0000
Message-ID:
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 2DC51D831ACAD248913C88B5A9E58022.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 2bdc8bd4-059f-4505-dd0f-08d981828888
x-ms-traffictypediagnostic: DBBPR08MB4489:|VI1PR08MB3231:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB323132BC465F8CAE1755B67F9EA79@VI1PR08MB3231.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 tXy7snAM1tZbhaWL4tULzxITkRO52ogIlS2290a4nXocR/u1FNQ176isjtQQojnPdJ+0Svp0irlkxTXcMCrNuFw+xuH3n9yfmZ1tl/Hd5ESeZI9J21sAT7v0kS2C45oEHw6iTmUV5K+G6ToL/833YIxt7tPRRLUGDBd8GOumeX6QAK/Tndawgm71QjI9/V2JebHGbq2kwt4vfaItCmN1KqSWy/Qh3VLb988WCn+4UiAqWebCp2r9M1M/s3FhH1ygmdXP7Nb4u61CQAEj36jS3UUb/V3QGRxsnvKI4iLEPY2rGKor2zPaBOyGzyfNfzPzeSYjxm5tzfsActzt9WOQqodW11KnnzwM7VoYKYchvkm73mM5Qg36W2jQb5vuIi7wBkoYEBy51EzAUNq/1P2Fy/BF4tGPETF1gEISZ4/lKKs3qN51J4gk1Tipluiyy2zR0tpZzkSOfw84OLId/RlZbhyU5IjHirhUOykDuA3Qbh3ar5U45bGHko/p0ZmXq8XF+gWmZ7weVKRy70LsuDN/Jh5x1hr+g/OcDoScTKeqZyB5PhwJGC/eDCkxndMj3oQ3D7SC9bUCDcBK6gF1kBZVCbNeXFeZHqi+iHt4DgFNMbj0UuU86qwr7N9d2myb4x7XPXSvXaY85qlsBjzuwYNHQCTK6ZHnQtXBoHBMaHutoEhhTzwFeuraLQLzci7CrSskvxWkk2rpzpN3Xt7xA73lhbhpPEXnMkji5wOlyuFveWnALB7134WvzPphijLvXdAOF9DqePwzfd2QUuaJ48MO7/HyALayZdTV8s328Qwsyjc=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(6916009)(122000001)(54906003)(86362001)(8936002)(76116006)(508600001)(5660300002)(7696005)(2906002)(966005)(8676002)(316002)(9686003)(71200400001)(55016002)(4326008)(66946007)(66446008)(6506007)(53546011)(26005)(64756008)(38070700005)(66556008)(66476007)(33656002)(52536014)(83380400001)(186003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4489
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	30ba2efe-1acc-4b50-a3e7-08d981827eae
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WdpZ0MQ6UFemApPgf0FZGq+vEG1XD2crzoirGWpVliPEkHT5qKvhYJhVq9sPEqeMKIPTMrnM1dEd5udnMWNkgWeo2NsXOWHDdOFNwIdz8CKsKLbrm5fNm30uIq1KrIix7kV0I66+XBwb3XdCR1DjtIE2cljMI6SQ+XUHUI6ucMjJwhjkhPNYL5aG3+dfuDNW9Vs9IPkWtCrnXQODWk1O/rlbGC7ZehOkNcDPWyCOKAzUoJxfTx9/G1LSUxzn/yCUZ4Yg/X1eVfOThC/8t7J1m22vT0QGV0gZnkVUv4aEriUBlGAr24ZuqWhFAp3RD4zo+p1W88IMQ56xbNWXE3RgkrvroYITVhRJDi+gY37TxnbIn4EC8m4ZlktErEe0WxCKEjWPffNhrEqZgPS5+zkDNbxjsEQavU5X2Tmav30T1Dy9urfo9nqmUjkQxMHEY1NslHZSI8shQWG9I0b4vUlA968YJDYdQ/14tNWduKAMSlIodV+JofRHfWFnrk+rrcAwrl4oDEPLyiHyt/VfYcLvu30kzGG3Vt8S7lIRZUARZqeeRkrMG+gWPyQQL5xRbSjQtjZjvTkMRKTwZP4ztKbZOjZHmDC9o1rt8zWkhyP6SiJO0SUBBDy3UmBze3g/vCnhtnr/uvOVftjlCA6gZCHYGZoxk/KR/G/PbrjwVds9ANCyuMRx85dHDgA4UZL3AjlCsJvOv6K/YOryFrAGoHi6vEyxGPWAHA8rbuowGPluQ9VN6Z37WbPyy50Et2Z2I24yebVqwXVJ+wHXfYgCEKnK08oYm5/l3ceDFlInnCVVOvM=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(356005)(82310400003)(5660300002)(83380400001)(52536014)(33656002)(81166007)(36860700001)(86362001)(70206006)(70586007)(2906002)(6506007)(8936002)(107886003)(186003)(966005)(336012)(508600001)(55016002)(53546011)(7696005)(4326008)(26005)(54906003)(316002)(8676002)(47076005)(6862004)(9686003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 06:46:28.5294
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2bdc8bd4-059f-4505-dd0f-08d981828888
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3231

SGkgU3RlZmFubywgSnVsaWVuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy
b206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDog
MjAyMeW5tDnmnIgyN+aXpSAxMzowMA0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+
DQo+IENjOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4t
DQo+IGRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQg
TWFycXVpcw0KPiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hAc3VzZS5jb207
IHJvZ2VyLnBhdUBjaXRyaXguY29tOw0KPiBhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tDQo+IFN1
YmplY3Q6IFJFOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9NRU1fQkFOS1MgdG8gb3Zl
cnJpZGUgZGVmYXVsdA0KPiBOUl9OT0RFX01FTUJMS1MNCj4gDQo+ICt4ODYgbWFpbnRhaW5lcnMN
Cj4gDQo+IE9uIE1vbiwgMjcgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gLS0tLS1P
cmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0
YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiA+IFNlbnQ6IDIwMjHlubQ55pyIMjfml6UgMTE6MjYN
Cj4gPiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiA+IENjOiBTdGVmYW5v
IFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tDQo+ID4gPiBkZXZlbEBs
aXN0cy54ZW5wcm9qZWN0Lm9yZzsganVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMNCj4g
PiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+ID4gPiBTdWJqZWN0OiBSRTogW1BBVENI
IDIyLzM3XSB4ZW4vYXJtOiB1c2UgTlJfTUVNX0JBTktTIHRvIG92ZXJyaWRlDQo+IGRlZmF1bHQN
Cj4gPiA+IE5SX05PREVfTUVNQkxLUw0KPiA+ID4NCj4gPiA+IE9uIFN1biwgMjYgU2VwIDIwMjEs
IFdlaSBDaGVuIHdyb3RlOg0KPiA+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+
ID4gPiA+ID4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3Jn
Pg0KPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ55pyIMjTml6UgOTozNQ0KPiA+ID4gPiA+IFRvOiBX
ZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiA+ID4gPiBDYzogeGVuLWRldmVsQGxpc3Rz
LnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGluaUBrZXJuZWwub3JnOw0KPiA+ID4ganVsaWVuQHhl
bi5vcmc7DQo+ID4gPiA+ID4gQmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0u
Y29tPg0KPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBO
Ul9NRU1fQkFOS1MgdG8gb3ZlcnJpZGUNCj4gPiA+IGRlZmF1bHQNCj4gPiA+ID4gPiBOUl9OT0RF
X01FTUJMS1MNCj4gPiA+ID4gPg0KPiA+ID4gPiA+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBD
aGVuIHdyb3RlOg0KPiA+ID4gPiA+ID4gQXMgYSBtZW1vcnkgcmFuZ2UgZGVzY3JpYmVkIGluIGRl
dmljZSB0cmVlIGNhbm5vdCBiZSBzcGxpdA0KPiBhY3Jvc3MNCj4gPiA+ID4gPiA+IG11bHRpcGxl
IG5vZGVzLiBTbyB3ZSBkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIGFzIE5SX01FTV9CQU5LUyBpbg0K
PiA+ID4gPiA+ID4gYXJjaCBoZWFkZXIuDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiBUaGlzIHN0YXRl
bWVudCBpcyB0cnVlIGJ1dCB3aGF0IGlzIHRoZSBnb2FsIG9mIHRoaXMgcGF0Y2g/IElzIGl0DQo+
IHRvDQo+ID4gPiA+ID4gcmVkdWNlIGNvZGUgc2l6ZSBhbmQgbWVtb3J5IGNvbnN1bXB0aW9uPw0K
PiA+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+IE5vLCB3aGVuIEp1bGllbiBhbmQgSSBkaXNjdXNz
ZWQgdGhpcyBpbiBsYXN0IHZlcnNpb25bMV0sIHdlIGhhZG4ndA0KPiA+ID4gdGhvdWdodA0KPiA+
ID4gPiBzbyBkZWVwbHkuIFdlIGp1c3QgdGhvdWdodCBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQg
aW4gRFQgY2Fubm90IGJlDQo+ID4gPiBzcGxpdA0KPiA+ID4gPiBhY3Jvc3MgbXVsdGlwbGUgbm9k
ZXMuIFNvIE5SX01FTV9CQU5LUyBzaG91bGQgYmUgZXF1YWwgdG8NCj4gTlJfTUVNX0JBTktTLg0K
PiA+ID4gPg0KPiA+ID4gPiBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hpdmVzL2h0
bWwveGVuLWRldmVsLzIwMjEtDQo+ID4gPiAwOC9tc2cwMDk3NC5odG1sDQo+ID4gPiA+DQo+ID4g
PiA+ID4gSSBhbSBhc2tpbmcgYmVjYXVzZSBOUl9NRU1fQkFOS1MgaXMgMTI4IGFuZA0KPiA+ID4g
PiA+IE5SX05PREVfTUVNQkxLUz0yKk1BWF9OVU1OT0RFUyB3aGljaCBpcyA2NCBieSBkZWZhdWx0
IHNvIGFnYWluDQo+ID4gPiA+ID4gTlJfTk9ERV9NRU1CTEtTIGlzIDEyOCBiZWZvcmUgdGhpcyBw
YXRjaC4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IEluIG90aGVyIHdvcmRzLCB0aGlzIHBhdGNoIGFs
b25lIGRvZXNuJ3QgbWFrZSBhbnkgZGlmZmVyZW5jZTsgYXQNCj4gbGVhc3QNCj4gPiA+ID4gPiBk
b2Vzbid0IG1ha2UgYW55IGRpZmZlcmVuY2UgdW5sZXNzIENPTkZJR19OUl9OVU1BX05PREVTIGlz
DQo+IGluY3JlYXNlZC4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IFNvLCBpcyB0aGUgZ29hbCB0byBy
ZWR1Y2UgbWVtb3J5IHVzYWdlIHdoZW4gQ09ORklHX05SX05VTUFfTk9ERVMNCj4gaXMNCj4gPiA+
ID4gPiBoaWdoZXIgdGhhbiA2ND8NCj4gPiA+ID4gPg0KPiA+ID4gPg0KPiA+ID4gPiBJIGFsc28g
dGhvdWdodCBhYm91dCB0aGlzIHByb2JsZW0gd2hlbiBJIHdhcyB3cml0aW5nIHRoaXMgcGF0Y2gu
DQo+ID4gPiA+IENPTkZJR19OUl9OVU1BX05PREVTIGlzIGluY3JlYXNpbmcsIGJ1dCBOUl9NRU1f
QkFOS1MgaXMgYSBmaXhlZA0KPiA+ID4gPiB2YWx1ZSwgdGhlbiBOUl9NRU1fQkFOS1MgY2FuIGJl
IHNtYWxsZXIgdGhhbiBDT05GSUdfTlJfTlVNQV9OT0RFUw0KPiA+ID4gPiBhdCBvbmUgcG9pbnQu
DQo+ID4gPiA+DQo+ID4gPiA+IEJ1dCBJIGFncmVlIHdpdGggSnVsaWVuJ3Mgc3VnZ2VzdGlvbiwg
TlJfTUVNX0JBTktTIGFuZA0KPiBOUl9OT0RFX01FTUJMS1MNCj4gPiA+ID4gbXVzdCBiZSBhd2Fy
ZSBvZiBlYWNoIG90aGVyLiBJIGhhZCB0aG91Z2h0IHRvIGFkZCBzb21lIEFTU0VSVCBjaGVjaywN
Cj4gPiA+ID4gYnV0IEkgZG9uJ3Qga25vdyBob3cgdG8gZG8gaXQgYmV0dGVyLiBTbyBJIHBvc3Qg
dGhpcyBwYXRjaCBmb3IgbW9yZQ0KPiA+ID4gPiBzdWdnZXN0aW9uLg0KPiA+ID4NCj4gPiA+IE9L
LiBJbiB0aGF0IGNhc2UgSSdkIHNheSB0byBnZXQgcmlkIG9mIHRoZSBwcmV2aW91cyBkZWZpbml0
aW9uIG9mDQo+ID4gPiBOUl9OT0RFX01FTUJMS1MgYXMgaXQgaXMgcHJvYmFibHkgbm90IG5lY2Vz
c2FyeSwgc2VlIGJlbG93Lg0KPiA+ID4NCj4gPiA+DQo+ID4gPg0KPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiBBbmQga2VlcCBkZWZhdWx0IE5SX05PREVfTUVNQkxLUyBpbiBjb21tb24gaGVhZGVyDQo+
ID4gPiA+ID4gPiBmb3IgdGhvc2UgYXJjaGl0ZWN0dXJlcyBOVU1BIGlzIGRpc2FibGVkLg0KPiA+
ID4gPiA+DQo+ID4gPiA+ID4gVGhpcyBsYXN0IHNlbnRlbmNlIGlzIG5vdCBhY2N1cmF0ZTogb24g
eDg2IE5VTUEgaXMgZW5hYmxlZCBhbmQNCj4gPiA+ID4gPiBOUl9OT0RFX01FTUJMS1MgaXMgc3Rp
bGwgZGVmaW5lZCBpbiB4ZW4vaW5jbHVkZS94ZW4vbnVtYS5oICh0aGVyZQ0KPiBpcw0KPiA+ID4g
bm8NCj4gPiA+ID4gPiB4ODYgZGVmaW5pdGlvbiBvZiBpdCkNCj4gPiA+ID4gPg0KPiA+ID4gPg0K
PiA+ID4gPiBZZXMuDQo+ID4gPiA+DQo+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IFNpZ25lZC1vZmYt
Ynk6IFdlaSBDaGVuIDx3ZWkuY2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gLS0tDQo+ID4gPiA+
ID4gPiAgeGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmggfCA4ICsrKysrKystDQo+ID4gPiA+ID4g
PiAgeGVuL2luY2x1ZGUveGVuL251bWEuaCAgICAgfCAyICsrDQo+ID4gPiA+ID4gPiAgMiBmaWxl
cyBjaGFuZ2VkLCA5IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkNCj4gPiA+ID4gPiA+DQo+
ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmggYi94ZW4v
aW5jbHVkZS9hc20tDQo+IGFybS9udW1hLmgNCj4gPiA+ID4gPiA+IGluZGV4IDhmMWM2N2UzZWIu
LjIxNTY5ZTYzNGIgMTAwNjQ0DQo+ID4gPiA+ID4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJt
L251bWEuaA0KPiA+ID4gPiA+ID4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4g
PiA+ID4gPiA+IEBAIC0zLDkgKzMsMTUgQEANCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiAgI2lu
Y2x1ZGUgPHhlbi9tbS5oPg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ICsjaW5jbHVkZSA8YXNt
L3NldHVwLmg+DQo+ID4gPiA+ID4gPiArDQo+ID4gPiA+ID4gPiAgdHlwZWRlZiB1OCBub2RlaWRf
dDsNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiAtI2lmbmRlZiBDT05GSUdfTlVNQQ0KPiA+ID4g
PiA+ID4gKyNpZmRlZiBDT05GSUdfTlVNQQ0KPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gKyNk
ZWZpbmUgTlJfTk9ERV9NRU1CTEtTIE5SX01FTV9CQU5LUw0KPiA+ID4gPiA+ID4gKw0KPiA+ID4g
PiA+ID4gKyNlbHNlDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gIC8qIEZha2Ugb25lIG5vZGUg
Zm9yIG5vdy4gU2VlIGFsc28gbm9kZV9vbmxpbmVfbWFwLiAqLw0KPiA+ID4gPiA+ID4gICNkZWZp
bmUgY3B1X3RvX25vZGUoY3B1KSAwDQo+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1
ZGUveGVuL251bWEuaCBiL3hlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4gPiA+ID4gPiA+IGluZGV4
IDE5NzhlMmJlMWIuLjE3MzFlMWNjNmIgMTAwNjQ0DQo+ID4gPiA+ID4gPiAtLS0gYS94ZW4vaW5j
bHVkZS94ZW4vbnVtYS5oDQo+ID4gPiA+ID4gPiArKysgYi94ZW4vaW5jbHVkZS94ZW4vbnVtYS5o
DQo+ID4gPiA+ID4gPiBAQCAtMTIsNyArMTIsOSBAQA0KPiA+ID4gPiA+ID4gICNkZWZpbmUgTUFY
X05VTU5PREVTICAgIDENCj4gPiA+ID4gPiA+ICAjZW5kaWYNCj4gPiA+ID4gPiA+DQo+ID4gPiA+
ID4gPiArI2lmbmRlZiBOUl9OT0RFX01FTUJMS1MNCj4gPiA+ID4gPiA+ICAjZGVmaW5lIE5SX05P
REVfTUVNQkxLUyAoTUFYX05VTU5PREVTKjIpDQo+ID4gPiA+ID4gPiArI2VuZGlmDQo+ID4gPg0K
PiA+ID4gVGhpcyBvbmUgd2UgY2FuIHJlbW92ZSBpdCBjb21wbGV0ZWx5IHJpZ2h0Pw0KPiA+DQo+
ID4gSG93IGFib3V0IGRlZmluZSBOUl9NRU1fQkFOS1MgdG86DQo+ID4gI2lmZGVmIENPTkZJR19O
Ul9OVU1BX05PREVTDQo+ID4gI2RlZmluZSBOUl9NRU1fQkFOS1MgKENPTkZJR19OUl9OVU1BX05P
REVTICogMikNCj4gPiAjZWxzZQ0KPiA+ICNkZWZpbmUgTlJfTUVNX0JBTktTIDEyOA0KPiA+ICNl
bmRpZg0KPiA+IGZvciBib3RoIHg4NiBhbmQgQXJtLiBGb3IgdGhvc2UgYXJjaGl0ZWN0dXJlcyBk
byBub3Qgc3VwcG9ydCBvciBlbmFibGUNCj4gPiBOVU1BLCB0aGV5IGNhbiBzdGlsbCB1c2UgIk5S
X01FTV9CQU5LUyAxMjgiLiBBbmQgcmVwbGFjZSBhbGwNCj4gTlJfTk9ERV9NRU1CTEtTDQo+ID4g
aW4gTlVNQSBjb2RlIHRvIE5SX01FTV9CQU5LUyB0byByZW1vdmUgTlJfTk9ERV9NRU1CTEtTIGNv
bXBsZXRlbHkuDQo+ID4gSW4gdGhpcyBjYXNlLCBOUl9NRU1fQkFOS1MgY2FuIGJlIGF3YXJlIG9m
IHRoZSBjaGFuZ2VzIG9mDQo+IENPTkZJR19OUl9OVU1BX05PREVTLg0KPiANCj4geDg2IGRvZXNu
J3QgaGF2ZSBOUl9NRU1fQkFOS1MgYXMgZmFyIGFzIEkgY2FuIHRlbGwuIEkgZ3Vlc3MgeW91IGFs
c28NCj4gbWVhbnQgdG8gcmVuYW1lIE5SX05PREVfTUVNQkxLUyB0byBOUl9NRU1fQkFOS1M/DQo+
IA0KDQpZZXMuDQoNCj4gQnV0IE5SX01FTV9CQU5LUyBpcyBub3QgZGlyZWN0bHkgcmVsYXRlZCB0
byBDT05GSUdfTlJfTlVNQV9OT0RFUyBiZWNhdXNlDQo+IHRoZXJlIGNhbiBiZSBtYW55IG1lbW9y
eSBiYW5rcyBmb3IgZWFjaCBudW1hIG5vZGUsIGNlcnRhaW5seSBtb3JlIHRoYW4NCj4gMi4gVGhl
IGV4aXN0aW5nIGRlZmluaXRpb24gb24geDg2Og0KPiANCj4gI2RlZmluZSBOUl9OT0RFX01FTUJM
S1MgKE1BWF9OVU1OT0RFUyoyKQ0KPiANCj4gRG9lc24ndCBtYWtlIGEgbG90IG9mIHNlbnNlIHRv
IG1lLiBXYXMgaXQganVzdCBhbiBhcmJpdHJhcnkgbGltaXQgZm9yDQo+IHRoZSBsYWNrIG9mIGEg
YmV0dGVyIHdheSB0byBzZXQgYSBtYXhpbXVtPw0KPiANCg0KQXQgdGhhdCB0aW1lLCB0aGlzIHdh
cyBwcm9iYWJseSB0aGUgbW9zdCBjb3N0LWVmZmVjdGl2ZSBhcHByb2FjaC4NCkVub3VnaCBhbmQg
ZWFzeS4gQnV0LCBpZiBtb3JlIG5vZGVzIG5lZWQgdG8gYmUgc3VwcG9ydGVkIGluIHRoZQ0KZnV0
dXJlLCBpdCBtYXkgYnJpbmcgbW9yZSBtZW1vcnkgYmxvY2tzLiBBbmQgdGhpcyBtYXhpbXVtIHZh
bHVlDQptaWdodCBub3QgYXBwbHkuIFRoZSBtYXhpbXVtIG1heSBuZWVkIHRvIHN1cHBvcnQgZHlu
YW1pYyBleHRlbnNpb24uDQoNCj4gDQo+IE9uIHRoZSBvdGhlciBoYW5kLCBOUl9NRU1fQkFOS1Mg
YW5kIE5SX05PREVfTUVNQkxLUyBzZWVtIHRvIGJlIHJlbGF0ZWQuDQo+IEluIGZhY3QsIHdoYXQn
cyB0aGUgZGlmZmVyZW5jZT8NCj4gDQo+IE5SX01FTV9CQU5LUyBpcyB0aGUgbWF4IG51bWJlciBv
ZiBtZW1vcnkgYmFua3MgKHdpdGggb3Igd2l0aG91dA0KPiBudW1hLW5vZGUtaWQpLg0KPiANCj4g
TlJfTk9ERV9NRU1CTEtTIGlzIHRoZSBtYXggbnVtYmVyIG9mIG1lbW9yeSBiYW5rcyB3aXRoIE5V
TUEgc3VwcG9ydA0KPiAod2l0aCBudW1hLW5vZGUtaWQpPw0KPiANCj4gVGhleSBhcmUgYmFzaWNh
bGx5IHRoZSBzYW1lIHRoaW5nLiBPbiBBUk0gSSB3b3VsZCBqdXN0IGRvOg0KPiANCg0KUHJvYmFi
bHkgbm90LCBOUl9NRU1fQkFOS1Mgd2lsbCBjb3VudCB0aG9zZSBtZW1vcnkgcmFuZ2VzIHdpdGhv
dXQNCm51bWEtbm9kZS1pZCBpbiBib290IG1lbW9yeSBwYXJzaW5nIHN0YWdlIChwcm9jZXNzX21l
bW9yeV9ub2RlIG9yDQpFRkkgcGFyc2VyKS4gQnV0IE5SX05PREVfTUVNQkxLUyB3aWxsIG9ubHkg
Y291bnQgdGhvc2UgbWVtb3J5IHJhbmdlcw0Kd2l0aCBudW1hLW5vZGUtaWQuDQoNCj4gI2RlZmlu
ZSBOUl9OT0RFX01FTUJMS1MgTUFYKE5SX01FTV9CQU5LUywgKENPTkZJR19OUl9OVU1BX05PREVT
ICogMikpDQo+IA0KPiANCj4gQW5kIG1heWJlIHRoZSBkZWZpbml0aW9uIGNvdWxkIGJlIGNvbW1v
biB3aXRoIHg4NiBpZiB3ZSBkZWZpbmUNCj4gTlJfTUVNX0JBTktTIHRvIDEyOCBvbiB4ODYgdG9v
Lg0KDQpKdWxpZW4gaGFkIGNvbW1lbnQgaGVyZSwgSSB3aWxsIGNvbnRpbnVlIGluIHRoYXQgZW1h
aWwuDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 06:53:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 06:53:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196344.349142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkW2-0000D8-IT; Mon, 27 Sep 2021 06:53:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196344.349142; Mon, 27 Sep 2021 06:53:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkW2-0000D1-EZ; Mon, 27 Sep 2021 06:53:30 +0000
Received: by outflank-mailman (input) for mailman id 196344;
 Mon, 27 Sep 2021 06:53:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUkW0-0000Cv-MN
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 06:53:28 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.51]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9d45bbb8-1f5f-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 06:53:26 +0000 (UTC)
Received: from DU2PR04CA0086.eurprd04.prod.outlook.com (2603:10a6:10:232::31)
 by DB6PR0802MB2165.eurprd08.prod.outlook.com (2603:10a6:4:86::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 06:53:24 +0000
Received: from DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:232:cafe::30) by DU2PR04CA0086.outlook.office365.com
 (2603:10a6:10:232::31) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 06:53:24 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT003.mail.protection.outlook.com (10.152.20.157) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 06:53:24 +0000
Received: ("Tessian outbound b324a1dbd9be:v103");
 Mon, 27 Sep 2021 06:53:24 +0000
Received: from 225e26587021.3
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5FAE5026-D7F1-4D65-A62F-613251D87655.1; 
 Mon, 27 Sep 2021 06:53:13 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 225e26587021.3
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 06:53:13 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB5898.eurprd08.prod.outlook.com (2603:10a6:10:20c::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 06:53:10 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 06:53:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d45bbb8-1f5f-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y69604WGFM8RocY7M+WTze36g7EYSACfEaBY0trGQs4=;
 b=yMU4mseWn5T0FfTzZXJGPYLrzWSbUt8l2F9mRmEOE59k+/aHR+WIbMr/IJ9uFqXlBDNWd5j308RDeG4/0Wv2y7BQ/iAzCtJd7v7Hxx9vE4UZigENZew7ngvxI6nNR7J6ZfqKL+aNFWxkOpWExgIlzfqmxPk14fuCgALgrZL9KkE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cpDudCo7r1DdSFLpzNXr6XtHTgtP4yk2GQvYFGf8QXpmM0VExh3oTMMhmPIs5TU54tANp6EZTIITgVcI68n9tE3if/Hf3uOPziMOISc3Zj/idef6YxUESaLIDzawLpxriyPPSQQbZZOuIMJyixQRAcOZKI5q7I2EDZuEKVikKxnmH3iU744NGkGauOmhlqn6VKG4uVCWsgBy+quu2EGZPWJEDV/CFM9tfRXXrxEn6sXx8Gep1QxIWGmKicbPCu+4pC/Eytb41tHDW4+URVibNxsxAUY4p7odskWeNoBgbdxQHiY/ylMho8BJxldu/c1+M5ov/1xCUkCiljBrnFw26w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Y69604WGFM8RocY7M+WTze36g7EYSACfEaBY0trGQs4=;
 b=CvJtZRVdjKM1z0vPRkoxblaT0xP5mRG3SSBHQJbB9FOCdvgKRjBSnvBTOGvWwHf3lkIhBX9CAZslRNfCJ7sqLcNKAZMyrfh4wMk6Ydc5CZTaJMwNY0lPUpnPDSMVEHrU7N94wh4PMuepAAJ+rLE0AJLqPHlDzAhwAetLW7wy8eeyUJzfY33phjrM2sM+hgL1xyIhUZW1qliVX7hqIKatHs4RXgPM+Yql8vGJ6nmo4If3VjhCsrD4zh26qI1Ve+f9fIhe3tjV6lXYXFwuYzolaa9f9rVJGIf/lc7TFr8iI0VB3p0NH151PEi64P9sivOX2VBGysWMNPNqatQWK0ud4w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y69604WGFM8RocY7M+WTze36g7EYSACfEaBY0trGQs4=;
 b=yMU4mseWn5T0FfTzZXJGPYLrzWSbUt8l2F9mRmEOE59k+/aHR+WIbMr/IJ9uFqXlBDNWd5j308RDeG4/0Wv2y7BQ/iAzCtJd7v7Hxx9vE4UZigENZew7ngvxI6nNR7J6ZfqKL+aNFWxkOpWExgIlzfqmxPk14fuCgALgrZL9KkE=
From: Wei Chen <Wei.Chen@arm.com>
To: Wei Chen <Wei.Chen@arm.com>, Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "andrew.cooper3@citrix.com"
	<andrew.cooper3@citrix.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index:
 AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CCAAP08AIAADO8QgAANUgCAABvh0IAAAsXQ
Date: Mon, 27 Sep 2021 06:53:10 +0000
Message-ID:
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
In-Reply-To:
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 972622BF6888314E89591205416934EB.0
x-checkrecipientchecked: true
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 2dd6fee6-2287-46e7-c71d-08d98183805d
x-ms-traffictypediagnostic: DBBPR08MB5898:|DB6PR0802MB2165:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB21651D137E5D16488E631A369EA79@DB6PR0802MB2165.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GhHGJnnjynD/yVRMIgql/cTmM1WMMvbx7GK/2bvA7JrEuq3vRxkWAIpqJvy32zkFiL60Gb7ZDbwETgMpyraykyNXBYLUgg0LT4Q1ttzIUqexh5C4EUvWfUqWL2cm9Q8yTban9gurOiINzTerIf2IS05gwIDbDbXuSSFhwZMeDjkkozK6WdvBm9FBx0CUI4Moxu7Cq+BSv0t0ymXKqPP5p185GoP2NvI9fLdKStIuOb8iO770uED2Kfbmn96jjSQPC2p+39eO57ybgBrxWbhLXWxT8c3zwrIIWL2MtdmLQG/OpC4knAmG408iGFEQHUOpMUUmetHdPfq9JoA1BpD55ailcXmW67yYseB58+7A5aMFmIW8skuSAgxSu+k8RQPsWRJPtWpoKz28Rj05M1fUrSvbugalejwXMvMKrVNFiRhRviuAsrVd0wRZ7L8GBFMkGBgM8iX87SVTOj2v2fZQeMw8xPn4ZqF8jdvAY0UQ+RdoTTmTY4YvIQjPeI+qkwFQjsKTI2uxoZkm7rocD40KGRV4WXYRD9hJQMJ01a/M1cuQva20jwmVq//pz0zj8AqJUKg5K/C00BOkAgiyeX8NZ2YA34zL7z1qVbx21whxG2lSkz8zsBQsJkDGCgRy11Rid3vrO5wPL+6trlTKDE9bbqvXsOVnKAoBD69tv3Ehh6RF0fyVALBFvfWmNM14WvotkEgrW2sruO3I7xBUI1V0DjgGPAPp6bAP004zYhheDBmaaVH97a7YhLg0KkO1CCya8yMapDD1jOv7qYKTriS42SX6YXiyK7VxE4OA2WFqSlI=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2940100002)(33656002)(316002)(71200400001)(66946007)(186003)(66476007)(66556008)(508600001)(76116006)(7696005)(5660300002)(64756008)(52536014)(66446008)(110136005)(4326008)(966005)(54906003)(86362001)(83380400001)(38070700005)(2906002)(9686003)(6506007)(26005)(122000001)(53546011)(55016002)(8676002)(38100700002)(8936002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5898
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b1981227-ffd9-4108-3daa-08d98183780b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	c/XXIJYNC9eY8t1/6j99HaO6OkiOwKEBNcHCOfsCUPawgO3ji/c14/IPIdROTAZEHBVWqBTmIkEeF/5BuAjYTs4SOqZtjrv2MOip9WX3yXi0g24JVZRkzwJYQXMk6kW62XqWPvGawhfpGBheexstjvw0+1c2t5vLvTdzbEAWYqSow9YJSuWrpR1tp15k8OpkKAFAVx0f77wo+mzCd2QfWD0AlI2R+KNgVU7KpsGxGJ+XpSH9O9GMA6btDZ+ALE9UkPxbTwrlTG6qAdBbbtnajGe9/gJZFMtoPzd+TtOFNLW71/5xS8PLOWgWeUC1brYLgVVrZF8KQpo3bUVoVoXxF4Tm5zaAVSi3AohEOWhN3qlvET0rn+brn1F139gP0NIShvSHOqmRDh0IKG1tQTCnGctjC53tFYmgST01/P2E7rXKk3AKl3YoGqAH+Wccu58j08kbCn4b71PDyRc0jhRtmjyy9jPUpBmrvYUNkGoo8PcHUUMOZNt8Fu46xlkVJ6qa8l2sjstezkzIqY9+qQYrUVC2tQs3u5HtOyHBnv5QqvckNNkQf8mKa35nXQpo4si+TyjioCa0zGLdOdEwFIv5VNPhWp8XTQN/EWUqWLQY1zUNnratRcDha/9B/y5COThUDsFTbzcmnaFOMwTi0ecFs4ItRmWE2qdPmYPuyobWZ4vepTUHu0EJvPEEyOaaF1pg0XdN12VJFANmeY4BMdxu1bPpEmyG98t0JRpi599kN6ql4Yfg4dB6CR0u5o6Bpoz0hqQXld4yDQQiqvj56XKj2LwSWuz/IjEmtGo2pIvMlwQ=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70206006)(966005)(316002)(86362001)(508600001)(52536014)(33656002)(4326008)(70586007)(356005)(8936002)(110136005)(5660300002)(54906003)(107886003)(53546011)(47076005)(83380400001)(81166007)(9686003)(82310400003)(2940100002)(26005)(6506007)(336012)(186003)(7696005)(36860700001)(8676002)(55016002)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 06:53:24.3894
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2dd6fee6-2287-46e7-c71d-08d98183805d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2165

SGkgSnVsaWVuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFhlbi1k
ZXZlbCA8eGVuLWRldmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFsZiBP
ZiBXZWkNCj4gQ2hlbg0KPiBTZW50OiAyMDIx5bm0OeaciDI35pelIDE0OjQ2DQo+IFRvOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IENjOiB4ZW4tZGV2ZWxA
bGlzdHMueGVucHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+
IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBqYmV1bGljaEBzdXNlLmNvbTsgcm9nZXIucGF1
QGNpdHJpeC5jb207DQo+IGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb20NCj4gU3ViamVjdDogUkU6
IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01FTV9CQU5LUyB0byBvdmVycmlkZSBkZWZh
dWx0DQo+IE5SX05PREVfTUVNQkxLUw0KPiANCj4gSGkgU3RlZmFubywgSnVsaWVuLA0KPiANCj4g
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiBTZW50OiAyMDIx5bm0OeaciDI35pelIDEz
OjAwDQo+ID4gVG86IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiA+IENjOiBTdGVmYW5v
IFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tDQo+ID4gZGV2ZWxAbGlz
dHMueGVucHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4g
PEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IGpiZXVsaWNoQHN1c2UuY29tOyByb2dlci5wYXVA
Y2l0cml4LmNvbTsNCj4gPiBhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tDQo+ID4gU3ViamVjdDog
UkU6IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01FTV9CQU5LUyB0byBvdmVycmlkZSBk
ZWZhdWx0DQo+ID4gTlJfTk9ERV9NRU1CTEtTDQo+ID4NCj4gPiAreDg2IG1haW50YWluZXJzDQo+
ID4NCj4gPiBPbiBNb24sIDI3IFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gLS0t
LS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+ID4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5p
IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4gPiBTZW50OiAyMDIx5bm0OeaciDI35pel
IDExOjI2DQo+ID4gPiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiA+ID4g
Q2M6IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0NCj4g
PiA+ID4gZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFu
ZCBNYXJxdWlzDQo+ID4gPiA+IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+ID4gPiA+IFN1
YmplY3Q6IFJFOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9NRU1fQkFOS1MgdG8gb3Zl
cnJpZGUNCj4gPiBkZWZhdWx0DQo+ID4gPiA+IE5SX05PREVfTUVNQkxLUw0KPiA+ID4gPg0KPiA+
ID4gPiBPbiBTdW4sIDI2IFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPiA+IC0t
LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPiA+ID4gPiBGcm9tOiBTdGVmYW5vIFN0YWJl
bGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4gPiA+ID4gPiBTZW50OiAyMDIx5bm0
OeaciDI05pelIDk6MzUNCj4gPiA+ID4gPiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNv
bT4NCj4gPiA+ID4gPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IHNzdGFi
ZWxsaW5pQGtlcm5lbC5vcmc7DQo+ID4gPiA+IGp1bGllbkB4ZW4ub3JnOw0KPiA+ID4gPiA+ID4g
QmVydHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPg0KPiA+ID4gPiA+ID4g
U3ViamVjdDogUmU6IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01FTV9CQU5LUyB0byBv
dmVycmlkZQ0KPiA+ID4gPiBkZWZhdWx0DQo+ID4gPiA+ID4gPiBOUl9OT0RFX01FTUJMS1MNCj4g
PiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90
ZToNCj4gPiA+ID4gPiA+ID4gQXMgYSBtZW1vcnkgcmFuZ2UgZGVzY3JpYmVkIGluIGRldmljZSB0
cmVlIGNhbm5vdCBiZSBzcGxpdA0KPiA+IGFjcm9zcw0KPiA+ID4gPiA+ID4gPiBtdWx0aXBsZSBu
b2Rlcy4gU28gd2UgZGVmaW5lIE5SX05PREVfTUVNQkxLUyBhcyBOUl9NRU1fQkFOS1MNCj4gaW4N
Cj4gPiA+ID4gPiA+ID4gYXJjaCBoZWFkZXIuDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gVGhp
cyBzdGF0ZW1lbnQgaXMgdHJ1ZSBidXQgd2hhdCBpcyB0aGUgZ29hbCBvZiB0aGlzIHBhdGNoPyBJ
cyBpdA0KPiA+IHRvDQo+ID4gPiA+ID4gPiByZWR1Y2UgY29kZSBzaXplIGFuZCBtZW1vcnkgY29u
c3VtcHRpb24/DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gTm8sIHdoZW4gSnVs
aWVuIGFuZCBJIGRpc2N1c3NlZCB0aGlzIGluIGxhc3QgdmVyc2lvblsxXSwgd2UgaGFkbid0DQo+
ID4gPiA+IHRob3VnaHQNCj4gPiA+ID4gPiBzbyBkZWVwbHkuIFdlIGp1c3QgdGhvdWdodCBhIG1l
bW9yeSByYW5nZSBkZXNjcmliZWQgaW4gRFQgY2Fubm90DQo+IGJlDQo+ID4gPiA+IHNwbGl0DQo+
ID4gPiA+ID4gYWNyb3NzIG11bHRpcGxlIG5vZGVzLiBTbyBOUl9NRU1fQkFOS1Mgc2hvdWxkIGJl
IGVxdWFsIHRvDQo+ID4gTlJfTUVNX0JBTktTLg0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gaHR0cHM6
Ly9saXN0cy54ZW5wcm9qZWN0Lm9yZy9hcmNoaXZlcy9odG1sL3hlbi1kZXZlbC8yMDIxLQ0KPiA+
ID4gPiAwOC9tc2cwMDk3NC5odG1sDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEkgYW0gYXNraW5n
IGJlY2F1c2UgTlJfTUVNX0JBTktTIGlzIDEyOCBhbmQNCj4gPiA+ID4gPiA+IE5SX05PREVfTUVN
QkxLUz0yKk1BWF9OVU1OT0RFUyB3aGljaCBpcyA2NCBieSBkZWZhdWx0IHNvIGFnYWluDQo+ID4g
PiA+ID4gPiBOUl9OT0RFX01FTUJMS1MgaXMgMTI4IGJlZm9yZSB0aGlzIHBhdGNoLg0KPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+IEluIG90aGVyIHdvcmRzLCB0aGlzIHBhdGNoIGFsb25lIGRvZXNu
J3QgbWFrZSBhbnkgZGlmZmVyZW5jZTsgYXQNCj4gPiBsZWFzdA0KPiA+ID4gPiA+ID4gZG9lc24n
dCBtYWtlIGFueSBkaWZmZXJlbmNlIHVubGVzcyBDT05GSUdfTlJfTlVNQV9OT0RFUyBpcw0KPiA+
IGluY3JlYXNlZC4NCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBTbywgaXMgdGhlIGdvYWwgdG8g
cmVkdWNlIG1lbW9yeSB1c2FnZSB3aGVuIENPTkZJR19OUl9OVU1BX05PREVTDQo+ID4gaXMNCj4g
PiA+ID4gPiA+IGhpZ2hlciB0aGFuIDY0Pw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPg0KPiA+ID4g
PiA+IEkgYWxzbyB0aG91Z2h0IGFib3V0IHRoaXMgcHJvYmxlbSB3aGVuIEkgd2FzIHdyaXRpbmcg
dGhpcyBwYXRjaC4NCj4gPiA+ID4gPiBDT05GSUdfTlJfTlVNQV9OT0RFUyBpcyBpbmNyZWFzaW5n
LCBidXQgTlJfTUVNX0JBTktTIGlzIGEgZml4ZWQNCj4gPiA+ID4gPiB2YWx1ZSwgdGhlbiBOUl9N
RU1fQkFOS1MgY2FuIGJlIHNtYWxsZXIgdGhhbiBDT05GSUdfTlJfTlVNQV9OT0RFUw0KPiA+ID4g
PiA+IGF0IG9uZSBwb2ludC4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IEJ1dCBJIGFncmVlIHdpdGgg
SnVsaWVuJ3Mgc3VnZ2VzdGlvbiwgTlJfTUVNX0JBTktTIGFuZA0KPiA+IE5SX05PREVfTUVNQkxL
Uw0KPiA+ID4gPiA+IG11c3QgYmUgYXdhcmUgb2YgZWFjaCBvdGhlci4gSSBoYWQgdGhvdWdodCB0
byBhZGQgc29tZSBBU1NFUlQNCj4gY2hlY2ssDQo+ID4gPiA+ID4gYnV0IEkgZG9uJ3Qga25vdyBo
b3cgdG8gZG8gaXQgYmV0dGVyLiBTbyBJIHBvc3QgdGhpcyBwYXRjaCBmb3INCj4gbW9yZQ0KPiA+
ID4gPiA+IHN1Z2dlc3Rpb24uDQo+ID4gPiA+DQo+ID4gPiA+IE9LLiBJbiB0aGF0IGNhc2UgSSdk
IHNheSB0byBnZXQgcmlkIG9mIHRoZSBwcmV2aW91cyBkZWZpbml0aW9uIG9mDQo+ID4gPiA+IE5S
X05PREVfTUVNQkxLUyBhcyBpdCBpcyBwcm9iYWJseSBub3QgbmVjZXNzYXJ5LCBzZWUgYmVsb3cu
DQo+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiBB
bmQga2VlcCBkZWZhdWx0IE5SX05PREVfTUVNQkxLUyBpbiBjb21tb24gaGVhZGVyDQo+ID4gPiA+
ID4gPiA+IGZvciB0aG9zZSBhcmNoaXRlY3R1cmVzIE5VTUEgaXMgZGlzYWJsZWQuDQo+ID4gPiA+
ID4gPg0KPiA+ID4gPiA+ID4gVGhpcyBsYXN0IHNlbnRlbmNlIGlzIG5vdCBhY2N1cmF0ZTogb24g
eDg2IE5VTUEgaXMgZW5hYmxlZCBhbmQNCj4gPiA+ID4gPiA+IE5SX05PREVfTUVNQkxLUyBpcyBz
dGlsbCBkZWZpbmVkIGluIHhlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4gKHRoZXJlDQo+ID4gaXMN
Cj4gPiA+ID4gbm8NCj4gPiA+ID4gPiA+IHg4NiBkZWZpbml0aW9uIG9mIGl0KQ0KPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IFllcy4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+ID4NCj4g
PiA+ID4gPiA+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPHdlaS5jaGVuQGFybS5jb20+DQo+
ID4gPiA+ID4gPiA+IC0tLQ0KPiA+ID4gPiA+ID4gPiAgeGVuL2luY2x1ZGUvYXNtLWFybS9udW1h
LmggfCA4ICsrKysrKystDQo+ID4gPiA+ID4gPiA+ICB4ZW4vaW5jbHVkZS94ZW4vbnVtYS5oICAg
ICB8IDIgKysNCj4gPiA+ID4gPiA+ID4gIDIgZmlsZXMgY2hhbmdlZCwgOSBpbnNlcnRpb25zKCsp
LCAxIGRlbGV0aW9uKC0pDQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+IGRpZmYgLS1naXQg
YS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaCBiL3hlbi9pbmNsdWRlL2FzbS0NCj4gPiBhcm0v
bnVtYS5oDQo+ID4gPiA+ID4gPiA+IGluZGV4IDhmMWM2N2UzZWIuLjIxNTY5ZTYzNGIgMTAwNjQ0
DQo+ID4gPiA+ID4gPiA+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gPiA+
ID4gPiA+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oDQo+ID4gPiA+ID4gPiA+IEBA
IC0zLDkgKzMsMTUgQEANCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gICNpbmNsdWRlIDx4
ZW4vbW0uaD4NCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gKyNpbmNsdWRlIDxhc20vc2V0
dXAuaD4NCj4gPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gPiAgdHlwZWRlZiB1OCBub2RlaWRf
dDsNCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gLSNpZm5kZWYgQ09ORklHX05VTUENCj4g
PiA+ID4gPiA+ID4gKyNpZmRlZiBDT05GSUdfTlVNQQ0KPiA+ID4gPiA+ID4gPiArDQo+ID4gPiA+
ID4gPiA+ICsjZGVmaW5lIE5SX05PREVfTUVNQkxLUyBOUl9NRU1fQkFOS1MNCj4gPiA+ID4gPiA+
ID4gKw0KPiA+ID4gPiA+ID4gPiArI2Vsc2UNCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
IC8qIEZha2Ugb25lIG5vZGUgZm9yIG5vdy4gU2VlIGFsc28gbm9kZV9vbmxpbmVfbWFwLiAqLw0K
PiA+ID4gPiA+ID4gPiAgI2RlZmluZSBjcHVfdG9fbm9kZShjcHUpIDANCj4gPiA+ID4gPiA+ID4g
ZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL3hlbi9udW1hLmggYi94ZW4vaW5jbHVkZS94ZW4vbnVt
YS5oDQo+ID4gPiA+ID4gPiA+IGluZGV4IDE5NzhlMmJlMWIuLjE3MzFlMWNjNmIgMTAwNjQ0DQo+
ID4gPiA+ID4gPiA+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4gPiA+ID4gPiA+ID4g
KysrIGIveGVuL2luY2x1ZGUveGVuL251bWEuaA0KPiA+ID4gPiA+ID4gPiBAQCAtMTIsNyArMTIs
OSBAQA0KPiA+ID4gPiA+ID4gPiAgI2RlZmluZSBNQVhfTlVNTk9ERVMgICAgMQ0KPiA+ID4gPiA+
ID4gPiAgI2VuZGlmDQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ICsjaWZuZGVmIE5SX05P
REVfTUVNQkxLUw0KPiA+ID4gPiA+ID4gPiAgI2RlZmluZSBOUl9OT0RFX01FTUJMS1MgKE1BWF9O
VU1OT0RFUyoyKQ0KPiA+ID4gPiA+ID4gPiArI2VuZGlmDQo+ID4gPiA+DQo+ID4gPiA+IFRoaXMg
b25lIHdlIGNhbiByZW1vdmUgaXQgY29tcGxldGVseSByaWdodD8NCj4gPiA+DQo+ID4gPiBIb3cg
YWJvdXQgZGVmaW5lIE5SX01FTV9CQU5LUyB0bzoNCj4gPiA+ICNpZmRlZiBDT05GSUdfTlJfTlVN
QV9OT0RFUw0KPiA+ID4gI2RlZmluZSBOUl9NRU1fQkFOS1MgKENPTkZJR19OUl9OVU1BX05PREVT
ICogMikNCj4gPiA+ICNlbHNlDQo+ID4gPiAjZGVmaW5lIE5SX01FTV9CQU5LUyAxMjgNCj4gPiA+
ICNlbmRpZg0KPiA+ID4gZm9yIGJvdGggeDg2IGFuZCBBcm0uIEZvciB0aG9zZSBhcmNoaXRlY3R1
cmVzIGRvIG5vdCBzdXBwb3J0IG9yIGVuYWJsZQ0KPiA+ID4gTlVNQSwgdGhleSBjYW4gc3RpbGwg
dXNlICJOUl9NRU1fQkFOS1MgMTI4Ii4gQW5kIHJlcGxhY2UgYWxsDQo+ID4gTlJfTk9ERV9NRU1C
TEtTDQo+ID4gPiBpbiBOVU1BIGNvZGUgdG8gTlJfTUVNX0JBTktTIHRvIHJlbW92ZSBOUl9OT0RF
X01FTUJMS1MgY29tcGxldGVseS4NCj4gPiA+IEluIHRoaXMgY2FzZSwgTlJfTUVNX0JBTktTIGNh
biBiZSBhd2FyZSBvZiB0aGUgY2hhbmdlcyBvZg0KPiA+IENPTkZJR19OUl9OVU1BX05PREVTLg0K
PiA+DQo+ID4geDg2IGRvZXNuJ3QgaGF2ZSBOUl9NRU1fQkFOS1MgYXMgZmFyIGFzIEkgY2FuIHRl
bGwuIEkgZ3Vlc3MgeW91IGFsc28NCj4gPiBtZWFudCB0byByZW5hbWUgTlJfTk9ERV9NRU1CTEtT
IHRvIE5SX01FTV9CQU5LUz8NCj4gPg0KPiANCj4gWWVzLg0KPiANCj4gPiBCdXQgTlJfTUVNX0JB
TktTIGlzIG5vdCBkaXJlY3RseSByZWxhdGVkIHRvIENPTkZJR19OUl9OVU1BX05PREVTIGJlY2F1
c2UNCj4gPiB0aGVyZSBjYW4gYmUgbWFueSBtZW1vcnkgYmFua3MgZm9yIGVhY2ggbnVtYSBub2Rl
LCBjZXJ0YWlubHkgbW9yZSB0aGFuDQo+ID4gMi4gVGhlIGV4aXN0aW5nIGRlZmluaXRpb24gb24g
eDg2Og0KPiA+DQo+ID4gI2RlZmluZSBOUl9OT0RFX01FTUJMS1MgKE1BWF9OVU1OT0RFUyoyKQ0K
PiA+DQo+ID4gRG9lc24ndCBtYWtlIGEgbG90IG9mIHNlbnNlIHRvIG1lLiBXYXMgaXQganVzdCBh
biBhcmJpdHJhcnkgbGltaXQgZm9yDQo+ID4gdGhlIGxhY2sgb2YgYSBiZXR0ZXIgd2F5IHRvIHNl
dCBhIG1heGltdW0/DQo+ID4NCj4gDQo+IEF0IHRoYXQgdGltZSwgdGhpcyB3YXMgcHJvYmFibHkg
dGhlIG1vc3QgY29zdC1lZmZlY3RpdmUgYXBwcm9hY2guDQo+IEVub3VnaCBhbmQgZWFzeS4gQnV0
LCBpZiBtb3JlIG5vZGVzIG5lZWQgdG8gYmUgc3VwcG9ydGVkIGluIHRoZQ0KPiBmdXR1cmUsIGl0
IG1heSBicmluZyBtb3JlIG1lbW9yeSBibG9ja3MuIEFuZCB0aGlzIG1heGltdW0gdmFsdWUNCj4g
bWlnaHQgbm90IGFwcGx5LiBUaGUgbWF4aW11bSBtYXkgbmVlZCB0byBzdXBwb3J0IGR5bmFtaWMg
ZXh0ZW5zaW9uLg0KPiANCj4gPg0KPiA+IE9uIHRoZSBvdGhlciBoYW5kLCBOUl9NRU1fQkFOS1Mg
YW5kIE5SX05PREVfTUVNQkxLUyBzZWVtIHRvIGJlIHJlbGF0ZWQuDQo+ID4gSW4gZmFjdCwgd2hh
dCdzIHRoZSBkaWZmZXJlbmNlPw0KPiA+DQo+ID4gTlJfTUVNX0JBTktTIGlzIHRoZSBtYXggbnVt
YmVyIG9mIG1lbW9yeSBiYW5rcyAod2l0aCBvciB3aXRob3V0DQo+ID4gbnVtYS1ub2RlLWlkKS4N
Cj4gPg0KPiA+IE5SX05PREVfTUVNQkxLUyBpcyB0aGUgbWF4IG51bWJlciBvZiBtZW1vcnkgYmFu
a3Mgd2l0aCBOVU1BIHN1cHBvcnQNCj4gPiAod2l0aCBudW1hLW5vZGUtaWQpPw0KPiA+DQo+ID4g
VGhleSBhcmUgYmFzaWNhbGx5IHRoZSBzYW1lIHRoaW5nLiBPbiBBUk0gSSB3b3VsZCBqdXN0IGRv
Og0KPiA+DQo+IA0KPiBQcm9iYWJseSBub3QsIE5SX01FTV9CQU5LUyB3aWxsIGNvdW50IHRob3Nl
IG1lbW9yeSByYW5nZXMgd2l0aG91dA0KPiBudW1hLW5vZGUtaWQgaW4gYm9vdCBtZW1vcnkgcGFy
c2luZyBzdGFnZSAocHJvY2Vzc19tZW1vcnlfbm9kZSBvcg0KPiBFRkkgcGFyc2VyKS4gQnV0IE5S
X05PREVfTUVNQkxLUyB3aWxsIG9ubHkgY291bnQgdGhvc2UgbWVtb3J5IHJhbmdlcw0KPiB3aXRo
IG51bWEtbm9kZS1pZC4NCj4gDQo+ID4gI2RlZmluZSBOUl9OT0RFX01FTUJMS1MgTUFYKE5SX01F
TV9CQU5LUywgKENPTkZJR19OUl9OVU1BX05PREVTICogMikpDQo+ID4NCj4gPg0KDQpRdW90ZSBK
dWxpZW4ncyBjb21tZW50IGZyb20gSFRNTCBlbWFpbCB0byBoZXJlOg0KIiBBcyB5b3Ugd3JvdGUg
YWJvdmUsIHRoZSBzZWNvbmQgcGFydCBvZiB0aGUgTUFYIGlzIHRvdGFsbHkgYXJiaXRyYXJ5Lg0K
SW4gZmFjdCwgaXQgaXMgdmVyeSBsaWtlbHkgdGhhbiBpZiB5b3UgaGF2ZSBtb3JlIHRoYW4gNjQg
bm9kZXMsIHlvdSBtYXkNCm5lZWQgYSBsb3QgbW9yZSB0aGFuIDIgcmVnaW9ucyBwZXIgbm9kZS4N
Cg0KU28sIGZvciBBcm0sIEkgd291bGQganVzdCBkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIGFzIGFu
IGFsaWFzIHRvIE5SX01FTV9CQU5LUw0Kc28gaXQgY2FuIGJlIHVzZWQgYnkgY29tbW9uIGNvZGUu
DQoiDQoNCkJ1dCBoZXJlIGNvbWVzIHRoZSBwcm9ibGVtOg0KSG93IGNhbiB3ZSBzZXQgdGhlIE5S
X01FTV9CQU5LUyBtYXhpbXVtIHZhbHVlLCAxMjggc2VlbXMgYW4gYXJiaXRyYXJ5IHRvbz8NCklm
ICNkZWZpbmUgTlJfTUVNX0JBTktTIChDT05GSUdfTlJfTlVNQV9OT0RFUyAqIE4pPyBBbmQgd2hh
dCBOIHNob3VsZCBiZT8NCg0KPiA+IEFuZCBtYXliZSB0aGUgZGVmaW5pdGlvbiBjb3VsZCBiZSBj
b21tb24gd2l0aCB4ODYgaWYgd2UgZGVmaW5lDQo+ID4gTlJfTUVNX0JBTktTIHRvIDEyOCBvbiB4
ODYgdG9vLg0KPiANCj4gSnVsaWVuIGhhZCBjb21tZW50IGhlcmUsIEkgd2lsbCBjb250aW51ZSBp
biB0aGF0IGVtYWlsLg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 06:58:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 06:58:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196352.349152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkaq-0001C0-8t; Mon, 27 Sep 2021 06:58:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196352.349152; Mon, 27 Sep 2021 06:58:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkaq-0001Bt-5o; Mon, 27 Sep 2021 06:58:28 +0000
Received: by outflank-mailman (input) for mailman id 196352;
 Mon, 27 Sep 2021 06:58:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKXO=OR=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mUkao-0001Bm-TG
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 06:58:27 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2aee7c6c-f13a-4068-9c2a-1443bd178401;
 Mon, 27 Sep 2021 06:58:25 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id i4so72878857lfv.4
 for <xen-devel@lists.xenproject.org>; Sun, 26 Sep 2021 23:58:25 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id t18sm1498351lfl.219.2021.09.26.23.58.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 26 Sep 2021 23:58:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2aee7c6c-f13a-4068-9c2a-1443bd178401
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=azoyT34yzw5FEZAw867wEMI2AGPZuHjULBEBEWOLTig=;
        b=Q4Nopw1q1uNPcyDa3BAy7fLMHBF/YqU9c/CWrCSShVgJJmqcNupvFahPWZAm7FNT4A
         ew0twYnzreXfX/7BvL/XDYu698vIb7W0h/hxgTLoTnXPvjrEpsAD+d8XDMCyL7L6qG1L
         1l0LwSoY1Bqt8VCWl1/pIpKLnwQf2+Hq9qX6ECydH8GGSPnvb1WwH8EKEQActWYHZ4Aj
         3yE/kPFBIaotCno/wt0bLQxzYS5i0tB2/oL/cRH7jX6OZ1MurO0WtrmUAjBHxusOpqhc
         Eoaub1gti6qpgA9qDMYiksLpBNjvHQSjmQqJe4EK+eH0KRfaYRmNcFUndwm0kt+GnlVS
         j5vA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=azoyT34yzw5FEZAw867wEMI2AGPZuHjULBEBEWOLTig=;
        b=3ntwNEKe3GqYBj6gDBsVvi9NSdrxUj8HqnhyyXck1y/g6x0AckEBGKEKQbdMMrHT2L
         SssxSCipotFTeGsGs03YxqvHJTP0WY7cV+bCMCMQVcEOtwJROopq2lMEHITbj9/qViXy
         kVuRX1PFqufKgCtPvHAeDXtMusXhcaa/oZ/MNgWk45XwasqfZGAMftolkX21A5ie7pdm
         6BMthLA99d9m9c9MR+Pc3k+PTOVj+tlAweWnNtwHbHA/HDgsVfItTAt4zxDm3BRao35L
         OM0qbxOHbFqEMaIbjXIFQCR2kfJf46aBih6ff+7O8xuFdAdOYDk0Ry1k6G3av0Rs8Fc9
         NcMQ==
X-Gm-Message-State: AOAM530Wax/EEGF0Hl2mz2hr7lozcJthtUmB1hMDSO4NIL/tX3bqF3D5
	okAHM1r8QTsEfDCbss2RyO5FpJIkjo5udQ==
X-Google-Smtp-Source: ABdhPJzWIN17tXgM6N/bt7MC6BQy3APMvXCiMcbgPs6xmHXFHnpkJI+ja20n13wTWFKVT3rW0DqbyA==
X-Received: by 2002:ac2:5d23:: with SMTP id i3mr17295747lfb.477.1632725904487;
        Sun, 26 Sep 2021 23:58:24 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Date: Mon, 27 Sep 2021 09:58:21 +0300
Message-Id: <20210927065822.350973-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Currently PCI backend implements multiple functionalities at a time.
To name a few:
1. It is used as a database for assignable PCI devices, e.g. xl
   pci-assignable-{add|remove|list} manipulates that list. So, whenever
   the toolstack needs to know which PCI devices can be passed through
   it reads that from the relevant sysfs entries of the pciback.
2. It is used to hold the unbound PCI devices list, e.g. when passing
   through a PCI device it needs to be unbound from the relevant device
   driver and bound to pciback (strictly speaking it is not required
   that the device is bound to pciback, but pciback is again used as a
   database of the passed through PCI devices, so we can re-bind the
   devices back to their original drivers when guest domain shuts down)
3. Device reset for the devices being passed through
4. Para-virtualised use-cases support

The para-virtualised part of the driver is not always needed as some
architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
model for PCI device passthrough. For such use-cases make the very
first step in splitting the xen-pciback driver into two parts: Xen
PCI stub and PCI PV backend drivers.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Changes since v3:
- Move CONFIG_XEN_PCIDEV_STUB to the second patch
Changes since v2:
 - swap the patch order
New in v2
---
 drivers/xen/Kconfig               | 4 ++++
 drivers/xen/Makefile              | 2 +-
 drivers/xen/xen-pciback/Makefile  | 1 +
 drivers/xen/xen-pciback/pciback.h | 5 +++++
 drivers/xen/xen-pciback/xenbus.c  | 6 +++++-
 5 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 22f5aff0c136..4efc95ddda41 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -181,10 +181,14 @@ config SWIOTLB_XEN
 	select DMA_OPS
 	select SWIOTLB
 
+config XEN_PCI_STUB
+	bool
+
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
 	depends on PCI && X86 && XEN
 	depends on XEN_BACKEND
+	select XEN_PCI_STUB
 	default m
 	help
 	  The PCI device backend driver allows the kernel to export arbitrary
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3434593455b2..5aae66e638a7 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
 obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
 obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
 obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
-obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
+obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
 obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
index e8d981d43235..e2cb376444a6 100644
--- a/drivers/xen/xen-pciback/Makefile
+++ b/drivers/xen/xen-pciback/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
+obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
 
 xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
 xen-pciback-y += conf_space.o conf_space_header.o \
diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
index 95e28ee48d52..9a64196e831d 100644
--- a/drivers/xen/xen-pciback/pciback.h
+++ b/drivers/xen/xen-pciback/pciback.h
@@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
 				    struct pci_dev *dev);
 void pcistub_put_pci_dev(struct pci_dev *dev);
 
+static inline bool xen_pcibk_pv_support(void)
+{
+	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
+}
+
 /* Ensure a device is turned off or reset */
 void xen_pcibk_reset_device(struct pci_dev *pdev);
 
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index c09c7ebd6968..f8ba2903a3ff 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
 
 int __init xen_pcibk_xenbus_register(void)
 {
+	if (!xen_pcibk_pv_support())
+		return 0;
+
 	xen_pcibk_backend = &xen_pcibk_vpci_backend;
 	if (passthrough)
 		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
@@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
 
 void __exit xen_pcibk_xenbus_unregister(void)
 {
-	xenbus_unregister_driver(&xen_pcibk_driver);
+	if (xen_pcibk_pv_support())
+		xenbus_unregister_driver(&xen_pcibk_driver);
 }
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 06:58:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 06:58:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196353.349164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkav-0001VG-H6; Mon, 27 Sep 2021 06:58:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196353.349164; Mon, 27 Sep 2021 06:58:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkav-0001V0-DG; Mon, 27 Sep 2021 06:58:33 +0000
Received: by outflank-mailman (input) for mailman id 196353;
 Mon, 27 Sep 2021 06:58:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKXO=OR=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mUkat-0001Bm-Ri
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 06:58:31 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 590cdffa-81e9-412d-b0ea-6ced6ad18a84;
 Mon, 27 Sep 2021 06:58:26 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id u18so71440400lfd.12
 for <xen-devel@lists.xenproject.org>; Sun, 26 Sep 2021 23:58:26 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id t18sm1498351lfl.219.2021.09.26.23.58.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 26 Sep 2021 23:58:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 590cdffa-81e9-412d-b0ea-6ced6ad18a84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=w4/DyioZ/3SsuR8yEN6bZ8EtrReqU8tTeSBuDAdb3VQ=;
        b=qgj5GJKpWfoP9ym013F4TV5ZqlbqGRkpbbOTMSa/RzePc5Cl0WuBwhXH576wBYsX7p
         WOLlznN6Kr77bgZZekIgSLpm6VvnaNmKfz9aqqdsi3JomAkBfozM6MRI69BomD4KuTSS
         B8sxRrTft5JxxGNwIGZQClT2t9NnFNHBESWr3Zh3VzXmfS5izueYMFBz3wlipBjSvAR7
         zDMz6sTg4hdftLJ1VWiMkthhUbL4/KrC1UIl8u33ksgActLJOx2hECcGib87HzFV+x8z
         30FQZKGSIC3T7lvv/OdspYzrGaiFv2+jxOVYibxospTYD9TPTDxkeeSex7yu6Kdppk3r
         4aGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=w4/DyioZ/3SsuR8yEN6bZ8EtrReqU8tTeSBuDAdb3VQ=;
        b=4wirqbOaoE/hdzOJCfeNoJyIamJ2j4uNXPbs6HnZxA51ESMTNioIb9ShKliu7njWCv
         OYQbS3ZyntB+f3yjRrVrD1f7OhpdVHadDxaUSL9aWFEfWD9Ntq4mKcObmZBrw1wX9eSA
         7ma8BGHURYbnHrV2Wc4fx5efrmw6OCQx716KBoOHYJrLf5suDoS2kiO+wQfswlXVbJB3
         wez7jRMFDrQ3+Pg7h/wSyr1qrmd8HWYtEdCHG7Vknta85bcgc7Bd0eol682fvuOIyKXb
         oGN9AQgjPIT1cuya8CNSUO6ewA8wIIx1S6ddlygwtY5rnsT0/XU5dgvmZ4rXCPhVyDNO
         dJfg==
X-Gm-Message-State: AOAM5336cDswAuoT2+cXurYnBuKFRB5uSU6iTBNN82w8xUjTG7S8klZb
	7KAalwEP9YUTLqJmZghvFjmgg5GdueOW5Q==
X-Google-Smtp-Source: ABdhPJxibM+BSNB4l/nZ5IbtJMeL7ldlPn+n0Q/3bzv1XqLCi8tXpVNxScA7/exJ1Svv6KIpcb+xUQ==
X-Received: by 2002:a2e:97ce:: with SMTP id m14mr27362812ljj.73.1632725905476;
        Sun, 26 Sep 2021 23:58:25 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: [PATCH v4 2/2] xen-pciback: allow compiling on other archs than x86
Date: Mon, 27 Sep 2021 09:58:22 +0300
Message-Id: <20210927065822.350973-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210927065822.350973-1-andr2000@gmail.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen-pciback driver was designed to be built for x86 only. But it
can also be used by other architectures, e.g. Arm.
Re-structure the driver in a way that it can be built for other
platforms as well.

Add new configuration option CONFIG_XEN_PCIDEV_STUB, so the driver
can be limited in its functionality, e.g. no support for
para-virtualised scenario when it acts as PCI backend.
x86 platform will continue using CONFIG_XEN_PCIDEV_BACKEND
for the fully featured backend driver.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>

---
Since v3:
 - Fix >>32 shift for 32-bit architectures
 - Move CONFIG_XEN_PCIDEV_STUB into this patch
Since v2:
 - swap the patch order
Since v1:
 - Do not move pci_xen_initial_domain as it is x86 specific
---
 arch/x86/include/asm/xen/pci.h              | 16 -----
 arch/x86/pci/xen.c                          | 74 +-------------------
 drivers/xen/Kconfig                         | 20 ++++++
 drivers/xen/pci.c                           | 75 +++++++++++++++++++++
 drivers/xen/xen-pciback/conf_space_header.c |  8 ++-
 drivers/xen/xen-pciback/pci_stub.c          |  3 +-
 drivers/xen/xen-pciback/xenbus.c            |  2 +-
 include/xen/pci.h                           | 28 ++++++++
 8 files changed, 133 insertions(+), 93 deletions(-)
 create mode 100644 include/xen/pci.h

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 3506d8c598c1..4b08c2b809ea 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -16,27 +16,11 @@ static inline int pci_xen_hvm_init(void)
 #endif
 #if defined(CONFIG_XEN_DOM0)
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
 	return -1;
 }
-static inline int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-static inline int xen_register_device_domain_owner(struct pci_dev *dev,
-						   uint16_t domain)
-{
-	return -1;
-}
-static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
 #endif
 
 #if defined(CONFIG_PCI_MSI)
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 3d41a09c2c14..4a45b0bf9ae4 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -23,6 +23,7 @@
 
 #include <xen/features.h>
 #include <xen/events.h>
+#include <xen/pci.h>
 #include <asm/xen/pci.h>
 #include <asm/xen/cpuid.h>
 #include <asm/apic.h>
@@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
-
-struct xen_device_domain_owner {
-	domid_t domain;
-	struct pci_dev *dev;
-	struct list_head list;
-};
-
-static DEFINE_SPINLOCK(dev_domain_list_spinlock);
-static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
-
-static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	list_for_each_entry(owner, &dev_domain_list, list) {
-		if (owner->dev == dev)
-			return owner;
-	}
-	return NULL;
-}
-
-int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-	int domain = -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (owner)
-		domain = owner->domain;
-	spin_unlock(&dev_domain_list_spinlock);
-	return domain;
-}
-EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
-
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
-{
-	struct xen_device_domain_owner *owner;
-
-	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
-	if (!owner)
-		return -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	if (find_device(dev)) {
-		spin_unlock(&dev_domain_list_spinlock);
-		kfree(owner);
-		return -EEXIST;
-	}
-	owner->domain = domain;
-	owner->dev = dev;
-	list_add_tail(&owner->list, &dev_domain_list);
-	spin_unlock(&dev_domain_list_spinlock);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
-
-int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (!owner) {
-		spin_unlock(&dev_domain_list_spinlock);
-		return -ENODEV;
-	}
-	list_del(&owner->list);
-	spin_unlock(&dev_domain_list_spinlock);
-	kfree(owner);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
 #endif
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 4efc95ddda41..75b9330dc6fd 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -184,6 +184,26 @@ config SWIOTLB_XEN
 config XEN_PCI_STUB
 	bool
 
+config XEN_PCIDEV_STUB
+	tristate "Xen PCI-device stub driver"
+	depends on PCI && !X86 && XEN
+	depends on XEN_BACKEND
+	select XEN_PCI_STUB
+	default m
+	help
+	  The PCI device stub driver provides limited version of the PCI
+	  device backend driver without para-virtualized support for guests.
+	  If you select this to be a module, you will need to make sure no
+	  other driver has bound to the device(s) you want to make visible to
+	  other guests.
+
+	  The "hide" parameter (only applicable if backend driver is compiled
+	  into the kernel) allows you to bind the PCI devices to this module
+	  from the default device drivers. The argument is the list of PCI BDFs:
+	  xen-pciback.hide=(03:00.0)(04:00.0)
+
+	  If in doubt, say m.
+
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
 	depends on PCI && X86 && XEN
diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
index 224df03ce42e..fc8c1249d49f 100644
--- a/drivers/xen/pci.c
+++ b/drivers/xen/pci.c
@@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_XEN_DOM0
+struct xen_device_domain_owner {
+	domid_t domain;
+	struct pci_dev *dev;
+	struct list_head list;
+};
+
+static DEFINE_SPINLOCK(dev_domain_list_spinlock);
+static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
+
+static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	list_for_each_entry(owner, &dev_domain_list, list) {
+		if (owner->dev == dev)
+			return owner;
+	}
+	return NULL;
+}
+
+int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+	int domain = -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (owner)
+		domain = owner->domain;
+	spin_unlock(&dev_domain_list_spinlock);
+	return domain;
+}
+EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
+
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
+{
+	struct xen_device_domain_owner *owner;
+
+	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
+	if (!owner)
+		return -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	if (find_device(dev)) {
+		spin_unlock(&dev_domain_list_spinlock);
+		kfree(owner);
+		return -EEXIST;
+	}
+	owner->domain = domain;
+	owner->dev = dev;
+	list_add_tail(&owner->list, &dev_domain_list);
+	spin_unlock(&dev_domain_list_spinlock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
+
+int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (!owner) {
+		spin_unlock(&dev_domain_list_spinlock);
+		return -ENODEV;
+	}
+	list_del(&owner->list);
+	spin_unlock(&dev_domain_list_spinlock);
+	kfree(owner);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
+#endif
diff --git a/drivers/xen/xen-pciback/conf_space_header.c b/drivers/xen/xen-pciback/conf_space_header.c
index ac45cdc38e85..981435103af1 100644
--- a/drivers/xen/xen-pciback/conf_space_header.c
+++ b/drivers/xen/xen-pciback/conf_space_header.c
@@ -236,8 +236,12 @@ static void *bar_init(struct pci_dev *dev, int offset)
 	else {
 		pos = (offset - PCI_BASE_ADDRESS_0) / 4;
 		if (pos && (res[pos - 1].flags & IORESOURCE_MEM_64)) {
-			bar->val = res[pos - 1].start >> 32;
-			bar->len_val = -resource_size(&res[pos - 1]) >> 32;
+			/*
+			 * Use ">> 16 >> 16" instead of direct ">> 32" shift
+			 * to avoid warnings on 32-bit architectures.
+			 */
+			bar->val = res[pos - 1].start >> 16 >> 16;
+			bar->len_val = -resource_size(&res[pos - 1]) >> 16 >> 16;
 			return bar;
 		}
 	}
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index f8e4faa96ad6..bba527620507 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -19,7 +19,8 @@
 #include <linux/sched.h>
 #include <linux/atomic.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
+#include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index f8ba2903a3ff..bde63ef677b8 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -14,7 +14,7 @@
 #include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
diff --git a/include/xen/pci.h b/include/xen/pci.h
new file mode 100644
index 000000000000..b8337cf85fd1
--- /dev/null
+++ b/include/xen/pci.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __XEN_PCI_H__
+#define __XEN_PCI_H__
+
+#if defined(CONFIG_XEN_DOM0)
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
+static inline int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+
+static inline int xen_register_device_domain_owner(struct pci_dev *dev,
+						   uint16_t domain)
+{
+	return -1;
+}
+
+static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+#endif
+
+#endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:13:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:13:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196364.349175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkp3-0004NN-PO; Mon, 27 Sep 2021 07:13:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196364.349175; Mon, 27 Sep 2021 07:13:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkp3-0004NG-LI; Mon, 27 Sep 2021 07:13:09 +0000
Received: by outflank-mailman (input) for mailman id 196364;
 Mon, 27 Sep 2021 07:13:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUkp2-0004N9-Gu
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:13:08 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5cf16c6d-1f62-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:13:07 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R5wd0B000498; 
 Mon, 27 Sep 2021 07:13:05 GMT
Received: from eur03-db5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2054.outbound.protection.outlook.com [104.47.10.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3batvnhbv8-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 07:13:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4883.eurprd03.prod.outlook.com (2603:10a6:208:103::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Mon, 27 Sep
 2021 07:13:02 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:13:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cf16c6d-1f62-11ec-bc42-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e6g6/hO7JxWdox9vQxSFiAOFGXLDyACpYzgGV/eoG8KeTaK8ImdaeoC1MmESXx1xT9TktfgRf0z5J2l08XFf/LdGoiCPUbbUSM7XxRF7gwI9VZ/2Cg14uyruATXtLu/LHOb8dXqkcy2srIHegOBC1c/Fq7vglColIOR7ePpmCHFk2y1ra80sQBI9unxRziPHdGAU0BNRWSfWUaJwUTIy3aeWNEq4UqsuPZA3011NiSrp1A4wofk9FPU/KoqZ+J/5JZTF0i5mVA0Me4UcTPE6usUHqktAvRf3Ubn6183Gxo5OG6z5CNLq7KhPGXw2zwbmnc3GXWpjylN16sY5jBtEfw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Y+ESMEu4hP8xXjB/6BY/qKC3yxOu5l7lpWUzJGOCDDk=;
 b=bLEwixzeGxse3Sksu7cpxpxeo6srRzKXmzHX4nBube7XFqsS0DlSK1C6ZMevrZid/cP7BTWrXA2My3Lb0Ksb1kL4gCZxaGcTnWZc2V8dok8iyYSaoNlhKGzO15xTKZM7W9SsC6ajsODfmz5NPAehpl/jw7hPGuaQH4AbNMswkhn79AsC8R6BxGpZXipWmsnDcHILuDyY7HY4XPZJA5GrS8ejowji94EDOmvmGHJ/9HyJK5Bc+q8QkmeWKo18HQq93dkCcW0J2bLRyplMfQyZgxcP7/uMf8vGeibDxMcsXfe1tvC9td7xmrDtY3fW0apmFOX1oienMgIOBZx1bLTbsA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y+ESMEu4hP8xXjB/6BY/qKC3yxOu5l7lpWUzJGOCDDk=;
 b=6ORVfs8nQvgw369C6mKiLqFlroARXfol8/Zv9jlNGRIY04iR7B4Kzf0zZnLu1O3VjP5gWzZ5WXKG8KKhh11tTwUQmX4vXI/Z6MeChh7XUkqDDqyjdjVvRG4cOLxWhgkx1KiB1Ws5Gsnoi0WjwDTB+kgfj/zMZX4/S/dn7EcIrhMb7HOZIv+6hOyMRREOVT0es+J6WDPMWIMrF23XbUTx3FiRQ3Mc2kY1RmOHx0Sqvxp1A18eG2197QKE4/ltg+Xh30lHopKWlFseQToRVW5PTrpYLq/iw04OOMZDWNZk3SEQGZmXvZGqtnF50Vl90rOprkBadBgrK0SvR783Qpx5zg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: AQHXsHoyHtx3VKSfZEKPQekyrs7GEauz3gOAgAOfP4A=
Date: Mon, 27 Sep 2021 07:13:02 +0000
Message-ID: <bf937f7d-4d7f-501a-2763-8b7a3bf72804@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241653510.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241653510.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 977131c9-b71e-4fd8-b46a-08d981863e7e
x-ms-traffictypediagnostic: AM0PR03MB4883:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4883AF021EDC72189ABAD9F0E7A79@AM0PR03MB4883.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:2449;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Yq7QpCu3JR9cc9yauSbKNbA5fLpMFCHgeu6FjpeboCImcusAARCUsZzENFIpCzhPfJv4qAPxiGUjNbvQUcwk84A6mJsUXKM8IYW3WmRxw4D79+3jOGK6OPYEljcxx9xztXG0xngynMKqkjalkucYaWklYpzmhuQjelHp7TQ/fe6vh9miMzyNRcl7U5xcrvKpY2bOiVL4KdV+3ryMzwEiuknEiXmdo+JbxSmyf7MFidpWPjwl0szB7mjP47z07nhun2fafnLlu9G7qHf0VJAXNAsLwK7FbifNusA0sfqj+NPNnr9rKm7fqZnECNS2JNt5aVqcjWY365dFtcKOQpRseEYjO632QA+S9oia+WneFkOtLfPCb0ulcBU7rbLJgI/WHWt2yYLKt2Pq6DPU4t5GgyNLUiC2loHiwzb3duZtSv9a2G1BdS9aUbM0G1LOKnIdR1xzJq95z3xxFCCI4GvjzXJiSHRB2FnjbugxNa36o6GhR9lF//2kojKOqCTscdRZHqyJTXPDAlhd5cEP1MqI1VqWpcC0/V3M3rNVfuaDW9r5G9B2yGUDh+3rPavJ9bx+dJAc1UJAklUAjCwifbw4hyksgElnEuLBYMnBe9yCcvI6KO0punBGgl7JvV2L42KmjFxlFSa2lWxYIYFlclEdH3J9O+3/XhPaN1NxKPTgn7AbWeBehhha/eQ4wjUMqDOcdx6PH2GkaGHe5lWcRgDJ5TbHy1f95hcGKKjLCJx4QwQKDaIJ7xrmcs/ebClU38R4
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(91956017)(122000001)(38100700002)(6916009)(2616005)(26005)(76116006)(8676002)(64756008)(66946007)(71200400001)(508600001)(316002)(186003)(66476007)(2906002)(38070700005)(6486002)(66446008)(66556008)(6512007)(55236004)(86362001)(6506007)(53546011)(5660300002)(107886003)(36756003)(31696002)(4326008)(31686004)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ZXk5cmtOeklLT25hRHhzUUlzbXhISWZjRy9TeGc0NEpqVjBrKzhLRXVHMHlP?=
 =?utf-8?B?NEU3RkFnaVprNkdFQktubXBURGloTlgyTzc0eUFuSmRPU0grRHRTUmFGS0hk?=
 =?utf-8?B?Uys0MEJNakRjRytBdUF4ZGV2cmR6WEw3cDVWZkFLUHFxeFArOXo2QkgxL2R3?=
 =?utf-8?B?VWRzdU5OM3luVlFub1p3d0hZSUY5THVqSjVFR3Nkc00zWVRvdmFwbk5leEpQ?=
 =?utf-8?B?TGg1LzZsTkNCTnJqa0UzUkozcnJRSUZwSFdLekxPY3VDak1ONnYrVTdRb3I5?=
 =?utf-8?B?QkxXNjRCVXhlTjNDYy8wTnNOdWN2UjUvY0phSEt1RUdjR2g3UXgvSFFvbTlj?=
 =?utf-8?B?STRJRkN2SXZUZElQT1BzSlVsdzZ0L1FFYzN5N1VwNE5USFlackpMUUtzajVQ?=
 =?utf-8?B?VlIyYkVRRXpJZHhXaFNjV2hiOUZNVnllNFpQTkJQZUtsVWh4WWdxZEVqV1Y3?=
 =?utf-8?B?TUhJSEYra2pNWXVMc1g4TkUvamVzNFF1UHY4cGRjYmFuK3kxbmNPMHFOOU93?=
 =?utf-8?B?R09DVjIrTUMycENGd1BFZ0toN0piMGpRdDNGeXpnUGJQaXRrMFg0dTI4VkdR?=
 =?utf-8?B?bjNNNndqM09oNHZ4S1NRZmNMZlVna1dPbjNHS0ovNW1ERlMza25Ed2xaTXlu?=
 =?utf-8?B?bFdvZlhrY2JWWUJMVmJ2TmRaZ3BJMXpUNmlSY1Y5QSsvNk5DUllCR2tlcWRI?=
 =?utf-8?B?SmMxZktwSUV4Vkl1ckVzTFhPZE5td0NLa0pjQnNLTGRNTDFYWlZtYnlsZHJo?=
 =?utf-8?B?RnZFbHVBMVJjdjBac08yRmdqd1RFYTBOMktMVVZxSVFHUDNuajhGaFVPS0NU?=
 =?utf-8?B?RFpLdzRSeG90cmowTWNmWDBhdENMRjR1UmVuNlBLVXA0Qml4UlZTeFp6b0kw?=
 =?utf-8?B?bzNML2hDN2J5c0FLY1FWRnRxSnZwVCtiQjcrOEt3VExaWFZsbUk0bmVlcEpn?=
 =?utf-8?B?anpRQ25mMFoxaUpDcThacXIwTGNseHh3QmpGQzlvWmJIc3V6TnRlekpNWU9h?=
 =?utf-8?B?cTBDTWNvaVdaMzdYb0txTlpHK3hMZSs1cnJRWGlyQ1lKTlNGZGtwOFF1Szlv?=
 =?utf-8?B?TkxLWUZ0NGxnYnN2clZQMEFNWnN3eXRjczg2dmdHSU5ySjFIbTI0OWROL05X?=
 =?utf-8?B?NzdINGV5ZllqaU5ScTdHVkNjakt4enAvakw2b3B5SnRuck5DQXBVZkcwTjcv?=
 =?utf-8?B?dEF2NEpIMnpWUHhFK2NvY05WMTBLVVE3eHpkMHI0MVkvTDBPUzhmVlZWOEJw?=
 =?utf-8?B?Q2RJM0phdERYZ2o3WXZOOTdQdE9vbUtjWW9NUlZxSlQ4cS9PTU0ydmQveVVn?=
 =?utf-8?B?ZjRUbEFBK3MrMFNibU52NFk1UlpuSGNVbnc5V25NSXE4cGVyQTh5TjVxMmNR?=
 =?utf-8?B?VkZOQ2ljWUhBZFFPa1NEVkVidTRBUlRDcXdiVlhnbk05dE9pRlBucVFsVWJ0?=
 =?utf-8?B?dmpicFJ5Y2t6ektpOWlrTzN4bXRsbWJoVnhPUHdXL1IxOGFscGlQRXBBOFNn?=
 =?utf-8?B?QTNZenZqd2JRR2E4TlNJWUUwbGU0aVBwaTBDMUJnTFNYUmlSL0tTa01yRCtl?=
 =?utf-8?B?WThkVGFVRUtvZnZqMGhRcHRtNkF3bkxqdUtUQXhmTzcvVnBwSVRUQnZkcldN?=
 =?utf-8?B?aW1UR3o4b2gyNnRlMlhwYUJmOGpRQzRZMGdQNHhxRk5CMVB4RFRyQ09rekJZ?=
 =?utf-8?B?Qm5Zcm8raXgyL2t4dG9Uc3FVbncvdmV6Nkx5a2ZiYWFaTlU1RXlQQU9WZEU5?=
 =?utf-8?B?Z0xpbEhyUFc5RVJrWU1vaUl5R3NnM21HRndlTFUwRWFSRk5PQXdZaWFiMko5?=
 =?utf-8?B?RXRUbnVjSDVTYW1QWlRmdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <EAF197FB324E6D40B0414424585E0040@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 977131c9-b71e-4fd8-b46a-08d981863e7e
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 07:13:02.2752
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: YA4dSOIElwI6ZJEzE+lbclLFB9Kjfn/s36kcGj+YM/h+vW2brAqVMrsnz6cn/rCKdeOktBQ/fqDTNphF245lkhnoRrTNaar03ryOrxk+JQsxaso/JJryl5x1B6zYfkvt
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4883
X-Proofpoint-GUID: VpeoV-92FZEyV2FM-992j8lofK9H1mLQ
X-Proofpoint-ORIG-GUID: VpeoV-92FZEyV2FM-992j8lofK9H1mLQ
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0
 mlxscore=0 impostorscore=0 spamscore=0 suspectscore=0 priorityscore=1501
 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 clxscore=1015
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109270047

DQpPbiAyNS4wOS4yMSAwMjo1NCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBXaGlsZSBhZGRpbmcgYSBQQ0kgZGV2aWNlIG1hcmsgaXQgYXMgc3VjaCwgc28gb3Ro
ZXIgZnJhbWV3b3Jrcw0KPj4gY2FuIGRpc3Rpbmd1aXNoIGl0IGZvcm0gRFQgZGV2aWNlcy4NCj4g
ICAgICAgICAgICAgICAgICAgICAgIF4gZnJvbQ0KV2lsbCBmaXgNCj4NCj4+IFNpZ25lZC1vZmYt
Ynk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFt
LmNvbT4NCj4gUmV2aWV3ZWQtYnk6IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2Vy
bmVsLm9yZz4NCj4NCj4NCj4+IC0tLQ0KPj4gU2luY2UgdjE6DQo+PiAgIC0gTW92ZWQgdGhlIGFz
c2lnbm1lbnQgZnJvbSBpb21tdV9hZGRfZGV2aWNlIHRvIGFsbG9jX3BkZXYNCj4+IC0tLQ0KPj4g
ICB4ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYyB8IDMgKysrDQo+PiAgIDEgZmlsZSBjaGFu
Z2VkLCAzIGluc2VydGlvbnMoKykNCj4+DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2RyaXZlcnMvcGFz
c3Rocm91Z2gvcGNpLmMgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gaW5kZXgg
NjMzZTg5YWMxMzExLi5mYzM0NjliYzEyZGMgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vZHJpdmVycy9w
YXNzdGhyb3VnaC9wY2kuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMN
Cj4+IEBAIC0zMjgsNiArMzI4LDkgQEAgc3RhdGljIHN0cnVjdCBwY2lfZGV2ICphbGxvY19wZGV2
KHN0cnVjdCBwY2lfc2VnICpwc2VnLCB1OCBidXMsIHU4IGRldmZuKQ0KPj4gICAgICAgKigodTgq
KSAmcGRldi0+YnVzKSA9IGJ1czsNCj4+ICAgICAgICooKHU4KikgJnBkZXYtPmRldmZuKSA9IGRl
dmZuOw0KPj4gICAgICAgcGRldi0+ZG9tYWluID0gTlVMTDsNCj4+ICsjaWZkZWYgQ09ORklHX0FS
TQ0KPj4gKyAgICBwY2lfdG9fZGV2KHBkZXYpLT50eXBlID0gREVWX1BDSTsNCj4+ICsjZW5kaWYN
Cj4+ICAgDQo+PiAgICAgICByYyA9IHBkZXZfbXNpX2luaXQocGRldik7DQo+PiAgICAgICBpZiAo
IHJjICkNCj4+IC0tIA0KPj4gMi4yNS4xDQo+Pg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:17:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:17:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196373.349185 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUktI-0005L9-Ff; Mon, 27 Sep 2021 07:17:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196373.349185; Mon, 27 Sep 2021 07:17:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUktI-0005L2-Cb; Mon, 27 Sep 2021 07:17:32 +0000
Received: by outflank-mailman (input) for mailman id 196373;
 Mon, 27 Sep 2021 07:17:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUktH-0005Kv-CS
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:17:31 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f9a85070-1f62-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:17:29 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R5wZwQ021746; 
 Mon, 27 Sep 2021 07:17:27 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bb7x2r8vm-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 07:17:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM4PR0301MB2196.eurprd03.prod.outlook.com (2603:10a6:200:4d::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 27 Sep
 2021 07:17:25 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:17:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9a85070-1f62-11ec-bc42-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NAa8K3EHsKHN/X7bEYwkQWe3FDlbaExopjU2vAQ2fpdCmVAzkyh6l1ANMLAV4Cx6sj/YIFaAJ8SwHhJv4Y4X8L+p1ZasPF/TTjURsBPrmTFCQx3bsRk5RctGJ1ADz0tmWqvyTT2gwVfj0/e76nroMXNdyJ+FgGshDLVHg8QL2HpPlvPl3iDc9JYabSFqY0kHQQRjsd9bz3TA6Qn9ofKxYJFyAUU543S0ZugiQWGsQg0UCsfMdJXBeQUPciRFt+DcAh3jL2EzI4DKzkO9/LYmr0Bzzqt9s3wiSza//mjxWJHmi1NUeWcymAcQg4lViQmHWT+vfc+jVQFzuhnLHE6CrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PmU+17d68n/+dmGiPuNg9noukYa/3K2MoitkwNtDdVQ=;
 b=dTw+f0uNDwdljl7BXytdiNO1BDj+GCkJS9ReVhCrOpMW3S2dMjn2vEm1w3vVn+OJeis2NaNMNcbECZllhILjHew0LcI0TUTxMIvPa8L1bVq3eL4up0Vd3Yrf88BKwOn1RjRHNW1SukIDcUHllM8Dhmq0bKE9xgC864xNewmLcTExWCYNA9529zqg7LN4yrqitUpo50+zx6qCCaV86syqRZoKmCuP/AACXHtkkgjbjKniJ6Cih1LnVzYqMLY1w0nlWeb+g/Nwp4fttdeFgJ/XGjyC+F2cotMlT2t05ZuMgtJa0hYwaQJWHFCFuWO7nl9tJqvfYqzCA4USNWqc373Iug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PmU+17d68n/+dmGiPuNg9noukYa/3K2MoitkwNtDdVQ=;
 b=q4Mqa5zXRZYI3yQrEyDvBescQw5n5vnSCWVpAgfHcnRUFHwGiJMltoEm/3QXCX83OHQFS4zkZBUsUcIxL3jy1Fhfd+nMhIFVttv9pwJZ1jTko9DNsejNB8EID5yOmNujJ61noL1BkEO0tmFabU2Ot5Pq3fN3uPJDadnP2OACU4Zxl5jLrQ4wSUmsF5/N2ZewAfpLNrPMbuVpucHaBbFrNzRHxlBKrXBZJ2CkXr1/+GM0/D+eNo+XIH+72GjsQhk03gmtCOPfO+lSMN/RoZmGN6Y3QK4moFL7ydgKOcF13Lh69E4Gz0lx/38VX0gWesgFftdRNk4Er+ZnU80lyttZ+Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        "roger.pau@citrix.com"
	<roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        Rahul Singh
	<rahul.singh@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>,
        Ian Jackson <iwj@xenproject.org>, Juergen
 Gross <jgross@suse.com>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 07/11] libxl: Allow removing PCI devices for all types
 of domains
Thread-Topic: [PATCH v2 07/11] libxl: Allow removing PCI devices for all types
 of domains
Thread-Index: AQHXsHo0SAN7/ZEcjkCS11geMNuE5quz35wAgAOe3wA=
Date: Mon, 27 Sep 2021 07:17:25 +0000
Message-ID: <f08a2fda-4148-d439-c4cd-575be8922764@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-8-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241658350.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241658350.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: eb31dc30-dfda-4b1c-9d20-08d98186db35
x-ms-traffictypediagnostic: AM4PR0301MB2196:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM4PR0301MB219617D803C0D3DAA40B9EDBE7A79@AM4PR0301MB2196.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 cPrDYLOfkq/JzPSlgA0rGkHvcgmRLruyidqMBqC9ArEtqnWsTp1lDPMssoPjQpapDPntkafXy6j2B0BNF+eJOvUjZqFrOErSv4Vt9x00WFRGKcVcHD//etObqZRwEcHkXMJDRtsucBa6yaZt9Cd49ib+j1L3ADWNVBc4iS0i8VDjcQt5zUNkKbUpgytcwIbQGQ/IMTlwInedCiJP6GEFcCN28XrUqu05dYRW5zYwJKgx0QoXjMtLor/WykEZeeohXz74yKP0l5snptA2RsflSqO8xqNMGrTQAHWnx4Bb34sEcjvlfhx4QLWT3JBgJ3YMcqj7TD3+U/lOtq1775XQ+mg91us3pzctpgoezjEYxhjwvsFC2+rUflbV35HGZoBZ+CGPZ0HB/iIt1e0kQMeJ4oK+JFE+wN3oM4FXND9ueOh5GTZpRO+lZDmSKo+JrUxc80ru0tpdHZ7Ks09579THLz8BOU5bW8Yj5PsVH/KjNkVveP0wFzuF5UpseHBUMKTNtdjfTDfT3ixHqLSed1Ewz6gKv4LOkpZdBpY05HWHImswHZbYbIInTFaRzjIoSWck88TtFinEO8CsOtLOYQ3CjNj3E7zP1cQUcCowQvibiChn64no3Br1ncytiX+mVgrOizhzqo9wK33s+q2gpqnfWqmN/hup73chQvGaVSb/iT65Dyp5FATGqPDbBPfpIhfjGabI65Gk1gPXy+agM4YVZmgg729bAsOn9lHEUtKgPiurLjq0hvMcoLvt1geiUQ28
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(66446008)(110136005)(38100700002)(71200400001)(2616005)(7416002)(6506007)(5660300002)(31686004)(6486002)(66946007)(86362001)(66556008)(31696002)(8676002)(64756008)(66476007)(53546011)(316002)(76116006)(4326008)(36756003)(2906002)(26005)(508600001)(54906003)(38070700005)(6512007)(186003)(8936002)(83380400001)(122000001)(55236004)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dHcxT25GSGVFM29Vakg4ODNMZTNsU3hENmkvdXBNUDl4Z09ybFQ0QmtMYkpi?=
 =?utf-8?B?Vk5FM0NkVENZYTh5SHF5ZnRTYms5MFFZbUFZV0JDOHdmWUFWb0lxUVo3bFFL?=
 =?utf-8?B?MkNqcnlUVHBoajVUWTUvbjRiK1RWVXlodjV3R1NsK2Mycy9nYmhnT0tybWdD?=
 =?utf-8?B?OXNpTmorWVZSdHdUbnlLQXhjaTF0cXlCcU4yaWhMMSs3WFp0eUU1aEQzUkR0?=
 =?utf-8?B?d2xVY1BxTGxuaFdWZUd2eHZRZmhWR1VPRXNDU2JVV1k5VDVHN003Q2pHTTdq?=
 =?utf-8?B?WWx1eS9mY2UxS0IvcnBUNCtueU4yNitIWVlFZi80M2tNWmdkUHZ4NGdFZWdt?=
 =?utf-8?B?Skk0cHk0QndxdTh3eHZqWW9pdkZvbzFKMVZtZHZIRmZKcWYxTkZ0aHBhcXVY?=
 =?utf-8?B?dHBJZ25TQWlmMFJaa2VYcS92Yng5bjZVRkoxNnJhaTJsUnRkM0kybmFYL2lU?=
 =?utf-8?B?a1BIa1BBRjR5QmpMN0ZaNHBCTU5xT0RuNEpqR3oydGdoMnFVZE0zQlEwUG5T?=
 =?utf-8?B?TGZEcjhoM1VVZklmRHFaT1ViTVcxNmJqZG9mUGUrY2hnNkp2TDdiaXhtRmxS?=
 =?utf-8?B?K1RUMTlZcmhyVm5GSGZxUzl5RFdFWkZpd0htUXgzQ1VQQXkrdTMzVkFwMzVG?=
 =?utf-8?B?WGtvMU9MbG9ZbktxSUFyN0ZPZG14WjlaaWtpcWRnTjJFSnJjQURwRlBzNGJa?=
 =?utf-8?B?aXB4VE5pWUF5aXBraE5NNXJ2VjZrcjNaU2VWSjY2M3FJelVNOEJGcitkTXlX?=
 =?utf-8?B?a0NEd1N5NC8yZ2hsWFdwV1Z2WFEvZ1ZUQU9YSi9tL0J5c0htUjZmOG9YeTg4?=
 =?utf-8?B?WHBkdUthK2FMNGRoRjF3bStFOXJwaDRrck9tcFpzM2xPSHJrWWk0a1dBVE9Z?=
 =?utf-8?B?U2txNStBN1JTY2JZVTJPWk5UUVNhRG9pTEZkdVZQYk8zaTdQVGl6VVZWNnI0?=
 =?utf-8?B?elpaYUp4NlpxdXRoRXpkQmtjSUVZY09GNVZ3Qkg2NHVBL05pTnlzdkY5d0pD?=
 =?utf-8?B?cWNYR3N1QStpcnVmbnczQUE0bjkza3RlaksrY0VBOUp5SUVjc3RqTlNncmUr?=
 =?utf-8?B?TEl5KzlRZjFzcENQdjl0VnFRTEtkc1loSzVkT3hZZFVvcWI2Yk9JV0sxc2dP?=
 =?utf-8?B?NzBVNDBNd1EzTnVOTy9OY0MwTUpqVkk2Qzg2amV0TDRBZ3o0b0pkU2V0NzRr?=
 =?utf-8?B?c1pzY0wrOG02TjYrNGNodE5lN2hqZnZ4eWNtdThJSEFlalVOWVMvRFdhem9i?=
 =?utf-8?B?eG0wRG96QlFWL0tkbVQwYUZscldtYmxUVnlManVic3NCNmNScG5Gc2NFc29R?=
 =?utf-8?B?L01CRENISjN4RE0zdkEwMDU3a255L1dDQ3d1S1dZS1Z4WnVhazhMaThPWExP?=
 =?utf-8?B?UzNvb2NBc2dYdURqRDFRYWU2bHZqeGs5ZVQ1Zi9XSnZJbWtpVytvUThLZ0R6?=
 =?utf-8?B?Ymp2cXhFdHFseVFEekd1cEltbzlvc0hzNjFHU0VPYTFBTWhoOGw3ZkJJRDhx?=
 =?utf-8?B?WmkrQVdkWGp5eTdoc1BTQ3FnVy9aTXBCK1FQbFQvMGYwVjFtUmNtWCsxeEdm?=
 =?utf-8?B?M052bE9rYkVVbTRxVHFIS2R0MHBLMkprMlBuQWVSazRwTVlPcElnSWhRUERM?=
 =?utf-8?B?NTFjajVJV1FzWlVzQVM3cWIzRjlxNDJQYUxUcnUwQXlkQklxTzhPZ1NicGNi?=
 =?utf-8?B?LzVOKzI2L1pSeDZVVUh3NHliUlBUbUJ0MndFT2VCTjgrejh5RStRbEpQM2pR?=
 =?utf-8?B?R2Jxc1kzb1pINlVEZGZJNHk5YmpIaW9PV3VBV2Q3N1JTeTc1S2tmYll4dDg2?=
 =?utf-8?B?ekIrWnQ0S001UDN0R2srZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <2E6F13F6B3EA124B9273677FFD4A55AC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eb31dc30-dfda-4b1c-9d20-08d98186db35
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 07:17:25.2134
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6eRBJCp4V6g2G3undPBWDNbDOv90+8Dmak3gi7fBNafnik2mshmerHWPW7x6uG43yCsQVYXi8l0E5zwXAE1pDJxZ1mwjZqfI3QS0jJxaAPmMDE7Oh54VCBeSXEmXafgT
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0301MB2196
X-Proofpoint-GUID: bB9NqoRiobZFn5KSZFlJDZurNcUTEhVB
X-Proofpoint-ORIG-GUID: bB9NqoRiobZFn5KSZFlJDZurNcUTEhVB
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0
 lowpriorityscore=0 adultscore=0 impostorscore=0 spamscore=0 bulkscore=0
 mlxlogscore=999 priorityscore=1501 malwarescore=0 clxscore=1015 mlxscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270048

DQpPbiAyNS4wOS4yMSAwMzowMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiAreDg2IG1h
aW50YWluZXJzDQo+DQo+IE9uIFRodSwgMjMgU2VwIDIwMjEsIE9sZWtzYW5kciBBbmRydXNoY2hl
bmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9h
bmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+IFRoZSBQQ0kgZGV2aWNlIHJlbW92ZSBwYXRo
IG1heSBub3cgYmUgdXNlZCBieSBQVkggb24gQVJNLCBzbyB0aGUNCj4+IGFzc2VydCBpcyBubyBs
b25nZXIgdmFsaWQuDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVu
a28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4gQ2M6IElhbiBKYWNrc29u
IDxpd2pAeGVucHJvamVjdC5vcmc+DQo+PiBDYzogSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2Uu
Y29tPg0KPj4NCj4+IC0tLQ0KPj4gICB0b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jIHwgMiAt
LQ0KPj4gICAxIGZpbGUgY2hhbmdlZCwgMiBkZWxldGlvbnMoLSkNCj4+DQo+PiBkaWZmIC0tZ2l0
IGEvdG9vbHMvbGlicy9saWdodC9saWJ4bF9wY2kuYyBiL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxf
cGNpLmMNCj4+IGluZGV4IDFhMWMyNjMwODAzYi4uNTlmMzY4NmZjODVlIDEwMDY0NA0KPj4gLS0t
IGEvdG9vbHMvbGlicy9saWdodC9saWJ4bF9wY2kuYw0KPj4gKysrIGIvdG9vbHMvbGlicy9saWdo
dC9saWJ4bF9wY2kuYw0KPj4gQEAgLTE5NDcsOCArMTk0Nyw2IEBAIHN0YXRpYyB2b2lkIGRvX3Bj
aV9yZW1vdmUobGlieGxfX2VnYyAqZWdjLCBwY2lfcmVtb3ZlX3N0YXRlICpwcnMpDQo+PiAgICAg
ICAgICAgICAgIGdvdG8gb3V0X2ZhaWw7DQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgfSBlbHNl
IHsNCj4+IC0gICAgICAgIGFzc2VydCh0eXBlID09IExJQlhMX0RPTUFJTl9UWVBFX1BWKTsNCj4g
VGhpcyBpcyBmaW5lIGZvciBBUk0sIGJ1dCBpcyBpdCBPSyBmcm9tIGFuIHg4NiBwb2ludCBvZiB2
aWV3IGNvbnNpZGVyaW5nDQo+IHRoZSBQVkggaW1wbGljYXRpb25zPw0KDQpMb25nIHRpbWUgYWdv
IEkgd2FzIGFza2luZyBSb2dlciBhYm91dCB0aGF0LiBBdCBmaXJzdCBnbGFuY2UgdGhlIGNoYW5n
ZQ0KDQpzZWVtZWQgdG8gYmUgb2ssIGJ1dCBSb2dlciBjb3VsZCB5b3UgcGxlYXNlIGNvbmZpcm0g
dGhpcz8NCg0KVGhhbmsgeW91IGluIGFkdmFuY2UsDQoNCk9sZWtzYW5kcg0KDQo+DQo+DQo+PiAg
ICAgICAgICAgY2hhciAqc3lzZnNfcGF0aCA9IEdDU1BSSU5URihTWVNGU19QQ0lfREVWIi8iUENJ
X0JERiIvcmVzb3VyY2UiLCBwY2ktPmRvbWFpbiwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4+ICAgICAg
ICAgICBGSUxFICpmID0gZm9wZW4oc3lzZnNfcGF0aCwgInIiKTsNCj4+IC0tIA0KPj4gMi4yNS4x
DQo+Pg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:24:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:24:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196380.349196 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkzd-0006nU-6g; Mon, 27 Sep 2021 07:24:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196380.349196; Mon, 27 Sep 2021 07:24:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUkzd-0006nN-3j; Mon, 27 Sep 2021 07:24:05 +0000
Received: by outflank-mailman (input) for mailman id 196380;
 Mon, 27 Sep 2021 07:24:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUkzc-0006nG-Cr
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:24:04 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e3ef4242-1f63-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:24:03 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 81A73220C3;
 Mon, 27 Sep 2021 07:24:02 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 476C613A1E;
 Mon, 27 Sep 2021 07:24:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id SZo1EJJxUWHRLAAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 07:24:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3ef4242-1f63-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632727442; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=e5hNkRe5fGF2KqqbgdIR0xCVAr+1JiYjbNfzjZzSJTw=;
	b=hUKuE9pY5REBOX0kthxmtgyl9/7Ck7GtZtLoQoKGLvrO/ToqmnGUEHirORklPokmbDPb2K
	75maP3NztR8TI4HLLbtYm2IfIr285EdyaaV2IE6I/5eytohBQBykA1n+NxaemzZg3G1WQY
	dA08SDt6ZHh/4ykHAimUP8rVCfL02m8=
Subject: Re: [PATCH v2] Config: use Mini-OS commit 9f09744aa3e5982 for
 xen-unstable
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20210908125232.20993-1-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <6c655f0e-5e32-6895-1a88-a967c8721bfd@suse.com>
Date: Mon, 27 Sep 2021 09:24:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210908125232.20993-1-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="lFDvP8srNXqFWN2KWsyBY2MVwMLzqD0G2"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--lFDvP8srNXqFWN2KWsyBY2MVwMLzqD0G2
Content-Type: multipart/mixed; boundary="2cMFTwsEcmBUQym1XwQnTGKDkwTKEGSyR";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Message-ID: <6c655f0e-5e32-6895-1a88-a967c8721bfd@suse.com>
Subject: Re: [PATCH v2] Config: use Mini-OS commit 9f09744aa3e5982 for
 xen-unstable
References: <20210908125232.20993-1-jgross@suse.com>
In-Reply-To: <20210908125232.20993-1-jgross@suse.com>

--2cMFTwsEcmBUQym1XwQnTGKDkwTKEGSyR
Content-Type: multipart/mixed;
 boundary="------------99BDCEB4E680E655D90EC18F"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------99BDCEB4E680E655D90EC18F
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

Ping?

On 08.09.21 14:52, Juergen Gross wrote:
> Switch the used Mini-OS commit to 9f09744aa3e5982 in xen-unstable.
>=20
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> V2:
> - use a specific commit instead of master (Ian Jackson, Jan Beulich)
> ---
>   Config.mk | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>=20
> diff --git a/Config.mk b/Config.mk
> index 4d723eec1d..e85bf18654 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?=3D git://xenbits.xen.org/mini=
-os.git
>   endif
>   OVMF_UPSTREAM_REVISION ?=3D b37cfdd2807181aed2fee1e17bd7ec1190db266a
>   QEMU_UPSTREAM_REVISION ?=3D master
> -MINIOS_UPSTREAM_REVISION ?=3D 051b87bb9c19609976fb038f386920e1ce5454c5=

> +MINIOS_UPSTREAM_REVISION ?=3D 9f09744aa3e5982a083ecf8e9cd2123f477081f9=

>  =20
>   SEABIOS_UPSTREAM_REVISION ?=3D rel-1.14.0
>  =20
>=20


--------------99BDCEB4E680E655D90EC18F
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------99BDCEB4E680E655D90EC18F--

--2cMFTwsEcmBUQym1XwQnTGKDkwTKEGSyR--

--lFDvP8srNXqFWN2KWsyBY2MVwMLzqD0G2
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFRcZEFAwAAAAAACgkQsN6d1ii/Ey9r
VAf+MbvZSGfvX2LqIvIHZu9g/BLQHNkhFfgRlXXY7H8yQoxyJySEuLyycvJ+bqINnWNqwEGDPVnE
j9d+2heXZQwzTT2UB3PgdLgirFOBG6V7fi1m76FxBq6MljN+E5wGvODWLRhHCWQNFymPAzx0eL/c
bMB+JkhWmxWirlN41quTAqvH1sGKAlOtvbrJeZAmech/xTGzqjZJBZbB8+O+WrIwcL9f44LqCjeY
Lyg852vpiaO08oIeuz6+7dUPRlywVaChFRi6E6dVbN/i7BzMZTvloX4vXTASxiSbq4logivWnc6R
0Tdf2IXOtJuMRAIcNzNyowCLl5bEqOkPnBbHN84uLQ==
=1HmS
-----END PGP SIGNATURE-----

--lFDvP8srNXqFWN2KWsyBY2MVwMLzqD0G2--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:24:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:24:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196384.349208 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl07-0007K1-HT; Mon, 27 Sep 2021 07:24:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196384.349208; Mon, 27 Sep 2021 07:24:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl07-0007Ju-DB; Mon, 27 Sep 2021 07:24:35 +0000
Received: by outflank-mailman (input) for mailman id 196384;
 Mon, 27 Sep 2021 07:24:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUl05-0007Je-So
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:24:33 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f5ac2d1a-1f63-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:24:33 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 502DF21C13;
 Mon, 27 Sep 2021 07:24:32 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 244F313A1E;
 Mon, 27 Sep 2021 07:24:32 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id sreBB7BxUWECLQAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 07:24:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5ac2d1a-1f63-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632727472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=Nbw1XeualNXHyoC+nXFWaQ8npzmocWj68COAdaf2Tzc=;
	b=kr/UmqoKOSMmsH3A+rnwgbmC51UThC/dOOcp8ZF5c5LZ9lbkeaXQ6gYJRvcbthudi33cHv
	j4G5uKMnNWzANj6rhD4hcyZiiHoBftKXyolv8vVR/h6GJyw7vgeW8dHsHb9vQUsFoeWDy3
	fh9AmhICNCrEP0eLYQ2l7rNnFny04rM=
Subject: Re: [PATCH] tools/libs: fix build of stubdoms
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210908124303.20680-1-jgross@suse.com>
 <7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <adf928b7-850d-963b-155b-983af7733662@suse.com>
Date: Mon, 27 Sep 2021 09:24:31 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ToDBKjXaUdfdt8bNZB23QXrVc1vE0Otjl"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ToDBKjXaUdfdt8bNZB23QXrVc1vE0Otjl
Content-Type: multipart/mixed; boundary="FSrDUMGxnnHMAPL00XRE4fDb0ravOq1QI";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
Message-ID: <adf928b7-850d-963b-155b-983af7733662@suse.com>
Subject: Re: [PATCH] tools/libs: fix build of stubdoms
References: <20210908124303.20680-1-jgross@suse.com>
 <7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>
In-Reply-To: <7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>

--FSrDUMGxnnHMAPL00XRE4fDb0ravOq1QI
Content-Type: multipart/mixed;
 boundary="------------80000EA62B67077F3591BB57"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------80000EA62B67077F3591BB57
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 08.09.21 14:53, Andrew Cooper wrote:
> On 08/09/2021 13:43, Juergen Gross wrote:
>> In case abi-dumper is available the stubdom builds will fail due to a
>> false dependency on dynamic loadable libraries. Fix that.
>>
>> Fixes: d7c9f7a7a3959913b4 ("tools/libs: Write out an ABI analysis when=
 abi-dumper is available")
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>=20
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
>=20

Could this please be committed?


Juergen

--------------80000EA62B67077F3591BB57
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------80000EA62B67077F3591BB57--

--FSrDUMGxnnHMAPL00XRE4fDb0ravOq1QI--

--ToDBKjXaUdfdt8bNZB23QXrVc1vE0Otjl
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFRca8FAwAAAAAACgkQsN6d1ii/Ey+9
Lwf+MtSKbyQeb5Dch5NBqhVXX8arl+ErXeIi9BvRpKqM/J6OK60NHM4x8Zv0QbIS/d5zqf5pA4hA
MnlJ1yPtoI/ac7Og9QtXHg0cp67bjfvTfakRuZodqA9ryb/Oum+/CNU4v39otr3NB1zkrcamH3Az
E2WWkujeUZtliAzGp47eL/7bWK7GdKZpQdV2RQE2nnR03TL52Ts7FmFHbTxzGwWS2d1aFc30txdC
IhiO/wLJ0slJ7hV1ucE00bzi0OsS3ZCjZTSjtvyyp90bBpHuofRuPybF/5F9nV4/Yqse+LU3x2Kk
nBHwxusqJynwDp1czA+NOnVFLJDa6WZhcJn/bOStSA==
=B7ym
-----END PGP SIGNATURE-----

--ToDBKjXaUdfdt8bNZB23QXrVc1vE0Otjl--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:26:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:26:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196394.349219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl1k-000846-3T; Mon, 27 Sep 2021 07:26:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196394.349219; Mon, 27 Sep 2021 07:26:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl1j-00083z-WA; Mon, 27 Sep 2021 07:26:15 +0000
Received: by outflank-mailman (input) for mailman id 196394;
 Mon, 27 Sep 2021 07:26:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUl1h-00083g-Sf
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:26:13 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 30cecf7e-1f64-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:26:12 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9442220092;
 Mon, 27 Sep 2021 07:26:11 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 47F4413A1E;
 Mon, 27 Sep 2021 07:26:11 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id GpjdDxNyUWELLgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 07:26:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30cecf7e-1f64-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632727571; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=XSbfL9aPdFmwEccXXyuzzyqynt1h/SE/bsO1U7HqAfI=;
	b=OFSm0aId9RN7TGShse6n8uOO1PqzOWbxXIzXXVdFSs4Px3n+MrcNHfD/qg/ToqmMgZqf73
	6VBEBxSM2idjdHin9CQQcXPLgbe8NVuckn1wjC28jPBpzYt5urkyIcALtV5LwE4vh1tV7v
	aUOU5fqfhh9lUqUrU5BUu/6Fj2B2DT4=
Subject: Re: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per
 default
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>,
 Community Manager <community.manager@xenproject.org>
References: <20210910055518.562-1-jgross@suse.com>
 <24891.31480.165445.521062@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <d894799e-27c6-bbbd-8d65-3e32c1aff843@suse.com>
Date: Mon, 27 Sep 2021 09:26:10 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24891.31480.165445.521062@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="7heJCTqVYiQRTKvueA9OX6ZKSp0fnMT5v"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--7heJCTqVYiQRTKvueA9OX6ZKSp0fnMT5v
Content-Type: multipart/mixed; boundary="81T9CDpotJZKTYLYKR51RMA775jv4iN8q";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>,
 Community Manager <community.manager@xenproject.org>
Message-ID: <d894799e-27c6-bbbd-8d65-3e32c1aff843@suse.com>
Subject: Re: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per
 default
References: <20210910055518.562-1-jgross@suse.com>
 <24891.31480.165445.521062@mariner.uk.xensource.com>
In-Reply-To: <24891.31480.165445.521062@mariner.uk.xensource.com>

--81T9CDpotJZKTYLYKR51RMA775jv4iN8q
Content-Type: multipart/mixed;
 boundary="------------A4F8299CCD8D5C4D42650031"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------A4F8299CCD8D5C4D42650031
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 10.09.21 17:34, Ian Jackson wrote:
> Juergen Gross writes ("[PATCH v3 0/3] disable building of pv-grub and q=
emu-trad per default"):
>> This is a first step of deprecating pv-grub and qemu-trad including
>> ioemu-stubdom. Switch the default to not building it.
>=20
> This is now fully acked.  But can we wait with committing it until we
> have a decision about whether to (a) have osstest explicitly enable
> the pv-grub and qemu-trad builds (b) have osstest stop testing these
> configurations ?

Any decisions made?

FWIW I'd be fine dropping pv-grub builds and tests in OSStets, but I'd
rather keep testing qemu-trad stubdom tests.


Juergen

--------------A4F8299CCD8D5C4D42650031
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------A4F8299CCD8D5C4D42650031--

--81T9CDpotJZKTYLYKR51RMA775jv4iN8q--

--7heJCTqVYiQRTKvueA9OX6ZKSp0fnMT5v
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFRchIFAwAAAAAACgkQsN6d1ii/Ey+m
wAf/RjbiWA+7JhhrzN6eheEK9tgIk6vteYpOTWA+znVWBGCi3snrkdcpmrdOvS9ioV4ljb0JUnPp
ubkeHKbS4SNvrHkd2Iz6nSnk3xLuPoYHXyvLX6HLYyv0VPgKghwoyr+vz7x0JjuGfu47jqMAMhGT
9Gn3NVsSRbqAISDZzOVgAc8M639rMvLF6/oxoA5vXfjPJtSY27XQG9YdDEnyMjfWvKTsiDqwF7Sa
zJKe/DTip9gqvZrgKNtgM69dTHyf79FlkeTiI8dHosBQ43SN5i+zTdPYToO0Qe45s4R9s7DBIeGA
dd3L8I62vH6DrbXchqHPCrkNNjytS11LQ4AKUAMUOw==
=byM1
-----END PGP SIGNATURE-----

--7heJCTqVYiQRTKvueA9OX6ZKSp0fnMT5v--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:26:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:26:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196399.349230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl2P-0000Ck-Dd; Mon, 27 Sep 2021 07:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196399.349230; Mon, 27 Sep 2021 07:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUl2P-0000Cd-AP; Mon, 27 Sep 2021 07:26:57 +0000
Received: by outflank-mailman (input) for mailman id 196399;
 Mon, 27 Sep 2021 07:26:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUl2O-0000CV-SQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:26:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e9f79e8c-ecfa-4109-982b-ac9be4c913b3;
 Mon, 27 Sep 2021 07:26:55 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-8-ugGp8uFDPgK88etsL4MnpA-1; Mon, 27 Sep 2021 09:26:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5328.eurprd04.prod.outlook.com (2603:10a6:803:59::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Mon, 27 Sep
 2021 07:26:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:26:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0106.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:19::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 07:26:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9f79e8c-ecfa-4109-982b-ac9be4c913b3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632727614;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6xjuVATs6D3wjKZlrb9lrWmJ+kcuNY9mgUVnAPrK/gA=;
	b=AWpv1BESMjaLcnvn81vXLye/gG234lrhHUFnGttiBPoyboVA3XZ1guDUxhBVd3lC/M25pR
	UFg8aVPP1G+IdceLegBsjRMNGG2BWow2Zs2fSIKn7JRY/aGz+F5IWEegnxjYXe09PdhrFh
	qcX352bLeDnJKrLOlhzwfD5ZX49tWto=
X-MC-Unique: ugGp8uFDPgK88etsL4MnpA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YdjgnNbZQtCYaQTtccvOF8N0+j1Aq3ojlJ/xes+QISBe8I8t2piGgjtDVC+46JzIjrspbeQg6Uxkv6aAW1+3H5+fhGXsc1GssuJKu++PnTJbkyV5Fug9joOZYcZtaXAnapI9n7CekSqcu4kvCC7EKcYvP5vRmk3ahoBglfoKnoMZuua+ahKScujHVZ2EJ5Ib0yIHJkbdtLqOhJDhpRwWRVBFI6Km9Su6+M9GZj/ZGuqO/V9J8GI9Ix1YwPNAdWUpXqZVnB9IWk5bEJKfw4930AB7KSZUFhyuSt0KjCrb6oKgzFKh3xabZu3C0luvc/1ic5tOCL/Q+ERm7T+qOBb/Wg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6xjuVATs6D3wjKZlrb9lrWmJ+kcuNY9mgUVnAPrK/gA=;
 b=nD5tmP/yiIAROEkm+oqa6X4rMSKvpdNXzCtI+tvyjtBpfllavj630D3D/stXQR57bsTwEuLE3Qnoy7FMbpZAQKudosDr1/Bli0mk3NnWf2+mGOQ1+xzPisktbQ4E1E9AjLrWTlLBOnYmlUW8qrL2SwFa/rQ23caoTYJLtnY4Woplx+AHb0nvZiDF2PsIh+YsfJcbrFVZXoVVaMLD3CfNMo8Ky1G1qWFRMK2JMqvCOwGcgs2DPkBi+Pzr/WMnmuKIOGzM2Muxbz8uRtI1cGdTzkmeDAA6HDN9u1hlBh0fp4ocfNLpmnbrkHB+unfi+3/uzICghB6hrE4eVqgbeMfbsA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: boris.ostrovsky@oracle.com, jgross@suse.com, julien@xen.org,
 sstabellini@kernel.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20210927065822.350973-1-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
Date: Mon, 27 Sep 2021 09:26:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210927065822.350973-1-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0106.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:19::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9038d1b7-e2e0-4cad-e8c4-08d981882cbb
X-MS-TrafficTypeDiagnostic: VI1PR04MB5328:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5328D591A10AD0F8C815F2B6B3A79@VI1PR04MB5328.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+vMYmP6i3vrL3r+WCsbUkwTo23NUZz0tMMKJOMG1lcOzFEZvOKRMa2/VI25NGN4TB6hyujfutFl4Z+oJqFbLSEr4gu5BaijlJRnowJ4IXQmMOqSMLFUAEYnJJsUOclGG6K90bwu4xqkJVSxvhDm2/K8T726FvkkvThtnoRjmA00Qdw86H3kMZEKMP9XuMCAiANqfOQ5l7a+0Dav5PmQ6y4+Ou9C2E6rSiVMVdpvBP+SMDCfFHgTlYfFOrT2eXVPBHcxW9BOajTEtbKxy8Gpddvj8lCKqD8WrC0lx8oqqOopvGd/RA149jDxHLitMHXD+lshrU7KyYjn5NujgMsj7M65z6le4ExnXzhrdJAi+1nT15egT0/7s/ZqOooQz77hrlVQ8tOf4OvXsm4LuSB75z7bIGtJf6783I6Dr2IUe0cKTioWGcRIU4r8zxj0ZZX3oqI3FBgvzoHx9h5Xydz+Hvfyu7bSdVZlf4fvH+sunHvp1/p/I6u51IY3jt0kdyiWWPKZwxkjhOgS5aNmo3s/6N8bJWUfjbQijUraEouhgAyfUTetyHEcjsmztuKAfgBhjmtGULUW9unK2fL9XG6XlKxQI/op6s/8dcrBdd1ixEDrZh2cGlF6N3GG6ArAeSX+uwBIUnbULv0JzhZZaNQyQnc4TdLABm4RZbYGYaP1RV2z6LnHKlNHHdeCfNV8SY3eBKiLyn/jHSUdK7OUO+wFbRi8A6dn225lf8JpZWsAZ+so=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(66946007)(53546011)(316002)(16576012)(8936002)(2906002)(4326008)(36756003)(66476007)(26005)(186003)(31696002)(2616005)(956004)(66556008)(86362001)(38100700002)(5660300002)(508600001)(6916009)(6486002)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eWJUTzBDb2VMNTBBMk1QcG04THR4T2d1TmRHTHVRbVhZSk9CSDUxOUpRVVNT?=
 =?utf-8?B?c2t2MnlnV20yVXdleW9QV3A1TFNKNmhYejRmbEUvaThGaWJxTzk0RUErblJT?=
 =?utf-8?B?ZHl6VjdycGFIWjFuVThGZm1SR1V2L1VqZmZCcVIvTFV1Tkw5UUZrcnFibUlp?=
 =?utf-8?B?elZDTlQycytaeGhpT29GTTVyM3NYcXhSU2Rka0hTUGYvb2lLZ2d5aFdHemNQ?=
 =?utf-8?B?N1lDUEtqbTIrVEFITzdpS0hMbUg3SWx2ZGI3bHlRb2l5LzIrenF4RE45UG0r?=
 =?utf-8?B?NDdFdlovMU00Mi9TUWJXRXMxakNnWHNNdWxQVmphMlB1RkZTbkJHVkltMCtF?=
 =?utf-8?B?QWtFRmhPcWsrRWdWSERocHlTSGVZTzlFNCsrZkR2MjloRHVjWG5wRmtPWWlk?=
 =?utf-8?B?YnFYSzdMZ3A1NGZXbFdHK3RvR0E4bmhxN0VOdnlqS1h5ckxLbk0xSUhDcldq?=
 =?utf-8?B?RUZYUElLaUhCc2lSUEpqdWxWcm1TQUIyUXk5cjF3OHVjdk1nVm5kNGtWQjVO?=
 =?utf-8?B?Qk54aXlsNXkrbWdVbzZXeE9Ec3NLaTJvbERWc2JBdk5rZStlSEZwTGlWT1Bm?=
 =?utf-8?B?WnVtbFVuYTRRYUdZOEt4SVNkUElpa0tlZXZqQ2hqMG92SXg0dE1ZOWNNN1Yy?=
 =?utf-8?B?QzhDVWZNMzhyWHNMR1QwVzF0eFlnVVJQTElEUk9KQmpyL250QURIbTdXZGd0?=
 =?utf-8?B?MmN1SGdzRHF1VWlRRWQ3R2dzeVVUYTI0NGhmdmpzUWIvVlBieHR4T0VtRC9o?=
 =?utf-8?B?U2tkNk8yMTUwZGJnQkNRcXN2NUgxM1JDdVoyWmF3eFhjWmpKdEdxWlF2Ly9l?=
 =?utf-8?B?Sk82MEdiZ2lFS3lneTV6OUlpV1hoUmk5ZHgvTXQrb2lLVDgybHNxTFZjbzha?=
 =?utf-8?B?STFOaHlacERINWI3NHBNY0V5eVliQzRWQjNtZjR4TnhtV2h0MkNJOHorV2o2?=
 =?utf-8?B?UVc3eWg2RWFwaEZGM3BVNktHUHpUNHRHOGJnN3VRM0t0L0wvemllY1crZ05T?=
 =?utf-8?B?dC9RQ3NmN213OE5MVmtMdnp3eFpTSVZ5VzZFVGd4d0VTellDTDMxQnpPNnhI?=
 =?utf-8?B?UjhsZXJUNzJ5enI0NWZMOWFHQjA5ZDBCNE5tKzRBcTFGQ0Rrc2lUTFFIVjly?=
 =?utf-8?B?V3NPL2hpYlVMNXNRK2pKdkJsN0lJQTV1MGRaOWw4VVZ1bjM0TUxMMkNCY28v?=
 =?utf-8?B?d1Vwd3pmbnhIOU9XSGtTTkdCTnBYMG45QkJJdk4zRFl0ckxIVU5jVHQwTkky?=
 =?utf-8?B?SXp4Q3dKKzFDSndGYUVzWG1udjhLcTZlUUVUdU5KaFYrVi9Xb3UvcXVwbE9Z?=
 =?utf-8?B?ODhlYmo0aWs0dDlBNkpQdWRha2NmWm52amxyMHNQYTFHTm9GYjRDTjNnSndQ?=
 =?utf-8?B?ZVJTS1RxYm1RZVFKRCtBc2VhREI1enpHRUMrNHlRRjNwVU1tZEFSWUhkNk05?=
 =?utf-8?B?Mk42amwyQnJTdUJBYjh5TWpveitTeWVPa0ZIZFo1VW5zWkw2RnVEbll0ZkFz?=
 =?utf-8?B?ZlpkSGNqbVlEU0FzYUthMjFiUjlwVE5EUWxvT1RRMDlGRHVKd0hTcFA5VGpt?=
 =?utf-8?B?RWtOUk0rVkRqaVdOZ1pvMlFncXRCQXdPMHRsK0hHQXh4Q3lYVm1DQ1FNa1B1?=
 =?utf-8?B?MGgzUFpQeWJ3MnN0dkZlRWk4UkkxdUlOOGJaV1hoZkRXNkh5WHJQUDNSVlli?=
 =?utf-8?B?b09YcFptNUg4SjQ1eS9TaHJWcGpGQkZVUVBvdDdZUEJKU3hCNXdNVmRRQkpi?=
 =?utf-8?Q?R7BU/Sq9mqBjaEw1hFML0dnuw1emZ70JsDw/Iil?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9038d1b7-e2e0-4cad-e8c4-08d981882cbb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:26:51.6912
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lBZTzvjc4Q+LnMGsRbvKb8/ezmuBwke1OZQQkNhFgcL2bI02Swlu5f5F87EdalcHHT8vd11LfDVwp3KGiTT+mQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5328

On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Currently PCI backend implements multiple functionalities at a time.
> To name a few:
> 1. It is used as a database for assignable PCI devices, e.g. xl
>    pci-assignable-{add|remove|list} manipulates that list. So, whenever
>    the toolstack needs to know which PCI devices can be passed through
>    it reads that from the relevant sysfs entries of the pciback.
> 2. It is used to hold the unbound PCI devices list, e.g. when passing
>    through a PCI device it needs to be unbound from the relevant device
>    driver and bound to pciback (strictly speaking it is not required
>    that the device is bound to pciback, but pciback is again used as a
>    database of the passed through PCI devices, so we can re-bind the
>    devices back to their original drivers when guest domain shuts down)
> 3. Device reset for the devices being passed through
> 4. Para-virtualised use-cases support
> 
> The para-virtualised part of the driver is not always needed as some
> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
> model for PCI device passthrough. For such use-cases make the very
> first step in splitting the xen-pciback driver into two parts: Xen
> PCI stub and PCI PV backend drivers.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Changes since v3:
> - Move CONFIG_XEN_PCIDEV_STUB to the second patch

I'm afraid this wasn't fully done:

> --- a/drivers/xen/xen-pciback/Makefile
> +++ b/drivers/xen/xen-pciback/Makefile
> @@ -1,5 +1,6 @@
>  # SPDX-License-Identifier: GPL-2.0
>  obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o

While benign when added here, this addition still doesn't seem to
belong here.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:35:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:35:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196407.349241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlAq-0001v1-8s; Mon, 27 Sep 2021 07:35:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196407.349241; Mon, 27 Sep 2021 07:35:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlAq-0001uu-5m; Mon, 27 Sep 2021 07:35:40 +0000
Received: by outflank-mailman (input) for mailman id 196407;
 Mon, 27 Sep 2021 07:35:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUlAo-0001uo-UQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:35:38 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42745704-7912-4ee9-b686-10c0f686db88;
 Mon, 27 Sep 2021 07:35:37 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18R7FFBk008580;
 Mon, 27 Sep 2021 07:35:36 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2057.outbound.protection.outlook.com [104.47.9.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bb9dd82k1-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 07:35:36 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7345.eurprd03.prod.outlook.com (2603:10a6:20b:273::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 07:35:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:35:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42745704-7912-4ee9-b686-10c0f686db88
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CPIKHTqGkPEpCx9R6pCch7gdELFMyCQ1PEg4uilZXUze9m1QxT+WcS+tUBJaiDOde2gkx7G/eJOYolKJlkIQD5/i7zt5nPRDxS21shJgZihdb3XLPAgLfJjD46bd1WrjRoHCcGVYQaLhrkatLfVcCir4Z+5w3/dM6NOKe3ufL/bzsxO4HTD2uq6GKGxq+lLOfCTZvSKu6J5LLCe3LMriWtPPe7Gu0v/CjW1tTWL5ifDx5YiTUnRIQ/RnT6gmSDDBV8VQE1n8yNVOyqMyI4zbD6oIfAtd9eRBZHEHWwaSvoXQValc8mO5SBg7h5kl0PbJSa5qBi6bvfEn7Rhs9WaMtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9nSSwAihiXK3S+MOrY7xQjjBWbyyoTUWXuyU+f9frco=;
 b=mHr1xj4TdNDUj9M87Ahuc4z12poAoh8HSA4EnBAAz+cDPQ7ITcySEHwGzD8bf5swXrOyf1z7s9AECrWOhbYINV1hyM0CIhh1y756wQ9ImYFbcU5wcqAPnQfEuoHHJu1Pah1aL3Wz4PYMDyY+u0Czd/dQipfGea11AtSqmGq+tJiw50V58nwU0J3K+lHawsGCtKUqFzHjXb76/A3VX70oXgvRrN6jSrCDWhOEZ8hDlnTJWuOx/zXA3/YgHz/u3TCoApjW6sZL8c82AaDx635q0QTu7XcC73vXEHVdeNzL0+yxDj1ntwJxyJYC/vY8SHXOQfkLNMXkXg88KJyJIIS32g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9nSSwAihiXK3S+MOrY7xQjjBWbyyoTUWXuyU+f9frco=;
 b=eE60XSqn0Zs4o8chBco3lq1AL2QMV7639V8z6zxgHbw+q34tseqT7PXVLJLzy6UNXFhyewgp8dl+N2tZcg8SfPONr0jP4mOks3HUohU5jNg8nDaktUqF8Z/yfSMUOQXOqvMORPuX2aNE1hyIsEIpurSN7rNva1VPbL/SN3jyABTXEYYsUfrht59x5eMUehWPrWqoG0XEjLmWD13iVLLmqIj9+ac2Oe5q6k7loWzh+yN4ZSUM2jYPDJ+f3MEVTNtsGKy4htgm0ZX2n0tj4RQdshLCSA/XKORwyrKT+lIX81Hp1FBOdi/jyaZoPvUeW4UZbcx0nqLPEKMXaik7ucCS6g==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "jgross@suse.com" <jgross@suse.com>, "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org" <sstabellini@kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbAA=
Date: Mon, 27 Sep 2021 07:35:33 +0000
Message-ID: <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
In-Reply-To: <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 1df5d8d6-5f25-498d-604a-08d9818963e5
x-ms-traffictypediagnostic: AM9PR03MB7345:
x-microsoft-antispam-prvs: 
 <AM9PR03MB7345D61C3F857661F958BBC2E7A79@AM9PR03MB7345.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 zib9rlm9eXEWaQszhi4fPevHqekS0czU1A6HMuKl3z6iQ22UOH3sOf8G+j6drtEwj6/S55pNXU67v7AEsI1nIQIBxFlGvEetWDjcADIObgOOxeTogccQ0bffHAx2FbFmSJAcbGide99gsvjvxX0c3J1n/sKWE/i4uvHZb+FhuWE1VBFN/I9FEkOXzgc2JsAvUPPp0qdFcSPqgWU1o4alkpMNWkfQEOzid9twSQHUmRMkN84P6qFP8UeSL9srvsVBzPgppO7jApga10uWX81wdxrfhyaXbsNIcJk9OghVM0THHcaDgVHmfzZ4EkozFtLsbvn+LUoX7k9MC3xqtfv9USPemtcItQZ5RRafQ96vFUEBWDoyjfNCGXMSKpZ+4/qsPyx98namgHH/af8H8GlZ/VxVfhwOs0pvaWBWKpwDpZYAiyTShNqcFHmCfU7SqMpVl/v4gvn4XvicMFXn5+f6pQkixT22cB73d+gwAwA/C8jYeRpnmwQGOgOaVVaocUHCgiE8ryADFWiNaxYetCNcVI+12B7nn7y2GvsCxv6vOp0MOpVjV4hzjp4O7biUD3++6cIvavjbcGGLUs7cFo/wvJVSGMEaVxccd190cZ0kPoULducw4ntCO2rH4rR2Qj0GcquAiDkAQBMcrdMv85yWCXTc+12p3B4eKT9anVOdC3caIM7Nqc4m+jASK6kEX6CmcH6/Y3PRi5zabauVXcUJX5PlUz7wH0HShO8vXl72h4VyASESqjSP81xRb001rT4d
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(36756003)(38100700002)(54906003)(122000001)(86362001)(8936002)(76116006)(508600001)(110136005)(2906002)(5660300002)(8676002)(316002)(71200400001)(4326008)(6512007)(64756008)(66946007)(66446008)(53546011)(6506007)(55236004)(2616005)(26005)(38070700005)(31686004)(6486002)(31696002)(66556008)(66476007)(83380400001)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?MTlzamdnUEdjZ1dmREJvbEY2cVVrRFBwdlpQQ09KOU1EK2srT3hkS0hneFNY?=
 =?utf-8?B?QytLSnJsWjgyaTNYVUxWdDY4MWVBTjM1bHAwa1IrcnF0K0NERE9abkRXcDh1?=
 =?utf-8?B?aCtzU2xaUlJYNnpCUXBScDlOZXB6eEZpZXRKUWhpM242a0F0bktHL0o0MXJ1?=
 =?utf-8?B?TjFvUlNpSTAxWkxibCtmQkprbXR6WVFndk4yeEg4RkRkMlF1UWlhckI5NjJ1?=
 =?utf-8?B?anp6YkdxbkVTbDBzZUczdEsrd3ByTDh4MGd2aEM1TjkzVEk2cExtZlFPNFp1?=
 =?utf-8?B?U01sWU1uVkdQM2o1LytORmhlb0hianNFSkVjNmkzWHp2Y3J2cmtCc2V3K1dm?=
 =?utf-8?B?UzNOdmJHeDVhck5PNE9qek5YeTNvVjlSc2wrRkFVVndMRy9IRW5HZWtNWEcx?=
 =?utf-8?B?YjFCekt4MmgvMFBYcXhnUTFXeWl0bTJJWmpmY3FSOHFmTVdZSVh4d3kydGtn?=
 =?utf-8?B?SVZKS25XTUxPWGZXNEpEQnpRMktVa05WeHMrLzhscEphaHRhamFNcHFWMkZw?=
 =?utf-8?B?U0h2emxPZUIxTnNTdk1PUWltejVFcDZwUzNuenU1bkRJY1FTY25NMDdpdVI2?=
 =?utf-8?B?dVpvL3hzVlhUZWN4TmFZUitjOWdtVThxK1d6U2NMa01vMGsyUk9CUVB6cDZj?=
 =?utf-8?B?NlZsNVdYdEU2RTdBMzIvdHd6WUlVS0dac0dYeUFtNGNya3YyMWRCN3ZlbGpQ?=
 =?utf-8?B?dG5xRG5ZczVBMEtkRzRFSUpDMGhGQ2pENGZtYytCRnJTTlZhc3VtZDhsZmtR?=
 =?utf-8?B?elg5NXVjMnFIQWdYWnY1VGEwTnZBUDhRL2xWdnN6YmZYZkpPcFJwNm9BQ095?=
 =?utf-8?B?cktHd2t1a1hreU1ESnJMN1hUQW1iOVlNc2pJeEFhWnBUQmZHMTdKWXZzTVU4?=
 =?utf-8?B?eEhMOHh3aFY3a3RGSVRkdURERVZYa1FEYkRZN0M0L1dqcXdFbzBkalU4bGdY?=
 =?utf-8?B?bndyc3IvMWEwVmRDTWkzWEZtWTFtc0E4NXF4VVkrUE1kei9mTExuWWJhWEcx?=
 =?utf-8?B?L1k4V05BR3FSZHVMKzlyVm9kWkNxMUNiRHRDRi9IUm9PODlFQy9qNUVlc09i?=
 =?utf-8?B?ODFpN3BFL0s4bTlGRlZCd3M1UDA4MDFwTEszNzBibzE2dFJvcWhkaW5XaUEz?=
 =?utf-8?B?TU1QR1lRVjNZcVozK3FadHpid1VRTTBLQ3NsRGtwb1NZZEN0NFV4SURlb1FW?=
 =?utf-8?B?YkZ5L2ZNUEdxMTh1VVY5RFEwUlAvNitnVW1ldnhYZ05lSlIySms5azJ0Nmd6?=
 =?utf-8?B?NXVHeVJDd1JkQi9vamVEVSs4ZG5vc0JWR1l1bzFqb1lFaVpicUNhdndDdnlq?=
 =?utf-8?B?cDFwaHM1cTR6NUt5UmZMUzRRaWlSWkxsUUNzNGt6MG9tSWYrbjdTamlpQU1W?=
 =?utf-8?B?V292VGo0dmw0eEZ4ZHVJUHA3OE9yck1WeUVaUzY0WTNrY3hvNnozUTBDTnU4?=
 =?utf-8?B?N1ptd205MDN4MmpVR01zYVFzQ2paUmF3M1UwZEwvY2p4bWJLd0JDTUo3Uisr?=
 =?utf-8?B?R1lheDdWbWNLQ2NmdU81TEVEOUoxSHV4T0VzdFNlVkl2QTBwZG5FQWU5Vi9Z?=
 =?utf-8?B?VjYrSGxhTVNEZjBzUzlsMGR5OWtIWnd4NkpnRkZpcmN4dWNRTnZxLy9vNGls?=
 =?utf-8?B?QmFxMkRBL1cyVEl4eUR6a1grYjlJcXljNXEzWFkvUWdDR3d5RUpYMm5Ob2xQ?=
 =?utf-8?B?VHdFYlpBUG9aejVidnlDR3V4aXl6Kys5QkpoU2ZON3NMQzNOSW9ML1ZKZStZ?=
 =?utf-8?B?L2llaExWVmFJbTZYNnB6VERONFdIZzc5elFrL0dYQmppT0tJM1pFV0hFOGpE?=
 =?utf-8?B?aXJ3VEpvT1I2b1lsdng2dz09?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <F6969D1A2AAD37408B0B3D31E9085D4A@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1df5d8d6-5f25-498d-604a-08d9818963e5
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 07:35:33.5448
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rSNRzDlRFlhzTtTqyYeiluDXogNQ8/2RvcsG3Mv8TiAdVXEWboXhtWp/4kLb97zJolZgvicnpraeTPRwtS3F6LDTtWBtHraTFNPtvUehoKABJX1dxUrqweepmWHbjALx
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7345
X-Proofpoint-GUID: nSkETDT8JU0GjmcNUTkm-BY_BB7BmUj3
X-Proofpoint-ORIG-GUID: nSkETDT8JU0GjmcNUTkm-BY_BB7BmUj3
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0
 malwarescore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=999
 bulkscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270051

DQpPbiAyNy4wOS4yMSAxMDoyNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MDg6NTgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+
IEN1cnJlbnRseSBQQ0kgYmFja2VuZCBpbXBsZW1lbnRzIG11bHRpcGxlIGZ1bmN0aW9uYWxpdGll
cyBhdCBhIHRpbWUuDQo+PiBUbyBuYW1lIGEgZmV3Og0KPj4gMS4gSXQgaXMgdXNlZCBhcyBhIGRh
dGFiYXNlIGZvciBhc3NpZ25hYmxlIFBDSSBkZXZpY2VzLCBlLmcuIHhsDQo+PiAgICAgcGNpLWFz
c2lnbmFibGUte2FkZHxyZW1vdmV8bGlzdH0gbWFuaXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywgd2hl
bmV2ZXINCj4+ICAgICB0aGUgdG9vbHN0YWNrIG5lZWRzIHRvIGtub3cgd2hpY2ggUENJIGRldmlj
ZXMgY2FuIGJlIHBhc3NlZCB0aHJvdWdoDQo+PiAgICAgaXQgcmVhZHMgdGhhdCBmcm9tIHRoZSBy
ZWxldmFudCBzeXNmcyBlbnRyaWVzIG9mIHRoZSBwY2liYWNrLg0KPj4gMi4gSXQgaXMgdXNlZCB0
byBob2xkIHRoZSB1bmJvdW5kIFBDSSBkZXZpY2VzIGxpc3QsIGUuZy4gd2hlbiBwYXNzaW5nDQo+
PiAgICAgdGhyb3VnaCBhIFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRo
ZSByZWxldmFudCBkZXZpY2UNCj4+ICAgICBkcml2ZXIgYW5kIGJvdW5kIHRvIHBjaWJhY2sgKHN0
cmljdGx5IHNwZWFraW5nIGl0IGlzIG5vdCByZXF1aXJlZA0KPj4gICAgIHRoYXQgdGhlIGRldmlj
ZSBpcyBib3VuZCB0byBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1c2VkIGFzIGENCj4+
ICAgICBkYXRhYmFzZSBvZiB0aGUgcGFzc2VkIHRocm91Z2ggUENJIGRldmljZXMsIHNvIHdlIGNh
biByZS1iaW5kIHRoZQ0KPj4gICAgIGRldmljZXMgYmFjayB0byB0aGVpciBvcmlnaW5hbCBkcml2
ZXJzIHdoZW4gZ3Vlc3QgZG9tYWluIHNodXRzIGRvd24pDQo+PiAzLiBEZXZpY2UgcmVzZXQgZm9y
IHRoZSBkZXZpY2VzIGJlaW5nIHBhc3NlZCB0aHJvdWdoDQo+PiA0LiBQYXJhLXZpcnR1YWxpc2Vk
IHVzZS1jYXNlcyBzdXBwb3J0DQo+Pg0KPj4gVGhlIHBhcmEtdmlydHVhbGlzZWQgcGFydCBvZiB0
aGUgZHJpdmVyIGlzIG5vdCBhbHdheXMgbmVlZGVkIGFzIHNvbWUNCj4+IGFyY2hpdGVjdHVyZXMs
IGUuZy4gQXJtIG9yIHg4NiBQVkggRG9tMCwgYXJlIG5vdCB1c2luZyBiYWNrZW5kLWZyb250ZW5k
DQo+PiBtb2RlbCBmb3IgUENJIGRldmljZSBwYXNzdGhyb3VnaC4gRm9yIHN1Y2ggdXNlLWNhc2Vz
IG1ha2UgdGhlIHZlcnkNCj4+IGZpcnN0IHN0ZXAgaW4gc3BsaXR0aW5nIHRoZSB4ZW4tcGNpYmFj
ayBkcml2ZXIgaW50byB0d28gcGFydHM6IFhlbg0KPj4gUENJIHN0dWIgYW5kIFBDSSBQViBiYWNr
ZW5kIGRyaXZlcnMuDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVu
a28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+IC0tLQ0KPj4gQ2hh
bmdlcyBzaW5jZSB2MzoNCj4+IC0gTW92ZSBDT05GSUdfWEVOX1BDSURFVl9TVFVCIHRvIHRoZSBz
ZWNvbmQgcGF0Y2gNCj4gSSdtIGFmcmFpZCB0aGlzIHdhc24ndCBmdWxseSBkb25lOg0KPg0KPj4g
LS0tIGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4+ICsrKyBiL2RyaXZlcnMv
eGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+PiBAQCAtMSw1ICsxLDYgQEANCj4+ICAgIyBTUERY
LUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMA0KPj4gICBvYmotJChDT05GSUdfWEVOX1BDSURF
Vl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQo+PiArb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZf
U1RVQikgKz0geGVuLXBjaWJhY2subw0KPiBXaGlsZSBiZW5pZ24gd2hlbiBhZGRlZCBoZXJlLCB0
aGlzIGFkZGl0aW9uIHN0aWxsIGRvZXNuJ3Qgc2VlbSB0bw0KPiBiZWxvbmcgaGVyZS4NCg0KTXkg
YmFkLiBTbywgaXQgc2VlbXMgd2l0aG91dCBDT05GSUdfWEVOX1BDSURFVl9TVFVCIHRoZSBjaGFu
Z2Ugc2VlbXMNCg0KdG8gYmUgbm9uLWZ1bmN0aW9uYWwuIFdpdGggQ09ORklHX1hFTl9QQ0lERVZf
U1RVQiB3ZSBmYWlsIHRvIGJ1aWxkIG9uIDMyLWJpdA0KDQphcmNoaXRlY3R1cmVzLi4uDQoNCldo
YXQgd291bGQgYmUgdGhlIHByZWZlcmVuY2UgaGVyZT8gU3RlZmFubyBzdWdnZXN0ZWQgdGhhdCB3
ZSBzdGlsbCBkZWZpbmUNCg0KQ09ORklHX1hFTl9QQ0lERVZfU1RVQiwgYnV0IGluIGRpc2FibGVk
IHN0YXRlLCBlLmcuIHdlIGFkZCB0cmlzdGF0ZSB0byBpdA0KDQppbiB0aGUgc2Vjb25kIHBhdGNo
DQoNCkFub3RoZXIgb3B0aW9uIGlzIGp1c3QgdG8gc3F1YXNoIHRoZSB0d28gcGF0Y2hlcy4NCg0K
Pg0KPiBKYW4NCj4=


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:36:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:36:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196415.349252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlBN-0002WV-Ma; Mon, 27 Sep 2021 07:36:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196415.349252; Mon, 27 Sep 2021 07:36:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlBN-0002WO-In; Mon, 27 Sep 2021 07:36:13 +0000
Received: by outflank-mailman (input) for mailman id 196415;
 Mon, 27 Sep 2021 07:36:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Pp17=OR=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mUlBM-0002Ut-0k
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:36:12 +0000
Received: from mail-ed1-x532.google.com (unknown [2a00:1450:4864:20::532])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 83070884-0034-48df-a105-50da553ea5e7;
 Mon, 27 Sep 2021 07:36:09 +0000 (UTC)
Received: by mail-ed1-x532.google.com with SMTP id ee50so65432656edb.13
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 00:36:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83070884-0034-48df-a105-50da553ea5e7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=AyOb/yB1i5HlOX5zI/WRQXD6X2rbNnjVSkFRlNH62ss=;
        b=qdZmSa8ut5PUpPvuP54SLEe9L8+4XSBCUHAMQRws5IvMRaoHj3n9fDaUaBhCQ0zaQX
         BtzJxbM2lFJiQaulaJkfm9DZ5v8ESKT7fSjyY7LT8IdzgMXCap5gRg0vfTvTsR8sZJ+K
         HSTgVTaHDiEfw8TB3NN/jvDu1uFbWX3a8uQCSEPAN30XagdGYcdt32KRK0PR3SA4TUMI
         F4FdbdcvJdiDPObBMvSPrvhripiyyidcN7dwCQwymHyRGtm4X/DwFOfoEeLiVEnzIJRR
         C3r7sYmuXtvXfN2Bz86nOp/P9w9xhPHA7hy4qIVmpAfgr9cQdRAk2seC8UOJB/lwCX2Y
         fLJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=AyOb/yB1i5HlOX5zI/WRQXD6X2rbNnjVSkFRlNH62ss=;
        b=3gGWAdo4WqYD6thMxNEaSsPyYA0Cqcid+CHGWujWGAGbhYmajIszX0zD12Jf4immHT
         R5NyP/Y3FzjKA9/1kvV+fNk5Ig7B7AKoElbccmSQD6mo7B3C2uWLjOz9xKjzM7Iz+V+L
         AMNYk9mPoLh369BR1geHru54cAs3fcoSpr26nkfE77L4ROgBY6/TrGJNLbvTFhsFsgWR
         LbETEk0EHy8XdzaQ8SiXsU3gf1OnzslZuFh4sLN+ZROs9NwTfAdlmdUjXO2gu6VlC4Kc
         o8GA9sF8KYldVnYQ5A3vwNCiKilbvAuteogkqW2xgJloI4QaCP+uTFCmrGq2O4uIVAHr
         p8YQ==
X-Gm-Message-State: AOAM532j3ER6mMsQQ3I13r5DHE85yULhbBUxOwlxFx982ond58R4x6pP
	HnLh7VVBJcF/gYq+PTe027RO48es+b5YrKWXbJo=
X-Google-Smtp-Source: ABdhPJx2YVwPwSdHk/ZSxNuJDz7nw+l/QnRStNjMxMJar6RdXnBlLI0ewpi3So1EyVe5x3gPx7C2FsZnmAQp88cMmOk=
X-Received: by 2002:a17:906:781:: with SMTP id l1mr25706464ejc.289.1632728168470;
 Mon, 27 Sep 2021 00:36:08 -0700 (PDT)
MIME-Version: 1.0
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
In-Reply-To: <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Mon, 27 Sep 2021 11:35:57 +0400
Message-ID: <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com>
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
To: Wei Chen <Wei.Chen@arm.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="0000000000007f1af905ccf5263f"

--0000000000007f1af905ccf5263f
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 27 Sep 2021, 08:53 Wei Chen, <Wei.Chen@arm.com> wrote:

> Hi Julien,
>
> > -----Original Message-----
> > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Wei
> > Chen
> > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 14:46
> > To: Stefano Stabellini <sstabellini@kernel.org>
> > Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>; jbeulich@suse.com; roger.pau@citrix.com;
> > andrew.cooper3@citrix.com
> > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override defaul=
t
> > NR_NODE_MEMBLKS
> >
> > Hi Stefano, Julien,
> >
> > > -----Original Message-----
> > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 13:00
> > > To: Wei Chen <Wei.Chen@arm.com>
> > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > <Bertrand.Marquis@arm.com>; jbeulich@suse.com; roger.pau@citrix.com;
> > > andrew.cooper3@citrix.com
> > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> default
> > > NR_NODE_MEMBLKS
> > >
> > > +x86 maintainers
> > >
> > > On Mon, 27 Sep 2021, Wei Chen wrote:
> > > > > -----Original Message-----
> > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26
> > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > > <Bertrand.Marquis@arm.com>
> > > > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> > > default
> > > > > NR_NODE_MEMBLKS
> > > > >
> > > > > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > > > Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35
> > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > > > julien@xen.org;
> > > > > > > Bertrand Marquis <Bertrand.Marquis@arm.com>
> > > > > > > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to
> override
> > > > > default
> > > > > > > NR_NODE_MEMBLKS
> > > > > > >
> > > > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > > > As a memory range described in device tree cannot be split
> > > across
> > > > > > > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANK=
S
> > in
> > > > > > > > arch header.
> > > > > > >
> > > > > > > This statement is true but what is the goal of this patch? Is
> it
> > > to
> > > > > > > reduce code size and memory consumption?
> > > > > > >
> > > > > >
> > > > > > No, when Julien and I discussed this in last version[1], we
> hadn't
> > > > > thought
> > > > > > so deeply. We just thought a memory range described in DT canno=
t
> > be
> > > > > split
> > > > > > across multiple nodes. So NR_MEM_BANKS should be equal to
> > > NR_MEM_BANKS.
> > > > > >
> > > > > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > > > > 08/msg00974.html
> > > > > >
> > > > > > > I am asking because NR_MEM_BANKS is 128 and
> > > > > > > NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 by default so ag=
ain
> > > > > > > NR_NODE_MEMBLKS is 128 before this patch.
> > > > > > >
> > > > > > > In other words, this patch alone doesn't make any difference;
> at
> > > least
> > > > > > > doesn't make any difference unless CONFIG_NR_NUMA_NODES is
> > > increased.
> > > > > > >
> > > > > > > So, is the goal to reduce memory usage when
> CONFIG_NR_NUMA_NODES
> > > is
> > > > > > > higher than 64?
> > > > > > >
> > > > > >
> > > > > > I also thought about this problem when I was writing this patch=
.
> > > > > > CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
> > > > > > value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NOD=
ES
> > > > > > at one point.
> > > > > >
> > > > > > But I agree with Julien's suggestion, NR_MEM_BANKS and
> > > NR_NODE_MEMBLKS
> > > > > > must be aware of each other. I had thought to add some ASSERT
> > check,
> > > > > > but I don't know how to do it better. So I post this patch for
> > more
> > > > > > suggestion.
> > > > >
> > > > > OK. In that case I'd say to get rid of the previous definition of
> > > > > NR_NODE_MEMBLKS as it is probably not necessary, see below.
> > > > >
> > > > >
> > > > >
> > > > > > >
> > > > > > > > And keep default NR_NODE_MEMBLKS in common header
> > > > > > > > for those architectures NUMA is disabled.
> > > > > > >
> > > > > > > This last sentence is not accurate: on x86 NUMA is enabled an=
d
> > > > > > > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h
> > (there
> > > is
> > > > > no
> > > > > > > x86 definition of it)
> > > > > > >
> > > > > >
> > > > > > Yes.
> > > > > >
> > > > > > >
> > > > > > > > Signed-off-by: Wei Chen <wei.chen@arm.com>
> > > > > > > > ---
> > > > > > > >  xen/include/asm-arm/numa.h | 8 +++++++-
> > > > > > > >  xen/include/xen/numa.h     | 2 ++
> > > > > > > >  2 files changed, 9 insertions(+), 1 deletion(-)
> > > > > > > >
> > > > > > > > diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-
> > > arm/numa.h
> > > > > > > > index 8f1c67e3eb..21569e634b 100644
> > > > > > > > --- a/xen/include/asm-arm/numa.h
> > > > > > > > +++ b/xen/include/asm-arm/numa.h
> > > > > > > > @@ -3,9 +3,15 @@
> > > > > > > >
> > > > > > > >  #include <xen/mm.h>
> > > > > > > >
> > > > > > > > +#include <asm/setup.h>
> > > > > > > > +
> > > > > > > >  typedef u8 nodeid_t;
> > > > > > > >
> > > > > > > > -#ifndef CONFIG_NUMA
> > > > > > > > +#ifdef CONFIG_NUMA
> > > > > > > > +
> > > > > > > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> > > > > > > > +
> > > > > > > > +#else
> > > > > > > >
> > > > > > > >  /* Fake one node for now. See also node_online_map. */
> > > > > > > >  #define cpu_to_node(cpu) 0
> > > > > > > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.=
h
> > > > > > > > index 1978e2be1b..1731e1cc6b 100644
> > > > > > > > --- a/xen/include/xen/numa.h
> > > > > > > > +++ b/xen/include/xen/numa.h
> > > > > > > > @@ -12,7 +12,9 @@
> > > > > > > >  #define MAX_NUMNODES    1
> > > > > > > >  #endif
> > > > > > > >
> > > > > > > > +#ifndef NR_NODE_MEMBLKS
> > > > > > > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > > > > > > > +#endif
> > > > >
> > > > > This one we can remove it completely right?
> > > >
> > > > How about define NR_MEM_BANKS to:
> > > > #ifdef CONFIG_NR_NUMA_NODES
> > > > #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)
> > > > #else
> > > > #define NR_MEM_BANKS 128
> > > > #endif
> > > > for both x86 and Arm. For those architectures do not support or
> enable
> > > > NUMA, they can still use "NR_MEM_BANKS 128". And replace all
> > > NR_NODE_MEMBLKS
> > > > in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.
> > > > In this case, NR_MEM_BANKS can be aware of the changes of
> > > CONFIG_NR_NUMA_NODES.
> > >
> > > x86 doesn't have NR_MEM_BANKS as far as I can tell. I guess you also
> > > meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?
> > >
> >
> > Yes.
> >
> > > But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES
> because
> > > there can be many memory banks for each numa node, certainly more tha=
n
> > > 2. The existing definition on x86:
> > >
> > > #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > >
> > > Doesn't make a lot of sense to me. Was it just an arbitrary limit for
> > > the lack of a better way to set a maximum?
> > >
> >
> > At that time, this was probably the most cost-effective approach.
> > Enough and easy. But, if more nodes need to be supported in the
> > future, it may bring more memory blocks. And this maximum value
> > might not apply. The maximum may need to support dynamic extension.
> >
> > >
> > > On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be relate=
d.
> > > In fact, what's the difference?
> > >
> > > NR_MEM_BANKS is the max number of memory banks (with or without
> > > numa-node-id).
> > >
> > > NR_NODE_MEMBLKS is the max number of memory banks with NUMA support
> > > (with numa-node-id)?
> > >
> > > They are basically the same thing. On ARM I would just do:
> > >
> >
> > Probably not, NR_MEM_BANKS will count those memory ranges without
> > numa-node-id in boot memory parsing stage (process_memory_node or
> > EFI parser). But NR_NODE_MEMBLKS will only count those memory ranges
> > with numa-node-id.
> >
> > > #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))
> > >
> > >
>
> Quote Julien's comment from HTML email to here:
> " As you wrote above, the second part of the MAX is totally arbitrary.
> In fact, it is very likely than if you have more than 64 nodes, you may
> need a lot more than 2 regions per node.
>
> So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to
> NR_MEM_BANKS
> so it can be used by common code.
> "
>
> But here comes the problem:
> How can we set the NR_MEM_BANKS maximum value, 128 seems an arbitrary too=
?
>

This is based on hardware we currently support (the last time we bumped the
value was, IIRC, for Thunder-X). In the case of booting UEFI, we can get a
lot of small ranges as we discover the RAM using the UEFI memory map.

If #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * N)? And what N should be.


N would have to be the maximum number of ranges you can find in a NUMA node=
.

We would also need to make sure this doesn't break existing platforms. So N
would have to be quite large or we need a MAX as Stefano suggested.

But I would prefer to keep the existing 128 and allow to configure it at
build time (not necessarily in this series). This avoid to have different
way to define the value based NUMA vs non-NUMA.


> > > And maybe the definition could be common with x86 if we define
> > > NR_MEM_BANKS to 128 on x86 too.
> >
> > Julien had comment here, I will continue in that email.
>

--0000000000007f1af905ccf5263f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Mon, 27 Sep 2021, 08:53 Wei Chen, &lt;<a href=3D"ma=
ilto:Wei.Chen@arm.com">Wei.Chen@arm.com</a>&gt; wrote:<br></div><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">Hi Julien,<br>
<br>
&gt; -----Original Message-----<br>
&gt; From: Xen-devel &lt;<a href=3D"mailto:xen-devel-bounces@lists.xenproje=
ct.org" target=3D"_blank" rel=3D"noreferrer">xen-devel-bounces@lists.xenpro=
ject.org</a>&gt; On Behalf Of Wei<br>
&gt; Chen<br>
&gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 14:46<br>
&gt; To: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.org" t=
arget=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;<br>
&gt; Cc: <a href=3D"mailto:xen-devel@lists.xenproject.org" target=3D"_blank=
" rel=3D"noreferrer">xen-devel@lists.xenproject.org</a>; <a href=3D"mailto:=
julien@xen.org" target=3D"_blank" rel=3D"noreferrer">julien@xen.org</a>; Be=
rtrand Marquis<br>
&gt; &lt;<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"_blank" rel=
=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;; <a href=3D"mailto:jbeulic=
h@suse.com" target=3D"_blank" rel=3D"noreferrer">jbeulich@suse.com</a>; <a =
href=3D"mailto:roger.pau@citrix.com" target=3D"_blank" rel=3D"noreferrer">r=
oger.pau@citrix.com</a>;<br>
&gt; <a href=3D"mailto:andrew.cooper3@citrix.com" target=3D"_blank" rel=3D"=
noreferrer">andrew.cooper3@citrix.com</a><br>
&gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override defau=
lt<br>
&gt; NR_NODE_MEMBLKS<br>
&gt; <br>
&gt; Hi Stefano, Julien,<br>
&gt; <br>
&gt; &gt; -----Original Message-----<br>
&gt; &gt; From: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel=
.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;<b=
r>
&gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 13:00<br>
&gt; &gt; To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@arm.com" target=3D"_b=
lank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; Cc: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.o=
rg" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;; xe=
n-<br>
&gt; &gt; <a href=3D"mailto:devel@lists.xenproject.org" target=3D"_blank" r=
el=3D"noreferrer">devel@lists.xenproject.org</a>; <a href=3D"mailto:julien@=
xen.org" target=3D"_blank" rel=3D"noreferrer">julien@xen.org</a>; Bertrand =
Marquis<br>
&gt; &gt; &lt;<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"_blank"=
 rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;; <a href=3D"mailto:jbe=
ulich@suse.com" target=3D"_blank" rel=3D"noreferrer">jbeulich@suse.com</a>;=
 <a href=3D"mailto:roger.pau@citrix.com" target=3D"_blank" rel=3D"noreferre=
r">roger.pau@citrix.com</a>;<br>
&gt; &gt; <a href=3D"mailto:andrew.cooper3@citrix.com" target=3D"_blank" re=
l=3D"noreferrer">andrew.cooper3@citrix.com</a><br>
&gt; &gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override =
default<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt;<br>
&gt; &gt; +x86 maintainers<br>
&gt; &gt;<br>
&gt; &gt; On Mon, 27 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; -----Original Message-----<br>
&gt; &gt; &gt; &gt; From: Stefano Stabellini &lt;<a href=3D"mailto:sstabell=
ini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org=
</a>&gt;<br>
&gt; &gt; &gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26<br>
&gt; &gt; &gt; &gt; To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@arm.com" ta=
rget=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; Cc: Stefano Stabellini &lt;<a href=3D"mailto:sstabellin=
i@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</=
a>&gt;; xen-<br>
&gt; &gt; &gt; &gt; <a href=3D"mailto:devel@lists.xenproject.org" target=3D=
"_blank" rel=3D"noreferrer">devel@lists.xenproject.org</a>; <a href=3D"mail=
to:julien@xen.org" target=3D"_blank" rel=3D"noreferrer">julien@xen.org</a>;=
 Bertrand Marquis<br>
&gt; &gt; &gt; &gt; &lt;<a href=3D"mailto:Bertrand.Marquis@arm.com" target=
=3D"_blank" rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to=
 override<br>
&gt; &gt; default<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; On Sun, 26 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; &gt; &gt; -----Original Message-----<br>
&gt; &gt; &gt; &gt; &gt; &gt; From: Stefano Stabellini &lt;<a href=3D"mailt=
o:sstabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@=
kernel.org</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35=
<br>
&gt; &gt; &gt; &gt; &gt; &gt; To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@a=
rm.com" target=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Cc: <a href=3D"mailto:xen-devel@lists.xenproj=
ect.org" target=3D"_blank" rel=3D"noreferrer">xen-devel@lists.xenproject.or=
g</a>; <a href=3D"mailto:sstabellini@kernel.org" target=3D"_blank" rel=3D"n=
oreferrer">sstabellini@kernel.org</a>;<br>
&gt; &gt; &gt; &gt; <a href=3D"mailto:julien@xen.org" target=3D"_blank" rel=
=3D"noreferrer">julien@xen.org</a>;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Bertrand Marquis &lt;<a href=3D"mailto:Bertra=
nd.Marquis@arm.com" target=3D"_blank" rel=3D"noreferrer">Bertrand.Marquis@a=
rm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Subject: Re: [PATCH 22/37] xen/arm: use NR_ME=
M_BANKS to override<br>
&gt; &gt; &gt; &gt; default<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; On Thu, 23 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; As a memory range described in device tr=
ee cannot be split<br>
&gt; &gt; across<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; multiple nodes. So we define NR_NODE_MEM=
BLKS as NR_MEM_BANKS<br>
&gt; in<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; arch header.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; This statement is true but what is the goal o=
f this patch? Is it<br>
&gt; &gt; to<br>
&gt; &gt; &gt; &gt; &gt; &gt; reduce code size and memory consumption?<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; No, when Julien and I discussed this in last versi=
on[1], we hadn&#39;t<br>
&gt; &gt; &gt; &gt; thought<br>
&gt; &gt; &gt; &gt; &gt; so deeply. We just thought a memory range describe=
d in DT cannot<br>
&gt; be<br>
&gt; &gt; &gt; &gt; split<br>
&gt; &gt; &gt; &gt; &gt; across multiple nodes. So NR_MEM_BANKS should be e=
qual to<br>
&gt; &gt; NR_MEM_BANKS.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; <a href=3D"https://lists.xenproject.org/archives/h=
tml/xen-devel/2021-" rel=3D"noreferrer noreferrer" target=3D"_blank">https:=
//lists.xenproject.org/archives/html/xen-devel/2021-</a><br>
&gt; &gt; &gt; &gt; 08/msg00974.html<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; I am asking because NR_MEM_BANKS is 128 and<b=
r>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 =
by default so again<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is 128 before this patch.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; In other words, this patch alone doesn&#39;t =
make any difference; at<br>
&gt; &gt; least<br>
&gt; &gt; &gt; &gt; &gt; &gt; doesn&#39;t make any difference unless CONFIG=
_NR_NUMA_NODES is<br>
&gt; &gt; increased.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; So, is the goal to reduce memory usage when C=
ONFIG_NR_NUMA_NODES<br>
&gt; &gt; is<br>
&gt; &gt; &gt; &gt; &gt; &gt; higher than 64?<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; I also thought about this problem when I was writi=
ng this patch.<br>
&gt; &gt; &gt; &gt; &gt; CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BAN=
KS is a fixed<br>
&gt; &gt; &gt; &gt; &gt; value, then NR_MEM_BANKS can be smaller than CONFI=
G_NR_NUMA_NODES<br>
&gt; &gt; &gt; &gt; &gt; at one point.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; But I agree with Julien&#39;s suggestion, NR_MEM_B=
ANKS and<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; must be aware of each other. I had thought to add =
some ASSERT<br>
&gt; check,<br>
&gt; &gt; &gt; &gt; &gt; but I don&#39;t know how to do it better. So I pos=
t this patch for<br>
&gt; more<br>
&gt; &gt; &gt; &gt; &gt; suggestion.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; OK. In that case I&#39;d say to get rid of the previous=
 definition of<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS as it is probably not necessary, see be=
low.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; And keep default NR_NODE_MEMBLKS in comm=
on header<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; for those architectures NUMA is disabled=
.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; This last sentence is not accurate: on x86 NU=
MA is enabled and<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is still defined in xen/inclu=
de/xen/numa.h<br>
&gt; (there<br>
&gt; &gt; is<br>
&gt; &gt; &gt; &gt; no<br>
&gt; &gt; &gt; &gt; &gt; &gt; x86 definition of it)<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; Yes.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; Signed-off-by: Wei Chen &lt;<a href=3D"m=
ailto:wei.chen@arm.com" target=3D"_blank" rel=3D"noreferrer">wei.chen@arm.c=
om</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; ---<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/asm-arm/numa.h | 8 +++=
++++-<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/xen/numa.h=C2=A0 =C2=
=A0 =C2=A0| 2 ++<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 2 files changed, 9 insertions(+), =
1 deletion(-)<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/asm-arm/numa.h =
b/xen/include/asm-<br>
&gt; &gt; arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; index 8f1c67e3eb..21569e634b 100644<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; --- a/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +++ b/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; @@ -3,9 +3,15 @@<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #include &lt;xen/mm.h&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#include &lt;asm/setup.h&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 typedef u8 nodeid_t;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; -#ifndef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#ifdef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#define NR_NODE_MEMBLKS NR_MEM_BANKS<br=
>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#else<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 /* Fake one node for now. See also=
 node_online_map. */<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define cpu_to_node(cpu) 0<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/xen/numa.h b/xe=
n/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; index 1978e2be1b..1731e1cc6b 100644<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; --- a/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +++ b/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; @@ -12,7 +12,9 @@<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define MAX_NUMNODES=C2=A0 =C2=A0 =
1<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #endif<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#ifndef NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define NR_NODE_MEMBLKS (MAX_NUMNO=
DES*2)<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#endif<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; This one we can remove it completely right?<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; How about define NR_MEM_BANKS to:<br>
&gt; &gt; &gt; #ifdef CONFIG_NR_NUMA_NODES<br>
&gt; &gt; &gt; #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)<br>
&gt; &gt; &gt; #else<br>
&gt; &gt; &gt; #define NR_MEM_BANKS 128<br>
&gt; &gt; &gt; #endif<br>
&gt; &gt; &gt; for both x86 and Arm. For those architectures do not support=
 or enable<br>
&gt; &gt; &gt; NUMA, they can still use &quot;NR_MEM_BANKS 128&quot;. And r=
eplace all<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS compl=
etely.<br>
&gt; &gt; &gt; In this case, NR_MEM_BANKS can be aware of the changes of<br=
>
&gt; &gt; CONFIG_NR_NUMA_NODES.<br>
&gt; &gt;<br>
&gt; &gt; x86 doesn&#39;t have NR_MEM_BANKS as far as I can tell. I guess y=
ou also<br>
&gt; &gt; meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?<br>
&gt; &gt;<br>
&gt; <br>
&gt; Yes.<br>
&gt; <br>
&gt; &gt; But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES =
because<br>
&gt; &gt; there can be many memory banks for each numa node, certainly more=
 than<br>
&gt; &gt; 2. The existing definition on x86:<br>
&gt; &gt;<br>
&gt; &gt; #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)<br>
&gt; &gt;<br>
&gt; &gt; Doesn&#39;t make a lot of sense to me. Was it just an arbitrary l=
imit for<br>
&gt; &gt; the lack of a better way to set a maximum?<br>
&gt; &gt;<br>
&gt; <br>
&gt; At that time, this was probably the most cost-effective approach.<br>
&gt; Enough and easy. But, if more nodes need to be supported in the<br>
&gt; future, it may bring more memory blocks. And this maximum value<br>
&gt; might not apply. The maximum may need to support dynamic extension.<br=
>
&gt; <br>
&gt; &gt;<br>
&gt; &gt; On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be re=
lated.<br>
&gt; &gt; In fact, what&#39;s the difference?<br>
&gt; &gt;<br>
&gt; &gt; NR_MEM_BANKS is the max number of memory banks (with or without<b=
r>
&gt; &gt; numa-node-id).<br>
&gt; &gt;<br>
&gt; &gt; NR_NODE_MEMBLKS is the max number of memory banks with NUMA suppo=
rt<br>
&gt; &gt; (with numa-node-id)?<br>
&gt; &gt;<br>
&gt; &gt; They are basically the same thing. On ARM I would just do:<br>
&gt; &gt;<br>
&gt; <br>
&gt; Probably not, NR_MEM_BANKS will count those memory ranges without<br>
&gt; numa-node-id in boot memory parsing stage (process_memory_node or<br>
&gt; EFI parser). But NR_NODE_MEMBLKS will only count those memory ranges<b=
r>
&gt; with numa-node-id.<br>
&gt; <br>
&gt; &gt; #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES *=
 2))<br>
&gt; &gt;<br>
&gt; &gt;<br>
<br>
Quote Julien&#39;s comment from HTML email to here:<br>
&quot; As you wrote above, the second part of the MAX is totally arbitrary.=
<br>
In fact, it is very likely than if you have more than 64 nodes, you may<br>
need a lot more than 2 regions per node.<br>
<br>
So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to NR_MEM_BANK=
S<br>
so it can be used by common code.<br>
&quot;<br>
<br>
But here comes the problem:<br>
How can we set the NR_MEM_BANKS maximum value, 128 seems an arbitrary too?<=
br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">T=
his is based on hardware we currently support (the last time we bumped the =
value was, IIRC, for Thunder-X). In the case of booting UEFI, we can get a =
lot of small ranges as we discover the RAM using the UEFI memory map.</div>=
<div dir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote"><b=
lockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px =
#ccc solid;padding-left:1ex">
If #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * N)? And what N should be.</=
blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">N woul=
d have to be the maximum number of ranges you can find in a NUMA node.</div=
><div dir=3D"auto"><br></div><div dir=3D"auto">We would also need to make s=
ure this doesn&#39;t break existing platforms. So N would have to be quite =
large or we need a MAX as Stefano suggested.</div><div dir=3D"auto"><br></d=
iv><div dir=3D"auto">But I would prefer to keep the existing 128 and allow =
to configure it at build time (not necessarily in this series). This avoid =
to have different way to define the value based NUMA vs non-NUMA.</div><div=
 dir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote"><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex">
<br>
&gt; &gt; And maybe the definition could be common with x86 if we define<br=
>
&gt; &gt; NR_MEM_BANKS to 128 on x86 too.<br>
&gt; <br>
&gt; Julien had comment here, I will continue in that email.<br>
</blockquote></div></div></div>

--0000000000007f1af905ccf5263f--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:42:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:42:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196425.349263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlH2-0004G5-Ev; Mon, 27 Sep 2021 07:42:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196425.349263; Mon, 27 Sep 2021 07:42:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlH2-0004Fy-BT; Mon, 27 Sep 2021 07:42:04 +0000
Received: by outflank-mailman (input) for mailman id 196425;
 Mon, 27 Sep 2021 07:42:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlH0-0004Fs-4N
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:42:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7e446ecd-ddaa-409c-8d02-c09fe806ff18;
 Mon, 27 Sep 2021 07:42:01 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-zSq9vyvgOH6xtsb7zHFtHA-1;
 Mon, 27 Sep 2021 09:41:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3391.eurprd04.prod.outlook.com (2603:10a6:803:3::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 07:41:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:41:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0004.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 07:41:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e446ecd-ddaa-409c-8d02-c09fe806ff18
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632728520;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6o1igZ1aVG15XpHjc0F0Q2prJ99ZMsbv9St3Ick7HT4=;
	b=WCHM3M9sv1nIV7CDP0Qkq832E0CRDj296L/JGc2PkH96vGzuLlB5pP8NcpAWQEQ9teVH0A
	QFGis+pNnIUplkkqoUB1vaeX5/Jmi757fNnavWUriytvTSbC/DfvIB75ssTsph7Y6yU5hd
	G2UlDT8YFpZBHMGeiuSNHvp0Oq2Ccak=
X-MC-Unique: zSq9vyvgOH6xtsb7zHFtHA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cYEqgQ7OklhOwI64W7r5OR/2PAVFQPNbKso26H42cT0A3MtGunw5DdA2IwbvNHihsVGGzLBPH3lFhYrj6Cl/xRvLt+8jDzd2GA7Ya1uwOt87yiqXaHBi8/GMphY7AKNykWCEYRQM8bLrSttgZ73PKzzc77lOFV0PZ3WzpYsSr970fwsFC6GNF7KpHfxOD9bnNxX6TdUp4zp0t8iSQJ6/cFMfu0FYj1wmeyI4vHtsYECggwBKtga39LK3Jbw7tK+bEFypAu7ET7wYafQS2rtWiYQHx4kV/MbK0tDf0muCUokDG/nP9ZQoydS6fiXdiy7RxswaaPqXpCMHqCV+AdsDxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6o1igZ1aVG15XpHjc0F0Q2prJ99ZMsbv9St3Ick7HT4=;
 b=ki9aPtqVyFfg0IEKp7zgQEEBtCJD4YryW2zWM7Fbavp8DRW7mQ4EAbXr2PA7M83AQRBWEVtzDDQNRCDMPnsQ7fFAQl/V7kqPxPDodzVLjE9s9F1MAl1Po9yl0LWrbNIm1HJYSQh3TqUTQ22kMq/ig7QY1pGl4cje9FJQMeHuWgDtdgHY5u94a4JcTS/ElytTY5q7fnAGp8B7xv9dFR/k8nhxJEN6he7tu4bSryAcXlYiJXzXYdWtB2eGK9NV3p27SnO9XMYyD3aRUojHA5KbEMZeix2QCPS74hj7gUPEDFJIDPw8H9yG9RI7+jFeCVgSrGwT2SRcXlBi3seFBuFqkw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 02/11] xen/arm: Add new device type for PCI
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-3-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <278da2e5-e55b-e36e-1210-d917bb5ede34@suse.com>
Date: Mon, 27 Sep 2021 09:41:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923125438.234162-3-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0004.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9260d593-6ffd-432d-4c74-08d9818a4826
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3391:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB339176932DA0744FFB077471B3A79@VI1PR0402MB3391.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vQUmnpCz2SdKwxEfSyZGwu4XWMt5UxNn4RxZlfA8YXxWvmJE3/7/6Dn2g8ETrYzx1NGG+7MKcQ3SG361fV2y4JL98qgv+FRVCO7rb3gsACBsI+DS4N1KB5UKgt+uWjA1JBfroMJ8YOJnVC2pdEg9v0yD76DG3dgJRGg9Eo1wb2bQxgqMsJVP5+lV9XWl/3dCqvPS4udcaVHXVDgdTOoJXyXqB0KtlLH7SMtTUAL7IYXW63Ef/Oveso1ww6+lsJTJg7HnJWWzUF5dW963hzLwjfNngwmQbvI5+PRI5GZcUGhNyPMH+MLfeCxm6cKFkocghjgJNalDzq7jXcDddTGzFNJO2/d/woWvxUfRWsu7VX/XPubSyTW6LyESyLr7eBPARuC4C61YL+q+D4PM0TEHCtU8MsTyMra9AJUo37oFVbKB2aeOSCVF4JGtkgxIjHzEpS/uFsfMLmdQawfnJra8AqMcooYn0fA3O85+t3kf73KKNI43f+k4PtHCIhnsYPcEEvtUezfj5aOfFMMaXXZOZzxvQqLfBmJtWnE+CxKSPZ/R9mr3evO7r1/Q22h0Jl2yaPY+VviJyhRDCPin6dtzFf6s8bc6LFFFTKuhnbNRZiIIHVKBk3512lPmH8He54+aGaQxI2vQOxeQBaNEOxEd1PlpYMYCVBkhr+GoazPd91l0hrnjqh7LJqaWZ10ivfBsWSzMb/T8jvfjFX8ae+4+jPnY+Z9sdAI+SjZKeF3LB4U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(316002)(53546011)(2906002)(4744005)(8676002)(8936002)(31686004)(5660300002)(4326008)(26005)(186003)(2616005)(956004)(86362001)(31696002)(508600001)(66946007)(6916009)(66556008)(66476007)(7416002)(36756003)(38100700002)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?U01LQmU5eWlFTnBjSGFSOHN3SEZMK3ZlVDZsTjdvbHJmNy9sWnVReUlaN3Br?=
 =?utf-8?B?T0xMQUdmOFVFWWhpUDJvVEM3T3VzdEpNUEU2b0tCcW9UbnVUVmJUYW1lR1ov?=
 =?utf-8?B?cGp6aU5DQ0ZNUnR2MW92RFpOZ0FDMGxrUU5raWpxZEdIVHJ1R2dLTVg3blQr?=
 =?utf-8?B?dzVCYnhNVlRJQ09IOGpneFgwUndjZ1g1L0NnRGlCR2xMOWpsOFUyWG44ZlVo?=
 =?utf-8?B?NXRnZ1ozcFk5WE13Z2txeVZuQi9SNDRpVG5DUTRWamdGNWh0SjZuREo1QjR4?=
 =?utf-8?B?KzBtanNkNC8yaEJicE5DeHVUaHpUQVJHdDZJbVFkUERvK2ZqYmtXSVRnMUgy?=
 =?utf-8?B?cnEvZ0hMa2E0ajM0N2hwTTUyNkhva0xpVjA2ejd5aGJadG9wVFVCNk56RnUr?=
 =?utf-8?B?TlN2MXpxb2xUK3lmSXJVOUhnYlNENVdkVk5BU2RmaUV4dWtDck9zTGhNaUc4?=
 =?utf-8?B?Tkk2V05FVHVEMlRqYVZRWFNJOTc0T1IwYmYxV3VLbnRLaklMa1M3TFh0SW5i?=
 =?utf-8?B?OVUyQ2ZrL2dPWld4VVZ3MHBRMHlZNmY4YlVFNUhUS1k5S25FNVhDT283SHB3?=
 =?utf-8?B?WmlPaEdZaDFJRG1NQ0VlSEpBVFRhYjM5QzQzQUhMM0ZkNU02VU8rUnpwU245?=
 =?utf-8?B?K3NQTmNuT3UzRkpnTVlOZG1HSzByc0FjU0hCSzkvb21TYjhDWXFqWktQOUxY?=
 =?utf-8?B?L3pyS1N1VW9SUHRsQVpHbXJmRWZtNXhtRzg5TlhBcTRsdHl5blBSRTFLNGJD?=
 =?utf-8?B?Z3kxUzZNZ3RLbDZLZ0RjdXRUSDVpTEpBK0RTS1BXZjByamQ4bHRxSXk3cVNT?=
 =?utf-8?B?cmdGZjRGdVdKa0o5KzZ1c0JTeHUvbyt2QnRVZTd2b2xBMDBNcUoveXIyM3E3?=
 =?utf-8?B?bHNCRjFySFJKNVJMa09NckxTWmRHb3pyUy9ycDY0YklkVG5TUWhoZDZKTkpn?=
 =?utf-8?B?aHoxbHBVdk43R1dNRVpUUllmQ1VMM0trVmI1M21rU05OeHd1U3lmR0dEVTd0?=
 =?utf-8?B?TWg2cWdKcXg2T1pGVTZFMWtpdFBndS9oRWNFYjlJaFR5ZkVJS3N2ZlRDQlBI?=
 =?utf-8?B?YUNDRUdPc2k4cFZFK1RDYjFZaFE5U3JhZEYxR3VoSlFPVzVqWHA2QjNsSTFL?=
 =?utf-8?B?aEE5aVJDYno1Q0lwR1dkM2YwOU56R1dUOWdaYjZYeVVvaTVuYVQySFh0Y2JU?=
 =?utf-8?B?WUp6bVlRd091Slk1Q0NRWDdUMkF2dFJYdVJIMDZ5aDUrYXhNdFlkVUZXK3VK?=
 =?utf-8?B?MndnTStsOUZaRzlVK1crMlZocmQ2bmlSZEV1dWpIZW9iN050a3F1Q0hNM0lV?=
 =?utf-8?B?d0dvL2M1Z2FpL0ZjRjN5RTc5Tzc4b2pqaS9CUFJFanZFNTh0L2U2YUYrV0V1?=
 =?utf-8?B?aFJycVhXaWNmTTBWUFJiQ20rZTNmaG9La2ZPUk1SWi9TNU9mS2p6U2pJV2NP?=
 =?utf-8?B?QUtOMWp4WmlVdE5mcTB3WU14V2MzUHBjUHZrcnd0QUpMaE05aXBLcUdjamlP?=
 =?utf-8?B?c3hoNUVOalcrd0s1aXNCdUxhMk1Td0hsUFNGQUx6dzFrZ28zY2tXeEpiaFQ2?=
 =?utf-8?B?UjgxeEdlSlVlT092dkZXZEZuVms1WDJYUG9xcFlNRWNadDdtUlV1OVdlWTMy?=
 =?utf-8?B?NnlhcnJGS1FsTlFoOWNoVkFhV0huTW9sWDlJT2NjYnlHWWJnL0FIelh6bUZp?=
 =?utf-8?B?bTRtbTdzS1djNTFBb2hyR2FQbFJDTGowMkZ0M3hHUGU3YTFsaTVGWHBLZWp4?=
 =?utf-8?Q?3GMF6ss6LuOLuapqUEs7ya8uMYP7bPJHDd+U7fu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9260d593-6ffd-432d-4c74-08d9818a4826
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:41:56.7234
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OKdp2mP5t0Cm3tcw4fkVvhvWPXiRCSmamlNos5qG+BzrEhPUG6Z4ijMG1JEQ2abN+S1s5MnOTcJUif2P6fSmGw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3391

On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -27,6 +27,16 @@ int arch_pci_clean_pirqs(struct domain *d)
>      return 0;
>  }
>  
> +struct pci_dev *dev_to_pci(struct device *dev)
> +{
> +    struct arch_pci_dev *arch_dev;
> +
> +    ASSERT(dev->type == DEV_PCI);
> +
> +    arch_dev = container_of((dev), struct arch_pci_dev, dev);

Nit: This not being a macro, the inner parentheses aren't needed.

> +    return container_of(arch_dev, struct pci_dev, arch);

Two successive container_of() on the same pointer look odd. Is
there a reason a single one wouldn't do?

    return container_of(dev, struct pci_dev, arch.dev);

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:46:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:46:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196431.349273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlKo-0004uR-V7; Mon, 27 Sep 2021 07:45:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196431.349273; Mon, 27 Sep 2021 07:45:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlKo-0004uK-S3; Mon, 27 Sep 2021 07:45:58 +0000
Received: by outflank-mailman (input) for mailman id 196431;
 Mon, 27 Sep 2021 07:45:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlKn-0004uE-R2
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:45:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e42d40b8-2252-43d2-8624-76bb858806f5;
 Mon, 27 Sep 2021 07:45:56 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-rB0Csll9MGaVqHBu945SqA-1; Mon, 27 Sep 2021 09:45:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7087.eurprd04.prod.outlook.com (2603:10a6:800:12a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 27 Sep
 2021 07:45:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:45:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0001.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Mon, 27 Sep 2021 07:45:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e42d40b8-2252-43d2-8624-76bb858806f5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632728755;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AwNwRqoEtd1Tv6Fa40/+aQ9YyaXVlHlq/LvEhc6yWpU=;
	b=O3mlmvEKQA47Wwn5Xkdovbo/DqXsTfPiKAeEcQGkqOE+4nsWyfbTtQfeRv0SB5hPEqa8Hm
	IKGnVy0Hv8DnoeXJTSTDKytftJARLhGfW8iiBWrA7MzSi6At/6VYX4r8A+zrcdhlq5+AZG
	4R2y8/94F7ioiDy6fuAMDcYkf7ELFJg=
X-MC-Unique: rB0Csll9MGaVqHBu945SqA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P0lfuZ29sJCzQbucmKptyVTdCraY0tHr8fgK9KBYOJOdkC8T7p2bUa4gs9Sqz95FOhe2/qcuC0U3WjheZ96KqDRpf9WJRNZFKSehyhzz8R3NJsiu7qPjF+ZSy0cR43K6ZMjp5TQETaaH2k3dSDyxz2GnHWgJ2pefZIJ3/ShoRj6Aq02gvRrTDQfjn47JuNkNupl9+7gcArptKiRETTmww42jVtVsJ6E7rC62bm8bw+UV3lEY6/w2LSwFIm6cFMMTxlIqnK6FTnMxsLPpI0TyPOgYYaYRv8KW1dyNgfQxpmZJ2Y6AAVP7eerIiPNYQHL+KdUdQRdg9fwEQYz1LplNZg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AwNwRqoEtd1Tv6Fa40/+aQ9YyaXVlHlq/LvEhc6yWpU=;
 b=nRsFXQGh0PwUZsQj8No+xtZH5XOuJz/o5etfwmKERs/XlXib5L7JlbO0WgM/SLjQUYZN0/D84iLeDwvLrjlmSjBA0u2eePVMcefbAmaHtijDCYPbFRRaEk/PViuRnBDN4Nhou4GRmvYkjyoKVnPA92y0y4e5exW/CycOd7hX8A7CdsOLD5wUXaYxpjAhxj1k4QrHw4SOf64hSzGz09qrJ3kRJWSjg9IvIqeDTpz4Wc4BIzW9KfIWxopRwvy6aOYn1cJqYu+ky3jjRgo+AqXz6YAbDTjO5Saxpi0N5xes9reblCySvwpxCjnV+srH0r4UQ04E5H2J966yH7IgQlk0BQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
Date: Mon, 27 Sep 2021 09:45:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923125438.234162-6-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0001.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 84099f6c-2308-40d5-ad20-08d9818ad4dd
X-MS-TrafficTypeDiagnostic: VI1PR04MB7087:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7087F7D8B6DDE796C64F4BE6B3A79@VI1PR04MB7087.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4125;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TDsGo2XHUylb4qGAW6H/7ByA0+HFqMiR/JMxirpoK1Po6v83w4axmG5HXOzK8f4Z/XjQ3561bGbde0yKIORtnBQ5FbDPJyAwz/8BEsEzHj2QsVMntPsQXATwb3qaq6Janf2M0Izl9tagn2EGOg6ZMCnCZoURFpo3xyagairXd0ToYrNplTFatUMlRDhsQR/EeF5THdrBeHGe1yFP4C/1r6KbkVtW2tFF8LKyRMGUglpm31qSdwFgZOfJZmZGwd0CMgEZfrtgUh3ExeF7mWH6yWOXhcyv/14z7px1kCDXakpuCMxYk1sucDGgJvmaTW3ERy6QiHxt9FFk/plpN1RVyHNAdNIyGCGGSyLPKxcI8vZi0pNO77T9Fb1CXjFKp2PM8sk0UZL+3ovGbNP2gH/zTpivg2R32IXKS/GzFvDnwn7X4L8UDLBYRQ7NQyIOn2dPSMDQfUnlmowRga3sii71ADa1KMdkz5u0PQmLAK1+liNA/W6Z1dOAAbNFf+jcYWZO5L8GgI0eJh8dJqcwRFChOfrDl5IYnNmB/aJetMiowg8TUYvgiAvpSMrC6bRuF5oR3SQBHOcxevzrDBayqRgQiYYml+78G1rqJNNm3gl5j3PTbz7Fb5lEZZiVnodLkt0cxyLOEsDHdefldbnTdb0r1hz8DNRSSKN68sTXZBvFlyV/4QBXfwdKqS6xyQDm4Os7C+wUILDdrpVtWYbAjpwk+rQW6q6ezW8nb7bEHw00HnI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(186003)(2616005)(316002)(16576012)(956004)(8936002)(2906002)(31686004)(53546011)(7416002)(66476007)(31696002)(38100700002)(4744005)(36756003)(6916009)(4326008)(6486002)(5660300002)(26005)(66556008)(86362001)(66946007)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bkxCeGt5Q0pnTThHZzhNOUdkNkZvM2l6bFF6WUhRUUg2bkNwYnM5aHNReWRl?=
 =?utf-8?B?NFJHYnFBOHMxYVYwRW42dVF2ekRMMDVBUk1qQitxZ0wzZnBDbXJOa09rWVpD?=
 =?utf-8?B?R084V05yVjZ2Z3J6UEhiMkZOWTZ0cklWbEtITVRyTVhxMG5Vei9NYXVQdmFS?=
 =?utf-8?B?cjBxRXRqRnU2V3ZFaXI1cDlmbGdjbmtVSDNFMGZ6U2JDNWJYTXhXNHV6M1VI?=
 =?utf-8?B?UytSVk1XcGZXTXdzaVk4MC93WmxZYnZLQ3VrK0xHRVo0bUViUUx5WkdzQUtj?=
 =?utf-8?B?RlJpeGptalBBaDdQNktJVGpoNGdaeE1pT3ZNSGhzVmZnazl1TlU1aEFucnFS?=
 =?utf-8?B?Y044bkhSRDVWclRtSldJNCtEMzEzNldjSTZFQ3Nna0hZTzJ1UUFScHhEQkdu?=
 =?utf-8?B?b1dibXpDU2U2T0d6Z0hBUkhpRVJ2cWNJdmhSdFAwcjl3V0N6aHlmaEVUcE5T?=
 =?utf-8?B?RVhsZmVOSGFnMncveWg0MDR6MTRxa1IwYTlVcFB3WVJlaXJSSVBSWUlTU3hy?=
 =?utf-8?B?dzhVbU5xU3Q2Um5RbmxtSEFoNkNEeDEvSEF1eFUxRjVSSWF5bldseHlRYXV2?=
 =?utf-8?B?eXhIbXAxVzdqRldhWTN4WERmTiswSU1ZSnRLM0NTQmxZcngxb3NqQXhCSmh6?=
 =?utf-8?B?QmlLeHIxWUZzSFRqLzZ0MDBJcERIaDBMQkQ0TjlHY1lnOGRabklBMlgvSjJX?=
 =?utf-8?B?VmEvT1BYL0gvT3dBZVJUWG1LdWdTT0l3bTZxOEROSkVlczR2ZllvZm94eXZw?=
 =?utf-8?B?QVNjMnlGN2VXTTI0SG0xbUlQWWNIbTNFUkFtOGFmdWZyL3loUXdQZWpENVZp?=
 =?utf-8?B?bXVMTUFYajBKR0dXeFRPSTZILzlJU1I3akxRNTgxNTN3UnNQNytFWUNBaGZT?=
 =?utf-8?B?amgrcjBJbS93S2NtNjBvWmkxSGtRTHBsVzNNaWdVR2srNmpkTVdhTGFCc3Nn?=
 =?utf-8?B?NDlUWXU3aVpLTnF2SXpISlBvdjNhSVVxL3RJM2krSnM2dGhyL0RCMkxNQnRP?=
 =?utf-8?B?dE5Nc0lFMDQxeU1xbHozTDFDOEZiajFnT3RCRHN1WVFXelJSOTRVK3Bud05k?=
 =?utf-8?B?MmwvYWhBOHU3ZGhCMklsdy9Yb3BXa2lYT0pYWlR6SHV3SWs5dWNuM05vQ0Nu?=
 =?utf-8?B?aUZwazZhdjE0NWZsbmI3R3JhOTNta0svY2FEYU00RjFvNXRSSGE4NVNxSnE2?=
 =?utf-8?B?Y1dTblBTRG1VOTNSVFlDcHdiOVY2MVpjakZZMFA0eUFISWwzK0EvQmEzMVRK?=
 =?utf-8?B?cmhHMHJqWVlpTmRLamNFZnA1NjJ2YU95UWxFKzdPSWdJMGMvR2V0cmtNN2V1?=
 =?utf-8?B?ZHRqVmk2eE5EQXlUN2U1VFdIeFBqcHg5Q2RpTjg0N3pQUE9kMisrbWRHcTBP?=
 =?utf-8?B?T1puTndjTlhFQ2o1N1BsSjZjR0dJN3hjaXJNeU9BY3FwTXl1N3YvUDFJdnkx?=
 =?utf-8?B?Ym02UURxbU5weldSWTBNS003eGdtUXZXbU1UZmQ1eVJDUlcveUhETkdrV1c1?=
 =?utf-8?B?b0VuZDh6TGhzampoYWVkWWdIZzlkbDNEZkhhd0M1dVpLUGtibmlsWktZY1ZL?=
 =?utf-8?B?MWhtcG82Q1pYSnhFZHZySm5rbVJxbzZLOUU1RzlWaVRzenZsVmlZdU5wUW13?=
 =?utf-8?B?WG9FVk5nZ1Q5U1VZaWtCalBxSkVPYTJWYUxZSUFtQWdHWFl1VGVNMTNTV1R5?=
 =?utf-8?B?Y2pXMlNrU0ZBMUJOWWdoWVRjVkdla09jMFU5TFVXQUdUNFNPV1ZsNithZzFL?=
 =?utf-8?Q?X0sOl5QKxdHd0jtvcz1UJI+FMOGntm5w/oLjct1?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 84099f6c-2308-40d5-ad20-08d9818ad4dd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:45:52.9041
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: J8g5uWHerCuZvF8Om25/NGJf0Fn4ARGpjLYK5Libj6xKXyzBFE5LyLvD7o4mz4bVNeM3BpEfRCOizNXWXTq2tg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7087

On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>      *((u8*) &pdev->bus) = bus;
>      *((u8*) &pdev->devfn) = devfn;
>      pdev->domain = NULL;
> +#ifdef CONFIG_ARM
> +    pci_to_dev(pdev)->type = DEV_PCI;
> +#endif

I have to admit that I'm not happy about new CONFIG_<arch> conditionals
here. I'd prefer to see this done by a new arch helper, unless there are
obstacles I'm overlooking.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:48:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:48:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196441.349285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlMx-0005kJ-CS; Mon, 27 Sep 2021 07:48:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196441.349285; Mon, 27 Sep 2021 07:48:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlMx-0005kC-87; Mon, 27 Sep 2021 07:48:11 +0000
Received: by outflank-mailman (input) for mailman id 196441;
 Mon, 27 Sep 2021 07:48:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlMw-0005k6-1D
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:48:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ddee715e-87bf-4987-800b-730a46a797b6;
 Mon, 27 Sep 2021 07:48:09 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2052.outbound.protection.outlook.com [104.47.5.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-PCpniL9XNvqwcCco3rh3wQ-1;
 Mon, 27 Sep 2021 09:48:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 07:32:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:32:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0015.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Mon, 27 Sep 2021 07:32:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddee715e-87bf-4987-800b-730a46a797b6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632728888;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FC8US9h+dvFJf/b7A4Sa/oFEziZP/JSAVuR1fpnU+7s=;
	b=USVYM1IIJk+BePAUqrBcj9ZRSeZo5cBIwCCuS1yjSc5MyliCMXVOvdQDzym8CEPRMBQmT5
	kgU8PSe2+pEcp5bwD/BLiAXTggXKE42R6ImzTIyiStih6WKlS04jUxqjmMDAGhFK1E2SKX
	P9sqnCKXOZSsW6r50kKTQU6tiAmbCAc=
X-MC-Unique: PCpniL9XNvqwcCco3rh3wQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n69m6RUPHi12saa+Z5aDpMW5fZjOV/wadpPq/k8VCWXV+R2MlRWTSXY95rdxUJ3S3M5KH5U3vHdsO78yQEmMe1sm04bVpafGskyUTXq+a5pR81QLypH/1R9Sb28EHpucNPudQBhLGkqAxvqRxepAmnfzcQTB2d9lWCO83GjJcge6B/HlCCAkD7As7vdZR3mfNWCnrKNuAWryJu7by7Fr89YM2NnBWsF9E1ADFVCQONiomEm/ZGt5mrg1pomGVvlr4QvsxwR7SnYOrJBN+WuNCoqIqLRj9QZVr3MuuoSgSlnkA4anM2wmKbLNAUyJO3kdmgNch0O+h77WnRwwbFZ84g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FC8US9h+dvFJf/b7A4Sa/oFEziZP/JSAVuR1fpnU+7s=;
 b=Ro9MqhTRcWlwraP+eM8G99v2vEjVtR2HVrLgrV4CQktaA6lPz5/2DqIHXD88BZdVvoZBXpz9QXZfp1NCxYRWVRdOUvy2kSBaDeWcKLel+YNCKvdSyROkPXO+NcNOsqTHLUrB2TC0hXXleGBdieBUJC9eS8hEGDcW0w8aHSc71ESejwW3m5QwNUxaE4smCm5HB8d/iRvgzHOq+uq8/NKyLHUWCHe6dFLjvXzZqEudSOS2qxtv21kFu5nMEhs3K6Hk6NJUHmqhWV6td5iWK+btxGSgeiSUJAsacfaDds97Y+KiD/o+SjMrgoz4RZ4KpM7RPKzLRMgW6MNkr8SILxMK2w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: linux-foundation.org; dkim=none (message not signed)
 header.d=none;linux-foundation.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [GIT PULL] xen: branch for v5.15-rc3
To: Juergen Gross <jgross@suse.com>
Cc: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
 boris.ostrovsky@oracle.com, torvalds@linux-foundation.org
References: <20210925143126.26439-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1402ea5e-d392-410f-16a6-ea3d4d7a3adb@suse.com>
Date: Mon, 27 Sep 2021 09:32:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210925143126.26439-1-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0015.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 40aa3432-47af-4e0a-cbc8-08d98188f5dc
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70242BB56F59A25BE043CC68B3A79@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TVWYkDkg2sn+yj17PUpJs1T8jC+QurttojVk0xkhuvdBcDXl5+S2kwj7L55LIC05AFUs2k2wiUZkhez1vCmIKJBp0NE1ryJoPPYwR9XJSckWDc5+c9KmDDEVT4UKkD/Wb58ORfqZyy7HOHPp6dA3OU7lmX7OllZTsPXsY53J7zBc98/uSSsAurZH5NazEtjFPEqmMMO09Yq3QBhNB9koI5+dr7sfQWrL8Tz48D1TsTHJ6JstjovyqMpi1nK0y1LUWJJkShIwEeLqf78fK+09xwfNj/8GmPFjnM3ZWnnWqFQ5dCQjCPWVw6wnN7ZGn4JwufPieLZJkmvlTAAo/enHspe0iAq6Ex5gazsCMCmEl3T62NwzAtRlDI3R4SI0KXJCMySUa7lNP9QNK5S7sqD5PxFQZ6JB2QTgRvPu8H3zgbU/Id7AwNHWSdfJnf/hTMKSkjseUMeN7GuSyh0QvO6iiupZnggOf3bpdiMSRL25Cqg25Gp5eAO2mUpP9vXzA2mL5DRsPaZuUbK3QYYRSykgc2Lo9WJy3DBTpU2sU1f+nkx9gGYY5ZH5fzYAqUu8XKHQj1Lda0oxpch64KaFePPZbKFCKJHZo3+MNCyaole76CFXszidkpNGxzaJPg1kBBw8CWTrJG8MhbkG5efZmxRkR6OrjCArhuJDAY0Ws2yTdsQZe3Pc5OmQY8M4ufw0HzoOZ2KAYYbcQuxMlN6Ud5QTZtQFKMssoQN/j7qV4LCBbPk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6666004)(2906002)(6862004)(8936002)(53546011)(66946007)(37006003)(31686004)(6636002)(38100700002)(5660300002)(83380400001)(4744005)(316002)(16576012)(956004)(26005)(36756003)(508600001)(66476007)(186003)(66556008)(4326008)(31696002)(8676002)(86362001)(2616005)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TEZab3lEcFR4TnNRSEFiSUZaVnMzVXNqM0UrRC9hQnNRSHFNV08xS09ZV0xT?=
 =?utf-8?B?TkRObWZYTHBPVFcxemd4V0IwY3p0a2R6RFY5NFdvWUVsc0R1K2YrSFdlbXEx?=
 =?utf-8?B?YW1uek5lQUhXMzM1QkZJQytTQk1qTjNXNWpqSmxrSWI2RjNpWmtaaEM0RURC?=
 =?utf-8?B?eFNZaXc4QmJOVmp4UTd6MWZPVjhyNjM0VDlNVWVlb3dwelQ3aWdlaktMZzF6?=
 =?utf-8?B?alRia1JiYUxSektnVTdtNXpVUlgrbHZ0WmFxanNIMGMyRWYyWWlJak1aeGtN?=
 =?utf-8?B?aU80VER3b2lmVkJ4RGNFR004ZThPWHFaV1hCMTUzc2J2K0dQenhaRUQvc3Nv?=
 =?utf-8?B?ZjFOeWdjVHlNZUdBZjRyVDBZZXhMYnRuRi9wcWtkWnRRd0QvQy8xSmplUkpn?=
 =?utf-8?B?SjJ0OFdTbnpVTUplTG5aN0lLNDhCTVlrdlRpVExac0xKcTZ5U0YvdytnWHRo?=
 =?utf-8?B?a0d0ckU5OHBvNTdXbmx1QjR5RG1NZisvSnVlRnN6aDNaWFdISGU0OTIzVHZP?=
 =?utf-8?B?Z2NFUlY5cktTdjJab0VvdkgrV1QrRXJRbkpYVml2VFZGMno0RWVlQWJvSThS?=
 =?utf-8?B?bW5aQmxDWnMzL1IvRXBDRE5sVlFsdmtsVkRiaXgvckduQWRzekU5WUluZ1NN?=
 =?utf-8?B?MEpTT3hXUm9KNjN1ZXI3blJxV1ZPRWhxMzBHaW5XSS9GaURFZlZXUktqS2ZZ?=
 =?utf-8?B?Z3FmTm1uSFRSVTdwQm5XWGtNQllxUXI4RVN3Vk16ditVOVowYkJZanNFN2sy?=
 =?utf-8?B?bUtXa1JxTkE0T1FTR3dJSms4VDVPdGI3S0VrcGt1b0JwNEsvNE9odjZkN21V?=
 =?utf-8?B?Nmh3K3A5d3ZqUUJVNExXc1AxOHdaYWkvM3B6K3dpM1RXbWhnemRnWXc0WUFD?=
 =?utf-8?B?VS8vR2dyYVBwVDlmVWY4MDViYXVYdmZwRzFEdmt6ZlFuM2NnbWFtNTFOR1I1?=
 =?utf-8?B?UndtaVR1TTFvR0V4b2RnT2lQcThXSGoyQlg3bldTOFY0QU4wKy83RSt3Qk9M?=
 =?utf-8?B?eCtocVV0aGd2azJSZUpWSSthV3pZSS9ES3ZiMlIxaGhWYmVDbkpUNCtvT05I?=
 =?utf-8?B?cmhPVkRraDdJVXREM3NhK0I2R01rc0c5Ym5UMUNvNFdNdnFNcGxscVQxVzQr?=
 =?utf-8?B?WHphK2MwTFk2K3l1aE1maUNNZWo3V0llenBmRnFtYXN1cXdhRmJjMk1qTWhC?=
 =?utf-8?B?L293RlA2Vzc0TFJYVmpRc2VFaEgvQ1FTQkYycjN6bXdmaFBVLzd1VmtlelV1?=
 =?utf-8?B?ekhpSEtYNGhiUjNOVkl5TGtXWnBSWExHNVVMN3ZiL2tyOFhBd0NiTW42cEww?=
 =?utf-8?B?YW82ZXhvdi83cHFsUm5QQUpoQi9pNm1aUjI0SzNPc09Ydk5vVDJaODNicG96?=
 =?utf-8?B?MEtIZzdZTEFkSGluKzVFRHFxUFdSNXJxQlV6N2VGdktkMk5NRDBxbW1ES052?=
 =?utf-8?B?a2lWUFJiRTBmdC91aUxoTTlaVDhRUDJ4anlIdWl0RVBPNGtsVUdOZGg3WGxh?=
 =?utf-8?B?dnFGRlN4ajdibHg3aTZtSDNGbWQ2MmJwelU2d0xJODZQcHl2WTVjTFE1K3Bi?=
 =?utf-8?B?VlFmREVjZ3EyMTRJdmh3NnVabkVaWC95OC9WWGpCN2N6U0hRbHc2UitsbkdX?=
 =?utf-8?B?dDNqOU5hLys0NGpsV3dKRVM3SDQyL0VwRXc3ZUxHbnpaVld3dUQ1NjdLNXEz?=
 =?utf-8?B?SDVMOHBGRzcza3dSVmphaS9vNjVoczhMbHBkNVNETGtEamRUSkIrbG45Z3l3?=
 =?utf-8?Q?OodfJmw0TkeofvqMZJt01gyKdaI7DCb9m9wFIhb?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40aa3432-47af-4e0a-cbc8-08d98188f5dc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:32:29.1678
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VpUjKOt+pB6BtZED2azVAKKQGEgQSPsBpJa9yqlo+3LnkNkARN6vcyEk5HUtCgaEMkJZUFBNxWAfM8dQGcg1jw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 25.09.2021 16:31, Juergen Gross wrote:
> Linus,
> 
> Please git pull the following tag:
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-5.15b-rc3-tag
> 
> xen: branch for v5.15-rc3
> 
> It contains some minor cleanups and fixes of some theoretical bugs, as
> well as a fix of a bug introduced in 5.15-rc1.

Just FTR that the above doesn't apply to ...

> Jan Beulich (7):
>       xen/x86: drop redundant zeroing from cpu_initialize_context()
>       Xen/gntdev: don't ignore kernel unmapping error
>       swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
>       PCI: only build xen-pcifront in PV-enabled environments
>       xen/pci-swiotlb: reduce visibility of symbols
>       swiotlb-xen: this is PV-only on x86
>       xen/x86: fix PV trap handling on secondary processors

... the last one here, which fixes a regression introduced in 5.14 and
a long-standing underlying bug.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:49:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:49:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196448.349296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlNk-0006Oo-Ny; Mon, 27 Sep 2021 07:49:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196448.349296; Mon, 27 Sep 2021 07:49:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlNk-0006Oh-KN; Mon, 27 Sep 2021 07:49:00 +0000
Received: by outflank-mailman (input) for mailman id 196448;
 Mon, 27 Sep 2021 07:48:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlNj-0006Ob-KD
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:48:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 97fd1c9e-fd0b-43ac-ba4e-114f95c15f80;
 Mon, 27 Sep 2021 07:48:58 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2055.outbound.protection.outlook.com [104.47.5.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-kKUSX5cKN3mZZjaxpD4jzA-2; Mon, 27 Sep 2021 09:48:56 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 07:48:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:48:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0075.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Mon, 27 Sep 2021 07:48:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97fd1c9e-fd0b-43ac-ba4e-114f95c15f80
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632728937;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4rwhkrQJ9kLkgjnOKQPT+JHOelIXQyjB/Af+cIBoY/g=;
	b=ShR6AMct/APX/jPoBSwYpkF5v29tUMphPLWFtXfX/e92OSXJM/GiggA7U58Dl50A4L7fAm
	yTrRvEBHou6Aae9B/oc1zpvhbaxU72p606E3fNkpmkzKxuhgVPtJ3dS0rlrnLpphC7dZi/
	/sd2u50bz2xS/zrRAWm9tRDL1I9dbvM=
X-MC-Unique: kKUSX5cKN3mZZjaxpD4jzA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kHi0XFL68jt+EhgIJVb+oysbP6DiqEkr59EOnu7SnCO48cWiGGoDqrGp5yXhr6ORc3Zgpf4DxSjDO+is6tnXbeUuM7+Wb8zMlTeEzKmDajF/Ss3KY30bgkwopQSjQXJiBa9UeHuFOX/J1YeLvaxronfDjfkwdLtlrR+TsU+fTL9/Cw4e23VFxpUoew9pqUROPAPdRm7DByKSD/LQPc9nxeR87ZDKrwUuDaZsj08hNaKw2HAQkKhw2Zf6jcnt9YQ3znMofxFP3VNJCYHjpukGanvnESkbQ5h8vTDV2R+SW/cLdOn56GxV54rSIxyfdtjjBMgLNSENvHmvUBKSpBJWnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4rwhkrQJ9kLkgjnOKQPT+JHOelIXQyjB/Af+cIBoY/g=;
 b=Ui9Mpp6RIXpW81lQegAykOes9pw0Iiqz51xWzponhwScX49iPEN8C/1H9+q/JzHowXYG16X7+pXEX2MnsAxuhI6KTy6ekbVQVEmf7FSg87lDTHEmFitCorW9973C2T3SIFGZxHwEHnzUZGSWVtoJ4PDQPrRtq95w4g8Vx49uBMQOSSEUcuY8YSekKRiuMGMyRWSTqhMb2uV9mfMoUmNht/R2S1ksQIxfZriMI2FEFWW/tikIqAbtjIYZEJ5OhGdJsRZHITSnDx2mBuJJS+BMsC1P3VVsS62gJGh4x7N0SbpwZZAhw1qDyW1iHKWgVNE2xJSvChvyAI5jqoxmihulBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: vger.kernel.org; dkim=none (message not signed)
 header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>, "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <750b9449-c986-025d-9b68-dd0f6967b083@suse.com>
Date: Mon, 27 Sep 2021 09:48:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0075.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1e::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9c2da389-15c7-47f3-0484-08d9818b3fe9
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024C32884B01F9305800CC5B3A79@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	orU9aeimQKbvP0ORGrN6x4BJ6DTvZlYYG2R3E+Uht1UAXYYeZKik08+kjapYQ/BUQ0D1EEzLws//aqmpK/cbDnHL30h7h65Mh8GmihE50J7twetG8m3U0r6D1vkYKYPXXTcfwhRfvmcgv353abf4NJvugQHmE7AkCjQhn9mEaQNfZ1BQYwPpcvN+yJsD9c1MSuP5KrTyicLaeZ+s13SqxWAuyjfrqTRrGtJAGGsKn2lf/yJ0L8pC6jaJKO9BFjV/g28/6D5M1J1RkNA4QM+QSeRgOTrC+yvJyJ+86cERJMAf32zcQKHgs+iYDPbPYr73AkcI+rOaYaxi0pKOd3LM1vuafObymshJbWj4/dmz9iZ1JWBxxn6UQmo2mel/5g0FQw/Ay5AH2XWLWqmeUkexSVOSeov3rgkB6t0TNkkNlpnn1yj48AED8a+/FP4OkmzJYhTrSEKg9+yPhXT2WwVD/nWNzxbSb6FoNTVXZKWcIoB5XofSWIBSFcGQwQ7DMsxbu6XhwzhHnpsEpNZpGQyAjgqlbkPH3k0+SKFqT2Bh9i8DQAH9ExsV3j5QesqSHTJhL0meOtw8LoBubKDsV7c4CPa01eSPtamvCOQafzXv/gdaxj7unuVngVRup4iyGeSCWSG3udJMSyV4JTqQwSjCUMNeT32mxSIGYa7FI0Joz1cK0mmPn3gq4p+75/SCSlIHkJbvAg1j8QEOrz+e9yTIC4pfjNFVzVyh2ALm/w4Xwy4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(508600001)(316002)(16576012)(956004)(26005)(2616005)(86362001)(6486002)(186003)(66556008)(66476007)(8676002)(4326008)(31696002)(66946007)(53546011)(31686004)(2906002)(8936002)(83380400001)(5660300002)(54906003)(38100700002)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?czVoMkpiM0NUTlVqODRjWHBhOXpRN3MycENwMU9ZeFBra25BcC9pU1FOSnl1?=
 =?utf-8?B?QXNQeDZHbUxScFQwTmZHcGJrZFRuaEVSZmZvVFg2N256cTVHVncrMlhhd29p?=
 =?utf-8?B?cmh2bGRxaHdQcWJ1UTA0QmVNMHZkUW5EaHBBU0ZTdmpYc0RZa2lRV21YRUhG?=
 =?utf-8?B?Q2JmcVVFaDAvSzQ5L2JRRkthcWM5bENEWkt0bmczdGxiZGFsenlXTStEZUZy?=
 =?utf-8?B?cE1BZzJrb0xmR1YzRmxMQ0k2RklGWGxpSVR2dU9QWGQwMytiV1REc3JGUzNo?=
 =?utf-8?B?d3VsTktLOVJtU1FqVm5tSjRqUnRFTDVvZVpwcTRuSGRqMXR4T3RGeWlzMDdE?=
 =?utf-8?B?eEJud2ZIejUzQzhhQ0cyUVZQTEVmQUJVN0NTdEpGWWJjaUNRTi9lTDJobXpw?=
 =?utf-8?B?RThSUzJVeVQrWXdtdWxWN2JhbVBVUVNrN1hQa095MjZaZmNxNVI5ZlN0NkZm?=
 =?utf-8?B?RUR2N1hyVUkxeDdKMnBVT01YV1htUXpNSDZxZGtxN2dBbUxuQkQ0K1JXWWdi?=
 =?utf-8?B?Nm5VanE4Yld0c2JLSjhpTmJBVkhLNUgyV0IrNEdZWEVhc2RvZGwzYUZqY1ZI?=
 =?utf-8?B?Q25jbTZiVkUrUFFEV1draW0wWW4xWDVEbW5LUE5WbHZRcGZFRmN0Q29hWlhv?=
 =?utf-8?B?K21USHlIVmx2Sllncm1MZkJEQzJnTi9oVXdnUit5bjRTdVNPUjZBWERZL0NT?=
 =?utf-8?B?M1RsRkt0cHA1YnErelJCb0NzRzZ4aHFGeDY4a244eHZTTkVTL2tyY2I2NzJx?=
 =?utf-8?B?dVBzSzVSS1c0SWlPajhJVnRIWU5zN0c0U29mTVlFVFUvckIvdlNUZVZ0SHlz?=
 =?utf-8?B?R2hra2pOcjBXaGpFaVFwcnAyemI2VzdncUpsK3p2dVNYZmFIVjVVNWhPS2sx?=
 =?utf-8?B?SEdSditiSW9HN2NTY200dzJJTU5kdHpKandxN0UvY2FPVE5COHFwekxhN3Rq?=
 =?utf-8?B?ZDJDUTlLMWpwamJ3a25SZWN5dWU1ZkRFWFM1R0hNNTFpQUV4d09VaGs4ckF1?=
 =?utf-8?B?Ykt1U2QwR3NQV3puMjRaSFNpVmdqZitEVFh4TVZYdWJXTytDYjR6Y25rSmRw?=
 =?utf-8?B?WWw2QkFTcGxhK3lmYXlHS0ZaYmlFdjkzTU9QOGgzZjNzVmFPdnE3QkNGeFky?=
 =?utf-8?B?SFI5U29aNGRBRi85ZmthckJVWmRwMjYwNkJ1V0xQQ20vWENzQzNjL3RUTDE5?=
 =?utf-8?B?VzhVdHEwRUhabGVhUW5PSEJGYXp3bkgzRmd5L1htdUdKTThtSzR4UWlKVWJZ?=
 =?utf-8?B?aUk0YjFhZEZsRnF5ZkV6UWs2R1NTejlmejlKWkN4NmhlRzBDZ2FOMGVkVWxo?=
 =?utf-8?B?TGJNMlR3dzBKdkx3TnBXY3FWVS9LY3c5WEFMS3gyWWYwNnJTSWxnVWwzNDdo?=
 =?utf-8?B?c3dxSXdwRk1abXNYbUczMXdkM2FDK3lMa0U2QVRyUjEyTktnTWdPRzZReWNV?=
 =?utf-8?B?a1VnWWdLeG9KSDBSd29aTHFNaVk0b2tmMTUzenBpNkx4OVZMdlBOSEZuTzNP?=
 =?utf-8?B?bndTWnMwS00zbnFOWjcvNW14UVJsSHhUTC81RWxrc0l0M21CK0cwZS81OElh?=
 =?utf-8?B?dVdnci8zTDlGVzJka1pMOCtXb0NkS0pVTjY5aTl6T3Q1a0JWY0ZiV2huek5m?=
 =?utf-8?B?Z00zMDRvQ0NPeXMxT2ZOVERpZUFsSzdnRGV3YSsxY1lic0JZWkVVYWpoOEFT?=
 =?utf-8?B?bS9hUUZPSHp4OWJ3dW5ENEp5V2ZNVEhSQ3N3R05jWWNGWUx4T1FsdjZnSlow?=
 =?utf-8?Q?EKWNPoY/xEJEs/k5iHplfnSdXoNG/d/UX6Ux6b0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c2da389-15c7-47f3-0484-08d9818b3fe9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:48:52.3338
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HuPWozKGCjYH1uF+sFOYQCrcCyvu3rNSEKI9cIt+99T+ZGtl4v7dPPnrFFMhkPXqYGb2cFuY3mDZe4RSzQ7MPw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 27.09.2021 09:35, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 10:26, Jan Beulich wrote:
>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> Currently PCI backend implements multiple functionalities at a time.
>>> To name a few:
>>> 1. It is used as a database for assignable PCI devices, e.g. xl
>>>     pci-assignable-{add|remove|list} manipulates that list. So, whenever
>>>     the toolstack needs to know which PCI devices can be passed through
>>>     it reads that from the relevant sysfs entries of the pciback.
>>> 2. It is used to hold the unbound PCI devices list, e.g. when passing
>>>     through a PCI device it needs to be unbound from the relevant device
>>>     driver and bound to pciback (strictly speaking it is not required
>>>     that the device is bound to pciback, but pciback is again used as a
>>>     database of the passed through PCI devices, so we can re-bind the
>>>     devices back to their original drivers when guest domain shuts down)
>>> 3. Device reset for the devices being passed through
>>> 4. Para-virtualised use-cases support
>>>
>>> The para-virtualised part of the driver is not always needed as some
>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
>>> model for PCI device passthrough. For such use-cases make the very
>>> first step in splitting the xen-pciback driver into two parts: Xen
>>> PCI stub and PCI PV backend drivers.
>>>
>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> ---
>>> Changes since v3:
>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>> I'm afraid this wasn't fully done:
>>
>>> --- a/drivers/xen/xen-pciback/Makefile
>>> +++ b/drivers/xen/xen-pciback/Makefile
>>> @@ -1,5 +1,6 @@
>>>   # SPDX-License-Identifier: GPL-2.0
>>>   obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
>> While benign when added here, this addition still doesn't seem to
>> belong here.
> 
> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seems
> 
> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build on 32-bit
> 
> architectures...
> 
> What would be the preference here? Stefano suggested that we still define
> 
> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate to it
> 
> in the second patch
> 
> Another option is just to squash the two patches.

I'd suggest to do so, but I'm not the maintainer of this code.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:51:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:51:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196455.349307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlQN-0007lt-5K; Mon, 27 Sep 2021 07:51:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196455.349307; Mon, 27 Sep 2021 07:51:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlQN-0007lm-2K; Mon, 27 Sep 2021 07:51:43 +0000
Received: by outflank-mailman (input) for mailman id 196455;
 Mon, 27 Sep 2021 07:51:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlQL-0007lg-6I
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:51:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id bf72dd6d-1f67-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:51:40 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-HpRfhkOrOLOtamoL_RvgyA-1;
 Mon, 27 Sep 2021 09:51:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7375.eurprd04.prod.outlook.com (2603:10a6:800:1a8::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 07:51:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 07:51:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0044.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:51::19) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 07:51:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf72dd6d-1f67-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632729099;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nT+T/72EmkOC555fFuA3/9XemJ5uoHyGK7Ods3SDSqE=;
	b=YLvNkv+FgbO9QA293k3Qj7ZnGX5spuvoVdBejjCsOdgksVJGJTtM66nVIr/tHdMc+pComu
	4Sm8HQ+RK51Ral/xTY80hAWZFiDCxTLmVUuQu1XeoQ/bOhOCiKHQXNzPLWgNM0Ot+vRjme
	h6mXj/dzlV8lZZVTHyEsmvC2DuZkI8k=
X-MC-Unique: HpRfhkOrOLOtamoL_RvgyA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=itJbM019zuBCp+3ZVwvzsIduLdMxS4DQIC3NpSj1mSuf6Zwk+2h9z4KVkwYS8SCXxsQHDboV8cCFNf9CIegIKdmVEMeARnRxmisn15/pryUSofG+mzYjyQnMF8Oa2FYkpyGqqhNrIjG6wcI7UsTNQ9Qf4jDJsbcWZdLdxPR/VLyfOU+VuJrRlmBEwxQUJf1DIFXbAZvFfYbfANaOpGX6L5Am66GfKVNgnL8oaRy01wxeTgZ7ZBTw4nl4aIU7mlGz3atB7nLM8O6G/JEhAeBrh/wopnaZWdXl0dY2GQk5imwi5hBebfrvYZOVncVM1ZPGPz8gfEziezZEF8JSGMpNJA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dPxg+uspY3zJhaK/lOcD+C16I8PeoXPTSu3CuLLjkB0=;
 b=Al6XWkUpBR3KpXit5SMVm2oivQUbuQweF8XYoEnsnyR/KUu4gay/T3IK//tyXUkdA0jnZpDOgQ5I14zk/wMu7fMKQBZWDVpX+/KqLT/09HbZG+B5/6LBd6iueZDjPiRHENZkzEVZgkGSULQGG1zCQvWO3yckw1bVq3d8ORyDPp/829M4V6QBQl04D7veIRsN2YJN/fUywW9goeE41Ra2eFcMUM9CK/6P/Lba7WUCdJltcxMu61MYXXOMctGEqcctpNyNrX8pf72UPifwFUD0aisz4TASOujYjuxIzA91ejODG4C68prgza012wL+dubVUmIUl1dRIqSR07XV4dqKEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
To: Dario Faggioli <dfaggioli@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
 "George.Dunlap@eu.citrix.com" <George.Dunlap@eu.citrix.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "iwj@xenproject.org" <iwj@xenproject.org>, "wl@xen.org" <wl@xen.org>,
 "mengxu@cis.upenn.edu" <mengxu@cis.upenn.edu>,
 "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
 <20210920172529.24932-2-andrew.cooper3@citrix.com>
 <da06dfe42adad13650755650518a232dd41ac46e.camel@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8e3f7318-2fb4-0011-f582-268816ebb59c@suse.com>
Date: Mon, 27 Sep 2021 09:51:37 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <da06dfe42adad13650755650518a232dd41ac46e.camel@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P193CA0044.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:51::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 96075057-4af9-4be0-0e82-08d9818ba184
X-MS-TrafficTypeDiagnostic: VE1PR04MB7375:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB7375784413721509A21C380CB3A79@VE1PR04MB7375.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	99DWcdhj2hgZSmQDh3r9xzetiZjoJ72X/ih6Lx7igrhR1EQZUWBu73B2XAB8QJjqhStBL+sW5JyQwQ5qlea6Ac9Io/10erISuonmRpl0vtYJhC+teOYQFKuun3Z6f8K5aWNCJVtLlF6dC1/TLukiJIGO459PTNYSR4Z51LOcbA2dXNWHDciJYnw0bsEN7PWScSiAiWMFpCHP3HCS44gWr0NAIuMbtIRabtvIAnaB7X4sjyeHPyP0ImO+K7+fC2MvgqUHOpK9OY4FG0x4VxiLCCBuJm5RXgRP53Hoct9X4KU00tul/X8nIROb6QWmurGqLKjm7NLkctPe04oclIX5nn/sM6bjETiyWMvlkqTfLXWGd47fyOqr7FnQCZPyk+S8IZ8HCXF8olQOba0+7gEX2B2Z8vQ5dV2Wbr6vtK9P01F4MMQZAiP3Ai7KUX3YPGQlqevzQyj1UlU+4PFdeRRlcYhyxxLFf89kfH5uQxX1zT3t7ilXdL7CkNdKbZr2bs3GpX6Uhbnm5XZqmTWq7lR/Jvb2bR1vzHwPlqg1HzxkGSVnE/PCrCzLxLf/wJ2mpuXhkoABA1hH2U2lE3f4J8Ga6hmAEhj1d1MrXgXvt8OMUQAJSU4cg6N6BowBVG00DZdqkpZ05X5QfJR4VkWrjjIMaCYPh1hAWSjZala1qdgectxs8g/FKHpoF+IvKTpsxbgFm94BV68RLXe34XEgBNmbl/R4Na+wLkBT+eWiXmf+hTo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(6862004)(53546011)(31696002)(26005)(8936002)(4326008)(66556008)(66946007)(36756003)(4744005)(54906003)(31686004)(508600001)(38100700002)(86362001)(5660300002)(2906002)(16576012)(8676002)(37006003)(186003)(316002)(6636002)(83380400001)(2616005)(66476007)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?nwpb/FcAnqWG9noH/eOR47b3+fZFkS6u4BWBbSn2e2RVFTPJiwik/LnH9Sxv?=
 =?us-ascii?Q?GtazpW0QDK7WhHWkbXPK+kfUWjK0fL48MsVVjNtWP/QSSNghT/qntguzWtZv?=
 =?us-ascii?Q?2XXtigg7w3gV/RdRHG7Fv3/EmGVZf+CcZotPGoQGhTUq4XMELclnRP9K1B3a?=
 =?us-ascii?Q?8M62MKz76Oz5J9zp1Tm/n3U0OjPFYz3eLi0vEJpnz+FGTqAhvB1QJ3Uwby6e?=
 =?us-ascii?Q?YuBHvCpR7vKsa4l9cM2erm/5nCCHsvSm/IUHHMWOdcAWSE9nWFC9LWmmXnXN?=
 =?us-ascii?Q?kZIgvH90zPSSGAAbwEMOynNs0+Sd7eFi9wjPSz5DiXj764WPvi1LjO6qsguI?=
 =?us-ascii?Q?+sVFL6pH+PuRdIfz/BAq2XmeZcm9wPetxBUY4JMQ120X+EdbIEWTgnCddYo2?=
 =?us-ascii?Q?h98plHz7yEr72XvGRe5xJ44UMX+DfU2tjbNDo2Jh9ohfUatkqgKzjQ0ODmya?=
 =?us-ascii?Q?Oj60Bfx0BFCOz4VhNp6r9CT++YdultEjdhzkvTmco8OIOHM08yGhLwG9bl8O?=
 =?us-ascii?Q?iahOrvsB/SUz8rh7QpUl5pBQudCkCmAD9UQA57uGQNE5qVugErfPrWgybuYL?=
 =?us-ascii?Q?mi9YpiV5gVBQFtXbyO/pec/BDQ/ez6gUMnmzaWDMBz9jjb+MHbhU55aYL03h?=
 =?us-ascii?Q?3wItkQJR8K1kMhxS71t/GWUTs47pxVFptxnWu2/3DY0ukZJzutOjZxuC6xpP?=
 =?us-ascii?Q?tfr2vIiPP5thAIH+DJ+JXw7Vnf9L0O5ODfck4KJTAx9TzDp6lx+JKFsxWi3f?=
 =?us-ascii?Q?Nq1kBaye5hZ7nOjuaum90jM+K2BnskNBIh5F2KhyxWoUdz7dsHjywMGwtzu7?=
 =?us-ascii?Q?0p1iyjWrWcIKaNOkg7rEjiQd4waMNEHl7YMOOlfs2QgW3C9R1+ibVJM7pS9t?=
 =?us-ascii?Q?J6Wt0+raYDDw6b7qMgUWwLVJ+ho4Aa8JU50hi6fjE4eMHA+J/4h4RpaTU58n?=
 =?us-ascii?Q?imwtwRWH6i4vsZubJsR0k0/u4os3kxkspOmr6ce+6gly3w37HGU6sjJ9AE5R?=
 =?us-ascii?Q?DSJqSxWoM88Cs88WQTojDPqCgsUM/6R5SgzZ9Jw2fmfRj48qx7inMNT+rU07?=
 =?us-ascii?Q?gAego9spsDjcdY5qaZTnot8Y9i+r/AuukG1tZaJ24kDnrDvU3TFA3F2n4zVe?=
 =?us-ascii?Q?8eVGFQmWlkvxTh9ntlxND5F3R5E6V1YLICmXT/LGH8j4Bf546o6RTBpt9fLF?=
 =?us-ascii?Q?uJGnOL1R1ebiNzs6ja7UtqFkkxyQjmxISbeXLGmkhgwhmvnxlzUGye7R92gd?=
 =?us-ascii?Q?fsNn+B2HmxxNHAprbQ7Unzn43C56OL0mYIvb2MTckZs3e5XnCxU4Vlr6P1sG?=
 =?us-ascii?Q?QGIQi2313ZOjQOB0H35AlAbn?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 96075057-4af9-4be0-0e82-08d9818ba184
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 07:51:36.1078
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KiKZ90r+xP8Z9J9DwuXbwsj90vB0D0anqAjevqu5IbdQWUeH9eTYLC3R4iVvvqNGhS/ubnQ4LNKcn6nA2ozEUw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7375

On 24.09.2021 16:51, Dario Faggioli wrote:
> On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
>=20
>> There is one buggy race record, TRC_RTDS_BUDGET_BURN.=C2=A0 As it must
>> remain
>> __packed (as cur_budget is misaligned), change bool has_extratime to
>> uint32_t
>> to compensate.
>>
> Mmm... maybe my understanding of data alignment inside structs is a bit
> lacking, but what the actual issue here, and what would we need to do
> to fix it (where, by fix, I mean us being able to get rid of the
> `__packed`)?
>=20
> If rearranging fields is not enough, we can think about making
> priority_level and has_extratime smaller, or even combining them in
> just one field and decode the information in xentrace.

I guess Andrew has tried to avoid re-arranging field order so that
the consumer side doesn't need to also change.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 07:54:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 07:54:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196461.349318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlT2-0008QM-J7; Mon, 27 Sep 2021 07:54:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196461.349318; Mon, 27 Sep 2021 07:54:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlT2-0008QF-Fm; Mon, 27 Sep 2021 07:54:28 +0000
Received: by outflank-mailman (input) for mailman id 196461;
 Mon, 27 Sep 2021 07:54:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUlT1-0008Q9-Tb
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 07:54:27 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 229efe52-1f68-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 07:54:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B3751220CC;
 Mon, 27 Sep 2021 07:54:25 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6AC6B13A1E;
 Mon, 27 Sep 2021 07:54:25 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id X1djGLF4UWH6OgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 07:54:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 229efe52-1f68-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632729265; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=PBFZrFjsQ8OSESSfPBFunak/TzFG+W7v0PpGK3A0e9M=;
	b=D4TqFONDxRKkNfuWAG1jcVJK7fopmr5EAUCL9RcL3dM2pvm+bgfob61vZYzHiau/jk7pcG
	uFWTHR+h3muge3GdjO/HM/NXbOzEU/rotCtDX6NWYoKPEv6aJDSf2M+Ua3m7mMSAdNNPyg
	vZmlhw0dyZM3KsX4lgPuQEPNtKTFUkc=
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
Date: Mon, 27 Sep 2021 09:54:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="iKFvM128gM5jX0PwofUW7M3zDjSDKgMDU"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--iKFvM128gM5jX0PwofUW7M3zDjSDKgMDU
Content-Type: multipart/mixed; boundary="FfizLD3OKvVSvGubtwh3UjobBCls8ZNcz";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Message-ID: <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
In-Reply-To: <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>

--FfizLD3OKvVSvGubtwh3UjobBCls8ZNcz
Content-Type: multipart/mixed;
 boundary="------------C218A4DAFE50988ABB3EB532"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------C218A4DAFE50988ABB3EB532
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
>=20
> On 27.09.21 10:26, Jan Beulich wrote:
>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>
>>> Currently PCI backend implements multiple functionalities at a time.
>>> To name a few:
>>> 1. It is used as a database for assignable PCI devices, e.g. xl
>>>      pci-assignable-{add|remove|list} manipulates that list. So, when=
ever
>>>      the toolstack needs to know which PCI devices can be passed thro=
ugh
>>>      it reads that from the relevant sysfs entries of the pciback.
>>> 2. It is used to hold the unbound PCI devices list, e.g. when passing=

>>>      through a PCI device it needs to be unbound from the relevant de=
vice
>>>      driver and bound to pciback (strictly speaking it is not require=
d
>>>      that the device is bound to pciback, but pciback is again used a=
s a
>>>      database of the passed through PCI devices, so we can re-bind th=
e
>>>      devices back to their original drivers when guest domain shuts d=
own)
>>> 3. Device reset for the devices being passed through
>>> 4. Para-virtualised use-cases support
>>>
>>> The para-virtualised part of the driver is not always needed as some
>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-fronte=
nd
>>> model for PCI device passthrough. For such use-cases make the very
>>> first step in splitting the xen-pciback driver into two parts: Xen
>>> PCI stub and PCI PV backend drivers.
>>>
>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.=
com>
>>>
>>> ---
>>> Changes since v3:
>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>> I'm afraid this wasn't fully done:
>>
>>> --- a/drivers/xen/xen-pciback/Makefile
>>> +++ b/drivers/xen/xen-pciback/Makefile
>>> @@ -1,5 +1,6 @@
>>>    # SPDX-License-Identifier: GPL-2.0
>>>    obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>> While benign when added here, this addition still doesn't seem to
>> belong here.
>=20
> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seems
>=20
> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build on 3=
2-bit
>=20
> architectures...
>=20
> What would be the preference here? Stefano suggested that we still defi=
ne
>=20
> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate to =
it
>=20
> in the second patch
>=20
> Another option is just to squash the two patches.

Squashing would be fine for me.


Juergen

--------------C218A4DAFE50988ABB3EB532
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------C218A4DAFE50988ABB3EB532--

--FfizLD3OKvVSvGubtwh3UjobBCls8ZNcz--

--iKFvM128gM5jX0PwofUW7M3zDjSDKgMDU
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFReLAFAwAAAAAACgkQsN6d1ii/Ey/w
Nwf+JVva89F5ZAa39ebUiKdOIuxr53em+baYyNNajTrPDayoieNFFaK5Bd3I6bSdCn2oISlDkKXM
Xp4dwrV6Y3lppD1rOyY5V60kszWAp40Bnii3dzzjKP4CITzEzyDUpfwmN32l4hVTgYAyZkdZ+02s
Ho8w6wbhcz9yCsN5DDr21+iXfKE22N2Ww7fgBnNE10cWLfionDK3QhFYaveDQqpJOWsCr6slbWxt
0uI9PTCk5Y0CpdvlZumTNBaEHQNkfwVvTLCccA7zp0AnX6PDVOgbaA4o58J2JAZ5mXDL1zdvc+o7
r4usAJoLPmjIEFNc3Bl8lmoOTXE8VPKsibTWbirz+Q==
=bZkF
-----END PGP SIGNATURE-----

--iKFvM128gM5jX0PwofUW7M3zDjSDKgMDU--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:06:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:06:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196474.349329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUleY-0002H5-W8; Mon, 27 Sep 2021 08:06:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196474.349329; Mon, 27 Sep 2021 08:06:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUleY-0002Gy-T1; Mon, 27 Sep 2021 08:06:22 +0000
Received: by outflank-mailman (input) for mailman id 196474;
 Mon, 27 Sep 2021 08:06:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUleX-0002Gs-8q
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:06:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 70289721-5838-4ff3-a296-7fec41166e18;
 Mon, 27 Sep 2021 08:06:20 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-o45J6dk8MKGyyamDK7_FmA-1; Mon, 27 Sep 2021 10:06:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5598.eurprd04.prod.outlook.com (2603:10a6:803:e9::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Mon, 27 Sep
 2021 08:06:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:06:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0063.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1d::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 08:06:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70289721-5838-4ff3-a296-7fec41166e18
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632729979;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZBc77q+KcNW4aZDXnGFVHZi2SMQ/1hAVmd8/In7Ex0A=;
	b=jjSDemvKORP7DfZyEvXp6rZMJEMLvDUqtB7cc5hw90l2mSQlY8aBMSThCIToDpqLeZwr6j
	23j5b5ZyeQ5X07ZnVmJavX4euZ53Z8ZnmEIo86V0EWuGKuCH5dFXsdQdb/8grXeSbLsc2N
	c+NSS/4q08kW3JxS6A0ISb7pDdgiD+E=
X-MC-Unique: o45J6dk8MKGyyamDK7_FmA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=md3g5j5F0EBEFss1X1iYUtpGyeOaXlHLAaewwWMI1+j+aXEGzJ8OedfFb0gIJXQLfdvw7sqhbEpSLfWfv2+D1w4CbDBaO6LeWR99Zuot0ZR9W2pU64+kuMkCWaX833aoWVsl67A08dVJSsxZIVb+53yzMwR5ecncLU+EPC/C4/k8NLmtmX+ffpS08C02UyG2QeU5Qx4a7s+v6MklbWx9X5JM6i7+FW/oclEhN1iz8vWcNTWjkONFDj3Pdl+Yu+2ag5Bpri4SUQ+oUyGsgPtVZKCbROUrftd4T7iEYgSZT5pfw54mTeYR9JxwTQwMV/F4Sk//LCy6FSK7iCwci/5MbA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZBc77q+KcNW4aZDXnGFVHZi2SMQ/1hAVmd8/In7Ex0A=;
 b=OsEkZrXW0CXb+o7yCQ4qnBZ5XUcoiYNaZFWobwL6/Zayg6Cb65AWn8MmlobNLkGF2z+9WQsANvuyfC73AXsM0RlT5/lLU1PFYoxoj2UAsVS6wsZZhW91G1DE5tvBa/+/haVHEX+jtmgFBRp1EqUVIniXQQs0BPnA48gDy4Qt+aTUETmF8oDAHHXKDKLOMUU9ZIBE4MQbAk7DSYmCyLX+DtVNjDbdvoSETKiXHmg/X6pir3Izbc7bEc5bkuHOkq281DA0EDHnuOYfMKjUiR7cyGAIWyzQ5oy6LCcDmVCMBE2JIxOIGOh2gCXlF/DGWD8QD1OjPMbkbDZxmBZP0JynNw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/11] xen/arm: Process pending vPCI map/unmap
 operations
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-12-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7bb451e0-9a99-e2b4-a2be-0377de38c0be@suse.com>
Date: Mon, 27 Sep 2021 10:06:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923125438.234162-12-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0063.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1d::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0c88b998-affb-4b1f-8ac6-08d9818dae1d
X-MS-TrafficTypeDiagnostic: VI1PR04MB5598:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB559816112BC034CF8FA7AC64B3A79@VI1PR04MB5598.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OdXJr7FICZTqVWLcYLqqmSIt6skfyFNiX01tUgewXDfEqT2zVm+VHQ6r9ytNXROkmovv9nLBeDg86M7cOh3xsSZqEiZe+aNbGhjy0ogkXfO1HP/eqhTObS/q94eF3IlmlmAcy+1xz+VXsIKSlApDIRBDtd5D4+gR24KB+ZriNsiNhoi67QwYbuKMstqDW6OAAAaxEvCvleuBWlyUmAyvaPR9K4DpWEWMEQiqRp0N17LQgGZHh6OCieiQXXOUNWqj+ntV5ozjInI1ZakTwfYNZDyPAMhl1BhZsH2JI8d8nr173x4RfopD2AwEQDsGnIPD6Qoxw0lN+nCTjFAKstvxgh8epV668iENFIwvWTz3Fwgxevj6lS+NxNpwFR2WKyXwaEHdiPyiUTs0Yam9iMtYheyJMkO5bWCLd+28iBBO6yLj4vR+lsQcteOLlu+b4x3JJHbeYUopRW88bHYfkFJPUq1d4kWQsJ4qfdFQ4X6pxZmRPCXjstzmwDSR6oYAXT4ypx3ZDL2XqRxrYPpS4B1t7zKny6iFcp43L5WXtnNlPf6WEFJRbhNvdNyjwNHPhMbbMmvNLhn2Go5Kli7+7lIUDs6SokxVPo/CQOaBkVEKRetMw/PRgvGJhKVfeRndRl5ZK8OzeoC9GaZcWHFePIF8NwnXj4NnJl/MJI+EoS7/+oemUyU+ZAKxXxEF2DPZ2lGmzaAH3DlhOKFmfqgFNgiymuLSRnRKG9ClC0/+ZXD7q+Q=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(53546011)(83380400001)(5660300002)(6486002)(6916009)(4326008)(2616005)(26005)(956004)(2906002)(38100700002)(16576012)(316002)(7416002)(31686004)(8936002)(36756003)(86362001)(66946007)(66476007)(66556008)(508600001)(31696002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Mzg4VDhzY3h3b0FKQWt5VXQ2bWQxSGZUWnh1TSthNjdoRDFLRldWcExqTVN4?=
 =?utf-8?B?dDdqTXlWNkhqRm01UktaQ051SXB6VDBhandTOUNrOUJLekhRTCtIb29xK1lw?=
 =?utf-8?B?Sk9iMjV3cGM4a2VpdmYvQ1dmeDRHRDZJQXZpUks0VXhkK0YrQXE0VWw3dktT?=
 =?utf-8?B?a3dLcXRSOVd4QmwvUjNMVnJRdityKy9zUHVQN2J3ZkVpSGRuRXl4WWMyMEJ4?=
 =?utf-8?B?U3ZZVGdEaGs0V3pTOHVMeG1Ua093ODBpVmtHRS9VbzcxbzF0YSswRktzdlEv?=
 =?utf-8?B?SGdWVXJzTFdVdnFvaytGL3hZYjJ6QnBZb1hjSUdtWGdqNlZmWWVyQkFKM3Zn?=
 =?utf-8?B?VlFzU3BnMCtDbGpKQXg5SnF3ZXZCcndlb0xaS1ZPRGQvcjYwV1plL0VFdkNX?=
 =?utf-8?B?MG0yNWtQcHpnNW43cWhUVGFCUmlHS2pMNkU1VGRyUUJZUTJNQTFacHJNc2dC?=
 =?utf-8?B?d2RpSTZMMjNOQkJjdlo3M21BWkMreUdRWjNnajlIY1plUFBiVVlQaG9zZ3BQ?=
 =?utf-8?B?RVdkalhUVnhxNmd1N0F2bElPMlFvZW1NcHQrTk1zeGg4VjRoNkE5angwRWxa?=
 =?utf-8?B?Z21UUnhhSzdXbHBVQlU4MjgvOTd6dkdKT2dGRjQ0UW8ybkNzTGJGQ0RwVWtS?=
 =?utf-8?B?Qi8xZUdBWitycUdsYjlrc0xuK2tpTWdjVXJ6bUdvZVREcm5lNTA1TkZLM1ps?=
 =?utf-8?B?SkdYbWtVbUNHVGNmVTJHbVAzaHIrL0toaDBqdENBTElmWEliaHZkVnM3Rzhz?=
 =?utf-8?B?Tm9rUDNzK2xwRUEzL2VCMDUrdUJRVFFUZzNRSWRUOFd3VGM4UnBPNWVmeVU3?=
 =?utf-8?B?cWtUc2JqN0tvT3FyWGQ5dDVKUENEYTFNaGR5aHlCU2tWaVFkRUV3QVFneTJR?=
 =?utf-8?B?UEplelhxOHdOUTVEd056MDMvZHlyczgwMEdHY2dBWFpWQlN3Z05jODBGZ3p2?=
 =?utf-8?B?ODI0dEtsajRYb0hxRlg1TDQxNDFhWmVNZXVhdHVqWlJwM3lXbHkvRDdkTzFx?=
 =?utf-8?B?UGFTRGF6UlJaU1JlbzUvOGhUQ0l4ZldlMzlyeEExVWFIQitxcWlrYlBjNDM1?=
 =?utf-8?B?b2pDTk9RUGlLZ29zUVRiajl4cWlNSWVmSUpTbU1hVXJoMDQ2WUlwZ1NsQ09P?=
 =?utf-8?B?VVEwRWg1QXVRdXVoQmQ5V3FuLzh2b2FZZWhGTFE4ZFgwcDNGVjcybkd0dDFk?=
 =?utf-8?B?ck1aNm1rdVBOOGcyUlNVVnJ2bm5hNXE4Q29zZ0N5SXFpcXo0QkpsTDBxZkpm?=
 =?utf-8?B?S1hRQnFBNWxub3FRY0VrL2g1Mlp5YnNZK3VwVGpXZG5SVC90OHNrb1lzMmVF?=
 =?utf-8?B?VTR3ZmRsSTJhQ1BOK2M1MGVFSVFUREpSaXlRT0lmYjBvSk44SGI1ck9mZ2kw?=
 =?utf-8?B?NFNucUN5S2ZqdmpzU3hIVENmbU1SR1RWZnlsMzFqMHk3ZzVodXBmTUFhMEJP?=
 =?utf-8?B?ZG84RTh1eEQ0Mm1zRk4xWkErczRFaFlic0txRlR0ZWNKMlFyUjdaTmdrdktl?=
 =?utf-8?B?RlhBMjREMXVxU0xjVXBkcEJzeDBZamtGOW9qM0ZFRjUvOFcrdGpLSzU3YSs4?=
 =?utf-8?B?VFg5NHBEWTR5Q1IySmllbGlCM255SnNwL3gyTHdhbDdIY2VGcnd3b0duS0Fw?=
 =?utf-8?B?ZmFEOFhlK2RiSEt3MjF6NEJCeTV3cm03Tk9DRFF6djdOQ2crTFNBbDEzNks1?=
 =?utf-8?B?R1JvYk5FditOL3hjVktvNzZHTFR0UmNpRWxrY1FuNTI3RG1PaU9jN0o1TVFw?=
 =?utf-8?Q?ovgOMC5uYkZm/8ZAUOZ6EVdOjyZRX+2dvquccdo?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0c88b998-affb-4b1f-8ac6-08d9818dae1d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 08:06:16.3248
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4zPR84w/ucCLMA2yJ6LfbTFqeCMbp12Ixs41jSESDrIhQGFlbRhHBBEwFZKd8cfUCm/gqwi5PYOKdXfousykPQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5598

+ Paul (retaining full context for this reason)

On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> vPCI may map and unmap PCI device memory (BARs) being passed through which
> may take a lot of time. For this those operations may be deferred to be
> performed later, so that they can be safely preempted.
> Run the corresponding vPCI code while switching a vCPU.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - Moved the check for pending vpci work from the common IOREQ code
>    to hvm_do_resume on x86

While perhaps obvious for Arm folks, I'd like to see the reason for this
spelled out in the description.

> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -549,6 +549,12 @@ void hvm_do_resume(struct vcpu *v)
>      if ( !vcpu_ioreq_handle_completion(v) )
>          return;
>  
> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )
> +    {
> +        raise_softirq(SCHEDULE_SOFTIRQ);
> +        return;
> +    }

Note that you're altering behavior here: Originally this was done ...

> @@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
>  
>  bool vcpu_ioreq_handle_completion(struct vcpu *v)
>  {
> -    struct domain *d = v->domain;
>      struct vcpu_io *vio = &v->io;
>      struct ioreq_server *s;
>      struct ioreq_vcpu *sv;
>      enum vio_completion completion;
>      bool res = true;
>  
> -    if ( has_vpci(d) && vpci_process_pending(v) )
> -    {
> -        raise_softirq(SCHEDULE_SOFTIRQ);
> -        return false;
> -    }

... first thing. And I think it wants (perhaps even needs) to remain
that way; otherwise you'll need to explain why not, and why the change
is correct / safe.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:14:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:14:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196480.349340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUllz-0003uk-P5; Mon, 27 Sep 2021 08:14:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196480.349340; Mon, 27 Sep 2021 08:14:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUllz-0003ud-Lj; Mon, 27 Sep 2021 08:14:03 +0000
Received: by outflank-mailman (input) for mailman id 196480;
 Mon, 27 Sep 2021 08:14:01 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUllx-0003uX-Lv
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:14:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ddfae81c-1f6a-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 08:14:00 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2059.outbound.protection.outlook.com [104.47.1.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-zeFcZcVPN6CzNfpcrPhFTQ-2; Mon, 27 Sep 2021 10:13:58 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3392.eurprd04.prod.outlook.com (2603:10a6:803:7::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.16; Mon, 27 Sep
 2021 08:13:55 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:13:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0027.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 08:13:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddfae81c-1f6a-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632730439;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1odbPXqDZnzI/wEIq/o1a4C0cKDMDP/eIYH7Bq3N3EU=;
	b=CcLjgBzcMuSq/hQZJJF/KPikfOJwY7tFuA6vDTEpFqMzpY06lwtKzSegLmVZ7ra5DYlBEZ
	OLq6sRFzHM0q5BIaLyheCT69Tba4VoidANkdAZcZ49q1qgeyvKV7qT7TxzxdHd54b7la+3
	idYb+0G/SsvR2aks4nyJlgUexIE/Eeg=
X-MC-Unique: zeFcZcVPN6CzNfpcrPhFTQ-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n4hd4tJoc0eb0YxE9QCEmXr9BMiMLybjWIymb6wKQJOPcEXxDZWqd63CyO8ifau6Or8Jcn3+YdzoCUgeRS+o0AqgfJru1RVKCjXRwXQgV47e3m/hccNN9cDPTU/fwR4QCRf8OU9gJ6RXuhqaIrZtygicyrboZVqSGo6u5J0IzWu+UiyJVfu1sLHoAldd6EPYyQ2lNVLJTc+7/C7Ag1meb4yf1nqjRJDL/lf87+dydHXtOnkvX5RzZtH+WPtELsLv/Facmi2suqN0udJ+/K5BEWxaZCThCoxK2twFuekXjF4aex7VWfEJvpgxwtFM4eIyC5ua9gQu+r8FtFuccNR24Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1odbPXqDZnzI/wEIq/o1a4C0cKDMDP/eIYH7Bq3N3EU=;
 b=JVAzSlzTj0gKFoQdnP+WjqGzJZExXGYIdOeGLyD+0T+33/1hRSKb0M35O5GebWJCi/L8ViRuANCJgMmvMegmJMxxSvwFJCen/rheairnOJ5TOh4rYxSI8rJNQSZUIFzrXxf2vfiSavHVee2Fj35EnP8z8UmQCzHIZ/WPY0IXKiWmDktkhjDYExmZiD+6DLZKaF5DOIJ72XyaAU6NzDQosYnHy1nj3nPNNSTnmDV6RIJotK7so0EU8l9EdB6utXX4h4X39Y91ULT/v1ahDjAVOuGR3vR5/lV8TpGvqZQJB1K5Y19IIRsX7Y4Z0DlKGSyV9ciDETRlFGpwmRjzrS5Umw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
To: Juergen Gross <jgross@suse.com>
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
Date: Mon, 27 Sep 2021 10:13:57 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210924150424.10047-2-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0027.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4b401e9a-2a99-48f7-9c23-08d9818ebffc
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3392:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3392D464A3C41F3F377DB4C1B3A79@VI1PR0402MB3392.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vCWbOIzksNNyfI5a6Hy5FOB8dPRrBUYkFXCjg4n485sqFXuYaj88MIFx5TyMrGrEo0T1s6q+8VELU2b5nWtCvAjmxX1eaNhRBSSffStQLb754hF7YyarK+5zcq+/JSGg+y7zxj476ytirlALbN5Yx4aG3PQVFcKuM8w32wSBhqE5qsVsHeGNezG58MGTy6T+GrM5oM1MMe/dGVlWodlRPnhqovxgij6FfkZB/E5Z9IW4cZo8caU1hpRoAzGilvc6taKgfo8+CqjD/Ws50jigt9gwXinZHErjBA9xVplApCsRtbn2fcWlQhE2mnkpM9PUCeVo5KADB4ej7s7PYgvypxp4LTzROIXyIVi3FQUBpC/mJg7FPl+H7+rJWAB6ZUp78PUaeq3gJ9Ytfz/s1r4nC+ZRSRMsRkx6/6rasJAGqWkbpf5vbasfhN36QmJSWgQCBs2ecRiwrXh20bICiiusOTO5QGl5dJtpPQzsNLA0bL7zNAkjNF26woo5G0W0UsrHoYFz0XrNfYsxRLVnjmrA0e2+/m8uAnrmXU79phslYiLNr44H/sH5H5p9/1h2vFKXEwPrPZ4l3+jc5nc1tuNaXAjuELCB4O9+IPkt6HPzEj1fc8ODyW0Yv/0FZ3C+yI0l2rvTykX2TQ94o7RF5bPCmKigYeYrDBCSeFbn3ATX0O0/40o5enFQUOUlXMUbfmCQmPlP2wpRNF2VnyZxGaSzb91w79Fscre2fVH/6dEp6qA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(31686004)(6862004)(83380400001)(4326008)(6636002)(5660300002)(66556008)(66476007)(66946007)(316002)(16576012)(2616005)(53546011)(2906002)(6486002)(86362001)(186003)(36756003)(956004)(8936002)(26005)(38100700002)(8676002)(31696002)(37006003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dTE4S1VQOGJxQWlRTGdlSXY3UzE2ZXFPNFYwbGhEb2s4NVhRU2JIQ0hPRG56?=
 =?utf-8?B?RmtFeDJIN0d5VWttRzRjLzFsNThMS0YyczA5aDg0MXVRc0lRT2lpbWFlWWhF?=
 =?utf-8?B?S084ZG16d3JzUysvYXcvQS9YMVl0TnNvbEs5MitjaDhVMVRjYXh4RTVvQy9J?=
 =?utf-8?B?N2dOM0NhVmUxWS9rRDdBL0Z5VzVXdDNwbU9LazBDM3dPdXVadXZ3Wk0rYlp4?=
 =?utf-8?B?SFRTQnpPUHlralpJcmc3K3RyUzBSWEszM29UUis3dVVRb3g0WHo4STMrQmkx?=
 =?utf-8?B?WmRxeHplMk1PWEFTRUh1MWpBVU55dGM2UUJhbzU0cGFOeEIwY2puN2hIR1ov?=
 =?utf-8?B?aVJTbUxUQzR1b0lEb2wxcDZ2dExNK2xML2gveTJVb0xZV0JEa3R3ZUUyVUVC?=
 =?utf-8?B?ZWxVb3REcUY2TTZTS1hybjQ5bklmalI4WWxvY3RwYlN2YjFlNDYwa0wxaGM2?=
 =?utf-8?B?bnFyM1Bqd0k3MHY0cGtvV3BIYzdsVVpTU01TMmJsVjJnNk1YcE1LVTM5cWVG?=
 =?utf-8?B?WDRXQTI4U0V4RnY2TXdDS3ZOM0owODU1dEdGMzh1bFhkZGJOS2pRd01relU2?=
 =?utf-8?B?L3NJeGdZeHVia0V5TDF6VFZYc0swZ1Z2NURiNlE4aFp2VHhWbnZjelRkdjg1?=
 =?utf-8?B?ZGt2TG9DSkgydHRTWG9BbndDcVBSV1I3QjZuckRKL2RTNVU1a2xoZUpWSTRY?=
 =?utf-8?B?S05LaUZtY1FNcUlMRjYvZFVDdFd6ZFFiQVJOd2VjZjdJUmoycDByTTAxT2lW?=
 =?utf-8?B?eHlZRU5DMWg0OUU3YTlDVHdTNkhiZ1JGa0F1emZLTFY0eWRVbE1ObXFKQk4z?=
 =?utf-8?B?WU9sY1MrdTE5bGNWcmpXWEF0aWdkRE02VUZuWE84SFNSODkrR0tJaEY4NjdB?=
 =?utf-8?B?ZU16WG9qVnJENGg2WnBoWEE5dG5RaFkrN3NVM2dBUGZrUVhJd0tFUnlEYmc1?=
 =?utf-8?B?YlJWVHAydExnRm1GN0grRFd4VHRVeXFNb0pnNFJZZENNRXBmWHd3eGFPOU1j?=
 =?utf-8?B?MlJqVzdNQUFmTUhXcDEzR3h6cDJjL0o2TUw5NmRadFhCUStweUwxR04xZ3BZ?=
 =?utf-8?B?Mkljb3FQNjdPaG4zZlQrUnFnUVVrM0k1dGdabXBReFRjWUVONkZyM1ZnanQv?=
 =?utf-8?B?OGRPdUl4bkV5L3NYeFBteUI2ZTRsUmhIYUt1dFg5WXpncWhJbWRYUW9yNCtw?=
 =?utf-8?B?cm1idUJRSW9UOHdSaGZ4bGxzZjhQbENZM0owaVlQWXZHNkx6S1JTVGdBMnll?=
 =?utf-8?B?VG5xa2xZV3gvSTl2UGEwUW1uQm9EbWNoVU8xRm9BK24yWmFZVDdWMEd2d3Rv?=
 =?utf-8?B?NTRwZ2tVQzl2cTU1NGE4ZXdCT2NlbHg2eEg3Vmd1N01tME40dTEveWdnSWRE?=
 =?utf-8?B?T24vVlpGa3NLUXN3cEJFMzdLcnRTSWRhYlBoYXFPbW5EdjFOUDFZWklyR2kr?=
 =?utf-8?B?SHV3aXJCK2twdlNlL1d0K1J5VldoczZrK0dzcUczV2dwbzI5OGdBLzM5eW80?=
 =?utf-8?B?V2I3eENoNEJ0bGt0TDlsVnVWdlFqa0VEc2k2K3JTWWx4dDlPUXBQNHl5RytJ?=
 =?utf-8?B?ZkdMdFdGN2hlZWVRa3ZOZnIybkRZQUxJNkovY0RCZzJtbmFOOWN2UDByR2gx?=
 =?utf-8?B?ZGNhcEl3SUZZV3RPSnJpK25ydnkvbC9OOUx1ZXIvbUFvSXlTVUtYMlE2WmFN?=
 =?utf-8?B?VWtIdlpNd3dIQkJmVitWWjgrZDlRaGxTYXFiNUtuUERocGRHM1h5QnVoR01p?=
 =?utf-8?Q?ShI7q8sp+N/e1AAYeQ3SDT1/ELfFbxhjWggajlp?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b401e9a-2a99-48f7-9c23-08d9818ebffc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 08:13:55.7657
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tsjC9JhBZWDi12L38zmTaKPcESJUeMURjpwyTsQfQedG9o55WC5Z2H7LymjRRhk1vNL3O7aqOmZgX0RjdrWUNQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3392

On 24.09.2021 17:04, Juergen Gross wrote:
> The interface definition of PV USB devices is lacking the specification
> of possible values of the status filed in a response. Those are

Nit: "field"?

> negative errno values as used in Linux, so they might differ in other
> OS's. Specify them via appropriate defines.

What if new errno values got used by the driver? Would we alter the
public header every time? Or is the likelihood of further values ever
getting used vanishingly small? In how far would it be possible to tie
these to Xen's public/errno.h instead?

> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  xen/include/public/io/usbif.h | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
> index c6a58639d6..fbd6f953f8 100644
> --- a/xen/include/public/io/usbif.h
> +++ b/xen/include/public/io/usbif.h
> @@ -221,6 +221,13 @@ struct usbif_urb_response {
>  	uint16_t id; /* request id */
>  	uint16_t start_frame;  /* start frame (ISO) */
>  	int32_t status; /* status (non-ISO) */
> +#define USBIF_STATUS_OK		0
> +#define USBIF_STATUS_NODEV	-19
> +#define USBIF_STATUS_INVAL	-22
> +#define USBIF_STATUS_STALL	-32
> +#define USBIF_STATUS_IOERROR	-71
> +#define USBIF_STATUS_BABBLE	-75
> +#define USBIF_STATUS_SHUTDOWN	-108

Nit: While probably benign for all practical uses, these negative
values nevertheless would better be parenthesized.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:18:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:18:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196489.349351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlqL-0004mt-B6; Mon, 27 Sep 2021 08:18:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196489.349351; Mon, 27 Sep 2021 08:18:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlqL-0004mg-80; Mon, 27 Sep 2021 08:18:33 +0000
Received: by outflank-mailman (input) for mailman id 196489;
 Mon, 27 Sep 2021 08:18:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUlqK-0004lH-1S
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:18:32 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7f33f8b8-1f6b-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 08:18:30 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R8EMtb022497; 
 Mon, 27 Sep 2021 08:18:27 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bba91g0dw-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 08:18:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM4PR0301MB2196.eurprd03.prod.outlook.com (2603:10a6:200:4d::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 27 Sep
 2021 08:18:24 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:18:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f33f8b8-1f6b-11ec-bc42-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T9Z8C0HWR5W67PjFtMGl2UFRK9PxwAu6zzfGitZ2TAs4OuR0OzMs8cLoImBRGone6qWnHsa2lk02ku6MYzftblQFmwu1eobXqhL9hgGPvKihZ8NLJIUzNXJvjTpbb/nvDt/giem6SDZviYoCfCSfMACbb7S++ChdHNE/P3mo10MCzLHYYEV0weE57Tib/y1/ihTOkCXhKgvywwl1ZNuSsvXd9KGCGaf7XpVyBD6Okm+jy6u1I5CzsfYvuyk0oi+O1HscFta3JMqDNpcgqarK151T0NvchhpXsAC+krsWuQ3B2CtrIZQ+jE13W1W3fcQV4SFVvbQICJJvm/q5Bvp8Fw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=krYohFNs8smbKXzf1DrxvP6rw7g3WhfrIED2O9IS+j8=;
 b=IbUW0QR7LcaaV6gn+bx646EB2mSjUagqBkYJeC8gWs5XuFRLAF41oGNSXd9FLwmfVOGnIjvQtvEpDNvLjkvsl8MbgJ25QbLEZl7GBqoM7rcalskKJJhIxefLgNVtPtMRC+FNh1cwuTHtGT6GmdoBcZQ58hKV+RqJMrLDit9SyuZ6LhZgwUU5yn7V1fIbG6PY6DnSmvuq20t2v2acMOGS+dDLHpQDmnRk1zTiAD+QhvA9EtglADMmstqnZmeSXRq/pe5th+HvfZCZIQmp59DQA0aA0F8G5iP4UVvpQxGPEu24nMGLNLN9ReQsdRDKxon6oCWBE/PvwLND8x4ZTn5iaQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=krYohFNs8smbKXzf1DrxvP6rw7g3WhfrIED2O9IS+j8=;
 b=aQHkdjZzgJMGhd4RC0O3HG/RkehhK8krYIb/NWKaRLZvDYKcfN0bOQs0xO+xRjtF/v+QYcZBPGJ/9UYcaLalPQ7hjaChTFQI/4ezPrZgS0LdGKthYuZbKbJb6u7Q8QRwOWz+cUp+wqRS+q+17S8YW1W4P3b9Y0M1+t/S2tmftMDr9Wh1KX6RiQFU5/IUtfTfYcZpjBA464i3lyV4a8BepN8PIxuoYmuX9FLDKNl2IBMhkfM7pd4PrrJfjT7hknK3niAPPZgWKG94Lc6g8q3gJlyXU8F4Yo/AqziaahZoUbQdvFWA5leyMF6iML9/02RV362gm2CjgOTNutXlUHNEJw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 02/11] xen/arm: Add new device type for PCI
Thread-Topic: [PATCH v2 02/11] xen/arm: Add new device type for PCI
Thread-Index: AQHXsHoxl5/FGbezwky3nlX6PVRH4qu3hVeAgAAKLIA=
Date: Mon, 27 Sep 2021 08:18:22 +0000
Message-ID: <093ba27e-37a7-ac7b-9419-65cdef5e5661@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-3-andr2000@gmail.com>
 <278da2e5-e55b-e36e-1210-d917bb5ede34@suse.com>
In-Reply-To: <278da2e5-e55b-e36e-1210-d917bb5ede34@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9f35a75e-c709-4b13-d1cc-08d9818f5f3c
x-ms-traffictypediagnostic: AM4PR0301MB2196:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM4PR0301MB2196D2FD723B7A3830EE6DFAE7A79@AM4PR0301MB2196.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6430;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ovxN54WUz4Hzt3t4BZfLIu8E7wIWKREjfzJu0o398kj3d/Fhi38xMWkaEdbjXlmjSpMeR4aUPhrBm+ahR7iJ4f5LB2FxD7w16gp8r0f/fawYjCL0S7D4zVEYapXehFBRwZyOFEaBdY6C5zqNq3w+QifK7Ww3GWNsvnmqTSGX7J899c/Ir0KVCO9ZJsnDiuTFH+oADnMfmUgnuxRfJneM9W3VMYctgBwmm1aLEp54k/1UPB7f4JD68+Efu0HEVhQkQmiHvfZyFRMklXDvFlEWSX0KOq+eNXekKAZpZMu6yTBIhYV7WR5k2lQqtWaYRAp/BBhgB8szc03EXacaq+tnEc/X+75dEg/HDh34vL3NlE43v82PPn07xfLb8W8F4l/8/oQMqpVZfI32tmY1IzvZNEcVxfXIkkT14Dc5/TYwRupY6I0T3Mmh7V5zpj+/UxLn62qZXM0dGNMCByx1jOueM/bYxJcH6h8Cj2hR3wLZO8piFjy/CdPdolBQ6RUpo+Bj50PbarlAHkrQ/jbUHayRVZsk0OAkNelp0tlZfWBdZomsGDOcVndm4T6L7yQsqybebMM3BoqDabaLhRueAFF0Lhj69wG85kTF6+zTTGFBnQroaUl/JO2ATgkh93jj95X4WxsTIZHJcyaNDCPYAYuhs0j5S0/r9plo0gLWTRNmmypH+7pHbLaxY4WvyGGJVODa3i0iLNQ53038d/gsmWVXZJm2fcdt++GvyU9AnAQdoqX3sn/sehWa99kth0UN333J
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(38100700002)(66446008)(71200400001)(2616005)(6506007)(5660300002)(31686004)(66946007)(86362001)(6486002)(6916009)(66556008)(31696002)(8676002)(64756008)(66476007)(53546011)(316002)(76116006)(4744005)(4326008)(36756003)(122000001)(2906002)(26005)(508600001)(54906003)(38070700005)(186003)(6512007)(8936002)(55236004)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SWY2RnpUS0FoRmxrdkRJZFZKSVJmUTlYYW55YXAzK0pseDE0ekNNZDZ4YWsx?=
 =?utf-8?B?R3R4emNaalhDZUZIcnhkREQwcldmeVVGa2FOa1ZXdEJha0lvL1RndDU2N25t?=
 =?utf-8?B?NmpwcHFZVUdtK2NwYUFFRUlYUXRjWm1DWVlEY1VVS1lYRFhrYWoxZjVLbnJN?=
 =?utf-8?B?S044ZnFCbjliZldCL0FybDJ2NlY5L0dXQTZVMGIxcmxpYU5zd3Z0Y056cldO?=
 =?utf-8?B?R1hsMVJSOVRpa3FSakxUTVc1bGdUU3V4SXFBZExBVzMzV2F0M05GdmIrbVlx?=
 =?utf-8?B?ZGJibHQ2aDZrQkNlYVpTMUZadTFGTVVKU2s0dzhEOXZuczFmVDJQL0xQRkV5?=
 =?utf-8?B?MXlPMUM4T0JINFBEMUdHdU9tc2NaMFFLazRmakc4TlBoLytaUmNuelBPRm9h?=
 =?utf-8?B?RXY3MlBUeHNSMGxpZGxtUlU1dnpCemxUZFdkbWJCenFuZjhza3U1RTU3Qmc0?=
 =?utf-8?B?ZFYySjhJdGlKd0prOW5LYWdzUThsRU8xMDJjQUpqdFNNeXQreENpOFkwblBy?=
 =?utf-8?B?UjNNR3NEMW4yNHlpcGt1cU9oOWl5RXBmeE5KY1drRTcrc1ZBTjlaN1NZejJH?=
 =?utf-8?B?VW5TTjRKOUdQZ0xOM1hxTHQ0bDB1Ymx4a1FzZTZ0dGlFaUs2ejUxNXJvVE9Z?=
 =?utf-8?B?ZXZkV09PaitTcVNaai9PdFZkUVp3UGFvdVZYVVNrQVhZUFlKM1FzZGdGS1lN?=
 =?utf-8?B?cVNaTlJZU0FBTjJQZnlKc0JrZ2I2WGxVZWZiMUdEbzhaUjBOT3lyYkFuOXI4?=
 =?utf-8?B?NHUzaGxkRDVLVUl5dXg1di9xSjNHT1R2SENLaTBEM3Z3NTluSmE5Z1YrOFJT?=
 =?utf-8?B?dCtjNVo4UDRwdUN6S2RKVUd6UEpSQjJ2aDV0V1dCdDZnSjRHUDQ2NEw1RUdF?=
 =?utf-8?B?YXd3Mi9qUXRWRDJ2SllGelp4eGx2WTdTcnZjclltaTVxNXZYMnZHOUlpT2xM?=
 =?utf-8?B?aW9rWXRVT0xMbG5KVEFRM1hoaW1sb0oya3FpKzJjeU5XaXFWd1VsSmR6c2lx?=
 =?utf-8?B?U2J1TGd5UUpQY09EL3kvbHdaR2JhN0ZDYlhyNGZwQkgrSnpFWWhMeXpHN0Vr?=
 =?utf-8?B?NVcwYUQrNlFSeEQxelNXUGs5ZGVGUWRQNUVMZFJibm1ac3FaemxBTEtOaTNF?=
 =?utf-8?B?eXBZLzBKRXRLME93b0FDZXNvSUU2TjkwOWtnaitzMmZDTjljbDMyNStCdkxi?=
 =?utf-8?B?VlF4Vk5JNEoxMHhGcjc5WnZVZjNvT0xSRWhwVnZ2dkJFMk9abFhwQTVLYVF1?=
 =?utf-8?B?Y0kwR2N4MW8rQVhlNDdVblhFV0FiWjN2ZDFrRk42TFVqemtGU3FZZnhyUzhp?=
 =?utf-8?B?QU9DdElVTGpWNGo4VnRnL2FDeHkxZ0Vzc3VHNWV1a3ZxNURRRGc1amduWDRu?=
 =?utf-8?B?SEZMUE5CSlhEbWhpZ1luOU54ZEdMM3FBUmp0QUdweUtBVGZ3Q3JqTHBpckFC?=
 =?utf-8?B?UDk2aUlMNmtzNzMyeW16WW45d3prRjVJNWU1dENLZ0I0SkcxKzZPMU92NGsz?=
 =?utf-8?B?OE92M1ZhZ1ZzQnRvZUVXK2lEOVd5dmZNZmhmcUxmalUwYlRhRjdzTTJoQkVE?=
 =?utf-8?B?ZTV3RW1KYWY0QjVBQUN1QkowbnppMnpDMkVNanNwcVMwZ0M4U0o5K0FYb3Nr?=
 =?utf-8?B?NHpyM2crY1BHemR5RVFud1kva1NLT3RDNlBEVUNtN3djSGdieVFWeFg3L2c5?=
 =?utf-8?B?bHI1UmErNG5oNkRvSUtMeldOMFUwaGZ1ZmUvYTFydmtock44MDNoU0FLOFJJ?=
 =?utf-8?B?UldZMzlRdmZsNENNTnJCRDhHQkwxU0lIeEhUazVqYVk5aHVCTFFlZEhqUE4v?=
 =?utf-8?B?RjV1aU9mTFltZ3ZsMHFpZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <AB6074E06B6E204EB8A1F2C711B27186@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f35a75e-c709-4b13-d1cc-08d9818f5f3c
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 08:18:22.6917
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SHUVknaGId8XPhPm9NTFjMonLejWY0qgrUBlV9rMXWjmYwz9MgoSaKTwlCd8ArU11p0YMFXo2knjRHpLY7SYZ1C+n92TTidItr6TnjvK9TwujP+w+TnEVyOqoJ4mFQzB
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0301MB2196
X-Proofpoint-ORIG-GUID: sIw-ONtog9wC-gUww8bo2OPlWQ5Gua2j
X-Proofpoint-GUID: sIw-ONtog9wC-gUww8bo2OPlWQ5Gua2j
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0
 suspectscore=0 mlxlogscore=842 adultscore=0 spamscore=0 malwarescore=0
 mlxscore=0 clxscore=1015 priorityscore=1501 bulkscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109270056

DQpPbiAyNy4wOS4yMSAxMDo0MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDIzLjA5LjIwMjEg
MTQ6NTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEveGVuL2FyY2gv
YXJtL3BjaS9wY2kuYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2kuYw0KPj4gQEAgLTI3
LDYgKzI3LDE2IEBAIGludCBhcmNoX3BjaV9jbGVhbl9waXJxcyhzdHJ1Y3QgZG9tYWluICpkKQ0K
Pj4gICAgICAgcmV0dXJuIDA7DQo+PiAgIH0NCj4+ICAgDQo+PiArc3RydWN0IHBjaV9kZXYgKmRl
dl90b19wY2koc3RydWN0IGRldmljZSAqZGV2KQ0KPj4gK3sNCj4+ICsgICAgc3RydWN0IGFyY2hf
cGNpX2RldiAqYXJjaF9kZXY7DQo+PiArDQo+PiArICAgIEFTU0VSVChkZXYtPnR5cGUgPT0gREVW
X1BDSSk7DQo+PiArDQo+PiArICAgIGFyY2hfZGV2ID0gY29udGFpbmVyX29mKChkZXYpLCBzdHJ1
Y3QgYXJjaF9wY2lfZGV2LCBkZXYpOw0KPiBOaXQ6IFRoaXMgbm90IGJlaW5nIGEgbWFjcm8sIHRo
ZSBpbm5lciBwYXJlbnRoZXNlcyBhcmVuJ3QgbmVlZGVkLg0KPg0KPj4gKyAgICByZXR1cm4gY29u
dGFpbmVyX29mKGFyY2hfZGV2LCBzdHJ1Y3QgcGNpX2RldiwgYXJjaCk7DQo+IFR3byBzdWNjZXNz
aXZlIGNvbnRhaW5lcl9vZigpIG9uIHRoZSBzYW1lIHBvaW50ZXIgbG9vayBvZGQuIElzDQo+IHRo
ZXJlIGEgcmVhc29uIGEgc2luZ2xlIG9uZSB3b3VsZG4ndCBkbz8NCj4NCj4gICAgICByZXR1cm4g
Y29udGFpbmVyX29mKGRldiwgc3RydWN0IHBjaV9kZXYsIGFyY2guZGV2KTsNCkdvb2QgY2F0Y2gs
IHRoYW5rIHlvdSEgTm8gcmVhc29uIGF0IGFsbCwgd2lsbCBmaXgNCj4NCj4gSmFuDQo+


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:20:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:20:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196498.349362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlsO-0006BH-RJ; Mon, 27 Sep 2021 08:20:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196498.349362; Mon, 27 Sep 2021 08:20:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlsO-0006BA-Nm; Mon, 27 Sep 2021 08:20:40 +0000
Received: by outflank-mailman (input) for mailman id 196498;
 Mon, 27 Sep 2021 08:20:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUlsN-0006B2-6Q
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:20:39 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cb6e9c60-1f6b-11ec-bc42-12813bfff9fa;
 Mon, 27 Sep 2021 08:20:38 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6930F1FF54;
 Mon, 27 Sep 2021 08:20:37 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 53A5913A42;
 Mon, 27 Sep 2021 08:20:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id dl41E9V+UWFgSQAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 08:20:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb6e9c60-1f6b-11ec-bc42-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632730837; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=+2pgmZ9PBvLLM0BIolHE/Tn+LBvVRbm1JLOt9XmsCMc=;
	b=j4Hgc7x75S6IBvzp4fm/pWg3FAobqB402nnZtY9SW3tSpr68Mcyw7WOcrjr1+bclAQKcZm
	QOiYM0/g6ypszovQ9xeeLmAydwRIz4SquDa6rh7datt+4ppUCHyoGPN9GVXuOOiFqmEbdj
	Br8n1LKYbjkD7QcBlYuR9LW3POA5wVw=
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
 <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
Message-ID: <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
Date: Mon, 27 Sep 2021 10:20:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="xpwh3UHZK6BgyEDnDPX0bKDMeqSKSvmLL"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--xpwh3UHZK6BgyEDnDPX0bKDMeqSKSvmLL
Content-Type: multipart/mixed; boundary="Knre5ARVZOlZWr52fSLknyf3PAa8097Uz";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Message-ID: <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
 <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
In-Reply-To: <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>

--Knre5ARVZOlZWr52fSLknyf3PAa8097Uz
Content-Type: multipart/mixed;
 boundary="------------7ED7949F2496B598604B06CA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7ED7949F2496B598604B06CA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 10:13, Jan Beulich wrote:
> On 24.09.2021 17:04, Juergen Gross wrote:
>> The interface definition of PV USB devices is lacking the specificatio=
n
>> of possible values of the status filed in a response. Those are
>=20
> Nit: "field"?

Yes, of course.

>=20
>> negative errno values as used in Linux, so they might differ in other
>> OS's. Specify them via appropriate defines.
>=20
> What if new errno values got used by the driver? Would we alter the
> public header every time? Or is the likelihood of further values ever
> getting used vanishingly small? In how far would it be possible to tie
> these to Xen's public/errno.h instead?

Those are the current values returned by the backend. Other ones used
internally in the backend should IMO tried to be mapped to the values
defined in the interface specification.

>=20
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   xen/include/public/io/usbif.h | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usb=
if.h
>> index c6a58639d6..fbd6f953f8 100644
>> --- a/xen/include/public/io/usbif.h
>> +++ b/xen/include/public/io/usbif.h
>> @@ -221,6 +221,13 @@ struct usbif_urb_response {
>>   	uint16_t id; /* request id */
>>   	uint16_t start_frame;  /* start frame (ISO) */
>>   	int32_t status; /* status (non-ISO) */
>> +#define USBIF_STATUS_OK		0
>> +#define USBIF_STATUS_NODEV	-19
>> +#define USBIF_STATUS_INVAL	-22
>> +#define USBIF_STATUS_STALL	-32
>> +#define USBIF_STATUS_IOERROR	-71
>> +#define USBIF_STATUS_BABBLE	-75
>> +#define USBIF_STATUS_SHUTDOWN	-108
>=20
> Nit: While probably benign for all practical uses, these negative
> values nevertheless would better be parenthesized.

Okay, fine with me.


Juergen

--------------7ED7949F2496B598604B06CA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7ED7949F2496B598604B06CA--

--Knre5ARVZOlZWr52fSLknyf3PAa8097Uz--

--xpwh3UHZK6BgyEDnDPX0bKDMeqSKSvmLL
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFRftQFAwAAAAAACgkQsN6d1ii/Ey9M
iQf/TPJrYZRCYEoqhWblye96zNt6yP4Pc8SKl7v4bSWqbEroGNnK5/0+MPxBosFW9f3vfCPL32fK
W+fd86VgNxK+B7g3wkAOmURhwFQeMg4BD+/9Oz5ZwMsxMjAOPhLVa5r97ZvMQ+QHBjetHHMN9fIa
m/y1P4FrAZ8h7g9aVq9osj8Y0ZK+z73vV7gIx5dS4A+KBH4klZPr+pWLLyxOSXzcaoXtJqmRArWc
BkfgRqfiK9WbOn5Gn9W3n63luJYgtJEBq+QJRV5ktYHyoouhh7AUqxOlxOxuFcUaywtZJhFARbT7
w0Ao+O2/U/d91NQDaOnOeBo5F24NEmeZUbxSKKmKSw==
=ZvU2
-----END PGP SIGNATURE-----

--xpwh3UHZK6BgyEDnDPX0bKDMeqSKSvmLL--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:27:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:27:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196505.349373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlyy-000753-IG; Mon, 27 Sep 2021 08:27:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196505.349373; Mon, 27 Sep 2021 08:27:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUlyy-00074w-FA; Mon, 27 Sep 2021 08:27:28 +0000
Received: by outflank-mailman (input) for mailman id 196505;
 Mon, 27 Sep 2021 08:27:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUlyx-00074p-D9
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:27:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 74d04a7c-e948-41f4-9c56-333b2ded4b1e;
 Mon, 27 Sep 2021 08:27:26 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-nPJbHdmjO0mSUSUAbpds_g-1; Mon, 27 Sep 2021 10:27:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6862.eurprd04.prod.outlook.com (2603:10a6:803:130::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 08:27:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:27:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0006.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Mon, 27 Sep 2021 08:27:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74d04a7c-e948-41f4-9c56-333b2ded4b1e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632731245;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4FJpTFZ8F6mjIpt0KUVzZy0+zeEyXsMPTLCOwRbaFdI=;
	b=jly6itxpvYR3KBU5FBMo8JaYCRkDg/NjzC2GGbZEsWJj7AfmUdXYahIVlOcQvw11JL/gfk
	anYospf/E2WpguI3r5oe3flNZXM5Ty9f4jwpldBTnSEDIjR0HMNp/4ldB4bLAAZC5obHEZ
	3wCvxN0M8ODo0eJvJAwQQY9kxIXTPQk=
X-MC-Unique: nPJbHdmjO0mSUSUAbpds_g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IHkFGTM8Im4iS8soP9Nl6sdLVWdlkg/sVZFK2lWeMU836iB8UaIlblw0mnETTjvK4+ISZgQYDUvKk61VtzyfTnNKeLw2DptfQBxB5q8bILQSNBTsV65CGmpS5onALF2EFWAobslNpQmaLldftFABqCQnBwPfB/RkkX6i7rK14bwEh5DvVFbchvl1qRWoJJeq0g6ghXV8KhcBaFrZWigPdvCLGed1HIWIQ8FhalEVOykcLPjh1stY8GwsAPLlvHgSzEthqm83el7NCXTbYNh1Zg1iADE9RgoNFfKuvnAfTjeY3SdU7NPgngY6p1QGq6B07mM17Vn5jANzhmqcD76UVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4FJpTFZ8F6mjIpt0KUVzZy0+zeEyXsMPTLCOwRbaFdI=;
 b=XDAapQAPdS3vkQxeL4mjm2/Z3ZQ54VHV1wzVwN2Pc/B1JstU+N06UTaE9bsxVpqQSloPvts10lLMkEQxgmsTeEu9JuF/O4xlfKNegE4KqhqIeKUX0xV7W4JWza5sdSsNd9F0mXPizH61gjrxs5krE2qz44V3UCuB/nPBXIv4M+nDFc7dPeHEBlb4Jg8ZlODNMrXRudWXit2ms/YHszjTqkmsvV690TcqoKeiVjqbY6hL4BopfLAJahOZuldDYiSXNnotM2PrINwI8KeNgJX5WWGMKE2Z6ziieGwtQ7odJOOoi59k5PNeBu0TQhmt+JadIGZF6EOUlzscWNCOjM40Gg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
 <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
 <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <795058eb-7489-8303-fd24-14b94762f74d@suse.com>
Date: Mon, 27 Sep 2021 10:27:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0006.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::18)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1a32e566-043c-4c02-b1fc-08d98190a016
X-MS-TrafficTypeDiagnostic: VI1PR04MB6862:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6862E48A7EC2858C0D47AF24B3A79@VI1PR04MB6862.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZDyHYokUfFFqiYK/30B+deYZ5xopETKeQGabzHpFBvICrM2uUcJqaHm/PcgDaMz3fip5FE2RRYhJ11KewxTgGwr4YHDh13HM+SJx+Yi6r/17DZqUSwCNk+g+pCRzJl4bewBSZrJzalIr4Ql3LNYZgMc9lTYlKAnqxCTa8kbDDFrA0X4qlhEhQ2/L6qf7+Mw3BTP7dMmFGGIWJF0oDlJRAbTFSrsIgF7ghVzUOv6gpBGrqd1MM8gXZBVHv6n/hewmf1PraZ695qOovgKmR2R0qYxb7rzjxuVal2OJ6hbsvP9AtpiBwv1aRbyqnhQxIVz11w3PfzpcQ0QpDWRjqE0uYlGVfBypHzOwDY/WwD1V+mnkhl66I9ttgfS9PsgbIyGKlfnSIW3/PHp4zMgvO20jrtWLdAjACVVHMEPBjzcnkkmFaOoANaMxikp/4dyhSB5oE0knp9kXjE9uL2s2gnkzvEvtgvX9L8G4wuZCQmG/BNDJYXv7rhRad0oYyEbjThZ4FQNA6MoRPMD93kJi5P9NL9ergaKCeuMJrZSRx2ycz+5+tB3syUM5pmUKacIZNKbdOadAxdW3Vk0FvPdiQ8HT4kT/uVYuwxt9/f6qvDKO7UZY31AalexM166pUTv4393YKdoUehUntNHn417SBVF3SNXXm/E0lvWb1PZdcvMhCEpmcY7/vDTHrtW79iAMWtyAC/2csdlScMqd0WTVaHB0COrbCDsE5g9VtFPJ+/rZ5qM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(38100700002)(66556008)(53546011)(6486002)(31696002)(316002)(66946007)(26005)(8936002)(86362001)(31686004)(66476007)(16576012)(6862004)(37006003)(2616005)(956004)(2906002)(508600001)(4326008)(6636002)(8676002)(36756003)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UEVrQmRGc2UwYzg3MDRMdGlUdlg5SE5ySTM0ejh6YzhoWXd5RXhZUDUvUHFS?=
 =?utf-8?B?ODZWSFpwdFpHTGpkeTB0K0xnR0llTllWcWRGMUErbU9EWU5EVFIwUkI1eTky?=
 =?utf-8?B?OW1ZR044RVhCWlBuUzFHUUhNVEZyeWVFeDJTVFFkbFZ4dHByMUlHS3NnZTBt?=
 =?utf-8?B?Y1pydHlvYUMraVFKU3dGSzRpeVBhTkt1Nkgxc04vWDVlSXR5K0ZKSERaSlla?=
 =?utf-8?B?Z1V6TjVJT2JXOHJzdGJEM0NMNW5nTFBhd2l3RjlZN2wvbkdZVElmTHVBQkd3?=
 =?utf-8?B?dU4wcmtDc1hBcmN3enJ1Q2JtL283U0Z4dk9zaDJWallQQU9MOXRyTzlpVGF6?=
 =?utf-8?B?TEZOTUF3ZUV3UTNXVVRxbzkzNHlyTkRYMmlQWEU2cFM5V2lYTWRwanR4QVlY?=
 =?utf-8?B?bEpGclJrZ0JwQndaNnFBTk9nZ3d0THluQjQxd3hZSWYxMDA3ZGZzbzhSeXBy?=
 =?utf-8?B?a0ZjeUkydEZ3YWxYd254K3RlUkdwaEkwNHlwSnIyR2FWRk90N1FLLzlMd0pl?=
 =?utf-8?B?ZjhvNHA3d3J1UU1GaDRNd05hUFFPTEZmeXJSS3JFNnpLS0tZNnczbktHaDho?=
 =?utf-8?B?K0RtL3JReUxJblpFc3RuOXFsSXBWYThBVEU4V0hsUm12cDl5aEVTbC9PVktI?=
 =?utf-8?B?NTl0cGliNmhPS1ZoaWxwT0Mvdm13Z24rb3lqamcwdlJ5enJFOFhFMzVobEVT?=
 =?utf-8?B?K0ZrUXF0MS9sQ2dVOFYwUVB6VkxSTURsQ0ZmS0JmL0ZzWVZLS285UG16Zmc3?=
 =?utf-8?B?UmdNZjNtWHZNRlVrUjJUSjdreHZWU3JRMWo5S2NXZFZqbDJrdjZ1dFB1N3RZ?=
 =?utf-8?B?cjNWMmY1TmlLNDhuMnhlMXRoNjM0ZlB6RU15R0xYdURkdVEyZXRZNXYybUtU?=
 =?utf-8?B?eCtVWFkwYXpWc1Q5eUtWcnFVYmpjWWU5bDFWZHo1QW0rV05Wa3RianczVDZQ?=
 =?utf-8?B?L2NVWmRyaFR4eTdsMWhyem51aXZ6cU1UamM2SEtxRFdVNE8wMllzemZobFRn?=
 =?utf-8?B?NWNXSTRKZ1ZxWmxralNUajNRYWcySDFmT0w3K2Y0QnJpengwN1pBU0IwQktK?=
 =?utf-8?B?OUNGOFJpMTVwNEF3N1BFWlVSbmFNTnRJbWl4cm1hVnRrWjhLM3ZoUFZ4ZG1u?=
 =?utf-8?B?WjVJOEI5VlhBUWNqWEhqeFhFWFRtKzYxa3pHNG9VZHhaeGNyK3dGUzE4ZnlU?=
 =?utf-8?B?dmt0WjFOUXB1SW5ZdmwzdXZuazA5OVRGTndCeGxYS3pXQTMrVHhwOEs1V0dK?=
 =?utf-8?B?THptQXViT05LS2hieUFjSytZdW1sYXdrUVFRVzZYYmVuUGtKbkJSa2t5Vlll?=
 =?utf-8?B?NS9pUENBdmJ5OEpzTlMxMkVyRmFZN21ucW5vSkpUYWF2ZEs0bFJJcFp1T1pt?=
 =?utf-8?B?aEVKcEMrdUFGQ2R5eGkzdzU1TjFMbjc4WFJDcW56cjU1bDEzajR6NjdEQVdQ?=
 =?utf-8?B?aHlpbzB3OFVrL2l4TC9zSTZGTTF4a3lmd0wzVUZNOW12UG0rYmNtNElOS1NF?=
 =?utf-8?B?Y05EMzVSd0ZrdGVFK05GZnFaZGUwZFBrbDN2SzhKVUg1Z0ZYNW8wWkE4MlFM?=
 =?utf-8?B?b3JZYWFZYVMwbFhLQzVnUXNnajk3NUR5YWRCbFI4eFpkT3h1T1NaSlBiSHF2?=
 =?utf-8?B?cHlIQ1ZuSWdicE5Rb2RlRG5VbHp5bG9YYm55THFhVDVGOEI3c2x0OFQydXEv?=
 =?utf-8?B?cGNxYjhlejhncHNWRDVRbmlXMURvRGtLQlZRV1pYYzJpSWZhVElxNXlxNGhD?=
 =?utf-8?Q?ymoF/q67uu79UPtElNlIPYJ2jMLtifoesN6kch7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a32e566-043c-4c02-b1fc-08d98190a016
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 08:27:21.1800
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jiWs9Q9YD+wmmJzCbCTiNyKTew0PTaStkec1W1bF3bCtEHsxq0zm1MjEQo/YSAL5WoYOxdpwanbic9X4IiwYpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6862

On 27.09.2021 10:20, Juergen Gross wrote:
> On 27.09.21 10:13, Jan Beulich wrote:
>> On 24.09.2021 17:04, Juergen Gross wrote:
>>> The interface definition of PV USB devices is lacking the specification
>>> of possible values of the status filed in a response. Those are
>>> negative errno values as used in Linux, so they might differ in other
>>> OS's. Specify them via appropriate defines.
>>
>> What if new errno values got used by the driver? Would we alter the
>> public header every time? Or is the likelihood of further values ever
>> getting used vanishingly small? In how far would it be possible to tie
>> these to Xen's public/errno.h instead?
> 
> Those are the current values returned by the backend. Other ones used
> internally in the backend should IMO tried to be mapped to the values
> defined in the interface specification.

In which case I'd like to reword my question: Is the set of values added
sufficiently rich? There's e.g. no ENOMEM equivalent.

One other observation: Why is this header using tab indentation? This is
by far the largest (albeit sadly not the only) of the style violators in
xen/include/public/io/.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:34:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:34:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196511.349384 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUm5A-0008VB-9Q; Mon, 27 Sep 2021 08:33:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196511.349384; Mon, 27 Sep 2021 08:33:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUm5A-0008V4-5h; Mon, 27 Sep 2021 08:33:52 +0000
Received: by outflank-mailman (input) for mailman id 196511;
 Mon, 27 Sep 2021 08:33:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUm59-0008Uy-0U
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:33:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4310cad7-7447-4375-8927-a1d1cbc27737;
 Mon, 27 Sep 2021 08:33:49 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2050.outbound.protection.outlook.com [104.47.13.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-KYw03PszPD6KURfOaR2G4A-1;
 Mon, 27 Sep 2021 10:33:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5325.eurprd04.prod.outlook.com (2603:10a6:803:60::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Mon, 27 Sep
 2021 08:33:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:33:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0249.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 08:33:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4310cad7-7447-4375-8927-a1d1cbc27737
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632731628;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=iyMZMmPtZ3Mp+s6sZOxcPkrbJcV0Hxysk8STaKxn9nY=;
	b=Cu8RCHsN6YPx87B+7BXMaN4r4VVNJNghfXjd/rwq4hJJjXS/C7hcN6A271P8Oc9aVgYD/j
	3yfcVnsyiGc0p9o7nfAlp5eWmagzvX0h5nrEImp8tlcV4Vd5zEtde/CBxiDz536J3vceP9
	Cb/Ca/bXJI5jeTBwgOBpy2giY0+x6ws=
X-MC-Unique: KYw03PszPD6KURfOaR2G4A-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BtgQhDKTyaZtPIGKWgp/aRW7Y2FqlogMlNj67QYdSCq0SBUSb81uWYEWtg9kDQ1pQvonaZ1PjJcCccxKnAkNnLpAMQbOMXFsdFbPNaBN/dl8HuJU/TdFgJYqkA6ie4JJH8VW7tPR8sFqeda+WZqwej6MDNQ5FXw/wjWGPrea4ncM6MxutM0s82ZjpVZKS5jljOuNcFKSQzCzva1aDFGrzJaoG9YJieik0Y1UuklZ69v85erlC9rvpGTwysZcPV9KGVwOfMTxA0KaH4Kflx0XI6CRcYslzfom0334o6B26PBVpe16SQlPkoMeSleHCqnp6cRTgVXsZdRVqEshwqk8mg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=iyMZMmPtZ3Mp+s6sZOxcPkrbJcV0Hxysk8STaKxn9nY=;
 b=dP9nib0dDHuSeMA/+/Jd9xZp1cT9SMRWsQDUhOpv2j2OPHzY4pzfC0JMQy0OBvn74p8GPhidfldwagpCJgBkpscM9KOLkHjTlO0jP1JuRgtlLDmgbHu9qV03u6T0NP0l1fclPkWLBRnz5p+BBoaQWB6gF+i62wMQoafkyfScVkLtCCZZj7hUvnxMITar6+ag0BfOF+nzvZp7MPdvCmvhfYhfwFY4izryqjJNSBzXmHGaDOnsDGMxwBtooe+tCJAILarc4WhbmnpGKGaLNUOoBN+slVrkWwpvroIYTGvLxflAcMll9OxFBtP1/6TLHyhU+m+QmxzqB38xkjVKl9PlzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
To: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>
Cc: xen-devel@lists.xenproject.org, julien@xen.org, Bertrand.Marquis@arm.com
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
 <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
 <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com>
Date: Mon, 27 Sep 2021 10:33:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0249.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::21)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fc6b3a52-69ba-4617-317c-08d9819184bd
X-MS-TrafficTypeDiagnostic: VI1PR04MB5325:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5325537C59E7A4FDB118B775B3A79@VI1PR04MB5325.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YNGJNGuKV/xDGPwNYJySnaG5UAI7CAiHn/hXk0YT/5S6wuDSHT1jBc9z6b0FP/bF8Q6JAX+fDR0UoVMcQDlZAK/yGkyV7TUa3bNC9UJRLLHAdj0IM7JQ1BpOvnGhFNAbssD2PZR73hruKTKXQqvvGkc7UUTd7TMjoSBMnaIkhKn6gMt2tJZi9yfF2cMnnx8lKVywTug2Mx+VztqgeeBNuQhaS8VdisA59H98hvg4NieqIKFr+IkmrBEmPeyNRZO+ia8Szrhsox4sarzWodvQo4EKMKtyVnzmYuplCNcGUjVKuhbX9pJbho/Png6PhwjGj1f/N7c6/xUjS7XvpksVkDn7zgK5aT9u2fOKaBzW0JLU/XjFu7uGaC7P7tPPQiP/+y3pBtC/HDpdzK1HNS91HX2huO3v9GotiduDqI92pTUrk53NiTZntquowKBu1O5rKlOXcjdnCWZ/Fu66v2Wvc4StwvDZjVUYzo0FZcOzgQD1lFTsYDBfXZklv41tv5t9rt+9O6tp+xLEfjXq3/Ohkr6Z+8Muy9T1/xBEfZ1UpuFDIJWiHOyGqhoOSfL28aojbYnsKQouQWnWARWBy1ZjFXP6oVDXrF2aY6QBJHS0ALOVEdsXc2z8po8jxPVycgqUSCPaPEQsoGzQd2QqxT5SY9ffTGAxbQIohoUTNc0ZN+o1PbnunEbs1EAohCL8dwaID8NS3HIBvQXJUw9qfjqZ5GmGBUblgTzR3A8mkOFDp91jgwAscX2egN1uYE9ZOm3+f975YpTXPYUg/+P0UXYNNPHVD/9X4ijnk+e8A90w91nHGnAI/poCWn0MlW49Xvuv
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(2616005)(508600001)(956004)(316002)(966005)(66946007)(8676002)(38100700002)(186003)(2906002)(110136005)(6486002)(31686004)(5660300002)(86362001)(36756003)(16576012)(26005)(53546011)(66556008)(4326008)(8936002)(66476007)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?U0xTZ3ZPR2RDdGorNTJZV0pWWXg5QmF6Q2xWdmV0bW84V1BtaDk4ay95MDJF?=
 =?utf-8?B?WjF5UGh5bXVtUnE0TmZUcWU0Q0dUc3ZUWVAzVUFKV1BtaWFBVzd4YXEwMytF?=
 =?utf-8?B?OGNZRll1U1lSZDc1WlVJSnUyc2llRmVaam05WU0xVUh6OU40MVg5alpCRVYw?=
 =?utf-8?B?Zkx6SktETFBUc29zUHJ3YU1CeUpMZUg4OEJvS2MvMVlUR3puci9nSDZBZDN1?=
 =?utf-8?B?a2ZBWXB0VWRmSUpFYUY5NG5oNWJkWktJZFpPL3dOVGlha2wyamwzVTQxMDlF?=
 =?utf-8?B?ZUlaV2hRdGJKa2xWcnB6U2hweGY2dHhWNjBwL2ljUkFhQmRpczRRNDNHS3RK?=
 =?utf-8?B?WGRzSEFuWXhUV2g2cWJObjNxeVhsaWlGaUdNZTRKeHRmM0NWaklxZmdqUHpt?=
 =?utf-8?B?ODQ4c0VJVENycU1WRzRzODVWUzJGTi9PaVRvOU9FbW4wTW1RcnNPQkdGWlg1?=
 =?utf-8?B?UEM4ZGs4WVROV05QQjB6bHNOT2tlWGU4OUJoQ1Zub0JldmJHOVZHYktBM05n?=
 =?utf-8?B?dGNpbCt1TFY3U0E0OGRlVThMWWI2OTY5Y3FQWWd2eFUzV2s4R0E4OHk1bnZC?=
 =?utf-8?B?b2FMUHBHSURwZnl2djRHVWZFSFh2NHFvdnRMTDREcHNTVzJMUUliWVJuY2c4?=
 =?utf-8?B?QzNsM0NVOVlZeHNDbkFTZVFPWlFoOTArTFhKUnNtYXBtSkIzczEzQ3FQSGpT?=
 =?utf-8?B?RjcwcE5DMW1qTU4wdWxFdDFZK1NTbFpaVlZ6Sm5ENEJoemNndWNRV1lVbU1p?=
 =?utf-8?B?ZnJjcnpIUmNIcVhQZFNJSzlaMmRHV2dpWlhuS2NUVHkvaHN4czV5WnRULyt1?=
 =?utf-8?B?U3Iyb25OK0tNNEhtMGl2Z3drTHZoT3EwY3lVdll3RzlGVWdWREN4alB1UC9P?=
 =?utf-8?B?Y3hrNUlPWURQOFMzNnY5Tjc1RjhvWWtqRVhOUzdVZVBFd3dTRWhIWEZZTVIx?=
 =?utf-8?B?YkpaWEE5TGtBek82S1oxMUE1YTRXaElDZWpVWGd4NTI4dnVIS20za2p5L0Fp?=
 =?utf-8?B?czFmNERRdllJVUZOSkVablpQMFZUZ1NpV2p3d01pQU9BWkdHZkVjUStoaUp2?=
 =?utf-8?B?ZDZGb0pHbklaSmFlSVkrUVJLYzN4NWdLbGxNa2cxRStsQjI2dDk0ekNBdDYx?=
 =?utf-8?B?ZTV4WHlJbkxSVHhEUmV5cnc0WjVYTkV4SWErekhwWUUxeFVqWkg1RkhzREZt?=
 =?utf-8?B?cDQxU2RNUFVuYTBpUDdSNTlhcVJmMCtyRmRjcnR5NXJ0VlFBSy80RHBrdTF5?=
 =?utf-8?B?T0xmL2EzeFhXRVVKcHNEZGxiQ2tES2M2b1o5THFjUnVBSE5GTFBCTG5BV0NW?=
 =?utf-8?B?cHJFd0MwNWsxN2orZjFOQXNNbHgrNysrTTFRRmtOSWdIZUNOWmh0OTVMQVJi?=
 =?utf-8?B?cUJJZmNPQjJkY0xYcDdINkxFdmhMV05iY09YL2lqTXFEeHo1cjdlSFBIVUhN?=
 =?utf-8?B?UzB4ZStsbnBFbnR2c2dQNHhObXM1UFdndzNqR2Jpc2tKUndrcDQzV3Y1ZVRj?=
 =?utf-8?B?d2ZiY2NQT1FKbG5hTFEyaDBwa3MyTlR1aksvK2pEaFBzYW54emNWdll5SHlN?=
 =?utf-8?B?SFV4MVVKY1UvTFRGdzZLY0ZtMUFLbHBRaU01L1dheml5b1dhT0F0NXFFU29K?=
 =?utf-8?B?bzNoREpnSWNDWm5iMDArTE9hNnlGWFFQeXc2TnE0cmlidDRPdHdXL0V6TWR4?=
 =?utf-8?B?cEZpVFdpOU9NYlhoakZVNTZRVm5mWkMxYWJieEU2RWh3eXB0eTlTL09ReFdi?=
 =?utf-8?Q?/fptN3Y8xsQ8WVgzV6OROLl2eLj967lePWxy+5D?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fc6b3a52-69ba-4617-317c-08d9819184bd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 08:33:44.8147
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DEueJpU4otjdUHRH6SGltBZpY4YrGJ280mUtDR2N5eXN6Tk+INtjTQ9Yhai647t+3JiPzd0J4pDTdrzHOvJgHg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5325

On 24.09.2021 21:39, Stefano Stabellini wrote:
> On Fri, 24 Sep 2021, Wei Chen wrote:
>> On 2021/9/24 11:31, Stefano Stabellini wrote:
>>> On Thu, 23 Sep 2021, Wei Chen wrote:
>>>> --- a/xen/arch/arm/Kconfig
>>>> +++ b/xen/arch/arm/Kconfig
>>>> @@ -34,6 +34,17 @@ config ACPI
>>>>   	  Advanced Configuration and Power Interface (ACPI) support for Xen is
>>>>   	  an alternative to device tree on ARM64.
>>>>   + config DEVICE_TREE_NUMA
>>>> +	def_bool n
>>>> +	select NUMA
>>>> +
>>>> +config ARM_NUMA
>>>> +	bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)" if
>>>> UNSUPPORTED
>>>> +	select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
>>>
>>> Should it be: depends on HAS_DEVICE_TREE ?
>>> (And eventually depends on HAS_DEVICE_TREE || ACPI)
>>>
>>
>> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
>> option can be selected by users. And depends on has_device_tree
>> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
>>
>> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
>> does it become a loop dependency?
>>
>> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html
> 
> OK, I am fine with that. I was just trying to catch the case where a
> user selects "ARM_NUMA" but actually neither ACPI nor HAS_DEVICE_TREE
> are selected so nothing happens. I was trying to make it clear that
> ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACPI
> because otherwise it is not going to work.
> 
> That said, I don't think this is important because HAS_DEVICE_TREE
> cannot be unselected. So if we cannot find a way to express the
> dependency, I think it is fine to keep the patch as is.

So how about doing things the other way around: ARM_NUMA has no prompt
and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt instead
(and, for Arm at least, ACPI_NUMA as well; this might even be worthwhile
to have on x86 down the road).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:39:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:39:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196518.349395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmAM-00010q-2j; Mon, 27 Sep 2021 08:39:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196518.349395; Mon, 27 Sep 2021 08:39:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmAL-00010j-Tt; Mon, 27 Sep 2021 08:39:13 +0000
Received: by outflank-mailman (input) for mailman id 196518;
 Mon, 27 Sep 2021 08:39:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUmAK-00010d-OB
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:39:12 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 21fa6869-18a8-48d7-8370-d93c1aae2b8f;
 Mon, 27 Sep 2021 08:39:11 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id DB6BD1FCA1;
 Mon, 27 Sep 2021 08:39:10 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BF67213A6A;
 Mon, 27 Sep 2021 08:39:10 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id zrCPLS6DUWFgUwAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 08:39:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21fa6869-18a8-48d7-8370-d93c1aae2b8f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632731950; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=/PepU/LSHcq6TFMs1pB5DOt7QZ46FEunMd4xzBPB8Kg=;
	b=lOvVl+S37dWhlanslNkNr+R9cEIAm7v4xwXqiELFLpvkN9GoZ0n/Z2MDOVUifx/Bx2sxeu
	tb98hgNqvxjQn3stPf7vJFLw+Km7D5/Wy7B5MD8sgpzc5qtwkGilLw7yAyE6hDIz5pf3yk
	aplDGCVpNzP/2pqu4Ltd8ByXZZvY5vA=
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
 <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
 <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
 <795058eb-7489-8303-fd24-14b94762f74d@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <e8de197a-78c2-fa79-217b-8d3e62b08576@suse.com>
Date: Mon, 27 Sep 2021 10:39:10 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <795058eb-7489-8303-fd24-14b94762f74d@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="fZOZ0TIThMgs5RRlYgAo5Tw6GHLWZxf08"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--fZOZ0TIThMgs5RRlYgAo5Tw6GHLWZxf08
Content-Type: multipart/mixed; boundary="YIMQNNkCwCgJGF6oeDmBuOaUlum7PWgzI";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Message-ID: <e8de197a-78c2-fa79-217b-8d3e62b08576@suse.com>
Subject: Re: [PATCH 1/2] include/public: add possible status values to usbif.h
References: <20210924150424.10047-1-jgross@suse.com>
 <20210924150424.10047-2-jgross@suse.com>
 <8d9f549c-108f-17a8-e8f3-fb9b52442d5f@suse.com>
 <28a9b815-0903-a06c-1b17-4aeceb3604f5@suse.com>
 <795058eb-7489-8303-fd24-14b94762f74d@suse.com>
In-Reply-To: <795058eb-7489-8303-fd24-14b94762f74d@suse.com>

--YIMQNNkCwCgJGF6oeDmBuOaUlum7PWgzI
Content-Type: multipart/mixed;
 boundary="------------97330C727B8B157167884C5A"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------97330C727B8B157167884C5A
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 10:27, Jan Beulich wrote:
> On 27.09.2021 10:20, Juergen Gross wrote:
>> On 27.09.21 10:13, Jan Beulich wrote:
>>> On 24.09.2021 17:04, Juergen Gross wrote:
>>>> The interface definition of PV USB devices is lacking the specificat=
ion
>>>> of possible values of the status filed in a response. Those are
>>>> negative errno values as used in Linux, so they might differ in othe=
r
>>>> OS's. Specify them via appropriate defines.
>>>
>>> What if new errno values got used by the driver? Would we alter the
>>> public header every time? Or is the likelihood of further values ever=

>>> getting used vanishingly small? In how far would it be possible to ti=
e
>>> these to Xen's public/errno.h instead?
>>
>> Those are the current values returned by the backend. Other ones used
>> internally in the backend should IMO tried to be mapped to the values
>> defined in the interface specification.
>=20
> In which case I'd like to reword my question: Is the set of values adde=
d
> sufficiently rich? There's e.g. no ENOMEM equivalent.

Those are mapped to ESHUTDOWN today.

> One other observation: Why is this header using tab indentation? This i=
s
> by far the largest (albeit sadly not the only) of the style violators i=
n
> xen/include/public/io/.

Good question. This seems to date back to its introduction back in
2009.

I can add a style fixup patch to the series.


Juergen

--------------97330C727B8B157167884C5A
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------97330C727B8B157167884C5A--

--YIMQNNkCwCgJGF6oeDmBuOaUlum7PWgzI--

--fZOZ0TIThMgs5RRlYgAo5Tw6GHLWZxf08
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFRgy4FAwAAAAAACgkQsN6d1ii/Ey9v
hgf/enOFGES+bH3bIVamGP6JFrCREZa4+wIuhJhjB9+QEVvSwwU6/YNXnBkK5NRL1WoV2ii3Fzen
YaOvI/o+5Hcqra8l8fo/ZyFaggwO1v9xqQbCsi2+QSOzPaMf89uDzghDFeLCZqMGh+57VlcDF8hU
svOygf0C7UJgvFvdGn038dAtmT/QuMq3AzMLG3d+2yuJSAP1wU4wfDvFsiH26zsF2aCJXIyxQ+BP
dXg2BEppnVGqSjaDooS19vx3Pq05tnQtVlJr08YdsWMe4rm2WAAf7hRXQYoFPG2UqDLzMQfUie96
fyc0aWPVFT8vEP/u/C+/W3uXilWt2Uaj8gK1Of463w==
=+p45
-----END PGP SIGNATURE-----

--fZOZ0TIThMgs5RRlYgAo5Tw6GHLWZxf08--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:39:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:39:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196520.349406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmAX-0001Lc-A3; Mon, 27 Sep 2021 08:39:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196520.349406; Mon, 27 Sep 2021 08:39:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmAX-0001LT-6j; Mon, 27 Sep 2021 08:39:25 +0000
Received: by outflank-mailman (input) for mailman id 196520;
 Mon, 27 Sep 2021 08:39:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUmAW-0001KO-D0
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:39:24 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6a264c70-1f6e-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 08:39:23 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R7wurf019347; 
 Mon, 27 Sep 2021 08:39:20 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bba1n06c5-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 08:39:20 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5794.eurprd03.prod.outlook.com (2603:10a6:208:157::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.20; Mon, 27 Sep
 2021 08:39:17 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:39:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a264c70-1f6e-11ec-bc43-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jYLMIC8fIB43CNcb7j9RRgSXPUdRliVyrM6hyWSxqKR+21nF7NZA6IbzS1KJoHsuvkk+Sey2OuH5Uw0BVhyPILB31Y6H1dk7pIjKnofZHIEsHsWSC2WLA3K+AL3bPYLLDRiLfrc9bjn+VIaDyEOUcdOXxCtifuh6TifDoCgTpKeoC8CJduzRky8vUi0DmEKFrhs4R5hYiiVvLRj9AzD08WszZaxBXNAnRw8N0JMcND9YhPXwFxzjE2mKCRPQkN4CAk+L8I0fo3vmLiit2t43sXrzj4qXqB4XTuxEdKOKbV6jMLjM4YXda6rlSLycocecLxsHO7qUGVBMHCqbBZFkoA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Q2aDLSpXQzKyLXfEm9nl2qdd4wWIywY4rPfbjIgqjlY=;
 b=bH6nj78xW+ki0GtcHpmEzibT0kf9RjLg+o/3wV9zVmt/cSf7JWultHzdXf0g6Q8JyncHdZOMbYmLQIYkYsrkrYsMUh52eTI1zwFTbW5O2FujWyeHZsCb2eF0btYzLHKYaBqlIHhHTnyk7T1yIAAjV7KxxxEbcrKumbyxPKwr41HEl3QUS4XoGD84f6Wv09fQE0nsB9L0M7f/r2wA3+xE9Jh/DLhWFK0f3Ju55NS/sgV5IpLXvu0/4bBE9sq5sanUijx9qUppftkXX2b07Be0WZ3NehboVHbXJ+suwNroujTsiIUvsXM1lM+183SGYFpsgrL6FcaKDaWT5C2UezIamQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q2aDLSpXQzKyLXfEm9nl2qdd4wWIywY4rPfbjIgqjlY=;
 b=r6wwJfOTrMFSgdsoy0UydrLf58q1H24+bgEWcqlM8p5Tu86NDuQyTU2QnC1k7TGPpPBNXaxpKN+0V0YD5Der2owF07VIF8Vl8fqXg3QVLy7tx1wui23R/d2uU4h7sYKULVp7SPi+QWgWYYKIHi918cgzkqLXou1PYxhweRi97J0d9YU/4+rfmSExTLEpF6hBMwBw2NpQRZfZyuKgNoGQYFD7bvNm9UCFC0fNhSDIU9sCrVxFl+s90+XHjZ6DZVxWgm2vLl6oTcIUeo+HwUfBtqqMutwQqcakcFIdoqMmN/vErB5/15ZKDPl4Im0y0IIBKIofNtkUN9sCXPhwVzQYBw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <andr2000@gmail.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 11/11] xen/arm: Process pending vPCI map/unmap
 operations
Thread-Topic: [PATCH v2 11/11] xen/arm: Process pending vPCI map/unmap
 operations
Thread-Index: AQHXsHo2LHAX3buXA02kyLB7DlUL6au3jCIAgAAJOQA=
Date: Mon, 27 Sep 2021 08:39:17 +0000
Message-ID: <c2a9d992-082f-534a-a649-746cebc8b1d2@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-12-andr2000@gmail.com>
 <7bb451e0-9a99-e2b4-a2be-0377de38c0be@suse.com>
In-Reply-To: <7bb451e0-9a99-e2b4-a2be-0377de38c0be@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e4611c79-d4f6-4c6e-dc9f-08d981924b22
x-ms-traffictypediagnostic: AM0PR03MB5794:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5794354BD3D37E5F1FC6F5B8E7A79@AM0PR03MB5794.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 jnj2SZ2gfH90A08M4rVr224oYhvl8tqnzWfL4HVYnBCDaMtjv+8sZIgIcVE2imo0v34PqYu56p/WnZn4sLjAJQTyOKG8B8/5bfr/lv1pK9BT3tZhA+uiMBnMs83WCaZsHxbRj4JkzBNfskHt7kEo4cs4cv/0NFEfCzScneqBhrw01cIBtWLHQZ1WxAUYHSjmSPcIV/l41XZZK0nu3M6XvcrCmEfNfPOqXzrh1wVfaNsAcBFC4X4n6Mx27kalrFDo+r9qNZbTWdquLMiaTLlkWiQZ8jRsGatGvACQY3QfxC4OrA4PN+C9gUJQilEqaYErgfIJaER8w2UTZ+Xh6caJXJwJgORM78ZW1npY3quLo2OK74NkUOv8fRSziamYlld9Pjuu3TSttZU75SbhbKhgqKJaqw197JXeAL02M1v2Teo9zJnnNUnPan5/GuLiyXWB7zkAZhXOl38sEcBGNX9IItbIso8jugzpHe73v9WYFDskMEIjM4CA68Swq+nrys1YcuyDQQXO5lu2Mtvd44j5oDye+QXMV1v5RZlUyQwlVDzUhClANr91OGicNXyJb9fVwKzE+czRmJcNSawx7qVi1VHP5Qn7WSpXEb4aL1KXdIe8ok6ctY59fZe8Zuarhwq11pIFarb29Nqw19HUxVRIXf9g8go6ZaXAZehG5KLlFD22fLO+7Eo++DGO1/C90zZ2dc+jnB0se/eMm0YmWs5p0O3+KMsrAxmINKoLRnLS3f8=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66446008)(38070700005)(66556008)(6486002)(31686004)(2616005)(186003)(2906002)(71200400001)(66946007)(8676002)(316002)(36756003)(64756008)(26005)(6916009)(86362001)(66476007)(5660300002)(38100700002)(83380400001)(6512007)(6506007)(122000001)(54906003)(4326008)(8936002)(91956017)(55236004)(53546011)(508600001)(31696002)(76116006)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?OHFwS3ZZYmhUTHRqYVREQlRxSEpGYXdFdXVxR2d0aWpIaCtHUDdRRnhNT3or?=
 =?utf-8?B?K3VEbHZsZVFXUDRuWlZiOEp2b2lQTWRCbGVuWHZESHFkRm1aQWQ0NnRFVnJU?=
 =?utf-8?B?dXRUcFNMRFFCbG9LdXJkYkJ2ZEVTd2JTWGVibEJJbWZ6UmpteU9CUXd5bktU?=
 =?utf-8?B?VWoxY2plYXE1UUJCMndHTnplbkd5eWdEcjNodTdubzVOOEF4WkdqSDQ5akp5?=
 =?utf-8?B?dlg5NHJXOXYwSExkZ1MzQnBaTncxMGNramNHZWJtZWRkQUJ2VmZjSjQzNnhX?=
 =?utf-8?B?dlV4YmpnaUUrSktqa1M0NEoxbjhEUjZWYzRwM0pBWTZRejNEcm9zRVM3aGRF?=
 =?utf-8?B?ZW5TRDk4a0tQWHRUQkxSWUs5NFhEaHk1Z1J2SmlkS001NjhTdDZqMXUrMEJ4?=
 =?utf-8?B?b1VBTmEyUm0rd3BSbG9IUFZGUXpRQWgvY3ltZEwzT09WNW5LZDR6NEZzYldZ?=
 =?utf-8?B?L0szcGRGK0lHV3RnWHZhVm9MTmw0UlVmWmJuVFZuSmh1V0Q3SkFxWmsybjYy?=
 =?utf-8?B?RnoxRU54a0pkU09PU1BBSEdrb3BEOWZ4ZDB5ZWFsQ1dtNWtWMWJMdTZyaXR4?=
 =?utf-8?B?cHp0ZFhienk5Q1lzUjdjTDlKWDlGUllOTVkvZXIxOFRWSEwxQU1lMXJ6NlRC?=
 =?utf-8?B?MGJxQU5DZkRMTUkrRnV3clZBSjdqL09tQTM0MTZIRG5FNlptM0dFSGRBVEwr?=
 =?utf-8?B?SGtnYm52QjNHV21jYjV5bGt6UVdmeHNJYWlic1lwNlFucksxY1ZTeUp6WUVz?=
 =?utf-8?B?RTgxYzd6L0pDbGZ6MlBPeTFSVG9jTEdHTXNXaEdJdmdkODZ4VkVQNmtlaHNM?=
 =?utf-8?B?aWhkRng3bEtTUkJ1Y0V2T0hPcFhuK1JIWkxmL0IyYW85c3YyS1FRa1ZQMi9u?=
 =?utf-8?B?VlpFbDdmdWR3cS9nOTlNc2YvVER0VHI5ZEd5MXl5TjBuK08vNUc2b3E1UlVE?=
 =?utf-8?B?V1liWU5EL1V5UVF0MU04TFlGM0hWQjJrU3BVSHZMRFdxcG9NcHNZV3Y4ei9X?=
 =?utf-8?B?bmp2UGs1SHRMMEF6VVNsVFRMd1N5S1ViQ2Nhc1JWSkhEK095NkNDczZGMlNt?=
 =?utf-8?B?U2VSa2JnMW11Si8zd2JWUGU0UmZhYkErNUUvbU9PS0xtYkRlcXp2eFU2dVFV?=
 =?utf-8?B?RWdGcEo1N0s1ZlhXa09jczdhUHNDS1J1cjRuZ3R4MnRod0xVQmxIWWNxeGEr?=
 =?utf-8?B?WWMzbUtwMFd4UjhBbkhoN3Zqa056cmNQeWthMm12RHdUd2hqOXZUNEFIS3Jn?=
 =?utf-8?B?ZVVTN2VWR2NKZy9JNVBBU3p2K2I5QkMvWjM2U3lDcGE1SUVYQ0JUNHRhTXM0?=
 =?utf-8?B?UjI2Rklnd2t1czh6TE0wT3NjRC9SY2dDZUx2a3FsbGdnT0dEYUtsaWNDK1Ru?=
 =?utf-8?B?QzRWalp4NWxBcUpqQU1CaHhrSVZQMmR0OStzaG5iKzdpYjZEWWdzZzlXNThS?=
 =?utf-8?B?NWNOaFIvN0hPR3krSzN6NWYySSs3WFVCRVVvMHYwb3h1RG14K25vUmRZSHZ3?=
 =?utf-8?B?Ty9JQ21JNTk0N01NTWVLTWp6V2dSZXhxbldHcEZ1cFFhN2g2YkhHdWxxMG8r?=
 =?utf-8?B?b2pnQXNYVDY0N2JWSXN5cFQrQnA5c0lUZEtsMXJhQWRqK1NOdW8yV0JtUHU3?=
 =?utf-8?B?eW4rUzU3WDZmNXA5ZWFnWHNsQks1ZHRNeWFGWHBzNDlTYytFeGxVRldrNFVu?=
 =?utf-8?B?V0NHekZ0Z01BZ09OSmpWZmRsNnIzMVRycGlsOEhDamljSXJBSXU3TTNqOFZY?=
 =?utf-8?B?YVRwaW5LZ2EvWXVRRzg5aG8rYWI1a1NPUnh5aFRwSFZxZ0ZaUTRRODF3Qlh0?=
 =?utf-8?B?dlJweUptcGFPU1B5V0pvZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <48883AD8706C784E87D39BD652AD1733@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e4611c79-d4f6-4c6e-dc9f-08d981924b22
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 08:39:17.3827
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vAWqRGhbpd135VCvgdk6rCQwZAi/nODxRfNSTAyY7QAYR+IpxaZKk6FHj79XBsZ23SD5U9VRcsS/QzfV1uhDtFggQ1cL6O9VA9Eb11fdynzjFkFPjn7ip9BRPK9XS89R
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5794
X-Proofpoint-GUID: Lh0rfd6SdPtnrdcBugNJqPEzLCuXvbBz
X-Proofpoint-ORIG-GUID: Lh0rfd6SdPtnrdcBugNJqPEzLCuXvbBz
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0
 lowpriorityscore=0 clxscore=1015 impostorscore=0 mlxscore=0 bulkscore=0
 adultscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270058

DQpPbiAyNy4wOS4yMSAxMTowNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+ICsgUGF1bCAocmV0YWlu
aW5nIGZ1bGwgY29udGV4dCBmb3IgdGhpcyByZWFzb24pDQo+DQo+IE9uIDIzLjA5LjIwMjEgMTQ6
NTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRyIEFu
ZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+IHZQ
Q0kgbWF5IG1hcCBhbmQgdW5tYXAgUENJIGRldmljZSBtZW1vcnkgKEJBUnMpIGJlaW5nIHBhc3Nl
ZCB0aHJvdWdoIHdoaWNoDQo+PiBtYXkgdGFrZSBhIGxvdCBvZiB0aW1lLiBGb3IgdGhpcyB0aG9z
ZSBvcGVyYXRpb25zIG1heSBiZSBkZWZlcnJlZCB0byBiZQ0KPj4gcGVyZm9ybWVkIGxhdGVyLCBz
byB0aGF0IHRoZXkgY2FuIGJlIHNhZmVseSBwcmVlbXB0ZWQuDQo+PiBSdW4gdGhlIGNvcnJlc3Bv
bmRpbmcgdlBDSSBjb2RlIHdoaWxlIHN3aXRjaGluZyBhIHZDUFUuDQo+Pg0KPj4gU2lnbmVkLW9m
Zi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVw
YW0uY29tPg0KPj4NCj4+IC0tLQ0KPj4gU2luY2UgdjE6DQo+PiAgIC0gTW92ZWQgdGhlIGNoZWNr
IGZvciBwZW5kaW5nIHZwY2kgd29yayBmcm9tIHRoZSBjb21tb24gSU9SRVEgY29kZQ0KPj4gICAg
IHRvIGh2bV9kb19yZXN1bWUgb24geDg2DQo+IFdoaWxlIHBlcmhhcHMgb2J2aW91cyBmb3IgQXJt
IGZvbGtzLCBJJ2QgbGlrZSB0byBzZWUgdGhlIHJlYXNvbiBmb3IgdGhpcw0KPiBzcGVsbGVkIG91
dCBpbiB0aGUgZGVzY3JpcHRpb24uDQoNCkkgd2lsbCBhZGQ6DQoNCklPUkVRIGlzIG5vdCBlbmFi
bGVkIGZvciBBcm0gYnkgZGVmYXVsdCwgc28gcGVuZGluZyB2UENJIHdvcmsgaGFzDQoNCm5vIGNo
YW5jZSB0byBiZSBleGVjdXRlZCBpZiB0aGUgcHJvY2Vzc2luZyBpcyBsZWZ0IGFzIGlzIGluIHRo
ZSBjb21tb24gSU9SRVENCg0KY29kZSBvbmx5LiBGb3IgdGhhdCByZWFzb24gQXJtIHByb2Nlc3Nl
cyB0aGF0IGluIGFyY2ggc3BlY2lmaWMgY29kZSB3aGljaCByZXN1bHRzDQoNCmluIHRoYXQgdGhl
IHByb2Nlc3NpbmcgaGFwcGVucyB0d2ljZSBmb3IgQXJtIHdoZW4gSU9SRVEgaXMgZW5hYmxlZC4N
Cg0KRm9yIHg4NiwgcHJvY2Vzc2luZyBvZiB2UENJIGluIElPUkVRIGNvZGUgYWxzbyBkb2Vzbid0
IHNlZW0gdG8gYmUgdGhlIHJpZ2h0DQoNCnBsYWNlLCBzbyBtb3ZlIHRoYXQgdG8gaHZtX2RvX3Jl
c3VtZS4NCg0KPg0KPj4gLS0tIGEveGVuL2FyY2gveDg2L2h2bS9odm0uYw0KPj4gKysrIGIveGVu
L2FyY2gveDg2L2h2bS9odm0uYw0KPj4gQEAgLTU0OSw2ICs1NDksMTIgQEAgdm9pZCBodm1fZG9f
cmVzdW1lKHN0cnVjdCB2Y3B1ICp2KQ0KPj4gICAgICAgaWYgKCAhdmNwdV9pb3JlcV9oYW5kbGVf
Y29tcGxldGlvbih2KSApDQo+PiAgICAgICAgICAgcmV0dXJuOw0KPj4gICANCj4+ICsgICAgaWYg
KCBoYXNfdnBjaSh2LT5kb21haW4pICYmIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHYpICkNCj4+ICsg
ICAgew0KPj4gKyAgICAgICAgcmFpc2Vfc29mdGlycShTQ0hFRFVMRV9TT0ZUSVJRKTsNCj4+ICsg
ICAgICAgIHJldHVybjsNCj4+ICsgICAgfQ0KPiBOb3RlIHRoYXQgeW91J3JlIGFsdGVyaW5nIGJl
aGF2aW9yIGhlcmU6IE9yaWdpbmFsbHkgdGhpcyB3YXMgZG9uZSAuLi4NCj4NCj4+IEBAIC0yMTIs
MTkgKzIxMCwxMiBAQCBzdGF0aWMgYm9vbCB3YWl0X2Zvcl9pbyhzdHJ1Y3QgaW9yZXFfdmNwdSAq
c3YsIGlvcmVxX3QgKnApDQo+PiAgIA0KPj4gICBib29sIHZjcHVfaW9yZXFfaGFuZGxlX2NvbXBs
ZXRpb24oc3RydWN0IHZjcHUgKnYpDQo+PiAgIHsNCj4+IC0gICAgc3RydWN0IGRvbWFpbiAqZCA9
IHYtPmRvbWFpbjsNCj4+ICAgICAgIHN0cnVjdCB2Y3B1X2lvICp2aW8gPSAmdi0+aW87DQo+PiAg
ICAgICBzdHJ1Y3QgaW9yZXFfc2VydmVyICpzOw0KPj4gICAgICAgc3RydWN0IGlvcmVxX3ZjcHUg
KnN2Ow0KPj4gICAgICAgZW51bSB2aW9fY29tcGxldGlvbiBjb21wbGV0aW9uOw0KPj4gICAgICAg
Ym9vbCByZXMgPSB0cnVlOw0KPj4gICANCj4+IC0gICAgaWYgKCBoYXNfdnBjaShkKSAmJiB2cGNp
X3Byb2Nlc3NfcGVuZGluZyh2KSApDQo+PiAtICAgIHsNCj4+IC0gICAgICAgIHJhaXNlX3NvZnRp
cnEoU0NIRURVTEVfU09GVElSUSk7DQo+PiAtICAgICAgICByZXR1cm4gZmFsc2U7DQo+PiAtICAg
IH0NCj4gLi4uIGZpcnN0IHRoaW5nLiBBbmQgSSB0aGluayBpdCB3YW50cyAocGVyaGFwcyBldmVu
IG5lZWRzKSB0byByZW1haW4NCj4gdGhhdCB3YXk7DQpXaWxsIG1ha2UgaXQgdGhlIGZpcnN0IG9w
ZXJhdGlvbg0KPiAgIG90aGVyd2lzZSB5b3UnbGwgbmVlZCB0byBleHBsYWluIHdoeSBub3QsIGFu
ZCB3aHkgdGhlIGNoYW5nZQ0KPiBpcyBjb3JyZWN0IC8gc2FmZS4NCj4NCj4gSmFuDQo+DQpUaGFu
ayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:45:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196532.349417 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmGd-000307-1Q; Mon, 27 Sep 2021 08:45:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196532.349417; Mon, 27 Sep 2021 08:45:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmGc-000300-Tk; Mon, 27 Sep 2021 08:45:42 +0000
Received: by outflank-mailman (input) for mailman id 196532;
 Mon, 27 Sep 2021 08:45:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUmGb-0002zu-KP
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:45:41 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3f79fda3-5b70-4732-b871-e781ed1b43f0;
 Mon, 27 Sep 2021 08:45:40 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R8dGBY028451; 
 Mon, 27 Sep 2021 08:45:38 GMT
Received: from eur01-db5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2053.outbound.protection.outlook.com [104.47.2.53])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbamm00w4-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 08:45:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7345.eurprd03.prod.outlook.com (2603:10a6:20b:273::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 08:45:36 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:45:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f79fda3-5b70-4732-b871-e781ed1b43f0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QVhr9Xj/lvjEnR49097+K07D8vDTz0C2UUVQZkINwgfevutkYViotzKKqgmNT7KCrW6YsnSeAfHdFi7GSvmC0hU2Rfk+N/1KAIwm0WznSqQH6nf8jkI9OqVF+eu3SfbEAL5tZ9WYZD7v78rTVBKEVXnX+jv0xvOH7/5PHxGpxIBFzEI3IwcVBvgXtyZpkIjuD1oEMeCAec6B9ClItdrKxLpflmptKJWY3nIrO0T00PpuqzM0o4wX10JacGtSAI4if66z0EW879Nzug4HTwzfpYMl9EpBhlBY4ScbWZeCxtkxnMThi+gm2h/z0LrFAoHSp0c94vxzYV0E/J3Ek/MdyQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hGETNBeow91mbr0JYZi6wOx7KspMQktzcVUBI+Y7uN8=;
 b=kt/yFDcajnTyOGcIBgk+FgGriBei+LDDH3dq8Kpnq4/3dDPZrNfQd/gfYzTgNP+NYXIzr0bWiqQXlnwj9XaGPDUu4ZeqXu+vo0mtXsyPoglQA7KEvoAki+ywd9GNIHQYxAmQYFHKT4zJpYO+uQvSWWeG3ft1oPeF+wJGRI0e1Ty8TTjOqljKrND3h4bc2C8cLk8Azwpk3ppqduA62P+lFsiw9cVtVas9m9LrPXQuHvD3L/4MUkrPlw9L2sR6He2xiKgzZc6ffHSO0aAWWnJyN7NE8uBtv66LrSVR/GdfXFyDhBHKKXFL9+nASCjgjU4FE8PQzsn4NZwe4Z+maIdexw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hGETNBeow91mbr0JYZi6wOx7KspMQktzcVUBI+Y7uN8=;
 b=rR1IbgBnQrGJvt1lGN04tYn13XUDN+87zjnNKPDYCzqU6MxwbnYnlz6eGkkhFH5cJCshMyAi8CJcqvkEO+wdFKKJNHkexBTwYpt9+dcGIjAvvXEvhzakqU6I4fCIc7sh//DCRr3r5yBqS6jOx944+d2Ljch/CzEJB3G+tcCNwvh77r4Mq3yFDT1fHaPcNoUuQgst6LhJHKwdRUCs+oq9H7pQWSxkiT5i31yQ63/t6ZCe5yRMGLQP67JwRyzglchgQbsTPQuUGbTzIzeaBnn1pFSZe3rj0cSrzKvswaFU9dPFfWNyyYZQOaH2UBaQGRHTuu2B42sx8HAgLjWtYP+e1g==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrIA=
Date: Mon, 27 Sep 2021 08:45:36 +0000
Message-ID: <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
In-Reply-To: <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 49599abe-8ac5-49db-d97f-08d981932cf3
x-ms-traffictypediagnostic: AM9PR03MB7345:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB7345DD057F56F0D36366B760E7A79@AM9PR03MB7345.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 0j28hTdQRCnJvewDrhgPexQbrdL8zWfvTw4OtVMoY4MK8MRRYGzAqoZRiyXKHM9hQGVEI05Rma8+UnflgMh3vIjxRg+U/cwahcOXCdS2EUoSApwFwLvRan6X1WtllI6+9CjT4f9+759mZHpHa4WM98Ju65n+f3ATlQBXMznQkjC8FX7yf9gs9V2Te3Qxd3NaXf2pZWMB0dvvoZV7kjtJYP7pgD0GaluYOgrPC7eDFh/uYbwoNF2xkPOCpdG7yc5WEbTN6L7+CLHfCqwjVgeX/VJGCFU3lvjM85riZAzogPWIGE6vXojp7A97fwBuhGALpD71fE7foFJcJHViLxKG3SI1aG74W6jy82dZr2XpyAy8hrs0guIgKl4ZgLUVB8q/jJ5NgDYqsGsrp6Q4iSW7u/L2LMYcKPziM7jJmBfodNrvhLQNUTJhf7IhdWyjVXwAK6XMwoINQDg2GpYcLzrAwIdHEo4PLcpaO8V+fR4f+3kZOCuKVaKQsOU2ghn1EyPYzN0HBBTg8VVKgj6AhbHTB/PH6GSYrLMo30nN+pa5/I4OzABMmjNHc4CDEmYmwARB9w2z9ERq5mXqPg/b1DMq42hAMu3SA4cSFftaCHB1z1gb0Hhq1SxsJcurf1IUzKyi2x7N+WrCLH4ycgdNd1hediKb8zRJcQaQUd31h6nT3NkVdQ/RBCxCH0BkXhi06oWCmAcHbEQJqnGs21h7iuuykTcBAVkxnm6PN1GiiExQFGbmcOOvj6FcTU+GlqbYduFi
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2616005)(4744005)(38070700005)(26005)(66946007)(66446008)(64756008)(55236004)(53546011)(6506007)(186003)(6486002)(31696002)(31686004)(66556008)(66476007)(2906002)(6916009)(122000001)(86362001)(91956017)(36756003)(54906003)(38100700002)(8676002)(316002)(4326008)(71200400001)(6512007)(76116006)(508600001)(8936002)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?cmNtM1I4cENHVHZtSjczMXMwQTd4cnFWYjJHVElRcjNxc0M0eFZuRTZVbVN6?=
 =?utf-8?B?TWpFV1RwUklNekxtVFlCeFp0UnNOQUJ6S2c5dGQyQmFGdS9KN1VPU3JZNnIx?=
 =?utf-8?B?eWt2ckQwVmxVYVIvb24wSnZscjNtQWRsOGM0MFF0RExJM3BoK2FZRUhXVEpy?=
 =?utf-8?B?d2Y2bktscnUvUDhVUXZROUNrOElYcjVwTUJKRmVlZXN0eHMxaFcwbkFMeW9i?=
 =?utf-8?B?ZWhEeDJYK0JKYldtRDRPeUhMb0FTTWVpVWtOSFRVNXhRa3NSeU95ZWFudlpG?=
 =?utf-8?B?MTVOU2Z0UzRtNHZ0VWY5R3JyVzBDTVZzY0t4aCtGbWhNcjNOUVlKM0FXak1O?=
 =?utf-8?B?UDFmS2RaQlYzR1NrQmZUdHhXQW5HNW5DYkttMzFTZEIrQUtyM1BTcFZ4VVhT?=
 =?utf-8?B?S3E1eDVJQlI5alU2ekxKVTVpekFOWTFIM0x3T2k3T2tkc0VXT1l1MnJQVG5z?=
 =?utf-8?B?dHJmcjRnRHFORkczbkFobm1VUDhSUXZIcHZhKzkrWFI3UStkSWlMSXZCemhM?=
 =?utf-8?B?SjNySlNyWTR4Sm4zYkJxYzBLVGtGMkFPak43Tjc0Vlo2dzVGbkFMNHIvTlpz?=
 =?utf-8?B?REdKRnpXVGhxVzJyWFVMM2tycmZ4Ui82WlkwRytlanQ1KzdzZTU2NlM2NE5S?=
 =?utf-8?B?Yk9xMFdvdXdiYWRPZHk2ZnprSEozS0I4ZzZPRGpNbExFcEJ3SmpJVzlFYmhy?=
 =?utf-8?B?VGhMNFB6clZTdjRERXdTYWF2V2ZNWjhoSU9CU2VQKzE5eE8rVndSSzIydHIr?=
 =?utf-8?B?SHdLeDRBK21FVHYrSGxBanF4MEtCUjdzTFF4ZFdvUlA1eHFYeTVkaUZKY2Qz?=
 =?utf-8?B?MXk4MW82VDl4RzVudmZJZmhpTG51UURlcitpeUhJV1pMeHFMakFBR2w5WFJZ?=
 =?utf-8?B?aXlONFplVUlkb2FVMElxRWl3MmlJT29wTjVQdTE4alNBakt1MUdBSkpXVnFT?=
 =?utf-8?B?a2hQVExjSm41SEozc3N2ZU5BU3NuSldBQm51QjFPY2VKcURiNXFIOXRUMGpl?=
 =?utf-8?B?SmxrblRTUlVjNExqRFE3c3UzNHl3WlBjUEpaYUZJSjhyQnI0RkVPNWZHb0tm?=
 =?utf-8?B?ZDdpTUF5K1IrL2N0T3FYbm84TkFWWGpjMXJ0cXBuWlJraE03MTBFSTBlcUN0?=
 =?utf-8?B?azl3RVhVK2srWTM0Y0NlSVg2SDBZNVBkQUcyS1YxLzA5U1dNNEJsSEFvc0kw?=
 =?utf-8?B?TVNYV08rRXQvbTllN05wOE5CbnRTT3RoRkphU0ZiU2NQcW9kMHVneUdGTWVF?=
 =?utf-8?B?OHYzaFVnR3RzY25VbE1jblhFL0EzZ0pQencvMGJLWE9nQ2ZJalJlZTFyc25n?=
 =?utf-8?B?VjN3dHBYRGJrOHN5cGlHYlhaTXhVY1F0dzZhSTIrM3M3RjNKU2JRU1Y3VTIy?=
 =?utf-8?B?aXJOdWRyZStTZmVrR200eUxSNmgweWw4bVFpZmlidmYxb045VlI2clVMU0FZ?=
 =?utf-8?B?a3V4MStsT2VOOTFZNCtDSlFZRjlrcXRGOVJlUGpNOXZObUVXZ1FhMGtGb09L?=
 =?utf-8?B?QWM1dDlCWHRmRXBtR2hyM2dvblFGRi93MWN0dVM5cExUYWlQbUwwSFRFV2pE?=
 =?utf-8?B?ZUtiN2lCbXpXVHNFMTNpY0svRmQzc3BXYm96K2hMVk90eUV0b2ExSkx3RTlP?=
 =?utf-8?B?djQ5NjdoL2xqTnd6Z2NpTDlPTGhRUFhxU0pLUU9XZzRNbnlkNEZyMDh0WXls?=
 =?utf-8?B?R2ZrMzR6M0t4cWZUdW5IODJoYWRuWUVOZGlWR0xuTDd4b2tualRadTBGb0ZX?=
 =?utf-8?B?NnJGejFJY3dCVVFObGJISHNVZEZlOW1vUCt1cW9Vb2dHbnowVFh3L3N1akdo?=
 =?utf-8?B?SlAxaTBvd2pQdlVEaWM5dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A7647998DE5B1644B0F9C805D2F7C145@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 49599abe-8ac5-49db-d97f-08d981932cf3
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 08:45:36.2257
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: TH+qC/8be+hXblMV+JKZDfADOorUV3O6yKWAewU2khSQ2aC88lDHB0fAiRtzTw1JXL5pfYhxGb/MhOJth9Tme46o6KjzzClPSEMzRASRhzf4CsuWgrz3ohBkHf3rEhK8
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7345
X-Proofpoint-GUID: BjDK9cGi4wZjwZPald8aqXbrQEAw1tzP
X-Proofpoint-ORIG-GUID: BjDK9cGi4wZjwZPald8aqXbrQEAw1tzP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=963 adultscore=0
 spamscore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0
 malwarescore=0 phishscore=0 bulkscore=0 impostorscore=0 mlxscore=0
 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270059

DQpPbiAyNy4wOS4yMSAxMDo0NSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDIzLjA5LjIwMjEg
MTQ6NTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gLS0tIGEveGVuL2RyaXZl
cnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3Bj
aS5jDQo+PiBAQCAtMzI4LDYgKzMyOCw5IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2RldiAqYWxsb2Nf
cGRldihzdHJ1Y3QgcGNpX3NlZyAqcHNlZywgdTggYnVzLCB1OCBkZXZmbikNCj4+ICAgICAgICoo
KHU4KikgJnBkZXYtPmJ1cykgPSBidXM7DQo+PiAgICAgICAqKCh1OCopICZwZGV2LT5kZXZmbikg
PSBkZXZmbjsNCj4+ICAgICAgIHBkZXYtPmRvbWFpbiA9IE5VTEw7DQo+PiArI2lmZGVmIENPTkZJ
R19BUk0NCj4+ICsgICAgcGNpX3RvX2RldihwZGV2KS0+dHlwZSA9IERFVl9QQ0k7DQo+PiArI2Vu
ZGlmDQo+IEkgaGF2ZSB0byBhZG1pdCB0aGF0IEknbSBub3QgaGFwcHkgYWJvdXQgbmV3IENPTkZJ
R188YXJjaD4gY29uZGl0aW9uYWxzDQo+IGhlcmUuIEknZCBwcmVmZXIgdG8gc2VlIHRoaXMgZG9u
ZSBieSBhIG5ldyBhcmNoIGhlbHBlciwgdW5sZXNzIHRoZXJlIGFyZQ0KPiBvYnN0YWNsZXMgSSdt
IG92ZXJsb29raW5nLg0KDQpEbyB5b3UgbWVhbiBzb21ldGhpbmcgbGlrZSBhcmNoX3BjaV9hbGxv
Y19wZGV2KGRldik/DQoNCklmIHNvLCB3aGVyZSBzaG91bGQgd2UgcHV0IHRoaXMgY2FsbD8gQXQg
dGhlIHZlcnkgYmVnaW5uaW5nIG9mIGFsbG9jX3BkZXYNCg0Kb3IgYXQgdGhlIGJvdHRvbSBqdXN0
IGJlZm9yZSByZXR1cm5pbmcgZnJvbSBhbGxvY19wZGV2Pw0KDQo+DQo+IEphbg0KPg0KVGhhbmsg
eW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:45:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:45:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196533.349428 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmGi-0003Hz-BZ; Mon, 27 Sep 2021 08:45:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196533.349428; Mon, 27 Sep 2021 08:45:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmGi-0003Hs-7w; Mon, 27 Sep 2021 08:45:48 +0000
Received: by outflank-mailman (input) for mailman id 196533;
 Mon, 27 Sep 2021 08:45:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Pp17=OR=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mUmGg-0002zu-Gz
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:45:46 +0000
Received: from mail-ed1-x535.google.com (unknown [2a00:1450:4864:20::535])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6fa954c4-3c12-412c-98ab-0dd73b776962;
 Mon, 27 Sep 2021 08:45:41 +0000 (UTC)
Received: by mail-ed1-x535.google.com with SMTP id s17so47433471edd.8
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 01:45:41 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fa954c4-3c12-412c-98ab-0dd73b776962
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=o2SPdkuE1Epdr/lPUP2/nUJBK5SmDI2SytgdxN+W2f8=;
        b=UoxY9yC+uuxiqX1fbmgoezrxxBHJjUVl3gLH4s2kcWvm/4Q+ICyx5nImiGMlJk/97C
         uBtr9yN/3+SA8/zH1j9SzU7k+ST0HzxM2tJFmYBwuY/q39nqfN0qP9rbM3lzTHlmVrX3
         WE9EyK8WQNcvjEuZSoaV1ZUwyNk8dBtTv6Iqv0re/KmAsFzc3w/OgJ4CQsFQlAZCBZTJ
         CAeVqSv+BB6tOpGECAuDPPaAKtxjbbaaAbQSyHLgcQatFpIokBmj9ho2qNakRQaBOynA
         fB9bHY49ANVjoxy8zMbEIBJbbgfazFWph3s4pAjKbRUcky8hd77RldMmRbVt0b11G8OB
         n2BQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=o2SPdkuE1Epdr/lPUP2/nUJBK5SmDI2SytgdxN+W2f8=;
        b=uEdg+fnO/eOWBsysu6iH9oPOHKFdAdlue+nHjfok4zO4v3RkLafiXXvne/YJZcsxKb
         GjeMtLfffSEKweLxqn/wCyOADudyV2Vsyo2xJO/hAi0x2QHyz7FRKx324UH1sp2tXLYj
         so/umAt9G/uQgNVt4BlJE0Hf0GbR2A2NA+xYtiOK4+bxCeNXemxvmCZ+W273glrexmn2
         czJ4/wtVeL0ZPF+ofZSPYWCOyakF9zHrx3HbcZ18h2CD4vxeU08lgR5ORN2n3geFRsjl
         m2UY/vg+Sq8adfHISN1WNjoPuG3sdFeO+cwAjsbRGYWZjEe/iN4/sJFNI58zs4qoqjhA
         Y0tA==
X-Gm-Message-State: AOAM531G+IBxBn1s0C7A34X9i8waNc2SGoJbCXtj0UxTIFzkzNC7NbIv
	kjZKU5h7X7ugnsgK/LBaoL7J+0KcrHx26VeeezU=
X-Google-Smtp-Source: ABdhPJzGXQuhhLI/jvUnszxKSrs+BQvYzIKUpP7DQCKPj1pPOT6sCYyjZeQWawcmxuOrsKsFz0xuF2TWoO6dhT9aqII=
X-Received: by 2002:aa7:df92:: with SMTP id b18mr21165119edy.47.1632732340888;
 Mon, 27 Sep 2021 01:45:40 -0700 (PDT)
MIME-Version: 1.0
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-37-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
 <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com> <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
 <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com>
In-Reply-To: <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Mon, 27 Sep 2021 12:45:30 +0400
Message-ID: <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com>
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable NUMA
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>, 
	xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>
Content-Type: multipart/alternative; boundary="00000000000031261505ccf61f83"

--00000000000031261505ccf61f83
Content-Type: text/plain; charset="UTF-8"

On Mon, 27 Sep 2021, 10:33 Jan Beulich, <jbeulich@suse.com> wrote:

> On 24.09.2021 21:39, Stefano Stabellini wrote:
> > On Fri, 24 Sep 2021, Wei Chen wrote:
> >> On 2021/9/24 11:31, Stefano Stabellini wrote:
> >>> On Thu, 23 Sep 2021, Wei Chen wrote:
> >>>> --- a/xen/arch/arm/Kconfig
> >>>> +++ b/xen/arch/arm/Kconfig
> >>>> @@ -34,6 +34,17 @@ config ACPI
> >>>>      Advanced Configuration and Power Interface (ACPI) support for
> Xen is
> >>>>      an alternative to device tree on ARM64.
> >>>>   + config DEVICE_TREE_NUMA
> >>>> +  def_bool n
> >>>> +  select NUMA
> >>>> +
> >>>> +config ARM_NUMA
> >>>> +  bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)"
> if
> >>>> UNSUPPORTED
> >>>> +  select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> >>>
> >>> Should it be: depends on HAS_DEVICE_TREE ?
> >>> (And eventually depends on HAS_DEVICE_TREE || ACPI)
> >>>
> >>
> >> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
> >> option can be selected by users. And depends on has_device_tree
> >> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
> >>
> >> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
> >> does it become a loop dependency?
> >>
> >>
> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html
> >
> > OK, I am fine with that. I was just trying to catch the case where a
> > user selects "ARM_NUMA" but actually neither ACPI nor HAS_DEVICE_TREE
> > are selected so nothing happens. I was trying to make it clear that
> > ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACPI
> > because otherwise it is not going to work.
> >
> > That said, I don't think this is important because HAS_DEVICE_TREE
> > cannot be unselected. So if we cannot find a way to express the
> > dependency, I think it is fine to keep the patch as is.
>
> So how about doing things the other way around: ARM_NUMA has no prompt
> and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt instead
> (and, for Arm at least, ACPI_NUMA as well; this might even be worthwhile
> to have on x86 down the road).
>

As I wrote before, I don't think the user should say "I want to enable NUMA
with Device-Tree or ACPI". Instead, they say whether they want to use NUMA
and let Xen decide to enable the DT/ACPI support.

In other word, the prompt should stay on ARM_NUMA.

Cheers,


> Jan
>
>

--00000000000031261505ccf61f83
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Mon, 27 Sep 2021, 10:33 Jan Beulich, &lt;<a href=3D=
"mailto:jbeulich@suse.com">jbeulich@suse.com</a>&gt; wrote:<br></div><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex">On 24.09.2021 21:39, Stefano Stabellini wrote:<br>
&gt; On Fri, 24 Sep 2021, Wei Chen wrote:<br>
&gt;&gt; On 2021/9/24 11:31, Stefano Stabellini wrote:<br>
&gt;&gt;&gt; On Thu, 23 Sep 2021, Wei Chen wrote:<br>
&gt;&gt;&gt;&gt; --- a/xen/arch/arm/Kconfig<br>
&gt;&gt;&gt;&gt; +++ b/xen/arch/arm/Kconfig<br>
&gt;&gt;&gt;&gt; @@ -34,6 +34,17 @@ config ACPI<br>
&gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 Advanced Configuration and Power Inter=
face (ACPI) support for Xen is<br>
&gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 an alternative to device tree on ARM64=
.<br>
&gt;&gt;&gt;&gt;=C2=A0 =C2=A0+ config DEVICE_TREE_NUMA<br>
&gt;&gt;&gt;&gt; +=C2=A0 def_bool n<br>
&gt;&gt;&gt;&gt; +=C2=A0 select NUMA<br>
&gt;&gt;&gt;&gt; +<br>
&gt;&gt;&gt;&gt; +config ARM_NUMA<br>
&gt;&gt;&gt;&gt; +=C2=A0 bool &quot;Arm NUMA (Non-Uniform Memory Access) Su=
pport (UNSUPPORTED)&quot; if<br>
&gt;&gt;&gt;&gt; UNSUPPORTED<br>
&gt;&gt;&gt;&gt; +=C2=A0 select DEVICE_TREE_NUMA if HAS_DEVICE_TREE<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; Should it be: depends on HAS_DEVICE_TREE ?<br>
&gt;&gt;&gt; (And eventually depends on HAS_DEVICE_TREE || ACPI)<br>
&gt;&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; As the discussion in RFC [1]. We want to make ARM_NUMA as a generi=
c<br>
&gt;&gt; option can be selected by users. And depends on has_device_tree<br=
>
&gt;&gt; or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.<br>
&gt;&gt;<br>
&gt;&gt; If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,<br=
>
&gt;&gt; does it become a loop dependency?<br>
&gt;&gt;<br>
&gt;&gt; <a href=3D"https://lists.xenproject.org/archives/html/xen-devel/20=
21-08/msg00888.html" rel=3D"noreferrer noreferrer" target=3D"_blank">https:=
//lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html</a><br=
>
&gt; <br>
&gt; OK, I am fine with that. I was just trying to catch the case where a<b=
r>
&gt; user selects &quot;ARM_NUMA&quot; but actually neither ACPI nor HAS_DE=
VICE_TREE<br>
&gt; are selected so nothing happens. I was trying to make it clear that<br=
>
&gt; ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACP=
I<br>
&gt; because otherwise it is not going to work.<br>
&gt; <br>
&gt; That said, I don&#39;t think this is important because HAS_DEVICE_TREE=
<br>
&gt; cannot be unselected. So if we cannot find a way to express the<br>
&gt; dependency, I think it is fine to keep the patch as is.<br>
<br>
So how about doing things the other way around: ARM_NUMA has no prompt<br>
and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt instead<br=
>
(and, for Arm at least, ACPI_NUMA as well; this might even be worthwhile<br=
>
to have on x86 down the road).<br></blockquote></div></div><div dir=3D"auto=
"><br></div><div dir=3D"auto">As I wrote before, I don&#39;t think the user=
 should say &quot;I want to enable NUMA with Device-Tree or ACPI&quot;. Ins=
tead, they say whether they want to use NUMA and let Xen decide to enable t=
he DT/ACPI support.</div><div dir=3D"auto"><br></div><div dir=3D"auto">In o=
ther word, the prompt should stay on ARM_NUMA.</div><div dir=3D"auto"><br><=
/div><div dir=3D"auto">Cheers,</div><div dir=3D"auto"><br></div><div dir=3D=
"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Jan<br>
<br>
</blockquote></div></div></div>

--00000000000031261505ccf61f83--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 08:47:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 08:47:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196549.349439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmIi-0004Vf-R5; Mon, 27 Sep 2021 08:47:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196549.349439; Mon, 27 Sep 2021 08:47:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmIi-0004VY-NV; Mon, 27 Sep 2021 08:47:52 +0000
Received: by outflank-mailman (input) for mailman id 196549;
 Mon, 27 Sep 2021 08:47:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUmIh-0004VS-Ce
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 08:47:51 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89f6f099-05d8-4638-a6ee-8d9d986ad72a;
 Mon, 27 Sep 2021 08:47:50 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R70bNX026217; 
 Mon, 27 Sep 2021 08:47:47 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bb96mgehh-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 08:47:47 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 08:47:45 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 08:47:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89f6f099-05d8-4638-a6ee-8d9d986ad72a
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bn1uJpqLqNhQSmpZywtc/kcWKi/C0ES5AZw85Vj55rltzEICCbW70vh73dGauFTvVrUfLLopOlekmHbUpxEFUZTt/M3+S5WY5iM9CmtOdrS1e2Z/whvLZydTcXlkhHvWqtBYZBvxJAVwrn+Ijk0T0LbTL/p26JdtHj5WdCl8IkBo+4vACQ0oUaDOGIPTMCUIFrepSrfRV+C0eefU/cjefIpQcUzsUoup8cnfF7jy+l4vQrw2jtl+lUUDkLhEYU5/rp9spYbSe/9/lBpc8LKdjWLJuu3X6/Ke8oB9bEZolA03lxV2y4fiUoMzX5ZNkB3IlYthwb/6hRwjmYtZLVirEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zoFPAhxzEd2nryFaPLGz3Dx6D3fkNJesHTJ3hanOgN0=;
 b=Szt6jZING72GHUjCMexYQY4HIR6JIPc5NDy1FSIvjZU+51sGpwqNJx+h2K46vkyQ87478JxnXOj10oib05CU4yJFhqiFgvUapyBF2keqs/gRcu/veMf8ggLa4pGYXOdbMWw2QG6vAVW2o5gRheH5UiUD3kjPJO54R/1zPYK58xN/2Fc30VdoR2m5brAZ6jBbBFwbZcqZ3xQTnhfy4PF9YmDSyyJOA9v8tpgKWWy+Ps2mF4KjfWDV47bg/NJ3hP1V7bhrhorZLOTq58QJIe9DRPc77Uvj8w93e0jR+oiCa9isqm/HwEUC9P43BNLuKyE7j5k29YlT2XFguiqRU2pGsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zoFPAhxzEd2nryFaPLGz3Dx6D3fkNJesHTJ3hanOgN0=;
 b=lBf6U5tA5zmeYzHN6KFOjdbxuzbiV1x71f31YsIXf562gcVk5Uy4r4lYfqtDqtD/Od66yAlJYuYOHuvHp2bk4iKobVlSzPJV95UcgS0DXErPCCm5PPVH/FfvQp7sghKYakSB8pp7faKYaOj3KDFuWKwz2TiO8mLpbggSb4PaOXVTlGJHJNpFdaRJJRi/DR0IVmTu/ZAvVfdImtLgIXWT/8ZnlTo6piKTh37CE9fu+Q2iFGR72HJn1yxSG+XMZi4/P3WAtVUcZfCr9cF9w90p8LsGGW4vJRZ8C+8JQoctXJ8AJnl9bj4VKIEJgvankmVBV4/0l0ZTGU+dcvdvuOFBnw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Topic: [PATCH v2 09/11] xen/arm: Setup MMIO range trap handlers for
 hardware domain
Thread-Index: AQHXsHo395a0MNZ4jUyUZjWX7U0qP6uz5McAgAOy8QA=
Date: Mon, 27 Sep 2021 08:47:44 +0000
Message-ID: <cf5e14a9-7bf0-6bf4-cf6e-3d85e0cdc6bd@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-10-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241713160.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241713160.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 432c899c-1a2a-4650-07be-08d9819379af
x-ms-traffictypediagnostic: AM0PR03MB6324:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB63240AF5C2A1CAA89B0D7F99E7A79@AM0PR03MB6324.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Oz37YsfRqrNDKyYUbbaBe2fZHoHQZQG7ALwRuqGD/QQ6Ygj2Y1zR9yTvOem4Hwd1amQkPoVmZ7GDhIokJpnRKRW0taLDIZT33s0LfucRivPwZ7ZHmpluVCJXuehCYgDdXAuQssTa0fip/wFXBddXv+QjTCYu3JW7lwlk3I+jpVBmrVWRh1wJilF7Qn1AIEbT8XlnKusooW8cMk+zVs6ky+kqD4OhZ7/6jg+r/ZVo0bp0Mdkn//cgqmUbXpcTxw6CJf+ljTjYnmNlYZqUJ3QlrgLsrEVaDsJv9QVUQ7grvO2DAqvOyhK4t7I86ZuUHx7dyGf2AhwDg6mrejwmIcn95CZvv4cpdgXLpJWOZeobZTS2b3fUnd36IeNLgw+NDyaZ3tFPtzToyYQhcsoUU5T2vF5k6ANCXzzfu1FtcgTgZ8rwrnPSDTa6zPto9vHVGBydHxwxcXFqLQ7xJtmuLzgbplauj2QenwVH1Wea8MtI0nzo22D/N8QpyKC5Yu84tib3oL3hrGruLJFwrHqYLAlupy1Yki4oW5I8rwk4Losws6KASUA6BCFfZ9Er6YACZgf+JzCsNO0Eb4gQAFsnDg6VyV9YbLqmWwDD804pEs8YaOE+3zm6kOu55RE43X8RCtTnIti5qLhYBH3KofFK5zFc7tbbmzOvLIUCo4ObqYX3aEwIIAI1CXiquxsku1u5wOnr8ErciLhIiPkb21/MLZCO6g0ic6JYnmdOWx3O1fKrYqrOopvw8zaSC9lyUJgddPBwioKmJt4v5xipz/VCt18TGQ==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6512007)(5660300002)(31686004)(66476007)(71200400001)(38070700005)(4326008)(66946007)(55236004)(86362001)(26005)(64756008)(186003)(36756003)(2906002)(8936002)(31696002)(66446008)(107886003)(508600001)(66556008)(6506007)(53546011)(8676002)(122000001)(54906003)(2616005)(316002)(83380400001)(6916009)(91956017)(38100700002)(6486002)(76116006)(45980500001)(309714004);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eldGYjVtWERVZGJpLytrQWtjOXlFZURLanRIL0kyaWI3ZVlzYU5ZQWtKUzM3?=
 =?utf-8?B?YlhpamNmSTRKam16MmlZeTR0ZElBWlZqOGVhNEx1RHRzMTlyR0F2bEdjNDVt?=
 =?utf-8?B?T1VXaXZBRlR1ZDZTTzlEZHhsMHRIaFJlRG44UDNYcUlkSDhEUFBqZnVEODU3?=
 =?utf-8?B?OEhaSzQ4dlYzMis3NDR3UlNEUXBFUmdxZ2dDeFNDVUdOMTUwN2JSdEJHZjU1?=
 =?utf-8?B?alFma3l1TnRROEFGSSsrNHVlNzVINzJOTEg2U1BwNWFTWkhTSXQ0SmlFbEtJ?=
 =?utf-8?B?YnlYem1EK3ZuZCt3aUJWVjJVUHQ0NHhTcVhMa3hzOUlReGx1YVNhSlNuMC95?=
 =?utf-8?B?S2JaL29rZkx0elpEWDVGRzhlUDR4QWdLRVRtMXd2STVhUktVZmtqV0tvUEdL?=
 =?utf-8?B?OUJWM3BSUTlna1JKaVUyeXB5aWJOZ1IxM3pNQjFRNDlMc05LWWVrNkRYVlhm?=
 =?utf-8?B?S2xBK0R5TTF1TzhCTlFDSXJSeThnVWJsM2VzK1lRZC95NlNmSnA2Ympkc1Jt?=
 =?utf-8?B?Tm5qK1ZUd2x2dTh2aFBqYkw5MnRjSXJkWXZEMXdqVDlBeGpndFNtcVdYaGRi?=
 =?utf-8?B?NmxoWURtZ1VLK1M1TExVZzVmVEZvazdub0tMWGNEOG92NGt4bnZDYmd4RThs?=
 =?utf-8?B?aUQ2eVdXdGFkNFhIS1hCSk4rVWNzUWZjQ2VyQkp0YXRXaWhOSldpWEhQSUYx?=
 =?utf-8?B?cXQvbXQyWDkzOEdTRUJYR0pTbUpkU0FXYnlyOGpRODJETmxVOTNRUFhGS0VH?=
 =?utf-8?B?U3VKOXpWeTZ4YlU1c25rTmhvanZVWlFJbzNOclVxWkpvdzB3SU1TYk80dHlQ?=
 =?utf-8?B?OEtKVGxCUVZDa2RKMHRDNTlhajh6L1FqcnVHTmI0MHNzanQvVmJwanN3WEhJ?=
 =?utf-8?B?T0Z3OUc4dWdhWTRLcUg1ZjFZbXhjYmtDZ0gxNnNmaTRJblgyTzQ1dC8vendk?=
 =?utf-8?B?cVJ6Rkp4R0U2QTJNRE1McXBIbnBjcVlhaytoekhmUU1ZV01LQWg4SCtlTWxN?=
 =?utf-8?B?WFVlRFhtTzAwdk1jWElRSXhNS2gvRHVBem1uK0w0TXgyclY5eWYyK2hPOE56?=
 =?utf-8?B?N1EzQVBISC9ITDZoRnd6NWp0M1lwNXJieW9FTjZnOGZ4aW0xSFNhUkRtQWds?=
 =?utf-8?B?TmNSTk5FeGFhcTZ1Wk5Ebnh1cGEwZThqQm5HK3ZBSXBRcU02aWwzRDZEQ3FM?=
 =?utf-8?B?V3JGZFZ5cDBVei9DTUVZTFVYcUhLd1ArYXhNaGtPWXdXWCsxVVFqdTdlTEts?=
 =?utf-8?B?bkpGb1g3V1c3d3BoOFNIL0NpM1BSbXpMNG13L1gvV2RiekE0eUFzYk8xNHdT?=
 =?utf-8?B?dGZTM0pFOUJTcS9VT1BMS3R3MHhQb3BCM2NFN0xNYzU5YysvQnBJV0xUUFFG?=
 =?utf-8?B?eG8xbjl3TzRVSHVOS1QwZy9CT0tmSVRXc2w4TzFsblBZeWIyQXJUdlpGOUo4?=
 =?utf-8?B?RGpBTXpHS1lZbjN1YjUwcStvWFB0NXcrYUsyYlpacFhpRGk0Q25hRkh0aGJJ?=
 =?utf-8?B?V0RTelRmbm1DMjR4RW9nUVF1Nk9MZDdLUzRFWW5iK1puUUxQOXlvbldmcTM4?=
 =?utf-8?B?b0JjbTEzRTRNM1VGMmxaMVRqRTdDUEhncmVjREJXQkMyTDlneW93Sno1TnVV?=
 =?utf-8?B?eDZ0Yk9MYjF1NE9iZ3FzRldlQlA2enhkWGFZVHorZGd4REhxaEZjQ3Z5dWZR?=
 =?utf-8?B?OVNtNm02VC9kUWRiOGxpQjNaMDZuNVdhbjlRNnNaN21KSVI1WDc5bGk1WmVJ?=
 =?utf-8?B?UThlREc5OStaVTFlcHlCTE1Ba0U5TS9tTDE1WG16U20yL0MrMmo2azFDR0tt?=
 =?utf-8?B?VGlMUmJ1SFlteTYwcCs5dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <6849F60890C78D47A4DA762E4D6AF751@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 432c899c-1a2a-4650-07be-08d9819379af
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 08:47:45.0011
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Zl5TqUegq19Ym3NJkZ5jcy5lLkuUr++GsARzTmkPAzfOIrSdteYuvyId0AvddEsVz5VcHYUyD7zOJsYh2fvAhbg0t3UV/ZiXKYMFCfOnTeXT87R13V7PByChA2qrL+8b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6324
X-Proofpoint-GUID: b7dJLZLjP70zOVmVeLq1G_jKWQ2dKMIz
X-Proofpoint-ORIG-GUID: b7dJLZLjP70zOVmVeLq1G_jKWQ2dKMIz
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_02,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0
 phishscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0
 priorityscore=1501 bulkscore=0 clxscore=1015 mlxlogscore=999
 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270059

DQpPbiAyNS4wOS4yMSAwMzoxOCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBJbiBvcmRlciBmb3IgdlBDSSB0byB3b3JrIGl0IG5lZWRzIHRvIG1haW50YWluIGd1
ZXN0IGFuZCBoYXJkd2FyZQ0KPj4gZG9tYWluJ3Mgdmlld3Mgb24gdGhlIGNvbmZpZ3VyYXRpb24g
c3BhY2UuIEZvciBleGFtcGxlLCBCQVJzIGFuZA0KPiAgICAgICAgICAgICAgICAgICBeIG9mDQo+
DQpBY2sNCj4+IENPTU1BTkQgcmVnaXN0ZXJzIHJlcXVpcmUgZW11bGF0aW9uIGZvciBndWVzdHMg
YW5kIHRoZSBndWVzdCB2aWV3DQo+PiBvbiB0aGUgcmVnaXN0ZXJzIG5lZWRzIHRvIGJlIGluIHN5
bmMgd2l0aCB0aGUgcmVhbCBjb250ZW50cyBvZiB0aGUNCj4gICAgXiBvZg0KQWNrDQo+DQo+PiBy
ZWxldmFudCByZWdpc3RlcnMuIEZvciB0aGF0IEVDQU0gYWRkcmVzcyBzcGFjZSBuZWVkcyB0byBh
bHNvIGJlDQo+PiB0cmFwcGVkIGZvciB0aGUgaGFyZHdhcmUgZG9tYWluLCBzbyB3ZSBuZWVkIHRv
IGltcGxlbWVudCBQQ0kgaG9zdA0KPj4gYnJpZGdlIHNwZWNpZmljIGNhbGxiYWNrcyB0byBwcm9w
ZXJseSBzZXR1cCBNTUlPIGhhbmRsZXJzIGZvciB0aG9zZQ0KPj4gcmFuZ2VzIGRlcGVuZGluZyBv
biBwYXJ0aWN1bGFyIGhvc3QgYnJpZGdlIGltcGxlbWVudGF0aW9uLg0KPj4NCj4+IFNpZ25lZC1v
ZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0Bl
cGFtLmNvbT4NCj4gVGhlIHBhdGNoIGxvb2tzIHByZXR0eSBnb29kLCBvbmx5IGEgY291cGxlIG9m
IG1pbm9yIGNvbW1lbnRzIGJlbG93DQo+DQo+DQo+PiAtLS0NCj4+IFNpbmNlIHYxOg0KPj4gICAt
IER5bmFtaWNhbGx5IGNhbGN1bGF0ZSB0aGUgbnVtYmVyIG9mIE1NSU8gaGFuZGxlcnMgcmVxdWly
ZWQgZm9yIHZQQ0kNCj4+ICAgICBhbmQgdXBkYXRlIHRoZSB0b3RhbCBudW1iZXIgYWNjb3JkaW5n
bHkNCj4+ICAgLSBzL2NsYi9jYg0KPj4gICAtIERvIG5vdCBpbnRyb2R1Y2UgYSBuZXcgY2FsbGJh
Y2sgZm9yIE1NSU8gaGFuZGxlciBzZXR1cA0KPj4gLS0tDQo+PiAgIHhlbi9hcmNoL2FybS9kb21h
aW4uYyAgICAgICAgICAgICAgfCAgMiArKw0KPj4gICB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0
LWNvbW1vbi5jIHwgMjggKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4gICB4ZW4vYXJjaC9h
cm0vdnBjaS5jICAgICAgICAgICAgICAgIHwgMzMgKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrDQo+PiAgIHhlbi9hcmNoL2FybS92cGNpLmggICAgICAgICAgICAgICAgfCAgNiArKysrKysN
Cj4+ICAgeGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCAgICAgICAgICB8ICA3ICsrKysrKysNCj4+
ICAgNSBmaWxlcyBjaGFuZ2VkLCA3NiBpbnNlcnRpb25zKCspDQo+Pg0KPj4gZGlmZiAtLWdpdCBh
L3hlbi9hcmNoL2FybS9kb21haW4uYyBiL3hlbi9hcmNoL2FybS9kb21haW4uYw0KPj4gaW5kZXgg
ODU0ZThmZWQwMzkzLi5jN2IyNWJjNzA0MzkgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0v
ZG9tYWluLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9kb21haW4uYw0KPj4gQEAgLTczMyw2ICs3
MzMsOCBAQCBpbnQgYXJjaF9kb21haW5fY3JlYXRlKHN0cnVjdCBkb21haW4gKmQsDQo+PiAgICAg
ICBpZiAoIChyYyA9IGRvbWFpbl92Z2ljX3JlZ2lzdGVyKGQsICZjb3VudCkpICE9IDAgKQ0KPj4g
ICAgICAgICAgIGdvdG8gZmFpbDsNCj4+ICAgDQo+PiArICAgIGNvdW50ICs9IGRvbWFpbl92cGNp
X2dldF9udW1fbW1pb19oYW5kbGVycyhkKTsNCj4+ICsNCj4+ICAgICAgIGlmICggKHJjID0gZG9t
YWluX2lvX2luaXQoZCwgY291bnQgKyBNQVhfSU9fSEFORExFUikpICE9IDAgKQ0KPj4gICAgICAg
ICAgIGdvdG8gZmFpbDsNCj4+ICAgDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BjaS9w
Y2ktaG9zdC1jb21tb24uYyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMNCj4+
IGluZGV4IDE1NjdiNmUyOTU2Yy4uMTU1ZjJhMjc0M2FmIDEwMDY0NA0KPj4gLS0tIGEveGVuL2Fy
Y2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9w
Y2ktaG9zdC1jb21tb24uYw0KPj4gQEAgLTMwMCw2ICszMDAsMzQgQEAgc3RydWN0IGR0X2Rldmlj
ZV9ub2RlICpwY2lfZmluZF9ob3N0X2JyaWRnZV9ub2RlKHN0cnVjdCBkZXZpY2UgKmRldikNCj4+
ICAgICAgIH0NCj4+ICAgICAgIHJldHVybiBicmlkZ2UtPmR0X25vZGU7DQo+PiAgIH0NCj4+ICsN
Cj4+ICtpbnQgcGNpX2hvc3RfaXRlcmF0ZV9icmlkZ2VzKHN0cnVjdCBkb21haW4gKmQsDQo+PiAr
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnQgKCpjYikoc3RydWN0IGRvbWFpbiAqZCwN
Cj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcGNpX2hv
c3RfYnJpZGdlICpicmlkZ2UpKQ0KPj4gK3sNCj4+ICsgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRn
ZSAqYnJpZGdlOw0KPj4gKyAgICBpbnQgZXJyOw0KPj4gKw0KPj4gKyAgICBsaXN0X2Zvcl9lYWNo
X2VudHJ5KCBicmlkZ2UsICZwY2lfaG9zdF9icmlkZ2VzLCBub2RlICkNCj4+ICsgICAgew0KPj4g
KyAgICAgICAgZXJyID0gY2IoZCwgYnJpZGdlKTsNCj4+ICsgICAgICAgIGlmICggZXJyICkNCj4+
ICsgICAgICAgICAgICByZXR1cm4gZXJyOw0KPj4gKyAgICB9DQo+PiArICAgIHJldHVybiAwOw0K
Pj4gK30NCj4+ICsNCj4+ICtpbnQgcGNpX2hvc3RfZ2V0X251bV9icmlkZ2VzKHZvaWQpDQo+PiAr
ew0KPj4gKyAgICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2U7DQo+PiArICAgIGludCBj
b3VudCA9IDA7DQo+PiArDQo+PiArICAgIGxpc3RfZm9yX2VhY2hfZW50cnkoIGJyaWRnZSwgJnBj
aV9ob3N0X2JyaWRnZXMsIG5vZGUgKQ0KPj4gKyAgICAgICAgY291bnQrKzsNCj4+ICsNCj4+ICsg
ICAgcmV0dXJuIGNvdW50Ow0KPj4gK30NCj4+ICsNCj4+ICAgLyoNCj4+ICAgICogTG9jYWwgdmFy
aWFibGVzOg0KPj4gICAgKiBtb2RlOiBDDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3Zw
Y2kuYyBiL3hlbi9hcmNoL2FybS92cGNpLmMNCj4+IGluZGV4IDc2YzEyYjkyODE0Zi4uMTQ5NDdl
OTc1ZDY5IDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL3ZwY2kuYw0KPj4gKysrIGIveGVu
L2FyY2gvYXJtL3ZwY2kuYw0KPj4gQEAgLTgwLDE3ICs4MCw1MCBAQCBzdGF0aWMgY29uc3Qgc3Ry
dWN0IG1taW9faGFuZGxlcl9vcHMgdnBjaV9tbWlvX2hhbmRsZXIgPSB7DQo+PiAgICAgICAud3Jp
dGUgPSB2cGNpX21taW9fd3JpdGUsDQo+PiAgIH07DQo+PiAgIA0KPj4gK3N0YXRpYyBpbnQgdnBj
aV9zZXR1cF9tbWlvX2hhbmRsZXIoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICsgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSkNCj4+
ICt7DQo+PiArICAgIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnID0gYnJpZGdlLT5jZmc7
DQo+PiArDQo+PiArICAgIHJlZ2lzdGVyX21taW9faGFuZGxlcihkLCAmdnBjaV9tbWlvX2hhbmRs
ZXIsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICBjZmctPnBoeXNfYWRkciwgY2ZnLT5z
aXplLCBOVUxMKTsNCj4+ICsgICAgcmV0dXJuIDA7DQo+PiArfQ0KPj4gKw0KPj4gICBpbnQgZG9t
YWluX3ZwY2lfaW5pdChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gICB7DQo+PiAgICAgICBpZiAoICFo
YXNfdnBjaShkKSApDQo+PiAgICAgICAgICAgcmV0dXJuIDA7DQo+PiAgIA0KPj4gKyAgICBpZiAo
IGlzX2hhcmR3YXJlX2RvbWFpbihkKSApDQo+PiArICAgICAgICByZXR1cm4gcGNpX2hvc3RfaXRl
cmF0ZV9icmlkZ2VzKGQsIHZwY2lfc2V0dXBfbW1pb19oYW5kbGVyKTsNCj4+ICsNCj4+ICsgICAg
LyogR3Vlc3QgZG9tYWlucyB1c2Ugd2hhdCBpcyBwcm9ncmFtbWVkIGluIHRoZWlyIGRldmljZSB0
cmVlLiAqLw0KPj4gICAgICAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsICZ2cGNpX21taW9faGFu
ZGxlciwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBHVUVTVF9WUENJX0VDQU1fQkFT
RSwgR1VFU1RfVlBDSV9FQ0FNX1NJWkUsIE5VTEwpOw0KPj4gICANCj4+ICAgICAgIHJldHVybiAw
Ow0KPj4gICB9DQo+PiAgIA0KPj4gK2ludCBkb21haW5fdnBjaV9nZXRfbnVtX21taW9faGFuZGxl
cnMoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICt7DQo+PiArICAgIGludCBjb3VudCA9IDA7DQo+PiAr
DQo+PiArICAgIGlmICggaXNfaGFyZHdhcmVfZG9tYWluKGQpICkNCj4+ICsgICAgICAgIC8qIEZv
ciBlYWNoIFBDSSBob3N0IGJyaWRnZSdzIGNvbmZpZ3VyYXRpb24gc3BhY2UuICovDQo+PiArICAg
ICAgICBjb3VudCArPSBwY2lfaG9zdF9nZXRfbnVtX2JyaWRnZXMoKTsNCj4gTklUOiB3aHkgKz0g
aW5zdGVhZCBvZiA9ID8NCldpbGwgZml4DQo+DQo+DQo+PiArICAgIGVsc2UNCj4+ICsgICAgICAg
IC8qDQo+PiArICAgICAgICAgKiBWUENJX01TSVhfTUVNX05VTSBoYW5kbGVycyBmb3IgTVNJLVgg
dGFibGVzIHBlciBlYWNoIFBDSSBkZXZpY2UNCj4+ICsgICAgICAgICAqIGJlaW5nIHBhc3NlZCB0
aHJvdWdoLiBNYXhpbXVtIG51bWJlciBvZiBzdXBwb3J0ZWQgZGV2aWNlcw0KPj4gKyAgICAgICAg
ICogaXMgMzIgYXMgdmlydHVhbCBidXMgdG9wb2xvZ3kgZW11bGF0ZXMgdGhlIGRldmljZXMgYXMg
ZW1iZWRkZWQNCj4+ICsgICAgICAgICAqIGVuZHBvaW50cy4NCj4+ICsgICAgICAgICAqICsxIGZv
ciBhIHNpbmdsZSBlbXVsYXRlZCBob3N0IGJyaWRnZSdzIGNvbmZpZ3VyYXRpb24gc3BhY2UuICov
DQo+PiArICAgICAgICBjb3VudCA9IFZQQ0lfTVNJWF9NRU1fTlVNICogMzIgKyAxOw0KPj4gKw0K
Pj4gKyAgICByZXR1cm4gY291bnQ7DQo+PiArfQ0KPj4gKw0KPj4gICAvKg0KPj4gICAgKiBMb2Nh
bCB2YXJpYWJsZXM6DQo+PiAgICAqIG1vZGU6IEMNCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9h
cm0vdnBjaS5oIGIveGVuL2FyY2gvYXJtL3ZwY2kuaA0KPj4gaW5kZXggZDhhN2IwZTNlODAyLi4y
N2EyYjA2OWFiZDIgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdnBjaS5oDQo+PiArKysg
Yi94ZW4vYXJjaC9hcm0vdnBjaS5oDQo+PiBAQCAtMTcsMTEgKzE3LDE3IEBADQo+PiAgIA0KPj4g
ICAjaWZkZWYgQ09ORklHX0hBU19WUENJDQo+PiAgIGludCBkb21haW5fdnBjaV9pbml0KHN0cnVj
dCBkb21haW4gKmQpOw0KPj4gK2ludCBkb21haW5fdnBjaV9nZXRfbnVtX21taW9faGFuZGxlcnMo
c3RydWN0IGRvbWFpbiAqZCk7DQo+PiAgICNlbHNlDQo+PiAgIHN0YXRpYyBpbmxpbmUgaW50IGRv
bWFpbl92cGNpX2luaXQoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICAgew0KPj4gICAgICAgcmV0dXJu
IDA7DQo+PiAgIH0NCj4+ICsNCj4+ICtzdGF0aWMgaW5saW5lIGludCBkb21haW5fdnBjaV9nZXRf
bnVtX21taW9faGFuZGxlcnMoc3RydWN0IGRvbWFpbiAqZCkNCj4+ICt7DQo+PiArICAgIHJldHVy
biAwOw0KPj4gK30NCj4+ICAgI2VuZGlmDQo+PiAgIA0KPj4gICAjZW5kaWYgLyogX19BUkNIX0FS
TV9WUENJX0hfXyAqLw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgg
Yi94ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oDQo+PiBpbmRleCA1YjEwMDU1NjIyNWUuLjc2MThm
MGI2NzI1YiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+ICsr
KyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+IEBAIC0xNSw2ICsxNSw4IEBADQo+PiAg
ICNpZm5kZWYgX19BUk1fUENJX0hfXw0KPj4gICAjZGVmaW5lIF9fQVJNX1BDSV9IX18NCj4+ICAg
DQo+PiArI2luY2x1ZGUgPGFzbS9tbWlvLmg+DQo+IFRoaXMgc2hvdWxkIGJlIG5vIGxvbmdlciBy
ZXF1aXJlZC4NCkknbGwgY2hlY2sgYW5kIHJlbW92ZQ0KPg0KPg0KPj4gICAjaWZkZWYgQ09ORklH
X0hBU19QQ0kNCj4+ICAgDQo+PiAgICNkZWZpbmUgcGNpX3RvX2RldihwY2lkZXYpICgmKHBjaWRl
diktPmFyY2guZGV2KQ0KPj4gQEAgLTEwOSw2ICsxMTEsMTEgQEAgc3RhdGljIGFsd2F5c19pbmxp
bmUgYm9vbCBpc19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCh2b2lkKQ0KPj4gICB7DQo+PiAgICAg
ICByZXR1cm4gISFwY2lfcGFzc3Rocm91Z2hfZW5hYmxlZDsNCj4+ICAgfQ0KPj4gKw0KPj4gK2lu
dCBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICsgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIGludCAoKmNsYikoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICsg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHBjaV9ob3N0X2Jy
aWRnZSAqYnJpZGdlKSk7DQo+PiAraW50IHBjaV9ob3N0X2dldF9udW1fYnJpZGdlcyh2b2lkKTsN
Cj4+ICAgI2Vsc2UgICAvKiFDT05GSUdfSEFTX1BDSSovDQo+PiAgIA0KPj4gICAjZGVmaW5lIHBj
aV9wYXNzdGhyb3VnaF9lbmFibGVkIChmYWxzZSkNCg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHIN
Cg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:04:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:04:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196558.349450 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmYq-00070K-8b; Mon, 27 Sep 2021 09:04:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196558.349450; Mon, 27 Sep 2021 09:04:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmYq-00070D-4j; Mon, 27 Sep 2021 09:04:32 +0000
Received: by outflank-mailman (input) for mailman id 196558;
 Mon, 27 Sep 2021 09:04:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUmYo-000707-Gh
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:04:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id eb74f396-1f71-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 09:04:29 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-WDu41gZKMLqyKG12tlhcDA-1; Mon, 27 Sep 2021 11:04:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7039.eurprd04.prod.outlook.com (2603:10a6:800:12b::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 09:04:25 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:04:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0006.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Mon, 27 Sep 2021 09:04:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb74f396-1f71-11ec-bc43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632733468;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rR5Bi4C/0nThtlZ0ncdcLtoL5SQYZkVOfglVAd2DgKg=;
	b=MH/fHbE+hxI9Bjh+jCobOhaexjJ+T3ZSUPdQ+bb/HROYKr/0FBY2le1N5Ig/w4G6VKWzVT
	SLIZL/FaSVeIwaBLpKupLR0aofPkscUoTzG6j2o82EPeCDauJGiBa7+oARx4xmLpkjmcIF
	PoHMiWJjz1i8h0ZILiCDQegT1Inn8rU=
X-MC-Unique: WDu41gZKMLqyKG12tlhcDA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oWu+ASa4VKwokixrn3Uvnjq+DCTfN/u3GAJbOufsOdhiazjNeZIuL4QpMN2IN8xPL46Ml92AsnfxlLYrl3zX4802on1GDcTzihrlxTn+91JKiNizYL/YhfHz3Gzc6NrDE72pR26VJa/AO/E+FzTorROzS1/eq61kLBrqPqy16K1Q2nykZ5MjKT/myLuFKStiEqP1wxbyif2N4rltRkndz70XnMFz/+KpooHFLXdEk0e0UzP95gbG6o4oeNNXQdOtUGruDcfgrmJTsmB4sm/0bBnLGKqXHPYFIqvIQFebkU+6DHYObCCqv5G9HxTAAZU3bkllBPhz4i51DsHEeIuLyA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XsFCFAVp28fbkgmW0sLX6BCKQZCn6WqvaZwT8rsKZf8=;
 b=SC4igtGScXi284bsTrUOfDeUaZ1BQ7pK53SlxvcA2dqTWKqGRvXFeS8OxoDz2Ku5JT6mGISiLbxOf7k5Y1Fy3Ahp9FB7NGEh3dk4qBD+cNHqMuUQDO9xXPHdejKsH0xCEnGTRhofCnQTklSYcrrE3BLTmii2heKSHR85aUvwVMzUhbIc7SByFZpeSr8yQDyvt0Du941jNBr80JEmeaI9y09dZzhRjuovxM05tx6nrfS/PWmkflV21LulX+AB1CB3lo8m0qrJ/NH6n1HCzQgD+tU9+ivRoXJCEVwZpnLjz1sv3ndTbtVAm1oeeD4DFOFjoIkMlvVBK6N60cySmz8gFQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 02/18] VT-d: have callers specify the target level for
 page table walks
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
 <6d01a401-ac53-d4ca-e0ac-165d045341af@suse.com>
 <YU3kedkf/mSsGcpD@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <91745efe-129b-4851-87a2-93eea8f6d63d@suse.com>
Date: Mon, 27 Sep 2021 11:04:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YU3kedkf/mSsGcpD@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0006.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6df64749-a66b-40ac-a5f2-08d98195cdcb
X-MS-TrafficTypeDiagnostic: VI1PR04MB7039:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70390797E5255D239FEA17E5B3A79@VI1PR04MB7039.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OYZulweWFfSIpzybj0c3CX8KtkGt1bWk2cEyZsn5dYzDOiuB6Ns5N2rV5ytxilkNalbNoT4qqB0C62TBPvFDEp6ubHM5tcY1J7iQMZ2FSmKM9Mhc4yJ436ZzCVMpD590VLLJKUhjKxi6yY7x06J3AE+m2Ib5zUypj/DJUQlmDwuxRRNuvq0osM5acCNDN+bcJdcKSvqY7My3iRFfx5iFhfmjP4/WG5ZnGlN0Lhl1qjYBTzy6PsUKwqvLQ+b6faGbq6rHmsXuTjivee8KoZApjKqqUk5dzQnZIwW13T1uxvf4P29/g8ir5WBUOslHF9jMkva3SIlm8M7Wezz2lu6gB4bXM8naRb9cjawIX3MSlLFaH0U3FV71NZoZa6QNZnJxHgBcCa3Al9ByaoYYVoiiVolRjRhYB8YAFTz8zYiKNzIQPGiKUjuVxnwOcU1hl4Ds91TasG8VjmJ5raCZtqMWj/4iwVc0Xn5fHMiT+Wp5mOZFq0Vo6WNLs40y60bbcRapZMd5Ei9RfFqcVqDfZozBUIAz/iVPYLx579EAfGHseOb+VQasOCj+98q9rGNS+dfPl9Ug49ob+U9u2fak0lD3fCd+JLuXxF1ozy+DT0Ka7ye5MZdCRDPtbRgESDkUjNo/ul6nmHPmdmlUHjEIj2bHReFRavMLeSrwVTDo7gpWyAUANJLixg9ivuPDuNaFL+yaVTkcHGhuQO2uO6F3inaGRX//KQyw+SLDu9DLW6nmTUw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(53546011)(66946007)(16576012)(8936002)(66556008)(66476007)(38100700002)(186003)(26005)(508600001)(2616005)(4326008)(956004)(31686004)(8676002)(6486002)(86362001)(2906002)(36756003)(5660300002)(316002)(54906003)(31696002)(110136005)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?1QJnb/hO7KIEZciTkLE927qEq+lH+9uNAKfsiH7R2GXuW9XldqU23TOyV1vs?=
 =?us-ascii?Q?ho0Uiu0BHtR+2VDsAgjmberrbpa93U9R2BceE6T2/KBfYeCqeScfJEJBH9uI?=
 =?us-ascii?Q?VMDyaIhCOTqTTtb6Wprw02i2bGwmNGyl7ys3ab9oDVtJqovpa8BeUjyG2P2o?=
 =?us-ascii?Q?QFE7HQAJNZY6kD1nsGxLu0IDxq8O0PB6P9HxRbmp2nwyByTYuawBiGQu4+Vs?=
 =?us-ascii?Q?D4CzPHUdpDN9jaOFkrwqck7sTSseqlyjknhayvoy6YSi96HbwkgiQk8HC2mQ?=
 =?us-ascii?Q?A8klWnccCftyNle1LlUwZeAUCcBKu6BhlaTexpkrPio3L8BdeCRljOwD/9G7?=
 =?us-ascii?Q?wXJV3mJDfns4vy63uJYSdA6E5gZufEfbfGwvFOe5yIQ27oWAlpnFRtD/1jx+?=
 =?us-ascii?Q?Kdsxi5fuWi5iKjYlszvgAHNi9D+HMwxnOkkedtjePGb0jlaExbmHfoPKCMZ+?=
 =?us-ascii?Q?91azHUQtMREbyjs5iGwNwvv5KQkVFS7zbcSmOJzBJ6Rwc7QbG1w7fMcAlVnG?=
 =?us-ascii?Q?dYrl8vNz/k5pN1EmfkHiAw2M2qDTgQeZOHyD7+97cJ2FihC4DTN46qWUQh6X?=
 =?us-ascii?Q?cb1cp32Wd9LoaENv4sv3ZKSYLp/QhoivgoNmxOccEQ4TGMYfXjBnS2f/D0Tg?=
 =?us-ascii?Q?gxhn44/+vlUmsQpmofuVhuVLmXE/PY2VGh5GQUnwBIUNQMhT4d7PmCg3u2Fi?=
 =?us-ascii?Q?otYnbBP4BWYqcI8qiG5vqiqgYpZ+BkFbiCDsRcu6uv4nnWuvp69dAqItylJL?=
 =?us-ascii?Q?0gEuhWoZqzJY1kr831zFtw9O6DSn81RTl5vxKzSTiLL2lvrkcb2YFtujcCJN?=
 =?us-ascii?Q?hhJe5/DET1V1c/5tcbc52X13sztD0avKgyzYL8+82bOQLfW577h7PRJfkl9W?=
 =?us-ascii?Q?OqI1QXgMDx1nJiov8iksuwDtCVWls9TMV4KqfwRmBXE4ExCwv3GK9aSxOApl?=
 =?us-ascii?Q?3i7cCHOb5KFAZsry0cqpze2h63NKvfDqPGKAzF3rNWNM4fXS897heQmp9Rm4?=
 =?us-ascii?Q?AuEzKWfKEPCjj+wS5RtrM5Bx3GtnFZZiqYbr29phUAOoWI0Unljt9WEuhemW?=
 =?us-ascii?Q?hroimY6oyJAAFUztuIMjCcI0sFfTH9KfzTw9kZNmlji0XZE94cKExFy+1diL?=
 =?us-ascii?Q?Sur0whSpzhxt4zYb8lfWftt7YY5sn8iZl38pI4tSQ4vNJsb9CbBMvgRmEoSR?=
 =?us-ascii?Q?i6l/+GUewQ+NwPStXvUu2UYrWP/5bIGGLqp65g4PNhJX4Uijzw1XVyX7I7oY?=
 =?us-ascii?Q?qUyCJDpng7idINWFeAr3lnNEAnknXal12FR+EAaw3kQjgJFq0yGNY4wyZ1dm?=
 =?us-ascii?Q?pWoJr+rkeETN+68izVXEhAaK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6df64749-a66b-40ac-a5f2-08d98195cdcb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 09:04:25.3590
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 59/EGjYP9KTsk2WckCxDCJ6/HrRuWhQYlL2sAz3PzZMYiiO7eIhwKdAHDf5pIcSVJgaCOp76CExPRb9y3qx51A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7039

On 24.09.2021 16:45, Roger Pau Monn=C3=A9 wrote:
> On Fri, Sep 24, 2021 at 11:42:13AM +0200, Jan Beulich wrote:
>> In order to be able to insert/remove super-pages we need to allow
>> callers of the walking function to specify at which point to stop the
>> walk.
>>
>> For intel_iommu_lookup_page() integrate the last level access into
>> the main walking function.
>>
>> dma_pte_clear_one() gets only partly adjusted for now: Error handling
>> and order parameter get put in place, but the order parameter remains
>> ignored (just like intel_iommu_map_page()'s order part of the flags).
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I have to admit that I don't understand why domain_pgd_maddr() wants to
>> populate all page table levels for DFN 0.
>=20
> I think it would be enough to create up to the level requested by the
> caller?
>=20
> Seems like a lazy way to always assert that the level requested by the
> caller would be present.

The caller doesn't request any level here. What the caller passes in
is the number of levels the respective IOMMU can deal with (varying
of which across all IOMMUs is somewhat funny anyway). Hence I _guess_
that it would really be sufficient to install as many levels as are
necessary for the loop at the end of the function to complete
successfully. Full depth population then would have happened only
because until here addr_to_dma_page_maddr() didn't have a way to
limit the number of levels. But then the comment is misleading. As
I'm merely guessing here, I'm still hoping for Kevin to have (and
share) some insight.

>> --- a/xen/drivers/passthrough/vtd/iommu.c
>> +++ b/xen/drivers/passthrough/vtd/iommu.c
>> @@ -264,63 +264,116 @@ static u64 bus_to_context_maddr(struct v
>>      return maddr;
>>  }
>> =20
>> -static u64 addr_to_dma_page_maddr(struct domain *domain, u64 addr, int =
alloc)
>> +/*
>> + * This function walks (and if requested allocates) page tables to the
>> + * designated target level. It returns
>> + * - 0 when a non-present entry was encountered and no allocation was
>> + *   requested,
>> + * - a small positive value (the level, i.e. below PAGE_SIZE) upon allo=
cation
>> + *   failure,
>> + * - for target > 0 the address of the page table holding the leaf PTE =
for
>                           ^ physical
>=20
> I think it's clearer, as the return type could be ambiguous.

Added.

>> + *   the requested address,
>> + * - for target =3D=3D 0 the full PTE.
>=20
> Could this create confusion if for example one PTE maps physical page
> 0? As the caller when getting back a full PTE with address 0 and some of
> the low bits set could interpret the result as an error.
>=20
> I think we already had this discussion on other occasions, but I would
> rather add a parameter to be used as a return placeholder (ie: a
> *dma_pte maybe?) and use the function return value just for errors
> because IMO it's clearer, but I know you don't usually like this
> approach, so I'm not going to insist.

MFN 0 is never used for anything. This in particular includes it not
getting used as a page table.

>> +static uint64_t addr_to_dma_page_maddr(struct domain *domain, daddr_t a=
ddr,
>> +                                       unsigned int target,
>> +                                       unsigned int *flush_flags, bool =
alloc)
>>  {
>>      struct domain_iommu *hd =3D dom_iommu(domain);
>>      int addr_width =3D agaw_to_width(hd->arch.vtd.agaw);
>>      struct dma_pte *parent, *pte =3D NULL;
>> -    int level =3D agaw_to_level(hd->arch.vtd.agaw);
>> -    int offset;
>> +    unsigned int level =3D agaw_to_level(hd->arch.vtd.agaw), offset;
>>      u64 pte_maddr =3D 0;
>> =20
>>      addr &=3D (((u64)1) << addr_width) - 1;
>>      ASSERT(spin_is_locked(&hd->arch.mapping_lock));
>> +    ASSERT(target || !alloc);
>=20
> Might be better to use an if with ASSERT_UNREACHABLE and return an
> error? (ie: level itself?)

I did consider this, but decided against because neither of the two
error indicators properly expresses that case. If you're concerned
of hitting the case in a release build, I'd rather switch to BUG_ON().

>> +
>>      if ( !hd->arch.vtd.pgd_maddr )
>>      {
>>          struct page_info *pg;
>> =20
>> -        if ( !alloc || !(pg =3D iommu_alloc_pgtable(domain)) )
>> +        if ( !alloc )
>> +            goto out;
>> +
>> +        pte_maddr =3D level;
>> +        if ( !(pg =3D iommu_alloc_pgtable(domain)) )
>>              goto out;
>> =20
>>          hd->arch.vtd.pgd_maddr =3D page_to_maddr(pg);
>>      }
>> =20
>> -    parent =3D (struct dma_pte *)map_vtd_domain_page(hd->arch.vtd.pgd_m=
addr);
>> -    while ( level > 1 )
>> +    pte_maddr =3D hd->arch.vtd.pgd_maddr;
>> +    parent =3D map_vtd_domain_page(pte_maddr);
>> +    while ( level > target )
>>      {
>>          offset =3D address_level_offset(addr, level);
>>          pte =3D &parent[offset];
>> =20
>>          pte_maddr =3D dma_pte_addr(*pte);
>> -        if ( !pte_maddr )
>> +        if ( !dma_pte_present(*pte) || (level > 1 && dma_pte_superpage(=
*pte)) )
>>          {
>>              struct page_info *pg;
>> +            /*
>> +             * Higher level tables always set r/w, last level page tabl=
e
>> +             * controls read/write.
>> +             */
>> +            struct dma_pte new_pte =3D { DMA_PTE_PROT };
>> =20
>>              if ( !alloc )
>> -                break;
>> +            {
>> +                pte_maddr =3D 0;
>> +                if ( !dma_pte_present(*pte) )
>> +                    break;
>> +
>> +                /*
>> +                 * When the leaf entry was requested, pass back the ful=
l PTE,
>> +                 * with the address adjusted to account for the residua=
l of
>> +                 * the walk.
>> +                 */
>> +                pte_maddr =3D pte->val +
>=20
> Wouldn't it be better to use dma_pte_addr(*pte) rather than accessing
> pte->val, and then you could drop the PAGE_MASK?
>=20
> Or is the addr parameter not guaranteed to be page aligned?

addr is page aligned, but may not be superpage aligned. Yet that's not
the point here. As per the comment at the top of the function (and as
per the needs of intel_iommu_lookup_page()) we want to return a proper
(even if fake) PTE here, i.e. in particular including the access
control bits. Is "full" in the comment not sufficient to express this?

>> +                    (addr & ((1UL << level_to_offset_bits(level)) - 1) =
&
>> +                     PAGE_MASK);
>> +                if ( !target )
>> +                    break;
>=20
> I'm confused by the conditional break here, why would you calculate
> pte_maddr unconditionally to get overwritten just the line below if
> target !=3D 0?

That's simply for style reasons - calculating ahead of the if() allows
for less indentation and hence better flowing of the overall expression.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:13:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:13:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196569.349460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmhK-0000Gv-8Z; Mon, 27 Sep 2021 09:13:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196569.349460; Mon, 27 Sep 2021 09:13:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmhK-0000Go-5a; Mon, 27 Sep 2021 09:13:18 +0000
Received: by outflank-mailman (input) for mailman id 196569;
 Mon, 27 Sep 2021 09:13:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUmhI-0000Gi-BD
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:13:16 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 246821b8-1f73-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 09:13:14 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2104.outbound.protection.outlook.com [104.47.18.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-85LuPaJpNVuc1bva3LyAbw-1; Mon, 27 Sep 2021 11:13:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2335.eurprd04.prod.outlook.com (2603:10a6:800:2e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Mon, 27 Sep
 2021 09:13:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:13:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR2P281CA0031.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Mon, 27 Sep 2021 09:13:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 246821b8-1f73-11ec-bc43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632733993;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=385TWVhCUgsps16snZlJb/hP9/SXebrRW9F9pdvegDY=;
	b=YIfqw6CL5ZYRx/2UZqU01rMSAmu+1Q/y0f9Z0tz0qH2i7l6Upbf0NKz9CzidzXfMzlbQ9l
	gsj2e/db+O6nO6RfxA2fSHi4/BqswppDZn2ToEgF0mLdlz1x+tvp1Uu97qS7raF6Sfxwji
	Gk6tNzZZppk61LGnxgMxn+PwsQS0LRc=
X-MC-Unique: 85LuPaJpNVuc1bva3LyAbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TpOUftQNqAEAjAJk7Nar3ifQWxDMqSKMw8v0DnW6igKWEYabljq9NmDRKm5M8Fr276EPZQO359Ka0OVE012MQYbl3kBqCpzWBFS3C9zxXJV9L/tQmawEAOHlRhRJgLBbau4z3h5RoBxwUDe6fHuJfGOlxac47Se3QXo1+1W6YS5B03w0IQqYUf8m4fK4aJzrNA9L/zC+qr4DPIRzYlBYmZyjAeiMQSWf2dZ+4kFLb1A7FSiUVw1z9Nrrx0IjisOovaUgTQPAhqIZkuW30QmN4dI6CCQIE+f9ruFZ3kIEZ3NAIOgexj504LlPEzDNGd81dABwkbHhdFN7hKI7duQMTw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DeHaMMvFOqxOkESfpOQMbNvuPqgWjvBdIjraCg0PN5M=;
 b=EpYcuNunX2cfnOycgEsIlWTXs+B3ZJN2YK6Z4CnVFn2mXtUrul5YSOjYxD1pZxRu3AMU8ELpf49awM6n78J/yNpdirZOhq/5vYvV6dgC5DBps18kqrd4pHpzVm1Dqopj4psolDwQULReaZtdjIP9/zm+Z19ETigEA9VP/qO6YWwlxqtzO/GM3Pqgt5W2Q54WR4uIbcTdZ1T1fZPfL0gpIwCnpMnnDKtMQ609OEek+vEkqw8BFfolo0KqmjEyCDxzfdPog3zwakMY5opCz/Kq70o0z0DPJm7wIXNg2m/3LnmiayxAWFY0r5PFsj6cyd6WpGemmSnWwzsaE1OwXJNifA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 02/18] VT-d: have callers specify the target level for
 page table walks
From: Jan Beulich <jbeulich@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <957f067b-9fe1-2350-4266-51982f09d3a9@suse.com>
 <6d01a401-ac53-d4ca-e0ac-165d045341af@suse.com>
 <YU3kedkf/mSsGcpD@MacBook-Air-de-Roger.local>
 <91745efe-129b-4851-87a2-93eea8f6d63d@suse.com>
Message-ID: <3ba86d34-1e24-7b29-658a-34d8ff7c862c@suse.com>
Date: Mon, 27 Sep 2021 11:13:11 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <91745efe-129b-4851-87a2-93eea8f6d63d@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR2P281CA0031.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:14::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: be2de79e-bad8-4760-1405-08d981970705
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2335:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB233598B6DCB48D1FCCC31965B3A79@VI1PR0401MB2335.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6repov2BftTXMvpPlYoMaxymDGB8dLvxNs67n9U7rB+DL519LVaUDtsQM+/AXr+oN8/1cZnaVzjfUKSGf0Kf/jPP9LqNkiofk3jiESLT2BKc/byJBIp6lyRIHW9TonbnmoXiv+uPdstaywYbSp0A23j2XCudjLmvkS8FB0B28aI3IR0TGdLpb0a/G9eLuYelP9fUSVDMjoD6Ll2rCNpM4N4OvSphRQh5sR3LSbSf1FC7u+9oLB+c9CxwMrSCbWldRJ3dfLMyknE/XaVTTp2SpJMKNlIQlRyd50ijwU/afkzoVSik7xLXkb2GDtan0CrAE5ZFLir0UnOvmyLsyzGfvbwai6nnybCtg22bFY4sTZYt+hMhhxvlmP2NsLjfltpXktPuDCtEfoqeh+720JBeJAip3/hZfqAOaI4WeHvCy9gHMobN0aOhjkjlT/1bBQTaz5MQluZtNBPj9Jt2J/9h4iBDWkpALu288wRW24mlgT2k/tepQ1gvR18ZW70f7OTfAbwA7oVHqzaI3hvE5YWXa9fFOf1ozpXVmPHxsUSpKy+EKQlCPBl5UFjlTScAYKd6uEDqEJjwvDLNNK8KR6dR1QTOA/pUr/3Wqb40DPGwx9nZAp8xL0guXL+CeOTI3KbB2iwfp9EmAi79+hfyBeEqbl1ru4efns+FoUezP90OXZb87b6U/9KQ8C8HEQXCpswmoPLsE9DefRQoSdy+3frzohXystqvKOVTnUw5+z41ysKc7xFuWwXMLF4TtcTvm4bK
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(36756003)(110136005)(86362001)(2616005)(316002)(83380400001)(16576012)(956004)(8676002)(8936002)(31696002)(53546011)(186003)(66946007)(31686004)(26005)(508600001)(54906003)(2906002)(66476007)(66556008)(38100700002)(5660300002)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?ar0cM/Ek7DtnA28WPLZ5d7xg4ardWAGtzaAaCHaFbaqu8FqFsgrdvW3H+UNr?=
 =?us-ascii?Q?/diRrFy638i7mAk/CaCc8NGqcZVJYwRnAEQBve6II81VaLyVOXQHW3puxfL8?=
 =?us-ascii?Q?u52k6PzQBZyeOQfHwnNl8VmICkw1majOe+6xRFOiR+UXbCLx7SX83ChY39/9?=
 =?us-ascii?Q?d36neO2MuliNkKTKJlR0C42noybmy94rShc1Ky9ptZRwdcLgHoD04asp9qVo?=
 =?us-ascii?Q?eyoHluMaSe38PtESt55qwAIPQdJLCCIL+RG60bl3d6jSMG9AqrGI4vUWVyXX?=
 =?us-ascii?Q?v/W19YHXMBJRG0IBldIHWkLyOdnNFVlxflI4TvGiUO+UD+zk+evn6k5EcKQe?=
 =?us-ascii?Q?dw/1hSVJjGRuadGD1FWyk8doXUjU5ME1Dz6X/pSMiGXrxDagEBLqP6g3RrkD?=
 =?us-ascii?Q?z03ZhwfK7B5qlHk492+/vwJ8Rvkv3QzVYK+BTY5iKTGz9tfNVU2c5s++9AJD?=
 =?us-ascii?Q?AwcFrVmHnlJyOxxfQ9jWyk+tHqSV6iI5OQkvr/cuFG+8LrnZS6ubRqEzvqN6?=
 =?us-ascii?Q?sIdWpROMjfXVnBaC2JHdW1bZVHg9x0z3z1bYAJkj8NMNHe/O1JHTnmXFuH4C?=
 =?us-ascii?Q?B+atoyl8R+RPFfELZYAe8m6WtoXbiW+ea5lXv3wRIPGyN6ASK0TodDb3oayr?=
 =?us-ascii?Q?7ubo0mOQuMLsqjKoaPiyB5pntqV/GcrKiiEvYpj2HIiLdo07h2AfOTKy83aT?=
 =?us-ascii?Q?dhLheAvpcoPruEsJW8e5AQROBjLUsFUd7WYWSp2yVEFkq1v/Cj7QwwCWHYth?=
 =?us-ascii?Q?7YBwBs1SFTZZ6zPFk4QdcMh5rAdXanOM0+r/gfEY286PKvIWwAOtnsD9o2B7?=
 =?us-ascii?Q?S6h1EgAQidXJWBh3TTnUKbEB++JBeVKawKnjAXNWcLd7CccM4Qid1/2QuVNf?=
 =?us-ascii?Q?8R0Ct4n1tgZ/Wviin6tLCzaa4tPLgYSQqIvjssSQcY3lKpzGAQYjKEEconcC?=
 =?us-ascii?Q?70lRPP+xyR6tWGC0dzmxlueY1WqzRhGmW9GdPZ3qQIjyklXPasY/VrI7/Uc0?=
 =?us-ascii?Q?6fwpnPbVOm+j2W130GsfyNxsYkj3q2Arcmgq0iWO+67QmsgDOUZ4jmYc8Anx?=
 =?us-ascii?Q?ku6e+VkDaqv6pMKxjkDYb5U/A1RdjHpIAuZrI4FLprjwnSS7+wR4dK/WUj2G?=
 =?us-ascii?Q?rjSWoeFbm+oaQ1ELGxJUh0EuKjSCwZ2yeFpouzc0RKn+Lg+f096Vd7B7UE/A?=
 =?us-ascii?Q?8dE/wmyZtOIXw+PX2Mq/CiTV9dT1ht64+hhHe/bBMWoP9oekxL2eY2zdcvvS?=
 =?us-ascii?Q?0ArMhzcaPkQUpmGExV/mq9JO5Zu1aMOwHd4HO6QEaZYzEhdJb3lHHviMNdOs?=
 =?us-ascii?Q?ipGPM3FmCHpBsFvVGsKEHKL+?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: be2de79e-bad8-4760-1405-08d981970705
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 09:13:10.8373
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ASqW5eVsvLS3W24W1baRY6BjW1Uq4ItVi0isfGNv2dUyAXFSavArh5M7VTRbfLZoZNZZ8tAu6ZPZPUcBZRpqXw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2335

On 27.09.2021 11:04, Jan Beulich wrote:
> On 24.09.2021 16:45, Roger Pau Monn=C3=A9 wrote:
>> On Fri, Sep 24, 2021 at 11:42:13AM +0200, Jan Beulich wrote:
>>> In order to be able to insert/remove super-pages we need to allow
>>> callers of the walking function to specify at which point to stop the
>>> walk.
>>>
>>> For intel_iommu_lookup_page() integrate the last level access into
>>> the main walking function.
>>>
>>> dma_pte_clear_one() gets only partly adjusted for now: Error handling
>>> and order parameter get put in place, but the order parameter remains
>>> ignored (just like intel_iommu_map_page()'s order part of the flags).
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> ---
>>> I have to admit that I don't understand why domain_pgd_maddr() wants to
>>> populate all page table levels for DFN 0.
>>
>> I think it would be enough to create up to the level requested by the
>> caller?
>>
>> Seems like a lazy way to always assert that the level requested by the
>> caller would be present.
>=20
> The caller doesn't request any level here. What the caller passes in
> is the number of levels the respective IOMMU can deal with (varying
> of which across all IOMMUs is somewhat funny anyway). Hence I _guess_
> that it would really be sufficient to install as many levels as are
> necessary for the loop at the end of the function to complete
> successfully. Full depth population then would have happened only
> because until here addr_to_dma_page_maddr() didn't have a way to
> limit the number of levels. But then the comment is misleading. As
> I'm merely guessing here, I'm still hoping for Kevin to have (and
> share) some insight.

I've extended this post-commit-message comment to:

I have to admit that I don't understand why domain_pgd_maddr() wants to
populate all page table levels for DFN 0. I _guess_ that despite the
comment there what is needed is really only population down to
nr_pt_levels (such that the loop at the end of the function would
succeed). Problem is that this gets done only upon first allocating the
root table, yet the loop may later get executed with a smaller
nr_pt_levels. IOW population would need to be done down to the smallest
of all possible iommu->nr_pt_levels. As per a comment in iommu_alloc()
this can be between 2 and 4, yet once again the code there isn't fully
in line with the comment, going all the way down to 0.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:16:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:16:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196575.349472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmkp-0000wO-P5; Mon, 27 Sep 2021 09:16:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196575.349472; Mon, 27 Sep 2021 09:16:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmkp-0000wH-Lp; Mon, 27 Sep 2021 09:16:55 +0000
Received: by outflank-mailman (input) for mailman id 196575;
 Mon, 27 Sep 2021 09:16:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUmko-0000w7-7l; Mon, 27 Sep 2021 09:16:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUmkn-0004Cc-VQ; Mon, 27 Sep 2021 09:16:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUmkn-0006eN-K4; Mon, 27 Sep 2021 09:16:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUmkn-0003Jd-JZ; Mon, 27 Sep 2021 09:16:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mOrVOw9pZt10N3ZJ9fGdbpaUsQUTsGrXAkqFvj6W2xw=; b=zpDlXDH1CbgS8xHOwRGHaNjqPZ
	6ZGAr3uMbr2usoteQTlis6QqqQGV3t9zEmuzsnQ5ACvISF7j5jmoqOi2UtIGNVTDPistLLseGdYma
	tMav/TalthEORPFG9JXgSclzmZGLkhJ4jgpxK5MboMIFopDHSH+3Y+abDUYKgM1AcXDE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165210-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165210: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=11a11998460ed84d9a127c025f50f7234e5a483f
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 09:16:53 +0000

flight 165210 qemu-mainline real [real]
flight 165216 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165210/
http://logs.test-lab.xenproject.org/osstest/logs/165216/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10       fail  like 164950
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                11a11998460ed84d9a127c025f50f7234e5a483f
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   15 days
Failing since        164967  2021-09-13 13:06:52 Z   13 days   27 attempts
Testing same since   165193  2021-09-25 11:09:10 Z    1 days    4 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5088 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:17:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:17:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196579.349485 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUml3-0001X1-B1; Mon, 27 Sep 2021 09:17:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196579.349485; Mon, 27 Sep 2021 09:17:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUml3-0001Wu-7c; Mon, 27 Sep 2021 09:17:09 +0000
Received: by outflank-mailman (input) for mailman id 196579;
 Mon, 27 Sep 2021 09:17:08 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUml2-0001GS-Ct
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:17:08 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b69856d3-ed26-4515-ad7b-33884b3e85a2;
 Mon, 27 Sep 2021 09:17:03 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-GNQkiIwDN4eqRvy3XK1mbA-1; Mon, 27 Sep 2021 11:17:01 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 09:17:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:17:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0282.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 09:16:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b69856d3-ed26-4515-ad7b-33884b3e85a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632734222;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=tNKvu3ro9YeaYI6egGAfV3uRkgbGoPHqahZniehNlok=;
	b=GDmfXn9Zg4flioi/rcIbsPgNLY9juVPkNQUOpKmjFaPpw0UOMN6b7/DQCMC1TFuswR7ZSV
	EEuY/yjtfZ8lZTj9vpFSS4YcRg9ERCPyWRgz1l3zC3pA3xsAJLeav4jryt2GqVl1VktPzs
	Lsn37INTpx/bty6Rd/TVHlzIzz1pkyI=
X-MC-Unique: GNQkiIwDN4eqRvy3XK1mbA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ak5IaRTf1oDK1zBRoZd/wumU1AKNZMq/GllqXquZyO3xgDmxYVNCqtoboTG5vY7dfYd5QepIi8eYF64JiMDKseNxDuygK7xdftgvuxfDsBax2KkG3w8mpsC0CiGhys+Ql4wXG1pd2WzEN9UY/nZVG3E9K7PaREzswMGQ3ORcBAjAYoKn+kl/MetUL8ToJ4Soz6zRWuuhroX+HvENFBZKcpbh54agtyW7mFPBKNv5DrXcXY4Y7cvrY/tLNCn3eao8ShQ75lXu6ZZWo/sUSSX8OETYRxeK4kveQ8gmyrHlkXK2oCxOgrbniEd6aW1mOXqJtHqXUkylFe44+Eg9cBsPXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tNKvu3ro9YeaYI6egGAfV3uRkgbGoPHqahZniehNlok=;
 b=gl2mKlkmAIS+LS3eQP2Dbo8FyMWsiuv9bLfknfCYkJDwy+xoWYr5wPQ2Bgjswk/1auzVtQTiL/tKj/Sur/fR7uMOQZ39/IxH9ePC3RgwmYkZi4+iHdboJH8/XNfjw7yP3M4Xu/SHkiFRw6HmC9O2KSEBURdNEoFzKL2/i1JxpEFqgSBYmPBqaUCVysJrMgfEwwOGVKCDwy82VaRoO5e+jsM6uw6jW8nmnM+c8VFMMlc85hf2RJ0CxaLRZtKWEMi+30VwJYurVBPyMLu7CdehazYmamAHuP7MZ285KJme4icFQDkK2AxRgFFWl3LOollCLBRzAAADeNTkYczA7ugxow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
To: Julien Grall <julien.grall.oss@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>,
 xen-devel <xen-devel@lists.xenproject.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
 <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
 <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
 <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com>
 <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <42eb1303-1b45-5489-eac3-855f4ab35eb5@suse.com>
Date: Mon, 27 Sep 2021 11:17:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0282.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae98d9b7-eb47-4b14-42d4-08d981978fd7
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2703719EF123BA6027972D5AB3A79@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IN5LFcd7EhLsxob6X+JdrcgTHqfThrJLyRftZ1t+eaxOxnjKAcbhgEsZv7ynpTp70tS9qzc1BywVw8Unlnt/SHaEj2qlTl4Y7Ai9janmQAUsqUKm2SGwcM5mpan6NEy8aLjzKLxGq8F5ucq8HaSL+k+ijNHGDBUbCWVJi2Al6RSd7Mo85E+RRezCiRaZztmfxCZgeQIZbGwesJkvocmNJwALE7yCbyXY+8IHNb2/vKXeWuPzrjw5TxBb9q7B4IUnsd0YMDEXulfbM/mY66sOsCs1j180nIrTf6mrNWhzFT09DL7o0Xa4xxv6WWXNjqzZGOwI2E/qeTJX+KTVcwMwgSoHSz4vnpzrRme4oiKKVY0MzB/+3sT31KTellM4BMdpN3Qs9WgpzU9X0BuBJvfbxT5lxcqhUP3WRPM5n76jzUuWoASCdJU5YdQz1bzRQhCU8z5aqgHPWMEgpflA4lG7sQ2u+TXD1twvUtWlvowC4LdSqwQ+Gbdt2UNhGSwkLBpOIB+HHJ6iinBBjuo6CCULgg6YeyeeLGfrGmwIv7f3TAtkMZXLT6Q8lwSaC+xzHsoiloJCauwZWJipCr0p5KFjFZw2VP7/L5h3ZrjYKbLPRcJ4xRuSRuJ6PcDt03BhfIkqxHsHTfssthPEMPtOfl//M05IopRzHaM8iMLZUAldHQbHnIbyRM07zVDagCL4hFjP6zXRFYpTiCBE2LydX4JCNqLzkQ8VTDHhA6x2fR/w3Poyqi10Z+RQBfCxDPE6Zfb5v12FIThnUSb+HvvsijiJ8ZXEBHw2/m59Vp87zANYb0MEq0kEIPPWUWWf8pYxSdX40WyGkZnFasKzAfiuH4AThQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(66946007)(956004)(53546011)(31686004)(8676002)(83380400001)(6666004)(4326008)(5660300002)(6486002)(54906003)(16576012)(316002)(2616005)(38100700002)(2906002)(186003)(36756003)(66556008)(66476007)(508600001)(31696002)(966005)(8936002)(26005)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M3hvZUVFYVBkdkZIRTRhajA2WUVsa2toUTlTODBkUFNwMXRZUHZQVkxpVVl3?=
 =?utf-8?B?UW0rVVF4eDNLSG5WNlRHdW51UjdoOTdWUlpId2tkTHh4UjlENjhBS3Jtc1lJ?=
 =?utf-8?B?QnBiTGh4TnA5Vi9VRlpORmFVLzdoaTVwSEdRTVhLZ2FBUEVsbDM5OStDcFFh?=
 =?utf-8?B?QUFkaWpraElsSVFiZWRyaExWd2htVXF3YnhLODlpOWRvbU91ZlR4eDBQZjA2?=
 =?utf-8?B?UXpwVzVrck5aTktuYXc1RnFDNFhuKzJIVjhCLzVSZFdkY3pNanFRcGtoNDdC?=
 =?utf-8?B?Y0JzaXhhamZrY1JKcTFuTVZCM2tYUzJTMVVQKzRGeHBmRVhCVnpiNnpJMjgv?=
 =?utf-8?B?TXQ3TU9NR28wWkJHdVdBbkN5NWNXSjV0MU9BaGE2RlAyM3BZMm9nV0UrQmxT?=
 =?utf-8?B?ZDBKUE00WjJhN2RxcFJaUWFiaHRBZHk1amVlRFd3ZTJyckdQWjV4eCt1VHIz?=
 =?utf-8?B?L2xoeUtoNDlicmZxcHNuQm03ZE5FMllYRTUwN21mWGJ5bXlvZjJHZkJlVFB6?=
 =?utf-8?B?L1JaOEVmeC9rd0xOSnBCQ3pkMUJpYUVzS3VNOHR6MFhJaVpyditKcFhvUzBD?=
 =?utf-8?B?NGdpZElsZ0dDekpFSnEyYkhQODJSbGhJYmdJcXUyU2Y5OGthS3prQWtZNGFX?=
 =?utf-8?B?K0xOYmxjV1RIOThkNnZ2dlUwUUlYN1pJMjV5OWZqOGg0bmR0M1hOVVVGNEdQ?=
 =?utf-8?B?cDFuc3g5UWhQZm1mblZKVmJGYjFhZzBlOG4veFN2Z2JnN1FvNEhaNmUrK3R0?=
 =?utf-8?B?VFpHelhpeENNbzNuQ3ZZeFdKbTdROW5IaDRDeitTd29vSnVCWVZMOWdac2tX?=
 =?utf-8?B?c0dGOW5VUkR0ZGIvajJyN05BeXRFWURObGhKOVNtbmVwdmh1ZDJxK0YvelZa?=
 =?utf-8?B?NGVoZVdZMUtKb3BpbUxTalFhQjlzV21DY0xaOHQ3VHJFbnE5UEtqdW5GeW10?=
 =?utf-8?B?OWk5RUFpWTlkSFFzOWdKL1A0a2hpeE5KdENYYlpaY2M1Z0ttNmtXdDR1QlZi?=
 =?utf-8?B?MUlRbG5LVTZTVGN2S3l2QVhVRjhpejRaajh4NGpHTmpUbU96S2hIZkNqdXZt?=
 =?utf-8?B?RkRwMFQ4Z3hXYWJMVXBMZGtueWRYQnFiSFg1aGVQVHFkNDlvKzJFY1pQNXlO?=
 =?utf-8?B?QmN1eHRxempObEFUOXlPR3RTWndNRGtGV0RVa2hBNDVxSlJjTzM3RmtKdnJT?=
 =?utf-8?B?WTRmMjduU2NHOVRuYWx2dHlHSjJEMGM4N1I5QjAvQkN6OHNVZlZyY0RtckdF?=
 =?utf-8?B?aFhlbVk0d0oyR1J1RjErL2JYRXplLzJGNWc3WmpWREdLNks0U3VvMmQvZ0w1?=
 =?utf-8?B?Sy8yVklBMHB0KythT0pLcTBwanovdjBEdS9HZTdUd0FlUFprb3k0ZnFlcHNR?=
 =?utf-8?B?Wk50VlZ0aTFlTVpFTThuY1hJZXFGRE83RDUyT09vWkY3Z0NBR09xU3pCVVcy?=
 =?utf-8?B?T1dWc2p2SUc2T2kxM0ZmWWhzQWdJK2NRKzM3L3hNRmM0Q3lCNTVWN3o3dmY1?=
 =?utf-8?B?SmhSVlZqODdvUHRNbnk4cWJKVytOVEVSSE5PeXp5bVdPSWNXbS9zWkIyYnVV?=
 =?utf-8?B?M2x1djh4U3UzTkhxWVI1QzI2eHFuV3N4UWJrRWsvN3RsSTZFY1R5VjZuY1JF?=
 =?utf-8?B?dVQwZWExVzVPQkpybjd5UU52Q0NENGJHakIzM2JCRHk5RnhiY3hnU3hmbzNP?=
 =?utf-8?B?Mysyc054OGxvTE5kaSszY0gweFNXQTNNbnRpQjl6cTQ5NHA2aGQ3ejY5U2dH?=
 =?utf-8?Q?PIGY7PwbSHSrgQf6Na5YE7h+ML+NgeFIDnkVQbV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae98d9b7-eb47-4b14-42d4-08d981978fd7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 09:17:00.3839
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 32OxsL5WquP7L4BUlUsJDnSOTqAgqXkDyjoyt8d04FDD2T0jB0wTX64rVBzYiaLTNV5WVANaH4j1IDsiYRc80w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

On 27.09.2021 10:45, Julien Grall wrote:
> On Mon, 27 Sep 2021, 10:33 Jan Beulich, <jbeulich@suse.com> wrote:
> 
>> On 24.09.2021 21:39, Stefano Stabellini wrote:
>>> On Fri, 24 Sep 2021, Wei Chen wrote:
>>>> On 2021/9/24 11:31, Stefano Stabellini wrote:
>>>>> On Thu, 23 Sep 2021, Wei Chen wrote:
>>>>>> --- a/xen/arch/arm/Kconfig
>>>>>> +++ b/xen/arch/arm/Kconfig
>>>>>> @@ -34,6 +34,17 @@ config ACPI
>>>>>>      Advanced Configuration and Power Interface (ACPI) support for
>> Xen is
>>>>>>      an alternative to device tree on ARM64.
>>>>>>   + config DEVICE_TREE_NUMA
>>>>>> +  def_bool n
>>>>>> +  select NUMA
>>>>>> +
>>>>>> +config ARM_NUMA
>>>>>> +  bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)"
>> if
>>>>>> UNSUPPORTED
>>>>>> +  select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
>>>>>
>>>>> Should it be: depends on HAS_DEVICE_TREE ?
>>>>> (And eventually depends on HAS_DEVICE_TREE || ACPI)
>>>>>
>>>>
>>>> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
>>>> option can be selected by users. And depends on has_device_tree
>>>> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
>>>>
>>>> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
>>>> does it become a loop dependency?
>>>>
>>>>
>> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html
>>>
>>> OK, I am fine with that. I was just trying to catch the case where a
>>> user selects "ARM_NUMA" but actually neither ACPI nor HAS_DEVICE_TREE
>>> are selected so nothing happens. I was trying to make it clear that
>>> ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACPI
>>> because otherwise it is not going to work.
>>>
>>> That said, I don't think this is important because HAS_DEVICE_TREE
>>> cannot be unselected. So if we cannot find a way to express the
>>> dependency, I think it is fine to keep the patch as is.
>>
>> So how about doing things the other way around: ARM_NUMA has no prompt
>> and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt instead
>> (and, for Arm at least, ACPI_NUMA as well; this might even be worthwhile
>> to have on x86 down the road).
>>
> 
> As I wrote before, I don't think the user should say "I want to enable NUMA
> with Device-Tree or ACPI". Instead, they say whether they want to use NUMA
> and let Xen decide to enable the DT/ACPI support.
> 
> In other word, the prompt should stay on ARM_NUMA.

Okay. In which case I'm confused by Stefano's question.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:19:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:19:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196592.349496 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmn1-0002Sg-OF; Mon, 27 Sep 2021 09:19:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196592.349496; Mon, 27 Sep 2021 09:19:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUmn1-0002SZ-LJ; Mon, 27 Sep 2021 09:19:11 +0000
Received: by outflank-mailman (input) for mailman id 196592;
 Mon, 27 Sep 2021 09:19:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUmn0-0002ST-BN
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:19:10 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 12803130-117c-4f34-b967-c01a463f0ee9;
 Mon, 27 Sep 2021 09:19:09 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2054.outbound.protection.outlook.com [104.47.12.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-kynDTfq9OIGmz_r5AopOTg-1; Mon, 27 Sep 2021 11:19:07 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7087.eurprd04.prod.outlook.com (2603:10a6:800:12a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 27 Sep
 2021 09:19:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:19:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0137.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Mon, 27 Sep 2021 09:19:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12803130-117c-4f34-b967-c01a463f0ee9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632734348;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JhnrBnwuUNF23CxQfoC4vO8DtJ34XGVGnTDJNQawta0=;
	b=OOrCVJvilJa4xYToXDMbkVZXo/yp6oNgD1u/mMDGPJRukSNBtMfU9ld8H8CeiChHoZxCaW
	VFl+SwWYFYVILdrVWjKVvEkWA90R7jMEHa8NnI2Ca9OyuI3DYn8BVaFfArf2CGBcna2tMq
	Zv0Y+obotXQ4UpCgVVjF1Fs9mADUj3w=
X-MC-Unique: kynDTfq9OIGmz_r5AopOTg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=P2qMDFCUhQGjVAqclSP8Vy4jpMJK7VBu+z6hcx2TQGNX/fu9kVHLwFaeV70Usv1o5iuHU/93iqYoG9k054sNNbPLm2ftUXJakIFMT7qKdn5PK19osKONoo6vkBmiDtdr8Chf5gxvY4T66VkkFeTxODW+taqiF/XzEZXjkUJ4UyBiOsrjQpXFZ10ZlUUGBYFDG/5YqdRUzwf1VapRh5uJt1jNWdKPo2GX2QKBARpDE+4kjQVq2q+e3YRgvCigTldX+Ey8PsDE4EOcTajGW098RYg/+mX78HmK/ro7xNoP9okVfwFS3ophBbaXdW+5xq1OzckfJ5BAKn4ZnTkNlOiUEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JhnrBnwuUNF23CxQfoC4vO8DtJ34XGVGnTDJNQawta0=;
 b=EETh8Ba730Jv8jmza//sSSeoCuKZ+08Zm9bWnkK+usk1nEaS8mb0K2/uMqXXiv9LgC1Pll8r7w5bOxKhEvwR+3wNUOLpp9iJ3KhczdVMEhWyvGGBZIunYabyILgCOOR3avbyCH9KydbTD17qpXq0QIClagq1S9KExQ82NJdQIGo9eVFO263eySDBgGGhw4LHAIAWaDCLQgbIyrpQyZSebmCxj5UO90X5irP2q+EbPfIjmamBlYSbWUChMXrcjE6vyUQ+vAjsubA0IYrf2sGu0LodfRE5giChVPdZTE4D9n8qUZAClr71h5xFpN+z/YoUKj/NuuukypoNwVGhfTEhiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
Date: Mon, 27 Sep 2021 11:19:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0137.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b4e3c112-6c2d-4c51-f7b0-08d98197d9e8
X-MS-TrafficTypeDiagnostic: VI1PR04MB7087:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB70871A44C1F75FC64E7C9E9DB3A79@VI1PR04MB7087.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZE1VHtjq6MJh5gtHTclFARm+hwxRqfUmfTij4Som4pa6/TE6CF6ILpSd4Z+LP824CiiDrJ/gzkc0oQgH3BKbxkhd8L/iteO9JVdDjotbPzyKmptjc7k/GSlSGgKPfL7uLSFqh7VA9BdqCQ+25D74IuUmMNiVPTnnmpAFs0TL8Mgxo3iwWNQMLDiGFe79FX5Je4MGMwVx8KhLIFeFZbUMehfsfgmqUNngKrruitq6j9kk9lf6SJBkc0g9Xe/wI99SxyQQo3ZJ1sSsL+XqhW9yNp2w8jeVW+BUPjaY7D02S4Lg18uVaCWf53ZdWHot2nogNVwsUgh1cTQgY+IPcROQUDRiQMDvxSjyP89AgeeIIvTwUC6aUhEHhP7DDfN8LDvKS5VdiXOhuKpTz/u9yBMzQI1BWnMSMsEtf+gb1u+IaxYHVpSqwK5e6xXIRoHD21THBf/eXZViBgM7FGvrd6K9x9GwxSpApauw7nOzfJ46qNuHT3gxej79BAZwyuqGiYt3P9JbQKL3fq0Ldj3ZNxm+MsJ9KmNYIUB1oqLZlFGPdfeDlsOcFCxOonGebZKwNM8zpubW16Ki3Po2Hvq7xHUEmiHSRLNDhCqkgIiHSvjrYivrWbXp15Yx4IPVLRBzXFbpvRPQpD55Uf4Cdyqbgcvt7AAr8mMQwHcs8ofNIBFyars6ZhYbacDuBzmFXFAz0aFcheRLZUw7w6jQWwk0NYW5RR2jdfPhDXtlUY9XW3LXFQ1zc9BHGyP5T52mZxDEJqKs
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(508600001)(2616005)(316002)(16576012)(956004)(54906003)(8936002)(2906002)(31686004)(53546011)(83380400001)(7416002)(66476007)(31696002)(36756003)(6916009)(38100700002)(4326008)(5660300002)(26005)(6486002)(66556008)(86362001)(66946007)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Wk53WGU2Myt6eTZHeElGNWxXTk9keFNDUjJPUnJkT1p5N1JUclJyZXkyTGJC?=
 =?utf-8?B?TGkwQkV1NlBjWjQvTWJRQlQ5T0g5WGwwcWVJdmJSREhVOFdnWEs3bm85ekZB?=
 =?utf-8?B?Y004QWgrSDRhWHlldEQxemVSRDFRci9LUmkvbjJiK2pzeC9zdm1oZWFkeUNQ?=
 =?utf-8?B?c3VwbXdLTVJOL0VVYkdCZTE1Wi9ybzl0R0l5MytjVWNGVlptL1ozTDVYVVFD?=
 =?utf-8?B?UFJYSVRueWJ1LzJXVE5aQUFxOTNzcGFOMDZXZHVVb2tKa2VwanlNQlFxRjV2?=
 =?utf-8?B?TXRZdVRqN2RvVEV0cnRLN1QvTnVpcmlJdkptL2NSQVdXRm9LMEhzSlFUVktp?=
 =?utf-8?B?TXVjMXhkQWJlbC9GcWVFS1ZGUUp1TUVYb09CRVBlQTh3bGc3a2dYZkhvWmp4?=
 =?utf-8?B?NHJ0VjVaTlVRQXZxNXlVZm9ibGF4bUxaV29oWllQYnp0S1pSYlpYelUvTjBv?=
 =?utf-8?B?YjAyZk1XVTZ3MTVGVy8ydjhBci9LREdIdFo2eERiNExUZm4xbGIrTWw2K1Nj?=
 =?utf-8?B?eWo1VWJkRVJrRWdZQmRzeGNkVDF5WXFieUhPTTV1VlJWLzJsYU5rOUVFcjFx?=
 =?utf-8?B?ZEp3Sk5sa1pjeHFaL1o5QW44aHZqZGFiMmlJU2ZvcEtaQW9SU3JEdTA1d1Rr?=
 =?utf-8?B?Sm1lZ3UrWXVrdU91OFdJL1YwS202U0t0UmV6QXNxdlJZTlFDak1Ea0JDVm5y?=
 =?utf-8?B?RnFlamdoZkRnLzVjVm8zRXdRKzJwMit5cXg1UGNaQTNmUjUzYUZNQnFQWWJK?=
 =?utf-8?B?MGNKblV2TVRkT2psTTRsS1dtb3BHTlJ6a2dONlExSk5lUmhlWTFJdCtpOEFq?=
 =?utf-8?B?T2VENmpveU9zeFg0bUp3T1gxUUxsckdodDI1RHNCNjllay9rL1IxRjRMbHJv?=
 =?utf-8?B?dWdrRkdubXN2dkx6akhmRitIVUNqbnRKS3JmejV3dTJucjlJWCthUExsVHVk?=
 =?utf-8?B?bnZpQmMrVkY2TTNIZ3lya1RRRmxqYUdVN25LcVBRSXZkNGUvU2FtYkU0UlBY?=
 =?utf-8?B?T0Z1OEJCcUVUQTNvRllncDRtbWhIOHVaazNKcVJ1bDh5S3FEaWQ2STEyZ3Fi?=
 =?utf-8?B?UlgyNHVuZFhtUmhVT1BZS2tMckVtNDQ3R2x2U0FLSTloYVphTEF6MEllRElO?=
 =?utf-8?B?RnhsZkhtdDFmTVNHREJZa3NjaTVHWEx2YzVUdDRWNHdYWit5UkdQMysyYk1E?=
 =?utf-8?B?cloweS9WaWp1Vm1NQmVoSmo3MWFHZVF6OWpnbnlkb0FiNHNHQnI1VjdnelFn?=
 =?utf-8?B?dmtUZ0JWaFAzbEhDRFlFZ3FvbjRkTGpqQmxLQkZYR2Y5Y0tZR2dVWEdsUXlC?=
 =?utf-8?B?OEVXSnVBNktDN0k2VlZRb2gzQmJqc1ZiQytrN1JaSVF2dDFVZ1hFTHNEZUo4?=
 =?utf-8?B?cE94TzlsYlJrbFNvK1JsYkVVcHFWdkdxd3VLWFZVaUVrNVZ5bHhHeWU4U1pB?=
 =?utf-8?B?alVzRVQxMVMzTlNiU0E2V21ST2JRZzYzc0dBN0U5N3F4b0hhd0pMYVRlOHpK?=
 =?utf-8?B?WWlSYUpSTlh0Y1ZDWkh0VDJLaWRrV2U4d3dNWHFZOWVBWUhicklkNVM3OGN3?=
 =?utf-8?B?N3JDMVlnWUdTZG50VkZUTnh3aXlNeFlrbTRWRWhKbVR5NzRNOEdxdWs2VGVZ?=
 =?utf-8?B?c1VaclBUV0VZbk1tS1RYQUFkOGJ0cWRUNm1uSGRsSFFEckxmRmh5MmdQVHVO?=
 =?utf-8?B?TzljaUp1c0dsRGI2QkU3WGwzZ2Rzd1pyd2xNZWRsMzFGaU5WZ0hoOW5XQjBo?=
 =?utf-8?Q?RZ+iuo5b29QCOoLpSJVtg0Tmkc+XhaZp8VmjbO9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b4e3c112-6c2d-4c51-f7b0-08d98197d9e8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 09:19:04.6303
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hD3Urmghqawufk2nwcw05FGXYfxRcM6cP733fMcqK8vofG2hLXNTyQiy7QQ2kSXI39s7ylvz9uCmUhnuh9L7ZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7087

On 27.09.2021 10:45, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 10:45, Jan Beulich wrote:
>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>>>       *((u8*) &pdev->bus) = bus;
>>>       *((u8*) &pdev->devfn) = devfn;
>>>       pdev->domain = NULL;
>>> +#ifdef CONFIG_ARM
>>> +    pci_to_dev(pdev)->type = DEV_PCI;
>>> +#endif
>> I have to admit that I'm not happy about new CONFIG_<arch> conditionals
>> here. I'd prefer to see this done by a new arch helper, unless there are
>> obstacles I'm overlooking.
> 
> Do you mean something like arch_pci_alloc_pdev(dev)?

I'd recommend against "alloc" in its name; "new" instead maybe?

> If so, where should we put this call? At the very beginning of alloc_pdev
> or at the bottom just before returning from alloc_pdev?

Right where you have the #ifdef right now, I would say (separated by
a blank line).

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:36:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:36:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196599.349508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUn3P-0004ut-4m; Mon, 27 Sep 2021 09:36:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196599.349508; Mon, 27 Sep 2021 09:36:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUn3P-0004um-14; Mon, 27 Sep 2021 09:36:07 +0000
Received: by outflank-mailman (input) for mailman id 196599;
 Mon, 27 Sep 2021 09:36:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUn3N-0004ug-JO
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:36:05 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2b642832-e845-424d-a1b0-e3c521a4dc3d;
 Mon, 27 Sep 2021 09:36:03 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18R9SnBp027392;
 Mon, 27 Sep 2021 09:36:00 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbbbv80t4-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 09:35:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6881.eurprd03.prod.outlook.com (2603:10a6:20b:286::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 09:35:57 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:35:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b642832-e845-424d-a1b0-e3c521a4dc3d
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aX8I/J6PVdXLbESHY0Fvd6iLkbYllXXvRUSP8GuBSBVKPwGe39WpYK9vaSBITPJf0BWO1ZyF7qt4KiJviqiOFi3jxju+ttFvOJdHxhey/L/DItTpPIomKmAvjhqRoKUBcxccfDPr/4NoxKuuEHjLfhx5tQm0MxKEdEsfUMPDFs00IbQh5yh/jU+ir+Vmlax5uiJBZtJmwk1RAeYJKyQ2qYnoLmU+Mf7cwrvup5dxXBXqu5wiBeJSGdeAJdzK32nRFNvA6nESE4/g2BkerqoiXYz/puaejIBloKC+dPlBCHi4f7VH9dw9p3vyfD0KEVL5R2j7Dal2E7wvGZdyi5eY3Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Gs2RrZQw62BoiJU6muXMOsnGwUgDw4TtrzVC3chvwr0=;
 b=capUWAQCEN+79BAkyeSOinKioDb8iyc7CNScrYjNFj7GQ/OnDVi/Q9TLMsivinTv6mVTLPWFE/r2QYLWLilDTJJ76dUb7Ns1Mt05T5e+KCuTxCNB/6ZmjeqbrSF38EOnijekop58fUwjfCCxEQw0MouDGl+o5PYvq6WBcboaYQ+8HZrQ39/asc1LEBVeHR5qC/r5ALswrnBJIsyiWE5+ErccIxr1FFk1WMGlCCkyoXADfPeevgTsdOdc8pIVISpxKICOt4poTLzHJaAhXEhn+Mf6s4QVzsJH2MYc1AbF5UBONZNd5KqyNTgsH+ZyuFD8i4xqGBswkw7KBD+MH5+GMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Gs2RrZQw62BoiJU6muXMOsnGwUgDw4TtrzVC3chvwr0=;
 b=J5sC+PJB3ryDHnlsTSSEb4A+UK/NMOYX5AzhxeMsYLfP2VLAqX38iMk/x3PLHP6x+pouWFu8gfh1CDvtmEdAMuLJbHtzH/FM+LZv60/sfhS8PTqmMwH7GyCD0siKocpMcopHHL08jre27PjtQ6lrfCS/JnpruWGGJBVmtGwB34T8HB2PnLcWTH1exV0wjd8R0WfcKteh7F0JrG5Zv2p+yMonz1dXhw9dWiVQfoysU5oyLuQucfqKc79CHyk/H2H/kcEjTEEreDCv848EtCB9wRPG2fWatZOj+h3yVvtqNz0WIx4uEc2p/qnl8DqsM2ZwYWNl07RPytby/p7tiOIi+g==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrICAAAldgIAABLUA
Date: Mon, 27 Sep 2021 09:35:57 +0000
Message-ID: <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
In-Reply-To: <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4ae5f236-e73b-45fc-b3f4-08d9819a3593
x-ms-traffictypediagnostic: AM9PR03MB6881:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB688156CFB0B119DFCDCC80C1E7A79@AM9PR03MB6881.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 7b4l0kO97BzaoPvW5RQWQOXuG1HszzkcmtiwHKhu2EUF43eGinMIW36zeh+SdE1cwecYRXd9vkagJ02sR+Wmhoggj4pvIhl5Jn3xYIhdryfLlwj133d1sw8EW6UWkJl9EwksoXq/aBSgDEetKfPO81gi7h46B748N/hs916cgXj86ZDIQ47rzeWm2Imw5dIe/qae3Z/CfZ/dx91hN0UIZo+pfaRuXqU75j+dfD7I8fks4DrBnWWz1zf3m/rYmbG9U1/ZZ4iGOy1EmkGuRkeltQDSmHTimu6FE8LufPJgCtnptYRFObZ0LZ/HwoSY4Sr4m5BGo3z2Sy9/Wh0YABxQgnd3Pa1D6qIyxBL+a5EG7GjH994uiP36jkIaomVXfBxpxHjR/YcVb6AKSiVir59qqbUdGw0zxXYeHNOuo21XGatGQQToXst9izrb+bWXCzittANZtA+DzYXcni6/kjmQpsyHWhNqzE6InmQdUknMVUOgzYk0Os3VurwbgrMrkShl/gXh0Bj6FmKa3f2wtaghpTAk8j3Jy2Nc8o2Rd2EJ4RFflIx4ZP7um7QiMrQhY8kWmjLgdAIiG53hR5lWaggljR3UKBucbi1wY8X4ndRtwlhYaX+v/5fcnrGco3nUujnii352rV/l9A0nFtHDdTisYQJlbSKFdauLGncFdywAe/O3//CMoTPv4wROq2nGf4EQqZit77rgba6n1py2m/JU6bl42wG6nZ5rA9whaoQVjWqp2x7J1yfCerADEY4VawY0
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31696002)(38100700002)(6512007)(55236004)(53546011)(54906003)(6506007)(508600001)(316002)(66946007)(4326008)(26005)(64756008)(6916009)(66446008)(2906002)(83380400001)(36756003)(66556008)(31686004)(107886003)(38070700005)(66476007)(8676002)(71200400001)(76116006)(2616005)(91956017)(186003)(122000001)(86362001)(5660300002)(8936002)(6486002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?RzdVZXN2djZJSmRnRkxkajFXWjJ2NmNKdmhhYWVUVytXMnNtLzB2VC9mcjJJ?=
 =?utf-8?B?NUViYlgwUkdZcHZQa0xxZUFOdkF6SnVwNmdsWG40MS9ZS3NOQlFNRGFrcDFI?=
 =?utf-8?B?a2xEK1VlSHk2ZE1JZlBJTWdSbW11UHRjd0JhQkd3SE9VQ3p3U0QyekU5SUVs?=
 =?utf-8?B?VXdyS2p6YURUOGI3T1lRUGRTeHh1bTRKcE9XK1pDdkVPaUtxWGp6ejdZR2JX?=
 =?utf-8?B?RTJrV3k4STh6bkV3c1lwNS8rTnhWck56djdxWUhKbWNZZWpSYjNXUTl1bHA2?=
 =?utf-8?B?UXpNNlRVbXZ5SXhVQ0RRWXdqU0U3RXh3VHdybGZWWndZS3ZIUlRVRUNXUTZs?=
 =?utf-8?B?NTdtZlhPV1pQbEczUEQybTdYb3FwMmgreEt0S1pnWklDbmI5ekpXOHVUYXRM?=
 =?utf-8?B?ZGU1RDM5SzZxQjhmWkZjZnFSUGU4K3lPQUErS0o1RFRqNEovVnY1QlIwVEV5?=
 =?utf-8?B?Y3NkRzJuNG14d1orRUJPdE1WaERpT3dRMXoxd1RMQ1Q0SDFkaXh4WnJONTNu?=
 =?utf-8?B?NjlsVllIeHZnQzVlS3ZCY0ZJY0FUWkd2RWpWRnVTYzdKb2FBMkx1MTgzMFJ5?=
 =?utf-8?B?akU2bm1XS1A3TTM2dlVlNjNVOFlLeFF5TmZBTjNlYmJnRWZpdnRlejdZM1FG?=
 =?utf-8?B?bmZMdGI0WU1xZXhGRG56cXpiVGE2dDlFbUdjVVUyUnhBWC9ZWGNYUnRId1NZ?=
 =?utf-8?B?UENGUlhVZnY4bDJ2aW5SYWpuUXBoblBDWnBWVzlnaVAweHh2R1A1M2dMR282?=
 =?utf-8?B?UlpBMDAzc2hnM2tnL2Q3dHpXRjBKU0pWWStxaVZiQVd3QlhTbTRhOWpKcmto?=
 =?utf-8?B?N2VsdVZDYitXT3kzRmxPUTNUd2pZK0tEdGttcnRUbC9XNmdQT3JCVlVxQkhD?=
 =?utf-8?B?K2VFVE1HQ1ZtY1U5OVVlajk4RGdqMnJMRVk1OTVqVVZ2N1c3ZHo1RGt1VzI4?=
 =?utf-8?B?cVlrSGhaMG1scDRjOWZSTm5ZYW1LR0lLMC90MHBCQVl1WXUvU1QyQ0ltZ241?=
 =?utf-8?B?SFdzSzhRaDdqaUFVVmVWS2NjeVBQU0VQOGl0QVhhM1JVTHdMeDRRMkoxcksz?=
 =?utf-8?B?eGJlbmRiK0JYL1JNUDUvdkFaaWNIcW1qRUpXUVl2UVNvem9DeTFRZVQwOENC?=
 =?utf-8?B?ZG1adUJOYk1mV0lGU3Zvd3pVU2dIOU9OVW4xcTdnWlgxR2JlUFFHYysvc293?=
 =?utf-8?B?VGg0VDQrdFoyM3pmd3Y2d09TTDRhK2UyRStFWlpkM3kwSU1OSTk3VGg1QXFN?=
 =?utf-8?B?V0VQSVN4Nit2VkVOS0c2NnM3dDR5NGNad3A2ZmJsQkJ4MExyVk1QOVU1bUV4?=
 =?utf-8?B?blMrTFQ4cnZaRC9IOFNNZTJja01EbTVIWWEwOUVFMnBUU2hIaDl1eFlIdWlZ?=
 =?utf-8?B?NVg1b3QzUE9xeVRvbVNzdkdQblViUUxKMVBKaVFXUit3a21rUDFqNWlCb0Fr?=
 =?utf-8?B?U1VUUDRoaFNzQTZGWnd0T1FuWmJpYlRrSGdObUJvOTh2aTh0QlpaSmlkM1h0?=
 =?utf-8?B?Mnd0RzlMWUx5UHhVc08wOGJQeURWMjdUMmc2UEhsQmpqY1NXS0ZkcUFZZlB2?=
 =?utf-8?B?RDJjREhaZmVUWFowdllQV2YvSVRZSXZjZ1FqbnJUV0NzMDA2ajM1SHBQR0Vv?=
 =?utf-8?B?MGU0ZEdwWCsxTDA1SkREYi9FMzkrdzJQTklQaUdtWHlhTW4zaWpWa214TFo4?=
 =?utf-8?B?RGdxSElaNnNZODZ1Vis3WEc2bnVqZmZtQWJWWXZPN2xSQndaVWhxV3Ztd3B6?=
 =?utf-8?B?eEFyU0pVNnNvcDErUHArcEd0Y2tyV05ma0wzRFdDUTNMSnc4eVJ6djg2UXlJ?=
 =?utf-8?B?T2dpalZBZU5McFVpUnEwZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <601BA19700D6074FB288583A66A12386@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4ae5f236-e73b-45fc-b3f4-08d9819a3593
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 09:35:57.2250
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: T+4emmEbaeCeCZzD6W/vGdUdTXIgnB8TjIqDvJClAUDjeTcJI5f445WytfPnXSKHOkrf7XM40Gm30JEy1GfYq1gq0LnA3Eph55syL7am29ZGiICfzUSkeJ27BEXLy9c1
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6881
X-Proofpoint-GUID: SfQMmFDMH63LHz5VjRPIg8AIspt2-63H
X-Proofpoint-ORIG-GUID: SfQMmFDMH63LHz5VjRPIg8AIspt2-63H
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_03,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0
 adultscore=0 mlxlogscore=895 priorityscore=1501 suspectscore=0 spamscore=0
 clxscore=1015 mlxscore=0 malwarescore=0 phishscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109270066

DQpPbiAyNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTA6NDUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTA6
NDUsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTQsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3Vn
aC9wY2kuYw0KPj4+PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+PiBA
QCAtMzI4LDYgKzMyOCw5IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2RldiAqYWxsb2NfcGRldihzdHJ1
Y3QgcGNpX3NlZyAqcHNlZywgdTggYnVzLCB1OCBkZXZmbikNCj4+Pj4gICAgICAgICooKHU4Kikg
JnBkZXYtPmJ1cykgPSBidXM7DQo+Pj4+ICAgICAgICAqKCh1OCopICZwZGV2LT5kZXZmbikgPSBk
ZXZmbjsNCj4+Pj4gICAgICAgIHBkZXYtPmRvbWFpbiA9IE5VTEw7DQo+Pj4+ICsjaWZkZWYgQ09O
RklHX0FSTQ0KPj4+PiArICAgIHBjaV90b19kZXYocGRldiktPnR5cGUgPSBERVZfUENJOw0KPj4+
PiArI2VuZGlmDQo+Pj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQgSSdtIG5vdCBoYXBweSBhYm91dCBu
ZXcgQ09ORklHXzxhcmNoPiBjb25kaXRpb25hbHMNCj4+PiBoZXJlLiBJJ2QgcHJlZmVyIHRvIHNl
ZSB0aGlzIGRvbmUgYnkgYSBuZXcgYXJjaCBoZWxwZXIsIHVubGVzcyB0aGVyZSBhcmUNCj4+PiBv
YnN0YWNsZXMgSSdtIG92ZXJsb29raW5nLg0KPj4gRG8geW91IG1lYW4gc29tZXRoaW5nIGxpa2Ug
YXJjaF9wY2lfYWxsb2NfcGRldihkZXYpPw0KPiBJJ2QgcmVjb21tZW5kIGFnYWluc3QgImFsbG9j
IiBpbiBpdHMgbmFtZTsgIm5ldyIgaW5zdGVhZCBtYXliZT8NCg0KSSBhbSBmaW5lIHdpdGggYXJj
aF9wY2lfbmV3X3BkZXYsIGJ1dCBhcmNoIHByZWZpeCBwb2ludHMgdG8gdGhlIGZhY3QgdGhhdA0K
DQp0aGlzIGlzIGp1c3QgYW4gYXJjaGl0ZWN0dXJlIHNwZWNpZmljIHBhcnQgb2YgdGhlIHBkZXYg
YWxsb2NhdGlvbiByYXRoZXIgdGhhbg0KDQphY3R1YWwgcGRldiBhbGxvY2F0aW9uIGl0c2VsZiwg
c28gd2l0aCB0aGlzIHJlc3BlY3QgYXJjaF9wY2lfYWxsb2NfcGRldiBzZWVtcw0KDQptb3JlIG5h
dHVyYWwgdG8gbWUuDQoNCj4NCj4+IElmIHNvLCB3aGVyZSBzaG91bGQgd2UgcHV0IHRoaXMgY2Fs
bD8gQXQgdGhlIHZlcnkgYmVnaW5uaW5nIG9mIGFsbG9jX3BkZXYNCj4+IG9yIGF0IHRoZSBib3R0
b20ganVzdCBiZWZvcmUgcmV0dXJuaW5nIGZyb20gYWxsb2NfcGRldj8NCj4gUmlnaHQgd2hlcmUg
eW91IGhhdmUgdGhlICNpZmRlZiByaWdodCBub3csIEkgd291bGQgc2F5IChzZXBhcmF0ZWQgYnkN
Cj4gYSBibGFuayBsaW5lKS4NCk9rDQo+DQo+IEphbg0KPg0KPg0KVGhhbmsgeW91LA0KDQpPbGVr
c2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:47:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:47:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196610.349519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnEQ-0006rV-8U; Mon, 27 Sep 2021 09:47:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196610.349519; Mon, 27 Sep 2021 09:47:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnEQ-0006rO-4l; Mon, 27 Sep 2021 09:47:30 +0000
Received: by outflank-mailman (input) for mailman id 196610;
 Mon, 27 Sep 2021 09:47:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ztgo=OR=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mUnEP-0006rI-O9
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:47:29 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ecff94cd-1f77-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 09:47:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecff94cd-1f77-11ec-bc43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632736048;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=pe2Yv5qXjCbIqx+PwmhV7IzjnJZEO3e5PooDSPv7Se0=;
  b=MVDkrgFau0XkHOB6b5/aZeeBN/OOA8GEkCMuhdl6Ge9IpclMvm3wHVvr
   kDKs6WNtQ2QlDMpKYRAEJ2PC/q5NsJ50ifrgPmgQIQ0/bp+bxC53sLXqU
   sntp/O0hbIkPSKk1drXdEAx0Ki0YTVXcCW2WfTKsQr/vBV3u6D/eRDrn8
   Q=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: EGGhILMTlmtiBVsJnK/TGDqlGDuPnfyZHCJnQwjCChIZobt577AuqjWQkSANQgEJIDVoTsRHDn
 tHxhwRuliqPEM00ofNyZYsl28tgVGacgeQQz5q8tqEooia2h4EqztTqHX8YJTI2bK1F70oNkoL
 tkulHGRXWGuwnjrUpCCzOmlngAFrnzCYhj3HieVPJ38PDBgsedFWReVWq1S/W9aPbQgRxtjlYT
 9YG5KjO4hPACGrHMaov1F30HKKOTiInAJtCbcGe2zPcJmN7Cq10AyQKFPgDUDdhRn3s74BkouZ
 mg1nYosIwexmKYBHwg7zOS1e
X-SBRS: 5.1
X-MesageID: 53706513
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:XxP0K6idXHXRunnhnhxzRr5LX1614hcKZh0ujC45NGQN5FlHY01je
 htvWjqGMvqPYGXzc41xOd+39koEu8OAnIQ3HgpqrykxQ34b9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y3oDga++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /0QuJW3eSwjBpHU2/8MQyAJSQtdYY9JreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHiOp8fvXdxiynUF/88TbjIQrnQ5M8e1zA17ixLNauFO
 JJHNGM0BPjGSzoTJ3JJCos9oPuPqCTkQhFahUOrpbVitgA/yyQuieOwYbI5YOeiXt5Jl0yVo
 mbH+WXRARwAMtGbjz2f/RqEmevnjS79HoUIG9WQ5vNsxVGe2GEXIBkXTkeg5+m0jFakXNBSI
 FBS/TAhxYAw/kG2Stj2XzWjvWWJ+BUbXrJ4EvI4wBGAzLLO5ASUDXRCSSROAPQkqcs3SDoCx
 lKP2dTzClRSXKa9ECzHsO3O9HXrZHZTfTRqiTI4oRUtvtTqp7EYhCrzVpV4Ab6M34ffOm3Sz
 GXfxMQhvIk7gckO3qS92FnIhTOwu5TEJjIIChXrsnGNtF0gO9/8D2C8wR2Lta8fcNbxokyp5
 iBspiSI0AwZ4XhhfgSjR/8RVJWg+vqIKjHVhVMH83IJrG/2oy/LkWy9+lhDyKZV3iQsJWGBj
 Kz741o5CHpv0J2CNvYfj2WZUZhC8EQYPY65Ps04l/IXCnSLSCeJ/Tt1eWmb1H33nU4nnMkXY
 MnAK5j8VCpLV/Q9llJaotvxN5dwn0jSIkuJGfjGI+mPi+LCNBZ5t59fWLdxUgzJxPzd+1iEm
 zquH8CL1w9eQIXDjtr/q+YuwaQxBSFjX/je8pUPHsbae1YOMDxxWpf5nOJ6E6Q4zvs9qws91
 izkMqOu4AGk3iOvxMTjQi0LVY4Dqr4l9i9kYXxwYwz4s5XhCK72hJoim1IMVeFP3IReITRcE
 pHpou2MXaZCTCrp4TMYYcWvpYBubk3z1wmPIzCkcH40eJs5H17F/drtfw3O8igSD3Xo6Zti8
 uP4jg6LE4AeQwlCDdrNbK79xV2GonVAyvl5WFHFI4cPdRy0opRqMSH4ktQ+P9oIdUfY3jKf2
 gvPWUUYqODBrpUb6t7MgazY/Y6lH/EnRhhRHnXB7KbwPi7fpzLxzYhFWeeOXDbcSGKrp/nyO
 bQLl6nxaaRVkkxLvoxwF6dQ4Zg/v9a/9aVHyglEHWnQawj5AL1XPXTbj9JEsbdAx+EFtFLuC
 F6P4NRTJZ6AJNjhTAwKPAMgY+mOiaMUlz3V4ahnKUn2/nYqrr+OUEEUNBiQki1NarByNdp9k
 +smvccX7S25iwYrbYna3nwFqTzUIyxSSbgju7EbHJTv21gixVx1aJDBDjP7vcOUYNJWP0h2e
 jKZicIuXViHKpYup5brKUXw4A==
IronPort-HdrOrdr: A9a23:LbFiKamRWS5VsOI2Nkcxs5n/tLjpDfIu3DAbv31ZSRFFG/Fxl6
 iV/cjz8SWE7wr5OUtQ/exoV5PtfZqxz/FICMwqTNGftWrdyQ6VxeNZnOjfKlTbckWUnINgPO
 VbAspD4bXLfCFHZK3BgDVQfexP/OW6
X-IronPort-AV: E=Sophos;i="5.85,326,1624334400"; 
   d="scan'208";a="53706513"
Date: Mon, 27 Sep 2021 10:46:28 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v3] xen: rework `checkpolicy` detection when using
 "randconfig"
Message-ID: <YVGS9K1XqNpyxAxe@perard>
References: <20210908111727.440265-1-anthony.perard@citrix.com>
 <bb40484f-fbfb-a206-fa98-0e927f5398fa@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <bb40484f-fbfb-a206-fa98-0e927f5398fa@suse.com>

On Thu, Sep 16, 2021 at 05:34:00PM +0200, Jan Beulich wrote:
> On 08.09.2021 13:17, Anthony PERARD wrote:
> > --- a/Config.mk
> > +++ b/Config.mk
> > @@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=y
> >  build_id_linker := --build-id=sha1
> >  endif
> >  
> > -ifndef XEN_HAS_CHECKPOLICY
> > -    CHECKPOLICY ?= checkpolicy
> > -    XEN_HAS_CHECKPOLICY := $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xen && echo y || echo n)
> > -    export XEN_HAS_CHECKPOLICY
> > -endif
> 
> Is there a particular reason to go from XEN_HAS_CHECKPOLICY to ...
> 
> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?= $(shell hostname)
> >  PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
> >  export PYTHON		?= $(PYTHON_INTERPRETER)
> >  
> > +export CHECKPOLICY	?= checkpolicy
> > +
> >  export BASEDIR := $(CURDIR)
> >  export XEN_ROOT := $(BASEDIR)/..
> >  
> > @@ -178,6 +180,8 @@ CFLAGS += $(CLANG_FLAGS)
> >  export CLANG_FLAGS
> >  endif
> >  
> > +export HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)
> 
> ... HAS_CHECKPOLICY? As soon as things get put in the environment,

Not really anymore, it's just left over from having put this in Kconfig
in previous version of the patch.

> I'm always suspecting possible name collisions ...

Yes, it's probably better to keep the XEN_ prefix.

> > @@ -189,14 +193,24 @@ ifeq ($(config-build),y)
> >  # *config targets only - make sure prerequisites are updated, and descend
> >  # in tools/kconfig to make the *config target
> >  
> > +# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
> > +# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
> > +filechk_kconfig_allconfig = \
> > +    $(if $(findstring n,$(HAS_CHECKPOLICY)),echo 'CONFIG_XSM_FLASK_POLICY=n';) \
> > +    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG), :)
> 
> Nit: It would be nice if you were consistent with the blanks after
> commas in $(if ...). Personally I'm also considering $(if ...)s the
> more difficult to follow the longer they are. Hence for the 2nd one
> I wonder whether
> 
>     $(if $(KCONFIG_ALLCONFIG),cat,:) $(KCONFIG_ALLCONFIG)
> 
> wouldn't be easier to read.

How about:

    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
    :

.. instead, as that would be more consistent with the previous line,
that is there would be only one branch to the $(if ) and no else, and
thus probably easier to read.

> > +.allconfig.tmp: FORCE
> > +	set -e; { $(call filechk_kconfig_allconfig); } > $@
> 
> Is there a particular reason for the .tmp suffix?

Yes, .*.tmp are already ignored via .gitignore.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 09:51:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 09:51:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196616.349530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnHw-0008Es-P2; Mon, 27 Sep 2021 09:51:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196616.349530; Mon, 27 Sep 2021 09:51:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnHw-0008El-Kw; Mon, 27 Sep 2021 09:51:08 +0000
Received: by outflank-mailman (input) for mailman id 196616;
 Mon, 27 Sep 2021 09:51:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUnHv-0008Ef-PK
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 09:51:07 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.87]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6e5ef4cc-1f78-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 09:51:05 +0000 (UTC)
Received: from AM6P192CA0008.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::21)
 by DBBPR08MB4394.eurprd08.prod.outlook.com (2603:10a6:10:d0::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 09:51:02 +0000
Received: from VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:83:cafe::72) by AM6P192CA0008.outlook.office365.com
 (2603:10a6:209:83::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 09:51:02 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT016.mail.protection.outlook.com (10.152.18.115) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 09:51:01 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Mon, 27 Sep 2021 09:50:59 +0000
Received: from cea30a26d627.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 BC1E8423-5779-4D69-8598-F669072250BE.1; 
 Mon, 27 Sep 2021 09:50:50 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cea30a26d627.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 09:50:50 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB9PR08MB6715.eurprd08.prod.outlook.com (2603:10a6:10:26f::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 09:50:46 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 09:50:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e5ef4cc-1f78-11ec-bc43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=R3/x3sqtUyU2wimZJo1Jllq4DKTHS4GZHQksye07BUI=;
 b=U+J7RQTl6sDfNf5JtGVRD5KvrXVxqMSHT41P9nsm2cINcTWtWgveAEfuiivPsqFSVVRnN3nnFx6UiL8e7bk49I3X9zxhhe7io0yIpB6JSTb9xLZ1rOe5zj8i4BgKeuLLRGYFVjQnEWE7jKfwByrClhpD1aQyOg1JHqvZoY7fLl4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SxM5z7AoHmJKR+wmQQsQKOkimZh51P1onYPjXCswEvC+6JWRG4pziIxFl+RIbXLJtJlVJgMGWrDk/DDnqUXaQ/0HGbz0Zr79XjIoKoqkXdcAtyaafeyhartaSiFlMh/p07IzTJSCD3UbyM8a6sP3280EgFhluWneND/yGs7K1iJuNFZOEr2bdiaodDkyXNQ5bawHZouM6LMukvzH1s6Jcxz717yNupTflJsQ9jb8theeY46tRN1ko43/T5ewDRfELYAAX5a+7InnyU5NF8NfohhwGPH006YU2vP3mWRr9p+Yg1W/MjSm9/jAsuaNOQmZDat3fzy3OJ16jYb7rEGNlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R3/x3sqtUyU2wimZJo1Jllq4DKTHS4GZHQksye07BUI=;
 b=UjRy6gz0W24j38w8ixQtt4rbPP3yvYLaF25an2emgE6vcMV9gIC65NGvCtbx27gV7PMpPuhMA/AhAxk5QCJcCTUK3QRwBwoXItow95OT5EneIVuAfwBZJuJ6jdMteGg+BH4gVy+M7z6NXNK6r5bIiLhKPL9i3JHcj+/yEzpYaz8/4mvW7WFHTgw89CURQqmT45H9rBPwHASn+zBJfIx6cx+64qMpPhReemlaUgafN8j7LAZaghb8cET1R0kfbW/N7j2MP+rLdXvG3RVh/0oH2EfdKMu9MdiI94z/c0mtYNV6BIAz5f1dghD4SWdstpNVBu+fzMwGSWPiM2xHfRmYZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=R3/x3sqtUyU2wimZJo1Jllq4DKTHS4GZHQksye07BUI=;
 b=U+J7RQTl6sDfNf5JtGVRD5KvrXVxqMSHT41P9nsm2cINcTWtWgveAEfuiivPsqFSVVRnN3nnFx6UiL8e7bk49I3X9zxhhe7io0yIpB6JSTb9xLZ1rOe5zj8i4BgKeuLLRGYFVjQnEWE7jKfwByrClhpD1aQyOg1JHqvZoY7fLl4=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Topic: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Index:
 AQHXsHMRZjZ8eZQrbkW8SEq2x0f00KuyVH0AgABDnECAAQJ+AIACfOAggAEi1ICAAB9HAIAAIXKw
Date: Mon, 27 Sep 2021 09:50:46 +0000
Message-ID:
 <DB9PR08MB685772C5CDE9DF885A063F479EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-9-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 7EBDD917EFE3414E9FEF5E7E02B172ED.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 82c771b3-ee02-4ddf-ae9f-08d9819c50cc
x-ms-traffictypediagnostic: DB9PR08MB6715:|DBBPR08MB4394:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB439452EE87290D0028B182A79EA79@DBBPR08MB4394.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 woavUwxf7hH6DukD/o/9RT1d7kNf1z4rR6Fi+yeSsD/mBpyWAl2+jexHZY01bKyM36GNVVPbmD5O0jLwY8pzPzWAmGwF5qNcFyNWBE6/GJtiC9/FsEqW0i4TL3FBETuVMedQt8rHXrRIHky4f8y8o7i73TOzfnFqU/LG0fWqF8eBI3y12j3o1CY0kyRyzwUS8z0kXZCkLbRX9GO1Bfl1lryWZotYBGnzSIepiO9+E/HFp8W40dxx+aYJwZl4KTGVVyjCkDL+kIyg4inj9vZbLzDcLRBmprrRCVVuBggn2TJ7DKEqOpav4oMslOw3T5j2C2jBzC6kgT9mj5IsDIb+Aqr4XKKpZYDjz3ag1Tg/W3fPTtL2cSLG2We3uYjsIov1CfV9E4GLDwnyXfaG39MoP/+IBzh/qwbtuib6eK6L9H8ofM02q6R7MZCue44L5kWi4jOmV/k/DEJxlDe5AuPc4EyocK3yNQ0bPM58Ii6RDtJKGljHytagc5Kh2wv7IZ0L6L+NbIr74RJf/0AQ1dTE943ROaEPvlJrWa+jeYqM4QfAiKF3c6cjtyt/ph9i8DL+hjW5mOSjHKzzDoDXaSp8dcqtI+DRNRKAX0oP8blWrvMUXfX7JrSD6JceyrChOLe5JY3Ymgj8BBCmq0aB1xC4xOxUtlc6uSHMfZaX17AvmaO0GBbo8pRsMSG8bTy80rmwGvDdr7fSIF3vgJVd5V1UEQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(9686003)(83380400001)(86362001)(66946007)(76116006)(52536014)(71200400001)(5660300002)(26005)(316002)(186003)(508600001)(55016002)(8936002)(54906003)(8676002)(38100700002)(66476007)(7696005)(66556008)(6506007)(4326008)(64756008)(53546011)(66446008)(122000001)(38070700005)(33656002)(2906002)(6916009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6715
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	45fd03b3-464a-4652-08c1-08d9819c4766
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eRNBIt2LTkqvYahjaJoaHJwofz+uOQ2sBi6/ybbyK07UeOhteaWEPNy7orcABNT+HWuvF9DvuOA5l833alyra0MvBNGYbyPPPvUSxaRdOoSRPpU/O5dt+N+SETfRRE2oFjo/QnBSlPHfhjgHn9EH49VD6IaDXof+XUlALDrHwysjrBYvqBO0D/dCr2DTYct0T9XYz3EoJOrnHERJZeRHMdVwptkUa50PgksAQo2lCfFm000REd9cF/7kjeBQfmJ9ZL5wRxQofHvtWs8810oq+NssL57/aLG6OEfZmHJoEW2dcTC2NFXaL2kA6p8rsdOOAiGHhywnnJMGJszqxtRzwtHxQuaVfs/zoG1J+DL1cT53Mo1rJXCkpwesIGH9U3FEwwiWUEoOrOg56QUe87PPn/P/2eVtO5ZvxZF8O6WFD1ci8IZmkGgO22dvWeaUBd6mNotbLRz2Gk1H+1o7HUp0a0ZcBASGbps4vXMlJ51QJRNVw1vMAcKQTQL07nUAPfKRU1RCkycWeRLJ0biiKYZO117obuzqFG/pzNGqUiRjs5CGWqHvJQUX3d6XYaufqEdGHtl+K3/TTpxHxVmFZ+HpI3RvDWG2/auM9G/mFMhXo7o1uVhb/Q4tfD7UEmeByAI3WlWVpBUdR6qfqq3t1Ep+07qxlT9UDJbS5QxsPYMds+LiRvJu6fmxaems4uf7hzjBz1Ni2p3x7GsTUOWpggpQtA==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(508600001)(6862004)(83380400001)(70206006)(70586007)(4326008)(82310400003)(6506007)(81166007)(86362001)(53546011)(7696005)(8676002)(9686003)(36860700001)(186003)(47076005)(356005)(33656002)(316002)(55016002)(52536014)(8936002)(2906002)(54906003)(26005)(5660300002)(336012);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 09:51:01.8710
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 82c771b3-ee02-4ddf-ae9f-08d9819c50cc
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4394

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMjfml6UgMTM6MDUNCj4gVG86IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2Vy
bmVsLm9yZz4NCj4gQ2M6IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPjsgeGVuLWRldmVsQGxp
c3RzLnhlbnByb2plY3Qub3JnOw0KPiBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcyA8
QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsNCj4gamJldWxpY2hAc3VzZS5jb207IGFuZHJldy5j
b29wZXIzQGNpdHJpeC5jb207IHJvZ2VyLnBhdUBjaXRyaXguY29tOw0KPiB3bEB4ZW4ub3JnDQo+
IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMDgvMzddIHhlbi94ODY6IGFkZCBkZXRlY3Rpb24gb2YgZGlz
Y29udGlub3VzIG5vZGUNCj4gbWVtb3J5IHJhbmdlDQo+IA0KPiBPbiBTdW4sIDI2IFNlcCAyMDIx
LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+ID4gT24gU3VuLCAyNiBTZXAgMjAyMSwgV2Vp
IENoZW4gd3JvdGU6DQo+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPiA+
IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiA+
ID4gU2VudDogMjAyMeW5tDnmnIgyNeaXpSAzOjUzDQo+ID4gPiA+IFRvOiBXZWkgQ2hlbiA8V2Vp
LkNoZW5AYXJtLmNvbT4NCj4gPiA+ID4gQ2M6IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxp
bmlAa2VybmVsLm9yZz47IHhlbi0NCj4gPiA+ID4gZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7
IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4gPiA+IDxCZXJ0cmFuZC5NYXJx
dWlzQGFybS5jb20+OyBqYmV1bGljaEBzdXNlLmNvbTsNCj4gYW5kcmV3LmNvb3BlcjNAY2l0cml4
LmNvbTsNCj4gPiA+ID4gcm9nZXIucGF1QGNpdHJpeC5jb207IHdsQHhlbi5vcmcNCj4gPiA+ID4g
U3ViamVjdDogUkU6IFtQQVRDSCAwOC8zN10geGVuL3g4NjogYWRkIGRldGVjdGlvbiBvZiBkaXNj
b250aW5vdXMNCj4gbm9kZQ0KPiA+ID4gPiBtZW1vcnkgcmFuZ2UNCj4gPiA+ID4NCj4gPiA+ID4g
T24gRnJpLCAyNCBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiA+ID4gPiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4gPiA+ID4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5p
IDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4gPiA+ID4gU2VudDogMjAyMeW5tDnmnIgy
NOaXpSA4OjI2DQo+ID4gPiA+ID4gPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+
ID4gPiA+ID4gPiBDYzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBzc3RhYmVsbGlu
aUBrZXJuZWwub3JnOw0KPiA+ID4gPiBqdWxpZW5AeGVuLm9yZzsNCj4gPiA+ID4gPiA+IEJlcnRy
YW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IGpiZXVsaWNoQHN1c2UuY29t
Ow0KPiA+ID4gPiA+ID4gYW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbTsgcm9nZXIucGF1QGNpdHJp
eC5jb207IHdsQHhlbi5vcmcNCj4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMDgvMzdd
IHhlbi94ODY6IGFkZCBkZXRlY3Rpb24gb2YNCj4gZGlzY29udGlub3VzIG5vZGUNCj4gPiA+ID4g
PiA+IG1lbW9yeSByYW5nZQ0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IENDJ2luZyB4ODYgbWFp
bnRhaW5lcnMNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBX
ZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPiA+ID4gT25lIE5VTUEgbm9kZSBtYXkgY29udGFpbiBz
ZXZlcmFsIG1lbW9yeSBibG9ja3MuIEluIGN1cnJlbnQNCj4gWGVuDQo+ID4gPiA+ID4gPiA+IGNv
ZGUsIFhlbiB3aWxsIG1haW50YWluIGEgbm9kZSBtZW1vcnkgcmFuZ2UgZm9yIGVhY2ggbm9kZSB0
bw0KPiBjb3Zlcg0KPiA+ID4gPiA+ID4gPiBhbGwgaXRzIG1lbW9yeSBibG9ja3MuIEJ1dCBoZXJl
IGNvbWVzIHRoZSBwcm9ibGVtLCBpbiB0aGUgZ2FwDQo+IG9mDQo+ID4gPiA+ID4gPiA+IG9uZSBu
b2RlJ3MgdHdvIG1lbW9yeSBibG9ja3MsIGlmIHRoZXJlIGFyZSBzb21lIG1lbW9yeSBibG9ja3MN
Cj4gZG9uJ3QNCj4gPiA+ID4gPiA+ID4gYmVsb25nIHRvIHRoaXMgbm9kZSAocmVtb3RlIG1lbW9y
eSBibG9ja3MpLiBUaGlzIG5vZGUncyBtZW1vcnkNCj4gcmFuZ2UNCj4gPiA+ID4gPiA+ID4gd2ls
bCBiZSBleHBhbmRlZCB0byBjb3ZlciB0aGVzZSByZW1vdGUgbWVtb3J5IGJsb2Nrcy4NCj4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gT25lIG5vZGUncyBtZW1vcnkgcmFuZ2UgY29udGFpbnMg
b3RoZSBub2RlcycgbWVtb3J5LCB0aGlzIGlzDQo+ID4gPiA+IG9idmlvdXNseQ0KPiA+ID4gPiA+
ID4gPiBub3QgdmVyeSByZWFzb25hYmxlLiBUaGlzIG1lYW5zIGN1cnJlbnQgTlVNQSBjb2RlIG9u
bHkgY2FuDQo+IHN1cHBvcnQNCj4gPiA+ID4gPiA+ID4gbm9kZSBoYXMgY29udGlub3VzIG1lbW9y
eSBibG9ja3MuIEhvd2V2ZXIsIG9uIGEgcGh5c2ljYWwNCj4gbWFjaGluZSwNCj4gPiA+ID4gdGhl
DQo+ID4gPiA+ID4gPiA+IGFkZHJlc3NlcyBvZiBtdWx0aXBsZSBub2RlcyBjYW4gYmUgaW50ZXJs
ZWF2ZWQuDQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+IFNvIGluIHRoaXMgcGF0Y2gsIHdl
IGFkZCBjb2RlIHRvIGRldGVjdCBkaXNjb250aW5vdXMgbWVtb3J5DQo+IGJsb2Nrcw0KPiA+ID4g
PiA+ID4gPiBmb3Igb25lIG5vZGUuIE5VTUEgaW5pdGlhbGl6dGlvbiB3aWxsIGJlIGZhaWxlZCBh
bmQgZXJyb3INCj4gbWVzc2FnZXMNCj4gPiA+ID4gPiA+ID4gd2lsbCBiZSBwcmludGVkIHdoZW4g
WGVuIGRldGVjdCBzdWNoIGhhcmR3YXJlIGNvbmZpZ3VyYXRpb24uDQo+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gQXQgbGVhc3Qgb24gQVJNLCBpdCBpcyBub3QganVzdCBtZW1vcnkgdGhhdCBjYW4g
YmUgaW50ZXJsZWF2ZWQsDQo+IGJ1dA0KPiA+ID4gPiBhbHNvDQo+ID4gPiA+ID4gPiBNTUlPIHJl
Z2lvbnMuIEZvciBpbnN0YW5jZToNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBub2RlMCBiYW5r
MCAwLTB4MTAwMDAwMA0KPiA+ID4gPiA+ID4gTU1JTyAweDEwMDAwMDAtMHgxMDAyMDAwDQo+ID4g
PiA+ID4gPiBIb2xlIDB4MTAwMjAwMC0weDIwMDAwMDANCj4gPiA+ID4gPiA+IG5vZGUwIGJhbmsx
IDB4MjAwMDAwMC0weDMwMDAwMDANCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBTbyBJIGFtIG5v
dCBmYW1pbGlhciB3aXRoIHRoZSBTUkFUIGZvcm1hdCwgYnV0IEkgdGhpbmsgb24gQVJNDQo+IHRo
ZQ0KPiA+ID4gPiBjaGVjaw0KPiA+ID4gPiA+ID4gd291bGQgbG9vayBkaWZmZXJlbnQ6IHdlIHdv
dWxkIGp1c3QgbG9vayBmb3IgbXVsdGlwbGUgbWVtb3J5DQo+IHJhbmdlcw0KPiA+ID4gPiA+ID4g
dW5kZXIgYSBkZXZpY2VfdHlwZSA9ICJtZW1vcnkiIG5vZGUgb2YgYSBOVU1BIG5vZGUgaW4gZGV2
aWNlDQo+IHRyZWUuDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPg0KPiA+ID4g
PiA+IFNob3VsZCBJIG5lZWQgdG8gaW5jbHVkZS9yZWZpbmUgYWJvdmUgbWVzc2FnZSB0byBjb21t
aXQgbG9nPw0KPiA+ID4gPg0KPiA+ID4gPiBMZXQgbWUgYXNrIHlvdSBhIHF1ZXN0aW9uIGZpcnN0
Lg0KPiA+ID4gPg0KPiA+ID4gPiBXaXRoIHRoZSBOVU1BIGltcGxlbWVudGF0aW9uIG9mIHRoaXMg
cGF0Y2ggc2VyaWVzLCBjYW4gd2UgZGVhbCB3aXRoDQo+ID4gPiA+IGNhc2VzIHdoZXJlIGVhY2gg
bm9kZSBoYXMgbXVsdGlwbGUgbWVtb3J5IGJhbmtzLCBub3QgaW50ZXJsZWF2ZWQ/DQo+ID4gPg0K
PiA+ID4gWWVzLg0KPiA+ID4NCj4gPiA+ID4gQW4gYW4gZXhhbXBsZToNCj4gPiA+ID4NCj4gPiA+
ID4gbm9kZTA6IDB4MCAgICAgICAgLSAweDEwMDAwMDAwDQo+ID4gPiA+IE1NSU8gOiAweDEwMDAw
MDAwIC0gMHgyMDAwMDAwMA0KPiA+ID4gPiBub2RlMDogMHgyMDAwMDAwMCAtIDB4MzAwMDAwMDAN
Cj4gPiA+ID4gTU1JTyA6IDB4MzAwMDAwMDAgLSAweDUwMDAwMDAwDQo+ID4gPiA+IG5vZGUxOiAw
eDUwMDAwMDAwIC0gMHg2MDAwMDAwMA0KPiA+ID4gPiBNTUlPIDogMHg2MDAwMDAwMCAtIDB4ODAw
MDAwMDANCj4gPiA+ID4gbm9kZTI6IDB4ODAwMDAwMDAgLSAweDkwMDAwMDAwDQo+ID4gPiA+DQo+
ID4gPiA+DQo+ID4gPiA+IEkgYXNzdW1lIHdlIGNhbiBkZWFsIHdpdGggdGhpcyBjYXNlIHNpbXBs
eSBieSBzZXR0aW5nIG5vZGUwIG1lbW9yeQ0KPiB0bw0KPiA+ID4gPiAweDAtMHgzMDAwMDAwMCBl
dmVuIGlmIHRoZXJlIGlzIGFjdHVhbGx5IHNvbWV0aGluZyBlbHNlLCBhIGRldmljZSwNCj4gdGhh
dA0KPiA+ID4gPiBkb2Vzbid0IGJlbG9uZyB0byBub2RlMCBpbiBiZXR3ZWVuIHRoZSB0d28gbm9k
ZTAgYmFua3M/DQo+ID4gPg0KPiA+ID4gV2hpbGUgdGhpcyBjb25maWd1cmF0aW9uIGlzIHJhcmUg
aW4gU29DIGRlc2lnbiwgYnV0IGl0IGlzIG5vdA0KPiBpbXBvc3NpYmxlLg0KPiA+DQo+ID4gRGVm
aW5pdGVseSwgSSBoYXZlIHNlZW4gaXQgYmVmb3JlLg0KPiA+DQo+ID4NCj4gPiA+ID4gSXMgaXQg
b25seSBvdGhlciBub2RlcycgbWVtb3J5IGludGVybGVhdmVkIHRoYXQgY2F1c2UgaXNzdWVzPyBJ
bg0KPiBvdGhlcg0KPiA+ID4gPiB3b3Jkcywgb25seSB0aGUgZm9sbG93aW5nIGlzIGEgcHJvYmxl
bWF0aWMgc2NlbmFyaW8/DQo+ID4gPiA+DQo+ID4gPiA+IG5vZGUwOiAweDAgICAgICAgIC0gMHgx
MDAwMDAwMA0KPiA+ID4gPiBNTUlPIDogMHgxMDAwMDAwMCAtIDB4MjAwMDAwMDANCj4gPiA+ID4g
bm9kZTE6IDB4MjAwMDAwMDAgLSAweDMwMDAwMDAwDQo+ID4gPiA+IE1NSU8gOiAweDMwMDAwMDAw
IC0gMHg1MDAwMDAwMA0KPiA+ID4gPiBub2RlMDogMHg1MDAwMDAwMCAtIDB4NjAwMDAwMDANCj4g
PiA+ID4NCj4gPiA+ID4gQmVjYXVzZSBub2RlMSBpcyBpbiBiZXR3ZWVuIHRoZSB0d28gcmFuZ2Vz
IG9mIG5vZGUwPw0KPiA+ID4gPg0KPiA+ID4NCj4gPiA+IEJ1dCBvbmx5IGRldmljZV90eXBlPSJt
ZW1vcnkiIGNhbiBiZSBhZGRlZCB0byBhbGxvY2F0aW9uLg0KPiA+ID4gRm9yIG1taW8gdGhlcmUg
YXJlIHR3byBjYXNlczoNCj4gPiA+IDEuIG1taW8gZG9lc24ndCBoYXZlIE5VTUEgaWQgcHJvcGVy
dHkuDQo+ID4gPiAyLiBtbWlvIGhhcyBOVU1BIGlkIHByb3BlcnR5LCBqdXN0IGxpa2Ugc29tZSBQ
Q0llIGNvbnRyb2xsZXJzLg0KPiA+ID4gICAgQnV0IHdlIGRvbuKAmXQgbmVlZCB0byBoYW5kbGUg
dGhlc2Uga2luZHMgb2YgTU1JTyBkZXZpY2VzDQo+ID4gPiAgICBpbiBtZW1vcnkgYmxvY2sgcGFy
c2luZy4gQmVjYXVzZSB3ZSBkb24ndCBuZWVkIHRvIGFsbG9jYXRlDQo+ID4gPiAgICBtZW1vcnkg
ZnJvbSB0aGVzZSBtbWlvIHJhbmdlcy4gQW5kIGZvciBhY2Nlc3NpbmcsIHdlIG5lZWQNCj4gPiA+
ICAgIGEgTlVNQS1hd2FyZSBQQ0llIGNvbnRyb2xsZXIgZHJpdmVyIG9yIGEgZ2VuZXJpYyBOVU1B
LWF3YXJlDQo+ID4gPiAgICBNTUlPIGFjY2Vzc2luZyBBUElzLg0KPiA+DQo+ID4gWWVzLCBJIGFt
IG5vdCB0b28gd29ycmllZCBhYm91dCBkZXZpY2VzIHdpdGggYSBOVU1BIGlkIHByb3BlcnR5IGJl
Y2F1c2UNCj4gPiB0aGV5IGFyZSBsZXNzIGNvbW1vbiBhbmQgdGhpcyBzZXJpZXMgZG9lc24ndCBo
YW5kbGUgdGhlbSBhdCBhbGwsIHJpZ2h0Pw0KPiA+IEkgaW1hZ2luZSB0aGV5IHdvdWxkIGJlIHRy
ZWF0ZWQgbGlrZSBhbnkgb3RoZXIgZGV2aWNlIHdpdGhvdXQgTlVNQQ0KPiA+IGF3YXJlbmVzcy4N
Cj4gPg0KPiA+IEkgYW0gdGhpbmtpbmcgYWJvdXQgdGhlIGNhc2Ugd2hlcmUgdGhlIG1lbW9yeSBv
ZiBlYWNoIE5VTUEgbm9kZSBpcyBtYWRlDQo+ID4gb2YgbXVsdGlwbGUgYmFua3MuIEkgdW5kZXJz
dGFuZCB0aGF0IHRoaXMgcGF0Y2ggYWRkcyBhbiBleHBsaWNpdCBjaGVjaw0KPiA+IGZvciBjYXNl
cyB3aGVyZSB0aGVzZSBiYW5rcyBhcmUgaW50ZXJsZWF2aW5nLCBob3dldmVyIHRoZXJlIGFyZSBt
YW55DQo+ID4gb3RoZXIgY2FzZXMgd2hlcmUgTlVNQSBtZW1vcnkgbm9kZXMgYXJlICpub3QqIGlu
dGVybGVhdmluZyBidXQgdGhleSBhcmUNCj4gPiBzdGlsbCBtYWRlIG9mIG11bHRpcGxlIGRpc2Nv
bnRpbnVvdXMgYmFua3MsIGxpa2UgaW4gdGhlIHR3byBleGFtcGxlDQo+ID4gYWJvdmUuDQo+ID4N
Cj4gPiBNeSBxdWVzdGlvbiBpcyB3aGV0aGVyIHRoaXMgcGF0Y2ggc2VyaWVzIGluIGl0cyBjdXJy
ZW50IGZvcm0gY2FuIGhhbmRsZQ0KPiA+IHRoZSB0d28gY2FzZXMgYWJvdmUgY29ycmVjdGx5LiBJ
ZiBzbywgSSBhbSB3b25kZXJpbmcgaG93IGl0IHdvcmtzIGdpdmVuDQo+ID4gdGhhdCB3ZSBvbmx5
IGhhdmUgYSBzaW5nbGUgInN0YXJ0IiBhbmQgInNpemUiIHBhcmFtZXRlciBwZXIgbm9kZS4NCj4g
Pg0KPiA+IE9uIHRoZSBvdGhlciBoYW5kIGlmIHRoaXMgc2VyaWVzIGNhbm5vdCBoYW5kbGUgdGhl
IHR3byBjYXNlcyBhYm92ZSwgbXkNCj4gPiBxdWVzdGlvbiBpcyB3aGV0aGVyIGl0IHdvdWxkIGZh
aWwgZXhwbGljaXRseSBvciBub3QuIFRoZSBuZXcNCj4gPiBjaGVjayBpc19ub2RlX21lbW9yeV9j
b250aW51b3VzIGRvZXNuJ3Qgc2VlbSB0byBiZSBhYmxlIHRvIGNhdGNoIHRoZW0uDQo+IA0KPiAN
Cj4gTG9va2luZyBhdCBudW1hX3VwZGF0ZV9ub2RlX21lbWJsa3MsIGl0IGlzIGNsZWFyIHRoYXQg
dGhlIGNvZGUgaXMgbWVhbnQNCj4gdG8gaW5jcmVhc2UgdGhlIHJhbmdlIG9mIGVhY2ggbnVtYSBu
b2RlIHRvIGNvdmVyIGV2ZW4gTU1JTyByZWdpb25zIGluDQo+IGJldHdlZW4gbWVtb3J5IGJhbmtz
LiBBbHNvIHNlZSB0aGUgY29tbWVudCBhdCB0aGUgdG9wIG9mIHRoZSBmaWxlOg0KPiANCj4gICog
QXNzdW1lcyBhbGwgbWVtb3J5IHJlZ2lvbnMgYmVsb25naW5nIHRvIGEgc2luZ2xlIHByb3hpbWl0
eSBkb21haW4NCj4gICogYXJlIGluIG9uZSBjaHVuay4gSG9sZXMgYmV0d2VlbiB0aGVtIHdpbGwg
YmUgaW5jbHVkZWQgaW4gdGhlIG5vZGUuDQo+IA0KPiBTbyBpZiB0aGVyZSBhcmUgbXVsdGlwbGUg
YmFua3MgZm9yIGVhY2ggbm9kZSwgc3RhcnQgYW5kIGVuZCBhcmUNCj4gc3RyZXRjaGVkIHRvIGNv
dmVyIHRoZSBob2xlcyBiZXR3ZWVuIHRoZW0sIGFuZCBpdCB3b3JrcyBhcyBsb25nIGFzDQo+IG1l
bW9yeSBiYW5rcyBvZiBkaWZmZXJlbnQgTlVNQSBub2RlcyBkb24ndCBpbnRlcmxlYXZlLg0KPiAN
Cj4gSSB3b3VsZCBhcHByZWNpYXRlIGlmIHlvdSBjb3VsZCBhZGQgYW4gaW4tY29kZSBjb21tZW50
IHRvIGV4cGxhaW4gdGhpcw0KPiBvbiB0b3Agb2YgbnVtYV91cGRhdGVfbm9kZV9tZW1ibGsuDQoN
ClllcywgSSB3aWxsIGRvIGl0Lg0KDQo+IA0KPiBIYXZlIHlvdSBoYWQgYSBjaGFuY2UgdG8gdGVz
dCB0aGlzPyBJZiBub3QgaXQgd291bGQgYmUgZmFudGFzdGljIGlmIHlvdQ0KPiBjb3VsZCBnaXZl
IGl0IGEgcXVpY2sgdGVzdCB0byBtYWtlIHN1cmUgaXQgd29ya3MgYXMgaW50ZW5kZWQ6IGZvcg0K
PiBpbnN0YW5jZSBieSBjcmVhdGluZyBtdWx0aXBsZSBtZW1vcnkgYmFua3MgZm9yIGVhY2ggTlVN
QSBub2RlIGJ5DQo+IHNwbGl0dGluZyBhbiByZWFsIGJhbmsgaW50byB0d28gc21hbGxlciBiYW5r
cyB3aXRoIGEgaG9sZSBpbiBiZXR3ZWVuIGluDQo+IGRldmljZSB0cmVlLCBqdXN0IGZvciB0aGUg
c2FrZSBvZiB0ZXN0aW5nLg0KDQpZZXMsIEkgaGF2ZSBjcmVhdGVkIHNvbWUgZmFrZSBOVU1BIG5v
ZGVzIGluIEZWUCBkZXZpY2UgdHJlZSB0byB0ZXN0IGl0Lg0KVGhlIGludGVydHdpbmUgb2Ygbm9k
ZXMnIGFkZHJlc3MgY2FuIGJlIGRldGVjdGVkLg0KDQooWEVOKSBTUkFUOiBOb2RlIDAgMDAwMDAw
MDA4MDAwMDAwMC0wMDAwMDAwMGZmMDAwMDAwDQooWEVOKSBTUkFUOiBOb2RlIDEgMDAwMDAwMDg4
MDAwMDAwMC0wMDAwMDAwOGMwMDAwMDAwDQooWEVOKSBOT0RFIDA6ICgwMDAwMDAwMDgwMDAwMDAw
LTAwMDAwMDA4ZDAwMDAwMDApIGludGVydHdpbmUgd2l0aCBOT0RFIDEgKDAwMDAwMDA4ODAwMDAw
MDAtMDAwMDAwMDhjMDAwMDAwMCkNCg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:01:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:01:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196626.349541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnRT-0001Xq-N2; Mon, 27 Sep 2021 10:00:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196626.349541; Mon, 27 Sep 2021 10:00:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnRT-0001Xj-JT; Mon, 27 Sep 2021 10:00:59 +0000
Received: by outflank-mailman (input) for mailman id 196626;
 Mon, 27 Sep 2021 10:00:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUnRS-0001Xd-7w
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:00:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ceed6980-1f79-11ec-bc43-12813bfff9fa;
 Mon, 27 Sep 2021 10:00:57 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-27ozLIvAM_aVh9HdJOSefw-1; Mon, 27 Sep 2021 12:00:55 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7024.eurprd04.prod.outlook.com (2603:10a6:800:124::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 10:00:54 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:00:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Mon, 27 Sep 2021 10:00:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ceed6980-1f79-11ec-bc43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632736856;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3QPHlbDmOoVFRDmyzTvu3eGnri1US6M7NCW0Mn9uJVg=;
	b=U/T/EdSx09F1lDC5XV7eNmhNMg8vupCjm8VYJhrVukOvwlAwRv6X8WNVf0+9XJ/x16TXIU
	/OzMpqC/PQDDUOSJH3Ja8vrH/djumR030z3qRFPV3uBTI1GAvI7EXxUqe+1qVHuyHL5k3J
	XeUTR36Qq9LoIgkVfUuoBa7YdZkWezw=
X-MC-Unique: 27ozLIvAM_aVh9HdJOSefw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XGRZhR9mbrSO9sfbctRfqWtlj2qBArFn3FU2gTfr6cQO1GtCCxq0hY2TlaHdQypNcxK9Afy3XiEMLpWZjZhKk2zosao/8a2KCAfIvhxMOslPbAQYlmXhREU9HwtSqGJ8cMjDS9Ft/uTmy/IsEhFkdY1QQBaW7fyz7hvnRygDKkL3u2q7R+lkg680ECwnlOhcJgPj0YaHDZ0DvKHfosxMdwFk3gbTmcQXfJHySOqJxfW7ZCWwfAONtJLjxtKo3JXRSYC3zBpfVsh6uw4RlJjA+JTG6Bgwkuz9IeekHzS6Dj9dHZ1WBoaGzhnYntMWMUbY4xoNR7U2DWJzXxDnEwMWIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3QPHlbDmOoVFRDmyzTvu3eGnri1US6M7NCW0Mn9uJVg=;
 b=B5HCpACQC8dL0SYMyycn6bcutVSl4YymYZqivQ8hQheG20CUkvMyNF0I/Pqnh3trBII3l86Fs08qlctgoT4hrAxhELdLuL6xQglQLA+ICpVrRB8ZJxhchcqc8nYY8RtoXgsbK2nQt2Mhlnhx/HeiE1wUfMDgGRVOGtAozFtWN9weC4HrfF/kVMIJhO4sqmwfuFz2mxYg4J/kHwn4Dmk/tuvdyFF011jbu9yTvD+E/9UCrxl6Cv35ZBdLdGOCsFWAoy8GOqjTJ8fUqkEIGWMkW1z1zrULsI2Tiq56b/7uX5r+tdhlfS/pAK+u6T11nbFYkpDzSn6D2XDHQOUopPhIzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
Date: Mon, 27 Sep 2021 12:00:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0070.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 43c36e06-8302-4bf6-f347-08d9819db17f
X-MS-TrafficTypeDiagnostic: VI1PR04MB7024:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7024F8EFB1B1530F4B70A4E7B3A79@VI1PR04MB7024.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IZTP4X112Po/rLuFO7mcApaMdaNZPmnDXFPZ+B8lrd/nyxrBzxcJZwvjRRsx9YKtY9YuI2mayB18UVA18NbweSoOejKNvc5it+rAglMPA2vmn8tQLJ1w32SQ4J/P2mYSL0ULU7Y7/JYtQwjVq5B6YBTtug1+hndUetrwUwph7IiuxrzMsQ68y0C7oMlBex8DGdA/69oKFzirUVncz1FNFUFu0WJwX9dOmYgOpzjPH288BBQtLMhMHev6HO3c7xSuivN35Zo/HJfOB0fQRrYoI7t/IchJQEsujuqrOsNkV0Mx7Ftufu32+Oj9sZv1oQdib59m830m/x+19jA8RuePPA+NwrWBN/FTwu30b9/UsTvoZ/HTxZk5vAaXol7XwOZzZMFfqeg6Fl1fkBgC8m85MG1YQ5V4M0QMmBXWmm7iRXsfrzVz0qoqyDS0feu3b65zLjJJsT3pM59AyaHeh6L5GmZeCHAvMLSG7WMaKANZClFKjsoQvRHr5ZiXntnNDS9YYX7I2td75P//Ej9g/nXnv44tF3p4qT1QElFVyExbBR47RuMkTg85FhZ/0GiZ6/2UHOL7h7FB9bVLABaFX7dUsoARG1YdpRwxpmdVX4W1Hfypqum2Xm26vrDSTTUGfAohAH45WEpdtnm4D29yPwfRq4eDNMenbetWNHx0S2i8HKOR/QsAM17L5p8vmKRsGshrK7fI3uY8jvSSVoNsKhNFn0Ra6sMd9tAB3hiHsmvQKfE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(8936002)(31686004)(6916009)(53546011)(66946007)(38100700002)(5660300002)(54906003)(508600001)(316002)(16576012)(956004)(26005)(36756003)(186003)(66556008)(66476007)(4326008)(31696002)(8676002)(86362001)(2616005)(6486002)(7416002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OUpacXZaNE5HNmxZZmphSGNWb1BoVWJpV25TZnRKbEpBcWEwSm82SUREUFRM?=
 =?utf-8?B?R2I3RzBjY0gvYVZtMkJyWXk1TWV4YkFIZEZqMldoSUJjclA5QW5pM0k0d2Q5?=
 =?utf-8?B?NkpCQkcva1pBV2dHYUpJYVArSm16ZkJndUhhTUY1N245VUMydmRTbmhLQVRB?=
 =?utf-8?B?MlozbWRFVE1oWk5WRDBnbllUQ3Uwa25VQ0RNLzk4S1o3TVNuWUxhOThEci91?=
 =?utf-8?B?cTcxK0VvS2tvSlVlTUExOGt6M0Y5b0dmcHFnV3lYRGRZdDVFMk9qaUd3cnJ2?=
 =?utf-8?B?cXpTMHBwZ25lRGRpcmN4dXlHcFNKWkkwTVAyd2l4NjBCS05RWU1qUCtFTElN?=
 =?utf-8?B?RURSeGNKaUh3TW9ZNEdKSkFCOXdNZGVUSlpKS2VEWnBWbnpQL1VxTnozODJW?=
 =?utf-8?B?ZXo3WlFQdkVKOThpZWpLc25wTSsrSzg3KzdlcjVXSGVtOUwrbHc4T3o5NEgw?=
 =?utf-8?B?NUFrQXNNNjZlcXlzSElXc3NubVFpbFdVNDhYeDBzeXJPS2d1TWN4UFFkdGxJ?=
 =?utf-8?B?d3JMQjFtVEFFNGpHbnlwalpLT1UzbXB6eHRXK2ZOMGtJSGFVd3NzZkJDK3Na?=
 =?utf-8?B?NkEwbHVvYmt0UUp0cHhQbnNsWFc2QjlUNVBMLzVJdkRKcU1yakZTVXUzd3Fh?=
 =?utf-8?B?WkgzT1d2SzZ2eDJiTEFvNlBOdFlkOWs4NkU3L1o4bzdIcUNHaElVU3BOaFFC?=
 =?utf-8?B?aHRPTDdueE5TQjB4UVFsWCt4U0toV1Q1c1RoZFJOR3Rsd2VMdVEyYytIM0Rk?=
 =?utf-8?B?SEN6RXJJOGhUOEt4Q1Y5ZTRzaE1uYjE3dTIxNlRzdERQRnA3UHJqT3c0Q1R4?=
 =?utf-8?B?ci8veUdSNUFwbGNBQzNCdkYyTVh6N0lWOFN2MmZPMEtSSHI5cnE2VDEwUjRL?=
 =?utf-8?B?NmVNcG8vZmd1L3ZGVWh0Vkw1dklnQkUwWUVYelRnc3VaS1B3UGFxNUw1aGpR?=
 =?utf-8?B?b1VaU0dsT1NMOXVXTXBGM29seWcvcXJlYUZzRG5FUTZSNUwyTkpYZHdoMTNZ?=
 =?utf-8?B?UGxFcG5lRjZGVmNKbld0a1Y4d28wOXRGWE0zalEvNWdSdVVOZ3FpQjUweDVO?=
 =?utf-8?B?ZTUzaklkd3R0STJpK1BYaG0xa29wSlNMMkVpY0lRMkg1Vm1IRzBUNzRJNU9V?=
 =?utf-8?B?Qi8zY3FiTFFzd3ZzbTViNXFKcUJrOUM3RCs5bzE5dVJYNzk0R2paTi90cEx2?=
 =?utf-8?B?WXZIYUh5OXpIZjg5UnAvaEtQdVI0WXh6V3IxcitCREFYTHZsMndObDFJWXV0?=
 =?utf-8?B?c3Zxc2NpbEhISzRyaXREdXJhc1l0WkY5VVAyMFdQNFFxRFhmYlUxS3ZXUjlB?=
 =?utf-8?B?WFB4WjNkTUJIRU5pbU9HWGFLZlVjY0prQWd2ZGJGT0ZQZTE0ak1jczlsMW5t?=
 =?utf-8?B?d1cvak1wcWlYMytEUDV6SnJOenY2c1BKMmNVeThoYTRFN2V4cjI2Z3Vwb09J?=
 =?utf-8?B?VXRwbU15b2czOGpwclgrd3EvekxUUDhnMnF6dUY2eVFTU0R0a3ZEeVpQNVVB?=
 =?utf-8?B?VTFNVUZqMzMreS9VM0VETGc0WXptdTZOcjgvRUFhUkEyamZIcWJzNFZYMjRQ?=
 =?utf-8?B?VjVlcEhtQTZWbFdYZkI3YTBsQndvMmk5Y1VrYkc3RVlSMDd0ZVNPTUZXM2ZX?=
 =?utf-8?B?cUpJbFVaMStDQlZrS2Rqby9odnd0TGdMd1NjRlNacXJ2U0RTdFhiVTlnTVkr?=
 =?utf-8?B?ajk4WEZ5Vzk1WG9TZEZJNlFyNFU1eUxMeHlYUWxHM3grVERRL3RadXVkOTJW?=
 =?utf-8?Q?Pmv0EYsvyWQMrwYs3MOhwsyG7trvmnJKFLv72/i?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43c36e06-8302-4bf6-f347-08d9819db17f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 10:00:53.8496
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HyOOirY3esoiZ5/3prpvhQonZnki1GamHPaf5be/ZkXem8pyyTcFbmznhTvwwNJmBbxLQhH4Jq/ss/3FLPKeWg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7024

On 27.09.2021 11:35, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 12:19, Jan Beulich wrote:
>> On 27.09.2021 10:45, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 10:45, Jan Beulich wrote:
>>>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>>>>>        *((u8*) &pdev->bus) = bus;
>>>>>        *((u8*) &pdev->devfn) = devfn;
>>>>>        pdev->domain = NULL;
>>>>> +#ifdef CONFIG_ARM
>>>>> +    pci_to_dev(pdev)->type = DEV_PCI;
>>>>> +#endif
>>>> I have to admit that I'm not happy about new CONFIG_<arch> conditionals
>>>> here. I'd prefer to see this done by a new arch helper, unless there are
>>>> obstacles I'm overlooking.
>>> Do you mean something like arch_pci_alloc_pdev(dev)?
>> I'd recommend against "alloc" in its name; "new" instead maybe?
> 
> I am fine with arch_pci_new_pdev, but arch prefix points to the fact that
> this is just an architecture specific part of the pdev allocation rather than
> actual pdev allocation itself, so with this respect arch_pci_alloc_pdev seems
> more natural to me.

The bulk of the function is about populating the just allocated struct.
There's no arch-specific part of the allocation (so far, leaving aside
MSI-X), you only want and arch-specific part of the initialization. I
would agree with "alloc" in the name if further allocation was to
happen there.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:03:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196634.349552 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnUN-0002Fj-8K; Mon, 27 Sep 2021 10:03:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196634.349552; Mon, 27 Sep 2021 10:03:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnUN-0002Fc-5I; Mon, 27 Sep 2021 10:03:59 +0000
Received: by outflank-mailman (input) for mailman id 196634;
 Mon, 27 Sep 2021 10:03:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUnUL-0002FW-IQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:03:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1af06e13-db75-4ea5-8b2f-1f183528c458;
 Mon, 27 Sep 2021 10:03:56 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2108.outbound.protection.outlook.com [104.47.18.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-6elVGHwmMoOOsWz9khq24w-1; Mon, 27 Sep 2021 12:03:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3118.eurprd04.prod.outlook.com (2603:10a6:802:a::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Mon, 27 Sep
 2021 10:03:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:03:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0071.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Mon, 27 Sep 2021 10:03:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1af06e13-db75-4ea5-8b2f-1f183528c458
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632737035;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JYRPyskItNKUUBpdGX7fZkaIYIjBrv5rwp9aY6ntDVI=;
	b=RSEWWU4zzLZZE96XVV+vDyQnlfhZFRIat9BvDYaNQUbzSpF+HrToavD5uArX7nM9gAvfHS
	vEmrB5WzQU604kFGrQYj5066NZ3IkmEBoJjIfmrqWCHxEXqUm4OtZYVLnX+yi8T0umDL1R
	Eo4YJQ76WHWxzJIwlJR/mL8QkAtpFe4=
X-MC-Unique: 6elVGHwmMoOOsWz9khq24w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YZ7h5QuUlnjq5y/3D0u5JvgJP7wQL62YgGAEnf7SgNjZ0unBqe9A9S3k5sWkMQS0YctEMqtVZsyJwpJwhTwEHeyVXkdsGVr9MuuFn1qkPMa6lMhLwWO1eleUQ/0cgR+6EChBBiAPg3B+ykuh7d6+Bb7yofIDqZl/H6et3JT94g4EoONUHGDyTsFHz6vmSKwkBXigcJBAEDbqJSw9asNlWdmtH0NhhB51++fgIWZrwyHybxmn1+4q+wpsMYjcO58Jn5PwnUzewtxyfvKTunBxp5qiEGZPZciSH0R0mfC78ePHQfdypMGR1heZt5Ya3qPjwyu68BazYORD948SzxvO5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JYRPyskItNKUUBpdGX7fZkaIYIjBrv5rwp9aY6ntDVI=;
 b=Bi75LK4LhhxRCOmNxozkhBnsf3WZEcwhgZQIeBcNY8+gJYvyziaLnCo4MFeNF/YouJC291wyHGarl07gM3W0fkI+knRVD+gCRzWCGIWmrsnIyf8aNAUpWUH/5e04QDhK3K4dQqnfZfAt+T1qZjEHAKv6JlWoUdwBY7Rlmci8N7FaWegHBTOmSHBUaiAO+/tf+gAI/pTjmiU/L4rofAa6DCZprHdoXt3RzJ6wDshx/XYdr4eVWSbmVznDTo/oARbySj2IPOBmdnQQLszEzugpAgFpq0sYH/pjCLhh5/ozVgQtdH2f0VvBdvV1INXT6ISZo5NQOXo07PPsEgTx53iIDQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v3] xen: rework `checkpolicy` detection when using
 "randconfig"
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210908111727.440265-1-anthony.perard@citrix.com>
 <bb40484f-fbfb-a206-fa98-0e927f5398fa@suse.com> <YVGS9K1XqNpyxAxe@perard>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ed0f7470-69f4-f1bd-50bf-fea19fbbe535@suse.com>
Date: Mon, 27 Sep 2021 12:03:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YVGS9K1XqNpyxAxe@perard>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0071.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 63a2a16b-8d4f-4c80-ebd5-08d9819e1b56
X-MS-TrafficTypeDiagnostic: VI1PR04MB3118:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB31184B18555649A1FDAE69D0B3A79@VI1PR04MB3118.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YZsQ4tmR4SZTFV5QG/1D9R/hD5jSzqk4Z9oItS2xinlaCAQ4QEUTd5GLP6X0miSkRT0ZS1u8+zU4H/eowjvfeQJai6LyBSef6NjzpvSSJIWp5hCD2BMC583k33uCXatQ8fkB/cYePAqGd9k3OdSlGp/J22E1EUkshLue92uWL99j5NduVizvJ0MnVgArGGKDOS0ERuKgGlSvdk7yiQui0XkcfLXJraD/oApMkUO27uaEAVAuc9mdJip6x62QRQLxIAJRLEUGEXR+j1ootWrhd/AFYWWiF9q3rRfI51wpM+jw8Yd3h6Zb7eBoMH6MSRacUZWu18+oLSY9ZeSwu2hxz0RsuLnw+JY0WSrPRYMFkPouTD3ft0EXymRJc4p/hxzF+g+RkQ+Br5pBnMhB0P4pk8MGePdoV8Ylkb5KDS7es7DYljrEhFpRJ7MaPXniAzjdr5bNjvTRzRHmTxpd1jt8vI4EF3iXEPUQ1HjMB95UknGsc1J30u7JUzgT5VbRdPmr5AUTX+5OJjZCdzd+qMF8GlEep1ky7/PBbRRZADuWbJdlKLHDlV4aMfUWByZdthRqaSFb2zP4Y3GnRtE8/u4cs4V+pKk1k5+sVejM0M+a3KCc4W95D7F/nfk9Mf+8HxUBvjOPaCpvxmYNyqEgaX2172R6ildkz3xI+ZvD85aBqQFgoWKfXddjQKJeQm7MrBeFGgXgtm+GtNjoQ0Wj2VnG6B+RSNm3HApCCxI2R0z6tnEMh2ptloJFtADIVftCNjDXLBA1mtsa8y+inlsPw1aDpQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(956004)(6916009)(16576012)(83380400001)(31686004)(53546011)(6486002)(2616005)(26005)(186003)(4326008)(8936002)(38100700002)(316002)(5660300002)(508600001)(2906002)(36756003)(66476007)(66556008)(66946007)(8676002)(86362001)(31696002)(81973001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SDZJQXhKUVNDdERtWnN3ckUxdllleUZyYmdvT1EySlhlcWlpR3J1Z2hmNGpI?=
 =?utf-8?B?dWFYOWFrWS9KNnhqUkh2ZkwvWVBEZ3QrWTFDV3BZUHEwUDNvamRiREpxaElQ?=
 =?utf-8?B?S1FSemU2bU5YZzlGb2tUNno4R0ZCSmp3QU5zUEJqVEtjWUI0b0JzcjRPV1Rh?=
 =?utf-8?B?bHd5TWdaZUFpcHc5NVBJQnQrY1ZQbDQwOFZ2aUF2OXlNOGM3TUNhUlgyY053?=
 =?utf-8?B?eEFjYkpxNllwUC95Z1crTGV2cjdzdnRrc3JGVERpWmEwYU9HWmdXSHRtNlgz?=
 =?utf-8?B?aEl2djJnUWdwNk82T2N4RHlpaG0xR2F1UVQ4YjhqL0lVaWN5aTBNNHR1bmt4?=
 =?utf-8?B?U3dNSVpKOHZLR1RCLzd4N3NWdXVSZStLWTVLcEkxQ3RUSHQ5TTVUMEpabk1Y?=
 =?utf-8?B?ck8reFFHa0FRUFYxYi9qdXJDc3k3bEZHT1RMZjFBaUdPczBQd0NxcHl4UmFL?=
 =?utf-8?B?cGpySFkxeEo2UDJHUWdkOVVnd0tNUC83RTkxZlNtVDR1Y2RqYmNvaitVb3hk?=
 =?utf-8?B?ck82SGg4MUdLenZGK1JiYndpVEpSSmF1eHlLMVJsRXl3S1czd0g4WlJycGJw?=
 =?utf-8?B?cTJlWTZKSExOcHQzVlRMVkwwVGorSlVpMDg5WTNtUGM3K0J4TzRpSWJxMFdu?=
 =?utf-8?B?TkFxV2UzUkRGdlNSKys4TGRueVFvYzg0WG83SVRaTHNNckw5dlJpMlFpd0Yx?=
 =?utf-8?B?ZjdxQW9XUGNPWVdXNU90Q0RJdzBEL1RHOHc1RE8xL3FIY1RWMUJlazZuTVha?=
 =?utf-8?B?UW5UL0RLVG5wd2J3b3BiZTA3TmtGTXRVVGVBUWEzSmFNTDZ6NEt5MlRxU2p4?=
 =?utf-8?B?QmlpeVpIUmNCVGZyV2tXMmh5U3Q5NDZ4RkZybFg3Um5VZy9CMTcvWDFKUVgy?=
 =?utf-8?B?MVBCcWNJanJ1ZFZHdDYySUFJZ3o5dkF3cjhoTytoOGNaa3hnelc3Vk5VbGhO?=
 =?utf-8?B?MzdrMXB6aVY5cURKaGowZlFSekordmJsTm8vRjRnWHk4V0J0dG16cnVHWnpn?=
 =?utf-8?B?ejVMT2luMzkxS1d3WUJzY3lnV1g5akwxaWdhL05xUk1MR1R2aUtjOEdvb2oy?=
 =?utf-8?B?MzJkYTFQNnRvR0JmWit4NEtBYnM1elJJRENWdThXWWphZkhndU9XK1E5VTYy?=
 =?utf-8?B?SUJCNlZVbmJSQ3FxTzlhdCtJekl4TGY1ZXE0cDRXbHdWZ1NIOEZpRW02MWpp?=
 =?utf-8?B?aUVzVkg0SGJtaFNPRm1mQ25VQmFYUzJNSDFNMkNmRDdTWHUwMmluVEk0ZTgz?=
 =?utf-8?B?K2dYYmpXQ2NCVDJ2NWFiVmtZZG4wRDYyNHloMmNySS80MXpHMXRZMzdrMmN1?=
 =?utf-8?B?anYyTXR6YytBbkRWY0RCQ01IZnozTFloZFl5WVNQK3h0V2gwc0lmd0NNUFRk?=
 =?utf-8?B?cmo5cWxrUEl3TkI5dXZMTXNxWFhHU09Od1JQeUtNSVlzOWxaaTc4OTRqSXc2?=
 =?utf-8?B?em9qWjlBRTMyeWQ2RkswSm5oY052OGdQTklZVW1LNyt4QzhQS1kxUzBWL3BW?=
 =?utf-8?B?OExDaUVrMWlXNGQzNFpMTE9IVE5TbCt3eXlLSS93VGoxbkJWcTl6Rzh6R2Ux?=
 =?utf-8?B?b0dzZFRxUlZPdmFTZW9KTldYYVlHRHZtYjNvOW5GMnNkdmo4NFNnSmlLZk1o?=
 =?utf-8?B?aDIyN0lqVGw4QkxyV1o1REJvcXBjcTF0d0RqeUFzYU1ONDF5RmJ0OHlPakdh?=
 =?utf-8?B?NEVIU05GbkM0MlRMZEhRcVdpQ0RnaUV0anlOSmwzcjc1OC9RVlJnTFN0aWkx?=
 =?utf-8?Q?6yT9C9a2zpKXswmWW9kMb7GYkFhGxTUlgxbV5m6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 63a2a16b-8d4f-4c80-ebd5-08d9819e1b56
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 10:03:51.4082
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zEoSPFpa/nsCEe6kJwK0IGDA1min9i04ybpSwB7JbArC8zllw+VNS05HVskxHBr9agS4IMxDKqx7Lu6gI9kQ4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3118

On 27.09.2021 11:46, Anthony PERARD wrote:
> On Thu, Sep 16, 2021 at 05:34:00PM +0200, Jan Beulich wrote:
>> On 08.09.2021 13:17, Anthony PERARD wrote:
>>> @@ -189,14 +193,24 @@ ifeq ($(config-build),y)
>>>  # *config targets only - make sure prerequisites are updated, and descend
>>>  # in tools/kconfig to make the *config target
>>>  
>>> +# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
>>> +# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
>>> +filechk_kconfig_allconfig = \
>>> +    $(if $(findstring n,$(HAS_CHECKPOLICY)),echo 'CONFIG_XSM_FLASK_POLICY=n';) \
>>> +    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG), :)
>>
>> Nit: It would be nice if you were consistent with the blanks after
>> commas in $(if ...). Personally I'm also considering $(if ...)s the
>> more difficult to follow the longer they are. Hence for the 2nd one
>> I wonder whether
>>
>>     $(if $(KCONFIG_ALLCONFIG),cat,:) $(KCONFIG_ALLCONFIG)
>>
>> wouldn't be easier to read.
> 
> How about:
> 
>     $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
>     :
> 
> .. instead, as that would be more consistent with the previous line,
> that is there would be only one branch to the $(if ) and no else, and
> thus probably easier to read.

Oh, sure, even better if that works.

>>> +.allconfig.tmp: FORCE
>>> +	set -e; { $(call filechk_kconfig_allconfig); } > $@
>>
>> Is there a particular reason for the .tmp suffix?
> 
> Yes, .*.tmp are already ignored via .gitignore.

I see. Could you add two words to the description saying so? Or
maybe even just a post-commit-message remark would do.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:04:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:04:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196639.349564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnV5-0002os-Ji; Mon, 27 Sep 2021 10:04:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196639.349564; Mon, 27 Sep 2021 10:04:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnV5-0002ol-EG; Mon, 27 Sep 2021 10:04:43 +0000
Received: by outflank-mailman (input) for mailman id 196639;
 Mon, 27 Sep 2021 10:04:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUnV3-0002oV-Su
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:04:41 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 53fa8056-9d09-4c6c-a12b-509460984814;
 Mon, 27 Sep 2021 10:04:40 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18R95F0o025045; 
 Mon, 27 Sep 2021 10:04:38 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbb1107y2-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 10:04:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6963.eurprd03.prod.outlook.com (2603:10a6:20b:2d5::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 10:04:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:04:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53fa8056-9d09-4c6c-a12b-509460984814
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CErWoN0gcqYMvW8iaX/sBLW3GSu19hkHKT9IcFOYoAQWGKNHiKEgblFzYNnyFLPJLi/KOIrw2bFoY2TAN6IFzRBJXmsgj5cVZ2+7QSM3Y0zPBFqjGg6F8JjDZSkUoUpYfuzG1HrONc9lt2NQDu4hUj++ptp+2bg3zLPtdd1OK2+jOXHpqSJ0LOYnaLM0qthPe+TcpcI1AXPFXk7N1TCUmsLIWHIb9GzeKR3CuWLqiJIbxLhLf7OYZ72zE1OcmCy8UButgQ0n6501vNjLoMmN6bXyLzkwBASbYI4rIx+NYftSqYFTvui4Pmgs8Wau0f60oYR/HI+IIg1dW4PmpoKVMQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gPYe77LdkDEmNrinKE2WVBUO6bpsOy5UdR7i9kPst/4=;
 b=CnpyjtaufdKACAS2M56cdjlFZjd37C1CJaiGN4QJuqllLaSsQlSLed4uQCD9ImNDRhnCpLdfuW/pUVEhSZ5UEz/C9odHkvjG5empW55xWSaIV6HCPIQcFrLx35Rzc/EiuVcLXCKvj7APk/TX+N4okB/u5kLVYmJEhBGwnmyyr2J+pMa1KA+ORt2F+ZshHeJLStRKJ/tjO/AZBV+nAmFHUYUws+zZBEeifV+QvtydvqCl/8VxVPZHfjcxUIp8Z/1GNu5YxchYpmop8l5iI8fun1yIVE9WqXxsiY6Sl9FV5hUUBPs6fjq1dKU4/+quANbdaa1YmFoxML/PdYvO6ml1kA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gPYe77LdkDEmNrinKE2WVBUO6bpsOy5UdR7i9kPst/4=;
 b=ET5h4+3voaHqnc051TMps7jchu4hBsJPOyrOLzEs3OHl2J0SveGwWYrvLK1i53CQJUfmvIOL1j3wrG2Cr4/TvcHKSNv3fdE4J6ymvy7ee5VNijdFUm9sP+Btxh+RGDvOhJJ7WEno3gabjnmaMSVO0hay51J8cbabKLWkUTgDGwIYflGjsCwDsQNpkqJzH9ZoUE5G3KD70WcN/leq0LdAG4yvbl/Fnuhb/RVBcRPUi/uJTYjCTetUaPxMObsrBXARohEUTuWj2qKoqbckHZL9GaL/BhatcMzRAHSh/IzDBTYazaSYhglRCECyc5c6UIVm7W3bSH1gVJdXFRJJOZp6Iw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: 
 AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrICAAAldgIAABLUAgAAG9wCAAAEGAA==
Date: Mon, 27 Sep 2021 10:04:33 +0000
Message-ID: <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
In-Reply-To: <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0686b6b9-2656-4607-871b-08d9819e3486
x-ms-traffictypediagnostic: AM9PR03MB6963:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB69631F820B096A31C987A3A0E7A79@AM9PR03MB6963.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 GZy7ZieUQuZxlpBNJykbm36ivZR9b1+2TgQSw6Rv0FpsRLQqEp1hbdnhqqxAQqSSYEXx7cIltZbuBFXIlOPWi0xqDJePMLZ4iWUQGNgMsmFMGmbzYekK4z0imYCT6i3BT/jURDodkpHlF218Z84BEMBCGdkFMDi837O0TYzrANh7XH4uCnZMELoYYAlDlF0YS0mSmCIzDnniwWD4cQS6b7ffyP8dyRqZU2JHdh/beBJCjKD7R8Qe7XWcIv0xIAvuh1hRni1Nqcm1wYp0+eq8JqOfFSqEl/3Lr27YGt4EPjVaXHW7GfDuava5dhg4xiOKJ5OBaS7SluUrikZW3S33OoHvLg7RxME5oLFL84OBvCGV5lwTfC9OO8IlElK/2FSK4DS/KzONbKqAkZHFxpflkx2pTqxSEFqGm76oZ9pmQlxO1Zjhij5yGIEoLqr/LAEG8itGLTUkU+XTK31dI+/LtNmEEQ9tvAmlKBXe/KCj462/UFRvq2jUqiUvVutCv/4/CguYrNAwyUIa1j2ccCKemB6+MTGsIiPc7uNRWnuSlmKy9JYgsRx3aSVq13MHK5mm6AWc1ZOvBCJT0DK/8AfBFKapEyzz9V7TZsJByHiHt+ZiLpYe7nd6/SWoQMXpiMPY3Ptlk99emIFZmh7sQqft1i1ybT1dWCV23/6U44y/+pDT6334gNXGP+BsYJMjF4Sfj4BBx8nT2hi3jVSXJS+h660l85OPrTNG6niVRtelJQbWrmMSu8Ad/mXpqOz196HE
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55236004)(86362001)(316002)(64756008)(31696002)(66946007)(66476007)(53546011)(6506007)(31686004)(66446008)(8936002)(2906002)(38070700005)(54906003)(6486002)(91956017)(76116006)(66556008)(4326008)(8676002)(36756003)(5660300002)(186003)(6512007)(122000001)(508600001)(2616005)(71200400001)(6916009)(38100700002)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?MXJhb21nOGx1NGZNWmZKbnJxVnlvV0d3bWRTT1l6dFgxb3NQT05uUkNXSWtP?=
 =?utf-8?B?T1l0c3dza2VKYmEreDRmVG95Mk0xdldKVU94WUZ1VlQ4Z2phc0JCOHpmSjAr?=
 =?utf-8?B?UU5SS0tIZWZ6Q005ODMwRzNtMUFqVG9GNHdhYUdsd3JqdUdVdEJobmxZbzRk?=
 =?utf-8?B?UTFGZk9NNEcrZnRUTFdac3pTOFZ6QzNNTXpnUTFMU0M0b0hDU2hoWFlqV0R6?=
 =?utf-8?B?aXBYQTR1aGpSZFovSjIwUkJpc0c2YnhrYjJ1clAwRUNGQnJ1UmFMNGFaYlRN?=
 =?utf-8?B?N3pITm9WY1R4ZWR1c3E0WWtvR3dLZis3djQ1OGlIdncwTTgweHZhallraU1n?=
 =?utf-8?B?LzN5WjlCS1Zuam9HeVl4N0hVWnZGRElhR283M3JSdnppVGMrWWkxR2FvcnFQ?=
 =?utf-8?B?d3YvZ3hNbXdFd1VGM2drR1gxY3BzTUZ6WjNaM2tqeXFYOGZUVm1NcElnUHQ1?=
 =?utf-8?B?cGdFZmJoUkNtZWtBY2gxUVlpVVRzYzNUcTlTVDhCa0ROelBUQ2poOGhQSW90?=
 =?utf-8?B?S1lYNFozNVdNMHB0aXRON3dLS3ZNbVVPdTVmVG84WkxrNWVBUmprRU9yWnp0?=
 =?utf-8?B?SlYzQnRLbUN3MUFPdG9PTmI0T1crclBOYXV2UFpBU1U5c1VHY2tkalJWVW81?=
 =?utf-8?B?WDY4U0VDRWxaTFMwdjVIb1YzaVJ1NXBPMGNkdktJcFFBS1ZlWDRKeGUwTnBj?=
 =?utf-8?B?eitZaGxhRlRyRUFNWGpoUjBRL1BsVnlWOXlQcGpsRXphbjBkRXI4TVZia0p1?=
 =?utf-8?B?YnpYOWlMRW9OK2VyKzhlZWRvTUpwUTJHSFo0dHd5N0VMMjFCTzZzS3drbjlZ?=
 =?utf-8?B?UWw2WENDMUQ5d0VyR2ljWXRrV0xSMksvdE5yVlBRdnRtaUxNUHc2WG03dks3?=
 =?utf-8?B?dDBaOUZ1OFcvZjVHN2RicFJUVW5ocUZBRFA5bzN4ZnhtVFhtWkY5VXYxTXRl?=
 =?utf-8?B?ekd6bTBmUXdGUm42U25YZ0Jpd0pzYlY1M0FhVmFqUVU5N1g2RFk0VDhpYi9i?=
 =?utf-8?B?VjRsbk9udmdZdXpwb0RHS0NGVGRISmdpZ0JteVBmM0ZLQnNaS2tzakIrMjZO?=
 =?utf-8?B?Znd1WHo5UmxkQ3M5TEl6d2tSM3NseENiREdoZlhhV3p1ZmFuVnM5QVN0ay9X?=
 =?utf-8?B?ZUtHN1RIak5RYUEyUW8xOStPaml5QWJKTUQ4bTZSSE5pV1RzZUdCbHN4MkZv?=
 =?utf-8?B?bERINEczaEhBbnlyckxKY0FicXRJMnlxVFdKS2Y5TnRRRXI5aDgvZjEvK2tP?=
 =?utf-8?B?SlJ4Y3JWTHBEeS9hSll5SFYyaW1RLzFIUGZHaEcrQ1ZUeWVDYzNkNEc2MXlj?=
 =?utf-8?B?L2xEUzFLL2pJUUtkOTBNQURzblh4RnF2TlNKWUo2a0x0bk5NSkVOVTFHSk5v?=
 =?utf-8?B?Nm1DSnBWZzl1Nm9BK0ZRNlQ3Rnk5OEpFazVERzlPOFcxYmtQa1lsczZQNW9V?=
 =?utf-8?B?VVhrdzlVOHVCaGZ5aXphREFoRnA1VzVvT1Y2N0xGTDA4cWNEREMxd0Q5QzYx?=
 =?utf-8?B?Ni9sK25mcWlxQm54UkhoTXl1K0Q5NnIxREtuc05wSTRwSmI1QlJjRXpzeHRY?=
 =?utf-8?B?dFNGNHFuYllLZEhGNzFoRkxXNDREQ1NJRVZlVnlMWFV6U3FreVFySHlnVzQv?=
 =?utf-8?B?VkdZRTRvYnpLZkJBYlhqaFhZUEFDMjU2Z1h1UE5ibytJcjQ0OUJ1YTdoWkc5?=
 =?utf-8?B?c1pkZFN3NlN0QVdKUlZJZVduOXB4L0dkTHZ3cTVFWHhJdEJ3NlFnQkFNN1V0?=
 =?utf-8?B?Zmg0R00xWkYvdjk2Q0xVbW9EQld6akJJOGU3MUFLNTN3amRnOXl5UDZEcFNp?=
 =?utf-8?B?bHN6Qkk0c29qUCs4aERVZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <3C35164A63C1B44492B0AC55A353AFEF@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0686b6b9-2656-4607-871b-08d9819e3486
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 10:04:33.2079
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rtSxvaKixPVCEC/gGuQbwyugjojAjG4aYAK3TscgeepNQnBVnxJ+hHdAmFZhY1zEjPvEMVT6K9FZwFdXAfUlnqKCy/hWkZTL2UtuzknMuLco+g4c3B1rdLHWl841LqM4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6963
X-Proofpoint-ORIG-GUID: efGINWN45lJvf-s6W0_Mab-6ofKi4Bp-
X-Proofpoint-GUID: efGINWN45lJvf-s6W0_Mab-6ofKi4Bp-
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0
 impostorscore=0 phishscore=0 clxscore=1015 mlxscore=0 bulkscore=0
 mlxlogscore=999 adultscore=0 suspectscore=0 malwarescore=0
 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109230001 definitions=main-2109270070

DQpPbiAyNy4wOS4yMSAxMzowMCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTE6MzUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTI6
MTksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIwMjEgMTA6NDUsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxMDo0NSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyMy4wOS4yMDIxIDE0OjU0LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0K
Pj4+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+Pj4gQEAgLTMy
OCw2ICszMjgsOSBAQCBzdGF0aWMgc3RydWN0IHBjaV9kZXYgKmFsbG9jX3BkZXYoc3RydWN0IHBj
aV9zZWcgKnBzZWcsIHU4IGJ1cywgdTggZGV2Zm4pDQo+Pj4+Pj4gICAgICAgICAqKCh1OCopICZw
ZGV2LT5idXMpID0gYnVzOw0KPj4+Pj4+ICAgICAgICAgKigodTgqKSAmcGRldi0+ZGV2Zm4pID0g
ZGV2Zm47DQo+Pj4+Pj4gICAgICAgICBwZGV2LT5kb21haW4gPSBOVUxMOw0KPj4+Pj4+ICsjaWZk
ZWYgQ09ORklHX0FSTQ0KPj4+Pj4+ICsgICAgcGNpX3RvX2RldihwZGV2KS0+dHlwZSA9IERFVl9Q
Q0k7DQo+Pj4+Pj4gKyNlbmRpZg0KPj4+Pj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQgSSdtIG5vdCBo
YXBweSBhYm91dCBuZXcgQ09ORklHXzxhcmNoPiBjb25kaXRpb25hbHMNCj4+Pj4+IGhlcmUuIEkn
ZCBwcmVmZXIgdG8gc2VlIHRoaXMgZG9uZSBieSBhIG5ldyBhcmNoIGhlbHBlciwgdW5sZXNzIHRo
ZXJlIGFyZQ0KPj4+Pj4gb2JzdGFjbGVzIEknbSBvdmVybG9va2luZy4NCj4+Pj4gRG8geW91IG1l
YW4gc29tZXRoaW5nIGxpa2UgYXJjaF9wY2lfYWxsb2NfcGRldihkZXYpPw0KPj4+IEknZCByZWNv
bW1lbmQgYWdhaW5zdCAiYWxsb2MiIGluIGl0cyBuYW1lOyAibmV3IiBpbnN0ZWFkIG1heWJlPw0K
Pj4gSSBhbSBmaW5lIHdpdGggYXJjaF9wY2lfbmV3X3BkZXYsIGJ1dCBhcmNoIHByZWZpeCBwb2lu
dHMgdG8gdGhlIGZhY3QgdGhhdA0KPj4gdGhpcyBpcyBqdXN0IGFuIGFyY2hpdGVjdHVyZSBzcGVj
aWZpYyBwYXJ0IG9mIHRoZSBwZGV2IGFsbG9jYXRpb24gcmF0aGVyIHRoYW4NCj4+IGFjdHVhbCBw
ZGV2IGFsbG9jYXRpb24gaXRzZWxmLCBzbyB3aXRoIHRoaXMgcmVzcGVjdCBhcmNoX3BjaV9hbGxv
Y19wZGV2IHNlZW1zDQo+PiBtb3JlIG5hdHVyYWwgdG8gbWUuDQo+IFRoZSBidWxrIG9mIHRoZSBm
dW5jdGlvbiBpcyBhYm91dCBwb3B1bGF0aW5nIHRoZSBqdXN0IGFsbG9jYXRlZCBzdHJ1Y3QuDQo+
IFRoZXJlJ3Mgbm8gYXJjaC1zcGVjaWZpYyBwYXJ0IG9mIHRoZSBhbGxvY2F0aW9uIChzbyBmYXIs
IGxlYXZpbmcgYXNpZGUNCj4gTVNJLVgpLCB5b3Ugb25seSB3YW50IGFuZCBhcmNoLXNwZWNpZmlj
IHBhcnQgb2YgdGhlIGluaXRpYWxpemF0aW9uLiBJDQo+IHdvdWxkIGFncmVlIHdpdGggImFsbG9j
IiBpbiB0aGUgbmFtZSBpZiBmdXJ0aGVyIGFsbG9jYXRpb24gd2FzIHRvDQo+IGhhcHBlbiB0aGVy
ZS4NCkhtLCB0aGVuIGFyY2hfcGNpX2luaXRfcGRldiBzb3VuZHMgbW9yZSByZWFzb25hYmxlDQo+
IEphbg0KPg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:06:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:06:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196647.349573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnWs-0003SN-TL; Mon, 27 Sep 2021 10:06:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196647.349573; Mon, 27 Sep 2021 10:06:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnWs-0003SG-QN; Mon, 27 Sep 2021 10:06:34 +0000
Received: by outflank-mailman (input) for mailman id 196647;
 Mon, 27 Sep 2021 10:06:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7LT4=OR=linaro.org=alex.bennee@srs-us1.protection.inumbo.net>)
 id 1mUnWs-0003SA-5T
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:06:34 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e0e9f269-53e9-4ad8-90a3-0ba5d1a28480;
 Mon, 27 Sep 2021 10:06:32 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id d6so50689670wrc.11
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 03:06:32 -0700 (PDT)
Received: from zen.linaroharston ([51.148.130.216])
 by smtp.gmail.com with ESMTPSA id d1sm18124520wrr.72.2021.09.27.03.06.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 27 Sep 2021 03:06:31 -0700 (PDT)
Received: from zen (localhost [127.0.0.1])
 by zen.linaroharston (Postfix) with ESMTP id F2AA81FF96;
 Mon, 27 Sep 2021 11:06:29 +0100 (BST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0e9f269-53e9-4ad8-90a3-0ba5d1a28480
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google;
        h=references:user-agent:from:to:cc:subject:date:in-reply-to
         :message-id:mime-version:content-transfer-encoding;
        bh=SsGrfYLyvw3STW3SK74psyLTGgh9Nxr1GGqYsTYAMcM=;
        b=JCYJqDmZcP6dpSmkqFq9/3KpGM/EFT6dpJQunFfKtIsXa7H/+qmthxv62mSihz9858
         xkYorIeoLH5z0wo24fwlSCH11vuUQQwoVlqrOlA354mdFgb8kfh7IujClMxTIqSub0p4
         wMp7PN/KsNJXVdJv0+GA/ovE7sJJPzCM0TIYBVXDYtTvc6vNR9ia+ZoNRy0iIrLcRrlj
         c4ieqfSDQ+ZtcYpPeGPPGN910dx9cjHmrPSDfdjq7E8DzUyLOkOu581dKaEIZCnhfAY2
         PxeZCLehlosYGsEkVE2xno4tT2nnV3YbM4cNxIeh0ZD0zlMwBfA399RkjbpfD1KhvnJ1
         Vatw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
         :in-reply-to:message-id:mime-version:content-transfer-encoding;
        bh=SsGrfYLyvw3STW3SK74psyLTGgh9Nxr1GGqYsTYAMcM=;
        b=5wRyg1ndIwqcJ7Y44PWhOA3ztaERpZquEDoVgzUIFWAP58xyAMpz2b3Pq4skpsOwzi
         flYIQNFiPBMuYYRj8l6Sc+Ps2O39pHaf97pHxSOoURQbMSghrDRKtXq1Sh55ziFW5DPa
         Jquo4rlDgHzqYQMfYvct1fnBDdh8wSTVujMRH/7qcIs1TNpQNhjPyR6EaHoTTrcnKk0Z
         6a2n+pg1nrTo2ZAtAhWeRitU2GYxo6g3WgwDQlD6dFAnURuAcQCjQ/uWJw6akjjR/VpM
         FLdH90D7spsw9hLdVL1JPMNuQTUUGU4l5RBwmQPv+WKyx9t6vhg5tqLA2zYqXpW/SqAx
         xBxw==
X-Gm-Message-State: AOAM530A2JNuj2ndt1I702rQP18D2vMNC/YWdmb7pkn3n8cumuJ2z8Me
	nrzmBpnfh7WHRi734pmd222hVg==
X-Google-Smtp-Source: ABdhPJx0ZluoFQhLnmLJIXQPudn0JMs9Zcw7fcd8S25Q/lMxhj02kSRXJITIIb0QoZ2iE/oFKFqXpw==
X-Received: by 2002:adf:d851:: with SMTP id k17mr11896387wrl.54.1632737191992;
        Mon, 27 Sep 2021 03:06:31 -0700 (PDT)
References: <87pmsylywy.fsf@linaro.org> <YU5mW396S04IsCBr@mail-itl>
User-agent: mu4e 1.7.0; emacs 28.0.50
From: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Stratos Mailing List <stratos-dev@op-lists.linaro.org>, Mike Holmes
 <mike.holmes@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>,
 Viresh Kumar <viresh.kumar@linaro.org>, Peter Griffin
 <peter.griffin@linaro.org>, xen-devel@lists.xenproject.org, wl@xen.org,
 Artem Mygaiev <Artem_Mygaiev@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Doug Goldstein <cardoe@cardoe.com>, Oleksandr Tyshchenko
 <olekstysh@gmail.com>, Rust-VMM Mailing List <rust-vmm@lists.opendev.org>,
 Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>, David
 Woodhouse <dwmw2@infradead.org>, Arnd Bergmann <arnd.bergmann@linaro.org>
Subject: Re: Xen Rust VirtIO demos work breakdown for Project Stratos
Date: Mon, 27 Sep 2021 10:50:56 +0100
In-reply-to: <YU5mW396S04IsCBr@mail-itl>
Message-ID: <874ka68h96.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com> writes:

> [[PGP Signed Part:Undecided]]
> On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrote:
>> Hi,
>
> Hi,
>
>> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
>> =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80
>>=20
>>   Currently the foreign memory mapping support only works for dom0 due
>>   to reference counting issues. If we are to support backends running in
>>   their own domains this will need to get fixed.
>>=20
>>   Estimate: 8w
>>=20
>>=20
>> [STR-57] <https://linaro.atlassian.net/browse/STR-57>
>
> I'm pretty sure it was discussed before, but I can't find relevant
> (part of) thread right now: does your model assumes the backend (running
> outside of dom0) will gain ability to map (or access in other way)
> _arbitrary_ memory page of a frontend domain? Or worse: any domain?

The aim is for some DomU's to host backends for other DomU's instead of
all backends being in Dom0. Those backend DomU's would have to be
considered trusted because as you say the default memory model of VirtIO
is to have full access to the frontend domains memory map.

> That is a significant regression in terms of security model Xen
> provides. It would give the backend domain _a lot more_ control over the
> system that it normally has with Xen PV drivers - negating significant
> part of security benefits of using driver domains.

It's part of the continual trade off between security and speed. For
things like block and network backends there is a penalty if data has to
be bounce buffered before it ends up in the guest address space.

> So, does the above require frontend agreeing (explicitly or implicitly)
> for accessing specific pages by the backend? There were several
> approaches to that discussed, including using grant tables (as PV
> drivers do), vIOMMU(?), or even drastically different model with no
> shared memory at all (Argo). Can you clarify which (if any) approach
> your attempt of VirtIO on Xen will use?

There are separate strands of work in Stratos looking at how we could
further secure VirtIO for architectures with distributed backends (e.g.
you may accept the block backend having access to the whole of memory
but an i2c multiplexer has different performance characteristics).

Currently the only thing we have prototyped is "fat virtqueues" which
Arnd has been working on. Here the only actual shared memory required is
the VirtIO config space and the relevant virt queues.

Other approaches have been discussed including using the virtio-iommu to
selectively make areas available to the backend or use memory zoning so
for example network buffers are only allocated in a certain region of
guest physical memory that is shared with the backend.

> A more general idea: can we collect info on various VirtIO on Xen
> approaches (since there is more than one) in a single place, including:
>  - key characteristics, differences
>  - who is involved
>  - status
>  - links to relevant threads, maybe
>
> I'd propose to revive https://wiki.xenproject.org/wiki/Virtio_On_Xen

>From the Stratos point of view Xen is a useful proving ground for
general VirtIO experimentation due to being both a type-1 and open
source. Our ultimate aim is have a high degree of code sharing for
backends regardless of the hypervisor choice so a guest can use a VirtIO
device model without having to be locked into KVM.

If your technology choice is already fixed with a Xen hypervisor and
portability isn't a concern you might well just stick to the existing
well tested Xen PV interfaces.

--=20
Alex Benn=C3=A9e


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:21:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:21:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196663.349591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnlI-0006FO-EP; Mon, 27 Sep 2021 10:21:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196663.349591; Mon, 27 Sep 2021 10:21:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnlI-0006FH-A2; Mon, 27 Sep 2021 10:21:28 +0000
Received: by outflank-mailman (input) for mailman id 196663;
 Mon, 27 Sep 2021 10:21:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUnlH-0006FB-CQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:21:27 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::625])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae0fcefb-10de-47c9-9714-88976eb94604;
 Mon, 27 Sep 2021 10:21:23 +0000 (UTC)
Received: from AM7PR02CA0023.eurprd02.prod.outlook.com (2603:10a6:20b:100::33)
 by HE1PR08MB2940.eurprd08.prod.outlook.com (2603:10a6:7:30::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 10:21:20 +0000
Received: from VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:100:cafe::28) by AM7PR02CA0023.outlook.office365.com
 (2603:10a6:20b:100::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 10:21:20 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT044.mail.protection.outlook.com (10.152.19.106) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 10:21:20 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Mon, 27 Sep 2021 10:21:18 +0000
Received: from 6fdb01dcc6bb.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7D02ABD9-6273-4F1A-B77D-D7DB6BCFAA19.1; 
 Mon, 27 Sep 2021 10:21:06 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6fdb01dcc6bb.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 10:21:06 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB7PR08MB3531.eurprd08.prod.outlook.com (2603:10a6:10:49::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 10:21:05 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:21:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae0fcefb-10de-47c9-9714-88976eb94604
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g8GDnxrpLGbzERDVYWOOOIaiHQFHMWwNJxjNOwQVUjY=;
 b=UbZpt9LkqnzqrFKv/UWMry3Etm+c+qSoaIqY3jYiGUWFyLHA/kDqlHaoBXpOQW7v/UsJA1oq71D68LgcguQRaRLAu92kDISXHF+Cvqe37JwfJj4UVJ68hVhaVmdYWV0vw/JTqyQf6Cy85W5+WbCBNCXJ44E+mJE0gQOuaQrVY60=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AAaedwvqqP72fZcuk7YCUvOWaBAPA7o72yBO5PR3Jkz59wnO1XSQrKFK0rT/+CKmU1BSNWNaD9zksiWgjF6OWKjQJw2AlvEEuceXi9vSGAX5UzdEaaBHQ4ETzFVI3jV1ABFEJv1WUbpJUVjSwrJrB2L4Sgi/8K20VeZQAaRlQSHBEMp7xKk8LNwQ6avYe2DImFE9OY52X8857OrgwNTeSGw67S49XBSZi6cNkSyhfjc4QXYTez4QoEZIkXwTWMWSB5qsy2wQWA+TpbSMsirqDnsTWF3MhcDMj1Q5w55Od7fQPFhkdExFijOZoSzeoTg81X9RWLzMPUGDUBR0fji8ow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=g8GDnxrpLGbzERDVYWOOOIaiHQFHMWwNJxjNOwQVUjY=;
 b=OoKEJWaFAo/29r3pVU6w+kVrUXPNOTPa/frA8BDvw8lnFhcF8/zFGL+dCBpDWMEnm/rCAqU++czb2N6EcV6eLXg+V75/ASzsePdQCZusdiYkfFKIsuA/cRyBQoGklx57PSk+m6guXSezfAB59VGY0F/LDietSH5UAu4CrxTmRObMMzEjy7Ci4pvxXHkY9PATGPV7NpAnNxLrPVwEnoqKrPF3VTCql94FoaMTDaBXZFdb/EdzICPnhZWG/fyWpBbgjUtXoM8hjWHRB/eUU9shZpTW7TcK5wI+x565Okt3fQhy4gz0Az7dM5rTbK7gHL6APiVjEM+DlehTnaGr9OHrQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=g8GDnxrpLGbzERDVYWOOOIaiHQFHMWwNJxjNOwQVUjY=;
 b=UbZpt9LkqnzqrFKv/UWMry3Etm+c+qSoaIqY3jYiGUWFyLHA/kDqlHaoBXpOQW7v/UsJA1oq71D68LgcguQRaRLAu92kDISXHF+Cvqe37JwfJj4UVJ68hVhaVmdYWV0vw/JTqyQf6Cy85W5+WbCBNCXJ44E+mJE0gQOuaQrVY60=
From: Wei Chen <Wei.Chen@arm.com>
To: Julien Grall <julien.grall.oss@gmail.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index:
 AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CCAAP08AIAADO8QgAANUgCAABvh0IAAAsXQgAANBICAACxScA==
Date: Mon, 27 Sep 2021 10:21:05 +0000
Message-ID:
 <DB9PR08MB6857316DC540769819C8155E9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com>
In-Reply-To:
 <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 9581C7C154771C41BF2D6C031E00F1CA.0
x-checkrecipientchecked: true
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 7fdcd977-bb10-4352-8f39-08d981a08c93
x-ms-traffictypediagnostic: DB7PR08MB3531:|HE1PR08MB2940:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR08MB294009A63C4488459BAD8E409EA79@HE1PR08MB2940.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 0D/mmbzKTzjRUCeMlpC2o2SWRd7CshQKzzEJ1WDCRpXDJMher1A9dnF0wx3rlX/2Dqjl+EcCEYJslHF05RZ1rRX/hwdwLPLvvOsO4QGVec0zKNoGFsNnnQr1iPHT2njJgf7F32S620HXprtNEy8XtamdToRNvfAODybNh+1xia1y94T1Yo9lXxIk2ZITvSPjxPsoT+arMcQp8JIRdV1s0Nd5oQpmTaobdK016bgdOsYlDDNyyOMzxMON6OCgB3rLDkyqRIUhVDS6R4pHWyUyxcBormzNOaj11EP3iW6YpXINH0LRK7tcuMt3S8eAtccIA2NxHkS23siIAEtkP15LYm4KAeTyBvT8Xas3pSRbJMakY1+EPwTCnfTlz5RMtLT9mCi8sdQHjsaY8/hcJKMn+foY+rooOmzfysrAjT+mzTNR5Qc8pl+K/RUK6zxVWL9RbK1IV8CBvByp6/58AdVz13fuIiEi9XodBS/FeYblTOJOLXI9UQsig2N7l7hhIk/mqhCLB2KAW1Z2a+BU20ftO4yD+xPGnzKN+KQKtu5h7dQv481FF0+8j8XL7g46bxM1T/csoPEhGegNl1zGnYvI+L5gmp7w4MNFphzPM0GOL1yg4JUqxf0b3/Ku8sSEc1z/Nh9jAV3OX1HiqIX5NPWXneVxxkUuMz1goG+9cR84SrtXo8GB1OJLEF1gcgediTe+8eZrKsx6V2V1ltk4yDRf2Yg2pyE01qbQwCLBZNux04ewgACCMR+WvmMi06+fBJi5TWc2SAY/NBp/PFibVuQKdkzxqav1GHe6ULu7ePVbRTk=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(38070700005)(9686003)(7696005)(8676002)(55016002)(6506007)(8936002)(53546011)(186003)(76116006)(4326008)(83380400001)(6916009)(52536014)(66476007)(2906002)(71200400001)(54906003)(66556008)(66446008)(64756008)(26005)(966005)(86362001)(38100700002)(316002)(33656002)(508600001)(122000001)(66946007);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3531
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	49086bbf-7ff7-44d8-6ee3-08d981a083b3
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	g4QiCbFnRE3kCvLtYwBCwc+snL4t1OveOyAsY6WVTHtAIKOeB4ByF50B4Cw2/BTBkCtYKEkqcP1cv/ypzEvWGqVc3C8DKL/+xzNBthtkiNgxfM7IVdtHRWGMi7I0D3s4HyHRekgw/xlo8zTlHM2Tda24/9gRXBrwid05xlVVHHbsDMFBxBNE/ZccSUevh8xMzNfhipDXFhzVUvwGml1CTjMIGm+uFYLDaG9t5xViqGwDknsN+Tz5IFwsvfd15FQ16JCLQ2VT5JqrKQrDeJv/652iTHGyYS+L9GGWfIxSjQ1evLgAeMd1K7MHhsezTMN+Bhf43ue3V7lK638sdayWlY0FMC2Yg7ks/1E62dtaPIZCL3YuA0DEmE2PGl+f74FSxfwnEz0IdjlSWUK2UGAdjCnS649oAxIOycM9nAgjB4aykFG4WA/uVoa+3gg0FPyup0IsLMKhIJpj+OTchHyAPSZj1sJefFs8HBKoIhun+Va6qvGaDzc4K9ZbMZLVhIfrWUp+mdVZ6MyRK/VJDGJuS+6ntZ6vy+GkqQcV4+7HrIjmJTKmF11tt3QkARGG8xqpz2Yreg+b0bMwsEcxBZCr1VXLue0d4orxq+JkBJSbyK/tNZGWyndExu/M7Y5eOH28M9yTKnS9NwJgRN4LWmjAS2Xq6VHtCvwvBY88QnPXKIYk88vfxJdLICXosxjJJ/WHxK4QzzPxeDBghSrlYe84g8XYa+0r01A3LeZcbBBYaNhZaHSYwvuqH+MNGKE0KoghanwevGldUBn+mGrHzt9ZlRZol/n0TXTxiv9rE6MhAE8=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(336012)(8936002)(186003)(83380400001)(26005)(5660300002)(7696005)(2906002)(86362001)(53546011)(70586007)(9686003)(52536014)(6862004)(47076005)(70206006)(6506007)(8676002)(356005)(30864003)(107886003)(54906003)(508600001)(55016002)(33656002)(82310400003)(966005)(316002)(36860700001)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 10:21:20.1817
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7fdcd977-bb10-4352-8f39-08d981a08c93
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR08MB2940

SGkgSnVsaWVuLA0KDQpGcm9tOiBKdWxpZW4gR3JhbGwgPGp1bGllbi5ncmFsbC5vc3NAZ21haWwu
Y29tPiANClNlbnQ6IDIwMjHlubQ55pyIMjfml6UgMTU6MzYNClRvOiBXZWkgQ2hlbiA8V2VpLkNo
ZW5AYXJtLmNvbT4NCkNjOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5v
cmc+OyB4ZW4tZGV2ZWwgPHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZz47IEJlcnRyYW5k
IE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IEphbiBCZXVsaWNoIDxqYmV1bGlj
aEBzdXNlLmNvbT47IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2VyLnBhdUBjaXRyaXguY29tPjsgQW5k
cmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4NClN1YmplY3Q6IFJlOiBbUEFU
Q0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9NRU1fQkFOS1MgdG8gb3ZlcnJpZGUgZGVmYXVsdCBO
Ul9OT0RFX01FTUJMS1MNCg0KDQpPbiBNb24sIDI3IFNlcCAyMDIxLCAwODo1MyBXZWkgQ2hlbiwg
PG1haWx0bzpXZWkuQ2hlbkBhcm0uY29tPiB3cm90ZToNCkhpIEp1bGllbiwNCg0KPiAtLS0tLU9y
aWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBYZW4tZGV2ZWwgPG1haWx0bzp4ZW4tZGV2ZWwt
Ym91bmNlc0BsaXN0cy54ZW5wcm9qZWN0Lm9yZz4gT24gQmVoYWxmIE9mIFdlaQ0KPiBDaGVuDQo+
IFNlbnQ6IDIwMjHlubQ55pyIMjfml6UgMTQ6NDYNCj4gVG86IFN0ZWZhbm8gU3RhYmVsbGluaSA8
bWFpbHRvOnNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IENjOiBtYWlsdG86eGVuLWRldmVsQGxp
c3RzLnhlbnByb2plY3Qub3JnOyBtYWlsdG86anVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1
aXMNCj4gPG1haWx0bzpCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBtYWlsdG86amJldWxpY2hA
c3VzZS5jb207IG1haWx0bzpyb2dlci5wYXVAY2l0cml4LmNvbTsNCj4gbWFpbHRvOmFuZHJldy5j
b29wZXIzQGNpdHJpeC5jb20NCj4gU3ViamVjdDogUkU6IFtQQVRDSCAyMi8zN10geGVuL2FybTog
dXNlIE5SX01FTV9CQU5LUyB0byBvdmVycmlkZSBkZWZhdWx0DQo+IE5SX05PREVfTUVNQkxLUw0K
PiANCj4gSGkgU3RlZmFubywgSnVsaWVuLA0KPiANCj4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2Ut
LS0tLQ0KPiA+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8bWFpbHRvOnNzdGFiZWxsaW5pQGtl
cm5lbC5vcmc+DQo+ID4gU2VudDogMjAyMeW5tDnmnIgyN+aXpSAxMzowMA0KPiA+IFRvOiBXZWkg
Q2hlbiA8bWFpbHRvOldlaS5DaGVuQGFybS5jb20+DQo+ID4gQ2M6IFN0ZWZhbm8gU3RhYmVsbGlu
aSA8bWFpbHRvOnNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tDQo+ID4gbWFpbHRvOmRldmVs
QGxpc3RzLnhlbnByb2plY3Qub3JnOyBtYWlsdG86anVsaWVuQHhlbi5vcmc7IEJlcnRyYW5kIE1h
cnF1aXMNCj4gPiA8bWFpbHRvOkJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IG1haWx0bzpqYmV1
bGljaEBzdXNlLmNvbTsgbWFpbHRvOnJvZ2VyLnBhdUBjaXRyaXguY29tOw0KPiA+IG1haWx0bzph
bmRyZXcuY29vcGVyM0BjaXRyaXguY29tDQo+ID4gU3ViamVjdDogUkU6IFtQQVRDSCAyMi8zN10g
eGVuL2FybTogdXNlIE5SX01FTV9CQU5LUyB0byBvdmVycmlkZSBkZWZhdWx0DQo+ID4gTlJfTk9E
RV9NRU1CTEtTDQo+ID4NCj4gPiAreDg2IG1haW50YWluZXJzDQo+ID4NCj4gPiBPbiBNb24sIDI3
IFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdl
LS0tLS0NCj4gPiA+ID4gRnJvbTogU3RlZmFubyBTdGFiZWxsaW5pIDxtYWlsdG86c3N0YWJlbGxp
bmlAa2VybmVsLm9yZz4NCj4gPiA+ID4gU2VudDogMjAyMeW5tDnmnIgyN+aXpSAxMToyNg0KPiA+
ID4gPiBUbzogV2VpIENoZW4gPG1haWx0bzpXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiBDYzog
U3RlZmFubyBTdGFiZWxsaW5pIDxtYWlsdG86c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0N
Cj4gPiA+ID4gbWFpbHRvOmRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBtYWlsdG86anVsaWVu
QHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiA+ID4gPG1haWx0bzpCZXJ0cmFuZC5NYXJx
dWlzQGFybS5jb20+DQo+ID4gPiA+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06
IHVzZSBOUl9NRU1fQkFOS1MgdG8gb3ZlcnJpZGUNCj4gPiBkZWZhdWx0DQo+ID4gPiA+IE5SX05P
REVfTUVNQkxLUw0KPiA+ID4gPg0KPiA+ID4gPiBPbiBTdW4sIDI2IFNlcCAyMDIxLCBXZWkgQ2hl
biB3cm90ZToNCj4gPiA+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gPiA+
ID4gPiBGcm9tOiBTdGVmYW5vIFN0YWJlbGxpbmkgPG1haWx0bzpzc3RhYmVsbGluaUBrZXJuZWwu
b3JnPg0KPiA+ID4gPiA+ID4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSA5OjM1DQo+ID4gPiA+ID4g
PiBUbzogV2VpIENoZW4gPG1haWx0bzpXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gQ2M6
IG1haWx0bzp4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IG1haWx0bzpzc3RhYmVsbGlu
aUBrZXJuZWwub3JnOw0KPiA+ID4gPiBtYWlsdG86anVsaWVuQHhlbi5vcmc7DQo+ID4gPiA+ID4g
PiBCZXJ0cmFuZCBNYXJxdWlzIDxtYWlsdG86QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPg0KPiA+
ID4gPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01FTV9C
QU5LUyB0byBvdmVycmlkZQ0KPiA+ID4gPiBkZWZhdWx0DQo+ID4gPiA+ID4gPiBOUl9OT0RFX01F
TUJMS1MNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkg
Q2hlbiB3cm90ZToNCj4gPiA+ID4gPiA+ID4gQXMgYSBtZW1vcnkgcmFuZ2UgZGVzY3JpYmVkIGlu
IGRldmljZSB0cmVlIGNhbm5vdCBiZSBzcGxpdA0KPiA+IGFjcm9zcw0KPiA+ID4gPiA+ID4gPiBt
dWx0aXBsZSBub2Rlcy4gU28gd2UgZGVmaW5lIE5SX05PREVfTUVNQkxLUyBhcyBOUl9NRU1fQkFO
S1MNCj4gaW4NCj4gPiA+ID4gPiA+ID4gYXJjaCBoZWFkZXIuDQo+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gVGhpcyBzdGF0ZW1lbnQgaXMgdHJ1ZSBidXQgd2hhdCBpcyB0aGUgZ29hbCBvZiB0aGlz
IHBhdGNoPyBJcyBpdA0KPiA+IHRvDQo+ID4gPiA+ID4gPiByZWR1Y2UgY29kZSBzaXplIGFuZCBt
ZW1vcnkgY29uc3VtcHRpb24/DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gTm8s
IHdoZW4gSnVsaWVuIGFuZCBJIGRpc2N1c3NlZCB0aGlzIGluIGxhc3QgdmVyc2lvblsxXSwgd2Ug
aGFkbid0DQo+ID4gPiA+IHRob3VnaHQNCj4gPiA+ID4gPiBzbyBkZWVwbHkuIFdlIGp1c3QgdGhv
dWdodCBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQgaW4gRFQgY2Fubm90DQo+IGJlDQo+ID4gPiA+
IHNwbGl0DQo+ID4gPiA+ID4gYWNyb3NzIG11bHRpcGxlIG5vZGVzLiBTbyBOUl9NRU1fQkFOS1Mg
c2hvdWxkIGJlIGVxdWFsIHRvDQo+ID4gTlJfTUVNX0JBTktTLg0KPiA+ID4gPiA+DQo+ID4gPiA+
ID4gaHR0cHM6Ly9saXN0cy54ZW5wcm9qZWN0Lm9yZy9hcmNoaXZlcy9odG1sL3hlbi1kZXZlbC8y
MDIxLQ0KPiA+ID4gPiAwOC9tc2cwMDk3NC5odG1sDQo+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEkg
YW0gYXNraW5nIGJlY2F1c2UgTlJfTUVNX0JBTktTIGlzIDEyOCBhbmQNCj4gPiA+ID4gPiA+IE5S
X05PREVfTUVNQkxLUz0yKk1BWF9OVU1OT0RFUyB3aGljaCBpcyA2NCBieSBkZWZhdWx0IHNvIGFn
YWluDQo+ID4gPiA+ID4gPiBOUl9OT0RFX01FTUJMS1MgaXMgMTI4IGJlZm9yZSB0aGlzIHBhdGNo
Lg0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IEluIG90aGVyIHdvcmRzLCB0aGlzIHBhdGNoIGFs
b25lIGRvZXNuJ3QgbWFrZSBhbnkgZGlmZmVyZW5jZTsgYXQNCj4gPiBsZWFzdA0KPiA+ID4gPiA+
ID4gZG9lc24ndCBtYWtlIGFueSBkaWZmZXJlbmNlIHVubGVzcyBDT05GSUdfTlJfTlVNQV9OT0RF
UyBpcw0KPiA+IGluY3JlYXNlZC4NCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBTbywgaXMgdGhl
IGdvYWwgdG8gcmVkdWNlIG1lbW9yeSB1c2FnZSB3aGVuIENPTkZJR19OUl9OVU1BX05PREVTDQo+
ID4gaXMNCj4gPiA+ID4gPiA+IGhpZ2hlciB0aGFuIDY0Pw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4g
Pg0KPiA+ID4gPiA+IEkgYWxzbyB0aG91Z2h0IGFib3V0IHRoaXMgcHJvYmxlbSB3aGVuIEkgd2Fz
IHdyaXRpbmcgdGhpcyBwYXRjaC4NCj4gPiA+ID4gPiBDT05GSUdfTlJfTlVNQV9OT0RFUyBpcyBp
bmNyZWFzaW5nLCBidXQgTlJfTUVNX0JBTktTIGlzIGEgZml4ZWQNCj4gPiA+ID4gPiB2YWx1ZSwg
dGhlbiBOUl9NRU1fQkFOS1MgY2FuIGJlIHNtYWxsZXIgdGhhbiBDT05GSUdfTlJfTlVNQV9OT0RF
Uw0KPiA+ID4gPiA+IGF0IG9uZSBwb2ludC4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IEJ1dCBJIGFn
cmVlIHdpdGggSnVsaWVuJ3Mgc3VnZ2VzdGlvbiwgTlJfTUVNX0JBTktTIGFuZA0KPiA+IE5SX05P
REVfTUVNQkxLUw0KPiA+ID4gPiA+IG11c3QgYmUgYXdhcmUgb2YgZWFjaCBvdGhlci4gSSBoYWQg
dGhvdWdodCB0byBhZGQgc29tZSBBU1NFUlQNCj4gY2hlY2ssDQo+ID4gPiA+ID4gYnV0IEkgZG9u
J3Qga25vdyBob3cgdG8gZG8gaXQgYmV0dGVyLiBTbyBJIHBvc3QgdGhpcyBwYXRjaCBmb3INCj4g
bW9yZQ0KPiA+ID4gPiA+IHN1Z2dlc3Rpb24uDQo+ID4gPiA+DQo+ID4gPiA+IE9LLiBJbiB0aGF0
IGNhc2UgSSdkIHNheSB0byBnZXQgcmlkIG9mIHRoZSBwcmV2aW91cyBkZWZpbml0aW9uIG9mDQo+
ID4gPiA+IE5SX05PREVfTUVNQkxLUyBhcyBpdCBpcyBwcm9iYWJseSBub3QgbmVjZXNzYXJ5LCBz
ZWUgYmVsb3cuDQo+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+DQo+ID4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4gPiBBbmQga2VlcCBkZWZhdWx0IE5SX05PREVfTUVNQkxLUyBpbiBjb21tb24gaGVhZGVy
DQo+ID4gPiA+ID4gPiA+IGZvciB0aG9zZSBhcmNoaXRlY3R1cmVzIE5VTUEgaXMgZGlzYWJsZWQu
DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gVGhpcyBsYXN0IHNlbnRlbmNlIGlzIG5vdCBhY2N1
cmF0ZTogb24geDg2IE5VTUEgaXMgZW5hYmxlZCBhbmQNCj4gPiA+ID4gPiA+IE5SX05PREVfTUVN
QkxLUyBpcyBzdGlsbCBkZWZpbmVkIGluIHhlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4gKHRoZXJl
DQo+ID4gaXMNCj4gPiA+ID4gbm8NCj4gPiA+ID4gPiA+IHg4NiBkZWZpbml0aW9uIG9mIGl0KQ0K
PiA+ID4gPiA+ID4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IFllcy4NCj4gPiA+ID4gPg0KPiA+ID4g
PiA+ID4NCj4gPiA+ID4gPiA+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPG1haWx0bzp3ZWku
Y2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gPiAtLS0NCj4gPiA+ID4gPiA+ID7CoCB4ZW4vaW5j
bHVkZS9hc20tYXJtL251bWEuaCB8IDggKysrKysrKy0NCj4gPiA+ID4gPiA+ID7CoCB4ZW4vaW5j
bHVkZS94ZW4vbnVtYS5owqAgwqAgwqB8IDIgKysNCj4gPiA+ID4gPiA+ID7CoCAyIGZpbGVzIGNo
YW5nZWQsIDkgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KPiA+ID4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmggYi94ZW4v
aW5jbHVkZS9hc20tDQo+ID4gYXJtL251bWEuaA0KPiA+ID4gPiA+ID4gPiBpbmRleCA4ZjFjNjdl
M2ViLi4yMTU2OWU2MzRiIDEwMDY0NA0KPiA+ID4gPiA+ID4gPiAtLS0gYS94ZW4vaW5jbHVkZS9h
c20tYXJtL251bWEuaA0KPiA+ID4gPiA+ID4gPiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL251
bWEuaA0KPiA+ID4gPiA+ID4gPiBAQCAtMyw5ICszLDE1IEBADQo+ID4gPiA+ID4gPiA+DQo+ID4g
PiA+ID4gPiA+wqAgI2luY2x1ZGUgPHhlbi9tbS5oPg0KPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+
ID4gPiArI2luY2x1ZGUgPGFzbS9zZXR1cC5oPg0KPiA+ID4gPiA+ID4gPiArDQo+ID4gPiA+ID4g
PiA+wqAgdHlwZWRlZiB1OCBub2RlaWRfdDsNCj4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4g
LSNpZm5kZWYgQ09ORklHX05VTUENCj4gPiA+ID4gPiA+ID4gKyNpZmRlZiBDT05GSUdfTlVNQQ0K
PiA+ID4gPiA+ID4gPiArDQo+ID4gPiA+ID4gPiA+ICsjZGVmaW5lIE5SX05PREVfTUVNQkxLUyBO
Ul9NRU1fQkFOS1MNCj4gPiA+ID4gPiA+ID4gKw0KPiA+ID4gPiA+ID4gPiArI2Vsc2UNCj4gPiA+
ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID7CoCAvKiBGYWtlIG9uZSBub2RlIGZvciBub3cuIFNlZSBh
bHNvIG5vZGVfb25saW5lX21hcC4gKi8NCj4gPiA+ID4gPiA+ID7CoCAjZGVmaW5lIGNwdV90b19u
b2RlKGNwdSkgMA0KPiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUveGVuL251
bWEuaCBiL3hlbi9pbmNsdWRlL3hlbi9udW1hLmgNCj4gPiA+ID4gPiA+ID4gaW5kZXggMTk3OGUy
YmUxYi4uMTczMWUxY2M2YiAxMDA2NDQNCj4gPiA+ID4gPiA+ID4gLS0tIGEveGVuL2luY2x1ZGUv
eGVuL251bWEuaA0KPiA+ID4gPiA+ID4gPiArKysgYi94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+
ID4gPiA+ID4gPiA+IEBAIC0xMiw3ICsxMiw5IEBADQo+ID4gPiA+ID4gPiA+wqAgI2RlZmluZSBN
QVhfTlVNTk9ERVPCoCDCoCAxDQo+ID4gPiA+ID4gPiA+wqAgI2VuZGlmDQo+ID4gPiA+ID4gPiA+
DQo+ID4gPiA+ID4gPiA+ICsjaWZuZGVmIE5SX05PREVfTUVNQkxLUw0KPiA+ID4gPiA+ID4gPsKg
ICNkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIChNQVhfTlVNTk9ERVMqMikNCj4gPiA+ID4gPiA+ID4g
KyNlbmRpZg0KPiA+ID4gPg0KPiA+ID4gPiBUaGlzIG9uZSB3ZSBjYW4gcmVtb3ZlIGl0IGNvbXBs
ZXRlbHkgcmlnaHQ/DQo+ID4gPg0KPiA+ID4gSG93IGFib3V0IGRlZmluZSBOUl9NRU1fQkFOS1Mg
dG86DQo+ID4gPiAjaWZkZWYgQ09ORklHX05SX05VTUFfTk9ERVMNCj4gPiA+ICNkZWZpbmUgTlJf
TUVNX0JBTktTIChDT05GSUdfTlJfTlVNQV9OT0RFUyAqIDIpDQo+ID4gPiAjZWxzZQ0KPiA+ID4g
I2RlZmluZSBOUl9NRU1fQkFOS1MgMTI4DQo+ID4gPiAjZW5kaWYNCj4gPiA+IGZvciBib3RoIHg4
NiBhbmQgQXJtLiBGb3IgdGhvc2UgYXJjaGl0ZWN0dXJlcyBkbyBub3Qgc3VwcG9ydCBvciBlbmFi
bGUNCj4gPiA+IE5VTUEsIHRoZXkgY2FuIHN0aWxsIHVzZSAiTlJfTUVNX0JBTktTIDEyOCIuIEFu
ZCByZXBsYWNlIGFsbA0KPiA+IE5SX05PREVfTUVNQkxLUw0KPiA+ID4gaW4gTlVNQSBjb2RlIHRv
IE5SX01FTV9CQU5LUyB0byByZW1vdmUgTlJfTk9ERV9NRU1CTEtTIGNvbXBsZXRlbHkuDQo+ID4g
PiBJbiB0aGlzIGNhc2UsIE5SX01FTV9CQU5LUyBjYW4gYmUgYXdhcmUgb2YgdGhlIGNoYW5nZXMg
b2YNCj4gPiBDT05GSUdfTlJfTlVNQV9OT0RFUy4NCj4gPg0KPiA+IHg4NiBkb2Vzbid0IGhhdmUg
TlJfTUVNX0JBTktTIGFzIGZhciBhcyBJIGNhbiB0ZWxsLiBJIGd1ZXNzIHlvdSBhbHNvDQo+ID4g
bWVhbnQgdG8gcmVuYW1lIE5SX05PREVfTUVNQkxLUyB0byBOUl9NRU1fQkFOS1M/DQo+ID4NCj4g
DQo+IFllcy4NCj4gDQo+ID4gQnV0IE5SX01FTV9CQU5LUyBpcyBub3QgZGlyZWN0bHkgcmVsYXRl
ZCB0byBDT05GSUdfTlJfTlVNQV9OT0RFUyBiZWNhdXNlDQo+ID4gdGhlcmUgY2FuIGJlIG1hbnkg
bWVtb3J5IGJhbmtzIGZvciBlYWNoIG51bWEgbm9kZSwgY2VydGFpbmx5IG1vcmUgdGhhbg0KPiA+
IDIuIFRoZSBleGlzdGluZyBkZWZpbml0aW9uIG9uIHg4NjoNCj4gPg0KPiA+ICNkZWZpbmUgTlJf
Tk9ERV9NRU1CTEtTIChNQVhfTlVNTk9ERVMqMikNCj4gPg0KPiA+IERvZXNuJ3QgbWFrZSBhIGxv
dCBvZiBzZW5zZSB0byBtZS4gV2FzIGl0IGp1c3QgYW4gYXJiaXRyYXJ5IGxpbWl0IGZvcg0KPiA+
IHRoZSBsYWNrIG9mIGEgYmV0dGVyIHdheSB0byBzZXQgYSBtYXhpbXVtPw0KPiA+DQo+IA0KPiBB
dCB0aGF0IHRpbWUsIHRoaXMgd2FzIHByb2JhYmx5IHRoZSBtb3N0IGNvc3QtZWZmZWN0aXZlIGFw
cHJvYWNoLg0KPiBFbm91Z2ggYW5kIGVhc3kuIEJ1dCwgaWYgbW9yZSBub2RlcyBuZWVkIHRvIGJl
IHN1cHBvcnRlZCBpbiB0aGUNCj4gZnV0dXJlLCBpdCBtYXkgYnJpbmcgbW9yZSBtZW1vcnkgYmxv
Y2tzLiBBbmQgdGhpcyBtYXhpbXVtIHZhbHVlDQo+IG1pZ2h0IG5vdCBhcHBseS4gVGhlIG1heGlt
dW0gbWF5IG5lZWQgdG8gc3VwcG9ydCBkeW5hbWljIGV4dGVuc2lvbi4NCj4gDQo+ID4NCj4gPiBP
biB0aGUgb3RoZXIgaGFuZCwgTlJfTUVNX0JBTktTIGFuZCBOUl9OT0RFX01FTUJMS1Mgc2VlbSB0
byBiZSByZWxhdGVkLg0KPiA+IEluIGZhY3QsIHdoYXQncyB0aGUgZGlmZmVyZW5jZT8NCj4gPg0K
PiA+IE5SX01FTV9CQU5LUyBpcyB0aGUgbWF4IG51bWJlciBvZiBtZW1vcnkgYmFua3MgKHdpdGgg
b3Igd2l0aG91dA0KPiA+IG51bWEtbm9kZS1pZCkuDQo+ID4NCj4gPiBOUl9OT0RFX01FTUJMS1Mg
aXMgdGhlIG1heCBudW1iZXIgb2YgbWVtb3J5IGJhbmtzIHdpdGggTlVNQSBzdXBwb3J0DQo+ID4g
KHdpdGggbnVtYS1ub2RlLWlkKT8NCj4gPg0KPiA+IFRoZXkgYXJlIGJhc2ljYWxseSB0aGUgc2Ft
ZSB0aGluZy4gT24gQVJNIEkgd291bGQganVzdCBkbzoNCj4gPg0KPiANCj4gUHJvYmFibHkgbm90
LCBOUl9NRU1fQkFOS1Mgd2lsbCBjb3VudCB0aG9zZSBtZW1vcnkgcmFuZ2VzIHdpdGhvdXQNCj4g
bnVtYS1ub2RlLWlkIGluIGJvb3QgbWVtb3J5IHBhcnNpbmcgc3RhZ2UgKHByb2Nlc3NfbWVtb3J5
X25vZGUgb3INCj4gRUZJIHBhcnNlcikuIEJ1dCBOUl9OT0RFX01FTUJMS1Mgd2lsbCBvbmx5IGNv
dW50IHRob3NlIG1lbW9yeSByYW5nZXMNCj4gd2l0aCBudW1hLW5vZGUtaWQuDQo+IA0KPiA+ICNk
ZWZpbmUgTlJfTk9ERV9NRU1CTEtTIE1BWChOUl9NRU1fQkFOS1MsIChDT05GSUdfTlJfTlVNQV9O
T0RFUyAqIDIpKQ0KPiA+DQo+ID4NCg0KPiBRdW90ZSBKdWxpZW4ncyBjb21tZW50IGZyb20gSFRN
TCBlbWFpbCB0byBoZXJlOg0KPiAiIEFzIHlvdSB3cm90ZSBhYm92ZSwgdGhlIHNlY29uZCBwYXJ0
IG9mIHRoZSBNQVggaXMgdG90YWxseSBhcmJpdHJhcnkuDQo+IEluIGZhY3QsIGl0IGlzIHZlcnkg
bGlrZWx5IHRoYW4gaWYgeW91IGhhdmUgbW9yZSB0aGFuIDY0IG5vZGVzLCB5b3UgbWF5DQo+IG5l
ZWQgYSBsb3QgbW9yZSB0aGFuIDIgcmVnaW9ucyBwZXIgbm9kZS4NCj4gDQo+IFNvLCBmb3IgQXJt
LCBJIHdvdWxkIGp1c3QgZGVmaW5lIE5SX05PREVfTUVNQkxLUyBhcyBhbiBhbGlhcyB0byBOUl9N
RU1fQkFOS1MNCj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgY29tbW9uIGNvZGUuDQo+ICINCj4gDQo+
ID4gQnV0IGhlcmUgY29tZXMgdGhlIHByb2JsZW06DQo+ID4gSG93IGNhbiB3ZSBzZXQgdGhlIE5S
X01FTV9CQU5LUyBtYXhpbXVtIHZhbHVlLCAxMjggc2VlbXMgYW4gYXJiaXRyYXJ5IHRvbz8NCj4g
DQo+IFRoaXMgaXMgYmFzZWQgb24gaGFyZHdhcmUgd2UgY3VycmVudGx5IHN1cHBvcnQgKHRoZSBs
YXN0IHRpbWUgd2UgYnVtcGVkIHRoZSB2YWx1ZSB3YXMsIElJUkMsIGZvciBUaHVuZGVyLVgpLiBJ
biB0aGUgY2FzZSBvZiBib290aW5nIFVFRkksIHdlIGNhbiBnZXQgYSBsb3Qgb2Ygc21hbGwgcmFu
Z2VzIGFzIHdlIGRpc2NvdmVyIHRoZSBSQU0gdXNpbmcgdGhlIFVFRkkgbWVtb3J5IG1hcC4NCj4g
DQoNClRoYW5rcyBmb3IgdGhlIGJhY2tncm91bmQuDQoNCj4gDQo+ID4gSWYgI2RlZmluZSBOUl9N
RU1fQkFOS1MgKENPTkZJR19OUl9OVU1BX05PREVTICogTik/IEFuZCB3aGF0IE4gc2hvdWxkIGJl
Lg0KPiANCj4gTiB3b3VsZCBoYXZlIHRvIGJlIHRoZSBtYXhpbXVtIG51bWJlciBvZiByYW5nZXMg
eW91IGNhbiBmaW5kIGluIGEgTlVNQSBub2RlLg0KPiANCj4gV2Ugd291bGQgYWxzbyBuZWVkIHRv
IG1ha2Ugc3VyZSB0aGlzIGRvZXNuJ3QgYnJlYWsgZXhpc3RpbmcgcGxhdGZvcm1zLiBTbyBOIHdv
dWxkIGhhdmUgdG8gYmUgcXVpdGUgbGFyZ2Ugb3Igd2UgbmVlZCBhIE1BWCBhcyBTdGVmYW5vIHN1
Z2dlc3RlZC4NCj4gDQo+IEJ1dCBJIHdvdWxkIHByZWZlciB0byBrZWVwIHRoZSBleGlzdGluZyAx
MjggYW5kIGFsbG93IHRvIGNvbmZpZ3VyZSBpdCBhdCBidWlsZCB0aW1lIChub3QgbmVjZXNzYXJp
bHkgaW4gdGhpcyBzZXJpZXMpLiBUaGlzIGF2b2lkIHRvIGhhdmUgZGlmZmVyZW50IHdheSB0byBk
ZWZpbmUgdGhlIHZhbHVlIGJhc2VkIE5VTUEgdnMgbm9uLU5VTUEuDQoNCkluIHRoaXMgY2FzZSwg
Y2FuIHdlIHVzZSBTdGVmYW5vJ3MNCiIjZGVmaW5lIE5SX05PREVfTUVNQkxLUyBNQVgoTlJfTUVN
X0JBTktTLCAoQ09ORklHX05SX05VTUFfTk9ERVMgKiAyKSkiDQppbiBuZXh0IHZlcnNpb24uIElm
IHllcywgc2hvdWxkIHdlIGNoYW5nZSB4ODYgcGFydD8gQmVjYXVzZSBOUl9NRU1fQkFOS1MNCmhh
cyBub3QgYmVlbiBkZWZpbmVkIGluIHg4Ni4NCg0KPiA+IEFuZCBtYXliZSB0aGUgZGVmaW5pdGlv
biBjb3VsZCBiZSBjb21tb24gd2l0aCB4ODYgaWYgd2UgZGVmaW5lDQo+ID4gTlJfTUVNX0JBTktT
IHRvIDEyOCBvbiB4ODYgdG9vLg0KPiANCj4gSnVsaWVuIGhhZCBjb21tZW50IGhlcmUsIEkgd2ls
bCBjb250aW51ZSBpbiB0aGF0IGVtYWlsLg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:26:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:26:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196669.349602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnqE-0006uK-2p; Mon, 27 Sep 2021 10:26:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196669.349602; Mon, 27 Sep 2021 10:26:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUnqD-0006uD-Un; Mon, 27 Sep 2021 10:26:33 +0000
Received: by outflank-mailman (input) for mailman id 196669;
 Mon, 27 Sep 2021 10:26:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUnqC-0006u7-F4
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:26:32 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6174ac66-1f7d-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 10:26:31 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2052.outbound.protection.outlook.com [104.47.13.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-RDFUha55OfuIfF9dDF9szg-1; Mon, 27 Sep 2021 12:26:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3934.eurprd04.prod.outlook.com (2603:10a6:803:1b::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.20; Mon, 27 Sep
 2021 10:26:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:26:26 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0021.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 10:26:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6174ac66-1f7d-11ec-bc44-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632738390;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YUY1eXGShgXO0KNmTzeyUwyq/wBL8aX1DbFFl/+h4R0=;
	b=Vry0M0oYlvGlXmg8R2ntkWxZ1HZX39jCNfQc7/eoCkXNXnMNGevwsQkimUU9+1A+Nn2a3C
	uzJkZBxiRHDIqbI/OvXoOVBugDhjOE56GFl+TuciSZVt++U9l8Xonv7AZpNO6ONPcygL3k
	gHxW9xPf+jMhEXO8LWmIlhVgGRi6qRo=
X-MC-Unique: RDFUha55OfuIfF9dDF9szg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W+MxufCO1QpkO07WRHLgKhX+2PQhH5Fm1t+hD3AXtlyR6/0ZECb8GMVaLmac0XW+nBKVs6v6tStt2Hf0iBFJpxdWDfg+/faHlGBH9rjkkt7tFubuxUzW0ie/jgcecujBCttNrwPIJ2J3QBocqxa3hVj9l5xisd1mMNCTrSVm9Z107SLTJQHfUjm0TP1L7SqN0MqRtVpl+C9txReY5pLjvtdxJO2o00UVyvje0ZQgYV1seSZ50rj+N2FQ9lfjIkYfq56/TM4zxPzbR0ryWrVySjvzy2oQjfNc2gjkXzgIpXNa69mvRHeHpjx4pS/nrlIc3BdjtNi7UQAVqP1JXlEzig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YUY1eXGShgXO0KNmTzeyUwyq/wBL8aX1DbFFl/+h4R0=;
 b=VJ3wnKay3upwtx8xmzuTbd91+EVKpS39AktO9wAYJLmbMmR4fDv3smvLNy6gWyIfqVwQYahdq5XbKeiBO0gyAPU9gDWNEjDXyLYb2kdpcOv1pkS96HQJSNG6FP3Cf0WX1AS2fJlg0y/gLvZhpYBtHVOPDsovsLfXzrOKxWqKe47CWS76gbw5ewUtguX/B0McsFxWogiL/SsTImTIJCTUeOjRoz/0LWZ4Kr4Yl+SRegqgAcIXI8fbUAGHDwJRMqxrfFpnZl3Lwd2VCXFUxJqMH1zGNZhp5BIirqHYSQzuKv73PsGHkT7rj50BLPAdzntQz3Le4hU3Y0ufJVr6LGuubw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
Date: Mon, 27 Sep 2021 12:26:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P193CA0021.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 929964b6-6702-43b3-5398-08d981a1433a
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3934:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3934C3D326F9BE60DCC62AB6B3A79@VI1PR0402MB3934.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rL0olxL2iDI11Mw/qJl8OQAk7cKQ90bSMdeObHL1rYng4IHH2kRL0K6lhPSne2BU2BpWLTjJ1fT+bM4t44XW1/O1doJRu9jRYjvqifxLhlJqMQSn/mdI0XFNdiPd2kVKxrip6Ikfu4dT198UoNHNmMVcR2DUHpenmz6BZ4c4WG5BKa5uZLj4Emx7eTvWMe9PdbWGSxMNIuVH0OI5EKTsApT+fVjLKL+FUBGNfWYUknRCzHHyXb0dX50dRwAtpZ6vXkQODt25/VV81csmr25uQ4ycDmhUMNV1ZkeTdZB3PR6JHKjC5W7FWTSLqtb7Bg4QjnwUffplaAb0AmbDXHREdgJPIZV/0TtJ9DhUTvGYeCJEWts5C+s0ZtPCZGYNRl+lrZYo2xVQPlrdngodWGbuIWPPC6KESFrq+owEljyy5+ndBaaMXGuBaoZbUWcayBv2hZhkxjVoNgmLRfmCEEQSzQeJjSmmauwNypMupvKD7nNjD4gFB1fus/HIGMxVT+GCaeqAbjPLLxeXB4gGHW6ODnmy+eQUz8qwMZZ7pQFk3DqQUds9sOVCJzSrhwJFI92OLpirjzlqv35UH5Ewhpd6u2qDWRdIQT36FDX+kJnWe2PrBjRJku7on3QQt2js8KCmTeH1fI6kY58QaZXCaRoq8hG8Jl/Xw9qgJdU7IBa0ej8zwjTITc3tx7ETbsy+dq7GgsES/Wv2Fj/TnDcS6vAjxb8x5aiYfLrce+2BinjtwkHve8RFggNHVk0v4X6yWukg
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(956004)(38100700002)(66946007)(6486002)(36756003)(66476007)(508600001)(66556008)(53546011)(31696002)(26005)(54906003)(86362001)(186003)(2906002)(8676002)(31686004)(8936002)(2616005)(6916009)(4326008)(316002)(7416002)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDg0VnJtNUY4YWd6c0VXaHBGU1g1aTU2endHbkV5WTFPZk5xVmplYWp6L3FI?=
 =?utf-8?B?NHRFOVFTbnE2UUpWMFh2RXExcnRndlU1Mmkxb0NjR0Y2c3ZWOTN2VEVVYnZv?=
 =?utf-8?B?c054TFpyV0pHWVV2dFRsRVJNSFpjczFkNUV4cDBveUlaU1pmZ3VMVXRvb0J1?=
 =?utf-8?B?SXpvQmxLZGhiaVU1bmVwaXlNOGp0N3NxeGVHVWdxWDc2bHFtSk16QStKOXl4?=
 =?utf-8?B?eDB0Kzljc3ZoRE9mV1pERHdBMzdvWWkyTEc1bFVGNFVzL0V2dStwNk1Rajd0?=
 =?utf-8?B?emhmckhNQ2FGWGZRQ3luL0hYa1VRdEE0alNDNHNTNHJ3bitSK1UzYzBXb0xj?=
 =?utf-8?B?ZWt6bmZSN21tWXBYbHlzTHk4QTlCZytMRHpkZzFqQzFXanJvVG1LQkZYZm81?=
 =?utf-8?B?Y1Zjam1BMmdsdnlIRkM5NzI0UDhSbTVmOVpBdDRPSngzUUExRDZza3RXRnBX?=
 =?utf-8?B?TFNqdkcvODVaQXRKV3pPMGxlaUd2SmRWYTFhenFmaVJScmZNNXgrOVVPZmRH?=
 =?utf-8?B?ZmtFT3N6SXpMdnpuOXJteWdPclVHdU9ndTMvNkJ3K0tBaXEvWlhnZ1JwL0tq?=
 =?utf-8?B?WTVUQmN6WDNETmZoMGN2UUpJZEgxSW9YWnlwL2wxRlFsVXAxNVNVQWE1clF2?=
 =?utf-8?B?MVk2bGhwWGwvYmNYYituRGhsTWpOdjNZK1g5YjhyTzA4eXRmQnRaNXpTdzdZ?=
 =?utf-8?B?aC9HajFYd08rMFlzRm1hVWYrZGR5RDA1aDJjUXBHRlYwS2swSFdtTXkxZXFi?=
 =?utf-8?B?aFFaNkR5L3lySWZGcDBReVErRVdwdUFrdnR3cVdYTGpOZE1EZGl3SCtIQlZF?=
 =?utf-8?B?R3pKT0xhaDZPTGQ0ZDRqd2EvNEtiTVdWd1piNnErUmZYSS90RWFlQjI3Z2hq?=
 =?utf-8?B?TUFENldOOEJsL2g1eEFiT21SWWJlS1RSaEg2VGJrV2Q2Y2JVM1ordUFmamZE?=
 =?utf-8?B?bGxSYnJPTElMZ011UUprcXZsQWdLcEd6YmdoYlh6aTZDbnpwaGQvOHc0akNq?=
 =?utf-8?B?SGh1VTNCdkE5bEltSW5BVHBhYTB0Q0RQaDVMSm5TR2JyMlV3ak80T3RWR1R3?=
 =?utf-8?B?dENEVkFwRXlrbVlxeEZ5dGlFeHk2bUtleGlVT1VIRDI2VFJLNnhPRVZobWw4?=
 =?utf-8?B?aUlWaWw2ZUUraXpMRm9sSTRqMVNHaHFOWmNzeFJZQitIZXBOUURDeFlYemtX?=
 =?utf-8?B?ejBYbXcxcUx4RndDNzVNMHJ0TEdVMHJ6UGxaTFA3WnMxUkluSFJNZEhRTk9L?=
 =?utf-8?B?WVh1elpocDEzY0J0K3h3QThUMGZOb1dnL3VucnpPM1VETWdhR3RzeSt1SnBy?=
 =?utf-8?B?cTFXRVBmL3dJQUhpNkdyakVDUDM2SDBNUkk3d2gyN0RYeWJsaVJ5WDVob2Jk?=
 =?utf-8?B?Q3ZOVW9iQWJ6VS9wMGVMY3hFNDE0dFVzaEFHZUF4WjVQRkttdkNsR0ZDRVMz?=
 =?utf-8?B?MmZJclNEaUsrMjNtUWF1dDZ2WlBaY0dkUW83MjVLZlNZM2ZhRmpQVUQrbE5t?=
 =?utf-8?B?WEcrM3FTRmlEdVVPc2VYc0RpWUh5d1BZK0JNY05YaDZjYWhOMU56a2R6Smtw?=
 =?utf-8?B?SGRKR295OUZTeEordmdXOEJwUFpzdUVWRFFwbUJwMmNjdStoOHA4bXJiQmla?=
 =?utf-8?B?SUlmZTAwYi95MEhPMk4wMUt2Mk5uUzhLcXI1TkpSWGNSc0JIYXg4WTMxOFNy?=
 =?utf-8?B?Q1k0Q0IyTEtoQjFtbXplajZQaDdDRnRwaVJYbUVGaWRUUFRTQnRyRWFPZ3RI?=
 =?utf-8?Q?gVmBn9q9n63tlClpa2pjr76M7mqU2m1wueYfLMm?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 929964b6-6702-43b3-5398-08d981a1433a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 10:26:26.8418
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JQoNspEhaEcBfANLcQcMdjz1IMLVQQbDY3/NcQRMLhmqjDcIxeAI3BlJ0xFGWzlwZw523AMDxTnOwX4xxLNG4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3934

On 27.09.2021 12:04, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 13:00, Jan Beulich wrote:
>> On 27.09.2021 11:35, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 12:19, Jan Beulich wrote:
>>>> On 27.09.2021 10:45, Oleksandr Andrushchenko wrote:
>>>>> On 27.09.21 10:45, Jan Beulich wrote:
>>>>>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>>>> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>>>>>>>         *((u8*) &pdev->bus) = bus;
>>>>>>>         *((u8*) &pdev->devfn) = devfn;
>>>>>>>         pdev->domain = NULL;
>>>>>>> +#ifdef CONFIG_ARM
>>>>>>> +    pci_to_dev(pdev)->type = DEV_PCI;
>>>>>>> +#endif
>>>>>> I have to admit that I'm not happy about new CONFIG_<arch> conditionals
>>>>>> here. I'd prefer to see this done by a new arch helper, unless there are
>>>>>> obstacles I'm overlooking.
>>>>> Do you mean something like arch_pci_alloc_pdev(dev)?
>>>> I'd recommend against "alloc" in its name; "new" instead maybe?
>>> I am fine with arch_pci_new_pdev, but arch prefix points to the fact that
>>> this is just an architecture specific part of the pdev allocation rather than
>>> actual pdev allocation itself, so with this respect arch_pci_alloc_pdev seems
>>> more natural to me.
>> The bulk of the function is about populating the just allocated struct.
>> There's no arch-specific part of the allocation (so far, leaving aside
>> MSI-X), you only want and arch-specific part of the initialization. I
>> would agree with "alloc" in the name if further allocation was to
>> happen there.
> Hm, then arch_pci_init_pdev sounds more reasonable

Fine with me.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:28:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:28:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196678.349613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUns4-0007nO-IK; Mon, 27 Sep 2021 10:28:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196678.349613; Mon, 27 Sep 2021 10:28:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUns4-0007nH-EZ; Mon, 27 Sep 2021 10:28:28 +0000
Received: by outflank-mailman (input) for mailman id 196678;
 Mon, 27 Sep 2021 10:28:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xxNw=OR=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mUns3-0007n9-SM
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:28:27 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.40]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc42aca2-1f43-4f06-8309-a1e0a88dfae0;
 Mon, 27 Sep 2021 10:28:25 +0000 (UTC)
Received: from DU2PR04CA0308.eurprd04.prod.outlook.com (2603:10a6:10:2b5::13)
 by DB7PR08MB3865.eurprd08.prod.outlook.com (2603:10a6:10:74::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 10:28:22 +0000
Received: from DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b5:cafe::1e) by DU2PR04CA0308.outlook.office365.com
 (2603:10a6:10:2b5::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 10:28:22 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT061.mail.protection.outlook.com (10.152.21.234) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 10:28:22 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Mon, 27 Sep 2021 10:28:21 +0000
Received: from 4899b7a5c6ad.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 56A48B6A-F6AA-46D0-B004-881EE5B454FB.1; 
 Mon, 27 Sep 2021 10:28:10 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4899b7a5c6ad.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 10:28:10 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB4316.eurprd08.prod.outlook.com (2603:10a6:10:c6::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 10:28:07 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 10:28:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc42aca2-1f43-4f06-8309-a1e0a88dfae0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EHcV30bC/NM+1UXbmY9tpGQ7gTKZXACVUKrEOY5FYig=;
 b=GfVHKmDIufvi0Zxq6h8fuuqd+tdww2RmHAqmTdHB7nu6Luc4RNsD9dEKloA07b84RLUibNqWYKm42s/0H6fvGzD2roZldZTEdnxPNkCrEtc8KU+iSDnidKUnDmNWDRBgh6K6OSqGDDwbqEKVlt7MViHC3pC+P8ORd7bdHf2OjXs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JYPQlvf0TxF7jqHPOLmEHZQG738akV6vx3lcBGiGLPqqeRyA1bFRKAk2z1fN3lDC3/v2QgjpYdnpFLk7sVFihgRUD6YVyEn1Opn1pYm4w98n3mQ52gHJFwa1sbq2zNnaNigcZ9MEfA4PuTx2RHyFnSmer+D3CjwWIGZQ7dmcaK7Hc/xa/4Nir1p696YdLRupSx7T2bNZKCnDhu8uyV6FQzeTnMpR7QRvO8jk/ofvFyvEkN9/WtWq/p3WB4yqKbYcUv/yfPmiZPVtVZfU0o2OOhH4Bgz2iHszOVcYT+DK70ByIt61qw1rrSXM0w8Xw9e9SU+6+/DAbPIOAXrLL57ceg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EHcV30bC/NM+1UXbmY9tpGQ7gTKZXACVUKrEOY5FYig=;
 b=MV1gykMxGgfTN0VXaR9O6vrPt5ymnUBDZEYu+aLP/QIyOI5wGjkYonU9hxD4Hz9kq8eoXYSvw1pePSEX20+pC8DCqH22bocLlO2Lu7bFuusbOhKAEbU/QGJa9zSYO966SgribLIEfLgwCNl4pF6Fi652BvExzKsAwQaR1GaByPsf54WzHjBDo/qg1cIqmNk0UVQ3da+fE8AMvFhRbpXnK41eLClzg73Gb9rMkAedxprcb0KbdeInNu+VRVo9sLKRUMjCEf2Qigzs/Roh49PMMPcZcZnaVnUpLjhY/8D0bnRmi3wrUEpHIcCS5e7ga2vNjhBfXYuMNgvAS27HdrYu0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EHcV30bC/NM+1UXbmY9tpGQ7gTKZXACVUKrEOY5FYig=;
 b=GfVHKmDIufvi0Zxq6h8fuuqd+tdww2RmHAqmTdHB7nu6Luc4RNsD9dEKloA07b84RLUibNqWYKm42s/0H6fvGzD2roZldZTEdnxPNkCrEtc8KU+iSDnidKUnDmNWDRBgh6K6OSqGDDwbqEKVlt7MViHC3pC+P8ORd7bdHf2OjXs=
From: Wei Chen <Wei.Chen@arm.com>
To: Wei Chen <Wei.Chen@arm.com>, Jan Beulich <jbeulich@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Stefano Stabellini <sstabellini@kernel.org>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Topic: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Index:
 AQHXsHMtR+l18+MmpE6rl/RjSXTBgquyYmkAgAA2xQCAADnbAIAAKWhQgAAGUwCAAxtPwIABk+Mw
Date: Mon, 27 Sep 2021 10:28:07 +0000
Message-ID:
 <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
 <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
In-Reply-To:
 <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: D62B2DFA3FA4B740989B8750363DDE61.0
x-checkrecipientchecked: true
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 97d8e9b0-0380-41ee-46c4-08d981a18800
x-ms-traffictypediagnostic: DBBPR08MB4316:|DB7PR08MB3865:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB3865CB577AA0EB0B59FE0FC69EA79@DB7PR08MB3865.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YR7Rv6dXF4M+1N+lC3aEtmXPllUGx2a+jX+H9ckyftcztI2Go2avFRe/hqJ4cvYm2qP4nFFE0G5wji18CJ24cifhWh7qsAEx54xgcrBreh/AIz0OkKYLJAJhHlNTKOyScjkJ1CpX9vcs9XO3JcnOqFVntcn1hf8n6W6s8yMp4NzvLe5FJ0Bi2WnqgYH3FpdxLQVSCWHPa0ElN67BGjFMBk/EkFp/669C27zKdaii8EVP8SjudIXIjNDxIgc3k+3TxW1hV1rcDmOUrclfCj0l0E8tXc430gUGvr/OQHaJysst7OsCs5kkbPN7ggILv+uSVOMZbK8q4fYjtr48HjhFM0iuj3/Da89IJjgNcACewX94f4UOGtVz3vChcbHrnB8o8F0ROVRpaYzj/ccUy4aYatweirC9MUoPTd9R+VkgYSkxB/RfJMaApSdhJ633YsQbo7CoW6yiriU89f27kEP774SFBSuUtAQ3nPY7MyTB24rKjYpJQ0MSV1DzwCrdRKzpkqHm5qdnRQv1xpzjJf2hQEH8LJfzKMAWU5qXNgce9vcjEMG0lipH5flXoKoK3rBe10zvFcoSSC9tN+Hs2cOF+cTKhHz+KABzmcFX4RaFsnfGaBpFNu++2gq2E2Kuku36rzfepOEF4yW4KWy2QM5eG4FTFIs5A4JNyoK0Rh1EVxApqO9c7RGtgaTG1vhSXdw58zMtVDzYopOFhaLmfqlu+2Jad5gx+NfY1ywlRSNmbYIhFwJzq6QWlFh0/94vJbRX5zYtu8pyEAafMOGWUuwXKnzvdyfZj8K7H0Jf8MeGt8Q=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(86362001)(54906003)(8936002)(966005)(7696005)(8676002)(508600001)(5660300002)(9686003)(6506007)(53546011)(83380400001)(55016002)(38070700005)(66476007)(33656002)(316002)(66446008)(64756008)(66556008)(76116006)(26005)(122000001)(110136005)(38100700002)(4326008)(71200400001)(52536014)(186003)(66946007);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4316
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ca9d317d-c2ca-45d7-d9bf-08d981a17f6f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	U9JhNjk5v1ml2qKcC26XqhebBGBcwtTNXHlHNJfsxrKTmxPpyklK0saRonutJjj4q7SCPWemkt+qrUwXyk1KmDrI9xwJbqjAu47Bx/ljkFxhg2sRerz2ybF97oncNsPUONDx1A3wG0cjcb36IaANNO5hCLfFTC5bod+hCkcmIk+MUevg6VNkkFYsuXi2I5YrNqRb9BnLqYrfv54jYwFCj/oKCGVTlPk483o2ShvdolifKoj4e8CDYEGvj+QwxgCZ70/nYn5WGF4bOGTMPu0CTebMPftMWp4VktbkhmhZuCAp0fJ+bQe7C7IzQE6zhnEX9d6X79x6aba0QprWoZrMQ/hhXHZ7UxzTmbAlEz3aRTwgrFJH6N+BprGnWdtq+I3pD71D8FS8seLp9SqAqOD5+dG628FXNj2ekaDl77tI7P6ikjVTzjfwAufNm+80U8fMUCytUWPUc+uv68fwT1DFkRl2kj/h6SHVi0jdTirKpKilqPCXxaglK7McVERWst7GUw0+KxC5MMlaONUqmSwjBmL20s26y5UqFZKxr5/+BfRSlrHSMYAz5Eyz10x4XLLQ+B0gdRJyyUSVpjSbK8DJ4vAuTvtNjPUJN+q2pGO4ajHqGSjGxMnDFY3voOQ5cr2BjNa4DqjbjC8sbPjQxVA7+u9XGOP5H+g9h6mNmeVwivjdRLzA9p40CfhoFJH9BoQjEkS7P+YnTuIsknsHNQlvEiQCdkFUkh3fEjYmAVioq+UU4Qi/h1IXYFIaCeC0lFs2sVGYK3q8bVZmkGrNcmyJn/O+XNRkxpFQNcicdau2tF8=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(9686003)(70586007)(356005)(36860700001)(2906002)(110136005)(47076005)(55016002)(81166007)(8676002)(8936002)(70206006)(26005)(54906003)(7696005)(82310400003)(52536014)(53546011)(966005)(6506007)(107886003)(186003)(33656002)(5660300002)(86362001)(4326008)(83380400001)(336012)(316002)(508600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 10:28:22.1013
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 97d8e9b0-0380-41ee-46c4-08d981a18800
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT061.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3865

SGkgSnVsaWVuLCBTdGVmYW5vLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy
b206IFhlbi1kZXZlbCA8eGVuLWRldmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9u
IEJlaGFsZiBPZiBXZWkNCj4gQ2hlbg0KPiBTZW50OiAyMDIxxOo51MIyNsjVIDE4OjI1DQo+IFRv
OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+DQo+IENjOiB4ZW4tZGV2ZWxAbGlzdHMu
eGVucHJvamVjdC5vcmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+IDxCZXJ0
cmFuZC5NYXJxdWlzQGFybS5jb20+OyBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtl
cm5lbC5vcmc+DQo+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMjAvMzddIHhlbjogaW50cm9kdWNlIENP
TkZJR19FRkkgdG8gc3R1YiBBUEkgZm9yIG5vbi0NCj4gRUZJIGFyY2hpdGVjdHVyZQ0KPiANCj4g
SGkgSmFuLA0KPiANCj4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IFhl
bi1kZXZlbCA8eGVuLWRldmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFs
ZiBPZg0KPiBKYW4NCj4gPiBCZXVsaWNoDQo+ID4gU2VudDogMjAyMcTqOdTCMjTI1SAxODo0OQ0K
PiA+IFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiBDYzogeGVuLWRldmVsQGxp
c3RzLnhlbnByb2plY3Qub3JnOyBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcw0KPiA+
IDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxs
aW5pQGtlcm5lbC5vcmc+DQo+ID4gU3ViamVjdDogUmU6IFtQQVRDSCAyMC8zN10geGVuOiBpbnRy
b2R1Y2UgQ09ORklHX0VGSSB0byBzdHViIEFQSSBmb3INCj4gbm9uLQ0KPiA+IEVGSSBhcmNoaXRl
Y3R1cmUNCj4gPg0KPiA+IE9uIDI0LjA5LjIwMjEgMTI6MzEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+
ID4+IEZyb206IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gPiA+PiBTZW50OiAy
MDIxxOo51MIyNMjVIDE1OjU5DQo+ID4gPj4NCj4gPiA+PiBPbiAyNC4wOS4yMDIxIDA2OjM0LCBX
ZWkgQ2hlbiB3cm90ZToNCj4gPiA+Pj4+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJl
bGxpbmlAa2VybmVsLm9yZz4NCj4gPiA+Pj4+IFNlbnQ6IDIwMjHE6jnUwjI0yNUgOToxNQ0KPiA+
ID4+Pj4NCj4gPiA+Pj4+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+
ID4+Pj4+IC0tLSBhL3hlbi9jb21tb24vS2NvbmZpZw0KPiA+ID4+Pj4+ICsrKyBiL3hlbi9jb21t
b24vS2NvbmZpZw0KPiA+ID4+Pj4+IEBAIC0xMSw2ICsxMSwxNiBAQCBjb25maWcgQ09NUEFUDQo+
ID4gPj4+Pj4gIGNvbmZpZyBDT1JFX1BBUktJTkcNCj4gPiA+Pj4+PiAgCWJvb2wNCj4gPiA+Pj4+
Pg0KPiA+ID4+Pj4+ICtjb25maWcgRUZJDQo+ID4gPj4+Pj4gKwlib29sDQo+ID4gPj4+Pg0KPiA+
ID4+Pj4gV2l0aG91dCB0aGUgdGl0bGUgdGhlIG9wdGlvbiBpcyBub3QgdXNlci1zZWxlY3RhYmxl
IChvciBkZS0NCj4gPiBzZWxlY3RhYmxlKS4NCj4gPiA+Pj4+IFNvIHRoZSBoZWxwIG1lc3NhZ2Ug
YmVsb3cgY2FuIG5ldmVyIGJlIHNlZW4uDQo+ID4gPj4+Pg0KPiA+ID4+Pj4gRWl0aGVyIGFkZCBh
IHRpdGxlLCBlLmcuOg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IGJvb2wgIkVGSSBzdXBwb3J0Ig0KPiA+
ID4+Pj4NCj4gPiA+Pj4+IE9yIGZ1bGx5IG1ha2UgdGhlIG9wdGlvbiBhIHNpbGVudCBvcHRpb24g
YnkgcmVtb3ZpbmcgdGhlIGhlbHAgdGV4dC4NCj4gPiA+Pj4NCj4gPiA+Pj4gT0ssIGluIGN1cnJl
bnQgWGVuIGNvZGUsIEVGSSBpcyB1bmNvbmRpdGlvbmFsbHkgY29tcGlsZWQuIEJlZm9yZQ0KPiA+
ID4+PiB3ZSBjaGFuZ2UgcmVsYXRlZCBjb2RlLCBJIHByZWZlciB0byByZW1vdmUgdGhlIGhlbHAg
dGV4dC4NCj4gPiA+Pg0KPiA+ID4+IEJ1dCB0aGF0J3Mgbm90IHRydWU6IEF0IGxlYXN0IG9uIHg4
NiBFRkkgZ2V0cyBjb21waWxlZCBkZXBlbmRpbmcgb24NCj4gPiA+PiB0b29sIGNoYWluIGNhcGFi
aWxpdGllcy4gVWx0aW1hdGVseSB3ZSBtYXkgaW5kZWVkIHdhbnQgYSB1c2VyDQo+ID4gPj4gc2Vs
ZWN0YWJsZSBvcHRpb24gaGVyZSwgYnV0IHVudGlsIHRoZW4gSSdtIGFmcmFpZCBoYXZpbmcgdGhp
cyBvcHRpb24NCj4gPiA+PiBhdCBhbGwgbWF5IGJlIG1pc2xlYWRpbmcgb24geDg2Lg0KPiA+ID4+
DQo+ID4gPg0KPiA+ID4gSSBjaGVjayB0aGUgYnVpbGQgc2NyaXB0cywgeWVzLCB5b3UncmUgcmln
aHQuIEZvciB4ODYsIEVGSSBpcyBub3QgYQ0KPiA+ID4gc2VsZWN0YWJsZSBvcHRpb24gaW4gS2Nv
bmZpZy4gSSBhZ3JlZSB3aXRoIHlvdSwgd2UgY2FuJ3QgdXNlIEtjb25maWcNCj4gPiA+IHN5c3Rl
bSB0byBkZWNpZGUgdG8gZW5hYmxlIEVGSSBidWlsZCBmb3IgeDg2IG9yIG5vdC4NCj4gPiA+DQo+
ID4gPiBTbyBob3cgYWJvdXQgd2UganVzdCB1c2UgdGhpcyBFRkkgb3B0aW9uIGZvciBBcm0gb25s
eT8gQmVjYXVzZSBvbiBBcm0sDQo+ID4gPiB3ZSBkbyBub3QgaGF2ZSBzdWNoIHRvb2xjaGFpbiBk
ZXBlbmRlbmN5Lg0KPiA+DQo+ID4gVG8gYmUgaG9uZXN0IC0gZG9uJ3Qga25vdy4gVGhhdCdzIGJl
Y2F1c2UgSSBkb24ndCBrbm93IHdoYXQgeW91IHdhbnQNCj4gPiB0byB1c2UgdGhlIG9wdGlvbiBm
b3Igc3Vic2VxdWVudGx5Lg0KPiA+DQo+IA0KPiBJbiBsYXN0IHZlcnNpb24sIEkgaGFkIGludHJv
ZHVjZWQgYW4gYXJjaC1oZWxwZXIgdG8gc3R1YiBFRklfQk9PVA0KPiBpbiBBcm0ncyBjb21tb24g
Y29kZSBmb3IgQXJtMzIuIEJlY2F1c2UgQXJtMzIgZG9lc24ndCBzdXBwb3J0IEVGSS4NCj4gU28g
SnVsaWVuIHN1Z2dlc3RlZCBtZSB0byBpbnRyb2R1Y2UgYSBDT05GSUdfRUZJIG9wdGlvbiBmb3Ig
bm9uLUVGSQ0KPiBzdXBwb3J0ZWQgYXJjaGl0ZWN0dXJlcyB0byBzdHViIGluIEVGSSBsYXllci4N
Cj4gDQo+IFsxXSBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hpdmVzL2h0bWwveGVu
LWRldmVsLzIwMjEtDQo+IDA4L21zZzAwODA4Lmh0bWwNCj4gDQoNCkFzIEphbicgcmVtaW5kZWQs
IHg4NiBkb2Vzbid0IGRlcGVuZCBvbiBLY29uZmlnIHRvIGJ1aWxkIEVGSSBjb2RlLg0KU28sIGlm
IHdlIENPTkZJR19FRkkgdG8gc3R1YiBFRkkgQVBJJ3MgZm9yIHg4Niwgd2Ugd2lsbCBlbmNvdW50
ZXINCnRoYXQgdG9vbGNoYWlucyBlbmFibGUgRUZJLCBidXQgS2NvbmZpZyBkaXNhYmxlIEVGSS4g
T3IgS2NvbmZpZw0KZW5hYmxlIEVGSSBidXQgdG9vbGNoYWluIGRvZXNuJ3QgcHJvdmlkZSBFRkkg
YnVpbGQgc3VwcG9ydHMuIEFuZA0KdGhlbiB4ODYgY291bGQgbm90IHdvcmsgd2VsbC4NCg0KSWYg
d2UgdXNlIENPTkZJR19FRkkgZm9yIEFybSBvbmx5LCB0aGF0IG1lYW5zIENPTkZJR19FRkkgZm9y
IHg4Ng0KaXMgb2ZmLCB0aGlzIHdpbGwgYWxzbyBjYXVzZSBwcm9ibGVtLg0KDQpTbywgY2FuIHdl
IHN0aWxsIHVzZSBwcmV2aW91cyBhcmNoX2hlbHBlcnMgdG8gc3R1YiBmb3IgQXJtMzI/DQp1bnRp
bCB4ODYgY2FuIHVzZSB0aGlzIHNlbGVjdGFibGUgb3B0aW9uPw0KDQo+ID4gSmFuDQo+ID4NCg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:40:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:40:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196687.349624 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUo3H-0001g1-ME; Mon, 27 Sep 2021 10:40:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196687.349624; Mon, 27 Sep 2021 10:40:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUo3H-0001fO-Ib; Mon, 27 Sep 2021 10:40:03 +0000
Received: by outflank-mailman (input) for mailman id 196687;
 Mon, 27 Sep 2021 10:40:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rT7O=OR=gmail.com=julien.grall@srs-us1.protection.inumbo.net>)
 id 1mUo3G-0001V3-Es
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:40:02 +0000
Received: from mail-oi1-x231.google.com (unknown [2607:f8b0:4864:20::231])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 008d67c5-ae9e-4f5b-994c-51083a48528e;
 Mon, 27 Sep 2021 10:39:59 +0000 (UTC)
Received: by mail-oi1-x231.google.com with SMTP id e24so15062268oig.11
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 03:39:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 008d67c5-ae9e-4f5b-994c-51083a48528e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=OB4tnKWET1q9U/7EbB1ovFbkJ0Rz+j6x5E3x6eSDVbE=;
        b=T+WuS8wW3hq4DovQKI4udHWt/6L2K9EMrKp4B1uYYcXk+FwZXJXDROmCmemKCrHNiG
         Php1djzDkfNWyKGrVBaDhhYzalTy8eo29EQiOK8R2eU5FpKhpSwHuu2emxZRluYyM1Bx
         XY/RdqzilRv2hrUH37fHVZ7WIBxD4i9l+8I+Qpv/s97/4JEC8L3TBF/T/pEvXlX7WB5M
         CF8EKIm/0ylh+kz0awL5gCQ2ZPKgJxOZKCpuXqlk+jIhECuzK0SgaSqdOnjesjeC26hN
         luxu4U9TyJJBk2EWvyRrHIA2mfYn4WxqfKafYWoLmsn87KqNyMKdtr3IajD036g+Xl/D
         Sbbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=OB4tnKWET1q9U/7EbB1ovFbkJ0Rz+j6x5E3x6eSDVbE=;
        b=cnn3mFqtXt/DPu7EMKyxB2fO7jhYGendUT3ScNl0DyRg2FMUO+LU1gUkLO7d1w7SH9
         q7W+LvoMdfdMNk7DLOL8e2Op5kk8ARyAOIMyL1RuyB24A0mgvC+wmf37QhkfwCwTDTaI
         NuwJ1sehRMaWQ/ZjE7QGTTKCrIJqN086JtQgCFIfBSDUnS6ahH7VyEJGrheh6x2DV/X9
         wFxwfR4YyX8pDd/5xnmNaMMT23krgLYqA1bAzag4NdaREeabvDau2M/LIaVxwa68se0P
         0Vc0LGlrOkaKWcnNP29+A/Q56m1nDN7l68ymppgOLyI2wMxSnbSSAlrjm0XJzEvBkrJl
         BbAg==
X-Gm-Message-State: AOAM533Irjl/41OqaCETWTQUi6+CgjfeNMW756XvevmqEvGGB6yyhg+S
	IfuwiYOY69gjKLLQIKN5BC7uFk9tjs/wCtr0kqs=
X-Google-Smtp-Source: ABdhPJwxmU8SQv0k4R5Stztam8sQ1v17wQ5MXWUykj5HTFo3qZhEm1MNf3uoT8in6MXBWNb/m+UK9d03LYDyMU2JtX4=
X-Received: by 2002:aca:c6c7:: with SMTP id w190mr11376304oif.96.1632739199132;
 Mon, 27 Sep 2021 03:39:59 -0700 (PDT)
MIME-Version: 1.0
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com> <DB9PR08MB6857316DC540769819C8155E9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
In-Reply-To: <DB9PR08MB6857316DC540769819C8155E9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
From: Julien Grall <julien.grall@gmail.com>
Date: Mon, 27 Sep 2021 14:39:48 +0400
Message-ID: <CAF3u54BNbPPo4_ezC2ft36NwodKDFSdtv_ekT-FFyvRrU3gQaA@mail.gmail.com>
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
To: Wei Chen <Wei.Chen@arm.com>
Cc: Julien Grall <julien.grall.oss@gmail.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000f9a62205ccf7b7d3"

--000000000000f9a62205ccf7b7d3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 27 Sep 2021, 12:22 Wei Chen, <Wei.Chen@arm.com> wrote:

> Hi Julien,
>
> From: Julien Grall <julien.grall.oss@gmail.com>
> Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 15:36
> To: Wei Chen <Wei.Chen@arm.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-devel <
> xen-devel@lists.xenproject.org>; Bertrand Marquis <
> Bertrand.Marquis@arm.com>; Jan Beulich <jbeulich@suse.com>; Roger Pau
> Monn=C3=A9 <roger.pau@citrix.com>; Andrew Cooper <andrew.cooper3@citrix.c=
om>
> Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
> NR_NODE_MEMBLKS
>
>
> On Mon, 27 Sep 2021, 08:53 Wei Chen, <mailto:Wei.Chen@arm.com> wrote:
> Hi Julien,
>
> > -----Original Message-----
> > From: Xen-devel <mailto:xen-devel-bounces@lists.xenproject.org> On
> Behalf Of Wei
> > Chen
> > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 14:46
> > To: Stefano Stabellini <mailto:sstabellini@kernel.org>
> > Cc: mailto:xen-devel@lists.xenproject.org; mailto:julien@xen.org;
> Bertrand Marquis
> > <mailto:Bertrand.Marquis@arm.com>; mailto:jbeulich@suse.com; mailto:
> roger.pau@citrix.com;
> > mailto:andrew.cooper3@citrix.com
> > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override defaul=
t
> > NR_NODE_MEMBLKS
> >
> > Hi Stefano, Julien,
> >
> > > -----Original Message-----
> > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
> > > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 13:00
> > > To: Wei Chen <mailto:Wei.Chen@arm.com>
> > > Cc: Stefano Stabellini <mailto:sstabellini@kernel.org>; xen-
> > > mailto:devel@lists.xenproject.org; mailto:julien@xen.org; Bertrand
> Marquis
> > > <mailto:Bertrand.Marquis@arm.com>; mailto:jbeulich@suse.com; mailto:
> roger.pau@citrix.com;
> > > mailto:andrew.cooper3@citrix.com
> > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> default
> > > NR_NODE_MEMBLKS
> > >
> > > +x86 maintainers
> > >
> > > On Mon, 27 Sep 2021, Wei Chen wrote:
> > > > > -----Original Message-----
> > > > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
> > > > > Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26
> > > > > To: Wei Chen <mailto:Wei.Chen@arm.com>
> > > > > Cc: Stefano Stabellini <mailto:sstabellini@kernel.org>; xen-
> > > > > mailto:devel@lists.xenproject.org; mailto:julien@xen.org;
> Bertrand Marquis
> > > > > <mailto:Bertrand.Marquis@arm.com>
> > > > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
> > > default
> > > > > NR_NODE_MEMBLKS
> > > > >
> > > > > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
> > > > > > > Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35
> > > > > > > To: Wei Chen <mailto:Wei.Chen@arm.com>
> > > > > > > Cc: mailto:xen-devel@lists.xenproject.org; mailto:
> sstabellini@kernel.org;
> > > > > mailto:julien@xen.org;
> > > > > > > Bertrand Marquis <mailto:Bertrand.Marquis@arm.com>
> > > > > > > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to
> override
> > > > > default
> > > > > > > NR_NODE_MEMBLKS
> > > > > > >
> > > > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > > > As a memory range described in device tree cannot be split
> > > across
> > > > > > > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANK=
S
> > in
> > > > > > > > arch header.
> > > > > > >
> > > > > > > This statement is true but what is the goal of this patch? Is
> it
> > > to
> > > > > > > reduce code size and memory consumption?
> > > > > > >
> > > > > >
> > > > > > No, when Julien and I discussed this in last version[1], we
> hadn't
> > > > > thought
> > > > > > so deeply. We just thought a memory range described in DT canno=
t
> > be
> > > > > split
> > > > > > across multiple nodes. So NR_MEM_BANKS should be equal to
> > > NR_MEM_BANKS.
> > > > > >
> > > > > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > > > > 08/msg00974.html
> > > > > >
> > > > > > > I am asking because NR_MEM_BANKS is 128 and
> > > > > > > NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 by default so ag=
ain
> > > > > > > NR_NODE_MEMBLKS is 128 before this patch.
> > > > > > >
> > > > > > > In other words, this patch alone doesn't make any difference;
> at
> > > least
> > > > > > > doesn't make any difference unless CONFIG_NR_NUMA_NODES is
> > > increased.
> > > > > > >
> > > > > > > So, is the goal to reduce memory usage when
> CONFIG_NR_NUMA_NODES
> > > is
> > > > > > > higher than 64?
> > > > > > >
> > > > > >
> > > > > > I also thought about this problem when I was writing this patch=
.
> > > > > > CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
> > > > > > value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NOD=
ES
> > > > > > at one point.
> > > > > >
> > > > > > But I agree with Julien's suggestion, NR_MEM_BANKS and
> > > NR_NODE_MEMBLKS
> > > > > > must be aware of each other. I had thought to add some ASSERT
> > check,
> > > > > > but I don't know how to do it better. So I post this patch for
> > more
> > > > > > suggestion.
> > > > >
> > > > > OK. In that case I'd say to get rid of the previous definition of
> > > > > NR_NODE_MEMBLKS as it is probably not necessary, see below.
> > > > >
> > > > >
> > > > >
> > > > > > >
> > > > > > > > And keep default NR_NODE_MEMBLKS in common header
> > > > > > > > for those architectures NUMA is disabled.
> > > > > > >
> > > > > > > This last sentence is not accurate: on x86 NUMA is enabled an=
d
> > > > > > > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h
> > (there
> > > is
> > > > > no
> > > > > > > x86 definition of it)
> > > > > > >
> > > > > >
> > > > > > Yes.
> > > > > >
> > > > > > >
> > > > > > > > Signed-off-by: Wei Chen <mailto:wei.chen@arm.com>
> > > > > > > > ---
> > > > > > > >  xen/include/asm-arm/numa.h | 8 +++++++-
> > > > > > > >  xen/include/xen/numa.h     | 2 ++
> > > > > > > >  2 files changed, 9 insertions(+), 1 deletion(-)
> > > > > > > >
> > > > > > > > diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-
> > > arm/numa.h
> > > > > > > > index 8f1c67e3eb..21569e634b 100644
> > > > > > > > --- a/xen/include/asm-arm/numa.h
> > > > > > > > +++ b/xen/include/asm-arm/numa.h
> > > > > > > > @@ -3,9 +3,15 @@
> > > > > > > >
> > > > > > > >  #include <xen/mm.h>
> > > > > > > >
> > > > > > > > +#include <asm/setup.h>
> > > > > > > > +
> > > > > > > >  typedef u8 nodeid_t;
> > > > > > > >
> > > > > > > > -#ifndef CONFIG_NUMA
> > > > > > > > +#ifdef CONFIG_NUMA
> > > > > > > > +
> > > > > > > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
> > > > > > > > +
> > > > > > > > +#else
> > > > > > > >
> > > > > > > >  /* Fake one node for now. See also node_online_map. */
> > > > > > > >  #define cpu_to_node(cpu) 0
> > > > > > > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.=
h
> > > > > > > > index 1978e2be1b..1731e1cc6b 100644
> > > > > > > > --- a/xen/include/xen/numa.h
> > > > > > > > +++ b/xen/include/xen/numa.h
> > > > > > > > @@ -12,7 +12,9 @@
> > > > > > > >  #define MAX_NUMNODES    1
> > > > > > > >  #endif
> > > > > > > >
> > > > > > > > +#ifndef NR_NODE_MEMBLKS
> > > > > > > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > > > > > > > +#endif
> > > > >
> > > > > This one we can remove it completely right?
> > > >
> > > > How about define NR_MEM_BANKS to:
> > > > #ifdef CONFIG_NR_NUMA_NODES
> > > > #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)
> > > > #else
> > > > #define NR_MEM_BANKS 128
> > > > #endif
> > > > for both x86 and Arm. For those architectures do not support or
> enable
> > > > NUMA, they can still use "NR_MEM_BANKS 128". And replace all
> > > NR_NODE_MEMBLKS
> > > > in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.
> > > > In this case, NR_MEM_BANKS can be aware of the changes of
> > > CONFIG_NR_NUMA_NODES.
> > >
> > > x86 doesn't have NR_MEM_BANKS as far as I can tell. I guess you also
> > > meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?
> > >
> >
> > Yes.
> >
> > > But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES
> because
> > > there can be many memory banks for each numa node, certainly more tha=
n
> > > 2. The existing definition on x86:
> > >
> > > #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
> > >
> > > Doesn't make a lot of sense to me. Was it just an arbitrary limit for
> > > the lack of a better way to set a maximum?
> > >
> >
> > At that time, this was probably the most cost-effective approach.
> > Enough and easy. But, if more nodes need to be supported in the
> > future, it may bring more memory blocks. And this maximum value
> > might not apply. The maximum may need to support dynamic extension.
> >
> > >
> > > On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be relate=
d.
> > > In fact, what's the difference?
> > >
> > > NR_MEM_BANKS is the max number of memory banks (with or without
> > > numa-node-id).
> > >
> > > NR_NODE_MEMBLKS is the max number of memory banks with NUMA support
> > > (with numa-node-id)?
> > >
> > > They are basically the same thing. On ARM I would just do:
> > >
> >
> > Probably not, NR_MEM_BANKS will count those memory ranges without
> > numa-node-id in boot memory parsing stage (process_memory_node or
> > EFI parser). But NR_NODE_MEMBLKS will only count those memory ranges
> > with numa-node-id.
> >
> > > #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))
> > >
> > >
>
> > Quote Julien's comment from HTML email to here:
> > " As you wrote above, the second part of the MAX is totally arbitrary.
> > In fact, it is very likely than if you have more than 64 nodes, you may
> > need a lot more than 2 regions per node.
> >
> > So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to
> NR_MEM_BANKS
> > so it can be used by common code.
> > "
> >
> > > But here comes the problem:
> > > How can we set the NR_MEM_BANKS maximum value, 128 seems an arbitrary
> too?
> >
> > This is based on hardware we currently support (the last time we bumped
> the value was, IIRC, for Thunder-X). In the case of booting UEFI, we can
> get a lot of small ranges as we discover the RAM using the UEFI memory ma=
p.
> >
>
> Thanks for the background.
>
> >
> > > If #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * N)? And what N should
> be.
> >
> > N would have to be the maximum number of ranges you can find in a NUMA
> node.
> >
> > We would also need to make sure this doesn't break existing platforms.
> So N would have to be quite large or we need a MAX as Stefano suggested.
> >
> > But I would prefer to keep the existing 128 and allow to configure it a=
t
> build time (not necessarily in this series). This avoid to have different
> way to define the value based NUMA vs non-NUMA.
>
> In this case, can we use Stefano's
> "#define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))"
> in next version. If yes, should we change x86 part? Because NR_MEM_BANKS
> has not been defined in x86.


What I meant by configuring dynamically is allowing NR_MEM_BANKS to be set
by the user.

The second part of the MAX makes no sense to me (at least on Arm). So I
really prefer if this is not part of the initial version.

We can refine the value, or introduce the MAX in the future if we have a
justification for it.


> > > And maybe the definition could be common with x86 if we define
> > > NR_MEM_BANKS to 128 on x86 too.
> >
> > Julien had comment here, I will continue in that email.
>

--000000000000f9a62205ccf7b7d3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Mon, 27 Sep 2021, 12:22 Wei Chen, &lt;<a href=3D"ma=
ilto:Wei.Chen@arm.com">Wei.Chen@arm.com</a>&gt; wrote:<br></div><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">Hi Julien,<br>
<br>
From: Julien Grall &lt;<a href=3D"mailto:julien.grall.oss@gmail.com" target=
=3D"_blank" rel=3D"noreferrer">julien.grall.oss@gmail.com</a>&gt; <br>
Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 15:36<br>
To: Wei Chen &lt;<a href=3D"mailto:Wei.Chen@arm.com" target=3D"_blank" rel=
=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
Cc: Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.org" target=
=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;; xen-devel &l=
t;<a href=3D"mailto:xen-devel@lists.xenproject.org" target=3D"_blank" rel=
=3D"noreferrer">xen-devel@lists.xenproject.org</a>&gt;; Bertrand Marquis &l=
t;<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"_blank" rel=3D"nore=
ferrer">Bertrand.Marquis@arm.com</a>&gt;; Jan Beulich &lt;<a href=3D"mailto=
:jbeulich@suse.com" target=3D"_blank" rel=3D"noreferrer">jbeulich@suse.com<=
/a>&gt;; Roger Pau Monn=C3=A9 &lt;<a href=3D"mailto:roger.pau@citrix.com" t=
arget=3D"_blank" rel=3D"noreferrer">roger.pau@citrix.com</a>&gt;; Andrew Co=
oper &lt;<a href=3D"mailto:andrew.cooper3@citrix.com" target=3D"_blank" rel=
=3D"noreferrer">andrew.cooper3@citrix.com</a>&gt;<br>
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR=
_NODE_MEMBLKS<br>
<br>
<br>
On Mon, 27 Sep 2021, 08:53 Wei Chen, &lt;mailto:<a href=3D"mailto:Wei.Chen@=
arm.com" target=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt; wrot=
e:<br>
Hi Julien,<br>
<br>
&gt; -----Original Message-----<br>
&gt; From: Xen-devel &lt;mailto:<a href=3D"mailto:xen-devel-bounces@lists.x=
enproject.org" target=3D"_blank" rel=3D"noreferrer">xen-devel-bounces@lists=
.xenproject.org</a>&gt; On Behalf Of Wei<br>
&gt; Chen<br>
&gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 14:46<br>
&gt; To: Stefano Stabellini &lt;mailto:<a href=3D"mailto:sstabellini@kernel=
.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&gt;<b=
r>
&gt; Cc: mailto:<a href=3D"mailto:xen-devel@lists.xenproject.org" target=3D=
"_blank" rel=3D"noreferrer">xen-devel@lists.xenproject.org</a>; mailto:<a h=
ref=3D"mailto:julien@xen.org" target=3D"_blank" rel=3D"noreferrer">julien@x=
en.org</a>; Bertrand Marquis<br>
&gt; &lt;mailto:<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"_blan=
k" rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;; mailto:<a href=3D"m=
ailto:jbeulich@suse.com" target=3D"_blank" rel=3D"noreferrer">jbeulich@suse=
.com</a>; mailto:<a href=3D"mailto:roger.pau@citrix.com" target=3D"_blank" =
rel=3D"noreferrer">roger.pau@citrix.com</a>;<br>
&gt; mailto:<a href=3D"mailto:andrew.cooper3@citrix.com" target=3D"_blank" =
rel=3D"noreferrer">andrew.cooper3@citrix.com</a><br>
&gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override defau=
lt<br>
&gt; NR_NODE_MEMBLKS<br>
&gt; <br>
&gt; Hi Stefano, Julien,<br>
&gt; <br>
&gt; &gt; -----Original Message-----<br>
&gt; &gt; From: Stefano Stabellini &lt;mailto:<a href=3D"mailto:sstabellini=
@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a=
>&gt;<br>
&gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 13:00<br>
&gt; &gt; To: Wei Chen &lt;mailto:<a href=3D"mailto:Wei.Chen@arm.com" targe=
t=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; Cc: Stefano Stabellini &lt;mailto:<a href=3D"mailto:sstabellini@k=
ernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kernel.org</a>&=
gt;; xen-<br>
&gt; &gt; mailto:<a href=3D"mailto:devel@lists.xenproject.org" target=3D"_b=
lank" rel=3D"noreferrer">devel@lists.xenproject.org</a>; mailto:<a href=3D"=
mailto:julien@xen.org" target=3D"_blank" rel=3D"noreferrer">julien@xen.org<=
/a>; Bertrand Marquis<br>
&gt; &gt; &lt;mailto:<a href=3D"mailto:Bertrand.Marquis@arm.com" target=3D"=
_blank" rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;; mailto:<a href=
=3D"mailto:jbeulich@suse.com" target=3D"_blank" rel=3D"noreferrer">jbeulich=
@suse.com</a>; mailto:<a href=3D"mailto:roger.pau@citrix.com" target=3D"_bl=
ank" rel=3D"noreferrer">roger.pau@citrix.com</a>;<br>
&gt; &gt; mailto:<a href=3D"mailto:andrew.cooper3@citrix.com" target=3D"_bl=
ank" rel=3D"noreferrer">andrew.cooper3@citrix.com</a><br>
&gt; &gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override =
default<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt;<br>
&gt; &gt; +x86 maintainers<br>
&gt; &gt;<br>
&gt; &gt; On Mon, 27 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; -----Original Message-----<br>
&gt; &gt; &gt; &gt; From: Stefano Stabellini &lt;mailto:<a href=3D"mailto:s=
stabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@ker=
nel.org</a>&gt;<br>
&gt; &gt; &gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8827=E6=97=A5 11:26<br>
&gt; &gt; &gt; &gt; To: Wei Chen &lt;mailto:<a href=3D"mailto:Wei.Chen@arm.=
com" target=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; Cc: Stefano Stabellini &lt;mailto:<a href=3D"mailto:sst=
abellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini@kerne=
l.org</a>&gt;; xen-<br>
&gt; &gt; &gt; &gt; mailto:<a href=3D"mailto:devel@lists.xenproject.org" ta=
rget=3D"_blank" rel=3D"noreferrer">devel@lists.xenproject.org</a>; mailto:<=
a href=3D"mailto:julien@xen.org" target=3D"_blank" rel=3D"noreferrer">julie=
n@xen.org</a>; Bertrand Marquis<br>
&gt; &gt; &gt; &gt; &lt;mailto:<a href=3D"mailto:Bertrand.Marquis@arm.com" =
target=3D"_blank" rel=3D"noreferrer">Bertrand.Marquis@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to=
 override<br>
&gt; &gt; default<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; On Sun, 26 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; &gt; &gt; -----Original Message-----<br>
&gt; &gt; &gt; &gt; &gt; &gt; From: Stefano Stabellini &lt;mailto:<a href=
=3D"mailto:sstabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sst=
abellini@kernel.org</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:35=
<br>
&gt; &gt; &gt; &gt; &gt; &gt; To: Wei Chen &lt;mailto:<a href=3D"mailto:Wei=
.Chen@arm.com" target=3D"_blank" rel=3D"noreferrer">Wei.Chen@arm.com</a>&gt=
;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Cc: mailto:<a href=3D"mailto:xen-devel@lists.=
xenproject.org" target=3D"_blank" rel=3D"noreferrer">xen-devel@lists.xenpro=
ject.org</a>; mailto:<a href=3D"mailto:sstabellini@kernel.org" target=3D"_b=
lank" rel=3D"noreferrer">sstabellini@kernel.org</a>;<br>
&gt; &gt; &gt; &gt; mailto:<a href=3D"mailto:julien@xen.org" target=3D"_bla=
nk" rel=3D"noreferrer">julien@xen.org</a>;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Bertrand Marquis &lt;mailto:<a href=3D"mailto=
:Bertrand.Marquis@arm.com" target=3D"_blank" rel=3D"noreferrer">Bertrand.Ma=
rquis@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; Subject: Re: [PATCH 22/37] xen/arm: use NR_ME=
M_BANKS to override<br>
&gt; &gt; &gt; &gt; default<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; On Thu, 23 Sep 2021, Wei Chen wrote:<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; As a memory range described in device tr=
ee cannot be split<br>
&gt; &gt; across<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; multiple nodes. So we define NR_NODE_MEM=
BLKS as NR_MEM_BANKS<br>
&gt; in<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; arch header.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; This statement is true but what is the goal o=
f this patch? Is it<br>
&gt; &gt; to<br>
&gt; &gt; &gt; &gt; &gt; &gt; reduce code size and memory consumption?<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; No, when Julien and I discussed this in last versi=
on[1], we hadn&#39;t<br>
&gt; &gt; &gt; &gt; thought<br>
&gt; &gt; &gt; &gt; &gt; so deeply. We just thought a memory range describe=
d in DT cannot<br>
&gt; be<br>
&gt; &gt; &gt; &gt; split<br>
&gt; &gt; &gt; &gt; &gt; across multiple nodes. So NR_MEM_BANKS should be e=
qual to<br>
&gt; &gt; NR_MEM_BANKS.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; <a href=3D"https://lists.xenproject.org/archives/h=
tml/xen-devel/2021-" rel=3D"noreferrer noreferrer" target=3D"_blank">https:=
//lists.xenproject.org/archives/html/xen-devel/2021-</a><br>
&gt; &gt; &gt; &gt; 08/msg00974.html<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; I am asking because NR_MEM_BANKS is 128 and<b=
r>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS=3D2*MAX_NUMNODES which is 64 =
by default so again<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is 128 before this patch.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; In other words, this patch alone doesn&#39;t =
make any difference; at<br>
&gt; &gt; least<br>
&gt; &gt; &gt; &gt; &gt; &gt; doesn&#39;t make any difference unless CONFIG=
_NR_NUMA_NODES is<br>
&gt; &gt; increased.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; So, is the goal to reduce memory usage when C=
ONFIG_NR_NUMA_NODES<br>
&gt; &gt; is<br>
&gt; &gt; &gt; &gt; &gt; &gt; higher than 64?<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; I also thought about this problem when I was writi=
ng this patch.<br>
&gt; &gt; &gt; &gt; &gt; CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BAN=
KS is a fixed<br>
&gt; &gt; &gt; &gt; &gt; value, then NR_MEM_BANKS can be smaller than CONFI=
G_NR_NUMA_NODES<br>
&gt; &gt; &gt; &gt; &gt; at one point.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; But I agree with Julien&#39;s suggestion, NR_MEM_B=
ANKS and<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; must be aware of each other. I had thought to add =
some ASSERT<br>
&gt; check,<br>
&gt; &gt; &gt; &gt; &gt; but I don&#39;t know how to do it better. So I pos=
t this patch for<br>
&gt; more<br>
&gt; &gt; &gt; &gt; &gt; suggestion.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; OK. In that case I&#39;d say to get rid of the previous=
 definition of<br>
&gt; &gt; &gt; &gt; NR_NODE_MEMBLKS as it is probably not necessary, see be=
low.<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; And keep default NR_NODE_MEMBLKS in comm=
on header<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; for those architectures NUMA is disabled=
.<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; This last sentence is not accurate: on x86 NU=
MA is enabled and<br>
&gt; &gt; &gt; &gt; &gt; &gt; NR_NODE_MEMBLKS is still defined in xen/inclu=
de/xen/numa.h<br>
&gt; (there<br>
&gt; &gt; is<br>
&gt; &gt; &gt; &gt; no<br>
&gt; &gt; &gt; &gt; &gt; &gt; x86 definition of it)<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; Yes.<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; Signed-off-by: Wei Chen &lt;mailto:<a hr=
ef=3D"mailto:wei.chen@arm.com" target=3D"_blank" rel=3D"noreferrer">wei.che=
n@arm.com</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; ---<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/asm-arm/numa.h | 8 +++=
++++-<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 xen/include/xen/numa.h=C2=A0 =C2=
=A0 =C2=A0| 2 ++<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 2 files changed, 9 insertions(+), =
1 deletion(-)<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/asm-arm/numa.h =
b/xen/include/asm-<br>
&gt; &gt; arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; index 8f1c67e3eb..21569e634b 100644<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; --- a/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +++ b/xen/include/asm-arm/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; @@ -3,9 +3,15 @@<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #include &lt;xen/mm.h&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#include &lt;asm/setup.h&gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 typedef u8 nodeid_t;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; -#ifndef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#ifdef CONFIG_NUMA<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#define NR_NODE_MEMBLKS NR_MEM_BANKS<br=
>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#else<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 /* Fake one node for now. See also=
 node_online_map. */<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define cpu_to_node(cpu) 0<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; diff --git a/xen/include/xen/numa.h b/xe=
n/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; index 1978e2be1b..1731e1cc6b 100644<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; --- a/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +++ b/xen/include/xen/numa.h<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; @@ -12,7 +12,9 @@<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define MAX_NUMNODES=C2=A0 =C2=A0 =
1<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #endif<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#ifndef NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt;=C2=A0 #define NR_NODE_MEMBLKS (MAX_NUMNO=
DES*2)<br>
&gt; &gt; &gt; &gt; &gt; &gt; &gt; +#endif<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; This one we can remove it completely right?<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; How about define NR_MEM_BANKS to:<br>
&gt; &gt; &gt; #ifdef CONFIG_NR_NUMA_NODES<br>
&gt; &gt; &gt; #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)<br>
&gt; &gt; &gt; #else<br>
&gt; &gt; &gt; #define NR_MEM_BANKS 128<br>
&gt; &gt; &gt; #endif<br>
&gt; &gt; &gt; for both x86 and Arm. For those architectures do not support=
 or enable<br>
&gt; &gt; &gt; NUMA, they can still use &quot;NR_MEM_BANKS 128&quot;. And r=
eplace all<br>
&gt; &gt; NR_NODE_MEMBLKS<br>
&gt; &gt; &gt; in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS compl=
etely.<br>
&gt; &gt; &gt; In this case, NR_MEM_BANKS can be aware of the changes of<br=
>
&gt; &gt; CONFIG_NR_NUMA_NODES.<br>
&gt; &gt;<br>
&gt; &gt; x86 doesn&#39;t have NR_MEM_BANKS as far as I can tell. I guess y=
ou also<br>
&gt; &gt; meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?<br>
&gt; &gt;<br>
&gt; <br>
&gt; Yes.<br>
&gt; <br>
&gt; &gt; But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES =
because<br>
&gt; &gt; there can be many memory banks for each numa node, certainly more=
 than<br>
&gt; &gt; 2. The existing definition on x86:<br>
&gt; &gt;<br>
&gt; &gt; #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)<br>
&gt; &gt;<br>
&gt; &gt; Doesn&#39;t make a lot of sense to me. Was it just an arbitrary l=
imit for<br>
&gt; &gt; the lack of a better way to set a maximum?<br>
&gt; &gt;<br>
&gt; <br>
&gt; At that time, this was probably the most cost-effective approach.<br>
&gt; Enough and easy. But, if more nodes need to be supported in the<br>
&gt; future, it may bring more memory blocks. And this maximum value<br>
&gt; might not apply. The maximum may need to support dynamic extension.<br=
>
&gt; <br>
&gt; &gt;<br>
&gt; &gt; On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be re=
lated.<br>
&gt; &gt; In fact, what&#39;s the difference?<br>
&gt; &gt;<br>
&gt; &gt; NR_MEM_BANKS is the max number of memory banks (with or without<b=
r>
&gt; &gt; numa-node-id).<br>
&gt; &gt;<br>
&gt; &gt; NR_NODE_MEMBLKS is the max number of memory banks with NUMA suppo=
rt<br>
&gt; &gt; (with numa-node-id)?<br>
&gt; &gt;<br>
&gt; &gt; They are basically the same thing. On ARM I would just do:<br>
&gt; &gt;<br>
&gt; <br>
&gt; Probably not, NR_MEM_BANKS will count those memory ranges without<br>
&gt; numa-node-id in boot memory parsing stage (process_memory_node or<br>
&gt; EFI parser). But NR_NODE_MEMBLKS will only count those memory ranges<b=
r>
&gt; with numa-node-id.<br>
&gt; <br>
&gt; &gt; #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES *=
 2))<br>
&gt; &gt;<br>
&gt; &gt;<br>
<br>
&gt; Quote Julien&#39;s comment from HTML email to here:<br>
&gt; &quot; As you wrote above, the second part of the MAX is totally arbit=
rary.<br>
&gt; In fact, it is very likely than if you have more than 64 nodes, you ma=
y<br>
&gt; need a lot more than 2 regions per node.<br>
&gt; <br>
&gt; So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to NR_MEM=
_BANKS<br>
&gt; so it can be used by common code.<br>
&gt; &quot;<br>
&gt; <br>
&gt; &gt; But here comes the problem:<br>
&gt; &gt; How can we set the NR_MEM_BANKS maximum value, 128 seems an arbit=
rary too?<br>
&gt; <br>
&gt; This is based on hardware we currently support (the last time we bumpe=
d the value was, IIRC, for Thunder-X). In the case of booting UEFI, we can =
get a lot of small ranges as we discover the RAM using the UEFI memory map.=
<br>
&gt; <br>
<br>
Thanks for the background.<br>
<br>
&gt; <br>
&gt; &gt; If #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * N)? And what N sh=
ould be.<br>
&gt; <br>
&gt; N would have to be the maximum number of ranges you can find in a NUMA=
 node.<br>
&gt; <br>
&gt; We would also need to make sure this doesn&#39;t break existing platfo=
rms. So N would have to be quite large or we need a MAX as Stefano suggeste=
d.<br>
&gt; <br>
&gt; But I would prefer to keep the existing 128 and allow to configure it =
at build time (not necessarily in this series). This avoid to have differen=
t way to define the value based NUMA vs non-NUMA.<br>
<br>
In this case, can we use Stefano&#39;s<br>
&quot;#define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))=
&quot;<br>
in next version. If yes, should we change x86 part? Because NR_MEM_BANKS<br=
>
has not been defined in x86.</blockquote></div></div><div dir=3D"auto"><br>=
</div><div dir=3D"auto">What I meant by configuring dynamically is allowing=
 NR_MEM_BANKS to be set by the user.</div><div dir=3D"auto"><br></div><div =
dir=3D"auto">The second part of the MAX makes no sense to me (at least on A=
rm). So I really prefer if this is not part of the initial version.</div><d=
iv dir=3D"auto"><br></div><div dir=3D"auto">We can refine the value, or int=
roduce the MAX in the future if we have a justification for it.</div><div d=
ir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote"><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc s=
olid;padding-left:1ex">
<br>
&gt; &gt; And maybe the definition could be common with x86 if we define<br=
>
&gt; &gt; NR_MEM_BANKS to 128 on x86 too.<br>
&gt; <br>
&gt; Julien had comment here, I will continue in that email.<br>
</blockquote></div></div></div>

--000000000000f9a62205ccf7b7d3--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:48:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196699.349635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBD-000319-KU; Mon, 27 Sep 2021 10:48:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196699.349635; Mon, 27 Sep 2021 10:48:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBD-000312-HU; Mon, 27 Sep 2021 10:48:15 +0000
Received: by outflank-mailman (input) for mailman id 196699;
 Mon, 27 Sep 2021 10:48:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUoBB-00030v-R5
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUoBB-0005qB-PF
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:13 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUoBB-0006eq-OI
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:13 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUoB6-0005b9-Vi; Mon, 27 Sep 2021 11:48:09 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=/w8eoH0CeKNEKy0FozyUDDGo5ckoS7P2jALjS6lX9BU=; b=i59kD1bBsMHHTMoLTsV4NrY3Qo
	91bEhzn2i3gxpm3zvGGYoCdCIrzK+g4ekJsTZO5nyoyFL2+/3prbyOmQCjlv51Okv0clMcCOHDybI
	9oAycTII0+HfiBMOY1PaAbrWN9ZIJVgRIlxOLuddjoHoXCol9/v/3epYQCarGVLOleMY=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.41320.660005.549137@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 11:48:08 +0100
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Oleksandr Tyshchenko <olekstysh@gmail.com>,
    xen-devel@lists.xenproject.org,
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
    Ian Jackson <iwj@xenproject.org>,
    Anthony PERARD <anthony.perard@citrix.com>,
    Juergen Gross <jgross@suse.com>,
    Julien Grall <julien@xen.org>,
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH V3 3/3] libxl/arm: Add handling of extended regions for
 DomU
In-Reply-To: <alpine.DEB.2.21.2109241614300.17979@sstabellini-ThinkPad-T480s>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
	<1632437334-12015-4-git-send-email-olekstysh@gmail.com>
	<alpine.DEB.2.21.2109241614300.17979@sstabellini-ThinkPad-T480s>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Stefano Stabellini writes ("Re: [PATCH V3 3/3] libxl/arm: Add handling of extended regions for DomU"):
> On Fri, 24 Sep 2021, Oleksandr Tyshchenko wrote:
> > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
...
> > Suggested-by: Julien Grall <jgrall@amazon.com>
> > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:48:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196701.349652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBH-0003Ks-Ap; Mon, 27 Sep 2021 10:48:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196701.349652; Mon, 27 Sep 2021 10:48:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBH-0003Jh-2A; Mon, 27 Sep 2021 10:48:19 +0000
Received: by outflank-mailman (input) for mailman id 196701;
 Mon, 27 Sep 2021 10:48:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUoBF-0003HF-Nf
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:17 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6b8e251c-1f80-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 10:48:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 148E22209B;
 Mon, 27 Sep 2021 10:48:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E5D0013F4B;
 Mon, 27 Sep 2021 10:48:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id MBL/Nm+hUWHBKgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 10:48:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b8e251c-1f80-11ec-bc44-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632739696; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0bTrNbQ0miBKyNabZ7OBv568TXSZHmfPJwYLlLSwrss=;
	b=Pl/LD+1gSxuMVta+tt/eO4ZEzBzgRAh0YlzD8rfBG3ybn9WUnHRuV3h1n4vB6BUPDAaSZk
	CGlvtWQxwq+0Ata56n89R9ybSNeVv45CwdRIN0EcYkL0piIPBNEHV363QHagqSrrkaH3YR
	4GJa+Qx4t7+iXWBRHSP5Jrfls7TgDjw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for xenstored
Date: Mon, 27 Sep 2021 12:48:13 +0200
Message-Id: <20210927104813.19772-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210927104813.19772-1-jgross@suse.com>
References: <20210927104813.19772-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a configuration item for the maximum number of open file
descriptors xenstored should be allowed to have.

The default should be "unlimited" in order not to restrict xenstored
in the number of domains it can support, but unfortunately the prlimit
command requires specification of a real value for the number of files,
so use 262144 as the default value. As an aid for the admin configuring
the value add a comment specifying the common needs of xenstored for
the different domain types.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- set ulimit form launch script (Julien Grall)
- split off from original patch (Julien Grall)
V4:
- switch to directly configuring the limit of file descriptors instead
  of domains (Ian Jackson)
---
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 13 +++++++++++++
 tools/hotplug/Linux/launch-xenstore.in             |  2 ++
 2 files changed, 15 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index b83101ab7e..ad020b7a50 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -32,6 +32,19 @@
 # Changing this requires a reboot to take effect.
 #XENSTORED=@XENSTORED@
 
+## Type: integer
+## Default: 262144
+#
+# Select maximum number of file descriptors xenstored is allowed to have
+# opened at one time.
+# For each HVM domain xenstored might need up to 5 open file descriptors,
+# PVH and PV domains will require up to 3 open file descriptors. Additionally
+# 20-30 file descriptors will be opened for internal uses. The default of
+# 262144 allows for about 8 open files for the theoretical maximum of 32752
+# domains.
+# Only evaluated if XENSTORETYPE is "daemon".
+#XENSTORED_MAX_OPEN_FDS=262144
+
 ## Type: string
 ## Default: ""
 #
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index 1747c96065..2bc41bb4f0 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -54,6 +54,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 
 [ "$XENSTORETYPE" = "daemon" ] && {
 	[ -z "$XENSTORED_TRACE" ] || XENSTORED_ARGS="$XENSTORED_ARGS -T @XEN_LOG_DIR@/xenstored-trace.log"
+	[ -z "$XENSTORED_MAX_OPEN_FDS" ] && XENSTORED_MAX_OPEN_FDS=262144
 	[ -z "$XENSTORED" ] && XENSTORED=@XENSTORED@
 	[ -x "$XENSTORED" ] || {
 		echo "No xenstored found"
@@ -70,6 +71,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1
 	XS_PID=`cat @XEN_RUN_DIR@/xenstored.pid`
 	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
+	prlimit --pid $XS_PID --nofile=$XENSTORED_MAX_OPEN_FDS
 
 	exit 0
 }
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:48:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196700.349646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBG-0003Hu-Sn; Mon, 27 Sep 2021 10:48:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196700.349646; Mon, 27 Sep 2021 10:48:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBG-0003Hl-P8; Mon, 27 Sep 2021 10:48:18 +0000
Received: by outflank-mailman (input) for mailman id 196700;
 Mon, 27 Sep 2021 10:48:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUoBF-0003Gz-BY
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:17 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 81c52922-a431-496e-9c83-27d06e7505dd;
 Mon, 27 Sep 2021 10:48:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B8F1022060;
 Mon, 27 Sep 2021 10:48:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8F83313A7F;
 Mon, 27 Sep 2021 10:48:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 2CHzIW+hUWHBKgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 10:48:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81c52922-a431-496e-9c83-27d06e7505dd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632739695; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=7Q8Q/1jLHjWLGdFh7x85OcFSIbNQybxr1Bod8P/V3So=;
	b=S8Y8tTPruO5kUWRDHOpbW+3tNSI4Sol3/9ga5LLb+Ye9RaPvQ0KeWq7Mjc9gt1ckq55kku
	o8TE0MGak9pyvkMhaSFJ8s0lLJTe7jUDID0PhC8lCY7ZUyBAuu/Q2FE6jIrh+KFbpBGLfm
	oCiVUa2Ba5y/NlXUcTdfvnDsyzHoSvI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 0/2] tools/xenstore: set resource limits of xenstored
Date: Mon, 27 Sep 2021 12:48:11 +0200
Message-Id: <20210927104813.19772-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Set some limits for xenstored in order to avoid it being killed by
OOM killer, or to run out of file descriptors.

Changes in V4:
- add comments
- switch to configure open file descriptors directly

Changes in V3:
- make oom score configurable

Changes in V2:
- split into 2 patches
- set limits from start script

Juergen Gross (2):
  tools/xenstore: set oom score for xenstore daemon on Linux
  tools/xenstore: set open file descriptor limit for xenstored

 .../Linux/init.d/sysconfig.xencommons.in      | 22 +++++++++++++++++++
 tools/hotplug/Linux/launch-xenstore.in        |  8 +++++++
 2 files changed, 30 insertions(+)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 10:48:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 10:48:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196702.349668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBL-0003ud-I5; Mon, 27 Sep 2021 10:48:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196702.349668; Mon, 27 Sep 2021 10:48:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUoBL-0003uU-DP; Mon, 27 Sep 2021 10:48:23 +0000
Received: by outflank-mailman (input) for mailman id 196702;
 Mon, 27 Sep 2021 10:48:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUoBK-0003Gz-7n
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:22 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea33450a-7757-403b-bd59-0628b7106e2d;
 Mon, 27 Sep 2021 10:48:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id E1D0C200DD;
 Mon, 27 Sep 2021 10:48:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BEB8C13A7F;
 Mon, 27 Sep 2021 10:48:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id EJhrLW+hUWHBKgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 10:48:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea33450a-7757-403b-bd59-0628b7106e2d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632739695; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8hffVnuWEiEzQ5SGP29hjShMeVnOse/BmKMl/NYs2rY=;
	b=j3d/BkoySyIIS8qrc3ZLM5AapZ7EJIhtiAGn/Ox/lWGCYahv7vWz7Fru/WdRsFXYywTGJw
	CJCEEOeBbKH0MJ16k3TtBQwFgBWhYQyj4QHuJHteXUl93N17YA1GI1RWh5ekszUdw6Oc+7
	Nv/f7Q5iejuENOatfN7yRkURIKvUc9o=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 1/2] tools/xenstore: set oom score for xenstore daemon on Linux
Date: Mon, 27 Sep 2021 12:48:12 +0200
Message-Id: <20210927104813.19772-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210927104813.19772-1-jgross@suse.com>
References: <20210927104813.19772-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Xenstored is absolutely mandatory for a Xen host and it can't be
restarted, so being killed by OOM-killer in case of memory shortage is
to be avoided.

Set /proc/$pid/oom_score_adj (if available) per default to -500 (this
translates to 50% of dom0 memory size) in order to allow xenstored to
use large amounts of memory without being killed.

The percentage of dom0 memory above which the oom killer is allowed to
kill xenstored can be set via XENSTORED_OOM_MEM_THRESHOLD in
xencommons.

Make sure the pid file isn't a left-over from a previous run delete it
before starting xenstored.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- set oom score from launch script (Julien Grall)
- split off open file descriptor limit setting (Julien Grall)
V3:
- make oom killer threshold configurable (Julien Grall)
V4:
- extend comment (Ian Jackson)
---
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 9 +++++++++
 tools/hotplug/Linux/launch-xenstore.in             | 6 ++++++
 2 files changed, 15 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 00cf7f91d4..b83101ab7e 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -48,6 +48,15 @@ XENSTORED_ARGS=
 # Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
+## Type: integer
+## Default: 50
+#
+# Percentage of dom0 memory size the xenstore daemon can use before the
+# OOM killer is allowed to kill it.
+# The specified value is multiplied by -10 and echoed to
+# /proc/PID/oom_score_adj.
+#XENSTORED_OOM_MEM_THRESHOLD=50
+
 ## Type: string
 ## Default: @LIBEXEC@/boot/xenstore-stubdom.gz
 #
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index 019f9d6f4d..1747c96065 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -59,11 +59,17 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 		echo "No xenstored found"
 		exit 1
 	}
+	[ -z "$XENSTORED_OOM_MEM_THRESHOLD" ] || XENSTORED_OOM_MEM_THRESHOLD=50
+	XS_OOM_SCORE=-$(($XENSTORED_OOM_MEM_THRESHOLD * 10))
+
+	rm -f @XEN_RUN_DIR@/xenstored.pid
 
 	echo -n Starting $XENSTORED...
 	$XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS
 
 	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1
+	XS_PID=`cat @XEN_RUN_DIR@/xenstored.pid`
+	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
 
 	exit 0
 }
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 11:32:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 11:32:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196743.349679 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUorW-00020n-0s; Mon, 27 Sep 2021 11:31:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196743.349679; Mon, 27 Sep 2021 11:31:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUorV-00020g-T0; Mon, 27 Sep 2021 11:31:57 +0000
Received: by outflank-mailman (input) for mailman id 196743;
 Mon, 27 Sep 2021 11:31:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUorU-00020a-6b
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 11:31:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 83b19402-1f86-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 11:31:54 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-XSyh8YLoNpaq3TPIZcpjHQ-1; Mon, 27 Sep 2021 13:31:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4190.eurprd04.prod.outlook.com (2603:10a6:803:4b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 11:31:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 11:31:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0066.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Mon, 27 Sep 2021 11:31:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83b19402-1f86-11ec-bc44-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632742313;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=o2GYJL857UsI0GbUPLDhmje2RvcSOCoELDigK3SKglo=;
	b=Kuoo7rfE5f6vAljUG6cklNnWXfahA1sgc/53tSF/Zewf9bSATxLQMhVFiqdz7J/+/7z/jz
	IeuvvJ/GzyOplZZxss4XKtsdkOMQdfIhiQ8eHqVxGYE1t333hXDpvh4xP6eQK8S7wO2lGl
	lwSQwdwB8kM5rUK2luVUXpmrVXV4XCs=
X-MC-Unique: XSyh8YLoNpaq3TPIZcpjHQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d2EbRzUaR2mhZ427BYm6uNqepfjHgdTfNzXFoMr7oWl/KZS+iG785gG6SlKDEFdwCyZOft6YDSzmpi5y+2XSbri0gFwzs7Ednmcr8Ze/36tbV34DySviadMi/HzfPnN86b09qFRs+dT3a6LP2+1Opm4SW++NxqawMBrySGC8dtr2w+B3QpEY52hMHxIbsjx0G55V30OspL3DNig1ltXA/I0HMOfFSW3LbPr6LJDLfJBDSnvwl9BQaljRCrNtCMtJ0R2nWv7tAnMs21fVatwIV+gAraRZQGVI6UGxMrLVSurndotpouqlVvjhJrKAoWwWuu/DiPt7dUKIh5M0KqnSgQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=o2GYJL857UsI0GbUPLDhmje2RvcSOCoELDigK3SKglo=;
 b=M920bz1+ojJpegEUxlCXpsaD3yN/ndqujsgd2Doo1m4AEzvHstvzcE43tySDNJpYHJh2YglBBVpUapYf9SYA1Us4tYWN3/5L4aYw7t8uxo4eucRx4xwJ/Bhm/HcoaFq/qTh25rCVGTK9KjFhFcxgNJ9ZWBwFwDirCZlycAg0i6ZPm1KG+xPuuUW3Y94nFMa52GathgvGVt8IfmtPbcoZnIXDWdP40qLizq170NrUTYtQAc591YBnlJzqwrs0XWEFLwlLmku7uOqFpCVYMBK9p5ACYQD/3yboQVAzM/5iScgx8aZLM6kfLqIKUL4FI0OqUOhK7PqOc4IOGqf5DMdnzw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
Date: Mon, 27 Sep 2021 13:31:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210923125501.234252-12-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0066.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bf03436a-900e-4834-e436-08d981aa64b9
X-MS-TrafficTypeDiagnostic: VI1PR04MB4190:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4190E900BC781B63ADEB2C7CB3A79@VI1PR04MB4190.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HwWfiTISFF5tSEM3ihqBoANfVmYOr3X4stLkdj7Ww7C6DWy7tUfUQB0oPSecWK+4Ahdk1Qb+0H/WP+AfhqP0hjR2T8i/W/2JOP2zltQ6uS5JmmObBwIy3HFzUJYAldgbk6dSMIBfqDHYg29K2esAltBqt5iYbgp3wycpYC5z3ng0sEHMQwGl9jPBKRKQamuKf8VFcxJPzeZuabTwb3sqeGC4ql/IjLA4z70EhHFVG/y2k+7jNsvjLMtc7q1CnGr7MNN6g7fMfJeI/mLSJTwG6U7UAUCA19cdVpBxWtsua94PnG0YcSBsTl4EJ60ObFIp1NDX4pdNVLsW7832TDWS4PbAo9TEV3RFjBZycWgxEXbvWOzPZLoBwJoatsOQK6ci9f6RvJA8P5aQjsrfGw/ucaEks/XltyLK/nQCP4VmvuMK8juYl+LTNvKsudEnFN5dc/472kMe9NbCEgHv4JHAsa/Xc9kt6JNKQsavgKlxmSduMhuddVwtWWPsbey+JNYuLH2QXpqTrgDzGF5tRL5KkSPTzaUNIX2Fom50J/4uWamya/fvuwKOF4GKwZzKlZSmj/YcpaaLo5ZIT3EOru5uy7oC9SmDfByO3xnjT1NiS4Jtvi8FyxvqQK05/oTvfg/R3/FFNglJiXPy/Mqnbq7yq+LvMkJg8IXkFQRLKxAnhgHOwgu00YUxn0NTNGRx3/IOD1i7JbLfKqJRwnNTuDjdVUBQEg1JrVXU3kgIlfmlClk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(956004)(31686004)(36756003)(316002)(16576012)(31696002)(53546011)(6486002)(83380400001)(6916009)(7416002)(2906002)(26005)(38100700002)(66476007)(4326008)(66946007)(8676002)(186003)(8936002)(508600001)(66556008)(2616005)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SFNEbSsxSnZCc1NrNmtxYVdzSGdLTEs4OWxhRWN0OTVRcktrakpDVXhzZlBO?=
 =?utf-8?B?U0pJbzBUSytZOHErQ1VOL2l3R3pDZHJxd1JLSWJNM3NVU291WlVwQXJsSzht?=
 =?utf-8?B?S1BFMkR3SkZ5MW1jbVZ1SlJpM0VGUnEzSzNiZkhXSUlYRVBFaDBjQjhianJn?=
 =?utf-8?B?bHFWRk9mU0k4QzlDQ29zQldwYTljWkd6SG1oWnRwb21BVmc4dk5Ta0dJVm9r?=
 =?utf-8?B?L2lHU1d2bitpWUhDRUhVcGJuVkllRzk1K1hvRFVuaGhoTDNIcHBtbGRHSmI1?=
 =?utf-8?B?SlVYSFJ5aVk0WG83c0xsbFlTMGlETDQyTXZwdEhqNzVxWFRUcWcxZTVrN2Y3?=
 =?utf-8?B?L0JiZXBZaFJYdE1zT3ZsMUNSSFAwRi9RaGdPUC84Wno1NDhUTkJHaGMxTGVy?=
 =?utf-8?B?MGJpSWFNdUV5VmNtM0xFY2FrS2RaRFBaYU1TMzYvMFRKVjVRSEVsY1dJR21F?=
 =?utf-8?B?M0dlOVN3ZHdpRVJLM1ljeU91ZlpoODhGb0RFRjI1V3Z4VFZ6dUl6d09waXF2?=
 =?utf-8?B?NDlSaloxcEZkVWJPWGJOYTc4Y0o0eGZRK1BJUXBEVmxhb0lkekVUcUlvZXlt?=
 =?utf-8?B?a29Va2QyL08rRWEwS1lMRDdRclY4UUtHMEFSeXg1ZTdvazlLTmxqOExqZUVD?=
 =?utf-8?B?MnFQdlkxdWRhRlI3Qy9PeFRENzh4ZnRkT2RQRFhvNmoxS3h4WHpwdUVHRVkw?=
 =?utf-8?B?UCtKVW9pTUlZdndVM1NmMm1sSXo5dFFqNTd1V1JPd2VaeFh0T3pPeWhQT0hs?=
 =?utf-8?B?VS95T2F6ZWYrdU13M0lPeHhMNDIyOXVrYkRHMWc2Y0NrR3FDMng5aExtUVk2?=
 =?utf-8?B?V0R5bzNMWmFKblVGb2NHeEJDVDZqaU9WZTByejA5SGVkK0p6ZU1zV2hRc0lD?=
 =?utf-8?B?ZXR4UmRmUjUrVjhBMU9TcWtMSTJWSmZwZnQ2NnJ3Umhydi85Y1hPeDZ4d1lI?=
 =?utf-8?B?Q2RMQVBibTh0QUp0OThhS3NXTUtnTnZxRmJCbkR2SkVxWWxmTXNRQmFtWjZ2?=
 =?utf-8?B?UkNGeCsrY1hJTXg5RzNnc2ZoVVBEelBlbXNXdHkzOEtQeHpDbUJjcm9TZ3RX?=
 =?utf-8?B?c3cwY09qS1BiT2pQZCtYM3pCSktRVVlzdjAzdDhHVVlMNXMwR0VlTmhON3No?=
 =?utf-8?B?aUppUStRNUNFL2thckZzZGpXSml5d1JRck84d0RJTWpuMlZZLzBZZnlHKzZF?=
 =?utf-8?B?VE43WWREWDV6VzlEdVE0S1ArUThKVzA5WXN6K2c3SGJJUEZFMy8relljbXFh?=
 =?utf-8?B?SjF1NnJ4enNvMk0veC84R2JXOEdyaEJsMWp4cTZmZ1RJM0ZSZDVGOTFnUWEz?=
 =?utf-8?B?REh3UDJQdktrRC9IVlZMVkNZcFphQmhzZkh5NkZOWTMyWUp0a1FVcE9HNGJR?=
 =?utf-8?B?eXhkK1VnVi9uNmUyZjZGVnh6bGtDa2QxY1oxNktDbHVKZW9GeHRMMVVNd1Jm?=
 =?utf-8?B?R01uL3Exd3dkelh2ZVVWM3RrL0hyZEZXT3FIdFBJSmx3U0VPcURGeWRiTG5w?=
 =?utf-8?B?YjEyVmtJa2lUUlVWM0p6S1FvN0FnQWE4MEY5UGprTDRjUGUwVjYyWXk1KzNo?=
 =?utf-8?B?enZZNUoyNjNVK08xUnhZbGorRjJCdTRicVN3ODUrNEJWaTREYXVpSjFyd0xI?=
 =?utf-8?B?U2pkMnp4MUNtZGdheTBqRDY4TUhmb1NUWUF3d3ZpWE5laWFNUjc2cHBvQndl?=
 =?utf-8?B?WVYrM25MT2dTT01vVU5WTUtrb3NyVy90bS9vZUZXcGlRaEViNHhZdlIvUVE2?=
 =?utf-8?Q?nTKF/X6DGIXFldvvyTfL94NoLTGKCHJR5Wnm+8Z?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf03436a-900e-4834-e436-08d981aa64b9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 11:31:48.5292
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZBDLwxVGeOYncOFF4VtliTHIqwgqVWsQsc3yxAz4t0zhqwY4TfE2w1L67VuVzCQyx/x+xzATNmVfAxxMgUDzwQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4190

On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> There are three  originators for the PCI configuration space access:
> 1. The domain that owns physical host bridge: MMIO handlers are
> there so we can update vPCI register handlers with the values
> written by the hardware domain, e.g. physical view of the registers
> vs guest's view on the configuration space.
> 2. Guest access to the passed through PCI devices: we need to properly
> map virtual bus topology to the physical one, e.g. pass the configuration
> space access to the corresponding physical devices.
> 3. Emulated host PCI bridge access. It doesn't exist in the physical
> topology, e.g. it can't be mapped to some physical host bridge.
> So, all access to the host bridge itself needs to be trapped and
> emulated.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> New in v2
> ---
>  xen/arch/arm/domain.c         |  1 +
>  xen/arch/arm/vpci.c           | 87 +++++++++++++++++++++++++++++++----
>  xen/arch/arm/vpci.h           |  3 ++
>  xen/drivers/passthrough/pci.c | 25 ++++++++++
>  xen/include/asm-arm/pci.h     |  1 +
>  xen/include/xen/pci.h         |  1 +
>  xen/include/xen/sched.h       |  2 +
>  7 files changed, 111 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index c7b25bc70439..c0ad6ad682d2 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -797,6 +797,7 @@ void arch_domain_destroy(struct domain *d)
>                         get_order_from_bytes(d->arch.efi_acpi_len));
>  #endif
>      domain_io_free(d);
> +    domain_vpci_free(d);
>  }
>  
>  void arch_domain_shutdown(struct domain *d)
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index 14947e975d69..012f958960d1 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -17,6 +17,14 @@
>  
>  #define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
>  
> +struct vpci_mmio_priv {
> +    /*
> +     * Set to true if the MMIO handlers were set up for the emulated
> +     * ECAM host PCI bridge.
> +     */
> +    bool is_virt_ecam;
> +};
> +
>  /* Do some sanity checks. */
>  static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
>  {
> @@ -38,6 +46,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>      pci_sbdf_t sbdf;
>      unsigned long data = ~0UL;
>      unsigned int size = 1U << info->dabt.size;
> +    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
>  
>      sbdf.sbdf = MMCFG_BDF(info->gpa);
>      reg = REGISTER_OFFSET(info->gpa);
> @@ -45,6 +54,13 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
>      if ( !vpci_mmio_access_allowed(reg, size) )
>          return 0;
>  
> +    /*
> +     * For the passed through devices we need to map their virtual SBDF
> +     * to the physical PCI device being passed through.
> +     */
> +    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v, &sbdf) )
> +            return 1;
> +
>      data = vpci_read(sbdf, reg, min(4u, size));
>      if ( size == 8 )
>          data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
> @@ -61,6 +77,7 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
>      pci_sbdf_t sbdf;
>      unsigned long data = r;
>      unsigned int size = 1U << info->dabt.size;
> +    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
>  
>      sbdf.sbdf = MMCFG_BDF(info->gpa);
>      reg = REGISTER_OFFSET(info->gpa);
> @@ -68,6 +85,13 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
>      if ( !vpci_mmio_access_allowed(reg, size) )
>          return 0;
>  
> +    /*
> +     * For the passed through devices we need to map their virtual SBDF
> +     * to the physical PCI device being passed through.
> +     */
> +    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v, &sbdf) )
> +            return 1;
> +
>      vpci_write(sbdf, reg, min(4u, size), data);
>      if ( size == 8 )
>          vpci_write(sbdf, reg + 4, 4, data >> 32);
> @@ -80,13 +104,48 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
>      .write = vpci_mmio_write,
>  };
>  
> +/*
> + * There are three  originators for the PCI configuration space access:
> + * 1. The domain that owns physical host bridge: MMIO handlers are
> + *    there so we can update vPCI register handlers with the values
> + *    written by the hardware domain, e.g. physical view of the registers/
> + *    configuration space.
> + * 2. Guest access to the passed through PCI devices: we need to properly
> + *    map virtual bus topology to the physical one, e.g. pass the configuration
> + *    space access to the corresponding physical devices.
> + * 3. Emulated host PCI bridge access. It doesn't exist in the physical
> + *    topology, e.g. it can't be mapped to some physical host bridge.
> + *    So, all access to the host bridge itself needs to be trapped and
> + *    emulated.
> + */
>  static int vpci_setup_mmio_handler(struct domain *d,
>                                     struct pci_host_bridge *bridge)
>  {
> -    struct pci_config_window *cfg = bridge->cfg;
> +    struct vpci_mmio_priv *priv;
> +
> +    priv = xzalloc(struct vpci_mmio_priv);
> +    if ( !priv )
> +        return -ENOMEM;
> +
> +    priv->is_virt_ecam = !is_hardware_domain(d);
>  
> -    register_mmio_handler(d, &vpci_mmio_handler,
> -                          cfg->phys_addr, cfg->size, NULL);
> +    if ( is_hardware_domain(d) )
> +    {
> +        struct pci_config_window *cfg = bridge->cfg;
> +
> +        bridge->mmio_priv = priv;
> +        register_mmio_handler(d, &vpci_mmio_handler,
> +                              cfg->phys_addr, cfg->size,
> +                              priv);
> +    }
> +    else
> +    {
> +        d->vpci_mmio_priv = priv;
> +        /* Guest domains use what is programmed in their device tree. */
> +        register_mmio_handler(d, &vpci_mmio_handler,
> +                              GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE,
> +                              priv);
> +    }
>      return 0;
>  }
>  
> @@ -95,13 +154,16 @@ int domain_vpci_init(struct domain *d)
>      if ( !has_vpci(d) )
>          return 0;
>  
> -    if ( is_hardware_domain(d) )
> -        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
> -
> -    /* Guest domains use what is programmed in their device tree. */
> -    register_mmio_handler(d, &vpci_mmio_handler,
> -                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
> +    return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
> +}
>  
> +static int domain_vpci_free_cb(struct domain *d,
> +                               struct pci_host_bridge *bridge)
> +{
> +    if ( is_hardware_domain(d) )
> +        XFREE(bridge->mmio_priv);
> +    else
> +        XFREE(d->vpci_mmio_priv);
>      return 0;
>  }
>  
> @@ -124,6 +186,13 @@ int domain_vpci_get_num_mmio_handlers(struct domain *d)
>      return count;
>  }
>  
> +void domain_vpci_free(struct domain *d)
> +{
> +    if ( !has_vpci(d) )
> +        return;
> +
> +    pci_host_iterate_bridges(d, domain_vpci_free_cb);
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> index 27a2b069abd2..38e5a28c0d95 100644
> --- a/xen/arch/arm/vpci.h
> +++ b/xen/arch/arm/vpci.h
> @@ -18,6 +18,7 @@
>  #ifdef CONFIG_HAS_VPCI
>  int domain_vpci_init(struct domain *d);
>  int domain_vpci_get_num_mmio_handlers(struct domain *d);
> +void domain_vpci_free(struct domain *d);
>  #else
>  static inline int domain_vpci_init(struct domain *d)
>  {
> @@ -28,6 +29,8 @@ static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
>  {
>      return 0;
>  }
> +
> +static inline void domain_vpci_free(struct domain *d) { }
>  #endif
>  
>  #endif /* __ARCH_ARM_VPCI_H__ */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index 4552ace855e0..579c6947cc35 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -890,6 +890,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>      return 0;
>  }
>  
> +/*
> + * Find the physical device which is mapped to the virtual device
> + * and translate virtual SBDF to the physical one.
> + */
> +bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf)

Why struct vcpu, when you only need ...

> +{
> +    struct domain *d = v->domain;

... this? It's also not really logical for this function to take a
struct vcpu, as the translation should be uniform within a domain.

Also - const please (as said elsewhere before, ideally wherever possible
and sensible).

> +    struct vpci_dev *vdev;
> +    bool found = false;
> +
> +    pcidevs_lock();
> +    list_for_each_entry ( vdev, &d->vdev_list, list )
> +    {
> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
> +        {
> +            /* Replace virtual SBDF with the physical one. */
> +            *sbdf = vdev->pdev->sbdf;
> +            found = true;
> +            break;
> +        }
> +    }

For a DomU with just one or at most a couple of devices, such a brute
force lookup may be fine. What about Dom0 though? The physical topology
gets split at the segment level, so maybe this would by a reasonable
granularity here as well?

> +    pcidevs_unlock();
> +    return found;

Nit: Blank line please ahead of the main "return" of a function.

> +}
> +
>  /* Caller should hold the pcidevs_lock */
>  static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
>                             uint8_t devfn)

Seeing this function in context (which patch 2 adds without any #ifdef
around it afaics), will this new function needlessly be built on x86 as
well? (I didn't look at other intermediate patches yet, so please
forgive if I've missed the addition of an #ifdef.)

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 12:08:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 12:08:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196774.349690 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpQo-00068P-0b; Mon, 27 Sep 2021 12:08:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196774.349690; Mon, 27 Sep 2021 12:08:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpQn-00068I-SP; Mon, 27 Sep 2021 12:08:25 +0000
Received: by outflank-mailman (input) for mailman id 196774;
 Mon, 27 Sep 2021 12:08:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUpQl-00068C-WF
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 12:08:24 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9b8a9060-1f8b-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 12:08:21 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18RBhAoC027647; 
 Mon, 27 Sep 2021 12:08:20 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbcrpg92h-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 12:08:19 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6884.eurprd03.prod.outlook.com (2603:10a6:20b:2de::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 12:08:17 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 12:08:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b8a9060-1f8b-11ec-bc44-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kR8u6GE5hrot0KiQ9FJ6faiJguBv/bu5wznLndEBANqf0FI20C/IoWO1Ussb4nmpfw+VHlhvd3mJzeB0w+FxGQkrLB/PQsZp9zk7BqYj1iqvOqjxSW3JSubwUFaTeqRbZJ3tJYR2wVxwEWvhiP0+NffasS1aMHiIRANfmAMLhyMngvzcw1JZjABwpiPoyFoBmWHWj6TDKdoA3WjUw74V4AlVVVyy9pF7Q2ISZAghuZJ88gEEztoZ42c29QwM2QjIu+BkTu/E99grM3RYpq4wxKOqJM8mY+v+NZuxAO1hmTZjyrjE1mpQyrVyz5vIYoK6z5NCFULZCLqela+u8B0C0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xexl7NyoROe4qEbtFGxV28fJJnFyqcHJo+rcYlERR9k=;
 b=YiTFwaVkEFGLiLbdz/kB29Ox+4rBppehGYjmS173cFFLU6fj3dswCKZjQmfT/ghAnO68fUIm0dZ0S4H/1tHJxO/g3syWGsV8xtuoTNvSuHbFHEeQDgqa7JC7vqveS+jpwDEcKR+wy9n7byu0L9HfeeczffMQCV3/J9mFpEIHZ/PneFMbol/ZYoIMAlgUn4lZoU4idKstWOvG8iUt+TR7bPhtLzU5kwZ3GPqqajDFh1gdvly4cOEEAk6+pKje4sUaUeO17GOiGIvs6s6vtvY5LVJ/iwUUfdW7LjKYBJ1OZ6ylB9suvtdkkfxCfGqn3zE2jOO6mu3j3EtwKWF4XDA1fw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xexl7NyoROe4qEbtFGxV28fJJnFyqcHJo+rcYlERR9k=;
 b=fdiwhJFgrhFlyh4/Tq8CEvd9tYeOVLaHQJyWPAWD17Ra6dzvC/4S2usNfJgOPIaR5++ubJ9WKlQhB6hcx9ht0jBb8w3RV8pQZg7LViR5lBmWnM/Tm8mYIIH9CoOl4zlUHKrtqkTH4/KeOpvmfn7b8dW5HIfFDJb8kJgfSa+6KQJvOFegU1WzNUqvozJNqOk4VJ2zYdjzKLRyGFV5NR4kQB8qGY0X8Mbhtwocmwyp+H50v3Fq3YVIymI/KHrXU2lwIer38d7gzff5XNOGEDqvYjVVBmOi1YEB/sjKB50+DAaSHscbk5dk7NVePWN3u6Ya8a2etY6ysNyHNnNG9hv0+w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: AQHXsHpEqeYVqdD8fU+X35KPuXAnA6u3xY6AgAAKMQA=
Date: Mon, 27 Sep 2021 12:08:16 +0000
Message-ID: <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
In-Reply-To: <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c923dbea-d6af-47d4-73f0-08d981af7d40
x-ms-traffictypediagnostic: AM9PR03MB6884:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB6884C96EFC64A844BC0E39DFE7A79@AM9PR03MB6884.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 uN6rgfMaT1kPldmO0Xb6aDzK8M/p3QagjsnK9qNpjOz5SQr1VcX5ZA4SA+37DCTyrsaPJA9IGeFLIffWi1tSlpCXlzll3PFXa4WcMm8oq2sKOmg5CoKgcoEDlMmZ1ktduglgODEe5ebMWfk5/5fL+bopmHSxCS4mpOsx48HdX7VWR0UjF3jveM375T2/9dtnQUOSg5dzTyDw2OYbP2S9+ku8IUk5DGQpWm6AS4chYTM89+YDd9Y9Y67Xcr56OkTXombSA80FGj/NasevOnvh0SGsxouoiPHFUSLNPW36KrRDhEgRsk8mBXUzJWhgUAPs6PiIEOBZm6HfVFImRV5JiYINAUskZ1Chg8NaAB1vq68EZK++OhBdTxUw4b3hreKiRf/TnSLZNk2u7ztr3l8mhexhKEAn+vD6eMeq+kbOhRELt/HmVIJ74OAI3rrYET8VPJ9d3rv/Du/vH7ldeENWb8QyioWySCAJnajpILcVXQhqGABjlh8UOGEgcFvZ7T0/J2asj1UJ5U8Q41OKpeJxjTi/kLxKtOjuGf4o/nqx0ePOYOKGj4QBfytdUk+Zz1aORiaeoYSnlbkSu4bUzP25vxPXUOPXggCsgMp1du2lIZys1Mf0AmOKp/ZkeywpNj3OaAPGYojVNbS5wkxx/SKt5AT6QQcXMaNs8phRkdLh5dfcKj9xCkx8Rk8w5cXTc8eJdG8vSTINbWXyp1EthZVdABuJQl6/7JEeWLrvVI1NF4RBTls3LUg31yVOZKlbuonG
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(316002)(6512007)(4326008)(91956017)(76116006)(66446008)(64756008)(66556008)(6486002)(66476007)(66946007)(36756003)(2616005)(31686004)(5660300002)(107886003)(54906003)(83380400001)(508600001)(186003)(2906002)(122000001)(71200400001)(38070700005)(6916009)(6506007)(53546011)(8676002)(26005)(31696002)(55236004)(38100700002)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ek43UTVnMktoakJ5NHlsamYySVpwQ2lxdlB5ZE5ScHIrWXdiWDFTYTMvbk90?=
 =?utf-8?B?Z2lCQzExSG5ZUXo2a3VTK05YSjJ5cXczaXZ5Z1JhUWhlazNJdkd6djFkWHFt?=
 =?utf-8?B?Wmt3MzFGblloUXFPZGFWeGU3TGpocVV2VXJjWDQrZUdrMXNVZkR5SXhCeUZv?=
 =?utf-8?B?TGphdFRiN3NrQVJlWWs2UnR1aEtvMDFSN3lNSkI3aXJqaFRhVFhHZ0FsSXRG?=
 =?utf-8?B?TlZzNTV1SXNrLysvSGgyR1VDSDUyazhQTmFVZ3FnWWpMSlo3TTFxV1BpWWZS?=
 =?utf-8?B?NzlwMlNkckgwR0tXRHZ1VTNmTnlTNDJVYzVPWUM3RnRxSjd6ZEgra2RucE4y?=
 =?utf-8?B?TzBjRnMwRDdIT09uWEYyUUMyNXNjejNvdS9TMS92OXd6RHBPazRKYlZUbDg4?=
 =?utf-8?B?dktzUDlkdDdLNENCQmJGUWdXbFhEOFVxMmFmSHFpZEhhQnYzNmpDWGZWQzlO?=
 =?utf-8?B?UjJRQ1hFS1Q0VkpOdlo5TGNaWGxScUVwUG9jM2UxdTFlUUFLTllyQkxXcWtk?=
 =?utf-8?B?WXZSOEwrd1VmKzJxTERiellQMzNpQTlyVGpReE1SOC90VmFMazBmelB6VDlu?=
 =?utf-8?B?Y0dwS1ZYYTVvcFVyUVNIWUlOUFl3bXpiUVc0MUFBbUtCZkh1Mm94YTVEa05q?=
 =?utf-8?B?T0dvbDYzU2FXVXpaOStZZXMycjBocjcraDFldEF4aXM3OHJlKzlXdW1saVgy?=
 =?utf-8?B?U1BtRDh3a1BHTFJvQ2NzbURYY2N0alhXeHIvRVQzblhZVGpYZjlnaWpyRng0?=
 =?utf-8?B?RHBLUTAvZk9nSGljOGJoQTAzQitCMk40SHlYS2VUOWs4dzF3MjJkQWtlNG1h?=
 =?utf-8?B?aHBqRkEyclpRTzVDU05XZXc1SjdCK0pCb2hQenBpYVIxYXpDQllxTmUvRHg3?=
 =?utf-8?B?SVpqbWdoWmgzVm1SWnEvY2VWRGo2OEZTa21LeFl5TEpONWg5WS83K0pTMWU2?=
 =?utf-8?B?Tk1McC9Gc1BLbFZLaExjRldyVDBNMmI3WXY0aEJCYlVDZnFCQlVhNWpWWmhH?=
 =?utf-8?B?WVJROGpkN29lWW1wbndYdzVYdm1oUTdydVBnenpDTThLMVl1aUEvV29vemZJ?=
 =?utf-8?B?YTd1djhUeXBIRlVPREVNNnF4eHk1ZlYxRXdrdEp6L3lnQlEvUzIwM2VmbGhL?=
 =?utf-8?B?Zlh1Zm9hcnBYOU5MZzlaWWN4ZWZaK3h6VGk2dWQ3VTdoRE14RVVkSCtCdXFR?=
 =?utf-8?B?YjAxek1pZUs4MHVpa2tQWTNRamtLN2ErSWRybWJFQm1VNExEY3lFYVFZaUlt?=
 =?utf-8?B?eEpFNFZheHFSZzhWRDkvOGhvblJNMXMzZ3JBSHZJdlNCZGl3b3c0VWlvV3RC?=
 =?utf-8?B?ck12RHYwTWRUamppdDdUQXlkcEFZU3N0VEdqdUtCelkxUTJEWGdtMHRWRWQ2?=
 =?utf-8?B?ZmNuUkN1SWsyTy9ML1Z1TmtINnNWaWhSSTZDN3F0WVRiU0ppVXpQZlJHMWJD?=
 =?utf-8?B?WnNRWXY4cEdqQmtIdlgrUURDRHl0eVNQa1dDVlhhL3hNSjh5UGdVeDdhWklC?=
 =?utf-8?B?Zlc0QlRETE5tQm1KK0hpRkp2eVRJc2dSMFcrNXFmMGp2d3pwUlgwMDR5N0xm?=
 =?utf-8?B?b2NKMnZJekpCNVVwbGY1bG02VnBxV3kxdXYvcm91S01XUHQ4c3NOQnUvOHl3?=
 =?utf-8?B?UFcrSk9RUUtpVEMxN2czbytUUXkzbE91ZDRBcHJTSGdoT2xKcmQvU25tSy93?=
 =?utf-8?B?czFJUUtCVlk4TjFtUUljY2xxcnpxVC9JUWlEakVoWExzT3VoeWZxNWdCN3BS?=
 =?utf-8?B?OWJXY1F0K3lFZWVaZHlSTHF5TFBJTTlxLy9QSUJPVjZEaEk2VmNlVW03N1FM?=
 =?utf-8?B?MGw2QjhjZ1VjQldTWFQ4Zz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <AE455761610DF846B2829E74C9B7AB83@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c923dbea-d6af-47d4-73f0-08d981af7d40
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 12:08:16.8509
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: GQx2cFyR9W+8pltCUqBvzNW5tWEm2PqYqVTyGsFmcqIObHDURj3fOxZ0Saip58mrC/8ayQXc7bCuUZ3sRzR2gAOfAoLwgDFq8yc+8YKPJM3mnFLTausjlb87lexv4MCG
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6884
X-Proofpoint-GUID: 9MuYMjv6HHzU9D-O3jUqFogSkMW3bXZ7
X-Proofpoint-ORIG-GUID: 9MuYMjv6HHzU9D-O3jUqFogSkMW3bXZ7
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 clxscore=1015
 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501
 bulkscore=0 suspectscore=0 spamscore=0 adultscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270084

DQpPbiAyNy4wOS4yMSAxNDozMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDIzLjA5LjIwMjEg
MTQ6NTUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRy
IEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+
IFRoZXJlIGFyZSB0aHJlZSAgb3JpZ2luYXRvcnMgZm9yIHRoZSBQQ0kgY29uZmlndXJhdGlvbiBz
cGFjZSBhY2Nlc3M6DQo+PiAxLiBUaGUgZG9tYWluIHRoYXQgb3ducyBwaHlzaWNhbCBob3N0IGJy
aWRnZTogTU1JTyBoYW5kbGVycyBhcmUNCj4+IHRoZXJlIHNvIHdlIGNhbiB1cGRhdGUgdlBDSSBy
ZWdpc3RlciBoYW5kbGVycyB3aXRoIHRoZSB2YWx1ZXMNCj4+IHdyaXR0ZW4gYnkgdGhlIGhhcmR3
YXJlIGRvbWFpbiwgZS5nLiBwaHlzaWNhbCB2aWV3IG9mIHRoZSByZWdpc3RlcnMNCj4+IHZzIGd1
ZXN0J3MgdmlldyBvbiB0aGUgY29uZmlndXJhdGlvbiBzcGFjZS4NCj4+IDIuIEd1ZXN0IGFjY2Vz
cyB0byB0aGUgcGFzc2VkIHRocm91Z2ggUENJIGRldmljZXM6IHdlIG5lZWQgdG8gcHJvcGVybHkN
Cj4+IG1hcCB2aXJ0dWFsIGJ1cyB0b3BvbG9neSB0byB0aGUgcGh5c2ljYWwgb25lLCBlLmcuIHBh
c3MgdGhlIGNvbmZpZ3VyYXRpb24NCj4+IHNwYWNlIGFjY2VzcyB0byB0aGUgY29ycmVzcG9uZGlu
ZyBwaHlzaWNhbCBkZXZpY2VzLg0KPj4gMy4gRW11bGF0ZWQgaG9zdCBQQ0kgYnJpZGdlIGFjY2Vz
cy4gSXQgZG9lc24ndCBleGlzdCBpbiB0aGUgcGh5c2ljYWwNCj4+IHRvcG9sb2d5LCBlLmcuIGl0
IGNhbid0IGJlIG1hcHBlZCB0byBzb21lIHBoeXNpY2FsIGhvc3QgYnJpZGdlLg0KPj4gU28sIGFs
bCBhY2Nlc3MgdG8gdGhlIGhvc3QgYnJpZGdlIGl0c2VsZiBuZWVkcyB0byBiZSB0cmFwcGVkIGFu
ZA0KPj4gZW11bGF0ZWQuDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hj
aGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+IC0tLQ0KPj4g
TmV3IGluIHYyDQo+PiAtLS0NCj4+ICAgeGVuL2FyY2gvYXJtL2RvbWFpbi5jICAgICAgICAgfCAg
MSArDQo+PiAgIHhlbi9hcmNoL2FybS92cGNpLmMgICAgICAgICAgIHwgODcgKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKy0tLS0NCj4+ICAgeGVuL2FyY2gvYXJtL3ZwY2kuaCAgICAgICAg
ICAgfCAgMyArKw0KPj4gICB4ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYyB8IDI1ICsrKysr
KysrKysNCj4+ICAgeGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCAgICAgfCAgMSArDQo+PiAgIHhl
bi9pbmNsdWRlL3hlbi9wY2kuaCAgICAgICAgIHwgIDEgKw0KPj4gICB4ZW4vaW5jbHVkZS94ZW4v
c2NoZWQuaCAgICAgICB8ICAyICsNCj4+ICAgNyBmaWxlcyBjaGFuZ2VkLCAxMTEgaW5zZXJ0aW9u
cygrKSwgOSBkZWxldGlvbnMoLSkNCj4+DQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL2Rv
bWFpbi5jIGIveGVuL2FyY2gvYXJtL2RvbWFpbi5jDQo+PiBpbmRleCBjN2IyNWJjNzA0MzkuLmMw
YWQ2YWQ2ODJkMiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9kb21haW4uYw0KPj4gKysr
IGIveGVuL2FyY2gvYXJtL2RvbWFpbi5jDQo+PiBAQCAtNzk3LDYgKzc5Nyw3IEBAIHZvaWQgYXJj
aF9kb21haW5fZGVzdHJveShzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gICAgICAgICAgICAgICAgICAg
ICAgICAgIGdldF9vcmRlcl9mcm9tX2J5dGVzKGQtPmFyY2guZWZpX2FjcGlfbGVuKSk7DQo+PiAg
ICNlbmRpZg0KPj4gICAgICAgZG9tYWluX2lvX2ZyZWUoZCk7DQo+PiArICAgIGRvbWFpbl92cGNp
X2ZyZWUoZCk7DQo+PiAgIH0NCj4+ICAgDQo+PiAgIHZvaWQgYXJjaF9kb21haW5fc2h1dGRvd24o
c3RydWN0IGRvbWFpbiAqZCkNCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vdnBjaS5jIGIv
eGVuL2FyY2gvYXJtL3ZwY2kuYw0KPj4gaW5kZXggMTQ5NDdlOTc1ZDY5Li4wMTJmOTU4OTYwZDEg
MTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdnBjaS5jDQo+PiArKysgYi94ZW4vYXJjaC9h
cm0vdnBjaS5jDQo+PiBAQCAtMTcsNiArMTcsMTQgQEANCj4+ICAgDQo+PiAgICNkZWZpbmUgUkVH
SVNURVJfT0ZGU0VUKGFkZHIpICAoIChhZGRyKSAmIDB4MDAwMDBmZmYpDQo+PiAgIA0KPj4gK3N0
cnVjdCB2cGNpX21taW9fcHJpdiB7DQo+PiArICAgIC8qDQo+PiArICAgICAqIFNldCB0byB0cnVl
IGlmIHRoZSBNTUlPIGhhbmRsZXJzIHdlcmUgc2V0IHVwIGZvciB0aGUgZW11bGF0ZWQNCj4+ICsg
ICAgICogRUNBTSBob3N0IFBDSSBicmlkZ2UuDQo+PiArICAgICAqLw0KPj4gKyAgICBib29sIGlz
X3ZpcnRfZWNhbTsNCj4+ICt9Ow0KPj4gKw0KPj4gICAvKiBEbyBzb21lIHNhbml0eSBjaGVja3Mu
ICovDQo+PiAgIHN0YXRpYyBib29sIHZwY2lfbW1pb19hY2Nlc3NfYWxsb3dlZCh1bnNpZ25lZCBp
bnQgcmVnLCB1bnNpZ25lZCBpbnQgbGVuKQ0KPj4gICB7DQo+PiBAQCAtMzgsNiArNDYsNyBAQCBz
dGF0aWMgaW50IHZwY2lfbW1pb19yZWFkKHN0cnVjdCB2Y3B1ICp2LCBtbWlvX2luZm9fdCAqaW5m
bywNCj4+ICAgICAgIHBjaV9zYmRmX3Qgc2JkZjsNCj4+ICAgICAgIHVuc2lnbmVkIGxvbmcgZGF0
YSA9IH4wVUw7DQo+PiAgICAgICB1bnNpZ25lZCBpbnQgc2l6ZSA9IDFVIDw8IGluZm8tPmRhYnQu
c2l6ZTsNCj4+ICsgICAgc3RydWN0IHZwY2lfbW1pb19wcml2ICpwcml2ID0gKHN0cnVjdCB2cGNp
X21taW9fcHJpdiAqKXA7DQo+PiAgIA0KPj4gICAgICAgc2JkZi5zYmRmID0gTU1DRkdfQkRGKGlu
Zm8tPmdwYSk7DQo+PiAgICAgICByZWcgPSBSRUdJU1RFUl9PRkZTRVQoaW5mby0+Z3BhKTsNCj4+
IEBAIC00NSw2ICs1NCwxMyBAQCBzdGF0aWMgaW50IHZwY2lfbW1pb19yZWFkKHN0cnVjdCB2Y3B1
ICp2LCBtbWlvX2luZm9fdCAqaW5mbywNCj4+ICAgICAgIGlmICggIXZwY2lfbW1pb19hY2Nlc3Nf
YWxsb3dlZChyZWcsIHNpemUpICkNCj4+ICAgICAgICAgICByZXR1cm4gMDsNCj4+ICAgDQo+PiAr
ICAgIC8qDQo+PiArICAgICAqIEZvciB0aGUgcGFzc2VkIHRocm91Z2ggZGV2aWNlcyB3ZSBuZWVk
IHRvIG1hcCB0aGVpciB2aXJ0dWFsIFNCREYNCj4+ICsgICAgICogdG8gdGhlIHBoeXNpY2FsIFBD
SSBkZXZpY2UgYmVpbmcgcGFzc2VkIHRocm91Z2guDQo+PiArICAgICAqLw0KPj4gKyAgICBpZiAo
IHByaXYtPmlzX3ZpcnRfZWNhbSAmJiAhcGNpX3RyYW5zbGF0ZV92aXJ0dWFsX2RldmljZSh2LCAm
c2JkZikgKQ0KPj4gKyAgICAgICAgICAgIHJldHVybiAxOw0KPj4gKw0KPj4gICAgICAgZGF0YSA9
IHZwY2lfcmVhZChzYmRmLCByZWcsIG1pbig0dSwgc2l6ZSkpOw0KPj4gICAgICAgaWYgKCBzaXpl
ID09IDggKQ0KPj4gICAgICAgICAgIGRhdGEgfD0gKHVpbnQ2NF90KXZwY2lfcmVhZChzYmRmLCBy
ZWcgKyA0LCA0KSA8PCAzMjsNCj4+IEBAIC02MSw2ICs3Nyw3IEBAIHN0YXRpYyBpbnQgdnBjaV9t
bWlvX3dyaXRlKHN0cnVjdCB2Y3B1ICp2LCBtbWlvX2luZm9fdCAqaW5mbywNCj4+ICAgICAgIHBj
aV9zYmRmX3Qgc2JkZjsNCj4+ICAgICAgIHVuc2lnbmVkIGxvbmcgZGF0YSA9IHI7DQo+PiAgICAg
ICB1bnNpZ25lZCBpbnQgc2l6ZSA9IDFVIDw8IGluZm8tPmRhYnQuc2l6ZTsNCj4+ICsgICAgc3Ry
dWN0IHZwY2lfbW1pb19wcml2ICpwcml2ID0gKHN0cnVjdCB2cGNpX21taW9fcHJpdiAqKXA7DQo+
PiAgIA0KPj4gICAgICAgc2JkZi5zYmRmID0gTU1DRkdfQkRGKGluZm8tPmdwYSk7DQo+PiAgICAg
ICByZWcgPSBSRUdJU1RFUl9PRkZTRVQoaW5mby0+Z3BhKTsNCj4+IEBAIC02OCw2ICs4NSwxMyBA
QCBzdGF0aWMgaW50IHZwY2lfbW1pb193cml0ZShzdHJ1Y3QgdmNwdSAqdiwgbW1pb19pbmZvX3Qg
KmluZm8sDQo+PiAgICAgICBpZiAoICF2cGNpX21taW9fYWNjZXNzX2FsbG93ZWQocmVnLCBzaXpl
KSApDQo+PiAgICAgICAgICAgcmV0dXJuIDA7DQo+PiAgIA0KPj4gKyAgICAvKg0KPj4gKyAgICAg
KiBGb3IgdGhlIHBhc3NlZCB0aHJvdWdoIGRldmljZXMgd2UgbmVlZCB0byBtYXAgdGhlaXIgdmly
dHVhbCBTQkRGDQo+PiArICAgICAqIHRvIHRoZSBwaHlzaWNhbCBQQ0kgZGV2aWNlIGJlaW5nIHBh
c3NlZCB0aHJvdWdoLg0KPj4gKyAgICAgKi8NCj4+ICsgICAgaWYgKCBwcml2LT5pc192aXJ0X2Vj
YW0gJiYgIXBjaV90cmFuc2xhdGVfdmlydHVhbF9kZXZpY2UodiwgJnNiZGYpICkNCj4+ICsgICAg
ICAgICAgICByZXR1cm4gMTsNCj4+ICsNCj4+ICAgICAgIHZwY2lfd3JpdGUoc2JkZiwgcmVnLCBt
aW4oNHUsIHNpemUpLCBkYXRhKTsNCj4+ICAgICAgIGlmICggc2l6ZSA9PSA4ICkNCj4+ICAgICAg
ICAgICB2cGNpX3dyaXRlKHNiZGYsIHJlZyArIDQsIDQsIGRhdGEgPj4gMzIpOw0KPj4gQEAgLTgw
LDEzICsxMDQsNDggQEAgc3RhdGljIGNvbnN0IHN0cnVjdCBtbWlvX2hhbmRsZXJfb3BzIHZwY2lf
bW1pb19oYW5kbGVyID0gew0KPj4gICAgICAgLndyaXRlID0gdnBjaV9tbWlvX3dyaXRlLA0KPj4g
ICB9Ow0KPj4gICANCj4+ICsvKg0KPj4gKyAqIFRoZXJlIGFyZSB0aHJlZSAgb3JpZ2luYXRvcnMg
Zm9yIHRoZSBQQ0kgY29uZmlndXJhdGlvbiBzcGFjZSBhY2Nlc3M6DQo+PiArICogMS4gVGhlIGRv
bWFpbiB0aGF0IG93bnMgcGh5c2ljYWwgaG9zdCBicmlkZ2U6IE1NSU8gaGFuZGxlcnMgYXJlDQo+
PiArICogICAgdGhlcmUgc28gd2UgY2FuIHVwZGF0ZSB2UENJIHJlZ2lzdGVyIGhhbmRsZXJzIHdp
dGggdGhlIHZhbHVlcw0KPj4gKyAqICAgIHdyaXR0ZW4gYnkgdGhlIGhhcmR3YXJlIGRvbWFpbiwg
ZS5nLiBwaHlzaWNhbCB2aWV3IG9mIHRoZSByZWdpc3RlcnMvDQo+PiArICogICAgY29uZmlndXJh
dGlvbiBzcGFjZS4NCj4+ICsgKiAyLiBHdWVzdCBhY2Nlc3MgdG8gdGhlIHBhc3NlZCB0aHJvdWdo
IFBDSSBkZXZpY2VzOiB3ZSBuZWVkIHRvIHByb3Blcmx5DQo+PiArICogICAgbWFwIHZpcnR1YWwg
YnVzIHRvcG9sb2d5IHRvIHRoZSBwaHlzaWNhbCBvbmUsIGUuZy4gcGFzcyB0aGUgY29uZmlndXJh
dGlvbg0KPj4gKyAqICAgIHNwYWNlIGFjY2VzcyB0byB0aGUgY29ycmVzcG9uZGluZyBwaHlzaWNh
bCBkZXZpY2VzLg0KPj4gKyAqIDMuIEVtdWxhdGVkIGhvc3QgUENJIGJyaWRnZSBhY2Nlc3MuIEl0
IGRvZXNuJ3QgZXhpc3QgaW4gdGhlIHBoeXNpY2FsDQo+PiArICogICAgdG9wb2xvZ3ksIGUuZy4g
aXQgY2FuJ3QgYmUgbWFwcGVkIHRvIHNvbWUgcGh5c2ljYWwgaG9zdCBicmlkZ2UuDQo+PiArICog
ICAgU28sIGFsbCBhY2Nlc3MgdG8gdGhlIGhvc3QgYnJpZGdlIGl0c2VsZiBuZWVkcyB0byBiZSB0
cmFwcGVkIGFuZA0KPj4gKyAqICAgIGVtdWxhdGVkLg0KPj4gKyAqLw0KPj4gICBzdGF0aWMgaW50
IHZwY2lfc2V0dXBfbW1pb19oYW5kbGVyKHN0cnVjdCBkb21haW4gKmQsDQo+PiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdl
KQ0KPj4gICB7DQo+PiAtICAgIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnID0gYnJpZGdl
LT5jZmc7DQo+PiArICAgIHN0cnVjdCB2cGNpX21taW9fcHJpdiAqcHJpdjsNCj4+ICsNCj4+ICsg
ICAgcHJpdiA9IHh6YWxsb2Moc3RydWN0IHZwY2lfbW1pb19wcml2KTsNCj4+ICsgICAgaWYgKCAh
cHJpdiApDQo+PiArICAgICAgICByZXR1cm4gLUVOT01FTTsNCj4+ICsNCj4+ICsgICAgcHJpdi0+
aXNfdmlydF9lY2FtID0gIWlzX2hhcmR3YXJlX2RvbWFpbihkKTsNCj4+ICAgDQo+PiAtICAgIHJl
Z2lzdGVyX21taW9faGFuZGxlcihkLCAmdnBjaV9tbWlvX2hhbmRsZXIsDQo+PiAtICAgICAgICAg
ICAgICAgICAgICAgICAgICBjZmctPnBoeXNfYWRkciwgY2ZnLT5zaXplLCBOVUxMKTsNCj4+ICsg
ICAgaWYgKCBpc19oYXJkd2FyZV9kb21haW4oZCkgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBz
dHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNmZyA9IGJyaWRnZS0+Y2ZnOw0KPj4gKw0KPj4gKyAg
ICAgICAgYnJpZGdlLT5tbWlvX3ByaXYgPSBwcml2Ow0KPj4gKyAgICAgICAgcmVnaXN0ZXJfbW1p
b19oYW5kbGVyKGQsICZ2cGNpX21taW9faGFuZGxlciwNCj4+ICsgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjZmctPnBoeXNfYWRkciwgY2ZnLT5zaXplLA0KPj4gKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHByaXYpOw0KPj4gKyAgICB9DQo+PiArICAgIGVsc2UNCj4+ICsgICAg
ew0KPj4gKyAgICAgICAgZC0+dnBjaV9tbWlvX3ByaXYgPSBwcml2Ow0KPj4gKyAgICAgICAgLyog
R3Vlc3QgZG9tYWlucyB1c2Ugd2hhdCBpcyBwcm9ncmFtbWVkIGluIHRoZWlyIGRldmljZSB0cmVl
LiAqLw0KPj4gKyAgICAgICAgcmVnaXN0ZXJfbW1pb19oYW5kbGVyKGQsICZ2cGNpX21taW9faGFu
ZGxlciwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBHVUVTVF9WUENJX0VDQU1f
QkFTRSwgR1VFU1RfVlBDSV9FQ0FNX1NJWkUsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcHJpdik7DQo+PiArICAgIH0NCj4+ICAgICAgIHJldHVybiAwOw0KPj4gICB9DQo+PiAg
IA0KPj4gQEAgLTk1LDEzICsxNTQsMTYgQEAgaW50IGRvbWFpbl92cGNpX2luaXQoc3RydWN0IGRv
bWFpbiAqZCkNCj4+ICAgICAgIGlmICggIWhhc192cGNpKGQpICkNCj4+ICAgICAgICAgICByZXR1
cm4gMDsNCj4+ICAgDQo+PiAtICAgIGlmICggaXNfaGFyZHdhcmVfZG9tYWluKGQpICkNCj4+IC0g
ICAgICAgIHJldHVybiBwY2lfaG9zdF9pdGVyYXRlX2JyaWRnZXMoZCwgdnBjaV9zZXR1cF9tbWlv
X2hhbmRsZXIpOw0KPj4gLQ0KPj4gLSAgICAvKiBHdWVzdCBkb21haW5zIHVzZSB3aGF0IGlzIHBy
b2dyYW1tZWQgaW4gdGhlaXIgZGV2aWNlIHRyZWUuICovDQo+PiAtICAgIHJlZ2lzdGVyX21taW9f
aGFuZGxlcihkLCAmdnBjaV9tbWlvX2hhbmRsZXIsDQo+PiAtICAgICAgICAgICAgICAgICAgICAg
ICAgICBHVUVTVF9WUENJX0VDQU1fQkFTRSwgR1VFU1RfVlBDSV9FQ0FNX1NJWkUsIE5VTEwpOw0K
Pj4gKyAgICByZXR1cm4gcGNpX2hvc3RfaXRlcmF0ZV9icmlkZ2VzKGQsIHZwY2lfc2V0dXBfbW1p
b19oYW5kbGVyKTsNCj4+ICt9DQo+PiAgIA0KPj4gK3N0YXRpYyBpbnQgZG9tYWluX3ZwY2lfZnJl
ZV9jYihzdHJ1Y3QgZG9tYWluICpkLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UpDQo+PiArew0KPj4gKyAgICBpZiAoIGlz
X2hhcmR3YXJlX2RvbWFpbihkKSApDQo+PiArICAgICAgICBYRlJFRShicmlkZ2UtPm1taW9fcHJp
dik7DQo+PiArICAgIGVsc2UNCj4+ICsgICAgICAgIFhGUkVFKGQtPnZwY2lfbW1pb19wcml2KTsN
Cj4+ICAgICAgIHJldHVybiAwOw0KPj4gICB9DQo+PiAgIA0KPj4gQEAgLTEyNCw2ICsxODYsMTMg
QEAgaW50IGRvbWFpbl92cGNpX2dldF9udW1fbW1pb19oYW5kbGVycyhzdHJ1Y3QgZG9tYWluICpk
KQ0KPj4gICAgICAgcmV0dXJuIGNvdW50Ow0KPj4gICB9DQo+PiAgIA0KPj4gK3ZvaWQgZG9tYWlu
X3ZwY2lfZnJlZShzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gK3sNCj4+ICsgICAgaWYgKCAhaGFzX3Zw
Y2koZCkgKQ0KPj4gKyAgICAgICAgcmV0dXJuOw0KPj4gKw0KPj4gKyAgICBwY2lfaG9zdF9pdGVy
YXRlX2JyaWRnZXMoZCwgZG9tYWluX3ZwY2lfZnJlZV9jYik7DQo+PiArfQ0KPj4gICAvKg0KPj4g
ICAgKiBMb2NhbCB2YXJpYWJsZXM6DQo+PiAgICAqIG1vZGU6IEMNCj4+IGRpZmYgLS1naXQgYS94
ZW4vYXJjaC9hcm0vdnBjaS5oIGIveGVuL2FyY2gvYXJtL3ZwY2kuaA0KPj4gaW5kZXggMjdhMmIw
NjlhYmQyLi4zOGU1YTI4YzBkOTUgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vdnBjaS5o
DQo+PiArKysgYi94ZW4vYXJjaC9hcm0vdnBjaS5oDQo+PiBAQCAtMTgsNiArMTgsNyBAQA0KPj4g
ICAjaWZkZWYgQ09ORklHX0hBU19WUENJDQo+PiAgIGludCBkb21haW5fdnBjaV9pbml0KHN0cnVj
dCBkb21haW4gKmQpOw0KPj4gICBpbnQgZG9tYWluX3ZwY2lfZ2V0X251bV9tbWlvX2hhbmRsZXJz
KHN0cnVjdCBkb21haW4gKmQpOw0KPj4gK3ZvaWQgZG9tYWluX3ZwY2lfZnJlZShzdHJ1Y3QgZG9t
YWluICpkKTsNCj4+ICAgI2Vsc2UNCj4+ICAgc3RhdGljIGlubGluZSBpbnQgZG9tYWluX3ZwY2lf
aW5pdChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gICB7DQo+PiBAQCAtMjgsNiArMjksOCBAQCBzdGF0
aWMgaW5saW5lIGludCBkb21haW5fdnBjaV9nZXRfbnVtX21taW9faGFuZGxlcnMoc3RydWN0IGRv
bWFpbiAqZCkNCj4+ICAgew0KPj4gICAgICAgcmV0dXJuIDA7DQo+PiAgIH0NCj4+ICsNCj4+ICtz
dGF0aWMgaW5saW5lIHZvaWQgZG9tYWluX3ZwY2lfZnJlZShzdHJ1Y3QgZG9tYWluICpkKSB7IH0N
Cj4+ICAgI2VuZGlmDQo+PiAgIA0KPj4gICAjZW5kaWYgLyogX19BUkNIX0FSTV9WUENJX0hfXyAq
Lw0KPj4gZGlmZiAtLWdpdCBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jIGIveGVuL2Ry
aXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+IGluZGV4IDQ1NTJhY2U4NTVlMC4uNTc5YzY5NDdj
YzM1IDEwMDY0NA0KPj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+ICsr
KyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+PiBAQCAtODkwLDYgKzg5MCwzMSBA
QCBpbnQgcGNpX3JlbW92ZV92aXJ0dWFsX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBz
dHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+ICAgICAgIHJldHVybiAwOw0KPj4gICB9DQo+PiAgIA0K
Pj4gKy8qDQo+PiArICogRmluZCB0aGUgcGh5c2ljYWwgZGV2aWNlIHdoaWNoIGlzIG1hcHBlZCB0
byB0aGUgdmlydHVhbCBkZXZpY2UNCj4+ICsgKiBhbmQgdHJhbnNsYXRlIHZpcnR1YWwgU0JERiB0
byB0aGUgcGh5c2ljYWwgb25lLg0KPj4gKyAqLw0KPj4gK2Jvb2wgcGNpX3RyYW5zbGF0ZV92aXJ0
dWFsX2RldmljZShzdHJ1Y3QgdmNwdSAqdiwgcGNpX3NiZGZfdCAqc2JkZikNCj4gV2h5IHN0cnVj
dCB2Y3B1LCB3aGVuIHlvdSBvbmx5IG5lZWQgLi4uDQo+DQo+PiArew0KPj4gKyAgICBzdHJ1Y3Qg
ZG9tYWluICpkID0gdi0+ZG9tYWluOw0KPiAuLi4gdGhpcz8gSXQncyBhbHNvIG5vdCByZWFsbHkg
bG9naWNhbCBmb3IgdGhpcyBmdW5jdGlvbiB0byB0YWtlIGENCj4gc3RydWN0IHZjcHUsIGFzIHRo
ZSB0cmFuc2xhdGlvbiBzaG91bGQgYmUgdW5pZm9ybSB3aXRoaW4gYSBkb21haW4uDQpBZ3JlZSwg
c3RydWN0IGRvbWFpbiBpcyBqdXN0IGVub3VnaA0KPg0KPiBBbHNvIC0gY29uc3QgcGxlYXNlIChh
cyBzYWlkIGVsc2V3aGVyZSBiZWZvcmUsIGlkZWFsbHkgd2hlcmV2ZXIgcG9zc2libGUNCj4gYW5k
IHNlbnNpYmxlKS4NCk9rDQo+DQo+PiArICAgIHN0cnVjdCB2cGNpX2RldiAqdmRldjsNCj4+ICsg
ICAgYm9vbCBmb3VuZCA9IGZhbHNlOw0KPj4gKw0KPj4gKyAgICBwY2lkZXZzX2xvY2soKTsNCj4+
ICsgICAgbGlzdF9mb3JfZWFjaF9lbnRyeSAoIHZkZXYsICZkLT52ZGV2X2xpc3QsIGxpc3QgKQ0K
Pj4gKyAgICB7DQo+PiArICAgICAgICBpZiAoIHZkZXYtPnNiZGYuc2JkZiA9PSBzYmRmLT5zYmRm
ICkNCj4+ICsgICAgICAgIHsNCj4+ICsgICAgICAgICAgICAvKiBSZXBsYWNlIHZpcnR1YWwgU0JE
RiB3aXRoIHRoZSBwaHlzaWNhbCBvbmUuICovDQo+PiArICAgICAgICAgICAgKnNiZGYgPSB2ZGV2
LT5wZGV2LT5zYmRmOw0KPj4gKyAgICAgICAgICAgIGZvdW5kID0gdHJ1ZTsNCj4+ICsgICAgICAg
ICAgICBicmVhazsNCj4+ICsgICAgICAgIH0NCj4+ICsgICAgfQ0KPiBGb3IgYSBEb21VIHdpdGgg
anVzdCBvbmUgb3IgYXQgbW9zdCBhIGNvdXBsZSBvZiBkZXZpY2VzLCBzdWNoIGEgYnJ1dGUNCj4g
Zm9yY2UgbG9va3VwIG1heSBiZSBmaW5lLiBXaGF0IGFib3V0IERvbTAgdGhvdWdoPyBUaGUgcGh5
c2ljYWwgdG9wb2xvZ3kNCj4gZ2V0cyBzcGxpdCBhdCB0aGUgc2VnbWVudCBsZXZlbCwgc28gbWF5
YmUgdGhpcyB3b3VsZCBieSBhIHJlYXNvbmFibGUNCj4gZ3JhbnVsYXJpdHkgaGVyZSBhcyB3ZWxs
Pw0KDQpOb3Qgc3VyZSBJIGFtIGZvbGxvd2luZyB3aHkgdG9wb2xvZ3kgbWF0dGVycyBoZXJlLiBX
ZSBhcmUganVzdCB0cnlpbmcgdG8NCg0KbWF0Y2ggb25lIFNCREYgKGFzIHNlZW4gYnkgdGhlIGd1
ZXN0KSB0byBvdGhlciBTQkRGIChwaHlzaWNhbCwNCg0KYXMgc2VlbiBieSBEb20wKSwgc28gd2Ug
Y2FuIHByb3h5IERvbVUncyBjb25maWd1cmF0aW9uIHNwYWNlIGFjY2Vzcw0KDQp0byB0aGUgcHJv
cGVyIGRldmljZSBpbiBEb20wLg0KDQo+DQo+PiArICAgIHBjaWRldnNfdW5sb2NrKCk7DQo+PiAr
ICAgIHJldHVybiBmb3VuZDsNCj4gTml0OiBCbGFuayBsaW5lIHBsZWFzZSBhaGVhZCBvZiB0aGUg
bWFpbiAicmV0dXJuIiBvZiBhIGZ1bmN0aW9uLg0KU3VyZQ0KPg0KPj4gK30NCj4+ICsNCj4+ICAg
LyogQ2FsbGVyIHNob3VsZCBob2xkIHRoZSBwY2lkZXZzX2xvY2sgKi8NCj4+ICAgc3RhdGljIGlu
dCBkZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgdWludDE2X3Qgc2VnLCB1aW50OF90
IGJ1cywNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDhfdCBkZXZmbikNCj4g
U2VlaW5nIHRoaXMgZnVuY3Rpb24gaW4gY29udGV4dCAod2hpY2ggcGF0Y2ggMiBhZGRzIHdpdGhv
dXQgYW55ICNpZmRlZg0KPiBhcm91bmQgaXQgYWZhaWNzKSwNCg0KSSBiZWxpZXZlIHlvdSBhcmUg
dGFsa2luZyBhYm91dCB2cGNpX2RlYXNzaWduX2RldmljZSBoZXJlDQoNCnZwY2lfe2Fzc2lnbnxk
ZWFzc2lnbn1fZGV2aWNlIHNlZW0gdG8gYmUgbm90IGNhbGxlZCBvbiB4ODYgUFZIIGFzIG9mIG5v
dywNCg0KdGhpcyBpcyB0cnVlLg0KDQo+ICAgd2lsbCB0aGlzIG5ldyBmdW5jdGlvbiBuZWVkbGVz
c2x5IGJlIGJ1aWx0IG9uIHg4NiBhcw0KPiB3ZWxsPw0KDQpJdCB3aWxsIGF0IHRoZSBtb21lbnQu
IEJ1dCBpbiBvcmRlciB0byBhdm9pZCBpZmRlZmVyeSBJIHdvdWxkIGxpa2UNCg0KdG8gc3RpbGwg
aW1wbGVtZW50IGl0IGFzIGFuIGVtcHR5IGZ1bmN0aW9uIGZvciB4ODYuDQoNCj4gICAoSSBkaWRu
J3QgbG9vayBhdCBvdGhlciBpbnRlcm1lZGlhdGUgcGF0Y2hlcyB5ZXQsIHNvIHBsZWFzZQ0KPiBm
b3JnaXZlIGlmIEkndmUgbWlzc2VkIHRoZSBhZGRpdGlvbiBvZiBhbiAjaWZkZWYuKQ0KDQpTbywg
SSBjYW4gZ2F0ZSB0aGlzIHdpdGggSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVCBpbiBwYXRjaCAyDQoN
CihIQVNfVlBDSV9HVUVTVF9TVVBQT1JUIGlzIGludHJvZHVjZWQgaW4gcGF0Y2ggNCwgc28gSSds
bCBtb3ZlIGl0IHRvIDIpDQoNCkRvZXMgdGhpcyBzb3VuZCBnb29kPw0KDQo+DQo+IEphbg0KPg0K
VGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 12:16:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 12:16:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196781.349701 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpYV-0007YG-RU; Mon, 27 Sep 2021 12:16:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196781.349701; Mon, 27 Sep 2021 12:16:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpYV-0007Y9-N2; Mon, 27 Sep 2021 12:16:23 +0000
Received: by outflank-mailman (input) for mailman id 196781;
 Mon, 27 Sep 2021 12:16:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ztgo=OR=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mUpYU-0007Y3-Gx
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 12:16:22 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id aaedc7b3-97a2-4f4e-b5f4-ec30717f7fad;
 Mon, 27 Sep 2021 12:16:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aaedc7b3-97a2-4f4e-b5f4-ec30717f7fad
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632744980;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=ISlS+b238wHQMto59LXb1YFZ97aWUwHgdekuHbUPwtU=;
  b=IBD3xLZtmlWFRKC4Ak1vMiD94S4MCDwFHkGCGe/4vH9lk3uS2RpQ2mwg
   y11BF/0/WDg8dZIAU0Of1yojmW91if/BG8skQc5ss7YZgmlu3pS/nsgbW
   v6LPXhyLYkg3rt9p95l4aDEA2zRffJTcBakI/Q6VbZhMP5u/oN0HA6flk
   Q=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: uCk6CuAccNsBOtg5btp5+HlDZuV5MX8AbitJcUPVQKaAJpbtzcztasK8hb/Jx8w5z/5zbSmGvx
 6t0MeUGSndKfeUxXUmjKGxdZsKw2lDCIs8zM96UGxm+qZ3rvclx2gLoogtRU0dBErpMLGn7BhQ
 keqJGQNbmU5KwtcdPsQxjqMFUsB2AH7bOMqXBBHCj/lKXL/I4SDGg6oTyHPQOV5zA99Uv6LmEn
 ePkrPnbBax4M+t0+3C9XX0UgR/HtvZbDCL8C6QLFdgY67nEutIUHwH5ZIsT3sxBE5Dv7MjWKxk
 ooOqpaLtrp5DMRPau7cgFmAz
X-SBRS: 5.1
X-MesageID: 53694552
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ZZOE6Kr5xiyKlbvxJ6gR5UGMETFeBmKhYhIvgKrLsJaIsI4StFCzt
 garIBmAPvneYmryfN91b4W/o0pQu5/Wn4RrQAZtrSxhFH8QpJuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlZT4vE2xbuKU5NTsY0idfic5Dnd84f5fs7Rh2Ncw0ILmW1rlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCnayBGCVuYIrloc1DXRtKVHhZAKh49ZaSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFIoZpnFnyyCfFfs8SIrPa67L+cVZzHE7gcUm8fP2O
 pFHOGYyMU6ojxtnM1w0T49igbyRqXjHKjhZq1u2mfMT2j2GpOB2+Oe0a4eEEjCQfu1OhVqRr
 G/C+2X/AzkZOcaZxD7D9Wij7sfNgCf6VYQ6BLC+sPlwjzW7ynQIDRwKVXOyufSjllOlQNVbN
 lAV/Sw166M18SSDQtbxUhKiqVaYrxUcXJxWCOR85waTooLW6QuEAmkPThZadccr8sQxQFQCx
 lKP2t/kGzFrmLmUUm6GsKeZqyuoPioYJnNEYjULJSMa5/HzrYd1iQjAJv5iD6ydntDzASv3w
 T2BsG45nbp7sCIQ//zlpxad2Wvq/8WXCF5ujunKYo67xjNINaziOauB0HGF4/RSa9++c1y/s
 VFRzqBy89syJZ2KkSWMRsAEE7eo++uJPVXgvLJ/I3Uy327zoiHyJOi89Bk7dB8wbZpUEdP8S
 BWL0T698qO/K5dDgUVfTYWrF4wOxLXsGM7pX/TZBjalSsMqL1fWlM2CiFT54owMrKTOuf1hU
 Xt4WZz1ZZr/NUiB5GHvLwv6+eV3rh3SPUuJGfjGI+2PiNJynkK9R7YfK0epZesk9q6Cqwi92
 48BbJHUm0wDAbOuOHO/HWsvwbYidiFT6Xfe8ZA/SwJ+ClA+RDFJ5wH5m9vNhLCJb4wKz7yVr
 xlRq2dTyUblhG2vFOl5QioLVV8bZr4m9ShTFXV1ZT6AgiF/Ca7yvPZ3X8ZmJtEPqb08pcOYu
 tFYIq1s9NwUEW+Zk9ncBLGgxLFfmOOD31nTYHb1PGZlL/aNhWXho7fZQ+cmzwFWZgLfiCf0i
 +HIOtrzTcVRSgJ8ItzRbf7znVq9sWJEwLB5XlfSI8kVc0LpqdA4Jyv0h/4xAscNNRScmWfKi
 1fIWU8V9bvXvos40NjVnqTY/Y2nJPRzQxhBFG7B4LfoaSSDpji/wZVNWfqjdCzGUD+m472rY
 OhYlqmuMPAOkFtQnZB7FrJnkfA369f1/ucIxQV4BnTbKV+sD+o4cHWB2MBOsIxLx6NY5lTqC
 h7epIECNOzQas3/EVMXKA40Vci51KkZymvI8PA4AETm/ysrrrCJZlpfYkuXgytHIborbI58m
 bU9uNQb4hCUgwYxNorUlThd8mmBIyBSU6gjsZ1GUobnhhBylwNHaJ3YTCT3/IuOe5NHNUxze
 m2Yg6/LhrJ9wEveciVsSSiRjLQF3Zle6gpXyFIiJkiSnouXj/A66xRd7DArQ1kH1R5Aye9yZ
 jBmOkAdyX9iJNu0aByvh1yRJjw=
IronPort-HdrOrdr: A9a23:XjjCGKFtnxdyg1lCpLqE6MeALOsnbusQ8zAXP0AYc3Jom+ij5q
 STdZUgpHrJYVkqNU3I9ertBEDEewK6yXcX2/hyAV7BZmnbUQKTRekIh7cKgQeQeBEWntQts5
 uIGJIeNDSfNzdHsfo=
X-IronPort-AV: E=Sophos;i="5.85,326,1624334400"; 
   d="scan'208";a="53694552"
Date: Mon, 27 Sep 2021 13:16:01 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, "Daniel De
 Graaf" <dgdegra@tycho.nsa.gov>, "Daniel P. Smith"
	<dpsmith@apertussolutions.com>
Subject: Ping: [XEN PATCH v2] build: add --full to version.sh to guess
 $(XEN_FULLVERSION)
Message-ID: <YVG2Ae95bRNSLVJf@perard>
References: <20210909143306.466836-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20210909143306.466836-1-anthony.perard@citrix.com>

Anyone to review this?

Thanks.

On Thu, Sep 09, 2021 at 03:33:06PM +0100, Anthony PERARD wrote:
> Running $(MAKE) like that in a $(shell ) while parsing the Makefile
> doesn't work reliably. In some case, make will complain with
> "jobserver unavailable: using -j1.  Add '+' to parent make rule.".
> Also, it isn't possible to distinguish between the output produced by
> the target "xenversion" and `make`'s own output.
> 
> Instead of running make, this patch "improve" `version.sh` to try to
> guess the output of `make xenversion`.
> 
> In order to have version.sh works in more scenario, it will use
> XEN_EXTRAVERSION and XEN_VENDORVERSION from the environment when
> present. As for the cases were those two variables are overridden by a
> make command line arguments, we export them when invoking version.sh
> via a new $(XEN_FULLVERSION) macro.
> 
> That should hopefully get us to having ./version.sh returning the same
> value that `make xenversion` would.
> 
> This fix GitLab CI's build job "debian-unstable-gcc-arm64".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> v2:
> - use $(SHELL) to execute ./version.sh
> - get XEN_EXTRAVERSION from the environment if exist
> - use XEN_VENDORVERSION when exist in the environment
> - introduce a new macro $(XEN_FULLVERSION) whose jobs is to export
>   $(XEN_EXTRAVERSION) and $(XEN_VENDORVERSION) in cases where those
>   are overridden by command line argument of make.
>   (we can't just use make's "export" because it doesn't work when
>   parsing the makefile which is when $(shell ) get's executed for
>   POLICY_FILENAME, so it wouldn't work when running `make -C
>   tools/flask/policy XEN_VENDORVERSION=-os`.)
> ---
>  tools/Rules.mk                     |  5 +++++
>  tools/flask/policy/Makefile.common |  2 +-
>  version.sh                         | 18 +++++++++++++++++-
>  3 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/Rules.mk b/tools/Rules.mk
> index 2907ed2d3972..b022da3336c4 100644
> --- a/tools/Rules.mk
> +++ b/tools/Rules.mk
> @@ -6,6 +6,11 @@ all:
>  -include $(XEN_ROOT)/config/Tools.mk
>  include $(XEN_ROOT)/Config.mk
>  
> +XEN_FULLVERSION=$(shell env \
> +    XEN_EXTRAVERSION=$(XEN_EXTRAVERSION) \
> +    XEN_VENDORVERSION=$(XEN_VENDORVERSION) \
> +    $(SHELL) $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile)
> +
>  export _INSTALL := $(INSTALL)
>  INSTALL = $(XEN_ROOT)/tools/cross-install
>  
> diff --git a/tools/flask/policy/Makefile.common b/tools/flask/policy/Makefile.common
> index bea5ba4b6a40..e5ed58200e75 100644
> --- a/tools/flask/policy/Makefile.common
> +++ b/tools/flask/policy/Makefile.common
> @@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER)
>  #
>  ########################################
>  
> -POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory)
> +POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
>  POLICY_LOADPATH = /boot
>  
>  # List of policy versions supported by the hypervisor
> diff --git a/version.sh b/version.sh
> index e894ee7e0469..c6a5692c1982 100755
> --- a/version.sh
> +++ b/version.sh
> @@ -1,5 +1,21 @@
>  #!/bin/sh
>  
> +opt_full=false
> +while [ $# -gt 1 ]; do
> +    case "$1" in
> +        --full) opt_full=true ;;
> +        *) break ;;
> +    esac
> +    shift
> +done
> +
>  MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "`
>  MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "`
> -printf "%d.%d" $MAJOR $MINOR
> +
> +if $opt_full; then
> +    extraversion=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g')
> +    : ${XEN_EXTRAVERSION:=${extraversion}${XEN_VENDORVERSION}}
> +else
> +    unset XEN_EXTRAVERSION
> +fi
> +printf "%d.%d%s" $MAJOR $MINOR $XEN_EXTRAVERSION
> -- 
> Anthony PERARD
> 

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 12:22:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 12:22:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196805.349711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpdz-0000o8-J1; Mon, 27 Sep 2021 12:22:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196805.349711; Mon, 27 Sep 2021 12:22:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpdz-0000o1-FN; Mon, 27 Sep 2021 12:22:03 +0000
Received: by outflank-mailman (input) for mailman id 196805;
 Mon, 27 Sep 2021 12:22:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUpdx-0000nr-AM; Mon, 27 Sep 2021 12:22:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUpdx-0007S7-1z; Mon, 27 Sep 2021 12:22:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUpdw-0007l0-Ke; Mon, 27 Sep 2021 12:22:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUpdw-0003Eu-KA; Mon, 27 Sep 2021 12:22:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XEAwwoWD06NquXPXK+Es20QWgo3CFTJpzKzHyW4Q02Y=; b=HV0NfwtLHmSTAWM1buiIa+Yjw7
	0z1FeqchR/Et21pECATLjhFAYseesG3y7NJTXmLUkT135MozdJbGv10fTFYYrf8lXHg69tZnkIXTY
	h6PlxGxvZR+RbngYL2bu8pH8qlyxFTWbFd+/Qw8dczIlFB3rEcueoJALUs9p9YDrU9Kg=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165211-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165211: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-rtds:guest-start:fail:heisenbug
    xen-unstable:test-arm64-arm64-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 12:22:00 +0000

flight 165211 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165211/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   8 xen-boot         fail in 165205 pass in 165211
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 165205 pass in 165211
 test-armhf-armhf-xl-rtds     14 guest-start      fail in 165205 pass in 165211
 test-arm64-arm64-xl-credit1   8 xen-boot                   fail pass in 165205

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit1 15 migrate-support-check fail in 165205 never pass
 test-arm64-arm64-xl-credit1 16 saverestore-support-check fail in 165205 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   16 days
Failing since        164951  2021-09-12 00:14:36 Z   15 days   31 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    2 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 12:24:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 12:24:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196812.349726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpgF-0001R8-0K; Mon, 27 Sep 2021 12:24:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196812.349726; Mon, 27 Sep 2021 12:24:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpgE-0001R1-Tg; Mon, 27 Sep 2021 12:24:22 +0000
Received: by outflank-mailman (input) for mailman id 196812;
 Mon, 27 Sep 2021 12:24:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gNz3=OR=apertussolutions.com=dpsmith@srs-us1.protection.inumbo.net>)
 id 1mUpgE-0001Qv-44
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 12:24:22 +0000
Received: from sender4-of-o51.zoho.com (unknown [136.143.188.51])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d5a1e1f2-1f8d-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 12:24:18 +0000 (UTC)
Received: from mail.zoho.com by mx.zohomail.com
 with SMTP id 1632745443098424.98610012230733;
 Mon, 27 Sep 2021 05:24:03 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5a1e1f2-1f8d-11ec-bc44-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; t=1632745450; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=SJO7sMW+0XmbjpryKWFx860kK4E+kpEhrGE2xvlPJcqIN/bN2RnK6tnZW7ySIokX3qqQywyZ1hIxoDmnNSw/gcWt3NVSiZhbThNIInkXuo952b53IHJc28crtBSDX9FKb0JJYIcgmv9g1XwSM5pOTHecpUdnmE10B+5oahBsTl0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1632745450; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; 
	bh=vtWywQbS2r9ACqp5dmOkIUXcJ7Z9kRfzvZcjloTeh+E=; 
	b=YMiNUbdcpwK3d1hft3PyZpnrR4gUqv5t+zwNk7jKyQWWEvyqhqZP71TgZvIOPJLUcrfW9fX7PGhUWUA8/2LMmMDOOcAZN5idnp2FScgQii4Biq2mPIJfamY6Jtpwq2ZMS8pU40eNv3g4gm+zyK4WqS5y4rrKmAujquVoZm21Mk4=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1632745450;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Date:From:To:Cc:Message-ID:In-Reply-To:References:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding;
	bh=vtWywQbS2r9ACqp5dmOkIUXcJ7Z9kRfzvZcjloTeh+E=;
	b=nZdiAVpmkmGat0iCB46yeSpgarEcaH75x3i1aKRBXcAPJwAoky/SQguFOxy2iX5R
	XE2ijTNnVnW0PO2XqlHXIGCUAStpJEzUqaiKgRKrz7NLOBNIoT69D4NUfeHv6801+Ih
	sG6jxQS6EREXbS8KU8PR3Og4eOXBse7UXkD8LEvQ=
Date: Mon, 27 Sep 2021 08:24:03 -0400
From: Daniel Smith <dpsmith@apertussolutions.com>
To: "Anthony PERARD" <anthony.perard@citrix.com>
Cc: "xen-devel" <xen-devel@lists.xenproject.org>,
	"Jan Beulich" <jbeulich@suse.com>,
	"Andrew Cooper" <andrew.cooper3@citrix.com>,
	"Ian Jackson" <iwj@xenproject.org>, "Wei Liu" <wl@xen.org>,
	"George Dunlap" <george.dunlap@citrix.com>,
	"Julien Grall" <julien@xen.org>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Daniel De Graaf" <dgdegra@tycho.nsa.gov>
Message-ID: <17c27364f04.c7fbc53a24194.6291604602462820326@apertussolutions.com>
In-Reply-To: <20210909143306.466836-1-anthony.perard@citrix.com>
References: <20210909143306.466836-1-anthony.perard@citrix.com>
Subject: Re: [XEN PATCH v2] build: add --full to version.sh to guess
 $(XEN_FULLVERSION)
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Importance: Medium
User-Agent: Zoho Mail
X-Mailer: Zoho Mail

Apologies for any formatting issues, responding via web mail client.

---- On Thu, 09 Sep 2021 10:33:06 -0400 Anthony PERARD <anthony.perard@citrix.com> wrote ----

 > Running $(MAKE) like that in a $(shell ) while parsing the Makefile 
 > doesn't work reliably. In some case, make will complain with 
 > "jobserver unavailable: using -j1.  Add '+' to parent make rule.". 
 > Also, it isn't possible to distinguish between the output produced by 
 > the target "xenversion" and `make`'s own output. 
 >  
 > Instead of running make, this patch "improve" `version.sh` to try to 
 > guess the output of `make xenversion`. 
 >  
 > In order to have version.sh works in more scenario, it will use 
 > XEN_EXTRAVERSION and XEN_VENDORVERSION from the environment when 
 > present. As for the cases were those two variables are overridden by a 
 > make command line arguments, we export them when invoking version.sh 
 > via a new $(XEN_FULLVERSION) macro. 
 >  
 > That should hopefully get us to having ./version.sh returning the same 
 > value that `make xenversion` would. 
 >  
 > This fix GitLab CI's build job "debian-unstable-gcc-arm64". 
 >  
 > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com> 
 > --- 
 > v2: 
 > - use $(SHELL) to execute ./version.sh 
 > - get XEN_EXTRAVERSION from the environment if exist 
 > - use XEN_VENDORVERSION when exist in the environment 
 > - introduce a new macro $(XEN_FULLVERSION) whose jobs is to export 
 >  $(XEN_EXTRAVERSION) and $(XEN_VENDORVERSION) in cases where those 
 >  are overridden by command line argument of make. 
 >  (we can't just use make's "export" because it doesn't work when 
 >  parsing the makefile which is when $(shell ) get's executed for 
 >  POLICY_FILENAME, so it wouldn't work when running `make -C 
 >  tools/flask/policy XEN_VENDORVERSION=-os`.) 
 > --- 
 >  tools/Rules.mk                     |  5 +++++ 
 >  tools/flask/policy/Makefile.common |  2 +- 
 >  version.sh                         | 18 +++++++++++++++++- 
 >  3 files changed, 23 insertions(+), 2 deletions(-) 
 >  
 > diff --git a/tools/Rules.mk b/tools/Rules.mk 
 > index 2907ed2d3972..b022da3336c4 100644 
 > --- a/tools/Rules.mk 
 > +++ b/tools/Rules.mk 
 > @@ -6,6 +6,11 @@ all: 
 >  -include $(XEN_ROOT)/config/Tools.mk 
 >  include $(XEN_ROOT)/Config.mk 
 >  
 > +XEN_FULLVERSION=$(shell env \ 
 > +    XEN_EXTRAVERSION=$(XEN_EXTRAVERSION) \ 
 > +    XEN_VENDORVERSION=$(XEN_VENDORVERSION) \ 
 > +    $(SHELL) $(XEN_ROOT)/version.sh --full $(XEN_ROOT)/xen/Makefile) 
 > + 
 >  export _INSTALL := $(INSTALL) 
 >  INSTALL = $(XEN_ROOT)/tools/cross-install 
 >  
 > diff --git a/tools/flask/policy/Makefile.common b/tools/flask/policy/Makefile.common 
 > index bea5ba4b6a40..e5ed58200e75 100644 
 > --- a/tools/flask/policy/Makefile.common 
 > +++ b/tools/flask/policy/Makefile.common 
 > @@ -35,7 +35,7 @@ OUTPUT_POLICY ?= $(BEST_POLICY_VER) 
 >  # 
 >  ######################################## 
 >  
 > -POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(shell $(MAKE) -C $(XEN_ROOT)/xen xenversion --no-print-directory) 
 > +POLICY_FILENAME = $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION) 
 >  POLICY_LOADPATH = /boot 
 >  
 >  # List of policy versions supported by the hypervisor 
 > diff --git a/version.sh b/version.sh 
 > index e894ee7e0469..c6a5692c1982 100755 
 > --- a/version.sh 
 > +++ b/version.sh 
 > @@ -1,5 +1,21 @@ 
 >  #!/bin/sh 
 >  
 > +opt_full=false 
 > +while [ $# -gt 1 ]; do 
 > +    case "$1" in 
 > +        --full) opt_full=true ;; 
 > +        *) break ;; 
 > +    esac 
 > +    shift 
 > +done 
 > + 
 >  MAJOR=`grep "export XEN_VERSION" $1 | sed 's/.*=//g' | tr -s " "` 
 >  MINOR=`grep "export XEN_SUBVERSION" $1 | sed 's/.*=//g' | tr -s " "` 
 > -printf "%d.%d" $MAJOR $MINOR 
 > + 
 > +if $opt_full; then 
 > +    extraversion=$(grep "export XEN_EXTRAVERSION" $1 | sed 's/^.* ?=\s\+//; s/\$([^)]*)//g; s/ //g') 
 > +    : ${XEN_EXTRAVERSION:=${extraversion}${XEN_VENDORVERSION}} 
 > +else 
 > +    unset XEN_EXTRAVERSION 
 > +fi 
 > +printf "%d.%d%s" $MAJOR $MINOR $XEN_EXTRAVERSION 
 > -- 
 > Anthony PERARD 
 >  
 > 

Provided there are no issues with rest of the changes, I see no issues made to the Flask Policy makefile.

Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 12:44:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 12:44:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196828.349737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpzr-0004Bv-Ml; Mon, 27 Sep 2021 12:44:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196828.349737; Mon, 27 Sep 2021 12:44:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUpzr-0004Bo-JN; Mon, 27 Sep 2021 12:44:39 +0000
Received: by outflank-mailman (input) for mailman id 196828;
 Mon, 27 Sep 2021 12:44:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUpzq-0004Bi-1I
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 12:44:38 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id abe46134-1f90-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 12:44:36 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18RBgoh5027442;
 Mon, 27 Sep 2021 12:44:33 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbbnu15eh-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 12:44:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3586.eurprd03.prod.outlook.com (2603:10a6:208:43::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep
 2021 12:44:23 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 12:44:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abe46134-1f90-11ec-bc44-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bIjdeTCFLVxQPh1o43+S0xlYlxBLfeNtZRnqzRkaak3aMPBVYZ6iwQfn8exhHXaxPDEtH8Yim9iFQawzcxOqSnXIHpT5IqR7OWXR8Ca8uDmR9l0DXXrqNJxW9TQ8IVbG6er8DL7Yb9opLR4OFt8FAJbm2kwDho4xK8c4z8JL1n0v66uHxz+oCfAyZOJ12QkK+cgOzMKEhnKBZGoGovQIoCNHnFOJr7Lv4X6+XC6JpVL1NSXkiab/WB3ald6ljxW9zGK4lzjhZvMbrvgmVjS6xNd9tG2h/lK1/KhDrMLTEwVLgRVHkdXt9N0hHVn4re4wcI+ZGj5Y9z7xuZpj8uksVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=VfjX0fwYhkAtqhUwrPCB6lLe2fG9DVOHPAjqtHHiKh0=;
 b=D9zH5jMP1RrMQWmWlS+1D9nUPTU75Oe5+6ClyfsPhk+wr1Iyo3FZbJcuFwGZSJ7dC2pXuFCxAlh7y2eUclC4GNFNO+q6htsejjAuTt+rQ4m0eGfNwtDZSVPVjnsgfmZ5gDtGu0PILt81EREeU/zRNEEr9uci14Tf9yrtaaRZkomLR69lmO1LhCqwU1t0FtVh0uIL3FWLD8UalZGpjrD0ulVYV2u7En2OkK4tfpWK4mXRQqr5TP32v6QyFf/TpPIrQuGvfawpVkgAZJ53cmhdLAs9wv66FERcK4GqXFvKxuFFbUajc31yGqJW2wx2th3gOlCWm+ToycDR+LXyEFr2qg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VfjX0fwYhkAtqhUwrPCB6lLe2fG9DVOHPAjqtHHiKh0=;
 b=5pRJFLqAkw4WXyRVaU6A19pdx0JCxiBxeXj4hznaATVZD8oN8PR0uXLU0AMEu8OQ5YhP/1BBQDZ5an+HUzc3GZw/MP8bJOY3rFgAtyf9L6qbYw4RSbmWqGwDzC3sYfaFo3EvIG/u3uQ6vfKfXrQVMEbSbOAvinu5/zjVaEc2rAWYaOu8mublh+rsf/nV/h10NKTIeRCzWr8yWhp8RGhV4BhoebkLJAPXenSj1NLo5mQGsErEpyg1qPCJO+jdOp4Jn2s+BqBwbT3xtdgAvAYE4Kn1I408N3zaAulydReSmN6mvhX3G0hI3Lm3y8uQ/MMweCEREsqafYP+ltGrzOwOGQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Topic: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Index: AQHXsHo2GZQHQL3fu0+Ql9qa7dzr/Kuz6/8AgAPt2IA=
Date: Mon, 27 Sep 2021 12:44:23 +0000
Message-ID: <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-11-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c56fee2b-6c16-43e8-5819-08d981b488d0
x-ms-traffictypediagnostic: AM0PR03MB3586:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB35869987039FE34E77B3C56EE7A79@AM0PR03MB3586.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5236;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 dzUnLvy0ul79CLhKgMSPCatn4zI/egxwJrWb2MIYmqkpnrA4i/X+D3kkyAeMsmNZHBHUUYNOCLGn6u3lz3le+qpnWo/FGWCYZQW8jU80hchklzlsGbpZPeEHE537glCYnRxYsjcK7PXG84hbjmIIf07oCL1CdvQC5wbp1p/5m+pnnL9QOaRsNdplPpPs3+hZYHDbQRiqDApfaKUzI2B3nHTSp9vQbw9ZOrufeRszpDhWh5JRvAmUZyZ+S243sEyjFM227QHap1xuMdhJF6Tea0HYOiTBjr6OV3puubrtMssT2NC3tflE2PY52xm/5wHpKWvH3EIaCtPAEG7O3166Mj/CjBQctiv63+/GBsL///6xCCjKuM9Lo9qqqkM+4Nt/aj1cpUKDNtVwI05hdGY5jX71zzx6OuuD3M9vVuWrhufOjlDlr0aj/Bte+NBrWegfk9vDDeJ2H9tTg21F2ROWEmyPEWF2rxFDvQXDepc4ReWAvXeekKq1fs5LnlYaeheX80AuL7XBLhUEKT60cRlnUBRARws3GhGARWWDxJNzY1yBezZ4Aj/IRWL0xJ8MYBpISFrAPp7U+x8p+f7dREVOOISO43sUkJ5GQJMM4SC+TAqa2UYwFz1XQ4FCg2hm0ExzJ+bEYxFql6+yXVqJehN7+KkGqfN66IUyy7PcsxJNAuUu2AzSXuSgnIHdJUAOH9U0vXRRo+aaOVcI3d1l1H0CxmxnsxdfDXoEIF9gV4Yd/HYltSwzS0Wwfkg8igKZf9AGIbhZgWbTIysbRPmN85OzlNEza840iJ+54P080SX5bxBSYrk/3ZHbHDLQbsQ3/zCMUCcQjrCvHWeHPSFBo8w3Gw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(966005)(31686004)(316002)(107886003)(66946007)(2616005)(5660300002)(4326008)(2906002)(36756003)(66476007)(66446008)(64756008)(66556008)(8676002)(38070700005)(508600001)(54906003)(76116006)(122000001)(83380400001)(71200400001)(31696002)(8936002)(86362001)(91956017)(55236004)(30864003)(6916009)(186003)(53546011)(6506007)(6512007)(38100700002)(6486002)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?bWZ0bEZueEZrcjJIMTcvOFdpUnBTYlA1UWQxb2cycllOSG5KaXR4bHpPbllH?=
 =?utf-8?B?enllRkFRdXRMMy9BVTM3aFIyU3FXWmlmbGc3OFM0bEdhMjBoK00zSUY4NWdy?=
 =?utf-8?B?RmdkK0ZsREJ3OXc1Unhub2oxRFNpWGc1UmZnZk9mWlZYV21IYXluVUlOaGdp?=
 =?utf-8?B?d1FQaFcwS25qNU1wUW5jSk5yTm5xZFFMa2pCU2EzNkl2TUFvN3dram5zcWtF?=
 =?utf-8?B?b2FuNVZYaEc1NlpTbWxyMlpxanJpV1N3cXhwY1l1QWpCRUp1RnhWdmNvZUpo?=
 =?utf-8?B?TEg5T29qd25NRzFER3BNUC90U05ieUErWmplak52Sld4L0FmdzYwYUNhdFNC?=
 =?utf-8?B?TnNQK2x5RlA0WVhaMjgxSHJkRlBIdVNYVUQra1doejZaWmJtdGJIeHJiRllW?=
 =?utf-8?B?NGZhbXkyd3JJUTZ2dk9SbmdYejlvVlo2ZCtiS3QwQXZYbTlEVkNqYks0L21u?=
 =?utf-8?B?UzgrZUN3TjdGZ1pqZk1QVDlNZnp1Mlg4cTNuRXdtTmUzZ1ZUNDRKaGRSTXpt?=
 =?utf-8?B?UDRrYmF6emc5eEhXT0JpSWttNFc2d3RjdjVGNFV0aVpIQUtESm5iN0lnT081?=
 =?utf-8?B?ZXlaOHZKYllpOVFSQms4amJGdW0xdGFhMk5qZFFwTXdTTkVuSkxYanVyS2p6?=
 =?utf-8?B?WkpOTHRidlFyaG91Z3B5aVZia05GMG5veHYyV0RpMDMwSXREUHdJQUh4UndV?=
 =?utf-8?B?MWt2U3FBV0Q1U1hzVG9WeWlhaUpGOE5rWWtmdmZyNnk3T1d1NkI4aTVWWmpi?=
 =?utf-8?B?elBlN05tbldVbFhucElNWHkyT3Z3d2ZUZk1EdHJSKzVydm5iSzh5TEdxZm9w?=
 =?utf-8?B?VGY0NDEyUDhuV21YMGkvVlU3N3JURnlOM2hvMEQ1enVVVmtmazRsTmhxUWFh?=
 =?utf-8?B?NmtKbGx6Q2ZDRlc1U3E0THFwTGJpbng3OGI0VlR3K3ZTTXRaWmRCTnZid0ZO?=
 =?utf-8?B?SlZDSW51N2NLQnUxNXVtbFdZUUwvM29VZ1I2WVRLOUowYjc4RllUM21TaWQy?=
 =?utf-8?B?L1FDc0p5cjFWMkxVajEzeEYxTTFENW5BZnpwZnpBM1hxYmU5bXA4ZnhOLzZ1?=
 =?utf-8?B?ZTI5WXYzczNYMG5OOHRpOFhvRTB1bHRqOVR3YjNOeE9QN2tjWlQxV3VaMHZC?=
 =?utf-8?B?S3RpWUs0VFBvMWlOSEFUVjRRZk9oNG9PdE8xMFNnMWlKWlIxVEFPbGk2Z3ho?=
 =?utf-8?B?T0tNZzJjQkVPNGFta2cxUDFYNWpwNjdoeVowM3g1V1huTjlYVGlNS1dxREsz?=
 =?utf-8?B?dVNyQ1BwVmliK0xOVDkrVzBQemJJSzFJd1lCZ3VaVEM0MTF2WEphS3J0dnNR?=
 =?utf-8?B?VHJiWlYrSWd5T25JZHV0Q0drYUUwK0V4Nm1OY2kyNTBSaHNpYzNocnFBUC9X?=
 =?utf-8?B?VlNXOFpndFpoN21QUnJ0ZEU1NXA1VTNHa2dDUVRIMXBER2hkU3EwRVpWNDA2?=
 =?utf-8?B?VWdLM20yeE56ZktRTXN0UFg1OU14QkhsUGg2L2RqbGZHRkd2OGNaRG80Tk12?=
 =?utf-8?B?Q2pabzVKUm0vTkVHcURIbWFsTW5RRWROdFVwM2d4ODJvTVczV0p3MFJPV3J4?=
 =?utf-8?B?eFRENko3WnBqRzBmZWlldDBaeUhrb3hIdnFxWlcwb1dQUjBTSFppdkF3dnVT?=
 =?utf-8?B?UkFTK1I1SXh4b0tKQUhrZ3NBK1ExakJlbmJva0dVdWx5WSs4YmNjbTV6cnJF?=
 =?utf-8?B?MnVxSkJKaHcvUUhIM05NL0QvSE56NGNaby9ZUHJDTkRuRUk3ZVJSME9OQjgv?=
 =?utf-8?B?ZmhDbG9PUnlnRmdTdE02VGI1NHdDOS9DVjlCWGVVdFUwK0dFbDBRQXJzdFZh?=
 =?utf-8?B?dm8wUVZ1OWpPTXdudXRoUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <55008F050F0D2C41BB8C5330B51BE5FF@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c56fee2b-6c16-43e8-5819-08d981b488d0
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 12:44:23.8599
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jf/k3EBIppLxC3YJuVWq4FI5LpbtddZ9iHEcUoTHx4T/lom4Jl23tq3HIaQqTmlrHN2B0/p7q/ibH1fMu4d1zs+FQRoP3gyUhxIjFy6K5D0O61zHmUxQh9k7hgyt6M5p
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3586
X-Proofpoint-GUID: pg-hgMKx3B4R1gyV0T-H9NwPUCH5ayrU
X-Proofpoint-ORIG-GUID: pg-hgMKx3B4R1gyV0T-H9NwPUCH5ayrU
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 mlxlogscore=999 lowpriorityscore=0 priorityscore=1501 adultscore=0
 spamscore=0 clxscore=1015 malwarescore=0 bulkscore=0 phishscore=0
 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270088

DQpPbiAyNS4wOS4yMSAwMzo0NCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBQQ0kgaG9zdCBicmlkZ2VzIGFyZSBzcGVjaWFsIGRldmljZXMgaW4gdGVybXMgb2Yg
aW1wbGVtZW50aW5nIFBDSQ0KPj4gcGFzc3Rocm91Z2guIEFjY29yZGluZyB0byBbMV0gdGhlIGN1
cnJlbnQgaW1wbGVtZW50YXRpb24gZGVwZW5kcyBvbg0KPj4gRG9tYWluLTAgdG8gcGVyZm9ybSB0
aGUgaW5pdGlhbGl6YXRpb24gb2YgdGhlIHJlbGV2YW50IFBDSSBob3N0DQo+PiBicmlkZ2UgaGFy
ZHdhcmUgYW5kIHBlcmZvcm0gUENJIGRldmljZSBlbnVtZXJhdGlvbi4gSW4gb3JkZXIgdG8NCj4+
IGFjaGlldmUgdGhhdCBvbmUgb2YgdGhlIHJlcXVpcmVkIGNoYW5nZXMgaXMgdG8gbm90IG1hcCBh
bGwgdGhlIG1lbW9yeQ0KPj4gcmFuZ2VzIGluIG1hcF9yYW5nZV90b19kb21haW4gYXMgd2UgdHJh
dmVyc2UgdGhlIGRldmljZSB0cmVlIG9uIHN0YXJ0dXANCj4+IGFuZCBwZXJmb3JtIHNvbWUgYWRk
aXRpb25hbCBjaGVja3MgaWYgdGhlIHJhbmdlIG5lZWRzIHRvIGJlIG1hcHBlZCB0bw0KPj4gRG9t
YWluLTAuDQo+Pg0KPj4gVGhlIGdlbmVyaWMgUENJIGhvc3QgY29udHJvbGxlciBkZXZpY2UgdHJl
ZSBiaW5kaW5nIHNheXMgWzJdOg0KPj4gLSByYW5nZXM6IEFzIGRlc2NyaWJlZCBpbiBJRUVFIFN0
ZCAxMjc1LTE5OTQsIGJ1dCBtdXN0IHByb3ZpZGUNCj4+ICAgICAgICAgICAgYXQgbGVhc3QgYSBk
ZWZpbml0aW9uIG9mIG5vbi1wcmVmZXRjaGFibGUgbWVtb3J5LiBPbmUNCj4+ICAgICAgICAgICAg
b3IgYm90aCBvZiBwcmVmZXRjaGFibGUgTWVtb3J5IGFuZCBJTyBTcGFjZSBtYXkgYWxzbw0KPj4g
ICAgICAgICAgICBiZSBwcm92aWRlZC4NCj4+DQo+PiAtIHJlZyAgIDogVGhlIENvbmZpZ3VyYXRp
b24gU3BhY2UgYmFzZSBhZGRyZXNzIGFuZCBzaXplLCBhcyBhY2Nlc3NlZA0KPj4gICAgICAgICAg
ICBmcm9tIHRoZSBwYXJlbnQgYnVzLiAgVGhlIGJhc2UgYWRkcmVzcyBjb3JyZXNwb25kcyB0bw0K
Pj4gICAgICAgICAgICB0aGUgZmlyc3QgYnVzIGluIHRoZSAiYnVzLXJhbmdlIiBwcm9wZXJ0eS4g
IElmIG5vDQo+PiAgICAgICAgICAgICJidXMtcmFuZ2UiIGlzIHNwZWNpZmllZCwgdGhpcyB3aWxs
IGJlIGJ1cyAwICh0aGUgZGVmYXVsdCkuDQo+Pg0KPj4gPkZyb20gdGhlIGFib3ZlIG5vbmUgb2Yg
dGhlIG1lbW9yeSByYW5nZXMgZnJvbSB0aGUgInJhbmdlcyIgcHJvcGVydHkNCj4+IG5lZWRzIHRv
IGJlIG1hcHBlZCB0byBEb21haW4tMCBhdCBzdGFydHVwIGFzIE1NSU8gbWFwcGluZyBpcyBnb2lu
ZyB0bw0KPj4gYmUgaGFuZGxlZCBkeW5hbWljYWxseSBieSB2UENJIGFzIHdlIGFzc2lnbiBQQ0kg
ZGV2aWNlcywgZS5nLiBlYWNoDQo+PiBkZXZpY2UgYXNzaWduZWQgdG8gRG9tYWluLTAvZ3Vlc3Qg
d2lsbCBoYXZlIGl0cyBNTUlPcyBtYXBwZWQvdW5tYXBwZWQNCj4+IGFzIG5lZWRlZCBieSBYZW4u
DQo+Pg0KPj4gVGhlICJyZWciIHByb3BlcnR5IGNvdmVycyBub3Qgb25seSBFQ0FNIHNwYWNlLCBi
dXQgbWF5IGFsc28gaGF2ZSBvdGhlcg0KPj4gdGhlbiB0aGUgY29uZmlndXJhdGlvbiBtZW1vcnkg
cmFuZ2VzIGRlc2NyaWJlZCwgZm9yIGV4YW1wbGUgWzNdOg0KPj4gLSByZWc6IFNob3VsZCBjb250
YWluIHJjX2RiaSwgY29uZmlnIHJlZ2lzdGVycyBsb2NhdGlvbiBhbmQgbGVuZ3RoLg0KPj4gLSBy
ZWctbmFtZXM6IE11c3QgaW5jbHVkZSB0aGUgZm9sbG93aW5nIGVudHJpZXM6DQo+PiAgICAgInJj
X2RiaSI6IGNvbnRyb2xsZXIgY29uZmlndXJhdGlvbiByZWdpc3RlcnM7DQo+PiAgICAgImNvbmZp
ZyI6IFBDSWUgY29uZmlndXJhdGlvbiBzcGFjZSByZWdpc3RlcnMuDQo+Pg0KPj4gVGhpcyBwYXRj
aCBtYWtlcyBpdCBwb3NzaWJsZSB0byBub3QgbWFwIGFsbCB0aGUgcmFuZ2VzIGZyb20gdGhlDQo+
PiAicmFuZ2VzIiBwcm9wZXJ0eSBhbmQgYWxzbyBFQ0FNIGZyb20gdGhlICJyZWciLiBBbGwgdGhl
IHJlc3QgZnJvbSB0aGUNCj4+ICJyZWciIHByb3BlcnR5IHN0aWxsIG5lZWRzIHRvIGJlIG1hcHBl
ZCB0byBEb21haW4tMCwgc28gdGhlIFBDSQ0KPj4gaG9zdCBicmlkZ2UgcmVtYWlucyBmdW5jdGlv
bmFsIGluIERvbWFpbi0wLg0KPj4NCj4+IFsxXSBodHRwczovL3VybGRlZmVuc2UuY29tL3YzL19f
aHR0cHM6Ly9saXN0cy54ZW5wcm9qZWN0Lm9yZy9hcmNoaXZlcy9odG1sL3hlbi1kZXZlbC8yMDIw
LTA3L21zZzAwNzc3Lmh0bWxfXzshIUdGXzI5ZGJjUUlVQlBBIWxyQ3VOUnpVVmtSZjRGTGdwM2hX
LTR1T2xkZ0tyNHFOcFpiX3VmSTBqVy1PMGVSSDExVkZEZUdjczVwUGR0S1pUMV9vdTRJblNnJCBb
bGlzdHNbLl14ZW5wcm9qZWN0Wy5db3JnXQ0KPj4gWzJdIGh0dHBzOi8vdXJsZGVmZW5zZS5jb20v
djMvX19odHRwczovL3d3dy5rZXJuZWwub3JnL2RvYy9Eb2N1bWVudGF0aW9uL2RldmljZXRyZWUv
YmluZGluZ3MvcGNpL2hvc3QtZ2VuZXJpYy1wY2kudHh0X187ISFHRl8yOWRiY1FJVUJQQSFsckN1
TlJ6VVZrUmY0RkxncDNoVy00dU9sZGdLcjRxTnBaYl91ZkkwalctTzBlUkgxMVZGRGVHY3M1cFBk
dEtaVDFfVDV5bjdHQSQgW2tlcm5lbFsuXW9yZ10NCj4+IFszXSBodHRwczovL3VybGRlZmVuc2Uu
Y29tL3YzL19faHR0cHM6Ly93d3cua2VybmVsLm9yZy9kb2MvRG9jdW1lbnRhdGlvbi9kZXZpY2V0
cmVlL2JpbmRpbmdzL3BjaS9oaXNpbGljb24tcGNpZS50eHRfXzshIUdGXzI5ZGJjUUlVQlBBIWxy
Q3VOUnpVVmtSZjRGTGdwM2hXLTR1T2xkZ0tyNHFOcFpiX3VmSTBqVy1PMGVSSDExVkZEZUdjczVw
UGR0S1pUMThpbV9ZMnR3JCBba2VybmVsWy5db3JnXQ0KPj4NCj4+IFNpZ25lZC1vZmYtYnk6IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiAtLS0NCj4+IFNpbmNlIHYxOg0KPj4gICAtIEFkZGVkIGJldHRlciBkZXNjcmlwdGlv
biBvZiB3aHkgYW5kIHdoYXQgbmVlZHMgdG8gYmUgbWFwcGVkIGludG8NCj4+ICAgICBEb21haW4t
MCdzIHAybSBhbmQgd2hhdCBkb2Vzbid0DQo+PiAgIC0gRG8gbm90IGRvIGFueSBtYXBwaW5ncyBm
b3IgUENJIGRldmljZXMgd2hpbGUgdHJhdmVyc2luZyB0aGUgRFQNCj4+ICAgLSBXYWxrIGFsbCB0
aGUgYnJpZGdlcyBhbmQgbWFrZSByZXF1aXJlZCBtYXBwaW5ncyBpbiBvbmUgZ28NCj4+IC0tLQ0K
Pj4gICB4ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMgICAgICAgIHwgMzggKysrKysrKysrKysr
KysrLS0tLS0tLS0NCj4+ICAgeGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMgICAgICAgICAgICB8IDE0
ICsrKysrKysrKw0KPj4gICB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIHwgNDgg
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQo+PiAgIHhlbi9hcmNoL2FybS9wY2kvcGNp
LWhvc3QtenlucW1wLmMgfCAgMSArDQo+PiAgIHhlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmggICAg
ICAgICAgfCAgOSArKysrKysNCj4+ICAgeGVuL2luY2x1ZGUvYXNtLWFybS9zZXR1cC5oICAgICAg
ICB8IDEzICsrKysrKysrDQo+PiAgIDYgZmlsZXMgY2hhbmdlZCwgMTExIGluc2VydGlvbnMoKyks
IDEyIGRlbGV0aW9ucygtKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vZG9tYWlu
X2J1aWxkLmMgYi94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMNCj4+IGluZGV4IDgzYWIwZDUy
Y2NlOS4uZTcyYzFiODgxY2FlIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gvYXJtL2RvbWFpbl9i
dWlsZC5jDQo+PiArKysgYi94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMNCj4+IEBAIC0xMCw3
ICsxMCw2IEBADQo+PiAgICNpbmNsdWRlIDxhc20vcmVncy5oPg0KPj4gICAjaW5jbHVkZSA8eGVu
L2Vycm5vLmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4vZXJyLmg+DQo+PiAtI2luY2x1ZGUgPHhlbi9k
ZXZpY2VfdHJlZS5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL2xpYmZkdC9saWJmZHQuaD4NCj4+ICAg
I2luY2x1ZGUgPHhlbi9ndWVzdF9hY2Nlc3MuaD4NCj4+ICAgI2luY2x1ZGUgPHhlbi9pb2NhcC5o
Pg0KPj4gQEAgLTQ3LDEyICs0Niw2IEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2RvbTBfbWVt
KGNvbnN0IGNoYXIgKnMpDQo+PiAgIH0NCj4+ICAgY3VzdG9tX3BhcmFtKCJkb20wX21lbSIsIHBh
cnNlX2RvbTBfbWVtKTsNCj4+ICAgDQo+PiAtc3RydWN0IG1hcF9yYW5nZV9kYXRhDQo+PiAtew0K
Pj4gLSAgICBzdHJ1Y3QgZG9tYWluICpkOw0KPj4gLSAgICBwMm1fdHlwZV90IHAybXQ7DQo+PiAt
fTsNCj4+IC0NCj4+ICAgLyogT3ZlcnJpZGUgbWFjcm9zIGZyb20gYXNtL3BhZ2UuaCB0byBtYWtl
IHRoZW0gd29yayB3aXRoIG1mbl90ICovDQo+PiAgICN1bmRlZiB2aXJ0X3RvX21mbg0KPj4gICAj
ZGVmaW5lIHZpcnRfdG9fbWZuKHZhKSBfbWZuKF9fdmlydF90b19tZm4odmEpKQ0KPj4gQEAgLTEz
ODgsOSArMTM4MSw4IEBAIHN0YXRpYyBpbnQgX19pbml0IG1hcF9kdF9pcnFfdG9fZG9tYWluKGNv
bnN0IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4gICAgICAgcmV0dXJuIDA7DQo+PiAg
IH0NCj4+ICAgDQo+PiAtc3RhdGljIGludCBfX2luaXQgbWFwX3JhbmdlX3RvX2RvbWFpbihjb25z
dCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+IC0gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHU2NCBhZGRyLCB1NjQgbGVuLA0KPj4gLSAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdm9pZCAqZGF0YSkNCj4+ICtpbnQgX19pbml0IG1hcF9yYW5n
ZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+PiArICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIHU2NCBhZGRyLCB1NjQgbGVuLCB2b2lkICpkYXRhKQ0K
Pj4gICB7DQo+PiAgICAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgKm1yX2RhdGEgPSBkYXRhOw0K
Pj4gICAgICAgc3RydWN0IGRvbWFpbiAqZCA9IG1yX2RhdGEtPmQ7DQo+PiBAQCAtMTQxNyw2ICsx
NDA5LDEzIEBAIHN0YXRpYyBpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3Ry
dWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgfQ0KPj4g
ICANCj4+ICsjaWZkZWYgQ09ORklHX0hBU19QQ0kNCj4+ICsgICAgaWYgKCBpc19wY2lfcGFzc3Ro
cm91Z2hfZW5hYmxlZCgpICYmDQo+PiArICAgICAgICAgKGRldmljZV9nZXRfY2xhc3MoZGV2KSA9
PSBERVZJQ0VfUENJKSAmJg0KPj4gKyAgICAgICAgICFtcl9kYXRhLT5tYXBfcGNpX2JyaWRnZSAp
DQo+PiArICAgICAgICBuZWVkX21hcHBpbmcgPSBmYWxzZTsNCj4+ICsjZW5kaWYNCj4gV2l0aCB0
aGUgY2hhbmdlIEkgc3VnZ2VzdGVkIGJlbG93IHR1cm5pbmcgbWFwX3BjaV9icmlkZ2UgaW50bw0K
PiBza2lwX21hcHBpbmcsIHRoZW4gdGhpcyBjaGVjayBjb3VsZCBnbyBhd2F5IGlmIHdlIGp1c3Qg
c2V0IG5lZWRfbWFwcGluZw0KPiBhcyBmb2xsb3dzOg0KPg0KPiBib29sIG5lZWRfbWFwcGluZyA9
ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJvdWdoKGRldikgJiYNCj4gICAgICAgICAgICAgICAgICAg
ICAgIW1yX2RhdGEtPnNraXBfbWFwcGluZzsNCg0KTm90IGV4YWN0bHkuIFRoaXMgY2hlY2ssIGUu
Zy4NCg0KImlzX3BjaV9wYXNzdGhyb3VnaF9lbmFibGVkKCkgJiYgKGRldmljZV9nZXRfY2xhc3Mo
ZGV2KSA9PSBERVZJQ0VfUENJKSINCg0KcmVhbGx5IHByb3RlY3RzIHVzIGZyb20gbWFwcGluZyBh
bnkgb2YgdGhlIHJhbmdlcyBiZWxvbmdpbmcgdG8gYSBQQ0kgZGV2aWNlOg0KDQp3ZSBzY2FuIHRo
ZSBkZXZpY2UgdHJlZSBhbmQgZm9yIGVhY2ggbm9kZSB3ZSBjYWxsIG1hcF9yYW5nZV90b19kb21h
aW4NCg0Kd2l0aCBza2lwX21hcHBpbmcgPT0gZmFsc2UgKGl0IGlzIGNhbGxlZCBmcm9tIG1hcF9k
ZXZpY2VfY2hpbGRyZW4pLg0KDQpTbywgaWYgdGhlcmUgaXMgbm8gY2hlY2sgdGhlbiB0aGUgbWFw
cGluZyBpcyBwZXJmb3JtZWQgZXZlbiBmb3IgUENJIGRldmljZXMNCg0Kd2hpY2ggd2UgZG8gbm90
IHdhbnQuDQoNCkJ1dCwgeWVzIHdlIGNhbiBzaW1wbGlmeSB0aGUgbG9naWMgdG86DQoNCmJvb2wg
bmVlZF9tYXBwaW5nID0gIWR0X2RldmljZV9mb3JfcGFzc3Rocm91Z2goZGV2KSAmJg0KIMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICFtcl9kYXRhLT5za2lwX21hcHBpbmc7
DQoNCiNpZmRlZiBDT05GSUdfSEFTX1BDSQ0KIMKgwqDCoCBpZiAoIG5lZWRfbWFwcGluZyAmJiBp
c19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCgpICYmDQogwqDCoMKgwqDCoMKgwqDCoCAoZGV2aWNl
X2dldF9jbGFzcyhkZXYpID09IERFVklDRV9QQ0kpICkNCiDCoMKgwqDCoMKgwqDCoCBuZWVkX21h
cHBpbmcgPSBmYWxzZTsNCiNlbmRpZg0KDQpidXQgSSBzZWUgbm8gYmlnIHByb2ZpdCBmcm9tIGl0
Lg0KDQo+DQo+DQo+PiAgICAgICBpZiAoIG5lZWRfbWFwcGluZyApDQo+PiAgICAgICB7DQo+PiAg
ICAgICAgICAgcmVzID0gbWFwX3JlZ2lvbnNfcDJtdChkLA0KPj4gQEAgLTE0NTAsNyArMTQ0OSwx
MSBAQCBzdGF0aWMgaW50IF9faW5pdCBtYXBfZGV2aWNlX2NoaWxkcmVuKHN0cnVjdCBkb21haW4g
KmQsDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3Ry
dWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgcDJtX3R5cGVfdCBwMm10KQ0KPj4gICB7DQo+PiAtICAgIHN0cnVjdCBtYXBf
cmFuZ2VfZGF0YSBtcl9kYXRhID0geyAuZCA9IGQsIC5wMm10ID0gcDJtdCB9Ow0KPj4gKyAgICBz
dHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSA9IHsNCj4+ICsgICAgICAgIC5kID0gZCwNCj4+
ICsgICAgICAgIC5wMm10ID0gcDJtdCwNCj4+ICsgICAgICAgIC5tYXBfcGNpX2JyaWRnZSA9IGZh
bHNlDQo+PiArICAgIH07DQo+PiAgICAgICBpbnQgcmV0Ow0KPj4gICANCj4+ICAgICAgIGlmICgg
ZHRfZGV2aWNlX3R5cGVfaXNfZXF1YWwoZGV2LCAicGNpIikgKQ0KPj4gQEAgLTE1ODIsNyArMTU4
NSwxMSBAQCBzdGF0aWMgaW50IF9faW5pdCBoYW5kbGVfZGV2aWNlKHN0cnVjdCBkb21haW4gKmQs
IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4gICAgICAgLyogR2l2ZSBwZXJtaXNzaW9u
IGFuZCBtYXAgTU1JT3MgKi8NCj4+ICAgICAgIGZvciAoIGkgPSAwOyBpIDwgbmFkZHI7IGkrKyAp
DQo+PiAgICAgICB7DQo+PiAtICAgICAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSA9
IHsgLmQgPSBkLCAucDJtdCA9IHAybXQgfTsNCj4+ICsgICAgICAgIHN0cnVjdCBtYXBfcmFuZ2Vf
ZGF0YSBtcl9kYXRhID0gew0KPj4gKyAgICAgICAgICAgIC5kID0gZCwNCj4+ICsgICAgICAgICAg
ICAucDJtdCA9IHAybXQsDQo+PiArICAgICAgICAgICAgLm1hcF9wY2lfYnJpZGdlID0gZmFsc2UN
Cj4+ICsgICAgICAgIH07DQo+PiAgICAgICAgICAgcmVzID0gZHRfZGV2aWNlX2dldF9hZGRyZXNz
KGRldiwgaSwgJmFkZHIsICZzaXplKTsNCj4+ICAgICAgICAgICBpZiAoIHJlcyApDQo+PiAgICAg
ICAgICAgew0KPj4gQEAgLTI3NTQsNyArMjc2MSwxNCBAQCBzdGF0aWMgaW50IF9faW5pdCBjb25z
dHJ1Y3RfZG9tMChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4gICAgICAgICAgIHJldHVybiByYzsNCj4+
ICAgDQo+PiAgICAgICBpZiAoIGFjcGlfZGlzYWJsZWQgKQ0KPj4gKyAgICB7DQo+PiAgICAgICAg
ICAgcmMgPSBwcmVwYXJlX2R0Yl9od2RvbShkLCAma2luZm8pOw0KPj4gKyNpZmRlZiBDT05GSUdf
SEFTX1BDSQ0KPj4gKyAgICAgICAgaWYgKCByYyA8IDAgKQ0KPj4gKyAgICAgICAgICAgIHJldHVy
biByYzsNCj4gVGhpcyBkb2Vzbid0IGxvb2sgZ3JlYXQgOi0pDQo+DQo+IEkgd291bGQgbW92ZSB0
aGUgY2FsbCB0byBwY2lfaG9zdF9icmlkZ2VfbWFwcGluZ3MoKSBiZWxvdyBqdXN0IGJlZm9yZQ0K
PiBjb25zdHJ1Y3RfZG9tYWluLg0KDQpJIHB1dCBpdCB0aGVyZSBmb3IgcHVycG9zZTogY3VycmVu
dGx5IHdlIG9ubHkgc3VwcG9ydCBkZXZpY2UtdHJlZSBhbmQNCg0KQUNQSSBpcyBub3QgY292ZXJl
ZCwgZS5nLiBwY2lfaG9zdF9icmlkZ2VfbWFwcGluZ3MgaXMgaW1wbGVtZW50ZWQNCg0Kd2l0aCBk
ZXZpY2UtdHJlZSBpbiBtaW5kLiBTbywgSSBkZWNpZGVkIHRvIHRpZSBpdCB0byBwcmVwYXJlX2R0
Yl9od2RvbQ0KDQp3aGljaCBpcyBjYWxsZWQgd2hlbiBhY3BpX2Rpc2FibGVkIGlzIHRydWUuDQoN
Cj4NCj4NCj4+ICsgICAgICAgIHJjID0gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKGQsIHAybV9t
bWlvX2RpcmVjdF9jKTsNCj4+ICsjZW5kaWYNCj4+ICsgICAgfQ0KPj4gICAgICAgZWxzZQ0KPj4g
ICAgICAgICAgIHJjID0gcHJlcGFyZV9hY3BpKGQsICZraW5mbyk7DQo+PiAgIA0KPj4gZGlmZiAt
LWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jIGIveGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMN
Cj4+IGluZGV4IDliODhiMWNlZGFhMi4uZWFlMTc3ZjJjYmMyIDEwMDY0NA0KPj4gLS0tIGEveGVu
L2FyY2gvYXJtL3BjaS9lY2FtLmMNCj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQo+
PiBAQCAtMzksNiArMzksMTkgQEAgdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVj
dCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+ICAgICAgIHJldHVybiBiYXNlICsgKFBDSV9E
RVZGTjIoc2JkZikgPDwgZGV2Zm5fc2hpZnQpICsgd2hlcmU7DQo+PiAgIH0NCj4+ICAgDQo+PiAr
Ym9vbCBwY2lfZWNhbV9uZWVkX3AybV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsDQo+PiArICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRn
ZSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDY0X3QgYWRkcikNCj4+
ICt7DQo+PiArICAgIHN0cnVjdCBwY2lfY29uZmlnX3dpbmRvdyAqY2ZnID0gYnJpZGdlLT5jZmc7
DQo+PiArDQo+PiArICAgIC8qDQo+PiArICAgICAqIFdlIGRvIG5vdCB3YW50IEVDQU0gYWRkcmVz
cyBzcGFjZSB0byBiZSBtYXBwZWQgaW4gRG9tYWluLTAncyBwMm0sDQo+PiArICAgICAqIHNvIHdl
IGNhbiB0cmFwIGFjY2VzcyB0byBpdC4NCj4+ICsgICAgICovDQo+PiArICAgIHJldHVybiBjZmct
PnBoeXNfYWRkciAhPSBhZGRyOw0KPj4gK30NCj4+ICsNCj4+ICAgLyogRUNBTSBvcHMgKi8NCj4+
ICAgY29uc3Qgc3RydWN0IHBjaV9lY2FtX29wcyBwY2lfZ2VuZXJpY19lY2FtX29wcyA9IHsNCj4+
ICAgICAgIC5idXNfc2hpZnQgID0gMjAsDQo+PiBAQCAtNDYsNiArNTksNyBAQCBjb25zdCBzdHJ1
Y3QgcGNpX2VjYW1fb3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0KPj4gICAgICAgICAgIC5t
YXBfYnVzICAgICAgICAgICAgICAgID0gcGNpX2VjYW1fbWFwX2J1cywNCj4+ICAgICAgICAgICAu
cmVhZCAgICAgICAgICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZpZ19yZWFkLA0KPj4gICAg
ICAgICAgIC53cml0ZSAgICAgICAgICAgICAgICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3dyaXRl
LA0KPj4gKyAgICAgICAgLm5lZWRfcDJtX21hcHBpbmcgICAgICAgPSBwY2lfZWNhbV9uZWVkX3Ay
bV9tYXBwaW5nLA0KPj4gICAgICAgfQ0KPj4gICB9Ow0KPj4gICANCj4+IGRpZmYgLS1naXQgYS94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2kt
aG9zdC1jb21tb24uYw0KPj4gaW5kZXggMTU1ZjJhMjc0M2FmLi5mMzUwODI2ZWEyNmIgMTAwNjQ0
DQo+PiAtLS0gYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+PiArKysgYi94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+PiBAQCAtMTgsNiArMTgsNyBAQA0K
Pj4gICANCj4+ICAgI2luY2x1ZGUgPHhlbi9pbml0Lmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4vcGNp
Lmg+DQo+PiArI2luY2x1ZGUgPGFzbS9zZXR1cC5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL3J3bG9j
ay5oPg0KPj4gICAjaW5jbHVkZSA8eGVuL3NjaGVkLmg+DQo+PiAgICNpbmNsdWRlIDx4ZW4vdm1h
cC5oPg0KPj4gQEAgLTMyOCw2ICszMjksNTMgQEAgaW50IHBjaV9ob3N0X2dldF9udW1fYnJpZGdl
cyh2b2lkKQ0KPj4gICAgICAgcmV0dXJuIGNvdW50Ow0KPj4gICB9DQo+PiAgIA0KPj4gK2ludCBf
X2luaXQgcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKHN0cnVjdCBkb21haW4gKmQsIHAybV90eXBl
X3QgcDJtdCkNCj4+ICt7DQo+PiArICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZTsN
Cj4+ICsgICAgc3RydWN0IG1hcF9yYW5nZV9kYXRhIG1yX2RhdGEgPSB7DQo+PiArICAgICAgICAu
ZCA9IGQsDQo+PiArICAgICAgICAucDJtdCA9IHAybXQsDQo+PiArICAgICAgICAubWFwX3BjaV9i
cmlkZ2UgPSB0cnVlDQo+PiArICAgIH07DQo+PiArDQo+PiArICAgIC8qDQo+PiArICAgICAqIEZv
ciBlYWNoIFBDSSBob3N0IGJyaWRnZSB3ZSBuZWVkIHRvIG9ubHkgbWFwIHRob3NlIHJhbmdlcw0K
Pj4gKyAgICAgKiB3aGljaCBhcmUgdXNlZCBieSBEb21haW4tMCB0byBwcm9wZXJseSBpbml0aWFs
aXplIHRoZSBicmlkZ2UsDQo+PiArICAgICAqIGUuZy4gd2UgZG8gbm90IHdhbnQgdG8gbWFwIEVD
QU0gY29uZmlndXJhdGlvbiBzcGFjZSB3aGljaCBsaXZlcyBpbg0KPj4gKyAgICAgKiAicmVnIiBv
ciAiYXNzaWduZWQtYWRkcmVzc2VzIiBkZXZpY2UgdHJlZSBwcm9wZXJ0eS4NCj4+ICsgICAgICog
TmVpdGhlciB3ZSB3YW50IHRvIG1hcCBhbnkgb2YgdGhlIE1NSU8gcmFuZ2VzIGZvdW5kIGluIHRo
ZSAicmFuZ2VzIg0KPj4gKyAgICAgKiBkZXZpY2UgdHJlZSBwcm9wZXJ0eS4NCj4+ICsgICAgICov
DQo+PiArICAgIGxpc3RfZm9yX2VhY2hfZW50cnkoIGJyaWRnZSwgJnBjaV9ob3N0X2JyaWRnZXMs
IG5vZGUgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25v
ZGUgKmRldiA9IGJyaWRnZS0+ZHRfbm9kZTsNCj4+ICsgICAgICAgIGludCBpOw0KPiBpIHNob3Vs
ZCBiZSB1bnNpZ25lZCBpbnQNCk9rDQo+DQo+DQo+PiArICAgICAgICBmb3IgKCBpID0gMDsgaSA8
IGR0X251bWJlcl9vZl9hZGRyZXNzKGRldik7IGkrKyApDQo+PiArICAgICAgICB7DQo+PiArICAg
ICAgICAgICAgdWludDY0X3QgYWRkciwgc2l6ZTsNCj4+ICsgICAgICAgICAgICBpbnQgZXJyOw0K
Pj4gKw0KPj4gKyAgICAgICAgICAgIGVyciA9IGR0X2RldmljZV9nZXRfYWRkcmVzcyhkZXYsIGks
ICZhZGRyLCAmc2l6ZSk7DQo+PiArICAgICAgICAgICAgaWYgKCBlcnIgKQ0KPj4gKyAgICAgICAg
ICAgIHsNCj4+ICsgICAgICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIlVuYWJsZSB0byBy
ZXRyaWV2ZSBhZGRyZXNzICV1IGZvciAlc1xuIiwNCj4gTWF5YmUgcmVwaHJhc2UgaXQgdG86DQo+
DQo+ICAgICAgVW5hYmxlIHRvIHJldHJpZXZlIGFkZHJlc3MgcmFuZ2UgaW5kZXg9JXUgZm9yICVz
DQpUaGlzIGlzIGEgY29weS1wYXN0ZSBmcm9tIHRoZSBvcmlnaW5hbCBjb2RlLCBidXQgb2sNCj4N
Cj4NCj4+ICsgICAgICAgICAgICAgICAgICAgICAgIGksIGR0X25vZGVfZnVsbF9uYW1lKGRldikp
Ow0KPj4gKyAgICAgICAgICAgICAgICByZXR1cm4gZXJyOw0KPj4gKyAgICAgICAgICAgIH0NCj4+
ICsNCj4+ICsgICAgICAgICAgICBpZiAoIGJyaWRnZS0+b3BzLT5uZWVkX3AybV9tYXBwaW5nKGQs
IGJyaWRnZSwgYWRkcikgKQ0KPiBUaGUgY3VycmVudCBpbXBsZW1lbnRhdGlvbiBvZiBuZWVkX3Ay
bV9tYXBwaW5nIGZpbHRlcnMgb3V0IEVDQU0gYW5kDQo+IG5vdGhpbmcgZWxzZS4gSnVzdCBkb3Vi
bGUtY2hlY2tpbmcgaGVyZTogaXMgdGhlcmUgYW55dGhpbmcgZWxzZSB3ZQ0KPiBzaG91bGQgZmls
dGVyIG91dD8gTG9va2luZyBhdCB0aGUgZGV2aWNlIHRyZWUgcGNpIG5vZGUgZm9yIFp5bnFNUDoN
Cj4NCj4gCQkJcmVnID0gPDB4MCAweGZkMGUwMDAwIDB4MCAweDEwMDAgMHgwIDB4ZmQ0ODAwMDAg
MHgwIDB4MTAwMCAweDgwIDB4MCAweDAgMHgxMDAwMDAwPjsNCj4gCQkJcmVnLW5hbWVzID0gImJy
ZWciLCAicGNpcmVnIiwgImNmZyI7DQo+DQo+IFdlIGFyZSBmaWx0ZXJpbmcgb3V0IGNmZywgYnV0
IGRvIHdlIG5lZWQgYm90aCAiYnJlZyIgYW5kICJwY2lyZWciIGhlcmU/DQoNCkl0IGlzIHZpY2Ug
dmVyc2E6IHdlIGFyZSBmaWx0ZXJpbmcgb3V0IGNmZyBvbmx5IGFuZCBhbGwgdGhlIHJlc3QgYXJl
ICJ1bmtub3duIHJlZ2lvbnMgd2UgZG8gbm90DQoNCndhbnQgdG8gYWx0ZXIiLg0KDQo+DQo+IElm
IG5vdCwgZG8gd2UgbmVlZCBhbm90aGVyIGZ1bmN0aW9uIGxpa2UgLmNmZ19yZWdfaW5kZXggdG8g
a25vdyB3aGF0IHdlDQo+IGFjdHVhbGx5IG5lZWQgdG8gbWFwPw0KDQpJIHdhcyB0aGlua2luZyB0
byB1c2UgLmNmZ19yZWdfaW5kZXggZmlyIHRoYXQsIGJ1dCBpdCBtZWFucyBJJ2xsIG5lZWQgdG8g
dHJhdmVyc2UNCg0KdGhlIGRldmljZS10cmVlIHRvIGdldCB0aGUgdmFsdWUgZm9yIC5jZmdfcmVn
X2luZGV4IHdoaWNoIGlzIGFscmVhZHkga25vd24NCg0KdG8gdGhlIGJyaWRnZS4gU28sIGl0IGlz
IGNoZWFwZXIgdG8gaGF2ZSBhIGNhbGxiYWNrIGFuZCBqdXN0IGNoZWNrIHRoYXQNCg0KY2ZnLT5w
aHlzX2FkZHIgIT0gYWRkciwgZS5nLiB3aGF0IHdlIHdhbnQgdG8gbWFwIGlzIG5vdCBjZmcuDQoN
Cj4NCj4NCj4+ICsgICAgICAgICAgICB7DQo+PiArICAgICAgICAgICAgICAgIGVyciA9IG1hcF9y
YW5nZV90b19kb21haW4oZGV2LCBhZGRyLCBzaXplLCAmbXJfZGF0YSk7DQo+PiArICAgICAgICAg
ICAgICAgIGlmICggZXJyICkNCj4+ICsgICAgICAgICAgICAgICAgICAgIHJldHVybiBlcnI7DQo+
PiArICAgICAgICAgICAgfQ0KPj4gKyAgICAgICAgfQ0KPj4gKyAgICB9DQo+PiArDQo+PiArICAg
IHJldHVybiAwOw0KPj4gK30NCj4+ICsNCj4+ICAgLyoNCj4+ICAgICogTG9jYWwgdmFyaWFibGVz
Og0KPj4gICAgKiBtb2RlOiBDDQo+PiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BjaS9wY2kt
aG9zdC16eW5xbXAuYyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtenlucW1wLmMNCj4+IGlu
ZGV4IGMyN2I0ZWE5ZjAyZi4uYWRiZTM2Mjc4NzFmIDEwMDY0NA0KPj4gLS0tIGEveGVuL2FyY2gv
YXJtL3BjaS9wY2ktaG9zdC16eW5xbXAuYw0KPj4gKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2kt
aG9zdC16eW5xbXAuYw0KPj4gQEAgLTMzLDYgKzMzLDcgQEAgY29uc3Qgc3RydWN0IHBjaV9lY2Ft
X29wcyBud2xfcGNpZV9vcHMgPSB7DQo+PiAgICAgICAgICAgLm1hcF9idXMgICAgICAgICAgICAg
ICAgPSBwY2lfZWNhbV9tYXBfYnVzLA0KPj4gICAgICAgICAgIC5yZWFkICAgICAgICAgICAgICAg
ICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3JlYWQsDQo+PiAgICAgICAgICAgLndyaXRlICAgICAg
ICAgICAgICAgICAgPSBwY2lfZ2VuZXJpY19jb25maWdfd3JpdGUsDQo+PiArICAgICAgICAubmVl
ZF9wMm1fbWFwcGluZyAgICAgICA9IHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmcsDQo+PiAgICAg
ICB9DQo+PiAgIH07DQo+PiAgIA0KPj4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL2FzbS1hcm0v
cGNpLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL3BjaS5oDQo+PiBpbmRleCA3NjE4ZjBiNjcyNWIu
LmI4MWY2NmU4MTNlZiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgN
Cj4+ICsrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+IEBAIC0xOSw2ICsxOSw4IEBA
DQo+PiAgIA0KPj4gICAjaWZkZWYgQ09ORklHX0hBU19QQ0kNCj4+ICAgDQo+PiArI2luY2x1ZGUg
PGFzbS9wMm0uaD4NCj4+ICsNCj4+ICAgI2RlZmluZSBwY2lfdG9fZGV2KHBjaWRldikgKCYocGNp
ZGV2KS0+YXJjaC5kZXYpDQo+PiAgICNkZWZpbmUgUFJJX3BjaSAiJTA0eDolMDJ4OiUwMnguJXUi
DQo+PiAgIA0KPj4gQEAgLTc5LDYgKzgxLDkgQEAgc3RydWN0IHBjaV9vcHMgew0KPj4gICAgICAg
ICAgICAgICAgICAgdWludDMyX3QgcmVnLCB1aW50MzJfdCBsZW4sIHVpbnQzMl90ICp2YWx1ZSk7
DQo+PiAgICAgICBpbnQgKCp3cml0ZSkoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLCB1
aW50MzJfdCBzYmRmLA0KPj4gICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMy
X3QgbGVuLCB1aW50MzJfdCB2YWx1ZSk7DQo+PiArICAgIGJvb2wgKCpuZWVkX3AybV9tYXBwaW5n
KShzdHJ1Y3QgZG9tYWluICpkLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3Ry
dWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdWludDY0X3QgYWRkcik7DQo+IEkgd291bGQgY2FsbCB0aGlzIGZ1bmN0aW9uOiBuZWVk
X3AybV9od2RvbV9tYXBwaW5nDQpPaw0KPg0KPg0KPj4gICB9Ow0KPj4gICANCj4+ICAgLyoNCj4+
IEBAIC0xMDIsNiArMTA3LDkgQEAgaW50IHBjaV9nZW5lcmljX2NvbmZpZ193cml0ZShzdHJ1Y3Qg
cGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsIHVpbnQzMl90IHNiZGYsDQo+PiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1aW50MzJfdCByZWcsIHVpbnQzMl90IGxlbiwgdWludDMyX3QgdmFs
dWUpOw0KPj4gICB2b2lkIF9faW9tZW0gKnBjaV9lY2FtX21hcF9idXMoc3RydWN0IHBjaV9ob3N0
X2JyaWRnZSAqYnJpZGdlLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWlu
dDMyX3Qgc2JkZiwgdWludDMyX3Qgd2hlcmUpOw0KPj4gK2Jvb2wgcGNpX2VjYW1fbmVlZF9wMm1f
bWFwcGluZyhzdHJ1Y3QgZG9tYWluICpkLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+PiArICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVpbnQ2NF90IGFkZHIpOw0KPj4gICBzdHJ1Y3QgcGNpX2hvc3RfYnJp
ZGdlICpwY2lfZmluZF9ob3N0X2JyaWRnZSh1aW50MTZfdCBzZWdtZW50LCB1aW50OF90IGJ1cyk7
DQo+PiAgIGludCBwY2lfZ2V0X2hvc3RfYnJpZGdlX3NlZ21lbnQoY29uc3Qgc3RydWN0IGR0X2Rl
dmljZV9ub2RlICpub2RlLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVp
bnQxNl90ICpzZWdtZW50KTsNCj4+IEBAIC0xMTYsNiArMTI0LDcgQEAgaW50IHBjaV9ob3N0X2l0
ZXJhdGVfYnJpZGdlcyhzdHJ1Y3QgZG9tYWluICpkLA0KPj4gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGludCAoKmNsYikoc3RydWN0IGRvbWFpbiAqZCwNCj4+ICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRn
ZSkpOw0KPj4gICBpbnQgcGNpX2hvc3RfZ2V0X251bV9icmlkZ2VzKHZvaWQpOw0KPj4gK2ludCBw
Y2lfaG9zdF9icmlkZ2VfbWFwcGluZ3Moc3RydWN0IGRvbWFpbiAqZCwgcDJtX3R5cGVfdCBwMm10
KTsNCj4+ICAgI2Vsc2UgICAvKiFDT05GSUdfSEFTX1BDSSovDQo+PiAgIA0KPj4gICAjZGVmaW5l
IHBjaV9wYXNzdGhyb3VnaF9lbmFibGVkIChmYWxzZSkNCj4+IGRpZmYgLS1naXQgYS94ZW4vaW5j
bHVkZS9hc20tYXJtL3NldHVwLmggYi94ZW4vaW5jbHVkZS9hc20tYXJtL3NldHVwLmgNCj4+IGlu
ZGV4IDk1ZGEwYjdhYjljZC4uMjE4NjNkZDJiYzU4IDEwMDY0NA0KPj4gLS0tIGEveGVuL2luY2x1
ZGUvYXNtLWFybS9zZXR1cC5oDQo+PiArKysgYi94ZW4vaW5jbHVkZS9hc20tYXJtL3NldHVwLmgN
Cj4+IEBAIC0yLDYgKzIsOCBAQA0KPj4gICAjZGVmaW5lIF9fQVJNX1NFVFVQX0hfDQo+PiAgIA0K
Pj4gICAjaW5jbHVkZSA8cHVibGljL3ZlcnNpb24uaD4NCj4+ICsjaW5jbHVkZSA8YXNtL3AybS5o
Pg0KPj4gKyNpbmNsdWRlIDx4ZW4vZGV2aWNlX3RyZWUuaD4NCj4+ICAgDQo+PiAgICNkZWZpbmUg
TUlOX0ZEVF9BTElHTiA4DQo+PiAgICNkZWZpbmUgTUFYX0ZEVF9TSVpFIFNaXzJNDQo+PiBAQCAt
NzcsNiArNzksMTQgQEAgc3RydWN0IGJvb3RpbmZvIHsNCj4+ICAgI2VuZGlmDQo+PiAgIH07DQo+
PiAgIA0KPj4gK3N0cnVjdCBtYXBfcmFuZ2VfZGF0YQ0KPj4gK3sNCj4+ICsgICAgc3RydWN0IGRv
bWFpbiAqZDsNCj4+ICsgICAgcDJtX3R5cGVfdCBwMm10Ow0KPj4gKyAgICAvKiBTZXQgaWYgbWFw
cGluZ3MgZm9yIFBDSSBob3N0IGJyaWRnZXMgbXVzdCBub3QgYmUgc2tpcHBlZC4gKi8NCj4+ICsg
ICAgYm9vbCBtYXBfcGNpX2JyaWRnZTsNCj4gVG8gbWFrZSB0aGlzIG1vcmUgZ2VuZXJhbGx5IGFw
cGxpY2FibGUsIEkgd291bGQgY2FsbCB0aGUgbmV3IHByb3BlcnR5Og0KPg0KPiAgICBib29sIHNr
aXBfbWFwcGluZzsNClNvdW5kcyBnb29kDQo+DQo+IGFuZCBpdCBjb3VsZCBhcHBseSB0byBhbnkg
Y2xhc3Mgb2YgZGV2aWNlcy4gQWxsIGN1cnJlbnQgdXNlcnMgd291bGQgc2V0DQo+IGl0IHRvIGZh
bHNlIGV4Y2VwdCBmb3IgcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzLg0KUGxlYXNlIHNlZSBQQ0kg
c3BlY2lhbCBjYXNlIGFib3ZlDQo+DQo+DQo+PiArfTsNCj4+DQo+PiAgIGV4dGVybiBzdHJ1Y3Qg
Ym9vdGluZm8gYm9vdGluZm87DQo+PiAgIA0KPj4gICBleHRlcm4gZG9taWRfdCBtYXhfaW5pdF9k
b21pZDsNCj4+IEBAIC0xMjQsNiArMTM0LDkgQEAgdm9pZCBkZXZpY2VfdHJlZV9nZXRfcmVnKGNv
bnN0IF9fYmUzMiAqKmNlbGwsIHUzMiBhZGRyZXNzX2NlbGxzLA0KPj4gICB1MzIgZGV2aWNlX3Ry
ZWVfZ2V0X3UzMihjb25zdCB2b2lkICpmZHQsIGludCBub2RlLA0KPj4gICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBjaGFyICpwcm9wX25hbWUsIHUzMiBkZmx0KTsNCj4+ICAgDQo+PiAr
aW50IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYs
DQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgdTY0IGFkZHIsIHU2NCBsZW4sIHZvaWQgKmRh
dGEpOw0KPj4gKw0KPj4gICAjZW5kaWYNCj4+ICAgLyoNCj4+ICAgICogTG9jYWwgdmFyaWFibGVz
Og0KPj4gLS0gDQo+PiAyLjI1LjENCj4+


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 13:02:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 13:02:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196852.349748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqGr-0006uI-Dz; Mon, 27 Sep 2021 13:02:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196852.349748; Mon, 27 Sep 2021 13:02:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqGr-0006uB-Aa; Mon, 27 Sep 2021 13:02:13 +0000
Received: by outflank-mailman (input) for mailman id 196852;
 Mon, 27 Sep 2021 13:02:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUqGp-0006u5-TZ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 13:02:11 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 20105d4a-1f93-11ec-bc44-12813bfff9fa;
 Mon, 27 Sep 2021 13:02:10 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18RCwPQ5031720; 
 Mon, 27 Sep 2021 13:02:08 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbapvh96s-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 13:02:07 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4884.eurprd03.prod.outlook.com (2603:10a6:208:fe::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 27 Sep
 2021 13:02:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 13:02:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20105d4a-1f93-11ec-bc44-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jAnVsz+UNOIb5ef5C2Fwv4Ix6NmaiZgA4ICMDlUIO5iOJeIMJeakkZkGr8ZvzpiX8c67rQ4raIqqitPKDRMOnCQqs8CdTY+9I8xLUiLXBdeKOrI7puOW+IyAgPuLnoEqkdeSkgGCZg+WZQJxALnxGYSgZ4EIBSSBKqzDOejCXr/5pPa3nWISmNtZsZVYkZo2zk4BozKwzRu8gMQ13WTRHXj4jNzn0qn4/N7mFrEZzJuNXb5sJ5W+mZwsx+gmt3EjNa1E3C4wvJcW1o2HpEivkApKGm0qYQA9NT6Sz6cPN2S/72uYYvb81zDKwQ/mlYnWPp7NmO0SDxuYk4PqOxz67A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=BXNqTsz23BP+4qe5sIekQ17PL4tWH7PxtLQ28xGn6Ms=;
 b=PAgziZrtTleVZRW01YoxHC7FFsb7Vha6aoNKad5XCAvGFDQCUOaMvW/bB8/0cm9jFZyHkNfTO0vXLX5L3IW4WBEK9qxSazgYuZkbfcSfqttTg1nIsn5sLdn5b+dkeKl/iIHlZKW+YMKYvGuUzZBjagMKDDHdWbpqENFj0RVw87Yh8adIG67FZJS9BA9L3huA+M+5ARjCSBdXLcLor8QNdonaqkVVL6bd2Nbu9yuNPAfBLofsFGg+y6iHWWNK+32swaxG2xGtWxF1EbkfJI/GOVSNJvSBpBnr8BPOZ04rIqcXUVZRdWajdp+443IKP/ZfKtVfdR3BW9g4WkuLpo028A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BXNqTsz23BP+4qe5sIekQ17PL4tWH7PxtLQ28xGn6Ms=;
 b=e1zzFO0sZcNts8mAWFi7E94xgR7Bo0bfIol2Y+QYkRoWQSAQ01WVD3CwmodJ6NlAyv0DwbfSWrRwNI1IJMqMqbFCd8FDIKsFh5wz3Y+qkDDoHsW9EzObORYwn4rM9ZwWjV8LyqkCZfwCvYiP54ZJDVWDBMeo0P5IhqyU+GNmgPid9kH6Ecv6k3hpGQl3K7XQu9HEQ6ATDa1EjJyIIRr52nJHiEm0jGZH2AGoDRb2oZ2kTYPafLvFVFvjhIvUirLXsqVJRE2s0wVOojo7RDK1oAxEecLQzCbI4WExeI7K9cg+U/e4BCisu/eEQXfz+SjA9JjW5GHTLoiHTI/8XoTKaw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Ian Jackson
	<iwj@xenproject.org>, Juergen Gross <jgross@suse.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>,
        "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v2 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Topic: [PATCH v2 08/11] libxl: Only map legacy PCI IRQs if they are
 supported
Thread-Index: AQHXsHo1c5aXAigXoEuHbQuK2gbS5Kuz4U+AgAP9dgA=
Date: Mon, 27 Sep 2021 13:02:02 +0000
Message-ID: <85922b59-2bd4-0f16-9109-7c1ca24f52d8@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-9-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241704050.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241704050.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b123b222-9092-4bd5-6cb2-08d981b70015
x-ms-traffictypediagnostic: AM0PR03MB4884:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB48841CF088A0CDAADEC8B1F2E7A79@AM0PR03MB4884.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Cs8UoHQJsMfF81nLnNpSsklvmDFUFSR2OT0ANVlzARFGPgNMLCJgm0SGefUZ3kV7OJWa3FeVClriJ9bCtNJ8+eefxKMEFAdEVu4seZGEf2OekFVzhZVOwyUk/WcK6nQTfx52ui1a/odquP2JBz8siSVrS04r+Pw4THM4wZxslnsJ3daj1wuOdYnjqDAaMIpcj649/aY3gu71VGEiFJTt/2Co6SS+pNFYT9x2hElURl5PNwixnxDcMdBWe8066vokhtPSwaoOjWSFD2I55Mtvp7BgVe1iA/4BX7VrF1yktCwg/47edh7Kl9yPFJS3SpWzKHBmrX5M3QJxGpG9vddZw+cGowIdGm7kU3HwxzOlwt6wPi1nqBi6h3jEBYedrHpJWSfw7JMoxzWOQC/+SOIAep5lvyPuzgacb1qecFeUHE1gT+y9Kk1rZsYv1nPvNWm7kEG3ABPfVhKsCHzMOcxYzsI7h2hv3py/ySKV7/Lidz5ArJV9MyO1ph2NFp8nwfJZQtPpDAOj/jDrGhzTOoF2cYbFv6M0Xurkv6Me6QunoYfrVOqdRLTC281umv1vV8JaavrCIoFpPslwYvsT2QmHNt469HPU/1pF0mR2rvXVARiRQ4n17LBjk0uOj1jBHju41MJDkXZHp0cJO0Mh/Zp1IU6zdhX5w7wvHROOPhW1pJd9YLb1YzDk8i4uVHiNCxVYs8MzZUE01stLb7KJlYoIHnEGnhVaI1HwFoUPslFlaQfGeXB1Th9qWWmIN/WRhd0j
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(64756008)(54906003)(66556008)(38100700002)(2616005)(316002)(66446008)(508600001)(4326008)(66476007)(6486002)(6512007)(186003)(71200400001)(31686004)(76116006)(66946007)(2906002)(8676002)(122000001)(7416002)(83380400001)(86362001)(31696002)(36756003)(91956017)(53546011)(8936002)(55236004)(110136005)(26005)(5660300002)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ZW1ZSDJ3K2ZuY3ZhMi85SXYvdzZBanVVY3pMNkVsa1FpcjB3akxnNUdORHk4?=
 =?utf-8?B?eVRQK05oc2ZnMmhTcmlEL1FWV1AwZk4rSTAraHVQWDNuRzlta1dndjhsczhD?=
 =?utf-8?B?WWE0MllKOEsrOHJIeFFkekpBSUhubUpwL1Fucjc4Q1dTRTBYMi9EQnIzMnRa?=
 =?utf-8?B?ZEtBc2FnYXhmbWF6a1BQOTNUQ2lsV1NVQnRtampZeFA5OHlxcVlFOE4xcW5R?=
 =?utf-8?B?R1J0b0F2dzJNUS9Bbm1HWng2TWtjazZ3OFVUbjlnNFZRQm1CbDROTjN4VTBW?=
 =?utf-8?B?T0htMXg2Q0hBMEJtRmZJRmN4UzI4bTcyZWxaQjduWUhqMmZHcit0Mlk3UUJp?=
 =?utf-8?B?K1YvQXlqQnl2UWR1cnQzTXg5R1k5UGJsOGlCMGpkVENOcFpKTFVwU3YrZ0lR?=
 =?utf-8?B?bGE1YWcyQkFaQitXQjQ2ejFaSDNOR2ZvRHJEdUd6bDNSU3lhaDVUY3FxRFlj?=
 =?utf-8?B?N2lsM0FPZWdnd1BFZTl1NnpqdG1rSU9ORHBsUmx0R01ETzNZc1RxVzlXUmhj?=
 =?utf-8?B?a1BWQ0x1U0cwZ29JNHgzZ1hYRkhGMHRNVjFSTGFleUFvMWVMV3FFdGE4Mmd1?=
 =?utf-8?B?Z0NuWjl1YXhoSGZrNC9MSVB0anFVemhUUnhiL2ROSFNyOFN6MW1vNC9ua1Ns?=
 =?utf-8?B?RVN1VGhOVHdudXZIK29WQ1k1YUJGRTVZT2huRWtRbHVqanJ0UVlwMjUvWlNH?=
 =?utf-8?B?d2FjekI3RU5RQUN4UVE4MTc4MkZTUnJ5T2dFS0VXNTFoMzd5ZHE1WUt1ZEFP?=
 =?utf-8?B?emxpVVREZVhCWmRvb2pxRzBqQ1orMFdoQWxIUUsrM0U0S1hxb2NyNFVjeVdz?=
 =?utf-8?B?akZjQXZIdlZtU0pVQk5UM2d3WXZXV25QdmdDS2k3TzdTKzlIb1pDcitVT0JI?=
 =?utf-8?B?OVR4RkNiQytSOUJVNFZUMnFJYTdSZ0tCejZITUFDdVJ6NnVjZE5nM3ZhaHl1?=
 =?utf-8?B?L0NDVGtxZStCY2dFRVF6NUhkTEROVFJRTVpHb292K0VDbkN2YkNWbzV1cjFF?=
 =?utf-8?B?VEFVaXpEQnY2ZVNEYksrbG90Sm16c3RRUmhnbTV6eUxLNEp3U2pzbWVzNXJz?=
 =?utf-8?B?cm1wdExQeHZHeWhQOVlpcGk1RXJ6dEhycTV5TkU4YlF1bmpVZmprZGQzc3hs?=
 =?utf-8?B?aHNpbHpPallFOThNTlE2bnZEMjJXVURnd1A0UWNvZTNDOGdIbjMxRFZaUEVF?=
 =?utf-8?B?WXVQZlErT2pIK3BjUkNFUEhidWZ4V3krMjVOMjdmK0R0NUg0K2xMOTNjc29x?=
 =?utf-8?B?R244VWRTenRTSWpEeENsL2FkL0dUYlI4SkI2K0lDbmhuK3E4M1pBNXNRN3g0?=
 =?utf-8?B?K1k4S3VpclYvU1l1TlA0NGp3WDhqNHRuMGlyaDhDQ2YrdW9XVlh6eThCcS9u?=
 =?utf-8?B?eHNDUUh2TzNXVnNNc2R6eHVicXNSbVBDekd1Ym42aDU4WE1wTkhiVnhFeWE1?=
 =?utf-8?B?NXN0SnpxdlRxK3hkMzZQdWhjK3huTE1IU2hTdWRLQUJNMFJDaWRpVCs4bktF?=
 =?utf-8?B?VUc5bktPS2hHMlhKSlhrMjNTRmttQktHenhwSmlIK09WRCt6L3hnd0lsRmFF?=
 =?utf-8?B?dGZJRE1HdHZnTFFEYTNod1RYREk0UEwwUTMrMzRHWDgzWHVTZmQwQ21wWlMr?=
 =?utf-8?B?SjhKeDRuR2hmME1OTGszcTBNMTdyS0Jyd1dCVVNJYWovZVh1YWliM2hhSTNT?=
 =?utf-8?B?VHc3UnoweHlpaE5jbXVkWmdVT2FKeC9uVC9XeU00c3RWNDdpRlRwelJYTklw?=
 =?utf-8?B?R1FBSnp2byszS1ZmL054UHo0TGZKWkRXOHIyU1ZNQUZhUDBWbUFGVWRJNy9j?=
 =?utf-8?B?WXdVUjFpVm9WMFVQVzU2Zz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D3BFE77BFFF17B4EB31EFBE3638C2AF9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b123b222-9092-4bd5-6cb2-08d981b70015
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 13:02:02.8972
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: JaNlmLf2xNzey+ggaaCXiKpWSXK/Z/GcpyW8MjL2b9oopij10f0caOaSB4/YUeILVCG4/gx9aESafZ3EWQo/EyLMk6ga4nlXEmBRLv7Jqdpf2XGI3WF3wxRi0u1PXm8Q
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4884
X-Proofpoint-GUID: NABmYIASCv9XxbngtWeBKWKykyfQBanQ
X-Proofpoint-ORIG-GUID: NABmYIASCv9XxbngtWeBKWKykyfQBanQ
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 priorityscore=1501
 adultscore=0 clxscore=1015 lowpriorityscore=0 spamscore=0 impostorscore=0
 bulkscore=0 phishscore=0 malwarescore=0 suspectscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109270090

DQpPbiAyNS4wOS4yMSAwMzowNiwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUaHUs
IDIzIFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiBBcm0ncyBQQ0kgcGFzc3Rocm91Z2ggaW1wbGVtZW50YXRpb24gZG9lc24ndCBzdXBw
b3J0IGxlZ2FjeSBpbnRlcnJ1cHRzLA0KPj4gYnV0IE1TSS9NU0ktWC4gVGhpcyBjYW4gYmUgdGhl
IGNhc2UgZm9yIG90aGVyIHBsYXRmb3JtcyB0b28uDQo+PiBGb3IgdGhhdCByZWFzb24gaW50cm9k
dWNlIGEgbmV3IENPTkZJR19QQ0lfU1VQUF9MRUdBQ1lfSVJRIGFuZCBhZGQNCj4+IGl0IHRvIHRo
ZSBDRkxBR1MgYW5kIGNvbXBpbGUgdGhlIHJlbGV2YW50IGNvZGUgaW4gdGhlIHRvb2xzdGFjayBv
bmx5IGlmDQo+PiBhcHBsaWNhYmxlLg0KPj4NCj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBB
bmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+IENjOiBJ
YW4gSmFja3NvbiA8aXdqQHhlbnByb2plY3Qub3JnPg0KPj4gQ2M6IEp1ZXJnZW4gR3Jvc3MgPGpn
cm9zc0BzdXNlLmNvbT4NCj4+DQo+PiAtLS0NCj4+IFNpbmNlIHYxOg0KPj4gICAtIE1pbmltaXpl
ZCAjaWRlZmVyeSBieSBpbnRyb2R1Y2luZyBwY2lfc3VwcF9sZWdhY3lfaXJxIGZ1bmN0aW9uDQo+
PiAgICAgZm9yIHJlbGV2YW50IGNoZWNrcw0KPj4gLS0tDQo+PiAgIHRvb2xzL2xpYnMvbGlnaHQv
TWFrZWZpbGUgICAgfCAgNCArKysrDQo+PiAgIHRvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMg
fCAxMyArKysrKysrKysrKysrDQo+PiAgIDIgZmlsZXMgY2hhbmdlZCwgMTcgaW5zZXJ0aW9ucygr
KQ0KPj4NCj4+IGRpZmYgLS1naXQgYS90b29scy9saWJzL2xpZ2h0L01ha2VmaWxlIGIvdG9vbHMv
bGlicy9saWdodC9NYWtlZmlsZQ0KPj4gaW5kZXggN2Q4YzUxZDQ5MjQyLi5iZDNmNmJlMmExODMg
MTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L01ha2VmaWxlDQo+PiArKysgYi90b29s
cy9saWJzL2xpZ2h0L01ha2VmaWxlDQo+PiBAQCAtNDYsNiArNDYsMTAgQEAgQ0ZMQUdTICs9IC1X
bm8tZm9ybWF0LXplcm8tbGVuZ3RoIC1XbWlzc2luZy1kZWNsYXJhdGlvbnMgXA0KPj4gICAJLVdu
by1kZWNsYXJhdGlvbi1hZnRlci1zdGF0ZW1lbnQgLVdmb3JtYXQtbm9ubGl0ZXJhbA0KPj4gICBD
RkxBR1MgKz0gLUkuDQo+PiAgIA0KPj4gK2lmZXEgKCQoQ09ORklHX1g4NikseSkNCj4+ICtDRkxB
R1MgKz0gLURDT05GSUdfUENJX1NVUFBfTEVHQUNZX0lSUQ0KPj4gK2VuZGlmDQo+IFRoaXMgcGF0
Y2ggaXMgYSBsb3QgYmV0dGVyIHRoYW4gdGhlIHByZXZpb3VzIHZlcnNpb24sIHRoYW5rcyENCj4N
Cj4gSSB0aGluayB0aGUgdXNhZ2Ugb2YgcGNpX3N1cHBfbGVnYWN5X2lycSBiZWxvdyBpcyBnb29k
IGFuZCB3ZSBjYW4ndCBkbw0KPiBiZXR0ZXIgdGhhbiB0aGF0Lg0KPg0KPiBJIHdvbmRlciBpZiB0
aGVyZSBpcyBhIGJldHRlciB3YXkgdGhhbiB0aGUgYWJvdmUgdG8gZXhwb3J0DQo+IENPTkZJR19Q
Q0lfU1VQUF9MRUdBQ1lfSVJRLiBTdWdnZXN0aW9ucz8NCg0KSSBzZWUgbm90aGluZyBiYWQgZG9p
bmcgaXQgdGhpcyB3YXksIG1heWJlIElhbiBvciBKdWVyZ2VuIGNhbiB0ZWxsDQoNCmlmIHRoaXMg
aXMgYWNjZXB0YWJsZT8NCg0KPg0KPg0KPj4gICBTUkNTLSQoQ09ORklHX1g4NikgKz0gbGlieGxf
Y3B1aWQuYw0KPj4gICBTUkNTLSQoQ09ORklHX1g4NikgKz0gbGlieGxfeDg2LmMNCj4+ICAgU1JD
Uy0kKENPTkZJR19YODYpICs9IGxpYnhsX3Bzci5jDQo+PiBkaWZmIC0tZ2l0IGEvdG9vbHMvbGli
cy9saWdodC9saWJ4bF9wY2kuYyBiL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+IGlu
ZGV4IDU5ZjM2ODZmYzg1ZS4uNGMyZDdhZWVmYmIyIDEwMDY0NA0KPj4gLS0tIGEvdG9vbHMvbGli
cy9saWdodC9saWJ4bF9wY2kuYw0KPj4gKysrIGIvdG9vbHMvbGlicy9saWdodC9saWJ4bF9wY2ku
Yw0KPj4gQEAgLTEzNjQsNiArMTM2NCwxNSBAQCBzdGF0aWMgdm9pZCBwY2lfYWRkX3RpbWVvdXQo
bGlieGxfX2VnYyAqZWdjLCBsaWJ4bF9fZXZfdGltZSAqZXYsDQo+PiAgICAgICBwY2lfYWRkX2Rt
X2RvbmUoZWdjLCBwYXMsIHJjKTsNCj4+ICAgfQ0KPj4gICANCj4+ICtzdGF0aWMgYm9vbCBwY2lf
c3VwcF9sZWdhY3lfaXJxKHZvaWQpDQo+PiArew0KPj4gKyNpZmRlZiBDT05GSUdfUENJX1NVUFBf
TEVHQUNZX0lSUQ0KPj4gKyAgICByZXR1cm4gdHJ1ZTsNCj4+ICsjZWxzZQ0KPj4gKyAgICByZXR1
cm4gZmFsc2U7DQo+PiArI2VuZGlmDQo+PiArfQ0KPj4gKw0KPj4gICBzdGF0aWMgdm9pZCBwY2lf
YWRkX2RtX2RvbmUobGlieGxfX2VnYyAqZWdjLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgcGNpX2FkZF9zdGF0ZSAqcGFzLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgaW50IHJjKQ0KPj4gQEAgLTE0MzQsNiArMTQ0Myw4IEBAIHN0YXRpYyB2b2lkIHBjaV9hZGRf
ZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgfQ0KPj4g
ICAgICAgZmNsb3NlKGYpOw0KPj4gKyAgICBpZiAoIXBjaV9zdXBwX2xlZ2FjeV9pcnEoKSkNCj4+
ICsgICAgICAgIGdvdG8gb3V0X25vX2lycTsNCj4+ICAgICAgIHN5c2ZzX3BhdGggPSBHQ1NQUklO
VEYoU1lTRlNfUENJX0RFViIvIlBDSV9CREYiL2lycSIsIHBjaS0+ZG9tYWluLA0KPj4gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5j
KTsNCj4+ICAgICAgIGYgPSBmb3BlbihzeXNmc19wYXRoLCAiciIpOw0KPj4gQEAgLTE5ODMsNiAr
MTk5NCw4IEBAIHN0YXRpYyB2b2lkIGRvX3BjaV9yZW1vdmUobGlieGxfX2VnYyAqZWdjLCBwY2lf
cmVtb3ZlX3N0YXRlICpwcnMpDQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgICAgIGZjbG9zZShm
KTsNCj4+ICAgc2tpcDE6DQo+PiArICAgICAgICBpZiAoIXBjaV9zdXBwX2xlZ2FjeV9pcnEoKSkN
Cj4+ICsgICAgICAgICAgICBnb3RvIHNraXBfaXJxOw0KPj4gICAgICAgICAgIHN5c2ZzX3BhdGgg
PSBHQ1NQUklOVEYoU1lTRlNfUENJX0RFViIvIlBDSV9CREYiL2lycSIsIHBjaS0+ZG9tYWluLA0K
Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGNpLT5idXMsIHBjaS0+ZGV2LCBw
Y2ktPmZ1bmMpOw0KPj4gICAgICAgICAgIGYgPSBmb3BlbihzeXNmc19wYXRoLCAiciIpOw0KPj4g
LS0gDQo+PiAyLjI1LjENCj4+


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 13:34:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 13:34:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196875.349759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqmI-0002BV-Pj; Mon, 27 Sep 2021 13:34:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196875.349759; Mon, 27 Sep 2021 13:34:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqmI-0002BO-Mm; Mon, 27 Sep 2021 13:34:42 +0000
Received: by outflank-mailman (input) for mailman id 196875;
 Mon, 27 Sep 2021 13:34:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUqmH-0002Az-9j
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 13:34:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a9bcd40c-1f97-11ec-bc45-12813bfff9fa;
 Mon, 27 Sep 2021 13:34:39 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2054.outbound.protection.outlook.com [104.47.6.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-SezIbczOMeygycLb4R9raA-1;
 Mon, 27 Sep 2021 15:34:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2958.eurprd04.prod.outlook.com (2603:10a6:802:a::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep
 2021 13:34:35 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 13:34:35 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0075.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 13:34:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9bcd40c-1f97-11ec-bc45-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632749678;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MBkKeAyOsdBo8BLeLFIlpGAfBFrCok3ULzRD7lKMQ3k=;
	b=a5dVm5TFDYqxZCz8xYRC6dV6KRwXBazmA5qvNvCNOHzs5N4Kgxsq43GzKOiWsuQP0KpMVw
	HxJe8/iVjlRkDBYWZlf1vAtLdiygeCtaiO9QJYpPeQxDslifAKHEBI30gKRpQc1H2VbAUE
	Uqc6hBkVFBMZx+S7qL/Kunov0B/fxFY=
X-MC-Unique: SezIbczOMeygycLb4R9raA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=digkVlnaD6pwyuHBlLR1Gs6X17Li2NKWmaOktEuUOyBBeWLUMID0Lhc1rFRQRCDFlTsvkgzQoZ0SeBAVMC3KrqzP7I5nuIZmQCllLaRoQi1jD2j302N5f+krjVGSYRMgfM2hAtaDq/TFBhC6KFbZF8kuf5qqgmNujHfFgZKI2qk90NWW2/kTYRUfxTyUhKPtvqOYHSqpniY2FdHmewmVFnEf3GyT1rx30b9wYmODvMsBMlXGhQNjVrVU/R8q+8zEz48/1dv1I6iId0/7gCtqJxYRkXG2vQQckQFGfM7J7zYeJhb8Lh7COCh4XdKoUCWEvT+Zwe5+iPfD7quw+H57ug==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=MBkKeAyOsdBo8BLeLFIlpGAfBFrCok3ULzRD7lKMQ3k=;
 b=Jsx0zjggaefKkyAsQBm+dYQPZGp+INSE/D4llL/NIwvz2LHJ3dVhF257YiHB33J4d7EfxubarrPeyrWYyh3KZTVJDw2s34wlaTipcOAR+kK1Y9Qah3n7YRWlx8MrRpSdw37kuCaztfFeSdlnhAcvbwCbY3+oQG5ntTtOa0nACPATKsM4xT5+/Yv2osn7NUlhNWdVPBe3eQQbVqR65gpwHuJtuqilDsdUAB9OzT/yZ9Fjp8rRlPnl4lw2y+n8Ys6WnQjQR+qi6XGQwEPxjhFUFo0bGE8S19I0heykxm12RjIjAg+BT5XcQSBp1w7EP5fAym546HVow14paDTzNCdbBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
Date: Mon, 27 Sep 2021 15:34:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0075.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 99bceb7b-7b13-46b7-9a70-08d981bb8bc9
X-MS-TrafficTypeDiagnostic: VI1PR04MB2958:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2958B3DD62C665BD57C8F1E8B3A79@VI1PR04MB2958.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	u6AKxlIx35nAaOF2btANBJy5j5pKVLtth+hSpao/xShJmSSGbbu1mTco4fjZlXfn8s9T2TFyvNg6mLLDDys6LlK5vFP3chL8Yd/6t3pYnaxZnrnLlJoEg971e2Ffrmv+ltCyF40IKSR5Hi4IQE8N9C8o8hXkEW2eT232lEf3v+NIq/Bomj1/2IyyfXLJJZAO2d2DSRY43r3efoQUDYfnn7D9hyXtHWLmZ2+u2vcsKE4EZUBCoEVXyc/K3s58RiW+9Ebw+2Pb3IcE1nXr7CfxeyW23hm+aJZhIPtr9jI3JUGTZamxZnevptVgQuk8Xejo9PgkIsxpfqi+qU+cm/4+lgJV36fzJOuVp2nEiAwE4QSj6pV7xQxpxiaLFksxuji0ZWhOHjUox+lUw7+c0IXI02gqCBsUk0u6F5rrSv/d2Md5ttLL3LvN0blv7+jf0kAR6irzNuqyiOvDOjdyq+sHSKi+vH09E5wJWil4FNTF9e0xCoqXoCrnAK6GdDXwHyzrwkbT34Ey5sVZJwxsiI/cHTK9IO5lYFZ5RYxnl2kevP7AArXeh0FZnMVr4fhOzKWKoXgrasEBqvQ5PECNd+TCwXHN374/IflWjbiYbqgU2Gtkmq+Lb4SazdAesLERVSukCmqTB0Phw9BYaSR+INRQyTKOg9st5NtExvu+VIy5VS+q4CPW+n4O/A0sOl1opQq9P1QeRw2SZIV91JA3nXa+SG0OXnmRbiqB89o7qIO1zbQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(66946007)(2616005)(31686004)(316002)(5660300002)(4326008)(36756003)(2906002)(66476007)(956004)(66556008)(508600001)(8676002)(54906003)(83380400001)(26005)(31696002)(8936002)(7416002)(6666004)(6916009)(86362001)(186003)(53546011)(6486002)(38100700002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WHRjVG94MnNWQWlmOFVvS0huMEtQVHUwTk44TkhTM2NxRG11UzZkb1JQV3ow?=
 =?utf-8?B?RGJaQ0Nad2dod3l0RUZaMWhadXZGWHVZNDFaVTNZTjlIQVJaTStFaE4xZXZF?=
 =?utf-8?B?eHMwT1plVldZdkJZb21MVVFweElyRm9MbUZlSlZHNHYzbXp1RDQzWFFQM1BR?=
 =?utf-8?B?cUViVnEvTW16TWlCaG5RRzlkMzhUOHk0WERnRk0wbG5YYjhQbkxKa2plbU9H?=
 =?utf-8?B?OGtsM1BtVlJwSVRJWTZVOGFuWXV5VTRRcnQ0T3duMEVWV2Vjb2NDYTVSeCtj?=
 =?utf-8?B?MFlYMy8zbytEbVA1aHZwUE8wazBtZzlGd0dRSVdiYytSM3BxUFpEU2o3clRj?=
 =?utf-8?B?VGpTdTd1b1FCWEU5aHI3UCtSaWFtWGQ1RVJYSnVzMEkza0NQY3p2L3E5RUtO?=
 =?utf-8?B?TVFJQTcxbmtEYVlZVnRkWFl2RmNnTFpYb252dS9VVHcrTXdMY3l3TGlTY2k2?=
 =?utf-8?B?SzZpeTAxejB5TWJrbUpOaHBXUGtHeWRsSWxVZnZ0cnlEb3RjYTArU0lCOXJh?=
 =?utf-8?B?Y2xiZzNpU0dmZmRmUTF5WEdlNFZ0akc4djFqREFNRWcrV1RGZXV6ZWNMNG9V?=
 =?utf-8?B?UjNqS0htaXMydzNCZ2NMeU02aGo4UmhoU3R2UHUwMkl2eXFETk5PUXlBWEtv?=
 =?utf-8?B?Z25mMEFlb3JpSHRCNjhQd253ZzE0L1BraGhHWSt3NkQzSFoxaW9lRFpjSTgr?=
 =?utf-8?B?eHJ4VnZhNy9NYkg1QTRrRnU1OVJzcEJSbHNOS3NGNU1ZOEZOUlBNMGFKL1BG?=
 =?utf-8?B?cjJqL2doVlFkTm5mY3hCcGl4cklIZDFtc0RxMlRRL3A1eTgyWm9CU08zaVls?=
 =?utf-8?B?REZtZTZaMTdhSjdwUHRUcW0yTCtqZTlITGppSWE1eFNkSHBKNDhqME53TWZv?=
 =?utf-8?B?OXpZRDlmaXdybXBCYXF2bTFTKzNnWW45eEF0ckJzN0kvZ2RlcWQzT2pLUmVj?=
 =?utf-8?B?RVk4SkZVMFVCY3lVNEJzaWxmZ1RYLzdZQnBUNmtrT0FCVFZzdzV2TUE3MFJs?=
 =?utf-8?B?NWIyNVRidTJhS25FWVpoVlFtNkpoSXoxMUk1QXViaGdOZThqcUIyOVd5Z1pG?=
 =?utf-8?B?TW9BdndXUXRDbmZ2QmJLNHZRRHpQY0V6K3p2QUtXN3NTSndvVjMyZDFYVE1P?=
 =?utf-8?B?NjUxeWxndHp3OHhUaXlGdDNmdHZGZ2xQWEs1WTNLUElNR0h5WnFZbjkzQklV?=
 =?utf-8?B?cG5WZ20yd01ld0drbk5idVNsRGxReGRSNmhWNTl0YXVvVzh3bFltZE5MM3hp?=
 =?utf-8?B?VTNuM2JzWWRNRVVOY1FPZ25jTkp5QSthSHZBMzNlbHZGRnVFS3h4SmJENHNi?=
 =?utf-8?B?SDdXd21DSTVTZ3A3a1dtSnB5WFBBWitYYkhpSHdaNTdwTWpVcG9sQnhrM09H?=
 =?utf-8?B?UG5KZXpQMlROSjNKOXZlNkdEUkxHbkVHa3Z0aDdQVEJzdUtpS0dSQVY2dDIy?=
 =?utf-8?B?MTYvZVF2NTVmczVIL1UrTE1jSjV3R0RMNzJYOVl0bnFVVldQOXdUM09EUUpm?=
 =?utf-8?B?VWZrSGVpcmpKeDJ3QmFqVzlFbUFpbjZTd1JEWG03SjlzQkNDMm1zSWkweVVI?=
 =?utf-8?B?U2FOS1AvQmdUTzNQb0J4SlU1T0crREFKUklXaW44bU1jWmZSSGU3RFl3MVBw?=
 =?utf-8?B?aFdMVkhpODFxcEpPdHkxbU5YSmRsemEzb2sydloxdDVUUjNxNEdpdDZIcFUw?=
 =?utf-8?B?K1ZjcHdhWWxvZ0piRXE0VlpJdGZJTFZEQWFSVlAwNnBwYVpHcENRcmFPK1Fv?=
 =?utf-8?Q?rGloOpvwBfRIRqdel+91nYpVpOSXvV0AQnqx7se?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 99bceb7b-7b13-46b7-9a70-08d981bb8bc9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 13:34:35.5477
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FB2NQup/cREjkhriNdXPf/fDLSf27nQifFMMd9nJk7LbNr+sfQoVNNHuUDOvehWlXl2oLp7Mf5N131QlmHLARA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2958

On 27.09.2021 14:08, Oleksandr Andrushchenko wrote:
> On 27.09.21 14:31, Jan Beulich wrote:
>> On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -890,6 +890,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>>       return 0;
>>>   }
>>>   
>>> +/*
>>> + * Find the physical device which is mapped to the virtual device
>>> + * and translate virtual SBDF to the physical one.
>>> + */
>>> +bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf)
>> Why struct vcpu, when you only need ...
>>
>>> +{
>>> +    struct domain *d = v->domain;
>> ... this? It's also not really logical for this function to take a
>> struct vcpu, as the translation should be uniform within a domain.
> Agree, struct domain is just enough
>>
>> Also - const please (as said elsewhere before, ideally wherever possible
>> and sensible).
> Ok
>>
>>> +    struct vpci_dev *vdev;
>>> +    bool found = false;
>>> +
>>> +    pcidevs_lock();
>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>> +    {
>>> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
>>> +        {
>>> +            /* Replace virtual SBDF with the physical one. */
>>> +            *sbdf = vdev->pdev->sbdf;
>>> +            found = true;
>>> +            break;
>>> +        }
>>> +    }
>> For a DomU with just one or at most a couple of devices, such a brute
>> force lookup may be fine. What about Dom0 though? The physical topology
>> gets split at the segment level, so maybe this would by a reasonable
>> granularity here as well?
> 
> Not sure I am following why topology matters here. We are just trying to
> match one SBDF (as seen by the guest) to other SBDF (physical,
> as seen by Dom0), so we can proxy DomU's configuration space access
> to the proper device in Dom0.

Topology here matters only in so far as I've suggested to have separate
lists per segment, to reduce look times. Other methods of avoiding a
fully linear search are of course possible as well.

>>> +    pcidevs_unlock();
>>> +    return found;
>> Nit: Blank line please ahead of the main "return" of a function.
> Sure
>>
>>> +}
>>> +
>>>   /* Caller should hold the pcidevs_lock */
>>>   static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
>>>                              uint8_t devfn)
>> Seeing this function in context (which patch 2 adds without any #ifdef
>> around it afaics),
> 
> I believe you are talking about vpci_deassign_device here
> vpci_{assign|deassign}_device seem to be not called on x86 PVH as of now,
> this is true.
> 
>>   will this new function needlessly be built on x86 as
>> well?
> 
> It will at the moment. But in order to avoid ifdefery I would like
> to still implement it as an empty function for x86.
> 
>>   (I didn't look at other intermediate patches yet, so please
>> forgive if I've missed the addition of an #ifdef.)
> 
> So, I can gate this with HAS_VPCI_GUEST_SUPPORT in patch 2
> (HAS_VPCI_GUEST_SUPPORT is introduced in patch 4, so I'll move it to 2)
> Does this sound good?

Yes.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 13:43:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 13:43:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196881.349770 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqv4-0003mX-NO; Mon, 27 Sep 2021 13:43:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196881.349770; Mon, 27 Sep 2021 13:43:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUqv4-0003mQ-Je; Mon, 27 Sep 2021 13:43:46 +0000
Received: by outflank-mailman (input) for mailman id 196881;
 Mon, 27 Sep 2021 13:43:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUqv3-0003mK-Qn
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 13:43:46 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ce064853-999d-45da-ada7-f24061ccfc8c;
 Mon, 27 Sep 2021 13:43:43 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18RDguLv015569;
 Mon, 27 Sep 2021 13:43:41 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbewr829b-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 13:43:41 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5697.eurprd03.prod.outlook.com (2603:10a6:208:176::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 13:43:39 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 13:43:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce064853-999d-45da-ada7-f24061ccfc8c
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Az1jILi20w4NZsxRs6zJusxvIk5Ad8Vhu4oBiHeIEHYFkvCe2wuTE2fc2WrVLIuq8Mc+kvy7BttUMssq0OUGZ3a2qByMkz/aMLsgcV07sHwTOyLsXqh1xxpwA+MVbOCEz84l6owe1xwPUfldVBE+XGmfJjpe5kb5MvihQ5Th1YNlZ3xYTpzkcAlOpe3loWVweF7dvQ2WjTeBWofuHKwwf/P91kci9Qiu3qqWwA6i/UNGNfYW6YZmQjcnKvUy8Ay4AXiR4NFKXmd3DR2Fw3vb3Yeigh0KTM6gV5CnyQh1rr1SZKieLA0nvuzrzRT8xA9EZdcufarf8kjcmf+X2Ak2Dg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OwstH2iTl88Yqp1+7Q5OgdCSPwcuaZstrSlZ+MRXWhM=;
 b=ZFR6BlnLC9O3eOG6Zo0sWGEOmRpuhY+upmnvj6+s+JaC8zjCpuXrAyyprq2HOgMbAshr7E9UmqXgZYwG9DyyG97oKPxQpDzjt6Mfh8O+2Q4oWIY8o9JQPAfv2qCc+XxlMVkSElhPLm8/CLP44vSKXPhCvPqqMlz405m37eKUPWqnOg9JY5ZemPiN+ep6sS7xLowF1bh9zqfUhi5gQqxUR7SxrZ3iYToH9ZaPFqj0Re2mq8YDn2+UsGdaftiBo+1M899lkQZQbaABAnm7vZJuIm1QlM1jx3g+xsI3vQS2jc2S1pmH0sXYSgkubCcwoL0WWIS0IL7UzSeQ0VVT5Jc68w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OwstH2iTl88Yqp1+7Q5OgdCSPwcuaZstrSlZ+MRXWhM=;
 b=Jo8183zooor6XvezoSAFdFZG89Q8lG0tiKgSqSl1ZkHSNRyPOgsbcCInncUNdUjfqzZy/FQjRp3XpKn+PZ+fknQu4xHJSk8thvABzTKxp8n9DxSe8Vkvakj7g6X7SLeU3sFtOPeEwt7p9jF8jwkpIMPXbYlnP+/0O6dqyYOeAWvt0PL6z1LI/mLgkYQbzsYgUX8LnrB9NnnTGRubZT3YJboCL1XngyKndKyVz+ijmQqZuZu9hUK8r4Kkp6qSHly6V13r0sRfZKC/G8rO8cEeS0HAC9ftlZAw7r+Y3rawPNakmGV7DRp86auleH5fEOX91eWDFjG5eB4ywrCsS1wheA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: AQHXsHpEqeYVqdD8fU+X35KPuXAnA6u3xY6AgAAKMQCAABgZgIAAAowA
Date: Mon, 27 Sep 2021 13:43:39 +0000
Message-ID: <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
 <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
In-Reply-To: <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 45d64c47-af92-4ade-2442-08d981bccfeb
x-ms-traffictypediagnostic: AM0PR03MB5697:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5697D1D8C34801302A200393E7A79@AM0PR03MB5697.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 V5vY7qzA13V89+jyHio76fumKpmeeZYE2cbjpHimFbBJBSuBX7hEnn6lssArdYgtw7ywopSsoajSE7XaCZC892o7falAmCd6FQTlQXOviPXTtQnQtdzv/CC/3r4e9HPojKnRXib6ePZ1qr0BPngUNzlW6JnkIVo9rGYc+bJfBcYiTbjAtNfll+HmXTeqwYuAO5SEfD3bxonGaQ6UBnBVyvbTOGQ+wQkxYPXdjoW1Jg/8suxASSEd3RsJbdjZURtdORwYHobPMW3ijrXsMFDTefN9b3H7T1iHFlbgLjkJMSxgs2q4XzgoWJd79BIwsUHntMZf4GO8m+gEc5iJOR7URwex4WzuVvsOfcIENNdES5LK729caU1jeYUNXAtjVXSvs8RldKevBZRt+WwLB+KwmKdGSueyUkTtPCAft+v3VPj86YeEVThFqJ9M0Z1TEQ2DMyNAWTuunBTUoc9/fpNe+Ch8oVhiNM+g9fXAU/7Kr5r3r8v7CqaSr21PGu5kgRkLC1CTB4/GjHJQyi9lAbmfApy2i4+xlLlBI/FrUo07O9684VW2MEXv1EKZZ/oKXjFGftSnp+B9W1VwN33iEOePcFX1TjdweGSqwF0XHXDMAT0LbK9bnvbn0QH+eeO/73r9Q26sj2Nmf98gHnLvgkBmd/ZIOQFULdVY63mRg7tNvuQs8EYJkTikHmNFJIUfYXLUPBLpsbsJV/yWvY6m7bzaSJZPIfEEKYZ2nQky9Zrc04+JuIjmQkq8Dg1rJshc/cbT
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(26005)(55236004)(6506007)(36756003)(86362001)(2616005)(38100700002)(508600001)(6486002)(122000001)(53546011)(5660300002)(2906002)(31686004)(6512007)(38070700005)(71200400001)(186003)(66476007)(4326008)(316002)(66946007)(66446008)(64756008)(8676002)(76116006)(8936002)(91956017)(107886003)(83380400001)(66556008)(54906003)(31696002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?c2gvNnMvODAwYW1ZTEtUdHNVUXFaNTJXWUdTVWdNN1U3SW50WXRKNDR4MXZh?=
 =?utf-8?B?M0U3T3ljVDNmdlA4Q1Z5QWM1RHI5TVFjNVFqakNQVXZhN3RiSWJuKzJHWWVt?=
 =?utf-8?B?SytOSmRmZnZyL1ZPNHgzMHdMM05mbVJ5S1F6bEpsT0IyQXNjNVQ2QnZ3aTRz?=
 =?utf-8?B?c0FpOVlWaGdMQ1FRYk1yR3JBZXNQaFZDS2hVaFZjdGVmN0UwbXdWY3d6UWdE?=
 =?utf-8?B?LzNMRUVKWCtkQ0pMM3dmam5jR3JIeDVKVzR5QWNxSVRvak9Md3kwMTVHSGli?=
 =?utf-8?B?Z0h1cjFFbU1CVS8xeXl5U2MyWDJGZS9BYXJsbWJ2cUkwV3ROWlV3cFQ3SWpJ?=
 =?utf-8?B?YlJ4WlZBck5WcVdEQ0dqMExrU0Z0ck9CT01jUzhCRVpnQlU2RVp2NzBxc3oz?=
 =?utf-8?B?bVdYTGJpaWZDTmhxaXBwWEduRHR5ZTRyRDhDYXdIcHNucDZ1dUt0MUtjbEp1?=
 =?utf-8?B?cnppYXNDNDk2aDYvdnp5T1pFRmRueWR5dE80UURnN2tpeTNVdStCUW5DdGNN?=
 =?utf-8?B?L1k5dkpoNUlleUZNNXVkWHA5Mm9Ua3dHS0p6cHdINzF3YVYvanRIc1NiQ0xL?=
 =?utf-8?B?N2hZWEZVWHhMQUV1WmY5dUhUWGc4Zkd2U1hMN2Vnb1h3OHFMMkNpQVhrSFFD?=
 =?utf-8?B?Z3EyWnBTTHlxQnN0dmVpSHpIT0MvRUpyY2E5M1Y1aW10SGVjMXBndjlGUzN4?=
 =?utf-8?B?MGFNejc4d0VPZlBKckorK0phc1ZQZ1IzcmduMVBrL2FpUGtjcW5pWGpEVU1k?=
 =?utf-8?B?UE4ybU15K0ZaTDBPa25ldDYvckxrWDdJa3Z6aEpid3ZyTTNnK04yVTlZYWJ4?=
 =?utf-8?B?d1pWOEtZTmJQajFady9obFZCZW5NN2czL3pXZ2d2YzRzbGw3ZzdoRWxqQ2xx?=
 =?utf-8?B?eEYxcW1OL3I2L3Z4bm55eWxxb0xuRVpCV0Y0TC85WE5vc1dYeEFWdUtEeWE3?=
 =?utf-8?B?SENOZ2JkNjRqQ0ErQk04dHlpYjluRUFKN3BvcGxGaEZHK05sZUdkTVlqVmJJ?=
 =?utf-8?B?V01qV0IxZTBXeXNCcVBBdmJuM3dMZWxOcWhKdXFoMDRmMk9YcnZhZmR2cm1K?=
 =?utf-8?B?d2phMnhUY0V4clNvc3ZRNnhXTlZlSVc2VnRHVTZnbHhFSkJzV3Z0bVdTY0sr?=
 =?utf-8?B?eVEzdUs2ZWpUTG9MbE5Vd2lYMzZIZDlLSC9uaVN6Y3owUnQxNmgwOVZqdVQy?=
 =?utf-8?B?THIwenRjNlVucWVqWlJ0RmQ5M090UXVjcWRpMWlhcVlFQStqdmNmei9RYUJM?=
 =?utf-8?B?RFlCYTYzakZybHZ2Wk8rcFc0bUlCSGh1WFcrUTFiK2swamNyTFhzVnA4MXFS?=
 =?utf-8?B?eFNUZ0lRZnErZkJUbzVpMGFJWC9RMWlDTW9uR3Bkbm9nOXFWUm9TMWF2c1Ru?=
 =?utf-8?B?RUpQVnhabzdESlIxYm5kNFd2R2VHcEdnNWFwNlNjQjk0NWFQVlZjYmNIdVE5?=
 =?utf-8?B?VDBObEtmaDQ4WEVwbU5maktQVWZEbGI1b0lJNHYrb1g3Z25UK3BRTVZ1TTJ4?=
 =?utf-8?B?a2pSSTRGenovcEJwWGd6UU9nSFF6MjE1OGdWOVJhMFpIOUt5M0tRSncyOC9q?=
 =?utf-8?B?UWJXb3BJbEZtNDVwNmFqYWFEaVlzazkvMkFsQjVEQldsQU03ZXVqVXQ0QVlE?=
 =?utf-8?B?UFVxazhrR1IwK0RsTmQxRDJ4eDBNMWUxRTgwOGYxMDd5ckhKbkVkOFNCZE4r?=
 =?utf-8?B?d0FZS3pFbXJKQzJJWWVUMHdIaHNqMFVBaVpkWlVvK1pITm4xYTYwNC9naFAv?=
 =?utf-8?B?M1ZQK1g2V0c3a25FWXBONGdjRUI1YjI3QVhVWkV0eENqbXZ4ZDZTVXptYzhz?=
 =?utf-8?B?UkRGRGNQc0dyemV3SklwQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <AD646CC93040EB449514B760F6629524@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 45d64c47-af92-4ade-2442-08d981bccfeb
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 13:43:39.1187
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: hhyeqPcnc/KNYrHcXw6KMVe8qxNvdiqeY6RWdd/CxxA5RIOYVrsmVFnxlRNjFb1iulLGSeWPK3vY19ufh4OTnzulaA51/86eQuXRmMFW77hMoDkSfdKdNVCxxHXfqAVk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5697
X-Proofpoint-ORIG-GUID: UT4S6YB7rzP8aPvA5MTtqqQLyE3z49q0
X-Proofpoint-GUID: UT4S6YB7rzP8aPvA5MTtqqQLyE3z49q0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 mlxlogscore=999 phishscore=0 mlxscore=0 bulkscore=0 lowpriorityscore=0
 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 spamscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270094

DQpPbiAyNy4wOS4yMSAxNjozNCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTQ6MDgsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTQ6
MzEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTUsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3Vn
aC9wY2kuYw0KPj4+PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+PiBA
QCAtODkwLDYgKzg5MCwzMSBAQCBpbnQgcGNpX3JlbW92ZV92aXJ0dWFsX2RldmljZShzdHJ1Y3Qg
ZG9tYWluICpkLCBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4gICAgICAgIHJldHVy
biAwOw0KPj4+PiAgICB9DQo+Pj4+ICAgIA0KPj4+PiArLyoNCj4+Pj4gKyAqIEZpbmQgdGhlIHBo
eXNpY2FsIGRldmljZSB3aGljaCBpcyBtYXBwZWQgdG8gdGhlIHZpcnR1YWwgZGV2aWNlDQo+Pj4+
ICsgKiBhbmQgdHJhbnNsYXRlIHZpcnR1YWwgU0JERiB0byB0aGUgcGh5c2ljYWwgb25lLg0KPj4+
PiArICovDQo+Pj4+ICtib29sIHBjaV90cmFuc2xhdGVfdmlydHVhbF9kZXZpY2Uoc3RydWN0IHZj
cHUgKnYsIHBjaV9zYmRmX3QgKnNiZGYpDQo+Pj4gV2h5IHN0cnVjdCB2Y3B1LCB3aGVuIHlvdSBv
bmx5IG5lZWQgLi4uDQo+Pj4NCj4+Pj4gK3sNCj4+Pj4gKyAgICBzdHJ1Y3QgZG9tYWluICpkID0g
di0+ZG9tYWluOw0KPj4+IC4uLiB0aGlzPyBJdCdzIGFsc28gbm90IHJlYWxseSBsb2dpY2FsIGZv
ciB0aGlzIGZ1bmN0aW9uIHRvIHRha2UgYQ0KPj4+IHN0cnVjdCB2Y3B1LCBhcyB0aGUgdHJhbnNs
YXRpb24gc2hvdWxkIGJlIHVuaWZvcm0gd2l0aGluIGEgZG9tYWluLg0KPj4gQWdyZWUsIHN0cnVj
dCBkb21haW4gaXMganVzdCBlbm91Z2gNCj4+PiBBbHNvIC0gY29uc3QgcGxlYXNlIChhcyBzYWlk
IGVsc2V3aGVyZSBiZWZvcmUsIGlkZWFsbHkgd2hlcmV2ZXIgcG9zc2libGUNCj4+PiBhbmQgc2Vu
c2libGUpLg0KPj4gT2sNCj4+Pj4gKyAgICBzdHJ1Y3QgdnBjaV9kZXYgKnZkZXY7DQo+Pj4+ICsg
ICAgYm9vbCBmb3VuZCA9IGZhbHNlOw0KPj4+PiArDQo+Pj4+ICsgICAgcGNpZGV2c19sb2NrKCk7
DQo+Pj4+ICsgICAgbGlzdF9mb3JfZWFjaF9lbnRyeSAoIHZkZXYsICZkLT52ZGV2X2xpc3QsIGxp
c3QgKQ0KPj4+PiArICAgIHsNCj4+Pj4gKyAgICAgICAgaWYgKCB2ZGV2LT5zYmRmLnNiZGYgPT0g
c2JkZi0+c2JkZiApDQo+Pj4+ICsgICAgICAgIHsNCj4+Pj4gKyAgICAgICAgICAgIC8qIFJlcGxh
Y2UgdmlydHVhbCBTQkRGIHdpdGggdGhlIHBoeXNpY2FsIG9uZS4gKi8NCj4+Pj4gKyAgICAgICAg
ICAgICpzYmRmID0gdmRldi0+cGRldi0+c2JkZjsNCj4+Pj4gKyAgICAgICAgICAgIGZvdW5kID0g
dHJ1ZTsNCj4+Pj4gKyAgICAgICAgICAgIGJyZWFrOw0KPj4+PiArICAgICAgICB9DQo+Pj4+ICsg
ICAgfQ0KPj4+IEZvciBhIERvbVUgd2l0aCBqdXN0IG9uZSBvciBhdCBtb3N0IGEgY291cGxlIG9m
IGRldmljZXMsIHN1Y2ggYSBicnV0ZQ0KPj4+IGZvcmNlIGxvb2t1cCBtYXkgYmUgZmluZS4gV2hh
dCBhYm91dCBEb20wIHRob3VnaD8gVGhlIHBoeXNpY2FsIHRvcG9sb2d5DQo+Pj4gZ2V0cyBzcGxp
dCBhdCB0aGUgc2VnbWVudCBsZXZlbCwgc28gbWF5YmUgdGhpcyB3b3VsZCBieSBhIHJlYXNvbmFi
bGUNCj4+PiBncmFudWxhcml0eSBoZXJlIGFzIHdlbGw/DQo+PiBOb3Qgc3VyZSBJIGFtIGZvbGxv
d2luZyB3aHkgdG9wb2xvZ3kgbWF0dGVycyBoZXJlLiBXZSBhcmUganVzdCB0cnlpbmcgdG8NCj4+
IG1hdGNoIG9uZSBTQkRGIChhcyBzZWVuIGJ5IHRoZSBndWVzdCkgdG8gb3RoZXIgU0JERiAocGh5
c2ljYWwsDQo+PiBhcyBzZWVuIGJ5IERvbTApLCBzbyB3ZSBjYW4gcHJveHkgRG9tVSdzIGNvbmZp
Z3VyYXRpb24gc3BhY2UgYWNjZXNzDQo+PiB0byB0aGUgcHJvcGVyIGRldmljZSBpbiBEb20wLg0K
PiBUb3BvbG9neSBoZXJlIG1hdHRlcnMgb25seSBpbiBzbyBmYXIgYXMgSSd2ZSBzdWdnZXN0ZWQg
dG8gaGF2ZSBzZXBhcmF0ZQ0KPiBsaXN0cyBwZXIgc2VnbWVudCwgdG8gcmVkdWNlIGxvb2sgdGlt
ZXMuIE90aGVyIG1ldGhvZHMgb2YgYXZvaWRpbmcgYQ0KPiBmdWxseSBsaW5lYXIgc2VhcmNoIGFy
ZSBvZiBjb3Vyc2UgcG9zc2libGUgYXMgd2VsbC4NCg0KQWgsIHdpdGggdGhhdCB0aGF0IHJlc3Bl
Y3QgdGhlbiBvZiBjb3Vyc2UuIEJ1dCBsZXQncyBiZSByZWFsaXN0aWMuDQoNCkhvdyBtYW55IFBD
SSBkZXZpY2VzIGFyZSBub3JtYWxseSBwYXNzZWQgdGhyb3VnaCB0byBhIGd1ZXN0Pw0KDQpJIGNh
biBhc3N1bWUgdGhpcyBpcyBwcm9iYWJseSBsZXNzIHRoYW4gMTAgbW9zdCBvZiB0aGUgdGltZS4N
Cg0KQnkgYXNzdW1pbmcgdGhhdCB0aGUgbnVtYmVyIG9mIGRldmljZXMgaXMgc21hbGwgSSBzZWUg
bm8gcHJvZml0LA0KDQpidXQgdW5uZWVkZWQgY29tcGxleGl0eSBpbiBhY2NvdW50aW5nIHZpcnR1
YWwgZGV2aWNlcyBwZXIgc2VnbWVudA0KDQphbmQgcGVyZm9ybWluZyB0aGUgcmVsZXZhbnQgbG9v
a3VwLiBTbywgSSB3b3VsZCBnbyB3aXRoIGEgc2luZ2xlIGxpc3QNCg0KYW5kICJicnV0ZSBmb3Jj
ZSBsb29rdXAiIHVubGVzcyBpdCBpcyBjbGVhcmx5IHNlZW4gdGhhdCB0aGlzIG5lZWRzIHRvIGJl
DQoNCm9wdGltaXplZC4NCg0KPg0KPj4+PiArICAgIHBjaWRldnNfdW5sb2NrKCk7DQo+Pj4+ICsg
ICAgcmV0dXJuIGZvdW5kOw0KPj4+IE5pdDogQmxhbmsgbGluZSBwbGVhc2UgYWhlYWQgb2YgdGhl
IG1haW4gInJldHVybiIgb2YgYSBmdW5jdGlvbi4NCj4+IFN1cmUNCj4+Pj4gK30NCj4+Pj4gKw0K
Pj4+PiAgICAvKiBDYWxsZXIgc2hvdWxkIGhvbGQgdGhlIHBjaWRldnNfbG9jayAqLw0KPj4+PiAg
ICBzdGF0aWMgaW50IGRlYXNzaWduX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLCB1aW50MTZfdCBz
ZWcsIHVpbnQ4X3QgYnVzLA0KPj4+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50
OF90IGRldmZuKQ0KPj4+IFNlZWluZyB0aGlzIGZ1bmN0aW9uIGluIGNvbnRleHQgKHdoaWNoIHBh
dGNoIDIgYWRkcyB3aXRob3V0IGFueSAjaWZkZWYNCj4+PiBhcm91bmQgaXQgYWZhaWNzKSwNCj4+
IEkgYmVsaWV2ZSB5b3UgYXJlIHRhbGtpbmcgYWJvdXQgdnBjaV9kZWFzc2lnbl9kZXZpY2UgaGVy
ZQ0KPj4gdnBjaV97YXNzaWdufGRlYXNzaWdufV9kZXZpY2Ugc2VlbSB0byBiZSBub3QgY2FsbGVk
IG9uIHg4NiBQVkggYXMgb2Ygbm93LA0KPj4gdGhpcyBpcyB0cnVlLg0KPj4NCj4+PiAgICB3aWxs
IHRoaXMgbmV3IGZ1bmN0aW9uIG5lZWRsZXNzbHkgYmUgYnVpbHQgb24geDg2IGFzDQo+Pj4gd2Vs
bD8NCj4+IEl0IHdpbGwgYXQgdGhlIG1vbWVudC4gQnV0IGluIG9yZGVyIHRvIGF2b2lkIGlmZGVm
ZXJ5IEkgd291bGQgbGlrZQ0KPj4gdG8gc3RpbGwgaW1wbGVtZW50IGl0IGFzIGFuIGVtcHR5IGZ1
bmN0aW9uIGZvciB4ODYuDQo+Pg0KPj4+ICAgIChJIGRpZG4ndCBsb29rIGF0IG90aGVyIGludGVy
bWVkaWF0ZSBwYXRjaGVzIHlldCwgc28gcGxlYXNlDQo+Pj4gZm9yZ2l2ZSBpZiBJJ3ZlIG1pc3Nl
ZCB0aGUgYWRkaXRpb24gb2YgYW4gI2lmZGVmLikNCj4+IFNvLCBJIGNhbiBnYXRlIHRoaXMgd2l0
aCBIQVNfVlBDSV9HVUVTVF9TVVBQT1JUIGluIHBhdGNoIDINCj4+IChIQVNfVlBDSV9HVUVTVF9T
VVBQT1JUIGlzIGludHJvZHVjZWQgaW4gcGF0Y2ggNCwgc28gSSdsbCBtb3ZlIGl0IHRvIDIpDQo+
PiBEb2VzIHRoaXMgc291bmQgZ29vZD8NCj4gWWVzLg0KSSdsbCBzZWUgaG93IEkgY2FuIGdldCBy
aWQgb2YgdGhlIGNvZGUgdGhhdCB4ODYgZG9lc24ndCB1c2UNCj4gSmFuDQo+DQpUaGFuayB5b3Us
DQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 13:51:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 13:51:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196890.349781 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUr2M-0005Su-Kc; Mon, 27 Sep 2021 13:51:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196890.349781; Mon, 27 Sep 2021 13:51:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUr2M-0005Sn-H9; Mon, 27 Sep 2021 13:51:18 +0000
Received: by outflank-mailman (input) for mailman id 196890;
 Mon, 27 Sep 2021 13:51:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUr2L-0005Sh-QN
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 13:51:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a845716c-7828-469a-b96a-de1177f1bb8c;
 Mon, 27 Sep 2021 13:51:16 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-_HjOz0EHM1afHNvivzdOjw-1; Mon, 27 Sep 2021 15:51:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep
 2021 13:51:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 13:51:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0137.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::29) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Mon, 27 Sep 2021 13:51:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a845716c-7828-469a-b96a-de1177f1bb8c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632750675;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=650pWNflc7v9BmVAoh8M7oE4XQqDtjRwLgovh2s6xyg=;
	b=aei6EDuoFb3OaimAFTaPgLY0jAcSSTJqyKueK+c5MwDIrBgcjZx/7VloyVxsJqgEnpkDAl
	B1zVfBFzCcnvbgtZvGY0/A2ffY+L6KDCwjoYowJcygC7YtlEf1EOjU1yxpjd5uDcW1AQ4K
	CB417w6NIYQbjNPauFQTvOan1r/4Yvw=
X-MC-Unique: _HjOz0EHM1afHNvivzdOjw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=K64kJVS5L5vbCSHy3wOP7OXL7/WxmokPGUTnZL4CQ4WJokljym5oPWHGUAcSzrieJIL0LR1Y8wEXDinIbTkf4E1kvqE2xo4dHgkr3I+gzI/czz98gz1TI2oRXDq5PIyk6ZkUQ5m3YBWCpij6l8CTK4wsoAAH6jJ0RvwFB0OovTQuNXCqerXSTWbD52OFcarI6VQiNYjOHxpgl7854M0Co2Em6niIHgOav46kIjktqzwR6z+j+v/JfFOflL2wKtyGlwUWWgYIUs5sHXaJ/a8l0FTOsW4YAUkfaxji1Nv6zuNB6kyEIktXi9jTeA0WGyD3e56rYGCfBsdghlLhW7TR8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=650pWNflc7v9BmVAoh8M7oE4XQqDtjRwLgovh2s6xyg=;
 b=To/56KLj4XrD3IZXaGA02wcfq4n/sdLXi/vpXYysiTSl/iyUD8Z2uW7MDoUFhjs0V/QVBi6421gX76RBFL2S8sIiHula2r/4wrMc1MHsjK02Zghxk2jU2g2Ru8dVYpTHM3a09GqXAjrh05Y44Yu4TTWRs/er5VDIDah3vvHNZ83dDKGsxl/LXZMLNc9JKIEjCHsfTn98i7kgCnjECMBQRx/k/Yl1b2Jb54d2EL/9JovgFOIyb0TR1C+MwCO3rDEv6ThPBbmF+BYkOTU8BgDwNPWYbMWH2HfWJ0e+MRO+pJFuUjAI6X/Fa1dZ/YdDvI1e++tjUTjIkZBnPG6MDVS5gA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
 <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
 <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <299f330a-3a99-e87a-d182-762247371e3d@suse.com>
Date: Mon, 27 Sep 2021 15:51:09 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0137.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9dcf8030-5861-4014-ee50-08d981bddd9f
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6304CC09A3D035AEE57775C1B3A79@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iRVin2oAlnarjt16F4WbVF0fN4mSzVmO618DPVKMqCmOdkPTuQ2unu883L3Mj7ovdj84e96v8pZ1+NxwV8EafI+ubBQt15m6CaKZpDd+TuUJqaeFqwNKU23PZQ5vVClEruXlnRISBJv4gRfTivQXMBuc2Ofb/7F+RRdWiWFFFH8nN7wKBhnzRLsViEJCi7w3LYM94PpxoMNm80olVg1sCXeqFTT63IeCZsHbO7EFx2MJ40ZfwcZBgNnXbFMMH7RK1jUPgPTDVx0Sk7XLt2S7q9fFIjiep9WTzrv23HaiJtO50o1n4J7DRu8d5my8eTU/VrkAfU6/wA9oIqjm6lKuUSbvnAIyVr7PpBAG7XSNIW8GG0kLuK+3etGdelC1xbF8c3XnRMo2lX59lSq/PYd91WUj4tbZR68fMDhetuSX3C0jy/GBwyhm531mQuu32W+jLkJTL/A0Y0j2wXTXPloTOkn5eYy081JB1AAd4IGFtXU+qJv8yPXzfS3hsov4XB5sM0V/UXwB2NZvCkVgc4IYm9C8lDd9D9oitDMvFdXDTk9OaUBVkKroBoVxbXnIsNYRTNlrBpd3aS/jEpkUw2ZZELSIKfh4RlNo6zaRUwNkYn9BTAjdbOdu3ZWX/sMUdd0UWuUhxNx9q+ZyGK7pn/hYkheaJch6ANZE5zIoOxPpfVjEpi03ccPRF5MdfGIG+nMfGBBGONi+aeuxYblMjrpJiND/KdiBVAyOzOcUrqsbzhEY7CEw2j3Kli28169xud0G
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(83380400001)(2906002)(6916009)(53546011)(26005)(8676002)(8936002)(316002)(16576012)(38100700002)(4326008)(54906003)(86362001)(7416002)(31686004)(2616005)(5660300002)(66556008)(36756003)(66476007)(6486002)(186003)(956004)(508600001)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eFBQVjV5NDI5UFdCY2k2Rk9mSmxwS1oxRkgzUnRodmJCejN0TWlHV0lFWEl2?=
 =?utf-8?B?MU0zdktPTzVMV0VmL3pDbU1nbmFDdG1nalo0ZE10aUhIbGZycFBwMU1HaGpr?=
 =?utf-8?B?REdpTFFuY3VCNTQycEJ3M1I4dmdLWTd4ZXNoYUVIRU9id1hDU0c4c3RSb1F2?=
 =?utf-8?B?eS9VdzZteld0Y3NjdGhwVVpWa0xJTXl2cUtEclFLaUExQ0prTkNQWnRHcVp2?=
 =?utf-8?B?a2ZWaUdieUNxZVVyVXVjRmRVbGlhMW1mZGhoaHUrWTVoSW9IMW83ZVlYcDcz?=
 =?utf-8?B?OVQxRm9EN3BYL1Bxc1labkdKczMzM0dkaGwwTWFoOGRWbWFnclpSYmdqc2xC?=
 =?utf-8?B?RlFYT29ISVBMN3k0N3dRdkxMTEIvcFFGZ25YaVJYNkVCTnBRT1dURnkvbGgw?=
 =?utf-8?B?UnF4WnRqN1dSTGJCVkN0bXprVzlnSEI3OTVxYVVncjlMYW03RWo4d1Q0STE0?=
 =?utf-8?B?WHBIWEVuUVZ5UFNWM3JNZ3NZY0ZLQXdtQ2tpakZnbk5tZHpUUDJ5TDdQVWoy?=
 =?utf-8?B?ZVBaY243S1d1VzZJczFmNGt2U3g0R05XRll0S2hqcStZQ2VrcHV4aURqYzZO?=
 =?utf-8?B?T0RKcU8zMUpkdmFlOFBYektCTVQ3b3Q3d1pkSlVwUERVQmU5dnA4d08wU0lh?=
 =?utf-8?B?K3JmTFJjclpTY2dUZkNQYjhNYXlHaVV3cVI2aUgvWmIxSzZoeDdQWnF2cGcr?=
 =?utf-8?B?YU43TWNGQkl0bTNLOWpFbWlKRlF4YjZkVTEydU5HNVNVUUp5ZjkvMzVnRUZF?=
 =?utf-8?B?MGl4UkE1WDVuWWFxQ0NFVXMzTElOUDBBNkJhdHBJVDF2UGpRMEloblZ0ZjRa?=
 =?utf-8?B?OWlmR2F3bCtDNkU0K3NOV0lJVTN3WDdQYkZ2U3BtZHd0bGhjM29RUUd0bWJE?=
 =?utf-8?B?cFB1SnNRcVlLNWhLWnd6UUJ0clJrTWlIR2k5VklpaUwybXlPbTNKTDZ1KzhR?=
 =?utf-8?B?K25SOXNmOFVwV0JSeXVUMExkVDFVT01Lek1yczg5bzRYTno3L2VSTGxoTS9R?=
 =?utf-8?B?VUxQTWZud3psL1JweGlzVE85VjRNdmVHb1c5aDNadE1KcDVROEUvWElOMkJU?=
 =?utf-8?B?TXZuMDhtRm9zL0RZRmQ1THpRWmZDdkZtbzVNL0ZvM1dtQW1vZ3FwRGYwTnJv?=
 =?utf-8?B?ZU1NUlp4VGkvS1pWUE5pZEUwVm1OQ1hxUjNiZEVMS3A5VXN4TDBSZUs2K2NJ?=
 =?utf-8?B?eTV2aDk1MStLK09JN0twWDhhcDk3RG9kK2pQS3ROSlR2Ym5XUFNaWkN3WHRr?=
 =?utf-8?B?YnZYekhNRTEybWVkQnJEdVFob3ZjOEtXNzRmNklKckVDU0l0elM3STQxU2tt?=
 =?utf-8?B?eVlNMGx6b08ybFFvU2VIaFdBaEVEUFgreTlLalVDaWdiRzdOdTdzdVljNUVZ?=
 =?utf-8?B?RjZZNVptREVWUnBkMUdzcUE5QngweUNueUxrNkI0RWpvOXYrMndZUENsVG51?=
 =?utf-8?B?ZWdnZC9xYWgwbmtPQ1ZlL3VrYk1HWWVvSUxNby9RMWJtUFgrSEF2RDB1VmNM?=
 =?utf-8?B?c1ptZGxsR3B1K3Exd1hzdTQ1ODREMENIVHVueEE0TFVGZlNIWjYxSGthV1Zn?=
 =?utf-8?B?SzlXbjJtaW1uUjlMOGlZNVlFRHBIeHRjWHVQNWxzNEpIeHQvZnBMQmNSL2N6?=
 =?utf-8?B?RDRUSlRxQ1A0SmFQMW9rTVhQMG5kdUk1TTd1M3dRTnZIamxyd0cyTmZ2b2I4?=
 =?utf-8?B?aUsraThnVVZPS1N5UVhWMVU1N3JkYjV4cHRuSUR2aWtHL2MrMGxkalhuWGdS?=
 =?utf-8?Q?HqQ6lO/NOc3irg+jionqyTE4Lm3qgu1nnHqPtBL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9dcf8030-5861-4014-ee50-08d981bddd9f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 13:51:11.7733
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9LGQhk/VDynsbg2d7bs1BbO7zU+EAqY4sZxrGuqEG75VKrABaaBFJQw+H88Axw1Cp8q3jFU7XHsCa3kxBKaCyA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

On 27.09.2021 15:43, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 16:34, Jan Beulich wrote:
>> On 27.09.2021 14:08, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 14:31, Jan Beulich wrote:
>>>> On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>> @@ -890,6 +890,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>>>>        return 0;
>>>>>    }
>>>>>    
>>>>> +/*
>>>>> + * Find the physical device which is mapped to the virtual device
>>>>> + * and translate virtual SBDF to the physical one.
>>>>> + */
>>>>> +bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf)
>>>> Why struct vcpu, when you only need ...
>>>>
>>>>> +{
>>>>> +    struct domain *d = v->domain;
>>>> ... this? It's also not really logical for this function to take a
>>>> struct vcpu, as the translation should be uniform within a domain.
>>> Agree, struct domain is just enough
>>>> Also - const please (as said elsewhere before, ideally wherever possible
>>>> and sensible).
>>> Ok
>>>>> +    struct vpci_dev *vdev;
>>>>> +    bool found = false;
>>>>> +
>>>>> +    pcidevs_lock();
>>>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>>>> +    {
>>>>> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
>>>>> +        {
>>>>> +            /* Replace virtual SBDF with the physical one. */
>>>>> +            *sbdf = vdev->pdev->sbdf;
>>>>> +            found = true;
>>>>> +            break;
>>>>> +        }
>>>>> +    }
>>>> For a DomU with just one or at most a couple of devices, such a brute
>>>> force lookup may be fine. What about Dom0 though? The physical topology
>>>> gets split at the segment level, so maybe this would by a reasonable
>>>> granularity here as well?
>>> Not sure I am following why topology matters here. We are just trying to
>>> match one SBDF (as seen by the guest) to other SBDF (physical,
>>> as seen by Dom0), so we can proxy DomU's configuration space access
>>> to the proper device in Dom0.
>> Topology here matters only in so far as I've suggested to have separate
>> lists per segment, to reduce look times. Other methods of avoiding a
>> fully linear search are of course possible as well.
> 
> Ah, with that that respect then of course. But let's be realistic.
> How many PCI devices are normally passed through to a guest?
> I can assume this is probably less than 10 most of the time.
> By assuming that the number of devices is small I see no profit,
> but unneeded complexity in accounting virtual devices per segment
> and performing the relevant lookup. So, I would go with a single list
> and "brute force lookup" unless it is clearly seen that this needs to be
> optimized.


Just to repeat my initial reply: "For a DomU with just one or at most
a couple of devices, such a brute force lookup may be fine. What about
Dom0 though?" If the code uses the simpler form because it's only
going to be used for DomU, then that's fine for now. But such latent
issues will want recording - e.g. by TODO comments or at the very
least suitable pointing out in the description.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 13:54:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 13:54:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196896.349792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUr5N-00066K-4F; Mon, 27 Sep 2021 13:54:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196896.349792; Mon, 27 Sep 2021 13:54:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUr5M-00066D-W6; Mon, 27 Sep 2021 13:54:24 +0000
Received: by outflank-mailman (input) for mailman id 196896;
 Mon, 27 Sep 2021 13:54:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DWny=OR=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mUr5L-000665-VM
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 13:54:24 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 69e5ed81-a244-4f2b-9e75-00b2c35d305e;
 Mon, 27 Sep 2021 13:54:23 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id i4so77797235lfv.4
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 06:54:23 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id f25sm1610780lfk.11.2021.09.27.06.54.21
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 27 Sep 2021 06:54:21 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69e5ed81-a244-4f2b-9e75-00b2c35d305e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=jGIe6yzxklf8s+KZ6dGRr06t9LLIe4sCf3QMi2lTE0k=;
        b=FiQZdjgArW3TEJvJs13Q/5Sl2jXn+8PESRkxlICuIKXXHZXiKe14GuHC/d/YqXSvWr
         f4VQnIueGbx9Qxu9TjAK3JSte0F+LCo1xmmEN8+IckFNAziUxtdLvEZUnDbAWjqCmD8i
         rX/YYcN45hKNY5Fax8RvRaumEFWIABVNEanrmJV410575fPxsVYS91EFNJNVinBEIF/7
         /KSqWtwdzmK+WC4GQLstjrGu/nSWNx1ywR4mfdW244xu1ASZrgSl9/3JAlq66SM5WEMm
         6/chjAMK7Qc0ebEHE6kxGcUPOV4ApLxnSPnVPqk2tB+ixSkaYPpiaztCwUD05tZn1Odi
         nZ8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=jGIe6yzxklf8s+KZ6dGRr06t9LLIe4sCf3QMi2lTE0k=;
        b=UJeVjPQ6303kJhIBHM8QD8iJbRJJEzjmLZHBtJER6fcJTNpP9ewiFAYZgVUaAmmx1q
         Wedroa7o5qHtf187c+zB9K3nDSWjDyGHnkeEvhxym0kL7EKTCxFEKYYmQAklAeMgGYgQ
         +sSOuU3Aav4g6hZC6tmrSCUNvA08LTmP0QTRfZO5GhuNQ2Ddtu4jNK2CrbVGOe4zDsc5
         K8z3KN1QCc7s8B7VwlUmXvv3GI4CLbaMmoNZpqrRGF8OVyN46cesCH1VfeA9fkEima4P
         Qwsi0jW18NssjDIhjK2NhT7UE0Tjc/CmMjGJfTgkIkuU7B0TMmaqBOeS+OAnROQ29pOa
         481Q==
X-Gm-Message-State: AOAM533EZeZNsx7HuPnjxxxtAWEozI31V6UR8TQvGxvvva5EgFyRHv5V
	DK91LkIDvmc+ya3HVm+UDa4M97NqZ6Y=
X-Google-Smtp-Source: ABdhPJzTwZ8eZoi+ltHQPL1GIa9m2GVyBhq96FjpW162MV4kBY6fPdKnFZlAD5PF92iAheKR0+Cn6A==
X-Received: by 2002:a05:6512:3052:: with SMTP id b18mr25269970lfb.561.1632750861833;
        Mon, 27 Sep 2021 06:54:21 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	tee-dev@lists.linaro.org
Subject: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for OPTEE_SMC_DISABLE_SHM_CACHE
Date: Mon, 27 Sep 2021 16:54:10 +0300
Message-Id: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Fix a possible copy-paste error in arm_smccc_smc's first argument (a0)
for OPTEE_SMC_DISABLE_SHM_CACHE case.

This error causes Linux > v5.14-rc5 (b5c10dd04b7418793517e3286cde5c04759a86de
optee: Clear stale cache entries during initialization) to stuck
repeatedly issuing OPTEE_SMC_DISABLE_SHM_CACHE call and waiting for
the result to be OPTEE_SMC_RETURN_ENOTAVAIL which will never happen.

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
I wonder whether this patch wants backporting to the old versions
since OPTEE support went in.
---
 xen/arch/arm/tee/optee.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 3453615..6df0d44 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -1692,7 +1692,7 @@ static bool optee_handle_call(struct cpu_user_regs *regs)
         return true;
 
     case OPTEE_SMC_DISABLE_SHM_CACHE:
-        arm_smccc_smc(OPTEE_SMC_ENABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
+        arm_smccc_smc(OPTEE_SMC_DISABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
                       OPTEE_CLIENT_ID(current->domain), &resp);
         set_user_reg(regs, 0, resp.a0);
         if ( resp.a0 == OPTEE_SMC_RETURN_OK ) {
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:03:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:03:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196904.349802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrE1-0007lE-Vw; Mon, 27 Sep 2021 14:03:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196904.349802; Mon, 27 Sep 2021 14:03:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrE1-0007l7-Sx; Mon, 27 Sep 2021 14:03:21 +0000
Received: by outflank-mailman (input) for mailman id 196904;
 Mon, 27 Sep 2021 14:03:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUrE0-0007kx-5k; Mon, 27 Sep 2021 14:03:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUrDz-0000p7-Vh; Mon, 27 Sep 2021 14:03:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUrDz-0003Sp-KR; Mon, 27 Sep 2021 14:03:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUrDz-00052D-Jy; Mon, 27 Sep 2021 14:03:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=n2KjTTTkc/eCX76XTBa+msAyEf1OkZPGQJSuRqJG6mI=; b=4sqo7W1HDuG/mfo9Dq7MeQX9OL
	HJ8aq2JFQba6sZx2UTwsvicNXICHQywiN6fshZB8UR7fueRkaV7RUGEm9QJR8ni2lgX+9XM7LK/YX
	GruC5Jf3h+QpqMD6s/HMO9smZXTSp9MoMTIa5zL2JP2YaYHaePJu3hrLd9afUyODu6W8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165214-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165214: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=49d74124a55fac3b2ed9574b7642887c1da49f0b
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 14:03:19 +0000

flight 165214 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165214/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              49d74124a55fac3b2ed9574b7642887c1da49f0b
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  444 days
Failing since        151818  2020-07-11 04:18:52 Z  443 days  434 attempts
Testing same since   165201  2021-09-26 04:20:07 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 75999 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:04:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:04:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196913.349817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrEz-0008Qx-FZ; Mon, 27 Sep 2021 14:04:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196913.349817; Mon, 27 Sep 2021 14:04:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrEz-0008Qq-Cb; Mon, 27 Sep 2021 14:04:21 +0000
Received: by outflank-mailman (input) for mailman id 196913;
 Mon, 27 Sep 2021 14:04:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUrEy-0008Qk-OF
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:04:20 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cf077c54-1f9b-11ec-bc4c-12813bfff9fa;
 Mon, 27 Sep 2021 14:04:19 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18RDvg4q026454; 
 Mon, 27 Sep 2021 14:04:17 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2051.outbound.protection.outlook.com [104.47.8.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbf7b81xc-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 14:04:17 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7348.eurprd03.prod.outlook.com (2603:10a6:20b:269::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep
 2021 14:04:14 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 14:04:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf077c54-1f9b-11ec-bc4c-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fTbzCzfH0B2P2OMk3QsNQzUqqM6vmglFhskXtKdnE2qmJ+pICWreVS2oRbEECvY7ag5MvUQWX048MnveN0MSQ5GLad3maEmu1HKOCMxN/cP9EqRir6NBLpOrbFi7PPktuaKTrWt1gauJtzc7PiR4UdGQGjWJ7M+QJlNBr0+txy4MVMg/2ru1LPi1LyeIE/7WHGmS4TtJ2H/LjyKJRLRmu6zB1zrRLPuSybqqmo+NEro6j9ArsuEIg2nICQF/QjK7gjt4r1ApcgVgBM0ro6gXIo5NJUOupnWrzzV6oA8u+HkeXPiL2ivz9W/7UevDrbEw2DG7f1kif7NW/0UmLAU2XA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+DEs6HovYwf86RRm/8PDLsyZPeEeRbHRTAHLb4SPBZQ=;
 b=FvsG56pTBqRRhZghITKVYxOUMSjT9vkvHm7UJRVw6UAkIw8elsIDeB9DLM92dJgts8H+FEgz62GQ1SMYkhwPHFkKYyC9V8Ert+1x5BU76G9iQ1Qd9ZQ0V9ZRic2psMKumH1eHKGTpSSd3WGJjS8BXe1n+lrrwbkdviuBPkwxD1fdaM8R1uapD+PBC5ubZulCTei3c7sKlu5cvy5sVT+fYcT8dsokdSbixOEoM6kZHOViy8Xplvqap+dZ43rlWIWz4WJIb3age3jqV3QGZn6B+cfMI08xdlvWyt7Fvu3WYKjyurBmZthEAIouKWwWkkyzfjPOQRJFm7uez0n5isl0TA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+DEs6HovYwf86RRm/8PDLsyZPeEeRbHRTAHLb4SPBZQ=;
 b=1CNPyVtw5JmAZj/6rOwbu7aWP8EQYiEZhYXUSeSJoHRYo9IQSKSVYKwuZnyeC3RN77Md9+k3kcksPJpVxiQQ6pi9fFhF1CqWh6H7xaDN05vRxRJx7tUrpfcXmWgOlRkfUZdw43DyCMWwauVGfBOBj5lH4okGtyhoHbhCxonzk4f15hPLz3oWpDsdW0tkpFcNiRMuHAjQEiu3eRdgeGFusym5xaGs8e1TipxJ3B0rYjlWy1xhdKWiQWRtYsyxSKfS08s9w5GK9j3keX/mVdF+RjvvL88CZutcvCAotQecabT2BgYK5k835pqpMzBYPZOvMkUvqLXJLbKsMbry3HnXVQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: 
 AQHXsHpEqeYVqdD8fU+X35KPuXAnA6u3xY6AgAAKMQCAABgZgIAAAowAgAACGoCAAAOoAA==
Date: Mon, 27 Sep 2021 14:04:14 +0000
Message-ID: <e0ee94e4-4076-ac51-f5a8-d895220d1522@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
 <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
 <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
 <299f330a-3a99-e87a-d182-762247371e3d@suse.com>
In-Reply-To: <299f330a-3a99-e87a-d182-762247371e3d@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 515fe925-83d2-4b8e-600f-08d981bfb06d
x-ms-traffictypediagnostic: AM9PR03MB7348:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB7348E848C5A76FB6578D66BAE7A79@AM9PR03MB7348.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 tCbsHXhCDwNuekN6Hfb+76Jvl4IKvCFtC7hJcLnABhZthkgXlq6DtOJ2USVtYyuoUoaKziVY3Ed0DqLssxnanhPqk6ZNzk3ZONORce7StyeqqCL6FOLrUzKNXAaksnxxKQfIGvCvwH93KlAKv8CpJ5yQxuUXolxWpD6kEyh+xELvmKMgtFNn6htTVHpb0x8K8rQt52XaMKKNCliegAiz8GH+FqfU9dWTXbwvivuww2/8+cutXzfhmj801/+FBFaNJKkTR4tOwp0p3HW+IgYmOcWsAgSVLFO27g6HHJWOU5NFsZlRZM6D/arYrX+EgpEXcTphijM5rZXdmr9MRr8dllgTlXKexu77YqeHsvfRt5AB59rt3wTeADhX8v20I+QrD72JlX6X7TCC2tP3eLG/ybOEwir1NMmRRfnfYw8LlOAmpN7n1vTWGQrQRT/q/042yU/Atk00xPc+BtC7Ia0ZBekHKKAbtFORk1p24RsVDsPlUZW0VCIO6dQ/bmDBdEa/fbrrDQwCVxsbCXikF9tCYY6duyrD2calrjrGK9d7KTc+Rl4FFvM+7n0wTBRITC8zYZ2Cjm/La3kZgmiEjk4xSIAW2xmQ46MNG8IY864JI9XWe7HDJs3SdBZO1Kyb1lJH2cQ+Gj3uVqBd/QZd8LGCMg1kbK9kMNngFzmaFsZ/n039ufZIrFh1KJ1gU4a/yTJz0bSJa4cT0UoHz+SrVf6CcYO8lECC5mkRQU4ErB/uDqVXqKSv8M1tTcGQ2IsEKLrY
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(31696002)(54906003)(508600001)(53546011)(4326008)(71200400001)(31686004)(36756003)(66556008)(64756008)(86362001)(66946007)(66446008)(2616005)(6916009)(66476007)(91956017)(2906002)(76116006)(6512007)(38100700002)(107886003)(122000001)(8936002)(5660300002)(316002)(83380400001)(8676002)(38070700005)(6506007)(55236004)(6486002)(186003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?ZjFWR3JlUElMam9yQ3kzaWNHYSthRUhzR3NPRkNQY3FaUTdlZjl6V1RRSWVY?=
 =?utf-8?B?S3Y1QmlZWnMreDVFZG5PVmdYbGw4MDNtY3FUZi9NM1A2Y092Q3MxNWhPb21K?=
 =?utf-8?B?M3RQM1hmelVRV2h6dGc3REc2R2tESGk1Z21DbFRTdTYrU2RrenBuaU1idU5p?=
 =?utf-8?B?ZWpEZXFMVmp6aWJwMFlaWlMxZ3RYQUl2eHJCUklzbFBZR2lGd1l3cUxoT1VF?=
 =?utf-8?B?SnV2K1BYQURHZzlzakc5QlRWMnJoMURPTTYwREdpMXFHb1F0NVJwdlorc1ph?=
 =?utf-8?B?bXh5WEpPTjJ6dUc4R3gzVEtPR1pDeWY5bEpCeGMyK0JlZXFHUUpiOUI4am42?=
 =?utf-8?B?TjBRZHlkOXlBMzdJaytpUEhHVjZjdXZQNFA0ZzJwMlVVZjdLbW44a3dIL1lI?=
 =?utf-8?B?OGFjWUl5TGlwMEY3b3IrdkpGaDQ1QURUcDR4alVQWktzU0NHSjN2dGRRQksv?=
 =?utf-8?B?Tkd0cFR5MzhML0xVRVZ5OERoSGQ5SGh3bk9CYnZrQWw4aUp0UXF1NksvcDBC?=
 =?utf-8?B?c1NMU3o4T1ZkeCtvN1hQeEtJYWczUThjYWxxandMSm5pWEwvUkZTTWhPVE9a?=
 =?utf-8?B?VVlDbmRKNDA4ZDBxQXpuaDNpNEx4UkZhZTJ6YS9iQlYyVGxZbHhyWGQ0Q1hF?=
 =?utf-8?B?WTdYV0RySE80Rm5LcjZyclJqaW94RW5FczRhTkZUU3l5N0thRlpKcVk1VWR5?=
 =?utf-8?B?Rjc0dlNPMTZwVUhZT0hFNjcyczZQUHRydHRaT1lwc1NSNlpDTzQ2ZVJvOUR2?=
 =?utf-8?B?dEdIcTJEd1BMSzVwR0lSRmdlK2FHMktWV1lDR1FQTUxOUDFSZUdMckZDaGZ0?=
 =?utf-8?B?dHB0UDhvTU1wYXRubERnazQ5eVFaR2Jzbkp2bEFER1UzNWRheUZUR0oyNVNI?=
 =?utf-8?B?VkRsRkNvZ3dTQ0t6dGZNTHhEWjRwUW9xZ2dnVnlhUTNqY0xGWUdlWUJSaENL?=
 =?utf-8?B?L0tLTTVmV0Jxa1NxSU1NS0ZKMkRoeVI1WFpMRTJxaW9DeWFEc0ZRc0hsMXNn?=
 =?utf-8?B?NWpNOVk5MVlqdFNYZ3JBa01BdVFvMTBydWlja25RL0VnZHFVYkVKUUR1clRD?=
 =?utf-8?B?T1NwQUVIU2lSd1NpRkNrUjFocW5udjdXTW4xcnBDT3Q3ZGQvTGlhMDlyZG1m?=
 =?utf-8?B?TXhkOWlvSVM1c3Z4ZlZaRi9PbjYxM2lGMlRXQVBrdytOUTI2ZnpwaEVtYlk4?=
 =?utf-8?B?Y0tKb2lQODR5aDFNVkVzTjhoT1k0blVpbHE4Unp0YTc3aWtFQVBvNlBzSXZD?=
 =?utf-8?B?aUFoRGpPeENSRnBieUxzWmJqQk1NSVRmNHM0dUhZcVE5enFkeTNUVXhEd3RZ?=
 =?utf-8?B?WEZiZ25yRE81RCs3SEtoUHdXQmNSRGRSRXlMTWxFMGVFdkYxS3JsRmREN0tQ?=
 =?utf-8?B?a1lzMnoydTNoRHVvYW8xOUl2WGtuTHk1WDVkZHlYOG5XS0p0d0ZCazlacWlz?=
 =?utf-8?B?bTVFU0xJKyszUnVOT3RCMlRMb0JDbFlvZTl1SXZmWUhaZmRFM0FMN2FUV2tu?=
 =?utf-8?B?c2JVM09vczR5c1pxTEc2UjA3SE15cG1KY2haSk1RYTFpUW9MeWsvaktQT015?=
 =?utf-8?B?a2RiNmNONkhNblR5cDZZOUI4UitCT1c3Qkthc2JseGJObGZmb0tvU1AzM0w3?=
 =?utf-8?B?cVdVeXhIbUc5QXdKRFZpNytCdGpWM3l4WHBrYmlIMzdieDRTWlpsMHlNQzdp?=
 =?utf-8?B?MFcyYXhDdnF1cHBoYjlqR3MxYTlXdjYyZU1RT2c4b0NDWEE1THQ4ODJKQkkw?=
 =?utf-8?B?L2I1anErdWtSektFVlU4c2FpUllLUWJqa2FQZlV6TFRhalIxbkRrcWZiWDFB?=
 =?utf-8?B?SklNSElnYXdXY1B4UmhVUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <BBAC5BFC898A714C806343DC845CD875@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 515fe925-83d2-4b8e-600f-08d981bfb06d
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 14:04:14.7922
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: aFb2n9PcnEufTmnJrtwoU2eFWvb4uHMkuP77R6tmqK2AZlNszy6ZcZkJ1Jsbmo6daSsQkJOwBGYGr4fY9dnWfUSVZsRgWi2bdhz1AkE3Jq+kBnDhY7LOrrNzH0LxIL0y
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7348
X-Proofpoint-GUID: -y-eHXSI7zKmulu6792kg_uQ7YbC-Uo8
X-Proofpoint-ORIG-GUID: -y-eHXSI7zKmulu6792kg_uQ7YbC-Uo8
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 clxscore=1015
 bulkscore=0 priorityscore=1501 malwarescore=0 mlxscore=0 suspectscore=0
 spamscore=0 lowpriorityscore=0 impostorscore=0 mlxlogscore=999
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270097

DQpPbiAyNy4wOS4yMSAxNjo1MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTU6NDMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTY6
MzQsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIwMjEgMTQ6MDgsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxNDozMSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyMy4wOS4yMDIxIDE0OjU1LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0K
Pj4+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+Pj4gQEAgLTg5
MCw2ICs4OTAsMzEgQEAgaW50IHBjaV9yZW1vdmVfdmlydHVhbF9kZXZpY2Uoc3RydWN0IGRvbWFp
biAqZCwgY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+Pj4+Pj4gICAgICAgICByZXR1cm4g
MDsNCj4+Pj4+PiAgICAgfQ0KPj4+Pj4+ICAgICANCj4+Pj4+PiArLyoNCj4+Pj4+PiArICogRmlu
ZCB0aGUgcGh5c2ljYWwgZGV2aWNlIHdoaWNoIGlzIG1hcHBlZCB0byB0aGUgdmlydHVhbCBkZXZp
Y2UNCj4+Pj4+PiArICogYW5kIHRyYW5zbGF0ZSB2aXJ0dWFsIFNCREYgdG8gdGhlIHBoeXNpY2Fs
IG9uZS4NCj4+Pj4+PiArICovDQo+Pj4+Pj4gK2Jvb2wgcGNpX3RyYW5zbGF0ZV92aXJ0dWFsX2Rl
dmljZShzdHJ1Y3QgdmNwdSAqdiwgcGNpX3NiZGZfdCAqc2JkZikNCj4+Pj4+IFdoeSBzdHJ1Y3Qg
dmNwdSwgd2hlbiB5b3Ugb25seSBuZWVkIC4uLg0KPj4+Pj4NCj4+Pj4+PiArew0KPj4+Pj4+ICsg
ICAgc3RydWN0IGRvbWFpbiAqZCA9IHYtPmRvbWFpbjsNCj4+Pj4+IC4uLiB0aGlzPyBJdCdzIGFs
c28gbm90IHJlYWxseSBsb2dpY2FsIGZvciB0aGlzIGZ1bmN0aW9uIHRvIHRha2UgYQ0KPj4+Pj4g
c3RydWN0IHZjcHUsIGFzIHRoZSB0cmFuc2xhdGlvbiBzaG91bGQgYmUgdW5pZm9ybSB3aXRoaW4g
YSBkb21haW4uDQo+Pj4+IEFncmVlLCBzdHJ1Y3QgZG9tYWluIGlzIGp1c3QgZW5vdWdoDQo+Pj4+
PiBBbHNvIC0gY29uc3QgcGxlYXNlIChhcyBzYWlkIGVsc2V3aGVyZSBiZWZvcmUsIGlkZWFsbHkg
d2hlcmV2ZXIgcG9zc2libGUNCj4+Pj4+IGFuZCBzZW5zaWJsZSkuDQo+Pj4+IE9rDQo+Pj4+Pj4g
KyAgICBzdHJ1Y3QgdnBjaV9kZXYgKnZkZXY7DQo+Pj4+Pj4gKyAgICBib29sIGZvdW5kID0gZmFs
c2U7DQo+Pj4+Pj4gKw0KPj4+Pj4+ICsgICAgcGNpZGV2c19sb2NrKCk7DQo+Pj4+Pj4gKyAgICBs
aXN0X2Zvcl9lYWNoX2VudHJ5ICggdmRldiwgJmQtPnZkZXZfbGlzdCwgbGlzdCApDQo+Pj4+Pj4g
KyAgICB7DQo+Pj4+Pj4gKyAgICAgICAgaWYgKCB2ZGV2LT5zYmRmLnNiZGYgPT0gc2JkZi0+c2Jk
ZiApDQo+Pj4+Pj4gKyAgICAgICAgew0KPj4+Pj4+ICsgICAgICAgICAgICAvKiBSZXBsYWNlIHZp
cnR1YWwgU0JERiB3aXRoIHRoZSBwaHlzaWNhbCBvbmUuICovDQo+Pj4+Pj4gKyAgICAgICAgICAg
ICpzYmRmID0gdmRldi0+cGRldi0+c2JkZjsNCj4+Pj4+PiArICAgICAgICAgICAgZm91bmQgPSB0
cnVlOw0KPj4+Pj4+ICsgICAgICAgICAgICBicmVhazsNCj4+Pj4+PiArICAgICAgICB9DQo+Pj4+
Pj4gKyAgICB9DQo+Pj4+PiBGb3IgYSBEb21VIHdpdGgganVzdCBvbmUgb3IgYXQgbW9zdCBhIGNv
dXBsZSBvZiBkZXZpY2VzLCBzdWNoIGEgYnJ1dGUNCj4+Pj4+IGZvcmNlIGxvb2t1cCBtYXkgYmUg
ZmluZS4gV2hhdCBhYm91dCBEb20wIHRob3VnaD8gVGhlIHBoeXNpY2FsIHRvcG9sb2d5DQo+Pj4+
PiBnZXRzIHNwbGl0IGF0IHRoZSBzZWdtZW50IGxldmVsLCBzbyBtYXliZSB0aGlzIHdvdWxkIGJ5
IGEgcmVhc29uYWJsZQ0KPj4+Pj4gZ3JhbnVsYXJpdHkgaGVyZSBhcyB3ZWxsPw0KPj4+PiBOb3Qg
c3VyZSBJIGFtIGZvbGxvd2luZyB3aHkgdG9wb2xvZ3kgbWF0dGVycyBoZXJlLiBXZSBhcmUganVz
dCB0cnlpbmcgdG8NCj4+Pj4gbWF0Y2ggb25lIFNCREYgKGFzIHNlZW4gYnkgdGhlIGd1ZXN0KSB0
byBvdGhlciBTQkRGIChwaHlzaWNhbCwNCj4+Pj4gYXMgc2VlbiBieSBEb20wKSwgc28gd2UgY2Fu
IHByb3h5IERvbVUncyBjb25maWd1cmF0aW9uIHNwYWNlIGFjY2Vzcw0KPj4+PiB0byB0aGUgcHJv
cGVyIGRldmljZSBpbiBEb20wLg0KPj4+IFRvcG9sb2d5IGhlcmUgbWF0dGVycyBvbmx5IGluIHNv
IGZhciBhcyBJJ3ZlIHN1Z2dlc3RlZCB0byBoYXZlIHNlcGFyYXRlDQo+Pj4gbGlzdHMgcGVyIHNl
Z21lbnQsIHRvIHJlZHVjZSBsb29rIHRpbWVzLiBPdGhlciBtZXRob2RzIG9mIGF2b2lkaW5nIGEN
Cj4+PiBmdWxseSBsaW5lYXIgc2VhcmNoIGFyZSBvZiBjb3Vyc2UgcG9zc2libGUgYXMgd2VsbC4N
Cj4+IEFoLCB3aXRoIHRoYXQgdGhhdCByZXNwZWN0IHRoZW4gb2YgY291cnNlLiBCdXQgbGV0J3Mg
YmUgcmVhbGlzdGljLg0KPj4gSG93IG1hbnkgUENJIGRldmljZXMgYXJlIG5vcm1hbGx5IHBhc3Nl
ZCB0aHJvdWdoIHRvIGEgZ3Vlc3Q/DQo+PiBJIGNhbiBhc3N1bWUgdGhpcyBpcyBwcm9iYWJseSBs
ZXNzIHRoYW4gMTAgbW9zdCBvZiB0aGUgdGltZS4NCj4+IEJ5IGFzc3VtaW5nIHRoYXQgdGhlIG51
bWJlciBvZiBkZXZpY2VzIGlzIHNtYWxsIEkgc2VlIG5vIHByb2ZpdCwNCj4+IGJ1dCB1bm5lZWRl
ZCBjb21wbGV4aXR5IGluIGFjY291bnRpbmcgdmlydHVhbCBkZXZpY2VzIHBlciBzZWdtZW50DQo+
PiBhbmQgcGVyZm9ybWluZyB0aGUgcmVsZXZhbnQgbG9va3VwLiBTbywgSSB3b3VsZCBnbyB3aXRo
IGEgc2luZ2xlIGxpc3QNCj4+IGFuZCAiYnJ1dGUgZm9yY2UgbG9va3VwIiB1bmxlc3MgaXQgaXMg
Y2xlYXJseSBzZWVuIHRoYXQgdGhpcyBuZWVkcyB0byBiZQ0KPj4gb3B0aW1pemVkLg0KPg0KPiBK
dXN0IHRvIHJlcGVhdCBteSBpbml0aWFsIHJlcGx5OiAiRm9yIGEgRG9tVSB3aXRoIGp1c3Qgb25l
IG9yIGF0IG1vc3QNCj4gYSBjb3VwbGUgb2YgZGV2aWNlcywgc3VjaCBhIGJydXRlIGZvcmNlIGxv
b2t1cCBtYXkgYmUgZmluZS4gV2hhdCBhYm91dA0KPiBEb20wIHRob3VnaD8iIElmIHRoZSBjb2Rl
IHVzZXMgdGhlIHNpbXBsZXIgZm9ybSBiZWNhdXNlIGl0J3Mgb25seQ0KPiBnb2luZyB0byBiZSB1
c2VkIGZvciBEb21VLCB0aGVuIHRoYXQncyBmaW5lIGZvciBub3cuIEJ1dCBzdWNoIGxhdGVudA0K
PiBpc3N1ZXMgd2lsbCB3YW50IHJlY29yZGluZyAtIGUuZy4gYnkgVE9ETyBjb21tZW50cyBvciBh
dCB0aGUgdmVyeQ0KPiBsZWFzdCBzdWl0YWJsZSBwb2ludGluZyBvdXQgaW4gdGhlIGRlc2NyaXB0
aW9uLg0KDQpBcyB3ZSBkbyBub3QgZW11bGF0ZSB2aXJ0dWFsIGJ1cyB0b3BvbG9neSBmb3IgRG9t
MCB0aGVuIGl0IGlzDQoNCmNsZWFybHkgc2VlbiB0aGF0IHRoZSBjb2RlIG1heSBvbmx5IGhhdmUg
aW1wYWN0IG9uIERvbVVzLg0KDQpCdXQgYW55d2F5cywgdmlydHVhbCBidXMgdG9wb2xvZ3kgZm9y
IERvbVVzIGlzIGVtdWxhdGVkIHdpdGgNCg0KYSBzaW5nbGUgc2VnbWVudCAwLiBXZSBoYXZlIGEg
c2luZ2xlIGxpc3Qgb2YgdmlydHVhbCBTQkRGcywNCg0KYWdhaW4sIGZvciB2aXJ0dWFsIHNlZ21l
bnQgMCwgd2hpY2ggbWFwcyB0aG9zZSB2aXJ0dWFsIFNCREZzDQoNCnRvIHBoeXNpY2FsIFNCREZz
LiBTbywgd2UgZ28gb3ZlciB0aGUgbGlzdCBvZiB2aXJ0dWFsIGRldmljZXMNCg0KYXNzaWduZWQg
dG8gdGhhdCBndWVzdCBhbmQgbWF0Y2ggdGhlIHZpcnR1YWwgU0JERiBpbiBxdWVzdGlvbiB0bw0K
DQppdHMgY291bnRlcnBhcnQgaW4gRG9tMC4gSSBjYW4ndCBzZWUgaG93IHRoaXMgY2FuIGJlIG9w
dGltaXplZCBvcg0KDQpuZWVkcyB0aGF0IG9wdGltaXphdGlvbiBiZWNhdXNlIG9mIHRoZSBmYWN0
IHRoYXQgRG9tMCBtYXkgaGF2ZQ0KDQptdWx0aXBsZSBzZWdtZW50cy4uLg0KDQpTbywgaG93IHdv
dWxkIHRoYXQgY29tbWVudCBsb29rIGxpa2U/DQoNCg0KPg0KPiBKYW4NCj4NClRoYW5rIHlvdSwN
Cg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:14:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:14:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196924.349827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrOd-0001fU-ER; Mon, 27 Sep 2021 14:14:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196924.349827; Mon, 27 Sep 2021 14:14:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrOd-0001fN-BQ; Mon, 27 Sep 2021 14:14:19 +0000
Received: by outflank-mailman (input) for mailman id 196924;
 Mon, 27 Sep 2021 14:14:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g88l=OR=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mUrOb-0001fB-QO
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:14:18 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 32a74c34-1f9d-11ec-bc4d-12813bfff9fa;
 Mon, 27 Sep 2021 14:14:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32a74c34-1f9d-11ec-bc4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632752056;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=tlVgqlUaCwykdxCPmHzNbthKGv0kqdLyw1pCdry7ZAU=;
  b=hv9u33Bg3C84hpQvwP7jRLp6mx173lNPzRSjoXlrIf/BE40cHzDlumsH
   WcuWcbbq2OP/pMjDOTtab9L7nVrmCQZ2cVrHz3yjGkKT729R1b8jcH+x2
   p5R+IsHf4EiZGkp8ZP7LzOuPvDwiGHgb6xsa7o14uWLDqjZ2EkH1PRuYL
   g=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: EpbPAmmryCrun1GMsck/hO0TaDRfQsfSjxwTo0Jkuc7IAyNV5AI14z+Ts+PpJ3h/M+yLCXwUoe
 L8XtiUlsh185UQ7m4T8HLIFxHzIPkUqHYqTLLKARvp0iIIeek/pV3dgbclmqj7yQ/6ktIDPbJI
 upi00q1+XfOD1LaKdPoT0dM5xVMkJXkSKx0VK0byUiRIUIxF43H/L2aTra0E/U4xOZKP4As4qT
 yoz//A3dST+nzrFRWQpYF70Fp0vbjRDkNEErEGdyMSwTu89ffPKWYXJvloHnRuhBXXITTNHBmq
 Thvgojj3flYVmDzsllBwjvEg
X-SBRS: 5.1
X-MesageID: 55533150
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:r8ZRzqPxL2puu7PvrR1UkMFynXyQoLVcMsEvi/4bfWQNrUp23jxWn
 DRJCGGHbqmNYTSneNh/OYrlp0oO7MOEyN9gQQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Eg8w7dRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYo2W0x81Y4
 t5OjKKhEEAGPfTWltUdUAYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgWpt3ZsXQK62i
 8wxMgo1YkvHfDp0OBQrCNFjwrauqmCjWmgNwL6SjfVuuDWCpOBr65D9PdyQdtGUSMF9mkeDu
 nmA72n/GgsdNtGU1XyC6H3Eru3AhyTgQ6oJCaa1sPVthTW72Wg7GBAQE1yhrpGRi1G8c8JSL
 VQO/SgjprR081akJuQRRDXh/iTC5ERFHYMNTatqs2lh15Y4/S7JOzAZZxRKa+UYpZZxaj9zz
 FG4n9D2UGkHXKKudVqR8bKdrDWXMCcTLHMfaSJscTbp8+UPs6lp0EuQH48L/Lqdy42tQG6sk
 1hmuQBk390uYdg3O7JXFLwtqxyrvISBagco6gjNUmuh42uVj6b+PNfwuDA3ARtaRbt1r2VtX
 lBYxaByD8hUVPlhcRBhps1WRdlFAN7fbFXhbaZHRcVJythU0yfLkXpsyD9/Plx1Fc0PZCXkZ
 kTe0SsIusQJYivzMf4pM9jrYyjP8UQGPY+8PhwzRoASCqWdiSfdpH0+DaJu9zqFfLcQfVEXZ
 s7ALJfE4YcyAqV71jumL9rxIpdxrh3SMVj7HMihpzz+iOL2TCfMFd8tbQvfBshkvfjsiFiEr
 L5i2z6ilkw3vBvWOXKMr+b+7DkicBAGOHwBg5YMK7HfclY4QjFJ5j246epJRrGJVp99z4/g1
 nq8RlVZ2Bz4g3jGIh+NcXdtdPXkWpMXkJ7xFXZE0Y+A1ydxbICxwr0YcpdrL7Ar+PY6lax/T
 uUfetXGCfNKE2yV9zMYZJj7jYpjaBX02l7eY3v7OGAyL8x6WgjE2t74ZQ+zpiMAOTW66Jklq
 Lq62wKFHZdaH1Z+DNzbYe6Exk+quSRPg/p7WkbFe4EBeEjl/IVwBTb2i/s7f5MFJRnZn2PI3
 AeKGxYI4+LKptZtotXOgKmFqaavEvd/QRUGTzWKs+7uOHCDrGS5wIJGXOKZRhznVTv5qPe4e
 OFY7/DgK/lbzlxEhJVxTuRwxqUk6tqx+7IDllZ4HG/GZkiAA697JiXUxtFGs6BAy+MLuQayX
 U7TqNBWNa/QZZHgGV8VYgEkcv6CxbcfnTyLtaY5J0Dz5SlW+rubUBoNY0nQ2XIFdLYlYpk4x
 eoBudIN713tgxUnBd+KkyRI+jneNXcHSagm6skXDYKDZtDHEb2ejUgw0hPL3aw=
IronPort-HdrOrdr: A9a23:T11zj6PtyZzrkcBcT1H155DYdb4zR+YMi2TDiHofdfUFSKClfp
 6V8cjztSWUtN4QMEtQ/OxoS5PwPk80kqQFnbX5XI3SITUO3VHHEGgM1/qb/9SNIVyYygcZ79
 YbT0EcMqyBMbEZt7eC3ODQKb9Jq7PmgcPY9ts2jU0dKT2CA5sQnjuRYTzrdHGeKjM2Z6bRWK
 Dsnfau8FGbCAoqh4mAdzQ4dtmGg+eOuIPtYBYACRJiwA6SjQmw4Lq/NxSDxB8RXx5G3L9nqA
 H+4kPEz5Tml8v+5g7X1mfV4ZgTsNz9yuFbDMjJrsQOMD3jhiuheYwkcbyfuzIepv2p9T8R4Z
 XxiiZlG/42x2Laf2mzrxeo8w780Aw243un8lOciWuLm72weBsKT+56wa5JeBrQ7EQt+Ptm1r
 hQ4m6fv51LSTvdgSXU/bHzJlFXv3vxhUBnvf8YjnRZX4dbQqRWt5Yj8ERcF4pFND7m6bogDP
 JlAKjnlbdrmGuhHjLkV1RUsZmRtixZJGbDfqFCgL3a79FupgE786NCr/Zv2Uvp9/oGOtB5Dq
 r/Q+JVfYp1P7orhJRGdZE8qPuMex7wqC33QRavyHTcZeo60iH22tTKCItc3pDcRHVP9upqpK
 j8
X-IronPort-AV: E=Sophos;i="5.85,326,1624334400"; 
   d="scan'208";a="55533150"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CY9UD5M/WlEmMf6dvno5ysP8IH8eKCxL0eFympj4KD7J5dbzG2sR+TMviL0ltai0i0rb4l4xSSn6J3weVPnkxSS4XXOJYHfyCNShLv4b6nLFDfPyUGwtZGF/gGzvoYneN/AlRRbJNWlMQEy+HdD6W67Gj9fiAGG1GgrHrXC+nvRIwxcvo6q536FEXY8f7lpcg1dum2RhqBQJ3EwlLEkVr/j0qbXYGx07Qt2dDR2VWpCUcf8KKbLsi9iqGtIS74EQQt0QHbURjISdP09D5hh1vMM5HWl3anXf7vAZ3YWg8UfCsP3DKA0eLfO5PxjlCi6DJvbrQfXr+QgT5sURZuNUXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tlVgqlUaCwykdxCPmHzNbthKGv0kqdLyw1pCdry7ZAU=;
 b=GPxTxGprj5UqSTB6Ik78QfGhA9c6BJR1j+FDpe4tj7YGrDZ5Ndfgbe4MuipDczKS/NsfUfAk4cwuKHomF3vxfWYL9R2RWC8oHcgcKKCgWH3cjezUscM0OGW+Dmz6Aef4we+Ge0Z+K4xd4/jq50f5HTOR8T8orisw20wsJ/3Y6KSm0q9sRnfV5frlxeIRGbeNq+mtLKLGY5T8ICDXXbJurx1OsIl9s2OMpTZIj6qrb/eEp6D3RNBaGYktkQi0OHoR64UhsVEC+mLeD7+We6YIYdV6q8wTvA68SLrZhQquJceYQn/AJsndd873Fioo++8eWTHvikwsHLMQ+/5Wu2g5nQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tlVgqlUaCwykdxCPmHzNbthKGv0kqdLyw1pCdry7ZAU=;
 b=jEJc5pkSK2BHv7H3R3U6e5QaZY75ahCFNpD5sVlfp8tBIdr4Eyt1Lq6uTtJ4F8tdZKgTXjkrrUPKxpmGQDWoB2Zf5x2Cs1As5n3vjfs9lHPSQvaXnsr6GeycaL105SQqj6+gq8egkGloyuoP7goI/rVUYoztJQuMfNpNhU5bmOM=
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
Message-ID: <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
Date: Mon, 27 Sep 2021 15:13:35 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210927104813.19772-3-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0157.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9::25) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 84d8ae59-a21f-42cf-f3b7-08d981c10256
X-MS-TrafficTypeDiagnostic: BYAPR03MB3496:
X-Microsoft-Antispam-PRVS: <BYAPR03MB3496B6A55E87D5EC49241A7ABAA79@BYAPR03MB3496.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2449;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: QtBG7/ozejH2ThHVV/O6r0KCd0mrVnlqdo7kQOysdCb4RvaXc+CDcuA2DM+WQWS6jHjQbUAF5RN1HBlb8P6qi9HVxfjhEmE+fXNPLTR+ezluaQD26+P/e9IMnqQSRQpBrC4rO0/yS9emL4anYW0pqlSQgqzYhxSPfwUSeI+Y+r0F62j9agzQNMaic2EgQFVFW96KZOXwH5zkKEEhwZn0rOnLvmMtxDItAKxZA0gjmNwI8v8JxVXEafSWlivQuJibvVAu4DVfPc9x2hSn71zK/3ObbHezTKIekUCxgwU7pixX+DHEQoqPcPyoRg5/X482r7v/V7Q45W22xkQQ7PsTbuKgC8hFVoWkolcD3+RJ2frYNAnkyWvopbz9GD4idBEz2jF364k6m33gESu2Zi24VVdJ0Hjr3IW8dT0n+7UELxvSoloqybSWjOfeY9zcKgYoczavmWjKvEY0VozYfHx53y1sFsQqFjFjGPuk2wqekFhvYSqODS7okbfqhlORos+JbBHutLVmWqEv3BU3+Nd4VBSjOBib6PHSbc9mewCguqGENPRd8exi2aCdZboR4HxrZ4jLKaVLqBG2mMdGRTcr6SQfm8eh5WpkXnQVU4SLLaXb97V7MdMAw+Xoi1OFgXgTWTo6PgmPswS5ztzP8wGM0+hGIcvzmSyvRHE6kcp7ODzO1gI/ASY2blMlYK8Z4Xhn3GKvEeEeDuwsxOzbdkZRZHRqwMeTeq5uZzcllnbIWdM6YzC7hkesT0Aw9gOmLc36xk2UHu1uaDArOJwSfgQuvbxEYaW1LurQs7Yaok+SK2eSolLxx3DGdg0Qf+QOcv/8
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(31686004)(8936002)(6486002)(53546011)(186003)(966005)(4744005)(2616005)(36756003)(66946007)(66476007)(66556008)(508600001)(26005)(956004)(8676002)(4326008)(2906002)(54906003)(6666004)(316002)(16576012)(38100700002)(31696002)(86362001)(83380400001)(55236004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZkRvRlB0dXNDbnRyalJsQmdrMHhseFZWRnBIc3JtUTQ3N3V3QlhhWXNDSHRM?=
 =?utf-8?B?MzUzZ1drNVJFRVhnb3dwYm82SE53TDkvREs5NHJEV01uVWdralpJci8rV0FZ?=
 =?utf-8?B?cGpkLy91Q1hmbVFuaVVhb2NQNW5RdStXNThIK1FyMCt5ckw2K1BFMTNzRHRt?=
 =?utf-8?B?b21HeXJLQjlkVWZlakU5THp2K0l0ZEpSRTg2c0tsN0ZmSktNS3ZKaFRscVNX?=
 =?utf-8?B?K0dEaVlxM1lUaFFQTTFUSjE4ZE05NFl3c1JsdWVjMlNOQlg5TExoQ0JPSm5m?=
 =?utf-8?B?ZlNCUFJidmtHdTlWMnk3NlVCNm9WSXFXKzNWejZDR3RMOFZkM0hMSlV3aC85?=
 =?utf-8?B?ZjJ3QndNLzdhdXN3YXNZbjkyb0pMZlFVQXhQRVgveG51OHUzQkR4Rk1zaFBH?=
 =?utf-8?B?ZkJuVlVJYWM5Z2lmdGVIWjFwZW40NWNtSUpyc3JYSWJCLzhsWnpnQWpLY1BE?=
 =?utf-8?B?dE5JQ1VEdFNpRnhSb2xFZHM1KzJHUlZaVjlOV0dHQnh4QWxVTDNFWFo3c3JZ?=
 =?utf-8?B?RVA2RUJybm5UczdCNEduRWZrN3h2bDVZTUJpS3pNMG52YmQ2c2k4elhqd2N0?=
 =?utf-8?B?aHdRTStOS2xiQjRPY3NDL1ptdmd4L2JQaU9HYmdLc0ZLUUk3eloxTExSUEpN?=
 =?utf-8?B?ZWF2MWpiWHFRNjIvQ0pyS2RhenBYYmxWdkFXSDlTZkdycGpUQWRxcWprVUQr?=
 =?utf-8?B?RlJiRThtSWRXVGVGSkFSVGhaM2FjS05JM0pCWmpjVGoyQjhiRmNOeHhKTk0w?=
 =?utf-8?B?MHNjNEtJT0g0NjN0T3J4NHdua2NMV2hsV1RPM1k3N3VoYlRQMmFldXZkR1px?=
 =?utf-8?B?U09CTGdQTkd2YktpUHk1RnBTUUtMZjhEcndERmVEQ3F3N0lMRjVSd1Z3VTh3?=
 =?utf-8?B?QkR3UUFtY2xFcmVRS29nY3FBZXdydXBIWkYvVVM5YVZNei9vbUVYc0k0NTND?=
 =?utf-8?B?bFNvUER2a0pycmxLZzk3b3pxTE95ekZHVjQwTHlyYmRmT1ZFREhWZjVmWW1R?=
 =?utf-8?B?RXg3bFRFbmFTaHJKc3VrUmI2Rzk2em9iRUdvekxydy9uVnFqcmk4elYweWNU?=
 =?utf-8?B?bUZJeEpzMkZSWHdmQk5GRWh3bUt0Q0YyTGdkWEhMblQvZzFZQTF0cS9hWVpq?=
 =?utf-8?B?MkdiTElXaVB5MWNoWXVnNWlHOU9HejQyWElDVHhoWFFHbFpoYllGQm1zdUN6?=
 =?utf-8?B?VHQyR05GVTRrVTlsbUhoY1RRT0JRUFJPazF5a3I5dUc0K0I0TituWmxEYzg2?=
 =?utf-8?B?aWoxN3NQcGRKUkRSOVh2VVBMTVRma2krbEEwSUlWL3pGM2hkWVQ3dTBOTzRx?=
 =?utf-8?B?R2QvZzdReHBSYVhaTHRTdkpLaGxuQXpWUFBDTmdTRzlOanJWN2FnUktnYWg5?=
 =?utf-8?B?SVJ3aVMyOG9ndjRZMjlsVWd0enMxNlQrbUxyYjRLcGk0UWRIWStaYzY1Y0hL?=
 =?utf-8?B?ekJ0dlFpNXRzSzY0cWc3ZnE1aU9sSjlBL3hiQmZLRzl5UHFYMkNWN2hRRG41?=
 =?utf-8?B?amxkZ2JzcnltWG9LSFNUbEt3MXJxcjNZUlgxQmZMR1ZrSElsV0NpMUFacWFK?=
 =?utf-8?B?YzhjUUdFeXo0S3A3NVdEUkIzNHhMR1loZ080RGZTcm1NVEtkK3FKTWRZY3dQ?=
 =?utf-8?B?U0twUTVVbUwvNHBIUTh3NUwwbzJ2WFdpMUdacFQzWDRXZXJqWFFxSm51YnRE?=
 =?utf-8?B?d2ppdXE2M0dXY3hZckdmVGZMaGM4S3V2QWlzdzJQQnNOb0p3bFhhVE5ZUldO?=
 =?utf-8?Q?Aww5mL5EdvvWVtQeK/uEsC4Nm0gKK0E9lTtUDmr?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 84d8ae59-a21f-42cf-f3b7-08d981c10256
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 14:13:42.1087
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4o8y7hq5ydgyfea0QUIQvlyuGyIC/tJOiIQiyDGdYn4zi5pcPa1aFDs+LLZoyQBe49xFMJbtsT6YzrkRYoLmfIMLRbZpkbo0uOQBYTyidAo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3496
X-OriginatorOrg: citrix.com

On 27/09/2021 11:48, Juergen Gross wrote:
> Add a configuration item for the maximum number of open file
> descriptors xenstored should be allowed to have.
>
> The default should be "unlimited" in order not to restrict xenstored
> in the number of domains it can support, but unfortunately the prlimit
> command requires specification of a real value for the number of files,
> so use 262144 as the default value.

Citation needed.

prlimit -nunlimited

prlimit --nofile=3Dunlimited

both work fine, and strace confirms they issue correct system calls.

Support for "unlimited" as a parameter has existed for the entire
lifetime of the utility,
https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e219dbc=
f62ec5ede84

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:17:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196933.349839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrRm-0002SV-U2; Mon, 27 Sep 2021 14:17:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196933.349839; Mon, 27 Sep 2021 14:17:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrRm-0002SO-Qs; Mon, 27 Sep 2021 14:17:34 +0000
Received: by outflank-mailman (input) for mailman id 196933;
 Mon, 27 Sep 2021 14:17:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=xtCR=OR=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mUrRl-0002SI-Hv
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:17:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 76bce6eb-f72b-4bf5-a924-21dcd9f9df83;
 Mon, 27 Sep 2021 14:17:32 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-H8Q34x9PODy28DvvTqhCNQ-1; Mon, 27 Sep 2021 16:17:30 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2957.eurprd04.prod.outlook.com (2603:10a6:802:4::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 14:17:28 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 14:17:28 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P192CA0010.EURP192.PROD.OUTLOOK.COM (2603:10a6:102:56::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 14:17:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76bce6eb-f72b-4bf5-a924-21dcd9f9df83
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632752251;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/q5MK7EPL24fpKUHgKon7EQFXJWgnC9xdYrW+2UuBKc=;
	b=MCS/odGUBiNZyQkxUg2qw5Q6qpTLBTdY8ct5I3GDwFVHfS5A1wjCsAXGGqNxXQY3WVsKOO
	ZDoJcTx1YvpXOtMKZEOoLoVU9MuT7Uzvp2yrwnOnmUgYFn8HEwOe95bbjCjQdB7kwQldDa
	bPSG5TMOF0IpTFYZJpXFOj/S0+sAOwQ=
X-MC-Unique: H8Q34x9PODy28DvvTqhCNQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ik222J2n8WKOrtkQq9c68GNpUM1h3450cFk5/zkE8Wb6riKi29jOQpm+jDXoJrp5ftpXgGVngOOLBopS2J7PK466zeXokfKbA1Y6XKVApXaPgOgGiVcS5Nzzs2baRol+1flww1NsIhCEseZ/x8t/L3XjjrvoqKdK4Lj4sHT31o9djBnANJ5OJSrFpzmOD3D2M6vy8bdAeEm1mV3Pe6lkJcIzjvSxLuhX9nCYac3sAtNI5xztgfWNeJ0pCe1XWx8Mjg7Uzl0pMgV+5qZzkG3p2mkWvP+H5Aw66eGELf9hD30R1LUcQKUB0+kts00gRY0+zwgCEgamZ19SM4X1ZWW6lQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/q5MK7EPL24fpKUHgKon7EQFXJWgnC9xdYrW+2UuBKc=;
 b=SCzg+hali+lSRSuRIruz9atVOxXjI3pEKrWspDbu5PoJK54vJEwCVxDXD/93NaNgOXHGZhNCUeDptzOKBOGVU3HyHAkkNgG+0ByWSh1gWlNRSgHFPPu3NK9xy+xUMJGRGWgXFMVuDABytmI77NfWeDe1vUcF7NqMRCqRrFZiPHy0xFw+D4vuhE5+VxsqXuHV6vMV0yfsva0O8BQ8nKU0//ILWQLfe1omd2wK5LguCdC23NxNqegTsiS2goOnaHdA1s0Wn5hEfy6LXvQDuzqcEuVVFkn8pcLEPvJIbYVqWi5RE4sLhhKge0Nw1WLp9umwMzGHuBemBTyF3Eg1teWBGA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
 <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
 <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
 <299f330a-3a99-e87a-d182-762247371e3d@suse.com>
 <e0ee94e4-4076-ac51-f5a8-d895220d1522@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <67bf6cff-5e1f-0770-f212-b6bc0d82838e@suse.com>
Date: Mon, 27 Sep 2021 16:16:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <e0ee94e4-4076-ac51-f5a8-d895220d1522@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P192CA0010.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:102:56::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7ebd5346-1300-435c-b2d2-08d981c18921
X-MS-TrafficTypeDiagnostic: VI1PR04MB2957:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2957FF930EA1235A0DE94AFAB3A79@VI1PR04MB2957.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	c8t/WbZTnSh/4RBA1TZc1h6jn7BdM0qDMXmfCBIQsyXsypuY3AWEs2kc8DFILAKVDlAcsATihOCw5WU9jfqMIkwLniuyGd89S8tw8QAp7I3aMP2UU5KFolf9K+sopEY7weEWO3s56R7bCV4HA+7ru5DxT/lMG4MfMEZomhmey3PjuClxO6VWkhHlzO/WqXEbwsngUW0pgQyCLV731dHAtt8Xv2VzpGZ04Y/iLcEnlw//MsK0iU/jD1PgRh2NmCPjYwNp8UcP5Z2Bf+U5nL8QND+ag+KIug2Ped0yF9qjxNYQr5ikQ2/K9wypRgKprCtYp8Zdw051GzPIEZDwsx8Hr+9r8a68OPCoxQ6E74DANo5IlvS4pVt+xqr1d5MWxtXcECVSaWCakGyIqURXkXrwUdRDTvy+l0eF56jldEHi0T4vuByK7+VduLyEbDLYvXciBt7GawzhK77zW6JZV2PTZ8IJQlxWDzVoV6qIcXxKrMg8XzGmjz68ESzt8BPYRkRx4ajNJOm3EHr8zeVRg8toEtx18bhkq7FPAIGUMaqcrIzXx/NEKU6wlCZktrhgWxv0FmRzgiw8lTqtv8Lj0UDHFgbvthKjvASIB+u7PuIEa6EbauhdIvz6Z7P5BtfzcXYY3vtcsuhde3ZJJXn0n0QMJcyq6FqFKNySE+6pyRwaqLOgTzZgUpmWU94NH0zTb2d1Ios6Fgx19P80mSWAPbPijLoQgLB5cr3+GICz1Z29BY009Eb74XHypIfnO46/buNj
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(508600001)(2906002)(53546011)(8676002)(186003)(36756003)(8936002)(31696002)(54906003)(956004)(6916009)(6486002)(38100700002)(316002)(16576012)(2616005)(83380400001)(6666004)(66476007)(5660300002)(31686004)(4326008)(66946007)(86362001)(26005)(7416002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UVRuT0VYem9BMjZZNC9XZ3BTMUhoQkVVRlZMUk1id002VU9sTjA1Ny9EWFdL?=
 =?utf-8?B?UWlKblFQVUw1OUZUREpjd3F5cFhTT1U0RDlldUtDeDBib0V0K1RlcVNzaGpv?=
 =?utf-8?B?MjBCVDBnZFpGUTEzelhHcDhmQU9CREdyVVZjSHdGOTBzS2pWc1hrRHF4YXds?=
 =?utf-8?B?aTM1RGtkMjFSNzgybXZNbkw5RkF3d29Yc0ZBQlNxRkxKSjZHL2xOTXB5djVR?=
 =?utf-8?B?b3UvM1hpWHJwUGYyZWo5cTMvSlBMTTh2UjJwS05lMDM0aXBkY2k2SVVqWUxm?=
 =?utf-8?B?azNucTBmL25lcnNDMGltMjJvbEQ3ZEdJZGQycDRtYVFTUHJYam95UzB0Rmk1?=
 =?utf-8?B?T2lvWE45NGxHR28yZ2NGVTkzc1VjZ1hHUmw2VVRJRHNoY1dSRG9qc1lpTmFN?=
 =?utf-8?B?STlvTUZ0ZU9sY2Jqbk5hbzNVWk1qa1BYOWllODduSUloUE1IZVNUZE9xeUJ1?=
 =?utf-8?B?VE8veXJPSE9Tendyb1EvT0tIc3VUZkJVczZ3b3hNZzgzVzBpM25iaklQY0hn?=
 =?utf-8?B?UWU2Tml3TlBqY3ZQeDAvQ292eXA0bTFSMUJFUFBxc0FXVFZOajVuNjdkYjQr?=
 =?utf-8?B?ZkZiZzkvYVV3bmJTT0xFV3hTMDR2ci9NVVFMM3VQWmQ0MHovUkhUb05SUVR1?=
 =?utf-8?B?Nloza0d3ZStidFhrM3V2RCtyVDduUU1SalJkZEZ3dSsxb1VGREkwZ3ZFNjcv?=
 =?utf-8?B?ZmpqN05hYm9KQzR6SGhJQVNCUUdNMEd5c21QdnlHZnZocU9QZzZZdGpCQ01m?=
 =?utf-8?B?SFlLcTIzRXYrSWR0d0x1SXhuYjVwc0tVa1RmV29Nb0lDSEF4Q1JMTU1BSFJN?=
 =?utf-8?B?aUw0dmY4dnNTMTJwcWZMMUxabHJlUTlJVzVkVXRtREd2d1Yra2M1K0R1VmxU?=
 =?utf-8?B?dVNnTm9PVU9TMDlEaTl4UExiRmltTzVNT1doQmZpUkNDOUtlRWRlKzRubDc0?=
 =?utf-8?B?QUVKUGp5cW44RG9uSkVkZ0crZ2ZnSGxxbytWTmJJTjNrQ0h6QTJNTnVMVEt4?=
 =?utf-8?B?TDVRK2tyNnQ1dUpKTUh6emJwdWl3TEN3clQwNzcwTjd0SS9sU0U1ZnpocWhY?=
 =?utf-8?B?THlsU2luZUYwWFdRWmVWRDhUc0hqeElsdmJFc3Q4bkExMGFZN0ptY1pFbnA4?=
 =?utf-8?B?ZEpOTFZVako1OEpFQ3JLQ2NNQmg2dVlLNnQzWEprQy9WWFp3amI4c2gvN0JR?=
 =?utf-8?B?RFp5aTZqZlJqdUYyN0NOTEtCaWRrSkh5RDIvb1lzcGZ0WlNyN04vN0Z2QkNN?=
 =?utf-8?B?MmY2aDB1MHo0RU53YzRPeDZoZUlQT0tHc0pjSnhTNitiY2N2SmpOeWtBQnBT?=
 =?utf-8?B?eFlNajcwem5tMFVIS2dZWnJMM3RoSmNpUjN5MjBOa29IczFyUHk3WnBJeSt4?=
 =?utf-8?B?djhTcC8rK0tBcXZzUUlTdUduRUx0NVl6UzV6SVd0SUFXdnMrU1FvZ3RNbUV6?=
 =?utf-8?B?bkNneThZU2crb2hVZG9JWHloMUJzT1kzUW5yL3pMSVArZnozVFFVK0xLVWd2?=
 =?utf-8?B?RURvM2tURERBazVDNlc2MVFLUGxnQlk1dUlLL3RuZjNVUnl6cEh2cVZXK0pJ?=
 =?utf-8?B?RllxRmRpZjAyVUpyTnNwTGdtdjIrbUt5SklrbVFKNldqK2tOaDJsaGkxcUU0?=
 =?utf-8?B?VkREKzhoZlAreUw1RXcvanVGVXU3N0hXWWFlSHliRGhxamRsZVRscTBKUUlO?=
 =?utf-8?B?OXRNaHlNSVRDNXp0SURlT05yMXRLbEJKbVA4VE1JUXFidVRmK1VheXloM3Vn?=
 =?utf-8?Q?gDpITFQaZlSJs7xRfeuCUnCEUNY4LI/QUWeOWFX?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7ebd5346-1300-435c-b2d2-08d981c18921
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 14:17:28.0173
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: stVNI6L1RmoQns9FnQ38lTL8uhTSqt4+Bq2RsAl8mIvuosyZZSsaCnxLyauqve/SHAYKCTWuEHWiBoRhRb6/FQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2957

On 27.09.2021 16:04, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 16:51, Jan Beulich wrote:
>> On 27.09.2021 15:43, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 16:34, Jan Beulich wrote:
>>>> On 27.09.2021 14:08, Oleksandr Andrushchenko wrote:
>>>>> On 27.09.21 14:31, Jan Beulich wrote:
>>>>>> On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
>>>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>>>> @@ -890,6 +890,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>>>>>>         return 0;
>>>>>>>     }
>>>>>>>     
>>>>>>> +/*
>>>>>>> + * Find the physical device which is mapped to the virtual device
>>>>>>> + * and translate virtual SBDF to the physical one.
>>>>>>> + */
>>>>>>> +bool pci_translate_virtual_device(struct vcpu *v, pci_sbdf_t *sbdf)
>>>>>> Why struct vcpu, when you only need ...
>>>>>>
>>>>>>> +{
>>>>>>> +    struct domain *d = v->domain;
>>>>>> ... this? It's also not really logical for this function to take a
>>>>>> struct vcpu, as the translation should be uniform within a domain.
>>>>> Agree, struct domain is just enough
>>>>>> Also - const please (as said elsewhere before, ideally wherever possible
>>>>>> and sensible).
>>>>> Ok
>>>>>>> +    struct vpci_dev *vdev;
>>>>>>> +    bool found = false;
>>>>>>> +
>>>>>>> +    pcidevs_lock();
>>>>>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>>>>>> +    {
>>>>>>> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
>>>>>>> +        {
>>>>>>> +            /* Replace virtual SBDF with the physical one. */
>>>>>>> +            *sbdf = vdev->pdev->sbdf;
>>>>>>> +            found = true;
>>>>>>> +            break;
>>>>>>> +        }
>>>>>>> +    }
>>>>>> For a DomU with just one or at most a couple of devices, such a brute
>>>>>> force lookup may be fine. What about Dom0 though? The physical topology
>>>>>> gets split at the segment level, so maybe this would by a reasonable
>>>>>> granularity here as well?
>>>>> Not sure I am following why topology matters here. We are just trying to
>>>>> match one SBDF (as seen by the guest) to other SBDF (physical,
>>>>> as seen by Dom0), so we can proxy DomU's configuration space access
>>>>> to the proper device in Dom0.
>>>> Topology here matters only in so far as I've suggested to have separate
>>>> lists per segment, to reduce look times. Other methods of avoiding a
>>>> fully linear search are of course possible as well.
>>> Ah, with that that respect then of course. But let's be realistic.
>>> How many PCI devices are normally passed through to a guest?
>>> I can assume this is probably less than 10 most of the time.
>>> By assuming that the number of devices is small I see no profit,
>>> but unneeded complexity in accounting virtual devices per segment
>>> and performing the relevant lookup. So, I would go with a single list
>>> and "brute force lookup" unless it is clearly seen that this needs to be
>>> optimized.
>>
>> Just to repeat my initial reply: "For a DomU with just one or at most
>> a couple of devices, such a brute force lookup may be fine. What about
>> Dom0 though?" If the code uses the simpler form because it's only
>> going to be used for DomU, then that's fine for now. But such latent
>> issues will want recording - e.g. by TODO comments or at the very
>> least suitable pointing out in the description.
> 
> As we do not emulate virtual bus topology for Dom0 then it is
> 
> clearly seen that the code may only have impact on DomUs.
> 
> But anyways, virtual bus topology for DomUs is emulated with
> 
> a single segment 0. We have a single list of virtual SBDFs,
> 
> again, for virtual segment 0, which maps those virtual SBDFs
> 
> to physical SBDFs. So, we go over the list of virtual devices
> 
> assigned to that guest and match the virtual SBDF in question to
> 
> its counterpart in Dom0. I can't see how this can be optimized or
> 
> needs that optimization because of the fact that Dom0 may have
> 
> multiple segments...
> 
> So, how would that comment look like?

Well - if the plan is that this code would never be used for Dom0,
then all is fine as is, I guess. But as you can see the Dom0 plans
on Arm wrt vPCI weren't clear to me at all.

Jan



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:20:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:20:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196941.349850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrUz-0003tv-Gi; Mon, 27 Sep 2021 14:20:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196941.349850; Mon, 27 Sep 2021 14:20:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrUz-0003to-Dh; Mon, 27 Sep 2021 14:20:53 +0000
Received: by outflank-mailman (input) for mailman id 196941;
 Mon, 27 Sep 2021 14:20:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M1Nw=OR=epam.com=prvs=9904516479=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mUrUz-0003ti-1v
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:20:53 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1e8897d4-1f9e-11ec-bc4d-12813bfff9fa;
 Mon, 27 Sep 2021 14:20:52 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18RDvol1014855; 
 Mon, 27 Sep 2021 14:20:48 GMT
Received: from eur02-am5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2052.outbound.protection.outlook.com [104.47.4.52])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbf3hg6tt-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 14:20:47 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7074.eurprd03.prod.outlook.com (2603:10a6:20b:2dc::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 14:20:44 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 14:20:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e8897d4-1f9e-11ec-bc4d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ecy+SwcG2eZsQDltZKSXT8hNdM7Qhvjfk3xs2RTnYCoFeZA22oRUf1FjeUnDTg3SJFqboi3X0nKn+jurgvVzQtGwfOUEhgWakKLsPWiWNm0WFccN49umJiZzmFdGowLuwo9TpDhK7ANXB3jxq/KZdM/5gtGFFU/kjo81TTHZS0M67sdxhqFYe2IlGoeni0dAbR9UwHhnHiKmHDBBeZykvdgjfK3bed2miys7AkjMjBORpJepf3z/XMKKx7jGFM8HHu34L8WQf2rSuuwrlueFMs7nQ3CWedDYlvtqy7iJ9ol/7WgoxPMbiDqEs7VcRTSy8CZYHYCLH9goAxI/LXkJgQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wXAQV8+kQ/itzK45s1YY/EqBxz8tyUcP1XPq5UjuqVc=;
 b=OkZGVXrvU4ikVJkWatiDcXCXXBufCVRUHcWj9Is7Pvbkflg0xPS5yP42KgR889kC0pmi8pTL/L3aZrjouBSE0l4HSifBy4Rlq4i5xdJbMTv+7CgNXaxlZwyWQkqeLFO/x/SgPYvh31LawfSQHTv00sBAurSsa+4OHRNDvT8XQlhdtVYPBEgD/6gx2dsm56aVzyU5Fmb9cJ647NJjgCOHZJjB5984Z6uzohKohzdjfRCAzEH8JVQUuxJsTzet1L+Nbvfb9JfxJ4p6q0WWmNwRxAu8yOyak0iZPXGkqvZTCb1T0ELpWG+E7KWNlD9TTJ8NFGQ6K/YM28uRNdPHN3VmUQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wXAQV8+kQ/itzK45s1YY/EqBxz8tyUcP1XPq5UjuqVc=;
 b=klR/8SG1RxJVBeP++EXA8GLjBIHoAc769Qq0HNOXa7AVruyTOom/qhMILxhdrpWZrW+0jJrV8HhWRI8yCcdWLm5NSpj1nb461CuetBdcHE1sn8DQ1bKRyKq5KBF6div4g3caQ9VrZQgFdKmpi8jipoYujBeiCTxoADLUDooY8gDRdCBLTzTboINcjLNdgXCAlEpvwAIf9qzkkp32vbv6zzo0d7NHe8lc73CweO5C8/AVqndY46Gs9keaREfeD/KOgF2tEA/FKvapblIwAWgXzDk8xEUZ3xpKKc/igw8R/zWUpV0PoCdEejJD49T1h9UPNrwYj1VXZn6pIXOqj1zUfw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v2 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: 
 AQHXsHpEqeYVqdD8fU+X35KPuXAnA6u3xY6AgAAKMQCAABgZgIAAAowAgAACGoCAAAOoAIAAA10AgAABPwA=
Date: Mon, 27 Sep 2021 14:20:44 +0000
Message-ID: <e2c8947a-8389-320e-f896-1a7254bb189e@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-12-andr2000@gmail.com>
 <df675ecd-3774-4b5e-241b-0a68e36c99f9@suse.com>
 <d08a9d34-5f75-f266-f8fb-95594fb0a8b0@epam.com>
 <3dcba7a0-6984-a9bf-6121-5003bbc2ccaa@suse.com>
 <27378eb1-7dad-29a4-a136-c9e5e55a2067@epam.com>
 <299f330a-3a99-e87a-d182-762247371e3d@suse.com>
 <e0ee94e4-4076-ac51-f5a8-d895220d1522@epam.com>
 <67bf6cff-5e1f-0770-f212-b6bc0d82838e@suse.com>
In-Reply-To: <67bf6cff-5e1f-0770-f212-b6bc0d82838e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 61fb08e7-ba6a-4343-8146-08d981c1fe69
x-ms-traffictypediagnostic: AM9PR03MB7074:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB7074FB8E68CBDE61A45DF1D2E7A79@AM9PR03MB7074.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 /e1oM/wWdRqe+JpEKqWPzWod8usD87nqSrv3zbhFBJr5AkTaWBFaD0mYcxjWwx9A0rbtRjKL9vV2h54oUWwirpquKEP5Fqjsh+fwHa7rYWvRC6FRJQ7ouPvsEhtjiOJ77ZEodgXz0lbpGOW7hhXSUtEIbunQB1uWXWKTWkd5tyvQJ36bOeJe4mk5YAEEaYK6LJbGjWDk8ejNaavxDUeVsCiSS8H/nSqDw5BdGwg4eJ167J3zU6mx4ob+VqesUdeo7jQQJ+13obd7oP59fScyctuBYk8OwYiZC+CYxYQcbqsMAb9Pk1yp6CiE+3C+ksrsLx0UdHe/YEujVO7g//nMbI16Y80/aP/qR72FneEtVVL3tL3ld0GfYJGiu4F4T0Be0utzIShU6wCfi0Dcl0kEDds8b17Ddi0pLSRpjVB2CPVk5qOEQ51SyN+/por1xyPsEMgtgpVg/ZyX/J0IfP936oSYSZFv9/L6c6ptY7a+MrziFkMNmKQCvEsWAqQEPNG43p39VGLEL9heBnIUATgkbDrRak2JHdDuwYbNv65DnHD6kywm4/cuJLVOkzwiSHrZ40PJQzBCvfKThMqy6TfVny2VhfewoIAxgQf+tB9Z1Ou7P6X16QzCdqeZesp1rDQaC6SU/NEQJ2Z9KI9Xy+2DtAt0xfgZdh9zf6FeJtxPh+/2kMPQ90I3a03oak1zL5IVAz6DIZWrTDuz2CQh1zsw8wEFPjiB1Owf4awJ7oe+a097yk+5V3DQBXUIMJbrJD2K
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(6512007)(66446008)(316002)(64756008)(86362001)(122000001)(76116006)(36756003)(91956017)(6486002)(66946007)(31696002)(6506007)(107886003)(66556008)(83380400001)(8676002)(4326008)(38070700005)(8936002)(66476007)(38100700002)(71200400001)(186003)(55236004)(31686004)(26005)(53546011)(6916009)(2906002)(508600001)(2616005)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?U3ZDSXRoNXVvSktmaTJGMEp5cEVoWHVId1NRbkx1bktnM2ExWmJ2Mk03c3Bk?=
 =?utf-8?B?aFBiRlc3M0JhY0dwb1F1dFNCWWJCZXZSa1RZRlJpZXV3bkdxUjc0Y0FIM09r?=
 =?utf-8?B?Z3ZZSWVRM1NaWDB4QTR1TVBxZGNvRDJsTVBzdGpBSzAzcVN0OFdibzI4N1g2?=
 =?utf-8?B?TEkxWFp3VC9kVTJwbm03VXRneTI5UUxDSEtkTEJjTlQ3eFFuNW4ybHZLajVE?=
 =?utf-8?B?OWRxNWlxdGp0VS9MT2t5bUtYdXU0UGxuM3V4MWZ2NkQzUWRUMkhweGdSbXMy?=
 =?utf-8?B?cWNuckdvTXVsT01QTjRYQjRvOTJQZ1JhN3hOL2hkcFRqY2xNV0RadnBPaUJJ?=
 =?utf-8?B?VVducEpLVm1pb3pPNXRLS2s3RTRaam1ZdndQTFVtSWU0VDRYRGx4OWJmTElE?=
 =?utf-8?B?cEROQXJqVVluSDc2d2NPVmNvWTNBQ3BLeTNidGNQWnhqYndBNy93UTByQS9x?=
 =?utf-8?B?Y0YwV1JpTXhoVWczL3U2RFJIRDF2bDUxOXF2RW03bVNhSGhCNWYxNGlNelUw?=
 =?utf-8?B?TmpLMXMxUmtRWjRmR3kwUXZaK1hpODJtUDBEV2ZzcnorMHY5NmxsRUYzSHRu?=
 =?utf-8?B?Q2JyM2VhK05MZkZvLzZINzg2UWhuYStwWkhQNVVYaXV5M3U0NjJOTXc0eXVl?=
 =?utf-8?B?cFRJQkQ0bEZyTUd2NFJSY09VZ250WVdESVAvbjk3Z3VlZVNuSU5zVDh1QVVi?=
 =?utf-8?B?U294eCtuL1FSVTNFQTdvY3NPazFxWFJPSjdIdUtJRksvZGpBaEJmNkJLK1lR?=
 =?utf-8?B?OEtQZTlOcXV5Z29qWWFmQVVSY1drVlpDRnRKT1pjOVdoYnZxd2REOGNzWFBm?=
 =?utf-8?B?ekZGd0RkNmVZUjlJbzlaQzhoWFNjd2R1Sy85ZUdYMzc0RnFKZ1RXWk5SSEhH?=
 =?utf-8?B?ajhVWXRxSjludU1vNzNHUzZQVUg4ci9seTc1SHJqQ2lrdm5yUWdFWjJzTUJN?=
 =?utf-8?B?dXlUeDdmUVkyK0pISm5scno5czk0d2E2WXlwb1NnajMxUy83TnBVS0gyVFZw?=
 =?utf-8?B?OGpkRGNuTDAxVzExRXV5ckFMcG4rMmtWOG5MYmJPMzlhdDlMSW91a0NLYVZX?=
 =?utf-8?B?TFJRZVdnakdwZ1hQczg4RXYxd211b2t4ZXUyK0l2VllVdEdFaEtYYkRyWnhH?=
 =?utf-8?B?RnNpL0lSM1VYZXpiMyt1NFpUVmpCUFREWjZBajZzbXA3OUhkcVpFV2gvWENT?=
 =?utf-8?B?cnU5c0VkV1Y1TFFMMndOb0RPelRTcHpGcUFSYVlzQmZmWmhuNFBlT3pPazRF?=
 =?utf-8?B?Qnpja3R2eHAzWDk4eGdXVE5tM1plaHZXWDVHeitvc1hEeUNWUi9Vdml5cmpU?=
 =?utf-8?B?TktreVdNNkhFNndabmRoYlJBT05uUUZMVE1ydHcxeEZzaVUxNFovSTBRVUZ0?=
 =?utf-8?B?bGxTVUxRMVorY0xxUTRIelNrd1htcmo3a1hnOG1iS0R4WVp0VWNaZ2xZSExO?=
 =?utf-8?B?U3RhZGlRMjk4YnhuRUphUWFtenc4QmFKbGtSeHVFVWhNOTZLNndaTlRYS1lH?=
 =?utf-8?B?YlphY25xNFJMMHNLWTdDRXlFSVhPdjFoWmNud3BUbCsvV1BScnphbDluNHdW?=
 =?utf-8?B?VmM4N2t5bWxReG1JM3UxY3lHREQyaTB2RFprbHNXOTVCTzAyREVzNzRsSFo1?=
 =?utf-8?B?OHEwamdiWkJmV1czaHZEK000cHZDMGxTZGtqbkwxWU5zd2tpL2hSeitWSEV3?=
 =?utf-8?B?d2hudUxabmR2VHdSaUFLRFh1elA1OFErWkdsRElTNGl4RHV2UUdwSUhMekd4?=
 =?utf-8?B?RDJsSlNZdHA3M0JBZjF5UUZCMzlkdWgvTDdkUE5EdjJsYzBJbisxejU4SG1u?=
 =?utf-8?B?K3BJb1EyUUlUdW5GQ0x6dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <1B6DD71F2D68ED44BCD240159EC379D1@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 61fb08e7-ba6a-4343-8146-08d981c1fe69
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 14:20:44.5708
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: GobgtISrZoPIcFlc1HvXYXWaVmXTiwnlq08zYW4GH/t4MpbhX/V4vHA/jtFY8kfagXeDgGcIR5HAvG1E2NUIM2apYZIDyUisyh5FZqA3pN71Syfsw0sSI7/Ky7IUexr6
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7074
X-Proofpoint-GUID: 9JznE2oD4tko5epW91P-3ObeCvhRyUWe
X-Proofpoint-ORIG-GUID: 9JznE2oD4tko5epW91P-3ObeCvhRyUWe
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_06,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0
 lowpriorityscore=0 priorityscore=1501 spamscore=0 clxscore=1015 mlxscore=0
 impostorscore=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270098

DQpPbiAyNy4wOS4yMSAxNzoxNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTY6MDQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTY6
NTEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIwMjEgMTU6NDMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxNjozNCwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMDIxIDE0OjA4LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAyNy4wOS4yMSAxNDozMSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+Pj4+
Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+Pj4+PiBAQCAtODkw
LDYgKzg5MCwzMSBAQCBpbnQgcGNpX3JlbW92ZV92aXJ0dWFsX2RldmljZShzdHJ1Y3QgZG9tYWlu
ICpkLCBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4+Pj4+ICAgICAgICAgIHJldHVy
biAwOw0KPj4+Pj4+Pj4gICAgICB9DQo+Pj4+Pj4+PiAgICAgIA0KPj4+Pj4+Pj4gKy8qDQo+Pj4+
Pj4+PiArICogRmluZCB0aGUgcGh5c2ljYWwgZGV2aWNlIHdoaWNoIGlzIG1hcHBlZCB0byB0aGUg
dmlydHVhbCBkZXZpY2UNCj4+Pj4+Pj4+ICsgKiBhbmQgdHJhbnNsYXRlIHZpcnR1YWwgU0JERiB0
byB0aGUgcGh5c2ljYWwgb25lLg0KPj4+Pj4+Pj4gKyAqLw0KPj4+Pj4+Pj4gK2Jvb2wgcGNpX3Ry
YW5zbGF0ZV92aXJ0dWFsX2RldmljZShzdHJ1Y3QgdmNwdSAqdiwgcGNpX3NiZGZfdCAqc2JkZikN
Cj4+Pj4+Pj4gV2h5IHN0cnVjdCB2Y3B1LCB3aGVuIHlvdSBvbmx5IG5lZWQgLi4uDQo+Pj4+Pj4+
DQo+Pj4+Pj4+PiArew0KPj4+Pj4+Pj4gKyAgICBzdHJ1Y3QgZG9tYWluICpkID0gdi0+ZG9tYWlu
Ow0KPj4+Pj4+PiAuLi4gdGhpcz8gSXQncyBhbHNvIG5vdCByZWFsbHkgbG9naWNhbCBmb3IgdGhp
cyBmdW5jdGlvbiB0byB0YWtlIGENCj4+Pj4+Pj4gc3RydWN0IHZjcHUsIGFzIHRoZSB0cmFuc2xh
dGlvbiBzaG91bGQgYmUgdW5pZm9ybSB3aXRoaW4gYSBkb21haW4uDQo+Pj4+Pj4gQWdyZWUsIHN0
cnVjdCBkb21haW4gaXMganVzdCBlbm91Z2gNCj4+Pj4+Pj4gQWxzbyAtIGNvbnN0IHBsZWFzZSAo
YXMgc2FpZCBlbHNld2hlcmUgYmVmb3JlLCBpZGVhbGx5IHdoZXJldmVyIHBvc3NpYmxlDQo+Pj4+
Pj4+IGFuZCBzZW5zaWJsZSkuDQo+Pj4+Pj4gT2sNCj4+Pj4+Pj4+ICsgICAgc3RydWN0IHZwY2lf
ZGV2ICp2ZGV2Ow0KPj4+Pj4+Pj4gKyAgICBib29sIGZvdW5kID0gZmFsc2U7DQo+Pj4+Pj4+PiAr
DQo+Pj4+Pj4+PiArICAgIHBjaWRldnNfbG9jaygpOw0KPj4+Pj4+Pj4gKyAgICBsaXN0X2Zvcl9l
YWNoX2VudHJ5ICggdmRldiwgJmQtPnZkZXZfbGlzdCwgbGlzdCApDQo+Pj4+Pj4+PiArICAgIHsN
Cj4+Pj4+Pj4+ICsgICAgICAgIGlmICggdmRldi0+c2JkZi5zYmRmID09IHNiZGYtPnNiZGYgKQ0K
Pj4+Pj4+Pj4gKyAgICAgICAgew0KPj4+Pj4+Pj4gKyAgICAgICAgICAgIC8qIFJlcGxhY2Ugdmly
dHVhbCBTQkRGIHdpdGggdGhlIHBoeXNpY2FsIG9uZS4gKi8NCj4+Pj4+Pj4+ICsgICAgICAgICAg
ICAqc2JkZiA9IHZkZXYtPnBkZXYtPnNiZGY7DQo+Pj4+Pj4+PiArICAgICAgICAgICAgZm91bmQg
PSB0cnVlOw0KPj4+Pj4+Pj4gKyAgICAgICAgICAgIGJyZWFrOw0KPj4+Pj4+Pj4gKyAgICAgICAg
fQ0KPj4+Pj4+Pj4gKyAgICB9DQo+Pj4+Pj4+IEZvciBhIERvbVUgd2l0aCBqdXN0IG9uZSBvciBh
dCBtb3N0IGEgY291cGxlIG9mIGRldmljZXMsIHN1Y2ggYSBicnV0ZQ0KPj4+Pj4+PiBmb3JjZSBs
b29rdXAgbWF5IGJlIGZpbmUuIFdoYXQgYWJvdXQgRG9tMCB0aG91Z2g/IFRoZSBwaHlzaWNhbCB0
b3BvbG9neQ0KPj4+Pj4+PiBnZXRzIHNwbGl0IGF0IHRoZSBzZWdtZW50IGxldmVsLCBzbyBtYXli
ZSB0aGlzIHdvdWxkIGJ5IGEgcmVhc29uYWJsZQ0KPj4+Pj4+PiBncmFudWxhcml0eSBoZXJlIGFz
IHdlbGw/DQo+Pj4+Pj4gTm90IHN1cmUgSSBhbSBmb2xsb3dpbmcgd2h5IHRvcG9sb2d5IG1hdHRl
cnMgaGVyZS4gV2UgYXJlIGp1c3QgdHJ5aW5nIHRvDQo+Pj4+Pj4gbWF0Y2ggb25lIFNCREYgKGFz
IHNlZW4gYnkgdGhlIGd1ZXN0KSB0byBvdGhlciBTQkRGIChwaHlzaWNhbCwNCj4+Pj4+PiBhcyBz
ZWVuIGJ5IERvbTApLCBzbyB3ZSBjYW4gcHJveHkgRG9tVSdzIGNvbmZpZ3VyYXRpb24gc3BhY2Ug
YWNjZXNzDQo+Pj4+Pj4gdG8gdGhlIHByb3BlciBkZXZpY2UgaW4gRG9tMC4NCj4+Pj4+IFRvcG9s
b2d5IGhlcmUgbWF0dGVycyBvbmx5IGluIHNvIGZhciBhcyBJJ3ZlIHN1Z2dlc3RlZCB0byBoYXZl
IHNlcGFyYXRlDQo+Pj4+PiBsaXN0cyBwZXIgc2VnbWVudCwgdG8gcmVkdWNlIGxvb2sgdGltZXMu
IE90aGVyIG1ldGhvZHMgb2YgYXZvaWRpbmcgYQ0KPj4+Pj4gZnVsbHkgbGluZWFyIHNlYXJjaCBh
cmUgb2YgY291cnNlIHBvc3NpYmxlIGFzIHdlbGwuDQo+Pj4+IEFoLCB3aXRoIHRoYXQgdGhhdCBy
ZXNwZWN0IHRoZW4gb2YgY291cnNlLiBCdXQgbGV0J3MgYmUgcmVhbGlzdGljLg0KPj4+PiBIb3cg
bWFueSBQQ0kgZGV2aWNlcyBhcmUgbm9ybWFsbHkgcGFzc2VkIHRocm91Z2ggdG8gYSBndWVzdD8N
Cj4+Pj4gSSBjYW4gYXNzdW1lIHRoaXMgaXMgcHJvYmFibHkgbGVzcyB0aGFuIDEwIG1vc3Qgb2Yg
dGhlIHRpbWUuDQo+Pj4+IEJ5IGFzc3VtaW5nIHRoYXQgdGhlIG51bWJlciBvZiBkZXZpY2VzIGlz
IHNtYWxsIEkgc2VlIG5vIHByb2ZpdCwNCj4+Pj4gYnV0IHVubmVlZGVkIGNvbXBsZXhpdHkgaW4g
YWNjb3VudGluZyB2aXJ0dWFsIGRldmljZXMgcGVyIHNlZ21lbnQNCj4+Pj4gYW5kIHBlcmZvcm1p
bmcgdGhlIHJlbGV2YW50IGxvb2t1cC4gU28sIEkgd291bGQgZ28gd2l0aCBhIHNpbmdsZSBsaXN0
DQo+Pj4+IGFuZCAiYnJ1dGUgZm9yY2UgbG9va3VwIiB1bmxlc3MgaXQgaXMgY2xlYXJseSBzZWVu
IHRoYXQgdGhpcyBuZWVkcyB0byBiZQ0KPj4+PiBvcHRpbWl6ZWQuDQo+Pj4gSnVzdCB0byByZXBl
YXQgbXkgaW5pdGlhbCByZXBseTogIkZvciBhIERvbVUgd2l0aCBqdXN0IG9uZSBvciBhdCBtb3N0
DQo+Pj4gYSBjb3VwbGUgb2YgZGV2aWNlcywgc3VjaCBhIGJydXRlIGZvcmNlIGxvb2t1cCBtYXkg
YmUgZmluZS4gV2hhdCBhYm91dA0KPj4+IERvbTAgdGhvdWdoPyIgSWYgdGhlIGNvZGUgdXNlcyB0
aGUgc2ltcGxlciBmb3JtIGJlY2F1c2UgaXQncyBvbmx5DQo+Pj4gZ29pbmcgdG8gYmUgdXNlZCBm
b3IgRG9tVSwgdGhlbiB0aGF0J3MgZmluZSBmb3Igbm93LiBCdXQgc3VjaCBsYXRlbnQNCj4+PiBp
c3N1ZXMgd2lsbCB3YW50IHJlY29yZGluZyAtIGUuZy4gYnkgVE9ETyBjb21tZW50cyBvciBhdCB0
aGUgdmVyeQ0KPj4+IGxlYXN0IHN1aXRhYmxlIHBvaW50aW5nIG91dCBpbiB0aGUgZGVzY3JpcHRp
b24uDQo+PiBBcyB3ZSBkbyBub3QgZW11bGF0ZSB2aXJ0dWFsIGJ1cyB0b3BvbG9neSBmb3IgRG9t
MCB0aGVuIGl0IGlzDQo+Pg0KPj4gY2xlYXJseSBzZWVuIHRoYXQgdGhlIGNvZGUgbWF5IG9ubHkg
aGF2ZSBpbXBhY3Qgb24gRG9tVXMuDQo+Pg0KPj4gQnV0IGFueXdheXMsIHZpcnR1YWwgYnVzIHRv
cG9sb2d5IGZvciBEb21VcyBpcyBlbXVsYXRlZCB3aXRoDQo+Pg0KPj4gYSBzaW5nbGUgc2VnbWVu
dCAwLiBXZSBoYXZlIGEgc2luZ2xlIGxpc3Qgb2YgdmlydHVhbCBTQkRGcywNCj4+DQo+PiBhZ2Fp
biwgZm9yIHZpcnR1YWwgc2VnbWVudCAwLCB3aGljaCBtYXBzIHRob3NlIHZpcnR1YWwgU0JERnMN
Cj4+DQo+PiB0byBwaHlzaWNhbCBTQkRGcy4gU28sIHdlIGdvIG92ZXIgdGhlIGxpc3Qgb2Ygdmly
dHVhbCBkZXZpY2VzDQo+Pg0KPj4gYXNzaWduZWQgdG8gdGhhdCBndWVzdCBhbmQgbWF0Y2ggdGhl
IHZpcnR1YWwgU0JERiBpbiBxdWVzdGlvbiB0bw0KPj4NCj4+IGl0cyBjb3VudGVycGFydCBpbiBE
b20wLiBJIGNhbid0IHNlZSBob3cgdGhpcyBjYW4gYmUgb3B0aW1pemVkIG9yDQo+Pg0KPj4gbmVl
ZHMgdGhhdCBvcHRpbWl6YXRpb24gYmVjYXVzZSBvZiB0aGUgZmFjdCB0aGF0IERvbTAgbWF5IGhh
dmUNCj4+DQo+PiBtdWx0aXBsZSBzZWdtZW50cy4uLg0KPj4NCj4+IFNvLCBob3cgd291bGQgdGhh
dCBjb21tZW50IGxvb2sgbGlrZT8NCj4gV2VsbCAtIGlmIHRoZSBwbGFuIGlzIHRoYXQgdGhpcyBj
b2RlIHdvdWxkIG5ldmVyIGJlIHVzZWQgZm9yIERvbTAsDQo+IHRoZW4gYWxsIGlzIGZpbmUgYXMg
aXMsIEkgZ3Vlc3MuIEJ1dCBhcyB5b3UgY2FuIHNlZSB0aGUgRG9tMCBwbGFucw0KPiBvbiBBcm0g
d3J0IHZQQ0kgd2VyZW4ndCBjbGVhciB0byBtZSBhdCBhbGwuDQoNCkl0IGlzIGFsbCBuZXcgdG8g
YWxsIG9mIHVzIDspIE5vIHByb2JsZW0uDQoNCg0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCg0KT2xl
a3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:24:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:24:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196949.349861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrYW-0004Y9-3W; Mon, 27 Sep 2021 14:24:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196949.349861; Mon, 27 Sep 2021 14:24:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrYV-0004Y2-Vk; Mon, 27 Sep 2021 14:24:31 +0000
Received: by outflank-mailman (input) for mailman id 196949;
 Mon, 27 Sep 2021 14:24:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUrYU-0004Xo-Lc
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:24:30 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a0057ea8-1f9e-11ec-bc4d-12813bfff9fa;
 Mon, 27 Sep 2021 14:24:29 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id E806F1FEAB;
 Mon, 27 Sep 2021 14:24:28 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C546013A91;
 Mon, 27 Sep 2021 14:24:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id d6p/LhzUUWHHNgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 14:24:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0057ea8-1f9e-11ec-bc4d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632752668; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=j+P+aYk43VBe0kn+iDMVZ4C/O64eAZ+0Vu/cA9/TBgQ=;
	b=FhJyaPpcknNEqJ4HZ4rugn0cseDfVcEfdwijVQwgpTcXAULkm26TSwLPSTwFlQnCeZgYUb
	O6JnmIlKp3WFWgQk2TLNDTb/uRaljcTv68/lb1mJdHRIBa7G6LyzdmOajaKnW3QywxGtts
	6y3fJX+Z2Vvl4DMOguXeakSTcL5fy5c=
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
Date: Mon, 27 Sep 2021 16:24:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="Jr57jY708Oh1UQ1qom055Trz8CNBNA5j2"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Jr57jY708Oh1UQ1qom055Trz8CNBNA5j2
Content-Type: multipart/mixed; boundary="aOrsLSMgnbM7WhEp3ynBSWhniBdHoFhfF";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
Message-ID: <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
In-Reply-To: <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>

--aOrsLSMgnbM7WhEp3ynBSWhniBdHoFhfF
Content-Type: multipart/mixed;
 boundary="------------3DD1FEE5FCE35A9D041F6383"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------3DD1FEE5FCE35A9D041F6383
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 16:13, Andrew Cooper wrote:
> On 27/09/2021 11:48, Juergen Gross wrote:
>> Add a configuration item for the maximum number of open file
>> descriptors xenstored should be allowed to have.
>>
>> The default should be "unlimited" in order not to restrict xenstored
>> in the number of domains it can support, but unfortunately the prlimit=

>> command requires specification of a real value for the number of files=
,
>> so use 262144 as the default value.
>=20
> Citation needed.
>=20
> prlimit -nunlimited
>=20
> prlimit --nofile=3Dunlimited
>=20
> both work fine, and strace confirms they issue correct system calls.

Not on my test system:

# prlimit --pid 734 --nofile=3Dunlimited
prlimit: failed to set the NOFILE resource limit: Operation not permitted=

# prlimit --pid 734 --nofile=3D262144
#

> Support for "unlimited" as a parameter has existed for the entire
> lifetime of the utility,
> https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e21=
9dbcf62ec5ede84

Yes, but not all systems seem to support raising the limit to
"unlimited".


Juergen

--------------3DD1FEE5FCE35A9D041F6383
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------3DD1FEE5FCE35A9D041F6383--

--aOrsLSMgnbM7WhEp3ynBSWhniBdHoFhfF--

--Jr57jY708Oh1UQ1qom055Trz8CNBNA5j2
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFR1BwFAwAAAAAACgkQsN6d1ii/Ey9C
zAf/dmuZepBr4HhlngzUraXbhwn2aEuLFL3o0Bl0ET249jTvZgy/q8zP6z+cnsYNS41rXgpG+aid
ULrlqEo7a7F7wFzBCbYT96poD0QcQnkgriPNVA5hIAPcV/MmGRCY+RbGf3129CAsm0J/e0hER4Xs
ys2VqunFYdJjWeLQPWjpejCzuFfBOYadjjR1+bj8y81m0xMyWE8rZ7/OPauhhfaNzIpQy3F9lRte
Q3OZmpyKYFJHyC8r3KE2QRr/Rd74xQNT6+qB8K++W+aphm0v1WIpQlnrPkwIAFXa7FArqUGiFxsD
JRjRUpd2+NCZQDf3579nap5RsqAhNgtUpO80e8fUiA==
=RDPg
-----END PGP SIGNATURE-----

--Jr57jY708Oh1UQ1qom055Trz8CNBNA5j2--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:24:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:24:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196951.349872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrYY-0004ox-CX; Mon, 27 Sep 2021 14:24:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196951.349872; Mon, 27 Sep 2021 14:24:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrYY-0004om-8e; Mon, 27 Sep 2021 14:24:34 +0000
Received: by outflank-mailman (input) for mailman id 196951;
 Mon, 27 Sep 2021 14:24:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrYW-0004mJ-Tz
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:24:32 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrYW-0001BJ-TG
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:24:32 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrYW-0003U8-SE
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:24:32 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUrYR-0000f5-Tz; Mon, 27 Sep 2021 15:24:27 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Reply-to:CC:Subject:To:Date:Message-ID:
	Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=fmHR2HKkcxwSkg8VSxsFQk8VEDEH0dEXe9jHJ5l4uFg=; b=IihPu8hyZe6PRs87MbjTJnsJMX
	qaev38foZPnp4z598Ji3f62G0YPb3Gs/J9/SM/aFmATG25PR/pRr92/nll/01IH96jstXU7lrJq5f
	sVpHRpXNwdQDdglz9gfmMTLliELypgkZA0aItzm3OxyitXYR9b1LWnX2tHzr68OGe558=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.54299.590779.955301@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 15:24:27 +0100
To: xen-announce@lists.xenproject.org
Subject: Xen 4.16 development update; request for regression bug reports
CC: committers@xenproject.org,
    xen-devel@lists.xenproject.org
Reply-to: xen-devel@lists.xenproject.org,
          iwj@xenproject.org

The mainline development branch of Xen is has taken the first step
towards stabilisation for 4.16: new features not yet submitted, at
least in draft form, will now be deferred to the following release.

We aim to make each Xen release as good as possible, and that includes
high quality and particularly an absence of serious regressions.
To help deliver this as Release Manager for Xen 4.16, I need the help
of the Xen developer and user community

In particular, please would users and developers make sure that I know
about:

 * Regressions in the current development branch (xen-unstable)
   compared to Xen 4.15;

 * Major or important work which is targeted for 4.16, but at risk
   of missing the deadlines.

Please email me, CC xen-devel (as in the Reply-To), and of course
other interested parties. including the maintainer(s) of relevant
subsystems (if you know who that would be).

Thanks for your attention,
Ian.


Planned release schedule for Xen 4.16 (recap):

  Friday 24th September   PASSED        Last posting date

    Patches adding new features should be posted to the mailing list
    by this cate, although perhaps not in their final version.
    (3 weeks)

  Friday 15th October                   Feature freeze
 
    Patches adding new features should be committed by this date.
    Straightforward bugfixes may continue to be accepted by
    maintainers.
    (3 weeks)

  Friday 5th November **tentatve**      Code freeze

    Bugfixes only, all changes to be approved by the Release Manager,
    on the basis of a (progressively stricter[*]) risk assessment.
    (2 weeks)

  Friday 19th November **tentative**    Hard code freeze [*]

    Bugfixes for serious bugs (including regressions), and low-risk
    fixes only.
    (0.5 weeks)

  Tuesday 23rd November **tentative**   Branch off staging-4.16
                                        
    xen-unstable open again - with caveats to avoid release disruption.
    (1.5 weeks)

  Friday 3rd December **tentative**     Final commits (docs/prep only)
  Week of 6th December **tentative**    Release
    (probably Tuesday or Wednesday)

Any patches containing substantial refactoring are to treated as
new features, even if they intent is to fix bugs.

Freeze exceptions will not be routine, but may be granted in
exceptional cases for small changes (on the basis of risk assessment,
like any release-ack).  Large series will not get exceptions.
Contributors *must not* rely on, or expect, a freeze exception, or
release schedule slip.

If as a feature proponent you feel your feature is at risk and there
is something the Xen Project could do to help, please consult me or
ask here on xen-devel.  In such situations please reach out earlier
rather than later.  I will try to put you in touch with people who may
be able to help.

[*] The distinction between Code Freeze and Hard Code Freeze is a
matter of degree, not kind; the Hard Code Freeze data and associated
tighter policy text is indicative rather than normative.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:27:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:27:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196978.349897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrax-0006aV-9S; Mon, 27 Sep 2021 14:27:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196978.349897; Mon, 27 Sep 2021 14:27:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrax-0006Zf-5o; Mon, 27 Sep 2021 14:27:03 +0000
Received: by outflank-mailman (input) for mailman id 196978;
 Mon, 27 Sep 2021 14:27:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j90C=OR=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mUrav-0006YD-ML
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:27:01 +0000
Received: from mail-pl1-x630.google.com (unknown [2607:f8b0:4864:20::630])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6913e688-0b1c-43f2-a9d6-8eb87dcd96c2;
 Mon, 27 Sep 2021 14:26:59 +0000 (UTC)
Received: by mail-pl1-x630.google.com with SMTP id j15so10472013plh.7
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 07:26:59 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 f2sm18320581pga.60.2021.09.27.07.26.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 Sep 2021 07:26:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6913e688-0b1c-43f2-a9d6-8eb87dcd96c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:from:to:cc:references:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=MOPOGElqqFMWUhHowVvWcfuLoaWIjv3H2lNBIDiv9Rw=;
        b=SGJ8OL2HIEh1QsmXMs+AylxKGaHye69DGzSIlTE1Ed2ZzIYXgz0A9iOJbI6Xw6VdZy
         4mA7TUsIpYEWe9RryXwJJQ8ZfvvTo34PTEMhzd/2uckvi+fi1i/rIyqTfNq+0m0yVNAt
         F6LSbbqUBHk5JnJXGJOXW+glgPHBHQpOWwwvscWZth5nlM3F4b8vzG3P7Dg6YRB5ArEh
         rL09qxtw7J5ldMSAhHLiocf7vlYu3QM4dcq92OEoew5/r/de1tkpI2r7Fh8eAoNqTerO
         PPxlOej9jAw7wEyPyYUDCYoansJMOTLVj3JMSNcVmD+FQjmTm1/Vw6GSs29yEF0wzk/p
         XhTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=MOPOGElqqFMWUhHowVvWcfuLoaWIjv3H2lNBIDiv9Rw=;
        b=U71o3UkWp+wrDii03rCj+jnGgREAFcJOGcrP3fTIn1iuogVmq2nfIchKAvrw6uNAlg
         5bKiUaUyiIkK+R+ipicPr2PqDXWeCY/uaq9x6azxgJhUTKG1n1xDNUTCsMjxJiL+W1UE
         rNUoDX0MvFzU+qnXYxUwSsjYONGQtzV//KbmFHe/ZP+UIEJUC4ONlNhocgAeAsKZQdSc
         KWz1Rt8l5xmaO5lzubTilolhwvI2aEHVdu9sPm67/E/ySXj3GzbWMK7vlIOf6d78KUjC
         sRyT9dXuUoGlQtEo3jarlV407lys0evTAlvVKkOfemVDVrr7wO9++bz2pODIAUULOO02
         jv7w==
X-Gm-Message-State: AOAM531soC/rQnKFlqnljzCRz9UmSEvp5zRYltaQblfHfYnWHET/DNTm
	ArJikuZiiwRH1Q4Yj35qeJM=
X-Google-Smtp-Source: ABdhPJyzuDnbCGygVxiOXAKTzO/7SeFtM0yROC/HMpbJMd17ZbAsAJOKXu6SedrDCMbcVZ49LU7GdA==
X-Received: by 2002:a17:90a:190:: with SMTP id 16mr268696pjc.152.1632752817722;
        Mon, 27 Sep 2021 07:26:57 -0700 (PDT)
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
From: Tianyu Lan <ltykernel@gmail.com>
To: Christoph Hellwig <hch@lst.de>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>,
 Michael Kelley <mikelley@microsoft.com>, KY Srinivasan <kys@microsoft.com>,
 Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <43e22b84-7273-4099-42ea-54b06f398650@gmail.com>
Message-ID: <e379a60b-4d74-9167-983f-f70c96bb279e@gmail.com>
Date: Mon, 27 Sep 2021 22:26:43 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <43e22b84-7273-4099-42ea-54b06f398650@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Christoph:
     Gentile ping. The swiotlb and shared memory mapping changes in this
patchset needs your reivew. Could you have a look?

Thanks.

On 9/22/2021 6:34 PM, Tianyu Lan wrote:
> Hi Christoph:
>      This patch follows your purposal in the previous discussion.
> Could you have a look?
>      "use vmap_pfn as in the current series.  But in that case I think
>      we should get rid of the other mapping created by vmalloc.  I
>      though a bit about finding a way to apply the offset in vmalloc
>      itself, but I think it would be too invasive to the normal fast
>      path.  So the other sub-option would be to allocate the pages
>      manually (maybe even using high order allocations to reduce TLB
>      pressure) and then remap them(https://lkml.org/lkml/2021/9/2/112)
> 
> Otherwise, I merge your previous change for swiotlb into patch 9
> “x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM”
> You previous change 
> link.(http://git.infradead.org/users/hch/misc.git/commit/8248f295928aded3364a1e54a4e0022e93d3610c) 
> Please have a look.
> 
> 
> Thanks.
> 
> 
> On 9/16/2021 12:21 AM, Michael Kelley wrote:
>> From: Tianyu Lan <ltykernel@gmail.com>  Sent: Tuesday, September 14, 
>> 2021 6:39 AM
>>>
>>> In Isolation VM, all shared memory with host needs to mark visible
>>> to host via hvcall. vmbus_establish_gpadl() has already done it for
>>> netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_
>>> pagebuffer() stills need to be handled. Use DMA API to map/umap
>>> these memory during sending/receiving packet and Hyper-V swiotlb
>>> bounce buffer dma address will be returned. The swiotlb bounce buffer
>>> has been masked to be visible to host during boot up.
>>>
>>> Allocate rx/tx ring buffer via alloc_pages() in Isolation VM and map
>>> these pages via vmap(). After calling vmbus_establish_gpadl() which
>>> marks these pages visible to host, unmap these pages to release the
>>> virtual address mapped with physical address below shared_gpa_boundary
>>> and map them in the extra address space via vmap_pfn().
>>>
>>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>>> ---
>>> Change since v4:
>>>     * Allocate rx/tx ring buffer via alloc_pages() in Isolation VM
>>>     * Map pages after calling vmbus_establish_gpadl().
>>>     * set dma_set_min_align_mask for netvsc driver.
>>>
>>> Change since v3:
>>>     * Add comment to explain why not to use dma_map_sg()
>>>     * Fix some error handle.
>>> ---
>>>   drivers/net/hyperv/hyperv_net.h   |   7 +
>>>   drivers/net/hyperv/netvsc.c       | 287 +++++++++++++++++++++++++++++-
>>>   drivers/net/hyperv/netvsc_drv.c   |   1 +
>>>   drivers/net/hyperv/rndis_filter.c |   2 +
>>>   include/linux/hyperv.h            |   5 +
>>>   5 files changed, 296 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/net/hyperv/hyperv_net.h 
>>> b/drivers/net/hyperv/hyperv_net.h
>>> index 315278a7cf88..87e8c74398a5 100644
>>> --- a/drivers/net/hyperv/hyperv_net.h
>>> +++ b/drivers/net/hyperv/hyperv_net.h
>>> @@ -164,6 +164,7 @@ struct hv_netvsc_packet {
>>>       u32 total_bytes;
>>>       u32 send_buf_index;
>>>       u32 total_data_buflen;
>>> +    struct hv_dma_range *dma_range;
>>>   };
>>>
>>>   #define NETVSC_HASH_KEYLEN 40
>>> @@ -1074,6 +1075,8 @@ struct netvsc_device {
>>>
>>>       /* Receive buffer allocated by us but manages by NetVSP */
>>>       void *recv_buf;
>>> +    struct page **recv_pages;
>>> +    u32 recv_page_count;
>>>       u32 recv_buf_size; /* allocated bytes */
>>>       struct vmbus_gpadl recv_buf_gpadl_handle;
>>>       u32 recv_section_cnt;
>>> @@ -1082,6 +1085,8 @@ struct netvsc_device {
>>>
>>>       /* Send buffer allocated by us */
>>>       void *send_buf;
>>> +    struct page **send_pages;
>>> +    u32 send_page_count;
>>>       u32 send_buf_size;
>>>       struct vmbus_gpadl send_buf_gpadl_handle;
>>>       u32 send_section_cnt;
>>> @@ -1731,4 +1736,6 @@ struct rndis_message {
>>>   #define RETRY_US_HI    10000
>>>   #define RETRY_MAX    2000    /* >10 sec */
>>>
>>> +void netvsc_dma_unmap(struct hv_device *hv_dev,
>>> +              struct hv_netvsc_packet *packet);
>>>   #endif /* _HYPERV_NET_H */
>>> diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
>>> index 1f87e570ed2b..7d5254bf043e 100644
>>> --- a/drivers/net/hyperv/netvsc.c
>>> +++ b/drivers/net/hyperv/netvsc.c
>>> @@ -20,6 +20,7 @@
>>>   #include <linux/vmalloc.h>
>>>   #include <linux/rtnetlink.h>
>>>   #include <linux/prefetch.h>
>>> +#include <linux/gfp.h>
>>>
>>>   #include <asm/sync_bitops.h>
>>>   #include <asm/mshyperv.h>
>>> @@ -150,11 +151,33 @@ static void free_netvsc_device(struct rcu_head 
>>> *head)
>>>   {
>>>       struct netvsc_device *nvdev
>>>           = container_of(head, struct netvsc_device, rcu);
>>> +    unsigned int alloc_unit;
>>>       int i;
>>>
>>>       kfree(nvdev->extension);
>>> -    vfree(nvdev->recv_buf);
>>> -    vfree(nvdev->send_buf);
>>> +
>>> +    if (nvdev->recv_pages) {
>>> +        alloc_unit = (nvdev->recv_buf_size /
>>> +            nvdev->recv_page_count) >> PAGE_SHIFT;
>>> +
>>> +        vunmap(nvdev->recv_buf);
>>> +        for (i = 0; i < nvdev->recv_page_count; i++)
>>> +            __free_pages(nvdev->recv_pages[i], alloc_unit);
>>> +    } else {
>>> +        vfree(nvdev->recv_buf);
>>> +    }
>>> +
>>> +    if (nvdev->send_pages) {
>>> +        alloc_unit = (nvdev->send_buf_size /
>>> +            nvdev->send_page_count) >> PAGE_SHIFT;
>>> +
>>> +        vunmap(nvdev->send_buf);
>>> +        for (i = 0; i < nvdev->send_page_count; i++)
>>> +            __free_pages(nvdev->send_pages[i], alloc_unit);
>>> +    } else {
>>> +        vfree(nvdev->send_buf);
>>> +    }
>>> +
>>>       kfree(nvdev->send_section_map);
>>>
>>>       for (i = 0; i < VRSS_CHANNEL_MAX; i++) {
>>> @@ -330,6 +353,108 @@ int netvsc_alloc_recv_comp_ring(struct 
>>> netvsc_device *net_device, u32 q_idx)
>>>       return nvchan->mrc.slots ? 0 : -ENOMEM;
>>>   }
>>>
>>> +void *netvsc_alloc_pages(struct page ***pages_array, unsigned int 
>>> *array_len,
>>> +             unsigned long size)
>>> +{
>>> +    struct page *page, **pages, **vmap_pages;
>>> +    unsigned long pg_count = size >> PAGE_SHIFT;
>>> +    int alloc_unit = MAX_ORDER_NR_PAGES;
>>> +    int i, j, vmap_page_index = 0;
>>> +    void *vaddr;
>>> +
>>> +    if (pg_count < alloc_unit)
>>> +        alloc_unit = 1;
>>> +
>>> +    /* vmap() accepts page array with PAGE_SIZE as unit while try to
>>> +     * allocate high order pages here in order to save page array 
>>> space.
>>> +     * vmap_pages[] is used as input parameter of vmap(). pages[] is to
>>> +     * store allocated pages and map them later.
>>> +     */
>>> +    vmap_pages = kmalloc_array(pg_count, sizeof(*vmap_pages), 
>>> GFP_KERNEL);
>>> +    if (!vmap_pages)
>>> +        return NULL;
>>> +
>>> +retry:
>>> +    *array_len = pg_count / alloc_unit;
>>> +    pages = kmalloc_array(*array_len, sizeof(*pages), GFP_KERNEL);
>>> +    if (!pages)
>>> +        goto cleanup;
>>> +
>>> +    for (i = 0; i < *array_len; i++) {
>>> +        page = alloc_pages(GFP_KERNEL | __GFP_ZERO,
>>> +                   get_order(alloc_unit << PAGE_SHIFT));
>>> +        if (!page) {
>>> +            /* Try allocating small pages if high order pages are 
>>> not available. */
>>> +            if (alloc_unit == 1) {
>>> +                goto cleanup;
>>> +            } else {
>>
>> The "else" clause isn't really needed because of the goto cleanup 
>> above.  Then
>> the indentation of the code below could be reduced by one level.
>>
>>> +                memset(vmap_pages, 0,
>>> +                       sizeof(*vmap_pages) * vmap_page_index);
>>> +                vmap_page_index = 0;
>>> +
>>> +                for (j = 0; j < i; j++)
>>> +                    __free_pages(pages[j], alloc_unit);
>>> +
>>> +                kfree(pages);
>>> +                alloc_unit = 1;
>>
>> This is the case where a large enough contiguous physical memory chunk 
>> could
>> not be found.  But rather than dropping all the way down to single pages,
>> would it make sense to try something smaller, but not 1?  For example,
>> cut the alloc_unit in half and try again.  But I'm not sure of all the 
>> implications.
>>
>>> +                goto retry;
>>> +            }
>>> +        }
>>> +
>>> +        pages[i] = page;
>>> +        for (j = 0; j < alloc_unit; j++)
>>> +            vmap_pages[vmap_page_index++] = page++;
>>> +    }
>>> +
>>> +    vaddr = vmap(vmap_pages, vmap_page_index, VM_MAP, PAGE_KERNEL);
>>> +    kfree(vmap_pages);
>>> +
>>> +    *pages_array = pages;
>>> +    return vaddr;
>>> +
>>> +cleanup:
>>> +    for (j = 0; j < i; j++)
>>> +        __free_pages(pages[i], alloc_unit);
>>> +
>>> +    kfree(pages);
>>> +    kfree(vmap_pages);
>>> +    return NULL;
>>> +}
>>> +
>>> +static void *netvsc_map_pages(struct page **pages, int count, int 
>>> alloc_unit)
>>> +{
>>> +    int pg_count = count * alloc_unit;
>>> +    struct page *page;
>>> +    unsigned long *pfns;
>>> +    int pfn_index = 0;
>>> +    void *vaddr;
>>> +    int i, j;
>>> +
>>> +    if (!pages)
>>> +        return NULL;
>>> +
>>> +    pfns = kcalloc(pg_count, sizeof(*pfns), GFP_KERNEL);
>>> +    if (!pfns)
>>> +        return NULL;
>>> +
>>> +    for (i = 0; i < count; i++) {
>>> +        page = pages[i];
>>> +        if (!page) {
>>> +            pr_warn("page is not available %d.\n", i);
>>> +            return NULL;
>>> +        }
>>> +
>>> +        for (j = 0; j < alloc_unit; j++) {
>>> +            pfns[pfn_index++] = page_to_pfn(page++) +
>>> +                (ms_hyperv.shared_gpa_boundary >> PAGE_SHIFT);
>>> +        }
>>> +    }
>>> +
>>> +    vaddr = vmap_pfn(pfns, pg_count, PAGE_KERNEL_IO);
>>> +    kfree(pfns);
>>> +    return vaddr;
>>> +}
>>> +
>>
>> I think you are proposing this approach to allocating memory for the send
>> and receive buffers so that you can avoid having two virtual mappings for
>> the memory, per comments from Christop Hellwig.  But overall, the 
>> approach
>> seems a bit complex and I wonder if it is worth it.  If allocating 
>> large contiguous
>> chunks of physical memory is successful, then there is some memory 
>> savings
>> in that the data structures needed to keep track of the physical pages is
>> smaller than the equivalent page tables might be.  But if you have to 
>> revert
>> to allocating individual pages, then the memory savings is reduced.
>>
>> Ultimately, the list of actual PFNs has to be kept somewhere.  Another 
>> approach
>> would be to do the reverse of what hv_map_memory() from the v4 patch
>> series does.  I.e., you could do virt_to_phys() on each virtual 
>> address that
>> maps above VTOM, and subtract out the shared_gpa_boundary to get the
>> list of actual PFNs that need to be freed.   This way you don't have 
>> two copies
>> of the list of PFNs -- one with and one without the 
>> shared_gpa_boundary added.
>> But it comes at the cost of additional code so that may not be a great 
>> idea.
>>
>> I think what you have here works, and I don't have a clearly better 
>> solution
>> at the moment except perhaps to revert to the v4 solution and just 
>> have two
>> virtual mappings.  I'll keep thinking about it.  Maybe Christop has other
>> thoughts.
>>
>>>   static int netvsc_init_buf(struct hv_device *device,
>>>                  struct netvsc_device *net_device,
>>>                  const struct netvsc_device_info *device_info)
>>> @@ -337,7 +462,7 @@ static int netvsc_init_buf(struct hv_device *device,
>>>       struct nvsp_1_message_send_receive_buffer_complete *resp;
>>>       struct net_device *ndev = hv_get_drvdata(device);
>>>       struct nvsp_message *init_packet;
>>> -    unsigned int buf_size;
>>> +    unsigned int buf_size, alloc_unit;
>>>       size_t map_words;
>>>       int i, ret = 0;
>>>
>>> @@ -350,7 +475,14 @@ static int netvsc_init_buf(struct hv_device 
>>> *device,
>>>           buf_size = min_t(unsigned int, buf_size,
>>>                    NETVSC_RECEIVE_BUFFER_SIZE_LEGACY);
>>>
>>> -    net_device->recv_buf = vzalloc(buf_size);
>>> +    if (hv_isolation_type_snp())
>>> +        net_device->recv_buf =
>>> +            netvsc_alloc_pages(&net_device->recv_pages,
>>> +                       &net_device->recv_page_count,
>>> +                       buf_size);
>>> +    else
>>> +        net_device->recv_buf = vzalloc(buf_size);
>>> +
>>
>> I wonder if it is necessary to have two different code paths here.  The
>> allocating and freeing of the send and receive buffers is not perf
>> sensitive, and it seems like netvsc_alloc_pages() could be used
>> regardless of whether SNP Isolation is in effect.  To my thinking,
>> one code path is better than two code paths unless there's a
>> compelling reason to have two.
>>
>>>       if (!net_device->recv_buf) {
>>>           netdev_err(ndev,
>>>                  "unable to allocate receive buffer of size %u\n",
>>> @@ -375,6 +507,27 @@ static int netvsc_init_buf(struct hv_device 
>>> *device,
>>>           goto cleanup;
>>>       }
>>>
>>> +    if (hv_isolation_type_snp()) {
>>> +        alloc_unit = (buf_size / net_device->recv_page_count)
>>> +                >> PAGE_SHIFT;
>>> +
>>> +        /* Unmap previous virtual address and map pages in the extra
>>> +         * address space(above shared gpa boundary) in Isolation VM.
>>> +         */
>>> +        vunmap(net_device->recv_buf);
>>> +        net_device->recv_buf =
>>> +            netvsc_map_pages(net_device->recv_pages,
>>> +                     net_device->recv_page_count,
>>> +                     alloc_unit);
>>> +        if (!net_device->recv_buf) {
>>> +            netdev_err(ndev,
>>> +                   "unable to allocate receive buffer of size %u\n",
>>> +                   buf_size);
>>> +            ret = -ENOMEM;
>>> +            goto cleanup;
>>> +        }
>>> +    }
>>> +
>>>       /* Notify the NetVsp of the gpadl handle */
>>>       init_packet = &net_device->channel_init_pkt;
>>>       memset(init_packet, 0, sizeof(struct nvsp_message));
>>> @@ -456,13 +609,21 @@ static int netvsc_init_buf(struct hv_device 
>>> *device,
>>>       buf_size = device_info->send_sections * 
>>> device_info->send_section_size;
>>>       buf_size = round_up(buf_size, PAGE_SIZE);
>>>
>>> -    net_device->send_buf = vzalloc(buf_size);
>>> +    if (hv_isolation_type_snp())
>>> +        net_device->send_buf =
>>> +            netvsc_alloc_pages(&net_device->send_pages,
>>> +                       &net_device->send_page_count,
>>> +                       buf_size);
>>> +    else
>>> +        net_device->send_buf = vzalloc(buf_size);
>>> +
>>>       if (!net_device->send_buf) {
>>>           netdev_err(ndev, "unable to allocate send buffer of size 
>>> %u\n",
>>>                  buf_size);
>>>           ret = -ENOMEM;
>>>           goto cleanup;
>>>       }
>>> +
>>>       net_device->send_buf_size = buf_size;
>>>
>>>       /* Establish the gpadl handle for this buffer on this
>>> @@ -478,6 +639,27 @@ static int netvsc_init_buf(struct hv_device 
>>> *device,
>>>           goto cleanup;
>>>       }
>>>
>>> +    if (hv_isolation_type_snp()) {
>>> +        alloc_unit = (buf_size / net_device->send_page_count)
>>> +                >> PAGE_SHIFT;
>>> +
>>> +        /* Unmap previous virtual address and map pages in the extra
>>> +         * address space(above shared gpa boundary) in Isolation VM.
>>> +         */
>>> +        vunmap(net_device->send_buf);
>>> +        net_device->send_buf =
>>> +            netvsc_map_pages(net_device->send_pages,
>>> +                     net_device->send_page_count,
>>> +                     alloc_unit);
>>> +        if (!net_device->send_buf) {
>>> +            netdev_err(ndev,
>>> +                   "unable to allocate receive buffer of size %u\n",
>>> +                   buf_size);
>>> +            ret = -ENOMEM;
>>> +            goto cleanup;
>>> +        }
>>> +    }
>>> +
>>>       /* Notify the NetVsp of the gpadl handle */
>>>       init_packet = &net_device->channel_init_pkt;
>>>       memset(init_packet, 0, sizeof(struct nvsp_message));
>>> @@ -768,7 +950,7 @@ static void netvsc_send_tx_complete(struct 
>>> net_device *ndev,
>>>
>>>       /* Notify the layer above us */
>>>       if (likely(skb)) {
>>> -        const struct hv_netvsc_packet *packet
>>> +        struct hv_netvsc_packet *packet
>>>               = (struct hv_netvsc_packet *)skb->cb;
>>>           u32 send_index = packet->send_buf_index;
>>>           struct netvsc_stats *tx_stats;
>>> @@ -784,6 +966,7 @@ static void netvsc_send_tx_complete(struct 
>>> net_device *ndev,
>>>           tx_stats->bytes += packet->total_bytes;
>>>           u64_stats_update_end(&tx_stats->syncp);
>>>
>>> +        netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>>>           napi_consume_skb(skb, budget);
>>>       }
>>>
>>> @@ -948,6 +1131,87 @@ static void netvsc_copy_to_send_buf(struct 
>>> netvsc_device *net_device,
>>>           memset(dest, 0, padding);
>>>   }
>>>
>>> +void netvsc_dma_unmap(struct hv_device *hv_dev,
>>> +              struct hv_netvsc_packet *packet)
>>> +{
>>> +    u32 page_count = packet->cp_partial ?
>>> +        packet->page_buf_cnt - packet->rmsg_pgcnt :
>>> +        packet->page_buf_cnt;
>>> +    int i;
>>> +
>>> +    if (!hv_is_isolation_supported())
>>> +        return;
>>> +
>>> +    if (!packet->dma_range)
>>> +        return;
>>> +
>>> +    for (i = 0; i < page_count; i++)
>>> +        dma_unmap_single(&hv_dev->device, packet->dma_range[i].dma,
>>> +                 packet->dma_range[i].mapping_size,
>>> +                 DMA_TO_DEVICE);
>>> +
>>> +    kfree(packet->dma_range);
>>> +}
>>> +
>>> +/* netvsc_dma_map - Map swiotlb bounce buffer with data page of
>>> + * packet sent by vmbus_sendpacket_pagebuffer() in the Isolation
>>> + * VM.
>>> + *
>>> + * In isolation VM, netvsc send buffer has been marked visible to
>>> + * host and so the data copied to send buffer doesn't need to use
>>> + * bounce buffer. The data pages handled by 
>>> vmbus_sendpacket_pagebuffer()
>>> + * may not be copied to send buffer and so these pages need to be
>>> + * mapped with swiotlb bounce buffer. netvsc_dma_map() is to do
>>> + * that. The pfns in the struct hv_page_buffer need to be converted
>>> + * to bounce buffer's pfn. The loop here is necessary because the
>>> + * entries in the page buffer array are not necessarily full
>>> + * pages of data.  Each entry in the array has a separate offset and
>>> + * len that may be non-zero, even for entries in the middle of the
>>> + * array.  And the entries are not physically contiguous.  So each
>>> + * entry must be individually mapped rather than as a contiguous unit.
>>> + * So not use dma_map_sg() here.
>>> + */
>>> +static int netvsc_dma_map(struct hv_device *hv_dev,
>>> +           struct hv_netvsc_packet *packet,
>>> +           struct hv_page_buffer *pb)
>>> +{
>>> +    u32 page_count =  packet->cp_partial ?
>>> +        packet->page_buf_cnt - packet->rmsg_pgcnt :
>>> +        packet->page_buf_cnt;
>>> +    dma_addr_t dma;
>>> +    int i;
>>> +
>>> +    if (!hv_is_isolation_supported())
>>> +        return 0;
>>> +
>>> +    packet->dma_range = kcalloc(page_count,
>>> +                    sizeof(*packet->dma_range),
>>> +                    GFP_KERNEL);
>>> +    if (!packet->dma_range)
>>> +        return -ENOMEM;
>>> +
>>> +    for (i = 0; i < page_count; i++) {
>>> +        char *src = phys_to_virt((pb[i].pfn << HV_HYP_PAGE_SHIFT)
>>> +                     + pb[i].offset);
>>> +        u32 len = pb[i].len;
>>> +
>>> +        dma = dma_map_single(&hv_dev->device, src, len,
>>> +                     DMA_TO_DEVICE);
>>> +        if (dma_mapping_error(&hv_dev->device, dma)) {
>>> +            kfree(packet->dma_range);
>>> +            return -ENOMEM;
>>> +        }
>>> +
>>> +        packet->dma_range[i].dma = dma;
>>> +        packet->dma_range[i].mapping_size = len;
>>> +        pb[i].pfn = dma >> HV_HYP_PAGE_SHIFT;
>>> +        pb[i].offset = offset_in_hvpage(dma);
>>
>> With the DMA min align mask now being set, the offset within
>> the Hyper-V page won't be changed by dma_map_single().  So I
>> think the above statement can be removed.
>>
>>> +        pb[i].len = len;
>>
>> A few lines above, the value of "len" is set from pb[i].len.  Neither
>> "len" nor "i" is changed in the loop, so this statement can also be
>> removed.
>>
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>   static inline int netvsc_send_pkt(
>>>       struct hv_device *device,
>>>       struct hv_netvsc_packet *packet,
>>> @@ -988,14 +1252,24 @@ static inline int netvsc_send_pkt(
>>>
>>>       trace_nvsp_send_pkt(ndev, out_channel, rpkt);
>>>
>>> +    packet->dma_range = NULL;
>>>       if (packet->page_buf_cnt) {
>>>           if (packet->cp_partial)
>>>               pb += packet->rmsg_pgcnt;
>>>
>>> +        ret = netvsc_dma_map(ndev_ctx->device_ctx, packet, pb);
>>> +        if (ret) {
>>> +            ret = -EAGAIN;
>>> +            goto exit;
>>> +        }
>>> +
>>>           ret = vmbus_sendpacket_pagebuffer(out_channel,
>>>                             pb, packet->page_buf_cnt,
>>>                             &nvmsg, sizeof(nvmsg),
>>>                             req_id);
>>> +
>>> +        if (ret)
>>> +            netvsc_dma_unmap(ndev_ctx->device_ctx, packet);
>>>       } else {
>>>           ret = vmbus_sendpacket(out_channel,
>>>                          &nvmsg, sizeof(nvmsg),
>>> @@ -1003,6 +1277,7 @@ static inline int netvsc_send_pkt(
>>>                          VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
>>>       }
>>>
>>> +exit:
>>>       if (ret == 0) {
>>>           atomic_inc_return(&nvchan->queue_sends);
>>>
>>> diff --git a/drivers/net/hyperv/netvsc_drv.c 
>>> b/drivers/net/hyperv/netvsc_drv.c
>>> index 382bebc2420d..c3dc884b31e3 100644
>>> --- a/drivers/net/hyperv/netvsc_drv.c
>>> +++ b/drivers/net/hyperv/netvsc_drv.c
>>> @@ -2577,6 +2577,7 @@ static int netvsc_probe(struct hv_device *dev,
>>>       list_add(&net_device_ctx->list, &netvsc_dev_list);
>>>       rtnl_unlock();
>>>
>>> +    dma_set_min_align_mask(&dev->device, HV_HYP_PAGE_SIZE - 1);
>>>       netvsc_devinfo_put(device_info);
>>>       return 0;
>>>
>>> diff --git a/drivers/net/hyperv/rndis_filter.c 
>>> b/drivers/net/hyperv/rndis_filter.c
>>> index f6c9c2a670f9..448fcc325ed7 100644
>>> --- a/drivers/net/hyperv/rndis_filter.c
>>> +++ b/drivers/net/hyperv/rndis_filter.c
>>> @@ -361,6 +361,8 @@ static void rndis_filter_receive_response(struct 
>>> net_device *ndev,
>>>               }
>>>           }
>>>
>>> +        netvsc_dma_unmap(((struct net_device_context *)
>>> +            netdev_priv(ndev))->device_ctx, &request->pkt);
>>>           complete(&request->wait_event);
>>>       } else {
>>>           netdev_err(ndev,
>>> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
>>> index c94c534a944e..81e58dd582dc 100644
>>> --- a/include/linux/hyperv.h
>>> +++ b/include/linux/hyperv.h
>>> @@ -1597,6 +1597,11 @@ struct hyperv_service_callback {
>>>       void (*callback)(void *context);
>>>   };
>>>
>>> +struct hv_dma_range {
>>> +    dma_addr_t dma;
>>> +    u32 mapping_size;
>>> +};
>>> +
>>>   #define MAX_SRV_VER    0x7ffffff
>>>   extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *icmsghdrp, 
>>> u8 *buf, u32 buflen,
>>>                   const int *fw_version, int fw_vercnt,
>>> -- 
>>> 2.25.1
>>


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:35:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:35:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.196995.349908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrjO-0008OG-8h; Mon, 27 Sep 2021 14:35:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 196995.349908; Mon, 27 Sep 2021 14:35:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrjO-0008O9-5O; Mon, 27 Sep 2021 14:35:46 +0000
Received: by outflank-mailman (input) for mailman id 196995;
 Mon, 27 Sep 2021 14:35:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrjM-0008O3-NK
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:35:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrjM-0001Mr-LX
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:35:44 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrjM-0004Yi-KK
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:35:44 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUrjJ-0000iJ-DJ; Mon, 27 Sep 2021 15:35:41 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=Ds0r5BeAigokeNiNFatsZ1/EKEVxckV8FbxIOcH/U1k=; b=3StHqKEPw7SwADXLiI5OrBs34g
	a7OxWBfBXt977lfedApA2oNaxorJB9Bj66uoxM/w1e7yshFgvDPQ6qpwj8j2wN5hWZLc09c1PlwRF
	EUPRzlP+Y10nCKAt8V3E1YWQQD8xmVuhqLoTvpNU4qRqFK4DklOFnmXR1o750rz2q+hQ=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.54973.125818.683581@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 15:35:41 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Ian Jackson <iwj@xenproject.org>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v2] Config: use Mini-OS commit 9f09744aa3e5982 for
 xen-unstable
In-Reply-To: <6c655f0e-5e32-6895-1a88-a967c8721bfd@suse.com>
References: <20210908125232.20993-1-jgross@suse.com>
	<6c655f0e-5e32-6895-1a88-a967c8721bfd@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v2] Config: use Mini-OS commit 9f09744aa3e5982 for xen-unstable"):
> Ping?
> 
> On 08.09.21 14:52, Juergen Gross wrote:
> > Switch the used Mini-OS commit to 9f09744aa3e5982 in xen-unstable.
> > 
> > Signed-off-by: Juergen Gross <jgross@suse.com>
> > ---
> > V2:
> > - use a specific commit instead of master (Ian Jackson, Jan Beulich)
> > ---
> >   Config.mk | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/Config.mk b/Config.mk
> > index 4d723eec1d..e85bf18654 100644
> > --- a/Config.mk
> > +++ b/Config.mk
> > @@ -246,7 +246,7 @@ MINIOS_UPSTREAM_URL ?= git://xenbits.xen.org/mini-os.git
> >   endif
> >   OVMF_UPSTREAM_REVISION ?= b37cfdd2807181aed2fee1e17bd7ec1190db266a
> >   QEMU_UPSTREAM_REVISION ?= master
> > -MINIOS_UPSTREAM_REVISION ?= 051b87bb9c19609976fb038f386920e1ce5454c5
> > +MINIOS_UPSTREAM_REVISION ?= 9f09744aa3e5982a083ecf8e9cd2123f477081f9

Acked-by: Ian Jackson <iwj@xenproject.org>

and committed.  I added this note to the commit message:

    [ 9f09744aa3e5982 is current mini-os.git#master -iwj. ]

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:36:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:36:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197001.349919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrkK-0000Vx-J3; Mon, 27 Sep 2021 14:36:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197001.349919; Mon, 27 Sep 2021 14:36:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrkK-0000Vq-Fu; Mon, 27 Sep 2021 14:36:44 +0000
Received: by outflank-mailman (input) for mailman id 197001;
 Mon, 27 Sep 2021 14:36:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrkJ-0000Vg-61
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:36:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrkJ-0001O5-5G
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:36:43 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrkJ-0004jJ-4S
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:36:43 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUrkE-0000jf-9g; Mon, 27 Sep 2021 15:36:38 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=6t322a88txgXQTfImP99DTQ2WYzj3jdM3xB9etmdhVE=; b=WKu9BfD4jliqqoAS8jIxfSeIrP
	i+rBvtvqepRepTdG0LBujlVbntlPjLUwDRJUzzOPZvQIkQiD0RX1CYMFG3X/akZpmZcfyIm7UGUVw
	bzWGE+J0rlrBnidSQccYO9P6dom4rXAP9KDuYrsnXtQpfwLSCw/+Of6E2LVdPMV6T7FQ=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.55030.57966.150042@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 15:36:38 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
    xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools/libs: fix build of stubdoms
In-Reply-To: <adf928b7-850d-963b-155b-983af7733662@suse.com>
References: <20210908124303.20680-1-jgross@suse.com>
	<7bfb511a-2262-948a-fa54-4175550f4104@citrix.com>
	<adf928b7-850d-963b-155b-983af7733662@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH] tools/libs: fix build of stubdoms"):
> On 08.09.21 14:53, Andrew Cooper wrote:
> > On 08/09/2021 13:43, Juergen Gross wrote:
> >> In case abi-dumper is available the stubdom builds will fail due to a
> >> false dependency on dynamic loadable libraries. Fix that.
> >>
> >> Fixes: d7c9f7a7a3959913b4 ("tools/libs: Write out an ABI analysis when abi-dumper is available")
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > 
> > Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > 
> 
> Could this please be committed?

Done


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:37:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:37:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197007.349930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrkn-0001AL-TH; Mon, 27 Sep 2021 14:37:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197007.349930; Mon, 27 Sep 2021 14:37:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrkn-0001AE-Ol; Mon, 27 Sep 2021 14:37:13 +0000
Received: by outflank-mailman (input) for mailman id 197007;
 Mon, 27 Sep 2021 14:37:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qIHc=OR=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mUrkl-00016J-SJ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:37:12 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.51]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f60ac457-dd18-4d55-a8cb-a4af47d464c2;
 Mon, 27 Sep 2021 14:37:09 +0000 (UTC)
Received: from AM9P195CA0020.EURP195.PROD.OUTLOOK.COM (2603:10a6:20b:21f::25)
 by VI1PR08MB4432.eurprd08.prod.outlook.com (2603:10a6:803:102::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 14:37:08 +0000
Received: from VE1EUR03FT057.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:21f:cafe::6b) by AM9P195CA0020.outlook.office365.com
 (2603:10a6:20b:21f::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Mon, 27 Sep 2021 14:37:08 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT057.mail.protection.outlook.com (10.152.19.123) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 14:37:07 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Mon, 27 Sep 2021 14:37:05 +0000
Received: from 33f85b5f2b51.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 11438B15-C593-4B76-A341-7111724D4A1A.1; 
 Mon, 27 Sep 2021 14:36:53 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 33f85b5f2b51.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 14:36:53 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6576.eurprd08.prod.outlook.com (2603:10a6:102:15a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Mon, 27 Sep
 2021 14:36:52 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Mon, 27 Sep 2021
 14:36:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f60ac457-dd18-4d55-a8cb-a4af47d464c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AA0A5XgcHo/DiPOEgEiB18CiJ3SV9SeYjxxmhRrAfbc=;
 b=4bmm3B7R/u3vq+hdW5wDCUJqZ7si4oEr01b8O/w6LOtc8DWrz5LQ90QKvtG0ci+jFpGooNTtWO2qxFQa6Koc5T/ydW+W3r9Lu6yf9P0hbfvCJcl16AtgqyDZHZG/AQGQrlEMkVw5CtSkks2j3U5srmkQxlVq1RWZw0MTDi2JZsQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 46ac2b8e162df1da
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OpgJRqO0P+bcake9HXvn67nVT7/z0B/pwhBxkcM8qRZZisrH++SHEFEIK9+w0zhRP3Y6mwVy7qRW2/6f6uow/0fzm4be6gCdeMJ9+Br62JMgToSCxs1o6ztUyBPpjmVfXsHm3U87hTPw+ZvMawVEAfFHRqR26f2EVkOgztIM68tiC+ZKd5ZJiiiW12TTkAcbrB7C2ElFZ8rBzMT6KlM9vhDK2N/lG7OmNHv+MwRwzRgYA8F036RC/I0wi2nVC0tKcZ2HEwSglC6xdjY1NUxFFNKOAH5B2bCrya+8CgiSvgxJo/ug2geddu0e0k9j1AfDARqCojR0fedQAkzY7NGefg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AA0A5XgcHo/DiPOEgEiB18CiJ3SV9SeYjxxmhRrAfbc=;
 b=UgRbUf1AyR+f976UhNqYVPFdyWw+nwlS2Po5AQpc0QRgaLdnWW6nk0BdDfa+ZuAaAHqVe6dVk4Oj1TUMEZ9OAGZmcwOjnphRcetozx1m+3UWy74zfJBIe+qcoYHOE5x+gafuwv4OGJhFjzFOdTbExG1e/mFdNKuz4kf4tB06MbthVhkLB9HChFatHJIHIhuASSSan6tfyUvPc8Wog2PJ2speGJ8lcGLPVEdPMG1JIh/BG93+aWmg7Jxaoi7slJxjMJ/DrVMBcLc/PyudogP0War9wPqTgoZHfVp9EZ5KEFy9vgJwDkN+y7ZbT3cIAC63uc5mluTPwv6tUDmA7SMCMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AA0A5XgcHo/DiPOEgEiB18CiJ3SV9SeYjxxmhRrAfbc=;
 b=4bmm3B7R/u3vq+hdW5wDCUJqZ7si4oEr01b8O/w6LOtc8DWrz5LQ90QKvtG0ci+jFpGooNTtWO2qxFQa6Koc5T/ydW+W3r9Lu6yf9P0hbfvCJcl16AtgqyDZHZG/AQGQrlEMkVw5CtSkks2j3U5srmkQxlVq1RWZw0MTDi2JZsQ=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>, "tee-dev@lists.linaro.org"
	<tee-dev@lists.linaro.org>
Subject: Re: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for
 OPTEE_SMC_DISABLE_SHM_CACHE
Thread-Topic: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for
 OPTEE_SMC_DISABLE_SHM_CACHE
Thread-Index: AQHXs6dCIyk0ePXwCku0OJ1w+54IXKu38umA
Date: Mon, 27 Sep 2021 14:36:51 +0000
Message-ID: <E0C80373-4139-4426-A9B5-6FBBA2E1AACE@arm.com>
References: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
In-Reply-To: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3dd589a3-5402-42f9-b31e-08d981c44867
x-ms-traffictypediagnostic: PAXPR08MB6576:|VI1PR08MB4432:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB4432647500A40B3F70698DEA9DA79@VI1PR08MB4432.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rP2mlVaYUBUllTzSZZsyYtAxXi6yLwaohgvgxBh7JC3QnLRTSMMgXjZxSgbYZeb98r560+T4wPNahhMSkxDfkgHStC2Eu2BdU8fPDDVN5Hm4BJglG931jHAMkO9EehgVTlKp1xrulzLvHxbxMOSlfW+yzeO7ZgQIkG/sM4nuxcJcIUoIVMw5kIEOYXSO/+NHgGWsxyZ0eBxDo36ZsrYO/rVTww6zGHSbfWRI7lb/+BYm/4Yv78R0X05tZFGXpcc5hbHz8WblDK/Etmcp+wQ6zU7vRLtaQ28DBcyT0uiE8GgwjscAflSokUYJgjCB3EQK7l2IZ4RV5C6kuIeFfgYaSNf3ZSjrdirGnnQMcdjiGZf6zPzWDRpStMWhUOEoqyRFo3Bzf9kVkZoSULPTiAGWx5RU57wBxXx50BpWN8JnnIR8l573QXQ8Bqf8+RA9CmhHjupTU/0ycRhNIPllKX1EdoYGiqOAgJIDPEt1pHVB5yKsDSRyEpk6f5XWxzle4+tHgXcgHZeQorqKt6usMkcwpe6Bst8rtzaODhKNA2bGN+V1FIsNEWR58CNnd0z0pdjeIKSm5I2u55eXFcP6VaLo7nmYevE47GC/HJomWxF6XbM0Yz0CRlqt7rY3Yjpwz546tnJUCEsGw/j4vRa5OcyYYmmlIWbyXCSl2MZlGloWS4avcv+ITytC/o3PEwqX31E5KlwmEHMDtxDbGbHOtOIgEa31jGNcyUNIeqQxw10W+bM8AeKFAnMy8+UZL/nhMLGl
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(71200400001)(83380400001)(2906002)(6916009)(26005)(8676002)(38100700002)(8936002)(316002)(4326008)(54906003)(53546011)(122000001)(86362001)(76116006)(6506007)(6486002)(2616005)(66556008)(66446008)(5660300002)(36756003)(33656002)(66476007)(38070700005)(186003)(6512007)(508600001)(64756008)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2ED8D24AC678B24787A71C272AD26DC5@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6576
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	02f9e51e-f392-42d1-84c3-08d981c43ef2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1VCNoTFIl6DHLAeomgYW/zC8HEhDKFxIbPtSu9AEMhofZuoTECUR+5Gq37YrUICTY4semkjrMt/O8xJUxP0prncTXpMcUB1Wt21S3n8649k4699AHrhR7J7ErX2xBt6FhaaBKwTc4XFfs/p1yB8uGEE7IKnldwUoQWd+/+CcpiWpCZar5anSSkJF5Zt2U04tPtRzObTYFXfb101QL9afdc0Yr1zUakQFBj9ZEBltz9P7xSQT3Gt3UIsoOaCGPeNxbujRx3nygcFWGPPqG93/zL9c8ybGvUZs5aN1BfSIsKELfCZ6aUQMXZTu+N16ifRj4brq89nGECVRKw1PsoKx05MGrsRd5aaVBIiX7I5utxHpSnzH+x/h8lSLu47HHkEZ7R3998CI2pukRJuXm+DVGY6IeTK/v8nM9TLmFgRUY9kTj0ro0bZT6Z9wPfsagT3PbpV0LhtzBHCUxp8e6Gx8NQ96UYZeRqNb7VYx+L1Mr2Z0Mfz2rcSEL371Sq1pzotaj0qmPgZm4jOD8ArtXkkH0RJUy+Gd87423B9rxJyOg1Hymy0WS+hAudBRAlNcLvJyzjuQwcsxDy2RGFWXXNFbQWtnjKfAkTyr68vfHaFTNFTfPO1FOljIsUJnx6BpQhAnoDB35jxUKybxTDY5VIaoRmwsfEtPwjp9tzdbr30uvM0hBAPq6lYN4jRkW+2K/mE9UMzRi8iUTzEuZ+MkCLXuLQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(316002)(82310400003)(4326008)(107886003)(6862004)(81166007)(70206006)(2616005)(6512007)(36756003)(356005)(6486002)(53546011)(8936002)(70586007)(26005)(8676002)(336012)(6506007)(5660300002)(47076005)(2906002)(54906003)(36860700001)(186003)(86362001)(83380400001)(508600001)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 14:37:07.6935
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3dd589a3-5402-42f9-b31e-08d981c44867
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT057.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4432

Hi Oleksandr,

> On 27 Sep 2021, at 14:54, Oleksandr Tyshchenko <olekstysh@gmail.com> wrot=
e:
>=20
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>=20
> Fix a possible copy-paste error in arm_smccc_smc's first argument (a0)
> for OPTEE_SMC_DISABLE_SHM_CACHE case.
>=20
> This error causes Linux > v5.14-rc5 (b5c10dd04b7418793517e3286cde5c04759a=
86de
> optee: Clear stale cache entries during initialization) to stuck
> repeatedly issuing OPTEE_SMC_DISABLE_SHM_CACHE call and waiting for
> the result to be OPTEE_SMC_RETURN_ENOTAVAIL which will never happen.
>=20
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Sound like a reasonable change :-)

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> I wonder whether this patch wants backporting to the old versions
> since OPTEE support went in.
> ---
> xen/arch/arm/tee/optee.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>=20
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..6df0d44 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -1692,7 +1692,7 @@ static bool optee_handle_call(struct cpu_user_regs =
*regs)
>         return true;
>=20
>     case OPTEE_SMC_DISABLE_SHM_CACHE:
> -        arm_smccc_smc(OPTEE_SMC_ENABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
> +        arm_smccc_smc(OPTEE_SMC_DISABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
>                       OPTEE_CLIENT_ID(current->domain), &resp);
>         set_user_reg(regs, 0, resp.a0);
>         if ( resp.a0 =3D=3D OPTEE_SMC_RETURN_OK ) {
> --=20
> 2.7.4
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:40:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:40:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197016.349941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrnw-0002hs-Bm; Mon, 27 Sep 2021 14:40:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197016.349941; Mon, 27 Sep 2021 14:40:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrnw-0002hl-8Y; Mon, 27 Sep 2021 14:40:28 +0000
Received: by outflank-mailman (input) for mailman id 197016;
 Mon, 27 Sep 2021 14:40:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g88l=OR=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mUrnv-0002hf-42
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:40:27 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f36aa982-cf7b-410e-ac89-1cea42103cd7;
 Mon, 27 Sep 2021 14:40:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f36aa982-cf7b-410e-ac89-1cea42103cd7
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632753625;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=oOEl2Y+nsFDfz8F3hMxC2R9NibfcB39NbSJvpyeONt8=;
  b=Wxs53KBMizbJLHqSOOSXrWU2o27WbHS1Bm1HLeVgIWxLr/4doUTugu4M
   kZM685ycQZAMVZCdFIoK1y10iHRHKJ8rz7rbk6+JSJwe2IMYaBxFOOC00
   lnuuZMC9IqRcWesRuhiGry9DIVYYafl1j0qc3KtQnCezec94mxAr5bNpm
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: savFAdl03aPR77hgo4pEBd/Hfnd6o3G/o6TUNx2gUsrPtU7pQRWutZfr+bC6hZZA2zzW53wM3Z
 2jL0MFsG6D39oIQgW4FALVv21NDYLKvKyMQJiyVozhvoM7FaE5FWccW4k/wjQigcpEKXT3xRz6
 ozKg6l5VIYdRaU/KK/zOJngvmZLdEEJBETr46Zbw8cTaWg5Co0wCr1erTs1IUQhgnosYFdZrZz
 ZSU8hw/0REgZFAXisSh+iP2La6DsWy17MS6O/pBmXYx2fmr6h9A3hCvwTyl4gVXZHwbM72Vs/x
 rmvZ7ILyg9ExA+lXxjf09rAL
X-SBRS: 5.1
X-MesageID: 53315451
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:O6+lvqDrR3KJIRVW/9jkw5YqxClBgxIJ4kV8jS/XYbTApGgkhWEFm
 zZNCjrXPPrbZ2SncthxaYXi9hgFvpfcz98wQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0007wobVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/ugrTh9t+z
 NZ0sL+2TystE+7nnOsgXEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFh2xp25wTRJ4yY
 eJHUzUyckSdQCFEZFcmCaMFvuCX2T7wJmgwRFW9+vNsvjm7IBZK+KjgNp/Zd8KHQe1Rn12Ev
 STW8mLhGBYYOdeDjz2f/RqEhODVmjjgcJkPD7D+/flv6HWMwkQDBRtQUkG0ydG4lUyWS99ZM
 1YT+Cclse417kPDczXmd0Tm+jje5EdaAocOVb1hgO2Q9kbKyzemXHAPQTpCUdkFsskaQRFp/
 VuGoty8UFSDr4apYX6a876Vqxa7Ni4UMXIOaEc4cOcV3zXwiNtt102eF76PBIbw3oWsRm2gq
 9yfhHJm74j/m/LnwElSEbrvrTW2utDtRxM56x/bV2aohu+STNX+PNT5gbQ3AOwpEWp4crVjl
 CRa8yR9xLpXZX1oqMBraL5UdF1Oz6zZWAAweXY1Q/EcG82FohZPh7x47jBkP1tOOc0ZYzLva
 0K7kVoPv8UNbCL2PPEnOdjZ5yEWIU7ITo2Nuhf8NIYmX3SMXFXfoHEGibC4hQgBb3TAYYlgY
 MzGIK5A/F4RCLh9zSreegvu+eVD+8zK/kuKHcqT503+idK2PSfJIZ9YYArmRr1ot8us/VSKm
 +uzwuPXkn2zpsWlOXKJmWPSRHhXRUUG6Wfe8ZcKKbLTflQ8SAnMyZb5mNscRmCspIwM/s/g9
 XChQE5Ijl35gHzMMwKRbX5/LrjoWP5CQbgTYUTA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:Dd9lm6ldfYpiaoCm7Jjngs5v7YXpDfIo3DAbv31ZSRFFG/Fw8P
 re+8jztCWE7Ar5PUtKpTnuAsW9qB/nmqKdgrNwAV7BZmfbUQKTRekJgLcKqAeAJwTOssJbyK
 d8Y+xfJbTLfD1HZB/BkWqF+gAbsbu6zJw=
X-IronPort-AV: E=Sophos;i="5.85,326,1624334400"; 
   d="scan'208";a="53315451"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CFuz92T6NghP7WdMUt5rleuaChtOsJrvqO5TMWiD2I390lceuiGDsBe4zVq/ogI0nL2J9A8y5927OI7VaGueRWfXjNt6hUPnbWbcMtuoBqmlMbjw2vjN0LqjbUoWMmrChwLMvHO9wd4LKz0NHVQCAtH171WN8Rj2e1anzpk8/FLw6K6FAzNHIHNNWniWnTz+ZECOCX9UNi5AhuA65DvvKTTEHRQy/37xjdx3d4I8JxszpzXM4IlECtZLDV/9ZU8unTPQ6M3Gy6ILPRr4hi+JHmSiQoYBg4s+poLtGnRJ5WrZKioziBLd8GvvAWrWzQLkYAgU4aR93iL1bWf9UNC8iQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oOEl2Y+nsFDfz8F3hMxC2R9NibfcB39NbSJvpyeONt8=;
 b=UDk+h/KmjgrPqPGS2azd77Zq+iDVC8cs0hvJvZEjw83lT887+tgFK5Zx3iXkdrw0SbvrTAgdAojfaNoMyn4e1KEsgHabMFkxXYziZZYMducvxc0E3OISAZC6Z0BSAri35GNtuRVYSwpPALBrDMWlyl+JJvwMNBh2JYrjfrOTL1qDl+eJEYLyf7aYviPTvolTsLX1KTZ20w04R3RqFAcNc4doZfROHZqnBOuKvNCm/oOx4C8F0vWYsY3Hy78j5uKV6FxxDTxU69/e4ptTQX6YGlSfmDbvY0V7cKsRj3NN4Fa0hcfgX80JsvjyQQgGVpLojTkgXGsrJoUT+Wr2813DnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oOEl2Y+nsFDfz8F3hMxC2R9NibfcB39NbSJvpyeONt8=;
 b=e0FShg4iILQYmZbDrewJaFhuVJllq0JyxK/KgoOT/Ef+ee5E17NpWs/T6IlZrYyTW42y14P5dIHmWkVHtKSOPjCy4vyDmRk8cT0Y0OzG7h/wIbk5Gnl5jVhJVPKxsYZAUoKb1QJdZaWEukILReNElvR2blqXvEZWko5fZp8nNf0=
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
Message-ID: <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
Date: Mon, 27 Sep 2021 15:40:09 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0050.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:60::14) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f2bb5e1d-8746-4082-cb67-08d981c4b826
X-MS-TrafficTypeDiagnostic: SJ0PR03MB6270:
X-Microsoft-Antispam-PRVS: <SJ0PR03MB62701D7DAB74A6BFEA3E2937BAA79@SJ0PR03MB6270.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2733;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PWsrKN0KQv4WeQN68whL9o7gkjLpHL7ny8J/Mc+4IuAjLNzmab4W7aR5BYUTnHPQwYbzxndqlOtnaiW0qIAr3HfmHk/HLYUJ3i7ztkYlII9hq1bMz2HAWdCO5p2vaznDgZgEKKd/4YXLcEqrbIGoxpgo9YJDybBlmSljvpij10VTWfB/6Is8GlFAVipOgZ0CQGbI+kYPcGFWL9EXQymAEfvfHkPNNkq56sDTJXuQ2lm5Lqd4HvYPcb0Ti9E0EojxPFBDmPCl9etPo9FyZ1FrCXYTan0mxogrms5Gc9mlJCGy2KbKMhgZs+cCoaQQmakgm82GD1/QN0OXBwGvjb5j6yssm861FloKs1CP5h4RR3VWBsCQp8868srFn42/+b4oEGBDDM6fI2WfhvBybU3+0pZUOgUBk0quL5wK9uxnTLNV2ZSsYTL3GgmlVA26ioS1J7cK3hhfEf4GHBVeCguSfSsb00zocG4Zxec4Y6TQ9uyDjxNW/ai8tTLHP9b3XhqzcGw69TEEBtVBazt1hAylb6lRVa6ZxnHokwPzDAUmmhK8yWRk3a4BEWyeA+6hnEYt4O4cCvrvt9j/LTZRVfGOgyzLwPzuBsfQNp0jCOOWAYI22Sn+5Lh6u5QxvslOV82PgGBIAXoigcTMHevvosLOpzCr1pdZdRYAnFnjca0cp/cIt7DDV4AQ+bJgSypaVddBeXq2alOnSp7fugD8VSsG4RIH20ZZSQRO2b6YrrNcT8y62TNX3TyHo9q/a4ZCP5n0joSdceRjM6t3NOEDdvdHuDnf7JaJspjzMAXg2oC3fQkL8k6uQHARhDlFDO3I1lmK/1dHv4ZAnvOVPXYg06rZpg==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(66946007)(86362001)(55236004)(2906002)(53546011)(6666004)(83380400001)(8936002)(956004)(966005)(31696002)(2616005)(8676002)(54906003)(5660300002)(26005)(36756003)(186003)(316002)(508600001)(66556008)(16576012)(6486002)(31686004)(66476007)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TUp3b0gxa3BsaHBOeDFRWm90L3BXamhxaDYydDhnY0drMGI5TVczbXFOVEpz?=
 =?utf-8?B?ZmJQZ0IrY1dMUWJXUjBXaWRrOC83KzkyVmFqcXR0UldGNHJ1NXhWcDFacTk3?=
 =?utf-8?B?cGs3MmtvK250Q21uOGVweFRacjRrVlBaSHEzRmtTU3lhSUFGZmJaOUlqZ0o5?=
 =?utf-8?B?TVpGQmltZThCSTVCb1hoUkY1cmF1TnVLelJZQ1FVbjk5QW8vckJhcys3R2Rl?=
 =?utf-8?B?VDFMUXl3YXRIUENOdW5ySkxxK290QkJGVEhvdDlEMVZzR2R2Z2tNZUFaYWJq?=
 =?utf-8?B?TFBUR3pPQTBrNGFJT0VGajZvemtqY0hZNGhtcHpCU0tCbjFpUTZlOGM1VXBV?=
 =?utf-8?B?YUpFVEZ5YWdCZVBBdGRWRkF2ZXNnN3kraE9vTmJVRllGMlZuR3R5cnlEV2d0?=
 =?utf-8?B?VDE4Z2dGNk1OR3BZZkxmNnF0MWNqbVdpRnVla0FkV2YzdmtTM09JVHFGQXdM?=
 =?utf-8?B?dENWSFFSclZtQy9TYmF3SGJheEFlVVRpNmNvNmU5ek5Vd2pja0pTYStEVHho?=
 =?utf-8?B?a3dha0hIWkdaeG5yYS95OHB3aksvS09qQnRmWVNBWmJKTnBrQ3lsd1ZiQ25G?=
 =?utf-8?B?ZDFCUVQwcTVKSncreG1ERDl3cHpoOHE5SmlnUVowbzVOZGo5OG1hcTdlK3FV?=
 =?utf-8?B?ZHVsdGZzM3pYS0twTHlzYzdOcjc3ZUx3TEgrTTVJSUw0S1lsZnJFWktMTlp4?=
 =?utf-8?B?a2d4d0tWcW5yWXdPMmdUZDRWMmlTdmJucys2eGlnMjdncXFHRVRRSW5jSVlP?=
 =?utf-8?B?L1lCU2JCVjA1ampwbHRZRUI0bFBtWmFtdzlBUmV0NWdDWWs3ZkdHeWZOa2Zv?=
 =?utf-8?B?ZXJUN3NOdFFyM3VsMVl4ZEFvZ3JyTXloU1lUTVkramFBWjRLUXJ6UEdVbUVw?=
 =?utf-8?B?OFBmWWxTU0c3elJPd1BZbVpzV3pra1FzT2h2UGFQalRaSzRlUmphM3RlU2kv?=
 =?utf-8?B?bXgxTnNYMEVtaDlURnVCQWtFZXFkWWZMbllBcjlhb3FnbDkya3huTXl4d1dW?=
 =?utf-8?B?bVllL2krQ3J1ZG9udmNWQ21KWGVEcTYyMi9MMDQwQWJBRGRDdDBOVmJ6QzVu?=
 =?utf-8?B?REpUN1NETUhnaTJaaXFVNWt2eVBJa1VUTEluVGtNOElYcWtrQi9xNVp6RXdN?=
 =?utf-8?B?dk51RVl3aGhxVTFDZSt0RnVLZzFoZDNpUVV1eG5jZFQ1UklnSURVNEJaVUxW?=
 =?utf-8?B?T0JPMGNZNzZnZU5ndjc3KytrZDJVQ1R0bXRMTkVDdXEvSDc1a2liN1Z4UHZE?=
 =?utf-8?B?NXc5dHdIUHJhdGF5RTJPNnN4NkpncXRLdmRsZjgzbE9GUlNFMWFpMk93Mzh5?=
 =?utf-8?B?WHBoNFVZSlV1Wk1DeTRrNUcrSnBPQ0dtdFhxUlZ5dktBOXo1MVBrNFRzM3pS?=
 =?utf-8?B?U2RQZUNQbXpqNnAySG5lWHc3RkRncTRUNE12V28ybkZpMkNSVkc0a1RrcTZ3?=
 =?utf-8?B?MktCbjFMazZzeHowcWRQclVTUitkMDJQems4L3dXcXZWUndOdi94WTJCV1Rm?=
 =?utf-8?B?cjZmMDFLdGh4c2Mwc0RweTFodGdqQVRHemk3Qi92SlBLZ3NScWhzWUxiM2NB?=
 =?utf-8?B?blRtMC9RVC82V2RTcVRjUi9RMG9zbE82QTNwR1VkY1RleklwbUJ1eDJVWFdJ?=
 =?utf-8?B?b0IvZ1RkL21ySHdlT1dUd3U3Z1M0NVRmTlpkQmdlYkJwSytMOUFNYTI3Wldw?=
 =?utf-8?B?d2RaTGJlaWk3N2hQVUZoOEpKSExaeTh6cGdJWHdJS01ha3UwbG1idU5IMFF3?=
 =?utf-8?Q?+UOIveGKXnH2OWdbYZPFnjKnCShuoiPX54A1Gn5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f2bb5e1d-8746-4082-cb67-08d981c4b826
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 14:40:15.3828
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xqOz6MJiYhJfTmjyRzm4DMOVeyYcHHgQl3TlJ7ZQoMry2Et5FqJ+09ktz0k9HM9dvp2WXhUv1ioisTkWkt9VBt5Kx7X/9Kpr7p7dTsxio+4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6270
X-OriginatorOrg: citrix.com

On 27/09/2021 15:24, Juergen Gross wrote:
> On 27.09.21 16:13, Andrew Cooper wrote:
>> On 27/09/2021 11:48, Juergen Gross wrote:
>>> Add a configuration item for the maximum number of open file
>>> descriptors xenstored should be allowed to have.
>>>
>>> The default should be "unlimited" in order not to restrict xenstored
>>> in the number of domains it can support, but unfortunately the prlimit
>>> command requires specification of a real value for the number of files,
>>> so use 262144 as the default value.
>>
>> Citation needed.
>>
>> prlimit -nunlimited
>>
>> prlimit --nofile=3Dunlimited
>>
>> both work fine, and strace confirms they issue correct system calls.
>
> Not on my test system:
>
> # prlimit --pid 734 --nofile=3Dunlimited
> prlimit: failed to set the NOFILE resource limit: Operation not permitted
> # prlimit --pid 734 --nofile=3D262144
> #

What does strace say in both of these cases?

>
>> Support for "unlimited" as a parameter has existed for the entire
>> lifetime of the utility,
>> https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e219=
dbcf62ec5ede84
>>
>
> Yes, but not all systems seem to support raising the limit to
> "unlimited".

That's as maybe, but

prlimit64(0, RLIMIT_NOFILE, {rlim_cur=3DRLIM64_INFINITY,
rlim_max=3DRLIM64_INFINITY}, NULL) =3D -1 EPERM (Operation not permitted)

is a Linux issue, not a prlimit bug.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:43:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:43:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197024.349952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrqi-0003O9-TL; Mon, 27 Sep 2021 14:43:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197024.349952; Mon, 27 Sep 2021 14:43:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrqi-0003O2-Q3; Mon, 27 Sep 2021 14:43:20 +0000
Received: by outflank-mailman (input) for mailman id 197024;
 Mon, 27 Sep 2021 14:43:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrqh-0003Nw-6X
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:43:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrqh-0001XX-5V
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:43:19 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrqh-0005M7-4E
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:43:19 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUrqd-0000lG-LW; Mon, 27 Sep 2021 15:43:15 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=1TRhJg8PYLjD3J6NES3KkqVbp7A6faSg0tvEFu/PVEQ=; b=WzE9SKC+WjHQCmbRwBVZVOaQrl
	FLmTGsgmoyU0C3XPPU3cBeGHSTX+VkFvjzhOahoKZZ6iQhLvwH2q+SQgdAZJqF9k/mbAvBDDztDen
	E+/DYAUCICKAttZO7ITriW1L0+nSqpwhS8WHYArGJ5iuuPexj6E5c+42vuSd6bEMN+mY=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.55427.402592.660538@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 15:43:15 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Andrew Cooper  <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>,
    Samuel Thibault <samuel.thibault@ens-lyon.org>,
    Community Manager <community.manager@xenproject.org>
Subject: Re: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per
 default
In-Reply-To: <d894799e-27c6-bbbd-8d65-3e32c1aff843@suse.com>
References: <20210910055518.562-1-jgross@suse.com>
	<24891.31480.165445.521062@mariner.uk.xensource.com>
	<d894799e-27c6-bbbd-8d65-3e32c1aff843@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v3 0/3] disable building of pv-grub and qemu-trad per default"):
> On 10.09.21 17:34, Ian Jackson wrote:
> > Juergen Gross writes ("[PATCH v3 0/3] disable building of pv-grub and qemu-trad per default"):
> >> This is a first step of deprecating pv-grub and qemu-trad including
> >> ioemu-stubdom. Switch the default to not building it.
> > 
> > This is now fully acked.  But can we wait with committing it until we
> > have a decision about whether to (a) have osstest explicitly enable
> > the pv-grub and qemu-trad builds (b) have osstest stop testing these
> > configurations ?
> 
> Any decisions made?

No-one seems to have had any opinions.   I'll take ...

> FWIW I'd be fine dropping pv-grub builds and tests in OSStets, but I'd
> rather keep testing qemu-trad stubdom tests.

... that as a proposal :-).  think that is probably the right tradeoff.

I think that means that means we need osstest patches to edit
make-flight and ts-xen-build which

 * Explictly enables stubdom qemu build
 * Drops the pv-grub tests

and those need to go in first.

With my RM hat on, I think those changes to osstest may need a release
ack since they have missed the LPD, but I will grant such an ack.

As for the patches themselves, I may get to that tomorrow, but
contributions would be very welcome.

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:44:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:44:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197031.349966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrrU-000407-75; Mon, 27 Sep 2021 14:44:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197031.349966; Mon, 27 Sep 2021 14:44:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUrrU-000400-3o; Mon, 27 Sep 2021 14:44:08 +0000
Received: by outflank-mailman (input) for mailman id 197031;
 Mon, 27 Sep 2021 14:44:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrrT-0003zl-2g
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:44:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrrT-0001Yw-1x
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:44:07 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUrrT-0005aI-19
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:44:07 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUrrP-0000mB-RV; Mon, 27 Sep 2021 15:44:03 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=smCdIbDJaJpow6x6NDDzRkknRgqMXv83TA7sdSVqiyE=; b=367nAY4oilSM3KCSzBjEUMEQ5t
	MplaC4VEWc5E5hD5qiXDhDUMdq2T3DEWONnX85EpWrKSomgFd8d+CgP7lRb311+KtnoPPFWwCZ+31
	K/2XELEBGJ0sCHMIowaGavwIDlKPSJg8jnSdg8IYPxA9mY/wWfy8OkROY4BAraThqUB4=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.55475.600304.918870@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 15:44:03 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: [PATCH v4 1/2] tools/xenstore: set oom score for xenstore daemon on Linux
In-Reply-To: <20210927104813.19772-2-jgross@suse.com>
References: <20210927104813.19772-1-jgross@suse.com>
	<20210927104813.19772-2-jgross@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("[PATCH v4 1/2] tools/xenstore: set oom score for xenstore daemon on Linux"):
> Xenstored is absolutely mandatory for a Xen host and it can't be
> restarted, so being killed by OOM-killer in case of memory shortage is
> to be avoided.

Reviewed-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:53:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:53:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197040.349982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs06-0005hK-74; Mon, 27 Sep 2021 14:53:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197040.349982; Mon, 27 Sep 2021 14:53:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs06-0005hD-3s; Mon, 27 Sep 2021 14:53:02 +0000
Received: by outflank-mailman (input) for mailman id 197040;
 Mon, 27 Sep 2021 14:53:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUs04-0005h7-AT
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:53:00 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9acb0c58-1fa2-11ec-bc50-12813bfff9fa;
 Mon, 27 Sep 2021 14:52:59 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 268FF221D5;
 Mon, 27 Sep 2021 14:52:58 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EA48613A91;
 Mon, 27 Sep 2021 14:52:57 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id XDzlN8naUWG4RwAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 14:52:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9acb0c58-1fa2-11ec-bc50-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632754378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=Bwuvl+k0GlLkveSbi/F8ZUNL1aQE14c7Sn0p7LqGkRQ=;
	b=rwOsY3QBH1rc4qZc9upD89YYweZ3VhlMQ2+yUv84Gahd8AyCgjVIEbKNGaJLkS3fUaTPq9
	f77m+FMo/e3vOXvv9AI89tqdLo5WeQ3zGwA1nU9mGIfNvn2uAOkHH79hAabfTSJZG0W17n
	DNfDj05V1iyVZEoUrg90xp3WxkgDQII=
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
 <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
Date: Mon, 27 Sep 2021 16:52:56 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="fsYiyn1EpBjwAz7vOI1QWRYrB6LQk0XFx"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--fsYiyn1EpBjwAz7vOI1QWRYrB6LQk0XFx
Content-Type: multipart/mixed; boundary="9tKOk0sK5O9AOfaD8O88T9OvYKplZuGB3";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
Message-ID: <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
 <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
In-Reply-To: <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>

--9tKOk0sK5O9AOfaD8O88T9OvYKplZuGB3
Content-Type: multipart/mixed;
 boundary="------------3177C107245117017D2D69CA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------3177C107245117017D2D69CA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 16:40, Andrew Cooper wrote:
> On 27/09/2021 15:24, Juergen Gross wrote:
>> On 27.09.21 16:13, Andrew Cooper wrote:
>>> On 27/09/2021 11:48, Juergen Gross wrote:
>>>> Add a configuration item for the maximum number of open file
>>>> descriptors xenstored should be allowed to have.
>>>>
>>>> The default should be "unlimited" in order not to restrict xenstored=

>>>> in the number of domains it can support, but unfortunately the prlim=
it
>>>> command requires specification of a real value for the number of fil=
es,
>>>> so use 262144 as the default value.
>>>
>>> Citation needed.
>>>
>>> prlimit -nunlimited
>>>
>>> prlimit --nofile=3Dunlimited
>>>
>>> both work fine, and strace confirms they issue correct system calls.
>>
>> Not on my test system:
>>
>> # prlimit --pid 734 --nofile=3Dunlimited
>> prlimit: failed to set the NOFILE resource limit: Operation not permit=
ted
>> # prlimit --pid 734 --nofile=3D262144
>> #
>=20
> What does strace say in both of these cases?

prlimit64(734, RLIMIT_NOFILE, {rlim_cur=3DRLIM64_INFINITY,=20
rlim_max=3DRLIM64_INFINITY}, NULL) =3D -1 EPERM (Operation not permitted)=

write(2, "prlimit: ", 9prlimit: )                =3D 9

vs.

prlimit64(734, RLIMIT_NOFILE, {rlim_cur=3D256*1024, rlim_max=3D256*1024},=
=20
NULL) =3D 0

>=20
>>
>>> Support for "unlimited" as a parameter has existed for the entire
>>> lifetime of the utility,
>>> https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e=
219dbcf62ec5ede84
>>>
>>
>> Yes, but not all systems seem to support raising the limit to
>> "unlimited".
>=20
> That's as maybe, but
>=20
> prlimit64(0, RLIMIT_NOFILE, {rlim_cur=3DRLIM64_INFINITY,
> rlim_max=3DRLIM64_INFINITY}, NULL) =3D -1 EPERM (Operation not permitte=
d)
>=20
> is a Linux issue, not a prlimit bug.

Nevertheless it isn't a good idea to use this setting in case it isn't
supported in all Linux distros/versions we care about.


Juergen

--------------3177C107245117017D2D69CA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------3177C107245117017D2D69CA--

--9tKOk0sK5O9AOfaD8O88T9OvYKplZuGB3--

--fsYiyn1EpBjwAz7vOI1QWRYrB6LQk0XFx
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFR2sgFAwAAAAAACgkQsN6d1ii/Ey9R
7ggAgbP8bF/SEhXVDUWx+SONIHoN1xOtjImFst3DTlXfGNgp1wbpJLM4+k914N2XZidIEDn886Om
D2qzGXIkP/HDMtKFNKzqqFu4kUP0icstUQci7VOrRxgq8hBFeHJzkGtBAl3qZJf7Aa0jZDKn3d1t
iV8CuLZSDN35zUZUusDndT0dwY1qzNuMEjPP8eqedS0XrrqA2pGYvfcO/66vgG+7wpdSCvzbKDsR
o7GzP9f85HhUiinB7adV94ksfs8FnmBah+LbPnAz8NyUj+G/C2CwiAPg3F1BhT2/3uguA7i61uOI
M1O3VtvNgn0TE8jKb2HhFCUh9ypPxMweyv74fk2lIQ==
=LclO
-----END PGP SIGNATURE-----

--fsYiyn1EpBjwAz7vOI1QWRYrB6LQk0XFx--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:54:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:54:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197046.349993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs1r-0006JZ-JF; Mon, 27 Sep 2021 14:54:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197046.349993; Mon, 27 Sep 2021 14:54:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs1r-0006JS-FY; Mon, 27 Sep 2021 14:54:51 +0000
Received: by outflank-mailman (input) for mailman id 197046;
 Mon, 27 Sep 2021 14:54:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g88l=OR=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mUs1q-0006JH-9a
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:54:50 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b6b6a64-15b7-4283-8096-c2ab2cdb3093;
 Mon, 27 Sep 2021 14:54:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b6b6a64-15b7-4283-8096-c2ab2cdb3093
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632754488;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=ZX0XlsrV/FGLqaNk70WhvfCFkjIFRFOBT9SCDpFUg/w=;
  b=GYeuN9XDCbQO0OmPOFMgPw8zS12zLo6J2EIIwluKOIVFEs3zh71ZIL65
   JjE20U1DVYPQCHaZw7gknSMaFsM+42sFlIo9+vKJ2ozgwhsbFWqHRwSEv
   VDD2nPqST69QNScdLJ4v46EAX/RM+355nBjdx51UugnGHziMINq1s2ThW
   M=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: j+2IFII3AUg0+7XFUt3deMqiV/LQMela43cwiMsBFKrCM95seulRsjzgIfyeWBJj22QhiI4lK/
 5BOQMXWWcPBAW2ZMr1/VgMf9HkSYdeBNy1nGhwglg1c+bH6KqRSCupJgbo3R2zLOAAYPLr46ea
 eRtxF/fZfjODLkz7PicYY9a4hA+QTKlehxEOYz/QHLmO4pWUaaXSjFexVRocnyFN5BDJUuf01/
 vGHvq5YL850e4ec61XI3BCXUCGuYVJeyHaGn+MD6rals4/M6WzlINEYR5zNTt+3971y89WIDeD
 ZdEd2YsSJlAdf3v0Ee4zraKL
X-SBRS: 5.1
X-MesageID: 54132478
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:DokkfKA5qw1/ahVW/9jkw5YqxClBgxIJ4kV8jS/XYbTApD1x1z1Uz
 DcYWTqDO/bbYGakf9p+Po3k9UxVvJ6Bn9ZnQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0007wobVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/jmmyxuhO+
 NJ2kaOvRS0NEbDLns0SakwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFgG5r3JwRTZ4yY
 eIbawBhaEjDcyF3I10RN60Cxdz0mVjWJmgwRFW9+vNsvjm7IBZK+KjgNp/Zd8KHQe1Rn12Ev
 STW8mLhGBYYOdeDjz2f/RqEhODVmjjgcJkPD7D+/flv6HWMwkQDBRtQUkG0ydG4lUyWS99ZM
 1YT+Cclse417kPDczXmd0Tm+jje5EdaAocOVb1hgO2Q9kbKywPFPTE8cQ5GU/gZq8McYRM30
 VmgkOq8UFSDr4apYX6a876Vqxa7Ni4UMXIOaEc4cOcV3zXwiNtt102eF76PBIbw3oWsRm2gq
 9yfhHJm74j/m/LnwElSEbrvrTW2utDtRxM56x/bV2aohu+STNX+PNT5gbQ3AOwpEWp4crVjl
 CRa8yR9xLpXZX1oqMBraL5UdF1Oz6zZWAAweXY1Q/EcG82FohZPh7x47jBkP1tOOc0ZYzLva
 0K7kVoPv8UNbCL2PPEnOdjZ5yEWIU7ITo2Nuhf8NIYmX3SMXFXfoHEGibC4hQgBb3TAYYlgY
 MzGIK5A/F4RCLh9zSreegvu+eVD+8zK/kuKHcqT503+idK2PSfJIZ9YYArmRr1ot8us/VSKm
 +uzwuPXkn2zpsWlOXKJmWPSRHhXRUUG6Wfe8ZcKKbLTflQ8SAnMyZb5mNscRmCspIwM/s/g9
 XChQE5Ijl35gHzMMwKRbX5/LrjoWP5CQbgTYUTA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:QNtaAq4mOYPdsS3OgAPXwViBI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwXJVoMkmsiqKdhrNhQYtKPTOWxVdASbsN0WKM+UyZJ8STzJ866U
 4kSdkFNDSSNykLsS+Z2njALz9I+rDum8rJ9ISuvEuFDzsaDJ2Ihz0JezpzeXcGJjWua6BJca
 Z0qvA33AZJLh8sH7qG7zQ+LqX+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+qemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lsdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNzN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wqJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABKnhkjizytSKeGXLzEO9k/seDlHhiXV6UkZoJlB9Tpa+CRF9U1ws67USPF/lq
 352+pT5fdzp/QtHNdA7dE6ML2K41z2MGDx2V2pUCDa/YE8SjnwQs3Mkf8IDN/DQu1+8HJ1ou
 WZbG9l
X-IronPort-AV: E=Sophos;i="5.85,326,1624334400"; 
   d="scan'208";a="54132478"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OcjW7GQmw2lWoyWZsYWNXWMYzxprJ5513cH93mkD8TLqJ23yI9q6Od9dgxjdQh0SE7iCg4jTMcPArIFtm4mg9CGmGXOWumZEiPr5ILhhYoMHvh0BwKNQbfgu8P9pNh6Tbz10KBDWiHr0TyystNvrFj5ggnvvHJhger2Ttu+yEnAYsYyXqiynjfNXqMalzCFa2bPYy0UwvH89MHsFYzug1GCDZmy97uFzXX7/Zk1mazKWNjvu20DmggtbEy3/t7/D5FY8W3Sk5ux25F9io6QDkmwogh77wZKr5TderfG72Y9oZbTp44DPHI5eqsAXOibzEnZCPtb7DGQK+wJwww0qQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZX0XlsrV/FGLqaNk70WhvfCFkjIFRFOBT9SCDpFUg/w=;
 b=DqUCXkvDAPjrYINi9z+SPlhAa2pOix9NvBwlRz5Gp34gzf6Ongveshk3b4jW9YhnPDOi+5MXjjYSeF0QuKIhd2/fPpT+SFSqgHsTpkv6onNZuYEOXcEynSXttvPUChqCqClmKvM2Ksiv97PvfBp4oAY05oZ3Qf3ulsW8Ud+gZ3sVSalBQUw/xVU4X3/X5/ZHWXuxv1A2SabLw2vkqrbRvf5EgXHaoC5O/lRciTwAOP+K2c5Irdij5BMYXcrZJJw1YCk7n0JUcoY5RW2EJFruTPGFCmS4NrfdmTBSkOb4VVUTv3hxaVXIvOg5ccBmV3qXb1p3aZZnDr7bGXUrVBaPKw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZX0XlsrV/FGLqaNk70WhvfCFkjIFRFOBT9SCDpFUg/w=;
 b=EFoLvUQuT1kCs50GfCHPhi4b4jy2pyIIgnvH1fGGcb9mURtbc25M+Y9w/N07O9HDomOoa4Wc1nVkCK9SG3loQHBg+ek+c5IzIfRUbmzkPK9HwCheHcHQwzIZst8A3xXl3kBZrvNrv8TsC+pJtytLZ/YnV6NqFgmKSl6SWzMQF0U=
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
To: Juergen Gross <jgross@suse.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
 <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
 <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>
Date: Mon, 27 Sep 2021 15:54:19 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0161.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:188::22) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7c4fa0cc-4177-46c4-b3ba-08d981c6b2d0
X-MS-TrafficTypeDiagnostic: SJ0PR03MB6357:
X-Microsoft-Antispam-PRVS: <SJ0PR03MB63577BD4048B5120DDDBE505BAA79@SJ0PR03MB6357.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: F5YE4hEOBedkGftBQE+KH9xLRRgjTyF6Vc+fYERMELTucQ42HvupUoe3IKNYcDwU5nBk2c00qpQbK6Gxc4ckzXdEy2809cFqJE6px+vUvBu/8jUet1TExEPCNpHh4B/CSPA8+fK3lUAu4WRsrn6rYtgqqt9Uqg73HJuVdY4zln39jLr/tOcM6a5lkMPAFfhTvkpZ0rNu8pAqtLQaCndafnwran+S9rdKLBhQlyaoUlUZqACb9U77HZqeyNvMG4vYo1c34IlNOiKwjvMOQD9VmgtDfGnjv6W9tq8nm2kIIJQLxM6YKpoXpAmZVJ2CbDd7jugEgq6UOA9H94y/UTxaK6gHTJ27hgHYK+Q06T11yDMR1/SOgwI3PLJNNPqXlDpxRjRa19lhw/3c6EYhi9D83g5/NsCA7+BcawV1c178K3dfYtmnKcGOrVh+j1aqNHZ/gUCiSOBfU7XpswqJpBVyQPOvXFcgJzAnuKZSGnD5gSvL1+H8XaA58ynvPNl3TsPC3GH/ldBb3ZAHycUPGDiPXn3M7OLjxO8WH9aTlMIIhqPwRXOaItXjaCSdFLKmBECXL9A4ay1DjseBdnpA5a+tHTzT96NoyVCLBZG9qcdQE7vei2thDyap1yBf8vJc+xCsGbDCynMFzosxOChh3ny0Ij35Sz7vBJy15z32zPI/GfpkRQ3psbIeV6w5l4rcbcPpP+EqYTc7LxitOfl45MMjORCxrydZu0eriM6QetfqMyMaTOKyl7ZtXkQLUIk90VDNNsmmOoSxJ+TwzZtZ4ZxGRGxdNuRuemvRhqMi9S4TJxSmskduflr+q0Sq/uYMIyIzcpy2TKuDPgG8GuthgUzDMQ==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(6486002)(31686004)(8676002)(54906003)(966005)(66946007)(31696002)(186003)(5660300002)(83380400001)(956004)(53546011)(86362001)(55236004)(2616005)(6666004)(316002)(2906002)(36756003)(16576012)(26005)(66476007)(4326008)(8936002)(66556008)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bTY4NVVGcjBnOE13azFqL3RTWktSWE9ESm5wZ1pmZkRQMmpXSGdqZG1lT1Nx?=
 =?utf-8?B?SDc2UHdUN3p3cis3VGhPem1wMW8xeGV2ZVBPNmRvZU1uNCs1NVhCeDVnU2ZY?=
 =?utf-8?B?dGdzeUhHdmdIZ1E4bTl1YzlCL1NwQlRwZ1Q3aVNuUEhNL1I3aWFuV28rM3lD?=
 =?utf-8?B?SjNReCtjNnh3NmJwdU5EekwwL3dvUEFGMUpFWEcyQXFaVWp1ZjdwZ1lQQmN5?=
 =?utf-8?B?WUN0WXFjN1ErbU5ZUTJ2VktMZzdJdmw1OCtHRGIwdXROWnJ6VUpQeUg1dUkr?=
 =?utf-8?B?T1RiK2VMdXAyWVZTUTkrU1JXdFd2ZXVmd0xwT3UwMG00em15R1RpSmtPdzFE?=
 =?utf-8?B?WGRRSEtTY0FpVzZ6UkFLZkRpdEx4c3N6djF4U1BTZXR5Z1oxRDJXeWprNlVq?=
 =?utf-8?B?ZFVsN1VYTGdxL2lYNTdWNS9FQy9Od1p2dE1RWE9BdVBWSGJPTXBEMkN5SVE3?=
 =?utf-8?B?QUV3QXlVMU1lNDJmU21qMXorKy9KdDlRYmF3L2hmY0F5WCsyRFF4M1MyUHli?=
 =?utf-8?B?YVNIVk5qU3pNOWtZbCtlbWlhS1p6RXdlQWp0WEtGMUFCY1JpQS9IUUpvVDlq?=
 =?utf-8?B?SnlFQ1BGR2JXQmtoTDlGbUdreUpLN3oxL1hlc1ZiZ2pDNERud2ZBclFheHlF?=
 =?utf-8?B?Y1NJVzJuTUtJMlUyU1IvZVZjdGJnVnErc3RPQjBpQlhuci9DVTNCZlo4NVE3?=
 =?utf-8?B?dHJId0krOEhLTHFldUMySFoxRW56M21NOVJVS291NElrRFR6c3g2T0ZiSFdw?=
 =?utf-8?B?RmxmSHVyNmpQQVJoUjM5bnFJek9sekRDbGl6SHZqS3BEZGlHNVRRVSttUURq?=
 =?utf-8?B?WXpEelNodU84RHVQNm9PMDlhTVVjeWRXbHhyTUJsc1pmbUdBSXRCbHo5Qy9w?=
 =?utf-8?B?U0k2R3lCdm5QbjMzeFRUMVJhc1IvbGQ5VTZheTFaYUpSN0FHeUxXcXNmNzU3?=
 =?utf-8?B?eE1uTWx2MkpJVEtCSjVZSUY1ODlMaE9LdCtxNUpxSkFTclluQklkUG9yZ3dx?=
 =?utf-8?B?bzZrUVhXMzdLeFpCak50UW9NTU5HS1lyRXpnNUx0YUdEck9mOUFoSTM5OVl0?=
 =?utf-8?B?RVpJNEZMNmlIOW9YLzJ4ek1WQytxb2JPd3NhV2d5aHZpT1JzengxZmN5T3Rj?=
 =?utf-8?B?WEllTm1ZMTNER2xXdG55aFdGdWlmUGRBVTN3ZEZLQ2VCU2F4UmxDK0FheXFo?=
 =?utf-8?B?ZnhubHV1cXA3VXZrM1k5NWNvV1pkWWNZd05mTldoTmt0ZnRUbzJ6KzJsTHdX?=
 =?utf-8?B?eW1tN1lKRW42NStkaW1RL1NadUJscFZNankvTUY2NjA4S3hvT0NQaGhSQ3hp?=
 =?utf-8?B?UU9lOGtFNmsraUZYUjc2dEppV0FDRFdPWS9wQVJORldkYUg2V1JlMkhSanlC?=
 =?utf-8?B?aDZRbDhRZWwydDdRWHg4UStydms5V2EzWXVia1ByV1BDdktzQXg2UmRJVmNM?=
 =?utf-8?B?R1VtcG5UbGxkU0ZnTTdjRjhYaS95cFdqYXVYeHgzSE55Yk5LT2lheTVJQnYx?=
 =?utf-8?B?OXA3ZFd5ZjlQdmIvbUl1M3E0cEZ5cC9nMWtCaXEvRUo1THQ1MmhudmtMUDBG?=
 =?utf-8?B?WEEwdVpzdDFZQm9NUk9vSHlGTUZDOUYxU2VzRWlBYlZpcE9KRlFYdzZVZlhM?=
 =?utf-8?B?RkFDQjNMcGY0dTVWazNoNkQxOE1ORXY3Qzg0ZmQ1d3NlTk9sREk3U21KbUJa?=
 =?utf-8?B?YzVxR3YybUNLYWQ0ZUx0OEJBTGJlNzBKQjlJYWNaVkpnYWNRUlI3RExQVE55?=
 =?utf-8?Q?inRcLj9LyT0EAccieMipDFJmiO5RGHebzms1fq4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c4fa0cc-4177-46c4-b3ba-08d981c6b2d0
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 14:54:25.4025
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HF71qufRq8MHlorcmYLOS6Hm5243rDUCC/+BOFPlXvBkTn60fTIBwO7kJYjrwWDQ8pmZGx0Fauw6fnE3YghcKY7ZKW6ulufMpAdvDOugxRA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6357
X-OriginatorOrg: citrix.com

On 27/09/2021 15:52, Juergen Gross wrote:
> On 27.09.21 16:40, Andrew Cooper wrote:
>> On 27/09/2021 15:24, Juergen Gross wrote:
>>> On 27.09.21 16:13, Andrew Cooper wrote:
>>>> On 27/09/2021 11:48, Juergen Gross wrote:
>>>>> Add a configuration item for the maximum number of open file
>>>>> descriptors xenstored should be allowed to have.
>>>>>
>>>>> The default should be "unlimited" in order not to restrict xenstored
>>>>> in the number of domains it can support, but unfortunately the
>>>>> prlimit
>>>>> command requires specification of a real value for the number of
>>>>> files,
>>>>> so use 262144 as the default value.
>>>>
>>>> Citation needed.
>>>>
>>>> prlimit -nunlimited
>>>>
>>>> prlimit --nofile=unlimited
>>>>
>>>> both work fine, and strace confirms they issue correct system calls.
>>>
>>> Not on my test system:
>>>
>>> # prlimit --pid 734 --nofile=unlimited
>>> prlimit: failed to set the NOFILE resource limit: Operation not
>>> permitted
>>> # prlimit --pid 734 --nofile=262144
>>> #
>>
>> What does strace say in both of these cases?
>
> prlimit64(734, RLIMIT_NOFILE, {rlim_cur=RLIM64_INFINITY,
> rlim_max=RLIM64_INFINITY}, NULL) = -1 EPERM (Operation not permitted)
> write(2, "prlimit: ", 9prlimit: )                = 9
>
> vs.
>
> prlimit64(734, RLIMIT_NOFILE, {rlim_cur=256*1024, rlim_max=256*1024},
> NULL) = 0
>
>>
>>>
>>>> Support for "unlimited" as a parameter has existed for the entire
>>>> lifetime of the utility,
>>>> https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e219dbcf62ec5ede84
>>>>
>>>>
>>>
>>> Yes, but not all systems seem to support raising the limit to
>>> "unlimited".
>>
>> That's as maybe, but
>>
>> prlimit64(0, RLIMIT_NOFILE, {rlim_cur=RLIM64_INFINITY,
>> rlim_max=RLIM64_INFINITY}, NULL) = -1 EPERM (Operation not permitted)
>>
>> is a Linux issue, not a prlimit bug.
>
> Nevertheless it isn't a good idea to use this setting in case it isn't
> supported in all Linux distros/versions we care about.

Ok, but at a minimum you need s/prlimit/Linux/ in the commit message.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 14:57:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 14:57:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197054.350004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs4b-00075f-5D; Mon, 27 Sep 2021 14:57:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197054.350004; Mon, 27 Sep 2021 14:57:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUs4b-00075Y-1W; Mon, 27 Sep 2021 14:57:41 +0000
Received: by outflank-mailman (input) for mailman id 197054;
 Mon, 27 Sep 2021 14:57:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eI72=OR=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mUs4Z-00075Q-Vr
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 14:57:40 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d0a3453b-9b7b-4f51-b009-4f1fb6563aa8;
 Mon, 27 Sep 2021 14:57:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3FD72221D8;
 Mon, 27 Sep 2021 14:57:38 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1DF9414007;
 Mon, 27 Sep 2021 14:57:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id imIXBuLbUWF3SgAAMHmgww
 (envelope-from <jgross@suse.com>); Mon, 27 Sep 2021 14:57:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0a3453b-9b7b-4f51-b009-4f1fb6563aa8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632754658; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=9f+e12+t4toTAQB2zgPoQ2ut+S5PWfdy5EFiT3SqVGs=;
	b=tLx3mCMKsZB2UCjoQ3oDtxDQ/AmmlsB3OHowUzaaf953cpIzWrJcPQ2l5zjvFVQ31Tu8tg
	Tl0TouMCM564KhUFUwbrqvYeWlYnKd8wI/B4ZQvHmsC2Uc46UFDIc+ghxv0mdxs10yjAtz
	q+s+m6pPKG3hEZSjHOc0n8oHo6ChdMA=
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
 <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
 <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
 <8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <e56b9881-5a82-0180-765a-102fafab13ed@suse.com>
Date: Mon, 27 Sep 2021 16:57:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="rP49qt83LZhsQV5Kjj8Ch7nP7Qzsixxs2"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--rP49qt83LZhsQV5Kjj8Ch7nP7Qzsixxs2
Content-Type: multipart/mixed; boundary="YBYEeWogkslpCsejnnM3Y4EV3mYdphfbI";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>
Message-ID: <e56b9881-5a82-0180-765a-102fafab13ed@suse.com>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210927104813.19772-1-jgross@suse.com>
 <20210927104813.19772-3-jgross@suse.com>
 <f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
 <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
 <9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
 <825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
 <8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>
In-Reply-To: <8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>

--YBYEeWogkslpCsejnnM3Y4EV3mYdphfbI
Content-Type: multipart/mixed;
 boundary="------------FF3089DB97FAA0C19F2C33AF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------FF3089DB97FAA0C19F2C33AF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 27.09.21 16:54, Andrew Cooper wrote:
> On 27/09/2021 15:52, Juergen Gross wrote:
>> On 27.09.21 16:40, Andrew Cooper wrote:
>>> On 27/09/2021 15:24, Juergen Gross wrote:
>>>> On 27.09.21 16:13, Andrew Cooper wrote:
>>>>> On 27/09/2021 11:48, Juergen Gross wrote:
>>>>>> Add a configuration item for the maximum number of open file
>>>>>> descriptors xenstored should be allowed to have.
>>>>>>
>>>>>> The default should be "unlimited" in order not to restrict xenstor=
ed
>>>>>> in the number of domains it can support, but unfortunately the
>>>>>> prlimit
>>>>>> command requires specification of a real value for the number of
>>>>>> files,
>>>>>> so use 262144 as the default value.
>>>>>
>>>>> Citation needed.
>>>>>
>>>>> prlimit -nunlimited
>>>>>
>>>>> prlimit --nofile=3Dunlimited
>>>>>
>>>>> both work fine, and strace confirms they issue correct system calls=
=2E
>>>>
>>>> Not on my test system:
>>>>
>>>> # prlimit --pid 734 --nofile=3Dunlimited
>>>> prlimit: failed to set the NOFILE resource limit: Operation not
>>>> permitted
>>>> # prlimit --pid 734 --nofile=3D262144
>>>> #
>>>
>>> What does strace say in both of these cases?
>>
>> prlimit64(734, RLIMIT_NOFILE, {rlim_cur=3DRLIM64_INFINITY,
>> rlim_max=3DRLIM64_INFINITY}, NULL) =3D -1 EPERM (Operation not permitt=
ed)
>> write(2, "prlimit: ", 9prlimit: )=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 9
>>
>> vs.
>>
>> prlimit64(734, RLIMIT_NOFILE, {rlim_cur=3D256*1024, rlim_max=3D256*102=
4},
>> NULL) =3D 0
>>
>>>
>>>>
>>>>> Support for "unlimited" as a parameter has existed for the entire
>>>>> lifetime of the utility,
>>>>> https://github.com/karelzak/util-linux/commit/6bac2825af7216c547114=
8e219dbcf62ec5ede84
>>>>>
>>>>>
>>>>
>>>> Yes, but not all systems seem to support raising the limit to
>>>> "unlimited".
>>>
>>> That's as maybe, but
>>>
>>> prlimit64(0, RLIMIT_NOFILE, {rlim_cur=3DRLIM64_INFINITY,
>>> rlim_max=3DRLIM64_INFINITY}, NULL) =3D -1 EPERM (Operation not permit=
ted)
>>>
>>> is a Linux issue, not a prlimit bug.
>>
>> Nevertheless it isn't a good idea to use this setting in case it isn't=

>> supported in all Linux distros/versions we care about.
>=20
> Ok, but at a minimum you need s/prlimit/Linux/ in the commit message.

Okay.

Should I send out another version, or can this be done when committing?


Juergen

--------------FF3089DB97FAA0C19F2C33AF
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------FF3089DB97FAA0C19F2C33AF--

--YBYEeWogkslpCsejnnM3Y4EV3mYdphfbI--

--rP49qt83LZhsQV5Kjj8Ch7nP7Qzsixxs2
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFR2+EFAwAAAAAACgkQsN6d1ii/Ey9a
Vwf9EEUPmEjMf1cT3riBeRdXTo1Xn9URWtSaPiQ3SX0fIVQ0b6lYFZkEBVgDb8Di/M9m0kzoUISE
gNW0ltrQhFkKeJOhUuXGm5ohGTOOxU0OJNAKFkNSJXFRNdsU40g1y9DDOE+ct8n2zdJC1tWbGCfp
oeXo7ZfiP3IbQb07XMsrCRl941zBRW5jXPrUo3JOzxH8q5DHosOixefTgcr7NrYtfX6VW5B8EhKX
DNW1YLCdWTGHfWv+q/7dHv2vvkJQVlTJ1S8rpgYvTh8uaoq5p4HLcuydPej5PckvwsYOZbrZ6/5T
YQWTHO2bFVymBFhUPhUphHr6ftxL1VtSdyGuV2S0MA==
=9/Ez
-----END PGP SIGNATURE-----

--rP49qt83LZhsQV5Kjj8Ch7nP7Qzsixxs2--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 15:12:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 15:12:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197067.350014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUsIg-0001En-EN; Mon, 27 Sep 2021 15:12:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197067.350014; Mon, 27 Sep 2021 15:12:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUsIg-0001Eg-BB; Mon, 27 Sep 2021 15:12:14 +0000
Received: by outflank-mailman (input) for mailman id 197067;
 Mon, 27 Sep 2021 15:12:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsIe-0001Ea-Rr
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:12:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsIe-00024P-41
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:12:12 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsIe-0000NB-31
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:12:12 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUsIZ-0000r5-AF; Mon, 27 Sep 2021 16:12:07 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=ZXOZy8ChzbHKuQbO2lzAzt/6jKad5k1rrr19sLbwr8U=; b=n48xGhdjLJmzuQv94o/aulkf0b
	85DcGIbv7vymYhkXylwfvsJVpvknpd9+Zfko0d5M9lroXpe/8RL/FPh5IKz6GIB/WANmqgD4NCFGv
	gvyjaB5TTCgCaDUHhHUeR7FMO+ySY05Bpr6BCGv++4SE77mb+zphxDM/dZnTYigkVyTc=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.57159.98506.99705@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 16:12:07 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
    xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
In-Reply-To: <20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
References: <20210927104813.19772-1-jgross@suse.com>
	<20210927104813.19772-3-jgross@suse.com>
	<f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
	<20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for xenstored"):
> On 27.09.21 16:13, Andrew Cooper wrote:
> > both work fine, and strace confirms they issue correct system calls.
> 
> Not on my test system:
> 
> # prlimit --pid 734 --nofile=unlimited
> prlimit: failed to set the NOFILE resource limit: Operation not permitted
> # prlimit --pid 734 --nofile=262144
> #
> 
> > Support for "unlimited" as a parameter has existed for the entire
> > lifetime of the utility,
> > https://github.com/karelzak/util-linux/commit/6bac2825af7216c5471148e219dbcf62ec5ede84
> 
> Yes, but not all systems seem to support raising the limit to
> "unlimited".

This is strange.  Are you running in some kind of restricted
environment ?  <strike>Or maybe prlimit is trying to adjust the soft
limit (only) but failing to remove the hard limit too ?  I confess
that I never use prlimit, just ulimit.</strike>

I just tried this on my laptop:

root(ian)@zealot:~> ulimit -Hn 1048577
bash: ulimit: open files: cannot modify limit: Operation not permitted
root(ian)@zealot:~> ulimit -Hn 1048576
root(ian)@zealot:~>

????

1048576 is 0x100000.
1048577 is 0x100001.

The intertubes caused me to check sysctl fs.file-max (1591013),
/etc/security/limits.conf (nothing uncommented).  Eventually a helpful
person pointed me to /proc/sys/fs/nr_open.

root(ian)@zealot:~> cat /proc/sys/fs/nr_open
1048576

This is completely deranged.  The internet is full of people working
around this (if you are unluckloy you try to set nofile unlimited in
some file in /etc/security and then you can't log in!).

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=60fd760fb9ff7034360bab7137c917c0330628c2
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c2d64fb6cae9aae480f6a46cfe79f8d7d48b59f

^ that explains why things are like this.  Oh woe is us what madness
have we collectively perpetrated.

I suggest the following workaround for our scripts: try to read
/proc/sys/fs/nr_open and use the value from there if there is one;
otherwise use "unlimited".

I think that is better than sort-of-guessing 262144.  What you do you
think ?

Thanks,
Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 15:19:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 15:19:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197077.350025 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUsPd-00027G-49; Mon, 27 Sep 2021 15:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197077.350025; Mon, 27 Sep 2021 15:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUsPd-000279-1E; Mon, 27 Sep 2021 15:19:25 +0000
Received: by outflank-mailman (input) for mailman id 197077;
 Mon, 27 Sep 2021 15:19:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsPb-000273-JD
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:19:23 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsPb-0002Dn-DB
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:19:23 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mUsPb-0000yv-B4
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 15:19:23 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mUsPW-0000t9-Hb; Mon, 27 Sep 2021 16:19:18 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=C0Jllf5TlpxccxZtyP4hirSV3tpXHtXnYRJGy/CTvKs=; b=s8Y35rwq5XWlojd5YXj/FEBGsu
	bSunfPlrX4KqOlk+gsxeQJ51BYgQrPRpDU9c1TosUgRVqlKlnd6eE7L4fiarQ83a8Hp0kmSCFRfBQ
	hZPnszH95rtyuzuU/sv7/tnYBfQ7kS08Gk8jAJ4l4x0ChMMg6CX72BoIOjZ1zjivF6qc=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24913.57590.165404.759779@mariner.uk.xensource.com>
Date: Mon, 27 Sep 2021 16:19:18 +0100
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
    xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
In-Reply-To: <e56b9881-5a82-0180-765a-102fafab13ed@suse.com>
References: <20210927104813.19772-1-jgross@suse.com>
	<20210927104813.19772-3-jgross@suse.com>
	<f858dda9-b844-48e4-c079-75296cc36316@citrix.com>
	<20bd7f1f-f0fc-b7a6-c0e8-05192714d384@suse.com>
	<9076ecc3-8509-f687-df11-82a561cb5b2d@citrix.com>
	<825e7a37-c39d-4cea-0992-268945ad118e@suse.com>
	<8d51e78c-e8b5-c65a-d130-cc0e527f9f92@citrix.com>
	<e56b9881-5a82-0180-765a-102fafab13ed@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for xenstored"):
> Should I send out another version, or can this be done when committing?

In any case I think you should send another version.

If you don't agree with my suggestion to check /proc/sys/fs/nr_open,
then you could usefully explain in the commit message why we're not
doing that.  If you do agree please include the links from my mail.

Ian.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 16:21:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 16:21:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197098.350040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtNf-0001gu-PZ; Mon, 27 Sep 2021 16:21:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197098.350040; Mon, 27 Sep 2021 16:21:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtNf-0001gn-LW; Mon, 27 Sep 2021 16:21:27 +0000
Received: by outflank-mailman (input) for mailman id 197098;
 Mon, 27 Sep 2021 16:21:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lDhJ=OR=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mUtNe-0001gh-He
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 16:21:26 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.40]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f4e32f3c-1fae-11ec-bc53-12813bfff9fa;
 Mon, 27 Sep 2021 16:21:24 +0000 (UTC)
Received: from DB6PR07CA0161.eurprd07.prod.outlook.com (2603:10a6:6:43::15) by
 AS8PR08MB6168.eurprd08.prod.outlook.com (2603:10a6:20b:294::9) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15; Mon, 27 Sep 2021 16:21:12 +0000
Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:43:cafe::b1) by DB6PR07CA0161.outlook.office365.com
 (2603:10a6:6:43::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Mon, 27 Sep 2021 16:21:12 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Mon, 27 Sep 2021 16:21:12 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Mon, 27 Sep 2021 16:21:11 +0000
Received: from 788ae38cee15.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 27B243E0-372B-49F7-9233-4ABC01622023.1; 
 Mon, 27 Sep 2021 16:20:57 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 788ae38cee15.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 27 Sep 2021 16:20:57 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0801MB1745.eurprd08.prod.outlook.com (2603:10a6:203:3a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Mon, 27 Sep
 2021 16:20:54 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 16:20:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4e32f3c-1fae-11ec-bc53-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FaMylyagjMBBWy5Z3AyOT51E7F76IMhGQYfHVy7jJIo=;
 b=IziVwc9wMlmIPUOPpRWyNq3S6kDIIlo1gqMAmPR4O+r+oEqguF3twLFz31ffwJ/jU8ic7X6piPyLVOAGquvJWCPcOb51sLxgFB/zQeZfg6VwFUoxA3e0yzXm/n9YCjP1am63NqKMFFStAmpghZRY2/vz8a8+KJNxLuPh66SVGUU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 11cc5260630cd775
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UWBS64EJvKlfS61dc7HsfQB8KFqVFo9AD+3u3f88uLj3WYlwKX1Lu1gFFPch/nxryREnl2ALvGYuHMUS/UKTrgT3pM4FKlAgiuonWX4vCFxOGzEYXOiUCf4TWLCddfqYAzhPfey1yLAbjS3794oA5PW6netw+TySEHoW7AIVhBC+CYJni3Cju6udNHIrerlDI31LGQzWC7+qHUps9M3ZOBmYtLDDR6HzEKF14EEJ/oh5pNNvUxDfPISjI4KWA6J7U6I0okLGorcYVEBg058L31TujAG2P53He6GaDQ9y4CbfIAEGMLyvP1qanNycr2zDi3HCuk1LO92UXvvxyy6oFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FaMylyagjMBBWy5Z3AyOT51E7F76IMhGQYfHVy7jJIo=;
 b=U3cwxv1s9rLoZV1XhvslQcMe2sqxF7y8Fks4v5Hga36j7YO71747NMf0F6F6/EV7ou5f4kmVQpBDUuH5u2bqVXjo4bLKqetUZgBApvuPrlOQ3PpV+2mPD9lO5HtmgWNA4vb8RCZPq+9NSYkB/Fd4Wj02sBQEdTbRxeHumpJBKQYLUUVA9B3TdkCkYh+ts88+vtxYzqIwzdnUERBDSAX3hFfRGwnQA00ESjUnvTgCEs4PhfskiDDyvx5VWob92d1TDX+IJ/HA49ba3laOTK1vTUW/49gbSy8qiZDI3VEGtXqm2KrumDuExLdFc49irYvP+EBZ3Vc1qFPYyKBDoE6YSg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FaMylyagjMBBWy5Z3AyOT51E7F76IMhGQYfHVy7jJIo=;
 b=IziVwc9wMlmIPUOPpRWyNq3S6kDIIlo1gqMAmPR4O+r+oEqguF3twLFz31ffwJ/jU8ic7X6piPyLVOAGquvJWCPcOb51sLxgFB/zQeZfg6VwFUoxA3e0yzXm/n9YCjP1am63NqKMFFStAmpghZRY2/vz8a8+KJNxLuPh66SVGUU=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: xen-devel <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index:
 AQHXr6aPsmXcRwLbA0y2fuTM/iBxWauw4OaAgAD7DQCAACKOAIABKMWAgACTbwCAAB1PAIAEP/YA
Date: Mon, 27 Sep 2021 16:20:53 +0000
Message-ID: <AEC72EBB-90D9-41F8-A67E-661F229F356C@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
 <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com>
 <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s>
 <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com>
 <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109241619270.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241619270.17979@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 2cb030c7-c90f-46d0-a82d-08d981d2d262
x-ms-traffictypediagnostic: AM5PR0801MB1745:|AS8PR08MB6168:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB6168E2F5AFD780EFE9FFA2C9FCA79@AS8PR08MB6168.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 JlGdSPeROV9ty6kREBP/v3cJvubdyT6/t7cXlkYRaflUAfbc03eZ1sRHGcYSFZwxlV6GU6yoo88XmIvGgF0pIjbHcLnR48ndR3Yp3XXGPSGrvYc+okZAzBtfOGE+AnWHCmjui22AffGWacWbZ3W6QxWpKMPYAORgmQy9gF+uVd5tsE1Q3DGG39yQNP3nGT5HtAycOZ3N2RRzBxNOluLipfhadnRuIFRzsG2OqTPDkkfaV8prf+q8bfaUf6i5vNjXzE/tWV9Zq7sspWM6ssuL2a1MVPoL/0i0DLJu0iuboZQnbogO5u0XLGHwmIBh7f82V1IcZWWgcz1++JWJaI1s8yjDCsAuY9hJgnPVxRKKrxK++X+xEdBd5Z+6v9LdQClt0CDLXNtAc2i3vY6Ys6tUMxBnLo/2koJBI8Mad5259XXxc2OKJkwAtCApEaAVAycyZxjOR/hnp+QVDTbMvxBSrt3huZaydgb3jZohJZW0FlAq3+t0Nxity+s4l0iUo2xhb5ezaOF+8GDSU3VM/+0Z8DfRMKLys7f75YX4q8hP1NSFt9ocqvk+GLo+pgEm2TRB2R27y/a4ZmhHGAWVUIzO8W2V9AKGKYOmb5jcb3bXe7/wnPzmjo+CPmgFAiMhdSRQDmb++2Zc0YWFRkvO56ICWyGbKdCG0pvv966XWBNg4TweRCLbUBHeoQOBC7LD6Z0hYezwqMRZPY3bzFJG9lyGH3p6GSnB4BkPv8z9O1yWfpeEEADh99xwVXJ/58Fx4VesKz7a276qcfFepSWylB8HCqeh+zUq4a65KsMuRuOQghtz5jVSP+CNiIgf2SLZdqQb6ur1TjPa7x4eGaDg8xO5Rg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(91956017)(2906002)(6486002)(66556008)(5660300002)(66446008)(66476007)(508600001)(86362001)(71200400001)(76116006)(316002)(6506007)(186003)(64756008)(66946007)(38070700005)(4326008)(6512007)(38100700002)(122000001)(6916009)(36756003)(54906003)(26005)(53546011)(8936002)(966005)(33656002)(8676002)(2616005)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <567123CCE58923469B71580E4F538D86@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1745
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	dd12ada9-523a-4cfd-3d4e-08d981d2c774
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UlS9NdyIN1dWAG9v2GZ1q3qbBkW+VOdM+WqPBvz5wnMNxAdRj31Y1TAGCP5e1s6U8nCiR08x2Zks0HqsZHKsVfUWqmwPUgYksa8gS4bg3wPfv74QhPQy//tbXDajQYep76LT3Nf6YTS26xr4RqQKdEsnasaZueBIgMfVkRS08c29m/3lgL+xnoV7H+mqEZDzj3bl6SvSXtNTVQDne+xE+Y2tfCmltMVNZPKqdkkFGtEY/erD4udaldry6BSqmqpeqOuTRzvmwU/8MQUksXUJimpb+bx9YkLV0LGCKbCDW/OxFmoKsg3rYPsztrHvaK/+su6z7qJswsG9H5wShXWArlf8CNoctNrwVcugjrhM2+BmoZfIJQjhHt7dhQWEQFN6+TsfnnnQR6LmNvWGxMa4GWhfrKsJP2Y9cfaaDkqZNwKLK9j7Rlbd//eQvYwXax10YwsjFxiMBS2o2LxZIH7oX8AapvQ0j/4UqbgMBvjdzTilmIpnUMtglBsq5kGZghwSPiN1Ebk5sNKjSElI6PRPo6RaF3FGePZizl4zlA34BAAkuy3Yw9Vxsf4hMAkmk5Z49OcnF7gIhNwaG2vERmdPIXts0SGznet+ZhII6HuBrSAHMChrjGQ53VZNsRISXhAMjohf1OkvVKxh3Zwc1bD2m4dLxLCBBJ84ZWH6dtG8HetTDh/jUnKESp1NSs9ONjTGvsWPhYemxEn9rsvUmFIJ7u8Amraju6BNA0mkpWubq2G0VL0YKDlNnCxkk+UYT2ngHjuPC0CEfQpHrzgx4uRu66TyRGbEzdyGzx9AcRUXECE=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(6486002)(2616005)(53546011)(6862004)(26005)(5660300002)(70586007)(186003)(316002)(70206006)(36860700001)(8676002)(33656002)(6506007)(8936002)(336012)(36756003)(966005)(4326008)(107886003)(86362001)(6512007)(54906003)(82310400003)(2906002)(81166007)(356005)(508600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 16:21:12.2352
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2cb030c7-c90f-46d0-a82d-08d981d2d262
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6168

Hi Stefano,

> On 25 Sep 2021, at 12:26 am, Stefano Stabellini <sstabellini@kernel.org> =
wrote:
>=20
> On Fri, 24 Sep 2021, Stefano Stabellini wrote:
>> On Fri, 24 Sep 2021, Rahul Singh wrote:
>>> Hi Stefano,
>>>=20
>>>> On 23 Sep 2021, at 8:12 pm, Stefano Stabellini <sstabellini@kernel.org=
> wrote:
>>>>=20
>>>> On Thu, 23 Sep 2021, Rahul Singh wrote:
>>>>>>> +            goto err_exit;
>>>>>>> +    }
>>>>>>=20
>>>>>> This is unnecessary at the moment, right? Can we get rid of ops->ini=
t ?
>>>>>=20
>>>>> No this is required for N1SDP board. Please check below patch.
>>>>> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d5=
7547087cff4ffc078dc515d5
>>>>=20
>>>> OK
>>>>=20
>>>>=20
>>>>>>> +int pci_host_common_probe(struct dt_device_node *dev, const void *=
data)
>>>>>>> +{
>>>>>>> +    struct pci_host_bridge *bridge;
>>>>>>> +    struct pci_config_window *cfg;
>>>>>>> +    struct pci_ecam_ops *ops;
>>>>>>> +    const struct dt_device_match *of_id;
>>>>>>> +    int err;
>>>>>>> +
>>>>>>> +    if ( dt_device_for_passthrough(dev) )
>>>>>>> +        return 0;
>>>>>>> +
>>>>>>> +    of_id =3D dt_match_node(dev->dev.of_match_table, dev->dev.of_n=
ode);
>>>>>>> +    ops =3D (struct pci_ecam_ops *) of_id->data;
>>>>>>=20
>>>>>> Do we really need dt_match_node and dev->dev.of_match_table to get
>>>>>> dt_device_match.data?
>>>>>>=20
>>>>>=20
>>>>>> data is passed as a parameter to pci_host_common_probe, isn't it eno=
ugh
>>>>>> to do:
>>>>>>=20
>>>>>> ops =3D (struct pci_ecam_ops *) data;
>>>>>=20
>>>>> As of now not required but in future we might need it if we implement=
 other ecam supported bridge
>>>>>=20
>>>>> static const struct dt_device_match gen_pci_dt_match[] =3D {         =
            =20
>>>>>   { .compatible =3D "pci-host-ecam-generic",                         =
          =20
>>>>>     .data =3D       &pci_generic_ecam_ops },
>>>>>=20
>>>>>   { .compatible =3D "pci-host-cam-generic",
>>>>>     .data =3D &gen_pci_cfg_cam_bus_ops },                            =
    =20
>>>>>=20
>>>>>   { },                                                               =
        =20
>>>>> };
>>>>=20
>>>> Even if we add another ECAM-supported bridge, the following:
>>>>=20
>>>> ops =3D (struct pci_ecam_ops *) data;
>>>>=20
>>>> could still work, right? The probe function will directly receive as
>>>> parameter the .data pointer. You shouldn't need the indirection via
>>>> dt_match_node?
>>>=20
>>> As per my understanding probe function will not get .data pointer.Probe=
 data argument is NULL in most of the cases in XEN
>>> Please have a look once dt_pci_init() -> device_init(..) call flow impl=
ementation.
>>=20
>> You are right. Looking at the code, nobody is currently using
>> dt_device_match.data and it is clear why: it is not passed to the
>> device_desc.init function at all. As it is today, it is basically
>> useless.
>>=20
>> And there is only one case where device_init has a non-NULL data
>> parameter and it is in xen/drivers/char/arm-uart.c. All the others are
>> not even using the data parameter of device_init.
>>=20
>> I think we need to change device_init so that dt_device_match.data can
>> be useful. Sorry for the scope-creep but I think we should do the
>> following:
>>=20
>> - do not add of_match_table to struct device
>>=20
>> - add one more parameter to device_desc.init:
>>  int (*init)(struct dt_device_node *dev, struct device_desc *desc, const=
 void *data);
>>=20
>> - change device_init to call desc->init with the right parameters:
>>  desc->init(dev, desc, data);
>>=20
>> This way pci_host_common_probe is just going to get a desc directly as
>> parameter. I think it would make a lot more sense from an interface
>> perspective. It does require a change in all the DT_DEVICE_START.init
>> functions adding a struct device_desc *desc parameter, but it should be
>> a mechanical change.
>>=20
>> Alternatively we could just change device_init to pass
>> device_desc.dt_match.data when the data parameter is NULL but it feels
>> like a hack.
>>=20
>>=20
>> What do you think?
>=20
>=20
> Another idea that doesn't require a device_desc.init change and also
> doesn't require a change to struct device is the following:
>=20
>=20
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-ho=
st-common.c
> index a88f20175e..1aa0ef4c1e 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -205,8 +205,7 @@ int pci_host_common_probe(struct dt_device_node *dev,=
 const void *data)
>     if ( dt_device_for_passthrough(dev) )
>         return 0;
>=20
> -    of_id =3D dt_match_node(dev->dev.of_match_table, dev->dev.of_node);
> -    ops =3D (struct pci_ecam_ops *) of_id->data;
> +    ops =3D (struct pci_ecam_ops *) data;
>=20
>     bridge =3D pci_alloc_host_bridge();
>     if ( !bridge )
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-h=
ost-generic.c
> index 6b3288d6f3..66fb843f49 100644
> --- a/xen/arch/arm/pci/pci-host-generic.c
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -20,15 +20,19 @@
> #include <asm/pci.h>
>=20
> static const struct dt_device_match gen_pci_dt_match[] =3D {
> -    { .compatible =3D "pci-host-ecam-generic",
> -      .data =3D       &pci_generic_ecam_ops },
> -
> +    { .compatible =3D "pci-host-ecam-generic" },
>     { },
> };
>=20
> +static int pci_host_generic_probe(struct dt_device_node *dev,
> +                                         const void *data)
> +{
> +    return pci_host_common_probe(dev, &pci_generic_ecam_ops);
> +}
> +
> DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> .dt_match =3D gen_pci_dt_match,
> -.init =3D pci_host_common_probe,
> +.init =3D pci_host_generic_probe,
> DT_DEVICE_END
>=20
> /*

I also think this is good idea to avoid modification for device_init(.) and=
 struct device{} .
I will use this patch in next version.

Regards,
Rahul



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 16:58:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 16:58:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197107.350051 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtxC-0005b9-QZ; Mon, 27 Sep 2021 16:58:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197107.350051; Mon, 27 Sep 2021 16:58:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtxC-0005b2-Nb; Mon, 27 Sep 2021 16:58:10 +0000
Received: by outflank-mailman (input) for mailman id 197107;
 Mon, 27 Sep 2021 16:58:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUtxA-0005as-W6
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 16:58:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 15fe2906-1fb4-11ec-bc54-12813bfff9fa;
 Mon, 27 Sep 2021 16:58:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 19BC160F3A;
 Mon, 27 Sep 2021 16:58:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15fe2906-1fb4-11ec-bc54-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632761886;
	bh=rqOK66Cvl2zrztemjN1qedLXqxCBOjzSu4R65OLMYQA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hq5yyhZlbXUE/OzPmuXxpnMBDrqtzOFzNW7o0IIIp6PV66OM8SFg3Mr2K2IHMn8aW
	 o1Qsx2qzb0ECSEBfGM0FMQw78Wvfo49iskglyNlTOFIloCzxhEaiR+F+7r9kU4bZ9l
	 Z5+JoPiWUKq4XPKlK3nP9Xu5I/ghLBN44gk6fEAZXMWx06FccqhmUPFC2Dv0Yq6WSt
	 Vo/hcMBw+3c9pKEOClhLX1F3rzL/fhdcir1Pw7zWLqSPHXxsYYmmPMI5njgcUKLEfv
	 iLaSaQk4RdGok4tFnFoZB9PwhymCAPY8jPxJdybkH6Pgl/913JKJNBXBzxV8JPLi8w
	 GYvPtwcstC9wQ==
Date: Mon, 27 Sep 2021 09:58:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Julien Grall <julien.grall@gmail.com>
cc: Wei Chen <Wei.Chen@arm.com>, Julien Grall <julien.grall.oss@gmail.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
In-Reply-To: <CAF3u54BNbPPo4_ezC2ft36NwodKDFSdtv_ekT-FFyvRrU3gQaA@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.2109270956270.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-23-wei.chen@arm.com> <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s> <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s> <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com> <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com> <DB9PR08MB6857316DC540769819C8155E9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CAF3u54BNbPPo4_ezC2ft36NwodKDFSdtv_ekT-FFyvRrU3gQaA@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-741841233-1632761885=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-741841233-1632761885=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 27 Sep 2021, Julien Grall wrote:
> On Mon, 27 Sep 2021, 12:22 Wei Chen, <Wei.Chen@arm.com> wrote:
>       Hi Julien,
> 
>       From: Julien Grall <julien.grall.oss@gmail.com>
>       Sent: 2021年9月27日 15:36
>       To: Wei Chen <Wei.Chen@arm.com>
>       Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-devel <xen-devel@lists.xenproject.org>; Bertrand Marquis
>       <Bertrand.Marquis@arm.com>; Jan Beulich <jbeulich@suse.com>; Roger Pau Monné <roger.pau@citrix.com>; Andrew Cooper
>       <andrew.cooper3@citrix.com>
>       Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
> 
> 
>       On Mon, 27 Sep 2021, 08:53 Wei Chen, <mailto:Wei.Chen@arm.com> wrote:
>       Hi Julien,
> 
>       > -----Original Message-----
>       > From: Xen-devel <mailto:xen-devel-bounces@lists.xenproject.org> On Behalf Of Wei
>       > Chen
>       > Sent: 2021年9月27日 14:46
>       > To: Stefano Stabellini <mailto:sstabellini@kernel.org>
>       > Cc: mailto:xen-devel@lists.xenproject.org; mailto:julien@xen.org; Bertrand Marquis
>       > <mailto:Bertrand.Marquis@arm.com>; mailto:jbeulich@suse.com; mailto:roger.pau@citrix.com;
>       > mailto:andrew.cooper3@citrix.com
>       > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
>       > NR_NODE_MEMBLKS
>       >
>       > Hi Stefano, Julien,
>       >
>       > > -----Original Message-----
>       > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
>       > > Sent: 2021年9月27日 13:00
>       > > To: Wei Chen <mailto:Wei.Chen@arm.com>
>       > > Cc: Stefano Stabellini <mailto:sstabellini@kernel.org>; xen-
>       > > mailto:devel@lists.xenproject.org; mailto:julien@xen.org; Bertrand Marquis
>       > > <mailto:Bertrand.Marquis@arm.com>; mailto:jbeulich@suse.com; mailto:roger.pau@citrix.com;
>       > > mailto:andrew.cooper3@citrix.com
>       > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
>       > > NR_NODE_MEMBLKS
>       > >
>       > > +x86 maintainers
>       > >
>       > > On Mon, 27 Sep 2021, Wei Chen wrote:
>       > > > > -----Original Message-----
>       > > > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
>       > > > > Sent: 2021年9月27日 11:26
>       > > > > To: Wei Chen <mailto:Wei.Chen@arm.com>
>       > > > > Cc: Stefano Stabellini <mailto:sstabellini@kernel.org>; xen-
>       > > > > mailto:devel@lists.xenproject.org; mailto:julien@xen.org; Bertrand Marquis
>       > > > > <mailto:Bertrand.Marquis@arm.com>
>       > > > > Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
>       > > default
>       > > > > NR_NODE_MEMBLKS
>       > > > >
>       > > > > On Sun, 26 Sep 2021, Wei Chen wrote:
>       > > > > > > -----Original Message-----
>       > > > > > > From: Stefano Stabellini <mailto:sstabellini@kernel.org>
>       > > > > > > Sent: 2021年9月24日 9:35
>       > > > > > > To: Wei Chen <mailto:Wei.Chen@arm.com>
>       > > > > > > Cc: mailto:xen-devel@lists.xenproject.org; mailto:sstabellini@kernel.org;
>       > > > > mailto:julien@xen.org;
>       > > > > > > Bertrand Marquis <mailto:Bertrand.Marquis@arm.com>
>       > > > > > > Subject: Re: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override
>       > > > > default
>       > > > > > > NR_NODE_MEMBLKS
>       > > > > > >
>       > > > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
>       > > > > > > > As a memory range described in device tree cannot be split
>       > > across
>       > > > > > > > multiple nodes. So we define NR_NODE_MEMBLKS as NR_MEM_BANKS
>       > in
>       > > > > > > > arch header.
>       > > > > > >
>       > > > > > > This statement is true but what is the goal of this patch? Is it
>       > > to
>       > > > > > > reduce code size and memory consumption?
>       > > > > > >
>       > > > > >
>       > > > > > No, when Julien and I discussed this in last version[1], we hadn't
>       > > > > thought
>       > > > > > so deeply. We just thought a memory range described in DT cannot
>       > be
>       > > > > split
>       > > > > > across multiple nodes. So NR_MEM_BANKS should be equal to
>       > > NR_MEM_BANKS.
>       > > > > >
>       > > > > > https://lists.xenproject.org/archives/html/xen-devel/2021-
>       > > > > 08/msg00974.html
>       > > > > >
>       > > > > > > I am asking because NR_MEM_BANKS is 128 and
>       > > > > > > NR_NODE_MEMBLKS=2*MAX_NUMNODES which is 64 by default so again
>       > > > > > > NR_NODE_MEMBLKS is 128 before this patch.
>       > > > > > >
>       > > > > > > In other words, this patch alone doesn't make any difference; at
>       > > least
>       > > > > > > doesn't make any difference unless CONFIG_NR_NUMA_NODES is
>       > > increased.
>       > > > > > >
>       > > > > > > So, is the goal to reduce memory usage when CONFIG_NR_NUMA_NODES
>       > > is
>       > > > > > > higher than 64?
>       > > > > > >
>       > > > > >
>       > > > > > I also thought about this problem when I was writing this patch.
>       > > > > > CONFIG_NR_NUMA_NODES is increasing, but NR_MEM_BANKS is a fixed
>       > > > > > value, then NR_MEM_BANKS can be smaller than CONFIG_NR_NUMA_NODES
>       > > > > > at one point.
>       > > > > >
>       > > > > > But I agree with Julien's suggestion, NR_MEM_BANKS and
>       > > NR_NODE_MEMBLKS
>       > > > > > must be aware of each other. I had thought to add some ASSERT
>       > check,
>       > > > > > but I don't know how to do it better. So I post this patch for
>       > more
>       > > > > > suggestion.
>       > > > >
>       > > > > OK. In that case I'd say to get rid of the previous definition of
>       > > > > NR_NODE_MEMBLKS as it is probably not necessary, see below.
>       > > > >
>       > > > >
>       > > > >
>       > > > > > >
>       > > > > > > > And keep default NR_NODE_MEMBLKS in common header
>       > > > > > > > for those architectures NUMA is disabled.
>       > > > > > >
>       > > > > > > This last sentence is not accurate: on x86 NUMA is enabled and
>       > > > > > > NR_NODE_MEMBLKS is still defined in xen/include/xen/numa.h
>       > (there
>       > > is
>       > > > > no
>       > > > > > > x86 definition of it)
>       > > > > > >
>       > > > > >
>       > > > > > Yes.
>       > > > > >
>       > > > > > >
>       > > > > > > > Signed-off-by: Wei Chen <mailto:wei.chen@arm.com>
>       > > > > > > > ---
>       > > > > > > >  xen/include/asm-arm/numa.h | 8 +++++++-
>       > > > > > > >  xen/include/xen/numa.h     | 2 ++
>       > > > > > > >  2 files changed, 9 insertions(+), 1 deletion(-)
>       > > > > > > >
>       > > > > > > > diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-
>       > > arm/numa.h
>       > > > > > > > index 8f1c67e3eb..21569e634b 100644
>       > > > > > > > --- a/xen/include/asm-arm/numa.h
>       > > > > > > > +++ b/xen/include/asm-arm/numa.h
>       > > > > > > > @@ -3,9 +3,15 @@
>       > > > > > > >
>       > > > > > > >  #include <xen/mm.h>
>       > > > > > > >
>       > > > > > > > +#include <asm/setup.h>
>       > > > > > > > +
>       > > > > > > >  typedef u8 nodeid_t;
>       > > > > > > >
>       > > > > > > > -#ifndef CONFIG_NUMA
>       > > > > > > > +#ifdef CONFIG_NUMA
>       > > > > > > > +
>       > > > > > > > +#define NR_NODE_MEMBLKS NR_MEM_BANKS
>       > > > > > > > +
>       > > > > > > > +#else
>       > > > > > > >
>       > > > > > > >  /* Fake one node for now. See also node_online_map. */
>       > > > > > > >  #define cpu_to_node(cpu) 0
>       > > > > > > > diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
>       > > > > > > > index 1978e2be1b..1731e1cc6b 100644
>       > > > > > > > --- a/xen/include/xen/numa.h
>       > > > > > > > +++ b/xen/include/xen/numa.h
>       > > > > > > > @@ -12,7 +12,9 @@
>       > > > > > > >  #define MAX_NUMNODES    1
>       > > > > > > >  #endif
>       > > > > > > >
>       > > > > > > > +#ifndef NR_NODE_MEMBLKS
>       > > > > > > >  #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
>       > > > > > > > +#endif
>       > > > >
>       > > > > This one we can remove it completely right?
>       > > >
>       > > > How about define NR_MEM_BANKS to:
>       > > > #ifdef CONFIG_NR_NUMA_NODES
>       > > > #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * 2)
>       > > > #else
>       > > > #define NR_MEM_BANKS 128
>       > > > #endif
>       > > > for both x86 and Arm. For those architectures do not support or enable
>       > > > NUMA, they can still use "NR_MEM_BANKS 128". And replace all
>       > > NR_NODE_MEMBLKS
>       > > > in NUMA code to NR_MEM_BANKS to remove NR_NODE_MEMBLKS completely.
>       > > > In this case, NR_MEM_BANKS can be aware of the changes of
>       > > CONFIG_NR_NUMA_NODES.
>       > >
>       > > x86 doesn't have NR_MEM_BANKS as far as I can tell. I guess you also
>       > > meant to rename NR_NODE_MEMBLKS to NR_MEM_BANKS?
>       > >
>       >
>       > Yes.
>       >
>       > > But NR_MEM_BANKS is not directly related to CONFIG_NR_NUMA_NODES because
>       > > there can be many memory banks for each numa node, certainly more than
>       > > 2. The existing definition on x86:
>       > >
>       > > #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
>       > >
>       > > Doesn't make a lot of sense to me. Was it just an arbitrary limit for
>       > > the lack of a better way to set a maximum?
>       > >
>       >
>       > At that time, this was probably the most cost-effective approach.
>       > Enough and easy. But, if more nodes need to be supported in the
>       > future, it may bring more memory blocks. And this maximum value
>       > might not apply. The maximum may need to support dynamic extension.
>       >
>       > >
>       > > On the other hand, NR_MEM_BANKS and NR_NODE_MEMBLKS seem to be related.
>       > > In fact, what's the difference?
>       > >
>       > > NR_MEM_BANKS is the max number of memory banks (with or without
>       > > numa-node-id).
>       > >
>       > > NR_NODE_MEMBLKS is the max number of memory banks with NUMA support
>       > > (with numa-node-id)?
>       > >
>       > > They are basically the same thing. On ARM I would just do:
>       > >
>       >
>       > Probably not, NR_MEM_BANKS will count those memory ranges without
>       > numa-node-id in boot memory parsing stage (process_memory_node or
>       > EFI parser). But NR_NODE_MEMBLKS will only count those memory ranges
>       > with numa-node-id.
>       >
>       > > #define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))
>       > >
>       > >
> 
>       > Quote Julien's comment from HTML email to here:
>       > " As you wrote above, the second part of the MAX is totally arbitrary.
>       > In fact, it is very likely than if you have more than 64 nodes, you may
>       > need a lot more than 2 regions per node.
>       >
>       > So, for Arm, I would just define NR_NODE_MEMBLKS as an alias to NR_MEM_BANKS
>       > so it can be used by common code.
>       > "
>       >
>       > > But here comes the problem:
>       > > How can we set the NR_MEM_BANKS maximum value, 128 seems an arbitrary too?
>       >
>       > This is based on hardware we currently support (the last time we bumped the value was, IIRC, for Thunder-X). In the case of
>       booting UEFI, we can get a lot of small ranges as we discover the RAM using the UEFI memory map.
>       >
> 
>       Thanks for the background.
> 
>       >
>       > > If #define NR_MEM_BANKS (CONFIG_NR_NUMA_NODES * N)? And what N should be.
>       >
>       > N would have to be the maximum number of ranges you can find in a NUMA node.
>       >
>       > We would also need to make sure this doesn't break existing platforms. So N would have to be quite large or we need a MAX as
>       Stefano suggested.
>       >
>       > But I would prefer to keep the existing 128 and allow to configure it at build time (not necessarily in this series). This
>       avoid to have different way to define the value based NUMA vs non-NUMA.
> 
>       In this case, can we use Stefano's
>       "#define NR_NODE_MEMBLKS MAX(NR_MEM_BANKS, (CONFIG_NR_NUMA_NODES * 2))"
>       in next version. If yes, should we change x86 part? Because NR_MEM_BANKS
>       has not been defined in x86.
> 
> 
> What I meant by configuring dynamically is allowing NR_MEM_BANKS to be set by the user.
> 
> The second part of the MAX makes no sense to me (at least on Arm). So I really prefer if this is not part of the initial version.
> 
> We can refine the value, or introduce the MAX in the future if we have a justification for it.

OK, so for clarity the suggestion is:

- define NR_NODE_MEMBLKS as NR_MEM_BANKS on ARM in this series
- in the future make NR_MEM_BANKS user-configurable via kconfig
- for now leave NR_MEM_BANKS as 128 on ARM

That's fine by me.
--8323329-741841233-1632761885=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 16:59:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 16:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197113.350062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtye-0006CW-75; Mon, 27 Sep 2021 16:59:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197113.350062; Mon, 27 Sep 2021 16:59:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUtye-0006CP-2D; Mon, 27 Sep 2021 16:59:40 +0000
Received: by outflank-mailman (input) for mailman id 197113;
 Mon, 27 Sep 2021 16:59:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Pp17=OR=gmail.com=julien.grall.oss@srs-us1.protection.inumbo.net>)
 id 1mUtyc-0006C9-1Y
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 16:59:38 +0000
Received: from mail-ed1-x52d.google.com (unknown [2a00:1450:4864:20::52d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42feefc6-c1d8-4741-b91d-c24351cb3b58;
 Mon, 27 Sep 2021 16:59:36 +0000 (UTC)
Received: by mail-ed1-x52d.google.com with SMTP id s17so53289943edd.8
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 09:59:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42feefc6-c1d8-4741-b91d-c24351cb3b58
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=+WLDGAQMdDej50CKean/med2vGPwv60xNujIKfagY/Y=;
        b=i5g4rVd+YZCv1hubU9Ey4s2NmIJnA5dyXIRA1oW5o1qVboJLCfhX4SAIIQ0Qc4U6jW
         AbI6jyXBfQA/RXBB1E+G0tZyJtyIhovKY1D1XRhulDc6kaOpuchvT2wu7L5ek1viy0nv
         ScE4Trf/nQXLVLy997ThWqZYXHudsMt1I7e/YVwteg39mw2guJ2QpUXpZYhgD+QjV2/x
         nTEstu1K5nZkKqJSNwqHyhe1PgYPaDDc5HCbT+BD8ZlIHG8jUC26KGAjShqKvAJBbH+6
         bZsuPWzRu8SUuQ2KBIYtZZIvhkuyr01jh6aEdJlqY0TSY+OjDQMx/HNm5Va2POEU+ucI
         yGqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=+WLDGAQMdDej50CKean/med2vGPwv60xNujIKfagY/Y=;
        b=ZZ5OjhtGiARJ70XAuEI1AL7UrM4noacBHTt17cdrfEmgstywSywbuLQS9kC9uRf7LT
         uyuw3cbicOIBTsES617bLQmzlVhxs/gF9f5oeVb1+Q2maZh//AioQxJlSF/Um4B29nFw
         /cUYc+lkLKsfZsepL5CndgNuAtm4idgklyS05g4mXQ74ORNt0xU6aGtslfB5iTllyeAO
         xvU2zu52n+jmS7VUlibNY5nD9t77yPu8AwxZuB2lUYd0BEdMA+uHSaE6f5QXyr85dnjl
         u1xzW12flbffZ1/N8rlUUk/1ln4lDm7f+aUOZb3KxnHPBFoxDBLMBPAyoBECWQQodF1A
         vF3Q==
X-Gm-Message-State: AOAM533rrur2mkpTFMrueph9HwjNgdYwgRk/bj3namA5t3yptxTBi9Kh
	x9k+PClpFZ47VnxJGye/b1snVMSX1WPm7A1IgaI=
X-Google-Smtp-Source: ABdhPJyonwJT+jaa4bgFc4Q16DpMAwV/K2uxET5ibvPmkfRxJFEiraJ/ppNmQUwao+vDTC4e6e1KNAmX/yQnIEniYug=
X-Received: by 2002:a50:cfcb:: with SMTP id i11mr1127843edk.347.1632761975817;
 Mon, 27 Sep 2021 09:59:35 -0700 (PDT)
MIME-Version: 1.0
References: <cover.1632307952.git.rahul.singh@arm.com> <c837d73b1fc8953ab48d36609482cf84832b7827.1632307952.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109221836190.17979@sstabellini-ThinkPad-T480s>
 <F87F4A4E-D6CC-4341-88CF-2DD52A1FD503@arm.com> <alpine.DEB.2.21.2109231206120.17979@sstabellini-ThinkPad-T480s>
 <1E52C080-6913-4009-997D-A9C0988B5A09@arm.com> <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109241422170.17979@sstabellini-ThinkPad-T480s>
From: Julien Grall <julien.grall.oss@gmail.com>
Date: Mon, 27 Sep 2021 18:59:24 +0200
Message-ID: <CAJ=z9a3J7xgEJu8waQpnORO4=gcucK1oHrcjpzBrW1Fs0HY3oQ@mail.gmail.com>
Subject: Re: [PATCH v2 11/17] xen/arm: PCI host bridge discovery within XEN on ARM
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Rahul Singh <Rahul.Singh@arm.com>, xen-devel <xen-devel@lists.xenproject.org>, 
	Bertrand Marquis <Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Type: multipart/alternative; boundary="00000000000092495705ccfd0589"

--00000000000092495705ccfd0589
Content-Type: text/plain; charset="UTF-8"

On Fri, 24 Sep 2021, 23:42 Stefano Stabellini, <sstabellini@kernel.org>
wrote:

> On Fri, 24 Sep 2021, Rahul Singh wrote:
> > Hi Stefano,
> >
> > > On 23 Sep 2021, at 8:12 pm, Stefano Stabellini <sstabellini@kernel.org>
> wrote:
> > >
> > > On Thu, 23 Sep 2021, Rahul Singh wrote:
> > >>>> +            goto err_exit;
> > >>>> +    }
> > >>>
> > >>> This is unnecessary at the moment, right? Can we get rid of
> ops->init ?
> > >>
> > >> No this is required for N1SDP board. Please check below patch.
> > >>
> https://gitlab.com/rahsingh/xen-integration/-/commit/6379ba5764df33d57547087cff4ffc078dc515d5
> > >
> > > OK
> > >
> > >
> > >>>> +int pci_host_common_probe(struct dt_device_node *dev, const void
> *data)
> > >>>> +{
> > >>>> +    struct pci_host_bridge *bridge;
> > >>>> +    struct pci_config_window *cfg;
> > >>>> +    struct pci_ecam_ops *ops;
> > >>>> +    const struct dt_device_match *of_id;
> > >>>> +    int err;
> > >>>> +
> > >>>> +    if ( dt_device_for_passthrough(dev) )
> > >>>> +        return 0;
> > >>>> +
> > >>>> +    of_id = dt_match_node(dev->dev.of_match_table,
> dev->dev.of_node);
> > >>>> +    ops = (struct pci_ecam_ops *) of_id->data;
> > >>>
> > >>> Do we really need dt_match_node and dev->dev.of_match_table to get
> > >>> dt_device_match.data?
> > >>>
> > >>
> > >>> data is passed as a parameter to pci_host_common_probe, isn't it
> enough
> > >>> to do:
> > >>>
> > >>> ops = (struct pci_ecam_ops *) data;
> > >>
> > >> As of now not required but in future we might need it if we implement
> other ecam supported bridge
> > >>
> > >> static const struct dt_device_match gen_pci_dt_match[] = {
>
> > >>    { .compatible = "pci-host-ecam-generic",
>
> > >>      .data =       &pci_generic_ecam_ops },
> > >>
> > >>    { .compatible = "pci-host-cam-generic",
> > >>      .data = &gen_pci_cfg_cam_bus_ops },
>
> > >>
> > >>    { },
>
> > >> };
> > >
> > > Even if we add another ECAM-supported bridge, the following:
> > >
> > > ops = (struct pci_ecam_ops *) data;
> > >
> > > could still work, right? The probe function will directly receive as
> > > parameter the .data pointer. You shouldn't need the indirection via
> > > dt_match_node?
> >
> > As per my understanding probe function will not get .data pointer.Probe
> data argument is NULL in most of the cases in XEN
> > Please have a look once dt_pci_init() -> device_init(..) call flow
> implementation.
>
> You are right. Looking at the code, nobody is currently using
> dt_device_match.data and it is clear why: it is not passed to the
> device_desc.init function at all. As it is today, it is basically
> useless.
>

IIRC it is used by the SMMU driver. But you need to lookup for the desc
manually in each init callback.

If I am not mistaken, this is how Linux is dealing with it as well.
However...


> And there is only one case where device_init has a non-NULL data
> parameter and it is in xen/drivers/char/arm-uart.c. All the others are
> not even using the data parameter of device_init.


> I think we need to change device_init so that dt_device_match.data can
> be useful. Sorry for the scope-creep but I think we should do the
> following:
>
> - do not add of_match_table to struct device
>
> - add one more parameter to device_desc.init:
>   int (*init)(struct dt_device_node *dev, struct device_desc *desc, const
> void *data);
>
> - change device_init to call desc->init with the right parameters:
>   desc->init(dev, desc, data);
>
> This way pci_host_common_probe is just going to get a desc directly as
> parameter. I think it would make a lot more sense from an interface
> perspective. It does require a change in all the DT_DEVICE_START.init
> functions adding a struct device_desc *desc parameter, but it should be
> a mechanical change.
>
> Alternatively we could just change device_init to pass
> device_desc.dt_match.data when the data parameter is NULL but it feels
> like a hack.
>
>
> What do you think?


... I like the idea of passing desc parameter (we could also simply pass
desc.data in an argument named "priv").

Cheers,

>

--00000000000092495705ccfd0589
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Fri, 24 Sep 2021, 23:42 Stefano Stabellini, &lt;<a =
href=3D"mailto:sstabellini@kernel.org">sstabellini@kernel.org</a>&gt; wrote=
:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor=
der-left:1px #ccc solid;padding-left:1ex">On Fri, 24 Sep 2021, Rahul Singh =
wrote:<br>
&gt; Hi Stefano,<br>
&gt; <br>
&gt; &gt; On 23 Sep 2021, at 8:12 pm, Stefano Stabellini &lt;<a href=3D"mai=
lto:sstabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellin=
i@kernel.org</a>&gt; wrote:<br>
&gt; &gt; <br>
&gt; &gt; On Thu, 23 Sep 2021, Rahul Singh wrote:<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto err_e=
xit;<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 }<br>
&gt; &gt;&gt;&gt; <br>
&gt; &gt;&gt;&gt; This is unnecessary at the moment, right? Can we get rid =
of ops-&gt;init ?<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; No this is required for N1SDP board. Please check below patch=
.<br>
&gt; &gt;&gt; <a href=3D"https://gitlab.com/rahsingh/xen-integration/-/comm=
it/6379ba5764df33d57547087cff4ffc078dc515d5" rel=3D"noreferrer noreferrer" =
target=3D"_blank">https://gitlab.com/rahsingh/xen-integration/-/commit/6379=
ba5764df33d57547087cff4ffc078dc515d5</a><br>
&gt; &gt; <br>
&gt; &gt; OK<br>
&gt; &gt; <br>
&gt; &gt; <br>
&gt; &gt;&gt;&gt;&gt; +int pci_host_common_probe(struct dt_device_node *dev=
, const void *data)<br>
&gt; &gt;&gt;&gt;&gt; +{<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 struct pci_host_bridge *bridge;<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 struct pci_config_window *cfg;<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 struct pci_ecam_ops *ops;<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 const struct dt_device_match *of_id;<b=
r>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 int err;<br>
&gt; &gt;&gt;&gt;&gt; +<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 if ( dt_device_for_passthrough(dev) )<=
br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;<br>
&gt; &gt;&gt;&gt;&gt; +<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 of_id =3D dt_match_node(dev-&gt;dev.of=
_match_table, dev-&gt;dev.of_node);<br>
&gt; &gt;&gt;&gt;&gt; +=C2=A0 =C2=A0 ops =3D (struct pci_ecam_ops *) of_id-=
&gt;data;<br>
&gt; &gt;&gt;&gt; <br>
&gt; &gt;&gt;&gt; Do we really need dt_match_node and dev-&gt;dev.of_match_=
table to get<br>
&gt; &gt;&gt;&gt; dt_device_match.data?<br>
&gt; &gt;&gt;&gt; <br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt;&gt; data is passed as a parameter to pci_host_common_probe, i=
sn&#39;t it enough<br>
&gt; &gt;&gt;&gt; to do:<br>
&gt; &gt;&gt;&gt; <br>
&gt; &gt;&gt;&gt; ops =3D (struct pci_ecam_ops *) data;<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; As of now not required but in future we might need it if we i=
mplement other ecam supported bridge<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt; static const struct dt_device_match gen_pci_dt_match[] =3D {=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 <br>
&gt; &gt;&gt;=C2=A0 =C2=A0 { .compatible =3D &quot;pci-host-ecam-generic&qu=
ot;,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <br>
&gt; &gt;&gt;=C2=A0 =C2=A0 =C2=A0 .data =3D=C2=A0 =C2=A0 =C2=A0 =C2=A0&amp;=
pci_generic_ecam_ops },<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt;=C2=A0 =C2=A0 { .compatible =3D &quot;pci-host-cam-generic&quo=
t;,<br>
&gt; &gt;&gt;=C2=A0 =C2=A0 =C2=A0 .data =3D &amp;gen_pci_cfg_cam_bus_ops },=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<br>
&gt; &gt;&gt; <br>
&gt; &gt;&gt;=C2=A0 =C2=A0 { },=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <br>
&gt; &gt;&gt; };<br>
&gt; &gt; <br>
&gt; &gt; Even if we add another ECAM-supported bridge, the following:<br>
&gt; &gt; <br>
&gt; &gt; ops =3D (struct pci_ecam_ops *) data;<br>
&gt; &gt; <br>
&gt; &gt; could still work, right? The probe function will directly receive=
 as<br>
&gt; &gt; parameter the .data pointer. You shouldn&#39;t need the indirecti=
on via<br>
&gt; &gt; dt_match_node?<br>
&gt; <br>
&gt; As per my understanding probe function will not get .data pointer.Prob=
e data argument is NULL in most of the cases in XEN<br>
&gt; Please have a look once dt_pci_init() -&gt; device_init(..) call flow =
implementation.<br>
<br>
You are right. Looking at the code, nobody is currently using<br>
dt_device_match.data and it is clear why: it is not passed to the<br>
device_desc.init function at all. As it is today, it is basically<br>
useless.<br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=
=3D"auto">IIRC it is used by the SMMU driver. But you need to lookup for th=
e desc manually in each init callback.</div><div dir=3D"auto"><br></div><di=
v dir=3D"auto">If I am not mistaken, this is how Linux is dealing with it a=
s well. However...</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div =
class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
And there is only one case where device_init has a non-NULL data<br>
parameter and it is in xen/drivers/char/arm-uart.c. All the others are<br>
not even using the data parameter of device_init.</blockquote></div></div><=
div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quot=
e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I think we need to change device_init so that dt_device_match.data can<br>
be useful. Sorry for the scope-creep but I think we should do the<br>
following:<br>
<br>
- do not add of_match_table to struct device<br>
<br>
- add one more parameter to device_desc.init:<br>
=C2=A0 int (*init)(struct dt_device_node *dev, struct device_desc *desc, co=
nst void *data);<br>
<br>
- change device_init to call desc-&gt;init with the right parameters:<br>
=C2=A0 desc-&gt;init(dev, desc, data);<br>
<br>
This way pci_host_common_probe is just going to get a desc directly as<br>
parameter. I think it would make a lot more sense from an interface<br>
perspective. It does require a change in all the DT_DEVICE_START.init<br>
functions adding a struct device_desc *desc parameter, but it should be<br>
a mechanical change.<br>
<br>
Alternatively we could just change device_init to pass<br>
device_desc.dt_match.data when the data parameter is NULL but it feels<br>
like a hack.<br>
<br>
<br>
What do you think?</blockquote></div></div><div dir=3D"auto"><br></div><div=
 dir=3D"auto">... I like the idea of passing desc parameter (we could also =
simply pass desc.data in an argument named &quot;priv&quot;).</div><div dir=
=3D"auto"><br></div><div dir=3D"auto">Cheers,</div><div dir=3D"auto"><div c=
lass=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 =
0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
</blockquote></div></div></div>

--00000000000092495705ccfd0589--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 17:17:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 17:17:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197125.350073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuFu-0000Ui-Sm; Mon, 27 Sep 2021 17:17:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197125.350073; Mon, 27 Sep 2021 17:17:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuFu-0000Ub-Oh; Mon, 27 Sep 2021 17:17:30 +0000
Received: by outflank-mailman (input) for mailman id 197125;
 Mon, 27 Sep 2021 17:17:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUuFt-0000UV-6J
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 17:17:29 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3a933486-6083-4b55-b22a-5945d7acf907;
 Mon, 27 Sep 2021 17:17:28 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 59D7961501;
 Mon, 27 Sep 2021 17:17:27 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a933486-6083-4b55-b22a-5945d7acf907
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632763047;
	bh=zxs65gmXcvyQSuEH9x9d7sqI8gTKLA2qdHecSUdJWCA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NOIwCldbnMoRYsFrZncx8cUCETfNadDAVyyKLAfzqI/dIz68bJh8VM04RILa0LpHT
	 uTd6moE6fyWDl8AxY0CkxfKVzL87GWIlyOfVNePrrMHx7+eqTchnEUTi03fNe7su2r
	 /hSb/jZeQL+rNGx6QqBRGgib9qRmt8fIlGY5lvwKgxLjho4ADejdoAXXOD7gPicLry
	 3QPxaacYPIxJ6H7eyHicPKy/lSdliOgiizWSJGKm8yetQYRh0xrd1Y4RR/jeML2sb5
	 SOgzyZnMWH/4SMIb7gmTogKp+eYpB3UA5Gy8OUm491iOeS0nsbgmcIBV/BcWrQ7me3
	 HASt71YMCN1Dg==
Date: Mon, 27 Sep 2021 10:17:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Jan Beulich <jbeulich@suse.com>
cc: Julien Grall <julien.grall.oss@gmail.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
In-Reply-To: <42eb1303-1b45-5489-eac3-855f4ab35eb5@suse.com>
Message-ID: <alpine.DEB.2.21.2109271006480.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-37-wei.chen@arm.com> <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s> <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com> <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
 <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com> <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com> <42eb1303-1b45-5489-eac3-855f4ab35eb5@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 Sep 2021, Jan Beulich wrote:
> On 27.09.2021 10:45, Julien Grall wrote:
> > On Mon, 27 Sep 2021, 10:33 Jan Beulich, <jbeulich@suse.com> wrote:
> > 
> >> On 24.09.2021 21:39, Stefano Stabellini wrote:
> >>> On Fri, 24 Sep 2021, Wei Chen wrote:
> >>>> On 2021/9/24 11:31, Stefano Stabellini wrote:
> >>>>> On Thu, 23 Sep 2021, Wei Chen wrote:
> >>>>>> --- a/xen/arch/arm/Kconfig
> >>>>>> +++ b/xen/arch/arm/Kconfig
> >>>>>> @@ -34,6 +34,17 @@ config ACPI
> >>>>>>      Advanced Configuration and Power Interface (ACPI) support for
> >> Xen is
> >>>>>>      an alternative to device tree on ARM64.
> >>>>>>   + config DEVICE_TREE_NUMA
> >>>>>> +  def_bool n
> >>>>>> +  select NUMA
> >>>>>> +
> >>>>>> +config ARM_NUMA
> >>>>>> +  bool "Arm NUMA (Non-Uniform Memory Access) Support (UNSUPPORTED)"
> >> if
> >>>>>> UNSUPPORTED
> >>>>>> +  select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> >>>>>
> >>>>> Should it be: depends on HAS_DEVICE_TREE ?
> >>>>> (And eventually depends on HAS_DEVICE_TREE || ACPI)
> >>>>>
> >>>>
> >>>> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
> >>>> option can be selected by users. And depends on has_device_tree
> >>>> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
> >>>>
> >>>> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
> >>>> does it become a loop dependency?
> >>>>
> >>>>
> >> https://lists.xenproject.org/archives/html/xen-devel/2021-08/msg00888.html
> >>>
> >>> OK, I am fine with that. I was just trying to catch the case where a
> >>> user selects "ARM_NUMA" but actually neither ACPI nor HAS_DEVICE_TREE
> >>> are selected so nothing happens. I was trying to make it clear that
> >>> ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or ACPI
> >>> because otherwise it is not going to work.
> >>>
> >>> That said, I don't think this is important because HAS_DEVICE_TREE
> >>> cannot be unselected. So if we cannot find a way to express the
> >>> dependency, I think it is fine to keep the patch as is.
> >>
> >> So how about doing things the other way around: ARM_NUMA has no prompt
> >> and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt instead
> >> (and, for Arm at least, ACPI_NUMA as well; this might even be worthwhile
> >> to have on x86 down the road).
> >>
> > 
> > As I wrote before, I don't think the user should say "I want to enable NUMA
> > with Device-Tree or ACPI". Instead, they say whether they want to use NUMA
> > and let Xen decide to enable the DT/ACPI support.
> > 
> > In other word, the prompt should stay on ARM_NUMA.
> 
> Okay. In which case I'm confused by Stefano's question.

Let me clarify: I think it is fine to have a single prompt for NUMA in
Kconfig. However, I am just pointing out that it is theoretically
possible with the current code to present an ARM_NUMA prompt to the user
but actually have no NUMA enabled at the end because both DEVICE TREE
and ACPI are disabled. This is only a theoretical problem because DEVICE
TREE support (HAS_DEVICE_TREE) cannot be disabled today. Also I cannot
imagine how a configuration with neither DEVICE TREE nor ACPI can be
correct. So I don't think it is a critical concern.

That said, you can see that, at least theoretically, ARM_NUMA depends on
either HAS_DEVICE_TREE or ACPI, so I suggested to add:

depends on HAS_DEVICE_TREE || ACPI

Wei answered that it might introduce a circular dependency, but I did
try the addition of "depends on HAS_DEVICE_TREE || ACPI" under ARM_NUMA
in Kconfig and everything built fine here.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 17:19:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 17:19:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197130.350084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuHq-000175-8d; Mon, 27 Sep 2021 17:19:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197130.350084; Mon, 27 Sep 2021 17:19:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuHq-00016y-5B; Mon, 27 Sep 2021 17:19:30 +0000
Received: by outflank-mailman (input) for mailman id 197130;
 Mon, 27 Sep 2021 17:19:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bXng=OR=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mUuHo-00016q-SC
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 17:19:28 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c0dcdec4-789e-46cd-b110-1e163b9894d6;
 Mon, 27 Sep 2021 17:19:27 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E27B061288;
 Mon, 27 Sep 2021 17:19:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0dcdec4-789e-46cd-b110-1e163b9894d6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632763167;
	bh=q3xP0gwq4BK0FWyI2NZbUHVL9Rc1Iwyg72CRKD1g5iQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=HOBQ8ACGvtKUSS8KM/hvuCA5ptJ80wYltJCRevgxP83qjCzi0bKm/kN2yXF4Su4Lm
	 sNds/fm2Tkfi7WPv/Qm4nbXMY+0yTBdaNOZWtoT/9Ndw8KKG8QJ+PXybKa6XqPLzko
	 I5a1jGumE1dDh3JdSbyDgL8+ixztu6EGqQjwAhKHEbHURq64uuLg8N4RYpvj8J8Az5
	 UuG0bYtDJECfxrQgMztBPupR4d9N5iNoiDiCFtZ27noHFnQQXWYpW+4ETzR1HLXfZY
	 6kbdV/Zfo41BIkQEONSIZ1uOoj4oieHpivrTa92dM7HcMNKLklMorUwMd1wcWDDdch
	 9f6duMRh8Rw6A==
Date: Mon, 27 Sep 2021 10:19:25 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    "jbeulich@suse.com" <jbeulich@suse.com>, 
    "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <DB9PR08MB685772C5CDE9DF885A063F479EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109271018220.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com> <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s> <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685772C5CDE9DF885A063F479EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-947565128-1632763165=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-947565128-1632763165=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 27 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月27日 13:05
> > To: Stefano Stabellini <sstabellini@kernel.org>
> > Cc: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> > julien@xen.org; Bertrand Marquis <Bertrand.Marquis@arm.com>;
> > jbeulich@suse.com; andrew.cooper3@citrix.com; roger.pau@citrix.com;
> > wl@xen.org
> > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node
> > memory range
> > 
> > On Sun, 26 Sep 2021, Stefano Stabellini wrote:
> > > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > > -----Original Message-----
> > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > Sent: 2021年9月25日 3:53
> > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > > <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > andrew.cooper3@citrix.com;
> > > > > roger.pau@citrix.com; wl@xen.org
> > > > > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous
> > node
> > > > > memory range
> > > > >
> > > > > On Fri, 24 Sep 2021, Wei Chen wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > > > Sent: 2021年9月24日 8:26
> > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > > > julien@xen.org;
> > > > > > > Bertrand Marquis <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > > > > > > andrew.cooper3@citrix.com; roger.pau@citrix.com; wl@xen.org
> > > > > > > Subject: Re: [PATCH 08/37] xen/x86: add detection of
> > discontinous node
> > > > > > > memory range
> > > > > > >
> > > > > > > CC'ing x86 maintainers
> > > > > > >
> > > > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > > > One NUMA node may contain several memory blocks. In current
> > Xen
> > > > > > > > code, Xen will maintain a node memory range for each node to
> > cover
> > > > > > > > all its memory blocks. But here comes the problem, in the gap
> > of
> > > > > > > > one node's two memory blocks, if there are some memory blocks
> > don't
> > > > > > > > belong to this node (remote memory blocks). This node's memory
> > range
> > > > > > > > will be expanded to cover these remote memory blocks.
> > > > > > > >
> > > > > > > > One node's memory range contains othe nodes' memory, this is
> > > > > obviously
> > > > > > > > not very reasonable. This means current NUMA code only can
> > support
> > > > > > > > node has continous memory blocks. However, on a physical
> > machine,
> > > > > the
> > > > > > > > addresses of multiple nodes can be interleaved.
> > > > > > > >
> > > > > > > > So in this patch, we add code to detect discontinous memory
> > blocks
> > > > > > > > for one node. NUMA initializtion will be failed and error
> > messages
> > > > > > > > will be printed when Xen detect such hardware configuration.
> > > > > > >
> > > > > > > At least on ARM, it is not just memory that can be interleaved,
> > but
> > > > > also
> > > > > > > MMIO regions. For instance:
> > > > > > >
> > > > > > > node0 bank0 0-0x1000000
> > > > > > > MMIO 0x1000000-0x1002000
> > > > > > > Hole 0x1002000-0x2000000
> > > > > > > node0 bank1 0x2000000-0x3000000
> > > > > > >
> > > > > > > So I am not familiar with the SRAT format, but I think on ARM
> > the
> > > > > check
> > > > > > > would look different: we would just look for multiple memory
> > ranges
> > > > > > > under a device_type = "memory" node of a NUMA node in device
> > tree.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Should I need to include/refine above message to commit log?
> > > > >
> > > > > Let me ask you a question first.
> > > > >
> > > > > With the NUMA implementation of this patch series, can we deal with
> > > > > cases where each node has multiple memory banks, not interleaved?
> > > >
> > > > Yes.
> > > >
> > > > > An an example:
> > > > >
> > > > > node0: 0x0        - 0x10000000
> > > > > MMIO : 0x10000000 - 0x20000000
> > > > > node0: 0x20000000 - 0x30000000
> > > > > MMIO : 0x30000000 - 0x50000000
> > > > > node1: 0x50000000 - 0x60000000
> > > > > MMIO : 0x60000000 - 0x80000000
> > > > > node2: 0x80000000 - 0x90000000
> > > > >
> > > > >
> > > > > I assume we can deal with this case simply by setting node0 memory
> > to
> > > > > 0x0-0x30000000 even if there is actually something else, a device,
> > that
> > > > > doesn't belong to node0 in between the two node0 banks?
> > > >
> > > > While this configuration is rare in SoC design, but it is not
> > impossible.
> > >
> > > Definitely, I have seen it before.
> > >
> > >
> > > > > Is it only other nodes' memory interleaved that cause issues? In
> > other
> > > > > words, only the following is a problematic scenario?
> > > > >
> > > > > node0: 0x0        - 0x10000000
> > > > > MMIO : 0x10000000 - 0x20000000
> > > > > node1: 0x20000000 - 0x30000000
> > > > > MMIO : 0x30000000 - 0x50000000
> > > > > node0: 0x50000000 - 0x60000000
> > > > >
> > > > > Because node1 is in between the two ranges of node0?
> > > > >
> > > >
> > > > But only device_type="memory" can be added to allocation.
> > > > For mmio there are two cases:
> > > > 1. mmio doesn't have NUMA id property.
> > > > 2. mmio has NUMA id property, just like some PCIe controllers.
> > > >    But we don’t need to handle these kinds of MMIO devices
> > > >    in memory block parsing. Because we don't need to allocate
> > > >    memory from these mmio ranges. And for accessing, we need
> > > >    a NUMA-aware PCIe controller driver or a generic NUMA-aware
> > > >    MMIO accessing APIs.
> > >
> > > Yes, I am not too worried about devices with a NUMA id property because
> > > they are less common and this series doesn't handle them at all, right?
> > > I imagine they would be treated like any other device without NUMA
> > > awareness.
> > >
> > > I am thinking about the case where the memory of each NUMA node is made
> > > of multiple banks. I understand that this patch adds an explicit check
> > > for cases where these banks are interleaving, however there are many
> > > other cases where NUMA memory nodes are *not* interleaving but they are
> > > still made of multiple discontinuous banks, like in the two example
> > > above.
> > >
> > > My question is whether this patch series in its current form can handle
> > > the two cases above correctly. If so, I am wondering how it works given
> > > that we only have a single "start" and "size" parameter per node.
> > >
> > > On the other hand if this series cannot handle the two cases above, my
> > > question is whether it would fail explicitly or not. The new
> > > check is_node_memory_continuous doesn't seem to be able to catch them.
> > 
> > 
> > Looking at numa_update_node_memblks, it is clear that the code is meant
> > to increase the range of each numa node to cover even MMIO regions in
> > between memory banks. Also see the comment at the top of the file:
> > 
> >  * Assumes all memory regions belonging to a single proximity domain
> >  * are in one chunk. Holes between them will be included in the node.
> > 
> > So if there are multiple banks for each node, start and end are
> > stretched to cover the holes between them, and it works as long as
> > memory banks of different NUMA nodes don't interleave.
> > 
> > I would appreciate if you could add an in-code comment to explain this
> > on top of numa_update_node_memblk.
> 
> Yes, I will do it.
 
Thank you


> > Have you had a chance to test this? If not it would be fantastic if you
> > could give it a quick test to make sure it works as intended: for
> > instance by creating multiple memory banks for each NUMA node by
> > splitting an real bank into two smaller banks with a hole in between in
> > device tree, just for the sake of testing.
> 
> Yes, I have created some fake NUMA nodes in FVP device tree to test it.
> The intertwine of nodes' address can be detected.
> 
> (XEN) SRAT: Node 0 0000000080000000-00000000ff000000
> (XEN) SRAT: Node 1 0000000880000000-00000008c0000000
> (XEN) NODE 0: (0000000080000000-00000008d0000000) intertwine with NODE 1 (0000000880000000-00000008c0000000)

Great thanks! And what if there are multiple non-contiguous memory banks
per node, but *not* intertwined. Does that all work correctly as
expected?
--8323329-947565128-1632763165=:5022--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 17:25:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 17:25:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197137.350095 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuNn-0002Vh-Sy; Mon, 27 Sep 2021 17:25:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197137.350095; Mon, 27 Sep 2021 17:25:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUuNn-0002Va-Pf; Mon, 27 Sep 2021 17:25:39 +0000
Received: by outflank-mailman (input) for mailman id 197137;
 Mon, 27 Sep 2021 17:25:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DWny=OR=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mUuNm-0002VU-N5
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 17:25:38 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7979ed08-db6a-4aa8-bff5-cd77488ebf11;
 Mon, 27 Sep 2021 17:25:37 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id z24so81572509lfu.13
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 10:25:37 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id u14sm1659605lfi.231.2021.09.27.10.25.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 Sep 2021 10:25:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7979ed08-db6a-4aa8-bff5-cd77488ebf11
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=BxtWH+zfZOEqiyTTmaQRt0CfvXvlFSx5l0UI63Q5xaM=;
        b=ddOTpbia4/+S1xHCQ3gyH9+KRouDsqA+F4EWRzeE+0xHng/Z7M3p0DIVaeCGoWunxb
         ZV97ayrb8hTMcQ/7vjGvhAdcNszIWxnd0yw72+ZUFAi7KXFc8D8e4Sj591SU2KAfJt+O
         FDgsUJN4ZqM0srqNTnJTr+plKMnQPqRnv6Ni5SYHIDTvyjxLSydJkSKl0LDIoxdp7H/Z
         yCjswkvBVc5Eyi1ZNKJYwsiegGIu9+dbe48BPEqYimW/v32OyAEl0wQRCZcgcEdZh/jW
         /qSLWsw/uKgxOG942DDngeT0Vdb9NYA0CRDzDWjOD4uCGVwB+mS0rKRzPJJ5eAwKwaQL
         E32w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=BxtWH+zfZOEqiyTTmaQRt0CfvXvlFSx5l0UI63Q5xaM=;
        b=KgREWVkGk697Y+qqDHrgPXuJjGWo/p7MiFIe1RhxQdfshtTkD5TtBX0aMtOvc7aUhd
         YnjpeVX/PrQy0BpL/lzZx/v7VsRbmDo6l/YsbJHj1q5pmrOR7VumsHyq3syFQEQbgFLN
         GaojzSZLoyMwO/GB48u1/gOwm0XUVy2hN3XC/TTWCTMg/Qky4vz/EcT/DUY2ROFXRZ5R
         DfyhYuBO4Ve3NuJs1f+0DfJaVVZ90s9FjJuWhe6aPahgsGxH7O56Po2pc4leqACSM0Dn
         8eX7hrGcqsmISlccB5s5KZmSDpjrE3fFa0ZYb0+akzqZclsvrIp0dAt0w4c36UgG3Atw
         3PZg==
X-Gm-Message-State: AOAM531NimxutkIoteQwzSftwTanhin/HVarLViolBw82Ex1HVWwecvM
	oEgYAfm3Au7PzBk26lU4L+4=
X-Google-Smtp-Source: ABdhPJwD85iEm4TMHrL2i4N0lju0rL0CtCBKrFAiVS1yG48GbVsbOta1nOP4NGbyGUSULJtbeb6CIw==
X-Received: by 2002:a2e:7008:: with SMTP id l8mr1053858ljc.340.1632763536739;
        Mon, 27 Sep 2021 10:25:36 -0700 (PDT)
Subject: Re: Xen Rust VirtIO demos work breakdown for Project Stratos
To: =?UTF-8?Q?Alex_Benn=c3=a9e?= <alex.bennee@linaro.org>
Cc: Stratos Mailing List <stratos-dev@op-lists.linaro.org>,
 Mike Holmes <mike.holmes@linaro.org>,
 Mathieu Poirier <mathieu.poirier@linaro.org>,
 Viresh Kumar <viresh.kumar@linaro.org>,
 Peter Griffin <peter.griffin@linaro.org>, xen-devel@lists.xenproject.org,
 wl@xen.org, Artem Mygaiev <Artem_Mygaiev@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Doug Goldstein <cardoe@cardoe.com>,
 Rust-VMM Mailing List <rust-vmm@lists.opendev.org>,
 Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>,
 David Woodhouse <dwmw2@infradead.org>
References: <87pmsylywy.fsf@linaro.org>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <47bce3dd-d271-1688-d445-43eee667ade3@gmail.com>
Date: Mon, 27 Sep 2021 20:25:35 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <87pmsylywy.fsf@linaro.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US


On 24.09.21 19:02, Alex Bennée wrote:

Hi Alex

[snip]

>
> [STR-56] <https://linaro.atlassian.net/browse/STR-56>
>
> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
> ───────────────────────────────────────────────────────────────
>
>    Currently the foreign memory mapping support only works for dom0 due
>    to reference counting issues. If we are to support backends running in
>    their own domains this will need to get fixed.
>
>    Estimate: 8w
>
>
> [STR-57] <https://linaro.atlassian.net/browse/STR-57>

If I got this paragraph correctly, this is already fixed on Arm [1]


[1] 
https://lore.kernel.org/xen-devel/1611884932-1851-17-git-send-email-olekstysh@gmail.com/


[snip]


-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 18:54:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 18:54:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197144.350105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUvm1-0003qG-D6; Mon, 27 Sep 2021 18:54:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197144.350105; Mon, 27 Sep 2021 18:54:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUvm1-0003q9-9m; Mon, 27 Sep 2021 18:54:45 +0000
Received: by outflank-mailman (input) for mailman id 197144;
 Mon, 27 Sep 2021 18:54:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUvlz-0003pz-Vs; Mon, 27 Sep 2021 18:54:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUvlz-0006Mm-O5; Mon, 27 Sep 2021 18:54:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUvlz-0004H1-FI; Mon, 27 Sep 2021 18:54:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUvlz-00078y-Eq; Mon, 27 Sep 2021 18:54:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aLXXraIHIqOFJRgu+KlxGLUc6Ek8j7OeQ1bbKB3RkSw=; b=QO0s9aO41BTb0+uv49DH1I1h3R
	oPuewb6/i5Qdegx2ihnuvjqSQL6ZsA1UhiY4+cGW5rl4UDc2Q6/pi3eI8ohgm9JWxyx1CFX0mMx6g
	xLG75Rs7lfwVmS+WNyWXl0n/3g2eSbvEpDB67OarMidk3Y5CTVei1NbPQ+LwQIUWUm9g=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165223-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165223: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2e46d73b4c7562f7b104e9e10fe302316af13959
X-Osstest-Versions-That:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 18:54:43 +0000

flight 165223 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165223/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2e46d73b4c7562f7b104e9e10fe302316af13959
baseline version:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155

Last test of basis   165181  2021-09-24 14:03:24 Z    3 days
Testing same since   165223  2021-09-27 15:01:42 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Ian Jackson <iwj@xenproject.org>
  Juergen Gross <jgross@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   dd6c062a7a..2e46d73b4c  2e46d73b4c7562f7b104e9e10fe302316af13959 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 20:26:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 20:26:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197159.350120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxCN-00059J-8n; Mon, 27 Sep 2021 20:26:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197159.350120; Mon, 27 Sep 2021 20:26:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxCN-00059C-5Y; Mon, 27 Sep 2021 20:26:03 +0000
Received: by outflank-mailman (input) for mailman id 197159;
 Mon, 27 Sep 2021 20:26:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ytfr=OR=xen.org=tim@srs-us1.protection.inumbo.net>)
 id 1mUxCM-000596-1O
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 20:26:02 +0000
Received: from deinos.phlegethon.org (unknown [2001:41d0:8:b1d7::1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a22acf48-6271-4cbd-9d38-62eb13b31ca8;
 Mon, 27 Sep 2021 20:26:00 +0000 (UTC)
Received: from tjd by deinos.phlegethon.org with local (Exim 4.94.2 (FreeBSD))
 (envelope-from <tim@xen.org>)
 id 1mUxCI-000Olf-Vj; Mon, 27 Sep 2021 20:25:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a22acf48-6271-4cbd-9d38-62eb13b31ca8
Date: Mon, 27 Sep 2021 21:25:58 +0100
From: Tim Deegan <tim@xen.org>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: sh_unshadow_for_p2m_change() vs p2m_set_entry()
Message-ID: <YVIo1sR283L/MUeN@deinos.phlegethon.org>
References: <ae8b6cf1-d1ae-8a55-7b88-7ef24db0c554@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <ae8b6cf1-d1ae-8a55-7b88-7ef24db0c554@suse.com>
X-SA-Known-Good: Yes
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: tim@xen.org
X-SA-Exim-Scanned: No (on deinos.phlegethon.org); SAEximRunCond expanded to false

Hi,

At 13:31 +0200 on 24 Sep (1632490304), Jan Beulich wrote:
> I'm afraid you're still my best guess to hopefully get an insight
> on issues like this one.

I'm now very rusty on all this but I'll do my best!  I suspect I'll
just be following you through the code.

> While doing IOMMU superpage work I was, just in the background,
> considering in how far the superpage re-coalescing to be used there
> couldn't be re-used for P2M / EPT / NPT. Which got me to think about
> shadow mode's using of p2m-pt.c: That's purely software use of the
> tables in that case, isn't it? In which case hardware support for
> superpages shouldn't matter at all.

ISTR at the time we used the same table for p2m and NPT.
If that's gone away, then yes, we could have superpages
in the p2m without caring about hardware support.

> The only place where I could spot that P2M superpages would actually
> make a difference to shadow code was sh_unshadow_for_p2m_change().
> That one appears to have been dealing with 2M pages (more below)
> already at the time of 0ca1669871f8a ("P2M: check whether hap mode
> is enabled before using 2mb pages"), so I wonder what "potential
> errors when hap is disabled" this commit's description might be
> talking about.

Sorry, I have no idea what the "potential errors" were here. :(

> As to sh_unshadow_for_p2m_change()'s readiness for at least 2Mb
> pages: The 4k page handling there differs from the 2M one primarily
> in the p2m type checks: "p2m_is_valid(...) || p2m_is_grant(...)"
> vs "p2m_is_valid(...)" plus later "!p2m_is_ram(...)", the first
> three acting on the type taken from the old PTE, while the latter
> acts on the type in the new (split) PTEs.

So I think the type tests on the old entry are correct - as you say,
p2m_is_grant() only applies to 4k entries and otherwise they're the
same.

> Shouldn't the exact same
> checks be used in both cases (less the p2m_is_grant() check which
> can't be true for superpages)? IOW isn't !p2m_is_ram() at least
> superfluous (and perhaps further redundant with the subsequent
> !mfn_eq(l1e_get_mfn(npte[i]), omfn))? Instead I'm missing an
> entry-is-present check, without which l1e_get_mfn(npte[i]) looks
> risky at best. Or is p2m_is_ram() considered a sufficient
> replacement, making assumptions on the behavior of a lot of other
> code?

AFAICT, p2m_is_ram(p2m_flags_to_type(l1e_get_flags(npte[i]))) implies
that npte[i] has a valid MFN in it, but I agree that it would be better
to check _PAGE_PRESENT.

And I think in general it would make sense to factor out the old->new
checks and make them the same between the L1 and L2.  I don't see
anything obviously wrong with the current code but the refactored code
would be more obviously right.

> The 2M logic also first checks _PAGE_PRESENT (and _PAGE_PSE), while
> the 4k logic appears to infer that the old page was present from
> p2m_is_{valid,grant}().

I think the p2m_type check is the right one (rather than
_PAGE_PRESENT), since that's the one that the p2m lookups will obey
when causing things to get shadowed.  But the extra _PAGE_PSE check
should stay.

> And isn't this 2M page handling code (because of the commit pointed
> at above) dead right now anyway? If not, where would P2M superpages
> come from?

Yep, that sounds right - p2m_set_entry still only sets 4k entries on
shadow domains, so I think this code is dead right now.  If you had
asked me I would not have remembered that was the case.

Cheers,

Tim.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 20:41:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 20:41:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197167.350131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxRH-0007kN-CL; Mon, 27 Sep 2021 20:41:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197167.350131; Mon, 27 Sep 2021 20:41:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxRH-0007kG-9A; Mon, 27 Sep 2021 20:41:27 +0000
Received: by outflank-mailman (input) for mailman id 197167;
 Mon, 27 Sep 2021 20:41:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DWny=OR=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mUxRF-0007kA-SZ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 20:41:25 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5b5f3a60-c7b4-4c32-acdc-1f5c0540db32;
 Mon, 27 Sep 2021 20:41:24 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id z24so83861013lfu.13
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 13:41:24 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id d7sm1704777lfn.27.2021.09.27.13.41.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 Sep 2021 13:41:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b5f3a60-c7b4-4c32-acdc-1f5c0540db32
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=tgzBmsSO6dG09PN9iOtRQkEfhICE0+hS9rhKv7aTfrU=;
        b=IgmdJfYw2FOSQp+AVZ1yt3bCmhbVbFG9GYowhZiUuGJ8lUcemd9pHb/4mNs0kxZeQE
         XeUNT7osS+fzO8ec++WdtJWHR8fuP8d9XJRXfQV/6waIZ3ZC3qZD4tQNPquphlLJjIJ1
         pU5L+j83JdvgtlgwmZxyrXUR4HiNR2KmaLA/8qe2o2MGjTXKcoz/q0N8ARST5gvnhRiv
         QkJfwK1ZzXVTen2mydOYgxVCTzzn04FxkULJuNLhtt6Jjk16Rm+8ss+XjmZBDSnaJ2kg
         cblHWLkRDvsMRGy31zOlTdapKvw2xPkTcr35R69ZZvfFUUavgidHpptKQUxu4Yhe2NdW
         8Ljg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=tgzBmsSO6dG09PN9iOtRQkEfhICE0+hS9rhKv7aTfrU=;
        b=iWA2IJwJkofVgkKeV+OiUyBbHQvK3ydrSKVmZPcrLNEheH7lQt/p6DiDMpvceRGUfH
         F6iczCm76TDeFlNCqpNd6IVE6xI5KPhnFaPkn8Bp9iIu+00CG5s5M5Mnd/cdpFT81KVY
         kb7g0TNwLYimUI5Vqxrlv7hAjCO4mYMNxxRKsxfvblCJyEDKzEA8z1mXWQyj1rL/eUzn
         EHnygvHWpcifh2EHcmeW1FtPxf51Wrcb1S9It2dse4PVzljkYDSrwBZcLVCOnWREVvgh
         Sr+v8ubXIhGQ0EFvypdb4te0BaB86TYCuKSovQDtk/3Va7lgcxOzojvBQm+w6P2FO/5A
         VJgA==
X-Gm-Message-State: AOAM532Q2JecJQ/1j61xaMRJXr6QsZYQ2xi0axvCi+houirPT1Jdoes9
	+Jryusb81iDi9TOnWnyeOKA=
X-Google-Smtp-Source: ABdhPJypb143HeGyegdyodh/txXPmGMs8AzmZnKEQd3JmMOCehnZRzd4kPi1v49DcKwvpvYXySueSg==
X-Received: by 2002:a19:f517:: with SMTP id j23mr1658922lfb.371.1632775283887;
        Mon, 27 Sep 2021 13:41:23 -0700 (PDT)
Subject: Re: [PATCH 0/2] grant table and add-to-physmap adjustments on top of
 XSAs 379 and 384
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <be1dc77b-829e-e8cc-eb04-2a826d6bc03b@gmail.com>
Date: Mon, 27 Sep 2021 23:41:22 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 13.09.21 09:39, Jan Beulich wrote:

Hi Jan

> I'm prepared for the "how" aspect of the 1st patch here to end up
> controversial. Since the observed quirk will imo want dealing with,
> I'd appreciate any objection to the proposed change to be accompanied
> by an alternative suggestion. An intention of mine was to not further
> increase the number of arch hooks needed. I further realize that this
> change conflicts with Oleksandr's "xen/gnttab: Store frame GFN in
> struct page_info on Arm", at the very least contextually.
FYI, I have a rebased version of my patch on top of your patch #1 
locally. I preliminary checked that combination on my setup (Arm64) and 
didn't see any issues.


>
> 1: gnttab: remove guest_physmap_remove_page() call from gnttab_map_frame()
> 2: memory: XENMEM_add_to_physmap (almost) wrapping checks
>
> Jan
>
>
-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 21:00:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 21:00:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197182.350141 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxjh-0001yf-6B; Mon, 27 Sep 2021 21:00:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197182.350141; Mon, 27 Sep 2021 21:00:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxjh-0001yY-2h; Mon, 27 Sep 2021 21:00:29 +0000
Received: by outflank-mailman (input) for mailman id 197182;
 Mon, 27 Sep 2021 21:00:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ukqu=OR=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mUxjf-0001yS-DO
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 21:00:27 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 57575a6e-72fb-4dfc-8457-a9109f6a3762;
 Mon, 27 Sep 2021 21:00:25 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxik-0003sZ-Tr; Mon, 27 Sep 2021 22:59:30 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxia-0001cX-8M; Mon, 27 Sep 2021 22:59:20 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxia-0001SG-3t; Mon, 27 Sep 2021 22:59:20 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57575a6e-72fb-4dfc-8457-a9109f6a3762
Date: Mon, 27 Sep 2021 22:59:17 +0200
From: Uwe =?utf-8?Q?Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
To: Uwe =?utf-8?Q?Kleine-K=C3=B6nig?= <uwe@kleine-koenig.org>
Cc: Bjorn Helgaas <helgaas@kernel.org>,
	Giovanni Cabiddu <giovanni.cabiddu@intel.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Sathya Prakash <sathya.prakash@broadcom.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Alexander Duyck <alexanderduyck@fb.com>,
	Russell Currey <ruscur@russell.cc>, x86@kernel.org,
	qat-linux@intel.com, oss-drivers@corigine.com,
	Oliver O'Halloran <oohall@gmail.com>,
	"H. Peter Anvin" <hpa@zytor.com>, Jiri Olsa <jolsa@redhat.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Marco Chiappero <marco.chiappero@intel.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	linux-scsi@vger.kernel.org, Michael Ellerman <mpe@ellerman.id.au>,
	=?utf-8?B?UmFmYcWCIE1pxYJlY2tp?= <zajec5@gmail.com>,
	Jesse Brandeburg <jesse.brandeburg@intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>, linux-pci@vger.kernel.org,
	linux-wireless@vger.kernel.org, Jakub Kicinski <kuba@kernel.org>,
	Yisen Zhuang <yisen.zhuang@huawei.com>,
	Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
	Fiona Trahe <fiona.trahe@intel.com>,
	Andrew Donnellan <ajd@linux.ibm.com>, Arnd Bergmann <arnd@arndb.de>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ido Schimmel <idosch@nvidia.com>,
	Simon Horman <simon.horman@corigine.com>,
	linuxppc-dev@lists.ozlabs.org,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Jack Xu <jack.xu@intel.com>, Borislav Petkov <bp@alien8.de>,
	Michael Buesch <m@bues.ch>, Jiri Pirko <jiri@nvidia.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Juergen Gross <jgross@suse.com>,
	Salil Mehta <salil.mehta@huawei.com>,
	Sreekanth Reddy <sreekanth.reddy@broadcom.com>,
	xen-devel@lists.xenproject.org, Vadym Kochan <vkochan@marvell.com>,
	MPT-FusionLinux.pdl@broadcom.com,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	linux-usb@vger.kernel.org,
	Wojciech Ziemba <wojciech.ziemba@intel.com>,
	linux-kernel@vger.kernel.org,
	Mathias Nyman <mathias.nyman@intel.com>,
	Zhou Wang <wangzhou1@hisilicon.com>, linux-crypto@vger.kernel.org,
	kernel@pengutronix.de, netdev@vger.kernel.org,
	Frederic Barrat <fbarrat@linux.ibm.com>,
	Paul Mackerras <paulus@samba.org>,
	Tomaszx Kowalik <tomaszx.kowalik@intel.com>,
	Taras Chornyi <tchornyi@marvell.com>,
	"David S. Miller" <davem@davemloft.net>,
	linux-perf-users@vger.kernel.org, Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH v4 0/8] PCI: Drop duplicated tracking of a pci_dev's
 bound driver
Message-ID: <20210927205917.e763q5mojkwk6per@pengutronix.de>
References: <20210927204326.612555-1-uwe@kleine-koenig.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="upxcpc44c7obtcwt"
Content-Disposition: inline
In-Reply-To: <20210927204326.612555-1-uwe@kleine-koenig.org>
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org


--upxcpc44c7obtcwt
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hello,

On Mon, Sep 27, 2021 at 10:43:18PM +0200, Uwe Kleine-K=F6nig wrote:
> From: Uwe Kleine-K=F6nig <u.kleine-koenig@pengutronix.de>

I sent the series from the wrong email address :-\ I should have used
the above address as sender. Also I failed to add Christoph Hellwig to
Cc: (fixed for this mail). I guess I'll have to send a v5, but I will
wait a bit until the build bots are done with this series.

Best regards
Uwe

--=20
Pengutronix e.K.                           | Uwe Kleine-K=F6nig            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

--upxcpc44c7obtcwt
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFSMKIACgkQwfwUeK3K
7AnalQgAlpBrfIgHu7fEFcJYkSR/33uv+V4CLZsCsu9MNXsSeds9vT38r8/y0bJl
rOEKhsH1blIcq3bsV8/AulLrFkmjYRkkih/gA/y9CeoqpbV0/NzhrS4Xo9kMos8z
n+0f+PzRO1qg1RVWyPL7K4pkXR5cMkqWGoie07ihkt3Y9mVY8ItYl9ny3oDxCRcU
r8KFjr7Jw0Vo8eI3Kr9lu62KyFZFByf1DDBurR5crF8ZcWM7e9kOezvJrxOQxGPP
Z82uFafVCkhtIkKaks/6/y9pMmJF9hzDd91ubgKfbkPIMvBjpL7n07Y/Sk4S34vM
5wdHQGNinogsdgdebn8YdZCULLmXSQ==
=/nd3
-----END PGP SIGNATURE-----

--upxcpc44c7obtcwt--


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 21:01:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 21:01:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197188.350153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxl2-0002Yz-Hw; Mon, 27 Sep 2021 21:01:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197188.350153; Mon, 27 Sep 2021 21:01:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUxl2-0002Ys-EK; Mon, 27 Sep 2021 21:01:52 +0000
Received: by outflank-mailman (input) for mailman id 197188;
 Mon, 27 Sep 2021 21:01:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g88l=OR=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mUxl1-0002Yi-1q
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 21:01:51 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3b7949de-3884-48d0-b34e-f56c2d8194c5;
 Mon, 27 Sep 2021 21:01:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b7949de-3884-48d0-b34e-f56c2d8194c5
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632776509;
  h=to:from:cc:subject:message-id:date:
   content-transfer-encoding:mime-version;
  bh=T9zPLtDbhZa8cqsLzs9J9E7mUk9bKUS79E9/0KA/6kg=;
  b=AVVKh5Wz1BrPqri1fS3xxGjWNTVlrXgTBOAgHcyUPZVa7cLrUwtwrlq3
   dJ1Ca+f3acNXUe0aG6TE5JmQ8Ua8FUrJARcwtmPDWEctY3YC9H22K29wJ
   6G9OVHKs5QTzBlwSFnjL1pij6Cz7jicHy6/da1Mzbqz0ouYDNrgDCQyRu
   U=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: POfzMSqZQxv3ibjBPQEmvpjMgroqr3ov2YT0WxncigJ1IL6SxRjyr86AFUQtDITMdgXho5P8Pm
 2/r0ZawD+aTJCcVC0gvI3h69bAmd135OsKsYc5m2KrFgA4bh2/kdosDHBE1rhLV4nB9UAUE/6Y
 Lv8h1DDCMQTMLLTVM9USHe+s4yxFSTjBKT9UsB9s1qvrsHV4R7oqPu5LU54UyK09cDA1ckXSyO
 OM81zBjCNzNUVvLORPGOgKGh4w38hYDZYsw5mdXU+LDQZdzFgw75MEaeXlKnQ642HlQiJrPfyf
 pf2SX+HvVTaDVSMfpsrTEvjv
X-SBRS: 5.1
X-MesageID: 53352501
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:uCNQGKDcn+x2oxVW/8Pkw5YqxClBgxIJ4kV8jS/XYbTApDp2gTIEz
 GtJUT2CPqyJZ2CgfNklPYmw8k1T78PUyYUyQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0007wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/jRqykslWm
 c525I2dFCo5AZyVht0fekwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFh2xt3J0WRJ4yY
 eIHbyAzXS/kTCdUN05KNawMjMKqjWLWJmgwRFW9+vNsvjm7IBZK+LrnPcfRe9eKbd5IhUver
 WXDl0zpDxdfONGBxD6t9nO3mvSJjS79QJgVFrCz6rhtmlL77nMXIA0bUx28u/bRolK/XvpPJ
 kpS/TAhxZXe72TyEIO7BUfh5ifZ4FhMALK8DtHW9imL5ZHZ+iHIHVNVXxMcRcwZhZYofD4Dg
 wrhc8zSORRjt7icSHS4/7iSrC+vNSV9EVLudRPoXiNfvYC88dhbYgbnC4Y7SfHp3rUZDBmqm
 1i3QD4Ca6L/ZCLh/4u850yPpzuxqpXTQgcx6207tUr+tVghNeZJi2GygGU3DMqszq7FEjFtX
 1BewqByCdzi67nWz0Rhp81XQNmUCw6tamG0vLKWN8BJG86R03CiZ5tMxzp1OV1kNM0JERewP
 hSJ5l0BvsYJYCLxBUOSX25XI59xpUQHPY65Ps04k/IUOsQhHON51HsGibGsM5DFzxF3zPBX1
 WazesewF3cKYZmLPxLsL9rxJYQDn3hkrUuKHMiT503+jdK2OS7EIZ9YYQDmRr1os8u5TPD9r
 o832z2ikE4EDoUTo0D/rOYuELz9BSJgWM+q+5AJLL7rz8gPMDhJNsI9CIgJIuRNt69Uiv3J7
 je6XEpZw0D4nnrJNUOBbXULVV8ldcwXQasTMXN+MFC29WIkZIrzvq4Te4FuJess9fB5zO4yR
 P4AIp3SDvNKQzXB2jIccZii89AyKEX13VqDb3i/fTwyX598XAiVqNXqSRTiqXsVBS2tuMpg/
 7D5jlHHQYAOThhJBdrNbK791Eu4uHUQwbogX0bBLtRJVl/r9Yxmd374gvMtepleIhTf3DqKk
 Q2RBE5A9+XKpoY09vjPhLyF8Nj1Q7cvQBICEjCCv7isNCTc8m6y+qN6Ub6FLWLHSWf52KS+f
 uEJnfvyB+IKwQRRuI1mHrc1ka9nv4nzp6VXxxhPFWnQawj5EatpJ3SL0JUdtqBJwbMF6wK6V
 ljWp4tfMLSNfsjkDEQQNEwuaeHajaMYnTzb7PIUJkTm5XAooOrbABsKZxTc2jZAKLZVMZ8+x
 bZzscEb3AWzlx42P4vUlStT7WmNciQNXqhPWkv222M3Zt7HEm1/XKE=
IronPort-HdrOrdr: A9a23:9OqVB6oJTi6Ut7rs6mef10UaV5u/L9V00zEX/kB9WHVpm5Oj+f
 xGzc516farslossREb+expOMG7MBThHO1OkPUs1NCZLXTbUQqTXftfBO7ZogEIdBeOjtK1uZ
 0QF5SWa+eAfGSS7/yKmTVQeuxIqLLsndHK9IWuv0uFJTsaFZ2IhD0JbDpzfHcGITWuSaBJb6
 Z1saF81kWdkDksH4iG7j5vZZm2m/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTsj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZuA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQPZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv2nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLbU5nghBgr/DWQZAV2Iv/fKXJy/fB9kgIm3UyR9nFohvD2xRw7hdQAo5ot3Z
 WNDk0nrsAWcie6BZgNc9vpevHHeFAldyi8eV56EW6XZ53vBEi93qIfwI9Frt1CK6Z4gafbpv
 z6ISVlXCgJChrTNfE=
X-IronPort-AV: E=Sophos;i="5.85,327,1624334400"; 
   d="scan'208";a="53352501"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KlLiULX0gt1APS0lRbxARHJFUMUcrW+Ncn88jVSBXWHqGhyUUPrGp4ANu20TM3J2gDciiaD4TnvwnprZlX3KwyLglgSwhXzbeMX1MkzENDtY+0tVcjeiBMom27y2cRj0RTE1AvNvy6LMrGhYuqy7xXGRE3Z0/hH1DjlfxPtIeIddphABdnQBo7wKgYu6hULj6hWDMRwTLucDzytw+bEdXC8yleT8lHiPyc3lTnmn5gzXcxfzOnNRdAWIv2jcj4S7ptwHiHlPIbRVujqIBEIXFkF9ns1t14C1WGog2ohG7GmfgmzLEhTYz5V+9kje+YGVbj/pu5Axm46pz5K4Rh9q6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=T9zPLtDbhZa8cqsLzs9J9E7mUk9bKUS79E9/0KA/6kg=;
 b=iIwGhLZ3mhUTnHl1mr4/sKWgaXRy/uWBTATFhDPC4uuOigU03SpML9Dh0lPh9IEh/TvwMpBQ8o3Ixnw2KsqDwMOLLXHnCrXK3SPlW3dWBuZOSDLwLjBTQw5QISocqPQ0oOcwd+DF+HiD9eSHrLuUevIs9RpIroA5Yt7f0d/MJT/WfOjImqinyHyBz2JmBU3ODnupmAVpE2ru4iZMrMeTV7sDAASY97ZBxxqpWt3Km5uI6nQkv+p9tfoHwHsn8WbtAklc3w5YKHbcG/D8dWH+Q3m5P9K2/QOIUyfgoHHT8OswbIDzRWVAScrIjWG7KBJp3o9k4zwAIFRofkISIbKc7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=T9zPLtDbhZa8cqsLzs9J9E7mUk9bKUS79E9/0KA/6kg=;
 b=fKTuiXN9zQp9IXYbHae1L0tVbcBT03zTA05bHptbrC9gqo1ybX+RkRlSW4Qg4Qv792hQ4h2HRxcCSNV+JkyAP8CJgKLGOSE9U7nlrA5GMkOx+ovy7MsBhSmYgoh+W4s0SgiEs8aefn2K/t2RDkeTrtUZ6Z2t5zFrSl+A9/rwUng=
To: xen-devel <xen-devel@lists.xenproject.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>, Tim Deegan <tim@xen.org>,
	=?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>
Subject: Semantics of XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION
Message-ID: <13e3368d-89e3-360b-36df-bc215227b7e3@citrix.com>
Date: Mon, 27 Sep 2021 22:01:10 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0386.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:f::14) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c556677a-0b7a-495e-b8bc-08d981f9f2c2
X-MS-TrafficTypeDiagnostic: BYAPR03MB4680:
X-LD-Processed: 335836de-42ef-43a2-b145-348c2ee9ca5b,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4680C710006E96AA293E160BBAA79@BYAPR03MB4680.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: c9Qs1rI3QdeGLQb084h9L4U1bnzlqob9gQujo1EqQxwyPvGxWohhDiooGhMMPOH7m3mMTd4KfSxBswtO3CYmL+F9482pg3xxa4RqPSiVIDIR/omnI44wxpE2K1lZ60FLQrN1znJzPvS9vvuVSyiCTNx9ztymIAfWcmnOoj2m4wUP2qodq66uOz6y3RxIQs7X7GnckMUYTmM/m49BYitaJ7DgCtJpvylqtFx2NA/6bDNodBCXMeHBvkSccPQBc5AtVSSl15jO58WYQsAwUz6+1MLA6QSBLK8I89PzZ8kXRMXkvblvUZFFWwYcLIAaauaq3YV4wj7uy2Tf+hwO/T2VDWInoy5pzqnRdOGeUhfC8FiJPKI/gz09/jkiTbDCNA5OZivVFdaNpZLlxhCrR7GkKgKjA0xBmrN9AznLl2h8oBBb83kAQ4KdV4xiNvsa543irK4AnllugU/pJXwspx2uyd2JcOBpqXeGLDeyZ7S9wBsDmsStmTaGtzGDgFCG1cG9zmvl+6glgu+n7MBCNr+wcZtZMkx2w0v1zUhkHKmAaZIudS5CHRKYbUncwf7k/LRxNR1oOq5BKtQ2UCtTkBOo2dMsCYp/gkUz20Ird3hENT0jv4Ofj2xn/+XCaxouKXbgsIx8B/VMhGCav2RtImtu4bG783cQU0RwUUqzzWs4mYXSO9rAIoW2U76BpLa9ehsVHWJipVQ+p9MxGfloqISpLUnlxbRevq3JXw6Osl/+qhI=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(36756003)(54906003)(31686004)(956004)(6916009)(8936002)(86362001)(316002)(4326008)(83380400001)(16576012)(66556008)(66476007)(6486002)(66946007)(38100700002)(31696002)(186003)(55236004)(6666004)(5660300002)(2906002)(2616005)(107886003)(26005)(8676002)(7116003)(508600001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bExaM0NyZm1yUzZTYmhZUTN2aWdoU0NjUHB2ZmZoOEdVR1ZDeHNINE9HMmRr?=
 =?utf-8?B?TDRxdGNKRzZ6a2I3eTZuTXc3aDV2QVdha0s5bzF6RnQ5UjV5alhReW1oMk03?=
 =?utf-8?B?WnZuNXA1dlJPenJtV3VpVUVnK0d0Vi9Na0xuVHZyQmFMcU1GWHY3aE8zcWYv?=
 =?utf-8?B?MDZNTThBZkdXU0NRSllsRHovYWNnbVdtZHJudDlxbHlRckNBR1dnVHFGOXBU?=
 =?utf-8?B?aXpzdVZzRW40ckpIOVFmZFllRjhMTmIxVXlmdVRLL2Z2aTRKT0g0S1B3bmJP?=
 =?utf-8?B?K3NrSUR6TGlFaU0yclI0OXM5d0tZZExWZFBXTFRlellPUmQ1Ylo2a0RaTVMw?=
 =?utf-8?B?THV1cHdFUVd4U2FPcGV5aGdZc0xCUTFvcW1DNXMrMDJod1VmUmhFd3gvQWR2?=
 =?utf-8?B?Qno2Q2d5enhGUW1SZTBuTEU1a3A3VllXZWk1NERwWW1SM1BQZlAyQi9taFRn?=
 =?utf-8?B?dFE1VzQyQVU0OVAyN09yVm5Ld1lJVHJlYnZzSks1UVcrWWlJaEJvalhzSkY0?=
 =?utf-8?B?OHFxa2RFVGRGRWx0UkgwSmEzdlUyci9VSWoyV2o1ellVTm1BUDNib2pkYStv?=
 =?utf-8?B?aktTVzR5RVhGR3dwNEk0M0N6NVVlL1BQNDJwV1BlNzFBUVZUY283N3BvM1hK?=
 =?utf-8?B?WHo3VnZlM3k5SXRwbUhuRTBpeEEvNlJzYlFCVmlGQjZ6cmFJcEFGSy9EZEpp?=
 =?utf-8?B?UzVqMkdDZjNGbHBtbVVtSkU0L2lSOCtLNVc1ZjQ2dngwUkd0RzNYd0ZVaVd6?=
 =?utf-8?B?OHI4RGFrL0pwOFNnQWNvMVF1b2hhQk9xUkJET0IzM291Qng0OGx0dXJFOHB0?=
 =?utf-8?B?aC8yT0F1RjB0czZlelBmNGZRTC9CaVlwTWFMY1VqSjlJblJ5c283a1gyeVZI?=
 =?utf-8?B?aytscXRHMC9oWHNiUWdpbjBlanNGVnEyaWo0N2NObWd1Y3hIeFNod0JRV0xp?=
 =?utf-8?B?VDlvcHpLaFVObEp1YUJsMzZPTzd6TldFWXJIOEx5Qm9nN2I4dkw0cWJaNll0?=
 =?utf-8?B?U0M0WFNKa0lIdFFncGwyS1JhMGRGSW1FZkk1V1FNOXUrZHlZUlFQbUM5L1Q2?=
 =?utf-8?B?Y21BWHUxaGtJL3dSOVZnbnpaeGNGVGIydTFNb3p3d2x5UVRHTStuWXpTZGt3?=
 =?utf-8?B?QjZxQlU1VmVQV0xENVF1U05pbzM4QXZVV1FTZWhWd1d6OU90U1p2VDlVd0Jm?=
 =?utf-8?B?OFF2YlRZdDB3WlJoTlZtNDBmWWYyczJMZVcxOUhqM2ZoNGYzNm5BNWxmMysy?=
 =?utf-8?B?SXZEaTR1U2pOeHUwdHdlRUR6dkorSWQ3dWF4WHBrWW4ydVpZRWRXK2hSMWZa?=
 =?utf-8?B?UFJkbUExd1U2M043SEwwZ2xwZVROaDhWZVpmU3YrMzh2YWczZUExYnEyVGov?=
 =?utf-8?B?b1d6U2s1cEpmZStETzlESXJocVNIbzlqZmdLaUovRHk4REJWdUg4M0NZY3ZD?=
 =?utf-8?B?bzdIQVR2Skx3QnVid0lnbFZ4U0FxdTNpMS92Yy9MOFJ3bVpVc2NiV2VNVG1K?=
 =?utf-8?B?cmpiaVJPQlJzRGp2YnFWcjVRM3JlMVB0K2t1bUp3b1g2QWNWTWZKRGNlbFNC?=
 =?utf-8?B?VUduMjhkV1ZLU3M5Rk9uSTVIenhXQXQ2OEg3aGVZZkxnT1JRSFRXRWxERnBr?=
 =?utf-8?B?N1FCcmNJMDM5TWxLYUZ6NHU5d0VSTlJIRm5jbkU5WE1KU2daQ2gzWGVUcXR6?=
 =?utf-8?B?blk0aW50TjBkSytlUFFQb0w1VFZIYkhYbUdPRFR2YWJqZmY5eVVJNUhBdWdt?=
 =?utf-8?Q?3+tld0xhwJUJJXUd6lSbThaMV+BsO+xFrmXAU0i?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c556677a-0b7a-495e-b8bc-08d981f9f2c2
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2021 21:01:17.2271
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AUy1Xp6oQo9qLc/SSEveZTQ9EvJ4hvo7ZfScgPk2bgKDM5qri5anyrR+lBBAr6xnko0d0F17LE0VjZJD2wsP6rpY7syzMd+OpRQI4b11QQs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4680
X-OriginatorOrg: citrix.com

Hello,

A recent ABI change in Xen caused a total breakage under the Xapi
toolstack, and the investigation had lead to this.

First of all, the memory pool really needs renaming, because (not naming
names) multiple developers were fooled into thinking that the bug was
caused by a VM being unexpectedly in shadow mode.

Second, any MB value >=3D 0x1000000 will truncate to 0 between
{hap,shadow}_domctl() and {hap,shadow}_set_allocation().


But for the main issue, passing 0 in at the hypercall level is broken.

hap_enable() forces a minimum of 256 pages.=C2=A0 A subsequent hypercall
trying to set 0 frees {tot 245, free 244, p2m 11} all the way down to
{tot 1, free 0, p2m 11} before failing with -ENOMEM because there are no
more free pages to free.=C2=A0 Getting -ENOMEM from this kind of operation
isn't really correct.


Passing 0 cannot possibly function.=C2=A0 There are non-zero p2m frames by
the time createdomain completes, as we allocate the top of the p2m, and
they cannot be freed without the teardown logic which releases memory in
the correct order.

In fact, passing anything lower than the current free size is guaranteed
to fail.=C2=A0 Continuations also mean that you can't pick a value which is
guaranteed not to fail, because even a well (poorly?) placed foreign map
in dom0 could change the properties of the pool.


The shadow side rejects hypercall attempts using 0 (but can be bypassed
with the above truncation bug), and will try to drop shadows to get down
to the limit.=C2=A0 This represents a difference vs HAP, and in practice 1M
granularity is probably enough to ensure that you can't fail to set the
shadow allocation that low.=C2=A0 There is also a reachable BUG() somewhere
in this path as reported several times on xen-devel, but I still haven't
figured out how to tickle it.


There is no code or working usecase for reducing the size of the shadow
pool, except on domain destruction.=C2=A0 I think we should prohibit the
ability to shrink the shadow pool, and defer most of this mess to anyone
who turns up with a plausible usecase.

~Andrew



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197197.350164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygB-0000mh-P4; Mon, 27 Sep 2021 22:00:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197197.350164; Mon, 27 Sep 2021 22:00:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygB-0000ma-M9; Mon, 27 Sep 2021 22:00:55 +0000
Received: by outflank-mailman (input) for mailman id 197197;
 Mon, 27 Sep 2021 22:00:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUyg7-0000mS-Gr
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:00:54 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a842f28d-6bee-44e0-8ddd-7e048daccf24;
 Mon, 27 Sep 2021 22:00:48 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004Sv0-HC; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: a842f28d-6bee-44e0-8ddd-7e048daccf24
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=vmoV/2yvLiwBFQQklXCjpC+/vAkb2GZd/kil2P+K9LY=; b=1N4JPqGLhLSNsS1Cgb2YIwsly7
	+aW4YwpCslJYpkXAks9L5PAknlJFJXGDyS2ZEpaD2M0hKp7uuxemDnq0kmn0PiKNEhZfT3xU5OE2B
	NEIidZ82kagd9xiTvY4QBVhmfjUldvqMwEvJy3GCCTADP8JJLfZaVXaqvSZjYvDbf4jwE6N+/UCCO
	nFCZarCbKs9WvILUUJzj8n4GKX9IADqiqMHNghfdRfc0+glrPy84n7AImXEvFcu5sIum2O+W4K3/v
	voAp9xetJTWmWSHjJcDRqvnSUKtPXEmI0SsH4et/rU0BJwVZAwXRe6CxRMvzIZNqW955VNmdGfG+F
	VLf4/Vkw==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 05/10] nvdimm/btt: use goto error labels on btt_blk_init()
Date: Mon, 27 Sep 2021 15:00:34 -0700
Message-Id: <20210927220039.1064193-6-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

This will make it easier to share common error paths.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvdimm/btt.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 29cc7325e890..23ee8c005db5 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -1520,10 +1520,11 @@ static int btt_blk_init(struct btt *btt)
 {
 	struct nd_btt *nd_btt = btt->nd_btt;
 	struct nd_namespace_common *ndns = nd_btt->ndns;
+	int rc = -ENOMEM;
 
 	btt->btt_disk = blk_alloc_disk(NUMA_NO_NODE);
 	if (!btt->btt_disk)
-		return -ENOMEM;
+		goto out;
 
 	nvdimm_namespace_disk_name(ndns, btt->btt_disk->disk_name);
 	btt->btt_disk->first_minor = 0;
@@ -1535,19 +1536,23 @@ static int btt_blk_init(struct btt *btt)
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, btt->btt_disk->queue);
 
 	if (btt_meta_size(btt)) {
-		int rc = nd_integrity_init(btt->btt_disk, btt_meta_size(btt));
-
-		if (rc) {
-			blk_cleanup_disk(btt->btt_disk);
-			return rc;
-		}
+		rc = nd_integrity_init(btt->btt_disk, btt_meta_size(btt));
+		if (rc)
+			goto out_cleanup_disk;
 	}
+
 	set_capacity(btt->btt_disk, btt->nlba * btt->sector_size >> 9);
 	device_add_disk(&btt->nd_btt->dev, btt->btt_disk, NULL);
+
 	btt->nd_btt->size = btt->nlba * (u64)btt->sector_size;
 	nvdimm_check_and_set_ro(btt->btt_disk);
 
 	return 0;
+
+out_cleanup_disk:
+	blk_cleanup_disk(btt->btt_disk);
+out:
+	return rc;
 }
 
 static void btt_blk_cleanup(struct btt *btt)
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197199.350186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygJ-0001Lf-DY; Mon, 27 Sep 2021 22:01:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197199.350186; Mon, 27 Sep 2021 22:01:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygJ-0001LY-9y; Mon, 27 Sep 2021 22:01:03 +0000
Received: by outflank-mailman (input) for mailman id 197199;
 Mon, 27 Sep 2021 22:01:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygH-0000mS-EQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:01 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5d3805c9-e55a-48d0-a539-d4a837333b2a;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SvO-N8; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 5d3805c9-e55a-48d0-a539-d4a837333b2a
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=vLYLL0oFZhxGylgSJPWXsGjPK2psQoUWyxkb9au47Xw=; b=vFdtNbirDyd+cqwuiMxjiJzgLG
	Jt0ChJnf2JuPRFu4a+wlNaF4kthrKz7PxnttkR0Ds40s2EhZvp4JmkbSX6L3ADzaVDhCnLDp3cCs/
	l0Ao4N5wA1GF1nIHrrOdTV9k0vDEX8dt7ZIGbkn5jeXk4By4iFu7i+TmcRDENgYIuV6M0/X0COZJ8
	PTZwba2aQr3+WagO31hkbEZG4Js9Hr92YGi4M//TLw0Zk2t95HtZSK54Goe3CUTOj1kxdO0jr9W83
	2Y8mWhaksDHH+sEiqrhCSZnBTG/J8tSzqfz5X0PYVM21EAFDp2zVD0/Chb9dwkwaSbB31hydbvvRb
	1P5EOUhg==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 08/10] nvdimm/blk: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:37 -0700
Message-Id: <20210927220039.1064193-9-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

Since nvdimm/blk uses devm we just need to move the devm
registration towards the end. And in hindsight, that seems
to also provide a fix given del_gendisk() should not be
called unless the disk was already added via add_disk().
The probably of that issue happening is low though, like
OOM while calling devm_add_action(), so the fix is minor.

We manually unwind in case of add_disk() failure prior
to the devm registration.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvdimm/blk.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
index 591fa1f86f1e..9f1eb41404ac 100644
--- a/drivers/nvdimm/blk.c
+++ b/drivers/nvdimm/blk.c
@@ -265,7 +265,9 @@ static int nsblk_attach_disk(struct nd_namespace_blk *nsblk)
 	}
 
 	set_capacity(disk, available_disk_size >> SECTOR_SHIFT);
-	device_add_disk(dev, disk, NULL);
+	rc = device_add_disk(dev, disk, NULL);
+	if (rc)
+		goto out_before_devm_err;
 
 	/* nd_blk_release_disk() is called if this fails */
 	if (devm_add_action_or_reset(dev, nd_blk_release_disk, disk))
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197198.350175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygE-00012h-21; Mon, 27 Sep 2021 22:00:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197198.350175; Mon, 27 Sep 2021 22:00:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygD-00012a-UK; Mon, 27 Sep 2021 22:00:57 +0000
Received: by outflank-mailman (input) for mailman id 197198;
 Mon, 27 Sep 2021 22:00:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygC-0000mS-EJ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:00:56 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 92853f46-28ba-4555-aa0d-75f551b4771f;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004Sue-Cr; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 92853f46-28ba-4555-aa0d-75f551b4771f
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=hYL8LO+ietZVLiEWE2oEBW5mwRVdCiTP81FH5yaWaJI=; b=1BIITuUWUphtncAHlR3onSNJxl
	M6CmnasxwYUC18J5dAsyxcU45RRwAvsUth2gvXvlZCPWdpLhxADiTdWmmJnwp3Uk6w06QS7PX+wFY
	xppaHnOXRVB6MSKkNlcAfcp3c19hINrak2wAZGn+6pA7bP0hbBcVLP+ysEnGBFB8hzMRoaOxh9d8N
	EmHK04hOyMeu/LPfTOWs+mRlcoafAuyFpBBN3nOB+ukUTwMoJKSfcf/9HcGCwOOXot+mH5FL/B6ke
	PyL9SssBEH7fnKGJu6Uq+ETgRscCbWtIhSkBzmnno/dRwxrZeb5mjc5Tr1f/Pm86EFqDvRow06OXL
	BrqrshjA==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 02/10] bcache: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:31 -0700
Message-Id: <20210927220039.1064193-3-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

This driver doesn't do any unwinding with blk_cleanup_disk()
even on errors after add_disk() and so we follow that
tradition.

Acked-by: Coly Li <colyli@suse.de>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/md/bcache/super.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index f2874c77ff79..f0c32cdd6594 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1082,7 +1082,9 @@ int bch_cached_dev_run(struct cached_dev *dc)
 		closure_sync(&cl);
 	}
 
-	add_disk(d->disk);
+	ret = add_disk(d->disk);
+	if (ret)
+		goto out;
 	bd_link_disk_holder(dc->bdev, dc->disk.disk);
 	/*
 	 * won't show up in the uevent file, use udevadm monitor -e instead
@@ -1534,10 +1536,11 @@ static void flash_dev_flush(struct closure *cl)
 
 static int flash_dev_run(struct cache_set *c, struct uuid_entry *u)
 {
+	int err = -ENOMEM;
 	struct bcache_device *d = kzalloc(sizeof(struct bcache_device),
 					  GFP_KERNEL);
 	if (!d)
-		return -ENOMEM;
+		goto err_ret;
 
 	closure_init(&d->cl, NULL);
 	set_closure_fn(&d->cl, flash_dev_flush, system_wq);
@@ -1551,9 +1554,12 @@ static int flash_dev_run(struct cache_set *c, struct uuid_entry *u)
 	bcache_device_attach(d, c, u - c->uuids);
 	bch_sectors_dirty_init(d);
 	bch_flash_dev_request_init(d);
-	add_disk(d->disk);
+	err = add_disk(d->disk);
+	if (err)
+		goto err;
 
-	if (kobject_add(&d->kobj, &disk_to_dev(d->disk)->kobj, "bcache"))
+	err = kobject_add(&d->kobj, &disk_to_dev(d->disk)->kobj, "bcache");
+	if (err)
 		goto err;
 
 	bcache_device_link(d, c, "volume");
@@ -1567,7 +1573,8 @@ static int flash_dev_run(struct cache_set *c, struct uuid_entry *u)
 	return 0;
 err:
 	kobject_put(&d->kobj);
-	return -ENOMEM;
+err_ret:
+	return err;
 }
 
 static int flash_devs_run(struct cache_set *c)
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197200.350197 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygN-0001ge-Lf; Mon, 27 Sep 2021 22:01:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197200.350197; Mon, 27 Sep 2021 22:01:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygN-0001gS-Hz; Mon, 27 Sep 2021 22:01:07 +0000
Received: by outflank-mailman (input) for mailman id 197200;
 Mon, 27 Sep 2021 22:01:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygM-0000mS-EQ
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:06 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 44ecce16-2260-4fbf-be97-3488afd89593;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SvG-Ll; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 44ecce16-2260-4fbf-be97-3488afd89593
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=hsYRy5mW2eSZFEqOw8JTWcpX/S1jq2Dljgzx3k3VBB0=; b=VUhrw8hzQFU5q/FEnWuCu9vNeC
	DCvMINggtEXTZJxRA2FK/rIecWzokE3v00zXQby0C8xhCkSHSknRiHDJShGvvg5/WB7yFgDIDckZK
	PpnRl/rPKh332VtV3wML8z1zrXFA3i55GxnQCfAaEYw70wUGbzTEjF/468zPMAbP4jhENcEZMIqJm
	2Mu0n0pLv7IctMpH2UsQFSSaGesroZ0HRe1tGdCY1b4Bk06qhpX4Xv9n9tTQWerqKdilNkIIQwwUJ
	1JFfO38L6zcxmIzVXP2nckVh/bimkBRtBsjOqfHtqRjhaQH2lAQbVW3sNXoQ6wYWq6VFluDHCOtLw
	Cug+py4Q==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 07/10] nvdimm/blk: avoid calling del_gendisk() on early failures
Date: Mon, 27 Sep 2021 15:00:36 -0700
Message-Id: <20210927220039.1064193-8-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

If nd_integrity_init() fails we'd get del_gendisk() called,
but that's not correct as we should only call that if we're
done with device_add_disk(). Fix this by providing unwinding
prior to the devm call being registered and moving the devm
registration to the very end.

This should fix calling del_gendisk() if nd_integrity_init()
fails. I only spotted this issue through code inspection. It
does not fix any real world bug.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvdimm/blk.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
index 088d3dd6f6fa..591fa1f86f1e 100644
--- a/drivers/nvdimm/blk.c
+++ b/drivers/nvdimm/blk.c
@@ -240,6 +240,7 @@ static int nsblk_attach_disk(struct nd_namespace_blk *nsblk)
 	resource_size_t available_disk_size;
 	struct gendisk *disk;
 	u64 internal_nlba;
+	int rc;
 
 	internal_nlba = div_u64(nsblk->size, nsblk_internal_lbasize(nsblk));
 	available_disk_size = internal_nlba * nsblk_sector_size(nsblk);
@@ -256,20 +257,26 @@ static int nsblk_attach_disk(struct nd_namespace_blk *nsblk)
 	blk_queue_logical_block_size(disk->queue, nsblk_sector_size(nsblk));
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, disk->queue);
 
-	if (devm_add_action_or_reset(dev, nd_blk_release_disk, disk))
-		return -ENOMEM;
-
 	if (nsblk_meta_size(nsblk)) {
-		int rc = nd_integrity_init(disk, nsblk_meta_size(nsblk));
+		rc = nd_integrity_init(disk, nsblk_meta_size(nsblk));
 
 		if (rc)
-			return rc;
+			goto out_before_devm_err;
 	}
 
 	set_capacity(disk, available_disk_size >> SECTOR_SHIFT);
 	device_add_disk(dev, disk, NULL);
+
+	/* nd_blk_release_disk() is called if this fails */
+	if (devm_add_action_or_reset(dev, nd_blk_release_disk, disk))
+		return -ENOMEM;
+
 	nvdimm_check_and_set_ro(disk);
 	return 0;
+
+out_before_devm_err:
+	blk_cleanup_disk(disk);
+	return rc;
 }
 
 static int nd_blk_probe(struct device *dev)
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197201.350208 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygS-00023x-1Z; Mon, 27 Sep 2021 22:01:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197201.350208; Mon, 27 Sep 2021 22:01:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygR-00023f-Sg; Mon, 27 Sep 2021 22:01:11 +0000
Received: by outflank-mailman (input) for mailman id 197201;
 Mon, 27 Sep 2021 22:01:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygR-0000mS-ES
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:11 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 38d858f2-84fe-4674-8f1f-42af4c0ee3ab;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004Suu-Fq; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 38d858f2-84fe-4674-8f1f-42af4c0ee3ab
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=DvHJoJ6f5lcZ7SxjJw+FCx0/QeyauIuzT+VR1mX8TXg=; b=elnPQS+ctA3KT/ki2GQVFEC0IV
	RTWRmV32tiJGwilpjX3SNpHglIJBAVIeGXZKH+X3ixYZ7S6APxsLuq0LBFdyBEm0bI5NylSr6YGJk
	QzCSLP3xfX3rbUJ7B1uiGuEVT/xJeyaVhGqxlGo2m9uPPj12ehaPIOrXhs5sc+AnUE0yB322PBODD
	eVZWnMKrVd6oBx/TZLLisZG4oTrBSlBnaHMoONzXuDEfAylMjbvMOj7kvRLv5BYXdpUYCctkN9jT4
	zAlwg4mNR2tncvqZkqop5VrsUe7e8ibXvwtwAWUbxE+wPFTxBdbc1EVxV5wEO/TWYCRG1WD3mm1Wl
	l3K/xSmA==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 04/10] nvdimm/btt: do not call del_gendisk() if not needed
Date: Mon, 27 Sep 2021 15:00:33 -0700
Message-Id: <20210927220039.1064193-5-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We know we don't need del_gendisk() if we haven't added
the disk, so just skip it. This should fix a bug on older
kernels, as del_gendisk() became able to deal with
disks not added only recently, after the patch titled
"block: add flag for add_disk() completion notation".

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvdimm/btt.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 52de60b7adee..29cc7325e890 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -1538,7 +1538,6 @@ static int btt_blk_init(struct btt *btt)
 		int rc = nd_integrity_init(btt->btt_disk, btt_meta_size(btt));
 
 		if (rc) {
-			del_gendisk(btt->btt_disk);
 			blk_cleanup_disk(btt->btt_disk);
 			return rc;
 		}
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197203.350219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygX-0002YA-BA; Mon, 27 Sep 2021 22:01:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197203.350219; Mon, 27 Sep 2021 22:01:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygX-0002Y1-84; Mon, 27 Sep 2021 22:01:17 +0000
Received: by outflank-mailman (input) for mailman id 197203;
 Mon, 27 Sep 2021 22:01:16 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygW-0000mS-EV
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:16 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a94eef9-03fa-4a35-bcd0-411b3f8d8836;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004Suk-EY; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 2a94eef9-03fa-4a35-bcd0-411b3f8d8836
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=7F58wxD/dEQonKrzNoWM7DkNk/UH9ZF8NZLJCFZZJo4=; b=ZnlFI8fLEZC+pXL09OaLsKjdn8
	sG2hdDcnkdd7ZQAAOmWguGc7OnVMevoBr9Iei6cSMyXAQixEc8APBeN/A8M4gCqUCYojYyBiRia7O
	7wDEec/aFp7xuN5F775zNLNzsTp/rSR3Ay3VTbK1PNHfZokfRNvqNAX9BvqfJMUKuSdi3rm3MuWeX
	L6a8Q4FjRtkn9UeJ4Lhp3Lt/jbaePXbm+u8tO6lhIOBWl8Zvgdk3hjBymTruCXrQy+1R0u429f+a4
	B0bkzO9hXCn6QDiaV77uGrK+Hfqt+kENvWzGuFAfXSbunwOgQu1cHy4uiqeb5kU5vGXxTPlHBSNa9
	7+WGTsLw==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 03/10] nvme-multipath: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:32 -0700
Message-Id: <20210927220039.1064193-4-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

Since we now can tell for sure when a disk was added, move
setting the bit NVME_NSHEAD_DISK_LIVE only when we did
add the disk successfully.

Nothing to do here as the cleanup is done elsewhere. We take
care and use test_and_set_bit() because it is protects against
two nvme paths simultaneously calling device_add_disk() on the
same namespace head.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvme/host/multipath.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index e8ccdd398f78..35cace4f3f5f 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -496,13 +496,22 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
 static void nvme_mpath_set_live(struct nvme_ns *ns)
 {
 	struct nvme_ns_head *head = ns->head;
+	int rc;
 
 	if (!head->disk)
 		return;
 
+	/*
+	 * test_and_set_bit() is used because it is protecting against two nvme
+	 * paths simultaneously calling device_add_disk() on the same namespace
+	 * head.
+	 */
 	if (!test_and_set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags)) {
-		device_add_disk(&head->subsys->dev, head->disk,
-				nvme_ns_id_attr_groups);
+		rc = device_add_disk(&head->subsys->dev, head->disk,
+				     nvme_ns_id_attr_groups);
+		if (rc)
+			return;
+		set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags);
 		nvme_add_ns_head_cdev(head);
 	}
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197205.350230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygc-00030T-LP; Mon, 27 Sep 2021 22:01:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197205.350230; Mon, 27 Sep 2021 22:01:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygc-00030F-HM; Mon, 27 Sep 2021 22:01:22 +0000
Received: by outflank-mailman (input) for mailman id 197205;
 Mon, 27 Sep 2021 22:01:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygb-0000mS-En
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:21 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 946b4533-7b5a-43da-948d-9031806784ea;
 Mon, 27 Sep 2021 22:00:49 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SvT-Oa; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 946b4533-7b5a-43da-948d-9031806784ea
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=n0t709e+/n6TLkwKfO+OryH6p8FtdPpV7nSeauOqbbE=; b=TLHZvka2DfAcoJlgj2gIUo2wwQ
	kBYYzdPNPYZC/AnRdkDO1kXKwqIG4rX1TuRVaeYtm8g8FZZLnXZznHA97zWn40Cv+a49q1z7q0fZa
	slzU1wKKIiAjk3kitEZmW44QvUwxkBOqzr/j2l5uAv8zMAnIa38hcxTP2ODG/t5RHM9iNanZDHu/7
	ZE0n1Z2FLyslzo/QQ7+mSDhGkk2EDFgc4z39HCJ9EZKnv11bxOqtIBEY2UOuJM8Cw3/2GicMyHVGG
	QLkdR4EXXZuimuvP3kbLVXuTIVA1Zlt1Xfds0uWdLzXPM0UJNwE3jIdqAPP3/+yI4lDSsg6Ur4uif
	adlVMKew==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 09/10] xen-blkfront: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:38 -0700
Message-Id: <20210927220039.1064193-10-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on device_add_disk() as this function
returned void. Now that this is fixed, use the shiny new error
handling. The function xlvbd_alloc_gendisk() typically does the
unwinding on error on allocating the disk and creating the tag,
but since all that error handling was stuffed inside
xlvbd_alloc_gendisk() we must repeat the tag free'ing as well.

We set the info->rq to NULL to ensure blkif_free() doesn't crash
on blk_mq_stop_hw_queues() on device_add_disk() error as the queue
will be long gone by then.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/block/xen-blkfront.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 72902104f111..86440b051766 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -2385,7 +2385,13 @@ static void blkfront_connect(struct blkfront_info *info)
 	for_each_rinfo(info, rinfo, i)
 		kick_pending_request_queues(rinfo);
 
-	device_add_disk(&info->xbdev->dev, info->gd, NULL);
+	err = device_add_disk(&info->xbdev->dev, info->gd, NULL);
+	if (err) {
+		blk_cleanup_disk(info->gd);
+		blk_mq_free_tag_set(&info->tag_set);
+		info->rq = NULL;
+		goto fail;
+	}
 
 	info->is_ready = 1;
 	return;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197207.350240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygh-0003Vl-1V; Mon, 27 Sep 2021 22:01:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197207.350240; Mon, 27 Sep 2021 22:01:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygg-0003VY-TX; Mon, 27 Sep 2021 22:01:26 +0000
Received: by outflank-mailman (input) for mailman id 197207;
 Mon, 27 Sep 2021 22:01:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygg-0000mS-Em
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:26 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c60016d9-c785-4540-8a33-235a217df72c;
 Mon, 27 Sep 2021 22:00:50 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SuN-7t; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: c60016d9-c785-4540-8a33-235a217df72c
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type:
	Content-ID:Content-Description:In-Reply-To:References;
	bh=f1z5HgkOwmjFBk4lUYxpkBwDgNqINDcFoZThfyywwx8=; b=eQhJDUwgI2a4kvwwC1LW4Grcbe
	Afldl7Y/5P+CtLt6Bf7Y9ytxFLTlAL7DQ1mrwOtj6+6Pnddn8Wp4NkxVuxqvoNw0Tyz5tv3XvMzt/
	95ad2a+T3zdPfY1HqSNzUwK9FALGPXQfHChw080YZLrustezGq9sWxl4CDGODiD+GUg6c3Ya2VpOL
	Lz+0XMYtVA7Q3XN0iSeVnjb/6bCKu9f0aIxF7V2Oxq68qH0zcltU5tGD/tQ+xGADbb+jbit4yqFIv
	EQ6Wyv4G24Wl6sIzPggmISuK4WSugRZrguNrS2KSbhhqFI7oFTnyVJEEiHe4GjAGQLWYlZNDx3Kcq
	wpi9PwkQ==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 00/10] block: second batch of add_disk() error handling conversions
Date: Mon, 27 Sep 2021 15:00:29 -0700
Message-Id: <20210927220039.1064193-1-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

This is the second series of driver conversions for add_disk()
error handling. You can find this set and the rest of the 7th set of
driver conversions on my 20210927-for-axboe-add-disk-error-handling
branch [0].

Changes on this v2 since the last first version of this
patch series:

  - rebased onto linux-next tag 20210927
  - nvme-multipath: used test_and_set_bit() as suggested by Keith Busch,         
    and justified this in the code with a comment as this race was not
    obvious
  - Added reviewed-by / Acked-by tags where one was provided 

[0] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git/log/?h=20210927-for-axboe-add-disk-error-handling

Luis Chamberlain (10):
  block/brd: add error handling support for add_disk()
  bcache: add error handling support for add_disk()
  nvme-multipath: add error handling support for add_disk()
  nvdimm/btt: do not call del_gendisk() if not needed
  nvdimm/btt: use goto error labels on btt_blk_init()
  nvdimm/btt: add error handling support for add_disk()
  nvdimm/blk: avoid calling del_gendisk() on early failures
  nvdimm/blk: add error handling support for add_disk()
  xen-blkfront: add error handling support for add_disk()
  zram: add error handling support for add_disk()

 drivers/block/brd.c           | 10 ++++++++--
 drivers/block/xen-blkfront.c  |  8 +++++++-
 drivers/block/zram/zram_drv.c |  6 +++++-
 drivers/md/bcache/super.c     | 17 ++++++++++++-----
 drivers/nvdimm/blk.c          | 21 +++++++++++++++------
 drivers/nvdimm/btt.c          | 24 +++++++++++++++---------
 drivers/nvme/host/multipath.c | 13 +++++++++++--
 7 files changed, 73 insertions(+), 26 deletions(-)

-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197209.350252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygm-00047e-BN; Mon, 27 Sep 2021 22:01:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197209.350252; Mon, 27 Sep 2021 22:01:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygm-00047X-6y; Mon, 27 Sep 2021 22:01:32 +0000
Received: by outflank-mailman (input) for mailman id 197209;
 Mon, 27 Sep 2021 22:01:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygl-0000mS-F7
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:31 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6cf0e5b8-187c-4d9c-80b9-ddd084603ee4;
 Mon, 27 Sep 2021 22:00:50 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SvY-Q6; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 6cf0e5b8-187c-4d9c-80b9-ddd084603ee4
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=Zp3peWChzsLwJZgR1i8Dtyh3FOO544lFeJzlWbkfyco=; b=xyicbK2H0CR7mxWgNNAbZZVOoG
	f/TY+aOtnMS/uoVtMXqtPXis+vbZhtpiS/ndUP0AdX9IDRGIobYET1+iWMgf0eic3hI0JSLbAulKG
	8wvXxEzmhDvXeOmmj8TPh+HGQ0Q2wvHySLXwxFAgsiQ0IWl+sk40LXHXcOXOHWUbo3breLF9OOmtC
	aU5QvqBV5mC+pgxhFyfAWSzB39tzj8teDlNoEJNNKk8zoulQfqxsIBVBuYTJraMMbw8Z6hDN9ZQpa
	LM244ocJ/fpPGEyLZkxh+GCiIuoIMgqFitcviGyRgvFztoaDWVexx1nlLNy/oEUY4relGyOFoN2oa
	SP4Bj1fA==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 10/10] zram: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:39 -0700
Message-Id: <20210927220039.1064193-11-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/block/zram/zram_drv.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index f61910c65f0f..59086e178fbd 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1953,7 +1953,9 @@ static int zram_add(void)
 		blk_queue_max_write_zeroes_sectors(zram->disk->queue, UINT_MAX);
 
 	blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES, zram->disk->queue);
-	device_add_disk(NULL, zram->disk, zram_disk_attr_groups);
+	ret = device_add_disk(NULL, zram->disk, zram_disk_attr_groups);
+	if (ret)
+		goto out_cleanup_disk;
 
 	strlcpy(zram->compressor, default_compressor, sizeof(zram->compressor));
 
@@ -1961,6 +1963,8 @@ static int zram_add(void)
 	pr_info("Added device: %s\n", zram->disk->disk_name);
 	return device_id;
 
+out_cleanup_disk:
+	blk_cleanup_disk(zram->disk);
 out_free_idr:
 	idr_remove(&zram_index_idr, device_id);
 out_free_dev:
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:01:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:01:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197211.350263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygr-0004nC-QH; Mon, 27 Sep 2021 22:01:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197211.350263; Mon, 27 Sep 2021 22:01:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUygr-0004ly-M4; Mon, 27 Sep 2021 22:01:37 +0000
Received: by outflank-mailman (input) for mailman id 197211;
 Mon, 27 Sep 2021 22:01:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygq-0000mS-FD
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:36 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42fe048f-1dbc-4ca2-b663-4e7f16da06f0;
 Mon, 27 Sep 2021 22:00:50 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004SvC-K7; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 42fe048f-1dbc-4ca2-b663-4e7f16da06f0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=RKU66AnJzLKhhgSWEHLOfZ4aIkidqzrddEWqP9yNa3s=; b=DlwowCHOstgFTHo1OasqvBBrRA
	Ln5Lnf/XerBeGqMUcmhKvtfbTCWtF7GYSUu6/OOKkAt4grh0hFRd5deBekspTSIWg8AjRZ2eeZKUD
	6NbxnpIf/ny+fShWjWFE6NSx57iAqpvB2STaOYmGxyb8gkLmyIU2giqEJ/wc7Q+0HvyHInp4oNX0U
	bQN6nQAoBsEQtyjaUCBNU3TPonPPX0fO9u+NSEcfBEKREBZlbiBhi12yjpAEzm6ia6MdWXUqfZHUV
	P/j52UBMOkCxN2xoIsyxrUw2HuJVLpC29l30Op1PLWCmJ9OOvtEigV+Jc9XvZfDpXkyxXLigTBOU0
	B5Qzou+A==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 06/10] nvdimm/btt: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:35 -0700
Message-Id: <20210927220039.1064193-7-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/nvdimm/btt.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 23ee8c005db5..57b921c5fbb5 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -1542,7 +1542,9 @@ static int btt_blk_init(struct btt *btt)
 	}
 
 	set_capacity(btt->btt_disk, btt->nlba * btt->sector_size >> 9);
-	device_add_disk(&btt->nd_btt->dev, btt->btt_disk, NULL);
+	rc = device_add_disk(&btt->nd_btt->dev, btt->btt_disk, NULL);
+	if (rc)
+		goto out_cleanup_disk;
 
 	btt->nd_btt->size = btt->nlba * (u64)btt->sector_size;
 	nvdimm_check_and_set_ro(btt->btt_disk);
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:07:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:07:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197241.350274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUymM-0006zA-DP; Mon, 27 Sep 2021 22:07:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197241.350274; Mon, 27 Sep 2021 22:07:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUymM-0006z3-9v; Mon, 27 Sep 2021 22:07:18 +0000
Received: by outflank-mailman (input) for mailman id 197241;
 Mon, 27 Sep 2021 22:07:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUymK-0006yA-Tg; Mon, 27 Sep 2021 22:07:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUymK-0001GJ-L5; Mon, 27 Sep 2021 22:07:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUymK-00072g-7Z; Mon, 27 Sep 2021 22:07:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUymK-0000k8-73; Mon, 27 Sep 2021 22:07:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UUb9MoaRVU8WdS/Dd5D+tEnW8d+GzZ10tUkrCskSMmY=; b=WiScS57TLvJXXx86Fxi2ph/uEK
	teRso4hN1W9BlK8xYhmRMeW4PO6N+z981fFv8XWeJ9pbZNnNvmK8pxWk6y5uLawcJlq31yoCBDgqn
	H5hIxSD20rqT92Not8Z9qqN8jMpUew4beTUY2eNLxYBl6YZ1G4KNOLDKCr2exWb73pFE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165215-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165215: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=5816b3e6577eaa676ceb00a848f0fd65fe2adc29
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 22:07:16 +0000

flight 165215 linux-linus real [real]
flight 165224 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165215/
http://logs.test-lab.xenproject.org/osstest/logs/165224/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-xsm 18 guest-start/debian.repeat fail pass in 165224-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                5816b3e6577eaa676ceb00a848f0fd65fe2adc29
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  423 days
Failing since        152366  2020-08-01 20:49:34 Z  422 days  738 attempts
Testing same since   165215  2021-09-27 04:24:58 Z    0 days    1 attempts

------------------------------------------------------------
7341 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2271626 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:07:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:07:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197253.350287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUyme-0007VP-Nz; Mon, 27 Sep 2021 22:07:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197253.350287; Mon, 27 Sep 2021 22:07:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUyme-0007V4-Jp; Mon, 27 Sep 2021 22:07:36 +0000
Received: by outflank-mailman (input) for mailman id 197253;
 Mon, 27 Sep 2021 22:07:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0LRd=OR=infradead.org=mcgrof@srs-us1.protection.inumbo.net>)
 id 1mUygv-0000mS-FF
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:01:41 +0000
Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f6487bb8-665e-4e21-b066-64e0c942043c;
 Mon, 27 Sep 2021 22:00:53 +0000 (UTC)
Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2
 (Red Hat Linux)) id 1mUyfw-004Sua-BV; Mon, 27 Sep 2021 22:00:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: f6487bb8-665e-4e21-b066-64e0c942043c
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:
	Reply-To:Content-Type:Content-ID:Content-Description;
	bh=vzfFGe77MiDtDAcsNEd4FFV6XUqpkLp7v2/+XiGswys=; b=1IjRffMWToIuI5hlT0QDQQle10
	h2cmW94BFoi6hungxxRyemGHngj6jAyo6qgzCDtNwyoUIWdx8yZsIACgBw+d6Nd4692A4UqE77TGE
	FCZRmyjQTpFaOofQoJtOcS2Pkb2YvtxiwdI6lkhFHGjLfTo7fvE2zEi7m6m4dPHjKrCOvij4KUPWH
	DSYAexzPrZhtGZMXRAVrRoXp4+plHqrbeuTjWRI7g5ndnkdAFIDfDHEv6r0wdbAfWet/1/8nTrT68
	8nOiI5xCjxlwlC2oda7p9NvctMKAA8h8pcRZ7GYw/ktOiSjdKEDnnDoQuyC514ICPIyH9sAgE7omc
	UHhUFlIg==;
From: Luis Chamberlain <mcgrof@kernel.org>
To: axboe@kernel.dk,
	colyli@suse.de,
	kent.overstreet@gmail.com,
	kbusch@kernel.org,
	sagi@grimberg.me,
	vishal.l.verma@intel.com,
	dan.j.williams@intel.com,
	dave.jiang@intel.com,
	ira.weiny@intel.com,
	konrad.wilk@oracle.com,
	roger.pau@citrix.com,
	boris.ostrovsky@oracle.com,
	jgross@suse.com,
	sstabellini@kernel.org,
	minchan@kernel.org,
	ngupta@vflare.org,
	senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org,
	nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org,
	linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v2 01/10] block/brd: add error handling support for add_disk()
Date: Mon, 27 Sep 2021 15:00:30 -0700
Message-Id: <20210927220039.1064193-2-mcgrof@kernel.org>
X-Mailer: git-send-email 2.31.1
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: Luis Chamberlain <mcgrof@infradead.org>

We never checked for errors on add_disk() as this function
returned void. Now that this is fixed, use the shiny new
error handling.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 drivers/block/brd.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 58ec167aa018..c2bf4946f4e3 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -372,6 +372,7 @@ static int brd_alloc(int i)
 	struct brd_device *brd;
 	struct gendisk *disk;
 	char buf[DISK_NAME_LEN];
+	int err = -ENOMEM;
 
 	brd = kzalloc(sizeof(*brd), GFP_KERNEL);
 	if (!brd)
@@ -410,14 +411,19 @@ static int brd_alloc(int i)
 	/* Tell the block layer that this is not a rotational device */
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, disk->queue);
 	blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, disk->queue);
-	add_disk(disk);
+	err = add_disk(disk);
+	if (err)
+		goto out_cleanup_disk;
+
 	list_add_tail(&brd->brd_list, &brd_devices);
 
 	return 0;
 
+out_cleanup_disk:
+	blk_cleanup_disk(disk);
 out_free_dev:
 	kfree(brd);
-	return -ENOMEM;
+	return err;
 }
 
 static void brd_probe(dev_t dev)
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:13:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:13:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197274.350298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUysC-0000h7-Ea; Mon, 27 Sep 2021 22:13:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197274.350298; Mon, 27 Sep 2021 22:13:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUysC-0000h0-Ba; Mon, 27 Sep 2021 22:13:20 +0000
Received: by outflank-mailman (input) for mailman id 197274;
 Mon, 27 Sep 2021 22:13:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v5uC=OR=kernel.org=kbusch@srs-us1.protection.inumbo.net>)
 id 1mUysA-0000gt-Ut
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:13:19 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1df77b4a-1fe0-11ec-bc5f-12813bfff9fa;
 Mon, 27 Sep 2021 22:13:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 014C460F94;
 Mon, 27 Sep 2021 22:13:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1df77b4a-1fe0-11ec-bc5f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632780797;
	bh=aUkvMSo84Ts+2hyfPmGVIgM4abSwV67b0JY7DU1pVzk=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=CR3ThJMTzNN7IxGCoQ9SihnqkQkkv3F41YJaAr67klqD8ay7cRcSeppUUhh/8+7PQ
	 DFxOyaBzoI962F2usbjfziq1easTinlTyXoLnTwCKEIgdylqvj9h8QEVuaeDIXyhCf
	 8pRA7I8t7831enPRtpE/358VVKwNsUwrPG5b/lRH5qyH1IXU821Yn+C1LYzW65TQiM
	 Kr2pNyLsYwmCCd+A88QwrYslEh5QjuzWUx1TBVYz6/tBtiRBBMVllLOKUI3Il17WX7
	 FhuTQ1Q9A8S1JL3ILSV0TNBfSOS3l81jmsFCnMM1Pvg1VT7bDaBh2KmwsWKeTC3U9m
	 OIVZBum14ZcHw==
Date: Mon, 27 Sep 2021 15:13:12 -0700
From: Keith Busch <kbusch@kernel.org>
To: Luis Chamberlain <mcgrof@kernel.org>
Cc: axboe@kernel.dk, colyli@suse.de, kent.overstreet@gmail.com,
	sagi@grimberg.me, vishal.l.verma@intel.com,
	dan.j.williams@intel.com, dave.jiang@intel.com, ira.weiny@intel.com,
	konrad.wilk@oracle.com, roger.pau@citrix.com,
	boris.ostrovsky@oracle.com, jgross@suse.com, sstabellini@kernel.org,
	minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org,
	xen-devel@lists.xenproject.org, nvdimm@lists.linux.dev,
	linux-nvme@lists.infradead.org, linux-bcache@vger.kernel.org,
	linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 03/10] nvme-multipath: add error handling support for
 add_disk()
Message-ID: <20210927221312.GD387558@dhcp-10-100-145-180.wdc.com>
References: <20210927220039.1064193-1-mcgrof@kernel.org>
 <20210927220039.1064193-4-mcgrof@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210927220039.1064193-4-mcgrof@kernel.org>

On Mon, Sep 27, 2021 at 03:00:32PM -0700, Luis Chamberlain wrote:
> +	/*
> +	 * test_and_set_bit() is used because it is protecting against two nvme
> +	 * paths simultaneously calling device_add_disk() on the same namespace
> +	 * head.
> +	 */
>  	if (!test_and_set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags)) {
> -		device_add_disk(&head->subsys->dev, head->disk,
> -				nvme_ns_id_attr_groups);
> +		rc = device_add_disk(&head->subsys->dev, head->disk,
> +				     nvme_ns_id_attr_groups);
> +		if (rc)
> +			return;
> +		set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags);

No need to set_bit() here since the test_and_set_bit() already took care
of that.


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:33:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:33:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197280.350310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzBu-0003MF-0Z; Mon, 27 Sep 2021 22:33:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197280.350310; Mon, 27 Sep 2021 22:33:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzBt-0003M8-T5; Mon, 27 Sep 2021 22:33:41 +0000
Received: by outflank-mailman (input) for mailman id 197280;
 Mon, 27 Sep 2021 22:33:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=5dgL=OR=kernel.dk=axboe@srs-us1.protection.inumbo.net>)
 id 1mUzBs-0003M2-Ix
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 22:33:40 +0000
Received: from mail-il1-x12f.google.com (unknown [2607:f8b0:4864:20::12f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6d6f68b7-db2f-4bf5-8530-fedd7bda7fec;
 Mon, 27 Sep 2021 22:33:39 +0000 (UTC)
Received: by mail-il1-x12f.google.com with SMTP id i13so21078829ilm.4
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 15:33:39 -0700 (PDT)
Received: from [192.168.1.116] ([66.219.217.159])
 by smtp.gmail.com with ESMTPSA id y2sm6427866iot.45.2021.09.27.15.33.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 27 Sep 2021 15:33:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d6f68b7-db2f-4bf5-8530-fedd7bda7fec
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=kernel-dk.20210112.gappssmtp.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=LOsuwHDiPC+PPoCEQN/qtQeyMVFZbR3g1AzPIbWGz1s=;
        b=YTqmdXk5vu2O6Z09DphFzhzdWxPQ8jmaJFHrIsMI+6Jhr8dovpFIgf0hfWsFZmWJGq
         EVBCGkS1YFFNZ8M01WgONYjDBxGTGP/DCZWSKVCmRzFam2Hh4IgqxVmjePf268PGtV81
         f5vMDXMmwBYUfauwYcel2Rv7+Z/vgk39qzYsRkTAosBsqgB4GwDsvm6cOxLKEb7WhBzu
         TSGL2cQt0huoDrGurVnZof3FIGpxyaHR8+jLKBQHjcW0vLlSgpEdy3xv/8pfNM0RjCzL
         S68pit92xtiJTYJshqe0ROVs/67npCPOQRoa5DA5iXWksCwd+SjdXeI2A9yZ8hXhKis6
         coXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=LOsuwHDiPC+PPoCEQN/qtQeyMVFZbR3g1AzPIbWGz1s=;
        b=QnyLgKsnVbL44MDpOMvcFR8XcfrpXJBp60aqjhLj7OQMTF6Kr7kkEupqvZDnTqVRnk
         CuhGZgG/V4e9lhSLHSywkq/NJ95Sbs93ANMaOk6k0PaJskqc5Bjq1Fa9vg0DDRoB9hK3
         oLOBwEx3befay3JQfqS5124X2J29B06qFo1oOSJHfwgM4Gkgjf8J7NzQR9UyT8kbJD1O
         Al/A03hixKWWHYzu+PWe8WWyOlSO8kePbZvHbd4rRZoRtVIX1Yu3dToMEQW/JDa57jmH
         U2gv/fBtRPKbuev8bHzBzCa9ZchoK9hFv2WLfNNGy1LXPEcJ1dl2wu7zm2aATntQFX2E
         b0pQ==
X-Gm-Message-State: AOAM531XDJYEUzTtq6FXmXY0RpJ5XHuq1Kwird0AMHyZarzRMFqqvPmo
	+bmkVlF/LWcYx9/hLemmpYDXHw==
X-Google-Smtp-Source: ABdhPJwueWxtKqjP8KyNWWEV43ZHbAJgLgbtzL1ErWyEowm3/77Murhw193eceS6jFlQ9sjaEQjTVg==
X-Received: by 2002:a92:da85:: with SMTP id u5mr1851944iln.213.1632782018707;
        Mon, 27 Sep 2021 15:33:38 -0700 (PDT)
Subject: Re: [PATCH v2 00/10] block: second batch of add_disk() error handling
 conversions
To: Luis Chamberlain <mcgrof@kernel.org>, colyli@suse.de,
 kent.overstreet@gmail.com, kbusch@kernel.org, sagi@grimberg.me,
 vishal.l.verma@intel.com, dan.j.williams@intel.com, dave.jiang@intel.com,
 ira.weiny@intel.com, konrad.wilk@oracle.com, roger.pau@citrix.com,
 boris.ostrovsky@oracle.com, jgross@suse.com, sstabellini@kernel.org,
 minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org, nvdimm@lists.linux.dev,
 linux-nvme@lists.infradead.org, linux-bcache@vger.kernel.org,
 linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20210927220039.1064193-1-mcgrof@kernel.org>
From: Jens Axboe <axboe@kernel.dk>
Message-ID: <c17004ed-884f-5a97-c333-602e3f9903e7@kernel.dk>
Date: Mon, 27 Sep 2021 16:33:37 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210927220039.1064193-1-mcgrof@kernel.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/27/21 4:00 PM, Luis Chamberlain wrote:
> This is the second series of driver conversions for add_disk()
> error handling. You can find this set and the rest of the 7th set of
> driver conversions on my 20210927-for-axboe-add-disk-error-handling
> branch [0].

Applied 1, thanks.

-- 
Jens Axboe



From xen-devel-bounces@lists.xenproject.org Mon Sep 27 22:47:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 22:47:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197288.350321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzP1-0005E4-AI; Mon, 27 Sep 2021 22:47:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197288.350321; Mon, 27 Sep 2021 22:47:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzP1-0005Dx-6o; Mon, 27 Sep 2021 22:47:15 +0000
Received: by outflank-mailman (input) for mailman id 197288;
 Mon, 27 Sep 2021 22:47:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUzOz-0005Dn-Ex; Mon, 27 Sep 2021 22:47:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUzOz-0001vj-4o; Mon, 27 Sep 2021 22:47:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mUzOy-0008NO-Pz; Mon, 27 Sep 2021 22:47:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mUzOy-0003Fk-PR; Mon, 27 Sep 2021 22:47:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=guyT0uxfrLoL5wwMfWvQDLimhxE8BRrt6nRrZhOVc5k=; b=kKFurEIM2XLSwMVAjz6E4LjZS4
	NU3PFTZMCaAp0lfE7pZMJ5xx5DFRuRDMR8ldFoDfRKrwAX0FaLsvB4MSVa4XhK6CIw3cj2ZcKMth8
	zb8jGF6mlELAg4JDi3Ng3MuQiWQujk9McJy2YwIBeq4W7gpPhqWM314dZEzxC8BdN5mc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165217-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165217: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:xen-boot:fail:heisenbug
    qemu-mainline:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=11a11998460ed84d9a127c025f50f7234e5a483f
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 27 Sep 2021 22:47:12 +0000

flight 165217 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165217/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                   fail pass in 165210

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds  20 guest-localmigrate/x10 fail in 165210 like 164950
 test-armhf-armhf-xl-rtds    15 migrate-support-check fail in 165210 never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-check fail in 165210 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                11a11998460ed84d9a127c025f50f7234e5a483f
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   16 days
Failing since        164967  2021-09-13 13:06:52 Z   14 days   28 attempts
Testing same since   165193  2021-09-25 11:09:10 Z    2 days    5 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5088 lines long.)


From xen-devel-bounces@lists.xenproject.org Mon Sep 27 23:13:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 27 Sep 2021 23:13:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197300.350334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzo3-0000Dv-Fl; Mon, 27 Sep 2021 23:13:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197300.350334; Mon, 27 Sep 2021 23:13:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mUzo3-0000Do-Cm; Mon, 27 Sep 2021 23:13:07 +0000
Received: by outflank-mailman (input) for mailman id 197300;
 Mon, 27 Sep 2021 23:13:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WuYw=OR=epam.com=prvs=990419b1bf=volodymyr_babchuk@srs-us1.protection.inumbo.net>)
 id 1mUzo1-0000Di-Vx
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 23:13:06 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 77e233ea-1fe8-11ec-bc60-12813bfff9fa;
 Mon, 27 Sep 2021 23:13:04 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18RN3oTN022921; 
 Mon, 27 Sep 2021 23:13:02 GMT
Received: from eur01-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2054.outbound.protection.outlook.com [104.47.1.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbq9tg14t-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 27 Sep 2021 23:13:02 +0000
Received: from PAXPR03MB7594.eurprd03.prod.outlook.com (2603:10a6:102:201::14)
 by PAXPR03MB8013.eurprd03.prod.outlook.com (2603:10a6:102:21c::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Mon, 27 Sep
 2021 23:12:59 +0000
Received: from PAXPR03MB7594.eurprd03.prod.outlook.com
 ([fe80::24db:d530:e190:518b]) by PAXPR03MB7594.eurprd03.prod.outlook.com
 ([fe80::24db:d530:e190:518b%5]) with mapi id 15.20.4544.021; Mon, 27 Sep 2021
 23:12:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77e233ea-1fe8-11ec-bc60-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=azWj0i0FTrFaL+NXMxE23idqUY66tExIdoyF0LVJRnukU2K7w78wYlfJ+iUGD0md7/eGs92Ik8Tn9iRtdcZOMwDJrde69BolSCAbB/ndodfcAf+6UGSgDvil3rkWGVgTY1Q98BAyORWfktvWQP4CATZwZpFKbRbF4EbZc1OTOZIS46StQBjhZ2DtGKwJ3gF4fCM1CrdGzuRaPCP6ycE86ozNE7LuGRgCDtibskHPKCBfiUc8eEFypSiYWyfq0U1nemmGs3+V5bdjA4c3XSFWjhERy3MvMjAPAeHXKxWyW/KcCFdL7Lf2MVCrkgLCrbqK9HM6phSLGbTijx8Lzf7DqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=holjH+HfB8CXCHfIgJ8fG7J1vTR3WfyEe4KZjexySjc=;
 b=hYu9nUurlw/HXaeDL/RyrYAgXNL36aFabdcYanwCiIFB6oCzNH8T2cThPsTIFlqxAgHnjs3Mkmpc1RFe9XKx4fne9pHJhjH1bCC9T+DFl9XOYKfEjydukRzKL6QkVhlUr89bNP8DFGwjEdkIL5eeMDGvnIcvmpmGba+WnkD22a/HCH8CyKmC9q3e9ExJKALp13WfwM68HAVVWj+C2o8i2JmtD4O5W78bnSD32J0Jc04vItxetTC9yzKtKflKEbUgB5i5HzxTJhcfPDyoT3yGgvP+8n73E5988T7729YEaC8497rdYID2OSkc/Oq0hv5Fy2HR8uIg0mI9zFlpS9Bdrg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=holjH+HfB8CXCHfIgJ8fG7J1vTR3WfyEe4KZjexySjc=;
 b=FKc/DaLkfRcK1sjmSY8CbSm04wf6vuZp7Uwr+wPh6N7zBJDtY0hwH3fw0o7TIuzwgRvhdcx1/sZtkAXDct3BlwAnTgPLt85QNx56MIrOhBh1rHKib0UgLokjWmCLq6k5CMMrOsPXwpeU9iFXCVvE92jKt9fxG66Js1uqGMt10e0Iq5En7qAKvUxgVPrfUhpi4ITRGZI2JnDdVavuKJfFTKC1EBik94pbThkR7ibnmWK+CrhM4rODoOBIVyNsOqqdEA+6/O2vJHeT2tiLeAGHcKTjdjdPxxaI5QFzBpiB7PpWbRemO5fnHacwMpWAG60EukCD/5KDQdczCHQNJJhpTw==
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        "tee-dev@lists.linaro.org" <tee-dev@lists.linaro.org>
Subject: Re: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for
 OPTEE_SMC_DISABLE_SHM_CACHE
Thread-Topic: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for
 OPTEE_SMC_DISABLE_SHM_CACHE
Thread-Index: AQHXs6cvn7zQUNndCEOzhQWRkGM4Xqu4gvCA
Date: Mon, 27 Sep 2021 23:12:59 +0000
Message-ID: <874ka5sjd1.fsf@epam.com>
References: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
In-Reply-To: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
user-agent: mu4e 1.6.5; emacs 27.2
authentication-results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b000a409-c351-42a9-2f36-08d9820c5931
x-ms-traffictypediagnostic: PAXPR03MB8013:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <PAXPR03MB80135B378CD2F5B27BCCC72BE6A79@PAXPR03MB8013.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ga6NMVOXyBBKKBXDlpObTEK25VNXU08BLXUyWxdHyh7p7+U0KRYPOg4Z5WLaKgJJiIfr1g3fbMry8Ml8I04EQ16p+ihc+g+rn6HhXiO4pw8zGFaC12CloAbvix9HEltw0S3GziwQAGCf/oayCMN3B2lZ6rs/K2E1o6PcLX7aic9oRoXPap0LwFeDKPkTumWiVGu3OEvUg8Sc/BvdIyPc9ioTHHW+bFRUp4+4Pw8n9kk4/qOqEXDCkWt91Dp95YnBi48e3Q/dD7uqRt7afDbyWeUmjvZMKTKtDrlfHYl/Lm65x+bSbBh6sUlkwz2ACQgxJUhtDvStwP0TWSKjIsLkoUbTKKgwEJsJWSj2vLVzMFh4vWoKibky4onPxh9L0AiAx0P+w5g6YNAHDq+1TEc70f76yc9H5B58P8r1x7CCmWILfQ4+zhvEkbAmTHlhFOTQuEAt0f5wS9CBVbNL90wmEaVzadGHQh+UxzHG/m1FwVJ5BjX6E6B59J4w23P2go7qppcrImWkeVef1GqDV40bGeB+IgUwCB+ktxrmyX+5Vm1aiz5PBNMfMZgDX+2VUMA0boxHIT0CZ33K4DShBz1QN3JMqcumRzlw4LH1LU1jy9mAMIqRqpLcXRU9DhmoASo4KeT/TmvGJgeTaPOP07WulJSEfWJwFX27mMlmio1ZxjhvHKHIlMzfqneHTquH+KlJRFchu5pW10TpmnD7qfhomw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR03MB7594.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2616005)(6512007)(6486002)(508600001)(8676002)(86362001)(36756003)(6506007)(316002)(2906002)(38070700005)(5660300002)(6916009)(76116006)(66946007)(4326008)(54906003)(55236004)(66556008)(38100700002)(64756008)(8936002)(122000001)(71200400001)(66446008)(26005)(66476007)(186003)(83380400001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?iso-8859-1?Q?YTOW39LzUDr2X+59xfNuCCfIm1SCumiSsXb+JX4nZOecvGpeVR1xQiGkNH?=
 =?iso-8859-1?Q?IA1f3rZ1UwYuqLIjGPVHtAXlqEO1hVQpdvLH/qIbjuuwbzLSWLgevQgy8Z?=
 =?iso-8859-1?Q?npReXwOENqgZfM0fKtrhr0S+MJ8o/SYGGdGNhtDbXa/4T90vn09Qj4u4JI?=
 =?iso-8859-1?Q?KyooUPUPyAP++ccPEa+9H3gOxxlezAjpNXc88qesaswFyTJhe1v5TPNmno?=
 =?iso-8859-1?Q?kp+55UKwsnTI4/SeB5ao1AS1w+/tH8wJD1y0Z9z1mZktZjbp/2U7xvcyLq?=
 =?iso-8859-1?Q?VYl+6zwU9tbtrhHhMAp4hewenZUlitwR0/Yg8uEFWujnb/LR1fj+zQAfk6?=
 =?iso-8859-1?Q?UyvpGztTWtDwh8lFFeNy5oinCG7a0HDcYCRi00H4eAlaCkOZy1b61l7Fad?=
 =?iso-8859-1?Q?l+vqjg/DvTS8oWjGjZtDezjCs/Iue66NIUSSsD2XhzyIDIDKLDUI9j873V?=
 =?iso-8859-1?Q?W0ESd+PWRSHUzVNaCAY1a4SbMz8n81VtLPt9Gjd3K4oaDXHHnHjtItYXfz?=
 =?iso-8859-1?Q?MjcY901EvU41uPNq2l7NxNkaDLdYTRTlIm57xWvnWexRqtWhuUHIn/fD/N?=
 =?iso-8859-1?Q?gGgASJiP6PDTf9fuBWUMKMT+FJ2qR4yfJGDKDEKnhAX9e4GZ3XZus9hkiU?=
 =?iso-8859-1?Q?6rKcwPdWuV6i1OYjik7r+Nb6jroTgbQXwFNRLSlATe8sE+SAtaBJkeBHbm?=
 =?iso-8859-1?Q?AVszmtmPvuXyf2gl93KbpUpRwrliAn9NVyn7Zklh08OcanYqXg1VNyswSB?=
 =?iso-8859-1?Q?B8YwRuOocm1+hfNMyvFSefoNQwjx6zouQA7Ce6OUgYfevOpu2xMNQfj5Pd?=
 =?iso-8859-1?Q?CpnRiMB9PCccfkaZ7c8fnfLK7og10b1po/ILo3h16RQUIKCPbUplIBiFdz?=
 =?iso-8859-1?Q?5KPpgVRIuEz8pf2ZaKP7IDediVPTpRwqAlt6K9zPLAonRTkOahNqbr0isb?=
 =?iso-8859-1?Q?/2x5x9JZeKCICYmT3K0xQBVcILyOVXIb9M2ggqin0eVOz2ZvC3hXe0sZVE?=
 =?iso-8859-1?Q?xxhat3Eps1Dbg+GYgRF/qP71WND3Ajq4JBF0G7nbb5RChoUhMVom3y4eGV?=
 =?iso-8859-1?Q?M7njE2SBPMvQt/PBS7CvZl4+KYkuqDn9veZFrKScoCzfV214pOaeyQAtDQ?=
 =?iso-8859-1?Q?V1N96AeehCN9NfooHqlLKADSLMtxmjlvcUUVfPe76pEdz/fFQXObbVPdB2?=
 =?iso-8859-1?Q?giX8SOjgw4FF1N3YvWwcMPi/RjGdlBueDhg9AijOr2NuAzPIRUvH0Acvpz?=
 =?iso-8859-1?Q?xnN5+AcLlgrXYXU/xymm/Yh4voDcQTzPBnv9DSVYipJ1L4EIMkNUVME9Xz?=
 =?iso-8859-1?Q?tuZr8y3aO0GFgcsHUApbfv+ogd9u3VBqBTz3Z4TLGAb8HZ86sOVkY9r390?=
 =?iso-8859-1?Q?Vrp7DYJ8xv?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PAXPR03MB7594.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b000a409-c351-42a9-2f36-08d9820c5931
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2021 23:12:59.6453
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: w2W2vNbPZP89od0oz3Gf/wM0shEmL5Ovv1ROSyWMftZtoi8yjSxcx9leEKy33o40SM8UJeDa+gOo4H22V6pDKqMMqZnP7EiVXiEaXmjoYGU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB8013
X-Proofpoint-GUID: 5tzVb_ET1UP5xDQKnpjWUgr3bVJneTWK
X-Proofpoint-ORIG-GUID: 5tzVb_ET1UP5xDQKnpjWUgr3bVJneTWK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-27_07,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 priorityscore=1501 clxscore=1011 spamscore=0 mlxscore=0 lowpriorityscore=0
 phishscore=0 bulkscore=0 mlxlogscore=940 suspectscore=0 impostorscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109270157


Hi Oleksandr,

Oleksandr Tyshchenko <olekstysh@gmail.com> writes:

> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> Fix a possible copy-paste error in arm_smccc_smc's first argument (a0)
> for OPTEE_SMC_DISABLE_SHM_CACHE case.
>
> This error causes Linux > v5.14-rc5 (b5c10dd04b7418793517e3286cde5c04759a=
86de
> optee: Clear stale cache entries during initialization) to stuck
> repeatedly issuing OPTEE_SMC_DISABLE_SHM_CACHE call and waiting for
> the result to be OPTEE_SMC_RETURN_ENOTAVAIL which will never happen.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

> ---
> I wonder whether this patch wants backporting to the old versions
> since OPTEE support went in.
> ---
>  xen/arch/arm/tee/optee.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..6df0d44 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -1692,7 +1692,7 @@ static bool optee_handle_call(struct cpu_user_regs =
*regs)
>          return true;
> =20
>      case OPTEE_SMC_DISABLE_SHM_CACHE:
> -        arm_smccc_smc(OPTEE_SMC_ENABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
> +        arm_smccc_smc(OPTEE_SMC_DISABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
>                        OPTEE_CLIENT_ID(current->domain), &resp);
>          set_user_reg(regs, 0, resp.a0);
>          if ( resp.a0 =3D=3D OPTEE_SMC_RETURN_OK ) {


--=20
Volodymyr Babchuk at EPAM=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 01:00:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 01:00:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197324.350346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV1TN-0003Jv-Ja; Tue, 28 Sep 2021 00:59:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197324.350346; Tue, 28 Sep 2021 00:59:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV1TN-0003Jo-GZ; Tue, 28 Sep 2021 00:59:53 +0000
Received: by outflank-mailman (input) for mailman id 197324;
 Tue, 28 Sep 2021 00:59:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV1TL-0003Ji-GU
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 00:59:51 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cf0f8921-a776-409c-bf65-30d0aa8943a9;
 Tue, 28 Sep 2021 00:59:50 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 5978060F11;
 Tue, 28 Sep 2021 00:59:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf0f8921-a776-409c-bf65-30d0aa8943a9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632790789;
	bh=o/9PrZaRyZAV7aKtqEdAGwyUZ3FkDjT+co7mytr5ZtI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=P2ofTRpjyJfmXzVs6JvQ3oDOHzA4eDcEzsJxfxU3HjU14HdsuGoPvmzqw8/FLMyiT
	 c2PTNoz7LMdoP0IFi+CkB1sMWX8LdKzKneOEWuqZckWOVVm895Z6bXMJPQ9yt9zYcD
	 GXdz3voj1pgoOz3dSLEhDAYUFaVNnzkEQB+c/ThvyTKJ4to7ddAReMb8/pvoUO6gfz
	 NrfNvYXNQHLdTF6PSlmHlehfk6EoVFdvH+FZjgkIOfd/Lrs1AZ6sxwmkM3sFllNRft
	 kdVO+qGtgqU5i79NuRIp+uFVLDDEu4NKjvHluzVtO3t1mTDT51m4AgO+7iCnCK6kdS
	 egPD8GIwRd/5A==
Date: Mon, 27 Sep 2021 17:59:48 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Jan Beulich <jbeulich@suse.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
In-Reply-To: <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109271420560.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-21-wei.chen@arm.com> <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com> <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com> <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com> <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1428592914-1632777812=:5022"
Content-ID: <alpine.DEB.2.21.2109271424150.5022@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1428592914-1632777812=:5022
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109271424151.5022@sstabellini-ThinkPad-T480s>

On Mon, 27 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Wei
> > Chen
> > Sent: 2021年9月26日 18:25
> > To: Jan Beulich <jbeulich@suse.com>
> > Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>; Stefano Stabellini <sstabellini@kernel.org>
> > Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-
> > EFI architecture
> > 
> > Hi Jan,
> > 
> > > -----Original Message-----
> > > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > Jan
> > > Beulich
> > > Sent: 2021年9月24日 18:49
> > > To: Wei Chen <Wei.Chen@arm.com>
> > > Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > <Bertrand.Marquis@arm.com>; Stefano Stabellini <sstabellini@kernel.org>
> > > Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for
> > non-
> > > EFI architecture
> > >
> > > On 24.09.2021 12:31, Wei Chen wrote:
> > > >> From: Jan Beulich <jbeulich@suse.com>
> > > >> Sent: 2021年9月24日 15:59
> > > >>
> > > >> On 24.09.2021 06:34, Wei Chen wrote:
> > > >>>> From: Stefano Stabellini <sstabellini@kernel.org>
> > > >>>> Sent: 2021年9月24日 9:15
> > > >>>>
> > > >>>> On Thu, 23 Sep 2021, Wei Chen wrote:
> > > >>>>> --- a/xen/common/Kconfig
> > > >>>>> +++ b/xen/common/Kconfig
> > > >>>>> @@ -11,6 +11,16 @@ config COMPAT
> > > >>>>>  config CORE_PARKING
> > > >>>>>  	bool
> > > >>>>>
> > > >>>>> +config EFI
> > > >>>>> +	bool
> > > >>>>
> > > >>>> Without the title the option is not user-selectable (or de-
> > > selectable).
> > > >>>> So the help message below can never be seen.
> > > >>>>
> > > >>>> Either add a title, e.g.:
> > > >>>>
> > > >>>> bool "EFI support"
> > > >>>>
> > > >>>> Or fully make the option a silent option by removing the help text.
> > > >>>
> > > >>> OK, in current Xen code, EFI is unconditionally compiled. Before
> > > >>> we change related code, I prefer to remove the help text.
> > > >>
> > > >> But that's not true: At least on x86 EFI gets compiled depending on
> > > >> tool chain capabilities. Ultimately we may indeed want a user
> > > >> selectable option here, but until then I'm afraid having this option
> > > >> at all may be misleading on x86.
> > > >>
> > > >
> > > > I check the build scripts, yes, you're right. For x86, EFI is not a
> > > > selectable option in Kconfig. I agree with you, we can't use Kconfig
> > > > system to decide to enable EFI build for x86 or not.
> > > >
> > > > So how about we just use this EFI option for Arm only? Because on Arm,
> > > > we do not have such toolchain dependency.
> > >
> > > To be honest - don't know. That's because I don't know what you want
> > > to use the option for subsequently.
> > >
> > 
> > In last version, I had introduced an arch-helper to stub EFI_BOOT
> > in Arm's common code for Arm32. Because Arm32 doesn't support EFI.
> > So Julien suggested me to introduce a CONFIG_EFI option for non-EFI
> > supported architectures to stub in EFI layer.
> > 
> > [1] https://lists.xenproject.org/archives/html/xen-devel/2021-
> > 08/msg00808.html
> > 
> 
> As Jan' reminded, x86 doesn't depend on Kconfig to build EFI code.
> So, if we CONFIG_EFI to stub EFI API's for x86, we will encounter
> that toolchains enable EFI, but Kconfig disable EFI. Or Kconfig
> enable EFI but toolchain doesn't provide EFI build supports. And
> then x86 could not work well.
> 
> If we use CONFIG_EFI for Arm only, that means CONFIG_EFI for x86
> is off, this will also cause problem.
> 
> So, can we still use previous arch_helpers to stub for Arm32?
> until x86 can use this selectable option?

EFI doesn't have to be necessarily a user-visible option in Kconfig at
this point. I think Julien was just asking to make the #ifdef based on
a EFI-related config rather than just based CONFIG_ARM64.

On x86 EFI is detected based on compiler support, setting XEN_BUILD_EFI
in xen/arch/x86/Makefile. Let's say that we keep using the same name
"XEN_BUILD_EFI" on ARM as well.

On ARM32, XEN_BUILD_EFI should be always unset.

On ARM64 XEN_BUILD_EFI should be always set.

That's it, right? I'd argue that CONFIG_EFI or HAS_EFI are better names
than XEN_BUILD_EFI, but that's OK anyway. So for instance you can make
XEN_BUILD_EFI an invisible symbol in xen/arch/arm/Kconfig and select it
only on ARM64.
--8323329-1428592914-1632777812=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 01:51:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 01:51:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197337.350357 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV2HT-0007tD-B5; Tue, 28 Sep 2021 01:51:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197337.350357; Tue, 28 Sep 2021 01:51:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV2HT-0007t5-4t; Tue, 28 Sep 2021 01:51:39 +0000
Received: by outflank-mailman (input) for mailman id 197337;
 Tue, 28 Sep 2021 01:51:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV2HR-0007sv-Q4; Tue, 28 Sep 2021 01:51:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV2HR-0003Mp-CI; Tue, 28 Sep 2021 01:51:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV2HQ-0001Ci-TE; Tue, 28 Sep 2021 01:51:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mV2HQ-0004ag-Ro; Tue, 28 Sep 2021 01:51:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OAIMUQWFZSn/9QPqhWO3rAL/QDL0AHZ8HW591hHMdLA=; b=m2S+SKa3IKZfxaXFSM0YIslOT4
	LdfBcc3EQIIc5FT8WZpBBvzAw4ymLz6SrcaznJQlHYD70LzwBBTR+BtXZygNuB4DmuMHOebHIWUM6
	nA0kWRPwyPTUJDhit1/BCcaJ5NH9+rww0O7ipRVY4nxzxBXqvYZ8XR90oxS+Mk59n5SI=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165218-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165218: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-xl-credit1:xen-boot:fail:heisenbug
    xen-unstable:test-arm64-arm64-libvirt-xsm:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=dd6c062a7a4abdb662c18af03d1396325969d155
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 01:51:36 +0000

flight 165218 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165218/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit1   8 xen-boot         fail in 165211 pass in 165218
 test-arm64-arm64-libvirt-xsm 18 guest-start/debian.repeat  fail pass in 165211

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  dd6c062a7a4abdb662c18af03d1396325969d155
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   17 days
Failing since        164951  2021-09-12 00:14:36 Z   16 days   32 attempts
Testing same since   165188  2021-09-25 01:37:10 Z    3 days    6 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 966 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 02:57:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 02:57:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197353.350371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3JF-0006ga-BJ; Tue, 28 Sep 2021 02:57:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197353.350371; Tue, 28 Sep 2021 02:57:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3JF-0006gT-7z; Tue, 28 Sep 2021 02:57:33 +0000
Received: by outflank-mailman (input) for mailman id 197353;
 Tue, 28 Sep 2021 02:57:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0YX1=OS=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mV3JD-0006gN-Et
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 02:57:31 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.59]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a296a7f-be43-4b24-b09d-effc7c1115b1;
 Tue, 28 Sep 2021 02:57:26 +0000 (UTC)
Received: from AM6P192CA0001.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::14)
 by AM0PR08MB5331.eurprd08.prod.outlook.com (2603:10a6:208:187::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 02:57:18 +0000
Received: from AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:83:cafe::88) by AM6P192CA0001.outlook.office365.com
 (2603:10a6:209:83::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Tue, 28 Sep 2021 02:57:18 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT015.mail.protection.outlook.com (10.152.16.132) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 02:57:18 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Tue, 28 Sep 2021 02:57:17 +0000
Received: from 15cbb1cb533c.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5853ACA9-BB35-439A-9E4D-F57508CDD076.1; 
 Tue, 28 Sep 2021 02:57:07 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 15cbb1cb533c.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 02:57:07 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5559.eurprd08.prod.outlook.com (2603:10a6:10:1a2::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 02:57:06 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 02:57:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a296a7f-be43-4b24-b09d-effc7c1115b1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=i0ZWj0So+f3AGvIMNKdZairzuaiPSyZDQgM8cCXrFUw=;
 b=Oo7tS2xoqT79t9J11jCUPUakEzP6M2Mxnvsz61gWzNgUeS2nGM5Hw4lfC0xpCezEHldCitCev3OU/UQVEV7aRLa/lRBYb1m8z96LeOAiJH76fT/LhxtROj5WGd5hvaBGTEyQ/1mI4zH0A4GekLp7TRG7Su4xmJPeAkSZvbo40ZA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TtD+507qJyVRS7WSWzAIqs8r1xKqDATuwS18J+fh3/JHG7ZYcUNVjbwP2mvwyBtpHf2gA+DodKNsxuV7om4uBB667Y5cTl7OHYTttDOyaJYWCK4t7kKg7kdq8+W1rPe/FbRMS/RiWNlHfKHY1adwgqvmn1FJPWVStyUImzAHxWfV/vDfZceqtitu8szzAUuSWCc3jWalvXmrsbt5R68Rhc5umMOG3VxCbD6G8fFSNC9mMSoeGXesYwLeYrcGANd6i52spYvKS0w+iIM1GEuh+VPS9aY47s7/JQSR5YRYEOaxaUt4CGabhB4qxQ0rKUVsOUu7g28SkGaGRk4jOsdtZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=i0ZWj0So+f3AGvIMNKdZairzuaiPSyZDQgM8cCXrFUw=;
 b=C1REErYB8+J+ABR0WsU5qnR/fDSAcGg+dJQZwPXiovLp+xlxzOpeoTaZG9tyU5oCy+n8B5MNmg51zO/uFu3zzYGTwXDmiaIBGhlKIuPhjn2cEKbuPhKxhEvCM/PN/OV5zIFp0YKIAsyd09c802Yki1e8cNB+lFca68QWfA4r9dGKirbQiOXFHipS4bPPTYY9JRrl4w54UOOdvNYsCNx/Fz8ibvD383KiGKW5voi9LbuWMVpH/ou7TcKE0O/J7QE2lznx/GxoZ+qHchYuQJsm3GPPft+67WrgTzcyxEr1mlCkA7srA8/SNUXWI9W0zXmQowau6vT9rpbpGzq90yGtdw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=i0ZWj0So+f3AGvIMNKdZairzuaiPSyZDQgM8cCXrFUw=;
 b=Oo7tS2xoqT79t9J11jCUPUakEzP6M2Mxnvsz61gWzNgUeS2nGM5Hw4lfC0xpCezEHldCitCev3OU/UQVEV7aRLa/lRBYb1m8z96LeOAiJH76fT/LhxtROj5WGd5hvaBGTEyQ/1mI4zH0A4GekLp7TRG7Su4xmJPeAkSZvbo40ZA=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien.grall@gmail.com>
CC: Julien Grall <julien.grall.oss@gmail.com>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Jan Beulich <jbeulich@suse.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>
Subject: RE: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Topic: [PATCH 22/37] xen/arm: use NR_MEM_BANKS to override default
 NR_NODE_MEMBLKS
Thread-Index:
 AQHXsHMrRDdmLUiNMk+huFUWk4EjYquyZ70AgAPY1CCAAP08AIAADO8QgAANUgCAABvh0IAAAsXQgAANBICAACxScIAABwwAgABpsYCAAKY5gA==
Date: Tue, 28 Sep 2021 02:57:05 +0000
Message-ID:
 <DB9PR08MB68574D931CF0ED15635AD4F69EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-23-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231828310.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685744A06D7C014DAE9BE73F9EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262021200.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857AD295D692F962AD76C219EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262123140.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857603316C2C808BAD8CD709EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB685744B09307DFCA38C0635C9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CAJ=z9a1D2ROHRyvRM7=kgnU_J2RkuHC_htYEWtqznUxtq=vZHA@mail.gmail.com>
 <DB9PR08MB6857316DC540769819C8155E9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <CAF3u54BNbPPo4_ezC2ft36NwodKDFSdtv_ekT-FFyvRrU3gQaA@mail.gmail.com>
 <alpine.DEB.2.21.2109270956270.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109270956270.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: FFED7C67FECC3349B07D3BDD77F538ED.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 72d2469e-1c70-4dbf-64c8-08d9822baf05
x-ms-traffictypediagnostic: DBAPR08MB5559:|AM0PR08MB5331:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB53314DE11DAB1E1587519F349EA89@AM0PR08MB5331.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 4Idu1O9ap0Q3G8bE40cLRUJ1BylJbXvgpck2+U6DunyW7u3WIUasntZZzUPFp343Zg6IuooMVM3FnFATcsX51iKNIUO8jzYMq2VTiove+5Gr79wz71mBSywWgJl0zqdsMiJgMufD1ra34VTxsMmQ/0k7G8fUWuRcdgptKI4VHKNCYRsdNLEjVlrnRhPbwNJW7fuBVJRf5PwcL7SGzyXxwgKw8jtAqfnC6cniJAoTSWTQCatBN0ngbJS9p4dxOfr7xlaSa4Kaw8D6zK1PfbRd7CG0xFU1fPfNiJg/RDE2l1XCRBvqak3a66xwLSHulvnTm5h01u6XGbDw1oidsXZbZIvLGeSHl0HoH+mABooGyixFlLi55ODIdsuaKyXYzF4XEcnyDvcjpDDWOFyjZz4cKMP9PChBuRFbmJGYrsU3vol3fFkomB8dHsGFWbwb1uU4pF2+JImMkgOuUslcd7UUrMmVDNhx6KIMl9R1ecacIC93F01i/cN2Xgw1EYawziknfdNmuJg7a2KTlkeBFcatEakMJdzMZCkn/vFsTNHhNHs8WZWJ+c+MGP9lmSP2p0CQZ4kFixzFiSbfREfiHE6IyS+5BGG8Uf5Jr2Bbp5zpNax4nGtq0jzL59v2tofif9uRH7WVsR1osKGT/l4rn6YQIzeuxgpwolfpD4f2qoeQqfDxaK1msERlP7Ua95rs5UY33cFr26U+p1m39GSJ7Q+CHHCGo8t/1tcmwk8sF1l49Hdu/JLzPVnkzBvMYbwuNrw1MfOdb8EGiwM+wVB7QPQjSBt9VOp6PZe6tJ5A9bfSgMo=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52536014)(26005)(9686003)(186003)(55016002)(8676002)(8936002)(54906003)(86362001)(38070700005)(33656002)(122000001)(83380400001)(38100700002)(316002)(53546011)(66476007)(508600001)(66446008)(66946007)(66556008)(76116006)(7696005)(966005)(110136005)(30864003)(6506007)(64756008)(71200400001)(2906002)(4326008)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5559
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	eb3ca722-37a5-4cb5-accd-08d9822ba7ba
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WK13ThznxNDONy71SdaVR0OlHQe30Hym81YssMmill/YIP2kFg35yR0h652sTNgddaKcQ3EChr943OSA5TL6BJH3A+sFBZeAwE0+2xwDF42PSQb4kDEzL+V0IPgRkJcPx5QNhM8lvcjAONX6i8buRDFVFeR0x08yKiWKH21Bgkxzpo4A/Xh+MP7Yhiypld0SRzN5CNq/hZkQvISwGKtdOJ1HDkJ6YuLa+vC/1axQymSs3dai0ac1F8NkEMhtiy7mtFGqHG9bzzt1cQcoMWxfXQWMAD1sq4gEuBb2/3jpOYbJt0BS+nU00e30+AWjIx2DuW3SfEMMaq0h61lN8ELxG+N20ChEcuRCyZNoAZERsJ+LiCQpKdeSWBtY5DD+6OoYn7DTKx50HaFz+i2rl+9K3tOfuNPG/tFA43BL2OdWyVzRO7h19T7Yb7B6wpONWKhTmkILTjitnUwaH0AbB+Z73SCeDtqfP5i3LLHS+oNdcqMoII55ai5Z+dV4hVRFFhgUnkuFLLwVzJ8FXqwg9L01j1ejV1oM4JmE3cvNCjw42HH7EuWYj1KYSdbppCgUcpDVcv5HN204B1LvW671KeBek7uQ1AD6SZXrYW9Uq+vG4hZC0uQ9DXmApGSJdbrRXmTse2RpdqgTdTcSIV1B8Z2c37yhiGHH1hiSISlrQn7YYkPd6sWnupCWQS585g3DeZ+trMv43y+1n3FqChLi55vkG95a3LYaZ/k8l3ULOyRbD9l0UO+5cL8CP0LY+4edUA+sBfjJqWLy4U21rH59yZHTYe22HsAKSOC1cskbAxPvwco=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(5660300002)(2906002)(8936002)(336012)(33656002)(186003)(30864003)(4326008)(26005)(82310400003)(86362001)(8676002)(47076005)(83380400001)(52536014)(36860700001)(316002)(6506007)(53546011)(356005)(7696005)(107886003)(9686003)(55016002)(508600001)(54906003)(81166007)(70586007)(110136005)(70206006)(966005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 02:57:18.0647
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 72d2469e-1c70-4dbf-64c8-08d9822baf05
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5331

SGkgU3RlZmFubywgSnVsaWVuLA0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy
b206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gU2VudDog
MjAyMeW5tDnmnIgyOOaXpSAwOjU4DQo+IFRvOiBKdWxpZW4gR3JhbGwgPGp1bGllbi5ncmFsbEBn
bWFpbC5jb20+DQo+IENjOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT47IEp1bGllbiBHcmFs
bCA8anVsaWVuLmdyYWxsLm9zc0BnbWFpbC5jb20+Ow0KPiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNz
dGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tZGV2ZWwgPHhlbi0NCj4gZGV2ZWxAbGlzdHMueGVu
cHJvamVjdC5vcmc+OyBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+
Ow0KPiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+OyBSb2dlciBQYXUgTW9ubsOpIDxy
b2dlci5wYXVAY2l0cml4LmNvbT47DQo+IEFuZHJldyBDb29wZXIgPGFuZHJldy5jb29wZXIzQGNp
dHJpeC5jb20+DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9N
RU1fQkFOS1MgdG8gb3ZlcnJpZGUgZGVmYXVsdA0KPiBOUl9OT0RFX01FTUJMS1MNCj4gDQo+IE9u
IE1vbiwgMjcgU2VwIDIwMjEsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gPiBPbiBNb24sIDI3IFNl
cCAyMDIxLCAxMjoyMiBXZWkgQ2hlbiwgPFdlaS5DaGVuQGFybS5jb20+IHdyb3RlOg0KPiA+ICAg
ICAgIEhpIEp1bGllbiwNCj4gPg0KPiA+ICAgICAgIEZyb206IEp1bGllbiBHcmFsbCA8anVsaWVu
LmdyYWxsLm9zc0BnbWFpbC5jb20+DQo+ID4gICAgICAgU2VudDogMjAyMeW5tDnmnIgyN+aXpSAx
NTozNg0KPiA+ICAgICAgIFRvOiBXZWkgQ2hlbiA8V2VpLkNoZW5AYXJtLmNvbT4NCj4gPiAgICAg
ICBDYzogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPjsgeGVuLWRl
dmVsIDx4ZW4tDQo+IGRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnPjsgQmVydHJhbmQgTWFycXVp
cw0KPiA+ICAgICAgIDxCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+OyBKYW4gQmV1bGljaCA8amJl
dWxpY2hAc3VzZS5jb20+OyBSb2dlcg0KPiBQYXUgTW9ubsOpIDxyb2dlci5wYXVAY2l0cml4LmNv
bT47IEFuZHJldyBDb29wZXINCj4gPiAgICAgICA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4N
Cj4gPiAgICAgICBTdWJqZWN0OiBSZTogW1BBVENIIDIyLzM3XSB4ZW4vYXJtOiB1c2UgTlJfTUVN
X0JBTktTIHRvIG92ZXJyaWRlDQo+IGRlZmF1bHQgTlJfTk9ERV9NRU1CTEtTDQo+ID4NCj4gPg0K
PiA+ICAgICAgIE9uIE1vbiwgMjcgU2VwIDIwMjEsIDA4OjUzIFdlaSBDaGVuLCA8bWFpbHRvOldl
aS5DaGVuQGFybS5jb20+DQo+IHdyb3RlOg0KPiA+ICAgICAgIEhpIEp1bGllbiwNCj4gPg0KPiA+
ICAgICAgID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiAgICAgICA+IEZyb206IFhl
bi1kZXZlbCA8bWFpbHRvOnhlbi1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPg0K
PiBPbiBCZWhhbGYgT2YgV2VpDQo+ID4gICAgICAgPiBDaGVuDQo+ID4gICAgICAgPiBTZW50OiAy
MDIx5bm0OeaciDI35pelIDE0OjQ2DQo+ID4gICAgICAgPiBUbzogU3RlZmFubyBTdGFiZWxsaW5p
IDxtYWlsdG86c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiAgICAgICA+IENjOiBtYWlsdG86
eGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOyBtYWlsdG86anVsaWVuQHhlbi5vcmc7DQo+
IEJlcnRyYW5kIE1hcnF1aXMNCj4gPiAgICAgICA+IDxtYWlsdG86QmVydHJhbmQuTWFycXVpc0Bh
cm0uY29tPjsgbWFpbHRvOmpiZXVsaWNoQHN1c2UuY29tOw0KPiBtYWlsdG86cm9nZXIucGF1QGNp
dHJpeC5jb207DQo+ID4gICAgICAgPiBtYWlsdG86YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbQ0K
PiA+ICAgICAgID4gU3ViamVjdDogUkU6IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01F
TV9CQU5LUyB0byBvdmVycmlkZQ0KPiBkZWZhdWx0DQo+ID4gICAgICAgPiBOUl9OT0RFX01FTUJM
S1MNCj4gPiAgICAgICA+DQo+ID4gICAgICAgPiBIaSBTdGVmYW5vLCBKdWxpZW4sDQo+ID4gICAg
ICAgPg0KPiA+ICAgICAgID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ICAgICAg
ID4gPiBGcm9tOiBTdGVmYW5vIFN0YWJlbGxpbmkgPG1haWx0bzpzc3RhYmVsbGluaUBrZXJuZWwu
b3JnPg0KPiA+ICAgICAgID4gPiBTZW50OiAyMDIx5bm0OeaciDI35pelIDEzOjAwDQo+ID4gICAg
ICAgPiA+IFRvOiBXZWkgQ2hlbiA8bWFpbHRvOldlaS5DaGVuQGFybS5jb20+DQo+ID4gICAgICAg
PiA+IENjOiBTdGVmYW5vIFN0YWJlbGxpbmkgPG1haWx0bzpzc3RhYmVsbGluaUBrZXJuZWwub3Jn
PjsgeGVuLQ0KPiA+ICAgICAgID4gPiBtYWlsdG86ZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7
IG1haWx0bzpqdWxpZW5AeGVuLm9yZzsNCj4gQmVydHJhbmQgTWFycXVpcw0KPiA+ICAgICAgID4g
PiA8bWFpbHRvOkJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IG1haWx0bzpqYmV1bGljaEBzdXNl
LmNvbTsNCj4gbWFpbHRvOnJvZ2VyLnBhdUBjaXRyaXguY29tOw0KPiA+ICAgICAgID4gPiBtYWls
dG86YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbQ0KPiA+ICAgICAgID4gPiBTdWJqZWN0OiBSRTog
W1BBVENIIDIyLzM3XSB4ZW4vYXJtOiB1c2UgTlJfTUVNX0JBTktTIHRvDQo+IG92ZXJyaWRlIGRl
ZmF1bHQNCj4gPiAgICAgICA+ID4gTlJfTk9ERV9NRU1CTEtTDQo+ID4gICAgICAgPiA+DQo+ID4g
ICAgICAgPiA+ICt4ODYgbWFpbnRhaW5lcnMNCj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+ID4g
T24gTW9uLCAyNyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gICAgICAgPiA+ID4gPiAt
LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ICAgICAgID4gPiA+ID4gRnJvbTogU3RlZmFu
byBTdGFiZWxsaW5pIDxtYWlsdG86c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4NCj4gPiAgICAgICA+
ID4gPiA+IFNlbnQ6IDIwMjHlubQ55pyIMjfml6UgMTE6MjYNCj4gPiAgICAgICA+ID4gPiA+IFRv
OiBXZWkgQ2hlbiA8bWFpbHRvOldlaS5DaGVuQGFybS5jb20+DQo+ID4gICAgICAgPiA+ID4gPiBD
YzogU3RlZmFubyBTdGFiZWxsaW5pIDxtYWlsdG86c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47DQo+
IHhlbi0NCj4gPiAgICAgICA+ID4gPiA+IG1haWx0bzpkZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9y
ZzsgbWFpbHRvOmp1bGllbkB4ZW4ub3JnOw0KPiBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4gICAgICAg
PiA+ID4gPiA8bWFpbHRvOkJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT4NCj4gPiAgICAgICA+ID4g
PiA+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMjIvMzddIHhlbi9hcm06IHVzZSBOUl9NRU1fQkFOS1Mg
dG8NCj4gb3ZlcnJpZGUNCj4gPiAgICAgICA+ID4gZGVmYXVsdA0KPiA+ICAgICAgID4gPiA+ID4g
TlJfTk9ERV9NRU1CTEtTDQo+ID4gICAgICAgPiA+ID4gPg0KPiA+ICAgICAgID4gPiA+ID4gT24g
U3VuLCAyNiBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gICAgICAgPiA+ID4gPiA+ID4g
LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiAgICAgICA+ID4gPiA+ID4gPiBGcm9tOiBT
dGVmYW5vIFN0YWJlbGxpbmkNCj4gPG1haWx0bzpzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+
ICAgICAgID4gPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ55pyIMjTml6UgOTozNQ0KPiA+ICAgICAg
ID4gPiA+ID4gPiA+IFRvOiBXZWkgQ2hlbiA8bWFpbHRvOldlaS5DaGVuQGFybS5jb20+DQo+ID4g
ICAgICAgPiA+ID4gPiA+ID4gQ2M6IG1haWx0bzp4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5v
cmc7DQo+IG1haWx0bzpzc3RhYmVsbGluaUBrZXJuZWwub3JnOw0KPiA+ICAgICAgID4gPiA+ID4g
bWFpbHRvOmp1bGllbkB4ZW4ub3JnOw0KPiA+ICAgICAgID4gPiA+ID4gPiA+IEJlcnRyYW5kIE1h
cnF1aXMgPG1haWx0bzpCZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+DQo+ID4gICAgICAgPiA+ID4g
PiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCAyMi8zN10geGVuL2FybTogdXNlIE5SX01FTV9CQU5L
Uw0KPiB0byBvdmVycmlkZQ0KPiA+ICAgICAgID4gPiA+ID4gZGVmYXVsdA0KPiA+ICAgICAgID4g
PiA+ID4gPiA+IE5SX05PREVfTUVNQkxLUw0KPiA+ICAgICAgID4gPiA+ID4gPiA+DQo+ID4gICAg
ICAgPiA+ID4gPiA+ID4gT24gVGh1LCAyMyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4g
ICAgICAgPiA+ID4gPiA+ID4gPiBBcyBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQgaW4gZGV2aWNl
IHRyZWUgY2Fubm90IGJlDQo+IHNwbGl0DQo+ID4gICAgICAgPiA+IGFjcm9zcw0KPiA+ICAgICAg
ID4gPiA+ID4gPiA+ID4gbXVsdGlwbGUgbm9kZXMuIFNvIHdlIGRlZmluZSBOUl9OT0RFX01FTUJM
S1MgYXMNCj4gTlJfTUVNX0JBTktTDQo+ID4gICAgICAgPiBpbg0KPiA+ICAgICAgID4gPiA+ID4g
PiA+ID4gYXJjaCBoZWFkZXIuDQo+ID4gICAgICAgPiA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4g
PiA+ID4gPiBUaGlzIHN0YXRlbWVudCBpcyB0cnVlIGJ1dCB3aGF0IGlzIHRoZSBnb2FsIG9mIHRo
aXMNCj4gcGF0Y2g/IElzIGl0DQo+ID4gICAgICAgPiA+IHRvDQo+ID4gICAgICAgPiA+ID4gPiA+
ID4gcmVkdWNlIGNvZGUgc2l6ZSBhbmQgbWVtb3J5IGNvbnN1bXB0aW9uPw0KPiA+ICAgICAgID4g
PiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+IE5vLCB3
aGVuIEp1bGllbiBhbmQgSSBkaXNjdXNzZWQgdGhpcyBpbiBsYXN0IHZlcnNpb25bMV0sDQo+IHdl
IGhhZG4ndA0KPiA+ICAgICAgID4gPiA+ID4gdGhvdWdodA0KPiA+ICAgICAgID4gPiA+ID4gPiBz
byBkZWVwbHkuIFdlIGp1c3QgdGhvdWdodCBhIG1lbW9yeSByYW5nZSBkZXNjcmliZWQgaW4NCj4g
RFQgY2Fubm90DQo+ID4gICAgICAgPiBiZQ0KPiA+ICAgICAgID4gPiA+ID4gc3BsaXQNCj4gPiAg
ICAgICA+ID4gPiA+ID4gYWNyb3NzIG11bHRpcGxlIG5vZGVzLiBTbyBOUl9NRU1fQkFOS1Mgc2hv
dWxkIGJlIGVxdWFsDQo+IHRvDQo+ID4gICAgICAgPiA+IE5SX01FTV9CQU5LUy4NCj4gPiAgICAg
ICA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4gPiA+ID4gaHR0cHM6Ly9saXN0cy54ZW5wcm9qZWN0
Lm9yZy9hcmNoaXZlcy9odG1sL3hlbi0NCj4gZGV2ZWwvMjAyMS0NCj4gPiAgICAgICA+ID4gPiA+
IDA4L21zZzAwOTc0Lmh0bWwNCj4gPiAgICAgICA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4gPiA+
ID4gPiBJIGFtIGFza2luZyBiZWNhdXNlIE5SX01FTV9CQU5LUyBpcyAxMjggYW5kDQo+ID4gICAg
ICAgPiA+ID4gPiA+ID4gTlJfTk9ERV9NRU1CTEtTPTIqTUFYX05VTU5PREVTIHdoaWNoIGlzIDY0
IGJ5IGRlZmF1bHQNCj4gc28gYWdhaW4NCj4gPiAgICAgICA+ID4gPiA+ID4gPiBOUl9OT0RFX01F
TUJMS1MgaXMgMTI4IGJlZm9yZSB0aGlzIHBhdGNoLg0KPiA+ICAgICAgID4gPiA+ID4gPiA+DQo+
ID4gICAgICAgPiA+ID4gPiA+ID4gSW4gb3RoZXIgd29yZHMsIHRoaXMgcGF0Y2ggYWxvbmUgZG9l
c24ndCBtYWtlIGFueQ0KPiBkaWZmZXJlbmNlOyBhdA0KPiA+ICAgICAgID4gPiBsZWFzdA0KPiA+
ICAgICAgID4gPiA+ID4gPiA+IGRvZXNuJ3QgbWFrZSBhbnkgZGlmZmVyZW5jZSB1bmxlc3MNCj4g
Q09ORklHX05SX05VTUFfTk9ERVMgaXMNCj4gPiAgICAgICA+ID4gaW5jcmVhc2VkLg0KPiA+ICAg
ICAgID4gPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+ID4gU28sIGlzIHRoZSBnb2FsIHRv
IHJlZHVjZSBtZW1vcnkgdXNhZ2Ugd2hlbg0KPiBDT05GSUdfTlJfTlVNQV9OT0RFUw0KPiA+ICAg
ICAgID4gPiBpcw0KPiA+ICAgICAgID4gPiA+ID4gPiA+IGhpZ2hlciB0aGFuIDY0Pw0KPiA+ICAg
ICAgID4gPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+
IEkgYWxzbyB0aG91Z2h0IGFib3V0IHRoaXMgcHJvYmxlbSB3aGVuIEkgd2FzIHdyaXRpbmcNCj4g
dGhpcyBwYXRjaC4NCj4gPiAgICAgICA+ID4gPiA+ID4gQ09ORklHX05SX05VTUFfTk9ERVMgaXMg
aW5jcmVhc2luZywgYnV0IE5SX01FTV9CQU5LUyBpcw0KPiBhIGZpeGVkDQo+ID4gICAgICAgPiA+
ID4gPiA+IHZhbHVlLCB0aGVuIE5SX01FTV9CQU5LUyBjYW4gYmUgc21hbGxlciB0aGFuDQo+IENP
TkZJR19OUl9OVU1BX05PREVTDQo+ID4gICAgICAgPiA+ID4gPiA+IGF0IG9uZSBwb2ludC4NCj4g
PiAgICAgICA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4gPiA+ID4gQnV0IEkgYWdyZWUgd2l0aCBK
dWxpZW4ncyBzdWdnZXN0aW9uLCBOUl9NRU1fQkFOS1MgYW5kDQo+ID4gICAgICAgPiA+IE5SX05P
REVfTUVNQkxLUw0KPiA+ICAgICAgID4gPiA+ID4gPiBtdXN0IGJlIGF3YXJlIG9mIGVhY2ggb3Ro
ZXIuIEkgaGFkIHRob3VnaHQgdG8gYWRkIHNvbWUNCj4gQVNTRVJUDQo+ID4gICAgICAgPiBjaGVj
aywNCj4gPiAgICAgICA+ID4gPiA+ID4gYnV0IEkgZG9uJ3Qga25vdyBob3cgdG8gZG8gaXQgYmV0
dGVyLiBTbyBJIHBvc3QgdGhpcw0KPiBwYXRjaCBmb3INCj4gPiAgICAgICA+IG1vcmUNCj4gPiAg
ICAgICA+ID4gPiA+ID4gc3VnZ2VzdGlvbi4NCj4gPiAgICAgICA+ID4gPiA+DQo+ID4gICAgICAg
PiA+ID4gPiBPSy4gSW4gdGhhdCBjYXNlIEknZCBzYXkgdG8gZ2V0IHJpZCBvZiB0aGUgcHJldmlv
dXMNCj4gZGVmaW5pdGlvbiBvZg0KPiA+ICAgICAgID4gPiA+ID4gTlJfTk9ERV9NRU1CTEtTIGFz
IGl0IGlzIHByb2JhYmx5IG5vdCBuZWNlc3NhcnksIHNlZSBiZWxvdy4NCj4gPiAgICAgICA+ID4g
PiA+DQo+ID4gICAgICAgPiA+ID4gPg0KPiA+ICAgICAgID4gPiA+ID4NCj4gPiAgICAgICA+ID4g
PiA+ID4gPg0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4gQW5kIGtlZXAgZGVmYXVsdCBOUl9OT0RF
X01FTUJMS1MgaW4gY29tbW9uIGhlYWRlcg0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4gZm9yIHRo
b3NlIGFyY2hpdGVjdHVyZXMgTlVNQSBpcyBkaXNhYmxlZC4NCj4gPiAgICAgICA+ID4gPiA+ID4g
Pg0KPiA+ICAgICAgID4gPiA+ID4gPiA+IFRoaXMgbGFzdCBzZW50ZW5jZSBpcyBub3QgYWNjdXJh
dGU6IG9uIHg4NiBOVU1BIGlzDQo+IGVuYWJsZWQgYW5kDQo+ID4gICAgICAgPiA+ID4gPiA+ID4g
TlJfTk9ERV9NRU1CTEtTIGlzIHN0aWxsIGRlZmluZWQgaW4NCj4geGVuL2luY2x1ZGUveGVuL251
bWEuaA0KPiA+ICAgICAgID4gKHRoZXJlDQo+ID4gICAgICAgPiA+IGlzDQo+ID4gICAgICAgPiA+
ID4gPiBubw0KPiA+ICAgICAgID4gPiA+ID4gPiA+IHg4NiBkZWZpbml0aW9uIG9mIGl0KQ0KPiA+
ICAgICAgID4gPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4g
PiA+IFllcy4NCj4gPiAgICAgICA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4gPiA+ID4gPg0KPiA+
ICAgICAgID4gPiA+ID4gPiA+ID4gU2lnbmVkLW9mZi1ieTogV2VpIENoZW4gPG1haWx0bzp3ZWku
Y2hlbkBhcm0uY29tPg0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4gLS0tDQo+ID4gICAgICAgPiA+
ID4gPiA+ID4gPsKgIHhlbi9pbmNsdWRlL2FzbS1hcm0vbnVtYS5oIHwgOCArKysrKysrLQ0KPiA+
ICAgICAgID4gPiA+ID4gPiA+ID7CoCB4ZW4vaW5jbHVkZS94ZW4vbnVtYS5owqAgwqAgwqB8IDIg
KysNCj4gPiAgICAgICA+ID4gPiA+ID4gPiA+wqAgMiBmaWxlcyBjaGFuZ2VkLCA5IGluc2VydGlv
bnMoKyksIDEgZGVsZXRpb24oLSkNCj4gPiAgICAgICA+ID4gPiA+ID4gPiA+DQo+ID4gICAgICAg
PiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4g
Yi94ZW4vaW5jbHVkZS9hc20tDQo+ID4gICAgICAgPiA+IGFybS9udW1hLmgNCj4gPiAgICAgICA+
ID4gPiA+ID4gPiA+IGluZGV4IDhmMWM2N2UzZWIuLjIxNTY5ZTYzNGIgMTAwNjQ0DQo+ID4gICAg
ICAgPiA+ID4gPiA+ID4gPiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJtL251bWEuaA0KPiA+ICAg
ICAgID4gPiA+ID4gPiA+ID4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9udW1hLmgNCj4gPiAg
ICAgICA+ID4gPiA+ID4gPiA+IEBAIC0zLDkgKzMsMTUgQEANCj4gPiAgICAgICA+ID4gPiA+ID4g
PiA+DQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPsKgICNpbmNsdWRlIDx4ZW4vbW0uaD4NCj4gPiAg
ICAgICA+ID4gPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPiArI2luY2x1ZGUgPGFz
bS9zZXR1cC5oPg0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4gKw0KPiA+ICAgICAgID4gPiA+ID4g
PiA+ID7CoCB0eXBlZGVmIHU4IG5vZGVpZF90Ow0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4NCj4g
PiAgICAgICA+ID4gPiA+ID4gPiA+IC0jaWZuZGVmIENPTkZJR19OVU1BDQo+ID4gICAgICAgPiA+
ID4gPiA+ID4gPiArI2lmZGVmIENPTkZJR19OVU1BDQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPiAr
DQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPiArI2RlZmluZSBOUl9OT0RFX01FTUJMS1MgTlJfTUVN
X0JBTktTDQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPiArDQo+ID4gICAgICAgPiA+ID4gPiA+ID4g
PiArI2Vsc2UNCj4gPiAgICAgICA+ID4gPiA+ID4gPiA+DQo+ID4gICAgICAgPiA+ID4gPiA+ID4g
PsKgIC8qIEZha2Ugb25lIG5vZGUgZm9yIG5vdy4gU2VlIGFsc28gbm9kZV9vbmxpbmVfbWFwLg0K
PiAqLw0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID7CoCAjZGVmaW5lIGNwdV90b19ub2RlKGNwdSkg
MA0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4gZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRlL3hlbi9u
dW1hLmgNCj4gYi94ZW4vaW5jbHVkZS94ZW4vbnVtYS5oDQo+ID4gICAgICAgPiA+ID4gPiA+ID4g
PiBpbmRleCAxOTc4ZTJiZTFiLi4xNzMxZTFjYzZiIDEwMDY0NA0KPiA+ICAgICAgID4gPiA+ID4g
PiA+ID4gLS0tIGEveGVuL2luY2x1ZGUveGVuL251bWEuaA0KPiA+ICAgICAgID4gPiA+ID4gPiA+
ID4gKysrIGIveGVuL2luY2x1ZGUveGVuL251bWEuaA0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID4g
QEAgLTEyLDcgKzEyLDkgQEANCj4gPiAgICAgICA+ID4gPiA+ID4gPiA+wqAgI2RlZmluZSBNQVhf
TlVNTk9ERVPCoCDCoCAxDQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPsKgICNlbmRpZg0KPiA+ICAg
ICAgID4gPiA+ID4gPiA+ID4NCj4gPiAgICAgICA+ID4gPiA+ID4gPiA+ICsjaWZuZGVmIE5SX05P
REVfTUVNQkxLUw0KPiA+ICAgICAgID4gPiA+ID4gPiA+ID7CoCAjZGVmaW5lIE5SX05PREVfTUVN
QkxLUyAoTUFYX05VTU5PREVTKjIpDQo+ID4gICAgICAgPiA+ID4gPiA+ID4gPiArI2VuZGlmDQo+
ID4gICAgICAgPiA+ID4gPg0KPiA+ICAgICAgID4gPiA+ID4gVGhpcyBvbmUgd2UgY2FuIHJlbW92
ZSBpdCBjb21wbGV0ZWx5IHJpZ2h0Pw0KPiA+ICAgICAgID4gPiA+DQo+ID4gICAgICAgPiA+ID4g
SG93IGFib3V0IGRlZmluZSBOUl9NRU1fQkFOS1MgdG86DQo+ID4gICAgICAgPiA+ID4gI2lmZGVm
IENPTkZJR19OUl9OVU1BX05PREVTDQo+ID4gICAgICAgPiA+ID4gI2RlZmluZSBOUl9NRU1fQkFO
S1MgKENPTkZJR19OUl9OVU1BX05PREVTICogMikNCj4gPiAgICAgICA+ID4gPiAjZWxzZQ0KPiA+
ICAgICAgID4gPiA+ICNkZWZpbmUgTlJfTUVNX0JBTktTIDEyOA0KPiA+ICAgICAgID4gPiA+ICNl
bmRpZg0KPiA+ICAgICAgID4gPiA+IGZvciBib3RoIHg4NiBhbmQgQXJtLiBGb3IgdGhvc2UgYXJj
aGl0ZWN0dXJlcyBkbyBub3Qgc3VwcG9ydA0KPiBvciBlbmFibGUNCj4gPiAgICAgICA+ID4gPiBO
VU1BLCB0aGV5IGNhbiBzdGlsbCB1c2UgIk5SX01FTV9CQU5LUyAxMjgiLiBBbmQgcmVwbGFjZSBh
bGwNCj4gPiAgICAgICA+ID4gTlJfTk9ERV9NRU1CTEtTDQo+ID4gICAgICAgPiA+ID4gaW4gTlVN
QSBjb2RlIHRvIE5SX01FTV9CQU5LUyB0byByZW1vdmUgTlJfTk9ERV9NRU1CTEtTDQo+IGNvbXBs
ZXRlbHkuDQo+ID4gICAgICAgPiA+ID4gSW4gdGhpcyBjYXNlLCBOUl9NRU1fQkFOS1MgY2FuIGJl
IGF3YXJlIG9mIHRoZSBjaGFuZ2VzIG9mDQo+ID4gICAgICAgPiA+IENPTkZJR19OUl9OVU1BX05P
REVTLg0KPiA+ICAgICAgID4gPg0KPiA+ICAgICAgID4gPiB4ODYgZG9lc24ndCBoYXZlIE5SX01F
TV9CQU5LUyBhcyBmYXIgYXMgSSBjYW4gdGVsbC4gSSBndWVzcw0KPiB5b3UgYWxzbw0KPiA+ICAg
ICAgID4gPiBtZWFudCB0byByZW5hbWUgTlJfTk9ERV9NRU1CTEtTIHRvIE5SX01FTV9CQU5LUz8N
Cj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+DQo+ID4gICAgICAgPiBZZXMuDQo+ID4gICAgICAg
Pg0KPiA+ICAgICAgID4gPiBCdXQgTlJfTUVNX0JBTktTIGlzIG5vdCBkaXJlY3RseSByZWxhdGVk
IHRvDQo+IENPTkZJR19OUl9OVU1BX05PREVTIGJlY2F1c2UNCj4gPiAgICAgICA+ID4gdGhlcmUg
Y2FuIGJlIG1hbnkgbWVtb3J5IGJhbmtzIGZvciBlYWNoIG51bWEgbm9kZSwgY2VydGFpbmx5DQo+
IG1vcmUgdGhhbg0KPiA+ICAgICAgID4gPiAyLiBUaGUgZXhpc3RpbmcgZGVmaW5pdGlvbiBvbiB4
ODY6DQo+ID4gICAgICAgPiA+DQo+ID4gICAgICAgPiA+ICNkZWZpbmUgTlJfTk9ERV9NRU1CTEtT
IChNQVhfTlVNTk9ERVMqMikNCj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+ID4gRG9lc24ndCBt
YWtlIGEgbG90IG9mIHNlbnNlIHRvIG1lLiBXYXMgaXQganVzdCBhbiBhcmJpdHJhcnkNCj4gbGlt
aXQgZm9yDQo+ID4gICAgICAgPiA+IHRoZSBsYWNrIG9mIGEgYmV0dGVyIHdheSB0byBzZXQgYSBt
YXhpbXVtPw0KPiA+ICAgICAgID4gPg0KPiA+ICAgICAgID4NCj4gPiAgICAgICA+IEF0IHRoYXQg
dGltZSwgdGhpcyB3YXMgcHJvYmFibHkgdGhlIG1vc3QgY29zdC1lZmZlY3RpdmUgYXBwcm9hY2gu
DQo+ID4gICAgICAgPiBFbm91Z2ggYW5kIGVhc3kuIEJ1dCwgaWYgbW9yZSBub2RlcyBuZWVkIHRv
IGJlIHN1cHBvcnRlZCBpbiB0aGUNCj4gPiAgICAgICA+IGZ1dHVyZSwgaXQgbWF5IGJyaW5nIG1v
cmUgbWVtb3J5IGJsb2Nrcy4gQW5kIHRoaXMgbWF4aW11bSB2YWx1ZQ0KPiA+ICAgICAgID4gbWln
aHQgbm90IGFwcGx5LiBUaGUgbWF4aW11bSBtYXkgbmVlZCB0byBzdXBwb3J0IGR5bmFtaWMNCj4g
ZXh0ZW5zaW9uLg0KPiA+ICAgICAgID4NCj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+ID4gT24g
dGhlIG90aGVyIGhhbmQsIE5SX01FTV9CQU5LUyBhbmQgTlJfTk9ERV9NRU1CTEtTIHNlZW0gdG8g
YmUNCj4gcmVsYXRlZC4NCj4gPiAgICAgICA+ID4gSW4gZmFjdCwgd2hhdCdzIHRoZSBkaWZmZXJl
bmNlPw0KPiA+ICAgICAgID4gPg0KPiA+ICAgICAgID4gPiBOUl9NRU1fQkFOS1MgaXMgdGhlIG1h
eCBudW1iZXIgb2YgbWVtb3J5IGJhbmtzICh3aXRoIG9yDQo+IHdpdGhvdXQNCj4gPiAgICAgICA+
ID4gbnVtYS1ub2RlLWlkKS4NCj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+ID4gTlJfTk9ERV9N
RU1CTEtTIGlzIHRoZSBtYXggbnVtYmVyIG9mIG1lbW9yeSBiYW5rcyB3aXRoIE5VTUENCj4gc3Vw
cG9ydA0KPiA+ICAgICAgID4gPiAod2l0aCBudW1hLW5vZGUtaWQpPw0KPiA+ICAgICAgID4gPg0K
PiA+ICAgICAgID4gPiBUaGV5IGFyZSBiYXNpY2FsbHkgdGhlIHNhbWUgdGhpbmcuIE9uIEFSTSBJ
IHdvdWxkIGp1c3QgZG86DQo+ID4gICAgICAgPiA+DQo+ID4gICAgICAgPg0KPiA+ICAgICAgID4g
UHJvYmFibHkgbm90LCBOUl9NRU1fQkFOS1Mgd2lsbCBjb3VudCB0aG9zZSBtZW1vcnkgcmFuZ2Vz
DQo+IHdpdGhvdXQNCj4gPiAgICAgICA+IG51bWEtbm9kZS1pZCBpbiBib290IG1lbW9yeSBwYXJz
aW5nIHN0YWdlIChwcm9jZXNzX21lbW9yeV9ub2RlDQo+IG9yDQo+ID4gICAgICAgPiBFRkkgcGFy
c2VyKS4gQnV0IE5SX05PREVfTUVNQkxLUyB3aWxsIG9ubHkgY291bnQgdGhvc2UgbWVtb3J5DQo+
IHJhbmdlcw0KPiA+ICAgICAgID4gd2l0aCBudW1hLW5vZGUtaWQuDQo+ID4gICAgICAgPg0KPiA+
ICAgICAgID4gPiAjZGVmaW5lIE5SX05PREVfTUVNQkxLUyBNQVgoTlJfTUVNX0JBTktTLA0KPiAo
Q09ORklHX05SX05VTUFfTk9ERVMgKiAyKSkNCj4gPiAgICAgICA+ID4NCj4gPiAgICAgICA+ID4N
Cj4gPg0KPiA+ICAgICAgID4gUXVvdGUgSnVsaWVuJ3MgY29tbWVudCBmcm9tIEhUTUwgZW1haWwg
dG8gaGVyZToNCj4gPiAgICAgICA+ICIgQXMgeW91IHdyb3RlIGFib3ZlLCB0aGUgc2Vjb25kIHBh
cnQgb2YgdGhlIE1BWCBpcyB0b3RhbGx5DQo+IGFyYml0cmFyeS4NCj4gPiAgICAgICA+IEluIGZh
Y3QsIGl0IGlzIHZlcnkgbGlrZWx5IHRoYW4gaWYgeW91IGhhdmUgbW9yZSB0aGFuIDY0IG5vZGVz
LA0KPiB5b3UgbWF5DQo+ID4gICAgICAgPiBuZWVkIGEgbG90IG1vcmUgdGhhbiAyIHJlZ2lvbnMg
cGVyIG5vZGUuDQo+ID4gICAgICAgPg0KPiA+ICAgICAgID4gU28sIGZvciBBcm0sIEkgd291bGQg
anVzdCBkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIGFzIGFuIGFsaWFzIHRvDQo+IE5SX01FTV9CQU5L
Uw0KPiA+ICAgICAgID4gc28gaXQgY2FuIGJlIHVzZWQgYnkgY29tbW9uIGNvZGUuDQo+ID4gICAg
ICAgPiAiDQo+ID4gICAgICAgPg0KPiA+ICAgICAgID4gPiBCdXQgaGVyZSBjb21lcyB0aGUgcHJv
YmxlbToNCj4gPiAgICAgICA+ID4gSG93IGNhbiB3ZSBzZXQgdGhlIE5SX01FTV9CQU5LUyBtYXhp
bXVtIHZhbHVlLCAxMjggc2VlbXMgYW4NCj4gYXJiaXRyYXJ5IHRvbz8NCj4gPiAgICAgICA+DQo+
ID4gICAgICAgPiBUaGlzIGlzIGJhc2VkIG9uIGhhcmR3YXJlIHdlIGN1cnJlbnRseSBzdXBwb3J0
ICh0aGUgbGFzdCB0aW1lIHdlDQo+IGJ1bXBlZCB0aGUgdmFsdWUgd2FzLCBJSVJDLCBmb3IgVGh1
bmRlci1YKS4gSW4gdGhlIGNhc2Ugb2YNCj4gPiAgICAgICBib290aW5nIFVFRkksIHdlIGNhbiBn
ZXQgYSBsb3Qgb2Ygc21hbGwgcmFuZ2VzIGFzIHdlIGRpc2NvdmVyIHRoZQ0KPiBSQU0gdXNpbmcg
dGhlIFVFRkkgbWVtb3J5IG1hcC4NCj4gPiAgICAgICA+DQo+ID4NCj4gPiAgICAgICBUaGFua3Mg
Zm9yIHRoZSBiYWNrZ3JvdW5kLg0KPiA+DQo+ID4gICAgICAgPg0KPiA+ICAgICAgID4gPiBJZiAj
ZGVmaW5lIE5SX01FTV9CQU5LUyAoQ09ORklHX05SX05VTUFfTk9ERVMgKiBOKT8gQW5kIHdoYXQg
Tg0KPiBzaG91bGQgYmUuDQo+ID4gICAgICAgPg0KPiA+ICAgICAgID4gTiB3b3VsZCBoYXZlIHRv
IGJlIHRoZSBtYXhpbXVtIG51bWJlciBvZiByYW5nZXMgeW91IGNhbiBmaW5kIGluDQo+IGEgTlVN
QSBub2RlLg0KPiA+ICAgICAgID4NCj4gPiAgICAgICA+IFdlIHdvdWxkIGFsc28gbmVlZCB0byBt
YWtlIHN1cmUgdGhpcyBkb2Vzbid0IGJyZWFrIGV4aXN0aW5nDQo+IHBsYXRmb3Jtcy4gU28gTiB3
b3VsZCBoYXZlIHRvIGJlIHF1aXRlIGxhcmdlIG9yIHdlIG5lZWQgYSBNQVggYXMNCj4gPiAgICAg
ICBTdGVmYW5vIHN1Z2dlc3RlZC4NCj4gPiAgICAgICA+DQo+ID4gICAgICAgPiBCdXQgSSB3b3Vs
ZCBwcmVmZXIgdG8ga2VlcCB0aGUgZXhpc3RpbmcgMTI4IGFuZCBhbGxvdyB0bw0KPiBjb25maWd1
cmUgaXQgYXQgYnVpbGQgdGltZSAobm90IG5lY2Vzc2FyaWx5IGluIHRoaXMgc2VyaWVzKS4gVGhp
cw0KPiA+ICAgICAgIGF2b2lkIHRvIGhhdmUgZGlmZmVyZW50IHdheSB0byBkZWZpbmUgdGhlIHZh
bHVlIGJhc2VkIE5VTUEgdnMgbm9uLQ0KPiBOVU1BLg0KPiA+DQo+ID4gICAgICAgSW4gdGhpcyBj
YXNlLCBjYW4gd2UgdXNlIFN0ZWZhbm8ncw0KPiA+ICAgICAgICIjZGVmaW5lIE5SX05PREVfTUVN
QkxLUyBNQVgoTlJfTUVNX0JBTktTLCAoQ09ORklHX05SX05VTUFfTk9ERVMgKg0KPiAyKSkiDQo+
ID4gICAgICAgaW4gbmV4dCB2ZXJzaW9uLiBJZiB5ZXMsIHNob3VsZCB3ZSBjaGFuZ2UgeDg2IHBh
cnQ/IEJlY2F1c2UNCj4gTlJfTUVNX0JBTktTDQo+ID4gICAgICAgaGFzIG5vdCBiZWVuIGRlZmlu
ZWQgaW4geDg2Lg0KPiA+DQo+ID4NCj4gPiBXaGF0IEkgbWVhbnQgYnkgY29uZmlndXJpbmcgZHlu
YW1pY2FsbHkgaXMgYWxsb3dpbmcgTlJfTUVNX0JBTktTIHRvIGJlDQo+IHNldCBieSB0aGUgdXNl
ci4NCj4gPg0KPiA+IFRoZSBzZWNvbmQgcGFydCBvZiB0aGUgTUFYIG1ha2VzIG5vIHNlbnNlIHRv
IG1lIChhdCBsZWFzdCBvbiBBcm0pLiBTbyBJDQo+IHJlYWxseSBwcmVmZXIgaWYgdGhpcyBpcyBu
b3QgcGFydCBvZiB0aGUgaW5pdGlhbCB2ZXJzaW9uLg0KPiA+DQo+ID4gV2UgY2FuIHJlZmluZSB0
aGUgdmFsdWUsIG9yIGludHJvZHVjZSB0aGUgTUFYIGluIHRoZSBmdXR1cmUgaWYgd2UgaGF2ZSBh
DQo+IGp1c3RpZmljYXRpb24gZm9yIGl0Lg0KPiANCj4gT0ssIHNvIGZvciBjbGFyaXR5IHRoZSBz
dWdnZXN0aW9uIGlzOg0KPiANCj4gLSBkZWZpbmUgTlJfTk9ERV9NRU1CTEtTIGFzIE5SX01FTV9C
QU5LUyBvbiBBUk0gaW4gdGhpcyBzZXJpZXMNCj4gLSBpbiB0aGUgZnV0dXJlIG1ha2UgTlJfTUVN
X0JBTktTIHVzZXItY29uZmlndXJhYmxlIHZpYSBrY29uZmlnDQo+IC0gZm9yIG5vdyBsZWF2ZSBO
Ul9NRU1fQkFOS1MgYXMgMTI4IG9uIEFSTQ0KPiANCj4gVGhhdCdzIGZpbmUgYnkgbWUuDQoNCk9r
LCBJIHdpbGwgb25seSBrZWVwDQojZGVmaW5lIE5SX05PREVfTUVNQkxLUyBOUl9NRU1fQkFOS1Mg
aW4gYXNtLWFybS9udW1hLmgsIGFuZCBsZWZ0DQp4ODYgTlJfTk9ERV9NRU1CTEtTIGRlZmluaXRp
b24gYXMgaXQgd2FzIGluIGFzbS14ODYvbnVtYS5oDQpCZWNhdXNlIGluIGN1cnJlbnQgc3RhZ2Us
IHdlIGNhbiBub3QgdW5pZnkgdGhlbSBpbiBvbiBwbGFjZS4NCkFuZCBJIHdpbGwgdXBkYXRlIHRo
ZSBjb21taXQgbWVzc2FnZSB0byBsb2cgc29tZSBvZiBvdXINCmRpc2N1c3Npb24gaW4gdGhpcyB0
dGhyZWFkLg0KDQoNCg0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 02:59:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 02:59:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197361.350381 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3Ky-0007LY-Sn; Tue, 28 Sep 2021 02:59:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197361.350381; Tue, 28 Sep 2021 02:59:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3Ky-0007LR-Pr; Tue, 28 Sep 2021 02:59:20 +0000
Received: by outflank-mailman (input) for mailman id 197361;
 Tue, 28 Sep 2021 02:59:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0YX1=OS=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mV3Ky-0007LJ-BM
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 02:59:20 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.52]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 121bc1aa-2008-11ec-bc63-12813bfff9fa;
 Tue, 28 Sep 2021 02:59:18 +0000 (UTC)
Received: from DB6PR1001CA0028.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:4:55::14)
 by DBBPR08MB5945.eurprd08.prod.outlook.com (2603:10a6:10:1f7::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 02:59:16 +0000
Received: from DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:55:cafe::86) by DB6PR1001CA0028.outlook.office365.com
 (2603:10a6:4:55::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
 Transport; Tue, 28 Sep 2021 02:59:16 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT038.mail.protection.outlook.com (10.152.21.84) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 02:59:16 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Tue, 28 Sep 2021 02:59:16 +0000
Received: from ab48ca72af33.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 B877F9A8-4119-421A-AE6A-EF9E19D38DB6.1; 
 Tue, 28 Sep 2021 02:59:04 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ab48ca72af33.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 02:59:04 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBAPR08MB5559.eurprd08.prod.outlook.com (2603:10a6:10:1a2::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 02:59:01 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 02:59:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 121bc1aa-2008-11ec-bc63-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YznWpNLGBTTBslejc45Z5WNnZBZJnNh3kX1oxPRNQE0=;
 b=JoFAiDbvREDC6LTVhivbd9CcyWCkgJmk3UzkBydTJpoEtvJwdofhTgMAghaEi+KJbXzBegBkN3nJLC8xNKklk5rQHKyhqDezyukzoRkMCgK+qQp+NYbIs3zMVBv47v/vipg52IB/ykTk4JePFYiPWH3g7UEvNhYX/AaB5bn3bUU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BFa6BQblO5XPD/aRsZQxLKrb/2VoQdy2rz1ty4Ja4Cx3hzgN2GAYTzytlEAirCp32p+DDwpgquNwsYuMV9VKMWvzfe0fBThAg3Otn8UxeKVuGVRd6sxHZRe1nCFPwVTc2dpwXFYhQ92WFmK03YQZ+YQy9oxPFfR0QL3JoQAyUzVpOIfqgEJb//0cTtQy36bk+gwdG1GidUh67yIl/7tEPGsTf4iveMgBwOj2KMH2LApRuzGslAW/BCEXdSUOVut0vp2OjrxLlzk0BY2RK0ucUpbANhYuC8ShO2SbVuJjBU6e2c3hWjBG+ybZc0sF0bQNLN5tH/LMjoqRw3pl49T+6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YznWpNLGBTTBslejc45Z5WNnZBZJnNh3kX1oxPRNQE0=;
 b=f5B28R6Atoqx/SEazwJwAVvWIJiCa0Y/IaaYOkNY2fvUHN7734be/9QjSIhrXK3uXgOrzUWwN0h7y7zDsUbzKUcL9eDJXzxt1O3C6xWU1+UnDBVH4AmwsZAFF1rPyAkyoyr4WZxXh4m0b1wPVe9v8Fbvl+/Fqm8zhOKcUxK15EANjBIL33BRXxRmYqIn+wjvunSb+T28iWPFmzXXJI5Erj7XBR3EdqWC7BPF3R+oUgo0d/dT63pzrnBs0HIW5u3yCe1cWxXrRMkhuEjMz2pztcn4jGIbh4zoUMSRq6Tyt9y2OV/NYa5va1eOIbjVMhPeeNE35QGLC+g1nMK5yimIcg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YznWpNLGBTTBslejc45Z5WNnZBZJnNh3kX1oxPRNQE0=;
 b=JoFAiDbvREDC6LTVhivbd9CcyWCkgJmk3UzkBydTJpoEtvJwdofhTgMAghaEi+KJbXzBegBkN3nJLC8xNKklk5rQHKyhqDezyukzoRkMCgK+qQp+NYbIs3zMVBv47v/vipg52IB/ykTk4JePFYiPWH3g7UEvNhYX/AaB5bn3bUU=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>, Jan Beulich
	<jbeulich@suse.com>
CC: Julien Grall <julien.grall.oss@gmail.com>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
Thread-Topic: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
Thread-Index:
 AQHXsHM4CZyvsR6BnEO/IHEJe48InquyiHQAgABwPACAAJ4wgIAD/P8AgAADRwCAAAjPgIAAhjmAgACh/HA=
Date: Tue, 28 Sep 2021 02:59:01 +0000
Message-ID:
 <DB9PR08MB6857EB6600193CCB7B77ECC39EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-37-wei.chen@arm.com>
 <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s>
 <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com>
 <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
 <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com>
 <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com>
 <42eb1303-1b45-5489-eac3-855f4ab35eb5@suse.com>
 <alpine.DEB.2.21.2109271006480.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109271006480.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: B9306C586C15294FA7A6F5EE81DC5DCA.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b5bf9a4b-237c-4a66-c496-08d9822bf584
x-ms-traffictypediagnostic: DBAPR08MB5559:|DBBPR08MB5945:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB5945E74933E43F6B530486FD9EA89@DBBPR08MB5945.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 7xyDVYhRROI6i29Rk57QzingHBhWsXSp0xYLlPcwycUsPN9OQS/kGgBCI16cCLJw2iiKV1zPY0yCodXphJakxs5xCYMjqKCgu2f0tVQGD9EP584s7IzUaDYhgjQi/X+yJhFmAT97bUEk15IeTntqO5JqfoefwSy4B1TKgc1D1MEY42VnYEZ1l4cKG2zNJl6veZ99iB6eNBo6+90YMC5oUPspeQKI0aLroLeJDvDvnla4j5Hzg8l5ADsCnNQaoP7NYXfQxKqsF9p74SZz7YizTS948BolTZD/0eM15z65RndS5igBTwdsycB2osHDXxllPssuDQxtjydmgvBuMDVZ+xl62XmyyAJZ5Xt0pVS8ql5vjbR72mMgb6VCYK9WP7mtN/Lx3qouq+H9atY2H5+Ss8/+T46VC43U97aGSZrTf/54ujEg0O0D7evSZb3x3qP0PqkjeJl9osuMLseIHZ4a4atRRFXJquVwPBnAB6L0j6jVYAF48+KcgMEoyxrhAZ5dlNw3wh0kpnL4cDYJwoKSIyYZWP1Z1yI5ScwS8ip0E1OfXD2IoR9Pna9H0CqgrBkKl4q5I8/S3W3F/7CdFOPSp1DTmvQZ0J6fv9v3ffbLRhlnHY3jfttnrX2V0kfh3+/4zvGcZLTyN8rIxLrjjW5Pw6WadSIn+JXSR5NK+MLTPnK8FB2y3S0vyWCnE7Yev2z4q2X1/QE1qctPxPm+qgrZTtzPniQagr8/TcTuayn+ubM9uk0gFM30f/qF/2oT77XNpiIbDpA0GhMVeQguRsbbmk6NCYeLfLfs9gyRuR8KK1o=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52536014)(26005)(9686003)(186003)(55016002)(8676002)(8936002)(54906003)(86362001)(38070700005)(33656002)(122000001)(83380400001)(38100700002)(316002)(53546011)(66476007)(508600001)(66446008)(66946007)(66556008)(76116006)(7696005)(966005)(110136005)(6506007)(64756008)(71200400001)(2906002)(4326008)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5559
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	208a95a9-8294-4a1d-ad0c-08d9822becdc
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+6d8YT3a5PAYjXjf3OcHIDKHC/GaUgllVdxYTe6GXIWOsFjKF+H5wIqJ1J7peFfUCWew6sB5OisBPzKW/hd2kKiKlPOx74p3Bjbv+6JZnnTQZX2hapMvgaIctYFCcVgI5NHx+wLgSiP9I4HOgssoL5F0xxsqOKQhxDXUHQfKAozx9UcYK9Z0BVo67HNhabfY77cy5bJ0Ijlmaocm7RZJgpqmvCjyWrUBUtYLh5np9f9HcGGZNgi+gifXnZFaDEL7HAKyAh6mnLz96M8P11gr8Ev+WVdAVxRMXejWk4GpQ34dR99wRxWeYT3gk7j4frJDx2MU2zWsZEfdyWULtTXqXoP6Q16JDRFV+N39GNhV8jl+DdvFhip+ViIKbDsfUlbVQyhrKqDkjcGctELYV/nRGrBUoJBYxSkzMNrJP7SlxkuFOwCFu08UjRJQceMctsdzy+jR16GpCCUFiPnq5tGxKfgiD5a1cFHq+FZjWQsmpnKT/hOuUEKLELIMdlM8AYp3+lwS4TQ6WY9lB0pQKvvJus/vM6uAHHmxWcf/EOko9i7Eod37w01yAKuDJAB9pkDxVq94d1bLcS+yua2nZd5IbzEUB9eQBMWr7Ayvo4LKt+skuBuhIFNxNzXbT/CaMUOVopr2dL/JavR2vDX7Hco+QSVyaQeVC7ncpHD2gUYh00ujhWRj6qSdFHfuxJ3akAkAau0N6DvAdKZmZAC8XzIgvICSDEnmDvECPT4phHqo/pHwzZrpnd6qaiBxl56Bm06HLUQKOjiAWepMf0Q6i2kje2bWC/ZPIddM2qqT3kJFc34=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(8936002)(4326008)(6506007)(54906003)(83380400001)(336012)(86362001)(356005)(186003)(26005)(81166007)(52536014)(7696005)(55016002)(9686003)(47076005)(110136005)(53546011)(8676002)(5660300002)(2906002)(36860700001)(70586007)(508600001)(316002)(70206006)(82310400003)(966005)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 02:59:16.3899
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b5bf9a4b-237c-4a66-c496-08d9822bf584
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT038.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5945

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHE6jnU
wjI4yNUgMToxNw0KPiBUbzogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPg0KPiBDYzog
SnVsaWVuIEdyYWxsIDxqdWxpZW4uZ3JhbGwub3NzQGdtYWlsLmNvbT47IFN0ZWZhbm8gU3RhYmVs
bGluaQ0KPiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0u
Y29tPjsgeGVuLWRldmVsIDx4ZW4tDQo+IGRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnPjsgQmVy
dHJhbmQgTWFycXVpcyA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPg0KPiBTdWJqZWN0OiBSZTog
W1BBVENIIDM2LzM3XSB4ZW4vYXJtOiBQcm92aWRlIEtjb25maWcgb3B0aW9ucyBmb3IgQXJtIHRv
DQo+IGVuYWJsZSBOVU1BDQo+IA0KPiBPbiBNb24sIDI3IFNlcCAyMDIxLCBKYW4gQmV1bGljaCB3
cm90ZToNCj4gPiBPbiAyNy4wOS4yMDIxIDEwOjQ1LCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+ID4g
PiBPbiBNb24sIDI3IFNlcCAyMDIxLCAxMDozMyBKYW4gQmV1bGljaCwgPGpiZXVsaWNoQHN1c2Uu
Y29tPiB3cm90ZToNCj4gPiA+DQo+ID4gPj4gT24gMjQuMDkuMjAyMSAyMTozOSwgU3RlZmFubyBT
dGFiZWxsaW5pIHdyb3RlOg0KPiA+ID4+PiBPbiBGcmksIDI0IFNlcCAyMDIxLCBXZWkgQ2hlbiB3
cm90ZToNCj4gPiA+Pj4+IE9uIDIwMjEvOS8yNCAxMTozMSwgU3RlZmFubyBTdGFiZWxsaW5pIHdy
b3RlOg0KPiA+ID4+Pj4+IE9uIFRodSwgMjMgU2VwIDIwMjEsIFdlaSBDaGVuIHdyb3RlOg0KPiA+
ID4+Pj4+PiAtLS0gYS94ZW4vYXJjaC9hcm0vS2NvbmZpZw0KPiA+ID4+Pj4+PiArKysgYi94ZW4v
YXJjaC9hcm0vS2NvbmZpZw0KPiA+ID4+Pj4+PiBAQCAtMzQsNiArMzQsMTcgQEAgY29uZmlnIEFD
UEkNCj4gPiA+Pj4+Pj4gICAgICBBZHZhbmNlZCBDb25maWd1cmF0aW9uIGFuZCBQb3dlciBJbnRl
cmZhY2UgKEFDUEkpIHN1cHBvcnQNCj4gZm9yDQo+ID4gPj4gWGVuIGlzDQo+ID4gPj4+Pj4+ICAg
ICAgYW4gYWx0ZXJuYXRpdmUgdG8gZGV2aWNlIHRyZWUgb24gQVJNNjQuDQo+ID4gPj4+Pj4+ICAg
KyBjb25maWcgREVWSUNFX1RSRUVfTlVNQQ0KPiA+ID4+Pj4+PiArICBkZWZfYm9vbCBuDQo+ID4g
Pj4+Pj4+ICsgIHNlbGVjdCBOVU1BDQo+ID4gPj4+Pj4+ICsNCj4gPiA+Pj4+Pj4gK2NvbmZpZyBB
Uk1fTlVNQQ0KPiA+ID4+Pj4+PiArICBib29sICJBcm0gTlVNQSAoTm9uLVVuaWZvcm0gTWVtb3J5
IEFjY2VzcykgU3VwcG9ydA0KPiAoVU5TVVBQT1JURUQpIg0KPiA+ID4+IGlmDQo+ID4gPj4+Pj4+
IFVOU1VQUE9SVEVEDQo+ID4gPj4+Pj4+ICsgIHNlbGVjdCBERVZJQ0VfVFJFRV9OVU1BIGlmIEhB
U19ERVZJQ0VfVFJFRQ0KPiA+ID4+Pj4+DQo+ID4gPj4+Pj4gU2hvdWxkIGl0IGJlOiBkZXBlbmRz
IG9uIEhBU19ERVZJQ0VfVFJFRSA/DQo+ID4gPj4+Pj4gKEFuZCBldmVudHVhbGx5IGRlcGVuZHMg
b24gSEFTX0RFVklDRV9UUkVFIHx8IEFDUEkpDQo+ID4gPj4+Pj4NCj4gPiA+Pj4+DQo+ID4gPj4+
PiBBcyB0aGUgZGlzY3Vzc2lvbiBpbiBSRkMgWzFdLiBXZSB3YW50IHRvIG1ha2UgQVJNX05VTUEg
YXMgYSBnZW5lcmljDQo+ID4gPj4+PiBvcHRpb24gY2FuIGJlIHNlbGVjdGVkIGJ5IHVzZXJzLiBB
bmQgZGVwZW5kcyBvbiBoYXNfZGV2aWNlX3RyZWUNCj4gPiA+Pj4+IG9yIEFDUEkgdG8gc2VsZWN0
IERFVklDRV9UUkVFX05VTUEgb3IgQUNQSV9OVU1BLg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IElmIHdl
IGFkZCBIQVNfREVWSUNFX1RSRUUgfHwgQUNQSSBhcyBkZXBlbmRlbmNpZXMgZm9yIEFSTV9OVU1B
LA0KPiA+ID4+Pj4gZG9lcyBpdCBiZWNvbWUgYSBsb29wIGRlcGVuZGVuY3k/DQo+ID4gPj4+Pg0K
PiA+ID4+Pj4NCj4gPiA+PiBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2FyY2hpdmVzL2h0
bWwveGVuLWRldmVsLzIwMjEtDQo+IDA4L21zZzAwODg4Lmh0bWwNCj4gPiA+Pj4NCj4gPiA+Pj4g
T0ssIEkgYW0gZmluZSB3aXRoIHRoYXQuIEkgd2FzIGp1c3QgdHJ5aW5nIHRvIGNhdGNoIHRoZSBj
YXNlIHdoZXJlIGENCj4gPiA+Pj4gdXNlciBzZWxlY3RzICJBUk1fTlVNQSIgYnV0IGFjdHVhbGx5
IG5laXRoZXIgQUNQSSBub3INCj4gSEFTX0RFVklDRV9UUkVFDQo+ID4gPj4+IGFyZSBzZWxlY3Rl
ZCBzbyBub3RoaW5nIGhhcHBlbnMuIEkgd2FzIHRyeWluZyB0byBtYWtlIGl0IGNsZWFyIHRoYXQN
Cj4gPiA+Pj4gQVJNX05VTUEgZGVwZW5kcyBvbiBoYXZpbmcgYXQgbGVhc3Qgb25lIGJldHdlZW4g
SEFTX0RFVklDRV9UUkVFIG9yDQo+IEFDUEkNCj4gPiA+Pj4gYmVjYXVzZSBvdGhlcndpc2UgaXQg
aXMgbm90IGdvaW5nIHRvIHdvcmsuDQo+ID4gPj4+DQo+ID4gPj4+IFRoYXQgc2FpZCwgSSBkb24n
dCB0aGluayB0aGlzIGlzIGltcG9ydGFudCBiZWNhdXNlIEhBU19ERVZJQ0VfVFJFRQ0KPiA+ID4+
PiBjYW5ub3QgYmUgdW5zZWxlY3RlZC4gU28gaWYgd2UgY2Fubm90IGZpbmQgYSB3YXkgdG8gZXhw
cmVzcyB0aGUNCj4gPiA+Pj4gZGVwZW5kZW5jeSwgSSB0aGluayBpdCBpcyBmaW5lIHRvIGtlZXAg
dGhlIHBhdGNoIGFzIGlzLg0KPiA+ID4+DQo+ID4gPj4gU28gaG93IGFib3V0IGRvaW5nIHRoaW5n
cyB0aGUgb3RoZXIgd2F5IGFyb3VuZDogQVJNX05VTUEgaGFzIG5vDQo+IHByb21wdA0KPiA+ID4+
IGFuZCBkZWZhdWx0cyB0byBBQ1BJX05VTUEgfHwgRFRfTlVNQSwgYW5kIERUX05VTUEgZ2FpbnMg
YSBwcm9tcHQNCj4gaW5zdGVhZA0KPiA+ID4+IChhbmQsIGZvciBBcm0gYXQgbGVhc3QsIEFDUElf
TlVNQSBhcyB3ZWxsOyB0aGlzIG1pZ2h0IGV2ZW4gYmUNCj4gd29ydGh3aGlsZQ0KPiA+ID4+IHRv
IGhhdmUgb24geDg2IGRvd24gdGhlIHJvYWQpLg0KPiA+ID4+DQo+ID4gPg0KPiA+ID4gQXMgSSB3
cm90ZSBiZWZvcmUsIEkgZG9uJ3QgdGhpbmsgdGhlIHVzZXIgc2hvdWxkIHNheSAiSSB3YW50IHRv
IGVuYWJsZQ0KPiBOVU1BDQo+ID4gPiB3aXRoIERldmljZS1UcmVlIG9yIEFDUEkiLiBJbnN0ZWFk
LCB0aGV5IHNheSB3aGV0aGVyIHRoZXkgd2FudCB0byB1c2UNCj4gTlVNQQ0KPiA+ID4gYW5kIGxl
dCBYZW4gZGVjaWRlIHRvIGVuYWJsZSB0aGUgRFQvQUNQSSBzdXBwb3J0Lg0KPiA+ID4NCj4gPiA+
IEluIG90aGVyIHdvcmQsIHRoZSBwcm9tcHQgc2hvdWxkIHN0YXkgb24gQVJNX05VTUEuDQo+ID4N
Cj4gPiBPa2F5LiBJbiB3aGljaCBjYXNlIEknbSBjb25mdXNlZCBieSBTdGVmYW5vJ3MgcXVlc3Rp
b24uDQo+IA0KPiBMZXQgbWUgY2xhcmlmeTogSSB0aGluayBpdCBpcyBmaW5lIHRvIGhhdmUgYSBz
aW5nbGUgcHJvbXB0IGZvciBOVU1BIGluDQo+IEtjb25maWcuIEhvd2V2ZXIsIEkgYW0ganVzdCBw
b2ludGluZyBvdXQgdGhhdCBpdCBpcyB0aGVvcmV0aWNhbGx5DQo+IHBvc3NpYmxlIHdpdGggdGhl
IGN1cnJlbnQgY29kZSB0byBwcmVzZW50IGFuIEFSTV9OVU1BIHByb21wdCB0byB0aGUgdXNlcg0K
PiBidXQgYWN0dWFsbHkgaGF2ZSBubyBOVU1BIGVuYWJsZWQgYXQgdGhlIGVuZCBiZWNhdXNlIGJv
dGggREVWSUNFIFRSRUUNCj4gYW5kIEFDUEkgYXJlIGRpc2FibGVkLiBUaGlzIGlzIG9ubHkgYSB0
aGVvcmV0aWNhbCBwcm9ibGVtIGJlY2F1c2UgREVWSUNFDQo+IFRSRUUgc3VwcG9ydCAoSEFTX0RF
VklDRV9UUkVFKSBjYW5ub3QgYmUgZGlzYWJsZWQgdG9kYXkuIEFsc28gSSBjYW5ub3QNCj4gaW1h
Z2luZSBob3cgYSBjb25maWd1cmF0aW9uIHdpdGggbmVpdGhlciBERVZJQ0UgVFJFRSBub3IgQUNQ
SSBjYW4gYmUNCj4gY29ycmVjdC4gU28gSSBkb24ndCB0aGluayBpdCBpcyBhIGNyaXRpY2FsIGNv
bmNlcm4uDQo+IA0KPiBUaGF0IHNhaWQsIHlvdSBjYW4gc2VlIHRoYXQsIGF0IGxlYXN0IHRoZW9y
ZXRpY2FsbHksIEFSTV9OVU1BIGRlcGVuZHMgb24NCj4gZWl0aGVyIEhBU19ERVZJQ0VfVFJFRSBv
ciBBQ1BJLCBzbyBJIHN1Z2dlc3RlZCB0byBhZGQ6DQo+IA0KPiBkZXBlbmRzIG9uIEhBU19ERVZJ
Q0VfVFJFRSB8fCBBQ1BJDQo+IA0KPiBXZWkgYW5zd2VyZWQgdGhhdCBpdCBtaWdodCBpbnRyb2R1
Y2UgYSBjaXJjdWxhciBkZXBlbmRlbmN5LCBidXQgSSBkaWQNCj4gdHJ5IHRoZSBhZGRpdGlvbiBv
ZiAiZGVwZW5kcyBvbiBIQVNfREVWSUNFX1RSRUUgfHwgQUNQSSIgdW5kZXIgQVJNX05VTUENCj4g
aW4gS2NvbmZpZyBhbmQgZXZlcnl0aGluZyBidWlsdCBmaW5lIGhlcmUuDQoNCk9rLCBJIHdpbGwg
YWRkICJkZXBlbmRzIG9uIEhBU19ERVZJQ0VfVFJFRSIgaW4gbmV4dCB2ZXJzaW9uLCBidXQgInx8
IEFDUEkiDQp3aWxsIGJlIGxhdGVyIHdoZW4gd2UgaGF2ZSBBQ1BJIE5VTUEgZm9yIEFybSA6ICkN
Cg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 03:30:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 03:30:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197372.350393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3pT-0003GI-9e; Tue, 28 Sep 2021 03:30:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197372.350393; Tue, 28 Sep 2021 03:30:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV3pT-0003GB-6e; Tue, 28 Sep 2021 03:30:51 +0000
Received: by outflank-mailman (input) for mailman id 197372;
 Tue, 28 Sep 2021 03:30:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV3pR-0003G4-WF
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 03:30:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4b534ac8-b758-4c59-84d7-2630518686dc;
 Tue, 28 Sep 2021 03:30:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 816C161052;
 Tue, 28 Sep 2021 03:30:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b534ac8-b758-4c59-84d7-2630518686dc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632799847;
	bh=wMBJSK3gpANuVYGHFbqZg2iREsk5ctDlTZuA9ErKpxI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EQVE3HGbAOxB3aYSrPhi1Lc84XXb1dvkfE07vHPbZAu1mHjC6QIvE98Su+r/3U4+e
	 5pJX8NWR+/6u0rEIsldYIuH34Gi2ZL8aitYYnaGR+zaEiJ1JRkt9bvOjnWo1t7R7Jv
	 Ox5g9ItYuQDgdQqfQDaPjjG/wck4iCQGORWhPAd9Bi7i/oScDsRq6mKDRmoqv0dY3N
	 pEPFOllq77IRqq88VMly6q+viSuuaa0ZoOVRPk2lfNzKaEbRP9WGDKwuldqWHi619b
	 ui4G85FRPTAHfzFAiFbV8qR1MgfTfINLHmGBKk5JK07NHO1zCjiKYNBS8V1l6Ty+ZG
	 JsgbBC2PIQLgA==
Date: Mon, 27 Sep 2021 20:30:47 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien.grall.oss@gmail.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to enable
 NUMA
In-Reply-To: <DB9PR08MB6857EB6600193CCB7B77ECC39EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109272030380.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-37-wei.chen@arm.com> <alpine.DEB.2.21.2109232029450.17979@sstabellini-ThinkPad-T480s> <56e0cc0e-7405-74b0-eb4b-07cd9cdae225@arm.com> <alpine.DEB.2.21.2109241237210.17979@sstabellini-ThinkPad-T480s>
 <4e6f7222-59c8-7853-cf2c-076620efa244@suse.com> <CAJ=z9a1wOd6Hct50O_3B5q3o-bvhgMFy+A95gYMPfz-YL8uFDQ@mail.gmail.com> <42eb1303-1b45-5489-eac3-855f4ab35eb5@suse.com> <alpine.DEB.2.21.2109271006480.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857EB6600193CCB7B77ECC39EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-487226256-1632799847=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-487226256-1632799847=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 28 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月28日 1:17
> > To: Jan Beulich <jbeulich@suse.com>
> > Cc: Julien Grall <julien.grall.oss@gmail.com>; Stefano Stabellini
> > <sstabellini@kernel.org>; Wei Chen <Wei.Chen@arm.com>; xen-devel <xen-
> > devel@lists.xenproject.org>; Bertrand Marquis <Bertrand.Marquis@arm.com>
> > Subject: Re: [PATCH 36/37] xen/arm: Provide Kconfig options for Arm to
> > enable NUMA
> > 
> > On Mon, 27 Sep 2021, Jan Beulich wrote:
> > > On 27.09.2021 10:45, Julien Grall wrote:
> > > > On Mon, 27 Sep 2021, 10:33 Jan Beulich, <jbeulich@suse.com> wrote:
> > > >
> > > >> On 24.09.2021 21:39, Stefano Stabellini wrote:
> > > >>> On Fri, 24 Sep 2021, Wei Chen wrote:
> > > >>>> On 2021/9/24 11:31, Stefano Stabellini wrote:
> > > >>>>> On Thu, 23 Sep 2021, Wei Chen wrote:
> > > >>>>>> --- a/xen/arch/arm/Kconfig
> > > >>>>>> +++ b/xen/arch/arm/Kconfig
> > > >>>>>> @@ -34,6 +34,17 @@ config ACPI
> > > >>>>>>      Advanced Configuration and Power Interface (ACPI) support
> > for
> > > >> Xen is
> > > >>>>>>      an alternative to device tree on ARM64.
> > > >>>>>>   + config DEVICE_TREE_NUMA
> > > >>>>>> +  def_bool n
> > > >>>>>> +  select NUMA
> > > >>>>>> +
> > > >>>>>> +config ARM_NUMA
> > > >>>>>> +  bool "Arm NUMA (Non-Uniform Memory Access) Support
> > (UNSUPPORTED)"
> > > >> if
> > > >>>>>> UNSUPPORTED
> > > >>>>>> +  select DEVICE_TREE_NUMA if HAS_DEVICE_TREE
> > > >>>>>
> > > >>>>> Should it be: depends on HAS_DEVICE_TREE ?
> > > >>>>> (And eventually depends on HAS_DEVICE_TREE || ACPI)
> > > >>>>>
> > > >>>>
> > > >>>> As the discussion in RFC [1]. We want to make ARM_NUMA as a generic
> > > >>>> option can be selected by users. And depends on has_device_tree
> > > >>>> or ACPI to select DEVICE_TREE_NUMA or ACPI_NUMA.
> > > >>>>
> > > >>>> If we add HAS_DEVICE_TREE || ACPI as dependencies for ARM_NUMA,
> > > >>>> does it become a loop dependency?
> > > >>>>
> > > >>>>
> > > >> https://lists.xenproject.org/archives/html/xen-devel/2021-
> > 08/msg00888.html
> > > >>>
> > > >>> OK, I am fine with that. I was just trying to catch the case where a
> > > >>> user selects "ARM_NUMA" but actually neither ACPI nor
> > HAS_DEVICE_TREE
> > > >>> are selected so nothing happens. I was trying to make it clear that
> > > >>> ARM_NUMA depends on having at least one between HAS_DEVICE_TREE or
> > ACPI
> > > >>> because otherwise it is not going to work.
> > > >>>
> > > >>> That said, I don't think this is important because HAS_DEVICE_TREE
> > > >>> cannot be unselected. So if we cannot find a way to express the
> > > >>> dependency, I think it is fine to keep the patch as is.
> > > >>
> > > >> So how about doing things the other way around: ARM_NUMA has no
> > prompt
> > > >> and defaults to ACPI_NUMA || DT_NUMA, and DT_NUMA gains a prompt
> > instead
> > > >> (and, for Arm at least, ACPI_NUMA as well; this might even be
> > worthwhile
> > > >> to have on x86 down the road).
> > > >>
> > > >
> > > > As I wrote before, I don't think the user should say "I want to enable
> > NUMA
> > > > with Device-Tree or ACPI". Instead, they say whether they want to use
> > NUMA
> > > > and let Xen decide to enable the DT/ACPI support.
> > > >
> > > > In other word, the prompt should stay on ARM_NUMA.
> > >
> > > Okay. In which case I'm confused by Stefano's question.
> > 
> > Let me clarify: I think it is fine to have a single prompt for NUMA in
> > Kconfig. However, I am just pointing out that it is theoretically
> > possible with the current code to present an ARM_NUMA prompt to the user
> > but actually have no NUMA enabled at the end because both DEVICE TREE
> > and ACPI are disabled. This is only a theoretical problem because DEVICE
> > TREE support (HAS_DEVICE_TREE) cannot be disabled today. Also I cannot
> > imagine how a configuration with neither DEVICE TREE nor ACPI can be
> > correct. So I don't think it is a critical concern.
> > 
> > That said, you can see that, at least theoretically, ARM_NUMA depends on
> > either HAS_DEVICE_TREE or ACPI, so I suggested to add:
> > 
> > depends on HAS_DEVICE_TREE || ACPI
> > 
> > Wei answered that it might introduce a circular dependency, but I did
> > try the addition of "depends on HAS_DEVICE_TREE || ACPI" under ARM_NUMA
> > in Kconfig and everything built fine here.
> 
> Ok, I will add "depends on HAS_DEVICE_TREE" in next version, but "|| ACPI"
> will be later when we have ACPI NUMA for Arm : )

Good point :)
--8323329-487226256-1632799847=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:01:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:01:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197383.350404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4IU-0006uC-MM; Tue, 28 Sep 2021 04:00:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197383.350404; Tue, 28 Sep 2021 04:00:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4IU-0006u5-JC; Tue, 28 Sep 2021 04:00:50 +0000
Received: by outflank-mailman (input) for mailman id 197383;
 Tue, 28 Sep 2021 04:00:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV4IT-0006ty-Hn
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:00:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d9fb0a6-dc2d-4a63-ad1a-66968e0e20c3;
 Tue, 28 Sep 2021 04:00:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1795F611F2;
 Tue, 28 Sep 2021 04:00:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d9fb0a6-dc2d-4a63-ad1a-66968e0e20c3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632801646;
	bh=8plRDu2QTjNKaoPcm7y+cJOGQ+Q0fXhT5TdsYLQyKyg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=k+WM1FKIODP9mm2A408I85xRJwDP6THtWlRcrE71hIGcHn01bGGA1fT7eGBijtbDQ
	 ev9Dt+XYZOgzsVT6qg8vkLSUTkC+dsGXle49i1Tac82SijW/R5AzkYxuo/gyN7n8rh
	 XOE/rYJGl2pnaLIMwj6ZbQkIor6rNDX0ypdcCJCEhKTvkjh9MUyvUNFZjKHX7LaMQs
	 /uf9cd/EkCNAAI3OTcgdRwjkqdAseiQIasXlueW5z68pRYoVhRsXQO7hzU2/C0GtZY
	 KdoXx+DbiZJ/5LyAP5q5wrtU0VhVbyiDUGsURz1BJMUQZwmGdv5SRHr1X/QGDYGir5
	 8bLK1JQrnmhhw==
Date: Mon, 27 Sep 2021 21:00:45 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
In-Reply-To: <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com>
Message-ID: <alpine.DEB.2.21.2109272046320.5022@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-11-andr2000@gmail.com> <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s> <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1264100615-1632801182=:5022"
Content-ID: <alpine.DEB.2.21.2109272053060.5022@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1264100615-1632801182=:5022
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109272053061.5022@sstabellini-ThinkPad-T480s>

On Mon, 27 Sep 2021, Oleksandr Andrushchenko wrote:
> On 25.09.21 03:44, Stefano Stabellini wrote:
> > On Thu, 23 Sep 2021, Oleksandr Andrushchenko wrote:
> >> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>
> >> PCI host bridges are special devices in terms of implementing PCI
> >> passthrough. According to [1] the current implementation depends on
> >> Domain-0 to perform the initialization of the relevant PCI host
> >> bridge hardware and perform PCI device enumeration. In order to
> >> achieve that one of the required changes is to not map all the memory
> >> ranges in map_range_to_domain as we traverse the device tree on startup
> >> and perform some additional checks if the range needs to be mapped to
> >> Domain-0.
> >>
> >> The generic PCI host controller device tree binding says [2]:
> >> - ranges: As described in IEEE Std 1275-1994, but must provide
> >>            at least a definition of non-prefetchable memory. One
> >>            or both of prefetchable Memory and IO Space may also
> >>            be provided.
> >>
> >> - reg   : The Configuration Space base address and size, as accessed
> >>            from the parent bus.  The base address corresponds to
> >>            the first bus in the "bus-range" property.  If no
> >>            "bus-range" is specified, this will be bus 0 (the default).
> >>
> >> >From the above none of the memory ranges from the "ranges" property
> >> needs to be mapped to Domain-0 at startup as MMIO mapping is going to
> >> be handled dynamically by vPCI as we assign PCI devices, e.g. each
> >> device assigned to Domain-0/guest will have its MMIOs mapped/unmapped
> >> as needed by Xen.
> >>
> >> The "reg" property covers not only ECAM space, but may also have other
> >> then the configuration memory ranges described, for example [3]:
> >> - reg: Should contain rc_dbi, config registers location and length.
> >> - reg-names: Must include the following entries:
> >>     "rc_dbi": controller configuration registers;
> >>     "config": PCIe configuration space registers.
> >>
> >> This patch makes it possible to not map all the ranges from the
> >> "ranges" property and also ECAM from the "reg". All the rest from the
> >> "reg" property still needs to be mapped to Domain-0, so the PCI
> >> host bridge remains functional in Domain-0.
> >>
> >> [1] https://urldefense.com/v3/__https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00777.html__;!!GF_29dbcQIUBPA!lrCuNRzUVkRf4FLgp3hW-4uOldgKr4qNpZb_ufI0jW-O0eRH11VFDeGcs5pPdtKZT1_ou4InSg$ [lists[.]xenproject[.]org]
> >> [2] https://urldefense.com/v3/__https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt__;!!GF_29dbcQIUBPA!lrCuNRzUVkRf4FLgp3hW-4uOldgKr4qNpZb_ufI0jW-O0eRH11VFDeGcs5pPdtKZT1_T5yn7GA$ [kernel[.]org]
> >> [3] https://urldefense.com/v3/__https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt__;!!GF_29dbcQIUBPA!lrCuNRzUVkRf4FLgp3hW-4uOldgKr4qNpZb_ufI0jW-O0eRH11VFDeGcs5pPdtKZT18im_Y2tw$ [kernel[.]org]
> >>
> >> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>
> >> ---
> >> Since v1:
> >>   - Added better description of why and what needs to be mapped into
> >>     Domain-0's p2m and what doesn't
> >>   - Do not do any mappings for PCI devices while traversing the DT
> >>   - Walk all the bridges and make required mappings in one go
> >> ---
> >>   xen/arch/arm/domain_build.c        | 38 +++++++++++++++--------
> >>   xen/arch/arm/pci/ecam.c            | 14 +++++++++
> >>   xen/arch/arm/pci/pci-host-common.c | 48 ++++++++++++++++++++++++++++++
> >>   xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
> >>   xen/include/asm-arm/pci.h          |  9 ++++++
> >>   xen/include/asm-arm/setup.h        | 13 ++++++++
> >>   6 files changed, 111 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> >> index 83ab0d52cce9..e72c1b881cae 100644
> >> --- a/xen/arch/arm/domain_build.c
> >> +++ b/xen/arch/arm/domain_build.c
> >> @@ -10,7 +10,6 @@
> >>   #include <asm/regs.h>
> >>   #include <xen/errno.h>
> >>   #include <xen/err.h>
> >> -#include <xen/device_tree.h>
> >>   #include <xen/libfdt/libfdt.h>
> >>   #include <xen/guest_access.h>
> >>   #include <xen/iocap.h>
> >> @@ -47,12 +46,6 @@ static int __init parse_dom0_mem(const char *s)
> >>   }
> >>   custom_param("dom0_mem", parse_dom0_mem);
> >>   
> >> -struct map_range_data
> >> -{
> >> -    struct domain *d;
> >> -    p2m_type_t p2mt;
> >> -};
> >> -
> >>   /* Override macros from asm/page.h to make them work with mfn_t */
> >>   #undef virt_to_mfn
> >>   #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
> >> @@ -1388,9 +1381,8 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
> >>       return 0;
> >>   }
> >>   
> >> -static int __init map_range_to_domain(const struct dt_device_node *dev,
> >> -                                      u64 addr, u64 len,
> >> -                                      void *data)
> >> +int __init map_range_to_domain(const struct dt_device_node *dev,
> >> +                               u64 addr, u64 len, void *data)
> >>   {
> >>       struct map_range_data *mr_data = data;
> >>       struct domain *d = mr_data->d;
> >> @@ -1417,6 +1409,13 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
> >>           }
> >>       }
> >>   
> >> +#ifdef CONFIG_HAS_PCI
> >> +    if ( is_pci_passthrough_enabled() &&
> >> +         (device_get_class(dev) == DEVICE_PCI) &&
> >> +         !mr_data->map_pci_bridge )
> >> +        need_mapping = false;
> >> +#endif
> > With the change I suggested below turning map_pci_bridge into
> > skip_mapping, then this check could go away if we just set need_mapping
> > as follows:
> >
> > bool need_mapping = !dt_device_for_passthrough(dev) &&
> >                      !mr_data->skip_mapping;
> 
> Not exactly. This check, e.g.
> 
> "is_pci_passthrough_enabled() && (device_get_class(dev) == DEVICE_PCI)"
> 
> really protects us from mapping any of the ranges belonging to a PCI device:
> 
> we scan the device tree and for each node we call map_range_to_domain
> 
> with skip_mapping == false (it is called from map_device_children).
> 
> So, if there is no check then the mapping is performed even for PCI devices
> 
> which we do not want.
> 
> But, yes we can simplify the logic to:
> 
> bool need_mapping = !dt_device_for_passthrough(dev) &&
>                      !mr_data->skip_mapping;
> 
> #ifdef CONFIG_HAS_PCI
>      if ( need_mapping && is_pci_passthrough_enabled() &&
>           (device_get_class(dev) == DEVICE_PCI) )
>          need_mapping = false;
> #endif
> 
> but I see no big profit from it.

Sorry I didn't follow your explanation.

My suggestion is to remove the #ifdef CONFIG_HAS_PCI completely from
map_range_to_domain. At the beginning of map_range_to_domain, there is
already this line:

bool need_mapping = !dt_device_for_passthrough(dev);

We can change it into:

bool need_mapping = !dt_device_for_passthrough(dev) &&
                    !mr_data->skip_mapping;


Then, in map_device_children we can set mr_data->skip_mapping to true
for PCI devices. There is already a pci check there:

 if ( dt_device_type_is_equal(dev, "pci") )

so it should be easy to do. What am I missing?



> >>       if ( need_mapping )
> >>       {
> >>           res = map_regions_p2mt(d,
> >> @@ -1450,7 +1449,11 @@ static int __init map_device_children(struct domain *d,
> >>                                         const struct dt_device_node *dev,
> >>                                         p2m_type_t p2mt)
> >>   {
> >> -    struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
> >> +    struct map_range_data mr_data = {
> >> +        .d = d,
> >> +        .p2mt = p2mt,
> >> +        .map_pci_bridge = false
> >> +    };
> >>       int ret;
> >>   
> >>       if ( dt_device_type_is_equal(dev, "pci") )
> >> @@ -1582,7 +1585,11 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
> >>       /* Give permission and map MMIOs */
> >>       for ( i = 0; i < naddr; i++ )
> >>       {
> >> -        struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
> >> +        struct map_range_data mr_data = {
> >> +            .d = d,
> >> +            .p2mt = p2mt,
> >> +            .map_pci_bridge = false
> >> +        };
> >>           res = dt_device_get_address(dev, i, &addr, &size);
> >>           if ( res )
> >>           {
> >> @@ -2754,7 +2761,14 @@ static int __init construct_dom0(struct domain *d)
> >>           return rc;
> >>   
> >>       if ( acpi_disabled )
> >> +    {
> >>           rc = prepare_dtb_hwdom(d, &kinfo);
> >> +#ifdef CONFIG_HAS_PCI
> >> +        if ( rc < 0 )
> >> +            return rc;
> > This doesn't look great :-)
> >
> > I would move the call to pci_host_bridge_mappings() below just before
> > construct_domain.
> 
> I put it there for purpose: currently we only support device-tree and
> 
> ACPI is not covered, e.g. pci_host_bridge_mappings is implemented
> 
> with device-tree in mind. So, I decided to tie it to prepare_dtb_hwdom
> 
> which is called when acpi_disabled is true.

That's OK, I don't mind. My comment was purely "code aesthetic". I think
this would look better:

    if ( acpi_disabled )
        rc = prepare_dtb_hwdom(d, &kinfo);
    else
        rc = prepare_acpi(d, &kinfo);

    if ( rc < 0 )
        return rc;

#ifdef CONFIG_HAS_PCI
    if ( acpi_disabled )
        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
    if ( rc < 0 )
        return rc;
#endif


Or even this would look better:

    if ( acpi_disabled )
    {
        rc = prepare_dtb_hwdom(d, &kinfo);
        if ( rc < 0 )
            return rc;
#ifdef CONFIG_HAS_PCI
        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
#endif
    } else
        rc = prepare_acpi(d, &kinfo);

    if ( rc < 0 )
        return rc;


Given that this comment is not about functionality but only about how
the code looks like I won't insist.



> >> +        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
> >> +#endif
> >> +    }
> >>       else
> >>           rc = prepare_acpi(d, &kinfo);
> >>   
> >> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> >> index 9b88b1cedaa2..eae177f2cbc2 100644
> >> --- a/xen/arch/arm/pci/ecam.c
> >> +++ b/xen/arch/arm/pci/ecam.c
> >> @@ -39,6 +39,19 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> >>       return base + (PCI_DEVFN2(sbdf) << devfn_shift) + where;
> >>   }
> >>   
> >> +bool pci_ecam_need_p2m_mapping(struct domain *d,
> >> +                               struct pci_host_bridge *bridge,
> >> +                               uint64_t addr)
> >> +{
> >> +    struct pci_config_window *cfg = bridge->cfg;
> >> +
> >> +    /*
> >> +     * We do not want ECAM address space to be mapped in Domain-0's p2m,
> >> +     * so we can trap access to it.
> >> +     */
> >> +    return cfg->phys_addr != addr;
> >> +}
> >> +
> >>   /* ECAM ops */
> >>   const struct pci_ecam_ops pci_generic_ecam_ops = {
> >>       .bus_shift  = 20,
> >> @@ -46,6 +59,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
> >>           .map_bus                = pci_ecam_map_bus,
> >>           .read                   = pci_generic_config_read,
> >>           .write                  = pci_generic_config_write,
> >> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
> >>       }
> >>   };
> >>   
> >> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> >> index 155f2a2743af..f350826ea26b 100644
> >> --- a/xen/arch/arm/pci/pci-host-common.c
> >> +++ b/xen/arch/arm/pci/pci-host-common.c
> >> @@ -18,6 +18,7 @@
> >>   
> >>   #include <xen/init.h>
> >>   #include <xen/pci.h>
> >> +#include <asm/setup.h>
> >>   #include <xen/rwlock.h>
> >>   #include <xen/sched.h>
> >>   #include <xen/vmap.h>
> >> @@ -328,6 +329,53 @@ int pci_host_get_num_bridges(void)
> >>       return count;
> >>   }
> >>   
> >> +int __init pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt)
> >> +{
> >> +    struct pci_host_bridge *bridge;
> >> +    struct map_range_data mr_data = {
> >> +        .d = d,
> >> +        .p2mt = p2mt,
> >> +        .map_pci_bridge = true
> >> +    };
> >> +
> >> +    /*
> >> +     * For each PCI host bridge we need to only map those ranges
> >> +     * which are used by Domain-0 to properly initialize the bridge,
> >> +     * e.g. we do not want to map ECAM configuration space which lives in
> >> +     * "reg" or "assigned-addresses" device tree property.
> >> +     * Neither we want to map any of the MMIO ranges found in the "ranges"
> >> +     * device tree property.
> >> +     */
> >> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> >> +    {
> >> +        const struct dt_device_node *dev = bridge->dt_node;
> >> +        int i;
> > i should be unsigned int
> Ok
> >
> >
> >> +        for ( i = 0; i < dt_number_of_address(dev); i++ )
> >> +        {
> >> +            uint64_t addr, size;
> >> +            int err;
> >> +
> >> +            err = dt_device_get_address(dev, i, &addr, &size);
> >> +            if ( err )
> >> +            {
> >> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> > Maybe rephrase it to:
> >
> >      Unable to retrieve address range index=%u for %s
> This is a copy-paste from the original code, but ok
> >
> >
> >> +                       i, dt_node_full_name(dev));
> >> +                return err;
> >> +            }
> >> +
> >> +            if ( bridge->ops->need_p2m_mapping(d, bridge, addr) )
> > The current implementation of need_p2m_mapping filters out ECAM and
> > nothing else. Just double-checking here: is there anything else we
> > should filter out? Looking at the device tree pci node for ZynqMP:
> >
> > 			reg = <0x0 0xfd0e0000 0x0 0x1000 0x0 0xfd480000 0x0 0x1000 0x80 0x0 0x0 0x1000000>;
> > 			reg-names = "breg", "pcireg", "cfg";
> >
> > We are filtering out cfg, but do we need both "breg" and "pcireg" here?
> 
> It is vice versa: we are filtering out cfg only and all the rest are "unknown regions we do not
> 
> want to alter".

Ah, OK. Can you please add a note about this to the in-code comment?
Maybe as follows:

For each PCI host bridge we need to only map those ranges which are used
by Domain-0 to properly initialize the bridge, e.g. we do not want to
map ECAM configuration space which lives in "reg" or
"assigned-addresses" device tree property, but we want to map other
regions of the host bridge. The PCI aperture defined by the "ranges"
device tree property should also be skipped.


> >
> > If not, do we need another function like .cfg_reg_index to know what we
> > actually need to map?
> 
> I was thinking to use .cfg_reg_index fir that, but it means I'll need to traverse
> 
> the device-tree to get the value for .cfg_reg_index which is already known
> 
> to the bridge. So, it is cheaper to have a callback and just check that
> 
> cfg->phys_addr != addr, e.g. what we want to map is not cfg.

Yeah that makes sense


> >> +            {
> >> +                err = map_range_to_domain(dev, addr, size, &mr_data);
> >> +                if ( err )
> >> +                    return err;
> >> +            }
> >> +        }
> >> +    }
> >> +
> >> +    return 0;
> >> +}
> >> +
> >>   /*
> >>    * Local variables:
> >>    * mode: C
> >> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> >> index c27b4ea9f02f..adbe3627871f 100644
> >> --- a/xen/arch/arm/pci/pci-host-zynqmp.c
> >> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> >> @@ -33,6 +33,7 @@ const struct pci_ecam_ops nwl_pcie_ops = {
> >>           .map_bus                = pci_ecam_map_bus,
> >>           .read                   = pci_generic_config_read,
> >>           .write                  = pci_generic_config_write,
> >> +        .need_p2m_mapping       = pci_ecam_need_p2m_mapping,
> >>       }
> >>   };
> >>   
> >> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> >> index 7618f0b6725b..b81f66e813ef 100644
> >> --- a/xen/include/asm-arm/pci.h
> >> +++ b/xen/include/asm-arm/pci.h
> >> @@ -19,6 +19,8 @@
> >>   
> >>   #ifdef CONFIG_HAS_PCI
> >>   
> >> +#include <asm/p2m.h>
> >> +
> >>   #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> >>   #define PRI_pci "%04x:%02x:%02x.%u"
> >>   
> >> @@ -79,6 +81,9 @@ struct pci_ops {
> >>                   uint32_t reg, uint32_t len, uint32_t *value);
> >>       int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf,
> >>                    uint32_t reg, uint32_t len, uint32_t value);
> >> +    bool (*need_p2m_mapping)(struct domain *d,
> >> +                             struct pci_host_bridge *bridge,
> >> +                             uint64_t addr);
> > I would call this function: need_p2m_hwdom_mapping
> Ok
> >
> >
> >>   };
> >>   
> >>   /*
> >> @@ -102,6 +107,9 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf,
> >>                               uint32_t reg, uint32_t len, uint32_t value);
> >>   void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> >>                                  uint32_t sbdf, uint32_t where);
> >> +bool pci_ecam_need_p2m_mapping(struct domain *d,
> >> +                               struct pci_host_bridge *bridge,
> >> +                               uint64_t addr);
> >>   struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
> >>   int pci_get_host_bridge_segment(const struct dt_device_node *node,
> >>                                   uint16_t *segment);
> >> @@ -116,6 +124,7 @@ int pci_host_iterate_bridges(struct domain *d,
> >>                                int (*clb)(struct domain *d,
> >>                                           struct pci_host_bridge *bridge));
> >>   int pci_host_get_num_bridges(void);
> >> +int pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt);
> >>   #else   /*!CONFIG_HAS_PCI*/
> >>   
> >>   #define pci_passthrough_enabled (false)
> >> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> >> index 95da0b7ab9cd..21863dd2bc58 100644
> >> --- a/xen/include/asm-arm/setup.h
> >> +++ b/xen/include/asm-arm/setup.h
> >> @@ -2,6 +2,8 @@
> >>   #define __ARM_SETUP_H_
> >>   
> >>   #include <public/version.h>
> >> +#include <asm/p2m.h>
> >> +#include <xen/device_tree.h>
> >>   
> >>   #define MIN_FDT_ALIGN 8
> >>   #define MAX_FDT_SIZE SZ_2M
> >> @@ -77,6 +79,14 @@ struct bootinfo {
> >>   #endif
> >>   };
> >>   
> >> +struct map_range_data
> >> +{
> >> +    struct domain *d;
> >> +    p2m_type_t p2mt;
> >> +    /* Set if mappings for PCI host bridges must not be skipped. */
> >> +    bool map_pci_bridge;
> > To make this more generally applicable, I would call the new property:
> >
> >    bool skip_mapping;
> Sounds good
> >
> > and it could apply to any class of devices. All current users would set
> > it to false except for pci_host_bridge_mappings.
> Please see PCI special case above
> >
> >
> >> +};
> >>
> >>   extern struct bootinfo bootinfo;
> >>   
> >>   extern domid_t max_init_domid;
> >> @@ -124,6 +134,9 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
> >>   u32 device_tree_get_u32(const void *fdt, int node,
> >>                           const char *prop_name, u32 dflt);
> >>   
> >> +int map_range_to_domain(const struct dt_device_node *dev,
> >> +                        u64 addr, u64 len, void *data);
> >> +
> >>   #endif
> >>   /*
> >>    * Local variables:
> >> -- 
> >> 2.25.1
> >>
--8323329-1264100615-1632801182=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:16:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:16:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197391.350415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4Xy-0000Mb-6R; Tue, 28 Sep 2021 04:16:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197391.350415; Tue, 28 Sep 2021 04:16:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4Xy-0000MU-31; Tue, 28 Sep 2021 04:16:50 +0000
Received: by outflank-mailman (input) for mailman id 197391;
 Tue, 28 Sep 2021 04:16:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0YX1=OS=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mV4Xw-0000MO-Mh
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:16:48 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.52]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 400d8a54-663b-4e4f-84a2-4d28e2405adf;
 Tue, 28 Sep 2021 04:16:44 +0000 (UTC)
Received: from AM6PR05CA0009.eurprd05.prod.outlook.com (2603:10a6:20b:2e::22)
 by VI1PR0801MB1951.eurprd08.prod.outlook.com (2603:10a6:800:8f::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 04:16:42 +0000
Received: from AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:2e:cafe::e8) by AM6PR05CA0009.outlook.office365.com
 (2603:10a6:20b:2e::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Tue, 28 Sep 2021 04:16:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT014.mail.protection.outlook.com (10.152.16.130) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 04:16:41 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Tue, 28 Sep 2021 04:16:40 +0000
Received: from f9d5b6d21a5f.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8DECE90E-85D4-4B05-A80E-47FB332C6C34.1; 
 Tue, 28 Sep 2021 04:16:30 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f9d5b6d21a5f.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 04:16:30 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DBBPR08MB5898.eurprd08.prod.outlook.com (2603:10a6:10:20c::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 04:16:28 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 04:16:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 400d8a54-663b-4e4f-84a2-4d28e2405adf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=htSWZT2gCJUbwc6XxBwv9NpNR3Vu/M5ZCRmtcB9KuFQ=;
 b=S8wze3Y7XGqJ64TbNtv4n1Vz1K4bzC/W182+IRH6v4ToSDRtUl1YDRN3DhAMuDDyxvxErJ1Q1AoOsEZPe77SHV5qPJhPRHT/Op2Nqnsg1S9HSL3dPKR3BtRcu1YIPd25OGwauFJMGH5ZoUmTcdECkZIohYAi5TqjxZNMoj4XoDw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HONvC8NJ4GjvT74QHmGkSxthEXqk4o9cIGBhbp8tVGjaBb+nqEmIe8OPIqsmJVmFBvHAUV4i7fy+xHTVZ/xNU7643FbzgJq8lvPqxfs/df8F7YwpSLJ2gxC5K2QEVBy71wIuLp7vqkZbODgdlLkO4ynYsv5aIjs1mwG4P7LqrDsaz9ZOiLvQJHDI0AgM3O+zy2+8bFdhELhY1CDqrQKb+rKuVKpD028MyYCGf0GxTxsdlBVhF8vYmw07k/e03/BR8O/UgdLmF/3jkLofUG2N+0RfQgwVESCjtIxLKG23G2RCMPjGlNZXMpK2UGg/wdQs0mTCr+X/jbDafvrSzbprxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=htSWZT2gCJUbwc6XxBwv9NpNR3Vu/M5ZCRmtcB9KuFQ=;
 b=KXizyDTjYJ4xJH228zUqcZ1WYt5aCCz5pmyASV6IWGuSt/p8+/kDY9lJXeH5rPObcuD+swR6s64JZPEGoWYGuY+uKEtRnPNVef6AvZzjUk37bNAkrS5ezuJOCRdtGkFNLfLVq3Q9YnnVRAO+e3DKx7YeCDfysG8vL/ybYg1l3U55xUcULRMl+hbA6SXnI10I3R4xjk4xvSoiJyOCnAHtHwEtGsXIDy91BGwv3MmWeXByKEDejqkciUiY4ElAkap3NmrevyIm5M6OGpCKcWVL6OZiyrNSY2zhQmQ5VQBRSHier3VcVf1Kf2yg0LRT+w1I/nHahxqQqUGR7HOrB7f9Hw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=htSWZT2gCJUbwc6XxBwv9NpNR3Vu/M5ZCRmtcB9KuFQ=;
 b=S8wze3Y7XGqJ64TbNtv4n1Vz1K4bzC/W182+IRH6v4ToSDRtUl1YDRN3DhAMuDDyxvxErJ1Q1AoOsEZPe77SHV5qPJhPRHT/Op2Nqnsg1S9HSL3dPKR3BtRcu1YIPd25OGwauFJMGH5ZoUmTcdECkZIohYAi5TqjxZNMoj4XoDw=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Jan Beulich <jbeulich@suse.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, "julien@xen.org" <julien@xen.org>, Bertrand
 Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Topic: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
Thread-Index:
 AQHXsHMtR+l18+MmpE6rl/RjSXTBgquyYmkAgAA2xQCAADnbAIAAKWhQgAAGUwCAAxtPwIABk+MwgAD1aACAACGFAA==
Date: Tue, 28 Sep 2021 04:16:28 +0000
Message-ID:
 <DB9PR08MB6857E57847E38C400F6571EA9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
 <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109271420560.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109271420560.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 43DA564848AA594593DFCAD0E1C69A63.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 5eaec923-77d5-4d74-2421-08d98236c64b
x-ms-traffictypediagnostic: DBBPR08MB5898:|VI1PR0801MB1951:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0801MB19511444B2F7F1700E0253629EA89@VI1PR0801MB1951.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YTJT0FJ3mLPsvmRH0BArZ4g3ZmsNRUmUj4NjmP3mgMqsSuP+QP3bIQiHKPlg0W+NLXphKSfX6MeQRHmc0KnyHbj4v6oDrHm2cvY3/LuABtsU1KJugIFnUh1b71JefFceYW7rugng9LkG97OYAhqs3MNqvDPjjcdODS3GolJMeQBcAwfEzgYJu+LOx1RTdg3xpBkSPuNwCLEGE4gQGRex5iCX2inEuBOwEeO3naHt0KgzZXWcAeUBED01nM9Y7E6LczpR68qqRHK6Jo4K28uWejSra5tEEacf5Ntp5gd9aGP4Mtbo0LAOX1/NqaERSjSoy6u0CxXsyNvZWh7YMeQ9gXTIZIIeBtkxrIQPHZ1RyWb+hSgpFzuSLkUMF41JRDjVB6pMQl2b8CfUw0G4F/qmDPbuG2/XIGe9rMA44V6RZIno6CFJH2Zofw2PxituEZF7bsNDWpw6CKj27L4owmWnPSjKA0J+aofItd1ByTgzUIdXEHL/MTNw+0Unvd3gIaAbjopTmlIkg8V2xj3CmZHXNF1WxfUEBAHRPQTGYuFNG1c4Qh0XG0ZxBSk5qPJL3rep5vZKV/Of1TdnikKI4/dzjg9xvuWp8U7oCvbDc2azxIUH2u1ud0M2kwuatddWLoTx5sCfhGq8VXTpREHBEKAPa1YKKAANDPNEJOF4yxDB0AgczQKrawVR1Tk7m5BJOXm38pdZvFEEPdXuyHL7NSXnH2PvBfIA7IuVYSSj+nejuNiN4j3j6d+X0Choh2w5RzKVAXA49Ddv6OU5Zu2h19ZeV4Nj8laZCMPseXcLgG2d63w=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(38100700002)(8676002)(86362001)(122000001)(316002)(55016002)(26005)(38070700005)(83380400001)(966005)(186003)(33656002)(66446008)(53546011)(7696005)(66946007)(76116006)(6916009)(52536014)(5660300002)(54906003)(66556008)(64756008)(508600001)(8936002)(9686003)(66476007)(6506007)(71200400001)(2906002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5898
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	ae9ecb16-ff66-4a39-ab9d-08d98236be42
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qdklKgi6iwJhdwWikmy05FVe738PpIe8dyvxYtfpKtAj6MFq71IRYIlexZ/e8un2UjCpzTBku1WQ+2ZS5Dcer5qeVQ5t4fDyqt4RQQ/MNAX9ZfRTkBtdjm8skJAv20g19sHyyMXZQTe9xh1jcD1vreSMBBrYTn8uWL+tbJDM3BLgn5MI2blIEgLmCGva+8n3irmeg+nTBSEY2135Fkxu4SIWqK3hGyKgmplJVDBhdNnMPYWZ1+hvYK///bKTh2sJr+cjqFncv36EsYRz8CBsrPvIIouye+jUBu5fkQZOUtziQqXRU1PBCEc/gqAbrcLuc1+38ZGUbVbgiR7T9Sn0BNavBrq0oc0WDt3/bpvxLQUeXmAdOyUqPSl7QTk70E1AVtcHzp0JNBEzXCwl4UqU4dBzNUGIXb/6OCyBN4/FA917iZVS3H9lG49TnAOqGFVrLM45sZmozPqZJaGGYjeZsbW+DXqaQJ+bGmui0EGygkwDhEAiJcM1swz6PKfO+aIDVW1XSrW1rql7VfPvjNlh8pFl5+jQPDXii+vL2BDLX6dk8snIyDahjwBrWRZhdb1ScSKsnnH+TTVhpMGunf1fOOYi6g+pvhp32PaKf7J7MoD67qY8G17r33WdTNdEUTzh8mjHUx1iZxy/WdjujnFUhlwlEE9ClzT2YfrrWn88FZA342rEMcHjfH5PturHcxpGS4p0KaOsHjphEC7dZ4/3Vr5g+aMLcX6eXHxN45c8w+RWjB33KxlYA668PKD/nNEGXA4uGzgWNeDRM2usbIbnTHGhCoXuRiL4AO0pO+8p15g=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(82310400003)(47076005)(186003)(53546011)(26005)(6506007)(508600001)(966005)(336012)(5660300002)(8936002)(4326008)(33656002)(6862004)(316002)(70206006)(356005)(81166007)(83380400001)(54906003)(8676002)(36860700001)(86362001)(70586007)(9686003)(2906002)(7696005)(52536014)(55016002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 04:16:41.5534
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5eaec923-77d5-4d74-2421-08d98236c64b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT014.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1951

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMjjml6UgOTowMA0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBK
YW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+OyB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVj
dC5vcmc7DQo+IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJx
dWlzQGFybS5jb20+OyBTdGVmYW5vDQo+IFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5v
cmc+DQo+IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMjAvMzddIHhlbjogaW50cm9kdWNlIENPTkZJR19F
RkkgdG8gc3R1YiBBUEkgZm9yIG5vbi0NCj4gRUZJIGFyY2hpdGVjdHVyZQ0KPiANCj4gT24gTW9u
LCAyNyBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3Nh
Z2UtLS0tLQ0KPiA+ID4gRnJvbTogWGVuLWRldmVsIDx4ZW4tZGV2ZWwtYm91bmNlc0BsaXN0cy54
ZW5wcm9qZWN0Lm9yZz4gT24gQmVoYWxmIE9mDQo+IFdlaQ0KPiA+ID4gQ2hlbg0KPiA+ID4gU2Vu
dDogMjAyMeW5tDnmnIgyNuaXpSAxODoyNQ0KPiA+ID4gVG86IEphbiBCZXVsaWNoIDxqYmV1bGlj
aEBzdXNlLmNvbT4NCj4gPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IGp1
bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4gPiA8QmVydHJhbmQuTWFycXVpc0Bh
cm0uY29tPjsgU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+
ID4gU3ViamVjdDogUkU6IFtQQVRDSCAyMC8zN10geGVuOiBpbnRyb2R1Y2UgQ09ORklHX0VGSSB0
byBzdHViIEFQSSBmb3INCj4gbm9uLQ0KPiA+ID4gRUZJIGFyY2hpdGVjdHVyZQ0KPiA+ID4NCj4g
PiA+IEhpIEphbiwNCj4gPiA+DQo+ID4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+
ID4gPiA+IEZyb206IFhlbi1kZXZlbCA8eGVuLWRldmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVj
dC5vcmc+IE9uIEJlaGFsZg0KPiBPZg0KPiA+ID4gSmFuDQo+ID4gPiA+IEJldWxpY2gNCj4gPiA+
ID4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSAxODo0OQ0KPiA+ID4gPiBUbzogV2VpIENoZW4gPFdl
aS5DaGVuQGFybS5jb20+DQo+ID4gPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5v
cmc7IGp1bGllbkB4ZW4ub3JnOyBCZXJ0cmFuZCBNYXJxdWlzDQo+ID4gPiA+IDxCZXJ0cmFuZC5N
YXJxdWlzQGFybS5jb20+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4gPHNzdGFiZWxsaW5pQGtlcm5l
bC5vcmc+DQo+ID4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMjAvMzddIHhlbjogaW50cm9kdWNl
IENPTkZJR19FRkkgdG8gc3R1YiBBUEkgZm9yDQo+ID4gPiBub24tDQo+ID4gPiA+IEVGSSBhcmNo
aXRlY3R1cmUNCj4gPiA+ID4NCj4gPiA+ID4gT24gMjQuMDkuMjAyMSAxMjozMSwgV2VpIENoZW4g
d3JvdGU6DQo+ID4gPiA+ID4+IEZyb206IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4N
Cj4gPiA+ID4gPj4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSAxNTo1OQ0KPiA+ID4gPiA+Pg0KPiA+
ID4gPiA+PiBPbiAyNC4wOS4yMDIxIDA2OjM0LCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPj4+
PiBGcm9tOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4g
PiA+ID4+Pj4gU2VudDogMjAyMeW5tDnmnIgyNOaXpSA5OjE1DQo+ID4gPiA+ID4+Pj4NCj4gPiA+
ID4gPj4+PiBPbiBUaHUsIDIzIFNlcCAyMDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPj4+
Pj4gLS0tIGEveGVuL2NvbW1vbi9LY29uZmlnDQo+ID4gPiA+ID4+Pj4+ICsrKyBiL3hlbi9jb21t
b24vS2NvbmZpZw0KPiA+ID4gPiA+Pj4+PiBAQCAtMTEsNiArMTEsMTYgQEAgY29uZmlnIENPTVBB
VA0KPiA+ID4gPiA+Pj4+PiAgY29uZmlnIENPUkVfUEFSS0lORw0KPiA+ID4gPiA+Pj4+PiAgCWJv
b2wNCj4gPiA+ID4gPj4+Pj4NCj4gPiA+ID4gPj4+Pj4gK2NvbmZpZyBFRkkNCj4gPiA+ID4gPj4+
Pj4gKwlib29sDQo+ID4gPiA+ID4+Pj4NCj4gPiA+ID4gPj4+PiBXaXRob3V0IHRoZSB0aXRsZSB0
aGUgb3B0aW9uIGlzIG5vdCB1c2VyLXNlbGVjdGFibGUgKG9yIGRlLQ0KPiA+ID4gPiBzZWxlY3Rh
YmxlKS4NCj4gPiA+ID4gPj4+PiBTbyB0aGUgaGVscCBtZXNzYWdlIGJlbG93IGNhbiBuZXZlciBi
ZSBzZWVuLg0KPiA+ID4gPiA+Pj4+DQo+ID4gPiA+ID4+Pj4gRWl0aGVyIGFkZCBhIHRpdGxlLCBl
LmcuOg0KPiA+ID4gPiA+Pj4+DQo+ID4gPiA+ID4+Pj4gYm9vbCAiRUZJIHN1cHBvcnQiDQo+ID4g
PiA+ID4+Pj4NCj4gPiA+ID4gPj4+PiBPciBmdWxseSBtYWtlIHRoZSBvcHRpb24gYSBzaWxlbnQg
b3B0aW9uIGJ5IHJlbW92aW5nIHRoZSBoZWxwDQo+IHRleHQuDQo+ID4gPiA+ID4+Pg0KPiA+ID4g
PiA+Pj4gT0ssIGluIGN1cnJlbnQgWGVuIGNvZGUsIEVGSSBpcyB1bmNvbmRpdGlvbmFsbHkgY29t
cGlsZWQuIEJlZm9yZQ0KPiA+ID4gPiA+Pj4gd2UgY2hhbmdlIHJlbGF0ZWQgY29kZSwgSSBwcmVm
ZXIgdG8gcmVtb3ZlIHRoZSBoZWxwIHRleHQuDQo+ID4gPiA+ID4+DQo+ID4gPiA+ID4+IEJ1dCB0
aGF0J3Mgbm90IHRydWU6IEF0IGxlYXN0IG9uIHg4NiBFRkkgZ2V0cyBjb21waWxlZCBkZXBlbmRp
bmcNCj4gb24NCj4gPiA+ID4gPj4gdG9vbCBjaGFpbiBjYXBhYmlsaXRpZXMuIFVsdGltYXRlbHkg
d2UgbWF5IGluZGVlZCB3YW50IGEgdXNlcg0KPiA+ID4gPiA+PiBzZWxlY3RhYmxlIG9wdGlvbiBo
ZXJlLCBidXQgdW50aWwgdGhlbiBJJ20gYWZyYWlkIGhhdmluZyB0aGlzDQo+IG9wdGlvbg0KPiA+
ID4gPiA+PiBhdCBhbGwgbWF5IGJlIG1pc2xlYWRpbmcgb24geDg2Lg0KPiA+ID4gPiA+Pg0KPiA+
ID4gPiA+DQo+ID4gPiA+ID4gSSBjaGVjayB0aGUgYnVpbGQgc2NyaXB0cywgeWVzLCB5b3UncmUg
cmlnaHQuIEZvciB4ODYsIEVGSSBpcyBub3QNCj4gYQ0KPiA+ID4gPiA+IHNlbGVjdGFibGUgb3B0
aW9uIGluIEtjb25maWcuIEkgYWdyZWUgd2l0aCB5b3UsIHdlIGNhbid0IHVzZQ0KPiBLY29uZmln
DQo+ID4gPiA+ID4gc3lzdGVtIHRvIGRlY2lkZSB0byBlbmFibGUgRUZJIGJ1aWxkIGZvciB4ODYg
b3Igbm90Lg0KPiA+ID4gPiA+DQo+ID4gPiA+ID4gU28gaG93IGFib3V0IHdlIGp1c3QgdXNlIHRo
aXMgRUZJIG9wdGlvbiBmb3IgQXJtIG9ubHk/IEJlY2F1c2Ugb24NCj4gQXJtLA0KPiA+ID4gPiA+
IHdlIGRvIG5vdCBoYXZlIHN1Y2ggdG9vbGNoYWluIGRlcGVuZGVuY3kuDQo+ID4gPiA+DQo+ID4g
PiA+IFRvIGJlIGhvbmVzdCAtIGRvbid0IGtub3cuIFRoYXQncyBiZWNhdXNlIEkgZG9uJ3Qga25v
dyB3aGF0IHlvdSB3YW50DQo+ID4gPiA+IHRvIHVzZSB0aGUgb3B0aW9uIGZvciBzdWJzZXF1ZW50
bHkuDQo+ID4gPiA+DQo+ID4gPg0KPiA+ID4gSW4gbGFzdCB2ZXJzaW9uLCBJIGhhZCBpbnRyb2R1
Y2VkIGFuIGFyY2gtaGVscGVyIHRvIHN0dWIgRUZJX0JPT1QNCj4gPiA+IGluIEFybSdzIGNvbW1v
biBjb2RlIGZvciBBcm0zMi4gQmVjYXVzZSBBcm0zMiBkb2Vzbid0IHN1cHBvcnQgRUZJLg0KPiA+
ID4gU28gSnVsaWVuIHN1Z2dlc3RlZCBtZSB0byBpbnRyb2R1Y2UgYSBDT05GSUdfRUZJIG9wdGlv
biBmb3Igbm9uLUVGSQ0KPiA+ID4gc3VwcG9ydGVkIGFyY2hpdGVjdHVyZXMgdG8gc3R1YiBpbiBF
RkkgbGF5ZXIuDQo+ID4gPg0KPiA+ID4gWzFdIGh0dHBzOi8vbGlzdHMueGVucHJvamVjdC5vcmcv
YXJjaGl2ZXMvaHRtbC94ZW4tZGV2ZWwvMjAyMS0NCj4gPiA+IDA4L21zZzAwODA4Lmh0bWwNCj4g
PiA+DQo+ID4NCj4gPiBBcyBKYW4nIHJlbWluZGVkLCB4ODYgZG9lc24ndCBkZXBlbmQgb24gS2Nv
bmZpZyB0byBidWlsZCBFRkkgY29kZS4NCj4gPiBTbywgaWYgd2UgQ09ORklHX0VGSSB0byBzdHVi
IEVGSSBBUEkncyBmb3IgeDg2LCB3ZSB3aWxsIGVuY291bnRlcg0KPiA+IHRoYXQgdG9vbGNoYWlu
cyBlbmFibGUgRUZJLCBidXQgS2NvbmZpZyBkaXNhYmxlIEVGSS4gT3IgS2NvbmZpZw0KPiA+IGVu
YWJsZSBFRkkgYnV0IHRvb2xjaGFpbiBkb2Vzbid0IHByb3ZpZGUgRUZJIGJ1aWxkIHN1cHBvcnRz
LiBBbmQNCj4gPiB0aGVuIHg4NiBjb3VsZCBub3Qgd29yayB3ZWxsLg0KPiA+DQo+ID4gSWYgd2Ug
dXNlIENPTkZJR19FRkkgZm9yIEFybSBvbmx5LCB0aGF0IG1lYW5zIENPTkZJR19FRkkgZm9yIHg4
Ng0KPiA+IGlzIG9mZiwgdGhpcyB3aWxsIGFsc28gY2F1c2UgcHJvYmxlbS4NCj4gPg0KPiA+IFNv
LCBjYW4gd2Ugc3RpbGwgdXNlIHByZXZpb3VzIGFyY2hfaGVscGVycyB0byBzdHViIGZvciBBcm0z
Mj8NCj4gPiB1bnRpbCB4ODYgY2FuIHVzZSB0aGlzIHNlbGVjdGFibGUgb3B0aW9uPw0KPiANCj4g
RUZJIGRvZXNuJ3QgaGF2ZSB0byBiZSBuZWNlc3NhcmlseSBhIHVzZXItdmlzaWJsZSBvcHRpb24g
aW4gS2NvbmZpZyBhdA0KPiB0aGlzIHBvaW50LiBJIHRoaW5rIEp1bGllbiB3YXMganVzdCBhc2tp
bmcgdG8gbWFrZSB0aGUgI2lmZGVmIGJhc2VkIG9uDQo+IGEgRUZJLXJlbGF0ZWQgY29uZmlnIHJh
dGhlciB0aGFuIGp1c3QgYmFzZWQgQ09ORklHX0FSTTY0Lg0KPiANCj4gT24geDg2IEVGSSBpcyBk
ZXRlY3RlZCBiYXNlZCBvbiBjb21waWxlciBzdXBwb3J0LCBzZXR0aW5nIFhFTl9CVUlMRF9FRkkN
Cj4gaW4geGVuL2FyY2gveDg2L01ha2VmaWxlLiBMZXQncyBzYXkgdGhhdCB3ZSBrZWVwIHVzaW5n
IHRoZSBzYW1lIG5hbWUNCj4gIlhFTl9CVUlMRF9FRkkiIG9uIEFSTSBhcyB3ZWxsLg0KPiANCj4g
T24gQVJNMzIsIFhFTl9CVUlMRF9FRkkgc2hvdWxkIGJlIGFsd2F5cyB1bnNldC4NCj4gDQo+IE9u
IEFSTTY0IFhFTl9CVUlMRF9FRkkgc2hvdWxkIGJlIGFsd2F5cyBzZXQuDQo+IA0KPiBUaGF0J3Mg
aXQsIHJpZ2h0PyBJJ2QgYXJndWUgdGhhdCBDT05GSUdfRUZJIG9yIEhBU19FRkkgYXJlIGJldHRl
ciBuYW1lcw0KPiB0aGFuIFhFTl9CVUlMRF9FRkksIGJ1dCB0aGF0J3MgT0sgYW55d2F5LiBTbyBm
b3IgaW5zdGFuY2UgeW91IGNhbiBtYWtlDQo+IFhFTl9CVUlMRF9FRkkgYW4gaW52aXNpYmxlIHN5
bWJvbCBpbiB4ZW4vYXJjaC9hcm0vS2NvbmZpZyBhbmQgc2VsZWN0IGl0DQo+IG9ubHkgb24gQVJN
NjQuDQoNClRoYW5rcywgdGhpcyBpcyBhIGdvb2QgYXBwcm9hY2guIEJ1dCBpZiB3ZSBwbGFjZSBY
RU5fQlVJTERfRUZJIGluIEtjb25maWcNCml0IHdpbGwgYmUgdHJhbnNmZXIgdG8gQ09ORklHX1hF
Tl9CVUlMRF9FRkkuIEhvdyBhYm91dCB1c2luZyBhbm90aGVyIG5hbWUNCmluIEtjb25maWcgbGlr
ZSBBUk1fRUZJLCBidXQgdXNlIENPTkZJR19BUk1fRUZJIGluIGNvbmZpZy5oIHRvIGRlZmluZQ0K
WEVOX0JVSUxEX0VGST8NCg0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:18:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:18:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197397.350426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4ZH-00015H-HN; Tue, 28 Sep 2021 04:18:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197397.350426; Tue, 28 Sep 2021 04:18:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4ZH-00015A-Di; Tue, 28 Sep 2021 04:18:11 +0000
Received: by outflank-mailman (input) for mailman id 197397;
 Tue, 28 Sep 2021 04:18:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV4ZF-00014r-UD
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:18:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1464b718-2013-11ec-bc64-12813bfff9fa;
 Tue, 28 Sep 2021 04:18:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A102A60FE3;
 Tue, 28 Sep 2021 04:18:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1464b718-2013-11ec-bc64-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632802685;
	bh=yBitKBKAgLE4ulPlX3HMhKhMpQBqvjuOFFQmaCDT27E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ZieJGm77q2ZvHLTHIlXZGwAVukchy3yejCqF3WMo5XFGGCxGIjbRnWBMwIjGSoDIE
	 7zBgCCSaROEieFZC9PMVVbFQSpoipWxxMwyX5lJ4LYCwNCkfX5sfi/OFMFkD7oWfB0
	 uYrF9N8Y+5cJGORwctit8HXbG2ZI8/2CqbDhdcCAErrajFtNsCYjG+m09O1hpTYcns
	 2dpzZDWH43Qyy8AgPSbCFhitvn32pN0nS5NNZgQgFRs5jV2n24s8uhCmcWz30TKix6
	 eCYu42ghyYZ2ja47EwCMs8jGg/d1fbsW+zHYazxzpBOYIKkIGDP6nbk2qpVq3ClN7e
	 smWR0a5Wa25Hw==
Date: Mon, 27 Sep 2021 21:18:05 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Juergen Gross <jgross@suse.com>
cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>, 
    Jan Beulich <jbeulich@suse.com>, 
    Oleksandr Andrushchenko <andr2000@gmail.com>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, 
    "sstabellini@kernel.org" <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
In-Reply-To: <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
Message-ID: <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
References: <20210927065822.350973-1-andr2000@gmail.com> <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com> <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com> <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 Sep 2021, Juergen Gross wrote:
> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
> > 
> > On 27.09.21 10:26, Jan Beulich wrote:
> > > On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
> > > > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > > > 
> > > > Currently PCI backend implements multiple functionalities at a time.
> > > > To name a few:
> > > > 1. It is used as a database for assignable PCI devices, e.g. xl
> > > >      pci-assignable-{add|remove|list} manipulates that list. So,
> > > > whenever
> > > >      the toolstack needs to know which PCI devices can be passed through
> > > >      it reads that from the relevant sysfs entries of the pciback.
> > > > 2. It is used to hold the unbound PCI devices list, e.g. when passing
> > > >      through a PCI device it needs to be unbound from the relevant
> > > > device
> > > >      driver and bound to pciback (strictly speaking it is not required
> > > >      that the device is bound to pciback, but pciback is again used as a
> > > >      database of the passed through PCI devices, so we can re-bind the
> > > >      devices back to their original drivers when guest domain shuts
> > > > down)
> > > > 3. Device reset for the devices being passed through
> > > > 4. Para-virtualised use-cases support
> > > > 
> > > > The para-virtualised part of the driver is not always needed as some
> > > > architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
> > > > model for PCI device passthrough. For such use-cases make the very
> > > > first step in splitting the xen-pciback driver into two parts: Xen
> > > > PCI stub and PCI PV backend drivers.
> > > > 
> > > > Signed-off-by: Oleksandr Andrushchenko
> > > > <oleksandr_andrushchenko@epam.com>
> > > > 
> > > > ---
> > > > Changes since v3:
> > > > - Move CONFIG_XEN_PCIDEV_STUB to the second patch
> > > I'm afraid this wasn't fully done:
> > > 
> > > > --- a/drivers/xen/xen-pciback/Makefile
> > > > +++ b/drivers/xen/xen-pciback/Makefile
> > > > @@ -1,5 +1,6 @@
> > > >    # SPDX-License-Identifier: GPL-2.0
> > > >    obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> > > > +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> > > While benign when added here, this addition still doesn't seem to
> > > belong here.
> > 
> > My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seems
> > 
> > to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build on 32-bit
> > 
> > architectures...
> > 
> > What would be the preference here? Stefano suggested that we still define
> > 
> > CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate to it
> > 
> > in the second patch
> > 
> > Another option is just to squash the two patches.
> 
> Squashing would be fine for me.
 
It is fine for me to squash the two patches.

But in any case, wouldn't it be better to modify that specific change to:

diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
index e2cb376444a6..e23c758b85ae 100644
--- a/drivers/xen/xen-pciback/Makefile
+++ b/drivers/xen/xen-pciback/Makefile
@@ -1,6 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
-obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
+obj-$(CONFIG_XEN_PCI_STUB) += xen-pciback.o
 
 xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
 xen-pciback-y += conf_space.o conf_space_header.o \


instead?


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:41:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:41:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197410.350437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4w2-0004T0-E1; Tue, 28 Sep 2021 04:41:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197410.350437; Tue, 28 Sep 2021 04:41:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV4w2-0004St-9N; Tue, 28 Sep 2021 04:41:42 +0000
Received: by outflank-mailman (input) for mailman id 197410;
 Tue, 28 Sep 2021 04:41:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=0YX1=OS=arm.com=Wei.Chen@srs-us1.protection.inumbo.net>)
 id 1mV4w1-0004Sn-BH
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:41:41 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.15.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a35a0ccf-06ee-4082-b745-d36fb7dd3d7f;
 Tue, 28 Sep 2021 04:41:37 +0000 (UTC)
Received: from DB6PR07CA0193.eurprd07.prod.outlook.com (2603:10a6:6:42::23) by
 AM0PR08MB4210.eurprd08.prod.outlook.com (2603:10a6:208:108::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep
 2021 04:41:35 +0000
Received: from DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:42:cafe::9e) by DB6PR07CA0193.outlook.office365.com
 (2603:10a6:6:42::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.7 via Frontend
 Transport; Tue, 28 Sep 2021 04:41:35 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT019.mail.protection.outlook.com (10.152.20.163) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 04:41:35 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Tue, 28 Sep 2021 04:41:34 +0000
Received: from c8bea324f26d.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 984DF8DC-CFCB-47CE-ADE2-4C4E3440EDED.1; 
 Tue, 28 Sep 2021 04:41:24 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c8bea324f26d.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 04:41:24 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com (2603:10a6:10:2a2::7)
 by DB6PR0801MB2040.eurprd08.prod.outlook.com (2603:10a6:4:75::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep
 2021 04:41:22 +0000
Received: from DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32]) by DB9PR08MB6857.eurprd08.prod.outlook.com
 ([fe80::2c28:50cf:49fd:da32%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 04:41:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a35a0ccf-06ee-4082-b745-d36fb7dd3d7f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zRhvPQCszJ4shTAAKhrpW7TQioWNu6P6FIit3FiR7r4=;
 b=WOlSaLl/qAvCwAw241R22gR0KWREco115FVNCoCH7HbQUG+7yJ76iqkU6Su+Cdwg+RcaIff1PL2Yvf170ANfPQy5XztXhaZ6J4cysQLIs6Pq/cUbvRoHIh9EZMrZZHzmhegURCTbCnc+azm7WfLC+izGmQRnqnBGXkD8ZseHYbA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cVcshjvslf1sbKxitwFbQGe7BpmMrICOcIGLDYacOfnF1rx7+kBE6Bf7LJU50N8FZF1qTn2XqnA0xRfRovc2bTXIsZ+LZxhJqUjsiby9vkvNrmYPSe/ZaDwd1SgCGH9cJoy6lNGxQ1dMi6T/susP1wWN1LW0rCVnGfTy50VPql1hY8f/8o/gJQSl6WYT6wGSvO84zl++/O+qXRjlSL5ZkVU1dhsKRosIPMi9zR5ELUN/H4c9clwCaG/uiQOB4mnuP0L+UEsl/iMBPPy1X2wjVIWO/ehMQ0g5i98e3w6xGU4ebFX0OY+eHhtMNZhVwtg+4PZwkZh4ZJ8OrnQy36zPaA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zRhvPQCszJ4shTAAKhrpW7TQioWNu6P6FIit3FiR7r4=;
 b=NLdBnQjdmInxUi6xAgPJcKV20B2ojvBvIa0HnXQBDSzhzjOFaMbWGyTDG3T67CGOtUOaLDgCcvqskp3NIxirtu8ZD5/t2AjTjzNiQjpF8/htgUkVSFK0sWMKUjhzt/ft3i9jmIE7c1f5VnpGKGFp0C5dMDoH60RjYuehbYp30/l4ub4DUpLXJaobO42hVcDGCi3/P0obI9EqBk0EU6jkGpEoqA69rfg4dYEH1xk+PDWRaODrfTwPGgDWYJlxq1nA4id06Xub8AcpsK9j9qiAXsFAh6mHD/7/aOe7WC+hHi0CSkH/z4kK6r+B2+hBjWUzemFXqt45eYiEi03NSJJ+YA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zRhvPQCszJ4shTAAKhrpW7TQioWNu6P6FIit3FiR7r4=;
 b=WOlSaLl/qAvCwAw241R22gR0KWREco115FVNCoCH7HbQUG+7yJ76iqkU6Su+Cdwg+RcaIff1PL2Yvf170ANfPQy5XztXhaZ6J4cysQLIs6Pq/cUbvRoHIh9EZMrZZHzmhegURCTbCnc+azm7WfLC+izGmQRnqnBGXkD8ZseHYbA=
From: Wei Chen <Wei.Chen@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"julien@xen.org" <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, "jbeulich@suse.com" <jbeulich@suse.com>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Topic: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
Thread-Index:
 AQHXsHMRZjZ8eZQrbkW8SEq2x0f00KuyVH0AgABDnECAAQJ+AIACfOAggAEi1ICAAB9HAIAAIXKwgACruYCAAL1BwA==
Date: Tue, 28 Sep 2021 04:41:21 +0000
Message-ID:
 <DB9PR08MB68573F34B7B9A73B976D4E4C9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-9-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s>
 <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685772C5CDE9DF885A063F479EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109271018220.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109271018220.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 0784BF3631E5D5439C5FFAAE0B28C835.0
x-checkrecipientchecked: true
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: d49e2d1f-843d-4604-6516-08d9823a4079
x-ms-traffictypediagnostic: DB6PR0801MB2040:|AM0PR08MB4210:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB4210CB29D44601809DD2F4989EA89@AM0PR08MB4210.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 qr2qZNHb9SituD42MXsz3/JRNgt9ghCuwjrBo/f53uBWi2+zRYLFrNLzfN3ZRIeJsQO7Bv8IO6RAOy5An9949HWdWriebp1EzOhrqqJmb9lGSp0ZUH5SuLXBK5dJDRSGeyO9dgYgH3K0h8FWQqLNfh/OIO348627MDisImAFpH4hcvKwRGC9hOWM/wtIrxFZ4XNwkJYGbxqPNa8FDsSd9oGd56yrcFmgpfQ67FbO4sdZ7srOhmIfx+/+kGUAof0rKETc3JN9qn+HzJxgRyfWgPr83m0rwtFAVJ6/8SkkirMjlWoio6TVUJ9/ce5YTKeod9wPmA/zMtclV6HjSn/gxSJzIs6KJr2l3Z7IOsWYd/iFWih4052TDOxm2eB8r/9bbNg6vMJVTg6MdxzCY17Tk1S/hw/a7x4RtNHqSz1pqZ42tcjZJhv+CLbJcbmgmUw20lj1csj/ZI1jGfuqsvJwpVy/eGAwc7ZZjaAq29R5bap2eI2u8vGLjI2xnkvNVavckiYIOI+WOMHTVP4FjhvTLpGcwD9N39pD7ReJzV1wbjmOZ2MbMyF7HeMhsZFEp+MSDq5v/fDPccN2PBMMnHZ26G/0cYKamBDwubMh60DVnEkhhNOxRGywsbJ+z8uUCXw9BLVvgLpMHYTRQQvLlodm2ZoE0K1awAEQ3E4W1EtNbaiz7gao3jUej7Cbyw6movCAVVnsaAxCG+D09tdvzVUNtg==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6857.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(33656002)(76116006)(6506007)(122000001)(508600001)(38070700005)(66556008)(52536014)(5660300002)(86362001)(38100700002)(53546011)(26005)(8676002)(186003)(71200400001)(9686003)(83380400001)(66946007)(6916009)(316002)(2906002)(8936002)(64756008)(4326008)(66476007)(7696005)(54906003)(55016002)(30864003)(66446008);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB2040
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c93cbcda-01a9-487a-5ed0-08d9823a38aa
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Cdlo3wVOmM4PLNbDL4F2gG3jY8vfhgsLmTYm7Wk+z9nMcbI+WmSNi7kU5Y5PH3KJTSHUkw4/6dEGK/Csi8VRAG2Obx6fGVfGvbhvXJgqOPgApX5mrNxattsydwWCpspRJ/0PesFQKPC6wjJH+wj+WJjwJYlRkgv6UMTKk14CVWD/ugeczNiEeXPYxLJaX5X3Jd2yFB0F7R2xUr+kz5OLv106MBxPabCj/LSpvm/A9+zyHGsZ9l/789Ao2DQ0auzuXSfovqoyN2XXLSFM4abBlndXT+jJYO1QD18mjkduYb+n6fjW1/zDWSjcgEXeTvaovVwMacbPg87V2s048ENugBPzf25nBXqPLDQt+c1VK/YA/RylMPevkDsxDFX41bCWmxyK6bRGSXgusvsCzGPyBBCIE9u2xshvLCUlIFnZvQ0kQJSiEvfPY9co6AYpN1HPeDAoCDenVokjJ2AqahPDcnO3Ssi3LG5SnzIsPioZ+TI0M3/ZbGUviR/nhSWdw3/25xaud/SFljVoY2FnVSEUOZ8JWm0gDiaVInE4b0CLUYRQdLjTKryWVwflHddXtFLOsnaqYeV1g+UDbqPdn+HQzk8OakxhK8tjaVeoMKjTbk2qeD/FVFgM4erQeN0U3tW9//eGUL8R/p/MetSUqomTuiXXI+svhWv26sCzpaJ4vjhWkW8XnOtWgEK8NAcuelI/DPAfU8BKywN83BMSLLt4+w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(55016002)(6862004)(30864003)(47076005)(86362001)(36860700001)(8676002)(316002)(82310400003)(70586007)(53546011)(81166007)(6506007)(4326008)(54906003)(356005)(83380400001)(70206006)(26005)(186003)(5660300002)(2906002)(52536014)(9686003)(508600001)(7696005)(336012)(33656002)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 04:41:35.0957
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d49e2d1f-843d-4604-6516-08d9823a4079
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4210

SGkgU3RlZmFubywNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IFNlbnQ6IDIwMjHlubQ5
5pyIMjjml6UgMToxOQ0KPiBUbzogV2VpIENoZW4gPFdlaS5DaGVuQGFybS5jb20+DQo+IENjOiBT
dGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+OyB4ZW4tDQo+IGRldmVs
QGxpc3RzLnhlbnByb2plY3Qub3JnOyBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcw0K
PiA8QmVydHJhbmQuTWFycXVpc0Bhcm0uY29tPjsgamJldWxpY2hAc3VzZS5jb207IGFuZHJldy5j
b29wZXIzQGNpdHJpeC5jb207DQo+IHJvZ2VyLnBhdUBjaXRyaXguY29tOyB3bEB4ZW4ub3JnDQo+
IFN1YmplY3Q6IFJFOiBbUEFUQ0ggMDgvMzddIHhlbi94ODY6IGFkZCBkZXRlY3Rpb24gb2YgZGlz
Y29udGlub3VzIG5vZGUNCj4gbWVtb3J5IHJhbmdlDQo+IA0KPiBPbiBNb24sIDI3IFNlcCAyMDIx
LCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4g
PiBGcm9tOiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4g
PiBTZW50OiAyMDIx5bm0OeaciDI35pelIDEzOjA1DQo+ID4gPiBUbzogU3RlZmFubyBTdGFiZWxs
aW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KPiA+ID4gQ2M6IFdlaSBDaGVuIDxXZWkuQ2hl
bkBhcm0uY29tPjsgeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnOw0KPiA+ID4ganVsaWVu
QHhlbi5vcmc7IEJlcnRyYW5kIE1hcnF1aXMgPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47DQo+
ID4gPiBqYmV1bGljaEBzdXNlLmNvbTsgYW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbTsgcm9nZXIu
cGF1QGNpdHJpeC5jb207DQo+ID4gPiB3bEB4ZW4ub3JnDQo+ID4gPiBTdWJqZWN0OiBSRTogW1BB
VENIIDA4LzM3XSB4ZW4veDg2OiBhZGQgZGV0ZWN0aW9uIG9mIGRpc2NvbnRpbm91cyBub2RlDQo+
ID4gPiBtZW1vcnkgcmFuZ2UNCj4gPiA+DQo+ID4gPiBPbiBTdW4sIDI2IFNlcCAyMDIxLCBTdGVm
YW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+ID4gPiA+IE9uIFN1biwgMjYgU2VwIDIwMjEsIFdlaSBD
aGVuIHdyb3RlOg0KPiA+ID4gPiA+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+
ID4gPiA+IEZyb206IFN0ZWZhbm8gU3RhYmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz4N
Cj4gPiA+ID4gPiA+IFNlbnQ6IDIwMjHlubQ55pyIMjXml6UgMzo1Mw0KPiA+ID4gPiA+ID4gVG86
IFdlaSBDaGVuIDxXZWkuQ2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gQ2M6IFN0ZWZhbm8gU3Rh
YmVsbGluaSA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IHhlbi0NCj4gPiA+ID4gPiA+IGRldmVs
QGxpc3RzLnhlbnByb2plY3Qub3JnOyBqdWxpZW5AeGVuLm9yZzsgQmVydHJhbmQgTWFycXVpcw0K
PiA+ID4gPiA+ID4gPEJlcnRyYW5kLk1hcnF1aXNAYXJtLmNvbT47IGpiZXVsaWNoQHN1c2UuY29t
Ow0KPiA+ID4gYW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbTsNCj4gPiA+ID4gPiA+IHJvZ2VyLnBh
dUBjaXRyaXguY29tOyB3bEB4ZW4ub3JnDQo+ID4gPiA+ID4gPiBTdWJqZWN0OiBSRTogW1BBVENI
IDA4LzM3XSB4ZW4veDg2OiBhZGQgZGV0ZWN0aW9uIG9mDQo+IGRpc2NvbnRpbm91cw0KPiA+ID4g
bm9kZQ0KPiA+ID4gPiA+ID4gbWVtb3J5IHJhbmdlDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
T24gRnJpLCAyNCBTZXAgMjAyMSwgV2VpIENoZW4gd3JvdGU6DQo+ID4gPiA+ID4gPiA+ID4gLS0t
LS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPiA+ID4gPiA+ID4gPiBGcm9tOiBTdGVmYW5vIFN0
YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+ID4gPiA+ID4gPiA+ID4gU2VudDog
MjAyMeW5tDnmnIgyNOaXpSA4OjI2DQo+ID4gPiA+ID4gPiA+ID4gVG86IFdlaSBDaGVuIDxXZWku
Q2hlbkBhcm0uY29tPg0KPiA+ID4gPiA+ID4gPiA+IENjOiB4ZW4tZGV2ZWxAbGlzdHMueGVucHJv
amVjdC5vcmc7IHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc7DQo+ID4gPiA+ID4gPiBqdWxpZW5AeGVu
Lm9yZzsNCj4gPiA+ID4gPiA+ID4gPiBCZXJ0cmFuZCBNYXJxdWlzIDxCZXJ0cmFuZC5NYXJxdWlz
QGFybS5jb20+Ow0KPiBqYmV1bGljaEBzdXNlLmNvbTsNCj4gPiA+ID4gPiA+ID4gPiBhbmRyZXcu
Y29vcGVyM0BjaXRyaXguY29tOyByb2dlci5wYXVAY2l0cml4LmNvbTsgd2xAeGVuLm9yZw0KPiA+
ID4gPiA+ID4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMDgvMzddIHhlbi94ODY6IGFkZCBkZXRl
Y3Rpb24gb2YNCj4gPiA+IGRpc2NvbnRpbm91cyBub2RlDQo+ID4gPiA+ID4gPiA+ID4gbWVtb3J5
IHJhbmdlDQo+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiBDQydpbmcgeDg2IG1haW50
YWluZXJzDQo+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiBPbiBUaHUsIDIzIFNlcCAy
MDIxLCBXZWkgQ2hlbiB3cm90ZToNCj4gPiA+ID4gPiA+ID4gPiA+IE9uZSBOVU1BIG5vZGUgbWF5
IGNvbnRhaW4gc2V2ZXJhbCBtZW1vcnkgYmxvY2tzLiBJbg0KPiBjdXJyZW50DQo+ID4gPiBYZW4N
Cj4gPiA+ID4gPiA+ID4gPiA+IGNvZGUsIFhlbiB3aWxsIG1haW50YWluIGEgbm9kZSBtZW1vcnkg
cmFuZ2UgZm9yIGVhY2ggbm9kZQ0KPiB0bw0KPiA+ID4gY292ZXINCj4gPiA+ID4gPiA+ID4gPiA+
IGFsbCBpdHMgbWVtb3J5IGJsb2Nrcy4gQnV0IGhlcmUgY29tZXMgdGhlIHByb2JsZW0sIGluIHRo
ZQ0KPiBnYXANCj4gPiA+IG9mDQo+ID4gPiA+ID4gPiA+ID4gPiBvbmUgbm9kZSdzIHR3byBtZW1v
cnkgYmxvY2tzLCBpZiB0aGVyZSBhcmUgc29tZSBtZW1vcnkNCj4gYmxvY2tzDQo+ID4gPiBkb24n
dA0KPiA+ID4gPiA+ID4gPiA+ID4gYmVsb25nIHRvIHRoaXMgbm9kZSAocmVtb3RlIG1lbW9yeSBi
bG9ja3MpLiBUaGlzIG5vZGUncw0KPiBtZW1vcnkNCj4gPiA+IHJhbmdlDQo+ID4gPiA+ID4gPiA+
ID4gPiB3aWxsIGJlIGV4cGFuZGVkIHRvIGNvdmVyIHRoZXNlIHJlbW90ZSBtZW1vcnkgYmxvY2tz
Lg0KPiA+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiA+IE9uZSBub2RlJ3MgbWVtb3J5
IHJhbmdlIGNvbnRhaW5zIG90aGUgbm9kZXMnIG1lbW9yeSwgdGhpcw0KPiBpcw0KPiA+ID4gPiA+
ID4gb2J2aW91c2x5DQo+ID4gPiA+ID4gPiA+ID4gPiBub3QgdmVyeSByZWFzb25hYmxlLiBUaGlz
IG1lYW5zIGN1cnJlbnQgTlVNQSBjb2RlIG9ubHkgY2FuDQo+ID4gPiBzdXBwb3J0DQo+ID4gPiA+
ID4gPiA+ID4gPiBub2RlIGhhcyBjb250aW5vdXMgbWVtb3J5IGJsb2Nrcy4gSG93ZXZlciwgb24g
YSBwaHlzaWNhbA0KPiA+ID4gbWFjaGluZSwNCj4gPiA+ID4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4g
PiA+ID4gYWRkcmVzc2VzIG9mIG11bHRpcGxlIG5vZGVzIGNhbiBiZSBpbnRlcmxlYXZlZC4NCj4g
PiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+ID4gPiBTbyBpbiB0aGlzIHBhdGNoLCB3ZSBh
ZGQgY29kZSB0byBkZXRlY3QgZGlzY29udGlub3VzDQo+IG1lbW9yeQ0KPiA+ID4gYmxvY2tzDQo+
ID4gPiA+ID4gPiA+ID4gPiBmb3Igb25lIG5vZGUuIE5VTUEgaW5pdGlhbGl6dGlvbiB3aWxsIGJl
IGZhaWxlZCBhbmQgZXJyb3INCj4gPiA+IG1lc3NhZ2VzDQo+ID4gPiA+ID4gPiA+ID4gPiB3aWxs
IGJlIHByaW50ZWQgd2hlbiBYZW4gZGV0ZWN0IHN1Y2ggaGFyZHdhcmUNCj4gY29uZmlndXJhdGlv
bi4NCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+IEF0IGxlYXN0IG9uIEFSTSwgaXQg
aXMgbm90IGp1c3QgbWVtb3J5IHRoYXQgY2FuIGJlDQo+IGludGVybGVhdmVkLA0KPiA+ID4gYnV0
DQo+ID4gPiA+ID4gPiBhbHNvDQo+ID4gPiA+ID4gPiA+ID4gTU1JTyByZWdpb25zLiBGb3IgaW5z
dGFuY2U6DQo+ID4gPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+ID4gPiBub2RlMCBiYW5rMCAwLTB4
MTAwMDAwMA0KPiA+ID4gPiA+ID4gPiA+IE1NSU8gMHgxMDAwMDAwLTB4MTAwMjAwMA0KPiA+ID4g
PiA+ID4gPiA+IEhvbGUgMHgxMDAyMDAwLTB4MjAwMDAwMA0KPiA+ID4gPiA+ID4gPiA+IG5vZGUw
IGJhbmsxIDB4MjAwMDAwMC0weDMwMDAwMDANCj4gPiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4g
PiA+IFNvIEkgYW0gbm90IGZhbWlsaWFyIHdpdGggdGhlIFNSQVQgZm9ybWF0LCBidXQgSSB0aGlu
ayBvbg0KPiBBUk0NCj4gPiA+IHRoZQ0KPiA+ID4gPiA+ID4gY2hlY2sNCj4gPiA+ID4gPiA+ID4g
PiB3b3VsZCBsb29rIGRpZmZlcmVudDogd2Ugd291bGQganVzdCBsb29rIGZvciBtdWx0aXBsZSBt
ZW1vcnkNCj4gPiA+IHJhbmdlcw0KPiA+ID4gPiA+ID4gPiA+IHVuZGVyIGEgZGV2aWNlX3R5cGUg
PSAibWVtb3J5IiBub2RlIG9mIGEgTlVNQSBub2RlIGluIGRldmljZQ0KPiA+ID4gdHJlZS4NCj4g
PiA+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiA+DQo+ID4gPiA+ID4g
PiA+IFNob3VsZCBJIG5lZWQgdG8gaW5jbHVkZS9yZWZpbmUgYWJvdmUgbWVzc2FnZSB0byBjb21t
aXQgbG9nPw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPiA+IExldCBtZSBhc2sgeW91IGEgcXVlc3Rp
b24gZmlyc3QuDQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gV2l0aCB0aGUgTlVNQSBpbXBsZW1l
bnRhdGlvbiBvZiB0aGlzIHBhdGNoIHNlcmllcywgY2FuIHdlIGRlYWwNCj4gd2l0aA0KPiA+ID4g
PiA+ID4gY2FzZXMgd2hlcmUgZWFjaCBub2RlIGhhcyBtdWx0aXBsZSBtZW1vcnkgYmFua3MsIG5v
dCBpbnRlcmxlYXZlZD8NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IFllcy4NCj4gPiA+ID4gPg0KPiA+
ID4gPiA+ID4gQW4gYW4gZXhhbXBsZToNCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPiBub2RlMDog
MHgwICAgICAgICAtIDB4MTAwMDAwMDANCj4gPiA+ID4gPiA+IE1NSU8gOiAweDEwMDAwMDAwIC0g
MHgyMDAwMDAwMA0KPiA+ID4gPiA+ID4gbm9kZTA6IDB4MjAwMDAwMDAgLSAweDMwMDAwMDAwDQo+
ID4gPiA+ID4gPiBNTUlPIDogMHgzMDAwMDAwMCAtIDB4NTAwMDAwMDANCj4gPiA+ID4gPiA+IG5v
ZGUxOiAweDUwMDAwMDAwIC0gMHg2MDAwMDAwMA0KPiA+ID4gPiA+ID4gTU1JTyA6IDB4NjAwMDAw
MDAgLSAweDgwMDAwMDAwDQo+ID4gPiA+ID4gPiBub2RlMjogMHg4MDAwMDAwMCAtIDB4OTAwMDAw
MDANCj4gPiA+ID4gPiA+DQo+ID4gPiA+ID4gPg0KPiA+ID4gPiA+ID4gSSBhc3N1bWUgd2UgY2Fu
IGRlYWwgd2l0aCB0aGlzIGNhc2Ugc2ltcGx5IGJ5IHNldHRpbmcgbm9kZTANCj4gbWVtb3J5DQo+
ID4gPiB0bw0KPiA+ID4gPiA+ID4gMHgwLTB4MzAwMDAwMDAgZXZlbiBpZiB0aGVyZSBpcyBhY3R1
YWxseSBzb21ldGhpbmcgZWxzZSwgYQ0KPiBkZXZpY2UsDQo+ID4gPiB0aGF0DQo+ID4gPiA+ID4g
PiBkb2Vzbid0IGJlbG9uZyB0byBub2RlMCBpbiBiZXR3ZWVuIHRoZSB0d28gbm9kZTAgYmFua3M/
DQo+ID4gPiA+ID4NCj4gPiA+ID4gPiBXaGlsZSB0aGlzIGNvbmZpZ3VyYXRpb24gaXMgcmFyZSBp
biBTb0MgZGVzaWduLCBidXQgaXQgaXMgbm90DQo+ID4gPiBpbXBvc3NpYmxlLg0KPiA+ID4gPg0K
PiA+ID4gPiBEZWZpbml0ZWx5LCBJIGhhdmUgc2VlbiBpdCBiZWZvcmUuDQo+ID4gPiA+DQo+ID4g
PiA+DQo+ID4gPiA+ID4gPiBJcyBpdCBvbmx5IG90aGVyIG5vZGVzJyBtZW1vcnkgaW50ZXJsZWF2
ZWQgdGhhdCBjYXVzZSBpc3N1ZXM/IEluDQo+ID4gPiBvdGhlcg0KPiA+ID4gPiA+ID4gd29yZHMs
IG9ubHkgdGhlIGZvbGxvd2luZyBpcyBhIHByb2JsZW1hdGljIHNjZW5hcmlvPw0KPiA+ID4gPiA+
ID4NCj4gPiA+ID4gPiA+IG5vZGUwOiAweDAgICAgICAgIC0gMHgxMDAwMDAwMA0KPiA+ID4gPiA+
ID4gTU1JTyA6IDB4MTAwMDAwMDAgLSAweDIwMDAwMDAwDQo+ID4gPiA+ID4gPiBub2RlMTogMHgy
MDAwMDAwMCAtIDB4MzAwMDAwMDANCj4gPiA+ID4gPiA+IE1NSU8gOiAweDMwMDAwMDAwIC0gMHg1
MDAwMDAwMA0KPiA+ID4gPiA+ID4gbm9kZTA6IDB4NTAwMDAwMDAgLSAweDYwMDAwMDAwDQo+ID4g
PiA+ID4gPg0KPiA+ID4gPiA+ID4gQmVjYXVzZSBub2RlMSBpcyBpbiBiZXR3ZWVuIHRoZSB0d28g
cmFuZ2VzIG9mIG5vZGUwPw0KPiA+ID4gPiA+ID4NCj4gPiA+ID4gPg0KPiA+ID4gPiA+IEJ1dCBv
bmx5IGRldmljZV90eXBlPSJtZW1vcnkiIGNhbiBiZSBhZGRlZCB0byBhbGxvY2F0aW9uLg0KPiA+
ID4gPiA+IEZvciBtbWlvIHRoZXJlIGFyZSB0d28gY2FzZXM6DQo+ID4gPiA+ID4gMS4gbW1pbyBk
b2Vzbid0IGhhdmUgTlVNQSBpZCBwcm9wZXJ0eS4NCj4gPiA+ID4gPiAyLiBtbWlvIGhhcyBOVU1B
IGlkIHByb3BlcnR5LCBqdXN0IGxpa2Ugc29tZSBQQ0llIGNvbnRyb2xsZXJzLg0KPiA+ID4gPiA+
ICAgIEJ1dCB3ZSBkb27igJl0IG5lZWQgdG8gaGFuZGxlIHRoZXNlIGtpbmRzIG9mIE1NSU8gZGV2
aWNlcw0KPiA+ID4gPiA+ICAgIGluIG1lbW9yeSBibG9jayBwYXJzaW5nLiBCZWNhdXNlIHdlIGRv
bid0IG5lZWQgdG8gYWxsb2NhdGUNCj4gPiA+ID4gPiAgICBtZW1vcnkgZnJvbSB0aGVzZSBtbWlv
IHJhbmdlcy4gQW5kIGZvciBhY2Nlc3NpbmcsIHdlIG5lZWQNCj4gPiA+ID4gPiAgICBhIE5VTUEt
YXdhcmUgUENJZSBjb250cm9sbGVyIGRyaXZlciBvciBhIGdlbmVyaWMgTlVNQS1hd2FyZQ0KPiA+
ID4gPiA+ICAgIE1NSU8gYWNjZXNzaW5nIEFQSXMuDQo+ID4gPiA+DQo+ID4gPiA+IFllcywgSSBh
bSBub3QgdG9vIHdvcnJpZWQgYWJvdXQgZGV2aWNlcyB3aXRoIGEgTlVNQSBpZCBwcm9wZXJ0eQ0K
PiBiZWNhdXNlDQo+ID4gPiA+IHRoZXkgYXJlIGxlc3MgY29tbW9uIGFuZCB0aGlzIHNlcmllcyBk
b2Vzbid0IGhhbmRsZSB0aGVtIGF0IGFsbCwNCj4gcmlnaHQ/DQo+ID4gPiA+IEkgaW1hZ2luZSB0
aGV5IHdvdWxkIGJlIHRyZWF0ZWQgbGlrZSBhbnkgb3RoZXIgZGV2aWNlIHdpdGhvdXQgTlVNQQ0K
PiA+ID4gPiBhd2FyZW5lc3MuDQo+ID4gPiA+DQo+ID4gPiA+IEkgYW0gdGhpbmtpbmcgYWJvdXQg
dGhlIGNhc2Ugd2hlcmUgdGhlIG1lbW9yeSBvZiBlYWNoIE5VTUEgbm9kZSBpcw0KPiBtYWRlDQo+
ID4gPiA+IG9mIG11bHRpcGxlIGJhbmtzLiBJIHVuZGVyc3RhbmQgdGhhdCB0aGlzIHBhdGNoIGFk
ZHMgYW4gZXhwbGljaXQNCj4gY2hlY2sNCj4gPiA+ID4gZm9yIGNhc2VzIHdoZXJlIHRoZXNlIGJh
bmtzIGFyZSBpbnRlcmxlYXZpbmcsIGhvd2V2ZXIgdGhlcmUgYXJlIG1hbnkNCj4gPiA+ID4gb3Ro
ZXIgY2FzZXMgd2hlcmUgTlVNQSBtZW1vcnkgbm9kZXMgYXJlICpub3QqIGludGVybGVhdmluZyBi
dXQgdGhleQ0KPiBhcmUNCj4gPiA+ID4gc3RpbGwgbWFkZSBvZiBtdWx0aXBsZSBkaXNjb250aW51
b3VzIGJhbmtzLCBsaWtlIGluIHRoZSB0d28gZXhhbXBsZQ0KPiA+ID4gPiBhYm92ZS4NCj4gPiA+
ID4NCj4gPiA+ID4gTXkgcXVlc3Rpb24gaXMgd2hldGhlciB0aGlzIHBhdGNoIHNlcmllcyBpbiBp
dHMgY3VycmVudCBmb3JtIGNhbg0KPiBoYW5kbGUNCj4gPiA+ID4gdGhlIHR3byBjYXNlcyBhYm92
ZSBjb3JyZWN0bHkuIElmIHNvLCBJIGFtIHdvbmRlcmluZyBob3cgaXQgd29ya3MNCj4gZ2l2ZW4N
Cj4gPiA+ID4gdGhhdCB3ZSBvbmx5IGhhdmUgYSBzaW5nbGUgInN0YXJ0IiBhbmQgInNpemUiIHBh
cmFtZXRlciBwZXIgbm9kZS4NCj4gPiA+ID4NCj4gPiA+ID4gT24gdGhlIG90aGVyIGhhbmQgaWYg
dGhpcyBzZXJpZXMgY2Fubm90IGhhbmRsZSB0aGUgdHdvIGNhc2VzIGFib3ZlLA0KPiBteQ0KPiA+
ID4gPiBxdWVzdGlvbiBpcyB3aGV0aGVyIGl0IHdvdWxkIGZhaWwgZXhwbGljaXRseSBvciBub3Qu
IFRoZSBuZXcNCj4gPiA+ID4gY2hlY2sgaXNfbm9kZV9tZW1vcnlfY29udGludW91cyBkb2Vzbid0
IHNlZW0gdG8gYmUgYWJsZSB0byBjYXRjaA0KPiB0aGVtLg0KPiA+ID4NCj4gPiA+DQo+ID4gPiBM
b29raW5nIGF0IG51bWFfdXBkYXRlX25vZGVfbWVtYmxrcywgaXQgaXMgY2xlYXIgdGhhdCB0aGUg
Y29kZSBpcw0KPiBtZWFudA0KPiA+ID4gdG8gaW5jcmVhc2UgdGhlIHJhbmdlIG9mIGVhY2ggbnVt
YSBub2RlIHRvIGNvdmVyIGV2ZW4gTU1JTyByZWdpb25zIGluDQo+ID4gPiBiZXR3ZWVuIG1lbW9y
eSBiYW5rcy4gQWxzbyBzZWUgdGhlIGNvbW1lbnQgYXQgdGhlIHRvcCBvZiB0aGUgZmlsZToNCj4g
PiA+DQo+ID4gPiAgKiBBc3N1bWVzIGFsbCBtZW1vcnkgcmVnaW9ucyBiZWxvbmdpbmcgdG8gYSBz
aW5nbGUgcHJveGltaXR5IGRvbWFpbg0KPiA+ID4gICogYXJlIGluIG9uZSBjaHVuay4gSG9sZXMg
YmV0d2VlbiB0aGVtIHdpbGwgYmUgaW5jbHVkZWQgaW4gdGhlIG5vZGUuDQo+ID4gPg0KPiA+ID4g
U28gaWYgdGhlcmUgYXJlIG11bHRpcGxlIGJhbmtzIGZvciBlYWNoIG5vZGUsIHN0YXJ0IGFuZCBl
bmQgYXJlDQo+ID4gPiBzdHJldGNoZWQgdG8gY292ZXIgdGhlIGhvbGVzIGJldHdlZW4gdGhlbSwg
YW5kIGl0IHdvcmtzIGFzIGxvbmcgYXMNCj4gPiA+IG1lbW9yeSBiYW5rcyBvZiBkaWZmZXJlbnQg
TlVNQSBub2RlcyBkb24ndCBpbnRlcmxlYXZlLg0KPiA+ID4NCj4gPiA+IEkgd291bGQgYXBwcmVj
aWF0ZSBpZiB5b3UgY291bGQgYWRkIGFuIGluLWNvZGUgY29tbWVudCB0byBleHBsYWluIHRoaXMN
Cj4gPiA+IG9uIHRvcCBvZiBudW1hX3VwZGF0ZV9ub2RlX21lbWJsay4NCj4gPg0KPiA+IFllcywg
SSB3aWxsIGRvIGl0Lg0KPiANCj4gVGhhbmsgeW91DQo+IA0KPiANCj4gPiA+IEhhdmUgeW91IGhh
ZCBhIGNoYW5jZSB0byB0ZXN0IHRoaXM/IElmIG5vdCBpdCB3b3VsZCBiZSBmYW50YXN0aWMgaWYN
Cj4geW91DQo+ID4gPiBjb3VsZCBnaXZlIGl0IGEgcXVpY2sgdGVzdCB0byBtYWtlIHN1cmUgaXQg
d29ya3MgYXMgaW50ZW5kZWQ6IGZvcg0KPiA+ID4gaW5zdGFuY2UgYnkgY3JlYXRpbmcgbXVsdGlw
bGUgbWVtb3J5IGJhbmtzIGZvciBlYWNoIE5VTUEgbm9kZSBieQ0KPiA+ID4gc3BsaXR0aW5nIGFu
IHJlYWwgYmFuayBpbnRvIHR3byBzbWFsbGVyIGJhbmtzIHdpdGggYSBob2xlIGluIGJldHdlZW4N
Cj4gaW4NCj4gPiA+IGRldmljZSB0cmVlLCBqdXN0IGZvciB0aGUgc2FrZSBvZiB0ZXN0aW5nLg0K
PiA+DQo+ID4gWWVzLCBJIGhhdmUgY3JlYXRlZCBzb21lIGZha2UgTlVNQSBub2RlcyBpbiBGVlAg
ZGV2aWNlIHRyZWUgdG8gdGVzdCBpdC4NCj4gPiBUaGUgaW50ZXJ0d2luZSBvZiBub2RlcycgYWRk
cmVzcyBjYW4gYmUgZGV0ZWN0ZWQuDQo+ID4NCj4gPiAoWEVOKSBTUkFUOiBOb2RlIDAgMDAwMDAw
MDA4MDAwMDAwMC0wMDAwMDAwMGZmMDAwMDAwDQo+ID4gKFhFTikgU1JBVDogTm9kZSAxIDAwMDAw
MDA4ODAwMDAwMDAtMDAwMDAwMDhjMDAwMDAwMA0KPiA+IChYRU4pIE5PREUgMDogKDAwMDAwMDAw
ODAwMDAwMDAtMDAwMDAwMDhkMDAwMDAwMCkgaW50ZXJ0d2luZSB3aXRoIE5PREUgMQ0KPiAoMDAw
MDAwMDg4MDAwMDAwMC0wMDAwMDAwOGMwMDAwMDAwKQ0KPiANCj4gR3JlYXQgdGhhbmtzISBBbmQg
d2hhdCBpZiB0aGVyZSBhcmUgbXVsdGlwbGUgbm9uLWNvbnRpZ3VvdXMgbWVtb3J5IGJhbmtzDQo+
IHBlciBub2RlLCBidXQgKm5vdCogaW50ZXJ0d2luZWQuIERvZXMgdGhhdCBhbGwgd29yayBjb3Jy
ZWN0bHkgYXMNCj4gZXhwZWN0ZWQ/DQoNClllcywgSSBhbSB1c2luZyBhIGRldmljZSB0cmVlIHNl
dHRpbmcgbGlrZSB0aGlzOg0KDQogICAgbWVtb3J5QDgwMDAwMDAwIHsNCiAgICAgICAgZGV2aWNl
X3R5cGUgPSAibWVtb3J5IjsNCiAgICAgICAgcmVnID0gPDB4MCAweDgwMDAwMDAwIDB4MCAweDgw
MDAwMDAwPjsNCiAgICAgICAgbnVtYS1ub2RlLWlkID0gPDA+Ow0KICAgIH07DQoNCiAgICBtZW1v
cnlAODgwMDAwMDAwIHsNCiAgICAgICAgZGV2aWNlX3R5cGUgPSAibWVtb3J5IjsNCiAgICAgICAg
cmVnID0gPDB4OCAweDgwMDAwMDAwIDB4MCAweDgwMDAwMDA+Ow0KICAgICAgICBudW1hLW5vZGUt
aWQgPSA8MD47DQogICAgfTsNCg0KICAgIG1lbW9yeUA4OTAwMDAwMDAgew0KICAgICAgICBkZXZp
Y2VfdHlwZSA9ICJtZW1vcnkiOw0KICAgICAgICByZWcgPSA8MHg4IDB4OTAwMDAwMDAgMHgwIDB4
ODAwMDAwMD47DQogICAgICAgIG51bWEtbm9kZS1pZCA9IDwwPjsNCiAgICB9Ow0KDQogICAgbWVt
b3J5QDhBMDAwMDAwMCB7DQogICAgICAgIGRldmljZV90eXBlID0gIm1lbW9yeSI7DQogICAgICAg
IHJlZyA9IDwweDggMHhBMDAwMDAwMCAweDAgMHg4MDAwMDAwPjsNCiAgICAgICAgbnVtYS1ub2Rl
LWlkID0gPDA+Ow0KICAgIH07DQoNCiAgICBtZW1vcnlAOEIwMDAwMDAwIHsNCiAgICAgICAgZGV2
aWNlX3R5cGUgPSAibWVtb3J5IjsNCiAgICAgICAgcmVnID0gPDB4OCAweEIwMDAwMDAwIDB4MCAw
eDgwMDAwMDA+Ow0KICAgICAgICBudW1hLW5vZGUtaWQgPSA8MD47DQogICAgfTsNCg0KICAgIG1l
bW9yeUA4QzAwMDAwMDAgew0KICAgICAgICBkZXZpY2VfdHlwZSA9ICJtZW1vcnkiOw0KICAgICAg
ICByZWcgPSA8MHg4IDB4QzAwMDAwMDAgMHgwIDB4ODAwMDAwMD47DQogICAgICAgIG51bWEtbm9k
ZS1pZCA9IDwxPjsNCiAgICB9Ow0KDQogICAgbWVtb3J5QDhEMDAwMDAwMCB7DQogICAgICAgIGRl
dmljZV90eXBlID0gIm1lbW9yeSI7DQogICAgICAgIHJlZyA9IDwweDggMHhEMDAwMDAwMCAweDAg
MHg4MDAwMDAwPjsNCiAgICAgICAgbnVtYS1ub2RlLWlkID0gPDE+Ow0KICAgIH07DQoNCiAgICBt
ZW1vcnlAOEUwMDAwMDAwIHsNCiAgICAgICAgZGV2aWNlX3R5cGUgPSAibWVtb3J5IjsNCiAgICAg
ICAgcmVnID0gPDB4OCAweEUwMDAwMDAwIDB4MCAweDgwMDAwMDA+Ow0KICAgICAgICBudW1hLW5v
ZGUtaWQgPSA8MT47DQogICAgfTsNCg0KICAgIG1lbW9yeUA4RjAwMDAwMDAgew0KICAgICAgICBk
ZXZpY2VfdHlwZSA9ICJtZW1vcnkiOw0KICAgICAgICByZWcgPSA8MHg4IDB4RjAwMDAwMDAgMHgw
IDB4ODAwMDAwMD47DQogICAgICAgIG51bWEtbm9kZS1pZCA9IDwxPjsNCiAgICB9Ow0KDQpBbmQg
aW4gWGVuIHdlIGdvdCB0aGUgb3V0cHV0Og0KDQooWEVOKSBEVDogTlVNQSBub2RlIDAgcHJvY2Vz
c29yIHBhcnNlZA0KKFhFTikgRFQ6IE5VTUEgbm9kZSAwIHByb2Nlc3NvciBwYXJzZWQNCihYRU4p
IERUOiBOVU1BIG5vZGUgMSBwcm9jZXNzb3IgcGFyc2VkDQooWEVOKSBEVDogTlVNQSBub2RlIDEg
cHJvY2Vzc29yIHBhcnNlZA0KKFhFTikgU1JBVDogTm9kZSAwIDAwMDAwMDAwODAwMDAwMDAtMDAw
MDAwMDBmZjAwMDAwMA0KKFhFTikgU1JBVDogTm9kZSAwIDAwMDAwMDA4ODAwMDAwMDAtMDAwMDAw
MDg4ODAwMDAwMA0KKFhFTikgU1JBVDogTm9kZSAwIDAwMDAwMDA4OTAwMDAwMDAtMDAwMDAwMDg5
ODAwMDAwMA0KKFhFTikgU1JBVDogTm9kZSAwIDAwMDAwMDA4YTAwMDAwMDAtMDAwMDAwMDhhODAw
MDAwMA0KKFhFTikgU1JBVDogTm9kZSAwIDAwMDAwMDA4YjAwMDAwMDAtMDAwMDAwMDhiODAwMDAw
MA0KKFhFTikgU1JBVDogTm9kZSAxIDAwMDAwMDA4YzAwMDAwMDAtMDAwMDAwMDhjODAwMDAwMA0K
KFhFTikgU1JBVDogTm9kZSAxIDAwMDAwMDA4ZDAwMDAwMDAtMDAwMDAwMDhkODAwMDAwMA0KKFhF
TikgU1JBVDogTm9kZSAxIDAwMDAwMDA4ZTAwMDAwMDAtMDAwMDAwMDhlODAwMDAwMA0KKFhFTikg
U1JBVDogTm9kZSAxIDAwMDAwMDA4ZjAwMDAwMDAtMDAwMDAwMDhmODAwMDAwMA0KKFhFTikgTlVN
QTogcGFyc2luZyBudW1hLWRpc3RhbmNlLW1hcA0KKFhFTikgTlVNQTogZGlzdGFuY2U6IE5PREUj
MC0+Tk9ERSMwOjEwDQooWEVOKSBOVU1BOiBkaXN0YW5jZTogTk9ERSMwLT5OT0RFIzE6MjANCihY
RU4pIE5VTUE6IGRpc3RhbmNlOiBOT0RFIzEtPk5PREUjMToxMA0KKFhFTikgTlVNQTogVXNpbmcg
MTYgZm9yIHRoZSBoYXNoIHNoaWZ0Lg0KKFhFTikgRG9tYWluIGhlYXAgaW5pdGlhbGlzZWQNCihY
RU4pIEJvb3RpbmcgdXNpbmcgRGV2aWNlIFRyZWUNCg0KRG9tMCBjYW4gYmUgYm9vdCBzdWNjZXNz
ZnVsbHksIHhsIGluZm8gZ290Og0KeGwgaW5mbw0KaG9zdCAgICAgICAgICAgICAgICAgICA6IFgt
RG9tMA0KcmVsZWFzZSAgICAgICAgICAgICAgICA6IDUuMTIuMA0KdmVyc2lvbiAgICAgICAgICAg
ICAgICA6ICMyMCBTTVAgUFJFRU1QVCBXZWQgSnVsIDI4IDEzOjQxOjI4IENTVCAyMDIxDQptYWNo
aW5lICAgICAgICAgICAgICAgIDogYWFyY2g2NA0KbnJfY3B1cyAgICAgICAgICAgICAgICA6IDQN
Cm1heF9jcHVfaWQgICAgICAgICAgICAgOiAzDQpucl9ub2RlcyAgICAgICAgICAgICAgIDogMg0K
Y29yZXNfcGVyX3NvY2tldCAgICAgICA6IDENCnRocmVhZHNfcGVyX2NvcmUgICAgICAgOiAxDQoN
ClhlbiBkZWJ1ZyBjb25zb2xlIHRvIGR1bXAgbnVtYSBpbmZvLCB3ZSBnb3Q6DQoNCihYRU4pICd1
JyBwcmVzc2VkIC0+IGR1bXBpbmcgbnVtYSBpbmZvIChub3cgPSAxMzIyOTM3MjI4MTAxMCkNCihY
RU4pIE5PREUwIHN0YXJ0LT41MjQyODggc2l6ZS0+ODYxNzk4NCBmcmVlLT4zODg3NDENCihYRU4p
IE5PREUxIHN0YXJ0LT45MTc1MDQwIHNpemUtPjIyOTM3NiBmcmVlLT4xMDY0NjANCihYRU4pIENQ
VTAuLi4xIC0+IE5PREUwDQooWEVOKSBDUFUyLi4uMyAtPiBOT0RFMQ0KKFhFTikgTWVtb3J5IGxv
Y2F0aW9uIG9mIGVhY2ggZG9tYWluOg0KKFhFTikgRG9tYWluIDAgKHRvdGFsOiAyNjIxNDQpOg0K
KFhFTikgICAgIE5vZGUgMDogMjYyMTQ0DQooWEVOKSAgICAgTm9kZSAxOiAwDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:52:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:52:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197422.350448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV567-00068B-HQ; Tue, 28 Sep 2021 04:52:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197422.350448; Tue, 28 Sep 2021 04:52:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV567-000684-Dj; Tue, 28 Sep 2021 04:52:07 +0000
Received: by outflank-mailman (input) for mailman id 197422;
 Tue, 28 Sep 2021 04:52:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV566-00067y-9T
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:52:06 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d2c524b4-2017-11ec-bc65-12813bfff9fa;
 Tue, 28 Sep 2021 04:52:03 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S4iu6s001926; 
 Tue, 28 Sep 2021 04:52:02 GMT
Received: from eur01-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2055.outbound.protection.outlook.com [104.47.1.55])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbv9pg0e7-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 04:52:01 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3469.eurprd03.prod.outlook.com (2603:10a6:803:18::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 04:51:57 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 04:51:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2c524b4-2017-11ec-bc65-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LdaL+3YR3tzBE9xz5RNFz4iFJYmQvS4wyoLY33YP4l4R5etFqrrngn8q+vZSjuughFwvS6rqTADiFxZmbA5MItq52K4Y6LmYTYLAZZwk9eX48Bc6/oRoyyTuwBnho+HCyHVvVehIEla1SQsxeMnO+PtFLGV1TjZ6JYlZaik+XXaSHP31PlmU51MnfO1YVWZNL59rxWjo9z4u1ws99laeY3O9wvuctpDDmGiL8B0hqIE71Q9RlFeLZe5WBSyaQZ5DeDV1nAEtrmD4rhbhUBy1Oc+8Pwuh0WymyOYbBid/SYadSWdBhtdFh+EKZ2oTETSIF0iZcjcDWHDlzq8liSJqFQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eXzQ5iNZAUjLJBIhwWKKv4Wf4V2QdIE91Aa/J4j5aJY=;
 b=RQYoEY0JOuoox5A9PoQJ4lZAQt8BKmoq5STMNO87hXYgjz+WTQzmtezz8XPON3o+DVJJGWTg0+EUDfaWN7LTL0W3i62EusEALv+AccP9irS1IPv7tEMfpKsEAQp5Pcxc0CtFFJhygiK71J/YvadI2PSCcXUyhH7UckkRudbGXsMdDfTC9KNa+lpk9ra5qjde9YPUkt3tk6tcDSNFqBdz6hp2GrR25ADqjKk90EjK7PWh9djpdbam1HmIoI4khLwr0gIaid6eYKYmr0poF3MV4Au2QVPT22z28a4UbBlRUFpHQNV10sS5uJHE/PciiNUuHa5W/HYGyeS7us0eQGJg6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eXzQ5iNZAUjLJBIhwWKKv4Wf4V2QdIE91Aa/J4j5aJY=;
 b=ckHiBbsPencGyQfxm1jpEpWQzL7iW45C/LMNZvew/JsiSyTLW4WoTpDyDMP/9EayM5vQFMP+d6+qvHkRxu/mcmBMAF7z0u3Z8PpT+CqUXx6uN46ONMJCQDesoYPCJBknf59SRZhlZ6TrLagOg2PnzevQcNTqSEg7urUImm+tl48BwZ0DtI907pGGc+x3DAlD91AkNHAzPfpxglL+ebXuaManiWtQS5vgZc2YYEfaqEmYA0CjfmAVSE0QIxxOjbUZuKpQcFyeoIBsnO3LpHIillYY1wjfYm4NJaIDxO4a7ytBuQVgw2zzI1k9q7pMSWSuDufjXkQ8kNrovXBCFDBlsQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Topic: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Index: AQHXsHo2GZQHQL3fu0+Ql9qa7dzr/Kuz6/8AgAPt2ICAAQAHgIAADkyA
Date: Tue, 28 Sep 2021 04:51:57 +0000
Message-ID: <cc05efe3-2b3a-be03-2c17-21578b09a013@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-11-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s>
 <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com>
 <alpine.DEB.2.21.2109272046320.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109272046320.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 734d6d73-3749-4fd8-8b4d-08d9823bb37d
x-ms-traffictypediagnostic: VI1PR0302MB3469:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB34697D0A9ABBBDF446E0CBD7E7A89@VI1PR0302MB3469.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5516;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 IKfoP+Y7bMLbBCziuXTCfa0oYHS1G2to+BMIpZkhnpmATMmvY6VkVKZC2NTWFGRDnoxeeX01FxEpqJVc17POWnECCsSisB91HGX2vdpjxQETp7tWWiGkQK1jwGNMPv4PMxePEoKxepdLRacTlKOnJnBtm9fl09KB3+UCByqL5W/kPtv3UyhJR+RoHsb2XCDsT3YyAhIUFbD7kxHT1weHVa1ioVLfGrJH7sNIL0lzK/HCLQ+tJ+Fu7H+mfy/R51CIO2688ZhuAFy5AEvKqfOsp0wtfWAtAaFnc+UCgWL15OZbrUllAwyKF8XT3iyjCYrqv7XQrcoR6CXQAFfMjGiUrD/G5KZAVTFmK6I4k7kyGJT2WNalcfA2sQ7LfI0EL7rUwH7vkoSlcL8/I4ZrCAG62PlL1n5Pyy1TUVaU+zioX9fljibeWDqQHkFUnSZ2yDr2VBmAj/rzUlEATWaakWDrTqmnSIp+vfHmN/hzgrFELSsQhnKH2Gzf9w+4IgMxkhdoSRw0ADxeuoSHgCyAoKUeJ9gSnr8Ezn7IKM1lItOvIsDmPHom2nM3zDHYlBRBBn18qJkYFtP6L+I1M8ZIIXr98HnyoSYvTfM7aq7Zq2u4sA+RWykpMmiw1PKrsdkv7Rv+IdTBtV91dB0E4lILLuk74GKyIvF11LewVrSR2Cix4RVv4G5uLuXdI1pQlagyrndHb8jKMe88xWpTskc8pZmhRKyuCm87IvK0rxPzD/LQ9kauTXEx2o+CBpS8Jfbvvb8ZRlSpL0U1Yj05fYrzotEIRt/ws+1Qjiw9xsZWga32RK9B2EXYvWNVeYHpGutsTYLipXjMK+o7lg46uB28AtifTQ==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(71200400001)(26005)(4326008)(30864003)(316002)(6916009)(2906002)(5660300002)(38100700002)(36756003)(122000001)(83380400001)(2616005)(508600001)(966005)(8936002)(6486002)(66446008)(64756008)(31686004)(66556008)(6512007)(31696002)(8676002)(53546011)(86362001)(66946007)(55236004)(66476007)(38070700005)(186003)(107886003)(91956017)(76116006)(6506007)(45980500001)(579004);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SlIxTlBEWHNGais1WHQvbmJLRmc0U09talhMVTVqQ0hTWll1ajlHbm1SL1Y2?=
 =?utf-8?B?THQ2QUsvMnhCSElsWFk2OEx6bW45S0NJbUdhUzNBMS9SNDBqdHE2cHc0R1Vq?=
 =?utf-8?B?WjhPaDEzRUZQSzd3eXRwdTFNL20yZTZEZjVGdlBYVXZoRVluenY0VTM5OFRu?=
 =?utf-8?B?SlF3ZkdKM3Ixc2pWckRBQ3JLcVJyS29lMW0wQmllTFZlcmZocnA5Uk1vNnlW?=
 =?utf-8?B?d0ZvbDg5U0ZoNlNKNW9NdkU4dTU2NHJHS2hiRzl0NktOWk1IdnF5VHBCajY3?=
 =?utf-8?B?L3ZRN1grVGxpajQzVWZpSTFwRHVqQ2o1TGozMXNrSzIxckMrd3EzaE96c0FQ?=
 =?utf-8?B?WkNuNm5hdDBxekNScjJQQTZ3NVZiVFhSSXkyYmtoNjIzNUhLWnpMZ0ZIZkJw?=
 =?utf-8?B?RExhZnZkVWoyWWUwVXdDQllVcUFEd09HZ1hiVVNRYlVpNVJCWlRnaytNSFNp?=
 =?utf-8?B?aGdrV09GVDZEUytVUkJMZ2kvK3M4R214U0tUdjZ1M1R2b2gwd1c1b2ZqTjJ0?=
 =?utf-8?B?bEQxaW5vYkFEM0N3WUhHWkRURklsWkVlUi85K3dPelRFaXVXK1ZZRlBRMDBh?=
 =?utf-8?B?SndPbnpaVk1nYjY1dXd6eWpSelpXbGMrcjRXWFBLY1dZQVpabGNLOUF1L1Mx?=
 =?utf-8?B?eE82N2JTNE1aUERwVUMrVW83MlJnZkVjNTBqTVdHbFlWYU12SVhGYkc2M3Aw?=
 =?utf-8?B?THFwcGFtTlBmTkZTMmU4Q3F6eG94SDQ1M1Z1S0p0aHR3N3EySlRrcGJHV0h6?=
 =?utf-8?B?d0syU2FpRmJxc2tJUXRzcnptUm1DVnlibFpmVU03c1E2ZlZUZThFWHNXVEs1?=
 =?utf-8?B?bXBhcWdxM2NwWlFqdlI3VEFDSXRCcGsxTi9sQVlEcHpQeDRKZ2NmU1lLVWk5?=
 =?utf-8?B?djJuS25MZjlVTVgvM1pPN2h5cmRnakk0N0U0d1pDZEVsSFZOdmpnWGJUQldo?=
 =?utf-8?B?SXk4TmJWbFllTm82ZG1WVTdiVXc5bUwrcjhRZ2p4bURaYy9wMEQ0bnlYZ2Jh?=
 =?utf-8?B?eUlxL3FsS0FTUEdnUmhOL0RMeHEwd1RFR1VQL01iYU52b1FET0dwb29xN25u?=
 =?utf-8?B?dDZMWHFxenJXbEcvTlM5RHBtcXlZNlVxOTVHanp3SloyMFNIMjRZbmRsWWJC?=
 =?utf-8?B?L2lZZHJmOXZLUWwyMmZQelNKRmQrT1JjVGRaOCt3aWpFbm5NMUl6RmQzcHhT?=
 =?utf-8?B?R1R3YzJQMzBiQWprRE9kVzJ6cUt2SVVIVzJwVllZVkx2U09wSHdocFlIMU1p?=
 =?utf-8?B?ZFhxNThCMnMvdDZzLzdtaDZOcWwyaWFQUTI5VVlEWWtoM2pwNk5MOTFWN1dY?=
 =?utf-8?B?UytZZkFMN0dIYlJJenpQOTF4S0hIZTIvWFI1VmNlV1dDMmlmcVU2U1Q2RzZ6?=
 =?utf-8?B?cTc2TXAxS2ZKSi9FbEJrYTA5WXR5SnVNdkRzRDdnd2VYeit1WkxwT3psdjN6?=
 =?utf-8?B?RE5xY2p2aGlUMFJZNmt5RU1tOUU2MlNlck1ZYW1XSXU2M2pyc3BGdjZwb0Jr?=
 =?utf-8?B?Z0diVjJuS0prWnhUSmJKdjZkUEVFb3pXSGF1c2s5YXA4WUdpM3dGd0RkZVd1?=
 =?utf-8?B?Z3l3eDU3MWx0cEpjeGhrMTMvbWVlcHdiSDFsK0xxRXo2dlA4UlRVdHl0dWJa?=
 =?utf-8?B?aTMxSi8zdllnN0YwVWJaWThnRlZMU293WEJySDVaYy9yMGVPSnhLK0RRSndW?=
 =?utf-8?B?Qk55RkRYdXQySFYxSlkxZDQxQXlraEVUMVlBSytRMlE3SGlLbnZOdGFkMkpH?=
 =?utf-8?B?K1lHczNCSWxEUGNQMDc3bzFWZXpvTHBLdk1OOEVqajRXOThtK1FDeHpza1p6?=
 =?utf-8?B?NnhzbWZoMWllRnUydVl3dz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <F9DE65464A7CF541B400C294F836FFB3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 734d6d73-3749-4fd8-8b4d-08d9823bb37d
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 04:51:57.4641
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: s93Mp3lQRNsfymgZ3pHbzquMiQp1IUqzSrCOH0NfxeFj8bNk9pR7KYS1rNYsz/uLJD3t6WNqqTUbXNYMNMj+MOpfCVLZwyGGQ72IubvkRo2pyH90Azg6LLwoe5AEwKXD
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3469
X-Proofpoint-ORIG-GUID: WdezLZVdar3r89jVhqF4bTk5fA-mKvgs
X-Proofpoint-GUID: WdezLZVdar3r89jVhqF4bTk5fA-mKvgs
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_03,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0
 suspectscore=0 adultscore=0 impostorscore=0 clxscore=1015 malwarescore=0
 bulkscore=0 mlxlogscore=999 spamscore=0 phishscore=0 priorityscore=1501
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280030

DQpPbiAyOC4wOS4yMSAwNzowMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBNb24s
IDI3IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IE9uIDI1LjA5
LjIxIDAzOjQ0LCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4gT24gVGh1LCAyMyBTZXAg
MjAyMSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IEZyb206IE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4N
Cj4+Pj4gUENJIGhvc3QgYnJpZGdlcyBhcmUgc3BlY2lhbCBkZXZpY2VzIGluIHRlcm1zIG9mIGlt
cGxlbWVudGluZyBQQ0kNCj4+Pj4gcGFzc3Rocm91Z2guIEFjY29yZGluZyB0byBbMV0gdGhlIGN1
cnJlbnQgaW1wbGVtZW50YXRpb24gZGVwZW5kcyBvbg0KPj4+PiBEb21haW4tMCB0byBwZXJmb3Jt
IHRoZSBpbml0aWFsaXphdGlvbiBvZiB0aGUgcmVsZXZhbnQgUENJIGhvc3QNCj4+Pj4gYnJpZGdl
IGhhcmR3YXJlIGFuZCBwZXJmb3JtIFBDSSBkZXZpY2UgZW51bWVyYXRpb24uIEluIG9yZGVyIHRv
DQo+Pj4+IGFjaGlldmUgdGhhdCBvbmUgb2YgdGhlIHJlcXVpcmVkIGNoYW5nZXMgaXMgdG8gbm90
IG1hcCBhbGwgdGhlIG1lbW9yeQ0KPj4+PiByYW5nZXMgaW4gbWFwX3JhbmdlX3RvX2RvbWFpbiBh
cyB3ZSB0cmF2ZXJzZSB0aGUgZGV2aWNlIHRyZWUgb24gc3RhcnR1cA0KPj4+PiBhbmQgcGVyZm9y
bSBzb21lIGFkZGl0aW9uYWwgY2hlY2tzIGlmIHRoZSByYW5nZSBuZWVkcyB0byBiZSBtYXBwZWQg
dG8NCj4+Pj4gRG9tYWluLTAuDQo+Pj4+DQo+Pj4+IFRoZSBnZW5lcmljIFBDSSBob3N0IGNvbnRy
b2xsZXIgZGV2aWNlIHRyZWUgYmluZGluZyBzYXlzIFsyXToNCj4+Pj4gLSByYW5nZXM6IEFzIGRl
c2NyaWJlZCBpbiBJRUVFIFN0ZCAxMjc1LTE5OTQsIGJ1dCBtdXN0IHByb3ZpZGUNCj4+Pj4gICAg
ICAgICAgICAgYXQgbGVhc3QgYSBkZWZpbml0aW9uIG9mIG5vbi1wcmVmZXRjaGFibGUgbWVtb3J5
LiBPbmUNCj4+Pj4gICAgICAgICAgICAgb3IgYm90aCBvZiBwcmVmZXRjaGFibGUgTWVtb3J5IGFu
ZCBJTyBTcGFjZSBtYXkgYWxzbw0KPj4+PiAgICAgICAgICAgICBiZSBwcm92aWRlZC4NCj4+Pj4N
Cj4+Pj4gLSByZWcgICA6IFRoZSBDb25maWd1cmF0aW9uIFNwYWNlIGJhc2UgYWRkcmVzcyBhbmQg
c2l6ZSwgYXMgYWNjZXNzZWQNCj4+Pj4gICAgICAgICAgICAgZnJvbSB0aGUgcGFyZW50IGJ1cy4g
IFRoZSBiYXNlIGFkZHJlc3MgY29ycmVzcG9uZHMgdG8NCj4+Pj4gICAgICAgICAgICAgdGhlIGZp
cnN0IGJ1cyBpbiB0aGUgImJ1cy1yYW5nZSIgcHJvcGVydHkuICBJZiBubw0KPj4+PiAgICAgICAg
ICAgICAiYnVzLXJhbmdlIiBpcyBzcGVjaWZpZWQsIHRoaXMgd2lsbCBiZSBidXMgMCAodGhlIGRl
ZmF1bHQpLg0KPj4+Pg0KPj4+PiA+RnJvbSB0aGUgYWJvdmUgbm9uZSBvZiB0aGUgbWVtb3J5IHJh
bmdlcyBmcm9tIHRoZSAicmFuZ2VzIiBwcm9wZXJ0eQ0KPj4+PiBuZWVkcyB0byBiZSBtYXBwZWQg
dG8gRG9tYWluLTAgYXQgc3RhcnR1cCBhcyBNTUlPIG1hcHBpbmcgaXMgZ29pbmcgdG8NCj4+Pj4g
YmUgaGFuZGxlZCBkeW5hbWljYWxseSBieSB2UENJIGFzIHdlIGFzc2lnbiBQQ0kgZGV2aWNlcywg
ZS5nLiBlYWNoDQo+Pj4+IGRldmljZSBhc3NpZ25lZCB0byBEb21haW4tMC9ndWVzdCB3aWxsIGhh
dmUgaXRzIE1NSU9zIG1hcHBlZC91bm1hcHBlZA0KPj4+PiBhcyBuZWVkZWQgYnkgWGVuLg0KPj4+
Pg0KPj4+PiBUaGUgInJlZyIgcHJvcGVydHkgY292ZXJzIG5vdCBvbmx5IEVDQU0gc3BhY2UsIGJ1
dCBtYXkgYWxzbyBoYXZlIG90aGVyDQo+Pj4+IHRoZW4gdGhlIGNvbmZpZ3VyYXRpb24gbWVtb3J5
IHJhbmdlcyBkZXNjcmliZWQsIGZvciBleGFtcGxlIFszXToNCj4+Pj4gLSByZWc6IFNob3VsZCBj
b250YWluIHJjX2RiaSwgY29uZmlnIHJlZ2lzdGVycyBsb2NhdGlvbiBhbmQgbGVuZ3RoLg0KPj4+
PiAtIHJlZy1uYW1lczogTXVzdCBpbmNsdWRlIHRoZSBmb2xsb3dpbmcgZW50cmllczoNCj4+Pj4g
ICAgICAicmNfZGJpIjogY29udHJvbGxlciBjb25maWd1cmF0aW9uIHJlZ2lzdGVyczsNCj4+Pj4g
ICAgICAiY29uZmlnIjogUENJZSBjb25maWd1cmF0aW9uIHNwYWNlIHJlZ2lzdGVycy4NCj4+Pj4N
Cj4+Pj4gVGhpcyBwYXRjaCBtYWtlcyBpdCBwb3NzaWJsZSB0byBub3QgbWFwIGFsbCB0aGUgcmFu
Z2VzIGZyb20gdGhlDQo+Pj4+ICJyYW5nZXMiIHByb3BlcnR5IGFuZCBhbHNvIEVDQU0gZnJvbSB0
aGUgInJlZyIuIEFsbCB0aGUgcmVzdCBmcm9tIHRoZQ0KPj4+PiAicmVnIiBwcm9wZXJ0eSBzdGls
bCBuZWVkcyB0byBiZSBtYXBwZWQgdG8gRG9tYWluLTAsIHNvIHRoZSBQQ0kNCj4+Pj4gaG9zdCBi
cmlkZ2UgcmVtYWlucyBmdW5jdGlvbmFsIGluIERvbWFpbi0wLg0KPj4+Pg0KPj4+PiBbMV0gaHR0
cHM6Ly91cmxkZWZlbnNlLmNvbS92My9fX2h0dHBzOi8vbGlzdHMueGVucHJvamVjdC5vcmcvYXJj
aGl2ZXMvaHRtbC94ZW4tZGV2ZWwvMjAyMC0wNy9tc2cwMDc3Ny5odG1sX187ISFHRl8yOWRiY1FJ
VUJQQSFsckN1TlJ6VVZrUmY0RkxncDNoVy00dU9sZGdLcjRxTnBaYl91ZkkwalctTzBlUkgxMVZG
RGVHY3M1cFBkdEtaVDFfb3U0SW5TZyQgW2xpc3RzWy5deGVucHJvamVjdFsuXW9yZ10NCj4+Pj4g
WzJdIGh0dHBzOi8vdXJsZGVmZW5zZS5jb20vdjMvX19odHRwczovL3d3dy5rZXJuZWwub3JnL2Rv
Yy9Eb2N1bWVudGF0aW9uL2RldmljZXRyZWUvYmluZGluZ3MvcGNpL2hvc3QtZ2VuZXJpYy1wY2ku
dHh0X187ISFHRl8yOWRiY1FJVUJQQSFsckN1TlJ6VVZrUmY0RkxncDNoVy00dU9sZGdLcjRxTnBa
Yl91ZkkwalctTzBlUkgxMVZGRGVHY3M1cFBkdEtaVDFfVDV5bjdHQSQgW2tlcm5lbFsuXW9yZ10N
Cj4+Pj4gWzNdIGh0dHBzOi8vdXJsZGVmZW5zZS5jb20vdjMvX19odHRwczovL3d3dy5rZXJuZWwu
b3JnL2RvYy9Eb2N1bWVudGF0aW9uL2RldmljZXRyZWUvYmluZGluZ3MvcGNpL2hpc2lsaWNvbi1w
Y2llLnR4dF9fOyEhR0ZfMjlkYmNRSVVCUEEhbHJDdU5SelVWa1JmNEZMZ3AzaFctNHVPbGRnS3I0
cU5wWmJfdWZJMGpXLU8wZVJIMTFWRkRlR2NzNXBQZHRLWlQxOGltX1kydHckIFtrZXJuZWxbLl1v
cmddDQo+Pj4+DQo+Pj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxv
bGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4NCj4+Pj4gLS0tDQo+Pj4+IFNp
bmNlIHYxOg0KPj4+PiAgICAtIEFkZGVkIGJldHRlciBkZXNjcmlwdGlvbiBvZiB3aHkgYW5kIHdo
YXQgbmVlZHMgdG8gYmUgbWFwcGVkIGludG8NCj4+Pj4gICAgICBEb21haW4tMCdzIHAybSBhbmQg
d2hhdCBkb2Vzbid0DQo+Pj4+ICAgIC0gRG8gbm90IGRvIGFueSBtYXBwaW5ncyBmb3IgUENJIGRl
dmljZXMgd2hpbGUgdHJhdmVyc2luZyB0aGUgRFQNCj4+Pj4gICAgLSBXYWxrIGFsbCB0aGUgYnJp
ZGdlcyBhbmQgbWFrZSByZXF1aXJlZCBtYXBwaW5ncyBpbiBvbmUgZ28NCj4+Pj4gLS0tDQo+Pj4+
ICAgIHhlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYyAgICAgICAgfCAzOCArKysrKysrKysrKysr
KystLS0tLS0tLQ0KPj4+PiAgICB4ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyAgICAgICAgICAgIHwg
MTQgKysrKysrKysrDQo+Pj4+ICAgIHhlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMg
fCA0OCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4+Pj4gICAgeGVuL2FyY2gvYXJt
L3BjaS9wY2ktaG9zdC16eW5xbXAuYyB8ICAxICsNCj4+Pj4gICAgeGVuL2luY2x1ZGUvYXNtLWFy
bS9wY2kuaCAgICAgICAgICB8ICA5ICsrKysrKw0KPj4+PiAgICB4ZW4vaW5jbHVkZS9hc20tYXJt
L3NldHVwLmggICAgICAgIHwgMTMgKysrKysrKysNCj4+Pj4gICAgNiBmaWxlcyBjaGFuZ2VkLCAx
MTEgaW5zZXJ0aW9ucygrKSwgMTIgZGVsZXRpb25zKC0pDQo+Pj4+DQo+Pj4+IGRpZmYgLS1naXQg
YS94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxkLmMgYi94ZW4vYXJjaC9hcm0vZG9tYWluX2J1aWxk
LmMNCj4+Pj4gaW5kZXggODNhYjBkNTJjY2U5Li5lNzJjMWI4ODFjYWUgMTAwNjQ0DQo+Pj4+IC0t
LSBhL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KPj4+PiArKysgYi94ZW4vYXJjaC9hcm0v
ZG9tYWluX2J1aWxkLmMNCj4+Pj4gQEAgLTEwLDcgKzEwLDYgQEANCj4+Pj4gICAgI2luY2x1ZGUg
PGFzbS9yZWdzLmg+DQo+Pj4+ICAgICNpbmNsdWRlIDx4ZW4vZXJybm8uaD4NCj4+Pj4gICAgI2lu
Y2x1ZGUgPHhlbi9lcnIuaD4NCj4+Pj4gLSNpbmNsdWRlIDx4ZW4vZGV2aWNlX3RyZWUuaD4NCj4+
Pj4gICAgI2luY2x1ZGUgPHhlbi9saWJmZHQvbGliZmR0Lmg+DQo+Pj4+ICAgICNpbmNsdWRlIDx4
ZW4vZ3Vlc3RfYWNjZXNzLmg+DQo+Pj4+ICAgICNpbmNsdWRlIDx4ZW4vaW9jYXAuaD4NCj4+Pj4g
QEAgLTQ3LDEyICs0Niw2IEBAIHN0YXRpYyBpbnQgX19pbml0IHBhcnNlX2RvbTBfbWVtKGNvbnN0
IGNoYXIgKnMpDQo+Pj4+ICAgIH0NCj4+Pj4gICAgY3VzdG9tX3BhcmFtKCJkb20wX21lbSIsIHBh
cnNlX2RvbTBfbWVtKTsNCj4+Pj4gICAgDQo+Pj4+IC1zdHJ1Y3QgbWFwX3JhbmdlX2RhdGENCj4+
Pj4gLXsNCj4+Pj4gLSAgICBzdHJ1Y3QgZG9tYWluICpkOw0KPj4+PiAtICAgIHAybV90eXBlX3Qg
cDJtdDsNCj4+Pj4gLX07DQo+Pj4+IC0NCj4+Pj4gICAgLyogT3ZlcnJpZGUgbWFjcm9zIGZyb20g
YXNtL3BhZ2UuaCB0byBtYWtlIHRoZW0gd29yayB3aXRoIG1mbl90ICovDQo+Pj4+ICAgICN1bmRl
ZiB2aXJ0X3RvX21mbg0KPj4+PiAgICAjZGVmaW5lIHZpcnRfdG9fbWZuKHZhKSBfbWZuKF9fdmly
dF90b19tZm4odmEpKQ0KPj4+PiBAQCAtMTM4OCw5ICsxMzgxLDggQEAgc3RhdGljIGludCBfX2lu
aXQgbWFwX2R0X2lycV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYs
DQo+Pj4+ICAgICAgICByZXR1cm4gMDsNCj4+Pj4gICAgfQ0KPj4+PiAgICANCj4+Pj4gLXN0YXRp
YyBpbnQgX19pbml0IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2RldmljZV9u
b2RlICpkZXYsDQo+Pj4+IC0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHU2
NCBhZGRyLCB1NjQgbGVuLA0KPj4+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICB2b2lkICpkYXRhKQ0KPj4+PiAraW50IF9faW5pdCBtYXBfcmFuZ2VfdG9fZG9tYWluKGNv
bnN0IHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LA0KPj4+PiArICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHU2NCBhZGRyLCB1NjQgbGVuLCB2b2lkICpkYXRhKQ0KPj4+PiAgICB7DQo+
Pj4+ICAgICAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgKm1yX2RhdGEgPSBkYXRhOw0KPj4+PiAg
ICAgICAgc3RydWN0IGRvbWFpbiAqZCA9IG1yX2RhdGEtPmQ7DQo+Pj4+IEBAIC0xNDE3LDYgKzE0
MDksMTMgQEAgc3RhdGljIGludCBfX2luaXQgbWFwX3JhbmdlX3RvX2RvbWFpbihjb25zdCBzdHJ1
Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+Pj4gICAgICAgICAgICB9DQo+Pj4+ICAgICAgICB9
DQo+Pj4+ICAgIA0KPj4+PiArI2lmZGVmIENPTkZJR19IQVNfUENJDQo+Pj4+ICsgICAgaWYgKCBp
c19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCgpICYmDQo+Pj4+ICsgICAgICAgICAoZGV2aWNlX2dl
dF9jbGFzcyhkZXYpID09IERFVklDRV9QQ0kpICYmDQo+Pj4+ICsgICAgICAgICAhbXJfZGF0YS0+
bWFwX3BjaV9icmlkZ2UgKQ0KPj4+PiArICAgICAgICBuZWVkX21hcHBpbmcgPSBmYWxzZTsNCj4+
Pj4gKyNlbmRpZg0KPj4+IFdpdGggdGhlIGNoYW5nZSBJIHN1Z2dlc3RlZCBiZWxvdyB0dXJuaW5n
IG1hcF9wY2lfYnJpZGdlIGludG8NCj4+PiBza2lwX21hcHBpbmcsIHRoZW4gdGhpcyBjaGVjayBj
b3VsZCBnbyBhd2F5IGlmIHdlIGp1c3Qgc2V0IG5lZWRfbWFwcGluZw0KPj4+IGFzIGZvbGxvd3M6
DQo+Pj4NCj4+PiBib29sIG5lZWRfbWFwcGluZyA9ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJvdWdo
KGRldikgJiYNCj4+PiAgICAgICAgICAgICAgICAgICAgICAgIW1yX2RhdGEtPnNraXBfbWFwcGlu
ZzsNCj4+IE5vdCBleGFjdGx5LiBUaGlzIGNoZWNrLCBlLmcuDQo+Pg0KPj4gImlzX3BjaV9wYXNz
dGhyb3VnaF9lbmFibGVkKCkgJiYgKGRldmljZV9nZXRfY2xhc3MoZGV2KSA9PSBERVZJQ0VfUENJ
KSINCj4+DQo+PiByZWFsbHkgcHJvdGVjdHMgdXMgZnJvbSBtYXBwaW5nIGFueSBvZiB0aGUgcmFu
Z2VzIGJlbG9uZ2luZyB0byBhIFBDSSBkZXZpY2U6DQo+Pg0KPj4gd2Ugc2NhbiB0aGUgZGV2aWNl
IHRyZWUgYW5kIGZvciBlYWNoIG5vZGUgd2UgY2FsbCBtYXBfcmFuZ2VfdG9fZG9tYWluDQo+Pg0K
Pj4gd2l0aCBza2lwX21hcHBpbmcgPT0gZmFsc2UgKGl0IGlzIGNhbGxlZCBmcm9tIG1hcF9kZXZp
Y2VfY2hpbGRyZW4pLg0KPj4NCj4+IFNvLCBpZiB0aGVyZSBpcyBubyBjaGVjayB0aGVuIHRoZSBt
YXBwaW5nIGlzIHBlcmZvcm1lZCBldmVuIGZvciBQQ0kgZGV2aWNlcw0KPj4NCj4+IHdoaWNoIHdl
IGRvIG5vdCB3YW50Lg0KPj4NCj4+IEJ1dCwgeWVzIHdlIGNhbiBzaW1wbGlmeSB0aGUgbG9naWMg
dG86DQo+Pg0KPj4gYm9vbCBuZWVkX21hcHBpbmcgPSAhZHRfZGV2aWNlX2Zvcl9wYXNzdGhyb3Vn
aChkZXYpICYmDQo+PiAgIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICFt
cl9kYXRhLT5za2lwX21hcHBpbmc7DQo+Pg0KPj4gI2lmZGVmIENPTkZJR19IQVNfUENJDQo+PiAg
IMKgwqDCoCBpZiAoIG5lZWRfbWFwcGluZyAmJiBpc19wY2lfcGFzc3Rocm91Z2hfZW5hYmxlZCgp
ICYmDQo+PiAgIMKgwqDCoMKgwqDCoMKgwqAgKGRldmljZV9nZXRfY2xhc3MoZGV2KSA9PSBERVZJ
Q0VfUENJKSApDQo+PiAgIMKgwqDCoMKgwqDCoMKgIG5lZWRfbWFwcGluZyA9IGZhbHNlOw0KPj4g
I2VuZGlmDQo+Pg0KPj4gYnV0IEkgc2VlIG5vIGJpZyBwcm9maXQgZnJvbSBpdC4NCj4gU29ycnkg
SSBkaWRuJ3QgZm9sbG93IHlvdXIgZXhwbGFuYXRpb24uDQo+DQo+IE15IHN1Z2dlc3Rpb24gaXMg
dG8gcmVtb3ZlIHRoZSAjaWZkZWYgQ09ORklHX0hBU19QQ0kgY29tcGxldGVseSBmcm9tDQo+IG1h
cF9yYW5nZV90b19kb21haW4uIEF0IHRoZSBiZWdpbm5pbmcgb2YgbWFwX3JhbmdlX3RvX2RvbWFp
biwgdGhlcmUgaXMNCj4gYWxyZWFkeSB0aGlzIGxpbmU6DQo+DQo+IGJvb2wgbmVlZF9tYXBwaW5n
ID0gIWR0X2RldmljZV9mb3JfcGFzc3Rocm91Z2goZGV2KTsNCj4NCj4gV2UgY2FuIGNoYW5nZSBp
dCBpbnRvOg0KPg0KPiBib29sIG5lZWRfbWFwcGluZyA9ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJv
dWdoKGRldikgJiYNCj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICFt
cl9kYXRhLT5za2lwX21hcHBpbmc7DQo+DQo+DQo+IFRoZW4sIGluIG1hcF9kZXZpY2VfY2hpbGRy
ZW4gd2UgY2FuIHNldCBtcl9kYXRhLT5za2lwX21hcHBpbmcgdG8gdHJ1ZQ0KPiBmb3IgUENJIGRl
dmljZXMuDQoNClRoaXMgaXMgdGhlIGtleS4gSSBhbSBmaW5lIHdpdGggdGhpcywgYnV0IGl0IGp1
c3QgbWVhbnMgd2UgbW92ZSB0aGUNCg0KY2hlY2sgdG8gdGhlIG91dHNpZGUgb2YgdGhpcyBmdW5j
dGlvbiB3aGljaCBsb29rcyBnb29kLiBXaWxsIGRvDQoNCj4gICBUaGVyZSBpcyBhbHJlYWR5IGEg
cGNpIGNoZWNrIHRoZXJlOg0KPg0KPiAgIGlmICggZHRfZGV2aWNlX3R5cGVfaXNfZXF1YWwoZGV2
LCAicGNpIikgKQ0KPg0KPiBzbyBpdCBzaG91bGQgYmUgZWFzeSB0byBkby4gV2hhdCBhbSBJIG1p
c3Npbmc/DQo+DQo+DQo+DQo+Pj4+ICAgICAgICBpZiAoIG5lZWRfbWFwcGluZyApDQo+Pj4+ICAg
ICAgICB7DQo+Pj4+ICAgICAgICAgICAgcmVzID0gbWFwX3JlZ2lvbnNfcDJtdChkLA0KPj4+PiBA
QCAtMTQ1MCw3ICsxNDQ5LDExIEBAIHN0YXRpYyBpbnQgX19pbml0IG1hcF9kZXZpY2VfY2hpbGRy
ZW4oc3RydWN0IGRvbWFpbiAqZCwNCj4+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+Pj4gICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1fdHlwZV90IHAybXQpDQo+Pj4+
ICAgIHsNCj4+Pj4gLSAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSA9IHsgLmQgPSBk
LCAucDJtdCA9IHAybXQgfTsNCj4+Pj4gKyAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0
YSA9IHsNCj4+Pj4gKyAgICAgICAgLmQgPSBkLA0KPj4+PiArICAgICAgICAucDJtdCA9IHAybXQs
DQo+Pj4+ICsgICAgICAgIC5tYXBfcGNpX2JyaWRnZSA9IGZhbHNlDQo+Pj4+ICsgICAgfTsNCj4+
Pj4gICAgICAgIGludCByZXQ7DQo+Pj4+ICAgIA0KPj4+PiAgICAgICAgaWYgKCBkdF9kZXZpY2Vf
dHlwZV9pc19lcXVhbChkZXYsICJwY2kiKSApDQo+Pj4+IEBAIC0xNTgyLDcgKzE1ODUsMTEgQEAg
c3RhdGljIGludCBfX2luaXQgaGFuZGxlX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLCBzdHJ1Y3Qg
ZHRfZGV2aWNlX25vZGUgKmRldiwNCj4+Pj4gICAgICAgIC8qIEdpdmUgcGVybWlzc2lvbiBhbmQg
bWFwIE1NSU9zICovDQo+Pj4+ICAgICAgICBmb3IgKCBpID0gMDsgaSA8IG5hZGRyOyBpKysgKQ0K
Pj4+PiAgICAgICAgew0KPj4+PiAtICAgICAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0
YSA9IHsgLmQgPSBkLCAucDJtdCA9IHAybXQgfTsNCj4+Pj4gKyAgICAgICAgc3RydWN0IG1hcF9y
YW5nZV9kYXRhIG1yX2RhdGEgPSB7DQo+Pj4+ICsgICAgICAgICAgICAuZCA9IGQsDQo+Pj4+ICsg
ICAgICAgICAgICAucDJtdCA9IHAybXQsDQo+Pj4+ICsgICAgICAgICAgICAubWFwX3BjaV9icmlk
Z2UgPSBmYWxzZQ0KPj4+PiArICAgICAgICB9Ow0KPj4+PiAgICAgICAgICAgIHJlcyA9IGR0X2Rl
dmljZV9nZXRfYWRkcmVzcyhkZXYsIGksICZhZGRyLCAmc2l6ZSk7DQo+Pj4+ICAgICAgICAgICAg
aWYgKCByZXMgKQ0KPj4+PiAgICAgICAgICAgIHsNCj4+Pj4gQEAgLTI3NTQsNyArMjc2MSwxNCBA
QCBzdGF0aWMgaW50IF9faW5pdCBjb25zdHJ1Y3RfZG9tMChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4+
PiAgICAgICAgICAgIHJldHVybiByYzsNCj4+Pj4gICAgDQo+Pj4+ICAgICAgICBpZiAoIGFjcGlf
ZGlzYWJsZWQgKQ0KPj4+PiArICAgIHsNCj4+Pj4gICAgICAgICAgICByYyA9IHByZXBhcmVfZHRi
X2h3ZG9tKGQsICZraW5mbyk7DQo+Pj4+ICsjaWZkZWYgQ09ORklHX0hBU19QQ0kNCj4+Pj4gKyAg
ICAgICAgaWYgKCByYyA8IDAgKQ0KPj4+PiArICAgICAgICAgICAgcmV0dXJuIHJjOw0KPj4+IFRo
aXMgZG9lc24ndCBsb29rIGdyZWF0IDotKQ0KPj4+DQo+Pj4gSSB3b3VsZCBtb3ZlIHRoZSBjYWxs
IHRvIHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5ncygpIGJlbG93IGp1c3QgYmVmb3JlDQo+Pj4gY29u
c3RydWN0X2RvbWFpbi4NCj4+IEkgcHV0IGl0IHRoZXJlIGZvciBwdXJwb3NlOiBjdXJyZW50bHkg
d2Ugb25seSBzdXBwb3J0IGRldmljZS10cmVlIGFuZA0KPj4NCj4+IEFDUEkgaXMgbm90IGNvdmVy
ZWQsIGUuZy4gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzIGlzIGltcGxlbWVudGVkDQo+Pg0KPj4g
d2l0aCBkZXZpY2UtdHJlZSBpbiBtaW5kLiBTbywgSSBkZWNpZGVkIHRvIHRpZSBpdCB0byBwcmVw
YXJlX2R0Yl9od2RvbQ0KPj4NCj4+IHdoaWNoIGlzIGNhbGxlZCB3aGVuIGFjcGlfZGlzYWJsZWQg
aXMgdHJ1ZS4NCj4gVGhhdCdzIE9LLCBJIGRvbid0IG1pbmQuIE15IGNvbW1lbnQgd2FzIHB1cmVs
eSAiY29kZSBhZXN0aGV0aWMiLiBJIHRoaW5rDQo+IHRoaXMgd291bGQgbG9vayBiZXR0ZXI6DQo+
DQo+ICAgICAgaWYgKCBhY3BpX2Rpc2FibGVkICkNCj4gICAgICAgICAgcmMgPSBwcmVwYXJlX2R0
Yl9od2RvbShkLCAma2luZm8pOw0KPiAgICAgIGVsc2UNCj4gICAgICAgICAgcmMgPSBwcmVwYXJl
X2FjcGkoZCwgJmtpbmZvKTsNCj4NCj4gICAgICBpZiAoIHJjIDwgMCApDQo+ICAgICAgICAgIHJl
dHVybiByYzsNCj4NCj4gI2lmZGVmIENPTkZJR19IQVNfUENJDQo+ICAgICAgaWYgKCBhY3BpX2Rp
c2FibGVkICkNCj4gICAgICAgICAgcmMgPSBwY2lfaG9zdF9icmlkZ2VfbWFwcGluZ3MoZCwgcDJt
X21taW9fZGlyZWN0X2MpOw0KPiAgICAgIGlmICggcmMgPCAwICkNCj4gICAgICAgICAgcmV0dXJu
IHJjOw0KPiAjZW5kaWYNCj4NCj4NCj4gT3IgZXZlbiB0aGlzIHdvdWxkIGxvb2sgYmV0dGVyOg0K
Pg0KPiAgICAgIGlmICggYWNwaV9kaXNhYmxlZCApDQo+ICAgICAgew0KPiAgICAgICAgICByYyA9
IHByZXBhcmVfZHRiX2h3ZG9tKGQsICZraW5mbyk7DQo+ICAgICAgICAgIGlmICggcmMgPCAwICkN
Cj4gICAgICAgICAgICAgIHJldHVybiByYzsNCj4gI2lmZGVmIENPTkZJR19IQVNfUENJDQo+ICAg
ICAgICAgIHJjID0gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKGQsIHAybV9tbWlvX2RpcmVjdF9j
KTsNCj4gI2VuZGlmDQo+ICAgICAgfSBlbHNlDQo+ICAgICAgICAgIHJjID0gcHJlcGFyZV9hY3Bp
KGQsICZraW5mbyk7DQo+DQo+ICAgICAgaWYgKCByYyA8IDAgKQ0KPiAgICAgICAgICByZXR1cm4g
cmM7DQo+DQo+DQo+IEdpdmVuIHRoYXQgdGhpcyBjb21tZW50IGlzIG5vdCBhYm91dCBmdW5jdGlv
bmFsaXR5IGJ1dCBvbmx5IGFib3V0IGhvdw0KPiB0aGUgY29kZSBsb29rcyBsaWtlIEkgd29uJ3Qg
aW5zaXN0Lg0KT2ssIEknbGwgcmUtYXJyYW5nZSB0aGUgY29kZQ0KPg0KPg0KPg0KPj4+PiArICAg
ICAgICByYyA9IHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5ncyhkLCBwMm1fbW1pb19kaXJlY3RfYyk7
DQo+Pj4+ICsjZW5kaWYNCj4+Pj4gKyAgICB9DQo+Pj4+ICAgICAgICBlbHNlDQo+Pj4+ICAgICAg
ICAgICAgcmMgPSBwcmVwYXJlX2FjcGkoZCwgJmtpbmZvKTsNCj4+Pj4gICAgDQo+Pj4+IGRpZmYg
LS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5j
DQo+Pj4+IGluZGV4IDliODhiMWNlZGFhMi4uZWFlMTc3ZjJjYmMyIDEwMDY0NA0KPj4+PiAtLS0g
YS94ZW4vYXJjaC9hcm0vcGNpL2VjYW0uYw0KPj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGNpL2Vj
YW0uYw0KPj4+PiBAQCAtMzksNiArMzksMTkgQEAgdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBf
YnVzKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+Pj4gICAgICAgIHJldHVybiBi
YXNlICsgKFBDSV9ERVZGTjIoc2JkZikgPDwgZGV2Zm5fc2hpZnQpICsgd2hlcmU7DQo+Pj4+ICAg
IH0NCj4+Pj4gICAgDQo+Pj4+ICtib29sIHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmcoc3RydWN0
IGRvbWFpbiAqZCwNCj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qg
cGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdWludDY0X3QgYWRkcikNCj4+Pj4gK3sNCj4+Pj4gKyAgICBzdHJ1Y3QgcGNpX2NvbmZp
Z193aW5kb3cgKmNmZyA9IGJyaWRnZS0+Y2ZnOw0KPj4+PiArDQo+Pj4+ICsgICAgLyoNCj4+Pj4g
KyAgICAgKiBXZSBkbyBub3Qgd2FudCBFQ0FNIGFkZHJlc3Mgc3BhY2UgdG8gYmUgbWFwcGVkIGlu
IERvbWFpbi0wJ3MgcDJtLA0KPj4+PiArICAgICAqIHNvIHdlIGNhbiB0cmFwIGFjY2VzcyB0byBp
dC4NCj4+Pj4gKyAgICAgKi8NCj4+Pj4gKyAgICByZXR1cm4gY2ZnLT5waHlzX2FkZHIgIT0gYWRk
cjsNCj4+Pj4gK30NCj4+Pj4gKw0KPj4+PiAgICAvKiBFQ0FNIG9wcyAqLw0KPj4+PiAgICBjb25z
dCBzdHJ1Y3QgcGNpX2VjYW1fb3BzIHBjaV9nZW5lcmljX2VjYW1fb3BzID0gew0KPj4+PiAgICAg
ICAgLmJ1c19zaGlmdCAgPSAyMCwNCj4+Pj4gQEAgLTQ2LDYgKzU5LDcgQEAgY29uc3Qgc3RydWN0
IHBjaV9lY2FtX29wcyBwY2lfZ2VuZXJpY19lY2FtX29wcyA9IHsNCj4+Pj4gICAgICAgICAgICAu
bWFwX2J1cyAgICAgICAgICAgICAgICA9IHBjaV9lY2FtX21hcF9idXMsDQo+Pj4+ICAgICAgICAg
ICAgLnJlYWQgICAgICAgICAgICAgICAgICAgPSBwY2lfZ2VuZXJpY19jb25maWdfcmVhZCwNCj4+
Pj4gICAgICAgICAgICAud3JpdGUgICAgICAgICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZp
Z193cml0ZSwNCj4+Pj4gKyAgICAgICAgLm5lZWRfcDJtX21hcHBpbmcgICAgICAgPSBwY2lfZWNh
bV9uZWVkX3AybV9tYXBwaW5nLA0KPj4+PiAgICAgICAgfQ0KPj4+PiAgICB9Ow0KPj4+PiAgICAN
Cj4+Pj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMgYi94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jDQo+Pj4+IGluZGV4IDE1NWYyYTI3NDNh
Zi4uZjM1MDgyNmVhMjZiIDEwMDY0NA0KPj4+PiAtLS0gYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1o
b3N0LWNvbW1vbi5jDQo+Pj4+ICsrKyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9u
LmMNCj4+Pj4gQEAgLTE4LDYgKzE4LDcgQEANCj4+Pj4gICAgDQo+Pj4+ICAgICNpbmNsdWRlIDx4
ZW4vaW5pdC5oPg0KPj4+PiAgICAjaW5jbHVkZSA8eGVuL3BjaS5oPg0KPj4+PiArI2luY2x1ZGUg
PGFzbS9zZXR1cC5oPg0KPj4+PiAgICAjaW5jbHVkZSA8eGVuL3J3bG9jay5oPg0KPj4+PiAgICAj
aW5jbHVkZSA8eGVuL3NjaGVkLmg+DQo+Pj4+ICAgICNpbmNsdWRlIDx4ZW4vdm1hcC5oPg0KPj4+
PiBAQCAtMzI4LDYgKzMyOSw1MyBAQCBpbnQgcGNpX2hvc3RfZ2V0X251bV9icmlkZ2VzKHZvaWQp
DQo+Pj4+ICAgICAgICByZXR1cm4gY291bnQ7DQo+Pj4+ICAgIH0NCj4+Pj4gICAgDQo+Pj4+ICtp
bnQgX19pbml0IHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5ncyhzdHJ1Y3QgZG9tYWluICpkLCBwMm1f
dHlwZV90IHAybXQpDQo+Pj4+ICt7DQo+Pj4+ICsgICAgc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAq
YnJpZGdlOw0KPj4+PiArICAgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSBtcl9kYXRhID0gew0KPj4+
PiArICAgICAgICAuZCA9IGQsDQo+Pj4+ICsgICAgICAgIC5wMm10ID0gcDJtdCwNCj4+Pj4gKyAg
ICAgICAgLm1hcF9wY2lfYnJpZGdlID0gdHJ1ZQ0KPj4+PiArICAgIH07DQo+Pj4+ICsNCj4+Pj4g
KyAgICAvKg0KPj4+PiArICAgICAqIEZvciBlYWNoIFBDSSBob3N0IGJyaWRnZSB3ZSBuZWVkIHRv
IG9ubHkgbWFwIHRob3NlIHJhbmdlcw0KPj4+PiArICAgICAqIHdoaWNoIGFyZSB1c2VkIGJ5IERv
bWFpbi0wIHRvIHByb3Blcmx5IGluaXRpYWxpemUgdGhlIGJyaWRnZSwNCj4+Pj4gKyAgICAgKiBl
LmcuIHdlIGRvIG5vdCB3YW50IHRvIG1hcCBFQ0FNIGNvbmZpZ3VyYXRpb24gc3BhY2Ugd2hpY2gg
bGl2ZXMgaW4NCj4+Pj4gKyAgICAgKiAicmVnIiBvciAiYXNzaWduZWQtYWRkcmVzc2VzIiBkZXZp
Y2UgdHJlZSBwcm9wZXJ0eS4NCj4+Pj4gKyAgICAgKiBOZWl0aGVyIHdlIHdhbnQgdG8gbWFwIGFu
eSBvZiB0aGUgTU1JTyByYW5nZXMgZm91bmQgaW4gdGhlICJyYW5nZXMiDQo+Pj4+ICsgICAgICog
ZGV2aWNlIHRyZWUgcHJvcGVydHkuDQo+Pj4+ICsgICAgICovDQo+Pj4+ICsgICAgbGlzdF9mb3Jf
ZWFjaF9lbnRyeSggYnJpZGdlLCAmcGNpX2hvc3RfYnJpZGdlcywgbm9kZSApDQo+Pj4+ICsgICAg
ew0KPj4+PiArICAgICAgICBjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiA9IGJyaWRn
ZS0+ZHRfbm9kZTsNCj4+Pj4gKyAgICAgICAgaW50IGk7DQo+Pj4gaSBzaG91bGQgYmUgdW5zaWdu
ZWQgaW50DQo+PiBPaw0KPj4+DQo+Pj4+ICsgICAgICAgIGZvciAoIGkgPSAwOyBpIDwgZHRfbnVt
YmVyX29mX2FkZHJlc3MoZGV2KTsgaSsrICkNCj4+Pj4gKyAgICAgICAgew0KPj4+PiArICAgICAg
ICAgICAgdWludDY0X3QgYWRkciwgc2l6ZTsNCj4+Pj4gKyAgICAgICAgICAgIGludCBlcnI7DQo+
Pj4+ICsNCj4+Pj4gKyAgICAgICAgICAgIGVyciA9IGR0X2RldmljZV9nZXRfYWRkcmVzcyhkZXYs
IGksICZhZGRyLCAmc2l6ZSk7DQo+Pj4+ICsgICAgICAgICAgICBpZiAoIGVyciApDQo+Pj4+ICsg
ICAgICAgICAgICB7DQo+Pj4+ICsgICAgICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19FUlIgIlVu
YWJsZSB0byByZXRyaWV2ZSBhZGRyZXNzICV1IGZvciAlc1xuIiwNCj4+PiBNYXliZSByZXBocmFz
ZSBpdCB0bzoNCj4+Pg0KPj4+ICAgICAgIFVuYWJsZSB0byByZXRyaWV2ZSBhZGRyZXNzIHJhbmdl
IGluZGV4PSV1IGZvciAlcw0KPj4gVGhpcyBpcyBhIGNvcHktcGFzdGUgZnJvbSB0aGUgb3JpZ2lu
YWwgY29kZSwgYnV0IG9rDQo+Pj4NCj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgaSwgZHRf
bm9kZV9mdWxsX25hbWUoZGV2KSk7DQo+Pj4+ICsgICAgICAgICAgICAgICAgcmV0dXJuIGVycjsN
Cj4+Pj4gKyAgICAgICAgICAgIH0NCj4+Pj4gKw0KPj4+PiArICAgICAgICAgICAgaWYgKCBicmlk
Z2UtPm9wcy0+bmVlZF9wMm1fbWFwcGluZyhkLCBicmlkZ2UsIGFkZHIpICkNCj4+PiBUaGUgY3Vy
cmVudCBpbXBsZW1lbnRhdGlvbiBvZiBuZWVkX3AybV9tYXBwaW5nIGZpbHRlcnMgb3V0IEVDQU0g
YW5kDQo+Pj4gbm90aGluZyBlbHNlLiBKdXN0IGRvdWJsZS1jaGVja2luZyBoZXJlOiBpcyB0aGVy
ZSBhbnl0aGluZyBlbHNlIHdlDQo+Pj4gc2hvdWxkIGZpbHRlciBvdXQ/IExvb2tpbmcgYXQgdGhl
IGRldmljZSB0cmVlIHBjaSBub2RlIGZvciBaeW5xTVA6DQo+Pj4NCj4+PiAJCQlyZWcgPSA8MHgw
IDB4ZmQwZTAwMDAgMHgwIDB4MTAwMCAweDAgMHhmZDQ4MDAwMCAweDAgMHgxMDAwIDB4ODAgMHgw
IDB4MCAweDEwMDAwMDA+Ow0KPj4+IAkJCXJlZy1uYW1lcyA9ICJicmVnIiwgInBjaXJlZyIsICJj
ZmciOw0KPj4+DQo+Pj4gV2UgYXJlIGZpbHRlcmluZyBvdXQgY2ZnLCBidXQgZG8gd2UgbmVlZCBi
b3RoICJicmVnIiBhbmQgInBjaXJlZyIgaGVyZT8NCj4+IEl0IGlzIHZpY2UgdmVyc2E6IHdlIGFy
ZSBmaWx0ZXJpbmcgb3V0IGNmZyBvbmx5IGFuZCBhbGwgdGhlIHJlc3QgYXJlICJ1bmtub3duIHJl
Z2lvbnMgd2UgZG8gbm90DQo+Pg0KPj4gd2FudCB0byBhbHRlciIuDQo+IEFoLCBPSy4gQ2FuIHlv
dSBwbGVhc2UgYWRkIGEgbm90ZSBhYm91dCB0aGlzIHRvIHRoZSBpbi1jb2RlIGNvbW1lbnQ/DQo+
IE1heWJlIGFzIGZvbGxvd3M6DQo+DQo+IEZvciBlYWNoIFBDSSBob3N0IGJyaWRnZSB3ZSBuZWVk
IHRvIG9ubHkgbWFwIHRob3NlIHJhbmdlcyB3aGljaCBhcmUgdXNlZA0KPiBieSBEb21haW4tMCB0
byBwcm9wZXJseSBpbml0aWFsaXplIHRoZSBicmlkZ2UsIGUuZy4gd2UgZG8gbm90IHdhbnQgdG8N
Cj4gbWFwIEVDQU0gY29uZmlndXJhdGlvbiBzcGFjZSB3aGljaCBsaXZlcyBpbiAicmVnIiBvcg0K
PiAiYXNzaWduZWQtYWRkcmVzc2VzIiBkZXZpY2UgdHJlZSBwcm9wZXJ0eSwgYnV0IHdlIHdhbnQg
dG8gbWFwIG90aGVyDQo+IHJlZ2lvbnMgb2YgdGhlIGhvc3QgYnJpZGdlLiBUaGUgUENJIGFwZXJ0
dXJlIGRlZmluZWQgYnkgdGhlICJyYW5nZXMiDQo+IGRldmljZSB0cmVlIHByb3BlcnR5IHNob3Vs
ZCBhbHNvIGJlIHNraXBwZWQuDQpTdXJlLCB3aWxsIGFkZA0KPg0KPg0KPj4+IElmIG5vdCwgZG8g
d2UgbmVlZCBhbm90aGVyIGZ1bmN0aW9uIGxpa2UgLmNmZ19yZWdfaW5kZXggdG8ga25vdyB3aGF0
IHdlDQo+Pj4gYWN0dWFsbHkgbmVlZCB0byBtYXA/DQo+PiBJIHdhcyB0aGlua2luZyB0byB1c2Ug
LmNmZ19yZWdfaW5kZXggZmlyIHRoYXQsIGJ1dCBpdCBtZWFucyBJJ2xsIG5lZWQgdG8gdHJhdmVy
c2UNCj4+DQo+PiB0aGUgZGV2aWNlLXRyZWUgdG8gZ2V0IHRoZSB2YWx1ZSBmb3IgLmNmZ19yZWdf
aW5kZXggd2hpY2ggaXMgYWxyZWFkeSBrbm93bg0KPj4NCj4+IHRvIHRoZSBicmlkZ2UuIFNvLCBp
dCBpcyBjaGVhcGVyIHRvIGhhdmUgYSBjYWxsYmFjayBhbmQganVzdCBjaGVjayB0aGF0DQo+Pg0K
Pj4gY2ZnLT5waHlzX2FkZHIgIT0gYWRkciwgZS5nLiB3aGF0IHdlIHdhbnQgdG8gbWFwIGlzIG5v
dCBjZmcuDQo+IFllYWggdGhhdCBtYWtlcyBzZW5zZQ0KPg0KPg0KPj4+PiArICAgICAgICAgICAg
ew0KPj4+PiArICAgICAgICAgICAgICAgIGVyciA9IG1hcF9yYW5nZV90b19kb21haW4oZGV2LCBh
ZGRyLCBzaXplLCAmbXJfZGF0YSk7DQo+Pj4+ICsgICAgICAgICAgICAgICAgaWYgKCBlcnIgKQ0K
Pj4+PiArICAgICAgICAgICAgICAgICAgICByZXR1cm4gZXJyOw0KPj4+PiArICAgICAgICAgICAg
fQ0KPj4+PiArICAgICAgICB9DQo+Pj4+ICsgICAgfQ0KPj4+PiArDQo+Pj4+ICsgICAgcmV0dXJu
IDA7DQo+Pj4+ICt9DQo+Pj4+ICsNCj4+Pj4gICAgLyoNCj4+Pj4gICAgICogTG9jYWwgdmFyaWFi
bGVzOg0KPj4+PiAgICAgKiBtb2RlOiBDDQo+Pj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0v
cGNpL3BjaS1ob3N0LXp5bnFtcC5jIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC16eW5xbXAu
Yw0KPj4+PiBpbmRleCBjMjdiNGVhOWYwMmYuLmFkYmUzNjI3ODcxZiAxMDA2NDQNCj4+Pj4gLS0t
IGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC16eW5xbXAuYw0KPj4+PiArKysgYi94ZW4vYXJj
aC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jDQo+Pj4+IEBAIC0zMyw2ICszMyw3IEBAIGNvbnN0
IHN0cnVjdCBwY2lfZWNhbV9vcHMgbndsX3BjaWVfb3BzID0gew0KPj4+PiAgICAgICAgICAgIC5t
YXBfYnVzICAgICAgICAgICAgICAgID0gcGNpX2VjYW1fbWFwX2J1cywNCj4+Pj4gICAgICAgICAg
ICAucmVhZCAgICAgICAgICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZpZ19yZWFkLA0KPj4+
PiAgICAgICAgICAgIC53cml0ZSAgICAgICAgICAgICAgICAgID0gcGNpX2dlbmVyaWNfY29uZmln
X3dyaXRlLA0KPj4+PiArICAgICAgICAubmVlZF9wMm1fbWFwcGluZyAgICAgICA9IHBjaV9lY2Ft
X25lZWRfcDJtX21hcHBpbmcsDQo+Pj4+ICAgICAgICB9DQo+Pj4+ICAgIH07DQo+Pj4+ICAgIA0K
Pj4+PiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaCBiL3hlbi9pbmNsdWRl
L2FzbS1hcm0vcGNpLmgNCj4+Pj4gaW5kZXggNzYxOGYwYjY3MjViLi5iODFmNjZlODEzZWYgMTAw
NjQ0DQo+Pj4+IC0tLSBhL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCj4+Pj4gKysrIGIveGVu
L2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KPj4+PiBAQCAtMTksNiArMTksOCBAQA0KPj4+PiAgICAN
Cj4+Pj4gICAgI2lmZGVmIENPTkZJR19IQVNfUENJDQo+Pj4+ICAgIA0KPj4+PiArI2luY2x1ZGUg
PGFzbS9wMm0uaD4NCj4+Pj4gKw0KPj4+PiAgICAjZGVmaW5lIHBjaV90b19kZXYocGNpZGV2KSAo
JihwY2lkZXYpLT5hcmNoLmRldikNCj4+Pj4gICAgI2RlZmluZSBQUklfcGNpICIlMDR4OiUwMng6
JTAyeC4ldSINCj4+Pj4gICAgDQo+Pj4+IEBAIC03OSw2ICs4MSw5IEBAIHN0cnVjdCBwY2lfb3Bz
IHsNCj4+Pj4gICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWludDMyX3QgbGVuLCB1
aW50MzJfdCAqdmFsdWUpOw0KPj4+PiAgICAgICAgaW50ICgqd3JpdGUpKHN0cnVjdCBwY2lfaG9z
dF9icmlkZ2UgKmJyaWRnZSwgdWludDMyX3Qgc2JkZiwNCj4+Pj4gICAgICAgICAgICAgICAgICAg
ICB1aW50MzJfdCByZWcsIHVpbnQzMl90IGxlbiwgdWludDMyX3QgdmFsdWUpOw0KPj4+PiArICAg
IGJvb2wgKCpuZWVkX3AybV9tYXBwaW5nKShzdHJ1Y3QgZG9tYWluICpkLA0KPj4+PiArICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcGNpX2hvc3RfYnJpZGdlICpicmlkZ2UsDQo+
Pj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQ2NF90IGFkZHIpOw0KPj4+IEkg
d291bGQgY2FsbCB0aGlzIGZ1bmN0aW9uOiBuZWVkX3AybV9od2RvbV9tYXBwaW5nDQo+PiBPaw0K
Pj4+DQo+Pj4+ICAgIH07DQo+Pj4+ICAgIA0KPj4+PiAgICAvKg0KPj4+PiBAQCAtMTAyLDYgKzEw
Nyw5IEBAIGludCBwY2lfZ2VuZXJpY19jb25maWdfd3JpdGUoc3RydWN0IHBjaV9ob3N0X2JyaWRn
ZSAqYnJpZGdlLCB1aW50MzJfdCBzYmRmLA0KPj4+PiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdWludDMyX3QgcmVnLCB1aW50MzJfdCBsZW4sIHVpbnQzMl90IHZhbHVlKTsNCj4+Pj4g
ICAgdm9pZCBfX2lvbWVtICpwY2lfZWNhbV9tYXBfYnVzKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2Ug
KmJyaWRnZSwNCj4+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVpbnQzMl90
IHNiZGYsIHVpbnQzMl90IHdoZXJlKTsNCj4+Pj4gK2Jvb2wgcGNpX2VjYW1fbmVlZF9wMm1fbWFw
cGluZyhzdHJ1Y3QgZG9tYWluICpkLA0KPj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCj4+Pj4gKyAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1aW50NjRfdCBhZGRyKTsNCj4+Pj4gICAgc3RydWN0IHBjaV9ob3N0
X2JyaWRnZSAqcGNpX2ZpbmRfaG9zdF9icmlkZ2UodWludDE2X3Qgc2VnbWVudCwgdWludDhfdCBi
dXMpOw0KPj4+PiAgICBpbnQgcGNpX2dldF9ob3N0X2JyaWRnZV9zZWdtZW50KGNvbnN0IHN0cnVj
dCBkdF9kZXZpY2Vfbm9kZSAqbm9kZSwNCj4+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICB1aW50MTZfdCAqc2VnbWVudCk7DQo+Pj4+IEBAIC0xMTYsNiArMTI0LDcgQEAgaW50
IHBjaV9ob3N0X2l0ZXJhdGVfYnJpZGdlcyhzdHJ1Y3QgZG9tYWluICpkLA0KPj4+PiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGludCAoKmNsYikoc3RydWN0IGRvbWFpbiAqZCwNCj4+
Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lf
aG9zdF9icmlkZ2UgKmJyaWRnZSkpOw0KPj4+PiAgICBpbnQgcGNpX2hvc3RfZ2V0X251bV9icmlk
Z2VzKHZvaWQpOw0KPj4+PiAraW50IHBjaV9ob3N0X2JyaWRnZV9tYXBwaW5ncyhzdHJ1Y3QgZG9t
YWluICpkLCBwMm1fdHlwZV90IHAybXQpOw0KPj4+PiAgICAjZWxzZSAgIC8qIUNPTkZJR19IQVNf
UENJKi8NCj4+Pj4gICAgDQo+Pj4+ICAgICNkZWZpbmUgcGNpX3Bhc3N0aHJvdWdoX2VuYWJsZWQg
KGZhbHNlKQ0KPj4+PiBkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9zZXR1cC5oIGIv
eGVuL2luY2x1ZGUvYXNtLWFybS9zZXR1cC5oDQo+Pj4+IGluZGV4IDk1ZGEwYjdhYjljZC4uMjE4
NjNkZDJiYzU4IDEwMDY0NA0KPj4+PiAtLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJtL3NldHVwLmgN
Cj4+Pj4gKysrIGIveGVuL2luY2x1ZGUvYXNtLWFybS9zZXR1cC5oDQo+Pj4+IEBAIC0yLDYgKzIs
OCBAQA0KPj4+PiAgICAjZGVmaW5lIF9fQVJNX1NFVFVQX0hfDQo+Pj4+ICAgIA0KPj4+PiAgICAj
aW5jbHVkZSA8cHVibGljL3ZlcnNpb24uaD4NCj4+Pj4gKyNpbmNsdWRlIDxhc20vcDJtLmg+DQo+
Pj4+ICsjaW5jbHVkZSA8eGVuL2RldmljZV90cmVlLmg+DQo+Pj4+ICAgIA0KPj4+PiAgICAjZGVm
aW5lIE1JTl9GRFRfQUxJR04gOA0KPj4+PiAgICAjZGVmaW5lIE1BWF9GRFRfU0laRSBTWl8yTQ0K
Pj4+PiBAQCAtNzcsNiArNzksMTQgQEAgc3RydWN0IGJvb3RpbmZvIHsNCj4+Pj4gICAgI2VuZGlm
DQo+Pj4+ICAgIH07DQo+Pj4+ICAgIA0KPj4+PiArc3RydWN0IG1hcF9yYW5nZV9kYXRhDQo+Pj4+
ICt7DQo+Pj4+ICsgICAgc3RydWN0IGRvbWFpbiAqZDsNCj4+Pj4gKyAgICBwMm1fdHlwZV90IHAy
bXQ7DQo+Pj4+ICsgICAgLyogU2V0IGlmIG1hcHBpbmdzIGZvciBQQ0kgaG9zdCBicmlkZ2VzIG11
c3Qgbm90IGJlIHNraXBwZWQuICovDQo+Pj4+ICsgICAgYm9vbCBtYXBfcGNpX2JyaWRnZTsNCj4+
PiBUbyBtYWtlIHRoaXMgbW9yZSBnZW5lcmFsbHkgYXBwbGljYWJsZSwgSSB3b3VsZCBjYWxsIHRo
ZSBuZXcgcHJvcGVydHk6DQo+Pj4NCj4+PiAgICAgYm9vbCBza2lwX21hcHBpbmc7DQo+PiBTb3Vu
ZHMgZ29vZA0KPj4+IGFuZCBpdCBjb3VsZCBhcHBseSB0byBhbnkgY2xhc3Mgb2YgZGV2aWNlcy4g
QWxsIGN1cnJlbnQgdXNlcnMgd291bGQgc2V0DQo+Pj4gaXQgdG8gZmFsc2UgZXhjZXB0IGZvciBw
Y2lfaG9zdF9icmlkZ2VfbWFwcGluZ3MuDQo+PiBQbGVhc2Ugc2VlIFBDSSBzcGVjaWFsIGNhc2Ug
YWJvdmUNCj4+Pg0KPj4+PiArfTsNCj4+Pj4NCj4+Pj4gICAgZXh0ZXJuIHN0cnVjdCBib290aW5m
byBib290aW5mbzsNCj4+Pj4gICAgDQo+Pj4+ICAgIGV4dGVybiBkb21pZF90IG1heF9pbml0X2Rv
bWlkOw0KPj4+PiBAQCAtMTI0LDYgKzEzNCw5IEBAIHZvaWQgZGV2aWNlX3RyZWVfZ2V0X3JlZyhj
b25zdCBfX2JlMzIgKipjZWxsLCB1MzIgYWRkcmVzc19jZWxscywNCj4+Pj4gICAgdTMyIGRldmlj
ZV90cmVlX2dldF91MzIoY29uc3Qgdm9pZCAqZmR0LCBpbnQgbm9kZSwNCj4+Pj4gICAgICAgICAg
ICAgICAgICAgICAgICAgICAgY29uc3QgY2hhciAqcHJvcF9uYW1lLCB1MzIgZGZsdCk7DQo+Pj4+
ICAgIA0KPj4+PiAraW50IG1hcF9yYW5nZV90b19kb21haW4oY29uc3Qgc3RydWN0IGR0X2Rldmlj
ZV9ub2RlICpkZXYsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAgICAgICB1NjQgYWRkciwgdTY0
IGxlbiwgdm9pZCAqZGF0YSk7DQo+Pj4+ICsNCj4+Pj4gICAgI2VuZGlmDQo+Pj4+ICAgIC8qDQo+
Pj4+ICAgICAqIExvY2FsIHZhcmlhYmxlczoNCj4+Pj4gLS0gDQo+Pj4+IDIuMjUuMQ0KPj4+ID4=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:52:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:52:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197429.350458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV56b-0006h1-Vl; Tue, 28 Sep 2021 04:52:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197429.350458; Tue, 28 Sep 2021 04:52:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV56b-0006gu-RD; Tue, 28 Sep 2021 04:52:37 +0000
Received: by outflank-mailman (input) for mailman id 197429;
 Tue, 28 Sep 2021 04:52:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV56Z-0006ac-Qg
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:52:35 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e5f9080e-16d9-431c-bb6b-24e578716788;
 Tue, 28 Sep 2021 04:52:35 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 76DE060FC0;
 Tue, 28 Sep 2021 04:52:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5f9080e-16d9-431c-bb6b-24e578716788
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632804754;
	bh=HipTk94d/rZILtl6Tw0eN64YlEZYb+K+S+IgGjMrL18=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RXpZAg0TvlO1SHsPT8qMX3NmsS3YWweS3UY61jDtDXOY6fJ6A9TdWBWh7qdlI9vvA
	 Ac4WcsKVosHMyu8vhDbfg3w8CvuSl0e2MPyajqvwCFXEogST2ik1PFDxyUlZ1EO/IT
	 ToReWAi2i7nnh7duPJ9X4wTTrb+dKsIFVT3aVpQsjg3S+qUvEwnP8q8iwO7Yhej2t3
	 CegKfELTbUtS+wLun5pg0Gj0HVlFRE+GrB88nE9skGx5gk8q2A7rx04kQVpnFHBb+6
	 jJ8K7alnig17hkAzOywFLFgzeKlGY4YtAkJJvXNIgc1wsEx3cmOSkaNgN/vdhmL0+7
	 ToAXv20qk+9Hw==
Date: Mon, 27 Sep 2021 21:52:34 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Volodymyr Babchuk <volodymyr_babchuk@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    tee-dev@lists.linaro.org
Subject: Re: [PATCH] xen/arm: optee: Fix arm_smccc_smc's a0 for
 OPTEE_SMC_DISABLE_SHM_CACHE
In-Reply-To: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109272151260.5022@sstabellini-ThinkPad-T480s>
References: <1632750850-28600-1-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 27 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Fix a possible copy-paste error in arm_smccc_smc's first argument (a0)
> for OPTEE_SMC_DISABLE_SHM_CACHE case.
> 
> This error causes Linux > v5.14-rc5 (b5c10dd04b7418793517e3286cde5c04759a86de
> optee: Clear stale cache entries during initialization) to stuck
> repeatedly issuing OPTEE_SMC_DISABLE_SHM_CACHE call and waiting for
> the result to be OPTEE_SMC_RETURN_ENOTAVAIL which will never happen.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

I added Fixes: and Backport: tags to the commit


> ---
> I wonder whether this patch wants backporting to the old versions
> since OPTEE support went in.
> ---
>  xen/arch/arm/tee/optee.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 3453615..6df0d44 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -1692,7 +1692,7 @@ static bool optee_handle_call(struct cpu_user_regs *regs)
>          return true;
>  
>      case OPTEE_SMC_DISABLE_SHM_CACHE:
> -        arm_smccc_smc(OPTEE_SMC_ENABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
> +        arm_smccc_smc(OPTEE_SMC_DISABLE_SHM_CACHE, 0, 0, 0, 0, 0, 0,
>                        OPTEE_CLIENT_ID(current->domain), &resp);
>          set_user_reg(regs, 0, resp.a0);
>          if ( resp.a0 == OPTEE_SMC_RETURN_OK ) {
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:53:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:53:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197435.350470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV57T-0007Ka-9S; Tue, 28 Sep 2021 04:53:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197435.350470; Tue, 28 Sep 2021 04:53:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV57T-0007KT-5K; Tue, 28 Sep 2021 04:53:31 +0000
Received: by outflank-mailman (input) for mailman id 197435;
 Tue, 28 Sep 2021 04:53:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV57S-0007KJ-6h
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:53:30 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1ba48712-0782-4cb0-9a67-33cf9cf2c361;
 Tue, 28 Sep 2021 04:53:29 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S4iuCc001921; 
 Tue, 28 Sep 2021 04:53:27 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2056.outbound.protection.outlook.com [104.47.0.56])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbv9pg0h6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 04:53:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3587.eurprd03.prod.outlook.com (2603:10a6:208:44::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 04:53:24 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 04:53:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ba48712-0782-4cb0-9a67-33cf9cf2c361
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DUEGE27lzY1gdjgV7uBtGFTYcgR0jqFxbtYlZiFu78GU0ala+Lwpr/Q/CJHSeweqj++1W8oCO4cSn9nQr2YuUc8xcTBhm8IRP42XIyRT3JX3GxFCcfExKe8Hzh+m9rRceL07nJPDbvBcLv9u1hIXNv7YAktkpjMGBIGYuU8ZAF57GjEcGSIw2X0vcFx4ep7LeFHecSELI3/iqez1p+jk2DRr8lX8YOhBpiekGBH2NCu7D7agXd0lvF5hW3zYf+UFyDbSwusJ70loVHnP7Sdxch4XXkXDDcyp5zu8pxjszFJ7Pm2IyvmbCwRSOw3/rp4E4BxKt/kr7MTPeRxSBdbAiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pUNoblaRZmNAu72mz3oju9rOzk8WG6gR72F8DGKJ3pg=;
 b=MjosxPEAZuO1zOoZ+9x1P02EYKgsxK9Sl1y7pm8WP/BcYqlsVWVO98yBmdmX6PZ5aBSaLUNFG4kGsX1qHyRZWUIrxs9se6bVkzfBhGljLiOeGQxoETqXdnW7RO2mSsfjiLMBD8+ELfwJ21TFrLhzzFmRw7Zq+BnfPIsI1iqLZ6wEx9wmDRlqeNIIUomCMu0AY9Mkvs5subD+nofWHyUbCvAJNkqA6GhDognLOs16qMo5eoBNdRdHnb1HN8XD/2Q7Gwlb8T6Z16U9e8xwRUfdTmNe/kMPJoKJB/HJfDdOKPBolvvHlePZi/iLmbOM4PXGprELZVHsfEHNAEHmLg+r/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pUNoblaRZmNAu72mz3oju9rOzk8WG6gR72F8DGKJ3pg=;
 b=55dk78j2smNJMUrCb71bV6+ggq7fuL88+Vq3ip4XMiLtXbYP+qUXo3bCnRArl32Q96Pz5UsNi9N+DfFIVM7GRMTBpJ8MuCJP/lnGwk+M21dhijSp8UPsmx054244JrGCEu7E61VSLpeGNzOdnM3IGwKY6opdH0d/ZxWnbbUntOROhxuLrR7SA3zbLvvGzTw/3A1WOLlgGlKE3E4i+GnNukghStNyvyH1CgNYxzKEA3A2EbviWaj3msDKVmYvEWGL7pWsDCP8HgO2TlUTFWGFaOCQdlhZ9atlZOlz8gWhV8B/tQ0n3rizgezXXnysQFAimsIi0lO6yic8e7CQhICdHw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
        Juergen Gross
	<jgross@suse.com>
CC: Jan Beulich <jbeulich@suse.com>,
        "boris.ostrovsky@oracle.com"
	<boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbACAAAVGAIABVeSAgAAJ3AA=
Date: Tue, 28 Sep 2021 04:53:24 +0000
Message-ID: <1394cf31-3fdf-a8df-ebaf-90295c47d740@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: cf201820-7316-49d5-978a-08d9823be76d
x-ms-traffictypediagnostic: AM0PR03MB3587:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR03MB35874EE8700471D5AFE90DFEE7A89@AM0PR03MB3587.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 9q+CVxIZhqR0QxgGSWzhknIhJ0RISH3lUpNaY3vp7qeQCsdiCLVN3CVYLl0uhTgcSwtloUj4sc3+EM6AduxlCRf7TAgTNq8LACmtPJim0j8MtbxSnKgpJ36lhEWGB24pIdrMSa37eL6Xsd/sTYN//h0SVvrLv7z/l6uhp0yUtVkwcbIo6ceF/h+9kzfIqroZVnpgMVtQEKjJR6gy4V38gGQBaPGf7+Mi/PufskzNB4Mr3L/1aGcA6nQljSiqS8UQKfYXkg9Q3E0dhhubGd02gOze2ek6LRGGKaZAH17dkzZZSsgYgianVAQBF3pV9gIetS7bQcu8VMmeDunNXgI59vDmpdxT4TksJqPXTwfret+yDuNzQF5+sb4J/rhcD+dpooyF1uLJDGuO+kyFVMzX1wO0llqh2CHTBLlgAgkqEjm5P0LKVewPXMJxImK+PwiO3n64tbYsVVxhLwPR6JNCebrZc07gkzjqI37PNyfa2h3/2GFz9OEqDMt1OQxkqV+Ahno7zyP2XIbcfVU2yoO6/rqXp/dxZD3w1gb6aP54CkERq4OOaPnHbB+44TgJFtofa4Xd2Vo6YqI8vuet631hGcWckPqJdEGQbzeHFxEswaQrBu8GXfagyYhrDJ2xCJQ7Pdoi18ixRkOUrShcL1ce27wIwESNbN4gNz8/TETs07qqS68IFa4kGKD9UxtCR5rcvIO7Hni7WTWPGI+ajE6M20vN102gw/p5eZ27TTNgverC/d53Zoj6jsSdRSZZtbEO
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(38100700002)(8936002)(316002)(122000001)(107886003)(6512007)(6486002)(4326008)(66946007)(186003)(66446008)(66476007)(31686004)(66556008)(64756008)(76116006)(8676002)(91956017)(26005)(54906003)(5660300002)(86362001)(6506007)(71200400001)(38070700005)(83380400001)(36756003)(508600001)(53546011)(31696002)(2616005)(55236004)(2906002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?YkpkWEFYenk1UlVDWVNIUWxMOFBlaWFpbTdkWnhnU1pBK3Q5T3hPeEIzNGdL?=
 =?utf-8?B?Q25WZHVwZ0pJNnVvN3hqbkxhTlQzUWlNOUVRNWp3U3pUUHhob09neFNHYTBt?=
 =?utf-8?B?T3VxbjArZ3V2Rko4NThKSThjTncrRCtzLzJiRWpHcnhTZTVQVUgrQU5GUTU2?=
 =?utf-8?B?bit5bFA4V0EzWlFJWFNIYnIwdEV1RUtCZm9lcEZZSXZCK2luRUFPVjRRYXJM?=
 =?utf-8?B?V1g5MHlheE9vOXNRa0hrWHM2cTBlK0xscXFYOTNWMlZBNWovN01hUEF2aUxo?=
 =?utf-8?B?SSt0TVB1SDQ0ak15V0kzWEZhU0NNN05SVTVXMlpCQ2x1S3I1WVRpdVJvWGV3?=
 =?utf-8?B?d2w2Yzl6NVJseVU4SGNNYXRqdlVheC8yTC9vZ2lQeFdIWkVZdGZ3QW9UN0k1?=
 =?utf-8?B?VDB0YmJwQ3M5aHJoMCtNSTBERWdaLzR4b2FYdWwzMkYwOGhWS0ZnNkhkS0RN?=
 =?utf-8?B?RmdGUmc1RzBRYURMN3d1RVQ3aU5vU01hamxxTFFkQVVUQ1AwL29BWjdjRTJG?=
 =?utf-8?B?WCt2NlBzSG5icU5zbzlNV25VZjhFSzJMa2x6QTN4SG1SWitleWJpQXZpUWZs?=
 =?utf-8?B?dHhVM0hGT2JFTnhndm9IQnRsU1UzSVdkNFd3MXpVNDFPK1V1V3BzeFUreHZq?=
 =?utf-8?B?S0lDSXlOdllacmJ6eStCd1YxekQwTUFYdkp5OHJtVll6U2lJVk13VFJOQzlX?=
 =?utf-8?B?UlZibnlsbEdKTjdqV3dGRWkwaUVyZjV2cHJ5VjRiR1VtYy9HQllvbXkwT3Nn?=
 =?utf-8?B?VVNTbmNFSE00NlRRSW1heGVDTUNyeWtsUGM5WkVEVWRzeGVpdXZNcWZXaTN1?=
 =?utf-8?B?VWM0S01EVjBZdXlGZzZqb3BzTGswdHJjZFp5VTYrbmRhTE5Bcm9XYlJMOG9o?=
 =?utf-8?B?RmJsaFNQZE81citDclZsVHpjSjl5c2NOY0N6ckpPRnc5TC9GeEN1V0VIK2R0?=
 =?utf-8?B?VC9BSFZkaUwvRnlJSkkrMHV5YnkrRmlYaExnRGhEd214VG04YVUrS3BSYUpS?=
 =?utf-8?B?ZEViNCsyQ2VvMW5HdmFkbDExbXdLYzVua0t4a01pYmVVSFBtR1BLcDhMZ2RM?=
 =?utf-8?B?SzVQTDVzdEpZQSs4L05hQW45RkpEbjc4Q2gxaGhkREpnK1I2ZWJBMHdsNlRp?=
 =?utf-8?B?d0VZLytXQW1QZDRsM2gyd2ltclQzMXRRSTQwZldac05nS1gwaXVXeUMvK2d5?=
 =?utf-8?B?OVl1a0c0WW9USmNmOWtTTmU1RmV1K2RUaW9XZjB4ZERHR1N6bWtZdW9oYi9K?=
 =?utf-8?B?cUNBTXE4QTNlR1VpeDZEc3FoRloyS1ZnaWFoZ0NScThSVTBIcDZwOEYzREQ4?=
 =?utf-8?B?U0JZLzNESG9uRkd5R1kyQUJIdDF4Q3JHdGJ0ZnRtcndpRmhlOUhqQXNnTHg3?=
 =?utf-8?B?T1l6K1VYdmRsYXNzR0IzWXRRNVQwVkZNNHRPaHdRVlRsUlRqcjh1dXh1S3o0?=
 =?utf-8?B?cWZQaUVYcm9pam0vbDlxWVZWR0lnUVRibzFyZkd5a294TXNRVFJXbmZTQTJx?=
 =?utf-8?B?TzdyV3BtSEVFa3pCT053NzUxU1lEUG5RZWJZbXFsWXRCREh6MzlSbVlZS3BN?=
 =?utf-8?B?RUVleTd4WnNJbnZhbkR4N2FMUUxneWVRd2dsazd1d0MrVWNQWThTS1BUd3hG?=
 =?utf-8?B?SmVLaXl0NERpMEc5cFRmUExqZHZoaUdSQUJDYWROVWhpU3VsbVZpVmtEcDl5?=
 =?utf-8?B?NXZQQldiU3hvMHA2elE4VHNHRmp0SGlMTVpaSzVjVnhSb1Z3akVHZS9oZEtF?=
 =?utf-8?B?bXdURjdOUnBxYml2d1gxZTZDQlJ6S3lxTUowRzVoK2hGbHhocFE1amM4aGZR?=
 =?utf-8?B?Rm95ZGNuZXFSWEVrUnpmdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <4F99D1FD42881B4E8ED17DB0ABF642CC@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cf201820-7316-49d5-978a-08d9823be76d
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 04:53:24.6229
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fuefLA2Uk6TzkvwYv+hx5cBmkz1K6pcTeRvLvfApOLBm/+PCY2lOlPBWM8M4kmshozxjjKzcNmwyRnHKSUCFsHeR2n6gUMH0df2nB4X7+ZgaHbT22S1+yAROTCoxRnOk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3587
X-Proofpoint-ORIG-GUID: QiHDHPtJT4t7S26omfZZTnoTa8xee28x
X-Proofpoint-GUID: QiHDHPtJT4t7S26omfZZTnoTa8xee28x
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_03,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0
 suspectscore=0 adultscore=0 impostorscore=0 clxscore=1015 malwarescore=0
 bulkscore=0 mlxlogscore=999 spamscore=0 phishscore=0 priorityscore=1501
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280030

DQpPbiAyOC4wOS4yMSAwNzoxOCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBNb24s
IDI3IFNlcCAyMDIxLCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMDk6MzUs
IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIxIDEwOjI2LCBK
YW4gQmV1bGljaCB3cm90ZToNCj4+Pj4gT24gMjcuMDkuMjAyMSAwODo1OCwgT2xla3NhbmRyIEFu
ZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8
b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pg0KPj4+Pj4gQ3VycmVudGx5
IFBDSSBiYWNrZW5kIGltcGxlbWVudHMgbXVsdGlwbGUgZnVuY3Rpb25hbGl0aWVzIGF0IGEgdGlt
ZS4NCj4+Pj4+IFRvIG5hbWUgYSBmZXc6DQo+Pj4+PiAxLiBJdCBpcyB1c2VkIGFzIGEgZGF0YWJh
c2UgZm9yIGFzc2lnbmFibGUgUENJIGRldmljZXMsIGUuZy4geGwNCj4+Pj4+ICAgICAgIHBjaS1h
c3NpZ25hYmxlLXthZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQgbGlzdC4gU28sDQo+
Pj4+PiB3aGVuZXZlcg0KPj4+Pj4gICAgICAgdGhlIHRvb2xzdGFjayBuZWVkcyB0byBrbm93IHdo
aWNoIFBDSSBkZXZpY2VzIGNhbiBiZSBwYXNzZWQgdGhyb3VnaA0KPj4+Pj4gICAgICAgaXQgcmVh
ZHMgdGhhdCBmcm9tIHRoZSByZWxldmFudCBzeXNmcyBlbnRyaWVzIG9mIHRoZSBwY2liYWNrLg0K
Pj4+Pj4gMi4gSXQgaXMgdXNlZCB0byBob2xkIHRoZSB1bmJvdW5kIFBDSSBkZXZpY2VzIGxpc3Qs
IGUuZy4gd2hlbiBwYXNzaW5nDQo+Pj4+PiAgICAgICB0aHJvdWdoIGEgUENJIGRldmljZSBpdCBu
ZWVkcyB0byBiZSB1bmJvdW5kIGZyb20gdGhlIHJlbGV2YW50DQo+Pj4+PiBkZXZpY2UNCj4+Pj4+
ICAgICAgIGRyaXZlciBhbmQgYm91bmQgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3BlYWtpbmcgaXQg
aXMgbm90IHJlcXVpcmVkDQo+Pj4+PiAgICAgICB0aGF0IHRoZSBkZXZpY2UgaXMgYm91bmQgdG8g
cGNpYmFjaywgYnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBhcyBhDQo+Pj4+PiAgICAgICBkYXRh
YmFzZSBvZiB0aGUgcGFzc2VkIHRocm91Z2ggUENJIGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5k
IHRoZQ0KPj4+Pj4gICAgICAgZGV2aWNlcyBiYWNrIHRvIHRoZWlyIG9yaWdpbmFsIGRyaXZlcnMg
d2hlbiBndWVzdCBkb21haW4gc2h1dHMNCj4+Pj4+IGRvd24pDQo+Pj4+PiAzLiBEZXZpY2UgcmVz
ZXQgZm9yIHRoZSBkZXZpY2VzIGJlaW5nIHBhc3NlZCB0aHJvdWdoDQo+Pj4+PiA0LiBQYXJhLXZp
cnR1YWxpc2VkIHVzZS1jYXNlcyBzdXBwb3J0DQo+Pj4+Pg0KPj4+Pj4gVGhlIHBhcmEtdmlydHVh
bGlzZWQgcGFydCBvZiB0aGUgZHJpdmVyIGlzIG5vdCBhbHdheXMgbmVlZGVkIGFzIHNvbWUNCj4+
Pj4+IGFyY2hpdGVjdHVyZXMsIGUuZy4gQXJtIG9yIHg4NiBQVkggRG9tMCwgYXJlIG5vdCB1c2lu
ZyBiYWNrZW5kLWZyb250ZW5kDQo+Pj4+PiBtb2RlbCBmb3IgUENJIGRldmljZSBwYXNzdGhyb3Vn
aC4gRm9yIHN1Y2ggdXNlLWNhc2VzIG1ha2UgdGhlIHZlcnkNCj4+Pj4+IGZpcnN0IHN0ZXAgaW4g
c3BsaXR0aW5nIHRoZSB4ZW4tcGNpYmFjayBkcml2ZXIgaW50byB0d28gcGFydHM6IFhlbg0KPj4+
Pj4gUENJIHN0dWIgYW5kIFBDSSBQViBiYWNrZW5kIGRyaXZlcnMuDQo+Pj4+Pg0KPj4+Pj4gU2ln
bmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28NCj4+Pj4+IDxvbGVrc2FuZHJfYW5k
cnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+DQo+Pj4+PiAtLS0NCj4+Pj4+IENoYW5nZXMgc2lu
Y2UgdjM6DQo+Pj4+PiAtIE1vdmUgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiB0byB0aGUgc2Vjb25k
IHBhdGNoDQo+Pj4+IEknbSBhZnJhaWQgdGhpcyB3YXNuJ3QgZnVsbHkgZG9uZToNCj4+Pj4NCj4+
Pj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+PiArKysgYi9k
cml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+Pj4gQEAgLTEsNSArMSw2IEBADQo+
Pj4+PiAgICAgIyBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMA0KPj4+Pj4gICAgIG9i
ai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNrLm8NCj4+Pj4+ICtv
YmotJChDT05GSUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+IFdoaWxl
IGJlbmlnbiB3aGVuIGFkZGVkIGhlcmUsIHRoaXMgYWRkaXRpb24gc3RpbGwgZG9lc24ndCBzZWVt
IHRvDQo+Pj4+IGJlbG9uZyBoZXJlLg0KPj4+IE15IGJhZC4gU28sIGl0IHNlZW1zIHdpdGhvdXQg
Q09ORklHX1hFTl9QQ0lERVZfU1RVQiB0aGUgY2hhbmdlIHNlZW1zDQo+Pj4NCj4+PiB0byBiZSBu
b24tZnVuY3Rpb25hbC4gV2l0aCBDT05GSUdfWEVOX1BDSURFVl9TVFVCIHdlIGZhaWwgdG8gYnVp
bGQgb24gMzItYml0DQo+Pj4NCj4+PiBhcmNoaXRlY3R1cmVzLi4uDQo+Pj4NCj4+PiBXaGF0IHdv
dWxkIGJlIHRoZSBwcmVmZXJlbmNlIGhlcmU/IFN0ZWZhbm8gc3VnZ2VzdGVkIHRoYXQgd2Ugc3Rp
bGwgZGVmaW5lDQo+Pj4NCj4+PiBDT05GSUdfWEVOX1BDSURFVl9TVFVCLCBidXQgaW4gZGlzYWJs
ZWQgc3RhdGUsIGUuZy4gd2UgYWRkIHRyaXN0YXRlIHRvIGl0DQo+Pj4NCj4+PiBpbiB0aGUgc2Vj
b25kIHBhdGNoDQo+Pj4NCj4+PiBBbm90aGVyIG9wdGlvbiBpcyBqdXN0IHRvIHNxdWFzaCB0aGUg
dHdvIHBhdGNoZXMuDQo+PiBTcXVhc2hpbmcgd291bGQgYmUgZmluZSBmb3IgbWUuDQo+ICAgDQo+
IEl0IGlzIGZpbmUgZm9yIG1lIHRvIHNxdWFzaCB0aGUgdHdvIHBhdGNoZXMuDQo+DQo+IEJ1dCBp
biBhbnkgY2FzZSwgd291bGRuJ3QgaXQgYmUgYmV0dGVyIHRvIG1vZGlmeSB0aGF0IHNwZWNpZmlj
IGNoYW5nZSB0bzoNCj4NCj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01h
a2VmaWxlIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4gaW5kZXggZTJjYjM3
NjQ0NGE2Li5lMjNjNzU4Yjg1YWUgMTAwNjQ0DQo+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2li
YWNrL01ha2VmaWxlDQo+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+
IEBAIC0xLDYgKzEsNSBAQA0KPiAgICMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAN
Cj4gLW9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNrLm8NCj4g
LW9iai0kKENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4gK29iai0k
KENPTkZJR19YRU5fUENJX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4gICANCj4gICB4ZW4tcGNp
YmFjay15IDo9IHBjaV9zdHViLm8gcGNpYmFja19vcHMubyB4ZW5idXMubw0KPiAgIHhlbi1wY2li
YWNrLXkgKz0gY29uZl9zcGFjZS5vIGNvbmZfc3BhY2VfaGVhZGVyLm8gXA0KPg0KPg0KPiBpbnN0
ZWFkPw0KDQpZZXMsIGxvb2tzIHNpbXBsZXIgdGhlbi4gSSdsbCBzcXVhc2ggYW5kIGFkZCB0aGlz
IGNoYW5nZQ0KDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 04:59:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 04:59:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197443.350485 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Cq-0008H6-V8; Tue, 28 Sep 2021 04:59:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197443.350485; Tue, 28 Sep 2021 04:59:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Cq-0008Gz-Rj; Tue, 28 Sep 2021 04:59:04 +0000
Received: by outflank-mailman (input) for mailman id 197443;
 Tue, 28 Sep 2021 04:59:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV5Cp-0008Gt-JB
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 04:59:03 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 58372ca8-302d-487d-a4b1-17afe1e1918d;
 Tue, 28 Sep 2021 04:59:01 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3238360FC0;
 Tue, 28 Sep 2021 04:59:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58372ca8-302d-487d-a4b1-17afe1e1918d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632805141;
	bh=8G5ud11qkyImUnBMBEMnayTKRHjNRpiwBh3dxN+opXM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=GWOjfdcBYA8wnTt8HVz+U/3i8I3lE2Ru1x0ltjHeVgPpXcVFm/6BpmTZ+IQSFGdHJ
	 xQZxzY5arFo6rDlDhTA4DDJfuN1tkmKHjFfCjjcEDGsVvoUVG45vhbr1ulompg3PKB
	 02+xgmjT0vl9kt6C7Sm6g41VVVfk/6eUGR31MhochK/l1S5XTTr39EfZ44mLJp54H3
	 AZBw7JTqrdH3BO9EzjYWHjPCgorJWTp7ElT9S95DxKYpOOsvVUm5joQ/UNFeVyS5sx
	 vgHu+xysrHysxK8r5BydHUuPhUDt8nLvGh7GsIbnVeNXBJ1sS3RlUKb4+ZOGIKkF9O
	 hCb84C0IHiwQQ==
Date: Mon, 27 Sep 2021 21:59:00 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>, 
    "jbeulich@suse.com" <jbeulich@suse.com>, 
    "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, "wl@xen.org" <wl@xen.org>
Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node memory
 range
In-Reply-To: <DB9PR08MB68573F34B7B9A73B976D4E4C9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109272158390.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-9-wei.chen@arm.com> <alpine.DEB.2.21.2109231719410.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB686474BADD786E523EAC026B9EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109241244070.17979@sstabellini-ThinkPad-T480s> <DB9PR08MB6857A3176752B3E08EAE4D739EA69@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109262002390.5022@sstabellini-ThinkPad-T480s> <alpine.DEB.2.21.2109262159500.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB685772C5CDE9DF885A063F479EA79@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109271018220.5022@sstabellini-ThinkPad-T480s> <DB9PR08MB68573F34B7B9A73B976D4E4C9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1263762048-1632805141=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1263762048-1632805141=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 28 Sep 2021, Wei Chen wrote:
> Hi Stefano,
> 
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月28日 1:19
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > <Bertrand.Marquis@arm.com>; jbeulich@suse.com; andrew.cooper3@citrix.com;
> > roger.pau@citrix.com; wl@xen.org
> > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node
> > memory range
> > 
> > On Mon, 27 Sep 2021, Wei Chen wrote:
> > > > -----Original Message-----
> > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > Sent: 2021年9月27日 13:05
> > > > To: Stefano Stabellini <sstabellini@kernel.org>
> > > > Cc: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
> > > > julien@xen.org; Bertrand Marquis <Bertrand.Marquis@arm.com>;
> > > > jbeulich@suse.com; andrew.cooper3@citrix.com; roger.pau@citrix.com;
> > > > wl@xen.org
> > > > Subject: RE: [PATCH 08/37] xen/x86: add detection of discontinous node
> > > > memory range
> > > >
> > > > On Sun, 26 Sep 2021, Stefano Stabellini wrote:
> > > > > On Sun, 26 Sep 2021, Wei Chen wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > > > Sent: 2021年9月25日 3:53
> > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-
> > > > > > > devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > > > > <Bertrand.Marquis@arm.com>; jbeulich@suse.com;
> > > > andrew.cooper3@citrix.com;
> > > > > > > roger.pau@citrix.com; wl@xen.org
> > > > > > > Subject: RE: [PATCH 08/37] xen/x86: add detection of
> > discontinous
> > > > node
> > > > > > > memory range
> > > > > > >
> > > > > > > On Fri, 24 Sep 2021, Wei Chen wrote:
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > > > > > Sent: 2021年9月24日 8:26
> > > > > > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > > > > > Cc: xen-devel@lists.xenproject.org; sstabellini@kernel.org;
> > > > > > > julien@xen.org;
> > > > > > > > > Bertrand Marquis <Bertrand.Marquis@arm.com>;
> > jbeulich@suse.com;
> > > > > > > > > andrew.cooper3@citrix.com; roger.pau@citrix.com; wl@xen.org
> > > > > > > > > Subject: Re: [PATCH 08/37] xen/x86: add detection of
> > > > discontinous node
> > > > > > > > > memory range
> > > > > > > > >
> > > > > > > > > CC'ing x86 maintainers
> > > > > > > > >
> > > > > > > > > On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > > > > > > One NUMA node may contain several memory blocks. In
> > current
> > > > Xen
> > > > > > > > > > code, Xen will maintain a node memory range for each node
> > to
> > > > cover
> > > > > > > > > > all its memory blocks. But here comes the problem, in the
> > gap
> > > > of
> > > > > > > > > > one node's two memory blocks, if there are some memory
> > blocks
> > > > don't
> > > > > > > > > > belong to this node (remote memory blocks). This node's
> > memory
> > > > range
> > > > > > > > > > will be expanded to cover these remote memory blocks.
> > > > > > > > > >
> > > > > > > > > > One node's memory range contains othe nodes' memory, this
> > is
> > > > > > > obviously
> > > > > > > > > > not very reasonable. This means current NUMA code only can
> > > > support
> > > > > > > > > > node has continous memory blocks. However, on a physical
> > > > machine,
> > > > > > > the
> > > > > > > > > > addresses of multiple nodes can be interleaved.
> > > > > > > > > >
> > > > > > > > > > So in this patch, we add code to detect discontinous
> > memory
> > > > blocks
> > > > > > > > > > for one node. NUMA initializtion will be failed and error
> > > > messages
> > > > > > > > > > will be printed when Xen detect such hardware
> > configuration.
> > > > > > > > >
> > > > > > > > > At least on ARM, it is not just memory that can be
> > interleaved,
> > > > but
> > > > > > > also
> > > > > > > > > MMIO regions. For instance:
> > > > > > > > >
> > > > > > > > > node0 bank0 0-0x1000000
> > > > > > > > > MMIO 0x1000000-0x1002000
> > > > > > > > > Hole 0x1002000-0x2000000
> > > > > > > > > node0 bank1 0x2000000-0x3000000
> > > > > > > > >
> > > > > > > > > So I am not familiar with the SRAT format, but I think on
> > ARM
> > > > the
> > > > > > > check
> > > > > > > > > would look different: we would just look for multiple memory
> > > > ranges
> > > > > > > > > under a device_type = "memory" node of a NUMA node in device
> > > > tree.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > Should I need to include/refine above message to commit log?
> > > > > > >
> > > > > > > Let me ask you a question first.
> > > > > > >
> > > > > > > With the NUMA implementation of this patch series, can we deal
> > with
> > > > > > > cases where each node has multiple memory banks, not interleaved?
> > > > > >
> > > > > > Yes.
> > > > > >
> > > > > > > An an example:
> > > > > > >
> > > > > > > node0: 0x0        - 0x10000000
> > > > > > > MMIO : 0x10000000 - 0x20000000
> > > > > > > node0: 0x20000000 - 0x30000000
> > > > > > > MMIO : 0x30000000 - 0x50000000
> > > > > > > node1: 0x50000000 - 0x60000000
> > > > > > > MMIO : 0x60000000 - 0x80000000
> > > > > > > node2: 0x80000000 - 0x90000000
> > > > > > >
> > > > > > >
> > > > > > > I assume we can deal with this case simply by setting node0
> > memory
> > > > to
> > > > > > > 0x0-0x30000000 even if there is actually something else, a
> > device,
> > > > that
> > > > > > > doesn't belong to node0 in between the two node0 banks?
> > > > > >
> > > > > > While this configuration is rare in SoC design, but it is not
> > > > impossible.
> > > > >
> > > > > Definitely, I have seen it before.
> > > > >
> > > > >
> > > > > > > Is it only other nodes' memory interleaved that cause issues? In
> > > > other
> > > > > > > words, only the following is a problematic scenario?
> > > > > > >
> > > > > > > node0: 0x0        - 0x10000000
> > > > > > > MMIO : 0x10000000 - 0x20000000
> > > > > > > node1: 0x20000000 - 0x30000000
> > > > > > > MMIO : 0x30000000 - 0x50000000
> > > > > > > node0: 0x50000000 - 0x60000000
> > > > > > >
> > > > > > > Because node1 is in between the two ranges of node0?
> > > > > > >
> > > > > >
> > > > > > But only device_type="memory" can be added to allocation.
> > > > > > For mmio there are two cases:
> > > > > > 1. mmio doesn't have NUMA id property.
> > > > > > 2. mmio has NUMA id property, just like some PCIe controllers.
> > > > > >    But we don’t need to handle these kinds of MMIO devices
> > > > > >    in memory block parsing. Because we don't need to allocate
> > > > > >    memory from these mmio ranges. And for accessing, we need
> > > > > >    a NUMA-aware PCIe controller driver or a generic NUMA-aware
> > > > > >    MMIO accessing APIs.
> > > > >
> > > > > Yes, I am not too worried about devices with a NUMA id property
> > because
> > > > > they are less common and this series doesn't handle them at all,
> > right?
> > > > > I imagine they would be treated like any other device without NUMA
> > > > > awareness.
> > > > >
> > > > > I am thinking about the case where the memory of each NUMA node is
> > made
> > > > > of multiple banks. I understand that this patch adds an explicit
> > check
> > > > > for cases where these banks are interleaving, however there are many
> > > > > other cases where NUMA memory nodes are *not* interleaving but they
> > are
> > > > > still made of multiple discontinuous banks, like in the two example
> > > > > above.
> > > > >
> > > > > My question is whether this patch series in its current form can
> > handle
> > > > > the two cases above correctly. If so, I am wondering how it works
> > given
> > > > > that we only have a single "start" and "size" parameter per node.
> > > > >
> > > > > On the other hand if this series cannot handle the two cases above,
> > my
> > > > > question is whether it would fail explicitly or not. The new
> > > > > check is_node_memory_continuous doesn't seem to be able to catch
> > them.
> > > >
> > > >
> > > > Looking at numa_update_node_memblks, it is clear that the code is
> > meant
> > > > to increase the range of each numa node to cover even MMIO regions in
> > > > between memory banks. Also see the comment at the top of the file:
> > > >
> > > >  * Assumes all memory regions belonging to a single proximity domain
> > > >  * are in one chunk. Holes between them will be included in the node.
> > > >
> > > > So if there are multiple banks for each node, start and end are
> > > > stretched to cover the holes between them, and it works as long as
> > > > memory banks of different NUMA nodes don't interleave.
> > > >
> > > > I would appreciate if you could add an in-code comment to explain this
> > > > on top of numa_update_node_memblk.
> > >
> > > Yes, I will do it.
> > 
> > Thank you
> > 
> > 
> > > > Have you had a chance to test this? If not it would be fantastic if
> > you
> > > > could give it a quick test to make sure it works as intended: for
> > > > instance by creating multiple memory banks for each NUMA node by
> > > > splitting an real bank into two smaller banks with a hole in between
> > in
> > > > device tree, just for the sake of testing.
> > >
> > > Yes, I have created some fake NUMA nodes in FVP device tree to test it.
> > > The intertwine of nodes' address can be detected.
> > >
> > > (XEN) SRAT: Node 0 0000000080000000-00000000ff000000
> > > (XEN) SRAT: Node 1 0000000880000000-00000008c0000000
> > > (XEN) NODE 0: (0000000080000000-00000008d0000000) intertwine with NODE 1
> > (0000000880000000-00000008c0000000)
> > 
> > Great thanks! And what if there are multiple non-contiguous memory banks
> > per node, but *not* intertwined. Does that all work correctly as
> > expected?
> 
> Yes, I am using a device tree setting like this:

Perfect! Thank you!


>     memory@80000000 {
>         device_type = "memory";
>         reg = <0x0 0x80000000 0x0 0x80000000>;
>         numa-node-id = <0>;
>     };
> 
>     memory@880000000 {
>         device_type = "memory";
>         reg = <0x8 0x80000000 0x0 0x8000000>;
>         numa-node-id = <0>;
>     };
> 
>     memory@890000000 {
>         device_type = "memory";
>         reg = <0x8 0x90000000 0x0 0x8000000>;
>         numa-node-id = <0>;
>     };
> 
>     memory@8A0000000 {
>         device_type = "memory";
>         reg = <0x8 0xA0000000 0x0 0x8000000>;
>         numa-node-id = <0>;
>     };
> 
>     memory@8B0000000 {
>         device_type = "memory";
>         reg = <0x8 0xB0000000 0x0 0x8000000>;
>         numa-node-id = <0>;
>     };
> 
>     memory@8C0000000 {
>         device_type = "memory";
>         reg = <0x8 0xC0000000 0x0 0x8000000>;
>         numa-node-id = <1>;
>     };
> 
>     memory@8D0000000 {
>         device_type = "memory";
>         reg = <0x8 0xD0000000 0x0 0x8000000>;
>         numa-node-id = <1>;
>     };
> 
>     memory@8E0000000 {
>         device_type = "memory";
>         reg = <0x8 0xE0000000 0x0 0x8000000>;
>         numa-node-id = <1>;
>     };
> 
>     memory@8F0000000 {
>         device_type = "memory";
>         reg = <0x8 0xF0000000 0x0 0x8000000>;
>         numa-node-id = <1>;
>     };
> 
> And in Xen we got the output:
> 
> (XEN) DT: NUMA node 0 processor parsed
> (XEN) DT: NUMA node 0 processor parsed
> (XEN) DT: NUMA node 1 processor parsed
> (XEN) DT: NUMA node 1 processor parsed
> (XEN) SRAT: Node 0 0000000080000000-00000000ff000000
> (XEN) SRAT: Node 0 0000000880000000-0000000888000000
> (XEN) SRAT: Node 0 0000000890000000-0000000898000000
> (XEN) SRAT: Node 0 00000008a0000000-00000008a8000000
> (XEN) SRAT: Node 0 00000008b0000000-00000008b8000000
> (XEN) SRAT: Node 1 00000008c0000000-00000008c8000000
> (XEN) SRAT: Node 1 00000008d0000000-00000008d8000000
> (XEN) SRAT: Node 1 00000008e0000000-00000008e8000000
> (XEN) SRAT: Node 1 00000008f0000000-00000008f8000000
> (XEN) NUMA: parsing numa-distance-map
> (XEN) NUMA: distance: NODE#0->NODE#0:10
> (XEN) NUMA: distance: NODE#0->NODE#1:20
> (XEN) NUMA: distance: NODE#1->NODE#1:10
> (XEN) NUMA: Using 16 for the hash shift.
> (XEN) Domain heap initialised
> (XEN) Booting using Device Tree
> 
> Dom0 can be boot successfully, xl info got:
> xl info
> host                   : X-Dom0
> release                : 5.12.0
> version                : #20 SMP PREEMPT Wed Jul 28 13:41:28 CST 2021
> machine                : aarch64
> nr_cpus                : 4
> max_cpu_id             : 3
> nr_nodes               : 2
> cores_per_socket       : 1
> threads_per_core       : 1
> 
> Xen debug console to dump numa info, we got:
> 
> (XEN) 'u' pressed -> dumping numa info (now = 13229372281010)
> (XEN) NODE0 start->524288 size->8617984 free->388741
> (XEN) NODE1 start->9175040 size->229376 free->106460
> (XEN) CPU0...1 -> NODE0
> (XEN) CPU2...3 -> NODE1
> (XEN) Memory location of each domain:
> (XEN) Domain 0 (total: 262144):
> (XEN)     Node 0: 262144
> (XEN)     Node 1: 0
> 
> 
--8323329-1263762048-1632805141=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:01:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:01:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197451.350496 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5FA-0001Xi-Fm; Tue, 28 Sep 2021 05:01:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197451.350496; Tue, 28 Sep 2021 05:01:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5FA-0001Xb-Cg; Tue, 28 Sep 2021 05:01:28 +0000
Received: by outflank-mailman (input) for mailman id 197451;
 Tue, 28 Sep 2021 05:01:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV5F9-0001XV-3C
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:01:27 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 21ab4fbd-2019-11ec-bc65-12813bfff9fa;
 Tue, 28 Sep 2021 05:01:25 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id E441861153;
 Tue, 28 Sep 2021 05:01:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21ab4fbd-2019-11ec-bc65-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632805285;
	bh=t8H8YPdMJiXnsapCfFlSV6luQZJAg7RLgrmgj8Lqnyk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JsL8A69uzmWM44kxnsge4W10L+BirbKicrAohCazz0ujbeFlI3zWJl6lnvBfOzEBj
	 x9vFvavrweHgRB7xNxIpLJvYlybQKRCxXt2fYwEq8lZSHTrlRkm9Iehsxn41F9WyAO
	 PIJN53S4bjsmANReJ1PnPDyi915/QSMW4NhME/+NJtVeZSjXmVDLBS4XPRiX0qVfPB
	 4z0L+4/4TgYpREEairvVYwt+Eh74U0JugCror3sX7WUC+JSAlXA+prgSkWh1gS+ESL
	 V7eBaWBN5xAVtXP9YApAkK8b+WQl8UCu0jKGok5F3DS/oNbUDMLZlyR9nsxyeX2SoW
	 4n79dwFpHrnOQ==
Date: Mon, 27 Sep 2021 22:01:24 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Wei Chen <Wei.Chen@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
In-Reply-To: <DB9PR08MB6857E57847E38C400F6571EA9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.2109272159590.5022@sstabellini-ThinkPad-T480s>
References: <20210923120236.3692135-1-wei.chen@arm.com> <20210923120236.3692135-21-wei.chen@arm.com> <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s> <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com> <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com> <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com> <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com> <alpine.DEB.2.21.2109271420560.5022@sstabellini-ThinkPad-T480s> <DB9PR08MB6857E57847E38C400F6571EA9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-921827469-1632805284=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-921827469-1632805284=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 28 Sep 2021, Wei Chen wrote:
> > -----Original Message-----
> > From: Stefano Stabellini <sstabellini@kernel.org>
> > Sent: 2021年9月28日 9:00
> > To: Wei Chen <Wei.Chen@arm.com>
> > Cc: Jan Beulich <jbeulich@suse.com>; xen-devel@lists.xenproject.org;
> > julien@xen.org; Bertrand Marquis <Bertrand.Marquis@arm.com>; Stefano
> > Stabellini <sstabellini@kernel.org>
> > Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-
> > EFI architecture
> > 
> > On Mon, 27 Sep 2021, Wei Chen wrote:
> > > > -----Original Message-----
> > > > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> > Wei
> > > > Chen
> > > > Sent: 2021年9月26日 18:25
> > > > To: Jan Beulich <jbeulich@suse.com>
> > > > Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > <Bertrand.Marquis@arm.com>; Stefano Stabellini <sstabellini@kernel.org>
> > > > Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for
> > non-
> > > > EFI architecture
> > > >
> > > > Hi Jan,
> > > >
> > > > > -----Original Message-----
> > > > > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf
> > Of
> > > > Jan
> > > > > Beulich
> > > > > Sent: 2021年9月24日 18:49
> > > > > To: Wei Chen <Wei.Chen@arm.com>
> > > > > Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
> > > > > <Bertrand.Marquis@arm.com>; Stefano Stabellini
> > <sstabellini@kernel.org>
> > > > > Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for
> > > > non-
> > > > > EFI architecture
> > > > >
> > > > > On 24.09.2021 12:31, Wei Chen wrote:
> > > > > >> From: Jan Beulich <jbeulich@suse.com>
> > > > > >> Sent: 2021年9月24日 15:59
> > > > > >>
> > > > > >> On 24.09.2021 06:34, Wei Chen wrote:
> > > > > >>>> From: Stefano Stabellini <sstabellini@kernel.org>
> > > > > >>>> Sent: 2021年9月24日 9:15
> > > > > >>>>
> > > > > >>>> On Thu, 23 Sep 2021, Wei Chen wrote:
> > > > > >>>>> --- a/xen/common/Kconfig
> > > > > >>>>> +++ b/xen/common/Kconfig
> > > > > >>>>> @@ -11,6 +11,16 @@ config COMPAT
> > > > > >>>>>  config CORE_PARKING
> > > > > >>>>>  	bool
> > > > > >>>>>
> > > > > >>>>> +config EFI
> > > > > >>>>> +	bool
> > > > > >>>>
> > > > > >>>> Without the title the option is not user-selectable (or de-
> > > > > selectable).
> > > > > >>>> So the help message below can never be seen.
> > > > > >>>>
> > > > > >>>> Either add a title, e.g.:
> > > > > >>>>
> > > > > >>>> bool "EFI support"
> > > > > >>>>
> > > > > >>>> Or fully make the option a silent option by removing the help
> > text.
> > > > > >>>
> > > > > >>> OK, in current Xen code, EFI is unconditionally compiled. Before
> > > > > >>> we change related code, I prefer to remove the help text.
> > > > > >>
> > > > > >> But that's not true: At least on x86 EFI gets compiled depending
> > on
> > > > > >> tool chain capabilities. Ultimately we may indeed want a user
> > > > > >> selectable option here, but until then I'm afraid having this
> > option
> > > > > >> at all may be misleading on x86.
> > > > > >>
> > > > > >
> > > > > > I check the build scripts, yes, you're right. For x86, EFI is not
> > a
> > > > > > selectable option in Kconfig. I agree with you, we can't use
> > Kconfig
> > > > > > system to decide to enable EFI build for x86 or not.
> > > > > >
> > > > > > So how about we just use this EFI option for Arm only? Because on
> > Arm,
> > > > > > we do not have such toolchain dependency.
> > > > >
> > > > > To be honest - don't know. That's because I don't know what you want
> > > > > to use the option for subsequently.
> > > > >
> > > >
> > > > In last version, I had introduced an arch-helper to stub EFI_BOOT
> > > > in Arm's common code for Arm32. Because Arm32 doesn't support EFI.
> > > > So Julien suggested me to introduce a CONFIG_EFI option for non-EFI
> > > > supported architectures to stub in EFI layer.
> > > >
> > > > [1] https://lists.xenproject.org/archives/html/xen-devel/2021-
> > > > 08/msg00808.html
> > > >
> > >
> > > As Jan' reminded, x86 doesn't depend on Kconfig to build EFI code.
> > > So, if we CONFIG_EFI to stub EFI API's for x86, we will encounter
> > > that toolchains enable EFI, but Kconfig disable EFI. Or Kconfig
> > > enable EFI but toolchain doesn't provide EFI build supports. And
> > > then x86 could not work well.
> > >
> > > If we use CONFIG_EFI for Arm only, that means CONFIG_EFI for x86
> > > is off, this will also cause problem.
> > >
> > > So, can we still use previous arch_helpers to stub for Arm32?
> > > until x86 can use this selectable option?
> > 
> > EFI doesn't have to be necessarily a user-visible option in Kconfig at
> > this point. I think Julien was just asking to make the #ifdef based on
> > a EFI-related config rather than just based CONFIG_ARM64.
> > 
> > On x86 EFI is detected based on compiler support, setting XEN_BUILD_EFI
> > in xen/arch/x86/Makefile. Let's say that we keep using the same name
> > "XEN_BUILD_EFI" on ARM as well.
> > 
> > On ARM32, XEN_BUILD_EFI should be always unset.
> > 
> > On ARM64 XEN_BUILD_EFI should be always set.
> > 
> > That's it, right? I'd argue that CONFIG_EFI or HAS_EFI are better names
> > than XEN_BUILD_EFI, but that's OK anyway. So for instance you can make
> > XEN_BUILD_EFI an invisible symbol in xen/arch/arm/Kconfig and select it
> > only on ARM64.
> 
> Thanks, this is a good approach. But if we place XEN_BUILD_EFI in Kconfig
> it will be transfer to CONFIG_XEN_BUILD_EFI. How about using another name
> in Kconfig like ARM_EFI, but use CONFIG_ARM_EFI in config.h to define
> XEN_BUILD_EFI?

I am OK with that. Another option is to rename XEN_BUILD_EFI to
CONFIG_XEN_BUILD_EFI on x86. Either way is fine by me. Jan, do you havea
preference?
--8323329-921827469-1632805284=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:02:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:02:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197175.350513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fr-000293-5l; Tue, 28 Sep 2021 05:02:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197175.350513; Tue, 28 Sep 2021 05:02:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fq-00028p-VT; Tue, 28 Sep 2021 05:02:10 +0000
Received: by outflank-mailman (input) for mailman id 197175;
 Mon, 27 Sep 2021 20:44:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ukqu=OR=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mUxUc-0008Pl-Ay
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 20:44:54 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3bcff2f8-f1f1-439e-860f-760bfbbf41b5;
 Mon, 27 Sep 2021 20:44:50 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTd-0001js-Ut; Mon, 27 Sep 2021 22:43:54 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTa-0001Yd-8z; Mon, 27 Sep 2021 22:43:50 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTZ-0001OH-5b; Mon, 27 Sep 2021 22:43:49 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3bcff2f8-f1f1-439e-860f-760bfbbf41b5
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <uwe@kleine-koenig.org>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Russell Currey <ruscur@russell.cc>,
	"Oliver O'Halloran" <oohall@gmail.com>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	x86@kernel.org,
	"H. Peter Anvin" <hpa@zytor.com>,
	Frederic Barrat <fbarrat@linux.ibm.com>,
	Andrew Donnellan <ajd@linux.ibm.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Mathias Nyman <mathias.nyman@intel.com>,
	linuxppc-dev@lists.ozlabs.org,
	linux-perf-users@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-usb@vger.kernel.org
Subject: [PATCH v4 7/8] PCI: Replace pci_dev::driver usage by pci_dev::dev.driver
Date: Mon, 27 Sep 2021 22:43:25 +0200
Message-Id: <20210927204326.612555-8-uwe@kleine-koenig.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210927204326.612555-1-uwe@kleine-koenig.org>
References: <20210927204326.612555-1-uwe@kleine-koenig.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Patch-Hashes: v=1; h=sha256; i=/pZgVMb346xYWFyKXoYzQiwqLbFb4Qw5wqFIqjkKa+4=; m=FyMqGYUG+9Ut7bWwCBeeUgHOxoK/KbMAjt+cyx+MULA=; p=7uJBwDup3eOXFa/jaxic6at398GqkPM+jAglE1o0ido=; g=d846534d35b8420e8bfb08112981cf20b1b1fab2
X-Patch-Sig: m=pgp; i=uwe@kleine-koenig.org; s=0x0D2511F322BFAB1C1580266BE2DCDD9132669BD6; b=iQEzBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFSLNwACgkQwfwUeK3K7AlNHwf/cK0 S4hDoGYclc0hVBG+zvNTPyNzWBwmEv6KLpH+WHGr672nd2H589gYO7W4HXL+jc+oA9TVgciT+/78D NdxhhLRv3qCJowREiQaQBOzfrC1dENMDPeQarjt0X1eTsH7JYw7sp0KCZCX//+xFD80mwzi0rNq2E MhaqAkYPs8WTeVbTUZMIQ7I9Se/6jwCr8aCP3azRqlL/1BQEkrdDGkZdXAO1VohdV0lDVhZPmhFtD 1Fhm+BCdcbqDf0bhoaeU7fBVYWRI3ynJc26WmiSz/259EAj21kJcUOC/c1vSCo78wHKjMY1J+A06h FHCAtJfZe48rUKTpNiwv6Ph4jI+pRLw==
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

From: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>

struct pci_dev::driver contains (apart from a constant offset) the same
data as struct pci_dev::dev->driver. Replace all remaining users of the
former pointer by the latter to allow removing the former.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
---
 arch/powerpc/kernel/eeh_driver.c | 10 ++++-----
 arch/x86/events/intel/uncore.c   |  2 +-
 arch/x86/kernel/probe_roms.c     |  2 +-
 drivers/misc/cxl/guest.c         | 24 ++++++++++++---------
 drivers/misc/cxl/pci.c           | 30 ++++++++++++++++----------
 drivers/pci/iov.c                | 25 ++++++++++++++--------
 drivers/pci/pci-driver.c         | 25 +++++++++++-----------
 drivers/pci/pci.c                |  4 ++--
 drivers/pci/pcie/err.c           | 36 ++++++++++++++++++--------------
 drivers/pci/xen-pcifront.c       |  4 ++--
 drivers/usb/host/xhci-pci.c      |  2 +-
 11 files changed, 93 insertions(+), 71 deletions(-)

diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3eff6a4888e7..350dab18e137 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -104,13 +104,13 @@ static bool eeh_edev_actionable(struct eeh_dev *edev)
  */
 static inline struct pci_driver *eeh_pcid_get(struct pci_dev *pdev)
 {
-	if (!pdev || !pdev->driver)
+	if (!pdev || !pdev->dev.driver)
 		return NULL;
 
-	if (!try_module_get(pdev->driver->driver.owner))
+	if (!try_module_get(pdev->dev.driver->owner))
 		return NULL;
 
-	return pdev->driver;
+	return to_pci_driver(pdev->dev.driver);
 }
 
 /**
@@ -122,10 +122,10 @@ static inline struct pci_driver *eeh_pcid_get(struct pci_dev *pdev)
  */
 static inline void eeh_pcid_put(struct pci_dev *pdev)
 {
-	if (!pdev || !pdev->driver)
+	if (!pdev || !pdev->dev.driver)
 		return;
 
-	module_put(pdev->driver->driver.owner);
+	module_put(pdev->dev.driver->owner);
 }
 
 /**
diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c
index c72e368dd164..f1ba6ab2e97e 100644
--- a/arch/x86/events/intel/uncore.c
+++ b/arch/x86/events/intel/uncore.c
@@ -1187,7 +1187,7 @@ static int uncore_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id
 	 * PCI slot and func to indicate the uncore box.
 	 */
 	if (id->driver_data & ~0xffff) {
-		struct pci_driver *pci_drv = pdev->driver;
+		struct pci_driver *pci_drv = to_pci_driver(pdev->dev.driver);
 
 		pmu = uncore_pci_find_dev_pmu(pdev, pci_drv->id_table);
 		if (pmu == NULL)
diff --git a/arch/x86/kernel/probe_roms.c b/arch/x86/kernel/probe_roms.c
index 9e1def3744f2..36e84d904260 100644
--- a/arch/x86/kernel/probe_roms.c
+++ b/arch/x86/kernel/probe_roms.c
@@ -80,7 +80,7 @@ static struct resource video_rom_resource = {
  */
 static bool match_id(struct pci_dev *pdev, unsigned short vendor, unsigned short device)
 {
-	struct pci_driver *drv = pdev->driver;
+	struct pci_driver *drv = to_pci_driver(pdev->dev.driver);
 	const struct pci_device_id *id;
 
 	if (pdev->vendor == vendor && pdev->device == device)
diff --git a/drivers/misc/cxl/guest.c b/drivers/misc/cxl/guest.c
index 186308f1f8eb..d997c9c3ebb5 100644
--- a/drivers/misc/cxl/guest.c
+++ b/drivers/misc/cxl/guest.c
@@ -25,28 +25,32 @@ static void pci_error_handlers(struct cxl_afu *afu,
 		return;
 
 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-		if (!afu_dev->driver)
+		struct pci_driver *afu_drv;
+
+		if (!afu_dev->dev.driver)
 			continue;
 
+		afu_drv = to_pci_driver(afu_dev->dev.driver);
+
 		switch (bus_error_event) {
 		case CXL_ERROR_DETECTED_EVENT:
 			afu_dev->error_state = state;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->error_detected)
-				afu_dev->driver->err_handler->error_detected(afu_dev, state);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->error_detected)
+				afu_drv->err_handler->error_detected(afu_dev, state);
 		break;
 		case CXL_SLOT_RESET_EVENT:
 			afu_dev->error_state = state;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->slot_reset)
-				afu_dev->driver->err_handler->slot_reset(afu_dev);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->slot_reset)
+				afu_drv->err_handler->slot_reset(afu_dev);
 		break;
 		case CXL_RESUME_EVENT:
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->resume)
-				afu_dev->driver->err_handler->resume(afu_dev);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->resume)
+				afu_drv->err_handler->resume(afu_dev);
 		break;
 		}
 	}
diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
index 2ba899f5659f..7e7545d01e27 100644
--- a/drivers/misc/cxl/pci.c
+++ b/drivers/misc/cxl/pci.c
@@ -1805,14 +1805,16 @@ static pci_ers_result_t cxl_vphb_error_detected(struct cxl_afu *afu,
 		return result;
 
 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-		if (!afu_dev->driver)
+		struct pci_driver *afu_drv;
+		if (!afu_dev->dev.driver)
 			continue;
 
+		afu_drv = to_pci_driver(afu_dev->dev.driver);
+
 		afu_dev->error_state = state;
 
-		if (afu_dev->driver->err_handler)
-			afu_result = afu_dev->driver->err_handler->error_detected(afu_dev,
-										  state);
+		if (afu_drv->err_handler)
+			afu_result = afu_drv->err_handler->error_detected(afu_dev, state);
 		/* Disconnect trumps all, NONE trumps NEED_RESET */
 		if (afu_result == PCI_ERS_RESULT_DISCONNECT)
 			result = PCI_ERS_RESULT_DISCONNECT;
@@ -2003,6 +2005,8 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
 			continue;
 
 		list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
+			struct pci_driver *afu_drv;
+
 			/* Reset the device context.
 			 * TODO: make this less disruptive
 			 */
@@ -2028,12 +2032,14 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
 			 * shouldn't start new work until we call
 			 * their resume function.
 			 */
-			if (!afu_dev->driver)
+			if (!afu_dev->dev.driver)
 				continue;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->slot_reset)
-				afu_result = afu_dev->driver->err_handler->slot_reset(afu_dev);
+			afu_drv = to_pci_driver(afu_dev->dev.driver);
+
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->slot_reset)
+				afu_result = afu_drv->err_handler->slot_reset(afu_dev);
 
 			if (afu_result == PCI_ERS_RESULT_DISCONNECT)
 				result = PCI_ERS_RESULT_DISCONNECT;
@@ -2074,9 +2080,11 @@ static void cxl_pci_resume(struct pci_dev *pdev)
 			continue;
 
 		list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-			if (afu_dev->driver && afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->resume)
-				afu_dev->driver->err_handler->resume(afu_dev);
+			struct pci_driver *afu_drv;
+			if (afu_dev->dev.driver &&
+			    (afu_drv = to_pci_driver(afu_dev->dev.driver))->err_handler &&
+			    afu_drv->err_handler->resume)
+				afu_drv->err_handler->resume(afu_dev);
 		}
 	}
 	spin_unlock(&adapter->afu_list_lock);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index dafdc652fcd0..f94660927544 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -164,13 +164,15 @@ static ssize_t sriov_vf_total_msix_show(struct device *dev,
 					char *buf)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pci_driver *pdrv;
 	u32 vf_total_msix = 0;
 
 	device_lock(dev);
-	if (!pdev->driver || !pdev->driver->sriov_get_vf_total_msix)
+	pdrv = to_pci_driver(dev->driver);
+	if (!pdrv || !pdrv->sriov_get_vf_total_msix)
 		goto unlock;
 
-	vf_total_msix = pdev->driver->sriov_get_vf_total_msix(pdev);
+	vf_total_msix = pdrv->sriov_get_vf_total_msix(pdev);
 unlock:
 	device_unlock(dev);
 	return sysfs_emit(buf, "%u\n", vf_total_msix);
@@ -183,6 +185,7 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 {
 	struct pci_dev *vf_dev = to_pci_dev(dev);
 	struct pci_dev *pdev = pci_physfn(vf_dev);
+	struct pci_driver *pdrv;
 	int val, ret;
 
 	ret = kstrtoint(buf, 0, &val);
@@ -193,13 +196,14 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 		return -EINVAL;
 
 	device_lock(&pdev->dev);
-	if (!pdev->driver || !pdev->driver->sriov_set_msix_vec_count) {
+	pdrv = to_pci_driver(pdev->dev.driver);
+	if (!pdrv || !pdrv->sriov_set_msix_vec_count) {
 		ret = -EOPNOTSUPP;
 		goto err_pdev;
 	}
 
 	device_lock(&vf_dev->dev);
-	if (vf_dev->driver) {
+	if (vf_dev->dev.driver) {
 		/*
 		 * A driver is already attached to this VF and has configured
 		 * itself based on the current MSI-X vector count. Changing
@@ -209,7 +213,7 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 		goto err_dev;
 	}
 
-	ret = pdev->driver->sriov_set_msix_vec_count(vf_dev, val);
+	ret = pdrv->sriov_set_msix_vec_count(vf_dev, val);
 
 err_dev:
 	device_unlock(&vf_dev->dev);
@@ -376,6 +380,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 				  const char *buf, size_t count)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pci_driver *pdrv;
 	int ret;
 	u16 num_vfs;
 
@@ -392,14 +397,16 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 		goto exit;
 
 	/* is PF driver loaded */
-	if (!pdev->driver) {
+	if (!pdev->dev.driver) {
 		pci_info(pdev, "no driver bound to device; cannot configure SR-IOV\n");
 		ret = -ENOENT;
 		goto exit;
 	}
 
+	pdrv = to_pci_driver(pdev->dev.driver);
+
 	/* is PF driver loaded w/callback */
-	if (!pdev->driver->sriov_configure) {
+	if (!pdrv->sriov_configure) {
 		pci_info(pdev, "driver does not support SR-IOV configuration via sysfs\n");
 		ret = -ENOENT;
 		goto exit;
@@ -407,7 +414,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 
 	if (num_vfs == 0) {
 		/* disable VFs */
-		ret = pdev->driver->sriov_configure(pdev, 0);
+		ret = pdrv->sriov_configure(pdev, 0);
 		goto exit;
 	}
 
@@ -419,7 +426,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 		goto exit;
 	}
 
-	ret = pdev->driver->sriov_configure(pdev, num_vfs);
+	ret = pdrv->sriov_configure(pdev, num_vfs);
 	if (ret < 0)
 		goto exit;
 
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index 50449ec622a3..4d20022b8631 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -457,7 +457,7 @@ static int pci_device_probe(struct device *dev)
 static void pci_device_remove(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	if (drv->remove) {
 		pm_runtime_get_sync(dev);
@@ -493,7 +493,7 @@ static void pci_device_remove(struct device *dev)
 static void pci_device_shutdown(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	pm_runtime_resume(dev);
 
@@ -589,7 +589,7 @@ static int pci_pm_reenable_device(struct pci_dev *pci_dev)
 static int pci_legacy_suspend(struct device *dev, pm_message_t state)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(dev->driver);
 
 	if (drv && drv->suspend) {
 		pci_power_t prev = pci_dev->current_state;
@@ -630,7 +630,7 @@ static int pci_legacy_suspend_late(struct device *dev, pm_message_t state)
 static int pci_legacy_resume(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	pci_fixup_device(pci_fixup_resume, pci_dev);
 
@@ -649,7 +649,7 @@ static void pci_pm_default_suspend(struct pci_dev *pci_dev)
 
 static bool pci_has_legacy_pm_support(struct pci_dev *pci_dev)
 {
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 	bool ret = drv && (drv->suspend || drv->resume);
 
 	/*
@@ -1242,11 +1242,11 @@ static int pci_pm_runtime_suspend(struct device *dev)
 	int error;
 
 	/*
-	 * If pci_dev->driver is not set (unbound), we leave the device in D0,
+	 * If pci_dev->dev.driver is not set (unbound), we leave the device in D0,
 	 * but it may go to D3cold when the bridge above it runtime suspends.
 	 * Save its config space in case that happens.
 	 */
-	if (!pci_dev->driver) {
+	if (!pci_dev->dev.driver) {
 		pci_save_state(pci_dev);
 		return 0;
 	}
@@ -1303,7 +1303,7 @@ static int pci_pm_runtime_resume(struct device *dev)
 	 */
 	pci_restore_standard_config(pci_dev);
 
-	if (!pci_dev->driver)
+	if (!dev->driver)
 		return 0;
 
 	pci_fixup_device(pci_fixup_resume_early, pci_dev);
@@ -1322,14 +1322,13 @@ static int pci_pm_runtime_resume(struct device *dev)
 
 static int pci_pm_runtime_idle(struct device *dev)
 {
-	struct pci_dev *pci_dev = to_pci_dev(dev);
 	const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
 
 	/*
-	 * If pci_dev->driver is not set (unbound), the device should
+	 * If dev->driver is not set (unbound), the device should
 	 * always remain in D0 regardless of the runtime PM status
 	 */
-	if (!pci_dev->driver)
+	if (!dev->driver)
 		return 0;
 
 	if (!pm)
@@ -1436,8 +1435,8 @@ static struct pci_driver pci_compat_driver = {
  */
 struct pci_driver *pci_dev_driver(const struct pci_dev *dev)
 {
-	if (dev->driver)
-		return dev->driver;
+	if (dev->dev.driver)
+		return to_pci_driver(dev->dev.driver);
 	else {
 		int i;
 		for (i = 0; i <= PCI_ROM_RESOURCE; i++)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ce2ab62b64cf..ccecf740de59 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5089,7 +5089,7 @@ EXPORT_SYMBOL_GPL(pci_dev_unlock);
 static void pci_dev_save_and_disable(struct pci_dev *dev)
 {
 	const struct pci_error_handlers *err_handler =
-			dev->driver ? dev->driver->err_handler : NULL;
+			dev->dev.driver ? to_pci_driver(dev->dev.driver)->err_handler : NULL;
 
 	/*
 	 * dev->driver->err_handler->reset_prepare() is protected against
@@ -5120,7 +5120,7 @@ static void pci_dev_save_and_disable(struct pci_dev *dev)
 static void pci_dev_restore(struct pci_dev *dev)
 {
 	const struct pci_error_handlers *err_handler =
-			dev->driver ? dev->driver->err_handler : NULL;
+			dev->dev.driver ? to_pci_driver(dev->dev.driver)->err_handler : NULL;
 
 	pci_restore_state(dev);
 
diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
index b576aa890c76..b314b54f7821 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -49,14 +49,15 @@ static int report_error_detected(struct pci_dev *dev,
 				 pci_channel_state_t state,
 				 enum pci_ers_result *result)
 {
+	struct pci_driver *pdrv;
 	pci_ers_result_t vote;
 	const struct pci_error_handlers *err_handler;
 
 	device_lock(&dev->dev);
 	if (!pci_dev_set_io_state(dev, state) ||
-		!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->error_detected) {
+		!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->error_detected) {
 		/*
 		 * If any device in the subtree does not have an error_detected
 		 * callback, PCI_ERS_RESULT_NO_AER_DRIVER prevents subsequent
@@ -70,7 +71,7 @@ static int report_error_detected(struct pci_dev *dev,
 			vote = PCI_ERS_RESULT_NONE;
 		}
 	} else {
-		err_handler = dev->driver->err_handler;
+		err_handler = pdrv->err_handler;
 		vote = err_handler->error_detected(dev, state);
 	}
 	pci_uevent_ers(dev, vote);
@@ -92,15 +93,16 @@ static int report_normal_detected(struct pci_dev *dev, void *data)
 static int report_mmio_enabled(struct pci_dev *dev, void *data)
 {
 	pci_ers_result_t vote, *result = data;
+	struct pci_driver *pdrv;
 	const struct pci_error_handlers *err_handler;
 
 	device_lock(&dev->dev);
-	if (!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->mmio_enabled)
+	if (!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->mmio_enabled)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	vote = err_handler->mmio_enabled(dev);
 	*result = merge_result(*result, vote);
 out:
@@ -112,14 +114,15 @@ static int report_slot_reset(struct pci_dev *dev, void *data)
 {
 	pci_ers_result_t vote, *result = data;
 	const struct pci_error_handlers *err_handler;
+	struct pci_driver *pdrv;
 
 	device_lock(&dev->dev);
-	if (!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->slot_reset)
+	if (!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->slot_reset)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	vote = err_handler->slot_reset(dev);
 	*result = merge_result(*result, vote);
 out:
@@ -130,15 +133,16 @@ static int report_slot_reset(struct pci_dev *dev, void *data)
 static int report_resume(struct pci_dev *dev, void *data)
 {
 	const struct pci_error_handlers *err_handler;
+	struct pci_driver *pdrv;
 
 	device_lock(&dev->dev);
 	if (!pci_dev_set_io_state(dev, pci_channel_io_normal) ||
-		!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->resume)
+		!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->resume)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	err_handler->resume(dev);
 out:
 	pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index f2d7f70a7a10..73831fb87a1e 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -601,12 +601,12 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	result = PCI_ERS_RESULT_NONE;
 
 	pcidev = pci_get_domain_bus_and_slot(domain, bus, devfn);
-	if (!pcidev || !pcidev->driver) {
+	if (!pcidev || !pcidev->dev.driver) {
 		dev_err(&pdev->xdev->dev, "device or AER driver is NULL\n");
 		pci_dev_put(pcidev);
 		return result;
 	}
-	pdrv = pcidev->driver;
+	pdrv = to_pci_driver(pcidev->dev.driver);
 
 	if (pdrv->err_handler && pdrv->err_handler->error_detected) {
 		pci_dbg(pcidev, "trying to call AER service\n");
diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c
index 2c9f25ca8edd..2f4729f4f1e0 100644
--- a/drivers/usb/host/xhci-pci.c
+++ b/drivers/usb/host/xhci-pci.c
@@ -103,7 +103,7 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci)
 	struct xhci_driver_data         *driver_data;
 	const struct pci_device_id      *id;
 
-	id = pci_match_id(pdev->driver->id_table, pdev);
+	id = pci_match_id(to_pci_driver(pdev->dev.driver)->id_table, pdev);
 
 	if (id && id->driver_data) {
 		driver_data = (struct xhci_driver_data *)id->driver_data;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:02:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:02:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197177.350519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fr-0002FT-H4; Tue, 28 Sep 2021 05:02:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197177.350519; Tue, 28 Sep 2021 05:02:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fr-0002EQ-9M; Tue, 28 Sep 2021 05:02:11 +0000
Received: by outflank-mailman (input) for mailman id 197177;
 Mon, 27 Sep 2021 20:45:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ukqu=OR=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mUxVC-0008QW-7e
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 20:45:30 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b2e0c24d-f998-4b1b-b17b-493016fe405c;
 Mon, 27 Sep 2021 20:45:27 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTe-0001jl-FQ; Mon, 27 Sep 2021 22:43:54 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTX-0001YL-VH; Mon, 27 Sep 2021 22:43:47 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTX-0001Li-Pk; Mon, 27 Sep 2021 22:43:47 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2e0c24d-f998-4b1b-b17b-493016fe405c
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <uwe@kleine-koenig.org>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Alexander Duyck <alexanderduyck@fb.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Andrew Donnellan <ajd@linux.ibm.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Arnd Bergmann <arnd@arndb.de>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Borislav Petkov <bp@alien8.de>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	"David S. Miller" <davem@davemloft.net>,
	Fiona Trahe <fiona.trahe@intel.com>,
	Frederic Barrat <fbarrat@linux.ibm.com>,
	Giovanni Cabiddu <giovanni.cabiddu@intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Ido Schimmel <idosch@nvidia.com>,
	Ingo Molnar <mingo@redhat.com>,
	Jack Xu <jack.xu@intel.com>,
	Jakub Kicinski <kuba@kernel.org>,
	Jesse Brandeburg <jesse.brandeburg@intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Jiri Pirko <jiri@nvidia.com>,
	Juergen Gross <jgross@suse.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Marco Chiappero <marco.chiappero@intel.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Mathias Nyman <mathias.nyman@intel.com>,
	Michael Buesch <m@bues.ch>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Namhyung Kim <namhyung@kernel.org>,
	"Oliver O'Halloran" <oohall@gmail.com>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <peterz@infradead.org>,
	=?UTF-8?q?Rafa=C5=82=20Mi=C5=82ecki?= <zajec5@gmail.com>,
	Russell Currey <ruscur@russell.cc>,
	Salil Mehta <salil.mehta@huawei.com>,
	Sathya Prakash <sathya.prakash@broadcom.com>,
	Simon Horman <simon.horman@corigine.com>,
	Sreekanth Reddy <sreekanth.reddy@broadcom.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
	Taras Chornyi <tchornyi@marvell.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Tomaszx Kowalik <tomaszx.kowalik@intel.com>,
	Vadym Kochan <vkochan@marvell.com>,
	Wojciech Ziemba <wojciech.ziemba@intel.com>,
	Yisen Zhuang <yisen.zhuang@huawei.com>,
	Zhou Wang <wangzhou1@hisilicon.com>,
	linux-crypto@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-perf-users@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	linux-scsi@vger.kernel.org,
	linux-usb@vger.kernel.org,
	linux-wireless@vger.kernel.org,
	MPT-FusionLinux.pdl@broadcom.com,
	netdev@vger.kernel.org,
	oss-drivers@corigine.com,
	qat-linux@intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v4 0/8] PCI: Drop duplicated tracking of a pci_dev's bound driver
Date: Mon, 27 Sep 2021 22:43:18 +0200
Message-Id: <20210927204326.612555-1-uwe@kleine-koenig.org>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

From: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>

Hello,

this is v4 of the quest to drop the "driver" member from struct pci_dev
which tracks the same data (apart from a constant offset) as dev.driver.

Changes since v3:
 - Add some Reviewed-by and Acked-by tags
 - Rebase to v5.15-rc3 (no conflicts)
 - Changes in patch #4 addressing review comments by Christoph Hellwig

I didn't do extensive build tests, so I might have missed a build
problem. I have some builds running, but want to get some feedback on
the changes suggested by Christoph.

Best regards
Uwe

Uwe Kleine-König (8):
  PCI: Simplify pci_device_remove()
  PCI: Drop useless check from pci_device_probe()
  xen/pci: Drop some checks that are always true
  PCI: replace pci_dev::driver usage that gets the driver name
  scsi: message: fusion: Remove unused parameter of mpt_pci driver's
    probe()
  crypto: qat - simplify adf_enable_aer()
  PCI: Replace pci_dev::driver usage by pci_dev::dev.driver
  PCI: Drop duplicated tracking of a pci_dev's bound driver

 arch/powerpc/include/asm/ppc-pci.h            |  9 ++-
 arch/powerpc/kernel/eeh_driver.c              | 10 +--
 arch/x86/events/intel/uncore.c                |  2 +-
 arch/x86/kernel/probe_roms.c                  |  2 +-
 drivers/bcma/host_pci.c                       |  7 ++-
 drivers/crypto/hisilicon/qm.c                 |  2 +-
 drivers/crypto/qat/qat_4xxx/adf_drv.c         |  7 +--
 drivers/crypto/qat/qat_c3xxx/adf_drv.c        |  7 +--
 drivers/crypto/qat/qat_c62x/adf_drv.c         |  7 +--
 drivers/crypto/qat/qat_common/adf_aer.c       | 10 +--
 .../crypto/qat/qat_common/adf_common_drv.h    |  2 +-
 drivers/crypto/qat/qat_dh895xcc/adf_drv.c     |  7 +--
 drivers/message/fusion/mptbase.c              |  7 +--
 drivers/message/fusion/mptbase.h              |  2 +-
 drivers/message/fusion/mptctl.c               |  4 +-
 drivers/message/fusion/mptlan.c               |  2 +-
 drivers/misc/cxl/guest.c                      | 24 ++++---
 drivers/misc/cxl/pci.c                        | 30 +++++----
 .../ethernet/hisilicon/hns3/hns3_ethtool.c    |  2 +-
 .../ethernet/marvell/prestera/prestera_pci.c  |  2 +-
 drivers/net/ethernet/mellanox/mlxsw/pci.c     |  2 +-
 .../ethernet/netronome/nfp/nfp_net_ethtool.c  |  2 +-
 drivers/pci/iov.c                             | 25 +++++---
 drivers/pci/pci-driver.c                      | 45 ++++++-------
 drivers/pci/pci.c                             |  4 +-
 drivers/pci/pcie/err.c                        | 36 ++++++-----
 drivers/pci/xen-pcifront.c                    | 63 +++++++++----------
 drivers/ssb/pcihost_wrapper.c                 |  8 ++-
 drivers/usb/host/xhci-pci.c                   |  2 +-
 include/linux/pci.h                           |  1 -
 30 files changed, 166 insertions(+), 167 deletions(-)


base-commit: 5816b3e6577eaa676ceb00a848f0fd65fe2adc29
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:02:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:02:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197173.350507 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fq-00025W-P8; Tue, 28 Sep 2021 05:02:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197173.350507; Tue, 28 Sep 2021 05:02:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5Fq-00025P-M9; Tue, 28 Sep 2021 05:02:10 +0000
Received: by outflank-mailman (input) for mailman id 197173;
 Mon, 27 Sep 2021 20:44:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ukqu=OR=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mUxTw-0008O6-8g
 for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 20:44:12 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5451584a-43a2-47fe-be1c-b8aff57295f3;
 Mon, 27 Sep 2021 20:44:06 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTa-0001jo-U0; Mon, 27 Sep 2021 22:43:50 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTY-0001YU-BC; Mon, 27 Sep 2021 22:43:48 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mUxTY-0001N9-9X; Mon, 27 Sep 2021 22:43:48 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5451584a-43a2-47fe-be1c-b8aff57295f3
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <uwe@kleine-koenig.org>
To: Bjorn Helgaas <helgaas@kernel.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>
Cc: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Stefano Stabellini <sstabellini@kernel.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	xen-devel@lists.xenproject.org,
	Christoph Hellwig <hch@lst.de>
Subject: [PATCH v4 3/8] xen/pci: Drop some checks that are always true
Date: Mon, 27 Sep 2021 22:43:21 +0200
Message-Id: <20210927204326.612555-4-uwe@kleine-koenig.org>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210927204326.612555-1-uwe@kleine-koenig.org>
References: <20210927204326.612555-1-uwe@kleine-koenig.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Patch-Hashes: v=1; h=sha256; i=7QOFRnoxyPrPqaGt2Yf8oJvx3xwgCvlMdm130sPrxr8=; m=Ixh7t2kv5MnS1JaJxFtVrqNhjBtz3xQyoCGo2anqbI0=; p=dOfuuSLv1l/M9DCuxyIBdCuX9kCH+yUzno229EZfUDs=; g=9e481f95a51887868b05f8a86c3e5a9f8aff47c6
X-Patch-Sig: m=pgp; i=uwe@kleine-koenig.org; s=0x0D2511F322BFAB1C1580266BE2DCDD9132669BD6; b=iQEzBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFSLMkACgkQwfwUeK3K7AnqIQgAnhz 0fOpo2dlkGtlWUJ6sTE0pwT6RIA2NSk6vrj9r36En7FhfasqWDQb4xX9gZspK5EgYSbybhxpPPFyq 80ehLBLcirNNq2x0pS57eDfwYdvbxOU+nqi2Hc1l1SF1j+CXEW8x9g5bn3MDb8kdclUwMMAK6yXVj bTii6/k2laEZQGj8CnAZmeIkDn8Ateozw5/Ukl6ojqTlWBH5AeuCCMyftDeAtZ42j4wLO45ZEiDx8 Tdw3LCh937pothK2cEWwbY7AdxW4F5JN0OLWnmNil0Cf/txxoSibymGe0GhVSJ1ClE3AM+N6IGrue IAT5IVmrMKtEZWZLy1ujTlCxs22jC6A==
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

From: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>

pcifront_common_process() has a check at the start that exits early if
pcidev or pdidev->driver are NULL. So simplify the following code by not
checking these two again.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
---
 drivers/pci/xen-pcifront.c | 57 +++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 32 deletions(-)

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index 2156c632524d..f2d7f70a7a10 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -594,7 +594,6 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	int devfn = pdev->sh_info->aer_op.devfn;
 	int domain = pdev->sh_info->aer_op.domain;
 	struct pci_dev *pcidev;
-	int flag = 0;
 
 	dev_dbg(&pdev->xdev->dev,
 		"pcifront AER process: cmd %x (bus:%x, devfn%x)",
@@ -609,40 +608,34 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	}
 	pdrv = pcidev->driver;
 
-	if (pdrv) {
-		if (pdrv->err_handler && pdrv->err_handler->error_detected) {
-			pci_dbg(pcidev, "trying to call AER service\n");
-			if (pcidev) {
-				flag = 1;
-				switch (cmd) {
-				case XEN_PCI_OP_aer_detected:
-					result = pdrv->err_handler->
-						 error_detected(pcidev, state);
-					break;
-				case XEN_PCI_OP_aer_mmio:
-					result = pdrv->err_handler->
-						 mmio_enabled(pcidev);
-					break;
-				case XEN_PCI_OP_aer_slotreset:
-					result = pdrv->err_handler->
-						 slot_reset(pcidev);
-					break;
-				case XEN_PCI_OP_aer_resume:
-					pdrv->err_handler->resume(pcidev);
-					break;
-				default:
-					dev_err(&pdev->xdev->dev,
-						"bad request in aer recovery "
-						"operation!\n");
-
-				}
-			}
+	if (pdrv->err_handler && pdrv->err_handler->error_detected) {
+		pci_dbg(pcidev, "trying to call AER service\n");
+		switch (cmd) {
+		case XEN_PCI_OP_aer_detected:
+			result = pdrv->err_handler->
+				 error_detected(pcidev, state);
+			break;
+		case XEN_PCI_OP_aer_mmio:
+			result = pdrv->err_handler->
+				 mmio_enabled(pcidev);
+			break;
+		case XEN_PCI_OP_aer_slotreset:
+			result = pdrv->err_handler->
+				 slot_reset(pcidev);
+			break;
+		case XEN_PCI_OP_aer_resume:
+			pdrv->err_handler->resume(pcidev);
+			break;
+		default:
+			dev_err(&pdev->xdev->dev,
+				"bad request in aer recovery "
+				"operation!\n");
 		}
+
+		return result;
 	}
-	if (!flag)
-		result = PCI_ERS_RESULT_NONE;
 
-	return result;
+	return PCI_ERS_RESULT_NONE;
 }
 
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:30:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:30:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197477.350540 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5hE-0006xu-TB; Tue, 28 Sep 2021 05:30:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197477.350540; Tue, 28 Sep 2021 05:30:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5hE-0006xn-PJ; Tue, 28 Sep 2021 05:30:28 +0000
Received: by outflank-mailman (input) for mailman id 197477;
 Tue, 28 Sep 2021 05:30:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV5hD-0006xd-Es; Tue, 28 Sep 2021 05:30:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV5hC-0007w9-M4; Tue, 28 Sep 2021 05:30:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV5hC-00027K-Cn; Tue, 28 Sep 2021 05:30:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mV5hC-0001k4-BE; Tue, 28 Sep 2021 05:30:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=K1UBKHyTibbMdlE1U6ATbeURCHifcskNFxkaJDsbvV4=; b=wDYrj/FVUYkfggLhecaEY40Y2X
	/ZI8agyjvXVOqyGAIAdlF+YrIoQiKY64SZkKPqREpu4GcHye4yyV+wGYDu5wboTz2P9BoCgOB6Sr5
	23TSU4vhJOzOPNnsVaCm4xQQC+kFbkjY7VFlUlvHaHILYegzlTW3BFVmQwCPJdEWeaQE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165225-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165225: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0513e464f9007b70b96740271a948ca5ab6e7dd7
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 05:30:26 +0000

flight 165225 linux-linus real [real]
flight 165229 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165225/
http://logs.test-lab.xenproject.org/osstest/logs/165229/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0513e464f9007b70b96740271a948ca5ab6e7dd7
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  423 days
Failing since        152366  2020-08-01 20:49:34 Z  422 days  739 attempts
Testing same since   165225  2021-09-27 22:11:09 Z    0 days    1 attempts

------------------------------------------------------------
7347 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2273419 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:39:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:39:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197487.350554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5pn-0007pc-R2; Tue, 28 Sep 2021 05:39:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197487.350554; Tue, 28 Sep 2021 05:39:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5pn-0007pV-No; Tue, 28 Sep 2021 05:39:19 +0000
Received: by outflank-mailman (input) for mailman id 197487;
 Tue, 28 Sep 2021 05:39:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EjFJ=OS=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mV5pm-0007pP-Rx
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:39:18 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6b8c4028-201e-11ec-bc68-12813bfff9fa;
 Tue, 28 Sep 2021 05:39:17 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id ED5FC67373; Tue, 28 Sep 2021 07:39:11 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b8c4028-201e-11ec-bc68-12813bfff9fa
Date: Tue, 28 Sep 2021 07:39:11 +0200
From: Christoph Hellwig <hch@lst.de>
To: Tianyu Lan <ltykernel@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>,
	"iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	vkuznets <vkuznets@redhat.com>,
	"parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>,
	Michael Kelley <mikelley@microsoft.com>,
	KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	"wei.liu@kernel.org" <wei.liu@kernel.org>,
	Dexuan Cui <decui@microsoft.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
	"jgross@suse.com" <jgross@suse.com>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>,
	"thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	"pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>,
	"sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
	"aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
	"saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>,
	"rientjes@google.com" <rientjes@google.com>
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
 netvsc driver
Message-ID: <20210928053911.GA29208@lst.de>
References: <20210914133916.1440931-1-ltykernel@gmail.com> <20210914133916.1440931-13-ltykernel@gmail.com> <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com> <43e22b84-7273-4099-42ea-54b06f398650@gmail.com> <e379a60b-4d74-9167-983f-f70c96bb279e@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e379a60b-4d74-9167-983f-f70c96bb279e@gmail.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Mon, Sep 27, 2021 at 10:26:43PM +0800, Tianyu Lan wrote:
> Hi Christoph:
>     Gentile ping. The swiotlb and shared memory mapping changes in this
> patchset needs your reivew. Could you have a look?

I'm a little too busy for a review of such a huge patchset right now.
That being said here are my comments from a very quick review:

 - the bare memremap usage in swiotlb looks strange and I'd
   definitively expect a well documented wrapper.
 - given that we can now hand out swiotlb memory for coherent mappings
   we need to carefully audit what happens when this memremaped
   memory gets mmaped or used through dma_get_sgtable
 - the netscv changes I'm not happy with at all.  A large part of it
   is that the driver already has a bad structure, but this series
   is making it significantly worse.  We'll need to find a way
   to use the proper dma mapping abstractions here.  One option
   if you want to stick to the double vmapped buffer would be something
   like using dma_alloc_noncontigous plus a variant of
   dma_vmap_noncontiguous that takes the shared_gpa_boundary into
   account.


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:39:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:39:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197494.350565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5qQ-0008Re-9D; Tue, 28 Sep 2021 05:39:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197494.350565; Tue, 28 Sep 2021 05:39:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV5qQ-0008RX-4V; Tue, 28 Sep 2021 05:39:58 +0000
Received: by outflank-mailman (input) for mailman id 197494;
 Tue, 28 Sep 2021 05:39:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Tiex=OS=suse.de=hare@srs-us1.protection.inumbo.net>)
 id 1mV5qP-0008RP-Aq
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:39:57 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e31740fb-978c-4aa5-ba91-342dbd110875;
 Tue, 28 Sep 2021 05:39:55 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 0C7BD201BB;
 Tue, 28 Sep 2021 05:39:55 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AB01913A98;
 Tue, 28 Sep 2021 05:39:54 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id JO5JKKqqUmHUNQAAMHmgww
 (envelope-from <hare@suse.de>); Tue, 28 Sep 2021 05:39:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e31740fb-978c-4aa5-ba91-342dbd110875
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa;
	t=1632807595; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=13TnURwhS8h4t01lwoZLD/d/5oPL9agrfycanDfeTRQ=;
	b=PT9kgFrx9DkgIx21N+ey4rvrG+p5lgzclkflex7M8ulg7b/MPe/8YBUPSSqsTo2GTahbjJ
	Gey5RuOSF1hNpbYAFgKbf2oL0K4d67fdRAt+eRdY+iU4RsWpBKefK/X7oBdujgebC7iQbz
	cH9ujOHUnez6nwSvlDTdjkQYCRSdxBg=
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de;
	s=susede2_ed25519; t=1632807595;
	h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=13TnURwhS8h4t01lwoZLD/d/5oPL9agrfycanDfeTRQ=;
	b=VnyBApxFuoR87JCq7TgLnoYycKzbt1M/1osAF+bIrqg5sP1C+l/tWJFxYA8+084TxkpCY5
	Hr8IhCD3XymifdAw==
Subject: Re: [PATCH v2 03/10] nvme-multipath: add error handling support for
 add_disk()
To: Luis Chamberlain <mcgrof@kernel.org>, axboe@kernel.dk, colyli@suse.de,
 kent.overstreet@gmail.com, kbusch@kernel.org, sagi@grimberg.me,
 vishal.l.verma@intel.com, dan.j.williams@intel.com, dave.jiang@intel.com,
 ira.weiny@intel.com, konrad.wilk@oracle.com, roger.pau@citrix.com,
 boris.ostrovsky@oracle.com, jgross@suse.com, sstabellini@kernel.org,
 minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org, nvdimm@lists.linux.dev,
 linux-nvme@lists.infradead.org, linux-bcache@vger.kernel.org,
 linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20210927220039.1064193-1-mcgrof@kernel.org>
 <20210927220039.1064193-4-mcgrof@kernel.org>
From: Hannes Reinecke <hare@suse.de>
Message-ID: <85d33b80-1d54-67ba-47f9-0298093b6d80@suse.de>
Date: Tue, 28 Sep 2021 07:39:54 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210927220039.1064193-4-mcgrof@kernel.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 9/28/21 12:00 AM, Luis Chamberlain wrote:
> We never checked for errors on add_disk() as this function
> returned void. Now that this is fixed, use the shiny new
> error handling.
> 
> Since we now can tell for sure when a disk was added, move
> setting the bit NVME_NSHEAD_DISK_LIVE only when we did
> add the disk successfully.
> 
> Nothing to do here as the cleanup is done elsewhere. We take
> care and use test_and_set_bit() because it is protects against
> two nvme paths simultaneously calling device_add_disk() on the
> same namespace head.
> 
> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
> ---
>   drivers/nvme/host/multipath.c | 13 +++++++++++--
>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index e8ccdd398f78..35cace4f3f5f 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -496,13 +496,22 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
>   static void nvme_mpath_set_live(struct nvme_ns *ns)
>   {
>   	struct nvme_ns_head *head = ns->head;
> +	int rc;
>   
>   	if (!head->disk)
>   		return;
>   
> +	/*
> +	 * test_and_set_bit() is used because it is protecting against two nvme
> +	 * paths simultaneously calling device_add_disk() on the same namespace
> +	 * head.
> +	 */
>   	if (!test_and_set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags)) {
> -		device_add_disk(&head->subsys->dev, head->disk,
> -				nvme_ns_id_attr_groups);
> +		rc = device_add_disk(&head->subsys->dev, head->disk,
> +				     nvme_ns_id_attr_groups);
> +		if (rc)
> +			return;
> +		set_bit(NVME_NSHEAD_DISK_LIVE, &head->flags);
>   		nvme_add_ns_head_cdev(head);
>   	}
>   
> Errm.
Setting the same bit twice?
And shouldn't you unset the bit if 'device_add_disk()' fails?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:55:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:55:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197503.350576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV65U-0002QU-K8; Tue, 28 Sep 2021 05:55:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197503.350576; Tue, 28 Sep 2021 05:55:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV65U-0002QN-Gi; Tue, 28 Sep 2021 05:55:32 +0000
Received: by outflank-mailman (input) for mailman id 197503;
 Tue, 28 Sep 2021 05:55:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV65S-0002QH-S6
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:55:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id af5a94a6-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:55:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1468861206;
 Tue, 28 Sep 2021 05:55:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af5a94a6-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808529;
	bh=C4JFrolUIj98OdoLWZp8qV8Z3V4eGkr6TAhkRHp8NXw=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=TXb2S6EQZVJH+hiEVbdFfoQGpIyUOpEh4cDQx9PjUR/Hey6rsSEpDqa8Watpvnf2X
	 qGDpGYuXBvM2GUJrSjKHFVXPK1O0IsYpu1M9qrGZOZEhNJ/YtmbeBRZYtuZk7i4fn3
	 xxOMgZfn6cJiVlMZxBdfioR7Qv5mndUSNBl0/2KS69IEcLpDXRKv+4dywiZh0lAPWI
	 Bk7WWnYheG9lWYgHdnn6aKIC464xfoRMd+tuLwbanHacydim8ZByKZ05XlQcNntUWD
	 TkFIvZVogdWMuto95dArHd7DsET/gZtQfaH4HEO1PToJAbwfHQWBihCPcoLN9JO540
	 +X7YEYbLazvhA==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 5.14 09/40] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:54:53 -0400
Message-Id: <20210928055524.172051-9-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055524.172051-1-sashal@kernel.org>
References: <20210928055524.172051-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 39a01c2a3058..32d5bc4919d8 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -499,7 +499,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:55:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197506.350586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV65r-0002uZ-SF; Tue, 28 Sep 2021 05:55:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197506.350586; Tue, 28 Sep 2021 05:55:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV65r-0002uS-PH; Tue, 28 Sep 2021 05:55:55 +0000
Received: by outflank-mailman (input) for mailman id 197506;
 Tue, 28 Sep 2021 05:55:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9GlL=OS=gmail.com=christopher.w.clark@srs-us1.protection.inumbo.net>)
 id 1mV65p-0002u1-T8
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:55:54 +0000
Received: from mail-oi1-x22e.google.com (unknown [2607:f8b0:4864:20::22e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 77f6268b-5510-49a2-9c78-e114cd2ee0fb;
 Tue, 28 Sep 2021 05:55:52 +0000 (UTC)
Received: by mail-oi1-x22e.google.com with SMTP id n64so9361677oih.2
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 22:55:52 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77f6268b-5510-49a2-9c78-e114cd2ee0fb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=5rEci0qlxlhucxbwe0MgbNSwwC4MRsbfUGnErsRIOcs=;
        b=gDUD4ch2MBk7xVOywI4CsqRw2wrGLzUsGpCiGyr8vb9/WLR0c011zWNCBced0zQg8M
         Xx6yaeqLxus6cVxXJTO8jD7ee3MLIlMcxXRF1GQ5a60gSPHPH2aEgcNb/KqoSUdz8nw8
         vsXnsGHXmxpAGR8xawClxxV9CdK6g9q6G1gpF+D3NYUb2fv1IF8o5ulJ3Yc0kPtyivYl
         fvEnnw9FRSOFVYm/XfjRSaeGGMqrMwqhjapPuJ+ZI53RBu4ouXM/rL/j2Am4YJ1rDjCX
         g92RRctSOVcbw3C7x3Jg6sw93yDp9gFqZ52uspT/46xeqk01VNiIqepsHEjcU9owwb5x
         HwzA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=5rEci0qlxlhucxbwe0MgbNSwwC4MRsbfUGnErsRIOcs=;
        b=vPlUo9MPPsrAppCZe5VNqNNTzhH8O7CEVJprFjf4tvXkLPAorTUgzJqVEeoYn+iq5y
         lhD/gYGZWc1APT34RhhGCADmzZ0hdJVeo2wxlybur+OhdUc6CTAqM6lkj2jkuoRLfI2y
         gIrvSkmXGrS0612AOjVn/nCDsnTf2pp7DvGDgftDh3AkNPaq+DGDnpEJVbNw1j/H3AOq
         Pc6gIYnJjsYvJjqiZSkxd1iqketSslEFg2qtPkP0XPftkekGEB52JjWmXgMkn6JLUbgv
         xQkxzke1btEHDMbj5e9YA7wLkE8PxBWIYj+YaA3RQQoaIUyM/EuPCIYEsraptBSrqaO9
         P/XA==
X-Gm-Message-State: AOAM5324Wx2+u2O5fK3VQDVT5zig2p15YOJ800pMxOaLPAOynZ0DXgOA
	eTrJmgCm7rZuZmFB+lVuZYyEXvpLSyN53HBlSEg=
X-Google-Smtp-Source: ABdhPJzMAKeODXFuOEYSpKf+BUrbUt+6Q3x3vuPi193tyJYLvMcUMaV6lsTRaSjJxG1flfVBH+kFhpoO2EYiBFyhdC8=
X-Received: by 2002:a54:4d8f:: with SMTP id y15mr2293640oix.122.1632808551678;
 Mon, 27 Sep 2021 22:55:51 -0700 (PDT)
MIME-Version: 1.0
References: <87pmsylywy.fsf@linaro.org> <YU5mW396S04IsCBr@mail-itl> <874ka68h96.fsf@linaro.org>
In-Reply-To: <874ka68h96.fsf@linaro.org>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Mon, 27 Sep 2021 22:55:40 -0700
Message-ID: <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
Subject: Re: [Stratos-dev] Xen Rust VirtIO demos work breakdown for Project Stratos
To: =?UTF-8?B?QWxleCBCZW5uw6ll?= <alex.bennee@linaro.org>
Cc: =?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Sergio Lopez <slp@redhat.com>, Wei Liu <wl@xen.org>, 
	Stefan Hajnoczi <stefanha@gmail.com>, Rust-VMM Mailing List <rust-vmm@lists.opendev.org>, 
	Doug Goldstein <cardoe@cardoe.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	xen-devel <xen-devel@lists.xenproject.org>, Arnd Bergmann <arnd.bergmann@linaro.org>, 
	David Woodhouse <dwmw2@infradead.org>, 
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>, Rich Persaud <persaur@gmail.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, Paul Durrant <paul@xen.org>, 
	openxt <openxt@googlegroups.com>
Content-Type: multipart/alternative; boundary="000000000000b588ee05cd07ddb6"

--000000000000b588ee05cd07ddb6
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 27, 2021 at 3:06 AM Alex Benn=C3=A9e via Stratos-dev <
stratos-dev@op-lists.linaro.org> wrote:

>
> Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com> writes:
>
> > [[PGP Signed Part:Undecided]]
> > On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrote:
> >> Hi,
> >
> > Hi,
> >
> >> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
> >> =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80
> >>
> >>   Currently the foreign memory mapping support only works for dom0 due
> >>   to reference counting issues. If we are to support backends running =
in
> >>   their own domains this will need to get fixed.
> >>
> >>   Estimate: 8w
> >>
> >>
> >> [STR-57] <https://linaro.atlassian.net/browse/STR-57>
> >
> > I'm pretty sure it was discussed before, but I can't find relevant
> > (part of) thread right now: does your model assumes the backend (runnin=
g
> > outside of dom0) will gain ability to map (or access in other way)
> > _arbitrary_ memory page of a frontend domain? Or worse: any domain?
>
> The aim is for some DomU's to host backends for other DomU's instead of
> all backends being in Dom0. Those backend DomU's would have to be
> considered trusted because as you say the default memory model of VirtIO
> is to have full access to the frontend domains memory map.
>

I share Marek's concern. I believe that there are Xen-based systems that
will want to run guests using VirtIO devices without extending this level
of trust to the backend domains.


>
> > That is a significant regression in terms of security model Xen
> > provides. It would give the backend domain _a lot more_ control over th=
e
> > system that it normally has with Xen PV drivers - negating significant
> > part of security benefits of using driver domains.
>
> It's part of the continual trade off between security and speed. For
> things like block and network backends there is a penalty if data has to
> be bounce buffered before it ends up in the guest address space.
>

I think we have significant flexibility in being able to modify several
layers of the stack here to make this efficient, and it would be beneficial
to avoid bounce buffering if possible without sacrificing the ability to
enforce isolation. I wonder if there's a viable approach possible with some
implementation of a virtual IOMMU (which enforces access control) that
would allow a backend to commission I/O on a physical device on behalf of a
guest, where the data buffers do not need to be mapped into the backend and
so avoid the need for a bounce?


>
> > So, does the above require frontend agreeing (explicitly or implicitly)
> > for accessing specific pages by the backend? There were several
> > approaches to that discussed, including using grant tables (as PV
> > drivers do), vIOMMU(?), or even drastically different model with no
> > shared memory at all (Argo). Can you clarify which (if any) approach
> > your attempt of VirtIO on Xen will use?
>
> There are separate strands of work in Stratos looking at how we could
> further secure VirtIO for architectures with distributed backends (e.g.
> you may accept the block backend having access to the whole of memory
> but an i2c multiplexer has different performance characteristics).
>
> Currently the only thing we have prototyped is "fat virtqueues" which
> Arnd has been working on. Here the only actual shared memory required is
> the VirtIO config space and the relevant virt queues.
>

I think the "fat virtqueues" work is a positive path for investigation and
I don't think shared memory between front and backend is hard requirement
for those to function: a VirtIO-Argo transport driver would be able to
operate with them without shared memory.


>
> Other approaches have been discussed including using the virtio-iommu to
> selectively make areas available to the backend or use memory zoning so
> for example network buffers are only allocated in a certain region of
> guest physical memory that is shared with the backend.
>
> > A more general idea: can we collect info on various VirtIO on Xen
> > approaches (since there is more than one) in a single place, including:
> >  - key characteristics, differences
> >  - who is involved
> >  - status
> >  - links to relevant threads, maybe
> >
> > I'd propose to revive https://wiki.xenproject.org/wiki/Virtio_On_Xen


Thanks for the reminder, Marek -- I've just overhauled that page to give an
overview of the several approaches in the Xen community to enabling VirtIO
on Xen, and have included a first pass at including the content you
describe. I'm happy to be involved in improving it further.


>
>
> From the Stratos point of view Xen is a useful proving ground for
> general VirtIO experimentation due to being both a type-1 and open
> source. Our ultimate aim is have a high degree of code sharing for
> backends regardless of the hypervisor choice so a guest can use a VirtIO
> device model without having to be locked into KVM.
>

Thanks, Alex - this context is useful.


>
> If your technology choice is already fixed with a Xen hypervisor and
> portability isn't a concern you might well just stick to the existing
> well tested Xen PV interfaces.
>

I wouldn't quite agree; there are additional reasons beyond portability to
be looking at other options than the traditional Xen PV interfaces: eg. an
Argo-based interdomain transport for PV devices will enable fine-grained
enforcement of Mandatory Access Control over the frontend / backend
communication, and will not depend on XenStore which is advantageous for
Hyperlaunch / dom0less Xen deployment configurations.

thanks,

Christopher



>
> --
> Alex Benn=C3=A9e
> --
> Stratos-dev mailing list
> Stratos-dev@op-lists.linaro.org
> https://op-lists.linaro.org/mailman/listinfo/stratos-dev
>

--000000000000b588ee05cd07ddb6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Mon, Sep 27, 2021 at 3:06 AM Alex Benn=
=C3=A9e via Stratos-dev &lt;<a href=3D"mailto:stratos-dev@op-lists.linaro.o=
rg">stratos-dev@op-lists.linaro.org</a>&gt; wrote:<br></div><div class=3D"g=
mail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Marek Marczykowski-G=C3=B3recki &lt;<a href=3D"mailto:marmarek@invisiblethi=
ngslab.com" target=3D"_blank">marmarek@invisiblethingslab.com</a>&gt; write=
s:<br>
<br>
&gt; [[PGP Signed Part:Undecided]]<br>
&gt; On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrote:<br>
&gt;&gt; Hi,<br>
&gt;<br>
&gt; Hi,<br>
&gt;<br>
&gt;&gt; 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])<br=
>
&gt;&gt; =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80<br>
&gt;&gt; <br>
&gt;&gt;=C2=A0 =C2=A0Currently the foreign memory mapping support only work=
s for dom0 due<br>
&gt;&gt;=C2=A0 =C2=A0to reference counting issues. If we are to support bac=
kends running in<br>
&gt;&gt;=C2=A0 =C2=A0their own domains this will need to get fixed.<br>
&gt;&gt; <br>
&gt;&gt;=C2=A0 =C2=A0Estimate: 8w<br>
&gt;&gt; <br>
&gt;&gt; <br>
&gt;&gt; [STR-57] &lt;<a href=3D"https://linaro.atlassian.net/browse/STR-57=
" rel=3D"noreferrer" target=3D"_blank">https://linaro.atlassian.net/browse/=
STR-57</a>&gt;<br>
&gt;<br>
&gt; I&#39;m pretty sure it was discussed before, but I can&#39;t find rele=
vant<br>
&gt; (part of) thread right now: does your model assumes the backend (runni=
ng<br>
&gt; outside of dom0) will gain ability to map (or access in other way)<br>
&gt; _arbitrary_ memory page of a frontend domain? Or worse: any domain?<br=
>
<br>
The aim is for some DomU&#39;s to host backends for other DomU&#39;s instea=
d of<br>
all backends being in Dom0. Those backend DomU&#39;s would have to be<br>
considered trusted because as you say the default memory model of VirtIO<br=
>
is to have full access to the frontend domains memory map.<br></blockquote>=
<div><br></div><div>I share Marek&#39;s concern. I believe that there are X=
en-based systems that will want to run guests using VirtIO devices without =
extending this level of trust to=C2=A0the backend domains.</div><div>=C2=A0=
<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8=
ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
&gt; That is a significant regression in terms of security model Xen<br>
&gt; provides. It would give the backend domain _a lot more_ control over t=
he<br>
&gt; system that it normally has with Xen PV drivers - negating significant=
<br>
&gt; part of security benefits of using driver domains.<br>
<br>
It&#39;s part of the continual trade off between security and speed. For<br=
>
things like block and network backends there is a penalty if data has to<br=
>
be bounce buffered before it ends up in the guest address space.<br></block=
quote><div><br></div><div>I think we have significant flexibility in being =
able to modify several layers of the stack here to make this efficient, and=
 it would be beneficial to avoid bounce buffering if possible without sacri=
ficing the ability to enforce isolation. I wonder if there&#39;s a viable a=
pproach possible with some implementation of a virtual IOMMU (which enforce=
s access control) that would allow a backend to commission I/O on a physica=
l device on behalf of a guest, where the data buffers do not need to be map=
ped into the backend and so avoid the need for a bounce?</div><div>=C2=A0<b=
r></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
&gt; So, does the above require frontend agreeing (explicitly or implicitly=
)<br>
&gt; for accessing specific pages by the backend? There were several<br>
&gt; approaches to that discussed, including using grant tables (as PV<br>
&gt; drivers do), vIOMMU(?), or even drastically different model with no<br=
>
&gt; shared memory at all (Argo). Can you clarify which (if any) approach<b=
r>
&gt; your attempt of VirtIO on Xen will use?<br>
<br>
There are separate strands of work in Stratos looking at how we could<br>
further secure VirtIO for architectures with distributed backends (e.g.<br>
you may accept the block backend having access to the whole of memory<br>
but an i2c multiplexer has different performance characteristics).<br>
<br>
Currently the only thing we have prototyped is &quot;fat virtqueues&quot; w=
hich<br>
Arnd has been working on. Here the only actual shared memory required is<br=
>
the VirtIO config space and the relevant virt queues.<br></blockquote><div>=
<br></div><div>I think the &quot;fat virtqueues&quot; work is a positive pa=
th for investigation and I don&#39;t think shared memory between front and =
backend is hard requirement for those to function: a VirtIO-Argo transport =
driver would be able to operate with them without shared memory.</div><div>=
=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Other approaches have been discussed including using the virtio-iommu to<br=
>
selectively make areas available to the backend or use memory zoning so<br>
for example network buffers are only allocated in a certain region of<br>
guest physical memory that is shared with the backend.<br>
<br>
&gt; A more general idea: can we collect info on various VirtIO on Xen<br>
&gt; approaches (since there is more than one) in a single place, including=
:<br>
&gt;=C2=A0 - key characteristics, differences<br>
&gt;=C2=A0 - who is involved<br>
&gt;=C2=A0 - status<br>
&gt;=C2=A0 - links to relevant threads, maybe<br>
&gt;<br>
&gt; I&#39;d propose to revive <a href=3D"https://wiki.xenproject.org/wiki/=
Virtio_On_Xen" rel=3D"noreferrer" target=3D"_blank">https://wiki.xenproject=
.org/wiki/Virtio_On_Xen</a></blockquote><div><br></div><div>Thanks for the =
reminder, Marek -- I&#39;ve just overhauled that page to give an overview o=
f the several approaches in the Xen community to enabling VirtIO on Xen, an=
d have included a first pass at including the content you describe. I&#39;m=
 happy to be involved in improving it further.</div><div>=C2=A0</div><block=
quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1=
px solid rgb(204,204,204);padding-left:1ex"><br>
<br>
>From the Stratos point of view Xen is a useful proving ground for<br>
general VirtIO experimentation due to being both a type-1 and open<br>
source. Our ultimate aim is have a high degree of code sharing for<br>
backends regardless of the hypervisor choice so a guest can use a VirtIO<br=
>
device model without having to be locked into KVM.<br></blockquote><div><br=
></div><div>Thanks, Alex - this context is useful.</div><div>=C2=A0</div><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft:1px solid rgb(204,204,204);padding-left:1ex">
<br>
If your technology choice is already fixed with a Xen hypervisor and<br>
portability isn&#39;t a concern you might well just stick to the existing<b=
r>
well tested Xen PV interfaces.<br></blockquote><div><br></div><div>I wouldn=
&#39;t quite agree; there are additional reasons beyond portability to be l=
ooking at other options than the traditional Xen PV interfaces: eg. an Argo=
-based interdomain transport for PV devices will enable fine-grained enforc=
ement of Mandatory Access Control over the frontend / backend communication=
, and will not depend on XenStore which is advantageous for Hyperlaunch / d=
om0less Xen deployment configurations.</div><div><br></div><div>thanks,</di=
v><div><br></div><div>Christopher</div><div><br></div><div>=C2=A0</div><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-- <br>
Alex Benn=C3=A9e<br>
-- <br>
Stratos-dev mailing list<br>
<a href=3D"mailto:Stratos-dev@op-lists.linaro.org" target=3D"_blank">Strato=
s-dev@op-lists.linaro.org</a><br>
<a href=3D"https://op-lists.linaro.org/mailman/listinfo/stratos-dev" rel=3D=
"noreferrer" target=3D"_blank">https://op-lists.linaro.org/mailman/listinfo=
/stratos-dev</a><br>
</blockquote></div></div>

--000000000000b588ee05cd07ddb6--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:56:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:56:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197514.350598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66U-0003XL-7l; Tue, 28 Sep 2021 05:56:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197514.350598; Tue, 28 Sep 2021 05:56:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66U-0003XE-4G; Tue, 28 Sep 2021 05:56:34 +0000
Received: by outflank-mailman (input) for mailman id 197514;
 Tue, 28 Sep 2021 05:56:32 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV66S-0003Ws-SR
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:56:32 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d4870fac-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:56:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 7095761262;
 Tue, 28 Sep 2021 05:56:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4870fac-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808591;
	bh=/vMdnTGIRNxo4mGYHll6b4s9a8WgufEfVbRp/DlK4b8=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=Z9EwQalNw0xG35eHzPyfGcQEJ3wCFvuNTOXo9MfXW8jiLehmG/WKcj52dkulktQpL
	 1dl5PgCBvzxVzlOhaHsKGRsjEjMuYKqa5J89bfT4VdlUBswhJFSiHFuSDgUZABgK2/
	 ImKldDTbfR54KLzEnTYkgc4SCeTLFQfD7DpT5vMGm4dqjCeHzqJpKa1f/PtSa9FewL
	 c805GDmDAzWGWyMSF5k/Tq3ABdd20hwbJ2DP0VHpuc4lIRw///8TrjfhhZKBlx7pSV
	 avKKm1x8MFGF9R7trm+qs4P6GRA8dCM2kozPqUBVSUEpLJcCMQjkicgnpn2pCVTP34
	 3zPIGvtZ390nQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	boris.ostrovsky@oracle.com,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.14 28/40] Xen/gntdev: don't ignore kernel unmapping error
Date: Tue, 28 Sep 2021 01:55:12 -0400
Message-Id: <20210928055524.172051-28-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055524.172051-1-sashal@kernel.org>
References: <20210928055524.172051-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit f28347cc66395e96712f5c2db0a302ee75bafce6 ]

While working on XSA-361 and its follow-ups, I failed to spot another
place where the kernel mapping part of an operation was not treated the
same as the user space part. Detect and propagate errors and add a 2nd
pr_debug().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/c2513395-74dc-aea3-9192-fd265aa44e35@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/xen/gntdev.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index a3e7be96527d..23fd09a9bbaf 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -396,6 +396,14 @@ static int __unmap_grant_pages(struct gntdev_grant_map *map, int offset,
 			map->unmap_ops[offset+i].handle,
 			map->unmap_ops[offset+i].status);
 		map->unmap_ops[offset+i].handle = INVALID_GRANT_HANDLE;
+		if (use_ptemod) {
+			if (map->kunmap_ops[offset+i].status)
+				err = -EINVAL;
+			pr_debug("kunmap handle=%u st=%d\n",
+				 map->kunmap_ops[offset+i].handle,
+				 map->kunmap_ops[offset+i].status);
+			map->kunmap_ops[offset+i].handle = INVALID_GRANT_HANDLE;
+		}
 	}
 	return err;
 }
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:56:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:56:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197515.350603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66U-0003aV-Hz; Tue, 28 Sep 2021 05:56:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197515.350603; Tue, 28 Sep 2021 05:56:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66U-0003ZA-Ce; Tue, 28 Sep 2021 05:56:34 +0000
Received: by outflank-mailman (input) for mailman id 197515;
 Tue, 28 Sep 2021 05:56:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV66T-0003X0-4u
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:56:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fbe3536b-f3d9-4dbd-bada-680816328013;
 Tue, 28 Sep 2021 05:56:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id CD9F46124B;
 Tue, 28 Sep 2021 05:56:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbe3536b-f3d9-4dbd-bada-680816328013
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808591;
	bh=yF8soBrMZR/O7Es8zvvSXK1ACmyYAeIPeI9GI8SIN4E=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=BR9Mz7tNJIpOMx5qLmLL20V6J+CCbJqnzz19y6p25N+bWH6doUl3rEufMzt3szdUr
	 MMpPdsax7GWcFfEBdnMBV4XDW877gtglxayrFvk3zQ0Znp0y7jjIMIbg1zqCrb3iPG
	 JTI0rh8X7/Ape0kSmKx0fT6brfXHpwiQ/hx8wHHcMa+yC4aabMKJLJRtWZ8fkwtE2X
	 spJM3WqZzgsGzNF1F2W5g2pR6rMj0pPPhnsolXYKtitgWQaSyEfnfgP9BkdEGM5UGl
	 l65tTf6tnqeBbJau43qA+ewWkEEv5+uXGFfuTrQxqipWlRFuDfkmQ8V7LxgIFUq9e8
	 jLfUbw/orNoEw==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	boris.ostrovsky@oracle.com,
	xen-devel@lists.xenproject.org,
	iommu@lists.linux-foundation.org
Subject: [PATCH AUTOSEL 5.14 29/40] swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests
Date: Tue, 28 Sep 2021 01:55:13 -0400
Message-Id: <20210928055524.172051-29-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055524.172051-1-sashal@kernel.org>
References: <20210928055524.172051-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 9074c79b62b6e0d91d7f716c6e4e9968eaf9e043 ]

While the hypervisor hasn't been enforcing this, we would still better
avoid issuing requests with GFNs not aligned to the requested order.
Instead of altering the value also in the call to panic(), drop it
there for being static and hence easy to determine without being part
of the panic message.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/7b3998e3-1233-4e5a-89ec-d740e77eb166@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/xen/swiotlb-xen.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index dbb18dc956f3..de4f55154d49 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -232,10 +232,11 @@ void __init xen_swiotlb_init_early(void)
 	/*
 	 * Get IO TLB memory from any location.
 	 */
-	start = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
+	start = memblock_alloc(PAGE_ALIGN(bytes),
+			       IO_TLB_SEGSIZE << IO_TLB_SHIFT);
 	if (!start)
-		panic("%s: Failed to allocate %lu bytes align=0x%lx\n",
-		      __func__, PAGE_ALIGN(bytes), PAGE_SIZE);
+		panic("%s: Failed to allocate %lu bytes\n",
+		      __func__, PAGE_ALIGN(bytes));
 
 	/*
 	 * And replace that memory with pages under 4GB.
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197522.350620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66t-0004fY-Tz; Tue, 28 Sep 2021 05:56:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197522.350620; Tue, 28 Sep 2021 05:56:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV66t-0004fR-R2; Tue, 28 Sep 2021 05:56:59 +0000
Received: by outflank-mailman (input) for mailman id 197522;
 Tue, 28 Sep 2021 05:56:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV66s-00049R-65
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:56:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d277f21d-9a22-42aa-8a42-02bb9381d72a;
 Tue, 28 Sep 2021 05:56:49 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A6D7661361;
 Tue, 28 Sep 2021 05:56:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d277f21d-9a22-42aa-8a42-02bb9381d72a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808608;
	bh=aXvnqcLO8Ou+S5mb1OFq4rMhDipi4tUId/TV8UmR9yE=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=fsNv/pA8GFyTZIWBUET7BM7qqwasXrq5k5G9x5Ql3j+dU69C0NoChAFdcoT/pqPFv
	 ik8KCqGPpUVEOnsZhzuhw0Aq5TTEyadEKqlAHZJVhuTv1pB3JtTmwVoLn3QVugi+4/
	 8SJCdhIdrnGBGSn5gtujSwSLZhk3FQNAhX2LzUK5OkNCHNvKjYhOC6aVE2OeXv3TsK
	 GqZplc26ZunwBVcM1FLCq9RbhLJ7Ihpqq6bXuHiKxdzYR43/GO/doQXn453crXorz2
	 PvIKYOftco2TuKzNC6ipoiYXU96yXZ/mKqOavJ1zlb4Ilm1XQywJP0vRkGxcKh4Yjn
	 EQy0vfT4Lb/Ag==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 5.10 08/23] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:56:29 -0400
Message-Id: <20210928055645.172544-8-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055645.172544-1-sashal@kernel.org>
References: <20210928055645.172544-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 986b56970961..b0cbc7fead74 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -499,7 +499,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197526.350631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV672-00054W-6S; Tue, 28 Sep 2021 05:57:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197526.350631; Tue, 28 Sep 2021 05:57:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV672-00054P-2u; Tue, 28 Sep 2021 05:57:08 +0000
Received: by outflank-mailman (input) for mailman id 197526;
 Tue, 28 Sep 2021 05:57:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV671-00053m-JG
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:57:07 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e935a92c-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:57:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2310461360;
 Tue, 28 Sep 2021 05:57:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e935a92c-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808626;
	bh=42D796lJ3lX0gb34lIi7EqCTnnN659dJBHy0opAhq9U=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=g2+VEN+olz3jKRAA9EL81jxoqZEUMlolyneyFXNtWxRLul/95x/+0/yZiS54JfByc
	 WkFLskh3BJl3EYAl0fHCzZchCBaZnx9abu0CBKcKLuy/oC6dtY9GYJyOmDhgw91Ozg
	 ZAws6PwLtwCOHTEVCqZoO+Ncbc6NnSiZZDM+rPG7lEhZiVvpKK8gcUmuPKLmGtH8Cv
	 PMmqvklXMCwQv6JWJ7kGEuvM21SKWLR8mAmgepWj/+eWHFjUQL4gOrCNMeKEYJXGEx
	 rPNssq9Cop+cNJbOw2SLdwn9Xcwl+1fZh4Qu+eg2vPVRfwcEtWZ6Z1SYbX3nxgHy0s
	 GH0AEH0AL0xBw==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 5.4 02/11] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:56:55 -0400
Message-Id: <20210928055704.172814-2-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055704.172814-1-sashal@kernel.org>
References: <20210928055704.172814-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index c213f2b81269..995566a2785f 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -492,7 +492,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197532.350642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67D-0005nF-Fm; Tue, 28 Sep 2021 05:57:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197532.350642; Tue, 28 Sep 2021 05:57:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67D-0005n4-C8; Tue, 28 Sep 2021 05:57:19 +0000
Received: by outflank-mailman (input) for mailman id 197532;
 Tue, 28 Sep 2021 05:57:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV67C-0005mF-RN
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:57:18 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id efe258ce-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:57:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 560526138D;
 Tue, 28 Sep 2021 05:57:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efe258ce-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808637;
	bh=9jFq3Ks7GD1NwvIEYZLWUZSvKCUXcz3duDRTepIsCuE=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=gi9rnGArhb9P7ssHmbG2DftbHnA4RZLG90lbdmxq14mABntN1E4f212OhBq8YlKyO
	 r+wGxsrWu2WSE0lgttjOvnlWzWvIChc1Ft6kYznuoZNMWft/0FPkFt56JXKxOAUfuj
	 Z1CF/ygDvJ0HTYYULDz0Sv7wH4HJBWY0yaAxlkIA+6I987kH16LTr8PVB09NJVr4Xd
	 Bo3Mevu03lPb5RAT67832/1KHdyFW4n+UIZSCSNLe+AOZsRxCD6FQZIxBfu45O0wPn
	 keM3asrGvdZpNsR9SowFfJpV7PpRCP9UuZhFB9d9p2GItZaOQNo/BjfA4nb3aUJDJX
	 xPlMmPz/w/brQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 4.19 02/10] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:57:08 -0400
Message-Id: <20210928055716.172951-2-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055716.172951-1-sashal@kernel.org>
References: <20210928055716.172951-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 41bdfb684d46..4d0d5501ca56 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -492,7 +492,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197534.350653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67O-0006Kv-O5; Tue, 28 Sep 2021 05:57:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197534.350653; Tue, 28 Sep 2021 05:57:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67O-0006Ki-Kf; Tue, 28 Sep 2021 05:57:30 +0000
Received: by outflank-mailman (input) for mailman id 197534;
 Tue, 28 Sep 2021 05:57:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV67N-0006J8-Rv
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:57:29 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f677cb10-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:57:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 75145613A8;
 Tue, 28 Sep 2021 05:57:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f677cb10-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808648;
	bh=Y4XjyZvXYrLEKqhTcaGzcA+k3/KH2wAv3Jyuw9cGryI=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=EianYCz/trToiQ0TXG2H1NnHFQZTY1kgJilWnwCT3Z4LFArFBpurBeWfylHHn6rPa
	 pGvhqI+2F3gcEn7sV0ShN2bhRdDDM8MgVS5jtM3rl2CZ0J3sS2atKm4kCoe7SXOVQi
	 6rBESfxDeUxwYzFfqyDZhTiqaBgYonjgoIZom4CeNdBY31c6hjAoAoGglOjsbB6IEk
	 3xYLt9xaUEPU1V69+IQxsv+3cr0dN78lQmn+OpK5Gnr28Fx5G5bNIMMKr6YRWwUwk9
	 1ausbTrBXiCbGO4cuGDQGsXkanvhD2jEQxzQO8qMtMcFmnCnkWIAuYkA2GoMF6VZEc
	 Z2EFGwUtJA76A==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 4.14 2/8] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:57:20 -0400
Message-Id: <20210928055727.173078-2-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055727.173078-1-sashal@kernel.org>
References: <20210928055727.173078-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index ad555a9a3eca..e1d6dbb4b770 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -492,7 +492,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197536.350664 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67W-0006q0-1P; Tue, 28 Sep 2021 05:57:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197536.350664; Tue, 28 Sep 2021 05:57:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67V-0006pr-T4; Tue, 28 Sep 2021 05:57:37 +0000
Received: by outflank-mailman (input) for mailman id 197536;
 Tue, 28 Sep 2021 05:57:37 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV67U-0006nF-V5
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:57:36 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fab01af2-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:57:36 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8A7E8613CD;
 Tue, 28 Sep 2021 05:57:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fab01af2-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808655;
	bh=2LruOHFY9TUIe84RMFTD1y3uv7jB33twD9v32n68Fu0=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=CrtuqrkKgT/8t4bDjMX6aa5aFU9XifZlLakbbu/mh8vkQWG0S629duKIwqFlT4plt
	 00BU6GYVmZWjUxiKTEYbawg+IQ1IEEy73F3EtRkCJsvcY4R6Nhzel1m4FIhESMohwc
	 DUPGms7Mgl4QoCw3zcZHLYxBGVaya2k3Pzh7wqc6Sw6WJ6BI9CTRq6+Tse3a6/Fwmf
	 0XY0WgsDgL7vcXvrmb2JW/MxeBLb/Km0EisE1a2MMOHi2xG4eti0UNAlGS47DdQH3z
	 PPuU9dg+wdKBoL603JHrcCtydhw9BfxEwEKsVqw1GDmFNI082HlkpS/BsSlHmWg0vM
	 bEs46EIWX6qNg==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 4.9 2/6] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:57:30 -0400
Message-Id: <20210928055734.173182-2-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210928055734.173182-1-sashal@kernel.org>
References: <20210928055734.173182-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index f7fd8b5a6a8c..3016869b4afd 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -492,7 +492,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 05:57:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 05:57:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197538.350675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67e-0007Mj-8f; Tue, 28 Sep 2021 05:57:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197538.350675; Tue, 28 Sep 2021 05:57:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV67e-0007Ma-5Y; Tue, 28 Sep 2021 05:57:46 +0000
Received: by outflank-mailman (input) for mailman id 197538;
 Tue, 28 Sep 2021 05:57:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqPr=OS=kernel.org=sashal@srs-us1.protection.inumbo.net>)
 id 1mV67c-0006nF-QX
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 05:57:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fe8a8cca-2020-11ec-bc6a-12813bfff9fa;
 Tue, 28 Sep 2021 05:57:42 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 06348613BD;
 Tue, 28 Sep 2021 05:57:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe8a8cca-2020-11ec-bc6a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632808662;
	bh=7/jeTgXBCZ19Re/PHtFD00k6FbJyqk+hhvotdk4Nf/0=;
	h=From:To:Cc:Subject:Date:From;
	b=SFJq/caJsa5hrdWd5cmAp19He1QCeoCqCaZoSgSNjhPuDlT+muqkxUu0XCS7HigHl
	 1oqmOfDbdBvCpHS/6px5q1hCKW4RKcBd0w+DN6y6jEJjUOE2uoIMsmavbbcMswpbUI
	 nq0Orl9MyrexdF3+uT5P5+1SWUu5X0sYANcHZYBamoWAV/gGNnbFBJuDxr7NbcJKwI
	 81SHp5gid5yK9NMX5VixotfPO7p1S7jI49b2f12HAxaGBYVLNvtZvT1TQOMwcHXrKX
	 KRKNzvxVoyh8WZTKH86TWp8bHsYTLlecS56wOTzKeuqrnnQghu84HIMtZYDRMUD262
	 3+RmFlh8VRgzg==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>,
	wei.liu@kernel.org,
	kuba@kernel.org,
	xen-devel@lists.xenproject.org,
	netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 4.4 1/5] xen-netback: correct success/error reporting for the SKB-with-fraglist case
Date: Tue, 28 Sep 2021 01:57:37 -0400
Message-Id: <20210928055741.173265-1-sashal@kernel.org>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
Content-Transfer-Encoding: 8bit

From: Jan Beulich <jbeulich@suse.com>

[ Upstream commit 3ede7f84c7c21f93c5eac611d60eba3f2c765e0f ]

When re-entering the main loop of xenvif_tx_check_gop() a 2nd time, the
special considerations for the head of the SKB no longer apply. Don't
mistakenly report ERROR to the frontend for the first entry in the list,
even if - from all I can tell - this shouldn't matter much as the overall
transmit will need to be considered failed anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/xen-netback/netback.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index c8c6afc0ab51..15c73ebe5efc 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -994,7 +994,7 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue,
 				 * the header's copy failed, and they are
 				 * sharing a slot, send an error
 				 */
-				if (i == 0 && sharedslot)
+				if (i == 0 && !first_shinfo && sharedslot)
 					xenvif_idx_release(queue, pending_idx,
 							   XEN_NETIF_RSP_ERROR);
 				else
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:26:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:26:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197565.350686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6Z9-00037P-Ib; Tue, 28 Sep 2021 06:26:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197565.350686; Tue, 28 Sep 2021 06:26:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6Z9-00037I-FK; Tue, 28 Sep 2021 06:26:11 +0000
Received: by outflank-mailman (input) for mailman id 197565;
 Tue, 28 Sep 2021 06:26:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV6Z8-00037C-Af
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:26:10 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f7466d2c-2024-11ec-bc6b-12813bfff9fa;
 Tue, 28 Sep 2021 06:26:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9FBF661159;
 Tue, 28 Sep 2021 06:26:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7466d2c-2024-11ec-bc6b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632810367;
	bh=hICJfTW5J5FttlptTDqGDMRdNurcagj721E0LCeA6/s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=K0izqKJML/d2ULbpUef8tfKchkBHhKRaTUYFM7pzpy6qGvta+ugrRnB10rtF3j04h
	 Dvr7OmQLV/xw3F7ZTQmA+LnnDcojIgwkUfSEOZq087XGbpNeAYVx/he0Dlww8DpHqB
	 Hy6QYnCYNpeoxDBox6elRysTeIXR282+LTOxLQwHUHKa0fu1FtKeiLpBODnjYYSfn3
	 cOyQINBIXrzZ+JUD/vjVSTDy6mV3mp62L0ze/itUtt0gGW6vVKvMqXgfYFemxF7/BT
	 zx95GpDF/EuXCmwcnFWe34FMeO6o0y1dslE15zSlkIRdcYLeWehfcSzkMN5D4Ev7ZM
	 PTeaHwrMisn1A==
Date: Mon, 27 Sep 2021 23:26:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Christopher Clark <christopher.w.clark@gmail.com>
cc: =?UTF-8?Q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>, 
    =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    Oleksandr Tyshchenko <olekstysh@gmail.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Sergio Lopez <slp@redhat.com>, 
    Wei Liu <wl@xen.org>, Stefan Hajnoczi <stefanha@gmail.com>, 
    Rust-VMM Mailing List <rust-vmm@lists.opendev.org>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    xen-devel <xen-devel@lists.xenproject.org>, 
    Arnd Bergmann <arnd.bergmann@linaro.org>, 
    David Woodhouse <dwmw2@infradead.org>, 
    Stratos Mailing List <stratos-dev@op-lists.linaro.org>, 
    Rich Persaud <persaur@gmail.com>, 
    Daniel Smith <dpsmith@apertussolutions.com>, Paul Durrant <paul@xen.org>, 
    openxt <openxt@googlegroups.com>
Subject: Re: [Stratos-dev] Xen Rust VirtIO demos work breakdown for Project
 Stratos
In-Reply-To: <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.2109272323160.5022@sstabellini-ThinkPad-T480s>
References: <87pmsylywy.fsf@linaro.org> <YU5mW396S04IsCBr@mail-itl> <874ka68h96.fsf@linaro.org> <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1811458571-1632810367=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1811458571-1632810367=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Mon, 27 Sep 2021, Christopher Clark wrote:
> On Mon, Sep 27, 2021 at 3:06 AM Alex Bennée via Stratos-dev <stratos-dev@op-lists.linaro.org> wrote:
> 
>       Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> writes:
> 
>       > [[PGP Signed Part:Undecided]]
>       > On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Bennée wrote:
>       >> Hi,
>       >
>       > Hi,
>       >
>       >> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57])
>       >> ───────────────────────────────────────────────────────────────
>       >>
>       >>   Currently the foreign memory mapping support only works for dom0 due
>       >>   to reference counting issues. If we are to support backends running in
>       >>   their own domains this will need to get fixed.
>       >>
>       >>   Estimate: 8w
>       >>
>       >>
>       >> [STR-57] <https://linaro.atlassian.net/browse/STR-57>
>       >
>       > I'm pretty sure it was discussed before, but I can't find relevant
>       > (part of) thread right now: does your model assumes the backend (running
>       > outside of dom0) will gain ability to map (or access in other way)
>       > _arbitrary_ memory page of a frontend domain? Or worse: any domain?
> 
>       The aim is for some DomU's to host backends for other DomU's instead of
>       all backends being in Dom0. Those backend DomU's would have to be
>       considered trusted because as you say the default memory model of VirtIO
>       is to have full access to the frontend domains memory map.
> 
> 
> I share Marek's concern. I believe that there are Xen-based systems that will want to run guests using VirtIO devices without extending
> this level of trust to the backend domains.

>From a safety perspective, it would be challenging to deploy a system
with privileged backends. From a safety perspective, it would be a lot
easier if the backend were unprivileged.

This is one of those times where safety and security requirements are
actually aligned.
--8323329-1811458571-1632810367=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:29:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197574.350697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6bw-0003yD-4v; Tue, 28 Sep 2021 06:29:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197574.350697; Tue, 28 Sep 2021 06:29:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6bw-0003y6-1k; Tue, 28 Sep 2021 06:29:04 +0000
Received: by outflank-mailman (input) for mailman id 197574;
 Tue, 28 Sep 2021 06:29:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV6bu-0003xy-LD
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:29:02 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 5e7f1ebd-2025-11ec-bc6b-12813bfff9fa;
 Tue, 28 Sep 2021 06:29:01 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 079E2D6E;
 Mon, 27 Sep 2021 23:29:01 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B61D23F718;
 Mon, 27 Sep 2021 23:28:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e7f1ebd-2025-11ec-bc6b-12813bfff9fa
Subject: Re: [PATCH V3 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Oleksandr Tyshchenko <olekstysh@gmail.com>, xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
 <1632437334-12015-2-git-send-email-olekstysh@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <27840222-d5be-3df6-88bf-fb41c978791e@arm.com>
Date: Tue, 28 Sep 2021 08:28:47 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <1632437334-12015-2-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 24.09.2021 00:48, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> We need to pass info about maximum supported guest address
> space size to the toolstack on Arm in order to properly
> calculate the base and size of the extended region (safe range)
> for the guest. The extended region is unused address space which
> could be safely used by domain for foreign/grant mappings on Arm.
> The extended region itself will be handled by the subsequents
> patch.
> 
> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
> hap_paddr_bits.
> 
> As we change the size of structure bump the interface version.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> Please note, that review comments for the RFC version [1] haven't been addressed yet.
> It is not forgotten, some clarification is needed. It will be addressed for the next version.
> 
> [1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/
> 
> Changes RFC -> V2:
>    - update patch subject/description
>    - replace arch-specific sub-struct with common gpaddr_bits
>      field and update code to reflect that
> 
> Changes V2 -> V3:
>    - make the field uint8_t and add uint8_t pad[7] after
>    - remove leading blanks in libxl.h
> ---
>  tools/include/libxl.h            | 7 +++++++
>  tools/libs/light/libxl.c         | 2 ++
>  tools/libs/light/libxl_types.idl | 2 ++
>  xen/arch/arm/sysctl.c            | 2 ++
>  xen/arch/x86/sysctl.c            | 2 ++
>  xen/include/public/sysctl.h      | 4 +++-
>  6 files changed, 18 insertions(+), 1 deletion(-)
> 

Don't you want to print gpaddr_bits field of xen_sysctl_physinfo from output_physinfo (xl_info.c)?

Apart from that:
Reviewed-by: Michal Orzel <michal.orzel@arm.com>

Cheers


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:30:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:30:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197580.350708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6dU-0005Ho-G8; Tue, 28 Sep 2021 06:30:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197580.350708; Tue, 28 Sep 2021 06:30:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6dU-0005Hh-Cw; Tue, 28 Sep 2021 06:30:40 +0000
Received: by outflank-mailman (input) for mailman id 197580;
 Tue, 28 Sep 2021 06:30:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WhrI=OS=gmail.com=stefanha@srs-us1.protection.inumbo.net>)
 id 1mV6dS-0005HZ-R4
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:30:38 +0000
Received: from mail-pl1-x634.google.com (unknown [2607:f8b0:4864:20::634])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e744136-7d11-4867-a4e8-d0c934905db7;
 Tue, 28 Sep 2021 06:30:38 +0000 (UTC)
Received: by mail-pl1-x634.google.com with SMTP id j15so12035165plh.7
 for <xen-devel@lists.xenproject.org>; Mon, 27 Sep 2021 23:30:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e744136-7d11-4867-a4e8-d0c934905db7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc:content-transfer-encoding;
        bh=7paCFUAiHOVANRBvVJ8Dzn9prsSI+D+TV8goUlnl8H0=;
        b=cqmEXjJ/pN29AERRFrZmGjvummNeZ2WhXHeKAcWY2J19ZE+1YeSPmMosWZ6/e83Pna
         p/SirQxGH1imDUljaWzxlNR1+ULAluNEJ6TRA/+vEvLBV66NnyzOvjfaElD0P49Bcg8h
         csoy0y5zArd1+WWpX/ScKA5ZpYjVrFuTVnOCL32vHKMeMkkmGIkVYJZAVCx8urQ7x90/
         dSrGjoZAkbWSIzojIFqwdqauHqbNLft4sKWBE0eeZUso0P63OfGejQLlL+9+x++qruYC
         1qBgXEX0QrswomEIX95j6xWdmUUyDQRHp163ti376VGPQ8kUzV36z3SB04kPNgt9N20t
         lSzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc:content-transfer-encoding;
        bh=7paCFUAiHOVANRBvVJ8Dzn9prsSI+D+TV8goUlnl8H0=;
        b=j7qmHRubJoWQxOaEJjd4/S6MeLQ8Zg931KfAcuqTpgXWElvvy4HKomUu1uq6Xl5sil
         XhbWBsinj/1Ut2Lz3oSn2qwtryxxpDL2gf3xiBmdBnKBUb8Ua39P3X91+7kCMd45ouov
         b7rYjdlnRHHl1QoSY6+ltEu/bNbLPg1266IprHG8uIvfxSCW4aZhXru6tn3stJzYr1Ez
         Wod1vmKKdJvcVQkRHdwGiRNGDIfcofOSVkNZkmXrCzHISg8cAxLBeOqsQliGOHleETDJ
         64pEQWbADEivI9Vx11EAkrz19PXg/W+G79DKOx91l7SbsIMrH+UXlAqF/67ZnrzKiGLE
         DpBA==
X-Gm-Message-State: AOAM533jJHyMd/VKji2jGMG9Iq7cAgDICaGZvBPToMyhH5lqXMm2VC3J
	cH/rbhus5trDQjY65uM6qcqaJNdA26DAznnRx3k=
X-Google-Smtp-Source: ABdhPJyTsxEYyKeFE9bsfXTF/sDV2g/48iIyKGGYqy5xpl0rYR45zmt9OenEsDfV18jz5KNyTnitE3scwpJtxTU6tRE=
X-Received: by 2002:a17:902:dac2:b0:13d:1db4:dcc2 with SMTP id
 q2-20020a170902dac200b0013d1db4dcc2mr3537039plx.77.1632810637182; Mon, 27 Sep
 2021 23:30:37 -0700 (PDT)
MIME-Version: 1.0
References: <87pmsylywy.fsf@linaro.org> <YU5mW396S04IsCBr@mail-itl>
 <874ka68h96.fsf@linaro.org> <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
In-Reply-To: <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
From: Stefan Hajnoczi <stefanha@gmail.com>
Date: Tue, 28 Sep 2021 08:30:26 +0200
Message-ID: <CAJSP0QW-CqiUwzzoaoBHXs+xWObFbRduPvPeugjoCN2vb40Ohg@mail.gmail.com>
Subject: Re: [Stratos-dev] Xen Rust VirtIO demos work breakdown for Project Stratos
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: =?UTF-8?B?QWxleCBCZW5uw6ll?= <alex.bennee@linaro.org>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Oleksandr Tyshchenko <olekstysh@gmail.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Sergio Lopez <slp@redhat.com>, Wei Liu <wl@xen.org>, 
	Rust-VMM Mailing List <rust-vmm@lists.opendev.org>, Doug Goldstein <cardoe@cardoe.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel <xen-devel@lists.xenproject.org>, 
	Arnd Bergmann <arnd.bergmann@linaro.org>, David Woodhouse <dwmw2@infradead.org>, 
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>, Rich Persaud <persaur@gmail.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, Paul Durrant <paul@xen.org>, 
	openxt <openxt@googlegroups.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Sep 28, 2021 at 7:55 AM Christopher Clark
<christopher.w.clark@gmail.com> wrote:
>
> On Mon, Sep 27, 2021 at 3:06 AM Alex Benn=C3=A9e via Stratos-dev <stratos=
-dev@op-lists.linaro.org> wrote:
>>
>>
>> Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com> writes=
:
>>
>> > [[PGP Signed Part:Undecided]]
>> > On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrote:
>> > That is a significant regression in terms of security model Xen
>> > provides. It would give the backend domain _a lot more_ control over t=
he
>> > system that it normally has with Xen PV drivers - negating significant
>> > part of security benefits of using driver domains.
>>
>> It's part of the continual trade off between security and speed. For
>> things like block and network backends there is a penalty if data has to
>> be bounce buffered before it ends up in the guest address space.
>
>
> I think we have significant flexibility in being able to modify several l=
ayers of the stack here to make this efficient, and it would be beneficial =
to avoid bounce buffering if possible without sacrificing the ability to en=
force isolation. I wonder if there's a viable approach possible with some i=
mplementation of a virtual IOMMU (which enforces access control) that would=
 allow a backend to commission I/O on a physical device on behalf of a gues=
t, where the data buffers do not need to be mapped into the backend and so =
avoid the need for a bounce?

This may not require much modification for Linux guest drivers.
Although the VIRTIO drivers traditionally assumed devices can DMA to
any memory location, there are already constraints in other situations
like Confidential Computing, where swiotlb is used for bounce
buffering.

Stefan


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:42:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:42:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197586.350719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6ol-0006uq-Ea; Tue, 28 Sep 2021 06:42:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197586.350719; Tue, 28 Sep 2021 06:42:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV6ol-0006uj-BG; Tue, 28 Sep 2021 06:42:19 +0000
Received: by outflank-mailman (input) for mailman id 197586;
 Tue, 28 Sep 2021 06:42:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV6oj-0006ud-OG
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:42:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d6c14b39-427f-4b24-b7d8-90ba95a10584;
 Tue, 28 Sep 2021 06:42:16 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-37-0G3-M-bhOWGx1SuMmKlHkw-2; Tue, 28 Sep 2021 08:42:14 +0200
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com (2603:10a6:208:125::12)
 by AM0PR04MB6002.eurprd04.prod.outlook.com (2603:10a6:208:117::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 06:42:10 +0000
Received: from AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b]) by AM0PR04MB5587.eurprd04.prod.outlook.com
 ([fe80::ac:a0be:c26c:908b%5]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 06:42:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0132.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1a::24) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 06:42:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6c14b39-427f-4b24-b7d8-90ba95a10584
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632811335;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WeglWZkQBdtppFapSy9NHOVS1TqpLDgaSzswyOhmizE=;
	b=Q2wdHyVeihmIIWilxP7WN0APOrRPXA2zywdhSWaExr5NPiHpZwc4jJ4PNPXmqx9eVTQx/1
	akyQrn5kaROZV+QOF/q+weXWjNJxJDFaZEoY1GZEKMOuP1Epz03tKEDnduFFfT197rhunj
	LeIOmJruZuyhkI1Ker4hY1QBi9Jsh+E=
X-MC-Unique: 0G3-M-bhOWGx1SuMmKlHkw-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OjAQ8TJ3uF3uByxqwzLz1k74PsIP74OTeGDJQinA20QYh/ODMLSNJ5QQ3mUJI1YEtYjs82f6jhY/l8Y9Lgckqlu0ZWQ8rtdRneyf14P/aKWHh4DbihSUw1n3pFhFz1LwP/JJpAP7diXG6mv2RimeIlpK9H5u/p4BUxo4F+khRUuh/KCQqACRQHaCGrSL3cLCu3zULECdiCOfHEtckYjeLJ2BQvvPSSJE7Ynrg/G/nNNknjpUX83Wdn/sUXSNgu2u/nmzNlA9rHw1iDNfLEHSK4+T993jUNiGLSPPt3c8OUeYEl4A52Dty0pqeZyrgmMmHL/ipJIV7jvRWm/G1IC2tw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WeglWZkQBdtppFapSy9NHOVS1TqpLDgaSzswyOhmizE=;
 b=l1XLrldlZgIUf/fvwRFYv19b+Ky68FVGoka7kULHCYTe4b+8iTCDb1B8hVdmWHbLzQLDmHL3aS1T20Zk9L5bPxpwK/GVxKP0pXLz3ZqgfXPF2xMhLIa9RvhKtzXiBMF/aoHiViy1l9Xr2qD8lN1X5knMemgzN6xEeoKzM4ARaJl3zXpStRnkbnVCdZ0xfflZGTfbnMHVs0pL3rB4a6m/qoL2SlrtsMVALX/+fVZVXWuE3W6rHraN+MCJamnM3jnjVSpWFo4nyomh9z1vKRzj7333ZoNnS5+X04GKOgoplA33BI31pW8jRgUbMksrylJDvi7xMc64m2KT93EkwJ/qAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 Juergen Gross <jgross@suse.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
Date: Tue, 28 Sep 2021 08:42:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0132.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1a::24) To AM0PR04MB5587.eurprd04.prod.outlook.com
 (2603:10a6:208:125::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5b6bfc79-2216-4690-4c88-08d9824b187b
X-MS-TrafficTypeDiagnostic: AM0PR04MB6002:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR04MB60022226134ABA6734736320B3A89@AM0PR04MB6002.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KyJTBonqIdK5Nq7JX+IMMfZPEiBIwpam3DWEaEUy8EDd77lCZj/SVHLlt5DDFT4QGrEq3yL8sgVhNL/8KPtWBn5jaecL1NcRzVMc06IzFqxFY/Xkq0AFh9VPHHTBTaNez8LJgKwVrOqD7XULGh3nhNgWnUaDPyUoXICYBKxpLyUlRXZJDwGaQFrGfahS+LZBYk2gNPSiD24Oa6upCAgbz4CIOJeHmURma3/ljI9ghlqIZSi9OCk8I3DO7wfcYBES3sIaEkTfmroTYhV0ZJ7DvcY51SdUctNFxY0rEqB05gwu4FeJAIug2K7unNy3ScZ9sciwl8ZwtvNh0tuFJZqhcPyJpiPYEt445VwhNR7SIPx+nFbcSu5X4VRthU4K3INbTLsjt7mTazMh7aRE/RzCx9vIXRzY2XXdmKLob0xKxRHhq3nTzdAalnLYzLEnajVxtgYB5wYmFBJvwNzg4YOtaCQ/OnUUWu1MgHk2ZcGl4dVWe/cO3fvDT/yK7OVxq7bRQPpuAYnywoIcKLE7Hw/ZMejykaIGYIOrKIrdN9EjA0ELtMW7DD0PUvkK09ry0Ciwj4kEzDklQvLWZZo0mMif3cmRpPlgGxltYZak4HGsmD07PfSmvjYo4SksIPXsZlKOh2Wx/Yuba9xKh8O6P8wKvppvsR6itazLdQyuS4VOl3rfRqspq8b+t1lHrdUndS9Joqt4NSvj3545rCXkSp7izlleKqItTa8RTXEdlyYuuuU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5587.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(5660300002)(316002)(107886003)(8936002)(8676002)(83380400001)(6486002)(186003)(53546011)(4326008)(54906003)(31686004)(16576012)(36756003)(26005)(31696002)(6916009)(66476007)(508600001)(86362001)(66556008)(956004)(2906002)(66946007)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VkNKRUszbE5vUlRwZ0NMMFdzbTh0Sm9QS0NWM3djc2pwS2F3Tk51YnZ0aEdP?=
 =?utf-8?B?Y1BDbmdkK29GdFhYRHpvSmFwWTJVSm1zVmpRNldWdFZkUG5na2d1ZmgzRHdR?=
 =?utf-8?B?Tmp5Z2JES0xNWUVwbUlpWnZKenJ0cEQ3M2pMNHIrL09LS2ZwL1dHam0ydWFX?=
 =?utf-8?B?QTJHZE82V3BYdzl1MFpGK3EzY3kzc3FiWUlZbW15MmVxbkZNb2hmM1ZHOVpa?=
 =?utf-8?B?dTZSV3UxSDJ2M1pjczF2ejM1ZzZRVk5KTjFURU1WRW5jdGpHTHdVd3loRlZZ?=
 =?utf-8?B?U3RUNTlrNnZ5M1g0cnZGNFlLM29PUkVYVERrUEFQaVh2RUgzMmh2Rm9DQi9t?=
 =?utf-8?B?VUkwK3c2d1pWNVZJNEV0Mm5NK21lcFJvYXg4UnBZdW9vUUFPT28zc255OXV2?=
 =?utf-8?B?elJtV2xxNjBnRnhSaGNlQmVGRFdObHdKUWU4NHU3bDVocTM5bnJRblllcndI?=
 =?utf-8?B?V3FrYy9tcy9aR3RSWG5QS2J1eUNjS052S0JIQ3hqWFpxYlBQVnBZQVMxU1JK?=
 =?utf-8?B?d0VRZmJEL3J1bGt5QWc3anZBSHFNMmdQREg4ai8zeE9TZWVPQklBSUhxSXQz?=
 =?utf-8?B?YmpWSldvazAzSGdLaDI4ajVmUUxHU2FmQzlpT0h0UUlXWGFVeVhIRG5qZ3NZ?=
 =?utf-8?B?WWU1aXJ3QmJrT1JvUnlVV25uUjF3Y0tiMUV5cFIrZHMxZjdlekhzQXFpQ0FE?=
 =?utf-8?B?YW9PZlFwemx4S2tCWXRTa1MvQ2R1RFZrdlNYR2VQUW5pR0Vaa0w3UmVUYVgy?=
 =?utf-8?B?eER1MzYxTmptK2dncmR0bElSK2ZjUVpVb3ErWHA3MHJGbE5EdmhDOVgrWncw?=
 =?utf-8?B?c3NPTk11R1F2a3A5WU41eWVvcU1QUzNwL3FNL3ZRVExrbDd6NlpnRmM4ZWJ2?=
 =?utf-8?B?TkRzT2VoSjd5NXExRS91WWdxWk9hWXNhRjN3QTBWek5jamZXZ05xNFlsczR1?=
 =?utf-8?B?Q1hYY3c2UjdtOUZwUHNDcFRRVElBL2Vwem40c0pZVHJRV2FsNEFVMDFsVmtq?=
 =?utf-8?B?RG1RUEJjVlphT1hTU1ozcERkNlJheTVEUmNwd2xXako2ckN6MFFpNlNZdlhX?=
 =?utf-8?B?bWcxSE1DenNLS0NVQVBaVndzcHkwcXNyaCtpQmZmTERmems1SUZRZVBDWjA5?=
 =?utf-8?B?b2hpOHZkSVFlTWM5UUNBRlgyN0gzcnBoR21wQUNRRlMrUVJmUzNKeWFmOG1W?=
 =?utf-8?B?STN0RlpjSGJuNGs5a2ZrWjJjVWlYY1lWbUlVZzhoUnNSWnJ2S0pIVmM1Mldj?=
 =?utf-8?B?TFRCQUhVSGV6TmVweGExRVJySll5RnROMy8xK2VyZFNPRUpiSzJiaWJNN2p4?=
 =?utf-8?B?bHNmbHdsdGpMTzR1bUNQK2VidkVjSjNyNDduT253ZUExclowenNQdGhLNC91?=
 =?utf-8?B?N2dMNFQzOStMUnVlVFprOVdFc1IwMUtLVlJubzZQc3RzWHRIS3BWWjhGSm00?=
 =?utf-8?B?emczbWxWZW94Q3N0VW9XMzdiaXdKMXkydXVETXdHMGZ1clpTNEg4OFVleWpw?=
 =?utf-8?B?SlRnTjBnZWRIT0daVWEvN0ZuaUx1Uzh2MkRZNHhFMmdER3Zoek9EYmZkSi9T?=
 =?utf-8?B?TTVWZmQ3RVliS2hJRlAydGxIUlhLMU90SEQvQTlmcVNGRW5BTUh1OXNjQ2M1?=
 =?utf-8?B?ZU1aQlkzQVlaaXJIZVpvTWtkOVZkZzlGaW84dWxiQVNIMnQ2SEFpakpqY3A5?=
 =?utf-8?B?ZmREWkFicm9VZTBiRTJiRnhuZ2Q5YjExNWN5MWNkbnM4a0lGYWMyTXljbkY1?=
 =?utf-8?Q?nhGLmG+RDd9nwXvxv8O1gFTlDZ2IpMs10N3gWZV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5b6bfc79-2216-4690-4c88-08d9824b187b
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5587.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 06:42:10.2703
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aMudJ5KM71Sf+5MfV+5mF0+1ALQRHAkYY9mZOKS+1cDAXiSaLMlNYieBztoYS/9xmSTQ3WjlOUy2ZDMKROz9zA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6002

On 28.09.2021 06:18, Stefano Stabellini wrote:
> On Mon, 27 Sep 2021, Juergen Gross wrote:
>> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
>>>
>>> On 27.09.21 10:26, Jan Beulich wrote:
>>>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> Currently PCI backend implements multiple functionalities at a time.
>>>>> To name a few:
>>>>> 1. It is used as a database for assignable PCI devices, e.g. xl
>>>>>      pci-assignable-{add|remove|list} manipulates that list. So,
>>>>> whenever
>>>>>      the toolstack needs to know which PCI devices can be passed through
>>>>>      it reads that from the relevant sysfs entries of the pciback.
>>>>> 2. It is used to hold the unbound PCI devices list, e.g. when passing
>>>>>      through a PCI device it needs to be unbound from the relevant
>>>>> device
>>>>>      driver and bound to pciback (strictly speaking it is not required
>>>>>      that the device is bound to pciback, but pciback is again used as a
>>>>>      database of the passed through PCI devices, so we can re-bind the
>>>>>      devices back to their original drivers when guest domain shuts
>>>>> down)
>>>>> 3. Device reset for the devices being passed through
>>>>> 4. Para-virtualised use-cases support
>>>>>
>>>>> The para-virtualised part of the driver is not always needed as some
>>>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
>>>>> model for PCI device passthrough. For such use-cases make the very
>>>>> first step in splitting the xen-pciback driver into two parts: Xen
>>>>> PCI stub and PCI PV backend drivers.
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko
>>>>> <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> ---
>>>>> Changes since v3:
>>>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>>>> I'm afraid this wasn't fully done:
>>>>
>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>> @@ -1,5 +1,6 @@
>>>>>    # SPDX-License-Identifier: GPL-2.0
>>>>>    obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
>>>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
>>>> While benign when added here, this addition still doesn't seem to
>>>> belong here.
>>>
>>> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seems
>>>
>>> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build on 32-bit
>>>
>>> architectures...
>>>
>>> What would be the preference here? Stefano suggested that we still define
>>>
>>> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate to it
>>>
>>> in the second patch
>>>
>>> Another option is just to squash the two patches.
>>
>> Squashing would be fine for me.
>  
> It is fine for me to squash the two patches.
> 
> But in any case, wouldn't it be better to modify that specific change to:
> 
> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
> index e2cb376444a6..e23c758b85ae 100644
> --- a/drivers/xen/xen-pciback/Makefile
> +++ b/drivers/xen/xen-pciback/Makefile
> @@ -1,6 +1,5 @@
>  # SPDX-License-Identifier: GPL-2.0
> -obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> -obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> +obj-$(CONFIG_XEN_PCI_STUB) += xen-pciback.o

But that wouldn't allow the driver to be a module anymore, would it?

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:56:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:56:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197593.350730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV72V-00006l-NF; Tue, 28 Sep 2021 06:56:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197593.350730; Tue, 28 Sep 2021 06:56:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV72V-00006e-Jf; Tue, 28 Sep 2021 06:56:31 +0000
Received: by outflank-mailman (input) for mailman id 197593;
 Tue, 28 Sep 2021 06:56:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV72U-00006Y-Cz
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:56:30 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89de428a-c1e2-47fc-b24e-99d3e001c1c3;
 Tue, 28 Sep 2021 06:56:28 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S6nakS012178; 
 Tue, 28 Sep 2021 06:56:26 GMT
Received: from eur02-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbx44g12d-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 06:56:26 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7425.eurprd03.prod.outlook.com (2603:10a6:20b:260::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Tue, 28 Sep
 2021 06:56:22 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 06:56:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89de428a-c1e2-47fc-b24e-99d3e001c1c3
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LGU3d31d63MDmf+dx2k+DhE9FU6oXqV1NqO/+sNRrfeys3ifKmTcBEY89cQLJrDpLC7EPxM7kLLPaXajHH6Xgo/Qb9OzXwOiLs/cx3Q4t/LbZupCzG3O1WDG+GwQs/4ctTF50H04XWur3r1/EpLv9DNzlu9yuLc4Ml+S2GsphgwkHWDJFju4EXtbkQnntC+89buKirZDj+E5aLS+xwNZoj/ybWEuUqr6+B28hYzsj4sZ4ik0yC1ho0ea/7g6rFMlcLZYeDT/ZSgG5Bgb//rOyllXoIXURbW9hHgFlQ6s0hM35V3hFZjiEXL/wNM+c2Tyxpsz+8r7GUTIVSw1lGrDSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=O57wXf8QTGFxw9gGS1emYmTJr7uuT/RTQdm/5MF46Ao=;
 b=AvUSfqouTnUxdz2USoHcmuzyMuYdVhBWlnurtIpY/LwapwXEQM96a51yQPU83fxd3frWcJlXQEky39t9EpkVnXpbD7h7ykaQBjdaVzOyXiXAis8vI4IRb6JF5OUFxny/lGqY7OERVRGN7bCyYHFbzN0EpOAZ8C2mGJV4GNL4aoEmrI6ZGeLKwe7/bQodfk4irxTdJfLhbSYFclFvyQvpFlEpuUUZfm/aJw98Egg8pHsf9I4lcXz0GWhfGvCxRY2DqfgShJMge73zUin6E/sak8plfLgGLhDNV3l3QGcvKwsgiFHZXes35y9h2DFLFO+1s9EJFSc8L4txDyGS4cpwfg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O57wXf8QTGFxw9gGS1emYmTJr7uuT/RTQdm/5MF46Ao=;
 b=Ex8602ceELHMPW3A0MxveSy3WRqHd0x/hUGKpmVjuiX5htcoCMOpEqYgITU4jWoyi+3fTetmY+54XERoC2zh9/CU6sFmGxAXV2fzvD2e110jiY0hE8YjKJdcS4vsDpzdxEfh15yLxzbgTbFxmB4HZsbzhz6tzoJlGEwMuTx5rnEKvFXoY07a7j0hnmw0rpR2aAy3xrxNuQgrfTzfwR0UTtIvlbzqVNGgDEY+J5uiA3kkVyyTPhlhY7SK4M3TDi5GvKn4Z/VtCRWrSp8iPPiRUYg5WQvl1Se6BHrK+A2MHDFHSIjrDQxkhbdKhprNl3OUNvTi3f+0tpU2ww4U3HEauw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>
CC: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: 
 AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbACAAAVGAIABVeSAgAAoPoCAAAP7gA==
Date: Tue, 28 Sep 2021 06:56:22 +0000
Message-ID: <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
In-Reply-To: <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 67fe1168-59ab-4668-061e-08d9824d14f9
x-ms-traffictypediagnostic: AM9PR03MB7425:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM9PR03MB74258D702110844B0CA69E3EE7A89@AM9PR03MB7425.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 T8YSrpALKmHRWdlIWjTHgCujExSPMLt3hyG2fb64HeP7Y3FbuyW0+TGBGxMlZnLZnQ3ydZw8XR5G0aK3aIDtQapLrGUbWiDewVEk9IKm5/I3UCDSE6oPcCiZn+t71KZYO+RItzNZyRgCj+4nDlK5u413XcjWeDmsf+YBSUMNd0K54hsis+mgogEyjim91ZU75qT5t+myxe11aFwDkjs2mMxj73XBSWWUSnQe1/6n3F3cyWP9lsBaBCQfTgSIdIrQXbt7aUWwvU20UOsoCU9mCVfVhLouWcV3tJbKbPqQ4c/V2iAwm7vOiD8vzAa4AhgsHbjJlLqPWBnB9LZPVnDvEgon7l/rAe3k+ei4i730bFZ9m6bErnPGhHOvgBMxoW4nUJIwg4eoDn1WSauBRn2Zog9iwNoiWDflGtyQ4XMaVnr0l9FSj6yDyNJblmVakeQ0P0UMbYjYW/XgcDNrddjCB6xo0fHNO3YDHLi21quwN8amUUNr/PNP1aMjzNNQwIy3TIygOarHjNKKI/r7+HW7qAmuMjCfS7r548AVqBHwcZD2r9Vuuk1hD0uGMOdhDCiq6RRSzPO/GWYV5KXXdyHMj+T6FfWjy7mZhlGOuOG03+73ZYXjZ2G4IEIxvbUHUNsimCgw/zpUaekLhFgSnu5/9AK8+i3/tr0wjSgVLrH2PEpyP/+7FXqxZg89RLsSvQdezAdbMGHJOfLvfFIA+GvNo3U+UGl5eVoXnEW956K9QpsyKopTMOp7C1qYsRa2VLSr
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(8676002)(316002)(54906003)(53546011)(6506007)(122000001)(26005)(4326008)(36756003)(83380400001)(2906002)(71200400001)(38100700002)(6486002)(64756008)(5660300002)(186003)(66946007)(55236004)(31686004)(86362001)(31696002)(2616005)(91956017)(76116006)(66476007)(6512007)(66446008)(66556008)(508600001)(38070700005)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?TEdndmdYZkdLOEppMGtpaHRXUFFodW5iam5GZUlPWjBESVRXRi8rclpyYnZU?=
 =?utf-8?B?V3FYaXEyTXp5R2FwQkxDNzJRdExpWlRDMk5GRWg2dTMraGlPNHppOFVCTnBy?=
 =?utf-8?B?MENqS0Q5ZVVGZlhRSHB2M3JPYXVUemZwU2RxQUJJL0c5eTRiMXpydDUxWkow?=
 =?utf-8?B?ZkxzL25USHRVTVBoWlJpKzZUVm91WkZpaXdTbGt1NWt3SmZiNkNPT2w1RkI4?=
 =?utf-8?B?Yyt4V082RWwwMGkxUG5pS3grcUpYMHBNbTEwNWxyaE5SQUh0aDg2UmluSVFk?=
 =?utf-8?B?emNlYUJSNERZc0FIanNPSGZzR1piMTBSTnlKYXJEYVh1REZtTSt0NTlCTGZk?=
 =?utf-8?B?SVZZaytmbUZQOGhWTyszclJaMjFWTCtWV1o4dzBPeWpkazUvS0ZreHk1Z2cy?=
 =?utf-8?B?bUpjZXROS3U5VVNxam5obkdReEx6akF2Zkd2OGN0K2wzZER6dnUvU2pFb05L?=
 =?utf-8?B?bW5nempDMjZjbGJJVXFFaU9JUmh3aDVmalFVMEhBVjhuZitCUFJVOVh1d0s5?=
 =?utf-8?B?dFlMSVduMUN6ZHQwNU9ERFIxaGs0LzdSV1JYdHVVelpoU1pra0c2RlgxS2U3?=
 =?utf-8?B?NVArT3lXYnhjQ2sxWFBWdVFoeUw5Z25mazNrUy9mcWd5TW93KzNIWkRPRERX?=
 =?utf-8?B?Q3lyM2N6V2tIeTg5cmo4SzNZV1lrMlgzeTZsa2JVcy96RlVaKzFCNVZVaXhF?=
 =?utf-8?B?VmNmUEh0VlQzS1ErMmdnQWhRcEpzRDFaQ3hoRXJLZGJSbVdEN1QzYTN5dVgz?=
 =?utf-8?B?S2xmSno5SDlyL2tGZTVXNi9zL2pEeWpCNzZoaWhKbnM2UGVsSGx0TnI3OG15?=
 =?utf-8?B?ZmQyRDVab3pLaS81OGJ4Y3pURlJxN3NDbDVkUmV5VmNSaDZNRmc0eFRKaVdJ?=
 =?utf-8?B?UmFhLzZxTGJDRTFTU3BQZk9URWJndnRmNUFRbkRVRHVIMm1iMzhsR2JIQU9D?=
 =?utf-8?B?T2RUd3N6c1p6NGhlSXBvbXhwTHZaYWVYVldJVVJBTEpwN3h2Uzd2a0RaQy95?=
 =?utf-8?B?Z0hzTlRzSFI3NTFXNHJqbTNOd3VWckpPZnNWeWtmQWxtVzFLR1p6RSs5Y3RE?=
 =?utf-8?B?RDUyTis1V2JaSjdIMGs3ajYwWWJycjY2YVA1YTdXdVFUQkNhYWNnSTNoU1dr?=
 =?utf-8?B?MkhhQlZuSEFPWDhGZXhFdTU0NnZBQWhkUUduYXUwd3JnT1ZPTzN5VTJjMHo0?=
 =?utf-8?B?TGlwZ1J5NXJMYkZNQ2MxVWc2YVFpY3JxQzhRbHJmcEVXWEZxWXlTQUpERm55?=
 =?utf-8?B?dFcxeFl2TDlUZWNQRU9sNEZkeVJVZnIxNDRZTHo2eXQ3SWRXbmpzTVlxSThB?=
 =?utf-8?B?S0gydXQ5ek5EN3VZOXNNMEU4N3pxUFAwREIwaE9ibm4xMFFpNUdjSEFZWUtT?=
 =?utf-8?B?Y2N1TGJsazRacjJvVFovM29zZkg1SFhkVzFFMGlpKzB3Wjh2Z3hoSXM4ZXNR?=
 =?utf-8?B?Zm1HdGJEN1BRazhWTGF2ZHYyRlZYM0pVazJlWkpjQmtLTkQ1VE9VNGNkOFBE?=
 =?utf-8?B?d29KMjU4b1JaM0dVbTNTMVlYKzlGTTFEbWFla1crRndIdUNpdURFL3ltOGRo?=
 =?utf-8?B?TEl4K1V2cVRIdHBJV1RZMmducklrU2tmRDhNcjJvSlVBZWF0a0NuanRtaG1s?=
 =?utf-8?B?R0VReiswbU91WWRqQzFtc25BWDI3cW9jR01tbzRyKzI4Vk9TUG9rR2pnL1pI?=
 =?utf-8?B?S2h4QUFKem9ERGxWUUdjbWtmZ2U4RDR4S1JlYjlJS0U2dUJwbUszUm9xcS9O?=
 =?utf-8?B?VHBCbTdzVU4raTJ1dzhGNTdzWm5VcjNPY2picDJVc3RnYVBhcnFZYXZXbU5a?=
 =?utf-8?B?YW1hQlcrSFREZnQ3aVc1UT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <866A7BCA3C583D409B70368A602F1C68@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 67fe1168-59ab-4668-061e-08d9824d14f9
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 06:56:22.4517
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: JJsI9iw6lEfQkLJyDNS/WORz01YWnq/PJowLO4jhFDQfONHXbDPTmoSNaTu8lSN09rmMAMr8ZiSyebp07e58TuNd/mkXNpzqImMbM7o05NKd+9aym30N7DgVez2p2d6e
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7425
X-Proofpoint-GUID: 86QRxxebSagseBb6eftrAI1xhtV5f20O
X-Proofpoint-ORIG-GUID: 86QRxxebSagseBb6eftrAI1xhtV5f20O
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 impostorscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0
 phishscore=0 adultscore=0 mlxscore=0 suspectscore=0 spamscore=0
 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2109230001 definitions=main-2109280039

DQpPbiAyOC4wOS4yMSAwOTo0MiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI4LjA5LjIwMjEg
MDY6MTgsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4+IE9uIE1vbiwgMjcgU2VwIDIwMjEs
IEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4gT24gMjcuMDkuMjEgMDk6MzUsIE9sZWtzYW5kciBB
bmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxMDoyNiwgSmFuIEJldWxpY2gg
d3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMDIxIDA4OjU4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byB3cm90ZToNCj4+Pj4+PiBGcm9tOiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRy
X2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pj4NCj4+Pj4+PiBDdXJyZW50bHkgUENJIGJh
Y2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlvbmFsaXRpZXMgYXQgYSB0aW1lLg0KPj4+
Pj4+IFRvIG5hbWUgYSBmZXc6DQo+Pj4+Pj4gMS4gSXQgaXMgdXNlZCBhcyBhIGRhdGFiYXNlIGZv
ciBhc3NpZ25hYmxlIFBDSSBkZXZpY2VzLCBlLmcuIHhsDQo+Pj4+Pj4gICAgICAgcGNpLWFzc2ln
bmFibGUte2FkZHxyZW1vdmV8bGlzdH0gbWFuaXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywNCj4+Pj4+
PiB3aGVuZXZlcg0KPj4+Pj4+ICAgICAgIHRoZSB0b29sc3RhY2sgbmVlZHMgdG8ga25vdyB3aGlj
aCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRocm91Z2gNCj4+Pj4+PiAgICAgICBpdCByZWFk
cyB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVudHJpZXMgb2YgdGhlIHBjaWJhY2suDQo+
Pj4+Pj4gMi4gSXQgaXMgdXNlZCB0byBob2xkIHRoZSB1bmJvdW5kIFBDSSBkZXZpY2VzIGxpc3Qs
IGUuZy4gd2hlbiBwYXNzaW5nDQo+Pj4+Pj4gICAgICAgdGhyb3VnaCBhIFBDSSBkZXZpY2UgaXQg
bmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFudA0KPj4+Pj4+IGRldmljZQ0KPj4+
Pj4+ICAgICAgIGRyaXZlciBhbmQgYm91bmQgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3BlYWtpbmcg
aXQgaXMgbm90IHJlcXVpcmVkDQo+Pj4+Pj4gICAgICAgdGhhdCB0aGUgZGV2aWNlIGlzIGJvdW5k
IHRvIHBjaWJhY2ssIGJ1dCBwY2liYWNrIGlzIGFnYWluIHVzZWQgYXMgYQ0KPj4+Pj4+ICAgICAg
IGRhdGFiYXNlIG9mIHRoZSBwYXNzZWQgdGhyb3VnaCBQQ0kgZGV2aWNlcywgc28gd2UgY2FuIHJl
LWJpbmQgdGhlDQo+Pj4+Pj4gICAgICAgZGV2aWNlcyBiYWNrIHRvIHRoZWlyIG9yaWdpbmFsIGRy
aXZlcnMgd2hlbiBndWVzdCBkb21haW4gc2h1dHMNCj4+Pj4+PiBkb3duKQ0KPj4+Pj4+IDMuIERl
dmljZSByZXNldCBmb3IgdGhlIGRldmljZXMgYmVpbmcgcGFzc2VkIHRocm91Z2gNCj4+Pj4+PiA0
LiBQYXJhLXZpcnR1YWxpc2VkIHVzZS1jYXNlcyBzdXBwb3J0DQo+Pj4+Pj4NCj4+Pj4+PiBUaGUg
cGFyYS12aXJ0dWFsaXNlZCBwYXJ0IG9mIHRoZSBkcml2ZXIgaXMgbm90IGFsd2F5cyBuZWVkZWQg
YXMgc29tZQ0KPj4+Pj4+IGFyY2hpdGVjdHVyZXMsIGUuZy4gQXJtIG9yIHg4NiBQVkggRG9tMCwg
YXJlIG5vdCB1c2luZyBiYWNrZW5kLWZyb250ZW5kDQo+Pj4+Pj4gbW9kZWwgZm9yIFBDSSBkZXZp
Y2UgcGFzc3Rocm91Z2guIEZvciBzdWNoIHVzZS1jYXNlcyBtYWtlIHRoZSB2ZXJ5DQo+Pj4+Pj4g
Zmlyc3Qgc3RlcCBpbiBzcGxpdHRpbmcgdGhlIHhlbi1wY2liYWNrIGRyaXZlciBpbnRvIHR3byBw
YXJ0czogWGVuDQo+Pj4+Pj4gUENJIHN0dWIgYW5kIFBDSSBQViBiYWNrZW5kIGRyaXZlcnMuDQo+
Pj4+Pj4NCj4+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbw0KPj4+
Pj4+IDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+Pg0KPj4+Pj4+IC0t
LQ0KPj4+Pj4+IENoYW5nZXMgc2luY2UgdjM6DQo+Pj4+Pj4gLSBNb3ZlIENPTkZJR19YRU5fUENJ
REVWX1NUVUIgdG8gdGhlIHNlY29uZCBwYXRjaA0KPj4+Pj4gSSdtIGFmcmFpZCB0aGlzIHdhc24n
dCBmdWxseSBkb25lOg0KPj4+Pj4NCj4+Pj4+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9NYWtlZmlsZQ0KPj4+Pj4+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxl
DQo+Pj4+Pj4gQEAgLTEsNSArMSw2IEBADQo+Pj4+Pj4gICAgICMgU1BEWC1MaWNlbnNlLUlkZW50
aWZpZXI6IEdQTC0yLjANCj4+Pj4+PiAgICAgb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VO
RCkgKz0geGVuLXBjaWJhY2subw0KPj4+Pj4+ICtvYmotJChDT05GSUdfWEVOX1BDSURFVl9TVFVC
KSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+PiBXaGlsZSBiZW5pZ24gd2hlbiBhZGRlZCBoZXJlLCB0
aGlzIGFkZGl0aW9uIHN0aWxsIGRvZXNuJ3Qgc2VlbSB0bw0KPj4+Pj4gYmVsb25nIGhlcmUuDQo+
Pj4+IE15IGJhZC4gU28sIGl0IHNlZW1zIHdpdGhvdXQgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiB0
aGUgY2hhbmdlIHNlZW1zDQo+Pj4+DQo+Pj4+IHRvIGJlIG5vbi1mdW5jdGlvbmFsLiBXaXRoIENP
TkZJR19YRU5fUENJREVWX1NUVUIgd2UgZmFpbCB0byBidWlsZCBvbiAzMi1iaXQNCj4+Pj4NCj4+
Pj4gYXJjaGl0ZWN0dXJlcy4uLg0KPj4+Pg0KPj4+PiBXaGF0IHdvdWxkIGJlIHRoZSBwcmVmZXJl
bmNlIGhlcmU/IFN0ZWZhbm8gc3VnZ2VzdGVkIHRoYXQgd2Ugc3RpbGwgZGVmaW5lDQo+Pj4+DQo+
Pj4+IENPTkZJR19YRU5fUENJREVWX1NUVUIsIGJ1dCBpbiBkaXNhYmxlZCBzdGF0ZSwgZS5nLiB3
ZSBhZGQgdHJpc3RhdGUgdG8gaXQNCj4+Pj4NCj4+Pj4gaW4gdGhlIHNlY29uZCBwYXRjaA0KPj4+
Pg0KPj4+PiBBbm90aGVyIG9wdGlvbiBpcyBqdXN0IHRvIHNxdWFzaCB0aGUgdHdvIHBhdGNoZXMu
DQo+Pj4gU3F1YXNoaW5nIHdvdWxkIGJlIGZpbmUgZm9yIG1lLg0KPj4gICANCj4+IEl0IGlzIGZp
bmUgZm9yIG1lIHRvIHNxdWFzaCB0aGUgdHdvIHBhdGNoZXMuDQo+Pg0KPj4gQnV0IGluIGFueSBj
YXNlLCB3b3VsZG4ndCBpdCBiZSBiZXR0ZXIgdG8gbW9kaWZ5IHRoYXQgc3BlY2lmaWMgY2hhbmdl
IHRvOg0KPj4NCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmls
ZSBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+PiBpbmRleCBlMmNiMzc2NDQ0
YTYuLmUyM2M3NThiODVhZSAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNr
L01ha2VmaWxlDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4g
QEAgLTEsNiArMSw1IEBADQo+PiAgICMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAN
Cj4+IC1vYmotJChDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQo+
PiAtb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfU1RVQikgKz0geGVuLXBjaWJhY2subw0KPj4gK29i
ai0kKENPTkZJR19YRU5fUENJX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4gQnV0IHRoYXQgd291
bGRuJ3QgYWxsb3cgdGhlIGRyaXZlciB0byBiZSBhIG1vZHVsZSBhbnltb3JlLCB3b3VsZCBpdD8N
Cg0KRXhhY3RseS4gSSBmb3Jnb3QgdGhhdCB3aGVuIHBsYXlpbmcgd2l0aCBtb2R1bGUvYnVpbHQt
aW4gSSB3YXMgbm90IGFibGUNCg0KdG8gY29udHJvbCB0aGF0IGFueW1vcmUgYmVjYXVzZSBDT05G
SUdfWEVOX1BDSV9TVFVCIHdpbGwgYWx3YXlzIGJlDQoNCmluICJ5IiBzdGF0ZSwgdGh1cyBldmVu
IGlmIHlvdSBoYXZlIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQ9bQ0KDQp5b3Ugd29uJ3QgYmUg
YWJsZSB0byBidWlsZCBpdCBhcyBtb2R1bGUuIFNvLCBJIHdpbGwgcHJvYmFibHkgcHV0IGEgY29t
bWVudA0KDQphYm91dCB0aGF0IGluIHRoZSBNYWtlZmlsZSBleHBsYWluaW5nIHRoZSBuZWVkIGZv
cg0KDQpvYmotJChDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQpv
YmotJChDT05GSUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQoNCj4NCj4gSmFu
DQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 06:59:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 06:59:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197602.350741 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV756-0000z8-94; Tue, 28 Sep 2021 06:59:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197602.350741; Tue, 28 Sep 2021 06:59:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV756-0000z1-5b; Tue, 28 Sep 2021 06:59:12 +0000
Received: by outflank-mailman (input) for mailman id 197602;
 Tue, 28 Sep 2021 06:59:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV755-0000yv-6j
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 06:59:11 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 942ec4c8-2029-11ec-bc6d-12813bfff9fa;
 Tue, 28 Sep 2021 06:59:09 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1E9E92233E;
 Tue, 28 Sep 2021 06:59:09 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BB65113513;
 Tue, 28 Sep 2021 06:59:08 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id rB/CKzy9UmHWVwAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 06:59:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 942ec4c8-2029-11ec-bc6d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632812349; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=8b2olotqmBe8T0BpUmGKRkSj3kbHQqlP9ZL2h4U6D6w=;
	b=phtozVx+GmmE9nDTx+OxHMDSUVol8CU7ms/13mLHmu9Ca7R51FPGBHNnO6DQZY0toZvuT8
	MYN+ihzpcUVfI9Y7V8/UqlLTxosoHJ3IhShAIxiq7x1LXGoKXdTUZa+xbuvR4JR+pmAhG6
	RZhIKpdgrV75Li7bMupdYD7AkDLDhGM=
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
Date: Tue, 28 Sep 2021 08:59:07 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="y4XK9qQ6Q6UB0y5bw2qxgXOrIOa8fCY7M"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--y4XK9qQ6Q6UB0y5bw2qxgXOrIOa8fCY7M
Content-Type: multipart/mixed; boundary="WBDbVkRjhr2Xl4zRhV5ApKwybLg9i6LKa";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Message-ID: <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
In-Reply-To: <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>

--WBDbVkRjhr2Xl4zRhV5ApKwybLg9i6LKa
Content-Type: multipart/mixed;
 boundary="------------2FAFFFE2618023AB1523ED08"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------2FAFFFE2618023AB1523ED08
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 08:56, Oleksandr Andrushchenko wrote:
>=20
> On 28.09.21 09:42, Jan Beulich wrote:
>> On 28.09.2021 06:18, Stefano Stabellini wrote:
>>> On Mon, 27 Sep 2021, Juergen Gross wrote:
>>>> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
>>>>> On 27.09.21 10:26, Jan Beulich wrote:
>>>>>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>>>
>>>>>>> Currently PCI backend implements multiple functionalities at a ti=
me.
>>>>>>> To name a few:
>>>>>>> 1. It is used as a database for assignable PCI devices, e.g. xl
>>>>>>>        pci-assignable-{add|remove|list} manipulates that list. So=
,
>>>>>>> whenever
>>>>>>>        the toolstack needs to know which PCI devices can be passe=
d through
>>>>>>>        it reads that from the relevant sysfs entries of the pciba=
ck.
>>>>>>> 2. It is used to hold the unbound PCI devices list, e.g. when pas=
sing
>>>>>>>        through a PCI device it needs to be unbound from the relev=
ant
>>>>>>> device
>>>>>>>        driver and bound to pciback (strictly speaking it is not r=
equired
>>>>>>>        that the device is bound to pciback, but pciback is again =
used as a
>>>>>>>        database of the passed through PCI devices, so we can re-b=
ind the
>>>>>>>        devices back to their original drivers when guest domain s=
huts
>>>>>>> down)
>>>>>>> 3. Device reset for the devices being passed through
>>>>>>> 4. Para-virtualised use-cases support
>>>>>>>
>>>>>>> The para-virtualised part of the driver is not always needed as s=
ome
>>>>>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-fr=
ontend
>>>>>>> model for PCI device passthrough. For such use-cases make the ver=
y
>>>>>>> first step in splitting the xen-pciback driver into two parts: Xe=
n
>>>>>>> PCI stub and PCI PV backend drivers.
>>>>>>>
>>>>>>> Signed-off-by: Oleksandr Andrushchenko
>>>>>>> <oleksandr_andrushchenko@epam.com>
>>>>>>>
>>>>>>> ---
>>>>>>> Changes since v3:
>>>>>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>>>>>> I'm afraid this wasn't fully done:
>>>>>>
>>>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>>>> @@ -1,5 +1,6 @@
>>>>>>>      # SPDX-License-Identifier: GPL-2.0
>>>>>>>      obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>>>>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>>> While benign when added here, this addition still doesn't seem to
>>>>>> belong here.
>>>>> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seem=
s
>>>>>
>>>>> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build =
on 32-bit
>>>>>
>>>>> architectures...
>>>>>
>>>>> What would be the preference here? Stefano suggested that we still =
define
>>>>>
>>>>> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate=
 to it
>>>>>
>>>>> in the second patch
>>>>>
>>>>> Another option is just to squash the two patches.
>>>> Squashing would be fine for me.
>>>   =20
>>> It is fine for me to squash the two patches.
>>>
>>> But in any case, wouldn't it be better to modify that specific change=
 to:
>>>
>>> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciba=
ck/Makefile
>>> index e2cb376444a6..e23c758b85ae 100644
>>> --- a/drivers/xen/xen-pciback/Makefile
>>> +++ b/drivers/xen/xen-pciback/Makefile
>>> @@ -1,6 +1,5 @@
>>>    # SPDX-License-Identifier: GPL-2.0
>>> -obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>> -obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>> +obj-$(CONFIG_XEN_PCI_STUB) +=3D xen-pciback.o
>> But that wouldn't allow the driver to be a module anymore, would it?
>=20
> Exactly. I forgot that when playing with module/built-in I was not able=

>=20
> to control that anymore because CONFIG_XEN_PCI_STUB will always be
>=20
> in "y" state, thus even if you have CONFIG_XEN_PCIDEV_BACKEND=3Dm
>=20
> you won't be able to build it as module. So, I will probably put a comm=
ent
>=20
> about that in the Makefile explaining the need for
>=20
> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o

In case the real split between both parts of xen-pciback is done this
will be needed anyway.


Juergen


--------------2FAFFFE2618023AB1523ED08
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------2FAFFFE2618023AB1523ED08--

--WBDbVkRjhr2Xl4zRhV5ApKwybLg9i6LKa--

--y4XK9qQ6Q6UB0y5bw2qxgXOrIOa8fCY7M
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFSvTsFAwAAAAAACgkQsN6d1ii/Ey+A
WQf+NRBEl6DmoTEEl+iQbKygzsYAUM2iPZECUpYS1qb/EZaYSPmnRAeP5rDEcSrAb4OUUJPVKoj+
pftLjG6fez7P7EWn/9uX9AxszDdQvHkGxZJOItIB1tzX0vIvb0PYt/5871fN4BGwtsxUK5YxcnJz
IqrkmS10nPq+Sk3xC0cHQuGO2WJbRR1tGKtHTKXbjzj3X4neQC2MQhBT4YcneAJWb3L6ddSyqhYh
RmOODbo3rDBPBgvvlX6wkz+CQYtAP9QjbQf0USKFGzFSGX1RCapw8HEoCcHQabiQWRzpTM1zSgWM
hxjutNZx9UYRHlTCYOZ/WE4r+F8ysdBXZixgcSOJsQ==
=YXjY
-----END PGP SIGNATURE-----

--y4XK9qQ6Q6UB0y5bw2qxgXOrIOa8fCY7M--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:12:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:12:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197608.350752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7I0-0003MO-GE; Tue, 28 Sep 2021 07:12:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197608.350752; Tue, 28 Sep 2021 07:12:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7I0-0003MH-Ca; Tue, 28 Sep 2021 07:12:32 +0000
Received: by outflank-mailman (input) for mailman id 197608;
 Tue, 28 Sep 2021 07:12:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sHmr=OS=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mV7Hz-0003MB-8j
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:12:31 +0000
Received: from mail-wr1-x433.google.com (unknown [2a00:1450:4864:20::433])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 46f61a1f-09da-4549-9cc3-3a05dd2271d0;
 Tue, 28 Sep 2021 07:12:30 +0000 (UTC)
Received: by mail-wr1-x433.google.com with SMTP id t8so56299225wri.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:12:30 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id r6sm2032203wrp.41.2021.09.28.00.12.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 00:12:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 46f61a1f-09da-4549-9cc3-3a05dd2271d0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=GaYYYFgnPlLR3k+/irQBsFsyl5Li0H3pN+dzpabEdF4=;
        b=EcS5Yt22pkp+j9/fwmecfi/V6S/6356P/dLrjU6yYABKDoEvmG+fWG6Yn89JaVcMTX
         BS7k6HO7vB/vcPcTiZQ1PIf5e22heEFF+2Fm8iHnRpJYdWH5DGAggqYC6trz2PIkk4ZY
         dZ4V9G2WTC8+sfYBN4IdXo5hcWfwFWDxIqoW8usWU0nVGJUYBvjVVE9omZigTi+OINxe
         l9nyBmQuwV+XkxRGtOgqqZHGW1/x+wMI1Hc0SEos+40+AYk517DdCCwaXb9Kdsyq692S
         Y0Bjrxudo8vmyWV4DYfLbn3Cm+NCanFjeA9aDpx+icKWYtptMmJo0efr97bh9EbreEAN
         yyPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=GaYYYFgnPlLR3k+/irQBsFsyl5Li0H3pN+dzpabEdF4=;
        b=PnGJGzjFHfFIqhRW5JtF7vaIndo2kKTr92VrKGfEeOsbnJv7pbk58szgIz9RKLTr1P
         RRCO8fehavd50YGs9EhNzOyWg66qwrnTAeZtsyWxoYZLe8rNVF5rRbhGr2GdTHX6AnA7
         4SVi+TN2L+ZdQcan9bXekUNbtkDrbCzIy/gwKfieoHV7hsY/EPFCWFlfcOL6UFKuuCCu
         11HeqwACJuAFUcXSsp+N3eo7zcRRPK4Cpp6+Ga0lXDgGeqnHW96aS0NYdNrksIpGCO/t
         5SP94aalTf4u80h8XIfxwuowYBJAlX4uw/e+0T+cAOKqIJExHFYW0sTwX6bLp/jDdabX
         3JeA==
X-Gm-Message-State: AOAM531a9y86Cz2lEIO+pTrznrRErLLvofN5lWPUdMtGDQyn71Vg/rtg
	9FS6lwYRQVykL9NAjO8ngeQ=
X-Google-Smtp-Source: ABdhPJyTtWBvdFtmruKesMEj/0t+gM9zzmpxxjy8MbSrdKRzujo67CTP/NR4DY/UNnrjrLTXa975ww==
X-Received: by 2002:adf:e30d:: with SMTP id b13mr4592901wrj.438.1632813149466;
        Tue, 28 Sep 2021 00:12:29 -0700 (PDT)
Message-ID: <f5dad7a7-0309-79f6-c592-18dda495e774@gmail.com>
Date: Tue, 28 Sep 2021 08:12:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Reply-To: paul@xen.org
Subject: Re: [PATCH v8 1/6] AMD/IOMMU: obtain IVHD type to use earlier
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <d5f76461-70d2-fc59-2213-99a093e3b57f@suse.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <d5f76461-70d2-fc59-2213-99a093e3b57f@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 22/09/2021 15:36, Jan Beulich wrote:
> Doing this in amd_iommu_prepare() is too late for it, in particular, to
> be used in amd_iommu_detect_one_acpi(), as a subsequent change will want
> to do. Moving it immediately ahead of amd_iommu_detect_acpi() is
> (luckily) pretty simple, (pretty importantly) without breaking
> amd_iommu_prepare()'s logic to prevent multiple processing.
> 
> This involves moving table checksumming, as
> amd_iommu_get_supported_ivhd_type() ->  get_supported_ivhd_type() will
> now be invoked before amd_iommu_detect_acpi()  -> detect_iommu_acpi(). In
> the course of doing so stop open-coding acpi_tb_checksum(), seeing that
> we have other uses of this originally ACPI-private function elsewhere in
> the tree.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Paul Durrant <paul@xen.org>


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:17:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:17:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197614.350763 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7MS-0004AX-2W; Tue, 28 Sep 2021 07:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197614.350763; Tue, 28 Sep 2021 07:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7MR-0004AQ-V8; Tue, 28 Sep 2021 07:17:07 +0000
Received: by outflank-mailman (input) for mailman id 197614;
 Tue, 28 Sep 2021 07:17:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV7MQ-0004AK-Ee
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:17:06 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id be6b7809-470f-4078-a60e-866deec3f95b;
 Tue, 28 Sep 2021 07:17:05 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S73jMb006884; 
 Tue, 28 Sep 2021 07:17:04 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbxb0r1uq-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 07:17:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6898.eurprd03.prod.outlook.com (2603:10a6:20b:2d6::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 07:17:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 07:17:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be6b7809-470f-4078-a60e-866deec3f95b
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OHxSZtAAYgSZx4fUo9YT7Qf60gg9XMngy6rh/m8I+AauKUod+NWPGbvJgCssbHu9Y60NbOxtUYgS2Mx6tGS6ke4RAM0A6MVSDfwWOjlB2R4pDohdFkXvZoWaGX7Taiy3cgrQGadi5n0g2y63jf//Li9dhxMDA9dSpifUjIaNn/5sQ3isNImVbSX0d/ggP06pdv8Wk4g7Cq9v/RgUbnl5KvsJa9ahLd5yd1MdiSayGuLwmd3iad33OnRhcCp1aruFSxaOXXqXb7GQC6Q1LylmWgiHU9+XqPjoTIBTCeJZE3xgOI/R+MkMz6Ot5sF1jaPQ2JFMai16qchuS0GezCt2ig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=aFfHw5u4eKbcsXOjtcO14Mn58U6WlfFn2mOLJ5ED/bE=;
 b=mIM4q3Cz5QcVEos/glIO/kRC4632EnWC7BOHK2Vvx5B6cdx4O/GWRu0IEhwg5RUOTblkiMCAMlVoVt8pFOK39s5vePnud+Fd4mVgxYfuSRLGDvolw6bmc/BexypFibXdiwhfIWBmlIai6MUH/AWZMt80mYazX2Nv4PJJFzibE8ed3aNQj4m3krMno8uwMencyORFzUTtU1ryydXFLAJczRajFExVJtcZcaOGL/Jbu9mkT+zk2TiYgYSDOBHk4lQ/siXy4/uUYtenC99b/ZY358U/NHIvY78LYsv3CpAXBvhK2HSCToamV3orAxufXRjfBWWoCzBPxxnuE7HEvmjfsA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aFfHw5u4eKbcsXOjtcO14Mn58U6WlfFn2mOLJ5ED/bE=;
 b=iulV8qGl+izbTWdB80zUzL/k83xAnwa8chVEwG3A/Jug4IMp4hwX9bWKRlT/Z8SdECkVxvxi4X0bSOzrSyI+TTBtwNsR+rRWmJz+D+YtzKSfcSeMGhaF31VrQlbfdpMhwMJrNI2uXhExUKtMFtzjQTpdx1gt9mpsejysI13nWB1cq3hliLrGvyq4klz7LhVJnRtCeskJ+QOuBAaqZalBM9BP/jn3ry8KL1WrUdCjxy38jYqQHnhn713wSRazmcaeGvdK7epBHaXS/PAtJizXPnnkTOuecI00f6NSENZbS3hNTrhl6DsBqO58Fb80jMUxJaaSeQw+fPpz7wCnNAdVyQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
        Stefano
 Stabellini <sstabellini@kernel.org>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: 
 AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbACAAAVGAIABVeSAgAAoPoCAAAP7gIAAAMWAgAAFAAA=
Date: Tue, 28 Sep 2021 07:17:00 +0000
Message-ID: <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
In-Reply-To: <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 80d5b3b3-0194-4077-1b57-08d9824ff708
x-ms-traffictypediagnostic: AM9PR03MB6898:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM9PR03MB689852C1196D1CD92D823A93E7A89@AM9PR03MB6898.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ZzrQySVK6qBbfvwQqoeosUfG5By63/+WbTmuyM2yxQ1NL/FNhpfXo61dQnhj3zrpReJQcLte/4au2/7YeXDiXIqYR/cseGWNVlfc/zu4e+MArvYRef8au9mYcIHBFT3PltzQTiC7u35+X0cSe6gTWCiWhpy19e+M4zdtHNyblC+hVlDKWoxkxtYbu+Qj5hFgsXlCL8pHPArBnwuISXDyaf9CmAGEx0OCWo6lvNFrxdRmU6HVtl/+jKSG8wh1TFToD5Cap2m0CpUbCxjfP3JeCgidipkAgggSBkj2CPQKlW1aiHUojB6Ir2F+hekokfhRL7Vbxa3c25glPn9H0jbuZcFMKB+rrRdEEPock7LGl8TNI893LaWujvY2ZaSdDNy1VnYkZQb7w6dVtv1y/0KGQ4E5dS5UVvwZBxqJBMyo6GU+l1NSHTNWpmRoNf30gSEFZLQW1j0VBRXsbsp5HGcY6M86mDhioVkLJ3y1jBoWdFfWQh5OQ1tQhcb+slPnd8bHavydLp2Oy84HqX6KizAD4dQ1o6OlaoKBpuV8I1Sn5itDSCTQOwnOl1X716YdxbZT/YHd0xYq5lggQJ4kl8+oegpK2rmlyJGjmGEUuK94WfhZ+ijVJLjk1O37LSzEa9d862zgWrbMdaViGY7ow/1dPpKRXcEosr93j+Px5BCUHN4eqmpa6YYPg3wIUD87c3ObMptTSmDu1tvF30y9wOV9YQr0rWpbEI4MoHg6oRHbJDe1lNZGDQ0MmHBlwehpt7JA
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(38100700002)(508600001)(36756003)(83380400001)(53546011)(38070700005)(6506007)(86362001)(31696002)(2616005)(6486002)(6512007)(66946007)(31686004)(66556008)(55236004)(66476007)(2906002)(5660300002)(26005)(64756008)(66446008)(91956017)(110136005)(54906003)(71200400001)(186003)(76116006)(8936002)(107886003)(316002)(4326008)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?N0dTaTFuVzVQcnhIRjRVOEh6QjEwdUk0enh4cFBrRjBQVXJLM040WGVrcUgr?=
 =?utf-8?B?M2JBZlUzRkJQTkFjVmtMVmRkMU9welc0cGdWclNyWU50MEQ5bERLWjN1M01D?=
 =?utf-8?B?L2tVVWJ1VXRTcjgxcmhBSy9lUnkyb1FNWmt4K0pIUHgrQVZaRllmMUNHdnRs?=
 =?utf-8?B?NXhUS3ZoWjZlcldsUGdyY2hXL0pDQzFIL1NsM0dGQmFtWjNiTEtFLy9LOW04?=
 =?utf-8?B?WHBqNEJSak8rbGowU0ttODVUYnlSSnFwelA4VzU5Tkh0QmVRcHNVZzRTdzhH?=
 =?utf-8?B?ajVWVTJocjA2YVgrajZPMEprRHVPVFFrZE1VZDNXVE5ucmlGSHBsOVR4RWd0?=
 =?utf-8?B?cTJ3eWUza2pjUDFoNWxZOVEvR1Q4TjRTb2t0RXUvTXFMalJsVFVDdWZyMTJm?=
 =?utf-8?B?RTN5R05lVVVVdVkyQlRqeWRnSlRoU3pVb2R1TGNnTG1BblV1MC9nR05RYUw0?=
 =?utf-8?B?d2NxUVlMTm5YRmJ4VlFDVWhIQmY5S3VxNjIvdXRDZE5Jc0FUeVhxUUF4cUdE?=
 =?utf-8?B?dFFKZEkxMU0yYThOREZLMFpGajI0c2JVSEZoTHBVS2psb1AyaXRta3lmemhs?=
 =?utf-8?B?RmMweXd0S2dObHdvVnMyd1V4ckZTQXdFbVJ3Wk15K1VKcnJTR0QzTnNnbHd2?=
 =?utf-8?B?SGpGRWNiekdPbjZpSDluZXZuanNLdmM0Y2loY05OTjB0TmJhenlyc1hFNTJW?=
 =?utf-8?B?MVhxbllwTXh2Qm5sT0hxMzNrRkt1RkRYVnRJT0pRSlZQTHI5RXhxb09PMlVu?=
 =?utf-8?B?ekdCb1QyZW9yWjhidFhDK3ltTWNZN2lobzQzMnR3cG9wQjcrQkROeGY5R2t5?=
 =?utf-8?B?UkRDUzBHN0Y1eDNKcVN0NW5YSFErT2JYV0tXcklzZmFoejdOendNdTB5UkJs?=
 =?utf-8?B?ZDRBN2gyOU1Xc1NLVlhMenMwdUM5QlNHYTliTGd6UmcyTVpKV282eTJ0bG9S?=
 =?utf-8?B?MzByaFBNbE5STGg0dG1aUEROM05WSG0vcDdSdGRkaVdYQkk1VEI2VkwzTEJu?=
 =?utf-8?B?VHczUWtRYXc3NUJRRjRQWU1VNWgvZEVlMXFwbU9vTmZRSnZJUXdNOXQzQzBF?=
 =?utf-8?B?WE1uNDZZMjZFT0s3akY3TmgrTWlKQ1B1TDlTaVdBZDUxZ0VnK2VTQWdMNVFM?=
 =?utf-8?B?TWZNdWI4ckVkMnhHWEs3dXJ2cWRmcy83dEJ1TDEzbi9CN1VEemlKbUtBSjRL?=
 =?utf-8?B?SHMvK0tVQWUvYXRrK1FtWmYreUJsVlUrdlU5TzJ5S1VzZlp2cGhEb3pUY05C?=
 =?utf-8?B?ZklNQXdGU3BvSUhqZjQ0VUhuYmMrTTFzTFoxdTNqN2lDWFNjeUtuNGx6SlM1?=
 =?utf-8?B?Nm1QVWtUNEkyeVNkdm9WTXY3RVZmdzM1VHl6TTFBKzRWc0JUVmc5Z1NLWkZi?=
 =?utf-8?B?YUN1ckJMWDJFVEJFRklHbUpmcEdwK2xlbCtXZHFjOTdud2lqZ3lTdFhFQXNC?=
 =?utf-8?B?cTRtUDAweENjYUhZczZDbGhaSnd1YWxSKzV4R2FDcDdsWHdZM3hoTEl5OUto?=
 =?utf-8?B?QTdpVyttQVJUTzlXdFhUUWtMdUo2S0Q0ZGFWNHVRMHVuVXMvaXBBbUNTOTNS?=
 =?utf-8?B?M2FHVldrMEp6T1hzNUhUTGUwZmhlbGNhb3RSejRSMHNOTk0za0JSY01qZGQ5?=
 =?utf-8?B?NGphWGhUb1FDSkI5V2xaQldGMTlUYTQ5eHV0ZU1xVmVwUUF6bytEbmRnMFh6?=
 =?utf-8?B?WlVhSVJ4YVIyakg5a3V0TzNqbG5nQ3ppZlcyR1BuNWZTL2ZIRUxKclBmeEpE?=
 =?utf-8?B?NVZzdzBhY2pselJGa2R5RUNpSG9DK0dNd0ZRWWtDbmFsZUk4UEpsR2o2YXFF?=
 =?utf-8?B?Ri9lWEZ0SUd1OUY5QnQ1UT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <4C25F2D4F64B514D9301FFB8CA93B3BD@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 80d5b3b3-0194-4077-1b57-08d9824ff708
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 07:17:00.7121
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4OEKDi4qYwCRfJDpoA6K9N29VWLQ5ToHl3C+gnulFZ3Z95RCPeX7qHBVopVU4amkj8iOkhNHb7GHG/RP2yx5rwv15ubd0EF0vogEt5WXpOwZCpmN+WoV0RTSw8ShEx12
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6898
X-Proofpoint-ORIG-GUID: bVoE9D8zBRjNci62zRmOuA_Bul4VlAsB
X-Proofpoint-GUID: bVoE9D8zBRjNci62zRmOuA_Bul4VlAsB
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0
 spamscore=0 lowpriorityscore=0 mlxlogscore=999 impostorscore=0
 adultscore=0 phishscore=0 priorityscore=1501 bulkscore=0 suspectscore=0
 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109280042

DQpPbiAyOC4wOS4yMSAwOTo1OSwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjguMDkuMjEg
MDg6NTYsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDI4LjA5LjIx
IDA5OjQyLCBKYW4gQmV1bGljaCB3cm90ZToNCj4+PiBPbiAyOC4wOS4yMDIxIDA2OjE4LCBTdGVm
YW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+Pj4+IE9uIE1vbiwgMjcgU2VwIDIwMjEsIEp1ZXJnZW4g
R3Jvc3Mgd3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMSAwOTozNSwgT2xla3NhbmRyIEFuZHJ1c2hj
aGVua28gd3JvdGU6DQo+Pj4+Pj4gT24gMjcuMDkuMjEgMTA6MjYsIEphbiBCZXVsaWNoIHdyb3Rl
Og0KPj4+Pj4+PiBPbiAyNy4wOS4yMDIxIDA4OjU4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3
cm90ZToNCj4+Pj4+Pj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJf
YW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiBDdXJyZW50bHkgUENJ
IGJhY2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlvbmFsaXRpZXMgYXQgYSB0aW1lLg0K
Pj4+Pj4+Pj4gVG8gbmFtZSBhIGZldzoNCj4+Pj4+Pj4+IDEuIEl0IGlzIHVzZWQgYXMgYSBkYXRh
YmFzZSBmb3IgYXNzaWduYWJsZSBQQ0kgZGV2aWNlcywgZS5nLiB4bA0KPj4+Pj4+Pj4gwqDCoMKg
wqDCoMKgIHBjaS1hc3NpZ25hYmxlLXthZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQg
bGlzdC4gU28sDQo+Pj4+Pj4+PiB3aGVuZXZlcg0KPj4+Pj4+Pj4gwqDCoMKgwqDCoMKgIHRoZSB0
b29sc3RhY2sgbmVlZHMgdG8ga25vdyB3aGljaCBQQ0kgZGV2aWNlcyBjYW4gYmUgcGFzc2VkIHRo
cm91Z2gNCj4+Pj4+Pj4+IMKgwqDCoMKgwqDCoCBpdCByZWFkcyB0aGF0IGZyb20gdGhlIHJlbGV2
YW50IHN5c2ZzIGVudHJpZXMgb2YgdGhlIHBjaWJhY2suDQo+Pj4+Pj4+PiAyLiBJdCBpcyB1c2Vk
IHRvIGhvbGQgdGhlIHVuYm91bmQgUENJIGRldmljZXMgbGlzdCwgZS5nLiB3aGVuIHBhc3NpbmcN
Cj4+Pj4+Pj4+IMKgwqDCoMKgwqDCoCB0aHJvdWdoIGEgUENJIGRldmljZSBpdCBuZWVkcyB0byBi
ZSB1bmJvdW5kIGZyb20gdGhlIHJlbGV2YW50DQo+Pj4+Pj4+PiBkZXZpY2UNCj4+Pj4+Pj4+IMKg
wqDCoMKgwqDCoCBkcml2ZXIgYW5kIGJvdW5kIHRvIHBjaWJhY2sgKHN0cmljdGx5IHNwZWFraW5n
IGl0IGlzIG5vdCByZXF1aXJlZA0KPj4+Pj4+Pj4gwqDCoMKgwqDCoMKgIHRoYXQgdGhlIGRldmlj
ZSBpcyBib3VuZCB0byBwY2liYWNrLCBidXQgcGNpYmFjayBpcyBhZ2FpbiB1c2VkIGFzIGENCj4+
Pj4+Pj4+IMKgwqDCoMKgwqDCoCBkYXRhYmFzZSBvZiB0aGUgcGFzc2VkIHRocm91Z2ggUENJIGRl
dmljZXMsIHNvIHdlIGNhbiByZS1iaW5kIHRoZQ0KPj4+Pj4+Pj4gwqDCoMKgwqDCoMKgIGRldmlj
ZXMgYmFjayB0byB0aGVpciBvcmlnaW5hbCBkcml2ZXJzIHdoZW4gZ3Vlc3QgZG9tYWluIHNodXRz
DQo+Pj4+Pj4+PiBkb3duKQ0KPj4+Pj4+Pj4gMy4gRGV2aWNlIHJlc2V0IGZvciB0aGUgZGV2aWNl
cyBiZWluZyBwYXNzZWQgdGhyb3VnaA0KPj4+Pj4+Pj4gNC4gUGFyYS12aXJ0dWFsaXNlZCB1c2Ut
Y2FzZXMgc3VwcG9ydA0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IFRoZSBwYXJhLXZpcnR1YWxpc2VkIHBh
cnQgb2YgdGhlIGRyaXZlciBpcyBub3QgYWx3YXlzIG5lZWRlZCBhcyBzb21lDQo+Pj4+Pj4+PiBh
cmNoaXRlY3R1cmVzLCBlLmcuIEFybSBvciB4ODYgUFZIIERvbTAsIGFyZSBub3QgdXNpbmcgYmFj
a2VuZC1mcm9udGVuZA0KPj4+Pj4+Pj4gbW9kZWwgZm9yIFBDSSBkZXZpY2UgcGFzc3Rocm91Z2gu
IEZvciBzdWNoIHVzZS1jYXNlcyBtYWtlIHRoZSB2ZXJ5DQo+Pj4+Pj4+PiBmaXJzdCBzdGVwIGlu
IHNwbGl0dGluZyB0aGUgeGVuLXBjaWJhY2sgZHJpdmVyIGludG8gdHdvIHBhcnRzOiBYZW4NCj4+
Pj4+Pj4+IFBDSSBzdHViIGFuZCBQQ0kgUFYgYmFja2VuZCBkcml2ZXJzLg0KPj4+Pj4+Pj4NCj4+
Pj4+Pj4+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvDQo+Pj4+Pj4+PiA8
b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+Pj4+Pg0KPj4+Pj4+Pj4gLS0t
DQo+Pj4+Pj4+PiBDaGFuZ2VzIHNpbmNlIHYzOg0KPj4+Pj4+Pj4gLSBNb3ZlIENPTkZJR19YRU5f
UENJREVWX1NUVUIgdG8gdGhlIHNlY29uZCBwYXRjaA0KPj4+Pj4+PiBJJ20gYWZyYWlkIHRoaXMg
d2Fzbid0IGZ1bGx5IGRvbmU6DQo+Pj4+Pj4+DQo+Pj4+Pj4+PiAtLS0gYS9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+Pj4+Pj4gKysrIGIvZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2svTWFrZWZpbGUNCj4+Pj4+Pj4+IEBAIC0xLDUgKzEsNiBAQA0KPj4+Pj4+Pj4gwqDCoMKgwqAg
IyBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMA0KPj4+Pj4+Pj4gwqDCoMKgwqAgb2Jq
LSQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCkgKz0geGVuLXBjaWJhY2subw0KPj4+Pj4+Pj4g
K29iai0kKENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4+Pj4+Pj4g
V2hpbGUgYmVuaWduIHdoZW4gYWRkZWQgaGVyZSwgdGhpcyBhZGRpdGlvbiBzdGlsbCBkb2Vzbid0
IHNlZW0gdG8NCj4+Pj4+Pj4gYmVsb25nIGhlcmUuDQo+Pj4+Pj4gTXkgYmFkLiBTbywgaXQgc2Vl
bXMgd2l0aG91dCBDT05GSUdfWEVOX1BDSURFVl9TVFVCIHRoZSBjaGFuZ2Ugc2VlbXMNCj4+Pj4+
Pg0KPj4+Pj4+IHRvIGJlIG5vbi1mdW5jdGlvbmFsLiBXaXRoIENPTkZJR19YRU5fUENJREVWX1NU
VUIgd2UgZmFpbCB0byBidWlsZCBvbiAzMi1iaXQNCj4+Pj4+Pg0KPj4+Pj4+IGFyY2hpdGVjdHVy
ZXMuLi4NCj4+Pj4+Pg0KPj4+Pj4+IFdoYXQgd291bGQgYmUgdGhlIHByZWZlcmVuY2UgaGVyZT8g
U3RlZmFubyBzdWdnZXN0ZWQgdGhhdCB3ZSBzdGlsbCBkZWZpbmUNCj4+Pj4+Pg0KPj4+Pj4+IENP
TkZJR19YRU5fUENJREVWX1NUVUIsIGJ1dCBpbiBkaXNhYmxlZCBzdGF0ZSwgZS5nLiB3ZSBhZGQg
dHJpc3RhdGUgdG8gaXQNCj4+Pj4+Pg0KPj4+Pj4+IGluIHRoZSBzZWNvbmQgcGF0Y2gNCj4+Pj4+
Pg0KPj4+Pj4+IEFub3RoZXIgb3B0aW9uIGlzIGp1c3QgdG8gc3F1YXNoIHRoZSB0d28gcGF0Y2hl
cy4NCj4+Pj4+IFNxdWFzaGluZyB3b3VsZCBiZSBmaW5lIGZvciBtZS4NCj4+Pj4gwqDCoCBJdCBp
cyBmaW5lIGZvciBtZSB0byBzcXVhc2ggdGhlIHR3byBwYXRjaGVzLg0KPj4+Pg0KPj4+PiBCdXQg
aW4gYW55IGNhc2UsIHdvdWxkbid0IGl0IGJlIGJldHRlciB0byBtb2RpZnkgdGhhdCBzcGVjaWZp
YyBjaGFuZ2UgdG86DQo+Pj4+DQo+Pj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNp
YmFjay9NYWtlZmlsZSBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+IGlu
ZGV4IGUyY2IzNzY0NDRhNi4uZTIzYzc1OGI4NWFlIDEwMDY0NA0KPj4+PiAtLS0gYS9kcml2ZXJz
L3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNp
YmFjay9NYWtlZmlsZQ0KPj4+PiBAQCAtMSw2ICsxLDUgQEANCj4+Pj4gwqDCoCAjIFNQRFgtTGlj
ZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wDQo+Pj4+IC1vYmotJChDT05GSUdfWEVOX1BDSURFVl9C
QUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+IC1vYmotJChDT05GSUdfWEVOX1BDSURFVl9T
VFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+ICtvYmotJChDT05GSUdfWEVOX1BDSV9TVFVCKSAr
PSB4ZW4tcGNpYmFjay5vDQo+Pj4gQnV0IHRoYXQgd291bGRuJ3QgYWxsb3cgdGhlIGRyaXZlciB0
byBiZSBhIG1vZHVsZSBhbnltb3JlLCB3b3VsZCBpdD8NCj4+DQo+PiBFeGFjdGx5LiBJIGZvcmdv
dCB0aGF0IHdoZW4gcGxheWluZyB3aXRoIG1vZHVsZS9idWlsdC1pbiBJIHdhcyBub3QgYWJsZQ0K
Pj4NCj4+IHRvIGNvbnRyb2wgdGhhdCBhbnltb3JlIGJlY2F1c2UgQ09ORklHX1hFTl9QQ0lfU1RV
QiB3aWxsIGFsd2F5cyBiZQ0KPj4NCj4+IGluICJ5IiBzdGF0ZSwgdGh1cyBldmVuIGlmIHlvdSBo
YXZlIENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQ9bQ0KPj4NCj4+IHlvdSB3b24ndCBiZSBhYmxl
IHRvIGJ1aWxkIGl0IGFzIG1vZHVsZS4gU28sIEkgd2lsbCBwcm9iYWJseSBwdXQgYSBjb21tZW50
DQo+Pg0KPj4gYWJvdXQgdGhhdCBpbiB0aGUgTWFrZWZpbGUgZXhwbGFpbmluZyB0aGUgbmVlZCBm
b3INCj4+DQo+PiBvYmotJChDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFj
ay5vDQo+PiBvYmotJChDT05GSUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+
DQo+IEluIGNhc2UgdGhlIHJlYWwgc3BsaXQgYmV0d2VlbiBib3RoIHBhcnRzIG9mIHhlbi1wY2li
YWNrIGlzIGRvbmUgdGhpcw0KPiB3aWxsIGJlIG5lZWRlZCBhbnl3YXkuDQoNClllcywgaXQgd2ls
bA0KDQpTbywgSSdsbCBwdXQgYSBjb21tZW50IGluIHRoZSBNYWtlZmlsZToNCg0KIyBOLkIuIFRo
aXMgY2Fubm90IGJlIGV4cHJlc3NlZCB3aXRoIGEgc2luZ2xlIGxpbmUgdXNpbmcgQ09ORklHX1hF
Tl9QQ0lfU1RVQg0KDQojIGFzIGl0IGFsd2F5cyByZW1haW5zIGluICJ5IiBzdGF0ZSwgdGh1cyBw
cmV2ZW50aW5nIHRoZSBkcml2ZXIgdG8gYmUgYnVpbHQgYXMNCg0KIyBhIG1vZHVsZS4NCg0Kb2Jq
LSQoQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORCkgKz0geGVuLXBjaWJhY2subw0Kb2JqLSQoQ09O
RklHX1hFTl9QQ0lERVZfU1RVQikgKz0geGVuLXBjaWJhY2subw0KDQpXaWxsIHRoaXMgYmUgb2sg
b3IgbmVlZHMgc29tZSByZS13b3JkaW5nPw0KDQo+DQo+DQo+IEp1ZXJnZW4NCj4NClRoYW5rIHlv
dSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:17:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197619.350774 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7Ms-0004iy-EU; Tue, 28 Sep 2021 07:17:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197619.350774; Tue, 28 Sep 2021 07:17:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7Ms-0004ir-BU; Tue, 28 Sep 2021 07:17:34 +0000
Received: by outflank-mailman (input) for mailman id 197619;
 Tue, 28 Sep 2021 07:17:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mV7Mr-0004if-E6
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:17:33 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4da8ecb7-fe45-492e-9720-cb973f554718;
 Tue, 28 Sep 2021 07:17:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4da8ecb7-fe45-492e-9720-cb973f554718
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632813452;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=5osQiy25+qlibc4sOvnrOTTHPxbvy7TZ7VPEO2+0X6w=;
  b=NBFoTyG2+CSyRWutm5Z3Sj99CAL1wkI5b3LfFlHbXb9uOm93uyjfmU0R
   UX3wR6d7JThswFMWuGqFwefI2r+GqGG5/uLTipK/Ihx9sseXyOY3UxQLh
   Qj1Z5yOywclhZlNCRBvnnVvw/OBIMif9HEXs7coLAREYD6LQNRY6ndplu
   0=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: oJY0Zz3tbisaqEJK4cSD9EfSY2YZiM60KyMsmlEP1x3M1WWcWJbVc/Q7xj8QPt8iLQQ2Y873lj
 OR4rmPzWLJbstJrR79CpB5UB0d1MXJ4cdAkuDUKD61dh8vUZXolI3EWSysLgursty6UBFbShTp
 r2iUKeu/bHQkhnj8u9NXl8ynyS64VWmrw6ePuw3puEP6IVAPQ1u7cgkmDohtULG21HXhusMaeP
 bvFz+J8xuhCTUVc395glxog24zxuf76dSb6jGRMXs/F7HIItlCmckix7xFGGZWY1ULGu2CWGsY
 ZQJwm7tkKziuv5PJ0OnptVvB
X-SBRS: 5.1
X-MesageID: 53797650
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:CBScM6vH4+WVkmxZjtnXagsuzefnVNRZMUV32f8akzHdYApBsoF/q
 tZmKWqFPKmPamvxLthza4zio05SuZfdydVlSlM9/Cs2Fy8a+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524jhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NplnMOAdl0GZoD3ov08fzYDFgpgeqBBweqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DNYUDunZm3HfBAOwvW5zrSKTW/95Imjw3g6iiGN6DP
 JVINmQyNHwsZTVIJXoQT74nxtyCgyP0WCZS9FevgIQotj27IAtZj+G2bYu9lsaxbcdImkeVo
 ErW8mK/BQsVXPSE0iaM+H+ogu7JnAv4VZgUGbn+8eRl6HWMwkQDBRtQUkG0ydG4lUyWS99ZM
 1YT+Cclse417kPDZsnwWVi0rWCJujYYWsFMCKsq5QeV0K3W7g2FQG8eQVZpatYrqcs3TjwCz
 UKSkpXiAjkHjVGOYSvDrPHO92r0YHVLaz9ZDcMZcecby/C8kL5poUridex6MYKa1djeB3b9x
 Anf+UDSmI4vpcIM0qy6+3XOjDStuoXFQ2YJ2+nHYo62xlknPdH0PuRE/XCeva4Zc9rJJrWUl
 CJc8/Vy+tziGn1keMalbuIXAPmN7uuJPSfQiFpid3XK32/2oCbyFWy8DSsXGauIDirmUWO1C
 KMwkVkIjHO2AJdMRfQsC25WI552pZUM7fy/CpjpgiNmO/CdjjOv8iB0flK31GvwikUqmqxXE
 c7FKpzzUyZAUP47lWHeqwIhPVkDnXtWKYT7H8yT8vha+eDGOC79pUktajNikdzVHIvb+V6Io
 r6zxuOByglFUf2WX8Uk2dR7ELz+FlBiXcqeg5UOLoare1M6cEl8W6656e5wIORNwvUK/tokC
 1ngAye0PnKk3iaZQehLA1g+AI7SsWFX9ylmYnB8YQjyixDOo++Htc8iSnf+RpF/nMQL8BK+Z
 6VtlxyoDqsdRzLZ1S4aaJWh/oVueA7y3VCFPja/YSh5dJllHlSb9tjhdwrp1S8PEivo6pdu/
 +z+jlvWEcgZWgBvLMfKc/bznVm/imcQxbBpVEzSL9gNJEi1qNp2Kzb8h+McKt0XLUmR3SOT0
 gubWE9KpeTEr4Iv3sPOgKSI89WgH+dkRxIIFGjH97emcyLd+zP7k4NHVe+JexHbVX/1p/r+N
 bkEkamkPaRezlhQsod6H7J69o4E5oPi9+1A0wBpPHTXdFD3WLluFWaLgJtUvapXy74H5QbvA
 hCT+sNXMKmiMd/+FAJDPxIsa+mO2K1GmjTW6vhpckz26DUuoeiCWERWeRKNlDZcPP1+N4Z8m
 bUtv8sf6gqejBs2M4nZ0nAIpjrUdnFQAb86spw6AZPwjlt5w15PVpXQFyvq7czdcN5LKEQrf
 meZiaeqa26wHaYen67fzUTw4Nc=
IronPort-HdrOrdr: A9a23:p/Z7C6tyX2AcY/SSfoED0FFf7skDdNV00zEX/kB9WHVpmszxra
 GTddAgpHjJYVcqKRUdcL+7VJVoLUmyyXcx2/h2AV7AZniChILLFvAA0WKK+VSJcEeSygce79
 YDT0EXMqyIMbEQt6bHCWeDfeod/A==
X-IronPort-AV: E=Sophos;i="5.85,328,1624334400"; 
   d="scan'208";a="53797650"
Date: Tue, 28 Sep 2021 08:17:21 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Ian Jackson <iwj@xenproject.org>
CC: <xen-devel@lists.xenproject.org>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] Config.mk: update OVMF to edk2-stable202108
Message-ID: <YVLBgcaFp/CzDgms@perard>
References: <20210831123637.22647-1-anthony.perard@citrix.com>
 <24878.13708.826632.607786@mariner.uk.xensource.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <24878.13708.826632.607786@mariner.uk.xensource.com>

On Tue, Aug 31, 2021 at 02:58:36PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[XEN PATCH] Config.mk: update OVMF to edk2-stable202108"):
> > Update to the latest stable tag.
> 
> Thanks.  I am OK with this but I think we should hold off committing
> it until the XSA fallout has been sorted.

Hopefully, this is sorted now. Time to commit the patch?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:20:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:20:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197628.350785 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7PX-0006CZ-U0; Tue, 28 Sep 2021 07:20:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197628.350785; Tue, 28 Sep 2021 07:20:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7PX-0006CS-Q3; Tue, 28 Sep 2021 07:20:19 +0000
Received: by outflank-mailman (input) for mailman id 197628;
 Tue, 28 Sep 2021 07:20:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV7PW-0006CM-6h
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:20:18 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a8e02a8d-7b2d-4807-82da-acc085ecfa9d;
 Tue, 28 Sep 2021 07:20:16 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id DACAF2232F;
 Tue, 28 Sep 2021 07:20:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9594813ACF;
 Tue, 28 Sep 2021 07:20:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 4uUzIi/CUmFEYgAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 07:20:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8e02a8d-7b2d-4807-82da-acc085ecfa9d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632813615; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=1FKOZwJG6HvIY+z5mULGzJSY89j94N90X87PspFD9nw=;
	b=oYwJQHMTPtMjozqkBMdy3KSlqyldGJp0kq8jbOVDjJ27Rfj9a4vKFB1c8gEXg6lhPwdtud
	pUdsIfEBZgfm2J6/jzlvxa8lsVVZsZmjmAkyBQA55lQzaEbNXm2GAwQfxSGtnhG+ukPNte
	WpiqTANEhKSYfwkT7J6f2718JEtnHfE=
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
Date: Tue, 28 Sep 2021 09:20:14 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="9JPaecaKNSLG4dtwkDnNuLHkoyV401xiQ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--9JPaecaKNSLG4dtwkDnNuLHkoyV401xiQ
Content-Type: multipart/mixed; boundary="lWJdWbU42RmXa51U7mt4OnICpSJdaD3nN";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Message-ID: <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
In-Reply-To: <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>

--lWJdWbU42RmXa51U7mt4OnICpSJdaD3nN
Content-Type: multipart/mixed;
 boundary="------------4E1FD0FE258D97863A978DA5"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------4E1FD0FE258D97863A978DA5
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 09:17, Oleksandr Andrushchenko wrote:
>=20
> On 28.09.21 09:59, Juergen Gross wrote:
>> On 28.09.21 08:56, Oleksandr Andrushchenko wrote:
>>>
>>> On 28.09.21 09:42, Jan Beulich wrote:
>>>> On 28.09.2021 06:18, Stefano Stabellini wrote:
>>>>> On Mon, 27 Sep 2021, Juergen Gross wrote:
>>>>>> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
>>>>>>> On 27.09.21 10:26, Jan Beulich wrote:
>>>>>>>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>>>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com=
>
>>>>>>>>>
>>>>>>>>> Currently PCI backend implements multiple functionalities at a =
time.
>>>>>>>>> To name a few:
>>>>>>>>> 1. It is used as a database for assignable PCI devices, e.g. xl=

>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pci-assignable-{add|remov=
e|list} manipulates that list. So,
>>>>>>>>> whenever
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the toolstack needs to kn=
ow which PCI devices can be passed through
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 it reads that from the re=
levant sysfs entries of the pciback.
>>>>>>>>> 2. It is used to hold the unbound PCI devices list, e.g. when p=
assing
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 through a PCI device it n=
eeds to be unbound from the relevant
>>>>>>>>> device
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 driver and bound to pciba=
ck (strictly speaking it is not required
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 that the device is bound =
to pciback, but pciback is again used as a
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 database of the passed th=
rough PCI devices, so we can re-bind the
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 devices back to their ori=
ginal drivers when guest domain shuts
>>>>>>>>> down)
>>>>>>>>> 3. Device reset for the devices being passed through
>>>>>>>>> 4. Para-virtualised use-cases support
>>>>>>>>>
>>>>>>>>> The para-virtualised part of the driver is not always needed as=
 some
>>>>>>>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-=
frontend
>>>>>>>>> model for PCI device passthrough. For such use-cases make the v=
ery
>>>>>>>>> first step in splitting the xen-pciback driver into two parts: =
Xen
>>>>>>>>> PCI stub and PCI PV backend drivers.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Oleksandr Andrushchenko
>>>>>>>>> <oleksandr_andrushchenko@epam.com>
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> Changes since v3:
>>>>>>>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>>>>>>>> I'm afraid this wasn't fully done:
>>>>>>>>
>>>>>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>>>>>> @@ -1,5 +1,6 @@
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 # SPDX-License-Identifier: GPL-2.0
>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0 obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D=
 xen-pciback.o
>>>>>>>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>>>>> While benign when added here, this addition still doesn't seem t=
o
>>>>>>>> belong here.
>>>>>>> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change se=
ems
>>>>>>>
>>>>>>> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to buil=
d on 32-bit
>>>>>>>
>>>>>>> architectures...
>>>>>>>
>>>>>>> What would be the preference here? Stefano suggested that we stil=
l define
>>>>>>>
>>>>>>> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add trista=
te to it
>>>>>>>
>>>>>>> in the second patch
>>>>>>>
>>>>>>> Another option is just to squash the two patches.
>>>>>> Squashing would be fine for me.
>>>>>  =C2=A0=C2=A0 It is fine for me to squash the two patches.
>>>>>
>>>>> But in any case, wouldn't it be better to modify that specific chan=
ge to:
>>>>>
>>>>> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pci=
back/Makefile
>>>>> index e2cb376444a6..e23c758b85ae 100644
>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>> @@ -1,6 +1,5 @@
>>>>>  =C2=A0=C2=A0 # SPDX-License-Identifier: GPL-2.0
>>>>> -obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>>>> -obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>> +obj-$(CONFIG_XEN_PCI_STUB) +=3D xen-pciback.o
>>>> But that wouldn't allow the driver to be a module anymore, would it?=

>>>
>>> Exactly. I forgot that when playing with module/built-in I was not ab=
le
>>>
>>> to control that anymore because CONFIG_XEN_PCI_STUB will always be
>>>
>>> in "y" state, thus even if you have CONFIG_XEN_PCIDEV_BACKEND=3Dm
>>>
>>> you won't be able to build it as module. So, I will probably put a co=
mment
>>>
>>> about that in the Makefile explaining the need for
>>>
>>> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>
>> In case the real split between both parts of xen-pciback is done this
>> will be needed anyway.
>=20
> Yes, it will
>=20
> So, I'll put a comment in the Makefile:
>=20
> # N.B. This cannot be expressed with a single line using CONFIG_XEN_PCI=
_STUB
>=20
> # as it always remains in "y" state, thus preventing the driver to be b=
uilt as
>=20
> # a module.
>=20
> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>=20
> Will this be ok or needs some re-wording?

I'd add that CONFIG_XEN_PCIDEV_BACKEND and CONFIG_XEN_PCIDEV_STUB are
mutually exclusive.


Juergen

--------------4E1FD0FE258D97863A978DA5
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------4E1FD0FE258D97863A978DA5--

--lWJdWbU42RmXa51U7mt4OnICpSJdaD3nN--

--9JPaecaKNSLG4dtwkDnNuLHkoyV401xiQ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFSwi4FAwAAAAAACgkQsN6d1ii/Ey+y
2Af9Gg7mt7iIcCHwDFkUPwaZYPLL2M/CgOZ9V8rsGKHxkAcS1dseTfKJ7MVhpV+AjxKvHXoSbN3e
1I+7/csF61/cMx+0u7UzPAhH74reMjgdv5Rhui6uidt05z2NB9AgKtXBrlL/HYKST3oRCpHyBAvs
J1bJKZphohkn5gG5WpXjKUpy8RzCzgNIyJ8fqMLAjsZHrtJU20dKPxmsACUkZGeVNmRkqV8LHPax
1ndZfVaArihLCpLSn5rblXCwnl7KaYQcCKMHF5uL0pIhsK3xmq2zWPrIlGEcyEZiv+oNUfVqBtPs
6sJ9tEYaN87Fb6vAIV9z92GZrxGQ7nkERZoqGTW9Uw==
=AoUA
-----END PGP SIGNATURE-----

--9JPaecaKNSLG4dtwkDnNuLHkoyV401xiQ--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:20:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:20:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197634.350796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7Q6-0006kj-5f; Tue, 28 Sep 2021 07:20:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197634.350796; Tue, 28 Sep 2021 07:20:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7Q6-0006kc-2d; Tue, 28 Sep 2021 07:20:54 +0000
Received: by outflank-mailman (input) for mailman id 197634;
 Tue, 28 Sep 2021 07:20:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mV7Q4-0006k7-Rg
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:20:52 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7da32dc4-e5c0-46c4-b990-a55eae9bf766;
 Tue, 28 Sep 2021 07:20:51 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 16EFF61052;
 Tue, 28 Sep 2021 07:20:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7da32dc4-e5c0-46c4-b990-a55eae9bf766
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632813651;
	bh=3/VYh1cvGM54Pp4563OgiGeaUEdkG80oWPlxS7UbG1g=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=KyFqPbfhRT40FZ4UGGI7ti7VxzCcWo1IXnC38NU7cP77hd1NiULpyDfTQ+ZjMOYW/
	 sMKQtIGbBb0GAd0iH0rhoZ8smzrw1iA0KxajEdbGXMltPYQqLJ5cav9iYkAbMqELZq
	 InJaJcuOKdO9WcokGEOAhgg7ETfm+SsbcFky1lHZG3UbK8hXUlBnTlC2SwMoMJ4aZq
	 wLfVmZ/0XpdkGtqISy7Ooscg77wMy82sIo041bnL413ES6KbUlAM70FoLRVp8amJGH
	 Un5seG2vdL311sHZQs3MJCnNS5XthXeRkfjXSedJ/N98Mv5fQm6Lx4zbgNVG8AUonk
	 wKkx5PlqCDqyg==
Date: Tue, 28 Sep 2021 00:20:50 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>, 
    "julien@xen.org" <julien@xen.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
In-Reply-To: <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
Message-ID: <alpine.DEB.2.21.2109280020150.5022@sstabellini-ThinkPad-T480s>
References: <20210927065822.350973-1-andr2000@gmail.com> <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com> <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com> <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com> <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com> <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com> <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com> <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-2095034342-1632813651=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-2095034342-1632813651=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 28 Sep 2021, Oleksandr Andrushchenko wrote:
> On 28.09.21 09:59, Juergen Gross wrote:
> > On 28.09.21 08:56, Oleksandr Andrushchenko wrote:
> >>
> >> On 28.09.21 09:42, Jan Beulich wrote:
> >>> On 28.09.2021 06:18, Stefano Stabellini wrote:
> >>>> On Mon, 27 Sep 2021, Juergen Gross wrote:
> >>>>> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
> >>>>>> On 27.09.21 10:26, Jan Beulich wrote:
> >>>>>>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
> >>>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>>>>>>>
> >>>>>>>> Currently PCI backend implements multiple functionalities at a time.
> >>>>>>>> To name a few:
> >>>>>>>> 1. It is used as a database for assignable PCI devices, e.g. xl
> >>>>>>>>        pci-assignable-{add|remove|list} manipulates that list. So,
> >>>>>>>> whenever
> >>>>>>>>        the toolstack needs to know which PCI devices can be passed through
> >>>>>>>>        it reads that from the relevant sysfs entries of the pciback.
> >>>>>>>> 2. It is used to hold the unbound PCI devices list, e.g. when passing
> >>>>>>>>        through a PCI device it needs to be unbound from the relevant
> >>>>>>>> device
> >>>>>>>>        driver and bound to pciback (strictly speaking it is not required
> >>>>>>>>        that the device is bound to pciback, but pciback is again used as a
> >>>>>>>>        database of the passed through PCI devices, so we can re-bind the
> >>>>>>>>        devices back to their original drivers when guest domain shuts
> >>>>>>>> down)
> >>>>>>>> 3. Device reset for the devices being passed through
> >>>>>>>> 4. Para-virtualised use-cases support
> >>>>>>>>
> >>>>>>>> The para-virtualised part of the driver is not always needed as some
> >>>>>>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
> >>>>>>>> model for PCI device passthrough. For such use-cases make the very
> >>>>>>>> first step in splitting the xen-pciback driver into two parts: Xen
> >>>>>>>> PCI stub and PCI PV backend drivers.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Oleksandr Andrushchenko
> >>>>>>>> <oleksandr_andrushchenko@epam.com>
> >>>>>>>>
> >>>>>>>> ---
> >>>>>>>> Changes since v3:
> >>>>>>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
> >>>>>>> I'm afraid this wasn't fully done:
> >>>>>>>
> >>>>>>>> --- a/drivers/xen/xen-pciback/Makefile
> >>>>>>>> +++ b/drivers/xen/xen-pciback/Makefile
> >>>>>>>> @@ -1,5 +1,6 @@
> >>>>>>>>      # SPDX-License-Identifier: GPL-2.0
> >>>>>>>>      obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> >>>>>>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> >>>>>>> While benign when added here, this addition still doesn't seem to
> >>>>>>> belong here.
> >>>>>> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change seems
> >>>>>>
> >>>>>> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to build on 32-bit
> >>>>>>
> >>>>>> architectures...
> >>>>>>
> >>>>>> What would be the preference here? Stefano suggested that we still define
> >>>>>>
> >>>>>> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tristate to it
> >>>>>>
> >>>>>> in the second patch
> >>>>>>
> >>>>>> Another option is just to squash the two patches.
> >>>>> Squashing would be fine for me.
> >>>>    It is fine for me to squash the two patches.
> >>>>
> >>>> But in any case, wouldn't it be better to modify that specific change to:
> >>>>
> >>>> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
> >>>> index e2cb376444a6..e23c758b85ae 100644
> >>>> --- a/drivers/xen/xen-pciback/Makefile
> >>>> +++ b/drivers/xen/xen-pciback/Makefile
> >>>> @@ -1,6 +1,5 @@
> >>>>    # SPDX-License-Identifier: GPL-2.0
> >>>> -obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> >>>> -obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> >>>> +obj-$(CONFIG_XEN_PCI_STUB) += xen-pciback.o
> >>> But that wouldn't allow the driver to be a module anymore, would it?
> >>
> >> Exactly. I forgot that when playing with module/built-in I was not able
> >>
> >> to control that anymore because CONFIG_XEN_PCI_STUB will always be
> >>
> >> in "y" state, thus even if you have CONFIG_XEN_PCIDEV_BACKEND=m
> >>
> >> you won't be able to build it as module. So, I will probably put a comment
> >>
> >> about that in the Makefile explaining the need for
> >>
> >> obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> >> obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> >
> > In case the real split between both parts of xen-pciback is done this
> > will be needed anyway.
> 
> Yes, it will
> 
> So, I'll put a comment in the Makefile:
> 
> # N.B. This cannot be expressed with a single line using CONFIG_XEN_PCI_STUB
> 
> # as it always remains in "y" state, thus preventing the driver to be built as
> 
> # a module.
> 
> obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
> obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
> 
> Will this be ok or needs some re-wording?

I am fine with it and honestly that was the only comment I had so you
can add my

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
--8323329-2095034342-1632813651=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:24:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:24:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197640.350807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7TX-0007Rb-MN; Tue, 28 Sep 2021 07:24:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197640.350807; Tue, 28 Sep 2021 07:24:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7TX-0007RU-Iv; Tue, 28 Sep 2021 07:24:27 +0000
Received: by outflank-mailman (input) for mailman id 197640;
 Tue, 28 Sep 2021 07:24:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV7TV-0007RO-S2
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:24:25 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c0acda29-5a6c-473c-a736-f6c46ef65ba6;
 Tue, 28 Sep 2021 07:24:24 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S7K45F007122; 
 Tue, 28 Sep 2021 07:24:23 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbxjhr0fa-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 07:24:23 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4753.eurprd03.prod.outlook.com (2603:10a6:208:cf::32)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep
 2021 07:24:20 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 07:24:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0acda29-5a6c-473c-a736-f6c46ef65ba6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gzE9eBe/B77iZa4XI3mqUJCRDTUrn8zaL5J9jxr7ZUFJ6BNLp+JYjRN2QBomEIFqY9/cISqmaPrVPOpZ5QxvgW20XpOU/mZhNp2XaQFIEhB6va9+Q2AA49qNE8m1lJa9+sZkcGh1uRlZMIXnTK89heMl7xwl9f7hIf7Md9x7KRKTedKxfG6czifmp1puJLa85hdk2xprR5Jds5kUm6iHE43bBynAVnCy12+FgQOISNZJdYwGXFt28InyfExPIN/mUWVSKgdfWKh3P11v0/KaasNy9JSGCbEqzDaJup54229jHotULk0MsXaPPj5AYf9k7q5VJkUuy+1RcZpuNpnscQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=mAxXwc/lmDEJhB8z+TYsCvxH9s7PYkzE+3n7odRNMe8=;
 b=FY1qcC7KV66Ofc9ws+5Q3IMR2FPPZawxFGlJkKwMq4e+OTRGepmiC/krym9B9ifAcTfBT9x7LiH9BO9s7B6rwc3cqbNE4GyCnEBFf49cMkaJGHAviUO8uQ9++1lFHk/zKsQp6aEc349JoSe2MyfeV3DGbXU26dZB69DX23BxKuxx9uYnMhVrT8brCk2Hq4lagNrFalAiRU+hl+FQ0XSU2NNKEDmr5mj7A/P51ck7m7v/d1YBlsBD0zK6EEpFSf2h5MNOafPub6bjrbFShrMl2rzYQTFCeZnUDXliUm3aXg3cok05xFxDXywu+6aOUQaSv8J+cDgKJyZGiVJt1pgYGw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=mAxXwc/lmDEJhB8z+TYsCvxH9s7PYkzE+3n7odRNMe8=;
 b=nO1l5S6yYAQu2MOgOr0xoxfhla7QUcS4TEwRY84wOVKGpY6AR3J07xxh+Ae0QED+wQnCdy/xdtKY2GdpMFXoUeR+D7Wl8ikcNIS8LayUh3AAiU5amBXDTj4FF0xYLc0WHVMromxQInmt1lE9q1MX4BwVSVtkAxH7HmbqkWX2H6xXnGOYzcXj1Rd3d1TzvQ5+Bc5QpoDkKIYbQL6i95GROMXNYQjlp5BCTeMLrhhb8S879riKBV4eS3EksioukWjgvkrluD2gsIUA1QkAWXRZ3wPtyHUboENhFjenAYHTMqrA/0XJ60hgsobtOhY4qz5pRKtcrJbgJlZTgmezOUH9yA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
        Stefano
 Stabellini <sstabellini@kernel.org>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: 
 AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbACAAAVGAIABVeSAgAAoPoCAAAP7gIAAAMWAgAAFAACAAADnAIAAASSA
Date: Tue, 28 Sep 2021 07:24:20 +0000
Message-ID: <b835b135-1113-2762-459a-b6e31053336a@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
 <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
In-Reply-To: <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 35720c18-cbc3-4583-75dc-08d98250fcea
x-ms-traffictypediagnostic: AM0PR03MB4753:
x-microsoft-antispam-prvs: 
 <AM0PR03MB4753BB921DCD325EEA801D1EE7A89@AM0PR03MB4753.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 cJ4fVsW5yWGUAQnDuQIYkLqDEh2p2deQ7Avtk2aFNohrevOfeQJBFBIZil6G2jbZFwsMwHg5sqn/jTXFS40/67FyEiLCQprccbMt5H6Ncl183AoE5nB/O873olRveli+lWu1bOPOru+yWjPNwM3xontvRULSwuiSA4O+n9oTSsS9LsQVpRTTknp/FveIJ6Q2WAI5ovhMa/Y2RYk28itu0TLP8Ei38tIuJOlSL5iG5MKazvDtcJWJhnylxx6AtH1A6SLQ4Ld49pT7SSVNHn+Ic3sarPf6YACQZg1xEh+PAeyrA4CNwXL1MtpAu2MT8xFcSnt3BNI2fXhec+HNvskq2SxlfXKtcR8NBHdXJP0mHSiTOEdrjIkTiBEYscbE/aKq4Or2lb+Hl80X/3wjmWZ16a/NFysZslGp0IFei2rPVf3uBJxbs3E3yLqsq1rRRYOlnIpQiIzOtVNrAV4x9ZXrlehn37cm1VQemddTHB9eol9N5nXEVeomtkurRcdvwyy6pISepupU8yE+dYRzzvIDUXr7t9AF4rsnJcWmZEjE7ytXrxrByMjmVORFwzBh0yQXrzL2yJ5nF6xPgqHBFIhGm+G+95KHsIKTJiQ7+UDhPoZsOvUv+l3f2dx7siE3/E6oMvu+03sV9VVfGuEezFi+O/cuGcdY+yDCC2rSkal6Whq+oSMGueiyOfkGcojC7lEeCPm0G4/Jp6i8RjVvf57YiTuUoJHPDaVmBCql4UdDQLRSurzAd78nmbl4u2cDkyWL
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(186003)(5660300002)(6512007)(122000001)(55236004)(26005)(71200400001)(6486002)(4326008)(2616005)(2906002)(6506007)(316002)(38100700002)(38070700005)(53546011)(83380400001)(54906003)(110136005)(86362001)(8936002)(36756003)(31686004)(66946007)(76116006)(66446008)(64756008)(66556008)(66476007)(91956017)(508600001)(31696002)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?YW9CWjgxR25RR0d3eEVzSFA2a28yZHlza0FydnhjVzlUZnpkUE9NY0pDMFZu?=
 =?utf-8?B?L0psd3gwdFp4WHB1WkVqNXJqUVZtci82VzNYZFU3SzZHZnJyb0hDdDVGcGww?=
 =?utf-8?B?c2hlbTcyOHdOem95eERka3BTcTJzaTh2Mi9pTmt4WjhDY2hoVXlET3dlSkdJ?=
 =?utf-8?B?blZZVVNLSmRqZXhNNnNHMkdMTEQvQUpRL2ZXZ0FxZVBOTGlDaHBuMlZrTlVO?=
 =?utf-8?B?aHYweTJPV1BlVnh6MlhPc2MrMFpYd0JrTmVvbjJoRWxsa0dDcTNNa2l2WGxN?=
 =?utf-8?B?aVVsdVZ1anI1dnVqbGp4QjV5Njdxa3UreDNYVHdyQTdPazhzLzhZc252K0Yv?=
 =?utf-8?B?MGVTU2lCMEFjNmdMalhwMmhad3VkY2VoQ0VOUFhLTTlVemFHeGF0bHdTV0ht?=
 =?utf-8?B?RTh3TExZbjhLREt2ZHBTQWhRM3AxQjFTcXVyME1WUlJMQWFISlEraTBGdkRE?=
 =?utf-8?B?dHFROHBFRy9WMDlLNE5Ncm5SL0NyVVVGdkJvNTBIT3ViZ0xMNCs3ZFBSOXBw?=
 =?utf-8?B?WVZOZ2oyWjRqdVNxdjY5UkJNWEN6WHFDMzBJN1hOYUdrOGo1THFRZXp4NUIy?=
 =?utf-8?B?cktjUlpHdzFFcjVZcVRyMkQ3ZWxCTldFU2hVUlZmQW1QdnNHOEZHQVRKSE1t?=
 =?utf-8?B?WVNHQjVHTWhFVGpNL3A0T2EwNCtoaVFSekMvZUFMQk5ka3R3dzhKa2pZd2FC?=
 =?utf-8?B?TjlDeVVvK01rSFJ5Nm9TSWxJUm5pd3k0dHZHY0Via012dEpqOGg5SFoyUHNz?=
 =?utf-8?B?eUpXMzRMc3NlMmhWeEJJNHZCd1BjY1BpVlFYNTdGWm8vcnl2RjhxTnBSeGly?=
 =?utf-8?B?Um1MS053cW1NWTlDN0loWWRWTUFLT041c2tjN0ZoTlNGR3RYSldyaVBLclZ5?=
 =?utf-8?B?WGlOSDVFMDA4K0Z4Vk54SCt1YTd3dXNKbDhYL0lpL3ZGV0o5Nm9xZi9tZFRw?=
 =?utf-8?B?d1UwRzJLU2o5TU9RZEdpOElJZHpvYWxWK3IzSDRjVnVNZUhMVG12ZzlzMHEv?=
 =?utf-8?B?d1pHTjZlaWEzOVJYSzZSbG9BcjlVeWN6VjZES2FBa3ZGeGpObnc3czJIZ2F1?=
 =?utf-8?B?RHRId0F6SURjYlNmY2tnT0k1M3JQbFJLSDVzSll6ODJEeHRWUWlOMk9PVG1R?=
 =?utf-8?B?Ukt3dXhUQ2pmbncvWWZaN1lRUnE5R09UNTlZNEs3blNRSjR0c3I0WTZJNjBo?=
 =?utf-8?B?TmhsME9ZUHR1TSsremFUZWFKd05MVjRISlh6enBTbkVxUGFZN0xtL2NiRVFR?=
 =?utf-8?B?U0tmUDBDNVZhWmNId2F1NitxaFZISXMrRnVFMThhdVV5Uld2R1EzSFh6WUpU?=
 =?utf-8?B?RVhNU05rUUE5MEtPeE5sQkNFYW5nWnZ2ZldCWlBrNFNpWlRUR2JMMU4zVTZ3?=
 =?utf-8?B?RnRpRUVoZDFpNWlUbFVva241dm1yak04cUt4U3lvdDBTN0V2NjBnYmxTSld0?=
 =?utf-8?B?QmVRSXFOWlo3Qkl1SXkyajNMVHBheGVTR0JFTXp3OVZSWUdSYVF2ajdjRnE4?=
 =?utf-8?B?cU0yRGRkOEFTd2hlMXBaM0djd0lCcDRGWWxPcVBOZGNBQVcyVldaSzBBTzNa?=
 =?utf-8?B?bVJ0cjR6anJPYzhLN1ZpMHZ4aXkzMkI4SlVFNVZxTjdkeU9iUDBJejI4bFBX?=
 =?utf-8?B?dHlrWHRZL292RXNzNXY2YkhmNlZGRXI1ZFRtRkJkQVNGYWhCTHB3S1ZPOEU0?=
 =?utf-8?B?dnh6RER6Zk4yeHUzZnZGbVhSeCt2a0p0TlltNEd5Ylo1eitVQkNkNStnRkt3?=
 =?utf-8?B?QlFybk1paWV2bVJwSE5NQUxBdG9pUFVvQVY5MUVtSVdjdDZjRXgxbXRRYW9i?=
 =?utf-8?B?WTluYnFkVDhyUmpIWGJuZz09?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <45E2F3D65AC7AC4DB51E3898B43EAD75@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 35720c18-cbc3-4583-75dc-08d98250fcea
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 07:24:20.0420
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Qwdfh/VQNcZ9nfnjwC1ihm8v0E0/aBRlKTYZCNbxzTg3MiVbnPO03dcEJDDL9+4Nn+Il695ONbKE4TrBjYuMdrMw/DNOFBoV7Iu8Gb5P/B5EXBcL+3IQQSv8XdROmFRw
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4753
X-Proofpoint-GUID: OMB24DAL9WbY340xhbfeUwvvIfUOi5lF
X-Proofpoint-ORIG-GUID: OMB24DAL9WbY340xhbfeUwvvIfUOi5lF
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 malwarescore=0
 mlxlogscore=999 adultscore=0 phishscore=0 clxscore=1015 priorityscore=1501
 impostorscore=0 suspectscore=0 mlxscore=0 spamscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280042

DQpPbiAyOC4wOS4yMSAxMDoyMCwgSnVlcmdlbiBHcm9zcyB3cm90ZToNCj4gT24gMjguMDkuMjEg
MDk6MTcsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4NCj4+IE9uIDI4LjA5LjIx
IDA5OjU5LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDI4LjA5LjIxIDA4OjU2LCBPbGVr
c2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4NCj4+Pj4gT24gMjguMDkuMjEgMDk6NDIs
IEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+Pj4gT24gMjguMDkuMjAyMSAwNjoxOCwgU3RlZmFubyBT
dGFiZWxsaW5pIHdyb3RlOg0KPj4+Pj4+IE9uIE1vbiwgMjcgU2VwIDIwMjEsIEp1ZXJnZW4gR3Jv
c3Mgd3JvdGU6DQo+Pj4+Pj4+IE9uIDI3LjA5LjIxIDA5OjM1LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+Pj4+IE9uIDI3LjA5LjIxIDEwOjI2LCBKYW4gQmV1bGljaCB3cm90
ZToNCj4+Pj4+Pj4+PiBPbiAyNy4wOS4yMDIxIDA4OjU4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byB3cm90ZToNCj4+Pj4+Pj4+Pj4gRnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtz
YW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBDdXJy
ZW50bHkgUENJIGJhY2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlvbmFsaXRpZXMgYXQg
YSB0aW1lLg0KPj4+Pj4+Pj4+PiBUbyBuYW1lIGEgZmV3Og0KPj4+Pj4+Pj4+PiAxLiBJdCBpcyB1
c2VkIGFzIGEgZGF0YWJhc2UgZm9yIGFzc2lnbmFibGUgUENJIGRldmljZXMsIGUuZy4geGwNCj4+
Pj4+Pj4+Pj4gwqDCoMKgwqDCoMKgwqAgcGNpLWFzc2lnbmFibGUte2FkZHxyZW1vdmV8bGlzdH0g
bWFuaXB1bGF0ZXMgdGhhdCBsaXN0LiBTbywNCj4+Pj4+Pj4+Pj4gd2hlbmV2ZXINCj4+Pj4+Pj4+
Pj4gwqDCoMKgwqDCoMKgwqAgdGhlIHRvb2xzdGFjayBuZWVkcyB0byBrbm93IHdoaWNoIFBDSSBk
ZXZpY2VzIGNhbiBiZSBwYXNzZWQgdGhyb3VnaA0KPj4+Pj4+Pj4+PiDCoMKgwqDCoMKgwqDCoCBp
dCByZWFkcyB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVudHJpZXMgb2YgdGhlIHBjaWJh
Y2suDQo+Pj4+Pj4+Pj4+IDIuIEl0IGlzIHVzZWQgdG8gaG9sZCB0aGUgdW5ib3VuZCBQQ0kgZGV2
aWNlcyBsaXN0LCBlLmcuIHdoZW4gcGFzc2luZw0KPj4+Pj4+Pj4+PiDCoMKgwqDCoMKgwqDCoCB0
aHJvdWdoIGEgUENJIGRldmljZSBpdCBuZWVkcyB0byBiZSB1bmJvdW5kIGZyb20gdGhlIHJlbGV2
YW50DQo+Pj4+Pj4+Pj4+IGRldmljZQ0KPj4+Pj4+Pj4+PiDCoMKgwqDCoMKgwqDCoCBkcml2ZXIg
YW5kIGJvdW5kIHRvIHBjaWJhY2sgKHN0cmljdGx5IHNwZWFraW5nIGl0IGlzIG5vdCByZXF1aXJl
ZA0KPj4+Pj4+Pj4+PiDCoMKgwqDCoMKgwqDCoCB0aGF0IHRoZSBkZXZpY2UgaXMgYm91bmQgdG8g
cGNpYmFjaywgYnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBhcyBhDQo+Pj4+Pj4+Pj4+IMKgwqDC
oMKgwqDCoMKgIGRhdGFiYXNlIG9mIHRoZSBwYXNzZWQgdGhyb3VnaCBQQ0kgZGV2aWNlcywgc28g
d2UgY2FuIHJlLWJpbmQgdGhlDQo+Pj4+Pj4+Pj4+IMKgwqDCoMKgwqDCoMKgIGRldmljZXMgYmFj
ayB0byB0aGVpciBvcmlnaW5hbCBkcml2ZXJzIHdoZW4gZ3Vlc3QgZG9tYWluIHNodXRzDQo+Pj4+
Pj4+Pj4+IGRvd24pDQo+Pj4+Pj4+Pj4+IDMuIERldmljZSByZXNldCBmb3IgdGhlIGRldmljZXMg
YmVpbmcgcGFzc2VkIHRocm91Z2gNCj4+Pj4+Pj4+Pj4gNC4gUGFyYS12aXJ0dWFsaXNlZCB1c2Ut
Y2FzZXMgc3VwcG9ydA0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBUaGUgcGFyYS12aXJ0dWFsaXNl
ZCBwYXJ0IG9mIHRoZSBkcml2ZXIgaXMgbm90IGFsd2F5cyBuZWVkZWQgYXMgc29tZQ0KPj4+Pj4+
Pj4+PiBhcmNoaXRlY3R1cmVzLCBlLmcuIEFybSBvciB4ODYgUFZIIERvbTAsIGFyZSBub3QgdXNp
bmcgYmFja2VuZC1mcm9udGVuZA0KPj4+Pj4+Pj4+PiBtb2RlbCBmb3IgUENJIGRldmljZSBwYXNz
dGhyb3VnaC4gRm9yIHN1Y2ggdXNlLWNhc2VzIG1ha2UgdGhlIHZlcnkNCj4+Pj4+Pj4+Pj4gZmly
c3Qgc3RlcCBpbiBzcGxpdHRpbmcgdGhlIHhlbi1wY2liYWNrIGRyaXZlciBpbnRvIHR3byBwYXJ0
czogWGVuDQo+Pj4+Pj4+Pj4+IFBDSSBzdHViIGFuZCBQQ0kgUFYgYmFja2VuZCBkcml2ZXJzLg0K
Pj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbw0KPj4+Pj4+Pj4+PiA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pj4+
Pj4+Pj4+DQo+Pj4+Pj4+Pj4+IC0tLQ0KPj4+Pj4+Pj4+PiBDaGFuZ2VzIHNpbmNlIHYzOg0KPj4+
Pj4+Pj4+PiAtIE1vdmUgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiB0byB0aGUgc2Vjb25kIHBhdGNo
DQo+Pj4+Pj4+Pj4gSSdtIGFmcmFpZCB0aGlzIHdhc24ndCBmdWxseSBkb25lOg0KPj4+Pj4+Pj4+
DQo+Pj4+Pj4+Pj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+
Pj4+Pj4+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+Pj4+Pj4+
IEBAIC0xLDUgKzEsNiBAQA0KPj4+Pj4+Pj4+PiDCoMKgwqDCoMKgICMgU1BEWC1MaWNlbnNlLUlk
ZW50aWZpZXI6IEdQTC0yLjANCj4+Pj4+Pj4+Pj4gwqDCoMKgwqDCoCBvYmotJChDT05GSUdfWEVO
X1BDSURFVl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+Pj4+Pj4+ICtvYmotJChDT05G
SUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+Pj4+Pj4gV2hpbGUgYmVu
aWduIHdoZW4gYWRkZWQgaGVyZSwgdGhpcyBhZGRpdGlvbiBzdGlsbCBkb2Vzbid0IHNlZW0gdG8N
Cj4+Pj4+Pj4+PiBiZWxvbmcgaGVyZS4NCj4+Pj4+Pj4+IE15IGJhZC4gU28sIGl0IHNlZW1zIHdp
dGhvdXQgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiB0aGUgY2hhbmdlIHNlZW1zDQo+Pj4+Pj4+Pg0K
Pj4+Pj4+Pj4gdG8gYmUgbm9uLWZ1bmN0aW9uYWwuIFdpdGggQ09ORklHX1hFTl9QQ0lERVZfU1RV
QiB3ZSBmYWlsIHRvIGJ1aWxkIG9uIDMyLWJpdA0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IGFyY2hpdGVj
dHVyZXMuLi4NCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiBXaGF0IHdvdWxkIGJlIHRoZSBwcmVmZXJlbmNl
IGhlcmU/IFN0ZWZhbm8gc3VnZ2VzdGVkIHRoYXQgd2Ugc3RpbGwgZGVmaW5lDQo+Pj4+Pj4+Pg0K
Pj4+Pj4+Pj4gQ09ORklHX1hFTl9QQ0lERVZfU1RVQiwgYnV0IGluIGRpc2FibGVkIHN0YXRlLCBl
LmcuIHdlIGFkZCB0cmlzdGF0ZSB0byBpdA0KPj4+Pj4+Pj4NCj4+Pj4+Pj4+IGluIHRoZSBzZWNv
bmQgcGF0Y2gNCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiBBbm90aGVyIG9wdGlvbiBpcyBqdXN0IHRvIHNx
dWFzaCB0aGUgdHdvIHBhdGNoZXMuDQo+Pj4+Pj4+IFNxdWFzaGluZyB3b3VsZCBiZSBmaW5lIGZv
ciBtZS4NCj4+Pj4+PiDCoMKgwqAgSXQgaXMgZmluZSBmb3IgbWUgdG8gc3F1YXNoIHRoZSB0d28g
cGF0Y2hlcy4NCj4+Pj4+Pg0KPj4+Pj4+IEJ1dCBpbiBhbnkgY2FzZSwgd291bGRuJ3QgaXQgYmUg
YmV0dGVyIHRvIG1vZGlmeSB0aGF0IHNwZWNpZmljIGNoYW5nZSB0bzoNCj4+Pj4+Pg0KPj4+Pj4+
IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZSBiL2RyaXZlcnMv
eGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+Pj4gaW5kZXggZTJjYjM3NjQ0NGE2Li5lMjNj
NzU4Yjg1YWUgMTAwNjQ0DQo+Pj4+Pj4gLS0tIGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFr
ZWZpbGUNCj4+Pj4+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+
Pj4+IEBAIC0xLDYgKzEsNSBAQA0KPj4+Pj4+IMKgwqDCoCAjIFNQRFgtTGljZW5zZS1JZGVudGlm
aWVyOiBHUEwtMi4wDQo+Pj4+Pj4gLW9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9
IHhlbi1wY2liYWNrLm8NCj4+Pj4+PiAtb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfU1RVQikgKz0g
eGVuLXBjaWJhY2subw0KPj4+Pj4+ICtvYmotJChDT05GSUdfWEVOX1BDSV9TVFVCKSArPSB4ZW4t
cGNpYmFjay5vDQo+Pj4+PiBCdXQgdGhhdCB3b3VsZG4ndCBhbGxvdyB0aGUgZHJpdmVyIHRvIGJl
IGEgbW9kdWxlIGFueW1vcmUsIHdvdWxkIGl0Pw0KPj4+Pg0KPj4+PiBFeGFjdGx5LiBJIGZvcmdv
dCB0aGF0IHdoZW4gcGxheWluZyB3aXRoIG1vZHVsZS9idWlsdC1pbiBJIHdhcyBub3QgYWJsZQ0K
Pj4+Pg0KPj4+PiB0byBjb250cm9sIHRoYXQgYW55bW9yZSBiZWNhdXNlIENPTkZJR19YRU5fUENJ
X1NUVUIgd2lsbCBhbHdheXMgYmUNCj4+Pj4NCj4+Pj4gaW4gInkiIHN0YXRlLCB0aHVzIGV2ZW4g
aWYgeW91IGhhdmUgQ09ORklHX1hFTl9QQ0lERVZfQkFDS0VORD1tDQo+Pj4+DQo+Pj4+IHlvdSB3
b24ndCBiZSBhYmxlIHRvIGJ1aWxkIGl0IGFzIG1vZHVsZS4gU28sIEkgd2lsbCBwcm9iYWJseSBw
dXQgYSBjb21tZW50DQo+Pj4+DQo+Pj4+IGFib3V0IHRoYXQgaW4gdGhlIE1ha2VmaWxlIGV4cGxh
aW5pbmcgdGhlIG5lZWQgZm9yDQo+Pj4+DQo+Pj4+IG9iai0kKENPTkZJR19YRU5fUENJREVWX0JB
Q0tFTkQpICs9IHhlbi1wY2liYWNrLm8NCj4+Pj4gb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfU1RV
QikgKz0geGVuLXBjaWJhY2subw0KPj4+DQo+Pj4gSW4gY2FzZSB0aGUgcmVhbCBzcGxpdCBiZXR3
ZWVuIGJvdGggcGFydHMgb2YgeGVuLXBjaWJhY2sgaXMgZG9uZSB0aGlzDQo+Pj4gd2lsbCBiZSBu
ZWVkZWQgYW55d2F5Lg0KPj4NCj4+IFllcywgaXQgd2lsbA0KPj4NCj4+IFNvLCBJJ2xsIHB1dCBh
IGNvbW1lbnQgaW4gdGhlIE1ha2VmaWxlOg0KPj4NCj4+ICMgTi5CLiBUaGlzIGNhbm5vdCBiZSBl
eHByZXNzZWQgd2l0aCBhIHNpbmdsZSBsaW5lIHVzaW5nIENPTkZJR19YRU5fUENJX1NUVUINCj4+
DQo+PiAjIGFzIGl0IGFsd2F5cyByZW1haW5zIGluICJ5IiBzdGF0ZSwgdGh1cyBwcmV2ZW50aW5n
IHRoZSBkcml2ZXIgdG8gYmUgYnVpbHQgYXMNCj4+DQo+PiAjIGEgbW9kdWxlLg0KPj4NCj4+IG9i
ai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNrLm8NCj4+IG9iai0k
KENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4+DQo+PiBXaWxsIHRo
aXMgYmUgb2sgb3IgbmVlZHMgc29tZSByZS13b3JkaW5nPw0KPg0KPiBJJ2QgYWRkIHRoYXQgQ09O
RklHX1hFTl9QQ0lERVZfQkFDS0VORCBhbmQgQ09ORklHX1hFTl9QQ0lERVZfU1RVQiBhcmUNCj4g
bXV0dWFsbHkgZXhjbHVzaXZlLg0KIyBOLkIuIFRoZSBiZWxvdyBjYW5ub3QgYmUgZXhwcmVzc2Vk
IHdpdGggYSBzaW5nbGUgbGluZSB1c2luZw0KIyBDT05GSUdfWEVOX1BDSV9TVFVCIGFzIGl0IGFs
d2F5cyByZW1haW5zIGluICJ5IiBzdGF0ZSwNCiMgdGh1cyBwcmV2ZW50aW5nIHRoZSBkcml2ZXIg
dG8gYmUgYnVpbHQgYXMgYSBtb2R1bGUuDQojIFBsZWFzZSBub3RlLCB0aGF0IENPTkZJR19YRU5f
UENJREVWX0JBQ0tFTkQgYW5kDQojIENPTkZJR19YRU5fUENJREVWX1NUVUIgYXJlIG11dHVhbGx5
IGV4Y2x1c2l2ZS4NCm9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2li
YWNrLm8NCm9iai0kKENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCg0K
Pg0KPg0KPiBKdWVyZ2VuDQoNClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:24:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:24:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197643.350817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7Tp-0007ur-3O; Tue, 28 Sep 2021 07:24:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197643.350817; Tue, 28 Sep 2021 07:24:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7To-0007ua-W0; Tue, 28 Sep 2021 07:24:44 +0000
Received: by outflank-mailman (input) for mailman id 197643;
 Tue, 28 Sep 2021 07:24:44 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV7To-0007uK-C4
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:24:44 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 264952d0-202d-11ec-bc6f-12813bfff9fa;
 Tue, 28 Sep 2021 07:24:43 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S7Jrq3006951; 
 Tue, 28 Sep 2021 07:24:41 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbxjhr0hm-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 07:24:41 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3876.eurprd03.prod.outlook.com (2603:10a6:208:73::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 07:24:39 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 07:24:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 264952d0-202d-11ec-bc6f-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bIn+DCEqwBA4dsdV+TQw31EfSUDEkFkK/2stLgYMjLug3C7o8AyW9Chn7X2hpkzqB+TZJn0S3IZtleeYrjy5PM+F0cgZN3TOIBwbjfi7+5WVyYzf2l5nlF2GJ6x7Ik40maHWASA7HQhv9XQpr84RUFqPyP0tKdsKcKAXYVpv8tXhLs5GCT82qofpZqG3SQvYwW95DL7NX+Dft7VtyjEOMMDmV6DIfOF15z1N9wWNhZCleyuYLuXKxK923YxEzq/qQUfDesNHgJBDRcvkh7g0qrzW1iSPdtWJt7U44u0leuruG8xi0I63BZ9oLxzTI4w4mI0wUTiwUxfixJR9pZ6Zrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sEDVmAc6mgaQQPZ1Vaf2nxtRSHHLsStBrGZTK+4Al9o=;
 b=Zv8GqoTpiCHRMlYkD+iHj0gUckMOJDkIO4c/KAxpWjErv3eY2VMcnuWQzXrgCanKf7G2Dvr7Ja8UIHaGfm4QKEjNx3h9pQ8JrnE01zG9Me1E0Ybl89ZrPz9LeYki+kAOnUMG4UT98Vr3+L1Q7xadDl2RPjiJfXyc6kcPts7KNSHKDig5d/ueJAoONmdZefG+A18+EZcNivSuJFs2msLsdu+WNUchjFSNGTpsPOePBK8MSs8MWfNjYMwsXeRrQymqQ2M1piUWWR484D8sZvaS5tYSzt1PS+OFZBoSGOHF4q9Rl1IDm32wrncDRn/iklkTcGMbTfMHkY4ONRxTNX1Iug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=sEDVmAc6mgaQQPZ1Vaf2nxtRSHHLsStBrGZTK+4Al9o=;
 b=lEBao8hoQEvCxveK/ryQ84GVxTLFCnYyREs9fspde7qQ6MUuFE609pKDQDdTwnt8Q9cRhMb+63HE7qIoTxwDw+I7XlWMeUZg1HCDaqy7LggWjepVo48sOqol7pywbhSfvsz0/vlgXdj8nPoSHjA8rlbIY/IsiO/tufHFC0Sl3fuXjKu3e18XEZSp+7YnE1e3xv8+HLpUl9QWuU+XAJk7QgHil20wEItCuxD9+kZWtF3rPe/FN91axbh80zkwrkWxehXMDms3haPy0nAAEE30OdrB2wX3rEz0hTp6flJhZci45ltVO+EK8mxHAHtpSk3VVP84s/VSBv/Uo1jno9I34Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Juergen Gross <jgross@suse.com>, Jan Beulich <jbeulich@suse.com>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org"
	<julien@xen.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
Thread-Topic: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and
 PV
Thread-Index: 
 AQHXs20TiZWMfq6cRkakGi7ds9CK56u3ezoAgAACbACAAAVGAIABVeSAgAAoPoCAAAP7gIAAAMWAgAAFAACAAAESAIAAAQ8A
Date: Tue, 28 Sep 2021 07:24:39 +0000
Message-ID: <7e65070c-6899-4277-daf1-1a1aa7ade094@epam.com>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
 <alpine.DEB.2.21.2109280020150.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109280020150.5022@sstabellini-ThinkPad-T480s>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6486f299-91ea-4a83-7b9c-08d982510839
x-ms-traffictypediagnostic: AM0PR03MB3876:
x-microsoft-antispam-prvs: 
 <AM0PR03MB3876A9C444408B0D693ACF56E7A89@AM0PR03MB3876.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 NLNkisMiPZzLMHRwa9bhciEkFhXzQF8ElxmGZxLR1pp1aG4ZsXkfGm1WMIK93aAwZu5hK5Vq6X/s+RrC4ZU6HY4FADJYmkdDiAbNxqgzWNPc8JK1Yp+J8ovg6UTZMRVUQzHkf8uvrKLvAv9E7zirlJjFU+vLGA9yYj6NghQoqB/uG6qa+/PHUaDC58J0nQx6HZBQzJnvLniI+s6sJCiClg7ePiVLJIkuU9WbREKaVOTBRvvfNHF3u/8kY5h+A8+GgyNcL8MgLS+ZeHHZoYyzPRllUzjUg+frIOLNvOQD2C6GCX2KII0ETaHes9T5Wl7UiEuYCVtjV+X/LM4/BhSbtNPE+Rkha8PENCKalVrKqgvCSemXqhplDRCZZQWU5wMQDUyfOVtVjDmaJ+i0EnlpwkcvFpFdCb1F4r1d+rFRwDiqFJP0rQCShjnr3P9fQCGn3VLqV/aJ7Vapv2+/iGLJhnLXE68aSDxTTKS7MvwSC5cuAKVMob9EJbX5kwUTDjWuv4eI0RPU+KKz8uqYWINUhHeoIjblcGCDvW67NGNh6t1KGtLgrcPIXlu+V3uTTx+8gP3IBxZUMb3dQvM+8JwyNTCiuNZjY2C5AqrKnw5raYHTDxIcxBuZe5Sedsv0REPGiqbxRfZLzto1u7b9kbvW/+sly5XpFW7SQJ4Rlb26v6f6XNnTfwpSergHaBBeyAQpr9I76Cs/KkIWV31CMHoju6ks2J0bvaQPZ4m5kfvJAi5ZO/6VhgeqHNWjyonotySW
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66556008)(38070700005)(66946007)(91956017)(66476007)(6916009)(66446008)(83380400001)(54906003)(86362001)(64756008)(2906002)(76116006)(2616005)(36756003)(31696002)(4326008)(8936002)(5660300002)(6506007)(53546011)(6486002)(55236004)(8676002)(71200400001)(26005)(186003)(122000001)(31686004)(316002)(6512007)(508600001)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?azVCSW9QMVA2QTljd3RIMGl0UktMcXU2QnNTMG8xWGtlYTNkODlXU1FPcDBQ?=
 =?utf-8?B?YWlkeVNrd1BvY3M3SWNEUDlTT2VjSTVOYTIzNFJONlBiYVdCNThPY0tZYjgw?=
 =?utf-8?B?UGd0U3pySVh2dEJKUm8xTndDZ0YzMzdJVDYxQTRpd3I4ZEZRaTc0VHA5aWZt?=
 =?utf-8?B?Tk1oSG9XTnZBOEdmN0FtRVVHVU5CMEZ3UWo4M0p4R1g3d1RudTVJdi95Y1lB?=
 =?utf-8?B?RU1kNEFyMmk0eW1zNEVQMW9VZ1pVTjBuS1J5bUNRVUI2WG9QbHBOb0VmOTFa?=
 =?utf-8?B?bGdnWUFBYVhOVEo5V3RoUURpUWN0Sk1Qa0ZkUTNmajJLVktKbkdBNXdnV2VL?=
 =?utf-8?B?N2V4QkI0bEpZSGprcWk0ZzlGT2dORVlBcHEySkhIT1V3dnZxQ2doOEo1dXg2?=
 =?utf-8?B?c0VjUktSN240UXViMGp2OTJVMnNBZHVHYjFlVnlIeVczcmJoYUU2eUo3aTNs?=
 =?utf-8?B?NEc1cU1kcHl0ZkxlYkc0TVFQVHNaSVdtNjl6VDdSajRraHlpZmp6bGVwV0VG?=
 =?utf-8?B?eVVadEI3akVBSmtlRkdacm9XT3RtYlpYN2MvbnZKZTNlS1hKcHZOODFaR0Qr?=
 =?utf-8?B?T2FOK3pqRVpGV0Z3cXhSVDJva1RXenVKc3pBL01KNkpLMjBPczlxMTZUdi9u?=
 =?utf-8?B?MlNQeFhnU29oZkF1aThKUWJSTW1JVk5lSzluVlgyUnJrM0JxUVBJZm4rY1Bp?=
 =?utf-8?B?cHdCUGUwRHpTVWdnbEkvWjVmMitVMTNLeDJhL25hUTNOOEVDNXpnWmFNTStw?=
 =?utf-8?B?QWFwenBlOUNVNnkybG9rMjJlRWYrVDQ4M25vb3VxZ0IxbFdHVU9jaklGVnU2?=
 =?utf-8?B?M2JoWk9lYlFMUDROMUovSmQzTW5lYlJqb3lNTmtOVnhIMWhmaXk0RHd1OFlk?=
 =?utf-8?B?c1NydUZtbk9DalFBbWRWUmczWjdRZk00T09POGdtRGNPQWJJNjhHcW1vN2s1?=
 =?utf-8?B?T3ZRZmw5SjZtZUR1TE9Fa21VVWJGRmo4Y1FSd29ReEx1cllzY25BOE1NdzMz?=
 =?utf-8?B?SHJtRW83MVdaUlVYNHVQTDBhYndqU3g3RGo2ZmY0ejRsUFB4QVphWWpXZzBY?=
 =?utf-8?B?b05PQVRTRm9DajQ4WkswZ3RYcDVkN3J0bjMzZW9xSTA0MW9aN3czNngvWmRV?=
 =?utf-8?B?R1BHWmVwaWs3ODVXcHJaNS8ydGNCcDVZcTJzNWxnVExieGJHT05PNmpCVndS?=
 =?utf-8?B?TnJzVkNpb0JMa2dzOG9yRUhCWlhVMDI5UkNjL0tHQ3JoUndlNzgvUlowSHFw?=
 =?utf-8?B?cVYzM1g0dmlmT0g2T3hEN0hyeHJkLysyQUxRS1h0cDFSVTdhY282TXpJV2Za?=
 =?utf-8?B?STlHdGtwTm9qck5SQ0cxa040N3QzY2hhZ1pCNlllaC9rSUJDQjdYNWhBaWoz?=
 =?utf-8?B?YVBGV0U1SU5aZGxveXp0YzZwcW1FdVQ5UlRYYVNrcnM2ay85RFNMa2JKRWlM?=
 =?utf-8?B?Z3NnaXBiQUIwTW0zZ2lhRmxXSUxYVkovMnZaM1NGYVBmcHIwWGZxQkVScjFy?=
 =?utf-8?B?dUJQMTQwL3B6ZlgxcXZFNnpQNm5wOWVUWmNxV2xLTlVwcnVlZi9pN3FSSHVp?=
 =?utf-8?B?UUx1T1RMbU4xaGxzTDVaTDhZSitoU1M2L1RXMEJwY3N5blFnSlE1U2VHdWpi?=
 =?utf-8?B?UGlBL1dvODVvaldvWkQ4S2w3R0lrSUh5cFFJK3VXTVdPNDhWc291Y2tpZFU5?=
 =?utf-8?B?c1BwTXQ2d2JSdjBJMktBQjhWV3QxVStIZ2lJYWNSZzFPanAveEFCd2tvTzNX?=
 =?utf-8?B?V1VLNjhyTUFicVVqN1J2QThZV1VHa0kweVRxZE9TMXJoNFk4dW5ieTE3WkZj?=
 =?utf-8?B?bGU5MGtlWnNMcHM0YjFaQT09?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <F1A27ECF0661B94FB27F706C97BB6A86@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6486f299-91ea-4a83-7b9c-08d982510839
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 07:24:39.0646
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: xpwu7zEM6xTVpq5THYhRVlUmbhN9eGvkJqJIlTlkWvmXTaDuCiFn14y6M2YZVmZPT33Ae9bzr2dgQAFRKQQynhPEyGUVbUk0wJfCx/MyyDOaDbcE1DhgU1XMDo3XUuK+
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3876
X-Proofpoint-GUID: V5LmgX9tBloon_ypB1sIDzE6yNsPjYO1
X-Proofpoint-ORIG-GUID: V5LmgX9tBloon_ypB1sIDzE6yNsPjYO1
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-24_02,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 malwarescore=0
 mlxlogscore=999 adultscore=0 phishscore=0 clxscore=1015 priorityscore=1501
 impostorscore=0 suspectscore=0 mlxscore=0 spamscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280042

DQpPbiAyOC4wOS4yMSAxMDoyMCwgU3RlZmFubyBTdGFiZWxsaW5pIHdyb3RlOg0KPiBPbiBUdWUs
IDI4IFNlcCAyMDIxLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+IE9uIDI4LjA5
LjIxIDA5OjU5LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4+IE9uIDI4LjA5LjIxIDA4OjU2LCBP
bGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4gT24gMjguMDkuMjEgMDk6NDIsIEph
biBCZXVsaWNoIHdyb3RlOg0KPj4+Pj4gT24gMjguMDkuMjAyMSAwNjoxOCwgU3RlZmFubyBTdGFi
ZWxsaW5pIHdyb3RlOg0KPj4+Pj4+IE9uIE1vbiwgMjcgU2VwIDIwMjEsIEp1ZXJnZW4gR3Jvc3Mg
d3JvdGU6DQo+Pj4+Pj4+IE9uIDI3LjA5LjIxIDA5OjM1LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5r
byB3cm90ZToNCj4+Pj4+Pj4+IE9uIDI3LjA5LjIxIDEwOjI2LCBKYW4gQmV1bGljaCB3cm90ZToN
Cj4+Pj4+Pj4+PiBPbiAyNy4wOS4yMDIxIDA4OjU4LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3
cm90ZToNCj4+Pj4+Pj4+Pj4gRnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5k
cl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+PiBDdXJyZW50
bHkgUENJIGJhY2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlvbmFsaXRpZXMgYXQgYSB0
aW1lLg0KPj4+Pj4+Pj4+PiBUbyBuYW1lIGEgZmV3Og0KPj4+Pj4+Pj4+PiAxLiBJdCBpcyB1c2Vk
IGFzIGEgZGF0YWJhc2UgZm9yIGFzc2lnbmFibGUgUENJIGRldmljZXMsIGUuZy4geGwNCj4+Pj4+
Pj4+Pj4gIMKgwqDCoMKgwqDCoCBwY2ktYXNzaWduYWJsZS17YWRkfHJlbW92ZXxsaXN0fSBtYW5p
cHVsYXRlcyB0aGF0IGxpc3QuIFNvLA0KPj4+Pj4+Pj4+PiB3aGVuZXZlcg0KPj4+Pj4+Pj4+PiAg
wqDCoMKgwqDCoMKgIHRoZSB0b29sc3RhY2sgbmVlZHMgdG8ga25vdyB3aGljaCBQQ0kgZGV2aWNl
cyBjYW4gYmUgcGFzc2VkIHRocm91Z2gNCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoCBpdCByZWFk
cyB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2ZzIGVudHJpZXMgb2YgdGhlIHBjaWJhY2suDQo+
Pj4+Pj4+Pj4+IDIuIEl0IGlzIHVzZWQgdG8gaG9sZCB0aGUgdW5ib3VuZCBQQ0kgZGV2aWNlcyBs
aXN0LCBlLmcuIHdoZW4gcGFzc2luZw0KPj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoMKgIHRocm91Z2gg
YSBQQ0kgZGV2aWNlIGl0IG5lZWRzIHRvIGJlIHVuYm91bmQgZnJvbSB0aGUgcmVsZXZhbnQNCj4+
Pj4+Pj4+Pj4gZGV2aWNlDQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqAgZHJpdmVyIGFuZCBib3Vu
ZCB0byBwY2liYWNrIChzdHJpY3RseSBzcGVha2luZyBpdCBpcyBub3QgcmVxdWlyZWQNCj4+Pj4+
Pj4+Pj4gIMKgwqDCoMKgwqDCoCB0aGF0IHRoZSBkZXZpY2UgaXMgYm91bmQgdG8gcGNpYmFjaywg
YnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBhcyBhDQo+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqAg
ZGF0YWJhc2Ugb2YgdGhlIHBhc3NlZCB0aHJvdWdoIFBDSSBkZXZpY2VzLCBzbyB3ZSBjYW4gcmUt
YmluZCB0aGUNCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoCBkZXZpY2VzIGJhY2sgdG8gdGhlaXIg
b3JpZ2luYWwgZHJpdmVycyB3aGVuIGd1ZXN0IGRvbWFpbiBzaHV0cw0KPj4+Pj4+Pj4+PiBkb3du
KQ0KPj4+Pj4+Pj4+PiAzLiBEZXZpY2UgcmVzZXQgZm9yIHRoZSBkZXZpY2VzIGJlaW5nIHBhc3Nl
ZCB0aHJvdWdoDQo+Pj4+Pj4+Pj4+IDQuIFBhcmEtdmlydHVhbGlzZWQgdXNlLWNhc2VzIHN1cHBv
cnQNCj4+Pj4+Pj4+Pj4NCj4+Pj4+Pj4+Pj4gVGhlIHBhcmEtdmlydHVhbGlzZWQgcGFydCBvZiB0
aGUgZHJpdmVyIGlzIG5vdCBhbHdheXMgbmVlZGVkIGFzIHNvbWUNCj4+Pj4+Pj4+Pj4gYXJjaGl0
ZWN0dXJlcywgZS5nLiBBcm0gb3IgeDg2IFBWSCBEb20wLCBhcmUgbm90IHVzaW5nIGJhY2tlbmQt
ZnJvbnRlbmQNCj4+Pj4+Pj4+Pj4gbW9kZWwgZm9yIFBDSSBkZXZpY2UgcGFzc3Rocm91Z2guIEZv
ciBzdWNoIHVzZS1jYXNlcyBtYWtlIHRoZSB2ZXJ5DQo+Pj4+Pj4+Pj4+IGZpcnN0IHN0ZXAgaW4g
c3BsaXR0aW5nIHRoZSB4ZW4tcGNpYmFjayBkcml2ZXIgaW50byB0d28gcGFydHM6IFhlbg0KPj4+
Pj4+Pj4+PiBQQ0kgc3R1YiBhbmQgUENJIFBWIGJhY2tlbmQgZHJpdmVycy4NCj4+Pj4+Pj4+Pj4N
Cj4+Pj4+Pj4+Pj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28NCj4+Pj4+
Pj4+Pj4gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KPj4+Pj4+Pj4+Pg0KPj4+
Pj4+Pj4+PiAtLS0NCj4+Pj4+Pj4+Pj4gQ2hhbmdlcyBzaW5jZSB2MzoNCj4+Pj4+Pj4+Pj4gLSBN
b3ZlIENPTkZJR19YRU5fUENJREVWX1NUVUIgdG8gdGhlIHNlY29uZCBwYXRjaA0KPj4+Pj4+Pj4+
IEknbSBhZnJhaWQgdGhpcyB3YXNuJ3QgZnVsbHkgZG9uZToNCj4+Pj4+Pj4+Pg0KPj4+Pj4+Pj4+
PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+Pj4+Pj4+PiArKysg
Yi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPj4+Pj4+Pj4+PiBAQCAtMSw1ICsx
LDYgQEANCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgICMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQ
TC0yLjANCj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgIG9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tF
TkQpICs9IHhlbi1wY2liYWNrLm8NCj4+Pj4+Pj4+Pj4gK29iai0kKENPTkZJR19YRU5fUENJREVW
X1NUVUIpICs9IHhlbi1wY2liYWNrLm8NCj4+Pj4+Pj4+PiBXaGlsZSBiZW5pZ24gd2hlbiBhZGRl
ZCBoZXJlLCB0aGlzIGFkZGl0aW9uIHN0aWxsIGRvZXNuJ3Qgc2VlbSB0bw0KPj4+Pj4+Pj4+IGJl
bG9uZyBoZXJlLg0KPj4+Pj4+Pj4gTXkgYmFkLiBTbywgaXQgc2VlbXMgd2l0aG91dCBDT05GSUdf
WEVOX1BDSURFVl9TVFVCIHRoZSBjaGFuZ2Ugc2VlbXMNCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiB0byBi
ZSBub24tZnVuY3Rpb25hbC4gV2l0aCBDT05GSUdfWEVOX1BDSURFVl9TVFVCIHdlIGZhaWwgdG8g
YnVpbGQgb24gMzItYml0DQo+Pj4+Pj4+Pg0KPj4+Pj4+Pj4gYXJjaGl0ZWN0dXJlcy4uLg0KPj4+
Pj4+Pj4NCj4+Pj4+Pj4+IFdoYXQgd291bGQgYmUgdGhlIHByZWZlcmVuY2UgaGVyZT8gU3RlZmFu
byBzdWdnZXN0ZWQgdGhhdCB3ZSBzdGlsbCBkZWZpbmUNCj4+Pj4+Pj4+DQo+Pj4+Pj4+PiBDT05G
SUdfWEVOX1BDSURFVl9TVFVCLCBidXQgaW4gZGlzYWJsZWQgc3RhdGUsIGUuZy4gd2UgYWRkIHRy
aXN0YXRlIHRvIGl0DQo+Pj4+Pj4+Pg0KPj4+Pj4+Pj4gaW4gdGhlIHNlY29uZCBwYXRjaA0KPj4+
Pj4+Pj4NCj4+Pj4+Pj4+IEFub3RoZXIgb3B0aW9uIGlzIGp1c3QgdG8gc3F1YXNoIHRoZSB0d28g
cGF0Y2hlcy4NCj4+Pj4+Pj4gU3F1YXNoaW5nIHdvdWxkIGJlIGZpbmUgZm9yIG1lLg0KPj4+Pj4+
ICDCoMKgIEl0IGlzIGZpbmUgZm9yIG1lIHRvIHNxdWFzaCB0aGUgdHdvIHBhdGNoZXMuDQo+Pj4+
Pj4NCj4+Pj4+PiBCdXQgaW4gYW55IGNhc2UsIHdvdWxkbid0IGl0IGJlIGJldHRlciB0byBtb2Rp
ZnkgdGhhdCBzcGVjaWZpYyBjaGFuZ2UgdG86DQo+Pj4+Pj4NCj4+Pj4+PiBkaWZmIC0tZ2l0IGEv
ZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9NYWtlZmlsZQ0KPj4+Pj4+IGluZGV4IGUyY2IzNzY0NDRhNi4uZTIzYzc1OGI4NWFlIDEwMDY0
NA0KPj4+Pj4+IC0tLSBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL01ha2VmaWxlDQo+Pj4+Pj4g
KysrIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4+Pj4+PiBAQCAtMSw2ICsx
LDUgQEANCj4+Pj4+PiAgwqDCoCAjIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wDQo+
Pj4+Pj4gLW9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNrLm8N
Cj4+Pj4+PiAtb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfU1RVQikgKz0geGVuLXBjaWJhY2subw0K
Pj4+Pj4+ICtvYmotJChDT05GSUdfWEVOX1BDSV9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+Pj4+
PiBCdXQgdGhhdCB3b3VsZG4ndCBhbGxvdyB0aGUgZHJpdmVyIHRvIGJlIGEgbW9kdWxlIGFueW1v
cmUsIHdvdWxkIGl0Pw0KPj4+PiBFeGFjdGx5LiBJIGZvcmdvdCB0aGF0IHdoZW4gcGxheWluZyB3
aXRoIG1vZHVsZS9idWlsdC1pbiBJIHdhcyBub3QgYWJsZQ0KPj4+Pg0KPj4+PiB0byBjb250cm9s
IHRoYXQgYW55bW9yZSBiZWNhdXNlIENPTkZJR19YRU5fUENJX1NUVUIgd2lsbCBhbHdheXMgYmUN
Cj4+Pj4NCj4+Pj4gaW4gInkiIHN0YXRlLCB0aHVzIGV2ZW4gaWYgeW91IGhhdmUgQ09ORklHX1hF
Tl9QQ0lERVZfQkFDS0VORD1tDQo+Pj4+DQo+Pj4+IHlvdSB3b24ndCBiZSBhYmxlIHRvIGJ1aWxk
IGl0IGFzIG1vZHVsZS4gU28sIEkgd2lsbCBwcm9iYWJseSBwdXQgYSBjb21tZW50DQo+Pj4+DQo+
Pj4+IGFib3V0IHRoYXQgaW4gdGhlIE1ha2VmaWxlIGV4cGxhaW5pbmcgdGhlIG5lZWQgZm9yDQo+
Pj4+DQo+Pj4+IG9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQpICs9IHhlbi1wY2liYWNr
Lm8NCj4+Pj4gb2JqLSQoQ09ORklHX1hFTl9QQ0lERVZfU1RVQikgKz0geGVuLXBjaWJhY2subw0K
Pj4+IEluIGNhc2UgdGhlIHJlYWwgc3BsaXQgYmV0d2VlbiBib3RoIHBhcnRzIG9mIHhlbi1wY2li
YWNrIGlzIGRvbmUgdGhpcw0KPj4+IHdpbGwgYmUgbmVlZGVkIGFueXdheS4NCj4+IFllcywgaXQg
d2lsbA0KPj4NCj4+IFNvLCBJJ2xsIHB1dCBhIGNvbW1lbnQgaW4gdGhlIE1ha2VmaWxlOg0KPj4N
Cj4+ICMgTi5CLiBUaGlzIGNhbm5vdCBiZSBleHByZXNzZWQgd2l0aCBhIHNpbmdsZSBsaW5lIHVz
aW5nIENPTkZJR19YRU5fUENJX1NUVUINCj4+DQo+PiAjIGFzIGl0IGFsd2F5cyByZW1haW5zIGlu
ICJ5IiBzdGF0ZSwgdGh1cyBwcmV2ZW50aW5nIHRoZSBkcml2ZXIgdG8gYmUgYnVpbHQgYXMNCj4+
DQo+PiAjIGEgbW9kdWxlLg0KPj4NCj4+IG9iai0kKENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQp
ICs9IHhlbi1wY2liYWNrLm8NCj4+IG9iai0kKENPTkZJR19YRU5fUENJREVWX1NUVUIpICs9IHhl
bi1wY2liYWNrLm8NCj4+DQo+PiBXaWxsIHRoaXMgYmUgb2sgb3IgbmVlZHMgc29tZSByZS13b3Jk
aW5nPw0KPiBJIGFtIGZpbmUgd2l0aCBpdCBhbmQgaG9uZXN0bHkgdGhhdCB3YXMgdGhlIG9ubHkg
Y29tbWVudCBJIGhhZCBzbyB5b3UNCj4gY2FuIGFkZCBteQ0KPg0KPiBSZXZpZXdlZC1ieTogU3Rl
ZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0KT2ssIHRoYW5rIHlvdQ==


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:26:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:26:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197656.350829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7VC-0000FS-GC; Tue, 28 Sep 2021 07:26:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197656.350829; Tue, 28 Sep 2021 07:26:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7VC-0000FL-CV; Tue, 28 Sep 2021 07:26:10 +0000
Received: by outflank-mailman (input) for mailman id 197656;
 Tue, 28 Sep 2021 07:26:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV7VB-0000FC-6R
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:26:09 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5896616a-202d-11ec-bc6f-12813bfff9fa;
 Tue, 28 Sep 2021 07:26:08 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 18DA522286;
 Tue, 28 Sep 2021 07:26:07 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D9A2413ACF;
 Tue, 28 Sep 2021 07:26:06 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id YTfDM47DUmGjZAAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 07:26:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5896616a-202d-11ec-bc6f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632813967; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=jQCgZdE7pv4+WIIfiYvyIaTfRhRo+++4IApjitHJov8=;
	b=I2C+IOaATHyAeIyxWtVFlb9RqisPyWOEN27HuyFxgZ9RL0tG96ih6aiQvU+YtpnR3aXoUB
	pghBdJL07/kF3eHQM0cpHmBBB0ZhDZg+YQbIFy6gVJnOFED3tBwQRY6tZ0DMGxkL5lSRJJ
	N7txiTmBajKUXH+ru/zFjBPn2BOokZI=
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
 <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
 <b835b135-1113-2762-459a-b6e31053336a@epam.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <a30fc7d9-0827-4970-845f-00cd711ff747@suse.com>
Date: Tue, 28 Sep 2021 09:26:06 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <b835b135-1113-2762-459a-b6e31053336a@epam.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="xIeC2D0vfBQmQvhVmgsuyjJnjFIpMibVn"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--xIeC2D0vfBQmQvhVmgsuyjJnjFIpMibVn
Content-Type: multipart/mixed; boundary="bxvJ1mioIrQTt0VSI5fRg2vdvvJnyBynD";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
 Jan Beulich <jbeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>
Cc: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "julien@xen.org" <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Message-ID: <a30fc7d9-0827-4970-845f-00cd711ff747@suse.com>
Subject: Re: [PATCH v4 1/2] xen-pciback: prepare for the split for stub and PV
References: <20210927065822.350973-1-andr2000@gmail.com>
 <e472468a-625e-6c4d-a9c2-85594e2ff908@suse.com>
 <accd0220-a9d7-145b-6632-9dee085ffc65@epam.com>
 <1cf5fbf3-6453-e258-3940-8b5bb96117b6@suse.com>
 <alpine.DEB.2.21.2109272112150.5022@sstabellini-ThinkPad-T480s>
 <0b952b8d-0ebd-1c8c-84d4-f02e05bc2a2b@suse.com>
 <b4c8b964-56d7-c693-98e0-ecb435925eb0@epam.com>
 <da921aa0-0259-cd2e-5f43-5e2934ad3f82@suse.com>
 <51e38ddc-49df-2e70-d094-71d045297ab1@epam.com>
 <2b4dc684-a00f-1a08-92b0-7b0b614aeb24@suse.com>
 <b835b135-1113-2762-459a-b6e31053336a@epam.com>
In-Reply-To: <b835b135-1113-2762-459a-b6e31053336a@epam.com>

--bxvJ1mioIrQTt0VSI5fRg2vdvvJnyBynD
Content-Type: multipart/mixed;
 boundary="------------30492A6117F16F81287211C1"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------30492A6117F16F81287211C1
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 09:24, Oleksandr Andrushchenko wrote:
>=20
> On 28.09.21 10:20, Juergen Gross wrote:
>> On 28.09.21 09:17, Oleksandr Andrushchenko wrote:
>>>
>>> On 28.09.21 09:59, Juergen Gross wrote:
>>>> On 28.09.21 08:56, Oleksandr Andrushchenko wrote:
>>>>>
>>>>> On 28.09.21 09:42, Jan Beulich wrote:
>>>>>> On 28.09.2021 06:18, Stefano Stabellini wrote:
>>>>>>> On Mon, 27 Sep 2021, Juergen Gross wrote:
>>>>>>>> On 27.09.21 09:35, Oleksandr Andrushchenko wrote:
>>>>>>>>> On 27.09.21 10:26, Jan Beulich wrote:
>>>>>>>>>> On 27.09.2021 08:58, Oleksandr Andrushchenko wrote:
>>>>>>>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.c=
om>
>>>>>>>>>>>
>>>>>>>>>>> Currently PCI backend implements multiple functionalities at =
a time.
>>>>>>>>>>> To name a few:
>>>>>>>>>>> 1. It is used as a database for assignable PCI devices, e.g. =
xl
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pci-assignable-{a=
dd|remove|list} manipulates that list. So,
>>>>>>>>>>> whenever
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the toolstack nee=
ds to know which PCI devices can be passed through
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 it reads that fro=
m the relevant sysfs entries of the pciback.
>>>>>>>>>>> 2. It is used to hold the unbound PCI devices list, e.g. when=
 passing
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 through a PCI dev=
ice it needs to be unbound from the relevant
>>>>>>>>>>> device
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 driver and bound =
to pciback (strictly speaking it is not required
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 that the device i=
s bound to pciback, but pciback is again used as a
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 database of the p=
assed through PCI devices, so we can re-bind the
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 devices back to t=
heir original drivers when guest domain shuts
>>>>>>>>>>> down)
>>>>>>>>>>> 3. Device reset for the devices being passed through
>>>>>>>>>>> 4. Para-virtualised use-cases support
>>>>>>>>>>>
>>>>>>>>>>> The para-virtualised part of the driver is not always needed =
as some
>>>>>>>>>>> architectures, e.g. Arm or x86 PVH Dom0, are not using backen=
d-frontend
>>>>>>>>>>> model for PCI device passthrough. For such use-cases make the=
 very
>>>>>>>>>>> first step in splitting the xen-pciback driver into two parts=
: Xen
>>>>>>>>>>> PCI stub and PCI PV backend drivers.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Oleksandr Andrushchenko
>>>>>>>>>>> <oleksandr_andrushchenko@epam.com>
>>>>>>>>>>>
>>>>>>>>>>> ---
>>>>>>>>>>> Changes since v3:
>>>>>>>>>>> - Move CONFIG_XEN_PCIDEV_STUB to the second patch
>>>>>>>>>> I'm afraid this wasn't fully done:
>>>>>>>>>>
>>>>>>>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>>>>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>>>>>>>> @@ -1,5 +1,6 @@
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # SPDX-License-Identifier: GP=
L-2.0
>>>>>>>>>>>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 obj-$(CONFIG_XEN_PCIDEV_BACKE=
ND) +=3D xen-pciback.o
>>>>>>>>>>> +obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>>>>>>> While benign when added here, this addition still doesn't seem=
 to
>>>>>>>>>> belong here.
>>>>>>>>> My bad. So, it seems without CONFIG_XEN_PCIDEV_STUB the change =
seems
>>>>>>>>>
>>>>>>>>> to be non-functional. With CONFIG_XEN_PCIDEV_STUB we fail to bu=
ild on 32-bit
>>>>>>>>>
>>>>>>>>> architectures...
>>>>>>>>>
>>>>>>>>> What would be the preference here? Stefano suggested that we st=
ill define
>>>>>>>>>
>>>>>>>>> CONFIG_XEN_PCIDEV_STUB, but in disabled state, e.g. we add tris=
tate to it
>>>>>>>>>
>>>>>>>>> in the second patch
>>>>>>>>>
>>>>>>>>> Another option is just to squash the two patches.
>>>>>>>> Squashing would be fine for me.
>>>>>>>  =C2=A0=C2=A0=C2=A0 It is fine for me to squash the two patches.
>>>>>>>
>>>>>>> But in any case, wouldn't it be better to modify that specific ch=
ange to:
>>>>>>>
>>>>>>> diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-p=
ciback/Makefile
>>>>>>> index e2cb376444a6..e23c758b85ae 100644
>>>>>>> --- a/drivers/xen/xen-pciback/Makefile
>>>>>>> +++ b/drivers/xen/xen-pciback/Makefile
>>>>>>> @@ -1,6 +1,5 @@
>>>>>>>  =C2=A0=C2=A0=C2=A0 # SPDX-License-Identifier: GPL-2.0
>>>>>>> -obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>>>>>> -obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>>>> +obj-$(CONFIG_XEN_PCI_STUB) +=3D xen-pciback.o
>>>>>> But that wouldn't allow the driver to be a module anymore, would i=
t?
>>>>>
>>>>> Exactly. I forgot that when playing with module/built-in I was not =
able
>>>>>
>>>>> to control that anymore because CONFIG_XEN_PCI_STUB will always be
>>>>>
>>>>> in "y" state, thus even if you have CONFIG_XEN_PCIDEV_BACKEND=3Dm
>>>>>
>>>>> you won't be able to build it as module. So, I will probably put a =
comment
>>>>>
>>>>> about that in the Makefile explaining the need for
>>>>>
>>>>> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>>>> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>>
>>>> In case the real split between both parts of xen-pciback is done thi=
s
>>>> will be needed anyway.
>>>
>>> Yes, it will
>>>
>>> So, I'll put a comment in the Makefile:
>>>
>>> # N.B. This cannot be expressed with a single line using CONFIG_XEN_P=
CI_STUB
>>>
>>> # as it always remains in "y" state, thus preventing the driver to be=
 built as
>>>
>>> # a module.
>>>
>>> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
>>> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o
>>>
>>> Will this be ok or needs some re-wording?
>>
>> I'd add that CONFIG_XEN_PCIDEV_BACKEND and CONFIG_XEN_PCIDEV_STUB are
>> mutually exclusive.
> # N.B. The below cannot be expressed with a single line using
> # CONFIG_XEN_PCI_STUB as it always remains in "y" state,
> # thus preventing the driver to be built as a module.
> # Please note, that CONFIG_XEN_PCIDEV_BACKEND and
> # CONFIG_XEN_PCIDEV_STUB are mutually exclusive.
> obj-$(CONFIG_XEN_PCIDEV_BACKEND) +=3D xen-pciback.o
> obj-$(CONFIG_XEN_PCIDEV_STUB) +=3D xen-pciback.o

Yes, that's fine.


Juergen


--------------30492A6117F16F81287211C1
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------30492A6117F16F81287211C1--

--bxvJ1mioIrQTt0VSI5fRg2vdvvJnyBynD--

--xIeC2D0vfBQmQvhVmgsuyjJnjFIpMibVn
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFSw44FAwAAAAAACgkQsN6d1ii/Ey89
GQgAlse+cel4CxZ3mB6WuwDFgMEicFjw8A/Hh1QGRLCC9wlATsNMbKjyA688TD67aA95dkjWEKzh
+U2stXEsoltlLv14yEzHbev7/1I6QTBDfr7T7yEpPjggrbT3LnFh/qqaEefiyh021l3h4IZI/Lbb
VuBMCXPFzF71Nl6Uo/XY8X+ywe44LLNfeZr3op1RcmAOfznzI+bnV3Tqir/OupACZFgKiLoITwra
XZtdrCPNrGIj8r7UsO3OJ5mTME57/xYwokRwS7UAlrBloYOu5P+GVyWqDR8lHkk6saEJA++TU5r8
rJ7MFdbdFPGQEdFh893g4qrmMxk3vDSt6Uqb5tu0uw==
=Z3vT
-----END PGP SIGNATURE-----

--xIeC2D0vfBQmQvhVmgsuyjJnjFIpMibVn--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:28:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:28:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197666.350840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7XJ-00017i-14; Tue, 28 Sep 2021 07:28:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197666.350840; Tue, 28 Sep 2021 07:28:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7XI-00017b-Tn; Tue, 28 Sep 2021 07:28:20 +0000
Received: by outflank-mailman (input) for mailman id 197666;
 Tue, 28 Sep 2021 07:28:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV7XH-00017V-4e
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:28:19 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id a697ae46-202d-11ec-bc6f-12813bfff9fa;
 Tue, 28 Sep 2021 07:28:18 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F0813D6E;
 Tue, 28 Sep 2021 00:28:17 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 264E33F7B4;
 Tue, 28 Sep 2021 00:28:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a697ae46-202d-11ec-bc6f-12813bfff9fa
Subject: Re: [PATCH V3 3/3] libxl/arm: Add handling of extended regions for
 DomU
To: Oleksandr Tyshchenko <olekstysh@gmail.com>, xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
 <1632437334-12015-4-git-send-email-olekstysh@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <bcb45e81-6df6-4aa9-e190-d69bff040bba@arm.com>
Date: Tue, 28 Sep 2021 09:28:08 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <1632437334-12015-4-git-send-email-olekstysh@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi,

On 24.09.2021 00:48, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain
>   currently.
> - The ACPI case is not covered.
> 
> ***
> 
> The algorithm to choose extended regions for non-direct mapped
> DomU is simpler in comparison with the algorithm for direct mapped
> Dom0. As we have a lot of unused space above 4GB, provide single
> 2MB-aligned region from the second RAM bank taking into the account
> the maximum supported guest address space size and the amount of
> memory assigned to the guest. The maximum size of the region is 128GB.
> The minimum size is 64MB.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> ---
> Changes RFC -> V2:
>    - update patch description
>    - drop uneeded "extended-region" DT property
>    - clear reg array in finalise_ext_region() and add a TODO
> 
> Changes V2 -> V3:
>    - update patch description, comments in code
>    - only pick up regions with size >= 64MB
>    - move the region calculation to make_hypervisor_node() and drop
>      finalise_ext_region()
>    - extend the list of arguments for make_hypervisor_node()
>    - do not show warning for 32-bit domain
>    - change the region alignment from 1GB to 2MB
>    - move EXT_REGION_SIZE to public/arch-arm.h
> ---
>  tools/libs/light/libxl_arm.c  | 70 +++++++++++++++++++++++++++++++++++++++----
>  xen/include/public/arch-arm.h |  3 ++
>  2 files changed, 68 insertions(+), 5 deletions(-)
> 
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
Tested-by: Michal Orzel <michal.orzel@arm.com>

Cheers


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:31:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:31:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197672.350851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7a5-0002We-F2; Tue, 28 Sep 2021 07:31:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197672.350851; Tue, 28 Sep 2021 07:31:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7a5-0002WX-Bs; Tue, 28 Sep 2021 07:31:13 +0000
Received: by outflank-mailman (input) for mailman id 197672;
 Tue, 28 Sep 2021 07:31:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV7a3-0002WR-Fy
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:31:11 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 53547801-c2da-4924-80a0-5ff2bb4ccb01;
 Tue, 28 Sep 2021 07:31:10 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 172BED6E;
 Tue, 28 Sep 2021 00:31:10 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 456C63F7B4;
 Tue, 28 Sep 2021 00:31:07 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53547801-c2da-4924-80a0-5ff2bb4ccb01
Subject: Re: [PATCH v2 01/11] xen/arm: Fix dev_is_dt macro definition
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-2-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <0eda8f1b-a090-5a9b-b9ea-3160be789ab8@arm.com>
Date: Tue, 28 Sep 2021 09:31:00 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125438.234162-2-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> This macro is not currently used, but still has an error in it:
> a missing parenthesis. Fix this, so the macro is properly defined.
> 
> Fixes: 6c5d3075d97e ("xen/arm: Introduce a generic way to describe device")
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> New in v2
> ---
>  xen/include/asm-arm/device.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index 582119c31ee0..64aaa2641b7f 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -29,7 +29,7 @@ typedef struct device device_t;
>  
>  /* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
>  #define dev_is_pci(dev) ((void)(dev), 0)
> -#define dev_is_dt(dev)  ((dev->type == DEV_DT)
> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
>  
>  enum device_class
>  {
> 

Reviewed-by: Michal Orzel <michal.orzel@arm.com>


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:34:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:34:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197678.350862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7d4-0003A3-VC; Tue, 28 Sep 2021 07:34:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197678.350862; Tue, 28 Sep 2021 07:34:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7d4-00039w-QI; Tue, 28 Sep 2021 07:34:18 +0000
Received: by outflank-mailman (input) for mailman id 197678;
 Tue, 28 Sep 2021 07:34:17 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sHmr=OS=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mV7d3-00039k-Nu
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:34:17 +0000
Received: from mail-wr1-x433.google.com (unknown [2a00:1450:4864:20::433])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2d5c09d3-1437-4560-9ee4-9c0485e98a36;
 Tue, 28 Sep 2021 07:34:16 +0000 (UTC)
Received: by mail-wr1-x433.google.com with SMTP id x20so11609018wrg.10
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:34:16 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id f17sm8890400wrm.83.2021.09.28.00.34.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 00:34:15 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d5c09d3-1437-4560-9ee4-9c0485e98a36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=NNakOyTZYGeX/Mb0ow/VeI5lJM8wKzbBa+NgDqXeOU8=;
        b=E9TbSX7TmQh18kx3b27rLiP9HQvBG+JL+iPCYZXwky1q3gpDGkay40e42z8Gf8TJ7B
         9ykc/grdjD1xcQMZCA6E+fc91CBDOmbtCgACWPVCnWpu0zSALSDH/08ciPP4sgZBboA7
         vOrXoR0bVFZzqP/0cSThUqiYQ/V2lsHZsn5fCQJjybMbvY6j3JaN835+evbVciV5lVKI
         6eSbo4En8EfXezy5K8aKwoVj5I/r+xV7AQRaJKMG8fiF+WuG4SGTs7m0uEtCFpx4I60i
         wGJbejchXO1j8YcbheFDOembvYM6FcQSBnnTpXLLIucotEpU18Npa9FixrE0BVG3gac8
         XIaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=NNakOyTZYGeX/Mb0ow/VeI5lJM8wKzbBa+NgDqXeOU8=;
        b=okXuP0z2V76cEW4jJWuGKTv+g4YJpY9HKiHsgJfotp8CjBO+yR7ae76dHpUZYnde5Z
         Fswe8wgLMRMhTULn0cOhGKFJG7HJ4LqyoL+GqxgLd5SiTXAsyE3MuFBKOQ69Mc76xLuB
         y+Dfoml2GRehyy2Xy5qUq2mUy6jvaMmmqb9Zm6ZQcZyhs4r0+P23WYdVFN+N/HWnZnmF
         w+gg3l2Imai40ynP9OyMCxbSA7aTX1poEJR7z3lcD7vcgkUc4oCM9DhTcRJRF50XVrTx
         1zemR76drWML18bcO/U3tMEXPSPEWknjFe+ELkC6QfjUFljFAZ6sQpMS4+3RY45FdR+N
         C2Yg==
X-Gm-Message-State: AOAM531EyVGmysNgr0A/ZkpYWO76M5kNsBhLvwERoLBvxC3nB5BaFSnh
	/Lr7P5LhzcQrTlHRORIMphM=
X-Google-Smtp-Source: ABdhPJwFbrXFBdbMbM9F7/OkBTJLFxXjBBygzt8cz5jlzteoOF6YfShY252y1hyqZU8/RuE5gM1AhQ==
X-Received: by 2002:adf:eb12:: with SMTP id s18mr4669884wrn.97.1632814455618;
        Tue, 28 Sep 2021 00:34:15 -0700 (PDT)
Message-ID: <d300aaf7-dbe9-1bf8-6abc-1693bb13ec56@gmail.com>
Date: Tue, 28 Sep 2021 08:34:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Reply-To: paul@xen.org
Subject: Re: [PATCH v8 4/6] AMD/IOMMU: respect AtsDisabled device flag
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 22/09/2021 15:37, Jan Beulich wrote:
> IVHD entries may specify that ATS is to be blocked for a device or range
> of devices. Honor firmware telling us so.
> 
> While adding respective checks I noticed that the 2nd conditional in
> amd_iommu_setup_domain_device() failed to check the IOMMU's capability.
> Add the missing part of the condition there, as no good can come from
> enabling ATS on a device when the IOMMU is not capable of dealing with
> ATS requests.
> 
> For actually using ACPI_IVHD_ATS_DISABLED, make its expansion no longer
> exhibit UB.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> TBD: As an alternative to adding the missing IOMMU capability check, we
>       may want to consider simply using dte->i in the 2nd conditional in
>       amd_iommu_setup_domain_device().
> Note that while ATS enabling/disabling gets invoked without any locks
> held, the two functions should not be possible to race with one another
> for any individual device (or else we'd be in trouble already, as ATS
> might then get re-enabled immediately after it was disabled, with the
> DTE out of sync with this setting).
> ---
> v7: New.
> 
> --- a/xen/drivers/passthrough/amd/iommu.h
> +++ b/xen/drivers/passthrough/amd/iommu.h
> @@ -120,6 +120,7 @@ struct ivrs_mappings {
>       uint16_t dte_requestor_id;
>       bool valid:1;
>       bool dte_allow_exclusion:1;
> +    bool block_ats:1;
>   
>       /* ivhd device data settings */
>       uint8_t device_flags;
> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
> @@ -55,8 +55,8 @@ union acpi_ivhd_device {
>   };
>   
>   static void __init add_ivrs_mapping_entry(
> -    uint16_t bdf, uint16_t alias_id, uint8_t flags, bool alloc_irt,
> -    struct amd_iommu *iommu)
> +    uint16_t bdf, uint16_t alias_id, uint8_t flags, unsigned int ext_flags,
> +    bool alloc_irt, struct amd_iommu *iommu)
>   {
>       struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(iommu->seg);
>   
> @@ -66,6 +66,7 @@ static void __init add_ivrs_mapping_entr
>       ivrs_mappings[bdf].dte_requestor_id = alias_id;
>   
>       /* override flags for range of devices */
> +    ivrs_mappings[bdf].block_ats = ext_flags & ACPI_IVHD_ATS_DISABLED;
>       ivrs_mappings[bdf].device_flags = flags;

I'm assuming the above indentation problem (which appears to be repeated 
in a few places below) is more to do with patch email generation rather 
than the actual code modifications so...

Reviewed-by: Paul Durrant <paul@xen.org>

>   
>       /* Don't map an IOMMU by itself. */
> @@ -499,7 +500,7 @@ static u16 __init parse_ivhd_device_sele
>           return 0;
>       }
>   
> -    add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, false,
> +    add_ivrs_mapping_entry(bdf, bdf, select->header.data_setting, 0, false,
>                              iommu);
>   
>       return sizeof(*select);
> @@ -545,7 +546,7 @@ static u16 __init parse_ivhd_device_rang
>       AMD_IOMMU_DEBUG(" Dev_Id Range: %#x -> %#x\n", first_bdf, last_bdf);
>   
>       for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
> -        add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting,
> +        add_ivrs_mapping_entry(bdf, bdf, range->start.header.data_setting, 0,
>                                  false, iommu);
>   
>       return dev_length;
> @@ -580,7 +581,7 @@ static u16 __init parse_ivhd_device_alia
>   
>       AMD_IOMMU_DEBUG(" Dev_Id Alias: %#x\n", alias_id);
>   
> -    add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, true,
> +    add_ivrs_mapping_entry(bdf, alias_id, alias->header.data_setting, 0, true,
>                              iommu);
>   
>       return dev_length;
> @@ -636,7 +637,7 @@ static u16 __init parse_ivhd_device_alia
>   
>       for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
>           add_ivrs_mapping_entry(bdf, alias_id, range->alias.header.data_setting,
> -                               true, iommu);
> +                               0, true, iommu);
>   
>       return dev_length;
>   }
> @@ -661,7 +662,8 @@ static u16 __init parse_ivhd_device_exte
>           return 0;
>       }
>   
> -    add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting, false, iommu);
> +    add_ivrs_mapping_entry(bdf, bdf, ext->header.data_setting,
> +                           ext->extended_data, false, iommu);
>   
>       return dev_length;
>   }
> @@ -708,7 +710,7 @@ static u16 __init parse_ivhd_device_exte
>   
>       for ( bdf = first_bdf; bdf <= last_bdf; bdf++ )
>           add_ivrs_mapping_entry(bdf, bdf, range->extended.header.data_setting,
> -                               false, iommu);
> +                               range->extended.extended_data, false, iommu);
>   
>       return dev_length;
>   }
> @@ -800,7 +802,7 @@ static u16 __init parse_ivhd_device_spec
>   
>       AMD_IOMMU_DEBUG("IVHD Special: %pp variety %#x handle %#x\n",
>                       &PCI_SBDF2(seg, bdf), special->variety, special->handle);
> -    add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, true,
> +    add_ivrs_mapping_entry(bdf, bdf, special->header.data_setting, 0, true,
>                              iommu);
>   
>       switch ( special->variety )
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -105,6 +105,7 @@ static int __must_check amd_iommu_setup_
>       int req_id, valid = 1, rc;
>       u8 bus = pdev->bus;
>       struct domain_iommu *hd = dom_iommu(domain);
> +    const struct ivrs_mappings *ivrs_dev;
>   
>       if ( QUARANTINE_SKIP(domain) )
>           return 0;
> @@ -122,20 +123,18 @@ static int __must_check amd_iommu_setup_
>       req_id = get_dma_requestor_id(iommu->seg, PCI_BDF2(bus, devfn));
>       table = iommu->dev_table.buffer;
>       dte = &table[req_id];
> +    ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id];
>   
>       spin_lock_irqsave(&iommu->lock, flags);
>   
>       if ( !dte->v || !dte->tv )
>       {
> -        const struct ivrs_mappings *ivrs_dev;
> -
>           /* bind DTE to domain page-tables */
>           amd_iommu_set_root_page_table(
>               dte, page_to_maddr(hd->arch.amd.root_table),
>               domain->domain_id, hd->arch.amd.paging_mode, valid);
>   
>           /* Undo what amd_iommu_disable_domain_device() may have done. */
> -        ivrs_dev = &get_ivrs_mappings(iommu->seg)[req_id];
>           if ( dte->it_root )
>           {
>               dte->int_ctl = IOMMU_DEV_TABLE_INT_CONTROL_TRANSLATED;
> @@ -146,6 +145,7 @@ static int __must_check amd_iommu_setup_
>           dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>   
>           if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> +             !ivrs_dev->block_ats &&
>                iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
>               dte->i = ats_enabled;
>   
> @@ -166,6 +166,8 @@ static int __must_check amd_iommu_setup_
>       ASSERT(pcidevs_locked());
>   
>       if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> +         !ivrs_dev->block_ats &&
> +         iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>            !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
>       {
>           if ( devfn == pdev->devfn )
> --- a/xen/include/acpi/actbl2.h
> +++ b/xen/include/acpi/actbl2.h
> @@ -851,7 +851,7 @@ struct acpi_ivrs_device8b {
>   
>   /* Values for extended_data above */
>   
> -#define ACPI_IVHD_ATS_DISABLED      (1<<31)
> +#define ACPI_IVHD_ATS_DISABLED      (1u << 31)
>   
>   /* Type 72: 8-byte device entry */
>   
> 



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:34:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:34:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197680.350873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7dH-0003Vt-5y; Tue, 28 Sep 2021 07:34:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197680.350873; Tue, 28 Sep 2021 07:34:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7dH-0003Vj-28; Tue, 28 Sep 2021 07:34:31 +0000
Received: by outflank-mailman (input) for mailman id 197680;
 Tue, 28 Sep 2021 07:34:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV7dF-0003SM-Sk
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:34:29 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 013943a2-4696-4d6b-9715-1c81b6345570;
 Tue, 28 Sep 2021 07:34:29 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C07DED6E;
 Tue, 28 Sep 2021 00:34:28 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B657B3F7B4;
 Tue, 28 Sep 2021 00:34:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 013943a2-4696-4d6b-9715-1c81b6345570
Subject: Re: [PATCH v2 08/11] vpci/header: Emulate PCI_COMMAND register for
 guests
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-9-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <c888991c-03a5-1d89-8714-f2794f1f4110@arm.com>
Date: Tue, 28 Sep 2021 09:34:19 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-9-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add basic emulation support for guests. At the moment only emulate
> PCI_COMMAND_INTX_DISABLE bit, the rest is not emulated yet and left
> as TODO.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> New in v2
> ---
>  xen/drivers/vpci/header.c | 35 ++++++++++++++++++++++++++++++++---
>  1 file changed, 32 insertions(+), 3 deletions(-)
> 

Reviewed-by: Michal Orzel <michal.orzel@arm.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:35:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:35:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197690.350883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7dy-0004Hl-Fy; Tue, 28 Sep 2021 07:35:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197690.350883; Tue, 28 Sep 2021 07:35:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7dy-0004He-Cq; Tue, 28 Sep 2021 07:35:14 +0000
Received: by outflank-mailman (input) for mailman id 197690;
 Tue, 28 Sep 2021 07:35:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j36k=OS=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mV7dw-0004G5-Lh
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:35:12 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 438346a3-6141-4d33-a93c-72bd64e9f6c3;
 Tue, 28 Sep 2021 07:35:04 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id i25so89231077lfg.6
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:35:04 -0700 (PDT)
Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id f10sm2154364ljp.55.2021.09.28.00.35.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 00:35:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 438346a3-6141-4d33-a93c-72bd64e9f6c3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=IvAnog9ZMyoFxSzlrPH+aaqQDW9SoDLgJFa60ylsI58=;
        b=PyMpk5bYkEid5T8O3pFG4k4h7C803IB7hNrmsqdMxsB5+YvP3zOhv2lzlyfG4auHzw
         2EVp5wWhIxysgpu/clFpQLuxJ1YC8u0K3z17cXcXP4piFMML30OxffYXe87FZj2lqvxc
         GfmSsePDbq+WfS+fEk/S2c0NwqQl3ekwpQSZqLaC9MNjNVBHTY6xLgiShtaLnI3efBaK
         vuzDiwc16ypOduzCuWmsLlEcaoIwGSUaQgGwYy9+US33EVsBUShCETevDmKrmG0L6m8v
         2gzl/AsXIkYapdT6ZOe5zR65aapubEPpZaA/sYQvEPq7RAl+LoIBaWTP5xGh3hor4Thg
         hN5A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=IvAnog9ZMyoFxSzlrPH+aaqQDW9SoDLgJFa60ylsI58=;
        b=QBg6qIVVYo/Db77+rPB08G1IwGNH4N/hYVtIN5WEWPXPbPuhXznrvElMznW6ycqlOc
         ijOjJgEKrqFuhTCOwVp73wE4m5HsKQ9yjtbnktSFC5A52v9TMEqToPxs5zdXbs8Yrmj4
         htkCOUWZpmPK2Kxyx8Sj0wV23qj3RfU1zVKxHh/MClSg7B/yOJ8FI8GuZ9hjOE2TDdmo
         W6okWlHxhBvxJWl5jhLy5mCuI9Nt2D/wxyCJinpJCBWZNeNKhQlBe/n2HV0CPiNxhJDF
         bzd7gj1pfBGW216ik/yNRB60WYZOztP55A1jNSU9JtdRytfHwaDiN6+/3S0jxD1qUmZ9
         2MhQ==
X-Gm-Message-State: AOAM533jn1frRJOP43kz7pY09Yzc2JKvsar9iSf14/DjQoQHHEqdDVID
	HKR1FQGQeglLvKKYwxJgheeYRlP9m+ixQg==
X-Google-Smtp-Source: ABdhPJwNuRnkJQcKmALDAYdnehNpGf2il6vwJMsKdbZLdh6bNsL/IlAX3qK3pOgr/3h+f2132Isu5w==
X-Received: by 2002:a2e:3018:: with SMTP id w24mr4234626ljw.352.1632814502928;
        Tue, 28 Sep 2021 00:35:02 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com,
	jgross@suse.com,
	julien@xen.org,
	sstabellini@kernel.org,
	jbeulich@suse.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Subject: [PATCH v5] xen-pciback: allow compiling on other archs than x86
Date: Tue, 28 Sep 2021 10:35:01 +0300
Message-Id: <20210928073501.433559-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen-pciback driver was designed to be built for x86 only. But it
can also be used by other architectures, e.g. Arm.

Currently PCI backend implements multiple functionalities at a time,
such as:
1. It is used as a database for assignable PCI devices, e.g. xl
   pci-assignable-{add|remove|list} manipulates that list. So, whenever
   the toolstack needs to know which PCI devices can be passed through
   it reads that from the relevant sysfs entries of the pciback.
2. It is used to hold the unbound PCI devices list, e.g. when passing
   through a PCI device it needs to be unbound from the relevant device
   driver and bound to pciback (strictly speaking it is not required
   that the device is bound to pciback, but pciback is again used as a
   database of the passed through PCI devices, so we can re-bind the
   devices back to their original drivers when guest domain shuts down)
3. Device reset for the devices being passed through
4. Para-virtualised use-cases support

The para-virtualised part of the driver is not always needed as some
architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend
model for PCI device passthrough.

For such use-cases make the very first step in splitting the
xen-pciback driver into two parts: Xen PCI stub and PCI PV backend
drivers.

For that add new configuration options CONFIG_XEN_PCI_STUB and
CONFIG_XEN_PCIDEV_STUB, so the driver can be limited in its
functionality, e.g. no support for para-virtualised scenario.
x86 platform will continue using CONFIG_XEN_PCIDEV_BACKEND for the
fully featured backend driver.

Please note, that CONFIG_XEN_PCIDEV_BACKEND and CONFIG_XEN_PCIDEV_STUB
are mutually exclusive.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes since v4:
- squashed two patches
- added comments to Makefile
Changes since v3:
 - Fix >>32 shift for 32-bit architectures
Changes since v2:
 - swap the patch order
Since v1:
 - Do not move pci_xen_initial_domain as it is x86 specific
---
 arch/x86/include/asm/xen/pci.h              | 16 -----
 arch/x86/pci/xen.c                          | 74 +-------------------
 drivers/xen/Kconfig                         | 24 +++++++
 drivers/xen/Makefile                        |  2 +-
 drivers/xen/pci.c                           | 75 +++++++++++++++++++++
 drivers/xen/xen-pciback/Makefile            |  7 ++
 drivers/xen/xen-pciback/conf_space_header.c |  8 ++-
 drivers/xen/xen-pciback/pci_stub.c          |  3 +-
 drivers/xen/xen-pciback/pciback.h           |  5 ++
 drivers/xen/xen-pciback/xenbus.c            |  8 ++-
 include/xen/pci.h                           | 28 ++++++++
 11 files changed, 155 insertions(+), 95 deletions(-)
 create mode 100644 include/xen/pci.h

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 3506d8c598c1..4b08c2b809ea 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -16,27 +16,11 @@ static inline int pci_xen_hvm_init(void)
 #endif
 #if defined(CONFIG_XEN_DOM0)
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
 	return -1;
 }
-static inline int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
-static inline int xen_register_device_domain_owner(struct pci_dev *dev,
-						   uint16_t domain)
-{
-	return -1;
-}
-static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	return -1;
-}
 #endif
 
 #if defined(CONFIG_PCI_MSI)
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 3d41a09c2c14..4a45b0bf9ae4 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -23,6 +23,7 @@
 
 #include <xen/features.h>
 #include <xen/events.h>
+#include <xen/pci.h>
 #include <asm/xen/pci.h>
 #include <asm/xen/cpuid.h>
 #include <asm/apic.h>
@@ -583,77 +584,4 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
-
-struct xen_device_domain_owner {
-	domid_t domain;
-	struct pci_dev *dev;
-	struct list_head list;
-};
-
-static DEFINE_SPINLOCK(dev_domain_list_spinlock);
-static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
-
-static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	list_for_each_entry(owner, &dev_domain_list, list) {
-		if (owner->dev == dev)
-			return owner;
-	}
-	return NULL;
-}
-
-int xen_find_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-	int domain = -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (owner)
-		domain = owner->domain;
-	spin_unlock(&dev_domain_list_spinlock);
-	return domain;
-}
-EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
-
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
-{
-	struct xen_device_domain_owner *owner;
-
-	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
-	if (!owner)
-		return -ENODEV;
-
-	spin_lock(&dev_domain_list_spinlock);
-	if (find_device(dev)) {
-		spin_unlock(&dev_domain_list_spinlock);
-		kfree(owner);
-		return -EEXIST;
-	}
-	owner->domain = domain;
-	owner->dev = dev;
-	list_add_tail(&owner->list, &dev_domain_list);
-	spin_unlock(&dev_domain_list_spinlock);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
-
-int xen_unregister_device_domain_owner(struct pci_dev *dev)
-{
-	struct xen_device_domain_owner *owner;
-
-	spin_lock(&dev_domain_list_spinlock);
-	owner = find_device(dev);
-	if (!owner) {
-		spin_unlock(&dev_domain_list_spinlock);
-		return -ENODEV;
-	}
-	list_del(&owner->list);
-	spin_unlock(&dev_domain_list_spinlock);
-	kfree(owner);
-	return 0;
-}
-EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
 #endif
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 22f5aff0c136..75b9330dc6fd 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -181,10 +181,34 @@ config SWIOTLB_XEN
 	select DMA_OPS
 	select SWIOTLB
 
+config XEN_PCI_STUB
+	bool
+
+config XEN_PCIDEV_STUB
+	tristate "Xen PCI-device stub driver"
+	depends on PCI && !X86 && XEN
+	depends on XEN_BACKEND
+	select XEN_PCI_STUB
+	default m
+	help
+	  The PCI device stub driver provides limited version of the PCI
+	  device backend driver without para-virtualized support for guests.
+	  If you select this to be a module, you will need to make sure no
+	  other driver has bound to the device(s) you want to make visible to
+	  other guests.
+
+	  The "hide" parameter (only applicable if backend driver is compiled
+	  into the kernel) allows you to bind the PCI devices to this module
+	  from the default device drivers. The argument is the list of PCI BDFs:
+	  xen-pciback.hide=(03:00.0)(04:00.0)
+
+	  If in doubt, say m.
+
 config XEN_PCIDEV_BACKEND
 	tristate "Xen PCI-device backend driver"
 	depends on PCI && X86 && XEN
 	depends on XEN_BACKEND
+	select XEN_PCI_STUB
 	default m
 	help
 	  The PCI device backend driver allows the kernel to export arbitrary
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3434593455b2..5aae66e638a7 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -24,7 +24,7 @@ obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
 obj-$(CONFIG_XEN_PVHVM_GUEST)		+= platform-pci.o
 obj-$(CONFIG_SWIOTLB_XEN)		+= swiotlb-xen.o
 obj-$(CONFIG_XEN_MCE_LOG)		+= mcelog.o
-obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= xen-pciback/
+obj-$(CONFIG_XEN_PCI_STUB)	        += xen-pciback/
 obj-$(CONFIG_XEN_PRIVCMD)		+= xen-privcmd.o
 obj-$(CONFIG_XEN_ACPI_PROCESSOR)	+= xen-acpi-processor.o
 obj-$(CONFIG_XEN_EFI)			+= efi.o
diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
index 224df03ce42e..fc8c1249d49f 100644
--- a/drivers/xen/pci.c
+++ b/drivers/xen/pci.c
@@ -254,3 +254,78 @@ static int xen_mcfg_late(void)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_XEN_DOM0
+struct xen_device_domain_owner {
+	domid_t domain;
+	struct pci_dev *dev;
+	struct list_head list;
+};
+
+static DEFINE_SPINLOCK(dev_domain_list_spinlock);
+static struct list_head dev_domain_list = LIST_HEAD_INIT(dev_domain_list);
+
+static struct xen_device_domain_owner *find_device(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	list_for_each_entry(owner, &dev_domain_list, list) {
+		if (owner->dev == dev)
+			return owner;
+	}
+	return NULL;
+}
+
+int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+	int domain = -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (owner)
+		domain = owner->domain;
+	spin_unlock(&dev_domain_list_spinlock);
+	return domain;
+}
+EXPORT_SYMBOL_GPL(xen_find_device_domain_owner);
+
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain)
+{
+	struct xen_device_domain_owner *owner;
+
+	owner = kzalloc(sizeof(struct xen_device_domain_owner), GFP_KERNEL);
+	if (!owner)
+		return -ENODEV;
+
+	spin_lock(&dev_domain_list_spinlock);
+	if (find_device(dev)) {
+		spin_unlock(&dev_domain_list_spinlock);
+		kfree(owner);
+		return -EEXIST;
+	}
+	owner->domain = domain;
+	owner->dev = dev;
+	list_add_tail(&owner->list, &dev_domain_list);
+	spin_unlock(&dev_domain_list_spinlock);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_register_device_domain_owner);
+
+int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	struct xen_device_domain_owner *owner;
+
+	spin_lock(&dev_domain_list_spinlock);
+	owner = find_device(dev);
+	if (!owner) {
+		spin_unlock(&dev_domain_list_spinlock);
+		return -ENODEV;
+	}
+	list_del(&owner->list);
+	spin_unlock(&dev_domain_list_spinlock);
+	kfree(owner);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
+#endif
diff --git a/drivers/xen/xen-pciback/Makefile b/drivers/xen/xen-pciback/Makefile
index e8d981d43235..d63df09de81c 100644
--- a/drivers/xen/xen-pciback/Makefile
+++ b/drivers/xen/xen-pciback/Makefile
@@ -1,5 +1,12 @@
 # SPDX-License-Identifier: GPL-2.0
+
+# N.B. The below cannot be expressed with a single line using
+# CONFIG_XEN_PCI_STUB as it always remains in "y" state,
+# thus preventing the driver to be built as a module.
+# Please note, that CONFIG_XEN_PCIDEV_BACKEND and
+# CONFIG_XEN_PCIDEV_STUB are mutually exclusive.
 obj-$(CONFIG_XEN_PCIDEV_BACKEND) += xen-pciback.o
+obj-$(CONFIG_XEN_PCIDEV_STUB) += xen-pciback.o
 
 xen-pciback-y := pci_stub.o pciback_ops.o xenbus.o
 xen-pciback-y += conf_space.o conf_space_header.o \
diff --git a/drivers/xen/xen-pciback/conf_space_header.c b/drivers/xen/xen-pciback/conf_space_header.c
index ac45cdc38e85..981435103af1 100644
--- a/drivers/xen/xen-pciback/conf_space_header.c
+++ b/drivers/xen/xen-pciback/conf_space_header.c
@@ -236,8 +236,12 @@ static void *bar_init(struct pci_dev *dev, int offset)
 	else {
 		pos = (offset - PCI_BASE_ADDRESS_0) / 4;
 		if (pos && (res[pos - 1].flags & IORESOURCE_MEM_64)) {
-			bar->val = res[pos - 1].start >> 32;
-			bar->len_val = -resource_size(&res[pos - 1]) >> 32;
+			/*
+			 * Use ">> 16 >> 16" instead of direct ">> 32" shift
+			 * to avoid warnings on 32-bit architectures.
+			 */
+			bar->val = res[pos - 1].start >> 16 >> 16;
+			bar->len_val = -resource_size(&res[pos - 1]) >> 16 >> 16;
 			return bar;
 		}
 	}
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index f8e4faa96ad6..bba527620507 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -19,7 +19,8 @@
 #include <linux/sched.h>
 #include <linux/atomic.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
+#include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 #include <xen/interface/physdev.h>
 #include "pciback.h"
diff --git a/drivers/xen/xen-pciback/pciback.h b/drivers/xen/xen-pciback/pciback.h
index 95e28ee48d52..9a64196e831d 100644
--- a/drivers/xen/xen-pciback/pciback.h
+++ b/drivers/xen/xen-pciback/pciback.h
@@ -71,6 +71,11 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
 				    struct pci_dev *dev);
 void pcistub_put_pci_dev(struct pci_dev *dev);
 
+static inline bool xen_pcibk_pv_support(void)
+{
+	return IS_ENABLED(CONFIG_XEN_PCIDEV_BACKEND);
+}
+
 /* Ensure a device is turned off or reset */
 void xen_pcibk_reset_device(struct pci_dev *pdev);
 
diff --git a/drivers/xen/xen-pciback/xenbus.c b/drivers/xen/xen-pciback/xenbus.c
index c09c7ebd6968..bde63ef677b8 100644
--- a/drivers/xen/xen-pciback/xenbus.c
+++ b/drivers/xen/xen-pciback/xenbus.c
@@ -14,7 +14,7 @@
 #include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
-#include <asm/xen/pci.h>
+#include <xen/pci.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
@@ -743,6 +743,9 @@ const struct xen_pcibk_backend *__read_mostly xen_pcibk_backend;
 
 int __init xen_pcibk_xenbus_register(void)
 {
+	if (!xen_pcibk_pv_support())
+		return 0;
+
 	xen_pcibk_backend = &xen_pcibk_vpci_backend;
 	if (passthrough)
 		xen_pcibk_backend = &xen_pcibk_passthrough_backend;
@@ -752,5 +755,6 @@ int __init xen_pcibk_xenbus_register(void)
 
 void __exit xen_pcibk_xenbus_unregister(void)
 {
-	xenbus_unregister_driver(&xen_pcibk_driver);
+	if (xen_pcibk_pv_support())
+		xenbus_unregister_driver(&xen_pcibk_driver);
 }
diff --git a/include/xen/pci.h b/include/xen/pci.h
new file mode 100644
index 000000000000..b8337cf85fd1
--- /dev/null
+++ b/include/xen/pci.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __XEN_PCI_H__
+#define __XEN_PCI_H__
+
+#if defined(CONFIG_XEN_DOM0)
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
+static inline int xen_find_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+
+static inline int xen_register_device_domain_owner(struct pci_dev *dev,
+						   uint16_t domain)
+{
+	return -1;
+}
+
+static inline int xen_unregister_device_domain_owner(struct pci_dev *dev)
+{
+	return -1;
+}
+#endif
+
+#endif
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:36:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:36:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197698.350894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7fb-00050Q-Vc; Tue, 28 Sep 2021 07:36:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197698.350894; Tue, 28 Sep 2021 07:36:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7fb-00050J-Sc; Tue, 28 Sep 2021 07:36:55 +0000
Received: by outflank-mailman (input) for mailman id 197698;
 Tue, 28 Sep 2021 07:36:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sHmr=OS=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mV7fa-00050D-OS
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:36:54 +0000
Received: from mail-wr1-x42d.google.com (unknown [2a00:1450:4864:20::42d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42772453-5762-4ab7-b011-07ecb0886482;
 Tue, 28 Sep 2021 07:36:54 +0000 (UTC)
Received: by mail-wr1-x42d.google.com with SMTP id u18so56337193wrg.5
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:36:54 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id h15sm18422829wrc.19.2021.09.28.00.36.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 00:36:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42772453-5762-4ab7-b011-07ecb0886482
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=v5ug0LzsbMRQwh3CsWK83e691mVxV+MasCT39d/p1F0=;
        b=RGhIruELcEu6cVD9rU8Q4GeBVWwDr6GdDYMdqR5Xn53hMzhHdb3dRiPWwDxg7Pob27
         QfRJRWoqwNm8pWqttZwBtSjpspO+n1SXnHVt3QzkpZOSS91U/SeZ3eIf9AclCfOEdUQ4
         aBi46UeEUpyhrcT/rZ7mfr6071ikP2l9A/p2emS+krfZC2E3NYc8AqV75v3EvQ0AD6xc
         epmZOnA1OJLUnk01D2PWuO7lgUjqQFR1uDOZUp4t9w7PSJ7pSf2W0ZfEnLqTmXz4zYcG
         ZPdbFM5t3sfmXqc3bNp4deBDRZARGqOoJ1qhL0Z0Vgo243rjxHERwLMWUHLQKyZQYbjV
         pt2w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=v5ug0LzsbMRQwh3CsWK83e691mVxV+MasCT39d/p1F0=;
        b=u5kEch72vxxl1v9x1YrzozmIBtowyddapKIsB3nB8eQaqPxaUPM+MgcJKWO3JP/sZ4
         VxQg1o+lSQYq70uvV3WTGoyUGxIz4TcUUB3/rmyUnRAiBVfJLW37z8Cwibrm1PJuMrZw
         tiXkobbk9HXL8t4qKWlzci5OKLeGnFHS6qX27ZPvzVO1N9fO1MvRg/0HLgSAe1yiphk9
         KrfNHjCMGqrkwB7N3oe/Np5WEoLgKoRGrnEe73zqqZsFPLjeb+SY7gnw0TFcVtxRAsnZ
         3tID+E445UAbqo5SJuuCrnkDMKyzwlUUQOsU5jCg1tsMPXs85BV7zYox7P8rdV0GstyD
         lxlw==
X-Gm-Message-State: AOAM531XkAUWh/QiUtwIyNeXvvkxE7Fh2X4BDNxmLHyOvsRcVwrtD3AP
	lX/qKEkcm0rnpYXHA4PVa7E=
X-Google-Smtp-Source: ABdhPJyqnCu9GbeBSKd18QLEKWPAlanEQ0sqC5irH14fIeVOfGMhpfSMk9CD9ytV06cED/EDwd3iCw==
X-Received: by 2002:adf:dd49:: with SMTP id u9mr4672361wrm.341.1632814613332;
        Tue, 28 Sep 2021 00:36:53 -0700 (PDT)
Message-ID: <d3c3b085-4408-f797-eaa6-d3fae074e692@gmail.com>
Date: Tue, 28 Sep 2021 08:36:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Reply-To: paul@xen.org
Subject: Re: [PATCH v8 5/6] AMD/IOMMU: pull ATS disabling earlier
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <34c081bd-ac6d-055c-e6c1-eeac62851389@suse.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <34c081bd-ac6d-055c-e6c1-eeac62851389@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 22/09/2021 15:38, Jan Beulich wrote:
> Disabling should be done in the opposite order of enabling: ATS wants to
> be turned off before adjusting the DTE, just like it gets enabled only
> after the DTE was suitably prepared. Note that we want ATS to be
> disabled as soon as any of the DTEs involved in the handling of a device
> (including phantom devices) gets adjusted respectively. For this reason
> the "devfn == pdev->devfn" of the original conditional gets dropped.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Paul Durrant <paul@xen.org>


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:38:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:38:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197705.350905 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7gs-0005nN-AM; Tue, 28 Sep 2021 07:38:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197705.350905; Tue, 28 Sep 2021 07:38:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7gs-0005nE-7F; Tue, 28 Sep 2021 07:38:14 +0000
Received: by outflank-mailman (input) for mailman id 197705;
 Tue, 28 Sep 2021 07:38:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV7gr-0005n2-QT
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:38:13 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 08ec92cd-202f-11ec-bc6f-12813bfff9fa;
 Tue, 28 Sep 2021 07:38:12 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 65D68D6E;
 Tue, 28 Sep 2021 00:38:12 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 70D2C3F7B4;
 Tue, 28 Sep 2021 00:38:09 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08ec92cd-202f-11ec-bc6f-12813bfff9fa
Subject: Re: [PATCH v2 09/11] vpci/header: Reset the command register when
 adding devices
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-10-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <f55e0c0f-5843-ba5f-aad7-95cecf5003f8@arm.com>
Date: Tue, 28 Sep 2021 09:38:03 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-10-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Reset the command register when passing through a PCI device:
> it is possible that when passing through a PCI device its memory
> decoding bits in the command register are already set. Thus, a
> guest OS may not write to the command register to update memory
> decoding, so guest mappings (guest's view of the BARs) are
> left not updated.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - do not write 0 to the command register, but respect host settings.
> ---
>  xen/drivers/vpci/header.c | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 

Reviewed-by: Michal Orzel <michal.orzel@arm.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:42:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:42:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197711.350916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7lI-0007CD-Si; Tue, 28 Sep 2021 07:42:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197711.350916; Tue, 28 Sep 2021 07:42:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7lI-0007C6-Ps; Tue, 28 Sep 2021 07:42:48 +0000
Received: by outflank-mailman (input) for mailman id 197711;
 Tue, 28 Sep 2021 07:42:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sHmr=OS=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mV7lI-0007C0-3F
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:42:48 +0000
Received: from mail-wm1-x334.google.com (unknown [2a00:1450:4864:20::334])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6a38e9ef-717e-4fdd-a30e-2fb3b88cbf87;
 Tue, 28 Sep 2021 07:42:46 +0000 (UTC)
Received: by mail-wm1-x334.google.com with SMTP id v127so2648968wme.5
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:42:46 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id o5sm7528635wrf.85.2021.09.28.00.42.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 00:42:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a38e9ef-717e-4fdd-a30e-2fb3b88cbf87
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=gCYg6qV9LtwuAYek9WtbGywdKoMVO4jikqVXZIdqHBc=;
        b=VG7AaA0U9BscB+iS7fD7aYibTgB7yHa3LN0E7eOAgv8Y6i/Z2xXZPSCg+ywfiwONPQ
         S5aaXVrAsWdd1GTTEejAlyEjZ9clFZyRNgaNedh1WQlmuCp4u96gkHNcZ4Whv+5t5aVL
         Zuoj/jJV/5jYmjW1JvXhcyHYVKjZC92GKW/waeEg3Lew8Yyba8w0CbHdIOMouQLJMCIh
         RKV1iPWRi2+V42Wrr15IEy9AhRKth1oQXzETGt5Sc7U7YXax1Lrvbp6lNnEITm8b0irN
         Fs5NxjXR6HYrEBQjS9JtrNbznRNsrhfMxdxAawvlGWBgMZKTbd0fvncBchIMl0KFe9Fg
         FhSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=gCYg6qV9LtwuAYek9WtbGywdKoMVO4jikqVXZIdqHBc=;
        b=hkqEOKQeKoErgMJGKCrm317YeqO8Kvtil2xqGo0AlZnWW7Llmo5H1F15lt+YvuHKcQ
         McmHgt3cex82EigOkLPGoQDqqeWIdHT52zTDZjXjRUiVf3a6vdxmGK7n37rAYFn3K/9+
         cyKKPTaFWIdfgATXNCwQY63rM1eJpgntrQ+nlB/RJ3pMM8X9A8qgKO+GO2kr6c+BcoqT
         XUZqPw7akZGN+aFafQnsZz6+fWCButPlYhTeMr59eAlFv6NX1rrjv3Uo0FIZUAfbEKRl
         iFg/75bdxOeKvNJzGpyffnGy/Qg0zI1g1xo6kJUMw9AfhbnGt+vJzZe0L/DKGA/wRZrk
         I7oA==
X-Gm-Message-State: AOAM5325djCsXsJbNF01tdlaYGyDSLm5/xceOFktbIPX7OcNv2jGd9a4
	QYQOwDVH/7Q1sP2yBacz1Dc=
X-Google-Smtp-Source: ABdhPJzHavlBPaI4vnHVFpOZZRr1LMDbRv+IL4vPXk3W95yWQVW8ymuxDZyC5utj0kGm6sk2Kxz9yg==
X-Received: by 2002:a7b:c932:: with SMTP id h18mr3229680wml.112.1632814965924;
        Tue, 28 Sep 2021 00:42:45 -0700 (PDT)
Message-ID: <ba9fe216-32a8-ef6d-173b-4f0d31a20db2@gmail.com>
Date: Tue, 28 Sep 2021 08:42:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Reply-To: paul@xen.org
Subject: Re: [PATCH v8 6/6] AMD/IOMMU: expose errors and warnings
 unconditionally
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <dc0cd7f7-a313-099d-3e89-e3862ed11f43@suse.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <dc0cd7f7-a313-099d-3e89-e3862ed11f43@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 22/09/2021 15:38, Jan Beulich wrote:
> Making these dependent upon "iommu=debug" isn't really helpful in the
> field. Where touching respective code anyway also make use of %pp and
> %pd.
> 
> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Paul Durrant <paul@xen.org>

... with one nit below...

> ---
[snip]
> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
> @@ -174,7 +174,7 @@ static int __init reserve_unity_map_for_
>           if ( unity_map->addr + unity_map->length > base &&
>                base + length > unity_map->addr )
>           {
> -            AMD_IOMMU_DEBUG("IVMD Error: overlap [%lx,%lx) vs [%lx,%lx)\n",
> +            AMD_IOMMU_ERROR("IVMD: overlap [%lx,%lx) vs [%lx,%lx)\n",
>                               base, base + length, unity_map->addr,
>                               unity_map->addr + unity_map->length);
>               return -EPERM;
> @@ -248,7 +248,7 @@ static int __init register_range_for_dev
>       iommu = find_iommu_for_device(seg, bdf);
>       if ( !iommu )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x!\n", bdf);
> +        AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x\n", bdf);
>           return -ENODEV;
>       }
>       req = ivrs_mappings[bdf].dte_requestor_id;
> @@ -318,7 +318,7 @@ static int __init parse_ivmd_device_sele
>       bdf = ivmd_block->header.device_id;
>       if ( bdf >= ivrs_bdf_entries )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: Invalid Dev_Id %#x\n", bdf);
> +        AMD_IOMMU_ERROR("IVMD: invalid Dev_Id %#x\n", bdf);
>           return -ENODEV;
>       }
>   
> @@ -335,16 +335,14 @@ static int __init parse_ivmd_device_rang
>       first_bdf = ivmd_block->header.device_id;
>       if ( first_bdf >= ivrs_bdf_entries )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: "
> -                        "Invalid Range_First Dev_Id %#x\n", first_bdf);
> +        AMD_IOMMU_ERROR("IVMD: invalid Range_First Dev_Id %#x\n", first_bdf);
>           return -ENODEV;
>       }
>   
>       last_bdf = ivmd_block->aux_data;
>       if ( (last_bdf >= ivrs_bdf_entries) || (last_bdf <= first_bdf) )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: "
> -                        "Invalid Range_Last Dev_Id %#x\n", last_bdf);
> +        AMD_IOMMU_ERROR("IVMD: invalid Range_Last Dev_Id %#x\n", last_bdf);
>           return -ENODEV;
>       }
>   
> @@ -367,7 +365,7 @@ static int __init parse_ivmd_device_iomm
>                                       ivmd_block->aux_data);
>       if ( !iommu )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: No IOMMU for Dev_Id %#x Cap %#x\n",
> +        AMD_IOMMU_ERROR("IVMD: no IOMMU for Dev_Id %#x Cap %#x\n",
>                           ivmd_block->header.device_id, ivmd_block->aux_data);
>           return -ENODEV;
>       }
> @@ -384,7 +382,7 @@ static int __init parse_ivmd_block(const
>   
>       if ( ivmd_block->header.length < sizeof(*ivmd_block) )
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Length!\n");
> +        AMD_IOMMU_ERROR("IVMD: invalid block length\n");
>           return -ENODEV;
>       }
>   
> @@ -402,8 +400,8 @@ static int __init parse_ivmd_block(const
>            (addr_bits < BITS_PER_LONG &&
>             ((start_addr + mem_length - 1) >> addr_bits)) )
>       {
> -        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not IOMMU addressable\n",
> -                        start_addr, start_addr + mem_length);
> +        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not IOMMU addressable\n",
> +                       start_addr, start_addr + mem_length);
>           return 0;
>       }
>   
> @@ -411,8 +409,8 @@ static int __init parse_ivmd_block(const
>       {
>           paddr_t addr;
>   
> -        AMD_IOMMU_DEBUG("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
> -                        base, limit + PAGE_SIZE);
> +        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
> +                       base, limit + PAGE_SIZE);
>   
>           for ( addr = base; addr <= limit; addr += PAGE_SIZE )
>           {
> @@ -423,7 +421,7 @@ static int __init parse_ivmd_block(const
>                   if ( e820_add_range(&e820, addr, addr + PAGE_SIZE,
>                                       E820_RESERVED) )
>                       continue;
> -                AMD_IOMMU_DEBUG("IVMD Error: Page at %lx couldn't be reserved\n",
> +                AMD_IOMMU_ERROR("IVMD: page at %lx couldn't be reserved\n",
>                                   addr);
>                   return -EIO;
>               }
> @@ -433,8 +431,7 @@ static int __init parse_ivmd_block(const
>                              RAM_TYPE_UNUSABLE)) )
>                   continue;
>   
> -            AMD_IOMMU_DEBUG("IVMD Error: Page at %lx can't be converted\n",
> -                            addr);
> +            AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);
>               return -EIO;
>           }
>       }
> @@ -448,7 +445,7 @@ static int __init parse_ivmd_block(const
>       }
>       else
>       {
> -        AMD_IOMMU_DEBUG("IVMD Error: Invalid Flag Field!\n");
> +        AMD_IOMMU_ERROR("IVMD: invalid flag field\n");
>           return -ENODEV;
>       }
>   
> @@ -471,7 +468,8 @@ static int __init parse_ivmd_block(const
>                                          iw, ir, exclusion);
>   
>       default:
> -        AMD_IOMMU_DEBUG("IVMD Error: Invalid Block Type!\n");
> +        AMD_IOMMU_ERROR("IVMD: unknown block type %#x\n",
> +                        ivmd_block->header.type);
>           return -ENODEV;
>       }
>   }
> @@ -481,7 +479,7 @@ static u16 __init parse_ivhd_device_padd
>   {
>       if ( header_length < (block_length + pad_length) )
>       {
> -        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
> +        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
>           return 0;
>       }
>   
> @@ -496,7 +494,7 @@ static u16 __init parse_ivhd_device_sele
>       bdf = select->header.id;
>       if ( bdf >= ivrs_bdf_entries )
>       {
> -        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Dev_Id %#x\n", bdf);
> +        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry Dev_Id %#x\n", bdf);
>           return 0;
>       }
>   
> @@ -515,14 +513,13 @@ static u16 __init parse_ivhd_device_rang
>       dev_length = sizeof(*range);
>       if ( header_length < (block_length + dev_length) )
>       {
> -        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
> +        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
>           return 0;
>       }
>   
>       if ( range->end.header.type != ACPI_IVRS_TYPE_END )
>       {
> -        AMD_IOMMU_DEBUG("IVHD Error: "
> -                        "Invalid Range: End_Type %#x\n",
> +        AMD_IOMMU_ERROR("IVHD Error: invalid range: End_Type %#x\n",

NIT: I guess you want to drop the 'Error' here like you did elsewhere.



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:47:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:47:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197720.350927 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7pp-00081k-GK; Tue, 28 Sep 2021 07:47:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197720.350927; Tue, 28 Sep 2021 07:47:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7pp-00081d-DO; Tue, 28 Sep 2021 07:47:29 +0000
Received: by outflank-mailman (input) for mailman id 197720;
 Tue, 28 Sep 2021 07:47:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV7pn-00081X-TA
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:47:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 52a2f6f8-2030-11ec-bc71-12813bfff9fa;
 Tue, 28 Sep 2021 07:47:26 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2053.outbound.protection.outlook.com [104.47.0.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-kJ1uxjtzOjWfG3LP9cstbw-1; Tue, 28 Sep 2021 09:47:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7360.eurprd04.prod.outlook.com (2603:10a6:800:1a3::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 07:47:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 07:47:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR0P281CA0011.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Tue, 28 Sep 2021 07:47:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52a2f6f8-2030-11ec-bc71-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632815245;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LxtsIan1PrQvtx+gwpLNKBF0Z/WvVABN8IPG8rsHM0s=;
	b=e9RnNWUc53EWKgoYFPB6sv96ATzdeqzoqDp0TR++X9N9M/qCu6/isO4h9AAhaXsRzYCrZi
	7EUMvdgiCED5DRZFkRNCpnb3bw6WW3gCQfb6nMcXgSOck/rDMlya1LpP+3zd6Em751VSLI
	Et8j5D3s0UdQM+zvzn/z/tC9DEFs8lw=
X-MC-Unique: kJ1uxjtzOjWfG3LP9cstbw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Rx7RTimmKan4Xvj3IKNCHnIYPiRGMMHBF6nXBzHPRORCbf8/hq9LURACLkIfeAtX1wSK1CumzCAleXSHDsKyOENW3mbvbGkYjsuv+B6DhRf9ooMEsyrkmYc4KkMXgvOY5mMax+FEz0Bsr4H6/YWG+rr54RWaJAj9Agw4GrUxSPlj01M8OOvdJcZtg2nYdGIj4uJEPSxr94QEEdk8cKd/sQgBXP/c0DqO0AcP2tUMfQOoASJdOW2OvbUE9psWK+xfJB0+i4wZVIJSN5k4RjDt85CLrW6XbWaIYC77/kaqBctzFpP35OFIHni2U8xNy8rX/wSzgtvxO6nZgTpQvOAg8w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LxtsIan1PrQvtx+gwpLNKBF0Z/WvVABN8IPG8rsHM0s=;
 b=ZnaEN9F+ixnXvrWDnoDY9zH8d8g4RI7TXOwEjMK2lf+LG3AjxQM07qkhVnMO1fBC/8QcGptXEIpJKWhMDxsm88OWyXl+BRsBYUJiCj9xA9Mww/6RoyacBGFBuNFrfeuc1ODq2thy/N9gTmpVT98F96wKKrdx6fsK+aRDJTIc1yqPETZByaeeZr+FqO8U+919Ri4WivKSK9rq+gyiWT3kiDRG505Y0+wWpCvsJmxaynv5ihkojt0USphBdDKk1XoiGjLhIGmancRyeSgqBUblcu9Ch0seK9ILSipVDiyOKp5FfzqLe0dWJ++IZxCAlhlfRZbojOcyfm0JQu0QQqTJGQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v8 4/6] AMD/IOMMU: respect AtsDisabled device flag
To: paul@xen.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com>
 <d300aaf7-dbe9-1bf8-6abc-1693bb13ec56@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a46e77e8-7686-875e-dad1-e0713b9fd46c@suse.com>
Date: Tue, 28 Sep 2021 09:47:21 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d300aaf7-dbe9-1bf8-6abc-1693bb13ec56@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR0P281CA0011.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:15::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a262fcdd-4eea-46ad-1a8a-08d9825434f3
X-MS-TrafficTypeDiagnostic: VE1PR04MB7360:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB736047D07C721C691312C680B3A89@VE1PR04MB7360.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VwD06BVdCpJptkir0V/qgmHPavVwTAzmyLRkfm9xtAKfA+JJqeWLH02ZkTXOT3BVjwqM1D1dDlQhevsVx2HqDqlqfXDVW9FmSzwNl/qdRXaP0mMeNrSRoGSTL//EfS8vwKc60MX6Sye/WeA72aZ3DB7qBMkHX02O5v5bq9MH3aYF8syvgPlTrVZ225asMt/STVbPnP91gnJdcw22NATg4fpEZgq/B3oG04VXhamYLBVhXlyJmGDLpV1p6Jyj2/FqoN+VlwRrtEyt9FrejQNi48U1SonXN61FkBqUp1y2WF0U46cIcK1fg1TC8abLbWnE0v2qIS7jxfs9AO8BLEpgpMLYv+K3JU99WvjeH6tD7lls5ixLCQA0iJPI+bYRSqwzkhYwJCcR/5XxwaXbxvJh4dBrCOkZod0VsMI2ZCOBRij6ZWc7N+kt/NmT/lorJoRJMhwJO3RJe03sw7Xnqkd5xkJiBCtr+a46D02sCIRYNysTCSQJivajK9ygSMTrq7F0OKOGqVLnlJn/wdCrvh4Ihdsn6yYPZsuBkdXnJOqwxHdp1+wewFbliPqEDmwaUx7kETal2bgTVKn3M3V0wQHJr2g/RAKZ3o1q2WQ50MNJNbqstXOV1+P0qzlkqKwHS3jg2RDF0i6kR19XeGAjLiR5muMU6X2e1AgERBihzANWnTGJ1oCSCHCEnPQKbBLSB8QWcly9DMdoE4mkaSlNM3nnQ4NHF18SdyQxP56O7YJflo0=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(508600001)(31696002)(38100700002)(54906003)(66476007)(26005)(86362001)(5660300002)(186003)(31686004)(2616005)(66946007)(8676002)(316002)(8936002)(16576012)(53546011)(6486002)(83380400001)(2906002)(4326008)(956004)(36756003)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bGRVTGwvRTlBY1I5cGJ4ZlFxck81ZDBpOUNqeUp5VHhUZERhRHU3NXFOMTI4?=
 =?utf-8?B?QWtvOTE1b21SQ0hJZ2ZkMG9tMlBSOXJOVVh1SEZCK29MUEE4WDdrN0hzU0d6?=
 =?utf-8?B?c3ovbHFJYUNObEpFODlhblo5WlBHNjJXSnFNNGpUNy9ITGd5SU80Tmh5a0Ix?=
 =?utf-8?B?d0xtM2I2RVVERHk2SWNZU0U2dE1SRmtHY2tBZ3NCQkdVamFFb1pkYVpkbWZ4?=
 =?utf-8?B?d2lMS3pPRjlhRmNxb0djV0VVeEhZNlpiR0g1Y1hYQ2JqbFRzNjVkc3RLVkpC?=
 =?utf-8?B?TVkwWjV4MnMyZUV6UU1YSmRrZ0xLSDFhOXgxU1lML3BTVWtjRE1YSkJGR1dP?=
 =?utf-8?B?Q1hiU0tjeHBMcVFRK0VibGhYdDZ4bkRmVUtyL1BHTDFMNWYxcUV6R3dBK1N6?=
 =?utf-8?B?WXhHNlMwLzFSaEIvNVVEUmQyUDBhbGZRby8vdWpjdi8reW1xUlFjRWlwVkJi?=
 =?utf-8?B?b09WRTg3ZGJRaXVPbThuOWhYMzdzR2F2SjArMWo2YlFRSXlvcmhvRWFZQTh4?=
 =?utf-8?B?YitnZXp5VVBXNHRnamFCc3R0UFJqUzlWZ0Z3RlZuSk1LS0JyMnFzbzZ4RTJt?=
 =?utf-8?B?NFpFQWdzRVdLWEtEMXhQSVlWMzBwMVdha1Q0SzlBRVZpWG1veExWc2k0YUIz?=
 =?utf-8?B?R0IrWm84eVNPSVVUVVo2M3ZraGx4UzkvOHByUlRocklCeTBvL3V0REtDVGdx?=
 =?utf-8?B?OHg2cC9haTFXL01jYXlKLzljMERmWjM3ZU5ReWlFNTdNUUZDcFZUSUJZL2V2?=
 =?utf-8?B?b0t4cWUyMUFLUGVEdTNjZnZocCtPS1BPa3pRZ1prZzZ3Z0Z0UW1OOHhaUXRk?=
 =?utf-8?B?bzJaWHhhQ0tLenpKU25WalBHMVZyU1d5L0VZWE1jc0RkdFF0ZDI2cCs1a1FS?=
 =?utf-8?B?eVNGcTNLUnAxRTJUa2QzNVkwS0FkRDZvYkZsMFBpcjVUaEprY05qc3o3L0U0?=
 =?utf-8?B?OFJmVUlGdlJGZlZsMkgzRjBBbkJZMlBhemZKWHpybEdhVE93U1J1VjVvN2FU?=
 =?utf-8?B?OGVpVkw0WnRPczg2TE81SFpEUDZ4dUNtazhybkpLOERFUDM4MlpNeW1JeVVL?=
 =?utf-8?B?UEt4UnphQnlKWnBxZ0NPTGJqRXlMSzFvRjdJYW8xRXM0VUVVeXpZOEFIZWlp?=
 =?utf-8?B?cFgvRUQwbElZVC9uaS96bnFZYi83UXFwQVFRc2JQRlVQZkFTZFVqRzRwNDE5?=
 =?utf-8?B?MkhUOWxVMnVXZytxOThVUjA1UjR6YnlOTE9VcmcxbkJEV3lENGlpK3ZBNFBG?=
 =?utf-8?B?WlNTQWpYMUJBeHp5Vy9KVkhGKzI5cisvT1kzZjQ4YnNoR09UR2JadTRySThE?=
 =?utf-8?B?RzU2QWVtb2NNYmtHdjM3emNLbG9sZXlXOC9kaGxLWGtlODNQQkhPMk5WWmxT?=
 =?utf-8?B?QkRpdnFYMGRLZUw0S3VnVFp6ZFVDYmF0SkRmZlg4ZnQvRjdMdjhrYjNsMU5R?=
 =?utf-8?B?SVQxaCtKNnlHRnZxTnZtblA0VytxNXcrekFML0VFUzFxTEFwNEdCaC9LUXV3?=
 =?utf-8?B?aHREVlNqTEMvZU53Yk41R0ZTcFZGYnE3WkUyNXNGbnRsZ1RhRFlzV2JGOE9Z?=
 =?utf-8?B?TlJ1cDNaeUZEbktHdGxVN0gyOWx5djdZV0lETE5CZ3MvT253YmRWbzN2Y3o1?=
 =?utf-8?B?bE1kVWl3UjB3V3N6QjFTRmgva0EvVWtUL2dSSDRWekFoWlArbEJuVXVxSjRq?=
 =?utf-8?B?L09DdjRWb2FLVlNYKzBKNDBEUTZoemE3OFh1bk1UT1V1K0o3eXErdlJVenNN?=
 =?utf-8?Q?qkzZwCFS5nr2FAkR5k1Doq4RgZ6EGncO4Fgf8yB?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a262fcdd-4eea-46ad-1a8a-08d9825434f3
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 07:47:22.7976
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PUDmJEh47bgtwudZwGD4ZDCsoP7QwsPWxV8BASxW1i/gJbyf3JLmupS0n3Xsx0jt1Z7CFITgM2AZuiIuXvORtA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7360

On 28.09.2021 09:34, Durrant, Paul wrote:
> On 22/09/2021 15:37, Jan Beulich wrote:
>> IVHD entries may specify that ATS is to be blocked for a device or range
>> of devices. Honor firmware telling us so.
>>
>> While adding respective checks I noticed that the 2nd conditional in
>> amd_iommu_setup_domain_device() failed to check the IOMMU's capability.
>> Add the missing part of the condition there, as no good can come from
>> enabling ATS on a device when the IOMMU is not capable of dealing with
>> ATS requests.
>>
>> For actually using ACPI_IVHD_ATS_DISABLED, make its expansion no longer
>> exhibit UB.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> TBD: As an alternative to adding the missing IOMMU capability check, we
>>       may want to consider simply using dte->i in the 2nd conditional in
>>       amd_iommu_setup_domain_device().
>> Note that while ATS enabling/disabling gets invoked without any locks
>> held, the two functions should not be possible to race with one another
>> for any individual device (or else we'd be in trouble already, as ATS
>> might then get re-enabled immediately after it was disabled, with the
>> DTE out of sync with this setting).
>> ---
>> v7: New.
>>
>> --- a/xen/drivers/passthrough/amd/iommu.h
>> +++ b/xen/drivers/passthrough/amd/iommu.h
>> @@ -120,6 +120,7 @@ struct ivrs_mappings {
>>       uint16_t dte_requestor_id;
>>       bool valid:1;
>>       bool dte_allow_exclusion:1;
>> +    bool block_ats:1;
>>   
>>       /* ivhd device data settings */
>>       uint8_t device_flags;
>> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
>> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
>> @@ -55,8 +55,8 @@ union acpi_ivhd_device {
>>   };
>>   
>>   static void __init add_ivrs_mapping_entry(
>> -    uint16_t bdf, uint16_t alias_id, uint8_t flags, bool alloc_irt,
>> -    struct amd_iommu *iommu)
>> +    uint16_t bdf, uint16_t alias_id, uint8_t flags, unsigned int ext_flags,
>> +    bool alloc_irt, struct amd_iommu *iommu)
>>   {
>>       struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(iommu->seg);
>>   
>> @@ -66,6 +66,7 @@ static void __init add_ivrs_mapping_entr
>>       ivrs_mappings[bdf].dte_requestor_id = alias_id;
>>   
>>       /* override flags for range of devices */
>> +    ivrs_mappings[bdf].block_ats = ext_flags & ACPI_IVHD_ATS_DISABLED;
>>       ivrs_mappings[bdf].device_flags = flags;
> 
> I'm assuming the above indentation problem (which appears to be repeated 
> in a few places below) is more to do with patch email generation rather 
> than the actual code modifications so...

Hmm, I first suspected a Thunderbird regression, but checking the list
archives I don't see any corruption. I'm therefore now suspecting the
problem may be at your end ...

> Reviewed-by: Paul Durrant <paul@xen.org>

Thanks.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:48:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:48:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197727.350938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7rA-0000GL-Va; Tue, 28 Sep 2021 07:48:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197727.350938; Tue, 28 Sep 2021 07:48:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7rA-0000GE-Sf; Tue, 28 Sep 2021 07:48:52 +0000
Received: by outflank-mailman (input) for mailman id 197727;
 Tue, 28 Sep 2021 07:48:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV7r8-0000G4-Tu
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:48:50 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id abd045a8-1dce-425c-ac5c-730b4152c4dd;
 Tue, 28 Sep 2021 07:48:49 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 108C6D6E;
 Tue, 28 Sep 2021 00:48:49 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5A4ED3F7B4;
 Tue, 28 Sep 2021 00:48:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abd045a8-1dce-425c-ac5c-730b4152c4dd
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
Date: Tue, 28 Sep 2021 09:48:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-11-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Assign SBDF to the PCI devices being passed through with bus 0.
> The resulting topology is where PCIe devices reside on the bus 0 of the
> root complex itself (embedded endpoints).
> This implementation is limited to 32 devices which are allowed on
> a single PCI bus.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> New in v2
> ---
>  xen/common/domain.c           |  1 +
>  xen/drivers/passthrough/pci.c | 57 +++++++++++++++++++++++++++++++++++
>  xen/drivers/vpci/vpci.c       | 18 +++++++++--
>  xen/include/xen/pci.h         | 18 +++++++++++
>  xen/include/xen/sched.h       |  6 ++++
>  5 files changed, 97 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 40d67ec34232..b80ff2e5e2e6 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -601,6 +601,7 @@ struct domain *domain_create(domid_t domid,
>  
>  #ifdef CONFIG_HAS_PCI
>      INIT_LIST_HEAD(&d->pdev_list);
> +    INIT_LIST_HEAD(&d->vdev_list);
>  #endif
>  
>      /* All error paths can depend on the above setup. */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index e1da283d73ad..4552ace855e0 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -833,6 +833,63 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>      return ret;
>  }
>  
> +static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
> +                                                const struct pci_dev *pdev)
> +{
> +    struct vpci_dev *vdev;
> +
> +    list_for_each_entry ( vdev, &d->vdev_list, list )
> +        if ( vdev->pdev == pdev )
> +            return vdev;
> +    return NULL;
> +}
> +
> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
> +{
> +    struct vpci_dev *vdev;
> +
> +    ASSERT(!pci_find_virtual_device(d, pdev));
> +
> +    /* Each PCI bus supports 32 devices/slots at max. */
> +    if ( d->vpci_dev_next > 31 )
> +        return -ENOSPC;
> +
> +    vdev = xzalloc(struct vpci_dev);
> +    if ( !vdev )
> +        return -ENOMEM;
> +
> +    /* We emulate a single host bridge for the guest, so segment is always 0. */
> +    *(u16*) &vdev->seg = 0;
Empty line hear would improve readability due to the asterisks being so close to each other.
Apart from that:
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
> +    /*
> +     * The bus number is set to 0, so virtual devices are seen
> +     * as embedded endpoints behind the root complex.
> +     */
> +    *((u8*) &vdev->bus) = 0;
> +    *((u8*) &vdev->devfn) = PCI_DEVFN(d->vpci_dev_next++, 0);
> +
> +    vdev->pdev = pdev;
> +    vdev->domain = d;
> +
> +    pcidevs_lock();
> +    list_add_tail(&vdev->list, &d->vdev_list);
> +    pcidevs_unlock();
> +
> +    return 0;
> +}
> +
> +int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
> +{
> +    struct vpci_dev *vdev;
> +
> +    pcidevs_lock();
> +    vdev = pci_find_virtual_device(d, pdev);
> +    if ( vdev )
> +        list_del(&vdev->list);
> +    pcidevs_unlock();
> +    xfree(vdev);
> +    return 0;
> +}
> +
>  /* Caller should hold the pcidevs_lock */
>  static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
>                             uint8_t devfn)
> diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
> index 0cdc0c3f75c4..a40a138a14f7 100644
> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -90,24 +90,36 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>  /* Notify vPCI that device is assigned to guest. */
>  int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
>  {
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +    int rc;
> +#endif
> +
>      /* It only makes sense to assign for hwdom or guest domain. */
>      if ( is_system_domain(d) || !has_vpci(d) )
>          return 0;
>  
>  #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> -    return vpci_bar_add_handlers(d, dev);
> -#else
> -    return 0;
> +    rc = vpci_bar_add_handlers(d, dev);
> +    if ( rc )
> +        return rc;
>  #endif
> +
> +    return pci_add_virtual_device(d, dev);
>  }
>  
>  /* Notify vPCI that device is de-assigned from guest. */
>  int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
>  {
> +    int rc;
> +
>      /* It only makes sense to de-assign from hwdom or guest domain. */
>      if ( is_system_domain(d) || !has_vpci(d) )
>          return 0;
>  
> +    rc = pci_remove_virtual_device(d, dev);
> +    if ( rc )
> +        return rc;
> +
>  #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>      return vpci_bar_remove_handlers(d, dev);
>  #else
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index 2b2dfb6f1b49..35ae1d093921 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -136,6 +136,22 @@ struct pci_dev {
>      struct vpci *vpci;
>  };
>  
> +struct vpci_dev {
> +    struct list_head list;
> +    /* Physical PCI device this virtual device is connected to. */
> +    const struct pci_dev *pdev;
> +    /* Virtual SBDF of the device. */
> +    const union {
> +        struct {
> +            uint8_t devfn;
> +            uint8_t bus;
> +            uint16_t seg;
> +        };
> +        pci_sbdf_t sbdf;
> +    };
> +    struct domain *domain;
> +};
> +
>  #define for_each_pdev(domain, pdev) \
>      list_for_each_entry(pdev, &(domain)->pdev_list, domain_list)
>  
> @@ -165,7 +181,9 @@ int pci_add_segment(u16 seg);
>  const unsigned long *pci_get_ro_map(u16 seg);
>  int pci_add_device(u16 seg, u8 bus, u8 devfn,
>                     const struct pci_dev_info *, nodeid_t node);
> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev);
>  int pci_remove_device(u16 seg, u8 bus, u8 devfn);
> +int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev);
>  int pci_ro_device(int seg, int bus, int devfn);
>  int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn);
>  struct pci_dev *pci_get_pdev(int seg, int bus, int devfn);
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 28146ee404e6..d304c7ebe766 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -444,6 +444,12 @@ struct domain
>  
>  #ifdef CONFIG_HAS_PCI
>      struct list_head pdev_list;
> +    struct list_head vdev_list;
> +    /*
> +     * Current device number used by the virtual PCI bus topology
> +     * to assign a unique SBDF to a passed through virtual PCI device.
> +     */
> +    int vpci_dev_next;
>  #endif
>  
>  #ifdef CONFIG_HAS_PASSTHROUGH
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:51:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:51:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197734.350949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7tI-0001au-C1; Tue, 28 Sep 2021 07:51:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197734.350949; Tue, 28 Sep 2021 07:51:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7tI-0001an-94; Tue, 28 Sep 2021 07:51:04 +0000
Received: by outflank-mailman (input) for mailman id 197734;
 Tue, 28 Sep 2021 07:51:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV7tH-0001af-KZ
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:51:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d3153dc8-2030-11ec-bc72-12813bfff9fa;
 Tue, 28 Sep 2021 07:51:02 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-SOftHYKTMMuythZLOczN5Q-1; Tue, 28 Sep 2021 09:50:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2959.eurprd04.prod.outlook.com (2603:10a6:802:8::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 07:50:57 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 07:50:57 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0003.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 07:50:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3153dc8-2030-11ec-bc72-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632815461;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WnWXYj8LX5CKb3mnxa2+jCxX1IwAdl0UKdT7BrkECrA=;
	b=bwV9SlRBAI5AtDMewJtJeQJJwGVBqIoLiqr48jxTVCgipmFcm9LxLYcxc/xMcNpivsILt9
	oUTv0VkKB3qQFpw946cHwF8RiZUkfPoHb6zu0J/NyMfYgNHDH/AURz6Ag56G5X5zB0Fq2+
	diWh4cLTzn21lLeohUEaT4+GN0qeM3o=
X-MC-Unique: SOftHYKTMMuythZLOczN5Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kafbRu4WxxJzJzkxEkfhyx2hPqc3gzj+y/Lv5Js/41K6/Ruu0xJ3vpxeB+o19saSkzrcnMRohihUiFvMHjm0ukkH6R15Oi+J6g3K7h2/qXrvxbD8UmfZAtC1PvHUH1asgP3HxKuvX5fb8zxqWLJQ4AelskMCTjB0rdQzt65Tx264bTU7+6XZG1mD8iVSGuFiv0xGpq97M9YES93FZs0F/RKUvq5zKuSV3i9KqN1t6KOAU4oj94t0sdM2+VRRUkrSGUjepLW8fRKSJpBn30POOuouARvOEt+qG/kQOrgn535H4wsGn6AM+LIt9J6YFQd/Hi/ygeA9gTu5o6AFa//Clw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WnWXYj8LX5CKb3mnxa2+jCxX1IwAdl0UKdT7BrkECrA=;
 b=fIyeAGdVmwrsvl8FZADDpD1nfm4RI7J7NR35nKurmHcuY91xPgMhgMPvXFZxNNDplcFPubFDra1r1Y3uguONC4yDlU50yWnx8jlM2ILPgAPdahP7gMhNwRZ5ZqHjBSkzc9t+RZPoUG8fmayYgG9fNq2sk6oIjCDx0VkYz9nnMWtvJeUQ5ih0fk+CyNZ30o3CP4bjgxxBcgpWBGNGG5WPF4mfKb/KlaHKnj9BtJx5xxCBeWEtu8/gJlITTrhcFBQtRvrdjbAbLwpfzH7yp8RFjqCzWl+CrL6L81RQB6DhCd6qyYUj3FBCZnzQywVjJTcbJFwp5BStA03wbb0tz0W2ug==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v8 6/6] AMD/IOMMU: expose errors and warnings
 unconditionally
To: paul@xen.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <dc0cd7f7-a313-099d-3e89-e3862ed11f43@suse.com>
 <ba9fe216-32a8-ef6d-173b-4f0d31a20db2@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <707b7bdb-bf81-8c16-a7a9-f55256f53ec6@suse.com>
Date: Tue, 28 Sep 2021 09:50:55 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <ba9fe216-32a8-ef6d-173b-4f0d31a20db2@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0003.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::15)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 353aa0df-d8be-42c6-a0f8-08d98254b521
X-MS-TrafficTypeDiagnostic: VI1PR04MB2959:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB295971C769F5EA26141F8347B3A89@VI1PR04MB2959.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3631;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	idX5zBhcqXW2M0aKU9TdgSMLY9KxMjmBrdnYivkqN1RTN4rW/1Kx+6d5wElwdpQmy1gDT5tX1TV7uyGgpmwI0zSuoe31XABHkW42brRWb4pPhAHlcGp61Uqco3b4ZJjObPaYI68zXBrxW/+795TEbQU5Dfz1dhUB6RZeWG2sTNdEca/1V4s9Z+D9qahxODDD9MP2o6Rue5eZ4ClJqlvnF+VJRZSj65SihYZYJ40X0DkH4VKR1S2zY8f1ov/Ye77o1bZjuFRwmJeqKrNiuYox+g1YYNkM4lJnNrtiHt8p3PNvLq9KdvgmYP/kGeJ8Udr1qEQm5P7u3qPKtJcN8RnL6+LgkaEyIBmZdXIne5S7UHsh3Xn6mqe/6wuMtIZXvSKlmA4KQqs41zk+eXFdd7Hzp8RNaHkp+KER++rwBfxuv+yxpOfz/Uv/pGABxmfdjr6YBjywXecY+disW67lX73J8VpZznUyJeZLYugYI8ZSWEIzM8AJ61l0/ZfNw/JGMJdZSQVvPurcbeUf+QaG5P2E9j59Yz9EZIvQhWIkY4C/Kk27giCNIQYoXf3/9knYymcjV6uicU6YUu+y9zog5R+26qKhGxUVes0kpNBy+lNfXskh9ZFK+jlNQ2idfO/+eOjWWlkguSmIkk+Hv4uus6i2wagqOr3XTELTJMKv+2tJEuciMX8xDcXhepZxYdHc3HiLnHObuDsulKI4MFcYsaacxdU9oBH+b/8O/F1JEH/nRgfyfAL9cP6E7U/yc+18Sg7H
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(54906003)(26005)(16576012)(4326008)(6916009)(316002)(2906002)(5660300002)(38100700002)(83380400001)(36756003)(2616005)(956004)(8936002)(6486002)(31686004)(66556008)(31696002)(8676002)(53546011)(508600001)(66946007)(86362001)(66476007)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WHIzSlRnQVU5V3RnaFZTaUxFQ1B3VUl1VUo5TnNwcW9DbEQ2K2ZSdnZPeXVW?=
 =?utf-8?B?alBMREJ1U2ZLQkhLY2ROMmtjNjNqT25jQXc0RzJ3RU5HM0grRk43SDEzODlh?=
 =?utf-8?B?SG8zREU3NTRodENxcmJFREV4b3g4VTNuVHkyeFM1MUdIVU15a2VlRjlOckll?=
 =?utf-8?B?UC9IeDcvcmVBTjArNWhGd3hraVVBbExmbWQ2SXVuejdLM1pQbzNUYTVrOGZi?=
 =?utf-8?B?bjRoa283Mm9pTmNZM0d1VzdCQVRVR0pNdFluS0hhTXczczZsK1BHU3ZYUm9I?=
 =?utf-8?B?OC95NGdvNUg2VXBrbVVkUmE4RkY0YkpIVmxOWWw0SGt6dncvelpSU040MGdt?=
 =?utf-8?B?c2pEWmwrQlZnQnpsNlVGdW4ySXNaRXM5R3hqK0xEcTJjU3NkLzE5b1lMSXJw?=
 =?utf-8?B?ZVlsNXc5dEczM0RiMVlVdXlrL2JBaUY0RWFYNlZHQ1BlcUdvVURHNk9neDlV?=
 =?utf-8?B?REI3U095N3lUOHNCYktBWkFrdks2cEJpR2lPeFZpWktyWHozY2ZpWVJkcHNI?=
 =?utf-8?B?OW16UkdXQ2Evb1pCV3NWYkQ4MFJIei81Q1pPaVhiZ0lqOVlNUkVQWmpkN3VE?=
 =?utf-8?B?L0ozL3dUbVREN0xWVGp0UjlORXJRK1I5MkY2U0xaeVo1V3BjUDhyN2wwNG82?=
 =?utf-8?B?QUYxMFJGNFBTUUhnT0dFLzNiaWlueks3bVFBUnJqVkx6YkVUb0w3dUxELzhr?=
 =?utf-8?B?L3c5RXh3cGIxd2FrL1NMeUprU2FFdm5XRTUzL0tyZElycXMwL09VZ3cwUXE5?=
 =?utf-8?B?SS9mUXJDVzM5bEFReCtHOUlXblhZam0rTWhpSVl3eUM2REp2a3Y0STF0QXEz?=
 =?utf-8?B?ekxqQ2xnSkpBYy85UjFzakZ2aEpmUm1qeXNhWklFR3UwSFlUOFF1TGJRMEVL?=
 =?utf-8?B?VWFvbEtXSDZwWjYwRUh1ZnZ0aGdQK0V4V0xTNkJ0RWV1SzNYYjBZMHEyVmxI?=
 =?utf-8?B?NnpoK1pmeFByWDN0YXBSbGZNUmxtUFNyTTBtMmtMdk9CS0ZtYzdjV1A3Z0Yx?=
 =?utf-8?B?aTcwY2JnRmF2SVVzeWtmRktWT0VpZkFIMnY2MWYvaFN1RWo5QWlPV1dOb0hy?=
 =?utf-8?B?ZnU5aUtwL2l5YTBrWnJoaU9aYzFqN1VwZzVYTG1ZbmNKNFR1MUdlenk2VXRS?=
 =?utf-8?B?aVdaV3pSY1JUQkVqSFdqcEhJRXpHUlNQT043VlFZZE1JbTZ1R2djWDBNOG9n?=
 =?utf-8?B?a1ZLT3QwZEFhSkRCQ3AwS3NBVzR3aTJscTBrenRuUWlqbFhsOTN5OHlyTXVJ?=
 =?utf-8?B?MnJBK3NLbW8xYVh0OC84OTd4a3NjSHlnWWhPSW54TitVYm5salNsTEZIUTVX?=
 =?utf-8?B?YW5QZHFtdStYamZ3Ly9heGg5L2VWb1JuS3BMWjJSb29ycUZjUWJ4Z0FaTzFJ?=
 =?utf-8?B?b2FIRVNReWVVdzlSUFRCait4ZFRqUWFFSnNlY1dqeUVHa01HNDJ3UXNuQVdS?=
 =?utf-8?B?Rk5Xc3phY204NjhpWlJNd082OVA2TmpqaHhWYXNhN08xa3NQYzl0T2hpNjlG?=
 =?utf-8?B?bDd3UHZ0T0tMenJZRmdIcUZhOWc5YWVBY1ExS0JzT2o3dTUrRHNaeW1XMlo3?=
 =?utf-8?B?b3AzQ3l3WUk3dkdUYVBDYVpCV3BzK3Y5cEFLb0dDcjZqaitucUVtTW1vVmNW?=
 =?utf-8?B?aVBBV3pZcm44RVdUWHJyL2tWdlVlbFkxeTVhazNxMWFZU214ZHhjMVhVbk5U?=
 =?utf-8?B?blpzUHdNZW9pTjlNc3ZLdnpPZnNCTWRwMjVrNnZ3Vzc2eXhNRmRDWUJiRGxw?=
 =?utf-8?Q?+7MBH5n6qdxqMofkeJRUvDMicJU8yDpF9f0Nh/Z?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 353aa0df-d8be-42c6-a0f8-08d98254b521
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 07:50:57.8229
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nNktrPSy8p4ScvHfDmUljFOPMgTdvnbxa+6+gdyESZnZ5LxeDi09RdRquHurvROe+yr85kI9BOliDbwaBcsZ1w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2959

On 28.09.2021 09:42, Durrant, Paul wrote:
> On 22/09/2021 15:38, Jan Beulich wrote:
>> Making these dependent upon "iommu=debug" isn't really helpful in the
>> field. Where touching respective code anyway also make use of %pp and
>> %pd.
>>
>> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Paul Durrant <paul@xen.org>

Thanks.

>> @@ -515,14 +513,13 @@ static u16 __init parse_ivhd_device_rang
>>       dev_length = sizeof(*range);
>>       if ( header_length < (block_length + dev_length) )
>>       {
>> -        AMD_IOMMU_DEBUG("IVHD Error: Invalid Device_Entry Length!\n");
>> +        AMD_IOMMU_ERROR("IVHD: invalid Device_Entry length\n");
>>           return 0;
>>       }
>>   
>>       if ( range->end.header.type != ACPI_IVRS_TYPE_END )
>>       {
>> -        AMD_IOMMU_DEBUG("IVHD Error: "
>> -                        "Invalid Range: End_Type %#x\n",
>> +        AMD_IOMMU_ERROR("IVHD Error: invalid range: End_Type %#x\n",
> 
> NIT: I guess you want to drop the 'Error' here like you did elsewhere.

Yes indeed. Fixed. Thanks for spotting.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:57:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:57:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197742.350961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7zL-0002SI-1h; Tue, 28 Sep 2021 07:57:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197742.350961; Tue, 28 Sep 2021 07:57:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV7zK-0002SB-Tm; Tue, 28 Sep 2021 07:57:18 +0000
Received: by outflank-mailman (input) for mailman id 197742;
 Tue, 28 Sep 2021 07:57:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sHmr=OS=gmail.com=xadimgnik@srs-us1.protection.inumbo.net>)
 id 1mV7zK-0002S5-7E
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:57:18 +0000
Received: from mail-wr1-x431.google.com (unknown [2a00:1450:4864:20::431])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c2d0c496-660e-467c-934e-00dfa5f9d2a6;
 Tue, 28 Sep 2021 07:57:17 +0000 (UTC)
Received: by mail-wr1-x431.google.com with SMTP id x20so11715783wrg.10
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 00:57:17 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id k10sm1812018wmr.32.2021.09.28.00.57.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 00:57:16 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2d0c496-660e-467c-934e-00dfa5f9d2a6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=6mo9efTGE4VOD+ie7b+QeqiAiR1rSgGGWGb2BC/STh8=;
        b=j3jeZ7JWsx5zGyYnEqvLWyk1dLZXvzkePCKRICmFRXB9wciS94XPoo2jYrAXLvU1gP
         662FyDWe6jVG8Ksw/Vz9Z/T+PfnszcJmqtorGWdqN96nAQS1hG/Gv4Fo6lkztEIGpa3Z
         rwZ/ZJnhGvVRXnQKz7WZ6UJLR0j4ASzKSRV1yt0iEriyYef0foEoBXQFFOVdp8XWQ03q
         A/z2UzK+Yr27oJLK+t1DRrCJvreVSodizrK5ia2V928tqhKHv5Y/tiRonAvmJ3YCCnpu
         ObbBl1uIKrA+kXNWFoFuJ73iLitqU1UFpYfebeXLYAI+LeEc81safV7Y/or5viZ71HHy
         sI5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=6mo9efTGE4VOD+ie7b+QeqiAiR1rSgGGWGb2BC/STh8=;
        b=T5R5ItHHuG7GROQQ+Z0VTEUjQYYT1tkIH3PwAxemfbK6V27XaNBRat4vM3sXu5O/gU
         4NcDTK9V1sb8PoDpMRhUTZ6U2QddltfpQMF/DdPDMyNubLtXeCYQbvC+F4hBdThK0Sjz
         xkYZwN8nNvSyfdVBBW9MY5QgMpz9H3qrNAwf61Wux6vJelNm7BJQOD3jmXmKFeA1PetO
         Uhks8drSnQS1vWP0tpncjxe3r/iOmIaXoS+g66eebuSzJFerotTP0eaIvpd04RgxZaDH
         X3clcb3l7ag5qxV9v6KqzaJUdiX4r3VCZfO3D78Ei83hfcz1sQ33Decd0bzHWHgsukq5
         LCwQ==
X-Gm-Message-State: AOAM5305i6phTJG+8Hj1fBLm84vSLZf72pw0jChITRgefgl92HT8Gzto
	ep00O0nzdTgHicc4zqYqPD4=
X-Google-Smtp-Source: ABdhPJzd4IjKkZhDEvBhuSU1QmAss3Na9EW0rb6ObLgLj4Pgmg1CxdIvFsY7qCizQr7kw3SXvmzKoQ==
X-Received: by 2002:adf:ee4d:: with SMTP id w13mr4494917wro.223.1632815836405;
        Tue, 28 Sep 2021 00:57:16 -0700 (PDT)
Message-ID: <70d9fb6c-9578-15d6-423f-366b741b2f08@gmail.com>
Date: Tue, 28 Sep 2021 08:57:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.1
Reply-To: paul@xen.org
Subject: Re: [PATCH v8 4/6] AMD/IOMMU: respect AtsDisabled device flag
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>, paul@xen.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <d249a759-188a-d689-316a-4743922827e5@suse.com>
 <14d1c1e1-d2d0-d041-e206-608e876d43a0@suse.com>
 <d300aaf7-dbe9-1bf8-6abc-1693bb13ec56@gmail.com>
 <a46e77e8-7686-875e-dad1-e0713b9fd46c@suse.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <a46e77e8-7686-875e-dad1-e0713b9fd46c@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 28/09/2021 08:47, Jan Beulich wrote:
> On 28.09.2021 09:34, Durrant, Paul wrote:
>> On 22/09/2021 15:37, Jan Beulich wrote:
>>> IVHD entries may specify that ATS is to be blocked for a device or range
>>> of devices. Honor firmware telling us so.
>>>
>>> While adding respective checks I noticed that the 2nd conditional in
>>> amd_iommu_setup_domain_device() failed to check the IOMMU's capability.
>>> Add the missing part of the condition there, as no good can come from
>>> enabling ATS on a device when the IOMMU is not capable of dealing with
>>> ATS requests.
>>>
>>> For actually using ACPI_IVHD_ATS_DISABLED, make its expansion no longer
>>> exhibit UB.
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> ---
>>> TBD: As an alternative to adding the missing IOMMU capability check, we
>>>        may want to consider simply using dte->i in the 2nd conditional in
>>>        amd_iommu_setup_domain_device().
>>> Note that while ATS enabling/disabling gets invoked without any locks
>>> held, the two functions should not be possible to race with one another
>>> for any individual device (or else we'd be in trouble already, as ATS
>>> might then get re-enabled immediately after it was disabled, with the
>>> DTE out of sync with this setting).
>>> ---
>>> v7: New.
>>>
>>> --- a/xen/drivers/passthrough/amd/iommu.h
>>> +++ b/xen/drivers/passthrough/amd/iommu.h
>>> @@ -120,6 +120,7 @@ struct ivrs_mappings {
>>>        uint16_t dte_requestor_id;
>>>        bool valid:1;
>>>        bool dte_allow_exclusion:1;
>>> +    bool block_ats:1;
>>>    
>>>        /* ivhd device data settings */
>>>        uint8_t device_flags;
>>> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
>>> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
>>> @@ -55,8 +55,8 @@ union acpi_ivhd_device {
>>>    };
>>>    
>>>    static void __init add_ivrs_mapping_entry(
>>> -    uint16_t bdf, uint16_t alias_id, uint8_t flags, bool alloc_irt,
>>> -    struct amd_iommu *iommu)
>>> +    uint16_t bdf, uint16_t alias_id, uint8_t flags, unsigned int ext_flags,
>>> +    bool alloc_irt, struct amd_iommu *iommu)
>>>    {
>>>        struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(iommu->seg);
>>>    
>>> @@ -66,6 +66,7 @@ static void __init add_ivrs_mapping_entr
>>>        ivrs_mappings[bdf].dte_requestor_id = alias_id;
>>>    
>>>        /* override flags for range of devices */
>>> +    ivrs_mappings[bdf].block_ats = ext_flags & ACPI_IVHD_ATS_DISABLED;
>>>        ivrs_mappings[bdf].device_flags = flags;
>>
>> I'm assuming the above indentation problem (which appears to be repeated
>> in a few places below) is more to do with patch email generation rather
>> than the actual code modifications so...
> 
> Hmm, I first suspected a Thunderbird regression, but checking the list
> archives I don't see any corruption. I'm therefore now suspecting the
> problem may be at your end ...

It certainly could be; I was forced to upgrade my copy of Thunderbird 
recently. It's odd that the issue is not consistent though. As long as 
the actual code looks ok though... no problem.

   Paul




From xen-devel-bounces@lists.xenproject.org Tue Sep 28 07:59:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 07:59:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197748.350972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV81R-000350-De; Tue, 28 Sep 2021 07:59:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197748.350972; Tue, 28 Sep 2021 07:59:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV81R-00034t-AM; Tue, 28 Sep 2021 07:59:29 +0000
Received: by outflank-mailman (input) for mailman id 197748;
 Tue, 28 Sep 2021 07:59:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV81P-00034n-QC
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 07:59:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b8568c98-62e2-4050-ba52-fc1b1bf3eb38;
 Tue, 28 Sep 2021 07:59:26 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-15-GjAfqdNWNAW5w_jW2ogtog-1; Tue, 28 Sep 2021 09:59:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2605.eurprd04.prod.outlook.com (2603:10a6:800:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.20; Tue, 28 Sep
 2021 07:59:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 07:59:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0021.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:52::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Tue, 28 Sep 2021 07:59:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8568c98-62e2-4050-ba52-fc1b1bf3eb38
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632815965;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=++SQqnMhac3su4SgIB0DoC0eO8NuHYjHSgc9CfMdRLU=;
	b=HTyBSWIB0gqpsu+kQCYgMPQvMq/HJKOlWGL0yX6IZ9o45ok/QtCOpapMuFWckioXL/p/ud
	57mc1G6e6YfYvHDrZeRwlG2gp58EzSTAnHNvmPDDNlN818toODT8WppHVmloTNoe/q7KBW
	QyzuKlKRPklRXKT9BxvqtQIXJFkCnUo=
X-MC-Unique: GjAfqdNWNAW5w_jW2ogtog-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HIQv51a+MyU1V/GcAIrGCRmPO8hixr+Qxnzyq8qv3uSmtghSddfxcIJM7Sb8wxxW2A/K8zuFEf6V0GnMQy+JxE11I/2ORRt5sENMfZ4t2otkSKmFFkYXQtekK8AqVT2dP6+/EWXwXcgw2fzAfsDIuWvc3TYG1aJNHD7Ohj19QIdzpR6DwHqfeSzQK7qKBx9hHf1vP8hpqIiHZRjP/FLvosRUUaCBoTFKb29qO/Cj0Vhy4MBZ8FEHAdLAece0rrNOTyLgCbHyD7OrlXflXWde2OiEcR1jKrO3+ECM1hTZ/6zAemXXeZUVIlpmTH3SSX55MtCu8DHm78BzY6pMZY7XlA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=++SQqnMhac3su4SgIB0DoC0eO8NuHYjHSgc9CfMdRLU=;
 b=d0X+AFxa8wVuCeyqc+kPIEBESpydDJabgkDtJrOAq1nCgyk9cIZMHNXOd7WlYovqEHlzpgm8ej9xDV9VfthDAQPaTzxO0fJARjWYr4zLRl8m4J+r8Um6sNlNvWEIEFId1Zdc7sKxj22uCQh++zEpvPMJdaTGe5eCaeUjVbMMoiRLlOtZwQ5k3CSh/2K5AixhhjyarrMfjDrXJSEyQahdeu06OF7Lv9PrgY8dxSwftt7Yl/gGYaJwvTXljUk3ETp8O42BpudE7zfXhDUAVkpwL+7MsN9SkHw7XzEPHlztqcQJwpdPgVKj0zG/BTjMtqs1SHwsxCUSZcc/a9kEBFpilA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Michal Orzel <michal.orzel@arm.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
Date: Tue, 28 Sep 2021 09:59:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P189CA0021.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:52::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f4466b74-829a-47db-c1e9-08d98255e185
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2605:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB260542622967F47690AA2C66B3A89@VI1PR0401MB2605.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	t7afz0Xca7mKgVtGmccTYXLifdOpCb5j+/o5OtB9mAMFz/TWThMepCdVJcu7W/tlA/nVMIG/1D3QKfCPLUBTLyswFvQBiF0cM5hswoaeHtYDoid+1wKZBFQUd7zVBZAe6/Dj+plvJu9wie1DEyazBLjpfpKG4+PrPp9MSvlt7nM0c1Je11NG1aQV/TejBmJIq2Cg+GWZ+um5OZq0wYI+Eamm9+Swzr9/xMi4utMFG0n/SiMidhoj1iptrdl2C6v/Hm5viTs6rPlypBnE3WTXc483zMi6n766zgMbfRTyV28Yta3KCxzX70k/IZTPU7VTfr0cDsjfD9ZFrDVXzyR6D21YH6T5pwLWLwrDlQYZlqtH+wLMnMInI8X7qsUA25W+UR6fK1Cdjb0Yp4jbE0+OUGPGkzy3BlO8X4DhqEnG6GabCu+mbizorkNXLlRTsH9e35DNyYKlfHjgBRtnEl6Mwa3I8I3wQZwdBKdMV+tGz3A6HaIAJgAK+uJrVWrsHxRCJkYK+IzDfYbTY3q35OMl/wRii50K6Aq6V+OMFPsV+CTuFkFNU/qInagw4CMq8CpusvwHmVOf6WmjlvuAFhaLJOrh5aXilVdh9lavCvUkcwxGL/GxQpFBOnG7Ha85L8MvvZ71fGFs1weamEeD36PRyVIAUInwSd7MBfcLSyHw0snKRaSL4h19BZIM8cEcmiSBz/rkrwlmA2s9eD7pERCrv+g4P67qs0ioIvPqhEsSizGjzMzUAqo/IrqhCBj6bgZB
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(36756003)(7416002)(31686004)(16576012)(316002)(110136005)(86362001)(83380400001)(31696002)(4326008)(508600001)(2906002)(38100700002)(6486002)(66946007)(2616005)(8676002)(53546011)(186003)(5660300002)(66476007)(956004)(66556008)(26005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MXBhUlM2QUdtQjBueGJTSE5WTWF1Rk5ad0xPb3ZqdmlHczByajBNcWt1bklW?=
 =?utf-8?B?T1B3cHBOZkx1UllUU21ta2MrREIxeG1DY1JCUmpsTHhCbHB6M2tseVRjMURU?=
 =?utf-8?B?Wm5MV1U3WXNBeUE4cFVvZTRaajBuaUpRbWVGNERWUFZTcSttSDlFRWlXRHlX?=
 =?utf-8?B?THlIdFdrQXFjWVllUjNNbXNLS09acXBsRkpXZW16MUx0RW5jbmtFNGhoS1Rh?=
 =?utf-8?B?aGp4SHVQYzBjTlowNExKdUxQWXMvYXdLVWxhNU9WQ1dwM2JGWnVVMzluWkNQ?=
 =?utf-8?B?MTgyRWNuTkVtSkxqcUZTZmlRWWlYOFQ4WTBEVXd3ZHh6N2ZhaVlBQXhrWG01?=
 =?utf-8?B?Qm81Z0J2SHdWUCtKTTNBckUvcitnZkpWaFNQT211a01pZWlhMDcwL2ZYYlN4?=
 =?utf-8?B?a2JQay92YzJwZ2UvYTVJT3FOZU5Tdlg4elNZYUp4YWgyOFQ5MjEvZDY3bEpy?=
 =?utf-8?B?S0FXRnoxMEJKOS9ybm11VCtER3VQdnBHR1p2U0Y4WVV0UHZHZ1o5enVWeHhv?=
 =?utf-8?B?MHVtbVBXU0ZjcDltRDJyd2NQaUxxdXVJcFJ0TjJkb2k2YjhTME1ib2ZGNkNM?=
 =?utf-8?B?TmUxM3JaTUxobmx6QWRsWHhXSW94SGtjUFErRmVITm5QdkNwNHRkeDNnS2xY?=
 =?utf-8?B?N0ZORWlFbmxTMVdzd1hka1FGcWU5M2dYOGk0aEFwb1h1cmhaUnNSOXBNcmxZ?=
 =?utf-8?B?dnhPMk9lRE1GOGt1SHZVb3J1dnJPOFNxYjJ6S1Y1blc5a0VYcnU1Z3M5M0Fu?=
 =?utf-8?B?c1AzT0R1SFgzYnM4YWVaNHlBeStnR3BSUW4rVjM2N2NsOExkUldlUzRhR2VS?=
 =?utf-8?B?cGMvTlJYeGRCRW0rUjV0Z1dLWXlVMS9MTExBNlBCRW5EakoyeG5VVkFPTk1G?=
 =?utf-8?B?a1RUR0RIWWtzajNKNlRoZDFkMVpMU2xaRDdzclNXMXJNOTluM2N2a0pzcXhK?=
 =?utf-8?B?R2pWUFNFTEJkL1Qyd240dU0zcUhoT0d2WVEwRWhyMTVrYWljK0pKVXd5Q3hl?=
 =?utf-8?B?ZklwYk1UL3VwZ2JNM29MWVc1NjRhaEk0VzN5UTFTOXg5aUl0MXNoaEt2cE1v?=
 =?utf-8?B?czFDeEhYa2U1WXRaR05TVDUwOWxweXNoUjZUTUp4TXgwdDgyVTBXRWtpdjJ2?=
 =?utf-8?B?bnE5cHpBcGo2bzFJTFBlazN4eDc0WGVMUk5CWHloUEJKT1NTQzVpc2hYUTRr?=
 =?utf-8?B?TVliYmhBZGxHSEhNUk5ibU5sVHhTZ3BxeXl0VTYvUm1PUHBWTmhObHdqcUd2?=
 =?utf-8?B?K0h1Mm5rU1pMejVwRmFDNXZQOTZuclN4REMwY01GaldQaHRBOG9ZZnVyMVgr?=
 =?utf-8?B?SHJaY1N5QXNZb2lSK25jeW5IQzFyLzdkQ0tyZ1NFWlFUeUk2TlZHM3JTRjZO?=
 =?utf-8?B?WWRGR3JsMWlsUzM4a2pOQTRyUHlKVFFWcCtTMjVLZy93b3hlNnRzV1NDT2tJ?=
 =?utf-8?B?eEd6KzhIUWdGSXR3cDMyRDRTdHZGYjZJV2J1TTM2YTgrUGY4MERXQzd4em1I?=
 =?utf-8?B?TWtHcjVjQWdoTXVVQUxtMk00MWU2azBmMFE5Q0hEaUcxYkdES1ZPeVJ6L2Jj?=
 =?utf-8?B?OUsvd3l4cTc1cldadFNFWEFWaEdCbkQvNGRrcWdOYmVMWGh3UnYrcVFKWWow?=
 =?utf-8?B?S0N1djJTZFExUDN5cTN6V1BtU2hsck9xVVJSZDZBdk1IVXcrOHlXanBNdTho?=
 =?utf-8?B?Sk1rVHdtYzgxS1oySnhaaDMzSWU2c2hvb0JCMkdXUWo3UFBjRzRIc1hWYnN4?=
 =?utf-8?Q?JaJ46MTJiyKJz4v4P3ism6AZz6EXUj377MmeBta?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f4466b74-829a-47db-c1e9-08d98255e185
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 07:59:21.8075
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bGKmgIW6a5gDMObsxS810w3OcohyL0McAnllWRvE2PEa+br96GVpnrPRQV9qjQpP/1M9ADFoXbjR8E/GzqsLFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2605

On 28.09.2021 09:48, Michal Orzel wrote:
> On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
>> --- a/xen/drivers/passthrough/pci.c
>> +++ b/xen/drivers/passthrough/pci.c
>> @@ -833,6 +833,63 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>>      return ret;
>>  }
>>  
>> +static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
>> +                                                const struct pci_dev *pdev)
>> +{
>> +    struct vpci_dev *vdev;
>> +
>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>> +        if ( vdev->pdev == pdev )
>> +            return vdev;
>> +    return NULL;
>> +}
>> +
>> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
>> +{
>> +    struct vpci_dev *vdev;
>> +
>> +    ASSERT(!pci_find_virtual_device(d, pdev));
>> +
>> +    /* Each PCI bus supports 32 devices/slots at max. */
>> +    if ( d->vpci_dev_next > 31 )
>> +        return -ENOSPC;
>> +
>> +    vdev = xzalloc(struct vpci_dev);
>> +    if ( !vdev )
>> +        return -ENOMEM;
>> +
>> +    /* We emulate a single host bridge for the guest, so segment is always 0. */
>> +    *(u16*) &vdev->seg = 0;
> Empty line hear would improve readability due to the asterisks being so close to each other.
> Apart from that:
> Reviewed-by: Michal Orzel <michal.orzel@arm.com>
>> +    /*
>> +     * The bus number is set to 0, so virtual devices are seen
>> +     * as embedded endpoints behind the root complex.
>> +     */
>> +    *((u8*) &vdev->bus) = 0;
>> +    *((u8*) &vdev->devfn) = PCI_DEVFN(d->vpci_dev_next++, 0);

All of these casts are (a) malformed and (b) unnecessary in the first
place, afaics at least.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:02:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:02:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197760.350983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV84l-00052l-9c; Tue, 28 Sep 2021 08:02:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197760.350983; Tue, 28 Sep 2021 08:02:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV84l-00052e-5f; Tue, 28 Sep 2021 08:02:55 +0000
Received: by outflank-mailman (input) for mailman id 197760;
 Tue, 28 Sep 2021 08:02:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV84k-00052F-5u
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:02:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 00f13cc1-b607-4581-851d-f343c70a64a3;
 Tue, 28 Sep 2021 08:02:52 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-ZOKMQGZ6NnCNYE3hEZA3fQ-1; Tue, 28 Sep 2021 10:02:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5904.eurprd04.prod.outlook.com (2603:10a6:803:e6::29)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 08:02:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 08:02:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P189CA0087.EURP189.PROD.OUTLOOK.COM (2603:10a6:102:b4::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 08:02:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00f13cc1-b607-4581-851d-f343c70a64a3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632816171;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sJ+GAgX6lMi3+SGbOZQeDp+v2NYvbnqOGkWbw3JziM0=;
	b=DfL9x7CqGvaay7CqNU93ETOQNZiu31OLi2OxsqzSIxFs36Q/cBAh1eEcBGV7L6ot4itzu4
	AymA679TWOIyt8r7M3BZycFgP9TQrkKXBX64GwYNwMzz+34JkuhZ3HExKC7sblRGCR09Yd
	PcElzu1I1qTxOYP4w7KPVOoqzExjAB4=
X-MC-Unique: ZOKMQGZ6NnCNYE3hEZA3fQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Co9mx8ssVjqdmgKHsRjroauvhQ7MsVInIRbVqQURlWpfy3Wjkma2EKPcPzqM3Y14HnZzaj/5yL+h3UlrP+BVVIIKd/ulXyoI3coQNsHkGdD3mJoC9/MSkM4nS9Hu90wPdtQyzI8pAiuY/fwI78qLMrXyO8PYasbZE7ANl4MkaTpwQ63zxk07wonYdam8/m2gGNjyfVhknSkKWv0KookoUvrw3ARfoh5c6zn4wL2EQ/W4odMugKgzK+cE+BnyGYiMYsMaIu1C3Iz9aO2vYCONgjh7WLSBYHdYY3Xru16QXGIBc4BFKk09TPZBHp4Xxvh2QgilI5gslwd7ZAUGqa+qdA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5yIna5NmEvZ/z/3EqYACdFIw5rgfUtwrjEFkAr2ZH1k=;
 b=XgW1KD/4t+2VcwieySn3g67ZbWsgYo34zXA9EbFKjZ+nLPb/mVZTeQR8NKGMeR1oKITCLWG3DtXCUwSjThyYrx2W8mSpM/NfR0w5OVN9OxZeprmEBh/2RUYjMisrQnVXsNb4iRT8Jg+JvowYpLdoQPyXZxU/4IBtcs6/W2ParHt6FomXZGo7MqS6qf4cpn5KUZzSQdnbORuwetj1DZ/Us/5eo2jXjaprUnkknusTCJNgBl+94tVnP7lGFscDvUidusTu9YViYp6IqgjkYFHGZjvMGBqyGGhbbFv3FcI7BiRYrglrUkFyejElHQGg0Hc0/+sZ5mUV7HXSNPAapV3eyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for non-EFI
 architecture
To: Stefano Stabellini <sstabellini@kernel.org>, Wei Chen <Wei.Chen@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "julien@xen.org" <julien@xen.org>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <20210923120236.3692135-1-wei.chen@arm.com>
 <20210923120236.3692135-21-wei.chen@arm.com>
 <alpine.DEB.2.21.2109231811420.17979@sstabellini-ThinkPad-T480s>
 <PAXPR08MB68640027443F267495804A529EA49@PAXPR08MB6864.eurprd08.prod.outlook.com>
 <b4433faf-bb70-d083-126c-0224da3d9a82@suse.com>
 <DB9PR08MB685742B691E39FD3161BFE289EA49@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <a295cc34-9cc0-468b-c85a-2e5d1238d9a3@suse.com>
 <DB9PR08MB6857EE6294A1062EE0FAF0289EA69@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <DB9PR08MB6857AB9DBB66A4E02140987B9EA79@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109271420560.5022@sstabellini-ThinkPad-T480s>
 <DB9PR08MB6857E57847E38C400F6571EA9EA89@DB9PR08MB6857.eurprd08.prod.outlook.com>
 <alpine.DEB.2.21.2109272159590.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f872f421-dddf-7fde-34ce-210ddc55a87f@suse.com>
Date: Tue, 28 Sep 2021 10:02:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109272159590.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P189CA0087.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:102:b4::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3e745ab6-24ed-4575-7fff-08d982565c47
X-MS-TrafficTypeDiagnostic: VI1PR04MB5904:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB590402EBF00BE5C706AFCBA5B3A89@VI1PR04MB5904.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oKbwNKghoHd7K7fiQJre5uaZ1G8FlvuAumdrNgRg1Wt0uuRnzkglugi9jibzCD6OLcAJz1zOyKEHb6nF0Zs330N8eeYjA7nRnq72nvfMIak9lumzSXMqXs3F/5RKCgRcsJT3jftL0CJuk7KeMbn+ReX4m6uXf9VMils/Cb9NdhnVKnU40FpqG2XzzTINW4LAzKeVwWzl2AnF58Ei8SXkFyvx9pW5fPmIa1DQnkWQqjZZHnhrregh2IBt5iEMdQwS9kRmXkOe/Kb+SgZx1qsPr4SYKt0l0H+SEyzTXYeybfcvK7n90YB2HCV9WQOwS+P9gu/UV5UmIBZCJS5ezrM97Y4HqN1ATbd8HgSmILtmB/2Uzr6OqUdoV9FdMc6/5y6NlBin9wTYKJeS+yO1aOuVGP93jQ91f6t7LloXk+D/F327d7m0Cq6g9D5J0kaB1GFywimTdLWa1Ek67nJHbdLecM5yQL1KDlO553AuIh1hX6KbImFWP7qg2xKr5cH58kQiwqruEPSSBcYQcpI20lKa2lkO3IiX3fv1HbGPPGZpxPDz86M31WzBdQlpKGae4xTgsbvA2Oa5HdgbEku8eRYw3PHIbMzPPTxpT5WhwJWUdPLkjv7LAHwkkBzNlwHviraTEP2dk8U4A393IxnY7utvaGx3dwL4C5PS+++p8UqkaCeRibMRZWVGaYFmLydZib5nLUMi2181Zph9neLRV0ZDPY4mh0VdVxLOEXTv5P/f9oBRf1s0Bsb9wfY1ZI1vtbv0fZYWMhOsuv46VgpMpXiFwisc8p0rFQv2/WEe+0xSfAC+n4laqXtzrLn0Nek3GIQUM9bAn6HpOzhyK1SAC8dsCA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(54906003)(956004)(53546011)(5660300002)(38100700002)(4326008)(8676002)(2906002)(86362001)(66476007)(6486002)(186003)(83380400001)(66946007)(31686004)(2616005)(26005)(508600001)(66556008)(316002)(966005)(31696002)(110136005)(8936002)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?0zVg27jEYq4vhGIHqyB7SwgN5FqbwKNZwrHbRal9TCGZJRgfvMWHs3e2/nE9?=
 =?us-ascii?Q?WDEo+wRB5lmWsP6JvwBqqAoyfOuRqPPxL0D8CsHvMyudMpCZ+/GmRqq6kzDw?=
 =?us-ascii?Q?baz5Db456vV4oggHvySwIBEvel2/ske5OZeTjjEKBv3dDqvYq7miQ8YvgY1B?=
 =?us-ascii?Q?SAVbsCayLZndonzkONH0+oPRBp6pRhtdYk2Hi3+jd2l8QxswzmaTkSbWfirt?=
 =?us-ascii?Q?fCq4XVG6P20kKN25FFCAw+CAqTPHHC/5i8ZBny4VxB7F//jLJ8Vhh909SF26?=
 =?us-ascii?Q?G0Jk/rmtvdMGAVSsJ3qSNkviHT5ppsPwFYGXSs+89MRlzkDabl37lUoU31nR?=
 =?us-ascii?Q?uGfBmkIQC4YZQ+QP8lcVMlBgxMO2YgT8FdM3dC25Wldp2jXchsXdmZXtqmkp?=
 =?us-ascii?Q?An5zHxM9gWewNGOjKbOODcuassi3SYPcqkwPcdL3A0jDj/k4bCy0XFyx3flV?=
 =?us-ascii?Q?Ewu9DgLV/mzX0WZ9QTp6ffQ1OGQd8GZoyzRgnH2tMHv8JrNfHjRBgFK077Ek?=
 =?us-ascii?Q?0x+7WpbYtxVw3cLmYHkyHsvieAJ+VrcMAWNr/99uFq0mMqrganyn4dbPY7vX?=
 =?us-ascii?Q?x0yqGgiUlKa8PE11oJ6RG9g6pVabEaIpootUpn4Tn4j8yf1Q6nzrXrHux5V4?=
 =?us-ascii?Q?Rl4VkJW0HkZjnKeJGQgZTM0YF0TJeSvx4C7+yBLcQdhyaRRujZhMlcgi5+zX?=
 =?us-ascii?Q?HM0wg5oyZsBBtbZuLb5qUU/1JWBY00PkCal0Pt219xtjYOv6ZIZTC9B5Tjeo?=
 =?us-ascii?Q?aLCvVfP/5VhzAiGUdPDqHSNGWF69PofbKZtGYRcvFKgq0eyfWj9LceQJe1BM?=
 =?us-ascii?Q?Hm12wzcxKBnhlesEUxCqS9vGKz7mvDJ1yNmIE4ak/eWfPvqCq7/z8nuN8kqS?=
 =?us-ascii?Q?p9jNoIZ/+sNjOlFlcF6xywCnCSio5ZHTQPm2j8zly5/XGnsnc7pM+XkrZslb?=
 =?us-ascii?Q?rUvvFT7XIgOOkPwGetshZZRqG/r4GkIchPWtbvvo5jem2phXFe/1gyytXroM?=
 =?us-ascii?Q?cWpmUKvoxaOrIeT2Kls7CO/n+s3SZiPVv4+jIviDh10CR3XcDXunI5eCYIzW?=
 =?us-ascii?Q?1FuwPaDj2VMtbqhj1QIwdxY8bgqmz1jEEnFEqQ+tOaxmS60QLv8xnaHBpG13?=
 =?us-ascii?Q?adV/wiBgEYqti+3x+u2e1zIlGYRVG4Wb+1lA+wdjn3mNmi1G5rixQbR6QSik?=
 =?us-ascii?Q?b9XOghIprMTVxQZyekV4VdyJL13Z9+HmxHgOMcvKcdPObSMlrjVvDOGW1vOa?=
 =?us-ascii?Q?AfnnjZcjM+BoYpHj9zy941LIt5RJzNc509yjm3G9GVB6oUR64W6Q3cMmkYoJ?=
 =?us-ascii?Q?6Lr0+15Xpti7g1PP9PPuh1ky?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3e745ab6-24ed-4575-7fff-08d982565c47
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:02:47.7593
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ehfa8JSxIGY4nAHfzeZPjek/dMrSehCr/tLLo/MSDsu05OtBsrNIfQ5qxZMhvaDHcdGtAiA5ghEwcFiwqDBzwg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5904

On 28.09.2021 07:01, Stefano Stabellini wrote:
> On Tue, 28 Sep 2021, Wei Chen wrote:
>>> -----Original Message-----
>>> From: Stefano Stabellini <sstabellini@kernel.org>
>>> Sent: 2021=E5=B9=B49=E6=9C=8828=E6=97=A5 9:00
>>> To: Wei Chen <Wei.Chen@arm.com>
>>> Cc: Jan Beulich <jbeulich@suse.com>; xen-devel@lists.xenproject.org;
>>> julien@xen.org; Bertrand Marquis <Bertrand.Marquis@arm.com>; Stefano
>>> Stabellini <sstabellini@kernel.org>
>>> Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for no=
n-
>>> EFI architecture
>>>
>>> On Mon, 27 Sep 2021, Wei Chen wrote:
>>>>> -----Original Message-----
>>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
>>> Wei
>>>>> Chen
>>>>> Sent: 2021=E5=B9=B49=E6=9C=8826=E6=97=A5 18:25
>>>>> To: Jan Beulich <jbeulich@suse.com>
>>>>> Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
>>>>> <Bertrand.Marquis@arm.com>; Stefano Stabellini <sstabellini@kernel.or=
g>
>>>>> Subject: RE: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for
>>> non-
>>>>> EFI architecture
>>>>>
>>>>> Hi Jan,
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf
>>> Of
>>>>> Jan
>>>>>> Beulich
>>>>>> Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 18:49
>>>>>> To: Wei Chen <Wei.Chen@arm.com>
>>>>>> Cc: xen-devel@lists.xenproject.org; julien@xen.org; Bertrand Marquis
>>>>>> <Bertrand.Marquis@arm.com>; Stefano Stabellini
>>> <sstabellini@kernel.org>
>>>>>> Subject: Re: [PATCH 20/37] xen: introduce CONFIG_EFI to stub API for
>>>>> non-
>>>>>> EFI architecture
>>>>>>
>>>>>> On 24.09.2021 12:31, Wei Chen wrote:
>>>>>>>> From: Jan Beulich <jbeulich@suse.com>
>>>>>>>> Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 15:59
>>>>>>>>
>>>>>>>> On 24.09.2021 06:34, Wei Chen wrote:
>>>>>>>>>> From: Stefano Stabellini <sstabellini@kernel.org>
>>>>>>>>>> Sent: 2021=E5=B9=B49=E6=9C=8824=E6=97=A5 9:15
>>>>>>>>>>
>>>>>>>>>> On Thu, 23 Sep 2021, Wei Chen wrote:
>>>>>>>>>>> --- a/xen/common/Kconfig
>>>>>>>>>>> +++ b/xen/common/Kconfig
>>>>>>>>>>> @@ -11,6 +11,16 @@ config COMPAT
>>>>>>>>>>>  config CORE_PARKING
>>>>>>>>>>>  	bool
>>>>>>>>>>>
>>>>>>>>>>> +config EFI
>>>>>>>>>>> +	bool
>>>>>>>>>>
>>>>>>>>>> Without the title the option is not user-selectable (or de-
>>>>>> selectable).
>>>>>>>>>> So the help message below can never be seen.
>>>>>>>>>>
>>>>>>>>>> Either add a title, e.g.:
>>>>>>>>>>
>>>>>>>>>> bool "EFI support"
>>>>>>>>>>
>>>>>>>>>> Or fully make the option a silent option by removing the help
>>> text.
>>>>>>>>>
>>>>>>>>> OK, in current Xen code, EFI is unconditionally compiled. Before
>>>>>>>>> we change related code, I prefer to remove the help text.
>>>>>>>>
>>>>>>>> But that's not true: At least on x86 EFI gets compiled depending
>>> on
>>>>>>>> tool chain capabilities. Ultimately we may indeed want a user
>>>>>>>> selectable option here, but until then I'm afraid having this
>>> option
>>>>>>>> at all may be misleading on x86.
>>>>>>>>
>>>>>>>
>>>>>>> I check the build scripts, yes, you're right. For x86, EFI is not
>>> a
>>>>>>> selectable option in Kconfig. I agree with you, we can't use
>>> Kconfig
>>>>>>> system to decide to enable EFI build for x86 or not.
>>>>>>>
>>>>>>> So how about we just use this EFI option for Arm only? Because on
>>> Arm,
>>>>>>> we do not have such toolchain dependency.
>>>>>>
>>>>>> To be honest - don't know. That's because I don't know what you want
>>>>>> to use the option for subsequently.
>>>>>>
>>>>>
>>>>> In last version, I had introduced an arch-helper to stub EFI_BOOT
>>>>> in Arm's common code for Arm32. Because Arm32 doesn't support EFI.
>>>>> So Julien suggested me to introduce a CONFIG_EFI option for non-EFI
>>>>> supported architectures to stub in EFI layer.
>>>>>
>>>>> [1] https://lists.xenproject.org/archives/html/xen-devel/2021-
>>>>> 08/msg00808.html
>>>>>
>>>>
>>>> As Jan' reminded, x86 doesn't depend on Kconfig to build EFI code.
>>>> So, if we CONFIG_EFI to stub EFI API's for x86, we will encounter
>>>> that toolchains enable EFI, but Kconfig disable EFI. Or Kconfig
>>>> enable EFI but toolchain doesn't provide EFI build supports. And
>>>> then x86 could not work well.
>>>>
>>>> If we use CONFIG_EFI for Arm only, that means CONFIG_EFI for x86
>>>> is off, this will also cause problem.
>>>>
>>>> So, can we still use previous arch_helpers to stub for Arm32?
>>>> until x86 can use this selectable option?
>>>
>>> EFI doesn't have to be necessarily a user-visible option in Kconfig at
>>> this point. I think Julien was just asking to make the #ifdef based on
>>> a EFI-related config rather than just based CONFIG_ARM64.
>>>
>>> On x86 EFI is detected based on compiler support, setting XEN_BUILD_EFI
>>> in xen/arch/x86/Makefile. Let's say that we keep using the same name
>>> "XEN_BUILD_EFI" on ARM as well.
>>>
>>> On ARM32, XEN_BUILD_EFI should be always unset.
>>>
>>> On ARM64 XEN_BUILD_EFI should be always set.
>>>
>>> That's it, right? I'd argue that CONFIG_EFI or HAS_EFI are better names
>>> than XEN_BUILD_EFI, but that's OK anyway. So for instance you can make
>>> XEN_BUILD_EFI an invisible symbol in xen/arch/arm/Kconfig and select it
>>> only on ARM64.
>>
>> Thanks, this is a good approach. But if we place XEN_BUILD_EFI in Kconfi=
g
>> it will be transfer to CONFIG_XEN_BUILD_EFI. How about using another nam=
e
>> in Kconfig like ARM_EFI, but use CONFIG_ARM_EFI in config.h to define
>> XEN_BUILD_EFI?
>=20
> I am OK with that. Another option is to rename XEN_BUILD_EFI to
> CONFIG_XEN_BUILD_EFI on x86. Either way is fine by me. Jan, do you havea
> preference?

Yes, I do: No new CONFIG_* settings please that don't originate from
Kconfig. Hence I'm afraid this is a "no" to your suggestion.

Mid-term we should try to get rid of the remaining CONFIG_* which
get #define-d in e.g. asm/config.h.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:14:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:14:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197768.350994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Fr-0006jr-BJ; Tue, 28 Sep 2021 08:14:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197768.350994; Tue, 28 Sep 2021 08:14:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Fr-0006jk-86; Tue, 28 Sep 2021 08:14:23 +0000
Received: by outflank-mailman (input) for mailman id 197768;
 Tue, 28 Sep 2021 08:14:22 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV8Fq-0006je-Cs
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:14:22 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 14ef27c4-2034-11ec-bc72-12813bfff9fa;
 Tue, 28 Sep 2021 08:14:20 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S7X6ur008553; 
 Tue, 28 Sep 2021 08:14:19 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbxjkg7ex-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 08:14:18 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6881.eurprd03.prod.outlook.com (2603:10a6:20b:286::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 08:09:15 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 08:09:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14ef27c4-2034-11ec-bc72-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CaQrLDAYolf39rXzjfJ7Soijgr+Bsxd4vQn2AUUJr78qdXbSlxNyC75VsQw0Konc4s6e2JBeCmoxh9K0t3nwxlflQqPXRrvMtr1Rqgtt/GWBqek5ptL1pEaepTe2E9C1MelAtVvr2n066J0wJdAvu2iujXdlj6LavT+EEAun27FXrt6qqjUNo1GSAR4VCjKbVLQ6XMOgLY/nMD4GrV9OSlQ9VDkdCY4gXfArXbrm6N6KqkTx0ENY1ROSeO8eJxlIZJQHecQclfvZC4EEc4G/120ihKIiwxFDkilqZzdf21v8W4loGfeoLsax/VG6MxDtgrJMxNmmFa/G6Q0qsXrXuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hsuIGcsZzN2xj8wYvl3rUSlfMOMLL9AzmFQUnjKsn+A=;
 b=GVWuPpsn/LghhVjvkt8qU50XgiwgL2zBp4iFVyrSmhd7NTrej64W2hyS12AlXgcZQ/esPfLKdRePbVu2V9yQWCnEVz+HS+X7Rwop1oZXj/LXWcrueeRQDX/QOBVUD0uYKnaEDXHWW24hGO1N/eVawhuCrdxsVlQb10Rp+2Z5+9gUaMQdEPx6DbAmRH08lXsbBNY0nZk2CV2YDh21XjtssPGYEpux56kNg0vNUVpiBgWAdgfAGZkFxfYUP/5kCZd+UN9Lada2jjKGT6MKr/ei/fvAF23SPe/ckoLvQDlLwtM4fLoLj5/s1Fg+3LFBY84hcUtDWEDem1efSOF0sbRjeQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hsuIGcsZzN2xj8wYvl3rUSlfMOMLL9AzmFQUnjKsn+A=;
 b=wi9o8ud6rgb5nW+leKeyP/Vsz+PeEbXuey1JfQWk3NS25VSlfqMYbq2R4jAR14yu2UkgyG1bY5elPZMyCoi1oStmaFuP21IAo2vkMYyUJQ1UGPPM/Rv82Gs2sZf4Bs2wtEoVDkoklM33Ig9njdm3b8pYhzCIbWCI7ktTXBEUp6jsXWu0MBVFmhVYbsdRhwUIZyvfeRmfNs+V5MxQNktDsewtmKTT30G+6UxNJq5ktPfuJkhgIAuU1SSycnqDF2K+oeWAUXBjnjxNkSLDiz/weq/fgae2An5kl0YvL6joqGqBHZa8dwbnDlMwZgcFksUVkkgZbTjUwjk9NKTT4mFqbA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: 
 AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrICAAAldgIAABLUAgAAG9wCAAAEGAIAABhwAgAFsAgA=
Date: Tue, 28 Sep 2021 08:09:15 +0000
Message-ID: <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
 <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
In-Reply-To: <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 07657679-a156-4e38-0844-08d98257433f
x-ms-traffictypediagnostic: AM9PR03MB6881:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB688175102C88204058ED7559E7A89@AM9PR03MB6881.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Gg8KWbL5/XEjtwbNMEuj4L9CizOqP5mCrT0saxAKbeJW6860IKiEzRdIB7lzEj4bV5wFeiDG5AsjZ6vkKajmxbKBMbyklmD2n6dowKV4w/1duLHLR9mXoDAoZ7ItHgCp2/yd3wjO3P/98s6HWDe5yak5JsJpu6TjPVqbHaj17HARdlTfDgxIjEheKMCCby1AQU3uK3ETkPOupEWqVzbpPLHprPRGCDUS1qOdedryq3Ohx4Yu9V3t3MTzSfCz1wIDcEkEPNMAR8lUMjZgwWbATPBMRK+RUULByQmcy/kc8ptkPyMU2Q3C13s9gYNGxoJA/NR/AWC6EvqOKWYOsCi7LzU4K5ZNgemLw1Tu8LC0kHsWyEDwQ1g4ZQfuedKlCS+GxYzXIm+TAzUoh3FVYgzYA4FIQPs64maiWJ0vQad0kkBQEsiDeaqzAxbyqnk4zgiuepflcoxS9gBCoR8bDCBfE/uXt88ZEtP9/Pv6RuXpO4qwsFV98jXehcLyEeBK8WkcKBW8L2HzD4HFpmkowxyD78tNRYEloAJMmtIVRm1K7OfDEPlLJZ7ytcATDdSVfG6+TEC8V/GztZMNmFF3jzRDV3ydnCRxlNL4ipnRNDc8cLxacXRyAcCN64xPFnLRwcD9hAJ2DHo/a6u/bLKO127tGF0fMMu8zmTCFvio/Ke2TWKvofG+e5Tr7xo6ftsB5CzSqV/P9sdTMZK2VHyk1Twbhx6Nosbkwjuf5XXpbBkxN8w4GKtiILnBLN14ofLyXnlR
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(55236004)(38070700005)(8676002)(26005)(8936002)(64756008)(66446008)(5660300002)(66556008)(66476007)(91956017)(66946007)(6512007)(186003)(31696002)(71200400001)(86362001)(53546011)(6486002)(6506007)(508600001)(76116006)(122000001)(2616005)(38100700002)(54906003)(31686004)(316002)(36756003)(4326008)(2906002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?TklCNGxyKysyN0hGNnBmYmJBYXBwQVJwMnE0LzZDZ2FidTIzS3ZiZkpmbXdn?=
 =?utf-8?B?Sll3REpLY29sU2NYRWZFdzkrS3UvNkZsaFB2dFpoZ2hXdDVScFBBMFg0SHVs?=
 =?utf-8?B?Q3RlbmhTQ1hHL3VabnlPaXFrUUN2NjUwTE4vT1VqTVlqQ3FPbTdJUkYvZE5x?=
 =?utf-8?B?djl0R0pQaGJHWElLSU4weWhMMjNZOFVCK1Fmc0ozYTRTbk4yNS9PelRHU1Vp?=
 =?utf-8?B?Y3V3bVVLNGRCaUhtaUU1U21TajhtQ05abkY1Z0RDWnIvOHNPS3pnNVB3anQy?=
 =?utf-8?B?NkMrdE12ZjdwNloxUzRJSThGaDRTM1JrbkRxZ1QxZmtRWDduM2hkb0Y2UHAr?=
 =?utf-8?B?TmpRVXhpeUVjckFsdll3M2ZHbGRzMUNXMlk3TnVmVjdqZ2ZsbWVkUWxGWkN6?=
 =?utf-8?B?aEN2dlR0dG1WbkxyYXhOYUxLQ25mb2J3TjVlVWxscW9iUG5Sdk53NlNvdDcr?=
 =?utf-8?B?c0FFbFhuT2hma2kzVzliV3pNU3NvQmtjd0lISzIvQVhLMXRzMkFnMWNvTXBQ?=
 =?utf-8?B?UHRuKzBXcHA1ZWR2RjdraWVkZG9zdExEVDZvQmJHaVFwQ21uL2k1REgwRjhF?=
 =?utf-8?B?MENDYlJkQWV0Nkh3TXFGOGVPemtUczdINXpHYmlpR2xrUUNGdDVRaEJUWC9Y?=
 =?utf-8?B?WXNnQ1Z3Q2tmM3NEbW51TlltVEZ5cEkrcER5QWwwcFZkMXRXSE9MQWhKWmVJ?=
 =?utf-8?B?NTcyYlFQclpZL0lsYml6eTFmcmNMcXp3amxPUThnNUh5YmVIM3BJZVJ5V3JW?=
 =?utf-8?B?N0dMaDFCNGVPeGhpclpGa0d0alo2b0JUbFZrM1Nlem1aZkhTcmo3RFNyaWha?=
 =?utf-8?B?MW15UXpNMk9xdzhIOTBNanN1SUJZYzRKTmdOWWtaYUlMRzhoRmpqa3JtRFFD?=
 =?utf-8?B?eUJmaE5Zcko1QWpwaVVDWUtDM0ZBbFk0Q0x3SzUxTlY2Ujk5Q1ViWGlaTHNX?=
 =?utf-8?B?Z3FLMDlDL3BDRG82TzNQWExQYWw3RUxocEZHSWtYNFIzdVRicjdLc3UzMUNN?=
 =?utf-8?B?cTdKUTBreWNQMkVmZ1NUcUFNUHRnQjdhTWtKWkRuc20yVzM0VFlTWkJYanJI?=
 =?utf-8?B?NW4zWE00V01IazVsdGhsMHRZRElxSTQ3MlBJK1UvWnpHK2V1YnFJdnk2VnZB?=
 =?utf-8?B?VlRVUmpYUXhmU2JZbTZaQVozQm95KzJTZ21Bb01UNk81T3c3czNXaGZsQVNj?=
 =?utf-8?B?WHlQd3Q1MXVINzdBaVFiZWZDMFAyc1pCZDhQeHM0YlBRQk5uQy9CT1MxcTJ4?=
 =?utf-8?B?cGNGVXJ4LzFYbURjdmFjOTFJOGVTbGw2MmlrK0RBdU5VcEgxV2M2MFBQb0Qy?=
 =?utf-8?B?Si91YVJBM1J1TlBGV1dkaDVBcXo3NU1Gc2lya1N2SzZJaTQrR1BLZkZzNStN?=
 =?utf-8?B?T1hIV3JuZW8xWDljb25nMDJ5d3hCUjFiRytpRnk1M2Q0NEtsWWNPanB2eTVN?=
 =?utf-8?B?aHdBZVBJWllHcmo1aEswNURGZnNYcVVNQUhKWlRKYmtteStsenVrdVRQRmI1?=
 =?utf-8?B?VVdZUldIZWtnbWZpcUdjZ1ZxWFU4QSt5eEx6ZDA1aUVwc28wREEwL2VlYnNx?=
 =?utf-8?B?OHNXdjI1U0drLytSYkpXTUliR21TMjY1RUN2djlrL1dxb1ozemMyZUNmNHZs?=
 =?utf-8?B?WnZkZ1dxcFFra0xWcnNiSC9vL3pNS2lOMU8zeGRCeklQbklBNzdhNWFJbVVJ?=
 =?utf-8?B?T1lZUzdLNjcyVFB1Vm54SVpGZFRRMlY1TS9FQll4dXgwamR0YW1JNTU3dmVy?=
 =?utf-8?B?SjlGQWpmT1RKbXhaM2FSMllZVjYwMmdCR0dtd21zaUNpR1BaOHpJN2x6aE1I?=
 =?utf-8?B?bHBXK3g4MnZNZ29PdUE3Zz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <7824461F8853B347899D2F7A62E95F77@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 07657679-a156-4e38-0844-08d98257433f
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 08:09:15.0640
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2qsFzBaagI7tNF3uHlP78+q45qik55YQ21PJ2knUWoqvPrkEpfiktousicSJtoM+nMBy+BXtNaNsonVdtakvK/+TWeo+h149YOpIp2Fvws6XC10nSH7D+HpmKRMxjHIP
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6881
X-Proofpoint-GUID: 6Na1QEaPdzrL6tSDb2M_XirLdSZRJ2ME
X-Proofpoint-ORIG-GUID: 6Na1QEaPdzrL6tSDb2M_XirLdSZRJ2ME
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 spamscore=0
 suspectscore=0 phishscore=0 priorityscore=1501 adultscore=0 mlxscore=0
 malwarescore=0 lowpriorityscore=0 clxscore=1015 bulkscore=0
 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109280048

DQpPbiAyNy4wOS4yMSAxMzoyNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI3LjA5LjIwMjEg
MTI6MDQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTM6
MDAsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIwMjEgMTE6MzUsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMDIxIDEwOjQ1LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAyNy4wOS4yMSAxMDo0NSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+Pj4+
Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+Pj4+PiBAQCAtMzI4
LDYgKzMyOCw5IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2RldiAqYWxsb2NfcGRldihzdHJ1Y3QgcGNp
X3NlZyAqcHNlZywgdTggYnVzLCB1OCBkZXZmbikNCj4+Pj4+Pj4+ICAgICAgICAgICooKHU4Kikg
JnBkZXYtPmJ1cykgPSBidXM7DQo+Pj4+Pj4+PiAgICAgICAgICAqKCh1OCopICZwZGV2LT5kZXZm
bikgPSBkZXZmbjsNCj4+Pj4+Pj4+ICAgICAgICAgIHBkZXYtPmRvbWFpbiA9IE5VTEw7DQo+Pj4+
Pj4+PiArI2lmZGVmIENPTkZJR19BUk0NCj4+Pj4+Pj4+ICsgICAgcGNpX3RvX2RldihwZGV2KS0+
dHlwZSA9IERFVl9QQ0k7DQo+Pj4+Pj4+PiArI2VuZGlmDQo+Pj4+Pj4+IEkgaGF2ZSB0byBhZG1p
dCB0aGF0IEknbSBub3QgaGFwcHkgYWJvdXQgbmV3IENPTkZJR188YXJjaD4gY29uZGl0aW9uYWxz
DQo+Pj4+Pj4+IGhlcmUuIEknZCBwcmVmZXIgdG8gc2VlIHRoaXMgZG9uZSBieSBhIG5ldyBhcmNo
IGhlbHBlciwgdW5sZXNzIHRoZXJlIGFyZQ0KPj4+Pj4+PiBvYnN0YWNsZXMgSSdtIG92ZXJsb29r
aW5nLg0KPj4+Pj4+IERvIHlvdSBtZWFuIHNvbWV0aGluZyBsaWtlIGFyY2hfcGNpX2FsbG9jX3Bk
ZXYoZGV2KT8NCj4+Pj4+IEknZCByZWNvbW1lbmQgYWdhaW5zdCAiYWxsb2MiIGluIGl0cyBuYW1l
OyAibmV3IiBpbnN0ZWFkIG1heWJlPw0KPj4+PiBJIGFtIGZpbmUgd2l0aCBhcmNoX3BjaV9uZXdf
cGRldiwgYnV0IGFyY2ggcHJlZml4IHBvaW50cyB0byB0aGUgZmFjdCB0aGF0DQo+Pj4+IHRoaXMg
aXMganVzdCBhbiBhcmNoaXRlY3R1cmUgc3BlY2lmaWMgcGFydCBvZiB0aGUgcGRldiBhbGxvY2F0
aW9uIHJhdGhlciB0aGFuDQo+Pj4+IGFjdHVhbCBwZGV2IGFsbG9jYXRpb24gaXRzZWxmLCBzbyB3
aXRoIHRoaXMgcmVzcGVjdCBhcmNoX3BjaV9hbGxvY19wZGV2IHNlZW1zDQo+Pj4+IG1vcmUgbmF0
dXJhbCB0byBtZS4NCj4+PiBUaGUgYnVsayBvZiB0aGUgZnVuY3Rpb24gaXMgYWJvdXQgcG9wdWxh
dGluZyB0aGUganVzdCBhbGxvY2F0ZWQgc3RydWN0Lg0KPj4+IFRoZXJlJ3Mgbm8gYXJjaC1zcGVj
aWZpYyBwYXJ0IG9mIHRoZSBhbGxvY2F0aW9uIChzbyBmYXIsIGxlYXZpbmcgYXNpZGUNCj4+PiBN
U0ktWCksIHlvdSBvbmx5IHdhbnQgYW5kIGFyY2gtc3BlY2lmaWMgcGFydCBvZiB0aGUgaW5pdGlh
bGl6YXRpb24uIEkNCj4+PiB3b3VsZCBhZ3JlZSB3aXRoICJhbGxvYyIgaW4gdGhlIG5hbWUgaWYg
ZnVydGhlciBhbGxvY2F0aW9uIHdhcyB0bw0KPj4+IGhhcHBlbiB0aGVyZS4NCj4+IEhtLCB0aGVu
IGFyY2hfcGNpX2luaXRfcGRldiBzb3VuZHMgbW9yZSByZWFzb25hYmxlDQo+IEZpbmUgd2l0aCBt
ZS4NCg0KRG8gd2Ugd2FudCB0aGlzIHRvIGJlIHZvaWQgb3IgcmV0dXJuaW5nIGFuIGVycm9yIGNv
ZGU/IElmIGVycm9yIGNvZGUgaXMgbmVlZGVkLA0KDQp0aGVuIHdlIHdvdWxkIGFsc28gbmVlZCBh
IHJvbGwtYmFjayBmdW5jdGlvbiwgZS5nLiBhcmNoX3BjaV9mcmVlX3BkZXYgb3INCg0KYXJjaF9w
Y2lfcmVsZWFzZV9wZGV2IG9yIGFyY2hfcGNpX2ZpbmlfcGRldiBvciBzb21ldGhpbmcsIHNvIGl0
IGNhbiBiZSB1c2VkIGluDQoNCmNhc2Ugb2YgZXJyb3Igb3IgaW4gZnJlZV9wZGV2IGZ1bmN0aW9u
Lg0KDQpJZiBzbywgdGhlbiB3aGF0J3MgeW91ciBwcmVmZXJlbmNlIG9uIHRoZSBuYW1lIG9mIHRo
YXQgZnVuY3Rpb24/DQoNCj4NCj4gSmFuDQo+DQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:17:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:17:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197776.351005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Iz-0007WE-Qf; Tue, 28 Sep 2021 08:17:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197776.351005; Tue, 28 Sep 2021 08:17:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Iz-0007W7-NO; Tue, 28 Sep 2021 08:17:37 +0000
Received: by outflank-mailman (input) for mailman id 197776;
 Tue, 28 Sep 2021 08:17:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mV8Iy-0007W1-NU
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:17:36 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id e8a0ead6-db31-427a-b54d-d5088b8c4365;
 Tue, 28 Sep 2021 08:17:35 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EC6796D;
 Tue, 28 Sep 2021 01:17:34 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 56FEE3F7B4;
 Tue, 28 Sep 2021 01:17:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8a0ead6-db31-427a-b54d-d5088b8c4365
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Jan Beulich <jbeulich@suse.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
Date: Tue, 28 Sep 2021 10:17:24 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 28.09.2021 09:59, Jan Beulich wrote:
> On 28.09.2021 09:48, Michal Orzel wrote:
>> On 23.09.2021 14:55, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -833,6 +833,63 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>>>      return ret;
>>>  }
>>>  
>>> +static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
>>> +                                                const struct pci_dev *pdev)
>>> +{
>>> +    struct vpci_dev *vdev;
>>> +
>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>> +        if ( vdev->pdev == pdev )
>>> +            return vdev;
>>> +    return NULL;
>>> +}
>>> +
>>> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>> +{
>>> +    struct vpci_dev *vdev;
>>> +
>>> +    ASSERT(!pci_find_virtual_device(d, pdev));
>>> +
>>> +    /* Each PCI bus supports 32 devices/slots at max. */
>>> +    if ( d->vpci_dev_next > 31 )
>>> +        return -ENOSPC;
>>> +
>>> +    vdev = xzalloc(struct vpci_dev);
>>> +    if ( !vdev )
>>> +        return -ENOMEM;
>>> +
>>> +    /* We emulate a single host bridge for the guest, so segment is always 0. */
>>> +    *(u16*) &vdev->seg = 0;
>> Empty line hear would improve readability due to the asterisks being so close to each other.
>> Apart from that:
>> Reviewed-by: Michal Orzel <michal.orzel@arm.com>
>>> +    /*
>>> +     * The bus number is set to 0, so virtual devices are seen
>>> +     * as embedded endpoints behind the root complex.
>>> +     */
>>> +    *((u8*) &vdev->bus) = 0;
>>> +    *((u8*) &vdev->devfn) = PCI_DEVFN(d->vpci_dev_next++, 0);
> 
> All of these casts are (a) malformed and (b) unnecessary in the first
> place, afaics at least.
> 
Agree.
*((u8*) &vdev->bus) = 0;
is the same as:
vdev->bus = 0;
> Jan
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:24:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:24:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197784.351015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Pr-0000dl-PD; Tue, 28 Sep 2021 08:24:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197784.351015; Tue, 28 Sep 2021 08:24:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8Pr-0000de-Lj; Tue, 28 Sep 2021 08:24:43 +0000
Received: by outflank-mailman (input) for mailman id 197784;
 Tue, 28 Sep 2021 08:24:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV8Pq-0000dY-FP
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:24:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cefaa5fc-3415-46c3-a974-611253d5786e;
 Tue, 28 Sep 2021 08:24:41 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-JrhyZcmgNPC6bcmC4LJLBw-1; Tue, 28 Sep 2021 10:24:39 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4192.eurprd04.prod.outlook.com (2603:10a6:803:4c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 08:24:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 08:24:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0150.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1b::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 08:24:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cefaa5fc-3415-46c3-a974-611253d5786e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632817480;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UElRKBsHD6IT9SKEi4casJP8K2M6ZLpkY9DNMpGqoWo=;
	b=UKGcnZv/EuQT3JBh9WtBteexGIkp6ROPOEo+NoYeCcn8umJTo0J0qeuKlO6Holhhz1Xemz
	d+R4sALPueHIbOM66pt1gpjns1ZH+XFoAd+zpl+twJuS3IeN3/FoWvgg7w7TaWNMY1eNxm
	YZCgbDsW/jKgwrup7+MguRiLiaUagCM=
X-MC-Unique: JrhyZcmgNPC6bcmC4LJLBw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CPaiTNYBRAyqRbaj2+EkPpGxWAkYGG+no+IPtRhUMUMcsHPGR2Ei1PdnqvDou+QcDC07OT6icpR/NT4d2thmA8+pKamMCOfX61VT/jZTp6p3UgHCT0iSNWV9VFirVGPMlygpdiA5KpxCaprXI0gkVAEl3ZRD/lcz3Jvig4M/1vY57m82qMFpF1QSBfjMbN2QvhpgH1K6K3irntV5r/wtK+KKoAc1IY50GH3zikjXtCXsqNYyKfMgUvh4Vk6sqnT+C9jaPclh2/kmfUVoR7cnqqM4RZdv3nlOMer/Lsx4cLYy34KOvpCFdSC+KhB8pWuYmCZb4nZezsE9kEfB/hQvHw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=tR4AXSaKVfroVxqGofVIKZsWC/DUTb78WRJF6HG7UP8=;
 b=QOXC8ssdJKgmbKE91mp3zO3qmObywkII0pWtMf9euyC2xswnEjWGCt1HhPv3FzmkGF3YY0jc0bfn0WwlptGpJZlHZ4SoFeftZ8TT0J3H6qmbSzgduzwTBr0CrOQIbHAhzgVxTCQ3P5tK812HLT6bBKmBJ8Qs6V0v0FgjmEY3lHbtOliLHK4SBC6XeWMVN4I1vbwBs+odCwdl5y4u79cBnO9BqeXtol0f+1Zls8toOmrWmk9K82amVybyYDQWOVRIsCfp8l9AntcYXWCv838Ztqf3BnJe2iHM/o7fFpdaX27tv9qYDiBrSn/IhkE5KkVam2Aq1OOzF0HMhD5XarmPDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: Semantics of XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Tim Deegan <tim@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>,
 xen-devel <xen-devel@lists.xenproject.org>
References: <13e3368d-89e3-360b-36df-bc215227b7e3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <0a9f9f66-2abb-7156-44cb-80a78415dd58@suse.com>
Date: Tue, 28 Sep 2021 10:24:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <13e3368d-89e3-360b-36df-bc215227b7e3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR0P264CA0150.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:100:1b::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9ea08fbd-d304-4870-20f9-08d9825968fc
X-MS-TrafficTypeDiagnostic: VI1PR04MB4192:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4192A4EDD4B1601465E7DA9CB3A89@VI1PR04MB4192.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1RuwHU/7JKt7upeYpsipSVb1/qsSwjeK+CdRW4ScXyK3ey+DU9vc/WY2Gpe5zez7pxQBx1C6bn8B/nABZMCp1ujPZWm4e7IzsDRHtoFL1kdSkcRPWn+RXSwMYvTs/se/kuCObyCQbTtbFBG5qHw1guY4kBa5t0PZBNFu5hCDElgS1sjtu7sodY4q6cwhft3DRts4BFPJ1eFJMAoDnMmrRq0PvGQB3vbj9+yAJgtYs2E08Q+qYSq4Vk1gQqgwNQnbeEihiedbW78xKmvEaqJuqKGhLCEwGuSWvxlpkPhJQvjhoHxzvU/Zie5NKolVDHCcEXgQRItnFsHCgLuW8dzWxGv6VqOi1WbQ93tJ/DvQBfcdLrAku3aONxl7JsLTcQBn0N5XRl7bpzM9D6rxE7sscv4EGNxSPIX36cj2OcB+GCkWhSnHjkHNx9bjNwmS5SDZBHxL5cBYdY+X8qWXp6nGEavMEuutd6XLBPeoPYKkYvbgOpusCtI9M9PuewwQzF5zQTs9dLaA1P/HHZ5HokTpK5RftDsc5orLIklqif4WIna6X0vff/zZubWmzF25zszxSDsSvJym9958y4XU63PSuNYvCmeLd4SE1l6rI9yW+7nesZs3+mVkAxzfFZg3+EQE0gwncgQTLgpYQuKCyaRltkkx319VQLOOkjjuCMb++pEHX4w07Y33T5pXb7MHa+AW+R/Z5UXnSpRTn+w1NG+KAgL1nbhZByHVLPbz+Kt+y7J7QArS79rD7Aahb3270kDu
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(16576012)(7116003)(26005)(31696002)(83380400001)(38100700002)(186003)(8676002)(316002)(508600001)(8936002)(66946007)(54906003)(53546011)(66556008)(86362001)(6916009)(2906002)(31686004)(36756003)(6486002)(2616005)(5660300002)(4326008)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?kCObOcUhZVl+WsWGuTNWzj9S+hdrOfn/6H9T4VdFY2AoYIhetLlxTs4ETrbj?=
 =?us-ascii?Q?3thWF3ggGuNF8d9ml78NDyOD82y4zUMSUqsB9dYjbqjGovbyXFiPtvG94DiG?=
 =?us-ascii?Q?pEFabReYwb6pg5Bk/uQJrZgG+DtFnMWi5sdkLYArIsi3gz9hxLCuD7/cF6e8?=
 =?us-ascii?Q?wwKsZYJCsKmz8wmhVNMt/wnNiBJfzRkswsnfewRwe1ROh3t6i522izST5B1g?=
 =?us-ascii?Q?tYbr84Y/Z3F44+YEIxak3v7NGVzL7P5Sza+YxCCtO5UdTXwlwVuVhLXcjZzf?=
 =?us-ascii?Q?BIfc2CAofaAIv7CHJHnz6IpmfLtBl+ktrnSVhzegJze8/S5r49lV82GE71fe?=
 =?us-ascii?Q?XW2gxck/g8yYVJNJBuAAy4+jg867yB7XNcfaWxkFph8MyeDAKkTed5UJn7CG?=
 =?us-ascii?Q?+53VXrK3QFd8aWEBqFFkK78vzq5lGzLUw0vq53UZIKJ3vGsNyXo0ZT9btdJw?=
 =?us-ascii?Q?K1QJYpwBMuAwxI3eDwRuh8m/JF3Anh+jZM5yVvhd/M5muY09lkGJ4I71kr6g?=
 =?us-ascii?Q?l4uTUlMbDS9Yoe+E3bgelLEIPhj5hbKL3n5X4m3BSAY/qYAOsMlgKrl5XdiM?=
 =?us-ascii?Q?QUPoIAZsl+J3KM49xaN8OK5OwaftvX4WD8sk7DPYuNodwuR+155kyRVb9n8F?=
 =?us-ascii?Q?S6BAa9cN6qLHQU5wbN1ryfefEbrGiBLsFON7k90AKj9QXQYuyqSc37jXxqkh?=
 =?us-ascii?Q?A8eSfUCctdWhed2PTtdi6Hq47ORO+S0hmMKFowXyAPkPzeQO5NJNV3qyJtp5?=
 =?us-ascii?Q?ykk7Uk2rvQi4qcNlV3IDiVrnsnuYmHM1okk3VOlnZXBr64KBvbOsp1Qv+0iz?=
 =?us-ascii?Q?lDKB3fNqSxS0Ftfb6bcA/KIqSMd/PZ9O6zXA4lwdPkvmqmQPlrc0oRP/JJ23?=
 =?us-ascii?Q?BjwVA1vidzEkXwUvHfxlpjONI6453/fSA8M/ID7P+NoPsqJvycgBX3VK16q0?=
 =?us-ascii?Q?GxzwO3/P+NfXerV4XKZkL/fuo9Ty1vBu7B2R96WtnE8VFFopjj0hErF3rB4O?=
 =?us-ascii?Q?EG6yter9hlm/YHEzl+gHahhHv9BvyUcjubUiMNzc4leag+ObQL8gXepu7jL3?=
 =?us-ascii?Q?7LyqR+2yGGzwrApgfGNdKY/fCFRZ3euRGs1y0dUvQkqPVivrMWr/N452ZboO?=
 =?us-ascii?Q?DMP+k4Q6SinY4WtH10DcQ2ccTE0vSCOEstdFuTPqMh8THEZQp/AeMkaSeDf3?=
 =?us-ascii?Q?eDjHf8Da1NsD8Y5w9b1gRgNBgD4coVCj+h2HWIllYCmbr4pgPJYbXUbvuRTN?=
 =?us-ascii?Q?/yAkAWDz3frHhLoSLizYctFp2q6/l9lE3F/yNuAz8w2h/IoHJcQnxUN9ekMG?=
 =?us-ascii?Q?xyjGZ7bQbFM37ShS6zaod7OI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ea08fbd-d304-4870-20f9-08d9825968fc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:24:37.5959
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nojxprHFaf/80yDzb1xodPqwz1WORmm8PF4ceMOzs/ZKP7+zDBEA6Ur7dE4cEIWp+APsrwtn5ICJuVmzItkEkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4192

On 27.09.2021 23:01, Andrew Cooper wrote:
> A recent ABI change in Xen caused a total breakage under the Xapi
> toolstack, and the investigation had lead to this.

I'm curious which change this was; while it's likely one of mine, I
can't seem to be able to easily guess.

> First of all, the memory pool really needs renaming, because (not naming
> names) multiple developers were fooled into thinking that the bug was
> caused by a VM being unexpectedly in shadow mode.
>=20
> Second, any MB value >=3D 0x1000000 will truncate to 0 between
> {hap,shadow}_domctl() and {hap,shadow}_set_allocation().

This wants fixing of course. I assume a patch is already in the works.
If not, let me know and I'll see about making one.

> But for the main issue, passing 0 in at the hypercall level is broken.
>=20
> hap_enable() forces a minimum of 256 pages.=C2=A0 A subsequent hypercall
> trying to set 0 frees {tot 245, free 244, p2m 11} all the way down to
> {tot 1, free 0, p2m 11} before failing with -ENOMEM because there are no
> more free pages to free.=C2=A0 Getting -ENOMEM from this kind of operatio=
n
> isn't really correct.

It's questionable, but I wouldn't call it outright "not correct": The
function was requested to obtain memory (from the pool), so one may
view this as allocation. The set-allocation functions really are both
allocations and frees at the same time (moving pages from one pool to
another).

> Passing 0 cannot possibly function.=C2=A0 There are non-zero p2m frames b=
y
> the time createdomain completes, as we allocate the top of the p2m, and
> they cannot be freed without the teardown logic which releases memory in
> the correct order.
>=20
> In fact, passing anything lower than the current free size is guaranteed
> to fail.=C2=A0 Continuations also mean that you can't pick a value which =
is
> guaranteed not to fail, because even a well (poorly?) placed foreign map
> in dom0 could change the properties of the pool.

Well, I suppose outside of domain cleanup shrinking of the pool was
always meant as kind of a best effort operation.

> The shadow side rejects hypercall attempts using 0

I haven't been able to spot this rejection logic. Instead I'm getting
the impression that the BUG() at the bottom of _shadow_prealloc()
would be hit if shrinking the pool beyond what can really be freed
(i.e. in particular if any pages are in use for the p2m).

> (but can be bypassed
> with the above truncation bug), and will try to drop shadows to get down
> to the limit.=C2=A0 This represents a difference vs HAP, and in practice =
1M
> granularity is probably enough to ensure that you can't fail to set the
> shadow allocation that low.=C2=A0 There is also a reachable BUG() somewhe=
re
> in this path as reported several times on xen-devel, but I still haven't
> figured out how to tickle it.

Any pointer to one such report? I don't recall any, and hence it's
not clear to me whether that's the one in _shadow_prealloc().

> There is no code or working usecase for reducing the size of the shadow
> pool, except on domain destruction.=C2=A0 I think we should prohibit the
> ability to shrink the shadow pool, and defer most of this mess to anyone
> who turns up with a plausible usecase.

No present use case for reducing is a fair argument for dropping support
for doing so. That'll still mean an error return, which - according to
what you have written near the top - may still upset the Xapi tool stack.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:26:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:26:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197790.351026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8RY-0001G2-5D; Tue, 28 Sep 2021 08:26:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197790.351026; Tue, 28 Sep 2021 08:26:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8RY-0001Fv-1q; Tue, 28 Sep 2021 08:26:28 +0000
Received: by outflank-mailman (input) for mailman id 197790;
 Tue, 28 Sep 2021 08:26:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV8RW-0001Fp-Tm
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:26:26 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c48f43a5-fe69-4d6f-8695-3b86eddaf072;
 Tue, 28 Sep 2021 08:26:26 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2053.outbound.protection.outlook.com [104.47.12.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-PUD8BdvWN4GJlfbiiMPs9Q-1; Tue, 28 Sep 2021 10:26:23 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7040.eurprd04.prod.outlook.com (2603:10a6:800:121::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 08:26:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 08:26:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P191CA0016.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.13 via Frontend Transport; Tue, 28 Sep 2021 08:26:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c48f43a5-fe69-4d6f-8695-3b86eddaf072
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632817585;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=5JiDH8eAcYjgcrjsbsX7whOu9DhoYFpXwJVQ+CN319M=;
	b=arND/g6RuAfUk7DegxhVhjWa8pYZRCmwq6Ii7EP/iSQvneurcDhSHk8ZdrTxmUwPd4iDUk
	zZk02uGvtvhHbnw7JM5Xw6Wt7UwgAP36GGzS5e0z0faoa5f0HlKFKCgIr4W4LRiGbT2m/C
	E7MFzJI3G9DODE3Sfd3PsTpJ6AyDoT8=
X-MC-Unique: PUD8BdvWN4GJlfbiiMPs9Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jToAnn84MYsk4ofZvTOET8GsnnO7f2oqczKArtve3rmKjilsURHiefeAjKDJ20r8kHQI8FqKIumd+AmvEQ0NAMPeTVxepOjDUe3gm/oNTqH8dF4hAKbs87b0M1mOLLKYLjJuXea1gmPPXvwfaNTpsm0P2ROhVhx7bOqky9RAlJqlp+Z3UV/PmnMidGzDmFiMpK6JlPSjK05LyAap1++WSCcaNh0SQjMSitd95Jhnm9DtBF9JBX3QpIUSK/l9uMk0IXPxsplSfGEEY+YTTXHNWeSOuYpJ4i3ZDTABO77dTHdBCFjP8BLw9h9xVFGd373tCzGtZJb36A2SAZyrO0Icmg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5JiDH8eAcYjgcrjsbsX7whOu9DhoYFpXwJVQ+CN319M=;
 b=m9UPtk+t5csGbLCvCIggmCep/TuaKihR3EqtgzOYj+lmAyV6fVffiC8H7bTVWBoF2/ICY7/afb7TbIjM2u4TjTPEYvUv4yhNE6LvqhK2hIOEGw39vJwT0D1jHO+TwX2NsDSlu1GtyColOC7fmh2XYSZS7WKwm1rbzcKkMHStVTLbTtW7yCI7NAgCdJ+Tf1ynYRSuLjchp6X1GlucWH/2zg0Z80SOWVvhjM3E1IuodFPJIZYqNMSRcDVMB0FEPcjDgPqGV+mq5SRn+YGRWfHnp+P6IZW+rKCbIUeKCjUKDR993KVQlLnFya6lj0a2883Id7/y3GJOjLoW4Gr++7Fi7A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
 <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
 <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <31dc3429-b1f4-85ab-968d-c9054b77ac4f@suse.com>
Date: Tue, 28 Sep 2021 10:26:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR3P191CA0016.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:102:54::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8fe78d5e-ed11-480e-c2c3-08d98259a6b0
X-MS-TrafficTypeDiagnostic: VI1PR04MB7040:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7040CD06CFFF939E357CFECEB3A89@VI1PR04MB7040.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7DG5NmHmRqshC7kyBYkCOmvPJnILA8Rb7incYY+d+k+AiQ+VRQ8f0GGo/5b1mCJsk6L21eO6OSFcmNbs9NRWPAYRP03+uvKDIyLrnrNRHrhRI05kz9u9hnJ2bRSfN1fO7IictM6jwlwd20ljgTktAPPIwtL7s0louq77KZAEYzyrxSwJzB2u/cqf17c4DISWZcezr79v/TSMUCDd6dDUYHNbf5OfE8oMkfVt3Jas8M29bHtphgiTQFcEglqUD+ctcxklSxEW/aKEtTR44KNACirgF9XneE3S3pS5WP/1ug04BDedyL1i3NjrQmkj9gHVQl4IhB7JlAr4sUcx7bFF/c4RGXLqNu2bh7TELNad8ZXg5DmNsxTrtp1w3304F5VbSTQAleuKN/1QQq0qbCewcEaqYJR1+hj78KiSeBCq8enqQdoWU0sTCCSWvRLocm1nHkvyIww5stV+3DSBByhTxQSTQbVUGCoii+Oj147TK8d5XFXx0t8M/rdHhgaa/MvxNVRXHzFoP5gY2IyDq+vHIry2Yy8+pJhvkhXXr5QKd2MjRvECkSp+Vp3N3epNwgBUnjeWo0lgUU9cJ1x6tsGtPn/QzFsvD6nfl1936aLnvuZ0dEJov26RXv2gskWf9Cn7BsTbRxuNn9+4BWEtCKrBnQi7Nkvx/fDgKDilzn9pIz9Qoj4XVwq7aKLL/HSGCRtsq+ABkUCVOH+zDdYSQpBWKnSUCWlEGxoU5kkx4XwnggH9TUtoKZDaNNqZ+oyiF3Iv
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(66476007)(6486002)(956004)(2616005)(7416002)(66946007)(186003)(8936002)(53546011)(2906002)(26005)(8676002)(38100700002)(66556008)(31696002)(31686004)(36756003)(4326008)(6916009)(54906003)(86362001)(16576012)(5660300002)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TFZ3cnRHTmlRLzdoZGdnY3NlRzJwaGN5VjNRU2RBU0NkL3l5SlJzczYwRDVJ?=
 =?utf-8?B?WkRsN0JWTVJKNStqY1RDbVgwYWpUd0dOWTRnSG10SHdBeGpFY1FrR09KLytC?=
 =?utf-8?B?S21vRGc0YmZJQ1VmT2VlbDBjUmNoSXNydXRKVm11R0oxVGVodzY2c1VxRnNP?=
 =?utf-8?B?TTRkd0FnRXpIeStuRzVUL0NTRmh6ZnliMEVkT0RxNlhlbTNONE45aTJLOFoy?=
 =?utf-8?B?d0xrOEQ4cEhSbkVkZ0hvb2F0MkJFdnhnbklrYnQzQnJac3NzNjRreE9CbWhM?=
 =?utf-8?B?R2gyYkIzT2xDdkdWOCtxc0NqNUhFbUl2UE5oYTcwYUtkNEdUS1h4MTkxVTR4?=
 =?utf-8?B?MU9vbmx5TUQyTFRpcHNaOHVTTUlRbHNjTVNueTIzM2poUXV0R2wvNzcyZURS?=
 =?utf-8?B?R3luQ2J0OFlCcmZoK1FCMDZUbVRnZzNoVU1tcC9YS2Q5bmVyMFBFc2lFK21t?=
 =?utf-8?B?Q3R3S1RXZElXOVRzTjRoZ1VQVE5HbjRpaWhnY1MyZFE0NW1BNkhVcE02ZkRS?=
 =?utf-8?B?Qi9rY05obXA0SG5vN0NMYlhuWFduRy9HeXk3N3AyMFRrV0pSMFZoVlVoYytB?=
 =?utf-8?B?R0MxZWhTV3pUNXpkbWUxRGtmMUZaZmxIVVc4QlZlcnZwaDZNaXNBRUhpYVJC?=
 =?utf-8?B?Z3FrbEJKR3Q0UTFNRS81cGhpV2hvQ3UvRkRvUmpPek9CaGoySWxWZ20wYjlH?=
 =?utf-8?B?dmRTdzhMWUo1Y1lLQytPcFI4SDRsSUZpTjZKNjhwZDUyUUZ5YlpLZVNXMDlh?=
 =?utf-8?B?WmVRU1REWjVyMktIbTRRMXhjWlBkMVZnYW11Ykd0WHdYcWppNHB0WERwd00v?=
 =?utf-8?B?VGZscGtzVGxTdThDYjdydm5iSkJERWU2ZmlsU3JNUHVDRk1wWnZaWWZrZ0RU?=
 =?utf-8?B?cEhvK2VzVmkvOEsycEdxWTVrKzlvanJyREx0S3FhZCtjL1VUZ0EzZW5PeEZO?=
 =?utf-8?B?ME9VTGJHUlE2UWx3UFpFbU5VTVlqRXVxTUx6bGFhelZVbGJvTHZmWTFFbDBs?=
 =?utf-8?B?YjdKV1g5SGhVZzh0ay9mUGVaL3NKdnd6azcyaExKdm02NW1WaEJTd3picXVk?=
 =?utf-8?B?amdOZlBmclB4RkhGUkFLL1h3VFI1Wk1ac2xzZFFoL1ZJS1RDT0RYNDFIMGV3?=
 =?utf-8?B?SDAzYnpLOENUdVE5aWQrbklQNGJlL3FHd1BnUTFuV3laS2dxU3pQS1dRcmZp?=
 =?utf-8?B?WllSaEVsZ1JwTlJUSFh3djlkVFl3WlAxM3lyS28zSUw0RW4yNlhBWFkxNEVa?=
 =?utf-8?B?SFFxQVIxUWx1R2cxYWt6MkgxNEltUWZ2SGlIUU1WYzEvZVhIcjFoRXhMNmRn?=
 =?utf-8?B?VmtkbTdRU0FMTU5qTUJlRjRudzJhQlhPbUVoVkxRcG9rSHROV2NaNytHeVJH?=
 =?utf-8?B?VytObjVyUitvZy9KUUQrVlZ2M29JWnphOGxyMW41UDNNUUQvWHMvbzRkNmFD?=
 =?utf-8?B?aVNnR05NMi9sODhwWkJORGlMcmhQQ0Z1eXFXQ3ByTE1GclZpRUd4b3E2MWha?=
 =?utf-8?B?UlMvVWxmaWZ3MHVIWUg4Q0FqSE50MnpMUzFFNHpZb2wyTGM3bXBQRitUcFBG?=
 =?utf-8?B?Z25BWmcwQXdSU04xSm85ajl2QUhIV1BrcVNDVWovc092ZTlWYjJJRk5TVEVo?=
 =?utf-8?B?dllkamhreENoaFQxcVRrRjUxVXBzaFRxUjhEQVlRMmkzWmVBbXBOWHpYRDVr?=
 =?utf-8?B?dk1hVVZLejBpMXo4dGRNZ1NyV2VOOEdUUUNQS1g1ZmU5UGJXN2RCanVnRk1H?=
 =?utf-8?Q?AHA6T87o1yqdb4a9rdU/Z9bDHhQBdTm6HFKgyC/?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fe78d5e-ed11-480e-c2c3-08d98259a6b0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:26:21.1127
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QqjQxbv0qXv4uCIwMAciJuN9ejJTFIrq8KVQWRo9F5XanolwMTs9lfL6wnvqrtN7gbo3KBaoDO7E6ykcKOuK3Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7040

On 28.09.2021 10:09, Oleksandr Andrushchenko wrote:
> 
> On 27.09.21 13:26, Jan Beulich wrote:
>> On 27.09.2021 12:04, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 13:00, Jan Beulich wrote:
>>>> On 27.09.2021 11:35, Oleksandr Andrushchenko wrote:
>>>>> On 27.09.21 12:19, Jan Beulich wrote:
>>>>>> On 27.09.2021 10:45, Oleksandr Andrushchenko wrote:
>>>>>>> On 27.09.21 10:45, Jan Beulich wrote:
>>>>>>>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>>>>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>>>>>> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>>>>>>>>>          *((u8*) &pdev->bus) = bus;
>>>>>>>>>          *((u8*) &pdev->devfn) = devfn;
>>>>>>>>>          pdev->domain = NULL;
>>>>>>>>> +#ifdef CONFIG_ARM
>>>>>>>>> +    pci_to_dev(pdev)->type = DEV_PCI;
>>>>>>>>> +#endif
>>>>>>>> I have to admit that I'm not happy about new CONFIG_<arch> conditionals
>>>>>>>> here. I'd prefer to see this done by a new arch helper, unless there are
>>>>>>>> obstacles I'm overlooking.
>>>>>>> Do you mean something like arch_pci_alloc_pdev(dev)?
>>>>>> I'd recommend against "alloc" in its name; "new" instead maybe?
>>>>> I am fine with arch_pci_new_pdev, but arch prefix points to the fact that
>>>>> this is just an architecture specific part of the pdev allocation rather than
>>>>> actual pdev allocation itself, so with this respect arch_pci_alloc_pdev seems
>>>>> more natural to me.
>>>> The bulk of the function is about populating the just allocated struct.
>>>> There's no arch-specific part of the allocation (so far, leaving aside
>>>> MSI-X), you only want and arch-specific part of the initialization. I
>>>> would agree with "alloc" in the name if further allocation was to
>>>> happen there.
>>> Hm, then arch_pci_init_pdev sounds more reasonable
>> Fine with me.
> 
> Do we want this to be void or returning an error code? If error code is needed,
> then we would also need a roll-back function, e.g. arch_pci_free_pdev or
> arch_pci_release_pdev or arch_pci_fini_pdev or something, so it can be used in
> case of error or in free_pdev function.

I'd start with void and make it return an error (and deal with necessary
cleanup) only once a need arises.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:29:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:29:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197796.351037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8UD-00025T-JV; Tue, 28 Sep 2021 08:29:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197796.351037; Tue, 28 Sep 2021 08:29:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8UD-00025M-GP; Tue, 28 Sep 2021 08:29:13 +0000
Received: by outflank-mailman (input) for mailman id 197796;
 Tue, 28 Sep 2021 08:29:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV8UC-00025G-6F
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:29:12 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 27edd01c-2036-11ec-bc73-12813bfff9fa;
 Tue, 28 Sep 2021 08:29:11 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S7hJfG007295; 
 Tue, 28 Sep 2021 08:29:09 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bbxwg863j-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 08:29:08 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6898.eurprd03.prod.outlook.com (2603:10a6:20b:2d6::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 08:29:05 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 08:29:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27edd01c-2036-11ec-bc73-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eRg/z8mAXplu6nyxXNu0OQEg+HGlD11OubK6cigq7XSGSW3BsSXfmOrxBw74gtiNPD1zE0x8CTlkTIt4ZxrBvpo0AXo50YHEdvjtQvUJNRwcnsgDWnUtmndhtzKP0eobHd4Ks390Mr/QFxEp3mSZ5WBRXFPdU3VtLT+s4pPEdR0KgRQD44CdvYm1e+V2oMNCcWPHiWAqgsC3FsH8zOtqiJgBdNdeZ2FP9OdbLvecqt7z3iEKqZejOA76LFwtAB7Uz6Fb6oezZUIM3LV0ULLahLMmF21PignvGv50gwjozlrvcN7+acHAbJfXXciVW/dwNqLYpMVJfU1wQJPncymACw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=44pgsq5FKu1KQgzRii0te2V/GLkUjvWitiQ8jmFKhXs=;
 b=VjDkCJbj0Gh6TowXPyYvfip5wky7BtR3f+DNsR2QO3IdkzTLVNYl1RLU8Eo2C6xeew5sepKy+PnYDgFYdkF+URQNfKhpWulP9VC9aUrWAgwaUOStkakHgzCuCAhSbJB8kgE8IUKbuFaS80heFZJHLtnh6YrnmMR61Prg/PosvM30Xyas/+7KsGudvCvVnAO9Dtd+UIlT9ITuUHycJCJokKVLbRzvZ40v2j9y7HFbKXLNHO9COckYpGmm9VEQsM6o4/l8u/8logJLlXz1A4O53+5oIdwxcu7831Qd43scE0qjX0hRzpUtjy/pEnK1pATGUc30OpIhvBKtqKTgJSXzZQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=44pgsq5FKu1KQgzRii0te2V/GLkUjvWitiQ8jmFKhXs=;
 b=B+0dkLwTrSEioR+XdZRVDEWBHtUlF6ftiZVTtfDOIMe6Fy+rSupjPloqgyAmZKAyO/x8BkJRLSvCG91Vtpk1T2x4uzx6qjidMPhp751jgMUNd2HIEvvjU4AkNA1/kOOdHm7LwPMPyI0XERcTuwREViNhz6DPjKDHGCGZpDjRncnb7X8f8M3KCPTP7dI5Q/DMDPthzHw89aDEwKqmfG1rOD7jVDoh+ZXuo4es1iJhC8x+LBhfHc+7cVu7U7gcqttnC1zf0VUkZLbwerv2fbx4jTpJE7vQ51v1SEzj1+iN+OImQZNsWLHYsmg7jU499ez5GismW+2Nqn7/Fq2MrA2Qwg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: 
 AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrICAAAldgIAABLUAgAAG9wCAAAEGAIAABhwAgAFsAgCAAATGgIAAAMUA
Date: Tue, 28 Sep 2021 08:29:04 +0000
Message-ID: <13b9cc76-c7e0-be7e-61d3-fc42a41dcc58@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
 <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
 <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
 <31dc3429-b1f4-85ab-968d-c9054b77ac4f@suse.com>
In-Reply-To: <31dc3429-b1f4-85ab-968d-c9054b77ac4f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ade56378-29a3-45a1-cfd7-08d9825a087f
x-ms-traffictypediagnostic: AM9PR03MB6898:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB689850D8626FB38BAAF7BA9CE7A89@AM9PR03MB6898.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 +mti7etz2V/ZLQC2QpEK0MUlt6TGcWiPPbonxHZeHChUZYtNqQlHBRVsYnjpnepbuUuustHpzKbQi70BUYSa9wRMcV+i4oa03wPshiGS4NSR0IDOZHCA9pRxSnHG5FIPWNVjPy+g5DixHDjrmeQTheajsqRsLJQFlK2XrqSBGQcevLcwNFTEup224kdQHLEE6Gx03u3WnhuKnP8YTeD7bvuCIB2p2oAcG4fjaC9oQYGhnoNCvQwPepDIJKJe/+Gh2mfx1Te9AOaUDQSalANogz4dLetxfHu8Y9tTPeSsOjauk/5whLBaU/d3qwoy7rgcWOL1DOML4RM1HGnPS66UIo41oqVmqQHzlx2lUF24Pw9/a/HXRWJFOF6TpOHWmWFkQDpqHy4DIt58W6pqBwPnk+jnpjhCWvu+00oaGX06RiUEdRYnl+oyjZmmtzbdT1gS+frT0G+CTqoX+qdmWnWkqLJ4PUICwLtNYaSEcLzWxwHzhOmv5juQjyycARO7aD0WrSbRM3fo0kBAGthTqVmzh+Qu3351vpRtQy2l+mGEKT3LFudOD66e+5g2B41dQsSWrk0NCHdxpvc17R4TAE1BuQM+hVEYsETBPzLzbrEE1UMN8Q+p9jjSkJQ1ebYcmQWe17TUA4B4vyFrcLTqqg3fGd2QGNUp82+Ktrt2DBa2yqywzI+OcFDJGAWkTt0tFgkSKIYAnsaqD9Ic66kymd/qWce15tZI/4FtHD3jUWT3EGgGWNvXAhvapZxf1bQSxYxZ
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(38100700002)(508600001)(36756003)(31696002)(53546011)(38070700005)(6916009)(6506007)(86362001)(6486002)(2616005)(6512007)(66946007)(66556008)(55236004)(66476007)(5660300002)(26005)(64756008)(66446008)(91956017)(2906002)(31686004)(54906003)(71200400001)(186003)(8936002)(107886003)(76116006)(316002)(4326008)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eEg3QXEwZG9WRHBPNE5sTStGWmNyUDBnS0xZNFlYTmNzL1dQMXZqcHBza0RC?=
 =?utf-8?B?Z3V2THRXM012WWtiTmRxTjBKd0xTZ1ZxMHhGWlpZNHh0bnFZOEcrSVgrZzU1?=
 =?utf-8?B?ZDZ3QXdPTkFGYUVvMmYrbGE5YndoZ3VmdmwzZUZKb3UyRnY1UkZFQTVBeUQ0?=
 =?utf-8?B?SkxGUnpIbDlVQzVZeDJZUG82aDhpRm1RSndlMjBwWTY4NkdGb0tYdHcxYWlq?=
 =?utf-8?B?NE03RXo4cU1NTFZSTDhmbFNteHNsNDlyODJPcHl0cEpqQXRmOHVaYWdpQVRq?=
 =?utf-8?B?NnpzZ1JWeXM5L045cHAzWDZQazh1aVFiaTR6b3hDdnI2a1dTUUxOZU9hcTR2?=
 =?utf-8?B?bzl6TlZRblBqeCtMdlJxOXpIZUNxWllmcWpUeEowZmFWdEpDbktHN2g1WDlR?=
 =?utf-8?B?NmdTVmp6MC9SM1dFVUVJK2lMRTNpTHYvZ3RwZnhva3NjeFBrbFhtc0Zkam9M?=
 =?utf-8?B?V2lWT3orQ1l0ZUI2eEtDalNobGk0SmFHcTQ2SlI0WmZXY0s5VGdoT0prRnM5?=
 =?utf-8?B?M243ZTJHNE5Lcm5pWmF0WDdDZmNVSmxvSUxHSEkwdCtrTUJwK0w1N0xpc0p6?=
 =?utf-8?B?d1N6M1ZjcElRd1FTeVhDN0U3ZUtQVHhaWHNibU1kT1BDcitpYWs4cUF1TFZD?=
 =?utf-8?B?TzVQZ1c3TTMrdE5Wcy83clFMeGd4ekxJcWY0dmJ5Zmg2aEFlL0llcXJhMXdz?=
 =?utf-8?B?Y2RYZWpvZGVnbFJjWmgzL25qWXZVOVA1Nit5SWpONnU4dmUrSmpEc3FnNFZo?=
 =?utf-8?B?Y3lWWUg5OGk4WDZHVzdRcHM3WkdxSnlmTStObWNjaDhaQjVZdHo5N0V6a2Z6?=
 =?utf-8?B?TXp6Wkl6QnpBbktmNS9hUENJNnVSL0tWWTdLNkpYSEZ4eCs1OFlOb3Viekgx?=
 =?utf-8?B?empwOFlKd0lmSFY0UnUzem1HbUJHTVZBZSt0bHEzVGdETFY5RmV2aWRKNUdn?=
 =?utf-8?B?UDA3VWtaWndRRGl1ajJ1N1M3eWJCbDhrcGU5Qm55K25KaENKSEUvS2Jia1cw?=
 =?utf-8?B?b3FsS1plTHUrMmZuNG9VVWtJRHM3RWJObmVRMFhZTGhQTTdZSmpONnpETG9E?=
 =?utf-8?B?Qnlua2FLY0FJQ0M0QWpNZzIzS3BIdFd6Y0hEQk02cEsyVlIzSGdxM1VMVDM0?=
 =?utf-8?B?Vll1eHE2dWFidjV0TExobkszZjkwS254alFrNVlWODI2aXlkUmVwR2c0M1k5?=
 =?utf-8?B?V0NNdUdJaEdFY2gybFIvU2lMam9vMXY0cFV1MWNtdjM5bk0zUFRLSXpaVHU2?=
 =?utf-8?B?NXBkNG0rWGVyVnJ4am5MWFowaU9oV0h6SVk5US9Gelh0ck9vT0dZa3FXRDdp?=
 =?utf-8?B?MjQ5RTFxbWRyRnFyeGVYSERiRVFSQWJMRE9YTG4wdVBQejFoRi9xS2ZFTDdF?=
 =?utf-8?B?cUp6dTh1OXA4UmhRQzdqby9BbVU2V094N2djWGw1V2RHejViUkdveEgzTWxN?=
 =?utf-8?B?VGVMelFIdHVVTFZxY01pclI3cWhkeXRFbkJIeDFaODhRQ0EzRTJORG9QWTUy?=
 =?utf-8?B?TDBlbG5BaDBaT2JCT2dOTjBsV2daZWcrMG9ZWHZNTG1zS05ndmN4TkZpUnE2?=
 =?utf-8?B?VE9zUGNpMW51SlNZbG5BUDdNT3E0Qlh4RkFTYWdNcTlualZFRXBjWjVQOS9x?=
 =?utf-8?B?ZjBsak81bkJrQUtGZmRpOFdRZE1jUWRLNnRObExnbjhVSTduc2g0MGdLbEFB?=
 =?utf-8?B?RGFzZDd3Ti9wa1FtR0ZjUnorallxeWZXemxhZ1RrVW5hMGhxQmVBRkhDN25M?=
 =?utf-8?B?SVQvcnA3R3FCK0pMT2tVUGo1aWxEenI3M2FaaGdHeko4dTlOOWJCbmZSSm9M?=
 =?utf-8?B?TDBoNUJRS2NjNHVaaHVxZz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <9FC9899DA0CFC744A81EE117ED7CBA31@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ade56378-29a3-45a1-cfd7-08d9825a087f
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 08:29:04.9991
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: db+a2QosUF1WeDbrnuRDJb6rVekR+yImi12yR49uOTqp/BkMoUA30rdKSBBqAaWQZ2Z5rYH9Z20qU6zzy7Z9fJEHn0unFVsif3Do21JaWA6imXMrHd1causVtonJdil5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6898
X-Proofpoint-ORIG-GUID: NhgsiyyECtfuhiEPGL2E-KhbO-2T-Ts8
X-Proofpoint-GUID: NhgsiyyECtfuhiEPGL2E-KhbO-2T-Ts8
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 spamscore=0
 malwarescore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0
 impostorscore=0 clxscore=1015 mlxscore=0 phishscore=0 mlxlogscore=999
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109280049

DQpPbiAyOC4wOS4yMSAxMToyNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI4LjA5LjIwMjEg
MTA6MDksIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjcuMDkuMjEgMTM6
MjYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI3LjA5LjIwMjEgMTI6MDQsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxMzowMCwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMDIxIDExOjM1LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAyNy4wOS4yMSAxMjoxOSwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDI3LjA5LjIwMjEgMTA6NDUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gT24gMjcuMDkuMjEgMTA6NDUsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+
Pj4+Pj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+Pj4+
Pj4+PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+Pj4+Pj4+PiBAQCAt
MzI4LDYgKzMyOCw5IEBAIHN0YXRpYyBzdHJ1Y3QgcGNpX2RldiAqYWxsb2NfcGRldihzdHJ1Y3Qg
cGNpX3NlZyAqcHNlZywgdTggYnVzLCB1OCBkZXZmbikNCj4+Pj4+Pj4+Pj4gICAgICAgICAgICoo
KHU4KikgJnBkZXYtPmJ1cykgPSBidXM7DQo+Pj4+Pj4+Pj4+ICAgICAgICAgICAqKCh1OCopICZw
ZGV2LT5kZXZmbikgPSBkZXZmbjsNCj4+Pj4+Pj4+Pj4gICAgICAgICAgIHBkZXYtPmRvbWFpbiA9
IE5VTEw7DQo+Pj4+Pj4+Pj4+ICsjaWZkZWYgQ09ORklHX0FSTQ0KPj4+Pj4+Pj4+PiArICAgIHBj
aV90b19kZXYocGRldiktPnR5cGUgPSBERVZfUENJOw0KPj4+Pj4+Pj4+PiArI2VuZGlmDQo+Pj4+
Pj4+Pj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQgSSdtIG5vdCBoYXBweSBhYm91dCBuZXcgQ09ORklH
XzxhcmNoPiBjb25kaXRpb25hbHMNCj4+Pj4+Pj4+PiBoZXJlLiBJJ2QgcHJlZmVyIHRvIHNlZSB0
aGlzIGRvbmUgYnkgYSBuZXcgYXJjaCBoZWxwZXIsIHVubGVzcyB0aGVyZSBhcmUNCj4+Pj4+Pj4+
PiBvYnN0YWNsZXMgSSdtIG92ZXJsb29raW5nLg0KPj4+Pj4+Pj4gRG8geW91IG1lYW4gc29tZXRo
aW5nIGxpa2UgYXJjaF9wY2lfYWxsb2NfcGRldihkZXYpPw0KPj4+Pj4+PiBJJ2QgcmVjb21tZW5k
IGFnYWluc3QgImFsbG9jIiBpbiBpdHMgbmFtZTsgIm5ldyIgaW5zdGVhZCBtYXliZT8NCj4+Pj4+
PiBJIGFtIGZpbmUgd2l0aCBhcmNoX3BjaV9uZXdfcGRldiwgYnV0IGFyY2ggcHJlZml4IHBvaW50
cyB0byB0aGUgZmFjdCB0aGF0DQo+Pj4+Pj4gdGhpcyBpcyBqdXN0IGFuIGFyY2hpdGVjdHVyZSBz
cGVjaWZpYyBwYXJ0IG9mIHRoZSBwZGV2IGFsbG9jYXRpb24gcmF0aGVyIHRoYW4NCj4+Pj4+PiBh
Y3R1YWwgcGRldiBhbGxvY2F0aW9uIGl0c2VsZiwgc28gd2l0aCB0aGlzIHJlc3BlY3QgYXJjaF9w
Y2lfYWxsb2NfcGRldiBzZWVtcw0KPj4+Pj4+IG1vcmUgbmF0dXJhbCB0byBtZS4NCj4+Pj4+IFRo
ZSBidWxrIG9mIHRoZSBmdW5jdGlvbiBpcyBhYm91dCBwb3B1bGF0aW5nIHRoZSBqdXN0IGFsbG9j
YXRlZCBzdHJ1Y3QuDQo+Pj4+PiBUaGVyZSdzIG5vIGFyY2gtc3BlY2lmaWMgcGFydCBvZiB0aGUg
YWxsb2NhdGlvbiAoc28gZmFyLCBsZWF2aW5nIGFzaWRlDQo+Pj4+PiBNU0ktWCksIHlvdSBvbmx5
IHdhbnQgYW5kIGFyY2gtc3BlY2lmaWMgcGFydCBvZiB0aGUgaW5pdGlhbGl6YXRpb24uIEkNCj4+
Pj4+IHdvdWxkIGFncmVlIHdpdGggImFsbG9jIiBpbiB0aGUgbmFtZSBpZiBmdXJ0aGVyIGFsbG9j
YXRpb24gd2FzIHRvDQo+Pj4+PiBoYXBwZW4gdGhlcmUuDQo+Pj4+IEhtLCB0aGVuIGFyY2hfcGNp
X2luaXRfcGRldiBzb3VuZHMgbW9yZSByZWFzb25hYmxlDQo+Pj4gRmluZSB3aXRoIG1lLg0KPj4g
RG8gd2Ugd2FudCB0aGlzIHRvIGJlIHZvaWQgb3IgcmV0dXJuaW5nIGFuIGVycm9yIGNvZGU/IElm
IGVycm9yIGNvZGUgaXMgbmVlZGVkLA0KPj4gdGhlbiB3ZSB3b3VsZCBhbHNvIG5lZWQgYSByb2xs
LWJhY2sgZnVuY3Rpb24sIGUuZy4gYXJjaF9wY2lfZnJlZV9wZGV2IG9yDQo+PiBhcmNoX3BjaV9y
ZWxlYXNlX3BkZXYgb3IgYXJjaF9wY2lfZmluaV9wZGV2IG9yIHNvbWV0aGluZywgc28gaXQgY2Fu
IGJlIHVzZWQgaW4NCj4+IGNhc2Ugb2YgZXJyb3Igb3IgaW4gZnJlZV9wZGV2IGZ1bmN0aW9uLg0K
PiBJJ2Qgc3RhcnQgd2l0aCB2b2lkIGFuZCBtYWtlIGl0IHJldHVybiBhbiBlcnJvciAoYW5kIGRl
YWwgd2l0aCBuZWNlc3NhcnkNCj4gY2xlYW51cCkgb25seSBvbmNlIGEgbmVlZCBhcmlzZXMuDQoN
ClNvdW5kcyByZWFzb25hYmxlLiBGb3IgeDg2IEkgdGhpbmsgd2UgY2FuIGRlYWwgd2l0aDoNCg0K
eGVuL2luY2x1ZGUveGVuL3BjaS5oOg0KDQojaWZkZWYgQ09ORklHX0FSTQ0Kdm9pZCBhcmNoX3Bj
aV9pbml0X3BkZXYoc3RydWN0IHBjaV9kZXYgKnBkZXYpOw0KI2Vsc2UNCnN0YXRpYyBpbmxpbmUg
dm9pZCBhcmNoX3BjaV9pbml0X3BkZXYoc3RydWN0IHBjaV9kZXYgKnBkZXYpDQp7DQogwqDCoMKg
IHJldHVybiAwOw0KfQ0KI2VuZGlmDQoNCj4NCj4gSmFuDQo+


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:38:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:38:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197805.351048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8cx-0003ig-Jg; Tue, 28 Sep 2021 08:38:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197805.351048; Tue, 28 Sep 2021 08:38:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8cx-0003iZ-Gm; Tue, 28 Sep 2021 08:38:15 +0000
Received: by outflank-mailman (input) for mailman id 197805;
 Tue, 28 Sep 2021 08:38:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV8cw-0003iT-Av
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:38:14 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2f37caef-b9cb-4d2d-ba7c-414ae0554c20;
 Tue, 28 Sep 2021 08:38:13 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6-znf6ldxrOD-zwj04Bi_IUw-1; Tue, 28 Sep 2021 10:38:11 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2446.eurprd04.prod.outlook.com (2603:10a6:800:4e::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Tue, 28 Sep
 2021 08:38:09 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 08:38:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0007.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.8 via Frontend Transport; Tue, 28 Sep 2021 08:38:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f37caef-b9cb-4d2d-ba7c-414ae0554c20
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632818292;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hx1xK7HNytdaZ6oyQpvXTcUcCpRyZzwnXo4wE9i9NKI=;
	b=H7/o4AU/AMIVaqN7PnqhgVJ7G06VruPqxRrC3CZEhoMssZG+cUxaZMAMkLGHCsLYn/N2Xt
	hGAz7mXTW1ZdOGfyXor0M0xGrhoF8jwRnrU7J/6KLJLNnjfdBBVMRSaUntMwuiXbasnFGT
	lNdb+9VtIWF4PRX1BqiyZTVmMiXKVMY=
X-MC-Unique: znf6ldxrOD-zwj04Bi_IUw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LL/55CX1yd+8ozBKCaM/vrN7Cq197z/F/E5NETA2sGbba2M7bdrWLG0xk1OOEqJpqk9putukjXMW6jwjtTTr+gGkJNEVL7S7Mb9k4NpDYWLyBS6EY+TCObVF1fQomGq0WHi+r3wmY+OK12c6I9uudv1y/nRxdIQk1z/Oldrw04ee0I5qe0hYn9bh19tmYlJLoEgJ7214jWBUIypyQOJ8g3FDEMX0wshkh/Z5ctcESwXCQmWOlirLOSf0WUGfks4TmBEN1lDHub1zTcgU2LJvpzAFUm+F+86M2afSPhoS4WT7IeNb6wyQsLYGp3UBEp4/J5VaIIh3uUkrajfrjPsuew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=hx1xK7HNytdaZ6oyQpvXTcUcCpRyZzwnXo4wE9i9NKI=;
 b=HOSMbEO2kpkR2SRzlDpo5iZeXRC2bHIezV1eKNVq4FVcsa92FlXusFxoDbwg+25mSfoyIhPUbwMS6Cu7fwwmjwagm7n5MEgVZkKmwBz7LoSK1qZbH8RH0VZBzvShrrXQMbyE+Iy+td3WgPzf+Br8lfl7XDF+8ipASfUYrRmVlEV1dflh99VPFuC1J0zFZloy7B3v21wqHMmb0cMazZvIGHhfXavJCVgkS3ak2ezvLxh1E3IBv75W4zbiIyfWJYnL/TiSTkaL48ecnoIHvYYzT2yp0BGAspMniFv4MZgZW+Isf7uOa8YFhbROfVLGJtJ7P70zZDf3/58Jjx+k5MHHNQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: sh_unshadow_for_p2m_change() vs p2m_set_entry()
To: Tim Deegan <tim@xen.org>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ae8b6cf1-d1ae-8a55-7b88-7ef24db0c554@suse.com>
 <YVIo1sR283L/MUeN@deinos.phlegethon.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7400a394-2866-b4e8-42b8-b04e6f0bf4ed@suse.com>
Date: Tue, 28 Sep 2021 10:38:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YVIo1sR283L/MUeN@deinos.phlegethon.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0007.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f867db37-3a40-4cd8-f2fd-08d9825b4cd6
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2446:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2446F99AAB4779DF4611FE3DB3A89@VI1PR0401MB2446.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	aNh2NBMoLLOkycwuvCuqEyZQxWsl3ot5YlUfie8e6yLyWwx2nGyvHjceU9bgVf+gN8ay+92iiH0/eOij0ZPgKwrrz6WKUd767TZZgfQ8lBK6nfmCluKHwEzVusTk3JwQ7HWOgTaT3jjZ4GyhPMz8vkyZIEm/gXrnZnqXapIR7RniujeEs11S5KcG5msMP8qMGUQbkJ/e0OgDWidO4aeUkRU0gRnFsn8tqfNCFYkjSqXXdc9YQYwOE6sbaOGmB01oZRW3JRAuREgo3fa0T8QRE2sjKyTay3bt504I3rNebsXGgJLBhcmwBCUsLecQ9qP/kJOdkEgCiUUlcjLjxG/zq+KWf+oIh/g8TSukHSGOk3hMTCx9waxChY8Uw5k7Puh7NdAjmYEappVO27VrH1DwjEtzShOiFqjy/IeI8YtsoNH4uk2WeDj3Ln3/AIKHR8D9KjeJfh4iQCpzWoOcZVNWrOrdyY6kZ52umLET/9Qc5QE6STOaydPnlb9tQluq4FDTmRZVHov9S7t6Ur7+TAAPcoWVrjUs+TKrjT84Lvk7kODFkAlgR0YgyciEU2MmTo4u4hsXMAnxETHrg+UCqfs9LF+ZrxVfSmzM5f9UTAmJR53pEBMj3Fke2Q8WPx6mmCRWucDW3QTllCPxcgw18J5lAtSqR0+ZWbk1dxgtoK++w/MIQ9wZ6BfKKafRR8rzCVfi4UnwSNw6bSstthE5g78z5WbbuLYdhQmoB/Qf0UCmSDy/kC/vgFEmrwqDzFZpxfW+lDnPQIPrY+hzPn6tJyzFrQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66476007)(86362001)(38100700002)(66946007)(31696002)(31686004)(316002)(36756003)(6486002)(4326008)(26005)(16576012)(8936002)(6916009)(8676002)(508600001)(53546011)(956004)(186003)(2906002)(2616005)(5660300002)(70780200001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bWg2Vk5KUm9EcjhxNHFOT1FTZ0xidDhGOW1yZzROR2lXd2VRVjhPVXgxZWFR?=
 =?utf-8?B?cUlqUi9IYkdObVliQTg2Tm9oN1ExdmFENVlxa3NSRUZYenhSMktHUkY5YXVU?=
 =?utf-8?B?d29SSzR2L3plY01tRmJOSGpFVW14eW1PV05GK3FGejI1Yjd5WXZDUSt3T3VQ?=
 =?utf-8?B?cEIvZjVaakMrN29NMGZRR0dsclY1UUFIZEwzcTBmNkwyMU1QY3J6a0Z4dmhL?=
 =?utf-8?B?VG1BQ056cWh1QSsvaENlV3VJQ2QvT1VTZnYvK3FRbE5mdWt2cURvRjZTbkdW?=
 =?utf-8?B?RDR3SGtzTjZKRktabytWNWQrVC9rSmNhaWdTOG9MR1FvQ1pDcG1OeTlNTTBh?=
 =?utf-8?B?QTM4NWE1TDRwVWp3cThGMkExUHdXSXJHdjRVV0dMS2RtVEJ3OStXV3Z6R3c4?=
 =?utf-8?B?MVpRcnFWb3QwQjdNWnIrKzlKamg0ZlQ4ZTRGUW5aRU1KMUs2M3JSeTRjbzZK?=
 =?utf-8?B?UHBrVUN0eHFkTStRWjI2YVdlOWVpTCtOc3IxNENxQXBQSm9ycDA3emRpT1hX?=
 =?utf-8?B?RGJYQkYwemUwRHFNVUpTZzFQN0Jsd3lNNXplQlV3RGVydDZZSDVET1dsSXFW?=
 =?utf-8?B?dFQyRy9lV0tvbHNhaUZNVzdXUmdtekptRWEvL2hpKzJlRXZoeHVEMTdmN0xE?=
 =?utf-8?B?TGt5K25TQW5XMDl2L3lWRmwrblE4Tm5MSlk1NGZMank5Z0FDMmZFc3Bmdk5l?=
 =?utf-8?B?RGlXaUdLY05xaExKalFpS0wxa2tKNzBMZ0tqdGJxeXdZTW5GcFRla01lMDFm?=
 =?utf-8?B?U3QzaXI5WG4wcXFIV0NHcFlTNldnNG1lM3AzTERpbXJiKzZudXZZaC9lcnpt?=
 =?utf-8?B?UGwrS2FHTHV4d2xLUWFaTDZDZGx2TEFyVmdOU1FsZ2crZTdxVFRNMVlidDFn?=
 =?utf-8?B?ZU1CZ2x6OUllV0ZpVnVnME9DTHFrajgrUm1Xb0xGN0RCcjJhRWRUTklKdm44?=
 =?utf-8?B?MjhzUDRaeDhkWUxWNElnK0N6TDI3TlFhVkhOWm5Hbkp0ZEJweThEd3BRUkt6?=
 =?utf-8?B?blVzaDd3WFJlQXhlZ0lFUE9WeEQzVGR0anMxeWVmaVdxYURWS0tzT3JLakRo?=
 =?utf-8?B?WVYvbUpvWENrbEdZcnNTTndFYVl2RDJnUW9Md0tBQmJ1VkNpRGFHOGJTMjNR?=
 =?utf-8?B?K1p4cWwxTzdYQnUrczFNYUZ3STlSSGlBTzJqWXd2L05HbHVIcG45QncrQ0lt?=
 =?utf-8?B?b3FtK2xHaGM4Z1ZrOVZxb3NxMHhFSURhZ1dCTDArNmNGa2xFTXBhNUtJV0Jy?=
 =?utf-8?B?NWcrdXQxdGVFVHowS3NyYlBZV0JPWmYxNjUwQWoxRXhIN3Vrb0IyWEJoRDBT?=
 =?utf-8?B?MCtPeFlhOFlWQWg1eXI2d2NvV1NQM1JzZjErUUU5QWhHbjM0NEJWNk9oT2p3?=
 =?utf-8?B?UmV4T0RCbDB2OEZVTi9QU253bFNIOWZpdjNzRkwya212QTFqSzZrUEJES0Vl?=
 =?utf-8?B?aVAxWm9md3pHZVNKWDdRYTJMMW1SWTJiTXA1cDBRWGsyU1FDVjJldGpZTEp4?=
 =?utf-8?B?Q1pvcm1vS0xUeTVBZ2I5b0xIUUpvNmlYMFQzL0x0KzNZMnAvTFQzK2FXTko4?=
 =?utf-8?B?ZldpZ01GMi9VV0YzaVRVbmlGTUlrcWVKMWlPUUxZbFhmaWg5b1dFMVhJUTJi?=
 =?utf-8?B?MnJBZ1A4NmpMdWdKSnhzR24xbTZrOG1zNHdrNk0zeHJxN0phYVB4eU45VHNt?=
 =?utf-8?B?aG5oUGZlU0JyTzNZRlhKSnZmTCt5a1ZtaWpvVVh3aXVOK2ppT2hlb20xS0dy?=
 =?utf-8?Q?Fl4TJ0Gzcjemlqj50M3cCDaOOFoFGdZguQrwC75?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f867db37-3a40-4cd8-f2fd-08d9825b4cd6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:38:09.3595
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Msdz0smahRDI+XRkMXJa2I/BM0Da8FMnUhGCi1+E4NwR9/WWW4zcFZ2vhohwhZf4wV5NF0UyuvVG5bFvMBobgQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2446

On 27.09.2021 22:25, Tim Deegan wrote:
> At 13:31 +0200 on 24 Sep (1632490304), Jan Beulich wrote:
>> I'm afraid you're still my best guess to hopefully get an insight
>> on issues like this one.
> 
> I'm now very rusty on all this but I'll do my best!  I suspect I'll
> just be following you through the code.

Thanks much!

>> While doing IOMMU superpage work I was, just in the background,
>> considering in how far the superpage re-coalescing to be used there
>> couldn't be re-used for P2M / EPT / NPT. Which got me to think about
>> shadow mode's using of p2m-pt.c: That's purely software use of the
>> tables in that case, isn't it? In which case hardware support for
>> superpages shouldn't matter at all.
> 
> ISTR at the time we used the same table for p2m and NPT.
> If that's gone away, then yes, we could have superpages
> in the p2m without caring about hardware support.

No, that code is still used two ways, but it can't be used for the
same domain in both of these ways. IOW I'm wondering whether the
check for 2M pages to be usable shouldn't be "!hap || hap_2mb", as
opposed to the 1G check continuing to be "hap && hap_1gb". Of
course once I make that change, I may end up learning what
"potential errors" that other commit was talking about ...

As to the further parts of your reply, I guess I'll try to
transform this (largely supporting my observations) and the above
into one or more patches then.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:40:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:40:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197811.351059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8ed-0004KW-WA; Tue, 28 Sep 2021 08:39:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197811.351059; Tue, 28 Sep 2021 08:39:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8ed-0004KP-Sz; Tue, 28 Sep 2021 08:39:59 +0000
Received: by outflank-mailman (input) for mailman id 197811;
 Tue, 28 Sep 2021 08:39:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mV8eb-0004KJ-Ur
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:39:57 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a8793d24-2037-11ec-bc73-12813bfff9fa;
 Tue, 28 Sep 2021 08:39:57 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-1-0zOHd_wSMy-5Dwt0vllfpQ-1; Tue, 28 Sep 2021 10:39:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 08:39:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 08:39:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0059.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::21) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.9 via Frontend Transport; Tue, 28 Sep 2021 08:39:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8793d24-2037-11ec-bc73-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632818396;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ayiNVvzyi8NkM/XX6T5mey5N/UX5qQalJLIXyS8enFY=;
	b=TgAl6OsIQdFyoJpHi/plT3vz4LoTvPweQmBwEsF63QUgGT7vb4fEIv4/2G+rWuHZglpdkr
	WgKEIwEX3iQUldpgn3XietB0O0gqcM9LBGDsCgAFCsyUzyTMF/G0MJbzTP+lKqGoxtrO0C
	Puo14GmGUDMA+r1LxjaPJrfng+6XY5A=
X-MC-Unique: 0zOHd_wSMy-5Dwt0vllfpQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ejolo0BbWCs99amFInIQLC3prPR1PC8K9bXX9UEqY/g3djgmBF3kErnGQJVPlPuhRjysUney1bn+EUXDDLE1EdjOsGOAer3jf6GJFxDvcZF8Os5t56xM8egLhZTQkbWjIg1XacdNCRjPqkt+4lW+fwAO8AIMjzGsCmPNzy/6KHrU/EAlnvR0FCPGt0adger1xIVgtigXu2SoM+x4JjnkI2LoTMAu9oLRBD4sz9cK9KVacJWqRK7bf2MGwl3/b9YXkAsNOf9SKpNCW2XAgF5qyrF8SxrkVpFpmV+IxTFjo0VptL/I46sysSYzTP+5COUvuJk53qTIetvXlh4mYsaLWg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0zFOdtgwqFf2erdagpj2hONURacGfoJvsaNbMGLLovk=;
 b=cZGzlg/0+7dhcxT+viRPLkNawSthd8SB4LGBhDImLdJs2oHIdfLFfxP+0W5mZtuq03fJQBMPXCN3H9+1Mya5t+sRzwHSzmVB/JIcAQzk6bxXF4rphRo3Cr8Esi/CQpkuHT0vxEG7eU2Wquubil29EHbhPe90ZYZMbpyszTPQ1XUlR7SNwrgUaGxWZMzS2ZZFyl3kEiL6VABie2JBctrP43h2+KpOvwfGkJP5eGt814F3vLge670t1qNyftTcFi30CcEMCkLWL9DBOgnyb9vmaYb5VivCaLYYd1HjdoowxcRDQdVJv8X8G32Z3sOSYw98RwoBzBI0Zzlt9V5hR0Yrow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
 <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
 <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
 <31dc3429-b1f4-85ab-968d-c9054b77ac4f@suse.com>
 <13b9cc76-c7e0-be7e-61d3-fc42a41dcc58@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8a3c020c-687c-1818-3a0f-5e8a553fe58e@suse.com>
Date: Tue, 28 Sep 2021 10:39:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <13b9cc76-c7e0-be7e-61d3-fc42a41dcc58@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: FR3P281CA0059.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:4b::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0da94c70-8883-438d-62c6-08d9825b8a8b
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6304B89427298E4CA5636F11B3A89@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5QQjhzWWmSwMB6sO65liybwbBI/HjLgnjM7cbqVC/K/1+u4FN/dcPYJ2aVM1ccCv5eV9vupHWbBQSvslMEhGYZ+ILcC4R95VQYcvoxltn5wHvydOrsiTkd3dYp0ZDjD5PFp1eaOTIZDdUl5UnbgU+tYUeB/aILyFpB6pHRW1m0NyltVx1TX5LySsHqi3ogvu+IthFa/yoPWItGPgeIvRWRONYBxXjOF1X5TAQIsVtQIPlek8Ip0+hHq6JZTMTh/8AenREg3INOzCQyWO73B78EJoDbFt13P3i40oihK1VYYkOjZNJvCJ52yS9Xk6juFCaewVsQmQR9/5IfKO+P7ey1FkhGjo8RtGG7avNWFusDbCoGQyH3rnxsIfTZGhePLLX5GeIB1oZqZ2O27a5XGKO2aZ3hdGjZ0RBWNbx83YppyFTcQkIZs6vYxcqet11OU5Z3aYUce4MqcxVMRFoJvNuyBk4BCSZiBLcONX7SZKzBnLWrfDOK4sM8nkAkVhxjvnQ2qAAeMXDtuwlWxjKlGnypQEN9BkwNPrEc0Ud0i4gRwdT1r76S1GS8oou6InOEqjq4+ZSG6D21BIMskWOzK98PVazZ0U9wpkqP/zEOtXVx3uxWnlJa64grzQeoKd3VvSUlMj6vxF1ILMY9iaQ7IZhi3WH/VVZfWpULjw1Jt70+FvOsSh0xPnVOqmzj6zsMy5qGI/mff/O9Ilv5zxTwa9MywG49+Otd5GXHITis9BQ62bg51rlONizrCehVH7tyPG
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(31686004)(956004)(2616005)(36756003)(8936002)(53546011)(6916009)(2906002)(31696002)(316002)(26005)(66946007)(86362001)(8676002)(66556008)(5660300002)(186003)(508600001)(66476007)(6486002)(38100700002)(54906003)(4326008)(7416002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?27/fm9kyQ8fOL70gmdoLufNQi1yzpnkSb1RLI2k4EAHuLekk3cMAxtqukHPp?=
 =?us-ascii?Q?ERFdZfgQlecWx2UoZXL+Kla7H8ezoU7TechTQh2s0SR3ORFVsH+oBGE+j5Tr?=
 =?us-ascii?Q?0FWoBHVr9a30a5lzElKBNbPQ1204bFnYRQlLwuBbERrrI4uQCzRN69oWaVGo?=
 =?us-ascii?Q?1crfWUQ3bQhQ5jM1nDUF447H1f3qkCejkzEoEpRrmJLvNdqRRmjQHXYOY40S?=
 =?us-ascii?Q?21vvST+Ezy5lBGwYHXuIdnE40+OhkyTlsCEHthaw/+18js1cFz7M3f8KY0Zv?=
 =?us-ascii?Q?KT1RtQBB3x0eSeBMvzIcwu3RgbJ0X7tZV2+R21XCd0VPTFYzaycOd3WmWhLB?=
 =?us-ascii?Q?D+0pT0/4U3vYov+cj0C+t5+HYYdvAiEBLpd/W/GLtro9QqhXo+vIXLGlxvSd?=
 =?us-ascii?Q?3zaa+ubq+k6CoHWTrIIBKzF0+sXnsPcSM1nA6MVtH7+uLvJg6s5CPXJLKfJN?=
 =?us-ascii?Q?61rbguQUTJgN6jLS8rFEPUcVCyI0e04By2ACSiazbvBm8+eFeBY25l9/DZzY?=
 =?us-ascii?Q?ssNq8SHHWshq5apS6Wxy1o7NBlAV6WSndmhU+pcmHvRE8ROA9ORdXFT0sHL8?=
 =?us-ascii?Q?ZF6ApuA7r8ju/NH/PFJ8O//D77ShULAU4FQtl/A4FICdMenmteLjj/N5JQqh?=
 =?us-ascii?Q?Gq9XMGTcIMSDms66r085rGAM1HQXQAK98iXmLl6DBs5k+Q6q911UEQrivDsV?=
 =?us-ascii?Q?ct3UnpYibyIYo8S4oiatigJBZtRtZkg3letu0cNmCpZoncjQ8GHzpgEYAg6k?=
 =?us-ascii?Q?4YIbCDsPLwq/ahO03eW0+ScHFBy/PIXZ7gD46NzcLoY2oR9HZxJOgxZsIJf+?=
 =?us-ascii?Q?7heYE101OoA914pp+HRyrjMCnxgMoAj02OIok/k7EliHitXNg/NU2rZ05nW4?=
 =?us-ascii?Q?MVHDJ8Htmyy1FrM233TzxdY+I07mkr0H+BA6PxaVzQ3kqiJuGYaj7EgYh620?=
 =?us-ascii?Q?wbdMktdHlNQuy1ksi15/Gk1ozbsxRZ4FY1JBBozHHNHR1DlOPJ80m5yzYJZ7?=
 =?us-ascii?Q?oBc9+9RolmVpZsk/M8iubWPE7PLQOFvxNgx+xgnmHJFxg35X4GbHbiVZbUes?=
 =?us-ascii?Q?RATYqXI5hJnsLXEFJeEDcWdcm8DoTLrPpBuc57ZrCm+PXa74StgOGAiZyxmw?=
 =?us-ascii?Q?4Ilw4/0WQ78D8Xs+kne1MZHYmPATr3NmKx66hkAZ4Wh0wDD0zN9bPR92+BKP?=
 =?us-ascii?Q?0RnCssb0qCUcmBtuf0ilHcuR6nGLsfen6YsBzyf7PaivVQ7IC/Np2QIp7ap1?=
 =?us-ascii?Q?pGfrqLfKtTHciYWXTc1LCqiBw6HrGNQ8Gg63JjK8Cc97xH+ilXfL6vnVfe8e?=
 =?us-ascii?Q?EmcJmQ9RJzu27j4k34Yjqngl?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0da94c70-8883-438d-62c6-08d9825b8a8b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:39:52.8720
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MjW1UHRPOrqG5TpAjfUe0uJDpHEqj1NuO8sG5DPqxBJBD71gSYh+azqb9Q+msL8indmlIultbqLq1qBYjhcx2w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

On 28.09.2021 10:29, Oleksandr Andrushchenko wrote:
>=20
> On 28.09.21 11:26, Jan Beulich wrote:
>> On 28.09.2021 10:09, Oleksandr Andrushchenko wrote:
>>> On 27.09.21 13:26, Jan Beulich wrote:
>>>> On 27.09.2021 12:04, Oleksandr Andrushchenko wrote:
>>>>> On 27.09.21 13:00, Jan Beulich wrote:
>>>>>> On 27.09.2021 11:35, Oleksandr Andrushchenko wrote:
>>>>>>> On 27.09.21 12:19, Jan Beulich wrote:
>>>>>>>> On 27.09.2021 10:45, Oleksandr Andrushchenko wrote:
>>>>>>>>> On 27.09.21 10:45, Jan Beulich wrote:
>>>>>>>>>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>>>>>>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>>>>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>>>>>>>> @@ -328,6 +328,9 @@ static struct pci_dev *alloc_pdev(struct pc=
i_seg *pseg, u8 bus, u8 devfn)
>>>>>>>>>>>           *((u8*) &pdev->bus) =3D bus;
>>>>>>>>>>>           *((u8*) &pdev->devfn) =3D devfn;
>>>>>>>>>>>           pdev->domain =3D NULL;
>>>>>>>>>>> +#ifdef CONFIG_ARM
>>>>>>>>>>> +    pci_to_dev(pdev)->type =3D DEV_PCI;
>>>>>>>>>>> +#endif
>>>>>>>>>> I have to admit that I'm not happy about new CONFIG_<arch> condi=
tionals
>>>>>>>>>> here. I'd prefer to see this done by a new arch helper, unless t=
here are
>>>>>>>>>> obstacles I'm overlooking.
>>>>>>>>> Do you mean something like arch_pci_alloc_pdev(dev)?
>>>>>>>> I'd recommend against "alloc" in its name; "new" instead maybe?
>>>>>>> I am fine with arch_pci_new_pdev, but arch prefix points to the fac=
t that
>>>>>>> this is just an architecture specific part of the pdev allocation r=
ather than
>>>>>>> actual pdev allocation itself, so with this respect arch_pci_alloc_=
pdev seems
>>>>>>> more natural to me.
>>>>>> The bulk of the function is about populating the just allocated stru=
ct.
>>>>>> There's no arch-specific part of the allocation (so far, leaving asi=
de
>>>>>> MSI-X), you only want and arch-specific part of the initialization. =
I
>>>>>> would agree with "alloc" in the name if further allocation was to
>>>>>> happen there.
>>>>> Hm, then arch_pci_init_pdev sounds more reasonable
>>>> Fine with me.
>>> Do we want this to be void or returning an error code? If error code is=
 needed,
>>> then we would also need a roll-back function, e.g. arch_pci_free_pdev o=
r
>>> arch_pci_release_pdev or arch_pci_fini_pdev or something, so it can be =
used in
>>> case of error or in free_pdev function.
>> I'd start with void and make it return an error (and deal with necessary
>> cleanup) only once a need arises.
>=20
> Sounds reasonable. For x86 I think we can deal with:
>=20
> xen/include/xen/pci.h:
>=20
> #ifdef CONFIG_ARM
> void arch_pci_init_pdev(struct pci_dev *pdev);
> #else
> static inline void arch_pci_init_pdev(struct pci_dev *pdev)
> {
>  =C2=A0=C2=A0=C2=A0 return 0;
> }
> #endif

But that's still #ifdef-ary. We have asm/pci.h.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:41:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:41:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197816.351071 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8g5-0005eM-BH; Tue, 28 Sep 2021 08:41:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197816.351071; Tue, 28 Sep 2021 08:41:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8g5-0005eF-8G; Tue, 28 Sep 2021 08:41:29 +0000
Received: by outflank-mailman (input) for mailman id 197816;
 Tue, 28 Sep 2021 08:41:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mV8g4-0005e9-IJ
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:41:28 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dc305a6c-cbd2-44fa-aa61-5865c96f6667;
 Tue, 28 Sep 2021 08:41:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc305a6c-cbd2-44fa-aa61-5865c96f6667
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632818486;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=uhD1dDEr+NgpbOz1ALc8V4poGGrcepyylD1B5aZfytU=;
  b=agEyC1jtfqjjzy9kniYbilHRhcH+2vnxSlRsRA8FHLVYXltWiU7CBJgW
   PIMtpJukX7bNMgeMwduC4xq8otzTddJUtu3G6mNhhC0Ztq4HPO5xpOTUa
   PF2KDgNTH9XjmVDXXB+s3zWq0bL89bA2VbuWjkwX8u64tu9vopEBrnISv
   k=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: Pkx9B9BcgzkNmzknTCEOGIWQAQrI8nRjgMpxwumPRInNtYhAI99cg+nMMZ3NoVdkd26G68kXlq
 56NoarFJKAJFVr3yvzWPYrHTRxrIs4rBbTbbi91OuwJaHNodDqAZuvVjOzwydtpdDGbN02gpfm
 JDVk7+0xJ2zQshXbiWRj31BXev1XNxMLflk24oLDMrpIBS1GkpuFyPVFCrEH2phKkwQNdABLYy
 ClXxiJ4SR+Hy76DrrzQi7CZ0jD2nNR4KmANlqpKME77MnkDenD1e6XPJig0eQCZOEHLDtb8rwL
 wwUWAJoY5hFOTb5NUCfh2nch
X-SBRS: 5.1
X-MesageID: 53783735
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:1ZBAQKIgimdQVTzkFE+RU5IlxSXFcZb7ZxGr2PjKsXjdYENS0GYHz
 mRMUTvVa/jfYzamfIolPo+z/RtS6JXXnddmSFNlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5y7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB2Mk4lA4
 /MO5Ka3Cik5Y/PIyf8dcwFhRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFlgNgjK8/6epsSoHh6wRnSDOo8QICFSKLPjTNd9Gpr2ZAVQq2DD
 yYfQRpGUhuRRiFfAFwKU7EDwsC6wXb7MDIN/Tp5ooJoujOOnWSdyoPFMsfRe9GMbdVYmACfv
 G2u13/iHhgQOdibyDyE2nGhnOnCmWX8Qo16PKK83u5nhhuU3GN7IA0bUx63rOe0jma6WslDM
 AoE9yw2t68w+Ue3CN7nUHWQoWGGlg4RXcJKFO834x3LzbDbiy6bDGUZSj9KaPQ9qdQ7Azct0
 ze0c8jBXGI19ufPEDTEq+nS/Wja1TUpwXEqYCNdfCAjoIjZp8Idnk+RSod4EI6tt4igcd3v+
 AyioC87jrQVqMcE0aSn4FzK6w6RSoj1oh0dvFuOATP8hu9tTMv1P9X3tQizAeNodt7xc7WXg
 JQTdyFyBsggCouR3AiEXekABr2g4/vt3Nb02gU0R8hJG9hA4ReekWFsDNNWeBsB3iUsI2aBj
 KrvVeV5v8Q70JyCN/MfXm5JI552pZUM7Py8PhwuUvJAY4JqaCiM9zx0aEib0gjFyRZ3y/5ha
 c3CK5r1Vh727JiLKhLtHI/xNpdxmkgDKZ77H8inn3xLL5LEDJJqdVv1GATXNb1ohE91iA7U7
 8xeJ6O3J+Z3CoXDjt3s2ddLdzgidCFjbbiv8pA/XrPTc2JORTB6Y9eMkOxJRmCQt/kM/gs+1
 irmAREwJZuWrSCvFDhmnVg5M+6wAcYj8yhrVcHuVH7xs0UejU+UxP93X/MKkXMPq4SPFNZ4E
 KsIfduuGPNKRmiV8jgRd8Cl/odjaA6qlUSFOC/8OGozeJtpRgro/N74f1SwqHlSX3Tv7cZu8
 ae90g77QIYYQ1gwBsjhd//ynUi6umIQmbwuUhKQcMVTYkjl7KNjNzf10q0sO8gJJBianmma2
 g+aDA02v+7Ip4NpotDFibrd99WiEvdkH1ocFG7etO7kOS7f92ul4IlBTOfXImyNCDKqoP2vP
 LwHwev9PfsLmEdxn7B9S7s7n7gj49bPpqNBylg2Fnv8cFn2WKhrJWOL3JcTu/QVlKNZowa/R
 mmG5sJeZeeSIMrgHVMceFglY+CE2a1GkzXe961ockDz5Ssx97ubS0RCeRKLjXUFfrdyNYokx
 8YnudIXtFPj2kZ7bI7eg3AG7XmII1wBT74j58MTD4LcgwY2zk1PPM7HASjs7ZDTM9hBPyHG+
 NNPaHYuU1iE+nf/Tg==
IronPort-HdrOrdr: A9a23:9nYjK6HWcmhPo9GspLqFW5HXdLJyesId70hD6qkvc3Fom52j/f
 xGws5x6fatskdrZJkh8erwW5VoMkmsj6KdgLNhcYtKOTOLhILGFvAE0WKP+Vzd8mjFh5ZgPM
 RbAudD4b/LfD5HZK/BiWHWferIguP3iZxA7t2urUuFODsaD52ImD0JbzpzfHcXeCB2Qb4CUL
 aM7MtOoDStPV4NaN6gO3UDV+/f4/XWiZPPe3c9dl8awTjLqQntxK/xEhCe0BtbeShI260e/W
 /MlBG8zrm/ssu81gTX2wbontRrcZrau5h+7f63+40owwbX+0KVjUNaKvq/VQUO0aOSAZAR4Z
 /xSlkbTp1OAjjqDxyISFPWqnXdOXAVmjHfIBaj8AXeScCVfkNEN+NRwY1eaRfX8EwmoZV117
 9KxXuQs95NAQrHhzmV3amCa/hGrDv8nZMZq59as5Wfa/pqVJZB6YgEuE9FGpYJGyz3rIghDe
 l1FcnZoPJba0mTYXzVtnRmhIXEZAV/Ij6WBkwZ/sCF2Tlfm350i0Me2cwEh38FsJYwUYNN6e
 jIOrlh0LtOUsgVZ6RgA/ppe7r8NkXdBRbXdG6CK1XuE68Kf3rLtp7s+b0woPqnfZQZpaFC7q
 gpkGkowlLaXniee/Fm8Kc7giwlcV/NKQgFkPsulqSRkoeMN4bWDQ==
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="53783735"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Ian
 Jackson" <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: [XEN PATCH v4] xen: rework `checkpolicy` detection when using "randconfig"
Date: Tue, 28 Sep 2021 09:39:43 +0100
Message-ID: <20210928083944.780398-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

This will help prevent the CI loop from having build failures when
`checkpolicy` isn't available when doing "randconfig" jobs.

To prevent "randconfig" from selecting XSM_FLASK_POLICY when
`checkpolicy` isn't available, we will actually override the config
output with the use of KCONFIG_ALLCONFIG.

Doing this way still allow a user/developer to set XSM_FLASK_POLICY
even when "checkpolicy" isn't available. It also prevent the build
system from reset the config when "checkpolicy" isn't available
anymore. And XSM_FLASK_POLICY is still selected automatically when
`checkpolicy` is available.
But this also work well for "randconfig", as it will not select
XSM_FLASK_POLICY when "checkpolicy" is missing.

This patch allows to easily add more override which depends on the
environment.

Also, move the check out of Config.mk and into xen/ build system.
Nothing in tools/ is using that information as it's done by
./configure.

We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
via .gitignore.

Remove '= y' in Kconfig as it isn't needed, only a value "y" is true,
anything else is considered false.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
v4:
- keep XEN_ prefix for HAS_CHECKPOLICY
- rework .allconfig.tmp file generation, so it is easier to read.
- remove .allconfig.tmp on clean, .*.tmp files aren't all cleaned yet,
  maybe for another time.
- add information about file name choice and Kconfig change in patch
  description.

v3:
- use KCONFIG_ALLCONFIG
- don't override XSM_FLASK_POLICY value unless we do randconfig.
- no more changes to the current behavior of kconfig, only to
  randconfig.

v2 was "[XEN PATCH v2] xen: allow XSM_FLASK_POLICY only if checkpolicy binary is available"
---
 Config.mk          |  6 ------
 xen/Makefile       | 20 +++++++++++++++++---
 xen/common/Kconfig |  2 +-
 3 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/Config.mk b/Config.mk
index e85bf186547f..d5490e35d03d 100644
--- a/Config.mk
+++ b/Config.mk
@@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=y
 build_id_linker := --build-id=sha1
 endif
 
-ifndef XEN_HAS_CHECKPOLICY
-    CHECKPOLICY ?= checkpolicy
-    XEN_HAS_CHECKPOLICY := $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xen && echo y || echo n)
-    export XEN_HAS_CHECKPOLICY
-endif
-
 define buildmakevars2shellvars
     export PREFIX="$(prefix)";                                            \
     export XEN_SCRIPT_DIR="$(XEN_SCRIPT_DIR)";                            \
diff --git a/xen/Makefile b/xen/Makefile
index f47423dacd9a..7c2ffce0fc77 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?= $(shell hostname)
 PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
 export PYTHON		?= $(PYTHON_INTERPRETER)
 
+export CHECKPOLICY	?= checkpolicy
+
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
 
@@ -178,6 +180,8 @@ CFLAGS += $(CLANG_FLAGS)
 export CLANG_FLAGS
 endif
 
+export XEN_HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)
+
 export root-make-done := y
 endif # root-make-done
 
@@ -189,14 +193,24 @@ ifeq ($(config-build),y)
 # *config targets only - make sure prerequisites are updated, and descend
 # in tools/kconfig to make the *config target
 
+# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
+# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
+filechk_kconfig_allconfig = \
+    $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \
+    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
+    :
+
+.allconfig.tmp: FORCE
+	set -e; { $(call filechk_kconfig_allconfig); } > $@
+
 config: FORCE
 	$(MAKE) $(kconfig) $@
 
 # Config.mk tries to include .config file, don't try to remake it
 %/.config: ;
 
-%config: FORCE
-	$(MAKE) $(kconfig) $@
+%config: .allconfig.tmp FORCE
+	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=$< $@
 
 else # !config-build
 
@@ -368,7 +382,7 @@ _clean: delete-unfresh-files
 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \;
 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
 	rm -f asm-offsets.s include/asm-*/asm-offsets.h
-	rm -f .banner
+	rm -f .banner .allconfig.tmp
 
 .PHONY: _distclean
 _distclean: clean
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index db687b1785e7..eb6c2edb7bfe 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -251,7 +251,7 @@ config XSM_FLASK_AVC_STATS
 
 config XSM_FLASK_POLICY
 	bool "Compile Xen with a built-in FLASK security policy"
-	default y if "$(XEN_HAS_CHECKPOLICY)" = "y"
+	default y if "$(XEN_HAS_CHECKPOLICY)"
 	depends on XSM_FLASK
 	---help---
 	  This includes a default XSM policy in the hypervisor so that the
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 08:54:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 08:54:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197826.351081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8sQ-0007OP-IL; Tue, 28 Sep 2021 08:54:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197826.351081; Tue, 28 Sep 2021 08:54:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV8sQ-0007OI-FJ; Tue, 28 Sep 2021 08:54:14 +0000
Received: by outflank-mailman (input) for mailman id 197826;
 Tue, 28 Sep 2021 08:54:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mV8sP-0007OA-Ta
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 08:54:13 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 45b2ca14-2f09-4759-940e-c07ea932a28e;
 Tue, 28 Sep 2021 08:54:12 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18S8hrRK019874; 
 Tue, 28 Sep 2021 08:54:10 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bbyswr1dg-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 08:54:10 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3373.eurprd03.prod.outlook.com (2603:10a6:803:1b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 08:54:07 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 08:54:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45b2ca14-2f09-4759-940e-c07ea932a28e
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WJpZUAn3EewuJHqzwk5DjPkkiGUistcphTQrl5SvPgjIJhs3i2ePy3Le+bLkmhJmM7SAXRwHEkcJZYsBOujF0b/B0dbLZQ6lJ24SuFSeuot92chhlVY8rpipY9UDMagmBGFF1sBCC4tTRMGKRs6OOZ+muaZTQxKY0p4EmddLGWOX5xWAkHmM5nLM2G55hvFAEksERbl3sN0CjpapAJfdwaGB9XynV0s4ootLbMX2x9LOPAyVhqitRv4G5Ai38UgivIAsWPOPKGsH35Di+K/dQTkqdSRh4th4xTAoPovMOzd1x1r+6Ic+Fy6ZVTv1Ag062bXbl/OMlOMQIU3EFgIUvQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IIQevOMQJIRgCmbg2Gul4rFNu8ZCy7XbPNgMYMOOn7I=;
 b=Czq59T7tz+cnQiZjQwNBvOF2hZaz/GJFvlFHGaMHZBN8Cc1s655nXERS7l/tCZUVx2819sD5BWAEQAHLye4WIuWmEGzfYXHBoxae07aricYTAJKptFXJS2ykflWFlfgHMFoeUigwAeGGhPlGytblUCOm+j3ANis++bMOJKdgMr2lyzHNSjq5k+VS2LWkRr7HllqppR6Hd7zsZPr/lt3qHc+8DxFoNNIX3DzC5ZjEzIkdiH0nBrbKt/oyzaC6HiXyMd7crNYkbQ9lvly9FPwkuRx8nZ1LU8V5E0qiRMl1EzzGFICGAw2mhIq0RFlvRNXZMJTA6Blkb5wuBUWa/2a5cg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IIQevOMQJIRgCmbg2Gul4rFNu8ZCy7XbPNgMYMOOn7I=;
 b=F53t8ZrMFUlzvQ540yoXU4Px+WSEv5CqQE2Fyv3TH8F4KuKkQ3nX2YOaixtgjEr42NzPY5Ozv+TmEEmAh5lQ8DgUSsYMUPyInsqKJubQc4TPHjkko5gaYXVl6JldXsfkmAVIEnDKrJAovZ5/rxOXoKrRhuGvucGMNkb0SXaXlXaNvbpGrha3tLE9lFth7unWAF7+/2lA/ayc99TRa75ztsqg4vV47/ofxCr8VuoFXuaPXMo0EdBhx5fn3jmMRF98tBsRBY6+3ykusXPLvyqbxVC2yMifVZgvqA68tPNNWJgh0KOLi0V09cgKfmKdNeuVjeMOiChG3oBShLDwRKRikQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Topic: [PATCH v2 05/11] xen/arm: Mark device as PCI while creating one
Thread-Index: 
 AQHXsHoyHtx3VKSfZEKPQekyrs7GEau3hnIAgAAQrICAAAldgIAABLUAgAAG9wCAAAEGAIAABhwAgAFsAgCAAATGgIAAAMUAgAADA4CAAAP7AA==
Date: Tue, 28 Sep 2021 08:54:07 +0000
Message-ID: <9a9d9517-f868-69a6-f75f-751a3828a926@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-6-andr2000@gmail.com>
 <443f57db-7b0d-e14a-88bb-78a31c0271d6@suse.com>
 <55beed13-3a7f-7269-870c-ee0361e23a1c@epam.com>
 <7b093c2f-33cf-ac9d-9583-f0d8b2bb4f48@suse.com>
 <23ae58de-6bd0-b299-a7d9-e6433d1b0689@epam.com>
 <56d30f56-d9fb-e7ca-f3e1-6795973d648c@suse.com>
 <a119740d-02c2-315f-432e-67aec806ada6@epam.com>
 <6eefff6f-97ed-e7f5-37f2-96065bd1f27e@suse.com>
 <2e8f4316-002f-17d8-b9ec-9886c6bc28fa@epam.com>
 <31dc3429-b1f4-85ab-968d-c9054b77ac4f@suse.com>
 <13b9cc76-c7e0-be7e-61d3-fc42a41dcc58@epam.com>
 <8a3c020c-687c-1818-3a0f-5e8a553fe58e@suse.com>
In-Reply-To: <8a3c020c-687c-1818-3a0f-5e8a553fe58e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ffd81a5e-1e72-43e1-cee0-08d9825d87ed
x-ms-traffictypediagnostic: VI1PR0302MB3373:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB3373B6DA3ECA9F04F44E1B5CE7A89@VI1PR0302MB3373.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 M/G4b95sJ13PHHQwVv32DYRmGJJYfp52XJsQNOV1Fd6M9P0FdnNHxoyqriv/JIcKD6ZYEuKQSaR3tq2VCWIesFnmobSbLoCzFbr2h8zItvnFmia1am0OkLyazZWFqQlFdUyYUt10v7ooiYLZJnrCqO7PY9J1m0wuZ/392wG7YPoP2UmwY3zxYmNECD9s4g7iJ+QHv/900A9jpbxH9sDfYkZOm+vIVWnu36aohhqRsK1RalLC+wiIuG9Zj+XpnAI8PM2hOdCvq5DCULfnf5vBWwjtWyGNbq4t12SzRfyuFjeUHeCpQmHNIKoM71TEUaDJGVZA8RSEUULfcOlcYNEWA47fcWCIMMxhIieuBthbXQFiP/NSaro80UinpUqzQNqWIZIHaUp051vPOf5XBKPfyPiRMEQ02PC4R2MAqXn0Dlsdq2Kj6l2R6aZ80od2GLzq3QpTdo0ytpybfpZ0bPgWOGQRhWN3K8zMZBz4QM9PmmnYbFN1pw/lQgjg1QU1vYWHePx8oo77PuGbcBKBdOjLkPDK/LzslTzW1zHdLOz7p/pXLaqzL3LFn7upi/svpyAtuD0v5ihZVW9GQe9UqXOjn4d2GaYJgI9hFQLmsigo3jW8qf4x84X48iTBvHjNdZta48PLSDC2J/MPJ1iz+0vUIOqCdBcGyOT/jQbF1i1m5CNHb0RCaSFTDO4/erC8KvdsHPNks5bzhqmYCGSuF9Y6DUJFNSHdv24hiYlI3CbBBPUdpoy5qzV35B1TDpZEoZoB
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(66476007)(5660300002)(55236004)(53546011)(2616005)(6506007)(54906003)(6916009)(186003)(31696002)(2906002)(6486002)(66556008)(316002)(38100700002)(64756008)(66446008)(122000001)(76116006)(71200400001)(91956017)(508600001)(26005)(31686004)(86362001)(66946007)(107886003)(38070700005)(8676002)(6512007)(8936002)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?WFB1L2w5RnU3TEMzZW9kVjZCeDVNVmJtQjVJbWlRVW1HUThHNGxJSUN1QUo0?=
 =?utf-8?B?UURwS3hxRHN4dFl2dFUvYjFPQnZ2SmNuNFROZTJyakU1VmpPSStjLzhuckpn?=
 =?utf-8?B?RThXQmhTNVJZY0Mzd3lLUFN5bjFlaVRVY2lSNnJTS0U5R2h1TzFMa1p3cG54?=
 =?utf-8?B?TTRvS1Z5bWJ1WVVveml4VkViVXZYZGRFLzBacXZHMEtDZ1Z2aUhRSkRMdy9R?=
 =?utf-8?B?c3JyZ2V0SWVmR3N6bTlNc0xHQkxrSkU1TTVlaEdSVlY0SGdqNkV3c2xEUGZR?=
 =?utf-8?B?cXovMEVLS3BpWmh2QXF4ZzNTL2s4cEdzcmQvOVpSbjk1NCt0WjJVcTJtY1Z4?=
 =?utf-8?B?VUd3b2VjbDdUVUVyUEkzMjQ4QnpFSjBsd2tmWUxOZ2s1UzgxZktBY0FXMEpo?=
 =?utf-8?B?MGtMNkIxbDZCTjRqY0V5ZFpUS1BhVS9sdXhUemtlM0J3aWlCcnNMU2tuMkZo?=
 =?utf-8?B?aURJeUpzVDM3bE1jbmRhcWsvMTJyY05HMzVXMDAyYmlzZTdVNnVTc21pNFIx?=
 =?utf-8?B?bUNEenpKS2VBUk5MWG1Ec2R3N1hIMHVOQVlzTTR6OHgwVG9KV3cvemFmRnha?=
 =?utf-8?B?WUc3RTErNEVLUWhUZHJxclFaOThnRm5YY3R4UnB2eElyNUZEVlM5RVRnakR3?=
 =?utf-8?B?YW95ZWR6MHZPazF3eTg3Wmh5b0lrWmRKbkxuNEpSb2EwTE9xWEVlQU1wNnVl?=
 =?utf-8?B?VFl3SlJrazA2T3NFWWxrOVJDcTRhTnBvTkVHN1BxbzdMWFlKNTZ0bld0RjRD?=
 =?utf-8?B?ZGlnSnJYOUE5YldPUjVhRExjbHl1L2JtNHFmc0JuM2JpRXhNdjdBNUwvVnpJ?=
 =?utf-8?B?RzlkWGRteWwzZmRqdTVnY0dJYTVmWU53bGpKWlhIclNSWkcxQzNrUExBL1VG?=
 =?utf-8?B?ZnJLaFdOcFlJd2tTeEMzaG81N0lBY3RWV3JKdGN0SGU1Rk1iU3dBRkhrM25t?=
 =?utf-8?B?WHo1Kzh4aHV5bmVBWWhLTWNZMWgyZFd5YitaVzVwZ3d4ZHpMQmo4U1czQ2w2?=
 =?utf-8?B?ZkN6ODRhVDJJK3p1cjBUOXI4WjBVWkxHUlVKY0ppMWFxV2tub00zZFJnS3Vh?=
 =?utf-8?B?aE1wVXhhUGR3NENRTzNHbHBVMElDaW5rNHV0MUVOdk9MK29xQmpYa0xYMndC?=
 =?utf-8?B?VFkzRkdXS21EQ1RWdkg1NjJsRStYdCt6bFo2eHk4ZXhPK2xaYk1wWXI5eVZO?=
 =?utf-8?B?aFZneHN4Q0RwRS9jVnBDcXdhR2h1VE5xZVRJTjgwdGxJTDJ3TlVpN2tLeVU3?=
 =?utf-8?B?SVBRNHhUZmhUY3JuWVNQcWxsZmVEUkdxRDZTS0FFZlpiRVNuT0piZE1xclZ4?=
 =?utf-8?B?d2ZDeVU5TGRNOGJ5VG5TT3N0Z3c1L1ZvZGd4NnlxQnJDSUlQYkJ6ZmI3cW9F?=
 =?utf-8?B?MmNMTmJjSm96TzJpQ1I5bFUwSWNUL0tnSll4Z3lTVDlTVGljc1VVVW9aSlJR?=
 =?utf-8?B?eGVETFFtclYvTy83aVh0dHhPbUVxY0Mxdm0vK1Rrb2cxTHdOTURYVUxxck9G?=
 =?utf-8?B?NGhwbnNsQzdxQmtoTFlMcEhhdWFIUzB4cHM0SXkySEtxa2FmT3NmbzgrYzB1?=
 =?utf-8?B?Qk05MEJiUm16bUl1S0dhWGoxSjhLTGJONDBKWG9PajBkdy9nZUdRQVlFV2pZ?=
 =?utf-8?B?TVJOdTdBUnM4em9iaHR1eENjbzltQ1BFV2NwOWpidTJkZStYRzIzOXhnQmtx?=
 =?utf-8?B?SXVoMms4dnBQd1JzZ3JsWmowcUlCY2J4U2lqcUhWckRTb2VKUm9oR0t2QldW?=
 =?utf-8?B?MTNWcXhzZnYvMXEwTW1iNDZvb2FhaXhEUmRFeVkrOXVzN0QrdVUybE1rZlBB?=
 =?utf-8?B?K0FsUmlUUTZSUWFoNUd0QT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C52C4DC56FCF4845831D84BE518C2533@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ffd81a5e-1e72-43e1-cee0-08d9825d87ed
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 08:54:07.2459
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Yjgrud8uJvszmjTVCXPwx5JoZS/lSkrqPE/JQPvAOF5n7Dr28xofzxZNUZk6R+kHbXHTGCXE1eH1QNTLmClbRh/zHFj6ZVWEXTmsJovmeSByhxy/bJnY35C48x+N26ZQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3373
X-Proofpoint-ORIG-GUID: z1mrsMWg4SI4tIaQcOJ8S1fuCCu8EkjK
X-Proofpoint-GUID: z1mrsMWg4SI4tIaQcOJ8S1fuCCu8EkjK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_04,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 malwarescore=0
 phishscore=0 lowpriorityscore=0 suspectscore=0 impostorscore=0 bulkscore=0
 mlxlogscore=999 mlxscore=0 adultscore=0 priorityscore=1501 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280052

DQpPbiAyOC4wOS4yMSAxMTozOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI4LjA5LjIwMjEg
MTA6MjksIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjguMDkuMjEgMTE6
MjYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI4LjA5LjIwMjEgMTA6MDksIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyNy4wOS4yMSAxMzoyNiwgSmFuIEJldWxp
Y2ggd3JvdGU6DQo+Pj4+PiBPbiAyNy4wOS4yMDIxIDEyOjA0LCBPbGVrc2FuZHIgQW5kcnVzaGNo
ZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAyNy4wOS4yMSAxMzowMCwgSmFuIEJldWxpY2ggd3JvdGU6
DQo+Pj4+Pj4+IE9uIDI3LjA5LjIwMjEgMTE6MzUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdy
b3RlOg0KPj4+Pj4+Pj4gT24gMjcuMDkuMjEgMTI6MTksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+
Pj4+Pj4+IE9uIDI3LjA5LjIwMjEgMTA6NDUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4+Pj4+PiBPbiAyNy4wOS4yMSAxMDo0NSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4+
Pj4+Pj4+PiBPbiAyMy4wOS4yMDIxIDE0OjU0LCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90
ZToNCj4+Pj4+Pj4+Pj4+PiAtLS0gYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4+
Pj4+Pj4+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+Pj4+Pj4+
Pj4gQEAgLTMyOCw2ICszMjgsOSBAQCBzdGF0aWMgc3RydWN0IHBjaV9kZXYgKmFsbG9jX3BkZXYo
c3RydWN0IHBjaV9zZWcgKnBzZWcsIHU4IGJ1cywgdTggZGV2Zm4pDQo+Pj4+Pj4+Pj4+Pj4gICAg
ICAgICAgICAqKCh1OCopICZwZGV2LT5idXMpID0gYnVzOw0KPj4+Pj4+Pj4+Pj4+ICAgICAgICAg
ICAgKigodTgqKSAmcGRldi0+ZGV2Zm4pID0gZGV2Zm47DQo+Pj4+Pj4+Pj4+Pj4gICAgICAgICAg
ICBwZGV2LT5kb21haW4gPSBOVUxMOw0KPj4+Pj4+Pj4+Pj4+ICsjaWZkZWYgQ09ORklHX0FSTQ0K
Pj4+Pj4+Pj4+Pj4+ICsgICAgcGNpX3RvX2RldihwZGV2KS0+dHlwZSA9IERFVl9QQ0k7DQo+Pj4+
Pj4+Pj4+Pj4gKyNlbmRpZg0KPj4+Pj4+Pj4+Pj4gSSBoYXZlIHRvIGFkbWl0IHRoYXQgSSdtIG5v
dCBoYXBweSBhYm91dCBuZXcgQ09ORklHXzxhcmNoPiBjb25kaXRpb25hbHMNCj4+Pj4+Pj4+Pj4+
IGhlcmUuIEknZCBwcmVmZXIgdG8gc2VlIHRoaXMgZG9uZSBieSBhIG5ldyBhcmNoIGhlbHBlciwg
dW5sZXNzIHRoZXJlIGFyZQ0KPj4+Pj4+Pj4+Pj4gb2JzdGFjbGVzIEknbSBvdmVybG9va2luZy4N
Cj4+Pj4+Pj4+Pj4gRG8geW91IG1lYW4gc29tZXRoaW5nIGxpa2UgYXJjaF9wY2lfYWxsb2NfcGRl
dihkZXYpPw0KPj4+Pj4+Pj4+IEknZCByZWNvbW1lbmQgYWdhaW5zdCAiYWxsb2MiIGluIGl0cyBu
YW1lOyAibmV3IiBpbnN0ZWFkIG1heWJlPw0KPj4+Pj4+Pj4gSSBhbSBmaW5lIHdpdGggYXJjaF9w
Y2lfbmV3X3BkZXYsIGJ1dCBhcmNoIHByZWZpeCBwb2ludHMgdG8gdGhlIGZhY3QgdGhhdA0KPj4+
Pj4+Pj4gdGhpcyBpcyBqdXN0IGFuIGFyY2hpdGVjdHVyZSBzcGVjaWZpYyBwYXJ0IG9mIHRoZSBw
ZGV2IGFsbG9jYXRpb24gcmF0aGVyIHRoYW4NCj4+Pj4+Pj4+IGFjdHVhbCBwZGV2IGFsbG9jYXRp
b24gaXRzZWxmLCBzbyB3aXRoIHRoaXMgcmVzcGVjdCBhcmNoX3BjaV9hbGxvY19wZGV2IHNlZW1z
DQo+Pj4+Pj4+PiBtb3JlIG5hdHVyYWwgdG8gbWUuDQo+Pj4+Pj4+IFRoZSBidWxrIG9mIHRoZSBm
dW5jdGlvbiBpcyBhYm91dCBwb3B1bGF0aW5nIHRoZSBqdXN0IGFsbG9jYXRlZCBzdHJ1Y3QuDQo+
Pj4+Pj4+IFRoZXJlJ3Mgbm8gYXJjaC1zcGVjaWZpYyBwYXJ0IG9mIHRoZSBhbGxvY2F0aW9uIChz
byBmYXIsIGxlYXZpbmcgYXNpZGUNCj4+Pj4+Pj4gTVNJLVgpLCB5b3Ugb25seSB3YW50IGFuZCBh
cmNoLXNwZWNpZmljIHBhcnQgb2YgdGhlIGluaXRpYWxpemF0aW9uLiBJDQo+Pj4+Pj4+IHdvdWxk
IGFncmVlIHdpdGggImFsbG9jIiBpbiB0aGUgbmFtZSBpZiBmdXJ0aGVyIGFsbG9jYXRpb24gd2Fz
IHRvDQo+Pj4+Pj4+IGhhcHBlbiB0aGVyZS4NCj4+Pj4+PiBIbSwgdGhlbiBhcmNoX3BjaV9pbml0
X3BkZXYgc291bmRzIG1vcmUgcmVhc29uYWJsZQ0KPj4+Pj4gRmluZSB3aXRoIG1lLg0KPj4+PiBE
byB3ZSB3YW50IHRoaXMgdG8gYmUgdm9pZCBvciByZXR1cm5pbmcgYW4gZXJyb3IgY29kZT8gSWYg
ZXJyb3IgY29kZSBpcyBuZWVkZWQsDQo+Pj4+IHRoZW4gd2Ugd291bGQgYWxzbyBuZWVkIGEgcm9s
bC1iYWNrIGZ1bmN0aW9uLCBlLmcuIGFyY2hfcGNpX2ZyZWVfcGRldiBvcg0KPj4+PiBhcmNoX3Bj
aV9yZWxlYXNlX3BkZXYgb3IgYXJjaF9wY2lfZmluaV9wZGV2IG9yIHNvbWV0aGluZywgc28gaXQg
Y2FuIGJlIHVzZWQgaW4NCj4+Pj4gY2FzZSBvZiBlcnJvciBvciBpbiBmcmVlX3BkZXYgZnVuY3Rp
b24uDQo+Pj4gSSdkIHN0YXJ0IHdpdGggdm9pZCBhbmQgbWFrZSBpdCByZXR1cm4gYW4gZXJyb3Ig
KGFuZCBkZWFsIHdpdGggbmVjZXNzYXJ5DQo+Pj4gY2xlYW51cCkgb25seSBvbmNlIGEgbmVlZCBh
cmlzZXMuDQo+PiBTb3VuZHMgcmVhc29uYWJsZS4gRm9yIHg4NiBJIHRoaW5rIHdlIGNhbiBkZWFs
IHdpdGg6DQo+Pg0KPj4geGVuL2luY2x1ZGUveGVuL3BjaS5oOg0KPj4NCj4+ICNpZmRlZiBDT05G
SUdfQVJNDQo+PiB2b2lkIGFyY2hfcGNpX2luaXRfcGRldihzdHJ1Y3QgcGNpX2RldiAqcGRldik7
DQo+PiAjZWxzZQ0KPj4gc3RhdGljIGlubGluZSB2b2lkIGFyY2hfcGNpX2luaXRfcGRldihzdHJ1
Y3QgcGNpX2RldiAqcGRldikNCj4+IHsNCj4+ICAgwqDCoMKgIHJldHVybiAwOw0KPj4gfQ0KPj4g
I2VuZGlmDQo+IEJ1dCB0aGF0J3Mgc3RpbGwgI2lmZGVmLWFyeS4gV2UgaGF2ZSBhc20vcGNpLmgu
DQpTdXJlLCB3aWxsIGRlZmluZSBpdCB0aGVyZQ0KPg0KPiBKYW4NCj4NCj4NClRoYW5rIHlvdSwN
Cg0KT2xla3NhbmRyDQo=


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 09:13:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 09:13:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197836.351092 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9Ab-0001bf-8a; Tue, 28 Sep 2021 09:13:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197836.351092; Tue, 28 Sep 2021 09:13:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9Ab-0001bY-5G; Tue, 28 Sep 2021 09:13:01 +0000
Received: by outflank-mailman (input) for mailman id 197836;
 Tue, 28 Sep 2021 09:13:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV9Aa-0001bO-6R; Tue, 28 Sep 2021 09:13:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV9AZ-0004D4-Oc; Tue, 28 Sep 2021 09:12:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mV9AZ-0003me-FF; Tue, 28 Sep 2021 09:12:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mV9AZ-0006bR-Eh; Tue, 28 Sep 2021 09:12:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=i2st3N1qr696SwEVcbHYKf5legmvNPGvFrG1y2xcsv0=; b=HhsR7Ta5tho6isgvliV4tbfBHR
	IEvSfyLWjd0W0fvfwjb0KcbGWC5gk24pipZxiuU/4mCbsP5GTdbxb98q1+3M6NQLA5TN2RD/Ucvir
	J7BNHdPdRjMj9O31e+LLlYYYVyLRzlJwm3jpYRpDNULVK22TKjgbqQzLxoptpKhtwZL4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165230-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165230: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1c3ed9c908732d19660fbe83580674d585464d4c
X-Osstest-Versions-That:
    xen=2e46d73b4c7562f7b104e9e10fe302316af13959
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 09:12:59 +0000

flight 165230 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165230/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1c3ed9c908732d19660fbe83580674d585464d4c
baseline version:
 xen                  2e46d73b4c7562f7b104e9e10fe302316af13959

Last test of basis   165223  2021-09-27 15:01:42 Z    0 days
Testing same since   165230  2021-09-28 05:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   2e46d73b4c..1c3ed9c908  1c3ed9c908732d19660fbe83580674d585464d4c -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 09:15:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 09:15:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197843.351106 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D0-0002Eu-M8; Tue, 28 Sep 2021 09:15:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197843.351106; Tue, 28 Sep 2021 09:15:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D0-0002En-Iv; Tue, 28 Sep 2021 09:15:30 +0000
Received: by outflank-mailman (input) for mailman id 197843;
 Tue, 28 Sep 2021 09:15:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV9Cy-0002Eh-JV
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 09:15:28 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9e7a1e42-203c-11ec-bc75-12813bfff9fa;
 Tue, 28 Sep 2021 09:15:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id CC199222BB;
 Tue, 28 Sep 2021 09:15:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8F18D13A4A;
 Tue, 28 Sep 2021 09:15:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id vKTnHy7dUmESHQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 09:15:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e7a1e42-203c-11ec-bc75-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632820526; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=K1gJm0tU9vRsReq+9DJ5wsBzXrvm3Rb7hqXFabtbdQo=;
	b=rsnuLQwJQb+B1cV/0ASiamB8vIJdrKEFKLMo80wPaJ6kX7GGS/Qw9+D6pYOM+oKtj4PjS3
	FH10YGaHS+KEILv2dDJ2x6P/HYh+Rce1/RS8PIxEEjWbSc4w4A+/JoqBuvr8RDS90u/n8y
	l6lLUfTzJBDUpcpH5oF1V+tI2OVY7SQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 0/2] tools/xenstore: set resource limits of xenstored
Date: Tue, 28 Sep 2021 11:15:15 +0200
Message-Id: <20210928091517.9761-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Set some limits for xenstored in order to avoid it being killed by
OOM killer, or to run out of file descriptors.

Changes in V5:
- respect /proc/sys/fs/nr_open (Ian Jackson)

Changes in V4:
- add comments
- switch to configure open file descriptors directly

Changes in V3:
- make oom score configurable

Changes in V2:
- split into 2 patches
- set limits from start script

Juergen Gross (2):
  tools/xenstore: set oom score for xenstore daemon on Linux
  tools/xenstore: set open file descriptor limit for xenstored

 .../Linux/init.d/sysconfig.xencommons.in      | 22 ++++++++++++++++
 tools/hotplug/Linux/launch-xenstore.in        | 26 +++++++++++++++++++
 2 files changed, 48 insertions(+)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 09:15:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 09:15:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197844.351118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D4-0002WX-Ue; Tue, 28 Sep 2021 09:15:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197844.351118; Tue, 28 Sep 2021 09:15:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D4-0002WQ-R7; Tue, 28 Sep 2021 09:15:34 +0000
Received: by outflank-mailman (input) for mailman id 197844;
 Tue, 28 Sep 2021 09:15:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV9D3-0002Eh-I4
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 09:15:33 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9e8c53aa-203c-11ec-bc75-12813bfff9fa;
 Tue, 28 Sep 2021 09:15:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id EA7801FD74;
 Tue, 28 Sep 2021 09:15:26 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C64DA13BAD;
 Tue, 28 Sep 2021 09:15:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id yAH9Li7dUmESHQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 09:15:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e8c53aa-203c-11ec-bc75-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632820526; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Q5L7RANFuY0lTjcILncAay9zDfSZ+zR/cGF7re2yY04=;
	b=C/YCNWfRaUUc5abiQF2BRob/tNIIM9tcO6Qj/ARQt7w8L6nmLhsuQitPnKbQUpbEd8xLaf
	LR6tL5sOHKvf3VzI4YdgZWTGLtUmBPzjWrHMfG9Px5jXguyeX1U1A6oqDkZz2ng2a97g7I
	AwEj5b6h2gTaBEXCLhIJOeUmGdlRWY4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 1/2] tools/xenstore: set oom score for xenstore daemon on Linux
Date: Tue, 28 Sep 2021 11:15:16 +0200
Message-Id: <20210928091517.9761-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210928091517.9761-1-jgross@suse.com>
References: <20210928091517.9761-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Xenstored is absolutely mandatory for a Xen host and it can't be
restarted, so being killed by OOM-killer in case of memory shortage is
to be avoided.

Set /proc/$pid/oom_score_adj (if available) per default to -500 (this
translates to 50% of dom0 memory size) in order to allow xenstored to
use large amounts of memory without being killed.

The percentage of dom0 memory above which the oom killer is allowed to
kill xenstored can be set via XENSTORED_OOM_MEM_THRESHOLD in
xencommons.

Make sure the pid file isn't a left-over from a previous run delete it
before starting xenstored.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Ian Jackson <iwj@xenproject.org>
---
V2:
- set oom score from launch script (Julien Grall)
- split off open file descriptor limit setting (Julien Grall)
V3:
- make oom killer threshold configurable (Julien Grall)
V4:
- extend comment (Ian Jackson)
---
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 9 +++++++++
 tools/hotplug/Linux/launch-xenstore.in             | 6 ++++++
 2 files changed, 15 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 00cf7f91d4..b83101ab7e 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -48,6 +48,15 @@ XENSTORED_ARGS=
 # Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
+## Type: integer
+## Default: 50
+#
+# Percentage of dom0 memory size the xenstore daemon can use before the
+# OOM killer is allowed to kill it.
+# The specified value is multiplied by -10 and echoed to
+# /proc/PID/oom_score_adj.
+#XENSTORED_OOM_MEM_THRESHOLD=50
+
 ## Type: string
 ## Default: @LIBEXEC@/boot/xenstore-stubdom.gz
 #
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index 019f9d6f4d..1747c96065 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -59,11 +59,17 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 		echo "No xenstored found"
 		exit 1
 	}
+	[ -z "$XENSTORED_OOM_MEM_THRESHOLD" ] || XENSTORED_OOM_MEM_THRESHOLD=50
+	XS_OOM_SCORE=-$(($XENSTORED_OOM_MEM_THRESHOLD * 10))
+
+	rm -f @XEN_RUN_DIR@/xenstored.pid
 
 	echo -n Starting $XENSTORED...
 	$XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS
 
 	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1
+	XS_PID=`cat @XEN_RUN_DIR@/xenstored.pid`
+	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
 
 	exit 0
 }
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 09:15:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 09:15:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197846.351129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D9-0002rw-Ag; Tue, 28 Sep 2021 09:15:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197846.351129; Tue, 28 Sep 2021 09:15:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9D9-0002rn-6u; Tue, 28 Sep 2021 09:15:39 +0000
Received: by outflank-mailman (input) for mailman id 197846;
 Tue, 28 Sep 2021 09:15:38 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mV9D8-0002Eh-IG
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 09:15:38 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9e8d5ce7-203c-11ec-bc75-12813bfff9fa;
 Tue, 28 Sep 2021 09:15:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1EFE1222BC;
 Tue, 28 Sep 2021 09:15:27 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id F084D13A4A;
 Tue, 28 Sep 2021 09:15:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 6EFqOS7dUmESHQAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 09:15:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e8d5ce7-203c-11ec-bc75-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632820527; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Vr+oyI16qKEiXKH2RoDyqCvitwKqYE25WQ2BT5cn+24=;
	b=gwkkbf0w7bfXGPDYcw2ZFEL/SWT4CxVDD1OgbbWDj0LLfo8Ww0aQAJNS483jeCqIiwGXtw
	uAkN98eN+zBBWbBjmktC2/SXiyHBu+Oh4eQhPXq64b60L0ONyGbQImr7a22/PUyAVyMGus
	P+zvAYaj1TxMK657tY5l71C1altbz9k=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for xenstored
Date: Tue, 28 Sep 2021 11:15:17 +0200
Message-Id: <20210928091517.9761-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210928091517.9761-1-jgross@suse.com>
References: <20210928091517.9761-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add a configuration item for the maximum number of open file
descriptors xenstored should be allowed to have.

The default should be "unlimited" in order not to restrict xenstored
in the number of domains it can support, but unfortunately the kernel
is normally limiting the maximum value via /proc/sys/fs/nr_open [1],
[2]. So check that file to exist and if it does, limit the maximum
value to the one specified by /proc/sys/fs/nr_open.

As an aid for the admin configuring the value add a comment specifying
the common needs of xenstored for the different domain types.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=60fd760fb9ff7034360bab7137c917c0330628c2
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c2d64fb6cae9aae480f6a46cfe79f8d7d48b59f

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- set ulimit form launch script (Julien Grall)
- split off from original patch (Julien Grall)
V4:
- switch to directly configuring the limit of file descriptors instead
  of domains (Ian Jackson)
V5:
- use /proc/sys/fs/nr_open (Ian Jackson)
---
 .../Linux/init.d/sysconfig.xencommons.in      | 13 ++++++++++++
 tools/hotplug/Linux/launch-xenstore.in        | 20 +++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index b83101ab7e..433e4849af 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -32,6 +32,19 @@
 # Changing this requires a reboot to take effect.
 #XENSTORED=@XENSTORED@
 
+## Type: string
+## Default: unlimited
+#
+# Select maximum number of file descriptors xenstored is allowed to have
+# opened at one time.
+# For each HVM domain xenstored might need up to 5 open file descriptors,
+# PVH and PV domains will require up to 3 open file descriptors. Additionally
+# 20-30 file descriptors will be opened for internal uses.
+# The specified value (including "unlimited") will be capped by the contents
+# of /proc/sys/fs/nr_open if existing.
+# Only evaluated if XENSTORETYPE is "daemon".
+#XENSTORED_MAX_OPEN_FDS=unlimited
+
 ## Type: string
 ## Default: ""
 #
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index 1747c96065..7a0334d880 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -54,6 +54,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 
 [ "$XENSTORETYPE" = "daemon" ] && {
 	[ -z "$XENSTORED_TRACE" ] || XENSTORED_ARGS="$XENSTORED_ARGS -T @XEN_LOG_DIR@/xenstored-trace.log"
+	[ -z "$XENSTORED_MAX_OPEN_FDS" ] && XENSTORED_MAX_OPEN_FDS=unlimited
 	[ -z "$XENSTORED" ] && XENSTORED=@XENSTORED@
 	[ -x "$XENSTORED" ] || {
 		echo "No xenstored found"
@@ -62,6 +63,24 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORED_OOM_MEM_THRESHOLD" ] || XENSTORED_OOM_MEM_THRESHOLD=50
 	XS_OOM_SCORE=-$(($XENSTORED_OOM_MEM_THRESHOLD * 10))
 
+	[ "$XENSTORED_MAX_OPEN_FDS" = "unlimited" ] || {
+		[ -z "${XENSTORED_MAX_OPEN_FDS//[0-9]}" ] &&
+		[ -n "$XENSTORED_MAX_OPEN_FDS" ] || {
+			echo "XENSTORED_MAX_OPEN_FDS=$XENSTORED_MAX_OPEN_FDS invalid"
+			echo "Setting to default \"unlimited\"."
+			XENSTORED_MAX_OPEN_FDS=unlimited
+		}
+	}
+	[ -r /proc/sys/fs/nr_open ] && {
+		MAX_FDS=`cat /proc/sys/fs/nr_open`
+		[ "$XENSTORED_MAX_OPEN_FDS" = "unlimited" ] && XENSTORED_MAX_OPEN_FDS=$MAX_FDS
+		[ $XENSTORED_MAX_OPEN_FDS -gt $MAX_FDS ] && {
+			echo "XENSTORED_MAX_OPEN_FDS exceeds system limit."
+			echo "Setting to \"$MAX_FDS\"."
+			XENSTORED_MAX_OPEN_FDS=$MAX_FDS
+		}
+	}
+
 	rm -f @XEN_RUN_DIR@/xenstored.pid
 
 	echo -n Starting $XENSTORED...
@@ -70,6 +89,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1
 	XS_PID=`cat @XEN_RUN_DIR@/xenstored.pid`
 	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
+	prlimit --pid $XS_PID --nofile=$XENSTORED_MAX_OPEN_FDS
 
 	exit 0
 }
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 09:23:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 09:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197864.351140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9L4-00056e-8i; Tue, 28 Sep 2021 09:23:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197864.351140; Tue, 28 Sep 2021 09:23:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mV9L4-00056X-4N; Tue, 28 Sep 2021 09:23:50 +0000
Received: by outflank-mailman (input) for mailman id 197864;
 Tue, 28 Sep 2021 09:23:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vk4k=OS=gmail.com=ltykernel@srs-us1.protection.inumbo.net>)
 id 1mV9L2-00056R-Tx
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 09:23:48 +0000
Received: from mail-pg1-x52e.google.com (unknown [2607:f8b0:4864:20::52e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 05fbcbc0-5bb8-4a1c-bb21-31810901836b;
 Tue, 28 Sep 2021 09:23:48 +0000 (UTC)
Received: by mail-pg1-x52e.google.com with SMTP id h3so20519398pgb.7
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 02:23:47 -0700 (PDT)
Received: from ?IPv6:2404:f801:0:5:8000::50b? ([2404:f801:9000:1a:efea::50b])
 by smtp.gmail.com with ESMTPSA id
 mv6sm1943328pjb.16.2021.09.28.02.23.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 02:23:46 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05fbcbc0-5bb8-4a1c-bb21-31810901836b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-language:content-transfer-encoding;
        bh=bNCbpKkDoRa/LEy6AE/asdoXmkV09NnhXW1RxZk9Sdg=;
        b=d7EQaIO2BrzYhwnuLWNgscYrGYoCF1llQTdP7VsnWfHPgbO7X8mwFpRL1rP0sxjRol
         /+BkBsfPxs/6kB2aGGiarfL3uCD04VsP3Qvt0ysEyyh6Gt2+3I1FWrhhioqM5RZZcaaQ
         08hitTmCaLcQBEott0EDr+wFaOq4jqZ1NJtcmmH1pPW9zZ2ctPp9TAkDzkUATz66B0OO
         APGXI2+QPUkq1LfuP1SdBGqjMyXvkbKRa4yN0N3eDg3lv3bc7gcKGRD1XvdkWanbXoPX
         wdfl6sy2mQQiSep/qjiZSaFK4BGckINkhPOtEFqjaiLg960K/wVSLQzxtSjHYSHxv/ie
         tbaw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-language
         :content-transfer-encoding;
        bh=bNCbpKkDoRa/LEy6AE/asdoXmkV09NnhXW1RxZk9Sdg=;
        b=J3pg20VjD3jcpnl+dMiZqMv+Ib/327FqqbXZjWgVyRNfsRogXYeSqSkbpqODlh9zsw
         rq/UsHDyc8MxLIOegFroX6itSQyOafDl7BpsNz0wJHJ7hxJMzMH4KftqHrPcIieMWDl1
         UOUWt5OA2Dj9s6lQHrnw7sSslKS2xvs12TP/bsE0stLM0QJHX7paICgLf6qcWsWja+uo
         NOgRC8WVlyvBpMpmYMj0bJ7iBorQPRveYNC+sAwXGYvVlDUSGx4uI6qfcSXFvyGKX6yM
         qR11VHSTAWA+Qf0VpZWsw+tZMSyikdSmRsUmsE9m8IsciwUwo//fRLB9AMCyO2upbQuZ
         hUFg==
X-Gm-Message-State: AOAM533rv5VgZ5yeSCbW0YX1zFun/CrT+MG7kDh3DB38OD9kdIa7j6ec
	l2Q7iBNjUopYp8cPau0SUGM=
X-Google-Smtp-Source: ABdhPJwGWB7Tw6GaWECCuWCSL7zfwRyp/J9yMIqD2mhp7FZqF/2KIG0FGrBjcU+pMCnt5vLHDbzqyg==
X-Received: by 2002:aa7:9f03:0:b0:447:dd09:6dda with SMTP id g3-20020aa79f03000000b00447dd096ddamr4300020pfr.36.1632821027229;
        Tue, 28 Sep 2021 02:23:47 -0700 (PDT)
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for netvsc
 driver
To: Christoph Hellwig <hch@lst.de>
Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
 "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
 "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
 vkuznets <vkuznets@redhat.com>,
 "parri.andrea@gmail.com" <parri.andrea@gmail.com>,
 "dave.hansen@intel.com" <dave.hansen@intel.com>,
 Michael Kelley <mikelley@microsoft.com>, KY Srinivasan <kys@microsoft.com>,
 Haiyang Zhang <haiyangz@microsoft.com>,
 Stephen Hemminger <sthemmin@microsoft.com>,
 "wei.liu@kernel.org" <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>,
 "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
 "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "luto@kernel.org" <luto@kernel.org>,
 "peterz@infradead.org" <peterz@infradead.org>,
 "konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
 "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
 "jgross@suse.com" <jgross@suse.com>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 "joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
 "davem@davemloft.net" <davem@davemloft.net>,
 "kuba@kernel.org" <kuba@kernel.org>, "jejb@linux.ibm.com"
 <jejb@linux.ibm.com>, "martin.petersen@oracle.com"
 <martin.petersen@oracle.com>,
 "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
 "arnd@arndb.de" <arnd@arndb.de>,
 "m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
 "robin.murphy@arm.com" <robin.murphy@arm.com>,
 "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
 Tianyu Lan <Tianyu.Lan@microsoft.com>,
 "thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
 "pgonda@google.com" <pgonda@google.com>,
 "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
 "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
 "rppt@kernel.org" <rppt@kernel.org>,
 "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
 "aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
 "saravanand@fb.com" <saravanand@fb.com>,
 "krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "tj@kernel.org" <tj@kernel.org>, "rientjes@google.com" <rientjes@google.com>
References: <20210914133916.1440931-1-ltykernel@gmail.com>
 <20210914133916.1440931-13-ltykernel@gmail.com>
 <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com>
 <43e22b84-7273-4099-42ea-54b06f398650@gmail.com>
 <e379a60b-4d74-9167-983f-f70c96bb279e@gmail.com>
 <20210928053911.GA29208@lst.de>
From: Tianyu Lan <ltykernel@gmail.com>
Message-ID: <de18b708-7252-642b-c60f-59e12ac27421@gmail.com>
Date: Tue, 28 Sep 2021 17:23:31 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210928053911.GA29208@lst.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

On 9/28/2021 1:39 PM, Christoph Hellwig wrote:
> On Mon, Sep 27, 2021 at 10:26:43PM +0800, Tianyu Lan wrote:
>> Hi Christoph:
>>      Gentile ping. The swiotlb and shared memory mapping changes in this
>> patchset needs your reivew. Could you have a look? >
> I'm a little too busy for a review of such a huge patchset right now.
> That being said here are my comments from a very quick review:
Hi Christoph:
       Thanks for your comments. Most patches in the series are Hyper-V
change. I will split patchset and make it easy to review.


> 
>   - the bare memremap usage in swiotlb looks strange and I'd
>     definitively expect a well documented wrapper.

OK. Should the wrapper in the DMA code? How about dma_map_decrypted() 
introduced in the V4?
https://lkml.org/lkml/2021/8/27/605

>   - given that we can now hand out swiotlb memory for coherent mappings
>     we need to carefully audit what happens when this memremaped
>     memory gets mmaped or used through dma_get_sgtable

OK. I check that.

>   - the netscv changes I'm not happy with at all.  A large part of it
>     is that the driver already has a bad structure, but this series
>     is making it significantly worse.  We'll need to find a way
>     to use the proper dma mapping abstractions here.  One option
>     if you want to stick to the double vmapped buffer would be something
>     like using dma_alloc_noncontigous plus a variant of
>     dma_vmap_noncontiguous that takes the shared_gpa_boundary into
>     account.
> 

OK. I will do that.




From xen-devel-bounces@lists.xenproject.org Tue Sep 28 11:16:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 11:16:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197909.351150 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVB5p-0008Sg-Ay; Tue, 28 Sep 2021 11:16:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197909.351150; Tue, 28 Sep 2021 11:16:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVB5p-0008SZ-7y; Tue, 28 Sep 2021 11:16:13 +0000
Received: by outflank-mailman (input) for mailman id 197909;
 Tue, 28 Sep 2021 11:16:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s6b1=OS=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVB5n-0008ST-HU
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:16:11 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id dd2e4523-fac9-46a5-9eaa-cd0e35859312;
 Tue, 28 Sep 2021 11:16:09 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1E0FE6D;
 Tue, 28 Sep 2021 04:16:09 -0700 (PDT)
Received: from [10.57.20.114] (unknown [10.57.20.114])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CB4563F7B4;
 Tue, 28 Sep 2021 04:16:06 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd2e4523-fac9-46a5-9eaa-cd0e35859312
Subject: Re: [PATCH v2 01/11] vpci: Make vpci registers removal a dedicated
 function
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-2-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <81b67339-08a5-402f-a60f-8db89a4e59da@arm.com>
Date: Tue, 28 Sep 2021 13:15:55 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-2-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> This is in preparation for dynamic assignment of the vpci register
> handlers depending on the domain: hwdom or guest.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - constify struct pci_dev where possible
> ---
>  xen/drivers/vpci/vpci.c | 7 ++++++-
>  xen/include/xen/vpci.h  | 2 ++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
Reviewed-by: Michal Orzel <michal.orzel@arm.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 11:28:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 11:28:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197915.351161 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBHl-0001nz-Eq; Tue, 28 Sep 2021 11:28:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197915.351161; Tue, 28 Sep 2021 11:28:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBHl-0001ns-Bs; Tue, 28 Sep 2021 11:28:33 +0000
Received: by outflank-mailman (input) for mailman id 197915;
 Tue, 28 Sep 2021 11:28:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVBHk-0001mw-MN; Tue, 28 Sep 2021 11:28:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVBHk-0006WI-HT; Tue, 28 Sep 2021 11:28:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVBHk-0002tu-5g; Tue, 28 Sep 2021 11:28:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVBHk-00009B-5G; Tue, 28 Sep 2021 11:28:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NfTEdPr73BXc74rw2Z8R8tUnzbtMZxZBU6WfifAg8BY=; b=rbupOhaq0LOQ7TE9Nd2FrIMI8Y
	NgbEXbQFHSnTwEX7bG9OGdGFgweBq7on/L8t7GGTLpdVVAjFc1AIzmgOJO7na4jkcXko5oH0GwsYo
	G6p6CeUJRL4+ErFEeOv0ypecuTFZelKlb68qHuMHOXcmObOofnWM8y+G+CXP73+ncmZQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165228-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165228: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=65499b4f090bf8b5c0699f27e755921f774fb1ee
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 11:28:32 +0000

flight 165228 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165228/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              65499b4f090bf8b5c0699f27e755921f774fb1ee
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  445 days
Failing since        151818  2020-07-11 04:18:52 Z  444 days  435 attempts
Testing same since   165228  2021-09-28 04:18:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 76205 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 11:38:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 11:38:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197925.351175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBR2-0003Qa-I3; Tue, 28 Sep 2021 11:38:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197925.351175; Tue, 28 Sep 2021 11:38:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBR2-0003QT-F0; Tue, 28 Sep 2021 11:38:08 +0000
Received: by outflank-mailman (input) for mailman id 197925;
 Tue, 28 Sep 2021 11:38:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVBR1-0003QN-3g
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:38:07 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8b4e89b6-2050-11ec-bc82-12813bfff9fa;
 Tue, 28 Sep 2021 11:38:05 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b4e89b6-2050-11ec-bc82-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632829085;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=vAduIc2QrRWnu3zSLOmundYjzP0SK/Ij9o6vUCp7Y9o=;
  b=gia80Cbm0DB0FmvES/mJ2ka6d061wznecysYZLjdNp37aHxVQLdil3Yc
   k8sNsi4jV7ReareOE3djVsGIua7Wh8n1F/GguAZnSCjTlorfxfCVq7cOM
   l1t1IPixcg2/iu1eOMwvT0hBSyMJQFCZdp4WANFKBTPiBvacYvHjUngqL
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: f6jeRmfa0fLm32iNtCjYM4V0k3NIrQrY5xbed/TEmrmEIph6N5Ocfugdyi5vk5wyJUiKgHSLxS
 U8K2rPJ9pDcsQCFBgMN+KCXWKePa1GHjKClA/y/wLw50CehnqmUgn/h1xBekNDC8GnQaGUgJDQ
 fYDINWX+/Iys0MZsMI8l/L0IzBmUXPUdGwaU7SLhDxx1WQ5DI3kki+ISY8z6TP4Axfh4KncLHk
 HVjGY5usu8B2OLkFx0LBn34i6s6ke2CgxpbhNfxMJgXlB9JArd66hiXoyHlV/38MPaKSxj7DXg
 u+HEeaEKpc2TfcWc3HaCjOr4
X-SBRS: 5.1
X-MesageID: 54211808
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:css1PaNI07j/WkfvrR08kcFynXyQoLVcMsEvi/4bfWQNrUongjNUy
 zRNXD2CbP+OMTP2e9pybYm/9U0B6pGDxtJhTgto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Egzw7RRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYoxKqh4BV7
 O5xj6TzSkQuG7XUo9pMfTANRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgWtq35keRaa2i
 8wxcAo3bAjEbyx2NFouKoIwnOSq3lDUSmgNwL6SjfVuuDWCpOBr65D1PcbYYJqFX8RbkEeej
 mbH+W3jBVcdLtP34SSE6H+3hqnMgCbyQqoWFbux7Pksh0ecrkQMDDUGWF39puO24mauVtQaJ
 0EK9y4Gqakp6FftXtT7Rwe/onOPolgbQdU4O/Ym8giRzbGR7wuHLm8cCzpAc8A98ss3QzUw0
 RmOhdyBLTVrva3TQ3+b8LqZhTezPyEPKikFfyBsZQ4M/9nirZx1lhXOVNduCoaxj9v8Aza2x
 CqFxAA7hrYey84M06C25njDgjSlvJ+PSRQ6ji3MWnqN9A51ZIe5IYev7DDz9+tbMMOHS1ypu
 Hkfh9PY6/gDC52AimqKWuplNL224unfGDzaj0RmE5Qo+3Kq4XHLVZtM7T93IkdgGskBcD7tJ
 kTUvGt5/4RPNXGnaat2ZYOZCMkwy6XkU9P/WZj8R8BSb51GUR6I9SBje2adx2no1kMrlMkXP
 pOWct2wHG0aIatixTuyAewa1NcDzSkkyWTebZvy1Rig3PyVY3v9dFseGALQNKZjtvrC+ViLt
 YYEXyeX9/lBeMrdennz1bUvF2hJLjs8IIjukPIGWMfWd2KKB1odI/PWxLogfalskKJUivrE8
 xmBZ6NI9LbsrSeZcFrbMxiPfJuqDc4m8i5rVcA5FQzws0XPd7pD+0v2m3EfU7496KRHxON4Q
 uMJcsGNahikYm+coGxMBXURQZYLSfhKue5sF3b5CNTcV8Q5L+AsxjMCVlC0nMXpJnDr3fbSW
 5X6imvmrWMrHmyO9vr+ZvO11E+WtnMAgu90VEagCoANIx+zrtgwcnat16RfzyQwxfPrnWDyO
 +G+W0twmAUwi9VtrImhaV6s9e9F7NeS7mIFRjKGvN5axAHR/3a5wJ8obQp7VWq1aY8Aw437P
 b8956ilaJUvxQ8W26IhQ+cD5f9vvLPH+u4FpjmI6V2WNjxH/Js7eSLYtSSO34UQroJkVfyeA
 BnSpYUKZevTaKsI0jc5fWIYUwhK7tlN8hH65vUpOkTqoihx+buMS0JJOBeQzidaKdNI3EkNm
 4/NYeYatF6yjAQEKNGDgnwG/miANCVYAa4mqosbEMngjQ9ykgNOZpnVCynX5pCTaooTbhl2c
 2HM3KeS1a5BwkficmYoESSf1+RqmplT6gtBy0UPJgrVl4Od1OM3xhBY7R8+Uh9Rkkdcy+t2N
 2Uybx91KKyC8i1Gns9GW2zwSQhNCAfAoh76ykcTlX2fRE6tDzSfIGo4MOeL3UYY72MDIWQLo
 ODGkD7oCG+4ctvw0y0+XV9eh8biFdEhpBffnM2HHtieG8VoazTSnaLzN3EDrAHqAJ1tiRSf9
 /Vq5ut5dYbyKTUU//8gE4Cf2LkdFEKEKWhFTa0z9a8FBziBKjS72DzIIEGtYMJdYffN9BbgW
 cBpI8tOUTW41TqP8W9HVfJdfecska57/scGd5PqOXUC4umWoTdeuZ7N8jTz2T0wSNJ0nMdhc
 o7celpuyIBLaae4T4MVkPR5Bw==
IronPort-HdrOrdr: A9a23:6yswjKnOWxdO2fs8qXAv5ot37kDpDfO+imdD5ihNYBxZY6Wkfp
 +V8sjzhCWatN9OYh0dcLC7WJVpQRvnhPpICPoqTMmftW7dyRSVxeBZnPffKljbehEWmdQtrp
 uIH5IObuEYSGIK8PoSgzPIYOrIouP3iJxA7N22pxwGIHAIGsMQnTuRSDzrdXGeLDM2dabRf6
 Dsn/avyQDQHEj/Iv7LfEXsCIP41qz2fd/dEFI7Li9izDPLoSKj6bb8HRTd9hACUwlXybNn1W
 TeiQT26oiqrvn+k3bnpiHuxqUTvOGk5spIBcSKhMRQAjLwijywbIAkf7GZpjg6rMym9V5vut
 jRpBULOdh19hrqDy6IiCqo/zOl/Ccl6nfkx1PdqXz/ofbhTDZ/MMZFjZIxSGqT12MQ+PVHlI
 5b1WOQsJRaSTnamj7m2tTOXxZ20mKpvHsLi4co/j9ieLpbTIUUgZ0U/UtTHptFNjn98pobHO
 5nC9yZzOpKcGmdc2vSsgBUsZyRt0wIb1K7q3U5y4ioO2A8pgE/86JY/r1fop44zuN+d3EejN
 60dJiBl9l1P4crhOxGdb48qPCMexjwqCT3QSuvyGTcZdQ60k322unKCZUOlauXkc8zvdYPcK
 qoaiIviYd1QTO3NfGz
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="54211808"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lvjSuG6dWP2615/D4/3my2mco9bkt2aXEi0wvbqhl0HwtwWiGBr3+O2f+tJ++GoL/xLBfBtfXsVWfhJ9Zy+gj4bWXSWuyOgsfe47xELm0d5mopnrGyp6P7r+z51+/VWf6REY5vLgdeXaZyScKGavJ6wPfZxIsNrxbb6fHpVXh/5DLY1D3TwS/Sc4QaWoRww/ncAHX/As81dTObb93Q4tSSc5N1vYWOuCxAGO0aDw0tGFQDNmqyqpa+g01XCCoKK+pE+Runepk9dU/2UY3Qalamw0UzVs/yBtoEk8grbfcZGxryt24YoVv84DbQrc2ztCICJAoAbFYODuPSjJX+ireA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vAduIc2QrRWnu3zSLOmundYjzP0SK/Ij9o6vUCp7Y9o=;
 b=mgjBnxmqOgVaGhYdrdLee76/q3UTMN/a71RjR4ljcFBd1TxiWk+fx0ElErslfx2qMuisF8YRGx5qHbhQc9LbvL5pPPg/1CETpVu2CJw2QxiW/t+iChPILPpLQhfuwQpBOw0RW9CecDM7XLSnM+mmG1vUWuVlN6Vsg7gL3zXcsmzusU+xli/4tAJWAsNADgRdPPsz9HbOyrDjSixyaw/hJYwR5lTIWRjvD7rQflHn3Aq2nS+ndtHAOu/WVGNPjlZ/43dVXKt680/S99PNTVqUtAApmFXj0odY2funJGWyAzWXDfK9w56DFaeMcmA8LtHIt+VuetCrhHaDCS/yeKBSmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vAduIc2QrRWnu3zSLOmundYjzP0SK/Ij9o6vUCp7Y9o=;
 b=mksjrlk/jtOcCr7mGRMJFdZglhUkMzRpc1aBVzpb7h+n77fkQZjI77xHmMbJwD60kFltkbi+sakkq8W+k3E63wknXjoRv+/WBN3ZKqD4vxKgzVNsCCgpBp6vM3HQJPvYQvmPjvMOOQxDu775u0HcNMGZiaUS77B3K2Z8ptl/vJs=
To: =?UTF-8?Q?Alex_Benn=c3=a9e?= <alex.bennee@linaro.org>, "Stratos Mailing
 List" <stratos-dev@op-lists.linaro.org>
CC: Mike Holmes <mike.holmes@linaro.org>, Mathieu Poirier
	<mathieu.poirier@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>, "Peter
 Griffin" <peter.griffin@linaro.org>, <xen-devel@lists.xenproject.org>,
	<wl@xen.org>, Artem Mygaiev <Artem_Mygaiev@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Doug Goldstein <cardoe@cardoe.com>, "Oleksandr
 Tyshchenko" <olekstysh@gmail.com>, Rust-VMM Mailing List
	<rust-vmm@lists.opendev.org>, Sergio Lopez <slp@redhat.com>, Stefan Hajnoczi
	<stefanha@gmail.com>, David Woodhouse <dwmw2@infradead.org>
References: <87pmsylywy.fsf@linaro.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Xen Rust VirtIO demos work breakdown for Project Stratos
Message-ID: <e1381c7d-a4e4-8fb4-a5a0-8b50ce273c61@citrix.com>
Date: Tue, 28 Sep 2021 12:37:35 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <87pmsylywy.fsf@linaro.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0512.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:13b::19) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: de385d32-d08a-4ee5-ac5e-08d982746286
X-MS-TrafficTypeDiagnostic: BYAPR03MB4807:
X-Microsoft-Antispam-PRVS: <BYAPR03MB480706F3FD11C0E2B73DC62DBAA89@BYAPR03MB4807.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1W4IPCEBm5NtrsyuXNJQvlY8ECrST+NVfpwwrepfyqH74yVwlSzLHG1mxubflQUxYQdPxASbakxnPYywBQCP9O7BUzq6TMqK7IkUYF0ffvwY8IuEq0iDsIcxtyQ29DJ9kbzpI3XQnKsK2+pnjoD4gTa7afDupFE0jB2d7JpqQVono0U4MrFqxFmPfvHuwbvA2T6W+a2m6aFgknF6g/hTuEor5eDexDB+bxRqJLED11PRYFz7QC8w7ny4bvzU/2af6fHzAGgXgAlRsH9sJwzZXVW/BBxxMYd7kDRpGUK0edjy906ngc+pPSHQW1cKb+mKJwBD3Y0du5bMPuVfCTePoUiEQnV5uVJoOVk2+LsKbssty0cc7nWrFkBvs1Kx7OHBJlTv7cOJDkslCFml5pOfCrgv5CWKjKinXRPHyYTsMd7kMISoUSNyf6Sb4FbWjyLraOK5HNYTtoGY9K7h6IHHeXWyYkuf/NICWE6Q2b+F3skra4TcOJD6aibZ7IoKKYEregXT6Syi8vuR6eyAAo6lIK8cOerBmJTL3piZHWgGCjjnkq1++3IvsD3eWPTAS7gJfrRNm8cGIu9px7JgNZ8ROBZ76ty9ESFTE1fyt7aIgp6uHO57XeIk515E1i4IYGIfsCV35sJ7VgeQIhzpv4aOsDE1/qJop3htjJnqX1KyBejST9kwPA6+70FTq97LScrSJA4LNQrq/Eo3vFS5K22x12RBmjkmDy+T0CknDLkuEqBdJBqsJqiCt8gxm/1LszUp7PcD9plPB131b6nMOf56Pt3e8gOSzv1KOJQjUSpdCX/nLZdCqkNeatWxaUZjRWcn
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(966005)(8676002)(66556008)(86362001)(316002)(6666004)(66946007)(66476007)(2906002)(508600001)(8936002)(66574015)(26005)(45080400002)(16576012)(38100700002)(36756003)(55236004)(53546011)(31696002)(83380400001)(956004)(4326008)(5660300002)(54906003)(110136005)(2616005)(31686004)(6486002)(7416002)(186003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?di8zb1pFRGt1TTg4Y2xqU0hDbmV5ZWVjaUlGWGJFQ0l1Ykl4UkZmZ0M2NFFn?=
 =?utf-8?B?SSt0WlRWd2JJc2NmVFJTaUhzQ2pUSUZ5QSt0ejRuSEFSaERoOGFHZ29hbG9D?=
 =?utf-8?B?U2gzSkFiUFN5UHZpQlJOSnpWTkhiZWQ0aS9ydDdYWlpGQUxkOFA5NGVDTGtx?=
 =?utf-8?B?UWpxYTFtVkpRNXRJcEVYbDRJQ1hTVmVqbGh3elV0VWdieW5LME95Zmp5YWVx?=
 =?utf-8?B?QWRnNTczUTQzNnUrZzlvaDMvMGljUVNmYko2OCtZeWxkdnpMZllQM0RDNzFS?=
 =?utf-8?B?Q2k0dk5zYWJQTXF5ai9rZHNDRnBqWUpXYklUcVdHOUs3MjlMdDBXMDNzTjcr?=
 =?utf-8?B?MVFIZ21zNjcxNVVaT1VxMkJ0VGZxMk9nTVZlejdGYWZGbWRsK056cjVlQ1I4?=
 =?utf-8?B?dXQyUDE1ZS84UjA0THJsWmV0eGcrMHV3bXkwdk90SCtUQ1pTcFVtbGRkMEw5?=
 =?utf-8?B?Y0ZCZjd0dG04MWFPME56VUU3ZURUOGxod1dMcmV0eUptaUR0cGRsYVpUNXVT?=
 =?utf-8?B?SnpYL096THFYTVQrME9TY1lHNEVqKzVnOGl1ZHk2ZFdMdGtrcHNNNVc5a2J5?=
 =?utf-8?B?b2p6UHFKMkxWL0h2QzZxZnAwaXdlTTQ2UytXenZsSkVhMXk5YUl3QzVSYlVx?=
 =?utf-8?B?K0ZUQUpKSjlqV1doNzdxZHFuQkxDb2ZBd1lQczFFOGVuS2JJNDFrU1RDV3No?=
 =?utf-8?B?bVovM2xuck8yQnc0KzdrK0U4TGFpeGh1RXFjK2JmYmdGRUdXak5nUTZvcTJV?=
 =?utf-8?B?YWgzYWpNZWVzRnJ2OWRnZ21iZ0JxOVpjdFA5bGdGaUcxMnUrWjJ1R1NqQ2ox?=
 =?utf-8?B?bktKUEJKWHpXb2dIdVBBU1pFdFRITWU4dEx4azZiQXdYb3IzemlZS2c5akpz?=
 =?utf-8?B?THNyaXNJU3pwVnVxU0tBaElpOWdIZVN6a293T1I3ZmZzVU5GU3lRSExsRVpI?=
 =?utf-8?B?UFRXZlc1VFpIWjZ6aS9yc2RLOElJZTlpSFRMa2tWQkJmV092NE91eWU2bWIw?=
 =?utf-8?B?NTlyYVlhTEhiK2l0NDFTY0ZaL2VzOTZ5VWlaLzJsbkNOM0dFdlJhamJzUWJp?=
 =?utf-8?B?cjVSeElUUVhGNEcyUlVoOUxlc25HR214YmJjTUFnV01KVjUvaUVGUm9ZL1pU?=
 =?utf-8?B?MXZVcHZnWWkyd3VTQVN4a2ozTmVLcDNoY2tNYWVXOEQ2S1RkaVBwRHNpazR2?=
 =?utf-8?B?NWIxS1FlNms2YXQ3MmJmNlZYOFN5UFM1cGdTZHJhbGhrNlBpNGk4WVFycTJK?=
 =?utf-8?B?WWtXSnkycXZMN01HVm9yVVVvME94TmVPUGUxamVGS3ZqM1hOKzU5eVJWQjZX?=
 =?utf-8?B?SGYwbXVqbktSdHlyVDBaL2FWbElGcENleUoyRzhMa0VNclcvWUVvbG5GU3RR?=
 =?utf-8?B?VGZ4Y0s5Q3p2NVF1TzUyRUpzTks5MmJQTkJMTGR3SEwwdmNvaHhzVC84eUhk?=
 =?utf-8?B?QVNodU1uV0xMaXpFTlNjTGtnR2VjNnhVKzlyR09jenNGZWtkcVh6SHdtam00?=
 =?utf-8?B?ZGI5VHpFSUlkWW53cytpdTZpNm5RZ3hoRFNGSmQ1ajR6QVRmNWNPdW5pdi9L?=
 =?utf-8?B?cTIzeG8wNDd0WGRzYlZGQmlOZEJDSmpKL3lPQTFnM0FDSWViTWhoeC9VeTNz?=
 =?utf-8?B?em83ZzBDUy80Z2k3Y01LL2pQWW45Nlc1cEV3ckJsQi95M2RYQ2NCMTVNWDB6?=
 =?utf-8?B?M1ZFV29YZTZ6OElxNFZseGxZNzFUM3l4TzdCTjJmWC9TNWFCdmNjSFQ5SU0y?=
 =?utf-8?Q?EwlcCwWN2KkyCtNixMyUpeGo6jclmCXd+Jmt59H?=
X-MS-Exchange-CrossTenant-Network-Message-Id: de385d32-d08a-4ee5-ac5e-08d982746286
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 11:37:43.2734
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uQoPBL6EyynzvFfEPjLJz6FU5f3/7QoYYDW070djq0uMcBtYDTE+nUBJavGdkxeoz4sh6cRY4D3ptJZVsNUrETsIKffyE3PJ98HMk8Y7ch4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4807
X-OriginatorOrg: citrix.com

T24gMjQvMDkvMjAyMSAxNzowMiwgQWxleCBCZW5uw6llIHdyb3RlOgo+IDEuMSBVcHN0cmVhbSBh
biAib2ZmaWNpYWwiIHJ1c3QgY3JhdGUgZm9yIFhlbiAoW1NUUi01Ml0pCj4g4pSA4pSA4pSA4pSA
4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA
4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA
4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSA4pSACj4KPiAgIFRvIHN0YXJ0
IHdpdGggd2Ugd2lsbCB3YW50IGFuIHVwc3RyZWFtIGxvY2F0aW9uIGZvciBmdXR1cmUgd29yayB0
byBiZQo+ICAgYmFzZWQgdXBvbi4gVGhlIGludGVudGlvbiBpcyB0aGUgY3JhdGUgaXMgaW5kZXBl
bmRlbnQgb2YgdGhlIHZlcnNpb24KPiAgIG9mIFhlbiBpdCBydW5zIG9uIChhYm92ZSB0aGUgYmFz
ZWxpbmUgdmVyc2lvbiBjaG9zZW4pLiBUaGlzIHdpbGwKPiAgIGVudGFpbDoKPgo+ICAg4oCiIOKY
kCBhZ3JlZWluZyB3aXRoIHVwc3RyZWFtIHRoZSBuYW1lL2xvY2F0aW9uIGZvciB0aGUgc291cmNl
CgpQcm9iYWJseSBnaXRodWIveGVuLXByb2plY3QvcnVzdC1iaW5kaW5ncyB1bmxlc3MgYW55b25l
IGhhcyBhIGJldHRlcgpzdWdnZXN0aW9uLgoKV2UgYWxtb3N0IGNlcnRhaW5seSB3YW50IGEgY29t
cGFuaW9uIHJlcG9zaXRvcnkgY29uZmlndXJlZCBhcyBhCmhlbGxvLXdvcmxkIGV4YW1wbGUgdXNp
bmcgdGhlIGJpbmRpbmdzIGFuZCAoY3Jvc3MtKWNvbXBpbGVkIGZvciBlYWNoCmJhY2tlbmQgdGFy
Z2V0LgoKPiAgIOKAoiDimJAgZG9jdW1lbnRpbmcgdGhlIHJ1bGVzIGZvciB0aGUgInN0YWJsZSIg
aHlwZXJjYWxsIEFCSQoKRWFzeS7CoCBUaGVyZSBzaGFsbCBiZSBubyB1c2Ugb2YgdW5zdGFibGUg
aW50ZXJmYWNlcyBhdCBhbGwuCgpUaGlzIGlzIHRoZSAqb25seSogd2F5IHRvIGF2b2lkIG1ha2lu
ZyB0aGUgYmluZGluZ3MgZGVwZW5kZW50IG9uIHRoZQp2ZXJzaW9uIG9mIHRoZSBoeXBlcnZpc29y
LCBhbmQgd2lsbCBiZSBhIG1ham9yIGltcHJvdmVtZW50IGluIHRoZSBYZW4KZWNvc3lzdGVtLgoK
QW55IHVuc3RhYmxlIGh5cGVyY2FsbCB3YW50aW5nIHRvIGJlIHVzZWQgc2hhbGwgYmUgc3RhYmls
aXNlZCBpbiBYZW4KZmlyc3QsIHdoaWNoIGhhcyBiZWVuIHZlaGVtZW50bHkgYWdyZWVkIHRvIGF0
IG11bHRpcGxlIGRldiBzdW1taXRzIGluCnRoZSBwYXN0LCBhbmQgd2lsbCBiZSBhIHVzZWZ1bCB3
YXkgb2YgZ3VpZGluZyB0aGUgc3RhYmlsaXNhdGlvbiBlZmZvcnQuCgo+ICAg4oCiIOKYkCBlc3Rh
Ymxpc2ggYW4gaW50ZXJuYWwgaW50ZXJmYWNlIHRvIGVsaWRlIGJldHdlZW4gaW9jdGwgbWVkaWF0
ZWQKPiAgICAgYW5kIGRpcmVjdCBoeXBlcmNhbGxzCj4gICDigKIg4piQIGVuc3VyZSB0aGUgY3Jh
dGUgaXMgbXVsdGktYXJjaCBhbmQgaGFzIGZlYXR1cmUgcGFyaXR5IGZvciBhcm02NAo+Cj4gICBB
cyBzdWNoIHdlIGV4cGVjdCB0aGUgaW1wbGVtZW50YXRpb24gdG8gYmUgc3RhbmRhbG9uZSwgaS5l
LiBub3QKPiAgIHdyYXBwaW5nIHRoZSBleGlzdGluZyBYZW4gbGlicmFyaWVzIGZvciBtZWRpYXRp
b24uIFRoZXJlIHNob3VsZCBiZSBhCj4gICBjbG9zZSAoMS10by0xKSBtYXBwaW5nIGJldHdlZW4g
dGhlIGludGVyZmFjZXMgaW4gdGhlIGNyYXRlIGFuZCB0aGUKPiAgIGV2ZW50dWFsIGh5cGVyY2Fs
bCBtYWRlIHRvIHRoZSBoeXBlcnZpc29yLgo+Cj4gICBFc3RpbWF0ZTogNHcgKGVsYXBzZWQgbGlr
ZWx5IGxvbmdlciBkdWUgdG8gZGlzY3Vzc2lvbikKPgo+Cj4gW1NUUi01Ml0gPGh0dHBzOi8vbmFt
MDQuc2FmZWxpbmtzLnByb3RlY3Rpb24ub3V0bG9vay5jb20vP3VybD1odHRwcyUzQSUyRiUyRmxp
bmFyby5hdGxhc3NpYW4ubmV0JTJGYnJvd3NlJTJGU1RSLTUyJmFtcDtkYXRhPTA0JTdDMDElN0NB
bmRyZXcuQ29vcGVyMyU0MGNpdHJpeC5jb20lN0M5NzMwODcyNDhmNDA0MDE4YTM2MDA4ZDk3Zjc4
YTBlYSU3QzMzNTgzNmRlNDJlZjQzYTJiMTQ1MzQ4YzJlZTljYTViJTdDMCU3QzAlN0M2Mzc2ODA5
NzgzNjMwNzA1NzYlN0NVbmtub3duJTdDVFdGcGJHWnNiM2Q4ZXlKV0lqb2lNQzR3TGpBd01EQWlM
Q0pRSWpvaVYybHVNeklpTENKQlRpSTZJazFoYVd3aUxDSlhWQ0k2TW4wJTNEJTdDMTAwMCZhbXA7
c2RhdGE9MGs2RVM5SVhDdlRscVA4alRpRmV1VE41MUdsTW5neGFlcFd2UUM2aE1BZyUzRCZhbXA7
cmVzZXJ2ZWQ9MD4KPgo+Cj4gMS4yIEJhc2ljIEh5cGVydmlzb3IgSW50ZXJhY3Rpb25zIGh5cGVy
Y2FsbHMgKFtTVFItNTNdKQo+IOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgAo+Cj4gICBUaGVzZSBhcmUgdGhlIGJhcmUgbWluaW11bSBoeXBlcmNhbGxz
IGltcGxlbWVudGVkIGFzIGJvdGggaW9jdGwgYW5kCj4gICBkaXJlY3QgY2FsbHMuIFRoZXNlIGFs
bG93IGZvciBhIHZlcnkgYmFzaWMgYmluYXJ5IHRvOgo+Cj4gICDigKIg4piQIGNvbnNvbGVfaW8g
LSBvdXRwdXQgSU8gdmlhIHRoZSBYZW4gY29uc29sZQo+ICAg4oCiIOKYkCBkb21jdGwgc3R1YiAt
IGJhc2ljIHN0dWIgZm9yIGRvbWFpbiBjb250cm9sIChkaWZmZXJlbnQgQVBJPykKPiAgIOKAoiDi
mJAgc3lzY3RsIHN0dWIgLSBiYXNpYyBzdHViIGZvciBzeXN0ZW0gY29udHJvbCAoZGlmZmVyZW50
IEFQST8pCj4KPiAgIFRoZSBpZGVhIHdvdWxkIGJlIHRoaXMgcHJvdmlkZXMgZW5vdWdoIGh5cGVy
Y2FsbCBpbnRlcmZhY2UgdG8gcXVlcnkKPiAgIHRoZSBsaXN0IG9mIGRvbWFpbnMgYW5kIG91dHB1
dCB0aGVpciBzdGF0dXMgdmlhIHRoZSB4ZW4gY29uc29sZS4gVGhlcmUKPiAgIGlzIGFuIG9wZW4g
cXVlc3Rpb24gYWJvdXQgaWYgdGhlIGRvbWN0bCBhbmQgc3lzY3RsIGh5cGVyY2FsbHMgYXJlIHdh
eQo+ICAgdG8gZ28uCgpjb25zb2xlX2lvIHByb2JhYmx5IHdhbnRzIGltcGxlbWVudGluZyBhcyBh
IGJhY2tlbmQgdG8gcHJpbnRsbiEoKSBvciB0aGUKbG9nIG1vZHVsZSwgYmVjYXVzZSB1c2VycyBv
ZiB0aGUgY3JhdGUgd29uJ3Qgd2FudCBjaGFuZ2UgaG93IHRoZXkKcHJpbnRmKCkvZXRjIGRlcGVu
ZGluZyBvbiB0aGUgdGFyZ2V0LgoKVGhhdCBzYWlkLCBjb25zb2xlX2lvIGh5cGVyY2FsbHMgb25s
eSBkbyBhbnl0aGluZyBmb3IgdW5wcml2bGVnZWQgVk1zIGluCmRlYnVnIGJ1aWxkcyBvZiB0aGUg
aHlwZXJ2aXNvci7CoCBUaGlzIGlzIGZpbmUgZm9yIGRldmVsb3BtZW50LCBhbmQgbGVzcwpmaW5l
IGluIHByb2R1Y3Rpb24sIHNvIGxvZ2dpbmcgb3VnaHQgdG8gdXNlIHRoZSBQViBjb25zb2xlIGlu
c3RlYWQgKHdpdGgKcm9vbSBmb3IgZnV0dXJlIGV4cGFuc2lvbiB0byBhbiBBcmdvIHRyYW5zcG9y
dCkuCgpkb21jdGwvc3lzY3RsIGFyZSB1bnN0YWJsZSBpbnRlcmZhY2VzLsKgIEkgZG9uJ3QgdGhp
bmsgdGhleSdsbCBiZQpuZWNlc3NhcnkgZm9yIGEgYmFzaWMgdmlydGlvIGJhY2tlbmQsIGFuZCB0
aGV5IHdpbGwgYmUgdGhlIG1vc3QKY29tcGxpY2F0ZWQgaHlwZXJjYWxscyB0byBzdGFiaWxpc2Uu
Cgo+Cj4gICBFc3RpbWF0ZTogNncKPgo+Cj4gW1NUUi01M10gPGh0dHBzOi8vbmFtMDQuc2FmZWxp
bmtzLnByb3RlY3Rpb24ub3V0bG9vay5jb20vP3VybD1odHRwcyUzQSUyRiUyRmxpbmFyby5hdGxh
c3NpYW4ubmV0JTJGYnJvd3NlJTJGU1RSLTUzJmFtcDtkYXRhPTA0JTdDMDElN0NBbmRyZXcuQ29v
cGVyMyU0MGNpdHJpeC5jb20lN0M5NzMwODcyNDhmNDA0MDE4YTM2MDA4ZDk3Zjc4YTBlYSU3QzMz
NTgzNmRlNDJlZjQzYTJiMTQ1MzQ4YzJlZTljYTViJTdDMCU3QzAlN0M2Mzc2ODA5NzgzNjMwNzA1
NzYlN0NVbmtub3duJTdDVFdGcGJHWnNiM2Q4ZXlKV0lqb2lNQzR3TGpBd01EQWlMQ0pRSWpvaVYy
bHVNeklpTENKQlRpSTZJazFoYVd3aUxDSlhWQ0k2TW4wJTNEJTdDMTAwMCZhbXA7c2RhdGE9MG1n
TVBTNGlrekQlMkY4ZmMlMkJjcVZqdFJ1U0t6TXUlMkJhOFhnT3M0aE9DOXBZNCUzRCZhbXA7cmVz
ZXJ2ZWQ9MD4KPgo+Cj4gMS4zIFsjMTBdIEFjY2VzcyB0byBYZW5TdG9yZSBzZXJ2aWNlIChbU1RS
LTU0XSkKPiDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDi
lIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDi
lIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIAKPgo+ICAgVGhpcyBpcyBhIHNoYXJl
ZCBjb25maWd1cmF0aW9uIHN0b3JhZ2Ugc3BhY2UgYWNjZXNzZWQgdmlhIGVpdGhlciBVbml4Cj4g
ICBzb2NrZXRzIChvbiBkb20wKSBvciB2aWEgdGhlIFhlbmJ1cy4gVGhpcyBpcyB1c2VkIHRvIGFj
Y2Vzcwo+ICAgY29uZmlndXJhdGlvbiBpbmZvcm1hdGlvbiBmb3IgdGhlIGRvbWFpbi4KPgo+ICAg
SXMgdGhpcyBuZWVkZWQgZm9yIGEgYmFja2VuZCB0aG91Z2g/IENhbiBldmVyeXRoaW5nIGp1c3Qg
YmUgcGFzc2VkCj4gICBkaXJlY3Qgb24gdGhlIGNvbW1hbmQgbGluZT8KCkN1cnJlbnRseSwgaWYg
eW91IHdhbnQgYSBzdHViZG9tIGFuZCB5b3Ugd2FudCB0byBpbnN0cnVjdCBpdCB0byBzaHV0CmRv
d24gY2xlYW5seSwgaXQgbmVlZHMgeGVuc3RvcmUuwqAgQW55IHN0dWJkb20gd2hpY2ggd2FudHMg
ZGlzayBvcgpuZXR3b3JrIG5lZWRzIHhlbnN0b3JlIHRvby4KCnhlbmJ1cyAodGhlIHRyYW5zcG9y
dCkgZG9lcyBuZWVkIHRvIHNwbGl0IGJldHdlZW4gaW9jdGwoKSdzIGFuZCByYXcKaHlwZXJjYWxs
cy7CoCB4ZW5zdG9yZSAodGhlIHByb3RvY29sKSBjb3VsZCBiZSBpbiB0aGUgeGVuIGNyYXRlLCBv
ciBhCnNlcGFyYXRlIG9uZSBhcyBpdCBpcyBhIHBpZWNlIG9mIGhpZ2hlciBsZXZlbCBmdW5jdGlv
bmFsaXR5LgoKSG93ZXZlciwgd2Ugc2hvdWxkIHBheSBhdHRlbnRpb24gdG8gbm9uLXhlbnN0b3Jl
IHVzZWNhc2VzIGFuZCBub3QgcGFpbnQKb3Vyc2VsdmVzIGludG8gYSBjb3JuZXIuwqAgU29tZSBz
ZWN1cml0eSB1c2VjYXNlcyB3b3VsZCBwcmVmZXIgbm90IHRvIHVzZQpzaGFyZWQgbWVtb3J5LCBh
bmQgZS5nLiBtaWdodCBjb25zaWRlciB1c2luZyBhbiBBcmdvIHRyYW5zcG9ydCBpbnN0ZWFkCm9m
IHRoZSB0cmFkaXRpb25hbCBncmFudC1zaGFyZWQgcGFnZS4KCj4KPiAgIEVzdGltYXRlOiA0dwo+
Cj4KPiBbU1RSLTU0XSA8aHR0cHM6Ly9uYW0wNC5zYWZlbGlua3MucHJvdGVjdGlvbi5vdXRsb29r
LmNvbS8/dXJsPWh0dHBzJTNBJTJGJTJGbGluYXJvLmF0bGFzc2lhbi5uZXQlMkZicm93c2UlMkZT
VFItNTQmYW1wO2RhdGE9MDQlN0MwMSU3Q0FuZHJldy5Db29wZXIzJTQwY2l0cml4LmNvbSU3Qzk3
MzA4NzI0OGY0MDQwMThhMzYwMDhkOTdmNzhhMGVhJTdDMzM1ODM2ZGU0MmVmNDNhMmIxNDUzNDhj
MmVlOWNhNWIlN0MwJTdDMCU3QzYzNzY4MDk3ODM2MzA3MDU3NiU3Q1Vua25vd24lN0NUV0ZwYkda
c2IzZDhleUpXSWpvaU1DNHdMakF3TURBaUxDSlFJam9pVjJsdU16SWlMQ0pCVGlJNklrMWhhV3dp
TENKWFZDSTZNbjAlM0QlN0MxMDAwJmFtcDtzZGF0YT1tVVJ4eG83dlF3ZlRrUjRjWDV5TjVsN2tQ
YXYyZ1hobHVPaG0yJTJCU0JBSXMlM0QmYW1wO3Jlc2VydmVkPTA+Cj4KPgo+IDEuNCBWaXJ0SU8g
c3VwcG9ydCBoeXBlcmNhbGxzIChbU1RSLTU1XSkKPiDilIDilIDilIDilIDilIDilIDilIDilIDi
lIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDi
lIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIDilIAKPgo+ICAgVGhlc2UgYXJlIHRo
ZSBoeXBlcmNhbGxzIHRoYXQgbmVlZCB0byBiZSBpbXBsZW1lbnRlZCB0byBzdXBwb3J0IGEKPiAg
IFZpcnRJTyBiYWNrZW5kLiBUaGlzIGluY2x1ZGVzIHRoZSBhYmlsaXR5IHRvIG1hcCBhbm90aGVy
IGd1ZXN0cyBtZW1vcnkKPiAgIGludG8gdGhlIGN1cnJlbnQgZG9tYWlucyBhZGRyZXNzIHNwYWNl
LCByZWdpc3RlciB0byByZWNlaXZlIElPUkVRCj4gICBldmVudHMgd2hlbiB0aGUgZ3Vlc3Qga25v
Y2tzIGF0IHRoZSBkb29yYmVsbCBhbmQgaW5qZWN0IGtpY2tzIGludG8gdGhlCj4gICBndWVzdC4g
VGhlIGh5cGVyY2FsbHMgd2UgbmVlZCB0byBzdXBwb3J0IHdvdWxkIGJlOgo+Cj4gICDigKIg4piQ
IGRtb3AgLSBkZXZpY2UgbW9kZWwgb3BzICgqX2lvcmVxX3NlcnZlciwgc2V0aXJxLCBucl92cHVz
KQo+ICAg4oCiIOKYkCBmb3JlaWdubWVtb3J5IC0gbWFwIGFuZCB1bm1hcCBndWVzdCBtZW1vcnkK
CmFsc28gZXZ0Y2huLCB3aGljaCB5b3UgbmVlZCBmb3IgaW9yZXEgbm90aWZpY2F0aW9ucy4KCj4g
ICBUaGUgRE1PUCBzcGFjZSBpcyBsYXJnZXIgdGhhbiB3aGF0IHdlIG5lZWQgZm9yIGFuIElPUkVR
IGJhY2tlbmQgc28KPiAgIEkndmUgYmFzZWQgaXQganVzdCBvbiB3aGF0IGFyY2gvYXJtL2RtLmMg
ZXhwb3J0cyB3aGljaCBpcyB0aGUgc3Vic2V0Cj4gICBpbnRyb2R1Y2VkIGZvciBFUEFNJ3Mgdmly
dGlvIHdvcmsuCgpPbmUgdGhpbmcgd2Ugd2lsbCB3YW50IHRvIGJlIGlzIGNhcmVmdWwgd2l0aCB0
aGUgaW50ZXJmYWNlLsKgIFRoZSBjdXJyZW50CkRNT1BzIGFyZSBhIG1lc3Mgb2YgdW5pdHMgKHBh
cnRpY3VsYXJseSBmcmFtZXMgdnMgYWRkcmVzc2VzLCB3aGljaCB3aWxsCm5lZWQgdG8gY2hhbmdl
IGluIFhlbiBpbiBkdWUgY291cnNlKSBhcyB3ZWxsIGFzIHJhbmdlCmluY2x1c2l2aXR5L2V4Y2x1
c2l2aXR5LgoKPgo+ICAgRXN0aW1hdGU6IDEydwo+Cj4KPiBbU1RSLTU1XSA8aHR0cHM6Ly9uYW0w
NC5zYWZlbGlua3MucHJvdGVjdGlvbi5vdXRsb29rLmNvbS8/dXJsPWh0dHBzJTNBJTJGJTJGbGlu
YXJvLmF0bGFzc2lhbi5uZXQlMkZicm93c2UlMkZTVFItNTUmYW1wO2RhdGE9MDQlN0MwMSU3Q0Fu
ZHJldy5Db29wZXIzJTQwY2l0cml4LmNvbSU3Qzk3MzA4NzI0OGY0MDQwMThhMzYwMDhkOTdmNzhh
MGVhJTdDMzM1ODM2ZGU0MmVmNDNhMmIxNDUzNDhjMmVlOWNhNWIlN0MwJTdDMCU3QzYzNzY4MDk3
ODM2MzA3MDU3NiU3Q1Vua25vd24lN0NUV0ZwYkdac2IzZDhleUpXSWpvaU1DNHdMakF3TURBaUxD
SlFJam9pVjJsdU16SWlMQ0pCVGlJNklrMWhhV3dpTENKWFZDSTZNbjAlM0QlN0MxMDAwJmFtcDtz
ZGF0YT10Y3pjR2w2WVdHUnduQm9vQndvZERYMkJITEt1NXRKRzMlMkZUZSUyRnBxZiUyQjl3JTNE
JmFtcDtyZXNlcnZlZD0wPgo+Cj4KPiAyIFhlbiBIeXBlcnZpc29yIFN1cHBvcnQgZm9yIFN0cmF0
b3MgKFtTVFItNTZdKQo+IOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKV
kOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKV
kOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkOKVkAo+Cj4gICBUaGVzZSB0
YXNrcyBpbmNsdWRlIHRhc2tzIG5lZWRlZCB0byBzdXBwb3J0IHRoZSB2YXJpb3VzIGRpZmZlcmVu
dAo+ICAgZGVwbG95bWVudHMgb2YgU3RyYXRvcyBjb21wb25lbnRzIGluIFhlbi4KPgo+Cj4gW1NU
Ui01Nl0gPGh0dHBzOi8vbmFtMDQuc2FmZWxpbmtzLnByb3RlY3Rpb24ub3V0bG9vay5jb20vP3Vy
bD1odHRwcyUzQSUyRiUyRmxpbmFyby5hdGxhc3NpYW4ubmV0JTJGYnJvd3NlJTJGU1RSLTU2JmFt
cDtkYXRhPTA0JTdDMDElN0NBbmRyZXcuQ29vcGVyMyU0MGNpdHJpeC5jb20lN0M5NzMwODcyNDhm
NDA0MDE4YTM2MDA4ZDk3Zjc4YTBlYSU3QzMzNTgzNmRlNDJlZjQzYTJiMTQ1MzQ4YzJlZTljYTVi
JTdDMCU3QzAlN0M2Mzc2ODA5NzgzNjMwNzA1NzYlN0NVbmtub3duJTdDVFdGcGJHWnNiM2Q4ZXlK
V0lqb2lNQzR3TGpBd01EQWlMQ0pRSWpvaVYybHVNeklpTENKQlRpSTZJazFoYVd3aUxDSlhWQ0k2
TW4wJTNEJTdDMTAwMCZhbXA7c2RhdGE9MzFsRnF4a2JadHlOZFFjeVZJMWQ4TTUzbDRKcVZobmkx
czFhb3dydG9YZyUzRCZhbXA7cmVzZXJ2ZWQ9MD4KPgo+IDIuMSBTdGFibGUgQUJJIGZvciBmb3Jl
aWdubWVtb3J5IG1hcHBpbmcgdG8gbm9uLWRvbTAgKFtTVFItNTddKQo+IOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKUgOKU
gOKUgAo+Cj4gICBDdXJyZW50bHkgdGhlIGZvcmVpZ24gbWVtb3J5IG1hcHBpbmcgc3VwcG9ydCBv
bmx5IHdvcmtzIGZvciBkb20wIGR1ZQo+ICAgdG8gcmVmZXJlbmNlIGNvdW50aW5nIGlzc3Vlcy4g
SWYgd2UgYXJlIHRvIHN1cHBvcnQgYmFja2VuZHMgcnVubmluZyBpbgo+ICAgdGhlaXIgb3duIGRv
bWFpbnMgdGhpcyB3aWxsIG5lZWQgdG8gZ2V0IGZpeGVkLgoKT2guwqAgSXQgYXBwZWFycyBhcyBp
ZiBzb21lIG9mIHRoaXMgd2FzIGNvbXBsZXRlZCBpbgpodHRwczovL3hlbmJpdHMueGVuLm9yZy9n
aXR3ZWIvP3A9eGVuLmdpdDthPWNvbW1pdGRpZmY7aD00OTIyY2FmMWRlNWEwOGQzZWVmYjQwNThk
ZTFiN2YwMTIyYzhmNzZmCgp+QW5kcmV3Cg==


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 11:39:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 11:39:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197931.351186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBSF-00041H-Rg; Tue, 28 Sep 2021 11:39:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197931.351186; Tue, 28 Sep 2021 11:39:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBSF-00041A-Ok; Tue, 28 Sep 2021 11:39:23 +0000
Received: by outflank-mailman (input) for mailman id 197931;
 Tue, 28 Sep 2021 11:39:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GBNq=OS=fedoraproject.org=pjp@srs-us1.protection.inumbo.net>)
 id 1mVBSD-000410-J9
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:39:21 +0000
Received: from sonic302-20.consmr.mail.sg3.yahoo.com (unknown [106.10.242.140])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b680a539-2050-11ec-bc82-12813bfff9fa;
 Tue, 28 Sep 2021 11:39:20 +0000 (UTC)
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic302.consmr.mail.sg3.yahoo.com with HTTP; Tue, 28 Sep 2021 11:39:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b680a539-2050-11ec-bc82-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1632829157; bh=ig9a+GEMENDbq6RaHsP6jNzuSOjps6RB21m4qLU3NP8=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=GpGydf3G3UBZu3hS4/pdykRd3vtdl3cpTLkM6myxVHltaV7AiGGF0pBZ690eRHao3HqEBdfLnxxjtbxsrWO+Rql8vUgBqXlWqsUKNd9vfng/hGuJ0ljT720QryQi+EqcowRCi9WehI8vwSiood7P73NK4IWg3CtcY4DovIMnBY+10s1rAb0ebEI7caMeeVnpA2gMYfqF2UJL58PndStnsdlOQfh+nTftQ8Z1dCsMPbY4waGVKHmbXoYpDNFcqiAeTfPY7aOacHtaUAwwOxOOHPTc3V7p+fwSQ2TumCq2tYWh98EapTL+rW5NfJD3Su8iwShtl0MTbNBIN/kRv4Rziw==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1632829157; bh=Yg0XwtaKmUI6Bqcl2pWhfgps3zqqwrtClGAMnyJ69t5=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=qBAUw+PbjatOWOht0Lb7FiZhmDguQtCgmdW6VfGiGKp9YmtwxcVKKroN8TcRzaHjrxwed/HykuDV527SJMl6p3ezxISsUzmwiYre9ERom9KgvHoPlYFpoNNKDOB70ulHZZGoQt1E7JkkFNflRSrJ9vJfK24keZpApAYu8uxJB9nwY5kWKdcaa1uchvOCLTq7JWAj7VJm0VowO78uyMGa12yweRKE+DggDT/rHHu3Pkl2kYqbmZbvPl1MyJ1yCkiw8cTX/6JojwxkG23qc125RJQuIggnLgUxZ6iW/NEpkPUc2dC9ui8L2+B2/9X7FG0JX4GszbKQ4bjzOlTj7B2Ljg==
X-YMail-OSG: 21yWVssVM1kZyBR0FJLTAYMpSYh9i7Xcwb8QaoZNQ3.ia14XfQPOB6d_lTzWhwd
 72N.kLvH.HRDGZlio_Hx0qX6HXOuk.M70qrJWGO.E.62LCNNgrtNS7v8LHGEAo9bvTBnDSQVRPbb
 fGrtXI9y2ZQpoPYLj1mj6jyToJju4f2jbsIVjxCPcJYnSCiyVrI3zbSBNtIkQf_3Ek6lL96KsVto
 tIEJ4R6eMvS7XmCB_jC6HgyFcqK9sZzM2VMoekL8_Ljae2JJ7Nle3ROLvXat5ZNz0KhUc_X2V6I.
 fZLv_JdRcMxNH6a8G7iY4qUmhQALx7W2iVLLDoNNz6xmgB_QnBjY1JVgpDdjlzW6eT4IPdJAUHBV
 vto1O.X4UOIgWYtYYzxjDDMTBo_0B6zjF5W9EOm3Ht.ecGaEpNMKsTz4f0qR0KrfOw_Ml3UD39xW
 KiLcWWsfxdXzSS2zSJnNuEdnX8hymcfxe38GvCVhlO66Zgm93sIIFii.DlZ8cYgnLBl.8YeM1IYC
 bwuSBNLZoynElC1YLsxEtYK4pGIzDzlWCS1UUsHybbyQeck2aU2HijsxY0nVinM4facUuQuyfcOv
 K45b3jzRr5TD1LtkqgZdTFwegXHP82pLRyid73iyxEeF1jfN1XleYefcl5lOg6p_NAJwYHpQ.kZz
 ZDo4RFmTjZtO4Dg4f6VXgcqkeFd_3UFPfZ7Ec_F1IJJLTUmxiaxIzjCzL4dw5NX4DVuQUX9p.def
 66SfrzTflyQWut0DewStrY2O714KUrH2LIyJhko1ghZgP2UX122w2NKLSSIJmE.UlT_mC4nB4gES
 1Xju08aR6TefB6Awqq8o6kZxSJTYnhuuM4khU_Cgodl6TNqHwrvwokpmPQFh2Svp_UiTYzaQLjIT
 RKYlJb364egGH0FI_1Q2pZAfjzF8rOI3ErFdOtIoFwYoEv8Wzg8WVZcMcNh4cSq3SUjUNyBztEdG
 l9jXtZ3uqYODhqziGo_nHNZegv134qOdFYivND4KD7IouHBMzcz8LUFfFImhLlc6RNYqxpbcUZ2i
 K6cVXtkQKeQ3n.UFC8FiOBT6rjC_u5ia4GE.DWomAs1hnVEPcPQnE650dSgiO48RYHpO4YY94RIZ
 3IpUwWfIM2BhchoE.j1Gs9B81DzTBOEoaowZDF_BVCLe1caDR1UWANzp5I1eoQwjAIz__PEiVm.l
 DXAVqzZkFc7kzYrCPlFdKQ4gwTTlx5hi.J01b4Tw_CuOewxHtv6dNbtEKvZYKG5KckmFnYUf1aml
 n4GaXVrxLnrey0fZKiDlKAlsz1MIlR58xCv8fwrD0JvvRdb.vQ8kO4M0RnK6QfGpij4GnLZzICjJ
 1v6CXoUDGztyX2E.VlpWHmc2Jh3bl1nloqIthXFV87GHzQdgS1N4ptKmPszcfi4Khy672e3pKC2y
 ftiT5Vy9SCfLIQBTatpwcY2fjBfNIMIoK.0Ih78C0Gu.uS_n7ZDEPum9cGsWDWzVBdq.E9.0YUBA
 bPT7AohTbvAnzWYvgoDWXPenEK8QFdnWN7F0QCbCKlLCL1mlp6eI89_NsoPAkOLj6yEd1g9vAT2W
 ecaxmfHWn72hdCFKqnrf_qZkFWx9R0TmRE4IzBTtxonn_UJTunbmfTPdAqJV_LTEWQVd1BTcDOSz
 3xs6NlAgaMZ2qrxPtboBrBjE.9SsPj.NNF1M.TOuZdhH7LTVagSAGAKBfjxXhogaQR0UJjRUY0uQ
 I0GfNcNFsVWSZJhT4O.l8BGQHtajaiQXX4JJDW52EooyUouErBc59Q3gDwsGVzaCqvH6cpsc03ON
 I5HFlFBLx6JWv4rWmIALiZ1R9rmRYDZmcwSHS_D31QsrFOHg6KDIhS6yyEsljRknGztf5Yc_nqZt
 5EPGzxab1EawskzQA7im_9TNvgW8t1vsuBlsxUZ.3Vss0GrtuseZAGUWyJx0uCc4U.QWGbfIS7mZ
 lc4hxlLhKw6VA3FVb8SSGtezv4vhfIWmXQyxax_MxhbwqSssKbUug3zZXr_XuIgHaSqqnYSqm6L9
 g2d2V24Noljs9MBOVOiHRt2Lk1KwZe_6Vc381gXgggAeAMY6YArS0Hp0ETnSK6VLJoNGZ5MlhK4B
 iAiJ9c_URdKmuuihfgb3I7o2gD5vA25qSjh_j8tfc.jbu0xpXXUW8LwBqJ__44DRIG.e3kBFTcgD
 uN8ecKQcMXkW3gg4BCqvZQj6V0hvljHWRk9rcj4tWLnv1OKxJ6Xk.rFRzymUpl7Y.ZhyDoLbdVhj
 DQ8rv1SxbKJuoh3QaYs7u0xtUynrSeg7lDMJbzaSc5f1uPbcPBW5g50v0r0ClmFnFRJdQHuao_Nn
 ZAnEXA1UMEMLsc3VtUKixgqhqpzTEZoHbwgMdb8pSVemCcHEbsGS_N2UfZb62ivHbJcNOnT0XKA-
 -
X-Sonic-MF: <pjp@fedoraproject.org>
Date: Tue, 28 Sep 2021 11:39:16 +0000 (UTC)
From: P J P <pjp@fedoraproject.org>
Reply-To: P J P <pj.pandit@yahoo.co.in>
To: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@redhat.com>, 
	=?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= <berrange@redhat.com>
Cc: "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, 
	Thomas Huth <thuth@redhat.com>, 
	"Michael S. Tsirkin" <mst@redhat.com>, 
	Markus Armbruster <armbru@redhat.com>, 
	Paolo Bonzini <pbonzini@redhat.com>, 
	Eduardo Habkost <ehabkost@redhat.com>, 
	=?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <f4bug@amsat.org>, 
	Eric Blake <eblake@redhat.com>, 
	Richard Henderson <richard.henderson@linaro.org>, 
	"qemu-block@nongnu.org" <qemu-block@nongnu.org>, 
	Peter Maydell <peter.maydell@linaro.org>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <1043465112.1767719.1632829156550@mail.yahoo.com>
In-Reply-To: <798304472.4432617.1631626227208@mail.yahoo.com>
References: <20210908232024.2399215-1-philmd@redhat.com> <YTnhy/Wo5M1qVs1k@redhat.com> <798304472.4432617.1631626227208@mail.yahoo.com>
Subject: Re: [RFC PATCH 00/10] security: Introduce
 qemu_security_policy_taint() API
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Mailer: WebService/1.1.19043 YMailNorrin
Content-Length: 2683

On Tuesday, 14 September, 2021, 07:00:27 pm IST, P J P <pjp@fedoraproject.o=
rg> wrote:
>* Thanks so much for restarting this thread. I've been at it intermittentl=
y last few
> months, thinking about how could we annotate the source/module objects.
>
> -> [*] https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04642.htm=
l
>
>* Last time we discussed about having both compile and run time options fo=
r users
> to be able to select the qualified objects/backends/devices as desired.
>
>* To confirm: How/Where is the security policy defined? Is it device/modul=
e specific OR QEMU project wide?
>
>>> it feels like we need
>> 'secure': 'bool'
>
>* Though we started the (above [*]) discussion with '--security' option in=
 mind,
> =C2=A0I wonder if 'secure' annotation is much specific. And if we could w=
iden its scope.
>
>
>Source annotations: I've been thinking over following approaches
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
>1) Segregate the QEMU sources under
>
> =C2=A0../staging/ <=3D devel/experimental, not for production usage
> =C2=A0../src/ <=3D good for production usage, hence security relevant
> =C2=A0../deprecated/ <=3D Bad for production usage, not security relevant
>
> =C2=A0- This is similar to Linux staging drivers' tree.
> =C2=A0- Staging drivers are not considered for production usage and hence=
 CVE allocation.
> =C2=A0- At build time by default we only build sources under ../src/ tree=
.
> =C2=A0- But we can still have options to build /staging/ and /deprecated/=
 trees.
> =C2=A0- It's readily understandable to end users.
>
>2) pkgconfig(1) way:
> =C2=A0- If we could define per device/backend a configuration (.pc) file =
which is then used
> =C2=A0at build/run time to decide which sources are suitable for the buil=
d or usage.
>
> =C2=A0- I'm trying to experiment with this.
>
>3) We annotate QEMU devices/backends/modules with macros which define its =
status.
> =C2=A0It can then be used at build/run times to decide if it's suitable f=
or usage.
> =C2=A0For ex:
>
> =C2=A0$ cat annotsrc.h
>
> =C2=A0#include <inttypes.h>
>
> =C2=A0enum SRCSTATUS {
> =C2=A0DEVEL =3D 0,
> =C2=A0STAGING,
> =C2=A0PRODUCTION,
> =C2=A0DEPRECATED
> =C2=A0};
>
...
>
>
>* Approach 3) above is similar to the _security_policy_taint() API,
> =C2=A0but works at the source/object file level, rather than specific 'st=
ruct type' field.
>=C2=A0
>* Does adding a field to struct type (ex. DeviceClass) scale to all object=
s/modules/backends etc?
> =C2=A0Does it have any limitations to include/cover other sources/objects=
?
>
>* I'd really appreciate your feedback/inputs/suggestions.


Ping...!?
---
=C2=A0 -P J P
http://feedmug.com


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 11:51:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 11:51:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197941.351198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBdl-0006Vx-5H; Tue, 28 Sep 2021 11:51:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197941.351198; Tue, 28 Sep 2021 11:51:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBdl-0006Vq-0u; Tue, 28 Sep 2021 11:51:17 +0000
Received: by outflank-mailman (input) for mailman id 197941;
 Tue, 28 Sep 2021 11:51:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBdk-0006Vk-G5
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:51:16 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBdk-0006sV-70
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:51:16 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBdk-00007J-64
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 11:51:16 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVBdg-0003c9-Mt; Tue, 28 Sep 2021 12:51:12 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=DoGyYLfie1QcWbgQl4SFVLtP4Lq+qwXggX3/2jWI3sk=; b=rCGAX9UDxYxB7B/vauiqQOdUuH
	AV/FlqEY9dqcxLr+94ssWPlQh0FMqZCD3eSfV2gj+QIOGnzWhU9tIMMYXZU1Fxpa4J4YVtB9q6Kjo
	w8IjkPGP8QYuUmV2x8XSL3qPh66Evgaj/tWyWyFBrgz+dc4tw2X/G5wndh353whVxR04=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24915.431.175302.559408@mariner.uk.xensource.com>
Date: Tue, 28 Sep 2021 12:51:11 +0100
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: <xen-devel@lists.xenproject.org>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    George Dunlap <george.dunlap@citrix.com>,
    "Jan  Beulich" <jbeulich@suse.com>,
    Julien Grall <julien@xen.org>,
    "Stefano  Stabellini" <sstabellini@kernel.org>,
    Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] Config.mk: update OVMF to edk2-stable202108
In-Reply-To: <YVLBgcaFp/CzDgms@perard>
References: <20210831123637.22647-1-anthony.perard@citrix.com>
	<24878.13708.826632.607786@mariner.uk.xensource.com>
	<YVLBgcaFp/CzDgms@perard>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Anthony PERARD writes ("Re: [XEN PATCH] Config.mk: update OVMF to edk2-stable202108"):
> On Tue, Aug 31, 2021 at 02:58:36PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[XEN PATCH] Config.mk: update OVMF to edk2-stable202108"):
> > > Update to the latest stable tag.
> > 
> > Thanks.  I am OK with this but I think we should hold off committing
> > it until the XSA fallout has been sorted.
> 
> Hopefully, this is sorted now. Time to commit the patch?

Well, things are still not good, but, yes, I have committed this.

git seems to have auto-merged this successfully with the Mini-OS
version update.

Ian.


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:02:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:02:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197956.351216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBos-0008P6-IN; Tue, 28 Sep 2021 12:02:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197956.351216; Tue, 28 Sep 2021 12:02:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBos-0008Oz-FR; Tue, 28 Sep 2021 12:02:46 +0000
Received: by outflank-mailman (input) for mailman id 197956;
 Tue, 28 Sep 2021 12:02:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBor-0008Or-8w
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:02:45 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBor-000766-5D
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:02:45 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVBor-0001cJ-4E
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:02:45 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVBon-0003dw-HD; Tue, 28 Sep 2021 13:02:41 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=pB5F4F+2HGAVnLSMVF8dXjxc9fgS3AtcWQAVWtHyk2I=; b=DtFYUiYl/T4QJdtTb5DXVrfGWr
	DRD/TX9KCq5+MWr8zplM3Vi294ZEOLuvFVyoTV8gxdrtRgVyIFfMyPCiu7ypvrcwFZGIyfFxhZel9
	/YIx78lOW2CQFWnIeEC1jqEqXwxYMa9rkYKU+DMjcfl0WtY6DGDnzntWSFUgqSvAdWzE=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24915.1121.8356.288414@mariner.uk.xensource.com>
Date: Tue, 28 Sep 2021 13:02:41 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for xenstored
In-Reply-To: <20210928091517.9761-3-jgross@suse.com>
References: <20210928091517.9761-1-jgross@suse.com>
	<20210928091517.9761-3-jgross@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("[PATCH v5 2/2] tools/xenstore: set open file descriptor limit for xenstored"):
> Add a configuration item for the maximum number of open file
> descriptors xenstored should be allowed to have.
> 
> The default should be "unlimited" in order not to restrict xenstored
> in the number of domains it can support, but unfortunately the kernel
> is normally limiting the maximum value via /proc/sys/fs/nr_open [1],
> [2]. So check that file to exist and if it does, limit the maximum
> value to the one specified by /proc/sys/fs/nr_open.
> 
> As an aid for the admin configuring the value add a comment specifying
> the common needs of xenstored for the different domain types.
...
>  	echo -n Starting $XENSTORED...
> @@ -70,6 +89,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
>  	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1
>  	XS_PID=`cat @XEN_RUN_DIR@/xenstored.pid`
>  	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
> +	prlimit --pid $XS_PID --nofile=$XENSTORED_MAX_OPEN_FDS

Thanks for this.  I have one comment/question, which I regret making
rather late:

I am uncomfortable with the use of prlimit here, because identifying
processes by pid is typically inherently not 100% reliable.

AIUI you are using it here because perhaps otherwise you would have to
mess about with both systemd and non-systemd approaches.  But in fact
this script "launch-xenstore" is simply a parent of xenstore.  It is
run either by systemd or from the init script, and it runs $XENSTORED
directly (so not via systemd or another process supervisor).

fd limits are inherited, so I think you can use ulimit rather than
prlimit ?

If you use ulimit I think you must set the hard and soft limits,
which requires two calls.

If you can't use ulimit then we should try to make some argument that
the prlimit can't target the wrong process eg due to a
misconfiguration or stale pid file or soemthing.  I think I see a way
that such an argument could be construted but it would be better just
to use ulimit.

Ian.


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:05:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:05:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197963.351227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBrp-0000dB-0B; Tue, 28 Sep 2021 12:05:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197963.351227; Tue, 28 Sep 2021 12:05:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVBro-0000d4-TY; Tue, 28 Sep 2021 12:05:48 +0000
Received: by outflank-mailman (input) for mailman id 197963;
 Tue, 28 Sep 2021 12:05:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVBrn-0000cy-JF
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:05:47 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 68d6b92c-2054-11ec-bc85-12813bfff9fa;
 Tue, 28 Sep 2021 12:05:45 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2051.outbound.protection.outlook.com [104.47.9.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-2vWm0Cy2PyKCTiKHQzYvZg-1; Tue, 28 Sep 2021 14:05:43 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2334.eurprd04.prod.outlook.com (2603:10a6:800:29::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 12:05:41 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 12:05:41 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR0P264CA0260.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 12:05:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68d6b92c-2054-11ec-bc85-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632830744;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OVk5ot41PxQFmAqjFPSjiuilqpeppXXrDTGViWAZDr0=;
	b=jl60jKpzGeCtWnFpFYG57YTktK70pUaK8HsJNjmb0oDbmSgysi0cIYjJ0XZO6hDLi08dwx
	qH2vYcakT29opLAvAdO8/7UMUx7OenkCM4UwCb4RaF59gZ1CKg15zJxuKMrzeP6d8dNXld
	YEAHyG4lH6qytNIikzyEepxd/iEwj98=
X-MC-Unique: 2vWm0Cy2PyKCTiKHQzYvZg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cZfD9QWI4BGkgo3z7EqK2IRrA+LMnL2hIHFnHd89N2ctpn0jwME4BCBjg0PCzo7gZ+YunAe8texl/LiS9xlQnC9IdtQGWKYFwLqvsZY/1z/JS8SRungzeTcUnu2U7ou/xRMZE1O2l+/CI190GWjSSlfn9FPxdqyrlnBscywPxCCKjZwq/yvebXun2puRo4aNlHJ1v0YYDRl4MCyMlBAp37MaQEILDIaapyE/Sj4jPnSiZbnMQXiHEGTtnlyE4iv6blWT0ysYS6j8ZLkx1tkNj+VddCN8FaIyqh7wAXbyFmbfJLaIpJ7ZNwOAC3QOoY3TJZmVuJrBdO8jRqrjzERxLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OVk5ot41PxQFmAqjFPSjiuilqpeppXXrDTGViWAZDr0=;
 b=AXl/CrbKe/QAX35qtmVNb1pNPGNIxy4R6TaUwMKmZT72I8z4ObAR0fIm23ppFZdB3RgHfuMB/TQfcmdPS2nBMNfBA5MQOAfNIljL3s9Sq25P+jtQe/AV2DnElmt39uyBehcLZzCU1pCZStWeQZJ7caZruxSCmHbNK/Ot8eaDD/LzPavyTaHK4sCBuxWD4LWa/jrsx2z1Nn4Gqr+l84WoLsdEKo/fl3WmdVUS4S7PkHJ0wH2pp1IcFBu9x57pxPW0UtZvHHYMAnYBcC70rzcRbdVjfpx9zkqNrycf07/V2RStCHsANavY6xNztWKeuT1e53BMcpDnBvhOXL9N53YHDA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 01/11] xen: reserve flags for internal usage in
 xen_domctl_createdomain
To: Penny Zheng <penny.zheng@arm.com>, xen-devel@lists.xenproject.org,
 sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com, Julien Grall <julien@xen.org>
References: <20210923031115.1429719-1-penny.zheng@arm.com>
 <20210923031115.1429719-2-penny.zheng@arm.com>
 <7da75d7c-5640-305d-d742-d64b8c219949@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <60d813f7-a756-172c-999d-44f81504e268@suse.com>
Date: Tue, 28 Sep 2021 14:05:39 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <7da75d7c-5640-305d-d742-d64b8c219949@xen.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR0P264CA0260.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100::32)
 To VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f305928e-7e1c-4d0f-2d05-08d982784ae1
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2334:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB23343AD63172BFBF1D3D2D8FB3A89@VI1PR0401MB2334.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gwIgxrm0xc2WExv4tAAYq33BbafvzqGRpp+u6viI80ZvdsX2QGJL4JYlUcVNoRbaKqjLa9GwYhT05pvz8RbWFdOTea+ntys0lWBGNOmy4ZnaT2BnvwvQYpp+NPid3bWVTwzEBZSWRVFuMcFuGNHPkavRcq5NTom4fbGptJ5hI9fQ/xi26YfswYKGpyCvutZQdIXq5lc7PkGiCi3VNft/MNECS0SYpjhuHNYU4crAT19Iw3zlW0nUKjTH2MJPjnSbg+/7KxGjGun6YYravouVNIPEDWuJqn520SGQFFsT8DcyfcDczhDZqm8sSvlKlGZbMB9Kj2lXJI9fQS+k5i6HJr7gZ2ePMCL+GE/u7I6Gf7dP+uCzPhbSmi3YwKMDPMkIYODkqWorzYZBWVMg53iVrGM82u8vIb/gJ6OpovTyQY7kWPeoe407FtuUVDrlpXiFIu5XMVxK+13FCZ+90hEPFkZ2f7q4p9rouLiG7RojsCRaJUBc6hxC8nRI4W37cRhxsl9VxmCJKKThctUOhKvW22s31R1vMGbYlhx/QbAkltDdtps8QD9olUziuv+T85AaNb4dXmgyVfz2lXq7xMzQSpZLt/ww/f0bzrVyYP9PMoZax11dVCBpqgeAcIABWG0QE/+6U4fyKjoppncCSsS349inSHbE3IHU7rC+A/DIwNKtFQU5k5yV9BFYz36cWJWbenyN2Qjw8okokuLeIHgpANY/LegdDtmTZaRgvgDbI0Y=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(53546011)(508600001)(86362001)(5660300002)(956004)(2906002)(2616005)(4744005)(31696002)(4326008)(6486002)(16576012)(38100700002)(26005)(8936002)(316002)(66476007)(31686004)(66556008)(186003)(66946007)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VDdzS0M1WmkyU255Q2JVOUNuaDZMdzRlOGlTRWd1Q0dCUStxVXVSYlR4d0wr?=
 =?utf-8?B?VVVDSU85TC81ZTExS3hnWXdvVEJKYmZMei95anJkcUpPR0QwZFN5V09DVGhH?=
 =?utf-8?B?Z3BoTmdXcnkxdGxFeExPK2xpaHpXNC96NmlDN0x0VkEzTVRPVVBXRk40ajQx?=
 =?utf-8?B?OGFhY0Y1WG9zVUw4RFpzYjdwVWs2dkdSa0U4NnFsMHVDaUV1TmpyeDIzVEx3?=
 =?utf-8?B?ZFRmd0FIcEdBU3J2cW5QSnVTWjA3ZWtZZ3FLRHg4U1ZnczUxL1FuREk2SUhS?=
 =?utf-8?B?QzUwQTN6a0dOTnEwRXlLNW1QdzNLeWxVUFV3d0RzTW5qb0VjUU1Cb2FqK00r?=
 =?utf-8?B?U3BDOCt3RW1oWHhuSmt6Q1ZxbFlHWWxSNDIyTXZMNWkwcU1ITDZsYWN1Rm9r?=
 =?utf-8?B?WnNxYWlmcDJrMHVmRlowS3FZc2lTOHY3VnlxbVh5ZUlpcTJQeVJEU0NRcXZ6?=
 =?utf-8?B?T3FodnA5cEgzMnk0cmJFT0xyWnJRcjlIakJ2VHR3S2VLemJQdGhXTUtxczZU?=
 =?utf-8?B?WVk3bjF6UlBvU2ZmcUx3R1MyMjJBQktGa0tXZDBSV3pmVjNkbEQ2VVY1dGox?=
 =?utf-8?B?OE9nc0VDdmNFSzh2RU43TFdhQ0tJaTJ6NWV6ZzZGNWJ5RWp5eVJURmFBOGRt?=
 =?utf-8?B?SHY4djJYVGowbVQ1VkNoZkZCOHlQYVFrZFRtbEEwbldJdVdEb044V1pNTktK?=
 =?utf-8?B?SVhCSEx6YXJtb3k3UUxKWlA2bXlObm1JY2pNRGxsS0JRZFdiaW4valp6UGtG?=
 =?utf-8?B?N2NzSVNERkVDL0JSNjBEWXM0UHYxVElISk9tancyWVQ3L0FsVFZmZVNGdVhw?=
 =?utf-8?B?dEdDRnBDWEJsYW9iSHB3d3hZcmkvcnVsUyt1SGJMYjhBQVltelRqNHhMTHZX?=
 =?utf-8?B?VUpXSXpFMnFENkdjZHJBck5BY3hGdCtZYnEvcjhGVlhWbzFHbmhiTVUzOFls?=
 =?utf-8?B?ZklkeXYwYzl2cDBGV1B5Nk1RYnNzc1NITlFmU2tjU3k2Ny9DSEx6M29nTTB0?=
 =?utf-8?B?QVI1dE9iM2svTytMY0hnTW9sOElqK2RSd0Y3QjF0RWxPNlJHT1FFdUJvUWFE?=
 =?utf-8?B?dUtzTC80SEFHbHlKVzRtbkRkN1cvYnh5U0ZCZkhxQmFTcTZrOVd6a2hOWU05?=
 =?utf-8?B?ejFxZWwxSDZuVVVjSGRQVW5LaXFWV0NLU29qTE55SHlac2VaSVdZMzlFSCs0?=
 =?utf-8?B?YVBSb1FYRGtuNDZaNkJIYUdtMjhaUTVrYndPdGFTRDBHa3Z3c3dMY0xFNWw0?=
 =?utf-8?B?MjdqS1BJTFplUi9ldmIzK1RGNkdrSjFKdWYwTTg5N1pETXJOMzZqWExLRk16?=
 =?utf-8?B?cHF5RERidUp1ZXBId1B0UUhRaGdNd0VRZ2VwbGlUaDIwbytYeW5hRVc4UG1C?=
 =?utf-8?B?MDFGdlZJdnJjWFh1RHJpTUFxczJ2Tm5CZ3BvL0tXMEtQQjdtSkR2YVRZTTh3?=
 =?utf-8?B?M2xVaEc4UC9sdSthWlJaS25vMHkwNEFOY3NwNU5WTStCNXB3NERRcEIrRnAz?=
 =?utf-8?B?ODI3aForRXZOOWpKQUx0OUdyejdoMU5MdHlWc1pJd2sxZ0JFWkRUTDJDRlJo?=
 =?utf-8?B?azVVRDhqZmF2Mm9PbldKZFFYdlcydENBUFRXS2ZNL1JLcDdoQzNRS2tZTitR?=
 =?utf-8?B?UWNudkgwaXo1Z1c4dDVvSlFTTklqSzFwM0dJZWs5YTZVS0FKM3p1ZlpiTGFR?=
 =?utf-8?B?TThKSmM1bzE4U3hOQURaQS84NnF5RklycTZ3UUNzQitiRW5hbCtsb1BMTUx0?=
 =?utf-8?Q?ntZyBure7mGITyM+dCbIs0Xh0OiWE64A2f+raDR?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f305928e-7e1c-4d0f-2d05-08d982784ae1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 12:05:41.5362
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: snkNe2fw3KAJVHnNbmCqiMOALWh2PZVai/RJp7pBb5ZH1bd6SZUmyKo8Yn8J7YZ8PXQPolUk1m5ILUcimbAx4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2334

On 23.09.2021 11:54, Julien Grall wrote:
> On 23/09/2021 08:11, Penny Zheng wrote:
>> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
>>
>> We are passing an extra special boolean flag at domain creation to
>> specify whether we want to the domain to be privileged (i.e. dom0) or
>> not. Another flag will be introduced later in this series.
>>
>> Reserve bits 16-31 from the existing flags bitfield in struct
>> xen_domctl_createdomain for internal Xen usage.
> 
> I am a bit split with this approach. This feels a bit of a hack to 
> reserve bits for internal purpose in external headers. But at the same 
> time I can see how this is easier to deal with it over repurposing the 
> last argument of domain_create().

I actually have trouble seeing why that's easier. It is a common thing
to widen a bool to "unsigned int flags" when more than one control is
needed. Plus this makes things needlessly harder once (in the future)
the low 16 bits are exhausted in the public interface.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:15:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:15:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197970.351239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVC1Q-0002G5-Va; Tue, 28 Sep 2021 12:15:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197970.351239; Tue, 28 Sep 2021 12:15:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVC1Q-0002Fy-SQ; Tue, 28 Sep 2021 12:15:44 +0000
Received: by outflank-mailman (input) for mailman id 197970;
 Tue, 28 Sep 2021 12:15:43 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TiGH=OS=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVC1P-0002Fs-Fz
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:15:43 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc92a241-2055-11ec-bc86-12813bfff9fa;
 Tue, 28 Sep 2021 12:15:42 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6789820202;
 Tue, 28 Sep 2021 12:15:41 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3CFEB13A82;
 Tue, 28 Sep 2021 12:15:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id AaQgDW0HU2EhfwAAMHmgww
 (envelope-from <jgross@suse.com>); Tue, 28 Sep 2021 12:15:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc92a241-2055-11ec-bc86-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632831341; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=NUNiFLtAHLJUOzZDI5flM+QhBET7pJWEcR1eZ9Iuzto=;
	b=ML6ZW73YQdOWb6vihX4NED7+SKLZtlxQbN5DzB0E4UnV+JFPqB0EjK+NUviaxNP0aCeXlA
	niGLFJHf9jda3/CwauSg4fv/XYW9pIbAvcBTDgzPBusSatuOP9qd7UrjxjXVmgZa/kNGsY
	LFGzQWSM0Nq4E+jmkk65C/nvun9tPgA=
Subject: Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20210928091517.9761-1-jgross@suse.com>
 <20210928091517.9761-3-jgross@suse.com>
 <24915.1121.8356.288414@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
Date: Tue, 28 Sep 2021 14:15:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24915.1121.8356.288414@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="d0wtOka4OxLM2Z4Gskn4hq5a1mmazPunv"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--d0wtOka4OxLM2Z4Gskn4hq5a1mmazPunv
Content-Type: multipart/mixed; boundary="JkvvaZn0wjfzVLkfML8Y2dXLPDLdgBca1";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Message-ID: <0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
Subject: Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210928091517.9761-1-jgross@suse.com>
 <20210928091517.9761-3-jgross@suse.com>
 <24915.1121.8356.288414@mariner.uk.xensource.com>
In-Reply-To: <24915.1121.8356.288414@mariner.uk.xensource.com>

--JkvvaZn0wjfzVLkfML8Y2dXLPDLdgBca1
Content-Type: multipart/mixed;
 boundary="------------7AA3DD2ABEF0A7F217775343"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------7AA3DD2ABEF0A7F217775343
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 14:02, Ian Jackson wrote:
> Juergen Gross writes ("[PATCH v5 2/2] tools/xenstore: set open file des=
criptor limit for xenstored"):
>> Add a configuration item for the maximum number of open file
>> descriptors xenstored should be allowed to have.
>>
>> The default should be "unlimited" in order not to restrict xenstored
>> in the number of domains it can support, but unfortunately the kernel
>> is normally limiting the maximum value via /proc/sys/fs/nr_open [1],
>> [2]. So check that file to exist and if it does, limit the maximum
>> value to the one specified by /proc/sys/fs/nr_open.
>>
>> As an aid for the admin configuring the value add a comment specifying=

>> the common needs of xenstored for the different domain types.
> ...
>>   	echo -n Starting $XENSTORED...
>> @@ -70,6 +89,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons &&=
 . @CONFIG_DIR@/@CONFIG_LEAF
>>   	systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED =
|| exit 1
>>   	XS_PID=3D`cat @XEN_RUN_DIR@/xenstored.pid`
>>   	echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj
>> +	prlimit --pid $XS_PID --nofile=3D$XENSTORED_MAX_OPEN_FDS
>=20
> Thanks for this.  I have one comment/question, which I regret making
> rather late:
>=20
> I am uncomfortable with the use of prlimit here, because identifying
> processes by pid is typically inherently not 100% reliable.
>=20
> AIUI you are using it here because perhaps otherwise you would have to
> mess about with both systemd and non-systemd approaches.  But in fact
> this script "launch-xenstore" is simply a parent of xenstore.  It is
> run either by systemd or from the init script, and it runs $XENSTORED
> directly (so not via systemd or another process supervisor).
>=20
> fd limits are inherited, so I think you can use ulimit rather than
> prlimit ?
>=20
> If you use ulimit I think you must set the hard and soft limits,
> which requires two calls.
>=20
> If you can't use ulimit then we should try to make some argument that
> the prlimit can't target the wrong process eg due to a
> misconfiguration or stale pid file or soemthing.  I think I see a way
> that such an argument could be construted but it would be better just
> to use ulimit.

Hmm, maybe I should just use:

prlimit --nofile=3D$XENSTORED_MAX_OPEN_FDS \
    $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS


Juergen

--------------7AA3DD2ABEF0A7F217775343
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------7AA3DD2ABEF0A7F217775343--

--JkvvaZn0wjfzVLkfML8Y2dXLPDLdgBca1--

--d0wtOka4OxLM2Z4Gskn4hq5a1mmazPunv
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFTB2wFAwAAAAAACgkQsN6d1ii/Ey82
QQf/Ux//Nw4xu2jJNrGCIvY5KKLz1vsVwXgxJf3lhNt3idaxHf2eJYVqkCzFlKw1J5pbaYon4jeK
XhrvNOK9Yktz14BhDMXk+bCxEDAZLxEV0mtebuVb8zwEZJ+51d0t5zr5Fj6DdiLVSlSeFMiozre3
7coJmCj/u3A5Ay9QWK2336GyzSaQHYhOA3+Kfb+EyvCeyEmhD8KrAcm85a/czK93z3ywrp1f5kTM
FqTT7Bm0+CIk9qC+l9W118xnkKw6KsRNOOXRvzHu1xJmT8RKT2u0kkaS7uQfgaLHmgquOu8tD6ia
VWhv4xT4AfPvRX6N5Qj+nv4F8fPGHo4azQ8lLdZDaQ==
=DtiI
-----END PGP SIGNATURE-----

--d0wtOka4OxLM2Z4Gskn4hq5a1mmazPunv--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:33:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:33:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197980.351250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCIN-0004n4-Hc; Tue, 28 Sep 2021 12:33:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197980.351250; Tue, 28 Sep 2021 12:33:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCIN-0004mx-Ed; Tue, 28 Sep 2021 12:33:15 +0000
Received: by outflank-mailman (input) for mailman id 197980;
 Tue, 28 Sep 2021 12:33:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IxrP=OS=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVCIM-0004mr-Er
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:33:14 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.14.73]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0838a0c6-605f-4b3c-979a-9bfe50561f81;
 Tue, 28 Sep 2021 12:33:11 +0000 (UTC)
Received: from AS9PR06CA0268.eurprd06.prod.outlook.com (2603:10a6:20b:45f::19)
 by PA4PR08MB6127.eurprd08.prod.outlook.com (2603:10a6:102:f3::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 12:33:09 +0000
Received: from VE1EUR03FT060.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45f:cafe::91) by AS9PR06CA0268.outlook.office365.com
 (2603:10a6:20b:45f::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Tue, 28 Sep 2021 12:33:09 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT060.mail.protection.outlook.com (10.152.19.187) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 12:33:09 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Tue, 28 Sep 2021 12:33:08 +0000
Received: from d8915662b8fc.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 695C1361-75E3-47B5-8277-5AFB596AE31E.1; 
 Tue, 28 Sep 2021 12:32:55 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d8915662b8fc.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 12:32:55 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6528.eurprd08.prod.outlook.com (2603:10a6:102:15a::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 12:32:54 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Tue, 28 Sep 2021
 12:32:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0838a0c6-605f-4b3c-979a-9bfe50561f81
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kMe0gzp8UMKJoHftubjUl2uIKsSHCR1TAN8fJuJayb8=;
 b=7dY5aT5htLa/u4OpmTzEjLnCDkfQAMUwIBPO4z9mliiU+HZlO7OwIz6O/nvVPMo0+SrhxbPTIIJdRICs2GpIIqhMxk6yJtyAMiV5RShHOHMfvNV62o9SMHwMVhLOWfF1ccWl1W/SVnMMCBq2q4irfFQdsCthixCNyL1l2M2G1hc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c32aa90628cbd2ff
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fc1KbGwYSQuS5L1/gHVcXXNgRO/wuoypKTKtK1NlxondqirTAQlJLY77FCtBP94NLaZ7ws8rWRHFJTyzBmnwerk87vYYK6avIlOsTfIzzCwaU49g1OLcSFY3LTU5+iGVG5FyL/BabEJieLHAHsuZb/w2Q7TpdlOybII+Shb+7TRIBWa50Kh0W+WyiFbNPS9lbs4Z86VySrlLMmp59wKuQocp5kbULhNNcGocXbbdNnb9GZHXzXDJA2cmpkCwFMWyeWZRYZQCSCcwaSYFsBrylZD241g66P+CVuUhBK7oVUpg5rURYAIVg/crSJtEGF3LZWPdVleybNj67KVX/FvwfQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kMe0gzp8UMKJoHftubjUl2uIKsSHCR1TAN8fJuJayb8=;
 b=fDWF+at0EBmfTK8i0LoX2A4o9bd3Ep0BDjB4nQ4igo4M6LY/TbBhWQizdbRE/RwfxHwiMSzC5+QjqTHRUfOpiCy5l3Sgofb6MkTWwYmHhzG6oGabYGNRzzNDoDttAwGsg1RnGutzSBmPogz8PaBV6V2/ePxyT+s8tEQJ+HGO9aIMzDa+Rdtj0BQD/puJsjLjs5gAdFWabZQnqHVwZgKGFBPp/iSrwfFjmapTiYZoNZNXDZol3+TzDv+iY9TaZiqw70kEzj9r0Uk8UgO+y6X26KsXepxcFvNEsj0eTKX60zIL+LB2WrD0oOD3+s0Fslam3CjiDHAbl2qGm96T16FoCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kMe0gzp8UMKJoHftubjUl2uIKsSHCR1TAN8fJuJayb8=;
 b=7dY5aT5htLa/u4OpmTzEjLnCDkfQAMUwIBPO4z9mliiU+HZlO7OwIz6O/nvVPMo0+SrhxbPTIIJdRICs2GpIIqhMxk6yJtyAMiV5RShHOHMfvNV62o9SMHwMVhLOWfF1ccWl1W/SVnMMCBq2q4irfFQdsCthixCNyL1l2M2G1hc=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Igor Druzhinin <igor.druzhinin@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"jbeulich@suse.com" <jbeulich@suse.com>, "paul@xen.org" <paul@xen.org>
Subject: Re: [PATCH v2] pci: fix handling of PCI bridges with subordinate bus
 number 0xff
Thread-Topic: [PATCH v2] pci: fix handling of PCI bridges with subordinate bus
 number 0xff
Thread-Index: AQHXsSQXPb+dzbvL/kOUSxwoDVQ8/6u5Z6IA
Date: Tue, 28 Sep 2021 12:32:54 +0000
Message-ID: <9C3D662F-A902-4F6A-80FD-935D01C3786B@arm.com>
References: <1632474612-9940-1-git-send-email-igor.druzhinin@citrix.com>
In-Reply-To: <1632474612-9940-1-git-send-email-igor.druzhinin@citrix.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 624dacb2-570d-4779-9241-08d9827c211d
x-ms-traffictypediagnostic: PAXPR08MB6528:|PA4PR08MB6127:
X-Microsoft-Antispam-PRVS:
	<PA4PR08MB61271519A175142DE67675E39DA89@PA4PR08MB6127.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 bZzxZSh5w7wvB0+gukW8YPDqDT6xnpnYasD8rOj3u8XpDS/PBtyVFYGzONUib0VCbIkurU2AYOl8GYSqnN/JJLd6+CJI/rAmXhtwkXYnqpSghNpGXSzwDAr838yqIELevuZvKSY6bUA0vusHnCndLKI8Z9aNDBddDzrsnhWRs7eeV1ynNNH3xLCf8c+R+OIa2vDEjnFqmnLNOekrdWq3PA2dSiEAA0kb8FyHWW6e7JavQMWQ4ENStzGBmlUPxoXILQfjQJIz1FGpCRknaQQn67U3wzowNsM4AnQR89om6+gUfl2YkR9UYmZUuQ1tlDZ3BeO4NC5RrT21eAnHr+ZH+78vtRL1YQoQmi6uCrRefpSS9IvDVc1EUykMhr7Hc35xyCYplT64RZ2uuqNazeP0sfS64g7D8sIjEqyjTXmePtta0+1E5liwDsPBxNuxo7wPNAPjKn/cp6cvPwCWfNWxdBNtDLXzWnUZ7nGI/eL1DdndxGo2V9n3Ll+gklA4Jc/OPK0IQWjkj+42+JMoapsFcqjkPQ6O2WbIipGmTuKsGsSNxiZDuyYqBzTc5orNk8foUqa5oLBkaXbT4SbwBx2dJJxO6X7vCFqDvd+SWNrrCLLrjJxYMOidrern82YhJfyQHaUeUi0GYfx4EE+3ghWu1pEpvxAky8/AuOSRYtEs3gYsorT3b7R1lR8cJoChIL3bvMfOSoTLKjuHZZtuI9VEJ/JqT0dsVgayhF3jRUEmR7FhO7IdEnU/nnYZnRjilCxq
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(33656002)(66946007)(2906002)(316002)(66476007)(54906003)(508600001)(6512007)(6486002)(8936002)(8676002)(6916009)(5660300002)(53546011)(38100700002)(4326008)(86362001)(36756003)(76116006)(38070700005)(2616005)(6506007)(186003)(71200400001)(26005)(66556008)(64756008)(66446008)(91956017)(122000001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D4EA74420D02FC4BB00D8A69B7E9D991@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6528
Original-Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2327c71c-c787-48d0-b637-08d9827c1842
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UvB7/Fy3w5FkjGpBrmrh2GxQ5pr4KKOLubbFddUMc22T1DsOsgw6sRsoWzVOCzE8S5nm94yTKOb9Y2CcxU+eDPRY4h8PrkdrvFpvr7Kzei9sb+4Q/RExQk8u2bTvUWcbsHNwVMhjtT0w7C1uWqMO89PJxkhtacoROUl2elJELPN5sOuAy+uc/709Wz004Ha8XdP395MaC1ACoC3C9MKSt3xOb/ExUaJs+/UKOZ/cgsJ5/xRLA6GaMZNtCcpuXdIm6/7G0ylkVA/UPPhsmo5BFgrZ6myyK2GbOyVlpfSeBpg83EsY0wLv8YqdQLlFkG9jowAZbgkXBEKn8L9Ew9UTpK1hyLy/1V8XeBHK4Q7FxjuyhLE0a/d9M1gyb9HvSvbBkJ57i/VAqZ04IqLx+1dtQsV/Bpodk7BEv9FReRlloMfcji20Lur8qOuUc4DXitl/Ta3xBWqmw3QE9Rm7IfeyDLyJlJepSCkvUZuDyfIpb8HDyf80hcx3iD7PeUTbQGLtSvi+W0jsvvjDjKsxzkPoPoTgEbCLRuMLOcEvCm0uFe8f6kdp/pUisQdQYNJFUHlPMpOJh/CJgKBQ+akLleGdxx9xGfaPfW0+kpdiZAGf0H5Xo5lJ/paalZJoqxB4OxI8AKxak8IlHlQs5SGKjVDnDbcHsy+9FHYFSlaGf//lpbik+yjTbXAdH04SQ1HGktBj6Qzo4OVL6GyEKJRBwybO1w==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(82310400003)(4326008)(508600001)(8676002)(26005)(6862004)(186003)(47076005)(83380400001)(336012)(81166007)(6506007)(53546011)(356005)(2906002)(70206006)(8936002)(6486002)(54906003)(36860700001)(5660300002)(2616005)(316002)(36756003)(6512007)(33656002)(70586007)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 12:33:09.1708
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 624dacb2-570d-4779-9241-08d9827c211d
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6127

Hi,

> On 24 Sep 2021, at 10:10, Igor Druzhinin <igor.druzhinin@citrix.com> wrot=
e:
>=20
> Bus number 0xff is valid according to the PCI spec. Using u8 typed sub_bu=
s
> and assigning 0xff to it will result in the following loop getting stuck.
>=20
>    for ( ; sec_bus <=3D sub_bus; sec_bus++ ) {...}
>=20
> Just change its type to unsigned int similarly to what is already done in
> dmar_scope_add_buses().
>=20
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand


> ---
> v2:
> - fix free_pdev() as well
> - style fixes
> ---
> xen/drivers/passthrough/pci.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>=20
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.=
c
> index fc4fa2e..d65cda8 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -363,8 +363,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pse=
g, u8 bus, u8 devfn)
>     /* update bus2bridge */
>     switch ( pdev->type =3D pdev_type(pseg->nr, bus, devfn) )
>     {
> -        u16 cap;
> -        u8 sec_bus, sub_bus;
> +        unsigned int cap, sec_bus, sub_bus;
>=20
>         case DEV_TYPE_PCIe2PCI_BRIDGE:
>         case DEV_TYPE_LEGACY_PCI_BRIDGE:
> @@ -431,7 +430,7 @@ static void free_pdev(struct pci_seg *pseg, struct pc=
i_dev *pdev)
>     /* update bus2bridge */
>     switch ( pdev->type )
>     {
> -        uint8_t sec_bus, sub_bus;
> +        unsigned int sec_bus, sub_bus;
>=20
>         case DEV_TYPE_PCIe2PCI_BRIDGE:
>         case DEV_TYPE_LEGACY_PCI_BRIDGE:
> --=20
> 2.7.4
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:36:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:36:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197989.351261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCLu-0005US-6o; Tue, 28 Sep 2021 12:36:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197989.351261; Tue, 28 Sep 2021 12:36:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCLu-0005UL-2z; Tue, 28 Sep 2021 12:36:54 +0000
Received: by outflank-mailman (input) for mailman id 197989;
 Tue, 28 Sep 2021 12:36:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVCLt-0005UE-0E
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:36:53 +0000
Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 58a6c5fc-84ef-49d0-b521-ac498aea12a7;
 Tue, 28 Sep 2021 12:36:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58a6c5fc-84ef-49d0-b521-ac498aea12a7
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632832611;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=Dcl3xvtyIEjWZIgMG1eaMInfPCnZCg1zs5WFrkQAlGo=;
  b=cI6XsgUmyuzjJw2AMOdgNBReHO9tOtoPqJTNKp9zkIM5qP7oLZSmAMdI
   yfUqERcSUHsYP8qZq/fhl2R5ycP7L3z89nadkyEf+ZacPPnnNMo48TdMJ
   7GLhQoLVE5wlrSc0lhjCIh59P/cAC9TvuA17a5rkDZVTuhRnuSbukE6GT
   k=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: CNImGH9CT0ZjBr4wPVxUaBkR+ZDyBk1Co3nXelbXcP2kW691Fk6eaF6q1+NuEVW0dXKNV1zxMN
 kowoDTSszUG/FJRV9p5L6Fc0B0sZIsPk3pEANQEo+4yDTivM6OthsMd8U0P5K3Y/rCn0q5qZAg
 9wInSwKzY9+TCovSlb4LmDaMANGQ0zZZT9fT+k/SXRT9VkSuUBEwJLfEQezQLWSaWTVg3Vlqiq
 y30UIWRL8NmNmFaeOHFBpG5YIG5wXdZcWvMxLAUbBnJJawEtrgL9qx4JOfwrPtJM1PlkF67+H9
 /RUoHvhgd2eBxTLfaI0sPtK3
X-SBRS: 5.1
X-MesageID: 53819662
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Orbkx6nfsWdocTppWZOLq8zo5gxiIURdPkR7XQ2eYbSJt1+Wr1Gzt
 xIeD2mFPK6LYzPwKthzOtuw9EgEu5PUmtE1GwRq+Hg1FSMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BClVlxJVF/fngqoDUUYYoAQgsA185IMsdoUg7wbdh09Q42YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 M1vto2BZTcTAoGPlPUDTD1kECJ4JaITrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBMDtIIMYvGAm1TzDBOwqaZvCX7/L9ZlT2zJYasVmQK2OP
 ptHN2AHgBLoeQRSOmweDpMExOaupFSvYmFAhEDEnP9ii4TU5FMoi+W8WDbPQfSLWsd9jkuev
 njB/WnyHlcdLtP34SWB2mKhgKnIhyyTcIMJHby79/gvjFiS2nYCIAwfU1e2sb+jlyaWS99Zb
 kAZ5Ccqhawz71CwCMnwWQWip3yJtQJaXMBfe9DW8ynUlPCSuVzAQDFZEHgRM7TKqfPaWxQy9
 FyYrv/ZPwBD87GEV225yrSXqyu9bH19wXA5WQcISg4M4t/GqY41jw7SQtsLLJNZnuEZChmqn
 WnV/XFWa6E7yJ5TjvrnrTgrlhr2/vD0ohgJChI7t45Pxi1+fpLtQ4W15VXB4f9EIe51pXHa5
 yNcyqByAA0UZKxhdRBhos1RQNlFBN7faVUwZGKD+bF6rFxBHFb5IehtDMlWfhsBDyr9UWaBj
 LXvkQ1Q/oRPG3ChcLV6ZYm8Y+xzk/O6Soq1DKmOMoMVCnSUSONh1Ho1DaJ39zqx+HXAbIllY
 cvLGSpSJSxy5VtbIMqeGL5GjO5DKtEWzmLPX5HrpylLIpLEDEN5vYwtaQPUBshgtfvsiFyMr
 753apvboz0CAbaWSnSGruYuwaUicCFT6Wbe8JcMKIZu42NORQkcNhMm6el6JtM5z/gLzLegE
 7PUchYw9WcTTEbvcW2iAk2Popu0NXqmhX5kbyEqI3iy3H0vPdSm4KsFLsNldrg77u1zi/VzS
 qBdKcmHB/1OTBXB+igcMsah/NAzKkzziFLcJTehbRg+Y4VkG17D9Oj7c1a97yIJFCe265cz+
 uXyygPBTJMfbA1+F8KKOum3xla8sCFFyuJ/VkfFOPdJf0Do/NQ4IiD9lKZvccoNNQ/C1n2R0
 APPWUUUouzEookU9tjVhP/b89f1QrUmRkcDRjvV97e7MyXe71GP+44YXbbaZy3ZWUP15L6mO
 bdfwcbjPaBVh11NqYd9TepmlPps+9v1qrZG5Q14B3GXPU+zA7ZtL3Taj8lCsqpBmu1QtQesA
 x/d/9BbPfOCOd//EU5XLw0gN7zR2fYRkzjUzPI0PESlu3MnoOvZCR1fb0uWlShQDLppK4d0k
 +4utfkf5xG7lhd3YM2NiTpZ9jjUI3ENO0n9Wkr23GM/ZtIX92x/
IronPort-HdrOrdr: A9a23:PZV2Ea8U6qoMJZNhPoduk+DcI+orL9Y04lQ7vn2ZLiYlFfBw9v
 re+MjzsCWetN9/Yh0dcLy7V5VoIkm9yXcW2+cs1N6ZNWGN1VdAR7sC0aLShxHmBi3i5qp8+M
 5bAs1D4QTLfDtHZBDBkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="53819662"
Date: Tue, 28 Sep 2021 13:36:45 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
CC: <xen-devel@lists.xenproject.org>, Ian Jackson <iwj@xenproject.org>, Wei
 Liu <wl@xen.org>
Subject: Re: [PATCH v3] tools/xl: fix autoballoon regex
Message-ID: <YVMMXarcqweUD4sK@perard>
References: <dd3a1e1a7a3f8e7bca18dd4779efbc2af01decc7.1631793876.git.isaikin-dmitry@yandex.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <dd3a1e1a7a3f8e7bca18dd4779efbc2af01decc7.1631793876.git.isaikin-dmitry@yandex.ru>

On Thu, Sep 16, 2021 at 03:15:21PM +0300, Dmitry Isaykin wrote:
> This regex is used for auto-balloon mode detection based on Xen command line.
> 
> The case of specifying a negative size was handled incorrectly.
> From misc/xen-command-line documentation:
> 
>     dom0_mem (x86)
>     = List of ( min:<sz> | max:<sz> | <sz> )
> 
>     If a size is positive, it represents an absolute value.
>     If a size is negative, it is subtracted from the total available memory.
> 
> Also add support for [tT] granularity suffix.
> Also add support for memory fractions (i.e. '50%' or '1G+25%').
> 
> Signed-off-by: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
> ---
>      ret = regcomp(&regex,
> -                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
> +                  "(^| )dom0_mem=((|min:|max:)(-?[0-9]+[bBkKmMgGtT]?)?(\+?[0-9]+%)?,?)+($| )",

It seems that by trying to match fractions, the new regex would match
too much. For example, if there is " dom0_mem= " on the command line, xl
would detect it as autoballoon=off, while it isn't the case without this
patch. I don't know if it is possible to have "dom0_mem=" on the command
line as I don't know what Xen would do in this case.

It might be better to make the regex more complicated and match
fraction like they are described in the doc, something like:
    ( <size> | (<size>\+)?<frac>% )

unless xen doesn't boot with bogus value for dom0_mem, but I haven't
checked. (we could use CPP macros to avoid duplicating the <size>
regex.)

Also, <frac> is supposed to be < 100, so [0-9]{1,2} would be better to
only match no more than 2 digit.

Thought?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:54:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:54:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.197995.351271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCd6-00084R-Nq; Tue, 28 Sep 2021 12:54:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 197995.351271; Tue, 28 Sep 2021 12:54:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCd6-00084K-Kw; Tue, 28 Sep 2021 12:54:40 +0000
Received: by outflank-mailman (input) for mailman id 197995;
 Tue, 28 Sep 2021 12:54:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVCd5-00084A-9p; Tue, 28 Sep 2021 12:54:39 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVCd5-0007zA-27; Tue, 28 Sep 2021 12:54:39 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVCd4-0008Fq-Lu; Tue, 28 Sep 2021 12:54:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVCd4-0002G5-LS; Tue, 28 Sep 2021 12:54:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=1egB6p6F0RgDKWOnPY55od2p1dsVOlle3fUmKzzCD38=; b=A+o1DxIcdYKuiAykYI7gaCx2RA
	ZtevivifiebWlf+/aT6g3HtbQ3z+8I+PeO2e21m9u5+uTrXPP4VVL8hqxh76cFihRv0RUN5I8RAQb
	nzKLq+lW49LrMbskXj7zf85vas7flABCjBfk/nWL7ghVg+TA2MlFxrRE9tILDuHG7ZRQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165226-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165226: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=de8ed1055c2ce18c95f597eb10df360dcb534f99
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 12:54:38 +0000

flight 165226 qemu-mainline real [real]
flight 165232 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165226/
http://logs.test-lab.xenproject.org/osstest/logs/165232/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                de8ed1055c2ce18c95f597eb10df360dcb534f99
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   16 days
Failing since        164967  2021-09-13 13:06:52 Z   14 days   29 attempts
Testing same since   165226  2021-09-27 23:09:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dongli Zhang <dongli.zhang@oracle.com>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Gupta <pankaj.gupta.linux@gmail.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5722 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 12:58:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 12:58:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198003.351285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCgc-0000Ql-8V; Tue, 28 Sep 2021 12:58:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198003.351285; Tue, 28 Sep 2021 12:58:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVCgc-0000Qe-5S; Tue, 28 Sep 2021 12:58:18 +0000
Received: by outflank-mailman (input) for mailman id 198003;
 Tue, 28 Sep 2021 12:58:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVCga-0000QV-TC
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 12:58:17 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id be8db59e-205b-11ec-bc8a-12813bfff9fa;
 Tue, 28 Sep 2021 12:58:15 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18SCnMRh006226; 
 Tue, 28 Sep 2021 12:58:11 GMT
Received: from eur02-he1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2059.outbound.protection.outlook.com [104.47.5.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bc3cv818s-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 12:58:11 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4435.eurprd03.prod.outlook.com (2603:10a6:208:c1::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Tue, 28 Sep
 2021 12:58:07 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 12:58:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be8db59e-205b-11ec-bc8a-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y3rJ0gcak0RMP+kRZxxofZ4TFcVU/L8lhy19a/H39EuLlBqUoDPWtfduELRxqXFqXFOlQfSGUYTIVlKN19VYgx/980w9mKuuQe9lDzDwMqedPA/lgm5EMN983rkaZeSMDtl3rN1R2SkKsLBR0zIALJbn0aLGF9BAqa6tgmeuBiW5kyi4BHliVQfC5plZ/yGG3ThURMETjR5mKMkV0LJtMDJq3G/rVkWNn+itFYY6s/d5lj8+UB6JN/VeHbLSdSmwAZJ6KxhyYAJSUkZzKIDzXFc+JhV4unKzk3KDY/r36USgZ5FVGvGA7qmQsuvmccIcO7i1ZT2kWho/dkKWTLbLew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9jH0gqOOD1UJNMg5HtmMPrxSxjV+7hrmY6Y6Km41ztQ=;
 b=QsLr/yHXokHHYNm/nEqLp9QtrrwFf9lkbBozy4jXc4pmLoT+P1j4KlgBHc6D661Gf/QBAFlMtQpPAERQE1eefk/vNoluj2Hu+ML1oTSh65INYE69sG3mREIs+lOvzHmiw0EOEiTAyqSbxbc4d7hTdNMNL9M0/NRLGn3CfhD9neaQ/bWJIKoUNCDKBtcDkUqgU+Y9fVlJq86Cmuwp1n+05yMEExGa5QmVp91yAid8mg3YO4hxatcOxyxDXpy94PGUuqotkrppmhPmqp/tLe+Cr5Ygh1/wIw4ofjfQM0aDE1ltN8CwJPgdkxx7YRfguQmE28MpdxwWOq5ERF7ItVgG4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9jH0gqOOD1UJNMg5HtmMPrxSxjV+7hrmY6Y6Km41ztQ=;
 b=UimeZIFE217Jepp4Cgs+oNq3YUGcwz4V64SAkWd+/9UeWr7VEeD/8GDPwWxgBfWxDA7Vil1qXURlpBzGChO/4XSYMCEdhJJ+DaFIzdjmu3hyBF0T2EoL8QNGYkwbEBM5b/1Fj3VnUL939XRzdTtN8M196hGrOdiyE+Qb12xa8xwokzJQA/1xDlgZLkVNrAyDQAcJy94ef8Up+E/XrbQjSmusgekzwouA3ZsiEXFmpNoinJ8m0lJjK4fDYm/XXWi/BlvH/nb6VsB+jf9YRHqE3PKt13qmhwmP/sI3M8jBbqCVviRTz8MsEtpiQ6ERZtQjo6SuoffLJOafsQoNA/JPZA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Michal Orzel <michal.orzel@arm.com>, Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyA
Date: Tue, 28 Sep 2021 12:58:06 +0000
Message-ID: <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
In-Reply-To: <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 965bed20-4e89-4ba3-6e2b-08d9827f9dce
x-ms-traffictypediagnostic: AM0PR03MB4435:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4435AA1AF6DF7684D96FA527E7A89@AM0PR03MB4435.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 EWNyaBp1sNgfiM+Rf5GsslE/LLcqf/wd3SR8pqwmWRjiTHFckV8mX6T7O4gARmTE9Txpuj6RZeKWh6Jg/eykDPb7gNkQtiRP4AmMWI59/hjbNMWUESxj3Ztkdk3WevT7jZgN0kDSlqd1DFnYAA+TGBG+1crEhVBoqzGJ4fnRsTcTh9gKGS7wzQP4y7tsFqIKo54W/Ggbzmm4FCPXYJvIjxaLEU6mOFbbOmwVXEUuonZJXnr2gSq3Yo6bsnSvRkIxnmfhuvm9m2f1EhUabn/o4Yvhg/pku94kzYGcRUR7LCy47c34qNNfm9JWjh5SL8XwwDJIyfEhBP/RxzhDEg9l1D5QSFpHgIwqhDgZilphndcmFafeai4upkYkQIMdGr1/4Q+1exiCUwO6u3JGyaEXr7uFkcWXwaK2+PG0wh8odboEarczbeYIf2MgimPvrOUqTPvLF67Q+qyC5Se+6KSp4BX+n7/D0X0YUJfMkYtF+XaSsfT+PLFWk6TpblqD4EoVB3d3NWWOCMl8WUwUQL9DGG6gJP7qa33QyjaBAeeNAvAwFV3fSZGrh1k/UHLNthEj728Bzrxws4Ph7NgRl1hIqHdSyTlpOZfA3u6nrpuDWvpWl4nud4KMM6QB4UY8VwTa7J2glrMsOq/G7blfcwGweUZuKSJGqJqp94CgLz1UEgeP40P/5wdmw4rBjYwS8PyZcDddi+lmjsW+dAabwlTgPQxoi/rBz5r4AXTSaPzQZJC5jtpP8C2daPxW022P3YR1
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(31686004)(83380400001)(86362001)(31696002)(316002)(6506007)(26005)(66446008)(64756008)(66556008)(66476007)(186003)(5660300002)(122000001)(55236004)(2906002)(6512007)(38070700005)(8936002)(71200400001)(76116006)(91956017)(38100700002)(53546011)(110136005)(54906003)(2616005)(4326008)(8676002)(6486002)(36756003)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SnA2TGFvT3BYNTA1aHoxZHJmUUVxRFJGQm0vRmd5N0RGR2ZFampjeW9DZmxp?=
 =?utf-8?B?Ym1VOGppY1hpY2cyYnVKVHlKeE5PYkNxb2JleHdBaGhBd2Vyd0FFeFAxMk9F?=
 =?utf-8?B?bzhrMVJ4THUxNTIzNGxKRHdMVmRyMTVuVGE5bzNNWEdmK1VZcWhNeGdGVzQ1?=
 =?utf-8?B?Z1pUZW9QblJpcGtXMkFENlp6TFMxNHZZUXp0RGM3dWNkRFFnQURrbkpJaVNV?=
 =?utf-8?B?TXhtZEJPdzlzU3htUFRaMXg4R08rQk8zckt0SDVvUmp4VE5oamxFZyt6R003?=
 =?utf-8?B?R28zR2cxaGNnVG50QnNhcm5pZEZsQVBNUDQxMEZYODBjcVluNEVhUGpTTU1F?=
 =?utf-8?B?U2g3U0NOWWFJY2FESWc5d2d6TDhiSy8zZXluaVRHZi9WUWtWdUVqdWlHYWR1?=
 =?utf-8?B?ZjA3Sk05V0g4T1VZYTlZUldvL0xSUUNLSmU4UFB1RUdteVFza1ZaajNyYlUv?=
 =?utf-8?B?YVlMbUVtUTZmUmVIOFZDVXZGRGdWNFcxeDRySVdLNWxWc0ZsYkN3RWdKYlhP?=
 =?utf-8?B?SndRbWh4c3dUVTdCYjNyZHpWcFJBRTUxQ0M4SlpjamUyZlA2VCt3eWQ2dVNF?=
 =?utf-8?B?NmdCYS9obVJLUEJMZDVucXBwWUZIdWNzQXlJRkZiQnpEenlubUhUcjlud0NM?=
 =?utf-8?B?N29MSXQ1VEwwR3c4NHlvTWxoSkY0TCtWbG1TM0JLQm9odndUaEQ3b2QrYjl5?=
 =?utf-8?B?NUtQUkVVZjk3K1lFREdEbjRIM1ZoOWNPSEljM2YyY1ZnUEFMUUwyNXloTlVn?=
 =?utf-8?B?T3FOZlNOTTEvRXVsRkJyeGtOOEJiQis4K0tKNVd0cjBqa014STMxL2craG55?=
 =?utf-8?B?Uk5jeVpLaTUxU3had2w5cVBwT0lVWGNTTDhyM3gwb0c4UzhIUUxDcHJaVGhQ?=
 =?utf-8?B?SjUxalRYaU44d3I1SmVjb0FKOHBMOFVzOFpHd0I4UGpoWk1PMlR3bTV1UzJu?=
 =?utf-8?B?Ym9jUjE4MisyeEZEUjdQQXNyYlBKMlB4ck42RGFjZlVKaFNWeis0eWJpNm02?=
 =?utf-8?B?TlJmVUdzT3JzT2lDUlhKaHp5ZXFyMXl0TlltaFgxRVNhT1BsMVRxWjVoVUU5?=
 =?utf-8?B?Vjdydmt0ZEp5b3hkYlE4bnpZZGcwZTNzV1M5dVNzNkd5aVlIeTZBQ1d0TDB6?=
 =?utf-8?B?OXpHV0pvU0QwVDRYZURqQ2cxbFByZklOSW0xWlc3ZW12MU54anpxYkFUY2dx?=
 =?utf-8?B?eE9CUG1wTVhHVDNxMkltM01DWW5PSEtXL1FoRGdsK3N0eHB1cDdhRDZLRUtF?=
 =?utf-8?B?R210alIxS0FyMGsrNnFHMVgyNnhKQ0wyRVBsaElReXd6UkJ1MGpSZUpGaDhr?=
 =?utf-8?B?MWJ0WTJXS1FEaGtDQmk1V2NnTEhVN0FGeDlMejdIRStPUk1qSjk3bDZjZDJk?=
 =?utf-8?B?SU1SbS83dWJaNFhZMDN2dEN2TGpJUU0xWng5Z2FFTzVvaW9ScEl6MUZSTHFr?=
 =?utf-8?B?c0JyVE9zcm91L0Q3NWxDVU9ZYkVsaU5KYnRhQlVNeXg3VVRaOUtwV2k5V3pW?=
 =?utf-8?B?R3Z2UGZVR2FKTkhiZkc0Mmtxd3FTTkJwOWR0Y0E1SVJUNk1WSVlqbnFLTGhX?=
 =?utf-8?B?S1FmRVpxZE9LU1FDRm1EMU5lbGRqa0pkakVuWklxejdhd1FacHVHWmM1dk1s?=
 =?utf-8?B?RGRRSjlyQ2dEZDhjNndjd2xMdFJETjM3VnQrVzVBSXJmbmM4MkpDRkxEVDFy?=
 =?utf-8?B?SFB6NTJyd0RwSVFqTWlNQlg3Y25BaHhpT0ZmUWtHK2ZCVm9vdG5yTTZualN0?=
 =?utf-8?B?bm9ia2dNUmxGcXRKbzMzVjk1Y2piejlGS3ErZnNaSkpLdFB3dDBNNGZNU1pD?=
 =?utf-8?B?OGhBaUFqaXYxTnI1VUZ1QT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <739A8BB36D1C8C42BA64F6776339AB2A@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 965bed20-4e89-4ba3-6e2b-08d9827f9dce
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 12:58:06.8181
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tw6iTRh2lXPi6vrjF37Cc9Hfn6GLTFwUvV02GWPNvhD69MC4p04n4cOH7J8szJHwltGT4QrOPhq8eY5BrNlLdUP0kOlrfhMXNsmUpN3zIItu4pDJTnL7UKUoOMVi1pSK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4435
X-Proofpoint-GUID: L2rjqNUkkc5Asi0e61S4kbsaPauqD51o
X-Proofpoint-ORIG-GUID: L2rjqNUkkc5Asi0e61S4kbsaPauqD51o
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_05,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0
 suspectscore=0 mlxlogscore=999 adultscore=0 priorityscore=1501 spamscore=0
 lowpriorityscore=0 mlxscore=0 phishscore=0 impostorscore=0 clxscore=1011
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109280074

DQpPbiAyOC4wOS4yMSAxMToxNywgTWljaGFsIE9yemVsIHdyb3RlOg0KPg0KPiBPbiAyOC4wOS4y
MDIxIDA5OjU5LCBKYW4gQmV1bGljaCB3cm90ZToNCj4+IE9uIDI4LjA5LjIwMjEgMDk6NDgsIE1p
Y2hhbCBPcnplbCB3cm90ZToNCj4+PiBPbiAyMy4wOS4yMDIxIDE0OjU1LCBPbGVrc2FuZHIgQW5k
cnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNp
LmMNCj4+Pj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+Pj4gQEAgLTgz
Myw2ICs4MzMsNjMgQEAgaW50IHBjaV9yZW1vdmVfZGV2aWNlKHUxNiBzZWcsIHU4IGJ1cywgdTgg
ZGV2Zm4pDQo+Pj4+ICAgICAgIHJldHVybiByZXQ7DQo+Pj4+ICAgfQ0KPj4+PiAgIA0KPj4+PiAr
c3RhdGljIHN0cnVjdCB2cGNpX2RldiAqcGNpX2ZpbmRfdmlydHVhbF9kZXZpY2UoY29uc3Qgc3Ry
dWN0IGRvbWFpbiAqZCwNCj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4+PiArew0KPj4+PiAr
ICAgIHN0cnVjdCB2cGNpX2RldiAqdmRldjsNCj4+Pj4gKw0KPj4+PiArICAgIGxpc3RfZm9yX2Vh
Y2hfZW50cnkgKCB2ZGV2LCAmZC0+dmRldl9saXN0LCBsaXN0ICkNCj4+Pj4gKyAgICAgICAgaWYg
KCB2ZGV2LT5wZGV2ID09IHBkZXYgKQ0KPj4+PiArICAgICAgICAgICAgcmV0dXJuIHZkZXY7DQo+
Pj4+ICsgICAgcmV0dXJuIE5VTEw7DQo+Pj4+ICt9DQo+Pj4+ICsNCj4+Pj4gK2ludCBwY2lfYWRk
X3ZpcnR1YWxfZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpw
ZGV2KQ0KPj4+PiArew0KPj4+PiArICAgIHN0cnVjdCB2cGNpX2RldiAqdmRldjsNCj4+Pj4gKw0K
Pj4+PiArICAgIEFTU0VSVCghcGNpX2ZpbmRfdmlydHVhbF9kZXZpY2UoZCwgcGRldikpOw0KPj4+
PiArDQo+Pj4+ICsgICAgLyogRWFjaCBQQ0kgYnVzIHN1cHBvcnRzIDMyIGRldmljZXMvc2xvdHMg
YXQgbWF4LiAqLw0KPj4+PiArICAgIGlmICggZC0+dnBjaV9kZXZfbmV4dCA+IDMxICkNCj4+Pj4g
KyAgICAgICAgcmV0dXJuIC1FTk9TUEM7DQo+Pj4+ICsNCj4+Pj4gKyAgICB2ZGV2ID0geHphbGxv
YyhzdHJ1Y3QgdnBjaV9kZXYpOw0KPj4+PiArICAgIGlmICggIXZkZXYgKQ0KPj4+PiArICAgICAg
ICByZXR1cm4gLUVOT01FTTsNCj4+Pj4gKw0KPj4+PiArICAgIC8qIFdlIGVtdWxhdGUgYSBzaW5n
bGUgaG9zdCBicmlkZ2UgZm9yIHRoZSBndWVzdCwgc28gc2VnbWVudCBpcyBhbHdheXMgMC4gKi8N
Cj4+Pj4gKyAgICAqKHUxNiopICZ2ZGV2LT5zZWcgPSAwOw0KPj4+IEVtcHR5IGxpbmUgaGVhciB3
b3VsZCBpbXByb3ZlIHJlYWRhYmlsaXR5IGR1ZSB0byB0aGUgYXN0ZXJpc2tzIGJlaW5nIHNvIGNs
b3NlIHRvIGVhY2ggb3RoZXIuDQpXaWxsIGFkZA0KPj4+IEFwYXJ0IGZyb20gdGhhdDoNCj4+PiBS
ZXZpZXdlZC1ieTogTWljaGFsIE9yemVsIDxtaWNoYWwub3J6ZWxAYXJtLmNvbT4NCj4+Pj4gKyAg
ICAvKg0KPj4+PiArICAgICAqIFRoZSBidXMgbnVtYmVyIGlzIHNldCB0byAwLCBzbyB2aXJ0dWFs
IGRldmljZXMgYXJlIHNlZW4NCj4+Pj4gKyAgICAgKiBhcyBlbWJlZGRlZCBlbmRwb2ludHMgYmVo
aW5kIHRoZSByb290IGNvbXBsZXguDQo+Pj4+ICsgICAgICovDQo+Pj4+ICsgICAgKigodTgqKSAm
dmRldi0+YnVzKSA9IDA7DQo+Pj4+ICsgICAgKigodTgqKSAmdmRldi0+ZGV2Zm4pID0gUENJX0RF
VkZOKGQtPnZwY2lfZGV2X25leHQrKywgMCk7DQo+PiBBbGwgb2YgdGhlc2UgY2FzdHMgYXJlIChh
KSBtYWxmb3JtZWQgYW5kIChiKSB1bm5lY2Vzc2FyeSBpbiB0aGUgZmlyc3QNCj4+IHBsYWNlLCBh
ZmFpY3MgYXQgbGVhc3QuDQo+Pg0KPiBBZ3JlZS4NCj4gKigodTgqKSAmdmRldi0+YnVzKSA9IDA7
DQo+IGlzIHRoZSBzYW1lIGFzOg0KPiB2ZGV2LT5idXMgPSAwOw0KDQpPdmVyZW5naW5lZXJpbmcg
YXQgaXRzIGJlc3QgOykNCg0KV2lsbCBmaXggdGhhdA0KDQo+PiBKYW4NCj4+DQpUaGFuayB5b3Us
DQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 13:37:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 13:37:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198014.351297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVDHt-0004wl-7X; Tue, 28 Sep 2021 13:36:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198014.351297; Tue, 28 Sep 2021 13:36:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVDHt-0004we-47; Tue, 28 Sep 2021 13:36:49 +0000
Received: by outflank-mailman (input) for mailman id 198014;
 Tue, 28 Sep 2021 13:36:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVDHs-0004wY-Dq
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 13:36:48 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e23392dd-0ab3-43ac-bd45-be19a73f1939;
 Tue, 28 Sep 2021 13:36:47 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2058.outbound.protection.outlook.com [104.47.14.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-jLCMAPYfOoKHvJ-5ZmA4aw-1; Tue, 28 Sep 2021 15:36:45 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Tue, 28 Sep
 2021 13:36:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 13:36:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR3P193CA0012.EURP193.PROD.OUTLOOK.COM (2603:10a6:102:50::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.22 via Frontend Transport; Tue, 28 Sep 2021 13:36:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e23392dd-0ab3-43ac-bd45-be19a73f1939
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632836206;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PXCticxT18Y17PZNDKIpCA0j/4DFPQ4recnAeJK47fs=;
	b=OuuJNMuoN0GlXIg9QJXkxHkxQlNV81lWfY6WhpJO/T/QRdCECJ3ZetSuhmtgy0DeYDGUwz
	nlUTNJtv34+kqbBEBvhEg33QLjARgunMu2eYDdn7QrA0ZWAJUkR1jN8pKIRweZedeNPIoF
	sXfayWf7SNxzzQ28ElijREe6GpEyu50=
X-MC-Unique: jLCMAPYfOoKHvJ-5ZmA4aw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lR5USHTjBOkoKJe8Wqz69QBzxyg+T0lKhaQCLQ/rsHK8AJyMLVIKSnDq4wbJ+/m2oWbk9kGvrxr2cwMBUYrtZFsFs2HF1E8qmXw/K6bgT7XA1QOND2SPlNhHJuzFHEMBmSL715iFt8RMnvtGpOsMZliufmNfcldsmplZyy+RYvZApVlVMDIZlr2FrmgWal6RrLiKNOvyZ7wq49SeoKBGOzcRvxSVNAPmIe1KhYxzymizhmSmR6r6YdXL+2H+uHZhoOePyN6z1MXYtGTXa75ltFTCZF1xGoEhWk81FoxcWQ+0Fs4CX/yWoIN4QKz8l3OKgw3tDScIZgTQ53f697pMEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5CZOjMZQ06ut6Cvo6a2Rn8E6/d3bWrOzmvQeZtCrL2A=;
 b=dznxUe6kpqU8M3WLOGHENsL/BrEuY5nn3KC+oSiXsH31jo2Ih7VSA4MylrdLLsATTIxMXvhdWgKV63bdesWTh/HRdBqPNkZHdiV0Koe7XaSHtX6fVvISROicJA0JGbRm6vHgnyd7qRa6jjEh1oaMxdFs/npXJx2w+8JeQRgBXj/ORwnJR86RSUnRxsqciZBWPy3GGxQGVh7joDl1BBZ1dinXFOdcGS/ZTeEEIoRQKylsy6tOWM73wuEHKs8YNwEN4/rYT085KxMWdpghAshKjO5ipRY+cs3tunl98sVa5K5GaJd4B+qMCD2UdfTkH16+qXHeiSfoUBKBkx5BRyEzjQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Ping: [PATCH 1/2] gnttab: remove guest_physmap_remove_page() call
 from gnttab_map_frame()
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <4f54456b-e8da-f67f-b6a0-b5ce2cf12cae@suse.com>
 <8b73ff7c-4dd6-ff2e-14b9-088fdce0beb9@suse.com>
 <YUhgdMyTbfL8Hyke@MacBook-Air-de-Roger.local>
 <5c1fd288-2972-d264-d6b0-3c7bc6d67be0@suse.com>
 <YUmYpzhJrBZkSYyd@MacBook-Air-de-Roger.local>
 <ae0fb20c-b7c9-2467-0951-b84b2f647382@suse.com>
 <YUr2zZL3kV4/nBQp@MacBook-Air-de-Roger.local>
 <8fd9e2d5-b875-ef7d-d80a-15b6ba2948b5@suse.com>
 <YUr+21K8GNqMFjKB@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4fb0f83a-8aef-c9a3-217c-99c5c2fcb810@suse.com>
Date: Tue, 28 Sep 2021 15:36:38 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUr+21K8GNqMFjKB@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: PR3P193CA0012.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:102:50::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 91b9c1fc-13e6-4199-7871-08d9828502bf
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB393365F83FB401003F8F34BBB3A89@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kndINbsAh377M571k3H/0gM6sfFlSRLc/YO5xB+YRecQDYo4S2Kd5u5N1V8r/1FFpfEp8apPmqjNQEXIn51SKxHkEgxSCGw+BmtjAvwFslHIUUOj6otzY6dhIG7DE0oup+/9MdB0L4zxtoMvF64Opb+ju8ITLm7ztx67144nqFO0GRlTRXCA5gchwJcDHDvvFRS3GGuW5qGZ0Trdoqu+4IVY/lZYQ/Rhydb6tdK4F0OfflsB1fF1l8QoeTmkLXHuTsuoFhcmCfACzZRdYyQgFF0GAR45bJZnN5iqlC0ti5gjfPY/2Y7LD2Vt5VAt0nP+UkX9Q89i7uhWoW393QOptF1Vp5HhuUSWg/n5e15W9n2QbcPsShNe6YsvYISLITrP3gkwggMHgIyc3bz1GcdloZrDnXgpWzTqgzqbCB3jT8rfwZqeUKSW4zM3sAjlbsVb1snroivJgoMK8Yu2tNR4VJ5iOUHoLMnPjQmIrLKyfz8wBkCdg+blEN4QYEZPFCBKSILm3J4n6RymuG8ZnyGWksl413vfxOQ0sB2TqiHcq7/aWxrDDxfPK4gYimIhqDsOBroHVhk4ktaOTxhXtS0GvjLSQLAJjRrNwIMcebc/THpppDky5tlX8AdHuE/UaXL7trVG6KqIp5kU8B7IEFbg+9he9c6d0r/XKTrp2AgYG2XoSUtPCm0NvcLGJlV9qsexWRAxfu23QU8lBGHpeOPYd6X8TElD1Pw3YDM26jZH8ycnFikme3tGBQP8F213Npy8eMBKFUH+xHl1/MqO0EL0wo5kjfry0LQXVXDSC5tlX0g=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(54906003)(5660300002)(6486002)(31696002)(53546011)(6666004)(38100700002)(316002)(4326008)(83380400001)(16576012)(110136005)(36756003)(2906002)(508600001)(186003)(66556008)(66476007)(66946007)(86362001)(8676002)(956004)(26005)(31686004)(2616005)(43170500006)(76704002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?C7D+3qNffIO+I5g2+uREWXD3wilhQ6L5U2bO6vX7wBRsT1GsA4gyHgqxZ4pc?=
 =?us-ascii?Q?gUfjrM0gnGsUS/fh49z/epm0D1l7ZHdF9+2neezi9HhU6bqI+c4OKsRqH3MO?=
 =?us-ascii?Q?1XFu2IYZ39ypqUI7J3FPh5JHg9PXL3TnS0ueKrg51SDhGRreXuADwrS+1Nyf?=
 =?us-ascii?Q?qLa96FglJpRXW2H0XJRcXq//ItMFjMA86GrZlpsDVGBypR/1nYXwjFSuahi9?=
 =?us-ascii?Q?fmwpPQV+Ctsh/lXT8au7X4azAms+DBoWBaIr8/eUDgP8Mzk1RbX17/c8BSy2?=
 =?us-ascii?Q?dj8FY9OSmzt5AOk4h7iZtgvy1zN701JsetPkbyinGeM+MC/6qJV6KKm88ayf?=
 =?us-ascii?Q?SXc2xtV4fTcxVupw6fvkKWg067XwpsSrFhnsnoVtXBsVYVqpX2GCqJ4mOU28?=
 =?us-ascii?Q?MXXloSbVcSwaiKfDWqjpBNfJfxAPNBYl+B3idd4pP7IqoxLA3mq07UWRNUuZ?=
 =?us-ascii?Q?MkLC0bJN/+E7wpnvpihGlf4cwsT/XONNZL2P0bGenspBAcwojST4fChMFRuD?=
 =?us-ascii?Q?6AD5du1IkNBhuLAi3G1DAx7JkaJQ9+ntZQLAQv4cAvE64fDFmrgSFNKJovV0?=
 =?us-ascii?Q?LdegGwpEYcj2MRCsWoEcThiF6c0jA+0d4W82JqWEXgrAXT8/X/gfv4x9i5Y1?=
 =?us-ascii?Q?4YIC8ZQ4vfOa1+nJE06gaqNwQ7p+aDVLwQ5lmS2fe22gR5Kjnm73Zbzab+Jh?=
 =?us-ascii?Q?YyG+8cZ7+wse7JWeff2Hp4Zm0D3rMCiS01u+lizstGbErlfUUolQnTq+4+t9?=
 =?us-ascii?Q?AIVX662INDAtp3hkfrqEThjcxEJTbM3jEJTQfUEgXyULLrKch282bbj1EumH?=
 =?us-ascii?Q?zjy0ndGnWs1cmQ8iWsYapHFtXlLwyPyPZRT7qZ2Hsw2pxWAwcZaQqkNh5bjf?=
 =?us-ascii?Q?dpTHiRxlw6xOwOKull7rncfuabsALNQMezbMsqZQZibh4yvjDoPWjqaMHhxA?=
 =?us-ascii?Q?mFUjs27mzb77WA9A1DNYC59QFlzVjnNH5L8e1ww68kVBF9z4lep4ibtwKr4f?=
 =?us-ascii?Q?/c8voaM0M4lJrwhgH05GMg/GBoXLlkhQ04VWht99exAqXItW+TcoaxfBZzU4?=
 =?us-ascii?Q?3p5IxcoBXJ3zy6gCNOIuH/onf3rSVY5hTxGX6iH+LJXL37nuFpNSIMQq5BSl?=
 =?us-ascii?Q?TznT4Z0O2RVchycW2izOom96dbsx8mNoP9JObCimw2yXTrDl4Xl1T18yymX3?=
 =?us-ascii?Q?iIvXIo3mn9VCsB3Cpjzk7FndxZp/uTMBH8nzUvts0Obt/eTT850p1IzHc0Rh?=
 =?us-ascii?Q?VyPB5mlo2j24H3SpuicytNeXcNfJWylOjpeqXe9JHum1PsDeVPH6D1qFjCAk?=
 =?us-ascii?Q?SIgVRRmePN7cCBxU3ri65lXL?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 91b9c1fc-13e6-4199-7871-08d9828502bf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 13:36:43.9219
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +lpLGQZ2mejguBpW7nxSocc+bsCELOs1JnW8+vEqGyEQ9aIkn7LoFuPX4FyQi9vl3cItfiK+jumCo9RQ7tn6Ig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 22.09.2021 12:00, Roger Pau Monn=C3=A9 wrote:
> On Wed, Sep 22, 2021 at 11:42:30AM +0200, Jan Beulich wrote:
>> On 22.09.2021 11:26, Roger Pau Monn=C3=A9 wrote:
>>> On Tue, Sep 21, 2021 at 12:12:05PM +0200, Jan Beulich wrote:
>>>> On 21.09.2021 10:32, Roger Pau Monn=C3=A9 wrote:
>>>>> On Mon, Sep 20, 2021 at 05:27:17PM +0200, Jan Beulich wrote:
>>>>>> On 20.09.2021 12:20, Roger Pau Monn=C3=A9 wrote:
>>>>>>> On Mon, Sep 13, 2021 at 08:41:47AM +0200, Jan Beulich wrote:
>>>>>>>> --- a/xen/include/asm-arm/grant_table.h
>>>>>>>> +++ b/xen/include/asm-arm/grant_table.h
>>>>>>>> +        if ( gfn_eq(ogfn, INVALID_GFN) || gfn_eq(ogfn, gfn) ||   =
        \
>>>>>>>
>>>>>>> I'm slightly confused by this checks, don't you need to check for
>>>>>>> gfn_eq(gfn, INVALID_GFN) (not ogfn) in order to call
>>>>>>> guest_physmap_remove_page?
>>>>>>
>>>>>> Why? It's ogfn which gets passed to the function. And it indeed is t=
he
>>>>>> prior GFN's mapping that we want to remove here.
>>>>>>
>>>>>>> Or assuming that ogfn is not invalid can be used to imply a removal=
?
>>>>>>
>>>>>> That implication can be (and on x86 is) used for the incoming argume=
nt,
>>>>>> i.e. "gfn". I don't think "ogfn" can serve this purpose.
>>>>>
>>>>> I guess I'm confused due to the ogfn checks done on the Arm side that
>>>>> are not performed on x86. So on Arm you always need to explicitly
>>>>> unhook the previous GFN before attempting to setup a new mapping,
>>>>> while on x86 you only need to do this when it's a removal in order to
>>>>> clear the entry?
>>>>
>>>> The difference isn't with guest_physmap_add_entry() (both x86 and
>>>> Arm only insert a new mapping there), but with
>>>> xenmem_add_to_physmap_one(): Arm's variant doesn't care about prior
>>>> mappings. And gnttab_map_frame() gets called only from there. This
>>>> is effectively what the first paragraph of the description is about.
>>>
>>> OK, sorry, it wasn't clear to me from the description. Could you
>>> explicitly mention in the description that the removal is moved into
>>> gnttab_set_frame_gfn on Arm in order to cope with the fact that
>>> xenmem_add_to_physmap_one doesn't perform it.
>>
>> Well, it's not really "in order to cope" - that's true for the placement
>> prior to this change as well, so not a justification for the change.
>> Nevertheless I've tried to make this more clear by changing the 1st
>> paragraph to:
>>
>> "Without holding appropriate locks, attempting to remove a prior mapping
>>  of the underlying page is pointless, as the same (or another) mapping
>>  could be re-established by a parallel request on another vCPU. Move the
>>  code to Arm's gnttab_set_frame_gfn(); it cannot be dropped there since
>>  xenmem_add_to_physmap_one() doesn't call it either (unlike on x86). Of
>>  course this new placement doesn't improve things in any way as far as
>>  the security of grant status frame mappings goes (see XSA-379). Proper
>>  locking would be needed here to allow status frames to be mapped
>>  securely."
>=20
> Thanks, this is indeed much clearer IMO:
>=20
> Acked-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Any chance of an Arm ack (or otherwise) here?

Thanks, Jan

> Albeit I still think we need to fix Arm side to do the removal in
> xenmem_add_to_physmap_one (or the x86 side to not do it).
>=20
> Thanks, Roger.
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 13:46:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 13:46:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198022.351308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVDQv-0006XJ-4D; Tue, 28 Sep 2021 13:46:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198022.351308; Tue, 28 Sep 2021 13:46:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVDQv-0006XC-1G; Tue, 28 Sep 2021 13:46:09 +0000
Received: by outflank-mailman (input) for mailman id 198022;
 Tue, 28 Sep 2021 13:46:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVDQt-0006X5-9P
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 13:46:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6d33c7d6-2062-11ec-bc91-12813bfff9fa;
 Tue, 28 Sep 2021 13:46:06 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-4s-QoxVtOh2q_3Ky3zEF_g-1; Tue, 28 Sep 2021 15:46:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4846.eurprd04.prod.outlook.com (2603:10a6:803:5b::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 13:46:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 13:46:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0010.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Tue, 28 Sep 2021 13:46:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d33c7d6-2062-11ec-bc91-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632836765;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1e2UxGNhGpbeevwfItRHbrV3DJCtEnSX9HXGLkrSQ9I=;
	b=OoxueNrhBL6PiL9mbYznSgoXru+pS50zUK//ph8ktOo1Hd7g73JqgZwXdZUxad8xyXL56U
	Qazg5bJRlylgbStZPIOV8df/MhVnAL/KLKlSwnrH5kosUZZYfdj5Mj2qVvadcWnLK+rAwH
	JjdXnY8dDqwJodhBrlx23MMxwS8lQkI=
X-MC-Unique: 4s-QoxVtOh2q_3Ky3zEF_g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WqS3WpDWRSs45zAbeS693Fy1rEAiKB9wL63hP6h/tbxM0OHlsUVhz1XgEh+PJwpAoZ8ls2Hehl2Xim+TlOKRZeKd1xMoBGNfRAKOyvc8qmHWPhe5o5ERPX8vEM+ruLmtxm/ObnzBrh6FvRXXEV75logO2Lv/9jfSos4tZ1801qFkjcghgEATJtSYb1ubddmdVUGdkRVplCoJP+Np6P6Rl0MSh+r2WI5uelNTPOlSYwFfRateSULeE0sGXmlfZn7whhG7SZkLssWKWFFqw7Uy8PRbfqD2iyv2BDWjcYYF9NpGEodFx8stXeLzuRLH3YTSIyvnHQtENFFKvieqJG6AUw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1e2UxGNhGpbeevwfItRHbrV3DJCtEnSX9HXGLkrSQ9I=;
 b=eqSih81HK8/FDD/Opcjfa9pACtbXY7EOb7TrHvJB+irtkb+etbG5qWVVBnyuE5iJsAkJu3xB90rhluviOS73K371JOfa/L6sbcIflJZGgWaexGlMRVT+GqX9XxJwx/o3ZrzIZX+zZmvvZwCPu0z0xfjFngBF6+QWc+ipopcJBTdDjf5fXYILrfyl9gpKAA0s1qMrwlaOVuOIPt7gJFitaBDJbt3lIOHTmlIAO51Bt+xpx/Wj2jHgzUG3u3e3TbKafv6H4ddwj7a4BNAoLFxgJMy8574bbD0fjmzfRlPw339fy5lLWWoY6CyK4nqvs2evCpE0VpUodRQ2CAfPU7CIEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v4] xen: rework `checkpolicy` detection when using
 "randconfig"
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210928083944.780398-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <bdd68129-0415-2018-c0bf-98687094dab4@suse.com>
Date: Tue, 28 Sep 2021 15:46:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210928083944.780398-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0010.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 772b6d68-1899-4ec5-8360-08d982864fde
X-MS-TrafficTypeDiagnostic: VI1PR04MB4846:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB48467DCD56C4EFFBBA92649CB3A89@VI1PR04MB4846.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	m7xoX4XmJzZ5GHwtKCf/bUY/xheh7sowhru8maHwiG9LhYocHDPRHWG3tFHxopFxzBpTpIdHJmm4OnNJcGitS2bQAz84FCNNa0JFYTXJFTl69wqoNpiqbThfP59kXR8UNVojDe7OBIqL+0QCIvAw7OWn64n9XzpseT36waMZqylN05pVw14TxeR928Fsbg7rVleJKDDuIDBS2b6J+946CR6Xq6nLjj5CuFZ+DgXTfXH2ZoBNbd7SfsTlYkiUtAoUPj8uY1TNrVgJyzo5DUTGy0QUnOgNqCQHisQMStslY9VUDtk/C7qsd536Q99zZMn7raCB3vlTi+qEtwVPES6TjDUyvwEHfOAnk3FyLQWMIcW93nKzLwIKW7lvYcsWT0mUUHxYDkDspn7CkfT4pWMRzs8oYWFiDUqFW3DLL12mW4BOA51eNCIhPkd6mA2FE/6+7pF4jGRSsUPthonxDI9brP4j7f/Hy/nolI4t80u6iOoiJsSMIcS1NOIRUdD+YfulJZAh4fI6jg8/ua0ybImtmzsq+MMzVKE5t0/+iUVUvSR+I/DzM9D5+o0e1NpgekxfrjMJoIg1OGrr6z07UKYR1Hc9Gl9NS7Gg3q5bsiWmYQgAfVZydjHJ6kDan3wQHPvImDOdaGxf3BRhIpa/n8Z1QIcN3WAcNcn2P3tVKrxWFCLf692JqbBsxLm2REtSSxqZauc2iAfdxVpEibyCCip2psGNgSOewLwy4hQlUsSqMR2q3T4qZjOvaChxXb6RZnXR9meMTFi+wCfX7MVUzOPpNQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(86362001)(31696002)(54906003)(316002)(8936002)(2906002)(66946007)(66476007)(53546011)(66556008)(6486002)(16576012)(31686004)(508600001)(36756003)(186003)(2616005)(8676002)(956004)(83380400001)(5660300002)(26005)(4326008)(6916009)(38100700002)(81973001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L3AxY3dzeVpUOXR2bk10SkZoK3JTWTBqR0U0eU1GYWhMUVZ0bWxPc0hvL1N6?=
 =?utf-8?B?TVdRMW9oTUZwRE95V2E0K1JwOFMyOTNQUHJDaWlkNkdCdlQ5WWo4L2NOelZx?=
 =?utf-8?B?MFYzaHFMcUdTcDR1Tm0wK0lsUGR3dklkSWVMdlpqNldIUnZKdkJTYUQwUnhz?=
 =?utf-8?B?UGl6MHZyenpoYm5Uc0RqMTAyN3VHZm1kQWJUcTV4ckR6cVdPenlaMHJPYWFj?=
 =?utf-8?B?bnU1SEc4MmthTDdmemJKdUJKRVZLMHBZNWRGSkFwRGpvN1ZES04wdTV5elN2?=
 =?utf-8?B?aFdDaWVyWDVyRjZ3blFqNVoyaHQrWHFFb3E0b0VXQ1pqZVUzNTRaWVZuTWRH?=
 =?utf-8?B?WnB3TjFSMnkwa1ZmbE91L1RKY3l6VVlBRis3R1FxTlRFeEJIcGN5NFdxbm8z?=
 =?utf-8?B?Y0loTTIyWkowOXQ0SjBCR0lSK2t2SUs2RFllT2dZNDJrSStPT2EybjhveXB0?=
 =?utf-8?B?Q2ptQ0p4R0V2MENsY1M3ZHY1SjQ4OTBoTmtIQ2JvMVBqWmVtZFRMdVExRVVW?=
 =?utf-8?B?USt5SFF4bjhVck1nMUdyRm5ZNUozYW9yU041UUtTbUt2VmVVdGJvZHNaUUdU?=
 =?utf-8?B?OUFOU1VKTWNtd1l5L0NNTnFCRGxUY1dVTVcweFZiVDB3SzBBbjNJN0dYaFpr?=
 =?utf-8?B?ZEZ0T0Y2bkVPNFk4UEhOSlR2QnljaDd2Z1JCRjdtVzRoeUJaMVY0MndhaU1P?=
 =?utf-8?B?VWQ2bEQxeStpT2VISStOVXBkT1lrcU5SUXlMM2UxWkV5Rk54bTFWY0FCc0x5?=
 =?utf-8?B?a2hicmZZTnh4bmdrZWlWWDQ1WWdPanBFVW14R2VYQm9sMWhManZOYVp5enNM?=
 =?utf-8?B?UEhMbjllNW9PK0FPaGFmcFpKT29zT3loTlhOYnliNnZqZHQ1cThLdm5pNkZa?=
 =?utf-8?B?K1RHYmdFUXF0MHpxUnFSUnZES1hBZ1FCNGF1K083UUNvNG8xKyt5TFVNZzFF?=
 =?utf-8?B?NEFOblZBVFBWNFFOa1R2c2JGSGhqQll2cElVeFFabTR3UHVLUUlQSnUwL1Fu?=
 =?utf-8?B?NkVtWWloS0F5cnBnV3pKRFptRjJqOGVxN0ZIWVVQTVhEQTBGcndsVmRHaGVC?=
 =?utf-8?B?eVJTSkQyZElOWEdvaWJCSVcvQnNqVEtWalE1TGx3S3EvbVUzYXpmZE5JMTd4?=
 =?utf-8?B?cmxkNTBCVXoza3VhTXE5ZzlOMU1JdWVPOWZGazVBZjJDQjRoUmtNOWppZHFi?=
 =?utf-8?B?Ui9Mcm1zZTVLVjQzV2I4SWhDa3k4UWFqZU05MWRnY1RXYTZoUERzNTd5K3pQ?=
 =?utf-8?B?TlNmT0ZlOE42amZGVmJBc2FiWWgzNHlFVW5iNk1YV0grWWgrdW9kRThaTFp3?=
 =?utf-8?B?b1FKMHdJYWVUZitVcVhCU1c2K29jSi95cUFvZVh2WHRXZjJBckRYNTg5QlNF?=
 =?utf-8?B?b0wzWGo3aDcxa2JXOEx2Ky90S1RKWUxwUTJOa0V2eWVvN1BXSFYzbFBKT1Vh?=
 =?utf-8?B?YmROdFU5ckZOdVFQZm5XdUxkT1gzZy9EYXhvUndmbWpqakxJajRRRjB1T0Jv?=
 =?utf-8?B?anRCc2V1MTdRRFZxVGlQQ2REVUFvZXRBZFAzaFhrVzBCZDArWEgzNFR0QnVo?=
 =?utf-8?B?R1pWN29TOEZkOTZ1TWllN1BZb3BaOGJVdkxJMFUzVVZvWjNXUkg4Zzdjem1H?=
 =?utf-8?B?NFN2MklJRDlLMzgzYVVhdHQ3SzJnT1dCWXMzWlA5aFpPb3BxdFdjRGZvZUU5?=
 =?utf-8?B?RlY0QXRBUlNmMkNPOGJkQUNJak5CaW4vcXRtZEFabU5WSUhiQ3hkUzJUT2Yz?=
 =?utf-8?Q?mYSc0eYp7cAO2irC3clRhuqCH9a0RH46kJ9jiB6?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 772b6d68-1899-4ec5-8360-08d982864fde
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 13:46:02.7776
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BDQVU6cPCm8+VCjUw4uideAavfU5xtPWPMzpSsq0tsbEHPoaMIbPXEsFgbP4FZIOKcpq6j979UuNc64CYlc+eQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4846

On 28.09.2021 10:39, Anthony PERARD wrote:
> This will help prevent the CI loop from having build failures when
> `checkpolicy` isn't available when doing "randconfig" jobs.
> 
> To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> `checkpolicy` isn't available, we will actually override the config
> output with the use of KCONFIG_ALLCONFIG.
> 
> Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> even when "checkpolicy" isn't available. It also prevent the build
> system from reset the config when "checkpolicy" isn't available
> anymore. And XSM_FLASK_POLICY is still selected automatically when
> `checkpolicy` is available.
> But this also work well for "randconfig", as it will not select
> XSM_FLASK_POLICY when "checkpolicy" is missing.
> 
> This patch allows to easily add more override which depends on the
> environment.
> 
> Also, move the check out of Config.mk and into xen/ build system.
> Nothing in tools/ is using that information as it's done by
> ./configure.
> 
> We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> via .gitignore.
> 
> Remove '= y' in Kconfig as it isn't needed, only a value "y" is true,
> anything else is considered false.

Seeing you say this explicitly makes me wonder - is this actually true?
At least when modules are enabled (which our kconfig is capable of even
if we don't use that part of it), "m" is also "kind of" true, and the
related logic really isn't quite boolean iirc.

Everything else looks goot to me now, thanks.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 14:34:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 14:34:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198032.351327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEBe-0003n6-Tr; Tue, 28 Sep 2021 14:34:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198032.351327; Tue, 28 Sep 2021 14:34:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEBe-0003mz-Qb; Tue, 28 Sep 2021 14:34:26 +0000
Received: by outflank-mailman (input) for mailman id 198032;
 Tue, 28 Sep 2021 14:34:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVEBc-0003mt-T6
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 14:34:25 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2c224ebe-2069-11ec-bc9b-12813bfff9fa;
 Tue, 28 Sep 2021 14:34:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c224ebe-2069-11ec-bc9b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632839663;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=f/M1jQ2L65vBY+5NOTnXuTSK69Cn7Gv109tQtoDyCZQ=;
  b=e+6osednOVPo8jlFrpPzUjR3fQqNh+7PMG5tfNRGVj2hpniQ3PcHOCMD
   97sJy1Fko0r4gnA1R8kAFhPWcsjpaGBvyOl/jdDID67iE9X4GTI9zw0w6
   BfyolqR1BDOc+gQMBVWmsXsVwpX9VhKWiEnuRpkMpzFKh3HOX3i8WdH1H
   s=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: XHx0p4GWcpWl7gaTST2+Tdfmmeveplcbwb0/+o1oFrOlW7eY9ICW7FW6/NbSqGU67QBFwgtt1s
 Bc9x71w8tSWoUxnm+d/DiKItlUkqLa+3bDefJl+7iqpdqhKiA6kPFeKJ9nJ2utWG/BuXvGHKE6
 g6z8FVfwkWjrteaEBeumYeRcBJsTKN4iafMp06l2n3UvFDLsLIxrKt3Q+b1lVy/VGpLSRsdO/g
 agbUfBIQodtNprQfHbutS4/vCVYQYT1Fcf442McHMMv5ktZUMTgx6tQKSMbloJZnxiF800uwQK
 2KyqUcOSRc/VAa6e47rLX3NO
X-SBRS: 5.1
X-MesageID: 55646089
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kTn6O6xJGVr7CMI7vdp6t+e1wSrEfRIJ4+MujC+fZmUNrF6WrkUFx
 2ZKC2uAbKyOYmf2Lt4jaNi09h9U6JLTzYRgSAI4qSAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrRp2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt+5V7
 tpQrcKpdSIoP4aSxucHSkRcLAgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVors0lMMnsOpJZonx6xCvVJf0nXYrCU+PB4towMDIY250QQaiGN
 5txhTxHdCTSfR9qYVQuBdEyt+X4pXDSUHpWpwfAzUYwyzeKl1EguFT3C/LOYcCDT8hRmkeep
 0rF8n7/DxVcM8aQoRKa9lq8i+mJmjn0MKoCGbv9+vN0jVm7wm0IFAZQRVa9ueO+iEO1R5RYM
 UN80igkoLU29UerZsLgRBD+q3mB1jYdRNh4A+A88BuKyKff/0CeHGdsZjxcbN0rsucmSDps0
 UWG9+4FHhQ27ufTEyjEsO7J83XiYkD5MFPuewc6aS0K/sG9ub0WjxPkEIhgCrfoiuLcTGSYL
 y+xkAAygLAajMgu3qq9/Ezajz/EmqUlXjLZ9S2NADv6v1oRiJqNItzwsAeHvZ6sOa7DFjG8U
 G44d99yBQzkJb+KjjDFZOwQELyz6/+BPVUwanY0RMJ5plxBF5O5FL28AQ2Sxm81bq7omhezO
 Sc/XD+9ArcIZxNGioctP+qM5zwCl/SIKDgcfqm8giBySpZwbhSb2ypleFSd2Wvg+GB1z/plZ
 snLK5/2US9BYUiC8NZQb7xGuVPM7npirV4/uLihl0j3uVZgTCT9pUg53KumMblisfLsTPT9+
 NdDLcqaoyizo8WkChQ7BbU7dAhQRVBiXMieg5UOKoarf1o3cEl8WqS56e5wJORYc1F9y76gE
 oeVARQDljISRBTvdG23V5yUQOm0AMkj/S5lZ3xE0JTB8yFLXLtDJZw3L/MfFYTLPsQzpRKtZ
 /VaKciGHNpVTTHLp2YUYZXn9dQwfxW3nwOeeSGiZWFnLZJnQgXI/P7ifxfuq3ZSXnbm65Nmr
 u3yzB7fTLoCWx9mUJTcZsWwwg7jpnMagu9zARfFe4EBZEX2/YF2ACXtlftrcdoUIBDOy2LCh
 QabCBsVv8fXpIox/IWbjKyItd7xQeB/AlBbDy/Q6rPvbXvW+W+qwIlhVueUfG+CCDOoqfv6P
 egMlqPyKvwKmlpOorFQKbczwPJs/cbrqp9b0h9gQCfBYWO0B+4yOXKBx8RO6PFAn+cLpQusV
 0uT0dBGIrHVatj9GVscKQd5POSO0fYYxmvb4fgveRio4SZ2+PyMUFlIPgnKgytYdeMnPIQgy
 OYnmcgX9w3g1UZ6bofY1nhZpzaWM3gNc6Q7rZVLUobkhz0ixkxGfZGBWDT954uCaokUP0QnS
 tNOaHEuW1iIKpL+TkcO
IronPort-HdrOrdr: A9a23:Ix1i1q4JSmKmWkK5QgPXwMjXdLJyesId70hD6qhwISY6TiW9rb
 HLoB17726QtN9/YhwdcLy7VJVoBEmskqKdgrNhX4tKPjOHhILAFugLhuHfKn/bak7DH4ZmpM
 FdmsNFaeEYY2IUsfrH
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="55646089"
Date: Tue, 28 Sep 2021 15:33:57 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v4] xen: rework `checkpolicy` detection when using
 "randconfig"
Message-ID: <YVMn1TW8mKpAwImA@perard>
References: <20210928083944.780398-1-anthony.perard@citrix.com>
 <bdd68129-0415-2018-c0bf-98687094dab4@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <bdd68129-0415-2018-c0bf-98687094dab4@suse.com>

On Tue, Sep 28, 2021 at 03:46:01PM +0200, Jan Beulich wrote:
> On 28.09.2021 10:39, Anthony PERARD wrote:
> > This will help prevent the CI loop from having build failures when
> > `checkpolicy` isn't available when doing "randconfig" jobs.
> > 
> > To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> > `checkpolicy` isn't available, we will actually override the config
> > output with the use of KCONFIG_ALLCONFIG.
> > 
> > Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> > even when "checkpolicy" isn't available. It also prevent the build
> > system from reset the config when "checkpolicy" isn't available
> > anymore. And XSM_FLASK_POLICY is still selected automatically when
> > `checkpolicy` is available.
> > But this also work well for "randconfig", as it will not select
> > XSM_FLASK_POLICY when "checkpolicy" is missing.
> > 
> > This patch allows to easily add more override which depends on the
> > environment.
> > 
> > Also, move the check out of Config.mk and into xen/ build system.
> > Nothing in tools/ is using that information as it's done by
> > ./configure.
> > 
> > We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> > via .gitignore.
> > 
> > Remove '= y' in Kconfig as it isn't needed, only a value "y" is true,
> > anything else is considered false.
> 
> Seeing you say this explicitly makes me wonder - is this actually true?

I've check that this was true by empirical testing before sending the
patch. But the documentation isn't clear to me about the meaning of
'default y if "m"'. So would you rather keep '= y' just to stay on the
safe side?

> At least when modules are enabled (which our kconfig is capable of even
> if we don't use that part of it), "m" is also "kind of" true, and the
> related logic really isn't quite boolean iirc.
> 
> Everything else looks goot to me now, thanks.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 14:39:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 14:39:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198039.351337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEGb-0004f3-K4; Tue, 28 Sep 2021 14:39:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198039.351337; Tue, 28 Sep 2021 14:39:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEGb-0004ew-H0; Tue, 28 Sep 2021 14:39:33 +0000
Received: by outflank-mailman (input) for mailman id 198039;
 Tue, 28 Sep 2021 14:39:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7uFz=OS=epam.com=prvs=0905849383=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVEGa-0004eq-3Y
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 14:39:32 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 80f22a12-8db6-436c-a9bd-de5658d03f52;
 Tue, 28 Sep 2021 14:39:29 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18SEcMHH008776; 
 Tue, 28 Sep 2021 14:39:27 GMT
Received: from eur02-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bc5038035-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 28 Sep 2021 14:39:26 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6881.eurprd03.prod.outlook.com (2603:10a6:20b:286::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 14:39:23 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::70f5:8ba9:da74:8994%4]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 14:39:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80f22a12-8db6-436c-a9bd-de5658d03f52
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Nj57B2r/SiHmTbTdUVusLotx/dS1Plz+e1ZZ8kr8IDVNFhqIuv1hIxmfkaDmh8a2HamwD9bmSv5i8WeUM/G7cxuIhvKEg0abu1NKYCTfGS55iV0AgZNoT0zYMm2f5dC1ozK5QbNLTuzWaliKqBxltepMQXUQjCSml3H/o8efZwYa46AVPRTdfpJjVvQjYiAMm5S+MbUZAp9VMy1oHLddAi5aXEKOVWINCpKoo3KY7pt0kL1JvRM8vf0BvB7xNzGBnf+UILZJy4f9rmhm41tTTFpwm4QzNM5MsP1o4tpXxmgZM1hw+xIRaK5Eo9ODyVU/EToj+fe75lFcy+HWt8GpwA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=F6bXOgWeU7l/tbE9ma2lBVAGN0HLK9OFOght1cUjvqk=;
 b=Ycnqq3o72j2/kvsa3GiEp+wPSVvJpDpycF8j9u36n39UkB9WAcPB7i3jnUYX544Hd5HMPHTQfjP1UaHGshl2S+WqamFK/Hx/sSiM/Y+35cPZKXnvExYmjR94sD8a9FFof3hadKvqzmDyX9aTWBDlkezKmZ0c481IE0OMS5R/kiC1+ofCWNGjzgdWC+oO3dCrWb++JgQSQm59gH6LFyDWtRxWR1D1Q10TcmOXheaW8WBbY2weLWMkjGkroRb+gIjhQVERDZ9wpJSGzj+P6/COydkghoNgBbegFcuL2drJQICkQXjjsvxg9Bf1ENCj/Ak1QwuHp0Rx0eH3WRqPnHmt5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=F6bXOgWeU7l/tbE9ma2lBVAGN0HLK9OFOght1cUjvqk=;
 b=tJijmeoYOaG8iEKEb9Kfnz9Q5RBcAxEU84ykA7Ylqsqai8il+MmfzsND1munOqgryGfmUY9Weqz/NSsyUSRJGV47o2Jo1Wf2mLTWv9Ym3OQPKwPd+rZLfxTqCKdCP1MNL+1z4K5ESr9iD7fISOGX0yz1SytI/ccRZCeRGRDjNt33bnO0TFr3WVKwEbW39B2XzY4g+gPpl79h1SQqFoSrD4eOER5goPs6ZkBVrKwRHgLn3UnSKCIKwk2AVAR7JfUWH91ZKlW2FrDTxh2QjBo93s280QuSxITJlIs9j0RQB/xou0CfCiqt2+oFQBOfvKrTIsmDNh+uzee0pl0tcy1hZw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "julien@xen.org" <julien@xen.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Topic: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
Thread-Index: AQHXsHo2GZQHQL3fu0+Ql9qa7dzr/Kuz6/8AgAPt2ICAAQAHgIAADkyAgACkIgA=
Date: Tue, 28 Sep 2021 14:39:23 +0000
Message-ID: <32ffee9e-8786-c91b-681c-7fa243f5d3fa@epam.com>
References: <20210923125438.234162-1-andr2000@gmail.com>
 <20210923125438.234162-11-andr2000@gmail.com>
 <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s>
 <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com>
 <alpine.DEB.2.21.2109272046320.5022@sstabellini-ThinkPad-T480s>
 <cc05efe3-2b3a-be03-2c17-21578b09a013@epam.com>
In-Reply-To: <cc05efe3-2b3a-be03-2c17-21578b09a013@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
authentication-results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7dd16646-4e1f-4139-a584-08d9828dc3b2
x-ms-traffictypediagnostic: AM9PR03MB6881:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB68816FFFD0F3BAB99190C7B8E7A89@AM9PR03MB6881.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3968;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ZIlvVAIpCjMd4skX3T3dGmQIMG+WLL/6EjZLuM/QG0VFgUmZgnDdNzZeayDUusHBZ6F29W3hMkwIgCnjp6cN+o8zCeQqLadTv//GOIQFRXfRLCopbMujxybNP213It6xNfs79RJEzEaq1TPMnL2caYu6JQbXD1VlFwcbQxRB2BzHv6f0/DhZbNdVzghVUIZEQVgXxgBdoPKhkoxL1TF0DfGXaWsQroTiSYnAB6T0uVjJzAGOD0XOof9VEBGBOYevRyd/LqYT4Lja5R/n8frTHa9eSSCl6ZvV/KIBfEmUKGPbetvvNrjpvE+Os5R0WPfyaYgF29YC5yhkQRGl8+gXTlPuMGXEkO1ciArZLt+Os2mqjqeKiNnDIN9NKMxB1Z2PgTMn2e58COqR1XumqDQcZB9M5Sfo0GID4NB8/v1DMXrR/kMbWMqFckayR2sLyBox/K+5lDKL2oM9Npn2gDejqF6zbA1jc1vDs7fyTp6yc4kmfd43jEG6xwSFkXBfLtRUIiwXN9dKU+QT7ALlmANW2ewW2cr8KVwtZPksMn/3gg4aIwPXY9uBFqAU8ppQ0MpfY4w7RW72hN/hLDlIa0GWhRl0ndcBxG5mB4U3pyyuvKkDfFUFqNeTsrSvz53vmdKdhw78dV1ILpyryCS+vzh4bKnD9dJOxC9k+ESVPMS1N/SPyD7oz2UsqssZM6kVAtROZJe7uxGT8t7moU/T9Clae9ki8+rh4AUnBiD+7B2K8Csc4w0mC4JAxjQR8QaVUzxI
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(55236004)(8676002)(26005)(6916009)(38070700005)(6512007)(64756008)(66446008)(5660300002)(66616009)(66556008)(66476007)(66946007)(91956017)(6486002)(186003)(31696002)(71200400001)(107886003)(6506007)(86362001)(2616005)(508600001)(36756003)(76116006)(122000001)(99936003)(83380400001)(38100700002)(31686004)(54906003)(4326008)(2906002)(316002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?OXJacTkxTnNXQVpHZU4xTWpWL1JWWDRheTNwWE9NbkVwdkpXdnlSWisvZW9P?=
 =?utf-8?B?aHVaVEorNWZrRVBjTVV5N2loUnJtckVKRDJjSmNzY0NWdWcwS283L2NKOFY5?=
 =?utf-8?B?M0htM0IwTERVdCtSWjNPVmsyS0tvc3NqVE40eTlQNFhUYUx0MnI2MEJXZ3dQ?=
 =?utf-8?B?VS9hdVcvUXhOMkVKdkZ4eEhWeEJTOVhkTHViSG5ReUdBTWZyeExBYTZ1ZVZV?=
 =?utf-8?B?L0pIdHNudUU0MFM2RUptNWxCdWdMR0x4TWJDWGdMQUF0TWYzYkM4MUdUMm1y?=
 =?utf-8?B?NENtbTQ5R0psMUVsa29LVXJ0bjNLb2dyWjlrY2UzdXMvb0RTUFNVREN1QWRT?=
 =?utf-8?B?di9NMGpRdjlFc0tkY0xHTy9xNUFmZXRjQTJLN3J5M3RlSDdTMm5aMkN4S3BD?=
 =?utf-8?B?NTFjdzhqOHVxd01nSnoyUFVnZTJ4dmY2VnRrN3ZlUUh4allGQzlCSFpZSFZT?=
 =?utf-8?B?c3dYNGlEU1pwOHFLdjRXK0N1ZkYyOE1EZDRhaC83QXJXOVFmY2l4a01JSnA1?=
 =?utf-8?B?elM4YzVLZXJNVlZvYmxhd1h2VWZPZ2MwTEt0YlBXbU40dldyMjA1ZmVOWkNM?=
 =?utf-8?B?c2lZaGt3eFZnOCt1TE51SDVIRlNJOTZYem00MXA5dVc0Mmk3bDQ5WW5RbDNs?=
 =?utf-8?B?ZjlLcTN1OE9UbzJIR1pDSDF1TTJzQ1RtVDVzU2Z0VU9rK1RhRjU5WlAxMWR4?=
 =?utf-8?B?dVVGNHZ2Wnp5Z3RrU1FZMmJMMm5lcW9vSVdRMjhmVjlqTlF3aHY1bEVzODVu?=
 =?utf-8?B?REhJSUFLMEJ3MGZiOW9VczVqVmV1d1VTZWJMMUcvL1MzSE95b0NUWlZYTXBl?=
 =?utf-8?B?L0F4akwraGh4YnFBVlhjc0VRUmdYN01IbHhuaW0vTy9pR3EycUVUNis0ZVMr?=
 =?utf-8?B?Yi9WN091ZXBGSWttVDVxaDJZYTNPREh2cjZtMWFxZHdiczYrREFTNW5CY090?=
 =?utf-8?B?Z3JNM0pibHg3Z3lmamlyVHE2aU1lM3VtZmFRRzRyMy91QVRWc05pQzdYTXhx?=
 =?utf-8?B?RDY3TUlFV2NSb1MreklZRUhBSGF0N0p4U0JTUEszVlE5NmhET3VWSEJJSlRq?=
 =?utf-8?B?cmoxaHdmYmcxY2JhaHljZzlCL1BOaTNhSGFaa3hjSGRXbHZxUk4yMCtvaXZS?=
 =?utf-8?B?aGU0ZGFaeXk5aDZ3MVZEOExjaFhEeHlpOU9GNnlCSmpVMUdHZUN5WHQwRTcx?=
 =?utf-8?B?MVBpaHFSL294NmsvTFpoN0Fjd0RtVFlSZzNZZGZTdUZmWFdFdjU0MTJlYURP?=
 =?utf-8?B?YXJ0VTFQeW5uWjJFZ3I5UGNpZWx3ZkhwK1N1VXhZcWFNUnVUNGZCby9TUUx6?=
 =?utf-8?B?amp3TjFwZWsrVnl2dlAyWEtTRzdRQmRzWmxQU0ZOL3M0Vkh4RXplZDRscTNC?=
 =?utf-8?B?cm51MFhxWWhSUGduM1RUWHlDY1U5UTJBQTlzanVFbmNqaExnaEJXcmZydDlv?=
 =?utf-8?B?enN5Vk4wVWFCWSsxZkx6L1oyZmxkcUxIQ1c4QWQwTWRMRVJFZkVtSjNkKzZD?=
 =?utf-8?B?WElBeHN0STRiU0Y0c2p0bDRVbkNtZHJZYWV0bThnSER6Ty9iRU56VTkxZ296?=
 =?utf-8?B?Uzg1M2Z5R0REMy9FVUpRVmNpelJmN0ZOdGV1aWNUUk5SN1owMk5vdTd5L2gz?=
 =?utf-8?B?bUw3THlTc0pxaExRS0tjVUlrZnhNOUM2eEkvZE5qVkdYNXRYQlBYZW5PMWdn?=
 =?utf-8?B?WGVsK2s2ZHowcS8rWkM3YkV0cGxnQ3Z2dTlSTzJxelZoR3dGN1dNdU85azFz?=
 =?utf-8?B?aTV3aVhFZTZneG50SjhlV2wwTjBZQjg0WkoycHFXYlA2UzlyV3F4cXhLam5m?=
 =?utf-8?B?ZThBR3dHMllsYS9BckovQT09?=
Content-Type: multipart/mixed;
	boundary="_002_32ffee9e8786c91b681c7fa243f5d3faepamcom_"
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7dd16646-4e1f-4139-a584-08d9828dc3b2
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Sep 2021 14:39:23.3327
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: xP9Mk55oTwMC1yVVNviwb3I4SKU4DgdvNmf/kU441zB0EyIALfQhKWcuRe44cDhBSC8o9zxHSJnisaRNGXG4EO1df8O9AqLmMDwpfV3q0tC7ExXHoylUgxniTd0rdcDo
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6881
X-Proofpoint-ORIG-GUID: ll-IMDOiphHiKkltRy--Nr2CvN8LyvZG
X-Proofpoint-GUID: ll-IMDOiphHiKkltRy--Nr2CvN8LyvZG
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-28_05,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0
 spamscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0
 adultscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 mlxscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109280084

--_002_32ffee9e8786c91b681c7fa243f5d3faepamcom_
Content-Type: text/plain; charset="utf-8"
Content-ID: <F8F529FE771DF948A17516EB7B8EFD97@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64

W3NuaXBdDQo+PiBTb3JyeSBJIGRpZG4ndCBmb2xsb3cgeW91ciBleHBsYW5hdGlvbi4NCj4+DQo+
PiBNeSBzdWdnZXN0aW9uIGlzIHRvIHJlbW92ZSB0aGUgI2lmZGVmIENPTkZJR19IQVNfUENJIGNv
bXBsZXRlbHkgZnJvbQ0KPj4gbWFwX3JhbmdlX3RvX2RvbWFpbi4gQXQgdGhlIGJlZ2lubmluZyBv
ZiBtYXBfcmFuZ2VfdG9fZG9tYWluLCB0aGVyZSBpcw0KPj4gYWxyZWFkeSB0aGlzIGxpbmU6DQo+
Pg0KPj4gYm9vbCBuZWVkX21hcHBpbmcgPSAhZHRfZGV2aWNlX2Zvcl9wYXNzdGhyb3VnaChkZXYp
Ow0KPj4NCj4+IFdlIGNhbiBjaGFuZ2UgaXQgaW50bzoNCj4+DQo+PiBib29sIG5lZWRfbWFwcGlu
ZyA9ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJvdWdoKGRldikgJiYNCj4+ICAgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgIW1yX2RhdGEtPnNraXBfbWFwcGluZzsNCj4+DQo+
Pg0KPj4gVGhlbiwgaW4gbWFwX2RldmljZV9jaGlsZHJlbiB3ZSBjYW4gc2V0IG1yX2RhdGEtPnNr
aXBfbWFwcGluZyB0byB0cnVlDQo+PiBmb3IgUENJIGRldmljZXMuDQo+IFRoaXMgaXMgdGhlIGtl
eS4gSSBhbSBmaW5lIHdpdGggdGhpcywgYnV0IGl0IGp1c3QgbWVhbnMgd2UgbW92ZSB0aGUNCj4N
Cj4gY2hlY2sgdG8gdGhlIG91dHNpZGUgb2YgdGhpcyBmdW5jdGlvbiB3aGljaCBsb29rcyBnb29k
LiBXaWxsIGRvDQo+DQo+PiAgICBUaGVyZSBpcyBhbHJlYWR5IGEgcGNpIGNoZWNrIHRoZXJlOg0K
Pj4NCj4+ICAgIGlmICggZHRfZGV2aWNlX3R5cGVfaXNfZXF1YWwoZGV2LCAicGNpIikgKQ0KPj4N
Cj4+IHNvIGl0IHNob3VsZCBiZSBlYXN5IHRvIGRvLiBXaGF0IGFtIEkgbWlzc2luZz8NCj4+DQo+
Pg0KPg0KSSBkaWQgc29tZSBleHBlcmltZW50cy4gSWYgd2UgbW92ZSB0aGUgY2hlY2sgdG8gbWFw
X2RldmljZV9jaGlsZHJlbg0KDQppdCBpcyBub3QgZW5vdWdoIGJlY2F1c2UgcGFydCBvZiB0aGUg
cmFuZ2VzIGlzIHN0aWxsIG1hcHBlZCBhdCBoYW5kbGVfZGV2aWNlIGxldmVsOg0KDQpoYW5kbGVf
ZGV2aWNlOg0KKFhFTikgLS0tIC9heGkvcGNpZUBmZDBlMDAwMCBuZWVkX21hcHBpbmcgMSBhZGRy
IGZkMGUwMDAwDQooWEVOKSAtLS0gL2F4aS9wY2llQGZkMGUwMDAwIG5lZWRfbWFwcGluZyAxIGFk
ZHIgZmQ0ODAwMDANCihYRU4pIC0tLSAvYXhpL3BjaWVAZmQwZTAwMDAgbmVlZF9tYXBwaW5nIDEg
YWRkciA4MDAwMDAwMDAwDQoNCm1hcF9kZXZpY2VfY2hpbGRyZW46DQooWEVOKSBNYXBwaW5nIGNo
aWxkcmVuIG9mIC9heGkvcGNpZUBmZDBlMDAwMCB0byBndWVzdCBza2lwIDENCihYRU4pIC0tLSAv
YXhpL3BjaWVAZmQwZTAwMDAgbmVlZF9tYXBwaW5nIDAgYWRkciBlMDAwMDAwMA0KKFhFTikgLS0t
IC9heGkvcGNpZUBmZDBlMDAwMCBuZWVkX21hcHBpbmcgMCBhZGRyIDYwMDAwMDAwMA0KDQpwY2lf
aG9zdF9icmlkZ2VfbWFwcGluZ3M6DQooWEVOKSAtLS0gL2F4aS9wY2llQGZkMGUwMDAwIG5lZWRf
bWFwcGluZyAxIGFkZHIgZmQwZTAwMDANCihYRU4pIC0tLSAvYXhpL3BjaWVAZmQwZTAwMDAgbmVl
ZF9tYXBwaW5nIDEgYWRkciBmZDQ4MDAwMA0KDQpTbywgSSBkaWQgbW9yZSBpbnRydXNpdmUgY2hh
bmdlOg0KDQpAQCAtMTU0MCw2ICsxNTM0LDEyIEBAIHN0YXRpYyBpbnQgX19pbml0IGhhbmRsZV9k
ZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQogwqDC
oMKgwqAgaW50IHJlczsNCiDCoMKgwqDCoCB1NjQgYWRkciwgc2l6ZTsNCiDCoMKgwqDCoCBib29s
IG5lZWRfbWFwcGluZyA9ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJvdWdoKGRldik7DQorwqDCoMKg
IHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSBtcl9kYXRhID0gew0KK8KgwqDCoMKgwqDCoMKgIC5kID0g
ZCwNCivCoMKgwqDCoMKgwqDCoCAucDJtdCA9IHAybXQsDQorwqDCoMKgwqDCoMKgwqAgLnNraXBf
bWFwcGluZyA9IGlzX3BjaV9wYXNzdGhyb3VnaF9lbmFibGVkKCkgJiYNCivCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIChkZXZpY2VfZ2V0X2NsYXNzKGRldikg
PT0gREVWSUNFX1BDSSkNCivCoMKgwqAgfTsNCg0KV2l0aCB0aGlzIEkgc2VlIHRoYXQgbm93IG1h
cHBpbmdzIGFyZSBkb25lIGNvcnJlY3RseToNCg0KaGFuZGxlX2RldmljZToNCihYRU4pIC0tLSAv
YXhpL3BjaWVAZmQwZTAwMDAgbmVlZF9tYXBwaW5nIDAgYWRkciBmZDBlMDAwMA0KKFhFTikgLS0t
IC9heGkvcGNpZUBmZDBlMDAwMCBuZWVkX21hcHBpbmcgMCBhZGRyIGZkNDgwMDAwDQooWEVOKSAt
LS0gL2F4aS9wY2llQGZkMGUwMDAwIG5lZWRfbWFwcGluZyAwIGFkZHIgODAwMDAwMDAwMA0KDQpt
YXBfZGV2aWNlX2NoaWxkcmVuOg0KKFhFTikgTWFwcGluZyBjaGlsZHJlbiBvZiAvYXhpL3BjaWVA
ZmQwZTAwMDAgdG8gZ3Vlc3Qgc2tpcCAxDQooWEVOKSAtLS0gL2F4aS9wY2llQGZkMGUwMDAwIG5l
ZWRfbWFwcGluZyAwIGFkZHIgZTAwMDAwMDANCihYRU4pIC0tLSAvYXhpL3BjaWVAZmQwZTAwMDAg
bmVlZF9tYXBwaW5nIDAgYWRkciA2MDAwMDAwMDANCg0KcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdz
Og0KKFhFTikgLS0tIC9heGkvcGNpZUBmZDBlMDAwMCBuZWVkX21hcHBpbmcgMSBhZGRyIGZkMGUw
MDAwDQooWEVOKSAtLS0gL2F4aS9wY2llQGZkMGUwMDAwIG5lZWRfbWFwcGluZyAxIGFkZHIgZmQ0
ODAwMDANCg0KU28sIGhhbmRsZV9kZXZpY2Ugc2VlbXMgdG8gYmUgdGhlIHJpZ2h0IHBsYWNlLiBX
aGlsZSBhdCBpdCBJIGhhdmUgYWxzbw0KDQpvcHRpbWl6ZWQgdGhlIHdheSB3ZSBzZXR1cCBzdHJ1
Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSBpbiBib3RoDQoNCmhhbmRsZV9kZXZpY2UgYW5kIG1h
cF9kZXZpY2VfY2hpbGRyZW46IEkgcmVtb3ZlZCBzdHJ1Y3R1cmUgaW5pdGlhbGl6YXRpb24NCg0K
ZnJvbSB3aXRoaW4gdGhlIHJlbGV2YW50IGxvb3AgYW5kIGFsc28gcGFzcyBtcl9kYXRhIHRvIG1h
cF9kZXZpY2VfY2hpbGRyZW4sDQoNCnNvIGl0IGRvZXNuJ3QgbmVlZCB0byBjcmVhdGUgaXRzIG93
biBjb3B5IG9mIHRoZSBzYW1lIGFuZCBwZXJmb3JtIHlldA0KDQphbm90aGVyIGNvbXB1dGF0aW9u
IGZvciAuc2tpcF9tYXBwaW5nOiBpdCBkb2VzIG5lZWQgdG8gbm90IG9ubHkga25vdw0KDQp0aGF0
IGRldiBpcyBhIFBDSSBkZXZpY2UgKHRoaXMgaXMgZG9uZSBieSB0aGUgZHRfZGV2aWNlX3R5cGVf
aXNfZXF1YWwoZGV2LCAicGNpIikNCg0KY2hlY2ssIGJ1dCBhbHNvIGFjY291bnQgb24gaXNfcGNp
X3Bhc3N0aHJvdWdoX2VuYWJsZWQoKS4NCg0KVGh1cywgdGhlIGNoYW5nZSB3aWxsIGJlIG1vcmUg
aW50cnVzaXZlLCBidXQgSSBob3BlIHdpbGwgc2ltcGxpZnkgdGhpbmdzLg0KDQpJIGFtIGF0dGFj
aGluZyB0aGUgZml4dXAgcGF0Y2ggZm9yIGp1c3QgaW4gY2FzZSB5b3Ugd2FudCBtb3JlIGRldGFp
bHMuDQoNClRoYW5rIHlvdSwNCg0KT2xla3NhbmRyDQoNCg0K

--_002_32ffee9e8786c91b681c7fa243f5d3faepamcom_
Content-Type: text/x-patch;
	name="0001-Fixes-4480fb1a5c83-xen-arm-Do-not-map-PCI-ECAM-and-M.patch"
Content-Description: 
 0001-Fixes-4480fb1a5c83-xen-arm-Do-not-map-PCI-ECAM-and-M.patch
Content-Disposition: attachment;
	filename="0001-Fixes-4480fb1a5c83-xen-arm-Do-not-map-PCI-ECAM-and-M.patch";
	size=10687; creation-date="Tue, 28 Sep 2021 14:39:23 GMT";
	modification-date="Tue, 28 Sep 2021 14:39:23 GMT"
Content-ID: <468C227AD385A7409DE1A325504F6E84@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64

RnJvbSAwN2Q2NTIzYmUyNTM1MjkzZDNlMzRmZmQxYzg1MDhhMDgxMmE0Y2Q4IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQ0KRnJvbTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9h
bmRydXNoY2hlbmtvQGVwYW0uY29tPg0KRGF0ZTogVHVlLCAyOCBTZXAgMjAyMSAxMzoyNDo0MiAr
MDMwMA0KU3ViamVjdDogW1BBVENIXSBGaXhlczogNDQ4MGZiMWE1YzgzICgieGVuL2FybTogRG8g
bm90IG1hcCBQQ0kgRUNBTSBhbmQgTU1JTw0KIHNwYWNlIHRvIERvbWFpbi0wJ3MgcDJtIikNCg0K
U2luY2UgdjI6DQogLSByZW1vdmVkIGNoZWNrIGluIG1hcF9yYW5nZV90b19kb21haW4gZm9yIFBD
SV9ERVYNCiAgIGFuZCBtb3ZlZCBpdCB0byBoYW5kbGVfZGV2aWNlLCBzbyB0aGUgY29kZSBpcw0K
ICAgc2ltcGxlcg0KIC0gcy9tYXBfcGNpX2JyaWRnZS9za2lwX21hcHBpbmcNCiAtIGV4dGVuZGVk
IGNvbW1lbnQgaW4gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzDQogLSBtaW5vciBjb2RlIHJlc3Ry
dWN0dXJlIGluIGNvbnN0cnVjdF9kb20wDQogLSBzLy5uZWVkX3AybV9tYXBwaW5nLy5uZWVkX3Ay
bV9od2RvbV9tYXBwaW5nIGFuZCByZWxhdGVkDQogICBjYWxsYmFja3MNCiAtIHVuc2lnbmVkIGlu
dCBpOyBpbiBwY2lfaG9zdF9icmlkZ2VfbWFwcGluZ3MNCg0KU2lnbmVkLW9mZi1ieTogT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRydXNoY2hlbmtvQGVwYW0uY29tPg0KLS0t
DQogeGVuL2FyY2gvYXJtL2RvbWFpbl9idWlsZC5jICAgICAgICB8IDQzICsrKysrKysrKysrLS0t
LS0tLS0tLS0tLS0tLS0tLQ0KIHhlbi9hcmNoL2FybS9wY2kvZWNhbS5jICAgICAgICAgICAgfCAg
OCArKystLS0NCiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LWNvbW1vbi5jIHwgMTUgKysrKysr
LS0tLS0NCiB4ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jIHwgIDIgKy0NCiB4ZW4v
aW5jbHVkZS9hc20tYXJtL3BjaS5oICAgICAgICAgIHwgMTIgKysrKy0tLS0tDQogeGVuL2luY2x1
ZGUvYXNtLWFybS9zZXR1cC5oICAgICAgICB8ICAyICstDQogNiBmaWxlcyBjaGFuZ2VkLCAzNiBp
bnNlcnRpb25zKCspLCA0NiBkZWxldGlvbnMoLSkNCg0KZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2Fy
bS9kb21haW5fYnVpbGQuYyBiL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KaW5kZXggZTcy
YzFiODgxY2FlLi4xN2YzZGI2YTFmNDggMTAwNjQ0DQotLS0gYS94ZW4vYXJjaC9hcm0vZG9tYWlu
X2J1aWxkLmMNCisrKyBiL3hlbi9hcmNoL2FybS9kb21haW5fYnVpbGQuYw0KQEAgLTEzODYsNyAr
MTM4Niw4IEBAIGludCBfX2luaXQgbWFwX3JhbmdlX3RvX2RvbWFpbihjb25zdCBzdHJ1Y3QgZHRf
ZGV2aWNlX25vZGUgKmRldiwNCiB7DQogICAgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSAqbXJfZGF0
YSA9IGRhdGE7DQogICAgIHN0cnVjdCBkb21haW4gKmQgPSBtcl9kYXRhLT5kOw0KLSAgICBib29s
IG5lZWRfbWFwcGluZyA9ICFkdF9kZXZpY2VfZm9yX3Bhc3N0aHJvdWdoKGRldik7DQorICAgIGJv
b2wgbmVlZF9tYXBwaW5nID0gIWR0X2RldmljZV9mb3JfcGFzc3Rocm91Z2goZGV2KSAmJg0KKyAg
ICAgICAgICAgICAgICAgICAgICAgICFtcl9kYXRhLT5za2lwX21hcHBpbmc7DQogICAgIGludCBy
ZXM7DQogDQogICAgIC8qDQpAQCAtMTQwOSwxMyArMTQxMCw2IEBAIGludCBfX2luaXQgbWFwX3Jh
bmdlX3RvX2RvbWFpbihjb25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCiAgICAgICAg
IH0NCiAgICAgfQ0KIA0KLSNpZmRlZiBDT05GSUdfSEFTX1BDSQ0KLSAgICBpZiAoIGlzX3BjaV9w
YXNzdGhyb3VnaF9lbmFibGVkKCkgJiYNCi0gICAgICAgICAoZGV2aWNlX2dldF9jbGFzcyhkZXYp
ID09IERFVklDRV9QQ0kpICYmDQotICAgICAgICAgIW1yX2RhdGEtPm1hcF9wY2lfYnJpZGdlICkN
Ci0gICAgICAgIG5lZWRfbWFwcGluZyA9IGZhbHNlOw0KLSNlbmRpZg0KLQ0KICAgICBpZiAoIG5l
ZWRfbWFwcGluZyApDQogICAgIHsNCiAgICAgICAgIHJlcyA9IG1hcF9yZWdpb25zX3AybXQoZCwN
CkBAIC0xNDQ1LDI3ICsxNDM5LDIxIEBAIGludCBfX2luaXQgbWFwX3JhbmdlX3RvX2RvbWFpbihj
b25zdCBzdHJ1Y3QgZHRfZGV2aWNlX25vZGUgKmRldiwNCiAgKiB0aGVuIHdlIG1heSBuZWVkIHRv
IHBlcmZvcm0gYWRkaXRpb25hbCBtYXBwaW5ncyBpbiBvcmRlciB0byBtYWtlDQogICogdGhlIGNo
aWxkIHJlc291cmNlcyBhdmFpbGFibGUgdG8gZG9tYWluIDAuDQogICovDQotc3RhdGljIGludCBf
X2luaXQgbWFwX2RldmljZV9jaGlsZHJlbihzdHJ1Y3QgZG9tYWluICpkLA0KLSAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpk
ZXYsDQotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwMm1fdHlwZV90IHAy
bXQpDQorc3RhdGljIGludCBfX2luaXQgbWFwX2RldmljZV9jaGlsZHJlbihjb25zdCBzdHJ1Y3Qg
ZHRfZGV2aWNlX25vZGUgKmRldiwNCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSAqbXJfZGF0YSkNCiB7DQotICAgIHN0cnVjdCBtYXBf
cmFuZ2VfZGF0YSBtcl9kYXRhID0gew0KLSAgICAgICAgLmQgPSBkLA0KLSAgICAgICAgLnAybXQg
PSBwMm10LA0KLSAgICAgICAgLm1hcF9wY2lfYnJpZGdlID0gZmFsc2UNCi0gICAgfTsNCi0gICAg
aW50IHJldDsNCi0NCiAgICAgaWYgKCBkdF9kZXZpY2VfdHlwZV9pc19lcXVhbChkZXYsICJwY2ki
KSApDQogICAgIHsNCisgICAgICAgIGludCByZXQ7DQorDQogICAgICAgICBkdF9kcHJpbnRrKCJN
YXBwaW5nIGNoaWxkcmVuIG9mICVzIHRvIGd1ZXN0XG4iLA0KICAgICAgICAgICAgICAgICAgICBk
dF9ub2RlX2Z1bGxfbmFtZShkZXYpKTsNCiANCi0gICAgICAgIHJldCA9IGR0X2Zvcl9lYWNoX2ly
cV9tYXAoZGV2LCAmbWFwX2R0X2lycV90b19kb21haW4sIGQpOw0KKyAgICAgICAgcmV0ID0gZHRf
Zm9yX2VhY2hfaXJxX21hcChkZXYsICZtYXBfZHRfaXJxX3RvX2RvbWFpbiwgbXJfZGF0YS0+ZCk7
DQogICAgICAgICBpZiAoIHJldCA8IDAgKQ0KICAgICAgICAgICAgIHJldHVybiByZXQ7DQogDQot
ICAgICAgICByZXQgPSBkdF9mb3JfZWFjaF9yYW5nZShkZXYsICZtYXBfcmFuZ2VfdG9fZG9tYWlu
LCAmbXJfZGF0YSk7DQorICAgICAgICByZXQgPSBkdF9mb3JfZWFjaF9yYW5nZShkZXYsICZtYXBf
cmFuZ2VfdG9fZG9tYWluLCBtcl9kYXRhKTsNCiAgICAgICAgIGlmICggcmV0IDwgMCApDQogICAg
ICAgICAgICAgcmV0dXJuIHJldDsNCiAgICAgfQ0KQEAgLTE1NDYsNiArMTUzNCwxMiBAQCBzdGF0
aWMgaW50IF9faW5pdCBoYW5kbGVfZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBkdF9k
ZXZpY2Vfbm9kZSAqZGV2LA0KICAgICBpbnQgcmVzOw0KICAgICB1NjQgYWRkciwgc2l6ZTsNCiAg
ICAgYm9vbCBuZWVkX21hcHBpbmcgPSAhZHRfZGV2aWNlX2Zvcl9wYXNzdGhyb3VnaChkZXYpOw0K
KyAgICBzdHJ1Y3QgbWFwX3JhbmdlX2RhdGEgbXJfZGF0YSA9IHsNCisgICAgICAgIC5kID0gZCwN
CisgICAgICAgIC5wMm10ID0gcDJtdCwNCisgICAgICAgIC5za2lwX21hcHBpbmcgPSBpc19wY2lf
cGFzc3Rocm91Z2hfZW5hYmxlZCgpICYmDQorICAgICAgICAgICAgICAgICAgICAgICAgKGRldmlj
ZV9nZXRfY2xhc3MoZGV2KSA9PSBERVZJQ0VfUENJKQ0KKyAgICB9Ow0KIA0KICAgICBuYWRkciA9
IGR0X251bWJlcl9vZl9hZGRyZXNzKGRldik7DQogDQpAQCAtMTU4NSwxMSArMTU3OSw2IEBAIHN0
YXRpYyBpbnQgX19pbml0IGhhbmRsZV9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGR0
X2RldmljZV9ub2RlICpkZXYsDQogICAgIC8qIEdpdmUgcGVybWlzc2lvbiBhbmQgbWFwIE1NSU9z
ICovDQogICAgIGZvciAoIGkgPSAwOyBpIDwgbmFkZHI7IGkrKyApDQogICAgIHsNCi0gICAgICAg
IHN0cnVjdCBtYXBfcmFuZ2VfZGF0YSBtcl9kYXRhID0gew0KLSAgICAgICAgICAgIC5kID0gZCwN
Ci0gICAgICAgICAgICAucDJtdCA9IHAybXQsDQotICAgICAgICAgICAgLm1hcF9wY2lfYnJpZGdl
ID0gZmFsc2UNCi0gICAgICAgIH07DQogICAgICAgICByZXMgPSBkdF9kZXZpY2VfZ2V0X2FkZHJl
c3MoZGV2LCBpLCAmYWRkciwgJnNpemUpOw0KICAgICAgICAgaWYgKCByZXMgKQ0KICAgICAgICAg
ew0KQEAgLTE2MDMsNyArMTU5Miw3IEBAIHN0YXRpYyBpbnQgX19pbml0IGhhbmRsZV9kZXZpY2Uo
c3RydWN0IGRvbWFpbiAqZCwgc3RydWN0IGR0X2RldmljZV9ub2RlICpkZXYsDQogICAgICAgICAg
ICAgcmV0dXJuIHJlczsNCiAgICAgfQ0KIA0KLSAgICByZXMgPSBtYXBfZGV2aWNlX2NoaWxkcmVu
KGQsIGRldiwgcDJtdCk7DQorICAgIHJlcyA9IG1hcF9kZXZpY2VfY2hpbGRyZW4oZGV2LCAmbXJf
ZGF0YSk7DQogICAgIGlmICggcmVzICkNCiAgICAgICAgIHJldHVybiByZXM7DQogDQpAQCAtMjc2
Myw5ICsyNzUyLDkgQEAgc3RhdGljIGludCBfX2luaXQgY29uc3RydWN0X2RvbTAoc3RydWN0IGRv
bWFpbiAqZCkNCiAgICAgaWYgKCBhY3BpX2Rpc2FibGVkICkNCiAgICAgew0KICAgICAgICAgcmMg
PSBwcmVwYXJlX2R0Yl9od2RvbShkLCAma2luZm8pOw0KLSNpZmRlZiBDT05GSUdfSEFTX1BDSQ0K
ICAgICAgICAgaWYgKCByYyA8IDAgKQ0KICAgICAgICAgICAgIHJldHVybiByYzsNCisjaWZkZWYg
Q09ORklHX0hBU19QQ0kNCiAgICAgICAgIHJjID0gcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKGQs
IHAybV9tbWlvX2RpcmVjdF9jKTsNCiAjZW5kaWYNCiAgICAgfQ0KZGlmZiAtLWdpdCBhL3hlbi9h
cmNoL2FybS9wY2kvZWNhbS5jIGIveGVuL2FyY2gvYXJtL3BjaS9lY2FtLmMNCmluZGV4IGVhZTE3
N2YyY2JjMi4uNGM1NDk5OGQxYjRiIDEwMDY0NA0KLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9lY2Ft
LmMNCisrKyBiL3hlbi9hcmNoL2FybS9wY2kvZWNhbS5jDQpAQCAtMzksOSArMzksOSBAQCB2b2lk
IF9faW9tZW0gKnBjaV9lY2FtX21hcF9idXMoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdl
LA0KICAgICByZXR1cm4gYmFzZSArIChQQ0lfREVWRk4yKHNiZGYpIDw8IGRldmZuX3NoaWZ0KSAr
IHdoZXJlOw0KIH0NCiANCi1ib29sIHBjaV9lY2FtX25lZWRfcDJtX21hcHBpbmcoc3RydWN0IGRv
bWFpbiAqZCwNCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHBjaV9ob3N0
X2JyaWRnZSAqYnJpZGdlLA0KLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1aW50NjRf
dCBhZGRyKQ0KK2Jvb2wgcGNpX2VjYW1fbmVlZF9wMm1faHdkb21fbWFwcGluZyhzdHJ1Y3QgZG9t
YWluICpkLA0KKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgcGNp
X2hvc3RfYnJpZGdlICpicmlkZ2UsDQorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHVpbnQ2NF90IGFkZHIpDQogew0KICAgICBzdHJ1Y3QgcGNpX2NvbmZpZ193aW5kb3cgKmNm
ZyA9IGJyaWRnZS0+Y2ZnOw0KIA0KQEAgLTU5LDcgKzU5LDcgQEAgY29uc3Qgc3RydWN0IHBjaV9l
Y2FtX29wcyBwY2lfZ2VuZXJpY19lY2FtX29wcyA9IHsNCiAgICAgICAgIC5tYXBfYnVzICAgICAg
ICAgICAgICAgID0gcGNpX2VjYW1fbWFwX2J1cywNCiAgICAgICAgIC5yZWFkICAgICAgICAgICAg
ICAgICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3JlYWQsDQogICAgICAgICAud3JpdGUgICAgICAg
ICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZpZ193cml0ZSwNCi0gICAgICAgIC5uZWVkX3Ay
bV9tYXBwaW5nICAgICAgID0gcGNpX2VjYW1fbmVlZF9wMm1fbWFwcGluZywNCisgICAgICAgIC5u
ZWVkX3AybV9od2RvbV9tYXBwaW5nID0gcGNpX2VjYW1fbmVlZF9wMm1faHdkb21fbWFwcGluZywN
CiAgICAgfQ0KIH07DQogDQpkaWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1j
b21tb24uYyBiL3hlbi9hcmNoL2FybS9wY2kvcGNpLWhvc3QtY29tbW9uLmMNCmluZGV4IGYzNTA4
MjZlYTI2Yi4uZjk5M2RlYmE4NzEyIDEwMDY0NA0KLS0tIGEveGVuL2FyY2gvYXJtL3BjaS9wY2kt
aG9zdC1jb21tb24uYw0KKysrIGIveGVuL2FyY2gvYXJtL3BjaS9wY2ktaG9zdC1jb21tb24uYw0K
QEAgLTMzNSwyMSArMzM1LDIxIEBAIGludCBfX2luaXQgcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdz
KHN0cnVjdCBkb21haW4gKmQsIHAybV90eXBlX3QgcDJtdCkNCiAgICAgc3RydWN0IG1hcF9yYW5n
ZV9kYXRhIG1yX2RhdGEgPSB7DQogICAgICAgICAuZCA9IGQsDQogICAgICAgICAucDJtdCA9IHAy
bXQsDQotICAgICAgICAubWFwX3BjaV9icmlkZ2UgPSB0cnVlDQorICAgICAgICAuc2tpcF9tYXBw
aW5nID0gZmFsc2UNCiAgICAgfTsNCiANCiAgICAgLyoNCiAgICAgICogRm9yIGVhY2ggUENJIGhv
c3QgYnJpZGdlIHdlIG5lZWQgdG8gb25seSBtYXAgdGhvc2UgcmFuZ2VzDQogICAgICAqIHdoaWNo
IGFyZSB1c2VkIGJ5IERvbWFpbi0wIHRvIHByb3Blcmx5IGluaXRpYWxpemUgdGhlIGJyaWRnZSwN
CiAgICAgICogZS5nLiB3ZSBkbyBub3Qgd2FudCB0byBtYXAgRUNBTSBjb25maWd1cmF0aW9uIHNw
YWNlIHdoaWNoIGxpdmVzIGluDQotICAgICAqICJyZWciIG9yICJhc3NpZ25lZC1hZGRyZXNzZXMi
IGRldmljZSB0cmVlIHByb3BlcnR5Lg0KLSAgICAgKiBOZWl0aGVyIHdlIHdhbnQgdG8gbWFwIGFu
eSBvZiB0aGUgTU1JTyByYW5nZXMgZm91bmQgaW4gdGhlICJyYW5nZXMiDQotICAgICAqIGRldmlj
ZSB0cmVlIHByb3BlcnR5Lg0KKyAgICAgKiAicmVnIiBvciAiYXNzaWduZWQtYWRkcmVzc2VzIiBk
ZXZpY2UgdHJlZSBwcm9wZXJ0eSwgYnV0IHdlIHdhbnQgdG8NCisgICAgICogbWFwIG90aGVyIHJl
Z2lvbnMgb2YgdGhlIGhvc3QgYnJpZGdlLiBUaGUgUENJIGFwZXJ0dXJlIGRlZmluZWQgYnkNCisg
ICAgICogdGhlICJyYW5nZXMiIGRldmljZSB0cmVlIHByb3BlcnR5IHNob3VsZCBhbHNvIGJlIHNr
aXBwZWQuDQogICAgICAqLw0KICAgICBsaXN0X2Zvcl9lYWNoX2VudHJ5KCBicmlkZ2UsICZwY2lf
aG9zdF9icmlkZ2VzLCBub2RlICkNCiAgICAgew0KICAgICAgICAgY29uc3Qgc3RydWN0IGR0X2Rl
dmljZV9ub2RlICpkZXYgPSBicmlkZ2UtPmR0X25vZGU7DQotICAgICAgICBpbnQgaTsNCisgICAg
ICAgIHVuc2lnbmVkIGludCBpOw0KIA0KICAgICAgICAgZm9yICggaSA9IDA7IGkgPCBkdF9udW1i
ZXJfb2ZfYWRkcmVzcyhkZXYpOyBpKysgKQ0KICAgICAgICAgew0KQEAgLTM1OSwxMiArMzU5LDEz
IEBAIGludCBfX2luaXQgcGNpX2hvc3RfYnJpZGdlX21hcHBpbmdzKHN0cnVjdCBkb21haW4gKmQs
IHAybV90eXBlX3QgcDJtdCkNCiAgICAgICAgICAgICBlcnIgPSBkdF9kZXZpY2VfZ2V0X2FkZHJl
c3MoZGV2LCBpLCAmYWRkciwgJnNpemUpOw0KICAgICAgICAgICAgIGlmICggZXJyICkNCiAgICAg
ICAgICAgICB7DQotICAgICAgICAgICAgICAgIHByaW50ayhYRU5MT0dfRVJSICJVbmFibGUgdG8g
cmV0cmlldmUgYWRkcmVzcyAldSBmb3IgJXNcbiIsDQorICAgICAgICAgICAgICAgIHByaW50ayhY
RU5MT0dfRVJSDQorICAgICAgICAgICAgICAgICAgICAgICAiVW5hYmxlIHRvIHJldHJpZXZlIGFk
ZHJlc3MgcmFuZ2UgaW5kZXg9JXUgZm9yICVzXG4iLA0KICAgICAgICAgICAgICAgICAgICAgICAg
aSwgZHRfbm9kZV9mdWxsX25hbWUoZGV2KSk7DQogICAgICAgICAgICAgICAgIHJldHVybiBlcnI7
DQogICAgICAgICAgICAgfQ0KIA0KLSAgICAgICAgICAgIGlmICggYnJpZGdlLT5vcHMtPm5lZWRf
cDJtX21hcHBpbmcoZCwgYnJpZGdlLCBhZGRyKSApDQorICAgICAgICAgICAgaWYgKCBicmlkZ2Ut
Pm9wcy0+bmVlZF9wMm1faHdkb21fbWFwcGluZyhkLCBicmlkZ2UsIGFkZHIpICkNCiAgICAgICAg
ICAgICB7DQogICAgICAgICAgICAgICAgIGVyciA9IG1hcF9yYW5nZV90b19kb21haW4oZGV2LCBh
ZGRyLCBzaXplLCAmbXJfZGF0YSk7DQogICAgICAgICAgICAgICAgIGlmICggZXJyICkNCmRpZmYg
LS1naXQgYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jIGIveGVuL2FyY2gvYXJt
L3BjaS9wY2ktaG9zdC16eW5xbXAuYw0KaW5kZXggYWRiZTM2Mjc4NzFmLi5hZWY1ZDZhZjdlZjUg
MTAwNjQ0DQotLS0gYS94ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jDQorKysgYi94
ZW4vYXJjaC9hcm0vcGNpL3BjaS1ob3N0LXp5bnFtcC5jDQpAQCAtMzMsNyArMzMsNyBAQCBjb25z
dCBzdHJ1Y3QgcGNpX2VjYW1fb3BzIG53bF9wY2llX29wcyA9IHsNCiAgICAgICAgIC5tYXBfYnVz
ICAgICAgICAgICAgICAgID0gcGNpX2VjYW1fbWFwX2J1cywNCiAgICAgICAgIC5yZWFkICAgICAg
ICAgICAgICAgICAgID0gcGNpX2dlbmVyaWNfY29uZmlnX3JlYWQsDQogICAgICAgICAud3JpdGUg
ICAgICAgICAgICAgICAgICA9IHBjaV9nZW5lcmljX2NvbmZpZ193cml0ZSwNCi0gICAgICAgIC5u
ZWVkX3AybV9tYXBwaW5nICAgICAgID0gcGNpX2VjYW1fbmVlZF9wMm1fbWFwcGluZywNCisgICAg
ICAgIC5uZWVkX3AybV9od2RvbV9tYXBwaW5nID0gcGNpX2VjYW1fbmVlZF9wMm1faHdkb21fbWFw
cGluZywNCiAgICAgfQ0KIH07DQogDQpkaWZmIC0tZ2l0IGEveGVuL2luY2x1ZGUvYXNtLWFybS9w
Y2kuaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vcGNpLmgNCmluZGV4IGVmOTg2YmQ1OTk0Zi4uM2My
MjJhY2M0NmJlIDEwMDY0NA0KLS0tIGEveGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KKysrIGIv
eGVuL2luY2x1ZGUvYXNtLWFybS9wY2kuaA0KQEAgLTc5LDkgKzc5LDkgQEAgc3RydWN0IHBjaV9v
cHMgew0KICAgICAgICAgICAgICAgICB1aW50MzJfdCByZWcsIHVpbnQzMl90IGxlbiwgdWludDMy
X3QgKnZhbHVlKTsNCiAgICAgaW50ICgqd3JpdGUpKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJy
aWRnZSwgdWludDMyX3Qgc2JkZiwNCiAgICAgICAgICAgICAgICAgIHVpbnQzMl90IHJlZywgdWlu
dDMyX3QgbGVuLCB1aW50MzJfdCB2YWx1ZSk7DQotICAgIGJvb2wgKCpuZWVkX3AybV9tYXBwaW5n
KShzdHJ1Y3QgZG9tYWluICpkLA0KLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dWludDY0X3QgYWRkcik7DQorICAgIGJvb2wgKCpuZWVkX3AybV9od2RvbV9tYXBwaW5nKShzdHJ1
Y3QgZG9tYWluICpkLA0KKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0
IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgdWludDY0X3QgYWRkcik7DQogfTsNCiANCiAvKg0KQEAgLTEwNSw5ICsxMDUsOSBAQCBp
bnQgcGNpX2dlbmVyaWNfY29uZmlnX3dyaXRlKHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRn
ZSwgdWludDMyX3Qgc2JkZiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDMyX3Qg
cmVnLCB1aW50MzJfdCBsZW4sIHVpbnQzMl90IHZhbHVlKTsNCiB2b2lkIF9faW9tZW0gKnBjaV9l
Y2FtX21hcF9idXMoc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqYnJpZGdlLA0KICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICB1aW50MzJfdCBzYmRmLCB1aW50MzJfdCB3aGVyZSk7DQotYm9v
bCBwY2lfZWNhbV9uZWVkX3AybV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsDQotICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRnZSwNCi0g
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDY0X3QgYWRkcik7DQorYm9vbCBwY2lf
ZWNhbV9uZWVkX3AybV9od2RvbV9tYXBwaW5nKHN0cnVjdCBkb21haW4gKmQsDQorICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBwY2lfaG9zdF9icmlkZ2UgKmJyaWRn
ZSwNCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDY0X3QgYWRkcik7
DQogc3RydWN0IHBjaV9ob3N0X2JyaWRnZSAqcGNpX2ZpbmRfaG9zdF9icmlkZ2UodWludDE2X3Qg
c2VnbWVudCwgdWludDhfdCBidXMpOw0KIGludCBwY2lfZ2V0X2hvc3RfYnJpZGdlX3NlZ21lbnQo
Y29uc3Qgc3RydWN0IGR0X2RldmljZV9ub2RlICpub2RlLA0KICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdWludDE2X3QgKnNlZ21lbnQpOw0KZGlmZiAtLWdpdCBhL3hlbi9pbmNsdWRl
L2FzbS1hcm0vc2V0dXAuaCBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vc2V0dXAuaA0KaW5kZXggMjE4
NjNkZDJiYzU4Li41YjMwMTM1ZmRhMzggMTAwNjQ0DQotLS0gYS94ZW4vaW5jbHVkZS9hc20tYXJt
L3NldHVwLmgNCisrKyBiL3hlbi9pbmNsdWRlL2FzbS1hcm0vc2V0dXAuaA0KQEAgLTg0LDcgKzg0
LDcgQEAgc3RydWN0IG1hcF9yYW5nZV9kYXRhDQogICAgIHN0cnVjdCBkb21haW4gKmQ7DQogICAg
IHAybV90eXBlX3QgcDJtdDsNCiAgICAgLyogU2V0IGlmIG1hcHBpbmdzIGZvciBQQ0kgaG9zdCBi
cmlkZ2VzIG11c3Qgbm90IGJlIHNraXBwZWQuICovDQotICAgIGJvb2wgbWFwX3BjaV9icmlkZ2U7
DQorICAgIGJvb2wgc2tpcF9tYXBwaW5nOw0KIH07DQogDQogZXh0ZXJuIHN0cnVjdCBib290aW5m
byBib290aW5mbzsNCi0tIA0KMi4yNS4xDQoNCg==

--_002_32ffee9e8786c91b681c7fa243f5d3faepamcom_--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 14:46:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 14:46:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198048.351348 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVENf-00067L-Hi; Tue, 28 Sep 2021 14:46:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198048.351348; Tue, 28 Sep 2021 14:46:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVENf-00067E-Eo; Tue, 28 Sep 2021 14:46:51 +0000
Received: by outflank-mailman (input) for mailman id 198048;
 Tue, 28 Sep 2021 14:46:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVENd-000678-M1
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 14:46:49 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.82]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 266d624f-491a-47df-9b41-871909b79ce1;
 Tue, 28 Sep 2021 14:46:46 +0000 (UTC)
Received: from AM6PR0502CA0041.eurprd05.prod.outlook.com
 (2603:10a6:20b:56::18) by VI1PR08MB3616.eurprd08.prod.outlook.com
 (2603:10a6:803:7f::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep
 2021 14:46:42 +0000
Received: from AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:56:cafe::fb) by AM6PR0502CA0041.outlook.office365.com
 (2603:10a6:20b:56::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Tue, 28 Sep 2021 14:46:41 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT015.mail.protection.outlook.com (10.152.16.132) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 14:46:41 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Tue, 28 Sep 2021 14:46:38 +0000
Received: from 11d9b4c7c2cd.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F563137F-3E88-4D13-A724-8743B04376FA.1; 
 Tue, 28 Sep 2021 14:46:22 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 11d9b4c7c2cd.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 14:46:22 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR3PR08MB5596.eurprd08.prod.outlook.com (2603:10a6:102:88::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 14:46:20 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 14:46:20 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P123CA0030.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 14:46:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 266d624f-491a-47df-9b41-871909b79ce1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Uw24404JNd1u24V3rFXoaO/pES0wg+9Qv+JR9AwHK/o=;
 b=enjIOUXokTvSU7SxTKOeNvi3r0D5EbuMeYD162vFixVIhpzvKGoJpALgX9CJUemftZgVmEwdaXNj+zdLjuk5BzdYR0x5cgJMV5CiG+yaDOlL7hjNB0iPG2Fhn+KtRTKIH57fuG1z6SxSJrEyxzmx3DqwMInJjLL8MXcuS1zeQ1g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ee3aeafacdfe2b96
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mPmSj1j+2U+h+wGWRr1SPffqQlfbJ/XI72dlUAUwQACiFxy58W1kF/TskRZTXjzNBDryFT24l95vTUziILh8S2GwwmxIer4KOtei5YvfLDCkdYIhznFYVv4On6wWDyNP24xsQXwbiWetrPia1Wyd6R37wA/AZkS8EcIUfBDqzVdbivJAIVnruI5u6XwnS4fq1JVMaUDR/HyQtj6Ldh/vXVANEISV1NiCg1q5Kgb5cRBJbIKU+NCsMA058b0HDkoy7VIug36hzgVLHtCcEI1yth7ERaxBNdiO0jARaYmLVXJvSQZzNu0NrZbpH4pkO/MtqJ6YLyJ3iv3EbmN2NVheyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Uw24404JNd1u24V3rFXoaO/pES0wg+9Qv+JR9AwHK/o=;
 b=ZeXOZqjfZ/b2mYnVwDlbfPznrgXT8cbMDXKYuSXebsRKZ8ooFBSUfv2PE0lKDX4NN7HAgekCN3AvCZU0Whv0nTzkE55qR3nKb6CQtC5c1pLg9K/U0WZys7fCYIhDsD9PE48PI0BWArdK9WU1X3UQN07HFa1C7bu9lrQnJMwHQ7QzffIWaJNPwvLezK5skzu8KWZe7V2uDOX/UsrXIrnBW/438CXelPLCltGFOoYc/wgSVCsm/9sU65CxiM1mhaHTkXK+MMbvJtFvkaPDXHGNyuY3lqQWrm9Iu/pprSPBiiB/ejiL2bA5CmfvztfkR077+HHfqJ9u+/F4GfGzDMonDA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Uw24404JNd1u24V3rFXoaO/pES0wg+9Qv+JR9AwHK/o=;
 b=enjIOUXokTvSU7SxTKOeNvi3r0D5EbuMeYD162vFixVIhpzvKGoJpALgX9CJUemftZgVmEwdaXNj+zdLjuk5BzdYR0x5cgJMV5CiG+yaDOlL7hjNB0iPG2Fhn+KtRTKIH57fuG1z6SxSJrEyxzmx3DqwMInJjLL8MXcuS1zeQ1g=
Authentication-Results-Original: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [XEN PATCH v4] xen: rework `checkpolicy` detection when using
 "randconfig"
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <20210928083944.780398-1-anthony.perard@citrix.com>
Date: Tue, 28 Sep 2021 15:46:13 +0100
Cc: xen-devel@lists.xenproject.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <53B2FB4B-DE19-45F5-A490-1A304B91BA32@arm.com>
References: <20210928083944.780398-1-anthony.perard@citrix.com>
To: Anthony PERARD <anthony.perard@citrix.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P123CA0030.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::18)
 To PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c54545a0-a11c-41ce-5fd1-08d9828ec8bd
X-MS-TrafficTypeDiagnostic: PR3PR08MB5596:|VI1PR08MB3616:
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB36163775244B1568C8941B61E4A89@VI1PR08MB3616.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 SVoOmX+8wMic3DbehSZTkVrJOa+5ZZlqFG+Vzm08cFX3F3FatyfBsYEj3C2GeCLt5RSTkjONPxA2yX7HBEd79nsE220gj2zsKbttHFcmmaM9AHkNFPfbRaE5tAWoUhtBNANeWP+/1GgXRVUNLsmsGNtkE0OsoKk3zfrudtLFi7uh2Y6EQXxRJ9cznBM6JPOjp0KZbx0VZ216mT+xcYfxVBNq2iR0fijBHe4pdmIOcUnOV5TWmKRwNSDdsJJ1T5r49S3zCx5ICSDcDZS44x+a1PembWu/dRvBHqOXXNLFkTB2ko1RmYkUN1m4J+Jln81+1nqgTPtKHLFLgWnXyoo+oedjYBFpn+//LA/m9u6FfuOa6L3K9eScV2HRR2fp8t0SmM/fjMBgacX90hsfPh+rw3UycZj1V8bY2wTmOiN4u8YWIblIIczXtCAe0Xx6r+ZPrDmIY76lcyQYD5FAtNWyfbcxy8jvK1LJrJnwxrXuWS2ljgveIxGmE+Pd4UHeH8qM4yScKjCYaUEZaCjNnGjtTHg9VbtPCm0XP5vIFrJiUmiQlYiWtt0b+Ndu7Ka1PHdRMfWR7qtrcgQQZyPxLnr7/4ziLJk/BCd6hJq/pouH3n4esTJ0m/J0mv2bnxgK5taKIg1dnNV3DxEEyWVNYydfXKQhngS1ysD/FORJHB++Q8h+eiVvD2h0hEaYyATvRzIucNW6qQ68++OTHAxGDJ/x/OuG+tTOC37102myddQaws7cQDWT9/QS/vFJB0ePp6Idqv1KD/mxQZI93CbziK6iQwFhhAETZpx+aKZZDx1RKSEthzQ4GcULEo56JJ04BM72hWnCQuuyjn7Oxx8LAiqWAw==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38350700002)(54906003)(508600001)(66476007)(38100700002)(2906002)(52116002)(66556008)(33656002)(6512007)(6666004)(86362001)(66946007)(6916009)(36756003)(6506007)(316002)(83380400001)(966005)(6486002)(53546011)(4326008)(8676002)(956004)(186003)(26005)(5660300002)(2616005)(8936002)(44832011)(81973001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5596
Original-Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bb84d48f-beb8-4217-9786-08d9828ebbcd
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hoeCbT4K98nRz/kyrB1BCyCRGcz2T5qRwdYzVJuY1lJFNE7y/jKA0XRvRK8hebc3H/acwmsa+OvGKl3DuMaShdhmi5T1HQV5cw0ADm4gWy06Vwes+lfx9dEit02zyN4PIw/xqZzCg82Tr4wVSAyBccAqYnHTbRugF55N4GXHhVWPcWC0Id0ZyDp0lwaZn7SDCrrRpHP/GYbOXMZKp8KsR/pvQqm/0l2YZcXolgxlj+N1s+6yOrVxo3CJbncuGwfJEA861ZLa6JMCkPreXWd8phWZ8VRFTh58gEoCLQ4XWIePuvtA0e7hvaj+D02ajBX8feANmdMelPBne2aA+xVEayY33oDujsgbsZmcE5sWbJkOj1RajYdDQEvJ728yW3TuhYux0Iv2M/z1BWs8db2Al/00w6XZIa/XUbL1AGSYut7mptvPqFzEUMOFcPBorCsQ19z5nwAV6fxASnXGK65ce0h6q6mZVmVN4qWY2t9jLXtSeAjhgO9MppssYWHx37e09s68n3Je2RGxp0mAOWVfAobEiRtBNwwKB/Gne2Dg5fJoGiheq8VTKK58jYUMwgMma1AILV/qLg7mbikjv7QQRYxI07nPzlPGubYo/3fFmLSwzyB/qhgtOosDTpQsdtmgw78kgdQqqP3JoJoBWYu/rFBg450rLI9QO/U6ur2/UK2Z0MnY2Q7uWMfrlZE8nTXcKN4sGqHHoRndeA7HoIA3OZz7O4NRr9UI29o0D9R6Gz3Hv8mTZnDGXgHZ3+vJMrRy0gqLhT5lbfIZ3TGPBFR777wEd/4pIoRPbWxdqBYrqutUvj1kmKMGUeYWA38lzY7h
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(86362001)(36860700001)(6512007)(2906002)(956004)(2616005)(336012)(8936002)(33656002)(47076005)(44832011)(186003)(36756003)(6666004)(966005)(82310400003)(4326008)(316002)(6486002)(356005)(70586007)(26005)(508600001)(53546011)(6862004)(83380400001)(81166007)(6506007)(70206006)(8676002)(54906003)(5660300002)(81973001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 14:46:41.2227
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c54545a0-a11c-41ce-5fd1-08d9828ec8bd
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3616



> On 28 Sep 2021, at 09:39, Anthony PERARD <anthony.perard@citrix.com> wrot=
e:
>=20
> This will help prevent the CI loop from having build failures when
> `checkpolicy` isn't available when doing "randconfig" jobs.
>=20
> To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> `checkpolicy` isn't available, we will actually override the config
> output with the use of KCONFIG_ALLCONFIG.
>=20
> Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> even when "checkpolicy" isn't available. It also prevent the build
> system from reset the config when "checkpolicy" isn't available
> anymore. And XSM_FLASK_POLICY is still selected automatically when
> `checkpolicy` is available.
> But this also work well for "randconfig", as it will not select
> XSM_FLASK_POLICY when "checkpolicy" is missing.
>=20
> This patch allows to easily add more override which depends on the
> environment.
>=20
> Also, move the check out of Config.mk and into xen/ build system.
> Nothing in tools/ is using that information as it's done by
> ./configure.
>=20
> We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> via .gitignore.
>=20
> Remove '=3D y' in Kconfig as it isn't needed, only a value "y" is true,
> anything else is considered false.

I don=E2=80=99t know if it is true, I=E2=80=99m having a look here:=20
https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt

And the section =E2=80=9CMenu dependencies=E2=80=9D states that:

An expression can have a value of 'n', 'm' or 'y' (or 0, 1, 2
respectively for calculations).

So it seems to me that m and y are evaluated as true, am I wrong?

Cheers,
Luca

>=20
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> v4:
> - keep XEN_ prefix for HAS_CHECKPOLICY
> - rework .allconfig.tmp file generation, so it is easier to read.
> - remove .allconfig.tmp on clean, .*.tmp files aren't all cleaned yet,
>  maybe for another time.
> - add information about file name choice and Kconfig change in patch
>  description.
>=20
> v3:
> - use KCONFIG_ALLCONFIG
> - don't override XSM_FLASK_POLICY value unless we do randconfig.
> - no more changes to the current behavior of kconfig, only to
>  randconfig.
>=20
> v2 was "[XEN PATCH v2] xen: allow XSM_FLASK_POLICY only if checkpolicy bi=
nary is available"
> ---
> Config.mk          |  6 ------
> xen/Makefile       | 20 +++++++++++++++++---
> xen/common/Kconfig |  2 +-
> 3 files changed, 18 insertions(+), 10 deletions(-)
>=20
> diff --git a/Config.mk b/Config.mk
> index e85bf186547f..d5490e35d03d 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=3Dy
> build_id_linker :=3D --build-id=3Dsha1
> endif
>=20
> -ifndef XEN_HAS_CHECKPOLICY
> -    CHECKPOLICY ?=3D checkpolicy
> -    XEN_HAS_CHECKPOLICY :=3D $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xe=
n && echo y || echo n)
> -    export XEN_HAS_CHECKPOLICY
> -endif
> -
> define buildmakevars2shellvars
>     export PREFIX=3D"$(prefix)";                                         =
   \
>     export XEN_SCRIPT_DIR=3D"$(XEN_SCRIPT_DIR)";                         =
   \
> diff --git a/xen/Makefile b/xen/Makefile
> index f47423dacd9a..7c2ffce0fc77 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?=3D $(shell hostname)
> PYTHON_INTERPRETER	:=3D $(word 1,$(shell which python3 python python2 2>/=
dev/null) python)
> export PYTHON		?=3D $(PYTHON_INTERPRETER)
>=20
> +export CHECKPOLICY	?=3D checkpolicy
> +
> export BASEDIR :=3D $(CURDIR)
> export XEN_ROOT :=3D $(BASEDIR)/..
>=20
> @@ -178,6 +180,8 @@ CFLAGS +=3D $(CLANG_FLAGS)
> export CLANG_FLAGS
> endif
>=20
> +export XEN_HAS_CHECKPOLICY :=3D $(call success,$(CHECKPOLICY) -h 2>&1 | =
grep -q xen)
> +
> export root-make-done :=3D y
> endif # root-make-done
>=20
> @@ -189,14 +193,24 @@ ifeq ($(config-build),y)
> # *config targets only - make sure prerequisites are updated, and descend
> # in tools/kconfig to make the *config target
>=20
> +# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
> +# This will be use by kconfig targets allyesconfig/allmodconfig/allnocon=
fig/randconfig
> +filechk_kconfig_allconfig =3D \
> +    $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_=
POLICY=3Dn';) \
> +    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
> +    :
> +
> +.allconfig.tmp: FORCE
> +	set -e; { $(call filechk_kconfig_allconfig); } > $@
> +
> config: FORCE
> 	$(MAKE) $(kconfig) $@
>=20
> # Config.mk tries to include .config file, don't try to remake it
> %/.config: ;
>=20
> -%config: FORCE
> -	$(MAKE) $(kconfig) $@
> +%config: .allconfig.tmp FORCE
> +	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=3D$< $@
>=20
> else # !config-build
>=20
> @@ -368,7 +382,7 @@ _clean: delete-unfresh-files
> 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} =
\;
> 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map=
 $(TARGET)-syms $(TARGET)-syms.map *~ core
> 	rm -f asm-offsets.s include/asm-*/asm-offsets.h
> -	rm -f .banner
> +	rm -f .banner .allconfig.tmp
>=20
> .PHONY: _distclean
> _distclean: clean
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index db687b1785e7..eb6c2edb7bfe 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -251,7 +251,7 @@ config XSM_FLASK_AVC_STATS
>=20
> config XSM_FLASK_POLICY
> 	bool "Compile Xen with a built-in FLASK security policy"
> -	default y if "$(XEN_HAS_CHECKPOLICY)" =3D "y"
> +	default y if "$(XEN_HAS_CHECKPOLICY)"
> 	depends on XSM_FLASK
> 	---help---
> 	  This includes a default XSM policy in the hypervisor so that the
> --=20
> Anthony PERARD
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:05:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:05:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198057.351360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEft-0000H7-5E; Tue, 28 Sep 2021 15:05:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198057.351360; Tue, 28 Sep 2021 15:05:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEft-0000H0-1j; Tue, 28 Sep 2021 15:05:41 +0000
Received: by outflank-mailman (input) for mailman id 198057;
 Tue, 28 Sep 2021 15:05:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVEfs-0000Gu-00
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:05:40 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e96eb628-464f-45af-a228-1fa21c972f1d;
 Tue, 28 Sep 2021 15:05:37 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e96eb628-464f-45af-a228-1fa21c972f1d
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632841537;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=t4N6GveHunfJ9nebUVgW4voK28bT2ubxAXBe44+qI+s=;
  b=CnjOOVLleaq2chW+Ti9L5xnMgOBzTwhozQ8dkoIwx/jTHjU/4HCjzzcD
   +02fx4MArchM38ORD4aeNPtj98BPl0wjy2hPqtdryU2TerTfXDKtC3WVc
   dboi3ec2zKhmC3vzcLRPIo6+VHE4qo4l6E6RC1V/2aL7NSijyZd5+mNAE
   U=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: WhM88Z2Tkik1qIBPkoytTeI0aWddivQWo+FuOjB4cCBlXcaJ6AWn+5/EEcBEeW60HoocPJj2tl
 aADFTszH42lQOEK3zJmNpPVa/7nlvW21Ro8HONt6qqGFV80+R1I/V/4sXNKBKSHvvadZZQoXgj
 wC6O9IcfKlO+DK6j9ZMDVXbQMp8IMpTmcXfgQauO9Mt36R01/b6KGosdIeDjYcf6dL9X7CxL0S
 UQAB04ZIIjuau6NHv5VpD4nxTXMfYbE/u+b3zphOhywrs4MAvaC3Wuol9zvPETh+DOga2KyWaH
 c5Z7pORopxbYRPa4VY9X90dW
X-SBRS: 5.1
X-MesageID: 53808741
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:9LvhfKL7kOvM2sG0FE+RU5IlxSXFcZb7ZxGr2PjKsXjdYENS3zAHn
 2seCm3UbP+CNDDwfdBxaN+08x8E6J/Qz99lSARlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokcxIn5BC5C5xZVG/fjgqoHUVaiUZ0ideSc+EH140UM5y7Zj6mJVqYPR7z2l6
 IuaT/L3YDdJ6xYsWo7Dw/vewP/HlK2aVAIw5jTSV9gS1LPtvyB94KYkDbOwNxPFrrx8RYZWc
 QphIIaRpQs19z91Yj+sfy2SnkciGtY+NiDW4pZatjTLbrGvaUXe345iXMfwZ3u7hB23pIxb1
 tpKuKbuTDhyY7/RlMstDBthRnQW0a1uoNcrIFC6uM2XiUbHb2Ht07NlC0Re0Y8wo7gtRzsUr
 LpBdW5LPkvra+GemdpXTsFlgNgjK8/6epsSoHh6wRnSDOo8QICFSKLPjTNd9Gpr1pgeQqqHD
 yYfQTxNMgvkWkZtBk8OIcwOl/65qSDUKSIN/Tp5ooJoujOOnWSdyoPFMsfRe9GMbdVYmACfv
 G2u13/iHhgQOdibyDyE2nGhnOnCmWX8Qo16PKK83u5nhhuU3GN7IA0bUx63rOe0jma6WslDM
 AoE9yw2t68w+Ue3CN7nUHWQoWGGlg4RXcJKFO834x3LzbDbiy6bDGUZSj9KaPQ9qdQ7Azct0
 ze0c8jBXGI19ufPEDTEq+nS/Wja1TUpwXEqWxRYZxUK5NTZ/oQM1U/dU9pMIPG3t4igcd3v+
 AyioC87jrQVqMcE0aSn4FzK6w6RSoj1oh0dvFuOATP8hu9tTMv1P9X3tQizAeNodt7xc7WXg
 JQTdyFyBsggCouR3AiEXekABr2g4/vt3Nb02gU0R8hJG9hA4ReekWFsDNNWeBsB3iUsI2aBj
 KrvVeV5v8Q70JyCN/MfXm5JI552pZUM7Py8PhwuUvJAY4JqaCiM9zx0aEib0gjFyRZ3y/5ha
 c3CK5r1Vh727JiLKhLtHI/xNpdxmkgDKZ77H8inn3xLL5LEDJJqdVv1GATXNb1ohE91iA7U7
 8xeJ6O3J+Z3CoXDjt3s2ddLdzgidCFjbbiv8pA/XrPTc2JORTB6Y9eMkOxJRmCQt/kM/gs+1
 irmAREwJZuWrSCvFDhmnVg5M+6wAcYj8yhrVcHuVH7xs0UejU+UxP93X/MKkXMPrYSPFNZ4E
 KsIfduuGPNKRmiV8jgRd8Cl/odjaA6qlUSFOC/8OGozeJtpRgro/N74f1SwqHlSX3Tv7cZu8
 ae90g77QIYYQ1gwBsjhd//ynUi6umIQmbwuUhKQcMVTYkjl7KNjNzf10q0sO8gJJBianmma2
 g+aDA02v+7Ip4NpotDFibrd99WiEvdkH1ocFG7etO7kOS7f92ul4IlBTOfXImyNCDKqoP2vP
 LwHwev9PfsLmEdxn7B9S7s7n7gj49bPpqNBylg2Fnv8cFn2WKhrJWOL3JcTu/QVlKNZowa/R
 mmG5sJeZeeSIMrgHVMceFglY+CE2a1GkzXe961ockDz5Ssx97ubS0RCeRKLjXUFfrdyNYokx
 8YnudIXtFPj2kZ7bI7eg3AG7XmII1wBT74j58MTD4LcgwY2zk1PPM7HASjs7ZDTM9hBPyHG+
 NNPaHYuU1iE+nf/Tg==
IronPort-HdrOrdr: A9a23:2envy693Ua6gtQr6yl1uk+DcI+orL9Y04lQ7vn2ZLiYlFfBw9v
 re+MjzsCWetN9/Yh0dcLy7V5VoIkm9yXcW2+cs1N6ZNWGN1VdAR7sC0aLShxHmBi3i5qp8+M
 5bAs1D4QTLfDtHZBDBkWuFL+o=
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="53808741"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Ian
 Jackson" <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>
Subject: [XEN PATCH v5] xen: rework `checkpolicy` detection when using "randconfig"
Date: Tue, 28 Sep 2021 16:04:51 +0100
Message-ID: <20210928150451.786582-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

This will help prevent the CI loop from having build failures when
`checkpolicy` isn't available when doing "randconfig" jobs.

To prevent "randconfig" from selecting XSM_FLASK_POLICY when
`checkpolicy` isn't available, we will actually override the config
output with the use of KCONFIG_ALLCONFIG.

Doing this way still allow a user/developer to set XSM_FLASK_POLICY
even when "checkpolicy" isn't available. It also prevent the build
system from reset the config when "checkpolicy" isn't available
anymore. And XSM_FLASK_POLICY is still selected automatically when
`checkpolicy` is available.
But this also work well for "randconfig", as it will not select
XSM_FLASK_POLICY when "checkpolicy" is missing.

This patch allows to easily add more override which depends on the
environment.

Also, move the check out of Config.mk and into xen/ build system.
Nothing in tools/ is using that information as it's done by
./configure.

We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
via .gitignore.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
v5:
- remove changes to common/Kconfig in order to avoid change in
  behavior for "default y if m" in future Kconfig update as the current
  behavior doesn't seems to be explicitly documented.

v4:
- keep XEN_ prefix for HAS_CHECKPOLICY
- rework .allconfig.tmp file generation, so it is easier to read.
- remove .allconfig.tmp on clean, .*.tmp files aren't all cleaned yet,
  maybe for another time.
- add information about file name choice and Kconfig change in patch
  description.

v3:
- use KCONFIG_ALLCONFIG
- don't override XSM_FLASK_POLICY value unless we do randconfig.
- no more changes to the current behavior of kconfig, only to
  randconfig.

v2 was "[XEN PATCH v2] xen: allow XSM_FLASK_POLICY only if checkpolicy binary is available"
---
 Config.mk    |  6 ------
 xen/Makefile | 20 +++++++++++++++++---
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/Config.mk b/Config.mk
index e85bf186547f..d5490e35d03d 100644
--- a/Config.mk
+++ b/Config.mk
@@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=y
 build_id_linker := --build-id=sha1
 endif
 
-ifndef XEN_HAS_CHECKPOLICY
-    CHECKPOLICY ?= checkpolicy
-    XEN_HAS_CHECKPOLICY := $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xen && echo y || echo n)
-    export XEN_HAS_CHECKPOLICY
-endif
-
 define buildmakevars2shellvars
     export PREFIX="$(prefix)";                                            \
     export XEN_SCRIPT_DIR="$(XEN_SCRIPT_DIR)";                            \
diff --git a/xen/Makefile b/xen/Makefile
index f47423dacd9a..7c2ffce0fc77 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?= $(shell hostname)
 PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
 export PYTHON		?= $(PYTHON_INTERPRETER)
 
+export CHECKPOLICY	?= checkpolicy
+
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
 
@@ -178,6 +180,8 @@ CFLAGS += $(CLANG_FLAGS)
 export CLANG_FLAGS
 endif
 
+export XEN_HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)
+
 export root-make-done := y
 endif # root-make-done
 
@@ -189,14 +193,24 @@ ifeq ($(config-build),y)
 # *config targets only - make sure prerequisites are updated, and descend
 # in tools/kconfig to make the *config target
 
+# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
+# This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig
+filechk_kconfig_allconfig = \
+    $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \
+    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
+    :
+
+.allconfig.tmp: FORCE
+	set -e; { $(call filechk_kconfig_allconfig); } > $@
+
 config: FORCE
 	$(MAKE) $(kconfig) $@
 
 # Config.mk tries to include .config file, don't try to remake it
 %/.config: ;
 
-%config: FORCE
-	$(MAKE) $(kconfig) $@
+%config: .allconfig.tmp FORCE
+	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=$< $@
 
 else # !config-build
 
@@ -368,7 +382,7 @@ _clean: delete-unfresh-files
 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \;
 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
 	rm -f asm-offsets.s include/asm-*/asm-offsets.h
-	rm -f .banner
+	rm -f .banner .allconfig.tmp
 
 .PHONY: _distclean
 _distclean: clean
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:07:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:07:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198062.351371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEhG-0000y7-GH; Tue, 28 Sep 2021 15:07:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198062.351371; Tue, 28 Sep 2021 15:07:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEhG-0000y0-D7; Tue, 28 Sep 2021 15:07:06 +0000
Received: by outflank-mailman (input) for mailman id 198062;
 Tue, 28 Sep 2021 15:07:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uYy3=OS=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVEhE-0000so-TX
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:07:04 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2583176c-00d0-4706-b517-1191cbec5667;
 Tue, 28 Sep 2021 15:07:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2583176c-00d0-4706-b517-1191cbec5667
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632841623;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:in-reply-to;
  bh=USxSn7DD6wpGToq+jqH/p+2Q4JiB0/26Ubd0hFYa9LA=;
  b=W/s9SRIcJaXfD+1av90AtqlhdTXr14yJQws6hxyjz3gkD6cKW7iPgYz4
   KmXJe04andWGQDI/ZEQG3FtoLjG+Gio669QOauPrVngc2YU5IjlGIzp4S
   HN5/Pj7bhmU0J9zTaxX9911L5SsWMME6UcTBz2m4r1zDpLzot01g9tMq1
   M=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: QJLeA7LiQ+cLlDzgZH4QFmps3txe7wsAMR/qdawHS3FkvwcFem9zD6i4jtqoZGVQeagYp+2Sgk
 dgviB1qwAsAgeqe5NUdAeViCAVkCOHsa9xJYUFbSJvhq3IpbQqrY1g8DQ6IK3DIzJr4hDEdrB/
 65Gv+EaAqznvtEXht4eS5QpAcMHPXGmBE/yDNS7rqlQ8M65TxpL45hw9cybg0dXW2plmlgkFZZ
 NZG8+wMmVkmKeVhmiY7cQUB4sW0PPWrsIStQdonz3vRGigp7eEpWO3c3t+igfegsz+cIa5Ww78
 erFR1W9pCz1OMX1qx5gFTzpP
X-SBRS: 5.1
X-MesageID: 53816312
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ZnxuFahOlxCGFB2/e47Yi7aSX1614hcKZh0ujC45NGQN5FlHY01je
 htvCGzUOv3fMWSmfY8gYI3n8EMGuJbXx9IyGwRtrCxhFSwb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y0YDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1R6qapYzxqMJHXmf0SCB5iCyFbZvFJreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHiOp8fvXdxiynUF/88TbjIQrnQ5M8e1zA17ixLNauFN
 5NHMmM1BPjGS00TYHoXUYsQp82TxX/0SR9StmrFn7VitgA/yyQuieOwYbI5YOeiXt5Jl0yVo
 mbH+WXRARwAMtGbjz2f/RqEmevnjS79HoUIG9WQ5vNsxVGe2GEXIBkXTkeg5+m0jFakXNBSI
 FBS/TAhxYAw/kG2Stj2XzWjvWWJ+BUbXrJ4EvI4wBGAzLLO5ASUDXRCSSROAPQkqcs3SDoCx
 lKP2dTzClRSXKa9ECzHsO3O9HXrZHZTfTRqiTI4oRUt/4bZuasIzRzzVdd6H/WytMX1PGqsz
 GXfxMQhvIk7gckO3qS92FnIhTOwu5TEJjIIChXrsnGNtF0gO9/8D2C8wR2Lta8fcNbxokyp5
 iBspiSI0AwZ4XhhfgSjR/8RVJWg+vqIKjHVhVMH83IJrG/2oy/LkWy9+lhDyKZV3iQsJWOBj
 Kz741o5CHpv0J2CNvQfj2WZUZhC8EQYPY65Ps04l/IXCnSLSCeJ/Tt1eWmb1H33nU4nnMkXY
 MnAK5nyXSxDU/g+l1JaotvxN5dwnUjSIkuJGfjGI+mPi+LCNBZ5t59fWLdxUgzJxPzd+1iEm
 zquH8CL1w9eQIXDjtr/q+YuwaQxBSFjX/je8pUPHsbae1YOMDxxWpf5nOJ6E6Q4zvs9qws91
 izkMqOu4AGk3iOvxMTjQi0LVY4Dqr4l9i9kYXxwYwz4s5XhCK72hJoim1IMVeFP3IReITRcF
 ZHpou2MXaZCTCrp4TMYYcWvpYBubk3z1wmPIzCkcH40eJs5H17F/drtfw3O8igSD3Xo6Zti8
 uP4jg6LE4AeQwlCDdrNbK79xV2GonVAyvl5WFHFI4cPdRy0opRqMSH4ktQ+P9oIdUfY3jKf2
 gvPWUUYqODBrpUb6t7MgazY/Y6lH/EnRhhRHnXB7KbwPi7fpzLxzYhFWeeOXDbcSGKrp/nyO
 bQLl6nxaaRVkkxLvoxwF6dQ4Zg/v9a/9aVHyglEHWnQawj5AL1XPXTbj9JEsbdAx+EFtFLuC
 F6P4NRTJZ6AJNjhTAwKPAMgY+mOiaMUlz3V4ahnKUn2/nYqrr+OUEEUNBiQki1NarByNdp9k
 +smvccX7S25iwYrbYna3nwFqTzUIyxSSbgju7EbHJTv21gixVx1aJDBDjP7vcOUYNJWP0h2e
 jKZicIuXViHKpYup5brKUXw4A==
IronPort-HdrOrdr: A9a23:a3qokaFRkZXYm9EppLqE6seALOsnbusQ8zAXP0AYc31om+ij5q
 eTdZUgpHvJYVkqNE3I9eruBEDEewK7yXcX2/h1AV7BZniEhILAFugLhuGO/9SjIVydygc079
 YYT0EUMr3N5DZB4/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,329,1624334400"; 
   d="scan'208";a="53816312"
Date: Tue, 28 Sep 2021 16:06:40 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, <xen-devel@lists.xenproject.org>
Subject: Re: [XEN PATCH v4] xen: rework `checkpolicy` detection when using
 "randconfig"
Message-ID: <YVMvgGxlcoD1ceB/@perard>
References: <20210928083944.780398-1-anthony.perard@citrix.com>
 <bdd68129-0415-2018-c0bf-98687094dab4@suse.com>
 <YVMn1TW8mKpAwImA@perard>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <YVMn1TW8mKpAwImA@perard>

On Tue, Sep 28, 2021 at 03:34:00PM +0100, Anthony PERARD wrote:
> On Tue, Sep 28, 2021 at 03:46:01PM +0200, Jan Beulich wrote:
> > On 28.09.2021 10:39, Anthony PERARD wrote:
> > > This will help prevent the CI loop from having build failures when
> > > `checkpolicy` isn't available when doing "randconfig" jobs.
> > > 
> > > To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> > > `checkpolicy` isn't available, we will actually override the config
> > > output with the use of KCONFIG_ALLCONFIG.
> > > 
> > > Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> > > even when "checkpolicy" isn't available. It also prevent the build
> > > system from reset the config when "checkpolicy" isn't available
> > > anymore. And XSM_FLASK_POLICY is still selected automatically when
> > > `checkpolicy` is available.
> > > But this also work well for "randconfig", as it will not select
> > > XSM_FLASK_POLICY when "checkpolicy" is missing.
> > > 
> > > This patch allows to easily add more override which depends on the
> > > environment.
> > > 
> > > Also, move the check out of Config.mk and into xen/ build system.
> > > Nothing in tools/ is using that information as it's done by
> > > ./configure.
> > > 
> > > We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> > > via .gitignore.
> > > 
> > > Remove '= y' in Kconfig as it isn't needed, only a value "y" is true,
> > > anything else is considered false.
> > 
> > Seeing you say this explicitly makes me wonder - is this actually true?
> 
> I've check that this was true by empirical testing before sending the
> patch. But the documentation isn't clear to me about the meaning of
> 'default y if "m"'. So would you rather keep '= y' just to stay on the
> safe side?

I've sent v5 with this change to the Kconfig file removed.

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:12:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:12:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198073.351382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEm8-0002UR-7I; Tue, 28 Sep 2021 15:12:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198073.351382; Tue, 28 Sep 2021 15:12:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEm8-0002UK-47; Tue, 28 Sep 2021 15:12:08 +0000
Received: by outflank-mailman (input) for mailman id 198073;
 Tue, 28 Sep 2021 15:12:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVEm6-0002UE-P5
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:12:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5b0b02ad-fadc-46c8-8aa5-2b6f5c6fe21f;
 Tue, 28 Sep 2021 15:12:05 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-j3b2l-9wM42InGZLzIRJlQ-1; Tue, 28 Sep 2021 17:12:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6304.eurprd04.prod.outlook.com (2603:10a6:803:fd::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 15:12:02 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 15:12:02 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 PR2P264CA0026.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101:1::14) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Tue, 28 Sep 2021 15:12:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b0b02ad-fadc-46c8-8aa5-2b6f5c6fe21f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632841924;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=XjsjJx6G5ArgzBcZBGnwnFpbdO/W90V41M1wnE6Ykb8=;
	b=Cr5Zo6gT5T/NwQOAZyCK2DKx0Cc9zaM9LOC1GPb6SGBNS/TyfyzAiSsePi5bC1H3HrIpF8
	ZbMiFT/D1s6NCjp2XVwZIQ6yS1Q1Ydt62CvGJLOdEazpWszZhdhgZWQDA9UlZhqpjNQk0x
	j2hS2KW7UA4H3EHIgl4jeFZgK56AO0I=
X-MC-Unique: j3b2l-9wM42InGZLzIRJlQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UUjHOOIXUNESJr03BwmXnDrvPHqTtXDLAUP9vejS6i1xUg9y857r9XhXF+xJIsYpWzboKwRabtaBV8zq/vgfwPNUuMFDMfwA3dOcLtLDRGeF3c4wj6JnhuxRDnq4MhCAaThJF62eiFJcClCIMR4jqxKDPHyR9jJ/Um6EBtSuvEG7VYThl4vp59fYMIDb+yfPMIs0GXbR62Mbq57wfIAe1I649VJZAdDSSG/oaNR0CW7XBtTiQBMxywinlpsybpjht3irx08WSOo2vdLE83AzYPONo+1dbB3c9RJJlAw1t8v9gKztp/GOnvpjWy1Ds0/hvySCGldPidJ6edGxYs1Obg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XjsjJx6G5ArgzBcZBGnwnFpbdO/W90V41M1wnE6Ykb8=;
 b=jTOHT8RiuL2M+XPlm93O8hjU5xpeyo+6JylWBpO25WcArA1rnPOhAwLbBLojG+uL0U+uc3OS9hDzvVksIZQYBdI7XQh4iuzYGcnHWoOs1XVssB/gxXxoNVGOTseYs8zRrSmmb9gn3g76IP1DCKqDuBARPSvsOvuNDIV18/y+ePZkxzuObLy/ggiiIKAps/OhOkvDGgSEWw14DKNVY8m+24jcyT7RwsEJ58WYB2VY7d2J/PKq4uN7/UbIhuDFwVvFtDZz0BicxaGlb9Wt2bsVFU9UyoB1q/CD3SgR+8CSONFSKB/6wcEKImSRJXP46v/YHk+amuk4sSz/KWcC/8CXDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [XEN PATCH v5] xen: rework `checkpolicy` detection when using
 "randconfig"
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210928150451.786582-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <667a6a25-c25d-2452-18a7-003a4bc4a0e3@suse.com>
Date: Tue, 28 Sep 2021 17:12:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210928150451.786582-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: PR2P264CA0026.FRAP264.PROD.OUTLOOK.COM
 (2603:10a6:101:1::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fe158f76-427a-454d-d7ec-08d98292533f
X-MS-TrafficTypeDiagnostic: VI1PR04MB6304:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6304DBFE7C946043B549C5D0B3A89@VI1PR04MB6304.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	y5qzum92qvGNq1S0WrnxVy7G9OyGK74P0O+RNlJ5E9VF4pb0Z8rkQC3/jiUjf3hpShnQ8ZA2PMgdP5rSYOuwZJmfBWd8YtoeJjc8XqLJkAp4FTZiX3Rr+smqnHEFPcWUbavIqxISHlmeAV1PrSIUksUq6jue4+gsEZG/0MpZgXM53yNS/GLg9XV3zKTL0JlAAIuq95QgP0JRDyHITZ1gqic+YHYEd/JdYClN71xcFbTJIfDICqSjFrsb7xYJXRKr8SMCaWuAtd79UMvfDLWhko+RawPRnbzkjD3hJkKG3P+pjifSKg4IaJklVOSTTFUGMcTi128lWx58pJizQaCSwRXMDxyPYt4bHH8qOHvmmPI4KNV3hLA2UoZa2/RUgUQBcknEbIc1BWXqKDf3tHGARh+x34c+0f5U35kyp+BxGyw9UybADf/WZK4sVYFfPrLKC0BI5ySTRc9ZibTmxfTriiYu3yF4tp7e9b2goJs3YDF/ihLS0spyi6Xlx3eLpfBQYk04aHNzwdgKhCpxzffkDAT8ju8ChMEwPseh9Qcna1AGUpoaSoh3Fj0LwwsC/2r6YsTj0qpkKWL00ubu0Q8LDBgrI4oRB/GO91xFOl4EYgGPGPtTD9IeJvzwKZcpU0rswO8T3kom9C3/tzAQudvX4s8Kon2D9lkdNs8+vKoNRr3JU30gkxS5lSj1ZwgM1cAjxONABclR0sGNNp1j8aOfFmSoH7odhU8l3yplEwjexGqRVOL8IoGdPBmBzJDj0Vuo
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(31686004)(956004)(2616005)(6916009)(36756003)(8936002)(53546011)(2906002)(31696002)(83380400001)(316002)(66556008)(5660300002)(186003)(66946007)(26005)(86362001)(8676002)(66476007)(508600001)(6486002)(38100700002)(54906003)(4326008)(81973001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QjlFSGI2d1lGaW4wcTRTZGowTUVndjZRN2FLTG5KZnpZNTFmd1h0VEdGMm9a?=
 =?utf-8?B?Y2pUNHNwYTdWMTltVjh0ZUZRRzZGUGliRzhSRWViMmQraFVzanI1MXJ1Lzl1?=
 =?utf-8?B?VStHREhjOHdOT2ZPSGxoNVhHMzBLRERMYzYvZCt3bEhVVTdUU0I5TTJFMHJ0?=
 =?utf-8?B?OXNtRUFDVmJJczZrSDZDb1pDQzFwTXBncmt5bDFPNklOZWFZOUFMZWoxbGxa?=
 =?utf-8?B?YjZTOUNmV0JKZDcwWU4wUG8zZFJRekJOQzFyaDdEZXMyM2tLYkI4Sks2TjF3?=
 =?utf-8?B?akhWUSswRVlVdm9uNjUyNjRJK3IxNDNCWE5NdUgwNnFJTjU1OGxXR0c5TmhP?=
 =?utf-8?B?ODhWd3lIN0JqeFJweFJSTVAwc0UxNzNBc0ErZ0VOZ29vVjFRMUQ3QkEzNHI5?=
 =?utf-8?B?NDFEVHN3Rk8wTjBQVFlsSjh1RW9POXdHVk1haE1vR0g0WlBPeU5ObnlCaFBl?=
 =?utf-8?B?Rk5nZGQvUFJYWEFtdS9ONUVKNjJOSXNobjI1THZXYXU0dFJQZ0pvMnZBVDR6?=
 =?utf-8?B?TEppcVFINzlreWlkeTEwRmRNRG9NTkJWalM4a0o0M2txMGZNZGcrd0NHbVJS?=
 =?utf-8?B?cmY5M2tRRDdrb1A3MXdxVnF4eFJYVTA4cEtud3BHOHdBeVRuWnY1aVJLaTVy?=
 =?utf-8?B?N0JMVFhZSVdXb0EySFdlYjVFMXVDd0ZXejAyUmIrbXJCb0ZoR1lZZzBIcm16?=
 =?utf-8?B?bWMvQUE4NVpQd0xCUDNmMFRuejlxWEpUUkZmRDRMaEQvRnpVejhLYzVHaDFr?=
 =?utf-8?B?Rkk1NFNEelEwbmJBRXFEbnlSWnpsb0ZYTlBxYW5ScXBPeE10eGJQUkJzQ2pz?=
 =?utf-8?B?Qmh0UUhnMktiQWM5blVCZi8wLzJUK1VKd0d3MFJiWjJ0S05sVWNIY0tvNGw1?=
 =?utf-8?B?VlptK25jaXg4ZmZBVFI1Q1ppQzhOSzRGNE9YN0c3dE14YW5aRytzUlNWY0FH?=
 =?utf-8?B?Skh2SGZPQ05ZMURwYmRCT3kvT2JvcXQ4Q0IwVkpLTFJVVExPTnpTM0hXNi9p?=
 =?utf-8?B?RDVUcjVtUlpSZkpvcStQd2Y4ZCtqK0RTV29DNy9adXppbDhWMTBWa29Yekxs?=
 =?utf-8?B?UjB5SVdNWUFNOUkwdzdEQmVJZ2I1dkJFUCtDMWZnNGlkN3EwYTM0ZHFXaC9a?=
 =?utf-8?B?Lzg0OFIxTXpEWkt0VitGWEMrQTFoYmZ5dnFxanRUVkNqVGVzSHlBYkp2cVU2?=
 =?utf-8?B?Wm9Oek5IMnUzZVR4YXZNUlZVRDM2U2F5dnpsWm90dWt4UktpV3JUUXluVjB0?=
 =?utf-8?B?ZnNUTUoyanVTVC9QZm5Hd0l4RW4rSFFXcm5pRnVEWFBmRlozaTBHc3BtWHRL?=
 =?utf-8?B?ZlBnWS8xZkdvK3B5dnBJeGR5WkZGbFdFM0QwRytMNFpHN2ViZ1lIZHh2c3JZ?=
 =?utf-8?B?VzFESUZkZ0g4TWQxSFFJc1FQdU12T1lURlJkS0labGROdEVkTU5ZZm9qdEhP?=
 =?utf-8?B?OUozTVc4a2ErS2F2YVhrMnZaQVUxZE9zajIvR3lpNnFRQ3g5Q0t2aWhWdnVu?=
 =?utf-8?B?aHV6bjN5MzU4RlU0Tm1JSDcwbVQ3clZrY0ltbW1HaXNseEZ4dUJGenhyQU1Q?=
 =?utf-8?B?YWdGYU5EbGpBZWZVbjRPRkpxUHd6SlQ3Z1VhUFZzYXBhY2NEM1o2L1gyb3Ay?=
 =?utf-8?B?UVcrUzl1TVZyR2g4NTBjQ1BaZVZCZEJUKzJ6amJBYmp0bEFXdldrTjJWRGx0?=
 =?utf-8?B?VitzRlBnRXF6YkIyYVFudkErbHZobHJJMG9ma3BVNGlrU1pwYlNJc1RPTEMy?=
 =?utf-8?Q?rc6H0Rz8wXAk2RBckZC7E8kP007RBYLEgrn9xGI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fe158f76-427a-454d-d7ec-08d98292533f
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 15:12:02.4064
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yOu5pbDhEzGwpgjIu7HRkDyKzNy+6JQwC7Iy6MFbIUbDEyTFqNXDwIS+yxR85udKiQQYL+aqXIsZN+WhblrWeQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6304

On 28.09.2021 17:04, Anthony PERARD wrote:
> This will help prevent the CI loop from having build failures when
> `checkpolicy` isn't available when doing "randconfig" jobs.
> 
> To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> `checkpolicy` isn't available, we will actually override the config
> output with the use of KCONFIG_ALLCONFIG.
> 
> Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> even when "checkpolicy" isn't available. It also prevent the build
> system from reset the config when "checkpolicy" isn't available
> anymore. And XSM_FLASK_POLICY is still selected automatically when
> `checkpolicy` is available.
> But this also work well for "randconfig", as it will not select
> XSM_FLASK_POLICY when "checkpolicy" is missing.
> 
> This patch allows to easily add more override which depends on the
> environment.
> 
> Also, move the check out of Config.mk and into xen/ build system.
> Nothing in tools/ is using that information as it's done by
> ./configure.
> 
> We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> via .gitignore.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:12:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:12:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198074.351393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEmD-0002lp-GW; Tue, 28 Sep 2021 15:12:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198074.351393; Tue, 28 Sep 2021 15:12:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVEmD-0002lg-Ct; Tue, 28 Sep 2021 15:12:13 +0000
Received: by outflank-mailman (input) for mailman id 198074;
 Tue, 28 Sep 2021 15:12:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVEmC-0002l2-57
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:12:12 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bae79525-66e4-4c5f-b46f-c60b7c70617b;
 Tue, 28 Sep 2021 15:12:09 +0000 (UTC)
Received: from AS8PR04CA0111.eurprd04.prod.outlook.com (2603:10a6:20b:31e::26)
 by PA4PR08MB5919.eurprd08.prod.outlook.com (2603:10a6:102:e0::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Tue, 28 Sep
 2021 15:12:00 +0000
Received: from AM5EUR03FT031.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:31e:cafe::4d) by AS8PR04CA0111.outlook.office365.com
 (2603:10a6:20b:31e::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Tue, 28 Sep 2021 15:11:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT031.mail.protection.outlook.com (10.152.16.111) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 15:11:59 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Tue, 28 Sep 2021 15:11:57 +0000
Received: from 7935e80e6af2.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E7C3994A-438D-489C-87FA-1DAAE49CD76B.1; 
 Tue, 28 Sep 2021 15:11:46 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7935e80e6af2.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 15:11:46 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6174.eurprd08.prod.outlook.com (2603:10a6:102:e6::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep
 2021 15:11:42 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 15:11:42 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0055.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:153::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.14 via Frontend Transport; Tue, 28 Sep 2021 15:11:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bae79525-66e4-4c5f-b46f-c60b7c70617b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZsYrC3/DJRwMFnLrXNHdwi3JPyH45hO+HXgH9FQ6r8Q=;
 b=TyRbDERXsFCpF55ctNsIEO4VnDx4+n5OdZeOppUkXbyGexissv+nLGfk5TasAmoWvCmHwYBOUdZnNTqRwacuoCSC/utYMwru7pPjuLRxsL4+jL9iXYuHULf+2Kb26i73znN6MyeXqGfN1RtsK9bblMJGp6dgUa8wSgRDw2/2Xl8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 5070607af8961cdc
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UUowX1yFkmqfoJJo0iD+8wODDFQ2/a0QOD0O0AAxNu9ixYygfEn98kMuZf8UhjXntRqGxBXOeNR4MQiwhTc7+woIklC5uxEu3vcRrzKo+u02mV27bEr8JYQgY+aLx48bi22yJg0dKsIeChp6KHeM3rRFy4+m6oKWSoXDfLUv0b7X5eyyV0n52kUBKhQmnBwHxevqI3UqgsZQIV7rSPM9nr1wykhBQ+UQeUztu1dDl8fTjaFzyI+u3VNGe5bfAZlGgUx0v7RXjIZfDXb+RTrgKzl/PZ+AFWpxBhHwaxEv4jCzBOSwCRLsV5xsxTXSdUsm87fzZVPKimf7pIDGFxTiig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZsYrC3/DJRwMFnLrXNHdwi3JPyH45hO+HXgH9FQ6r8Q=;
 b=NP9+/QSzZYeCEBqR/j+y1+9ujYo9C5ZaY0kLSWBZyr6aJTUQMBxnFEyxwDd6MTEONl89RgVsUuRKuAnRHge11ZOoU2aHXpC4VjwdKomL7aNU4pT1jpMppDMSDPttv01+SPefxKeVs3eItb8cn6lP4raj/E72FvWcApFNcmlqMp9/Er7SIVGQ4uKYtdC3TUbgTKk1GVrLcHFh7fJv66KQFLcCt+ZkJQ+lilRhYav18mADFOBxqDFCIN7n+wYL8y6LvHL2zhHJ5VuMUIE0nGGktkAFxRzf4MQmSGSWmXRFrPcQk70qy6TWuD9fTEfTEuWJqo1UAeCj25b3Kgt5h5hOsA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZsYrC3/DJRwMFnLrXNHdwi3JPyH45hO+HXgH9FQ6r8Q=;
 b=TyRbDERXsFCpF55ctNsIEO4VnDx4+n5OdZeOppUkXbyGexissv+nLGfk5TasAmoWvCmHwYBOUdZnNTqRwacuoCSC/utYMwru7pPjuLRxsL4+jL9iXYuHULf+2Kb26i73znN6MyeXqGfN1RtsK9bblMJGp6dgUa8wSgRDw2/2Xl8=
Authentication-Results-Original: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [XEN PATCH v5] xen: rework `checkpolicy` detection when using
 "randconfig"
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <20210928150451.786582-1-anthony.perard@citrix.com>
Date: Tue, 28 Sep 2021 16:11:35 +0100
Cc: xen-devel@lists.xenproject.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <018FB4C2-2995-4295-A53A-87485AA45FF7@arm.com>
References: <20210928150451.786582-1-anthony.perard@citrix.com>
To: Anthony PERARD <anthony.perard@citrix.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0055.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:153::6) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3d0c3bfd-036b-48c5-8296-08d98292516c
X-MS-TrafficTypeDiagnostic: PA4PR08MB6174:|PA4PR08MB5919:
X-Microsoft-Antispam-PRVS:
	<PA4PR08MB59191EA746AA437EDA4F4E0DE4A89@PA4PR08MB5919.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 2wUkLL1dIjrJ1esWcOF6IWQMoFbcas7DoLdcMLhJG5+Hh7XcOG5aCV24dQkCGZwJWSc4tWlbC4XRQCEdDSG+AuWu6IUdgNBEFHyhwZWa8sRnq+elxAGo31QqEtkHli8zJPqnJf0rY7RVqVwqMjdd5qEozl4ekhE+6yIXQ81GJAefnKja4b/k/l0QRMu2jpfI412wEaFBFNnpJMqLSCWaffFim0D+XA2Dz0B/QOrR0MYsIYWsiOZ8ovQtzySKzT2WdxFBB5rk86s+tYS823XuTgXA9NSZr6+EtZzaMV79JWdVudJKIpn28e4M9r/XaligUbPJS9ECGvLCn9BaxZr9BIulp6p4X53z/obkeO6xG2i7rR90fcAiVcZoKvGBofk/fRLGHgSkUSMberfAhw2/keqQJN3n4ykU6c0jSRN+tPzpDATrHlsn80d7Ptd0o7b9jqoHJbG/quo6i2MfkW7Zj/Mq/nf4iPUWE5z1DYLSigMbgkrwkvHhYL3qugMNyO7oDbWQI3pwC1D2FL+xqdZmBR4+TN+SjdgAIMP3V9ZRF5FuVz48BtqCEUkfDWRtY3lzY0j4pNJAeHMYG2EqRH1+j3U2u313hudkHi7gPcxSpif007k8e5PriErQMcR5g3rmSAPZaldi8N7TDBbvBlLpM5E/QCe+B2Hu+/2DS/n8a3Qai95cJDF3lVN2NjBrWDVKiJAKi8BdCc4d6LeCv2B+JwtjCcx6sp1Vp3FW7nV5G/T8qrYbp/vAvcR+R3iDyxZSXSQxqP5H5thF+gQv2jlHaQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(4326008)(36756003)(8936002)(38100700002)(66476007)(8676002)(66556008)(38350700002)(86362001)(83380400001)(508600001)(5660300002)(6916009)(956004)(2616005)(6486002)(6512007)(186003)(6506007)(2906002)(54906003)(26005)(44832011)(53546011)(33656002)(52116002)(6666004)(316002)(81973001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6174
Original-Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	40c3f6ae-73d2-4532-8227-08d982924701
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WeLAy8RDUPk6yktWmTH25YOJ/9ektkk3oiixY1i/fZgXBCzAXY8cR5yyYYiWPNI1Jw6Ayav58M4LIIDBfmCEWIgNybgJtH8WKhOHt57m4X2rsucbg6zV3q+eAmRQEHvrlYqUGnJkjpRaV4yyVx9MybmbvBH6odIkbDOqyuX8DCEl9ACOOGcZxGLjKy+dgCs1fGAobYE/huEXI0bcNj2tdPfteD9pek5uvgbweVfVShB3txBpCd0vUr6s9XeHrpLOwnEKKkC7073rTcgZaeqD2TEqst0FBgXckk0mTtL/xzSdchUO7SkfWmlTaetml/aCMnIYNmKm37GghrvK01fyOmcnN91E4imc4IDBbNGYwd44R2sAkZHGvCcwg9sx9zIEHZRIus6/cdYV6g256Wm88ArftkbhIxTnz/WOEga8gbmBBGPwL4RsRr2LWEzxuvI6kXVa6rv46ZmeyF3NnfZRJGyDbAlWA5Bk2TMKqoy2NkCAzNu9MFrXlCcn4/Q4YA6oRDLfRFF0iQE9RzS7br9IlWg6KEgM3h9RSleo7rOs4Y67GofDU8QhqTFE9phc0Whvz58GzUOsBkuGXccRaiL03NLuBJ7Z3DWP4D/QOmwAx55GKRYlYa6tUV7UDYLchvTaTEGJTuVkelVQrL7hDmUuiluklikgva5QiqAAjcfK6py4sdXQ14MVO3vIde7POifJ7gV8Y9LzKGuiwc0X7agXxjHF6NoGyDPOowakYQDtPLo=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(8936002)(6506007)(6512007)(44832011)(4326008)(336012)(356005)(508600001)(2906002)(81166007)(54906003)(82310400003)(86362001)(33656002)(8676002)(956004)(26005)(316002)(186003)(6666004)(83380400001)(6486002)(47076005)(53546011)(70586007)(36860700001)(2616005)(6862004)(5660300002)(70206006)(81973001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 15:11:59.1928
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d0c3bfd-036b-48c5-8296-08d98292516c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT031.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5919



> On 28 Sep 2021, at 16:04, Anthony PERARD <anthony.perard@citrix.com> wrot=
e:
>=20
> This will help prevent the CI loop from having build failures when
> `checkpolicy` isn't available when doing "randconfig" jobs.
>=20
> To prevent "randconfig" from selecting XSM_FLASK_POLICY when
> `checkpolicy` isn't available, we will actually override the config
> output with the use of KCONFIG_ALLCONFIG.
>=20
> Doing this way still allow a user/developer to set XSM_FLASK_POLICY
> even when "checkpolicy" isn't available. It also prevent the build
> system from reset the config when "checkpolicy" isn't available
> anymore. And XSM_FLASK_POLICY is still selected automatically when
> `checkpolicy` is available.
> But this also work well for "randconfig", as it will not select
> XSM_FLASK_POLICY when "checkpolicy" is missing.
>=20
> This patch allows to easily add more override which depends on the
> environment.
>=20
> Also, move the check out of Config.mk and into xen/ build system.
> Nothing in tools/ is using that information as it's done by
> ./configure.
>=20
> We named the new file ".allconfig.tmp" as ".*.tmp" are already ignored
> via .gitignore.
>=20
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

> ---
> v5:
> - remove changes to common/Kconfig in order to avoid change in
>  behavior for "default y if m" in future Kconfig update as the current
>  behavior doesn't seems to be explicitly documented.
>=20
> v4:
> - keep XEN_ prefix for HAS_CHECKPOLICY
> - rework .allconfig.tmp file generation, so it is easier to read.
> - remove .allconfig.tmp on clean, .*.tmp files aren't all cleaned yet,
>  maybe for another time.
> - add information about file name choice and Kconfig change in patch
>  description.
>=20
> v3:
> - use KCONFIG_ALLCONFIG
> - don't override XSM_FLASK_POLICY value unless we do randconfig.
> - no more changes to the current behavior of kconfig, only to
>  randconfig.
>=20
> v2 was "[XEN PATCH v2] xen: allow XSM_FLASK_POLICY only if checkpolicy bi=
nary is available"
> ---
> Config.mk    |  6 ------
> xen/Makefile | 20 +++++++++++++++++---
> 2 files changed, 17 insertions(+), 9 deletions(-)
>=20
> diff --git a/Config.mk b/Config.mk
> index e85bf186547f..d5490e35d03d 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -137,12 +137,6 @@ export XEN_HAS_BUILD_ID=3Dy
> build_id_linker :=3D --build-id=3Dsha1
> endif
>=20
> -ifndef XEN_HAS_CHECKPOLICY
> -    CHECKPOLICY ?=3D checkpolicy
> -    XEN_HAS_CHECKPOLICY :=3D $(shell $(CHECKPOLICY) -h 2>&1 | grep -q xe=
n && echo y || echo n)
> -    export XEN_HAS_CHECKPOLICY
> -endif
> -
> define buildmakevars2shellvars
>     export PREFIX=3D"$(prefix)";                                         =
   \
>     export XEN_SCRIPT_DIR=3D"$(XEN_SCRIPT_DIR)";                         =
   \
> diff --git a/xen/Makefile b/xen/Makefile
> index f47423dacd9a..7c2ffce0fc77 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -17,6 +17,8 @@ export XEN_BUILD_HOST	?=3D $(shell hostname)
> PYTHON_INTERPRETER	:=3D $(word 1,$(shell which python3 python python2 2>/=
dev/null) python)
> export PYTHON		?=3D $(PYTHON_INTERPRETER)
>=20
> +export CHECKPOLICY	?=3D checkpolicy
> +
> export BASEDIR :=3D $(CURDIR)
> export XEN_ROOT :=3D $(BASEDIR)/..
>=20
> @@ -178,6 +180,8 @@ CFLAGS +=3D $(CLANG_FLAGS)
> export CLANG_FLAGS
> endif
>=20
> +export XEN_HAS_CHECKPOLICY :=3D $(call success,$(CHECKPOLICY) -h 2>&1 | =
grep -q xen)
> +
> export root-make-done :=3D y
> endif # root-make-done
>=20
> @@ -189,14 +193,24 @@ ifeq ($(config-build),y)
> # *config targets only - make sure prerequisites are updated, and descend
> # in tools/kconfig to make the *config target
>=20
> +# Create a file for KCONFIG_ALLCONFIG which depends on the environment.
> +# This will be use by kconfig targets allyesconfig/allmodconfig/allnocon=
fig/randconfig
> +filechk_kconfig_allconfig =3D \
> +    $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_=
POLICY=3Dn';) \
> +    $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \
> +    :
> +
> +.allconfig.tmp: FORCE
> +	set -e; { $(call filechk_kconfig_allconfig); } > $@
> +
> config: FORCE
> 	$(MAKE) $(kconfig) $@
>=20
> # Config.mk tries to include .config file, don't try to remake it
> %/.config: ;
>=20
> -%config: FORCE
> -	$(MAKE) $(kconfig) $@
> +%config: .allconfig.tmp FORCE
> +	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=3D$< $@
>=20
> else # !config-build
>=20
> @@ -368,7 +382,7 @@ _clean: delete-unfresh-files
> 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} =
\;
> 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map=
 $(TARGET)-syms $(TARGET)-syms.map *~ core
> 	rm -f asm-offsets.s include/asm-*/asm-offsets.h
> -	rm -f .banner
> +	rm -f .banner .allconfig.tmp
>=20
> .PHONY: _distclean
> _distclean: clean
> --=20
> Anthony PERARD
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:24:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:24:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198090.351404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVExt-0004hh-Mt; Tue, 28 Sep 2021 15:24:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198090.351404; Tue, 28 Sep 2021 15:24:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVExt-0004ha-IA; Tue, 28 Sep 2021 15:24:17 +0000
Received: by outflank-mailman (input) for mailman id 198090;
 Tue, 28 Sep 2021 15:24:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=3WUc=OS=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVExr-0004hU-Nt
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:24:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2318f366-2070-11ec-bca5-12813bfff9fa;
 Tue, 28 Sep 2021 15:24:14 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2051.outbound.protection.outlook.com [104.47.4.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-d1uoH8ElNwOgqGdskh5hrg-1; Tue, 28 Sep 2021 17:24:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3775.eurprd04.prod.outlook.com (2603:10a6:803:1a::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Tue, 28 Sep
 2021 15:24:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Tue, 28 Sep 2021
 15:24:10 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0014.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Tue, 28 Sep 2021 15:24:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2318f366-2070-11ec-bca5-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632842653;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=bVjcc+lf5Xi6q3iAJ2+J5FXpDka7fe4/UpOPtnl/kho=;
	b=foO1090//iD8mA+pd59VmRM80SodVjO0PYSHH1JSkg42W04uX9TPHU2sI21OomndO0z1BE
	sHSoYTqRtsfIXaxFQvBLmqpyiijmwGdwJoInNyMKisECRkFl8Ybc+MuL1XNOHcNqPy87mm
	hU0uqN76d3EoPsijYgEV2Nq+3ZkDtmk=
X-MC-Unique: d1uoH8ElNwOgqGdskh5hrg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Zt/LdWA9YFNTp8PeQJNjaeV5hlkUovoXTqCJLuNXIx8sVCLLg1Ap4FF0ErMCABSkfVkAvpV4ZzuIMSsT/X8DbgiLXspG6KNMzhxSaBgnqnttuPtubxuCq+cCA6+r/+Iq5Vkt1CDhcBZS9XjxrWQMYgfWb4rGWpOxTApchZ6fOvpYZqRNwm7PsdT59sn9h0ivpsaN9n1HqGCj3+Luy9Nu/dOfDBMc6Dn8OhDNuSLP+0c9Q7I0mpx4G60YB6hwDys98wC62a/8kwAOrxfV8rN6zEuVQ4i2OEcA94yVKATvyK0eGXLKVa5bKk1YDqBAIuL05r/ZinxSYS1KTAvjA1i7RA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bVjcc+lf5Xi6q3iAJ2+J5FXpDka7fe4/UpOPtnl/kho=;
 b=cL46ktlba3N+/Mp3Li4EmErijdScFf3pTb/tjV+8xC2f6MiBbc97U34oofO/idPk/soYsLi0Xow2VRGJNk730qS45MiEJsBvs1dOT28J89Vvc+7KcS/mbpNGC3huWHJ6zPUEXrLxZ4AoAp3jWNPvZ4ugSFuQIrxGb9+bHLhfUDWg4zh+PpOnkUOQJBBo5S9xByN13AHvtCRfzuYlyHJFFd+t2xZb1n8GgYMiN/NVw1K3wAb6FtgNmSAYMGHOgxh0GJzaXviLFtu4Dh22trr1JPEAQfucvVhsB1jNdyuXaeavGgQ9T/PtMq5e1Z7bXjUb5HU4JlSV1JB86dFJ4xAt0A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
To: Julien Grall <julien.grall.oss@gmail.com>,
 Ian Jackson <iwj@xenproject.org>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Stefano Stabellini <sstabellini@kernel.org>, dpsmith@apertussolutions.com
References: <osstest-164996-mainreport@xen.org>
 <d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
 <24904.44119.940679.241639@mariner.uk.xensource.com>
 <alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
 <19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
 <alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
 <CAJ=z9a1W9BqeOGFu3F2+qbXM6UNo8ZO=0y-zYsnK01MNSNdbkw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <dda286dc-1edc-4505-a049-e82047b44590@suse.com>
Date: Tue, 28 Sep 2021 17:24:08 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <CAJ=z9a1W9BqeOGFu3F2+qbXM6UNo8ZO=0y-zYsnK01MNSNdbkw@mail.gmail.com>
Content-Type: multipart/mixed;
 boundary="------------AA895F193E45FFD3D63817C7"
Content-Language: en-US
X-ClientProxiedBy: FR3P281CA0014.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f5bf7880-c2d8-4174-b737-08d9829404f1
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3775:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB37754BD5CEA03BB9A1041CEAB3A89@VI1PR0402MB3775.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rsHwtHMII1rnux4huVEuOd6PCxVqp1fzDdkqXf2RN0IX18uQNgfQgwmmkkdW+Jfbrlf8GiugjwQVwO/uifZKZCM8hCQqNT17EGjzub7JuTSe0RLVMqvQy/Ns/yG0MVyP3YAQRhhjF+4DgkSA1tGL44Y3SpgooL5AbML+lWDIia8gLJQmR4ASphMOXWCKB/amVIU1lCu9i13oRBoj3Ap8kC92j5ML/ftqmejr2XCkQBbVO8fyYClP2F5MYaJwMaJ+tWW+HHmdWj9entCjstKP1zGAm3mfnJwQjQvHyGH4DuCy2XRt/7JmHXPd28ReUICHatGsLqn+3apdX5kT91hIXITUfowQXQMn0ct0zWm+SScSj5rjW0rYXYmEimH+tpfC32WwWQkNzqDV8c+J0qLI0oi8NmlzZ1OvpnD8eJ9o9d+hBMmJqgYZosAVYwd8YfzymveR/C48ycmAZjimyzFJ7yLOCVuM6klF78w3D9Os2utzorrSRMTzm1XV+woyxd0asQVoa0rwJky9fhIXoWpo/LBwKXT0uUbT1752fDXeeVuhORj3dEjYuGY+yxrfT7fQIZ3XZh7kE6F0lnNXqtQNWlZ6WfTIMtU9C4wJUL9VRrczybvZx40tXQUNMZsnMjkTyaXyDiodiN1eBejGwoiw6nNuESHWgsHWJH3ohhoyKBFVrwwAqk+El4Ac/eYxEUtVzB3BKniKj2PVRrzXDTGHPE2RPvlYoulPKaZhGICapvst5j6tAqx6W2NoqM6Xqrj6cjMmeMhI/xt9KrMVBCAx4H6Hva9nWDK829ch8SucTftnjeV/L5ErVO//p1HGsOSN
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(8936002)(8676002)(83380400001)(186003)(36756003)(2616005)(956004)(66946007)(66476007)(66616009)(86362001)(38100700002)(66556008)(53546011)(31696002)(54906003)(6486002)(33964004)(2906002)(316002)(16576012)(508600001)(235185007)(110136005)(26005)(966005)(5660300002)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SC95Z1AvK0o5Ymlpd3VDcWx6bmFmS3laalpoRmZMMzVUMUk0c2gxZDMzNWNM?=
 =?utf-8?B?R0tldHVVSWcyaUZ5ZTF6eGxONmVKeHJNaTIyODc3aldxQ3E1U3JuOFU4RkE3?=
 =?utf-8?B?UW5TTHlMQ1JHVWZCSGtYU1hDd0wvSnE2eWpzK0VNQTAydkgvdnpFUTB5d2gz?=
 =?utf-8?B?OVJneXFzbTAydUdkdGd4b1FQUVovZTlLZXBjcytackJZUjk3ZVI4OFNrMEhY?=
 =?utf-8?B?bXFKSzFoWUh3VW9yclFCK2cyM1ZvdWxhaFdEYTJJelNhd1ZBU0NkTFpuL0c4?=
 =?utf-8?B?d2JlekIrS3duditVdWxPR3I0MUxLTlNGQXB6WkRPVW1lVVlNR0NqWjVwYjdr?=
 =?utf-8?B?eFZwLzZvOFkwUEt2eVBRUll3Yk1aSU5zL0EyaFI1ZWg5YS9LQzMvU0RBSTRK?=
 =?utf-8?B?ZHdtVE84bFZoS0taVjJXYkZuY1ZQei9RMDM3YStFemVnSGpVcXFZV21veERw?=
 =?utf-8?B?N1huQmIxbUVPaWpjbjluUGpvSnQ3MUg0TWZTbDBVZWorUlp1eFFzT0VHSHdE?=
 =?utf-8?B?RXNnWnhQRFpoelJVMHlmR214b3k4d04wUDdBQTRuc0lOUzZ5a2o3S2tJSS8v?=
 =?utf-8?B?NkFVWVlkM1hMMmE4dE5BTkJEV013dDFuY3JNUHAwc0lLSG1HTTBTYjlDNU12?=
 =?utf-8?B?YjlwNlVqM2FkWm9iNFZmams0MjhMeWt2aFY3QTBqSDZCWmhtNWdsS1hndlhZ?=
 =?utf-8?B?TkI5VUFJQU5qWEp3ZTE3eExEbGNJd1VMemQwTHA4ejlMazEwVU93U1pqOTVy?=
 =?utf-8?B?V0FPdlM1ZS9XcnBZNWcvcGFlN3ZOajdrOGNmRk9BZXhCSlQvd3puTlRNZUdV?=
 =?utf-8?B?MGtYUXdhbDdqYkN0REh5bk5lcU5DaCtRckQ0UE4vTldFeHlENktuSUtOYTMy?=
 =?utf-8?B?a0Z2bE9LbjN5dlVXRndPUHFYd2crT21iNDNDN2tRb0phNWNrTjg4VU5JckU4?=
 =?utf-8?B?TE1obC9IeHBwNS9CenRpODZGUG9xeHowOFA3UWRaUkszczJsdTIvdU56bHJH?=
 =?utf-8?B?VG9waERiTUQ3b0g2cmY2SUJmOS8xdzhWd3JlVkxyK3pEZ2FVdjBCTWFxYUFP?=
 =?utf-8?B?MUNTdm1FMzFQSDdHUHlMUlpkN1E0RjU3d3ZCSTN6VDJSRXl6ZTc3clVkRFlS?=
 =?utf-8?B?VW9ROHYyVG5NRWxOOHRWYkM0clhhaXJZMUg5L1ZrUmxTQTNlbWxMbnN1UFBv?=
 =?utf-8?B?OEptNnNsYUZXbXF4MFovTGpyTkE0SithYncvS0FHYURGakMyb1VySEh1bUN3?=
 =?utf-8?B?VEJ3Q1Z0UC9LSExORGZIYzNxTlRBR1BkeWh6M0EvZ2R6UVY0cTgzZDM0ZlQy?=
 =?utf-8?B?YlZXbThUSTZGMnRWQmI3VFRONmFHeGlyN1RXN0EydzRCaW85Z0Q5NlN5NHp2?=
 =?utf-8?B?SEVkYUpQMlJMa2dSdzR6c2lUNzd1VHlwK0lnKzFMVGlUZUh6dGRNeXBqQ0kr?=
 =?utf-8?B?MHNnMFZ6dXRRWWRYZFg5T1JZK0UrUnc3RzBOWEtXWnRoKzgxKy9qUFNUQXVa?=
 =?utf-8?B?MFpCbjJZeTRVQU04NWh1V21Ca0ovMFR0MklIVGVoSjNrV3IwZGhLd3pGK0xz?=
 =?utf-8?B?bmxxWkNHU3d1MGFZaHB2SFZ4a3VYd1lrV1BuSnI4a05XUzl6dDBhZ25nOHlW?=
 =?utf-8?B?ckRlZW1ML3ZkdGtySFI0MUlMcCs4QjVJcnhaTG9WZWtVU3NOaEx3UXZxYjhy?=
 =?utf-8?B?ODE4WGVDUFIyUUpLb2hscHlvdGVHY2NXR0hacS9tUVdMUEtCN2UrS3RZbysx?=
 =?utf-8?Q?/l/xbNoXuX56Ya9vR9iOucacOq2kYCvEjeYEG+e?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f5bf7880-c2d8-4174-b737-08d9829404f1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 15:24:10.0335
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lQUNkmJDulU+tGvS4ZqY2shBrTgQLZTxsmvsrGv7svn7oIXHK7isuTb6rTsl7a+3tF6acOk0NfO+WPPNmd6/zg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3775

--------------AA895F193E45FFD3D63817C7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

On 23.09.2021 04:56, Julien Grall wrote:
> We could push the patch in the branch we have. However the Linux we use is
> not fairly old (I think I did a push last year) and not even the latest
> stable.

I don't think that's a problem here - this looks to be 5.4.17-ish, which
the patch should be good for (and it does apply cleanly to plain 5.4.0).

Ian, for your setting up of a one-off flight (as just talked about),
you can find the patch at
https://lists.xen.org/archives/html/xen-devel/2021-09/msg01691.html
(and perhaps in your mailbox). In case that's easier I've also attached
it here.

Jan

--------------AA895F193E45FFD3D63817C7
Content-Type: text/plain; charset=UTF-8;
 name="linux-5.15-rc2-xen-privcmd-mmap-kvcalloc.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="linux-5.15-rc2-xen-privcmd-mmap-kvcalloc.patch"

eGVuL3ByaXZjbWQ6IHJlcGxhY2Uga2NhbGxvYygpIGJ5IGt2Y2FsbG9jKCkgd2hlbiBhbGxvY2F0
aW5nIGVtcHR5IHBhZ2VzCgpPc3N0ZXN0IGhhcyBiZWVuIHN1ZmZlcmluZyB0ZXN0IGZhaWx1cmVz
IGZvciBhIGxpdHRsZSB3aGlsZSBmcm9tIG9yZGVyLTQKYWxsb2NhdGlvbiBmYWlsdXJlcywgcmVz
dWx0aW5nIGZyb20gYWxsb2NfZW1wdHlfcGFnZXMoKSBjYWxsaW5nCmtjYWxsb2MoKS4gQXMgdGhl
cmUncyBubyBuZWVkIGZvciBwaHlzaWNhbGx5IGNvbnRpZ3VvdXMgc3BhY2UgaGVyZSwKc3dpdGNo
IHRvIGt2Y2FsbG9jKCkuCgpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxpY2hAc3Vz
ZS5jb20+CkNjOiBzdGFibGVAdmdlci5rZXJuZWwub3JnClJldmlld2VkLWJ5OiBKdWVyZ2VuIEdy
b3NzIDxqZ3Jvc3NAc3VzZS5jb20+Ci0tLQpSRkM6IEkgY2Fubm90IHJlYWxseSB0ZXN0IHRoaXMs
IGFzIGFsbG9jX2VtcHR5X3BhZ2VzKCkgb25seSBnZXRzIHVzZWQgaW4KICAgICB0aGUgYXV0by10
cmFuc2xhdGVkIGNhc2UgKGkuZS4gb24gQXJtIG9yIFBWSCBEb20wLCB0aGUgbGF0dGVyIG9mCiAg
ICAgd2hpY2ggSSdtIG5vdCB0cnVzdGluZyBlbm91Z2ggeWV0IHRvIGFjdHVhbGx5IHN0YXJ0IHBs
YXlpbmcgd2l0aAogICAgIGd1ZXN0cykuCgpUaGVyZSBhcmUgcXVpdGUgYSBmZXcgbW9yZSBrY2Fs
bG9jKCkgd2hlcmUgaXQncyBub3QgaW1tZWRpYXRlbHkgY2xlYXIKaG93IGxhcmdlIHRoZSBlbGVt
ZW50IGNvdW50cyBjb3VsZCBwb3NzaWJseSBncm93IG5vciB3aGV0aGVyIGl0IHdvdWxkIGJlCmZp
bmUgdG8gcmVwbGFjZSB0aGVtIChpLmUuIHBoeXNpY2FsbHkgY29udGlndW91cyBzcGFjZSBub3Qg
cmVxdWlyZWQpLgoKSSB3YXNuJ3Qgc3VyZSB3aGV0aGVyIHRvIENjIHN0YWJsZUAgaGVyZTsgdGhl
IGlzc3VlIGNlcnRhaW5seSBoYXMgYmVlbgpwcmVzZW50IGZvciBxdWl0ZSBzb21lIHRpbWUuIEJ1
dCBpdCBkaWRuJ3QgbG9vayB0byBjYXVzZSBpc3N1ZXMgdW50aWwKcmVjZW50bHkuCgotLS0gYS9k
cml2ZXJzL3hlbi9wcml2Y21kLmMKKysrIGIvZHJpdmVycy94ZW4vcHJpdmNtZC5jCkBAIC00MjAs
NyArNDIwLDcgQEAgc3RhdGljIGludCBhbGxvY19lbXB0eV9wYWdlcyhzdHJ1Y3Qgdm1fYQogCWlu
dCByYzsKIAlzdHJ1Y3QgcGFnZSAqKnBhZ2VzOwogCi0JcGFnZXMgPSBrY2FsbG9jKG51bXBncywg
c2l6ZW9mKHBhZ2VzWzBdKSwgR0ZQX0tFUk5FTCk7CisJcGFnZXMgPSBrdmNhbGxvYyhudW1wZ3Ms
IHNpemVvZihwYWdlc1swXSksIEdGUF9LRVJORUwpOwogCWlmIChwYWdlcyA9PSBOVUxMKQogCQly
ZXR1cm4gLUVOT01FTTsKIApAQCAtNDI4LDcgKzQyOCw3IEBAIHN0YXRpYyBpbnQgYWxsb2NfZW1w
dHlfcGFnZXMoc3RydWN0IHZtX2EKIAlpZiAocmMgIT0gMCkgewogCQlwcl93YXJuKCIlcyBDb3Vs
ZCBub3QgYWxsb2MgJWQgcGZucyByYzolZFxuIiwgX19mdW5jX18sCiAJCQludW1wZ3MsIHJjKTsK
LQkJa2ZyZWUocGFnZXMpOworCQlrdmZyZWUocGFnZXMpOwogCQlyZXR1cm4gLUVOT01FTTsKIAl9
CiAJQlVHX09OKHZtYS0+dm1fcHJpdmF0ZV9kYXRhICE9IE5VTEwpOwpAQCAtOTEyLDcgKzkxMiw3
IEBAIHN0YXRpYyB2b2lkIHByaXZjbWRfY2xvc2Uoc3RydWN0IHZtX2FyZWEKIAllbHNlCiAJCXBy
X2NyaXQoInVuYWJsZSB0byB1bm1hcCBNRk4gcmFuZ2U6IGxlYWtpbmcgJWQgcGFnZXMuIHJjPSVk
XG4iLAogCQkJbnVtcGdzLCByYyk7Ci0Ja2ZyZWUocGFnZXMpOworCWt2ZnJlZShwYWdlcyk7CiB9
CiAKIHN0YXRpYyB2bV9mYXVsdF90IHByaXZjbWRfZmF1bHQoc3RydWN0IHZtX2ZhdWx0ICp2bWYp
Cg==

--------------AA895F193E45FFD3D63817C7--



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:26:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:26:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198096.351416 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVF0D-0005Md-AO; Tue, 28 Sep 2021 15:26:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198096.351416; Tue, 28 Sep 2021 15:26:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVF0D-0005MW-4r; Tue, 28 Sep 2021 15:26:41 +0000
Received: by outflank-mailman (input) for mailman id 198096;
 Tue, 28 Sep 2021 15:26:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVF0C-0005MO-2Z
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:26:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVF0B-0002EU-WA
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:26:39 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVF0B-00063G-V3
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 15:26:39 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVF08-000462-Db; Tue, 28 Sep 2021 16:26:36 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=zJxTKZel3/62yNh9z5yeO+0UsJntiRh8Zl0HuGTLCkE=; b=1/MKSoCXm5Y1as+lrlqLhuEEEl
	6fxcVQtbk3GFTsc3Jojl1P20nSl2rl1ct0qqtkcmTFzRwtBAXFGD+5lZIJZ7+0Ffhn9xhOUduzt7D
	cREQlsvN4mgrsqvZhJdSnkBQKJvW3rVczpRrqyjIdFBcOunmvfJaeTO2cm2E72TXMjQ8=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24915.13356.139165.259646@mariner.uk.xensource.com>
Date: Tue, 28 Sep 2021 16:26:36 +0100
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
In-Reply-To: <0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
References: <20210928091517.9761-1-jgross@suse.com>
	<20210928091517.9761-3-jgross@suse.com>
	<24915.1121.8356.288414@mariner.uk.xensource.com>
	<0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Juergen Gross writes ("Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for xenstored"):
> Hmm, maybe I should just use:
> 
> prlimit --nofile=$XENSTORED_MAX_OPEN_FDS \
>     $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS

I guess that would probably work (although it involves another
exec) but I don't understand what's wrong with ulimit, which is a
shell builtin.

I think this script has to run only on Linux and all reasonable Linux
/bin/sh have `ulimit`.  (I have checked dash and bash.)

So I think just

  ulimit -n $XENSTORED_MAX_OPEN_FDS

  $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS

will DTRT.  You could also do this

  ulimit -n $XENSTORED_MAX_OPEN_FDS || true

which will arrange that if, somehow, this fails, the system is likely
to continue to mostly-work despite the error.  Whether that would be
desirable is a matter of taste I think.

(I have RTFM again, and setting -H and -S separately is not needed;
omitting -H or -S means to set both.)

Ian.


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 15:43:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 15:43:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198106.351426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFGO-0007uC-OE; Tue, 28 Sep 2021 15:43:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198106.351426; Tue, 28 Sep 2021 15:43:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFGO-0007u5-KT; Tue, 28 Sep 2021 15:43:24 +0000
Received: by outflank-mailman (input) for mailman id 198106;
 Tue, 28 Sep 2021 15:43:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVFGN-0007tv-DC; Tue, 28 Sep 2021 15:43:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVFGN-0002Wf-0T; Tue, 28 Sep 2021 15:43:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVFGM-0007Jp-LR; Tue, 28 Sep 2021 15:43:22 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVFGM-000113-Kx; Tue, 28 Sep 2021 15:43:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6DNv/bCkxZu42mGO4461OECQ9jyYVdtd6x8OiQkSVOE=; b=glilndfGPCv/JZuxGE9zb4PcPi
	QQTCf4lvftpycO0+GzsuFPyVVjZonP8WVmc8xfgy+Vt6QB/rS8E3ABcKOoCmYG38dz6/BN6zU/TWE
	HNkT9CgBYBq8hYJjJ1l2Rg2c6ODBiz+cnCIQiWLfNNxjaFwbfyfhJVpl/tt7cIJTESDA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165227-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165227: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2e46d73b4c7562f7b104e9e10fe302316af13959
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 15:43:22 +0000

flight 165227 xen-unstable real [real]
flight 165235 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165227/
http://logs.test-lab.xenproject.org/osstest/logs/165235/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2e46d73b4c7562f7b104e9e10fe302316af13959
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   17 days
Failing since        164951  2021-09-12 00:14:36 Z   16 days   33 attempts
Testing same since   165227  2021-09-28 01:55:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 992 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:06:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:06:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198116.351440 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFcH-0002eF-KV; Tue, 28 Sep 2021 16:06:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198116.351440; Tue, 28 Sep 2021 16:06:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFcH-0002e8-Gk; Tue, 28 Sep 2021 16:06:01 +0000
Received: by outflank-mailman (input) for mailman id 198116;
 Tue, 28 Sep 2021 16:06:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Q7JL=OS=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mVFcG-0002e2-Au
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:06:00 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dda31691-6ecd-440a-8d4c-91a535e1f221;
 Tue, 28 Sep 2021 16:05:59 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id b15so92973603lfe.7
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 09:05:59 -0700 (PDT)
Received: from [192.168.1.7] ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id n11sm2610772ljg.72.2021.09.28.09.05.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 28 Sep 2021 09:05:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dda31691-6ecd-440a-8d4c-91a535e1f221
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=subject:to:cc:references:from:message-id:date:user-agent
         :mime-version:in-reply-to:content-transfer-encoding:content-language;
        bh=lHlbLzNTI/p4hTFJf4CQRi5qwBbxQpaCQ9ivmAzEE7o=;
        b=ozvajN56jjIgkSwzidW4QNgqyR0ZKWwTJd3UTZeCvLSdnqQmpgeT8y6XwSW4QDfKQk
         hGfYNeWNrxEvVvXarqDosC2PLXhrbR13vAH3NEVZKtDTImnkxwZidn8gDlwO894GY9vv
         qmnkqneLOwemDBG+PKZ+B0sD0KSl9HpgqNiRfflSLdxGJVRt+NPN9lpShq9MIdCYwoDF
         DJBmxj2O/0t5SIVQea3FSV6gs3YH5Ep//RDt0sRbYcygbKpX66o0o9upht9Ew+4XzHmR
         rblqV+tv+7tTU9Aq/40KsIQneTwrMGUUJx+VWZ0TSjY/0HSMpdGYERbCQfCl87V/WUR4
         btaw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:message-id:date
         :user-agent:mime-version:in-reply-to:content-transfer-encoding
         :content-language;
        bh=lHlbLzNTI/p4hTFJf4CQRi5qwBbxQpaCQ9ivmAzEE7o=;
        b=puLGt6OwAXv1l22iSbBHorD8sGNcYB8AsISzXrk0N4+Oj4qryQhOdGYFI1eBU+Uzol
         9FiqijDPBWmU/eqAW2FAGBLcGPsPxxfSeMlow4AwNHm6EPHAqPwecabKThiZzBrSf9XZ
         qL5P7YTjO89U/0Py8+BAXF/Fm6RU6Qza1N9779o9/B8anq4saVuTVw6XEfV1xen5eOlB
         wic0qjGqXHdEu21TvZz9yvmXhGpgQa3THiocolRYQeb2jZ5fDn2TONKg3nErxnzL1Se0
         hyPMELllHmCZ+/rPZIUFv1QGMzU10g9gOdejJWx1b9S8uTR2aSUwIgXlZZtbKWaeUn/P
         XYCA==
X-Gm-Message-State: AOAM530DVrpXCJVNtuOwzLeJQ5kaq2GHJBVlcNj3+wU+nWVJss8st7Xy
	H6u8s/GhnpyyyW9bIOfQsZ0=
X-Google-Smtp-Source: ABdhPJw+/QCpKYDfoocPbq97sFjnXHE+gY6imh8HYgUJC8dR1++oYlYQPU4Ooqquz69ksQkSNbuxlw==
X-Received: by 2002:a05:651c:549:: with SMTP id q9mr51627ljp.238.1632845143236;
        Tue, 28 Sep 2021 09:05:43 -0700 (PDT)
Subject: Re: [PATCH V3 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
To: Michal Orzel <michal.orzel@arm.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <1632437334-12015-1-git-send-email-olekstysh@gmail.com>
 <1632437334-12015-2-git-send-email-olekstysh@gmail.com>
 <27840222-d5be-3df6-88bf-fb41c978791e@arm.com>
From: Oleksandr <olekstysh@gmail.com>
Message-ID: <95aa98bf-f372-9893-ecc8-bc114af841cd@gmail.com>
Date: Tue, 28 Sep 2021 19:05:41 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <27840222-d5be-3df6-88bf-fb41c978791e@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US


On 28.09.21 09:28, Michal Orzel wrote:
> Hi Oleksandr,

Hi Michal



>
> On 24.09.2021 00:48, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> We need to pass info about maximum supported guest address
>> space size to the toolstack on Arm in order to properly
>> calculate the base and size of the extended region (safe range)
>> for the guest. The extended region is unused address space which
>> could be safely used by domain for foreign/grant mappings on Arm.
>> The extended region itself will be handled by the subsequents
>> patch.
>>
>> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
>> hap_paddr_bits.
>>
>> As we change the size of structure bump the interface version.
>>
>> Suggested-by: Julien Grall <jgrall@amazon.com>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> Please note, that review comments for the RFC version [1] haven't been addressed yet.
>> It is not forgotten, some clarification is needed. It will be addressed for the next version.
>>
>> [1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/
>>
>> Changes RFC -> V2:
>>     - update patch subject/description
>>     - replace arch-specific sub-struct with common gpaddr_bits
>>       field and update code to reflect that
>>
>> Changes V2 -> V3:
>>     - make the field uint8_t and add uint8_t pad[7] after
>>     - remove leading blanks in libxl.h
>> ---
>>   tools/include/libxl.h            | 7 +++++++
>>   tools/libs/light/libxl.c         | 2 ++
>>   tools/libs/light/libxl_types.idl | 2 ++
>>   xen/arch/arm/sysctl.c            | 2 ++
>>   xen/arch/x86/sysctl.c            | 2 ++
>>   xen/include/public/sysctl.h      | 4 +++-
>>   6 files changed, 18 insertions(+), 1 deletion(-)
>>
> Don't you want to print gpaddr_bits field of xen_sysctl_physinfo from output_physinfo (xl_info.c)?

Good point, will do, thank you.


>
> Apart from that:
> Reviewed-by: Michal Orzel <michal.orzel@arm.com>

Thanks


>
> Cheers

-- 
Regards,

Oleksandr Tyshchenko



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:12:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:12:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198123.351451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFi5-0004En-EY; Tue, 28 Sep 2021 16:12:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198123.351451; Tue, 28 Sep 2021 16:12:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFi5-0004Eg-9k; Tue, 28 Sep 2021 16:12:01 +0000
Received: by outflank-mailman (input) for mailman id 198123;
 Tue, 28 Sep 2021 16:11:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVFi3-0004EK-P5
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:11:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da718de5-93a7-426e-8c75-50d5a68fe433;
 Tue, 28 Sep 2021 16:11:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D666B60F24;
 Tue, 28 Sep 2021 16:11:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da718de5-93a7-426e-8c75-50d5a68fe433
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632845517;
	bh=OFwRnUPTbnyoe0TzkxFtbbQ7PcfvetoGmafQ5IKdHLE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Elz6LBXZ5fgppsnMq6FKn9e5YNZkQWh3a1YaVfsiLEt/1JfCjSPZECSRv8BfUGIxz
	 4U9dMn59/JC9/O7O/DRKZolbYWqxgmil+lagOYpXViVUjjtyF/JsqTitcekSUpcF3E
	 Tl6qRdSoPEee2YT6/mV2tdhlV1uRydAPbjxDkSwgFcW14vkl8rXVz06oD6tBwKu5Vg
	 9KjHLK63zZjVx7U2GAdPn1L427++gZpK7pBNpOwR806FdtU2ac9VQf/ByV2/3cb8ti
	 gBuC2nfzYZWUmNLLVMw6lTqCdwlQurUTySzCHGpfMMrLEvjm66gvqMPUmpZkZVhU43
	 cxj32fKKr4tyg==
Date: Tue, 28 Sep 2021 09:11:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "julien@xen.org" <julien@xen.org>, 
    Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Artem Mygaiev <Artem_Mygaiev@epam.com>, 
    "roger.pau@citrix.com" <roger.pau@citrix.com>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Rahul Singh <rahul.singh@arm.com>
Subject: Re: [PATCH v2 10/11] xen/arm: Do not map PCI ECAM and MMIO space to
 Domain-0's p2m
In-Reply-To: <32ffee9e-8786-c91b-681c-7fa243f5d3fa@epam.com>
Message-ID: <alpine.DEB.2.21.2109280911030.5022@sstabellini-ThinkPad-T480s>
References: <20210923125438.234162-1-andr2000@gmail.com> <20210923125438.234162-11-andr2000@gmail.com> <alpine.DEB.2.21.2109241724210.17979@sstabellini-ThinkPad-T480s> <d5704571-eade-4bc8-b6c6-f4b3810b8b21@epam.com> <alpine.DEB.2.21.2109272046320.5022@sstabellini-ThinkPad-T480s>
 <cc05efe3-2b3a-be03-2c17-21578b09a013@epam.com> <32ffee9e-8786-c91b-681c-7fa243f5d3fa@epam.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1136262037-1632845517=:5022"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1136262037-1632845517=:5022
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 28 Sep 2021, Oleksandr Andrushchenko wrote:
> [snip]
> >> Sorry I didn't follow your explanation.
> >>
> >> My suggestion is to remove the #ifdef CONFIG_HAS_PCI completely from
> >> map_range_to_domain. At the beginning of map_range_to_domain, there is
> >> already this line:
> >>
> >> bool need_mapping = !dt_device_for_passthrough(dev);
> >>
> >> We can change it into:
> >>
> >> bool need_mapping = !dt_device_for_passthrough(dev) &&
> >>                       !mr_data->skip_mapping;
> >>
> >>
> >> Then, in map_device_children we can set mr_data->skip_mapping to true
> >> for PCI devices.
> > This is the key. I am fine with this, but it just means we move the
> >
> > check to the outside of this function which looks good. Will do
> >
> >>    There is already a pci check there:
> >>
> >>    if ( dt_device_type_is_equal(dev, "pci") )
> >>
> >> so it should be easy to do. What am I missing?
> >>
> >>
> >
> I did some experiments. If we move the check to map_device_children
> 
> it is not enough because part of the ranges is still mapped at handle_device level:
> 
> handle_device:
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd0e0000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd480000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr 8000000000
> 
> map_device_children:
> (XEN) Mapping children of /axi/pcie@fd0e0000 to guest skip 1
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr e0000000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr 600000000
> 
> pci_host_bridge_mappings:
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd0e0000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd480000
> 
> So, I did more intrusive change:
> 
> @@ -1540,6 +1534,12 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
>       int res;
>       u64 addr, size;
>       bool need_mapping = !dt_device_for_passthrough(dev);
> +    struct map_range_data mr_data = {
> +        .d = d,
> +        .p2mt = p2mt,
> +        .skip_mapping = is_pci_passthrough_enabled() &&
> +                        (device_get_class(dev) == DEVICE_PCI)
> +    };
> 
> With this I see that now mappings are done correctly:
> 
> handle_device:
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr fd0e0000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr fd480000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr 8000000000
> 
> map_device_children:
> (XEN) Mapping children of /axi/pcie@fd0e0000 to guest skip 1
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr e0000000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 0 addr 600000000
> 
> pci_host_bridge_mappings:
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd0e0000
> (XEN) --- /axi/pcie@fd0e0000 need_mapping 1 addr fd480000
> 
> So, handle_device seems to be the right place. While at it I have also
> 
> optimized the way we setup struct map_range_data mr_data in both
> 
> handle_device and map_device_children: I removed structure initialization
> 
> from within the relevant loop and also pass mr_data to map_device_children,
> 
> so it doesn't need to create its own copy of the same and perform yet
> 
> another computation for .skip_mapping: it does need to not only know
> 
> that dev is a PCI device (this is done by the dt_device_type_is_equal(dev, "pci")
> 
> check, but also account on is_pci_passthrough_enabled().
> 
> Thus, the change will be more intrusive, but I hope will simplify things.
> 
> I am attaching the fixup patch for just in case you want more details.

Yes, thanks, this is what I had in mind. Hopefully the resulting
combined patch will be simpler.

Cheers,

Stefano
--8323329-1136262037-1632845517=:5022--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:16:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:16:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198128.351462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFm4-0004so-VH; Tue, 28 Sep 2021 16:16:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198128.351462; Tue, 28 Sep 2021 16:16:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVFm4-0004sh-RM; Tue, 28 Sep 2021 16:16:08 +0000
Received: by outflank-mailman (input) for mailman id 198128;
 Tue, 28 Sep 2021 16:16:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVFm4-0004sb-8w
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:16:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVFm4-0003bk-3j
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:16:08 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVFm4-0003Ip-2f
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:16:08 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVFlz-0004J0-CB; Tue, 28 Sep 2021 17:16:03 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=B0DwB6TERYc9bT67O8Pi5glYrbEF01l5SGQwpTN/Ne4=; b=GSjST3W4UcpD6eX2xrnwwHetxg
	3JqJE4Rp7JAL6Om1wvCTz3ipFeAZqdjEhCIJNf3AFs24sSlP8vbUYOeughQxAl/yCobLYD8RhloTs
	lg81Ln1a5dBMeSomf30D9AJnjGyTZW6freASum6U6YpiEKCB1RRBTGGdYfM5uQP8LBKk=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24915.16323.60012.172236@mariner.uk.xensource.com>
Date: Tue, 28 Sep 2021 17:16:03 +0100
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien.grall.oss@gmail.com>,
    xen-devel <xen-devel@lists.xenproject.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    dpsmith@apertussolutions.com
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <dda286dc-1edc-4505-a049-e82047b44590@suse.com>
References: <osstest-164996-mainreport@xen.org>
	<d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
	<24904.44119.940679.241639@mariner.uk.xensource.com>
	<alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
	<19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
	<alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
	<CAJ=z9a1W9BqeOGFu3F2+qbXM6UNo8ZO=0y-zYsnK01MNSNdbkw@mail.gmail.com>
	<dda286dc-1edc-4505-a049-e82047b44590@suse.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Jan Beulich writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> Ian, for your setting up of a one-off flight (as just talked about),
> you can find the patch at
> https://lists.xen.org/archives/html/xen-devel/2021-09/msg01691.html
> (and perhaps in your mailbox). In case that's easier I've also attached
> it here.
...
> [DELETED ATTACHMENT linux-5.15-rc2-xen-privcmd-mmap-kvcalloc.patch, plain text]

Thanks.  The attachment didn't git-am but I managed to make a tree
with it in (but a bogus commit message).

I have a repro of 165218 test-arm64-arm64-libvirt-raw (that's the last
xen-unstable flight) running.  If all goes well it will rebuild Linux
from my branch (new flight 165241) and then run the test using that
kernel (new flight 165242).  I have told it to report to the people on
this thread (and the list).

It will probably report in an hour or two (since it needs to rebuild a
kernel and then negotiate to get a host to run the repro on).
I didn't ask it to keep the host for me, but it ought to publish the
logs and as I say, send an email report here.

Ian.

For my reference:

./mg-transient-task ./mg-repro-setup -P -E...,iwj@xenproject.org,... 165218 test-arm64-arm64-libvirt-raw X --rebuild +linux=https://xenbits.xen.org/git-http/people/iwj/linux.git#164996-fix alloc:equiv-rochester



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:32:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:32:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198138.351473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG1v-0007RM-Bk; Tue, 28 Sep 2021 16:32:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198138.351473; Tue, 28 Sep 2021 16:32:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG1v-0007RF-8g; Tue, 28 Sep 2021 16:32:31 +0000
Received: by outflank-mailman (input) for mailman id 198138;
 Tue, 28 Sep 2021 16:32:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVG1t-0007R3-Oz
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:32:29 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id bdecb2e3-0929-4951-ac8f-c2a56f2aacaa;
 Tue, 28 Sep 2021 16:32:27 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7ECA1D6E;
 Tue, 28 Sep 2021 09:32:27 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.199.61])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E9E2E3F718;
 Tue, 28 Sep 2021 09:32:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdecb2e3-0929-4951-ac8f-c2a56f2aacaa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 0/3] arm/efi: Add dom0less support to UEFI boot
Date: Tue, 28 Sep 2021 17:32:06 +0100
Message-Id: <20210928163209.49611-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1

This serie introduces a way to start a dom0less setup when Xen is booting as EFI
application.
Using the device tree it's now possible to fetch from the disk and load in
memory all the modules needed to start any domU defined in the DT.
Dom0less for now is supported only by the arm architecture.

Luca Fancellu (3):
  arm/efi: Introduce uefi,cfg-load DT property
  arm/efi: Use dom0less configuration when using EFI boot
  arm/efi: load dom0 modules from DT using UEFI

 docs/misc/arm/device-tree/booting.txt |  37 +++
 docs/misc/efi.pandoc                  | 263 +++++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 361 +++++++++++++++++++++++++-
 xen/arch/x86/efi/efi-boot.h           |   6 +
 xen/common/efi/boot.c                 |  54 ++--
 5 files changed, 696 insertions(+), 25 deletions(-)

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:32:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:32:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198139.351484 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG1w-0007hC-J4; Tue, 28 Sep 2021 16:32:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198139.351484; Tue, 28 Sep 2021 16:32:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG1w-0007h5-G8; Tue, 28 Sep 2021 16:32:32 +0000
Received: by outflank-mailman (input) for mailman id 198139;
 Tue, 28 Sep 2021 16:32:31 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVG1u-0007R9-UI
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:32:30 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ac33e685-2079-11ec-bcab-12813bfff9fa;
 Tue, 28 Sep 2021 16:32:29 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D682101E;
 Tue, 28 Sep 2021 09:32:29 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.199.61])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B20413F718;
 Tue, 28 Sep 2021 09:32:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac33e685-2079-11ec-bcab-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 1/3] arm/efi: Introduce uefi,cfg-load DT property
Date: Tue, 28 Sep 2021 17:32:07 +0100
Message-Id: <20210928163209.49611-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210928163209.49611-1-luca.fancellu@arm.com>
References: <20210928163209.49611-1-luca.fancellu@arm.com>

Introduce the uefi,cfg-load DT property of /chosen
node for ARM whose presence decide whether to force
the load of the UEFI Xen configuration file.

The logic is that if any multiboot,module is found in
the DT, then the uefi,cfg-load property is used to see
if the UEFI Xen configuration file is needed.

Modify a comment in efi_arch_use_config_file, removing
the part that states "dom0 required" because it's not
true anymore with this commit.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
v3 changes:
- add documentation to misc/arm/device-tree/booting.txt
- Modified variable name and logic from skip_cfg_file to
load_cfg_file
- Add in the commit message that I'm modifying a comment.
v2 changes:
- Introduced uefi,cfg-load property
- Add documentation about the property
---
 docs/misc/arm/device-tree/booting.txt |  8 ++++++++
 docs/misc/efi.pandoc                  |  2 ++
 xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
 3 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 44cd9e1a9a..cf878b478e 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
 for Dom0 and bootargs for native Linux.
 
 
+UEFI boot and DT
+================
+
+When Xen is booted using UEFI, it doesn't read the configuration file if any
+multiboot module is specified. To force Xen to load the configuration file, the
+boolean property uefi,cfg-load must be declared in the /chosen node.
+
+
 Creating Multiple Domains directly from Xen
 ===========================================
 
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ac3cd58cae..e289c5e7ba 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
 bootloader provides a device tree containing modules then any configuration
 files are ignored, and the bootloader is responsible for populating all
 relevant device tree nodes.
+The property "uefi,cfg-load" can be specified in the /chosen node to force Xen
+to load the configuration file even if multiboot modules are found.
 
 Once built, `make install-xen` will place the resulting binary directly into
 the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index cf9c37153f..4f1b01757d 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
 
 static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
 {
+    bool load_cfg_file = true;
     /*
      * For arm, we may get a device tree from GRUB (or other bootloader)
      * that contains modules that have already been loaded into memory.  In
-     * this case, we do not use a configuration file, and rely on the
-     * bootloader to have loaded all required modules and appropriate
-     * options.
+     * this case, we search for the property uefi,cfg-load in the /chosen node
+     * to decide whether to skip the UEFI Xen configuration file or not.
      */
 
     fdt = lookup_fdt_config_table(SystemTable);
     dtbfile.ptr = fdt;
     dtbfile.need_to_free = false; /* Config table memory can't be freed. */
-    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
+
+    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
+    {
+        /* Locate chosen node */
+        int node = fdt_subnode_offset(fdt, 0, "chosen");
+        const void *cfg_load_prop;
+        int cfg_load_len;
+
+        if ( node > 0 )
+        {
+            /* Check if uefi,cfg-load property exists */
+            cfg_load_prop = fdt_getprop(fdt, node, "uefi,cfg-load",
+                                        &cfg_load_len);
+            if ( !cfg_load_prop )
+                load_cfg_file = false;
+        }
+    }
+
+    if ( !fdt || load_cfg_file )
     {
         /*
          * We either have no FDT, or one without modules, so we must have a
-         * Xen EFI configuration file to specify modules.  (dom0 required)
+         * Xen EFI configuration file to specify modules.
          */
         return true;
     }
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:32:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:32:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198140.351494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG20-000804-S8; Tue, 28 Sep 2021 16:32:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198140.351494; Tue, 28 Sep 2021 16:32:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG20-0007zt-P6; Tue, 28 Sep 2021 16:32:36 +0000
Received: by outflank-mailman (input) for mailman id 198140;
 Tue, 28 Sep 2021 16:32:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVG1y-0007R3-Nf
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:32:34 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 4405c5ac-9b16-4a03-aa54-cd30a5cfe7dc;
 Tue, 28 Sep 2021 16:32:31 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 130D8D6E;
 Tue, 28 Sep 2021 09:32:31 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.199.61])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 623E33F718;
 Tue, 28 Sep 2021 09:32:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4405c5ac-9b16-4a03-aa54-cd30a5cfe7dc
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 2/3] arm/efi: Use dom0less configuration when using EFI boot
Date: Tue, 28 Sep 2021 17:32:08 +0100
Message-Id: <20210928163209.49611-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210928163209.49611-1-luca.fancellu@arm.com>
References: <20210928163209.49611-1-luca.fancellu@arm.com>

This patch introduces the support for dom0less configuration
when using UEFI boot on ARM, it permits the EFI boot to
continue if no dom0 kernel is specified but at least one domU
is found.

Introduce the new property "uefi,binary" for device tree boot
module nodes that are subnode of "xen,domain" compatible nodes.
The property holds a string containing the file name of the
binary that shall be loaded by the uefi loader from the filesystem.

Update efi documentation about how to start a dom0less
setup using UEFI

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v3:
- fixed documentation
- fixed name len in strlcpy
- fixed some style issues
- closed filesystem handle before calling blexit
- passed runtime errors up to the stack instead
of calling blexit
- renamed names and function to make them more
general in prevision to load also Dom0 kernel
and ramdisk from DT
Changes in v2:
- remove array of struct file
- fixed some int types
- Made the code use filesystem even when configuration
file is skipped.
- add documentation of uefi,binary in booting.txt
- add documentation on how to boot all configuration
for Xen using UEFI in efi.pandoc
---
 docs/misc/arm/device-tree/booting.txt |  21 ++
 docs/misc/efi.pandoc                  | 203 +++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 305 +++++++++++++++++++++++++-
 xen/arch/x86/efi/efi-boot.h           |   6 +
 xen/common/efi/boot.c                 |  42 ++--
 5 files changed, 562 insertions(+), 15 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index cf878b478e..354bb43fe1 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
 
     Command line parameters for the guest kernel.
 
+- uefi,binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 The ramdisk sub-node has the following properties:
 
 - compatible
@@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
     Specifies the physical address of the ramdisk in RAM and its
     length.
 
+- uefi,binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 
 Example
 =======
@@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
     Specifies the physical address of the device tree binary fragment
     RAM and its length.
 
+- uefi,binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 As an example:
 
         module@0xc000000 {
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index e289c5e7ba..800e67a233 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,3 +167,206 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+
+## UEFI boot and dom0less on ARM
+
+Dom0less feature is supported by ARM and it is possible to use it when Xen is
+started as an EFI application.
+The way to specify the domU domains is by Device Tree as specified in the
+[dom0less](dom0less.html) documentation page under the "Device Tree
+configuration" section, but instead of declaring the reg property in the boot
+module, the user must specify the "uefi,binary" property containing the name
+of the binary file that has to be loaded in memory.
+The UEFI stub will load the binary in memory and it will add the reg property
+accordingly.
+
+An example here:
+
+domU1 {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	compatible = "xen,domain";
+	memory = <0 0x20000>;
+	cpus = <1>;
+	vpl011;
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "console=ttyAMA0";
+	};
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "initrd-3.0.31-0.4-xen";
+	};
+	module@3 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "passthrough.dtb";
+	};
+};
+
+## How to boot different Xen setup using UEFI
+
+These are the different ways to boot a Xen system from UEFI:
+
+ - Boot Xen and Dom0 (minimum required)
+ - Boot Xen and DomU(s) (true dom0less, only on ARM)
+ - Boot Xen, Dom0 and DomU(s) (only on ARM)
+
+### Boot Xen and Dom0
+
+This configuration can be started using the Xen configuration file in the
+example above.
+
+### Boot Xen and DomU(s)
+
+This configuration needs the domU domain(s) specified in the /chosen node,
+examples of how to do that are provided by the documentation about dom0less
+and the example above shows how to use the "uefi,binary" property to use the
+UEFI stub for module loading.
+When adding DomU modules to device tree, also add the property
+uefi,cfg-load under chosen for Xen to load the Xen config file.
+Otherwise, Xen will skip the config file and rely on device tree alone.
+
+Example 1 of how to boot a true dom0less configuration:
+
+Xen configuration file: skipped.
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "<Xen command line>"
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+Example 2 of how to boot a true dom0less configuration:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	uefi,cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+### Boot Xen, Dom0 and DomU(s)
+
+This configuration is a mix of the two configuration above, to boot this one
+the configuration file must be processed so the /chosen node must have the
+"uefi,cfg-load" property.
+
+Here an example:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
+ramdisk=initrd-3.0.31-0.4-xen
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	uefi,cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 4f1b01757d..4f7c913f86 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -8,9 +8,49 @@
 #include <asm/setup.h>
 #include <asm/smp.h>
 
+typedef struct {
+    char *name;
+    unsigned int name_len;
+    EFI_PHYSICAL_ADDRESS addr;
+    UINTN size;
+} module_name;
+
+/*
+ * Binaries will be translated into bootmodules, the maximum number for them is
+ * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
+ */
+#define MAX_UEFI_MODULES (MAX_MODULES - 2)
+static struct file __initdata module_binary;
+static module_name __initdata modules[MAX_UEFI_MODULES];
+static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
+static unsigned int __initdata modules_idx;
+
+#define ERROR_BINARY_FILE_NOT_FOUND (-1)
+#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
+#define ERROR_ALLOC_MODULE_NAME     (-2)
+#define ERROR_MISSING_DT_PROPERTY   (-3)
+#define ERROR_RENAME_MODULE_NAME    (-4)
+#define ERROR_SET_REG_PROPERTY      (-5)
+#define ERROR_DT_MODULE_DOMU        (-1)
+#define ERROR_DT_CHOSEN_NODE        (-2)
+
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
 
+static int get_module_file_index(const char *name, unsigned int name_len);
+static void PrintMessage(const CHAR16 *s);
+static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                const char *name,
+                                unsigned int name_len);
+static int handle_module_node(EFI_FILE_HANDLE dir_handle,
+                              int module_node_offset,
+                              int reg_addr_cells,
+                              int reg_size_cells);
+static bool is_boot_module(int dt_module_offset);
+static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                       int domain_node);
+static int efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle);
+
 #define DEVICE_TREE_GUID \
 {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
 
@@ -552,8 +592,260 @@ static void __init efi_arch_handle_module(const struct file *file,
                          kernel.size) < 0 )
             blexit(L"Unable to set reg property.");
     }
-    else
+    else if ( file != &module_binary )
+        /*
+         * If file is not a dom0 module file and it's not a domU module,
+         * stop here.
+         */
         blexit(L"Unknown module type");
+
+    /*
+     * modules_available is decremented here because for each dom0
+     * file added, there will be an additional bootmodule, so the number
+     * of dom0less module files will be decremented because there is
+     * a maximum amount of bootmodules that can be loaded.
+     */
+    modules_available--;
+}
+
+/*
+ * This function checks for a binary previously loaded with a give name, it
+ * returns the index of the file in the modules array or a negative number if no
+ * file with that name is found.
+ */
+static int __init get_module_file_index(const char *name,
+                                        unsigned int name_len)
+{
+    unsigned int i;
+    int ret = ERROR_BINARY_FILE_NOT_FOUND;
+
+    for ( i = 0; i < modules_idx; i++ )
+    {
+        module_name *mod = &modules[i];
+        if ( (mod->name_len == name_len) &&
+             (strncmp(mod->name, name, name_len) == 0) )
+        {
+            ret = i;
+            break;
+        }
+    }
+    return ret;
+}
+
+static void __init PrintMessage(const CHAR16 *s)
+{
+    PrintStr(s);
+    PrintStr(newline);
+}
+
+/*
+ * This function allocates a binary and keeps track of its name, it returns the
+ * index of the file in the modules array or a negative number on error.
+ */
+static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                       const char *name,
+                                       unsigned int name_len)
+{
+    module_name *file_name;
+    union string module_name;
+    int ret;
+
+    /*
+     * Check if there is any space left for a module, the variable
+     * modules_available is updated each time we use read_file(...)
+     * successfully.
+     */
+    if ( !modules_available )
+    {
+        PrintMessage(L"No space left for modules");
+        return ERROR_ALLOC_MODULE_NO_SPACE;
+    }
+
+    module_name.cs = name;
+    ret = modules_idx;
+
+    /* Save at this index the name of this binary */
+    file_name = &modules[ret];
+
+    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
+                              (void**)&file_name->name) != EFI_SUCCESS )
+    {
+        PrintMessage(L"Error allocating memory for module binary name");
+        return ERROR_ALLOC_MODULE_NAME;
+    }
+
+    /* Save name and length of the binary in the data structure */
+    strlcpy(file_name->name, name, name_len + 1);
+    file_name->name_len = name_len;
+
+    /* Load the binary in memory */
+    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
+
+    /* Save address and size */
+    file_name->addr = module_binary.addr;
+    file_name->size = module_binary.size;
+
+    /* s2w(...) allocates some memory, free it */
+    efi_bs->FreePool(module_name.w);
+
+    modules_idx++;
+
+    return ret;
+}
+
+/*
+ * This function checks for the presence of the uefi,binary property in the
+ * module, if found it loads the binary as module and sets the right address
+ * for the reg property into the module DT node.
+ */
+static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
+                                     int module_node_offset,
+                                     int reg_addr_cells,
+                                     int reg_size_cells)
+{
+    const void *uefi_name_prop;
+    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
+    int uefi_name_len, file_idx;
+    module_name *file;
+
+    /* Read uefi,binary property to get the file name. */
+    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
+                                 &uefi_name_len);
+
+    if ( !uefi_name_prop )
+        /* Property not found */
+        return 0;
+
+    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
+    if ( file_idx < 0 )
+    {
+        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
+                                        uefi_name_len);
+        if ( file_idx < 0 )
+            return file_idx;
+    }
+
+    file = &modules[file_idx];
+
+    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
+
+    /* Rename the module to be module@{address} */
+    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
+    {
+        PrintMessage(L"Unable to modify module node name.");
+        return ERROR_RENAME_MODULE_NAME;
+    }
+
+    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
+                     file->addr, file->size) < 0 )
+    {
+        PrintMessage(L"Unable to set module reg property.");
+        return ERROR_SET_REG_PROPERTY;
+    }
+
+    return 0;
+}
+
+static bool __init is_boot_module(int dt_module_offset)
+{
+    if ( (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,kernel") == 0) ||
+         (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,ramdisk") == 0) ||
+         (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,device-tree") == 0) )
+        return true;
+
+    return false;
+}
+
+/*
+ * This function checks for boot modules under the domU guest domain node
+ * in the DT.
+ * Returns 0 on success, negative number on error.
+ */
+static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                              int domain_node)
+{
+    int module_node, addr_cells, size_cells, len;
+    const struct fdt_property *prop;
+
+    /* Get #address-cells and #size-cells from domain node */
+    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#address-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#size-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    /*
+     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
+     * inside this node
+     */
+    for ( module_node = fdt_first_subnode(fdt, domain_node);
+          module_node > 0;
+          module_node = fdt_next_subnode(fdt, module_node) )
+        if ( is_boot_module(module_node) )
+        {
+            int ret = handle_module_node(dir_handle, module_node, addr_cells,
+                                         size_cells);
+            if ( ret < 0 )
+                return ret;
+        }
+
+    return 0;
+}
+
+/*
+ * This function checks for xen domain nodes under the /chosen node for possible
+ * domU guests to be loaded.
+ * Returns the number of modules loaded or a negative number for error.
+ */
+static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
+{
+    int chosen, node, addr_len, size_len;
+    unsigned int i = 0;
+
+    /* Check for the chosen node in the current DTB */
+    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
+    if ( chosen < 0 )
+    {
+        PrintMessage(L"Unable to setup chosen node");
+        return ERROR_DT_CHOSEN_NODE;
+    }
+
+    /* Check for nodes compatible with xen,domain under the chosen node */
+    for ( node = fdt_first_subnode(fdt, chosen);
+          node > 0;
+          node = fdt_next_subnode(fdt, node) )
+    {
+        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
+        {
+            /* Found a node with compatible xen,domain; handle this node. */
+            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
+                return ERROR_DT_MODULE_DOMU;
+        }
+    }
+
+    /* Free dom0less file names if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free dom0less binary names */
+        efi_bs->FreePool(modules[i].name);
+    }
+
+    return modules_idx;
 }
 
 static void __init efi_arch_cpu(void)
@@ -562,8 +854,19 @@ static void __init efi_arch_cpu(void)
 
 static void __init efi_arch_blexit(void)
 {
+    unsigned int i = 0;
+
     if ( dtbfile.need_to_free )
         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
+    /* Free dom0less files if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free dom0less binary names */
+        efi_bs->FreePool(modules[i].name);
+        /* Free dom0less binaries */
+        efi_bs->FreePages(modules[i].addr,
+                          PFN_UP(modules[i].size));
+    }
     if ( memmap )
         efi_bs->FreePool(memmap);
 }
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 9b0cc29aae..f5538bfad4 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
     efi_bs->FreePool(ptr);
 }
 
+static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
+{
+    /* x86 doesn't support device tree boot */
+    return 0;
+}
+
 static void __init efi_arch_cpu(void)
 {
     uint32_t eax = cpuid_eax(0x80000000);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 758f9d74d2..c8c57fbb54 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
     EFI_LOADED_IMAGE *loaded_image;
     EFI_STATUS status;
-    unsigned int i, argc;
-    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
+    unsigned int i, argc = 0;
+    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
     UINTN gop_mode = ~0;
     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
     union string section = { NULL }, name;
     bool base_video = false;
-    const char *option_str;
+    const char *option_str = NULL;
     bool use_cfg_file;
+    int dt_module_found;
+    EFI_FILE_HANDLE dir_handle;
 
     __set_bit(EFI_BOOT, &efi_flags);
     __set_bit(EFI_LOADER, &efi_flags);
@@ -1216,9 +1218,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
     efi_arch_relocate_image(0);
 
+    /* Get the file system interface. */
+    dir_handle = get_parent_handle(loaded_image, &file_name);
+
     if ( use_cfg_file )
     {
-        EFI_FILE_HANDLE dir_handle;
         UINTN depth, cols, rows, size;
 
         size = cols = rows = depth = 0;
@@ -1229,9 +1233,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
         gop = efi_get_gop();
 
-        /* Get the file system interface. */
-        dir_handle = get_parent_handle(loaded_image, &file_name);
-
         /* Read and parse the config file. */
         if ( read_section(loaded_image, L"config", &cfg, NULL) )
             PrintStr(L"Using builtin config file\r\n");
@@ -1285,14 +1286,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
             efi_bs->FreePool(name.w);
         }
 
-        if ( !name.s )
-            blexit(L"No Dom0 kernel image specified.");
-
         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
 
-        option_str = split_string(name.s);
+        if ( name.s )
+            option_str = split_string(name.s);
 
-        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
+        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
+             name.s )
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
@@ -1361,12 +1361,26 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
         cfg.addr = 0;
 
-        dir_handle->Close(dir_handle);
-
         if ( gop && !base_video )
             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
     }
 
+    dt_module_found = efi_arch_check_dt_boot(dir_handle);
+
+    dir_handle->Close(dir_handle);
+
+    if (dt_module_found < 0)
+        /* efi_arch_check_dt_boot throws some error */
+        blexit(L"Error processing boot modules on DT.");
+    /*
+     * Check if a proper configuration is provided to start Xen:
+     *  - Dom0 specified (minimum required)
+     *  - Dom0 and DomU(s) specified
+     *  - DomU(s) specified
+     */
+    if ( !dt_module_found && !kernel.addr )
+        blexit(L"No Dom0 kernel image specified.");
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:32:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:32:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198141.351506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG25-0008Mb-9l; Tue, 28 Sep 2021 16:32:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198141.351506; Tue, 28 Sep 2021 16:32:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG25-0008MO-5q; Tue, 28 Sep 2021 16:32:41 +0000
Received: by outflank-mailman (input) for mailman id 198141;
 Tue, 28 Sep 2021 16:32:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hUli=OS=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVG23-0007R3-Nl
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:32:39 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 893ccb0f-36a9-4a86-9c12-3d2e591339f3;
 Tue, 28 Sep 2021 16:32:33 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B3AB1101E;
 Tue, 28 Sep 2021 09:32:32 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.199.61])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 478163F718;
 Tue, 28 Sep 2021 09:32:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 893ccb0f-36a9-4a86-9c12-3d2e591339f3
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 3/3] arm/efi: load dom0 modules from DT using UEFI
Date: Tue, 28 Sep 2021 17:32:09 +0100
Message-Id: <20210928163209.49611-4-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210928163209.49611-1-luca.fancellu@arm.com>
References: <20210928163209.49611-1-luca.fancellu@arm.com>

Add support to load Dom0 boot modules from
the device tree using the uefi,binary property.

Update documentation about that.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v3:
- new patch
---
 docs/misc/arm/device-tree/booting.txt |  8 ++++
 docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
 xen/arch/arm/efi/efi-boot.h           | 36 +++++++++++++--
 xen/common/efi/boot.c                 | 12 ++---
 4 files changed, 108 insertions(+), 12 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 354bb43fe1..e73f6476d4 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -70,6 +70,14 @@ Each node contains the following properties:
 	priority of this field vs. other mechanisms of specifying the
 	bootargs for the kernel.
 
+- uefi,binary (UEFI boot only)
+
+	String property that specifies the file name to be loaded by the UEFI
+	boot for this module. If this is specified, there is no need to specify
+	the reg property because it will be created by the UEFI stub on boot.
+	This option is needed only when UEFI boot is used, the node needs to be
+	compatible with multiboot,kernel or multiboot,ramdisk.
+
 Examples
 ========
 
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index 800e67a233..4cebc47a18 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,6 +167,28 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+## UEFI boot and Dom0 modules on ARM
+
+When booting using UEFI on ARM, it is possible to specify the Dom0 modules
+directly from the device tree without using the Xen configuration file, here an
+example:
+
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "initrd-3.0.31-0.4-xen";
+	};
+}
 
 ## UEFI boot and dom0less on ARM
 
@@ -326,10 +348,10 @@ chosen {
 ### Boot Xen, Dom0 and DomU(s)
 
 This configuration is a mix of the two configuration above, to boot this one
-the configuration file must be processed so the /chosen node must have the
-"uefi,cfg-load" property.
+the configuration file can be processed or the Dom0 modules can be read from
+the device tree.
 
-Here an example:
+Here the first example:
 
 Xen configuration file:
 
@@ -369,4 +391,40 @@ chosen {
 };
 ```
 
+Here the second example:
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		uefi,binary = "initrd-3.0.31-0.4-xen";
+	};
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
 
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			uefi,binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 4f7c913f86..df63387136 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -31,8 +31,10 @@ static unsigned int __initdata modules_idx;
 #define ERROR_MISSING_DT_PROPERTY   (-3)
 #define ERROR_RENAME_MODULE_NAME    (-4)
 #define ERROR_SET_REG_PROPERTY      (-5)
+#define ERROR_DOM0_ALREADY_FOUND    (-6)
 #define ERROR_DT_MODULE_DOMU        (-1)
 #define ERROR_DT_CHOSEN_NODE        (-2)
+#define ERROR_DT_MODULE_DOM0        (-3)
 
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
@@ -45,7 +47,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int handle_module_node(EFI_FILE_HANDLE dir_handle,
                               int module_node_offset,
                               int reg_addr_cells,
-                              int reg_size_cells);
+                              int reg_size_cells,
+                              bool is_domu_module);
 static bool is_boot_module(int dt_module_offset);
 static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
                                        int domain_node);
@@ -701,7 +704,8 @@ static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
                                      int module_node_offset,
                                      int reg_addr_cells,
-                                     int reg_size_cells)
+                                     int reg_size_cells,
+                                     bool is_domu_module)
 {
     const void *uefi_name_prop;
     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
@@ -743,6 +747,24 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
         return ERROR_SET_REG_PROPERTY;
     }
 
+    if ( !is_domu_module &&
+         (fdt_node_check_compatible(fdt, module_node_offset,
+                                    "multiboot,kernel") == 0) )
+    {
+        /*
+         * This is the Dom0 kernel, wire it to the kernel variable because it
+         * will be verified by the shim lock protocol later in the common code.
+         */
+        if ( kernel.addr )
+        {
+            PrintMessage(L"Dom0 kernel already found in cfg file.");
+            return ERROR_DOM0_ALREADY_FOUND;
+        }
+        kernel.need_to_free = false; /* Freed using the module array */
+        kernel.addr = file->addr;
+        kernel.size = file->size;
+    }
+
     return 0;
 }
 
@@ -799,7 +821,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
         if ( is_boot_module(module_node) )
         {
             int ret = handle_module_node(dir_handle, module_node, addr_cells,
-                                         size_cells);
+                                         size_cells, true);
             if ( ret < 0 )
                 return ret;
         }
@@ -809,7 +831,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
 
 /*
  * This function checks for xen domain nodes under the /chosen node for possible
- * domU guests to be loaded.
+ * dom0 and domU guests to be loaded.
  * Returns the number of modules loaded or a negative number for error.
  */
 static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
@@ -836,6 +858,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
                 return ERROR_DT_MODULE_DOMU;
         }
+        else if ( is_boot_module(node) )
+        {
+            if ( handle_module_node(dir_handle, node, addr_len, size_len,
+                                    false) < 0 )
+                return ERROR_DT_MODULE_DOM0;
+        }
     }
 
     /* Free dom0less file names if any */
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index c8c57fbb54..b221494a06 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
-
-            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
-                            (void **)&shim_lock)) &&
-                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
-                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
         }
 
         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
@@ -1372,6 +1367,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     if (dt_module_found < 0)
         /* efi_arch_check_dt_boot throws some error */
         blexit(L"Error processing boot modules on DT.");
+
+    /* If Dom0 is specified, verify it */
+    if ( kernel.ptr &&
+         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
+                                           (void **)&shim_lock)) &&
+        (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
+        PrintErrMesg(L"Dom0 kernel image could not be verified", status);
     /*
      * Check if a proper configuration is provided to start Xen:
      *  - Dom0 specified (minimum required)
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:37:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:37:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198162.351517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG6N-0001eR-U6; Tue, 28 Sep 2021 16:37:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198162.351517; Tue, 28 Sep 2021 16:37:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVG6N-0001eK-Qd; Tue, 28 Sep 2021 16:37:07 +0000
Received: by outflank-mailman (input) for mailman id 198162;
 Tue, 28 Sep 2021 16:37:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mVG2r-0007R3-PW
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 16:33:29 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.55]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7aef723d-36b8-417f-b96f-ed62e803d2df;
 Tue, 28 Sep 2021 16:33:19 +0000 (UTC)
Received: from DB8P191CA0012.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:130::22)
 by AM0PR08MB5073.eurprd08.prod.outlook.com (2603:10a6:208:15c::27) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep
 2021 16:33:17 +0000
Received: from DB5EUR03FT052.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:130:cafe::23) by DB8P191CA0012.outlook.office365.com
 (2603:10a6:10:130::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Tue, 28 Sep 2021 16:33:17 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT052.mail.protection.outlook.com (10.152.21.82) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 16:33:17 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Tue, 28 Sep 2021 16:33:16 +0000
Received: from c1534b78cb71.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1DC65BD7-9CC4-4D7E-93B1-37FD3EB7D8FE.1; 
 Tue, 28 Sep 2021 16:33:05 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c1534b78cb71.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 28 Sep 2021 16:33:05 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6342.eurprd08.prod.outlook.com (2603:10a6:20b:31a::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Tue, 28 Sep
 2021 16:32:53 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.022; Tue, 28 Sep 2021
 16:32:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7aef723d-36b8-417f-b96f-ed62e803d2df
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=w/JEL8gsVihe+EMjYMhR2HrrRVt3HBZQnGRkx6R/jyw=;
 b=IAoR/xBmcCnfE16iQHmPcqfaPRARgdBL6nLJTaGKfYUXmkNLT7y09O8npe2fJ2e2fLC8tWgr6pLSQnXyr6RAmkJyEsLx3WNY/6mRfScleW7xgu8adjx7X8weAw4DqSW5FOUVM8ts6gLbeiACTCaT0C9iOmQphWciYACyavfL+wY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d90e10f3e6d110af
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KbQeAfIrhQBfq1gRMUAQdEkfn1AGJrOWh+sW2bF/1NC/qT3VheoviUMz+18Xi/FuRf+m2HqKJMTLITd8U7zDWMsqTkFeHSr9cMESxzsX79iSQXm2eLAwVBN5AD9RdWVM8l0ZdxSi4L3/2vKbvJQ1PngVbi3r7E+JJsI0OkDjENPx9qcpz2QPsr8H+nnv1TyPCEq1RjBLtWUWma2Hj3bg1RZd7KXYm7TPQI27jQEF+NGVMFI3jFP4QVlf14KjWIh0c+1QlzmttsdWsTw8IqXWOHThhw7EwD/9yDbThNTh3dGoNnRuHv7dxrxhp+khtNpyWJVav0ZNo9FDSrlrcreUAw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=w/JEL8gsVihe+EMjYMhR2HrrRVt3HBZQnGRkx6R/jyw=;
 b=HLzOp3FMsX+FCoqblQKb3D+eKU6CZmdPKrtYIiSHDj2PKErd08QTTR8isEyePLyghouBCHKN7r/PtL2Htcuht0au6x5d5NAy9R4QjYz6NY8aVgWTppjxADj164YRNSFEKDS6+uthGtz4BdeiEEtNKyFDs42MgR0ZxOsZT+2pODl/EvUeP3JwB0gL42Zz3nASAGaQ6N/VAV/b4mqyGxaOUdAdzY6iTji4/1WOWzwedYw4hzCqrKYbYBznMlzJ/feG5iWk/+WFb25Bw5RJn72yhB09YHKjtyfJSReNO0R2PkVncadIzLIeF7Jwe27Kw00sRV5zRE577m6hMkzVzx85PQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=w/JEL8gsVihe+EMjYMhR2HrrRVt3HBZQnGRkx6R/jyw=;
 b=IAoR/xBmcCnfE16iQHmPcqfaPRARgdBL6nLJTaGKfYUXmkNLT7y09O8npe2fJ2e2fLC8tWgr6pLSQnXyr6RAmkJyEsLx3WNY/6mRfScleW7xgu8adjx7X8weAw4DqSW5FOUVM8ts6gLbeiACTCaT0C9iOmQphWciYACyavfL+wY=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Paul Durrant <paul@xen.org>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Topic: [PATCH v2 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
Thread-Index: AQHXr6at03qE3GlVrUSsAw4u98X5EKuy0MOAgAbc5gA=
Date: Tue, 28 Sep 2021 16:32:53 +0000
Message-ID: <AB8CE929-1B53-4595-819E-AB1E3762BA39@arm.com>
References: <cover.1632307952.git.rahul.singh@arm.com>
 <06e0cf146ca4d82526282c1960177fad97346ad5.1632307952.git.rahul.singh@arm.com>
 <fe23eeb0-c31f-0525-dcda-f7cf548ea0cf@suse.com>
In-Reply-To: <fe23eeb0-c31f-0525-dcda-f7cf548ea0cf@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: b5123bc6-d3bf-474d-24e9-08d9829dad1c
x-ms-traffictypediagnostic: AS8PR08MB6342:|AM0PR08MB5073:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB5073609668603400712B8138FCA89@AM0PR08MB5073.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:4714;OLM:4714;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 wDQDwJr1Y/Y9fK6OwOpOEd1SPrK2Drmu+BuCVJl8RAOh6ikohGETcDFq0aBD2hpwTSR1DarO8GGAEZg61Mg8XFus93rKQZjzRUo8uNOHFE7xrdD4UDsl7wZRk+sdFTQLX0Tjk8N31dDLxUixfBJxsPGFroU8FqqiBo6VxfGfIZhFiIYgqOWpf6Fn6H8NAobgN1lyCTO6vxErPAQVjZrNk6BKqAbBXfWut6lemraOzJA+iwqji5p4C6yIk0QYPOG4MFLCGZVofun0KsUdGRKtNfX8CMrO2k2KQDbacxLehkACeEfj5t5kpccpLr/oKpppRxZcDhTITPRy2Rj3OgLFVBSrQjVaU1JTEv8uSiHBgyrSKlwdyFh4dJDkHQyJeBUa4aRhY0JhMIXAEg3HO1wkFcwX3SFumAEZZXtDGvHDig9RvZ4ZRdD4ufnZtugxV7WhZ5WpVNF0cKw+DIZqbquG/N6q4l9439Jvbl5bXZnysXnQZ3DfhSFalGOEs2HmTHJszXvJewigQPChNd5pXVriDOnBlBIN/Q5tPEaJgEIxCBBZtjfXMhicJO8/Zk4ErK0HGx6nbN4i03wT6xN5UpuszylaRWQYw1nGoRBa3eTXRc8iEj4JTdYvNBmusHGK6+bOCykOmYWhko3LoEmFe5GDUFziGLO6jMori8zCjfq4IxrVG7jAkjtY1ni/IYYqPhpuFsTLgP3FUwpzIeFBNb2/zVnaP7jeY1LnSzRLl1Yk0WIjXZZQzNGxx1uCTLOTkmQF
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(33656002)(38100700002)(6916009)(122000001)(6512007)(71200400001)(38070700005)(54906003)(36756003)(186003)(8936002)(86362001)(8676002)(7416002)(2616005)(4326008)(53546011)(76116006)(91956017)(6506007)(6486002)(5660300002)(2906002)(508600001)(64756008)(66446008)(316002)(26005)(66556008)(66476007)(66946007)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <6113B935DFA4A644BDE412999C02AC24@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6342
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5c83bf4c-eb2e-4dee-15d1-08d9829d9eb6
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0RX+D0upyYimdNKFb+R2Jql6ol686uVaVKTok/+kvbL2cn60qDKvb8+EK//hK+CS1aAlxypoYms6n8SL7Iv9ULRn9xN1l8/4VspiBZjaQB5DYri22REZlZzpnqljUTFsp3T0bqEX+eWl+OSJ1Pf6JXlQdmC6m0MtoEbL2HRV1sH7rXE+r9N9aUUIujoNeL1jVL7clFSj0AA6i3H+CMxX3BpODxmfXIx3QnxIHiRHDfi0oeLF8V7KqhbzZ0sqgBA+84Dfdvrq2a8gB2lbHo0ZUKUQcogUUG2x/0wyNsTHsSdQgenxhvT/bbi/1Ob8hDfvnY1VGz7ZcML8BebmMwXPg30Bd3zIuFcGyCX2naElKY7Uh4Z4wxFY11yn9JTn7GG4rN1420XG1yUM4fnhV/wHfuSz+rJFq3W5qzT1Z5f8gGZU4wqDt4zloSvPEzR7D1wns0cIgM+CnW7Xa2WxbpLUUWUqLK5yEJpdeqeKmJx61N3dC6YigbP2qu/sfG4fEbphtLcN+/nGF13B6fihUx77ah3WYpeZODP1s7ediQs2gixPFFi8VCFI9UbxpJvFUunXZmR7I3vwRCYl4KcoRbEvIKRj2G35NKggfDrXY5V5FQWQ91r6Pypt+Rbp/i5Pv5vKwe+NdXTBOI56LZ/vZtGkjf1mwDvPZKC1sbx8JYJol/Fw2TJp7HE1hxom9Qwvi9oiySF48E5ykLwmgrowuVn3pQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2616005)(6486002)(316002)(6512007)(2906002)(508600001)(356005)(36860700001)(186003)(6862004)(47076005)(26005)(86362001)(8676002)(5660300002)(54906003)(53546011)(36756003)(6506007)(336012)(81166007)(70586007)(4326008)(33656002)(82310400003)(70206006)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 16:33:17.5345
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b5123bc6-d3bf-474d-24e9-08d9829dad1c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT052.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5073

Hi Jan

> On 24 Sep 2021, at 8:44 am, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 22.09.2021 13:35, Rahul Singh wrote:
>> @@ -623,7 +624,7 @@ int arch_sanitise_domain_config(struct xen_domctl_cr=
eatedomain *config)
>>     unsigned int max_vcpus;
>>=20
>>     /* HVM and HAP must be set. IOMMU may or may not be */
>> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=3D
>> +    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci)=
 !=3D
>>          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>>     {
>>         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
>=20
> While you accept the new flag here and ...
>=20
>> --- a/xen/common/domain.c
>> +++ b/xen/common/domain.c
>> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_=
createdomain *config)
>>          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>>            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>>            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
>> -           XEN_DOMCTL_CDF_nested_virt) )
>> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
>>     {
>>         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>>         return -EINVAL;
>=20
> ... here, you need to somehow reject it on x86, until DomU support
> there gets added (unless I have misunderstood things and you're
> aiming at enabing that support for x86 here at the same time). I
> cannot spot existing code which would take care of such a newly
> added flag.

Ok. I will reject the flag in x86 arch_sanitise_domain_config().
>=20
>=20
>> --- a/xen/include/asm-x86/pci.h
>> +++ b/xen/include/asm-x86/pci.h
>> @@ -6,8 +6,6 @@
>> #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
>> #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
>>=20
>> -#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
>=20
> While there was a reason for the padding blank after the first
> opening parentheses here, ...
>=20
>> --- a/xen/include/xen/pci.h
>> +++ b/xen/include/xen/pci.h
>> @@ -41,6 +41,8 @@
>> #define PCI_SBDF3(s,b,df) \
>>     ((pci_sbdf_t){ .sbdf =3D (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
>>=20
>> +#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
>=20
> ... that blank ends up bogus here.
Ack . I will remove the extra blank in next version.

Regards,
Rahul
>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 16:55:34 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 16:55:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198172.351528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVGNr-00047T-FL; Tue, 28 Sep 2021 16:55:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198172.351528; Tue, 28 Sep 2021 16:55:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVGNr-00047M-CD; Tue, 28 Sep 2021 16:55:11 +0000
Received: by outflank-mailman (input) for mailman id 198172;
 Tue, 28 Sep 2021 16:55:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVGNq-00047B-1C; Tue, 28 Sep 2021 16:55:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVGNp-0004HG-Pi; Tue, 28 Sep 2021 16:55:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVGNp-00028W-GR; Tue, 28 Sep 2021 16:55:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVGNp-0004IC-Fv; Tue, 28 Sep 2021 16:55:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=J11LYJyhbISe6oZgZyGP8C6+oibIVcVlVnPM4KPZ3R4=; b=g0KlU55KzZCF06tCCGpoSpmlF/
	y4BREfQ+3fmNrFoiQch1kj1YdbQHMCBFR7J8SwNwT1asdxS6IGvOz2l2GRYd7Z4YkCaxa62FoV88O
	3RcOttJ8bKjFej5NhaBkVGLBaS/NUew766XY/PGR4/MfqTkDH5P1egPDcM35AKbTvVos=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165233-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165233: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=890ceb9453171c85e881103e65dbb5cdcf81659e
X-Osstest-Versions-That:
    xen=1c3ed9c908732d19660fbe83580674d585464d4c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 16:55:09 +0000

flight 165233 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165233/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  890ceb9453171c85e881103e65dbb5cdcf81659e
baseline version:
 xen                  1c3ed9c908732d19660fbe83580674d585464d4c

Last test of basis   165230  2021-09-28 05:00:28 Z    0 days
Testing same since   165233  2021-09-28 12:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Ian Jackson <iwj@xenproject.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   1c3ed9c908..890ceb9453  890ceb9453171c85e881103e65dbb5cdcf81659e -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 17:19:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 17:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198184.351542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVGlK-0006zn-AU; Tue, 28 Sep 2021 17:19:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198184.351542; Tue, 28 Sep 2021 17:19:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVGlK-0006zg-7U; Tue, 28 Sep 2021 17:19:26 +0000
Received: by outflank-mailman (input) for mailman id 198184;
 Tue, 28 Sep 2021 17:19:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>) id 1mVGlI-0006za-2e
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 17:19:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVGlH-0004i2-Ml; Tue, 28 Sep 2021 17:19:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVGlH-0003a8-Dm; Tue, 28 Sep 2021 17:19:23 +0000
Received: from iwj by osstest.test-lab.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVGlH-0006re-DA; Tue, 28 Sep 2021 17:19:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
To: xen-devel@lists.xenproject.org,jbeulich@suse.com,julien.grall.oss@gmail.com,iwj@xenproject.org,sstabellini@kernel.org,dpsmith@apertussolutions.com
Subject: [adhoc test] 165241: all pass
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Message-Id: <E1mVGlH-0006re-DA@osstest.test-lab.xenproject.org>
From: iwj@xenbits.xen.org
Date: Tue, 28 Sep 2021 17:19:23 +0000

[adhoc adhoc] <testing.git master /dev/pts/25>
harness 3a3089c9: mfi-common: Drop Linux dom0 i386 tests for newer Lin...
165241: all pass

flight 165241 xen-unstable adhoc [adhoc]
http://logs.test-lab.xenproject.org/osstest/logs/165241/

Perfect :-)
All tests in this flight passed as required

jobs:
 build-arm64-pvops                                            pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:19:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:19:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198195.351553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHgw-0005Ku-1E; Tue, 28 Sep 2021 18:18:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198195.351553; Tue, 28 Sep 2021 18:18:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHgv-0005Kn-Tx; Tue, 28 Sep 2021 18:18:57 +0000
Received: by outflank-mailman (input) for mailman id 198195;
 Tue, 28 Sep 2021 18:18:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHgu-0005Kh-22
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:18:56 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id beb59975-95b3-40f6-b880-fd6109e272c1;
 Tue, 28 Sep 2021 18:18:53 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C32E6D;
 Tue, 28 Sep 2021 11:18:53 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 36DED3F793;
 Tue, 28 Sep 2021 11:18:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: beb59975-95b3-40f6-b880-fd6109e272c1
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Paul Durrant <paul@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Juergen Gross <jgross@suse.com>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v3 00/17] PCI devices passthrough on Arm
Date: Tue, 28 Sep 2021 19:18:09 +0100
Message-Id: <cover.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello All,

The purpose of this patch series is to add PCI passthrough support to Xen on
Arm. PCI passthrough support on ARM is the collaboration work between EPAM and
ARM. ARM submitted the partial RFC [1][2] last year to get early feedback. We
tried to fix all the comments and added more features to this patch series.

Working POC with all the features can be found at [3]. Working POC is tested
on x86 so that there will be no regression on x86. Design presentation can be
found at [4]

PCI passthrough support is divided into different patches. This patch series
includes following features: 

Preparatory work to implement the PCI passthrough support for the ARM:
- Refactor MSI code.
- Fixed compilation error when HAS_PCI enabled for ARM.

Discovering PCI Host Bridge in XEN:
- PCI init to initialize the PCI driver.
- PCI host bridge discovery in XEN and map the PCI ECAM configuration space to
  the XEN memory.
- PCI access functions.

Discovering PCI devices:
- To support the PCI passthrough, XEN should be aware of the PCI
  devices.
- Hardware domain is in charge of doing the PCI enumeration and will discover
  the PCI devices and then communicate to the XEN via a hypercall to add the
  PCI devices in XEN.

Enable the existing x86 virtual PCI support for ARM:
- Add VPCI trap handler for each of the PCI device added for config space
  access.
- Register the trap handler in XEN for each of the host bridge PCI ECAM config
  space access.

Emulated PCI device tree node in libxl:
- Create a virtual PCI device tree node in libxl to enable the guest OS to
  discover the virtual PCI during guest boot.

This patch series does not inlcude the following features. Following features
will be send for review in the next version of the patch series once initial
patch series merged.

- VPCI support for DOMU guests (Non-identity mappings guest view of the BARs)
- Virtual bus topology implementation
- IOMMU related changes (generic, SMMUv2, SMMUv3)
- MSI support for DOMU guests.
- Virual ITS support for DOMU guests

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
[3] https://gitlab.com/rahsingh/xen-integration/-/commits/pci-passthrough-upstream-all
[4] https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf

Oleksandr Andrushchenko (1):
  xen/arm: Add support for Xilinx ZynqMP PCI host controller

Rahul Singh (16):
  xen/pci: Refactor MSI code that implements MSI functionality within
    XEN
  xen/arm: pci: Add stubs to allow selecting HAS_PCI
  xen/arm: solve compilation error on ARM with ACPI && HAS_PCI
  xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
  xen/arm: Add PHYSDEVOP_pci_device_* support for ARM
  xen/device-tree: Add dt_property_read_variable_u32_array helper
  xen/device-tree: Add dt_property_read_u32_array helper
  xen/device-tree: Add dt_get_pci_domain_nr helper
  xen/arm: Add support for PCI init to initialize the PCI driver.
  xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
  xen/arm: PCI host bridge discovery within XEN on ARM
  xen/arm: Implement pci access functions
  xen/arm: Enable the existing x86 virtual PCI support for ARM.
  xen/arm: Transitional change to build HAS_VPCI on ARM.
  arm/libxl: Emulated PCI device tree node in libxl
  xen/arm: Add linux,pci-domain property for hwdom if not available.

 docs/misc/xen-command-line.pandoc   |   7 +
 tools/include/libxl.h               |   6 +
 tools/libs/ctrl/xc_domain.c         |   9 +
 tools/libs/light/libxl_arm.c        | 105 ++++++++++
 tools/libs/light/libxl_create.c     |   3 +
 tools/libs/light/libxl_types.idl    |   1 +
 tools/xl/xl_parse.c                 |   2 +
 xen/arch/arm/Makefile               |   2 +
 xen/arch/arm/domain.c               |   8 +-
 xen/arch/arm/domain_build.c         |  19 ++
 xen/arch/arm/pci/Makefile           |   6 +
 xen/arch/arm/pci/ecam.c             |  61 ++++++
 xen/arch/arm/pci/pci-access.c       | 140 ++++++++++++++
 xen/arch/arm/pci/pci-host-common.c  | 287 ++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-generic.c |  46 +++++
 xen/arch/arm/pci/pci-host-zynqmp.c  |  63 ++++++
 xen/arch/arm/pci/pci.c              |  98 ++++++++++
 xen/arch/arm/physdev.c              |   5 +-
 xen/arch/arm/vpci.c                 | 102 ++++++++++
 xen/arch/arm/vpci.h                 |  36 ++++
 xen/arch/x86/Kconfig                |   1 +
 xen/arch/x86/domain.c               |   6 +
 xen/arch/x86/physdev.c              |  50 +----
 xen/arch/x86/x86_64/physdev.c       |   4 +-
 xen/common/Makefile                 |   1 +
 xen/common/device_tree.c            |  73 +++++++
 xen/common/domain.c                 |   2 +-
 xen/common/physdev.c                |  87 +++++++++
 xen/drivers/passthrough/Makefile    |   1 +
 xen/drivers/passthrough/arm/iommu.c |   9 +
 xen/drivers/passthrough/msi.c       |  83 ++++++++
 xen/drivers/passthrough/pci.c       |  69 +++----
 xen/drivers/pci/Kconfig             |   4 +
 xen/drivers/vpci/Makefile           |   3 +-
 xen/drivers/vpci/header.c           |   2 +
 xen/include/asm-arm/device.h        |   1 +
 xen/include/asm-arm/domain.h        |   8 +-
 xen/include/asm-arm/hypercall.h     |   2 -
 xen/include/asm-arm/numa.h          |   5 +
 xen/include/asm-arm/pci.h           | 116 ++++++++++-
 xen/include/asm-x86/hypercall.h     |   9 +-
 xen/include/asm-x86/pci.h           |  10 +-
 xen/include/public/arch-arm.h       |  17 ++
 xen/include/public/domctl.h         |   4 +-
 xen/include/xen/device_tree.h       |  73 +++++++
 xen/include/xen/hypercall.h         |   8 +
 xen/include/xen/msi.h               |  43 +++++
 xen/include/xen/pci.h               |   5 +-
 xen/xsm/flask/hooks.c               |   8 +-
 49 files changed, 1593 insertions(+), 117 deletions(-)
 create mode 100644 xen/arch/arm/pci/Makefile
 create mode 100644 xen/arch/arm/pci/ecam.c
 create mode 100644 xen/arch/arm/pci/pci-access.c
 create mode 100644 xen/arch/arm/pci/pci-host-common.c
 create mode 100644 xen/arch/arm/pci/pci-host-generic.c
 create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
 create mode 100644 xen/arch/arm/pci/pci.c
 create mode 100644 xen/arch/arm/vpci.c
 create mode 100644 xen/arch/arm/vpci.h
 create mode 100644 xen/common/physdev.c
 create mode 100644 xen/drivers/passthrough/msi.c
 create mode 100644 xen/include/xen/msi.h

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:19:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:19:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198197.351563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHhR-0005qX-9r; Tue, 28 Sep 2021 18:19:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198197.351563; Tue, 28 Sep 2021 18:19:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHhR-0005qQ-71; Tue, 28 Sep 2021 18:19:29 +0000
Received: by outflank-mailman (input) for mailman id 198197;
 Tue, 28 Sep 2021 18:19:28 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHhQ-0005qD-DE
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:19:28 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 9cc18f95-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:19:26 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B05B36D;
 Tue, 28 Sep 2021 11:19:25 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B4F903F793;
 Tue, 28 Sep 2021 11:19:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cc18f95-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Paul Durrant <paul@xen.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v3 01/17] xen/pci: Refactor MSI code that implements MSI functionality within XEN
Date: Tue, 28 Sep 2021 19:18:10 +0100
Message-Id: <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

On Arm, the initial plan is to only support GICv3 ITS which doesn't
require us to manage the MSIs because the HW will protect against
spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
for ARM.

No functional change intended.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
Change in v3: none 
Change in v2: Fixed minor comments
---
 xen/arch/x86/Kconfig             |  1 +
 xen/drivers/passthrough/Makefile |  1 +
 xen/drivers/passthrough/msi.c    | 83 ++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/pci.c    | 54 +++++----------------
 xen/drivers/pci/Kconfig          |  4 ++
 xen/include/xen/msi.h            | 43 +++++++++++++++++
 xen/xsm/flask/hooks.c            |  8 +--
 7 files changed, 149 insertions(+), 45 deletions(-)
 create mode 100644 xen/drivers/passthrough/msi.c
 create mode 100644 xen/include/xen/msi.h

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 1f83518ee0..b4abfca46f 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -20,6 +20,7 @@ config X86
 	select HAS_NS16550
 	select HAS_PASSTHROUGH
 	select HAS_PCI
+	select HAS_PCI_MSI
 	select HAS_PDX
 	select HAS_SCHED_GRANULARITY
 	select HAS_UBSAN
diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile
index 445690e3e5..a5efa22714 100644
--- a/xen/drivers/passthrough/Makefile
+++ b/xen/drivers/passthrough/Makefile
@@ -7,3 +7,4 @@ obj-y += iommu.o
 obj-$(CONFIG_HAS_PCI) += pci.o
 obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
 obj-$(CONFIG_HAS_PCI) += ats.o
+obj-$(CONFIG_HAS_PCI_MSI) += msi.o
diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.c
new file mode 100644
index 0000000000..ce1a450f6f
--- /dev/null
+++ b/xen/drivers/passthrough/msi.c
@@ -0,0 +1,83 @@
+#include <xen/init.h>
+#include <xen/pci.h>
+#include <xen/msi.h>
+#include <asm/hvm/io.h>
+
+int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
+{
+    int rc;
+
+    if ( pdev->msix )
+    {
+        rc = pci_reset_msix_state(pdev);
+        if ( rc )
+            return rc;
+        msixtbl_init(d);
+    }
+
+    return 0;
+}
+
+int pdev_msi_init(struct pci_dev *pdev)
+{
+    unsigned int pos;
+
+    INIT_LIST_HEAD(&pdev->msi_list);
+
+    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
+                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSI);
+    if ( pos )
+    {
+        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
+
+        pdev->msi_maxvec = multi_msi_capable(ctrl);
+    }
+
+    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
+                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSIX);
+    if ( pos )
+    {
+        struct arch_msix *msix = xzalloc(struct arch_msix);
+        uint16_t ctrl;
+
+        if ( !msix )
+            return -ENOMEM;
+
+        spin_lock_init(&msix->table_lock);
+
+        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
+        msix->nr_entries = msix_table_size(ctrl);
+
+        pdev->msix = msix;
+    }
+
+    return 0;
+}
+
+void pdev_msi_deinit(struct pci_dev *pdev)
+{
+    XFREE(pdev->msix);
+}
+
+void pdev_dump_msi(const struct pci_dev *pdev)
+{
+    const struct msi_desc *msi;
+
+    if ( list_empty(&pdev->msi_list) )
+        return;
+
+    printk(" - MSIs < ");
+    list_for_each_entry ( msi, &pdev->msi_list, list )
+        printk("%d ", msi->irq);
+    printk(">");
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index fc4fa2e5c3..8996403161 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -32,8 +32,8 @@
 #include <xen/softirq.h>
 #include <xen/tasklet.h>
 #include <xen/vpci.h>
+#include <xen/msi.h>
 #include <xsm/xsm.h>
-#include <asm/msi.h>
 #include "ats.h"
 
 struct pci_seg {
@@ -314,6 +314,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
 {
     struct pci_dev *pdev;
     unsigned int pos;
+    int rc;
 
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
         if ( pdev->bus == bus && pdev->devfn == devfn )
@@ -327,35 +328,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     *((u8*) &pdev->bus) = bus;
     *((u8*) &pdev->devfn) = devfn;
     pdev->domain = NULL;
-    INIT_LIST_HEAD(&pdev->msi_list);
-
-    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                              PCI_CAP_ID_MSI);
-    if ( pos )
-    {
-        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
-
-        pdev->msi_maxvec = multi_msi_capable(ctrl);
-    }
 
-    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
-                              PCI_CAP_ID_MSIX);
-    if ( pos )
+    rc = pdev_msi_init(pdev);
+    if ( rc )
     {
-        struct arch_msix *msix = xzalloc(struct arch_msix);
-        uint16_t ctrl;
-
-        if ( !msix )
-        {
-            xfree(pdev);
-            return NULL;
-        }
-        spin_lock_init(&msix->table_lock);
-
-        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
-        msix->nr_entries = msix_table_size(ctrl);
-
-        pdev->msix = msix;
+        xfree(pdev);
+        return NULL;
     }
 
     list_add(&pdev->alldevs_list, &pseg->alldevs_list);
@@ -449,7 +427,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev)
     }
 
     list_del(&pdev->alldevs_list);
-    xfree(pdev->msix);
+    pdev_msi_deinit(pdev);
     xfree(pdev);
 }
 
@@ -1271,18 +1249,16 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev)
 static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
 {
     struct pci_dev *pdev;
-    struct msi_desc *msi;
 
     printk("==== segment %04x ====\n", pseg->nr);
 
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
     {
-        printk("%pp - %pd - node %-3d - MSIs < ",
+        printk("%pp - %pd - node %-3d",
                &pdev->sbdf, pdev->domain,
                (pdev->node != NUMA_NO_NODE) ? pdev->node : -1);
-        list_for_each_entry ( msi, &pdev->msi_list, list )
-               printk("%d ", msi->irq);
-        printk(">\n");
+        pdev_dump_msi(pdev);
+        printk("\n");
     }
 
     return 0;
@@ -1422,13 +1398,9 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
     ASSERT(pdev && (pdev->domain == hardware_domain ||
                     pdev->domain == dom_io));
 
-    if ( pdev->msix )
-    {
-        rc = pci_reset_msix_state(pdev);
-        if ( rc )
-            goto done;
-        msixtbl_init(d);
-    }
+    rc = pdev_msix_assign(d, pdev);
+    if ( rc )
+        goto done;
 
     pdev->fault.count = 0;
 
diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
index 7da03fa13b..c6a7bc8007 100644
--- a/xen/drivers/pci/Kconfig
+++ b/xen/drivers/pci/Kconfig
@@ -1,3 +1,7 @@
 
 config HAS_PCI
 	bool
+
+config HAS_PCI_MSI
+	bool
+	depends on HAS_PCI
diff --git a/xen/include/xen/msi.h b/xen/include/xen/msi.h
new file mode 100644
index 0000000000..c903d0050c
--- /dev/null
+++ b/xen/include/xen/msi.h
@@ -0,0 +1,43 @@
+#ifndef __XEN_MSI_H_
+#define __XEN_MSI_H_
+
+#include <xen/pci.h>
+
+#ifdef CONFIG_HAS_PCI_MSI
+
+#include <asm/msi.h>
+
+int pdev_msix_assign(struct domain *d, struct pci_dev *pdev);
+int pdev_msi_init(struct pci_dev *pdev);
+void pdev_msi_deinit(struct pci_dev *pdev);
+void pdev_dump_msi(const struct pci_dev *pdev);
+
+#else /* !CONFIG_HAS_PCI_MSI */
+
+static inline int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
+{
+    return 0;
+}
+
+static inline int pdev_msi_init(struct pci_dev *pdev)
+{
+    return 0;
+}
+
+static inline void pdev_msi_deinit(struct pci_dev *pdev) {}
+static inline void pci_cleanup_msi(struct pci_dev *pdev) {}
+static inline void pdev_dump_msi(const struct pci_dev *pdev) {}
+
+#endif /* CONFIG_HAS_PCI_MSI */
+
+#endif /* __XEN_MSI_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 25e87180b4..ea9a12bd71 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -21,7 +21,7 @@
 #include <xen/guest_access.h>
 #include <xen/xenoprof.h>
 #include <xen/iommu.h>
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
 #include <asm/msi.h>
 #endif
 #include <public/xen.h>
@@ -114,7 +114,7 @@ static int get_irq_sid(int irq, u32 *sid, struct avc_audit_data *ad)
         }
         return security_irq_sid(irq, sid);
     }
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     {
         struct irq_desc *desc = irq_to_desc(irq);
         if ( desc->msi_desc && desc->msi_desc->dev ) {
@@ -874,7 +874,7 @@ static int flask_map_domain_pirq (struct domain *d)
 static int flask_map_domain_msi (struct domain *d, int irq, const void *data,
                                  u32 *sid, struct avc_audit_data *ad)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     const struct msi_info *msi = data;
     u32 machine_bdf = (msi->seg << 16) | (msi->bus << 8) | msi->devfn;
 
@@ -940,7 +940,7 @@ static int flask_unmap_domain_pirq (struct domain *d)
 static int flask_unmap_domain_msi (struct domain *d, int irq, const void *data,
                                    u32 *sid, struct avc_audit_data *ad)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_PCI_MSI
     const struct pci_dev *pdev = data;
     u32 machine_bdf = (pdev->seg << 16) | (pdev->bus << 8) | pdev->devfn;
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:19:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:19:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198203.351575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHhj-0006N6-JV; Tue, 28 Sep 2021 18:19:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198203.351575; Tue, 28 Sep 2021 18:19:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHhj-0006Mz-G1; Tue, 28 Sep 2021 18:19:47 +0000
Received: by outflank-mailman (input) for mailman id 198203;
 Tue, 28 Sep 2021 18:19:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHhh-0006Ma-SN
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:19:45 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id c278eeda-7b05-41fe-ba82-559089251627;
 Tue, 28 Sep 2021 18:19:44 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 11D9E6D;
 Tue, 28 Sep 2021 11:19:44 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 118903F793;
 Tue, 28 Sep 2021 11:19:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c278eeda-7b05-41fe-ba82-559089251627
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 02/17] xen/arm: pci: Add stubs to allow selecting HAS_PCI
Date: Tue, 28 Sep 2021 19:18:11 +0100
Message-Id: <c752df37298c75f2f1872565d2272fc59063547f.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

In a follow-up we will enable PCI support in Xen on Arm (i.e select
HAS_PCI).

The generic code expects the arch to implement a few functions:
arch_iommu_use_permitted()
arch_pci_clean_pirqs()

Note that this is not yet sufficient to enable HAS_PCI and will be
addressed in follow-ups.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Change in v3:
- Modify commit message.
- Added Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Change in v2:
- Remove pci_conf_read*(..) dummy implementation
- Add in code comment for arch_pci_clean_pirqs() and arch_iommu_use_permitted()
- Fixed minor comments.
---
 xen/arch/arm/Makefile               |  1 +
 xen/arch/arm/pci/Makefile           |  1 +
 xen/arch/arm/pci/pci.c              | 33 +++++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/iommu.c |  9 ++++++++
 xen/include/asm-arm/pci.h           | 31 ++++++++++++++++++++++++---
 5 files changed, 72 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/arm/pci/Makefile
 create mode 100644 xen/arch/arm/pci/pci.c

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 3d3b97b5b4..44d7cc81fa 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) += arm32/
 obj-$(CONFIG_ARM_64) += arm64/
 obj-$(CONFIG_ARM_64) += efi/
 obj-$(CONFIG_ACPI) += acpi/
+obj-$(CONFIG_HAS_PCI) += pci/
 ifneq ($(CONFIG_NO_PLAT),y)
 obj-y += platforms/
 endif
diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
new file mode 100644
index 0000000000..a98035df4c
--- /dev/null
+++ b/xen/arch/arm/pci/Makefile
@@ -0,0 +1 @@
+obj-y += pci.o
diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
new file mode 100644
index 0000000000..a7a7bc3213
--- /dev/null
+++ b/xen/arch/arm/pci/pci.c
@@ -0,0 +1,33 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+
+/*
+ * PIRQ event channels are not supported on Arm, so nothing to do.
+ */
+int arch_pci_clean_pirqs(struct domain *d)
+{
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c
index db3b07a571..ee653a9c48 100644
--- a/xen/drivers/passthrough/arm/iommu.c
+++ b/xen/drivers/passthrough/arm/iommu.c
@@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 {
 }
+
+/*
+ * Unlike x86, Arm doesn't support mem-sharing, mem-paging and log-dirty (yet).
+ * So there is no restriction to use the IOMMU.
+ */
+bool arch_iommu_use_permitted(const struct domain *d)
+{
+    return true;
+}
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index de13359f65..7dd9eb3dba 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -1,7 +1,32 @@
-#ifndef __X86_PCI_H__
-#define __X86_PCI_H__
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
 
+#ifndef __ARM_PCI_H__
+#define __ARM_PCI_H__
+
+#ifdef CONFIG_HAS_PCI
+
+#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
+
+/* Arch pci dev struct */
 struct arch_pci_dev {
+    struct device dev;
 };
 
-#endif /* __X86_PCI_H__ */
+#else   /*!CONFIG_HAS_PCI*/
+
+struct arch_pci_dev { };
+
+#endif  /*!CONFIG_HAS_PCI*/
+#endif /* __ARM_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:20:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:20:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198210.351586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHi2-0007Zu-UC; Tue, 28 Sep 2021 18:20:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198210.351586; Tue, 28 Sep 2021 18:20:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHi2-0007Zl-Ol; Tue, 28 Sep 2021 18:20:06 +0000
Received: by outflank-mailman (input) for mailman id 198210;
 Tue, 28 Sep 2021 18:20:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHi1-0006l6-HU
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:20:05 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ae47c80b-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:19:56 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 734E46D;
 Tue, 28 Sep 2021 11:19:56 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8E1703F793;
 Tue, 28 Sep 2021 11:19:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae47c80b-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with ACPI && HAS_PCI
Date: Tue, 28 Sep 2021 19:18:12 +0100
Message-Id: <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

prelink.o: In function `pcie_aer_get_firmware_first’:
drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'

Compilation error is observed when ACPI and HAS_PCI is enabled for ARM
architecture. APEI not supported on ARM yet move the code under
CONFIG_X86 flag to gate the code for ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
Change in v3: Added Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Change in v2: Add in code comment "APEI not supported on ARM yet"
---
 xen/drivers/passthrough/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 8996403161..d774a6154e 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1150,7 +1150,8 @@ void __hwdom_init setup_hwdom_pci_devices(
     pcidevs_unlock();
 }
 
-#ifdef CONFIG_ACPI
+/* APEI not supported on ARM yet. */
+#if defined(CONFIG_ACPI) && defined(CONFIG_X86)
 #include <acpi/acpi.h>
 #include <acpi/apei.h>
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:20:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:20:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198217.351597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHiI-0008Be-AW; Tue, 28 Sep 2021 18:20:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198217.351597; Tue, 28 Sep 2021 18:20:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHiI-0008BX-6j; Tue, 28 Sep 2021 18:20:22 +0000
Received: by outflank-mailman (input) for mailman id 198217;
 Tue, 28 Sep 2021 18:20:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHiG-0006l6-HX
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:20:20 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id b8248dc2-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:20:11 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9833E6D;
 Tue, 28 Sep 2021 11:20:11 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 89C913F793;
 Tue, 28 Sep 2021 11:20:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8248dc2-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3 04/17] xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
Date: Tue, 28 Sep 2021 19:18:13 +0100
Message-Id: <9d796d7c5c6d6c2d206809f99ed0be7ec9ebe056.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

ARM architecture does not implement I/O ports. Ignore this call on ARM
to avoid the overhead of making a hypercall just for Xen to return
-ENOSYS.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Change in v3: Added Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Change in v2:
- Instead of returning success in XEN, ignored the call in xl.
---
 tools/libs/ctrl/xc_domain.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index 23322b70b5..25c95f6596 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -1348,6 +1348,14 @@ int xc_domain_ioport_permission(xc_interface *xch,
                                 uint32_t nr_ports,
                                 uint32_t allow_access)
 {
+#if defined(__arm__) || defined(__aarch64__)
+    /*
+     * The ARM architecture does not implement I/O ports.
+     * Avoid the overhead of making a hypercall just for Xen to return -ENOSYS.
+     * It is safe to ignore this call on ARM so we just return 0.
+     */
+    return 0;
+#else
     DECLARE_DOMCTL;
 
     domctl.cmd = XEN_DOMCTL_ioport_permission;
@@ -1357,6 +1365,7 @@ int xc_domain_ioport_permission(xc_interface *xch,
     domctl.u.ioport_permission.allow_access = allow_access;
 
     return do_domctl(xch, &domctl);
+#endif
 }
 
 int xc_availheap(xc_interface *xch,
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:20:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:20:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198223.351608 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHiR-0000Dr-I2; Tue, 28 Sep 2021 18:20:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198223.351608; Tue, 28 Sep 2021 18:20:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHiR-0000Di-ET; Tue, 28 Sep 2021 18:20:31 +0000
Received: by outflank-mailman (input) for mailman id 198223;
 Tue, 28 Sep 2021 18:20:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHiQ-0006l6-Hr
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:20:30 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id c076fd34-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:20:25 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8CAAF6D;
 Tue, 28 Sep 2021 11:20:25 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E05A63F793;
 Tue, 28 Sep 2021 11:20:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c076fd34-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for ARM
Date: Tue, 28 Sep 2021 19:18:14 +0100
Message-Id: <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Hardware domain is in charge of doing the PCI enumeration and will
discover the PCI devices and then will communicate to XEN via hyper
call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.

Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.

As most of the code for PHYSDEVOP_pci_device_* is the same between x86
and ARM, move the code to a common file to avoid duplication.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3: Fixed minor comment.
Change in v2:
- Add support for PHYSDEVOP_pci_device_remove()
- Move code to common code
---
 xen/arch/arm/physdev.c          |  5 +-
 xen/arch/x86/physdev.c          | 50 +-------------------
 xen/arch/x86/x86_64/physdev.c   |  4 +-
 xen/common/Makefile             |  1 +
 xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
 xen/include/asm-arm/hypercall.h |  2 -
 xen/include/asm-arm/numa.h      |  5 ++
 xen/include/asm-x86/hypercall.h |  9 ++--
 xen/include/xen/hypercall.h     |  8 ++++
 9 files changed, 106 insertions(+), 59 deletions(-)
 create mode 100644 xen/common/physdev.c

diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
index e91355fe22..4e00b03aab 100644
--- a/xen/arch/arm/physdev.c
+++ b/xen/arch/arm/physdev.c
@@ -8,10 +8,9 @@
 #include <xen/lib.h>
 #include <xen/errno.h>
 #include <xen/sched.h>
-#include <asm/hypercall.h>
+#include <xen/hypercall.h>
 
-
-int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd);
     return -ENOSYS;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 23465bcd00..c00cc99404 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
 }
 #endif /* COMPAT */
 
-ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     int irq;
     ret_t ret;
@@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         break;
     }
 
-    case PHYSDEVOP_pci_device_add: {
-        struct physdev_pci_device_add add;
-        struct pci_dev_info pdev_info;
-        nodeid_t node;
-
-        ret = -EFAULT;
-        if ( copy_from_guest(&add, arg, 1) != 0 )
-            break;
-
-        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
-        if ( add.flags & XEN_PCI_DEV_VIRTFN )
-        {
-            pdev_info.is_virtfn = 1;
-            pdev_info.physfn.bus = add.physfn.bus;
-            pdev_info.physfn.devfn = add.physfn.devfn;
-        }
-        else
-            pdev_info.is_virtfn = 0;
-
-        if ( add.flags & XEN_PCI_DEV_PXM )
-        {
-            uint32_t pxm;
-            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
-                                sizeof(add.optarr[0]);
-
-            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
-                break;
-
-            node = pxm_to_node(pxm);
-        }
-        else
-            node = NUMA_NO_NODE;
-
-        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
-        break;
-    }
-
-    case PHYSDEVOP_pci_device_remove: {
-        struct physdev_pci_device dev;
-
-        ret = -EFAULT;
-        if ( copy_from_guest(&dev, arg, 1) != 0 )
-            break;
-
-        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
-        break;
-    }
-
     case PHYSDEVOP_prepare_msix:
     case PHYSDEVOP_release_msix: {
         struct physdev_pci_device dev;
diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c
index 0a50cbd4d8..5f72652ff7 100644
--- a/xen/arch/x86/x86_64/physdev.c
+++ b/xen/arch/x86/x86_64/physdev.c
@@ -9,9 +9,10 @@ EMIT_FILE;
 #include <compat/xen.h>
 #include <compat/event_channel.h>
 #include <compat/physdev.h>
-#include <asm/hypercall.h>
+#include <xen/hypercall.h>
 
 #define do_physdev_op compat_physdev_op
+#define arch_physdev_op arch_compat_physdev_op
 
 #define physdev_apic               compat_physdev_apic
 #define physdev_apic_t             physdev_apic_compat_t
@@ -82,6 +83,7 @@ CHECK_physdev_pci_device
 typedef int ret_t;
 
 #include "../physdev.c"
+#include "../../../common/physdev.c"
 
 /*
  * Local variables:
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 54de70d422..bcb1c8fb03 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -29,6 +29,7 @@ obj-y += notifier.o
 obj-y += page_alloc.o
 obj-$(CONFIG_HAS_PDX) += pdx.o
 obj-$(CONFIG_PERF_COUNTERS) += perfc.o
+obj-y += physdev.o
 obj-y += preempt.o
 obj-y += random.o
 obj-y += rangeset.o
diff --git a/xen/common/physdev.c b/xen/common/physdev.c
new file mode 100644
index 0000000000..20a5530269
--- /dev/null
+++ b/xen/common/physdev.c
@@ -0,0 +1,81 @@
+
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <xen/init.h>
+
+#ifndef COMPAT
+typedef long ret_t;
+#endif
+
+ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    ret_t ret;
+
+    switch ( cmd )
+    {
+#ifdef CONFIG_HAS_PCI
+    case PHYSDEVOP_pci_device_add: {
+        struct physdev_pci_device_add add;
+        struct pci_dev_info pdev_info;
+        nodeid_t node;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&add, arg, 1) != 0 )
+            break;
+
+        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
+        if ( add.flags & XEN_PCI_DEV_VIRTFN )
+        {
+            pdev_info.is_virtfn = 1;
+            pdev_info.physfn.bus = add.physfn.bus;
+            pdev_info.physfn.devfn = add.physfn.devfn;
+        }
+        else
+            pdev_info.is_virtfn = 0;
+
+        if ( add.flags & XEN_PCI_DEV_PXM )
+        {
+            uint32_t pxm;
+            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
+                                sizeof(add.optarr[0]);
+
+            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
+                break;
+
+            node = pxm_to_node(pxm);
+        }
+        else
+            node = NUMA_NO_NODE;
+
+        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
+        break;
+    }
+
+    case PHYSDEVOP_pci_device_remove: {
+        struct physdev_pci_device dev;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&dev, arg, 1) != 0 )
+            break;
+
+        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
+        break;
+    }
+#endif
+    default:
+        ret = arch_physdev_op(cmd, arg);
+        break;
+    }
+
+    return ret;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/hypercall.h b/xen/include/asm-arm/hypercall.h
index a0c5a31a2f..c6e4a04fd6 100644
--- a/xen/include/asm-arm/hypercall.h
+++ b/xen/include/asm-arm/hypercall.h
@@ -2,8 +2,6 @@
 #define __ASM_ARM_HYPERCALL_H__
 
 #include <public/domctl.h> /* for arch_do_domctl */
-int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-
 long do_arm_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg);
 
 long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
index 31a6de4e23..dbd1c2467f 100644
--- a/xen/include/asm-arm/numa.h
+++ b/xen/include/asm-arm/numa.h
@@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
 #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
 #define __node_distance(a, b) (20)
 
+static inline nodeid_t pxm_to_node(unsigned pxm)
+{
+    return 0xff;
+}
+
 #endif /* __ARCH_ARM_NUMA_H */
 /*
  * Local variables:
diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
index 0ae3b8b043..7afba98186 100644
--- a/xen/include/asm-x86/hypercall.h
+++ b/xen/include/asm-x86/hypercall.h
@@ -105,10 +105,6 @@ do_update_va_mapping(
     u64 val64,
     unsigned long flags);
 
-extern long
-do_physdev_op(
-    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-
 extern long
 do_update_va_mapping_otherdomain(
     unsigned long va,
@@ -151,6 +147,11 @@ compat_physdev_op(
     int cmd,
     XEN_GUEST_HANDLE_PARAM(void) arg);
 
+extern int
+arch_compat_physdev_op(
+    int cmd,
+    XEN_GUEST_HANDLE_PARAM(void) arg);
+
 extern int
 arch_compat_vcpu_op(
     int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg);
diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 3771487a30..5cc74f40b7 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -45,6 +45,14 @@ extern long
 do_platform_op(
     XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op);
 
+extern long
+do_physdev_op(
+    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
+
+extern long
+arch_physdev_op(
+    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
+
 /*
  * To allow safe resume of do_memory_op() after preemption, we need to know
  * at what point in the page list to resume. For this purpose I steal the
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:20:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:20:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198235.351618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHin-00014o-R2; Tue, 28 Sep 2021 18:20:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198235.351618; Tue, 28 Sep 2021 18:20:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHin-00014d-O1; Tue, 28 Sep 2021 18:20:53 +0000
Received: by outflank-mailman (input) for mailman id 198235;
 Tue, 28 Sep 2021 18:20:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHim-00012z-4e
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:20:52 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id cf4a5dd8-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:20:50 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 844996D;
 Tue, 28 Sep 2021 11:20:50 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9F3F73F793;
 Tue, 28 Sep 2021 11:20:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf4a5dd8-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v3 06/17] xen/device-tree: Add dt_property_read_variable_u32_array helper
Date: Tue, 28 Sep 2021 19:18:15 +0100
Message-Id: <9dba5f3457d9ead9084ac73297bf5a65e665a8d9.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3

Import the Linux helper of_property_read_variable_u32_array. This
function find and read an array of 32 bit integers from a property,
with bounds on the minimum and maximum array size.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Modify commit message to include upstream Linux commit-id not stable 
  Linux commit-id
Change in v2: Patch introduced in v2
---
 xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
 xen/include/xen/device_tree.h | 26 +++++++++++++++
 2 files changed, 87 insertions(+)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 03d25a81ce..53160d61f8 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_node *np,
     return 0;
 }
 
+/**
+ * dt_find_property_value_of_size
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @min:    minimum allowed length of property value
+ * @max:    maximum allowed length of property value (0 means unlimited)
+ * @len:    if !=NULL, actual length is written to here
+ *
+ * Search for a property in a device node and valid the requested size.
+ *
+ * Return: The property value on success, -EINVAL if the property does not
+ * exist, -ENODATA if property does not have a value, and -EOVERFLOW if the
+ * property data is too small or too large.
+ */
+static void *dt_find_property_value_of_size(const struct dt_device_node *np,
+                                            const char *propname, u32 min,
+                                            u32 max, size_t *len)
+{
+    const struct dt_property *prop = dt_find_property(np, propname, NULL);
+
+    if ( !prop )
+        return ERR_PTR(-EINVAL);
+    if ( !prop->value )
+        return ERR_PTR(-ENODATA);
+    if ( prop->length < min )
+        return ERR_PTR(-EOVERFLOW);
+    if ( max && prop->length > max )
+        return ERR_PTR(-EOVERFLOW);
+
+    if ( len )
+        *len = prop->length;
+
+    return prop->value;
+}
+
+int dt_property_read_variable_u32_array(const struct dt_device_node *np,
+                                        const char *propname, u32 *out_values,
+                                        size_t sz_min, size_t sz_max)
+{
+    size_t sz, count;
+    const __be32 *val = dt_find_property_value_of_size(np, propname,
+                        (sz_min * sizeof(*out_values)),
+                        (sz_max * sizeof(*out_values)),
+                        &sz);
+
+    if ( IS_ERR(val) )
+        return PTR_ERR(val);
+
+    if ( !sz_max )
+        sz = sz_min;
+    else
+        sz /= sizeof(*out_values);
+
+    count = sz;
+    while ( count-- )
+        *out_values++ = be32_to_cpup(val++);
+
+    return sz;
+}
+
 int dt_property_match_string(const struct dt_device_node *np,
                              const char *propname, const char *string)
 {
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index b02696be94..1693fb8e8c 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
 bool_t dt_property_read_u64(const struct dt_device_node *np,
                             const char *name, u64 *out_value);
 
+
+/**
+ * dt_property_read_variable_u32_array - Find and read an array of 32 bit
+ * integers from a property, with bounds on the minimum and maximum array size.
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @out_values: pointer to return found values.
+ * @sz_min: minimum number of array elements to read
+ * @sz_max: maximum number of array elements to read, if zero there is no
+ *      upper limit on the number of elements in the dts entry but only
+ *      sz_min will be read.
+ *
+ * Search for a property in a device node and read 32-bit value(s) from
+ * it.
+ *
+ * Return: The number of elements read on success, -EINVAL if the property
+ * does not exist, -ENODATA if property does not have a value, and -EOVERFLOW
+ * if the property data is smaller than sz_min or longer than sz_max.
+ *
+ * The out_values is modified only if a valid u32 value can be decoded.
+ */
+int dt_property_read_variable_u32_array(const struct dt_device_node *np,
+                                        const char *propname, u32 *out_values,
+                                        size_t sz_min, size_t sz_max);
+
 /**
  * dt_property_read_bool - Check if a property exists
  * @np: node to get the value
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:21:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:21:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198245.351630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHjK-0001kk-4z; Tue, 28 Sep 2021 18:21:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198245.351630; Tue, 28 Sep 2021 18:21:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHjK-0001kZ-0p; Tue, 28 Sep 2021 18:21:26 +0000
Received: by outflank-mailman (input) for mailman id 198245;
 Tue, 28 Sep 2021 18:21:25 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHjJ-0001kM-3o
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:21:25 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id e351fd90-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:21:24 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 249EB6D;
 Tue, 28 Sep 2021 11:21:24 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3F6233F793;
 Tue, 28 Sep 2021 11:21:23 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e351fd90-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v3 07/17] xen/device-tree: Add dt_property_read_u32_array helper
Date: Tue, 28 Sep 2021 19:18:16 +0100
Message-Id: <89788e202bc91e2bc7f9c3c33eb91ab7478381ee.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3

Import the Linux helper of_property_read_u32_array. This function find
and read an array of 32 bit integers from a property.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Modify commit message to include upstream Linux commit-id not stable
  Linux commit-id
Change in v2: Patch introduced in v2
---
 xen/include/xen/device_tree.h | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 1693fb8e8c..3ffe3eb3d2 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -392,6 +392,36 @@ int dt_property_read_variable_u32_array(const struct dt_device_node *np,
                                         const char *propname, u32 *out_values,
                                         size_t sz_min, size_t sz_max);
 
+/**
+ * dt_property_read_u32_array - Find and read an array of 32 bit integers
+ * from a property.
+ *
+ * @np:     device node from which the property value is to be read.
+ * @propname:   name of the property to be searched.
+ * @out_values: pointer to return value, modified only if return value is 0.
+ * @sz:     number of array elements to read
+ *
+ * Search for a property in a device node and read 32-bit value(s) from
+ * it.
+ *
+ * Return: 0 on success, -EINVAL if the property does not exist,
+ * -ENODATA if property does not have a value, and -EOVERFLOW if the
+ * property data isn't large enough.
+ *
+ * The out_values is modified only if a valid u32 value can be decoded.
+ */
+static inline int dt_property_read_u32_array(const struct dt_device_node *np,
+                                             const char *propname,
+                                             u32 *out_values, size_t sz)
+{
+    int ret = dt_property_read_variable_u32_array(np, propname, out_values,
+                              sz, 0);
+    if ( ret >= 0 )
+        return 0;
+    else
+        return ret;
+}
+
 /**
  * dt_property_read_bool - Check if a property exists
  * @np: node to get the value
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:21:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198249.351640 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHje-0002JJ-Cl; Tue, 28 Sep 2021 18:21:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198249.351640; Tue, 28 Sep 2021 18:21:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHje-0002JC-9q; Tue, 28 Sep 2021 18:21:46 +0000
Received: by outflank-mailman (input) for mailman id 198249;
 Tue, 28 Sep 2021 18:21:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHjd-0002Ii-4Q
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:21:45 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 6ae4ebc5-0c61-4d20-af29-907f25ad8507;
 Tue, 28 Sep 2021 18:21:44 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DAB296D;
 Tue, 28 Sep 2021 11:21:43 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 014CB3F793;
 Tue, 28 Sep 2021 11:21:42 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ae4ebc5-0c61-4d20-af29-907f25ad8507
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v3 08/17] xen/device-tree: Add dt_get_pci_domain_nr helper
Date: Tue, 28 Sep 2021 19:18:17 +0100
Message-Id: <f70ac215cd0c62d71231b8f61f12c1f8d801710e.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Based Linux commit 41e5c0f81d3e676d671d96a0a1fafb27abfbd9d7

Import the Linux helper of_get_pci_domain_nr. This function will try to
obtain the host bridge domain number by finding a property called
"linux,pci-domain" of the given device node.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Modify commit message to include upstream Linux commit-id not stable
  Linux commit-id
- Remove return value as those are not valid for XEN
Change in v2: Patch introduced in v2
---
 xen/common/device_tree.c      | 12 ++++++++++++
 xen/include/xen/device_tree.h | 17 +++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 53160d61f8..ea93da1725 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
     dt_alias_scan();
 }
 
+int dt_get_pci_domain_nr(struct dt_device_node *node)
+{
+    u32 domain;
+    int error;
+
+    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
+    if ( !error )
+        return -EINVAL;
+
+    return (u16)domain;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 3ffe3eb3d2..2297c59ce6 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -832,6 +832,23 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
                                const char *list_name,
                                const char *cells_name);
 
+/**
+ * dt_get_pci_domain_nr - Find the host bridge domain number
+ *            of the given device node.
+ * @node: Device tree node with the domain information.
+ *
+ * This function will try to obtain the host bridge domain number by finding
+ * a property called "linux,pci-domain" of the given device node.
+ *
+ * Return:
+ * * > 0    - On success, an associated domain number.
+ * * -EINVAL    - The property "linux,pci-domain" does not exist.
+ *
+ * Returns the associated domain number from DT in the range [0-0xffff], or
+ * a negative value if the required property is not found.
+ */
+int dt_get_pci_domain_nr(struct dt_device_node *node);
+
 #ifdef CONFIG_DEVICE_TREE_DEBUG
 #define dt_dprintk(fmt, args...)  \
     printk(XENLOG_DEBUG fmt, ## args)
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:21:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:21:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198253.351651 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHjp-0002my-P3; Tue, 28 Sep 2021 18:21:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198253.351651; Tue, 28 Sep 2021 18:21:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHjp-0002mr-M6; Tue, 28 Sep 2021 18:21:57 +0000
Received: by outflank-mailman (input) for mailman id 198253;
 Tue, 28 Sep 2021 18:21:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHjo-0002iO-W4
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:21:57 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 30f8032f-a30a-47aa-b339-519f9609134c;
 Tue, 28 Sep 2021 18:21:56 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CEE896D;
 Tue, 28 Sep 2021 11:21:55 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CF1FF3F793;
 Tue, 28 Sep 2021 11:21:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 30f8032f-a30a-47aa-b339-519f9609134c
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 09/17] xen/arm: Add support for PCI init to initialize the PCI driver.
Date: Tue, 28 Sep 2021 19:18:18 +0100
Message-Id: <31f2e3baf45f059a8ddac22fefc7cdfe1bc63ef5.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

pci_init(..) will be called during xen startup to initialize and probe
the PCI host-bridge driver.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Some nit for device_init(..) return logic
- Remove inline from acpi_pci_init(..)
- Modify return value for apci_pci_init(..) to return -EOPNOTSUPP
Change in v2:
- ACPI init function to return int
- pci_segments_init() called before dt/acpi init
---
 xen/arch/arm/pci/pci.c       | 51 ++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/device.h |  1 +
 2 files changed, 52 insertions(+)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index a7a7bc3213..e359bab9ea 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -12,6 +12,10 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/acpi.h>
+#include <xen/device_tree.h>
+#include <xen/errno.h>
+#include <xen/init.h>
 #include <xen/pci.h>
 
 /*
@@ -22,6 +26,53 @@ int arch_pci_clean_pirqs(struct domain *d)
     return 0;
 }
 
+static int __init dt_pci_init(void)
+{
+    struct dt_device_node *np;
+    int rc;
+
+    dt_for_each_device_node(dt_host, np)
+    {
+        rc = device_init(np, DEVICE_PCI, NULL);
+        /*
+         * Ignore the following error codes:
+         *   - EBADF: Indicate the current device is not a pci device.
+         *   - ENODEV: The pci device is not present or cannot be used by
+         *     Xen.
+         */
+        if( !rc || rc == -EBADF || rc == -ENODEV )
+            continue;
+
+        return rc;
+    }
+
+    return 0;
+}
+
+#ifdef CONFIG_ACPI
+static int __init acpi_pci_init(void)
+{
+    printk(XENLOG_ERR "ACPI pci init not supported \n");
+    return -EOPNOTSUPP;
+}
+#else
+static int __init acpi_pci_init(void)
+{
+    return -EINVAL;
+}
+#endif
+
+static int __init pci_init(void)
+{
+    pci_segments_init();
+
+    if ( acpi_disabled )
+        return dt_pci_init();
+    else
+        return acpi_pci_init();
+}
+__initcall(pci_init);
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index ee7cff2d44..5ecd5e7bd1 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -34,6 +34,7 @@ enum device_class
     DEVICE_SERIAL,
     DEVICE_IOMMU,
     DEVICE_GIC,
+    DEVICE_PCI,
     /* Use for error */
     DEVICE_UNKNOWN,
 };
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:22:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:22:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198261.351663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHk6-0003Rm-47; Tue, 28 Sep 2021 18:22:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198261.351663; Tue, 28 Sep 2021 18:22:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHk6-0003Rf-0q; Tue, 28 Sep 2021 18:22:14 +0000
Received: by outflank-mailman (input) for mailman id 198261;
 Tue, 28 Sep 2021 18:22:13 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHk4-0003C1-W1
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:22:13 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id fc9694f0-2088-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:22:06 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8A2D26D;
 Tue, 28 Sep 2021 11:22:06 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DB6093F793;
 Tue, 28 Sep 2021 11:22:04 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc9694f0-2088-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v3 10/17] xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
Date: Tue, 28 Sep 2021 19:18:19 +0100
Message-Id: <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Add cmdline boot option "pci-passthrough = = <boolean>" to enable or
disable the PCI passthrough support on ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Remove "define pci_passthrough_enabled (false)"
- Fixed coding style and minor comment
Change in v2:
- Add option in xen-command-line.pandoc
- Change pci option to pci-passthrough
- modify option from custom_param to boolean param
---
 docs/misc/xen-command-line.pandoc |  7 +++++++
 xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
 xen/common/physdev.c              |  6 ++++++
 xen/include/asm-arm/pci.h         | 11 +++++++++++
 xen/include/asm-x86/pci.h         |  8 ++++++++
 5 files changed, 46 insertions(+)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 177e656f12..c8bf96ccf8 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1808,6 +1808,13 @@ All numbers specified must be hexadecimal ones.
 
 This option can be specified more than once (up to 8 times at present).
 
+### pci-passthrough (arm)
+> `= <boolean>`
+
+> Default: `false`
+
+Flag to enable or disable support for PCI passthrough
+
 ### pcid (x86)
 > `= <boolean> | xpti=<bool>`
 
diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index e359bab9ea..84d8f0d634 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -16,6 +16,7 @@
 #include <xen/device_tree.h>
 #include <xen/errno.h>
 #include <xen/init.h>
+#include <xen/param.h>
 #include <xen/pci.h>
 
 /*
@@ -62,8 +63,21 @@ static int __init acpi_pci_init(void)
 }
 #endif
 
+/*
+ * By default pci passthrough is disabled
+ */
+bool __read_mostly pci_passthrough_enabled = false;
+boolean_param("pci-passthrough", pci_passthrough_enabled);
+
 static int __init pci_init(void)
 {
+    /*
+     * Enable PCI passthrough when has been enabled explicitly
+     * (pci-passthrough=on)
+     */
+    if ( !pci_passthrough_enabled )
+        return 0;
+
     pci_segments_init();
 
     if ( acpi_disabled )
diff --git a/xen/common/physdev.c b/xen/common/physdev.c
index 20a5530269..2d5fc886fc 100644
--- a/xen/common/physdev.c
+++ b/xen/common/physdev.c
@@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         struct pci_dev_info pdev_info;
         nodeid_t node;
 
+        if ( !is_pci_passthrough_enabled() )
+            return -ENOSYS;
+
         ret = -EFAULT;
         if ( copy_from_guest(&add, arg, 1) != 0 )
             break;
@@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_pci_device_remove: {
         struct physdev_pci_device dev;
 
+        if ( !is_pci_passthrough_enabled() )
+            return -ENOSYS;
+
         ret = -EFAULT;
         if ( copy_from_guest(&dev, arg, 1) != 0 )
             break;
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 7dd9eb3dba..0cf849e26f 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -19,14 +19,25 @@
 
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
 
+extern bool_t pci_passthrough_enabled;
+
 /* Arch pci dev struct */
 struct arch_pci_dev {
     struct device dev;
 };
 
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return pci_passthrough_enabled;
+}
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
 
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return false;
+}
+
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
index cc05045e9c..3f806ce7a8 100644
--- a/xen/include/asm-x86/pci.h
+++ b/xen/include/asm-x86/pci.h
@@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned int *seg,
 extern int pci_mmcfg_config_num;
 extern struct acpi_mcfg_allocation *pci_mmcfg_config;
 
+/*
+ * Unlike ARM, PCI passthrough is always enabled for x86.
+ */
+static always_inline bool is_pci_passthrough_enabled(void)
+{
+    return true;
+}
+
 #endif /* __X86_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:22:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:22:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198272.351674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHkT-0004BI-Ca; Tue, 28 Sep 2021 18:22:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198272.351674; Tue, 28 Sep 2021 18:22:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHkT-0004B7-9Q; Tue, 28 Sep 2021 18:22:37 +0000
Received: by outflank-mailman (input) for mailman id 198272;
 Tue, 28 Sep 2021 18:22:35 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHkR-00049L-Gr
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:22:35 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 0c953712-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:22:33 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6A9946D;
 Tue, 28 Sep 2021 11:22:33 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BDD7C3F793;
 Tue, 28 Sep 2021 11:22:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c953712-2089-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN on ARM
Date: Tue, 28 Sep 2021 19:18:20 +0100
Message-Id: <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XEN during boot will read the PCI device tree node “reg” property
and will map the PCI config space to the XEN memory.

As of now only "pci-host-ecam-generic" compatible board is supported.

"linux,pci-domain" device tree property assigns a fixed PCI domain
number to a host bridge, otherwise an unstable (across boots) unique
number will be assigned by Linux. XEN access the PCI devices based on
Segment:Bus:Device:Function. A Segment number in the XEN is same as a
domain number in Linux. Segment number and domain number has to be in
sync to access the correct PCI devices.

XEN will read the “linux,pci-domain” property from the device tree node
and configure the host bridge segment number accordingly. If this
property is not available XEN will allocate the unique segment number
to the host bridge.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Modify commit msg based on received comments.
- Remove added struct match_table{} struct in struct device{}
- Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
- Remove bus_start,bus_end and void *sysdata from struct pci_host_bridge{}
- Move "#include <asm/pci.h>" in "xen/pci.h" after pci_sbdf_t sbdf declaration
- Add pci_host_generic_probe() function 
Change in v2:
- Add more info in commit msg
- Add callback to parse register index.
- Merge patch pci_ecam_operation into this patch to avoid confusion
- Add new struct in struct device for match table
---
 xen/arch/arm/pci/Makefile           |   4 +
 xen/arch/arm/pci/ecam.c             |  61 +++++++
 xen/arch/arm/pci/pci-access.c       |  83 ++++++++++
 xen/arch/arm/pci/pci-host-common.c  | 247 ++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-generic.c |  46 ++++++
 xen/include/asm-arm/pci.h           |  56 +++++++
 xen/include/xen/pci.h               |   3 +-
 7 files changed, 499 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/pci/ecam.c
 create mode 100644 xen/arch/arm/pci/pci-access.c
 create mode 100644 xen/arch/arm/pci/pci-host-common.c
 create mode 100644 xen/arch/arm/pci/pci-host-generic.c

diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
index a98035df4c..6f32fbbe67 100644
--- a/xen/arch/arm/pci/Makefile
+++ b/xen/arch/arm/pci/Makefile
@@ -1 +1,5 @@
 obj-y += pci.o
+obj-y += pci-access.o
+obj-y += pci-host-generic.o
+obj-y += pci-host-common.o
+obj-y += ecam.o
diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
new file mode 100644
index 0000000000..602d00799c
--- /dev/null
+++ b/xen/arch/arm/pci/ecam.c
@@ -0,0 +1,61 @@
+/*
+ * Based on Linux drivers/pci/ecam.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+#include <xen/sched.h>
+
+/*
+ * Function to implement the pci_ops->map_bus method.
+ */
+void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
+                               pci_sbdf_t sbdf, uint32_t where)
+{
+    const struct pci_config_window *cfg = bridge->cfg;
+    struct pci_ecam_ops *ops =
+        container_of(bridge->ops, struct pci_ecam_ops, pci_ops);
+    unsigned int devfn_shift = ops->bus_shift - 8;
+    void __iomem *base;
+
+    unsigned int busn = PCI_BUS(sbdf.bdf);
+
+    if ( busn < cfg->busn_start || busn > cfg->busn_end )
+        return NULL;
+
+    busn -= cfg->busn_start;
+    base = cfg->win + (busn << ops->bus_shift);
+
+    return base + (PCI_DEVFN2(sbdf.bdf) << devfn_shift) + where;
+}
+
+/* ECAM ops */
+const struct pci_ecam_ops pci_generic_ecam_ops = {
+    .bus_shift  = 20,
+    .pci_ops    = {
+        .map_bus                = pci_ecam_map_bus,
+        .read                   = pci_generic_config_read,
+        .write                  = pci_generic_config_write,
+    }
+};
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
new file mode 100644
index 0000000000..3cd14a4b87
--- /dev/null
+++ b/xen/arch/arm/pci/pci-access.c
@@ -0,0 +1,83 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+#include <asm/io.h>
+
+#define INVALID_VALUE (~0U)
+
+int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t *value)
+{
+    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+
+    if ( !addr )
+    {
+        *value = INVALID_VALUE;
+        return -ENODEV;
+    }
+
+    switch ( len )
+    {
+    case 1:
+        *value = readb(addr);
+        break;
+    case 2:
+        *value = readw(addr);
+        break;
+    case 4:
+        *value = readl(addr);
+        break;
+    default:
+        ASSERT_UNREACHABLE();
+    }
+
+    return 0;
+}
+
+int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                             uint32_t reg, uint32_t len, uint32_t value)
+{
+    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+
+    if ( !addr )
+        return -ENODEV;
+
+    switch ( len )
+    {
+    case 1:
+        writeb(value, addr);
+        break;
+    case 2:
+        writew(value, addr);
+        break;
+    case 4:
+        writel(value, addr);
+        break;
+    default:
+        ASSERT_UNREACHABLE();
+    }
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
new file mode 100644
index 0000000000..a08e06cea1
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -0,0 +1,247 @@
+/*
+ * Based on Linux drivers/pci/ecam.c
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/init.h>
+#include <xen/pci.h>
+#include <xen/rwlock.h>
+#include <xen/sched.h>
+#include <xen/vmap.h>
+
+/*
+ * List for all the pci host bridges.
+ */
+
+static LIST_HEAD(pci_host_bridges);
+
+static atomic_t domain_nr = ATOMIC_INIT(-1);
+
+static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
+{
+    return ioremap_nocache(start, len);
+}
+
+static void pci_ecam_free(struct pci_config_window *cfg)
+{
+    if ( cfg->win )
+        iounmap(cfg->win);
+
+    xfree(cfg);
+}
+
+static struct pci_config_window * __init
+gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
+{
+    int err, cfg_reg_idx;
+    u32 bus_range[2];
+    paddr_t addr, size;
+    struct pci_config_window *cfg;
+
+    cfg = xzalloc(struct pci_config_window);
+    if ( !cfg )
+        return NULL;
+
+    err = dt_property_read_u32_array(dev, "bus-range", bus_range,
+                                     ARRAY_SIZE(bus_range));
+    if ( err ) {
+        cfg->busn_start = 0;
+        cfg->busn_end = 0xff;
+        printk(XENLOG_INFO "%s: No bus range found for pci controller\n",
+               dt_node_full_name(dev));
+    } else {
+        cfg->busn_start = bus_range[0];
+        cfg->busn_end = bus_range[1];
+        if ( cfg->busn_end > cfg->busn_start + 0xff )
+            cfg->busn_end = cfg->busn_start + 0xff;
+    }
+
+    if ( ops->cfg_reg_index )
+    {
+        cfg_reg_idx = ops->cfg_reg_index(dev);
+        if ( cfg_reg_idx < 0 )
+            goto err_exit;
+    }
+    else
+        cfg_reg_idx = 0;
+
+    /* Parse our PCI ecam register address */
+    err = dt_device_get_address(dev, cfg_reg_idx, &addr, &size);
+    if ( err )
+        goto err_exit;
+
+    cfg->phys_addr = addr;
+    cfg->size = size;
+
+    /*
+     * On 64-bit systems, we do a single ioremap for the whole config space
+     * since we have enough virtual address range available.  On 32-bit, we
+     * ioremap the config space for each bus individually.
+     * As of now only 64-bit is supported 32-bit is not supported.
+     *
+     * TODO: For 32-bit implement the ioremap/iounmap of config space
+     * dynamically for each read/write call.
+     */
+    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
+    if ( !cfg->win )
+    {
+        printk(XENLOG_ERR "ECAM ioremap failed\n");
+        goto err_exit;
+    }
+    printk("ECAM at [mem 0x%"PRIpaddr"-0x%"PRIpaddr"] for [bus %x-%x] \n",
+            cfg->phys_addr, cfg->phys_addr + cfg->size - 1,
+            cfg->busn_start, cfg->busn_end);
+
+    if ( ops->init )
+    {
+        err = ops->init(cfg);
+        if ( err )
+            goto err_exit;
+    }
+
+    return cfg;
+
+err_exit:
+    pci_ecam_free(cfg);
+
+    return NULL;
+}
+
+struct pci_host_bridge *pci_alloc_host_bridge(void)
+{
+    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
+
+    if ( !bridge )
+        return NULL;
+
+    INIT_LIST_HEAD(&bridge->node);
+
+    return bridge;
+}
+
+void pci_add_host_bridge(struct pci_host_bridge *bridge)
+{
+    list_add_tail(&bridge->node, &pci_host_bridges);
+}
+
+static int pci_get_new_domain_nr(void)
+{
+    return atomic_inc_return(&domain_nr);
+}
+
+static int pci_bus_find_domain_nr(struct dt_device_node *dev)
+{
+    static int use_dt_domains = -1;
+    int domain;
+
+    domain = dt_get_pci_domain_nr(dev);
+
+    /*
+     * Check DT domain and use_dt_domains values.
+     *
+     * If DT domain property is valid (domain >= 0) and
+     * use_dt_domains != 0, the DT assignment is valid since this means
+     * we have not previously allocated a domain number by using
+     * pci_get_new_domain_nr(); we should also update use_dt_domains to
+     * 1, to indicate that we have just assigned a domain number from
+     * DT.
+     *
+     * If DT domain property value is not valid (ie domain < 0), and we
+     * have not previously assigned a domain number from DT
+     * (use_dt_domains != 1) we should assign a domain number by
+     * using the:
+     *
+     * pci_get_new_domain_nr()
+     *
+     * API and update the use_dt_domains value to keep track of method we
+     * are using to assign domain numbers (use_dt_domains = 0).
+     *
+     * All other combinations imply we have a platform that is trying
+     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
+     * which is a recipe for domain mishandling and it is prevented by
+     * invalidating the domain value (domain = -1) and printing a
+     * corresponding error.
+     */
+    if ( domain >= 0 && use_dt_domains )
+    {
+        use_dt_domains = 1;
+    }
+    else if ( domain < 0 && use_dt_domains != 1 )
+    {
+        use_dt_domains = 0;
+        domain = pci_get_new_domain_nr();
+    }
+    else
+    {
+        domain = -1;
+    }
+
+    return domain;
+}
+
+int pci_host_common_probe(struct dt_device_node *dev, const void *data)
+{
+    struct pci_host_bridge *bridge;
+    struct pci_config_window *cfg;
+    struct pci_ecam_ops *ops;
+    int err;
+
+    if ( dt_device_for_passthrough(dev) )
+        return 0;
+
+    ops = (struct pci_ecam_ops *) data;
+
+    bridge = pci_alloc_host_bridge();
+    if ( !bridge )
+        return -ENOMEM;
+
+    /* Parse and map our Configuration Space windows */
+    cfg = gen_pci_init(dev, ops);
+    if ( !cfg )
+    {
+        err = -ENOMEM;
+        goto err_exit;
+    }
+
+    bridge->dt_node = dev;
+    bridge->cfg = cfg;
+    bridge->ops = &ops->pci_ops;
+
+    bridge->segment = pci_bus_find_domain_nr(dev);
+    if ( bridge->segment < 0 )
+    {
+        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
+        BUG();
+    }
+    pci_add_host_bridge(bridge);
+
+    return 0;
+
+err_exit:
+    xfree(bridge);
+
+    return err;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
new file mode 100644
index 0000000000..5e4f8f28a0
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -0,0 +1,46 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/device.h>
+#include <xen/pci.h>
+#include <asm/pci.h>
+
+static const struct dt_device_match gen_pci_dt_match[] = {
+    { .compatible = "pci-host-ecam-generic" },
+    { },
+};
+
+static int pci_host_generic_probe(struct dt_device_node *dev,
+                                  const void *data)
+{
+    return pci_host_common_probe(dev, &pci_generic_ecam_ops);
+}
+
+DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
+.dt_match = gen_pci_dt_match,
+.init = pci_host_generic_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 0cf849e26f..bb7eda6705 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -26,6 +26,62 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/*
+ * struct to hold the mappings of a config space window. This
+ * is expected to be used as sysdata for PCI controllers that
+ * use ECAM.
+ */
+struct pci_config_window {
+    paddr_t         phys_addr;
+    paddr_t         size;
+    uint8_t         busn_start;
+    uint8_t         busn_end;
+    void __iomem    *win;
+};
+
+/*
+ * struct to hold pci host bridge information
+ * for a PCI controller.
+ */
+struct pci_host_bridge {
+    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
+    struct list_head node;           /* Node in list of host bridges */
+    uint16_t segment;                /* Segment number */
+    struct pci_config_window* cfg;   /* Pointer to the bridge config window */
+    struct pci_ops *ops;
+};
+
+struct pci_ops {
+    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                             uint32_t offset);
+    int (*read)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                uint32_t reg, uint32_t len, uint32_t *value);
+    int (*write)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                 uint32_t reg, uint32_t len, uint32_t value);
+};
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_ecam_ops {
+    unsigned int            bus_shift;
+    struct pci_ops          pci_ops;
+    int (*cfg_reg_index)(struct dt_device_node *dev);
+    int (*init)(struct pci_config_window *);
+};
+
+/* Default ECAM ops */
+extern const struct pci_ecam_ops pci_generic_ecam_ops;
+
+int pci_host_common_probe(struct dt_device_node *dev, const void *data);
+int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                            uint32_t reg, uint32_t len, uint32_t *value);
+int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+                             uint32_t reg, uint32_t len, uint32_t value);
+void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
+                               pci_sbdf_t sbdf, uint32_t where);
+
 static always_inline bool is_pci_passthrough_enabled(void)
 {
     return pci_passthrough_enabled;
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 8e3d4d9454..70ac25345c 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -15,7 +15,6 @@
 #include <xen/pfn.h>
 #include <asm/device.h>
 #include <asm/numa.h>
-#include <asm/pci.h>
 
 /*
  * The PCI interface treats multi-function devices as independent
@@ -62,6 +61,8 @@ typedef union {
     };
 } pci_sbdf_t;
 
+#include <asm/pci.h>
+
 struct pci_dev_info {
     /*
      * VF's 'is_extfn' field is used to indicate whether its PF is an extended
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:22:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:22:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198278.351685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHkk-0004j9-Qo; Tue, 28 Sep 2021 18:22:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198278.351685; Tue, 28 Sep 2021 18:22:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHkk-0004j2-NC; Tue, 28 Sep 2021 18:22:54 +0000
Received: by outflank-mailman (input) for mailman id 198278;
 Tue, 28 Sep 2021 18:22:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHkj-0004g0-Eq
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:22:53 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 8c62079f-236d-49c5-9a98-bd6b520a552e;
 Tue, 28 Sep 2021 18:22:52 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0810B6D;
 Tue, 28 Sep 2021 11:22:52 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E1B873F793;
 Tue, 28 Sep 2021 11:22:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c62079f-236d-49c5-9a98-bd6b520a552e
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host controller
Date: Tue, 28 Sep 2021 19:18:21 +0100
Message-Id: <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add support for Xilinx ZynqMP PCI host controller to map the PCI config
space to the XEN memory.

Patch helps to understand how the generic infrastructure for PCI
host-bridge discovery will be used for future references.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Change in v3:
- nwl_cfg_reg_index(..) as static function
- Add support for pci_host_generic_probe() 
Change in v2:
- Add more info in commit msg
---
 xen/arch/arm/pci/Makefile          |  1 +
 xen/arch/arm/pci/pci-host-zynqmp.c | 63 ++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)
 create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c

diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
index 6f32fbbe67..1d045ade01 100644
--- a/xen/arch/arm/pci/Makefile
+++ b/xen/arch/arm/pci/Makefile
@@ -3,3 +3,4 @@ obj-y += pci-access.o
 obj-y += pci-host-generic.o
 obj-y += pci-host-common.o
 obj-y += ecam.o
+obj-y += pci-host-zynqmp.o
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
new file mode 100644
index 0000000000..6ccbfd15c9
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -0,0 +1,63 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ * Based on xen/arch/arm/pci/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/device.h>
+#include <xen/pci.h>
+#include <asm/pci.h>
+
+static int nwl_cfg_reg_index(struct dt_device_node *np)
+{
+    return dt_property_match_string(np, "reg-names", "cfg");
+}
+
+/* ECAM ops */
+const struct pci_ecam_ops nwl_pcie_ops = {
+    .bus_shift  = 20,
+    .cfg_reg_index = nwl_cfg_reg_index,
+    .pci_ops    = {
+        .map_bus                = pci_ecam_map_bus,
+        .read                   = pci_generic_config_read,
+        .write                  = pci_generic_config_write,
+    }
+};
+
+static const struct dt_device_match nwl_pcie_dt_match[] = {
+    { .compatible = "xlnx,nwl-pcie-2.11" },
+    { },
+};
+
+static int pci_host_generic_probe(struct dt_device_node *dev,
+                                  const void *data)
+{
+    return pci_host_common_probe(dev, &nwl_pcie_ops);
+}
+
+DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
+.dt_match = nwl_pcie_dt_match,
+.init = pci_host_generic_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:22:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:22:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198280.351696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHko-00054c-3F; Tue, 28 Sep 2021 18:22:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198280.351696; Tue, 28 Sep 2021 18:22:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHkn-00054R-VN; Tue, 28 Sep 2021 18:22:57 +0000
Received: by outflank-mailman (input) for mailman id 198280;
 Tue, 28 Sep 2021 18:22:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHkn-0004g0-Bj
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:22:57 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id a725ef51-be9c-48fa-a825-9e0c02c51380;
 Tue, 28 Sep 2021 18:22:55 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 220626D;
 Tue, 28 Sep 2021 11:22:55 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 214DB3F793;
 Tue, 28 Sep 2021 11:22:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a725ef51-be9c-48fa-a825-9e0c02c51380
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 13/17] xen/arm: Implement pci access functions
Date: Tue, 28 Sep 2021 19:18:22 +0100
Message-Id: <33cc6b7a133787700ea8ba4e54a03141d3aca1ea.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

Implement generic pci access functions to read/write the configuration
space.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Remove PRI_pci as not used.
- Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
Change in v2: Fixed comments
---
 xen/arch/arm/pci/pci-access.c      | 57 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
 xen/include/asm-arm/pci.h          |  1 +
 3 files changed, 77 insertions(+)

diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
index 3cd14a4b87..9f9aac43d7 100644
--- a/xen/arch/arm/pci/pci-access.c
+++ b/xen/arch/arm/pci/pci-access.c
@@ -16,6 +16,7 @@
 #include <asm/io.h>
 
 #define INVALID_VALUE (~0U)
+#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
 
 int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
                             uint32_t reg, uint32_t len, uint32_t *value)
@@ -72,6 +73,62 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
     return 0;
 }
 
+static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
+                                unsigned int len)
+{
+    uint32_t val = PCI_ERR_VALUE(len);
+    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+
+    if ( unlikely(!bridge) )
+        return val;
+
+    if ( unlikely(!bridge->ops->read) )
+        return val;
+
+    bridge->ops->read(bridge, sbdf, reg, len, &val);
+
+    return val;
+}
+
+static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
+                             unsigned int len, uint32_t val)
+{
+    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+
+    if ( unlikely(!bridge) )
+        return;
+
+    if ( unlikely(!bridge->ops->write) )
+        return;
+
+    bridge->ops->write(bridge, sbdf, reg, len, val);
+}
+
+/*
+ * Wrappers for all PCI configuration access functions.
+ */
+
+#define PCI_OP_WRITE(size, type)                            \
+    void pci_conf_write##size(pci_sbdf_t sbdf,              \
+                              unsigned int reg, type val)   \
+{                                                           \
+    pci_config_write(sbdf, reg, size / 8, val);             \
+}
+
+#define PCI_OP_READ(size, type)                             \
+    type pci_conf_read##size(pci_sbdf_t sbdf,               \
+                              unsigned int reg)             \
+{                                                           \
+    return pci_config_read(sbdf, reg, size / 8);            \
+}
+
+PCI_OP_READ(8, uint8_t)
+PCI_OP_READ(16, uint16_t)
+PCI_OP_READ(32, uint32_t)
+PCI_OP_WRITE(8, uint8_t)
+PCI_OP_WRITE(16, uint16_t)
+PCI_OP_WRITE(32, uint32_t)
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index a08e06cea1..c5941b10e9 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -236,6 +236,25 @@ err_exit:
     return err;
 }
 
+/*
+ * This function will lookup an hostbridge based on the segment and bus
+ * number.
+ */
+struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
+{
+    struct pci_host_bridge *bridge;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        if ( bridge->segment != segment )
+            continue;
+        if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn_end) )
+            continue;
+        return bridge;
+    }
+
+    return NULL;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index bb7eda6705..49c9622902 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -81,6 +81,7 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
                              uint32_t reg, uint32_t len, uint32_t value);
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                pci_sbdf_t sbdf, uint32_t where);
+struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:23:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:23:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198297.351707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlE-00063V-Be; Tue, 28 Sep 2021 18:23:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198297.351707; Tue, 28 Sep 2021 18:23:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlE-00063O-82; Tue, 28 Sep 2021 18:23:24 +0000
Received: by outflank-mailman (input) for mailman id 198297;
 Tue, 28 Sep 2021 18:23:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHlD-0005WT-65
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:23:23 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 28da69b0-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:23:21 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-297-iyZcQTaONDm3MWfcKShKpA-1; Tue, 28 Sep 2021 14:23:19 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 810A51023F50;
 Tue, 28 Sep 2021 18:23:16 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 0433660877;
 Tue, 28 Sep 2021 18:22:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28da69b0-2089-11ec-bcc4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853400;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding;
	bh=8/a5gkIL9yAofs88ckFRESPQGSjIdBv52g14ib29Qfw=;
	b=fSH7GZ1PtF8GFEluGZIH5MaHzfZmrJ0fCrzXSiBry14JbWeXeaLeHDjwI3SeP8eg4r3kwj
	BfKmdxL2T6eZ3KZ4kd5P45j/Ggg0I/y7dCDGsEUBun/Fw18Wiyx0Txn8ONFh3itKTxdEgg
	i9JFgvNmB6zvROnf0Y5t2mMXK+yFigQ=
X-MC-Unique: iyZcQTaONDm3MWfcKShKpA-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 0/8] proc/vmcore: sanitize access to virtio-mem memory
Date: Tue, 28 Sep 2021 20:22:50 +0200
Message-Id: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

As so often with virtio-mem changes that mess with common MM
infrastructure, this might be a good candiate to go via Andrew's tree.

--

After removing /dev/kmem, sanitizing /proc/kcore and handling /dev/mem,
this series tackles the last sane way how a VM could accidentially access
logically unplugged memory managed by a virtio-mem device: /proc/vmcore

When dumping memory via "makedumpfile", PG_offline pages, used by
virtio-mem to flag logically unplugged memory, are already properly
excluded; however, especially when accessing/copying /proc/vmcore "the
usual way", we can still end up reading logically unplugged memory part of
a virtio-mem device.

Patch #1-#3 are cleanups. Patch #4 extends the existing oldmem_pfn_is_ram
mechanism. Patch #5-#7 are virtio-mem refactorings for patch #8, which
implements the virtio-mem logic to query the state of device blocks.

Patch #8:

"
Although virtio-mem currently supports reading unplugged memory in the
hypervisor, this will change in the future, indicated to the device via
a new feature flag. We similarly sanitized /proc/kcore access recently.
[...]
Distributions that support virtio-mem+kdump have to make sure that the
virtio_mem module will be part of the kdump kernel or the kdump initrd;
dracut was recently [2] extended to include virtio-mem in the generated
initrd. As long as no special kdump kernels are used, this will
automatically make sure that virtio-mem will be around in the kdump initrd
and sanitize /proc/vmcore access -- with dracut.
"

This is the last remaining bit to support
VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE [3] in the Linux implementation of
virtio-mem.

Note: this is best-effort. We'll never be able to control what runs inside
the second kernel, really, but we also don't have to care: we only care
about sane setups where we don't want our VM getting zapped once we
touch the wrong memory location while dumping. While we usually expect sane
setups to use "makedumfile", nothing really speaks against just copying
/proc/vmcore, especially in environments where HWpoisioning isn't typically
expected. Also, we really don't want to put all our trust completely on the
memmap, so sanitizing also makes sense when just using "makedumpfile".

[1] https://lkml.kernel.org/r/20210526093041.8800-1-david@redhat.com
[2] https://github.com/dracutdevs/dracut/pull/1157
[3] https://lists.oasis-open.org/archives/virtio-comment/202109/msg00021.html

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>
Cc: x86@kernel.org
Cc: xen-devel@lists.xenproject.org
Cc: virtualization@lists.linux-foundation.org
Cc: kexec@lists.infradead.org
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-mm@kvack.org

David Hildenbrand (8):
  x86/xen: update xen_oldmem_pfn_is_ram() documentation
  x86/xen: simplify xen_oldmem_pfn_is_ram()
  proc/vmcore: let pfn_is_ram() return a bool
  proc/vmcore: convert oldmem_pfn_is_ram callback to more generic vmcore
    callbacks
  virtio-mem: factor out hotplug specifics from virtio_mem_init() into
    virtio_mem_init_hotplug()
  virtio-mem: factor out hotplug specifics from virtio_mem_probe() into
    virtio_mem_init_hotplug()
  virtio-mem: factor out hotplug specifics from virtio_mem_remove() into
    virtio_mem_deinit_hotplug()
  virtio-mem: kdump mode to sanitize /proc/vmcore access

 arch/x86/kernel/aperture_64.c |  13 +-
 arch/x86/xen/mmu_hvm.c        |  31 ++--
 drivers/virtio/virtio_mem.c   | 297 ++++++++++++++++++++++++----------
 fs/proc/vmcore.c              | 105 ++++++++----
 include/linux/crash_dump.h    |  26 ++-
 5 files changed, 332 insertions(+), 140 deletions(-)


base-commit: 5816b3e6577eaa676ceb00a848f0fd65fe2adc29
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:23:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:23:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198299.351717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlM-0006Qo-JV; Tue, 28 Sep 2021 18:23:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198299.351717; Tue, 28 Sep 2021 18:23:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlM-0006Qh-G1; Tue, 28 Sep 2021 18:23:32 +0000
Received: by outflank-mailman (input) for mailman id 198299;
 Tue, 28 Sep 2021 18:23:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHlK-0005t2-KC
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:23:30 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id b27d2215-ddaa-491a-8f86-d205518f6048;
 Tue, 28 Sep 2021 18:23:27 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-392-V1xArUnZPgOcWIzpkWqGiQ-1; Tue, 28 Sep 2021 14:23:25 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 582378145EF;
 Tue, 28 Sep 2021 18:23:22 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id DE92A60854;
 Tue, 28 Sep 2021 18:23:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b27d2215-ddaa-491a-8f86-d205518f6048
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853407;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gYZMK3UAt+mBMIBRezGxf4/V47kcWFAO89PKtDrJxFs=;
	b=ChmAjoD//b+aNe5MaIKdYHcTLx7Vltb/wMTpQwt64J0j3ThdQnX9vEZLsGvW8BsvOU7bH2
	XrGw7WZW+zuFXUH0w5hnPxwziuDyLBPEStGkjgMkZO48imP24ted8HxtuT/Fk/GbpS+soY
	MhZEcxm3SKuXmq53Tf8oeNE41BZ+hvo=
X-MC-Unique: V1xArUnZPgOcWIzpkWqGiQ-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 1/8] x86/xen: update xen_oldmem_pfn_is_ram() documentation
Date: Tue, 28 Sep 2021 20:22:51 +0200
Message-Id: <20210928182258.12451-2-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

The callback is only used for the vmcore nowadays.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 arch/x86/xen/mmu_hvm.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/arch/x86/xen/mmu_hvm.c b/arch/x86/xen/mmu_hvm.c
index 57409373750f..b242d1f4b426 100644
--- a/arch/x86/xen/mmu_hvm.c
+++ b/arch/x86/xen/mmu_hvm.c
@@ -9,12 +9,9 @@
 
 #ifdef CONFIG_PROC_VMCORE
 /*
- * This function is used in two contexts:
- * - the kdump kernel has to check whether a pfn of the crashed kernel
- *   was a ballooned page. vmcore is using this function to decide
- *   whether to access a pfn of the crashed kernel.
- * - the kexec kernel has to check whether a pfn was ballooned by the
- *   previous kernel. If the pfn is ballooned, handle it properly.
+ * The kdump kernel has to check whether a pfn of the crashed kernel
+ * was a ballooned page. vmcore is using this function to decide
+ * whether to access a pfn of the crashed kernel.
  * Returns 0 if the pfn is not backed by a RAM page, the caller may
  * handle the pfn special in this case.
  */
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:23:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:23:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198302.351729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlW-0006yv-TL; Tue, 28 Sep 2021 18:23:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198302.351729; Tue, 28 Sep 2021 18:23:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlW-0006yl-Q3; Tue, 28 Sep 2021 18:23:42 +0000
Received: by outflank-mailman (input) for mailman id 198302;
 Tue, 28 Sep 2021 18:23:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHlU-0005t2-KV
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:23:40 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 113a5697-4cfd-44ca-9739-44131b0b555f;
 Tue, 28 Sep 2021 18:23:36 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 629266D;
 Tue, 28 Sep 2021 11:23:36 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9B1DA3F793;
 Tue, 28 Sep 2021 11:23:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 113a5697-4cfd-44ca-9739-44131b0b555f
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH v3 14/17] xen/arm: Enable the existing x86 virtual PCI support for ARM.
Date: Tue, 28 Sep 2021 19:18:23 +0100
Message-Id: <d7f62249e6aa7b33017ef707d6bc451b5e2c7047.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The existing VPCI support available for X86 is adapted for Arm.
When the device is added to XEN via the hyper call
“PHYSDEVOP_pci_device_add”, VPCI handler for the config space
access is added to the Xen to emulate the PCI devices config space.

A MMIO trap handler for the PCI ECAM space is registered in XEN
so that when guest is trying to access the PCI config space,XEN
will trap the access and emulate read/write using the VPCI and
not the real PCI hardware.

For Dom0less systems scan_pci_devices() would be used to discover the
PCI device in XEN and VPCI handler will be added during XEN boots.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Use is_pci_passthrough_enabled() in place of pci_passthrough_enabled variable
- Reject XEN_DOMCTL_CDF_vpci for x86 in arch_sanitise_domain_config()
- Remove IS_ENABLED(CONFIG_HAS_VPCI) from has_vpci()
Change in v2:
- Add new XEN_DOMCTL_CDF_vpci flag
- modify has_vpci() to include XEN_DOMCTL_CDF_vpci
- enable vpci support when pci-passthough option is enabled.
---
 xen/arch/arm/Makefile         |   1 +
 xen/arch/arm/domain.c         |   8 ++-
 xen/arch/arm/domain_build.c   |   3 +
 xen/arch/arm/vpci.c           | 102 ++++++++++++++++++++++++++++++++++
 xen/arch/arm/vpci.h           |  36 ++++++++++++
 xen/arch/x86/domain.c         |   6 ++
 xen/common/domain.c           |   2 +-
 xen/drivers/passthrough/pci.c |  12 ++++
 xen/include/asm-arm/domain.h  |   8 ++-
 xen/include/asm-x86/pci.h     |   2 -
 xen/include/public/arch-arm.h |   7 +++
 xen/include/public/domctl.h   |   4 +-
 xen/include/xen/pci.h         |   2 +
 13 files changed, 186 insertions(+), 7 deletions(-)
 create mode 100644 xen/arch/arm/vpci.c
 create mode 100644 xen/arch/arm/vpci.h

diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 44d7cc81fa..fb9c976ea2 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
 obj-y += platforms/
 endif
 obj-$(CONFIG_TEE) += tee/
+obj-$(CONFIG_HAS_VPCI) += vpci.o
 
 obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
 obj-y += bootfdt.init.o
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d..fbb52f78f1 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -39,6 +39,7 @@
 #include <asm/vgic.h>
 #include <asm/vtimer.h>
 
+#include "vpci.h"
 #include "vuart.h"
 
 DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
@@ -622,8 +623,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
     unsigned int max_vcpus;
 
-    /* HVM and HAP must be set. IOMMU may or may not be */
-    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
+    /* HVM and HAP must be set. IOMMU and VPCI may or may not be */
+    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci) !=
          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
@@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
     if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
         goto fail;
 
+    if ( (rc = domain_vpci_init(d)) != 0 )
+        goto fail;
+
     return 0;
 
 fail:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d634c1..1731ae2028 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2773,6 +2773,9 @@ void __init create_dom0(void)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
+    if ( is_pci_passthrough_enabled() )
+        dom0_cfg.flags |= XEN_DOMCTL_CDF_vpci;
+
     dom0 = domain_create(0, &dom0_cfg, true);
     if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
         panic("Error creating domain 0\n");
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
new file mode 100644
index 0000000000..76c12b9281
--- /dev/null
+++ b/xen/arch/arm/vpci.c
@@ -0,0 +1,102 @@
+/*
+ * xen/arch/arm/vpci.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+#include <xen/sched.h>
+
+#include <asm/mmio.h>
+
+#define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
+
+/* Do some sanity checks. */
+static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
+{
+    /* Check access size. */
+    if ( len > 8 )
+        return false;
+
+    /* Check that access is size aligned. */
+    if ( (reg & (len - 1)) )
+        return false;
+
+    return true;
+}
+
+static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
+                          register_t *r, void *p)
+{
+    unsigned int reg;
+    pci_sbdf_t sbdf;
+    unsigned long data = ~0UL;
+    unsigned int size = 1U << info->dabt.size;
+
+    sbdf.sbdf = MMCFG_BDF(info->gpa);
+    reg = REGISTER_OFFSET(info->gpa);
+
+    if ( !vpci_mmio_access_allowed(reg, size) )
+        return 0;
+
+    data = vpci_read(sbdf, reg, min(4u, size));
+    if ( size == 8 )
+        data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
+
+    *r = data;
+
+    return 1;
+}
+
+static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
+                           register_t r, void *p)
+{
+    unsigned int reg;
+    pci_sbdf_t sbdf;
+    unsigned long data = r;
+    unsigned int size = 1U << info->dabt.size;
+
+    sbdf.sbdf = MMCFG_BDF(info->gpa);
+    reg = REGISTER_OFFSET(info->gpa);
+
+    if ( !vpci_mmio_access_allowed(reg, size) )
+        return 0;
+
+    vpci_write(sbdf, reg, min(4u, size), data);
+    if ( size == 8 )
+        vpci_write(sbdf, reg + 4, 4, data >> 32);
+
+    return 1;
+}
+
+static const struct mmio_handler_ops vpci_mmio_handler = {
+    .read  = vpci_mmio_read,
+    .write = vpci_mmio_write,
+};
+
+int domain_vpci_init(struct domain *d)
+{
+    if ( !has_vpci(d) )
+        return 0;
+
+    register_mmio_handler(d, &vpci_mmio_handler,
+                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
+
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
new file mode 100644
index 0000000000..d8a7b0e3e8
--- /dev/null
+++ b/xen/arch/arm/vpci.h
@@ -0,0 +1,36 @@
+/*
+ * xen/arch/arm/vpci.h
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ARCH_ARM_VPCI_H__
+#define __ARCH_ARM_VPCI_H__
+
+#ifdef CONFIG_HAS_VPCI
+int domain_vpci_init(struct domain *d);
+#else
+static inline int domain_vpci_init(struct domain *d)
+{
+    return 0;
+}
+#endif
+
+#endif /* __ARCH_ARM_VPCI_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ef1812dc14..d7a48781c5 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -662,6 +662,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    if ( config->flags & XEN_DOMCTL_CDF_vpci )
+    {
+        dprintk(XENLOG_INFO, "VPCI not supported\n");
+        return -EINVAL;
+    }
+
     if ( config->vmtrace_size )
     {
         unsigned int size = config->vmtrace_size;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..40d67ec342 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
-           XEN_DOMCTL_CDF_nested_virt) )
+           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
     {
         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
         return -EINVAL;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index d774a6154e..633e89ac13 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
     else
         iommu_enable_device(pdev);
 
+#ifdef CONFIG_ARM
+    /*
+     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
+     * Dom0 inform XEN to add the PCI devices in XEN.
+     */
+    ret = vpci_add_handlers(pdev);
+    if ( ret ) {
+        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
+        goto out;
+    }
+#endif
+
     pci_enable_acs(pdev);
 
 out:
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c9277b5c6d..5d9dd6b0d0 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -2,6 +2,7 @@
 #define __ASM_DOMAIN_H__
 
 #include <xen/cache.h>
+#include <xen/nospec.h>
 #include <xen/timer.h>
 #include <asm/page.h>
 #include <asm/p2m.h>
@@ -262,7 +263,12 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
 
 #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
 
-#define has_vpci(d)    ({ (void)(d); false; })
+/*
+ * For X86 VPCI is enabled and tested for PVH DOM0 only but
+ * for ARM we enable support VPCI for guest domain also.
+ */
+#define has_vpci(d) ((void)(d), \
+        evaluate_nospec(d->options & XEN_DOMCTL_CDF_vpci))
 
 #endif /* __ASM_DOMAIN_H__ */
 
diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
index 3f806ce7a8..61b940c91d 100644
--- a/xen/include/asm-x86/pci.h
+++ b/xen/include/asm-x86/pci.h
@@ -6,8 +6,6 @@
 #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
 #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
 
-#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
-
 #define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \
                         || id == 0x01268086 || id == 0x01028086 \
                         || id == 0x01128086 || id == 0x01228086 \
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 6b5a5f818a..727541a321 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -416,6 +416,13 @@ typedef uint64_t xen_callback_t;
 #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
 #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
 
+/*
+ * 256 MB is reserved for VPCI configuration space based on calculation
+ * 256 buses × 32 devices × 8 functions × 4 KB = 256 MB
+ */
+#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
+#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
+
 /* ACPI tables physical address */
 #define GUEST_ACPI_BASE xen_mk_ullong(0x20000000)
 #define GUEST_ACPI_SIZE xen_mk_ullong(0x02000000)
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..4245da3f45 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -70,9 +70,11 @@ struct xen_domctl_createdomain {
 #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
 #define _XEN_DOMCTL_CDF_nested_virt   6
 #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
+#define _XEN_DOMCTL_CDF_vpci          7
+#define XEN_DOMCTL_CDF_vpci           (1U << _XEN_DOMCTL_CDF_vpci)
 
 /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
-#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
+#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_vpci
 
     uint32_t flags;
 
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 70ac25345c..43b8a08170 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -40,6 +40,8 @@
 #define PCI_SBDF3(s,b,df) \
     ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
 
+#define MMCFG_BDF(addr)  (((addr) & 0x0ffff000) >> 12)
+
 typedef union {
     uint32_t sbdf;
     struct {
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:23:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198309.351740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlj-0007fz-FD; Tue, 28 Sep 2021 18:23:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198309.351740; Tue, 28 Sep 2021 18:23:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlj-0007ez-A1; Tue, 28 Sep 2021 18:23:55 +0000
Received: by outflank-mailman (input) for mailman id 198309;
 Tue, 28 Sep 2021 18:23:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHlh-0007E1-Uc
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:23:53 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 3796cc6f-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:23:45 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-440-5NJ-y1XEOa2GaoLjlra03g-1; Tue, 28 Sep 2021 14:23:44 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8A7EA40C6;
 Tue, 28 Sep 2021 18:23:41 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id A78BE60877;
 Tue, 28 Sep 2021 18:23:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3796cc6f-2089-11ec-bcc4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853425;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=iXipVHA7HDBxHYsFQ8A47J68c7im5DD1EeD2KWkQL+g=;
	b=LpJTz1uGsziiZ185xv3fNcqHaXbhuTXq0cAUbF6brEUYvl4hrhukpgOC+EQjvTX3IZYCz9
	KZl5xw2TPg8ePtcTRu3FsSFyzXPu+rAL0hyAB6rPbFjqetroq+H57YxDn9h9Axf9hlfuxP
	RyB4vH39NI3QkpnP+7/TOWxmdoW+Aac=
X-MC-Unique: 5NJ-y1XEOa2GaoLjlra03g-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
Date: Tue, 28 Sep 2021 20:22:52 +0200
Message-Id: <20210928182258.12451-3-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Let's simplify return handling.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 arch/x86/xen/mmu_hvm.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/arch/x86/xen/mmu_hvm.c b/arch/x86/xen/mmu_hvm.c
index b242d1f4b426..eb61622df75b 100644
--- a/arch/x86/xen/mmu_hvm.c
+++ b/arch/x86/xen/mmu_hvm.c
@@ -21,23 +21,16 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
 		.domid = DOMID_SELF,
 		.pfn = pfn,
 	};
-	int ram;
 
 	if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
 		return -ENXIO;
 
 	switch (a.mem_type) {
 	case HVMMEM_mmio_dm:
-		ram = 0;
-		break;
-	case HVMMEM_ram_rw:
-	case HVMMEM_ram_ro:
+		return 0;
 	default:
-		ram = 1;
-		break;
+		return 1;
 	}
-
-	return ram;
 }
 #endif
 
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:24:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:24:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198311.351751 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlo-00083b-MO; Tue, 28 Sep 2021 18:24:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198311.351751; Tue, 28 Sep 2021 18:24:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlo-00083U-IS; Tue, 28 Sep 2021 18:24:00 +0000
Received: by outflank-mailman (input) for mailman id 198311;
 Tue, 28 Sep 2021 18:23:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHlm-0007E1-Ul
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:23:58 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 3c09fb0f-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:23:53 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-441-77qRMUygOxyTob3UgChZqw-1; Tue, 28 Sep 2021 14:23:52 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F21C1A40C2;
 Tue, 28 Sep 2021 18:23:49 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 3200E60854;
 Tue, 28 Sep 2021 18:23:41 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c09fb0f-2089-11ec-bcc4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853433;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rRKr51/8zMLS50oc29PqKZsNRavWkybmtHfcIniBEhA=;
	b=bm0FYZtUD4yWki63YmFUxIAN8X3u/zoe3j9b6zhFlBmZP86nQaS4e2xEPzt3bna2UzDjJE
	Pi0tfURFhQr42liWzPfH0EBrhnGuD7yuUO029QhquD6zeAUMsJILhJT+KzJw9Mg7aOLZRM
	s5Tr6jRi8bmquPutYekjrJH7dVsEKTA=
X-MC-Unique: 77qRMUygOxyTob3UgChZqw-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 3/8] proc/vmcore: let pfn_is_ram() return a bool
Date: Tue, 28 Sep 2021 20:22:53 +0200
Message-Id: <20210928182258.12451-4-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

The callback should deal with errors internally, it doesn't make sense to
expose these via pfn_is_ram(). We'll rework the callbacks next. Right now
we consider errors as if "it's RAM"; no functional change.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 fs/proc/vmcore.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
index 9a15334da208..a9bd80ab670e 100644
--- a/fs/proc/vmcore.c
+++ b/fs/proc/vmcore.c
@@ -84,11 +84,11 @@ void unregister_oldmem_pfn_is_ram(void)
 }
 EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
 
-static int pfn_is_ram(unsigned long pfn)
+static bool pfn_is_ram(unsigned long pfn)
 {
 	int (*fn)(unsigned long pfn);
 	/* pfn is ram unless fn() checks pagetype */
-	int ret = 1;
+	bool ret = true;
 
 	/*
 	 * Ask hypervisor if the pfn is really ram.
@@ -97,7 +97,7 @@ static int pfn_is_ram(unsigned long pfn)
 	 */
 	fn = oldmem_pfn_is_ram;
 	if (fn)
-		ret = fn(pfn);
+		ret = !!fn(pfn);
 
 	return ret;
 }
@@ -124,7 +124,7 @@ ssize_t read_from_oldmem(char *buf, size_t count,
 			nr_bytes = count;
 
 		/* If pfn is not ram, return zeros for sparse dump files */
-		if (pfn_is_ram(pfn) == 0)
+		if (!pfn_is_ram(pfn))
 			memset(buf, 0, nr_bytes);
 		else {
 			if (encrypted)
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:24:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:24:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198312.351762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlp-0008Nu-Uy; Tue, 28 Sep 2021 18:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198312.351762; Tue, 28 Sep 2021 18:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHlp-0008Nm-Qo; Tue, 28 Sep 2021 18:24:01 +0000
Received: by outflank-mailman (input) for mailman id 198312;
 Tue, 28 Sep 2021 18:24:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHlo-0007ni-KZ
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:24:00 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 60b7dc57-d8fb-42d8-aaa2-b15f0d1259c7;
 Tue, 28 Sep 2021 18:23:56 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 725F86D;
 Tue, 28 Sep 2021 11:23:56 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 57A203F793;
 Tue, 28 Sep 2021 11:23:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60b7dc57-d8fb-42d8-aaa2-b15f0d1259c7
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 15/17] xen/arm: Transitional change to build HAS_VPCI on ARM.
Date: Tue, 28 Sep 2021 19:18:24 +0100
Message-Id: <adabef3cd1774ca51f1b409bd97a2ae353eaac2b.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

This patch will be reverted once we add support for VPCI MSI/MSIX
support on ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3: none
Change in v2: Patch introduced in v2
---
 xen/drivers/vpci/Makefile | 3 ++-
 xen/drivers/vpci/header.c | 2 ++
 xen/include/asm-arm/pci.h | 8 ++++++++
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
index 55d1bdfda0..1a1413b93e 100644
--- a/xen/drivers/vpci/Makefile
+++ b/xen/drivers/vpci/Makefile
@@ -1 +1,2 @@
-obj-y += vpci.o header.o msi.o msix.o
+obj-y += vpci.o header.o
+obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index ba9a036202..f8cd55e7c0 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
      * FIXME: punching holes after the p2m has been set up might be racy for
      * DomU usage, needs to be revisited.
      */
+#ifdef CONFIG_HAS_PCI_MSI
     if ( map && !rom_only && vpci_make_msix_hole(pdev) )
         return;
+#endif
 
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 49c9622902..5532ce3977 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -26,6 +26,14 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/* Arch-specific MSI data for vPCI. */
+struct vpci_arch_msi {
+};
+
+/* Arch-specific MSI-X entry data for vPCI. */
+struct vpci_arch_msix_entry {
+};
+
 /*
  * struct to hold the mappings of a config space window. This
  * is expected to be used as sysdata for PCI controllers that
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:24:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:24:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198325.351773 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHm8-00012y-Ae; Tue, 28 Sep 2021 18:24:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198325.351773; Tue, 28 Sep 2021 18:24:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHm8-00012r-6l; Tue, 28 Sep 2021 18:24:20 +0000
Received: by outflank-mailman (input) for mailman id 198325;
 Tue, 28 Sep 2021 18:24:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHm6-0000b7-Vy
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:24:19 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 497e2fd0-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:24:15 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 89F666D;
 Tue, 28 Sep 2021 11:24:15 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1F8893F793;
 Tue, 28 Sep 2021 11:24:14 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 497e2fd0-2089-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 16/17] arm/libxl: Emulated PCI device tree node in libxl
Date: Tue, 28 Sep 2021 19:18:25 +0100
Message-Id: <7f19d0802a9ac7544ddefe96f282ba7f97caefe9.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

libxl will create an emulated PCI device tree node in the device tree to
enable the guest OS to discover the virtual PCI during guest boot.
Emulated PCI device tree node will only be created when there is any
device assigned to guest.

A new area has been reserved in the arm guest physical map at
which the VPCI bus is declared in the device tree (reg and ranges
parameters of the node).

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Make GUEST_VPCI_MEM_ADDR address 2MB aligned
Change in v2:
- enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain.
---
 tools/include/libxl.h            |   6 ++
 tools/libs/light/libxl_arm.c     | 105 +++++++++++++++++++++++++++++++
 tools/libs/light/libxl_create.c  |   3 +
 tools/libs/light/libxl_types.idl |   1 +
 tools/xl/xl_parse.c              |   2 +
 xen/include/public/arch-arm.h    |  10 +++
 6 files changed, 127 insertions(+)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d698..3362073b21 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -358,6 +358,12 @@
  */
 #define LIBXL_HAVE_BUILDINFO_ARM_VUART 1
 
+/*
+ * LIBXL_HAVE_BUILDINFO_ARM_VPCI indicates that the toolstack supports virtual
+ * PCI for ARM.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARM_VPCI 1
+
 /*
  * LIBXL_HAVE_BUILDINFO_GRANT_LIMITS indicates that libxl_domain_build_info
  * has the max_grant_frames and max_maptrack_frames fields.
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6e00..52f1ddce48 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
     return fdt_property(fdt, "reg", regs, sizeof(regs));
 }
 
+static int fdt_property_values(libxl__gc *gc, void *fdt,
+        const char *name, unsigned num_cells, ...)
+{
+    uint32_t prop[num_cells];
+    be32 *cells = &prop[0];
+    int i;
+    va_list ap;
+    uint32_t arg;
+
+    va_start(ap, num_cells);
+    for (i = 0 ; i < num_cells; i++) {
+        arg = va_arg(ap, uint32_t);
+        set_cell(&cells, 1, arg);
+    }
+    va_end(ap);
+
+    return fdt_property(fdt, name, prop, sizeof(prop));
+}
+
+static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
+                                    unsigned addr_cells,
+                                    unsigned size_cells,
+                                    unsigned num_regs, ...)
+{
+    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
+    be32 *cells = &regs[0];
+    int i;
+    va_list ap;
+    uint64_t arg;
+
+    va_start(ap, num_regs);
+    for (i = 0 ; i < num_regs; i++) {
+        /* Set the memory bit field */
+        arg = va_arg(ap, uint32_t);
+        set_cell(&cells, 1, arg);
+
+        /* Set the vpci bus address */
+        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
+        set_cell(&cells, addr_cells , arg);
+
+        /* Set the cpu bus address where vpci address is mapped */
+        set_cell(&cells, addr_cells, arg);
+
+        /* Set the vpci size requested */
+        arg = size_cells ? va_arg(ap, uint64_t) : 0;
+        set_cell(&cells, size_cells, arg);
+    }
+    va_end(ap);
+
+    return fdt_property(fdt, "ranges", regs, sizeof(regs));
+}
+
 static int make_root_properties(libxl__gc *gc,
                                 const libxl_version_info *vers,
                                 void *fdt)
@@ -668,6 +720,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+static int make_vpci_node(libxl__gc *gc, void *fdt,
+        const struct arch_info *ainfo,
+        struct xc_dom_image *dom)
+{
+    int res;
+    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
+    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
+    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "device_type", "pci");
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
+    if (res) return res;
+
+    res = fdt_property_values(gc, fdt, "bus-range", 2, 0, 255);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 3);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#size-cells", 2);
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "status", "okay");
+    if (res) return res;
+
+    res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+        GUEST_ROOT_SIZE_CELLS, 2,
+        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE,
+        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
+        GUEST_VPCI_PREFETCH_MEM_SIZE);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
 static const struct arch_info *get_arch_info(libxl__gc *gc,
                                              const struct xc_dom_image *dom)
 {
@@ -971,6 +1070,9 @@ next_resize:
         if (info->tee == LIBXL_TEE_TYPE_OPTEE)
             FDT( make_optee_node(gc, fdt) );
 
+        if (libxl_defbool_val(info->arch_arm.vpci))
+            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
+
         if (pfdt)
             FDT( copy_partial_fdt(gc, fdt, pfdt) );
 
@@ -1189,6 +1291,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
     /* ACPI is disabled by default */
     libxl_defbool_setdefault(&b_info->acpi, false);
 
+    /* VPCI is disabled by default */
+    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
+
     if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
         return;
 
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index e356b2106d..529e475489 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -632,6 +632,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
             create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
 
+        if ( libxl_defbool_val(b_info->arch_arm.vpci) )
+            create.flags |= XEN_DOMCTL_CDF_vpci;
+
         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff653a..78b1ddf0b8 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
+                               ("vpci", libxl_defbool),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd5..ffafbeffb4 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
         }
         if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
             libxl_defbool_set(&b_info->u.pv.e820_host, true);
+        if (d_config->num_pcidevs)
+            libxl_defbool_set(&b_info->arch_arm.vpci, true);
     }
 
     if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 727541a321..acd97eb327 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -431,6 +431,11 @@ typedef uint64_t xen_callback_t;
 #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
 #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
 
+/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
+#define GUEST_VPCI_ADDR_TYPE_MEM            xen_mk_ullong(0x02000000)
+#define GUEST_VPCI_MEM_ADDR                 xen_mk_ullong(0x23000000)
+#define GUEST_VPCI_MEM_SIZE                 xen_mk_ullong(0x10000000)
+
 /*
  * 16MB == 4096 pages reserved for guest to use as a region to map its
  * grant table in.
@@ -446,6 +451,11 @@ typedef uint64_t xen_callback_t;
 #define GUEST_RAM0_BASE   xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1GB */
 #define GUEST_RAM0_SIZE   xen_mk_ullong(0xc0000000)
 
+/* 4GB @ 4GB Prefetch Memory for VPCI */
+#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM   xen_mk_ullong(0x42000000)
+#define GUEST_VPCI_PREFETCH_MEM_ADDR        xen_mk_ullong(0x100000000)
+#define GUEST_VPCI_PREFETCH_MEM_SIZE        xen_mk_ullong(0x100000000)
+
 #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ 8GB */
 #define GUEST_RAM1_SIZE   xen_mk_ullong(0xfe00000000)
 
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:24:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:24:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198336.351784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHma-0001qS-KO; Tue, 28 Sep 2021 18:24:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198336.351784; Tue, 28 Sep 2021 18:24:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHma-0001qL-Gf; Tue, 28 Sep 2021 18:24:48 +0000
Received: by outflank-mailman (input) for mailman id 198336;
 Tue, 28 Sep 2021 18:24:47 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4iZ8=OS=arm.com=rahul.singh@srs-us1.protection.inumbo.net>)
 id 1mVHmZ-0001pr-Fg
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:24:47 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 5bd91d34-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:24:46 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5DC786D;
 Tue, 28 Sep 2021 11:24:46 -0700 (PDT)
Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com
 [10.1.199.1])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5DFCC3F793;
 Tue, 28 Sep 2021 11:24:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5bd91d34-2089-11ec-bcc4-12813bfff9fa
From: Rahul Singh <rahul.singh@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Andre.Przywara@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v3 17/17] xen/arm: Add linux,pci-domain property for hwdom if not available.
Date: Tue, 28 Sep 2021 19:18:26 +0100
Message-Id: <f290d8e2f9b3780c5a956a8032aba6f35de83a90.1632847120.git.rahul.singh@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
In-Reply-To: <cover.1632847120.git.rahul.singh@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>

If the property is not present in the device tree node for host bridge,
XEN while creating the dtb for hwdom will create this property and
assigns the already allocated segment to the host bridge
so that XEN and linux will have the same segment for the host bridges.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
Change in v3:
- Use is_pci_passthrough_enabled()
Change in v2:
- Add linux,pci-domain only when pci-passthrough command line option is enabeld
---
 xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
 xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
 xen/include/asm-arm/pci.h          |  9 +++++++++
 3 files changed, 46 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 1731ae2028..026c9e5c6c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
             return res;
     }
 
+    if ( is_pci_passthrough_enabled() && dt_device_type_is_equal(node, "pci") )
+    {
+        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
+        {
+            uint16_t segment;
+
+            res = pci_get_host_bridge_segment(node, &segment);
+            if ( res < 0 )
+                return res;
+
+            res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment);
+            if ( res )
+                return res;
+        }
+    }
+
     /*
      * Override the property "status" to disable the device when it's
      * marked for passthrough.
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index c5941b10e9..593beeb48c 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -255,6 +255,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
 
     return NULL;
 }
+
+/*
+ * This function will lookup an hostbridge based on config space address.
+ */
+int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                uint16_t *segment)
+{
+    struct pci_host_bridge *bridge;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        if ( bridge->dt_node != node )
+            continue;
+
+        *segment = bridge->segment;
+        return 0;
+    }
+
+    return -EINVAL;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 5532ce3977..7cb2e2f1ed 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -90,6 +90,8 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                pci_sbdf_t sbdf, uint32_t where);
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
+int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                uint16_t *segment);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
@@ -104,5 +106,12 @@ static always_inline bool is_pci_passthrough_enabled(void)
     return false;
 }
 
+static inline int pci_get_host_bridge_segment(const struct dt_device_node *node,
+                                              uint16_t *segment)
+{
+    ASSERT_UNREACHABLE();
+    return -EINVAL;
+}
+
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:25:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:25:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198345.351795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHnA-0002c9-2u; Tue, 28 Sep 2021 18:25:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198345.351795; Tue, 28 Sep 2021 18:25:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHn9-0002c2-Um; Tue, 28 Sep 2021 18:25:23 +0000
Received: by outflank-mailman (input) for mailman id 198345;
 Tue, 28 Sep 2021 18:25:23 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHn8-0002MP-Va
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:25:23 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.129.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 68e8b53e-2089-11ec-bcc4-12813bfff9fa;
 Tue, 28 Sep 2021 18:25:08 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-27-tfQVnEQpPz2XZ-E6EUT2ZA-1; Tue, 28 Sep 2021 14:25:06 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 57A0C824FA8;
 Tue, 28 Sep 2021 18:25:04 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id D16BB60854;
 Tue, 28 Sep 2021 18:24:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68e8b53e-2089-11ec-bcc4-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853508;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nJ4bVTfY2RBZ7dKHJReSBumHsyhYfakPMn9Oqd8JPC0=;
	b=etqVQ8C2Y8Dqe5zlDHBVyetBME4Hm9bFB3v+8JFC7aq8+e2ViAtogD7w7qbNiaXRBHyM3s
	96OBiBGVN9l9xmQ/qrq372XN0gv13Y10H5K7CW41ODSqj3wPj9pcQhvhoHQzrX57GIY9mK
	2qa5462w6tJhf/+r/lcb1evdglxBmUE=
X-MC-Unique: tfQVnEQpPz2XZ-E6EUT2ZA-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 6/8] virtio-mem: factor out hotplug specifics from virtio_mem_probe() into virtio_mem_init_hotplug()
Date: Tue, 28 Sep 2021 20:22:56 +0200
Message-Id: <20210928182258.12451-7-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Let's prepare for a new virtio-mem kdump mode in which we don't actually
hot(un)plug any memory but only observe the state of device blocks.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 drivers/virtio/virtio_mem.c | 87 +++++++++++++++++++------------------
 1 file changed, 45 insertions(+), 42 deletions(-)

diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
index 2ba7e8d6ba8d..1be3ee7f684d 100644
--- a/drivers/virtio/virtio_mem.c
+++ b/drivers/virtio/virtio_mem.c
@@ -260,6 +260,8 @@ static void virtio_mem_fake_offline_going_offline(unsigned long pfn,
 static void virtio_mem_fake_offline_cancel_offline(unsigned long pfn,
 						   unsigned long nr_pages);
 static void virtio_mem_retry(struct virtio_mem *vm);
+static int virtio_mem_create_resource(struct virtio_mem *vm);
+static void virtio_mem_delete_resource(struct virtio_mem *vm);
 
 /*
  * Register a virtio-mem device so it will be considered for the online_page
@@ -2395,7 +2397,8 @@ static int virtio_mem_init_vq(struct virtio_mem *vm)
 static int virtio_mem_init_hotplug(struct virtio_mem *vm)
 {
 	const struct range pluggable_range = mhp_get_pluggable_range(true);
-	uint64_t sb_size, addr;
+	uint64_t unit_pages, sb_size, addr;
+	int rc;
 
 	/* bad device setup - warn only */
 	if (!IS_ALIGNED(vm->addr, memory_block_size_bytes()))
@@ -2474,7 +2477,48 @@ static int virtio_mem_init_hotplug(struct virtio_mem *vm)
 		dev_info(&vm->vdev->dev, "big block size: 0x%llx",
 			 (unsigned long long)vm->bbm.bb_size);
 
+	/* create the parent resource for all memory */
+	rc = virtio_mem_create_resource(vm);
+	if (rc)
+		return rc;
+
+	/* use a single dynamic memory group to cover the whole memory device */
+	if (vm->in_sbm)
+		unit_pages = PHYS_PFN(memory_block_size_bytes());
+	else
+		unit_pages = PHYS_PFN(vm->bbm.bb_size);
+	rc = memory_group_register_dynamic(vm->nid, unit_pages);
+	if (rc < 0)
+		goto out_del_resource;
+	vm->mgid = rc;
+
+	/*
+	 * If we still have memory plugged, we have to unplug all memory first.
+	 * Registering our parent resource makes sure that this memory isn't
+	 * actually in use (e.g., trying to reload the driver).
+	 */
+	if (vm->plugged_size) {
+		vm->unplug_all_required = true;
+		dev_info(&vm->vdev->dev, "unplugging all memory is required\n");
+	}
+
+	/* register callbacks */
+	vm->memory_notifier.notifier_call = virtio_mem_memory_notifier_cb;
+	rc = register_memory_notifier(&vm->memory_notifier);
+	if (rc)
+		goto out_unreg_group;
+	rc = register_virtio_mem_device(vm);
+	if (rc)
+		goto out_unreg_mem;
+
 	return 0;
+out_unreg_mem:
+	unregister_memory_notifier(&vm->memory_notifier);
+out_unreg_group:
+	memory_group_unregister(vm->mgid);
+out_del_resource:
+	virtio_mem_delete_resource(vm);
+	return rc;
 }
 
 static int virtio_mem_init(struct virtio_mem *vm)
@@ -2578,7 +2622,6 @@ static bool virtio_mem_has_memory_added(struct virtio_mem *vm)
 static int virtio_mem_probe(struct virtio_device *vdev)
 {
 	struct virtio_mem *vm;
-	uint64_t unit_pages;
 	int rc;
 
 	BUILD_BUG_ON(sizeof(struct virtio_mem_req) != 24);
@@ -2608,40 +2651,6 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 	if (rc)
 		goto out_del_vq;
 
-	/* create the parent resource for all memory */
-	rc = virtio_mem_create_resource(vm);
-	if (rc)
-		goto out_del_vq;
-
-	/* use a single dynamic memory group to cover the whole memory device */
-	if (vm->in_sbm)
-		unit_pages = PHYS_PFN(memory_block_size_bytes());
-	else
-		unit_pages = PHYS_PFN(vm->bbm.bb_size);
-	rc = memory_group_register_dynamic(vm->nid, unit_pages);
-	if (rc < 0)
-		goto out_del_resource;
-	vm->mgid = rc;
-
-	/*
-	 * If we still have memory plugged, we have to unplug all memory first.
-	 * Registering our parent resource makes sure that this memory isn't
-	 * actually in use (e.g., trying to reload the driver).
-	 */
-	if (vm->plugged_size) {
-		vm->unplug_all_required = true;
-		dev_info(&vm->vdev->dev, "unplugging all memory is required\n");
-	}
-
-	/* register callbacks */
-	vm->memory_notifier.notifier_call = virtio_mem_memory_notifier_cb;
-	rc = register_memory_notifier(&vm->memory_notifier);
-	if (rc)
-		goto out_unreg_group;
-	rc = register_virtio_mem_device(vm);
-	if (rc)
-		goto out_unreg_mem;
-
 	virtio_device_ready(vdev);
 
 	/* trigger a config update to start processing the requested_size */
@@ -2649,12 +2658,6 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 	queue_work(system_freezable_wq, &vm->wq);
 
 	return 0;
-out_unreg_mem:
-	unregister_memory_notifier(&vm->memory_notifier);
-out_unreg_group:
-	memory_group_unregister(vm->mgid);
-out_del_resource:
-	virtio_mem_delete_resource(vm);
 out_del_vq:
 	vdev->config->del_vqs(vdev);
 out_free_vm:
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:26:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:26:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198348.351806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHnz-0003HE-Bc; Tue, 28 Sep 2021 18:26:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198348.351806; Tue, 28 Sep 2021 18:26:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHnz-0003H7-8C; Tue, 28 Sep 2021 18:26:15 +0000
Received: by outflank-mailman (input) for mailman id 198348;
 Tue, 28 Sep 2021 18:26:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHnx-0003Ga-8U
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:26:13 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id f078858f-39f8-4a43-a981-d574e6b88e2a;
 Tue, 28 Sep 2021 18:26:11 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-18-4ipKgBKjNMS9GXMmOtGfLA-1; Tue, 28 Sep 2021 14:26:09 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 690D3A40C1;
 Tue, 28 Sep 2021 18:26:07 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 893F260854;
 Tue, 28 Sep 2021 18:25:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f078858f-39f8-4a43-a981-d574e6b88e2a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853571;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uzlLhr7xjIoW1XJLU3ED3fLv2V7GEnOWprvFn/XGCFI=;
	b=f/2+TviPXKP4Dumks4c2kr57KjcyuUDkfA8hljxIS8xn26gmGpFxiHDM3n8bQ5EfjxqQDf
	QFOR2ygalqXi77BcgVd5jk+5KYApuTEvCLfb1so1gGuNojAPOVnUAxNPj+MO8dZ+ydfJxS
	DA+o3iqTxjxJyIF1mm3xIYeGN8rzbVo=
X-MC-Unique: 4ipKgBKjNMS9GXMmOtGfLA-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 8/8] virtio-mem: kdump mode to sanitize /proc/vmcore access
Date: Tue, 28 Sep 2021 20:22:58 +0200
Message-Id: <20210928182258.12451-9-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Although virtio-mem currently supports reading unplugged memory in the
hypervisor, this will change in the future, indicated to the device via
a new feature flag. We similarly sanitized /proc/kcore access recently. [1]

Let's register a vmcore callback, to allow vmcore code to check if a PFN
belonging to a virtio-mem device is either currently plugged and should
be dumped or is currently unplugged and should not be accessed, instead
mapping the shared zeropage or returning zeroes when reading.

This is important when not capturing /proc/vmcore via tools like
"makedumpfile" that can identify logically unplugged virtio-mem memory via
PG_offline in the memmap, but simply by e.g., copying the file.

Distributions that support virtio-mem+kdump have to make sure that the
virtio_mem module will be part of the kdump kernel or the kdump initrd;
dracut was recently [2] extended to include virtio-mem in the generated
initrd. As long as no special kdump kernels are used, this will
automatically make sure that virtio-mem will be around in the kdump initrd
and sanitize /proc/vmcore access -- with dracut.

With this series, we'll send one virtio-mem state request for every
~2 MiB chunk of virtio-mem memory indicated in the vmcore that we intend
to read/map.

In the future, we might want to allow building virtio-mem for kdump
mode only, even without CONFIG_MEMORY_HOTPLUG and friends: this way,
we could support special stripped-down kdump kernels that have many
other config options disabled; we'll tackle that once required. Further,
we might want to try sensing bigger blocks (e.g., memory sections)
first before falling back to device blocks on demand.

Tested with Fedora rawhide, which contains a recent kexec-tools version
(considering "System RAM (virtio_mem)" when creating the vmcore header) and
a recent dracut version (including the virtio_mem module in the kdump
initrd).

[1] https://lkml.kernel.org/r/20210526093041.8800-1-david@redhat.com
[2] https://github.com/dracutdevs/dracut/pull/1157

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 drivers/virtio/virtio_mem.c | 136 ++++++++++++++++++++++++++++++++----
 1 file changed, 124 insertions(+), 12 deletions(-)

diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
index 76d8aef3cfd2..ec0b2ab37acb 100644
--- a/drivers/virtio/virtio_mem.c
+++ b/drivers/virtio/virtio_mem.c
@@ -223,6 +223,9 @@ struct virtio_mem {
 	 * When this lock is held the pointers can't change, ONLINE and
 	 * OFFLINE blocks can't change the state and no subblocks will get
 	 * plugged/unplugged.
+	 *
+	 * In kdump mode, used to serialize requests, last_block_addr and
+	 * last_block_plugged.
 	 */
 	struct mutex hotplug_mutex;
 	bool hotplug_active;
@@ -230,6 +233,9 @@ struct virtio_mem {
 	/* An error occurred we cannot handle - stop processing requests. */
 	bool broken;
 
+	/* Cached valued of is_kdump_kernel() when the device was probed. */
+	bool in_kdump;
+
 	/* The driver is being removed. */
 	spinlock_t removal_lock;
 	bool removing;
@@ -243,6 +249,13 @@ struct virtio_mem {
 	/* Memory notifier (online/offline events). */
 	struct notifier_block memory_notifier;
 
+#ifdef CONFIG_PROC_VMCORE
+	/* vmcore callback for /proc/vmcore handling in kdump mode */
+	struct vmcore_cb vmcore_cb;
+	uint64_t last_block_addr;
+	bool last_block_plugged;
+#endif /* CONFIG_PROC_VMCORE */
+
 	/* Next device in the list of virtio-mem devices. */
 	struct list_head next;
 };
@@ -2293,6 +2306,12 @@ static void virtio_mem_run_wq(struct work_struct *work)
 	uint64_t diff;
 	int rc;
 
+	if (unlikely(vm->in_kdump)) {
+		dev_warn_once(&vm->vdev->dev,
+			     "unexpected workqueue run in kdump kernel\n");
+		return;
+	}
+
 	hrtimer_cancel(&vm->retry_timer);
 
 	if (vm->broken)
@@ -2521,6 +2540,86 @@ static int virtio_mem_init_hotplug(struct virtio_mem *vm)
 	return rc;
 }
 
+#ifdef CONFIG_PROC_VMCORE
+static int virtio_mem_send_state_request(struct virtio_mem *vm, uint64_t addr,
+					 uint64_t size)
+{
+	const uint64_t nb_vm_blocks = size / vm->device_block_size;
+	const struct virtio_mem_req req = {
+		.type = cpu_to_virtio16(vm->vdev, VIRTIO_MEM_REQ_STATE),
+		.u.state.addr = cpu_to_virtio64(vm->vdev, addr),
+		.u.state.nb_blocks = cpu_to_virtio16(vm->vdev, nb_vm_blocks),
+	};
+	int rc = -ENOMEM;
+
+	dev_dbg(&vm->vdev->dev, "requesting state: 0x%llx - 0x%llx\n", addr,
+		addr + size - 1);
+
+	switch (virtio_mem_send_request(vm, &req)) {
+	case VIRTIO_MEM_RESP_ACK:
+		return virtio16_to_cpu(vm->vdev, vm->resp.u.state.state);
+	case VIRTIO_MEM_RESP_ERROR:
+		rc = -EINVAL;
+		break;
+	default:
+		break;
+	}
+
+	dev_dbg(&vm->vdev->dev, "requesting state failed: %d\n", rc);
+	return rc;
+}
+
+static bool virtio_mem_vmcore_pfn_is_ram(struct vmcore_cb *cb,
+					 unsigned long pfn)
+{
+	struct virtio_mem *vm = container_of(cb, struct virtio_mem,
+					     vmcore_cb);
+	uint64_t addr = PFN_PHYS(pfn);
+	bool is_ram;
+	int rc;
+
+	if (!virtio_mem_contains_range(vm, addr, addr + PAGE_SIZE))
+		return true;
+	if (!vm->plugged_size)
+		return false;
+
+	/*
+	 * We have to serialize device requests and access to the information
+	 * about the block queried last.
+	 */
+	mutex_lock(&vm->hotplug_mutex);
+
+	addr = ALIGN_DOWN(addr, vm->device_block_size);
+	if (addr != vm->last_block_addr) {
+		rc = virtio_mem_send_state_request(vm, addr,
+						   vm->device_block_size);
+		/* On any kind of error, we're going to signal !ram. */
+		if (rc == VIRTIO_MEM_STATE_PLUGGED)
+			vm->last_block_plugged = true;
+		else
+			vm->last_block_plugged = false;
+		vm->last_block_addr = addr;
+	}
+
+	is_ram = vm->last_block_plugged;
+	mutex_unlock(&vm->hotplug_mutex);
+	return is_ram;
+}
+#endif /* CONFIG_PROC_VMCORE */
+
+static int virtio_mem_init_kdump(struct virtio_mem *vm)
+{
+#ifdef CONFIG_PROC_VMCORE
+	dev_info(&vm->vdev->dev, "memory hot(un)plug disabled in kdump kernel\n");
+	vm->vmcore_cb.pfn_is_ram = virtio_mem_vmcore_pfn_is_ram;
+	register_vmcore_cb(&vm->vmcore_cb);
+	return 0;
+#else /* CONFIG_PROC_VMCORE */
+	dev_warn(&vm->vdev->dev, "disabled in kdump kernel without vmcore\n");
+	return -EBUSY;
+#endif /* CONFIG_PROC_VMCORE */
+}
+
 static int virtio_mem_init(struct virtio_mem *vm)
 {
 	uint16_t node_id;
@@ -2530,15 +2629,6 @@ static int virtio_mem_init(struct virtio_mem *vm)
 		return -EINVAL;
 	}
 
-	/*
-	 * We don't want to (un)plug or reuse any memory when in kdump. The
-	 * memory is still accessible (but not mapped).
-	 */
-	if (is_kdump_kernel()) {
-		dev_warn(&vm->vdev->dev, "disabled in kdump kernel\n");
-		return -EBUSY;
-	}
-
 	/* Fetch all properties that can't change. */
 	virtio_cread_le(vm->vdev, struct virtio_mem_config, plugged_size,
 			&vm->plugged_size);
@@ -2562,6 +2652,12 @@ static int virtio_mem_init(struct virtio_mem *vm)
 	if (vm->nid != NUMA_NO_NODE && IS_ENABLED(CONFIG_NUMA))
 		dev_info(&vm->vdev->dev, "nid: %d", vm->nid);
 
+	/*
+	 * We don't want to (un)plug or reuse any memory when in kdump. The
+	 * memory is still accessible (but not exposed to Linux).
+	 */
+	if (vm->in_kdump)
+		return virtio_mem_init_kdump(vm);
 	return virtio_mem_init_hotplug(vm);
 }
 
@@ -2640,6 +2736,7 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 	hrtimer_init(&vm->retry_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
 	vm->retry_timer.function = virtio_mem_timer_expired;
 	vm->retry_timer_ms = VIRTIO_MEM_RETRY_TIMER_MIN_MS;
+	vm->in_kdump = is_kdump_kernel();
 
 	/* register the virtqueue */
 	rc = virtio_mem_init_vq(vm);
@@ -2654,8 +2751,10 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 	virtio_device_ready(vdev);
 
 	/* trigger a config update to start processing the requested_size */
-	atomic_set(&vm->config_changed, 1);
-	queue_work(system_freezable_wq, &vm->wq);
+	if (!vm->in_kdump) {
+		atomic_set(&vm->config_changed, 1);
+		queue_work(system_freezable_wq, &vm->wq);
+	}
 
 	return 0;
 out_del_vq:
@@ -2732,11 +2831,21 @@ static void virtio_mem_deinit_hotplug(struct virtio_mem *vm)
 	}
 }
 
+static void virtio_mem_deinit_kdump(struct virtio_mem *vm)
+{
+#ifdef CONFIG_PROC_VMCORE
+	unregister_vmcore_cb(&vm->vmcore_cb);
+#endif /* CONFIG_PROC_VMCORE */
+}
+
 static void virtio_mem_remove(struct virtio_device *vdev)
 {
 	struct virtio_mem *vm = vdev->priv;
 
-	virtio_mem_deinit_hotplug(vm);
+	if (vm->in_kdump)
+		virtio_mem_deinit_kdump(vm);
+	else
+		virtio_mem_deinit_hotplug(vm);
 
 	/* reset the device and cleanup the queues */
 	vdev->config->reset(vdev);
@@ -2750,6 +2859,9 @@ static void virtio_mem_config_changed(struct virtio_device *vdev)
 {
 	struct virtio_mem *vm = vdev->priv;
 
+	if (unlikely(vm->in_kdump))
+		return;
+
 	atomic_set(&vm->config_changed, 1);
 	virtio_mem_retry(vm);
 }
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:27:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:27:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198358.351817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHpE-00040M-Mo; Tue, 28 Sep 2021 18:27:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198358.351817; Tue, 28 Sep 2021 18:27:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHpE-00040F-JT; Tue, 28 Sep 2021 18:27:32 +0000
Received: by outflank-mailman (input) for mailman id 198358;
 Tue, 28 Sep 2021 18:27:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHmc-0007ni-MI
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:24:50 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 1c185ed4-4f73-4c6c-bb5b-2b88a20728ba;
 Tue, 28 Sep 2021 18:24:10 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-449-rpm4hXWDPGGhvyyl4qPVLg-1; Tue, 28 Sep 2021 14:24:08 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F05E41B2C980;
 Tue, 28 Sep 2021 18:24:05 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 6147C60854;
 Tue, 28 Sep 2021 18:23:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c185ed4-4f73-4c6c-bb5b-2b88a20728ba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853449;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=YxVxDaMF8Dvr5dJnhnKtXBDkasxDLaxdtEfkC3HanCE=;
	b=fMzU62lrwd+Yy9xtxXsot4PJFkC5SzASmXH8wLg35TY8j9GvgYOC/v/IhuwUsVIKi/1t95
	oY54T9sno+VXRkBACZcnW0+wFM4GeQ4Z9yqlxMZGRYRQjP4Fd1bH4l1Z+zUSQNzljDYHnW
	U7za+EJ2Qve+lppsIFlqoDCR0W/ANNk=
X-MC-Unique: rpm4hXWDPGGhvyyl4qPVLg-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 4/8] proc/vmcore: convert oldmem_pfn_is_ram callback to more generic vmcore callbacks
Date: Tue, 28 Sep 2021 20:22:54 +0200
Message-Id: <20210928182258.12451-5-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Let's support multiple registered callbacks, making sure that
registering vmcore callbacks cannot fail. Make the callback return a
bool instead of an int, handling how to deal with errors internally.
Drop unused HAVE_OLDMEM_PFN_IS_RAM.

We soon want to make use of this infrastructure from other drivers:
virtio-mem, registering one callback for each virtio-mem device, to
prevent reading unplugged virtio-mem memory.

Handle it via a generic vmcore_cb structure, prepared for future
extensions: for example, once we support virtio-mem on s390x where the
vmcore is completely constructed in the second kernel, we want to detect
and add plugged virtio-mem memory ranges to the vmcore in order for them
to get dumped properly.

Handle corner cases that are unexpected and shouldn't happen in sane
setups: registering a callback after the vmcore has already been opened
(warn only) and unregistering a callback after the vmcore has already been
opened (warn and essentially read only zeroes from that point on).

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 arch/x86/kernel/aperture_64.c | 13 ++++-
 arch/x86/xen/mmu_hvm.c        | 15 +++---
 fs/proc/vmcore.c              | 99 ++++++++++++++++++++++++-----------
 include/linux/crash_dump.h    | 26 +++++++--
 4 files changed, 113 insertions(+), 40 deletions(-)

diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c
index 10562885f5fc..af3ba08b684b 100644
--- a/arch/x86/kernel/aperture_64.c
+++ b/arch/x86/kernel/aperture_64.c
@@ -73,12 +73,23 @@ static int gart_mem_pfn_is_ram(unsigned long pfn)
 		      (pfn >= aperture_pfn_start + aperture_page_count));
 }
 
+#ifdef CONFIG_PROC_VMCORE
+static bool gart_oldmem_pfn_is_ram(struct vmcore_cb *cb, unsigned long pfn)
+{
+	return !!gart_mem_pfn_is_ram(pfn);
+}
+
+static struct vmcore_cb gart_vmcore_cb = {
+	.pfn_is_ram = gart_oldmem_pfn_is_ram,
+};
+#endif
+
 static void __init exclude_from_core(u64 aper_base, u32 aper_order)
 {
 	aperture_pfn_start = aper_base >> PAGE_SHIFT;
 	aperture_page_count = (32 * 1024 * 1024) << aper_order >> PAGE_SHIFT;
 #ifdef CONFIG_PROC_VMCORE
-	WARN_ON(register_oldmem_pfn_is_ram(&gart_mem_pfn_is_ram));
+	register_vmcore_cb(&gart_vmcore_cb);
 #endif
 #ifdef CONFIG_PROC_KCORE
 	WARN_ON(register_mem_pfn_is_ram(&gart_mem_pfn_is_ram));
diff --git a/arch/x86/xen/mmu_hvm.c b/arch/x86/xen/mmu_hvm.c
index eb61622df75b..49bd4a6a5858 100644
--- a/arch/x86/xen/mmu_hvm.c
+++ b/arch/x86/xen/mmu_hvm.c
@@ -12,10 +12,10 @@
  * The kdump kernel has to check whether a pfn of the crashed kernel
  * was a ballooned page. vmcore is using this function to decide
  * whether to access a pfn of the crashed kernel.
- * Returns 0 if the pfn is not backed by a RAM page, the caller may
+ * Returns "false" if the pfn is not backed by a RAM page, the caller may
  * handle the pfn special in this case.
  */
-static int xen_oldmem_pfn_is_ram(unsigned long pfn)
+static bool xen_vmcore_pfn_is_ram(struct vmcore_cb *cb, unsigned long pfn)
 {
 	struct xen_hvm_get_mem_type a = {
 		.domid = DOMID_SELF,
@@ -23,15 +23,18 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
 	};
 
 	if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
-		return -ENXIO;
+		return true;
 
 	switch (a.mem_type) {
 	case HVMMEM_mmio_dm:
-		return 0;
+		return false;
 	default:
-		return 1;
+		return true;
 	}
 }
+static struct vmcore_cb xen_vmcore_cb = {
+	.pfn_is_ram = xen_vmcore_pfn_is_ram,
+};
 #endif
 
 static void xen_hvm_exit_mmap(struct mm_struct *mm)
@@ -65,6 +68,6 @@ void __init xen_hvm_init_mmu_ops(void)
 	if (is_pagetable_dying_supported())
 		pv_ops.mmu.exit_mmap = xen_hvm_exit_mmap;
 #ifdef CONFIG_PROC_VMCORE
-	WARN_ON(register_oldmem_pfn_is_ram(&xen_oldmem_pfn_is_ram));
+	register_vmcore_cb(&xen_vmcore_cb);
 #endif
 }
diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
index a9bd80ab670e..7a04b2eca287 100644
--- a/fs/proc/vmcore.c
+++ b/fs/proc/vmcore.c
@@ -62,46 +62,75 @@ core_param(novmcoredd, vmcoredd_disabled, bool, 0);
 /* Device Dump Size */
 static size_t vmcoredd_orig_sz;
 
-/*
- * Returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error
- * The called function has to take care of module refcounting.
- */
-static int (*oldmem_pfn_is_ram)(unsigned long pfn);
-
-int register_oldmem_pfn_is_ram(int (*fn)(unsigned long pfn))
+static DECLARE_RWSEM(vmcore_cb_rwsem);
+/* List of registered vmcore callbacks. */
+static LIST_HEAD(vmcore_cb_list);
+/* Whether we had a surprise unregistration of a callback. */
+static bool vmcore_cb_unstable;
+/* Whether the vmcore has been opened once. */
+static bool vmcore_opened;
+
+void register_vmcore_cb(struct vmcore_cb *cb)
 {
-	if (oldmem_pfn_is_ram)
-		return -EBUSY;
-	oldmem_pfn_is_ram = fn;
-	return 0;
+	down_write(&vmcore_cb_rwsem);
+	INIT_LIST_HEAD(&cb->next);
+	list_add_tail(&cb->next, &vmcore_cb_list);
+	/*
+	 * Registering a vmcore callback after the vmcore was opened is
+	 * very unusual (e.g., manual driver loading).
+	 */
+	if (vmcore_opened)
+		pr_warn_once("Unexpected vmcore callback registration\n");
+	up_write(&vmcore_cb_rwsem);
 }
-EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
+EXPORT_SYMBOL_GPL(register_vmcore_cb);
 
-void unregister_oldmem_pfn_is_ram(void)
+void unregister_vmcore_cb(struct vmcore_cb *cb)
 {
-	oldmem_pfn_is_ram = NULL;
-	wmb();
+	down_write(&vmcore_cb_rwsem);
+	list_del(&cb->next);
+	/*
+	 * Unregistering a vmcore callback after the vmcore was opened is
+	 * very unusual (e.g., forced driver removal), but we cannot stop
+	 * unregistering.
+	 */
+	if (vmcore_opened) {
+		pr_warn_once("Unexpected vmcore callback unregistration\n");
+		vmcore_cb_unstable = true;
+	}
+	up_write(&vmcore_cb_rwsem);
 }
-EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
+EXPORT_SYMBOL_GPL(unregister_vmcore_cb);
 
 static bool pfn_is_ram(unsigned long pfn)
 {
-	int (*fn)(unsigned long pfn);
-	/* pfn is ram unless fn() checks pagetype */
+	struct vmcore_cb *cb;
 	bool ret = true;
 
-	/*
-	 * Ask hypervisor if the pfn is really ram.
-	 * A ballooned page contains no data and reading from such a page
-	 * will cause high load in the hypervisor.
-	 */
-	fn = oldmem_pfn_is_ram;
-	if (fn)
-		ret = !!fn(pfn);
+	lockdep_assert_held_read(&vmcore_cb_rwsem);
+	if (unlikely(vmcore_cb_unstable))
+		return false;
+
+	list_for_each_entry(cb, &vmcore_cb_list, next) {
+		if (unlikely(!cb->pfn_is_ram))
+			continue;
+		ret = cb->pfn_is_ram(cb, pfn);
+		if (!ret)
+			break;
+	}
 
 	return ret;
 }
 
+static int open_vmcore(struct inode *inode, struct file *file)
+{
+	down_read(&vmcore_cb_rwsem);
+	vmcore_opened = true;
+	up_read(&vmcore_cb_rwsem);
+
+	return 0;
+}
+
 /* Reads a page from the oldmem device from given offset. */
 ssize_t read_from_oldmem(char *buf, size_t count,
 			 u64 *ppos, int userbuf,
@@ -117,6 +146,7 @@ ssize_t read_from_oldmem(char *buf, size_t count,
 	offset = (unsigned long)(*ppos % PAGE_SIZE);
 	pfn = (unsigned long)(*ppos / PAGE_SIZE);
 
+	down_read(&vmcore_cb_rwsem);
 	do {
 		if (count > (PAGE_SIZE - offset))
 			nr_bytes = PAGE_SIZE - offset;
@@ -136,8 +166,10 @@ ssize_t read_from_oldmem(char *buf, size_t count,
 				tmp = copy_oldmem_page(pfn, buf, nr_bytes,
 						       offset, userbuf);
 
-			if (tmp < 0)
+			if (tmp < 0) {
+				up_read(&vmcore_cb_rwsem);
 				return tmp;
+			}
 		}
 		*ppos += nr_bytes;
 		count -= nr_bytes;
@@ -147,6 +179,7 @@ ssize_t read_from_oldmem(char *buf, size_t count,
 		offset = 0;
 	} while (count);
 
+	up_read(&vmcore_cb_rwsem);
 	return read;
 }
 
@@ -537,14 +570,19 @@ static int vmcore_remap_oldmem_pfn(struct vm_area_struct *vma,
 			    unsigned long from, unsigned long pfn,
 			    unsigned long size, pgprot_t prot)
 {
+	int ret;
+
 	/*
 	 * Check if oldmem_pfn_is_ram was registered to avoid
 	 * looping over all pages without a reason.
 	 */
-	if (oldmem_pfn_is_ram)
-		return remap_oldmem_pfn_checked(vma, from, pfn, size, prot);
+	down_read(&vmcore_cb_rwsem);
+	if (!list_empty(&vmcore_cb_list) || vmcore_cb_unstable)
+		ret = remap_oldmem_pfn_checked(vma, from, pfn, size, prot);
 	else
-		return remap_oldmem_pfn_range(vma, from, pfn, size, prot);
+		ret = remap_oldmem_pfn_range(vma, from, pfn, size, prot);
+	up_read(&vmcore_cb_rwsem);
+	return ret;
 }
 
 static int mmap_vmcore(struct file *file, struct vm_area_struct *vma)
@@ -668,6 +706,7 @@ static int mmap_vmcore(struct file *file, struct vm_area_struct *vma)
 #endif
 
 static const struct proc_ops vmcore_proc_ops = {
+	.proc_open	= open_vmcore,
 	.proc_read	= read_vmcore,
 	.proc_lseek	= default_llseek,
 	.proc_mmap	= mmap_vmcore,
diff --git a/include/linux/crash_dump.h b/include/linux/crash_dump.h
index 2618577a4d6d..0c547d866f1e 100644
--- a/include/linux/crash_dump.h
+++ b/include/linux/crash_dump.h
@@ -91,9 +91,29 @@ static inline void vmcore_unusable(void)
 		elfcorehdr_addr = ELFCORE_ADDR_ERR;
 }
 
-#define HAVE_OLDMEM_PFN_IS_RAM 1
-extern int register_oldmem_pfn_is_ram(int (*fn)(unsigned long pfn));
-extern void unregister_oldmem_pfn_is_ram(void);
+/**
+ * struct vmcore_cb - driver callbacks for /proc/vmcore handling
+ * @pfn_is_ram: check whether a PFN really is RAM and should be accessed when
+ *              reading the vmcore. Will return "true" if it is RAM or if the
+ *              callback cannot tell. If any callback returns "false", it's not
+ *              RAM and the page must not be accessed; zeroes should be
+ *              indicated in the vmcore instead. For example, a ballooned page
+ *              contains no data and reading from such a page will cause high
+ *              load in the hypervisor.
+ * @next: List head to manage registered callbacks internally; initialized by
+ *        register_vmcore_cb().
+ *
+ * vmcore callbacks allow drivers managing physical memory ranges to
+ * coordinate with vmcore handling code, for example, to prevent accessing
+ * physical memory ranges that should not be accessed when reading the vmcore,
+ * although included in the vmcore header as memory ranges to dump.
+ */
+struct vmcore_cb {
+	bool (*pfn_is_ram)(struct vmcore_cb *cb, unsigned long pfn);
+	struct list_head next;
+};
+extern void register_vmcore_cb(struct vmcore_cb *cb);
+extern void unregister_vmcore_cb(struct vmcore_cb *cb);
 
 #else /* !CONFIG_CRASH_DUMP */
 static inline bool is_kdump_kernel(void) { return 0; }
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:27:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:27:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198364.351828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHpa-0004gt-4V; Tue, 28 Sep 2021 18:27:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198364.351828; Tue, 28 Sep 2021 18:27:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHpa-0004gm-0Y; Tue, 28 Sep 2021 18:27:54 +0000
Received: by outflank-mailman (input) for mailman id 198364;
 Tue, 28 Sep 2021 18:27:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHn1-0007ni-Mg
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:25:15 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 4af91939-3118-4460-a099-6c4ad581574e;
 Tue, 28 Sep 2021 18:24:30 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-516-hoClpQ93OYSgO_fkG_Ydfw-1; Tue, 28 Sep 2021 14:24:28 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 711ED1023F4F;
 Tue, 28 Sep 2021 18:24:26 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 73B1260854;
 Tue, 28 Sep 2021 18:24:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4af91939-3118-4460-a099-6c4ad581574e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853469;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UbXdn+v660VV5bsfwvBwOBATR8SkkHZUOKA5MNOOqCo=;
	b=Neq77v/UWED9mxC/TuCedK0kwo68aPVlZxHOpCr2R67FBa1GtapbWEic2a+03weAH+FBWw
	QW4e0re6EbMHkzAcyxDicq36PYgcsFKbTA5yVBM6IgMyRepxa7MvXNaY/vgOoYIO+qHxGK
	B9eD3Su5oPaoyLpNjd05n9CwsZapieY=
X-MC-Unique: hoClpQ93OYSgO_fkG_Ydfw-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 5/8] virtio-mem: factor out hotplug specifics from virtio_mem_init() into virtio_mem_init_hotplug()
Date: Tue, 28 Sep 2021 20:22:55 +0200
Message-Id: <20210928182258.12451-6-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Let's prepare for a new virtio-mem kdump mode in which we don't actually
hot(un)plug any memory but only observe the state of device blocks.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 drivers/virtio/virtio_mem.c | 81 ++++++++++++++++++++-----------------
 1 file changed, 44 insertions(+), 37 deletions(-)

diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
index bef8ad6bf466..2ba7e8d6ba8d 100644
--- a/drivers/virtio/virtio_mem.c
+++ b/drivers/virtio/virtio_mem.c
@@ -2392,41 +2392,10 @@ static int virtio_mem_init_vq(struct virtio_mem *vm)
 	return 0;
 }
 
-static int virtio_mem_init(struct virtio_mem *vm)
+static int virtio_mem_init_hotplug(struct virtio_mem *vm)
 {
 	const struct range pluggable_range = mhp_get_pluggable_range(true);
 	uint64_t sb_size, addr;
-	uint16_t node_id;
-
-	if (!vm->vdev->config->get) {
-		dev_err(&vm->vdev->dev, "config access disabled\n");
-		return -EINVAL;
-	}
-
-	/*
-	 * We don't want to (un)plug or reuse any memory when in kdump. The
-	 * memory is still accessible (but not mapped).
-	 */
-	if (is_kdump_kernel()) {
-		dev_warn(&vm->vdev->dev, "disabled in kdump kernel\n");
-		return -EBUSY;
-	}
-
-	/* Fetch all properties that can't change. */
-	virtio_cread_le(vm->vdev, struct virtio_mem_config, plugged_size,
-			&vm->plugged_size);
-	virtio_cread_le(vm->vdev, struct virtio_mem_config, block_size,
-			&vm->device_block_size);
-	virtio_cread_le(vm->vdev, struct virtio_mem_config, node_id,
-			&node_id);
-	vm->nid = virtio_mem_translate_node_id(vm, node_id);
-	virtio_cread_le(vm->vdev, struct virtio_mem_config, addr, &vm->addr);
-	virtio_cread_le(vm->vdev, struct virtio_mem_config, region_size,
-			&vm->region_size);
-
-	/* Determine the nid for the device based on the lowest address. */
-	if (vm->nid == NUMA_NO_NODE)
-		vm->nid = memory_add_physaddr_to_nid(vm->addr);
 
 	/* bad device setup - warn only */
 	if (!IS_ALIGNED(vm->addr, memory_block_size_bytes()))
@@ -2496,10 +2465,6 @@ static int virtio_mem_init(struct virtio_mem *vm)
 					      vm->offline_threshold);
 	}
 
-	dev_info(&vm->vdev->dev, "start address: 0x%llx", vm->addr);
-	dev_info(&vm->vdev->dev, "region size: 0x%llx", vm->region_size);
-	dev_info(&vm->vdev->dev, "device block size: 0x%llx",
-		 (unsigned long long)vm->device_block_size);
 	dev_info(&vm->vdev->dev, "memory block size: 0x%lx",
 		 memory_block_size_bytes());
 	if (vm->in_sbm)
@@ -2508,10 +2473,52 @@ static int virtio_mem_init(struct virtio_mem *vm)
 	else
 		dev_info(&vm->vdev->dev, "big block size: 0x%llx",
 			 (unsigned long long)vm->bbm.bb_size);
+
+	return 0;
+}
+
+static int virtio_mem_init(struct virtio_mem *vm)
+{
+	uint16_t node_id;
+
+	if (!vm->vdev->config->get) {
+		dev_err(&vm->vdev->dev, "config access disabled\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * We don't want to (un)plug or reuse any memory when in kdump. The
+	 * memory is still accessible (but not mapped).
+	 */
+	if (is_kdump_kernel()) {
+		dev_warn(&vm->vdev->dev, "disabled in kdump kernel\n");
+		return -EBUSY;
+	}
+
+	/* Fetch all properties that can't change. */
+	virtio_cread_le(vm->vdev, struct virtio_mem_config, plugged_size,
+			&vm->plugged_size);
+	virtio_cread_le(vm->vdev, struct virtio_mem_config, block_size,
+			&vm->device_block_size);
+	virtio_cread_le(vm->vdev, struct virtio_mem_config, node_id,
+			&node_id);
+	vm->nid = virtio_mem_translate_node_id(vm, node_id);
+	virtio_cread_le(vm->vdev, struct virtio_mem_config, addr, &vm->addr);
+	virtio_cread_le(vm->vdev, struct virtio_mem_config, region_size,
+			&vm->region_size);
+
+	/* Determine the nid for the device based on the lowest address. */
+	if (vm->nid == NUMA_NO_NODE)
+		vm->nid = memory_add_physaddr_to_nid(vm->addr);
+
+	dev_info(&vm->vdev->dev, "start address: 0x%llx", vm->addr);
+	dev_info(&vm->vdev->dev, "region size: 0x%llx", vm->region_size);
+	dev_info(&vm->vdev->dev, "device block size: 0x%llx",
+		 (unsigned long long)vm->device_block_size);
 	if (vm->nid != NUMA_NO_NODE && IS_ENABLED(CONFIG_NUMA))
 		dev_info(&vm->vdev->dev, "nid: %d", vm->nid);
 
-	return 0;
+	return virtio_mem_init_hotplug(vm);
 }
 
 static int virtio_mem_create_resource(struct virtio_mem *vm)
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:28:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:28:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198372.351838 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHps-0005GI-Bt; Tue, 28 Sep 2021 18:28:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198372.351838; Tue, 28 Sep 2021 18:28:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVHps-0005GB-92; Tue, 28 Sep 2021 18:28:12 +0000
Received: by outflank-mailman (input) for mailman id 198372;
 Tue, 28 Sep 2021 18:28:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jiAA=OS=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVHna-0007ni-Nl
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 18:25:50 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 6fe75553-7baf-49e2-921e-2b1fb77390a7;
 Tue, 28 Sep 2021 18:25:39 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-97-b_LGb36yMIiFspS9xQbtiA-1; Tue, 28 Sep 2021 14:25:37 -0400
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
 [10.5.11.13])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 507C01018721;
 Tue, 28 Sep 2021 18:25:35 +0000 (UTC)
Received: from t480s.redhat.com (unknown [10.39.194.120])
 by smtp.corp.redhat.com (Postfix) with ESMTP id B481560854;
 Tue, 28 Sep 2021 18:25:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fe75553-7baf-49e2-921e-2b1fb77390a7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632853539;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R7+JUDG91uhwuwaeltTRSiOKiaWOQXIYAMDdQ0A5p1w=;
	b=Kj43dMu7139d57FIT7+X6axopiYK5m05wonndObsxwzICcp6IEp3+sqfdgZC3FLtwrTSOA
	7disbkCL6Vx4DwfSU5b6hcyMUMDdWw+F+aRiFVX13hwludVeQt+PC4IYrVzS9quNqacMtP
	5erP7fWiFNp7Jb3ehKr3SZQ/ZIZ59iA=
X-MC-Unique: b_LGb36yMIiFspS9xQbtiA-1
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jason Wang <jasowang@redhat.com>,
	Dave Young <dyoung@redhat.com>,
	Baoquan He <bhe@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Oscar Salvador <osalvador@suse.de>,
	Mike Rapoport <rppt@kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	x86@kernel.org,
	xen-devel@lists.xenproject.org,
	virtualization@lists.linux-foundation.org,
	kexec@lists.infradead.org,
	linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org
Subject: [PATCH v1 7/8] virtio-mem: factor out hotplug specifics from virtio_mem_remove() into virtio_mem_deinit_hotplug()
Date: Tue, 28 Sep 2021 20:22:57 +0200
Message-Id: <20210928182258.12451-8-david@redhat.com>
In-Reply-To: <20210928182258.12451-1-david@redhat.com>
References: <20210928182258.12451-1-david@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

Let's prepare for a new virtio-mem kdump mode in which we don't actually
hot(un)plug any memory but only observe the state of device blocks.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 drivers/virtio/virtio_mem.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c
index 1be3ee7f684d..76d8aef3cfd2 100644
--- a/drivers/virtio/virtio_mem.c
+++ b/drivers/virtio/virtio_mem.c
@@ -2667,9 +2667,8 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 	return rc;
 }
 
-static void virtio_mem_remove(struct virtio_device *vdev)
+static void virtio_mem_deinit_hotplug(struct virtio_mem *vm)
 {
-	struct virtio_mem *vm = vdev->priv;
 	unsigned long mb_id;
 	int rc;
 
@@ -2716,7 +2715,8 @@ static void virtio_mem_remove(struct virtio_device *vdev)
 	 * away. Warn at least.
 	 */
 	if (virtio_mem_has_memory_added(vm)) {
-		dev_warn(&vdev->dev, "device still has system memory added\n");
+		dev_warn(&vm->vdev->dev,
+			 "device still has system memory added\n");
 	} else {
 		virtio_mem_delete_resource(vm);
 		kfree_const(vm->resource_name);
@@ -2730,6 +2730,13 @@ static void virtio_mem_remove(struct virtio_device *vdev)
 	} else {
 		vfree(vm->bbm.bb_states);
 	}
+}
+
+static void virtio_mem_remove(struct virtio_device *vdev)
+{
+	struct virtio_mem *vm = vdev->priv;
+
+	virtio_mem_deinit_hotplug(vm);
 
 	/* reset the device and cleanup the queues */
 	vdev->config->reset(vdev);
-- 
2.31.1



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:52:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:52:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198386.351850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVIDb-0000m0-DK; Tue, 28 Sep 2021 18:52:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198386.351850; Tue, 28 Sep 2021 18:52:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVIDb-0000lt-8w; Tue, 28 Sep 2021 18:52:43 +0000
Received: by outflank-mailman (input) for mailman id 198386;
 Tue, 28 Sep 2021 18:52:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIDZ-0000lj-QU; Tue, 28 Sep 2021 18:52:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIDZ-0006ZA-K5; Tue, 28 Sep 2021 18:52:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIDZ-0007JZ-8Z; Tue, 28 Sep 2021 18:52:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIDZ-0001VP-84; Tue, 28 Sep 2021 18:52:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7yZM0wDvFngq8gTo5MSQ+7he2uHjm/RAD+IDAkJxxI0=; b=ad8ohcsQeOLBkjYWWwzM2WKFEY
	99ASshpVveZhuWcxNoisDmkv9h7p21v1r5YytHjLKsZpZWuFWOTCDvUzXG+M6ZjshmgRSpmhdrz5p
	wq+gUig099o3gvpcJTjCtmWw/sz/nzcBFl6dohG9quv6khRZl9XdxyuklsBsu2WK+lqE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165231-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165231: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:guest-start/debianhvm.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0513e464f9007b70b96740271a948ca5ab6e7dd7
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 18:52:41 +0000

flight 165231 linux-linus real [real]
flight 165244 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165231/
http://logs.test-lab.xenproject.org/osstest/logs/165244/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-rtds 20 guest-localmigrate/x10 fail in 165225 pass in 165231
 test-armhf-armhf-xl-credit1  14 guest-start      fail in 165225 pass in 165231
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 20 guest-start/debianhvm.repeat fail pass in 165225

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0513e464f9007b70b96740271a948ca5ab6e7dd7
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  423 days
Failing since        152366  2020-08-01 20:49:34 Z  422 days  740 attempts
Testing same since   165225  2021-09-27 22:11:09 Z    0 days    2 attempts

------------------------------------------------------------
7347 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2273419 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 18:58:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 18:58:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198395.351864 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVIJI-0001kW-89; Tue, 28 Sep 2021 18:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198395.351864; Tue, 28 Sep 2021 18:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVIJI-0001kP-4g; Tue, 28 Sep 2021 18:58:36 +0000
Received: by outflank-mailman (input) for mailman id 198395;
 Tue, 28 Sep 2021 18:58:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIJH-0001kF-4X; Tue, 28 Sep 2021 18:58:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIJG-0006gY-Rr; Tue, 28 Sep 2021 18:58:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIJG-0007Vp-Gi; Tue, 28 Sep 2021 18:58:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVIJG-0006cb-GF; Tue, 28 Sep 2021 18:58:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TndT9+KRAnnNMqSlHDSgldgYJE9AKAiwicstxOEFADU=; b=GIWbb44pAqzciu13vuJ3UUSDoD
	z6tICke9xqPLAXr1jtNDYjnjW34JMtE3UGyhjvquV3oNu3robeenB6/6nxM9dVQug41y92S05XaDh
	ksYNvcEGDYzen5IugHJAvLa+Iqw++PbIVJ7aSLeIs9RbZscE1hP8m3Cgon7VE8FD+zeA=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165243-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165243: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9c3b9800e2019c93ab22da69e4a0b22d6fb059ec
X-Osstest-Versions-That:
    xen=890ceb9453171c85e881103e65dbb5cdcf81659e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 18:58:34 +0000

flight 165243 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165243/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf                   6 xen-build                fail REGR. vs. 165233

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9c3b9800e2019c93ab22da69e4a0b22d6fb059ec
baseline version:
 xen                  890ceb9453171c85e881103e65dbb5cdcf81659e

Last test of basis   165233  2021-09-28 12:00:26 Z    0 days
Testing same since   165243  2021-09-28 17:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Igor Druzhinin <igor.druzhinin@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 9c3b9800e2019c93ab22da69e4a0b22d6fb059ec
Author: Igor Druzhinin <igor.druzhinin@citrix.com>
Date:   Tue Sep 28 16:04:50 2021 +0200

    pci: fix handling of PCI bridges with subordinate bus number 0xff
    
    Bus number 0xff is valid according to the PCI spec. Using u8 typed sub_bus
    and assigning 0xff to it will result in the following loop getting stuck.
    
        for ( ; sec_bus <= sub_bus; sec_bus++ ) {...}
    
    Just change its type to unsigned int similarly to what is already done in
    dmar_scope_add_buses().
    
    Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

commit 1578322ac6bc4d66800a5a3caf6685f556b64054
Author: Jan Beulich <jbeulich@suse.com>
Date:   Tue Sep 28 16:03:38 2021 +0200

    x86/PVH: actually show Dom0's register state from debug key '0'
    
    vcpu_show_registers() didn't do anything for HVM so far. Note though
    that some extra hackery is needed for VMX - see the code comment.
    
    Note further that the show_guest_stack() invocation is left alone here:
    While strictly speaking guest_kernel_mode() should be predicated by a
    PV / !HVM check, show_guest_stack() itself will bail immediately for
    HVM.
    
    While there and despite not being PVH-specific, take the opportunity and
    filter offline vCPU-s: There's not really any register state associated
    with them, so avoid spamming the log with useless information while
    still leaving an indication of the fact.
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:19:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:19:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198408.351878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJZF-00024k-A5; Tue, 28 Sep 2021 20:19:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198408.351878; Tue, 28 Sep 2021 20:19:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJZF-00024d-76; Tue, 28 Sep 2021 20:19:09 +0000
Received: by outflank-mailman (input) for mailman id 198408;
 Tue, 28 Sep 2021 20:19:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Q7JL=OS=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mVJZD-00024X-6r
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:19:07 +0000
Received: from mail-wm1-x32e.google.com (unknown [2a00:1450:4864:20::32e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 29e99244-c8fa-4ff3-a0d9-4b5cbd0ee0b4;
 Tue, 28 Sep 2021 20:19:05 +0000 (UTC)
Received: by mail-wm1-x32e.google.com with SMTP id s24so7858wmh.4
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:19:05 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29e99244-c8fa-4ff3-a0d9-4b5cbd0ee0b4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=y+fMd4RTS2D4OXvXgqrntp+86PEPSos3v6YDgp1EsMg=;
        b=mpTnH8XNnCJyV9bOOdc+UyyisgWimas/3NVgbzQ8TnikvXrxEzc4VLSGHGAI4cGXVF
         E3McmRKMYEWKzGPuj9LwW9QUYqWnXPJx+pPPxBDM52VcVNqg7K491dWGyZWrzyWyoISr
         Gl0/G4FwsBVVIjJ/1Mespx0myvFlGvCSnuYm480vPap7T4k+Z2jHOXl9yX0LR1TJmU1m
         T8YtCPS+q6HPJkHyBKn6vYFcPY0D+aFle0q8gfJtSsHqlGwkGPMIeVwgD2/CqTstJ4gj
         MWmJ4AOFx9MBw0QUFdZv5IagINAqmt6T9cZomMyRk72K7HcKgJdt2dparf2u9cLXxhWL
         TOVg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=y+fMd4RTS2D4OXvXgqrntp+86PEPSos3v6YDgp1EsMg=;
        b=iAglU1EJ4FBx7LLeYzQWOKIvWdEeNm/5w3khgsZTIvLTcjCoHj+OmnEMMVfGM8XLof
         Bw08OSkPd64ENzc9wK8oHFKc8T092nVPmonclqW6U1V/ScPT8PUyAZlmKk6PAieLr/jX
         Fz5KlGi7ZSzwmTD9WV8h0WhfFsfXkiU6zL18vQtTuYog+n3sKCjiFV+vi5UGH6yItEaV
         Hih4pKSBxY6s3jYHL3Ba6mwgy9d+iIkkLobl7Tzx6sx/2L6r/PFBeRn0uoizl5OIL6+q
         O1ZpSbPkSv9KjGiCPrRLevCKfzGrWpWAUlKwj0LnS08mm5MIQbvc4kaP58av1WWTpBxG
         rF1A==
X-Gm-Message-State: AOAM533wAF6ZMGC1d1p7YyQrGpEQs1cttHGST+waV1TstROuHjtD/qvc
	cNvncmIaW7m2U7V8yMTnt5eLJNilF5JTyR6JsI0=
X-Google-Smtp-Source: ABdhPJxqFVoNAliDhBwpjCCTIRD4B75LaUGT9z/EtbJpuvy/F6c0IPHLEBfnGuJ6ltFSPfJ+84S7qWx0IcSv/BylLVo=
X-Received: by 2002:a05:600c:350f:: with SMTP id h15mr6692437wmq.144.1632860344161;
 Tue, 28 Sep 2021 13:19:04 -0700 (PDT)
MIME-Version: 1.0
References: <87pmsylywy.fsf@linaro.org> <YU5mW396S04IsCBr@mail-itl>
 <874ka68h96.fsf@linaro.org> <CACMJ4GbgnQVQbH1N3Duxmx10n-Qv+zzncqarNyuhmKhE-wqdqA@mail.gmail.com>
 <alpine.DEB.2.21.2109272323160.5022@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109272323160.5022@sstabellini-ThinkPad-T480s>
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
Date: Tue, 28 Sep 2021 23:18:53 +0300
Message-ID: <CAPD2p-=MdGB_a+oEsFrPQpLo7GeKkMwYyAWcQt3z0qrGr3vR7w@mail.gmail.com>
Subject: Re: [Stratos-dev] Xen Rust VirtIO demos work breakdown for Project Stratos
To: Stefano Stabellini <sstabellini@kernel.org>, xen-devel <xen-devel@lists.xenproject.org>
Cc: Christopher Clark <christopher.w.clark@gmail.com>, =?UTF-8?B?QWxleCBCZW5uw6ll?= <alex.bennee@linaro.org>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
	Artem Mygaiev <Artem_Mygaiev@epam.com>, Sergio Lopez <slp@redhat.com>, Wei Liu <wl@xen.org>, 
	Stefan Hajnoczi <stefanha@gmail.com>, Rust-VMM Mailing List <rust-vmm@lists.opendev.org>, 
	Doug Goldstein <cardoe@cardoe.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Arnd Bergmann <arnd.bergmann@linaro.org>, David Woodhouse <dwmw2@infradead.org>, 
	Stratos Mailing List <stratos-dev@op-lists.linaro.org>, Rich Persaud <persaur@gmail.com>, 
	Daniel Smith <dpsmith@apertussolutions.com>, Paul Durrant <paul@xen.org>, 
	openxt <openxt@googlegroups.com>
Content-Type: multipart/alternative; boundary="000000000000c8170e05cd13ec94"

--000000000000c8170e05cd13ec94
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Sep 28, 2021 at 9:26 AM Stefano Stabellini <sstabellini@kernel.org>
wrote:

Hi Stefano, all

[Sorry for the possible format issues]


On Mon, 27 Sep 2021, Christopher Clark wrote:
> > On Mon, Sep 27, 2021 at 3:06 AM Alex Benn=C3=A9e via Stratos-dev <
> stratos-dev@op-lists.linaro.org> wrote:
> >
> >       Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>
> writes:
> >
> >       > [[PGP Signed Part:Undecided]]
> >       > On Fri, Sep 24, 2021 at 05:02:46PM +0100, Alex Benn=C3=A9e wrot=
e:
> >       >> Hi,
> >       >
> >       > Hi,
> >       >
> >       >> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57]=
)
> >       >> =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80
> >       >>
> >       >>   Currently the foreign memory mapping support only works for
> dom0 due
> >       >>   to reference counting issues. If we are to support backends
> running in
> >       >>   their own domains this will need to get fixed.
> >       >>
> >       >>   Estimate: 8w
> >       >>
> >       >>
> >       >> [STR-57] <https://linaro.atlassian.net/browse/STR-57>
> >       >
> >       > I'm pretty sure it was discussed before, but I can't find
> relevant
> >       > (part of) thread right now: does your model assumes the backend
> (running
> >       > outside of dom0) will gain ability to map (or access in other
> way)
> >       > _arbitrary_ memory page of a frontend domain? Or worse: any
> domain?
> >
> >       The aim is for some DomU's to host backends for other DomU's
> instead of
> >       all backends being in Dom0. Those backend DomU's would have to be
> >       considered trusted because as you say the default memory model of
> VirtIO
> >       is to have full access to the frontend domains memory map.
> >
> >
> > I share Marek's concern. I believe that there are Xen-based systems tha=
t
> will want to run guests using VirtIO devices without extending
> > this level of trust to the backend domains.
>
> From a safety perspective, it would be challenging to deploy a system
> with privileged backends. From a safety perspective, it would be a lot
> easier if the backend were unprivileged.
>
> This is one of those times where safety and security requirements are
> actually aligned.


Well, the foreign memory mapping has one advantage in the context of Virtio
use-case
which is that Virtio infrastructure in Guest doesn't require any
modifications to run on top Xen.
The only issue with foreign memory here is that Guest memory actually
mapped without its agreement
which doesn't perfectly fit into the security model. (although there is one
more issue with XSA-300,
but I think it will go away sooner or later, at least there are some
attempts to eliminate it).
While the ability to map any part of Guest memory is not an issue for the
backend running in Dom0
(which we usually trust), this will certainly violate Xen security model if
we want to run it in other
domain, so I completely agree with the existing concern.

It was discussed before [1], but I couldn't find any decisions regarding
that. As I understand,
the one of the possible ideas is to have some entity in Xen (PV
IOMMU/virtio-iommu/whatever)
that works in protection mode, so it denies all foreign mapping requests
from the backend running in DomU
by default and only allows requests with mapping which were *implicitly*
granted by the Guest before.
For example, Xen could be informed which MMIOs hold the queue PFN and
notify registers
(as it traps the accesses to these registers anyway) and could
theoretically parse the frontend request
and retrieve descriptors to make a decision which GFNs are actually
*allowed*.

I can't say for sure (sorry not familiar enough with the topic), but
implementing the virtio-iommu device
in Xen we could probably avoid Guest modifications at all. Of course, for
this to work
the Virtio infrastructure in Guest should use DMA API as mentioned in [1].

Would the =E2=80=9Crestricted foreign mapping=E2=80=9D solution retain the =
Xen security
model and be accepted
by the Xen community? I wonder, has someone already looked in this
direction, are there any
pitfalls here or is this even feasible?

[1]
https://lore.kernel.org/xen-devel/464e91ec-2b53-2338-43c7-a018087fc7f6@arm.=
com/



--=20
Regards,

Oleksandr Tyshchenko

--000000000000c8170e05cd13ec94
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Tue, Sep 28, 2021 at 9:26 AM Stefa=
no Stabellini &lt;<a href=3D"mailto:sstabellini@kernel.org">sstabellini@ker=
nel.org</a>&gt; wrote:<br></div><div dir=3D"ltr" class=3D"gmail_attr"><br><=
/div><div class=3D"gmail_attr">Hi Stefano, all</div><div class=3D"gmail_att=
r"><br></div><div class=3D"gmail_attr">[Sorry for the possible format issue=
s]</div><div dir=3D"ltr" class=3D"gmail_attr"><br></div><div dir=3D"ltr" cl=
ass=3D"gmail_attr"><br></div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">On Mon, 27 Sep 2021, Christopher Clark wrote:<br>
&gt; On Mon, Sep 27, 2021 at 3:06 AM Alex Benn=C3=A9e via Stratos-dev &lt;<=
a href=3D"mailto:stratos-dev@op-lists.linaro.org" target=3D"_blank">stratos=
-dev@op-lists.linaro.org</a>&gt; wrote:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Marek Marczykowski-G=C3=B3recki &lt;<a href=
=3D"mailto:marmarek@invisiblethingslab.com" target=3D"_blank">marmarek@invi=
siblethingslab.com</a>&gt; writes:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; [[PGP Signed Part:Undecided]]<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; On Fri, Sep 24, 2021 at 05:02:46PM +010=
0, Alex Benn=C3=A9e wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt; Hi,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Hi,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt; 2.1 Stable ABI for foreignmemory ma=
pping to non-dom0 ([STR-57])<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt; =E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=
=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=
=E2=94=80<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;=C2=A0 =C2=A0Currently the foreign m=
emory mapping support only works for dom0 due<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;=C2=A0 =C2=A0to reference counting i=
ssues. If we are to support backends running in<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;=C2=A0 =C2=A0their own domains this =
will need to get fixed.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;=C2=A0 =C2=A0Estimate: 8w<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;&gt; [STR-57] &lt;<a href=3D"https://lin=
aro.atlassian.net/browse/STR-57" rel=3D"noreferrer" target=3D"_blank">https=
://linaro.atlassian.net/browse/STR-57</a>&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; I&#39;m pretty sure it was discussed be=
fore, but I can&#39;t find relevant<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; (part of) thread right now: does your m=
odel assumes the backend (running<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; outside of dom0) will gain ability to m=
ap (or access in other way)<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; _arbitrary_ memory page of a frontend d=
omain? Or worse: any domain?<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0The aim is for some DomU&#39;s to host backe=
nds for other DomU&#39;s instead of<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0all backends being in Dom0. Those backend Do=
mU&#39;s would have to be<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0considered trusted because as you say the de=
fault memory model of VirtIO<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0is to have full access to the frontend domai=
ns memory map.<br>
&gt; <br>
&gt; <br>
&gt; I share Marek&#39;s concern. I believe that there are Xen-based system=
s that will want to run guests using VirtIO devices without extending<br>
&gt; this level of trust to=C2=A0the backend domains.<br>
<br>
>From a safety perspective, it would be challenging to deploy a system<br>
with privileged backends. From a safety perspective, it would be a lot<br>
easier if the backend were unprivileged.<br>
<br>
This is one of those times where safety and security requirements are<br>
actually aligned.</blockquote><div><br></div><div>Well, the foreign memory =
mapping has one advantage in the context of Virtio use-case<br>which is tha=
t Virtio infrastructure in Guest doesn&#39;t require any modifications to r=
un on top Xen. <br>The only issue with foreign memory here is that Guest me=
mory actually mapped without its agreement<br>which doesn&#39;t perfectly f=
it into=C2=A0the security model. (although there is one more issue with XSA=
-300,<br>but I think it will go away sooner or later, at least there are so=
me attempts to eliminate it).<br>While the ability to map any part of Guest=
 memory is not an issue for the backend running in Dom0<br>(which we usuall=
y trust), this will certainly violate Xen security model if we want to run =
it in other<br>domain, so I completely agree with the existing concern.<br>=
<br>It was discussed before [1], but I couldn&#39;t find any decisions rega=
rding that. As I understand,<br>the one of the possible ideas is to have so=
me entity in Xen (PV IOMMU/virtio-iommu/whatever)<br>that works in protecti=
on mode, so it denies all foreign mapping requests from the backend running=
 in DomU<br>by default and only allows requests with mapping which were *im=
plicitly* granted by the Guest before.<br>For example, Xen could be informe=
d which MMIOs hold the queue PFN and notify registers<br>(as it traps the a=
ccesses to these registers anyway) and could theoretically parse the fronte=
nd request<br>and retrieve descriptors to make a decision which GFNs are ac=
tually *allowed*.<br><br>I can&#39;t say for sure (sorry not familiar enoug=
h with the topic), but implementing the virtio-iommu device<br>in Xen we co=
uld probably avoid Guest modifications at all. Of course, for this to work<=
br>the Virtio infrastructure in Guest should use DMA API as mentioned in [1=
].</div><div><br>Would the =E2=80=9Crestricted foreign mapping=E2=80=9D sol=
ution retain the Xen security model and be accepted<br>by the Xen community=
? I wonder, has someone already looked in this direction, are there any<br>=
pitfalls here or is this even feasible?<br><br>[1] <a href=3D"https://lore.=
kernel.org/xen-devel/464e91ec-2b53-2338-43c7-a018087fc7f6@arm.com/">https:/=
/lore.kernel.org/xen-devel/464e91ec-2b53-2338-43c7-a018087fc7f6@arm.com/</a=
><br></div></div><div class=3D"gmail_quote"><br><div>=C2=A0</div></div><div=
><br></div>-- <br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"lt=
r"><div><div dir=3D"ltr"><div><div dir=3D"ltr"><span style=3D"background-co=
lor:rgb(255,255,255)"><font size=3D"2"><span style=3D"color:rgb(51,51,51);f=
ont-family:Arial,sans-serif">Regards,</span></font></span></div><div dir=3D=
"ltr"><br></div><div dir=3D"ltr"><div><span style=3D"background-color:rgb(2=
55,255,255)"><font size=3D"2">Oleksandr Tyshchenko</font></span></div></div=
></div></div></div></div></div></div>

--000000000000c8170e05cd13ec94--


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:30:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:30:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198415.351900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJke-0004fF-Mu; Tue, 28 Sep 2021 20:30:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198415.351900; Tue, 28 Sep 2021 20:30:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJke-0004f6-I7; Tue, 28 Sep 2021 20:30:56 +0000
Received: by outflank-mailman (input) for mailman id 198415;
 Tue, 28 Sep 2021 20:30:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJkd-0004O5-8X
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:30:55 +0000
Received: from mail-pl1-x629.google.com (unknown [2607:f8b0:4864:20::629])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c46b5e31-a2db-4dc4-8b19-4e4c8adf388f;
 Tue, 28 Sep 2021 20:30:50 +0000 (UTC)
Received: by mail-pl1-x629.google.com with SMTP id l6so14980324plh.9
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:50 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: c46b5e31-a2db-4dc4-8b19-4e4c8adf388f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ymbhtWJDlmnuoU0qSCptZumb6o8HnxQbmRZItmihOOU=;
        b=hr3dpobqy60MQ0/xIdFAi4fhITENVYsP9/vWT8arSnZjQqR85SucjebewT3a7Cip7s
         rDz2XoyDlJmCsWxgBwPMh67obBCcuVDQDJFqFD4AkJXIx7vEYNWlsGX7WSOJls20gJqt
         1HNSWnnW8fFDv0k9Ok0Uk7ElpDGUtqxbDhigXsUrgmjrgz41LHOls+/SBAxaXVRNEt+r
         H6CKHkmIcShaC4+G/UP2Y7Zlx4/8X0NDW1/tVAHqoGiiX/o1mWCZQfMQJW6J+/D3WbbC
         z7sJHI20GYgOSBHqaPrrUfGaWeTyT/dA3xqF5Ml1OPjB1yudTRnsliOlS9ajk/hAQ1uA
         vR/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=ymbhtWJDlmnuoU0qSCptZumb6o8HnxQbmRZItmihOOU=;
        b=3CMAcHGQ8FwfLbpS0S16/jcB9p3aF2yRm10gU2JxzHO8/vblA9FPXZTE6a/mCO8+Zt
         rL5t37AgPY2zfTYwv8Jfx5RHlfP9HXiu5IGNy8TzXZnnGFJCRMDRgKdYtgK41XHGlRPj
         Km7TKLhC/wV+JThvtg9feVx5wIv9FNxTf7Om8krSvxWxX4OdUDq7XMkbwewdRwkgMqqy
         YOfLQGYpEuuiY+HBfuyYUAqCqwNpz4LC50jRDGqpghUJqBOXsXtwyoUTGs2nlcC4sZOy
         IQoYWLRWLLj+JeFtu1PZio9fivfSKrGyVSg7Yrj8xP+sBgo9CiKD7QjfEDMUHuEUVBtd
         WdlQ==
X-Gm-Message-State: AOAM530O+KBlm/R810fkc6Ne7h4pRita7cM3eqv4JZYFl0pJaIJcMhZT
	gz6OXSADfeaZIxCSJ1r0V5SPBfPxUKgx7xRg
X-Google-Smtp-Source: ABdhPJwHV0Vtsr8CP1xdXrs4yT4Ov+taT2HHe+f6VfNoQo38IkWERMKnhvVM8wpIKHE/y1MgOqA5kQ==
X-Received: by 2002:a17:90b:17ca:: with SMTP id me10mr2070593pjb.171.1632861049495;
        Tue, 28 Sep 2021 13:30:49 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 1/6] arm/traps: remove debugger_trap_fatal() calls
Date: Tue, 28 Sep 2021 13:30:24 -0700
Message-Id: <4c4f45280547564b2c16fd9ca09443314b8f3b6e.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

ARM doesn't actually use debugger_trap_* anything, and is stubbed out.

This commit simply removes the unneeded calls.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
 xen/arch/arm/traps.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 4ccb6e7d18..889650ba63 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -41,7 +41,6 @@
 #include <asm/acpi.h>
 #include <asm/cpuerrata.h>
 #include <asm/cpufeature.h>
-#include <asm/debugger.h>
 #include <asm/event.h>
 #include <asm/hsr.h>
 #include <asm/mmio.h>
@@ -1266,10 +1265,6 @@ int do_bug_frame(const struct cpu_user_regs *regs, vaddr_t pc)
 
     case BUGFRAME_bug:
         printk("Xen BUG at %s%s:%d\n", prefix, filename, lineno);
-
-        if ( debugger_trap_fatal(TRAP_invalid_op, regs) )
-            return 0;
-
         show_execution_state(regs);
         panic("Xen BUG at %s%s:%d\n", prefix, filename, lineno);
 
@@ -1281,8 +1276,6 @@ int do_bug_frame(const struct cpu_user_regs *regs, vaddr_t pc)
 
         printk("Assertion '%s' failed at %s%s:%d\n",
                predicate, prefix, filename, lineno);
-        if ( debugger_trap_fatal(TRAP_invalid_op, regs) )
-            return 0;
         show_execution_state(regs);
         panic("Assertion '%s' failed at %s%s:%d\n",
               predicate, prefix, filename, lineno);
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:30:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:30:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198414.351889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJka-0004OI-DO; Tue, 28 Sep 2021 20:30:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198414.351889; Tue, 28 Sep 2021 20:30:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJka-0004OB-AC; Tue, 28 Sep 2021 20:30:52 +0000
Received: by outflank-mailman (input) for mailman id 198414;
 Tue, 28 Sep 2021 20:30:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJkY-0004O5-CI
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:30:50 +0000
Received: from mail-pl1-x62e.google.com (unknown [2607:f8b0:4864:20::62e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3f939467-5c9f-4369-ab96-b758b676660c;
 Tue, 28 Sep 2021 20:30:49 +0000 (UTC)
Received: by mail-pl1-x62e.google.com with SMTP id t11so14978206plq.11
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:49 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:47 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 3f939467-5c9f-4369-ab96-b758b676660c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=iMtwLZAVHJpOgTEsa8hy0GrKBs2oDP5skUKZH5QnAzE=;
        b=MdvmXi2rLQvaG35/zPwUQl1d0zlrBnvQd+JZZB+3OBpO4ucZ0L0cR2iMgIgPdnFh40
         AUNMeDwXUTD3E7sK1zFCCcL42xG6x8ZeBIKKIcQj6QU/+VTeoYF+A7X8chfvxKE0ANhm
         wYa/vxi7IkYa0lkiyfLCE7BHBNv0bE3COmv3TltALbCx+bgrmFE/MH/MbUL3iF2xznVA
         BjMODhCb7RhTgjtBQwVxeBreOqD1z9kLCVlJIwNq+ySbvWlucKBk/XpTp6rFuOmr07U4
         Z4enqqdDMEuI6cXGoCyqIL5VBQ2sCoJoDN/P1pGx+sN2A8wML9eZcgYtym8IMjAMQAuG
         lfgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :mime-version:content-transfer-encoding;
        bh=iMtwLZAVHJpOgTEsa8hy0GrKBs2oDP5skUKZH5QnAzE=;
        b=X3VQ1rDfLLzcfBGTRAKYw9AWJoYOqH0nKzJ1fMV5njrBa+gwZ0xge1GlcjVzPDAoai
         eMDQddb6SLOEIvZa/nWLtGb85bx+P5fiw1EdQoh1b0MJxIqA1oddP/jydLXZt5xiISZ8
         QDZJZFhEda3woWASzWOQzwC6VlE03nVxLbWKr3rWBPbIze5ux4tWp+scnR7zbdKpiVTE
         kMDBnG+n9G6mP55aUMcaMTTj6cNwRNJ+Moq0lUjyjxhUIhgCJXDGArKKs/eyBsL0iUdq
         lIbOhhteAcdUxIodL56mRocKadCBwdaVqTABvy00Qfgo5hka2I+t/kOY6wx4nm/Uj+p5
         bZhg==
X-Gm-Message-State: AOAM532j1jvIXHQBSxuv3jJHqN1t+V3YhIvV43EKgnTWpbFnWrynmy6w
	M7nbmzCO2d1RSI7aQuuuJmOMmgUB1cjzr2kk
X-Google-Smtp-Source: ABdhPJwasL3UEmuqkANz46DT2jx1ehatfui8bqdx78Wjd7m9F9uqIeOmdvEAXPfWmoAvdox65yG51w==
X-Received: by 2002:a17:90b:224b:: with SMTP id hk11mr2044203pjb.231.1632861048275;
        Tue, 28 Sep 2021 13:30:48 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 0/6] Remove unconditional arch dependency on asm/debugger.h
Date: Tue, 28 Sep 2021 13:30:23 -0700
Message-Id: <cover.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This series removes the unconditional requirement that all architectures
implement asm/debugger.h. It additionally removes arm's debugger.h and
disentangles some of the x86 gdbsx/gdbstub/generic debugger code.

Additionally, this series does the following:
- Provides generic stubs when !CONFIG_CRASH_DEBUG
- Adds stronger separation between gdbstub, gdbsx, and generic debugger
  code.

v4 simply includes the review feedback from v3 with no other big changes
(as was the case for v3 in comparison to v2).

Bobby Eshleman (6):
  arm/traps: remove debugger_trap_fatal() calls
  x86/debugger: separate Xen and guest debugging debugger_trap_*
    functions
  arch/x86: rename debug.c to gdbsx.c
  x86/gdbsx: expand dbg_rw_mem() inline
  arch/x86: move domain_pause_for_debugger() to domain.h
  x86: change asm/debugger.h to xen/debugger.h

 xen/arch/arm/traps.c              |  7 ----
 xen/arch/x86/Makefile             |  2 +-
 xen/arch/x86/domain.c             |  2 +-
 xen/arch/x86/domctl.c             | 12 +-----
 xen/arch/x86/{debug.c => gdbsx.c} | 28 ++++++-------
 xen/arch/x86/nmi.c                |  1 -
 xen/arch/x86/traps.c              | 52 +++++++++++++++----------
 xen/common/domain.c               |  2 +-
 xen/common/gdbstub.c              |  2 +-
 xen/common/keyhandler.c           |  2 +-
 xen/common/shutdown.c             |  2 +-
 xen/drivers/char/console.c        |  2 +-
 xen/include/asm-arm/debugger.h    | 15 -------
 xen/include/asm-x86/debugger.h    | 65 +------------------------------
 xen/include/asm-x86/domain.h      |  2 +
 xen/include/asm-x86/gdbsx.h       | 19 +++++++++
 xen/include/xen/debugger.h        | 51 ++++++++++++++++++++++++
 17 files changed, 125 insertions(+), 141 deletions(-)
 rename xen/arch/x86/{debug.c => gdbsx.c} (89%)
 delete mode 100644 xen/include/asm-arm/debugger.h
 create mode 100644 xen/include/asm-x86/gdbsx.h
 create mode 100644 xen/include/xen/debugger.h

-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:31:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:31:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198416.351911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkk-00051C-4V; Tue, 28 Sep 2021 20:31:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198416.351911; Tue, 28 Sep 2021 20:31:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkk-000515-1T; Tue, 28 Sep 2021 20:31:02 +0000
Received: by outflank-mailman (input) for mailman id 198416;
 Tue, 28 Sep 2021 20:31:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJki-0004O5-8W
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:31:00 +0000
Received: from mail-pf1-x430.google.com (unknown [2607:f8b0:4864:20::430])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 545b2fb6-0eea-4f78-a11d-edba8a427652;
 Tue, 28 Sep 2021 20:30:51 +0000 (UTC)
Received: by mail-pf1-x430.google.com with SMTP id b68so1940566pfb.10
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:51 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:50 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 545b2fb6-0eea-4f78-a11d-edba8a427652
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=H5uWePdNAFApbwchB0lEmCcQHqVtRGoM5xxH6A0jNPE=;
        b=OU8kbq/6odgsBLbuRYNvQcjSGO1agLQ/zNMWt3bSZnEt/x0KR4ZZVBJKRp646+p6a8
         wFLoSKNevE3pDOho2bHyMPS1QohH6VwQ6R1WCZtYiLmU5ut4kE4rwXA4w/vGHRvb6WoP
         2RBltHjW+YpY/nrFOPiKeg6cEpX5j/U/VpIv2LZD/7jSKWkaWVkthLzp2pf0TshKol3Y
         dmMpIiW5f5OR0Wf9C8uZm7wUrzd9gDkVUV+jiHXxzy//i9yj2GB62tqiGPJJOJA9i1uF
         Pd5GsekypPhshYhKjGPlzo1iG1RCBeLMyK4MM1ENZXf1m75IMxj0x4kJTnmevCYlJoiA
         9Adw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=H5uWePdNAFApbwchB0lEmCcQHqVtRGoM5xxH6A0jNPE=;
        b=2MySHqMZYEwtT/m7mTG5BNW+3j0/4sr1cXRVNcXF2y+xucQSYpQ0tS85xpI0FNLTTI
         8KYKFzP09P4SdhOm4oPW1VvOKh0kKUCmTnGUHuRLyT7LZFequK/nq29YX0eeHN7qzCCL
         MQkP4bNp8y3aAfr3u/3c0xS5uwKAOCy8B2eRJpqTbOLICRIFMJM4GL0RyH1sSSanh0w4
         y904VqdcFasakPrXEa+/RAgb3oD1BuLxA3IL7CA9n94AwCR0GNJ6SKK+VX3tevjhQZvf
         L/FOd/UqvsiKXtmmT02FhtwcDv/6AydOKDFfOp0fPCA+SP/H0re0iaHA//F8B75U62om
         k6IA==
X-Gm-Message-State: AOAM5325TX0a9ai8ExhvZG097UPxKvxaNpkts7tWIGuDNIkgnDKgNxhA
	v0f0KSG7GQp6VMc+7jhkWKFrYH4Kna0wUHB6
X-Google-Smtp-Source: ABdhPJxz/2L818H7HQdZtDgvXYda6K8y0D3UwBb5Vo5vRi3oUcVuGZc0LXJGRokADf5XuTPMpykXTw==
X-Received: by 2002:a62:7dd3:0:b0:447:dd44:1580 with SMTP id y202-20020a627dd3000000b00447dd441580mr7427104pfc.19.1632861050693;
        Tue, 28 Sep 2021 13:30:50 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 2/6] x86/debugger: separate Xen and guest debugging debugger_trap_* functions
Date: Tue, 28 Sep 2021 13:30:25 -0700
Message-Id: <47e32bf1eb0cd18210df4273d564fa17d389f3ae.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The functions debugger_trap_fatal(), debugger_trap_immediate(), and
debugger_trap_entry() are generic hook functions for debugger support.
In practice, debugger_trap_fatal() and debugger_trap_immediate() are
only used in the debugging of Xen itself and debugger_trap_entry() is
only used in the debugging of guests. That is, debugger_trap_entry() is
part of gdbsx functionality and not the Xen gdstub. This is evidenced by
debugger_trap_fatal()'s usage of domain_pause_for_debugger(). Because of
this, debugger_trap_entry() many not belong alongside the Xen debug
functions.

This commit fixes this by expanding inline debugger_trap_entry() into
its usage sites in x86/traps.c and stubbing out
domain_pause_for_debugger() when !CONFIG_GDBSX. Placing what
was debugger_trap_entry() under the scope of gdbsx instead of gdbstub.

The function calls that caused an effective no-op and early exit out of
debugger_trap_entry() are removed completely (when the trapnr is not
int3/debug).

This commit is one of a series geared towards removing the unnecessary
requirement that all architectures to implement <asm/debugger.h>.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
Changes in v4:
- Reword commit message for accuracy (make weaker claims)
- Fix "if { return } else if { return }" anti-pattern

 xen/arch/x86/domain.c          |  2 +-
 xen/arch/x86/traps.c           | 50 ++++++++++++++++++++--------------
 xen/include/asm-x86/debugger.h | 42 ++--------------------------
 3 files changed, 33 insertions(+), 61 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ef1812dc14..70894ff826 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -2541,7 +2541,7 @@ __initcall(init_vcpu_kick_softirq);
 
 void domain_pause_for_debugger(void)
 {
-#ifdef CONFIG_CRASH_DEBUG
+#ifdef CONFIG_GDBSX
     struct vcpu *curr = current;
     struct domain *d = curr->domain;
 
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index e60af16ddd..772e2a5bfc 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -858,13 +858,20 @@ static void do_trap(struct cpu_user_regs *regs)
     if ( regs->error_code & X86_XEC_EXT )
         goto hardware_trap;
 
-    if ( debugger_trap_entry(trapnr, regs) )
-        return;
-
     ASSERT(trapnr < 32);
 
     if ( guest_mode(regs) )
     {
+        struct vcpu *curr = current;
+        if ( (trapnr == TRAP_debug || trapnr == TRAP_int3) &&
+              guest_kernel_mode(curr, regs) &&
+              curr->domain->debugger_attached )
+        {
+            if ( trapnr != TRAP_debug )
+                curr->arch.gdbsx_vcpu_event = trapnr;
+            domain_pause_for_debugger();
+            return;
+        }
         pv_inject_hw_exception(trapnr,
                                (TRAP_HAVE_EC & (1u << trapnr))
                                ? regs->error_code : X86_EVENT_NO_EC);
@@ -1094,9 +1101,6 @@ void do_invalid_op(struct cpu_user_regs *regs)
     int id = -1, lineno;
     const struct virtual_region *region;
 
-    if ( debugger_trap_entry(TRAP_invalid_op, regs) )
-        return;
-
     if ( likely(guest_mode(regs)) )
     {
         if ( pv_emulate_invalid_op(regs) )
@@ -1201,8 +1205,7 @@ void do_invalid_op(struct cpu_user_regs *regs)
 
 void do_int3(struct cpu_user_regs *regs)
 {
-    if ( debugger_trap_entry(TRAP_int3, regs) )
-        return;
+    struct vcpu *curr = current;
 
     if ( !guest_mode(regs) )
     {
@@ -1216,6 +1219,13 @@ void do_int3(struct cpu_user_regs *regs)
         return;
     }
 
+    if ( guest_kernel_mode(curr, regs) && curr->domain->debugger_attached )
+    {
+        curr->arch.gdbsx_vcpu_event = TRAP_int3;
+        domain_pause_for_debugger();
+        return;
+    }
+
     pv_inject_hw_exception(TRAP_int3, X86_EVENT_NO_EC);
 }
 
@@ -1492,9 +1502,6 @@ void do_page_fault(struct cpu_user_regs *regs)
     /* fixup_page_fault() might change regs->error_code, so cache it here. */
     error_code = regs->error_code;
 
-    if ( debugger_trap_entry(TRAP_page_fault, regs) )
-        return;
-
     perfc_incr(page_faults);
 
     /* Any shadow stack access fault is a bug in Xen. */
@@ -1593,9 +1600,6 @@ void do_general_protection(struct cpu_user_regs *regs)
     struct vcpu *v = current;
 #endif
 
-    if ( debugger_trap_entry(TRAP_gp_fault, regs) )
-        return;
-
     if ( regs->error_code & X86_XEC_EXT )
         goto hardware_gp;
 
@@ -1888,9 +1892,6 @@ void do_debug(struct cpu_user_regs *regs)
     /* Stash dr6 as early as possible. */
     dr6 = read_debugreg(6);
 
-    if ( debugger_trap_entry(TRAP_debug, regs) )
-        return;
-
     /*
      * At the time of writing (March 2018), on the subject of %dr6:
      *
@@ -1995,6 +1996,12 @@ void do_debug(struct cpu_user_regs *regs)
         return;
     }
 
+    if ( guest_kernel_mode(v, regs) && v->domain->debugger_attached )
+    {
+        domain_pause_for_debugger();
+        return;
+    }
+
     /* Save debug status register where guest OS can peek at it */
     v->arch.dr6 |= (dr6 & ~X86_DR6_DEFAULT);
     v->arch.dr6 &= (dr6 | ~X86_DR6_DEFAULT);
@@ -2014,9 +2021,6 @@ void do_entry_CP(struct cpu_user_regs *regs)
     const char *err = "??";
     unsigned int ec = regs->error_code;
 
-    if ( debugger_trap_entry(TRAP_debug, regs) )
-        return;
-
     /* Decode ec if possible */
     if ( ec < ARRAY_SIZE(errors) && errors[ec][0] )
         err = errors[ec];
@@ -2028,6 +2032,12 @@ void do_entry_CP(struct cpu_user_regs *regs)
      */
     if ( guest_mode(regs) )
     {
+        struct vcpu *curr = current;
+        if ( guest_kernel_mode(curr, regs) && curr->domain->debugger_attached )
+        {
+            domain_pause_for_debugger();
+            return;
+        }
         gprintk(XENLOG_ERR, "Hit #CP[%04x] in guest context %04x:%p\n",
                 ec, regs->cs, _p(regs->rip));
         ASSERT_UNREACHABLE();
diff --git a/xen/include/asm-x86/debugger.h b/xen/include/asm-x86/debugger.h
index 99803bfd0c..cd6b9477f7 100644
--- a/xen/include/asm-x86/debugger.h
+++ b/xen/include/asm-x86/debugger.h
@@ -5,19 +5,12 @@
  * 
  * Each debugger should define two functions here:
  * 
- * 1. debugger_trap_entry(): 
- *  Called at start of any synchronous fault or trap, before any other work
- *  is done. The idea is that if your debugger deliberately caused the trap
- *  (e.g. to implement breakpoints or data watchpoints) then you can take
- *  appropriate action and return a non-zero value to cause early exit from
- *  the trap function.
- * 
- * 2. debugger_trap_fatal():
+ * 1. debugger_trap_fatal():
  *  Called when Xen is about to give up and crash. Typically you will use this
  *  hook to drop into a debug session. It can also be used to hook off
  *  deliberately caused traps (which you then handle and return non-zero).
  *
- * 3. debugger_trap_immediate():
+ * 2. debugger_trap_immediate():
  *  Called if we want to drop into a debugger now.  This is essentially the
  *  same as debugger_trap_fatal, except that we use the current register state
  *  rather than the state which was in effect when we took the trap.
@@ -49,31 +42,6 @@ static inline bool debugger_trap_fatal(
 /* Int3 is a trivial way to gather cpu_user_regs context. */
 #define debugger_trap_immediate() __asm__ __volatile__ ( "int3" );
 
-static inline bool debugger_trap_entry(
-    unsigned int vector, struct cpu_user_regs *regs)
-{
-    /*
-     * This function is called before any checks are made.  Amongst other
-     * things, be aware that during early boot, current is not a safe pointer
-     * to follow.
-     */
-    struct vcpu *v = current;
-
-    if ( vector != TRAP_int3 && vector != TRAP_debug )
-        return false;
-
-    if ( guest_mode(regs) && guest_kernel_mode(v, regs) &&
-         v->domain->debugger_attached  )
-    {
-        if ( vector != TRAP_debug ) /* domain pause is good enough */
-            current->arch.gdbsx_vcpu_event = vector;
-        domain_pause_for_debugger();
-        return true;
-    }
-
-    return false;
-}
-
 #else
 
 static inline bool debugger_trap_fatal(
@@ -84,12 +52,6 @@ static inline bool debugger_trap_fatal(
 
 #define debugger_trap_immediate() ((void)0)
 
-static inline bool debugger_trap_entry(
-    unsigned int vector, struct cpu_user_regs *regs)
-{
-    return false;
-}
-
 #endif
 
 #ifdef CONFIG_GDBSX
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:31:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:31:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198417.351922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkp-0005Qz-GI; Tue, 28 Sep 2021 20:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198417.351922; Tue, 28 Sep 2021 20:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkp-0005Qm-By; Tue, 28 Sep 2021 20:31:07 +0000
Received: by outflank-mailman (input) for mailman id 198417;
 Tue, 28 Sep 2021 20:31:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJkn-0004O5-8a
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:31:05 +0000
Received: from mail-pj1-x1029.google.com (unknown [2607:f8b0:4864:20::1029])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1e1b3f39-e5a9-43d1-aa06-bc1981eaae04;
 Tue, 28 Sep 2021 20:30:53 +0000 (UTC)
Received: by mail-pj1-x1029.google.com with SMTP id kn18so2846331pjb.5
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:53 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:51 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 1e1b3f39-e5a9-43d1-aa06-bc1981eaae04
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=3cH5NMmmhhrKVlr9rQQI0c7DM+lnZiRUgrGaScpUArg=;
        b=MIwX4bL4cVoR5aRrDq6die7FAJDBGUpZdW1160GnVJ8yqc3g22G9120i7ZUT0ZlJCW
         GTE/j7wTTcWgOsbdmQQwA3g3GeKLZa3+aMkhnGMpN0NgPD9Sbo4FpyRqWjvnp363KTyS
         aiNoDK/AnsJGgB/ZjeIg4OcAGC59R8CkHpa3coNOVSXUI60uOF2YQyldH3hcrkmx/NaN
         Ls9sQKjAGKuyU8IgS8XIQmNXbL0LEIuorTSlPL7gvMFJ3TRdmoELARs0ubn3SwJvwCvD
         NZG8dNMy57Wua1xE63bM7SUL8NPSdF8W1Wzs8LD2/KqKdmRbrBVerAfR4mY9hDQf/TSA
         Tqng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=3cH5NMmmhhrKVlr9rQQI0c7DM+lnZiRUgrGaScpUArg=;
        b=vCeQn2UyLkpgW8ceEONqKrmZUgpxuxrGQ9q6ZpyNzp8jSOdlJGEHSD1909YPAsDMtF
         01fmd5QXpoCFlvJXOiUetdSFxLHgl/X9AFAd3bcYNBuXlZRdJzLT83mB9+63bKtdpjuy
         os4p2tknOBXGdfUNGwJTJNIAYuQOTqgkiE+aZSzUjBPq6AapFuB4WPpvsdjStr63t/z6
         sicMtrQJt762iyTwhsyZZC0s2qvTvkAA9japv2Dira1WzuYJ2u6gFgYBtRXLngGlXcgH
         MKG64p3bl3ArKmvd/6fAMhUYhaAX/Pl6KB+hQ9AncGVW+uapWB08gh8oyVLUYEB1uK19
         qq4g==
X-Gm-Message-State: AOAM5329xsZeTmxodGgitdFtCdoxrOROYDuOQC0ZNQn4bh4eSKb7Q3TM
	87/RsjrOuqy6YKwoIcJDe2/Rcsoq0t1MVqdl
X-Google-Smtp-Source: ABdhPJxlhP9wJVBhJuh8dHrjEZf/bP4hh4MfpGHaHA7u6jZFR1drBojqRdGgD/wzzQbgbQQvae6JtQ==
X-Received: by 2002:a17:902:7c17:b0:13e:2dd5:e5c4 with SMTP id x23-20020a1709027c1700b0013e2dd5e5c4mr6932711pll.68.1632861052219;
        Tue, 28 Sep 2021 13:30:52 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 3/6] arch/x86: rename debug.c to gdbsx.c
Date: Tue, 28 Sep 2021 13:30:26 -0700
Message-Id: <389ae979063a4afc38d8dbadaf539e7f411a24ba.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This commit renames debug.c to gdbsx.c to clarify its purpose.

The function gdbsx_guest_mem_io() is moved from domctl.c to gdbsx.c.

Although gdbsx_guest_mem_io() is conditionally removed from its single
call site in domctl.c upon !CONFIG_GDBSX and so no stub is technically
necessary, this commit adds a stub that would preserve the functioning
of that call site if the #ifdef CONFIG_GDBSX were to ever be removed or
the function were to ever be called outside of such an ifdef block.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
Changes in v4:
- Alphebetize Makefile addition
- Fix broken header guard
- Include errno.h in gdbsx.h

 xen/arch/x86/Makefile             |  2 +-
 xen/arch/x86/domctl.c             | 12 +-----------
 xen/arch/x86/{debug.c => gdbsx.c} | 12 ++++++++++--
 xen/include/asm-x86/debugger.h    |  6 ------
 xen/include/asm-x86/gdbsx.h       | 19 +++++++++++++++++++
 5 files changed, 31 insertions(+), 20 deletions(-)
 rename xen/arch/x86/{debug.c => gdbsx.c} (93%)
 create mode 100644 xen/include/asm-x86/gdbsx.h

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index fe38cfd544..9fa2ea9aa1 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -20,7 +20,6 @@ obj-y += cpuid.o
 obj-$(CONFIG_PV) += compat.o
 obj-$(CONFIG_PV32) += x86_64/compat.o
 obj-$(CONFIG_KEXEC) += crash.o
-obj-$(CONFIG_GDBSX) += debug.o
 obj-y += delay.o
 obj-y += desc.o
 obj-bin-y += dmi_scan.init.o
@@ -32,6 +31,7 @@ obj-y += emul-i8254.o
 obj-y += extable.o
 obj-y += flushtlb.o
 obj-$(CONFIG_CRASH_DEBUG) += gdbstub.o
+obj-$(CONFIG_GDBSX) += gdbsx.o
 obj-y += hypercall.o
 obj-y += i387.o
 obj-y += i8259.o
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 26a76d2be9..a492fe140e 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -20,6 +20,7 @@
 #include <xen/console.h>
 #include <xen/iocap.h>
 #include <xen/paging.h>
+#include <asm/gdbsx.h>
 #include <asm/irq.h>
 #include <asm/hvm/emulate.h>
 #include <asm/hvm/hvm.h>
@@ -33,20 +34,9 @@
 #include <public/vm_event.h>
 #include <asm/mem_sharing.h>
 #include <asm/xstate.h>
-#include <asm/debugger.h>
 #include <asm/psr.h>
 #include <asm/cpuid.h>
 
-#ifdef CONFIG_GDBSX
-static int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
-{
-    iop->remain = dbg_rw_mem(iop->gva, guest_handle_from_ptr(iop->uva, void),
-                             iop->len, domid, iop->gwr, iop->pgd3val);
-
-    return iop->remain ? -EFAULT : 0;
-}
-#endif
-
 static int update_domain_cpu_policy(struct domain *d,
                                     xen_domctl_cpu_policy_t *xdpc)
 {
diff --git a/xen/arch/x86/debug.c b/xen/arch/x86/gdbsx.c
similarity index 93%
rename from xen/arch/x86/debug.c
rename to xen/arch/x86/gdbsx.c
index d90dc93056..adea0f017b 100644
--- a/xen/arch/x86/debug.c
+++ b/xen/arch/x86/gdbsx.c
@@ -19,7 +19,7 @@
 #include <xen/mm.h>
 #include <xen/domain_page.h>
 #include <xen/guest_access.h>
-#include <asm/debugger.h>
+#include <asm/gdbsx.h>
 #include <asm/p2m.h>
 
 typedef unsigned long dbgva_t;
@@ -158,7 +158,7 @@ static unsigned int dbg_rw_guest_mem(struct domain *dp, unsigned long addr,
  * pgd3: value of init_mm.pgd[3] in guest. see above.
  * Returns: number of bytes remaining to be copied.
  */
-unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
+static unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
                         unsigned int len, domid_t domid, bool toaddr,
                         uint64_t pgd3)
 {
@@ -174,6 +174,14 @@ unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
     return len;
 }
 
+int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
+{
+    iop->remain = dbg_rw_mem(iop->gva, guest_handle_from_ptr(iop->uva, void),
+                             iop->len, domid, iop->gwr, iop->pgd3val);
+
+    return iop->remain ? -EFAULT : 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-x86/debugger.h b/xen/include/asm-x86/debugger.h
index cd6b9477f7..ed4d5c829b 100644
--- a/xen/include/asm-x86/debugger.h
+++ b/xen/include/asm-x86/debugger.h
@@ -54,10 +54,4 @@ static inline bool debugger_trap_fatal(
 
 #endif
 
-#ifdef CONFIG_GDBSX
-unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
-                        unsigned int len, domid_t domid, bool toaddr,
-                        uint64_t pgd3);
-#endif
-
 #endif /* __X86_DEBUGGER_H__ */
diff --git a/xen/include/asm-x86/gdbsx.h b/xen/include/asm-x86/gdbsx.h
new file mode 100644
index 0000000000..473229a7fb
--- /dev/null
+++ b/xen/include/asm-x86/gdbsx.h
@@ -0,0 +1,19 @@
+#ifndef __X86_GDBX_H__
+#define __X86_GDBX_H__
+
+#include <xen/errno.h>
+
+#ifdef CONFIG_GDBSX
+
+int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop);
+
+#else
+
+static inline int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
+{
+    return -EOPNOTSUPP;
+}
+
+#endif
+
+#endif
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:31:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:31:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198418.351933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkt-0005sK-OG; Tue, 28 Sep 2021 20:31:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198418.351933; Tue, 28 Sep 2021 20:31:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkt-0005s9-Kl; Tue, 28 Sep 2021 20:31:11 +0000
Received: by outflank-mailman (input) for mailman id 198418;
 Tue, 28 Sep 2021 20:31:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJks-0004O5-8v
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:31:10 +0000
Received: from mail-pf1-x42b.google.com (unknown [2607:f8b0:4864:20::42b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 78580ae5-8e8d-4452-b28d-b3fc6525a363;
 Tue, 28 Sep 2021 20:30:54 +0000 (UTC)
Received: by mail-pf1-x42b.google.com with SMTP id n23so15363pfv.4
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:54 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:53 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 78580ae5-8e8d-4452-b28d-b3fc6525a363
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=l4OVntz5YtvdVmlmtqYpHPgYE8ss+lH0o06oBOChZro=;
        b=q8F07rbAtlARTMyYiUPOm4yjeaBuF2S8aiD0hqdDsNYCRCZVqvGispWehmLcZxVwVl
         AsBpzR8uUaYwyaUqYDCWEupSDoj19lYbk4IX1Z55FLLzbXLjfi3eNTl+RIMQ/3cmSn/k
         HRbnbHHsXxFrtUREZRcFIO58VxwQC5tnT6TneKpocP8rVM451qHDvrTG2Zl+On5zBMpb
         cRWF5o2o+qw3H/ZNgtGDIZOwXOs0MOYQLFpawLWi0HNbHQhyikWtekMlLoi25EDmT9iy
         5hFIt0LxoumBW4XJoqjG5NiQt7b6Rn66usK/E5vQ4s9+mZGUL0E/tda4FAWlNflo6Y69
         YUtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=l4OVntz5YtvdVmlmtqYpHPgYE8ss+lH0o06oBOChZro=;
        b=IgaBcWmhA/eLS378Xr72vvqF0M2ZGm2r7s6DcMBP/cnMFSnI8ee7QUXs0njIbWxhIJ
         NQmzol5cDGX1ONZNE3ndyIrHwX6AIhgaEpSCWtuNctmZvf3gUaLtgkvHgfONLV587uqj
         aq9T0YDmA074PkXubZBPNR3wnLIE6FJtH1JYBLNhmKdPXTwsPzv0iywnre+0KDq1FKAa
         TgftaMhloqKCt790PB+lY+BqeNRtAYuo5z4hnjrWQpLGb/YEuEZdwGsqA9za4G46/1WJ
         jZPQ7eqZnQkMAsB3xEcIB2ltO5CjnXEIhVSV+kRautvkvz5dCxyp54iDU0sZx7F57U3u
         he6Q==
X-Gm-Message-State: AOAM532WuBH5C9OOVCVN5zJkt0NGm4kw9xLZk5JLFzmXavwLzidrtURo
	5P/2Ev7vAywGtcxQOi9mtj1lsj2LFH7DxCxs
X-Google-Smtp-Source: ABdhPJzKTkAV1Mkw5cnVcJddv8t7cBFdUB0I737XGLV/GUex1rPODiz9G01QHQf0VBWiE0Wt3nqEjg==
X-Received: by 2002:a63:24a:: with SMTP id 71mr6307537pgc.285.1632861053496;
        Tue, 28 Sep 2021 13:30:53 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 4/6] x86/gdbsx: expand dbg_rw_mem() inline
Date: Tue, 28 Sep 2021 13:30:27 -0700
Message-Id: <472215cdfa754634f8070386bb3122bf3e3cedc1.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Because dbg_rw_mem() has only a single call site, this commit
expands it inline.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
Changes in v4:
- Add DCO

 xen/arch/x86/gdbsx.c | 30 +++++++++---------------------
 1 file changed, 9 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/gdbsx.c b/xen/arch/x86/gdbsx.c
index adea0f017b..9c8827c6c4 100644
--- a/xen/arch/x86/gdbsx.c
+++ b/xen/arch/x86/gdbsx.c
@@ -151,33 +151,21 @@ static unsigned int dbg_rw_guest_mem(struct domain *dp, unsigned long addr,
     return len;
 }
 
-/*
- * addr is guest addr
- * buf is debugger buffer.
- * if toaddr, then addr = buf (write to addr), else buf = addr (rd from guest)
- * pgd3: value of init_mm.pgd[3] in guest. see above.
- * Returns: number of bytes remaining to be copied.
- */
-static unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
-                        unsigned int len, domid_t domid, bool toaddr,
-                        uint64_t pgd3)
+int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
 {
     struct domain *d = rcu_lock_domain_by_id(domid);
 
-    if ( d )
+    if ( d && !d->is_dying )
     {
-        if ( !d->is_dying )
-            len = dbg_rw_guest_mem(d, gva, buf, len, toaddr, pgd3);
-        rcu_unlock_domain(d);
+        iop->remain = dbg_rw_guest_mem(
+                d, iop->gva, guest_handle_from_ptr(iop->uva, void),
+                iop->len, domid, iop->pgd3val);
     }
+    else
+        iop->remain = iop->len;
 
-    return len;
-}
-
-int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
-{
-    iop->remain = dbg_rw_mem(iop->gva, guest_handle_from_ptr(iop->uva, void),
-                             iop->len, domid, iop->gwr, iop->pgd3val);
+    if ( d )
+        rcu_unlock_domain(d);
 
     return iop->remain ? -EFAULT : 0;
 }
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:31:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:31:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198420.351944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJky-0006H9-41; Tue, 28 Sep 2021 20:31:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198420.351944; Tue, 28 Sep 2021 20:31:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJkx-0006Gl-Ta; Tue, 28 Sep 2021 20:31:15 +0000
Received: by outflank-mailman (input) for mailman id 198420;
 Tue, 28 Sep 2021 20:31:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJkx-0004O5-9D
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:31:15 +0000
Received: from mail-pj1-x102b.google.com (unknown [2607:f8b0:4864:20::102b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3abad95c-4747-4e64-8b3d-839af013ab0a;
 Tue, 28 Sep 2021 20:30:55 +0000 (UTC)
Received: by mail-pj1-x102b.google.com with SMTP id
 on12-20020a17090b1d0c00b001997c60aa29so2589570pjb.1
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:55 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: 3abad95c-4747-4e64-8b3d-839af013ab0a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=iOTjHkk69+BVgT4XjnVLkopnx7Nv/jMSlcVqlEzwOf4=;
        b=ixGQGSdM1o7e/kVIG+47VRcbnwP/vXS0o8jMQexxab0GzYf8PJNNWViiyQKF52ALca
         j4oGXHHOTtMJzG5g/hLNhLFXTERigp4f+CI2c3e2jHPdxPnjKffk5O+HONJ3NHExE1iT
         ASJZTXQnJ0KGEZoDGy+1nw1ikTkNyxO7RsXqbJrVygFa84P2DxMDgGV6IvyUOZIy2SxH
         8NqzI5MVwg8mOtj2ASKCAKWVxWdIg93BrmAPu828QfCrm7PF3et0N24cubRHJ+reX2wU
         sFmAcm0iJ1R5Rr7T2rgqIoOrfkm1tpl61hqGbGwH+0NrVbiKa4PcXY0WA2NI49/1R08M
         nTuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=iOTjHkk69+BVgT4XjnVLkopnx7Nv/jMSlcVqlEzwOf4=;
        b=d6tCFfaHEhxiEM1XI7AKx3oNc/JTSXMwNnWzM1kIioWBe5gUzFb/QxTKDD8cjI51tR
         Gx54k1pUi2Uc6Q0PKxGWMwJmoz0ljFqeZMGZFKoJBzZnSb7vOtriWKKJEO6iwE48Fous
         Wju90TSo7gLR4UcseRocEc5tzp0mIb+sMn+cnjkDmWyyS9KQV+ZBihIiUTjpyQ1gP2J2
         KFP+iCZWJbaOMbIgZiOoe202BKZLilVlbVBnO9/HZadsBPFKi+S20wMDsxFZpst4dqXC
         EpXSdAOW53lXA4AGOJzu7BVDjRMQ7cxEZV4pZdCZsjxvE5lEUyVXzA+twPmR/Iu0/COR
         s10g==
X-Gm-Message-State: AOAM53305xQd0LINvXmensioMEeiAyZvUj88ecK14ykhww+N5Ryc1XbC
	Z6EiRKSzdOCy9kMG4IrHV4OBtpqqKHHN3C99
X-Google-Smtp-Source: ABdhPJymLtx9yWhdUPkYTejlM8Zfr8ITa+y4p33yqB84Ts+javy1zYt7RsZzAu8n9wjVBBspMqjfpg==
X-Received: by 2002:a17:902:bf8c:b0:13e:2b58:c2af with SMTP id v12-20020a170902bf8c00b0013e2b58c2afmr6916505pls.74.1632861054943;
        Tue, 28 Sep 2021 13:30:54 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 5/6] arch/x86: move domain_pause_for_debugger() to domain.h
Date: Tue, 28 Sep 2021 13:30:28 -0700
Message-Id: <5de82c39ffae1d65c8ab7142b05d304f9d4ddae9.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

domain_pause_for_debugger() was previously in debugger.h.  This commit
moves it to domain.h because its implementation is in domain.c.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
Changes in v3:
- domain_pause_for_debugger() is now moved into debugger.h, not a new
  file debugger.c

Changes in v4:
- Don't unnecessarily include <asm/domain.h>

 xen/arch/x86/nmi.c             | 1 -
 xen/arch/x86/traps.c           | 1 -
 xen/include/asm-x86/debugger.h | 2 --
 xen/include/asm-x86/domain.h   | 2 ++
 4 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index ab94a96c4d..11d5f5a917 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -30,7 +30,6 @@
 #include <asm/msr.h>
 #include <asm/mpspec.h>
 #include <asm/nmi.h>
-#include <asm/debugger.h>
 #include <asm/div64.h>
 #include <asm/apic.h>
 
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 772e2a5bfc..742fa9e2ca 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -62,7 +62,6 @@
 #include <asm/uaccess.h>
 #include <asm/i387.h>
 #include <asm/xstate.h>
-#include <asm/debugger.h>
 #include <asm/msr.h>
 #include <asm/nmi.h>
 #include <asm/xenoprof.h>
diff --git a/xen/include/asm-x86/debugger.h b/xen/include/asm-x86/debugger.h
index ed4d5c829b..8f6222956e 100644
--- a/xen/include/asm-x86/debugger.h
+++ b/xen/include/asm-x86/debugger.h
@@ -26,8 +26,6 @@
 #include <asm/regs.h>
 #include <asm/processor.h>
 
-void domain_pause_for_debugger(void);
-
 #ifdef CONFIG_CRASH_DEBUG
 
 #include <xen/gdbstub.h>
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 92d54de0b9..de854b5bfa 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -672,6 +672,8 @@ void update_guest_memory_policy(struct vcpu *v,
 
 void domain_cpu_policy_changed(struct domain *d);
 
+void domain_pause_for_debugger(void);
+
 bool update_runstate_area(struct vcpu *);
 bool update_secondary_system_time(struct vcpu *,
                                   struct vcpu_time_info *);
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:31:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:31:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198423.351955 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJl3-0006tJ-FP; Tue, 28 Sep 2021 20:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198423.351955; Tue, 28 Sep 2021 20:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVJl3-0006t8-Bs; Tue, 28 Sep 2021 20:31:21 +0000
Received: by outflank-mailman (input) for mailman id 198423;
 Tue, 28 Sep 2021 20:31:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=sVw8=OS=gmail.com=bobby.eshleman@srs-us1.protection.inumbo.net>)
 id 1mVJl2-0004O5-9I
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:31:20 +0000
Received: from mail-pf1-x42f.google.com (unknown [2607:f8b0:4864:20::42f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ad3a4beb-484e-4986-b3e6-d1314893e9a2;
 Tue, 28 Sep 2021 20:30:57 +0000 (UTC)
Received: by mail-pf1-x42f.google.com with SMTP id w19so19832006pfn.12
 for <xen-devel@lists.xenproject.org>; Tue, 28 Sep 2021 13:30:57 -0700 (PDT)
Received: from localhost.localdomain ([2601:1c2:4c02:4c00::5])
 by smtp.gmail.com with ESMTPSA id p15sm44201pfn.31.2021.09.28.13.30.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 28 Sep 2021 13:30:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
X-Inumbo-ID: ad3a4beb-484e-4986-b3e6-d1314893e9a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=sender:from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=er5faQ/st6GZye51adynMEU5HSlpMR7iwTJec4b9TtQ=;
        b=o85TT2UfMEB+cmfXfH4HpYaImrks1IEaD2YggbPCmcwkX9St2K8GM9HI9JyW2Yj0Hr
         g0p6meZ1346IppM4mS0I7rXJ+KGkorCm3594/SQAQJafbuORffI4kzanmc9pHNITfxXJ
         sLvvznt1/Qsd4TCUP/kYJJ57eigIqtg2QZEE5aNc9xzPFQPHB2aCHovOND/TMOZbV0yd
         zjM512H29H+ooNuLUMCnM3vPcv2Mxmwa4spCgR4Jnunzu6/AwzQncwF7a66b7bhrKAHz
         tt5J2vdvHVroEuJA8lnCs43rPN8LS+bV777aP3kTnxzx2OsskhcR7kS5mzQr9n1j+4Pq
         1vmw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:sender:from:to:cc:subject:date:message-id
         :in-reply-to:references:mime-version:content-transfer-encoding;
        bh=er5faQ/st6GZye51adynMEU5HSlpMR7iwTJec4b9TtQ=;
        b=YVKH3UyHBRAm+SfliQJxU6wre/HxVD2NBKxmQO/KhITvfASJ880Ng3GReD1MlI+L9j
         QSvRyLlZUP4P7/2YwRTPXQ8/ZQFbpSeugksh0KUpNSAwrPFoWAlrlzI0V2RZib1ph+k0
         zSBINFqPYrIrTGd5DqR4z4gOdklS7Br7Lb3HnoGmqua4ftLdmENv4WzJ2Fondw73xYKp
         f8xPmYEzYDRX7Sl6gD/21cNEHv13hf4cONl5fjoTDHd/FuOb/r+SmekT3VD1uKhntNOw
         hSq1M6oJcCWflelUicqmSjqk+ZwBQj/E2gWU0p7SGQo2+8DbW5EqIEodRZV3rhzcPtyw
         eGwg==
X-Gm-Message-State: AOAM533fo6CucR60qcYs8tgypfVw8VG52Al4wF0k2ZT+W+dswG0dPRCi
	DI+eu1AVs161bVLYRCLlfFVnEV/lSlqvCykI
X-Google-Smtp-Source: ABdhPJxiI0TvKS/plq0MCTX0bETK0mqEoV9I6fx4ued2x3fw34vZ90o7HgedhDr+VfoqMwzxvzdthg==
X-Received: by 2002:a63:3408:: with SMTP id b8mr6418520pga.340.1632861056227;
        Tue, 28 Sep 2021 13:30:56 -0700 (PDT)
Sender: Bobby Eshleman <bobbyeshleman@gmail.com>
From: Bobby Eshleman <bobby.eshleman@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobby.eshleman@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Elena Ufimtseva <elena.ufimtseva@oracle.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>
Subject: [PATCH v4 6/6] x86: change asm/debugger.h to xen/debugger.h
Date: Tue, 28 Sep 2021 13:30:29 -0700
Message-Id: <8c340b24b41af349069ad55e1f0421e17fad1d6f.1632860589.git.bobby.eshleman@gmail.com>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <cover.1632860589.git.bobby.eshleman@gmail.com>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This commit allows non-x86 architecture to omit the file asm/debugger.h
if they do not require it.  It changes debugger.h to be a general
xen/debugger.h which, if CONFIG_CRASH_DEBUG, resolves to include
asm/debugger.h.

It also changes all asm/debugger.h includes to xen/debugger.h.

Because it is no longer required, arm/debugger.h is removed.

Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>
---
Changes in v4:
- Replace #include <asm/regs.h> with `struct cpu_user_regs`

 xen/arch/x86/traps.c           |  1 +
 xen/common/domain.c            |  2 +-
 xen/common/gdbstub.c           |  2 +-
 xen/common/keyhandler.c        |  2 +-
 xen/common/shutdown.c          |  2 +-
 xen/drivers/char/console.c     |  2 +-
 xen/include/asm-arm/debugger.h | 15 ----------
 xen/include/asm-x86/debugger.h | 15 ----------
 xen/include/xen/debugger.h     | 51 ++++++++++++++++++++++++++++++++++
 9 files changed, 57 insertions(+), 35 deletions(-)
 delete mode 100644 xen/include/asm-arm/debugger.h
 create mode 100644 xen/include/xen/debugger.h

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 742fa9e2ca..36d7fc6238 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -35,6 +35,7 @@
 #include <xen/shutdown.h>
 #include <xen/guest_access.h>
 #include <asm/regs.h>
+#include <xen/debugger.h>
 #include <xen/delay.h>
 #include <xen/event.h>
 #include <xen/spinlock.h>
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6b71c6d6a9..a87d814b38 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -11,6 +11,7 @@
 #include <xen/err.h>
 #include <xen/param.h>
 #include <xen/sched.h>
+#include <xen/debugger.h>
 #include <xen/domain.h>
 #include <xen/mm.h>
 #include <xen/event.h>
@@ -33,7 +34,6 @@
 #include <xen/xenoprof.h>
 #include <xen/irq.h>
 #include <xen/argo.h>
-#include <asm/debugger.h>
 #include <asm/p2m.h>
 #include <asm/processor.h>
 #include <public/sched.h>
diff --git a/xen/common/gdbstub.c b/xen/common/gdbstub.c
index 848c1f4327..1d7b98cdac 100644
--- a/xen/common/gdbstub.c
+++ b/xen/common/gdbstub.c
@@ -38,12 +38,12 @@
 #include <xen/serial.h>
 #include <xen/irq.h>
 #include <xen/watchdog.h>
-#include <asm/debugger.h>
 #include <xen/init.h>
 #include <xen/param.h>
 #include <xen/smp.h>
 #include <xen/console.h>
 #include <xen/errno.h>
+#include <xen/debugger.h>
 #include <xen/delay.h>
 #include <asm/byteorder.h>
 
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 8b9f378371..1eafaef9b2 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -3,6 +3,7 @@
  */
 
 #include <asm/regs.h>
+#include <xen/debugger.h>
 #include <xen/delay.h>
 #include <xen/keyhandler.h>
 #include <xen/param.h>
@@ -20,7 +21,6 @@
 #include <xen/mm.h>
 #include <xen/watchdog.h>
 #include <xen/init.h>
-#include <asm/debugger.h>
 #include <asm/div64.h>
 
 static unsigned char keypress_key;
diff --git a/xen/common/shutdown.c b/xen/common/shutdown.c
index abde48aa4c..a933ee001e 100644
--- a/xen/common/shutdown.c
+++ b/xen/common/shutdown.c
@@ -2,13 +2,13 @@
 #include <xen/lib.h>
 #include <xen/param.h>
 #include <xen/sched.h>
+#include <xen/debugger.h>
 #include <xen/domain.h>
 #include <xen/delay.h>
 #include <xen/watchdog.h>
 #include <xen/shutdown.h>
 #include <xen/console.h>
 #include <xen/kexec.h>
-#include <asm/debugger.h>
 #include <public/sched.h>
 
 /* opt_noreboot: If true, machine will need manual reset on error. */
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 7d0a603d03..3d1cdde821 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -15,6 +15,7 @@
 #include <xen/init.h>
 #include <xen/event.h>
 #include <xen/console.h>
+#include <xen/debugger.h>
 #include <xen/param.h>
 #include <xen/serial.h>
 #include <xen/softirq.h>
@@ -26,7 +27,6 @@
 #include <xen/kexec.h>
 #include <xen/ctype.h>
 #include <xen/warning.h>
-#include <asm/debugger.h>
 #include <asm/div64.h>
 #include <xen/hypercall.h> /* for do_console_io */
 #include <xen/early_printk.h>
diff --git a/xen/include/asm-arm/debugger.h b/xen/include/asm-arm/debugger.h
deleted file mode 100644
index ac776efa78..0000000000
--- a/xen/include/asm-arm/debugger.h
+++ /dev/null
@@ -1,15 +0,0 @@
-#ifndef __ARM_DEBUGGER_H__
-#define __ARM_DEBUGGER_H__
-
-#define debugger_trap_fatal(v, r) (0)
-#define debugger_trap_immediate() ((void) 0)
-
-#endif /* __ARM_DEBUGGER_H__ */
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/include/asm-x86/debugger.h b/xen/include/asm-x86/debugger.h
index 8f6222956e..b9eeed395c 100644
--- a/xen/include/asm-x86/debugger.h
+++ b/xen/include/asm-x86/debugger.h
@@ -25,9 +25,6 @@
 #include <xen/sched.h>
 #include <asm/regs.h>
 #include <asm/processor.h>
-
-#ifdef CONFIG_CRASH_DEBUG
-
 #include <xen/gdbstub.h>
 
 static inline bool debugger_trap_fatal(
@@ -40,16 +37,4 @@ static inline bool debugger_trap_fatal(
 /* Int3 is a trivial way to gather cpu_user_regs context. */
 #define debugger_trap_immediate() __asm__ __volatile__ ( "int3" );
 
-#else
-
-static inline bool debugger_trap_fatal(
-    unsigned int vector, struct cpu_user_regs *regs)
-{
-    return false;
-}
-
-#define debugger_trap_immediate() ((void)0)
-
-#endif
-
 #endif /* __X86_DEBUGGER_H__ */
diff --git a/xen/include/xen/debugger.h b/xen/include/xen/debugger.h
new file mode 100644
index 0000000000..ddaa4a938b
--- /dev/null
+++ b/xen/include/xen/debugger.h
@@ -0,0 +1,51 @@
+/******************************************************************************
+ * Generic hooks into arch-dependent Xen.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Each debugger should define two functions here:
+ *
+ * 1. debugger_trap_fatal():
+ *  Called when Xen is about to give up and crash. Typically you will use this
+ *  hook to drop into a debug session. It can also be used to hook off
+ *  deliberately caused traps (which you then handle and return non-zero).
+ *
+ * 2. debugger_trap_immediate():
+ *  Called if we want to drop into a debugger now.  This is essentially the
+ *  same as debugger_trap_fatal, except that we use the current register state
+ *  rather than the state which was in effect when we took the trap.
+ *  For example: if we're dying because of an unhandled exception, we call
+ *  debugger_trap_fatal; if we're dying because of a panic() we call
+ *  debugger_trap_immediate().
+ */
+
+#ifndef __XEN_DEBUGGER_H__
+#define __XEN_DEBUGGER_H__
+
+#ifdef CONFIG_CRASH_DEBUG
+
+#include <asm/debugger.h>
+
+#else
+
+struct cpu_user_regs;
+
+static inline bool debugger_trap_fatal(
+    unsigned int vector, const struct cpu_user_regs *regs)
+{
+    return false;
+}
+
+static inline void debugger_trap_immediate(void)
+{
+}
+
+#endif /* CONFIG_CRASH_DEBUG */
+
+#endif /* __XEN_DEBUGGER_H__ */
-- 
2.32.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 20:53:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 20:53:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198461.351965 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVK6K-0002JC-9q; Tue, 28 Sep 2021 20:53:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198461.351965; Tue, 28 Sep 2021 20:53:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVK6K-0002J5-6X; Tue, 28 Sep 2021 20:53:20 +0000
Received: by outflank-mailman (input) for mailman id 198461;
 Tue, 28 Sep 2021 20:53:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVK6J-0002Iz-65
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 20:53:19 +0000
Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6efac599-38b6-44ab-a6a3-fbaf65b5f0e2;
 Tue, 28 Sep 2021 20:53:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6efac599-38b6-44ab-a6a3-fbaf65b5f0e2
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632862397;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=o/gQsaBCDL+vcmmpLo9NMplWGRLceVYuOJOdZyPcfrU=;
  b=Sere78e1mvIYszIU/nmckDghz1ZT9KA/ANSZrIG14qhhtg6CELDAV2QD
   vIQoKc9xBwrLD7gtBfkU4k2aKq6wuX+8pb6TVcOp2KoZQfTBSa8GKYqDW
   rdtU3kNrvWQV7PFeEKSGbL+vEhIS41CGjIEm/sWb8O4tfu4ckjFTKqiIu
   8=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: PnVj/hs2/wCpLn3pVSkJUBkVqrRR2+CxQYukDWP/aWkuRHBzGyUBKjmoYB7FKzJSynrt2G0caU
 CpkZ8veUi34LP8iRYGam2jba0rczK5U4CLLZci1zU7zZN+zpu4fWNaqPBYE9VfkW2EvxnJW1kP
 ShXtQCuAlIVflInYy+po2aagO6ePpTPWDiKLw6ZElIXC1AWkN58d6hnD+ob6yDzSXkbg0YjtKe
 DLE1j0KagEuzDZlxWa7XgxkkhaO9wk4bZUaPup4vT8CM0/k6RFWg4yc7lg142d7KJFP5qaCD/4
 NREx+Hdgv8E/ZQ0aWLdPiNJo
X-SBRS: 5.1
X-MesageID: 55685815
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KsWEWaC70XfVjhVW/13lw5YqxClBgxIJ4kV8jS/XYbTApGsigWYAn
 TdMCz+CPvncNmrze9gjPojjpx5T7ceAm9JnQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00I7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/ljiwuYB1i
 8Rxtt+1U1Y2HKfQwNslakwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFgWlg25AXQJ4yY
 eIoax1rRRn/SiFAO1ENJYhjk/+53VvGJmgwRFW9+vNsvjm7IBZK+KP2LNPfd9iORMNUtkWVv
 GTL+yL+GB5yHN6CzTuI9Fq8i+mJmjn0MKoYE7ey57hnm12Xy2kaDjUZUFK6pb+yjUvWc8JSL
 QkY9zQjqYA29Ve3VZ/tUhugunmGsxUAHd1KHIUS1gaJzabF5heDMUINRDVBdd8Oudc/QHoh0
 Vrht97tCDpm9qCbSHeB7bqKpBu1IyEeKWJEbigBJSME/NvLsIw1lgjITNtoDOiylNKdMTP6z
 i2OrSM+r64OlsNN3KK+lXjYhxq8q56PSRQ6jjg7RUr8sFk/PtT8IdX1tx6Ltp6sMbp1UHGcs
 UEiy8zd/dwNTpSXxTOvSvlVI4mQsqPt3CLnvXZjGJwo9jKI8nGlfJxN7DwWGHqFIvroahezP
 haM6Fo5CIt7eSvyNPArM+pdHuxwlcDd+cLZuuc4hzakSrZ2bhPP2ChzaUOK02npnSDAeolkY
 szDIK5A4ZsAYJmLLQZapc9BjdfHJQhknAs/oKwXKTz8itJyg1bPFd843KOmNLxR0U99iFy9H
 yxjH8WL0Q5Dd+b1fzPa94UeRXhTcyNhWs6m8ZUNKrfYSuaDJI3HI6SKqV/GU9Y595m5a8+Sp
 i3tMqOm4ACXaYL7xfWiNSk4NeKHsWdXpnMnJy08VWtEKFB5CbtDGJw3LsNtFZF+rbQL5actE
 5EtJpXRatwSG2+v02lMMvHAQHlKKU3DafSmZHH+PlDSvvdIGmT0xzMTVlG0qXZSXnbp75tWT
 n/J/lqzfKfvjj9KVa7+QPmu00mwrT4anudzVFHPOd5dZAPn940CFsA7pqZmSy3VARmclDacy
 SiMBhIU+bvEr4MvqYGbjqGYtYa5VeB5GxMCTWXc6L+3Mwjc/3aintAcALrZI2iFWTOm4rima
 MVU0+r4bK8NkmFVvtcuCL1s168/uYfi/ucI0gR+EXzXRF23Ebc8cGKe1MxCu/QVlL9UsAe7Q
 GyV/dxeNenbMc/pCgdJdgEkcv6CxbcfnTyLtaY5J0Dz5SlW+rubUBoNY0nQ2XIFdLYsadEr2
 +YsvsIS+jeTsBtyP4bUlD1Q+kSNMmcED/ctuKYFDdK5kQEs0FxDP8DRU3ek/JGVZtxQGUA2O
 TvI1rHajrFRy0eeIXo+EX/BgbhUiZgU4U0YyVYDIxKCm8bfh+9x1xpUqGxlQgNQxxRB8uRyJ
 mk0aBElefTQp29l1JpZQmShOwBdHxnIqEX+xmwAmHDdU0T1BHfGK3cwOLrV8U0Um46GkuO3I
 F1MJL7ZbAvX
IronPort-HdrOrdr: A9a23:iJUiEqxLz+fhe383aqGLKrPwKL1zdoMgy1knxilNoHtuA6ulfq
 GV7ZAmPHrP4wr5N0tNpTntAsa9qBDnlaKdg7N+AV7KZmCP0gaVxepZjLfK8nnNHDD/6/4Y9Y
 oISdkaNDQoNykYsS8t2njbL+od
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="55685815"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d6OsxfCaspSy50YuOaIS66Vt4WurlNCR0ynH234Ww9GkPlczb7PNpib29qcuE8VCkFFw19Sl/p1bMuzTJ9c2bgClu/HlZykmThQzSqnrllctT3B4tBb6rQSBJmjlx9fRxuqgeFdNAgt83kI+f9pDHiP18E6w3X3U50RLeQ4b/qV4oUs6xpwFCx2IL4vDPaRrav6RRTzyZUACI1XIuQj8hfsesGOmQSvZ36dV52oGF8L5xnQQGdFHeMOhhHistNOpNZ6Pq3lXtFmKB/SwsG4iWRywZdlRBhz0ckNHJBjip58eRVSPJlCgjXI2Q9pG/J+8KBBShZTOUFuCYIj4RT+04g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eJvj6b8iMf2UBiRV3R4j4nZib8IPWx0Hcuyv/lqleVE=;
 b=Jx61mVOqyWl/JrCh+tA9oiThG5b14IuZGzk9aFzqSfwLJ6Y6pITGfdAkFd6bnYfnMUIdXaZGhabDXmrX8w2gqo0cCOsD5yBcQvcknDag/cEb5eiaA3TLK9mLSeiqEX8f+UiTxIeEc5oUAVyS1hyuPG8K7L01h1RSe50IJmQzDlbuGgSPeRe2VsNoZmTcew0hLr/P9CB1jUdAYkYg6bI2uI/L2BbD6GCZGbTFaZNwjKwTBpn1FMQvaJdd6kRPRYJkSqOkhntWmujdas6XCgaPJDOFY/cmxlBgQu4EhVx846VEhhbYmjHMkbqu177BCFomJjzrBYhXci6ffsKYZXUfkA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eJvj6b8iMf2UBiRV3R4j4nZib8IPWx0Hcuyv/lqleVE=;
 b=jl4Ik3IQwu/G7gmkks5sjL1/AIRsn9xg/Y6qq1To7oody5iPzaN4yWcDkbGUrawczwY4dPBhXZ2uXMx8Y3CO6w7A5rQ5GM6weZYOwXgggQEdiVnlWabmCy1WKBENUS0oH3tBPLQUWwhk4EVFZsy9gpfDvSjZL6ERcDJC1YXEziI=
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, George
 Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <47e32bf1eb0cd18210df4273d564fa17d389f3ae.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 2/6] x86/debugger: separate Xen and guest debugging
 debugger_trap_* functions
Message-ID: <8a3e66c5-e724-7018-da4e-47b61951915f@citrix.com>
Date: Tue, 28 Sep 2021 21:52:48 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <47e32bf1eb0cd18210df4273d564fa17d389f3ae.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0200.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:9e::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ffb45135-6802-4870-a0b7-08d982c1f332
X-MS-TrafficTypeDiagnostic: BY5PR03MB4966:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB496695FFE7A2DAF59AEEACA6BAA89@BY5PR03MB4966.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: zpVdjAE4ZuID9AhvzI6O2uLqnRSbXmfT3QPT1GW2eaQXfvfu+LJt2oq5hFoubyuVTgSk2ahuDxWRVG830SeRHtOummZJRog0kHpI8h6hxZ3d3bkXUB+9BYXIbWkLJdusUom6rPyDdfJgxUDH+axYlA83mzmTg+3NKKY8VCPZDws3T9AyiQ5W/BTyXA2gXCCt9SWxDpJ4YlOb92FmEd59lFSIc+WSmU0zZHj1ayTAodZ5BmRB9qFWgLl/jjvBxubQ23bchcoGPWwHH8uyMKf2KxR5Z4gIOPZ3vV4RZab2X4ezyDRWrpYDDk09fLasV+AKSdnxSIuZbYHjEDSwtK806gXaTiM8vC83BZyqaJN3TnBDjbNIO18SjbZIvj7u+JJrLC6fjqQ6Qc8moWVgKkiRuBblNC+/OK53F0kEa8HTdOtqEaA8Zlt64xdYsxfHcYNfb6VumpiSmN3xhJ3nqmtwtuV2DOyM+AiTQHyGF1mr3ZSCEihpBG7kwQZP+8E6ljHXSI3XinAclob+g3ppS7FIuYRQhDMTk/9A3WopeLp+veIgOjpPPY3R0DgQ+axZPwsCA6MJiHkbCvEhgcB4tJjLtKIbOH0mA+SaaNq2rXEY4h5K2FeRpThkjgPmB72RMQ85AxLpF8PGjI6k96x/Go7umazTaPZQB0LI1iJHty6nA1QfyZwwyVn11XGQdBb+viEG2Ew26o4kAzXeKCa9NNqklvZLABN2r4UmK/086s4xCDc=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(31696002)(2616005)(8936002)(36756003)(5660300002)(508600001)(55236004)(31686004)(38100700002)(956004)(54906003)(66946007)(6666004)(4326008)(2906002)(186003)(53546011)(66556008)(8676002)(316002)(6486002)(16576012)(86362001)(66476007)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QzQrNE5ncndEM05pRDhJZDVqeWc1NzkyWmRnTXZvR0puTWJxSzJKN2tFZjBU?=
 =?utf-8?B?M2NZS2txNGJleE43UFU5T1ZMZjBzbVVsOTdNNzN4Y3NLWlBkVE5BZmdjM1V5?=
 =?utf-8?B?a0I5TFV5VGUyNE55bjUzV3JYZTFvclZmWXNUcWV3U0hoNi8zaHVrK2ZRSElr?=
 =?utf-8?B?dm1XekNQb3pudGtsWTBLRFExaXJkZ3N2eUNXWk9GbUVxb3pBZlByRG5wQXJu?=
 =?utf-8?B?M3NUUlhSUmlVZ0dOTFpOVkdEYUpkajdwSUVXMmZMK1l4VW1jV0k5bHpQUDEr?=
 =?utf-8?B?ZzBSQUpFSVBlc1Y3YlFxbUYvdU9tQ0MxK1NpRzRVeEd0RDhyNEQzbmJDQzd2?=
 =?utf-8?B?L04vRHBJYW5tWDd6eGk2ekJwOERybzdtSGVWL1VnaDl6TEpwb3RlbHJFUTJo?=
 =?utf-8?B?aDByN25YZWZKenVQMHQ4blZ2czlOQXJIdFU1Zzd1alI3YWtQV2l2VVBxZVFv?=
 =?utf-8?B?Z0VIdHQzMFgrV3M1SjRsTU10c25RR2hjN3k4YVZ0aktSdUdCZkhlQVJjZzll?=
 =?utf-8?B?SFlaRG1RMXhBUjkwcVFlRFY4VVVhODF1STU1WGczbmhvbk1BY21TMUtZSDJR?=
 =?utf-8?B?dUdmdXV3Zi9UWHB6b08rZTBLaHZQQTFKUXNucFJqcVVxbHpPNjU0VHRCTjZI?=
 =?utf-8?B?cVNQWlorREk1RCtFV3ZheXRFV2Z1UTZHMUNoVWZ1NlUyaVhPSE5saFNKdERm?=
 =?utf-8?B?a3czTVNHcTR4Z3Y0Q0cyclBnWWt0UGxjaHA3NFRMU2J0UUI1WHBGbFI4aWI5?=
 =?utf-8?B?MjFmeXNIdXl5WE9zVmVYSk0wZW1oUDFqYSt2OWJhNVdwV0ZsOGluMWtBeC9V?=
 =?utf-8?B?NkIwV2w2QnBRRDA1Z2w5VkxwV0xDdHFGbTRZSXdDZm5XSkgyY1V3SUlXUWNn?=
 =?utf-8?B?WUhrOXlYY1BUUW5BQ29HdHZINDVtNENDd1FYZFk1bjBTWXZLODFjVUdxTUth?=
 =?utf-8?B?VXg0T1Y3dGdnM1ZPS1o5RCt1MC93M3JBMjBEdERDOEVhd1U3ZVRPT2IzWFhU?=
 =?utf-8?B?UjU1eU1MY1FvVFpIeTRmdXVKb2svQ2IzbWdxTHQrMGEzQWNxNG5RbFJBZzJX?=
 =?utf-8?B?OXNKZ3hrY0Z5RVMzZ3Avc1BYNkVXVEkrOUtFaVpVMjl3VmRjdVhLUE9scjFw?=
 =?utf-8?B?UjV4UjdVRDhEekJTb3FwU1lZZjdtYXFWczFRbXBYdlduVFBJWlVsWnNabGVY?=
 =?utf-8?B?Z0xCSnQwY1d2VFVOVHdXaUxiaEE3RHZNZjFrVUd3SWR6M0I2Y0MwRklCb0Nl?=
 =?utf-8?B?cW96dGMzRmVGa1hGaWtsclBQd0lkcWJtTVp5K2NMY3BTd2V0N0Q2MG4yaFE2?=
 =?utf-8?B?TzhpMUJnUjRaRGluSmFHMzExcldsaHEza2FQb1Q5OEVwa0FPajduZCtaQm5P?=
 =?utf-8?B?Yk8wd1pnUXNSczA4dHBrc1NoTzR1cVRxVDlnMVJHSzUzNjIyQ0FaUHk3NVZF?=
 =?utf-8?B?VkRxNzBwSTZ1TU15c1Y0Q0F4RDhwVjhONHYrWkJiYTZic0ttU3NONDBXMnRZ?=
 =?utf-8?B?Sllvck14M2JWSHlrUkhGRU9NM3NYVEhMUGJmcC9ja293Z05HUnB6ZFcrZVVn?=
 =?utf-8?B?UW9xKzVTYk0zUTJMZDAwcmFuQUV6SEpWUkpham5walA2V01aWFRmT0dTRWNT?=
 =?utf-8?B?TEZXYXB5aEtaTFpuZFNoLy96MVFZU250eHhBQXVMUG92SkNXVm85dXBVNG1V?=
 =?utf-8?B?T1IwdGdNOUFLNkVNVnpZSHJyTDVRbVE5ZHRUcWZMUVdIbDBncGRzbVZWbjMy?=
 =?utf-8?Q?DuWpWNru99j5qEVvEYC4iym/witJSD4lfoP0cij?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ffb45135-6802-4870-a0b7-08d982c1f332
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 20:52:57.2245
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2V4kPL1ed8dN8uIZa9yvyjW2p9kSid5dVgiMQjVDNJ4zKi3Ye+302j23F3njs1aMH5W0ZR2pmn9DlsoIcEtMBo1A8Qvo7rCQG0yoqMH7E6M=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB4966
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
> index e60af16ddd..772e2a5bfc 100644
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -858,13 +858,20 @@ static void do_trap(struct cpu_user_regs *regs)
>      if ( regs->error_code & X86_XEC_EXT )
>          goto hardware_trap;
> =20
> -    if ( debugger_trap_entry(trapnr, regs) )
> -        return;
> -
>      ASSERT(trapnr < 32);
> =20
>      if ( guest_mode(regs) )
>      {
> +        struct vcpu *curr =3D current;
> +        if ( (trapnr =3D=3D TRAP_debug || trapnr =3D=3D TRAP_int3) &&
> +              guest_kernel_mode(curr, regs) &&
> +              curr->domain->debugger_attached )
> +        {
> +            if ( trapnr !=3D TRAP_debug )
> +                curr->arch.gdbsx_vcpu_event =3D trapnr;
> +            domain_pause_for_debugger();
> +            return;
> +        }

This is unreachable.=C2=A0 do_trap() isn't used for TRAP_debug or TRAP_int3=
.

> @@ -2014,9 +2021,6 @@ void do_entry_CP(struct cpu_user_regs *regs)
>      const char *err =3D "??";
>      unsigned int ec =3D regs->error_code;
> =20
> -    if ( debugger_trap_entry(TRAP_debug, regs) )
> -        return;
> -
>      /* Decode ec if possible */
>      if ( ec < ARRAY_SIZE(errors) && errors[ec][0] )
>          err =3D errors[ec];
> @@ -2028,6 +2032,12 @@ void do_entry_CP(struct cpu_user_regs *regs)
>       */
>      if ( guest_mode(regs) )
>      {
> +        struct vcpu *curr =3D current;
> +        if ( guest_kernel_mode(curr, regs) && curr->domain->debugger_att=
ached )
> +        {
> +            domain_pause_for_debugger();
> +            return;
> +        }

Urgh.=C2=A0 The TRAP_debug above was a copy/paste error.

I'll submit a patch, as it wants backporting for a couple of releases,
after which there should be no additions in do_entry_CP().

Everything else looks good.

~Andrew



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:00:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:00:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198467.351977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKDO-0003uQ-1l; Tue, 28 Sep 2021 21:00:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198467.351977; Tue, 28 Sep 2021 21:00:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKDN-0003uJ-Uz; Tue, 28 Sep 2021 21:00:37 +0000
Received: by outflank-mailman (input) for mailman id 198467;
 Tue, 28 Sep 2021 21:00:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKDN-0003uD-7s
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:00:37 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e0c79031-4ef4-4a9e-9bf8-90859dc81a44;
 Tue, 28 Sep 2021 21:00:36 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0c79031-4ef4-4a9e-9bf8-90859dc81a44
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632862836;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=jyvMptSu2lickOgtgaBHZNpuR05J4lIWaQ3xth9DrgA=;
  b=I3C0iHZD6TPB8ZBhV3RnH/0A2SiGkHbn2K4HlzgRnSHuWoQMJik0+DmZ
   QL/ov4DVXOtEN5DdF73fbvy3np9fm2/emsljJsFWxYoVrMDljWWPw5M/H
   jTaJq9UBM757Zug2A7PrtbZXUeYfizSpdU50rnCbrZS+Qb6J3U7FeK70p
   k=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: 3pGIScqYZWlgbvqDQXtR7T7VKGUC7awDT85X5Xoghm8WdobPJZYJd1aIOanEPcpgDLKdlA4gFl
 W72thHOZTLumNn0oBn7eMI2TIyA4b6aPkv1b65voIchKG1LTd9nsthm+xPrXZsvhb/9A6n74Ib
 07OwBluWObkPZPvWQCwe+ead25cCgfMjtLV7qzoNCHRO8HchTFrcFq6EP581xfg4t7AsCHxLwK
 tB9m87TW30uGDAueEHGxtyWt3UebQtF3ceIJU7xJVMC78jdMBfFaiU9HGGWgg1zEKkHLa6nnLP
 nPVS87u8qoBW4OJav3ixZ2VW
X-SBRS: 5.1
X-MesageID: 53842725
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:Ut9dFqvVwdwHbCcKxQj/BCIOa+fnVLZZMUV32f8akzHdYApBsoF/q
 tZmKTyOPKyKNDPwc9slPI+xpkwHuJXdz9A3S1E5ry80HiIQ+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y524jhWGthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 NpliJm6EzpwJL/wh/0bSxB7MSojH6hv5+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DP
 JpFM2U1MXwsZTUTMQcQF6wcv9uFm1zCTzleiXmvgqgetj27IAtZj+G2bYu9lsaxbdpRtlaVo
 CTB5WuRKhsdPtaOjzee8nahg+bJtSz+UYMWUra/85ZCkFCVg2AeFhASfV+6uuWizF6zXcpFL
 E4Z8TZoqrI9nGSJZNTgWxyzoFafowURHdFXFoUHBBqlk/SOpVzDXy5dE2AHOIdOWNIKqSIC6
 1qDxe6wHy5Tr6STUXu+1LzJjzabJn1ARYMdXhMsQQwA6tjlhYg8iBPTU9pueJKIYs3J9SLYm
 W7T83ln71kHpYtbj/zkoQCvbyeE/MChc+Ij2unAsotJBCtXY5S5L6ih9FTW9/pJKIvxorKp5
 yNfw5n2AAzjF/iweM2xrAclQO3BCxWtamS0bbtT838JrG/FF5mLJ9w43d2GDB01WvvogBewC
 KMphStf5YVIIFyhZrJtboS6BqwClPa7T4S4Cq2LNYYRPfCdkTNrGgk1PiZ8OEi3zCARfVwXY
 8/HIa5A815AYUiY8NZGb7hEiuJ6rszP7WjSWYr633yaPUm2PxaopUM+GALWNIgRtfrcyC2Mq
 oo3H5bamn13DbylCgGKoNF7ELz/BSVibXwAg5cMLbDrz8sPMDxJNsI9Npt7INE6w/QJyL2Yl
 px/M2cBoGfCabT8AV3iQhhehHnHB/6TdFo3Yn4hO0iGwX8mbdr95asTbcJvL7Im6PZi3bh/S
 PxcI5eMBfFGSzLm/TUBbMai8Nw+JUrz3Q/ebTC4ZDUffoJ7Q1Cb8NHTYQaypjIFCTC6tJVir
 uT4hB/bW5cKWy9rEN3SNKC011q0sHVEwLByUkLEL8N9YkLp9IQ2eSX9guVue5MHKAnZxyvc3
 AGTWE9Kqe7Iqo4z0d/ImaHb8Nv5T7ogRhJXRjCJ46y3OC/W+nuY7bVBCOvYLyrAUG7U+bm5Y
 bkHxf/LL/Bazk1BtJBxEug3wPtmtcfvvbJT0i9tAG7PMwawErplL3SLgZtPu6lKyuMLsAe6Q
 BvSqNxTOLHPM8L5Cl8BYgEia73bh/0TnzDT69UzIVn7u3ArrObWDx0KMknekjFZIZt0LJghk
 LUot8Mh4gCijgYnb4SdhSdO+mXQdnENXs3LbH3B7FMHXubz9mx/XA==
IronPort-HdrOrdr: A9a23:lCcKT6AqXf70akHlHemi55DYdb4zR+YMi2TC1yhKKCC9Ffbo8f
 xG/c5rsiMc5wxhO03I9eruBEDiewK5yXcW2/hyAV7KZmCP0wHEQL2KhbGSugEIcBefygcy78
 ldmtBFebjNMWQ=
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="53842725"
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich
	<JBeulich@suse.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Bobby Eshleman
	<bobby.eshleman@gmail.com>
Subject: [PATCH] x86/traps: Fix typo in do_entry_CP()
Date: Tue, 28 Sep 2021 22:00:14 +0100
Message-ID: <20210928210014.27954-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

The call to debugger_trap_entry() should pass the correct vector.  The
break-for-gdbsx logic is in practice unreachable because PV guests can't
generate #CP, but it will interfere with anyone inserting custom debugging
into debugger_trap_entry().

Fixes: 5ad05b9c2490 ("x86/traps: Implement #CP handler and extend #PF for shadow stacks")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Bobby Eshleman <bobby.eshleman@gmail.com>
---
 xen/arch/x86/traps.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index c2e2603c394b..63676b0a68ff 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2047,7 +2047,7 @@ void do_entry_CP(struct cpu_user_regs *regs)
     const char *err = "??";
     unsigned int ec = regs->error_code;
 
-    if ( debugger_trap_entry(TRAP_debug, regs) )
+    if ( debugger_trap_entry(X86_EXC_CP, regs) )
         return;
 
     /* Decode ec if possible */
-- 
2.11.0



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:02:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:02:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198475.351988 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKEy-0004ZP-Fm; Tue, 28 Sep 2021 21:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198475.351988; Tue, 28 Sep 2021 21:02:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKEy-0004ZI-CG; Tue, 28 Sep 2021 21:02:16 +0000
Received: by outflank-mailman (input) for mailman id 198475;
 Tue, 28 Sep 2021 21:02:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKEx-0004Z8-1K
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:02:15 +0000
Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5a7e798c-209f-11ec-bccf-12813bfff9fa;
 Tue, 28 Sep 2021 21:02:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a7e798c-209f-11ec-bccf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632862933;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=6UE6d1M7eYdCGTmwe2ZPJbBVHToWHugrAczuoQ0Il4I=;
  b=DfQiS/hwuqr8sIKLNkf/OSrVaxtBZ/HoX5LYt7e/T5rUxriWTL0+bE3a
   yxdOrPgzCvBglajlLPKHEUNiVTfpKD9y8qu3bJFDHu3mMDP/5sacpk/4J
   VVeSdlry5unlH8Z2LZ9EZ57LjEX9MkwBGz7Tuc+pYPHgfHbsKkdLOJNt5
   k=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: ZpgM0Zh2jkKA71TlQpN5Kpw2HnqvBVD7uf2oiXURlF3gR+oVaT5BipZJlf5QWCk0FaMWf8clgL
 V+5DMbSeJtUjaAt6uoM8Z8OYS53zNYMB3WBBYW+lChC8td/00vs0tVfp+1BKtafHIzKCTTxp40
 QOk+ehQkJoEdSXyGCn/uzSJIhw7Q74DWwNPG3E+bWVgkWI9yewRKmH8Mexfo9W5LmBKakJOpPB
 UhEpx3Ps7RVBLiE1Y3Rt7YLZIseqhZQvFQfcLRuFOHY0LpH/spGc7EEBDXxVS8TuMmIa0e5CzZ
 WLguJW8XFvWlyuCcGhjXAitC
X-SBRS: 5.1
X-MesageID: 53453677
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:+b6V2a947nyH1sfStR/+DrUD9XiTJUtcMsCJ2f8bNWPcYEJGY0x3z
 GocX2rVOazYZWagLdBxbI+y8EwBsMLSxtFrTAZq/3w8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhGGeIdA970Ug6w79j0tYz6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPh+0
 d5GuZqwcT4IYIPMluQUczN0Sw1haPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguwKKsXxMZxZkXZn1TzDVt4tQIzZQrWM7thdtNs1rp0VQauFO
 ZVGAdZpRAWePzFVZE03Mrl9u/y5pCHzWAV4lGvA8MLb5ECMlVcsgdABKuH9YceWTM9YmkKZo
 GPu/GnjBBwectuFxlKt8G+oh+LJtTP2XsQVDrLQ3vRtileCgGEICBkbUFK9ifa8g0+6HdlYL
 iQ86ico6KQ/6kGvZt38RAGj5m6JuAYGXNhdGPF87xuCooLq5AKeCnkBXyR2QtUsv88rRhQnz
 laM2djuAFRHvbmYT3vb67mVqC+pNDYcBWgYbCQAQE0O5NyLiIAsjDrfQ9B7Cqm3g9bpXzbqz
 FiiriIzmrEShs4jzLig8BbMhDfEjofNZh444EPQRG3N0+9iTNf7PcryswGdtKseatbCJrWcg
 JQas8eH5+sCC5G1rQ3TbuUDHLiJzdrdHCKJ1DaDAKId3ziq/neie6VZ7zd/OFplP644RNP5X
 KPAkVkPvcACZRNGeYcyM9roVJh6pUT1PY69Ps04eOaidXSYmOWv0iZ0eQa01n3hnVIgmKUyU
 XtwWZ33Vi5LYUiLISDfegv87VPJ7nxirY8wbcqip/hC7VZ4TCXOIVviGAHSBt3VFIve/G3oH
 y93bqNmMSl3XuzkeTXw+oUON10MJnVTLcmo8JYHK7LefVE6Rz5J5xrtLVUJINcNc0N9zLugw
 51AchUAlAqXaYPvc21mlUyPmJuwBM0i/BrXzAQnPEqy2mhLXGpcxPx3SnfDRpF+rLYL5actF
 5EtIpzcatwSGmWv02lMNvHV8d09HClHcCrTZkJJlhBkJMU+L+EIk/e5FjbSGN4mVHHq6pdu/
 +P6hms2g/MrHmxfMSofU9r2p3uZtnkBguNiGUzOJ9hYYkL394Z2bSf2i5cKzwskcH0vHxOWi
 FSbBwk2v+7Ir9Nn+dXFn/nc/YyoD/F/DgxRGGyCteS6MizT/2yCx45cUbnXIWCBBT2soKjyN
 /9Iy/zcMeEcmAoYuYRLDLs2n7k14MHipuEGw108TmnLdVmiFphpPmKCgZtUrqRIy7IA4Vm2V
 0uD98N0I7KMPM+5QlcdKBB8NraI1O0OmymU5vMweR2o6Chy9buBcENTIxjT13ANcOoraNsom
 L5zttQX5gqzjgsRHuyH1i0EpX6RKnEgUrk8ss1ICoHclQd2mEpJZobRC3Gq7cjXOclMKEQjP
 hSdmLHG2+ZH3kPHfnc+SSrN0O5aichcsRxG1gZfdVGAm96Djf4rxhxBtz8wS10NnBlA1utyP
 EltNlF0evrSr2s52pAbUjD+ARxFCT2Y5lf1mgkAm2DuRkW1UnDAcT8mMuGX8UFFq29Rc1C3J
 l1DJLoJhdoyQPzM4w==
IronPort-HdrOrdr: A9a23:st5XNa0otwey7ArKyUdZVwqjBSVyeYIsimQD101hICG9Lfb2qy
 n+ppgmPEHP5Qr5OEtApTiBUJPwJU80hqQFnrX5Wo3SIDUO2VHYUb2KiLGN/9SOIVyHygcw79
 YHT0E6MqyLMbEYt7eL3ODbKadY/DDvysnB7o2/vhQdPj2CKZsQiDuRYjzrY3GeLzM2Z6bReq
 Dsg/av6wDQAkj+Oa+Adwc4tqX41pH2vaOjRSRDKw8s6QGIgz/twLnmEyKA1hNbdz9U278t/U
 XMjgS8v8yYwr+G4y6Z81WWw4VdmdPnxNcGLMuQivINIjGprgqzfoxuV5CLoThwiuCy71QBls
 XKvn4bTotOwkKUWlvwjQrm2gHm3jprw3j+yWWAiX+mmsD9TCJSMbsLuatpNj/ir2YwttB116
 xGm0iDsYBMMB/GlCPho/DVShBDjCOP0DkfuN9Wq0YafZoVabdXo4Ba1lhSCo08ECXz751iOP
 VyDfvb+O1dfTqhHjDkV1FUsZmRt0kIb1O7qhBogL3T79EWpgE586Ig/r1cop9an6hNDaWt5I
 z/Q+xVff91P5YrhQ8UPpZ3fSKNMB25ffv7ChPaHb3WLtB0B5vzke+C3FwU3pDhRHVa9up+pH
 z+OGkow1LaPXieUfGz4A==
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="53453677"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TaaIF2OTHUSoBpozmdiQ329EwYYmOHxJDwlaaQxJ3tGUh3k2NuHSollODHbXI91sHUdUbrh1K5H7v+Qz+Rj3H/oFv3sHDTXlZFFo+mL+V8yUJ2vuYnycofXMTgSkPHF+BZh7w+ycpyaqPrDaPoWM9vAYVoB1VEcC4jO7BQH3ohbdmJp4ayKOfVywvzFbbPyfKq57OoTFrgPDRfFSCFvS1wo0kjPXVWLGXOctZRmAjTWqIA9fw5wS1ZC6j1uvSSe3yQap1B5liDgGw4NTGPRjwil+hlxFXGSIZ70ocwRIyoXS11Mq7IdWAnUoMvqMuxdHQzvYYy6pBnsN15wU/QMx6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6UE6d1M7eYdCGTmwe2ZPJbBVHToWHugrAczuoQ0Il4I=;
 b=VXQIg1FB/n5cq2v9YbYcUorL1oZU13YmJLIYmYmQRP6Z0et7rIit/ugbzXZO2pyAUn+C5RjGfwofGv49fUk7fZ35u9lSrdCsOnchtlb2EgF1VAJVM0+xKQJqxGt86diZp7SAA7x8IKjbbFlv1oljJETOp1qem3JtjDPtgaNnfqV408MiBWZxwwwH6565kxhABoYwr0DAb8VnVcfWHeW93FxOovDqxepIfaa99sHTbUv2cO6uDeHmnk/kcuHAPr9R+EF8I2U33amXkaLQc1RixowD5vNZlVhApMwfNU7zL3nmxPaRkTAPEqXmYMDzEfi3Z7PVS5tp+XSLlqN0OAN9lg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6UE6d1M7eYdCGTmwe2ZPJbBVHToWHugrAczuoQ0Il4I=;
 b=cgL55hKiK+TeflAJ4i6hAdWk3RKEaC8TL0OsdCbrMZs2d8rlTWPN7Y/np4py1Qc7yFTxU7IACXW1nkxQkHXDmqYx0RnRZFEsY8DEMotIXDz/B4P1K/E1yWhMZiWsGWsot5bqaAcQe075hpnqJQU2edFy5azy3QxuQm64CuUlI6g=
Subject: Re: [PATCH v4 1/6] arm/traps: remove debugger_trap_fatal() calls
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, George
 Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <4c4f45280547564b2c16fd9ca09443314b8f3b6e.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <05ff86aa-8db7-908a-1b96-851c110e399e@citrix.com>
Date: Tue, 28 Sep 2021 22:01:59 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <4c4f45280547564b2c16fd9ca09443314b8f3b6e.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0432.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a0::36) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae4c5457-4322-4f23-a101-08d982c33b23
X-MS-TrafficTypeDiagnostic: BYAPR03MB3429:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB3429F59B6E88A7C992B56C4BBAA89@BYAPR03MB3429.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:215;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: /smxpm6rdL+WkFCXusJzEyzfoDKysVPJlcYKikqZvJAVA4FJsL5ZYnuYAi6y+kVQTi+6If2UdwNqtAKS6tI0Jizz0f/QpYTn5Mcr73UItIk56gMGQ4NyJg3C2C43dnv5cRDi1mtCcabJUFx9CWHR+XNYAtQWrIFrt8cKxCwSjVDJJMhk6a0WtTyccDw4VFvDxv1nXEztRMKqj2MKQjQT9ZcX7spjZv8LhfM7JaeYlht6dz4AraT1s8b8aNVDz50roU681Q6+8NCB2gHszvwf236w267We2b5QRlKDK6ElBY9f9xO6JFmtFimlxXys+Xzw0Alx2xCD3pIPnGKuOqrVn3ah7tTX8l2VcpGzEa4nsmA8cc4v02ryj0wdeqfO/joC9Jam+CSDwj6WSoUTJ5Wi/1s/CkXKmQaLuulgvl4fXDMst0Ve2I5J/U06ncMlHEW8vbkyP4oMvXl449p8WEB8C0gwban20WuSJ+GbvhRAMfgSL/iiM9gE4DHdLuoo9ZZ+UUXECZHUEvEi2X4iUUBxCOiMDbb1QvpshKhz/uLEcRdtR6j2VuNKqrcey7WXOcPeiaAjfbI6jxsu9fi8RQ2OtmSuzAJ+qF5c3F1DtU0/cWL8yH+YbsM1orsp/PYUt0dtkxMHxnJPfQH/S1borMVQcBolwU3j4+GqgqGB3owDPU/nHvx39WGyb9k3Wbiwd26Xnh24lGI8hYiX8XpHue55b5lf3+C4FI4HYIHmt9OZojdQI1AqWwgqWVBgBz3I4oT
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(54906003)(31686004)(5660300002)(186003)(31696002)(55236004)(16576012)(36756003)(66556008)(86362001)(316002)(2906002)(8936002)(66476007)(53546011)(38100700002)(26005)(4326008)(66946007)(8676002)(4744005)(6666004)(2616005)(6486002)(508600001)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QU11NFc3dU16SkRmdHMzUk5NK1NML1hmdEpHV0lhVDdSZE1PNkQvNDhmMXdQ?=
 =?utf-8?B?NlY2NzhMQkFjY1l2NS8zenc3L0pwcTBFNnVRY3g2b1hpcm5uUVB2TjRtR3V6?=
 =?utf-8?B?bkRxY2E5UGdvV1BFK2VWS0ZHclRGOXhjM04xYmpmWEI1dEFhTmp3ZGRoVjJy?=
 =?utf-8?B?SWk4SVp4eldpek1wYTY2WTZKWS9kaUs3TWdGbGVuWi9iU05xZGk4UFZkd0dw?=
 =?utf-8?B?Tjd6ZFQ0RlFVeFMrWngzWGF1RWdQWnltcHN0U2NLTEpCWWVLcDY4clI4clgr?=
 =?utf-8?B?NU9sSnpydTNSeW9abkhieWpMRzdCT0xlYU5LZGdReTIwN3lNMjd0UC9uRWJG?=
 =?utf-8?B?VjFLZ2YwWFpjMG9xcHJUbHpOSVJSdEpYL2FUMjUyL1ByV1VUM1pHZ0ZCbi9Y?=
 =?utf-8?B?T2tOdGJTTWxJZWxFaVE3em1zWTNWQXpUei9ybllYdmZibmZ6dGhNTkNUTThF?=
 =?utf-8?B?blJ4eVM5bmZrZnZ4dFlmMWV3WDJDRlVPYXBxUnVGd0JBZDBoMGpWMTVkYWxi?=
 =?utf-8?B?dnJIVmtkZ0FPcFlGRXd2VmdxR1hBRVozLzhkaHA4dFQ5MjZERVZaengvd2R6?=
 =?utf-8?B?RWpVT1F5TXRvYWVoL3FjdGh3SlNYMTNsZnY2ZFQzcGlRS3ZzNU0rV1lWM2tH?=
 =?utf-8?B?RFV0bGZIQm9od1l0eFdYWlFSOFF0OUpxSFdaTnFmSlAxV2JkdVR5SmgyVkUw?=
 =?utf-8?B?aVNkcWRjVlhDQWtOZXZlSWdrdVJQNlpkTjg2REM0cS9LMlBaV25iTDN3bUgr?=
 =?utf-8?B?My8zNFUwRndxb0p5YlJOQTlPM2dFZWpNUTI0TStNMmRmN1Q2R3lucFkwczUz?=
 =?utf-8?B?OTZkNXBtbUErajh1Y2VqRHhSQjJ0bWlGbklMQk9DdVMxVmFCNUxscVBxUFhi?=
 =?utf-8?B?Q2xxRGtCTENkaGlLWjFEL0ZkdmcxYnEyc2w0ZldGYUtKU09PdkhSRm1EbjNm?=
 =?utf-8?B?bk9RRnJCSFkvVTh0RUFmNzVUSzZIOXlOeHVEQlAvUy9KQlkrUTVubjNFVEdZ?=
 =?utf-8?B?VTV2K0ozcXRrbmtpeHRYKzBPaFFrd1FrdW84aVcyWUF4b0ZMN082L1RSc0ZP?=
 =?utf-8?B?VDlUTUJGOFg2djBCZnhYYjJ4bm5MLzhueTl4STRraVJuTlArTk5TSzgxdVZJ?=
 =?utf-8?B?OHZyNU5qMFkxMjZIYUJqWlNEMmNKL1h6U1dTNGlReGN3NC9mM3hjanpBeHFy?=
 =?utf-8?B?WG15UHQwUG8xVkcxUHBDV2YyZmQ1OFM2eWlsT2hST1kzZHpiSkJpTGNXRkhX?=
 =?utf-8?B?NHlNelltMXN0S3dWNGFSU0JOQjVKbVlCeHB0NDFYWkxzeHNGUEYySDBpMHlk?=
 =?utf-8?B?a2ZqTjRhZUM5NGFIcnQyV0tRM04xYUZEUi91OWdaZk9JbDZQVllLbnZYT0ho?=
 =?utf-8?B?NmU5UzFiRlhteUx6MmVhT1ZYeUVNREU3WnVVYWtOT0MydEZMVFdpbjZIbWRH?=
 =?utf-8?B?SkF3REQ0SmhQa295Q2ZXZXg4RUV0WTRMeE5MZkhudkJ5aVpNTGNmeDZlMkxt?=
 =?utf-8?B?bHlWaVZrNktoZDJwcVQ5MURtY2JES1o5aHk1S0NhSUhNbWxaMlRtclBnM2ZM?=
 =?utf-8?B?RTkwTEl5M0ViTkxXTmNzeHVSRnV5OFFpRFpSRThpbEVrZEYxTDBGYVJBODFG?=
 =?utf-8?B?WW9adXJKQzJOdHBnNjJSOE5tWHllK3FPbDluaG9NdFBkRlpId2I1SHB0WDEv?=
 =?utf-8?B?VVNWcFZlWDlGbjZXbytlQkxDalhHY1NjejR5VzJYWmJ5anhhTWc1bHlCZTRU?=
 =?utf-8?Q?9ieKb/dzLAtKojN2DMh+ZrjntnZ0LkmDgtYjTZL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ae4c5457-4322-4f23-a101-08d982c33b23
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 21:02:07.3399
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dKuIYiKQytC5vk9S3DS13WL0zqqNbx/C/egjoGV62q6izrbq3MElLrufZufW2HV5xzr2wOp7As/rPB+a4vMMHT210B/eD3JJ0poDIf76qz4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB3429
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> ARM doesn't actually use debugger_trap_* anything, and is stubbed out.
>
> This commit simply removes the unneeded calls.
>
> Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>

Julien already acked this patch on v3.  You should carry the tag on
future revisions.

Acked-by: Julien Grall <jgrall@amazon.com>



From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:10:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:10:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198481.351998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKMo-0006AW-8M; Tue, 28 Sep 2021 21:10:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198481.351998; Tue, 28 Sep 2021 21:10:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKMo-0006AP-5J; Tue, 28 Sep 2021 21:10:22 +0000
Received: by outflank-mailman (input) for mailman id 198481;
 Tue, 28 Sep 2021 21:10:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKMm-0006AJ-DB
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:10:20 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a1fe70fe-3ef6-485d-b8f0-0d3bec0622b3;
 Tue, 28 Sep 2021 21:10:19 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1fe70fe-3ef6-485d-b8f0-0d3bec0622b3
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632863419;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=9N4+fRGodNsBNVOC0/rVAcY6uxVQT5zufJHm3nvET68=;
  b=hYr1i3bI3bsZ4/HSxYRFHpUha5mPzu7u8C33vQxNAc9D1o77DXbsIM0q
   BlXc1jYJvaHr8hU2PttQXZD0Hh0tfgI55hZbuaXKa4CMVe19DDOz0PkMl
   HDocTjm4jK6H5qUbMD85dh01R0G9jq2tKybi745A58v4JT/XDb4hVyDHD
   s=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: wqRDboxWTUxU/Xm7MitI1O1eFmfu5R0YIXYI7CMAxS7HMw+19Gwvw/oIGH2bApfe9aorVrd9vw
 oct0oFBo1aCarKmHv/rjlNbMWkMqnzMHKmGRmhNTBDmg1bu6AELa9tfqmvskkJRhYbTnR7g/iS
 28ZoLVCURPUzpGGvBzOkb2F1hJZ1Q2eL/r+kQZkUgSj6u5iVJJ8RLei1z9PjdrpB+yD4nut3gM
 kPapDh2ELAV8JCrXaSJOHfV2Q/ZaDOJhTYBTjP1IS24a3s0LR3ACg+VWOFADgW7QdQaWlB8rC4
 zC5u0OeWQzQJISKy0ZRFEJKo
X-SBRS: 5.1
X-MesageID: 53843409
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fz/Xd6CCj2kXOhVW/1zlw5YqxClBgxIJ4kV8jS/XYbTApGsl1D1Wz
 GcWUT2HPvaPZDD3cowlOY2zp0tQvpLQmt5nQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00I7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/0BTVhd9o6
 fV0mKPpSRgZP6KPuugtakwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFh2ds3ZwWTJ4yY
 eJeNSZubw7EZCQQGX0wIYsjl92M2nPWJmgwRFW9+vNsvjm7IBZK+KP2LNPfd9iORMNUtkWVv
 GTL+yL+GB5yHN6CzTuI9Fq8i+mJmjn0MKoYE7ey57hnm12Xy2kaDjUZUFK6pb+yjUvWc8JSL
 QkY9zQjqYA29Ve3VZ/tUhugunmGsxUAHd1KHIUS1gaJzabF5heDMUINRDVBdd8Oudc/QHoh0
 Vrht97tCDpm9qCbSHeB7bqKpBu1IyEeKWJEbigBJSME/NvLsIw1lgjITNtoDOiylNKdMTP6z
 i2OrSM+r64OlsNN3KK+lXjYhxq8q56PSRQ6jjg7RUr8sFk/PtT8IdX1tx6Ltp6sMbp1UHHan
 SQelNG/zNkjMsiNixyMT6YvEIGQsqPt3CLnvXZjGJwo9jKI8nGlfJxN7DwWGHqFIvroahezP
 haM6Fo5CIt7eSvyNPArM+pdHuxwlcDd+cLZuuc4hzakSrZ2bhPP2ChzaUOK02npnSDAeolkY
 szDIK5A4ZsAYJmLLQZapc9BjdfHJQhknAs/oKwXKTz8itJyg1bPFd843KOmNLxR0U99iFy9H
 yxjH8WL0Q5Dd+b1fzPa94UeRXhTcyNhVc2q9pMMKrHcSuaDJI3HI6SKqV/GU9Y595m5a8+Sp
 i3tMqOm4ACXaYL7xfWiNSk4NeKHsWdXpnMnJy08VWtEKFB4CbtDGJw3LsNtFZF+rbQL5actE
 5EtJpXRatwSG2+v02lMMvHAQHlKKU3DafSmZHH+PlDSvvdIGmT0xzMTVlG0qXZSXnbp75tWT
 n/J/lqzfKfvjj9KVa7+QPmu00mwrT4anudzVFHPOd5dZAPn940CFsA7pqNfzxgkJUqRyz2E+
 RyRBBtE9+DBr5VsqIvChLyerpfvGOx7RxIIE27e5LewFC/b4mv8ntMQDLfWJWjQBDHu5aGvR
 eRJ1PWgYvcJq0lH7thnGLFxwKNgu9a2/+1Gzh5pFWngZkiwDu8yOWGP2MRC7/UfxrJQtQasd
 FiI/91WZeeANM//SQZDLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIeIsYo0/w
 OontMoH0CCFi0InYoSckyRZ12WQNXhcAa8pgY4XXd3wgQ0xx1AcPZGFUn3q4IuCYslnO1UxJ
 mPGn7LLgrlRyxaQc3c3EnSRj+NRiY5X5UJPxV4GYV+IhsDElrk82xgIqWY7SQFczxNm1eNvO
 zc0ax0pdPvWpzo41tJeW22MGh1aAEzL80P8/FIFiWnFQhT6TWfKNmA8Zb6A8U1xH7iwpdSHE
 GV0EFrYbAs=
IronPort-HdrOrdr: A9a23:btn9bKi4xDmZjXTRjIUItzvjO3BQX1N13DAbv31ZSRFFG/FwyP
 rBoB1L73DJYWgqNE3I+erhBEGBKUmskaKdkrNhQ4tKOzOWx1dATbsSkbcKpgeAJ8SQzJ8n6U
 4NSdkZNDS0NykGsS+Y2njKLz9D+qj+zEnAv463pB0BPGIaCdAT0+46MHf9LqQffng3OXNTLu
 vk2iMonUvERZ1aVLXAOpFTNNKz1uEj2aiWLiIuNloC0k2jnDmo4Ln1H1yx2QofaSpGxfMH/X
 LemwL0y62/u7XjoyWsllP73tBzop/M29FDDMuDhow8LSjtsB+hYMBEV6eZtD44jemz4BIBkc
 XKoT0nI8NvgkmhMF2dkF/I4U3NwTwu43jtxRuxhmbim9XwQHYAB89IletiA1HkwntlmOs5/L
 NA3mqfuZYSJwjHhj7B69/BUAwvvlaooFI5+NRjzUB3YM87Uvt8vIYf9ERaHNMrByTh8r0qF+
 FoEYX1+OtWS1WHdHrU11Mfg+BEZk5DWituf3Jy/fB8i1Nt7TZEJgojtYwid047hdcAo8Iu3Z
 WcDkxq/Is+BfP+I5gNXNvo7KOMexrwqCn3QRavyGLcZek60kL22uvKCYoOlZaXkbwzvdYPcc
 f6IR9lXVBbQTOmNSTJ5uwRzizw
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="53843409"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=F2tdrNspkSHjGtq4iUAA7KtTEoz0eT43JFEcUVZ0cI1rhOdlxSnjjEd+G1/4d/HeSOOoreckMN6fMn8fXNaj+4yF1h37FACtRpgLKy9nBl1wH1FyLEabZJrBDlZwuHSp44tzUoeg4mayn7PgXlq1PfAE2Nx6XKA5SkvNpRsvFH6QxKCGy8kb2hMFazi8HchvHP62m7LRbRc4QEcWpJiNDXZ0tA2seA7sMqRWwzoqOb5DCKz0Jq+2ULBArB9J/vjOmMi0VmDbU2t5MOWgo7gTeY6GJnYQQwJUsIC5Sdpzr13mH1gg6K6LhuU3ozbWTMpKR2ahuUkPvWTHdpj0+d9EwQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YoszqKLQEyHUcm4Qjqr/BlfgU6gwos2Ob3tY8yPqMuA=;
 b=l1I7YPyuTDze4PDJicBjPrbWk0Vgj4pwEymQsYtn6TVHNp1HLDd63Cu6Ee6jPplhhtvEkoqrD2+QdJGa2wDTzsIeqaOe3mzaJLZtLIykEmNwCX35PmJT7stZ71v4byWL38Ic8FN0ulOvnr0HfmkyUmyoUMGUnVl9OS/G26yLOvCBI0+vExoN4OYNiin18LDwoo8bey4Vs+IR2PA6+oc56SpvvBvOCMCZvqc3MstINafZ6z8kSiDMqntAOcCXqc7nBB25Lo2y45vJQjAbGTkRI2TLD8fwNN3n0Si1iJdrKw9IG1Xb+cHpIj6peWrk5Q57EoE61cH2IuH8FlIBxhWyaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YoszqKLQEyHUcm4Qjqr/BlfgU6gwos2Ob3tY8yPqMuA=;
 b=oeGBu0o1gqbi6I+a3siLef42RYjDFIIbqs46vgLJ5CbC6a+caJSUXfM3xavzU/uSDEGnoFOuodFc/oApWdJ6VUnTcfbTn+udHebC9xFEe2G5mgAn3rmiPe52NL6mYwnzYl2JmTFf5fJ3SEnGw/2u81T/Y3eiKuvCZbiZNV9mJNg=
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <389ae979063a4afc38d8dbadaf539e7f411a24ba.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 3/6] arch/x86: rename debug.c to gdbsx.c
Message-ID: <a094d614-2296-bcc9-1c7a-084ef88b05ad@citrix.com>
Date: Tue, 28 Sep 2021 22:09:32 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <389ae979063a4afc38d8dbadaf539e7f411a24ba.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0444.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:1a9::17) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d5692131-0b02-4f57-a0e6-08d982c44938
X-MS-TrafficTypeDiagnostic: BY5PR03MB5156:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BY5PR03MB5156E5D022E8F5C448330281BAA89@BY5PR03MB5156.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 8oGfICafwYK6Ku/S1yLUkZxvlSIelnJBIFoigfhVdqSAlWPWxSBPga6e8GyEc88x6Zmc9ourP+QZaAIZGK1vFIcT1NFNYS21i7jX+iYGVmkeZgZm7BFi7IvLfbFF+3lFNo5UZMHdvFlsvtcXaCbE4lS9iwezRc7MuUJSPmJgOSqUnSxg3C+tgusDzDDA+HUBCPU8blpn3+CEfzqTBJnF6To/ziTXtyShUcVojphBPBF++NQTgIObIT17StqkFWhYHBJILtPVrPnAf9ObzgQZIMyc2NjitMGcDvtbavs86kHuYX0w8LLYYztXVUFElKGT8IuKS/tkctKCN1fgaz6/qfsaQkn5/yD0sBYaA3Fu5BbnM0B60rXwaW5fHEwlgMSp/o3fYj5r6c9BUbhKxhtRaCsSs5w4j5VWEe03ZG3yJ6PYwguwBPyzTCJpH4Lv5CN4xoniO2K7mXoSJjt0/pRTWsjbdU+sQiwKNXh40jSOedEERkOq9VWSm/nVo16ScTAc/vTRUKjTYlVE5nT2C+/IU/vRmwNLBKGzifraNlUy7Gf0NV6W4el8ucpQ01X57Pax0loMCg7+aX0XCkpmzfFfyzzuodCMJroh4Y0ZaV09gylvWfmea8QYJ0aLUtRWOZHwz/ThANbSx3D/VK00lFw7zOSe5eWE9FOOkZ4K9c3cnyqoqFUNKhOrigEO1d6ce5CkOS7Xz22EsbmsDFlYnkCnE7s4Q525qnSvuQi4pI8dhonwug/vsTw32L53PpGlKaX/
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31686004)(508600001)(83380400001)(6666004)(86362001)(31696002)(316002)(66556008)(66476007)(186003)(5660300002)(55236004)(2906002)(38100700002)(8936002)(26005)(16576012)(53546011)(956004)(2616005)(4326008)(6486002)(4744005)(8676002)(36756003)(54906003)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bnFKcmd6LzJ5djFDRlg5K1JDRUdIcFNMNk82SDJmSGhGazlLa0docXI0aUIy?=
 =?utf-8?B?KytLYyt0c2hreU5wc0tZVndxYmVVME56NGR4bXFncFAzaGRVTHVnRlRSbEgy?=
 =?utf-8?B?L3BkTXRoQTcvem9TejBUa3l2TEJMeHYySW40bG44Q0pkZmdzK1pGTStrZ09s?=
 =?utf-8?B?SUg0U2d2ZEJ2bWtrM2lnaEhaUStGY0hJallkaVhaRnlYc0kweFp1RnJBTXpI?=
 =?utf-8?B?VjMrR3FVOHVYQW91cnRSQmluOXJhVDJTeG1aMzZ2UDlFMm4zZStSM0VOeFZ0?=
 =?utf-8?B?SnFwdFNwOENLSjRpTTBXL3NJVUxQMnRvcEVFYjNWRXlSVndJL0h0TVRBTUtJ?=
 =?utf-8?B?bGppam1helNqWFZ0QmF1QkI3Ui9vbXNtU3BIYmxGWFhEbHBvVmR6N2xMd3NP?=
 =?utf-8?B?V2F2QkY4VWVMZ2ZBMEgrYW4zYThxamtZOVZHSy9HNGUwV2E4T0p3VWw1Nnor?=
 =?utf-8?B?OStHN0V6V2U0Y1E4ZTNTamZqdHhpNWtDcmFWOFFLeTEzSzZTa1dyM1FzRGE0?=
 =?utf-8?B?dVNuSEhuNmNtVDV1YnpLTndiSXdUTy80aVBtTmgwTnJQYURWcUJXRTF4NURk?=
 =?utf-8?B?U0xZMnFvbjZOSXFLcURPR2ZmWXpRZ2VrVTRJcUxoTW5zYnFuak5KZGczNHRO?=
 =?utf-8?B?Tlk4ejJjZW1icVhrQXh5cldLS2ovdGFqZ2p1dVNkN0dCVjJLekh6RktrYlQv?=
 =?utf-8?B?cGU4RDdxTDM5dUZDckxEa1pkRUxBZ3VUSHlpZmoyYVpXVUdYM094OERuS1NH?=
 =?utf-8?B?TEsya2xGVXZrekwvcllGWU8xMHBaWmJ5YjlWanRvYjA2SGk5RTBqaXJpZ2xl?=
 =?utf-8?B?a09Qcmc0R1VGdjhKclBTZWpydGJaaE1kTUtxclUwaXU2OW14dTdLQmJRN0tj?=
 =?utf-8?B?K1ZNRUtFczZFVjNmSFB2d1BDTGRTUnJ3czlwNFk2a3N1RjJtdzhrdklzRkNp?=
 =?utf-8?B?MXp2WCswYWtMbndPak1kenVWNDJURjlNZ0xvbmU4OGF6WkRRbFdXY0NrOUht?=
 =?utf-8?B?SU1vZ0JVM2U1bWkvU1E4d3VzclhxRG81VjEzUUx3L2dxMjBSSzB0WnJGeHJn?=
 =?utf-8?B?Vy9EckVuemJDQUQ1UVpvQ1JYL1pOeWVXU3p1bmtkelJDL0JpLy9sOWFWUmlK?=
 =?utf-8?B?MWhYOHAwZzFQNFdxNUdteExINWdvYlh3Nk9lNS9KZGJydnRGWUlCeEdqanZL?=
 =?utf-8?B?L2E2ZXkxK1dUU1k0V3plT3oxZk9PUDYzYXZ5TFJ3em5CZ3l1bS9aUkFITE1E?=
 =?utf-8?B?RTBkR1Ira3hXSWdLTEY5YmJnQ3NwRmJxTFV6czJKeU53NXNvaHdCRktrZXBN?=
 =?utf-8?B?Sm82Q1AwL3Z0TGJkTFluU2h2bEZHbXg0VHZHdDdJdW1CVUhrbUpWY3k4dmRs?=
 =?utf-8?B?TUVtSVNGLzFUMHk3cmJvTGRDSlRaM0Y4T3BFQ3A5Z2paRitMQUNLT2J6eEIy?=
 =?utf-8?B?anp3ZytPeWdIaVZIOStZaXlyWnVNbHY2aFdOcDFIem1XUW11L09IR1BIVEZm?=
 =?utf-8?B?bGJrbjlMUlNxemFRdkp4STNiVXoxejMxd09XUFg0MVc3YlNPTmhZVDV5TE5Q?=
 =?utf-8?B?TGQ0SmNMR29LQ0k2aDhWbGRIc25kemVaM0l0a1Z1RnliSVluSEh5VnFUa0V5?=
 =?utf-8?B?UE8yNnFBaXF4Yk11NjhxQTRXdHFIVW1IWUZVd3IzTDArcmo3QUN5bHRic1R2?=
 =?utf-8?B?SmhaZmJrWjA3aEpjbjNtYk5JNEtvckJPU3ZZQXdRYjhyVGZ1aG1wLzJjTDQ0?=
 =?utf-8?Q?rnTvphWnF44eO6VT8yRBwBpyoy+xgW/2ajPl2CA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d5692131-0b02-4f57-a0e6-08d982c44938
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 21:09:40.4653
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CL2k9F/xgU/mxhv+0Kjx6T6G5gRSNsXiSKjCniDAlZrbUbfyXjSp2yoGhXhgFJogNgN5BDTdSw8Be4h0DuiLlWhPU28bgPrlfZIgljpr1MU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5156
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> diff --git a/xen/include/asm-x86/gdbsx.h b/xen/include/asm-x86/gdbsx.h
> new file mode 100644
> index 0000000000..473229a7fb
> --- /dev/null
> +++ b/xen/include/asm-x86/gdbsx.h
> @@ -0,0 +1,19 @@
> +#ifndef __X86_GDBX_H__
> +#define __X86_GDBX_H__
> +
> +#include <xen/errno.h>

The errno include wants to move below....

However, you need to avoid latent build errors based on the order of
includes.=C2=A0 I'd include public/domctl.h which will get you both domid_t
and struct xen_domctl_gdbsx_memio.

> +
> +#ifdef CONFIG_GDBSX
> +
> +int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop=
);
> +
> +#else
> +

... specifically here.

~Andrew

> +static inline int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gd=
bsx_memio *iop)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
> +#endif
> +
> +#endif




From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:13:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:13:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198487.352010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKPY-0006ma-Mt; Tue, 28 Sep 2021 21:13:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198487.352010; Tue, 28 Sep 2021 21:13:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKPY-0006mT-Jl; Tue, 28 Sep 2021 21:13:12 +0000
Received: by outflank-mailman (input) for mailman id 198487;
 Tue, 28 Sep 2021 21:13:10 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKPW-0006mN-Qa
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:13:10 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e1a77bb0-20a0-11ec-bcd0-12813bfff9fa;
 Tue, 28 Sep 2021 21:13:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1a77bb0-20a0-11ec-bcd0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632863589;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=g9H30OweOlqfCyXAeGD66cP02ubERI/faXG+IFhCBqU=;
  b=gPDNkH872QIPX61j7Scx/9OxZUWlVR6bri8ebfe6sZLYcXBYHOpq64uh
   YhFf47Y2Wx+wmHZGtN3Hx+xIa3axAw6JNfEubsW4JbEQG9gWmIsM9isTh
   Ts7NFusakOt+DQWywB2n+aVl23hLRUsqQwCOeUYzmz38/rCUH9QXyt3BO
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: tTzCYtk5pN0EpRb7gqKOQpKMYzoiOUdWlUiO5JBkOl+HuJCF/O6LviHAfn3ARCKl6t1uEzIKQR
 j3Qx9Ge/9dBs2cl7JOdowmmrGNMdOWW1uPq46FiyBRWVkQ3mjfd1kEYA1wjRXf/8KT+4ka27+t
 XlWLa5sYZNt0MopWpijhIthX9g6dOnHPO1/8R1/G/os04sxdLHKBZGUtcjc3+LNblm9S2VlCm0
 hcFuMYHOmfHzjhbMPBkARueANt6/dlPcXgGCxB2Vw1qpAE+NkSEVRsYLCpEZWLTNLN13Mj1qhu
 pVm7QZ8CIy3CTkZjSUNCdTXd
X-SBRS: 5.1
X-MesageID: 54270071
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:W8MPPqAtalB4BRVW/13lw5YqxClBgxIJ4kV8jS/XYbTApDxz32EDz
 GsZWDuGb/vcajCmfNt2Pdng9EsCv8SAmNU3QQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w00I7wIbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/1SqLlMJB0
 sl0mt+pEAE1OfLukco5akwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTY+9gnMk8auLsO5sSoCpIxjDFF/c2B5vERs0m4PcFgG1v3pgRRJ4yY
 eICWBF0TBfLQCRPP2UyFLExjO2j2XzGJmgwRFW9+vNsvjm7IBZK+KP2LNPfd9iORMNUtkWVv
 GTL+yL+GB5yHN6CzTuI9Fq8i+mJmjn0MKoYE7ey57hnm12Xy2kaDjUZUFK6pb+yjUvWc8JSL
 QkY9zQjqYA29Ve3VZ/tUhugunmGsxUAHd1KHIUS1gaJzabF5heDMUINRDVBdd8Oudc/QHoh0
 Vrht97tCDpm9qCbSHeB7bqKpBu1IyEeKWJEbigBJSME/NvLsIw1lgjITNtoDOiylNKdMTP6z
 i2OrSM+r64OlsNN3KK+lXjYhxq8q56PSRQ6jjg7RUr8sFk/PtT8IdX1tx6Ltp6sMbp1UHHfo
 3oPifiE8NkhTojUkjCKH8cEMpWQsqPt3CLnvXZjGJwo9jKI8nGlfJxN7DwWGHqFIvroahezP
 haM6Fo5CIt7eSvyNPArM+pdHuxwlcDd+cLZuuc4hzakSrZ2bhPP2ChzaUOK02npnSDAeolkY
 szDIK5A4ZsAYJmLLQZapc9BjdfHJQhknAs/oKwXKTz8itJyg1bPFd843KOmNLxR0U99iFy9H
 yxjH8WL0Q5Dd+b1fzPa94UeRXhTcyNhVc2q9pMMKrHcSuaDJI3HI6SKqV/GU9Y595m5a8+Sp
 i3tMqOm4ACXaYL7xfWiNSk4NeKHsWdXpnMnJy08VWtEKFB4CbtDGJw3LsNtFZF+rbQL5actE
 5EtJpXRatwSG2+v02lMMvHAQHlKKU3DafSmZHH+PlDSvvdIGmT0xzMTVlG0qXZSXnbp75tWT
 n/J/lqzfKfvjj9KVa7+QPmu00mwrT4anudzVFHPOd5dZAPn940CFsA7pqZnSy3VARmclDacy
 SiMBhIU+bvEr4MvqYGbjqGYtYa5VeB5GxMCTWXc6L+3Mwjc/3aintAcALrZI2iFWTOm4rima
 MVU0+r4bK8NkmFVvtcuCL1s168/uYfi/ucI0gR+EXzXRF23Ebc8cGKe1MxCu/QVlL9UsAe7Q
 GyV/dxeNenbMc/pCgdJdgEkcv6CxbcfnTyLtaY5J0Dz5SlW+rubUBoNY0nQ2XIFdLYsadEr2
 +YsvsIS+jeTsBtyP4bUlD1Q+kSNMmcED/ctuKYFDdK5kQEs0FxDP8DRU3ek/JGVZtxQGUA2O
 TvI1rHajrFRy0eeIXo+EX/BgbhUiZgU4U0YyVYDIxKCm8bfh+9x1xpUqGxlQgNQxxRB8uRyJ
 mk0aBElefTQp29l1JpZQmShOwBdHxnIqEX+xmwAmHDdU0T1BHfGK3cwOLrV8U0Um46GkuO3I
 F1MJL7ZbAvX
IronPort-HdrOrdr: A9a23:Irv/kKr3z1/X+KuvPdk0CdsaV5oyeYIsimQD101hICG9Ffb4qy
 nOppomPHDP5Ar5NEtKpTn4Asm9qBDnmqKdg7NhXotKNTOO0FdAR7sO0WKN+VfdJxE=
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="54270071"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hLPa0KxSeEi0aCNSmhYXpZHsTb/OUWHOWrggT2NCr/XK2OBVYAOEwPliIdGqMmYVFLIkv+5TUP6i6XoJzha+N3rTRZ8iBvVQqZKVcjZpZpYjEeIbjgZbm59AEPQ0HpdsH+SvlckuayQHxfzXHmYl88jJDoKGrxY5suS2QV6fPorDaDQLq45ke+ALquS6SdJMf6glbfPP8Q+xTarxg57TL8SldRX8Cu8Ld/7Tg2f3xkbqV+2VX8l6Ba/26PSIyH3Yuv2w7sT9GBuhq/hgMY2O7biJIwreydEhpNqsfihQybcL9xB+oTsE1OcQZOGRLvKsJDQfKImnJ4q1sRMQr/rwOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FFky1Tf/IERdq9HC3apLODEKv5NpKwN1ps/14CdbYw0=;
 b=R35twKqXzaYRnCSwZZSRh1quaeco4aaUV61hNpX1F1mBy3omV9KM5z+VZ/gbcjc2uAeZyX4IuFxIGcPyx5Uh+Dj/ibzI3sOiVRJXGUhxHKNxMPzooB7dactDXOpWMGRvo+2/IWFr6YcJOC0dZ68TeUB3GXwgzMzvKEyEAA6IYVKiDWew+jF/f/WLKYxldXCZDtCNqcvT4LdXKN28gLq/Rbl6yUtmKXn0gbZFClbFJEVmwjGXZGuJF1sohUSYYYby1iZVfZH+L7AjRXzl6eBRdlL+8pSEEWXIEUzNEuqIJT9Fh4cDEOLTJgTn4sEQSx9feEOkrQtPT0jAZtQ+lAq14Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FFky1Tf/IERdq9HC3apLODEKv5NpKwN1ps/14CdbYw0=;
 b=Sz+0ecyjVoHBFnzP4nQ0eF+o3hfs08C2+hvd87/htmc80Ci3H4pW5PlLCBLFNsxgGZ+mk41Kn4MarmqIRYwXJh/V55mcfAjTJ59Ty+FiXqqwuVV9Dh71puL0MEIQkRCXaKFXnhDc/112D86fjAhp27iyo0Md2b8Ibq/8Sjt5JNo=
Subject: Re: [PATCH v4 5/6] arch/x86: move domain_pause_for_debugger() to
 domain.h
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, George
 Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <5de82c39ffae1d65c8ab7142b05d304f9d4ddae9.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <0d8bc733-33b8-1b4c-9e75-5293135405a5@citrix.com>
Date: Tue, 28 Sep 2021 22:12:43 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <5de82c39ffae1d65c8ab7142b05d304f9d4ddae9.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0405.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:f::33) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fd69e5da-45b9-497f-99dd-08d982c4bace
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5454:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB54545BD6F2E3C0192EA86797BAA89@SJ0PR03MB5454.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:330;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: QFhGKrvw7acNK7St4MG3ckxYKg/mUXA317aG13GOOm4ePL5NUx0Y1LQZ4ZRMIv/mOjSkwV1NnOySpoCMfOtZhA0EwABxm/gzB7TcS9DTU6DYiE6KVZzSf18sq30Tm9hVJLdMcK1A8h18TdU0V5jZ6uXRDyYJjZq2c+YCExIImKoeEhS5dZwjya44M56Bkkcp07TwLUq+wGTd+LQbFZGmqyOmJaTcNn2Uh736Na5Om90lrqmo72+41UUFounlcZdHAWBt4FNFZV8jCMo0nZ3Fcb5kV9lDE1qhp6CPrpuC+xAJNBSc282jD4/4dPbCkw2dit8jUCTR8Wslaclej9FtKKUoEGdAKUhh7JewcQpMkMJlT/bjYi+V8/Y/eqRkxYny+7w1CaCZ44lVvJDsPSO/Wl6RS7tnHE4xdFR5vUnhWoH5NOF2beWTvp5GUQHtHMa7X/d1S/+ZSgt3zT6BAkaJZt6VCg9pQYTqkQNRcgushAOHiolAOBmjjcpKipkD3IOWCZ95SK1S4Xwg3qGwL8qroR02ogkwfiV7zYjqKmWX4ZcyniB78LnBgEBFQT42lLHjAmcUQziTetIxY0VM+IWupDaRpLIzuLiX3gmPUjc5WhRXJmI6O1SDdFyJyiDdRnIvuIR4de1/fJ8m2cTYHZsagKnLLL3w0SQuWQszPq86F90fjys/qjWnRcUoS38jCeOhtdSeEIbs+XHyaBUjR7H00lsXmAgQHchqYq54gCysZO2oopiEzdr4lrc8nxNFWPKF
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8936002)(55236004)(54906003)(38100700002)(4326008)(6486002)(186003)(26005)(558084003)(31696002)(8676002)(53546011)(86362001)(316002)(508600001)(31686004)(2906002)(66946007)(2616005)(66556008)(36756003)(66476007)(5660300002)(6666004)(956004)(16576012)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VHgva0hIVXFxR1p5TUVuSHFYYXNnVDRzbVRjbE1ZQVRUYkhET2Y3U21lYUtp?=
 =?utf-8?B?UEpydXZoSnB0Q094SUlRSDdZY05abHVIVHpCYWlkZUJtT0duZmRobEJjRC9k?=
 =?utf-8?B?RG1uc2dEMW5BSVFOVkJmME9MT21XRWVrSHdNOFp3S014WWFZaDdwVEpRM1FM?=
 =?utf-8?B?eEh0VjliMDR0Vy9HUnlUbW5zSkI2M1pOcllaZkc0MmJJK3pKYXRGbnJkbTN5?=
 =?utf-8?B?Y0xPZmZkY01MakdRT29pMCtFQnJaTUZYbE5UWGowMUt6OGRHRmIvMjBwK3B2?=
 =?utf-8?B?a2pJYzhXcDFTR05kc1dKOVVCZjRvOFFQMk0yRjQ4VnJzTmdkZVQ1SThaMElo?=
 =?utf-8?B?bkZremxzUkJZb3FRL1FweXc1OEZNYmxVeDNWYWtCM0t1Zlo2VUoxSkIxMUR1?=
 =?utf-8?B?dlJXMEZlRGpIcDZLc1Q3UkU3WXpxQ1lGK2FOR04xbG1hcTM5aXRyYmJFME50?=
 =?utf-8?B?ay9LaGRiZERIMXpnZk14djQwU3k4emswRHZiTW9LRHJKbUhWcFVrOEtBT0xC?=
 =?utf-8?B?RkxWTnBEMmYrb3RDMVZTbjQ4NU1hVGpNRHptWW5YRkxNYW1vTmNnaTQ2MG8w?=
 =?utf-8?B?Qi9rUVptOFpqVXB5TVNzU1RMaEd2cnpDVllsc0t0dm9wY1dUQTJmbDhnMTRB?=
 =?utf-8?B?VkFJTnExQmkzMStZbUs1T2hFV2FUYWE4aXRVZng5UzZHWkxGWHZaM3FoTkx4?=
 =?utf-8?B?ZHlvaWZLT2tVeVVUQnYrL0o1TXJxelppam1CUXdwMHlRU3VaRHNmUjhNdkpY?=
 =?utf-8?B?NGVWak1XcTEyZGppSUdvV1ZTVHBSYmdDWGxZS25CaS9nK2VQelg0NWNQYits?=
 =?utf-8?B?b3dzeVZjSkdVSDVuYmJUYmJyS2pVdG9xcFRDN2k5MmVQUHR2ZC9HMjdzTENp?=
 =?utf-8?B?anl1UkNaRzBjVWFtSG5KQTdwK1ZMeC9xbUpOVFJKWmVkemdGbFNnOGROYVNv?=
 =?utf-8?B?ZTdLMlJYRlcrbzllWWk3Q2hoNHVpejBsQ2xCMlN5dzRoRElKc3FWb3JQcWNo?=
 =?utf-8?B?NzQvWU1oZlFITXM2M3hqRXp4bVJGR3VHOHpUTkEzeEdBUXRnaGJNMndNZ2hq?=
 =?utf-8?B?QTBMWXVSUmNUN3dRRnVmdDFBTmpYOGFiRnFVWDZWRzVTcTUyUUdjbGZ1b2hp?=
 =?utf-8?B?Ri9HUjRpS1p0cDJRODN2Yk91cGFJRklrRWZpcHhsS2VKaGQrdGVydHpkWTlK?=
 =?utf-8?B?LzdEWGo0emhoRkVlQkY3U2NVV2pNMEVlbFZwcE04b2tqVGJEVUVWczFFdHVR?=
 =?utf-8?B?V1dsODFRdnNyZXNNanJ0S1VXT1FrYVo4eGRXZCt5UytLTWduNTFXQWFZMGZx?=
 =?utf-8?B?ZS9RSFowdm1ZdFJHWnhSbjg5R2RXYXpHV3NrZVErdndqUk56a21TbWVZckZN?=
 =?utf-8?B?aTI4UE5uODRNL0hNcG5zSkFkbjVpZExXVnVtcUhNTnM3WTBiSjNDU2wvSUZl?=
 =?utf-8?B?M3Roa21xand1eURDTlkrWlYwMVprOWRWb3g5N2drYmQvUFJGRnBDbWh4MEty?=
 =?utf-8?B?anp1bUJyU2tndjdaZkhJZEpHNUhPbEJvL3BiVFJBcjNvaWd1UDVpbjByTG9m?=
 =?utf-8?B?Z0xKM0g0RGFyeUZSK3lMam5ONXVnVTBFWTVoU2dpaHFBd0pFcnBNRkU1dkRJ?=
 =?utf-8?B?YXN6WTBFK0JkOGc4YlNuMHpPVUx4bHIwejNTbzhJNlFmOTJ3V0xnNWJhb2o5?=
 =?utf-8?B?SGlBMEVmcGRjVzVqTFhtd1VWQmlYdlpJU1pObHU1ODFqTHovajFqMXZKdXRa?=
 =?utf-8?Q?082YUrfIErCu+kggDTXvpHlAypw9C2CWxcVMDRf?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fd69e5da-45b9-497f-99dd-08d982c4bace
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 21:12:51.0487
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WyCD0MZDalS/kH6VQfoQl9zZ5QmYWkKbXZ0ZFzEkBhdcBvOiQXUWUgjku/mq1w18lcWk9KnWKpaNl3YgMD5+ldwadCOqNV3Vkr93Yshktj4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5454
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> domain_pause_for_debugger() was previously in debugger.h.  This commit
> moves it to domain.h because its implementation is in domain.c.
>
> Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:23:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:23:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198495.352021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKZD-0008RP-Pn; Tue, 28 Sep 2021 21:23:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198495.352021; Tue, 28 Sep 2021 21:23:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKZD-0008RI-MA; Tue, 28 Sep 2021 21:23:11 +0000
Received: by outflank-mailman (input) for mailman id 198495;
 Tue, 28 Sep 2021 21:23:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKZC-0008RC-Ij
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:23:10 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4d7a31f4-9a1b-4d84-a557-d25183b2e48e;
 Tue, 28 Sep 2021 21:23:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d7a31f4-9a1b-4d84-a557-d25183b2e48e
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632864188;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=YmuC0MxgQv3gWGOe9bQXdJuMkj0hqsaqclxYjYBzEEc=;
  b=aZAuZjL3W7N07/+uxDWATxMy1U2dyYyFfDe+RgaZEQ9obBt8ceQhZoK9
   Oab0mOc7KKdU0eZsqDgGuu75jiytjV+ZWnror02qoHAeeYFMT6bkz+0a6
   n7E6T2A35uXZtPb6+Xhetk7QSdRaoWXh11DxUwYKXrjpyDB9Yp6dwSpPy
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: 58Ol8i+C4rm434DQYfPIcgX1YmQBHfDfhEhZorncbsXx94qsgKak+wNn9SqzfXxBBYe1YSjB4A
 TMK9iBfTWw7fgOmEaMYICxHsXMcO6l2D45KsWDuJP62O8VavXA2UmLAmv6TXl0iYbMGbwDmdmk
 Kz1Ul3oeyhjyJA10iLF8lf43trcrJHKM0ZVcoqwcsqLip9oocDaw112xNZjU6fQ8Qa0kiI7129
 xK/KXJ7JaglIc/t8c3DU7xhqEBaSKlXOsvE2iJ3uv5bTxJGYpIH4I07MA1nxPsCX5mPZOSmh64
 QC9NEjITZ+GDlSbzOdiUV1JP
X-SBRS: 5.1
X-MesageID: 54270631
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:DmPPS6ir8rxOXTeiBMctxIXoX161FRYKZh0ujC45NGQN5FlHY01je
 htvWmvUPvjcZjPxeNp/aoW080wDsMXQzII3TgJvr388EXkb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0FU/NtTo5w7Rg2t8y0YDia++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1Ou6O2bSobNJTxv9kmCCZiGhBEAJxZreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHCOo8Ft24m5jbeFfs8GrjIQrnQ5M8e1zA17ixLNauCP
 ZVBN2E2BPjGSz50EHQpA5wsoOupmnjBSwZArxXNn5NitgA/yyQuieOwYbI5YOeiXt5Jl0yVo
 mbH+WXRARwAMtGbjz2f/RqEhPLLnC79cJIfEvu/7PECqFeUzGYCThANVV+2pPC/okG7UtNbb
 UcT/0IGvaU0sUCmUNT5dxm5u2Kf+A4RXcJKFO834x3LzbDbiy6nAWwDQi9EecYRnsY8Tjw31
 XeEh9rsQzdotdW9RHaQ8L7SsD2zPDQOIHELTSgeRAAB7p/op4RbphPQTP5zHaitlNr3FDrsh
 TeQo0AWhb8ekMoK3KWT5k3cjnSnoZ2hZhExzhXaWCSi9AwRWWK+T9X2sx6Bt68GddvHCAna1
 JQZpySAxOMAMpepuR6HedgqIYry2MyMAQfhsEE6SvHN6A+R03KkeIlR5hR3K0FoLtsIdFfVX
 aPDhe9CzMUNZST7NMebd6r0W5xzkvSwfTjwfqqMBueidKSdY+NuEMtGS0mKwyjJmVUgnLszM
 JOWGSpHJS1BUvw2pNZaqv107FPK+szc7T+PLXwY507+uVZ7WJJzYexeWLdpRrplhJ5oWC2Pr
 75i2zKikn2zqtHWbCjN6pI0JlsXN3U9Dp2eg5UJLbTTf1Y7Rzp9VKC5LVYdl2pNxP89egDgp
 C3VZ6Ol4ACn2S2vxfuiMBiPl48Drb4g9ClmbETAzH6j2mQ5YJbH0UvsX8BfQFXTz8Q6laQcZ
 6BcI62oW60TIhyaq2V1RcSs9+RKKUX07T9iygL4OVDTibY7HFeXkjIlFyOynBQz4t2f75Bj/
 OHwil2BEfLuhW1KVa7rVR5m9Hvo1VA1k+NuRUrYZN5VfUTn6o9xLCLtyPQwJqkxxd/rnVN2D
 i6aXkUVo/fjuYgw/IWbjKyItd7xQeB/AlBbDy/Q6rPvbXvW+W+qwIlhVueUfG+CCDOoqfv6P
 egFnevhNPAnnUpRt9YuGbhc0q9jtcDkoKVXz1o4ESyTPUirEL5pPlKPwdJL6v9W3rZctAbvA
 hCP99BWNK+nIsTgFFJNdgMpYv7ajaMfmyXI7ORzK0L/vXcl8L2CWERUHh+NlC0Cc+clbNJ7m
 b8s4ZdE5Ra+hxwmNse9ohpVr2ncfGYdV6gHt40BBNO5gAQc1VwfM4fXDTX74c/TZowUYFUqO
 DKdmIHLm69YmhjZa3M2GHXAgbhdiJAJtEwYxVMOPQ3UyN/Mh/tx1xxN6zUnCA9Sy0wfge50P
 2FqMWxzJLmPoGg01JQSATj0FlETHgCd92zw10ANxT/QQESfX2DQKHEwZLSW90cD/mMAJjVW8
 dl0Eoo+veoGqC0p4hYPZA==
IronPort-HdrOrdr: A9a23:psYckK6DplwutkWHAQPXwV+BI+orL9Y04lQ7vn2ZFiY7TiXIra
 yTdaoguCMc6AxxZJkh8erwXJVoMkmsiqKdhrNhQYtKPTOWxVdASbsN0WKM+UyZJ8STzJ866U
 4kSdkFNDSSNykLsS+Z2njALz9I+rDum8rJ9ISuvEuFDzsaDJ2Ihz0JezpzeXcGJjWua6BJca
 Z0qvA33AZJLh8sH7qG7zQ+LqT+juyOsKijTQ8NBhYh5gXLpTS06ITiGxzd+hsFSTtAzZor7G
 CAymXCl+uemsD+7iWZ+37Y7pxQltek4txfBPaUgsxQDjn3kA6naKloRrXHljEop+OE7kosjb
 D30lgdFvU2z0mUUnC+oBPr1QWl+DEy60X6wVvdunfnqdyRfkNwN+NxwaZiNjfJ4Uspu99xlI
 hR2XiCipZRBRTc2Azg+tnhTXhR5wmJiEtntdRWo21UUIMYZrMUh5cY5llpHJAJGz+/wJw7Ed
 NpENrX6J9tABOnhkjizyxSKeGXLzAO9k/seDlEhiXV6UkWoJlB9Tpb+CRF9U1wsq7USPF/lq
 z52+pT5ehzpmJ/V9MLOA47e7rDNoX6e2OEDIujGyWUKEg5AQO4l3fW2sR+2Aj4Qu1E8HMN8K
 6xJm+w81RCI37TNQ==
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="54270631"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lAqMaVMTRu29tzAee0e5sm93JHildLzhpCoIIbbtr/vV0fDTUf0Ko8hEmPpccUTHonGXh6c4xhUA9bjjZpCSMDUfk2tAYRGGD9lk3bBrh4eEXh4yMqESwS5nbGbVzGA4XkUd3oXgC6wWJlFQRPuqIMftQOc8d6RmW1fmXcMyTAx8zuspyS7vHeSAKn1jquB4gf+MWzADJU8YZmbSqoO9KofayJ+As91oFKJ2IGKYPh4s76bStl8KVrjSLy/BYs/jQ5T/3tLtTnv1iwCMwU51i7esxWJKe4oRlQQtXeIWC55UKdBWRQqNLKan7WzFWo6mEOZ2HgcsH0ojt+/WknP07A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=YmuC0MxgQv3gWGOe9bQXdJuMkj0hqsaqclxYjYBzEEc=;
 b=AZivaa6KyswFhb+ZFkfz5IkZYuxZOQtopSN+8ou6rJa5n+jKx/iYGsg0oPQOt+w9OpPMPKIwgS3ecuqj9x/xfixe9BVeSHp5SOkyR/9z33Eu2c36RQNxxP/3j3gAETxGDRFLoRPe72D6GeLdrhcd8gQo7pR4wjd8ZeXtzPSgf59GqGp1COMYR3BmBSmyrPXBku5jiZjF6lizuTdiodDnwyD7xhRlpxA0apWvMcbOuKO+I4jhBHQSaKiiTAYStFoRhytN2BgVZzeW1ltih5XmjBY4z1AnwSnuTCb7IzuGlU0cMRLRx1jCj+bLAqyEbza2vyYisROMLDcxNPek7jr6fg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YmuC0MxgQv3gWGOe9bQXdJuMkj0hqsaqclxYjYBzEEc=;
 b=A0fQgvfQ+5R9U8bwLFxespfdSfX1k/6MKRFuGRP2r15NKj8jlIcRTPlCJ8bBQVHkJBCdSRp8jkl31q8dhw745JsKrh6Y4RKk7yeh20VGKL80CQqPqsO2tPxdY/TuYmwFbzEU0bDxpkCWA91VtGkkxXC3iNoxPkvvQa4cyEY+sS8=
Subject: Re: [PATCH v4 4/6] x86/gdbsx: expand dbg_rw_mem() inline
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, George
 Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <472215cdfa754634f8070386bb3122bf3e3cedc1.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <f66fef02-14d4-b70e-30ce-ab372cc03977@citrix.com>
Date: Tue, 28 Sep 2021 22:22:07 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <472215cdfa754634f8070386bb3122bf3e3cedc1.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0389.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:f::17) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d8dc57b5-39f0-4f0f-1563-08d982c60a82
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5693:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5693734E8A8552F75C00A49DBAA89@SJ0PR03MB5693.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:65;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: JrT9ws2knFE0ZbU8nfW4tvyUNawHW1n6ZqM4XK216a1LaKdpaq4UwT9MTDh9PMWlUnR7BuO/SoqG1z9OOj5numNRgvfjloEmurBQ+tSqZkzPdlhu8xfOHlW8QhGgOA29IcPEnrcOLnTaGU7JNvkovTwQEfblOoD7lmKfBUIkKjbH0fyT4LtnZmkU6C15NLzeFduqleal6l68392Vg+slghwDPa5Gg3ZP7SUGlYdJzXDKARS2gflVAExB4Sto5BLVF4LSUPu4ZwGNeO3y33i6ls/p+UxWBBvmIub27XSVEA5bQ8/9Foen07L1CPWI2pM8FYfFx0YCmwqK7NsMMcqM7TMiaSkIFlwOm3lb9VCzDRs+yWTRhpkQT7GM2UDFzFtGp4ODGvHhm8uUWWL4QJnjnJ6fdx7kIM7GuTL8RBOKy8T+IhTAn+aG22h9lfTSDAtpO/1ezvbM5xRU3hJOnPu/xQoSlsG28uV37X8CvQPyRikewqApvosqKwcK9WXAehsTSSvCGAd2hojy5H151Z2XErYMBepgre3pb1FKdP00DyMq9JsTUrNBCm9yOPRpCLqorLQB0hXszUghK6m/0bpnVIIwhdHdih51ON+5kd3pYlERsmPq4QvBIDMAbuD0mCswcwMv4IMdyKGBkIcDYrHyKCCFEHWdL20g6jGCV7REFBmk7v6qPaADFysM00pPeiqxIBPYkfsqEJxnBOMaJDIx/Edhz+OIctxo6z5ewWg2SaOnmKy2AcgzYuiz7oZnG1VA
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(66476007)(66556008)(86362001)(508600001)(31696002)(6666004)(186003)(8676002)(2616005)(956004)(2906002)(6486002)(4326008)(36756003)(558084003)(5660300002)(31686004)(26005)(316002)(54906003)(53546011)(38100700002)(55236004)(16576012)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SURLWmhsNkFCWVdtRi9DenUyWmRSMU9iTnRrN1BOTjJITHJuZ0VSTXdCL1Rs?=
 =?utf-8?B?NzNuQ3VhRGZTZUlIMVVQbGc3a3BoV1UvUkNrNm01UkNDVE5RbkZqV25IOE1D?=
 =?utf-8?B?cm9Ncy9HSlhhck5QbDc3ZWJZL2JqdC9ZazV4YVZzMWdxeWZyM25Pbmd5cmFv?=
 =?utf-8?B?Y2d4RjZUWVpvZk9Rck5DQWU3czczekhKcHp3aXh1enFaUGM3Wkh4MElYTUdB?=
 =?utf-8?B?WEM4Nllqa3VOUnA5R1d0U09BNktiOUxNZDR0K2xTRWY4Tm1hYmRnYzhjMjlY?=
 =?utf-8?B?S3NEU2RleDhOWFdrV2NkUEFsNVJVZ3Npd3QycWpDQjJGelZRSkx2dCtjVFho?=
 =?utf-8?B?S01DczZMclN5anZRaDYyWHdKOXgzTHBHVGJrWkdhQ3E5Z3pHVzBqRTN2QXBi?=
 =?utf-8?B?V3RUN2w5b3daWkJYZmVLdk5WY0xzWFEyd0dpNVU0bldVMzhLdFRacTN0RWd6?=
 =?utf-8?B?NWczV3FzdjQ2SEYzNlBYdDdONXAxTzR3cmt5UnRKT1NvSjF0Y2ZiTzNLTlY2?=
 =?utf-8?B?d0R5SXFmNmJBZ1R2elV4dXBBY2hsamtEM0hvWmhOZWdVR2hxL0lGdlNDSjI0?=
 =?utf-8?B?MzhCZjQyRVFjT1NpWDcvd1VCM2pMOGpSMGZkYlZDZ20vY04wdVRzdnZCT0Vl?=
 =?utf-8?B?UkJpV1dsSEJYcFZNRUNYZnl1b0gyQ2l2UWUrNkdzRW4vUVU4dlNCL3ZLblpW?=
 =?utf-8?B?eEpJcDVvYzFrd2M2RmphQSsvMjhWT2I5Q2d0WlM5TW9BeW1PZEgrRU9PanZL?=
 =?utf-8?B?YXVXcDI2UXRKaDJNK2V3ZWlCZlhFRlBXbmlFTTlLYVRiU0FzcG95SjNlQWlP?=
 =?utf-8?B?ZWlXbkl3cGlEa1FuOUg1eGJuaHRqNmZPbjdZL0pEdGt3cEkxK1hKTlpxTjUy?=
 =?utf-8?B?ZVg0RkxsOGdsVllqYjZCOGMzaWZGNUlxOHQ5WTM0d0VqTlk4UWsrRCt5dlFC?=
 =?utf-8?B?TS9RUU1EVURUTGFZV2RjdEduSGtTUG5WbDdSN1RLalo1am95cmRDZGF0NFJY?=
 =?utf-8?B?VktiWkJ5cTlLdmphS2ZnY1IxSlFVaGpkZXVKYmVlOEJVV2lFUS9ScmUxTVR1?=
 =?utf-8?B?ZFEwcU9TRXg4NE9yWnRFRDRHVGNZVnljTTNPZEM2Y1VlamdFNVptRHhpMjNv?=
 =?utf-8?B?dEJSU3FGTDBsT1lySUxQQjFQd0NaMm01dklidEFnd2o0RkFkOStMSUhaM3VP?=
 =?utf-8?B?Z1M3WGVwZVBmbEh3RjE1M0Vpa3pmZXhHMFlveFhqQ0Z2ek50d3ZXaDJRcDZY?=
 =?utf-8?B?dzVZR0dOK3dyZG1xVXFLRy9DWkh0ZUQxQ0k1ZXdtbFJTRHA1Q1hBQlZUK2xp?=
 =?utf-8?B?eHdFOTNUYmozTXhUWDBjeExCbUVXM00wdGxYM3diS01wUytmcndiZE8zMUs5?=
 =?utf-8?B?aGY0WFU1UjdFVTFNV3BGWFMvZGI1UmhjLzVTd0RoTnZKZEM3ZlpVcUhPQVAw?=
 =?utf-8?B?endjSzlvalpXTDBjZkNSbzNKd2piY0F1NWkwNUVXWndXdGw5SEc3L0xIZFVE?=
 =?utf-8?B?SjNRYjg4SWRoY2d5QnFiRC9NVUN4b2NVeXVtMVNPbDVmaFVPcWZ1bnlBS2V6?=
 =?utf-8?B?Z2JRYUFZNGlOMVJ6c1A4N29IbDJ5NWxHRisrQXRQS1Z4ejZ5eEMyY1pOelZL?=
 =?utf-8?B?KzgyUlJnMm80c2xsdXQyS0lBVzlBOEN6cTVHTzRtaFNHVFpuNVhlRjlXMlB5?=
 =?utf-8?B?RkRhRTcvN3FUUnFaZzNjWnpTZlIwVHg2YXJrTk9rNWhzQzhwOWdoL0czVWtu?=
 =?utf-8?Q?Z3I87ZfTQZ5NZRSq4BfXCkcj7TmYJclM5vBUne9?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d8dc57b5-39f0-4f0f-1563-08d982c60a82
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 21:22:14.2267
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3Wbu25jG56gpLUki0ynwVZWoNgQGHbVifwNn6ggZuCc7NAw1tCJNOqXnV+e9YBMSP7jzTS3ljg8WOQndUrP8UyB2WUz6NFRLbaqqe+L1w00=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5693
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> Because dbg_rw_mem() has only a single call site, this commit
> expands it inline.
>
> Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 21:25:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 21:25:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198500.352032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKbm-0000dh-74; Tue, 28 Sep 2021 21:25:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198500.352032; Tue, 28 Sep 2021 21:25:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVKbm-0000da-3p; Tue, 28 Sep 2021 21:25:50 +0000
Received: by outflank-mailman (input) for mailman id 198500;
 Tue, 28 Sep 2021 21:25:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XEZ0=OS=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVKbl-0000dR-Cj
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 21:25:49 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 246451a2-4a84-4326-8df4-f31d0b11b926;
 Tue, 28 Sep 2021 21:25:48 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 246451a2-4a84-4326-8df4-f31d0b11b926
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632864348;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=r36IDjkSiVvGqD4nOqlPWvK6xFaUyhejF+Z45ussmkI=;
  b=ID6Xi9AQJomxVpS/upURu2kEv6YznzFPr4EBpOXkWYe4t//LrdvWIQw/
   O9X97RpeN0cqSYUTtlvPWe9Iszr5xboSyLy6VFuMkfN+9kO5932ZSA+cN
   TX9JKe6KbB5OY6igtNBvY82S+B5R+cvtnhdHpZqrrrD0e2khy9elKwVBh
   E=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: VVAZle1NcVjyPDZhe5BqVFoVG6kz8g+6PD2H3UycCgEFGeNe2/oYioVwu/oAqSxzQPOt9HAI2u
 yd7uQgxd5E/k23wxXwQlt60c2pvB0C03Z1i2AyxbrcBKUNshLTHVc9ZScQqrRJdHGqPQXRTOio
 +9qEyU+qN2Ea30ry97+w1i0xFvtlEJfD5F3QK7Lzc5FThoX2mn9Jsjty5N1YKS0EAu4qw3Dq6z
 RINOsCYPWgZ/6QOBq3fPpIRYUlwoBL8hqZ06/pxL3v1S1j92dcsFWjcyiJV1QvhF1fd3AeKxSJ
 3wW7NVDxGh3pYQ8oZq1mLa1V
X-SBRS: 5.1
X-MesageID: 53844264
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:XYRm0qNVd21QDsnvrR3QkcFynXyQoLVcMsEvi/4bfWQNrUpw1WZTn
 DccDD2POquJNDP2KYtyO9ux/U4P7JeGyIA2SAto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Egzw7VRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYoxHZtvkqw
 YVWiZahbiR5JJPPgcpCbBYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgWxg2pwUQ6q2i
 8wxZ2o3STHSbzZ2Hlo6FJ4fkraXqWfCSmgNwL6SjfVuuDWCpOBr65D2K8bccNGOQcRTn26bq
 3jA8mC/BQsVXPSd1D6E/3SEluLJ2yThV+o6Hrqy/O8sjUaTwmEWARs+Wl6yoP3/gUm7M/pPJ
 kpR9icwoKwa8E2wUsK7TxC+uGSDvBMXR5xXCeJSwD+Kzq3Y8gOIHF8uRzRKaMElnMIuTDls3
 ViM9/vkATFpufuIR3OY6qyZszqaPjIcJmsPIyQDSGMt4sbhiJE+iArVSdRuG7Lzicf6cRnyy
 T2XqCk1h50IkNUGka68+DjvnDaEtpXPCAkv6W3qsnmNt10jItT/PsrxtAadva0owJulokepm
 CgtspOXvPsyB4Cqt3yXfuYUMu6W6KPQWNHDumJHE54k/jWr3nese4FM/T1zTHtU3tY4lSzBO
 hGK5lIIjHNHFD76NfYoP9LZ59ECl/CIKDjzahzDgjOiiLBfcxWbtAVneEKdxWzklEVEfUoXY
 szAKZfE4Zr3E81aINuKqwU1jeNDKsMWnzq7qXXHI/KPiuH2iJm9E+ttDbd2RrplhJ5oWS2Mm
 zqlC+OEyg9ETMr1aTTN/IgYIDgidCZgX8Cr95EILbLcfWKK/V3N7deKmtvNnKQ/w8xoehrgp
 CnhCie0NnKl7ZE4Fel6Qi86M+6+NXqOhXk6ITYtLT6VN4sLOu6SAFMkX8JvJ9EPrbU7pdYtF
 qVtU5jQU5xnF2WckxxAPMaVkWCXXEny7e54F3H+O2ZXkl8Jb1Ghx+IIiSO1r3RRUnrr65Fuy
 1BivyuCKac+q81ZJJ++QNqkzk+rvGhbn+R3XkDSJcJUdlmq+49vQxEdRNdtSy3VARmclDacy
 SiMBhIU+bvEr4MvqYGbjqGYtYa5VeB5GxMCTWXc6L+3Mwjc/3aintAcALrZI2iFWTOm4rima
 MVU0+r4bK8NkmFVvtcuCL1s168/uYfi/ucI0gR+EXzXRF23Ebc8cGKe1MxCu/QVlL9UsAe7Q
 GyV/dxeNenbMc/pCgdJdgEkcv6CxbcfnTyLtaY5J0Dz5SlW+rubUBoNY0nQ2XIFdLYsadEr2
 +YsvsIS+jeTsBtyP4bUlD1Q+kSNMmcED/ctuKYFDdK5kQEs0FxDP8DRU3ek/JGVZtxQGUA2O
 TvI1rHajrFRy0eeIXo+EX/BgbhUiZgU4U0YyVYDIxKCm8bfh+9x1xpUqGxlQgNQxxRB8uRyJ
 mk0aBElefTQp29l1JpZQmShOwBdHxnIqEX+xmwAmHDdU0T1BHfGK3cwOLrV8U0Um46GkuO3I
 F1MJL7ZbAvX
IronPort-HdrOrdr: A9a23:/gyrVqEjkZ5/D+g0pLqFYZHXdLJyesId70hD6qkvc3Jom52j+P
 xGws526faVslYssHFJo6HlBEDyewKjyXcT2/hvAV7CZnibhILMFuBfBOTZskbd8kHFh4hgPO
 JbAtVD4b7LfCtHZKTBkXGF+r8bqbHtms3Y5pa9vgVQpENRGsZdBm9Ce3am+yZNNXB77PQCZf
 +hD4Z81kCdkSN9VLXLOpBJZZmNm/T70LbdJTIWDR8u7weDyRuu9b7BChCdmjMTSSlGz7sO+X
 XM11WR3NTjj9iLjjvnk0PD5ZVfn9XsjvNFGcy3k8AQbhHhkByhaohNU6CL+Bo1vOaswlA3l8
 SkmWZvA+1Dr1fqOk2lqxrk3AftlB4o9n/Z0FedxUDupMToLQhKQfZptMZ8SF/0+kAgtNZz3O
 ZgxGSCradaChvGgWDU+8XIfwsCrDv3nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De
 hVCt3G7vo+SyLYU5nghBgq/DWQZAV1Iv/fKXJy/PB9kgIm3EyR9nFogfD2xRw7hcsAo5ot3Z
 WODk0nrsAXcie6BZgNctvpevHHflAldyi8eF56EW6XYZ3vBEi93KIfwI9Fqd1CK6Z4gKfbpv
 z6IRplXCgJChnTNfE=
X-IronPort-AV: E=Sophos;i="5.85,330,1624334400"; 
   d="scan'208";a="53844264"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QqrsB+p+vWAVd1ITKTTmP/ehRe6aJFHcG8zg5LPqom6yIvubE0ePtk4Uhp/2KKtsycTSz5EXaYDpI3FFSOWEr1htj5s8rZZUJHJuK116VO63CZcVlATBM+qJ81q7puGEFoYnrBABH32LaD4rz4J2WUW5QDRkiMOU46WKZp4ueACWUFxK9PqQHWYF65o09g8MSRsU7VNlENnuLOkWviqsEETLn68nb4vH8XKL1jQBBX6+IEpZdOMs4Wskz5DpEPE6TGIapBF/PIX3J7lTVh5CyZ4R1uy09uxMYUptYaW0Tt+eh3+kk1AVpajz2ShSLmAh2/ZKJukw6MwfYuhS8LAhbA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FfVQOJQU/3cOO6NgzgvvzaHk93JVCT/jCrFjKqyGcbI=;
 b=kGOPXSmfZlDY9aoAqxybah/6HEpD0CQXR5zfdAr60xwSLi2iSES2V6wqub+G6xk7cM+m8EHUp42mDCo+FbvuSWVRXP8ygpB7LYk8F3cBauR8yN+R9T/wROP5J4iCectn1mtXGvMwSW82A+RNmZydVyRx+ODxyyR1SmAl7GNt95dw/5FJ3vDQAyqE9lorQwS4seWid4v5ph03ovfLd+0KWT+IGeJ5gqZi8Qydn+txVqV+sKwyzR4iJSw3eI29rUuo+GcQgB87x2/usQc4aOBAG41x8ATcc23tMea4DwORrL5gI5Y7oqZFqgSAZPSk/EOe/hqe2G9P0ogLkK+8fH545Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FfVQOJQU/3cOO6NgzgvvzaHk93JVCT/jCrFjKqyGcbI=;
 b=C1JnPUjzgGzDjf9OhSm0JVVw0FvTrwCtXdSNJgQDWqOq8PYJq54gIwm8XbeT0lDPz5PqfOVwiqexUZzSSpthCaZ/ToKtUXvdjDf6ERiezYb96CP4lN4RDj9QbAQz+C5Cg0eyBR9vZPLjNNkdkxU23uEvZFGVtSAQ5l6JiRA2HS4=
To: Bobby Eshleman <bobby.eshleman@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan Beulich
	<jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <8c340b24b41af349069ad55e1f0421e17fad1d6f.1632860589.git.bobby.eshleman@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v4 6/6] x86: change asm/debugger.h to xen/debugger.h
Message-ID: <01339331-1607-dcbe-7d6c-2033d5ae3874@citrix.com>
Date: Tue, 28 Sep 2021 22:25:27 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <8c340b24b41af349069ad55e1f0421e17fad1d6f.1632860589.git.bobby.eshleman@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0382.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a3::34) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a00f317-2cbe-43bf-872c-08d982c68246
X-MS-TrafficTypeDiagnostic: SJ0PR03MB6342:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB6342EBEC2DC1D086904FCDB9BAA89@SJ0PR03MB6342.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 36oikk8XTPQEfO3UiG1hFe+jntmWNLSNiA/OYkzdKycdUISX3plEKzEb35iMKg+Jy4gPEEJf+iPH6KMpYaHRScgK9w3LvgTMV+jB8ErIxil48iv5uW+leKVLcA4ecsdALWt1vXi5XjRfm/M89XMicftq4IM4kaNnilCAUAbqUd37BgeYBVf8mLQgKjVt55esXwlad3HNzd/9ozGxTlMXxT/SFi/OrJy/KACV/HyRM0GX6h895XiOpkJiB3MdJjU+yNBAKVKM8VYzYREaUd/xpJuKVE0T3MbKenGmzIvqfJW5dCiQvRooWUYrkO4h+HC14LgZHevoOW8v7buqw0oUAomJqIXPxkLfZtlsmoCmeBVEuoWnKO7nbtBW8XDkqluHuauLIXucHdYzvbwjm1nrz1CG998MYohZX+Es7ZmQNewwvrguqf79ZWJmf4YBtY69UyjZdZ01bcs+vU0kpczPN5y1AvEdhDVA+/0e+1j1AqQhnb0H0TitH4HN6AUA7jdFkbS2X40dn2W7a/cMExG51j+TUGy54ekfT6IlE2e+8/vFuZgE4XeSIabPmpNEU475RxCV/D2N6BuFXem+RpuQSLC/cIkh+R15ARa6sGkP53pqKotHSx7NKmZxozY6O1t7v8py6yqUfrNg1k0AXyHOVgorGhEdgQbthUHtGeEy3nnr7dL7VISgHdvY2lhMCLz9hFnGXszY/T6JU1nOK0UdT72dQkTFCRn9kGw0cIxKdpFemqOeRkYeQlhWktgQYaKoWTwhOMQ+zkCcEJVALUhMT57O+ou2U1DU7WZ0YT9N2QfUkdV/DyqNSTFaihowqRAgTFqsyf4Zd8vmzJ8EB0JfMA==
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(31686004)(36756003)(2906002)(86362001)(31696002)(54906003)(508600001)(26005)(2616005)(53546011)(956004)(8676002)(8936002)(316002)(16576012)(5660300002)(66946007)(38100700002)(66556008)(55236004)(66476007)(186003)(6486002)(6666004)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SGVSL1hwdmNtNDF4MVhtZ3BNMVI3R3hkT25JMTJvNzA2T1JJemZDTGp3SWl1?=
 =?utf-8?B?dTU1M3NNbGtsMWFmVE04b3dTMHM4cjZZU1NvTUloS0dtRnc3L1VXS0RmUXho?=
 =?utf-8?B?SHFOdmpQcE44cHpLeDA1WXpuZXFrN0xzZHpwcFhvZFRham5RelJLbVRwUnlM?=
 =?utf-8?B?elYybW5oTFR0VmdGa0pmeUhiNGl3NEZqM01oa01zK1IvUnR4WGQ0Wm5LdVRq?=
 =?utf-8?B?M3BWWktRWmxpM3BXWHgzYUNtcGdtWXdDZmUyVEc3YVhLTnRsQkkvVkIwVmQv?=
 =?utf-8?B?MTdEL2FFU1pLSnFmcys3V0NJckEwZkN2em12OGIxckZMR1dHeVJoTk83VDdy?=
 =?utf-8?B?cHNnamJWZWlOUTBtOXYyZ3hnZjZjVEkrUFl1QjZkd1dLR1JmTmVxem1YUnpP?=
 =?utf-8?B?WE1wR0c4RUw4U2VMSjkrSHkwck9yRi81Sm41NHhQdktkemgxN0FvRWN5QTVv?=
 =?utf-8?B?V2o0MXc2bEE0elpsODc3cHIvMTZ4TWc5YUtrWXgwYmxreVhic2FRUnczb1VR?=
 =?utf-8?B?dnUxR3pEZS9NYUtiY202UVBlK3RoVGxQOWlCOWd4SkRPcUFSV0FwN0p5Ui9V?=
 =?utf-8?B?Z0EzVlRBUHk1K2dIN2ZHY3hiYjlxUXM5bW1MeFRvRm5pWkxySFFXNE5vNk9H?=
 =?utf-8?B?a3pZMFZCek1UMWd4ZU0wSDJYenNjTml0TUowemplUlV3aGxQRFhWbDYwS1Nt?=
 =?utf-8?B?QlA1RjkydVVaQmx4dUlOM0xRMkxQd0hwRS8ya0FQY2x6WUZXR3B5TlBSZjFs?=
 =?utf-8?B?ZE03ejRyTE1VUHlRYjYyYkxMQ3FzYlpib3ZOaE85VEdvckF6VlVkYkNDMDdE?=
 =?utf-8?B?YXlsUEJyc1hIU0FRQ3FYaDJ1enJIdkZMeTVFckZCTHYxYThHUUovWmMxTzhM?=
 =?utf-8?B?S1VFc2p0bjJaUWt6TUpENXcxTHpmRi9yMnBzaFQ3SzRWaWhnL0QwZkdDNGdh?=
 =?utf-8?B?MGhWcUJ5VUFXa2RzaVllLzJpM2NSbEt1bElWWUxEVGROMjdRUW5RbndGeVlj?=
 =?utf-8?B?Zzh4Snkzd2N6QjJrQ0VTNnlMWWJuQ20wVkQva25NenBYODJGb1pCd3RoMXdo?=
 =?utf-8?B?djdLOGZrMnVyTzVnNVJvZm5GL1JZK1VrZ01EeXgzNitSZ0VwY2pzVVIzNUpa?=
 =?utf-8?B?OG03SzMrQXRXbklJM05xWTJnc3RTWC9OTHFkVFJreXdtNXZ1T3F3QVpxamo3?=
 =?utf-8?B?aWNUUEJMWVhVNnlac0xCdW9yanI3djAxWkRRQWowM0lOc09BWDhYR2l3Snh5?=
 =?utf-8?B?SHE1bC9xbFI2dU9FY1BMblgzSEs4ZE9VUUpBdTFJKy8zY1doMTNkcldpa2Z0?=
 =?utf-8?B?R3Bib0xKMEJOeGxHeDZEYThPQ0tIaC9RZU5wbElNWmt2WEF1OWxMb210WGxP?=
 =?utf-8?B?MmtzamtvQWVVQkV2Ty9WTFdOVm5hRGRiZnpYZWk3SDRGNCtxOUlaZVpNSjJr?=
 =?utf-8?B?OHVvWkdLTlpLVVZnYmp6clE5SHdqOTMyYkFmMnFlM2NMUGFINGNkbTNBQ25a?=
 =?utf-8?B?ZjhXV3hhb1YrdjdoaE55eGFiT2hOOGVBb1RUbW9pUWMyZnloNTRMR0VaWkFK?=
 =?utf-8?B?Yi94dHFzUU9KYnZ2TU54UlJEYUw1N3N3RXp6OEc1R2hJZkdlM3U4aGJNaEs2?=
 =?utf-8?B?WFg1WEN3aDdUdHQzbGw5VHNkSU5YZTEwSzBRbk5JOWJZNGhBSnFoeDgwQUpq?=
 =?utf-8?B?YUtFSEsxQ0pPdmpkeVo2S1dJNUM3dmJFbU5SSFV2YmlZUm1ySkxYd1F5RFVl?=
 =?utf-8?Q?WEYovQzmZCmh1fb8tjXEZuJRetDl2dIpiV+FpGJ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a00f317-2cbe-43bf-872c-08d982c68246
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 21:25:35.3020
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5TFNOqD1j7q2zPhKX3dOnKqpv5/1x7O5HPSumnTXCblTPQmspn/5XeLQsxZIesh55/YDKVZ4jtcjSe1bi6nTaON0SiZq+D7hTCvMRwjraMg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6342
X-OriginatorOrg: citrix.com

On 28/09/2021 21:30, Bobby Eshleman wrote:
> This commit allows non-x86 architecture to omit the file asm/debugger.h
> if they do not require it.  It changes debugger.h to be a general
> xen/debugger.h which, if CONFIG_CRASH_DEBUG, resolves to include
> asm/debugger.h.
>
> It also changes all asm/debugger.h includes to xen/debugger.h.
>
> Because it is no longer required, arm/debugger.h is removed.
>
> Signed-off-by: Bobby Eshleman <bobby.eshleman@gmail.com>

Julien also acked this patch.

> diff --git a/xen/include/xen/debugger.h b/xen/include/xen/debugger.h
> new file mode 100644
> index 0000000000..ddaa4a938b
> --- /dev/null
> +++ b/xen/include/xen/debugger.h
> @@ -0,0 +1,51 @@
> +/***********************************************************************=
*******
> + * Generic hooks into arch-dependent Xen.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; If not, see <http://www.gnu.org/licenses/>.
> + *
> + * Each debugger should define two functions here:
> + *
> + * 1. debugger_trap_fatal():
> + *  Called when Xen is about to give up and crash. Typically you will us=
e this
> + *  hook to drop into a debug session. It can also be used to hook off
> + *  deliberately caused traps (which you then handle and return non-zero=
).
> + *
> + * 2. debugger_trap_immediate():
> + *  Called if we want to drop into a debugger now.  This is essentially =
the
> + *  same as debugger_trap_fatal, except that we use the current register=
 state
> + *  rather than the state which was in effect when we took the trap.
> + *  For example: if we're dying because of an unhandled exception, we ca=
ll
> + *  debugger_trap_fatal; if we're dying because of a panic() we call
> + *  debugger_trap_immediate().

This comment is now duplicated in x86's asm/debugger.h.=C2=A0 The x86 copy
wants deleting as part of this move.

> + */
> +
> +#ifndef __XEN_DEBUGGER_H__
> +#define __XEN_DEBUGGER_H__
> +
> +#ifdef CONFIG_CRASH_DEBUG
> +
> +#include <asm/debugger.h>
> +
> +#else

#include <xen/types.h> because you need bool and false to make this compile=
.

~Andrew

> +
> +struct cpu_user_regs;
> +
> +static inline bool debugger_trap_fatal(
> +    unsigned int vector, const struct cpu_user_regs *regs)
> +{
> +    return false;
> +}
> +
> +static inline void debugger_trap_immediate(void)
> +{
> +}
> +
> +#endif /* CONFIG_CRASH_DEBUG */
> +
> +#endif /* __XEN_DEBUGGER_H__ */




From xen-devel-bounces@lists.xenproject.org Tue Sep 28 22:03:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 22:03:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198508.352043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLCK-0005GV-6z; Tue, 28 Sep 2021 22:03:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198508.352043; Tue, 28 Sep 2021 22:03:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLCK-0005GO-3Q; Tue, 28 Sep 2021 22:03:36 +0000
Received: by outflank-mailman (input) for mailman id 198508;
 Tue, 28 Sep 2021 22:03:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVLCI-0005GI-Qf
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 22:03:34 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8f0dfba6-681e-4ac6-8f7b-0f305addd103;
 Tue, 28 Sep 2021 22:03:33 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 82CFB6136A;
 Tue, 28 Sep 2021 22:03:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f0dfba6-681e-4ac6-8f7b-0f305addd103
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632866612;
	bh=X58llmZjRCUwrjn8CXsxQA2PeJX9NB7ziIvsYon/dZg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=RxOQgknj//rnwjfjkvYaqWwRHIvTDf5FRykuTaXqKdfJ8LASz+Zu6zP40DzlvIMY4
	 jKXkuR2KaMOGfbbBuU8jqybL6/QQAa70E+VC6HnH4fpwB8nU43TE+Dm9m+buqOwmSK
	 Hjqel68JgCju/H001d0Wfai8tP0QbMxnGKD9M+2DwkX9Z+nYNdJ+BcYGCx+zFIa5oM
	 yXfgGlIT4WHOibKXetiEy5Eh5j2S8Vc8rfj2/3qdZ7vIm8HMiL0gvcpX0CWCU7tkEh
	 jHKFlU0BL2hL83Jkla5elQ486q/SZ+YBLTR3vuHOdHcV8ppsgXkuzXjoJzubos8xaw
	 KAin8mcUOz+TA==
Date: Tue, 28 Sep 2021 15:03:32 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 1/3] arm/efi: Introduce uefi,cfg-load DT property
In-Reply-To: <20210928163209.49611-2-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109281459250.5022@sstabellini-ThinkPad-T480s>
References: <20210928163209.49611-1-luca.fancellu@arm.com> <20210928163209.49611-2-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Luca Fancellu wrote:
> Introduce the uefi,cfg-load DT property of /chosen
> node for ARM whose presence decide whether to force
> the load of the UEFI Xen configuration file.
> 
> The logic is that if any multiboot,module is found in
> the DT, then the uefi,cfg-load property is used to see
> if the UEFI Xen configuration file is needed.
> 
> Modify a comment in efi_arch_use_config_file, removing
> the part that states "dom0 required" because it's not
> true anymore with this commit.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

The patch looks good. Only one minor change: given that this is a Xen
parameter that we are introducing and not a parameter defined by UEFI
Forum, I think uefi,cfg-load should be called xen,uefi-cfg-load instead.
Because "xen," is our prefix, while "uefi," is not.

With that minor change:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


Note that the uefi,binary property is different because that property is
for xen,domain nodes, so we are already in a Xen specific namespace when
we are using it. Instead this property is for /chosen which is not a Xen
specific node.



> ---
> v3 changes:
> - add documentation to misc/arm/device-tree/booting.txt
> - Modified variable name and logic from skip_cfg_file to
> load_cfg_file
> - Add in the commit message that I'm modifying a comment.
> v2 changes:
> - Introduced uefi,cfg-load property
> - Add documentation about the property
> ---
>  docs/misc/arm/device-tree/booting.txt |  8 ++++++++
>  docs/misc/efi.pandoc                  |  2 ++
>  xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
>  3 files changed, 33 insertions(+), 5 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 44cd9e1a9a..cf878b478e 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
>  for Dom0 and bootargs for native Linux.
>  
>  
> +UEFI boot and DT
> +================
> +
> +When Xen is booted using UEFI, it doesn't read the configuration file if any
> +multiboot module is specified. To force Xen to load the configuration file, the
> +boolean property uefi,cfg-load must be declared in the /chosen node.
> +
> +
>  Creating Multiple Domains directly from Xen
>  ===========================================
>  
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ac3cd58cae..e289c5e7ba 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
>  bootloader provides a device tree containing modules then any configuration
>  files are ignored, and the bootloader is responsible for populating all
>  relevant device tree nodes.
> +The property "uefi,cfg-load" can be specified in the /chosen node to force Xen
> +to load the configuration file even if multiboot modules are found.
>  
>  Once built, `make install-xen` will place the resulting binary directly into
>  the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..4f1b01757d 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
>  
>  static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
>  {
> +    bool load_cfg_file = true;
>      /*
>       * For arm, we may get a device tree from GRUB (or other bootloader)
>       * that contains modules that have already been loaded into memory.  In
> -     * this case, we do not use a configuration file, and rely on the
> -     * bootloader to have loaded all required modules and appropriate
> -     * options.
> +     * this case, we search for the property uefi,cfg-load in the /chosen node
> +     * to decide whether to skip the UEFI Xen configuration file or not.
>       */
>  
>      fdt = lookup_fdt_config_table(SystemTable);
>      dtbfile.ptr = fdt;
>      dtbfile.need_to_free = false; /* Config table memory can't be freed. */
> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
> +
> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
> +    {
> +        /* Locate chosen node */
> +        int node = fdt_subnode_offset(fdt, 0, "chosen");
> +        const void *cfg_load_prop;
> +        int cfg_load_len;
> +
> +        if ( node > 0 )
> +        {
> +            /* Check if uefi,cfg-load property exists */
> +            cfg_load_prop = fdt_getprop(fdt, node, "uefi,cfg-load",
> +                                        &cfg_load_len);
> +            if ( !cfg_load_prop )
> +                load_cfg_file = false;
> +        }
> +    }
> +
> +    if ( !fdt || load_cfg_file )
>      {
>          /*
>           * We either have no FDT, or one without modules, so we must have a
> -         * Xen EFI configuration file to specify modules.  (dom0 required)
> +         * Xen EFI configuration file to specify modules.
>           */
>          return true;
>      }
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 22:46:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 22:46:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198517.352053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLrT-0001Yl-Az; Tue, 28 Sep 2021 22:46:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198517.352053; Tue, 28 Sep 2021 22:46:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLrT-0001Ye-89; Tue, 28 Sep 2021 22:46:07 +0000
Received: by outflank-mailman (input) for mailman id 198517;
 Tue, 28 Sep 2021 22:46:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVLrR-0001YY-48
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 22:46:05 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6603ce58-ea49-47b4-bc35-cfa249efeed0;
 Tue, 28 Sep 2021 22:46:02 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BDC1060F4F;
 Tue, 28 Sep 2021 22:46:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6603ce58-ea49-47b4-bc35-cfa249efeed0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632869161;
	bh=iYhQyp0qqty8xbtUNQeFlurSIIW0Kb9Ood31Of3E/PA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=eLGt8Y6E66B38X00n8TBeKPC6aNPOU9ovUW0Qm3BdAy+RZzy2ICThNA2YbwoBSuCx
	 IP7DJ4uMlUWva7ye9hCEecFKjOA20YafUFiGFpDfnBhCW3ixkszF4uOZk/2n3Tt/eR
	 Uya1Dla7V2/FlVZFBCxty8hpNathlL5AnzRdU6jNsClUW6ZKK6meDwjl3KhgioTgGy
	 oF+7d+ha/C9eNY9zylCHfO0baHhd+3IDjojq/CCGR9w4dWTUZfayVO3l+3CdqMuQx0
	 mhMc6Bre8L//PtJIiwDG6b6aAHfMlKvcNQM4KShWo7LYfiSVkjwTeblDp5IlHwIAbi
	 1+qbpt1q0sCNQ==
Date: Tue, 28 Sep 2021 15:46:01 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 3/3] arm/efi: load dom0 modules from DT using UEFI
In-Reply-To: <20210928163209.49611-4-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109281534480.5022@sstabellini-ThinkPad-T480s>
References: <20210928163209.49611-1-luca.fancellu@arm.com> <20210928163209.49611-4-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Luca Fancellu wrote:
> Add support to load Dom0 boot modules from
> the device tree using the uefi,binary property.
> 
> Update documentation about that.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

It is great how simple this patch is!

The patch looks all correct. Only one question: do we need a check to
make sure the dom0 ramdisk is not loaded twice? Once via uefi,binary and
another time via the config file? In other words...

> ---
> Changes in v3:
> - new patch
> ---
>  docs/misc/arm/device-tree/booting.txt |  8 ++++
>  docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
>  xen/arch/arm/efi/efi-boot.h           | 36 +++++++++++++--
>  xen/common/efi/boot.c                 | 12 ++---
>  4 files changed, 108 insertions(+), 12 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 354bb43fe1..e73f6476d4 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -70,6 +70,14 @@ Each node contains the following properties:
>  	priority of this field vs. other mechanisms of specifying the
>  	bootargs for the kernel.
>  
> +- uefi,binary (UEFI boot only)
> +
> +	String property that specifies the file name to be loaded by the UEFI
> +	boot for this module. If this is specified, there is no need to specify
> +	the reg property because it will be created by the UEFI stub on boot.
> +	This option is needed only when UEFI boot is used, the node needs to be
> +	compatible with multiboot,kernel or multiboot,ramdisk.
> +
>  Examples
>  ========
>  
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index 800e67a233..4cebc47a18 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,6 +167,28 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +## UEFI boot and Dom0 modules on ARM
> +
> +When booting using UEFI on ARM, it is possible to specify the Dom0 modules
> +directly from the device tree without using the Xen configuration file, here an
> +example:
> +
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "initrd-3.0.31-0.4-xen";
> +	};
> +}
>  
>  ## UEFI boot and dom0less on ARM
>  
> @@ -326,10 +348,10 @@ chosen {
>  ### Boot Xen, Dom0 and DomU(s)
>  
>  This configuration is a mix of the two configuration above, to boot this one
> -the configuration file must be processed so the /chosen node must have the
> -"uefi,cfg-load" property.
> +the configuration file can be processed or the Dom0 modules can be read from
> +the device tree.
>  
> -Here an example:
> +Here the first example:
>  
>  Xen configuration file:
>  
> @@ -369,4 +391,40 @@ chosen {
>  };
>  ```
>  
> +Here the second example:
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "initrd-3.0.31-0.4-xen";
> +	};
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
>  
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 4f7c913f86..df63387136 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -31,8 +31,10 @@ static unsigned int __initdata modules_idx;
>  #define ERROR_MISSING_DT_PROPERTY   (-3)
>  #define ERROR_RENAME_MODULE_NAME    (-4)
>  #define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DOM0_ALREADY_FOUND    (-6)
>  #define ERROR_DT_MODULE_DOMU        (-1)
>  #define ERROR_DT_CHOSEN_NODE        (-2)
> +#define ERROR_DT_MODULE_DOM0        (-3)
>  
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
> @@ -45,7 +47,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
>  static int handle_module_node(EFI_FILE_HANDLE dir_handle,
>                                int module_node_offset,
>                                int reg_addr_cells,
> -                              int reg_size_cells);
> +                              int reg_size_cells,
> +                              bool is_domu_module);
>  static bool is_boot_module(int dt_module_offset);
>  static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>                                         int domain_node);
> @@ -701,7 +704,8 @@ static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
>  static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>                                       int module_node_offset,
>                                       int reg_addr_cells,
> -                                     int reg_size_cells)
> +                                     int reg_size_cells,
> +                                     bool is_domu_module)
>  {
>      const void *uefi_name_prop;
>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> @@ -743,6 +747,24 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>          return ERROR_SET_REG_PROPERTY;
>      }
>  
> +    if ( !is_domu_module &&
> +         (fdt_node_check_compatible(fdt, module_node_offset,
> +                                    "multiboot,kernel") == 0) )
> +    {
> +        /*
> +         * This is the Dom0 kernel, wire it to the kernel variable because it
> +         * will be verified by the shim lock protocol later in the common code.
> +         */
> +        if ( kernel.addr )
> +        {
> +            PrintMessage(L"Dom0 kernel already found in cfg file.");
> +            return ERROR_DOM0_ALREADY_FOUND;
> +        }
> +        kernel.need_to_free = false; /* Freed using the module array */
> +        kernel.addr = file->addr;
> +        kernel.size = file->size;
> +    }

... is it necessary to update ramdisk as well or check if it is already
set? As far as I can tell it is the only other potential conflict that
we could have.


>      return 0;
>  }
>  
> @@ -799,7 +821,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>          if ( is_boot_module(module_node) )
>          {
>              int ret = handle_module_node(dir_handle, module_node, addr_cells,
> -                                         size_cells);
> +                                         size_cells, true);
>              if ( ret < 0 )
>                  return ret;
>          }
> @@ -809,7 +831,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>  
>  /*
>   * This function checks for xen domain nodes under the /chosen node for possible
> - * domU guests to be loaded.
> + * dom0 and domU guests to be loaded.
>   * Returns the number of modules loaded or a negative number for error.
>   */
>  static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> @@ -836,6 +858,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>              if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>                  return ERROR_DT_MODULE_DOMU;
>          }
> +        else if ( is_boot_module(node) )
> +        {
> +            if ( handle_module_node(dir_handle, node, addr_len, size_len,
> +                                    false) < 0 )
> +                return ERROR_DT_MODULE_DOM0;
> +        }
>      }
>  
>      /* Free dom0less file names if any */
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index c8c57fbb54..b221494a06 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> -
> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> -                            (void **)&shim_lock)) &&
> -                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> -                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
>          }
>  
>          if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
> @@ -1372,6 +1367,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      if (dt_module_found < 0)
>          /* efi_arch_check_dt_boot throws some error */
>          blexit(L"Error processing boot modules on DT.");
> +
> +    /* If Dom0 is specified, verify it */
> +    if ( kernel.ptr &&
> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> +                                           (void **)&shim_lock)) &&
> +        (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status);
>      /*
>       * Check if a proper configuration is provided to start Xen:
>       *  - Dom0 specified (minimum required)
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 22:47:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 22:47:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198524.352064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLt4-0002Ip-Me; Tue, 28 Sep 2021 22:47:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198524.352064; Tue, 28 Sep 2021 22:47:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVLt4-0002Ii-JG; Tue, 28 Sep 2021 22:47:46 +0000
Received: by outflank-mailman (input) for mailman id 198524;
 Tue, 28 Sep 2021 22:47:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gf2A=OS=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVLt3-0002Ia-6k
 for xen-devel@lists.xenproject.org; Tue, 28 Sep 2021 22:47:45 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1710e306-20ae-11ec-bcd3-12813bfff9fa;
 Tue, 28 Sep 2021 22:47:42 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 1A72C6101A;
 Tue, 28 Sep 2021 22:47:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1710e306-20ae-11ec-bcd3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632869262;
	bh=cgHPUBGn6e1OCk62LANteEpGO6CUsa8C+XxARpkB/4Q=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TL65Abj076qaverf7vf2clRZPHiA29idlwOP2V0NtrD3sPm1hMeKSUlprAhhD6qV4
	 58PolIYqFXwncNU//0WxosmCXbAukcnQqLfeX6UVP2zo50Y/0JU5fzy3YONBUlDz9+
	 jPriOC14ckCvW0u9bqjSkRkcqCy170+NrTeFfwFOin3tGTWK/y8NpxADFxEcgT8BLw
	 U5b1uZjv5t8yKRPJUDYTP6kEK0WSg6+91RXJ+nUAm1yjvWk1KyaFPS2ftsDJumIOhR
	 uZ0DTW/Z01wcFuGvlPeOetHpH3ZRxI7ctOx77hiHisK62nP+y+/tJoywlMs4M9+72v
	 JGk7dcMPuNzJA==
Date: Tue, 28 Sep 2021 15:47:41 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v3 2/3] arm/efi: Use dom0less configuration when using
 EFI boot
In-Reply-To: <20210928163209.49611-3-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109281510030.5022@sstabellini-ThinkPad-T480s>
References: <20210928163209.49611-1-luca.fancellu@arm.com> <20210928163209.49611-3-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Luca Fancellu wrote:
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "uefi,binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Some minor feedback about code style and comments below. With those
addressed:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v3:
> - fixed documentation
> - fixed name len in strlcpy
> - fixed some style issues
> - closed filesystem handle before calling blexit
> - passed runtime errors up to the stack instead
> of calling blexit
> - renamed names and function to make them more
> general in prevision to load also Dom0 kernel
> and ramdisk from DT
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
>  docs/misc/arm/device-tree/booting.txt |  21 ++
>  docs/misc/efi.pandoc                  | 203 +++++++++++++++++
>  xen/arch/arm/efi/efi-boot.h           | 305 +++++++++++++++++++++++++-
>  xen/arch/x86/efi/efi-boot.h           |   6 +
>  xen/common/efi/boot.c                 |  42 ++--
>  5 files changed, 562 insertions(+), 15 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index cf878b478e..354bb43fe1 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
>  
>      Command line parameters for the guest kernel.
>  
> +- uefi,binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  The ramdisk sub-node has the following properties:
>  
>  - compatible
> @@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
>      Specifies the physical address of the ramdisk in RAM and its
>      length.
>  
> +- uefi,binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  
>  Example
>  =======
> @@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
>      Specifies the physical address of the device tree binary fragment
>      RAM and its length.
>  
> +- uefi,binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  As an example:
>  
>          module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index e289c5e7ba..800e67a233 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "uefi,binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	compatible = "xen,domain";
> +	memory = <0 0x20000>;
> +	cpus = <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		uefi,binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "console=ttyAMA0";
> +	};
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		uefi,binary = "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +These are the different ways to boot a Xen system from UEFI:
> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in the
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen node,
> +examples of how to do that are provided by the documentation about dom0less
> +and the example above shows how to use the "uefi,binary" property to use the
> +UEFI stub for module loading.
> +When adding DomU modules to device tree, also add the property
> +uefi,cfg-load under chosen for Xen to load the Xen config file.

Just a note to remind to update uefi,cfg-load to xen,uefi-cfg-load


> +Otherwise, Xen will skip the config file and rely on device tree alone.
> +
> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	uefi,cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this one
> +the configuration file must be processed so the /chosen node must have the
> +"uefi,cfg-load" property.

also here


> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	uefi,cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			uefi,binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 4f1b01757d..4f7c913f86 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,49 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_UEFI_MODULES (MAX_MODULES - 2)
> +static struct file __initdata module_binary;
> +static module_name __initdata modules[MAX_UEFI_MODULES];
> +static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
> +static unsigned int __initdata modules_idx;
> +
> +#define ERROR_BINARY_FILE_NOT_FOUND (-1)
> +#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
> +#define ERROR_ALLOC_MODULE_NAME     (-2)
> +#define ERROR_MISSING_DT_PROPERTY   (-3)
> +#define ERROR_RENAME_MODULE_NAME    (-4)
> +#define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DT_MODULE_DOMU        (-1)
> +#define ERROR_DT_CHOSEN_NODE        (-2)
> +
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int get_module_file_index(const char *name, unsigned int name_len);
> +static void PrintMessage(const CHAR16 *s);
> +static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                const char *name,
> +                                unsigned int name_len);
> +static int handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                              int module_node_offset,
> +                              int reg_addr_cells,
> +                              int reg_size_cells);
> +static bool is_boot_module(int dt_module_offset);
> +static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                       int domain_node);
> +static int efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle);
> +
>  #define DEVICE_TREE_GUID \
>  {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
>  
> @@ -552,8 +592,260 @@ static void __init efi_arch_handle_module(const struct file *file,
>                           kernel.size) < 0 )
>              blexit(L"Unable to set reg property.");
>      }
> -    else
> +    else if ( file != &module_binary )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>          blexit(L"Unknown module type");
> +
> +    /*
> +     * modules_available is decremented here because for each dom0
> +     * file added, there will be an additional bootmodule, so the number
> +     * of dom0less module files will be decremented because there is

the number of "boot" module files


> +     * a maximum amount of bootmodules that can be loaded.
> +     */
> +    modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the modules array or a negative number if no
> + * file with that name is found.
> + */
> +static int __init get_module_file_index(const char *name,
> +                                        unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret = ERROR_BINARY_FILE_NOT_FOUND;
> +
> +    for ( i = 0; i < modules_idx; i++ )
> +    {
> +        module_name *mod = &modules[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +static void __init PrintMessage(const CHAR16 *s)
> +{
> +    PrintStr(s);
> +    PrintStr(newline);
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it returns the
> + * index of the file in the modules array or a negative number on error.
> + */
> +static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                       const char *name,
> +                                       unsigned int name_len)
> +{
> +    module_name *file_name;
> +    union string module_name;
> +    int ret;
> +
> +    /*
> +     * Check if there is any space left for a module, the variable
> +     * modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !modules_available )
> +    {
> +        PrintMessage(L"No space left for modules");
> +        return ERROR_ALLOC_MODULE_NO_SPACE;
> +    }
> +
> +    module_name.cs = name;
> +    ret = modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &modules[ret];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +    {
> +        PrintMessage(L"Error allocating memory for module binary name");
> +        return ERROR_ALLOC_MODULE_NAME;
> +    }
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len + 1);
> +    file_name->name_len = name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr = module_binary.addr;
> +    file_name->size = module_binary.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    modules_idx++;
> +
> +    return ret;
> +}
> +
> +/*
> + * This function checks for the presence of the uefi,binary property in the
> + * module, if found it loads the binary as module and sets the right address
> + * for the reg property into the module DT node.
> + */
> +static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                                     int module_node_offset,
> +                                     int reg_addr_cells,
> +                                     int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx;
> +    module_name *file;
> +
> +    /* Read uefi,binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return 0;
> +
> +    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
> +    if ( file_idx < 0 )
> +    {
> +        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
> +                                        uefi_name_len);
> +        if ( file_idx < 0 )
> +            return file_idx;
> +    }
> +
> +    file = &modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +    {
> +        PrintMessage(L"Unable to modify module node name.");
> +        return ERROR_RENAME_MODULE_NAME;
> +    }
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +    {
> +        PrintMessage(L"Unable to set module reg property.");
> +        return ERROR_SET_REG_PROPERTY;
> +    }
> +
> +    return 0;
> +}
> +
> +static bool __init is_boot_module(int dt_module_offset)
> +{
> +    if ( (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,kernel") == 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,ramdisk") == 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,device-tree") == 0) )
> +        return true;
> +
> +    return false;
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + * Returns 0 on success, negative number on error.
> + */
> +static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                              int domain_node)
> +{
> +    int module_node, addr_cells, size_cells, len;
> +    const struct fdt_property *prop;
> +
> +    /* Get #address-cells and #size-cells from domain node */
> +    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#address-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#size-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    /*
> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
> +     * inside this node
> +     */
> +    for ( module_node = fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +        if ( is_boot_module(module_node) )
> +        {
> +            int ret = handle_module_node(dir_handle, module_node, addr_cells,
> +                                         size_cells);
> +            if ( ret < 0 )
> +                return ret;
> +        }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + * Returns the number of modules loaded or a negative number for error.
> + */
> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen, node, addr_len, size_len;
> +    unsigned int i = 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +    {
> +        PrintMessage(L"Unable to setup chosen node");
> +        return ERROR_DT_CHOSEN_NODE;
> +    }
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
> +        {
> +            /* Found a node with compatible xen,domain; handle this node. */
> +            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> +                return ERROR_DT_MODULE_DOMU;
> +        }
> +    }
> +
> +    /* Free dom0less file names if any */

Free boot modules file names if any


> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free dom0less binary names */

Free boot modules binary names


> +        efi_bs->FreePool(modules[i].name);
> +    }
> +
> +    return modules_idx;
>  }
>  
>  static void __init efi_arch_cpu(void)
> @@ -562,8 +854,19 @@ static void __init efi_arch_cpu(void)
>  
>  static void __init efi_arch_blexit(void)
>  {
> +    unsigned int i = 0;
> +
>      if ( dtbfile.need_to_free )
>          efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free dom0less files if any */

Free boot module files if any


> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free dom0less binary names */

Free boot modules binary names


> +        efi_bs->FreePool(modules[i].name);
> +        /* Free dom0less binaries */

Free boot modules binaries


> +        efi_bs->FreePages(modules[i].addr,
> +                          PFN_UP(modules[i].size));
> +    }
>      if ( memmap )
>          efi_bs->FreePool(memmap);
>  }
> diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
> index 9b0cc29aae..f5538bfad4 100644
> --- a/xen/arch/x86/efi/efi-boot.h
> +++ b/xen/arch/x86/efi/efi-boot.h
> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
>      efi_bs->FreePool(ptr);
>  }
>  
> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    /* x86 doesn't support device tree boot */
> +    return 0;
> +}
> +
>  static void __init efi_arch_cpu(void)
>  {
>      uint32_t eax = cpuid_eax(0x80000000);
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..c8c57fbb54 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
>      EFI_LOADED_IMAGE *loaded_image;
>      EFI_STATUS status;
> -    unsigned int i, argc;
> -    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
> +    unsigned int i, argc = 0;
> +    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
>      UINTN gop_mode = ~0;
>      EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    int dt_module_found;
> +    EFI_FILE_HANDLE dir_handle;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1218,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>      efi_arch_relocate_image(0);
>  
> +    /* Get the file system interface. */
> +    dir_handle = get_parent_handle(loaded_image, &file_name);
> +
>      if ( use_cfg_file )
>      {
> -        EFI_FILE_HANDLE dir_handle;
>          UINTN depth, cols, rows, size;
>  
>          size = cols = rows = depth = 0;
> @@ -1229,9 +1233,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>          gop = efi_get_gop();
>  
> -        /* Get the file system interface. */
> -        dir_handle = get_parent_handle(loaded_image, &file_name);
> -
>          /* Read and parse the config file. */
>          if ( read_section(loaded_image, L"config", &cfg, NULL) )
>              PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1286,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +        if ( name.s )
> +            option_str = split_string(name.s);
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
> +             name.s )
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> @@ -1361,12 +1361,26 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +    dt_module_found = efi_arch_check_dt_boot(dir_handle);
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if (dt_module_found < 0)

code style if ( dt_module_found < 0 )


> +        /* efi_arch_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");
> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !dt_module_found && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");
> +
>      efi_arch_edd();
>  
>      /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Tue Sep 28 23:22:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 28 Sep 2021 23:22:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198533.352075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVMQA-0006oG-KD; Tue, 28 Sep 2021 23:21:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198533.352075; Tue, 28 Sep 2021 23:21:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVMQA-0006o9-HJ; Tue, 28 Sep 2021 23:21:58 +0000
Received: by outflank-mailman (input) for mailman id 198533;
 Tue, 28 Sep 2021 23:21:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVMQ9-0006nx-5D; Tue, 28 Sep 2021 23:21:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVMQ8-0002vA-Pg; Tue, 28 Sep 2021 23:21:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVMQ8-00063A-F5; Tue, 28 Sep 2021 23:21:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVMQ8-00021V-EZ; Tue, 28 Sep 2021 23:21:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Vv5s9w5gMjZXQFG8qF0PSrnSYDK4YgiFl+0YmGZKXlM=; b=xz723aoyTPqlYemIGdJ3sjVi9V
	eRx1qQV/X++yvpqhp4huZvTgEUtNB4vtoxRLHdXwegIh1vEFg8I5LrkE+2hntywwDetpA4bNAaIsZ
	UUckRmMi5qcFnnMY3WwpI6be9eAxRFIAhJkGP/gsUfLMZzQnuiKwG8mDLbK5G/4iy0lQ=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165234-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165234: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    qemu-mainline:test-armhf-armhf-xl:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-credit2:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-cubietruck:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-multivcpu:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-vhd:<job status>:broken:regression
    qemu-mainline:test-amd64-i386-qemuu-rhel6hvm-amd:<job status>:broken:regression
    qemu-mainline:test-amd64-coresched-i386-xl:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-credit1:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-arndale:<job status>:broken:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:<job status>:broken:regression
    qemu-mainline:test-amd64-i386-xl-vhd:<job status>:broken:regression
    qemu-mainline:test-armhf-armhf-xl-credit2:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl-multivcpu:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl-arndale:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl-vhd:host-install(5):broken:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:host-install(5):broken:regression
    qemu-mainline:test-amd64-coresched-i386-xl:host-install(5):broken:regression
    qemu-mainline:test-amd64-i386-qemuu-rhel6hvm-amd:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl-credit1:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl:host-install(5):broken:regression
    qemu-mainline:test-armhf-armhf-xl-cubietruck:host-install(5):broken:regression
    qemu-mainline:test-amd64-i386-xl-xsm:debian-install:fail:regression
    qemu-mainline:test-amd64-i386-xl-vhd:debian-di-install:fail:regression
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:build-armhf-libvirt:libvirt-build:fail:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:host-install(5):broken:allowable
    qemu-mainline:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    qemu-mainline:test-amd64-i386-xl-vhd:capture-logs(13):broken:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=14f02d8a9ec1746823c106933a4c8f062f9e0f95
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 28 Sep 2021 23:21:56 +0000

flight 165234 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165234/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl             <job status>                 broken
 test-armhf-armhf-xl-credit2     <job status>                 broken
 test-armhf-armhf-xl-cubietruck    <job status>                 broken
 test-armhf-armhf-xl-multivcpu    <job status>                 broken
 test-armhf-armhf-xl-rtds        <job status>                 broken
 test-armhf-armhf-xl-vhd         <job status>                 broken
 test-amd64-i386-qemuu-rhel6hvm-amd    <job status>                 broken
 test-amd64-coresched-i386-xl    <job status>                 broken
 test-armhf-armhf-xl-credit1     <job status>                 broken
 test-armhf-armhf-xl-arndale     <job status>                 broken
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm    <job status>             broken
 test-amd64-i386-xl-vhd          <job status>                 broken
 test-armhf-armhf-xl-credit2   5 host-install(5)        broken REGR. vs. 164950
 test-armhf-armhf-xl-multivcpu  5 host-install(5)       broken REGR. vs. 164950
 test-armhf-armhf-xl-arndale   5 host-install(5)        broken REGR. vs. 164950
 test-armhf-armhf-xl-vhd       5 host-install(5)        broken REGR. vs. 164950
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 5 host-install(5) broken REGR. vs. 164950
 test-amd64-coresched-i386-xl  5 host-install(5)        broken REGR. vs. 164950
 test-amd64-i386-qemuu-rhel6hvm-amd  5 host-install(5)  broken REGR. vs. 164950
 test-armhf-armhf-xl-credit1   5 host-install(5)        broken REGR. vs. 164950
 test-armhf-armhf-xl           5 host-install(5)        broken REGR. vs. 164950
 test-armhf-armhf-xl-cubietruck  5 host-install(5)      broken REGR. vs. 164950
 test-amd64-i386-xl-xsm       12 debian-install           fail REGR. vs. 164950
 test-amd64-i386-xl-vhd       12 debian-di-install        fail REGR. vs. 164950
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 164950

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      5 host-install(5)        broken REGR. vs. 164950

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-amd64-i386-xl-vhd       13 capture-logs(13)             broken never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                14f02d8a9ec1746823c106933a4c8f062f9e0f95
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   17 days
Failing since        164967  2021-09-13 13:06:52 Z   15 days   30 attempts
Testing same since   165234  2021-09-28 12:56:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cleber Rosa <crosa@redhat.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dongli Zhang <dongli.zhang@oracle.com>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Gupta <pankaj.gupta.linux@gmail.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Wainer dos Santos Moschetta <wainersm@redhat.com>
  Willian Rampazzo <willianr@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          broken  
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 broken  
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  broken  
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           broken  
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  broken  
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  broken  
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  broken  
 test-armhf-armhf-xl-cubietruck                               broken  
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                broken  
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     broken  
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      broken  
 test-amd64-i386-xl-vhd                                       broken  


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-xl broken
broken-job test-armhf-armhf-xl-credit2 broken
broken-job test-armhf-armhf-xl-cubietruck broken
broken-job test-armhf-armhf-xl-multivcpu broken
broken-job test-armhf-armhf-xl-rtds broken
broken-job test-armhf-armhf-xl-vhd broken
broken-job test-amd64-i386-qemuu-rhel6hvm-amd broken
broken-job test-amd64-coresched-i386-xl broken
broken-job test-armhf-armhf-xl-credit1 broken
broken-job test-armhf-armhf-xl-arndale broken
broken-job test-amd64-i386-xl-qemuu-debianhvm-i386-xsm broken
broken-job test-amd64-i386-xl-vhd broken
broken-step test-amd64-i386-xl-vhd capture-logs(13)
broken-step test-armhf-armhf-xl-credit2 host-install(5)
broken-step test-armhf-armhf-xl-multivcpu host-install(5)
broken-step test-armhf-armhf-xl-arndale host-install(5)
broken-step test-armhf-armhf-xl-rtds host-install(5)
broken-step test-armhf-armhf-xl-vhd host-install(5)
broken-step test-amd64-i386-xl-qemuu-debianhvm-i386-xsm host-install(5)
broken-step test-amd64-coresched-i386-xl host-install(5)
broken-step test-amd64-i386-qemuu-rhel6hvm-amd host-install(5)
broken-step test-armhf-armhf-xl-credit1 host-install(5)
broken-step test-armhf-armhf-xl host-install(5)
broken-step test-armhf-armhf-xl-cubietruck host-install(5)

Not pushing.

(No revision log; it would be 6138 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 01:00:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 01:00:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198553.352090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVNxK-0007Vz-OE; Wed, 29 Sep 2021 01:00:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198553.352090; Wed, 29 Sep 2021 01:00:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVNxK-0007VY-Kz; Wed, 29 Sep 2021 01:00:18 +0000
Received: by outflank-mailman (input) for mailman id 198553;
 Wed, 29 Sep 2021 01:00:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YPmi=OT=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mVNxI-0006Wx-AB
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 01:00:16 +0000
Received: from mx0a-00069f02.pphosted.com (unknown [205.220.165.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 995f63eb-20c0-11ec-bcd3-12813bfff9fa;
 Wed, 29 Sep 2021 01:00:14 +0000 (UTC)
Received: from pps.filterd (m0246629.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18T0uGFR027809; 
 Wed, 29 Sep 2021 00:59:48 GMT
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79])
 by mx0b-00069f02.pphosted.com with ESMTP id 3bbj90t6c6-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 00:59:48 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1])
 by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18T0u7oZ080542;
 Wed, 29 Sep 2021 00:59:47 GMT
Received: from nam12-bn8-obe.outbound.protection.outlook.com
 (mail-bn8nam12lp2176.outbound.protection.outlook.com [104.47.55.176])
 by userp3020.oracle.com with ESMTP id 3bc3cde2ch-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 00:59:46 +0000
Received: from CH0PR10MB5020.namprd10.prod.outlook.com (2603:10b6:610:c0::22)
 by CH0PR10MB5369.namprd10.prod.outlook.com (2603:10b6:610:dd::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 00:59:44 +0000
Received: from CH0PR10MB5020.namprd10.prod.outlook.com
 ([fe80::f44b:db92:7a0:782b]) by CH0PR10MB5020.namprd10.prod.outlook.com
 ([fe80::f44b:db92:7a0:782b%7]) with mapi id 15.20.4544.021; Wed, 29 Sep 2021
 00:59:44 +0000
Received: from [10.74.101.163] (160.34.89.163) by
 SJ0PR03CA0382.namprd03.prod.outlook.com (2603:10b6:a03:3a1::27) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 00:59:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 995f63eb-20c0-11ec-bcd3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=V/TLEl3TSdMAyJh6JfXUElRd52hZpPV6ykH3Gpm4PSs=;
 b=hVXvaBtCgcikPRqZata01px0F8UK+neel2ByL2FAl7uMtDCLvSYQ+cFQF3RQKqdD0ZZX
 7ijg5nbacJrgLM7nROys9NTrK/Q5FHnZdtSiWN1xQzLE940J66uuoOG4RRK7M6U4tJdo
 I2u6jq8gh2k5e0sT0C5m46+Rk4yx05TsGhW4sTmv+5ej2BF8EO7H/gIMd77jnx4P1LAu
 zqCB42TxiM9HZnJ6K1cPmo13F1eMF6juj8pAmeNfu5X+uEFZnNl4CJ0kSLX0cZPevmVe
 Zc7a6OmAxv2Fi3mODU1YyTPBBTh67/uoio83Ny5ie340gRnQ5Ir7kmx+OzBZkspK1eNy TQ== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bRYPEpugJ9/kn192HWLjN0kgdKt0uh/28VBdW95/iRGcgCcBx4a1qaRZ+p7FUz6RqxbIzFstU8F5jAC1izzXmjkvm/JQkEIhvq+l8h/L3mqMfDtSW0FX2tRcRPWSaz8qH3O8TCm5MQguu3aVcKwjBWN/NYjkeHHTVRZZgYwWmzPj84fRI8IMvUUPCh7mnRdLPTaR02LZ2FL/TKFAY4Q3bO7mT1BM21kvf7EYdl1v3Jtsi/CIZEShKzpvfr6JjpOB+k+VrMfkzSB5qGWszq8/9Ln359tTzKyPCJqimJ+LeTKO4c20eWbesjM6klB68aDnNdpLXrCdqd/TpbEruUHydQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V/TLEl3TSdMAyJh6JfXUElRd52hZpPV6ykH3Gpm4PSs=;
 b=d8L81g3gafUvnJP8AkTeD0CHfm0Ftpk75HzwWLY2F1l0VhEWwljn55V7j0zKf3BPFMFWT++NVA3SyR6slWElV8/Gd5rL14ZehChFtAIIuo7avqY7Eq0iHNn+QhNO3Hd1R9akHL6H7fE83muDm3AUAdpErFETG9IBPAahDCvuYIgCEoC5vUy+VlWVRS9D9KW6T5yGgZ0MNNscsW2r42hT1z28QKIHPyqT55ucbsoXDiUc6mXYl4O8gWZAsoZvhzLlAtMEOxwoCLGtTjHHf2cU6/zoqQaNq6S+Yz9p2BVs/phq0ktTsNvcnHEDTUp7BEuI5Ixeh/6ZwtaQEB/o1iy/eA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=V/TLEl3TSdMAyJh6JfXUElRd52hZpPV6ykH3Gpm4PSs=;
 b=LFX7LrDEUGDw60gGrAfcpceYQzyKT2dymiTJdlpyzx38aasi6CKRGH/hoH71/S7shvzA6UTY1KYmmN2LoBRjaWmV4xPBq39qlnQgtoBv+Sg3UjLcIfmhNewP68tGx07i540Ad+NYzq/YhONIdQgVI+LCqRUr9tvwU4QD/j81Ctg=
Authentication-Results: kvack.org; dkim=none (message not signed)
 header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
        Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
 <sstabellini@kernel.org>,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
        Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
        Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
        Mike Rapoport <rppt@kernel.org>,
        "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
        xen-devel@lists.xenproject.org,
        virtualization@lists.linux-foundation.org, kexec@lists.infradead.org,
        linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-3-david@redhat.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
Date: Tue, 28 Sep 2021 20:59:36 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210928182258.12451-3-david@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-ClientProxiedBy: SJ0PR03CA0382.namprd03.prod.outlook.com
 (2603:10b6:a03:3a1::27) To CH0PR10MB5020.namprd10.prod.outlook.com
 (2603:10b6:610:c0::22)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b2ecdb2c-8bf6-4a70-f1aa-08d982e46cc3
X-MS-TrafficTypeDiagnostic: CH0PR10MB5369:
X-Microsoft-Antispam-PRVS: 
	<CH0PR10MB53693BAD65805A7C130FCE4D8AA99@CH0PR10MB5369.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	/GLtQduKEu0nuKS491KxNqoTHQpCM5FfFHZrVUnc1wlIbJWQBnbvxLZPOgmiM1aKQqV2tYfWDD4K8EkovZ3OJe1KV+mp6DciQ/j5YMGYScE/JXE5wgnEzJuF1bTOwQRdNpdY1pBg0I49F1UNXERxLa52fdOjZlp5tdHRacWQt24TUuGLJuuCjuOGCA4lGPpzpCA+P/tzTyEFDV50rH6iWAtXI51PXzA+jz6AUTcCugpVbuSrEOFWibhPONYZzXahetG0yCyHN1f5I7Im12udigrkz3MIO+Y9hHK+6pDdAmEm7ppVQyno/DVQxmk7MU0Nfa6WINFIsrktR3N2tlEIb9vp5JCK0xxo0zMo94aH6TDc1IJicyOEYvrhYpeCBk5I2nxvklGd71/r/jaN2dJwYkfQYadGTsG+PGngOcUvVKhHHV3PzgRAbZFa4qK/6eHSm/aTyaxhup3T36GHgfGUcBmO2RA7ZnS62NQUpjuEhhG+XmFY89biT2eaIUdte7Bf/YdaubCMeiHPWwVb1j5Q/rm25ICfa+wsSbIPVhiKvZ+KJ2oYrdogE3fsp/uf7PLFJcGa0n/hNFzCJ4BMWNXfOo1tOSj37EIE/e9FzWG87CZ7M0fyF0M6qvrwLiXCy924XGfk3ZqpXvcoYQkdR2npLk1ChxSeGIXkh7Pkv35201JcH6PX6hapKUDpu1pk/W/OHdIIiTBnxUORsJQJ1oZadC4IjXDVNs39eqooRNXsYrPfy8vdD6HcUZhqqzI390pr
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH0PR10MB5020.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8676002)(31686004)(86362001)(8936002)(44832011)(5660300002)(7416002)(38100700002)(31696002)(83380400001)(186003)(6486002)(508600001)(2616005)(36756003)(956004)(4326008)(2906002)(66556008)(66476007)(316002)(16576012)(66946007)(4744005)(6666004)(54906003)(26005)(53546011)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?M29aVzR2Q0txNnpqNmpzRXJxWm1xaTY0SnhOa3lZMDJzN0tKT2JJU3hxZzMy?=
 =?utf-8?B?dTFFZlFEMDRwaHkvWUU1amZNT2dvOHNVZEQ1SFprTm80c20yaHZ3NWFFMDB1?=
 =?utf-8?B?RDBxWUNNenBuVWpvOUg1WDM3L3p5Qkd3dDlWWnlScnl6UVNHME05T3RycUN0?=
 =?utf-8?B?bUc4ekdmN3lyTDh5S3FiZk1JSlpzQndSbmdMS1ltWTlacWs1dzlHd0ZXeFJr?=
 =?utf-8?B?SmEwWjdIVHdUd3dDU0NNeThUSVVpL2x4LzMxVjVvU2xMNDJ2ZkdVZEFFQUlN?=
 =?utf-8?B?SDBscWMvQ08rOURweWFDUllxWnRQUlg5UnFHYTZMM0ZabU1tQlZwY0ZXdjkv?=
 =?utf-8?B?a1JLcTB4SDVib010QlZWMndpRU93cXlFYXhsTU1MNHdielZKRkU5Mmp5VXBn?=
 =?utf-8?B?RW5uSVhORWZkMnNTaUlMMk5aUmZWQzNZUGJ1ZTd3VnZ3cmdVeEVwazNYbzFl?=
 =?utf-8?B?bVRVK2ZBa2pYd0tySjlQVGFWcTVUYTU1elFnMjZsNk05d3NoT1VwWmRZbGlv?=
 =?utf-8?B?V3ZiSnp5WjZzeU1pamVPUW1sRTJHOHlmSnFpNktmenNsRG5UNnNUblgwOVdk?=
 =?utf-8?B?T2NyMUlacEtTQzd5RzY3cDJUMGRiQndtVG5zMmJsOXQ4OVgrTzExTkVwbHZh?=
 =?utf-8?B?ZmdxMkovZG9FOEs5eGJhZnU0dk9ucFByVW9aOXBpaHAyaUN4eFQvNFFDdG9T?=
 =?utf-8?B?VDZObXorVUthV1lLMHZBTUpGaDdNNDNhSWYwQ2oyOVZNMitidEJQYUFrdTlP?=
 =?utf-8?B?eEd6OGRVTDNFeThsSGRLNFpNSHUxZWlBanpMN05yVWdMKzM5bmVBVFpTWm9F?=
 =?utf-8?B?bEV2UkZzb0syU25xdzlQeWVremJSVW1tRDloOWRycE1ESlVxNTQrd015enJu?=
 =?utf-8?B?ODRCVjN0aFZWOTZ4NVhFRGdJMlFhVm9GQ2c4c1ZtdEJ2V0NJNEpPVTdFWXJ5?=
 =?utf-8?B?cHJVYXJHa1Y0ZFplSkxMbkJ1bGc3QnVHRzRYTE84TlhNSEdDdStWV0JJTnp3?=
 =?utf-8?B?M2xBMmxyS0Ewd2d6Z3drMldHMW91WStlY01RV0I5bnhvSVQzQU5nWjJYNFNB?=
 =?utf-8?B?UnBzYWlPVC9SZFFjTG9KUHNPVDlDeGlXQ1dPTGtrSHhJKy9YUTN4Y21sQTVh?=
 =?utf-8?B?d1BNKzV2b0xGS1lGWnM5bmxQNDZpRm9ZbTFMZ1A4WG12K1VTNnVNYW0wVjR4?=
 =?utf-8?B?THNBRWNnQ21PcVJXclM1ckVMM1VIVlR4UjJOT1JNdFduR2JnYWhMRnBhelJB?=
 =?utf-8?B?d3pqbmlocFF4RTJwY0E4YWxxdERRSytSWXlYdGhYOURoT21wRHdYOXBWc1Z1?=
 =?utf-8?B?WDk1ZXlBaEozYmFtUHB2RUgySTBSZTFxditoMncvbzAwd2NFUENSWUMrV241?=
 =?utf-8?B?cmFNWTU4MlgzQjdSK3ZsWUhEZUxoSW9SOSttaFYyNmNxSzVLYjFqdXNPQ3Qy?=
 =?utf-8?B?VE40VnVyVUM2SDBTb1pEM3pPV1VPamJRR09Cdml4NkRsODRHU0ROUnpHbVR0?=
 =?utf-8?B?TFR6SUxGREdMUlg0M2docGtocDdGZ09GS21aamxnQ3Y2U3BFOEY3YTRxZmJE?=
 =?utf-8?B?U296Q3VRSWFCVXBrcjRONm51ajFEbnNHQXhQb2FKV2Y1WWRiajc3REl2ZlVB?=
 =?utf-8?B?cXNwRUErTkZLb2MwZmNIa1JMRFRMR2FMK1VieUc1N25BM3p4emVtTXVteE5K?=
 =?utf-8?B?cWwwQmlmWmV4VW9salA2dHliQzJvQVlXT1ZUWm9tWTRuSVJETVJnV2xIaFpj?=
 =?utf-8?Q?qM/9ursR9C93lhAgnOXohypRvjed+/zsY142v8g?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2ecdb2c-8bf6-4a70-f1aa-08d982e46cc3
X-MS-Exchange-CrossTenant-AuthSource: CH0PR10MB5020.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 00:59:44.0970
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jPnPpC4o1PIuxrZIeWMkZEYrwo6PjuOGte00wgc+DM0DOjWAOj+8yJDzhAK6MOQHJlhUXNznmcuIQ9C927e/5D2C9Jv9a+BBOgJ3yhubdSQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB5369
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10121 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 mlxlogscore=999
 phishscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109290002
X-Proofpoint-GUID: pElT9iCsyLFVrZ-JexoHkUs6hWbbmLap
X-Proofpoint-ORIG-GUID: pElT9iCsyLFVrZ-JexoHkUs6hWbbmLap


On 9/28/21 2:22 PM, David Hildenbrand wrote:
> Let's simplify return handling.
>
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  arch/x86/xen/mmu_hvm.c | 11 ++---------
>  1 file changed, 2 insertions(+), 9 deletions(-)
>
> diff --git a/arch/x86/xen/mmu_hvm.c b/arch/x86/xen/mmu_hvm.c
> index b242d1f4b426..eb61622df75b 100644
> --- a/arch/x86/xen/mmu_hvm.c
> +++ b/arch/x86/xen/mmu_hvm.c
> @@ -21,23 +21,16 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
>  		.domid = DOMID_SELF,
>  		.pfn = pfn,
>  	};
> -	int ram;
>  
>  	if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
>  		return -ENXIO;
>  
>  	switch (a.mem_type) {
>  	case HVMMEM_mmio_dm:
> -		ram = 0;
> -		break;
> -	case HVMMEM_ram_rw:
> -	case HVMMEM_ram_ro:
> +		return 0;
>  	default:
> -		ram = 1;
> -		break;
> +		return 1;
>  	}
> -
> -	return ram;
>  }
>  #endif
>  


How about

    return a.mem_type != HVMMEM_mmio_dm;


Result should be promoted to int and this has added benefit of not requiring changes in patch 4.


-boris



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 01:17:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 01:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198561.352101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVODT-0001Dj-BB; Wed, 29 Sep 2021 01:16:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198561.352101; Wed, 29 Sep 2021 01:16:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVODT-0001Dc-7v; Wed, 29 Sep 2021 01:16:59 +0000
Received: by outflank-mailman (input) for mailman id 198561;
 Wed, 29 Sep 2021 01:16:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YPmi=OT=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mVODR-0001DW-MR
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 01:16:57 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f0375cf0-20c2-11ec-bcd3-12813bfff9fa;
 Wed, 29 Sep 2021 01:16:56 +0000 (UTC)
Received: from pps.filterd (m0246631.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18SNwmnt008676; 
 Wed, 29 Sep 2021 01:16:33 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3bbh6nutf0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 01:16:33 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18T1APhx096729;
 Wed, 29 Sep 2021 01:16:32 GMT
Received: from nam10-bn7-obe.outbound.protection.outlook.com
 (mail-bn7nam10lp2104.outbound.protection.outlook.com [104.47.70.104])
 by userp3030.oracle.com with ESMTP id 3bc3bj69f0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 01:16:31 +0000
Received: from CH0PR10MB5020.namprd10.prod.outlook.com (2603:10b6:610:c0::22)
 by CH0PR10MB5050.namprd10.prod.outlook.com (2603:10b6:610:c3::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 01:16:29 +0000
Received: from CH0PR10MB5020.namprd10.prod.outlook.com
 ([fe80::f44b:db92:7a0:782b]) by CH0PR10MB5020.namprd10.prod.outlook.com
 ([fe80::f44b:db92:7a0:782b%7]) with mapi id 15.20.4544.021; Wed, 29 Sep 2021
 01:16:29 +0000
Received: from [10.74.101.163] (160.34.89.163) by
 BY3PR03CA0012.namprd03.prod.outlook.com (2603:10b6:a03:39a::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 01:16:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0375cf0-20c2-11ec-bcd3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=29kxuht8FBylVu7vhTw9VNiUY7i0G7ocmqZ4RJmM6eM=;
 b=S5mM7ni+vydjM1PAgwgSB5ZZAQMuxsvs2j1wRGIAOwvN9jgBXNYnV32olK27P6Q7I3Lp
 NANOz66tM9WvJHQUqnjh47LxH+ziNzsocR0mTQgB73n4yE1ZFjySFVjJvboNhg2K/shZ
 wL5Mk5XK8rt9jO8BLXNf3Y7/5OLT/Wuak1vm1mifQ21QPfviSiEa/x6ur8+TwpoP7rCT
 tcu+iZj1cRA6SvO1IrHrU4t4K2VAM3LMyGUwawTqtboGbPeL4UjAWtrlTq2P7H3NuRaP
 jtoznPRwLfqdcjQCHXWA9hMhZUYfhLQMRF/adtHQAn5XXuoHMfTNaWk0WWUBMgCLGBAZ Xg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QxP5g7WRfn+x3DGjt5jJz1wfvQDGyxRlxAoXlhe1F4CpRph+eVJ+pcINkYPguHUcvS8P2gcGWwWyV30ImlAig7DnFdlYBqzbcU3Bx4ypLqV+RW8yi11TBpMdPj7CY7YiLqYoC+LwYXNWvrADU40gZl5RNNr12sLyhbsslgB8ZtVGTxxaRYl/I4l6P7Z0fjNX04hENQreKqbO4djBVFY2yTWxjLYT8kfkhldw25OCVVdYKoe0r2MBkIzMc5hvgTwBuYQ9lXXeiIr0SN5KiU3MtaydR9RHT4ACMbtGYSLvt8bhCEcZhfM/7lxUXIr7aJsiiJ0lg3HI/ZAHssgETUqMaw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=29kxuht8FBylVu7vhTw9VNiUY7i0G7ocmqZ4RJmM6eM=;
 b=JSHWHz+r+EV4ZU7ruWc4ZPcGs3F/b1CyaSaMflhS5qpNc9SmZ2bEB/GDmtTsGpnuEaQpzZARSGkCYDbdzVrkz9731Ji3bw8CDkNfecI2xFqtzOxol+wN//o/V/bzZ0Vgs3PREv0pRlEOD9SIVX28E/9xgjFiVAmSsWd0/tswiM4ZkVdsxNwY9C/kvLmy9LNq8PZupgYbOrUKXJjFFf0ThqGYo9mvS6jijuXkKborgiZouGaX+R0IcqUmFphgCAQKQKxx6mEuqXxpnUPytQNRAr5vzUMgYzB603cnrG+rGEwkKIt47rcQzrhejgAWkGOGtwPJZFzvkDMnIfioYb+1bw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=29kxuht8FBylVu7vhTw9VNiUY7i0G7ocmqZ4RJmM6eM=;
 b=yPcgJj8v/tWyp5G+0Dzl9NhxjnfVKdHY6Rb8fl7e9kyGERyo1nw2PO1eOLoLtQGKDZ+mS0WkDie6wyz9vo6kzyHBWiazwcRlKS2tgW5TAYVUODkd2UUjvVgPj//V69i0mkOP2hrlHLq1QGtxDEV/TmHVzKxVopan2GsdJ/m1IvU=
Authentication-Results: kvack.org; dkim=none (message not signed)
 header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v1 1/8] x86/xen: update xen_oldmem_pfn_is_ram()
 documentation
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
        Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
 <sstabellini@kernel.org>,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
        Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
        Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
        Mike Rapoport <rppt@kernel.org>,
        "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
        xen-devel@lists.xenproject.org,
        virtualization@lists.linux-foundation.org, kexec@lists.infradead.org,
        linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-2-david@redhat.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <69da3297-d4ab-54b2-3a43-d536e6dee888@oracle.com>
Date: Tue, 28 Sep 2021 21:16:21 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20210928182258.12451-2-david@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-ClientProxiedBy: BY3PR03CA0012.namprd03.prod.outlook.com
 (2603:10b6:a03:39a::17) To CH0PR10MB5020.namprd10.prod.outlook.com
 (2603:10b6:610:c0::22)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a616770d-cbad-4270-d866-08d982e6c3e4
X-MS-TrafficTypeDiagnostic: CH0PR10MB5050:
X-Microsoft-Antispam-PRVS: 
	<CH0PR10MB50503EF6FB495FB1375400738AA99@CH0PR10MB5050.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3383;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	16H6WlWiCX4aJmp+b7S6ufcVG9yGpz0+NqmjE3IISJ6YJ9aYwFWS4mSIZlenZ5+6dwLGjfoCmnD0a1pUdSPFraSkFTFhzkBove1xExAbDCE4DVtJQI0Xy8gw9Vx/IPiwQMrGjgwac7YBKNdIybqfc0kPxH0j2YVKk4P860oj18APLrGQt7ZA9RhwLUvbCl7If3Jt1wUXEDou2eYLlltsWY2WlOslw3SXItGWWZ26ct9Pei5/oPKmGpTSC8Ue0LDQmAxdpg8KWrC+xyvgFUpYKYfd7OQmgyzGUz4G8XBPhy9UpB95XnVvtedshuFsh5a9aBO9P33PIPjUKCe8UDXtPlsFy3Ivhosc/oNKbyDct0yBIhEFbZY0k2FoScM9+BrmecfnJzddc8xLM9LeTkbm/rc8dNGZAWTeUOmpWk+V9nKVz1OHLmPUyCQR6oCvWvCSd/qvbwopbNeqpyFu2Bdxl9yxY57+E+NIByFjEqhFW1n5/n9X7SMVrWNWurwqYWHx2zF0i5pNXoJMCmZWshel85WGmx+v0eYItuV1V4WaG6F/GJkTD97xljzFYdWiJW39nx5YtdL8oJtxdb8jsPRT35ewOuPPsp3NybBW5pLfrsdX/P9Iu75mlDH6GIOALoM9FTRw/9u7x0VRULOqDjg1f00Lr5vH21sVLje+g5FU3YHkDmBe+dH53fE79ghQT6MwbgJ66C/LofczwBL7up4PPb573NYWsZ1ZfjpsINZcyes=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH0PR10MB5020.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(5660300002)(36756003)(186003)(26005)(8676002)(6486002)(16576012)(86362001)(53546011)(54906003)(8936002)(316002)(66476007)(508600001)(31696002)(956004)(38100700002)(31686004)(44832011)(2616005)(558084003)(4326008)(83380400001)(66556008)(6666004)(66946007)(7416002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?ZFVPZHl0SUp6cWlFMWt5NENlZDY2UmZIODVFcFBobUwxa1RMRDZheTBKNWVR?=
 =?utf-8?B?cEhrMXY2MnFrZmw5Y3BPWU9wcTVpYlYrYnV1eHJVaHM5SzhVNE9IUjcxR3JD?=
 =?utf-8?B?TEVFMG9aL1UreWk0NUFubWQrWEVKZEo2V0Zvci9uSWZUSTl1ZHFha096blV4?=
 =?utf-8?B?aC9rTFVsajVwQ0M5MmxOQkluNG5SM1dRbEhYY0FpZnQ1L1FMbGNpTjZwMlIr?=
 =?utf-8?B?OEUzdmJHN1RJZHYxTUFRMnVPRExjdHJXTFA1dGM3S0V3UVhjR1dZdEVrSDBN?=
 =?utf-8?B?TWZRckxVNnJXSSs1NmZ4YmlVM0Z6RHlLZTVTTFdpaEVINE12R0creTRPWVJ0?=
 =?utf-8?B?OFJMdE5FR2lIb3RDTVNXWnMvNHdPVVVCcUxnendON1Noa2h3NC9JWGhrSU9t?=
 =?utf-8?B?eC94eHQ1dDJUV2h6WFkreGwzRWRYRkZrcDZ3MFBaOWRCWUlVNW4rNkt2WC9p?=
 =?utf-8?B?OEhQSWhkeUpMeURxcCtpY1c5WVZ0TTBjdElhbmY4U0NPS1V0UGVpY0NsNEtR?=
 =?utf-8?B?cHhQUUROckEvVVVzK3VhL3VtZ092ZCs1VWJGOXk4OVBMUGR3c0h3QkhTaUFC?=
 =?utf-8?B?Y0tJdUdkRVFsMksrd05XdVdhSHpVcE9sSW5RQS9nWjZzS3lsSG9KaXlkVWJa?=
 =?utf-8?B?aTVoTXpveExUMUJKTFpmUXYwTEhucExOY1NlSm1ZV1MzMmNUa0wxYVpsaHZD?=
 =?utf-8?B?ZEx4VW1LMnphVUZLd2lGdVZkTlRoVmI2WHNqbkFwOVVneVA0TTZSREs4L2Rn?=
 =?utf-8?B?WWZCN2lzdUZjVytaVVA4NHB2RTBvbktZSkdJc0ZYU01YbWd6L0pVYWRMVS9G?=
 =?utf-8?B?OGoyNGhKMGpQQ3EvaWxkUzVuVlpxV1YwOThwakx6YWZhdGZkVG92K3dHdkxv?=
 =?utf-8?B?dW4yeVY2OTRZN1JXTEdvTHM4VWhGSmpVRUZjOUxiNWRvNndaV2VsSHllSFZN?=
 =?utf-8?B?YWpJbTN3emNzdTlZWU83WUNlYVdxbXVvMlRXYnVpR0FBbXpnY2paY2ZWR0w0?=
 =?utf-8?B?Z09CWS8rZm15emJUQzB0bkFBK0xma0pZbXpmRm1Ldmt3d0srcnFLQzVBSmZP?=
 =?utf-8?B?dnpVYnBmV05SemljeGhCVGg2VXo1V3pWcUZZQWRhQ3dJdEVlOGtmUUpwaWZJ?=
 =?utf-8?B?Slk2MmpvTlRtOUZjek5yWWJScy9TbjJxUnpJSTdXb0tpSmZ1cE1TTFpkNnAz?=
 =?utf-8?B?cUV3WFlaNVhnbW9KYWhucXpqQjFKdjJyZWk3SFgyVHlRK2orOVJVKzVEeDZF?=
 =?utf-8?B?MkVUeTVRUFFxNkNDS0hCdzNOMWJkVUxsbG9nZThEQnIyUVliSWk1THl6b3Nr?=
 =?utf-8?B?L2NwVFpMRFhTR1F2VnVyOFVJTGM4Qktoa2krSXM0Z2U0dFZUeFpjWTU5WTBs?=
 =?utf-8?B?TUhlR2ZpM0hHeE8yMHMzU2MwNkMrYjBVN0ZNTU9OMk82UDhETzdNcGhTaVU5?=
 =?utf-8?B?Uml5ckhvWThTNUhBNTNqN3E5SlArdERWb0R3dUVFS0NFNzFiMEtvc2o5Kys5?=
 =?utf-8?B?L2hDemw3Zmt0NUlybDdyZEN1YktDZHV0U09pelVsZnNEb1lmbzZxRDRMR2Ju?=
 =?utf-8?B?cDEyNG42U2VWM2NDblJDK3RiQ2VaNVMwUkpOeEhleFpzVTQ5amtjRHlXQzNw?=
 =?utf-8?B?UXZ3UFZOUWtrWVlMUE1ubjVYTmRjRTltOGZtK2hPdmkzUlNmY2FNWG9RRG02?=
 =?utf-8?B?eG1SM2p0Q2ZCTnhrL2dIb1FXSlNNSlpMbnZ3MlAzaXo3U095K0lvODRRVVlE?=
 =?utf-8?Q?lJIzvyWsXoohpGXi/HRQWiAVc8VLhlwRN+ObALT?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a616770d-cbad-4270-d866-08d982e6c3e4
X-MS-Exchange-CrossTenant-AuthSource: CH0PR10MB5020.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 01:16:29.2698
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jhJNUDbPq9E7buvUhHDySSAnSBj4fRjvG9y5fnk+MaoBLDtX27618HkrnWHdJK3xpzGy+yEmdFGArONk3O23keonihVfIN3TJZZKU5DmGr4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB5050
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10121 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0
 mlxscore=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290004
X-Proofpoint-ORIG-GUID: 5V-l9q6ebGhxpzbciIoppgWmMIrUUE-Z
X-Proofpoint-GUID: 5V-l9q6ebGhxpzbciIoppgWmMIrUUE-Z


On 9/28/21 2:22 PM, David Hildenbrand wrote:
> The callback is only used for the vmcore nowadays.
>
> Signed-off-by: David Hildenbrand <david@redhat.com>


Reviewed-by: Boris Ostrovsky <boris.ostrvsky@oracle.com>




From xen-devel-bounces@lists.xenproject.org Wed Sep 29 03:24:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 03:24:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198570.352112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVQCO-0006kJ-IR; Wed, 29 Sep 2021 03:24:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198570.352112; Wed, 29 Sep 2021 03:24:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVQCO-0006kC-FF; Wed, 29 Sep 2021 03:24:00 +0000
Received: by outflank-mailman (input) for mailman id 198570;
 Wed, 29 Sep 2021 03:23:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVQCN-0006k2-TN; Wed, 29 Sep 2021 03:23:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVQCN-0005rh-AZ; Wed, 29 Sep 2021 03:23:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVQCN-00043R-1l; Wed, 29 Sep 2021 03:23:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVQCN-0002yr-1I; Wed, 29 Sep 2021 03:23:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hHjy9WVntmWy4XX4GdkY4cv9CIAQVZ5DymJc9b3wx9k=; b=P8gjU7iwxsUvEPxKRcncJL5NC5
	oMnFLmDQgwcn1KNCbasj4SnF+XcZO0i8HgXG4mc6r5JD/o/Fi/7dmLZI3ElOKd937Egg8c9eVTy4e
	dlbi8zJKfWRWufZy4EDtgR6V75NpFIxLUxJFcapxjiC782rsZFyy3VbFg5Xg2XAnPklE=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165236-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165236: regressions - trouble: blocked/broken/fail/pass
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-qemuu-rhel6hvm-intel:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-pvshim:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-ovmf-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-shadow:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-vhd:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-xsm:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-credit1:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-credit2:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-seattle:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-thunderx:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-vhd:<job status>:broken:regression
    xen-unstable:test-arm64-arm64-xl-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-shadow:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-rtds:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qcow2:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-pvshim:<job status>:broken:regression
    xen-unstable:build-armhf:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-pvhv2-intel:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-pvhv2-amd:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-credit2:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-dom0pvh-xl-intel:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-credit1:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-libvirt:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-qemuu-nested-intel:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-libvirt-vhd:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-libvirt-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-migrupgrade:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-pair:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-pygrub:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:<job status>:broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-coresched-amd64-xl:<job status>:broken:regression
    xen-unstable:test-amd64-coresched-i386-xl:<job status>:broken:regression
    xen-unstable:test-amd64-i386-freebsd10-amd64:<job status>:broken:regression
    xen-unstable:test-amd64-i386-freebsd10-i386:<job status>:broken:regression
    xen-unstable:test-amd64-i386-libvirt:<job status>:broken:regression
    xen-unstable:test-amd64-i386-libvirt-pair:<job status>:broken:regression
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-i386-libvirt-raw:<job status>:broken:regression
    xen-unstable:test-amd64-i386-libvirt-xsm:<job status>:broken:regression
    xen-unstable:test-amd64-i386-livepatch:<job status>:broken:regression
    xen-unstable:test-amd64-i386-migrupgrade:<job status>:broken:regression
    xen-unstable:test-amd64-i386-pair:<job status>:broken:regression
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-amd:<job status>:broken:regression
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-intel:<job status>:broken:regression
    xen-unstable:test-amd64-i386-qemuu-rhel6hvm-amd:<job status>:broken:regression
    xen-unstable:test-xtf-amd64-amd64-2:<job status>:broken:regression
    xen-unstable:test-xtf-amd64-amd64-4:<job status>:broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-pvhv2-amd:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-amd:host-install(5):broken:regression
    xen-unstable:test-amd64-coresched-i386-xl:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-libvirt:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-migrupgrade:host-install/src_host(6):broken:regression
    xen-unstable:test-amd64-amd64-libvirt-vhd:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-libvirt-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-dom0pvh-xl-intel:host-install(5):broken:regression
    xen-unstable:test-xtf-amd64-amd64-4:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-qemuu-nested-intel:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qcow2:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-credit2:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-intel:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-ovmf-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-freebsd10-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-pygrub:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-credit1:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-pvshim:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-pvhv2-intel:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-shadow:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-freebsd10-i386:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-libvirt:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-libvirt-raw:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-pvshim:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-pair:host-install/src_host(6):broken:regression
    xen-unstable:test-amd64-i386-xl-shadow:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-xl-vhd:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-libvirt-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-qemuu-rhel6hvm-amd:host-install(5):broken:regression
    xen-unstable:test-arm64-arm64-xl-seattle:host-install(5):broken:regression
    xen-unstable:test-arm64-arm64-xl-vhd:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:host-install(5):broken:regression
    xen-unstable:test-arm64-arm64-xl-credit2:capture-logs(22):broken:regression
    xen-unstable:test-amd64-coresched-amd64-xl:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-pair:host-install/dst_host(7):broken:regression
    xen-unstable:test-arm64-arm64-xl-xsm:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-migrupgrade:host-install/src_host(6):broken:regression
    xen-unstable:test-amd64-i386-migrupgrade:host-install/dst_host(7):broken:regression
    xen-unstable:test-amd64-amd64-migrupgrade:host-install/dst_host(7):broken:regression
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-amd64-pair:host-install/src_host(6):broken:regression
    xen-unstable:test-amd64-amd64-pair:host-install/dst_host(7):broken:regression
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-qemuu-rhel6hvm-intel:host-install(5):broken:regression
    xen-unstable:test-xtf-amd64-amd64-2:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-livepatch:host-install(5):broken:regression
    xen-unstable:test-amd64-i386-libvirt-pair:host-install/src_host(6):broken:regression
    xen-unstable:test-amd64-i386-libvirt-pair:host-install/dst_host(7):broken:regression
    xen-unstable:test-arm64-arm64-xl-thunderx:host-install(5):broken:regression
    xen-unstable:build-armhf:host-install(4):broken:regression
    xen-unstable:test-xtf-amd64-amd64-5:xen-install:fail:regression
    xen-unstable:test-amd64-amd64-qemuu-freebsd12-amd64:freebsd-install:fail:regression
    xen-unstable:test-amd64-i386-examine:examine-iommu:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64:debian-hvm-install:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:debian-hvm-install:fail:regression
    xen-unstable:test-arm64-arm64-xl-credit1:guest-start/debian.repeat:fail:regression
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    xen-unstable:test-amd64-amd64-xl-rtds:host-install(5):broken:allowable
    xen-unstable:test-armhf-armhf-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:build-armhf-libvirt:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-libvirt:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-coresched-i386-xl:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-freebsd10-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-amd:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-dom0pvh-xl-intel:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ovmf-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qcow2:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-pygrub:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-pvhv2-intel:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-intel:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-rtds:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-freebsd10-i386:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-freebsd11-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-intel:capture-logs(6):broken:nonblocking
    xen-unstable:test-xtf-amd64-amd64-4:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-shadow:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-credit2:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-credit1:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-coresched-amd64-xl:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-shadow:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-pvshim:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-pvhv2-amd:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-qemuu-rhel6hvm-amd:capture-logs(6):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:capture-logs(6):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:capture-logs(13):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:capture-logs(19):broken:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-pair:capture-logs/src_host(8):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-amd64-migrupgrade:capture-logs/src_host(8):broken:nonblocking
    xen-unstable:test-amd64-i386-migrupgrade:capture-logs/dst_host(9):broken:nonblocking
    xen-unstable:test-amd64-amd64-pair:capture-logs/dst_host(9):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:capture-logs(6):broken:nonblocking
    xen-unstable:test-amd64-i386-xl-vhd:capture-logs(6):broken:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1c3ed9c908732d19660fbe83580674d585464d4c
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 Sep 2021 03:23:59 +0000

flight 165236 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165236/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-qemuu-rhel6hvm-intel    <job status>                 broken
 test-amd64-i386-xl              <job status>                 broken
 test-amd64-i386-xl-pvshim       <job status>                 broken
 test-amd64-i386-xl-qemut-debianhvm-amd64    <job status>                broken
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm    <job status>             broken
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm    <job status>    broken
 test-amd64-i386-xl-qemut-win7-amd64    <job status>                 broken
 test-amd64-i386-xl-qemut-ws16-amd64    <job status>                 broken
 test-amd64-i386-xl-qemuu-debianhvm-amd64    <job status>                broken
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow    <job status>         broken
 test-amd64-i386-xl-qemuu-ovmf-amd64    <job status>                 broken
 test-amd64-i386-xl-qemuu-win7-amd64    <job status>                 broken
 test-amd64-i386-xl-qemuu-ws16-amd64    <job status>                 broken
 test-amd64-i386-xl-shadow       <job status>                 broken
 test-amd64-i386-xl-vhd          <job status>                 broken
 test-amd64-i386-xl-xsm          <job status>                 broken
 test-arm64-arm64-xl-credit1     <job status>                 broken
 test-arm64-arm64-xl-credit2     <job status>                 broken
 test-arm64-arm64-xl-seattle     <job status>                 broken
 test-arm64-arm64-xl-thunderx    <job status>                 broken
 test-arm64-arm64-xl-vhd         <job status>                 broken
 test-arm64-arm64-xl-xsm         <job status>                 broken
 test-amd64-amd64-xl-xsm         <job status>                 broken
 test-amd64-amd64-xl-shadow      <job status>                 broken
 test-amd64-amd64-xl-rtds        <job status>                 broken
 test-amd64-amd64-xl-qemuu-ws16-amd64    <job status>                 broken
 test-amd64-amd64-xl-qemuu-win7-amd64    <job status>                 broken
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict    <job status>   broken
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm    <job status>            broken
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow    <job status>        broken
 test-amd64-amd64-xl-qemut-ws16-amd64    <job status>                 broken
 test-amd64-amd64-xl-qemut-win7-amd64    <job status>                 broken
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm    <job status>   broken
 test-amd64-amd64-xl-qemut-debianhvm-amd64    <job status>               broken
 test-amd64-amd64-xl-qcow2       <job status>                 broken
 test-amd64-amd64-xl-pvshim      <job status>                 broken
 build-armhf                     <job status>                 broken
 test-amd64-amd64-xl-pvhv2-intel    <job status>                 broken
 test-amd64-amd64-xl-pvhv2-amd    <job status>                 broken
 test-amd64-amd64-xl-credit2     <job status>                 broken
 test-amd64-amd64-dom0pvh-xl-amd    <job status>                 broken
 test-amd64-amd64-dom0pvh-xl-intel    <job status>                 broken
 test-amd64-amd64-xl-credit1     <job status>                 broken
 test-amd64-amd64-libvirt        <job status>                 broken
 test-amd64-amd64-xl             <job status>                 broken
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm    <job status>      broken
 test-amd64-amd64-qemuu-nested-intel    <job status>                 broken
 test-amd64-amd64-libvirt-vhd    <job status>                 broken
 test-amd64-amd64-libvirt-xsm    <job status>                 broken
 test-amd64-amd64-qemuu-nested-amd    <job status>                 broken
 test-amd64-amd64-migrupgrade    <job status>                 broken
 test-amd64-amd64-pair           <job status>                 broken
 test-amd64-amd64-pygrub         <job status>                 broken
 test-amd64-amd64-qemuu-freebsd11-amd64    <job status>                 broken
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict    <job status>    broken
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm    <job status>             broken
 test-amd64-coresched-amd64-xl    <job status>                 broken
 test-amd64-coresched-i386-xl    <job status>                 broken
 test-amd64-i386-freebsd10-amd64    <job status>                 broken
 test-amd64-i386-freebsd10-i386    <job status>                 broken
 test-amd64-i386-libvirt         <job status>                 broken
 test-amd64-i386-libvirt-pair    <job status>                 broken
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm    <job status>       broken
 test-amd64-i386-libvirt-raw     <job status>                 broken
 test-amd64-i386-libvirt-xsm     <job status>                 broken
 test-amd64-i386-livepatch       <job status>                 broken
 test-amd64-i386-migrupgrade     <job status>                 broken
 test-amd64-i386-pair            <job status>                 broken
 test-amd64-i386-qemut-rhel6hvm-amd    <job status>                 broken
 test-amd64-i386-qemut-rhel6hvm-intel    <job status>                 broken
 test-amd64-i386-qemuu-rhel6hvm-amd    <job status>                 broken
 test-xtf-amd64-amd64-2          <job status>                 broken
 test-xtf-amd64-amd64-4          <job status>                 broken
 test-amd64-amd64-xl-qemuu-ws16-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemut-ws16-amd64  5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl-pvhv2-amd  5 host-install(5)       broken REGR. vs. 164945
 test-amd64-i386-qemut-rhel6hvm-amd  5 host-install(5)  broken REGR. vs. 164945
 test-amd64-coresched-i386-xl  5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-libvirt      5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-migrupgrade 6 host-install/src_host(6) broken REGR. vs. 164945
 test-amd64-amd64-libvirt-vhd  5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-libvirt-xsm  5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-dom0pvh-xl-intel  5 host-install(5)   broken REGR. vs. 164945
 test-xtf-amd64-amd64-4        5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-qemuu-nested-intel  5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl           5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-qcow2     5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-credit2   5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-xsm        5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl-qemut-debianhvm-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-qemut-rhel6hvm-intel 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-ovmf-amd64  5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-freebsd10-amd64  5 host-install(5)     broken REGR. vs. 164945
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-ws16-amd64  5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-pygrub       5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-qemuu-freebsd11-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl-qemut-debianhvm-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl-credit1   5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-xsm       5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-pvshim    5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-xl-pvhv2-intel  5 host-install(5)     broken REGR. vs. 164945
 test-amd64-amd64-xl-shadow    5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-freebsd10-i386  5 host-install(5)      broken REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-libvirt       5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl            5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-debianhvm-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-libvirt-raw   5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-pvshim     5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-pair        6 host-install/src_host(6) broken REGR. vs. 164945
 test-amd64-i386-xl-shadow     5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-xl-qemuu-win7-amd64  5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-xl-vhd        5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-libvirt-xsm   5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-dom0pvh-xl-amd  5 host-install(5)     broken REGR. vs. 164945
 test-amd64-i386-qemuu-rhel6hvm-amd  5 host-install(5)  broken REGR. vs. 164945
 test-arm64-arm64-xl-seattle   5 host-install(5)        broken REGR. vs. 164945
 test-arm64-arm64-xl-vhd       5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-xl-qemut-win7-amd64 5 host-install(5) broken REGR. vs. 164945
 test-arm64-arm64-xl-credit2  22 capture-logs(22)       broken REGR. vs. 164945
 test-amd64-coresched-amd64-xl  5 host-install(5)       broken REGR. vs. 164945
 test-amd64-i386-pair        7 host-install/dst_host(7) broken REGR. vs. 164945
 test-arm64-arm64-xl-xsm       5 host-install(5)        broken REGR. vs. 164945
 test-amd64-amd64-qemuu-nested-amd  5 host-install(5)   broken REGR. vs. 164945
 test-amd64-i386-migrupgrade 6 host-install/src_host(6) broken REGR. vs. 164945
 test-amd64-i386-migrupgrade 7 host-install/dst_host(7) broken REGR. vs. 164945
 test-amd64-amd64-migrupgrade 7 host-install/dst_host(7) broken REGR. vs. 164945
 test-amd64-i386-xl-qemut-win7-amd64  5 host-install(5) broken REGR. vs. 164945
 test-amd64-amd64-pair       6 host-install/src_host(6) broken REGR. vs. 164945
 test-amd64-amd64-pair       7 host-install/dst_host(7) broken REGR. vs. 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 5 host-install(5) broken REGR. vs. 164945
 test-amd64-i386-qemuu-rhel6hvm-intel 5 host-install(5) broken REGR. vs. 164945
 test-xtf-amd64-amd64-2        5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-livepatch     5 host-install(5)        broken REGR. vs. 164945
 test-amd64-i386-libvirt-pair 6 host-install/src_host(6) broken REGR. vs. 164945
 test-amd64-i386-libvirt-pair 7 host-install/dst_host(7) broken REGR. vs. 164945
 test-arm64-arm64-xl-thunderx  5 host-install(5)        broken REGR. vs. 164945
 build-armhf                   4 host-install(4)        broken REGR. vs. 164945
 test-xtf-amd64-amd64-5        7 xen-install              fail REGR. vs. 164945
 test-amd64-amd64-qemuu-freebsd12-amd64 12 freebsd-install fail REGR. vs. 164945
 test-amd64-i386-examine      13 examine-iommu            fail REGR. vs. 164945
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 12 debian-hvm-install fail REGR. vs. 164945
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 12 debian-hvm-install fail REGR. vs. 164945
 test-arm64-arm64-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 164945
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 164945

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds      5 host-install(5)        broken REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-examine      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 build-armhf-libvirt           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-rtds      1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl-vhd       1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-libvirt       6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-xsm       6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-dom0pvh-xl-amd  6 capture-logs(6)    broken blocked in 164945
 test-amd64-amd64-libvirt-xsm  6 capture-logs(6)       broken blocked in 164945
 test-amd64-coresched-i386-xl  6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-freebsd10-amd64  6 capture-logs(6)    broken blocked in 164945
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-libvirt-xsm   6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-qemut-rhel6hvm-amd  6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemuu-win7-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-xsm        6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-dom0pvh-xl-intel  6 capture-logs(6)  broken blocked in 164945
 test-amd64-amd64-libvirt-vhd  6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-xl-qemut-debianhvm-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemuu-ovmf-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-qcow2     6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-pygrub       6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-xl-pvshim     6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-pvhv2-intel  6 capture-logs(6)    broken blocked in 164945
 test-amd64-amd64-qemuu-nested-intel 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl-rtds      6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-qemut-debianhvm-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-freebsd10-i386  6 capture-logs(6)     broken blocked in 164945
 test-amd64-i386-xl            6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-xl-qemut-ws16-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-qemuu-freebsd11-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-qemut-rhel6hvm-intel 6 capture-logs(6) broken blocked in 164945
 test-xtf-amd64-amd64-4        6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-shadow    6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-credit2   6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-credit1   6 capture-logs(6)       broken blocked in 164945
 test-amd64-coresched-amd64-xl  6 capture-logs(6)      broken blocked in 164945
 test-amd64-i386-xl-qemuu-debianhvm-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-amd64-xl           6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-xl-shadow     6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-pvshim    6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-pvhv2-amd  6 capture-logs(6)      broken blocked in 164945
 test-amd64-i386-qemuu-rhel6hvm-amd  6 capture-logs(6) broken blocked in 164945
 test-arm64-arm64-xl-seattle   6 capture-logs(6)       broken blocked in 164945
 test-arm64-arm64-xl-vhd       6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 capture-logs(13) broken blocked in 164945
 test-arm64-arm64-xl-credit1  19 capture-logs(19)      broken blocked in 164945
 test-amd64-amd64-xl-qemut-win7-amd64 6 capture-logs(6) broken blocked in 164945
 test-amd64-i386-pair       8 capture-logs/src_host(8) broken blocked in 164945
 test-arm64-arm64-xl-xsm       6 capture-logs(6)       broken blocked in 164945
 test-amd64-amd64-migrupgrade 8 capture-logs/src_host(8) broken blocked in 164945
 test-amd64-i386-migrupgrade 9 capture-logs/dst_host(9) broken blocked in 164945
 test-amd64-amd64-pair      9 capture-logs/dst_host(9) broken blocked in 164945
 test-arm64-arm64-xl-thunderx  6 capture-logs(6)       broken blocked in 164945
 test-amd64-i386-libvirt-raw   6 capture-logs(6)              broken never pass
 test-amd64-i386-xl-vhd        6 capture-logs(6)              broken never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1c3ed9c908732d19660fbe83580674d585464d4c
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   18 days
Failing since        164951  2021-09-12 00:14:36 Z   17 days   34 attempts
Testing same since   165236  2021-09-28 16:08:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  broken  
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          blocked 
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       broken  
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       broken  
 test-xtf-amd64-amd64-5                                       fail    
 test-amd64-amd64-xl                                          broken  
 test-amd64-coresched-amd64-xl                                broken  
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          blocked 
 test-amd64-i386-xl                                           broken  
 test-amd64-coresched-i386-xl                                 broken  
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           broken  
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            broken  
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        broken  
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         broken  
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  broken  
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 broken  
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  broken  
 test-amd64-amd64-libvirt-xsm                                 broken  
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  broken  
 test-amd64-amd64-xl-xsm                                      broken  
 test-arm64-arm64-xl-xsm                                      broken  
 test-amd64-i386-xl-xsm                                       broken  
 test-amd64-amd64-qemuu-nested-amd                            broken  
 test-amd64-amd64-xl-pvhv2-amd                                broken  
 test-amd64-i386-qemut-rhel6hvm-amd                           broken  
 test-amd64-i386-qemuu-rhel6hvm-amd                           broken  
 test-amd64-amd64-dom0pvh-xl-amd                              broken  
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    broken  
 test-amd64-i386-xl-qemut-debianhvm-amd64                     broken  
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     broken  
 test-amd64-i386-freebsd10-amd64                              broken  
 test-amd64-amd64-qemuu-freebsd11-amd64                       broken  
 test-amd64-amd64-qemuu-freebsd12-amd64                       fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          broken  
 test-amd64-amd64-xl-qemut-win7-amd64                         broken  
 test-amd64-i386-xl-qemut-win7-amd64                          broken  
 test-amd64-amd64-xl-qemuu-win7-amd64                         broken  
 test-amd64-i386-xl-qemuu-win7-amd64                          broken  
 test-amd64-amd64-xl-qemut-ws16-amd64                         broken  
 test-amd64-i386-xl-qemut-ws16-amd64                          broken  
 test-amd64-amd64-xl-qemuu-ws16-amd64                         broken  
 test-amd64-i386-xl-qemuu-ws16-amd64                          broken  
 test-armhf-armhf-xl-arndale                                  blocked 
 test-amd64-amd64-xl-credit1                                  broken  
 test-arm64-arm64-xl-credit1                                  broken  
 test-armhf-armhf-xl-credit1                                  blocked 
 test-amd64-amd64-xl-credit2                                  broken  
 test-arm64-arm64-xl-credit2                                  broken  
 test-armhf-armhf-xl-credit2                                  blocked 
 test-armhf-armhf-xl-cubietruck                               blocked 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        broken  
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         broken  
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     blocked 
 test-amd64-i386-examine                                      fail    
 test-amd64-i386-freebsd10-i386                               broken  
 test-amd64-amd64-qemuu-nested-intel                          broken  
 test-amd64-amd64-xl-pvhv2-intel                              broken  
 test-amd64-i386-qemut-rhel6hvm-intel                         broken  
 test-amd64-i386-qemuu-rhel6hvm-intel                         broken  
 test-amd64-amd64-dom0pvh-xl-intel                            broken  
 test-amd64-amd64-libvirt                                     broken  
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      broken  
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    broken  
 test-amd64-amd64-migrupgrade                                 broken  
 test-amd64-i386-migrupgrade                                  broken  
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                blocked 
 test-amd64-amd64-pair                                        broken  
 test-amd64-i386-pair                                         broken  
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 broken  
 test-amd64-amd64-xl-pvshim                                   broken  
 test-amd64-i386-xl-pvshim                                    broken  
 test-amd64-amd64-pygrub                                      broken  
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-amd64-amd64-xl-qcow2                                    broken  
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  broken  
 test-amd64-amd64-xl-rtds                                     broken  
 test-armhf-armhf-xl-rtds                                     blocked 
 test-arm64-arm64-xl-seattle                                  broken  
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             broken  
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              broken  
 test-amd64-amd64-xl-shadow                                   broken  
 test-amd64-i386-xl-shadow                                    broken  
 test-arm64-arm64-xl-thunderx                                 broken  
 test-amd64-amd64-libvirt-vhd                                 broken  
 test-arm64-arm64-xl-vhd                                      broken  
 test-armhf-armhf-xl-vhd                                      blocked 
 test-amd64-i386-xl-vhd                                       broken  


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-i386-qemuu-rhel6hvm-intel broken
broken-job test-amd64-i386-xl broken
broken-job test-amd64-i386-xl-pvshim broken
broken-job test-amd64-i386-xl-qemut-debianhvm-amd64 broken
broken-job test-amd64-i386-xl-qemut-debianhvm-i386-xsm broken
broken-job test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm broken
broken-job test-amd64-i386-xl-qemut-win7-amd64 broken
broken-job test-amd64-i386-xl-qemut-ws16-amd64 broken
broken-job test-amd64-i386-xl-qemuu-debianhvm-amd64 broken
broken-job test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow broken
broken-job test-amd64-i386-xl-qemuu-ovmf-amd64 broken
broken-job test-amd64-i386-xl-qemuu-win7-amd64 broken
broken-job test-amd64-i386-xl-qemuu-ws16-amd64 broken
broken-job test-amd64-i386-xl-shadow broken
broken-job test-amd64-i386-xl-vhd broken
broken-job test-amd64-i386-xl-xsm broken
broken-job test-arm64-arm64-xl-credit1 broken
broken-job test-arm64-arm64-xl-credit2 broken
broken-job test-arm64-arm64-xl-seattle broken
broken-job test-arm64-arm64-xl-thunderx broken
broken-job test-arm64-arm64-xl-vhd broken
broken-job test-arm64-arm64-xl-xsm broken
broken-job test-amd64-amd64-xl-xsm broken
broken-job test-amd64-amd64-xl-shadow broken
broken-job test-amd64-amd64-xl-rtds broken
broken-job test-amd64-amd64-xl-qemuu-ws16-amd64 broken
broken-job test-amd64-amd64-xl-qemuu-win7-amd64 broken
broken-job test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict broken
broken-job test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm broken
broken-job test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow broken
broken-job test-amd64-amd64-xl-qemut-ws16-amd64 broken
broken-job test-amd64-amd64-xl-qemut-win7-amd64 broken
broken-job test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm broken
broken-job test-amd64-amd64-xl-qemut-debianhvm-amd64 broken
broken-job test-amd64-amd64-xl-qcow2 broken
broken-job test-amd64-amd64-xl-pvshim broken
broken-job build-armhf broken
broken-job test-amd64-amd64-xl-pvhv2-intel broken
broken-job test-amd64-amd64-xl-pvhv2-amd broken
broken-job test-amd64-amd64-xl-credit2 broken
broken-job test-amd64-amd64-dom0pvh-xl-amd broken
broken-job test-amd64-amd64-dom0pvh-xl-intel broken
broken-job test-amd64-amd64-xl-credit1 broken
broken-job test-amd64-amd64-libvirt broken
broken-job test-amd64-amd64-xl broken
broken-job test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm broken
broken-job test-amd64-amd64-qemuu-nested-intel broken
broken-job test-amd64-amd64-libvirt-vhd broken
broken-job test-amd64-amd64-libvirt-xsm broken
broken-job test-amd64-amd64-qemuu-nested-amd broken
broken-job test-amd64-amd64-migrupgrade broken
broken-job test-amd64-amd64-pair broken
broken-job test-amd64-amd64-pygrub broken
broken-job test-amd64-amd64-qemuu-freebsd11-amd64 broken
broken-job test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict broken
broken-job test-amd64-i386-xl-qemuu-debianhvm-i386-xsm broken
broken-job test-amd64-coresched-amd64-xl broken
broken-job test-amd64-coresched-i386-xl broken
broken-job test-amd64-i386-freebsd10-amd64 broken
broken-job test-amd64-i386-freebsd10-i386 broken
broken-job test-amd64-i386-libvirt broken
broken-job test-amd64-i386-libvirt-pair broken
broken-job test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm broken
broken-job test-amd64-i386-libvirt-raw broken
broken-job test-amd64-i386-libvirt-xsm broken
broken-job test-amd64-i386-livepatch broken
broken-job test-amd64-i386-migrupgrade broken
broken-job test-amd64-i386-pair broken
broken-job test-amd64-i386-qemut-rhel6hvm-amd broken
broken-job test-amd64-i386-qemut-rhel6hvm-intel broken
broken-job test-amd64-i386-qemuu-rhel6hvm-amd broken
broken-job test-xtf-amd64-amd64-2 broken
broken-job test-xtf-amd64-amd64-4 broken
broken-step test-amd64-amd64-libvirt capture-logs(6)
broken-step test-amd64-i386-libvirt capture-logs(6)
broken-step test-amd64-i386-libvirt-raw capture-logs(6)
broken-step test-amd64-amd64-xl-xsm capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm capture-logs(6)
broken-step test-amd64-amd64-dom0pvh-xl-amd capture-logs(6)
broken-step test-amd64-amd64-libvirt-xsm capture-logs(6)
broken-step test-amd64-coresched-i386-xl capture-logs(6)
broken-step test-amd64-i386-freebsd10-amd64 capture-logs(6)
broken-step test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm capture-logs(6)
broken-step test-amd64-i386-libvirt-xsm capture-logs(6)
broken-step test-amd64-i386-qemut-rhel6hvm-amd capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-win7-amd64 capture-logs(6)
broken-step test-amd64-i386-xl-vhd capture-logs(6)
broken-step test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm capture-logs(6)
broken-step test-amd64-i386-xl-qemut-debianhvm-i386-xsm capture-logs(6)
broken-step test-amd64-i386-xl-xsm capture-logs(6)
broken-step test-amd64-amd64-dom0pvh-xl-intel capture-logs(6)
broken-step test-amd64-amd64-libvirt-vhd capture-logs(6)
broken-step test-amd64-i386-xl-qemut-debianhvm-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-ws16-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow capture-logs(6)
broken-step test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-ovmf-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-win7-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict capture-logs(6)
broken-step test-amd64-amd64-xl-qcow2 capture-logs(6)
broken-step test-amd64-amd64-pygrub capture-logs(6)
broken-step test-amd64-i386-xl-pvshim capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-ws16-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl-pvhv2-intel capture-logs(6)
broken-step test-amd64-amd64-qemuu-nested-intel capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-debianhvm-i386-xsm capture-logs(6)
broken-step test-amd64-amd64-xl-rtds capture-logs(6)
broken-step test-amd64-amd64-xl-qemut-debianhvm-amd64 capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict capture-logs(6)
broken-step test-amd64-i386-freebsd10-i386 capture-logs(6)
broken-step test-amd64-i386-xl capture-logs(6)
broken-step test-amd64-i386-xl-qemut-ws16-amd64 capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow capture-logs(6)
broken-step test-amd64-amd64-qemuu-freebsd11-amd64 capture-logs(6)
broken-step test-amd64-i386-qemut-rhel6hvm-intel capture-logs(6)
broken-step test-xtf-amd64-amd64-4 capture-logs(6)
broken-step test-amd64-amd64-xl-shadow capture-logs(6)
broken-step test-amd64-amd64-xl-credit2 capture-logs(6)
broken-step test-amd64-amd64-xl-credit1 capture-logs(6)
broken-step test-amd64-coresched-amd64-xl capture-logs(6)
broken-step test-amd64-i386-xl-qemuu-debianhvm-amd64 capture-logs(6)
broken-step test-amd64-amd64-xl capture-logs(6)
broken-step test-amd64-i386-xl-shadow capture-logs(6)
broken-step test-amd64-amd64-xl-pvshim capture-logs(6)
broken-step test-amd64-amd64-xl-pvhv2-amd capture-logs(6)
broken-step test-amd64-i386-qemuu-rhel6hvm-amd capture-logs(6)
broken-step test-arm64-arm64-xl-seattle capture-logs(6)
broken-step test-arm64-arm64-xl-vhd capture-logs(6)
broken-step test-amd64-amd64-xl-qemuu-ws16-amd64 host-install(5)
broken-step test-amd64-i386-xl-qemut-ws16-amd64 host-install(5)
broken-step test-amd64-amd64-xl-pvhv2-amd host-install(5)
broken-step test-amd64-i386-qemut-rhel6hvm-amd host-install(5)
broken-step test-amd64-coresched-i386-xl host-install(5)
broken-step test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow host-install(5)
broken-step test-amd64-i386-xl-qemut-debianhvm-i386-xsm host-install(5)
broken-step test-amd64-amd64-libvirt host-install(5)
broken-step test-amd64-amd64-migrupgrade host-install/src_host(6)
broken-step test-amd64-amd64-libvirt-vhd host-install(5)
broken-step test-amd64-amd64-libvirt-xsm host-install(5)
broken-step test-amd64-amd64-dom0pvh-xl-intel host-install(5)
broken-step test-xtf-amd64-amd64-4 host-install(5)
broken-step test-amd64-amd64-qemuu-nested-intel host-install(5)
broken-step test-amd64-amd64-xl host-install(5)
broken-step test-amd64-amd64-xl-qcow2 host-install(5)
broken-step test-amd64-amd64-xl-credit2 host-install(5)
broken-step test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm host-install(5)
broken-step test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow host-install(5)
broken-step test-amd64-amd64-xl-rtds host-install(5)
broken-step test-amd64-i386-xl-xsm host-install(5)
broken-step test-amd64-i386-xl-qemut-debianhvm-amd64 host-install(5)
broken-step test-amd64-i386-qemut-rhel6hvm-intel host-install(5)
broken-step test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm host-install(5)
broken-step test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm host-install(5)
broken-step test-amd64-i386-xl-qemuu-ovmf-amd64 host-install(5)
broken-step test-amd64-i386-freebsd10-amd64 host-install(5)
broken-step test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm host-install(5)
broken-step test-amd64-i386-xl-qemuu-ws16-amd64 host-install(5)
broken-step test-amd64-amd64-pygrub host-install(5)
broken-step test-amd64-amd64-qemuu-freebsd11-amd64 host-install(5)
broken-step test-amd64-amd64-xl-qemut-debianhvm-amd64 host-install(5)
broken-step test-amd64-amd64-xl-credit1 host-install(5)
broken-step test-amd64-amd64-xl-xsm host-install(5)
broken-step test-amd64-amd64-xl-pvshim host-install(5)
broken-step test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict host-install(5)
broken-step test-amd64-amd64-xl-pvhv2-intel host-install(5)
broken-step test-amd64-amd64-xl-shadow host-install(5)
broken-step test-amd64-i386-freebsd10-i386 host-install(5)
broken-step test-amd64-amd64-xl-qemuu-win7-amd64 host-install(5)
broken-step test-amd64-i386-libvirt host-install(5)
broken-step test-amd64-i386-xl host-install(5)
broken-step test-amd64-i386-xl-qemuu-debianhvm-i386-xsm host-install(5)
broken-step test-amd64-i386-xl-qemuu-debianhvm-amd64 host-install(5)
broken-step test-amd64-i386-libvirt-raw host-install(5)
broken-step test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict host-install(5)
broken-step test-amd64-i386-xl-pvshim host-install(5)
broken-step test-amd64-i386-pair host-install/src_host(6)
broken-step test-amd64-i386-xl-shadow host-install(5)
broken-step test-amd64-i386-xl-qemuu-win7-amd64 host-install(5)
broken-step test-amd64-i386-xl-vhd host-install(5)
broken-step test-amd64-i386-libvirt-xsm host-install(5)
broken-step test-amd64-amd64-dom0pvh-xl-amd host-install(5)
broken-step test-amd64-i386-qemuu-rhel6hvm-amd host-install(5)
broken-step test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm capture-logs(13)
broken-step test-arm64-arm64-xl-seattle host-install(5)
broken-step test-arm64-arm64-xl-vhd host-install(5)
broken-step test-arm64-arm64-xl-credit1 capture-logs(19)
broken-step test-amd64-amd64-xl-qemut-win7-amd64 host-install(5)
broken-step test-amd64-amd64-xl-qemut-win7-amd64 capture-logs(6)
broken-step test-arm64-arm64-xl-credit2 capture-logs(22)
broken-step test-amd64-coresched-amd64-xl host-install(5)
broken-step test-amd64-i386-pair host-install/dst_host(7)
broken-step test-amd64-i386-pair capture-logs/src_host(8)
broken-step test-arm64-arm64-xl-xsm host-install(5)
broken-step test-amd64-amd64-qemuu-nested-amd host-install(5)
broken-step test-amd64-i386-migrupgrade host-install/src_host(6)
broken-step test-amd64-i386-migrupgrade host-install/dst_host(7)
broken-step test-arm64-arm64-xl-xsm capture-logs(6)
broken-step test-amd64-amd64-migrupgrade host-install/dst_host(7)
broken-step test-amd64-i386-xl-qemut-win7-amd64 host-install(5)
broken-step test-amd64-amd64-pair host-install/src_host(6)
broken-step test-amd64-amd64-pair host-install/dst_host(7)
broken-step test-amd64-amd64-xl-qemut-ws16-amd64 host-install(5)
broken-step test-amd64-amd64-migrupgrade capture-logs/src_host(8)
broken-step test-amd64-i386-qemuu-rhel6hvm-intel host-install(5)
broken-step test-xtf-amd64-amd64-2 host-install(5)
broken-step test-amd64-i386-livepatch host-install(5)
broken-step test-amd64-i386-libvirt-pair host-install/src_host(6)
broken-step test-amd64-i386-libvirt-pair host-install/dst_host(7)
broken-step test-amd64-i386-migrupgrade capture-logs/dst_host(9)
broken-step test-amd64-amd64-pair capture-logs/dst_host(9)
broken-step test-arm64-arm64-xl-thunderx host-install(5)
broken-step test-arm64-arm64-xl-thunderx capture-logs(6)
broken-step build-armhf host-install(4)

Not pushing.

(No revision log; it would be 1013 lines long.)


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 04:36:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 04:36:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198578.352126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVRJx-00062e-3S; Wed, 29 Sep 2021 04:35:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198578.352126; Wed, 29 Sep 2021 04:35:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVRJx-00062X-0V; Wed, 29 Sep 2021 04:35:53 +0000
Received: by outflank-mailman (input) for mailman id 198578;
 Wed, 29 Sep 2021 04:35:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2G/I=OT=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mVRJu-00062P-B8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 04:35:51 +0000
Received: from forward104p.mail.yandex.net (unknown [77.88.28.107])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bc5b1271-e584-4c9d-b31e-9e2714efe1fb;
 Wed, 29 Sep 2021 04:35:46 +0000 (UTC)
Received: from postback11j.mail.yandex.net (postback11j.mail.yandex.net
 [IPv6:2a02:6b8:c04:242:0:640:f45d:5029])
 by forward104p.mail.yandex.net (Yandex) with ESMTP id 90AA13C1E6F6;
 Wed, 29 Sep 2021 07:35:44 +0300 (MSK)
Received: from forward500j.mail.yandex.net (forward500j.mail.yandex.net
 [IPv6:2a02:6b8:0:801:2::110])
 by postback11j.mail.yandex.net (Yandex) with ESMTP id 8363670790CB;
 Wed, 29 Sep 2021 07:35:44 +0300 (MSK)
Received: from myt5-d8fb82618a34.qloud-c.yandex.net
 (myt5-d8fb82618a34.qloud-c.yandex.net
 [IPv6:2a02:6b8:c12:59a3:0:640:d8fb:8261])
 by forward500j.mail.yandex.net (Yandex) with ESMTP id 805B06CB6402;
 Wed, 29 Sep 2021 07:35:44 +0300 (MSK)
Received: from 2a02:6b8:c12:2e:0:640:e9df:8ad7
 (2a02:6b8:c12:2e:0:640:e9df:8ad7 [2a02:6b8:c12:2e:0:640:e9df:8ad7])
 by myt5-d8fb82618a34.qloud-c.yandex.net (mxback/Yandex) with HTTP id
 gZc3NS0D3eA1-ZhDSTo54; Wed, 29 Sep 2021 07:35:43 +0300
Received: by myt3-e9df8ad73dde.qloud-c.yandex.net with HTTP;
 Wed, 29 Sep 2021 07:35:43 +0300
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc5b1271-e584-4c9d-b31e-9e2714efe1fb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1632890143;
	bh=H2Kd/bWdxrcpWzKwfKi9rgk2rakJspIEmpOdfcRI3/I=;
	h=References:Date:Message-Id:Cc:Subject:In-Reply-To:To:From;
	b=wLkFkdJcQ4x6zsZLKeqeH0sRGbKtzZdE+XQQKcy20XyX1AJGuqmjv8YqxhEd6MDnU
	 ueopnJONIksMuTDG7QJZZ+L27OS/048k7iZclvOFTKH04BmzwycTkS2bkgyadRBCuL
	 t/lowLaeQ6TDEN911g+gQDjcbPiy2a+04BdKWiu8=
Authentication-Results: myt5-d8fb82618a34.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>
In-Reply-To: <YVMMXarcqweUD4sK@perard>
References: <dd3a1e1a7a3f8e7bca18dd4779efbc2af01decc7.1631793876.git.isaikin-dmitry@yandex.ru> <YVMMXarcqweUD4sK@perard>
Subject: Re: [PATCH v3] tools/xl: fix autoballoon regex
MIME-Version: 1.0
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Wed, 29 Sep 2021 07:35:43 +0300
Message-Id: <1504001632889454@iva7-56e9317134d0.qloud-c.yandex.net>
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=utf-8

VGhhbmtzISBUaGF0J3MgYSBnb29kIGlkZWEuIEkgd2lsbCBkbyBpdCBpbiB2NCBwYXRjaC48ZGl2
PjxiciAvPjwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxNnB4Ij5JdCBzZWVtcyB0
aGF0ICJkb20wX21lbT0iIGlzIGEgY29ycmVjdCBzZXR0aW5nLiBJdCBtZWFucyAiZ2l2ZSBhbGwg
dW51c2VkIGJ5IGh5cGVydmlzb3IgbWVtb3J5IHRvIGRvbTAiLjwvc3Bhbj48YnIgLz48YnIgLz4x
NTozNiwgMjgg0YHQtdC90YLRj9Cx0YDRjyAyMDIxINCzLiwgQW50aG9ueSBQRVJBUkQgJmx0O2Fu
dGhvbnkucGVyYXJkQGNpdHJpeC5jb20mZ3Q7OjxiciAvPjxibG9ja3F1b3RlIGNsYXNzPSIyMTBl
N2E4NDhlOGZjYjQ1d21pLXF1b3RlIj48cD5PbiBUaHUsIFNlcCAxNiwgMjAyMSBhdCAwMzoxNToy
MVBNICswMzAwLCBEbWl0cnkgSXNheWtpbiB3cm90ZTo8YnIgLz48L3A+PGJsb2NrcXVvdGUgY2xh
c3M9IjIxMGU3YTg0OGU4ZmNiNDV3bWktcXVvdGUiPsKgVGhpcyByZWdleCBpcyB1c2VkIGZvciBh
dXRvLWJhbGxvb24gbW9kZSBkZXRlY3Rpb24gYmFzZWQgb24gWGVuIGNvbW1hbmQgbGluZS48YnIg
Lz7CoDxiciAvPsKgVGhlIGNhc2Ugb2Ygc3BlY2lmeWluZyBhIG5lZ2F0aXZlIHNpemUgd2FzIGhh
bmRsZWQgaW5jb3JyZWN0bHkuPGJyIC8+wqBGcm9tIG1pc2MveGVuLWNvbW1hbmQtbGluZSBkb2N1
bWVudGF0aW9uOjxiciAvPsKgPGJyIC8+wqDCoMKgwqDCoGRvbTBfbWVtICh4ODYpPGJyIC8+wqDC
oMKgwqDCoD0gTGlzdCBvZiAoIG1pbjombHQ7c3omZ3Q7IHwgbWF4OiZsdDtzeiZndDsgfCAmbHQ7
c3omZ3Q7ICk8YnIgLz7CoDxiciAvPsKgwqDCoMKgwqBJZiBhIHNpemUgaXMgcG9zaXRpdmUsIGl0
IHJlcHJlc2VudHMgYW4gYWJzb2x1dGUgdmFsdWUuPGJyIC8+wqDCoMKgwqDCoElmIGEgc2l6ZSBp
cyBuZWdhdGl2ZSwgaXQgaXMgc3VidHJhY3RlZCBmcm9tIHRoZSB0b3RhbCBhdmFpbGFibGUgbWVt
b3J5LjxiciAvPsKgPGJyIC8+wqBBbHNvIGFkZCBzdXBwb3J0IGZvciBbdFRdIGdyYW51bGFyaXR5
IHN1ZmZpeC48YnIgLz7CoEFsc28gYWRkIHN1cHBvcnQgZm9yIG1lbW9yeSBmcmFjdGlvbnMgKGku
ZS4gJzUwJScgb3IgJzFHKzI1JScpLjxiciAvPsKgPGJyIC8+wqBTaWduZWQtb2ZmLWJ5OiBEbWl0
cnkgSXNheWtpbiAmbHQ7PGEgaHJlZj0ibWFpbHRvOmlzYWlraW4tZG1pdHJ5QHlhbmRleC5ydSI+
aXNhaWtpbi1kbWl0cnlAeWFuZGV4LnJ1PC9hPiZndDs8YnIgLz7CoC0tLTxiciAvPsKgwqDCoMKg
wqDCoHJldCA9IHJlZ2NvbXAoJmFtcDtyZWdleCw8YnIgLz7CoC0gICAgICAgICAgICAgICAgICAi
KF58IClkb20wX21lbT0oKHxtaW46fG1heDopWzAtOV0rW2JCa0ttTWdHXT8sPykrKCR8ICkiLDxi
ciAvPsKgKyAgICAgICAgICAgICAgICAgICIoXnwgKWRvbTBfbWVtPSgofG1pbjp8bWF4OikoLT9b
MC05XStbYkJrS21NZ0d0VF0/KT8oXCs/WzAtOV0rJSk/LD8pKygkfCApIiw8YnIgLz48L2Jsb2Nr
cXVvdGU+PHA+PGJyIC8+SXQgc2VlbXMgdGhhdCBieSB0cnlpbmcgdG8gbWF0Y2ggZnJhY3Rpb25z
LCB0aGUgbmV3IHJlZ2V4IHdvdWxkIG1hdGNoPGJyIC8+dG9vIG11Y2guIEZvciBleGFtcGxlLCBp
ZiB0aGVyZSBpcyAiIGRvbTBfbWVtPSAiIG9uIHRoZSBjb21tYW5kIGxpbmUsIHhsPGJyIC8+d291
bGQgZGV0ZWN0IGl0IGFzIGF1dG9iYWxsb29uPW9mZiwgd2hpbGUgaXQgaXNuJ3QgdGhlIGNhc2Ug
d2l0aG91dCB0aGlzPGJyIC8+cGF0Y2guIEkgZG9uJ3Qga25vdyBpZiBpdCBpcyBwb3NzaWJsZSB0
byBoYXZlICJkb20wX21lbT0iIG9uIHRoZSBjb21tYW5kPGJyIC8+bGluZSBhcyBJIGRvbid0IGtu
b3cgd2hhdCBYZW4gd291bGQgZG8gaW4gdGhpcyBjYXNlLjxiciAvPjxiciAvPkl0IG1pZ2h0IGJl
IGJldHRlciB0byBtYWtlIHRoZSByZWdleCBtb3JlIGNvbXBsaWNhdGVkIGFuZCBtYXRjaDxiciAv
PmZyYWN0aW9uIGxpa2UgdGhleSBhcmUgZGVzY3JpYmVkIGluIHRoZSBkb2MsIHNvbWV0aGluZyBs
aWtlOjxiciAvPsKgwqDCoMKgKCAmbHQ7c2l6ZSZndDsgfCAoJmx0O3NpemUmZ3Q7XCspPyZsdDtm
cmFjJmd0OyUgKTxiciAvPjxiciAvPnVubGVzcyB4ZW4gZG9lc24ndCBib290IHdpdGggYm9ndXMg
dmFsdWUgZm9yIGRvbTBfbWVtLCBidXQgSSBoYXZlbid0PGJyIC8+Y2hlY2tlZC4gKHdlIGNvdWxk
IHVzZSBDUFAgbWFjcm9zIHRvIGF2b2lkIGR1cGxpY2F0aW5nIHRoZSAmbHQ7c2l6ZSZndDs8YnIg
Lz5yZWdleC4pPGJyIC8+PGJyIC8+QWxzbywgJmx0O2ZyYWMmZ3Q7IGlzIHN1cHBvc2VkIHRvIGJl
ICZsdDsgMTAwLCBzbyBbMC05XXsxLDJ9IHdvdWxkIGJlIGJldHRlciB0bzxiciAvPm9ubHkgbWF0
Y2ggbm8gbW9yZSB0aGFuIDIgZGlnaXQuPGJyIC8+PGJyIC8+VGhvdWdodD88YnIgLz48YnIgLz5U
aGFua3MsPGJyIC8+PGJyIC8+PC9wPjxzcGFuIGNsYXNzPSJmNTViYmI0ZWVlZjIwOGU4d21pLXNp
Z24iPi0tIDxiciAvPkFudGhvbnkgUEVSQVJEPGJyIC8+PC9zcGFuPjwvYmxvY2txdW90ZT48YnIg
Lz48YnIgLz4tLSA8YnIgLz7QntGC0L/RgNCw0LLQu9C10L3QviDQuNC3INC80L7QsdC40LvRjNC9
0L7Qs9C+INC/0YDQuNC70L7QttC10L3QuNGPINCv0L3QtNC10LrRgS7Qn9C+0YfRgtGLPC9kaXY+


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 05:31:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 05:31:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198588.352137 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSBu-0004cd-EA; Wed, 29 Sep 2021 05:31:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198588.352137; Wed, 29 Sep 2021 05:31:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSBu-0004cW-A9; Wed, 29 Sep 2021 05:31:38 +0000
Received: by outflank-mailman (input) for mailman id 198588;
 Wed, 29 Sep 2021 05:31:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVSBs-0004cQ-K0
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 05:31:36 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 82b83dd8-20e6-11ec-bcdd-12813bfff9fa;
 Wed, 29 Sep 2021 05:31:35 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id ABF30223A5;
 Wed, 29 Sep 2021 05:31:34 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 89CA013A81;
 Wed, 29 Sep 2021 05:31:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 583FHzb6U2EzOgAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 05:31:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82b83dd8-20e6-11ec-bcdd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632893494; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=/8KMw82i0GYCOWtgXPFwQhIxcONcrhtasQJPp0u+f2k=;
	b=NprFYsf18Kx3SZ8QMxhLxlHCKeJ2xMqM8+zryHM1fG0DRk53vNJ2H4EgUQ2Uk89jZdz+oX
	cvtgbr20k39UC47qfWHDslT80UW/zMXuHz/4OSDVwHHw8kpnIroZy7WqjSht4R6YoikP9e
	5v1bnNbyAXhiGvWZ3iSFarVm5+21/6g=
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20210928091517.9761-1-jgross@suse.com>
 <20210928091517.9761-3-jgross@suse.com>
 <24915.1121.8356.288414@mariner.uk.xensource.com>
 <0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
 <24915.13356.139165.259646@mariner.uk.xensource.com>
From: Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
Message-ID: <3c9bfe5d-c471-c771-d6a7-a15cca466cb6@suse.com>
Date: Wed, 29 Sep 2021 07:31:33 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <24915.13356.139165.259646@mariner.uk.xensource.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="DPu5AieWRRI95BwA4gKHhHllglUSOqoFZ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--DPu5AieWRRI95BwA4gKHhHllglUSOqoFZ
Content-Type: multipart/mixed; boundary="Vtw6uKjlyWkuHLsvHUAu7H25qYt1qZmNh";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Ian Jackson <iwj@xenproject.org>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Message-ID: <3c9bfe5d-c471-c771-d6a7-a15cca466cb6@suse.com>
Subject: Re: [PATCH v5 2/2] tools/xenstore: set open file descriptor limit for
 xenstored
References: <20210928091517.9761-1-jgross@suse.com>
 <20210928091517.9761-3-jgross@suse.com>
 <24915.1121.8356.288414@mariner.uk.xensource.com>
 <0bd81ff3-a7ac-4000-4c0c-d7127b1c1985@suse.com>
 <24915.13356.139165.259646@mariner.uk.xensource.com>
In-Reply-To: <24915.13356.139165.259646@mariner.uk.xensource.com>

--Vtw6uKjlyWkuHLsvHUAu7H25qYt1qZmNh
Content-Type: multipart/mixed;
 boundary="------------56B2A43F42BBA97E1EC2F8AA"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------56B2A43F42BBA97E1EC2F8AA
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 17:26, Ian Jackson wrote:
> Juergen Gross writes ("Re: [PATCH v5 2/2] tools/xenstore: set open file=
 descriptor limit for xenstored"):
>> Hmm, maybe I should just use:
>>
>> prlimit --nofile=3D$XENSTORED_MAX_OPEN_FDS \
>>      $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS=

>=20
> I guess that would probably work (although it involves another
> exec) but I don't understand what's wrong with ulimit, which is a
> shell builtin.

My main concern with ulimit is that this would set the open file limit
for _all_ children of the script. I don't think right now this is a real
problem, but it feels wrong to me (systemd-notify ought to be fine, but
you never know ...).


Juergen

--------------56B2A43F42BBA97E1EC2F8AA
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------56B2A43F42BBA97E1EC2F8AA--

--Vtw6uKjlyWkuHLsvHUAu7H25qYt1qZmNh--

--DPu5AieWRRI95BwA4gKHhHllglUSOqoFZ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFT+jUFAwAAAAAACgkQsN6d1ii/Ey98
rAf+P5JhunS71brNgN/5BH1Jxx1PVoAsg0ZAFMIzbGLgsXmf7V9l6Q4fcL0aQ2m+LvpYSP6Frd6i
58+xlm5cktdr+h2a9u6Sq5zoilyE4lFLbwCn0exVk3Ta72isG35jt2nXidL1/vlcI4u9dV4kjkyy
l0Z+I2gBrjJ37YZOYGoXW9sN5EHn4L4DXB0gh5KB5wgu/jJ+y6vm7h79U4mnC1ug/YGL8s5+rSXU
J0TqvHUfI7KcQprXdshR4THIE02HMtIy1RVZd+NxYrzPci2KA3XqazLb9gwSCxmUIIAKI3mhQne/
Mm0vYN6soCV3gIFrgUno+ZML6CpOK0WCHWVouPqXsA==
=NPA2
-----END PGP SIGNATURE-----

--DPu5AieWRRI95BwA4gKHhHllglUSOqoFZ--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 05:46:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 05:46:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198594.352148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSPf-0006Fy-KW; Wed, 29 Sep 2021 05:45:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198594.352148; Wed, 29 Sep 2021 05:45:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSPf-0006Fr-HS; Wed, 29 Sep 2021 05:45:51 +0000
Received: by outflank-mailman (input) for mailman id 198594;
 Wed, 29 Sep 2021 05:45:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVSPe-0006Fl-5V
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 05:45:50 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7f7a6662-20e8-11ec-bcdd-12813bfff9fa;
 Wed, 29 Sep 2021 05:45:49 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 3B0AB1FD8E;
 Wed, 29 Sep 2021 05:45:48 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E004013A81;
 Wed, 29 Sep 2021 05:45:47 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id d2a2M4v9U2GpPwAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 05:45:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f7a6662-20e8-11ec-bcdd-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632894348; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=9dS3pKrH9tIElFltXULjwBVW0jJRE0pVrJdjPgg+l+E=;
	b=XbL3ujQ3fdlQpIlvtXc2AngxfRAtihnbZCB97SdpjlLMKb19am+VHiRJdaa1MUx7Q7xNOd
	G6rdPHRRG4ft5aAXIuamcz70GgebYrnbA+hhc3tDpy2oCj0wYpi86o4fMs6q2LAiIOyX/I
	LXsucskLpXV3yFKJ6EDbO3VpQDaaJKI=
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
 <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
Date: Wed, 29 Sep 2021 07:45:47 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="lhXQsMPmsgWmJyuKUUGOejYSBwDkhwWna"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--lhXQsMPmsgWmJyuKUUGOejYSBwDkhwWna
Content-Type: multipart/mixed; boundary="GjTgUdmNpyn9pBSNTzNGE4ZYXvbDrmkf1";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
 <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
In-Reply-To: <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>

--GjTgUdmNpyn9pBSNTzNGE4ZYXvbDrmkf1
Content-Type: multipart/mixed;
 boundary="------------9D5EF408F6473EEBABC4A4D3"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------9D5EF408F6473EEBABC4A4D3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 23.09.21 17:31, Jan Beulich wrote:
> On 23.09.2021 17:25, Juergen Gross wrote:
>> On 23.09.21 17:19, Jan Beulich wrote:
>>> On 23.09.2021 17:15, Juergen Gross wrote:
>>>> On 23.09.21 17:10, Jan Beulich wrote:
>>>>> On 23.09.2021 16:59, Juergen Gross wrote:
>>>>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>>>>> This was effectively lost while dropping PVHv1 code. Move the fun=
ction
>>>>>>> and arrange for it to be called the same way as done in PV mode. =
Clearly
>>>>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve=
_ad
>>>>>>> check that was recently removed, as that's a PV-only feature.
>>>>>>>
>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>
>>>>>>> --- a/arch/x86/xen/enlighten.c
>>>>>>> +++ b/arch/x86/xen/enlighten.c
>>>>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>>>>      	return ((per_cpu(xen_vcpu, cpu) =3D=3D NULL) ? -ENODEV : 0)=
;
>>>>>>>      }
>>>>>>>     =20
>>>>>>> +void __init xen_banner(void)
>>>>>>> +{
>>>>>>> +	unsigned version =3D HYPERVISOR_xen_version(XENVER_version, NUL=
L);
>>>>>>> +	struct xen_extraversion extra;
>>>>>>
>>>>>> Please add a blank line here.
>>>>>
>>>>> Oops.
>>>>>
>>>>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>>>>> +
>>>>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name)=
;
>>>>>>
>>>>>> Is this correct? I don't think the kernel needs to be paravirtuali=
zed
>>>>>> with PVH (at least not to the same extend as for PV).
>>>>>
>>>>> What else do you suggest the message to say? Simply drop
>>>>> "paravirtualized"? To some extent it is applicable imo, further
>>>>> qualified by pv_info.name. And that's how it apparently was with
>>>>> PVHv1.
>>>>
>>>> The string could be selected depending on CONFIG_XEN_PV.
>>>
>>> Hmm, now I'm confused: Doesn't this setting control whether the kerne=
l
>>> can run in PV mode? If so, that functionality being present should ha=
ve
>>> no effect on the functionality of the kernel when running in PVH mode=
=2E
>>> So what you suggest would end up in misleading information imo.
>>
>> Hmm, yes, I mixed "paravirtualized" with "capable to run
>> paravirtualized".
>>
>> So the string should depend on xen_pv_domain().
>=20
> But that's already expressed by pv_info.name then being "Xen PV".

True. Okay, I'm fine with just dropping "paravirtualized".


Juergen

--------------9D5EF408F6473EEBABC4A4D3
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------9D5EF408F6473EEBABC4A4D3--

--GjTgUdmNpyn9pBSNTzNGE4ZYXvbDrmkf1--

--lhXQsMPmsgWmJyuKUUGOejYSBwDkhwWna
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFT/YsFAwAAAAAACgkQsN6d1ii/Ey+D
AQf/cSwoaFn4oE5iEPPrrvbW0sl3fdd0HlAXrpDZloMtgQDevpgrqj920qo6w/wMOfaBYrh/P1Et
BMXih3QQHaB3vmLWTN4CQ9YY6PLXTEOBsJrOE2RSIfDVgiSaG5s7lcqxF8Jplt0G779/BcRg15oT
gzPyOA0u+pB2W+dBnRgZn1opMQWjIwvXwgVGj5fQ5PoDxNBhjCws/cnTI9DSspYFs03kOORxIYcY
Gd1HfOFMY0cvxSgfupIIcCjHfIjQdB1uCZe+wg56FIcai0/WyyauOdtvVDH7aS+Ulhb9O3AqPpMd
rv6ZgwqolCVuutQlgqFa9Nr0xG8yLCpzAbvEiQzWMw==
=ezIX
-----END PGP SIGNATURE-----

--lhXQsMPmsgWmJyuKUUGOejYSBwDkhwWna--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:03:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:03:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198600.352159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSh0-0000Sr-5K; Wed, 29 Sep 2021 06:03:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198600.352159; Wed, 29 Sep 2021 06:03:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSh0-0000Sk-1W; Wed, 29 Sep 2021 06:03:46 +0000
Received: by outflank-mailman (input) for mailman id 198600;
 Wed, 29 Sep 2021 06:03:45 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSgz-0000Se-5L
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:03:45 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 007b8d98-20eb-11ec-bcde-12813bfff9fa;
 Wed, 29 Sep 2021 06:03:44 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 878CD61357;
 Wed, 29 Sep 2021 06:03:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 007b8d98-20eb-11ec-bcde-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895423;
	bh=lFRzRQMb+/Pr9VGvjQ8zg38ynqeuX82osa/6lfy+IMI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VCiFs3CcgSiOQFR9hzwmxELpaXptRjaX5TxOSCbMADZlvvbloQN2Xo7763Lsdmn2L
	 4LBLnj9XPOC7b701ev54efCkrx1NLIPsllzYMUfT5LN1xQezQT8xtZjBQhTY9Q5FBl
	 Bf8LxK4P+YcswJwhFTRhnk5QwToLf1ZI4/fWXohGf/eGny2uvAd/K4YG5vT2bbtlRC
	 yXiMeRlhOLylHbjs827ioOOOKMLEuyN2pXsmCzhoDiRC2oDIN+yo906YmcluGKfju4
	 DGqO4+4y+U1E+wctLDlfzwhzQLmT5WfNXACL3NGXE/qzrV+ZtRooWABElaWKsSjbCx
	 9Nd/BBetrriQw==
Date: Tue, 28 Sep 2021 23:03:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: iwj@xenbits.xen.org
cc: xen-devel@lists.xenproject.org, jbeulich@suse.com, 
    julien.grall.oss@gmail.com, iwj@xenproject.org, sstabellini@kernel.org, 
    dpsmith@apertussolutions.com
Subject: Re: [adhoc test] 165241: all pass
In-Reply-To: <E1mVGlH-0006re-DA@osstest.test-lab.xenproject.org>
Message-ID: <alpine.DEB.2.21.2109282301540.5022@sstabellini-ThinkPad-T480s>
References: <E1mVGlH-0006re-DA@osstest.test-lab.xenproject.org>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

That's great, Ian.

Should I go ahead and add the patch to linux-arm-xen? Or do you want to
do it?


On Tue, 28 Sep 2021, iwj@xenbits.xen.org wrote:
> [adhoc adhoc] <testing.git master /dev/pts/25>
> harness 3a3089c9: mfi-common: Drop Linux dom0 i386 tests for newer Lin...
> 165241: all pass
> 
> flight 165241 xen-unstable adhoc [adhoc]
> http://logs.test-lab.xenproject.org/osstest/logs/165241/
> 
> Perfect :-)
> All tests in this flight passed as required
> 
> jobs:
>  build-arm64-pvops                                            pass    
> 
> 
> ------------------------------------------------------------
> sg-report-flight on osstest.test-lab.xenproject.org
> logs: /home/logs/logs
> images: /home/logs/images
> 
> Logs, config files, etc. are available at
>     http://logs.test-lab.xenproject.org/osstest/logs
> 
> Explanation of these reports, and of osstest in general, is at
>     http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
>     http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
> 
> Test harness code can be found at
>     http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:05:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:05:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198607.352170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSil-00017E-K0; Wed, 29 Sep 2021 06:05:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198607.352170; Wed, 29 Sep 2021 06:05:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSil-000177-G0; Wed, 29 Sep 2021 06:05:35 +0000
Received: by outflank-mailman (input) for mailman id 198607;
 Wed, 29 Sep 2021 06:05:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSij-00016x-OZ
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:05:33 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 40f6bc94-20eb-11ec-bcde-12813bfff9fa;
 Wed, 29 Sep 2021 06:05:32 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B2398613CD;
 Wed, 29 Sep 2021 06:05:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40f6bc94-20eb-11ec-bcde-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895531;
	bh=NMjzbixTMRRafiuzCNgRh5z82dS6yvBE9TTjAzLLOlk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JIyeU8PIWSa9t60329USfcO3nRus9QL9jS0hZlFhYpBmxrHUguWKgMxxPirU4qZAV
	 YF+9Z71jIaNBK2JTX92OtQ4LYcyVX8sCLlcuVsLtaKAzte8D2FL1r42ji1nfDA6uvs
	 gm0Fi8nDxgh143ZbU7rDIoCzBVQPKVKdfqYc4cQR6MedEUnV1kM42oHWRicp3OhDwn
	 8EXCYyxLhWktC35PezeOr0+TQyu6ZOLhyPhcgIBZvbKD5kUqbkaY17M8fPh9Eb36pq
	 NoYfUlvtSOhRvz8fPwn/MZxm7v421qamLXBCxKu9DBV74K/Iycpkfv+QeROq8a05Ny
	 qXmvH8s0QMBYQ==
Date: Tue, 28 Sep 2021 23:05:31 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v3 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support
 for ARM
In-Reply-To: <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281611220.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Hardware domain is in charge of doing the PCI enumeration and will
> discover the PCI devices and then will communicate to XEN via hyper
> call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
> 
> Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
> 
> As most of the code for PHYSDEVOP_pci_device_* is the same between x86
> and ARM, move the code to a common file to avoid duplication.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3: Fixed minor comment.
> Change in v2:
> - Add support for PHYSDEVOP_pci_device_remove()
> - Move code to common code
> ---
>  xen/arch/arm/physdev.c          |  5 +-
>  xen/arch/x86/physdev.c          | 50 +-------------------
>  xen/arch/x86/x86_64/physdev.c   |  4 +-
>  xen/common/Makefile             |  1 +
>  xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/hypercall.h |  2 -
>  xen/include/asm-arm/numa.h      |  5 ++
>  xen/include/asm-x86/hypercall.h |  9 ++--
>  xen/include/xen/hypercall.h     |  8 ++++
>  9 files changed, 106 insertions(+), 59 deletions(-)
>  create mode 100644 xen/common/physdev.c
> 
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index e91355fe22..4e00b03aab 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -8,10 +8,9 @@
>  #include <xen/lib.h>
>  #include <xen/errno.h>
>  #include <xen/sched.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>  
> -
> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  {
>      gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd);
>      return -ENOSYS;
> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
> index 23465bcd00..c00cc99404 100644
> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
>  }
>  #endif /* COMPAT */
>  
> -ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  {
>      int irq;
>      ret_t ret;
> @@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          break;
>      }
>  
> -    case PHYSDEVOP_pci_device_add: {
> -        struct physdev_pci_device_add add;
> -        struct pci_dev_info pdev_info;
> -        nodeid_t node;
> -
> -        ret = -EFAULT;
> -        if ( copy_from_guest(&add, arg, 1) != 0 )
> -            break;
> -
> -        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
> -        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> -        {
> -            pdev_info.is_virtfn = 1;
> -            pdev_info.physfn.bus = add.physfn.bus;
> -            pdev_info.physfn.devfn = add.physfn.devfn;
> -        }
> -        else
> -            pdev_info.is_virtfn = 0;
> -
> -        if ( add.flags & XEN_PCI_DEV_PXM )
> -        {
> -            uint32_t pxm;
> -            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
> -                                sizeof(add.optarr[0]);
> -
> -            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> -                break;
> -
> -            node = pxm_to_node(pxm);
> -        }
> -        else
> -            node = NUMA_NO_NODE;
> -
> -        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
> -        break;
> -    }
> -
> -    case PHYSDEVOP_pci_device_remove: {
> -        struct physdev_pci_device dev;
> -
> -        ret = -EFAULT;
> -        if ( copy_from_guest(&dev, arg, 1) != 0 )
> -            break;
> -
> -        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
> -        break;
> -    }
> -
>      case PHYSDEVOP_prepare_msix:
>      case PHYSDEVOP_release_msix: {
>          struct physdev_pci_device dev;
> diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.c
> index 0a50cbd4d8..5f72652ff7 100644
> --- a/xen/arch/x86/x86_64/physdev.c
> +++ b/xen/arch/x86/x86_64/physdev.c
> @@ -9,9 +9,10 @@ EMIT_FILE;
>  #include <compat/xen.h>
>  #include <compat/event_channel.h>
>  #include <compat/physdev.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>  
>  #define do_physdev_op compat_physdev_op
> +#define arch_physdev_op arch_compat_physdev_op
>  
>  #define physdev_apic               compat_physdev_apic
>  #define physdev_apic_t             physdev_apic_compat_t
> @@ -82,6 +83,7 @@ CHECK_physdev_pci_device
>  typedef int ret_t;
>  
>  #include "../physdev.c"
> +#include "../../../common/physdev.c"
>  
>  /*
>   * Local variables:
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 54de70d422..bcb1c8fb03 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -29,6 +29,7 @@ obj-y += notifier.o
>  obj-y += page_alloc.o
>  obj-$(CONFIG_HAS_PDX) += pdx.o
>  obj-$(CONFIG_PERF_COUNTERS) += perfc.o
> +obj-y += physdev.o
>  obj-y += preempt.o
>  obj-y += random.o
>  obj-y += rangeset.o
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> new file mode 100644
> index 0000000000..20a5530269
> --- /dev/null
> +++ b/xen/common/physdev.c
> @@ -0,0 +1,81 @@
> +
> +#include <xen/guest_access.h>
> +#include <xen/hypercall.h>
> +#include <xen/init.h>
> +
> +#ifndef COMPAT
> +typedef long ret_t;
> +#endif
> +
> +ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +{
> +    ret_t ret;
> +
> +    switch ( cmd )
> +    {
> +#ifdef CONFIG_HAS_PCI
> +    case PHYSDEVOP_pci_device_add: {
> +        struct physdev_pci_device_add add;
> +        struct pci_dev_info pdev_info;
> +        nodeid_t node;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&add, arg, 1) != 0 )
> +            break;
> +
> +        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
> +        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> +        {
> +            pdev_info.is_virtfn = 1;
> +            pdev_info.physfn.bus = add.physfn.bus;
> +            pdev_info.physfn.devfn = add.physfn.devfn;
> +        }
> +        else
> +            pdev_info.is_virtfn = 0;
> +
> +        if ( add.flags & XEN_PCI_DEV_PXM )
> +        {
> +            uint32_t pxm;
> +            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
> +                                sizeof(add.optarr[0]);
> +
> +            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> +                break;
> +
> +            node = pxm_to_node(pxm);
> +        }
> +        else
> +            node = NUMA_NO_NODE;
> +
> +        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
> +        break;
> +    }
> +
> +    case PHYSDEVOP_pci_device_remove: {
> +        struct physdev_pci_device dev;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&dev, arg, 1) != 0 )
> +            break;
> +
> +        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
> +        break;
> +    }
> +#endif
> +    default:
> +        ret = arch_physdev_op(cmd, arg);
> +        break;
> +    }
> +
> +    return ret;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/hypercall.h b/xen/include/asm-arm/hypercall.h
> index a0c5a31a2f..c6e4a04fd6 100644
> --- a/xen/include/asm-arm/hypercall.h
> +++ b/xen/include/asm-arm/hypercall.h
> @@ -2,8 +2,6 @@
>  #define __ASM_ARM_HYPERCALL_H__
>  
>  #include <public/domctl.h> /* for arch_do_domctl */
> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> -
>  long do_arm_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg);
>  
>  long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 31a6de4e23..dbd1c2467f 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +static inline nodeid_t pxm_to_node(unsigned pxm)
> +{
> +    return 0xff;
> +}
> +
>  #endif /* __ARCH_ARM_NUMA_H */
>  /*
>   * Local variables:
> diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
> index 0ae3b8b043..7afba98186 100644
> --- a/xen/include/asm-x86/hypercall.h
> +++ b/xen/include/asm-x86/hypercall.h
> @@ -105,10 +105,6 @@ do_update_va_mapping(
>      u64 val64,
>      unsigned long flags);
>  
> -extern long
> -do_physdev_op(
> -    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> -
>  extern long
>  do_update_va_mapping_otherdomain(
>      unsigned long va,
> @@ -151,6 +147,11 @@ compat_physdev_op(
>      int cmd,
>      XEN_GUEST_HANDLE_PARAM(void) arg);
>  
> +extern int
> +arch_compat_physdev_op(
> +    int cmd,
> +    XEN_GUEST_HANDLE_PARAM(void) arg);
> +
>  extern int
>  arch_compat_vcpu_op(
>      int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg);
> diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
> index 3771487a30..5cc74f40b7 100644
> --- a/xen/include/xen/hypercall.h
> +++ b/xen/include/xen/hypercall.h
> @@ -45,6 +45,14 @@ extern long
>  do_platform_op(
>      XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op);
>  
> +extern long
> +do_physdev_op(
> +    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> +
> +extern long
> +arch_physdev_op(
> +    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> +
>  /*
>   * To allow safe resume of do_memory_op() after preemption, we need to know
>   * at what point in the page list to resume. For this purpose I steal the
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:07:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:07:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198613.352180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSk7-0001ir-TV; Wed, 29 Sep 2021 06:06:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198613.352180; Wed, 29 Sep 2021 06:06:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSk7-0001ik-QO; Wed, 29 Sep 2021 06:06:59 +0000
Received: by outflank-mailman (input) for mailman id 198613;
 Wed, 29 Sep 2021 06:06:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSk6-0001ic-6C
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:06:58 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7374e0a6-20eb-11ec-bcde-12813bfff9fa;
 Wed, 29 Sep 2021 06:06:57 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 6F422613CD;
 Wed, 29 Sep 2021 06:06:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7374e0a6-20eb-11ec-bcde-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895616;
	bh=WDCuyF8HnhujNqB8M39oQRzxnY7gu7he4k0+FNDp5vQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=R92ZdSeHN7R+LPwpLesX5enwZh9lWhV2619w3Rx0ILTsnePXyXNWzMRt5aRTi5fb4
	 Sih69sRklSVZOZV0/+EeU7Xlnz6rxf5LxhTtniqq+UJuo2WXlelJrdmjhV1HlEy/AN
	 dcy7AQiPF1eYjAKtVtHOoobUGUqj0wyjBq/WpbAAVZ5cxpb9YmrCehbomwW6rMVzcG
	 XwQTAXjJKkkOS1/I4aRPhnHBGpjIcYTHSzkVZL/BOWD6SGm8zp+5KzEG+AovM9FJmK
	 wXN97uXRu8jQRObrMVAq3dR5wa0P9qHr1YH7ZX+EKjKaiG0ovBt7cswuenQ/KFxAoh
	 qsAZwwFJiE2Tg==
Date: Tue, 28 Sep 2021 23:06:56 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v3 06/17] xen/device-tree: Add dt_property_read_variable_u32_array
 helper
In-Reply-To: <9dba5f3457d9ead9084ac73297bf5a65e665a8d9.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281615260.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <9dba5f3457d9ead9084ac73297bf5a65e665a8d9.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
> 
> Import the Linux helper of_property_read_variable_u32_array. This
> function find and read an array of 32 bit integers from a property,
> with bounds on the minimum and maximum array size.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable 
>   Linux commit-id
> Change in v2: Patch introduced in v2
> ---
>  xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
>  xen/include/xen/device_tree.h | 26 +++++++++++++++
>  2 files changed, 87 insertions(+)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 03d25a81ce..53160d61f8 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_node *np,
>      return 0;
>  }
>  
> +/**
> + * dt_find_property_value_of_size
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @min:    minimum allowed length of property value
> + * @max:    maximum allowed length of property value (0 means unlimited)
> + * @len:    if !=NULL, actual length is written to here
> + *
> + * Search for a property in a device node and valid the requested size.
> + *
> + * Return: The property value on success, -EINVAL if the property does not
> + * exist, -ENODATA if property does not have a value, and -EOVERFLOW if the
> + * property data is too small or too large.
> + */
> +static void *dt_find_property_value_of_size(const struct dt_device_node *np,
> +                                            const char *propname, u32 min,
> +                                            u32 max, size_t *len)
> +{
> +    const struct dt_property *prop = dt_find_property(np, propname, NULL);
> +
> +    if ( !prop )
> +        return ERR_PTR(-EINVAL);
> +    if ( !prop->value )
> +        return ERR_PTR(-ENODATA);
> +    if ( prop->length < min )
> +        return ERR_PTR(-EOVERFLOW);
> +    if ( max && prop->length > max )
> +        return ERR_PTR(-EOVERFLOW);
> +
> +    if ( len )
> +        *len = prop->length;
> +
> +    return prop->value;
> +}
> +
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_values,
> +                                        size_t sz_min, size_t sz_max)
> +{
> +    size_t sz, count;
> +    const __be32 *val = dt_find_property_value_of_size(np, propname,
> +                        (sz_min * sizeof(*out_values)),
> +                        (sz_max * sizeof(*out_values)),
> +                        &sz);
> +
> +    if ( IS_ERR(val) )
> +        return PTR_ERR(val);
> +
> +    if ( !sz_max )
> +        sz = sz_min;
> +    else
> +        sz /= sizeof(*out_values);
> +
> +    count = sz;
> +    while ( count-- )
> +        *out_values++ = be32_to_cpup(val++);
> +
> +    return sz;
> +}
> +
>  int dt_property_match_string(const struct dt_device_node *np,
>                               const char *propname, const char *string)
>  {
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index b02696be94..1693fb8e8c 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_node *np,
>  bool_t dt_property_read_u64(const struct dt_device_node *np,
>                              const char *name, u64 *out_value);
>  
> +
> +/**
> + * dt_property_read_variable_u32_array - Find and read an array of 32 bit
> + * integers from a property, with bounds on the minimum and maximum array size.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return found values.
> + * @sz_min: minimum number of array elements to read
> + * @sz_max: maximum number of array elements to read, if zero there is no
> + *      upper limit on the number of elements in the dts entry but only
> + *      sz_min will be read.
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: The number of elements read on success, -EINVAL if the property
> + * does not exist, -ENODATA if property does not have a value, and -EOVERFLOW
> + * if the property data is smaller than sz_min or longer than sz_max.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_values,
> +                                        size_t sz_min, size_t sz_max);
> +
>  /**
>   * dt_property_read_bool - Check if a property exists
>   * @np: node to get the value
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:07:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:07:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198619.352191 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSks-0002Sm-5q; Wed, 29 Sep 2021 06:07:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198619.352191; Wed, 29 Sep 2021 06:07:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSks-0002Sf-2w; Wed, 29 Sep 2021 06:07:46 +0000
Received: by outflank-mailman (input) for mailman id 198619;
 Wed, 29 Sep 2021 06:07:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSkq-0002ST-CP
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:07:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 48bc7a58-8a58-4a16-8f73-c60a89512a3c;
 Wed, 29 Sep 2021 06:07:43 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A5C46124B;
 Wed, 29 Sep 2021 06:07:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 48bc7a58-8a58-4a16-8f73-c60a89512a3c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895663;
	bh=gi7yLRdXEuHiDcYD8QGkIxiima8qvnbL7FOBGGfTlwE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jzmINPKH6x37HfW0DNtB8JiZ1mHSAUtVd8yyg3qjka+NtJlf2bGuUU+mtuSs3VabA
	 zkPt2Qi/g6b1mDg6J1iU+BnYwArLX9/79N33sMwomFxJOUTK8qjysCI6Q4dt/CMdI+
	 SzVNOqBttPtscI1q/j8lQXHW2MWzADvnMIxfs0IQR4h0QwJoV3VjfvisnM+HT/mXTY
	 mPocivfLbEYwgQz1HYu6DMOvZyN/irGNsuy+CW8rjHjWmF2okq+asZtDkpb5iLYlIE
	 udAOdQzZ+Ikpq5sEW5K9AOtiQgruT7VqnLoqt9rVwM49fkU2O3H2RP0aV89R5y/lmr
	 LuoIwgyhYMPNg==
Date: Tue, 28 Sep 2021 23:07:42 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v3 07/17] xen/device-tree: Add dt_property_read_u32_array
 helper
In-Reply-To: <89788e202bc91e2bc7f9c3c33eb91ab7478381ee.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281618560.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <89788e202bc91e2bc7f9c3c33eb91ab7478381ee.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
> 
> Import the Linux helper of_property_read_u32_array. This function find
> and read an array of 32 bit integers from a property.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable
>   Linux commit-id
> Change in v2: Patch introduced in v2
> ---
>  xen/include/xen/device_tree.h | 30 ++++++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)
> 
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 1693fb8e8c..3ffe3eb3d2 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -392,6 +392,36 @@ int dt_property_read_variable_u32_array(const struct dt_device_node *np,
>                                          const char *propname, u32 *out_values,
>                                          size_t sz_min, size_t sz_max);
>  
> +/**
> + * dt_property_read_u32_array - Find and read an array of 32 bit integers
> + * from a property.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return value, modified only if return value is 0.
> + * @sz:     number of array elements to read
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: 0 on success, -EINVAL if the property does not exist,
> + * -ENODATA if property does not have a value, and -EOVERFLOW if the
> + * property data isn't large enough.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +static inline int dt_property_read_u32_array(const struct dt_device_node *np,
> +                                             const char *propname,
> +                                             u32 *out_values, size_t sz)
> +{
> +    int ret = dt_property_read_variable_u32_array(np, propname, out_values,
> +                              sz, 0);
> +    if ( ret >= 0 )
> +        return 0;
> +    else
> +        return ret;
> +}
> +
>  /**
>   * dt_property_read_bool - Check if a property exists
>   * @np: node to get the value
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:09:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:09:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198625.352203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSmE-00038K-Ia; Wed, 29 Sep 2021 06:09:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198625.352203; Wed, 29 Sep 2021 06:09:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSmE-00038D-DZ; Wed, 29 Sep 2021 06:09:10 +0000
Received: by outflank-mailman (input) for mailman id 198625;
 Wed, 29 Sep 2021 06:09:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSmD-00037k-9O
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:09:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 39b977f4-5401-44a2-b89b-097d376cf577;
 Wed, 29 Sep 2021 06:09:08 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B31D46124B;
 Wed, 29 Sep 2021 06:09:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39b977f4-5401-44a2-b89b-097d376cf577
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895747;
	bh=eb2qUmJqXqj9nrH6P/sXBPB9SqhnS3wPQqwTvxUa4yk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tqalPHkFnEAVfGaJXy14dSAMysoCU45POOYIg8ZwOOCo1inQKYQ0hhc94zglPW6Mi
	 fEOXH9Q9XPIPepjYSfm95K45N9aEinzNcexg4lNBidtEJh0fTff0jJBIp96aOWUBNk
	 hj4bxv1onOjz0Nqw93WK+U24e2yEUGLmwMaLUym8h+7VTuRAtgmcp9sS9LAkhYMaUr
	 zAyZvCaRj3gLZ9D+7pKR4TznHttkof6KevJoDJ/Cke9SgoBbmvc3SXripXwSd6cXRf
	 4NObRbzvhKdDS0aYZ997GT5+3l5i39XCEN7mnPQydevm8YcKkXKXaZGW9MQXigJfpA
	 Y0tPTAgeTdTug==
Date: Tue, 28 Sep 2021 23:09:07 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>
Subject: Re: [PATCH v3 08/17] xen/device-tree: Add dt_get_pci_domain_nr
 helper
In-Reply-To: <f70ac215cd0c62d71231b8f61f12c1f8d801710e.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281621310.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <f70ac215cd0c62d71231b8f61f12c1f8d801710e.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Based Linux commit 41e5c0f81d3e676d671d96a0a1fafb27abfbd9d7
> 
> Import the Linux helper of_get_pci_domain_nr. This function will try to
> obtain the host bridge domain number by finding a property called
> "linux,pci-domain" of the given device node.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable
>   Linux commit-id
> - Remove return value as those are not valid for XEN
> Change in v2: Patch introduced in v2
> ---
>  xen/common/device_tree.c      | 12 ++++++++++++
>  xen/include/xen/device_tree.h | 17 +++++++++++++++++
>  2 files changed, 29 insertions(+)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 53160d61f8..ea93da1725 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
>      dt_alias_scan();
>  }
>  
> +int dt_get_pci_domain_nr(struct dt_device_node *node)
> +{
> +    u32 domain;
> +    int error;
> +
> +    error = dt_property_read_u32(node, "linux,pci-domain", &domain);
> +    if ( !error )
> +        return -EINVAL;
> +
> +    return (u16)domain;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 3ffe3eb3d2..2297c59ce6 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -832,6 +832,23 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
>                                 const char *list_name,
>                                 const char *cells_name);
>  
> +/**
> + * dt_get_pci_domain_nr - Find the host bridge domain number
> + *            of the given device node.
> + * @node: Device tree node with the domain information.
> + *
> + * This function will try to obtain the host bridge domain number by finding
> + * a property called "linux,pci-domain" of the given device node.
> + *
> + * Return:
> + * * > 0    - On success, an associated domain number.
> + * * -EINVAL    - The property "linux,pci-domain" does not exist.
> + *
> + * Returns the associated domain number from DT in the range [0-0xffff], or
> + * a negative value if the required property is not found.
> + */
> +int dt_get_pci_domain_nr(struct dt_device_node *node);
> +
>  #ifdef CONFIG_DEVICE_TREE_DEBUG
>  #define dt_dprintk(fmt, args...)  \
>      printk(XENLOG_DEBUG fmt, ## args)
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:10:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:10:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198633.352214 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSnq-0004WK-1t; Wed, 29 Sep 2021 06:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198633.352214; Wed, 29 Sep 2021 06:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSnp-0004WD-Tf; Wed, 29 Sep 2021 06:10:49 +0000
Received: by outflank-mailman (input) for mailman id 198633;
 Wed, 29 Sep 2021 06:10:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSno-0004W2-C9
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:10:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fcbb5b4c-20eb-11ec-bcde-12813bfff9fa;
 Wed, 29 Sep 2021 06:10:47 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id B7D506124B;
 Wed, 29 Sep 2021 06:10:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcbb5b4c-20eb-11ec-bcde-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895846;
	bh=UlUjuhgKaaH7cczSJ/gLOUPfURWGnqJjL+P6NuLpDsw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=g+WjhMve5LmikvavwcKQfTH8+k20DHGoyB3sWIVGww/P01DrvxzLb0krgToOYWMVw
	 BTZmBffDd8uqybX0D6Kt3WIPdHtrnYGOFmlFv0lc3WKFMgBpAXXWK4ag/1imH+HjTY
	 M9ByPYlUxIlOwKujVgg/w7MHcUQbeQ08ntbDzcvos4SblfLEgMbvSGhCtV719O7WR5
	 B1Oz4WWLQnSitpYIEoX3BLltsDm2AKi7lXxLkS/wiOdzhQ12YeRz4xd9WhKCOV7aWv
	 rbQBSlkdBBr9o1rVfsRp2fUjx8ZDA66jsXo2IMFWF7BnOr/cCIMAr43xD1NunfSpXm
	 LX1jUYKfOIGBQ==
Date: Tue, 28 Sep 2021 23:10:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
In-Reply-To: <31f2e3baf45f059a8ddac22fefc7cdfe1bc63ef5.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281625430.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <31f2e3baf45f059a8ddac22fefc7cdfe1bc63ef5.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> pci_init(..) will be called during xen startup to initialize and probe
> the PCI host-bridge driver.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Some nit for device_init(..) return logic
> - Remove inline from acpi_pci_init(..)
> - Modify return value for apci_pci_init(..) to return -EOPNOTSUPP
> Change in v2:
> - ACPI init function to return int
> - pci_segments_init() called before dt/acpi init
> ---
>  xen/arch/arm/pci/pci.c       | 51 ++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/device.h |  1 +
>  2 files changed, 52 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index a7a7bc3213..e359bab9ea 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -12,6 +12,10 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>  
> +#include <xen/acpi.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
>  #include <xen/pci.h>
>  
>  /*
> @@ -22,6 +26,53 @@ int arch_pci_clean_pirqs(struct domain *d)
>      return 0;
>  }
>  
> +static int __init dt_pci_init(void)
> +{
> +    struct dt_device_node *np;
> +    int rc;
> +
> +    dt_for_each_device_node(dt_host, np)
> +    {
> +        rc = device_init(np, DEVICE_PCI, NULL);
> +        /*
> +         * Ignore the following error codes:
> +         *   - EBADF: Indicate the current device is not a pci device.
> +         *   - ENODEV: The pci device is not present or cannot be used by
> +         *     Xen.
> +         */
> +        if( !rc || rc == -EBADF || rc == -ENODEV )
> +            continue;
> +
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
> +#ifdef CONFIG_ACPI
> +static int __init acpi_pci_init(void)
> +{
> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
> +    return -EOPNOTSUPP;
> +}
> +#else
> +static int __init acpi_pci_init(void)
> +{
> +    return -EINVAL;
> +}
> +#endif
> +
> +static int __init pci_init(void)
> +{
> +    pci_segments_init();
> +
> +    if ( acpi_disabled )
> +        return dt_pci_init();
> +    else
> +        return acpi_pci_init();
> +}
> +__initcall(pci_init);
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index ee7cff2d44..5ecd5e7bd1 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -34,6 +34,7 @@ enum device_class
>      DEVICE_SERIAL,
>      DEVICE_IOMMU,
>      DEVICE_GIC,
> +    DEVICE_PCI,
>      /* Use for error */
>      DEVICE_UNKNOWN,
>  };
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 06:12:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 06:12:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198639.352225 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSoy-00057M-BI; Wed, 29 Sep 2021 06:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198639.352225; Wed, 29 Sep 2021 06:12:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVSoy-00057F-8H; Wed, 29 Sep 2021 06:12:00 +0000
Received: by outflank-mailman (input) for mailman id 198639;
 Wed, 29 Sep 2021 06:11:59 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVSox-000574-2M
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 06:11:59 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 26cdc910-20ec-11ec-bcde-12813bfff9fa;
 Wed, 29 Sep 2021 06:11:58 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 49DA760560;
 Wed, 29 Sep 2021 06:11:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26cdc910-20ec-11ec-bcde-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632895917;
	bh=T58myEBQBInPXM+HTVDBnF2QadmvX8E8f1gGzvv/Z/4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=toJ/4iHpCaE5Fse8SSBwXzLQMjLOoWDr8DAiOT4ZcvGi/UmxmrZv68HZlwRxP+jtk
	 JiUCApaeepWuCQxfWVoAHviu06OgHUx49B/UNkEAMrTu/8dNCm+cVhBM2S3VM1paIO
	 j04IVhfiB0BjcjtGPJLx+VlKUG5MIolF6HwIyRWCKzbRuvhsB6gNAsBzFyx1Ic3mb7
	 tMdZAzoaCTWYz6DkV2H8shxPZ14xRYswzZomuqG8BFnGPVmgI4yzyYDpbPSO3kx7W9
	 FUqXmTzn+ny5emM67X7F+JBEHhCx9CciIhY3t0j8i71fxryXY5kVO0ar2iuNr6M5us
	 2ZPq+qrNCpv7g==
Date: Tue, 28 Sep 2021 23:11:57 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH v3 10/17] xen/arm: Add cmdline boot option "pci-passthrough
 = <boolean>"
In-Reply-To: <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281629480.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Add cmdline boot option "pci-passthrough = = <boolean>" to enable or
> disable the PCI passthrough support on ARM.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Remove "define pci_passthrough_enabled (false)"
> - Fixed coding style and minor comment
> Change in v2:
> - Add option in xen-command-line.pandoc
> - Change pci option to pci-passthrough
> - modify option from custom_param to boolean param
> ---
>  docs/misc/xen-command-line.pandoc |  7 +++++++
>  xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
>  xen/common/physdev.c              |  6 ++++++
>  xen/include/asm-arm/pci.h         | 11 +++++++++++
>  xen/include/asm-x86/pci.h         |  8 ++++++++
>  5 files changed, 46 insertions(+)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 177e656f12..c8bf96ccf8 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1808,6 +1808,13 @@ All numbers specified must be hexadecimal ones.
>  
>  This option can be specified more than once (up to 8 times at present).
>  
> +### pci-passthrough (arm)
> +> `= <boolean>`
> +
> +> Default: `false`
> +
> +Flag to enable or disable support for PCI passthrough
> +
>  ### pcid (x86)
>  > `= <boolean> | xpti=<bool>`
>  
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index e359bab9ea..84d8f0d634 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -16,6 +16,7 @@
>  #include <xen/device_tree.h>
>  #include <xen/errno.h>
>  #include <xen/init.h>
> +#include <xen/param.h>
>  #include <xen/pci.h>
>  
>  /*
> @@ -62,8 +63,21 @@ static int __init acpi_pci_init(void)
>  }
>  #endif
>  
> +/*
> + * By default pci passthrough is disabled
> + */
> +bool __read_mostly pci_passthrough_enabled = false;
> +boolean_param("pci-passthrough", pci_passthrough_enabled);
> +
>  static int __init pci_init(void)
>  {
> +    /*
> +     * Enable PCI passthrough when has been enabled explicitly
> +     * (pci-passthrough=on)
> +     */
> +    if ( !pci_passthrough_enabled )
> +        return 0;
> +
>      pci_segments_init();
>  
>      if ( acpi_disabled )
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> index 20a5530269..2d5fc886fc 100644
> --- a/xen/common/physdev.c
> +++ b/xen/common/physdev.c
> @@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          struct pci_dev_info pdev_info;
>          nodeid_t node;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>          ret = -EFAULT;
>          if ( copy_from_guest(&add, arg, 1) != 0 )
>              break;
> @@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_pci_device_remove: {
>          struct physdev_pci_device dev;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>          ret = -EFAULT;
>          if ( copy_from_guest(&dev, arg, 1) != 0 )
>              break;
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 7dd9eb3dba..0cf849e26f 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -19,14 +19,25 @@
>  
>  #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>  
> +extern bool_t pci_passthrough_enabled;
> +
>  /* Arch pci dev struct */
>  struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return pci_passthrough_enabled;
> +}
>  #else   /*!CONFIG_HAS_PCI*/
>  
>  struct arch_pci_dev { };
>  
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return false;
> +}
> +
>  #endif  /*!CONFIG_HAS_PCI*/
>  #endif /* __ARM_PCI_H__ */
> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
> index cc05045e9c..3f806ce7a8 100644
> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned int *seg,
>  extern int pci_mmcfg_config_num;
>  extern struct acpi_mcfg_allocation *pci_mmcfg_config;
>  
> +/*
> + * Unlike ARM, PCI passthrough is always enabled for x86.
> + */
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return true;
> +}
> +
>  #endif /* __X86_PCI_H__ */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:28:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:28:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198647.352247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1K-0004rn-Ag; Wed, 29 Sep 2021 07:28:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198647.352247; Wed, 29 Sep 2021 07:28:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1K-0004rg-75; Wed, 29 Sep 2021 07:28:50 +0000
Received: by outflank-mailman (input) for mailman id 198647;
 Wed, 29 Sep 2021 07:28:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU1J-0004rA-BO
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:28:49 +0000
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (unknown
 [40.107.2.72]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e249a754-20f6-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 07:28:47 +0000 (UTC)
Received: from DB8PR09CA0029.eurprd09.prod.outlook.com (2603:10a6:10:a0::42)
 by DB7PR08MB3658.eurprd08.prod.outlook.com (2603:10a6:10:27::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 07:28:45 +0000
Received: from DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:a0:cafe::bf) by DB8PR09CA0029.outlook.office365.com
 (2603:10a6:10:a0::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 07:28:45 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT017.mail.protection.outlook.com (10.152.20.114) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:28:45 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Wed, 29 Sep 2021 07:28:42 +0000
Received: from 4de97b46f34d.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 492B0BBB-9F73-4B2D-B8BB-146672599C75.1; 
 Wed, 29 Sep 2021 07:28:33 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4de97b46f34d.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:28:33 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5690.eurprd08.prod.outlook.com (2603:10a6:102:86::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 07:28:30 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:28:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e249a754-20f6-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NGDZGitNYmmrPj9EmZb9g5FOxAHzyr9GPT7FF64YxAY=;
 b=0nuJSE0SzaWfz2og2qLWmC6yEhtE7SBYbgmHV+6x3lnRmIEFj8j2yCOV9eVNH9B6Q1PDYME1EiPvQHL5HRhiSQRRz89xrKNNxx1++jlfIekJUyfnc8z5Q8gj6Or9HKbSoTqxHfK/tswqkUUw0iyFwClCQFKf0JNVo5u9W8V843Q=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 06e75e1dc0a7fb0b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FzEQuje99Iw7gYf0ZT6sRlFErcEBCZo+TMUSONe5G5VVJXWZSbVebJ5fZo73XG3uOqqHFUtNJAjJ9idsXVAPqYzfHG9mROjDoOFq0+Zw2NYLttm9WSkAzI3T17Ya6xPVD0b7munMFcRzplK2fx1niD0+EeJCJ6dMaU/yV5LNh9yZqf92jCWPVhmqalheBP5gZFjB8VtDYuhVXvMuWjbxWo5RImeSCgBiRvt8FgQR+yZpJsSeKfytHxNxPxYqghjElJ7XBWZMOaBmJxE+nqRZQUZZ1C0XIZbPxVbav3CBdv/wsdmPMmdHDjwy8kj0HMWnTHVQaesjvC2mvqfvhmEdnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NGDZGitNYmmrPj9EmZb9g5FOxAHzyr9GPT7FF64YxAY=;
 b=DqOGrK+oducbvaw2Hmb6fDdNdTqi3Et9Px9LoUSCHUQ1+T91FgvtuP3JZboey+EugPuNfcGRZVo6QJTam3HxGYiF0qSSeY1pMp3cCycA2ZanJolJskN+4wTDfZb181QlI2WenNRjc+4+VFXqPeFNsEtGU2KWyo0u8QkDot+6Ooqh8VIX2W6KihKyHtwENwJFtZ9VonrTh5gPCZIJvJ4RHnLOALLvb1zL3r6fB3xdUE8CfZpNLAouBIvcoKW0F7cYQT01BxHQe9+Lzt3cp3Xhfrp25g8aj5YAYdXmugK6v5kueU6oisPObAbOYKEJgh/kLLcmYbGbbpfG1zdQdEeKVw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NGDZGitNYmmrPj9EmZb9g5FOxAHzyr9GPT7FF64YxAY=;
 b=0nuJSE0SzaWfz2og2qLWmC6yEhtE7SBYbgmHV+6x3lnRmIEFj8j2yCOV9eVNH9B6Q1PDYME1EiPvQHL5HRhiSQRRz89xrKNNxx1++jlfIekJUyfnc8z5Q8gj6Or9HKbSoTqxHfK/tswqkUUw0iyFwClCQFKf0JNVo5u9W8V843Q=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Paul Durrant
	<paul@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>, "Daniel P. Smith"
	<dpsmith@apertussolutions.com>
Subject: Re: [PATCH v3 01/17] xen/pci: Refactor MSI code that implements MSI
 functionality within XEN
Thread-Topic: [PATCH v3 01/17] xen/pci: Refactor MSI code that implements MSI
 functionality within XEN
Thread-Index: AQHXtJVlTSED9jePKUmBgpHXuYbHAKu6ngYA
Date: Wed, 29 Sep 2021 07:28:28 +0000
Message-ID: <10818773-E678-4704-8AA0-6953004FB62B@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: cce10795-2a4a-45ff-52ff-08d9831ac59f
x-ms-traffictypediagnostic: PR3PR08MB5690:|DB7PR08MB3658:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB3658D76B1A89A287355C9E129DA99@DB7PR08MB3658.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 X8/jkeFHEM+aOq3MQX3SnxlgzHUrGY3VE2HvZPykJsrcxUXewCcwbBlwV6AGOn0yduEvRDVzslnpJHvv9oUYnr6pJukTN0AD86apapJxX3bFRiC3BSVHG3bR8IBA2vHqYHoq63jk+a8sM6w7ubh98LUDnQqY2obEHLeQeFBWf621A9McjptL+iVqkMbK4Mrx4aXp/OYAlNDcFnhR4bS905/LjZGmcgUpQ5XU8qqDcyE+eVnOdc8iYigwMpdnMzhm7VVB9Xxe4RW1K68d6NDQhI9fEyt9l9yB/L7U+oZwnsB6+fQuC82qnaJCCamHV/0YNzrbuU59PctP9kRejYOOmEvNoM7c31UH5LsH0eOXVEr0r4cuJUsfwWcQyP9J8MQQT/BBOA2RWEZ06WUcdQBWIuDhzXYdU6HpP7PRnzs+UDAOPlVKvvmltRagCg0xpMYQUHldNgxtYsXR0Hf6VBmoio66+GiF1C14v+ZWf+LKb933sAB1muvK84wP4JLWkjZzNZs/NZuhqBdHy5N6Zo15uokbcsCCD3WG3O2i/0vSHXS0W+B0BgzVRE4GarcERqd5V+XKLI8B4MNeIyb4lBdgCV6Ss2hj0Zh9ks7IAUN+o0IssTqSshsEm7UtZbAZZWiOlTv+mbc9W+QrEgOi767+A8muG+G2wTPGQbvidrix3uYpUVJ3CMwJE6kkc1OpxOcAb4BhpYKTHDVwNRCfn1ijU4zbqreZMFX1IrjS/uz4N3OX71vDejjcgrRF+UAreT0g
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(76116006)(91956017)(86362001)(33656002)(26005)(66446008)(64756008)(66556008)(66476007)(66946007)(37006003)(5660300002)(6636002)(508600001)(316002)(71200400001)(4326008)(54906003)(6486002)(186003)(8676002)(6862004)(38070700005)(122000001)(7416002)(8936002)(2906002)(83380400001)(2616005)(6512007)(38100700002)(53546011)(6506007)(36756003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <F0C152C72C25B8418CC199D13569CB70@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5690
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	7c22141a-5e18-4977-daa2-08d9831abbf6
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3eQYQXYyhDzyBOl1Tq0wKGmfIlOJsu3eZWhj4jChho3h6c5hu2uKnvf3XMWty8grleuPVXaso84b4kim5UnPRAGoyukNY4pBQurMakmz9+xmexOotmb5qFo+zxMFWJUFANL4FkhRE+l3uzky1ItU+SL3oAKD4ewSqFjsRWu+MwHcrrSSn3uXq5lvZybPkn7bbu2NFnIQJTNk3CzJLcnESPyFhNRnusZhzXaFihH6QktgrhaAQjhvZYVsYx9kDuMeDD+JTNpRr4SchHuuiYLGKQ4U9Y1grxk6n7+2kNSeRY1dRg3/51RH7A4YSpZR6xKIHCLtJzIGIk5Y46OOfD479IzNz8dMVjH3FuBBewWARDoQi2k24g5ZEs7XycScVkKpy9eAFAVMniIUBHI7rxBcbsMgj7L2LdhoDHNFM0rZFfW360GSB5pXV3/p+6poYtBYoAfRbTDdpnvulazla1XYNeMIs6R3XeEubvetrwm2rfsPO/zo+s7/0dKEQm0CNvzUH+9TRp124+RueCiLP7bebNwWvhBp5udgj7I4Cd8QEwsfiTEOnlxAO0HUrEYPq3HyJjqUWXC4Ox/F9/10FWFrvq4Cx6yWsKOgVnr6/YKj6p89QtEGLTlkE/LmZZNblLHzjYGj1oVdDrxeGXtX77djhNTWZkX/L5oODLefnucFfbb7qIH4S91PirSFzuzcVcRaNVX/CFvZtwrZu3MAaAFKcA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(336012)(47076005)(70206006)(53546011)(4326008)(2906002)(356005)(8936002)(70586007)(83380400001)(81166007)(86362001)(82310400003)(26005)(2616005)(6506007)(186003)(6862004)(107886003)(5660300002)(6636002)(6486002)(6512007)(508600001)(37006003)(36756003)(54906003)(36860700001)(316002)(8676002)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:28:45.7477
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: cce10795-2a4a-45ff-52ff-08d9831ac59f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT017.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3658

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> On Arm, the initial plan is to only support GICv3 ITS which doesn't
> require us to manage the MSIs because the HW will protect against
> spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
> for ARM.
>=20
> No functional change intended.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3: none=20
> Change in v2: Fixed minor comments
> ---
> xen/arch/x86/Kconfig             |  1 +
> xen/drivers/passthrough/Makefile |  1 +
> xen/drivers/passthrough/msi.c    | 83 ++++++++++++++++++++++++++++++++
> xen/drivers/passthrough/pci.c    | 54 +++++----------------
> xen/drivers/pci/Kconfig          |  4 ++
> xen/include/xen/msi.h            | 43 +++++++++++++++++
> xen/xsm/flask/hooks.c            |  8 +--
> 7 files changed, 149 insertions(+), 45 deletions(-)
> create mode 100644 xen/drivers/passthrough/msi.c
> create mode 100644 xen/include/xen/msi.h
>=20
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 1f83518ee0..b4abfca46f 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -20,6 +20,7 @@ config X86
> 	select HAS_NS16550
> 	select HAS_PASSTHROUGH
> 	select HAS_PCI
> +	select HAS_PCI_MSI
> 	select HAS_PDX
> 	select HAS_SCHED_GRANULARITY
> 	select HAS_UBSAN
> diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/M=
akefile
> index 445690e3e5..a5efa22714 100644
> --- a/xen/drivers/passthrough/Makefile
> +++ b/xen/drivers/passthrough/Makefile
> @@ -7,3 +7,4 @@ obj-y +=3D iommu.o
> obj-$(CONFIG_HAS_PCI) +=3D pci.o
> obj-$(CONFIG_HAS_DEVICE_TREE) +=3D device_tree.o
> obj-$(CONFIG_HAS_PCI) +=3D ats.o
> +obj-$(CONFIG_HAS_PCI_MSI) +=3D msi.o
> diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.=
c
> new file mode 100644
> index 0000000000..ce1a450f6f
> --- /dev/null
> +++ b/xen/drivers/passthrough/msi.c
> @@ -0,0 +1,83 @@
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <xen/msi.h>
> +#include <asm/hvm/io.h>
> +
> +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
> +{
> +    int rc;
> +
> +    if ( pdev->msix )
> +    {
> +        rc =3D pci_reset_msix_state(pdev);
> +        if ( rc )
> +            return rc;
> +        msixtbl_init(d);
> +    }
> +
> +    return 0;
> +}
> +
> +int pdev_msi_init(struct pci_dev *pdev)
> +{
> +    unsigned int pos;
> +
> +    INIT_LIST_HEAD(&pdev->msi_list);
> +
> +    pos =3D pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->dev=
fn),
> +                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSI);
> +    if ( pos )
> +    {
> +        uint16_t ctrl =3D pci_conf_read16(pdev->sbdf, msi_control_reg(po=
s));
> +
> +        pdev->msi_maxvec =3D multi_msi_capable(ctrl);
> +    }
> +
> +    pos =3D pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->dev=
fn),
> +                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSIX);
> +    if ( pos )
> +    {
> +        struct arch_msix *msix =3D xzalloc(struct arch_msix);
> +        uint16_t ctrl;
> +
> +        if ( !msix )
> +            return -ENOMEM;
> +
> +        spin_lock_init(&msix->table_lock);
> +
> +        ctrl =3D pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
> +        msix->nr_entries =3D msix_table_size(ctrl);
> +
> +        pdev->msix =3D msix;
> +    }
> +
> +    return 0;
> +}
> +
> +void pdev_msi_deinit(struct pci_dev *pdev)
> +{
> +    XFREE(pdev->msix);
> +}
> +
> +void pdev_dump_msi(const struct pci_dev *pdev)
> +{
> +    const struct msi_desc *msi;
> +
> +    if ( list_empty(&pdev->msi_list) )
> +        return;
> +
> +    printk(" - MSIs < ");
> +    list_for_each_entry ( msi, &pdev->msi_list, list )
> +        printk("%d ", msi->irq);
> +    printk(">");
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.=
c
> index fc4fa2e5c3..8996403161 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -32,8 +32,8 @@
> #include <xen/softirq.h>
> #include <xen/tasklet.h>
> #include <xen/vpci.h>
> +#include <xen/msi.h>
> #include <xsm/xsm.h>
> -#include <asm/msi.h>
> #include "ats.h"
>=20
> struct pci_seg {
> @@ -314,6 +314,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pse=
g, u8 bus, u8 devfn)
> {
>     struct pci_dev *pdev;
>     unsigned int pos;
> +    int rc;
>=20
>     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
>         if ( pdev->bus =3D=3D bus && pdev->devfn =3D=3D devfn )
> @@ -327,35 +328,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *p=
seg, u8 bus, u8 devfn)
>     *((u8*) &pdev->bus) =3D bus;
>     *((u8*) &pdev->devfn) =3D devfn;
>     pdev->domain =3D NULL;
> -    INIT_LIST_HEAD(&pdev->msi_list);
> -
> -    pos =3D pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC=
(devfn),
> -                              PCI_CAP_ID_MSI);
> -    if ( pos )
> -    {
> -        uint16_t ctrl =3D pci_conf_read16(pdev->sbdf, msi_control_reg(po=
s));
> -
> -        pdev->msi_maxvec =3D multi_msi_capable(ctrl);
> -    }
>=20
> -    pos =3D pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC=
(devfn),
> -                              PCI_CAP_ID_MSIX);
> -    if ( pos )
> +    rc =3D pdev_msi_init(pdev);
> +    if ( rc )
>     {
> -        struct arch_msix *msix =3D xzalloc(struct arch_msix);
> -        uint16_t ctrl;
> -
> -        if ( !msix )
> -        {
> -            xfree(pdev);
> -            return NULL;
> -        }
> -        spin_lock_init(&msix->table_lock);
> -
> -        ctrl =3D pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
> -        msix->nr_entries =3D msix_table_size(ctrl);
> -
> -        pdev->msix =3D msix;
> +        xfree(pdev);
> +        return NULL;
>     }
>=20
>     list_add(&pdev->alldevs_list, &pseg->alldevs_list);
> @@ -449,7 +427,7 @@ static void free_pdev(struct pci_seg *pseg, struct pc=
i_dev *pdev)
>     }
>=20
>     list_del(&pdev->alldevs_list);
> -    xfree(pdev->msix);
> +    pdev_msi_deinit(pdev);
>     xfree(pdev);
> }
>=20
> @@ -1271,18 +1249,16 @@ bool_t pcie_aer_get_firmware_first(const struct p=
ci_dev *pdev)
> static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
> {
>     struct pci_dev *pdev;
> -    struct msi_desc *msi;
>=20
>     printk("=3D=3D=3D=3D segment %04x =3D=3D=3D=3D\n", pseg->nr);
>=20
>     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
>     {
> -        printk("%pp - %pd - node %-3d - MSIs < ",
> +        printk("%pp - %pd - node %-3d",
>                &pdev->sbdf, pdev->domain,
>                (pdev->node !=3D NUMA_NO_NODE) ? pdev->node : -1);
> -        list_for_each_entry ( msi, &pdev->msi_list, list )
> -               printk("%d ", msi->irq);
> -        printk(">\n");
> +        pdev_dump_msi(pdev);
> +        printk("\n");
>     }
>=20
>     return 0;
> @@ -1422,13 +1398,9 @@ static int assign_device(struct domain *d, u16 seg=
, u8 bus, u8 devfn, u32 flag)
>     ASSERT(pdev && (pdev->domain =3D=3D hardware_domain ||
>                     pdev->domain =3D=3D dom_io));
>=20
> -    if ( pdev->msix )
> -    {
> -        rc =3D pci_reset_msix_state(pdev);
> -        if ( rc )
> -            goto done;
> -        msixtbl_init(d);
> -    }
> +    rc =3D pdev_msix_assign(d, pdev);
> +    if ( rc )
> +        goto done;
>=20
>     pdev->fault.count =3D 0;
>=20
> diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
> index 7da03fa13b..c6a7bc8007 100644
> --- a/xen/drivers/pci/Kconfig
> +++ b/xen/drivers/pci/Kconfig
> @@ -1,3 +1,7 @@
>=20
> config HAS_PCI
> 	bool
> +
> +config HAS_PCI_MSI
> +	bool
> +	depends on HAS_PCI
> diff --git a/xen/include/xen/msi.h b/xen/include/xen/msi.h
> new file mode 100644
> index 0000000000..c903d0050c
> --- /dev/null
> +++ b/xen/include/xen/msi.h
> @@ -0,0 +1,43 @@
> +#ifndef __XEN_MSI_H_
> +#define __XEN_MSI_H_
> +
> +#include <xen/pci.h>
> +
> +#ifdef CONFIG_HAS_PCI_MSI
> +
> +#include <asm/msi.h>
> +
> +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev);
> +int pdev_msi_init(struct pci_dev *pdev);
> +void pdev_msi_deinit(struct pci_dev *pdev);
> +void pdev_dump_msi(const struct pci_dev *pdev);
> +
> +#else /* !CONFIG_HAS_PCI_MSI */
> +
> +static inline int pdev_msix_assign(struct domain *d, struct pci_dev *pde=
v)
> +{
> +    return 0;
> +}
> +
> +static inline int pdev_msi_init(struct pci_dev *pdev)
> +{
> +    return 0;
> +}
> +
> +static inline void pdev_msi_deinit(struct pci_dev *pdev) {}
> +static inline void pci_cleanup_msi(struct pci_dev *pdev) {}
> +static inline void pdev_dump_msi(const struct pci_dev *pdev) {}
> +
> +#endif /* CONFIG_HAS_PCI_MSI */
> +
> +#endif /* __XEN_MSI_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
> index 25e87180b4..ea9a12bd71 100644
> --- a/xen/xsm/flask/hooks.c
> +++ b/xen/xsm/flask/hooks.c
> @@ -21,7 +21,7 @@
> #include <xen/guest_access.h>
> #include <xen/xenoprof.h>
> #include <xen/iommu.h>
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
> #include <asm/msi.h>
> #endif
> #include <public/xen.h>
> @@ -114,7 +114,7 @@ static int get_irq_sid(int irq, u32 *sid, struct avc_=
audit_data *ad)
>         }
>         return security_irq_sid(irq, sid);
>     }
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>     {
>         struct irq_desc *desc =3D irq_to_desc(irq);
>         if ( desc->msi_desc && desc->msi_desc->dev ) {
> @@ -874,7 +874,7 @@ static int flask_map_domain_pirq (struct domain *d)
> static int flask_map_domain_msi (struct domain *d, int irq, const void *d=
ata,
>                                  u32 *sid, struct avc_audit_data *ad)
> {
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>     const struct msi_info *msi =3D data;
>     u32 machine_bdf =3D (msi->seg << 16) | (msi->bus << 8) | msi->devfn;
>=20
> @@ -940,7 +940,7 @@ static int flask_unmap_domain_pirq (struct domain *d)
> static int flask_unmap_domain_msi (struct domain *d, int irq, const void =
*data,
>                                    u32 *sid, struct avc_audit_data *ad)
> {
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>     const struct pci_dev *pdev =3D data;
>     u32 machine_bdf =3D (pdev->seg << 16) | (pdev->bus << 8) | pdev->devf=
n;
>=20
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:28:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:28:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198646.352236 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1A-0004ZN-1P; Wed, 29 Sep 2021 07:28:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198646.352236; Wed, 29 Sep 2021 07:28:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU19-0004ZG-U9; Wed, 29 Sep 2021 07:28:39 +0000
Received: by outflank-mailman (input) for mailman id 198646;
 Wed, 29 Sep 2021 07:28:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVU18-0004Z4-L6
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:28:38 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 4ec6ee1d-6973-4e3e-947d-c9f6be05151a;
 Wed, 29 Sep 2021 07:28:37 +0000 (UTC)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2055.outbound.protection.outlook.com [104.47.9.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-VrPEJvPBNCOUaM56QZpXUw-1; Wed, 29 Sep 2021 09:28:32 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4607.eurprd04.prod.outlook.com (2603:10a6:803:71::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 07:28:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 07:28:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR05CA0090.eurprd05.prod.outlook.com (2603:10a6:207:1::16) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:28:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ec6ee1d-6973-4e3e-947d-c9f6be05151a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632900514;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=bRLUKn+ePbvuD7VZ655+hKZQqJV+M970FDPCaA9nB3k=;
	b=kDPaDwzkqRst4ST64wnkFiXb5MFbcg2OpGuPLpzET/3MZ5hxE58JVGiagGCQeac9ftIYQQ
	G2gxVQ4UyPSNbLoGDFL+PZXy8kJqzSujkgqzTh0HRGAiVJ0JxCEMoTnsxx6u4kS44BQAR4
	HJu0FjcfQKfaZjFsPPtI33ng6Ce9oDA=
X-MC-Unique: VrPEJvPBNCOUaM56QZpXUw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UdR0GRmXwq9YQtF/hOUARRFdl18IYEeCXYs+zuBvgcUTx0FCDz8viXcjvzWq3fDL36Kk2mDAUIa14quavI3n5V3GGZpmztXz+ZuG32qip5Yxgpdj9Haw5QGr4CiCVkksnQMy5N74bJXyiGT5EaO7zYz4SotQEk+wKDX6EODF3L5RJuR+WLtO5jPdBttgB63ypIVfJkYUR1yeMDVN5WH7ffqNgia5YaEYoqcENtJ3QC1spRTAFUEFh8t5M+FE8kD7EjkhQ+bxnGT/r5c+1pMHZGZMSfUZtgp4IpnMdVnC9oISCCs2/vwtV2QYcYem9+sMDl7uj3eqzu+dTR+jQf1hxw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=bRLUKn+ePbvuD7VZ655+hKZQqJV+M970FDPCaA9nB3k=;
 b=AeS3x7aQ6zf4vJEultWNKNvuLFRyPxCH1bfmPm4JU8FChFXXmpI/LfPExiwxjkfJRPnDxpeCyY9Lb/H4SPgC6jwZMNnP8XLrB5lEjV5F1rEggRB9Sh0bbwKZtjC7wQHDkNeJr8parRRWJXr+Ed77CvA+wNE3qsNxMK+N81idx3uh7cuAlrJG+sFA+JxGkpiCRzUFNoD7m7j9x2eZkIwZ+W0nIWbrs2DACdsxO3z1vtuHVGHWUE6xPTeJ8gc7Tg/GQJxp1HebOp9/Q+nspA9omnnkaXJmXZAMeAjNslVCJMRCcgTF4K/mpq4FnEKgclcbdbEkx7FZJ1IIsHSrGlZVzg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
 <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
 <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ae73b283-fca4-de3f-d654-bde94405e05f@suse.com>
Date: Wed, 29 Sep 2021 09:28:26 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR05CA0090.eurprd05.prod.outlook.com
 (2603:10a6:207:1::16) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fa44c12b-65b2-48b2-60ad-08d9831abb60
X-MS-TrafficTypeDiagnostic: VI1PR04MB4607:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4607872D28E4A14937649260B3A99@VI1PR04MB4607.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vGNbndID5ocMMvInaIVxkyEOO8tfOT8ceutK7ahDrmB7zQnoCDiSe0xerqKljEi8e0TfakhOaT40hqH0gDPE/QsNuoIrmftI8PBJP1aPCEiUFc4eJVVCnIF4sQoVOXj0z22WBvayf/3qLMSWRe1J86VJJ2G6wY1i2kmXsXjoJhppw3Wts/ELVTBNOop3lLk3usXFgifaN2t6oH9q4z1MOR6c+tx5qTWNRXDwph3ksL9xr8wyxYWJRAm4JEYeY0Rbg5nmHMduFB8QxVT8+5RqCVHIwNEmdHX9AQDP+ip8WWoZDWlsWQlTqHWlXnjkV9qVAMVQqxADsx7h/EhklpE4lwHvHnhkbPDFnuIizNOPYvYWE98bJnXlWuR9HahWGkOfZ2iIJGznlByqOd1fIQjfB5X79GBDRgVjB2G2b7mup+oLpbHa7EEKG4Izpw2cm7WiiDVNdTWRSRWntaMk86V+d5HSp37WKPPqlQQcec567EIf/h4ZMIcesQroiWr9Iq8otz1y8Xrh/z9FczCF5CXJBC8tsiw9OljChtnkEgJ+ycf3QDvUlDyw38js+R4MDn7dWH/UZZv1DNB/JCIEIbZi0bdpV2pJYjpe3XieV0q72gu7hlVIaQWA7/YlXX1H7uSkOMk/CExyqKbXzy+ng2SmUY1nbkhtuQwE+um2/6kiPCYY9lHKdpufoeXtKNtxcnPEubOZVqLb6fxLAM9bDvKDZcE7AAru2xkH7ANs8Hlhq1Didq3Muznz7V3/dOO/llSa/rQSTwfoWZNFyEg6qAjXp/vXMDGoZoHt5HhFriMb/e8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(54906003)(6862004)(316002)(4326008)(38100700002)(36756003)(26005)(186003)(83380400001)(956004)(2616005)(37006003)(6486002)(66556008)(86362001)(53546011)(5660300002)(31696002)(31686004)(66476007)(8936002)(6636002)(8676002)(66946007)(508600001)(2906002)(52103002)(158003001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aHBiVlFIaE5RM3UxSEpPS2hLTjYwQ0g5N1lNSEhBMkdPR3djU0Zoa3ZFYkdp?=
 =?utf-8?B?R3BBMWRmczJxbTZvc2t0ZUdOdEJHakRwNmtpdmZMcnN4bkFzNnA0UFMwa1hS?=
 =?utf-8?B?WERuRnZzYVQ5UklEOEJNdERhZUFBKzBRYjIwNWExcTZuamNZZlVjbU5GTXY4?=
 =?utf-8?B?bGQ2bVh0Rm5taUZLYW1oTU5oek5QczFvUVQ4QU8xMmZySTZsZXNpbEUxNmdz?=
 =?utf-8?B?Z054TGVPc2t3OEtaS01vUXZqYkFDQnQ4VVBScDhPOG00dTkwRXJlS1VSTjNP?=
 =?utf-8?B?MFpaZWxQaWJFcjkzUUlZZE5FV29PWTRoT2czTWQ4a3g1YnNhdGNaWUVmYUtu?=
 =?utf-8?B?WDkzbWtjRlgzWGg5YXpsbjJJcmhKYnVGUnhpcDJVbTFIVjVGMWJycm5xNklD?=
 =?utf-8?B?SEdlK3kvVDIzTWU4dlVybmJVRys2Z1BqWTBUWmVPYk9tc3Y2QkY3QmxmeTZX?=
 =?utf-8?B?TTRRRTJMWkx1blFRVnc0bUJMaHF3ZGFNYWNjeXdvbFRuMVQ2SVdEWEhveXpL?=
 =?utf-8?B?US9rTThQV0FjQ0pOSG9wQkJCZndYN01pL0hsYTlaa28rOG1EYjJaWVZ0YXJq?=
 =?utf-8?B?ajNvSjM2Q2tabkgvZHhFRW1pQTBQODB0ampuVWVZeWdwK2ZTRWpsTUw5dEZu?=
 =?utf-8?B?MC9YcU9QWTFDLzMvcFpyTXc0Q1gzNG9YdjAxUElVRkM0WndWSWhNaFlzVVF4?=
 =?utf-8?B?RXRSYWdaMEh1MmcrUDZKL0tCTHBEamJqSzdPZFl3MFE0cmRLZ25JVGZ6M0Vr?=
 =?utf-8?B?OGJDcHBZZGhlNXIrZzkvQUxxUVRzNmFvVGd5QWMvMUxmYTRCZktKU1VRbER5?=
 =?utf-8?B?SEhCbGp6aGozRU1Jd2k0ZXlkcjh5WU1XaHg4VjBoTkJ0c2N0cHRiU00wM0FZ?=
 =?utf-8?B?cWpka0poR1A1VkxYR3FlMnZNb1AxNnUvOXh0cldjYkJqdUlBWkVMK3Zhdy9o?=
 =?utf-8?B?M2owb2FObHhiejVDbjhGSU5LaGx4SysxQzJiYnNqd2F2ZUFnUzBMQllVTWZV?=
 =?utf-8?B?amQ2d2YxY2hEbW9nWi9ZaWxsQ1pLYmZnVjVxNWhMeUI0cVFGK2svdlRlTStX?=
 =?utf-8?B?bmtWSEhlbUVOY2N4TjRDcVh4UEttRXFsdTAxc3A5cjF0NnVGclE4OUY4RzRs?=
 =?utf-8?B?MTl0WGVlekxUSGxaa1lRemYvd2JIamhwRGcxeUVucEkwSWFkNzJSR2Z6SEh3?=
 =?utf-8?B?WEl0MTROV2tWK3pFT2xnSW1tYjVibWloYzFTQSs1Q2NsZGdmbTZJSnEzK1ky?=
 =?utf-8?B?SElLSnNCcFdTQkI4WG5vLzF2VXZFTlJYajRlSGEvaTFhRy8vRVk3bDZpalQ2?=
 =?utf-8?B?VjVYeUxaOENIS0lxTXFKRkRxN05YbTBNblc1QlNZZUpTZ2h0dzZZV25JZjZo?=
 =?utf-8?B?QXBYNFVJc3NhaEppditGcTRjV1YydFlxRTBQYlJHemR0S0Z4UnV0QjJMSUtG?=
 =?utf-8?B?aFRDMHRFK2tQdTRkVUFTcGRuRjRsQ1VsRDl3dkRlMmtRSmwrWVhGOU1reDZO?=
 =?utf-8?B?U3FhZ1ZYYkN6RmV2OU0yeCtGdTI0RVplNXlnY3BsaHNPd3lVQklybC83R1Vu?=
 =?utf-8?B?VE5tSU4xazZTT1NnbkFpMExIN04vMUhPRTlPZWRqa2VzMnVIdU9uS0htWjBU?=
 =?utf-8?B?a2N2ek5nQXl5RUdyb0JBa2hFUHdnUkNKYVI4Z3ZOL1dKL2F6emRiZTY3bW9N?=
 =?utf-8?B?eUhQV1MwWXRjTFlvYU8vYnJJZzVhUDJZQWFjRlpoeFpwd0pQQnlaQ3AxV0tp?=
 =?utf-8?Q?OcEW7L/gSBUkJ3iHQ1dLWjooQahiZH1vZxIxOp8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fa44c12b-65b2-48b2-60ad-08d9831abb60
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:28:28.9527
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Hd1EzYtFh7472YvRbJeUMaojYn1G/CppMPSZe0RrxRy9kxGjVXt2YrM4doVC4qMMFx4Vu7sumNfEUwX2yuRzQA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4607

On 29.09.2021 07:45, Juergen Gross wrote:
> On 23.09.21 17:31, Jan Beulich wrote:
>> On 23.09.2021 17:25, Juergen Gross wrote:
>>> On 23.09.21 17:19, Jan Beulich wrote:
>>>> On 23.09.2021 17:15, Juergen Gross wrote:
>>>>> On 23.09.21 17:10, Jan Beulich wrote:
>>>>>> On 23.09.2021 16:59, Juergen Gross wrote:
>>>>>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>>>>>> This was effectively lost while dropping PVHv1 code. Move the function
>>>>>>>> and arrange for it to be called the same way as done in PV mode. Clearly
>>>>>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
>>>>>>>> check that was recently removed, as that's a PV-only feature.
>>>>>>>>
>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>
>>>>>>>> --- a/arch/x86/xen/enlighten.c
>>>>>>>> +++ b/arch/x86/xen/enlighten.c
>>>>>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>>>>>      	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
>>>>>>>>      }
>>>>>>>>      
>>>>>>>> +void __init xen_banner(void)
>>>>>>>> +{
>>>>>>>> +	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
>>>>>>>> +	struct xen_extraversion extra;
>>>>>>>
>>>>>>> Please add a blank line here.
>>>>>>
>>>>>> Oops.
>>>>>>
>>>>>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>>>>>> +
>>>>>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
>>>>>>>
>>>>>>> Is this correct? I don't think the kernel needs to be paravirtualized
>>>>>>> with PVH (at least not to the same extend as for PV).
>>>>>>
>>>>>> What else do you suggest the message to say? Simply drop
>>>>>> "paravirtualized"? To some extent it is applicable imo, further
>>>>>> qualified by pv_info.name. And that's how it apparently was with
>>>>>> PVHv1.
>>>>>
>>>>> The string could be selected depending on CONFIG_XEN_PV.
>>>>
>>>> Hmm, now I'm confused: Doesn't this setting control whether the kernel
>>>> can run in PV mode? If so, that functionality being present should have
>>>> no effect on the functionality of the kernel when running in PVH mode.
>>>> So what you suggest would end up in misleading information imo.
>>>
>>> Hmm, yes, I mixed "paravirtualized" with "capable to run
>>> paravirtualized".
>>>
>>> So the string should depend on xen_pv_domain().
>>
>> But that's already expressed by pv_info.name then being "Xen PV".
> 
> True. Okay, I'm fine with just dropping "paravirtualized".

Done. Do you want me to also make pr_info() vs printk(KERN_INFO ...)
consistent in the function at this occasion? If so - which of the two?

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:29:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:29:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198657.352257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1k-0005es-PZ; Wed, 29 Sep 2021 07:29:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198657.352257; Wed, 29 Sep 2021 07:29:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1k-0005dn-MR; Wed, 29 Sep 2021 07:29:16 +0000
Received: by outflank-mailman (input) for mailman id 198657;
 Wed, 29 Sep 2021 07:29:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVU1j-0005a5-Rs
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:29:15 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cb613999-5047-4698-aa39-578e7a473992;
 Wed, 29 Sep 2021 07:29:15 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 0E194224EC;
 Wed, 29 Sep 2021 07:29:14 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DE0F713A81;
 Wed, 29 Sep 2021 07:29:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id GzfrNMkVVGG9bQAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 07:29:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb613999-5047-4698-aa39-578e7a473992
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632900554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=4h7YMX3LLHXl2S8vcVGzM/WIVpMM9pP4B8+ifC55tfk=;
	b=qsDMJVPQ25FCCCnLnyvSFuiyu4QNd+rP2d1OfjdiFc0qQK+eyWF5yoIExoc9+/PFqxYTKC
	F1KxRE/P/qnLfhacw+uq9XS0HPVhvZKtZVjbu2YswGVKS65ukEeEmSGwG+mW/gQNjUx9/D
	9QWabMtI7eIUL/zE3jzwu4gpbmZYAFc=
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
 <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
 <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
 <ae73b283-fca4-de3f-d654-bde94405e05f@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <7a8e5cdb-a871-0e86-86b6-66c30201b946@suse.com>
Date: Wed, 29 Sep 2021 09:29:13 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <ae73b283-fca4-de3f-d654-bde94405e05f@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="MMQcAhhCjtW07P3U4QBPLy4Z2cS5VX1JL"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--MMQcAhhCjtW07P3U4QBPLy4Z2cS5VX1JL
Content-Type: multipart/mixed; boundary="UaLZCwVVKZGxpPOyBu4Wt0fV4LDwM5FmX";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <7a8e5cdb-a871-0e86-86b6-66c30201b946@suse.com>
Subject: Re: [PATCH 7/9] xen/x86: hook up xen_banner() also for PVH
References: <4efa804e-3250-227f-00c7-347581366cd4@suse.com>
 <5af11027-cf9d-cf78-9f48-b2ce2edd6e62@suse.com>
 <2ded8c58-b9c3-89dc-6883-1794d1c4126a@suse.com>
 <d9b6e98c-4e75-73f3-1e6d-42df300cfd49@suse.com>
 <89cb2d39-2bfc-dccf-8e92-39e4e952750d@suse.com>
 <f613a83c-2b29-23eb-ca78-a8a75a67f651@suse.com>
 <e095eec1-f35b-ca35-9ad1-54c817e61408@suse.com>
 <1bcb3b62-8327-7da8-abdb-65ee965714a7@suse.com>
 <86a4d837-5603-39d0-4643-c31597d524df@suse.com>
 <ae73b283-fca4-de3f-d654-bde94405e05f@suse.com>
In-Reply-To: <ae73b283-fca4-de3f-d654-bde94405e05f@suse.com>

--UaLZCwVVKZGxpPOyBu4Wt0fV4LDwM5FmX
Content-Type: multipart/mixed;
 boundary="------------E692F45CC8D35ECDC1CD735D"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------E692F45CC8D35ECDC1CD735D
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 29.09.21 09:28, Jan Beulich wrote:
> On 29.09.2021 07:45, Juergen Gross wrote:
>> On 23.09.21 17:31, Jan Beulich wrote:
>>> On 23.09.2021 17:25, Juergen Gross wrote:
>>>> On 23.09.21 17:19, Jan Beulich wrote:
>>>>> On 23.09.2021 17:15, Juergen Gross wrote:
>>>>>> On 23.09.21 17:10, Jan Beulich wrote:
>>>>>>> On 23.09.2021 16:59, Juergen Gross wrote:
>>>>>>>> On 07.09.21 12:11, Jan Beulich wrote:
>>>>>>>>> This was effectively lost while dropping PVHv1 code. Move the f=
unction
>>>>>>>>> and arrange for it to be called the same way as done in PV mode=
=2E Clearly
>>>>>>>>> this then needs re-introducing the XENFEAT_mmu_pt_update_preser=
ve_ad
>>>>>>>>> check that was recently removed, as that's a PV-only feature.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>>>
>>>>>>>>> --- a/arch/x86/xen/enlighten.c
>>>>>>>>> +++ b/arch/x86/xen/enlighten.c
>>>>>>>>> @@ -261,6 +261,18 @@ int xen_vcpu_setup(int cpu)
>>>>>>>>>       	return ((per_cpu(xen_vcpu, cpu) =3D=3D NULL) ? -ENODEV :=
 0);
>>>>>>>>>       }
>>>>>>>>>      =20
>>>>>>>>> +void __init xen_banner(void)
>>>>>>>>> +{
>>>>>>>>> +	unsigned version =3D HYPERVISOR_xen_version(XENVER_version, N=
ULL);
>>>>>>>>> +	struct xen_extraversion extra;
>>>>>>>>
>>>>>>>> Please add a blank line here.
>>>>>>>
>>>>>>> Oops.
>>>>>>>
>>>>>>>>> +	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
>>>>>>>>> +
>>>>>>>>> +	pr_info("Booting paravirtualized kernel on %s\n", pv_info.nam=
e);
>>>>>>>>
>>>>>>>> Is this correct? I don't think the kernel needs to be paravirtua=
lized
>>>>>>>> with PVH (at least not to the same extend as for PV).
>>>>>>>
>>>>>>> What else do you suggest the message to say? Simply drop
>>>>>>> "paravirtualized"? To some extent it is applicable imo, further
>>>>>>> qualified by pv_info.name. And that's how it apparently was with
>>>>>>> PVHv1.
>>>>>>
>>>>>> The string could be selected depending on CONFIG_XEN_PV.
>>>>>
>>>>> Hmm, now I'm confused: Doesn't this setting control whether the ker=
nel
>>>>> can run in PV mode? If so, that functionality being present should =
have
>>>>> no effect on the functionality of the kernel when running in PVH mo=
de.
>>>>> So what you suggest would end up in misleading information imo.
>>>>
>>>> Hmm, yes, I mixed "paravirtualized" with "capable to run
>>>> paravirtualized".
>>>>
>>>> So the string should depend on xen_pv_domain().
>>>
>>> But that's already expressed by pv_info.name then being "Xen PV".
>>
>> True. Okay, I'm fine with just dropping "paravirtualized".
>=20
> Done. Do you want me to also make pr_info() vs printk(KERN_INFO ...)
> consistent in the function at this occasion? If so - which of the two?

pr_info(), please.


Juergen


--------------E692F45CC8D35ECDC1CD735D
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------E692F45CC8D35ECDC1CD735D--

--UaLZCwVVKZGxpPOyBu4Wt0fV4LDwM5FmX--

--MMQcAhhCjtW07P3U4QBPLy4Z2cS5VX1JL
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFUFckFAwAAAAAACgkQsN6d1ii/Ey+z
DQf/X8d9dogA6BM6662Ei5fM97QDmK4Q5eLr+oLP7OniSOhbVemUfgyUFdQoI5tx5qFnADmQrqsu
wSwr+0+ci7F1C6mUVbjQxeYWs0n1nbY5xS4bBVoNjBk+KDM7lhZhpghA3jbUHiZsK2nJlhooObCN
yntYic5yPi7YJ0vkdfCOdM/YWhP9OyfYBBmussH/OD1sz64ViL04T+qgHD3pZU7nc86tmvJZ0RZs
RgOxDEeqT03Q1rx0B0/ffKB7NyTvvw8J3u27DoEcAQYcvoYfdZ+YPwJvsXWJjWG4VDP3+P0s1Xi6
EEiPxPa3Nr0ixMWI5z5hV3Nik0We/uW9OAaK8agttQ==
=8Tbb
-----END PGP SIGNATURE-----

--MMQcAhhCjtW07P3U4QBPLy4Z2cS5VX1JL--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:29:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:29:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198658.352269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1r-0005yK-2Q; Wed, 29 Sep 2021 07:29:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198658.352269; Wed, 29 Sep 2021 07:29:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU1q-0005y9-VF; Wed, 29 Sep 2021 07:29:22 +0000
Received: by outflank-mailman (input) for mailman id 198658;
 Wed, 29 Sep 2021 07:29:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU1p-0005x4-5G
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:29:21 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.1.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 902985fd-efbb-4d6a-a0a4-7b00f1dd974f;
 Wed, 29 Sep 2021 07:29:18 +0000 (UTC)
Received: from AM6P192CA0034.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::47)
 by DB8PR08MB5276.eurprd08.prod.outlook.com (2603:10a6:10:bf::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Wed, 29 Sep
 2021 07:29:16 +0000
Received: from VE1EUR03FT021.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:83:cafe::f6) by AM6P192CA0034.outlook.office365.com
 (2603:10a6:209:83::47) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 07:29:16 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT021.mail.protection.outlook.com (10.152.18.117) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:29:15 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Wed, 29 Sep 2021 07:29:13 +0000
Received: from 01c892c53fc1.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 FF670FE8-1CE6-4F24-9F8C-135413F8525F.1; 
 Wed, 29 Sep 2021 07:29:00 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 01c892c53fc1.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:29:00 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB5901.eurprd08.prod.outlook.com (2603:10a6:102:ed::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:28:55 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:28:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 902985fd-efbb-4d6a-a0a4-7b00f1dd974f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JgvSnB62SJlV8IRf/y5r+hJoxiTMPzmggENzWdc/40s=;
 b=b9RooUGf+1saYThjIEJ6ciDcIRPbPfZNeh9y6mggV4OsSqENgB2r7agLqEtPfqVnzlKsJFWls9yTkLL/xSXiySWqeHVIjebr53yoCIuOXSUiMoXHVFjiig/Yhu4fDKQ51axBrXyOyNEWHfoYM32FzmyrQZAwd9Ii1Up1U1KrUh4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 018ebaf0476903df
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UD13/0dwn6Owem4k2joJGEC8AzuQE8/b57zU1I2vLtjkCQxFM7qwyjwbMFIwkOYGxcm3MLRU17SoZpc+p/KitV3OjIByJFBip30SeYrLrCQ6Pcbb4XDXV2EhORsPfBZAdhlBXxajGofWL7mXbQJqGA3ArzWQhCKc19eE7PV5NV0hEjX1XhlTLAkH1TxVPNZIIbwn1smDGDACvBvaGlLY2nJqv4bwhSETH3fP5rOTkuHFpQDUEazKmxlJw+g2LEFBSt6eczDt01wcEadsmbtYzLTSY6tvJn97jcb4SkMt3QkcDW0bXLuNVq8XvtoatnLKNUCyVG/aRAnrHqUkd4+Z8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JgvSnB62SJlV8IRf/y5r+hJoxiTMPzmggENzWdc/40s=;
 b=U76UOX/8Rja9aYjySukR9WKeU+7K+5aae44TRKNdPpPA66ui7TvhrHPjduehK7oZzx7cslPvBG/FqMkzB6Rib9pwceiu/I9CqzRlqLl7MTEfl0LIY+9B7s/qWUdUYfgZHyfAgczYAm1rlExq0MlC1XROLk3MouTmOhx2K3sh+vpSub0ZR/BUlx6GSm69O2m0eJUnuql55lL3IUm1YTn+CylTzDpbETK7AbNPYr0MnQHHHIpvC5e3jIo7B6uA2neWkDmg/rLo2tjEc3Ch2SdrdROZL9BPAPks4YrZA/Atlf/uh7I7PMfYvVeDIqsM7BRewjGQeYr2v0jZc7C9fg67AA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JgvSnB62SJlV8IRf/y5r+hJoxiTMPzmggENzWdc/40s=;
 b=b9RooUGf+1saYThjIEJ6ciDcIRPbPfZNeh9y6mggV4OsSqENgB2r7agLqEtPfqVnzlKsJFWls9yTkLL/xSXiySWqeHVIjebr53yoCIuOXSUiMoXHVFjiig/Yhu4fDKQ51axBrXyOyNEWHfoYM32FzmyrQZAwd9Ii1Up1U1KrUh4=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 02/17] xen/arm: pci: Add stubs to allow selecting
 HAS_PCI
Thread-Topic: [PATCH v3 02/17] xen/arm: pci: Add stubs to allow selecting
 HAS_PCI
Thread-Index: AQHXtJV0mB13hzAW+EyNCJ6p2ZFT7au6niUA
Date: Wed, 29 Sep 2021 07:28:55 +0000
Message-ID: <D938C740-9BBD-4D01-B26F-F4112A186983@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <c752df37298c75f2f1872565d2272fc59063547f.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <c752df37298c75f2f1872565d2272fc59063547f.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: abb1e648-d5cd-4262-0806-08d9831ad790
x-ms-traffictypediagnostic: PA4PR08MB5901:|DB8PR08MB5276:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB5276D2AB61A19CF30BB2EA3D9DA99@DB8PR08MB5276.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:317;OLM:317;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 A98c7V5YK+3SI/2HUmqgH4IH/KmnUKBBM9REkFpLXl9jyoWit/kcpZdVScX1rRLkSQ+Aa853y5/y/m8xv6x3JBDGAaJn05oAeuuTg24qrePkFwk1ZO6Ki0N+gE1O+EwSmuAjL8TTXlwS95Jr3dfsC3+nni/UzJDZqw7n9g51AtzxT30cJPYfBpX7vlUQ09NhLbWqWWPSW/ePGqCMeaJvNkxS/6xVhLo9kZswVIdyAfzVytHhGXM6gYY3tjHtZsf4mp1E7kdbrs7xmmRdzttuZvAOJBv3gg+6cJzlm3/Y+zjY4qO9/v9UhHs2JvlUHXn9DMY7RoYbqZzi0E8PMOR9J1FsEHxOkmr/FRoAy6crVCrQBB2rZhR14xz78K1g+gMPX1fXICXNAforLKIXcUlLL1LuSkmUIw431OB8+GOVKlHbS2uj3UOpJ8LUiFfHzQELu3HKhKeA6LdJygCro5zGnd3CeMIrgaY9t2UH0chgluEGUizyEzm882T1MVRLwW8kkJI7svugKoq7gO71IhLOZv01rH/zTjZDU933wU3bIeEW3ahrvdSSi5NBe4jKTboVxBGdW91/PaUkGjlfHb2dC6ljB4MDkpTP91VA64m6Negx/H9bto85I6P2GGGgzd++OK+yHaKfPB3oum2hW04WNGg3VtZA4jsVXv3BqCN+NqNkfBuJHzCd3JaOwX7/tDKxvEKtu0f8biTN5QNlxMJodeSRQtBcsJqQkJy2+66SzvSJ5zZVuRhQRRTtbaElWLiD2EB+4tYrSUaT7IdHg9hZ3qctS6dRxO0ni+UVVyy0DEo5GSnlSCs8Sz3sCTpL/flOKfEGbkKi4Eyuxe3H2YIh5WB7nDrOGd9cYqnqItdVlW0=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(8936002)(6862004)(8676002)(64756008)(36756003)(186003)(122000001)(38070700005)(71200400001)(66556008)(66446008)(66946007)(33656002)(91956017)(76116006)(2616005)(66476007)(83380400001)(38100700002)(86362001)(6486002)(37006003)(6636002)(26005)(6506007)(53546011)(54906003)(2906002)(316002)(508600001)(6512007)(5660300002)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <928CF9AA34A5114383BF295891501434@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5901
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	04b3c901-69f9-4296-cb4a-08d9831acb5f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cI7YXuSpvFt1Ed6Pz9YHp1qqnHrRdIGHFbtsRkA1YbQL82IRxfdthouFESFuY7W/i2d6sYLf3LHDM/OIC9tO3WAszyeU0eyhHWgVDX4vc5k141DzxzmFAIRTOD76TQSF7dJPoNvI+cFK3KrVkHReSQU36+4HeyITk7x3TwuXSp7yTw3HjmSzKwm0gBy0BaWEK88VakWJyBIOAXSTAP9heZ+BaL/SiWEUMsPfJGxDq3ICfpbQkmIauLfjEslnd2RAZ7z8OjJbYc3/25/3h+L/Y79Aao6PKmdyiHDyU8FdxGwazqXZtc1JYqnozmMrOoijGwqRcUJTzd8HWZmoSTPhC9g6KXkY7LNiHRhxP1bPic6RzU8oh/v7ma/U9E53VAbNnBNnEXTIH5vESYPlUZ/ngGPMjTTKijbKHc7QmAuxJxBl+jJdWmepWJzhC4kWoreGIKwrKnC8WQYgb17bw4NxPTK86tJevv8kFGH9wvYvxGchuH0PhV4yFpOamv7gIpYjXX+79C2MdEBC7gxeShgshzT+BprUKp8vCHeqhEhHj6bb5ayl6gRh51NkBowoBcjYJV2IYVxBuXHrd6fOjWZL5Of9PFZldJlB5FNNHLqEVuqqLSfRdLu0JlwMH3CBU9uqypO46TLQE8tZt6uGh3owJGdgXR+vaNhRwlHa8pzU8LRqCbL1uxDF0eRiVCyQR2VdHVT9W9SUntANhm7RKgDrs5Bj4mwbwgIFVkKj51YpHu7I2Irk6gnLHTNb/g0TLzYsxFHHs+OggvbUJZFCXRSHJuQijeUB32pQa6JAB+ZOPqJ0hWvJpZxGXdLCB+D+ROAl
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(26005)(4326008)(53546011)(6506007)(36860700001)(47076005)(6862004)(6636002)(5660300002)(8936002)(186003)(70206006)(70586007)(356005)(81166007)(8676002)(83380400001)(33656002)(2616005)(336012)(107886003)(36756003)(508600001)(6512007)(82310400003)(54906003)(86362001)(6486002)(2906002)(316002)(37006003)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:29:15.7515
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: abb1e648-d5cd-4262-0806-08d9831ad790
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5276

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> In a follow-up we will enable PCI support in Xen on Arm (i.e select
> HAS_PCI).
>=20
> The generic code expects the arch to implement a few functions:
> arch_iommu_use_permitted()
> arch_pci_clean_pirqs()
>=20
> Note that this is not yet sufficient to enable HAS_PCI and will be
> addressed in follow-ups.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Modify commit message.
> - Added Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> Change in v2:
> - Remove pci_conf_read*(..) dummy implementation
> - Add in code comment for arch_pci_clean_pirqs() and arch_iommu_use_permi=
tted()
> - Fixed minor comments.
> ---
> xen/arch/arm/Makefile               |  1 +
> xen/arch/arm/pci/Makefile           |  1 +
> xen/arch/arm/pci/pci.c              | 33 +++++++++++++++++++++++++++++
> xen/drivers/passthrough/arm/iommu.c |  9 ++++++++
> xen/include/asm-arm/pci.h           | 31 ++++++++++++++++++++++++---
> 5 files changed, 72 insertions(+), 3 deletions(-)
> create mode 100644 xen/arch/arm/pci/Makefile
> create mode 100644 xen/arch/arm/pci/pci.c
>=20
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 3d3b97b5b4..44d7cc81fa 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -2,6 +2,7 @@ obj-$(CONFIG_ARM_32) +=3D arm32/
> obj-$(CONFIG_ARM_64) +=3D arm64/
> obj-$(CONFIG_ARM_64) +=3D efi/
> obj-$(CONFIG_ACPI) +=3D acpi/
> +obj-$(CONFIG_HAS_PCI) +=3D pci/
> ifneq ($(CONFIG_NO_PLAT),y)
> obj-y +=3D platforms/
> endif
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> new file mode 100644
> index 0000000000..a98035df4c
> --- /dev/null
> +++ b/xen/arch/arm/pci/Makefile
> @@ -0,0 +1 @@
> +obj-y +=3D pci.o
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> new file mode 100644
> index 0000000000..a7a7bc3213
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci.c
> @@ -0,0 +1,33 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +
> +/*
> + * PIRQ event channels are not supported on Arm, so nothing to do.
> + */
> +int arch_pci_clean_pirqs(struct domain *d)
> +{
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthroug=
h/arm/iommu.c
> index db3b07a571..ee653a9c48 100644
> --- a/xen/drivers/passthrough/arm/iommu.c
> +++ b/xen/drivers/passthrough/arm/iommu.c
> @@ -135,3 +135,12 @@ void arch_iommu_domain_destroy(struct domain *d)
> void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
> {
> }
> +
> +/*
> + * Unlike x86, Arm doesn't support mem-sharing, mem-paging and log-dirty=
 (yet).
> + * So there is no restriction to use the IOMMU.
> + */
> +bool arch_iommu_use_permitted(const struct domain *d)
> +{
> +    return true;
> +}
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index de13359f65..7dd9eb3dba 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -1,7 +1,32 @@
> -#ifndef __X86_PCI_H__
> -#define __X86_PCI_H__
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
>=20
> +#ifndef __ARM_PCI_H__
> +#define __ARM_PCI_H__
> +
> +#ifdef CONFIG_HAS_PCI
> +
> +#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
> +
> +/* Arch pci dev struct */
> struct arch_pci_dev {
> +    struct device dev;
> };
>=20
> -#endif /* __X86_PCI_H__ */
> +#else   /*!CONFIG_HAS_PCI*/
> +
> +struct arch_pci_dev { };
> +
> +#endif  /*!CONFIG_HAS_PCI*/
> +#endif /* __ARM_PCI_H__ */
> --=20
> 2.17.1
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:29:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:29:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198665.352280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU27-0006ed-Cl; Wed, 29 Sep 2021 07:29:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198665.352280; Wed, 29 Sep 2021 07:29:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU27-0006eW-9d; Wed, 29 Sep 2021 07:29:39 +0000
Received: by outflank-mailman (input) for mailman id 198665;
 Wed, 29 Sep 2021 07:29:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU26-0006On-Py
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:29:38 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.88]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 88aa1a4e-2a46-4fc1-b404-ca3fd44d6b92;
 Wed, 29 Sep 2021 07:29:37 +0000 (UTC)
Received: from AS9PR06CA0238.eurprd06.prod.outlook.com (2603:10a6:20b:45e::8)
 by VI1PR08MB3373.eurprd08.prod.outlook.com (2603:10a6:803:7b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:29:30 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45e:cafe::70) by AS9PR06CA0238.outlook.office365.com
 (2603:10a6:20b:45e::8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 29 Sep 2021 07:29:30 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 29 Sep 2021 07:29:30 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Wed, 29 Sep 2021 07:29:28 +0000
Received: from 019ffb66a0b4.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 684F7AAA-E560-4CE6-AC52-8DACCAE843AD.1; 
 Wed, 29 Sep 2021 07:29:17 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 019ffb66a0b4.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:29:17 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB5901.eurprd08.prod.outlook.com (2603:10a6:102:ed::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:29:14 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:29:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88aa1a4e-2a46-4fc1-b404-ca3fd44d6b92
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9pHI4/IPOhsGJdfgmEp9Vt1QPCN0cKeOB0pTCObUcCA=;
 b=FaeuACWmYnlCC+W7fXE+40Lp3SMMTM62kABZ5cx6MLR2Yp1rtOYCCUg7Z6e2JLXPCutV2eUIVb66zzPM5gsiYCOyd0srQe07U07iVkaCFcpSOKC7nqQzBnoON4zfsmPyA+vmlFv1V+h3109UZyMkrua2Eby484woqgBmg0KXXzs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: dc4f99d7ef2ad4a8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RmmV5hfMI3sTO98+md0MUoWinu7Nil/iulIhHFxsfq2MddNTXknUKUq0mZOLFFPpYJanttkv+lvy61w5MTGkjo/ZWxnZFfTE8P2D3LlNAhs5oVKx57K/Q33sIAjKljay6Jg9t7AeiY04wg5yes0KbouKemD0JdAPv0qw4uD0Y6Y05uXyVeJo1itmW0cqYBmZ0MdQFcmiC53NtfYOS0tfEjJ6Vt1grAmjsLRPi4zLEw6CfbLk7Y8oMqrNNMPRYhMAi7PBsGIT05gmHHn+ZQjEw1ZCqqcKmdgQMvt1Q9kRI/IHYtyTHpmSrCK7CuzLwTbrB3IKxNeJMrOVzlJvHIPzFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=9pHI4/IPOhsGJdfgmEp9Vt1QPCN0cKeOB0pTCObUcCA=;
 b=Nwx1GFp8UVAM22rKkA+P/BgztUw5rM8dSm6X0tZWuZs6pGDcYYAF8NKOV/5P20ctN2VEACDrex9Rd5kRLMVCjUHR8fhR5XgMAatvrpxoq+BsbAcnyo9FSVPXoDwmKrrpoH9+QMEcMsrONG1Gb+ru3SSTOvC7kd8QhizgZlMrkhp3F4mjQ2DIZeY1iWhIDET1FlCARPWQK6pi75bpgHKcET/lyrjkuPvVJyv6chlAgGDTDAv0A91Gv+fWnnbnhLJEDLHy130gb7LSX3eNO2CZZ57Z0Gf2RNk4QgBLYYYztsKPnw+0qEbiGxsgmpcM1qoy7qYIOo2U9nGKiXSKu4Avng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9pHI4/IPOhsGJdfgmEp9Vt1QPCN0cKeOB0pTCObUcCA=;
 b=FaeuACWmYnlCC+W7fXE+40Lp3SMMTM62kABZ5cx6MLR2Yp1rtOYCCUg7Z6e2JLXPCutV2eUIVb66zzPM5gsiYCOyd0srQe07U07iVkaCFcpSOKC7nqQzBnoON4zfsmPyA+vmlFv1V+h3109UZyMkrua2Eby484woqgBmg0KXXzs=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andre
 Przywara <Andre.Przywara@arm.com>, Jan Beulich <jbeulich@suse.com>, Paul
 Durrant <paul@xen.org>
Subject: Re: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Topic: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Index: AQHXtJV6dalQJX/UXEa/GQQbUgx/4qu6njuA
Date: Wed, 29 Sep 2021 07:29:14 +0000
Message-ID: <28B10A6E-6224-41F8-A368-EB8A3503A717@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a44e69e1-ab0d-4396-c843-08d9831ae01b
x-ms-traffictypediagnostic: PA4PR08MB5901:|VI1PR08MB3373:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB3373A187764922DDD863482D9DA99@VI1PR08MB3373.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 iRk4KapPK6fdJrZmlPBmy9zQqY5uUm8fTw61FzOv9X9Bc5Tql5haTLqBuObzdBQlpGvPCb/9nMffogun4YULzamTCS8UWvKszzF1gvrH41eBxoxqt9iJ3xDNRvS+Z5Ikc66aEsySm0EcL1iXbYPJIHLBfdF9UtH9WiKMxeD2HQG41zFurR2ttzrmDarnTRq8E7BinbQqvIxne5jYbg7SbggVpk7zvG57iCCLjtILT1QmfDersLZGaLhYXMd7S5QdtA2D3mAmxGcfAogN1+yL4dEnEniX7UNMvfiWctspIpa9xr2h73nIbvYFdCS/W9Iv7sa2uyKz3EeY/7QNQsq2MVunH2oxObYJUag4Ab/G/FrFGZM/3T0gjLs+jcOm6w+kjA32mQHbflhLyVYqfsSzOLtYSBa3FS0ZsvZcG7NUU3lR4HsHhtS/yjPuj83LsijrAMDlBCVWc9IiHdTvI0sXDiIzreHIzQj/vQqdJbaRXPGFE/TKNkLeKJkqSmu4ZNcX1dVf6VPdrUQ0DOlZslcTPLe0p14RZtimjSSJZQKwjPWJpP9GgTyLZXpX3pcX/kvTnGXaUi0XdEnM37ao1+zmGgWD3FidJgw0BJ4+WaDw+cb0ru/xfobhQKCG5hwYhRpNnvBvw2HU134ksUUzXL1/wwUrHlJlb4KZSqlscVs1QCRPkC5gVpZexnq1ZM4ypguR7XMWcTGXP0QSWmihb/Y/A/XupR4bocbyAA49xNJTwIcZtc+Ihh5Batwt7bUgT0BM
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(8936002)(6862004)(8676002)(64756008)(36756003)(186003)(122000001)(38070700005)(71200400001)(66556008)(66446008)(66946007)(33656002)(91956017)(76116006)(2616005)(66476007)(83380400001)(38100700002)(86362001)(6486002)(37006003)(6636002)(26005)(6506007)(53546011)(54906003)(2906002)(316002)(508600001)(6512007)(5660300002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <3F44232FC889CF448D88103632C79D74@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5901
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	44b5b92c-676c-4896-a1c9-08d9831ad6c8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XXXXvEaMNZJBSa+6AL04Tb6NVb6PUM4Szy3sny0SE7bsqU95bR5XW2XRf4ZOUbXVtrq21d3uztdqSPHREQ0MWm0/6xo1REveJd645RLOJallp17OQSdJJOqDHXakuEdD+otB25ch1+ikQznnzp80ftnA4I8FOmAJc9/PZGe10a3XZdAhYNSrFoUVclHWCf5v9mmeAXlVRWBrbuQJpiPigDpZF1eP145wGvHgwc4RwB24y/TeoXn2iiHqJijD6OGVm8cq0OJ27pyP8JZnor+qAgM4G2I2LI/nWKzqPXO1YZDcg8ozGxMDeAPf+3GM9TgonGa42iiMmSL56tt+u8vVaLrVbqncwRxo7y6Wiz1h96P9zSob18S+QhC1IT1TYZnSauj4AuHJsmMI29f83WS+d+BMHEvyatF3lCvK7nlsqKBpJNylDnJ4L71M2t0cyPtQD7zCGY09FfzdAEQIA6KhDJA+vI36O2u9r2CagX0kO2oroABRsCHjo+PFVNcWCmE9Lz/uQfSLmg1G3xzuasTwWTvWrSUNXKcqrlX9VgL9cZ9xW6/sQkTSMBgsoM0sZhFe9ECqCrvDRYmsUa7gZXFsyyi+BtkKLYi4AbwQNDsA6Y+8tySrvW7nlxgPhAe+Figss++vmFYfsdpX8Y+3jG+VpGNPfaG3GxI5KQ40OpsVvwWFDRt5/o0b84iXGOakfPC3w/XgrwjvPHfZHvu24J+Wqw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(6636002)(2906002)(2616005)(82310400003)(33656002)(5660300002)(83380400001)(6862004)(356005)(508600001)(70206006)(70586007)(81166007)(336012)(316002)(54906003)(47076005)(8936002)(8676002)(6486002)(36860700001)(6512007)(186003)(26005)(53546011)(6506007)(4326008)(37006003)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:29:30.1331
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a44e69e1-ab0d-4396-c843-08d9831ae01b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3373

SGkgUmFodWwsDQoNCj4gT24gMjggU2VwIDIwMjEsIGF0IDE5OjE4LCBSYWh1bCBTaW5naCA8cmFo
dWwuc2luZ2hAYXJtLmNvbT4gd3JvdGU6DQo+IA0KPiBwcmVsaW5rLm86IEluIGZ1bmN0aW9uIGBw
Y2llX2Flcl9nZXRfZmlybXdhcmVfZmlyc3TigJk6DQo+IGRyaXZlcnMvcGFzc3Rocm91Z2gvcGNp
LmM6MTI1MTogdW5kZWZpbmVkIHJlZmVyZW5jZSB0byBgYXBlaV9oZXN0X3BhcnNlJw0KPiANCj4g
Q29tcGlsYXRpb24gZXJyb3IgaXMgb2JzZXJ2ZWQgd2hlbiBBQ1BJIGFuZCBIQVNfUENJIGlzIGVu
YWJsZWQgZm9yIEFSTQ0KPiBhcmNoaXRlY3R1cmUuIEFQRUkgbm90IHN1cHBvcnRlZCBvbiBBUk0g
eWV0IG1vdmUgdGhlIGNvZGUgdW5kZXINCj4gQ09ORklHX1g4NiBmbGFnIHRvIGdhdGUgdGhlIGNv
ZGUgZm9yIEFSTS4NCj4gDQo+IFNpZ25lZC1vZmYtYnk6IFJhaHVsIFNpbmdoIDxyYWh1bC5zaW5n
aEBhcm0uY29tPg0KPiBBY2tlZC1ieTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBr
ZXJuZWwub3JnPg0KUmV2aWV3ZWQtYnk6IEJlcnRyYW5kIE1hcnF1aXMgPGJlcnRyYW5kLm1hcnF1
aXNAYXJtLmNvbT4NCg0KQ2hlZXJzDQpCZXJ0cmFuZA0KDQo+IC0tLQ0KPiBDaGFuZ2UgaW4gdjM6
IEFkZGVkIEFja2VkLWJ5OiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5v
cmc+DQo+IENoYW5nZSBpbiB2MjogQWRkIGluIGNvZGUgY29tbWVudCAiQVBFSSBub3Qgc3VwcG9y
dGVkIG9uIEFSTSB5ZXQiDQo+IC0tLQ0KPiB4ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYyB8
IDMgKystDQo+IDEgZmlsZSBjaGFuZ2VkLCAyIGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkN
Cj4gDQo+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYyBiL3hlbi9k
cml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+IGluZGV4IDg5OTY0MDMxNjEuLmQ3NzRhNjE1NGUg
MTAwNjQ0DQo+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+ICsrKyBiL3hl
bi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+IEBAIC0xMTUwLDcgKzExNTAsOCBAQCB2b2lk
IF9faHdkb21faW5pdCBzZXR1cF9od2RvbV9wY2lfZGV2aWNlcygNCj4gICAgIHBjaWRldnNfdW5s
b2NrKCk7DQo+IH0NCj4gDQo+IC0jaWZkZWYgQ09ORklHX0FDUEkNCj4gKy8qIEFQRUkgbm90IHN1
cHBvcnRlZCBvbiBBUk0geWV0LiAqLw0KPiArI2lmIGRlZmluZWQoQ09ORklHX0FDUEkpICYmIGRl
ZmluZWQoQ09ORklHX1g4NikNCj4gI2luY2x1ZGUgPGFjcGkvYWNwaS5oPg0KPiAjaW5jbHVkZSA8
YWNwaS9hcGVpLmg+DQo+IA0KPiAtLSANCj4gMi4xNy4xDQo+IA0KDQo=


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:30:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:30:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198679.352290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU3C-0008KM-R4; Wed, 29 Sep 2021 07:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198679.352290; Wed, 29 Sep 2021 07:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU3C-0008KF-Np; Wed, 29 Sep 2021 07:30:46 +0000
Received: by outflank-mailman (input) for mailman id 198679;
 Wed, 29 Sep 2021 07:30:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU3B-0008K2-LS
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:30:45 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.42]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 997078b8-59c1-452b-904f-726b8c31a740;
 Wed, 29 Sep 2021 07:30:44 +0000 (UTC)
Received: from DB6P192CA0006.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::16) by
 DB8PR08MB5434.eurprd08.prod.outlook.com (2603:10a6:10:116::20) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15; Wed, 29 Sep 2021 07:30:38 +0000
Received: from DB5EUR03FT009.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:b8:cafe::25) by DB6P192CA0006.outlook.office365.com
 (2603:10a6:4:b8::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 07:30:38 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT009.mail.protection.outlook.com (10.152.20.117) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:30:38 +0000
Received: ("Tessian outbound 173d710607ad:v103");
 Wed, 29 Sep 2021 07:30:38 +0000
Received: from 36f130a07c45.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8D3FF14B-8DB4-49E6-9421-7799B6264A4C.1; 
 Wed, 29 Sep 2021 07:30:25 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 36f130a07c45.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:30:25 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:30:24 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:30:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 997078b8-59c1-452b-904f-726b8c31a740
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dOejfLP7LhKJrTKaI5uSV9u/URoVTJbE7lE88j5sq7U=;
 b=lGMj2SoRHRwvYgJMwWCub6oWwKtm7FePlLQ1+7UizfL2Izj9D85MvMY0xsFFCAQLW28rGX6zeDZaWhCO2D964TzyVVSgJAsdBedGYH5uj20NY4y7UhJdmDOuADpK65SWGJRWjH+/8lvzOIZFeffHVYGdfGf7YJxElGn6jreZWdo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fe8c31bef69c5875
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fh8wxvp2Cpq4qwrS/Q24udg4/f5eEKVeDpbQTPINsTMq/iwpLE65aZpnZUBHVgXiG34/qwRmiZSsVAjJ/20jmpbkmFudY3zMnAdgCw91MDrjR7zdhiWJINVe0yLpdQ1hK/PRFNa9QVfuVQoA6MIlb2FvTV7deqPgkQjQNAziJXq78kTk/neJg1UzT6dF69FetCTwzkcdwl84EBvxD2IFcphAfd9DvjG/GYmw7Kzt0i83Nfzy0lgms8BmfeAQqj+QWIG/SuIscjN6wYIB6A/k8CG9kF4rVLWGTEVxjqWI476Y1ax5HdjcLpJaOpyMGbX0XwoEjKjeBHYApBJmoUauIw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dOejfLP7LhKJrTKaI5uSV9u/URoVTJbE7lE88j5sq7U=;
 b=NV4+4kQx5uSKUY7MFTmbv1nTJb2/gppXCiaim63n0EBIdY/OaNXTwWiHATV2XwoFPImSgixObQKcavgJiGwu3qImbJmj7TLUZzVdVBMtB1i0NyDfdUDjQTUdpok6ZDoEcPQQ1k88TwHU5oB0YEEpHkHd6JEXjxcstQDrfFPL5rikcfEjZti7ZGanoZURPFpqXgMfdDVxRs+ZlICCxHMqgXaV0Q36KwQuUsDH2qZGxx0Du+YK5dFMbW/mIyBOdPYnJVxLYkeZoiS5J6YwKb/fAdWGMMpbXsQHtJR1EXJ7AIZgrXdsPc9IM9D/cD+QWY/+kfWqnboKXgV6LD17L5azYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dOejfLP7LhKJrTKaI5uSV9u/URoVTJbE7lE88j5sq7U=;
 b=lGMj2SoRHRwvYgJMwWCub6oWwKtm7FePlLQ1+7UizfL2Izj9D85MvMY0xsFFCAQLW28rGX6zeDZaWhCO2D964TzyVVSgJAsdBedGYH5uj20NY4y7UhJdmDOuADpK65SWGJRWjH+/8lvzOIZFeffHVYGdfGf7YJxElGn6jreZWdo=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andre
 Przywara <Andre.Przywara@arm.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v3 04/17] xen/arm: xc_domain_ioport_permission(..) not
 supported on ARM.
Thread-Topic: [PATCH v3 04/17] xen/arm: xc_domain_ioport_permission(..) not
 supported on ARM.
Thread-Index: AQHXtJWDr/ZkVrSogUyoFTUx7ZyQTKu6no+A
Date: Wed, 29 Sep 2021 07:30:24 +0000
Message-ID: <FC836F6B-3873-4E08-8606-2FD0D398B8B5@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <9d796d7c5c6d6c2d206809f99ed0be7ec9ebe056.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <9d796d7c5c6d6c2d206809f99ed0be7ec9ebe056.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 67e1a731-757f-4278-16f3-08d9831b08d1
x-ms-traffictypediagnostic: PAXPR08MB6816:|DB8PR08MB5434:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB8PR08MB5434751F4D768209C7A375419DA99@DB8PR08MB5434.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 MEwYmG//WQiAFA3M1ysLLVLndSx+jloznVbSF6YSS+mJCPhJHkl9SSQ8LTj819KM5eZJD2ibbxbIA05bZskSdBX+Tfc4tOVym+2qPNprpN1+QochuuGnL/lOEeFRLSIIs6Dwmq056bK7JtLajxy7lakxnQNHb9c1tCxnvHtBJeNf89mC6h6fOr+q8VadmSsPtYwnsPSExDmZVCM9h49Mg6rUlNIzJLStVZRED7m26rcDRS60eDluqwB76xd3HJ63O1ricpgeBoJz+xd9SspbX2nZaVI16UO6308qm6OVwDhODyPPse9Lm5m8fU86AKpxbhDnBvFzhAThzAfTuZ47bhLGGAyhAQ9/Txkys37BvFBo0MegAvtjP0XoOjBm4e9xTEEQiBQjYvNk3sM3JvRI8Tj53r42KKiSO77mFanvys3u7SgFGyIZvZeaN65lNqnfBXlBuZPfLW9r8hXnJy/5n87aP+ycy9BbaWUXr73P2hLaJpl2DqVnTQz1LcFP8kymGsb6cAjO84eDfURSQhaWdYPoOf4ZBjzA+khwSmjcCD3ql79iHfIhOi0uImbZM16C0pdclg/thmfStP0RcbSizmAC1nZw0lx3XvMV+7dpDQAzSjALS0HOHYp2xsrJ46NehDrFsFjto6Ax7FCIEqcaLM00E7yF7nJ34bCyzfTEn7rNKZHxMyIi/04VHd7l/PQ3AazBjxSYXmm4PYNoejm+VuiFq7MRVUhVkLVbwG2xfWFb9a0fNDYc2bQe6plhLbmd
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6512007)(71200400001)(186003)(6862004)(53546011)(5660300002)(6506007)(4326008)(6636002)(122000001)(38100700002)(83380400001)(38070700005)(8676002)(66446008)(64756008)(91956017)(2906002)(76116006)(2616005)(66946007)(66476007)(66556008)(33656002)(316002)(37006003)(54906003)(86362001)(26005)(508600001)(6486002)(8936002)(36756003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <4EBA6CC05A424941BD90461754F11996@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6816
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT009.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	bf4fe2e9-8259-4f22-d7cf-08d9831b0080
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2oxuq2FLzmJbaf4atu3KbMp7wwvqDIaenYTtyx2YafgFhXSDVJws5jGFtMQyGAbHFHWxPzoEbup8aGyk9EqTX3tpGp7wAAaM+zMgZogzdBPOb70GeRmkfxU3DA33RTa+ROhbqUd3U3o0wS6ajcM68n2M36a4vIrHbOOtlEoH9/i2BUD7cjEsUnP6Dm/eHFeQDwWJ9V3BHrGYj4u8rruRvqTXY+LIZSj/Jqjm4QW4EWVz2mdKhCQqynEOUhjoLlaAsLFfJLcvY+tGrzatti5YGtgiIpeBvtnKvCPWQOluK2RUoY3MSSjcBpg1VQdHCS84zk4ydmM0E+DHf41d65rKvFiKAP1lcTLerTiHp3oQLp23v6uG4CUSoXzKhdW1y1OxQl19NnhpjhRjbZjwMP44yXUHJWf/koK+Jvugh9l6Fwblamqrondg4czmK5gvCNNshFjj/0gMhroZyW20Kdjc14Ywt0GZ12gJUqUL19mghq2wW4XUsxeTa9am+raQhYQGK5dStIDFzs9zK+aincCluy2mFIaHXLbj1/CylwTjawVhjwrdqW4DxUxJYAwvhQ+IC2MHhsRcoWxOSiN7IJko/LB9FW7jy8KksmLsQ82dDlp9bI6BMzPulfwdFEMYnyxKV7SSwwMh6IdT14QUJzPnnyKrSPQqw8wlfAD+AWbkc7lkyuKCdyHtX/2zHZpf7qRWiUOdT5VCyUaQGuRaT+kmuQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(2616005)(5660300002)(316002)(54906003)(2906002)(36860700001)(83380400001)(6486002)(107886003)(6636002)(82310400003)(6862004)(37006003)(8676002)(86362001)(53546011)(81166007)(4326008)(6506007)(6512007)(186003)(70206006)(8936002)(26005)(336012)(33656002)(47076005)(508600001)(356005)(36756003)(70586007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:30:38.4847
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 67e1a731-757f-4278-16f3-08d9831b08d1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT009.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5434

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> ARM architecture does not implement I/O ports. Ignore this call on ARM
> to avoid the overhead of making a hypercall just for Xen to return
> -ENOSYS.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3: Added Reviewed-by: Stefano Stabellini <sstabellini@kernel.o=
rg>
> Change in v2:
> - Instead of returning success in XEN, ignored the call in xl.
> ---
> tools/libs/ctrl/xc_domain.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>=20
> diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
> index 23322b70b5..25c95f6596 100644
> --- a/tools/libs/ctrl/xc_domain.c
> +++ b/tools/libs/ctrl/xc_domain.c
> @@ -1348,6 +1348,14 @@ int xc_domain_ioport_permission(xc_interface *xch,
>                                 uint32_t nr_ports,
>                                 uint32_t allow_access)
> {
> +#if defined(__arm__) || defined(__aarch64__)
> +    /*
> +     * The ARM architecture does not implement I/O ports.
> +     * Avoid the overhead of making a hypercall just for Xen to return -=
ENOSYS.
> +     * It is safe to ignore this call on ARM so we just return 0.
> +     */
> +    return 0;
> +#else
>     DECLARE_DOMCTL;
>=20
>     domctl.cmd =3D XEN_DOMCTL_ioport_permission;
> @@ -1357,6 +1365,7 @@ int xc_domain_ioport_permission(xc_interface *xch,
>     domctl.u.ioport_permission.allow_access =3D allow_access;
>=20
>     return do_domctl(xch, &domctl);
> +#endif
> }
>=20
> int xc_availheap(xc_interface *xch,
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:34:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:34:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198685.352302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU79-0000dB-Dr; Wed, 29 Sep 2021 07:34:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198685.352302; Wed, 29 Sep 2021 07:34:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU79-0000d4-Ap; Wed, 29 Sep 2021 07:34:51 +0000
Received: by outflank-mailman (input) for mailman id 198685;
 Wed, 29 Sep 2021 07:34:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU77-0000cy-DW
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:34:49 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.81]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d2047621-2d2a-4e5f-8a61-12f486c15c34;
 Wed, 29 Sep 2021 07:34:47 +0000 (UTC)
Received: from DB6PR0601CA0033.eurprd06.prod.outlook.com (2603:10a6:4:17::19)
 by DBAPR08MB5672.eurprd08.prod.outlook.com (2603:10a6:10:1ad::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 07:34:45 +0000
Received: from DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:17:cafe::7a) by DB6PR0601CA0033.outlook.office365.com
 (2603:10a6:4:17::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Wed, 29 Sep 2021 07:34:45 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT003.mail.protection.outlook.com (10.152.20.157) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:34:45 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Wed, 29 Sep 2021 07:34:44 +0000
Received: from 6c7c8b32a4bc.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7A0F8E00-9347-4ED5-A6D2-67BBD9297996.1; 
 Wed, 29 Sep 2021 07:34:32 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6c7c8b32a4bc.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:34:32 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6271.eurprd08.prod.outlook.com (2603:10a6:102:eb::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 07:34:30 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:34:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2047621-2d2a-4e5f-8a61-12f486c15c34
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AhikygCoi2aCzLqr64xAZt0QJL4w6dKkf9BDgcmYptc=;
 b=a+897nCAZeZb3suIxR1fRKbiIi0hFDjZqRCoPM/kSVcCFTSL82gaoDYNnKVqCyNRn6bEVugIWZUlM0axl3mfOf+dvtVBggOKTo+PMOdxUqn28jaGhfovrOZ2+OZlpAJW+VZfnkaLCNJQahzts7f8YIkzKyYIqhb80fcUpFYB/HY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 86b4a94b74d641dc
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OkRI3lC3J/LeqvOMeEsWd4QYQokOLAmFnbT89E726vYRQOOppbqYp3K0MDDywP+5AiHpsXkItxxAouaLh/6fa1yyignoSiNlq/MD0QX/rnJZs0Rdy9PnjbYLO2gO3QhqzwfBHr08/Odl19JSTcjPhkwkc+9XCLdGhiosa6OEZPBDtlE637eOiKahzT59/UdKCs871t+cc7A9gdaaBrXJjn2lTROY4Vs2K2Rc4K6yPB5mopbTOblYVJlIDM34jy4z66Zs7pXk/+CAB8R3DMZVlUFcgGAVLxYfcu9m06Dqfh8y989iJJ/lUNtw68JL0jXuWH95omuB63WS1j1DmZIBTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AhikygCoi2aCzLqr64xAZt0QJL4w6dKkf9BDgcmYptc=;
 b=dETxNysCvyVcCpwa37bALP/xHIN78hxGIAy24eTGhyVboJjnt99LgbUo3OtFtZFi/Na5+5xmWcAEaQ57+x2txnZUli1Yp/C+9T8XyQGC8RFfcHjB1eYrOGLzTglV3GrmfsxJVeciPkKxMbg3UHc6mzMn6rEiUvoumGiEWvnYzw5nhy0eXo8X25xFiGcGuiDLNiVqUsOV573qUeKpa94q1agD0L9Xx2RoA7Ixfn9g8MyiI8AF9K7jtl705yvgXAJO21hNePI2m5NuBjMbJUw3wW9UG3nV2exXNzPTfcN/+hIleOlpRDPymBBlkBKj19EUP4DmsN+GjRQXef63wujl+w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AhikygCoi2aCzLqr64xAZt0QJL4w6dKkf9BDgcmYptc=;
 b=a+897nCAZeZb3suIxR1fRKbiIi0hFDjZqRCoPM/kSVcCFTSL82gaoDYNnKVqCyNRn6bEVugIWZUlM0axl3mfOf+dvtVBggOKTo+PMOdxUqn28jaGhfovrOZ2+OZlpAJW+VZfnkaLCNJQahzts7f8YIkzKyYIqhb80fcUpFYB/HY=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan Beulich
	<jbeulich@suse.com>, Wei Liu <wl@xen.org>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>
Subject: Re: [PATCH v3 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for
 ARM
Thread-Topic: [PATCH v3 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for
 ARM
Thread-Index: AQHXtJWO1YfLgI0NjECk5ZqoHOeawKu6n7SA
Date: Wed, 29 Sep 2021 07:34:29 +0000
Message-ID: <605BDD89-17FE-49DE-95EA-8108F7F02213@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 8c62ab83-93eb-4fd6-b8cb-08d9831b9c05
x-ms-traffictypediagnostic: PA4PR08MB6271:|DBAPR08MB5672:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DBAPR08MB5672701ACDBB92584692740D9DA99@DBAPR08MB5672.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +2se+A5+AMjc1FdpNXizoE5f9nkzD2vIXDNWkjicfaBQIkRn7KiHxgpCIseynr7QsK5N2lJGum3Nmkhh7E0QOvlQ4o/3XpVCVveJ6PBP0WD9jL3LO5tef+5uLsNxZenUUzOIKzzQ3YQA59COJIN0QE5t8bGWr7Gmph3IZBIYmD2JLOVOG5/wAn4OEYLir593Ea6hUyOoIFzjRmXGRkFhdz8cNd7PduS/rKbqnmZRs5Vgdwel0DsRLDcBozZHDZhQMiH+NmOcrUlSTW7w1zy7iIl6zsnJ3++oAPFaINxgeYjS56d12R15dGHPkIwwuBY3yh6tgmy55AjtUssqXvo5ptXW3fu1So0i44onnFY8OFT4sBoR6n4uxpUk0pKu4kWVHy5KoQjj1M82DyzMenbCsvN6P4oxIJzglgdaF0+el3Y6j7BzhFeCvNQQcc2XLomeeah5dWWRZes3HwKO9rDB9kXbZYwG6JQQrW+xObJF01zALplpNeRJDxmm7kCjjGLchvcLfjIRcTFZIDJDFaPIvJyleNBZOc64HhdzCkqKVqNyjuheFhpyNhyc3JPqHpV+uWFob2Aay7I3e2ngxpzCsc2NHeJQWfR3AkM6EJkPqQRGPKq9FYs/Y5uthJXW4Evhm+V0Vd9mDALsVkYpKHwty5I4HkBacqE/jtKfJATZddyq2JRXIG2hC+QOw7NNDK4pPW7aJ56TeTbfrU5Q0zuO5SBPPvkW25q4eA/h5XeWRaJyX3/tqWhUfedz8icbP85g
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(33656002)(53546011)(54906003)(2906002)(37006003)(71200400001)(316002)(7416002)(6506007)(38070700005)(86362001)(122000001)(38100700002)(64756008)(66476007)(66556008)(8676002)(66446008)(66946007)(76116006)(91956017)(6486002)(6512007)(8936002)(6862004)(4326008)(36756003)(508600001)(83380400001)(6636002)(2616005)(5660300002)(186003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <D5D645AD94469843AC027D32EAE43CD9@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6271
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	eff0cf00-84b9-4cf1-69eb-08d9831b92d8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ks1Q52LUta+v3KE/zYzlJcoMTIq3EieFz0vTCn9bXb33HI3CtNkKH1wWr5pLtiDbFlGQmD2Lhpkh1l5GCUGIMBkmhUnNFSUeJdDQz/un8p1jYSx18b2l6fnTasXH5irZB12/ON8O/fHkIXw/LpUIc6gfwMbmIORZAdhQH44pws5vDAOWrNpq5OtpoLdn9kl9JK/Y7L1Gg4rY5XOuBpUuef7DsuxuGQ1YOmBY0btmo++WWtjK3+2OxRqU9chb/p7U6wzo7tB1G/rFAe0jNdtig0jxNoFS/DpyKYuX1F40GrJUHGHt/r/Ot1cbHOfSXxtJTI4d0aRKaoIDBaINnkLuM2vVVNAFLfd/vAzRYQzTRwItGS0+xBnijKaqCuw92OmkltidxirZeguv6U084vf5V9/sI21AvNe9WtZ1hxP/NLm0r2HEC3p59BH/V18pXmY3yG3uiaTQixnzHoPwZK32VNdmnqcM2/NMvptEqwdvkOlAL3UflRb/9bKi3z/xzPmUkvA3OT48mjfdrDPw/9svxXAiiq9oSmZntFHVTrhSc6XBASmv2dwBPhetbC0w+IKPqhtzouNr6MUhWumnWamUwJbWFjd4er+IpQPG+utwiLWBG0n9t+UCGP8Fa1c7AONAyZn8jfD70L7d+mjvV5CzTM8MkDf1wOXTuNKgOnZtDtzMS8lADmFHh4d9PX+sUPosRe58zgILXpO0yLqPhroLxw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(82310400003)(107886003)(36756003)(54906003)(186003)(70206006)(6636002)(336012)(508600001)(6486002)(6862004)(26005)(83380400001)(5660300002)(8676002)(70586007)(37006003)(316002)(36860700001)(2906002)(33656002)(2616005)(8936002)(6512007)(81166007)(47076005)(6506007)(53546011)(4326008)(86362001)(356005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:34:45.4542
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c62ab83-93eb-4fd6-b8cb-08d9831b9c05
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5672

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Hardware domain is in charge of doing the PCI enumeration and will
> discover the PCI devices and then will communicate to XEN via hyper
> call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
>=20
> Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
>=20
> As most of the code for PHYSDEVOP_pci_device_* is the same between x86
> and ARM, move the code to a common file to avoid duplication.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3: Fixed minor comment.
> Change in v2:
> - Add support for PHYSDEVOP_pci_device_remove()
> - Move code to common code
> ---
> xen/arch/arm/physdev.c          |  5 +-
> xen/arch/x86/physdev.c          | 50 +-------------------
> xen/arch/x86/x86_64/physdev.c   |  4 +-
> xen/common/Makefile             |  1 +
> xen/common/physdev.c            | 81 +++++++++++++++++++++++++++++++++
> xen/include/asm-arm/hypercall.h |  2 -
> xen/include/asm-arm/numa.h      |  5 ++
> xen/include/asm-x86/hypercall.h |  9 ++--
> xen/include/xen/hypercall.h     |  8 ++++
> 9 files changed, 106 insertions(+), 59 deletions(-)
> create mode 100644 xen/common/physdev.c
>=20
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index e91355fe22..4e00b03aab 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -8,10 +8,9 @@
> #include <xen/lib.h>
> #include <xen/errno.h>
> #include <xen/sched.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>=20
> -
> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +long arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> {
>     gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=3D%d: not implemented\n", cmd);
>     return -ENOSYS;
> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
> index 23465bcd00..c00cc99404 100644
> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
> }
> #endif /* COMPAT */
>=20
> -ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +ret_t arch_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> {
>     int irq;
>     ret_t ret;
> @@ -480,54 +480,6 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(=
void) arg)
>         break;
>     }
>=20
> -    case PHYSDEVOP_pci_device_add: {
> -        struct physdev_pci_device_add add;
> -        struct pci_dev_info pdev_info;
> -        nodeid_t node;
> -
> -        ret =3D -EFAULT;
> -        if ( copy_from_guest(&add, arg, 1) !=3D 0 )
> -            break;
> -
> -        pdev_info.is_extfn =3D !!(add.flags & XEN_PCI_DEV_EXTFN);
> -        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> -        {
> -            pdev_info.is_virtfn =3D 1;
> -            pdev_info.physfn.bus =3D add.physfn.bus;
> -            pdev_info.physfn.devfn =3D add.physfn.devfn;
> -        }
> -        else
> -            pdev_info.is_virtfn =3D 0;
> -
> -        if ( add.flags & XEN_PCI_DEV_PXM )
> -        {
> -            uint32_t pxm;
> -            size_t optarr_off =3D offsetof(struct physdev_pci_device_add=
, optarr) /
> -                                sizeof(add.optarr[0]);
> -
> -            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> -                break;
> -
> -            node =3D pxm_to_node(pxm);
> -        }
> -        else
> -            node =3D NUMA_NO_NODE;
> -
> -        ret =3D pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, =
node);
> -        break;
> -    }
> -
> -    case PHYSDEVOP_pci_device_remove: {
> -        struct physdev_pci_device dev;
> -
> -        ret =3D -EFAULT;
> -        if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
> -            break;
> -
> -        ret =3D pci_remove_device(dev.seg, dev.bus, dev.devfn);
> -        break;
> -    }
> -
>     case PHYSDEVOP_prepare_msix:
>     case PHYSDEVOP_release_msix: {
>         struct physdev_pci_device dev;
> diff --git a/xen/arch/x86/x86_64/physdev.c b/xen/arch/x86/x86_64/physdev.=
c
> index 0a50cbd4d8..5f72652ff7 100644
> --- a/xen/arch/x86/x86_64/physdev.c
> +++ b/xen/arch/x86/x86_64/physdev.c
> @@ -9,9 +9,10 @@ EMIT_FILE;
> #include <compat/xen.h>
> #include <compat/event_channel.h>
> #include <compat/physdev.h>
> -#include <asm/hypercall.h>
> +#include <xen/hypercall.h>
>=20
> #define do_physdev_op compat_physdev_op
> +#define arch_physdev_op arch_compat_physdev_op
>=20
> #define physdev_apic               compat_physdev_apic
> #define physdev_apic_t             physdev_apic_compat_t
> @@ -82,6 +83,7 @@ CHECK_physdev_pci_device
> typedef int ret_t;
>=20
> #include "../physdev.c"
> +#include "../../../common/physdev.c"
>=20
> /*
>  * Local variables:
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 54de70d422..bcb1c8fb03 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -29,6 +29,7 @@ obj-y +=3D notifier.o
> obj-y +=3D page_alloc.o
> obj-$(CONFIG_HAS_PDX) +=3D pdx.o
> obj-$(CONFIG_PERF_COUNTERS) +=3D perfc.o
> +obj-y +=3D physdev.o
> obj-y +=3D preempt.o
> obj-y +=3D random.o
> obj-y +=3D rangeset.o
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> new file mode 100644
> index 0000000000..20a5530269
> --- /dev/null
> +++ b/xen/common/physdev.c
> @@ -0,0 +1,81 @@
> +
> +#include <xen/guest_access.h>
> +#include <xen/hypercall.h>
> +#include <xen/init.h>
> +
> +#ifndef COMPAT
> +typedef long ret_t;
> +#endif
> +
> +ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +{
> +    ret_t ret;
> +
> +    switch ( cmd )
> +    {
> +#ifdef CONFIG_HAS_PCI
> +    case PHYSDEVOP_pci_device_add: {
> +        struct physdev_pci_device_add add;
> +        struct pci_dev_info pdev_info;
> +        nodeid_t node;
> +
> +        ret =3D -EFAULT;
> +        if ( copy_from_guest(&add, arg, 1) !=3D 0 )
> +            break;
> +
> +        pdev_info.is_extfn =3D !!(add.flags & XEN_PCI_DEV_EXTFN);
> +        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> +        {
> +            pdev_info.is_virtfn =3D 1;
> +            pdev_info.physfn.bus =3D add.physfn.bus;
> +            pdev_info.physfn.devfn =3D add.physfn.devfn;
> +        }
> +        else
> +            pdev_info.is_virtfn =3D 0;
> +
> +        if ( add.flags & XEN_PCI_DEV_PXM )
> +        {
> +            uint32_t pxm;
> +            size_t optarr_off =3D offsetof(struct physdev_pci_device_add=
, optarr) /
> +                                sizeof(add.optarr[0]);
> +
> +            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> +                break;
> +
> +            node =3D pxm_to_node(pxm);
> +        }
> +        else
> +            node =3D NUMA_NO_NODE;
> +
> +        ret =3D pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, =
node);
> +        break;
> +    }
> +
> +    case PHYSDEVOP_pci_device_remove: {
> +        struct physdev_pci_device dev;
> +
> +        ret =3D -EFAULT;
> +        if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
> +            break;
> +
> +        ret =3D pci_remove_device(dev.seg, dev.bus, dev.devfn);
> +        break;
> +    }
> +#endif
> +    default:
> +        ret =3D arch_physdev_op(cmd, arg);
> +        break;
> +    }
> +
> +    return ret;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/hypercall.h b/xen/include/asm-arm/hyperc=
all.h
> index a0c5a31a2f..c6e4a04fd6 100644
> --- a/xen/include/asm-arm/hypercall.h
> +++ b/xen/include/asm-arm/hypercall.h
> @@ -2,8 +2,6 @@
> #define __ASM_ARM_HYPERCALL_H__
>=20
> #include <public/domctl.h> /* for arch_do_domctl */
> -int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> -
> long do_arm_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(=
void) arg);
>=20
> long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
> diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h
> index 31a6de4e23..dbd1c2467f 100644
> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
> #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
> #define __node_distance(a, b) (20)
>=20
> +static inline nodeid_t pxm_to_node(unsigned pxm)
> +{
> +    return 0xff;
> +}
> +
> #endif /* __ARCH_ARM_NUMA_H */
> /*
>  * Local variables:
> diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hyperc=
all.h
> index 0ae3b8b043..7afba98186 100644
> --- a/xen/include/asm-x86/hypercall.h
> +++ b/xen/include/asm-x86/hypercall.h
> @@ -105,10 +105,6 @@ do_update_va_mapping(
>     u64 val64,
>     unsigned long flags);
>=20
> -extern long
> -do_physdev_op(
> -    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> -
> extern long
> do_update_va_mapping_otherdomain(
>     unsigned long va,
> @@ -151,6 +147,11 @@ compat_physdev_op(
>     int cmd,
>     XEN_GUEST_HANDLE_PARAM(void) arg);
>=20
> +extern int
> +arch_compat_physdev_op(
> +    int cmd,
> +    XEN_GUEST_HANDLE_PARAM(void) arg);
> +
> extern int
> arch_compat_vcpu_op(
>     int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg);
> diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
> index 3771487a30..5cc74f40b7 100644
> --- a/xen/include/xen/hypercall.h
> +++ b/xen/include/xen/hypercall.h
> @@ -45,6 +45,14 @@ extern long
> do_platform_op(
>     XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op);
>=20
> +extern long
> +do_physdev_op(
> +    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> +
> +extern long
> +arch_physdev_op(
> +    int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
> +
> /*
>  * To allow safe resume of do_memory_op() after preemption, we need to kn=
ow
>  * at what point in the page list to resume. For this purpose I steal the
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:35:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:35:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198690.352313 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7S-00015X-RI; Wed, 29 Sep 2021 07:35:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198690.352313; Wed, 29 Sep 2021 07:35:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7S-00015N-OB; Wed, 29 Sep 2021 07:35:10 +0000
Received: by outflank-mailman (input) for mailman id 198690;
 Wed, 29 Sep 2021 07:35:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU7S-0000z7-EI
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:35:10 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.53]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a06d33ed-1c63-48a9-9f98-bbdd67f58764;
 Wed, 29 Sep 2021 07:35:08 +0000 (UTC)
Received: from AM6P193CA0114.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:85::19)
 by VI1PR08MB3168.eurprd08.prod.outlook.com (2603:10a6:803:49::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:35:05 +0000
Received: from AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:85:cafe::7a) by AM6P193CA0114.outlook.office365.com
 (2603:10a6:209:85::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 29 Sep 2021 07:35:05 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT062.mail.protection.outlook.com (10.152.17.120) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 29 Sep 2021 07:35:04 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Wed, 29 Sep 2021 07:35:03 +0000
Received: from 4feecd25145a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 689AD135-4C96-422E-AA2B-CEAA657D4D57.1; 
 Wed, 29 Sep 2021 07:34:51 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4feecd25145a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:34:51 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6271.eurprd08.prod.outlook.com (2603:10a6:102:eb::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 07:34:49 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:34:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a06d33ed-1c63-48a9-9f98-bbdd67f58764
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6I8CkTLCs7oBtYa/e5HNW/QH8dygDMbatGsUJoZIdWw=;
 b=a+7mhTOSGMB9vqmlp7EvGD+dPJPBieB0Poz5sFegv3mKdTWJBy3d3sy9gy3/lMUjJB5YrVkTpQZfrlUFoQvdQwY5r6woyTJwp3alsj/gA4ryuAiUQcddRfIcCpMn6Tz0sVvHhP3hVR+eiA2zIK11F/dvydTKAxmknAbBdD5XOJI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ebef743791d2ad0e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Kf/1bZGCYH9F/wFgPZruUFtfGXio45l4QLE9XpvEBGfMBAeZhmfBChJYHM8ElqkvsfuUvQQVeu2S03HOKrjn/5KFRm5RtwQXpuG+rfPZbkOFkkN0mw5P1WljKC8O36FxC6PSQiCFSB8OKB9znhShUUYfy9B+kDHYxEUHhxkdqSdLiBMt3PsC1sOB4IRkHuhzKlG6YWnDaiq6z6NwbnPe8GNoSaehc7U9Tc68g2Xi2jg58RYKdiSHWhvI6FLn61ExF/dufog6L/BZ+b3hUw0P3qrDX7kMMmcDid1o8zgBggkp/RkkCBTQ5WgB0eMG7jGmfLJ+cDt64A2bpDqdUuGODQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=6I8CkTLCs7oBtYa/e5HNW/QH8dygDMbatGsUJoZIdWw=;
 b=cjiC/kH+6khrh57avPFs0kAKyDldr5fjUrgJHgi2tW+EY0AMV+vQXtTYkCIhJMxGzgplGNk8+nOhFdXBV6ONfGBcDexRjAuIWhWt2AGsPbODwpXUgRD4s6DY3qZwW/x4Y9FHs53gxGQjZt0GJqP8xlN1Z1ZAl5xr+hKGQ/P6hyxv6/MXXhYzKzCt19GzO1lY6rxeUKYwxt2cCATyDvhd7TgUQd8lekINf6DOBWkMglNPGOwuRGuxLxkCQY0JEUwYzqlOaFj9/DXxze2UhxdCj+QVTn8jjlOqv1CYQBfNbAREtHuZ/ppbCKjMMf9z6YHAx6YoKLuBzUpQN6JsIsoBqQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6I8CkTLCs7oBtYa/e5HNW/QH8dygDMbatGsUJoZIdWw=;
 b=a+7mhTOSGMB9vqmlp7EvGD+dPJPBieB0Poz5sFegv3mKdTWJBy3d3sy9gy3/lMUjJB5YrVkTpQZfrlUFoQvdQwY5r6woyTJwp3alsj/gA4ryuAiUQcddRfIcCpMn6Tz0sVvHhP3hVR+eiA2zIK11F/dvydTKAxmknAbBdD5XOJI=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>
Subject: Re: [PATCH v3 06/17] xen/device-tree: Add
 dt_property_read_variable_u32_array helper
Thread-Topic: [PATCH v3 06/17] xen/device-tree: Add
 dt_property_read_variable_u32_array helper
Thread-Index: AQHXtJWYkyWVi1zlh0y0nXHkIxlt4Ku6n8oA
Date: Wed, 29 Sep 2021 07:34:49 +0000
Message-ID: <3A325D3C-714A-4DA4-91F5-B905728D163F@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <9dba5f3457d9ead9084ac73297bf5a65e665a8d9.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <9dba5f3457d9ead9084ac73297bf5a65e665a8d9.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 947ac49c-6172-497e-5697-08d9831ba7a3
x-ms-traffictypediagnostic: PA4PR08MB6271:|VI1PR08MB3168:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB316813AB8742599C88FA4F079DA99@VI1PR08MB3168.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2958;OLM:2958;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 S16HjX2y78Zl2u8l/zTdogZxvbTCKcPLsYCM6dp5VVCJKfAglVytm8jnERO7nd5KWUhvXukeI3Ny4vGOVhslhs3RQBK52/27M10PSpeUUfUhjP2yQDV03zEwXFKyJ1XA/7DnqYG6TqYDkMArrnwU5IxSIT7P6hfWYJA3EGLEJOSIwvSG4oEW8pThF1xqn+1Q2QKCpZz12cM8vMy/fmO1Wq13oyWXPjKyxhwbCQCnaAOjXgXfBLpJiNPoP0z40s5B/lQE98vU9MtAEzLlPNEHfJR8S03j2Q9FsfMpPLoQysQw5AhZ9PHvWpvesyHsfwimSSeEH5kUUA0/MnBMC1JTGVA39DA+rIYyg3yywQgrwjq1RTU52mAH07CYFxXF5DvntA7b8yGPUi4PfsAjhmfL+Td/4o7wdQXt8ZPLcCsQyw48k+jDdbyEMRlOVh3WsglM8ZDerdNR9btcbItHt9RJl58bw9gU3nvKdA6pyyT+d6E+7MSGpgY+xu9QojzSLTldVYdykVfOjj1bUZUTen9/hWAIkGQY6GgZtGUVkNhg1HYs0fPft3AOgu8uO2d08jgrHjeJJ7OaBnGs4tp/LX0Sw+/BisvCE8lhnD/TK9JUPiYRE349oGdnIzwtIUN5Dek6UspMa5wtR4GTSHN8Eip2iNk8d+BkOQ64Lf3CSTXrZfgnVytTmhESnXKe5KCidQDduRCPY7uIcjzoMIcYevzDomJPADkqO6lgwRBPA2BA13Ffa0IbtmkKNDMNbzjGrh4m
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(33656002)(53546011)(54906003)(2906002)(37006003)(71200400001)(316002)(6506007)(38070700005)(86362001)(122000001)(38100700002)(64756008)(66476007)(66556008)(8676002)(66446008)(66946007)(76116006)(91956017)(6486002)(6512007)(8936002)(6862004)(4326008)(36756003)(508600001)(83380400001)(6636002)(2616005)(5660300002)(186003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <B5EFB03D8EDF5449A04A7A8D880F7667@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6271
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	481da0f5-fdeb-4683-53d3-08d9831b9e6b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zyocVGLUiz+qavYEaFMJ3077sYT9j8SG+96XBW+GERbhp48BZkU4VWh41Rq7nhA1/of+5pm3BkQAaGT0hS9GhqrqXskfUzgT1oIAgygMfMNsN1W/+QOu/bEGosimviPo3dSQN9xZYqtmaZQ8dLn7NwTeXzYWcu3CynQ1pY3gKzZw9nvsbY4lm+v8haspxnBmEqGi8GonvcgiF6msuER6mAH6Pxq0lZ4sSMRBjU8t3jMyxff4X+kvElnb1ObU/VsF6dIZY4bLObZw+wAvMJSzGz7ods/Il77S3FRRaJysKmqqw2vDD7A11pHdSImNlA/8/gNSE7CVbr+D9JogwA0smEKkSEYKOxGob22mmnrqtCNox4EzupYJlpGOyOTwpPJqXqxpb2zvPSZcJ/4qwYQV6junJ7BBV+7KKfmbrsz+67yly4ZA5fvS5We1C1+i4UEIcN2EPsRGs3yndHUmr9MzTVmSenUPAySHNxbcUh2NVuMiBv5O+2r4msJ98VbiKyxRpUglQ2IIbP/DnmpVuK85uNPLwKpaGCSO6/AWmyubce2ilBHQwesHkAjk9xVOSweqFmLTeI4P6cjIScd+rcFmg/BoYbvpCyrrT8N/Rw4EWmd1WZtYN8baUxo5O2QXjiQF42l9j2nZOjfZnOUiDRXrAs2FjzMKA46D+u2wk1msXMnlkVc+8D4i/yQOVcHc0tU4ilVXljcaZ6lckGXHjnEDlA==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6862004)(356005)(86362001)(36756003)(6512007)(6486002)(336012)(508600001)(2616005)(81166007)(33656002)(47076005)(83380400001)(8936002)(4326008)(70586007)(6636002)(54906003)(8676002)(70206006)(186003)(5660300002)(37006003)(36860700001)(316002)(82310400003)(26005)(6506007)(53546011)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:35:04.8911
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 947ac49c-6172-497e-5697-08d9831ba7a3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3168

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
>=20
> Import the Linux helper of_property_read_variable_u32_array. This
> function find and read an array of 32 bit integers from a property,
> with bounds on the minimum and maximum array size.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable=20
>  Linux commit-id
> Change in v2: Patch introduced in v2
> ---
> xen/common/device_tree.c      | 61 +++++++++++++++++++++++++++++++++++
> xen/include/xen/device_tree.h | 26 +++++++++++++++
> 2 files changed, 87 insertions(+)
>=20
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 03d25a81ce..53160d61f8 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -208,6 +208,67 @@ int dt_property_read_string(const struct dt_device_n=
ode *np,
>     return 0;
> }
>=20
> +/**
> + * dt_find_property_value_of_size
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @min:    minimum allowed length of property value
> + * @max:    maximum allowed length of property value (0 means unlimited)
> + * @len:    if !=3DNULL, actual length is written to here
> + *
> + * Search for a property in a device node and valid the requested size.
> + *
> + * Return: The property value on success, -EINVAL if the property does n=
ot
> + * exist, -ENODATA if property does not have a value, and -EOVERFLOW if =
the
> + * property data is too small or too large.
> + */
> +static void *dt_find_property_value_of_size(const struct dt_device_node =
*np,
> +                                            const char *propname, u32 mi=
n,
> +                                            u32 max, size_t *len)
> +{
> +    const struct dt_property *prop =3D dt_find_property(np, propname, NU=
LL);
> +
> +    if ( !prop )
> +        return ERR_PTR(-EINVAL);
> +    if ( !prop->value )
> +        return ERR_PTR(-ENODATA);
> +    if ( prop->length < min )
> +        return ERR_PTR(-EOVERFLOW);
> +    if ( max && prop->length > max )
> +        return ERR_PTR(-EOVERFLOW);
> +
> +    if ( len )
> +        *len =3D prop->length;
> +
> +    return prop->value;
> +}
> +
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_v=
alues,
> +                                        size_t sz_min, size_t sz_max)
> +{
> +    size_t sz, count;
> +    const __be32 *val =3D dt_find_property_value_of_size(np, propname,
> +                        (sz_min * sizeof(*out_values)),
> +                        (sz_max * sizeof(*out_values)),
> +                        &sz);
> +
> +    if ( IS_ERR(val) )
> +        return PTR_ERR(val);
> +
> +    if ( !sz_max )
> +        sz =3D sz_min;
> +    else
> +        sz /=3D sizeof(*out_values);
> +
> +    count =3D sz;
> +    while ( count-- )
> +        *out_values++ =3D be32_to_cpup(val++);
> +
> +    return sz;
> +}
> +
> int dt_property_match_string(const struct dt_device_node *np,
>                              const char *propname, const char *string)
> {
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.=
h
> index b02696be94..1693fb8e8c 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -366,6 +366,32 @@ bool_t dt_property_read_u32(const struct dt_device_n=
ode *np,
> bool_t dt_property_read_u64(const struct dt_device_node *np,
>                             const char *name, u64 *out_value);
>=20
> +
> +/**
> + * dt_property_read_variable_u32_array - Find and read an array of 32 bi=
t
> + * integers from a property, with bounds on the minimum and maximum arra=
y size.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return found values.
> + * @sz_min: minimum number of array elements to read
> + * @sz_max: maximum number of array elements to read, if zero there is n=
o
> + *      upper limit on the number of elements in the dts entry but only
> + *      sz_min will be read.
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: The number of elements read on success, -EINVAL if the proper=
ty
> + * does not exist, -ENODATA if property does not have a value, and -EOVE=
RFLOW
> + * if the property data is smaller than sz_min or longer than sz_max.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +int dt_property_read_variable_u32_array(const struct dt_device_node *np,
> +                                        const char *propname, u32 *out_v=
alues,
> +                                        size_t sz_min, size_t sz_max);
> +
> /**
>  * dt_property_read_bool - Check if a property exists
>  * @np: node to get the value
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:35:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:35:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198697.352323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7o-0001is-4D; Wed, 29 Sep 2021 07:35:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198697.352323; Wed, 29 Sep 2021 07:35:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7o-0001ij-1F; Wed, 29 Sep 2021 07:35:32 +0000
Received: by outflank-mailman (input) for mailman id 198697;
 Wed, 29 Sep 2021 07:35:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU7n-0001iA-Bf
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:35:31 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.0.78]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 45b23c9b-1dd7-426d-845b-f7dcf8a97595;
 Wed, 29 Sep 2021 07:35:30 +0000 (UTC)
Received: from DU2PR04CA0215.eurprd04.prod.outlook.com (2603:10a6:10:2b1::10)
 by AM9PR08MB5937.eurprd08.prod.outlook.com (2603:10a6:20b:285::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:35:28 +0000
Received: from DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:2b1:cafe::94) by DU2PR04CA0215.outlook.office365.com
 (2603:10a6:10:2b1::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 07:35:28 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT036.mail.protection.outlook.com (10.152.20.185) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:35:28 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Wed, 29 Sep 2021 07:35:18 +0000
Received: from 4784a93395da.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F90C5D43-6A0B-495A-AC5E-CFC6458AE053.1; 
 Wed, 29 Sep 2021 07:35:06 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4784a93395da.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:35:06 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6271.eurprd08.prod.outlook.com (2603:10a6:102:eb::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 07:35:05 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:35:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45b23c9b-1dd7-426d-845b-f7dcf8a97595
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1FdtwCajRj+TzN7SdJwtQacsfMcLW+OGcADYIRdUN4Q=;
 b=8RpiBFYGxi5Ch1DWzO39HBp5vt8bygwLRlJPefWxIoIxDZOglNnsUTBv0OO7jElhwg8EiLyuZhi/dZ0p+OIk7JOH3hQpXneNMlnqScYYpI0vpL3zOa2N10vzgaN6p0ZnUH1H04m5J+g1F2vPJ5bIbIeaRe8lcVT11wTBVKyAR/o=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 8afd9cf5120f7d4e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZwzNeteHmBkOzcQwoH0Md1EXw3chh3GtSR9CK4JTzJctVRJRlBrmaPgy903NwcrElYIiZvOf+0EuUJtmtSGi6JA7xEbCkchokjRcGsKi2TI3PmMNy1wCwjASmc+DQYfUN2A3NkB3MbubUiiw3Ra0Ba0EM4ZL++2+ykk4u2gLxHR/lLN2OiPXy3NtMF469s+hEbWuMxCRhzlh1UrWXDXEh0F8BMdd3TZB7G1t2fPrTR4JcBMeAV0xd/KzVMOVYAL/OiVSSUBLeRmEIfLjrVXbPkZKyBp94+KyCZ3kakpzpI6ruhJg3ozuTqoTqqUzBQcByaVv5dcBmtYT50jFmapPnw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1FdtwCajRj+TzN7SdJwtQacsfMcLW+OGcADYIRdUN4Q=;
 b=B59M2IFrfBt5/mWxmT1LtwiYoRwTFqUM5IR+ZdJ7jCIKls3LdNhPKW3oo+8W3v7a0Ng8b4yDMhRfLpyQFl7jyHw2jCRQgVSIiP29zTita8igoyKCuI3oxqnsWjlBbT+esyy/AFQ1PwfzLPFxt/0YuqGFKLNprqecOI6ZdFfdllPH228kFNmeq0Jg8CuLUPl94q7aeHDKscdQNgc3DHukXsjR5ClWCQ82Q+JlsuHb3mGvkQvEultla6rfTI6vJFb/AEKnKVZbPOUWelA+wiZSj9TzsjIzQmh9iqK/TJQLXD2BT9NOszhGUib4WwTkw5Syow+oE+hBNUIZUBzB4+lWQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1FdtwCajRj+TzN7SdJwtQacsfMcLW+OGcADYIRdUN4Q=;
 b=8RpiBFYGxi5Ch1DWzO39HBp5vt8bygwLRlJPefWxIoIxDZOglNnsUTBv0OO7jElhwg8EiLyuZhi/dZ0p+OIk7JOH3hQpXneNMlnqScYYpI0vpL3zOa2N10vzgaN6p0ZnUH1H04m5J+g1F2vPJ5bIbIeaRe8lcVT11wTBVKyAR/o=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andre
 Przywara <Andre.Przywara@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Subject: Re: [PATCH v3 07/17] xen/device-tree: Add dt_property_read_u32_array
 helper
Thread-Topic: [PATCH v3 07/17] xen/device-tree: Add dt_property_read_u32_array
 helper
Thread-Index: AQHXtJWv7GtV+W3zUUON47opBlrdw6u6n90A
Date: Wed, 29 Sep 2021 07:35:04 +0000
Message-ID: <D6B739AB-788C-471A-84B3-73F9539FFA58@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <89788e202bc91e2bc7f9c3c33eb91ab7478381ee.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <89788e202bc91e2bc7f9c3c33eb91ab7478381ee.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 79485988-0776-4f2f-d041-08d9831bb594
x-ms-traffictypediagnostic: PA4PR08MB6271:|AM9PR08MB5937:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB59377F5A2559D3464CF61A6A9DA99@AM9PR08MB5937.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2582;OLM:2582;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GXcdvl/L2JtfASiI/M/jCm9wHPaYBxysBtF8raqjVXTU31RNNKA4MkYsGf1g1RoAc34s+qrWzyecyotJOub9Mfdar7B+RZLgiZRzgZ6BbQyBSEiCBaosLHg9uGLy3+NUe3MlvTdHdbM2BM6XDQyJTbDJOCTxybet973LiGDxkh78f+QfrSHzMRrDqtddy974wvT1GlH8RVv9UlRQiI4NrV1mp+WItoI8TJOmYYOrKNiPvs7/Le/olNAu8mDEDCa81qT+z9j7/876jGJJUEiw6HSv5y0rme5/sFSyKfcZ1ncOij1Juq+Gq1rYLwq3aG2B1uyRrLCbmNBemKQKBKbrQ+nCVYQvrWK+TU7w3T7tyIczzRdspHhmgnUdIjoeNsEn6LYLzZzrnznGl18QlEE1nb7JVmAc5BReQlJwifIfFCfFrHVR2DUpfGoa/bucvvq/MMkzPi4wdd26RWQKdbxEq1T+8/beu7YrE/WfvXgirlHqh5oFDHmbczX9sCQylJOBQGg+6MIfGdjzOfnr3frTB1P8SoPNOK1zCq0/7DEm86QDOtZAaOk3FQkhgqW4cgj/DXCtSVZcmCeiyYnVtkSPjG/xNPx7mnNgP0ZN/VD6gCwAfGgqPUyhiQUK42A89zon+8zvOrzPzseQhdIYsBa+tMnpZx5FoiIinM7DOLC1I3vzkN0cD2tjHmr2JLbKo8YLzUyQe3DMBIadFzWbGUO5hVs6wk7Ls7bbF6rFlaAB0XGyN782F6fNVb/XCtwLO5T7
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(33656002)(53546011)(54906003)(2906002)(37006003)(71200400001)(316002)(6506007)(38070700005)(86362001)(122000001)(38100700002)(64756008)(66476007)(66556008)(8676002)(66446008)(66946007)(76116006)(91956017)(6486002)(6512007)(8936002)(6862004)(4326008)(36756003)(508600001)(83380400001)(6636002)(2616005)(5660300002)(186003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1F0560E49C64E842873C8E71410DCBE3@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6271
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	0b5654c5-ccce-41c9-082b-08d9831ba7c8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Yf3NVzUwHcACspdM+LTQvsKekacJ+hUmdcWvD4O3P9JjnR+1CWSHHCyolrNdedh2DZw+n9gr0Qmr0CIL1aS/T2y3H781r8Vt5KDxEVDUCQuklmbK3YqVP5fIRzhJ2eLAbipETMBM9XXR+ggIfLXo+LIasLePtvZu0QApb5TFvpYej1/5NQ9NgWpi5HjwzhD3pHBILeL3t9HExUWqLH0KN8z/nvY/47QhDaB0XVXycgalg4n8/4K8iyW8Gr7G12uaC7oNgx0Whv8dPAO1iFGcYp2hz2fvIM6/FYsIdfcKGuSwNSrjkobtP+Wplip4jeMORt7EwaY09ykQqNy7ozlyokjnywl1kJbC5Vg1sqx6i40o5v9nWyRIdP5C5f7num/HcEbDxGqaQfwtJzQclJiTFbxLQO1TTF/0ALYziCiAJcGvyr4347FER8lO8KUxvfXjqKaShFQ5ZG8+AcpZh0GLYsNoKdFvlvCVF+bJDVa9EtXUbDOxDECP87uy2ASlDgvIPN5paWxXf7K94xv5ETef2gF9nV0fWJt/SJPoluPChRrz/5AT9WbXwu9s3iGXhiwjCm3z26Ue2D+p4IT9FI9YO/zlX/ozZoyVMxfL1tdEUhr8YRhGYTdZJS1h6TFYCcNwv28uNDxSS9CJXZgQmgohfNIaVkBFTx3u0WFnsYyIuzJ4czGIWPE8V18nDjQ8FSGAIGwi4H0QrhHlnioqg2d80w==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(47076005)(508600001)(26005)(186003)(2616005)(8936002)(6862004)(6506007)(54906003)(36756003)(53546011)(86362001)(6636002)(82310400003)(316002)(5660300002)(6486002)(37006003)(2906002)(33656002)(81166007)(70206006)(70586007)(83380400001)(336012)(36860700001)(356005)(4326008)(8676002)(6512007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:35:28.2080
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 79485988-0776-4f2f-d041-08d9831bb594
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB5937

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
>=20
> Import the Linux helper of_property_read_u32_array. This function find
> and read an array of 32 bit integers from a property.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable
>  Linux commit-id
> Change in v2: Patch introduced in v2
> ---
> xen/include/xen/device_tree.h | 30 ++++++++++++++++++++++++++++++
> 1 file changed, 30 insertions(+)
>=20
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.=
h
> index 1693fb8e8c..3ffe3eb3d2 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -392,6 +392,36 @@ int dt_property_read_variable_u32_array(const struct=
 dt_device_node *np,
>                                         const char *propname, u32 *out_va=
lues,
>                                         size_t sz_min, size_t sz_max);
>=20
> +/**
> + * dt_property_read_u32_array - Find and read an array of 32 bit integer=
s
> + * from a property.
> + *
> + * @np:     device node from which the property value is to be read.
> + * @propname:   name of the property to be searched.
> + * @out_values: pointer to return value, modified only if return value i=
s 0.
> + * @sz:     number of array elements to read
> + *
> + * Search for a property in a device node and read 32-bit value(s) from
> + * it.
> + *
> + * Return: 0 on success, -EINVAL if the property does not exist,
> + * -ENODATA if property does not have a value, and -EOVERFLOW if the
> + * property data isn't large enough.
> + *
> + * The out_values is modified only if a valid u32 value can be decoded.
> + */
> +static inline int dt_property_read_u32_array(const struct dt_device_node=
 *np,
> +                                             const char *propname,
> +                                             u32 *out_values, size_t sz)
> +{
> +    int ret =3D dt_property_read_variable_u32_array(np, propname, out_va=
lues,
> +                              sz, 0);
> +    if ( ret >=3D 0 )
> +        return 0;
> +    else
> +        return ret;
> +}
> +
> /**
>  * dt_property_read_bool - Check if a property exists
>  * @np: node to get the value
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:35:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:35:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198699.352335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7y-00028D-DL; Wed, 29 Sep 2021 07:35:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198699.352335; Wed, 29 Sep 2021 07:35:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU7y-000286-9q; Wed, 29 Sep 2021 07:35:42 +0000
Received: by outflank-mailman (input) for mailman id 198699;
 Wed, 29 Sep 2021 07:35:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU7w-00025H-Kv
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:35:40 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.73]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3e14de3-4705-4b92-9b1a-70e28bc43979;
 Wed, 29 Sep 2021 07:35:39 +0000 (UTC)
Received: from AM5PR0602CA0020.eurprd06.prod.outlook.com
 (2603:10a6:203:a3::30) by DB9PR08MB6828.eurprd08.prod.outlook.com
 (2603:10a6:10:26e::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Wed, 29 Sep
 2021 07:35:36 +0000
Received: from VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:203:a3:cafe::73) by AM5PR0602CA0020.outlook.office365.com
 (2603:10a6:203:a3::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 07:35:36 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT048.mail.protection.outlook.com (10.152.19.8) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:35:36 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Wed, 29 Sep 2021 07:35:34 +0000
Received: from b3de1c9bdd44.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 761E49FD-EF59-427A-9E53-B4C10B9143C9.1; 
 Wed, 29 Sep 2021 07:35:24 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b3de1c9bdd44.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:35:24 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6511.eurprd08.prod.outlook.com (2603:10a6:102:12d::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:35:20 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:35:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3e14de3-4705-4b92-9b1a-70e28bc43979
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QIjdK7QjOT4c/q9rcaKd2cg7zUzTEW/PNV6L3CcU7c0=;
 b=p8P+c3ZUomDWvYAUBGcnvtgxVH0V4fwwWmtHGt8cbl58JqYTZk7MrpIm162bhs+YzdsSHhGqBCBbIWgfcSfLykfBjIH0NmndJurohwsZzETt52ecMUIXl6hZ64nL6Ebq/YrlNf9Ig51xEG+CVDODBDQ+U+QOXCf8PIfn9RPQ0bA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f76c3558eb2d61ad
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GAkSt+PBMqep8Zg3IF8IT5ZCrRABLADc1PRUtQsL/yAANg+chU3Oh3wHD37aeNQS2U7V+4P+LJoGxXYJ5nTJ4+TKX5xisQRnug2iY1yxyOyFdQa91YEwSUutG3VMZUWkOg5fefDXRVYQXIweejKd93dWNhxUau/+7Bamamb7jDSwsI8Qz53A5qogIu8DL8BShIFB2i5WMNm5TRsS2pk8T9oW8rTf56dPAkK9Wwt5VsIXpUh7DXLj5Gcj2LvRfQIwaHewMgLaXPVFZnMoCQ+lub5bLRQ0gl71wVj0xL4Qme7w2BUPjxlYNQ3NbwLsyqV7v+kqadf2jpYfzSWiVWKSXQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QIjdK7QjOT4c/q9rcaKd2cg7zUzTEW/PNV6L3CcU7c0=;
 b=QI/RH4cPetgIlDEZXxtJj/jVhzzi10SP2FQXvs4W3iTVFOGdM0pjdHDTy+Y8Fu2KCgQhjbz+zvpqVFqozL9SacX3PVl7q9+w5mc67hl6apGMgJqvsLI9L0RKZigL8KKJ1spSpw43NKzrLZHj2464drlVtrnHi5nFlHQ+9O43ntgZ9awKcXKLLirQKB9Sxe+/msQdYS5RLqTO2W1Ys8Lh6ZXaaVfqOfUGEjJhs7aBIhJLME7PF5vkO8g699ihGSDqx5TSUnxLxmrxBvw07Tm8w5dSAU+AxOl8JeSEvMVDO0MVki/T5aqX3Z5zxA6moCs3hHeJ4Nnf1CXj0KMdqeVziA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QIjdK7QjOT4c/q9rcaKd2cg7zUzTEW/PNV6L3CcU7c0=;
 b=p8P+c3ZUomDWvYAUBGcnvtgxVH0V4fwwWmtHGt8cbl58JqYTZk7MrpIm162bhs+YzdsSHhGqBCBbIWgfcSfLykfBjIH0NmndJurohwsZzETt52ecMUIXl6hZ64nL6Ebq/YrlNf9Ig51xEG+CVDODBDQ+U+QOXCf8PIfn9RPQ0bA=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andre
 Przywara <Andre.Przywara@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Subject: Re: [PATCH v3 08/17] xen/device-tree: Add dt_get_pci_domain_nr helper
Thread-Topic: [PATCH v3 08/17] xen/device-tree: Add dt_get_pci_domain_nr
 helper
Thread-Index: AQHXtJW4akzPy06fZESxWckJ42Gz9au6n/AA
Date: Wed, 29 Sep 2021 07:35:20 +0000
Message-ID: <E66E6BF0-5C3D-4290-BF0E-F23473A0B012@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <f70ac215cd0c62d71231b8f61f12c1f8d801710e.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <f70ac215cd0c62d71231b8f61f12c1f8d801710e.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 946df0d2-12ae-4e94-a32b-08d9831bba63
x-ms-traffictypediagnostic: PAXPR08MB6511:|DB9PR08MB6828:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB9PR08MB6828543FAC78CAD60539B3639DA99@DB9PR08MB6828.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:2000;OLM:2000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 zeBpK7+Mq3dzbJ07VAvVTT1YmU/HZUDEWkVZhqd72+Xgf3hUm5BLSlrp+u/iqS50Ncxf43QTTquFlaWMuHblDieEw77XmBgqxoc2xRfpv+HVf2ai0ecQ2pLqqZr2dgpAHKG9LmJzhRtTzOmE1JMtvOj+5dNQWzvTYTtrMGucw4/udu7frWVqjKntPwv6l46sf08TjqvlF956XurHAPcuEuC0IbN53UPFLYbjJxmqQeM+iFO2RfDr0J8ne1r238G+Ev6siUZW0HTLFvgYDjufM9bJ+Wer0/uRT0KSH7zozvNul7SfGXupQ0VixLllGI4wc6btj9EqmK2V+0Bsr0wVCI5lN+RZ2kp+mViYQ99JLPAK4bndDxSGDNsJbUL2o2330oH6TNES5ZDyk8Zh+RDQbYnvuOuLtIzB8YMTjFU+uJ2neh9QgK4gqmPA4VRqspkFCxbZhpQzupqeo6GMBtsUpzRvDiEQN7u2JcoLEOZQMl3Sz449JVfIjxDarTi4rTJ4kQgmEOKCW0ZQoGiV0+uc6q1lMj/NBPbX+JZLcZqjwPiHnukBGgwVkDS0KKsEgcaF42YGzyScG4cp8Am06kMFO+jP66wxl9AqI163dFdebYf7PPSGNz+t/B5dhnnDuxo7ivLvwmXrKwAPNL8BF1R/XYZrBfYgT7ffwqegMaO9OboBqSs48xNUF0pst9Y+r5NUVdnFmSuqIIwBU1bq07UWdYedQo4mkrNB3W0BAWHav65pBFvS/J7Ld88PF92jvWdF
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(8936002)(86362001)(2906002)(5660300002)(38100700002)(66946007)(66476007)(37006003)(316002)(4326008)(36756003)(66556008)(76116006)(71200400001)(186003)(66446008)(64756008)(54906003)(83380400001)(91956017)(6862004)(122000001)(8676002)(26005)(6636002)(2616005)(6486002)(6506007)(38070700005)(33656002)(6512007)(53546011)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <06703B33A1400A409071D27CF6B12A00@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6511
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2874122f-2970-4dc8-13ad-08d9831bb11c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lcvPLHGlBnrGQKE4IkwNtvjsTgDXHNf7CkKyCFEXkdrCIsTExbHmM2YjJ2FAXuTeWaNLTIepELwMq3wLjSedtLTsaB1VTclj9Weki0rrpn7glLubrtp8rHKO551nXymTKtjrg5WtI9l+B/u1342cLxmN9CjqOMSIjdEo+puADNlJ68YiQjRDhycAniTFqIRTpCR9PpUF4aNjppvGqNc98jhAQyTvZcdTyfurJZTKy6sfr3zDDW8uz1wIg9Oy860qLx16oNp3KrGSte7tsIMtBcePytFf5zS5CliW1E3xxxHeyc5Y51ahRcLvIR6WunmX/RfXvJxrvBpZHgDpSxyiGx3sWptmeaE6+thzG+PHcMGEmQgTVDFXnWE+Pn5yjQDwgkHyK/g+hzPHXa1GlTNOzn5QTvZfJqXQkG7hVQ0p4hrKhGXTWNKSNqsewFvMkroG3fkbXJXxjuk3GSV49QF1jO5LWMHgYmzo6rnLydYZlBNRBLPLsyxOS6P25uCUhAZHMV7iHbx3aHHHMFYzefdSYr1+GzFt4DgBPVRYBadjtOZ8LT8ZmofO8w09Wa4c9C2OJfgePYrc9GTeg9LZpIvQstQtvGtzkd5v7xqF7iNo/vQYHxBYtqShgQTHDWJCnSSO0Gc8e+Z49T62Y4Lj2zqZ85WaUi4pvKjpxusOk/QB5TfvQEYPs2r+Dg/geWVW4b1iW/WylZndlDYqvouE8zqAxw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(70586007)(6486002)(47076005)(2616005)(70206006)(33656002)(36756003)(316002)(6512007)(26005)(54906003)(37006003)(6636002)(6862004)(336012)(53546011)(186003)(6506007)(508600001)(86362001)(356005)(36860700001)(82310400003)(81166007)(8936002)(5660300002)(8676002)(2906002)(83380400001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:35:36.2934
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 946df0d2-12ae-4e94-a32b-08d9831bba63
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6828

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Based Linux commit 41e5c0f81d3e676d671d96a0a1fafb27abfbd9d7
>=20
> Import the Linux helper of_get_pci_domain_nr. This function will try to
> obtain the host bridge domain number by finding a property called
> "linux,pci-domain" of the given device node.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Modify commit message to include upstream Linux commit-id not stable
>  Linux commit-id
> - Remove return value as those are not valid for XEN
> Change in v2: Patch introduced in v2
> ---
> xen/common/device_tree.c      | 12 ++++++++++++
> xen/include/xen/device_tree.h | 17 +++++++++++++++++
> 2 files changed, 29 insertions(+)
>=20
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 53160d61f8..ea93da1725 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -2183,6 +2183,18 @@ void __init dt_unflatten_host_device_tree(void)
>     dt_alias_scan();
> }
>=20
> +int dt_get_pci_domain_nr(struct dt_device_node *node)
> +{
> +    u32 domain;
> +    int error;
> +
> +    error =3D dt_property_read_u32(node, "linux,pci-domain", &domain);
> +    if ( !error )
> +        return -EINVAL;
> +
> +    return (u16)domain;
> +}
> +
> /*
>  * Local variables:
>  * mode: C
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.=
h
> index 3ffe3eb3d2..2297c59ce6 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -832,6 +832,23 @@ int dt_count_phandle_with_args(const struct dt_devic=
e_node *np,
>                                const char *list_name,
>                                const char *cells_name);
>=20
> +/**
> + * dt_get_pci_domain_nr - Find the host bridge domain number
> + *            of the given device node.
> + * @node: Device tree node with the domain information.
> + *
> + * This function will try to obtain the host bridge domain number by fin=
ding
> + * a property called "linux,pci-domain" of the given device node.
> + *
> + * Return:
> + * * > 0    - On success, an associated domain number.
> + * * -EINVAL    - The property "linux,pci-domain" does not exist.
> + *
> + * Returns the associated domain number from DT in the range [0-0xffff],=
 or
> + * a negative value if the required property is not found.
> + */
> +int dt_get_pci_domain_nr(struct dt_device_node *node);
> +
> #ifdef CONFIG_DEVICE_TREE_DEBUG
> #define dt_dprintk(fmt, args...)  \
>     printk(XENLOG_DEBUG fmt, ## args)
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:36:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:36:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198707.352346 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU8L-0002xQ-T4; Wed, 29 Sep 2021 07:36:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198707.352346; Wed, 29 Sep 2021 07:36:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU8L-0002xJ-PM; Wed, 29 Sep 2021 07:36:05 +0000
Received: by outflank-mailman (input) for mailman id 198707;
 Wed, 29 Sep 2021 07:36:04 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU8K-00025H-L7
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:36:04 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0c::61c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 906553f6-ccd4-48dc-95b0-6321f163a02f;
 Wed, 29 Sep 2021 07:36:00 +0000 (UTC)
Received: from AS9PR06CA0098.eurprd06.prod.outlook.com (2603:10a6:20b:465::13)
 by AM6PR08MB5542.eurprd08.prod.outlook.com (2603:10a6:20b:71::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 07:35:57 +0000
Received: from AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:465:cafe::a3) by AS9PR06CA0098.outlook.office365.com
 (2603:10a6:20b:465::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 07:35:57 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT037.mail.protection.outlook.com (10.152.17.241) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:35:57 +0000
Received: ("Tessian outbound ab2dc3678fa9:v103");
 Wed, 29 Sep 2021 07:35:54 +0000
Received: from 11bc9da7fabd.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 23E42074-F32B-40D7-839A-CC6591765930.1; 
 Wed, 29 Sep 2021 07:35:43 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 11bc9da7fabd.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:35:43 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5804.eurprd08.prod.outlook.com (2603:10a6:102:83::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 07:35:41 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:35:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 906553f6-ccd4-48dc-95b0-6321f163a02f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=br4YWecE0/dKzQxIu3myDDCGg0qQtfSf+Xoe6/yi/Ws=;
 b=yf2qpabwx5Cx42oCipcCZDAgOXHBscH+PtDPD0dLLaNOxHDacvIgw30d9bDektio0iRwefp9xDOJgYLg/eKbuPF+EuSvZ9/A9iOMpwd42A1vNnAzZaEaKMwZj89W6bqsRqylvFyO5tXb3UIITLaRBi5+mDuDtZMet+O7GFZXG5g=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ef6d481e05ba4f41
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nNcaf4utAUU/qZgyIIED6jljHtXqi8HH2h0jDD92AfKF05z6OM5Ix/NVc3fFDLA3bxR2NGmtKVXi7ZCRsUFQxA6VW7XlzlGkE7FBbeQ/cmlklysdz9MpfhahUWfQUFwT/TqP2Vl66ZfIPMOsbPTB+lOWoUsLCIwrXIv0X6rCBWAwFZy9wJ7NaFU2hjGpvo3+KiqUtNq34oRjTaavLaaKNGTIjV470HD57pmTjDBrMiCiG8lsuYUKSVx6DMPZYSGzcMd4MtIKTpLJWbDYb3+UZiPsYoQOEZuQgPe1J5o4lD0iuD5b3Ha0l7w+V04gcqgPriTHbKlMZ9B4DVSzDAXQhg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=br4YWecE0/dKzQxIu3myDDCGg0qQtfSf+Xoe6/yi/Ws=;
 b=SMsIfJg+TMU44/f8AGoLujpu9qGixgt0iU5Bm5mmh1dcq9cBDBN3U4kB/VwU+BwQaq3cickJl4dO26ETELAwP0twzi+8rvoEv4Qhb4Ixe5Kjh8hER1nfLQ0GZm/1Sumh3wcuB6O0cpHB5f0mc+iOa7NVMO+tNo9lhnrmz5MpxdpGfL4+SSNWjSzJFE+mhEg8db9uZPSQ9hktodZNKDq4XpVAuYRMq0sEaSNPTociIh4TBj9PxRP3c8MTJYkeuuQjzQ3sfTO64jyCSqW1OWWzlI43gbGrbUueV2NSY/1IXrmFJG/YIxQt8KzjbO/k5dYOw0Ax7Pec9HpXu9zH6MlX+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=br4YWecE0/dKzQxIu3myDDCGg0qQtfSf+Xoe6/yi/Ws=;
 b=yf2qpabwx5Cx42oCipcCZDAgOXHBscH+PtDPD0dLLaNOxHDacvIgw30d9bDektio0iRwefp9xDOJgYLg/eKbuPF+EuSvZ9/A9iOMpwd42A1vNnAzZaEaKMwZj89W6bqsRqylvFyO5tXb3UIITLaRBi5+mDuDtZMet+O7GFZXG5g=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andre
 Przywara <Andre.Przywara@arm.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Topic: [PATCH v3 09/17] xen/arm: Add support for PCI init to initialize
 the PCI driver.
Thread-Index: AQHXtJXC6tCzxs8NFU+yBY1A0K6UAau6oAmA
Date: Wed, 29 Sep 2021 07:35:41 +0000
Message-ID: <BCA0C9DB-C244-4CF9-8965-43410FE5A14E@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <31f2e3baf45f059a8ddac22fefc7cdfe1bc63ef5.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <31f2e3baf45f059a8ddac22fefc7cdfe1bc63ef5.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 53638ebc-45ec-40cd-c281-08d9831bc6b3
x-ms-traffictypediagnostic: PR3PR08MB5804:|AM6PR08MB5542:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB55427640E8819437B357FBA79DA99@AM6PR08MB5542.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 tSKLWmCx4eW1kMdSS1qmyQ+WZZ6FojAGC3RdMaG8K2FCJxt2A7PamBRYMJse/0LIJlbnTRo41VzLQACTd4/uKF2OVbfA1gjhIkUqrUvkvXxC8xzRJRxEisdg8Kom/a/dIhfVeiL/YZSTDNwQCTkbejBXflcfuuub4BihD6/wj/H6YnPlME3g/oEqm0MJS0s+ES2f3/tgFy5GqP/JjxzF5ImKyk+eIY7QHsgLzrJj+vlW3T7UlAW3Zp3Gicai6tzeFUsTqGApA1PdpMGHM1psDIZvfDBG8/tcKT4Oe79dEeNK7RZTb15vdbz1TrIO/UByGJyUzBHkfEjejivYawnPECZ7bnWse1i1ob4sOsUG/52Nw/pAO7Zw/5658Sn63b2hhVBokf1aEHhFzz5I4rERUGGausZwfmdFYQVHY9eGwJtaxcfzwTs1kSMbZ8BPTg96Q2uWfT9XqZ+szPscGEqi3Wx1m7Dr7j6f7oAV8DQeTbplLq+1ScOQvhIP4njOLCecNLA0ZdlF7guaZSJafdLdupmT7G3WKx/3bQNMKNz0F8Zb/jvPNUyruVwJNk2JQWRyUTB4oZOTcVQL5L0ce9A/PPn42t8yXUfgc86jTwFILl/10yh6pSd80oxRBa3y2aCD6/QxTjGttVReL49lB7tOlR1s4spqiHFF/ycohMjOEuwYPNSMxM3mGunmPScXpVuAjznxT0PhDr4Eo2ldovRePp2P2LYtixfEKee+WAwQhHdiZ+08J1Z7cmAUH6pXJGzNWm9iVjCaDQ4pVBVO+tHniOrTxSdHUTr0yTPYDQMcKDjRthNNY4CtHNMpjcHqHT6vbx4I8sLtyIKku+CATTSL7Q==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(2616005)(316002)(6512007)(37006003)(2906002)(4326008)(38100700002)(122000001)(36756003)(33656002)(5660300002)(6486002)(71200400001)(6636002)(54906003)(83380400001)(186003)(76116006)(91956017)(66476007)(66946007)(26005)(86362001)(6506007)(53546011)(8676002)(66556008)(508600001)(6862004)(8936002)(64756008)(66446008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8CB8C42FD2E2864D82FF4A3ECE8F85EF@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5804
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	177cc5ba-2abf-45b8-3c3a-08d9831bbd7c
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lz3fLUcg9fIU8SWeZIt9BXQZ4F3To0F+itFMvBPD1BEtPsR9ah42sKihuz4Z9fo7GReSUCtY7Ge0saxoAdxci2Vr5ITg+PyxRv+yk2U95cApyIirRnVHpDBbnJ4LWIYDIq01gamU7e+iBusYOkluOh+I8lj6D+6hMpGsjhMzUo0+XVsIewvpmgXi0ve2vwny5GcDOt+Aph5NYMZwzQDOL8lJbsEyfUmcyCNZwRI6RyukZKrBpJ5OcjHY/Dem2mI5t7AhjbZQ861UnvJHA2hq4fM7AaKYRO5IDKJiM+YkDUUWAnhuVoK4DCd4hPl/1RvPFavyEyD2KCce4bPUacwOFewev7t/37RjOBbnEhd0XpIshhq/htJvcencrnhuA266e9G9D0eNKbpeGPM0TK9DvNaFRBKrHyaw9+pLutBdQ4EpSzK3hGRNtRDhuEti31NAWOhCaVTVFXfL0QEKnDsGxllTkj/PIaorcD0hx0jEnxX27C14vBexN2Df2d6bZW28VgnkR+y22EDbbHBZgsMgIJN5nn8HVO1t86HClI89pxIIKlW9gkTwzeSRrb3Qm5PYbY6nIAJSw+LkatPbmLR9cNyVg8W9NrarJ34DfuoluT8UUD9gHy4aDoFPUHCNakTPOh0dnXZIW8pjH5khfNpZjBIQpTAGZYZAJkjoaYhdUYmCyqaAi7BM9xz/Q0O/hjL5XYbsITB8C7AR7J1wOsR+13Rdni0gxHgIdbuaT+Gmw5TgQze/+i0x57ek4VM+nQOdjMxXpBeNyYjq81pNlDFFyyhGw0yOQDts0R0k4IUy5e0=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(36860700001)(336012)(70206006)(107886003)(316002)(83380400001)(8936002)(6862004)(26005)(70586007)(81166007)(6512007)(33656002)(47076005)(37006003)(6486002)(186003)(5660300002)(6636002)(82310400003)(6506007)(86362001)(2906002)(508600001)(53546011)(54906003)(2616005)(36756003)(4326008)(8676002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:35:57.0071
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 53638ebc-45ec-40cd-c281-08d9831bc6b3
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5542

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> pci_init(..) will be called during xen startup to initialize and probe
> the PCI host-bridge driver.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Some nit for device_init(..) return logic
> - Remove inline from acpi_pci_init(..)
> - Modify return value for apci_pci_init(..) to return -EOPNOTSUPP
> Change in v2:
> - ACPI init function to return int
> - pci_segments_init() called before dt/acpi init
> ---
> xen/arch/arm/pci/pci.c       | 51 ++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/device.h |  1 +
> 2 files changed, 52 insertions(+)
>=20
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index a7a7bc3213..e359bab9ea 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -12,6 +12,10 @@
>  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>  */
>=20
> +#include <xen/acpi.h>
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/init.h>
> #include <xen/pci.h>
>=20
> /*
> @@ -22,6 +26,53 @@ int arch_pci_clean_pirqs(struct domain *d)
>     return 0;
> }
>=20
> +static int __init dt_pci_init(void)
> +{
> +    struct dt_device_node *np;
> +    int rc;
> +
> +    dt_for_each_device_node(dt_host, np)
> +    {
> +        rc =3D device_init(np, DEVICE_PCI, NULL);
> +        /*
> +         * Ignore the following error codes:
> +         *   - EBADF: Indicate the current device is not a pci device.
> +         *   - ENODEV: The pci device is not present or cannot be used b=
y
> +         *     Xen.
> +         */
> +        if( !rc || rc =3D=3D -EBADF || rc =3D=3D -ENODEV )
> +            continue;
> +
> +        return rc;
> +    }
> +
> +    return 0;
> +}
> +
> +#ifdef CONFIG_ACPI
> +static int __init acpi_pci_init(void)
> +{
> +    printk(XENLOG_ERR "ACPI pci init not supported \n");
> +    return -EOPNOTSUPP;
> +}
> +#else
> +static int __init acpi_pci_init(void)
> +{
> +    return -EINVAL;
> +}
> +#endif
> +
> +static int __init pci_init(void)
> +{
> +    pci_segments_init();
> +
> +    if ( acpi_disabled )
> +        return dt_pci_init();
> +    else
> +        return acpi_pci_init();
> +}
> +__initcall(pci_init);
> +
> /*
>  * Local variables:
>  * mode: C
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index ee7cff2d44..5ecd5e7bd1 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -34,6 +34,7 @@ enum device_class
>     DEVICE_SERIAL,
>     DEVICE_IOMMU,
>     DEVICE_GIC,
> +    DEVICE_PCI,
>     /* Use for error */
>     DEVICE_UNKNOWN,
> };
> --=20
> 2.17.1
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:37:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:37:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198716.352357 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU9G-0003rC-6Q; Wed, 29 Sep 2021 07:37:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198716.352357; Wed, 29 Sep 2021 07:37:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVU9G-0003r5-3L; Wed, 29 Sep 2021 07:37:02 +0000
Received: by outflank-mailman (input) for mailman id 198716;
 Wed, 29 Sep 2021 07:37:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVU8o-00025H-Lm
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:36:34 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.70]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e32ab1ce-fd50-43ee-86df-dacafeefb7d6;
 Wed, 29 Sep 2021 07:36:29 +0000 (UTC)
Received: from AS9PR06CA0355.eurprd06.prod.outlook.com (2603:10a6:20b:466::22)
 by PAXPR08MB6766.eurprd08.prod.outlook.com (2603:10a6:102:136::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 07:36:26 +0000
Received: from AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:466:cafe::36) by AS9PR06CA0355.outlook.office365.com
 (2603:10a6:20b:466::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 29 Sep 2021 07:36:26 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT050.mail.protection.outlook.com (10.152.17.47) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 29 Sep 2021 07:36:26 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Wed, 29 Sep 2021 07:36:21 +0000
Received: from 48f8ed936d97.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 21236CCF-406B-4EB4-9E1F-1BE12C480568.1; 
 Wed, 29 Sep 2021 07:36:10 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 48f8ed936d97.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 07:36:10 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5804.eurprd08.prod.outlook.com (2603:10a6:102:83::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 07:36:07 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 07:36:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e32ab1ce-fd50-43ee-86df-dacafeefb7d6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pMKp31vnIj/RtCFImjfzeDtmcIxvXYsVsVjToRnQjT8=;
 b=AyonFObRZXTRge3B2A+s3kyR5802c4qRBfL1NjIE8Vsqbba5zO6joLD36sl7McfgDtUwv6b/SBrWL57XpaXV+MoyGedXsMksI4xzYdKPPFL/UjJ0OztNl4Omz3UIutVdTjgRC76b7+SUhYZfcKwdw3ztzEWbXMg24YA6wmEoKCI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 340cb0bfd07c6147
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=D9D6yoUY5Ur1OnsmVcxOmIPDxih79rElC5g0NfjI7F5D26zdRehcmqYs2kaAQvkJR9wJOa/cPyyywMH9FTqJskBughR0yl9QLAqnhorLgyxSNqzFdGgZ5tTunjqMn7+lulijy591lSoffJTyMJFVSfXk3K8gmHmJ4g4BDk7sKaKspbH3U1eunvVbbIBhA3EegB2Ohb9VDKA4Y20Pd8aGVodFg0CP5zrU/bOsehIXMMx+Wq48SPoqFJa+A9LWBFkmMg0IHLD22CB6u9gw4s3DcVe8EDFgXxbtb1w6irSAvAtSqkJOeVmJNrNVAesrwmn+l9AR2dLT/a+dqdA5lymgpA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pMKp31vnIj/RtCFImjfzeDtmcIxvXYsVsVjToRnQjT8=;
 b=EYfrjK0iONFljEf5SWoJrJZYGxUEHkddKFpwzAWxduocyugKGSdOQ6eQptrG4W+rVxuWIt4cnqEICGatTigc6EYkXF2Bhis/cAdDXZVIeefYDdWKRUy+23abO+s9LjE67A2SSoPoSX9sNAGGn2+rRpMy4VH71UASO7wawk9PnLGB1r6uj6SnM85AZ2LLt5nLBSa4qg4e9bkQsJHCG7wlDszwATzyazXMqg1AadHNTUehGmKgFOkZE0Lma/pmcGNraErnygRM0H6339nTevFmuUWaKNhY3um9weayvMsn0zAMRCaiz9BZmttNUBxNczZT8WWi4PILDI4VU2GEQDiptA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pMKp31vnIj/RtCFImjfzeDtmcIxvXYsVsVjToRnQjT8=;
 b=AyonFObRZXTRge3B2A+s3kyR5802c4qRBfL1NjIE8Vsqbba5zO6joLD36sl7McfgDtUwv6b/SBrWL57XpaXV+MoyGedXsMksI4xzYdKPPFL/UjJ0OztNl4Omz3UIutVdTjgRC76b7+SUhYZfcKwdw3ztzEWbXMg24YA6wmEoKCI=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>
Subject: Re: [PATCH v3 10/17] xen/arm: Add cmdline boot option
 "pci-passthrough = <boolean>"
Thread-Topic: [PATCH v3 10/17] xen/arm: Add cmdline boot option
 "pci-passthrough = <boolean>"
Thread-Index: AQHXtJXG05lNP2fKPECPSZT9mpQWvau6oCcA
Date: Wed, 29 Sep 2021 07:36:07 +0000
Message-ID: <F5D5D147-432A-47B4-B251-BE0786345671@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4e058d69-a9a7-4a9c-896e-08d9831bd825
x-ms-traffictypediagnostic: PR3PR08MB5804:|PAXPR08MB6766:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB67668B190B2D427E3EAFAD809DA99@PAXPR08MB6766.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +7zB/5AGzBjBNDaOCB4BD4pxedI7WOlc/PUQEVygRR6eQAAq9L8oyGa2Amj/XVZtguChVV83I0jJvQyRm61O4NuJ3OLr3F+L4HjAMf314cBY2T6E2Qsx7rYLnVCsO0/XcBRQf/FebciDAlypvHswhEqY+uwDyWqmDACQcW+sEXl0MAYiVt9I6443+6L+msBo9UQsj+O4tzieS8LS3rLRFJ3uStp+UPJriQY+gqqYDOd1cveq56W82aOxQkPzrS4cED2K6KKM0qTzBXsopFHLkb4zY5bHw1vZ9QQHheiFgf/wKrSdXdGUZ/4Jg7NwdvAsx3czTtPDH0EpRWhDp0rmDMzl1qn3htjZLL1CSz5XsMEfVe/0eZ6PQXlvOqpo1x15IfRnAlitR5kDIoTXHemlxJy8ys85pICZS74A0v6ltQHUWfVr/8wLKgirU/Cmhi9lbNZNHD6nevuNrbfhrau+2PvgXlHPZ/HNWATLidJVpGvh7S28QcZx992JRL7oGorpx4AHvFHGosPXa7pOEitd9kLclqYk8kVmAvL3Le8luu+7Tu3Hf1Nb9VWr06wuMJ1+lMZFE6p7aMaqR5hx8/MF9EdLDqDh/ZMX/5X6gwpME3zQlluyr0xjoYjnWvo1NC9ZhsrvrC0DcbgYGR5J1RgMozCnZk+xqORCvc5eR9lu5aUhHInaAqDKYeNiWO8Vc+xsKtFZIcfxpVlYfZvjsPnqhDvR9fKuwhAab49rnCs+vrbj0xFs7FpUZGhx6kKB5zsc
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(2616005)(316002)(6512007)(37006003)(2906002)(4326008)(38100700002)(122000001)(36756003)(33656002)(5660300002)(6486002)(71200400001)(7416002)(6636002)(54906003)(83380400001)(186003)(76116006)(91956017)(66476007)(66946007)(26005)(86362001)(6506007)(53546011)(8676002)(66556008)(508600001)(6862004)(8936002)(64756008)(66446008)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <D1ABB6CB3146E04B9D86AD68BDEE1797@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5804
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6eed6070-aa0f-46e1-5430-08d9831bccde
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F2wbOTFPk94EaPQbjk4JluIyxjRMLREBQDH7Z10i3KJuJmjEYq34Y92nT1v/BkDTX851053IYPehuyyWBei3YlPlc6hsMpjRVoUdCecw2ctvkeat/yt+FmnWW7VcSE0OXiw3frKrki7mH2JJ6ClwNSwwWE3tHFJerTLNDdv9BtMaDmtD7lLo5rhnMC2Brt+Trkc1c5CEzB11//MhSpjGMBeYJrgQPCJ1rAgLd1HeAbidbYrnLrpJYwxDyh3KImrAaLqALCWCbdiPn4nIgH9yhRUx0YDhuxQS7a3cXiUl1BVWugY7Hsxelojieq6qFrEQ9/vRB7bMyrNC3fjTDDm5MoiyC+iY0Re1K/ONmrAXK+gofQci1WIVwqhGp1jR6A3ouqfpERBiaDBofgwL7B+dOuKdYkfbRr7GPcifKvxcAVuLVqCirLa1LYkaXRPkst4xcN99pf7fF5lJV1KyQmrsQpKC1YDwh0yjGVFo0cUojQ/DTQL9xODPPHL/1iST/l6elx9owhwSnMopguhJg2ii7wI6xBRGkhq6rxLvv1zmwCYV90sVdFBbawxN5HNGxEbAU8KOnpZzQgOS1ZKQCWyTFyIA9YIp5kPme1X371Oujp2h84Bg1r91Mkywx3F7fgrJmWi7Jj2sOBp/EIhB9gTrP9tS670EhFGS6FIj4ZpwTtWg5xAMHamklacYUT8PG5yfq6cPp8GsoAH/dVVblhUB2Q==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(6486002)(508600001)(6512007)(37006003)(86362001)(2906002)(8676002)(33656002)(107886003)(2616005)(5660300002)(54906003)(4326008)(6862004)(53546011)(26005)(356005)(81166007)(336012)(316002)(36756003)(186003)(83380400001)(47076005)(6636002)(70586007)(70206006)(82310400003)(6506007)(8936002)(36860700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:36:26.2780
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e058d69-a9a7-4a9c-896e-08d9831bd825
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6766

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Add cmdline boot option "pci-passthrough =3D =3D <boolean>" to enable or
> disable the PCI passthrough support on ARM.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Remove "define pci_passthrough_enabled (false)"
> - Fixed coding style and minor comment
> Change in v2:
> - Add option in xen-command-line.pandoc
> - Change pci option to pci-passthrough
> - modify option from custom_param to boolean param
> ---
> docs/misc/xen-command-line.pandoc |  7 +++++++
> xen/arch/arm/pci/pci.c            | 14 ++++++++++++++
> xen/common/physdev.c              |  6 ++++++
> xen/include/asm-arm/pci.h         | 11 +++++++++++
> xen/include/asm-x86/pci.h         |  8 ++++++++
> 5 files changed, 46 insertions(+)
>=20
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-li=
ne.pandoc
> index 177e656f12..c8bf96ccf8 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1808,6 +1808,13 @@ All numbers specified must be hexadecimal ones.
>=20
> This option can be specified more than once (up to 8 times at present).
>=20
> +### pci-passthrough (arm)
> +> `=3D <boolean>`
> +
> +> Default: `false`
> +
> +Flag to enable or disable support for PCI passthrough
> +
> ### pcid (x86)
>> `=3D <boolean> | xpti=3D<bool>`
>=20
> diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
> index e359bab9ea..84d8f0d634 100644
> --- a/xen/arch/arm/pci/pci.c
> +++ b/xen/arch/arm/pci/pci.c
> @@ -16,6 +16,7 @@
> #include <xen/device_tree.h>
> #include <xen/errno.h>
> #include <xen/init.h>
> +#include <xen/param.h>
> #include <xen/pci.h>
>=20
> /*
> @@ -62,8 +63,21 @@ static int __init acpi_pci_init(void)
> }
> #endif
>=20
> +/*
> + * By default pci passthrough is disabled
> + */
> +bool __read_mostly pci_passthrough_enabled =3D false;
> +boolean_param("pci-passthrough", pci_passthrough_enabled);
> +
> static int __init pci_init(void)
> {
> +    /*
> +     * Enable PCI passthrough when has been enabled explicitly
> +     * (pci-passthrough=3Don)
> +     */
> +    if ( !pci_passthrough_enabled )
> +        return 0;
> +
>     pci_segments_init();
>=20
>     if ( acpi_disabled )
> diff --git a/xen/common/physdev.c b/xen/common/physdev.c
> index 20a5530269..2d5fc886fc 100644
> --- a/xen/common/physdev.c
> +++ b/xen/common/physdev.c
> @@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(voi=
d) arg)
>         struct pci_dev_info pdev_info;
>         nodeid_t node;
>=20
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>         ret =3D -EFAULT;
>         if ( copy_from_guest(&add, arg, 1) !=3D 0 )
>             break;
> @@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(voi=
d) arg)
>     case PHYSDEVOP_pci_device_remove: {
>         struct physdev_pci_device dev;
>=20
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;
> +
>         ret =3D -EFAULT;
>         if ( copy_from_guest(&dev, arg, 1) !=3D 0 )
>             break;
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 7dd9eb3dba..0cf849e26f 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -19,14 +19,25 @@
>=20
> #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
>=20
> +extern bool_t pci_passthrough_enabled;
> +
> /* Arch pci dev struct */
> struct arch_pci_dev {
>     struct device dev;
> };
>=20
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return pci_passthrough_enabled;
> +}
> #else   /*!CONFIG_HAS_PCI*/
>=20
> struct arch_pci_dev { };
>=20
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return false;
> +}
> +
> #endif  /*!CONFIG_HAS_PCI*/
> #endif /* __ARM_PCI_H__ */
> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
> index cc05045e9c..3f806ce7a8 100644
> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned=
 int *seg,
> extern int pci_mmcfg_config_num;
> extern struct acpi_mcfg_allocation *pci_mmcfg_config;
>=20
> +/*
> + * Unlike ARM, PCI passthrough is always enabled for x86.
> + */
> +static always_inline bool is_pci_passthrough_enabled(void)
> +{
> +    return true;
> +}
> +
> #endif /* __X86_PCI_H__ */
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:46:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:46:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198726.352367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIL-0005Me-4A; Wed, 29 Sep 2021 07:46:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198726.352367; Wed, 29 Sep 2021 07:46:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIL-0005MX-13; Wed, 29 Sep 2021 07:46:25 +0000
Received: by outflank-mailman (input) for mailman id 198726;
 Wed, 29 Sep 2021 07:46:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVUIK-0005MR-5w
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:46:24 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5759d530-20f9-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 07:46:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 58885224FB;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2B41513A81;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 9FUICc4ZVGFRdwAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 07:46:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5759d530-20f9-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632901582; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=1Tc7GaYAsTN7KMj95qPWPS6UvvWEuv9iFEBZxd/LJPY=;
	b=MryowTe2ECgIzvHCnP/meR5mUTgV4qbfb+zerpwhOKS6bgOAIM0Q2wXJfxtlw2fp7/NA0h
	LKYQ27zK8mW7dRCp78Dd4Ub9TIno1X/NVrbvkbifEB8F2R79tGLu6C0YnVo0/pBkI7F3bC
	wpeiZqleBJTS94O32zuD/8/qBteJ8M0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 0/3] include/public: update usbif.h
Date: Wed, 29 Sep 2021 09:46:17 +0200
Message-Id: <20210929074620.28495-1-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add some missing defines and documentation to the pvUSB header file.

Changes in V2:
- add patch 3
- minor fixes in patch 1

Juergen Gross (3):
  include/public: add possible status values to usbif.h
  include/public: add better interface description to usbif.h
  include/public: fix style of usbif.h

 xen/include/public/io/usbif.h | 377 ++++++++++++++++++++++++----------
 1 file changed, 274 insertions(+), 103 deletions(-)

-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:46:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:46:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198727.352379 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIQ-0005ea-GR; Wed, 29 Sep 2021 07:46:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198727.352379; Wed, 29 Sep 2021 07:46:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIQ-0005eR-Cu; Wed, 29 Sep 2021 07:46:30 +0000
Received: by outflank-mailman (input) for mailman id 198727;
 Wed, 29 Sep 2021 07:46:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVUIP-0005MR-4R
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:46:29 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 575f9920-20f9-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 07:46:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 69CB91FFD6;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 539081407B;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id 6JtFE84ZVGFRdwAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 07:46:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 575f9920-20f9-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632901582; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=fBKFAtiMuRXfQlNUaLpeC377i///8nXozdU/6NrtRs8=;
	b=Z4zCcwx+Rifii1g8JPWcVyg40ABUdEZxBKuk8fterrpmuqYBmiKXC3O95eM3/xJQOt/Ekg
	kBEqohA6Hu0QbhOJnWcatZByfRpmeBP6fHArdsWd1EHIfgXdEaLSRbZidJpnCZdeYqI5XX
	ofUTt0msxDDFGhOz11xSKelLBlQpiew=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 1/3] include/public: add possible status values to usbif.h
Date: Wed, 29 Sep 2021 09:46:18 +0200
Message-Id: <20210929074620.28495-2-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210929074620.28495-1-jgross@suse.com>
References: <20210929074620.28495-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The interface definition of PV USB devices is lacking the specification
of possible values of the status field in a response. Those are
negative errno values as used in Linux, so they might differ in other
OS's. Specify them via appropriate defines.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- add parentheses to macro values (Jan Beulich)
---
 xen/include/public/io/usbif.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
index c6a58639d6..96b9fb661d 100644
--- a/xen/include/public/io/usbif.h
+++ b/xen/include/public/io/usbif.h
@@ -221,6 +221,13 @@ struct usbif_urb_response {
 	uint16_t id; /* request id */
 	uint16_t start_frame;  /* start frame (ISO) */
 	int32_t status; /* status (non-ISO) */
+#define USBIF_STATUS_OK		0
+#define USBIF_STATUS_NODEV	(-19)
+#define USBIF_STATUS_INVAL	(-22)
+#define USBIF_STATUS_STALL	(-32)
+#define USBIF_STATUS_IOERROR	(-71)
+#define USBIF_STATUS_BABBLE	(-75)
+#define USBIF_STATUS_SHUTDOWN	(-108)
 	int32_t actual_length; /* actual transfer length */
 	int32_t error_count; /* number of ISO errors */
 };
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:46:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:46:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198729.352389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIV-0005zW-ON; Wed, 29 Sep 2021 07:46:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198729.352389; Wed, 29 Sep 2021 07:46:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIV-0005zP-L5; Wed, 29 Sep 2021 07:46:35 +0000
Received: by outflank-mailman (input) for mailman id 198729;
 Wed, 29 Sep 2021 07:46:34 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVUIU-0005MR-4U
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:46:34 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 57685452-20f9-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 07:46:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 8696F202F0;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6FB5D13A81;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id iP03Gs4ZVGFRdwAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 07:46:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57685452-20f9-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632901582; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=yC6/hWmQnsbsA7pZUB2nnPsLfC8C4S0jE8iXwiOaPUU=;
	b=n/YMEYpI24SqMHiTAz0ZdTL89EXqsDujOjOGdMH8c1fHX+Za1FwPRrZNsyk0H+i6c+AMw5
	rVe6ZcIsftimmTQNoACHRWVwAosfVhujWvnRslGsUzDfd9uLZ6HICtd9fCchzp6hGeQNjX
	YTQ97diZRaPdFdFUIssZ4qYKMo06CBc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 2/3] include/public: add better interface description to usbif.h
Date: Wed, 29 Sep 2021 09:46:19 +0200
Message-Id: <20210929074620.28495-3-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210929074620.28495-1-jgross@suse.com>
References: <20210929074620.28495-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The PV USB protocol is poorly described. Add a more detailed
description to the usbif.h header file.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xen/include/public/io/usbif.h | 164 ++++++++++++++++++++++++++++++++++
 1 file changed, 164 insertions(+)

diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
index 96b9fb661d..dd378bcba5 100644
--- a/xen/include/public/io/usbif.h
+++ b/xen/include/public/io/usbif.h
@@ -32,6 +32,34 @@
 #include "../grant_table.h"
 
 /*
+ * Detailed Interface Description
+ * ==============================
+ * The pvUSB interface is using a split driver design: a frontend driver in
+ * the guest and a backend driver in a driver domain (normally dom0) having
+ * access to the physical USB device(s) being passed to the guest.
+ *
+ * The frontend and backend drivers use XenStore to initiate the connection
+ * between them, the I/O activity is handled via two shared ring pages and an
+ * event channel. As the interface between frontend and backend is at the USB
+ * host connector level, multiple (up to 31) physical USB devices can be
+ * handled by a single connection.
+ *
+ * The Xen pvUSB device name is "qusb", so the frontend's XenStore entries are
+ * to be found under "device/qusb", while the backend's XenStore entries are
+ * under "backend/<guest-dom-id>/qusb".
+ *
+ * When a new pvUSB connection is established, the frontend needs to setup the
+ * two shared ring pages for communication and the event channel. The ring
+ * pages need to be made available to the backend via the grant table
+ * interface.
+ *
+ * One of the shared ring pages is used by the backend to inform the frontend
+ * about USB device plug events (device to be added or removed). This is the
+ * "conn-ring".
+ *
+ * The other ring page is used for USB I/O communication (requests and
+ * responses). This is the "urb-ring".
+ *
  * Feature and Parameter Negotiation
  * =================================
  * The two halves of a Xen pvUSB driver utilize nodes within the XenStore to
@@ -99,6 +127,142 @@
  *      The machine ABI rules governing the format of all ring request and
  *      response structures.
  *
+ * Protocol Description
+ * ====================
+ *
+ *-------------------------- USB device plug events --------------------------
+ *
+ * USB device plug events are send via the "conn-ring" shared page. As only
+ * events are being sent, the respective requests from the frontend to the
+ * backend are just dummy ones.
+ * The events sent to the frontend have the following layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |    portnum     |     speed      | 4
+ * +----------------+----------------+----------------+----------------+
+ *   id - uint16_t, event id (taken from the actual frontend dummy request)
+ *   portnum - uint8_t, port number (1 ... 31)
+ *   speed - uint8_t, device USBIF_SPEED_*, USBIF_SPEED_NONE == unplug
+ *
+ * The dummy request:
+ *         0                1        octet
+ * +----------------+----------------+
+ * |               id                | 2
+ * +----------------+----------------+
+ *   id - uint16_t, guest supplied value (no need for being unique)
+ *
+ *-------------------------- USB I/O request ---------------------------------
+ *
+ * A single USB I/O request on the "urb-ring" has the following layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |         nr_buffer_segs          | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                               pipe                                | 8
+ * +----------------+----------------+----------------+----------------+
+ * |         transfer_flags          |          buffer_length          | 12
+ * +----------------+----------------+----------------+----------------+
+ * |                       request type specific                       | 16
+ * |                               data                                | 20
+ * +----------------+----------------+----------------+----------------+
+ * |                              seg[0]                               | 24
+ * |                               data                                | 28
+ * +----------------+----------------+----------------+----------------+
+ * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
+ * +----------------+----------------+----------------+----------------+
+ * |             seg[USBIF_MAX_SEGMENTS_PER_REQUEST - 1]               | 144
+ * |                               data                                | 148
+ * +----------------+----------------+----------------+----------------+
+ * Bit field bit number 0 is always least significant bit, undefined bits must
+ * be zero.
+ *   id - uint16_t, guest supplied value
+ *   nr_buffer_segs - uint16_t, number of segment entries in seg[] array
+ *   pipe - uint32_t, bit field with multiple information:
+ *     bits 0-4: port request to send to
+ *     bit 5: unlink request with specified id (cancel I/O) if set (see below)
+ *     bit 7: direction (1 = read from device)
+ *     bits 8-14: device number on port
+ *     bits 15-18: endpoint of device
+ *     bits 30-31: request type: 00 = isochronous, 01 = interrupt,
+ *                               10 = control, 11 = bulk
+ *   transfer_flags - uint16_t, bit field with processing flags:
+ *     bit 0: less data than specified allowed
+ *   buffer_length - uint16_t, total length of data
+ *   request type specific data - 8 bytes, see below
+ *   seg[] - array with 8 byte elements, see below
+ *
+ * Request type specific data for isochronous request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |            interval             |           start_frame           | 4
+ * +----------------+----------------+----------------+----------------+
+ * |       number_of_packets         |       nr_frame_desc_segs        | 8
+ * +----------------+----------------+----------------+----------------+
+ *   interval - uint16_t, time interval in msecs between frames
+ *   start_frame - uint16_t, start frame number
+ *   number_of_packets - uint16_t, number of packets to transfer
+ *   nr_frame_desc_segs - uint16_t number of seg[] frame descriptors elements
+ *
+ * Request type specific data for interrupt request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |            interval             |                0                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *   interval - uint16_t, time in msecs until interruption
+ *
+ * Request type specific data for control request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                      data of setup packet                         | 4
+ * |                                                                   | 8
+ * +----------------+----------------+----------------+----------------+
+ *
+ * Request type specific data for bulk request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 4
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *
+ * Request type specific data for unlink request:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |           unlink_id             |                0                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                                 0                                 | 8
+ * +----------------+----------------+----------------+----------------+
+ *   unlink_id - uint16_t, request id of request to terminate
+ *
+ * seg[] array element layout:
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |                               gref                                | 4
+ * +----------------+----------------+----------------+----------------+
+ * |             offset              |             length              | 8
+ * +----------------+----------------+----------------+----------------+
+ *   gref - uint32_t, grant reference of buffer page
+ *   offset - uint16_t, offset of buffer start in page
+ *   length - uint16_t, length of buffer in page
+ *
+ *-------------------------- USB I/O response --------------------------------
+ *
+ *         0                1                 2               3        octet
+ * +----------------+----------------+----------------+----------------+
+ * |               id                |          start_frame            | 4
+ * +----------------+----------------+----------------+----------------+
+ * |                              status                               | 8
+ * +----------------+----------------+----------------+----------------+
+ * |                          actual_length                            | 12
+ * +----------------+----------------+----------------+----------------+
+ * |                           error_count                             | 16
+ * +----------------+----------------+----------------+----------------+
+ *   id - uint16_t, id of the request this response belongs to
+ *   start_frame - uint16_t, start_frame this response (iso requests only)
+ *   status - int32_t, USBIF_STATUS_* (non-iso requests)
+ *   actual_length - uint32_t, actual size of data transferred
+ *   error_count - uint32_t, number of errors (iso requests)
  */
 
 enum usb_spec_version {
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:46:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:46:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198731.352400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIa-0006Lx-1g; Wed, 29 Sep 2021 07:46:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198731.352400; Wed, 29 Sep 2021 07:46:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUIZ-0006Lm-UK; Wed, 29 Sep 2021 07:46:39 +0000
Received: by outflank-mailman (input) for mailman id 198731;
 Wed, 29 Sep 2021 07:46:39 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=17VY=OT=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVUIZ-0005MR-4k
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:46:39 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 575f9921-20f9-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 07:46:23 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A43DB202F7;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8CA9D13A81;
 Wed, 29 Sep 2021 07:46:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id mNlYIc4ZVGFRdwAAMHmgww
 (envelope-from <jgross@suse.com>); Wed, 29 Sep 2021 07:46:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 575f9921-20f9-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1632901582; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=U60TGLLCuRAteDFsQE6aIpjaYyccPZyDPUn8cXp1mP4=;
	b=KnE/iJi48/IW9TXMt/sZTLIPKEK60yyhqJ7GZ/YYpfp3DXKJWvyjQd8GAu01TYbBn1u9iz
	G6qBn6QHfVQ6XY8fPBz5Svb2Dob0J8aONL9nTVm31HeNut0wGDrTEDX86efCW5+K5YvTgo
	PNz1EBtGky5nVuggn2xCGSmRU/TzCxw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 3/3] include/public: fix style of usbif.h
Date: Wed, 29 Sep 2021 09:46:20 +0200
Message-Id: <20210929074620.28495-4-jgross@suse.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20210929074620.28495-1-jgross@suse.com>
References: <20210929074620.28495-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

usbif.h is violating the Xen hypervisor coding style. Fix that.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- new patch (Jan Beulich)
---
 xen/include/public/io/usbif.h | 196 +++++++++++++++++-----------------
 1 file changed, 98 insertions(+), 98 deletions(-)

diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
index dd378bcba5..c0a552e195 100644
--- a/xen/include/public/io/usbif.h
+++ b/xen/include/public/io/usbif.h
@@ -266,134 +266,134 @@
  */
 
 enum usb_spec_version {
-	USB_VER_UNKNOWN = 0,
-	USB_VER_USB11,
-	USB_VER_USB20,
-	USB_VER_USB30,	/* not supported yet */
+    USB_VER_UNKNOWN = 0,
+    USB_VER_USB11,
+    USB_VER_USB20,
+    USB_VER_USB30,    /* not supported yet */
 };
 
 /*
  *  USB pipe in usbif_request
  *
- *  - port number:	bits 0-4
- *				(USB_MAXCHILDREN is 31)
+ *  - port number:      bits 0-4
+ *                              (USB_MAXCHILDREN is 31)
  *
- *  - operation flag:	bit 5
- *				(0 = submit urb,
- *				 1 = unlink urb)
+ *  - operation flag:   bit 5
+ *                              (0 = submit urb,
+ *                               1 = unlink urb)
  *
- *  - direction:		bit 7
- *				(0 = Host-to-Device [Out]
- *				 1 = Device-to-Host [In])
+ *  - direction:        bit 7
+ *                              (0 = Host-to-Device [Out]
+ *                               1 = Device-to-Host [In])
  *
- *  - device address:	bits 8-14
+ *  - device address:   bits 8-14
  *
- *  - endpoint:		bits 15-18
+ *  - endpoint:         bits 15-18
  *
- *  - pipe type:	bits 30-31
- *				(00 = isochronous, 01 = interrupt,
- *				 10 = control, 11 = bulk)
+ *  - pipe type:        bits 30-31
+ *                              (00 = isochronous, 01 = interrupt,
+ *                               10 = control, 11 = bulk)
  */
 
-#define USBIF_PIPE_PORT_MASK	0x0000001f
-#define USBIF_PIPE_UNLINK	0x00000020
-#define USBIF_PIPE_DIR		0x00000080
-#define USBIF_PIPE_DEV_MASK	0x0000007f
-#define USBIF_PIPE_DEV_SHIFT	8
-#define USBIF_PIPE_EP_MASK	0x0000000f
-#define USBIF_PIPE_EP_SHIFT	15
-#define USBIF_PIPE_TYPE_MASK	0x00000003
-#define USBIF_PIPE_TYPE_SHIFT	30
-#define USBIF_PIPE_TYPE_ISOC	0
-#define USBIF_PIPE_TYPE_INT	1
-#define USBIF_PIPE_TYPE_CTRL	2
-#define USBIF_PIPE_TYPE_BULK	3
+#define USBIF_PIPE_PORT_MASK    0x0000001f
+#define USBIF_PIPE_UNLINK       0x00000020
+#define USBIF_PIPE_DIR          0x00000080
+#define USBIF_PIPE_DEV_MASK     0x0000007f
+#define USBIF_PIPE_DEV_SHIFT    8
+#define USBIF_PIPE_EP_MASK      0x0000000f
+#define USBIF_PIPE_EP_SHIFT     15
+#define USBIF_PIPE_TYPE_MASK    0x00000003
+#define USBIF_PIPE_TYPE_SHIFT   30
+#define USBIF_PIPE_TYPE_ISOC    0
+#define USBIF_PIPE_TYPE_INT     1
+#define USBIF_PIPE_TYPE_CTRL    2
+#define USBIF_PIPE_TYPE_BULK    3
 
-#define usbif_pipeportnum(pipe)			((pipe) & USBIF_PIPE_PORT_MASK)
-#define usbif_setportnum_pipe(pipe, portnum)	((pipe) | (portnum))
+#define usbif_pipeportnum(pipe)                 ((pipe) & USBIF_PIPE_PORT_MASK)
+#define usbif_setportnum_pipe(pipe, portnum)    ((pipe) | (portnum))
 
-#define usbif_pipeunlink(pipe)			((pipe) & USBIF_PIPE_UNLINK)
-#define usbif_pipesubmit(pipe)			(!usbif_pipeunlink(pipe))
-#define usbif_setunlink_pipe(pipe)		((pipe) | USBIF_PIPE_UNLINK)
+#define usbif_pipeunlink(pipe)                  ((pipe) & USBIF_PIPE_UNLINK)
+#define usbif_pipesubmit(pipe)                  (!usbif_pipeunlink(pipe))
+#define usbif_setunlink_pipe(pipe)              ((pipe) | USBIF_PIPE_UNLINK)
 
-#define usbif_pipein(pipe)			((pipe) & USBIF_PIPE_DIR)
-#define usbif_pipeout(pipe)			(!usbif_pipein(pipe))
+#define usbif_pipein(pipe)                      ((pipe) & USBIF_PIPE_DIR)
+#define usbif_pipeout(pipe)                     (!usbif_pipein(pipe))
 
-#define usbif_pipedevice(pipe)			\
-		(((pipe) >> USBIF_PIPE_DEV_SHIFT) & USBIF_PIPE_DEV_MASK)
+#define usbif_pipedevice(pipe)                  \
+        (((pipe) >> USBIF_PIPE_DEV_SHIFT) & USBIF_PIPE_DEV_MASK)
 
-#define usbif_pipeendpoint(pipe)		\
-		(((pipe) >> USBIF_PIPE_EP_SHIFT) & USBIF_PIPE_EP_MASK)
+#define usbif_pipeendpoint(pipe)                \
+        (((pipe) >> USBIF_PIPE_EP_SHIFT) & USBIF_PIPE_EP_MASK)
 
-#define usbif_pipetype(pipe)			\
-		(((pipe) >> USBIF_PIPE_TYPE_SHIFT) & USBIF_PIPE_TYPE_MASK)
-#define usbif_pipeisoc(pipe)	(usbif_pipetype(pipe) == USBIF_PIPE_TYPE_ISOC)
-#define usbif_pipeint(pipe)	(usbif_pipetype(pipe) == USBIF_PIPE_TYPE_INT)
-#define usbif_pipectrl(pipe)	(usbif_pipetype(pipe) == USBIF_PIPE_TYPE_CTRL)
-#define usbif_pipebulk(pipe)	(usbif_pipetype(pipe) == USBIF_PIPE_TYPE_BULK)
+#define usbif_pipetype(pipe)                    \
+        (((pipe) >> USBIF_PIPE_TYPE_SHIFT) & USBIF_PIPE_TYPE_MASK)
+#define usbif_pipeisoc(pipe)    (usbif_pipetype(pipe) == USBIF_PIPE_TYPE_ISOC)
+#define usbif_pipeint(pipe)     (usbif_pipetype(pipe) == USBIF_PIPE_TYPE_INT)
+#define usbif_pipectrl(pipe)    (usbif_pipetype(pipe) == USBIF_PIPE_TYPE_CTRL)
+#define usbif_pipebulk(pipe)    (usbif_pipetype(pipe) == USBIF_PIPE_TYPE_BULK)
 
 #define USBIF_MAX_SEGMENTS_PER_REQUEST (16)
-#define USBIF_MAX_PORTNR	31
-#define USBIF_RING_SIZE	4096
+#define USBIF_MAX_PORTNR        31
+#define USBIF_RING_SIZE         4096
 
 /*
  * RING for transferring urbs.
  */
 struct usbif_request_segment {
-	grant_ref_t gref;
-	uint16_t offset;
-	uint16_t length;
+    grant_ref_t gref;
+    uint16_t offset;
+    uint16_t length;
 };
 
 struct usbif_urb_request {
-	uint16_t id; /* request id */
-	uint16_t nr_buffer_segs; /* number of urb->transfer_buffer segments */
+    uint16_t id;                  /* request id */
+    uint16_t nr_buffer_segs;      /* number of urb->transfer_buffer segments */
 
-	/* basic urb parameter */
-	uint32_t pipe;
-	uint16_t transfer_flags;
-#define USBIF_SHORT_NOT_OK	0x0001
-	uint16_t buffer_length;
-	union {
-		uint8_t ctrl[8]; /* setup_packet (Ctrl) */
+    /* basic urb parameter */
+    uint32_t pipe;
+    uint16_t transfer_flags;
+#define USBIF_SHORT_NOT_OK      0x0001
+    uint16_t buffer_length;
+    union {
+        uint8_t ctrl[8];                 /* setup_packet (Ctrl) */
 
-		struct {
-			uint16_t interval; /* maximum (1024*8) in usb core */
-			uint16_t start_frame; /* start frame */
-			uint16_t number_of_packets; /* number of ISO packet */
-			uint16_t nr_frame_desc_segs; /* number of iso_frame_desc segments */
-		} isoc;
+        struct {
+            uint16_t interval;           /* maximum (1024*8) in usb core */
+            uint16_t start_frame;        /* start frame */
+            uint16_t number_of_packets;  /* number of ISO packet */
+            uint16_t nr_frame_desc_segs; /* number of iso_frame_desc segments */
+        } isoc;
 
-		struct {
-			uint16_t interval; /* maximum (1024*8) in usb core */
-			uint16_t pad[3];
-		} intr;
+        struct {
+            uint16_t interval;           /* maximum (1024*8) in usb core */
+            uint16_t pad[3];
+        } intr;
 
-		struct {
-			uint16_t unlink_id; /* unlink request id */
-			uint16_t pad[3];
-		} unlink;
+        struct {
+            uint16_t unlink_id;          /* unlink request id */
+            uint16_t pad[3];
+        } unlink;
 
-	} u;
+    } u;
 
-	/* urb data segments */
-	struct usbif_request_segment seg[USBIF_MAX_SEGMENTS_PER_REQUEST];
+    /* urb data segments */
+    struct usbif_request_segment seg[USBIF_MAX_SEGMENTS_PER_REQUEST];
 };
 typedef struct usbif_urb_request usbif_urb_request_t;
 
 struct usbif_urb_response {
-	uint16_t id; /* request id */
-	uint16_t start_frame;  /* start frame (ISO) */
-	int32_t status; /* status (non-ISO) */
-#define USBIF_STATUS_OK		0
-#define USBIF_STATUS_NODEV	(-19)
-#define USBIF_STATUS_INVAL	(-22)
-#define USBIF_STATUS_STALL	(-32)
-#define USBIF_STATUS_IOERROR	(-71)
-#define USBIF_STATUS_BABBLE	(-75)
-#define USBIF_STATUS_SHUTDOWN	(-108)
-	int32_t actual_length; /* actual transfer length */
-	int32_t error_count; /* number of ISO errors */
+    uint16_t id;           /* request id */
+    uint16_t start_frame;  /* start frame (ISO) */
+    int32_t status;        /* status (non-ISO) */
+#define USBIF_STATUS_OK         0
+#define USBIF_STATUS_NODEV      (-19)
+#define USBIF_STATUS_INVAL      (-22)
+#define USBIF_STATUS_STALL      (-32)
+#define USBIF_STATUS_IOERROR    (-71)
+#define USBIF_STATUS_BABBLE     (-75)
+#define USBIF_STATUS_SHUTDOWN   (-108)
+    int32_t actual_length; /* actual transfer length */
+    int32_t error_count;   /* number of ISO errors */
 };
 typedef struct usbif_urb_response usbif_urb_response_t;
 
@@ -404,18 +404,18 @@ DEFINE_RING_TYPES(usbif_urb, struct usbif_urb_request, struct usbif_urb_response
  * RING for notifying connect/disconnect events to frontend
  */
 struct usbif_conn_request {
-	uint16_t id;
+    uint16_t id;
 };
 typedef struct usbif_conn_request usbif_conn_request_t;
 
 struct usbif_conn_response {
-	uint16_t id; /* request id */
-	uint8_t portnum; /* port number */
-	uint8_t speed; /* usb_device_speed */
-#define USBIF_SPEED_NONE	0
-#define USBIF_SPEED_LOW		1
-#define USBIF_SPEED_FULL	2
-#define USBIF_SPEED_HIGH	3
+    uint16_t id;           /* request id */
+    uint8_t portnum;       /* port number */
+    uint8_t speed;         /* usb_device_speed */
+#define USBIF_SPEED_NONE        0
+#define USBIF_SPEED_LOW         1
+#define USBIF_SPEED_FULL        2
+#define USBIF_SPEED_HIGH        3
 };
 typedef struct usbif_conn_response usbif_conn_response_t;
 
-- 
2.26.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 07:50:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 07:50:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198752.352412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVULy-0000KI-IW; Wed, 29 Sep 2021 07:50:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198752.352412; Wed, 29 Sep 2021 07:50:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVULy-0000KB-FT; Wed, 29 Sep 2021 07:50:10 +0000
Received: by outflank-mailman (input) for mailman id 198752;
 Wed, 29 Sep 2021 07:50:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVULx-0000Jo-DN
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 07:50:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 43fac505-82f1-470b-8cf8-c59989ef13ba;
 Wed, 29 Sep 2021 07:50:08 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-ZQAA83LVNcWOUcel7qB7Tw-1; Wed, 29 Sep 2021 09:50:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Wed, 29 Sep
 2021 07:50:03 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 07:50:03 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM3PR05CA0134.eurprd05.prod.outlook.com (2603:10a6:207:3::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 07:50:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43fac505-82f1-470b-8cf8-c59989ef13ba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632901807;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgShhN3zrmhC7mXQRzGue7jBiclU++igG4SklAgWBnw=;
	b=A0YufSx2yoWNQ9CoXDvWq4VM6GhlDsxZ4wv4z+yx5YCrzzx7Aam4uNPwOXkdI38erVegwk
	M9nzrWmrn/R/ZuLq478mPD9n14nRrJAdJ4wBK9yAKdGohQyl4bblSfrC46FEwbaNXg9Gzt
	1UMoeXXorpeWvIFMigpSs9gQSbvgYlE=
X-MC-Unique: ZQAA83LVNcWOUcel7qB7Tw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WH+tEfrv/hPWC0hGLFBE6u5+VjpDBnPcSubET54bWtCNo1mQdRUsYHJXfQWtq6+27cQ+Hdh2bE48oOu4lP/Nm9ROQaJZ1OJ5JKnDgXHsvNsVqrBHhWcdFGujjGoq3CyQ6KgUsAwmxayrgWSQIa4WvSVaqxS1ZMk7sqg2oTrvaY3IDYI7ZM6113PAZTG5m+Xsrko0V5S2Mgh57u5JB8rSrXMJVjoc+rfUakkiQF+yD6d8AAnCQ0NFUC+BiQwJL8+WP+wkgfVW0ru1d65YJ6I2a6IVb96TvTzgpHEmoLo5HeIV/9jr70mI2LMGS7A8rPKafn3e6Qlf5pawJ7n6I5vbng==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=zgShhN3zrmhC7mXQRzGue7jBiclU++igG4SklAgWBnw=;
 b=VMeHHLDZ5+uAaypsBjzZuq6BkAyu2s7Htk8q5xPKan0MIHXVRVlWpdCf37Hm83SOFo0BliBu29A6rQsXTLmgjZV9cufVP+O9EPyHyoqSPixum7hZ3GZHutSaVKW/xPSPrvbX8U+EPrIih0G+wFq9H8HjIGRF8RXCCir4cejGY9Izc6jgEk6jgAIPk0VdpmCZAjXSFIOichrtRNrwcEYt5Ub7SjemcVgT9mD6YZJcJGEAQOKCNj5Rel5GGbqVeNAXfvHEANgeJfqH9QGYEPP4bQRFpKOLv/sItIO6bdZXDTyqybOK2E1p25kP85Y4Ec18sEJlxMHp/V+OYERYPx0cCw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 2/3] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20210928163209.49611-1-luca.fancellu@arm.com>
 <20210928163209.49611-3-luca.fancellu@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <120aae99-d8f9-eef3-e6ac-b1b9b842083e@suse.com>
Date: Wed, 29 Sep 2021 09:50:00 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210928163209.49611-3-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM3PR05CA0134.eurprd05.prod.outlook.com
 (2603:10a6:207:3::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 590b3a1a-58f9-43bb-1d7c-08d9831dbecb
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB39336895CE238C2DAB3997E5B3A99@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2201;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4IM1OQ9DkGj61JQ4tem0HfR0rjweU6Q2RkT5BvlkZ17GBNX16hEZwe29tqrw8vPnhmUXpRy9Dek+vUF3Q2o9VN8laFDky7dvKXL91Z9AxMmhQ1dfjPiyaVt+AkZRK3JN52Z3DaUWk15889MC/ynie4LU1wbsJ3oYSOaLqkqYbD2bsiAhxBq+d1oEDfYYJFzKruK8DuQgHsXYUEbBv7NP17mN8wsJUJbzq1uRTGU6n6HC3+DWe5FJo/YcMlVx+Cuv0V2bLGXSZnKR6VpGme39BQcLae+5yx6rsOA3qRtPQVN45zhRUmmalUhq0DNivFA/CGGMGe0K0Hx+fZKgn/Why3BanQmyMzB8F/h7ljnvIgLzTrpCitseSxiYrB5BK6c18lttl2BvvWob8oyzq0JoM+ip/Z6CDe/LIO4hNygHJxpS6V3aP1lFZKldn38u4id1VQIJhMd+I0dAPqM+Lh0SClsNQDUoQ5AMCtZk4fJD4yjHk9qBUau5IS+wUfe1F/psVgAVORkY4qGkqTczz+8VkXLiM6upphR3nn8AjLhiPyzO0tKPrFoemkPU8OF9pOBabFQhulbZv1bVjsdjpsKxMdbJ4RmVAZlnCq4cnXmxP54xYWu1N6TMM8gg4tCtZhVI4dNQe4ZyuTbriZFc1GKUV+jONXmLJAwfWO3EkBth7IMl+I+Aw8v1UOCvvPSI6LLK49wdmnq60bOkrC9kev9qYKDnC58VjuTZrtRnUkaPV1eJQS2E0Lrthx3z6IsgtZP+9CCSyFIBTzjQ+CKhdSiiYg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(38100700002)(54906003)(5660300002)(316002)(53546011)(31696002)(16576012)(4326008)(83380400001)(6486002)(6916009)(2616005)(508600001)(36756003)(186003)(66556008)(66476007)(31686004)(66946007)(7416002)(8676002)(2906002)(956004)(26005)(86362001)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bzFyRDlXL3ZYbUhaUnRzcW56SGgySTdYaG52NGI0T1V2b3REekQ4cUJmYitq?=
 =?utf-8?B?SXcyZWF0N2NPZWVVNy9nQ0orNmVBeVU0ajU4ZmoxZHdoOXhjbEhuVlFmR3pC?=
 =?utf-8?B?MkJOWkErRVd1dzVTajY1TjV1ekhDZE5WRkhDTzV1ZldTbnF4dSticndtUUZT?=
 =?utf-8?B?QkgvdGFBSEgzWU9IQmk2SFdyVGRwdC9EV0dhVk1KUDRINjE3RW5MRkZQVWhE?=
 =?utf-8?B?ODVqMW9wQ25EMDlwNys3Zi9OTkgxZEEwR1Jrbk1SOSs1VDJWNmYzMlFkRzk5?=
 =?utf-8?B?NjgyUVFMYnhsd3duc3cxQjQ5THNkWnIrWWZ0VU51UmRkZ2s0VXI0OEJTek1y?=
 =?utf-8?B?WGlNd0UrSXcvSUdscER4cVJBZ2tRWlV3Q0ZRTUZwb1NEUTBCMkl0cnpyZXBO?=
 =?utf-8?B?SGdVa2pyejRRb3R3bUtMYVl5TEVadmg1THp3UURyaEdyY0g0ODIvVG5jdlpr?=
 =?utf-8?B?c2REMnZLZS93czQ5OFRBMGxpZm5VRlE0RzljTER5ZHBiakwxSllQTFdRRU1X?=
 =?utf-8?B?UUJYVmF1SDg5TkJvaVpPUU02WWIwODJ3VHRiVEowU3prSGFVdkJpaXJodm8r?=
 =?utf-8?B?czgzNDFXNzdzUVRZUjhIY1RzY3RLbWFKSkl2NE1FSUpiR0tNdjRxTmtZOWp5?=
 =?utf-8?B?bW9OVm5Obkd3TTBsbTdGMGFzWS84ZDk0Umo1NEZFUUs5MUlweUQ4TzNQdDVl?=
 =?utf-8?B?OFlQeHdkZm9oNUVxQUkvSWpyTVdxSnU4OXA5aVNOYmpQdjlNd0JwWHp6d29w?=
 =?utf-8?B?ZU1BZEo4ek4yUGE0WFJsTDc1QWljU2I0dTNoSUx3OXF3OXY1RXQ1cGRvN2JX?=
 =?utf-8?B?V3RkSElUTDUzNE5jNUNLai9US0U0VXl3NE85U0I3NFdKVVVEVDMwMmJtYU8x?=
 =?utf-8?B?VURRbUZnWUs5Yjl0THROQWV1Ni9MS0dvcGFaUXZKSVBIalkzMGNCRm8rYTFP?=
 =?utf-8?B?bUFVWnFFc3NwMzQyZ2I4dUhkczRBcG9XaXRHUWZVNzRrUmhlZm5yTkJTRlls?=
 =?utf-8?B?RE1MMi90R21VZE0razZEUjFlMDgvTllaeEx5YmtoUnE5ZTd1YWNFNWJ3WGdP?=
 =?utf-8?B?NXYrY0NQUTZkL29XOFQ0WUtwRW1WekxBYWhQWlRkd1B1eTAzWUxFWEJhQmtj?=
 =?utf-8?B?bHBIT2tMb2NPZm11WlBpcEZpSWYrQlVUQTY1N1ROMFEwOFFZci8vVmZjdTUy?=
 =?utf-8?B?SE5HcEE0L3M5UXVLRDFhTTdlc2U0WFdTRElrTnp3TlhOcDl5QkpuMVBleVZC?=
 =?utf-8?B?S0l6WVROR05KOTNlek5WNWxmYnRFOG4rbWd0QUhlUkRKRXYzRzFMaU50TjYv?=
 =?utf-8?B?czB4WkV0eXdwUzBLcngzZzBscGxTV3ZPUC9WNjE4bmhrK3dPelp3aUpxOXR0?=
 =?utf-8?B?c3hlRS9DSmRaVXVOMnJkUURHZThITFFud0ZpbjNDaDZnTXlEaTAzSWEyTjZU?=
 =?utf-8?B?ZlZvTUtUYnRsQWV6TnczTy91VGYxWnVXU2Y3TnlaUC9PSDNYQUc3U3NsYTNu?=
 =?utf-8?B?SVJ1ZTBJR3JtenBaWnNSV3N2YnhvbGxjMjRGTEdISXR2K2FDc3RXcDZXV3Jh?=
 =?utf-8?B?WnZ2czBRNUhrWmkrYzFTaWlBdVBqb29HVWRvTkhneENUdTlmUUtjeFBDR3N5?=
 =?utf-8?B?UHpBRVpwMG5WQWZGOUNHamluQUozQXVRbDFURS9oZCt5MlNtTG1ET3FobU84?=
 =?utf-8?B?enZLTDhHT1pvWFRYaEQzcHlLczhhUFk4ZHdFY0UvK0dFNE1OaGxHR0xzb1lp?=
 =?utf-8?Q?m9Hlg46tx4zeAUdWr9Sp2g96ppRs3UjlRqYM3SK?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 590b3a1a-58f9-43bb-1d7c-08d9831dbecb
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 07:50:03.0137
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KTLJiHi1G60GGzRUUesqZlE/lJmSF0l+mhRVlgBgMq3zYtkHFse4jzvXwDQhZluEIktrgdfrH1VgV7QMGOO7hQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

On 28.09.2021 18:32, Luca Fancellu wrote:
> --- a/xen/arch/x86/efi/efi-boot.h
> +++ b/xen/arch/x86/efi/efi-boot.h
> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
>      efi_bs->FreePool(ptr);
>  }
>  
> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    /* x86 doesn't support device tree boot */
> +    return 0;
> +}

Every time I see this addition I'm getting puzzled. As a result I'm
afraid I now need to finally ask you to do something about this (and
I'm sorry for doing so only now). There would better be no notion of
DT in x86 code, and there would better also not be a need for
architectures not supporting DT to each supply such a stub. Instead
I think you want to put this stub in xen/common/efi/boot.c, inside a
suitable #ifdef.

> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
>      EFI_LOADED_IMAGE *loaded_image;
>      EFI_STATUS status;
> -    unsigned int i, argc;
> -    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
> +    unsigned int i, argc = 0;
> +    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
>      UINTN gop_mode = ~0;
>      EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    int dt_module_found;

I think this variable either wants to be bool or be named differently.

> @@ -1361,12 +1361,26 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +    dt_module_found = efi_arch_check_dt_boot(dir_handle);
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if (dt_module_found < 0)
> +        /* efi_arch_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");

For this use, bool would seem appropriate, but ...

> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !dt_module_found && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");

... this (and my brief looking at the Arm code) rather suggests a
count gets returned, and hence it may want renaming instead. Maybe
simply to dt_modules_found.

Considering the new conditional I also wonder whether the error
message can't end up being misleading on Arm (it certainly should
remain as is on x86).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:01:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:01:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198763.352423 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUWU-0002Xi-TW; Wed, 29 Sep 2021 08:01:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198763.352423; Wed, 29 Sep 2021 08:01:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUWU-0002Xb-QC; Wed, 29 Sep 2021 08:01:02 +0000
Received: by outflank-mailman (input) for mailman id 198763;
 Wed, 29 Sep 2021 08:01:01 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUWT-0002XV-Lq
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:01:01 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ff93d627-ae99-4d62-a9a3-82ddb9c2da3b;
 Wed, 29 Sep 2021 08:01:00 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2105.outbound.protection.outlook.com [104.47.17.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-8-fptSl23KNRiKu2VoaUiReA-1; Wed, 29 Sep 2021 10:00:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4941.eurprd04.prod.outlook.com (2603:10a6:803:5a::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 08:00:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:00:55 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR07CA0025.eurprd07.prod.outlook.com (2603:10a6:208:ac::38) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Wed, 29 Sep 2021 08:00:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff93d627-ae99-4d62-a9a3-82ddb9c2da3b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632902459;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kNCv2JfxAmGrCKSm1zBbx09ONKsK59pscIEd/URrupE=;
	b=Wv6XhOmr1FCYyqWzbbcMR0B/+W7nAEXn7wq6aaEdhd8dWka8G1/q+Ib1Z+HcZzIcsCyKMo
	GUTOTG/0WRfln7H05zE64HDpi68XgNcUw8zkO7dSiDMf1i6MNOf89lfoiHcCacIW80dUlC
	Jl7+yGbJhsHkClODkZWqM8RFCfOuvwI=
X-MC-Unique: fptSl23KNRiKu2VoaUiReA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fRskDfeSb3hwMq/cMbYH5pvdR0YRa1OdI83JLMzIA7f+6J+C09jOqggOXLwJZftQNHaAibMCZN17eZSEWefW+XtDSWDgqwVYLjgm/OFMQz+nsXvf7WiE7bEkdE8l4b8TBwDhekw0rdKhDPdutYFmOuLmWwvefFHStcLIF30tnC8O3JTZGoJ4Rcp2r2zPlyPP16Tw3B9qqIblMKT07+DkG6kOJMZF+bOnXlBHWCQIWuRSGA4SnYq3I3vuOqq9bsBoGj3LOdKADnAt2K1x3Nybj1QNCNJOriyz81o8nwCBtQCg0Ig5Vpi/PFwGiJDgAxZcjyYZqGu1e/w5XfFyYYsPMg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kNCv2JfxAmGrCKSm1zBbx09ONKsK59pscIEd/URrupE=;
 b=jrvzm6QtJ5Z6h76QtSrkHhOfbTnELpWq2GjWBglqCBGJyANkN14e1yV4SdIqFNuQFyfrKkdo6MRAF3Dg+lqXc3NnHmMLjDcv94kTOx9/kIFYTZY8aSUww+y9Oye9Cem4qQ1c8Zfjh6JyE8JcEXZRXlzGLhNEULduv0kEIXTgbZ3bWMWSfenmL1yohQUDkaPYnBkfAsBbVT5j0BMANgYKgnoQWIrYnPmALf/fwsaMeuPoFrZX5ZoUEZgADBaZrD/bj4WrSkNT8mrxh/5JrSFVsC8sdSEkzYXhs6ligIVeqgYwa8qJ5xzNAqUaqygTWcKTTnQ17krN4cRcfvwwuc/V/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 3/3] arm/efi: load dom0 modules from DT using UEFI
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: bertrand.marquis@arm.com, wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20210928163209.49611-1-luca.fancellu@arm.com>
 <20210928163209.49611-4-luca.fancellu@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8f7422f7-8408-f33b-af2d-8e84dc0eab0a@suse.com>
Date: Wed, 29 Sep 2021 10:00:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210928163209.49611-4-luca.fancellu@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR07CA0025.eurprd07.prod.outlook.com
 (2603:10a6:208:ac::38) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bc9f16f5-4b17-4a4b-f456-08d9831f43f9
X-MS-TrafficTypeDiagnostic: VI1PR04MB4941:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB49415EDB677E08011610E588B3A99@VI1PR04MB4941.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	OcXKO1Kv+8+UXdPkGtZuw5qBOetqaQUHvbphGphGfAZfYuQe8eaMwKohlD+xmhWBpKEJq2znWA/h1ewjAqRG9kjPPOdwsN9DouAhFVbJXnb8YW1Gu3URQJeYMRoXJz6tPScnRrggiwZ7tmCFR9CkultWLfQ1cyVVK0WTBcTxSPaXwd5HRitWuc+jpgE8zWMWlr989m+Xk0ZEbFDbqbCoOuJciWc7KVva9WBccIH4YUDowjNtKeeru1ghIha9Rj0H7RYEO49yhe1cwR1lz3Fsctm4UITJekdxxSyd957paN3qBphNKh8MtL+f4mecxIJN6EDPa2nd5Q5a6GW3n31upmoLi8wOgTywpdywVJie3+YrOX++58lsYVCs0g88QmA+JNZX2MZtkFYG4LNgAR85UJybcC2ZmOPQUNgIpDmwseqROZsqHAK5Cs3aiu3yqwv6Re7GzrGBq+j1YRnP/Ak35FPcqF9XGiKEgEAzbxBkr/fPD5eZB73gi16rUhE6ypu7FgdTIOAsINSKYX3t5/aG4hIxrH/ot8Hl2ImTfcrpMBj+Toq823OPkMbGrnxRSWiYHK/v92NgmkpH7+GpXhA82AsXA+HZqgiTzHwvEiYUacbDP0hk+wkgyL/9DWF5nJeUeBtVeArhell+b431Qmr1KUy3BbiOx0EMy18HEPMDxelOVQ5b6NEcqEV/2wd2JXGFCoWcE5PWC5fqxF0K3G2IXbvy9CeQbmEtAvuqAe71AcM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(66476007)(2906002)(86362001)(66556008)(8676002)(31696002)(83380400001)(316002)(508600001)(36756003)(53546011)(7416002)(4326008)(186003)(2616005)(54906003)(26005)(31686004)(16576012)(66946007)(6486002)(38100700002)(6916009)(5660300002)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dnhMSXlESUJ0VTJ3ZEZaVWdXMFB3MlZoQmtwTDlSRTAweXF5amk1YWEzdDFl?=
 =?utf-8?B?WFFLMFlhTENOUTdSTzhnNkFwQnBoc0FuUlVaTEFoZzZxV2k2Rm95NHlBeUpm?=
 =?utf-8?B?eHZkalI5UnQ3WTdPVU9Jdzd0TkdqYktuV3l2NmgyUE1pdDFxeVBGeHV2TVhD?=
 =?utf-8?B?K3l1b2NNbVlKMHZ2OEl6ZWFuV1ZnQmVvNW5hWVN4UDM0NXV3aXlDK0VVWmdJ?=
 =?utf-8?B?WGZWVVR0MUR0RnY3bGl1K3QwL1RGT3JpZERrejNGYTQ5enJWTDdlZEtQaHBj?=
 =?utf-8?B?NnFEdHR5ZXRVREJneS9sVmtyQW5KQm5CamM5UzJTckhzSVJocmlDMkNCNXM2?=
 =?utf-8?B?bmN2VG42NjU3ZFNnYWU3OVZFYXBJVjY1R1VqbGNRWWY5b1JFbkVOMWFkaW53?=
 =?utf-8?B?elVQTUhaeFlXam0xQzljSllMY3dYOWk5bk84VE9FbjI3S0t6a0ZWMWJMOEZD?=
 =?utf-8?B?WWRyL3ZOQnV0U24xaGJoL3lyWmJnWitVaVA5TWsrQXkxQXk5emNMV3JRUlpN?=
 =?utf-8?B?YlBqcWlzekd2WEY5b25sbTRwbEd0MmppT1pQc0c2d01yRkdJY08xdUdhNTIz?=
 =?utf-8?B?alVxQ3ZhazZNYTUvaG5zSUM3Qll3NUNTSlZXZVArVk81VE55cXlWNmR1aysv?=
 =?utf-8?B?SDNlUGdoZWdHQXNJVGtVTlhqSXBtSTIxTG8xekRlTC93VWdtMWZaSUNaT2ha?=
 =?utf-8?B?MDFXM1NsV2VmOHJzSDZvbkxoTCtwd3MyYXBRZlVXL25ETS9vQk05RVJxSmpk?=
 =?utf-8?B?ME1hYnB0c01peGtxQlVmTHVTY0w4V0UrMk56NVVWUHZuV1VuRnNhVlpwZWFj?=
 =?utf-8?B?QndoaGptTXRWMkFJS00zRThQaGY0dm9LZEtJUG1SenZEWUdacTlaMTFGaUgr?=
 =?utf-8?B?VzJiZFQ0Rlo0YktrLzRPZG42aVlhZldOOGllWDRaTmtPaVUxTUhJNFhjZFJX?=
 =?utf-8?B?N3pPb291MnM5czZRNnBVVUIwd2V4OUdVdm8zc0Nhd3lheU5TNEFQZGJTaTZw?=
 =?utf-8?B?ZTVVb3FodVVmRVNydWxQR2JEb24zWWxPMU9SVWFBS1YwNlEwYkNDQ3BCNFZG?=
 =?utf-8?B?MlBZbjE3L3NRS1pXQ3dtSFdrOHZoT0hWajViYU1FVDFsZ3lrWU40bzhhL0RI?=
 =?utf-8?B?N005dkU5THlBaFpOb092THB2K1lOSCtlYXdzMW8yOUluanRPSWVKazdNNHpj?=
 =?utf-8?B?eG1BNkhhZ2Y5bFRpMUJOcW5EMUJ2UVZYcDlqaGoxa3B6Vkt1WGpsU1d1Mlpu?=
 =?utf-8?B?dGdGMW9kc1U3V0Z0bGtGTXJqL3JDVnJYczQ0RjV3UkJEeUxYODJuRXJMTlNE?=
 =?utf-8?B?OXlqaFJYZjZoOWwyNUhvKzF0N0RjczNrR21yazJRWFdOQWp4V0NFaDJiWHlN?=
 =?utf-8?B?T2Z4czdQa3dXR09OZ3JoNURMQ0dRdnc0ZlZjY3g2TWpaaW9tT2E1K2hVOHQv?=
 =?utf-8?B?K250WFRycnE4YjQ5aFozWUl4b0xEb0F0dU1sbzE3NzN4cFdHZFBZeXJVQnI5?=
 =?utf-8?B?YjZqeUh4NGFVcGVEdDJSS3ZsdGRmejJhbFhCVmJ0RXNvTUlOMk5EKzlpdjZ3?=
 =?utf-8?B?alYvcEJkOHJNZkpGUkl2UW85S0o4YU5RTWZlMG05T2VtV05DcUJDWFJrTFNm?=
 =?utf-8?B?eWlIL0NEelRnWmo5bmFVRVg3VUp6aUtaN1FZY1dMVU5OcFEzZVdURzBrZm9w?=
 =?utf-8?B?U21jZjUzd1RPdHNsNWNDVlJCUWZ4cFg0NzNTSkpiZnRVbU1lSXJ2aFo4QmQz?=
 =?utf-8?Q?S6ti1tJXMZOrQFwnRN9yoSEEHlXv7YVRgZf8PjT?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bc9f16f5-4b17-4a4b-f456-08d9831f43f9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:00:55.9034
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8iTZB6JErxzSJAl83PGW37Wt8H+o0DFgWrGL6nX3q26Jhq9DGlFXlAMbRpeihPKmZjJ6h6zvWOIDyYWKQJTLMg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4941

On 28.09.2021 18:32, Luca Fancellu wrote:
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> -
> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> -                            (void **)&shim_lock)) &&
> -                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> -                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
>          }
>  
>          if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
> @@ -1372,6 +1367,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      if (dt_module_found < 0)
>          /* efi_arch_check_dt_boot throws some error */
>          blexit(L"Error processing boot modules on DT.");
> +
> +    /* If Dom0 is specified, verify it */
> +    if ( kernel.ptr &&
> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> +                                           (void **)&shim_lock)) &&
> +        (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status);

Why does this code need moving in the first place? That's (to me at least)
not obvious from looking just at the common (i.e. non-Arm-specific) part.
Hence I would wish for the comment to be slightly extended to indicate
that the kernel image may also have been loaded by <whichever function>.

Also, two nits: You've broken indentation here (you've lost a space too
much compared to the original code) and ...

>      /*
>       * Check if a proper configuration is provided to start Xen:
>       *  - Dom0 specified (minimum required)
> 

... you will want to insert a blank line not only before, but also after
your insertion (or, imo less desirable, neither of the two blank lines).

Further I wonder whether your addition wouldn't better be after the code
following this comment.

And finally I wonder (looking back at the earlier patch) why you use
kernel.addr there when kernel.ptr is being used here. Unless there's a
reason for the difference, being consistent would be better.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:06:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:06:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198769.352434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUbU-0003DI-J2; Wed, 29 Sep 2021 08:06:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198769.352434; Wed, 29 Sep 2021 08:06:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUbU-0003DB-EP; Wed, 29 Sep 2021 08:06:12 +0000
Received: by outflank-mailman (input) for mailman id 198769;
 Wed, 29 Sep 2021 08:06:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUbT-0003D5-MT
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:06:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1b2a2012-20fc-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:06:10 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2108.outbound.protection.outlook.com [104.47.17.108])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-7zv2f_2CO_CdSG24ucAuVw-1; Wed, 29 Sep 2021 10:06:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6174.eurprd04.prod.outlook.com (2603:10a6:803:fa::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 08:06:06 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:06:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0110.eurprd02.prod.outlook.com (2603:10a6:20b:28c::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Wed, 29 Sep 2021 08:06:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b2a2012-20fc-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632902769;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gEBkLD4p5OLJiZ72tr+7qMCKDXAVO1EX8Bfr7fliDis=;
	b=nxqnyow8YmpK2JTizNbv6j3T7e50kvgtSz/c9w9LaQ5TfvfcgXMgtbrDG6+FwXiXhOeNbi
	WHRtOi+Y9la+3IgcW3/Klylw5Mf81vIdyXyNHj9LVyN+BgMdgdmReqj3+zfnOrOxJZho6G
	ucC4iwyr9bZnPzhLP8ZjAkDcxhjv/Wo=
X-MC-Unique: 7zv2f_2CO_CdSG24ucAuVw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GSNiKAfRSL0o8B4CQly1hp/f0EFJ9vsUI07acxteRF9HA6X4mFnqqyx2zRVDY9cljeI2tvwHdYF1phbb990LLz43tK0KiDdFLRhZQp3t5GD/wLL8r37kSl7JFvod930LQaeiZFgFFARnWZMcLv/a0MhGN4tEJzUh04ob3bgfUUl4u0Gw4AyRhw8abvYeJ6P0MnWVTdzUTRVmzZ+S3VuKX4yQiI+42YPv7lxnmIE/bHrsiWxj0X2nVrPT2RPcMgJLW4PGhTdnNXnXwxYY9GBicdSLkx/vUtmoh/jzXEPbVMcDJiDkpwOfVxzxgAaAJHW4Bo9Ja4uhUULt6PbVaZ60jg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gEBkLD4p5OLJiZ72tr+7qMCKDXAVO1EX8Bfr7fliDis=;
 b=TBJWYRHLiJfBmVv0rmwXdiRHTv80z2gXqZTuE8dhzkFkNOuKaEdtJV9UzMf78lnA0MOwP0aSmE7/ZEXM1BIvG/uDym01Hhg7RD6uPjlKLJ2L3TLzKao767BGGZajIxfLlyI+9e41ajhZO1mZjoPVF86GTTdzWpYiFqTQ+NpUdhfV9uwYM0mU9vUZRCXtBIdpFiWXiTFt7GR0g9HpZrZ4UfRqpRMF9m4w79fbv0WnteDxjHbpN3Xnf/b8rWz76BIPztJQW3NKC0aZwLnqEz+Grr8V3F4IMT/mRdi+3KfBifwt4d/ZrTkfrfWWR+U2piuqLKDHQS7RwUlkc2two6KlYw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 3/3] include/public: fix style of usbif.h
To: Juergen Gross <jgross@suse.com>
References: <20210929074620.28495-1-jgross@suse.com>
 <20210929074620.28495-4-jgross@suse.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <eebe75ad-e1c6-165c-3941-cd4dfa3e088d@suse.com>
Date: Wed, 29 Sep 2021 10:06:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210929074620.28495-4-jgross@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR02CA0110.eurprd02.prod.outlook.com
 (2603:10a6:20b:28c::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6a585be1-f93f-4bc3-ba83-08d9831ffcbd
X-MS-TrafficTypeDiagnostic: VI1PR04MB6174:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB61745645FCF11FED912AB71EB3A99@VI1PR04MB6174.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:820;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yIcWmR7WjWPsBr1Rex3QgCaAHKPNF1NxBJXS+IhyDhfakcZOrlDWuF8SWVfzPfR0Sd3a3J5yoz/pojEgrkipwE8tTzKOjWy/N1es/9LaLmpdFkGxEPf4l57TQiOl8MKaz8FzNlycx2URU2qrbQYuiaHk2Dd2ZoRAEA9SnpzG6TdZ8lg2M7X6pmuBlbh4woV0IPNsRvDxVcyTw9EXJZIhVRAEK0jwR/6zpTh55YD5RzwwtnHKNkTiqdVAYeOtyuD4XKBzrYnFUSLUCPShn0JRAZ30Si6nkSDOCbJUwCphfnX8wKFqWyzfbk4gLdS77LBvpvqoFECnP6cc1MhsErabldpFH4PDzaMyrstDtlfp7grNSrXMDuKqYSiAkxogt089WHlAV/wG0+uCB6HjtUwQipfVtTKt8vgnaphy0vRxo59XLjXaNVo55pTL/XCZs3KjvVAVE8yGAT9lWDknxsYzJOkd4/iCcIqam4V2gzZRgJgaW6GDlWzaSDZoNOJLCVncoFo4wdv/qBHEcLe0CPeIDYPpmy4QZ5Y7JqRas1Izp5wfyi5RghI9p/wd0cRvV1ObtXPQpGeSikhw94/DAP3qi7Dnw+YL9T6YSEIJ34OdfJiCrMf4xLo8fxaP6XqTZsVW5XiQfuWnmsgInlImwkhG5Qyd683wz5xHILjxZpSvy4EL0fjOAIDqqFOeaUYztRY1CJRj1VWpeivef3EmOhw80GyNXqiQSgGlkmEbyUm807I=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(31686004)(186003)(53546011)(4326008)(6862004)(316002)(26005)(558084003)(16576012)(66946007)(66476007)(66556008)(8936002)(6636002)(37006003)(956004)(86362001)(2906002)(38100700002)(508600001)(6486002)(2616005)(5660300002)(31696002)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?czhoaUE0anNqTmh4VWVnMStyTDk4YmYrOEhFVjVDUVphL3lEZ2M4T2RNVWhh?=
 =?utf-8?B?WmwwYWI4WWhTQU95Z0NxSXVIR2VTRkRHS3NFNk9VM0Q0TEJjTnF1SmVDSFpZ?=
 =?utf-8?B?eTRsU0ZJQUZhTFd4M2VKcDJVK1pwTW54SmJjbDUrNnVtaVoxTkpEQ21ZWG04?=
 =?utf-8?B?UXRYTnBUVmtCeTFrM0h5cnM3dExXSW9CekMwSXdZaU5CdnY2dXdnb2Z0Uitj?=
 =?utf-8?B?WXY1WUdaVzh3VnVZbUVyNVJVNS8xRVVxcXF6RUFQdC9tdFR3UGQ2eVcyRGRF?=
 =?utf-8?B?UWVScUIxY2dIbXVuekhFcTJRR011YWQ1WlBmeTVDSytTRFNmakJWTDBUS2hk?=
 =?utf-8?B?Qm1LR25jdVgzL1Ixc2FVbGdlTzFkd2RFSmlTQ1JyUGhsZDlSOGZXSkQ2ZDFm?=
 =?utf-8?B?MkM2N1ZMQkNzNzRVV0MzaHE3U2t6ZlNJblVheXZyeUJUSXc5R0FhNFVyeHlp?=
 =?utf-8?B?QW11NDVWYUM2YXdtd1Qwa0xGaHBEWHBlNXdZeEhITTg3Z0JjQU16VlcxcDRX?=
 =?utf-8?B?YnppaWpKcklEdEdBWjBVUWV2MWpFTTBVdklzY3ZJMkdRVTk5MTFHM2p5Rmhu?=
 =?utf-8?B?UVg1U3poQi85VWtoekdid0lTMmFBMUQvMUQ3L1RMTnBrTVJEUU0zNmhodG1F?=
 =?utf-8?B?QlFvU0tiSjhzd1hVMEx3cGkxVnE3b2Juc2pURWcwenpXdDlBa2U2ejRxeDNR?=
 =?utf-8?B?TVNjVWpKUkNvSkxyR09JSi9VZTUyUkd3SDBYSFprTFdzeUVaYVErWmhJR3dD?=
 =?utf-8?B?QlZqOVI1dHZSblBpYldwTW4zcGZNWGIxUmJFWXlaZ2g2N1BWeG11Q3dJVVZS?=
 =?utf-8?B?b2pzaWZrbGlPS1M4MFFjQlNMaXQ2N1psOHkrZEVhVTYzWGc2VzNPVmREUUkw?=
 =?utf-8?B?amVNajFJTmZ6SG4rb1NTTEFmcFUrM0FVSE5jMjdNRmVJNWFGV0hZeHFWSWc0?=
 =?utf-8?B?azNjbnJtdDd6RHdpSHRKTm1wY1YrNDlTenFZSjNScmRkYkNtcm1XRWVKWjAz?=
 =?utf-8?B?eUF2M2p5c3JPNXBYYXlsWk1Kd1hWNWt4Uno1QWdPTGpPa3NGUmNmZ0JzS1Jj?=
 =?utf-8?B?NXVrUVBaUjRhbzE4UHN1T2ovcVcwK0VTRnkwUUlPZzV5ZWpkWWtvODN0R21K?=
 =?utf-8?B?eWRqYndTQXlVbmJIcVJ3NnQ3VjhTZlJPN0JQTWtrdnhucHZrbEtWQlYvSzhS?=
 =?utf-8?B?U1EwMmNYZWFaUERsbWtmTFhDU0N3Qmh2YWNBZS8xTDNJLy9GbjJ3cnVwZmhC?=
 =?utf-8?B?NzRjUFNVaE14aEZ1cWNsQU5kM3FtR0JOY1VOcGczUGtvV0dpZStDZTVBdmp6?=
 =?utf-8?B?M0pUbVhWcitqT2dCS3RaOVZvRThBMncyN3JvWjlKWDJUOTJESWFXU2kvc3pZ?=
 =?utf-8?B?SUhJbkVQR2ZpOHJ5bG1iT0tYV3RjWXM5WnNvakljUEJZMTBHcUZPc2I3NjFO?=
 =?utf-8?B?bGowWTY0K1liVDVhUEoxK1drNUh2MFZFdXA5OWptWVVubkpBL2dJTkFvTyt3?=
 =?utf-8?B?YWRDZVJiVXNaaU50bXdZMktwTG9SSXhCaTFkay9YeEw1QmtiYVlCZFgrZmxs?=
 =?utf-8?B?MUFWZ0MrZVo4WXd3TkE4WmlzY2ZlSVFWSDZCb2ZKd3U0dW5ZNDMyRHpyUUNl?=
 =?utf-8?B?dE84RzJ5R2dISmNzV1R0YVgvMHFBa3JPcjhwYmljcFNLZWtJb002bWx3Snk5?=
 =?utf-8?B?RmRVMDlYM2Fyd29aRzlPeWR6bFNDYTBseGJ3bjdyWUJVK0VDZTM5N3QyZS9L?=
 =?utf-8?Q?uUduKX6ZaWd/DEsNsJw4jkz20faj1Ak4D/pjaLA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a585be1-f93f-4bc3-ba83-08d9831ffcbd
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:06:05.8382
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gyb/obETuKvNb89bRmRc/VRsGEqLCaQa8WhJERm8jUYAzBb5OoeYKtzVXLBtIt3YgSuLAx+csvq2rstcv8pjRg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6174

On 29.09.2021 09:46, Juergen Gross wrote:
> usbif.h is violating the Xen hypervisor coding style. Fix that.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:07:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:07:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198775.352445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUd6-0003y0-Ui; Wed, 29 Sep 2021 08:07:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198775.352445; Wed, 29 Sep 2021 08:07:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUd6-0003xt-Qm; Wed, 29 Sep 2021 08:07:52 +0000
Received: by outflank-mailman (input) for mailman id 198775;
 Wed, 29 Sep 2021 08:07:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUd5-0003xl-J8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:07:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5655cfe2-20fc-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:07:50 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2051.outbound.protection.outlook.com [104.47.5.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-10-gMy4twG1MF2-ga6iUtEQNQ-1; Wed, 29 Sep 2021 10:07:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB6176.eurprd04.prod.outlook.com (2603:10a6:803:f6::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 08:07:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:07:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR01CA0130.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::35) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 08:07:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5655cfe2-20fc-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632902869;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=+B5/iZO428rZrFD5p7xGraQznxRlLXoSw0d8Pfv2pnI=;
	b=G1nVrXn0FMJnj0SG9R7aGDzE5WHMSv4oz+aclr29GG60Yzw5BxNQz+8y8xxuCdXHrYlR72
	Z/4m6wSnznQoXKq3DDArjLRwcEADRrgwnGZ6zcUqy3dhFimnRqKlJY5Gwq4dI7Uu8Ln0ef
	h3yr+Gu876X2YtS0EdlKgqEc06xiDpU=
X-MC-Unique: gMy4twG1MF2-ga6iUtEQNQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MM6JalX8jC8z7VolZjLFbAW3SNq9iKY9zJVkR0KxxyRmh+5LFLM1N/eI6b10Zb4orA1pWnOb28n59HfqDVY50Ayn/n+JtEdCpQQbymUG9fcGx9JtlNBfzLz9ZcqN0lBdgra5t+0wDuILe3eprYM3ghPHPV1COTy0SuA/gukRzVYrs5qa97ZkyLc6AGBj08TXATtnlkKkUKXmQoK4VJ2PwyyHdW4noxqw+Zje+YTvDJdnxY3+sP60atW/ezo0+Jdhz3qC1RjvxlBtBh8pPIKW9ELV/fTrsbKFbqcj0rCQMMKjZ8hHyH6jXyaWxz+Lg+MVnBvq6E/sCn83Y9EpKyl/aA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+B5/iZO428rZrFD5p7xGraQznxRlLXoSw0d8Pfv2pnI=;
 b=EkdW0Wwmow0aPFmuaYtMWg7JxCbqYXMc3IWCyfJP4gIYxDRQwxCGw3W1fQipIJGypD+u4HadlNdCCwE1JCqMQJudq79kfU3kRjwsQNO5HikX0nUACme35xDikqB89JjxUEiTuvxmiKcj4ec6XMuiDGDM73M2cI0fgcuv9gorh4zVj8bvM8u24QYHXyuq4FUyhX4FV9p6JZhwHgYNX8WAP8vUgDnLqXUZublOGsYLlpPKZKoM0Bcrgfz5xOTr71LqZNytkKonoCQCF+om5G53odh1wKwnCfktHNTs2zTc9K+Swk+H1lZsAcaAhrhYAJ5U6VRDSuykkF+KDDfs0cZSNg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] x86/traps: Fix typo in do_entry_CP()
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Bobby Eshleman <bobby.eshleman@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20210928210014.27954-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3408f175-d71a-267b-515a-972b3ed8c4e8@suse.com>
Date: Wed, 29 Sep 2021 10:07:43 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210928210014.27954-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM0PR01CA0130.eurprd01.prod.exchangelabs.com
 (2603:10a6:208:168::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9e94ca02-46a8-4588-5a23-08d9832037ef
X-MS-TrafficTypeDiagnostic: VI1PR04MB6176:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB6176C4EEF0DA81EB839960D3B3A99@VI1PR04MB6176.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4303;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KV5jTFTUUifStb4Gz0mmMJIphXMQoI8+FeDPVz5NuyVzRWk45TxT2wLe53aQViD0g6yIkmAl30eOFjwe0ZJy64xX7wRrjL8ja2iCgX4uHVU2w0OirqBdiX3LuH5EdEPZ9+wwNsO0D5fCEUfpHD3AU4EaTyNYw1bdaEFgAIYc37lA7jo406ZUY7Bvt214nYqBGA8N/L02tGGf1N+LEhlh3Kuwv6YZ7MKScVETBU5cSIKEdeABb1giEpI1/lD/G5LQSXDpcBK5wKdkrEAaYakG7/XuE/AIUWqKrF9/BqKpZA09sXx0ZlG4bBFNRwLLUIFYA/myxaCww27QIpHRqk0iHhaYIcGE3Nrr+KaIf/l11Zk//5Wy3x5JVeVsesryUxbXkv2gJ0Xwpte7kxVSHNKvTjvILEi92C3+2/6KhAbqhRPIfwr70ojemGlL7uznH/SCPHy/5O4FRsFb3/rNta2yo5AlIW3JS39pLMBpgNAYjfF8pUMCZUb7D2j04voLzHK/4IGGFj4pk7Ca/FnDFe4O/P6sWRH5/Ij0Oi4NaF+1C42Vk595/rwDXEJeG75AUdyzOTZZZEdIdax9mzDNArOvzO21qOUgcrtPxou31gau8daRgS23GmB4nJcAQ6EzLJqIoEOeP0YdtYBI7sofm41kutoYDrpWTw68f8FFVqP2InImpDrjPn6mE0cfH4I1Xxq8CHqAxRnmm4uOS+YMG29cfA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(54906003)(4326008)(956004)(316002)(6486002)(16576012)(38100700002)(66946007)(8936002)(2906002)(66556008)(66476007)(53546011)(5660300002)(8676002)(4744005)(2616005)(86362001)(186003)(31686004)(508600001)(36756003)(6916009)(31696002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VnVCUk5JRVJUR1UwSXozMGFRaVBMRTVGd2dOUDhXNlBXQk90RE0wcEhiTDhP?=
 =?utf-8?B?NHU3Z0Z2bGNCM21EYVpSS0VwQzZlRFc4cURENkltT05DdmZGODd5OUovOVoy?=
 =?utf-8?B?OER2RVRaeVY1Q0dYa2tNUWpoT1NJbGlPVzNrN05xbG9pRjdCLzdCV0dWYVdu?=
 =?utf-8?B?a1pnY0haL21QamQxV1dETlR0b0Z1ZHV1dFBxOVFkcXd6N0U1SzNzdEhTd0ZL?=
 =?utf-8?B?dmNaNUtjUmVnQmZCT0R1ZXZBWGtDRm5ob3dkdUNkbjg3bDdXL1VQbWZVZ0kv?=
 =?utf-8?B?cXpCMmxqSGlsZjNkYklIcU5XeEo2RVRoc0Y4SGRmN0NDLy9xYk13UWo5MFRB?=
 =?utf-8?B?L2RxVjN6dkdJTTZxemNJRVdPaU9SVkVocFVHWDVNcVc0N2FqaDdUMGFKL3kv?=
 =?utf-8?B?d25yU0g4aDJEQlBOTUFYQXFqK0p3NVV0YTlDcjZCSGdGMngrbnhtV09SdnFX?=
 =?utf-8?B?aWkybXJLWFFzRi9pdzU5SG4wTnUyVU1BclFudjV0eXlmUGpuVUYza2F1MjB3?=
 =?utf-8?B?UTBlN0pjRXhkNFlSZnZIOUJtN1M0ZVk0RCtyYmlwa3B5cThucFpWakVsMThF?=
 =?utf-8?B?OUErVEtMT0syNlZiUkpTWDQvcEdGZSthYWJMQzdQWXlDbG45RkJaQjMrSk9q?=
 =?utf-8?B?TGR1Zmc3Nm5yTzRDZTdiaXpjTnpOQWl4SG1ZelYycmVyWHo0TDE2SDdLNVJU?=
 =?utf-8?B?R1RaakpMYm9tZlZQZDR1QTZTTHFHOXhUbEg1RWZmdEJYQTh2dGlTQUlIK2dp?=
 =?utf-8?B?VnQwSm1RWUNuSTFzbjVWeCt3Y29qZExPdG1EWW9vTkV0cURCbWkrNWNJdDFl?=
 =?utf-8?B?UW9ZejQwZ0MrcEJ3UXVMcjFVVG5UZm1RSVBrcG9IRWNsc1EwY3JZcHJOTGZn?=
 =?utf-8?B?SGZCNHNUd0ZOdVJ6Y09NYWhGK0laeUJxVnRsaDZZVjcxNHBLbUdVd3hNdldh?=
 =?utf-8?B?WmFMTjg3YTc1NnlZKzFXSk1vWFU2WHJQQ3dEd25nd2J1cjdjK1FYVzRFMW9o?=
 =?utf-8?B?MjUvWDZNNGFRMjZCTzlaeVV0MUtCc0tyNC9iWnVxNXNWeHF6dnRON0Zqakgr?=
 =?utf-8?B?WlYrdG14NTFjSHkwcGQ3VTYyTkVLazliRENNb2EwZmFXc3dUcU5ETTg0UlV3?=
 =?utf-8?B?QnIvSEdpSGxQS1BvL0RyWHlGbmpyS2VPdkZHMkJhM0tEWTl5K1RneUdNdnJp?=
 =?utf-8?B?eDRabWpjdFpOY081Q0U3OERCVktYVFRLOWtVWFZkMmcveDdsYXRJRGFqOXNo?=
 =?utf-8?B?U0N0cUNkUHd3OXVuL3NqZm9VU09BQkZXUzlSWXlWTjhFbG1yOHFOd2dYdkFL?=
 =?utf-8?B?bnpMUklKalhNSHREU1E1bnI0WTh2MjI5RVZsWXllZmZ4Wm01V0cxVzFwWE1k?=
 =?utf-8?B?NDFudFZ3bnRRTFUxVlNPT2hhNVNoR3NleEFHeEQ5dmtLZHA0eTFBL2VYcWFB?=
 =?utf-8?B?VExhRERWMnRIZDlTdk96NXZLd0F4cFk4RTdCV01vUzNueXVuNUlDMHIzd2Y1?=
 =?utf-8?B?cVpKYWlLSDkxcFBVTnpuK1RVSnR6NCtxQzM5ZHkxK0N6VEM2WitnZjBrRFNR?=
 =?utf-8?B?K0hZR3JZZkxIdFp5MHhkQ1pFS2tTUVdXZlZhVWlXWDJFeGpvNGNyQURFL0JQ?=
 =?utf-8?B?bkUwWHNXRWd1bjVpVGRiNU8yMmgrNFB1K056YnBUa2hnWlBQNGhTaXIwWWha?=
 =?utf-8?B?aHozVkRyeTJURFQ1S1cvSmJuUGJLZS93ZDhRU3Rqa2U4K1NkQ3NoZVNTODdu?=
 =?utf-8?Q?+lM3m2uQdU4Tgu1VDpro+Jtadbfu4HIxf6BF7qz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e94ca02-46a8-4588-5a23-08d9832037ef
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:07:45.1240
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dIEmFoSMPtm1zGEwNYhv4Wz1duI6gVOePkfKN8GjCaoSOZ4Kg/MjSVHuA7foYIaicnzmzmHf8vSOPfAelzb0Vg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB6176

On 28.09.2021 23:00, Andrew Cooper wrote:
> The call to debugger_trap_entry() should pass the correct vector.  The
> break-for-gdbsx logic is in practice unreachable because PV guests can't
> generate #CP, but it will interfere with anyone inserting custom debugging
> into debugger_trap_entry().
> 
> Fixes: 5ad05b9c2490 ("x86/traps: Implement #CP handler and extend #PF for shadow stacks")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:10:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:10:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198782.352456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUfE-000567-EJ; Wed, 29 Sep 2021 08:10:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198782.352456; Wed, 29 Sep 2021 08:10:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUfE-00055Y-BE; Wed, 29 Sep 2021 08:10:04 +0000
Received: by outflank-mailman (input) for mailman id 198782;
 Wed, 29 Sep 2021 08:10:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUfC-0004rK-Gf
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:10:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id abacd0eb-063f-4697-85e0-86b4d5fce817;
 Wed, 29 Sep 2021 08:10:01 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-oB5jKPakP2Ovu2tTxMdYzA-1; Wed, 29 Sep 2021 10:09:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2703.eurprd04.prod.outlook.com (2603:10a6:800:ae::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Wed, 29 Sep
 2021 08:09:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:09:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM0PR02CA0192.eurprd02.prod.outlook.com (2603:10a6:20b:28e::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.22 via Frontend
 Transport; Wed, 29 Sep 2021 08:09:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abacd0eb-063f-4697-85e0-86b4d5fce817
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632903000;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=dof4yb1LmJn8mR4Z9d27TQQYfPB6VwuVKCZ4rJkEVaU=;
	b=ff6cAT3LCQbUOTSDNavZBoDBnsz7+qSiGz6aMIzYA4SWy/zOOo2RB/TcoD9nslZmt4p2ub
	6mzqjEep4Jwino2tsgwIe3G0JIfsqpXALtwKiTPHn6MT9BAYgZhoABs6qVa+XvrGKFSuH3
	O7+v6V0BVM7GbDJngD6k+06ucDex6sA=
X-MC-Unique: oB5jKPakP2Ovu2tTxMdYzA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XoPL0wZU7mLL6cE51dt9lIm8MQSH4yj6zqIzgkpG5GEXh22wfKDYSRCSSaSu/NT+Kmx+i94u1JsXnBhuoXxcw+iS5QN9XdmgfbTg9yOZFpf8QJLd+xqURVTVVFSEam4VkscKCSrV4wlV8zmp9itOlHzodivq9dwgYNfO2TC5MryHlVvaGAju255XFvu14IWmgZB0Y2tvAp9xaP9kiAqJ4l9YUytq/UvC22L2v1+dQ7bDMWnCgUF5TzAkyckaZbp829V6qdD6Q3h1YPk81spO+jZvwZqMKXwOllcAAzy8pt80UAausBlPD7BV201WSKoonJis0sjSLtNlAQH7wFAN+A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=27xD78VhI9s2N/437ueyYiX2G7LUxR9hoJJ4q60h99g=;
 b=RSLTBTBnv5eUl51blbd3jw/uNNZv/OJgtsjrdOwsj9Jlyr7iLO4AOOzqZjtAZVd2yum7FxjM8K7ecuuTCdrxELdRbrpKREViihwz5vsD3yojZw+iUCeUeoe218G63ebbjMsimL0IDEYS/1xjMTmMWpIhZq79QQpDKskpcTUyHus0yjJaQjvXntRbL/S0uE3cWq5IaGXAekHioOu45RIGatkdnvPSry8CTHhjuitMUSM+2CkGFbP+g+DB71bTrLxlRy5umf981EutXrG2q+HONjqLN2uUBmrRGBPgChyC4CaLA5LYNZXCgLmHypuWnXNrhzJjFCGonnskQ8j+KmVwxg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v4 3/6] arch/x86: rename debug.c to gdbsx.c
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Bobby Eshleman <bobby.eshleman@gmail.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Elena Ufimtseva <elena.ufimtseva@oracle.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 xen-devel@lists.xenproject.org
References: <cover.1632860589.git.bobby.eshleman@gmail.com>
 <389ae979063a4afc38d8dbadaf539e7f411a24ba.1632860589.git.bobby.eshleman@gmail.com>
 <a094d614-2296-bcc9-1c7a-084ef88b05ad@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <a578a18a-ef43-c257-bbf7-cf29b81533c2@suse.com>
Date: Wed, 29 Sep 2021 10:09:53 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <a094d614-2296-bcc9-1c7a-084ef88b05ad@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM0PR02CA0192.eurprd02.prod.outlook.com
 (2603:10a6:20b:28e::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3ec70415-dc7b-4fcc-c185-08d98320861a
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2703:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB2703C0EF0930D337A88CE2E7B3A99@VI1PR0402MB2703.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:4502;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Eph23FYTL4s3T9FZ7H/zCj53NmYzaUeX9wzT2kKgHOGcOlEPHCgFUKPmJFvzH1wXiVCM9WOdurVwk5HK3vvFkg7gltsIHeTx0U8csGQJd60rUhJ2uD+mCGdlHMz/qdTBJt9tOqHaFflVvpV7TbPe7q7QM6ii9ijTWx8QZWbEkzHYhGDn1wCBlw8hdf3Q1kBrTMeqNpqgtAIyx1AkjBSIyLzaNXUr8yiSD5CyrB1eEaXb872hYD+byW5xAkmoLWIAKAFMMgMuCcElvJWoro+bj+6FN1+6zjjDqN2g58o3nObYP3U36MLA7A/6JCqrJsRdWOIFlKYZ8JL5jAFkOiXSho6vPvAlcrVyujVbK+zZLPMJiE79C1G2XIGwo+SgdAJP5aNzxcqZofW3cIf0aghBuvUB84nol66cU3cR0hYSLrciBGQspKmEa73ulM6tnBxLYTClc7oqhFEOI2gS3Qyh3LV2ZUigSolmK98Uixb9GcTHov1G4gf1uxh2MibEECGEIHZFZ4wSKaW+feB55g9Px5lfXYt5ZKP7t4QB3WiZg/kf8avxO9RuYsh/IqMXW0BDey8ox4cmBHVwN3V6QKI03ztqjtqkq091vNhTK0c93qrYZYt6aZISHSzzAm932QDBoZF9lyxqgo/lngcYmMJWynL1v/J8rdw00ZlvayZuE5KKPWpOYAlN4dljmH2HDb9arK0KBRmOusXhX1UiRImJysyDfU9ZgckpJeYR5UdT+TQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(2906002)(16576012)(4326008)(36756003)(8676002)(66556008)(38100700002)(66946007)(66476007)(83380400001)(316002)(54906003)(508600001)(8936002)(7416002)(5660300002)(86362001)(110136005)(31696002)(53546011)(956004)(2616005)(31686004)(186003)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?EZWCsdIyXUQtaRmpx4j1Rdk0jFWkQ1EdzTX6Xx1j+mwFVbHRkxGCB4bOqdvi?=
 =?us-ascii?Q?En1ngSMzolS/Ls8wWHMKhSaj5qPrkqfdOBe9rNe2Ov/6LZsOnGvBO0ldVNjm?=
 =?us-ascii?Q?UG3jdkGP+kVsO/QnQSMlsbr+WA9xuDmiLwQyNC3aWMlLpEpMKW2D7S5YPQQ0?=
 =?us-ascii?Q?r52zHyXnQMsu+V4QZxQovZsImyoJE43Af6cI2cXOxvt/+xJ6DLsjnqlJCmbn?=
 =?us-ascii?Q?g4d1lO9SRY2DL70T1cgODbQ2J97Zp7o2n8WkNn/bl6CH+ykfpDYkaE5C5wpT?=
 =?us-ascii?Q?7aG+pHdrJCjjZ4EnaxMpxSxPw2wDs91UMZRGztHJTajZsYQn/Ef0Y0QAiU6t?=
 =?us-ascii?Q?f6R1h+Ct4Z1hdcrX6gfRvSTz9P2Jz0o5oGR+EVchlKGWET9yrxxy34jt/A5r?=
 =?us-ascii?Q?3i7mycW5yyesHmnlyoYyj8P9VV8OOps1a9lYb3OK2RG0BMIdmdrc42EwpwrX?=
 =?us-ascii?Q?wIF0AiXK74VrarA6d+koLUPz/vur1GhQx6uzg0m8Oyszz8Lfh4QdpX/Dz1XK?=
 =?us-ascii?Q?eRXVaUjamvRl4MtkHbk7uMofVJGHhwnx0eHcjRzdED2uD+eFZErkc+Vnm8kx?=
 =?us-ascii?Q?kN1YOn0zr5byceaBW7kwmtnTeCBqyjQvW0xJ3RqoSesB4vOn3G81il+jqhLw?=
 =?us-ascii?Q?IlQZbi2dY08su4DSGJBBzTBITTgYlFNUVlrJQeFDJEfmWxFslKOn9xKpc6Xj?=
 =?us-ascii?Q?n1lzKvZQ9ja84skaW4lqllZP+cD0wK7IlVSlagVttU31X2UH4StrDhRnQCB7?=
 =?us-ascii?Q?YW3RlUk6YjF1iUjkhBIhpA7XaG/FrZIjD49lS2LW206xTkA4isHH5jllHUuG?=
 =?us-ascii?Q?Uhr2a44pR7qeRfeF4A/3Tv0VBN8Ctxex5MfljtG6byRMWLCuJCoOytqzVi0w?=
 =?us-ascii?Q?NVRbsBqSMx2/imqnm9M52HbQ5k17J90ailfi52g3j765LYPwgQwYlHJMlezR?=
 =?us-ascii?Q?cXCujIGB18rQwP0nIlaSA4BVAxXL7NrjQuMnVk1EVvkAqlJ3L6bt3odO2uXt?=
 =?us-ascii?Q?QoqCQtQXRVygtlv+EEyWnvJDIds/gDqFxrzy5syGcK14uQzmD3xqtsb92QiU?=
 =?us-ascii?Q?cIYwSyu7agNQxnHWfIxonCJnVfcPcjvlFXwEjNHq2XUfZG0q65oeXJ5ndA6l?=
 =?us-ascii?Q?8TxWQMlM38Dyvn94CaCCmh9AeRCdnXVQ3QIa8MjohxU16uGRv19JvgPeko3W?=
 =?us-ascii?Q?+K0d+RXcsE3CQw31/wXoULljV/vJDmoM/odlISUD/EsU5gLYJjmZxCJYfQRT?=
 =?us-ascii?Q?ljPmK9qpOi1pAJA/tRGIXZuiz4QMuFlRwh7VspVe+T/yOJ6IerVmJUDIfBpe?=
 =?us-ascii?Q?hqq5y+d9kmchmTFMTKn71GEt?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3ec70415-dc7b-4fcc-c185-08d98320861a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:09:56.2717
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gaJE7tfq09zwDTdqAWRbj+Sj7KTMpqfW2F4De5pzu1VwgTqJGUfAJtIxM7gHTrhIbaBdLB4arh9lofxkNHeMuA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2703

On 28.09.2021 23:09, Andrew Cooper wrote:
> On 28/09/2021 21:30, Bobby Eshleman wrote:
>> diff --git a/xen/include/asm-x86/gdbsx.h b/xen/include/asm-x86/gdbsx.h
>> new file mode 100644
>> index 0000000000..473229a7fb
>> --- /dev/null
>> +++ b/xen/include/asm-x86/gdbsx.h
>> @@ -0,0 +1,19 @@
>> +#ifndef __X86_GDBX_H__
>> +#define __X86_GDBX_H__
>> +
>> +#include <xen/errno.h>
>=20
> The errno include wants to move below....
>=20
> However, you need to avoid latent build errors based on the order of
> includes.=C2=A0 I'd include public/domctl.h which will get you both domid=
_t
> and struct xen_domctl_gdbsx_memio.

public/xen.h should suffice as ...

>> +#ifdef CONFIG_GDBSX
>> +
>> +int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *io=
p);
>> +
>> +#else
>> +
>=20
> ... specifically here.
>=20
> ~Andrew
>=20
>> +static inline int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_g=
dbsx_memio *iop)
>> +{
>> +    return -EOPNOTSUPP;
>> +}

... for struct xen_domctl_gdbsx_memio a forward declaration is all that's
needed here.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:13:38 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:13:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198788.352467 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUiZ-00061D-TW; Wed, 29 Sep 2021 08:13:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198788.352467; Wed, 29 Sep 2021 08:13:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUiZ-000616-Qa; Wed, 29 Sep 2021 08:13:31 +0000
Received: by outflank-mailman (input) for mailman id 198788;
 Wed, 29 Sep 2021 08:13:30 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jpj4=OT=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVUiX-000610-WB
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:13:30 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 20a20022-20fd-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:13:28 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 889C46D;
 Wed, 29 Sep 2021 01:13:28 -0700 (PDT)
Received: from [10.57.23.68] (unknown [10.57.23.68])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 78FE63F793;
 Wed, 29 Sep 2021 01:13:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20a20022-20fd-11ec-bcdf-12813bfff9fa
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
Date: Wed, 29 Sep 2021 10:13:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-8-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

Hi Oleksandr,

On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Take into account guest's BAR view and program its p2m accordingly:
> gfn is guest's view of the BAR and mfn is the physical BAR value as set
> up by the host bridge in the hardware domain.
> This way hardware doamin sees physical BAR values and guest sees
> emulated ones.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - s/MSI/MSI-X in comments
> ---
>  xen/drivers/vpci/header.c | 35 +++++++++++++++++++++++++++++++++--
>  1 file changed, 33 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index 9c603d26d302..bdd18599b205 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -30,6 +30,10 @@
>  
>  struct map_data {
>      struct domain *d;
> +    /* Start address of the BAR as seen by the guest. */
> +    gfn_t start_gfn;
> +    /* Physical start address of the BAR. */
> +    mfn_t start_mfn;
>      bool map;
>  };
>  
> @@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>                       unsigned long *c)
>  {
>      const struct map_data *map = data;
> +    gfn_t start_gfn;
>      int rc;
>  
>      for ( ; ; )
>      {
>          unsigned long size = e - s + 1;
>  
> +        /*
> +         * Any BAR may have holes in its memory we want to map, e.g.
> +         * we don't want to map MSI-X regions which may be a part of that BAR,
> +         * e.g. when a single BAR is used for both MMIO and MSI-X.
> +         * In this case MSI-X regions are subtracted from the mapping, but
> +         * map->start_gfn still points to the very beginning of the BAR.
> +         * So if there is a hole present then we need to adjust start_gfn
> +         * to reflect the fact of that substraction.
> +         */
> +        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
> +
> +        printk(XENLOG_G_DEBUG
> +               "%smap [%lx, %lx] -> %#"PRI_gfn" for d%d\n",
> +               map->map ? "" : "un", s, e, gfn_x(start_gfn),
> +               map->d->domain_id);
>          /*
>           * ARM TODOs:
>           * - On ARM whether the memory is prefetchable or not should be passed
> @@ -52,8 +72,10 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>           * - {un}map_mmio_regions doesn't support preemption.
>           */
>  
> -        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
> -                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
> +        rc = map->map ? map_mmio_regions(map->d, start_gfn,
> +                                         size, _mfn(s))
> +                      : unmap_mmio_regions(map->d, start_gfn,
> +                                           size, _mfn(s));
>          if ( rc == 0 )
>          {
>              *c += size;
> @@ -69,6 +91,7 @@ static int map_range(unsigned long s, unsigned long e, void *data,
>          ASSERT(rc < size);
>          *c += rc;
>          s += rc;
> +        gfn_add(map->start_gfn, rc);
>          if ( general_preempt_check() )
>                  return -ERESTART;
>      }
> @@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
>              if ( !bar->mem )
>                  continue;
>  
> +            data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ?
> +                _gfn(PFN_DOWN(bar->addr)) :
> +                _gfn(PFN_DOWN(bar->guest_addr));
I believe this would look better with the following alignment:
data.start_gfn = is_hardware_domain(v->vpci.pdev->domain)
                 ? _gfn(PFN_DOWN(bar->addr))
                 : _gfn(PFN_DOWN(bar->guest_addr));
> +            data.start_mfn = _mfn(PFN_DOWN(bar->addr));
>              rc = rangeset_consume_ranges(bar->mem, map_range, &data);
>  
>              if ( rc == -ERESTART )
> @@ -194,6 +221,10 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
>          if ( !bar->mem )
>              continue;
>  
> +        data.start_gfn = is_hardware_domain(d) ?
> +            _gfn(PFN_DOWN(bar->addr)) :
> +            _gfn(PFN_DOWN(bar->guest_addr));
This one also.
> +        data.start_mfn = _mfn(PFN_DOWN(bar->addr));
>          while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
>                                                &data)) == -ERESTART )
>              process_pending_softirqs();
> 

Cheers,
Michal


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:16:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:16:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198801.352482 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUlT-0006is-E2; Wed, 29 Sep 2021 08:16:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198801.352482; Wed, 29 Sep 2021 08:16:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUlT-0006il-AS; Wed, 29 Sep 2021 08:16:31 +0000
Received: by outflank-mailman (input) for mailman id 198801;
 Wed, 29 Sep 2021 08:16:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUlS-0006id-Jy
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:16:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 47a9f5b1-c1e9-49ab-8117-2cdc21a99152;
 Wed, 29 Sep 2021 08:16:29 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-L9TdVp7hNw-jgNqo7fzfrQ-1; Wed, 29 Sep 2021 10:16:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2333.eurprd04.prod.outlook.com (2603:10a6:800:28::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 08:16:25 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:16:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM9P195CA0021.EURP195.PROD.OUTLOOK.COM (2603:10a6:20b:21f::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Wed, 29 Sep 2021 08:16:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47a9f5b1-c1e9-49ab-8117-2cdc21a99152
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632903388;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kAJtOFAhDq6K0VGn51U2Rco8e5YgN526LGuD4cWCvlY=;
	b=KUdXJRpYafvwYVHdDW8fMjJ+NHUOMdzU5WuUsdhCC9PODzTeBezc8buqD0OX2GWlsmJjAl
	U0FY3vu7aZRCne42Nt5ElhH8FL6/0LhsV8EyUS5uH24rZb7scQRbabGzmXcMWoBnIDtF+w
	gC5Hfg+M4vorxsmWf/S2l19W9H14VeA=
X-MC-Unique: L9TdVp7hNw-jgNqo7fzfrQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XR2ajgrY9HG3b4lc73KiABrSI3HjgBzoW3EZgj/ZK2nRVB6hKgyr8lxvbU8uFumaCl1fFuLaaQcSFjdjXKq2OlXLHvsKmCJ+6qHDQvgOPWqUX0CT8AepYwZz8JvyR5O6oxpWXUtnSULS8nyCDMM9JcH5D9hHTSatkndfgDb2k/hR7qaPxwXEAZ1GBmjTjIYjD0BWSAYIQTA6nKdfALcqZEOsEaR5w19NQTnOECueWSkwN99POMbFy/bIFaGYvaeM1++ujrUcK9VLuyqCR1Z4xqdE68MQWc+AJ5+ZZ7HEbNtTDrdDS/F9QiTulGdfXmK80DRpy2f/l//WMnaFSmbmbQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kAJtOFAhDq6K0VGn51U2Rco8e5YgN526LGuD4cWCvlY=;
 b=gAY+YKyCGfSZCIqLhdPXekBMFpaBwm88ocQD0pN1DHCgj52QWclrlUEBV388MNc0dExMhYupNB7af3Agf8BvfkC0tckutEvud9vOXyF4n966KfUqc9fX0Typ6eiJVlzfwdvZ+WNj/mCFSnD9cf/yi1LZpYgiK9uHfIBc+V4nAh9/6dMn6DjM1On6PA0kaY+00BVGCK4pDM3vCPSFzeAs1lh7qtopTuoX0R9T24nU9q7AFvimC2TnEv7+JkzjlytrcqV9Yxs8RY1csoeFA/YySlBfBxPHzOuVj2alZ6UqMXegB1BdwXsbqq0DkHnjNdcSSMDhMMlhjNJP96DajX/53g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
To: Michal Orzel <michal.orzel@arm.com>,
 Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
 <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f59b48fd-564e-64f8-1422-fb842957f213@suse.com>
Date: Wed, 29 Sep 2021 10:16:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM9P195CA0021.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:20b:21f::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c89c0ea1-9437-492f-bcd0-08d983216dee
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2333BD0807F1D0420198ACCDB3A99@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	u2u2ug+yY/0cNZV3e/zHNJJgI+EG4x6xzLOdoqjhhclAgpGPXNGjLvRDhi8kEPIpoJaLzEnWkHcJUylsVIbzSINhsdlfIBaVSrFe86q06s7vnT4thjaeUZxL906bSyCqwmjYq/0cYLhFGUQX4T0DuW+Gdo8ub9VamchSi5kbQrEJwphNt6WAzVs5Zn8nZkk9G0J7tAsyvcmtJE32gNGYqmR+FLFNRNix1QaGNp+E3bbOZYNQQzbwXEWC41GH7fj3U2W8HP7M+wJIs5wkzqFw9+9vWyFxVhoGoEMctL87VTDjQLJB8HVDIbmZptkrqtfTfg6EW3qTMgBE3bi3yuLWfsWTchzcvOIlZ02rhJjgxVeOZSNUSfLgKMpEL4P4Q4QznmGvBZXqc8F7ShFt/9gUdBS2H8Hw9oImLG28sQe+SeycDY4ThywiNLUAFeeIFm30JrBM8XVtNUgE1h4xjCGo12FcV3g+wnqRXS4fslMGRD2G/r3VzbUrwEmhhRiJGz3iUIZ5Natuv1QBOTpNafb7EdlU0Y4WmoFgc/nVLo//fOON0LuSDQkS/oS4nnL0YV5YQw+aj7BK3K0b8KIfzYw7jd4qEGXE8KHRx1WONmMXMQAg7zBoDAbGZtAuSi4EAa974eaIb+SBeE9lv02oOVAMEXhGJ4xYP1w0GN9jEOlAwlRANh6105wyehP2CelfNe+UiHxr8ZhaJ27UEjUkloquO/PpgS6anw6N2pAYEXiscDk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(26005)(508600001)(38100700002)(4744005)(4326008)(53546011)(8676002)(5660300002)(956004)(2616005)(31686004)(86362001)(16576012)(66556008)(66476007)(66946007)(8936002)(110136005)(316002)(6486002)(7416002)(31696002)(2906002)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ODV6NVhxM09FRTN4TzJhY3d3VFBtb0J2bzJJZVNHQ1RObXhTRklvVzM5SDNU?=
 =?utf-8?B?SWtrRS9ndFhicWtadk1lR1VtWFBJR2lldzBFbWU5QW9zWHZ6bkdtTEM2Qisx?=
 =?utf-8?B?SVlWd0lYVVlHdGQ0T3NsVzZOMEw5c1V6N1NLVEowT1NFVmNUcWZ5OHdVSm1H?=
 =?utf-8?B?SVlEOFY5VEtibWFmNHRpbzVTVGRWTTBuREJaYjBscTlxRUJxTGROSXFEWWFY?=
 =?utf-8?B?dGJJNllhWHRTWU4vKzdXRS9uTTloajNhYTJ4bzRvcGN6eWhFR1FNUDA3KzBX?=
 =?utf-8?B?WjhWUTFXRTNCbUFuNDVocy9GcERHcFRPUGM5RlA1ZDg0NFk4OVRrREdTblJU?=
 =?utf-8?B?SXVVQ1o4aTZDUmkzMXdzeVVzUzJEVkVLWXM0VGNUMDZjSWhSOTZiU01OVGdR?=
 =?utf-8?B?U2lRUTB2azA4d2E4RlNoK2kxRVQwNlB1ckQyREp3ZGJGeTdyMzgxTk50eDVp?=
 =?utf-8?B?TUpNTk1VR01vYlIyT09nYnk2QkswWHg0SlZLWW5LRm51UVdvWG9qVnlEYlhY?=
 =?utf-8?B?d01kOWczWTUrall6OUVZT2x2eTFJUFA3bnE5SUYvV1d5THl3S2hzMkJiRnZ6?=
 =?utf-8?B?Q2RVcldtSzFrTFBKUU00VTdPNDcybWZ6R3N2V0pqTTBzMjdOcGV6UEx2Z2ZM?=
 =?utf-8?B?M0lMTk1ONHlWNG44c1pUcGppa2lkbkhhWUJaaTV1cmNGQTl1VkdwMUo2RzJx?=
 =?utf-8?B?OXJaUEtoT2VTcmJNaENzcm50WGFWNjY1VDFDaVlOb2dWY3gxanE1NDBBbmtU?=
 =?utf-8?B?OVdISjNvQ2VuVGx1Y1FqZlo3K01wYTlHRzBiL0lLNmJWQ2pMZVhmdVFnZ01B?=
 =?utf-8?B?TlRQQ0RVR3BkWGpvWE1YUy8rUytpNDlMWTNSa1hISFA5RjFXUkRlaUhMTGNo?=
 =?utf-8?B?a250RjcyaVhIdi9DRENuUG1FQ25oaFc4Ni9BbktSK2RycGR1WjRTWEFKQ2FR?=
 =?utf-8?B?cHQxcUVGMEl1S2t1bmp3U0NYUjZIMnBtcUFsNXMzNU5vcThXL2d3TVh4Qmhx?=
 =?utf-8?B?T2tEWW1aenhXcHE0RmdPNGNBMGtUc3F3QnBuT3loUHdSNFBnVzNETDV4WTZR?=
 =?utf-8?B?RkY4RHR1WUs4ak10cnhaK0JQamFNRGdqOVZTWEYwaS9LQjdJbkh2aHRVTnI1?=
 =?utf-8?B?RjlxQ2t1TU83VkxFT2E0dEJXNERBSmtHSEpsOExtc0M0MEUxTnhsNHFpdjFp?=
 =?utf-8?B?aFgxQXpxaEZ4R2ZDYUVkb0llQlhUc01FTWlkMlFQUUdVOHh1dzhwSjBUd1Bl?=
 =?utf-8?B?Yk9xZld0bXdUbjBnTXFFMHovejBkQmRiTHhhMDBiRTBKa2h0anhKbjRzZkxW?=
 =?utf-8?B?bzY0WWRvUUY3OTFyK2RvMzEvMFZYWkJDZjNnK3NGK09zemcwZ2YzOG1KWk9o?=
 =?utf-8?B?aDZ1dVo0dTB5VTV6YlNwVTNBeUNOTVU0NFNSVEU4RGRjdVB0V1l5QXNSelBP?=
 =?utf-8?B?bnVFNzBCejIzTFVaZXg2Y2U1ajRVMDVQRnhhNElKNjh1eUlwZnpPYWVlelpj?=
 =?utf-8?B?b2NFRmhpaXZyUUtmcTBPV2NWQk9SdVhQS2w5TzRrdElqZ0FMNWIra3M1a29q?=
 =?utf-8?B?L2FIUUVxMFNvZTJXT2dpRittbnQxWTFaSmNPVTJlZ3Zna080cVVhRGZmTDJ4?=
 =?utf-8?B?WGNvc2tnKzU2clpjQ2NpUEgvWllJcGc1RFVDWUhDb2lVSEtWbSttMWlOUSt5?=
 =?utf-8?B?WkpkZnkyQkRLcnlLVm15WGJndU51NHdIaVg2RXVPSzNEZUxXOXhSSkZCVDVQ?=
 =?utf-8?Q?cosPIev/vSbvXfafJXtSFXYau+0yD8QLlvy+XoS?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c89c0ea1-9437-492f-bcd0-08d983216dee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:16:25.2147
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RqP4RaQCQRGucRIqrojXu7ix1iHpG+PqdmqzglQk9Kk1ki6vuwXj6RyHEp2T8/bzh0gjHv2Td663Nn1aCp6XCg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

On 29.09.2021 10:13, Michal Orzel wrote:
> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>> @@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
>>              if ( !bar->mem )
>>                  continue;
>>  
>> +            data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ?
>> +                _gfn(PFN_DOWN(bar->addr)) :
>> +                _gfn(PFN_DOWN(bar->guest_addr));
> I believe this would look better with the following alignment:
> data.start_gfn = is_hardware_domain(v->vpci.pdev->domain)
>                  ? _gfn(PFN_DOWN(bar->addr))
>                  : _gfn(PFN_DOWN(bar->guest_addr));

FWIW I agree, yet personally I think the conditional operator here
even wants to move inside the _gfn(PFN_DOWN()).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:17:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:17:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198806.352493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUm2-0007PV-NZ; Wed, 29 Sep 2021 08:17:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198806.352493; Wed, 29 Sep 2021 08:17:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUm2-0007PO-KD; Wed, 29 Sep 2021 08:17:06 +0000
Received: by outflank-mailman (input) for mailman id 198806;
 Wed, 29 Sep 2021 08:17:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVUm1-0007PI-4u
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:17:05 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9ec8781a-af25-4cdf-944f-38545501e0a5;
 Wed, 29 Sep 2021 08:17:04 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18T5wVTJ024723; 
 Wed, 29 Sep 2021 08:16:59 GMT
Received: from eur04-he1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2051.outbound.protection.outlook.com [104.47.13.51])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bcg2wrra7-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 08:16:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3713.eurprd03.prod.outlook.com (2603:10a6:208:41::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 08:16:55 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 08:16:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ec8781a-af25-4cdf-944f-38545501e0a5
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n46qY9F7QXQ7CBLVLG6GPN3LSjj9lVBlwZWh5YT+NUbiz9ZbNbZ5W8MQJiTcOeUPM3YBbAL2VkokTN5dQDmGd12LsTTB6EgPGtZZq8m+sR7mUz172HEUdeddEcbzKE0Mkfxe+prjN/9vLF5QofdXXDd4YJqjW2Q2brPO+S8s/XQU60HoFVDQRaR1zdRM0MFGpyoBUEU5omjUD1GxzlAtwJkv0owAEq0Lr6iQKGiG7IM3aeH+gSmHGhkER0Q/fq15gVTGvtItpVmavR39bgTbsWAa5leHIJxEfsZvkRM6vtvL4OwX/Fp65koSQV/XxLNNqeRJqV36Oafkr/m/6FLlhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GOpDnCXDg05zX46BeGyQhbthEhLxry7AJ9p8Mph3xEU=;
 b=nvj+gHjhkfIyrLZvlIs/oiTKePp7+ocMGPCra7SXFkr+RGMD71KBTbKEmGc86tiI0/CYjb/7cUnvZNNWprauCsXn91ZihAZQg+QMzaVhPPRlF/ULHtKFqlJLMTV6s7pxomJby6dV/NmZsYPfTp6K0uHKjkcPQ4FlGiP231Fks6Rh+LsNP6+xYtHL0ma6hz0lKfqrHB/Vuu1sDiyQht7Ein1z2adFEF48p9vSET22/NYLzppCADHD/kPSqFuuXdfWGUaSoH9cP3TUhbKh/K63dkYCPbZbkk79/jFJgNseIz3O9vG29/mTZhEQiQ/KZRt1kIaOCOyGwPyEGD1HycWfag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GOpDnCXDg05zX46BeGyQhbthEhLxry7AJ9p8Mph3xEU=;
 b=i6chVexfdNSFs5+h5NNOnOogz4NWCT/zDIPa9qnys3obIlsEBeiPlig57DRhV4+29KIUH3Eocpr0hEbnD570Bu2UCSMRAIVLJf4I/I/tWqfjiAD+SX/t7MuIqvTfnZzDNGjKLpgUH5BAZ0lu2TZDN41mdcAaLZdozPXODuVY40ZN0F1dxftZrCswFkN/xe6XmKWf6OzH4+Ia7gIv0ns2CWlQA81ITa40m0FNzY/hSayRaC0E3AZIbaYbt3vBCaNx/xl6LuewO9NwXK4VoQ05Imk7A38I+chrvbGlcp/TDRoZc2WD6kZ2c3zAoH6bFO9KL4S85o7+IPkd3tIZLVwqdA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Michal Orzel <michal.orzel@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        "jbeulich@suse.com"
	<jbeulich@suse.com>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        Rahul Singh
	<rahul.singh@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Index: AQHXsHpCfe0n3k8oF0eTl7KOfxT3R6u6ssAAgAABBoA=
Date: Wed, 29 Sep 2021 08:16:55 +0000
Message-ID: <2a2a3122-ff1f-9002-9ad1-58f339bb4c42@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
 <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
In-Reply-To: <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4d549037-58f9-4312-6ecc-08d983218047
x-ms-traffictypediagnostic: AM0PR03MB3713:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB371318298FDAAE74B295DDD1E7A99@AM0PR03MB3713.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3276;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 a8lLMMIhykQF33xcgWPaudqL7WiXnjs6N84CieBElNEZ17jtTnluhxtY/L0IB4o9GpyyiUyw5DqZSgRlSMx37BmoXQYyNaGz0rXLZgHMjFDy4k0o/l/yMZW4p40nmeSIhBTGq4q5YztpaquonGvz62w/ZH0hm6qHw32Cai/xh3wNdE3AdwmCIha+1qjq0+vcDveorz0oZKTMTRhaEHIv+dGz8i7bTlSaNKVRqq7WE7eAg04ryOfqe9K8Eq0Hhj9XxqLDsOAas+2b6bo1+AVY7h9JxXAatIrBb+FGKNor7Fx1m3m385nHks6debw0RZZlcBEv4YBcF0727dX+jT7eMlHr7veAEfNS/n7XmCOuIaOi2J0xe7pcThYlYQ7wTo0KR9ZuPNtjpPrz1b96/5Nfb9c0ZaZLmNT3OM3yS8P5p7IlljpMFWrni3xwaKA2m6AEnn4BdqBEOz7TN6ZWAgPh0PQwNIJ4+aUVmFQQ4ZZYgBf07CdNjIix6ekT964g5VAxjdz8AqmXCV498gtWXtar0A6CcxSxpZR5r/Wpm96rxLrvQ0jONSgzox7ASj+5CX+rT9dXko9KogN624VwSZqlr8euV9mJMCWN4vibHxnVpIseWT6XUHv7WSiUhrO9Re0KZWJBX6LvC9mJczIcu9BokPxA8MZtsnQ7h5IfGiMYByWAaQT2Ac9fRmg7fn/LfEbc9oM/1yFODT8rDVVgFiT0p1LuTnRK9q5JkToL/ysqVqbeU+QGAolwgQRaB3rYVpFh
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(31686004)(6506007)(36756003)(66476007)(64756008)(2616005)(83380400001)(55236004)(66556008)(66446008)(508600001)(4326008)(38070700005)(66946007)(76116006)(107886003)(71200400001)(53546011)(6486002)(2906002)(316002)(38100700002)(122000001)(6512007)(8936002)(31696002)(54906003)(110136005)(26005)(186003)(86362001)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?amRUeDRMMGpYbFZ2SUJqSEx5MFNkWXFPbXpsT1AvMmQ4S0Q3NDRJWWhkVUR2?=
 =?utf-8?B?RUVXYlJ0eG13MG90bFVhY0NRYWxNRUNlREZNRytpTG1qOVo4bGVkc0VnZG9W?=
 =?utf-8?B?bWFnQndJWEZJVWNlODVhT2dSV2ZhVG1Mc25nTnAwUVhvUmp5Sk9zYm5yT1A1?=
 =?utf-8?B?NVJoVE5xTFFvQ0pnNlZBVSs0ajJrcU9EbERwUS9rbys0Yng3TEdyWUFHNEpk?=
 =?utf-8?B?K1M5dWc2TzdGZko2Z2kxbUlhZ2hjWGVVdlhpNmNIV3AxMk5QZURuRHVTT2tS?=
 =?utf-8?B?NjJmVGJJdUxGQ1cxZCt4RXduRnZaak9BcXZ0Q1Y1Sks5b1JEMFZWanhmZnpQ?=
 =?utf-8?B?amVQSUM3NGZPRzNYSEN6ZEhaenhlVmQzWmQrYTIrSkIvQXU2aE9KYXAzQVQ1?=
 =?utf-8?B?WCs0VTV5M3djam4zVldKbjdwTm5iNzQxWFNzeFJXdWZjSUFHMlJPNXJocURR?=
 =?utf-8?B?T1NqYjRhYXB4V1hiSkF0dzhjSW1Wc0tQMGZ1R0dQejJVSHFEcjg0YjZKUHVX?=
 =?utf-8?B?QzQxVEI5R05haEsyellqMXRmQWxGQkJHeHdwSU4rKzUzV3ZSNHdjYmpSem4r?=
 =?utf-8?B?VjdrSGNjT2NmMndLMXlwZjVJUGZRME9mZUIzblduT0VYeStMaGExcG9OTzZD?=
 =?utf-8?B?VjY3TEpMdjRRM1FaQWN2VHV2SS94VFFmdjVlV2U3c1VaK3V3YnZIQzFSajNC?=
 =?utf-8?B?eVlUdDliSGpVN3kyUU52a05neDNQRDhML3ZoRlRhQkM3bXhyN2tSckJlRWtV?=
 =?utf-8?B?bGovWGpheG1mbjhVSHlKd2lKMmI5N2V3Y2VXYXhyb0JNOEtwNU5OSHIweURE?=
 =?utf-8?B?MlpQTE5hbnNLWGo1WCt1dFgyRnVsZnJveFdEb291MGJNQW44YVRieTZ3UGRI?=
 =?utf-8?B?Q2RMUXpiaE1ORERJdEw0MkVZUXJqRjhNMTQzWmUxL3N2cEo2bUVEaGJ6Um50?=
 =?utf-8?B?cVM1VisvRlU0NXpuNC8rM0JveGJrZ2Nob3RpNmlrc2plMnN6a2lhcVZreVlZ?=
 =?utf-8?B?ZnlGK293dGFGZzlkTjZNNHFFN2hyd1VyZWd2UUFpbDcvdi9LWWxDTEhLTFc3?=
 =?utf-8?B?NFZWd250MWdJeFFBRlFUMm9yZXpIZWlQVVAxNTFna2ZpQXBOMzVhQVBvdHF4?=
 =?utf-8?B?bkJRL0FUZE8xbkdUeUFLbjZrUlZwVDRpeXA1bjAvS1lVU2tidU05QnJWeXRk?=
 =?utf-8?B?WmM2d3hmK1A4NmhWTHI2Qks4aU5Va2QvU2lCNDY1Ti9nTkx2dXB3bzlyS2x1?=
 =?utf-8?B?R2JWMEtHcVJNQ2lhb3FkaHR1aUJKQ2Z6UVpZb1JIUllZZUFWa2w4dU56VTZn?=
 =?utf-8?B?a3ZJYkRzcDQ4bDdFamljRWJETTQrc0tycVpxREh3aVJHaXdScitlWGpONDZm?=
 =?utf-8?B?RTJDa0dDRmhFamNsdkdjbDNWaytrekpqRkZWeElGSXVMMUI1TmxMTktHRWpP?=
 =?utf-8?B?bWJIRG9GZlZ5M3YvY0ZSZkZUUER1VC81dEkvbGdsTWp6S09KamlBVmF2eFFP?=
 =?utf-8?B?TWhOYUZVeG9CdjFMZmhIZXV2OFVnR0QveElqcndGTmovTTFQaHRwaVdpTmJ6?=
 =?utf-8?B?SUYzaUhQb0YxTzVMdzVGclFuSmNVSWZraSthYXN6MVZNSVhSanpZQzBWZnM1?=
 =?utf-8?B?eG1zRUtvdXRPaHQ0U2d4STlEOFdnZnppSmdyM3d6YjhrMHhQMUQyMWR2V2VM?=
 =?utf-8?B?RlZJWTB1dGtxRkhidko2OU0vazZ5RTBrZXpwTE5pNmFuMzhSdHV4YVlFWDdH?=
 =?utf-8?B?blA5V0R4UXVmQUpQLy9xVnIyNXc1QXExRCtNVXdEVWNnRnpSZjJLbWlSNCs3?=
 =?utf-8?B?bHZPQ2d2ei83ZHNXWisrQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <3D9D997E9A1A894E8E0B534C8AC9F1D2@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4d549037-58f9-4312-6ecc-08d983218047
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 08:16:55.7247
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Y3agMluNOHZTXFSM4pT3O5ujbQiLSBXvWZsuRWCsrpmBg/bGJkyNRyrf+gDgA6EwmGbovtZq8h6GejHN5qORw3BpcV1+GKo/PXMw6gFy3HGO76hlFt3gDFwcpscvXw28
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3713
X-Proofpoint-GUID: qLaRTHc3N9hjO4fKK5m8mBjngETbbk3L
X-Proofpoint-ORIG-GUID: qLaRTHc3N9hjO4fKK5m8mBjngETbbk3L
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_02,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0
 impostorscore=0 suspectscore=0 spamscore=0 adultscore=0 malwarescore=0
 clxscore=1015 phishscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109290048

SGksIE1pY2hhbCENCg0KT24gMjkuMDkuMjEgMTE6MTMsIE1pY2hhbCBPcnplbCB3cm90ZToNCj4g
SGkgT2xla3NhbmRyLA0KPg0KPiBPbiAyMy4wOS4yMDIxIDE0OjU0LCBPbGVrc2FuZHIgQW5kcnVz
aGNoZW5rbyB3cm90ZToNCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2Fu
ZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiBUYWtlIGludG8gYWNjb3VudCBndWVz
dCdzIEJBUiB2aWV3IGFuZCBwcm9ncmFtIGl0cyBwMm0gYWNjb3JkaW5nbHk6DQo+PiBnZm4gaXMg
Z3Vlc3QncyB2aWV3IG9mIHRoZSBCQVIgYW5kIG1mbiBpcyB0aGUgcGh5c2ljYWwgQkFSIHZhbHVl
IGFzIHNldA0KPj4gdXAgYnkgdGhlIGhvc3QgYnJpZGdlIGluIHRoZSBoYXJkd2FyZSBkb21haW4u
DQo+PiBUaGlzIHdheSBoYXJkd2FyZSBkb2FtaW4gc2VlcyBwaHlzaWNhbCBCQVIgdmFsdWVzIGFu
ZCBndWVzdCBzZWVzDQo+PiBlbXVsYXRlZCBvbmVzLg0KPj4NCj4+IFNpZ25lZC1vZmYtYnk6IE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4N
Cj4+DQo+PiAtLS0NCj4+IFNpbmNlIHYxOg0KPj4gICAtIHMvTVNJL01TSS1YIGluIGNvbW1lbnRz
DQo+PiAtLS0NCj4+ICAgeGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYyB8IDM1ICsrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKy0tDQo+PiAgIDEgZmlsZSBjaGFuZ2VkLCAzMyBpbnNlcnRp
b25zKCspLCAyIGRlbGV0aW9ucygtKQ0KPj4NCj4+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy92
cGNpL2hlYWRlci5jIGIveGVuL2RyaXZlcnMvdnBjaS9oZWFkZXIuYw0KPj4gaW5kZXggOWM2MDNk
MjZkMzAyLi5iZGQxODU5OWIyMDUgMTAwNjQ0DQo+PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL2hl
YWRlci5jDQo+PiArKysgYi94ZW4vZHJpdmVycy92cGNpL2hlYWRlci5jDQo+PiBAQCAtMzAsNiAr
MzAsMTAgQEANCj4+ICAgDQo+PiAgIHN0cnVjdCBtYXBfZGF0YSB7DQo+PiAgICAgICBzdHJ1Y3Qg
ZG9tYWluICpkOw0KPj4gKyAgICAvKiBTdGFydCBhZGRyZXNzIG9mIHRoZSBCQVIgYXMgc2VlbiBi
eSB0aGUgZ3Vlc3QuICovDQo+PiArICAgIGdmbl90IHN0YXJ0X2dmbjsNCj4+ICsgICAgLyogUGh5
c2ljYWwgc3RhcnQgYWRkcmVzcyBvZiB0aGUgQkFSLiAqLw0KPj4gKyAgICBtZm5fdCBzdGFydF9t
Zm47DQo+PiAgICAgICBib29sIG1hcDsNCj4+ICAgfTsNCj4+ICAgDQo+PiBAQCAtMzcsMTIgKzQx
LDI4IEBAIHN0YXRpYyBpbnQgbWFwX3JhbmdlKHVuc2lnbmVkIGxvbmcgcywgdW5zaWduZWQgbG9u
ZyBlLCB2b2lkICpkYXRhLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25n
ICpjKQ0KPj4gICB7DQo+PiAgICAgICBjb25zdCBzdHJ1Y3QgbWFwX2RhdGEgKm1hcCA9IGRhdGE7
DQo+PiArICAgIGdmbl90IHN0YXJ0X2dmbjsNCj4+ICAgICAgIGludCByYzsNCj4+ICAgDQo+PiAg
ICAgICBmb3IgKCA7IDsgKQ0KPj4gICAgICAgew0KPj4gICAgICAgICAgIHVuc2lnbmVkIGxvbmcg
c2l6ZSA9IGUgLSBzICsgMTsNCj4+ICAgDQo+PiArICAgICAgICAvKg0KPj4gKyAgICAgICAgICog
QW55IEJBUiBtYXkgaGF2ZSBob2xlcyBpbiBpdHMgbWVtb3J5IHdlIHdhbnQgdG8gbWFwLCBlLmcu
DQo+PiArICAgICAgICAgKiB3ZSBkb24ndCB3YW50IHRvIG1hcCBNU0ktWCByZWdpb25zIHdoaWNo
IG1heSBiZSBhIHBhcnQgb2YgdGhhdCBCQVIsDQo+PiArICAgICAgICAgKiBlLmcuIHdoZW4gYSBz
aW5nbGUgQkFSIGlzIHVzZWQgZm9yIGJvdGggTU1JTyBhbmQgTVNJLVguDQo+PiArICAgICAgICAg
KiBJbiB0aGlzIGNhc2UgTVNJLVggcmVnaW9ucyBhcmUgc3VidHJhY3RlZCBmcm9tIHRoZSBtYXBw
aW5nLCBidXQNCj4+ICsgICAgICAgICAqIG1hcC0+c3RhcnRfZ2ZuIHN0aWxsIHBvaW50cyB0byB0
aGUgdmVyeSBiZWdpbm5pbmcgb2YgdGhlIEJBUi4NCj4+ICsgICAgICAgICAqIFNvIGlmIHRoZXJl
IGlzIGEgaG9sZSBwcmVzZW50IHRoZW4gd2UgbmVlZCB0byBhZGp1c3Qgc3RhcnRfZ2ZuDQo+PiAr
ICAgICAgICAgKiB0byByZWZsZWN0IHRoZSBmYWN0IG9mIHRoYXQgc3Vic3RyYWN0aW9uLg0KPj4g
KyAgICAgICAgICovDQo+PiArICAgICAgICBzdGFydF9nZm4gPSBnZm5fYWRkKG1hcC0+c3RhcnRf
Z2ZuLCBzIC0gbWZuX3gobWFwLT5zdGFydF9tZm4pKTsNCj4+ICsNCj4+ICsgICAgICAgIHByaW50
ayhYRU5MT0dfR19ERUJVRw0KPj4gKyAgICAgICAgICAgICAgICIlc21hcCBbJWx4LCAlbHhdIC0+
ICUjIlBSSV9nZm4iIGZvciBkJWRcbiIsDQo+PiArICAgICAgICAgICAgICAgbWFwLT5tYXAgPyAi
IiA6ICJ1biIsIHMsIGUsIGdmbl94KHN0YXJ0X2dmbiksDQo+PiArICAgICAgICAgICAgICAgbWFw
LT5kLT5kb21haW5faWQpOw0KPj4gICAgICAgICAgIC8qDQo+PiAgICAgICAgICAgICogQVJNIFRP
RE9zOg0KPj4gICAgICAgICAgICAqIC0gT24gQVJNIHdoZXRoZXIgdGhlIG1lbW9yeSBpcyBwcmVm
ZXRjaGFibGUgb3Igbm90IHNob3VsZCBiZSBwYXNzZWQNCj4+IEBAIC01Miw4ICs3MiwxMCBAQCBz
dGF0aWMgaW50IG1hcF9yYW5nZSh1bnNpZ25lZCBsb25nIHMsIHVuc2lnbmVkIGxvbmcgZSwgdm9p
ZCAqZGF0YSwNCj4+ICAgICAgICAgICAgKiAtIHt1bn1tYXBfbW1pb19yZWdpb25zIGRvZXNuJ3Qg
c3VwcG9ydCBwcmVlbXB0aW9uLg0KPj4gICAgICAgICAgICAqLw0KPj4gICANCj4+IC0gICAgICAg
IHJjID0gbWFwLT5tYXAgPyBtYXBfbW1pb19yZWdpb25zKG1hcC0+ZCwgX2dmbihzKSwgc2l6ZSwg
X21mbihzKSkNCj4+IC0gICAgICAgICAgICAgICAgICAgICAgOiB1bm1hcF9tbWlvX3JlZ2lvbnMo
bWFwLT5kLCBfZ2ZuKHMpLCBzaXplLCBfbWZuKHMpKTsNCj4+ICsgICAgICAgIHJjID0gbWFwLT5t
YXAgPyBtYXBfbW1pb19yZWdpb25zKG1hcC0+ZCwgc3RhcnRfZ2ZuLA0KPj4gKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc2l6ZSwgX21mbihzKSkNCj4+ICsgICAgICAg
ICAgICAgICAgICAgICAgOiB1bm1hcF9tbWlvX3JlZ2lvbnMobWFwLT5kLCBzdGFydF9nZm4sDQo+
PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNpemUsIF9tZm4o
cykpOw0KPj4gICAgICAgICAgIGlmICggcmMgPT0gMCApDQo+PiAgICAgICAgICAgew0KPj4gICAg
ICAgICAgICAgICAqYyArPSBzaXplOw0KPj4gQEAgLTY5LDYgKzkxLDcgQEAgc3RhdGljIGludCBt
YXBfcmFuZ2UodW5zaWduZWQgbG9uZyBzLCB1bnNpZ25lZCBsb25nIGUsIHZvaWQgKmRhdGEsDQo+
PiAgICAgICAgICAgQVNTRVJUKHJjIDwgc2l6ZSk7DQo+PiAgICAgICAgICAgKmMgKz0gcmM7DQo+
PiAgICAgICAgICAgcyArPSByYzsNCj4+ICsgICAgICAgIGdmbl9hZGQobWFwLT5zdGFydF9nZm4s
IHJjKTsNCj4+ICAgICAgICAgICBpZiAoIGdlbmVyYWxfcHJlZW1wdF9jaGVjaygpICkNCj4+ICAg
ICAgICAgICAgICAgICAgIHJldHVybiAtRVJFU1RBUlQ7DQo+PiAgICAgICB9DQo+PiBAQCAtMTQ5
LDYgKzE3MiwxMCBAQCBib29sIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQ0K
Pj4gICAgICAgICAgICAgICBpZiAoICFiYXItPm1lbSApDQo+PiAgICAgICAgICAgICAgICAgICBj
b250aW51ZTsNCj4+ICAgDQo+PiArICAgICAgICAgICAgZGF0YS5zdGFydF9nZm4gPSBpc19oYXJk
d2FyZV9kb21haW4odi0+dnBjaS5wZGV2LT5kb21haW4pID8NCj4+ICsgICAgICAgICAgICAgICAg
X2dmbihQRk5fRE9XTihiYXItPmFkZHIpKSA6DQo+PiArICAgICAgICAgICAgICAgIF9nZm4oUEZO
X0RPV04oYmFyLT5ndWVzdF9hZGRyKSk7DQo+IEkgYmVsaWV2ZSB0aGlzIHdvdWxkIGxvb2sgYmV0
dGVyIHdpdGggdGhlIGZvbGxvd2luZyBhbGlnbm1lbnQ6DQo+IGRhdGEuc3RhcnRfZ2ZuID0gaXNf
aGFyZHdhcmVfZG9tYWluKHYtPnZwY2kucGRldi0+ZG9tYWluKQ0KPiAgICAgICAgICAgICAgICAg
ICA/IF9nZm4oUEZOX0RPV04oYmFyLT5hZGRyKSkNCj4gICAgICAgICAgICAgICAgICAgOiBfZ2Zu
KFBGTl9ET1dOKGJhci0+Z3Vlc3RfYWRkcikpOw0KSSBjYW4gY2hhbmdlIHRoYXQgaWYgaXQgaW1w
cm92ZXMgcmVhZGFiaWxpdHkNCj4+ICsgICAgICAgICAgICBkYXRhLnN0YXJ0X21mbiA9IF9tZm4o
UEZOX0RPV04oYmFyLT5hZGRyKSk7DQo+PiAgICAgICAgICAgICAgIHJjID0gcmFuZ2VzZXRfY29u
c3VtZV9yYW5nZXMoYmFyLT5tZW0sIG1hcF9yYW5nZSwgJmRhdGEpOw0KPj4gICANCj4+ICAgICAg
ICAgICAgICAgaWYgKCByYyA9PSAtRVJFU1RBUlQgKQ0KPj4gQEAgLTE5NCw2ICsyMjEsMTAgQEAg
c3RhdGljIGludCBfX2luaXQgYXBwbHlfbWFwKHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVj
dCBwY2lfZGV2ICpwZGV2LA0KPj4gICAgICAgICAgIGlmICggIWJhci0+bWVtICkNCj4+ICAgICAg
ICAgICAgICAgY29udGludWU7DQo+PiAgIA0KPj4gKyAgICAgICAgZGF0YS5zdGFydF9nZm4gPSBp
c19oYXJkd2FyZV9kb21haW4oZCkgPw0KPj4gKyAgICAgICAgICAgIF9nZm4oUEZOX0RPV04oYmFy
LT5hZGRyKSkgOg0KPj4gKyAgICAgICAgICAgIF9nZm4oUEZOX0RPV04oYmFyLT5ndWVzdF9hZGRy
KSk7DQo+IFRoaXMgb25lIGFsc28uDQpkaXR0bw0KPj4gKyAgICAgICAgZGF0YS5zdGFydF9tZm4g
PSBfbWZuKFBGTl9ET1dOKGJhci0+YWRkcikpOw0KPj4gICAgICAgICAgIHdoaWxlICggKHJjID0g
cmFuZ2VzZXRfY29uc3VtZV9yYW5nZXMoYmFyLT5tZW0sIG1hcF9yYW5nZSwNCj4+ICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICZkYXRhKSkgPT0gLUVSRVNU
QVJUICkNCj4+ICAgICAgICAgICAgICAgcHJvY2Vzc19wZW5kaW5nX3NvZnRpcnFzKCk7DQo+Pg0K
PiBDaGVlcnMsDQo+IE1pY2hhbA==


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:19:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:19:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198817.352503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUnu-0008AO-8O; Wed, 29 Sep 2021 08:19:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198817.352503; Wed, 29 Sep 2021 08:19:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUnu-0008AH-5S; Wed, 29 Sep 2021 08:19:02 +0000
Received: by outflank-mailman (input) for mailman id 198817;
 Wed, 29 Sep 2021 08:19:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUns-0008A9-C7
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:19:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 281f411f-6543-4785-ab68-5b9638b250c2;
 Wed, 29 Sep 2021 08:18:59 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2056.outbound.protection.outlook.com [104.47.5.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-19-jmkjQlbkNxiDy7szrwI5CQ-1; Wed, 29 Sep 2021 10:18:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4445.eurprd04.prod.outlook.com (2603:10a6:803:6e::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Wed, 29 Sep
 2021 08:18:53 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:18:53 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0058.eurprd06.prod.outlook.com (2603:10a6:20b:463::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14 via Frontend
 Transport; Wed, 29 Sep 2021 08:18:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 281f411f-6543-4785-ab68-5b9638b250c2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632903538;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=vTFJCWFjxefy2x10SalVUMklw74woPCuOh9Rm2fxbV8=;
	b=Pw8AZl04THMKR43JKQbxSYBrUg/P2VigrqCB75uvLtpu7YCkwa/+PPf73lET2+Fv8+0mrd
	avk1q52EjPc2ito0RMFT19IDm7ZIldnDTZjB9gxCWkqtaTZ41sawxcXdpwqj5KvzP6J5l5
	wLExnE80Ovidyq3DFihhlpBmQ0tG3V8=
X-MC-Unique: jmkjQlbkNxiDy7szrwI5CQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MS+XvJ2kTErcE1DlvqjG/m3DxXBiMdkjQ4/kPGRJNrvxN2w5PWn8rPoSOVBsWCAxGZnYzL+Zi+DPwsHlc/+s/n/k4cHP2S9KUwUT7f8SOv43TIdkpdQndUzr0o+VgZDH36CvHeebDG25PS4Smz8+pRkBa8vj4S5j+//I37xWrSLPvWL1zexdYgYBtcTUvB/QPZhOLVs0AFgzSwMySwKHwadZ5eFx/hBMexK80/Ll1yreH55nxAx4W+BxG+/g7ieC844PsCzppP/RXv7/8kEVtDNFil/jfblcHERh/17epUjkoA27Ptn46Tb0jgxf0aPgrvPYfbaDa3StTpBUeHt/vg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=vTFJCWFjxefy2x10SalVUMklw74woPCuOh9Rm2fxbV8=;
 b=Pey5OQ6LxCuXfevoEkRjyDDAtKGMADIKn4TKKq5xPrRBsr1BFVw1eoqkxifRAWXWvR1Mc4fpedPrTy8WuLe20VcsWNUraz20KHClICkvOXbVdixF+fb8GMWDpLBBqsEbTEIqoMcphvsj4so5BVS9q7HaqXjr8MAIiu/TZckPwTPpfh3sX/26t2sWWJZBFqFvUBCG7XvtQl2x2B1Rw5z4gqOxSp+SjhPKxCEnANW/s45944r3moKWUTKMhItctna3MszMf5uJ21+dZmUS2O+KLlF+DIydMaZhOOEet0Fp/zh2ixvfO16MVM9A0+oWB94PbFduVf9Ucv6+Uu/23kZruA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [adhoc test] 165241: all pass
To: Stefano Stabellini <sstabellini@kernel.org>, iwj@xenbits.xen.org
Cc: xen-devel@lists.xenproject.org, julien.grall.oss@gmail.com,
 iwj@xenproject.org, dpsmith@apertussolutions.com
References: <E1mVGlH-0006re-DA@osstest.test-lab.xenproject.org>
 <alpine.DEB.2.21.2109282301540.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <9dd9b794-881b-e157-45d9-80affa19f443@suse.com>
Date: Wed, 29 Sep 2021 10:18:50 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109282301540.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0058.eurprd06.prod.outlook.com
 (2603:10a6:20b:463::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: da0a94a5-6661-43a0-417b-08d98321c618
X-MS-TrafficTypeDiagnostic: VI1PR04MB4445:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4445613701A94A005DAF2BBBB3A99@VI1PR04MB4445.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	M+nf/iIkG/bTa1DjxP5wowTDmBiY2kJLGVsZa7Q3JdZp4A2OXAkfkJyVJnvHT3KqrPFyaOSixOnh5leyGkMfGbI0bFqkE8tsFIKke/BHkYLLp6cgGYwkHSYPpCyRfRfrtK6a7Zi+5Ixtr20eayJ7YZHZL5C9IK1Wumh8u9kQCmY5PXB0hEl5IlJLVvucm9lH0DzhrVrcO/kAeI55QSkPOUjBprNErM8T6/vejH1ZhHiuu6maRrkN0x1Ano3yJXaCTjKeNu0DGOAf+/xRnr6vCH0RHvxZUpb30K9yropK3uya3vXDK37dTesaCgO8jy9Nag02UwqV3yjfp9Fa3wi+QFDTkRJwZqhuMJgwf6nfmLZZ31uNNxT5WiTz3lxIbD4p5YqHPl7VyzQ17SGLpl31FGxiG82BtdHSi/sd4E9lymTfYgkVX3p7Uo1tlntiRWk7trfp1aIe9aX7dt6D8YnYXt6EALDnVb5wueg3YLpwWZcBOBaqunU95Md0IBhayvVyinOw/91Qa4zO66oIsgC6E8mi0/gaAjDylbxAMrX2QaspiLd5HB6cjMqvQj0FFTj5Gk4Uf7EjLC8QrlwsKE2PbqcJUbWOvvSLznhAXc3KMWRTrJ+XwwkqIigg74KsZfcPj8Fp/KEa+sawr1+iddKfErPteNBOe6XSJfixgAX6S7zHbuLZezVkgJkH6UPkCdaxVigwABqTGv5B+Kv0G3UE+ixbVfD3HSLQYUKc5OODqNw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(508600001)(558084003)(8936002)(6486002)(8676002)(86362001)(38100700002)(26005)(66946007)(186003)(36756003)(31686004)(31696002)(2906002)(316002)(5660300002)(53546011)(16576012)(66476007)(66556008)(4326008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dnA5bTVDb0RudmNCaUFqMml4eXFBL0hVTlovV0M2RmFGVVdVUGNna2F0UUw5?=
 =?utf-8?B?WkRCTXVTNzBPQ3QyQS82dTBtRXpCcVYrdUErdzUxTHY0OGFUY055OTJ1UGxG?=
 =?utf-8?B?SVZoMXIwWmhCWWhQd0x0WVRhZzF1VDJEeUVGNnNobzhIWUp6S3MrYmc0aTVW?=
 =?utf-8?B?ZU1HMkNvYUVwV2NjT3piYzFrcHRVdnUxS1JCR05iOE9EdFBFVnlqNFNFQktN?=
 =?utf-8?B?VDRCbnlKdEJoem1tbDArNENQYkFQdkNjNnJkZVpvWTlnVkJlOE1FcUlxL2dP?=
 =?utf-8?B?b0xWUFVCelhsd012YWxONldUVUNmdlM4NVZNS1hGNTIxU2cvSTQzQURlZ0pE?=
 =?utf-8?B?Rys5UGhsYU9TZFEwNEZYMXFEb2JKRWQzQUIvUDZqWGtraGZqdE9rWlRRMmpL?=
 =?utf-8?B?Q0JBdUlvdXpmNUcwTTBpdGYrM0FBcm8rYVg4UHpzWWFScDBwcExTQkdBMm9C?=
 =?utf-8?B?TndxeVJKS3ptR0U2K0RuN3VOSG41S2R3dU1rN256S0tqSEhtZURkUmJLUTNx?=
 =?utf-8?B?d2VFMlRIbVcwbjlsbVVkZE9GWitIc3JZVU95dFdwckp3WjZzZC9zT2tidEJX?=
 =?utf-8?B?VVR2WVloSzN3QkV0QW8ybk1yN1RFTzBRVzdYSXBlVVZwZE8xOTFsWWJnWmx6?=
 =?utf-8?B?d1NVa1BaOGZpdGYrUGVyNGtxMjdXZUpKRGNYenR0Z1llYldZd2NNZlE0WVQ3?=
 =?utf-8?B?VmdUZkloRGY4WlVtRHJNdWc4OEkvMmo5S1ZGMHoyUXJCN2drbTE5dTl3RWsv?=
 =?utf-8?B?YVZiVWpGOVc3L25vWm5HaUdnUDNJa256ZGdxenQrNFc5Slo2YnhXZThacTlu?=
 =?utf-8?B?U01FdHQxek9icWc5OFpGMkk3cDYzdjdaa3U0d3VFM3NvclYvRVFNRVFwam5X?=
 =?utf-8?B?VHBpODhaUVZ3MkxySm5rMVpPU01aRVlhQ1VHOEZNMnBJNnpLM001Qy9adE45?=
 =?utf-8?B?L3pvSWJkUWZ3c2xVQ2FhaDJYTGFFaWl5VzlKTFhpdkRTSUhzdHlvdEkrdTBU?=
 =?utf-8?B?SDhPcUh5YVVjOXhEL2dPYmJHem5DMEVXT0Q5TlFyc2t6dysrZ2ZVcHpBMmNq?=
 =?utf-8?B?OHExbUNYYXhXcE5rUEl4d1dsc3pLNHc1cGhTdTZ5SHdkaW9JR1FmNnRZU1ZV?=
 =?utf-8?B?U0piSlVUSStTeE9XdE9QUVFxRkhuM3JaTVhXN0pTajl4QlllL2sxSnBhZmVv?=
 =?utf-8?B?MlZsSGQ1bW5FY2ZESzNYZFZMMVVCK0V0dm9WVU9jdUw4RHNkUDNybWVKcTFt?=
 =?utf-8?B?Q1ZHcHpwYTZSdHJrdnRqbENvVFp2N29FVmNrSCtpUUFjUVNPTXB5N0NJbmE5?=
 =?utf-8?B?YzBjQWRZUHVXYzR6cmhpZXpZS21xUDdlZ25Vc2h5WWEvTGRoRlhSdlVuQml3?=
 =?utf-8?B?SzYxaTVtaHJ4MXR0c3BIYWV0dWYxOWkwTXNqV3VvMWM4ak9SVmYySTNPNHVS?=
 =?utf-8?B?bjhjZjZIbDdWRUM5em5ZVnFlV1lianRySWR4MUhaQjdRYitrd0dzSkpqSEN1?=
 =?utf-8?B?L00wUFlPdG5ZaDdDYUphQUdQM2pQUVYvVG5HSy9YRi9RYkYxUlZSU040dVFw?=
 =?utf-8?B?eUl2eUtnR0F0Y3pxYXRUYVJBdG9POXU4My9hRmlQWHJsVlNLaWxGU2Y1enAw?=
 =?utf-8?B?TXhzMzRvajliVmlQckxnbU9KaXpMaENyelIya0dSdVREVCtDeGhyVkNSelZ1?=
 =?utf-8?B?U0Vzb0tJU29Vd3lDcnk0OExOaXlHbTEwNGtRVS9CVWpzM1N6aUl0SjlaY3gz?=
 =?utf-8?Q?e4B+in7orEieWvIzczJw+qUQr4LV9gFbi+XtP1r?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: da0a94a5-6661-43a0-417b-08d98321c618
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:18:53.1333
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BxdzEpAjphWYiWWJOyUfKJckeVzdyLDN1NPDCkUF5q7OsAYjjUACpSN+ISX+3xsSchMCXL+9HO8hRCuWEKUO5Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4445

On 29.09.2021 08:03, Stefano Stabellini wrote:
> That's great, Ian.
> 
> Should I go ahead and add the patch to linux-arm-xen? Or do you want to
> do it?

This was only the build flight afaict; we still want to wait for the actual
test to hopefully indicate success.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:24:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:24:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198823.352515 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUsv-00018E-Rw; Wed, 29 Sep 2021 08:24:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198823.352515; Wed, 29 Sep 2021 08:24:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUsv-000187-Ov; Wed, 29 Sep 2021 08:24:13 +0000
Received: by outflank-mailman (input) for mailman id 198823;
 Wed, 29 Sep 2021 08:24:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVUsu-000181-6I
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:24:12 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 158a316d-e6ed-4f47-8d43-e507269985ec;
 Wed, 29 Sep 2021 08:24:11 +0000 (UTC)
Received: from pps.filterd (m0174681.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18T6U8uo024294; 
 Wed, 29 Sep 2021 08:24:07 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bcjxa8bqr-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 08:24:06 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM4PR0302MB2802.eurprd03.prod.outlook.com (2603:10a6:200:93::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 08:24:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 08:24:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 158a316d-e6ed-4f47-8d43-e507269985ec
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YBVpCuG66MRVuauC5xNxtX51wyl4CUTXc7JF0a7Xl6gR7teh6yMJm3MlvzWfBpnifPlKdsJjsj1U9XB1b/9FgsvCFxRX66S2wWxrTO4Z2iOdX45pQmPpATNvZDXysGftsVaIrjVA1UkiG1jAY5O9x3Uyw96yvmqE1hpyUJ5zOxJy11+7f+9KR3iKdV4yVgOZmd8Q2rEhAVU6EYlauDl2Ng7oGZKI5smmd+MCPXUhiK84g4KX36TR6KxbwcKorsP5NMAiU1/DFDLoNarivVu2UTUjasjuIqXWq+65JWrVKhPH5bE3UQCi5WFd9wI2pLLzzi8bV/Y2Smat2Yt0CZH8HA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4T6pSvEDVTQ1HJIa1MyHSz6tDGVwQKQ+pKNWCmHVUqc=;
 b=NwOITaERwHMa5xIwFsgpila6G9hvm9cRmzgGtDVybP5OOziirbPI4MFvRYOaukDxECEi7/MwX4T98fcObPG6b6RwW6y+mSizfaV0v6Ym/A+ATCH62QATXN1bgb/95XobbffvpJo9oKLAvx4Ux00GANAzbirEDXt28T7OUXqP7Ku3N1MI+QZa4+nCZZPqfDHHwbeAFZuIK62KI56uyJVEItYyeoN10oZQvuo6fybPuw5OyFL9G6PCzoh93nPM92lkIbyL4+yGC+ugyQhwCmHMTRZSlNEStoZzBlywU4FLgHsb3DANI25A52Np1pz2im9h7RH686JlCp8qIY0/+EIGkw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4T6pSvEDVTQ1HJIa1MyHSz6tDGVwQKQ+pKNWCmHVUqc=;
 b=ELkVyJLsxDar8bkfY5zjvSsbAZ9LXwDXnq1c2t1AHAm0MaWnCRqhYAQy5H3vPfQu9spOdmYraUow4VhhZVYgk6Vu6Bm8lWVaUDzJbWJWG7gio3qtn8dZoZfIX9jMNx5CnvTJeIB4YWnerq4p5ii65kThqYgeHvGqjsgFbX1+37Np/P5gTyojvZGsq9oaXuTx0JvKJZD0Ia/8y4DaoHm40d1EMke9wF+0epV0SDSLu7gqk5BOg3DOI4NHTgTG+IVYyXVMqZKMyycyJr/96fMw0lGmXBh1EoncT2zEkEn1WWEulTCbVHFSXFU29ygQUT6z4ShNax+R9bcXdgCG220utw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>, Michal Orzel <michal.orzel@arm.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Index: AQHXsHpCfe0n3k8oF0eTl7KOfxT3R6u6ssAAgAAA3gCAAAIlAA==
Date: Wed, 29 Sep 2021 08:24:03 +0000
Message-ID: <7ae0fad7-499b-1bda-5c9e-9313265abf8c@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
 <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
 <f59b48fd-564e-64f8-1422-fb842957f213@suse.com>
In-Reply-To: <f59b48fd-564e-64f8-1422-fb842957f213@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 851b1c07-8a40-4cec-f5b0-08d983227f2c
x-ms-traffictypediagnostic: AM4PR0302MB2802:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM4PR0302MB280231291F080309CC45F4EDE7A99@AM4PR0302MB2802.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 aGVo7xuI/7auVbLLZ+9OTNcbOK1IAweOvP7/4x5FcLOiEqMmSMeMcnBCiXChP4LuvfiMMZrzuzdU625I16g0v7XPCue+67fYXPagCjjV9VzbasudfvJvQZTm5qGSzsSfGlNdH2KD6kehFDiyOmHGn48bL9rS6p3E2U68YJ3BS7IeE1Cr+3EJxMdkcUCO61SKkWyxGYqVdyvpiPqE6ONWOvNM8GzD99F+HEscrm17nLqrbJnYSXmcihxyLKsur7ndEaDnt0jOutFK3UkMr9jkkHNePhvstgrZH1jioVfLSE33M6Q5DhfLzADNXzIHQipmBzPj6c4hrVZ7EaMhSI00tIBnH22ZTEOQy/B0lh0sX2iVpI4F5Vm+mDt0PxI7m/tbZzpgy5R6iFdDUz+Q8RzPGs2trHRIvCXmbp8R52JSw99vHv2yMqgzhCk03jwEtwiR2cz5wcJvDOX0B5KnFt4a6ELaYmAc7fUlc8XOQKNNB2WDgTKkRt+yBCsa+EIv9L428ISW+yEtGdAk4l2ak/ygQuHWEHmnUh6T2m5m3UOYG2dZ/6I7bkIUG87Ds+3cP/6HTGiQmgDK+utw/AiOudiEgToc+SQbcoveIT9Y2t20diwymg2zOdrTBsq8xE3F62yJk+MhCURKkSQoceaUydhEot/pkUSvvtkhmXciw95hKTqgTYFbSpDeaHnk4wVMVIoLSd0c+KmDP6aN8eU9cTQdvG/XFWOWzk1nj/Pdi2JrWYA=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66556008)(64756008)(66476007)(110136005)(38070700005)(66446008)(508600001)(53546011)(55236004)(6506007)(86362001)(71200400001)(54906003)(2906002)(186003)(26005)(66946007)(76116006)(4744005)(122000001)(8936002)(4326008)(6512007)(8676002)(316002)(2616005)(6486002)(31686004)(36756003)(5660300002)(31696002)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dFhBZWIvWlg5bmQ1R0hyMDE3UHo2T2FUSzdqQ2p5emNUV1FhaFd2VUJDL2Fa?=
 =?utf-8?B?anVlOXlCdnlhRUlYZUVQbDYyV0EvbE1iandnMTV4Wm55SnhQRHNzTUJCTU9n?=
 =?utf-8?B?YzFrc0tDWng1NkZDODYrZ0RveWRwZjJVYUdZM203SXRCeW9UdnNOdmo3MlA2?=
 =?utf-8?B?Tk0vSTJRZzF5S2w4OE9Qc2Z4UURCVVYvNjk5VnlSZUsvS0FkdDR0ZDdycWxW?=
 =?utf-8?B?c2NEODFsWERKRE5SS0UxOGhhblVEZk8vcCtPZHFrQ0pqa1h1bkF6OUZuU2lj?=
 =?utf-8?B?RGtGRTRoc3Z5V05jQ3ZPOHpjNjFKU1Z5cHNmM3Q1Q1RXZHZ1aDBYeWx3SjJY?=
 =?utf-8?B?RWNUSHpuZFNHdSt1SEo4Q1JWa1RiaGxodjNGQkdLM2pVRVVpcFJPZStqMzNE?=
 =?utf-8?B?V0FRSmFzLzFqSEluZ2k3dk14cUFxczNlQmhJQ1ZkZ0l1R2ozRHh3MWJxTk1o?=
 =?utf-8?B?cms4aGJuUWRoRHh2L01jNjA0aEpGWVE2LzNZNjJMRUJuUUNpUEhMVCthQ1Nw?=
 =?utf-8?B?bVVyYmxOemFkaUFieExLOS9HU2dkM0dORVJkbWdGK0dpZTE1SUdyMzFjUGlB?=
 =?utf-8?B?YkhJMVRhdGtuQUNlQ1kzTE1Ba1BsWUp1b2tyMzRMWkt6VGN6cHdFQ2RCMkpk?=
 =?utf-8?B?WnVxQkljVzVJa0RPalFaeXErd3ZmdmpyRkRacFB4OXJETXRZN09mbW90Mitm?=
 =?utf-8?B?K3dPaUIwdmlQdDA5U3JBRmk0cFpDbzA1SG5aZDBRL21ZaDhFMUZocGh6MnpI?=
 =?utf-8?B?Y0hBKzlqSmhtK3pqdUVsb3Y4b3BUdGgrdnU2SXB3cldMVkc1WTdaOG9LKzZy?=
 =?utf-8?B?cS9Qa0U0TGIxSFA5QTNyeStGa3ducWtQTkFqS244NnJkcXhZWE5heGJMMy9l?=
 =?utf-8?B?ak9QcnRYbExCOFRHdzgrQ0E5UVkrY2xTQ2VmbDY3RE92V2MzNW5aWkNYYnpI?=
 =?utf-8?B?akRydnlXTFJTYWlMUDRENkNzcFVZUUM1MHQvMHVBUzhHNWRFamN1U1V5NCtw?=
 =?utf-8?B?ZmxJWDJkaWYralV3N00xbURGQVQwL1hUQWxwS2lucit0UGdZTWhaZzdBWHJU?=
 =?utf-8?B?b1NGcXhtVXFBaWRNdWRnN3RlVzhVNnIxTlVJbTdUbVNGeGpyclVYWWxRWUVm?=
 =?utf-8?B?d1NoenZUdzRMRW9hOVE4NEh3dVI0MytQb2VYVnFsUElaNlI1L1N5RnVibDNp?=
 =?utf-8?B?RHJ5YTlWVjIyV29Jc042Q0pTVS9QNkwzNEF0cXBVM0VuVHNEZXlOV0VEVTBN?=
 =?utf-8?B?a2hLZExDVWNtb3BXYm1rdk9sb1lDUWtsQ3UvY0JzR0NHUmtOb3lMbld1ek9v?=
 =?utf-8?B?L3R4aTVhbHZoQjFEUWtjSy9xLzRKYW8wU2lZa2gwbnhoU3BORW54eTRBVkZG?=
 =?utf-8?B?WkZCUmdJOEpIMDIyZVR1ZWVqRDVFSGlrcXpDa1dwUWl5KzU2MnBndExySmFI?=
 =?utf-8?B?WUtiSTgwTnNqYkt4UFV1QkYxTkR5TkxDbHVubHdqSG83WmFuZjFsQ3RJR0lx?=
 =?utf-8?B?ZFFTL0lRZUZ4MDBTa2d3ZlZ6TDZEVlNObVc4MG1uSlJNVTE4bXl4R2J2VEFp?=
 =?utf-8?B?c3RaZGhsaEYxRW1BUnVLQ0N1MCszQ2tIOWM3YjBNTnkvQWwzd2hGenF5RDFk?=
 =?utf-8?B?TG9IazdWMDQ1QkEwMmxWTjdKekNDYWhKMytUTFplK3kxZVdxaG5Za2xqdFMv?=
 =?utf-8?B?aExScXBmY0xUMlpQOEdyckp5NDV2NmpseGFRRTA1NTFxTUpqVWhwRVMxdzFO?=
 =?utf-8?B?MHJraGRzdUR1a0thWHJKYXo0bjh3QmRZM0dtR1FYd04wVjdzUHI0STlBYkJq?=
 =?utf-8?B?T3hldktQUUNINklFK2cydz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <178AF666B789784897F7097CB2F977B2@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 851b1c07-8a40-4cec-f5b0-08d983227f2c
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 08:24:03.3623
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: oy90Bu9wZ8Nm1GPMcj03Qd1k+M0M3VIy/JDBeLaiG2+4KESxDBCAS+iQSdKqfe4mMqrto3J3TtBSIFrJprfaXh+kLFjpBioAjfIxZBbrjkBQLRsoWen04hkFaSBMA687
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0302MB2802
X-Proofpoint-GUID: Z9DqsPZ9OCHJxSAfyoFTEMl3jvcx9vAd
X-Proofpoint-ORIG-GUID: Z9DqsPZ9OCHJxSAfyoFTEMl3jvcx9vAd
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_02,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 adultscore=0 spamscore=0 mlxscore=0 suspectscore=0 malwarescore=0
 lowpriorityscore=0 clxscore=1015 phishscore=0 bulkscore=0 impostorscore=0
 mlxlogscore=747 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290049

DQpPbiAyOS4wOS4yMSAxMToxNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI5LjA5LjIwMjEg
MTA6MTMsIE1pY2hhbCBPcnplbCB3cm90ZToNCj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTQsIE9sZWtz
YW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+IEBAIC0xNDksNiArMTcyLDEwIEBAIGJvb2wg
dnBjaV9wcm9jZXNzX3BlbmRpbmcoc3RydWN0IHZjcHUgKnYpDQo+Pj4gICAgICAgICAgICAgICBp
ZiAoICFiYXItPm1lbSApDQo+Pj4gICAgICAgICAgICAgICAgICAgY29udGludWU7DQo+Pj4gICAN
Cj4+PiArICAgICAgICAgICAgZGF0YS5zdGFydF9nZm4gPSBpc19oYXJkd2FyZV9kb21haW4odi0+
dnBjaS5wZGV2LT5kb21haW4pID8NCj4+PiArICAgICAgICAgICAgICAgIF9nZm4oUEZOX0RPV04o
YmFyLT5hZGRyKSkgOg0KPj4+ICsgICAgICAgICAgICAgICAgX2dmbihQRk5fRE9XTihiYXItPmd1
ZXN0X2FkZHIpKTsNCj4+IEkgYmVsaWV2ZSB0aGlzIHdvdWxkIGxvb2sgYmV0dGVyIHdpdGggdGhl
IGZvbGxvd2luZyBhbGlnbm1lbnQ6DQo+PiBkYXRhLnN0YXJ0X2dmbiA9IGlzX2hhcmR3YXJlX2Rv
bWFpbih2LT52cGNpLnBkZXYtPmRvbWFpbikNCj4+ICAgICAgICAgICAgICAgICAgID8gX2dmbihQ
Rk5fRE9XTihiYXItPmFkZHIpKQ0KPj4gICAgICAgICAgICAgICAgICAgOiBfZ2ZuKFBGTl9ET1dO
KGJhci0+Z3Vlc3RfYWRkcikpOw0KPiBGV0lXIEkgYWdyZWUsIHlldCBwZXJzb25hbGx5IEkgdGhp
bmsgdGhlIGNvbmRpdGlvbmFsIG9wZXJhdG9yIGhlcmUNCj4gZXZlbiB3YW50cyB0byBtb3ZlIGlu
c2lkZSB0aGUgX2dmbihQRk5fRE9XTigpKS4NCg0KSSBjYW4gZG8gaXQgYXMgd2VsbDoNCg0KZGF0
YS5zdGFydF9nZm4gPSBfZ2ZuKFBGTl9ET1dOKGlzX2hhcmR3YXJlX2RvbWFpbih2LT52cGNpLnBk
ZXYtPmRvbWFpbikgPyBiYXItPmFkZHIgOiBiYXItPmd1ZXN0X2FkZHIpKQ0KQnV0LCBoZWxwIG1l
IHBsZWFzZSBicmVha2luZyBpdCBpbnRvIG11bHRpbGluZSB3aXRoIDgwIGNoYXJzIHJlc3BlY3Rl
ZA0KDQo+DQo+IEphbg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:24:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:24:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198825.352526 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUtB-0001Vc-4M; Wed, 29 Sep 2021 08:24:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198825.352526; Wed, 29 Sep 2021 08:24:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUtB-0001VS-16; Wed, 29 Sep 2021 08:24:29 +0000
Received: by outflank-mailman (input) for mailman id 198825;
 Wed, 29 Sep 2021 08:24:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUt9-0001TL-5w
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:24:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a81c7a55-20fe-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:24:26 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-44mRcsR-M-mjLuHpgOnEuQ-1; Wed, 29 Sep 2021 10:24:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 08:24:20 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:24:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0026.eurprd06.prod.outlook.com (2603:10a6:20b:462::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 08:24:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a81c7a55-20fe-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632903865;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=dc35Spxy70nhW60kM/sdbASAJlGjvSvVzTsJkyPWwmI=;
	b=hZZ1EmsX846a1+zWTgDQ5JGUM7HFj7YXZi0FPkxpOawKTLEOA7xDslO+rOoErWmS6eBksQ
	oYCh7GA47Z5l5zT2xFPc38pmbpU/y/b0GGn8kNzO1OIvA1kaohtfwKDL6D/syjSKEk8kxS
	27sZ3uTRBdxiODUZsF8d4z63pJmyces=
X-MC-Unique: 44mRcsR-M-mjLuHpgOnEuQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JdNWJb5YoOLW2QNKqjzoIxtFagHVNWIg9TRjjNPbtoXI2aHjFxz4k3Gqa7bsWHshj077ZQ4jTdvN4tRuRYtJPUX+q/mEGn7VARRJdjSNqW9pfKb9iTsSVemwxYL8RIIV7cwhNzSIk7uNOgXlgOrzmn+vtvQCRLog9P6S7XId5eHQYJk9RKPY0uDVFYC/OHF0Qvbnrkqc/TjlblCV5JrkMPmcrdA9lU6TlVDNKpcqTXgIFD+swutRn4dUURU/xcaIh9BmWpnztnB0lzyaPZfu09X5FcDcB1wDqlOgsNob05UjDCoBPrLYvZ9NmrWX9tof4hYEEG0BwncBVYkb+hp23w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xYLDlTKQG/veH3t6BApJ/5eZwO8H4btcFF9540vj7no=;
 b=bee6KJmrc0V849gs2cEh3G/naaUj6Lk/8fLJL+U08htNDaF2LCKj6GmpQw4TjGwkjZsVVCR+qcJOFWN8+Qpb5MbE2f96L6Ns5C/apMA6XZtKLkV0q3Ye23v5dEJZqoFHwbMfl2+3lQYdStF9LWPPIc/CDO63MmA0N3mhD8Cf7E72JXKVmnmwJxNPbuxfSAY9KX/fpC6F9ZPAFmvlhbIRsgRyxC5fMpEd8j3eMGmuELNhRA/0i2eMuZL1JELv2dnRAncie0SM6vVZpuwfD4OtSs7Bxfuc8sSFOZkNqeyklfu27bYJ55vYKGGlO3gPrvtOL01sDAaZPAB0f7J9oJ/yAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
To: Rahul Singh <rahul.singh@arm.com>
CC: bertrand.marquis@arm.com, Andre.Przywara@arm.com,
 Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1632847120.git.rahul.singh@arm.com>
 <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <67ae3cd5-e90a-5d62-b7f6-b9802ec8296c@suse.com>
Date: Wed, 29 Sep 2021 10:24:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS9PR06CA0026.eurprd06.prod.outlook.com
 (2603:10a6:20b:462::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 649277ad-e545-410a-574f-08d9832288f9
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB647950EA8E3CACAF67085CA7B3A99@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XXHNYjJ2xctKOTgiMSOrXfGocfYHdmvXGSQhhWKKPQW62vValBvdb6hFRkMjTv39ksV0DYYwmvx3P38AEDCUvPrzxBef0KMXJE3+WKXppKd0Oig/dzl2jQceTHxeoiN2nnKinNpv8bT0zZS23WDR1UCxyow5nvgdT3QslaB9Js2zEg/qJ/dgEdc+T4RnpruJ/445E81Wq3QgxkoryRGf7dzCELh36PhDmLRk8apSmfx76hTSajAVMiXUMYJzJAT5uQJDh1AadE9Z13iAsomZAZYjVeZOwrTgmLw9sMw/aYFwCJzCHb/zhJ0XWjc6Wm8PfdZYgqGVLkulO3ESnG9G5Lm97Y3IBwENSslpZ9pjZlGnXAwSkSkLlN4gxltR3AmEG45qmxnsCXqR1N8AYoAmoVBxlJRDhRnxXLTaHo5Wuou1HVErqxEzvLoVDfS3zg3tT9lH5UlQzvOXNXR+vly05IiDvbukXRPSGCDFZEr/Sgs6FIPqEXzKgNGpRdKKs8FQwm2bNxDqInOsTbpf2zwNK3MJb8lBnINh+nK5w9Dr6IoQi5k6V6uANKbZ6hXrMu3QKeDNHVt+n9W5tG0ulljqDvrx2zWtYE/8FEQXKTOnNzPcWLACUuBHJ0whAZbG4BsyjXb5czLYYl/D6Lw/A2ANs7CzGtbX/BWkJrKU88ftvuLD9HC2PXE08k0yh00k2DpLKzzZlmpVrWmuUzSovmBpOD60mUzPpwpgkk3lF7oO1L8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(4744005)(36756003)(956004)(6916009)(186003)(86362001)(26005)(16576012)(316002)(53546011)(508600001)(66946007)(38100700002)(6486002)(31686004)(4326008)(2616005)(66476007)(31696002)(66556008)(5660300002)(8676002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?I/NGQKPg4RHYlrSialoQgwye/hJ8CQRL0gKhJp2mI3E2RPg6aX5RdGkLfCsT?=
 =?us-ascii?Q?yvNw9ZhYebq+WNEBl1sAgaIq0IjL5VKOQMLTho9zpo/e6L2JH5NBXT1hMXy2?=
 =?us-ascii?Q?GIle7Ldu4QEVFjkQgu6LCOmVkg7YmarzT+EOCvtofiuTux9shFFfGzJg2aLg?=
 =?us-ascii?Q?SfKYpzlFhp2W/gtRqic2YXvOaiOqt+QZ30rIdAQmbLaDvlwzuTKA9j0VeE2t?=
 =?us-ascii?Q?YoWBhi379C8IPpJnu3thbXiYRZx8B64W83AlgVWcIppfo/b91n8adXrGtw/l?=
 =?us-ascii?Q?JW73B2prR4Zbx1HnIFe0bsTXHWmylH71i7Fc7QrS51p9BTpWOwJcmP8I+otv?=
 =?us-ascii?Q?RziswwVa6PzNGrKaQxYwZNzCCasyzo7hDfc8wkQ26Ot5nFbSmXS9DvjZQf9A?=
 =?us-ascii?Q?thEBvLpuAtOeZF8Y0pv09XL/iLz2FDKjihh3HKdyTKfp8xDuf8YHlZ5MWKiT?=
 =?us-ascii?Q?Z+6eahPwn7nxzqHv41SzpIG9d0Z1UIlXRnT13BHC5TMu3u32IBR2z+X2GRuC?=
 =?us-ascii?Q?7I3ZT30XqG2fNxssLj2kmrraaDdfyWNRSYwdRbNmr+fbBAv9A/6khmSblLPS?=
 =?us-ascii?Q?uoTjTfFdCnZ/DAhxQGiDChXQWrmYPNaZvmr3QTJKIYurwCNqq9UkahcfaepR?=
 =?us-ascii?Q?fFYr+7xiWcGTBjrWXBJ/ADLbr/LJMez4m1LUTY79RRw2ktf2VDC7P6VO7vYz?=
 =?us-ascii?Q?Cpq3IlW6BqNQxKnncrP8bupCCAd0OoIZjrlPqDNJMbTRvkQ7yt4riv74MA4b?=
 =?us-ascii?Q?gHxJG9euwukeUzN9DsWVgR2fMwYw/1mDgM6vOMwR4pu4n1XCCOmfEt7y78bM?=
 =?us-ascii?Q?hFd6DrQfl0o3Ap+Sm+Y/54mek8OnAHK4Nkw0N2Xhak0u3Zik1nwnqIwBkwKw?=
 =?us-ascii?Q?0g3/qLepOuUHaenY+Beel8YZEdgBOSmeOPuj2jNXCrVWeK5Ei6q/CQvCCzIA?=
 =?us-ascii?Q?LSI1Lj11Zaew1iDVao2G0yo6CgCKUcgpJ8l5ZDYvCK1voQNTjtYzDROfHqVi?=
 =?us-ascii?Q?0zljgykQjHWtEAMghbmp/9hqPo/JQRWbb0/lDBs9MHnYAvzNlGpEqnc8BiLu?=
 =?us-ascii?Q?hT5LV5EeCtNxta7qWbKuRnM8bEccb65cjdh9NLI25ITRVT3dfy21N+Pk105A?=
 =?us-ascii?Q?/4mE7hrvVLxHxGWysiKl8rBm2Olz3J+b/QM6jcLTS5TyRTpKUwbAjQNMAXCx?=
 =?us-ascii?Q?PKuHkhApFWoBIFyuvPSD0x/FK0JYPuqFtePN+mOBBrnxFCTUHuZ9MXNYsK3R?=
 =?us-ascii?Q?bXCZ/hQ1RZSOh9b1xH0bmz5aSDgVjOnz+4RAq1OzXVa0mAHkYLeRFYs7uRLe?=
 =?us-ascii?Q?KkKM/4CZg/qfMJDj2H0z75qh?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 649277ad-e545-410a-574f-08d9832288f9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:24:20.1173
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fFMxhI9roMktrnNSvSMDFlczM/GBtm1QdHS1laOUrKclfbFUIZhzL73Bl0WZFc9rNSJwdAFshMQLzmcwGZ+9/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

On 28.09.2021 20:18, Rahul Singh wrote:
> prelink.o: In function `pcie_aer_get_firmware_first=E2=80=99:
> drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'
>=20
> Compilation error is observed when ACPI and HAS_PCI is enabled for ARM
> architecture. APEI not supported on ARM yet move the code under
> CONFIG_X86 flag to gate the code for ARM.

Like iirc Julien has pointed out for another, similarly titled patch,
"solve compilation error" doesn't really seem appropriate here, as
with the tree at this point of the series no such error should be
observable. Unless I'm overlooking something.

I'd like to suggest that when you receive review comments, you apply
them throughout the series. If you follow the approach take in the
other patch ...

> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Acked-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:31:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:31:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198838.352536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUzj-0003Lf-0d; Wed, 29 Sep 2021 08:31:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198838.352536; Wed, 29 Sep 2021 08:31:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVUzi-0003LY-Tm; Wed, 29 Sep 2021 08:31:14 +0000
Received: by outflank-mailman (input) for mailman id 198838;
 Wed, 29 Sep 2021 08:31:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVUzh-0003LO-U0
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:31:13 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89fb1a1f-eff7-41f9-b762-58ee5cf3fb27;
 Wed, 29 Sep 2021 08:31:12 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-LMAMmD3eOhinRMmJGKqoLw-1;
 Wed, 29 Sep 2021 10:31:10 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4384.eurprd04.prod.outlook.com (2603:10a6:803:6f::26)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 08:31:08 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:31:08 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P193CA0043.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:8e::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 08:31:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89fb1a1f-eff7-41f9-b762-58ee5cf3fb27
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632904271;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xe0x5OaQDL0HmkLEDdt0SbYhwmDomn5FddKPmy1Fq4Q=;
	b=gro6E/yekiY9OVp2KsjMTFLquSsKLQpD+abuz0A/7OD7r98F2n1SV0wGlprWhQtnRVpjJV
	Qb/hvA7raghsFrpGT1s8vNc10Pjjb93ZKXjH7ve50LVff5A6ff+AAyuje0fbKQeRY+zlhb
	VzvWHX1O+llm3+OeZJLMNaFEvkmaIho=
X-MC-Unique: LMAMmD3eOhinRMmJGKqoLw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=lqFkNVzTUAkHUdUn+O8GIo8iSE3HCwJ5rHXnIq05SsJZwoSx8LXb7IOXXgtKaEfTsEIzGsFCZAAQfcekcG5zl7TMmScmOg2hHr7bRbulQwg7JQk5W3NPSo48QKOnafxg1YFxbkTOZcaBHVlUdBkwgHyx+W+hzTvRcqenC75reCa6ZCEz8wN7FK36fymfKDsFxv4QCb1WixoNMTJQYKTk0lJ/PEMe6n/bEbtCNhaX1VBI4GNf9YmImY0fmvX1qr0jreKJyHDDydGdqsLr0a+DAdG+WW/smstzYp37q6DGjCY5Xz+Blfb0t0tTk4UxYU0zcx2r4gvH1XCsDu4KehCVMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=V7EBOjmP8zODTq02nDA9aoNua2xdPpk/NgTvxxufaQ4=;
 b=kjGSg3NvCdpZypGOy9X0umTdOJ944kBcXUxOlLlJYP3JDXLRxHRUs1xU18Mh9otXtK68wkh7wXVnKSkhXkR6OKZ53xlcECQni4rl/oUjC2mzljpN6lGbGARACS8v31MxSOyHUUFL+nrQtl+u8sXaE4dgAE+sd1kyqscgKQuWWzi7KxFe+KiQExrr8eQRh8wyE65DsVyFcMz1oc1G/+lB2r/mPO5zDhiKLsAowQM+FgfkEC0vfJ/iPfxTEXRk6+iA7Cllt3R2JZvZaOxHWV8H2Ead9fti+7c7CuWvwrbO+bwuP+z0oGiIujKQ0Y+2ldUXeueQfPuYqNsMO0pGoTjtwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
To: Rahul Singh <rahul.singh@arm.com>
CC: bertrand.marquis@arm.com, Andre.Przywara@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1632847120.git.rahul.singh@arm.com>
 <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <27b97adb-c7f2-949e-140b-bbdb8f01457e@suse.com>
Date: Wed, 29 Sep 2021 10:31:05 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM6P193CA0043.EURP193.PROD.OUTLOOK.COM
 (2603:10a6:209:8e::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0d48ee57-0fd1-4491-a10d-08d983237c2b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4384:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43843B296F10E24841539785B3A99@VI1PR04MB4384.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1850;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mGhWO6fXldbtTYvF/ltw85jiqnvGUpR1tyi6LUcbTZHeo7G2aoAuDldFrffyI+6LUPM1M1RIHxCgllG18OY03lhZf4VIS9Xuu8DDwRa1iaJgQUk1lIjbRQlSVAMODQJiaD1+EpvQxhpGrSRMv06OXFFT0tv866rZhdbdp1VTJEMEL0kVe+VdGD8sG1uIJyvV0qeWGd0S+xmVOqvEfDkexzp+Rt7WoF+1D/EZ4yT32E7yc8eJPF6x7nOwAeGjxYvyCTogVdN9ayIgHBwQqdsFIhaLmD87nzsQp+Hwim9tSp9fqtGdfp6nkQCQoRrMi8ikKknIWkt8ZzBWH1eLRwDdOt0DLi764DSwmrFeNdKBJvqNbv2D2LSnW54kw96Q6MdJRo3UcHZXXhW4RKA10l3Wl0YbeTu2K4gJwsB55LdLTgZgHxnxV0T7jeuU8Ifp5B7qDim2hilTSbQJ8WWAg+MeZaNoSu5EcvPIfmdHypej3imzbv+Om3rgFbiLmfgBk9kzi7xC4Z/vPuQBO/PyN8TAHHFvTbCB14OYp6Zw29MEBNOYXsbPUhKblYRp2AINQFmMXD8pI7nuIgQ2wJRkBcwbhCwaR0bvTJdG+7MXCO/imYPKHvwxfbeVuDbVEUl3uXNNZnsWGpHaYGKnWBguR180LSWK54MqNQdVgs/0RT5spTC7Dl94yeAKnfTO3mdfYUXIIGLiVoCthnBxVLwJDyAO7q7t+BKy/SMYcN00UxgIUco=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(5660300002)(186003)(66946007)(26005)(66556008)(4326008)(66476007)(54906003)(316002)(16576012)(7416002)(8936002)(2906002)(38100700002)(53546011)(8676002)(86362001)(6916009)(83380400001)(31696002)(31686004)(36756003)(956004)(508600001)(6486002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?X3BCDceAFdVHpGtbjcEhErWjX9IDZaRbNrnBovblOg1gXAAV6t7zBB8njuhE?=
 =?us-ascii?Q?Z1thqZneEgK8XN4pMaNHd/aT0tuDJQo60OxSYr/j1SbwQ/k190kntFbGD2Ry?=
 =?us-ascii?Q?7A1hOZAxMhsfnU198qlKg935rRTlNytvIklUaa1Q/9y+Ma6OmmhLkz/GcScT?=
 =?us-ascii?Q?1hS/oLnxcnZ1wE/kWMa9Sg/3nAwmy7ObotB3BtXrMNXnFiS9wQp0QYa0x9j9?=
 =?us-ascii?Q?xGXp/lRfTpOgGKGloycCRNbphee18DffOwFGnq3nIGdZ/6X5yAy8drQy40EG?=
 =?us-ascii?Q?UwEshSRY0nDDeNn57ejtGj4pTlPrqswJkJ3segugxJnsE+/2raEb6kI2UJuJ?=
 =?us-ascii?Q?wxEscDPT5OhiMYk1ZuBW7RBIp2u+J8r7sSc1GfJVcys4DWDL57eyj2JpGSOq?=
 =?us-ascii?Q?NEjIWMFBaaJf0DYWQ+mb2IfJVnMB//hxohdxvlhvf4Jfs01+sAyplxARyrUn?=
 =?us-ascii?Q?g1VLKH2yYlM/zbbtIDgzwRRTfLv4iGVply9rWX2/cQrXLtsQHBEqmdG3JlbC?=
 =?us-ascii?Q?56fwsXoGSm+eNpiPM9GNH+LzRQctQvJJwXEBtD5H53JsOZm7TvTs9o+w1SfA?=
 =?us-ascii?Q?yKUeiuQOeJQYEsBuI5agsb0x2ubieNcvNUZ8Ryt3VMMUq19LerO+rJTfM+SB?=
 =?us-ascii?Q?+g/fH3NyJK+SJ8g3jVO5aXcZelNBcJuT+S0nPbE0+E7DaoaqXX0ihfYbXBm3?=
 =?us-ascii?Q?AOHPJLO4OyNmGxQJ02nSRKVa6vDA5IPX//r23+OjiaBVpqnu9jTCHbzJm25v?=
 =?us-ascii?Q?bqJ7khjpUTIJngDoHGxhZrVCmXxzqiOEYzAck/EfIs4eHuI4+QBG4qH1Z+Sy?=
 =?us-ascii?Q?uKrldpNDoZ298f7lrKYJz7zLNd+s9JyeNsI5txh3BbJbw30cMaxPkSqrDkHe?=
 =?us-ascii?Q?fjxO8Xdcnk+Sws2WxJxAEPjUh4fKbsXm0bjLkatUJ112NviChJpHM7JI56YK?=
 =?us-ascii?Q?3axZKRM7Kd+/QjuPkosJXMNZ59s5zSmgHKHismAxGYyvKzRepzYK1Wn0H/aP?=
 =?us-ascii?Q?UFkLF3ZxfSREEx4+Llwoik3ziPTY7yx7vA05WrLKRCF2GCC4dicfPqCEy7J4?=
 =?us-ascii?Q?kUK30fhR3He0YIUQIoxaHhOqctA0HzAVsuAZm2TvvFhP5+1EqPx7vq1S7vl8?=
 =?us-ascii?Q?nBebIewsYQRmZKmOPxv0Iv2udTjQb9Dx7UQ9Htb4eLQqaOn8Ta3HW8NP1hLj?=
 =?us-ascii?Q?yEn3Z8vqswXyuCx/SzgCAU0+Ols1jo7xg8bfMX7nfikYgtTEDy/oAV+HcKYg?=
 =?us-ascii?Q?Uswcps9i+HTi799pAmIUw3wMI5ej44p52A8xgi1rhfbyl7D6crOky4YDwrGi?=
 =?us-ascii?Q?1vYlzL5g/WOHXpHdhUlV83gI?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0d48ee57-0fd1-4491-a10d-08d983237c2b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:31:08.1033
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KepSlAbGRRKan5LmZd1BQJzlRlRRDK7NGAgMrgxwvLBkeo7e8VzRajaYMLaUGD4jFkCxhn2GJ1QaRxYTpZ15UA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4384

On 28.09.2021 20:18, Rahul Singh wrote:
> XEN during boot will read the PCI device tree node =E2=80=9Creg=E2=80=9D =
property
> and will map the PCI config space to the XEN memory.
>=20
> As of now only "pci-host-ecam-generic" compatible board is supported.
>=20
> "linux,pci-domain" device tree property assigns a fixed PCI domain
> number to a host bridge, otherwise an unstable (across boots) unique
> number will be assigned by Linux. XEN access the PCI devices based on
> Segment:Bus:Device:Function. A Segment number in the XEN is same as a
> domain number in Linux. Segment number and domain number has to be in
> sync to access the correct PCI devices.
>=20
> XEN will read the =E2=80=9Clinux,pci-domain=E2=80=9D property from the de=
vice tree node
> and configure the host bridge segment number accordingly. If this
> property is not available XEN will allocate the unique segment number
> to the host bridge.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> ---
> Change in v3:
> - Modify commit msg based on received comments.
> - Remove added struct match_table{} struct in struct device{}
> - Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
> - Remove bus_start,bus_end and void *sysdata from struct pci_host_bridge{=
}
> - Move "#include <asm/pci.h>" in "xen/pci.h" after pci_sbdf_t sbdf declar=
ation

This part, if not split into a separate patch in the first place, wants
mentioning in at least half a sentence of the description. Then ...

> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -15,7 +15,6 @@
>  #include <xen/pfn.h>
>  #include <asm/device.h>
>  #include <asm/numa.h>
> -#include <asm/pci.h>
> =20
>  /*
>   * The PCI interface treats multi-function devices as independent
> @@ -62,6 +61,8 @@ typedef union {
>      };
>  } pci_sbdf_t;
> =20
> +#include <asm/pci.h>
> +
>  struct pci_dev_info {
>      /*
>       * VF's 'is_extfn' field is used to indicate whether its PF is an ex=
tended
>=20

.. this part
Acked-by: Jan Beulich <jbeulich@suse.com>
(also if you decide to move this to a separate patch)

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:36:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:36:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198844.352548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVV5A-000403-Lm; Wed, 29 Sep 2021 08:36:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198844.352548; Wed, 29 Sep 2021 08:36:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVV5A-0003zw-Ie; Wed, 29 Sep 2021 08:36:52 +0000
Received: by outflank-mailman (input) for mailman id 198844;
 Wed, 29 Sep 2021 08:36:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVV59-0003zq-5x
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:36:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7fee0070-9d55-4b7d-9212-0d67028e1410;
 Wed, 29 Sep 2021 08:36:50 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2058.outbound.protection.outlook.com [104.47.13.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-uAHHky32PAK0QKTLJBJ9JQ-1; Wed, 29 Sep 2021 10:36:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2958.eurprd04.prod.outlook.com (2603:10a6:802:a::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 08:36:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 08:36:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR0701CA0018.eurprd07.prod.outlook.com (2603:10a6:203:51::28) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.8 via Frontend
 Transport; Wed, 29 Sep 2021 08:36:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fee0070-9d55-4b7d-9212-0d67028e1410
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632904609;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oefe4BjwUZx0knaWRCIB7JYsRVkYHVMnovphYg5Xm0A=;
	b=lSb5O3kvBqi1jGfHaovQq+wZbcFMfzAoPXp9/G/ta2YAoJGx2u80WpC88NtGCab5SQrVbd
	FsuA8sFzUtPetQfkFx0KLCIKsUdZJc6B46FzpFbRfpfn0aDeZuNam8hXeDtpBxO4rQfsNX
	4UaaTLmNGWMTTF22pJQsaqqRdShuf38=
X-MC-Unique: uAHHky32PAK0QKTLJBJ9JQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=l5jiwILP65IbWKx8pwshEGoNLnkfXTbi0UdJyxD/IdGP2qiD4wf3bNQCb8+vTvlvHYJDdD7q/Jir7Z9z7SiXSTgUk780IX1rAZXIiZfqD+zw0t5cr3v4Ima0whaWdQQm+fA7L5ue38XifoSp8ESthtlnEoKvc0VnLX96/GtOtNgArkGfoGGl7nzVQsQMhQr2UXTwYbyS1ODnQHypPPHeSAYHk2G6uBM9/xzZRlHljG+Jg+vSgYCiqiwBu/LGH4+BNOYBxWDFywll2D8QLD0QjhG/qcQGpSw3J8oHDFaY0jfhKgqnm3bL+Spt/bldq3Q4gKtbDAA7/HJob1o+DD4V1g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oefe4BjwUZx0knaWRCIB7JYsRVkYHVMnovphYg5Xm0A=;
 b=KVlZFZ+vYr+z+sq2BXL8RY87n3QCXWkF9Odt1+3ESvjjkuC06v7vf2wY7EtkLaqporuL5MsiybjS7NcfxJXApf+TXhbFWe1zNKk9YrmhNhaC7UbzYU8GmTpT0Izy3ixXFi+xYamD2Et0q/XE70kTJ7et3E62zs3rHABMcZgJYyp441osDMZj7HUswnP2kL7EN7Ob+zoOSYzIagkyoYGfdx0f3L83+qLP4ddeU1v25FFsy9wJY50ALwWPouNFB9XY+O0tDdO789wFYGUR1VVIRPjcVMFLzrHxd9sOpYtKOZjW0HE5Eu53CclE6MQhFo8VayesmcXNS65eP4370+HXKg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Michal Orzel <michal.orzel@arm.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
 <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
 <f59b48fd-564e-64f8-1422-fb842957f213@suse.com>
 <7ae0fad7-499b-1bda-5c9e-9313265abf8c@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <30b8c6b8-5dc4-6972-eaf9-76024f7e97bb@suse.com>
Date: Wed, 29 Sep 2021 10:36:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <7ae0fad7-499b-1bda-5c9e-9313265abf8c@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR0701CA0018.eurprd07.prod.outlook.com
 (2603:10a6:203:51::28) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6fe340dc-626d-4a06-de93-08d9832444f7
X-MS-TrafficTypeDiagnostic: VI1PR04MB2958:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB29583B5C45F805552AA0BCE1B3A99@VI1PR04MB2958.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5236;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4PxWfZJ3bXArex3/TFO7w+sY0J6PGldPH+KzuB2s+bUTdHwRYnA37p+PEUda5dPUwciyWcDA6Vdjr4fzYfO4DG7cZrp06oionAziGXCJmMDU7JC/8s8dxTF/eELeXZWV5eFg79ISX+uksg2WtjcIsAIk4kL+q/uiuGfFMiQnQ7JX8X780cNhbQC5b4lpg+NKA0sNVgG2qlnpkHrpF4eGUWyuLQm5lf/q10f2/EKLd8yImEYYDEhqNgnfwtfKsZx4XDBXryr3ThF9Obhtpomr7msxtu8C9F66LgAS9EvGm4AMZEgr9ujUHqMNmReV48j/tRFKhmngQd3KLykG/2TIvmxevZXCpQqB1Xf09X155nUECW6/bkNlkWIlwJEzFOm7/Dx5994XVwEcpVOAkfjrozg2wMCzz8onceKPCgHAAHGvig9o/schq+YvYwL8SYYhF0XUX9EQoTp6AkHed8ddrATkxM4ccqlnoLCCXgEU+MAI9p6aAQeadNKirDyDZml6FNJFtDCCfLE0qBw6gBv3rcJ8x5bRfNM8nstaY1xxrYKhy5jcYri/U45HFutz+gyZk5Y8FMUFyqFQ33FL90Lq5yhdTDZODryZH5s2sn5LZm+yzgi+Ont6ffdhb35F0WxlaUqatJQsBQKu1j8Ctq5qL6eDA52OM+0zVZPWUuXEk25y1DPP/GWL72lW+PORHzqJjl9HBR7f4g5P3zXwYj6kwMRIIE84YkYDHaaF2wVrUu8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(26005)(508600001)(54906003)(316002)(186003)(2906002)(66556008)(16576012)(53546011)(66946007)(66476007)(31686004)(8676002)(2616005)(956004)(5660300002)(7416002)(86362001)(31696002)(38100700002)(6486002)(4326008)(8936002)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MzNTbFlKUUhmY01teVRUNmh0ZkhLN3JieGl3bWdPSDk4S1l6Z2p3MVBqR3RU?=
 =?utf-8?B?aU53ZndpQUlPM1ppSjl1cWNLT1JWeUFKZUhzZVYwVC9aSTdZNVQzbU4xK0xQ?=
 =?utf-8?B?UkRvdTB1bnl5TzErcnd4MUsxL0M1MGQ4NU56dVJISll4NHU4Sm1KWVFTNi90?=
 =?utf-8?B?NTF1b010Y1pQQ2JwOW85aTA1NGpaSkZaWWNaMkF1OWJFM3kycUo2eVZKWGQr?=
 =?utf-8?B?Z0NlYTBVTUhjdzlFY0tsZGFGY3Y0ZW8vYmU4QWprTFo0VHBiVkl5WXFnY0Fw?=
 =?utf-8?B?TXd4Wlp2VnpzcWRHc1ZEZnpFejJRSlZFakx6eEhCaXpvd1pKYnV5QjVtZnRo?=
 =?utf-8?B?aURWNHpZUVJhU3pTa1Yva01qaUMrNmk2cTByajlERjVvRU9vNGYvbnQ1M2NH?=
 =?utf-8?B?R1R2elJya3FCajRTMVlsM3ZSa1BpYVg1dXQxRnR4bmJJRGNCVm1VbWJkZUNz?=
 =?utf-8?B?N3E0Wk5LNnR3Mk1LR2tQcHUxVkM1Z25QcmwzYjJqb0hCZnhnQ1VTTGhhazQ3?=
 =?utf-8?B?amVma0NlUEExeWgvTFI5cUxkRVdRczkyQzhtd01ZL3ZtcmwzNTNMdTFNaWpF?=
 =?utf-8?B?YWdvak5zTHlLR1FVSlU4MmJsMnpScW5RNi9Sb0FlZ1ZrVmNXMllBcXB2aDVK?=
 =?utf-8?B?S1cxQ0FqcEs0SE1rbW92dHphVHF5S3F3UGxrOUF6T1VQWndzWVo5cmtBSlhK?=
 =?utf-8?B?a2Z4cDFtT284YkdKZWlTNnlYOXZWZldGWWZMd0hPcEdtRXhRZ3Fhc04xcnJx?=
 =?utf-8?B?Tzh4dWhWbmxqSVRyejJRd29OcE1lemRXakwzcHNqcmVkdEQxTEhoQVRjRTZT?=
 =?utf-8?B?TS95elN4NzAwUHdkbVlPV293bnc4RGRZS1R0dU1SaGNobGdnSlBPMzBSMFp3?=
 =?utf-8?B?ajNiQm5Cby9sT0RwUlZuaU1XS1pYS2VPMVNvVXpDdWVteVd3eHZVN1JFNFZh?=
 =?utf-8?B?dk5mK2d4ZGN0bUhKVXZEaTE3VW8xWlAxVGh3RzZveGhsNWgwRWo0Ni9aVFdR?=
 =?utf-8?B?UTVIOWdRZWN1dVBNZWFlKzJnOWN1bDF2d2ZkWS80RlNVeWxXQ0xXSmVoZEVY?=
 =?utf-8?B?VXB4YjBucmFDN1h0ay81a05RdVEwc2h0ZEg4T1J3R1NDeU9jOEpnZlE4ZGtv?=
 =?utf-8?B?QWkyMSs0VFMwS3o3T1FXc2NtbkFXa25qMXRzWG4yZ21rSmgwZlEwb2EzOVh4?=
 =?utf-8?B?bFNSSzdPVlZ1dlJ0T1JwNmlWNzY3Y1EzSDNnd1VOaGVGRHdGVzFxSXhXZVZY?=
 =?utf-8?B?SXpLZVdEWVZpSEpMbUdQejFYaSt0amdDc1dDOUpFcEpKU2tkb1JpcWxyK3FI?=
 =?utf-8?B?dHJhMUoyYVpqZWxZVHhSUVUzeW9MaFBTeGxwTVJhUlRIaFpKK3VlaXpuUDNr?=
 =?utf-8?B?ZDhtUUtOY3RpaEZpRWlQTmRySklwSWdlM2xleFdFRU0yMDFESm01V3BvVzR0?=
 =?utf-8?B?cVk1NitpMFFQWVVWRkdPQmo3NUJIYk9JSEJmZXRVREp1SHpJTWdqeThKQ2tP?=
 =?utf-8?B?OGVwSmV0cEl5N0VVdUdEZys4MHMyUmFVbCtIZGZvTFVxZTV3c3Ezc3Z5bXBs?=
 =?utf-8?B?cklYSXRxOG4zb0djQTFUVzk0cFJJN1ZnU3M2bTNXMnhjcWhZSzU5VzR5a3NM?=
 =?utf-8?B?K0xWYlY1bVJzMVNvc2VyK3VFd2RMREVHRkVxajFieDlYN1p2WlFkUW5NcHMv?=
 =?utf-8?B?dER3WmlkM1I0YTFuSkpsd2IxSXN5cFdsOUtJTnNNbDZ5cTlYRUVZdEhuODNU?=
 =?utf-8?Q?ZyZK943AuJ33T07FqpdqwW34PLe5hASecqwjGcu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6fe340dc-626d-4a06-de93-08d9832444f7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:36:45.0161
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: c5fp8ysXILXGoPHEVfdCgOpZYzrG5ya/dqXU3Qf1gYYslLzseyC3EHyytNivVh9Lbs9/1XVkphgdWiHWGrwHZQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2958

On 29.09.2021 10:24, Oleksandr Andrushchenko wrote:
> 
> On 29.09.21 11:16, Jan Beulich wrote:
>> On 29.09.2021 10:13, Michal Orzel wrote:
>>> On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
>>>> @@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
>>>>               if ( !bar->mem )
>>>>                   continue;
>>>>   
>>>> +            data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ?
>>>> +                _gfn(PFN_DOWN(bar->addr)) :
>>>> +                _gfn(PFN_DOWN(bar->guest_addr));
>>> I believe this would look better with the following alignment:
>>> data.start_gfn = is_hardware_domain(v->vpci.pdev->domain)
>>>                   ? _gfn(PFN_DOWN(bar->addr))
>>>                   : _gfn(PFN_DOWN(bar->guest_addr));
>> FWIW I agree, yet personally I think the conditional operator here
>> even wants to move inside the _gfn(PFN_DOWN()).
> 
> I can do it as well:
> 
> data.start_gfn = _gfn(PFN_DOWN(is_hardware_domain(v->vpci.pdev->domain) ? bar->addr : bar->guest_addr))
> But, help me please breaking it into multiline with 80 chars respected

Besides the option of latching v->vpci.pdev->domain or
is_hardware_domain(v->vpci.pdev->domain) into a helper variable,

            data.start_gfn =
                 _gfn(PFN_DOWN(is_hardware_domain(v->vpci.pdev->domain)
                               ? bar->addr : bar->guest_addr));

or

            data.start_gfn =
                 _gfn(PFN_DOWN(is_hardware_domain(v->vpci.pdev->domain)
                               ? bar->addr
                               : bar->guest_addr));

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:38:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:38:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198850.352558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVV6R-0004jw-0A; Wed, 29 Sep 2021 08:38:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198850.352558; Wed, 29 Sep 2021 08:38:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVV6Q-0004jp-TU; Wed, 29 Sep 2021 08:38:10 +0000
Received: by outflank-mailman (input) for mailman id 198850;
 Wed, 29 Sep 2021 08:38:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NQqk=OT=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVV6P-0004jP-6m
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:38:09 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id bfdec212-d0ea-4532-946d-be04bda7a37a;
 Wed, 29 Sep 2021 08:38:08 +0000 (UTC)
Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com
 [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-161-L-gDeITiP5KIMHHqZx5kIg-1; Wed, 29 Sep 2021 04:38:04 -0400
Received: by mail-wm1-f70.google.com with SMTP id
 r66-20020a1c4445000000b0030cf0c97157so882621wma.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 01:38:04 -0700 (PDT)
Received: from [192.168.3.132] (p4ff23c3b.dip0.t-ipconnect.de. [79.242.60.59])
 by smtp.gmail.com with ESMTPSA id
 f17sm1497220wrm.83.2021.09.29.01.38.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 Sep 2021 01:38:02 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfdec212-d0ea-4532-946d-be04bda7a37a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632904688;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/rI4ipp+6p4YsURf5SVEvaXl3JhZmp49Hm7auACiD3c=;
	b=DMfSawGAybfbS151O4N/LvnBQadruDMRXtuIB36ZdHqhhac5RFCmu5iv5l3bSIMCgAcBAM
	yhX2qak3Q1iF10Y96ktyzLl8tZMzLWBHT2iGIMZPZnsin0CxyhXFXdqC1NDhDKY10/vXud
	st+nBFwaXHnmt2iob5qQ2IvrAdlD+Gc=
X-MC-Unique: L-gDeITiP5KIMHHqZx5kIg-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:organization
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=/rI4ipp+6p4YsURf5SVEvaXl3JhZmp49Hm7auACiD3c=;
        b=GN10XZc87nSHORmOkyCZgOnI5b6MD0CC1V8ZQ/xI8+57UPQvLrWJTPBiFvqTwXhBzx
         2D72zNGpiYFZ3voljhZ0Fa1UvcUCmuMF0DTjyWyh7yKdj8vUnl8KRULXCWa5MNs6UqEA
         luV0RZ9ONLlUsj5TTbSwPdbY4LD1Yy/o35USVy9cvRysA9uIt7zSaj/aqyCI1FWOTMPx
         lOZedHdbi2/jZnkYfrrT2lJG7Ov03kHz3l2Zp5XjiMMARz2LsxbAo9jHC0+p6dsKPlKV
         pgbmMFyDcie2KEOjRw+pqLy98hw68qUeNv2OfKsYmYlLAMaqE425fB51ESw2Xu7BfzbQ
         XRFg==
X-Gm-Message-State: AOAM530W2rSu2+Lpd0jBwFusEHAprzxI+yrZEPI6l/XJI0lGPIo/S94k
	1jqwwRuRhwKpkYdpL6rRZur0nfZ4tGNUkT9zCSoY+/dJjcMt3O4oZORJQ6LdyXk6Ux5CK8WtI93
	luFQIodxkY9ashDC+NlbCCLuxelo=
X-Received: by 2002:a05:600c:3652:: with SMTP id y18mr8982006wmq.66.1632904683468;
        Wed, 29 Sep 2021 01:38:03 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJw1UYD13jo4zi2cV8ML8JLWg6MbDn1wfsjZAud8HZY/udqsA+Bzs8BSxqVjAL69LYYlXkKLHw==
X-Received: by 2002:a05:600c:3652:: with SMTP id y18mr8981980wmq.66.1632904683305;
        Wed, 29 Sep 2021 01:38:03 -0700 (PDT)
Subject: Re: [PATCH v1 8/8] virtio-mem: kdump mode to sanitize /proc/vmcore
 access
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>, Juergen Gross
 <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 "Michael S. Tsirkin" <mst@redhat.com>, Jason Wang <jasowang@redhat.com>,
 Dave Young <dyoung@redhat.com>, Baoquan He <bhe@redhat.com>,
 Vivek Goyal <vgoyal@redhat.com>, Michal Hocko <mhocko@suse.com>,
 Oscar Salvador <osalvador@suse.de>, Mike Rapoport <rppt@kernel.org>,
 "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
 xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org,
 kexec@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-9-david@redhat.com>
From: David Hildenbrand <david@redhat.com>
Organization: Red Hat
Message-ID: <e01cdc7f-cda8-3268-c971-1255a71fb8ac@redhat.com>
Date: Wed, 29 Sep 2021 10:38:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210928182258.12451-9-david@redhat.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit

[...]

> +
> +static bool virtio_mem_vmcore_pfn_is_ram(struct vmcore_cb *cb,
> +					 unsigned long pfn)
> +{
> +	struct virtio_mem *vm = container_of(cb, struct virtio_mem,
> +					     vmcore_cb);
> +	uint64_t addr = PFN_PHYS(pfn);
> +	bool is_ram;
> +	int rc;
> +
> +	if (!virtio_mem_contains_range(vm, addr, addr + PAGE_SIZE))

Some more testing revealed that this has to be

if (!virtio_mem_contains_range(vm, addr, PAGE_SIZE))


-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:45:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:45:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198856.352569 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVDr-0006B9-PF; Wed, 29 Sep 2021 08:45:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198856.352569; Wed, 29 Sep 2021 08:45:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVDr-0006B2-M2; Wed, 29 Sep 2021 08:45:51 +0000
Received: by outflank-mailman (input) for mailman id 198856;
 Wed, 29 Sep 2021 08:45:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NQqk=OT=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVVDp-0006Av-NP
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:45:49 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 00b06123-e617-4a03-876b-6382a1d85a59;
 Wed, 29 Sep 2021 08:45:49 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-319-ea6IgI18M3OMzyRxCb88eQ-1; Wed, 29 Sep 2021 04:45:47 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 17-20020a05600c029100b00305eac9f29aso579110wmk.1
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 01:45:47 -0700 (PDT)
Received: from [192.168.3.132] (p4ff23c3b.dip0.t-ipconnect.de. [79.242.60.59])
 by smtp.gmail.com with ESMTPSA id
 t126sm893773wma.4.2021.09.29.01.45.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 Sep 2021 01:45:45 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00b06123-e617-4a03-876b-6382a1d85a59
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632905148;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Gvu4L83YOBGLGvWqLGUuGEaO/DEUZyEQ1rFR5bn6b8M=;
	b=JX8sXbowIGFwf7cZnZnH3wolVvbBg/04UwcHDM4LArKbK7y5RdnLF7GKBfNlCIqxj7WeYb
	EJQIzjv+Oko8xuDDKM8M1tO52KUIC4K0DaVpqAhlyP0JmqIwixnUFtYcSt5jJSOtPXDc77
	LK4n2WibCqgob34LQwQqKU6A/2Ads4g=
X-MC-Unique: ea6IgI18M3OMzyRxCb88eQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:organization
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=Gvu4L83YOBGLGvWqLGUuGEaO/DEUZyEQ1rFR5bn6b8M=;
        b=gFNDIsRUnCWhwCrxyg8q/sxxkd/Wg0yKyGaTHmROkaWBba8R5JC3Zis3p2ZdULvBG2
         PQP9IFYZxMWDkl/6t3+LLRaTzvk7+XhBnANIyMChdjFlKyNwCzW5pD43aDjTGHGpOhRz
         dnAp2muBWcSehGsWtzAPq5anf41RYNjlNKZZ7bdwAbKiNni+iIczlqGav1lYs+NBfgTT
         DHJt9ic/AMpVHolLu4Hqq0ZHcJ9HG0PcAqFCLPDsNjC5o3ayjnijqKuc2v3bLvjn+uSt
         MxiiySvutsna2DO0BVk12ASAZ9k3UvJJKlxlB6WCQr+qaCVR/X11/+uAtzo65BdePL1M
         q+BQ==
X-Gm-Message-State: AOAM533hZjR5jUlDjzh3MMBAac/viXeBP1Njz9M5VGgDOLuJBycEhWDj
	0molHrw83NA+/DEjqUSGKd2dNoFNn9I0UrbkQr2oxR38NVg3ibPcTHlnRnVU/H3oXjViYBKSU/0
	PlkHmaRyby2Z9tEoAHS9iy2ypefY=
X-Received: by 2002:a1c:7c0f:: with SMTP id x15mr8734511wmc.149.1632905146275;
        Wed, 29 Sep 2021 01:45:46 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyaikWIkzzrJGqHValULa/ZmnNfg4rl1LyjKqzMWVt+gz9mbskxaXhni5v3m042+vSd753h3g==
X-Received: by 2002:a1c:7c0f:: with SMTP id x15mr8734485wmc.149.1632905146123;
        Wed, 29 Sep 2021 01:45:46 -0700 (PDT)
Subject: Re: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Juergen Gross <jgross@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, "Michael S. Tsirkin" <mst@redhat.com>,
 Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
 Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
 Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
 Mike Rapoport <rppt@kernel.org>,
 "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
 xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org,
 kexec@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-3-david@redhat.com>
 <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
From: David Hildenbrand <david@redhat.com>
Organization: Red Hat
Message-ID: <bfe72f46-9a0d-1a87-64bd-4b03999edd1e@redhat.com>
Date: Wed, 29 Sep 2021 10:45:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

> 
> How about
> 
>      return a.mem_type != HVMMEM_mmio_dm;
> 

Ha, how could I have missed that :)

> 
> Result should be promoted to int and this has added benefit of not requiring changes in patch 4.
> 

Can we go one step further and do


@@ -20,24 +20,11 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
         struct xen_hvm_get_mem_type a = {
                 .domid = DOMID_SELF,
                 .pfn = pfn,
+               .mem_type = HVMMEM_ram_rw,
         };
-       int ram;
  
-       if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
-               return -ENXIO;
-
-       switch (a.mem_type) {
-       case HVMMEM_mmio_dm:
-               ram = 0;
-               break;
-       case HVMMEM_ram_rw:
-       case HVMMEM_ram_ro:
-       default:
-               ram = 1;
-               break;
-       }
-
-       return ram;
+       HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a);
+       return a.mem_type != HVMMEM_mmio_dm;
  }
  #endif


Assuming that if HYPERVISOR_hvm_op() fails that
.mem_type is not set to HVMMEM_mmio_dm.

-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:53:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:53:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198864.352580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVLG-0007n8-Nd; Wed, 29 Sep 2021 08:53:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198864.352580; Wed, 29 Sep 2021 08:53:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVLG-0007n1-Kf; Wed, 29 Sep 2021 08:53:30 +0000
Received: by outflank-mailman (input) for mailman id 198864;
 Wed, 29 Sep 2021 08:53:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CEC3=OT=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mVVLF-0007mv-E7
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:53:29 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a47ce3c0-13fb-495d-b33e-59341eaab64d;
 Wed, 29 Sep 2021 08:53:27 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL4-0001ii-CE; Wed, 29 Sep 2021 10:53:18 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL1-0004fj-Sj; Wed, 29 Sep 2021 10:53:15 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL1-0000QL-Rd; Wed, 29 Sep 2021 10:53:15 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a47ce3c0-13fb-495d-b33e-59341eaab64d
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
To: Bjorn Helgaas <helgaas@kernel.org>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>
Cc: Christoph Hellwig <hch@lst.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Stefano Stabellini <sstabellini@kernel.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 03/11] xen/pci: Drop some checks that are always true
Date: Wed, 29 Sep 2021 10:52:58 +0200
Message-Id: <20210929085306.2203850-4-u.kleine-koenig@pengutronix.de>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Patch-Hashes: v=1; h=sha256; i=7QOFRnoxyPrPqaGt2Yf8oJvx3xwgCvlMdm130sPrxr8=; m=Ixh7t2kv5MnS1JaJxFtVrqNhjBtz3xQyoCGo2anqbI0=; p=dOfuuSLv1l/M9DCuxyIBdCuX9kCH+yUzno229EZfUDs=; g=9e481f95a51887868b05f8a86c3e5a9f8aff47c6
X-Patch-Sig: m=pgp; i=u.kleine-koenig@pengutronix.de; s=0x0D2511F322BFAB1C1580266BE2DCDD9132669BD6; b=iQEzBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFUKUwACgkQwfwUeK3K7AlyQAf9Gjp TCADTkJE36ON9dioloBuG0iVyAvFHq07aALxCH74D0NK9kgDK47u4JVyMBhswUfFEriH9thGirCW/ Sx6wuhA9gbPQtOzLKitgpb6Kq2+LIr7e28Ht5lwu9rQKdL7iybpKHPL0Ydr65XKY87CyOIIEfyUXm Byf0jInzlMfR1aCRec4WfMUdCEAsfUldp4k6pqUVfe/LemxULqopkQSf1oQawlTaPEKn/UEo/P/Qd 51uPnQ7NnWP80mPyCMNFDBq5HS/AyHyimwYuRfZWdSKDahmfequCzVKFmwqzGgzMVNUdqUsU7AooA 9gZhAzE3K9TTpQQYi+AgIZpi5f2MsiQ==
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

pcifront_common_process() has a check at the start that exits early if
pcidev or pdidev->driver are NULL. So simplify the following code by not
checking these two again.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
---
 drivers/pci/xen-pcifront.c | 57 +++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 32 deletions(-)

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index 2156c632524d..f2d7f70a7a10 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -594,7 +594,6 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	int devfn = pdev->sh_info->aer_op.devfn;
 	int domain = pdev->sh_info->aer_op.domain;
 	struct pci_dev *pcidev;
-	int flag = 0;
 
 	dev_dbg(&pdev->xdev->dev,
 		"pcifront AER process: cmd %x (bus:%x, devfn%x)",
@@ -609,40 +608,34 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	}
 	pdrv = pcidev->driver;
 
-	if (pdrv) {
-		if (pdrv->err_handler && pdrv->err_handler->error_detected) {
-			pci_dbg(pcidev, "trying to call AER service\n");
-			if (pcidev) {
-				flag = 1;
-				switch (cmd) {
-				case XEN_PCI_OP_aer_detected:
-					result = pdrv->err_handler->
-						 error_detected(pcidev, state);
-					break;
-				case XEN_PCI_OP_aer_mmio:
-					result = pdrv->err_handler->
-						 mmio_enabled(pcidev);
-					break;
-				case XEN_PCI_OP_aer_slotreset:
-					result = pdrv->err_handler->
-						 slot_reset(pcidev);
-					break;
-				case XEN_PCI_OP_aer_resume:
-					pdrv->err_handler->resume(pcidev);
-					break;
-				default:
-					dev_err(&pdev->xdev->dev,
-						"bad request in aer recovery "
-						"operation!\n");
-
-				}
-			}
+	if (pdrv->err_handler && pdrv->err_handler->error_detected) {
+		pci_dbg(pcidev, "trying to call AER service\n");
+		switch (cmd) {
+		case XEN_PCI_OP_aer_detected:
+			result = pdrv->err_handler->
+				 error_detected(pcidev, state);
+			break;
+		case XEN_PCI_OP_aer_mmio:
+			result = pdrv->err_handler->
+				 mmio_enabled(pcidev);
+			break;
+		case XEN_PCI_OP_aer_slotreset:
+			result = pdrv->err_handler->
+				 slot_reset(pcidev);
+			break;
+		case XEN_PCI_OP_aer_resume:
+			pdrv->err_handler->resume(pcidev);
+			break;
+		default:
+			dev_err(&pdev->xdev->dev,
+				"bad request in aer recovery "
+				"operation!\n");
 		}
+
+		return result;
 	}
-	if (!flag)
-		result = PCI_ERS_RESULT_NONE;
 
-	return result;
+	return PCI_ERS_RESULT_NONE;
 }
 
 
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:54:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:54:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198870.352592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVMd-0008O2-2T; Wed, 29 Sep 2021 08:54:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198870.352592; Wed, 29 Sep 2021 08:54:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVMc-0008Nv-VU; Wed, 29 Sep 2021 08:54:54 +0000
Received: by outflank-mailman (input) for mailman id 198870;
 Wed, 29 Sep 2021 08:54:53 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CEC3=OT=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mVVMb-0008Nh-7Y
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:54:53 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 49f7b25e-6941-4109-a2e2-d3dab0f3d936;
 Wed, 29 Sep 2021 08:54:50 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL6-0001jM-6d; Wed, 29 Sep 2021 10:53:20 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL4-0004gO-VW; Wed, 29 Sep 2021 10:53:18 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL3-0000RO-SQ; Wed, 29 Sep 2021 10:53:17 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49f7b25e-6941-4109-a2e2-d3dab0f3d936
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Russell Currey <ruscur@russell.cc>,
	"Oliver O'Halloran" <oohall@gmail.com>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	x86@kernel.org,
	"H. Peter Anvin" <hpa@zytor.com>,
	Frederic Barrat <fbarrat@linux.ibm.com>,
	Andrew Donnellan <ajd@linux.ibm.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Mathias Nyman <mathias.nyman@intel.com>,
	linuxppc-dev@lists.ozlabs.org,
	linux-perf-users@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	linux-usb@vger.kernel.org
Subject: [PATCH v5 10/11] PCI: Replace pci_dev::driver usage by pci_dev::dev.driver
Date: Wed, 29 Sep 2021 10:53:05 +0200
Message-Id: <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
X-Patch-Hashes: v=1; h=sha256; i=/pZgVMb346xYWFyKXoYzQiwqLbFb4Qw5wqFIqjkKa+4=; m=FyMqGYUG+9Ut7bWwCBeeUgHOxoK/KbMAjt+cyx+MULA=; p=7uJBwDup3eOXFa/jaxic6at398GqkPM+jAglE1o0ido=; g=d846534d35b8420e8bfb08112981cf20b1b1fab2
X-Patch-Sig: m=pgp; i=u.kleine-koenig@pengutronix.de; s=0x0D2511F322BFAB1C1580266BE2DCDD9132669BD6; b=iQEzBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFUKWMACgkQwfwUeK3K7AmtpggAmyR 6rO1OqNj4Fquhf8pUeQsYWPCNyaSBg+ipgJHefnapmLAKQVspu/dH/PC55+8isG9gHDw/cc2aZGT8 XD7Ez6VRDOV8ryX2WGNNU4Hy5D5TzpGmHWt6tWKJzvQ0MGyFHZuHYguOR+PGGyw8BGGQNYCQLqVrJ 9sdmFeHKuAL0gwF6ePXdUaBNPcRNub3BEpt4IhiDwL57NELfETNaCvcCh1qZR9CtNrrLDG6JtCEXq 3voBvjuH9ajF8bZvq3Cjf8S5ZJ+/RCab6EtFb3jgLednPhUmO4kvRzYD7Bqoi0yIMCcUKwTAxnVJc jbe8dIQIb17LLK3TWwX+zPegpmSeziQ==
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

struct pci_dev::driver contains (apart from a constant offset) the same
data as struct pci_dev::dev->driver. Replace all remaining users of the
former pointer by the latter to allow removing the former.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
---
 arch/powerpc/kernel/eeh_driver.c | 10 ++++-----
 arch/x86/events/intel/uncore.c   |  2 +-
 arch/x86/kernel/probe_roms.c     |  2 +-
 drivers/misc/cxl/guest.c         | 24 ++++++++++++---------
 drivers/misc/cxl/pci.c           | 30 ++++++++++++++++----------
 drivers/pci/iov.c                | 25 ++++++++++++++--------
 drivers/pci/pci-driver.c         | 25 +++++++++++-----------
 drivers/pci/pci.c                |  4 ++--
 drivers/pci/pcie/err.c           | 36 ++++++++++++++++++--------------
 drivers/pci/xen-pcifront.c       |  4 ++--
 drivers/usb/host/xhci-pci.c      |  2 +-
 11 files changed, 93 insertions(+), 71 deletions(-)

diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3eff6a4888e7..350dab18e137 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -104,13 +104,13 @@ static bool eeh_edev_actionable(struct eeh_dev *edev)
  */
 static inline struct pci_driver *eeh_pcid_get(struct pci_dev *pdev)
 {
-	if (!pdev || !pdev->driver)
+	if (!pdev || !pdev->dev.driver)
 		return NULL;
 
-	if (!try_module_get(pdev->driver->driver.owner))
+	if (!try_module_get(pdev->dev.driver->owner))
 		return NULL;
 
-	return pdev->driver;
+	return to_pci_driver(pdev->dev.driver);
 }
 
 /**
@@ -122,10 +122,10 @@ static inline struct pci_driver *eeh_pcid_get(struct pci_dev *pdev)
  */
 static inline void eeh_pcid_put(struct pci_dev *pdev)
 {
-	if (!pdev || !pdev->driver)
+	if (!pdev || !pdev->dev.driver)
 		return;
 
-	module_put(pdev->driver->driver.owner);
+	module_put(pdev->dev.driver->owner);
 }
 
 /**
diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c
index c72e368dd164..f1ba6ab2e97e 100644
--- a/arch/x86/events/intel/uncore.c
+++ b/arch/x86/events/intel/uncore.c
@@ -1187,7 +1187,7 @@ static int uncore_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id
 	 * PCI slot and func to indicate the uncore box.
 	 */
 	if (id->driver_data & ~0xffff) {
-		struct pci_driver *pci_drv = pdev->driver;
+		struct pci_driver *pci_drv = to_pci_driver(pdev->dev.driver);
 
 		pmu = uncore_pci_find_dev_pmu(pdev, pci_drv->id_table);
 		if (pmu == NULL)
diff --git a/arch/x86/kernel/probe_roms.c b/arch/x86/kernel/probe_roms.c
index 9e1def3744f2..36e84d904260 100644
--- a/arch/x86/kernel/probe_roms.c
+++ b/arch/x86/kernel/probe_roms.c
@@ -80,7 +80,7 @@ static struct resource video_rom_resource = {
  */
 static bool match_id(struct pci_dev *pdev, unsigned short vendor, unsigned short device)
 {
-	struct pci_driver *drv = pdev->driver;
+	struct pci_driver *drv = to_pci_driver(pdev->dev.driver);
 	const struct pci_device_id *id;
 
 	if (pdev->vendor == vendor && pdev->device == device)
diff --git a/drivers/misc/cxl/guest.c b/drivers/misc/cxl/guest.c
index 186308f1f8eb..d997c9c3ebb5 100644
--- a/drivers/misc/cxl/guest.c
+++ b/drivers/misc/cxl/guest.c
@@ -25,28 +25,32 @@ static void pci_error_handlers(struct cxl_afu *afu,
 		return;
 
 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-		if (!afu_dev->driver)
+		struct pci_driver *afu_drv;
+
+		if (!afu_dev->dev.driver)
 			continue;
 
+		afu_drv = to_pci_driver(afu_dev->dev.driver);
+
 		switch (bus_error_event) {
 		case CXL_ERROR_DETECTED_EVENT:
 			afu_dev->error_state = state;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->error_detected)
-				afu_dev->driver->err_handler->error_detected(afu_dev, state);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->error_detected)
+				afu_drv->err_handler->error_detected(afu_dev, state);
 		break;
 		case CXL_SLOT_RESET_EVENT:
 			afu_dev->error_state = state;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->slot_reset)
-				afu_dev->driver->err_handler->slot_reset(afu_dev);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->slot_reset)
+				afu_drv->err_handler->slot_reset(afu_dev);
 		break;
 		case CXL_RESUME_EVENT:
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->resume)
-				afu_dev->driver->err_handler->resume(afu_dev);
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->resume)
+				afu_drv->err_handler->resume(afu_dev);
 		break;
 		}
 	}
diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
index 2ba899f5659f..7e7545d01e27 100644
--- a/drivers/misc/cxl/pci.c
+++ b/drivers/misc/cxl/pci.c
@@ -1805,14 +1805,16 @@ static pci_ers_result_t cxl_vphb_error_detected(struct cxl_afu *afu,
 		return result;
 
 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-		if (!afu_dev->driver)
+		struct pci_driver *afu_drv;
+		if (!afu_dev->dev.driver)
 			continue;
 
+		afu_drv = to_pci_driver(afu_dev->dev.driver);
+
 		afu_dev->error_state = state;
 
-		if (afu_dev->driver->err_handler)
-			afu_result = afu_dev->driver->err_handler->error_detected(afu_dev,
-										  state);
+		if (afu_drv->err_handler)
+			afu_result = afu_drv->err_handler->error_detected(afu_dev, state);
 		/* Disconnect trumps all, NONE trumps NEED_RESET */
 		if (afu_result == PCI_ERS_RESULT_DISCONNECT)
 			result = PCI_ERS_RESULT_DISCONNECT;
@@ -2003,6 +2005,8 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
 			continue;
 
 		list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
+			struct pci_driver *afu_drv;
+
 			/* Reset the device context.
 			 * TODO: make this less disruptive
 			 */
@@ -2028,12 +2032,14 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
 			 * shouldn't start new work until we call
 			 * their resume function.
 			 */
-			if (!afu_dev->driver)
+			if (!afu_dev->dev.driver)
 				continue;
 
-			if (afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->slot_reset)
-				afu_result = afu_dev->driver->err_handler->slot_reset(afu_dev);
+			afu_drv = to_pci_driver(afu_dev->dev.driver);
+
+			if (afu_drv->err_handler &&
+			    afu_drv->err_handler->slot_reset)
+				afu_result = afu_drv->err_handler->slot_reset(afu_dev);
 
 			if (afu_result == PCI_ERS_RESULT_DISCONNECT)
 				result = PCI_ERS_RESULT_DISCONNECT;
@@ -2074,9 +2080,11 @@ static void cxl_pci_resume(struct pci_dev *pdev)
 			continue;
 
 		list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
-			if (afu_dev->driver && afu_dev->driver->err_handler &&
-			    afu_dev->driver->err_handler->resume)
-				afu_dev->driver->err_handler->resume(afu_dev);
+			struct pci_driver *afu_drv;
+			if (afu_dev->dev.driver &&
+			    (afu_drv = to_pci_driver(afu_dev->dev.driver))->err_handler &&
+			    afu_drv->err_handler->resume)
+				afu_drv->err_handler->resume(afu_dev);
 		}
 	}
 	spin_unlock(&adapter->afu_list_lock);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index dafdc652fcd0..f94660927544 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -164,13 +164,15 @@ static ssize_t sriov_vf_total_msix_show(struct device *dev,
 					char *buf)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pci_driver *pdrv;
 	u32 vf_total_msix = 0;
 
 	device_lock(dev);
-	if (!pdev->driver || !pdev->driver->sriov_get_vf_total_msix)
+	pdrv = to_pci_driver(dev->driver);
+	if (!pdrv || !pdrv->sriov_get_vf_total_msix)
 		goto unlock;
 
-	vf_total_msix = pdev->driver->sriov_get_vf_total_msix(pdev);
+	vf_total_msix = pdrv->sriov_get_vf_total_msix(pdev);
 unlock:
 	device_unlock(dev);
 	return sysfs_emit(buf, "%u\n", vf_total_msix);
@@ -183,6 +185,7 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 {
 	struct pci_dev *vf_dev = to_pci_dev(dev);
 	struct pci_dev *pdev = pci_physfn(vf_dev);
+	struct pci_driver *pdrv;
 	int val, ret;
 
 	ret = kstrtoint(buf, 0, &val);
@@ -193,13 +196,14 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 		return -EINVAL;
 
 	device_lock(&pdev->dev);
-	if (!pdev->driver || !pdev->driver->sriov_set_msix_vec_count) {
+	pdrv = to_pci_driver(pdev->dev.driver);
+	if (!pdrv || !pdrv->sriov_set_msix_vec_count) {
 		ret = -EOPNOTSUPP;
 		goto err_pdev;
 	}
 
 	device_lock(&vf_dev->dev);
-	if (vf_dev->driver) {
+	if (vf_dev->dev.driver) {
 		/*
 		 * A driver is already attached to this VF and has configured
 		 * itself based on the current MSI-X vector count. Changing
@@ -209,7 +213,7 @@ static ssize_t sriov_vf_msix_count_store(struct device *dev,
 		goto err_dev;
 	}
 
-	ret = pdev->driver->sriov_set_msix_vec_count(vf_dev, val);
+	ret = pdrv->sriov_set_msix_vec_count(vf_dev, val);
 
 err_dev:
 	device_unlock(&vf_dev->dev);
@@ -376,6 +380,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 				  const char *buf, size_t count)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
+	struct pci_driver *pdrv;
 	int ret;
 	u16 num_vfs;
 
@@ -392,14 +397,16 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 		goto exit;
 
 	/* is PF driver loaded */
-	if (!pdev->driver) {
+	if (!pdev->dev.driver) {
 		pci_info(pdev, "no driver bound to device; cannot configure SR-IOV\n");
 		ret = -ENOENT;
 		goto exit;
 	}
 
+	pdrv = to_pci_driver(pdev->dev.driver);
+
 	/* is PF driver loaded w/callback */
-	if (!pdev->driver->sriov_configure) {
+	if (!pdrv->sriov_configure) {
 		pci_info(pdev, "driver does not support SR-IOV configuration via sysfs\n");
 		ret = -ENOENT;
 		goto exit;
@@ -407,7 +414,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 
 	if (num_vfs == 0) {
 		/* disable VFs */
-		ret = pdev->driver->sriov_configure(pdev, 0);
+		ret = pdrv->sriov_configure(pdev, 0);
 		goto exit;
 	}
 
@@ -419,7 +426,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,
 		goto exit;
 	}
 
-	ret = pdev->driver->sriov_configure(pdev, num_vfs);
+	ret = pdrv->sriov_configure(pdev, num_vfs);
 	if (ret < 0)
 		goto exit;
 
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index 50449ec622a3..4d20022b8631 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -457,7 +457,7 @@ static int pci_device_probe(struct device *dev)
 static void pci_device_remove(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	if (drv->remove) {
 		pm_runtime_get_sync(dev);
@@ -493,7 +493,7 @@ static void pci_device_remove(struct device *dev)
 static void pci_device_shutdown(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	pm_runtime_resume(dev);
 
@@ -589,7 +589,7 @@ static int pci_pm_reenable_device(struct pci_dev *pci_dev)
 static int pci_legacy_suspend(struct device *dev, pm_message_t state)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(dev->driver);
 
 	if (drv && drv->suspend) {
 		pci_power_t prev = pci_dev->current_state;
@@ -630,7 +630,7 @@ static int pci_legacy_suspend_late(struct device *dev, pm_message_t state)
 static int pci_legacy_resume(struct device *dev)
 {
 	struct pci_dev *pci_dev = to_pci_dev(dev);
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 
 	pci_fixup_device(pci_fixup_resume, pci_dev);
 
@@ -649,7 +649,7 @@ static void pci_pm_default_suspend(struct pci_dev *pci_dev)
 
 static bool pci_has_legacy_pm_support(struct pci_dev *pci_dev)
 {
-	struct pci_driver *drv = pci_dev->driver;
+	struct pci_driver *drv = to_pci_driver(pci_dev->dev.driver);
 	bool ret = drv && (drv->suspend || drv->resume);
 
 	/*
@@ -1242,11 +1242,11 @@ static int pci_pm_runtime_suspend(struct device *dev)
 	int error;
 
 	/*
-	 * If pci_dev->driver is not set (unbound), we leave the device in D0,
+	 * If pci_dev->dev.driver is not set (unbound), we leave the device in D0,
 	 * but it may go to D3cold when the bridge above it runtime suspends.
 	 * Save its config space in case that happens.
 	 */
-	if (!pci_dev->driver) {
+	if (!pci_dev->dev.driver) {
 		pci_save_state(pci_dev);
 		return 0;
 	}
@@ -1303,7 +1303,7 @@ static int pci_pm_runtime_resume(struct device *dev)
 	 */
 	pci_restore_standard_config(pci_dev);
 
-	if (!pci_dev->driver)
+	if (!dev->driver)
 		return 0;
 
 	pci_fixup_device(pci_fixup_resume_early, pci_dev);
@@ -1322,14 +1322,13 @@ static int pci_pm_runtime_resume(struct device *dev)
 
 static int pci_pm_runtime_idle(struct device *dev)
 {
-	struct pci_dev *pci_dev = to_pci_dev(dev);
 	const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
 
 	/*
-	 * If pci_dev->driver is not set (unbound), the device should
+	 * If dev->driver is not set (unbound), the device should
 	 * always remain in D0 regardless of the runtime PM status
 	 */
-	if (!pci_dev->driver)
+	if (!dev->driver)
 		return 0;
 
 	if (!pm)
@@ -1436,8 +1435,8 @@ static struct pci_driver pci_compat_driver = {
  */
 struct pci_driver *pci_dev_driver(const struct pci_dev *dev)
 {
-	if (dev->driver)
-		return dev->driver;
+	if (dev->dev.driver)
+		return to_pci_driver(dev->dev.driver);
 	else {
 		int i;
 		for (i = 0; i <= PCI_ROM_RESOURCE; i++)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ce2ab62b64cf..ccecf740de59 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5089,7 +5089,7 @@ EXPORT_SYMBOL_GPL(pci_dev_unlock);
 static void pci_dev_save_and_disable(struct pci_dev *dev)
 {
 	const struct pci_error_handlers *err_handler =
-			dev->driver ? dev->driver->err_handler : NULL;
+			dev->dev.driver ? to_pci_driver(dev->dev.driver)->err_handler : NULL;
 
 	/*
 	 * dev->driver->err_handler->reset_prepare() is protected against
@@ -5120,7 +5120,7 @@ static void pci_dev_save_and_disable(struct pci_dev *dev)
 static void pci_dev_restore(struct pci_dev *dev)
 {
 	const struct pci_error_handlers *err_handler =
-			dev->driver ? dev->driver->err_handler : NULL;
+			dev->dev.driver ? to_pci_driver(dev->dev.driver)->err_handler : NULL;
 
 	pci_restore_state(dev);
 
diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
index b576aa890c76..b314b54f7821 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -49,14 +49,15 @@ static int report_error_detected(struct pci_dev *dev,
 				 pci_channel_state_t state,
 				 enum pci_ers_result *result)
 {
+	struct pci_driver *pdrv;
 	pci_ers_result_t vote;
 	const struct pci_error_handlers *err_handler;
 
 	device_lock(&dev->dev);
 	if (!pci_dev_set_io_state(dev, state) ||
-		!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->error_detected) {
+		!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->error_detected) {
 		/*
 		 * If any device in the subtree does not have an error_detected
 		 * callback, PCI_ERS_RESULT_NO_AER_DRIVER prevents subsequent
@@ -70,7 +71,7 @@ static int report_error_detected(struct pci_dev *dev,
 			vote = PCI_ERS_RESULT_NONE;
 		}
 	} else {
-		err_handler = dev->driver->err_handler;
+		err_handler = pdrv->err_handler;
 		vote = err_handler->error_detected(dev, state);
 	}
 	pci_uevent_ers(dev, vote);
@@ -92,15 +93,16 @@ static int report_normal_detected(struct pci_dev *dev, void *data)
 static int report_mmio_enabled(struct pci_dev *dev, void *data)
 {
 	pci_ers_result_t vote, *result = data;
+	struct pci_driver *pdrv;
 	const struct pci_error_handlers *err_handler;
 
 	device_lock(&dev->dev);
-	if (!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->mmio_enabled)
+	if (!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->mmio_enabled)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	vote = err_handler->mmio_enabled(dev);
 	*result = merge_result(*result, vote);
 out:
@@ -112,14 +114,15 @@ static int report_slot_reset(struct pci_dev *dev, void *data)
 {
 	pci_ers_result_t vote, *result = data;
 	const struct pci_error_handlers *err_handler;
+	struct pci_driver *pdrv;
 
 	device_lock(&dev->dev);
-	if (!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->slot_reset)
+	if (!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->slot_reset)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	vote = err_handler->slot_reset(dev);
 	*result = merge_result(*result, vote);
 out:
@@ -130,15 +133,16 @@ static int report_slot_reset(struct pci_dev *dev, void *data)
 static int report_resume(struct pci_dev *dev, void *data)
 {
 	const struct pci_error_handlers *err_handler;
+	struct pci_driver *pdrv;
 
 	device_lock(&dev->dev);
 	if (!pci_dev_set_io_state(dev, pci_channel_io_normal) ||
-		!dev->driver ||
-		!dev->driver->err_handler ||
-		!dev->driver->err_handler->resume)
+		!dev->dev.driver ||
+		!(pdrv = to_pci_driver(dev->dev.driver))->err_handler ||
+		!pdrv->err_handler->resume)
 		goto out;
 
-	err_handler = dev->driver->err_handler;
+	err_handler = pdrv->err_handler;
 	err_handler->resume(dev);
 out:
 	pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index f2d7f70a7a10..73831fb87a1e 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -601,12 +601,12 @@ static pci_ers_result_t pcifront_common_process(int cmd,
 	result = PCI_ERS_RESULT_NONE;
 
 	pcidev = pci_get_domain_bus_and_slot(domain, bus, devfn);
-	if (!pcidev || !pcidev->driver) {
+	if (!pcidev || !pcidev->dev.driver) {
 		dev_err(&pdev->xdev->dev, "device or AER driver is NULL\n");
 		pci_dev_put(pcidev);
 		return result;
 	}
-	pdrv = pcidev->driver;
+	pdrv = to_pci_driver(pcidev->dev.driver);
 
 	if (pdrv->err_handler && pdrv->err_handler->error_detected) {
 		pci_dbg(pcidev, "trying to call AER service\n");
diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c
index 2c9f25ca8edd..2f4729f4f1e0 100644
--- a/drivers/usb/host/xhci-pci.c
+++ b/drivers/usb/host/xhci-pci.c
@@ -103,7 +103,7 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci)
 	struct xhci_driver_data         *driver_data;
 	const struct pci_device_id      *id;
 
-	id = pci_match_id(pdev->driver->id_table, pdev);
+	id = pci_match_id(to_pci_driver(pdev->dev.driver)->id_table, pdev);
 
 	if (id && id->driver_data) {
 		driver_data = (struct xhci_driver_data *)id->driver_data;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:55:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:55:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198877.352602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVNX-0000Yo-D8; Wed, 29 Sep 2021 08:55:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198877.352602; Wed, 29 Sep 2021 08:55:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVNX-0000Yh-AB; Wed, 29 Sep 2021 08:55:51 +0000
Received: by outflank-mailman (input) for mailman id 198877;
 Wed, 29 Sep 2021 08:55:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CEC3=OT=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mVVNV-0000YU-Er
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:55:49 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a00477e7-3e9b-409a-904e-985d7576e1e5;
 Wed, 29 Sep 2021 08:55:47 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL8-0001if-2j; Wed, 29 Sep 2021 10:53:22 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL1-0004fa-GN; Wed, 29 Sep 2021 10:53:15 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVVL1-0000Ow-C0; Wed, 29 Sep 2021 10:53:15 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a00477e7-3e9b-409a-904e-985d7576e1e5
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>,
	linux-pci@vger.kernel.org,
	kernel@pengutronix.de,
	Alexander Duyck <alexanderduyck@fb.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Andrew Donnellan <ajd@linux.ibm.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Arnd Bergmann <arnd@arndb.de>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Borislav Petkov <bp@alien8.de>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	"David S. Miller" <davem@davemloft.net>,
	Fiona Trahe <fiona.trahe@intel.com>,
	Frederic Barrat <fbarrat@linux.ibm.com>,
	Giovanni Cabiddu <giovanni.cabiddu@intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Ido Schimmel <idosch@nvidia.com>,
	Ingo Molnar <mingo@redhat.com>,
	Jack Xu <jack.xu@intel.com>,
	Jakub Kicinski <kuba@kernel.org>,
	Jesse Brandeburg <jesse.brandeburg@intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Jiri Pirko <jiri@nvidia.com>,
	Juergen Gross <jgross@suse.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Marco Chiappero <marco.chiappero@intel.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Mathias Nyman <mathias.nyman@intel.com>,
	Michael Buesch <m@bues.ch>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Namhyung Kim <namhyung@kernel.org>,
	"Oliver O'Halloran" <oohall@gmail.com>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <peterz@infradead.org>,
	=?UTF-8?q?Rafa=C5=82=20Mi=C5=82ecki?= <zajec5@gmail.com>,
	Russell Currey <ruscur@russell.cc>,
	Salil Mehta <salil.mehta@huawei.com>,
	Sathya Prakash <sathya.prakash@broadcom.com>,
	Simon Horman <simon.horman@corigine.com>,
	Sreekanth Reddy <sreekanth.reddy@broadcom.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
	Taras Chornyi <tchornyi@marvell.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Tomaszx Kowalik <tomaszx.kowalik@intel.com>,
	Vadym Kochan <vkochan@marvell.com>,
	Wojciech Ziemba <wojciech.ziemba@intel.com>,
	Yisen Zhuang <yisen.zhuang@huawei.com>,
	Zhou Wang <wangzhou1@hisilicon.com>,
	linux-crypto@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-perf-users@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	linux-scsi@vger.kernel.org,
	linux-usb@vger.kernel.org,
	linux-wireless@vger.kernel.org,
	MPT-FusionLinux.pdl@broadcom.com,
	netdev@vger.kernel.org,
	oss-drivers@corigine.com,
	qat-linux@intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v5 00/11] PCI: Drop duplicated tracking of a pci_dev's bound driver
Date: Wed, 29 Sep 2021 10:52:55 +0200
Message-Id: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org

Hello,

this is v5 of the quest to drop the "driver" member from struct pci_dev
which tracks the same data (apart from a constant offset) as dev.driver.

Changes since v4:
 - split some changes out of "PCI: replace pci_dev::driver usage that
   gets the driver name" and reworked them a bit as suggested by Bjorn.
 - Add a line break in
   drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c to please
   checkpatch --strict and Simon Horman.
 - Fixed a build problem in "crypto: qat - simplify adf_enable_aer()",
   thanks to Giovanni Cabiddu for spotting and a suggested fix.

I didn't replace :: by . as suggested by Bjorn as I'm unsure if his
preference is stronger than mine.

This patch stack survived an allmodconfig build on arm64, m68k, powerpc,
riscv, s390, sparc64 and x86_64 on top of v5.15-rc3.

Best regards
Uwe

Uwe Kleine-König (11):
  PCI: Simplify pci_device_remove()
  PCI: Drop useless check from pci_device_probe()
  xen/pci: Drop some checks that are always true
  bcma: simplify reference to the driver's name
  powerpc/eeh: Don't use driver member of struct pci_dev and further
    cleanups
  ssb: Simplify determination of driver name
  PCI: Replace pci_dev::driver usage that gets the driver name
  scsi: message: fusion: Remove unused parameter of mpt_pci driver's
    probe()
  crypto: qat - simplify adf_enable_aer()
  PCI: Replace pci_dev::driver usage by pci_dev::dev.driver
  PCI: Drop duplicated tracking of a pci_dev's bound driver

 arch/powerpc/include/asm/ppc-pci.h            |  5 --
 arch/powerpc/kernel/eeh.c                     |  8 +++
 arch/powerpc/kernel/eeh_driver.c              | 10 +--
 arch/x86/events/intel/uncore.c                |  2 +-
 arch/x86/kernel/probe_roms.c                  |  2 +-
 drivers/bcma/host_pci.c                       |  6 +-
 drivers/crypto/hisilicon/qm.c                 |  2 +-
 drivers/crypto/qat/qat_4xxx/adf_drv.c         |  7 +--
 drivers/crypto/qat/qat_c3xxx/adf_drv.c        |  7 +--
 drivers/crypto/qat/qat_c62x/adf_drv.c         |  7 +--
 drivers/crypto/qat/qat_common/adf_aer.c       | 10 +--
 .../crypto/qat/qat_common/adf_common_drv.h    |  3 +-
 drivers/crypto/qat/qat_dh895xcc/adf_drv.c     |  7 +--
 drivers/message/fusion/mptbase.c              |  7 +--
 drivers/message/fusion/mptbase.h              |  2 +-
 drivers/message/fusion/mptctl.c               |  4 +-
 drivers/message/fusion/mptlan.c               |  2 +-
 drivers/misc/cxl/guest.c                      | 24 ++++---
 drivers/misc/cxl/pci.c                        | 30 +++++----
 .../ethernet/hisilicon/hns3/hns3_ethtool.c    |  2 +-
 .../ethernet/marvell/prestera/prestera_pci.c  |  2 +-
 drivers/net/ethernet/mellanox/mlxsw/pci.c     |  2 +-
 .../ethernet/netronome/nfp/nfp_net_ethtool.c  |  3 +-
 drivers/pci/iov.c                             | 25 +++++---
 drivers/pci/pci-driver.c                      | 45 ++++++-------
 drivers/pci/pci.c                             |  4 +-
 drivers/pci/pcie/err.c                        | 36 ++++++-----
 drivers/pci/xen-pcifront.c                    | 63 +++++++++----------
 drivers/ssb/pcihost_wrapper.c                 |  6 +-
 drivers/usb/host/xhci-pci.c                   |  2 +-
 include/linux/pci.h                           |  1 -
 31 files changed, 161 insertions(+), 175 deletions(-)

Range-diff against v4:
 1:  8d064bbc74b0 =  1:  c2b53ab26a6b PCI: Simplify pci_device_remove()
 2:  966b49c308b4 =  2:  2c733e1d5186 PCI: Drop useless check from pci_device_probe()
 3:  ce710d6e8a1b =  3:  547ca5a7aa16 xen/pci: Drop some checks that are always true
 -:  ------------ >  4:  40eb07353844 bcma: simplify reference to the driver's name
 -:  ------------ >  5:  bab59c1dff6d powerpc/eeh: Don't use driver member of struct pci_dev and further cleanups
 -:  ------------ >  6:  abd70de9782d ssb: Simplify determination of driver name
 4:  3e4e6994a59d !  7:  735845bd26b9 PCI: replace pci_dev::driver usage that gets the driver name
    @@ Metadata
     Author: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
     
      ## Commit message ##
    -    PCI: replace pci_dev::driver usage that gets the driver name
    +    PCI: Replace pci_dev::driver usage that gets the driver name
     
         struct pci_dev::driver holds (apart from a constant offset) the same
         data as struct pci_dev::dev->driver. With the goal to remove struct
    @@ Commit message
     
         Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
     
    - ## arch/powerpc/include/asm/ppc-pci.h ##
    -@@ arch/powerpc/include/asm/ppc-pci.h: void eeh_sysfs_remove_device(struct pci_dev *pdev);
    - 
    - static inline const char *eeh_driver_name(struct pci_dev *pdev)
    - {
    --	return (pdev && pdev->driver) ? pdev->driver->name : "<null>";
    -+	if (pdev) {
    -+		const char *drvstr = dev_driver_string(&pdev->dev);
    -+
    -+		if (strcmp(drvstr, ""))
    -+			return drvstr;
    -+	}
    -+
    -+	return "<null>";
    - }
    - 
    - #endif /* CONFIG_EEH */
    -
    - ## drivers/bcma/host_pci.c ##
    -@@ drivers/bcma/host_pci.c: static int bcma_host_pci_probe(struct pci_dev *dev,
    - 	if (err)
    - 		goto err_kfree_bus;
    - 
    --	name = dev_name(&dev->dev);
    --	if (dev->driver && dev->driver->name)
    --		name = dev->driver->name;
    -+	name = dev_driver_string(&dev->dev);
    -+	if (!strcmp(name, ""))
    -+		name = dev_name(&dev->dev);
    -+
    - 	err = pci_request_regions(dev, name);
    - 	if (err)
    - 		goto err_pci_disable;
    -
      ## drivers/crypto/hisilicon/qm.c ##
     @@ drivers/crypto/hisilicon/qm.c: static int qm_alloc_uacce(struct hisi_qm *qm)
      	};
    @@ drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c: nfp_get_drvinfo(struct nfp
      	char nsp_version[ETHTOOL_FWVERS_LEN] = {};
      
     -	strlcpy(drvinfo->driver, pdev->driver->name, sizeof(drvinfo->driver));
    -+	strlcpy(drvinfo->driver, dev_driver_string(&pdev->dev), sizeof(drvinfo->driver));
    ++	strlcpy(drvinfo->driver, dev_driver_string(&pdev->dev),
    ++		sizeof(drvinfo->driver));
      	nfp_net_get_nspinfo(app, nsp_version);
      	snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version),
      		 "%s %s %s %s", vnic_version, nsp_version,
    -
    - ## drivers/ssb/pcihost_wrapper.c ##
    -@@ drivers/ssb/pcihost_wrapper.c: static int ssb_pcihost_probe(struct pci_dev *dev,
    - 	err = pci_enable_device(dev);
    - 	if (err)
    - 		goto err_kfree_ssb;
    --	name = dev_name(&dev->dev);
    --	if (dev->driver && dev->driver->name)
    --		name = dev->driver->name;
    -+
    -+	name = dev_driver_string(&dev->dev);
    -+	if (*name == '\0')
    -+		name = dev_name(&dev->dev);
    -+
    - 	err = pci_request_regions(dev, name);
    - 	if (err)
    - 		goto err_pci_disable;
 5:  574b743327b8 =  8:  1e58019165b9 scsi: message: fusion: Remove unused parameter of mpt_pci driver's probe()
 6:  674c6efd3dab !  9:  dea72a470141 crypto: qat - simplify adf_enable_aer()
    @@ drivers/crypto/qat/qat_4xxx/adf_drv.c: static struct pci_driver adf_driver = {
      	.probe = adf_probe,
      	.remove = adf_remove,
      	.sriov_configure = adf_sriov_configure,
    -+	.err_handler = adf_err_handler,
    ++	.err_handler = &adf_err_handler,
      };
      
      module_pci_driver(adf_driver);
    @@ drivers/crypto/qat/qat_c3xxx/adf_drv.c: static struct pci_driver adf_driver = {
      	.probe = adf_probe,
      	.remove = adf_remove,
      	.sriov_configure = adf_sriov_configure,
    -+	.err_handler = adf_err_handler,
    ++	.err_handler = &adf_err_handler,
      };
      
      static void adf_cleanup_pci_dev(struct adf_accel_dev *accel_dev)
    @@ drivers/crypto/qat/qat_c62x/adf_drv.c: static struct pci_driver adf_driver = {
      	.probe = adf_probe,
      	.remove = adf_remove,
      	.sriov_configure = adf_sriov_configure,
    -+	.err_handler = adf_err_handler,
    ++	.err_handler = &adf_err_handler,
      };
      
      static void adf_cleanup_pci_dev(struct adf_accel_dev *accel_dev)
    @@ drivers/crypto/qat/qat_common/adf_common_drv.h: void adf_ae_fw_release(struct ad
      int adf_ae_stop(struct adf_accel_dev *accel_dev);
      
     -int adf_enable_aer(struct adf_accel_dev *accel_dev);
    ++extern const struct pci_error_handlers adf_err_handler;
     +void adf_enable_aer(struct adf_accel_dev *accel_dev);
      void adf_disable_aer(struct adf_accel_dev *accel_dev);
      void adf_reset_sbr(struct adf_accel_dev *accel_dev);
    @@ drivers/crypto/qat/qat_dh895xcc/adf_drv.c: static struct pci_driver adf_driver =
      	.probe = adf_probe,
      	.remove = adf_remove,
      	.sriov_configure = adf_sriov_configure,
    -+	.err_handler = adf_err_handler,
    ++	.err_handler = &adf_err_handler,
      };
      
      static void adf_cleanup_pci_dev(struct adf_accel_dev *accel_dev)
 7:  0e022deb0f75 = 10:  b4165dda38ea PCI: Replace pci_dev::driver usage by pci_dev::dev.driver
 8:  edd9d24df02a = 11:  d93a138bd7ab PCI: Drop duplicated tracking of a pci_dev's bound driver

base-commit: 5816b3e6577eaa676ceb00a848f0fd65fe2adc29
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:59:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198886.352614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVQe-0001Sn-0a; Wed, 29 Sep 2021 08:59:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198886.352614; Wed, 29 Sep 2021 08:59:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVQd-0001Sg-Ti; Wed, 29 Sep 2021 08:59:03 +0000
Received: by outflank-mailman (input) for mailman id 198886;
 Wed, 29 Sep 2021 08:59:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVVQd-0001Sa-As
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:59:03 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7d7a7094-2103-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:59:01 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18T84kr4019554;
 Wed, 29 Sep 2021 08:58:57 GMT
Received: from eur01-db5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2059.outbound.protection.outlook.com [104.47.2.59])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bcg0n0t1q-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 08:58:57 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6321.eurprd03.prod.outlook.com (2603:10a6:20b:157::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 08:58:53 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 08:58:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d7a7094-2103-11ec-bcdf-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VgXLDAgFNTrivUDY3h1uFxMsyfKSbi19j524k8TmEQiLCobo5uCEXWpCXe/RYBpfiKRsUbZ3BueMOcj6leOfCK/cyxwn065NEPC+oMHyWkbIhSP8KVYgx3t/Im87HM+0G8DfhS02UhZGFPmKYx7NeXlfCy2yyfQGycUhmaerFtgsWpVZwAFYdTWzzqArJVCIWAQdAkvRQQtTfsnDmOkgfsX1XC2fwuGA8qxPCAceLZtPAnpkorysx3Sjb5F9tVXrdfYqL1mXTdfRvudIwkl/iKMo6sZzGgK0VJXvczATLDfAEiUl9vsvA3FJs8/fsfiFGWg17DbwDXUdzxzFqYHF1Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Y9yIjwUgEDjl0IhuAuptyB1XfD7qbghho+J9Lpgir7E=;
 b=O8xpdbKcKIiVw5H26FUPY+INu4TadXddB6ijbI1aGBw1WVIxgD0JjszAq8RoNoGbaJUFCTfdPn72CXk2IOnq8gMqJYtDCMBVMp+ETmxbGVyuEgf/sQrEIfqLirk1Y/SPUNGwnGZtTWz3wFR00EmGZSRi9NxPwDrb6Sir8xbP5KZVNq0pwuPOR54OcSj/bxMn8ngGQaYDdTVgDr++ng0SgeE72PEQlJ69WlMLCAa3w+pCEY0BB2TD3J8sRob9aFml97F2dhEByMHoeUSMp8hu9fA+EXp9RX1H1J9asjQPDXId43pJALsUYbxbSi3PTMW4uRNhBhJ4VZrVzzT/4viccw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y9yIjwUgEDjl0IhuAuptyB1XfD7qbghho+J9Lpgir7E=;
 b=Kqg0FMXGm0eBqOr9YHLfZk69D6HhEfUGPmmc1qBt2Uxpv74NwAWb0qyzWITQN83cTwD2sFqhInyxBNnmzhopWIZIiG46lU3zlF39pigiUTa/9B7RGTpZctj9WHVwlS1Qt+SX+VqC9K5WK6yE7nsCP51RA47aY0LyLgw1GEeBwjAYUhY7n/WV4r/zFpeFAi93AwCivMrdtnpFdiXnmWXldowM9o8lzeYsBZCoiFUwIlcqlXLiVhjJqpxXAYqaAK8Uqdv6ik9hO6/cL32PXz1t+icUF6d+an45i2qwCISsfwT0FNln8W6lgJAkrmaE2qpebkT5Omvq6aMt54fVsHeRxA==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Michal
 Orzel <michal.orzel@arm.com>
Subject: Re: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Topic: [PATCH v2 07/11] vpci/header: program p2m with guest BAR view
Thread-Index: AQHXsHpCfe0n3k8oF0eTl7KOfxT3R6u6ssAAgAAA3gCAAAIlAIAAA4oAgAAGMQA=
Date: Wed, 29 Sep 2021 08:58:53 +0000
Message-ID: <fd611978-b6f8-3ded-5b5d-f132c8a6de5c@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-8-andr2000@gmail.com>
 <1a341bc1-f7ad-e7ae-1241-a2d5f38f368f@arm.com>
 <f59b48fd-564e-64f8-1422-fb842957f213@suse.com>
 <7ae0fad7-499b-1bda-5c9e-9313265abf8c@epam.com>
 <30b8c6b8-5dc4-6972-eaf9-76024f7e97bb@suse.com>
In-Reply-To: <30b8c6b8-5dc4-6972-eaf9-76024f7e97bb@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ece8db52-d26f-4b28-ed9a-08d983275cc5
x-ms-traffictypediagnostic: AM0PR03MB6321:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB6321E5042F833AF29B19D275E7A99@AM0PR03MB6321.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5236;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 sUq94av34TcaakcrW70YQsXEq5kIGYdkn6rjmRF7gami1O9cxHTGqU4bInhenLT9qb90zT05KmrtlKKzUzU4Ej7hLWaUboDdkngASRcIIETbdnnzd/Oicx+P0oQeSysSXQvVIc8uN3VI7LGi1CVQkBPXlKhZtPRijMNIcOjF8lprKduVNfwg0qPDarGoJIbq6mWdZbPUPNbQZvXAIHoG8QbdLcWK/sxpuepeywoMoawI/4RP2cettPdDEldvnv5ffvEidWJhRVBjMZAV9BEjJJ/zRxaPR50fDYHV0jj+wuGZM/kS0UJ4WnleO/eqJg+pC84zjR3/DFcbrcw+AFUpaZpDT0oPDn4JW8Z5W9LUnbj2c26nUuJRx/z2deQ3F2nBi+3q2+jBjql6FFFTfR6RnqJyKSJv+9oeudo8TP0tBZ5/qoeM9/4m6wkGhDmYMyY1mJ9WnXfZR9YL/KRPLky99D4u8sCRZ0isjxDpyvxmVqu6meJ9UJA+AITpvIzy43vgeN6PNB0eX/oT9AKaqXVL6gFPq2gsZ80U1S9/RSnVKRNJkOXnj4nc3luvYM1gA7lTXI4d+IpoHYSFWpZaCP8hREn6+j/ExTLkyqIZ2/oWgWf65QResA8PFteChsi58q/4nFbuwjeZks23+srt7e6s/RbCkPsBKXWx9N3o5S5cMvlMZK8sHqZ0mzysZdC3/Mx6QmvgJMhXyGx4g1Nnv0g4by6+k7AHyFNM/F6XGWq4Gko=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(71200400001)(2616005)(508600001)(36756003)(316002)(4326008)(6916009)(38070700005)(76116006)(122000001)(38100700002)(26005)(31696002)(86362001)(186003)(2906002)(8676002)(6512007)(66446008)(64756008)(66556008)(66476007)(31686004)(55236004)(6486002)(53546011)(5660300002)(54906003)(6506007)(8936002)(66946007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NmNwNjAydFFoYkpjbFhpQ3RaaWlRUnZmdWFuaXhueGFGQnJ5NTA4cFcyWTNz?=
 =?utf-8?B?ZDl3UkNhZHdUSG9TVHFSTmg5WVZNb1lCUnF3TVlldUxuMVdKMlJtWHJPSE5h?=
 =?utf-8?B?WFcyR1ZkVDBXY0g0b1UyQnBGbmVFMEdiOUpBWVRHTEd0SFRWblNScmM0dEJM?=
 =?utf-8?B?NXBJdnZBTXM0WUtoM3F1akJLcGZaQ3FPbVhSVzFDY0RCVlN5ai90N0dJd0tN?=
 =?utf-8?B?R2dLUlpWcVQ5Wktjdmg2bmFmR2tnUDA3aUtJSGJqNWVJYWZkNmsyYUpDdmRn?=
 =?utf-8?B?UThpM2UxcFcrMUFIbDFxbWs3ZGtqWml6ZHl5dWlISnovTTd2ajV3QUlLdHpF?=
 =?utf-8?B?aTBFb0JEUjRHT01kRHRHUVk2cTV1ZzVDYzZiNzRpWm1neFNSc3N0M3ZSb3FI?=
 =?utf-8?B?Q2gzdllzZHBMMXROWm9LZTIxWTdDVkJNNXhpemltTTkrSHFPL2hQaU56Ukwr?=
 =?utf-8?B?VG96R21iVUpBRHJFUU5rTkM2eUVIRjRxM1NBV2U1RExvQzBKRWQrcm03WkNC?=
 =?utf-8?B?bFZWZGRaQUc0TmV2b09keTIwOTRDbzlWQytCSWE1MjRZR2lVSkRaN3p3WktW?=
 =?utf-8?B?YndjUHE1V1RXeHRLYVpzV0lKd3BPUUE5Y0M4N3pVY3l6ZUxZMVFYd1BRTGU3?=
 =?utf-8?B?U2dUWmhqemFaR0IvQXgzMmN2NTVqWnkzTHIrdnRoeDFlbEluOE1BMWtrREhI?=
 =?utf-8?B?aHM3VXl6aU9XekxkSVBxb29mRFQzLzRkZWsvRmE0L1d4Q0FMeDE2MXFpM3hh?=
 =?utf-8?B?ZC9VVjZNdkd1RlBUNi95bVdDc3VDNzZsZURONEpIVTZFVDMzZDloMU53V21h?=
 =?utf-8?B?TEJYeFR5dHhVN1E5czl0aE9DUU0zbGZubUtuakgvcUhJQ0RSRktzWThhRG54?=
 =?utf-8?B?ZmJNYy9CTnUwQ2cwKzN6QmUzdmN3RUtRNjdyaVY1bTRIWkVOTUxvcjRlQ3du?=
 =?utf-8?B?WEo2QzZ1bDY0VEx6ZENxNWxZZUZURWxxcTEvOFl6ZElEUHRFTEZpemZVRXA5?=
 =?utf-8?B?alZ4T1BWT1VxUjhCcm81ZnI0WEp2VU9nUHFaNlgraHBMK2pRdENsVk91ZWR5?=
 =?utf-8?B?YjVqeEdra00wdkZQMGNtTGJVZjVlK09RMnBiRWxNUnc5M1ZRbHFOdVE4bHBT?=
 =?utf-8?B?V2tCOGxlQ094TmpyYmZWaHFzQk14dnJuV2N2djJPQ1dZZGdPZHFtL3JpNS9t?=
 =?utf-8?B?cFlBTG52L2pzcTIzUnNNNGlCTTZsdHcwRE9CK01HTWNlMVh0ZG1Gblh0V2JM?=
 =?utf-8?B?bnFoOGtmNXBTVXhxSFRWSWtLcFVOVDdvcXQrV0w5NXVPT2NlYWo2djJiVGJl?=
 =?utf-8?B?czAxa0k3SkRrcTJTR3BHQmRES2ZKZ0JMa05BZGN2d2tXUEVUVXdGUDRvM3k2?=
 =?utf-8?B?M1NPS2J1YmdWZEk0bmtpdUQxUmZHTFJxem9pVkRSSUp2RkF6UUZIaEh3STk0?=
 =?utf-8?B?cmdla2YxN1RkK2V1M0tmRjBWcVJIZlFQcGxkWnJOQzNDaDNuTysxVVZya2Vv?=
 =?utf-8?B?OWZldkE0aVRlWG1UNGxFbTE5NEo1ZDVXQ3drQVNSSzlZR0lDc1gzbS9zaWtl?=
 =?utf-8?B?SG9MTjNhSlEwL1VQc1g4SWd4dk95NU1BSTFDOUZUQjR4SlJqMzA0cEJwdTlT?=
 =?utf-8?B?Wkx4VkthZ2hvL2dpZ0dnOHhGR0hmZm4zRHllcFFpelJCd3FsMDg4U1k3aW5G?=
 =?utf-8?B?UG0xYlhjdEFveXB5UGtKNVN6bE1McFBaTnkwQ0FjbVhMQTZDUEptSFNKNTRM?=
 =?utf-8?B?d1ZEb3FFZXBnZmRSOTlyZUdLTUdJeldNdXJESlM3QUdOVTdLa2h6d05NZEZk?=
 =?utf-8?B?Tk9POWxRUEVZTGJFL2E2UT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <1EA9130C972C854584E84C300159F5C3@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ece8db52-d26f-4b28-ed9a-08d983275cc5
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 08:58:53.1939
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2YyUE0/6/PI3dzwvortDLBEjbJM4xTBw6HNVNbYBvWPK/xtNjs9ydwX0IJFIZ/m/L/BR+brD8wc6v/qKPsRz9iyj0GBAtVaz+d/t0cVjmQbx0yOqzPYxLzZcpp9yvzdn
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6321
X-Proofpoint-ORIG-GUID: ALes0xBjEU6Cnxdob-dgk9DJ8l-a9hSL
X-Proofpoint-GUID: ALes0xBjEU6Cnxdob-dgk9DJ8l-a9hSL
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_02,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0
 mlxlogscore=730 lowpriorityscore=0 impostorscore=0 adultscore=0
 bulkscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 spamscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290053

DQpPbiAyOS4wOS4yMSAxMTozNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI5LjA5LjIwMjEg
MTA6MjQsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjkuMDkuMjEgMTE6
MTYsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI5LjA5LjIwMjEgMTA6MTMsIE1pY2hhbCBP
cnplbCB3cm90ZToNCj4+Pj4gT24gMjMuMDkuMjAyMSAxNDo1NCwgT2xla3NhbmRyIEFuZHJ1c2hj
aGVua28gd3JvdGU6DQo+Pj4+PiBAQCAtMTQ5LDYgKzE3MiwxMCBAQCBib29sIHZwY2lfcHJvY2Vz
c19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQ0KPj4+Pj4gICAgICAgICAgICAgICAgaWYgKCAhYmFy
LT5tZW0gKQ0KPj4+Pj4gICAgICAgICAgICAgICAgICAgIGNvbnRpbnVlOw0KPj4+Pj4gICAgDQo+
Pj4+PiArICAgICAgICAgICAgZGF0YS5zdGFydF9nZm4gPSBpc19oYXJkd2FyZV9kb21haW4odi0+
dnBjaS5wZGV2LT5kb21haW4pID8NCj4+Pj4+ICsgICAgICAgICAgICAgICAgX2dmbihQRk5fRE9X
TihiYXItPmFkZHIpKSA6DQo+Pj4+PiArICAgICAgICAgICAgICAgIF9nZm4oUEZOX0RPV04oYmFy
LT5ndWVzdF9hZGRyKSk7DQo+Pj4+IEkgYmVsaWV2ZSB0aGlzIHdvdWxkIGxvb2sgYmV0dGVyIHdp
dGggdGhlIGZvbGxvd2luZyBhbGlnbm1lbnQ6DQo+Pj4+IGRhdGEuc3RhcnRfZ2ZuID0gaXNfaGFy
ZHdhcmVfZG9tYWluKHYtPnZwY2kucGRldi0+ZG9tYWluKQ0KPj4+PiAgICAgICAgICAgICAgICAg
ICAgPyBfZ2ZuKFBGTl9ET1dOKGJhci0+YWRkcikpDQo+Pj4+ICAgICAgICAgICAgICAgICAgICA6
IF9nZm4oUEZOX0RPV04oYmFyLT5ndWVzdF9hZGRyKSk7DQo+Pj4gRldJVyBJIGFncmVlLCB5ZXQg
cGVyc29uYWxseSBJIHRoaW5rIHRoZSBjb25kaXRpb25hbCBvcGVyYXRvciBoZXJlDQo+Pj4gZXZl
biB3YW50cyB0byBtb3ZlIGluc2lkZSB0aGUgX2dmbihQRk5fRE9XTigpKS4NCj4+IEkgY2FuIGRv
IGl0IGFzIHdlbGw6DQo+Pg0KPj4gZGF0YS5zdGFydF9nZm4gPSBfZ2ZuKFBGTl9ET1dOKGlzX2hh
cmR3YXJlX2RvbWFpbih2LT52cGNpLnBkZXYtPmRvbWFpbikgPyBiYXItPmFkZHIgOiBiYXItPmd1
ZXN0X2FkZHIpKQ0KPj4gQnV0LCBoZWxwIG1lIHBsZWFzZSBicmVha2luZyBpdCBpbnRvIG11bHRp
bGluZSB3aXRoIDgwIGNoYXJzIHJlc3BlY3RlZA0KPiBCZXNpZGVzIHRoZSBvcHRpb24gb2YgbGF0
Y2hpbmcgdi0+dnBjaS5wZGV2LT5kb21haW4gb3INCj4gaXNfaGFyZHdhcmVfZG9tYWluKHYtPnZw
Y2kucGRldi0+ZG9tYWluKSBpbnRvIGEgaGVscGVyIHZhcmlhYmxlLA0KPg0KPiAgICAgICAgICAg
ICAgZGF0YS5zdGFydF9nZm4gPQ0KPiAgICAgICAgICAgICAgICAgICBfZ2ZuKFBGTl9ET1dOKGlz
X2hhcmR3YXJlX2RvbWFpbih2LT52cGNpLnBkZXYtPmRvbWFpbikNCj4gICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICA/IGJhci0+YWRkciA6IGJhci0+Z3Vlc3RfYWRkcikpOw0KSSdsbCBn
byB3aXRoIHRoaXMgb25lLCB0aGFuayB5b3UNCj4NCj4gb3INCj4NCj4gICAgICAgICAgICAgIGRh
dGEuc3RhcnRfZ2ZuID0NCj4gICAgICAgICAgICAgICAgICAgX2dmbihQRk5fRE9XTihpc19oYXJk
d2FyZV9kb21haW4odi0+dnBjaS5wZGV2LT5kb21haW4pDQo+ICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgPyBiYXItPmFkZHINCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICA6IGJhci0+Z3Vlc3RfYWRkcikpOw0KPg0KPiBKYW4NCj4=


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 08:59:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 08:59:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198892.352625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVR4-0001zC-9k; Wed, 29 Sep 2021 08:59:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198892.352625; Wed, 29 Sep 2021 08:59:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVR4-0001z5-6a; Wed, 29 Sep 2021 08:59:30 +0000
Received: by outflank-mailman (input) for mailman id 198892;
 Wed, 29 Sep 2021 08:59:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5AY/=OT=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mVVR2-0001yt-W8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 08:59:29 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 8c8eb0e0-2103-11ec-bcdf-12813bfff9fa;
 Wed, 29 Sep 2021 08:59:27 +0000 (UTC)
Received: from AM6PR10CA0064.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:80::41)
 by AM9PR08MB6915.eurprd08.prod.outlook.com (2603:10a6:20b:30f::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Wed, 29 Sep
 2021 08:59:25 +0000
Received: from VE1EUR03FT010.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:80:cafe::b0) by AM6PR10CA0064.outlook.office365.com
 (2603:10a6:209:80::41) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 08:59:25 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT010.mail.protection.outlook.com (10.152.18.113) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 08:59:25 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Wed, 29 Sep 2021 08:59:23 +0000
Received: from 039d4f51148a.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5B183071-0C17-4C8F-8569-D2B040691BFE.1; 
 Wed, 29 Sep 2021 08:59:14 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 039d4f51148a.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 08:59:14 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 08:59:09 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.022; Wed, 29 Sep 2021
 08:59:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c8eb0e0-2103-11ec-bcdf-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4AeOF3KYIOF4Q8MoXwvV1uJhsAmxB4uwLExzJVehyrc=;
 b=nbGLQRZGeMwWWo/L62vDZ6+Vs/Bnx09Oz83ClZwsSZ9js6/mD9+wRjRZWvDJ33lX2qVLy+LejkDwBpCZTad53IUTvNw3sVzgV7yExBrypriDOuRd3zaT/Oe+sotlzWjFv75MKDgVCsNuLxOt+T38szxXPG1qXZZ/R3fjtRDI8kk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 220ff5382ab26446
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ey73Hy5HUL+0WSXm0fXcDIQrtj1A4B3nMZkRYHv8q1zfj/QDPuxI7YJdekJa7HBTsBLIAkB2x2ZUcV3pDjnmDmm9RMFIujk+SLg6yUjhYsouuY6qfopAkPMseisw9firUcGZsjpa5YGhhec5Q4de12b0CTLrldo9VrZnSZZoQUL6YkxL7guIMKVloffmxyUdcuc+RBvMInYRe+PGtF/LNZt+1IQEWzBEeq6y/jYFue696aLduX2N3HbAF7JgUXV3PkZ1Kjm6VHhM3egQz1LqihOOF7BiIIxwLiuOYkj/MBRDT/2Lf97AT81K9abm6hWxnglYniRHFQGgbclMp7yiEw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=4AeOF3KYIOF4Q8MoXwvV1uJhsAmxB4uwLExzJVehyrc=;
 b=gL1kl+mLsxUobpo3dsbkithMX2WYjDtQVea/2SXtBr09sCrb4uLPw3eQn/1scqCz7zjqIF6HJNbFoSe9fxlrw3UK6Irnd4ItCav0rUaYVfAqMnf2y/qoLo4V0gS2VrVvazyWwXj0NAV76YMHQ9QdVxi6KKAybLFT914zYMzH4/Dtb7fH1A7e5SBTBU/7yprzo2E/MyIs7xybmIyUjPm4cdXsjJ3xvGtTAJgdEfRz5VSUGHVUzfc8cikFnadw6rOMjDLB1ShrWGPCsQj7hyAf6ZkH4DYlUrKiHtt7JrbjYjP9cRT4wlE9uds6Upk9wo76VzkUgB69G3yWxcbQfQmZlQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4AeOF3KYIOF4Q8MoXwvV1uJhsAmxB4uwLExzJVehyrc=;
 b=nbGLQRZGeMwWWo/L62vDZ6+Vs/Bnx09Oz83ClZwsSZ9js6/mD9+wRjRZWvDJ33lX2qVLy+LejkDwBpCZTad53IUTvNw3sVzgV7yExBrypriDOuRd3zaT/Oe+sotlzWjFv75MKDgVCsNuLxOt+T38szxXPG1qXZZ/R3fjtRDI8kk=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
Thread-Topic: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
Thread-Index: AQHXtJXbn2GMwMXPv0CHEFVVsLXHhKu6r4SAgAAH1gA=
Date: Wed, 29 Sep 2021 08:59:09 +0000
Message-ID: <31E0D735-C7A4-48C1-AE5A-2906A140617E@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
 <27b97adb-c7f2-949e-140b-bbdb8f01457e@suse.com>
In-Reply-To: <27b97adb-c7f2-949e-140b-bbdb8f01457e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 381feac0-5c86-4f02-d2d6-08d983277010
x-ms-traffictypediagnostic: AS8PR08MB6919:|AM9PR08MB6915:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM9PR08MB6915FCEE1473FF7067A1464BFCA99@AM9PR08MB6915.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:1332;OLM:1332;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 y/c4O7xZxacaXShbUNOFhpCyBFA8Hq7B2YJCSJqvMwq/g115yoIYYTDMsdUEutNsoA59JKmLTUdAgngVGK+PglI3V8z3+6ZB67lqqe200uUVW4TeNyDs+JYiurpdsZzYMh3GjbuDUde5iwWBPWz66PnMVCRpSYu/AVSwd6X79Uin6gPNhQmaHhw1Zob35k4s5uCTz4eSrorsFJ5HWnDf6vmxb7gn3+wOWO7S65IWrmhmcAglUgFNaLNtColzJfPZ2t9zU2BIYNkEXt4qasnZ2RoO240Kn1wRRJfhvuxLNBwvJcxVEfofi/UOm9o9hbrn3/goBNX6V2HvFYef2sHlTt8GwvC04bnDKhbbi6XPYiZJU+1EEv88pI9qosoZwmWpNHpYuQjbClRT6mAgUNDjToXa8td2XPvENOfKy30Zv72l+w4TGH+1bVpJ+bPBSlYgY+zGtoM6iFj1pnx5hF4rN1rNKmzawhoTTojflOuOirhV7oNu72yn50j1CSXwHw7vvUqPgCFFYgT9K3Hicv4+BMc5CsyguIw20e/9cjt/HF7e8xLpjGemQPWPEpbfJ3rRQHLJFSAqFw/WYh6sXJg/2DP20pbNqDdu5+K1+3tR9eZ6pY4ilrPSj9uB5PvVAK5jr67AaQjQaItSKoly3TWyR5+dAAxV2a1gjujGwC/KYZQwYpFdiw+set2wUP6K48owGpeUQ9NidVCoI2e09XP8tuybOiv9eFnVdnSX/v11Y/CBXV/Wq+27HjBOOJhfW3fN
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(5660300002)(66476007)(6506007)(38100700002)(508600001)(6512007)(66946007)(53546011)(4326008)(76116006)(91956017)(64756008)(66556008)(66446008)(186003)(2906002)(54906003)(2616005)(83380400001)(71200400001)(122000001)(26005)(6486002)(6916009)(316002)(33656002)(8676002)(38070700005)(36756003)(8936002)(86362001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <382926DEEFB26E4393E8DB6021043B38@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6919
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	a5b4cf52-2ebf-4684-2d6b-08d98327666a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LiRIFbh2E/RmcZzX9inhkZLBdJTcw1vu7OyRf420p2C+wkIci4IHLv6hHfLEh8x1Ut3dw6KtpuzsQlBsx1iBEnDlibWBGsZC36NfjtGzrgrGxoJROQMAZcXcgPfHuv936B0HlkG4rOKQEZR9JaXALnfD/wK0jp3iAkwUy9TPSijWRtfEZvQLkDFaGzOL/krVzF+xNmZy7P6qNS8DNxvDMGbJUiKOS8qFF3+tvqeTYm/xJdIsjhpqOZ54iXRZqFinCci/H197L2C5minPq9wTaKiQA1AlF1iHP+AGfMU+frvkLU5pUiVYA9YJun/UcpaYrV/3l/4F/ZZU19ILRpfiNolT2F+LMxaoMegnqs6u8guTkiZOnziBKJTqeIbGJiiS3cBbuIXkj81TckZmwVG8vi3r8QI/I3p5KUYOWEQKl/6qkJlAzXXXFlo/l6Sc8sI9FHO1wjE0R8d6xul+sSgEDHbKmALrYWQaqEE04dHbtC88L09JgadilCRPC64559lagFJkJaXZ3NhCr+0Kd3rEgJsEIfItkcfKTtIQGOLY6rrQUSBTs9TDlST5IMduXgmt8LShJzRXupGq4ro8AAwAkRKnS1fQi06dw4OslF5YL52F6YJFkVq00VtGfpGYcnDkbiRTI6dmGOcy1sPxa3w7aR4ia6Pb5PU6HbGzj0K6dyRaEcvxjv1tG06jqOYpr7+A+S2xW992WrlxOewW3lJWnQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(83380400001)(81166007)(6506007)(53546011)(508600001)(8936002)(4326008)(5660300002)(8676002)(356005)(70206006)(2616005)(54906003)(70586007)(6862004)(2906002)(26005)(186003)(316002)(6512007)(6486002)(86362001)(36860700001)(47076005)(36756003)(82310400003)(336012)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 08:59:25.5650
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 381feac0-5c86-4f02-d2d6-08d983277010
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT010.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6915

SGkgSmFuLA0KDQo+IE9uIDI5IFNlcCAyMDIxLCBhdCA5OjMxIGFtLCBKYW4gQmV1bGljaCA8amJl
dWxpY2hAc3VzZS5jb20+IHdyb3RlOg0KPiANCj4gT24gMjguMDkuMjAyMSAyMDoxOCwgUmFodWwg
U2luZ2ggd3JvdGU6DQo+PiBYRU4gZHVyaW5nIGJvb3Qgd2lsbCByZWFkIHRoZSBQQ0kgZGV2aWNl
IHRyZWUgbm9kZSDigJxyZWfigJ0gcHJvcGVydHkNCj4+IGFuZCB3aWxsIG1hcCB0aGUgUENJIGNv
bmZpZyBzcGFjZSB0byB0aGUgWEVOIG1lbW9yeS4NCj4+IA0KPj4gQXMgb2Ygbm93IG9ubHkgInBj
aS1ob3N0LWVjYW0tZ2VuZXJpYyIgY29tcGF0aWJsZSBib2FyZCBpcyBzdXBwb3J0ZWQuDQo+PiAN
Cj4+ICJsaW51eCxwY2ktZG9tYWluIiBkZXZpY2UgdHJlZSBwcm9wZXJ0eSBhc3NpZ25zIGEgZml4
ZWQgUENJIGRvbWFpbg0KPj4gbnVtYmVyIHRvIGEgaG9zdCBicmlkZ2UsIG90aGVyd2lzZSBhbiB1
bnN0YWJsZSAoYWNyb3NzIGJvb3RzKSB1bmlxdWUNCj4+IG51bWJlciB3aWxsIGJlIGFzc2lnbmVk
IGJ5IExpbnV4LiBYRU4gYWNjZXNzIHRoZSBQQ0kgZGV2aWNlcyBiYXNlZCBvbg0KPj4gU2VnbWVu
dDpCdXM6RGV2aWNlOkZ1bmN0aW9uLiBBIFNlZ21lbnQgbnVtYmVyIGluIHRoZSBYRU4gaXMgc2Ft
ZSBhcyBhDQo+PiBkb21haW4gbnVtYmVyIGluIExpbnV4LiBTZWdtZW50IG51bWJlciBhbmQgZG9t
YWluIG51bWJlciBoYXMgdG8gYmUgaW4NCj4+IHN5bmMgdG8gYWNjZXNzIHRoZSBjb3JyZWN0IFBD
SSBkZXZpY2VzLg0KPj4gDQo+PiBYRU4gd2lsbCByZWFkIHRoZSDigJxsaW51eCxwY2ktZG9tYWlu
4oCdIHByb3BlcnR5IGZyb20gdGhlIGRldmljZSB0cmVlIG5vZGUNCj4+IGFuZCBjb25maWd1cmUg
dGhlIGhvc3QgYnJpZGdlIHNlZ21lbnQgbnVtYmVyIGFjY29yZGluZ2x5LiBJZiB0aGlzDQo+PiBw
cm9wZXJ0eSBpcyBub3QgYXZhaWxhYmxlIFhFTiB3aWxsIGFsbG9jYXRlIHRoZSB1bmlxdWUgc2Vn
bWVudCBudW1iZXINCj4+IHRvIHRoZSBob3N0IGJyaWRnZS4NCj4+IA0KPj4gU2lnbmVkLW9mZi1i
eTogUmFodWwgU2luZ2ggPHJhaHVsLnNpbmdoQGFybS5jb20+DQo+PiAtLS0NCj4+IENoYW5nZSBp
biB2MzoNCj4+IC0gTW9kaWZ5IGNvbW1pdCBtc2cgYmFzZWQgb24gcmVjZWl2ZWQgY29tbWVudHMu
DQo+PiAtIFJlbW92ZSBhZGRlZCBzdHJ1Y3QgbWF0Y2hfdGFibGV7fSBzdHJ1Y3QgaW4gc3RydWN0
IGRldmljZXt9DQo+PiAtIFJlcGxhY2UgdWludDMyX3Qgc2JkZiB0byBwY2lfc2JkZl90IHNiZGYg
dG8gYXZvaWQgdHlwZWNhc3QNCj4+IC0gUmVtb3ZlIGJ1c19zdGFydCxidXNfZW5kIGFuZCB2b2lk
ICpzeXNkYXRhIGZyb20gc3RydWN0IHBjaV9ob3N0X2JyaWRnZXt9DQo+PiAtIE1vdmUgIiNpbmNs
dWRlIDxhc20vcGNpLmg+IiBpbiAieGVuL3BjaS5oIiBhZnRlciBwY2lfc2JkZl90IHNiZGYgZGVj
bGFyYXRpb24NCj4gDQo+IFRoaXMgcGFydCwgaWYgbm90IHNwbGl0IGludG8gYSBzZXBhcmF0ZSBw
YXRjaCBpbiB0aGUgZmlyc3QgcGxhY2UsIHdhbnRzDQo+IG1lbnRpb25pbmcgaW4gYXQgbGVhc3Qg
aGFsZiBhIHNlbnRlbmNlIG9mIHRoZSBkZXNjcmlwdGlvbi4gVGhlbiAuLi4NCg0KTGV0IG1lIG1v
dmUgdGhpcyB0byBuZXcgcGF0Y2ggaW4gbmV4dCB2ZXJzaW9uLg0KPiANCj4+IC0tLSBhL3hlbi9p
bmNsdWRlL3hlbi9wY2kuaA0KPj4gKysrIGIveGVuL2luY2x1ZGUveGVuL3BjaS5oDQo+PiBAQCAt
MTUsNyArMTUsNiBAQA0KPj4gI2luY2x1ZGUgPHhlbi9wZm4uaD4NCj4+ICNpbmNsdWRlIDxhc20v
ZGV2aWNlLmg+DQo+PiAjaW5jbHVkZSA8YXNtL251bWEuaD4NCj4+IC0jaW5jbHVkZSA8YXNtL3Bj
aS5oPg0KPj4gDQo+PiAvKg0KPj4gICogVGhlIFBDSSBpbnRlcmZhY2UgdHJlYXRzIG11bHRpLWZ1
bmN0aW9uIGRldmljZXMgYXMgaW5kZXBlbmRlbnQNCj4+IEBAIC02Miw2ICs2MSw4IEBAIHR5cGVk
ZWYgdW5pb24gew0KPj4gICAgIH07DQo+PiB9IHBjaV9zYmRmX3Q7DQo+PiANCj4+ICsjaW5jbHVk
ZSA8YXNtL3BjaS5oPg0KPj4gKw0KPj4gc3RydWN0IHBjaV9kZXZfaW5mbyB7DQo+PiAgICAgLyoN
Cj4+ICAgICAgKiBWRidzICdpc19leHRmbicgZmllbGQgaXMgdXNlZCB0byBpbmRpY2F0ZSB3aGV0
aGVyIGl0cyBQRiBpcyBhbiBleHRlbmRlZA0KPj4gDQo+IA0KPiAuLiB0aGlzIHBhcnQNCj4gQWNr
ZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gKGFsc28gaWYgeW91IGRl
Y2lkZSB0byBtb3ZlIHRoaXMgdG8gYSBzZXBhcmF0ZSBwYXRjaCkNCg0KUmVnYXJkcywNClJhaHVs
DQo+IA0KPiBKYW4NCg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:03:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:03:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198902.352636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVUd-0003Sc-0k; Wed, 29 Sep 2021 09:03:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198902.352636; Wed, 29 Sep 2021 09:03:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVUc-0003SV-T5; Wed, 29 Sep 2021 09:03:10 +0000
Received: by outflank-mailman (input) for mailman id 198902;
 Wed, 29 Sep 2021 09:03:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVVUb-0003SP-D3
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:03:09 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 074228cf-8055-43ad-9131-69ea4828a387;
 Wed, 29 Sep 2021 09:03:08 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18T84lhI019579;
 Wed, 29 Sep 2021 09:03:03 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bcg0n0tr8-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 09:03:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6242.eurprd03.prod.outlook.com (2603:10a6:20b:15c::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 09:03:00 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 09:03:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 074228cf-8055-43ad-9131-69ea4828a387
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dtALfyxxNmaU/48YZTJxNw6Ysoaj2QQdD3N0nwL4l+r/7H03WATekiGlujRT0fDyoRF0scVV3mJFma2MVArkeGMokS8L3SPdglXzE1OXKfAR+HDpX0J055e/WAJqXxC+EFUZeiJ3MKQyR0aa8HT8HoIryk72XQphdA8P8Y2W6oB/VGz/K1jwDTsAifAWR6tMh0FmRWb5o/W5UVX/2M+zPmbsKbIpWO5XVn0w6hbHQEcCDkr0szc9qHbXF9G0oq47tft0qFSkIslTHNigQGNab7Oi7sNcCTxzzKYPvnnQPbl+P48lIZc+2Kj3ozvbtyglWqOIcvkBZ1YC0Z9Ykk2+LQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=THBeAc5jY29gXIe48ZUd/K3VVnp852+rISS9da95/mo=;
 b=kX8Oq8+tmd3RMbD7KFkWIYExALk050C32M6Zims69o/SugRrbS1whlPw+Yk4rbk0CZu0FT5frckIvPJy4swRfWV/5mwdd9XWYwtOdhR1H4HeQqWdsUuElnpw2JtBTH5B1l6T88+AN9JszuXbDeCJlbKvwPV8CpkUPkUxmHKAYL9R//8jrCwUjvw31aeHcWy3FByi9eqw2pJ+izDjmnzib5MrvI5cYJOiNdFqOklvNkpNTcH0YvCMTZ/8Fz/MxD0LkFJfcpA0hU2LwvdZynJTQ8FxdJMcbg8+lfBnEm7X/gWCbrrnEQG+cDE/ntgVcq0HX6Lo1+LVzwjO4N2LIoJ8mQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=THBeAc5jY29gXIe48ZUd/K3VVnp852+rISS9da95/mo=;
 b=GchXwt1mV70i5S6C6zZrNwl9wQkUeb+xR73hPVq0RAslSFBaGN89rHtgkB1DhoqJxnIU0myBpnaSTj+wOTP1jvpIPy58/3yv4NtoOOqYlu60H1EjxtptPFS4GHXj4ICjyPIg5Cr1NTK/oMenvki3I6zKSuY6CenBfW3zmMERvLiSiQN8nwCuDFP6q8O5zxgs+i1spaE2b6H0nrxUErKoa3kVl0InpMA+/zN2zsKi41V3vJwa/OjbO9amZWWIuNi8nmLTKA3pHgGVsDXvfb5jYHxv/qmJ09FJJL80/Sx0K/C5jFZ5g6YpUOdJiygw6w9FFfIjP9uZp1jzt3xe4v9g9w==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Michal
 Orzel <michal.orzel@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyAgAFQpAA=
Date: Wed, 29 Sep 2021 09:03:00 +0000
Message-ID: <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
In-Reply-To: <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 6ed6eaaa-edd7-488b-b33e-08d98327efff
x-ms-traffictypediagnostic: AM0PR03MB6242:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB6242C621B606727219525E47E7A99@AM0PR03MB6242.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 GMqyo3Ef33T7nrF0hUrMxnWOX62TTyXcy64C7j1956Trh3bRlWzeinUCf8KXJG9CZmjwWcHt5Hbmx1tAd2rl3M+VT99Ee8rPcQ2KpUbl23sY/iJ37isUELEB2Db7e+T7bqv/FTconXlEOILeHP3lxBWvf2tEZgJyoCS44rf2nJAu3hfPMfjMbxkus7z270PcyQpufp8J+2VG09PzLrCDKyeVyYYHqgd3GZ77X9SoWydpeoxU0vOI+7C3OOdNYe2qNjF6QQtvrgH+pXAJ5iUgiv65yRFwmsVbHe1iiuzDo80JbKdpfJW7cRdvDpTPxX+U+1c5DkL0/FJ+J8XezC21Sol9s9OZg0f3LnmoZqZACkss22IwD2EfKnndkqE8CvOQHvW4Hq7xFJgcyCPtfu1z9CAlCDgWofgBI5fDwfGYLdOXjiuscBL3HtACpmKfcg7zmBluYnjDpODCATsSbnJ+YYx1iZAjREpDjLc3UDzcUeRHdep0wNafn+GAsUvvCpn3M7ljcF6jlQh9QgwmUfVCY9D5ccC2EeWbuypa6EzU3WYhkHHU5GfKE3TbWpjCmjrNmF4jREqYDjspCS1i33s3malc5Dl4bTw3YDPMmloBpKot7Es0+GgcqnDhLb9KrhuVhBupF7PoSDfEXrXhRMpcEHSlJhcqRQpZMhBgVlUIsuPajwlDyJs2HVIPtmN2rXtL+aLpeeqqmhthtNQoaQUTUe+ruzHYm9if4TbS9eInEWLgZ+Hjdtg1goMdNUaVMUQN
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(86362001)(6916009)(26005)(83380400001)(55236004)(36756003)(4326008)(316002)(2616005)(2906002)(5660300002)(71200400001)(38070700005)(186003)(8676002)(66446008)(107886003)(31686004)(6512007)(54906003)(64756008)(508600001)(66556008)(66476007)(76116006)(8936002)(6506007)(91956017)(53546011)(31696002)(66946007)(6486002)(122000001)(38100700002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?WlRHSGpwVWpNaFdrTWFNYVdnU2d6ZWlCUjhodUtuTFAxTHVwVVQ5QTB1WVRH?=
 =?utf-8?B?enBvT2xva1RUS3gydVlZRFErS0VrYkFtQldpUkhrSmNrek9jS3ZMbm1IbzB1?=
 =?utf-8?B?c0NZZm1IaFZCRENxZjVIWElVL24rYVdTaWFUeDBxNWFCMTg2RzRVdEF0bDhP?=
 =?utf-8?B?NmordlltaTZINXdORVRXcVhiMHZaUkE5OW95aGdzZTNuTUFFUGRaZ2Z4YjYr?=
 =?utf-8?B?MUtNQ3NNQldnM1hwczU2Wk8zTHU2ZVhEejBIN1IzTkw0Skl6Uko4S2NCNnlK?=
 =?utf-8?B?VGFabHNkNDFVQUxjK2Y1TWJvdkFCbUdCRjMrcXRFWXVqZzBEemFQaXJZMG9p?=
 =?utf-8?B?b3dQejRZQ3IxclV3V3FrWGc4eHU5d1BRUjhlN09PNnUvR1ljZjF2L3p2U3Bv?=
 =?utf-8?B?TzA5NnUyaVZYczZoMngvV05Sa0F6MnBiQzdUaU5TV05sbjJyeWVWZnhYVEtm?=
 =?utf-8?B?RzkyNUFQdzhMcDRrU25keHhYd2ZuRFI0T2pjNGVPbCtaZU53YTJwVHRndm00?=
 =?utf-8?B?MERNbmlaTjY2dWVLdnlXVUFwdHdMRDNFVjNCWnNOR0xJSGMzVHAzN2RYTlZ5?=
 =?utf-8?B?bWIrdEpvWS9QaTdQa3NVSGlSRGVsaGlVWDNId25nTmlPamRVTlpNb1kzRlNS?=
 =?utf-8?B?UmlvU05uUWxMQkxhK3FvaVNFWE5tZHZEN1Q2a2RodHFYTFM4a3ZPTlN5dzNy?=
 =?utf-8?B?NlZvYm40YzFSWC9HekQ5aTV1UEdmU2toZjZxUHpDYTVnNzlZMmVKcW1PQXdl?=
 =?utf-8?B?MWJxSHkwWmlhd0tEaUhCK2R5aGJJYXZoS05zYVV3czhzbm5NWVRMT2xnZG12?=
 =?utf-8?B?YTFnWEhka3Y0SGI0aE9tMUVTNEwwTmEyTHpCREJZSVo5cXdOemtvRjVxbVB2?=
 =?utf-8?B?cS9uNUw0b0xTMVZZYWhKbnVTR0NMdk04YmFrRG5QSXRnUi9iMlFEVCtFdjln?=
 =?utf-8?B?aGRtZmVpcWVidm93WDRvSmg1dExDclhUNlZ6TUJlTmdkbHFjbWV2MTlKM2ZS?=
 =?utf-8?B?ZDZHL1U1dlozeWhRNWFzRmV5U0ZHMkFkQmJTYkFxczZNdlBmaksvUnJlYkpl?=
 =?utf-8?B?WGRIZjkvTTdTQ3NyYmhXVVAyRW5tdFQ0WjhtaFhkZ2FKTmtsRjBiOHJaTmFy?=
 =?utf-8?B?MGNleE1YRDFGSEJMVmFZMnhQbURYMFJBYW9DNndPUGlDVDFpWUNTWFB5NDg3?=
 =?utf-8?B?aE5GVUw4VlNMaFQzc3FRK3I5NGlpcDV3ajAvL3ZURzVkMmRVclY2TFdFVDlC?=
 =?utf-8?B?NWVUOXJrNDduT3oxK2Fwc3RoaDR2RWljeE9RVnFDV29semlxWWNaM0RBbU9X?=
 =?utf-8?B?dXZZNmt3M2NQaHNiS3FEdUlPTXZieFM1ZkpPVU1BY2RJWHlNaDl3bkRjNmxa?=
 =?utf-8?B?Q2dSdWpNbGVncGt5QkhMeWhzQnZKQWh4L2REN2VwUmNkWkVLQVZvNFYxTE5Y?=
 =?utf-8?B?bndFQ2VzcytITG9wQUFYdzhJOE9JYVU4am5PZ1VwUkgyWFI2R3dPSkowbzFk?=
 =?utf-8?B?ZXlURGRGZHU1Qi8xb1BGUzJQSzc3TFVudlJOM3ZjUjBnUVFwdDlOaXl0NUJY?=
 =?utf-8?B?bEpzaVVwZjJ3UFJGM1FyQUV1OHJ0eU9QVHhFenR4V0pOWjNobDNqWlErNUs2?=
 =?utf-8?B?UFdwU3hhWjRyUUt4TS8xeXFaY0M4T3FiWXZiMUoySE1qa3U3ODU3TzZrdG9U?=
 =?utf-8?B?N3MxMUhTeWdtOG95bTRaOHM5a2NiSGhwT1oyeUg5SXhPbWJkUkJvelA2cXFK?=
 =?utf-8?B?N283bjU1ZmJLSW5xaGcwSk5BOFNHak9Rb1p1c2d2Tk13R0IxQVpaVENvMEg2?=
 =?utf-8?B?cmgzcmlTbWQzOEZRWkVCUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <87860C08B85D9742923CFAAF1CE02867@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ed6eaaa-edd7-488b-b33e-08d98327efff
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 09:03:00.1577
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sRlIidDIiXlyO0zX5SateYioKuZX6w0jP+GdTs1EbKrvFqmIDIrLo5JdT7O75qwlR90HIs4yzWFIHBR5ALvDAqAOm+6S6rVx6YQXbkuEqzbSsnGQnJnwGBJbmvaeJuF0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6242
X-Proofpoint-ORIG-GUID: -n36QO8qPpBaxIrifhObUOFpCxoocXa7
X-Proofpoint-GUID: -n36QO8qPpBaxIrifhObUOFpCxoocXa7
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_02,2021-09-28_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0
 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 adultscore=0
 bulkscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 spamscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290054

SGksIEphbiENCg0KU29ycnkgZm9yIHRvcCBwb3N0aW5nLCBidXQgdGhpcyBpcyBhIGdlbmVyYWwg
cXVlc3Rpb24gb24gdGhpcyBwYXRjaC9mdW5jdGlvbmFsaXR5Lg0KDQpEbyB5b3Ugc2VlIHdlIG5l
ZWQgdG8gZ2F0ZSBhbGwgdGhpcyB3aXRoIENPTkZJR19IQVNfVlBDSV9HVUVTVF9TVVBQT1JUDQoN
CmFzIHRoaXMgcmVuZGVycyBpbiBzb21ld2hhdCBkZWFkIGNvZGUgZm9yIHg4NiBmb3Igbm93PyBJ
IGRvIHRoaW5rIHRoaXMgc3RpbGwNCg0KbmVlZHMgdG8gYmUgaW4gdGhlIGNvbW1vbiBjb2RlIHRo
b3VnaC4NCg0KVGhhbmsgeW91IGluIGFkdmFuY2UsDQoNCk9sZWtzYW5kcg0KDQpPbiAyOC4wOS4y
MSAxNTo1OCwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+IE9uIDI4LjA5LjIxIDEx
OjE3LCBNaWNoYWwgT3J6ZWwgd3JvdGU6DQo+PiBPbiAyOC4wOS4yMDIxIDA5OjU5LCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+PiBPbiAyOC4wOS4yMDIxIDA5OjQ4LCBNaWNoYWwgT3J6ZWwgd3JvdGU6
DQo+Pj4+IE9uIDIzLjA5LjIwMjEgMTQ6NTUsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3Rl
Og0KPj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gvcGNpLmMNCj4+Pj4+ICsrKyBi
L3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+PiBAQCAtODMzLDYgKzgzMyw2MyBA
QCBpbnQgcGNpX3JlbW92ZV9kZXZpY2UodTE2IHNlZywgdTggYnVzLCB1OCBkZXZmbikNCj4+Pj4+
ICAgICAgICByZXR1cm4gcmV0Ow0KPj4+Pj4gICAgfQ0KPj4+Pj4gICAgDQo+Pj4+PiArc3RhdGlj
IHN0cnVjdCB2cGNpX2RldiAqcGNpX2ZpbmRfdmlydHVhbF9kZXZpY2UoY29uc3Qgc3RydWN0IGRv
bWFpbiAqZCwNCj4+Pj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4+ICt7DQo+Pj4+PiArICAg
IHN0cnVjdCB2cGNpX2RldiAqdmRldjsNCj4+Pj4+ICsNCj4+Pj4+ICsgICAgbGlzdF9mb3JfZWFj
aF9lbnRyeSAoIHZkZXYsICZkLT52ZGV2X2xpc3QsIGxpc3QgKQ0KPj4+Pj4gKyAgICAgICAgaWYg
KCB2ZGV2LT5wZGV2ID09IHBkZXYgKQ0KPj4+Pj4gKyAgICAgICAgICAgIHJldHVybiB2ZGV2Ow0K
Pj4+Pj4gKyAgICByZXR1cm4gTlVMTDsNCj4+Pj4+ICt9DQo+Pj4+PiArDQo+Pj4+PiAraW50IHBj
aV9hZGRfdmlydHVhbF9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0IHBjaV9k
ZXYgKnBkZXYpDQo+Pj4+PiArew0KPj4+Pj4gKyAgICBzdHJ1Y3QgdnBjaV9kZXYgKnZkZXY7DQo+
Pj4+PiArDQo+Pj4+PiArICAgIEFTU0VSVCghcGNpX2ZpbmRfdmlydHVhbF9kZXZpY2UoZCwgcGRl
dikpOw0KPj4+Pj4gKw0KPj4+Pj4gKyAgICAvKiBFYWNoIFBDSSBidXMgc3VwcG9ydHMgMzIgZGV2
aWNlcy9zbG90cyBhdCBtYXguICovDQo+Pj4+PiArICAgIGlmICggZC0+dnBjaV9kZXZfbmV4dCA+
IDMxICkNCj4+Pj4+ICsgICAgICAgIHJldHVybiAtRU5PU1BDOw0KPj4+Pj4gKw0KPj4+Pj4gKyAg
ICB2ZGV2ID0geHphbGxvYyhzdHJ1Y3QgdnBjaV9kZXYpOw0KPj4+Pj4gKyAgICBpZiAoICF2ZGV2
ICkNCj4+Pj4+ICsgICAgICAgIHJldHVybiAtRU5PTUVNOw0KPj4+Pj4gKw0KPj4+Pj4gKyAgICAv
KiBXZSBlbXVsYXRlIGEgc2luZ2xlIGhvc3QgYnJpZGdlIGZvciB0aGUgZ3Vlc3QsIHNvIHNlZ21l
bnQgaXMgYWx3YXlzIDAuICovDQo+Pj4+PiArICAgICoodTE2KikgJnZkZXYtPnNlZyA9IDA7DQo+
Pj4+IEVtcHR5IGxpbmUgaGVhciB3b3VsZCBpbXByb3ZlIHJlYWRhYmlsaXR5IGR1ZSB0byB0aGUg
YXN0ZXJpc2tzIGJlaW5nIHNvIGNsb3NlIHRvIGVhY2ggb3RoZXIuDQo+IFdpbGwgYWRkDQo+Pj4+
IEFwYXJ0IGZyb20gdGhhdDoNCj4+Pj4gUmV2aWV3ZWQtYnk6IE1pY2hhbCBPcnplbCA8bWljaGFs
Lm9yemVsQGFybS5jb20+DQo+Pj4+PiArICAgIC8qDQo+Pj4+PiArICAgICAqIFRoZSBidXMgbnVt
YmVyIGlzIHNldCB0byAwLCBzbyB2aXJ0dWFsIGRldmljZXMgYXJlIHNlZW4NCj4+Pj4+ICsgICAg
ICogYXMgZW1iZWRkZWQgZW5kcG9pbnRzIGJlaGluZCB0aGUgcm9vdCBjb21wbGV4Lg0KPj4+Pj4g
KyAgICAgKi8NCj4+Pj4+ICsgICAgKigodTgqKSAmdmRldi0+YnVzKSA9IDA7DQo+Pj4+PiArICAg
ICooKHU4KikgJnZkZXYtPmRldmZuKSA9IFBDSV9ERVZGTihkLT52cGNpX2Rldl9uZXh0KyssIDAp
Ow0KPj4+IEFsbCBvZiB0aGVzZSBjYXN0cyBhcmUgKGEpIG1hbGZvcm1lZCBhbmQgKGIpIHVubmVj
ZXNzYXJ5IGluIHRoZSBmaXJzdA0KPj4+IHBsYWNlLCBhZmFpY3MgYXQgbGVhc3QuDQo+Pj4NCj4+
IEFncmVlLg0KPj4gKigodTgqKSAmdmRldi0+YnVzKSA9IDA7DQo+PiBpcyB0aGUgc2FtZSBhczoN
Cj4+IHZkZXYtPmJ1cyA9IDA7DQo+IE92ZXJlbmdpbmVlcmluZyBhdCBpdHMgYmVzdCA7KQ0KPg0K
PiBXaWxsIGZpeCB0aGF0DQo+DQo+Pj4gSmFuDQo+Pj4NCj4gVGhhbmsgeW91LA0KPg0KPiBPbGVr
c2FuZHI=


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:03:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:03:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198903.352646 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVUh-0003kj-BG; Wed, 29 Sep 2021 09:03:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198903.352646; Wed, 29 Sep 2021 09:03:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVUh-0003kc-8F; Wed, 29 Sep 2021 09:03:15 +0000
Received: by outflank-mailman (input) for mailman id 198903;
 Wed, 29 Sep 2021 09:03:14 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NQqk=OT=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVVUg-0003SP-Fe
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:03:14 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 7889e37e-e89e-4f29-851e-9ebf35dd9aaf;
 Wed, 29 Sep 2021 09:03:13 +0000 (UTC)
Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com
 [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-182-Rna-RbHZOvaQvXp5YvfnTw-1; Wed, 29 Sep 2021 05:03:12 -0400
Received: by mail-wm1-f69.google.com with SMTP id
 y142-20020a1c7d94000000b0030cdc76dedeso2424447wmc.5
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 02:03:12 -0700 (PDT)
Received: from [192.168.3.132] (p4ff23c3b.dip0.t-ipconnect.de. [79.242.60.59])
 by smtp.gmail.com with ESMTPSA id
 t11sm1548498wrz.65.2021.09.29.02.03.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 Sep 2021 02:03:10 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7889e37e-e89e-4f29-851e-9ebf35dd9aaf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632906193;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=7W9Fc0fF86WyXYecPV/lI8dmSptGKk8vSEWO1VbX84s=;
	b=VgUT5Wkwz6V88ZQWns3PibHMln4HGhgp//Lqv9xtWWLTUU31B+jaqn1HXLjgQC/QydaJh8
	NDkadZGDpMr/7/xCagBYELPHCNxaXtcFe2exkJT+esc+RwifmAL59hZ6W7hNVbT8JgYUIs
	S8u2Og2LtVssz/7JezgZKEYnn+I6IDY=
X-MC-Unique: Rna-RbHZOvaQvXp5YvfnTw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:from:to:cc:references:organization
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=7W9Fc0fF86WyXYecPV/lI8dmSptGKk8vSEWO1VbX84s=;
        b=HPwGRsODjLbesCMLqTQU311gHgqkyT5SbKmMjEPayekjV6ewl01CmsKrLPEkRbgWuk
         T6tkPndx6eCU1yTYBOjPROwOnCYx5HWfUn6eeCh2UsUr2ymMASSMZvqOuuw9bNDYM1q9
         knOW0NlcgCuNoTGuSpH8H0qAf+x+/f207YjIil7yQmb45IXAWdiaAcWfe6csO5tMCRsi
         eVV/2UCZfttvZrjtb+ghxC7ITpMcHKkZhjldyvn75FARczH8uX0iJ15ydXJUIoY3FV4O
         5rbpLLdEQkIpnGm0gYhJnvMW98vovgo4o3Em0+TmDaiFTl9CYzSyS93x9yyChl8p5nNU
         vhUw==
X-Gm-Message-State: AOAM531ja3Og2SVKkw6eZtLmm9kGIqXqWrIWqugZbN4CoQtRJLxkICG7
	IYV/Z5t7rlgAaclLMtFLXaHfcSXbS2Z9CgeIkwnkCLe2kXerMyW5mZOaumGhMOhhyJSCq158z+g
	alEkhqMssWtQuSs+5Xzm0RMlggTU=
X-Received: by 2002:a7b:c219:: with SMTP id x25mr9238902wmi.125.1632906190953;
        Wed, 29 Sep 2021 02:03:10 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJyAVvczz8jUTCdMNZ0BC5qvezoqKUBOcf5OajF0CWmKLmPAPf0K3OR8WEfpMUG5vsV5dclhgg==
X-Received: by 2002:a7b:c219:: with SMTP id x25mr9238872wmi.125.1632906190770;
        Wed, 29 Sep 2021 02:03:10 -0700 (PDT)
Subject: Re: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
From: David Hildenbrand <david@redhat.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Juergen Gross <jgross@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, "Michael S. Tsirkin" <mst@redhat.com>,
 Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
 Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
 Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
 Mike Rapoport <rppt@kernel.org>,
 "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
 xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org,
 kexec@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-3-david@redhat.com>
 <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
 <bfe72f46-9a0d-1a87-64bd-4b03999edd1e@redhat.com>
Organization: Red Hat
Message-ID: <e9a230f9-85cb-d4c1-8027-508b7c344d94@redhat.com>
Date: Wed, 29 Sep 2021 11:03:09 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <bfe72f46-9a0d-1a87-64bd-4b03999edd1e@redhat.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 29.09.21 10:45, David Hildenbrand wrote:
>>
>> How about
>>
>>       return a.mem_type != HVMMEM_mmio_dm;
>>
> 
> Ha, how could I have missed that :)
> 
>>
>> Result should be promoted to int and this has added benefit of not requiring changes in patch 4.
>>
> 
> Can we go one step further and do
> 
> 
> @@ -20,24 +20,11 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
>           struct xen_hvm_get_mem_type a = {
>                   .domid = DOMID_SELF,
>                   .pfn = pfn,
> +               .mem_type = HVMMEM_ram_rw,
>           };
> -       int ram;
>    
> -       if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
> -               return -ENXIO;
> -
> -       switch (a.mem_type) {
> -       case HVMMEM_mmio_dm:
> -               ram = 0;
> -               break;
> -       case HVMMEM_ram_rw:
> -       case HVMMEM_ram_ro:
> -       default:
> -               ram = 1;
> -               break;
> -       }
> -
> -       return ram;
> +       HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a);
> +       return a.mem_type != HVMMEM_mmio_dm;
>    }
>    #endif
> 
> 
> Assuming that if HYPERVISOR_hvm_op() fails that
> .mem_type is not set to HVMMEM_mmio_dm.
> 

Okay we can't, due to "__must_check" ...

-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:09:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:09:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198917.352657 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVaT-0004xv-0m; Wed, 29 Sep 2021 09:09:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198917.352657; Wed, 29 Sep 2021 09:09:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVaS-0004xo-U2; Wed, 29 Sep 2021 09:09:12 +0000
Received: by outflank-mailman (input) for mailman id 198917;
 Wed, 29 Sep 2021 09:09:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVVaS-0004xi-6B
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:09:12 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7cb8d5ce-912e-4d9f-9513-04b76876399f;
 Wed, 29 Sep 2021 09:09:11 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-29-Up4EDLqoNSqxn5C8R3HwZw-1; Wed, 29 Sep 2021 11:09:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5597.eurprd04.prod.outlook.com (2603:10a6:803:dd::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 09:09:05 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 09:09:05 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR0301CA0009.eurprd03.prod.outlook.com (2603:10a6:206:14::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 09:09:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7cb8d5ce-912e-4d9f-9513-04b76876399f
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632906550;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EAo9xfYpnf8kmt+M2iJxEWkckj4VOOa8jqFTnO/RFFE=;
	b=EaVPFE/J+Wz+0Ci0uJgI45iwwEtN/W/xIrY+IZyZZAu1qwSA2T+LlVy89g9WoOM7Cr00vJ
	89hfqaRI+kL5VGrrv+PM4xrTuw+KdTo9VXPGo1lFVUwCKPnzmw+D2oaxrQuPW3lGnRSsxp
	OuDFNuMNMpx3r7fA1UQEhSVDTBIUZ1Y=
X-MC-Unique: Up4EDLqoNSqxn5C8R3HwZw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kF35Gv0J4Rd+qhATvE27RsNhKCBh7JmDiEnpkQcNyBhXy2QJpOwXzkEGSAT5LPnjmBHbflFZI/V1JMmOZRuz6TgJBQKlrBtsnXauMPWDM/KTPOeULa7I5DTgJDNDEFMlpBWOV57gORFhLL2BAteVQCUQ51uzGiFMsdFQvwujTVrNpqnTjchuYblV33jhRzkqbZllDERkTM/4ucvWRvyuWX+RMUjUP0U8w42AeeSTU9e0W735lVmn8BzPIiJv1Zi2M5Ef7C/QvTxA/TQPukYppRWfQPSicyT4fHdfqGhIJPBQvqRMORtf1OV/0yJ3Jsq7oSLCCXqhfRulVhwe5BvplQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EAo9xfYpnf8kmt+M2iJxEWkckj4VOOa8jqFTnO/RFFE=;
 b=EsLn6ZDHf6MwimQJ9e+aLMm7R33ehnTws2YtVkO1nClQFhGGNx/WwYg6y4fxnw5Bxdp1LBzBVQtdRyiG0YuwNPkofyIcdAC969MxLirKj0v/8zl3v1oAuuR+V0u48+iGLTRwhd6AIsEDF1s73ef25vBu+JpZ3iSAClMDITTS2POUCHBEU8QrwfVce8FvDfeJeyrDp6vG+kq3M+Guq/2afL6E9hVWbzHySfWXdIjIFwor1OasR9ScQQMgWBC0QQMf5jxGYEaoseQ/nB9qQYXZl2qKyjugizABvJF4V0oek7iCye9/yVZZSt3adPFGlST8unLFzQs3st1gUcXKo1a8Zw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Michal Orzel <michal.orzel@arm.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
Date: Wed, 29 Sep 2021 11:09:03 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR0301CA0009.eurprd03.prod.outlook.com
 (2603:10a6:206:14::22) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b8839526-1097-4b9d-b891-08d98328c9d0
X-MS-TrafficTypeDiagnostic: VI1PR04MB5597:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB55972F2E4A1A381BFA38C575B3A99@VI1PR04MB5597.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yiEiGZMeT69ily8CaczSbyVZd5T/DD0Vx9XFgRMSq7z/PrjYsWQdRG/vV0BFa3qPe0Zyh+vw/QecRoGg7DUiAa64lptiXnWlnpr6pz+RbSQ3V5g58O7EsLwSjArRVZfYxad1XW1XdsBJe73Rk2vFDWc2fq+Ry7PIGFwlQA7cuj/tHWzDA7FYdp8Z+7Pv25Jea+8sFt+ePY8KNko6cF9mTZM0axUWom4C+EcBqilYSqVennO3/buYoCWS07UV6GMHQx41v0ZnA75dO5eLiVF5SlW1UwfKgkwGRp+73JXQud7yiA3lfx8VWUhQbpkPB/qiDzHP4epbTcp6xcD5B/xBw3ZN2q9JwSQDrElWE9+ITqyKyunkem54ZhGE6+UCHKioTNSuoM3TJF9c1EIlFfJiUAWb4miBS7hjzxGZQ9r5K2EmQTQz+8tNoiNEpIimueMUXIEgU7l50EihwKvMjAtOy4QmJNl92xpEwfhwefTn9l/ue0s9o8V5u+5h7XJj0KID0lAly3Zi0ONGKBfZwKdJs7BWNKxq3SJMHS7LvFhA9gDV4eF6NFSrcHmlkhgpsjupZptztMxMXBZINGiF+9vLTz+Tas4iEtkfgQLFNFlZw4Duunx/jDbH60MuQnmzdsn5mMRNlJY0Rdw/M1j3/kJF5IRmRkIjz9RgL9C926aDteBMAQcY9zdYkLrtSt4YlIbMOUXNI+dNI+2PRoMoSNjdWGmM8+5IKmIbB0z8vgJJnrs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(7416002)(86362001)(316002)(186003)(38100700002)(956004)(4744005)(2616005)(2906002)(36756003)(5660300002)(8936002)(54906003)(66556008)(66476007)(66946007)(16576012)(31696002)(8676002)(26005)(53546011)(508600001)(6486002)(6916009)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TkZVY0l0TFJuNUVRMFhPYjkvMjlhcFh6QlNLSEhZNGQxbG00dG50bXJSV2Q4?=
 =?utf-8?B?ZjF1b1NUa1JCdE1wcWNFT2tuNlRZSXlZS0c0UEo4Y2lQdkg3WVloVTFoRmMv?=
 =?utf-8?B?amRWQWJ3eGtGL3lSeGxEVTZvMThoUjBzOWhrSXhYdDBDWWhXY3NSTEUxMVY1?=
 =?utf-8?B?VERsQ3prOXUyOVdWVjBabi9rcU45c2VZNCtlTE55c1FTOWlrcms3b3AvenN4?=
 =?utf-8?B?VFpDN2ZPTW5oVG9qVVh0Q0J2MG9acGMvMGIrODJ2cjNTOUZ6Z0RDZTdDZWhC?=
 =?utf-8?B?eXovc3JDZDJ1NDREanhkZElqT0RseEJMcDBVYzBYN21XcG4welgvVlhoc2hh?=
 =?utf-8?B?SERTeER3WWlvb3RoOEF2NEdSYTBXQUl6YXhBci9VQlFqQ21uSzRQakRpaFVB?=
 =?utf-8?B?RzdHTWc0LzE5WWRINGNKYzRrZDhMN2lEdEFwbVpsQXFwYW16a0R6eGJhRGVY?=
 =?utf-8?B?VC9XdlBHRUQ3MFRCaytONEwzeTdOSnFlNWl0TUlMZ0dib3prNEkvSWllK1Az?=
 =?utf-8?B?Qk5PcXJIK1JPK29aVUY4TVNzZVFRS29xeVI4Z1hZQ0EyZllRaFF6Q2JwdFRo?=
 =?utf-8?B?OHVUV2dPa0hlenRYdW5hNE4vQWFyS1kzeGsrcUVFa0pjMEVXSVkzblk3RVo0?=
 =?utf-8?B?VENTK3V2d2c2SVNOS0xBRnljdlBmckJHeWppYlJOVDE1VFBOQStnRmFKVWw5?=
 =?utf-8?B?QUhjZW0wdjI1VFphUk4xN2hwL0ttQ21oSkNLdVliMVp1OFVZMlF2YTgzWFlJ?=
 =?utf-8?B?UVZkMXgzRHQvVm00akd6KzJjQ2N2L09XZkpybTJoYmNvVDBCUzFNTndTYXFG?=
 =?utf-8?B?RWQ2eHlTblpGV1ZadDJ4amdCNHIyYXNjKzFCeDhPbGU1Vmw0aWU5dDRzZkM4?=
 =?utf-8?B?SE03eUg3Q3Vld2R5K3V3c0w0UnhYOHVSRHpZZCsvMmNzS3NuS0NEMFliSVVr?=
 =?utf-8?B?V0hsZVZyMm9SRjZXTFRVUFREYkdjaGZlaGs3Q0t5SmVFbktWdWtXZjBncmxD?=
 =?utf-8?B?aDhLcFowWjNiWEZ6LzEraGZxWkJTbk13SzB1Unh4RCtkd0o1Q1hCYWRJUFdY?=
 =?utf-8?B?dWhSYXQ0WVdvYm9LZ0hvcmplR2hJSUhFNDN4K3N3c3ZTRU91NUYvUTEzTVpJ?=
 =?utf-8?B?czdNckhsaWs5U2tCSU1wcUlOUHBhb1dBTEMzRERXR3J1N2c5aHIwanZNbTBI?=
 =?utf-8?B?Q3liM3p0QnRlcS9WSE9BQ0JQd2srWUpQVzVhRDZ2MzRYYW45bk55U3BMRUNY?=
 =?utf-8?B?KzNCVHZsOVBaenB3N2FkT1NsQjYwdmRSbXRlajlRWkI5WEc0bjltcEpVaFkw?=
 =?utf-8?B?NlIwNmpGb1pjQll2eThkenZuTVlXRjRtUjFsVlVsVHlnaGluNS9yUU02VXl4?=
 =?utf-8?B?OFFEOXJRUWlIbi9CZGM0K2lPYTlNL05tRWZoSm5EOWRoc2YrMFZmN1lWV2Ey?=
 =?utf-8?B?YjNqMkdEZURGbW56L1FUWDB5ZTZmYTUzMXlzOS93UzNIVjBkTndFRGUxVDQ0?=
 =?utf-8?B?dEpwMFhtbzJ2TktLSFlnSWJtVElMbXpySW1hTjdKNmtsTkpqNDM2WlRRRWpP?=
 =?utf-8?B?Q0x4TXgyWlU5Wk0vODhaVTBqZjlRYjZ2ZHJpN1JhZHU0bE9SNnNUT2k4U2dB?=
 =?utf-8?B?QmZPb1J0L0gzQ3FqalNHR1llSzJoL3dlL0hDVEZiV2dXeWdrRjRzNmU5Q2pj?=
 =?utf-8?B?YUpFbmMzekdtY1RMbzJEQngzeUVVUms1cmlzMmJzU1hWYUVBQnR6OVJKZlhH?=
 =?utf-8?Q?gqPVIccQbt5fv5aHX12T2oozHvnr1joSeCbtKJs?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b8839526-1097-4b9d-b891-08d98328c9d0
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:09:05.8145
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yonnvQ7w5OtUREUZrPznNoyYtVtQ3LljdBOYz81KiA+KInLqZxZt0JEZDxUnqIGSP9lhtFVLzzozd+IR6CvCxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5597

On 29.09.2021 11:03, Oleksandr Andrushchenko wrote:
> Sorry for top posting, but this is a general question on this patch/functionality.
> 
> Do you see we need to gate all this with CONFIG_HAS_VPCI_GUEST_SUPPORT
> as this renders in somewhat dead code for x86 for now? I do think this still
> needs to be in the common code though.

I agree it wants to live in common code, but I'd still like the code to
not bloat x86 binaries. Hence yes, I think there want to be
"if ( !IS_ENABLED() )" early bailout paths or, whenever this isn't
possible without breaking the build, respective #ifdef-s.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:28:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:28:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198923.352668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVsZ-0007Ve-IU; Wed, 29 Sep 2021 09:27:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198923.352668; Wed, 29 Sep 2021 09:27:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVsZ-0007VX-Fc; Wed, 29 Sep 2021 09:27:55 +0000
Received: by outflank-mailman (input) for mailman id 198923;
 Wed, 29 Sep 2021 09:27:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jpj4=OT=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVVsY-0007VR-FE
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:27:54 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id efab97b2-9884-4b70-83c1-07140577f68b;
 Wed, 29 Sep 2021 09:27:52 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2ADC6D6E;
 Wed, 29 Sep 2021 02:27:52 -0700 (PDT)
Received: from [10.57.23.68] (unknown [10.57.23.68])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1D1073F793;
 Wed, 29 Sep 2021 02:27:49 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efab97b2-9884-4b70-83c1-07140577f68b
Subject: Re: [PATCH v2 05/11] vpci/header: Implement guest BAR register
 handlers
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-6-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <4ab8c8fb-7821-fcc1-acb5-1d8cec49e4c0@arm.com>
Date: Wed, 29 Sep 2021 11:27:43 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-6-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Emulate guest BAR register values: this allows creating a guest view
> of the registers and emulates size and properties probe as it is done
> during PCI device enumeration by the guest.
> 
> ROM BAR is only handled for the hardware domain and for guest domains
> there is a stub: at the moment PCI expansion ROM is x86 only, so it
> might not be used by other architectures without emulating x86. Other
> use-cases may include using that expansion ROM before Xen boots, hence
> no emulation is needed in Xen itself. Or when a guest wants to use the
> ROM code which seems to be rare.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - re-work guest read/write to be much simpler and do more work on write
>    than read which is expected to be called more frequently
>  - removed one too obvious comment
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
>  xen/drivers/vpci/header.c | 30 +++++++++++++++++++++++++++++-
>  xen/include/xen/vpci.h    |  3 +++
>  2 files changed, 32 insertions(+), 1 deletion(-)
> 
FWICS you addressed all Jan's comments from v1 so:

Reviewed-by: Michal Orzel <michal.orzel@arm.com>


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:29:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:29:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198936.352684 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVuM-0008Cg-0C; Wed, 29 Sep 2021 09:29:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198936.352684; Wed, 29 Sep 2021 09:29:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVVuL-0008CZ-Sh; Wed, 29 Sep 2021 09:29:45 +0000
Received: by outflank-mailman (input) for mailman id 198936;
 Wed, 29 Sep 2021 09:29:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5AY/=OT=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mVVuK-0008CR-Kh
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:29:44 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.86]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 84e245bd-baa2-4fe7-b4e6-99b49acf798b;
 Wed, 29 Sep 2021 09:29:42 +0000 (UTC)
Received: from AM6P193CA0120.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:85::25)
 by VI1PR08MB3101.eurprd08.prod.outlook.com (2603:10a6:803:45::32) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 09:29:39 +0000
Received: from VE1EUR03FT018.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:209:85:cafe::c7) by AM6P193CA0120.outlook.office365.com
 (2603:10a6:209:85::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 09:29:39 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT018.mail.protection.outlook.com (10.152.18.135) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 09:29:38 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Wed, 29 Sep 2021 09:29:37 +0000
Received: from 6eecb24ba9d5.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 5017ED9A-962C-454B-AEBF-8203B05766D1.1; 
 Wed, 29 Sep 2021 09:29:24 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6eecb24ba9d5.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 09:29:24 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM6PR08MB4803.eurprd08.prod.outlook.com (2603:10a6:20b:c4::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 09:29:21 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.022; Wed, 29 Sep 2021
 09:29:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84e245bd-baa2-4fe7-b4e6-99b49acf798b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HgDWpFjyYRgPv97da3w2BM8h441w53bay0cYkUStBkg=;
 b=N+0LMgY6srKlSWFj0WJb9AQrm6B25AWn5YK9cAeEZuHpT4fgj/AGnGP3BC6M0DkbArRk3qh7kf4i3lnZcOgFRK+tEsUlurB569+H/yN2Ddlda84JsENlB1Dhq8S3HGZZnomGkO/Q7nTnMjeBc4vx0zK9SQLP34w9N+kGAmmj3MY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 87c3b6975f67332e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WVJBVCWL8/kQjmzo2Njs3103WNqCSmo+01GG2v8oJuYBt1aWUbNvbA4MFP83Tpf1e6q0nW+y8DtV7WK5n6SEsAalhNeoxsUKUoERSUQi9S8MHcE+JWIRpZuyGQ4+qpaOmeGrHc5e/A0v5VR3RVvqyE4duVOOkFURA34ZKr7MeqWKHgAefky+egZEdbLdzoZZJSDoii0SrSZyQ634vLo3ooeF5h38i/cZKMQgakfFjPXLC4f1MM+cLNiImHQZfmLFRSReujjZy+5bRiUajbRiZWq6IwVjNC6PxphoWxPDmOJ0Nv0qBebxA2yn4jl75EKjkWkbtsqVofJay0E4haLU/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=HgDWpFjyYRgPv97da3w2BM8h441w53bay0cYkUStBkg=;
 b=VrCMAHuFk6rtkcOOzNEHOJfyxt6W3ZbLjKeGO89vbaxKJzZcJPQBJdmHX2ya78xvnjny8HEQf6darJDBk5+f3WT4JtgSDUGr57svB4hngvGbCUzRfXMfjPA9fpixHeJztz5PvPdVzJcrvRZK9RFd/Vxs5Xr1FdJji0GY0AJsMzem/Swcki3exh/dGzchhXI9akk4GmPLyN4cxbEzS1fBOmVuatNqUVomvemgOPue1Kn+keA0hdmqm+kzpg/4gK7xJ0qgxxKwxt7ps6rVPEFyZT7StERmhxRyTLRdHZUjZpz80Rkn8cKorg7icViz7kU8guNLB2jKdwXHnXhpl9DLpw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HgDWpFjyYRgPv97da3w2BM8h441w53bay0cYkUStBkg=;
 b=N+0LMgY6srKlSWFj0WJb9AQrm6B25AWn5YK9cAeEZuHpT4fgj/AGnGP3BC6M0DkbArRk3qh7kf4i3lnZcOgFRK+tEsUlurB569+H/yN2Ddlda84JsENlB1Dhq8S3HGZZnomGkO/Q7nTnMjeBc4vx0zK9SQLP34w9N+kGAmmj3MY=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>, Andre Przywara
	<Andre.Przywara@arm.com>, Paul Durrant <paul@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Topic: [PATCH v3 03/17] xen/arm: solve compilation error on ARM with
 ACPI && HAS_PCI
Thread-Index: AQHXtJV6eXi8DNccokiopfNmJVlfDau6rZ8AgAASK4A=
Date: Wed, 29 Sep 2021 09:29:20 +0000
Message-ID: <FE4E065F-7DA3-4608-874E-C3CC48CA1E9F@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <b95685e16fccf18580c094a5c57db4b83d5b4a29.1632847120.git.rahul.singh@arm.com>
 <67ae3cd5-e90a-5d62-b7f6-b9802ec8296c@suse.com>
In-Reply-To: <67ae3cd5-e90a-5d62-b7f6-b9802ec8296c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 6df7a686-0acf-4a1d-a34e-08d9832ba8d8
x-ms-traffictypediagnostic: AM6PR08MB4803:|VI1PR08MB3101:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB31018C15AADAC0E786DA5D6FFCA99@VI1PR08MB3101.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 kVKRorL9xTTdm/eNKufrIgwWPaCPnpI/Srytkne5n4w/BSoObBhiUofmFgS0rmiS94oc8D0UhlAwiIhHbYAsUNG8h2p/MAuG+pzAXmENPedTHqUvXQSpMTig0gwq3D3dj060JGQKD/RXoc74J+nD5QjvN5t//itwVnD3ZESdAeg+gXY4yJHOT8N4XX1BISvmp1z0/XR1goq9fEgSi/RApQ/aiRrShhGzvzNOcT95VlPbtmKXw/x/ZC3umo8FwnvpHS8eMTMX8scq7dMsnYd156vNihGwTjzW5Qn2Aw7J79Y9qysKa2EnQgO5kY+9sdLChyb9w+b4740jyf11Qa9qQ1fBbDUkSKcL8mXWah2K9sXGufu6g15s2UzO3eJm4UURY+uarbD8YQzkdisgftVCIWzN/yzI7pOD9ElmZMfpX8PoCTiyUedDzB0C0QfUwyw71ul8eW7/WZjqMhP34PWvkhsHpt2bF5MHpqr/qyW2iQJ25Ywk6ZFTizjQReDBEgELUlSWXdDgyxHqmTYCimJSdNn/LOI5yDmTrLP5DI2vgqHkgDPPBw9CQv+fIxohC/g2SPrfRAj/KKYunMpII9+GVY77AVMOR3vewTwwV+ivEJjw0MFeVK9D5GsLdJH5wEw6zCqktDFqpe4uid5JtH8+PI4icAjAjrMXtqT1bgBUUWOKbWbJH1HWEzzjpXAu7JL7IktJoZ44UNBlmn1c5cjUkhR3auQK6eA73NmEVwn5qDvQSYNpH2Fgr0ZXtEltk/9f
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(6506007)(6486002)(8936002)(53546011)(8676002)(54906003)(2616005)(4326008)(6916009)(26005)(6512007)(5660300002)(186003)(71200400001)(66946007)(76116006)(91956017)(86362001)(33656002)(122000001)(36756003)(508600001)(66446008)(64756008)(66556008)(66476007)(38070700005)(2906002)(38100700002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <0CBE0B683A0E3F479964F11F157EC799@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4803
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT018.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f5e4c861-53bf-450f-512f-08d9832b9e2f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5MZHAn5Qn0BLbflF5y5IuPvPXbW3WGRDUjV0qcw1y00JHEalzvC2T+aGMJ33GlU3JS1BQkK+dKEvxou+9HLXf8slGZnH4mUJfiBIbtuLpLfTX8zaT68ju6fz+WzxRmAhPRoX0y9fMT8rjY2hAB3S5e5JFZZ2Cfa6M3sDIZhNCJLlLWeNf9ZPWPL5rRIf5ee6ZyVooE6V+3UmIgzltM5zCJC9FngVDj4zc8yImqwUV7m8IRDZM27/g0qcyv8JsMDE3tg3d1CO3DqpfzgratFHNHX//1vkl1b73sYH5JJo5lXYAcykxH+AfaSdrltsSiqDTOufi98jEzjlFWQEyztbefaXBX9uUxoNahQPDQbqoXxJw/lqqFg85w/Pw2PxlrG791uEEP3uUgM77I4X3P9gKgQkxvKy1AMVL04dAXbHzQffEfB7gfw/CvWbRiDiPNZRDSpMYLwdqnzMzxsTzWQ03eVr9zEDYD6w6wqLWzsE7zEnLHO5y+zkhm42pzpM13YQWy+8xidHDFM9cK6ELkXptd/EgZNxbOflghco9/AfnN6m08H0y6ccHD4Y1V90J4ZNbIih+SsdfhEMniTl9JLAJiJ/R4uUorxFLVwKII1pKTPqmmZ+7jXh35JxwzwBf11xRL0SOrPH530zZrft/5xZelx0L2d1KIM5EgkjTPOmlGYMgKlZJ8za7nVG+N6ao3q+GhO2SNdyrmRxPfXU0mkbDQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(6486002)(36860700001)(508600001)(70206006)(2616005)(6862004)(86362001)(82310400003)(70586007)(316002)(336012)(356005)(6506007)(53546011)(8936002)(81166007)(4326008)(5660300002)(2906002)(47076005)(54906003)(6512007)(36756003)(8676002)(26005)(33656002)(186003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:29:38.8128
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6df7a686-0acf-4a1d-a34e-08d9832ba8d8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT018.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3101

SGkgSmFuLA0KDQo+IE9uIDI5IFNlcCAyMDIxLCBhdCA5OjI0IGFtLCBKYW4gQmV1bGljaCA8amJl
dWxpY2hAc3VzZS5jb20+IHdyb3RlOg0KPiANCj4gT24gMjguMDkuMjAyMSAyMDoxOCwgUmFodWwg
U2luZ2ggd3JvdGU6DQo+PiBwcmVsaW5rLm86IEluIGZ1bmN0aW9uIGBwY2llX2Flcl9nZXRfZmly
bXdhcmVfZmlyc3TigJk6DQo+PiBkcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jOjEyNTE6IHVuZGVm
aW5lZCByZWZlcmVuY2UgdG8gYGFwZWlfaGVzdF9wYXJzZScNCj4+IA0KPj4gQ29tcGlsYXRpb24g
ZXJyb3IgaXMgb2JzZXJ2ZWQgd2hlbiBBQ1BJIGFuZCBIQVNfUENJIGlzIGVuYWJsZWQgZm9yIEFS
TQ0KPj4gYXJjaGl0ZWN0dXJlLiBBUEVJIG5vdCBzdXBwb3J0ZWQgb24gQVJNIHlldCBtb3ZlIHRo
ZSBjb2RlIHVuZGVyDQo+PiBDT05GSUdfWDg2IGZsYWcgdG8gZ2F0ZSB0aGUgY29kZSBmb3IgQVJN
Lg0KPiANCj4gTGlrZSBpaXJjIEp1bGllbiBoYXMgcG9pbnRlZCBvdXQgZm9yIGFub3RoZXIsIHNp
bWlsYXJseSB0aXRsZWQgcGF0Y2gsDQo+ICJzb2x2ZSBjb21waWxhdGlvbiBlcnJvciIgZG9lc24n
dCByZWFsbHkgc2VlbSBhcHByb3ByaWF0ZSBoZXJlLCBhcw0KPiB3aXRoIHRoZSB0cmVlIGF0IHRo
aXMgcG9pbnQgb2YgdGhlIHNlcmllcyBubyBzdWNoIGVycm9yIHNob3VsZCBiZQ0KPiBvYnNlcnZh
YmxlLiBVbmxlc3MgSSdtIG92ZXJsb29raW5nIHNvbWV0aGluZy4NCkkgd2lsIGZpeCB0aGUgY29t
bWVudCBhcyBiZWxvdzoNCg0KeGVuL3BjaTogZ2F0ZSBBUEVJIHN1cHBvcnQgb24gQVJNDQoNCkFQ
RUkgbm90IHN1cHBvcnRlZCBvbiBBUk0geWV0IG1vdmUgdGhlIGNvZGUgdW5kZXINCkNPTkZJR19Y
ODYgZmxhZyB0byBnYXRlIHRoZSBjb2RlIGZvciBBUk0uDQoNClRoaXMgcGF0Y2ggaXMgdGhlIHBy
ZXBhcmF0b3J5IHdvcmsgdG8gZW5hYmxlIEhBU19QQ0kgb24gQVJNIHRvIGF2b2lkIA0KY29tcGls
YXRpb24gZXJyb3Igb24gQVJNLg0KDQpwcmVsaW5rLm86IEluIGZ1bmN0aW9uIGBwY2llX2Flcl9n
ZXRfZmlybXdhcmVfZmlyc3TigJk6DQpkcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jOjEyNTE6IHVu
ZGVmaW5lZCByZWZlcmVuY2UgdG8gYGFwZWlfaGVzdF9wYXJzZScgDQoNCj4gIA0KPiBJJ2QgbGlr
ZSB0byBzdWdnZXN0IHRoYXQgd2hlbiB5b3UgcmVjZWl2ZSByZXZpZXcgY29tbWVudHMsIHlvdSBh
cHBseQ0KPiB0aGVtIHRocm91Z2hvdXQgdGhlIHNlcmllcy4gSWYgeW91IGZvbGxvdyB0aGUgYXBw
cm9hY2ggdGFrZSBpbiB0aGUNCj4gb3RoZXIgcGF0Y2ggLi4uDQoNCk9rLiBJIHdpbGwgdGFrZSBj
YXJlIG9mIHRoaXMgaW4gZnV0dXJlIHBhdGNoIHNlcmllcy4NCg0KUmVnYXJkcywNClJhaHVsDQo+
IA0KPj4gU2lnbmVkLW9mZi1ieTogUmFodWwgU2luZ2ggPHJhaHVsLnNpbmdoQGFybS5jb20+DQo+
PiBBY2tlZC1ieTogU3RlZmFubyBTdGFiZWxsaW5pIDxzc3RhYmVsbGluaUBrZXJuZWwub3JnPg0K
PiANCj4gQWNrZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4NCj4gDQo+IEph
bg0KPiANCg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:35:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:35:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198944.352695 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW07-0001EJ-QQ; Wed, 29 Sep 2021 09:35:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198944.352695; Wed, 29 Sep 2021 09:35:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW07-0001EC-N7; Wed, 29 Sep 2021 09:35:43 +0000
Received: by outflank-mailman (input) for mailman id 198944;
 Wed, 29 Sep 2021 09:35:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jpj4=OT=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVW07-0001E6-4X
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:35:43 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 0488ef5b-c184-44d5-8f03-5836a7de54ea;
 Wed, 29 Sep 2021 09:35:42 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 97677D6E;
 Wed, 29 Sep 2021 02:35:41 -0700 (PDT)
Received: from [10.57.23.68] (unknown [10.57.23.68])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8DECE3F793;
 Wed, 29 Sep 2021 02:35:39 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0488ef5b-c184-44d5-8f03-5836a7de54ea
Subject: Re: [PATCH v2 02/11] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-3-andr2000@gmail.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <974e0546-545d-633b-08be-5a331e689ef3@arm.com>
Date: Wed, 29 Sep 2021 11:35:33 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210923125501.234252-3-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit



On 23.09.2021 14:54, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> When a PCI device gets assigned/de-assigned some work on vPCI side needs
> to be done for that device. Introduce a pair of hooks so vPCI can handle
> that.
> 
> Please note, that in the current design the error path is handled by
> the toolstack via XEN_DOMCTL_assign_device/XEN_DOMCTL_deassign_device,
> so this is why it is acceptable not to de-assign devices if vPCI's
> assign fails, e.g. the roll back will be handled on deassign_device when
> it is called by the toolstack.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v1:
>  - constify struct pci_dev where possible
>  - do not open code is_system_domain()
>  - extended the commit message
> ---
Reviewed-by: Michal Orzel <michal.orzel@arm.com>



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:40:44 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:40:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198950.352706 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW4w-0002kq-Ed; Wed, 29 Sep 2021 09:40:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198950.352706; Wed, 29 Sep 2021 09:40:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW4w-0002kj-9a; Wed, 29 Sep 2021 09:40:42 +0000
Received: by outflank-mailman (input) for mailman id 198950;
 Wed, 29 Sep 2021 09:40:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVW4v-0002kd-DA
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:40:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1ed13f0b-ce7d-44dd-9d0a-b6f766762e18;
 Wed, 29 Sep 2021 09:40:40 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-OOKK9Rb4MO-cYcBH1pjSJQ-1;
 Wed, 29 Sep 2021 11:40:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 09:40:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 09:40:37 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR0301CA0012.eurprd03.prod.outlook.com (2603:10a6:206:14::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 09:40:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ed13f0b-ce7d-44dd-9d0a-b6f766762e18
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632908439;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=yjRuo993pVi6TU0drSbNhT2mOgdyCXG9oveYnvgv3tc=;
	b=SZvFLkybAtlwTPJX9/ZiZtefiFGpnnwcrpjyuY/Gfv7Lj7L2LSl7am2OwZHTAUIUKTDcOj
	ID9qT0+tOHaeXy3SVe1Moagr53znEU6G8lf3fFReI+o2lm1uYTaJanhZhObCDhE1mCV0CW
	Ef2WKou1RJCD+AiB/Nc39yw+aeDtRtU=
X-MC-Unique: OOKK9Rb4MO-cYcBH1pjSJQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QiMmfDRk+Fc+KyxSlAfp4LVDeqY8ip6ttkO5+4MKMmrBtdOhTo9CQ0rBX5oyy+W049cS92xVj1z9uz70Hz9KleXi8l6ZmoAGhK2GJoqiwO/nlHQ9o37HUPo2O2E10fTu18lE2R3qT9M3AISg18WVz8z7OufUpbIvQajv9HM0rdxMo4MDyC8LY7ryZWbj1lsal0TDXOBjNYxT5w3aYvhndgqh3ucGpkOd+KbYbMH+rrEEoN05rcKVpmOO55os3FiJBxzfocN7jq/u/JFsWi6teS2ai53kWJVGG1zGKEHe4DjI3L5DEX4ypWJbAmsbyZB205SlXOVWZO1+JLp4DmUFFg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=yjRuo993pVi6TU0drSbNhT2mOgdyCXG9oveYnvgv3tc=;
 b=ah4rnQjB899Lg1CmKKzriCSGlPLFAYvLOMAQFhP5qQwn5mKn/gYFGC+I9XG3AM69FIQitBFDoIV6zsLkLrHo2FgXtgIhWaLicjx8gIvXWqhMH3AN9NZK49mBNnD1oKjf3eQtQNXqT0dvJcI2I0wFA+TwpkHy04hylOywiTfwRyKIHP4QtJfja+5xs6fkukhVgEr9My3RCBNajcxhLlmkqGpbuYObAidJV6XyM96ZTLUdf4bUctqMr5PLrkG7qf+glklTLlB4vyOlHW+qdVwUxRHI4biNIfzePqM7/tnAmTkIMbNvcEmCTFinsnM2JqLITaFwQxO2Aq3Q3Vuvhx4yBw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/3] x86/PV: '0' debug key stack dumping adjustments
Message-ID: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Date: Wed, 29 Sep 2021 11:40:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR0301CA0012.eurprd03.prod.outlook.com
 (2603:10a6:206:14::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a8f3b91-8dad-4437-a197-08d9832d3109
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB31200503FC525AE521FA1A4FB3A99@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yPYyGHNCISvv7GWOhztpIYdtEwouOPyJsOyU4ftN/6i+pfu4kt/8Lz/2wjWu02luvOHx189u8ans1YwsIYtiWpAbN2rUPc8oGwAcpz/5rHtu2VfkQpiiEWhKsoTTkrjxNPB74JchGzT/Xvp+1e5z14OiDWiai2rZCY9wgxhxyVMqnVSq95VxMuaN+pzbX8duUch9A/9rg5g4Swi9dOCS+g7lVsWzu2ycPDtH3Q7KxZB8po9bGAh2j61ZgQ9H7pKGkH6yY3jHobGlfx4dFchmcfsDCilLn1EiwEfJzrrTgtiwWaA+hmYqy8ec/x4l6GTG2Ph9BbVGqWqOjRBS7ANhuhYedNb/sMfelfy5DdG/PgmSkpo2IA3NArrUcf4CZ0edY1yqhOqQGZbPoO3QvRTt89q+b9urOsIayCuDSO7ZHHtmU7KIrWJcp1S0n0pDNp7lXlnFwU7vJ87sCJKii3ZNzUTt+DTD5liAH8Rwo2FFHu2DObUq4fbkK42KPDUGOXdzVXPEUrCyfVvkuFV16/y905au4eOfk3gamIQrQ+YPO29CsOeUmCVEHygiMOcO4GcomPwk7RTV7GliQsKn77gC3bK3uH2Q5zngTqDDaXEAKpqnADN3n01Uz3VzEzCB39wmCfooV63LuXT4mRQox1um8TCfioxZ1Ef3iJCtd3X/REaiQb2aiTwP36upmOKqML8MtRVGDXIO1da8mL/xUe4FY6OtDT8+VjdtaCjE6n55KVe8LsbzhaRzxNoGYQsdeNYi
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(38100700002)(5660300002)(8676002)(26005)(8936002)(186003)(316002)(31686004)(508600001)(6916009)(54906003)(66946007)(66556008)(16576012)(4744005)(2616005)(956004)(36756003)(66476007)(86362001)(2906002)(31696002)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cFVSSlV6M1IvcEIvTW5wOVRZSUhBeDVxeUVuTDJCYTk2MldHT1I4TG1FVmkv?=
 =?utf-8?B?TnNpWEZNeElPSENMY1FWdlYwaFZmUC9TbEorZGpyUlVqanlRVEdGSXYxT1py?=
 =?utf-8?B?MHA0bGd4NlNMUjRqN3lkL2diOENVVDRaME1KaWxDeVNFRUxrVEVDQjdIdnNy?=
 =?utf-8?B?RTd2RVRDa0xOekMzdzJ2ai9GL201NjA4b1A4NzRoenhqTXZiV20wTWlWQ2Rq?=
 =?utf-8?B?S0Q4UFJ0SElYVzFnZ240Yk1XRXFtNm80MWNCbEpyTzNSY0FJc1k4ZTJLdjUx?=
 =?utf-8?B?NytPYnR4cnBBckx5YW82ZkoyR25sMnREZGczdmtqWnRuQmwzZjFVY2tsS2U1?=
 =?utf-8?B?azc5WU1RZHVqT1loamFiMThBY005d29yMm5sZnBDcnFzd3FFVjBEUjZBU0R6?=
 =?utf-8?B?UEVMbWRqOWtLOW1kOTgwTGhLRjhGTmp2ZjlDK1JiWTlKR2Y5YWo3N1hQcGZ5?=
 =?utf-8?B?c1FPNVNrQVhnc25jZkh6SXh5Zi82b3VGUjJta00vMFhTRUZQOVV1UzM2UEM1?=
 =?utf-8?B?eHAxdGVzNE10Z0xaUld6MXh4U01xejdOeTU0ZUhRV1V5L3JwczJaemFXMm1w?=
 =?utf-8?B?Y2tVRWVId1JSczZxc1VuS25Hc3M1WW11M0dVUmxNemhmdW1Mc24yM0tidmc0?=
 =?utf-8?B?bU5DU2twUCtJaEFrWEc5M2VGVVVSMEdMMXR2MGZZVXBrK1FPMVZJYUZzU1pl?=
 =?utf-8?B?WThDRDVqdWhhOXhsZ3FSbWUweFU2ZER6YUxYRUZXN3dJVmdBRGFpUFBjd0dM?=
 =?utf-8?B?bURveVpjM2Y5YnZ3VndWajdBTjJmYi8vMG5kR0NvMURzOVhQTjkwd2p6UkVr?=
 =?utf-8?B?Ykw2Q3JSb3ZzYy9oQkRGTjh3VHQ2VUhzRTR5NE9zTlV6b2JJM3U4Mm1JVHFC?=
 =?utf-8?B?L1NuTk1kQjhreFdpMDFxZU9VUlRHZG1zR1ROVnQ0b1c2c3hhdUFRUmkvYmdz?=
 =?utf-8?B?aFNIVktBNUg2SVNhV2xXb3NiTkVZRTZiSkIzV3pCVnRvRDQrd29GbTFmRlBt?=
 =?utf-8?B?V1ZXODM0K2FxTEVFN21IQUdGZndSQkpjWWVOdkNHUXlaQ2kyVnYyQlFJeGZ3?=
 =?utf-8?B?MllqL2NpU1FIdGw2Mmlram5uQW5iS0I3UFVOUmw2emhGWlBZWjgyZlphck42?=
 =?utf-8?B?NEU1SEMzejVOQTVBa1E2SHlSamIxV2s0cS9VeHl6OXBHa2pEbkxQL08zQm1D?=
 =?utf-8?B?MzJnSjNLck9wdHhRckh2ZFdzeVI2alZzWkUzMEx4VWxkV1BRb0pORGdGcTNX?=
 =?utf-8?B?SmRZVFZnMFpKSEVLT3JnRU9iSmNJekVPY1h1SnJIMlMzMkk1RXJmQmNoVWtI?=
 =?utf-8?B?cml4SjNhbzkra2FOeTNvOXVrQmlPaTdUMGt6dUtHNXI0WHZ1UkRlVjV4ZTdj?=
 =?utf-8?B?V056U3pXMXJIN2NGZVB1Y0VMN3ZMcU42Z1hhaS8zUDFLejl1QWJxL2Z6MGhh?=
 =?utf-8?B?NThETzMzV08zak0rNjhyclorK0xheGJIYllTVzN4V01KZE1BUjBlWWlBSXg1?=
 =?utf-8?B?RDlmaTlzZ01zV01kNzI5TE1iRHBYcTR4VzhXMDZpcFBFTElQRmprMXQ5THhY?=
 =?utf-8?B?TzZIdEQ5SVJrbzFTSlY3NzQ3cEkxNytObHNmSG5FRk1vMnNXOU92OUZYR0lh?=
 =?utf-8?B?SU5YVjA0aEtYL0xoK1pYR2lyeVY5elBMbWRpRkpldDNxV2pWaHB6WnNkcUVU?=
 =?utf-8?B?bjFVZ3o2b2swUm54WUlENDN2VWs2NU5KUVVvQ1dwcEJKY0RFdUZhMDgvc1NU?=
 =?utf-8?Q?98XfQxwS8txFB/mxGnvv8OFg/CAaUrDSK3HJZOi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a8f3b91-8dad-4437-a197-08d9832d3109
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:40:36.9797
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0wGuTMc5q+qMlTVbVIR3gZ1G6+lK41IYu593dsYRgAY2gTZQnxnNDSNzSenu5jfsjUGX7Olkc0iFK0RfBDCG9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

Having played with '0' for PVH quite a bit lately, it just so happened
that I also tried it with PV, finding that it has been broken for quite
some time.

1: make '0' debug key dump Dom0's stacks again
2: replace assertions in '0' debug key stack dumping
3: drop "vcpu" local variable from show_guest_stack()

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:42:45 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:42:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198956.352717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW6r-0003My-Ou; Wed, 29 Sep 2021 09:42:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198956.352717; Wed, 29 Sep 2021 09:42:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW6r-0003Mr-Lu; Wed, 29 Sep 2021 09:42:41 +0000
Received: by outflank-mailman (input) for mailman id 198956;
 Wed, 29 Sep 2021 09:42:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVW6q-0003Mj-Gw
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:42:40 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 01a44e8a-43e1-40b0-ac43-f9a34e6c3f33;
 Wed, 29 Sep 2021 09:42:39 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2057.outbound.protection.outlook.com [104.47.8.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-f8SUCLSMMBS9Ghzb7whrQg-1; Wed, 29 Sep 2021 11:42:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7152.eurprd04.prod.outlook.com (2603:10a6:800:12b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Wed, 29 Sep
 2021 09:42:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 09:42:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0017.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.7 via Frontend Transport; Wed, 29 Sep 2021 09:42:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01a44e8a-43e1-40b0-ac43-f9a34e6c3f33
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632908558;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oLIHrV3tAS35bw50cNvZyc6rMif9PxUtegze2vI4lwY=;
	b=VgGKXLu+fY+Qb3WxUUdcZjxYZwNl8Uf3GvqXwr3TZNlBhDU9QdA1MEPZx52hfgwtApim2G
	UdBrL65H0Nk7uYiSuR1s7EJwC2WWgP1Dhg/s5Z6UP0mN3HuxyAqb0x5ZP6eIX/ab099dCC
	I4i0nRWwQPlAzuubFliMnj7+7MUBSUs=
X-MC-Unique: f8SUCLSMMBS9Ghzb7whrQg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GOhnbwJ6RW4GGHpRGXWyDQR8HD0efmyGHTSmFA1GUppXsd+kZoSoeOSUuCfKuUYeJ87IQPh9teeINy4/HRE7lMWKO3dRf2875LbO4SoNwNVrSa4tiIVKkqiGUbWTQKRePPnrrXI0nx6EsdjwzpyLVwdCkUs/GUL+xTAivvB46B20GiL50b7ajrB46sv3qN1gwCLEJ6yA9Ooe10UqhPu7OtE63+cwnQyP4ocH4w+67ESCDVCUMZgBhYCQX7XbuqsXljjYW7pMSFrVBzn2SqPenIvV4+S5xGr3NrqPDXB+/gqiiS5hNRK4rYdhrQ7bUZQVfgZ+FVOK7zfHoxlrRM7gDw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oLIHrV3tAS35bw50cNvZyc6rMif9PxUtegze2vI4lwY=;
 b=TbAPriCDm+1TWb9PRb7gdUnhZPs94+R+0U7PlQ906ThB8+hc0hP7b07532LtejwVMRzzkxnQsLDKYiFRYPOIoiatkaUHgIYwUFUNlc5Pb5HyS1V0F81fXgYBIV4Ch+Lx/kt0wQ8s9JonFmPWA1SzEp2iIJ7R/Il0+cqV4qKHmrb9qupUJ4Flw+jyisoomICosW8idKm1K8wZmJSeP+0bANgE3yCeAOsHVXW1O+Ajl/ziYHvs7qZYf8I7sZXB1i+wPUV+mnNJH4P27aecQmX0mxbOULEmVkxoby+KAsbn/OES/rXyWYRVwuHbtzCB5EaKU7sd21n4rHgiOFfjpAVqAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 1/3] x86/PV: make '0' debug key dump Dom0's stacks again
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Message-ID: <268da650-1261-0ac6-9908-7916f0b4d2ec@suse.com>
Date: Wed, 29 Sep 2021 11:42:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0017.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::20) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5d9e66e2-a46f-4efc-36c1-08d9832d7805
X-MS-TrafficTypeDiagnostic: VI1PR04MB7152:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7152699CDA74E15B23091520B3A99@VI1PR04MB7152.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zM9e/6QihDAFCTwJt+Rvv7YlzATzQXgmqDSk4U4yQDCo3FETTNUX0P42Hb80iKAdx82cHmOhmw0dqKMEjLp8A9udPLAZkj1O+6DbMp/us76tNX4EZucdQQUlF2DL2vL10wjiZGuuhDF9hM80vEVVUL8pWtg3mDXI39C2nhzOBaW2TndrIp2a0LHyPWYP39YghQv2aJAAcKh5S9H98hpaue+tLwaEXaYtMXqorA4+HuPvdBRAtr91C2VISLDBqmTEqHCfro+0jWicoqf8a1eoJws5Mi5L1PWVGbokfDyBpaMZh5MEbupJF2MkehM/bn1/0ayBmAKLGdeTraPIrhFRjYdqlTsSjcoyBNv8JNF7GnQPU9nUVB4sxtfGfmCeHiugtIoVX07mLzg1CWrrKRbTS3zxMZwUc8WGwGfGf51AphpOlBoOToOmCd7+EPjcFnc5LaMTCqqUWw/z1NLMS4mUnRtWoKgI2hzNw6euYYDfHWwHC0KeBA5q0RsAkTQ4c+TdpTMVxnSoMkGQO8qbKftIcDDOgHhkg/7BA/OQ/FuBxPlhZeGsQR/FlImd1noQVPS5a8PnXhKBWlqYp5BHkUAjAtbVM/Bkpv4ElV9XJJXb8An1Nf5ESrmFxjaw3E+fhvNMu5bmJT+F/1OIf8kyJ42ne/nO0PjYeZIwUM5WK1ucHJtpG2bwGm4ibT7AdyVIvkPDuP4I/8VVRKCsTrpO4aaf9yz947cM69/jIy7UvdZCIUA0nMFlOUqsneFOmpHdDUid
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31686004)(66946007)(508600001)(956004)(2616005)(4326008)(186003)(6916009)(36756003)(86362001)(2906002)(5660300002)(83380400001)(8936002)(8676002)(31696002)(16576012)(316002)(38100700002)(54906003)(26005)(6486002)(66556008)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YkdEYlREbXhDS2xsYkxsRFZyTjdzNnI2Uk91WGdDam1Nb2xHWVUydnBYRlR1?=
 =?utf-8?B?NW5VT3pHaUI2MFBjRkhpR0JLck84cng3WW9nblkrem42d3p1OXV2OVMzay9H?=
 =?utf-8?B?SHU4bFRCQy9taGNRL0lpbkRhWlIyZzR2T0xPQm5TdXQzWWdYc3l4MExjUUY2?=
 =?utf-8?B?ZVVSYjhPc3BQd3hKTmVBaVQveTMxSE10TjY3ZXFLaFVhU25QdWxRblpZbVk2?=
 =?utf-8?B?Rytnd2haWHY4Q29qV1EwbllyQlVkK0NpOUZoWkk0aHQwN3NGa0dyV2h1V1Zr?=
 =?utf-8?B?ZXdGcG41RG1xNFRYbjFxaEpRNHcveVVBWXpFWXNJaHYzMkpHR1Zxam5mYWxF?=
 =?utf-8?B?VXFpTDZJbzJsZmo1Wk0wSDhqK0hZZHVNcW9TLzNwN0FvdWUxRmRNcTBpWmtr?=
 =?utf-8?B?dEUvMnNjMVdwRmpmSHYwUGlRQVdyZHZlMThxSVRkU3ZXdWV6Z2daamltSE1O?=
 =?utf-8?B?Qy8vMjFzS1hrU3MrQmNORmlaMzU1WDlOMm1KWURoZHVONjY5d1cxUUd5QTdY?=
 =?utf-8?B?ODlxbUNGL2psYWR1WjRzZUl0Wi94NHM1U3h2eEtaa2srNEZSTnV5eTloVFlC?=
 =?utf-8?B?QXhLSVNrM1lkTVhNd0xiWjBYRjAxTTNrT0lFZjFRZ0lsc0RsMlVXTVFOMUNJ?=
 =?utf-8?B?K2owa1N5ZlRqTWNJTnI1R1pQSHpKeDVzYkEzeEp5eEhXbzZxOGQya3NtV3N4?=
 =?utf-8?B?Q1h1MEdMZ1FIYWZxeUsxemtmcTFPRjJIZWtRbmNDOE85R0VEQnNNVE9aRkFI?=
 =?utf-8?B?SzRKckFYTlNKTFFKak5Vang5dVRNdW5GTUVJYWtyZVM4SzFuMHZTSCsyeDdS?=
 =?utf-8?B?c0xiNm91cHFzQVR1bTJHUk0xMXdlQlprdmJIZkZXZVBtQ0RNZlpjeVhwS0xS?=
 =?utf-8?B?U1pPVEJDSlA5cUxsRjdWUEdCeW9Fcy9Sem1nZCtJUFRMdzlJdjV1TTlBZFRS?=
 =?utf-8?B?bFRUc1BuSlVDSlJmRU5ENUR0V0w2UW9rbWI0cCtXVFVyMTRJdzRsWjdPMmxy?=
 =?utf-8?B?THpLbTkwZHdQYjVkOHdrZlRGQmZsVENOQm8xZnVacXVLdFpGTWNmL0FYVFMx?=
 =?utf-8?B?V1ptckZzZUd0VXA1OWhBbHA0NnREbm52TEc2TUJSN3k5YVhuUTd3N1pydmlO?=
 =?utf-8?B?alFTd3QzU2o4bE9ubmlWeStGWmdVV0l2ZXpCSjBnKzl2aXRTR3BCQ0EzMnFP?=
 =?utf-8?B?OXBKZ2lycHo0bXlvSzl6amFKRU1PcTJ6VFFVSWhUdDdoVThHeTRpSXFMaFFZ?=
 =?utf-8?B?UnpsZ0x0aFoyL1Zqa0J2a0hWdHRvTys4aks3SHNtWFVUeXRrTU9FcFdLNTJ2?=
 =?utf-8?B?ZTlnMVRlZGxHWU5hQ1lwZlhVbXptd290cEVqM0FEeWhyMUhJc25jQndESW40?=
 =?utf-8?B?Q0xjM1RHS1hWTVluRVFFenI3a2pnaVVueXJqWGZreU8xa0p4b3g0eXFxalZN?=
 =?utf-8?B?c09SL0VnRDBWUVZ2WHRpUmZhN1ZENk5ZQW1sUHYrSnNyNEJWMEpxMkIwWVJY?=
 =?utf-8?B?K2lBOFJBYlBTck44ZTVvVTNGelVCOVpWYi9iSE5Vb2hJRjZmQ2s5YzZMemVO?=
 =?utf-8?B?TS9ybHBETWRMU1VlMW9RYlQyblpiK3VOby8yVWEzK3ZWazF1akZxODlQU0dZ?=
 =?utf-8?B?a0I1eXp5czJDaWRBd2ZzVVJtTnRabG1WWlF6VFo2SGpEOGprQWY1TXBBRkdI?=
 =?utf-8?B?c0xUYnd0SmJYbFpoQ09rUTYrT09Bd2JLTVJBZjN2REFPNE56OXRuQm1xQnpR?=
 =?utf-8?Q?PScLhkxEnZ9+SUzlktvEQTp4ym+xwV+TyChW5Oe?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5d9e66e2-a46f-4efc-36c1-08d9832d7805
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:42:36.0590
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: i+QiDNiXxTVZ0rDwTT+UpNSjUeB6aeIjrAULzH2vNt5QkSlxCuK/wUaZhYxNl/h46xrcWv8a7E46OIqyVCyHtw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7152

The conversion to __get_guest() failed to account for the fact that for
remote vCPU-s dumping gets done through a pointer obtained from
map_domain_page(): __get_guest() arranges for (apparent) accesses to
hypervisor space to cause #GP(0).

Fixes: 6a1d72d3739e ('x86: split __{get,put}_user() into "guest" and "unsafe" variants')
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Using get_unsafe() might be an option as well, instead of the added
extra conditional.

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -275,7 +275,9 @@ static void compat_show_guest_stack(stru
     {
         if ( (((long)stack - 1) ^ ((long)(stack + 1) - 1)) & mask )
             break;
-        if ( __get_guest(addr, stack) )
+        if ( stack_page )
+            addr = *stack;
+        else if ( __get_guest(addr, stack) )
         {
             if ( i != 0 )
                 printk("\n    ");
@@ -344,7 +346,9 @@ static void show_guest_stack(struct vcpu
     {
         if ( (((long)stack - 1) ^ ((long)(stack + 1) - 1)) & mask )
             break;
-        if ( __get_guest(addr, stack) )
+        if ( stack_page )
+            addr = *stack;
+        else if ( __get_guest(addr, stack) )
         {
             if ( i != 0 )
                 printk("\n    ");



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:43:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:43:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198961.352728 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW7E-0003sv-4H; Wed, 29 Sep 2021 09:43:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198961.352728; Wed, 29 Sep 2021 09:43:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW7D-0003sm-Vq; Wed, 29 Sep 2021 09:43:03 +0000
Received: by outflank-mailman (input) for mailman id 198961;
 Wed, 29 Sep 2021 09:43:02 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVW7C-0003rE-Kp
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:43:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a2af4adc-2109-11ec-bce0-12813bfff9fa;
 Wed, 29 Sep 2021 09:43:01 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-lanrSw__Njytv9-yh5Do4w-1; Wed, 29 Sep 2021 11:42:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 09:42:56 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 09:42:56 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR3P281CA0013.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1d::18) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 09:42:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2af4adc-2109-11ec-bce0-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632908580;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Y7ccXA22nntJPvmIaAdNM7vs28zhnf2Wqchn+7jd8mY=;
	b=ljjRPDW5u1dQbR7uDFYL1RCts44lqVduLvAjgef2gI0xpi9M+bAqgtmcxigtB8PUBqaece
	jlyWMdUKJ8yGu4EyC0Tykk77sJCRsaxnezL4Kqd6jjZt8suxjkXeHiRp8lfU0uTL9tXNpc
	tR6pvJLnknZVfJ7995U+qlRmGW+QlkY=
X-MC-Unique: lanrSw__Njytv9-yh5Do4w-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YRtGTDVearZ/+MjEozVlI3PIqqe/quFylaWisaQiEz3rndBeaWcB7aVPNH/1wkSu70g5Z/7sB+OCGmHzrubNQIUovpAKnLOQMRh/4Je0rjppDwZja7ibGwN2RP9xByASS3axQGej+LubUdZ+Kv5dYpSZnVEVo0YcdWyoF9a/9A3RKnFbfmP9zIZXdmGXzFjHM1pRvgxQIJjR9pztdL+9tyXnjpy8Vxs4Jv+cmpkSEtkTn9zgtk/yMDlJ1h69ihRYEqk1NWclPc7CvRKZfShZUdI10Ca0l28hW1AYJtmBKwQiWtPOMPaTm6dHL/rW7UQH/2Pdqh50/V1bIzgAUGCLYw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Y7ccXA22nntJPvmIaAdNM7vs28zhnf2Wqchn+7jd8mY=;
 b=cOzLO5BBOgxZ1teDP8yhZNedntYQ7d/3BOlwhf4rXpPa0tHXdlCPtD/dw682wyAtOaMl+/6matDsiSZX7jTfdml/l6vmpPtOihkAtYRT9pJGVwiZccMhQ2iRL8P+0f4OyMjFCSEkBGjStsyTsOuL/6iO5NHEVScsvvayGZegFXt11waLJro1ALZnQoLKRuixDa29UsRGCFqTzEHHqb8eGxlISpftKVp6UIG/Pdo9qYW+yrv1S0K/wJiiKbocgOFi139s3173K7pJQbsfLEMvzkJnvlwyK7rCUWQWNe/BKsWNLO6OjvAz8cAl+m53i3C9asSDJ1hmr8Bsli++FIPPHA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 2/3] x86/PV: replace assertions in '0' debug key stack dumping
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Message-ID: <5ab88195-660e-5d73-cc34-67734ae161fa@suse.com>
Date: Wed, 29 Sep 2021 11:42:54 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR3P281CA0013.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:1d::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 937547cd-7f96-4ba6-6399-08d9832d8403
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB312093A598E6C3C4236E87C0B3A99@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SVN65TBD2eDwJXpIDni64FCxegQrh/xmGe/7zCh/Zm0+HuXGgb2FOO5DYFPfJF3+tq/RL/K4NaFuEhdrjpxUA02KBSA/leFz1iwl0yRRwGj0MoQoLzddFL4pOlGDux0Jq+KHmaQEPCRvqHIvdsmQ6UNMlNk/xmo5blZpWU7nMHixW8xDAIr4O62VrgbLA/scX4pyYjXWyIAPCy4yFTwEuk+qxGWBi+1PzPzB80QQaNsrFEKQTgS0TTh8NS/IfXyRZbBa0z0fDG0Z6/UQLQvnzDnmNzPAzav3/iRcrYEbIe4+ypu4R9UIT+L71udk3CL2zUNiru4e+gXeQzqJF8nysgaKsAhK0blRhLttd0PM72pvxjaE6empy8NP7lbvwanqsIDD5rLsmdrP4sYWgsx9YPE5AM0QSqLEbzfL2hmtwtbRNF7RUw3UH/7TS7wnKus6Oez8kcffbZAze5U5Nz6C62w9xOgmNHO/bMbU0tNaO4kKZfW5qMBJfPhBBFprdp8F8BusREI2cjkv3qyX5JJ6GsSK2rXijZv1/ivrbmWCICw16wtGhAnpgaRCV1tUOchnWO1lSXhuP+kehYbv5EtMnn/4Bb0Cyxdv+7Ib+v3hhqZOdAoEydVjBl8RQO7eoOCUB6LtUofFJYQmZu43qRL8hB+h0eF8nQb9K7n8NBWuEcPJPav7FwwXWODVPAOLxdTlx5p4LAu2Uj9jHRRu0Ut0pcd8uzTn48ewpqesdi9PXGM=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66946007)(16576012)(6916009)(54906003)(86362001)(2906002)(4326008)(31696002)(956004)(2616005)(36756003)(83380400001)(66476007)(316002)(26005)(8676002)(8936002)(6486002)(5660300002)(38100700002)(186003)(508600001)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?a3lCYjZyUHlPMlRER0hTdUJlZjRNTVJnN1dGOCtwSjd1dkxOVHN2TkZoVlJ1?=
 =?utf-8?B?bGw3Nkt3b1h0YUY4U2Z4WmZFU1NIMjBUNEt1ejhrcjhPUUNOb0NwSm82QjY5?=
 =?utf-8?B?NGJldncrZ1Jmajl4djlvU2l6a2RxM1NlNmQ3MnpqTGU3aWdkais2VVpyS0Nl?=
 =?utf-8?B?YW05bFFienp4Rk43YWF0Tk95Y0ZwUUtUckkySys3VHo3bm8vL0hvQ3J3c0Vy?=
 =?utf-8?B?c3ptVUIrYm4ySUR3SmFUZzkvaDZ2R2tRcW8raGU2M0RiSXdweU40bnVtc1lw?=
 =?utf-8?B?YXBpTXhrTkl0OE5URXBseEtUcVU1am50cXplbngxbHM5d2tqblFmbEJFVkNo?=
 =?utf-8?B?T1hOSHRnTGR3b3BFeTcyUFV3RytjSm9YYUp5MlFEU3R6eUJ3M05jQzI4T2xi?=
 =?utf-8?B?aENLVlJqWUpscm5YTzRJZURLRmtLNzVkWU9FclFKdkhkNzNpdGNCY0hlekNG?=
 =?utf-8?B?akNNclhjTTNMcm4yMTcwelBHSSt6RnVZQXI1OHJDUEU0eDE3VVIwTVczejUz?=
 =?utf-8?B?QzZxY3YrcGdmelEzdmR2bG16bnB1WUkxeGFCUkJlbVhuNGs2UGJGdUFoUXg0?=
 =?utf-8?B?TUNCY1Q3NzFpR2xNOFFOVnNwRE9XU0tjclJaanJVaTJVKzhxZG5YeVRkbncz?=
 =?utf-8?B?eWMwYTNqQkpwQXFLWFNUMThHd2pHUHpjMXp6R0hLSlo1YWM4OHBPcEd3WFBn?=
 =?utf-8?B?RlVzL0hoeEpwY3B6NFYySWhQNHYxTlBQdW9qSVhDdE9GMFJjQktoemFKVVVS?=
 =?utf-8?B?ZVprZ2RvZStJV3Nic00waWduc3VCWitISEYzeWlyNkZqUmppMVAvWDYwNllQ?=
 =?utf-8?B?Vm9KaWh6UkFRNVpiK1R6ZHY4NzhDTVVMTVZGN2NMemlGSUhyTVMxWHh6K0E2?=
 =?utf-8?B?T0t2dGZMbUk1RWtrS3RONHdDNzJ3OVd5czZhZkhMSG1sVUtsU3kwUUo1bWhy?=
 =?utf-8?B?aWV3VzIxbmQ0WlREajQyVjJ2aDhOK0hiYkd3UGNPT2lydUNRMm0rY1BOb21N?=
 =?utf-8?B?QmU0RFVvLzA2cEtYRmdHZk53WXg5eTVkMG4xTmVBcVptcW1uZ2tPNUw3aXRL?=
 =?utf-8?B?TUNxTkZPc095dUpCZENFcnRvc2JHS09xTTJQOHdwZnc3M3FKd3NYbFhKd09w?=
 =?utf-8?B?ejVpanVCMEpVMlNKTjdoUUVIbHd5RW5mbmRNNVBlM1RWbURKbzhTaG11eFBJ?=
 =?utf-8?B?T1JLa2g1TU5XZThDU3JoZy9VSi9vZldkK0N1Nkd6dkVPRG9wMEp3L0MxbmlJ?=
 =?utf-8?B?T3B0MithQUZsNVlMeVFuUDMyL0VwdlZJMlgyVnhhdVpiYWEyVUtrSEdLQ3RQ?=
 =?utf-8?B?NjhzZzdNYnRZbnU3MlplV1M3cHJrOXBpOTFDL2ROUHZBSDJKTzRoUTh1aUFt?=
 =?utf-8?B?b0IzTGs1cXBVdHVXektHRVF3cmYvTTdYaCtVZko5ZklUT2I3K052aHQ1cVpx?=
 =?utf-8?B?RU54aHRJa0JEMTNDUFlBa0V1dlcwdjM2V3lBcUR2azkvSVMrV2FnbnVPd2sv?=
 =?utf-8?B?Z1FZM0hrTjV4ZmNtYWRSR3NHVmpENUxCU3BMenAyd2JEZFIySFNmZzhCem5l?=
 =?utf-8?B?cUVwV3pESkY1V0laU0pOOUtDb01VSU9DREJtSWlnMzcwdjNnTGlidVZ5bEtG?=
 =?utf-8?B?MUFrTXNSMERSZ0FCUXdrMEY2cFFLaTlSTEUrQ1dMemN6bmJyWndmc0pOMEtw?=
 =?utf-8?B?NWd5NlJIQ1VCdzhFMGR1R2VtQnZsL1FqSDZVaW04ZXg1dWtpbFpVMy9kRjFW?=
 =?utf-8?Q?WZySqqyMsHTfyvoV5iw3gy7+USaMJXpI43aLTm7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 937547cd-7f96-4ba6-6399-08d9832d8403
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:42:56.1807
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: i4KUhHJMOq4vR5HFvK8DLn/FfFoqRaT/9MyLvBxQpEEdojunTwg80kZan0fyli44pN1RO4TYLt6hyU55C5orSA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

While it was me to add them, I'm afraid I don't see justification for
the assertions: A vCPU may very well have got preempted while in user
mode. Limit compat guest user mode stack dumps to the containing page
(like is done when using do_page_walk()), and suppress their dumping
altogether for 64-bit Dom0.

Fixes: cc0de53a903c ("x86: improve output resulting from sending '0' over serial")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
An alternative to suppressing the dump for 64-bit would be to make
do_page_fault() guest-user-mode aware.

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -254,7 +254,6 @@ static void compat_show_guest_stack(stru
         struct vcpu *vcpu;
         unsigned long mfn;
 
-        ASSERT(guest_kernel_mode(v, regs));
         mfn = read_cr3() >> PAGE_SHIFT;
         for_each_vcpu( v->domain, vcpu )
             if ( pagetable_get_pfn(vcpu->arch.guest_table) == mfn )
@@ -269,6 +268,8 @@ static void compat_show_guest_stack(stru
             }
             mask = PAGE_SIZE;
         }
+        else if ( !guest_kernel_mode(v, regs) )
+            mask = PAGE_SIZE;
     }
 
     for ( i = 0; i < debug_stack_lines * 8; i++ )
@@ -328,7 +329,12 @@ static void show_guest_stack(struct vcpu
     {
         struct vcpu *vcpu;
 
-        ASSERT(guest_kernel_mode(v, regs));
+        if ( !guest_kernel_mode(v, regs) )
+        {
+            printk("User mode stack\n");
+            return;
+        }
+
         vcpu = maddr_get_owner(read_cr3()) == v->domain ? v : NULL;
         if ( !vcpu )
         {



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:43:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:43:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198969.352739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW7n-0004a9-Hx; Wed, 29 Sep 2021 09:43:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198969.352739; Wed, 29 Sep 2021 09:43:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVW7n-0004a0-Eq; Wed, 29 Sep 2021 09:43:39 +0000
Received: by outflank-mailman (input) for mailman id 198969;
 Wed, 29 Sep 2021 09:43:38 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVW7m-0004Zq-LY
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:43:38 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a51f58e9-2f41-45b2-9694-77ebc0d8c3ab;
 Wed, 29 Sep 2021 09:43:37 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2055.outbound.protection.outlook.com [104.47.2.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-drNc4Ob3PL-ghuOg8DSjYw-1; Wed, 29 Sep 2021 11:43:35 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3120.eurprd04.prod.outlook.com (2603:10a6:802:e::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 09:43:34 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 09:43:34 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0145.eurprd06.prod.outlook.com (2603:10a6:20b:467::27) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 09:43:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a51f58e9-2f41-45b2-9694-77ebc0d8c3ab
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632908616;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AublGjh33/bNoaWEfJ8R3GvMAzXTnFnEdFM3kcKmyl8=;
	b=gDPKnL3JmeqZtyivP+NLwt1kaMqGrT24uM+SwJinRiGX85DIH/NSTnNt/PEL3EfRUC59um
	DQDGUiGQnN8VwDbzlh4W7Dqx/WdWuxrob+K50KXGH3XXqKQy+bvPOWKBgaosqIyagfad3f
	+lJ+/KONVCOxqRO+CCguE2x0HDvhv6c=
X-MC-Unique: drNc4Ob3PL-ghuOg8DSjYw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U1F6iLcDOY3Qb5MxIPl5VfyaDbRi4fBvyHlSxfBrrgNilvwnPih2vHny/LQRiG/QSUBhGvcuX4BJt+iz0pncJLqjFMyt6lNVi1RJRgXXVxr5pe23V7DCdiVgSgs9Z2cCqde5Yyd3Cmjh/c1R92pbOp8+uqucfV1qPnh9rPyj95jDHS5pIVVDxHi1PYfff/TH9RiY3YHESB/Vlf/Tt/QEdsDV7+hQpk4pg2YlU8kpkpb9KWvVQwKyZKJSgPWJx66gtycvTkIEe/8LL7UQ1wL0qY7laAGbw4VMySNiR1VcnNadqlWw0HkT8zV68Nn4xOApK/xiib2urOSHmC+Z8KV0Qw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=AublGjh33/bNoaWEfJ8R3GvMAzXTnFnEdFM3kcKmyl8=;
 b=dztgOZDVSYUmEY3ERFwq1/HnaeOV2ps+k6rSSkCA61sKbMdPP1DKmuNiKPgJzzm/VArRAWGQaBX8ssv7D8J3MM/dm9sqUKXtxV6tugJq1uTSh8BW1Czag9sOtJsoGZHsGizt9xrq8ngBfp49qcKjKkjmZvCIeRGg4W7Ii7/TId81vL1Bd0PuMNAD6H0xqwR4oqNBJ8QuBgr64CBPqp55M0y4waexwsR+mYT6lpmqPiED62x7vTmFI7mkQbk2SbogiOVv0C7TJTSxHr991HjDDlksutvzRmb6EaNYMd2HlFemavZDzdf3hqLpYQUdNLUnDh0lDL7VedoKlu+UWEZupg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH 3/3] x86/PV: drop "vcpu" local variable from
 show_guest_stack()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Message-ID: <6dad4cfb-708d-027b-0fa1-77bb39319fe6@suse.com>
Date: Wed, 29 Sep 2021 11:43:32 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2a14f2d0-2a53-287f-da36-dfb054c7f944@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0145.eurprd06.prod.outlook.com
 (2603:10a6:20b:467::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 89cc0893-c283-434c-344b-08d9832d9ad6
X-MS-TrafficTypeDiagnostic: VI1PR04MB3120:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB312026FE274A4CB371DCD9C9B3A99@VI1PR04MB3120.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1388;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	67nxH89wlRlubXJu4n2Gn5yP094aZQONQxTCAcWDAbipwxncqljfTkTq4gRzjWgcD6D94QAexl6ZVCoove8CzevfZJIyZaNia2vruKlgSyO6FIvAMi32dXhGyYlgH1A10WehUW/QsqQv3LFwQhAZoZnee9NM8zWgTGlut13g2EG0fa29l+WFerP9xWeF/75A5Ud0jHLZuloJOueONdS/CSSGEIPxRgAOeFNtwB7sOzmOErYPLL2swRZl4FgvHhByM5sMaUBwQKB6C0VIYF1SNDk7qq9WTzUsBKA+981vDANMhaN6Akbid+FronD2x/ikan6IMF5v0fKXDPMkrCCBbhsnWuPWNl7tNURHoRC9M+m1za36WTtQ9+anIaT9ioeokzgq5aDojQes04gOjM4S7vLNX4sMs12ZVUWZX1frIcdC1buZa/ku5GubqRshmkdkWHaG77JTNbKzPa2TQtIoRJxF8U22c1weQq1iLjXE3jL4giUqOcjze7OzQy9pXrymbKW4FJn+c63qxWUiL75aW2CkuaCemQh65/66TqdgFbMiSpLCrPyTgWhSebTkcfaw/YE/RsAFGX4QLTa/7l+DufMtQbE9zyqEkR47ayG+Aa5hWxQbezRqjHZZyookTXCQR+RTT7vDkwAeDYEObK0JokEwDED53eAST7Lb3imd5YO7ue7ml2qen1WCA/ik0oSQi9u/Uo09hUDqgNwwGXOuNNCcxsLiD1iyUPBFnMVACi8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(66946007)(16576012)(4744005)(6916009)(54906003)(86362001)(2906002)(4326008)(31696002)(956004)(2616005)(36756003)(66476007)(316002)(26005)(8676002)(8936002)(6486002)(5660300002)(38100700002)(186003)(508600001)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UXlDR3c5OHN1YnRlRnpjNDA4OFBkTjVVekd4bjlzQ3hqMFNDRjE3S3JDS2JW?=
 =?utf-8?B?dG1QMVBZekNMMS94WDAxTHpBNjhiTlU0WEx4K3E1L2pHdHhad3RxQ1pIS3Ew?=
 =?utf-8?B?aUNOdENLTWk0Z29Hb3R0MTdKVHdSQ3prYVQ4MUJ3clFmelltbXF0c25xWXB4?=
 =?utf-8?B?SjNpcGthNlErMXcxc2p4ZGVFNHlRUTlZWGhlY1hDeGp5cll4RGFmVXRZUC94?=
 =?utf-8?B?cUJnLzJCQWh6R01yS1E2N0NlMDdSc1F4dXdGaXg0RHdwQnc3aEc4ejcxV0c2?=
 =?utf-8?B?N0ZJNWdVY1N0RHZiUm9Dd0REWjNPOFBjSUJpL012MjBCM2lubjF3cE5hQUhu?=
 =?utf-8?B?SXdEK3lhVFJjT3JmVE0rNDg4ZStwZGkwL29CQnpDWkxuVWs2cG1tczFIYkFo?=
 =?utf-8?B?RzloUW5sSlZqc0RXdXlXMFdtRXJ2UFBKZW4vUDRLVlhEZXRKdi9sdXNXaWRt?=
 =?utf-8?B?b3dUQXFFSzlIU29EMnRWaDRORkVuT2VDM0EwMG5oSVNUaWc4R0R2YmExTEsz?=
 =?utf-8?B?NFRJVTRVSGlDTGFiUkVOeTFSV1FIbmhqaWNHeGxPSWVtWTFUeVo1UFMvZnJX?=
 =?utf-8?B?NjR3Wm5RVUZJRk1QR2lIcDJkZ0ZnU2pHdHd3cmUxSzdOcWlsVzRZenA3Qzc4?=
 =?utf-8?B?OWw0RlFVTHRIUmZRZU0yUm1aNWx6RVB2YnpqZVNQZmIyVENKNVdpMFVkZlo1?=
 =?utf-8?B?ZDJaVkF1K1ZyYmNkaDM2NUZacHowUDNXbmJReHA4RkJoZTA5dHFFTk9MMVYz?=
 =?utf-8?B?cEJEcHFISTF5cmJ4UVA5Nlh5SGtmRDdtNDViNG9aRzVKQldkb1BYV1FHeVMw?=
 =?utf-8?B?NFZ6amNvcEN3VmU5MHNtSHpCT2hHNkp6M0JOU1luSzlKZzVuQ05SY2c5MFE4?=
 =?utf-8?B?ZEhPd29JdG96Q1c2RWJYSWY0eHZINXNvOFc3OXVaSnhrcFVxL1RYR0FHUVVG?=
 =?utf-8?B?bG9pSlJyYWZCbjdpbmZkMG4xcC8vaHBoMUs0bFA3a1E2QUdESlNoRVN3ZkNv?=
 =?utf-8?B?MS9xU0dxOXZVRUdBR3QwcFNJLzhZNmp1T0I0UHIvdS9Hd2pjUG5rcU1SRTQ4?=
 =?utf-8?B?TnJsK24xN0FWaytQUCtVVkQ3ZXRGbUVWaExOMU5qNjg5STBEbXl6c1U2S3hX?=
 =?utf-8?B?Yk9jclZDMmMrUSs2eWxkU0d5VW4xemNmRWRFWnZySklQUmh0d1UrOVdHYU5N?=
 =?utf-8?B?Z2FhR1RhV0VjeTgvVlpDMHEzYU1PYlFKWU1xMU9oNDBSNHl4Y2hjeTYzQmRw?=
 =?utf-8?B?VVBTQ1RpZmRkWThDQkJKZmtudVpQNGlzamVkek5jLzY3b296LzZXdzBKWHBF?=
 =?utf-8?B?MXI2U1p4R2E3cVk4bjJIaE5SSmt0WXVvL0VBbVR1Z0VzQVFJbmxTdnI4Y2hl?=
 =?utf-8?B?VmhVT0l1MVVNQTNQc2Q0c1RkVExWV0F1MVFJYWdDNkNmTTRpYytEYnlobDJL?=
 =?utf-8?B?cXBQeUo2S3k5cWVYUGlmM0ZmWjM5MmJLRnU2aW5udHdGcjhLSEdGM3pHWnFl?=
 =?utf-8?B?V2h6eWQ1YzI1UEN5RllVZGQ4RG0xWGc5dGNadXEyTTFSaTJMMUZMWXpJZVFT?=
 =?utf-8?B?SzlGMmp4UUxRL0dBRkx6cXRyV0tnd1NzaW9vMUo3ZVh3OUY0cFhMb2hhc1hv?=
 =?utf-8?B?QnNnSC95a0tsd0V6WFUrRmRMd3Yrb3J1R0RoQ2dPSThMVmpjbW0yeHpUSnlR?=
 =?utf-8?B?cHBzZzZ0aTdjQlJ2anB6WVlZNkJ2LzRmb2IwLzRMSUZqRitMWEdWbnFFQlk0?=
 =?utf-8?Q?hfNTyXWHCFk+wpBv1DDD3nTGIHlEhy3Fog0Tv4q?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89cc0893-c283-434c-344b-08d9832d9ad6
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:43:34.4751
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: phtsauR9LWLyIiwRZwJodX85sgDCFWqCROI1f4Clu1GF+nO0FyMIpS/CzRB7CraeDbKw4sqC7eve2CUwMN1sVw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3120

It's not really needed and has been misleading me more than once to try
and spot its "actual" use(s). It should really have been dropped when
the 32-bit specific logic was purged from here.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -327,16 +327,13 @@ static void show_guest_stack(struct vcpu
 
     if ( v != current )
     {
-        struct vcpu *vcpu;
-
         if ( !guest_kernel_mode(v, regs) )
         {
             printk("User mode stack\n");
             return;
         }
 
-        vcpu = maddr_get_owner(read_cr3()) == v->domain ? v : NULL;
-        if ( !vcpu )
+        if ( maddr_get_owner(read_cr3()) != v->domain )
         {
             stack_page = stack = do_page_walk(v, (unsigned long)stack);
             if ( (unsigned long)stack < PAGE_SIZE )



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:50:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:50:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198976.352750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWEi-0006D2-Aa; Wed, 29 Sep 2021 09:50:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198976.352750; Wed, 29 Sep 2021 09:50:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWEi-0006Cv-7U; Wed, 29 Sep 2021 09:50:48 +0000
Received: by outflank-mailman (input) for mailman id 198976;
 Wed, 29 Sep 2021 09:50:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVWEh-0006Cp-Bg
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:50:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVWEh-0005HU-9H
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:50:47 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVWEh-0004uA-8F
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:50:47 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVWEf-0006dJ-Iu; Wed, 29 Sep 2021 10:50:45 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Subject:CC:To:Date:Message-ID:
	Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=RNtcMmJ/l9k3QCli44qClrpgE0n4GrwL9Rc22SiJOdA=; b=196KT+AtWobipRJbhP+lWjFosT
	yFjvY4DwOx45TLskD13QFPNSMd5GOTX729UQoQopkogVL1YB8DwUfn6neYHY/cbYo7vjd84H5nlNA
	Lc1Nutv/r7vsH9cOmfk3kvFmmR4mvBl2RjdXJvAb0onv+0cT+egKDWQVCDJGqb4tqsdw=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24916.14069.358118.417330@mariner.uk.xensource.com>
Date: Wed, 29 Sep 2021 10:50:45 +0100
To: xen-devel@lists.xenproject.org
CC: committers@xenproject.org
Subject: osstest down, PDU failure
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Currently, osstest is not working.  We have lost one of our PDUs,
meaning that about half a rack is out of action, including one of the
VM hosts.

There has been quite a bit of outstanding maintenance which has been
deferred due to the pandemic.  I am trying to see if we can get
someone on-site to the colo, in Massachusetts, soon.  A complication
is that the replacement PDU is in still New York.  Again, due to the
pandemic.

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 09:57:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 09:57:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198982.352760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWLA-00072c-0z; Wed, 29 Sep 2021 09:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198982.352760; Wed, 29 Sep 2021 09:57:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWL9-00072V-UO; Wed, 29 Sep 2021 09:57:27 +0000
Received: by outflank-mailman (input) for mailman id 198982;
 Wed, 29 Sep 2021 09:57:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xzIg=OT=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVWL8-00072P-9v
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 09:57:26 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::626])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 134cf356-c946-425e-b1f1-27305a21cba1;
 Wed, 29 Sep 2021 09:57:23 +0000 (UTC)
Received: from AS9PR06CA0125.eurprd06.prod.outlook.com (2603:10a6:20b:467::21)
 by AM6PR08MB4534.eurprd08.prod.outlook.com (2603:10a6:20b:ba::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 09:57:22 +0000
Received: from VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:467:cafe::fa) by AS9PR06CA0125.outlook.office365.com
 (2603:10a6:20b:467::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 09:57:22 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT048.mail.protection.outlook.com (10.152.19.8) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 09:57:21 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Wed, 29 Sep 2021 09:57:18 +0000
Received: from 07b611582d59.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 40E3DC09-E419-4D77-93F2-C58DF2994371.1; 
 Wed, 29 Sep 2021 09:57:05 +0000
Received: from FRA01-MR2-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 07b611582d59.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 09:57:05 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR2PR08MB4716.eurprd08.prod.outlook.com (2603:10a6:101:28::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Wed, 29 Sep
 2021 09:57:03 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 09:57:03 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO3P123CA0006.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:ba::11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 09:57:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 134cf356-c946-425e-b1f1-27305a21cba1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nPlE21yHvhQGy9bHY5vx0+F3mBZzPuTgUiQXsij4Bsw=;
 b=ZAESu1wlDK+fAxVxNQWCBsyp/uov8oV5X0CKkaGdg+B88+EyQQyeQohlANskX8TqWKVbRtfHgiTTHP4fhfrXUe8FAw+Gytqyvrr0HAPyBwVEWqkp9ScDsMpfpdnZU6fJAyRgWTuWGntyGQm5SiVGUnwNT5tBRH3BmhHrbE7vHjo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fa68625dfd9d0556
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nzph+zflLcgsNr7hCJSi2P/O+VtygZVPtoDrzXHdlp9QPm+9fW0meLjSuWlFYC9HfjUeMlbxrTyjvJEnG+1uOYYC5/oNAzXcRvInylcVl7zWh5l1o2Y+hyTw1ibWyO+S0V1vXVztKcpZCvQqPlKk0QIeXCnLUNyiImUsdO4CsJTF/TS3UBha/WKUXa9lqLu+uCs8EOvwmyO/dtB1eYsT0L724mlqJc3EnGQQkfNmeNpZXT/8NOYP4yLtA0c+Q2b/OXbu8KbwU6XTDhs7l+d/oJBx+2t2OgZS2YYTdlj4WdK+s+SNc2BLvmgQd4X3P8MRJpHBpwZu79tOHhdrzfrfMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nPlE21yHvhQGy9bHY5vx0+F3mBZzPuTgUiQXsij4Bsw=;
 b=iOZqRt9a9+sge7tsKv5kzLndd0S1XMNFiwsFVlNxchtAlQP1XUf+VK2shq9w9Mt7CbZwS0GL2qzgNpwfecf4WCk6XAOC654vvEBCZy9NuurBQyid2sJmkKaxt0sVWe9/am26xKPaaNUS8MJ4iG7fOY+vr3UM5JMvLuY1joZ/3T+7Ff5uL9XIoSFOIJVPcPPM3+vrXoycV3+tqaeZFnz/pIa5YRJuCfrp1zqWs4fiZStAMZu0sROdSfpcKmoON7V2F8S0gxkxRsH9hqqQBivf4Jxjh1OWdQyYxUrK4vgvvFajE80KZJPq9hG6z9Qg6IjhNPoXzMaBzBmhFGJBV/gxsw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nPlE21yHvhQGy9bHY5vx0+F3mBZzPuTgUiQXsij4Bsw=;
 b=ZAESu1wlDK+fAxVxNQWCBsyp/uov8oV5X0CKkaGdg+B88+EyQQyeQohlANskX8TqWKVbRtfHgiTTHP4fhfrXUe8FAw+Gytqyvrr0HAPyBwVEWqkp9ScDsMpfpdnZU6fJAyRgWTuWGntyGQm5SiVGUnwNT5tBRH3BmhHrbE7vHjo=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v3 1/3] arm/efi: Introduce uefi,cfg-load DT property
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <AB8FEF63-EA1F-427F-AF2B-13C1E930F682@arm.com>
Date: Wed, 29 Sep 2021 10:56:56 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <46255EF5-17E6-41AC-A6D9-5D276056C0DA@arm.com>
References: <alpine.DEB.2.21.2109281459250.5022@sstabellini-ThinkPad-T480s>
 <AB8FEF63-EA1F-427F-AF2B-13C1E930F682@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO3P123CA0006.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:ba::11) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8990b43b-f3fe-4805-cfd0-08d9832f880f
X-MS-TrafficTypeDiagnostic: PR2PR08MB4716:|AM6PR08MB4534:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB45349153581CA368E2763237E4A99@AM6PR08MB4534.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 GHZwp+liuNA5uLD0NN6x+kDQnE5VLyt4rlxIrT4bnH52AdmE21FOZDqD9rupOcGi9Gp3epD5puDUrZDN2qN1GqNYF0lUA9NsrfrrdjCJMxIlgk+9IPXt2Xp/oTFOV3cn5lnl1OakS3o8G+H7e/oNmQqaKaSn961UdKd+MXTiLCRNi4ijUEQIT9pSxRUKOKhcXzKUoduZGUIfNM8jnk9n/NHQ0/6riZhOFi/zOR/n/Wno3ZPFkjMPTNtpmNWPUpPx75EviVvDByX48g3ODXKrKdRpS/FyzPWnDyrd/JGJXcCRYvyuBgCxmMzrSm2puXD3fImNTeZbeZ/0R9Uv3Y7YxhQ5/TtB5vDT8eobZVAuQlhRZrUL0Tr5aWoVvsMf991dvqNrA+snEnVHP9MD7yvGcEqKneB5LF7rF2yokZXoX5QkuRdKRukGGij9MbDHJybnYjkxIBFcqGV9hpscgwq6zLFKoi23ol+1GzYJaLDXJdXaUpa5lnqZjOvEAhTseJ6QdeYlIrqrYWKPObergPTplBtplr4Tv3/TjE4AyyNOocoSozCLKGJcPpZKSB9gEwfxUZ1WJoDc+dvmTN6LJcaurkibOlKpUIi0X8y2i/d5TO1YNccV+l8s3NDyV4YcuGTP5VSHTrnDN45A1SXgNKDJe+dYU26wwXcB6WKya5U3jH5SATMTmMzpArbHik3Bv1pLHdC7Pdx9grsdFjgFMYWMgqTmhfZOVNyp20fEqE6OrxU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(44832011)(508600001)(316002)(86362001)(6666004)(33656002)(6486002)(6512007)(5660300002)(6916009)(36756003)(8936002)(66556008)(2906002)(956004)(8676002)(186003)(6506007)(66476007)(4326008)(66946007)(54906003)(2616005)(38350700002)(38100700002)(52116002)(26005)(83380400001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4716
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	920b8cba-71bf-4a07-478d-08d9832f7cd8
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/DpjiD0xhh0V88VYI9Wpi1OvrCqWNY4Lv30o2wBrSVLmgU1yoLILOZnbRm1jFpRy4TX/pI9FoyY4uUuuWFdwgGjnAthbsmKAbEQd8nt9DoJ7ghC7Ph+d+FtYEVkLG4sXeL7UE/1nEwQOANFBLrjkCi2jIBcLttMgArF1CiPfUqXBT9TEcgC4DZ0n5Wad/0FlYAtY8jZEm4XAOVeB/So6sesEHGD0v6wFL+ybssAANcbrOJsxjWDygTE4aOEEUWA0HP0QANv71/Z28S0FZN7QA72VGkSMgIL+xW0xeD6hE39jSlyR6PD7lgkJCpnlSlagpDGcvWFBs9dJfccb36lurlxavRGzO34E10tzJCA3dIaKtJRT3Dgoo0gyvCSOHJD/shoHFnFf9lkNRwwlP7Bt4QlZewOmGXGJX+7sDA2sqELrzRVUjOZZwGXELoeVwVP1RduqTMe2h2ar+FxNObS7RSUNvL7GvhPE0dGYctOSTe4H2120+aNKvI0dpbXombg4ExaXRtpNUdQCb/o52/7RDTfBpTahaLR55wI49UKqqQ9dlUxtzdmhXwVueipzeJfNmlJ+i3qeB11H8p2Jtr12YY9kZEdLeiAZDGB4tOLTxmvaKI9Zry0Q4l3eeRQ3tfOuzsCMwNIuN8hncsmPDpuRUu2RE62BEBvTheSLIalL6yX4f84HBdD/CfjvzrkU568/rzUMQZSY0OnaxdbaED65Zg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6666004)(8936002)(33656002)(86362001)(186003)(508600001)(356005)(36860700001)(336012)(5660300002)(6862004)(6506007)(316002)(47076005)(4326008)(83380400001)(70586007)(2616005)(8676002)(956004)(26005)(2906002)(70206006)(36756003)(81166007)(44832011)(82310400003)(6486002)(54906003)(6512007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 09:57:21.7585
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 8990b43b-f3fe-4805-cfd0-08d9832f880f
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4534

> On Tue, 28 Sep 2021, Luca Fancellu wrote:
>> Introduce the uefi,cfg-load DT property of /chosen
>> node for ARM whose presence decide whether to force
>> the load of the UEFI Xen configuration file.
>>=20
>> The logic is that if any multiboot,module is found in
>> the DT, then the uefi,cfg-load property is used to see
>> if the UEFI Xen configuration file is needed.
>>=20
>> Modify a comment in efi_arch_use_config_file, removing
>> the part that states "dom0 required" because it's not
>> true anymore with this commit.
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>=20
> The patch looks good. Only one minor change: given that this is a Xen
> parameter that we are introducing and not a parameter defined by UEFI
> Forum, I think uefi,cfg-load should be called xen,uefi-cfg-load instead.
> Because "xen," is our prefix, while "uefi," is not.
>=20
> With that minor change:
>=20
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Yes I will rename it.

>=20
>=20
> Note that the uefi,binary property is different because that property is
> for xen,domain nodes, so we are already in a Xen specific namespace when
> we are using it. Instead this property is for /chosen which is not a Xen
> specific node.

Given that uefi,binary will be used also for multiboot,module directly unde=
r /chosen
for Dom0, do you think I should rename also that to xen,uefi-binary?

Cheers,
Luca

>=20
>=20
>=20
>> ---
>> v3 changes:
>> - add documentation to misc/arm/device-tree/booting.txt
>> - Modified variable name and logic from skip_cfg_file to
>> load_cfg_file
>> - Add in the commit message that I'm modifying a comment.
>> v2 changes:
>> - Introduced uefi,cfg-load property
>> - Add documentation about the property
>> ---
>> docs/misc/arm/device-tree/booting.txt |  8 ++++++++
>> docs/misc/efi.pandoc                  |  2 ++
>> xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
>> 3 files changed, 33 insertions(+), 5 deletions(-)
>>=20
>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/devic=
e-tree/booting.txt
>> index 44cd9e1a9a..cf878b478e 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs f=
or Xen, xen,dom0-bootargs
>> for Dom0 and bootargs for native Linux.
>>=20
>>=20
>> +UEFI boot and DT
>> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> +
>> +When Xen is booted using UEFI, it doesn't read the configuration file i=
f any
>> +multiboot module is specified. To force Xen to load the configuration f=
ile, the
>> +boolean property uefi,cfg-load must be declared in the /chosen node.
>> +
>> +
>> Creating Multiple Domains directly from Xen
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>=20
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index ac3cd58cae..e289c5e7ba 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tr=
ee provided to Xen.  If a
>> bootloader provides a device tree containing modules then any configurat=
ion
>> files are ignored, and the bootloader is responsible for populating all
>> relevant device tree nodes.
>> +The property "uefi,cfg-load" can be specified in the /chosen node to fo=
rce Xen
>> +to load the configuration file even if multiboot modules are found.
>>=20
>> Once built, `make install-xen` will place the resulting binary directly =
into
>> the EFI boot partition, provided `EFI_VENDOR` is set in the environment =
(and
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index cf9c37153f..4f1b01757d 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LO=
ADED_IMAGE *loaded_image)
>>=20
>> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTabl=
e)
>> {
>> +    bool load_cfg_file =3D true;
>>     /*
>>      * For arm, we may get a device tree from GRUB (or other bootloader)
>>      * that contains modules that have already been loaded into memory. =
 In
>> -     * this case, we do not use a configuration file, and rely on the
>> -     * bootloader to have loaded all required modules and appropriate
>> -     * options.
>> +     * this case, we search for the property uefi,cfg-load in the /chos=
en node
>> +     * to decide whether to skip the UEFI Xen configuration file or not=
.
>>      */
>>=20
>>     fdt =3D lookup_fdt_config_table(SystemTable);
>>     dtbfile.ptr =3D fdt;
>>     dtbfile.need_to_free =3D false; /* Config table memory can't be free=
d. */
>> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,modul=
e") < 0 )
>> +
>> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 =
)
>> +    {
>> +        /* Locate chosen node */
>> +        int node =3D fdt_subnode_offset(fdt, 0, "chosen");
>> +        const void *cfg_load_prop;
>> +        int cfg_load_len;
>> +
>> +        if ( node > 0 )
>> +        {
>> +            /* Check if uefi,cfg-load property exists */
>> +            cfg_load_prop =3D fdt_getprop(fdt, node, "uefi,cfg-load",
>> +                                        &cfg_load_len);
>> +            if ( !cfg_load_prop )
>> +                load_cfg_file =3D false;
>> +        }
>> +    }
>> +
>> +    if ( !fdt || load_cfg_file )
>>     {
>>         /*
>>          * We either have no FDT, or one without modules, so we must hav=
e a
>> -         * Xen EFI configuration file to specify modules.  (dom0 requir=
ed)
>> +         * Xen EFI configuration file to specify modules.
>>          */
>>         return true;
>>     }
>> --=20
>> 2.17.1
>>=20




From xen-devel-bounces@lists.xenproject.org Wed Sep 29 10:04:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 10:04:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198993.352784 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWRb-0000JK-1d; Wed, 29 Sep 2021 10:04:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198993.352784; Wed, 29 Sep 2021 10:04:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVWRa-0000JD-T8; Wed, 29 Sep 2021 10:04:06 +0000
Received: by outflank-mailman (input) for mailman id 198993;
 Wed, 29 Sep 2021 10:04:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xzIg=OT=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVWRZ-0000J2-2O
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 10:04:05 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7e1a::62c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 711fef06-2b91-4447-9ed1-fec14ef88117;
 Wed, 29 Sep 2021 10:04:02 +0000 (UTC)
Received: from AS9PR06CA0015.eurprd06.prod.outlook.com (2603:10a6:20b:462::6)
 by AM5PR0802MB2515.eurprd08.prod.outlook.com (2603:10a6:203:9f::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 10:03:58 +0000
Received: from VE1EUR03FT047.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:462:cafe::98) by AS9PR06CA0015.outlook.office365.com
 (2603:10a6:20b:462::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 10:03:57 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT047.mail.protection.outlook.com (10.152.19.218) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 10:03:56 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Wed, 29 Sep 2021 10:03:53 +0000
Received: from 2f8a9b4fb81e.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 A120EDD4-E026-46F4-B277-AE70494909B4.1; 
 Wed, 29 Sep 2021 10:03:39 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2f8a9b4fb81e.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 10:03:39 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6927.eurprd08.prod.outlook.com (2603:10a6:102:133::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 10:03:36 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 10:03:36 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO2P265CA0513.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13b::20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.15 via Frontend Transport; Wed, 29 Sep 2021 10:03:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 711fef06-2b91-4447-9ed1-fec14ef88117
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wWT9PM0Vv9G5RAU1SZ2ct5762l6Ys9Hi/rjB47VZijs=;
 b=6AY+kZlL0PI6GT18cosx2SEHkzsSZogzc3gUj0ZAagtSvuR58lETiMld0eSBu8VMPwrT1IrMF0MS/l3J7ldotj/s8BgH5VLoZUxyEVItQANlNxwaq7zxQGiGGwwYM26YXaQjFYYY3iDeke90buvCRglbsLgClrOaqXGdMIqtFGo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 4418426482f44a76
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dtlKQlt9z5bqGCMbLiv6hy/nMRo7KrdQtdjits4GjsMIlV6uO3ksie17shH8gOmXZu5yqoIemeVhGmXeirnI7XWnsxJFrUGjcoXDFTvBgflkP7+gemX01zc8PcE4CRQvzerg28WqaV5ABguIQ6CrEHf/B8AWGvVip35a/EbPBwtuM7MEGQlK2yrliRktA8FqtM02QK7DyQBQ1WoAJTRG0LDNnu3Z9cCWZRkTY3DvZQQ8iELpeGqdzR5VXwliYpY83YJoqttqx0/vdDKx1Au0EE0BoYO96Tl4Qw+EmTL9B/sQOmtHNnrCMnbdiJjQ0xhrvpGr+QYOv16DeP8vHOwolQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wWT9PM0Vv9G5RAU1SZ2ct5762l6Ys9Hi/rjB47VZijs=;
 b=ALebTyhRa8fS6uSegiFmbUVsXeUvzTXpRlEVQY3eJCUMiHR9gkFl1vHRfPrMklpoyg16Itr9rDrz/67XfK4hivTs2rG8dJRP+nd/idz3gshMZFldAKBXynEI5ykgxMqOKAi6hgzgvUMlRdQkNYG0JB4o2EKhAQJqDctx1s1mLYIzz2VPSe6E037J19SE5YdoOrijAZh/j1noT2KdSWUF1HmdG6j4r6SNbCEYlx+5oA4PXKQOd2REcQwWaWo1aJARK8h4KPbcLTowy213Ua30zpjTb16lRg+sOqQ740oxX92LYH9VaKSb2Xrx0kDXUot33Nu6fThE25pwZ5y+/6vgqw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wWT9PM0Vv9G5RAU1SZ2ct5762l6Ys9Hi/rjB47VZijs=;
 b=6AY+kZlL0PI6GT18cosx2SEHkzsSZogzc3gUj0ZAagtSvuR58lETiMld0eSBu8VMPwrT1IrMF0MS/l3J7ldotj/s8BgH5VLoZUxyEVItQANlNxwaq7zxQGiGGwwYM26YXaQjFYYY3iDeke90buvCRglbsLgClrOaqXGdMIqtFGo=
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v3 3/3] arm/efi: load dom0 modules from DT using UEFI
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1E500DE6-1680-41F6-BE8E-5B7ECBAD24BA@arm.com>
Date: Wed, 29 Sep 2021 11:03:30 +0100
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <69B0AE5A-D327-4224-B7CA-79844C25FD2C@arm.com>
References: <alpine.DEB.2.21.2109281534480.5022@sstabellini-ThinkPad-T480s>
 <1E500DE6-1680-41F6-BE8E-5B7ECBAD24BA@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0513.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:13b::20) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7427ae72-bddb-4824-beb7-08d983307392
X-MS-TrafficTypeDiagnostic: PAXPR08MB6927:|AM5PR0802MB2515:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<AM5PR0802MB25155BF8FC95B1CE7BCE0BF4E4A99@AM5PR0802MB2515.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 41Q+3PFo0Ojk7vQ8s8jw9rpx/uqr0pMRAh0NkP4aPUN+BsUiipYmfmDrg16B9Gma+KSeiRGmfIUhIBWLihbQ64/D4+QyOj2L3JTn/vPMkGKnwUJg1zvwFnhdPCavOCc59vCLKsE30dpc6BgTKLFo258v8Ftxg/jJGbmSnbfW4iGU+GHxjYfl9JEsB/EZl3QQiDhW9kM2BLAb46F2THEgwY3n4SGUH50cK3X8o8BGoDl/iP/V2CLRr1zJhH6WtddXNaEUC0u7QwO3nx6tXCH5V6XiBEtgCVGdxzM/YxWuQuLeqK+ecEhhoFqXWHk0qmhpfqrjjsDhpVk74EvD9GtYOg3ObpkmJP2m3ltVgc4CeOhdWTcQbrB8EkIKMaA+zMZGjlGO1qvDW/X1WR10C11B5kwADjqmhqGJixEScD+dWmMUC7liQjE9cCbBs35FHKXJlViXXmkMnMtalfc3rhUw7nm3RgGrfBEMPZ/OuukxPCX0IoYWkI+LuiJTXgpJPAy5IzkrJHIGjvbjwCl7h/aIdXAPUNNBSqBiKqUHNmxE0YjlYTL/JXOi8/cNaaoNZAj9/uNHykix2t6ceeEuoTQ9QcwwtKAEOM3trQUyXVbPpFhyAy7DEQ0mGQ5FPUzZSUqtk031ETwM2qOudtNvljnptaxuYTnUFKDjpi7VBW4K0pZL/EYHWuPlnkKOFym2eqApDJG+JRZs9WNZRH0zhflApPg5Xh0J1MX9L7jbjsicjCyxBWMPbqqw9wF8Rp79U+eZ
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(36756003)(33656002)(316002)(54906003)(186003)(52116002)(6916009)(8676002)(38350700002)(6506007)(38100700002)(26005)(8936002)(6486002)(4326008)(5660300002)(2616005)(956004)(86362001)(66556008)(66476007)(6666004)(44832011)(83380400001)(66946007)(2906002)(6512007)(508600001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6927
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	75a3333b-bd63-4ee2-24f9-08d983306766
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nw/yT0XrvTv/q/TVt3S01mf5ZxG3yTtgSCA4vSB/yZyeK+YOrJG+TFquONxGvLdj2+3fKyBlE4vR/ARxoNnrOlfKqfE6lYFnkJ/o8p4Vf+Znwv9VwDEPY+jz1mAXQ2S8A9TJ5GdLbbr8p5QMuqVgq1ZUnZNMmOlp9HNyGsNAy7y68DwF8b4k4TS1PnZ3mo2zoK2m6UsHQpp+Q3mdkDpENmOOVMNw5GoOe0W1aX0el9+zzXQOidz+V58np+I5XLIvX8vcAYGvHYK5hFrmpUiVGvrU/VI+7jGNWy5TK7dZuEgvja7il8B3bhGYeJu2ADgH4so6EXhemaRZWCmz2doy27Ebfiap8/AiYQ45yWJJYxa6ZD6jY7g42Oiab8pdVd9cTdQK9uuBMN3Z6jvmwYtfp67WhYiT5yoDxONKZNzvMTG0+5SRzDqpZ7O1zQE0dyA8HA2/euKaSpnIhAVV5O2wzMjnMWQ099xQy7UUMZQCxHA81c/5aoP1VkXaVLqYZb5FHF5e1rOt3l0AEN9eSBkkI+52e6o1YyY4IMQaNu9qULI4kh/LAMV+Fni6W451RzJ8CDJyhHgCzg8muPtK57ktf6L0cTQtEcI8MESegzLK7G4rdMIDL9NUUmafUVHxh7GUMJ3HIi/lzmm4naiULuQxbPK3TDMGI2R7427TxfjwaHY4v4Fz/sNt/mvCbR8dlJZ5+IMGhA9dEKIvX/6zBFOHww==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(83380400001)(336012)(2616005)(6512007)(81166007)(5660300002)(54906003)(8676002)(6666004)(47076005)(6506007)(956004)(8936002)(44832011)(70586007)(186003)(33656002)(26005)(82310400003)(316002)(86362001)(36860700001)(4326008)(2906002)(6862004)(36756003)(70206006)(508600001)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 10:03:56.8852
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7427ae72-bddb-4824-beb7-08d983307392
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT047.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2515

> On Tue, 28 Sep 2021, Luca Fancellu wrote:
>> Add support to load Dom0 boot modules from
>> the device tree using the uefi,binary property.
>>=20
>> Update documentation about that.
>>=20
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>=20
> It is great how simple this patch is!
>=20
> The patch looks all correct. Only one question: do we need a check to
> make sure the dom0 ramdisk is not loaded twice? Once via uefi,binary and
> another time via the config file? In other words...

Yes I will add a check for it

>=20
>> ---
>> Changes in v3:
>> - new patch
>> ---
>> docs/misc/arm/device-tree/booting.txt |  8 ++++
>> docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
>> xen/arch/arm/efi/efi-boot.h           | 36 +++++++++++++--
>> xen/common/efi/boot.c                 | 12 ++---
>> 4 files changed, 108 insertions(+), 12 deletions(-)
>>=20
>> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/devic=
e-tree/booting.txt
>> index 354bb43fe1..e73f6476d4 100644
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -70,6 +70,14 @@ Each node contains the following properties:
>> priority of this field vs. other mechanisms of specifying the
>> bootargs for the kernel.
>>=20
>> +- uefi,binary (UEFI boot only)
>> +
>> + String property that specifies the file name to be loaded by the UEFI
>> + boot for this module. If this is specified, there is no need to specif=
y
>> + the reg property because it will be created by the UEFI stub on boot.
>> + This option is needed only when UEFI boot is used, the node needs to b=
e
>> + compatible with multiboot,kernel or multiboot,ramdisk.
>> +
>> Examples
>> =3D=3D=3D=3D=3D=3D=3D=3D
>>=20
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index 800e67a233..4cebc47a18 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -167,6 +167,28 @@ sbsign \
>> --output xen.signed.efi \
>> xen.unified.efi
>> ```
>> +## UEFI boot and Dom0 modules on ARM
>> +
>> +When booting using UEFI on ARM, it is possible to specify the Dom0 modu=
les
>> +directly from the device tree without using the Xen configuration file,=
 here an
>> +example:
>> +
>> +chosen {
>> + #size-cells =3D <0x1>;
>> + #address-cells =3D <0x1>;
>> + xen,xen-bootargs =3D "[Xen boot arguments]"
>> +
>> + module@1 {
>> + compatible =3D "multiboot,kernel", "multiboot,module";
>> + uefi,binary =3D "vmlinuz-3.0.31-0.4-xen";
>> + bootargs =3D "[domain 0 command line options]";
>> + };
>> +
>> + module@2 {
>> + compatible =3D "multiboot,ramdisk", "multiboot,module";
>> + uefi,binary =3D "initrd-3.0.31-0.4-xen";
>> + };
>> +}
>>=20
>> ## UEFI boot and dom0less on ARM
>>=20
>> @@ -326,10 +348,10 @@ chosen {
>> ### Boot Xen, Dom0 and DomU(s)
>>=20
>> This configuration is a mix of the two configuration above, to boot this=
 one
>> -the configuration file must be processed so the /chosen node must have =
the
>> -"uefi,cfg-load" property.
>> +the configuration file can be processed or the Dom0 modules can be read=
 from
>> +the device tree.
>>=20
>> -Here an example:
>> +Here the first example:
>>=20
>> Xen configuration file:
>>=20
>> @@ -369,4 +391,40 @@ chosen {
>> };
>> ```
>>=20
>> +Here the second example:
>> +
>> +Device tree:
>> +
>> +```
>> +chosen {
>> + #size-cells =3D <0x1>;
>> + #address-cells =3D <0x1>;
>> + xen,xen-bootargs =3D "[Xen boot arguments]"
>> +
>> + module@1 {
>> + compatible =3D "multiboot,kernel", "multiboot,module";
>> + uefi,binary =3D "vmlinuz-3.0.31-0.4-xen";
>> + bootargs =3D "[domain 0 command line options]";
>> + };
>> +
>> + module@2 {
>> + compatible =3D "multiboot,ramdisk", "multiboot,module";
>> + uefi,binary =3D "initrd-3.0.31-0.4-xen";
>> + };
>> +
>> + domU1 {
>> + #size-cells =3D <0x1>;
>> + #address-cells =3D <0x1>;
>> + compatible =3D "xen,domain";
>> + cpus =3D <0x1>;
>> + memory =3D <0x0 0xc0000>;
>> + vpl011;
>>=20
>> + module@1 {
>> + compatible =3D "multiboot,kernel", "multiboot,module";
>> + uefi,binary =3D "Image-domu1.bin";
>> + bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
>> + };
>> + };
>> +};
>> +```
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index 4f7c913f86..df63387136 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -31,8 +31,10 @@ static unsigned int __initdata modules_idx;
>> #define ERROR_MISSING_DT_PROPERTY   (-3)
>> #define ERROR_RENAME_MODULE_NAME    (-4)
>> #define ERROR_SET_REG_PROPERTY      (-5)
>> +#define ERROR_DOM0_ALREADY_FOUND    (-6)
>> #define ERROR_DT_MODULE_DOMU        (-1)
>> #define ERROR_DT_CHOSEN_NODE        (-2)
>> +#define ERROR_DT_MODULE_DOM0        (-3)
>>=20
>> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>> void __flush_dcache_area(const void *vaddr, unsigned long size);
>> @@ -45,7 +47,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_ha=
ndle,
>> static int handle_module_node(EFI_FILE_HANDLE dir_handle,
>>                               int module_node_offset,
>>                               int reg_addr_cells,
>> -                              int reg_size_cells);
>> +                              int reg_size_cells,
>> +                              bool is_domu_module);
>> static bool is_boot_module(int dt_module_offset);
>> static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>>                                        int domain_node);
>> @@ -701,7 +704,8 @@ static int __init allocate_module_file(EFI_FILE_HAND=
LE dir_handle,
>> static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>>                                      int module_node_offset,
>>                                      int reg_addr_cells,
>> -                                     int reg_size_cells)
>> +                                     int reg_size_cells,
>> +                                     bool is_domu_module)
>> {
>>     const void *uefi_name_prop;
>>     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + =
\0 */
>> @@ -743,6 +747,24 @@ static int __init handle_module_node(EFI_FILE_HANDL=
E dir_handle,
>>         return ERROR_SET_REG_PROPERTY;
>>     }
>>=20
>> +    if ( !is_domu_module &&
>> +         (fdt_node_check_compatible(fdt, module_node_offset,
>> +                                    "multiboot,kernel") =3D=3D 0) )
>> +    {
>> +        /*
>> +         * This is the Dom0 kernel, wire it to the kernel variable beca=
use it
>> +         * will be verified by the shim lock protocol later in the comm=
on code.
>> +         */
>> +        if ( kernel.addr )
>> +        {
>> +            PrintMessage(L"Dom0 kernel already found in cfg file.");
>> +            return ERROR_DOM0_ALREADY_FOUND;
>> +        }
>> +        kernel.need_to_free =3D false; /* Freed using the module array =
*/
>> +        kernel.addr =3D file->addr;
>> +        kernel.size =3D file->size;
>> +    }
>=20
> ... is it necessary to update ramdisk as well or check if it is already
> set? As far as I can tell it is the only other potential conflict that
> we could have.

ramdisk variable can be left unset, we are setting the kernel variable only=
 to
reuse the code that verifies the kernel image, however if the user specify =
the
ramdisk in both cfg and device tree we will end up in two multiboot,ramdisk
modules being added to the device tree.
I will add a check to return an error if ramdisk was declared previously.

>=20
>=20
>>     return 0;
>> }
>>=20
>> @@ -799,7 +821,7 @@ static int __init handle_dom0less_domain_node(EFI_FI=
LE_HANDLE dir_handle,
>>         if ( is_boot_module(module_node) )
>>         {
>>             int ret =3D handle_module_node(dir_handle, module_node, addr=
_cells,
>> -                                         size_cells);
>> +                                         size_cells, true);
>>             if ( ret < 0 )
>>                 return ret;
>>         }
>> @@ -809,7 +831,7 @@ static int __init handle_dom0less_domain_node(EFI_FI=
LE_HANDLE dir_handle,
>>=20
>> /*
>>  * This function checks for xen domain nodes under the /chosen node for =
possible
>> - * domU guests to be loaded.
>> + * dom0 and domU guests to be loaded.
>>  * Returns the number of modules loaded or a negative number for error.
>>  */
>> static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>> @@ -836,6 +858,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_H=
ANDLE dir_handle)
>>             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>>                 return ERROR_DT_MODULE_DOMU;
>>         }
>> +        else if ( is_boot_module(node) )
>> +        {
>> +            if ( handle_module_node(dir_handle, node, addr_len, size_le=
n,
>> +                                    false) < 0 )
>> +                return ERROR_DT_MODULE_DOM0;
>> +        }
>>     }
>>=20
>>     /* Free dom0less file names if any */
>> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
>> index c8c57fbb54..b221494a06 100644
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>>         {
>>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>>             efi_bs->FreePool(name.w);
>> -
>> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NUL=
L,
>> -                            (void **)&shim_lock)) &&
>> -                 (status =3D shim_lock->Verify(kernel.ptr, kernel.size)=
) !=3D EFI_SUCCESS )
>> -                PrintErrMesg(L"Dom0 kernel image could not be verified"=
, status);
>>         }
>>=20
>>         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
>> @@ -1372,6 +1367,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>>     if (dt_module_found < 0)
>>         /* efi_arch_check_dt_boot throws some error */
>>         blexit(L"Error processing boot modules on DT.");
>> +
>> +    /* If Dom0 is specified, verify it */
>> +    if ( kernel.ptr &&
>> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
>> +                                           (void **)&shim_lock)) &&
>> +        (status =3D shim_lock->Verify(kernel.ptr, kernel.size)) !=3D EF=
I_SUCCESS )
>> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status=
);
>>     /*
>>      * Check if a proper configuration is provided to start Xen:
>>      *  - Dom0 specified (minimum required)
>> --=20
>> 2.17.1




From xen-devel-bounces@lists.xenproject.org Wed Sep 29 10:53:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 10:53:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.198999.352795 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXDO-0005rW-QZ; Wed, 29 Sep 2021 10:53:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 198999.352795; Wed, 29 Sep 2021 10:53:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXDO-0005rP-NK; Wed, 29 Sep 2021 10:53:30 +0000
Received: by outflank-mailman (input) for mailman id 198999;
 Wed, 29 Sep 2021 10:53:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVXDN-0005rJ-Mc
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 10:53:29 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 36040277-d798-444c-b842-3169c871b1e3;
 Wed, 29 Sep 2021 10:53:28 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-h_Bu1teEP0-WXqwyzVOmRg-1; Wed, 29 Sep 2021 12:53:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5167.eurprd04.prod.outlook.com (2603:10a6:803:5b::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 10:53:24 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 10:53:24 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR10CA0096.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:8c::37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 10:53:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36040277-d798-444c-b842-3169c871b1e3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632912807;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aBvmVN+IpqsWcOvuRmgv053p8T677yHv+WGh+bolzo4=;
	b=DKHaBRsdm+45On1uHlPHRtwkAkRdkv7YSDxIx3CtftgXnQISHlPuM5Iuzx9BJnW09G2vSz
	mj77Kk2pJpnhDKX6riRut9TerUfw4++6aiJbTu4QV3FbdJBZSYqghigQUGMDs0vyJPNIZp
	xcGg1g7ZRr50OlJlLj1xvc7MEOkKLgc=
X-MC-Unique: h_Bu1teEP0-WXqwyzVOmRg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FQh+EArVp4u1VKtoH7ngVnuvRYmo31hFcR/vdfMD3YgbOjOb6xnwqRev5//CSeHggyEu2kSH23GoBX5rzAOyJreubO85C5DNq7QAWSQEv/INYW6y9UQapk6pl9Ta2ifultUy310j8z0rhYHYZ0d92+UOweS/x9klT5nn8DewjFoeQChZlCfMuub/k/GvW0D9tdch/prAVNaZiEgpIy5tWhrthLuhcw4uwsx4RnYUkj5baMwvKLzp8OyoHHI1rh+IGCrPQFP8fLQn0zp3ryJc3V1dBWU1q5Gqsb9enZu+kZjK6SCD4GRK250+XPKcLhTp+unZmfjPQMjqwJb9N35JJw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=EkUe7PDWn48Y6jUAL8tFmgb8Z61St5FYaV0Td7l5Dl0=;
 b=SmtBcb9HIVuIzMQaV6XEBBvsKgMh1lOe61tWcnncah/afh7IX8K/Z0vKXqQwEFzkH4hynZNxamtI7YjRSMphvSSOIwSFUig2zpau+Icggkdli8Zzsg2jia/ZMxI+A2VjWBJQJ1Alg7TNfUtuEvWVYqasBfLhDS2BKVCXuA7rXb/dXqc8kHV9yxfuWmC9cxzEqLYyHWPq7W5wg2+oXg7dezOfA2GnYf9NUmDYdX10/Vum7Q5eKDdq14eD1D0mjFAmgMCTwirUGFLWaEDTo7CaBbPUl/os2l+AC3HDb2ZD/xbsKaewXc8PbWYvnudSabzMJF4tUH5CLEFivtHn7+CzFQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 1/9] x86/PVH: improve Dom0 memory size calculation
To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <e4959bab-0e0b-1037-c5da-3d2f14592c20@suse.com>
 <562a9ee5-102e-ee25-e4cd-721e44b7409a@suse.com>
 <YUsapfsfoL+ODa6+@MacBook-Air-de-Roger.local>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <6d64b491-cfb1-addb-9f5b-e1447d3797da@suse.com>
Date: Wed, 29 Sep 2021 12:53:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YUsapfsfoL+ODa6+@MacBook-Air-de-Roger.local>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM6PR10CA0096.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:209:8c::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a06b4060-393c-4839-f307-08d983375c3d
X-MS-TrafficTypeDiagnostic: VI1PR04MB5167:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5167F2270B22E548DE0650FDB3A99@VI1PR04MB5167.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1227;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	m0+EH2pyWmbTvEY6SqZVVMqJgiEB1a35GF0LwhGDBoZPeRUbpFHuYT2rNFQdSgZFAquMJNQpVyvFFuGokveCiSMPBJ00Ro6VrmLNFm1vMpQbnve4/jgNBPjpu8lPnvbkqctAgqUbNfUa0ZFfR1xiw2xToAoqvhdtoTqB3U8q5Mt3xZ8/sNxsCdPJs4SM8UEcXIqaEeXbzPJNsFwjcw9LPt5lBTUmbKZf7iyQQiAn65xYAAIQ/e7znDgWCwfPZoyp/c+i0m2D6rlGMhvGsoHHnf8SeiUUHUq540vHlhCgQDwGNz0w1wbwRQvadxlg12/cQpZITgHAOWdJVdpHuhOG5Mw/N9Q2B9bVMYpMUajc5DKsmcvA9sqtI+OZTWyaRo4uoOXr5HivG/9FUGwOMwvXpYN0iDM+okTyu+TcS0gjAzQhpuMshB0bnYuF4khg++yQlcfyoBOQfHVgsmJmqtQ5OA+MXntR4qb1TnZ9MNUujB95wV2xpu6cziIZptCIR9Tkpesw2361oC8TCUsVlGtCaYcrl+P9GCW0KSrhxdV5RLAJsqJLzRQG98ea50bv3jzijMi7Ck9SemP3VC016xbCI0geVX5HZNqJVJdjdb2Mi0WdKetSLB3Qvbkwui4li2K7OpCygKBOqsgp9Pdg0/KU/cDJj1s/51hY5KFpO/9KKXg5WyAKdTUQvqoM2Njt8CaovuhbgVInBzkORc7uE1m6mnvBqbcBAG7qJJ34yrs6cUyWu/Es98zVSDkDmvgNMYBF
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(956004)(5660300002)(66476007)(54906003)(2616005)(2906002)(16576012)(6486002)(316002)(38100700002)(66556008)(6916009)(26005)(508600001)(31696002)(53546011)(186003)(8676002)(83380400001)(8936002)(36756003)(66946007)(31686004)(86362001)(25903002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?lLwuMkGts4sEXVKpuESQs0beP1ERs6MhokOLEiik1ObenXxa+ZVk2sVGn2+Z?=
 =?us-ascii?Q?iPPOeVfH4mbEyfUz978mT8QYU/lenistLfqlBjo55ZTNlz3cIbo9Ly1rrtxe?=
 =?us-ascii?Q?vjGjjPn1zEHYLwPmRVdUws+LiGYfrsox6S4E0Nbmrq5p31gtAmAq+L0d87Ue?=
 =?us-ascii?Q?X9jmgKczt04dpn3koQZ9W2vAORvgk+docInaUmnZ5bkKi1GfuXxQPj5LH9LT?=
 =?us-ascii?Q?EYJvlYjo27qOp7S0d3lyTuFJu1Y0E0kUo1JnVo3fauCB63muNwe6Ctpx7W5H?=
 =?us-ascii?Q?R/eQ784/NK9cYNpMr2MOcTasBdhxUy5XhCPz8JFPQ/kToeuDUhCAIFijWOyM?=
 =?us-ascii?Q?P9Hgn5hCqpvJ8Jv9zbbaCnLwkIJVzgw0gThr6McnbZmUqsqd6wqPkmKbETHc?=
 =?us-ascii?Q?IVxEDIy+gH3Ym9EoNbZPXwOVhS3aIXlJdifgLv/dCBQoJ18TDdxu4+ARTXv4?=
 =?us-ascii?Q?OnruY0EWFIMULrFjsvd+1b55t4T1VabsWKYOngLrFn6LB+9mU6iEG/j3J3l6?=
 =?us-ascii?Q?bsJZeF/hprRlft5tktd5yfaV26LUYHLgy62Vr1TJjbIuhZrocjMBA8pg+rUN?=
 =?us-ascii?Q?YJUn+zJQe43n7vaBPM4X3jCk7EJJv/HS0AgxFYGJqBpVg0IYX/H+dZ1xgzYF?=
 =?us-ascii?Q?2VnwylRwiyahJw536eshBOPwdPdyd6vAX3nNJX06gIvIcls2I+3zqNmWMZbG?=
 =?us-ascii?Q?Hyp4PwCX6czzEFBOZyu4/9+Fl/s37Sf/8+bw6GCYsALWskIBcUy1LPNF94r6?=
 =?us-ascii?Q?JqKFv1hT0fsG5f0FcClvv9G566lYM6zE9TRDHBPMqP8mcslYIOnQ210BjldH?=
 =?us-ascii?Q?2lQYWoT58V0zThjdREae4nwWt8I40oxsWFEIAGUrIYfzS48g+TcAKaBXL4dP?=
 =?us-ascii?Q?GByDpdKmMfE85+B58y551IRqDkU7aEQnSxlQfqybqhUbgW73r9JA8/Etit5n?=
 =?us-ascii?Q?IwStOhpUDqWNPGR1pu6D4s5vNeIu++gkQiGYmrUw7Y0I6rhzBP6yqTj9OnuW?=
 =?us-ascii?Q?O/rDhZjqEgDgkzHf/Ooxr9robPFLALIzP92RzkUyolkyfy2+6g1TQ/7EbQBn?=
 =?us-ascii?Q?CmEM0nRKnoJk/wxieYG8LMuLV3CcW5COFEjPgmjhCwDNAsSjRHIDT6RX6TpI?=
 =?us-ascii?Q?Xq7e/Cg6qAAOXSpK8MfLeuG2hh4yZ9obX2cWUzNU443wUfISGuU79gKuPOm0?=
 =?us-ascii?Q?RaUBKmH4GKBQs4W5HoW4G0i5GJmbRWj47Aq7J9fiRuJ6dKdFir/zch+7L/sA?=
 =?us-ascii?Q?4rC49+b8yzVQCpAmf6Z1ywMnec8VmUzSVsMVJbbldHuUsC+svCUxfNN2nbq2?=
 =?us-ascii?Q?LwaWfbSiLf/aOhVqa6XJgOb9?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a06b4060-393c-4839-f307-08d983375c3d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 10:53:24.5148
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RYZYQNifpUL/8NiZFcr8PWvvGUQPQiQP997hYtzAIUsQ5I7GdLpcFObWaGQC3RzKe5JZTZDumISv1cj8etfo/g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5167

On 22.09.2021 13:59, Roger Pau Monn=C3=A9 wrote:
> On Tue, Sep 21, 2021 at 09:16:44AM +0200, Jan Beulich wrote:
>> @@ -337,18 +337,23 @@ unsigned long __init dom0_compute_nr_pag
>>          avail -=3D d->max_vcpus - 1;
>> =20
>>      /* Reserve memory for iommu_dom0_init() (rough estimate). */
>> -    if ( is_iommu_enabled(d) )
>> +    if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
>>      {
>>          unsigned int s;
>> =20
>>          for ( s =3D 9; s < BITS_PER_LONG; s +=3D 9 )
>> -            avail -=3D max_pdx >> s;
>> +            iommu_pages +=3D max_pdx >> s;
>> +
>> +        avail -=3D iommu_pages;
>>      }
>> =20
>> -    need_paging =3D is_hvm_domain(d) &&
>> -        (!iommu_use_hap_pt(d) || !paging_mode_hap(d));
>> +    need_paging =3D is_hvm_domain(d)
>> +                  ? !iommu_use_hap_pt(d) || !paging_mode_hap(d)
>> +                  : opt_dom0_shadow;
>>      for ( ; ; need_paging =3D false )
>>      {
>> +        unsigned long paging_pages;
>> +
>>          nr_pages =3D get_memsize(&dom0_size, avail);
>>          min_pages =3D get_memsize(&dom0_min_size, avail);
>>          max_pages =3D get_memsize(&dom0_max_size, avail);
>> @@ -377,11 +382,20 @@ unsigned long __init dom0_compute_nr_pag
>>          nr_pages =3D min(nr_pages, max_pages);
>>          nr_pages =3D min(nr_pages, avail);
>> =20
>> -        if ( !need_paging )
>> -            break;
>> +        paging_pages =3D paging_mode_enabled(d) || need_paging
>> +                       ? dom0_paging_pages(d, nr_pages) : 0;
>> =20
>>          /* Reserve memory for shadow or HAP. */
>> -        avail -=3D dom0_paging_pages(d, nr_pages);
>> +        if ( !need_paging )
>> +        {
>> +            if ( paging_pages <=3D iommu_pages )
>> +                break;
>> +
>> +            avail -=3D paging_pages - iommu_pages;
>> +        }
>> +        else
>> +            avail -=3D paging_pages;
>> +        iommu_pages =3D paging_pages;
>>      }
>=20
> I always found this loop extremely confusing to reason about. Now that
> we account for the iommu page tables using separate logic, do we
> really need a loop here?
>=20
> In fact I would suggest something like:
>=20
> unsigned long cpu_pages =3D 0;
>=20
> if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
> {
>     unsigned int s;
>=20
>     for ( s =3D 9; s < BITS_PER_LONG; s +=3D 9 )
>         iommu_pages +=3D max_pdx >> s;
> }
>=20
> [perform all the nr_pages adjustments]
>=20
> if ( paging_mode_enabled(d) ||
>      opt_dom0_shadow /* shadow paging gets enabled later for PV dom0. */ =
)
>     cpu_pages =3D dom0_paging_pages(d, nr_pages);
>=20
> if ( is_hvm_domain(d) && iommu_use_hap_pt(d) && paging_mode_hap(d) )
>     avail -=3D max(iommu_pages, cpu_pages);
> else
>     avail -=3D cpu_pages + iommu_pages;
>=20
> There will be a slight over estimation of cpu_pages, as the value
> passed in doesn't account for the iommu pages in case they are used,
> but still it's better to over estimate than to under estimate.

Overestimating cpu_pages isn't a primary concern, I agree. But we
want to get the min/max clamping reasonably close. Therefore, while
dropping the loop as you suggest I've introduced two instances of
that logic, before and after calculating cpu_pages. I'll see to
send out v4 soonish (provided the result actually works for, in
particular, the case that I needed the fix for in the first place).

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 11:23:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 11:23:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199008.352806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXg1-00018U-8z; Wed, 29 Sep 2021 11:23:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199008.352806; Wed, 29 Sep 2021 11:23:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXg1-00018N-5i; Wed, 29 Sep 2021 11:23:05 +0000
Received: by outflank-mailman (input) for mailman id 199008;
 Wed, 29 Sep 2021 11:23:04 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xzIg=OT=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVXfz-00018H-Vo
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 11:23:04 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (unknown
 [40.107.20.75]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9b6485a4-2117-11ec-bce3-12813bfff9fa;
 Wed, 29 Sep 2021 11:23:02 +0000 (UTC)
Received: from DU2PR04CA0071.eurprd04.prod.outlook.com (2603:10a6:10:232::16)
 by DB7PR08MB3593.eurprd08.prod.outlook.com (2603:10a6:10:49::31) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 11:22:48 +0000
Received: from DB5EUR03FT029.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:232:cafe::c5) by DU2PR04CA0071.outlook.office365.com
 (2603:10a6:10:232::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 11:22:48 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT029.mail.protection.outlook.com (10.152.20.131) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 11:22:48 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Wed, 29 Sep 2021 11:22:46 +0000
Received: from 8725d1e8244a.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 7698802B-D068-4C98-9568-35C73FBFB507.1; 
 Wed, 29 Sep 2021 11:22:35 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8725d1e8244a.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 11:22:35 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR2PR08MB4794.eurprd08.prod.outlook.com (2603:10a6:101:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 11:22:32 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 11:22:32 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LO4P123CA0003.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:150::8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.13 via Frontend Transport; Wed, 29 Sep 2021 11:22:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b6485a4-2117-11ec-bce3-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NDZ4UAaVY9nZhTYxXxBTCLNMI93JXhz8iuH2JFsvWMQ=;
 b=mEtFnTgHzI2lfLE9tWtWu4pJS5E/JVBjDHI1J92iETLnBn8sZm/+tIH424v+9b5JfK1+RgQzQWrLYU5C3k7rDraPuIWtLJ1vRIpYvlKnHI2XJ3qHh74t0+xzLrJZoFgtMRbGn7Kn07zqw8bc5kCf+LvxzUjrHbAOTFRMcB8Ch8k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e46c1532b93b9f67
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AUiQDHmyooUefWf8Jgd6m0vQo6gNlTzPzAmcvGbevnsxdwVM/ruJViCctaXKiur9Mamo8Obwo4prHQp/kAMSG2VJE/rtc9NVYTABbgU9dqdB6/qOG53WvsIBxyYdGLrSz3mnfqsaJnvif7OeezbfLz15FLapnYwt2+ea94VUJbDTswRlMrq+O/qrKlLk3k8dBlL5J8ARhMMAMsoU3wpk67dagwg2ZgmvvdcjdX8I5oyCYvoyyt1b9HGPHxLoUrQDVt7GqnIiT2VfLcXtXIDuUoym53otMLxW+BjRNmZusn5b8bzBhvsnaCin1D+s/8G6k7b5EPGv95ganxaZuL2z0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NDZ4UAaVY9nZhTYxXxBTCLNMI93JXhz8iuH2JFsvWMQ=;
 b=G+6Fn+5xDQN/HFYtn2EqfDNLrMtmcl2W/UQiY7RAAYMwPUyfduef3Danhnkh1bi8+GB0zqHbQtFVTz2A/+ktD4hcfAWNsBWFPCbfqCLIJSn+8UdSUKIS3eAwW8gzpQ2aoCUv2gtBgf7iHOnbjkqUd76YS7pe42hFnWeEtE+G4CAhqf0hdp93WOHej4G9SRgRGVWmosLEnG+InCxQf1+Lox0qfU/fWhS57aDuETkf9tz34/1E3coiOjHiJn3UrXdMIFmiIdcXDdtIwVKAujfdyjPk382BJM/92Yxct/21MljZwoNeGERLpctMj44830R28ktuzMwfcftrtWYpmtqvoQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NDZ4UAaVY9nZhTYxXxBTCLNMI93JXhz8iuH2JFsvWMQ=;
 b=mEtFnTgHzI2lfLE9tWtWu4pJS5E/JVBjDHI1J92iETLnBn8sZm/+tIH424v+9b5JfK1+RgQzQWrLYU5C3k7rDraPuIWtLJ1vRIpYvlKnHI2XJ3qHh74t0+xzLrJZoFgtMRbGn7Kn07zqw8bc5kCf+LvxzUjrHbAOTFRMcB8Ch8k=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH v3 3/3] arm/efi: load dom0 modules from DT using UEFI
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <8f7422f7-8408-f33b-af2d-8e84dc0eab0a@suse.com>
Date: Wed, 29 Sep 2021 12:22:25 +0100
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <9ABAA1A6-9756-461C-868D-145637B67EEC@arm.com>
References: <20210928163209.49611-1-luca.fancellu@arm.com>
 <20210928163209.49611-4-luca.fancellu@arm.com>
 <8f7422f7-8408-f33b-af2d-8e84dc0eab0a@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0003.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:150::8) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c5d3579b-a2c5-490e-c5dd-08d9833b77b1
X-MS-TrafficTypeDiagnostic: PR2PR08MB4794:|DB7PR08MB3593:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<DB7PR08MB359378DAB3A738B94AEBAA81E4A99@DB7PR08MB3593.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 aUCgG8b4Gnc0DONUwyfFuCSfwSnGDeMaclzML7BRC0jcEN7W6K4/IeU62GL/vOSdRwvobUV5FmszsLsbPjTyT7g6xiaOw3yzJ1prz0sXWY2WsTChYPIi2g8YdUBBcrCUuvoRgre3uJtiWIm1n8Gvr6OX12IVoFzg9fhbo4xk2zWGZKecGYZFKOTqi4Hlk7XrQRBunM667azDSFDwY3toQ1egu/6Cw2RY1mW6TG/8qb/lbGI+C8Z0oUSU+t/15KNkvU47E8p1r1UsYq1F8PN1ED5uxV6C/gUXJqRl/reB29T2IuPX7B/vBFu2r/7sZWZ26Vc5NXTOTO4zP70MHdKSIfe/3UJn4n7QI/zUbpEMxQ+ZTG14z5kxTl0VosrXsAwKQEa1M9DVR4cjv4Rso2HXvasmNMVXs5Xc9k7H5kMV8xeQreg8+EPTHTi9F62tC+PecGzqxgy3Z1DSwr/8r8i0hYar0jvXnd8hE77oWqk9663C9mZ/UsgwxhZlSajymIHXmnXdjL/Wpn309XkG0eRfFSOM1wbGceDJYzuxhEnMaBZ5i3oc/TJztNof7e7+KkkhVfDf9imhMfhGDYF//6ZDVIiOucjNFMcgEUJGfF0lXrDHTkd8Ahfgc8ooNOgKv/9YA23LKWCi4AzYSVR3RcpZuXsbuDGnJyxiSGrtz92PDhUAk4VzOK8BWYETQNFXle3jvXUPn6st+7USDxjv8U0FC7klHqDSnjImsh/pmNuPymUQ75+mvfbR9Ap99GdSHwvw
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52116002)(38100700002)(38350700002)(8676002)(86362001)(6506007)(53546011)(508600001)(956004)(36756003)(6512007)(6486002)(2616005)(83380400001)(6666004)(6916009)(5660300002)(66476007)(66556008)(4326008)(33656002)(26005)(66946007)(8936002)(186003)(2906002)(316002)(44832011)(54906003)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4794
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	e58d8091-fba9-41b6-dc34-08d9833b6e42
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ziF6rQTcwJwlBtRd0OMkcM+ECSRw+uqud9BJOD8EetjwxGAC9LR1LFbZaar+2WQjebDcKHss1xY90Sky6kjZGzbsn/362DuuW1Iywh+IPWdPxZnu4QzsoWA8evem7vJKbWwZmJFGALNzppixnkIFALt+s0GL4+BPf63VE+Nm9tk9WANeAIE7bLrwatjnPNkKGNMC9IadzBY1Pgdcr9Bsc0bfp3XCPx7lBTpgIkZsThucr2idrGXlRECg4Sb1UWwHTUSPFY24jj2H+10tdL5tmvbEVsiOQNjlnXD5H+C/b4TGhdnw7Be05iXro4TmYfqWorPtWVwAU5kA3a7kjlOhabirTq5iY+vuDrNKVpO56wE2oL8uMxDBYty3EjVztPrPQj4cB2RGK2dIz+8lqKU2mqaw1H9ZKQa6MRgsIKUI4y0hFS5nUobl8Va7Npa1Abr8IgnReF0nXsxmaRzYMTC4K3EWd8yLaXslONGaZGUCGiDFUqAGB5ClmTWgFsyCfOGtTIiDaogxt/zaGjUHM4VhylBjwE779/8/sBjBs7KrmPpUcQJt20g83Ay4yK0SauFoV2KNLjgMfSzhg+7nMQkeh4O736G30Za5zmpVfCH8jkqVbn127s5TzSf7P/PmJf+ztbnbkfCvr1rmZIoo89EcVSxjDhr/YoWoWA3xEjwE8TepH1BGHDRtda5gAg9AiUZY7l4p7ZzfQW+KuEZ0l7cSnQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(81166007)(956004)(5660300002)(54906003)(2906002)(6506007)(6862004)(44832011)(6486002)(316002)(26005)(70586007)(508600001)(336012)(356005)(53546011)(2616005)(186003)(8676002)(83380400001)(33656002)(8936002)(36756003)(70206006)(47076005)(82310400003)(36860700001)(6512007)(6666004)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 11:22:48.4014
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c5d3579b-a2c5-490e-c5dd-08d9833b77b1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3593



> On 29 Sep 2021, at 09:00, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 28.09.2021 18:32, Luca Fancellu wrote:
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>>         {
>>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>>             efi_bs->FreePool(name.w);
>> -
>> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NUL=
L,
>> -                            (void **)&shim_lock)) &&
>> -                 (status =3D shim_lock->Verify(kernel.ptr, kernel.size)=
) !=3D EFI_SUCCESS )
>> -                PrintErrMesg(L"Dom0 kernel image could not be verified"=
, status);
>>         }
>>=20
>>         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
>> @@ -1372,6 +1367,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>>     if (dt_module_found < 0)
>>         /* efi_arch_check_dt_boot throws some error */
>>         blexit(L"Error processing boot modules on DT.");
>> +
>> +    /* If Dom0 is specified, verify it */
>> +    if ( kernel.ptr &&
>> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
>> +                                           (void **)&shim_lock)) &&
>> +        (status =3D shim_lock->Verify(kernel.ptr, kernel.size)) !=3D EF=
I_SUCCESS )
>> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status=
);
>=20
> Why does this code need moving in the first place? That's (to me at least=
)
> not obvious from looking just at the common (i.e. non-Arm-specific) part.
> Hence I would wish for the comment to be slightly extended to indicate
> that the kernel image may also have been loaded by <whichever function>.
>=20

Sure I will improve the comment to explain that.

> Also, two nits: You've broken indentation here (you've lost a space too
> much compared to the original code) and ...

Yes sorry for that, we didn=E2=80=99t see it, I will fix it.

>=20
>>     /*
>>      * Check if a proper configuration is provided to start Xen:
>>      *  - Dom0 specified (minimum required)
>>=20
>=20
> ... you will want to insert a blank line not only before, but also after
> your insertion (or, imo less desirable, neither of the two blank lines).
>=20
> Further I wonder whether your addition wouldn't better be after the code
> following this comment.
>=20
> And finally I wonder (looking back at the earlier patch) why you use
> kernel.addr there when kernel.ptr is being used here. Unless there's a
> reason for the difference, being consistent would be better.

I will do all of the above for the v4.

Thanks for all the feedbacks.

Cheers,
Luca

>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 11:30:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 11:30:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199014.352817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXnI-0002jv-0q; Wed, 29 Sep 2021 11:30:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199014.352817; Wed, 29 Sep 2021 11:30:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVXnH-0002jo-U0; Wed, 29 Sep 2021 11:30:35 +0000
Received: by outflank-mailman (input) for mailman id 199014;
 Wed, 29 Sep 2021 11:30:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xzIg=OT=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVXnG-0002ji-1x
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 11:30:34 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (unknown
 [40.107.22.56]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ea6b023a-47b3-40d6-b4af-360c45085e91;
 Wed, 29 Sep 2021 11:30:31 +0000 (UTC)
Received: from AS9PR06CA0132.eurprd06.prod.outlook.com (2603:10a6:20b:467::15)
 by VI1PR08MB4302.eurprd08.prod.outlook.com (2603:10a6:803:fb::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 11:30:29 +0000
Received: from AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:467:cafe::7a) by AS9PR06CA0132.outlook.office365.com
 (2603:10a6:20b:467::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 11:30:29 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT026.mail.protection.outlook.com (10.152.16.155) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.13 via Frontend Transport; Wed, 29 Sep 2021 11:30:28 +0000
Received: ("Tessian outbound 3c48586a377f:v103");
 Wed, 29 Sep 2021 11:30:24 +0000
Received: from e7bb2a9d1c02.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 859288AF-1049-4564-8CD6-CBF20ACD0BB2.1; 
 Wed, 29 Sep 2021 11:30:11 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e7bb2a9d1c02.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 11:30:11 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PAXPR08MB6971.eurprd08.prod.outlook.com (2603:10a6:102:1da::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 11:30:09 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 11:30:09 +0000
Received: from smtpclient.apple (82.8.129.65) by
 LNXP123CA0016.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:d2::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 11:30:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea6b023a-47b3-40d6-b4af-360c45085e91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7Jxhl/zWf/A9H0Crfm8bJ+EM6HO9RJUL4QN57A5IDwY=;
 b=2WzBolNjtmt7mG2wednKIphHW/8TfzzQTEuFMvxSLEglDYzB6BZBe/cg7QZ0wUKFMGQ5OR/0XeRhSmwEr6m25OUSLO6reQI4dOFUAtCBvoYgJ5/1B0Jher/dPnwe8roIwDhfsrFjdh4A/Oce4k1Bgs6iDlnO8vOvwJNVl73P1pM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ca96aaf5e52b76de
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j44HozNIEdnDJbbAEQY+BgNHt3rfN+9yt9p+lUst0YQWVohTdXIEUXrWjUljoLYcRncbxoCIg3n1r2zkkw9x4ueR3GwKoYzhjESYdJgGwXIRkPy3seNdkcWsDrQVpxuoC0PWZAbgDKwlqejuMYwX9+yaMy+XxJ9FHM+Lf8xlNCyB2UcvEQ/ffe45ZmPTdDj4N5UXckWz1oZqszS56w29MBKPv4rSAvU5fAvYu80/G9OdrjUcT4fl1qPpb1/GwEuOlRp3FyOLjgvZS19teDdS88spV335p/FUwewqqbY30EevHFbfrCpn8ew/isnZbmlsGZ2iMxC2jlQMU0/pEB2haw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7Jxhl/zWf/A9H0Crfm8bJ+EM6HO9RJUL4QN57A5IDwY=;
 b=l4onwVjn9AkFo7PUC2vWJg3KNb9llXfFFdz8IUxBLWOBI/hWCZtDg70HyhPAMmb5+EfhoyK0RBDMomtbdqb+WkcmI8xx5Ncb1BGZlGrPZa/gdSP8EzLgHL0fq6NPur1eQIHN1TTVq2OY3+v7OhO1B4Hj62ualf/QqeSt1q5m7zHSPe7hJHap6hIKzNamfaQXZDEfoeUgtDapzcPgbNyzZ64tD5/3aloSppynRjHJjGbY9JP3nASvJUzV/MSnJ+krk2YqREXePHmkIsK0LN/QyUgVW9Ei2iRB+EuEZsMt74zkrqOk1MarZpnVSFb3djjOz2y1nzGYxHdf9GgB6UXAJw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7Jxhl/zWf/A9H0Crfm8bJ+EM6HO9RJUL4QN57A5IDwY=;
 b=2WzBolNjtmt7mG2wednKIphHW/8TfzzQTEuFMvxSLEglDYzB6BZBe/cg7QZ0wUKFMGQ5OR/0XeRhSmwEr6m25OUSLO6reQI4dOFUAtCBvoYgJ5/1B0Jher/dPnwe8roIwDhfsrFjdh4A/Oce4k1Bgs6iDlnO8vOvwJNVl73P1pM=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=utf-8
Subject: Re: [PATCH v3 2/3] arm/efi: Use dom0less configuration when using EFI
 boot
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <120aae99-d8f9-eef3-e6ac-b1b9b842083e@suse.com>
Date: Wed, 29 Sep 2021 12:30:02 +0100
Cc: Bertrand Marquis <bertrand.marquis@arm.com>,
 wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>,
 Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>,
 =?utf-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <AB8612F0-623B-4E64-95FD-1D73E49FB169@arm.com>
References: <20210928163209.49611-1-luca.fancellu@arm.com>
 <20210928163209.49611-3-luca.fancellu@arm.com>
 <120aae99-d8f9-eef3-e6ac-b1b9b842083e@suse.com>
To: Jan Beulich <jbeulich@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LNXP123CA0016.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:d2::28) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f87c3753-c90d-4498-5c8f-08d9833c8a0e
X-MS-TrafficTypeDiagnostic: PAXPR08MB6971:|VI1PR08MB4302:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR08MB43020BD00C19292A2E606950E4A99@VI1PR08MB4302.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:2150;OLM:2150;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 PWa9VMl1LtK7PvkwfarnORnO25KQoH1pjNvnxFMW8/vQWodh+piWXbsp0xlZZwpqztidMfGunLd3TyjkwC2n5TNzY9zo9OofpyFUGq5/c5Jv1cILteKj1xIAzLUi6j1RZuMNAn73Zfw/EdHzbs6zrfFlvMHpItcpIKBeXlnTDQu54Xyt1mvJX3eHYj/Ju0rsy+KmbVHaaLT3BqAnU4vzfe3v6gMQ18bCjYyb8MjppzLFfz5JLRiE9iv7XjUSM/THG/t0qrFYwxB/CB1gK9z/dMwaN6jb5ERp07/Be2dv10yQcXIxQEkV8pF2pznS1VHkA74mtlM6NMPN5lPwASMnTLQz/3GiIi0Wtu7Ku9Tlk4R4N4GkEER0+33DGe/bi3sP6BqniDEbQe7ACr7JNCra9FkSkPin/c5r3S5l5tXvABUUxbYtsg5yCvfiPM9Vj17PeRysVeK5CXL/FKsXbIt3rXEMqWIBKNWsnQpY5nj1+gDjzGdR/+wxBHt2dAPkYsgwwqq2aMTPr3lB5J0wV8p/6GmaeKDIlltFTOy6hMjUhR6j5INsMlRKG3eP+frexA1DDn+pTWttwle7cpp2Rb9jgNxTIiyuhnmNkqAQWiLKw6+QmVRHDIZoxcWKduLOYGmSszV143Zd7cA88LInBNj0mtvUjPMKlVWBJ0u1p4c/mv0iHPmfX18FWmYFXkk8av2rXl3eOXNTQ1veFzPrdYxsuq4DXb321t1Fm2qBZMShzqYq7KVnOjPeo4LTzc9Fn/5rc93vigs10NbyLa5kVrgs7A==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(52116002)(26005)(6506007)(54906003)(53546011)(6666004)(6486002)(508600001)(316002)(5660300002)(44832011)(6512007)(2906002)(36756003)(38350700002)(186003)(4326008)(8936002)(8676002)(7416002)(2616005)(956004)(66946007)(33656002)(6916009)(66476007)(83380400001)(66556008)(86362001)(38100700002)(32563001)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6971
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	f39550cd-1c21-4301-e2a9-08d9833c7e2b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	6U/VzkTVu+mTNInbjUc4lBQg5dwtf4P3S0IrCplfZBDAGy3rebpTh5H/kKpaqn88O2HzwE/9w7TCIe7lqAIeRb7fGUPrTebha9eurcpye04utIFpRb3gN97kVt0hUty93Opr8DgFky7PMsBxrMm9qGqGyTghW0c8DCDy+N8TG9MFQVVPS2lg5afIccmj+Yjf5fVwWA6flpntHJxjJSG5sjCntYd3hQ8he33XfxIr1Cei3pnd0ydlsxeNjn9nt5zE6oRNhq0jOrm09t9/TcCmhMZeUHAFRBgZwkpA1T29GZrXEBBo+ECs5SvF4YMwlMGuJgLHp4IRubJAyiionHAWJtYJ3QfGCeddIcm+Hw3cKAbkYVjVmxlFrEMy+WOeuEeUfFyMeUHPW61dt05VaEyYjncfNJbG0zrePPsKlkv2WqSYL9Y/VjAa1lacEor3NXmPf7HSOZnbQHcg11+bBVcGOozq0ztfs7SQi3ibYAPs8GoptztT0vY8Hc3iRnxAzUHVm7Jx76F6zoMBD+W0v3F4DZn3rijaXEHCdq0QAPLnbRImnXSZPbuE0D4oRPrDhQz8alL50coRqJeWSZibBFJe1qKO/6flZUDdCUhbNgZ3qB4jg5DfffpTl0lwPEq0DVCWsaiIAaqyuERAVEV+6FACrzEBbSqOAImKRhk+4AsCE1RjD+g2Zdfw4UdUFjgopm3EtJow9vk3ODBtvs2P+4n2z3WBu7XKZKSdjswA1exmM88=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(5660300002)(82310400003)(36860700001)(44832011)(4326008)(6862004)(2906002)(81166007)(47076005)(36756003)(186003)(2616005)(316002)(33656002)(956004)(83380400001)(70586007)(26005)(6512007)(70206006)(8676002)(6666004)(54906003)(86362001)(6486002)(8936002)(508600001)(53546011)(336012)(6506007)(32563001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 11:30:28.6249
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f87c3753-c90d-4498-5c8f-08d9833c8a0e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4302



> On 29 Sep 2021, at 08:50, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 28.09.2021 18:32, Luca Fancellu wrote:
>> --- a/xen/arch/x86/efi/efi-boot.h
>> +++ b/xen/arch/x86/efi/efi-boot.h
>> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const str=
uct file *file,
>>     efi_bs->FreePool(ptr);
>> }
>>=20
>> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>> +{
>> +    /* x86 doesn't support device tree boot */
>> +    return 0;
>> +}
>=20
> Every time I see this addition I'm getting puzzled. As a result I'm
> afraid I now need to finally ask you to do something about this (and
> I'm sorry for doing so only now). There would better be no notion of
> DT in x86 code, and there would better also not be a need for
> architectures not supporting DT to each supply such a stub. Instead
> I think you want to put this stub in xen/common/efi/boot.c, inside a
> suitable #ifdef.

Sure I will enclose it in #ifdef CONFIG_ARM and remove the x86 stub.

>=20
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>     static EFI_GUID __initdata shim_lock_guid =3D SHIM_LOCK_PROTOCOL_GUI=
D;
>>     EFI_LOADED_IMAGE *loaded_image;
>>     EFI_STATUS status;
>> -    unsigned int i, argc;
>> -    CHAR16 **argv, *file_name, *cfg_file_name =3D NULL, *options =3D NU=
LL;
>> +    unsigned int i, argc =3D 0;
>> +    CHAR16 **argv, *file_name =3D NULL, *cfg_file_name =3D NULL, *optio=
ns =3D NULL;
>>     UINTN gop_mode =3D ~0;
>>     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop =3D NULL;
>>     union string section =3D { NULL }, name;
>>     bool base_video =3D false;
>> -    const char *option_str;
>> +    const char *option_str =3D NULL;
>>     bool use_cfg_file;
>> +    int dt_module_found;
>=20
> I think this variable either wants to be bool or be named differently.
>=20
>> @@ -1361,12 +1361,26 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TAB=
LE *SystemTable)
>>         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>>         cfg.addr =3D 0;
>>=20
>> -        dir_handle->Close(dir_handle);
>> -
>>         if ( gop && !base_video )
>>             gop_mode =3D efi_find_gop_mode(gop, cols, rows, depth);
>>     }
>>=20
>> +    dt_module_found =3D efi_arch_check_dt_boot(dir_handle);
>> +
>> +    dir_handle->Close(dir_handle);
>> +
>> +    if (dt_module_found < 0)
>> +        /* efi_arch_check_dt_boot throws some error */
>> +        blexit(L"Error processing boot modules on DT.");
>=20
> For this use, bool would seem appropriate, but ...
>=20
>> +    /*
>> +     * Check if a proper configuration is provided to start Xen:
>> +     *  - Dom0 specified (minimum required)
>> +     *  - Dom0 and DomU(s) specified
>> +     *  - DomU(s) specified
>> +     */
>> +    if ( !dt_module_found && !kernel.addr )
>> +        blexit(L"No Dom0 kernel image specified.");
>=20
> ... this (and my brief looking at the Arm code) rather suggests a
> count gets returned, and hence it may want renaming instead. Maybe
> simply to dt_modules_found.

Yes that=E2=80=99s a better name, I will also add a comment just above the
efi_arch_check_dt_boot to explain it is returning the number of modules
found in the DT or an error (<0)

>=20
> Considering the new conditional I also wonder whether the error
> message can't end up being misleading on Arm (it certainly should
> remain as is on x86).

Do you think that a message like this: =E2=80=9CNo guest kernel image speci=
fied=E2=80=9D
can work for both x86 and arm architecture?

>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 11:54:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 11:54:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199020.352828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYA3-0005Mr-V6; Wed, 29 Sep 2021 11:54:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199020.352828; Wed, 29 Sep 2021 11:54:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYA3-0005Mk-Rp; Wed, 29 Sep 2021 11:54:07 +0000
Received: by outflank-mailman (input) for mailman id 199020;
 Wed, 29 Sep 2021 11:54:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVYA3-0005Me-6B
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 11:54:07 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id f20fa97a-211b-11ec-bce7-12813bfff9fa;
 Wed, 29 Sep 2021 11:54:05 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-CgckERszOWWT22jBwUymNA-1; Wed, 29 Sep 2021 13:54:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6480.eurprd04.prod.outlook.com (2603:10a6:803:11d::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 11:54:01 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 11:54:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR04CA0058.eurprd04.prod.outlook.com (2603:10a6:20b:f0::35) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.13 via Frontend Transport; Wed, 29 Sep 2021 11:53:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f20fa97a-211b-11ec-bce7-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632916444;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8z51UwVKPDWSBfBs3ps8xHTJL4MdVWelWkGkGl6tl3c=;
	b=ak+z841ZZ113xFMjU+03MT0x8IQ9tVEW1qsonsnfV1KQAZnwAcU6TMlddTB1eVegfKVRUh
	diRBFu56m6MrxL6qnpI160y3420Afv2lCr6OJB/gVgkVTPTdvbJOQKvTvdueTZL6siWf9U
	iHY6M6VIDy3JFHQa3v8yFJlQiqX1OU0=
X-MC-Unique: CgckERszOWWT22jBwUymNA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FAX0+2E76jqnGOi8jd/pBuNipq77Q9K2zrTNL7pJQnpJK0LExgG9N++i+hK7XNEZEDOKx4DpV07jzuCKmClBHtY0B8DqoEFs2QZqXxv3aDKvwf6+iDYJE6SPfnxtVrhw4gPv5wu/T5HZIxMcVjIXnT031S5/rHmBIBwopc2nyQUF3V/3EW8C/tK3tJsPzcYThyzlwJjcsVvh6QpldK4GksWcDkesWiI3citXfy4iOF1+0ZVawbnDxIeClWH7lFPQuH5tqosCmuxxRwHCSB5zLmpBjCYgkwZO1B81ULXPGl0P3mZLAf/cLbQAk3C4s7ibAbl6RDhHe11JQ+1qJZmCag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=8z51UwVKPDWSBfBs3ps8xHTJL4MdVWelWkGkGl6tl3c=;
 b=PG51bbUGl9fRQsum3lMMKT7fpmPw8TpDsLXgQ/YTeAP/p6lzIgivT8J5rU4ksiJW4kfp/k+/RAjRejTn/CNMFiGabupR9IIihfwCfZFRiOty8YVWP/zmelyH9bnjcKsYEFXs/Po/wjI8DY7dQJ6VkKmfE8hV3TN1TYKnl0TJZrHZ0YPFvTGD5bW8UhlZ28xqVZoRH/eOi34DbMo/RltLumsyctZbFt85v3CU72eRyjMslw+fsJPlRKL9smZCS51wqqqr4JZtiJUwOXHvj8naMHJEDlJhNCOiaARJXb/wDoEFRud4C7ztuY0jmK1PFHSutvJo96e3oy3BPh5hhDWBmQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 2/3] arm/efi: Use dom0less configuration when using EFI
 boot
To: Luca Fancellu <luca.fancellu@arm.com>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <20210928163209.49611-1-luca.fancellu@arm.com>
 <20210928163209.49611-3-luca.fancellu@arm.com>
 <120aae99-d8f9-eef3-e6ac-b1b9b842083e@suse.com>
 <AB8612F0-623B-4E64-95FD-1D73E49FB169@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7f64b6b7-e394-c57a-b4c1-b88ef09a9609@suse.com>
Date: Wed, 29 Sep 2021 13:53:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <AB8612F0-623B-4E64-95FD-1D73E49FB169@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR04CA0058.eurprd04.prod.outlook.com
 (2603:10a6:20b:f0::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: af19d288-9a38-4940-792f-08d9833fd3ae
X-MS-TrafficTypeDiagnostic: VE1PR04MB6480:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6480E2FC2D5B45ABCC7D69B8B3A99@VE1PR04MB6480.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yF43TbKNOLojtuPvAmpOfwfPWKF4s23ySOFw1UU6FcH9H48NHtyH5ULWMhB4dUXB5i+acZ08G4CgdPhjt8D5wVZRUiSKcXPfkLIQYTvG1tNuTU48rRqzrcK0AogY4QDTQZsKDtlQA745w4MIcVZbX7MwlGqnkenxDL0oB7rjSZHgspryrZtEuZ1S0cVnyXW8uABkXRz5ZpO5QcX/Foe7epG7sB1CH4x0od/vkKlmFUr7WCK8su0VMzBNV9yRW9uapet1ZL8GxgLaQA3x3ffYkBcqwE9+POSyvNSpzeuK2GbEYRtB5rV1svS2jiPPqslBH5LVx5phUt4i/TXEXpSb58nR/y2cxVHvpKCYwfyvhjBSHnySF6pYD3AVo+m6KlGLQyAEsM61kYnq5m4+ls48c3OE6zEfw0PpzSYwke/44r7YVNPVItyUvjfMR12pR+2E7Y2oOav+fKoiZ7ZM1dFpGUfg8jcuxNDNY3u1Rwb918J9Ft6UiKoSryiOZeptT/M0kKQZH6cN/OiQPRrWZ/lFCUtm820WYMbD+zfvYkbl4+6lUFmc2aejesSu1K8bJxfJe5D+LvU+U/xVabOttiD3uyIkgLDncuxEpCb1rQq3/NXZlEIKLHJgCxlW+aIuPOLB81VJbXFvy8HHn2FVO9ynzMiWzeEM2V3yUQEulYZs+c6u4HwT3ahKWKTnC3NLkaoWutI0Mh/m6WaHCRyyd8+kYwYqi77yXEEq3Jpilen1bHaHPbak3TwG9yh7GGCQUoq2Kw2afPCB5ALSB5SfOG6EqA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31696002)(83380400001)(2906002)(38100700002)(26005)(6916009)(53546011)(8676002)(66476007)(8936002)(66946007)(6486002)(186003)(31686004)(36756003)(54906003)(66556008)(4326008)(5660300002)(2616005)(316002)(16576012)(956004)(508600001)(7416002)(86362001)(32563001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ejhrdERzZ24ybFVqRmdzN2FvUUUxaU0rcmVyWXZNSmY0UFJueUgvMVZEWUYx?=
 =?utf-8?B?VnpoeUIramFQSW9WQkw2OGFPVU9sRjBoakRNdjR0YTJXd3ZWV1M2c1hTb3pE?=
 =?utf-8?B?MDhBK3ZJQmNYMHkzSWN0QitlcUZxS3luU1pzTDFyWXRyUXM4dHhETWIwMkw2?=
 =?utf-8?B?aHdsWldOUGZXS1NMdHZZeFNUYTJTV3dQTnVZVmt1ejlGSE1JcXpWSCtwMkxp?=
 =?utf-8?B?T25VaWFoQmt6YnJnWUtYWmZvOFZEY3FSbEk2RE5jQU5TenRzYXgyMEVOTzNz?=
 =?utf-8?B?cDFmRWU5Z1hCUng5TXNyOFNMOHZ1VXpQdm5xZ2x0TXVUcVFXa3FxSXJzbFh2?=
 =?utf-8?B?T2tPKzJHWERCQURtMjZnenBXa29LVVdqb2RrRlFlTWttS0RaTTArSVFGMDJG?=
 =?utf-8?B?MWlIT2RRMTFKU0h6UWQ2L1B0cWgrZjZwWVVlNjFMQmpCL3BnVFVOV1hQeE5y?=
 =?utf-8?B?THRZVnMvbU5zcmtRSG1NR0txRFRBUjZsTENKUmFJbkJacjFIQTk0anEycm5v?=
 =?utf-8?B?dHRuR0wxMEQybFp6ZVVFY2ZacVllUy9LOWRKdkNaSVMyV1FuSzFzUXZQVU45?=
 =?utf-8?B?dExPR0ZKekZYdmYrNnVLY1Q4MU50YTQvcUJGTzRJWGhFd0Z6TUN2cTFuSEor?=
 =?utf-8?B?enZJSTY0dGd4amh1ckQyMnkwd0hOQllWb24zbmxBNHhzZDNSc0NvdSs5N3lS?=
 =?utf-8?B?amZENUcrU1FmNEFKWUlKWVpJVHpKd1NyMVlFOTNXcENpYlJESHVuSjRDcGUv?=
 =?utf-8?B?bzE0VTRWQ2VUZGFWaWYvb1lSVk0vcDAxeDBCSkRWUEZUYjV6L3M5T2ZzOEY0?=
 =?utf-8?B?bU5tLzJrYmpFcUNIU1h1YXgycU1GbnBtbENQeWNBSkN6UU9oYU95TjRxQjFS?=
 =?utf-8?B?M21hMUdCZHRFdy9ab3FWYUxjRitDZG9JT0thaUs5clZEbE9tNkt3RStUdWlJ?=
 =?utf-8?B?cFpGK3FvWlBxdzJUZWM2NjJkd3pXc1JYZUtLTTl5RE02SjJmMzdOZ2JUSE45?=
 =?utf-8?B?bURaQUVhcVRzOTYrVTlEbXpFWi8rb0NDTjhxUzF3bjZzanhQNStHTnZncmVS?=
 =?utf-8?B?MVRGVmlFUFVCS2hJbGpHQlJuak5UamxBdld1dllDaGVmTWp5NzVZK3dPYzFE?=
 =?utf-8?B?MlhTcDY5OWsvbXE2RURoVzF6SDluSXRNWG5lNGUraG1YZkhFeXFDcDFZcHJG?=
 =?utf-8?B?MU93Q2k3N21ydEtHU2FMVlNEUzRRd2hJUGYxVTlWVFBNV2hvNEx5ZmFiMU1Q?=
 =?utf-8?B?MWFhR0dvWTNKUWtDNk1hbWtlc1cwUkpkdWRwNjcxeUJJVkdEQlE5MnlvNEY4?=
 =?utf-8?B?anNtWmFtS1ZIWU8vOWdDMW5vRzJMTGp3MkZVY2h4cTJjL1VSMFlHS0MwUGRS?=
 =?utf-8?B?NGxkclg0bUY5NS9DWjFxUWx4TFg2bVBzTFRGTWdyY1Q4VHBYa25GNzZGV0dw?=
 =?utf-8?B?QmZ0SUs5OTJHZDdMcjNDTnphTG9tVU42eUY3VEZwK0JnUmVXNmFSUWduNTd6?=
 =?utf-8?B?czRWOVp6MmFKbFBaenNVYWFRL2pVa3VielpaTjVxRStWKzdmVXFkY1JuQnhT?=
 =?utf-8?B?QkwrbVgvNWpHeEdkSzh5ZDhmczU4MmdCRUd5eHh2bG5XWG1pdU1acjRoNHNU?=
 =?utf-8?B?SjB2Z1VoTzFMS1dwNDh6MzQxMXNPQjhPeDgzcHBScXJBWVB4YkFhcGFUNVhD?=
 =?utf-8?B?ajFSQjlZWS83UVBCVC9nNytmcDE1bFBsTUkweDZjaHJkZXBZb0p0eHpMemZm?=
 =?utf-8?Q?vUle45PuFCdS1Vl9+twlfL8TWbC59VsoRV3FW9o?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: af19d288-9a38-4940-792f-08d9833fd3ae
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 11:54:00.8790
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: U5CmOGjd6g5p9+gcgwnb69unGfU7SfPLSmRCVMsdg0cigbpw5eAAcHoQ0g99mL7WsXOrrp92l4rcgRx5gjJpFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6480

On 29.09.2021 13:30, Luca Fancellu wrote:
> 
> 
>> On 29 Sep 2021, at 08:50, Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 28.09.2021 18:32, Luca Fancellu wrote:
>>> --- a/xen/arch/x86/efi/efi-boot.h
>>> +++ b/xen/arch/x86/efi/efi-boot.h
>>> @@ -678,6 +678,12 @@ static void __init efi_arch_handle_module(const struct file *file,
>>>     efi_bs->FreePool(ptr);
>>> }
>>>
>>> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>>> +{
>>> +    /* x86 doesn't support device tree boot */
>>> +    return 0;
>>> +}
>>
>> Every time I see this addition I'm getting puzzled. As a result I'm
>> afraid I now need to finally ask you to do something about this (and
>> I'm sorry for doing so only now). There would better be no notion of
>> DT in x86 code, and there would better also not be a need for
>> architectures not supporting DT to each supply such a stub. Instead
>> I think you want to put this stub in xen/common/efi/boot.c, inside a
>> suitable #ifdef.
> 
> Sure I will enclose it in #ifdef CONFIG_ARM and remove the x86 stub.

Hmm, so you've taken my reply in way quite different from how it was
meant, albeit I can't see how the last sentence of what I've said could
be interpreted in this way. The more generic stub wants enclosing in
"#ifndef CONFIG_HAS_DEVICE_TREE" imo.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 11:56:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 11:56:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199028.352839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYCR-000629-Gu; Wed, 29 Sep 2021 11:56:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199028.352839; Wed, 29 Sep 2021 11:56:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYCR-000622-DD; Wed, 29 Sep 2021 11:56:35 +0000
Received: by outflank-mailman (input) for mailman id 199028;
 Wed, 29 Sep 2021 11:56:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVYCP-00061u-Ra
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 11:56:34 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c49f5cb7-6ad7-4af6-a3a9-ae886a1f8102;
 Wed, 29 Sep 2021 11:56:32 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TBnKju020531; 
 Wed, 29 Sep 2021 11:56:27 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bcp1crbsf-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 11:56:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by VI1PR0302MB3472.eurprd03.prod.outlook.com (2603:10a6:803:25::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 11:56:24 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 11:56:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c49f5cb7-6ad7-4af6-a3a9-ae886a1f8102
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fxwMfsNziAeyefWfP6oI6n+OafV8PRc8Yzks1LtnJ0yCNKeGixebGW95bicuo4D5MSSu5UoA8czk9y6ERFAAc6Y1wIWcBh1HUOLvxxf3U8tUxwSdwD2157OxfHSdiUXkfSkVv2w2m2JuTG0p0D8aLv59AiaPzXduxB1HheSL7m3iB5xqTvcnfigrQpZNnF9tjDOBXay0y6RhkVZz8daFR6X1diu2CYTdzVkeC9RBgFeo//03etAzdqkBpyfWSIhsatKdrQzTaxgDT/0xI/wO8p2688jRy6HebhlyVom+hRPPmnlZe9Bcf9TRFMqLHnbHmjAVzUU6uUL9dc49Z0ec1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WkZ3YPW1QEUhLOUhvyxRkmwjv6LmF2TOVLG0kFX4W5U=;
 b=N7NJT0HkaMLLeL87GeIoAVV4JZqaqzhBE6lIjCoXNZ97crS0Ocf8322I7oOJxA9pP1JXyWoksLEnk3Pi+KPEKtt1xxwsdIJ3cz5n5zZbWohXGHZ7Ac4iuwPrR5LplefM2Luty6y6Yq/7VThkfLEpPKvRHeVXAKQPpK7vQXboDLoYmXjRIexxAZefkdX9IjenCR6KxIhGSpo0EXVUgxJ8RUN/1TuqnAGrJIwFrne9//WNR44qPg5yQCxKzEm8LJXPR6e4IA8dhUN1SlXIL/jki5wd2aOVLnM1Uw2dUDQONE6rr6VZlWAxljlUl77Ywpzv4Cd/+COHdE5lXzX8U9eLJw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WkZ3YPW1QEUhLOUhvyxRkmwjv6LmF2TOVLG0kFX4W5U=;
 b=4PtrdQdM3ICJ/KcI+RxcXFaSfvWTvBh0S2DM5tTTlaByBX9lj/76pISNFVreIZEHwOw2htuhau7KhcXLdl8FuWJ2Z2tk6R4EuvgS5jxxeuhjS+atksRhADqA8kMUPOW+O655IJh7krqjw0ZbJufU+2Je+4B32JObV/pMDVvIjUphVK8ljqfKlGMDCjfTCEX2MSGMryKu0h8u+llgc42mfVS+tVg53ZPvrzW1j2Qrp0pzs5eAdq/5I0PykhegnvU5t2NAwNLfxIQaY4m48ZTEtAT6ETfR7yy7LGSwZMmsEEFsn6MuHk9ChemwyuJO5fMEPcQIBW7+3o7ZcuGHjJ+QRQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        Michal Orzel
	<michal.orzel@arm.com>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: 
 AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyAgAFQpACAAAGzgIAALsGA
Date: Wed, 29 Sep 2021 11:56:24 +0000
Message-ID: <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
In-Reply-To: <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 9ce2ffc4-4be3-4304-27c0-08d98340292e
x-ms-traffictypediagnostic: VI1PR0302MB3472:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <VI1PR0302MB347292C981ED42A0DC6512DFE7A99@VI1PR0302MB3472.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 LZ8fc7/f5iPS0BfovvIwYCas9hB71dGzKCPH+ScXa5/ZqYdqUV/ONAJf8U/5hWxH4KDrhSeUw18gYiYB5to6iX9xD2j2hkP5KlUEIWnASYstEsXZO/qwnNLR+u1Zu1WtbCxrsqTjRsOgu0g4q8n8hePjAv9w+nOJO/jcXOxAFVFJsGWp4VfCNrdic2U3G1bbNLw3Y1quGIrHtGl1+y5jUAowcN5iQrf+tIjAn2ow8ulhsRIwb1ZDs+Twr14LCoaVtZG3QffhhaVXlxeI9cMCaZarvg4ADoeZ8zv+UAukUD+SneT23zIkzUzFUIX5eJBsmIVrpLcuGpLeDalhsMvzE/fowMjt7gLe+MwtD0tbQf2VI8f5/+X43vLoJK9lXYw6AVDk7TB1lKiLF6oScXx3E92/OUKO7qMV49ps4rbrD1e1w/Wr0r7v+UYhKOkXCXCWcl3JNpyHkMnJ4HxL6hVD8644aQYyQW79HRfTrZ/tCPEgANWTiCUUvrBEpP4ce0/aIzOAKpeKjvXMZr8y0ziEaleHltG7DHEyh0XpWxe08oVwssX2zYdeiaEExHMMi4BP9DXeKMeSNqMq1ceX9F4XV2+gVXQg0qPTk0Hp7gHVgKIRV5tt63SXOywLnISh3cf7uy9VGqRHDnDT0AyxkQ7eZJaL9LUFyNGsaiKAOTfT89rixwFTsFhw6pbzprjguyeQoFmc4BrfQYG8N88CGKPkV8OqUqJqbA/2pVjJh0JV/PcQry4KlxYHpIFyVDgmNk0G
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(122000001)(38100700002)(38070700005)(2906002)(6512007)(71200400001)(8936002)(508600001)(31696002)(5660300002)(186003)(55236004)(26005)(6916009)(66476007)(91956017)(54906003)(36756003)(4326008)(31686004)(8676002)(6486002)(2616005)(66556008)(316002)(76116006)(6506007)(66446008)(66946007)(64756008)(53546011)(86362001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UnJMeUVnblBVRU04TU5NSTVCMGVCeUpuVjNDK3kvelFkWXY5TjRWSjVOYVc1?=
 =?utf-8?B?VHVPY1FtWWIzbHNOZ3pVSFYzWkJhSWl4L3FDbDBWenNDN1dXZlZRY3MybXN4?=
 =?utf-8?B?TVlRVFpkRG8rOVl2RzFkL25xNWYrU2FhcGdheDJyWHAyTHhwTFByQTFzeVp3?=
 =?utf-8?B?MjNUQWp1WEdqcGtuRjJPMHUzWkVjRDNrbVY4Z3diWXFsS3BLT3hmdnNndmdo?=
 =?utf-8?B?UzlRTFFSeTc3ZjNxbHN6anA2ZmdXeWV3QTJGcWhJSTdlSm0rbTgxMC85VDBt?=
 =?utf-8?B?UmdrT1F4dFpwOHJQdWlkSVBRVDVLZER6cE04Zzc5V0hkUlo0ajVBdmtyM1lR?=
 =?utf-8?B?MU9wdC9YRkx4Z2o1RGVFUDU1cUJUdG15UlphL09aSm1maDdIZHRReHB5UGZo?=
 =?utf-8?B?TGcwUXN6T0dPdFZacnFkZzI0TElyT056UzhjQ3YvY0NPVTc3UW9SVzg0elRG?=
 =?utf-8?B?VzRTZUtmYXFxWmdrckVxZ3daOW83aWYyTGoyaExXbTR6NmFJck1ENjYzd3Fm?=
 =?utf-8?B?b0g0L3F6TVFaWmlDcDhuT1grakE4OU9wdmM1U2Vya24yQ0hoSGFoOEFoWkJL?=
 =?utf-8?B?OG9EcW9oTlNxaDJETk9UM2MxUDZZTmdpUG1rQW9SK29Na0hJd1FtRFJTcXFE?=
 =?utf-8?B?MXk0a3FDWjEveVF0YUxYQS94UGp0ZkVHSkRmVmgydkVUajRDa1FralB0WUYz?=
 =?utf-8?B?NHNGakZqWUowQml6cldES3VIZkRWVzZDdmJjWWxKZmsrNEhHZnQ2ajM0UlFv?=
 =?utf-8?B?YXZTaFpsejdHbkRJQUdVYXFyTFVhUXBDZ01kckY3S1A2Y0p2aEEzT0cybTFp?=
 =?utf-8?B?V2FZZm1JaU9HWnh4cHVsRWowRzJldERmTFlWbEV6TGVmK2szaW13M2cxSHRJ?=
 =?utf-8?B?YWJBdis3VlBJb2p1UXZtak9CVHZGbWIrN1o3WVBKTmJLTFpTdXJNeTUxenNW?=
 =?utf-8?B?UEVmV2ViKzdQcG5wTnRFWll3UFpsb3VYV21KaGt1dzRCWXlKbU84TFdpYnAv?=
 =?utf-8?B?TWJobGFyV21iU1QreTcrby9NaU9XZFhqN1pJOU5QRkxQeHU3KzR6UWJPTWFT?=
 =?utf-8?B?Uk5laVZwWC9kbHBxZUhSQjdQaTlPdjV3N0VMU3M3QjNyaFNwN0NGVE5iVTVU?=
 =?utf-8?B?MDM2VWNGUzFremNwcHdTZEhLOFp1YzRMUjN2QWNoMFF6SWZIU1QrY1hqTGc2?=
 =?utf-8?B?N3RVRnc1QnVzNXRWMXRXL0gyQXUyZS9CNm9DR1FJL3diRU5jQnU4Tm5QRzc4?=
 =?utf-8?B?UTFiNzhZTTlMMzFoZkdNM1h6UUJRajJvcFN2UFkwUjBnU0FZTWM4NlFJeHJK?=
 =?utf-8?B?b0QwVHZuWHlrMllUMXdwQjZSSklWbUxKUnFmTUtUSHJKSmlXSlFlUmFHL2lK?=
 =?utf-8?B?NDZpeXZzdkh6NFRyWmM1MUFCRXRuQmVKbDdPVXJSd2d3NER5RG1IcFN1Um8x?=
 =?utf-8?B?SWNDejRqOGpmQUFxbmw3UEZZRFMyZ25OaUVtOTVjK0Z0bENRRmYwZE5SRGxV?=
 =?utf-8?B?dVhsK1RlQStRenViOUpEZjRGY0dIVHh4T3B3YXpPMzBCV2FDbVpxaVdMbU9P?=
 =?utf-8?B?Q1pIN2VTRDBhNU5mNjhQUFdieGxiY1hlVWN0ek9CZU1KRkUybDhWR3dFdEl2?=
 =?utf-8?B?dWw2Sm1ZVFN1T1ZlSkI3U09hSmJtRXZwMzVCTkVoZUw4Y1QyQUFRRFZDeXlu?=
 =?utf-8?B?U1dVZmhjQld1cTJhdHRUT2V5TGZkVVJYR1BHbWhPalQ0cEgycWJwZmozT2lh?=
 =?utf-8?B?UWxZTHpNNGZrbFh0dXJNQXIveWJYbjA1enlSUCtxYm5pbFU1MlB3OTY5T3hV?=
 =?utf-8?B?Ty9zWDlSUTI0V0dEVmJudz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B9F507F76FDF2E4297A2B70503E6B1CB@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ce2ffc4-4be3-4304-27c0-08d98340292e
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 11:56:24.0726
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: wBQwLEK3Q4ftzjvf3sE+kSAiIlfqV0OadPHyAi/8X1Uho0HX2MAt4vGrvZDqp2+dMvV8vvJFaSF+QxEJ7TY6F4aDUS3h73jpUmAS3srKRms5tkOBgt9Wl6V0OBtt/YZI
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB3472
X-Proofpoint-GUID: V25B-ehXs9yaE_ysZvyYdqIWU-jskfeq
X-Proofpoint-ORIG-GUID: V25B-ehXs9yaE_ysZvyYdqIWU-jskfeq
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_04,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
 mlxlogscore=875 impostorscore=0 phishscore=0 spamscore=0 bulkscore=0
 malwarescore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0
 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290072

DQpPbiAyOS4wOS4yMSAxMjowOSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI5LjA5LjIwMjEg
MTE6MDMsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gU29ycnkgZm9yIHRvcCBw
b3N0aW5nLCBidXQgdGhpcyBpcyBhIGdlbmVyYWwgcXVlc3Rpb24gb24gdGhpcyBwYXRjaC9mdW5j
dGlvbmFsaXR5Lg0KPj4NCj4+IERvIHlvdSBzZWUgd2UgbmVlZCB0byBnYXRlIGFsbCB0aGlzIHdp
dGggQ09ORklHX0hBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+IGFzIHRoaXMgcmVuZGVycyBpbiBz
b21ld2hhdCBkZWFkIGNvZGUgZm9yIHg4NiBmb3Igbm93PyBJIGRvIHRoaW5rIHRoaXMgc3RpbGwN
Cj4+IG5lZWRzIHRvIGJlIGluIHRoZSBjb21tb24gY29kZSB0aG91Z2guDQo+IEkgYWdyZWUgaXQg
d2FudHMgdG8gbGl2ZSBpbiBjb21tb24gY29kZSwgYnV0IEknZCBzdGlsbCBsaWtlIHRoZSBjb2Rl
IHRvDQo+IG5vdCBibG9hdCB4ODYgYmluYXJpZXMuIEhlbmNlIHllcywgSSB0aGluayB0aGVyZSB3
YW50IHRvIGJlDQo+ICJpZiAoICFJU19FTkFCTEVEKCkgKSIgZWFybHkgYmFpbG91dCBwYXRocyBv
ciwgd2hlbmV2ZXIgdGhpcyBpc24ndA0KPiBwb3NzaWJsZSB3aXRob3V0IGJyZWFraW5nIHRoZSBi
dWlsZCwgcmVzcGVjdGl2ZSAjaWZkZWYtcy4NCg0KVGhlbiBpdCBuZWVkcyB0byBiZSBkZWZpbmVk
IGFzICh4ZW4vZHJpdmVycy9LY29uZmlnKToNCg0KY29uZmlnIEhBU19WUENJX0dVRVNUX1NVUFBP
UlQNCiDCoMKgwqAgIyB2UENJIGd1ZXN0IHN1cHBvcnQgaXMgb25seSBlbmFibGVkIGZvciBBcm0g
bm93DQogwqDCoMKgIGRlZl9ib29sIHkgaWYgQVJNDQogwqDCoMKgIGRlcGVuZHMgb24gSEFTX1ZQ
Q0kNCg0KQmVjYXVzZSBpdCBuZWVkcyB0byBiZSBkZWZpbmVkIGFzICJ5IiBmb3IgQXJtIHdpdGgg
dlBDSSBzdXBwb3J0Lg0KDQpPdGhlcndpc2UgaXQgYnJlYWtzIHRoZSBQQ0kgcGFzc3Rocm91Z2gg
ZmVhdHVyZSwgZS5nLiBpdCBjb21waWxlcywNCg0KYnV0IHRoZSByZXN1bHRpbmcgYmluYXJ5IGJl
aGF2ZXMgd3JvbmcuDQoNCkRvIHlvdSBzZWUgdGhpcyBhcyBhbiBhY2NlcHRhYmxlIHNvbHV0aW9u
Pw0KDQo+DQo+IEphbg0KPg0KPg0KVGhhbmsgeW91LA0KDQpPbGVrc2FuZHINCg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 12:45:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 12:45:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199041.352849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYxG-0003Ih-Ay; Wed, 29 Sep 2021 12:44:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199041.352849; Wed, 29 Sep 2021 12:44:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVYxG-0003Ia-82; Wed, 29 Sep 2021 12:44:58 +0000
Received: by outflank-mailman (input) for mailman id 199041;
 Wed, 29 Sep 2021 12:44:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2G/I=OT=yandex.ru=isaikin-dmitry@srs-us1.protection.inumbo.net>)
 id 1mVYxE-0003IR-1q
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 12:44:56 +0000
Received: from forward102p.mail.yandex.net (unknown
 [2a02:6b8:0:1472:2741:0:8b7:102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5e13ae57-78b4-40f3-8600-e51c00de66f5;
 Wed, 29 Sep 2021 12:44:52 +0000 (UTC)
Received: from iva4-b3068118e41e.qloud-c.yandex.net
 (iva4-b3068118e41e.qloud-c.yandex.net
 [IPv6:2a02:6b8:c0c:14a6:0:640:b306:8118])
 by forward102p.mail.yandex.net (Yandex) with ESMTP id 66918393E589;
 Wed, 29 Sep 2021 15:44:49 +0300 (MSK)
Received: from iva3-dd2bb2ff2b5f.qloud-c.yandex.net
 (2a02:6b8:c0c:7611:0:640:dd2b:b2ff [2a02:6b8:c0c:7611:0:640:dd2b:b2ff])
 by iva4-b3068118e41e.qloud-c.yandex.net (mxback/Yandex) with ESMTP id
 JFJqoQ9Dp9-imDq9flO; Wed, 29 Sep 2021 15:44:49 +0300
Received: by iva3-dd2bb2ff2b5f.qloud-c.yandex.net (smtp/Yandex) with ESMTPS id
 D9RmO3nEvp-ikxqcVOA; Wed, 29 Sep 2021 15:44:46 +0300
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
 (Client certificate not present)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e13ae57-78b4-40f3-8600-e51c00de66f5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1632919489;
	bh=bO0ezWmpyOcx1EZo5WBqFSk2HkPLSeWwEOd1JNPNNR0=;
	h=Message-Id:Date:Subject:To:From:Cc;
	b=bgHPrd6ROwwLuzvJkz/emHb2qOqT8thC7a19wt2t5u85DbY1RtbfH+/2V7cLdEDg5
	 jBxukboCLAW2XPmVzMfGg1PanHnEQVk90K4iLGfllYH3MiVg2ZM3u1b/l2Z/YEvcgj
	 pfVPK7QmaBUTpUmxpyw9HPtvu/fsD/ihm/CADCkQ=
Authentication-Results: iva4-b3068118e41e.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru
From: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
To: xen-devel@lists.xenproject.org
Cc: Dmitry Isaykin <isaikin-dmitry@yandex.ru>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4] tools/xl: fix autoballoon regex
Date: Wed, 29 Sep 2021 15:44:33 +0300
Message-Id: <57f69075b77651c64b7dc6d86acd03fe03b4a18f.1632907890.git.isaikin-dmitry@yandex.ru>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This regex is used for auto-balloon mode detection based on Xen command line.

The case of specifying a negative size was handled incorrectly.
>From misc/xen-command-line documentation:

    dom0_mem (x86)
    = List of ( min:<sz> | max:<sz> | <sz> )

    If a size is positive, it represents an absolute value.
    If a size is negative, it is subtracted from the total available memory.

Also add support for [tT] granularity suffix.
Also add support for memory fractions (i.e. '50%' or '1G+25%').

Signed-off-by: Dmitry Isaykin <isaikin-dmitry@yandex.ru>
---
Changes in v4:
- improve regex after code review

Changes in v3:
- add support for [tT] granularity suffix
- add support for memory fractions

Changes in v2:
- add missing Signed-off-by tag

NB! New autoballon regex pattern is good enough, but not fully correct.
See test below.

```
import pytest
import re

size_pattern = r"-?[0-9]+[bBkKmMgGtT]?"
pattern = r"(^| )dom0_mem=((|min:|max:)({size}|({size}\+)?[0-9]{{1,2}}%),?)+($| )".format(size=size_pattern)

@pytest.mark.parametrize('cmdline', [
    'dom0_mem=1',
    ' dom0_mem=1',
    'dom0_mem=1G',
    'dom0_mem=25%',
    'dom0_mem=1g+25% ',
    ' dom0_mem=1G+10%,max:100T+50%',
    'dom0_mem=-1B',
    'dom0_mem=-10240M+5%,min:512M ',
    ' dom0_mem=min:-2048m,max:-2048M ',
])
def test_autoballoon_regex_match(cmdline):
    assert re.match(pattern, cmdline)

@pytest.mark.parametrize('cmdline', [
    'dom0_mem = 1',
    'dom0_mem= ',
    'dom0_mem',
    'DOM0_mem=10G',
    'dom0_mem=-10%', # does not pass
    ' dom0_mem=-10%+10G ',
    'dom0_mem=1G,', # does not pass
    'dom0_mem=1G,max:25%,', #does not pass
])
def test_autoballoon_regex_dont_match(cmdline):
    assert not re.match(pattern, cmdline)
```
---
 tools/xl/xl.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 4107d10fd4..58af7936fa 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -80,9 +80,14 @@ static int auto_autoballoon(void)
     if (!info)
         return 1; /* default to on */
 
+#define SIZE_PATTERN "-?[0-9]+[bBkKmMgGtT]?"
+
     ret = regcomp(&regex,
-                  "(^| )dom0_mem=((|min:|max:)[0-9]+[bBkKmMgG]?,?)+($| )",
+                  "(^| )dom0_mem=((|min:|max:)(" SIZE_PATTERN "|(" SIZE_PATTERN "\\+)?[0-9]{1,2}%),?)+($| )",
                   REG_NOSUB | REG_EXTENDED);
+
+#undef SIZE_PATTERN
+
     if (ret)
         return 1;
 
-- 
2.33.0



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 12:47:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 12:47:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199048.352861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZ08-000440-QO; Wed, 29 Sep 2021 12:47:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199048.352861; Wed, 29 Sep 2021 12:47:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZ08-00043t-NJ; Wed, 29 Sep 2021 12:47:56 +0000
Received: by outflank-mailman (input) for mailman id 199048;
 Wed, 29 Sep 2021 12:47:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZ08-00043l-3W
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 12:47:56 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b3070277-e099-42ad-bdc4-93bb23a647a6;
 Wed, 29 Sep 2021 12:47:55 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2052.outbound.protection.outlook.com [104.47.2.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-16-tbeCnbqfPpGDHa0oJfEM6Q-1; Wed, 29 Sep 2021 14:47:53 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5741.eurprd04.prod.outlook.com (2603:10a6:803:df::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 12:47:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 12:47:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR10CA0024.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:89::37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 12:47:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3070277-e099-42ad-bdc4-93bb23a647a6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632919674;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=q8LufTiCUQsqHdGqUe2Z53cyWIXeXkkmydlokoMNKK4=;
	b=kPTfCItiHOrWtNHOct9XOIQqtVtE9DMBcezRIlUc8tqUS9Sa5Au8J32Rhk2H3r+hvnoPHz
	hwhBnsQwZ8IgSmsAjBbdZWRRqA+1GYfv3NRw/vXl/9q1oqT7qziVC4EUF10yWx3sQj7s4D
	dn4Hl9Tc0ZLlJiz0bQky88DNy+jNMMY=
X-MC-Unique: tbeCnbqfPpGDHa0oJfEM6Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=I5Fcwr0jtB+7+qt66ta0uxT9Of0oDiet3IpknjAG0d8ZHMeWV6bFznNReFVNYITq841f9ovvlyMhbqNrhvfmE1dIibPq1UCCejW142TTMWMZUoBLDHF58yyLqMj9GaOxifxvXRzu1/Gnm51AEf6nnHKEg+Y57ejCMqsvrAE9TSt6RTsbaV5V7bdtGCV/TFh9CgtcfUxM8w853YdMYwu08PTPYu57LmUT7BcO+970fNSjT8hp2AtEpqtnLP6rwNNVCtblYADTOSvF2Nyp2ZQAThzsStox2yJRpIpSf5ySyj6LzyKF7SpOAaNcFNtYBj5djHaft646m173nbNBpP3yKQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=q8LufTiCUQsqHdGqUe2Z53cyWIXeXkkmydlokoMNKK4=;
 b=YLnCyRzo0kE/YJrwz2kjctT2hXNrEP3cagd4FtkV86V9kf0D7fTBDpt6dipHjlIT0VukC6NF3ydMJXOOlq+xYu2uuO44/DpTXpYZkP07YSGelNWXKrPdV25Yh3nrNZqSUkT7Is8xkjD+CQk33wASxyHVH+pMx3P5obOFkEBgk0pD2Tfnu1hcs2tUQkfcu6IU9ZlYZiynLOd0btkI56on90bpAnqWo7izrQQsazKgTicY5t0I498YM6LbMD9NnZZ0yYbYWHQ3etU+zjMRimgTqD3DBpyDZw5LGbQuI4bc0UZJz7zn/1ICXqn4fsaayeDdPQDuy/omc/T5hiDZHwRRQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/paging: restrict physical address width reported to
 guests
Message-ID: <e0acfa40-0901-2fc9-31b6-c2609625de22@suse.com>
Date: Wed, 29 Sep 2021 14:47:49 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR10CA0024.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:209:89::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 57e79f26-3d35-4cbe-9f64-08d98347594e
X-MS-TrafficTypeDiagnostic: VI1PR04MB5741:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB57413B7124CCBA9D363504B0B3A99@VI1PR04MB5741.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5797;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2hzOjIQ9Kj72tyCwMqle4AvB1pBoQiUiSlU3NkqItd+KD0WznVSiwXEcIZUX0JifQQ/qVGkEZrRIR2V+ftRtXrBmPgRoQuJv1I7ApVuY6gWlZ++u41I+nedar0XC0rCaO12D0Kt5ahWJld82v3Of1osVKsJMJY1hXw9VNC9L4OBDRSax6gOfIExRRluR1Nfo0UZertX8sI3ho1mXSEr3WirQgiXwhthlcvdBSvMPB9JhJeckiCuOYoLlTYFVRu4g3wypEKS40UhmkyhYVPBbvsv7lG3o8Z2ZArx8RbHTDYDStFOLwLap2PvfSrvNI5/NTEItk6hzGAoujxI0JALrtD3K9DDuSSrWFshXc2Kor7G6BqwQnziD2PDZqEnF3HszbAaC7Pdua4w5SEUKLM9n0sBRJY3PIHS6QPPpDmsyPuKZIX+ajGQnWGPNzp+k8zevIGWcjZawjTnA90ot3jNS5IT/ekMuqcclbEUuZ1h17xNBFPdLTzOcuR+1eOQbFvVRAt03QBiV0Xww8zF8NIokFJdOWAgsL+DdptweIXLuEeZzLo2M88a93/AzdjicwLOeAquUBeEOkVA0Q3OM4mDkLlKVmNEsJyxmgucikYwcyMxqTujgJwenD1XJ7G0dXpmFJzRQFi9RoxIjoZie7BMYoBvEOkLLYAd0FuDcA+gEKJruyA6m6YSl8IjQWi0pYhMltMzfpOjlC4d4WweVWLS11bdDfrt48MKHi7lyP5KPE8w=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66556008)(2906002)(66946007)(38100700002)(66476007)(956004)(8676002)(54906003)(36756003)(6486002)(8936002)(2616005)(5660300002)(31696002)(31686004)(508600001)(16576012)(4326008)(6916009)(26005)(86362001)(186003)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WkNOd3lZcmY0M1RqeE1Bc013MnFuSitzK0tDeUxYYmRhRjZPVTV0RlZueTZE?=
 =?utf-8?B?K3JpMkdJcDZCVldlK0dRZy92NTdOeFRHUnVvWXJvZEREaGVIU0ZnanBYMEdu?=
 =?utf-8?B?V3JQWFpldEVKK2tjLzF3WWloelJFRGNMMFRzQU53d2VkdEtEaisrRkpiRDJn?=
 =?utf-8?B?Ymg4cWlFWkRLSDF4YW1yMjNoalpUK2VGeVRvMEU4WThpTS9VYjhLZ01OYUF0?=
 =?utf-8?B?QXFYSWxkdTlqOXhLL1NWZng2VGxCeXkyY2R5RFhHVmE4UVZMb3N1aXBuOWVr?=
 =?utf-8?B?VnBJTGlxb2NoOFhOVkFmMEpiTTUxRldpMUg1SlVYajJRL3NrOTEzekt5UUJZ?=
 =?utf-8?B?a3F2elNrRDZnZXZpNVBzajhPU21BcENOUHBzeDBEbXVFb2NMQlBrNnFaN2Rz?=
 =?utf-8?B?Nm5WWEVaMERYMDdja0c0SVg1dy9oaGNHdHJjdmtZTytxWGp5QzErbTFnK2h2?=
 =?utf-8?B?aWR5bThacjRaaFNtbEtMMDJjVFRXaTBidjRmaFZET0tkL3B1U1V0dEFETXBH?=
 =?utf-8?B?N080WU5kUWlBV0YvbDdVT1ZJQm5QelpZU1R4dklPSWpvR0FvMU81VDUyb0Rx?=
 =?utf-8?B?UU9hOFkzcGdxbDVNNXZSSU1ZcEVFSnJ5Q2FKanVUMjB2NVR3V2lTamNkQ0gy?=
 =?utf-8?B?REkrNnozbTBVTUNTYjJNMXpURllvcU5DT1piT1JNbzRsMzN2dzQvb245Yjk5?=
 =?utf-8?B?L3JuSnFRZDA3K1RmY1J3TGZDblNMdldESmNrS1RhWUNqZWw1dFVxV2dQbDlV?=
 =?utf-8?B?VHMvTG4wQzRoQWNYQjVVYkJVNmk5dXBKVHB5NmFCVWxyTFF1b3MzcXo2cGQ0?=
 =?utf-8?B?V1RKMTgxdjZ3T1FzbnQ1Yi9vb3NjYWcvam1ISTM0R2orYVU3cjFKUTJpMk5Y?=
 =?utf-8?B?akZpR1BPNGY5eE43VjNLb05XK0ZVUGZQdk4vRTN3YmR0OTFzWS9XRGJEZ1Vr?=
 =?utf-8?B?aVRtZGc5WktEaHlJakRlUjg4M2ZvNUFOdmEwQm1xcnFoT2lJT2xWb1FQU3RR?=
 =?utf-8?B?RFB5VHJsTGxITk1TbTE1b3dPYm4yeVg5VGtNclluOE44T2Z0dXFhRUxqazYy?=
 =?utf-8?B?QU5XZVdFcUNjVkROR1ZOWkphTllVNXF3aTZKQitWSzlSSmxFajhGZDk2a1ZM?=
 =?utf-8?B?dWM4YXFldnpaM1cwajE4eUpoaS90RnlHQ0xxdEdtSWhvQTZSN0dhVG1vK2M1?=
 =?utf-8?B?QjN0aEtXTVlWVXdodjhDYmJjTWxNWUlSZjZaSE1tR0JNVEFqRWdvUnI0QUVV?=
 =?utf-8?B?UzRoRFBDeDhDT2sxV3BnSFNpTFFqaUh1eG9kNHpJdkRhMlpLN0ZheXNHem45?=
 =?utf-8?B?eENDMytYZXJpblFjRy9XSEFRelZBbDFLTmNhQ01WY2I3dDVaMGVQeGd4eGIx?=
 =?utf-8?B?aFozN0VNSEVjMk85ZVNOY2ZnSk1yMUdETjFwRmwvQ1pCN0kwL0syNmhjR2Nt?=
 =?utf-8?B?azJWMGd3Mng1NXRraVZMaDl5MkRKc2ZxRmtLUjJubHFSVXdEdHhvK3UyZExY?=
 =?utf-8?B?T1dSNXQvZ3hUbi83aUlDY0tMRlJtYW5DYU9HcWxCb1ZXZHpyMXVwTFYyS0Rn?=
 =?utf-8?B?Tm1EVUJWM3ZvWURHd0ZkeFZnMU1Qanowd3dCMFVFcFo1MVhHOW9VNVMwUFU1?=
 =?utf-8?B?eXNwbFAzZHhVaXlPQW1VQ3RJSWtadEFrbHlTQ3g4Mkx1ME4zMTRxK0tyVGZa?=
 =?utf-8?B?TTZVYms1K3VpTnFzMXpCcFI5b2E0OWxEWk5zYU5yckQxellqVnQreHVCanhB?=
 =?utf-8?Q?hj8jv75Pu//jDjA1vRT4ZeQRDWpak4idZs47O8u?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 57e79f26-3d35-4cbe-9f64-08d98347594e
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 12:47:51.4676
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ovCQe4XfL5DP/FReXXGijNvFa41NB63jqh4ukMeog0w7gcdyibpQ+5ByxN6ZQrFhlxxq/nkOKFN6dDRmDF0NMg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5741

Modern hardware may report more than 48 bits of physical address width.
For paging-external guests our P2M implementation does not cope with
larger values. Telling the guest of more available bits means misleading
it into perhaps trying to actually put some page there (like was e.g.
intermediately done in OVMF for the shared info page).

While there also convert the PV check to a paging-external one (which in
our current code base are synonyms of one another anyway).

Fixes: 5dbd60e16a1f ("x86/shadow: Correct guest behaviour when creating PTEs above maxphysaddr")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/include/asm-x86/paging.h
+++ b/xen/include/asm-x86/paging.h
@@ -401,11 +401,18 @@ static always_inline unsigned int paging
 {
     unsigned int bits = paging_mode_hap(d) ? hap_paddr_bits : paddr_bits;
 
-    if ( !IS_ENABLED(CONFIG_BIGMEM) && paging_mode_shadow(d) &&
-         !is_pv_domain(d) )
+    if ( paging_mode_external(d) )
     {
-        /* Shadowed superpages store GFNs in 32-bit page_info fields. */
-        bits = min(bits, 32U + PAGE_SHIFT);
+        if ( !IS_ENABLED(CONFIG_BIGMEM) && paging_mode_shadow(d) )
+        {
+            /* Shadowed superpages store GFNs in 32-bit page_info fields. */
+            bits = min(bits, 32U + PAGE_SHIFT);
+        }
+        else
+        {
+            /* Both p2m-ept and p2m-pt only support 4-level page tables. */
+            bits = min(bits, 48U);
+        }
     }
 
     return bits;



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 12:55:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 12:55:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199053.352871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZ6r-0005TV-HU; Wed, 29 Sep 2021 12:54:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199053.352871; Wed, 29 Sep 2021 12:54:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZ6r-0005TO-EJ; Wed, 29 Sep 2021 12:54:53 +0000
Received: by outflank-mailman (input) for mailman id 199053;
 Wed, 29 Sep 2021 12:54:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZ6q-0005TI-92
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 12:54:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 651d428e-b153-44db-a19c-e4c22b56cb6e;
 Wed, 29 Sep 2021 12:54:51 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2053.outbound.protection.outlook.com [104.47.0.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-b10iZLWqN8Oio8cmDRAMaQ-1; Wed, 29 Sep 2021 14:54:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4351.eurprd04.prod.outlook.com (2603:10a6:803:49::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 12:54:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 12:54:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0057.eurprd06.prod.outlook.com (2603:10a6:20b:463::33) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 12:54:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 651d428e-b153-44db-a19c-e4c22b56cb6e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632920090;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R6lQRwJmICDJFvYRvRneNnb78+i48pz1dYJPNPoVD/8=;
	b=O+V6IeFhezD1GjtRIPE/+/R1YWBDDXU7CNzhdN4InSE6dCLH3PHkiLbSX/xgZ7Ybs3MLAO
	xNYaAjAJennWowMHEj0Gyf/22iqSUl8rFCNczKmCgEO+7hJMFMokKXz0j5/E+Lyty4hrWQ
	c0UK6KxGKWm9rvoFZfnn9fHnIAuDXT4=
X-MC-Unique: b10iZLWqN8Oio8cmDRAMaQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U6Bwsd/mSFzMf7ppUv81MDqILPDrz/JiRpfZfNA/bKJlEn3gLKLLM3RXZJ5wfrL2arX/PbZxqL3uk2BfEjA+6bXZr+rjiu9ZEp4NNIozEoyXuV55LATh2GWxIMe89eT32D82b8aXp7nkv3dBa0ih7MunXLa+iCHH97KfLoON5RKgHy2a44ZvwDA2IkWA3KRFph1qpqJ3oMjePDmHWvZ99vPqhjQ74rmFmyqHoHB6M/8autoI5LZ9aZMyOS9VRurs0VDL6+kIOgbwDGdK/dCFEFPL0GJEiv1xZbsDjawkiUM8xWHpf2rQBj98Pi3jOLfXuvV248aqH1pqxlCZ0EWAOg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=R/QiLLZSov1Cp8mTGsC2wZm/hqDv/QWpizr2+S/QJ4U=;
 b=TVf5Dg5uiWtRKtoxd9OkEhiCr3avl+p6llWQPo0zuNkgxlsmqO9qDEouabnuN+vnCE9mbWCuNO8G6RbANnRr+5wm871NVy3imJq7QIsNymeKcs9avWmoSJw6U5cnZ0MTjoNFrzG8/GJcFpV1JDzeo/89L5jL7yHz5tTExbo2YubHeYlT3boaUheeEt2wWUZ5TiNew9WDRQPs7AxklcBltvEc/7oo+y0BKryPuIseZThlIXQNSsQcNX9qQpqO8pyhHx39cvUDeEMHcxvrHgreYbQNOz6XjfXDbKqZ/ZZYEd3qFtvl9zHCSW4z637uHO58d41FP6d8v0iav9cMmr9hxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Michal Orzel <michal.orzel@arm.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
Date: Wed, 29 Sep 2021 14:54:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS9PR06CA0057.eurprd06.prod.outlook.com
 (2603:10a6:20b:463::33) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b3ea754f-a4bf-4dab-73e2-08d983485004
X-MS-TrafficTypeDiagnostic: VI1PR04MB4351:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4351685D5C24D43CA5615D7BB3A99@VI1PR04MB4351.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kJl3IVsadClgSpnIYDX05zD3rZRMPtaxgCfCxPjZf3U6ivlr5BSF5ZCnvr3/+2Di9e6pv7p6Mf7qGOP4vxu3bdCbYA5HGxL00p1njVWumCvbbihf/lmXted1l0UXa5lhlI9o4Q2YgORIm4SUR5LB+nbXa5qnRtNyqzSIfRZHw3a9vgWqSl6ONH5j1hHOjfSYpLoPWrZeiS/k90y/icLl9lraptL4PED3caaWpaxWCSVNCDbHT7UOv/UHDtY6Pumgq/3P1RKX1LbodsIzKoGl6XDG8fTQr2JCuOaG+arrTn317I8NAbr02mAiOiNFaj481t6PKJJ1k07DVG73dQqb4qTzWlzsLr92ZT2bn1QMh+9CBSI4O5k3xfPz987uRCdgOoNZeuLldhQbTyYGHRvcRMqnpRHuCxlPqRbHoaEV9cTKUz714cDnDNuqArmlUaRkZbb0LdwaKNs6fiTZoEgmVuQ6yYdKhJtT9naWQwK0ZQqOGeaP0APFawcBbsoaS9lQaLh2npFf3rJR23l5ikuaBej19pcP9cu/QsdSgJ2/DsvKq/QGOZ4LgZfgkwpLTXhBCTYmjWOYM6TZGk8Zd+gn9sMVMlTsD3XZG7Vyx94RJxrRVkANQXSI/4R0CD767+yvoIcKceDKx/TbL+H0wIzvSbnRjLKusam/rc4XaCBvlOdiHtb5nhAyLdxuAyxK8uUYQGjxjQR6HSJ46sQSWow+IHBGH7P9Fvb86FXFftkpvRSVkAWW/VnVHIwJEa/9KQvI
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31686004)(6916009)(53546011)(4326008)(5660300002)(38100700002)(186003)(508600001)(36756003)(66946007)(8676002)(31696002)(2616005)(7416002)(66476007)(66556008)(2906002)(956004)(316002)(54906003)(16576012)(26005)(6486002)(8936002)(86362001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?hiokbUgcWSzDQncClePIqz9afOPrG0b2w0y+2zq2Ww6tFrsGMIbEhPBZHl3O?=
 =?us-ascii?Q?Uf+ObDQEv3h1BDnMwke6Lsu14iszYaMckJKG2UMpjjqvJOG9WTNnuwRaKgh9?=
 =?us-ascii?Q?gUJcQM//4GMTlfN0IooNIxtRcPSNriAzAB3zGu6CplAgHsdyg9wk0x7smROo?=
 =?us-ascii?Q?XCnmakp+dLItrvwo6XTCP887JKTtoiV0+ybo9OQ67vcNcy/ITfsoMnmUTg+O?=
 =?us-ascii?Q?lMs0/k6ECHNgSFN8RMKm/d7Xp8UemODsSxPdoL7e3mDJuFQSuyyidpTaju1N?=
 =?us-ascii?Q?/NA7gM/beT9MTS6LxDKvOhIpJfgHCHnvdsJ+XMYMX1QJFo26mRNNCZvR16c3?=
 =?us-ascii?Q?wlLpB0JSpZyKdzyqLeRK8ZzzPIEoMfYREiAjjQkA+mBBvoqQnmNMqAhZfn//?=
 =?us-ascii?Q?4jv8xFmFMc+2n1Y5yM9IxBPmwbIGTvHjSaWuyWxps4St3lbh8qDBIERPZ3uT?=
 =?us-ascii?Q?SjpayfSDoMOXctuhvWWu7FPOercgVyYpmFIJXceI9SBGCH36mjRCtIW34599?=
 =?us-ascii?Q?LgiKDf+ZYgkMoLoS+EIijRJ5uTdiUEPJzLc8Dkb2oWE/v5cQMrH7/ZHdvYEx?=
 =?us-ascii?Q?fN+1bGvLvePpcugujuq/gF0SMpK6YqzO3+ZJSA/Sz2uu3EykCoV6lc6DVyna?=
 =?us-ascii?Q?h7GZKtdmMgDeY9dyyKlXoqAumyuyaMH0mNdWxRoiJKCMd87X5a7XKFfA1UUJ?=
 =?us-ascii?Q?ry/sF1f6SbzCkEzOsCcDFOptueb2qrV7qz7VR9L25pG0dK5WFwXtc/KnQDze?=
 =?us-ascii?Q?6kUsJG2YGJS6Jro4n3/P/7ux+mK7hu/8n9cx3pWU6+kJIKyvc9Y8IMalYaRV?=
 =?us-ascii?Q?/LJxc1tVP9BFOhyw02CiZUHf2y8SpLE6yrsaZtpbxbVuRzReJE6rs67TYNFx?=
 =?us-ascii?Q?KeoJQmhqOI8fYIV2Mso08Blo9OeMcWXSgg+Ij9WSVwOHu9cEFPvf7/p1nVxC?=
 =?us-ascii?Q?8kA5pr+8LBPgAnxY9x54zRGylSxJQZNU2B61caUAG2mFJGMzeGJ+MX63Jzxq?=
 =?us-ascii?Q?ZNL+Tmb0ED2HbPiz/ES8JFVzelTCt3z+rDnPfW2OeMKmZTS/vzijwScNlQ2U?=
 =?us-ascii?Q?hGtlSeqENL1zbhG0oDJLO5DEVDz94bA76YuOwXb+zQK52AHjJziODAYAnR42?=
 =?us-ascii?Q?RfSg/QL/ebF4lFwYJ/IsTkKEs0wgwQgYd/gOJiJkJDEw7HGcI31B7T+8/zHR?=
 =?us-ascii?Q?FysilX6MuwCZOBfY3dC/OIaDH5qCxNs8/Tyg7HRn+teje8l57I+kJxPC50V7?=
 =?us-ascii?Q?IbYkwvdlCbtZClKKgatP3dBcjnvWLCIcXyouFPooeYk5/sXWrg4dEE1lGD/6?=
 =?us-ascii?Q?r/i8Z620if22a8qd6gEqeUif?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b3ea754f-a4bf-4dab-73e2-08d983485004
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 12:54:45.3778
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RJeysp1j+dxIFyRLLqnAyTNGIVQ0jkHN0g7J/6okIDg3nF54dGERYgPkreJZEToudeoY1OVCmicQ0D++NqNV4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4351

On 29.09.2021 13:56, Oleksandr Andrushchenko wrote:
>=20
> On 29.09.21 12:09, Jan Beulich wrote:
>> On 29.09.2021 11:03, Oleksandr Andrushchenko wrote:
>>> Sorry for top posting, but this is a general question on this patch/fun=
ctionality.
>>>
>>> Do you see we need to gate all this with CONFIG_HAS_VPCI_GUEST_SUPPORT
>>> as this renders in somewhat dead code for x86 for now? I do think this =
still
>>> needs to be in the common code though.
>> I agree it wants to live in common code, but I'd still like the code to
>> not bloat x86 binaries. Hence yes, I think there want to be
>> "if ( !IS_ENABLED() )" early bailout paths or, whenever this isn't
>> possible without breaking the build, respective #ifdef-s.
>=20
> Then it needs to be defined as (xen/drivers/Kconfig):
>=20
> config HAS_VPCI_GUEST_SUPPORT
>  =C2=A0=C2=A0=C2=A0 # vPCI guest support is only enabled for Arm now
>  =C2=A0=C2=A0=C2=A0 def_bool y if ARM
>  =C2=A0=C2=A0=C2=A0 depends on HAS_VPCI
>=20
> Because it needs to be defined as "y" for Arm with vPCI support.
>=20
> Otherwise it breaks the PCI passthrough feature, e.g. it compiles,
>=20
> but the resulting binary behaves wrong.
>=20
> Do you see this as an acceptable solution?

Like all (or at least the majority) of other HAS_*, it ought to be
"select"-ed (by arm/Kconfig). Is there a reason this isn't possible?
(I don't mind the "depends on", as long as it's clear that it exists
solely to allow kconfig to warn about bogus "select"s.)

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:12:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:12:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199060.352883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZNQ-00082Q-10; Wed, 29 Sep 2021 13:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199060.352883; Wed, 29 Sep 2021 13:11:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZNP-00082J-Tx; Wed, 29 Sep 2021 13:11:59 +0000
Received: by outflank-mailman (input) for mailman id 199060;
 Wed, 29 Sep 2021 13:11:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZNO-00082D-8S
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:11:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d23241c0-2126-11ec-bcfe-12813bfff9fa;
 Wed, 29 Sep 2021 13:11:56 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-7RCA3DEfOsiLzOM7fYah2Q-1; Wed, 29 Sep 2021 15:11:54 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 13:11:52 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:11:52 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR0202CA0049.eurprd02.prod.outlook.com (2603:10a6:20b:3a::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:11:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d23241c0-2126-11ec-bcfe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921115;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=WtmTa1abaYwRr4a1lAGNRAUiwhSXtLQOad3omEWetMo=;
	b=XNXEvTq39iNruoZK3lXwXpfx6B/sOXeL+nhZNu0zoBMVEhel+kb5AMfGmcNQUpU0rtMcCC
	L2hxvjFu/5MIWWiql+Q5/9L/5QjgKLhBzZq4Y2otw/4fXxXt/MlNjLjzh6O62AmvxiCP0S
	ss6XPefHDESO3I0oKUK7UqnnU1crzAM=
X-MC-Unique: 7RCA3DEfOsiLzOM7fYah2Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DMvw07FLyHLvJr4UIjnunBwuLQ7DsBsJFdyh+YDnCQJhVMgZx8xH43MPujj6O/EyBAr9oEujsat6Rdo/eSRGfqHNPZZTAYVk5EXnIX0uYOvNHctOEpfHgL2k0oOlGRpm4KIlAXpxhW6peReJWNQKjRFMa14LnmCS6LAePw5uFeMtPVC9xn9rba2dHsGTNQQEY11rT0Nu3XKsak6shu5Kf4pPxsCzrT5QfDlUkK7qXzDW7h4aFJQDsvS/q8ISmFLHPuOdUS1ioHtDVspIAIa9d9npEJ2FLWBWqNF/r8YMrBSoAYzAL7Zf7y5+VZ61P96JGX4VsDBYQN7vVTh6XcNn/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=WtmTa1abaYwRr4a1lAGNRAUiwhSXtLQOad3omEWetMo=;
 b=nGtE5Hd9JPTYQslYULPTTwa9C5g9Tcz/Wox26+YbjbWgaScgE7s+ZRD3/8g3+YWQlOKF+uUfuTwXMq2rIJdIYLbPGdaU6PADkvArX6w+0Jp37gw5/Me3YcG8WN4l4F3vrWZcuVKI7m9O+74jd0FrTE8pQkLvtj2NyC+Mj1BnWy1lMvBVQqHNQtURfd6qGqIWBo0RQiD0r1avqYG1NeZ8Lk75OL2+3SzmbvcbGxEzSJ75ZQW+zfVeOQKYSE/4k1FRbv3mJah4dS5U7vKN670QRvFO4+jwYaW6Q75/zL61dOfXsXY9A4tnlhIMoCry9VOie45C4mpGm8PrHkB+JX+XRA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v4 0/6] x86/PVH: Dom0 building adjustments
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
Message-ID: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Date: Wed, 29 Sep 2021 15:11:51 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR0202CA0049.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 49aae4ac-647c-487f-8834-08d9834ab475
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435253C16CCCFC48CD31FF41B3A99@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XhEErkFVYiwxfGRNQwGVfxtFykZ1o2h47Ic/i7lqqmMxbb5gGYvv06iVApPKp5zwVeGB2eDcACqaC6wwPKDDIXbIwakTwldHM1nVBEkpEv56n/lCeBFUx+9jbaqHnz02eksZlKTrqGIFH1acSuaHXkNSMKNx64GS542ZZPFf14LZ9n/PIGW+fdfzNmkfV2vjDTpbpDkuFdMK9d0yJ+6dweClxQXfqAej7BVDQOk2jYCxECLKyoyhy+mO6WvZDthwBMHareJQsHlHBexpLppHLWIzS6kPQA99xzDb3NtmtW/bW3BXq9iOGc1Lt7LNhizD13lS6uhCwLoLPu0OO4rKKQCtt97IoH1CLGB4T7t9wkRFPGvqj9z1caY5E+QbP3lGKGtJsGUsICKvX9IvLV4v2ACX8aLYFkZ3iahpqOApLU5RQpjQiVJ/Nax3h22IhyKYfA5gjcCDyyTwgoiE71m+Q2NWEOgw90TDcrYNv2+SyMj+2904gRan58a2A20CExXiF/uFw0z65Q65g9jITDcF/i8ZfiR3J4d/+w4A61Hjr84VjY4+i94lw8mMcGJCfQiv3oe83Yg5jkt89ky9/KNf5VgZfHl+HlPVX8DNKcMHX4+0+MfIWppMQru/5lkZ7an5NS5Z1F+Ljr4FX7H4HLIU04X8ky+3RSG+FFqP2SDlU2Yorb2sOhBzmoTG2rsdoLrDDdsjAoZgoiKe28PAbd+hBM+quFOOTbDUJlzYfNbqrXw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(6916009)(5660300002)(38100700002)(54906003)(8676002)(956004)(8936002)(186003)(26005)(31696002)(316002)(86362001)(66946007)(31686004)(4326008)(2906002)(36756003)(16576012)(66476007)(66556008)(508600001)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TEE1WXN2V2FOK1daQm9WM3d1OHo2NU1yYVJDVlFMM3Jxc1NabWlVa1NvNXp6?=
 =?utf-8?B?MnF1Zmd2ckt0clNyV2xHZlgyT2tsUVJlaUpOS2U0V0t4VUZlamJxUXlzRWVG?=
 =?utf-8?B?Y1dacUZGejlReFJvSkhsd2NzcEc4c1ZoNURZK1lnSkc5OTJNNmxRUlZGZWQx?=
 =?utf-8?B?ZHI2TkFwVWFEZjd2QXdVMzBWSlMvSTJkR1ZuUUZ5WGJCV2xNeUw4SmUvK24r?=
 =?utf-8?B?R01GVjlzRDB2UjErRG5yaDlrR29qTGV0NVFPdFdtcE9pcXUwL1RhRzF6bTdj?=
 =?utf-8?B?UFZBdTBIWk44eEZLVElFNnFKQWhWbkE5QkFnSUhpTzVWVjFINzZrUkdUWWlB?=
 =?utf-8?B?ck9IdDhpYnpBUENYaEdDQU1vUVh1TTFSTHk0ZlRSSmg2WE9mTmw4SnU0alNR?=
 =?utf-8?B?Rit2SzRuSVhEKzZ6MVJJSVJWSWM3U3VYeG1yeldqM0N5T0gxam40L2pMQ3J4?=
 =?utf-8?B?RE91N1luUWIrWklQd0lpWjVTaEJTdWJ5d0trSEtiZnZHa2NBWGlPZ2syL2Vh?=
 =?utf-8?B?MlVwckZNckhINEIwT3M1NFpxSzIxVUc2WkNET2tqZGZ0RlNkR0tLTnZBek9h?=
 =?utf-8?B?Y0lpcWtIeVVralhqVmtTc1YwbjcvTit4eFZPLzJpNzBvdmJpNW82ZDFSeWZS?=
 =?utf-8?B?VVlhUGE3UEZDSWtERnNvaEdSVm5VRGtzV29yQXBLaGhmamJxWVROUDVwUis0?=
 =?utf-8?B?VWI1eVh6b1JJNFFjaExnYTc3NTdya3BuZWpzSEZhSzZKKy9RYmpRUE12b3Fk?=
 =?utf-8?B?V2I2akdBWXIrTjRrY1ZOWi8vMm1NQ3hiWDQ4SUxvQ002MGZCRkR3U2k0c0h0?=
 =?utf-8?B?OFFvc0NWWHV0OS9NakNOSzduRkM2RFpmVjBxUG9Jc3pJQ3B3OXd1WjlWVTg4?=
 =?utf-8?B?Q1o5bDl1T0ZkdlFDOVZ6aWZMOHVrc2RQb1BhTHlNbmR5cXBrdkoya2g1bjZ6?=
 =?utf-8?B?UytueU8vL3ZkazJZcDk0QlE5ZlNtZGtERC8wY254ZndjbzlnNmdSaWtJa21o?=
 =?utf-8?B?RDNRWFg0aHFXVTB5OHFFWFFrd3RnRU03Z0tna0xZcHE0bmExS3BCY1hISzFu?=
 =?utf-8?B?Ri8vQWNjd0dDdGlnamd2bVg2QjZEc3RnTDFuWEs2cDFzaUNQaWY3Y1UwelJB?=
 =?utf-8?B?ckdzakJ4ZEY2bjF1clNMaGhjV1lWdTFna2NCa2k3Rko3TWtUUDNaU2NzbjMy?=
 =?utf-8?B?azRmRWw1ckFRLzZLUEw4MGNHVHlNZXdJQW9mMnRUSVFuQXdvTDducjFjeWdq?=
 =?utf-8?B?STZOVVVEUXJRNDV5M0pwUU9PSU9WS1BvTkV4bDQ5MSswK21FbnlyU3BIazV1?=
 =?utf-8?B?WkttYU5pZVZMNGgycmJnb1RRcHFMbURWOWYySGhBWkJldEZQNFFwY2RWVlVr?=
 =?utf-8?B?bnFRKzRCN294Vll3cVNnT3RGajhGV2xJTHA4UFdweHFKN1VUZXZIZ2grc3pO?=
 =?utf-8?B?NHlQZTJXT0sxWDZJTGx3aTR0eUdVTXN6eEpDVmpqZFNjSXVEQlhYMXhKZG5n?=
 =?utf-8?B?V09IaHhidXRIMnBoRVY2S3Q3Qy9rRjZzRzU1cjB5Y1B2T0xUaWU5MmQ3bVlN?=
 =?utf-8?B?OWs3NUYreTVkOUJNbWl3dGd0VUNBWmh1UDZtZWZMakNJVTFSdGFkVGVxNU53?=
 =?utf-8?B?R1A3d1JZQTJYd0xBWjkzOTErQnNmY1FiSHRJSFg3aHVKSHZzbThXaTZoblR2?=
 =?utf-8?B?NGxOU1RKRTFRdko5K3YyakhaVnNiemhXb3MvQ2NuKzVUbVJxV1g4Vk5YTnJn?=
 =?utf-8?Q?lHbsyGwx7OV+PJeKlv9XGrRBmbJwNBePhjbaGMW?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 49aae4ac-647c-487f-8834-08d9834ab475
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:11:52.8815
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VIux4VuSnYpMfxro35gXxbaCs3s2b2/sormgCp5huhlo/Xi5jO0LUBGRL4tXFYVUQPyQSPA0MwF2SOHVnGBkrA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

In the course of fixing XSA-378 fallout I ran into a number of
issues, the remaining parts of which this series is trying to deal
with. The majority of the changes is pretty much independent of
one another.

There was another rather basic issue to fight first (patch
was submitted separately as RFC): vPCI wasn't aware of hidden PCI
devices, hitting an ASSERT(). Obviously I couldn't afford not
having a functioning serial console.

Compared to v3 there is one new patch here, and the video mode
propagation one has been dropped for effectively having got
rejected. I realize that discussion on what exactly to include in
patch 3 has not settled yet; that patch is deliberately unchanged.
See individual patches for details.

1: PVH: improve Dom0 memory size calculation
2: PV: properly set shadow allocation for Dom0
3: PVH: permit more physdevop-s to be used by Dom0
4: HVM: also dump stacks from show_execution_state()
5: HVM: skip offline vCPU-s when dumping VMCBs/VMCSes
6: P2M: relax permissions of PVH Dom0's MMIO entries

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:13:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:13:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199068.352894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZOt-0000Fm-Fu; Wed, 29 Sep 2021 13:13:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199068.352894; Wed, 29 Sep 2021 13:13:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZOt-0000Ff-Cm; Wed, 29 Sep 2021 13:13:31 +0000
Received: by outflank-mailman (input) for mailman id 199068;
 Wed, 29 Sep 2021 13:13:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZOs-0000FX-3a
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:13:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 555b93f5-22d5-4a13-a4b9-60bd8575a494;
 Wed, 29 Sep 2021 13:13:29 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2050.outbound.protection.outlook.com [104.47.14.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-39-ZXZ70dRTNBqTnbNPCsaFTw-1; Wed, 29 Sep 2021 15:13:26 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 13:13:26 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:13:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0094.eurprd04.prod.outlook.com (2603:10a6:20b:31e::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:13:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 555b93f5-22d5-4a13-a4b9-60bd8575a494
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921208;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=K8riSR7WaBkT+h9mazx+moSRxeE0eWzN0eTXIGip8o4=;
	b=eIG2E56kxFd1N8jEN00fltlinortw/dCKUW2ECpVy8AcT2ruVnsyEXqs7PM4uenOrhA+Mg
	8YtJ+En5P3E8VoSzwSgKkWY+bvge+KWxH7ZRotEcnwsIA6oXoAgb4oGMA6sli5JzdSqQC2
	RQ65fjEnTZywzduF9pSCcP+x24d9el4=
X-MC-Unique: ZXZ70dRTNBqTnbNPCsaFTw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fW7nLgIeU3yeeIdJ7tE9Neir+vYLBsKSxsSDUYI8hIhg2wgojB0uXfLF1gJW1NArR80ZBwNlJ5A1zM1hbwRyo2i8ka0wOMkoHI9JzcNqEMmyAi/kAMioV6WJeM+bFRtFt1hl3BwFsyEyOeNcUq0n7U/NlPDJmDibiuHUARR9aVP5ACNvNuWQxJFOT3wUNplXOIjwYQV0v5uC+ozGnL3lVugffchRA66/5wSnLmUqaTyB5RKTXv4SYt6dmYuwQ6SFTU64be1SXJ2BHPVTKTXV93Q8gyQKtwH147+PC+lhsBPUFyB+y4XPfofdzmXNz/QRQFFLZJix9ZlOsrU9YxlZrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=K8riSR7WaBkT+h9mazx+moSRxeE0eWzN0eTXIGip8o4=;
 b=CZLJkZGrUrGUA7F0b4tN8oglObcB+ep+pphNA8ICHT7E5JbUGr+ZvolYTVaIVhqf2I+UcPMy2USGRGk1Dn1FjHTu1JWZfsVhgh8EJOeHApKz2wHdrEifCpacwL/prV8WbVQfDs7LdCJvCarA9sJLgQFkd10fnevHBO4EO/z9WAICRh/CJn+fcPrCJ/H+AuIa4pj9voTGTSfw47dlKj0apkkeO90nYq9sDugLPdTttRGfjyFmh6cvPKxUXchmtMesoTXLkD4z+w71Pt4lhkWTZdnvuk691q0EDa2VqivS4dtszccgSmoVqrs40b3k0JjGuVkCsmdpX51IaiZHrt9pUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 1/6] x86/PVH: improve Dom0 memory size calculation
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <4684f314-2c4e-f662-bd4e-132be001da9e@suse.com>
Date: Wed, 29 Sep 2021 15:13:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0094.eurprd04.prod.outlook.com
 (2603:10a6:20b:31e::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2aa6dc84-7f8a-4ca5-997c-08d9834aebe9
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435289E55FE18BC8715FF693B3A99@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3044;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EVJM8OMEARLRE00bSES6mAfvXvn1lYwUhwYQwIsvfhBLyL25Gr/TjnpAW+XqrS+2im+co2XJZG4POspn9Ly5RDzf6cXh8KJd354nhQO1W+zemsAebmkjRWaGf2+W2Qc8wyKw/024+XSbZWQJ+hkE6nCn+53bCkHhFN3DSPkdl862mGUxMG/fbZfL0zRHOiAtD+DBQ8g+7f3+FrKUr7em5j3S/wtwZXiRI7ues79YjA+hBrqGjDaE9+UMYwexzoNVpDGcYXjE73GWCc9MwbjvNql9H0gia+GioZw4agzdsdmaw1aIqLy5/FxubNjN5Mhbb2gx5f4qFRBzLdM/wEg+jjlA2CMw5H+oQqXee4sRGbmtV07ULDzvjh6mCy/yIgJZB4rPb7WEjj2OiY9QigdluOjTGObk4wGC5+Z4wC9ooquBTE8X6GBKC8hC5XeW4sWEyg7hivezQCywCal+gZ92IaHHXk7PmZta4Aeikee/WgIGtSYrBRyObiCOSqUFgOHbOpM4Dxc1K7A6hLQLlcyUPfyELgQRSFRKvkML1nMwTZ0alJw9JeUClnPly8H+Q7wzGZSayDPIlN5XP037W30H7p3bUO/U8XT9mC5SPGQCV2n+jmFvToY8oiAZIK5H3Xb12gnlfp9kG7LZmuqzzXqYMJYGAQghhIICpK3/RBl2NY0dfRNEoe7/J8WZgBfmDRZh5AMkaiX0fr7BBO3GKx1oaqMv1l3DehjxtE0DXJhGGoxMYVfOkVpLv+Fk4y3bJ1pTOU/vuTy/YzXgoM225LEDMA==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(83380400001)(6916009)(5660300002)(38100700002)(54906003)(8676002)(956004)(8936002)(186003)(26005)(31696002)(316002)(86362001)(66946007)(31686004)(4326008)(2906002)(36756003)(16576012)(66476007)(66556008)(508600001)(6486002)(25903002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bHJ2S3VITXVZcG9DNnhVTkNnaE9Ka2p1VjB2cy96d0h5dldDb1F1TVg3Z1Z2?=
 =?utf-8?B?NDlCT2JYMmVPS3lPMCtQdHVQYWhtWG1WdHBXSEw4NXUvRjZkaDZTR3RPcWp5?=
 =?utf-8?B?czhramhTaUdNZi9zOWxER3Rxc2xVTE9DREpsRXRNRDdVY3dGRkJmVlpHbmhz?=
 =?utf-8?B?QmJ4N2FrUnFxcit1a04yUTdnOXVxeWtJdURCNncvYzcweklEa2l0VjZMQ2ZM?=
 =?utf-8?B?ekRaYmRFcHN4bllVRllpMG5VNTV3Wjc5NjdpVFdFbEFmeW5jNlNMR3FjRE5n?=
 =?utf-8?B?cmtjcDZGc09IQlpwcTFJZGI5Q1Y4cnhQNXN5SmZKUXBwSFZDU2FubDlGVDhT?=
 =?utf-8?B?bGNscDJlaFdEdEhPdlpkNmxlZlJzZXA2M050WGVxdFI1VGZpZExjMlBoNTYw?=
 =?utf-8?B?YmR1VWFqSmdWRG4ranQ0bDc2eGs5WHIwZ0gzNDVrTWY5TlFWbkpYbzNzeU1X?=
 =?utf-8?B?VitOZVFwWkxDSEdwa29iQUMzSE5wdkFHMjc0eVM4clR1a1c0dmRpU2hOZ2p5?=
 =?utf-8?B?alVWKzUvY0w0dWM4dkNZZTZQYm5SbnZNeUdwNlAzV1UrbTJJL2cwNXpQdnlz?=
 =?utf-8?B?TEt1SU55c0JLLzVVTXJ4RzNvYWNrQUNaSWVINUJXRyt2T0g3TXAyMHI1Ly9G?=
 =?utf-8?B?NVVLQVV2MG1XWDB4Sm0za3duQWVnODRWdmNNTlh5c0xVUmpKNitzaWZ0UmdC?=
 =?utf-8?B?RmNpZnRBOHZBVENJRHR1U1crOHBzc3FEaWh4YmVzc3NUUE1EUDNJOWpLZVV6?=
 =?utf-8?B?UDd5a04yaklZSHFNYlYwRkZMWEI2cjRmOWNjUkRHV3F6UHEvMHZSV2ZEeFF3?=
 =?utf-8?B?UlFMTW8zYmRNMitHcWFjd0hrTEtDbHYvZTk4disxSnZwY29zOTZtQlAzVmJU?=
 =?utf-8?B?dnZhd2F4WXJvckl4T2RrWlZhczFHV1EvdTB5Z2tuckNpRGorOXAxektUeTg0?=
 =?utf-8?B?K20vVXhHbGNVTEF0REIwSFlRa3ZPNjlCdzYzYzU3MjcvU05YVFNOWWN5NEU2?=
 =?utf-8?B?dCtFTS9HWFEzSUZIR0l4M1FwbGs1RlpJRjZXcS8ydytRdElXbGI2dS9GZkVF?=
 =?utf-8?B?dkN3ZnlkVXZXVUFBRjFrWGlyVUMwdDFBTnMrWmZMUHRnclA3S2IxQjdEM0JH?=
 =?utf-8?B?eFVGZ1BuUHJtRFdsMDJ0QWxXT3lXdlhnNWhpc0p1QTJlNGNXWWxEL01Cd3da?=
 =?utf-8?B?Ni9wSlRhdVdud0F3QkVrSjdkN1VDM0EveUp5VWRiUmsyRks0UFhsTkgyVE4y?=
 =?utf-8?B?eGJMak9KalJzUG81cjR5VXhOQVdOQTlXcjFsOFpMVTZkZXgzM3YwWFJER1BT?=
 =?utf-8?B?T3p1cnhkS3gya2w4enkvUThwcGZBS0N2V2t1dzJENXkwZ3RvM1FNbzVpc2k2?=
 =?utf-8?B?WmU2bk9WUlR1ZmxJemszYkJ2UzF3MzdwRFpQQ3UydGZxL0FqRUdYRGwzcW9v?=
 =?utf-8?B?bUFZck5Fd1ViT05Kanh2NW85eXJDdjQ1VS92NjI4Q0tiS3dzQWdFeGJkVlBU?=
 =?utf-8?B?b21zbHErWEhycU9qaDhCQUFQLzFJbHBkZSs0MTFtVEhqQVhvWW9LN3hXVk9T?=
 =?utf-8?B?TFI4MTVMWmZ1ZU1qajFnR01CMHRCWVRDcDBGV2ptSnJHQkFUK2piU1hEZElF?=
 =?utf-8?B?RXlGcUFyQjlHcHBwVlFqNWtwenMwYzhHNXNPek80TzdZZWFZRTZtcUgyVUV3?=
 =?utf-8?B?SmNQZlIwd2c5VVVWZ0FRWUt5VTBqR1NveGFFbnFodXA2VWpBbVgzSjJLVzZH?=
 =?utf-8?Q?KI0MoQkEIw9I61SQRtEJxMuJILiTa8S+UT+MNpS?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2aa6dc84-7f8a-4ca5-997c-08d9834aebe9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:13:25.8999
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: m6ZTRnhIRCxP4aRn5naJvXYlmpR7WVURBaOasgnJGSiQQeFG5JQEmRINkP0x+hIUh0FFz/SZ/BAGl1PW2OxqqQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

Assuming that the accounting for IOMMU page tables will also take care
of the P2M needs was wrong: dom0_paging_pages() can determine a far
higher value, high enough for the system to run out of memory while
setting up Dom0. Hence in the case of shared page tables the larger of
the two values needs to be used (without shared page tables the sum of
both continues to be applicable).

To not further complicate the logic, eliminate the up-to-2-iteration
loop in favor of doing a few calculations twice (before and after
calling dom0_paging_pages()). While this will lead to slightly too high
a value in "cpu_pages", it is deemed better to account a few too many
than a few too little.

Also uniformly use paging_mode_enabled(), not is_hvm_domain().

While there also account for two further aspects in the PV case: With
"iommu=dom0-passthrough" no IOMMU page tables would get allocated, so
none need accounting for. And if shadow mode is to be enabled, setting
aside a suitable amount for the P2M pool to get populated is also
necessary (i.e. similar to the non-shared-page-tables case of PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I wonder whether this isn't enough to drop the "PVH dom0 without
dom0_mem" warning.

--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -318,8 +318,7 @@ unsigned long __init dom0_compute_nr_pag
     struct domain *d, struct elf_dom_parms *parms, unsigned long initrd_len)
 {
     nodeid_t node;
-    unsigned long avail = 0, nr_pages, min_pages, max_pages;
-    bool need_paging;
+    unsigned long avail = 0, nr_pages, min_pages, max_pages, iommu_pages = 0;
 
     /* The ordering of operands is to work around a clang5 issue. */
     if ( CONFIG_DOM0_MEM[0] && !dom0_mem_set )
@@ -337,53 +336,65 @@ unsigned long __init dom0_compute_nr_pag
         avail -= d->max_vcpus - 1;
 
     /* Reserve memory for iommu_dom0_init() (rough estimate). */
-    if ( is_iommu_enabled(d) )
+    if ( is_iommu_enabled(d) && !iommu_hwdom_passthrough )
     {
         unsigned int s;
 
         for ( s = 9; s < BITS_PER_LONG; s += 9 )
-            avail -= max_pdx >> s;
+            iommu_pages += max_pdx >> s;
+
+        avail -= iommu_pages;
+    }
+
+    nr_pages = get_memsize(&dom0_size, avail);
+
+    /*
+     * If allocation isn't specified, reserve 1/16th of available memory for
+     * things like DMA buffers. This reservation is clamped to a maximum of
+     * 128MB.
+     */
+    if ( !nr_pages )
+    {
+        nr_pages = avail - (pv_shim ? pv_shim_mem(avail)
+                            : min(avail / 16, 128UL << (20 - PAGE_SHIFT)));
+        if ( paging_mode_enabled(d) )
+            /*
+             * Temporary workaround message until internal (paging) memory
+             * accounting required to build a pvh dom0 is improved.
+             */
+            printk("WARNING: PVH dom0 without dom0_mem set is still unstable. "
+                   "If you get crashes during boot, try adding a dom0_mem parameter\n");
     }
 
-    need_paging = is_hvm_domain(d) &&
-        (!iommu_use_hap_pt(d) || !paging_mode_hap(d));
-    for ( ; ; need_paging = false )
+    if ( paging_mode_enabled(d) || opt_dom0_shadow )
     {
-        nr_pages = get_memsize(&dom0_size, avail);
-        min_pages = get_memsize(&dom0_min_size, avail);
-        max_pages = get_memsize(&dom0_max_size, avail);
+        unsigned long cpu_pages;
 
         /*
-         * If allocation isn't specified, reserve 1/16th of available memory
-         * for things like DMA buffers. This reservation is clamped to a
-         * maximum of 128MB.
+         * Clamp according to min/max limits and available memory
+         * (preliminary).
          */
-        if ( !nr_pages )
-        {
-            nr_pages = avail - (pv_shim ? pv_shim_mem(avail)
-                                 : min(avail / 16, 128UL << (20 - PAGE_SHIFT)));
-            if ( is_hvm_domain(d) && !need_paging )
-                /*
-                 * Temporary workaround message until internal (paging) memory
-                 * accounting required to build a pvh dom0 is improved.
-                 */
-                printk("WARNING: PVH dom0 without dom0_mem set is still unstable. "
-                       "If you get crashes during boot, try adding a dom0_mem parameter\n");
-        }
-
-
-        /* Clamp according to min/max limits and available memory. */
-        nr_pages = max(nr_pages, min_pages);
-        nr_pages = min(nr_pages, max_pages);
+        nr_pages = max(nr_pages, get_memsize(&dom0_min_size, avail));
+        nr_pages = min(nr_pages, get_memsize(&dom0_max_size, avail));
         nr_pages = min(nr_pages, avail);
 
-        if ( !need_paging )
-            break;
+        cpu_pages = dom0_paging_pages(d, nr_pages);
 
-        /* Reserve memory for shadow or HAP. */
-        avail -= dom0_paging_pages(d, nr_pages);
+        if ( !iommu_use_hap_pt(d) )
+            avail -= cpu_pages;
+        else if ( cpu_pages > iommu_pages )
+            avail -= cpu_pages - iommu_pages;
     }
 
+    nr_pages = get_memsize(&dom0_size, avail);
+    min_pages = get_memsize(&dom0_min_size, avail);
+    max_pages = get_memsize(&dom0_max_size, avail);
+
+    /* Clamp according to min/max limits and available memory (final). */
+    nr_pages = max(nr_pages, min_pages);
+    nr_pages = min(nr_pages, max_pages);
+    nr_pages = min(nr_pages, avail);
+
     if ( is_pv_domain(d) &&
          (parms->p2m_base == UNSET_ADDR) && !memsize_gt_zero(&dom0_size) &&
          (!memsize_gt_zero(&dom0_min_size) || (nr_pages > min_pages)) )



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:13:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:13:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199073.352905 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZPF-0000mU-Ph; Wed, 29 Sep 2021 13:13:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199073.352905; Wed, 29 Sep 2021 13:13:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZPF-0000mN-Mc; Wed, 29 Sep 2021 13:13:53 +0000
Received: by outflank-mailman (input) for mailman id 199073;
 Wed, 29 Sep 2021 13:13:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZPF-0000mA-06
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:13:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 169cc90c-2127-11ec-bcfe-12813bfff9fa;
 Wed, 29 Sep 2021 13:13:51 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-21-WOIg97fLNrqFzSoAqQtHEw-1; Wed, 29 Sep 2021 15:13:49 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 13:13:48 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:13:48 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0109.eurprd04.prod.outlook.com (2603:10a6:20b:31e::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Wed, 29 Sep 2021 13:13:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 169cc90c-2127-11ec-bcfe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921230;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=tZu5l7MohBWThL4D4aCcbxZpD036n9ZohT52VmFvYFE=;
	b=e2xohw6Zc32sgvCWh0kK1RK4w1I0TJKfeSrtfRwkUoMsd4Deurwr6CvPVoEULS0oz9Mx6R
	nTAF8uGW5FIfeET0yxPuDwHzhdjl70ITjge6aNkb4bcdlnQLJZT6AVFQHIrgQUKkR1np7C
	9A5qT+ObAzMVnJQ3zHhWcH457bLmssE=
X-MC-Unique: WOIg97fLNrqFzSoAqQtHEw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=l+Rikq+LTX/UwQp7WaEpQONCOFK00ku5+GfhXga7RcKwwrbnFDl79dxF8Ac1GGQfZxApemiooJBq6xvV3f6kIBPBWwItAEWsreFHdigKryfN3QsONtlsmKEUX2Mu0g0PUN4Leq1ey0xzH+NQxVcCukOUCUaqbGQYriSHaQd6HPsYkKfhGXyUuwFAZ+KwK6poovjP4W9YB8HAoDzCP1ChRJHfGq2KEsS0hZQcLcc6xvH0sfcTAVgjVFcGjsvh/Z9iOwMmya+YMamfBUexypOU8YkJNACiMEmJNHsXNDzruBEqaivIjPscq7+WmL5IShU/q8z2pooPwAzHcFL5EhO2xw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=w1MgNXQPIZRI2m7FHe+SKi50wAOF1utIT72Ti22BWx0=;
 b=Eg24EybU/QF+yhw6CmwU4Bhw/3Twfq8tXgD+7kueMKjGCKQZbfpyYe49pC6rU1MyIt/KHTgnCU/caw4cPnpVkbudqtZ+1E+vXwhc1VaX8uj56AG82C6FHFn7vkDA1EXIu4Wx2RmsS56+OZOWxxZaoA+BBGdGDxZ0H3nGNcwEjcbT84rbiQKZgSeqxR3y8lOandvhiiDbSRl/KAWnjKvePB7PX6OHFFeDzrPxwNFlGDDzw0qI/g34FTKgLxBIbnOy17PvGmz70m3hVHYvp0FfnFKeTj10DdxE3rFOtjoDMoB0tL5L8Tvesjq203gubCTuY2i809kPglRr2PvF5YJiGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 2/6] x86/PV: properly set shadow allocation for Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Tim Deegan <tim@xen.org>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <803917ef-2bd2-0223-8bf8-23d129baf277@suse.com>
Date: Wed, 29 Sep 2021 15:13:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS8PR04CA0109.eurprd04.prod.outlook.com
 (2603:10a6:20b:31e::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f8351a5e-cdb8-4dfc-cd92-08d9834af985
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB435240EA1B7503B0E87B2C8EB3A99@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZFT817FByKbhEIlq1RcaVsOLMYzAg0sqRhFpzHfMhIpiBWgCn6ZBxnpBPxX075otoV+peuGYE8sspYV4hYj/fvACK/mlTyJGdzXJHgc+6vbvj3h22SGocC8SJIDgNhmFi0i/mVMdWQ9+ii+Caqtsx7ceVFyMfp8I4HonC3BPgbbLDzoIo5hptX312nTI85iMXtwhkfzdjoP6KWk17eDiLMwd9p7+Y4ND0k9cjx5WAQmPHg6FzI32qKXaVRFTDMky1Xpi4v+F3UbxEdNMzZ9Lf5mADbClzxE+p9LOEhloih/0E26PnAp2A9gpR0Tl28gFF9l+LiuMNtxSeqZMAWRH1/CrKdfcWnYqb9NWaLKXNwXarRpISvue/suLFG01dRARlI+Ga777fC24QxXgv4n8/I6c/SNPRrZBKrQSiMogERYZQ5A2xUjMaJBQDKm+E8vSGQA1tzODTculvBdNmS/MHWniz8Nd3hKxj8W4DCz/6OXKb9v3cb4gFE3qBxsWVs8/GxxBSF4f0YO3LylxySYCQNGRvHPW1BT3us4YJFHumuXSm8Vo+3ikWd9+Ki/9vjMKV6vjGqdCG6e8bxRP682XZyUySFIT8Z0tTlRa6vjioNvU5oXiQ9dv3Zaa3S5Wxb2fo1DsX7m/F76HFbp3M7PS+ooc++fzvzqsPQZXxs28dIZtQ9CA0/NpOjpq0nQhoJf6aeu9jv4bKaimLtw1U10Mf68uEgJFd7MEKJLjm5oYbVU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(83380400001)(6916009)(5660300002)(38100700002)(54906003)(8676002)(956004)(8936002)(186003)(26005)(31696002)(316002)(86362001)(66946007)(31686004)(4326008)(2906002)(36756003)(16576012)(66476007)(66556008)(508600001)(6486002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?fg7GhRs2ZXn+VCjAw9y+GOkRxFxywyhwnSgJZ7GFqnlS6gjDrxd/hrEPI4Xs?=
 =?us-ascii?Q?xIN7HOKxZj4JBeO0AE64v7LvF38d+qxNxWtKyewMDALIMBVnawiwGe3qi0be?=
 =?us-ascii?Q?XLvAYD3W0jYlNKt+M9mR3AXbJfP6JvSR+Bi1AsmStFUvrWcqHawoW8r7VdZj?=
 =?us-ascii?Q?L22cBx+zDjfl/qk64KlACs+44gdZ63/O5mhUE1si1ZE2cxTv88W97sMOZoii?=
 =?us-ascii?Q?rlj+GXT/70859p+zmXnut8Fs9ooubwohWEEtfQpFeFh/GCIpPq9+iSmBHEbp?=
 =?us-ascii?Q?tXcy5WBhKMA5LJPfB4iW/7yCyLet4/M/5sF6QFUVBIVs//0q9MMzXml4MjjN?=
 =?us-ascii?Q?DWLwgd7B/7otkdTpN4xiFj041uGudixDqjOgFFwXJICFPEr1SdWdtgE49R+J?=
 =?us-ascii?Q?35sUU7lJuP1Uqa3VaBsMEEsdXKEQIFk95Rdpg+HHfMLo/dMRkC3DtPu36/pA?=
 =?us-ascii?Q?1ovUvhOpO1DVxVuj19tx8rpSPYdcL/Z3h8W9XWpq0/mdi69xXabIVlrGARzH?=
 =?us-ascii?Q?/DLGetotFHpV/ycJv9XFmlEoaOsTexYohC3cyy/mZ8LvPcd22zShOs3ZgTYX?=
 =?us-ascii?Q?6eiNuSJxLArSzTJ3YsxjKWGjBpZngH1ByxhIBhdio2GG4ZGZsCImZYW6Caum?=
 =?us-ascii?Q?kvD8bEOKCcrL76qrJHZtsCJdV+WeH3NMYeCU6IL2gkvFGeEX9WFUk8FVTcFG?=
 =?us-ascii?Q?6tLYdu2sUPCv5KSk9iqEHlzw+w7FtOJ35wzjddMnP4q9xgkxmslG6KKC5DBs?=
 =?us-ascii?Q?8PRepUB1wneBNQemtakCJBlJWpS7al2zc16w5LxjtMJgkytocfyuJPFkiB74?=
 =?us-ascii?Q?NwGmWXcCkivEJfVVUxa1MfuXugjag5mUel3AfbxTuq3gFAg5Ny3u5Kumrm2X?=
 =?us-ascii?Q?/71DQ+I3NpPkJUGE2LFjf73lav+NfrxR4ZbbrJh8E16K//jUt1bjufvOVqmI?=
 =?us-ascii?Q?PFGanFl8tCw6UNG/QJXWhqAa9UbBt7sl0fSKTxA2EXzIdW5fcil0nFTnWrrc?=
 =?us-ascii?Q?Mlpr50F40LZyl9r/c0SRXgOyu4T0ptAB4++2+G3z5bBwNHk+lYRKwKCHfiJ9?=
 =?us-ascii?Q?wmGy9YQ/b9xilpSqr47mMp7oPTy+4jMnd0yQf/r7WCHGPPnsuO3HVEJ5zTwd?=
 =?us-ascii?Q?5qS5mKQ3N7Tcme4UfDVX+n7NYZLzPd/uvnzRdAnoEv/y6ltvhU7Ryl4L5arx?=
 =?us-ascii?Q?ns7hvRj6WmMIhVEWfb33hvelJCltIJlGj0cZ+Set/oKTNpxL/7rxrIeb5cMy?=
 =?us-ascii?Q?lHxaNnljGLr+VBO2841pKO46rlVtVtxiYz8F6c5aUZVc8HlMN3dMRoJIlPYn?=
 =?us-ascii?Q?QWgcUtn3VmRbvwuakOYee/q3?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f8351a5e-cdb8-4dfc-cd92-08d9834af985
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:13:48.7289
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mPTuO0/Kqn+RYdyiMuQHaPNYdf7MebcM5k3oCr+aB6i8LP5TT5ZpCSdbc49jX2D+b2DfZG7jRl2YdBqS5h40FA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

Leaving shadow setup just to the L1TF tasklet means running Dom0 on a
minimally acceptable shadow memory pool, rather than what normally
would be used (also, for example, for PVH). Populate the pool before
triggering the tasklet (or in preparation for L1TF checking logic to
trigger it), on a best effort basis (again like done for PVH).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
Reviewed-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
---
v4: Also fill pool when opt_pv_l1tf_hwdom is set.
v2: Latch dom0_paging_pages() result.

--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1298,7 +1298,7 @@ int shadow_set_allocation(struct domain
 {
     struct page_info *sp;
=20
-    ASSERT(paging_locked_by_me(d));
+    ASSERT(paging_locked_by_me(d) || system_state < SYS_STATE_active);
=20
     if ( pages > 0 )
     {
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -21,6 +21,7 @@
 #include <asm/page.h>
 #include <asm/pv/mm.h>
 #include <asm/setup.h>
+#include <asm/shadow.h>
=20
 /* Allow ring-3 access in long mode as guest cannot use ring 1 ... */
 #define BASE_PROT (_PAGE_PRESENT|_PAGE_RW|_PAGE_ACCESSED|_PAGE_USER)
@@ -928,8 +929,22 @@ int __init dom0_construct_pv(struct doma
     if ( d->domain_id =3D=3D hardware_domid )
         iommu_hwdom_init(d);
=20
-    /* Activate shadow mode, if requested.  Reuse the pv_l1tf tasklet. */
 #ifdef CONFIG_SHADOW_PAGING
+    /* Fill the shadow pool if necessary. */
+    if ( opt_dom0_shadow || opt_pv_l1tf_hwdom )
+    {
+        bool preempted;
+
+        nr_pt_pages =3D dom0_paging_pages(d, nr_pages);
+
+        do {
+            preempted =3D false;
+            shadow_set_allocation(d, nr_pt_pages, &preempted);
+            process_pending_softirqs();
+        } while ( preempted );
+    }
+
+    /* Activate shadow mode, if requested.  Reuse the pv_l1tf tasklet. */
     if ( opt_dom0_shadow )
     {
         printk("Switching dom0 to using shadow paging\n");



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:14:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:14:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199079.352916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZPg-0001Ne-2S; Wed, 29 Sep 2021 13:14:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199079.352916; Wed, 29 Sep 2021 13:14:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZPf-0001NX-V8; Wed, 29 Sep 2021 13:14:19 +0000
Received: by outflank-mailman (input) for mailman id 199079;
 Wed, 29 Sep 2021 13:14:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZPe-0001NJ-Up
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:14:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 6f65c171-facb-4f78-9ad8-bd66e0b63ff2;
 Wed, 29 Sep 2021 13:14:18 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2056.outbound.protection.outlook.com [104.47.14.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-5-1Hr4OE5JMYGecOgmdzLptQ-1;
 Wed, 29 Sep 2021 15:14:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4352.eurprd04.prod.outlook.com (2603:10a6:803:4a::25)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 13:14:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:14:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR01CA0070.eurprd01.prod.exchangelabs.com (2603:10a6:20b:e0::47) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:14:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f65c171-facb-4f78-9ad8-bd66e0b63ff2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921257;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Le5BJdNW9ne/Xmuzk0IFXev47snchE6r7a8FNfwFXaI=;
	b=jVzA3KkMCAgyEEzDyakJPzQvqXQDvaNxtdFT7cSpdHODA03ztpxdw23t+deBhPhNlQPgxO
	GPHZAzD2Df6M1r6uf6ueJ5Zn6H2LHyOeArrERaOLsw0lpZBIYL8toZ+uPyxkIRPK+OECTQ
	RGR64opFRVbow6ItAd+dJJlDuOq6VgM=
X-MC-Unique: 1Hr4OE5JMYGecOgmdzLptQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Pa70P8gc42nrhMzTduy6SDxlH5qd3UO/ngbw/+cAe3IVXwkSZ2qKjkip8D+/unc72MTUZfnmodjNlqdszToP9P4NN0d4DuF6r1mciUUBpO62X25urL4zcSLYpen2xxvu2SoFWAt/HZaikGyMldbuBDDVzEASmH6DHuZCR8lcfqN6Qr9nPJLgT+mDTsOqjeK6+gUhAcXNkanVUwwjVXbqHGtcePYgoBdrATgqkTmJVN2BkIuDS4EFYkVRo8D8I4caRxy/C4ytq/3Gxe+soqv+VqnnhD0jpQB0GYP/e+6ibDKgLNmpDu71b9ZXRkFRAigCJbMc4IPuaA0LjghRVomeyg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Le5BJdNW9ne/Xmuzk0IFXev47snchE6r7a8FNfwFXaI=;
 b=gQpLzv+oOu6U5g7uL1B8nU321iyQ6d7bKLQKBSnBGFRe69dOiMPKKKwi4FqGrNmM3vcACEo7M7u/64e4VzeTmebefSdw+bcUHPsopXr7z1XKXqF50RAF95DuKmCfGTPkc+UXtbvaNnnVUi+0ujaxQJlz1A7ge2dGkZXulVVutTon9YwfGIg9Nu5Z9PWx3eVoUspTzPPfHakX3zweGDg7fJasoouZP5KQpeE/LOYK5TLt4l5v8B1dXusbWSZZ2H0CXk9LhrfEcQ4sFOVEghgK8XwPNc1R3bGxjnDLetAJIG0vklz53oVGZirFHZbDbGdzT2X4M+ZtPGAu2q5MXZ9xAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 3/6] x86/PVH: permit more physdevop-s to be used by Dom0
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <f7023f65-5cb7-fd2b-f320-c46a8e6e1ac1@suse.com>
Date: Wed, 29 Sep 2021 15:14:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR01CA0070.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::47) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: cdfd7ed1-bf90-4928-51da-08d9834b097b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4352:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB4352577B15E7CF548F0EEF11B3A99@VI1PR04MB4352.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8fg1eFR7mqDHdR2QtkDYfmq3AFHlXbA4h2xrfxmREYz5CYm5BpsDEGj6Z01lot/bmnX58Y1l7fbDRCpP724EVqZPNjv6bw/gj5/2ssK1bW5qHGCzrv+dOVsmow+0a/HfvZvO+xktB182VoUzcvP+vHkhUHAxTWMPb6zrJEYXGEc7NAAR15TSpyEDl/W63sYpV5dWYfBQ1yvg7pyDL78AD8hcJ51Y4sjLJBAjsDMU7lID0OoILEE9CpiQ7Wqcy/izCbd9ikHyjrngSmb6+HjWmmuR1Ol5CYU2GnOcl/3rP337ZobrCQYoQUcykq2VRTh4by3fWoujhkMdRQLoq1cjDRPQvZNzS5FMGDM9Z0IX/+GC8KBd/eBV0ktvPrpcnOpJZ2tO8CXx49IuKpo+hyQFTNi8uMwJw6clrnbfHULY7uFprrjkJgJWKXk3vZKz5kTdFg6rpcbNYAfFRd6nqMJ/8tDGANyLIcdNAKx+KUvu2JjGJ/mt0XVz8dIQPM9+qByIsuJ7uKHivyjLLR4DohtvhiRL9jMl0l2juBgXxhEXhy0OAFKQkJn4ZljSw5Rmk1z4VAcVI8wN6+/eXAJ43TCIG5K9liqAS2+srX1jy8+vXx5NsQBW2He5XDJePxUNscGzWjjBy++/xcefpscxNEH6Kv+51/A/hCYDE4KSAS9hbgyCZIGXPi+kzyepmHHPv7nc+nGWruZghQq+aKqf8Q4W+A==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(83380400001)(6916009)(5660300002)(38100700002)(54906003)(8676002)(956004)(8936002)(186003)(26005)(31696002)(316002)(86362001)(66946007)(31686004)(4326008)(2906002)(36756003)(16576012)(66476007)(66556008)(508600001)(6486002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bkdmVmR3c1lvMTl2bkVUSDE4c2w3RThUSnZSUGJ2a0tHWXZxM3VScVhSQU1N?=
 =?utf-8?B?dzZOSFAzZm9FVzVza1lzQ0FveUhKaDNOSDFrVGprTHRjRUFNZU5YUnRiWVF3?=
 =?utf-8?B?RjJ1am5UWElmYnR4dkxEcDVyUC9kd09ZdjBXWm1RQkVXUjMzTHU0bk5sVUlJ?=
 =?utf-8?B?MTJJaWZ2dGc2cGJGdTJrVHp4MVpYaDlBK0QvZlBCeVNvYjJTT3J2cDNWZTFt?=
 =?utf-8?B?QzhNN3J3Kyt1bzdRUlBSYmZYbVAzU1dDMG11MEt5MEwwaG5ONU44QVlMWWNl?=
 =?utf-8?B?Lytrbks3bStHSXBzb1BLN3o4OFhmcGpPb25zWVVyVzdiTFcyOGdMSzIzbHpj?=
 =?utf-8?B?Mjc1U01RVW1UM1FLVzRwNEFudnVNMFQ4YkhVMm42aEFRZ3ZHbkV2WUM2c25B?=
 =?utf-8?B?UkxkN29GaXlDS3gwWUNPbUNWMW9jSlBzckM3UjFLMXpvbXBmMXdaaW1SQVd1?=
 =?utf-8?B?WmxiOFVKM1VoUGxUZ2Zua3hKeWpBclpnODlGaVhkRHZYdnlFL3FwbGNlcE1U?=
 =?utf-8?B?MmFrdlIzclEzRWtLU3pUT1BmU1NDSUlWK09tWGlvbnFSN3p1K01NYysxNU9n?=
 =?utf-8?B?ck5pSG5rMkswdGZrTmZrRXZjWkx1WE1pdnhDNDBnWmhZbkJJMUdxTnY0RGpk?=
 =?utf-8?B?TUZmRnVyb2xKTVlXWU50am1ETHVwMzdjdzFVeWVEZXI0MzF3OVhZMWM3VTIw?=
 =?utf-8?B?WjN2eCs5Y2R1WVM3NGduZjk1SnRQSUt0MUczaUtHeVFFU2drN0xpZys2dU0v?=
 =?utf-8?B?OGFGSTBSSEptQVFFWmdlNFpTRzRMMmtJMk1sbG84NDFxcUdmTElSY21ZQ0ti?=
 =?utf-8?B?Q055cUhJZjlXaXdKb043djl4RzZ3YVVpVmZ6dTZVZGZXeEtmTUFHeTRhSEh4?=
 =?utf-8?B?c2Q3Q3NLendWQ3JJV0MwTHc5dzBIWXJ1bEpJdU4xbnRSZ05xVnFKRmszS1Ev?=
 =?utf-8?B?OHI1b3VYbGNtUXh3TlB3OVBNa0h4eTdkRHZBS2JHNzA4WTBSN1gxd29ydTVy?=
 =?utf-8?B?OUljSnFzVlU5L3NhWmQwYmtJanR4WTdNTXpTaC9NREkzUDhWeWlIM0tGSTRw?=
 =?utf-8?B?NVpCem5yS3kwYVlMVUJ3T3pUeER0bEZWUXFSTWtlNGZQdmkvWmU1V2N5Z0Jh?=
 =?utf-8?B?dVFUWkRFM0RVcEpIbWUrV3k2NVJlSlhkNVEvWVJkZktFSWFOMUFKZm9mUDJU?=
 =?utf-8?B?dzc5Q0p1T0xzUDczSm9Oa2N1SFBKWHNzd0EzUE1hMzd4bVptcytZekorM05n?=
 =?utf-8?B?RHo3OWJscCt6QjdENHdXdU12OTRDdlV4dGpNcGppVWc5VUZId2JBZmtoVDBk?=
 =?utf-8?B?R2NkNnRJMjJLczdRaVA5cWxWYXBnRnhHeGdnNnBzaVFSNDBieFFQalRhN1Y1?=
 =?utf-8?B?NVd3VDZsbEc1UWRaMXdmY3pZeGpuNDhaRlhkY25BSk93UlNqU2ZBYnovR0J6?=
 =?utf-8?B?QnBvN2FONXA2K1dCOGxMbDgrRGNic2ZLTnBaQlIwdTV5OVZ2dlF2VUE2dVg1?=
 =?utf-8?B?dDlqVWx4L2dveWN4Sk5Jc2hyNlR3eXdCb1pMR3FlR3RvYWU0VEVvcWRPZFI1?=
 =?utf-8?B?OXFtMnhDb29jM2ExS2Q5Y1ZNRzhzWnhPaTNUc3FFNHhxOHlybHVOaGxjdjVt?=
 =?utf-8?B?czhwQ0JDNk0vUjRva3pJVmw5NUVJR3luaTFPNHVQSVA5eFJYQUtwT2ttTEpF?=
 =?utf-8?B?bGZBV1BsNjlCNWx5VjJTOE9peUxJNFgrdW1Ta0dEMnRNNXNyT2E4TkUrWVZL?=
 =?utf-8?Q?8XeYtbEdrJgsJ1FIf0hLlo2AI+C8j5mZUZfqJT1?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cdfd7ed1-bf90-4928-51da-08d9834b097b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:14:15.5447
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UR8z8fFJToAsruZwi84K/Tl1R3UgHsesdCpTcyKMP29xs+w5YTbNEjuHpuinF5VvSp0S+xAOR3Xug+wBYhN61A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4352

Certain notifications of Dom0 to Xen are independent of the mode Dom0 is
running in. Permit further PCI related ones (only their modern forms).
Also include the USB2 debug port operation at this occasion.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I'm uncertain about the has_vpci() part of the check: I would think
is_hardware_domain() is both sufficient and concise. Without vPCI a PVH
Dom0 won't see any PCI devices in the first place (and hence would
effectively be non-functioning). Dropping this would in particular make
PHYSDEVOP_dbgp_op better fit in the mix.
---
v3: New.

--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -94,6 +94,12 @@ static long hvm_physdev_op(int cmd, XEN_
         break;
 
     case PHYSDEVOP_pci_mmcfg_reserved:
+    case PHYSDEVOP_pci_device_add:
+    case PHYSDEVOP_pci_device_remove:
+    case PHYSDEVOP_restore_msi_ext:
+    case PHYSDEVOP_dbgp_op:
+    case PHYSDEVOP_prepare_msix:
+    case PHYSDEVOP_release_msix:
         if ( !has_vpci(currd) || !is_hardware_domain(currd) )
             return -ENOSYS;
         break;



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:14:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:14:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199086.352927 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZQF-000241-GH; Wed, 29 Sep 2021 13:14:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199086.352927; Wed, 29 Sep 2021 13:14:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZQF-00023u-DG; Wed, 29 Sep 2021 13:14:55 +0000
Received: by outflank-mailman (input) for mailman id 199086;
 Wed, 29 Sep 2021 13:14:53 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZQD-00023a-LH
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:14:53 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3b126ad0-2127-11ec-bcfe-12813bfff9fa;
 Wed, 29 Sep 2021 13:14:52 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2054.outbound.protection.outlook.com [104.47.10.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-Q5pSGNuuPzuE5k3J6YPxDg-1; Wed, 29 Sep 2021 15:14:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:14:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:14:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR2P281CA0028.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::15) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Wed, 29 Sep 2021 13:14:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3b126ad0-2127-11ec-bcfe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921291;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=eW/2cF3lu6QYOHTIXUs9q6QXeBV+Luo/7Lua57wsjCU=;
	b=RFwuWuizdH85sv3rGnCKzKjZkR86mObT8tviw5uiDjaaDcong++gAcrJsWlHTamv88v+Lx
	gqM81AMiWlJypjllbhTxlSp2H7JUpLsctOqvAMg91JXEEFQUAeEtb1VXLSzqpHqi2eyvc9
	WuN6To0h1VKxVs+yfUZkmpDPyESMW80=
X-MC-Unique: Q5pSGNuuPzuE5k3J6YPxDg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hO+bq2XZYZuU4NVXmDz0OiVOj56/iXgRQE3QFAH9IUJ4BDp2oQdTVaSlvUFnryPn6XwHnxWLwY3RwSDIO/dLUzHnYqBqLvdpngf3SyL6J+nnxMokpJtupMEq3gwZD4M/YMbQ7QaTDXTUfBsKgpLfHWpkkH+Yz2q41fZyhXtEbvC8tlAs6mooBg2+uQ4rTOkilYAbqDnjzptx+5sU3tSxD20Fk0KqpML6jZn1Tq1Fxa5KeCj91XCr7POeNL54/m+c4OQS3R11hlTARF7ayz9W5ffJwTfL9Q/Wny7JYhNvt9so0yXL8gE9OV8oZnwytV7/G24zR1Th5+KHnSmD/1wQPg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eW/2cF3lu6QYOHTIXUs9q6QXeBV+Luo/7Lua57wsjCU=;
 b=LE7xNH0JaXtIoBkxN4JgLUV3+TfN1ISvfg2NbNU5E2EfMho/eapi/8XljV0II4pY9mZuYP3AnwOJCr8UlmrGT0WnDeg2pRJ82xRbvp7VE7hDGU3jtKqvcUhZRhjMHvxWjV4bFX+6jYojxXfgDLJsMISVVRxrMOdnYL+V/S80gGc4jpcD8Aj5l07NRe4/O7tR81OOaZHPj8vcGyNVPoX/d3x+mta+yON4UomsXNVlK8ppR3OoRnZZftst74PA3Ag9laWtuPPf8UsrN26tbx8LNYh3YGmelsUSKCIrO9N3pu/d6Sr0evgRYTgx8/T8ldoe0tfjuy/rbcbpoFgxbeKiDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 4/6] x86/HVM: also dump stacks from show_execution_state()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <db023918-618a-d19f-e600-ee0a683a01af@suse.com>
Date: Wed, 29 Sep 2021 15:14:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR2P281CA0028.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:14::15) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0bb73bed-6c89-412a-2339-08d9834b1cb7
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71505054E009C1ED4777752EB3A99@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3ZNlQTUaLyH8/6MqA5ptf9H4lJCoeMLgR5EWxt0q0aFA+aI+CDmJQHMvPdFA0JEz6CX9kcdipFCbplFOoloHtpDw9l6QEioW4DVdTnSGZWfcg+8vQqJMz57YiRIOL7+V58PNl/hCfXfAV7GxL55hu4PyjIryd/PHEA6ioNrpvqM/5Qq3bWx4VCvhI+0dmaZRg0tESvchrKqA23JhhtIvB8oFW469qp+iXUWFvlBtNWbLGRI33V7FXsECWeqC1SNfEz1NmnKXZBqYE/vTVMJY6wtv/ZJ5wA6NcneTGiFCxLhQ25cZyE4QMMrhCXpQ0YMJW7Dgj9Ew+9fYTBrffCUFBL4RwK616H4hGtqvyR49aZpNA5YpKlZRWhQhW0HC+l94ifd5VcsIBHyW2UwMcZuJK+3okURHcBNYs+Lid1wn8gJSr50iRXdX+PXqmGWFeiBd2ijxVwZ53hSGdjr04ltQ5k2wFkzFmEgtyxl9pPhvk71hoWWyro9B3lpP/7Z84rU3UQkTUX5/5BE+FE382ZQTUDA5M9qvksfIIfOrwjYA984Jl0i8zCR+iWxIfGSvjcyCS3aOuwoEXgpx9GFkKLH2cR5qVc0mTbd558FSp6x0KDEdjIUcluU2AtykoOi4e3cXxnqlf1E9SqREsKX4IEJfJMxfB2mlrOwr1lUFCeSwtvHrjRc8fvaUzp7eXIXEhZ4E4O9N82Yzsz4ikNAVb+LERjAXR7ghe+LJvepiOqhBKsk=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(66556008)(186003)(2906002)(16576012)(316002)(5660300002)(26005)(66946007)(8936002)(31686004)(83380400001)(8676002)(54906003)(66476007)(6486002)(86362001)(508600001)(36756003)(6916009)(956004)(4326008)(31696002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YmowbVNNNU5weklQZGhPMW1LSUwwbmZsb0VhMG1uaVVHajZJNlRVTis2WTJI?=
 =?utf-8?B?SVVPS3JSb3lVME5DUWJWYzVKdDBDVFJnV0JDNnFTWjlOMnRCTDNGSmxqYkVH?=
 =?utf-8?B?Njh3MGZ4bWExWEcyNnhCYmJpQnNtRHgwVHRHQ3psa3kyNERBa25MTnd2elpx?=
 =?utf-8?B?OGl0dDBKMm5Ebis0TnFvcVVVQTFSeFRBMVVsS0c0dkxTVkp0dVlGMDNzSWJs?=
 =?utf-8?B?eHl6dk9vbXpzSVF1L1BNNHFFbnZYTERSTzk1QUpnckpZQzJoazZFNlFUMHRY?=
 =?utf-8?B?YlB5OUI0RlBCWDFlMXNIcWtXcTdYUFlIcDd2bTQzbjdYVWIxNDhnQndPK1hn?=
 =?utf-8?B?V2FQdHNhSjV1VEpkdTAydUdoV3JObWIwUWovdFBjRXNtS3pxUHA1MWgzbG00?=
 =?utf-8?B?KzJpM3Fia2NjajU2MmR3bTZmdkdyTGZpSnEra1l5L0tNTm4vZmQzOXZqREov?=
 =?utf-8?B?T2FpWURVQlVJaEFNOHdNR0g3MW83bzFFS2lxZVJ4QVljN3dSOVBDbUJyQm1C?=
 =?utf-8?B?RUxHRXcybm1wQTVwSU5KZzlXRnNSaWd2WFRQb2wrRCtOdFhqWGJaWXZJcXpr?=
 =?utf-8?B?ZFlGdmkrSkFvQnc1bytuMlhGeVAzRXB0ZEFvYnBqQzdFOHIvY3o5NmhrQVlv?=
 =?utf-8?B?cTJQK2N3bkhyZ0xQNmE2N1UyeUd5SUtuajZVVGtQOEZQNEJKUkk0Tm9DVnYv?=
 =?utf-8?B?L2duSFpuenBjMlpuamZWVGtaeUNtOWFVTDR3YW9CelBVeW1ldFh4QkFxa3RT?=
 =?utf-8?B?NUZVbFBUOHlrTW9MVThtS09lS2xaN1d5TUlzek9hbHNvUjF6NkhiYkY0Z0Y5?=
 =?utf-8?B?Yk9Mc3hrLzkydTVxdTZoZk81WE5PNnlTbDd4VFEyVkNxVVp4bHdtQ1l0MFBl?=
 =?utf-8?B?dkoySGVtanR4UHpEQXdWQW9aR1ZLdXptNndrRFNFS2w4OHRudWFjczFPRkdO?=
 =?utf-8?B?NGhnYW5uM0xpTFlORGMzOU52T25ENmRZdUpJYUcvbFNZRlM1dTZ4VWF2OGh3?=
 =?utf-8?B?dXBkSkFERlpiV09QQ091R05ZbjJ0RTkvTXhSQUdpVU5aQkh6RHFUZTIwK3pu?=
 =?utf-8?B?eTJZSW13bEFpdnlYdHQxeTJjWlRBL2FHS2NUY3k2V1R3bkdBYXExNjdIMCtv?=
 =?utf-8?B?SzdPaFNjc2hsT0o4cEVyVEk2cGxxZS9oV3VRR3BYVHMwQUdocDBoQVVOMXdx?=
 =?utf-8?B?K3k0TXgrcER6cnFEb216UFBvdVNoK0c3R29lYzF0dmEycGJYbmo3dWErS1Uy?=
 =?utf-8?B?QjZlSm1aL0czbzdTOGh5NzBGaFZPY2dtZjNFYTg4R2p6bnJBRDVrczhoOFhJ?=
 =?utf-8?B?SzBXdjFQR1VuVDhLZmUwVldhYjZ6bkQyRjFPQ1dYOGNjM3duS0Rra2Fiejht?=
 =?utf-8?B?dWs0bUMvWlZvQy9kR3ZDMVRMYytoVFFwYlBTQ001OSswWGhCRzZ5RDNucE4v?=
 =?utf-8?B?NE5lM25obFFJNjVUWGtmZ1h5K2ZrSmJSOWkxTjM3RjhJNEpHNzVQeFpKVjlW?=
 =?utf-8?B?K1d0LzZoNzNyRWpDNS9STm82U3pja1dVb3V6N0I2dnFaN3hOMFNmOTBvVUhi?=
 =?utf-8?B?Y01aUzlBdFVMUXhvM2xSLzkyVEJDTDR2YWZMOWRqL0lsUFl6VlJIOStaVWN0?=
 =?utf-8?B?WWcreTJCb2cya29LclFUamhVLzEvNWdWc1FLNVdwMkhTNlF0bys3WExpcHNp?=
 =?utf-8?B?MGFjd1NacXZpRU1BMGxrcStvOUt3TDM2OHhtMFI1VXJEQjZXbDN2WkN2ekRj?=
 =?utf-8?Q?71kUPElaM53sIe3nxRQvkYgXEMkbIMTEtaiPE66?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0bb73bed-6c89-412a-2339-08d9834b1cb7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:14:47.7835
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UtEVlkbDpvc1RMt3KX76A2rb6wPkfkVDI1HcTqlp3HH0PG4XefMcWPEqLs8ZiiCT+BCvQn9r/lvHQivQSEfCmw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

Wire up show_hvm_stack() also on this path. Move the show_guest_stack()
invocation out of show_stack(), rendering dead the is-HVM check there.

While separating guest and host paths, also move the show_code()
invocation - the function bails immediately when guest_mode() returns
"true".

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v4: New.

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -303,10 +303,6 @@ static void show_guest_stack(struct vcpu
     unsigned long mask = STACK_SIZE;
     void *stack_page = NULL;
 
-    /* Avoid HVM as we don't know what the stack looks like. */
-    if ( is_hvm_vcpu(v) )
-        return;
-
     if ( is_pv_32bit_vcpu(v) )
     {
         compat_show_guest_stack(v, regs, debug_stack_lines);
@@ -611,14 +607,11 @@ static void show_trace(const struct cpu_
     printk("\n");
 }
 
-void show_stack(const struct cpu_user_regs *regs)
+static void show_stack(const struct cpu_user_regs *regs)
 {
     unsigned long *stack = ESP_BEFORE_EXCEPTION(regs), *stack_bottom, addr;
     int i;
 
-    if ( guest_mode(regs) )
-        return show_guest_stack(current, regs);
-
     printk("Xen stack trace from "__OP"sp=%p:\n  ", stack);
 
     stack_bottom = _p(get_stack_dump_bottom(regs->rsp));
@@ -687,8 +680,30 @@ void show_execution_state(const struct c
     unsigned long flags = console_lock_recursive_irqsave();
 
     show_registers(regs);
-    show_code(regs);
-    show_stack(regs);
+
+    if ( guest_mode(regs) )
+    {
+        struct vcpu *curr = current;
+
+        if ( is_hvm_vcpu(curr) )
+        {
+            /*
+             * Stop interleaving prevention: The necessary P2M lookups
+             * involve locking, which has to occur with IRQs enabled.
+             */
+            console_unlock_recursive_irqrestore(flags);
+
+            show_hvm_stack(curr, regs);
+            return;
+        }
+
+        show_guest_stack(curr, regs);
+    }
+    else
+    {
+        show_code(regs);
+        show_stack(regs);
+    }
 
     console_unlock_recursive_irqrestore(flags);
 }
--- a/xen/include/asm-x86/processor.h
+++ b/xen/include/asm-x86/processor.h
@@ -493,7 +493,6 @@ static always_inline void rep_nop(void)
 #define cpu_relax() rep_nop()
 
 void show_code(const struct cpu_user_regs *regs);
-void show_stack(const struct cpu_user_regs *regs);
 void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs);
 void show_registers(const struct cpu_user_regs *regs);
 void show_execution_state(const struct cpu_user_regs *regs);



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:15:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:15:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199093.352938 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZQp-0002ex-QK; Wed, 29 Sep 2021 13:15:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199093.352938; Wed, 29 Sep 2021 13:15:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZQp-0002eq-Lw; Wed, 29 Sep 2021 13:15:31 +0000
Received: by outflank-mailman (input) for mailman id 199093;
 Wed, 29 Sep 2021 13:15:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZQo-0002eb-3s
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:15:30 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a1860cd-3f21-4fe4-8e07-12f36a17a590;
 Wed, 29 Sep 2021 13:15:29 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-7-TzB_t_pHM_m-6GiNITrmCQ-1;
 Wed, 29 Sep 2021 15:15:27 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:15:25 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:15:25 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0103.eurprd06.prod.outlook.com (2603:10a6:20b:465::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 29 Sep 2021 13:15:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a1860cd-3f21-4fe4-8e07-12f36a17a590
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921328;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cLGJUBAqG8d0gm/NqmHESrd2ZgrMqEgXRIhq0Lc+mj8=;
	b=FDK/RSjbJRAJhbIuXuSllpUsmXOJUmPXP534atJyfu43EKE07FaUsAZIrndp70mrBJAXIZ
	CmTBsfHtZ4zCSN0s6W1Gp1mkmNnwuT2gmMYoGYumZl7u4AjByYRESc1csI4KyGzRmq4Jjg
	opEqQIS20cAJYkXUcEni76UM0qgd9ow=
X-MC-Unique: TzB_t_pHM_m-6GiNITrmCQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MyPyY0X3CdbrApH4GlNlmy7go66YHNLd1U/FZwWghNZe8U+E5pHT85SoMBQwQwyKOkCyn4V0ErC1KmB++r6Chehn3GDorgYnT+gzfc3EGTqYpGiOqfTurwWXAF9qr0+hSh2Ed49iBGfQZ9uRWjebj7lg9Jw1yjW5kuBTirY1Xjy6H8G2N4alnIodBsNXcu2v9e7UNUag32KPnArIgfDsXivQVOx/dBm0BY4AXorhaoHKFLDjZkNlpVmTY18+S4G7+ku8Cx/vyruvjt8j+5HSfzjBHXgUHYknZfMXOiqJWGexggQD6vv9p8IrDV8/Xpp9TxAa1YYz/nrGgrAGGhb65A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/SsiSshy4JYyNEDKdw1hBdETYO6OhRZqVzfvWAITRUY=;
 b=Lhkk3xVRweitjsrTY7AFdUvT0e1wTNFpMEbBlr9Qs6v+O91JhfpWFFYN9gK5p4OdbEcjbzmKadK+T15GtVyIbcSfIPyWqe7LruY17HvXj7J2mzsnuICoqiCSfzX5/VPFUaj40Al6v7HaYwOcCRKmlTrOK9hx2ReDgvssQBSR98UVzxHBTt4m7PjPZjFy7KRGFm+3PQmJf7ruFT/knuso6c+6I+DXZOrUg14fSnKKCrNJ6sqH1wwcefs+Xje2LL3Uk84F8Dyxm4GmH7oQHFxDRKf2Zu2E98q85e7QEbCHowuA8SoNv9H0HAHP4FaqbfWkrMLOljBEheKgykYsspT5VA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 5/6] x86/HVM: skip offline vCPU-s when dumping VMCBs/VMCSes
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <4df38415-4154-e696-aca5-fa9498d1c318@suse.com>
Date: Wed, 29 Sep 2021 15:15:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS9PR06CA0103.eurprd06.prod.outlook.com
 (2603:10a6:20b:465::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: af689673-9a1d-4cc4-3b4b-08d9834b3356
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7150D9C687F5FF2A4B2B015BB3A99@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3968;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	cc28zFlhsXgLqQ0kwEthlvWtdnBbsNFE+ZFlpy8t0jvpWxt+vOTnaSRkpSIKO9kreyCijupeM9Bda+bdibXZ/3RbhyXfAN+LXh6S2JJy05Wp9AvHeQUiq63PWyY/l4nctMnbWPxnaTAFVz9Dzk3HsbW9pDdvhP5c2NbmXcV2UXXnHzKhAEForL6/sMa1Zp9kycvH/JtJrT1ox5kZuFmL6NQ8ErR5tCZ0eMFAeRbNUZ2DV8Oj2VFUalopZ7f2F0hBQAUxmhI6B3QRWhv2Us2uGE4m5SHZPMHclneKvDkXFZH/spJgcKn2C1x9ENi8rdE2FCjDh76hpF2BS2qB2pKLLJAMVLzfe3F51tsv1MhFjpknoL+6psGSaHqmem0XP2vGswhBXshvyNXrnW4bWD82/zbxZl+qCM0WqkiwIY1T4S4mtpfKuzCfMEDq3Es9CTmRuTS9BxOMU587fUsqVeiKTH/HgXA6o0aF3XVlWKIrRRH4kog8k1uygN0OmeIVAUsK6uK6x3YlfyfShrmJvWPzjrbGIxUYJod8unLqeimNkr2oggXQntAYM6IZqNZDo3p+uyrM1D7mNAl7j8c2UCzAcFneG9Ii/Dpak0vhZ3fHzhGfo+bf5I1yvL+r/KVInbirDADyxL7fJnVFebxqgVQLrOU9RpJsSJI5uWRYXSlfsqafalBYJJ4HmjhXNdDLxYfjR6tEPFPjZl0vbjh6CMGPKJTgKta2vnMwaRy2TeX5zoU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(66556008)(186003)(2906002)(16576012)(316002)(5660300002)(26005)(66946007)(8936002)(31686004)(8676002)(54906003)(66476007)(6486002)(86362001)(508600001)(36756003)(6916009)(956004)(4326008)(31696002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?YA7aZMWKlUbXXCt2rRTnxx83JNH0m5vRdTrTvhQrchcaMQb9vrHKGP6cccTV?=
 =?us-ascii?Q?gLko0+LhaAhr6iJHsd4H+PlzUx893594MzBlKzqZcCTPRI+4M9qTynWiGA8q?=
 =?us-ascii?Q?opgn3aHFkD7pVlvRJZ5Dg/eMe1g6bFcnBv1lqbU6tTPB9hzkGXB3xpRUrNYY?=
 =?us-ascii?Q?sCOj5kAV/d9SarNAiJzqQMzXHMGA0VJg7HmDikEdEsm+7bSeOhrftb+heygG?=
 =?us-ascii?Q?9lnLiA265gDiFa0ZmUr2fCxi5QeHRUeMDrVnSjcQmLqr1Ju2flQnsFq5rfgP?=
 =?us-ascii?Q?3WFh0iwZfEskoPM+CV1tP/WibDK0rzDCYfeCJj+cJAH2UxvCTP9/T38MWF6r?=
 =?us-ascii?Q?FZNDRAicTzMrXxa5KYlClsKmLShrZW7XPAr+JV3h8aj3yeZDu7t4P/9jo22n?=
 =?us-ascii?Q?7yd9q4POW7XFPVqQnP/rSlQFTvmY4MFCXCPP7OnGJKU8Ih1aKqlF0r1Dih4b?=
 =?us-ascii?Q?DJ6pEWousoibz6Q+dhY14YvdcUV4t8XQjGAHBmhAesIKR6Gjzze0j/dQCP5g?=
 =?us-ascii?Q?RGh2rmej+P5Rq9OiORZ4gHJ6ajstYq4YbRyY+6V80zIdyKa957z4uQ4NmKB1?=
 =?us-ascii?Q?FE3GYtjryj1F4ipObklOJ+Nh8H2F1PElbZNgQ5cTZSI7+nYpMdUnKVYPqfO2?=
 =?us-ascii?Q?E2zcvlQ/DZvRp4LEjurXODgx4MTMMPUrYOhQ7LNG9mxTUhjVVPyjfy6YP0Fd?=
 =?us-ascii?Q?GAtY7eIPeudmi8YX09VQvU039H6XpJDU+5BCTSZfof3BWAZL9XBPLUQ4GvYR?=
 =?us-ascii?Q?WIOK6SKtlq9ZxcbbRbi9VAtbfTWkkBLHp6L5XnQZF+/5Vgps2BEamQdMg0cR?=
 =?us-ascii?Q?g1ewePpXuU0bLNGSqr/iEDvfEJRton6knK7mwsHohdtLUy8s56NF2Q7osgqS?=
 =?us-ascii?Q?jnbidlRLOSvHbKlHA6X+297H2Y+0iPhm9xYFTmgWogEWMnogi6MU3tQlemEc?=
 =?us-ascii?Q?rJOhGLXrX3ORIwkRGWNfUGWNikYXhIP7icdJ5rsmsFvc0RyOjnjb3togSS4w?=
 =?us-ascii?Q?iKCL6ZTZOMwyqjy6nm1RZKMXhK9R+KczrShIaRlxqSnoVELrvJA33ckGuCxE?=
 =?us-ascii?Q?/a3G9rhzwXZ12kMA/nNna0M2r+aNv5zJ3bkFp+wV7Q8aGugCNDMeGVKLgJle?=
 =?us-ascii?Q?GchdShc+rqVU7087YvTp/PYJ6X9xIGXThML3ULZ4CR49R3CBt4ftXXIN1Py9?=
 =?us-ascii?Q?+jqkjhiyXlndZzstZxf858Y4PWGtNslCLaCdZRhN4MwL3b6GB+mS90iWBrj8?=
 =?us-ascii?Q?hxXCEwHAr3YKvh8sez/VbThEEppML3LsVThBmXe5tp3kX0PFYDT/pu47sEkf?=
 =?us-ascii?Q?nZs7J2kI5Ze2wm+6J8rUi9OY?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: af689673-9a1d-4cc4-3b4b-08d9834b3356
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:15:25.7280
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YwfY+YHvq24ey5gN3wokohBnbVGPQG1mVzl+v5YMfmCLCOHipZZ07vuN7qR82XMMxxEhi+F34ZTv7goRtOi+Cg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

There's not really any register state associated with vCPU-s that
haven't been initialized yet, so avoid spamming the log with largely
useless information while still leaving an indication of the fact.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
---
v4: Key off of v->is_initialised.
v2: New.

--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -242,6 +242,11 @@ static void vmcb_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( !v->is_initialised )
+            {
+                printk("\tVCPU %u: not initialized\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             svm_vmcb_dump("key_handler", v->arch.hvm.svm.vmcb);
         }
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -2133,6 +2133,11 @@ static void vmcs_dump(unsigned char ch)
         printk("\n>>> Domain %d <<<\n", d->domain_id);
         for_each_vcpu ( d, v )
         {
+            if ( !v->is_initialised )
+            {
+                printk("\tVCPU %u: not initialized\n", v->vcpu_id);
+                continue;
+            }
             printk("\tVCPU %d\n", v->vcpu_id);
             vmcs_dump_vcpu(v);
         }



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:15:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:15:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199098.352949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRE-0003FW-38; Wed, 29 Sep 2021 13:15:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199098.352949; Wed, 29 Sep 2021 13:15:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRD-0003FP-VC; Wed, 29 Sep 2021 13:15:55 +0000
Received: by outflank-mailman (input) for mailman id 199098;
 Wed, 29 Sep 2021 13:15:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZRC-0003F3-Np
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:15:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5f78e1a6-2127-11ec-bcfe-12813bfff9fa;
 Wed, 29 Sep 2021 13:15:53 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2051.outbound.protection.outlook.com [104.47.4.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-uBEukRGdPEi98zdElX4-CQ-1;
 Wed, 29 Sep 2021 15:15:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:15:51 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:15:51 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P191CA0060.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::37) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Wed, 29 Sep 2021 13:15:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f78e1a6-2127-11ec-bcfe-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921352;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=289UvYu1z6hhybJctb/uHf8lFebZQ7uBwKVUfRIW7tA=;
	b=Y2/9WnbZflcsVK3/RcugaVUFAe+t95Az81fMM+8rCvFDQ+GjNPWcLgJbT3QU7zJAb6W+nD
	ZjMZBVeepWoukFyfud8vxSDIJt/Vmjpwz2pCJXCugGonOLjL8uP2bJF4A1uF+qfYyAn5eP
	j6AUnAKyg9yBMfWv/0J3m1/UTkTkU/w=
X-MC-Unique: uBEukRGdPEi98zdElX4-CQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WZTQ2Qo/kf5nIVak7nvHSIiGkONhIf6lO7Vfr/kFAPJ3sZL3H+wRrupGfpfuXQCHpeEK96ULyFMbaCtxcmoSRd7Q4dBUacBprvqiegLaGY9GRhM9ivNk2Of5qcgJHMlktJKgan41WprcINwANoqJ0We2xCNu7g1D1+Io0sW8Zuc0Ej6vIpl/688A3/ynQBbiI1r3OxMPk+Ref3t0XXHgzSAmFNXSYhDEla/Gx1dmQrPYaavCTNJ1iGPFmEvr5inx/XHw444JDE6CDEYLw7CmS5fD5yX19xkTWqkJ8vPpgVUr/Y067YCrJjtLcyhP4YRDHDc9Y0Qa0Stlx2vuiZ4RNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=289UvYu1z6hhybJctb/uHf8lFebZQ7uBwKVUfRIW7tA=;
 b=UpuLwH+Px761mS2wamIljFLDTjk6ivaXUPQUcEyEOmv0gq7qc3pQf4EqofHho/yTQf5k/5guaQunxloNhPBe3Usl0C5W1jnlLRanMGpmRIBquPfWuxsnTgqBCFo3uZS8zFKcUEg57sGJUYcL5+1MVQLVaEv+KS3qfVDvUGszO44nrziYRLwTdLqIs51ICajnmEp83zQb/vLxzrwe3HVxZ6LpmUSflx7WXYqTPxMzLAzhIxxkWtuUaCFl/blWsSvS9GQAVxTCzhRcZzf4GgBRad2on2zHwwKNEc78Df9tj4iF01K62fWfY1v7ohHVIQODwvtEPJYGgnkfZo1eM7dMdg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v4 6/6] x86/P2M: relax permissions of PVH Dom0's MMIO entries
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Message-ID: <4b8c48ba-a563-d036-d4bf-f4f8bdf0efb1@suse.com>
Date: Wed, 29 Sep 2021 15:15:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <cbdc93a6-e675-420c-f7e8-53aa407766ba@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P191CA0060.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:209:7f::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 80f1b0d1-d613-4907-1926-08d9834b4264
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7150A3160589C7DF992C7DADB3A99@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TPRDrz6BCE3Jc+Eeoa8oNpdKL8tqpPF6k/XQFGBrsFDBk4qC6hsk53E18kYRqP0aH8vDMvTl4ejbAXJ5JhBxU+ef+AVTzSIIXfgY+6ZnKbctC9tiWuzuCsDRGWL1abdXFqTOTtX7zMT8oxVnA1eEVZbmHlrBhBXzihx/zivFlS7IoxNpH6uMFEQNHCQ4XP1ukrytH6rZtiDZHomabruV2H8VqWazqg4og+qiCBkLb3voCI2HBdDr5Zyrl1dpZENwdw6RU0/rh55wFSqgKNSeRPXuUChhy5vYJSekpH6RrVUiyhXygLcvV8dfGGaDFj32R3Y5eQ4Rne08rQA24rUeFMRiP0HJLx5bSOZrQEyQUe7x6zaajDdbVzFTjqYfVoa67GWWRevFmtKMDhdINRMPjQQldtV7sO/04vZXXfCd7cUUWmFFrxnOub5NIy/OGEgFC4uryPCvx6KLt/gDnb95BTt28JGZP3x0gQm+qblXBU9fl52EDtpLI7R1KbIiZgTQoIG9wsrKouRj17FAev5G6g4XczCkCcGbIdVBE7Eqnxbggq6S0ss7qHvoo1FIb3IV4pzfhyIAAj17JgrqSptN4rIFWyRGWQPViYCAiPPxLMXEHFgXtqMXV60dlgkm5087bGwoiZ1zbYlT942QaSdF6MzR1BiNb1EFyBErTgFpgMpjVCnweJDK5LUzNeJA9bEt16w5e01Vit5oGZE15QFAL2163oFyafShI11RVi1iBbw=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(66556008)(186003)(2906002)(16576012)(316002)(5660300002)(26005)(66946007)(8936002)(31686004)(83380400001)(8676002)(54906003)(66476007)(6486002)(86362001)(508600001)(36756003)(6916009)(956004)(4326008)(31696002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZTArcWh5WWJvUktxalhWeFNwcVM3eTdpNlFDaXY5U3AwVXIwR24zWnVGME9O?=
 =?utf-8?B?dVF1NlkwM2RrVy8xM1p6N3FOOWZrZUJJZGptQmNZcXZxZEM1bkNuczVCWmkx?=
 =?utf-8?B?L3FCckdQeFhmbHBaL0RWNmNjQ0RyRUdObjl5ajBSTmREUFNncGl1Um0yNVRh?=
 =?utf-8?B?TmY4bmZmYklmYURxYjhPbmp4eFROTzNNbXkvY2FYOXhUSi9xMHNxQjVVcTdS?=
 =?utf-8?B?dWUyQ2FZZ2o2b2oybEIxSDdVYm5rRTFTcExyTXQySCtnaWxqTW9RckVaNGE0?=
 =?utf-8?B?WVJEL2JQSitXMTNMdzNwY2xMSm80eVJrV24wd3UrWTYvV2xPNW1qbTdiYm1j?=
 =?utf-8?B?WmVTcXhlRVdneHNvUHVLMm8rZlJ2NHNWTHg0VlRjRStxT0d0SGFTL2huZ1FP?=
 =?utf-8?B?VEhrbklJNTR4RU51YWFqT0EvNTIrZEZiclZhN2pFQlhaaE9yTEZVSStqbG9Q?=
 =?utf-8?B?ZFRzY1IxcTJtZ2FRM0NLSTRHYTRqNk5lQzFvYWVjdW9KMU54QmxhcElzdkRU?=
 =?utf-8?B?ZS96d2I5cGlPK1BDL29MOUNDWEJVaXBuaG9jbkNjS25VTEJhSEZLNDBTTlhq?=
 =?utf-8?B?U1lSMmN6M09TRG9hWjBNUXZsMW9OOUZJTVQyUkJyZjNkdmdxQTZYcUtiYmhk?=
 =?utf-8?B?QUJFTk5yVjJpUlZPNGxoOS9idVovZUJaLytYVjcvM0FlZWRQWTNxQ05TdkJB?=
 =?utf-8?B?R1Q1c0dRV050RnVYbWtrd1B1eVFEcEdzTE0wZS93aEk1MzI1NzRKQnlaN09a?=
 =?utf-8?B?TmJlVkJGMWlLdFVXRm0zR1ZJcng1cjNJVzBwOXNodXFzaGU2b0ZYZENuWXRM?=
 =?utf-8?B?OSs0WkpTZlpVNmJ3RVBmTXFYR1AvOTJKZVhrMzNRWkZ3R3IyNm5mMlQ3Rjkr?=
 =?utf-8?B?d1RJZXZONk9Pb3Bab3ZQTG1ySFBZLzZPRzZkSGF0Z3Q2YS9mMVY1T2NxZFVs?=
 =?utf-8?B?R0Y5azhiMjZjV2lxNlp1QU5pOUJiNzNwTU1kS2pCUEFOZGhmaXJwMUtmTEVo?=
 =?utf-8?B?YXhEaCtUY0hoZFErVEd6R3c3Tk50SXdNUXFmTW9wMFhUOW03VG1BTERBTTNC?=
 =?utf-8?B?cHZYek9JMDBPNEUrRWszTXluMjV3L3FZaUNJZFpmWmQzWkxoR29mREo2ZFdv?=
 =?utf-8?B?OEZMRXJ2MGo0YTdmWi9ZMlJ0OVVqYjlpTUJyRFgzRkNhZ1FsM3IrL2twY3p6?=
 =?utf-8?B?bmJGMy9IV0lKNHQ5THZJby96TnhQaEVXOGNpWU0yei9CbkNqaU1Pb1RBSUNj?=
 =?utf-8?B?NlBQRCtWTDZuNHVRYlBnYTMxdWYwZUIyNHRuT1ZUc3dhZ0FDUkxqTkRsdldZ?=
 =?utf-8?B?amlLWkdtUzVGSSttNHRicjM3bDFpTWRwL2pHaVUvbmlLbVJhblVVUFo0Q1ls?=
 =?utf-8?B?VzNkK1EyNTIrMWMzZGdVN0djM2lMUi9xdW8vNkdiK2pSZm9QbndFaTZ6N0FU?=
 =?utf-8?B?UStDVVk1cDVvU2VOQkVNem02NHR5MC94Nlk4KzdvYUZ5dmZtZDRWVHc1d3k0?=
 =?utf-8?B?dE1ZcTdHWXdkNlNaa0ZoRlBCbyszb2FiWEJkWXZOV1h3bGExallBcVRWeXVM?=
 =?utf-8?B?WjZ1emR3ck5YNFovejBtaVltMmFiRGhHM3lYclp0WmdEZnBXeU5YdXcza0ps?=
 =?utf-8?B?TVRIMkxwbnlNYWpLbjFsUXZWemxIUEljWFdoUDEwTWIxMStpTDdkZ0p1ZVNx?=
 =?utf-8?B?K04vVktBZ0J6QmwyZVJuYWc4bnlpTGhuVU5EaXhtODRIVTR3UCs4TW5wOWww?=
 =?utf-8?Q?fNq6leKW4YGiCCSuAPUgfYyOBPHbaotNynNAX75?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 80f1b0d1-d613-4907-1926-08d9834b4264
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:15:50.9967
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9HJ4sRWrMZc2S6To2MvFLOtPaXfT5CH6tecAqvOr/mZF7RBc4ieRZ9IE3Fwd7xY/gPkYcMnE8frKGws3GoLcEQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

To become independent of the sequence of mapping operations, permit
"access" to accumulate for Dom0, noting that there's not going to be an
introspection agent for it which this might interfere with. While e.g.
ideally only ROM regions would get mapped with X set, getting there is
quite a bit of work. Plus the use of p2m_access_* here is abusive in the
first place.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: Move last in series, for being controversial.
v2: Split off from original patch. Accumulate all of R, W, and X.

--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1319,6 +1319,18 @@ static int set_typed_p2m_entry(struct do
             return -EPERM;
         }
 
+        /*
+         * Gross bodge, to go away again rather sooner than later:
+         *
+         * For MMIO allow access permissions to accumulate, but only for Dom0.
+         * Since set_identity_p2m_entry() and set_mmio_p2m_entry() differ in
+         * the way they specify "access", this will allow the ultimate result
+         * to be independent of the sequence of operations.
+         */
+        if ( is_hardware_domain(d) && gfn_p2mt == p2m_mmio_direct &&
+             access <= p2m_access_rwx && a <= p2m_access_rwx )
+            access |= a;
+
         if ( access == a )
         {
             gfn_unlock(p2m, gfn, order);



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:16:31 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:16:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199106.352960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRm-0003vl-Dd; Wed, 29 Sep 2021 13:16:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199106.352960; Wed, 29 Sep 2021 13:16:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRm-0003ve-Ac; Wed, 29 Sep 2021 13:16:30 +0000
Received: by outflank-mailman (input) for mailman id 199106;
 Wed, 29 Sep 2021 13:16:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVZRk-0003pX-Me
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:16:28 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0aaaf67c-707c-4d58-9bdd-4c6db898ded2;
 Wed, 29 Sep 2021 13:16:23 +0000 (UTC)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TCYjvd027853; 
 Wed, 29 Sep 2021 13:16:18 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bcnhnru2m-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 13:16:18 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB5939.eurprd03.prod.outlook.com (2603:10a6:208:15e::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:16:13 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 13:16:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0aaaf67c-707c-4d58-9bdd-4c6db898ded2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mM3dmphBoB1ny7yVlImfhSWZSIlwK7Cqa3WoGcUMbt9u8Cq+punH5opiQcpmHZ9rbkh3w6mscMiM3482RDP1GsdyRLAOunVLD1cLWF+RVRd/S0vu9ia9fFKTlBFKH/Ebgtu5+98YZAJo/ak/7GbvmpIur6hqtjMlbRKeO4zRBS6QSooxXVN35FlGvLo6GWqnAbm/uMbmLeddmbTDv16895pW3C+QksjvvA3P++mV5IPFs7IFwmjfkC8ZfoQsKvvzmjWPqYPG4Lk76pd0x6lPAiaTy96JtbaqTUKBAWoe7OVUn0fVaQxNVGQrOXA46uk2wDvXyicQQA5sce5xX7Tdkw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=xul+tKc9Ri+hyTebX1TzVUIGgFxYiU4G3VIfMrQDQJg=;
 b=ZFNFreQTGVqY39Gu9EfSHjHgt60jdETfldX0hTQmk395FLZ5gELsNVPYfs67Si8Kl58zu8GHfp2X8s2zVwsVj4Tnpt3Ntqs/XsOnM0h9M9X9n8GrXLMsvk48Fdwh52BUvjK1fzTjoG+EfYd17biKmb8xzOipQqbFa6hgK3F+2n6dbdku5QXn9eZ/9klW2HOYtN8jmMRfWXAVHkxs34TUHB49pUiZbgI9jm7zkROxbpU2SbmK3PBqI3htX4Y7IRzxSNNHpYG8h8WKIvVGf7nnuSGnqEB3fQMh6F4U4HuEX9HRtbMqftsB3VsXGS06YLqBuVJOYbs3RE6i7sN6pGYlJg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xul+tKc9Ri+hyTebX1TzVUIGgFxYiU4G3VIfMrQDQJg=;
 b=aOB2h5rFOD7MpK1rPLkpPr96bEnqy4jhri9nONCSxF8vQ8EG5EhrI2sDksRZOa9H41JeAu8Q6us2+HndVithzACZ/eOv9/zZQ79nDT8ckYBl3gzaXPxWmIPJ5otveruZ/3C4Os+gDgO/nt2f0gwonvcGKqSyLBIjLkxsAleqx2LDOjM83HBcEBY+86y47v/DARaBIPK/Ox5k7wllL0E6MDAA4/fdMGXG4w++bQwgEYCb18OzlZ4kbe666QffxTReAiUF0aJc3GPkWsXzSDkroP6QQy4Jp8KeDWPOJDtMcsVnoYjMj4ouq805mcoA9zhQkvPGMOFy9KKNERsZfhc2lQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Michal
 Orzel <michal.orzel@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: 
 AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyAgAFQpACAAAGzgIAALsGAgAAQSwCAAAYBgA==
Date: Wed, 29 Sep 2021 13:16:11 +0000
Message-ID: <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
 <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
In-Reply-To: <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2a123798-a3e4-4cc0-6fa5-08d9834b4edb
x-ms-traffictypediagnostic: AM0PR03MB5939:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB5939635D1C0437D51256C25BE7A99@AM0PR03MB5939.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5797;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 QWTfi83xE3jONs9fIP3Cp157WsgQge0xK8sl0Nv+m6aoxijyrogELsUMDCSE9qApyJCPGnYNlBrPFKTdajwBgtYlpsKq2OjHIyol67gsiRjVS1lG6z0TrD3c3MiYZPIE6PNQ8HtdIH4tC7u0BDNLyTBOkl29cheky7L1URr+HHuO1nL0Y/gJSoColpMWovBgMt+HAGZQeXqnnA4KsH7k+n4JwxM6RhI0nJRY1W1Sj+dtkBbAurCc2u0SoHrjcBt8/cN4Yptrjq3VTrKeymbPLItB4IjH9Jy3Z0NyoDEr7Oe7EbfN0QL/61KtAcG9w6MZfpCIlhFlDaMyOyo6N4uG6jISs/xBtpLipfoHM9RAhEO1ewOUDVyGYPdOSVniiLU8JbCrWUdUxkvvPHbck4ea9gfrk+lWoVv6FtVElLvOy5K3tlBhgWWn3QIJHkBZCcyPvLdKXhYCD9chECNCBAE9ClEddiKm11wH3y7R/+5H04OHgFeOVS6YOLo173G4t7+xkn9PKqya0a0sgVlZwKv5g3M+vO7oq51ZoxFFrWjX+0pygwhXkBhiUmJCjx6lIo0OmiAP9c/rOMwyR+rW81BG9GGZj7HgDPR/fUF6YDEGKD/mWTh/Rh+P4Qh3is4xIjC1886hvPoH1wo1ylsRuw7gkFpTZEDKcrvD06krhZ9/hyHXaUWqEGZeHQplcRb+ULe/sAZRw1jZybemRLbEwnHZDPCpt7ObzbmL49MrIB/3WTM=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(53546011)(54906003)(2906002)(55236004)(71200400001)(316002)(6506007)(31686004)(38070700005)(66946007)(31696002)(86362001)(64756008)(66556008)(66476007)(8676002)(38100700002)(122000001)(66446008)(76116006)(91956017)(6486002)(6512007)(4326008)(36756003)(8936002)(83380400001)(107886003)(508600001)(186003)(2616005)(5660300002)(6916009)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Y1VSZE5VMm9hQmZTVEdoQXdyVXA1aXJMT2x6UFZGbjYzKzQrRm9CM1Z3SUZQ?=
 =?utf-8?B?aVZxMmFFWWtjc0wwSksyZlp1OFMyRk5oa1hlUzUrZWJxMGN6cE12anQvZUdp?=
 =?utf-8?B?NnBrQ1RBRUNZZTRSOHYyL1o2NkppQ0FJTDNrWFRBa0RYN3J1NXRBOFdBSDU4?=
 =?utf-8?B?Tk5naHFqbHBEUTRSUDdBQXdia3ZJMDJqUUswRHZ3UjNHanMwRlN5cTBWWkwr?=
 =?utf-8?B?Q2hGT1hTa2RJNFFZQkZhQkhnSjJ4OU45ZVZCa29RNFFPWlVMTFRmMkdhZWJF?=
 =?utf-8?B?RFhuWDJKSWlPa1Q5OUlnUEpsTkxITk9ONTFnOVRNNnFRSEdHZXRNNzRQekFV?=
 =?utf-8?B?NjhURXRMa2hkN3RzR09pR3h1OG5rd3VSZmMxc1JEUWt2dTVteTA4VGt1ckNw?=
 =?utf-8?B?dTBaMFh5YmV3YXRMYW1ZQ0R3SkxhMVNjQXVWTE0rcUJ2UWdFZjFYTTAydll0?=
 =?utf-8?B?cnNoZ0xxQVA2ZS8rRjNsM3hmK1U2bUhUSzBwSFYxNEpYazJHQTZrOUJuK280?=
 =?utf-8?B?RFF4dFRBbExUYWdpcWowR1ZxR1JlamkxU1lTblo4NTFMSUh4SmxZY0puRjNG?=
 =?utf-8?B?dzg5Q1oxbnd6TkNvZFJSbXg3Sms0akZvNTRtWmpaNXM3bTFvQnBJaXl5elVK?=
 =?utf-8?B?TDNvenJXQzlsTEgrMFZ0UGtEYVgrMFZhY0h6MzRObVgwUk9zM1FCaGs5NDdG?=
 =?utf-8?B?ako2bDM3VE1rQ3FHM1NVcEZ3ZnM2WjUwWkwyTXlGNnd3SENIOUhMem81aVc5?=
 =?utf-8?B?bGdYY1hTbUJITXZNMjdjdFN5MWc3MjFVUUpsTktvbFpmQy83QkFZbnV0WlY5?=
 =?utf-8?B?QkhDdjJuTUxDZFJ4cDcxUmJhOUJOYnlZd21jWnFRbEtiSkdXTUg2NUtEMUQ3?=
 =?utf-8?B?Sm9TNkliWlRtbC9iZ1Z6Z1pYam9DbHErbDVZNWlwcEhjd2NOVy8rNFJ5d08v?=
 =?utf-8?B?QVNCdFExanhwdGl0MXd1Q1NYK0k4QngzZGd2QXY2dkF3eTFhSVZFRmtTaFJa?=
 =?utf-8?B?VllmVExhcVhpOXBMc21RbUFPVlNnYmtmQUFQdmlLczlxUFBoZlFKQVdxU0hZ?=
 =?utf-8?B?a05NQWQreDdVaWdrR08xVnJabDRIVFhqcjhGVUpGRG9QUUhrbjRaZmlRQS8v?=
 =?utf-8?B?aS85QTM5elRyb2I4ZEZtWmg0VHE4UFdOSlB1ZnRoQ1U3NlVVUDFqRW5SMWtN?=
 =?utf-8?B?Q3lpSmZCclBVb1Q0M1VuUTBXenE1ZWFBdERJVkFkYXVHTll0NUg2Wjd3Z284?=
 =?utf-8?B?NjdjaytGM24zNWVnRTVCd1JzV05EcjBVSGZDS1ZvTmZMSVF5dkprNlFZbTB4?=
 =?utf-8?B?Q2xyeC96NDZpWmZFZkJYcnNFNmExMG1XcEQwbVBVL1ZjWHRBTjUrVW1rSHNv?=
 =?utf-8?B?ZEh3S1ZhVU1tQWJzRkFxaTRDaFBpd1JWanBjQkJRRitUSWVvbWRJZUhGbmNt?=
 =?utf-8?B?NzNUNFhZSEhsbEJiUDViOEhrUklEcEtma3hTZXpUWlVDQ3NTcjgyN0FXNFBq?=
 =?utf-8?B?WUI3dTBDN3dkeWtpaXRBNkNwZlJPZ0F5UmhBS1NDWi9kcDlZQ0MzUUs0WXVs?=
 =?utf-8?B?aGpOcXY3VFRmN3A5V0FIM1hNSURqKzZkL1IyM1FSaDZQbXZ2VmFMRWdIL3Ar?=
 =?utf-8?B?elo1NiswQSswUVZWNEdJNXp3T0ZlanYvejdlQXpNWko2U2J2UnRGczRHT1Nj?=
 =?utf-8?B?OGtLOFNob2U1N0pjaFdTd3NTanJIV2ppMXZRYklsQ3FRL2tJTTA2T1hOY05n?=
 =?utf-8?B?WG1Rb2pra1VCSE1wWmtUNm85Yjd0WUhrY3BCMkxsNGV6VnFGM3VRRHh4bFpO?=
 =?utf-8?B?Qko3bnBGMzFoYllETlVYQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <23D11C7B88ED394A9DA60A83B71581E7@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2a123798-a3e4-4cc0-6fa5-08d9834b4edb
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 13:16:11.7001
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fA1zQQ0pYrVLV4KlHH29AIyUptQSri3g354c1mQN4J+/lvrUj9rfw+FMGmvTKpGFIdRpySW6fZmY8/DmAW0jYM+IAFYlSDUu5fPlAIU2CVLyTDzZa6MTbl+YMJp980Ea
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB5939
X-Proofpoint-ORIG-GUID: gWf2HQS8KXZANbUgU5u_z5etfw3S1_VP
X-Proofpoint-GUID: gWf2HQS8KXZANbUgU5u_z5etfw3S1_VP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_05,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 mlxlogscore=669 mlxscore=0 phishscore=0 malwarescore=0 bulkscore=0
 spamscore=0 adultscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290083

DQpPbiAyOS4wOS4yMSAxNTo1NCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+IE9uIDI5LjA5LjIwMjEg
MTM6NTYsIE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4gT24gMjkuMDkuMjEgMTI6
MDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI5LjA5LjIwMjEgMTE6MDMsIE9sZWtzYW5k
ciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBTb3JyeSBmb3IgdG9wIHBvc3RpbmcsIGJ1dCB0
aGlzIGlzIGEgZ2VuZXJhbCBxdWVzdGlvbiBvbiB0aGlzIHBhdGNoL2Z1bmN0aW9uYWxpdHkuDQo+
Pj4+DQo+Pj4+IERvIHlvdSBzZWUgd2UgbmVlZCB0byBnYXRlIGFsbCB0aGlzIHdpdGggQ09ORklH
X0hBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+Pj4gYXMgdGhpcyByZW5kZXJzIGluIHNvbWV3aGF0
IGRlYWQgY29kZSBmb3IgeDg2IGZvciBub3c/IEkgZG8gdGhpbmsgdGhpcyBzdGlsbA0KPj4+PiBu
ZWVkcyB0byBiZSBpbiB0aGUgY29tbW9uIGNvZGUgdGhvdWdoLg0KPj4+IEkgYWdyZWUgaXQgd2Fu
dHMgdG8gbGl2ZSBpbiBjb21tb24gY29kZSwgYnV0IEknZCBzdGlsbCBsaWtlIHRoZSBjb2RlIHRv
DQo+Pj4gbm90IGJsb2F0IHg4NiBiaW5hcmllcy4gSGVuY2UgeWVzLCBJIHRoaW5rIHRoZXJlIHdh
bnQgdG8gYmUNCj4+PiAiaWYgKCAhSVNfRU5BQkxFRCgpICkiIGVhcmx5IGJhaWxvdXQgcGF0aHMg
b3IsIHdoZW5ldmVyIHRoaXMgaXNuJ3QNCj4+PiBwb3NzaWJsZSB3aXRob3V0IGJyZWFraW5nIHRo
ZSBidWlsZCwgcmVzcGVjdGl2ZSAjaWZkZWYtcy4NCj4+IFRoZW4gaXQgbmVlZHMgdG8gYmUgZGVm
aW5lZCBhcyAoeGVuL2RyaXZlcnMvS2NvbmZpZyk6DQo+Pg0KPj4gY29uZmlnIEhBU19WUENJX0dV
RVNUX1NVUFBPUlQNCj4+ICAgwqDCoMKgICMgdlBDSSBndWVzdCBzdXBwb3J0IGlzIG9ubHkgZW5h
YmxlZCBmb3IgQXJtIG5vdw0KPj4gICDCoMKgwqAgZGVmX2Jvb2wgeSBpZiBBUk0NCj4+ICAgwqDC
oMKgIGRlcGVuZHMgb24gSEFTX1ZQQ0kNCj4+DQo+PiBCZWNhdXNlIGl0IG5lZWRzIHRvIGJlIGRl
ZmluZWQgYXMgInkiIGZvciBBcm0gd2l0aCB2UENJIHN1cHBvcnQuDQo+Pg0KPj4gT3RoZXJ3aXNl
IGl0IGJyZWFrcyB0aGUgUENJIHBhc3N0aHJvdWdoIGZlYXR1cmUsIGUuZy4gaXQgY29tcGlsZXMs
DQo+Pg0KPj4gYnV0IHRoZSByZXN1bHRpbmcgYmluYXJ5IGJlaGF2ZXMgd3JvbmcuDQo+Pg0KPj4g
RG8geW91IHNlZSB0aGlzIGFzIGFuIGFjY2VwdGFibGUgc29sdXRpb24/DQo+IExpa2UgYWxsIChv
ciBhdCBsZWFzdCB0aGUgbWFqb3JpdHkpIG9mIG90aGVyIEhBU18qLCBpdCBvdWdodCB0byBiZQ0K
PiAic2VsZWN0Ii1lZCAoYnkgYXJtL0tjb25maWcpLiBJcyB0aGVyZSBhIHJlYXNvbiB0aGlzIGlz
bid0IHBvc3NpYmxlPw0KPiAoSSBkb24ndCBtaW5kIHRoZSAiZGVwZW5kcyBvbiIsIGFzIGxvbmcg
YXMgaXQncyBjbGVhciB0aGF0IGl0IGV4aXN0cw0KPiBzb2xlbHkgdG8gYWxsb3cga2NvbmZpZyB0
byB3YXJuIGFib3V0IGJvZ3VzICJzZWxlY3Qicy4pDQoNClRoZXJlIGlzIHlldCBubyBLY29uZmln
IGV4aXN0cyAoZXZlbiBmb3IgQXJtKSB0aGF0IGVuYWJsZXMgSEFTX1ZQQ0ksDQoNCnRodXMgSSBj
YW4ndCBkbyB0aGF0IGF0IHRoZSBtb21lbnQgZXZlbiBpZiBJIHdhbnQgdG8uIFllcywgdGhpcyBj
YW4gYmUgZGVmZXJyZWQNCg0KdG8gdGhlIGxhdGVyIHN0YWdlIHdoZW4gd2UgZW5hYmxlIFZQQ0kg
Zm9yIEFybSwgYml0IHRoaXMgY29uZmlnIG9wdGlvbiBpcyBzdGlsbA0KDQpjb25zaWRlcmVkIGFz
ICJjb21tb24gY29kZSIgYXMgdGhlIGZ1bmN0aW9uYWxpdHkgYmVpbmcgYWRkZWQNCg0KdG8gdGhl
IGNvbW1vbiBjb2RlIGlzIGp1c3QgZ2F0ZWQgd2l0aCBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQ
UE9SVC4NCg0KV2l0aCB0aGlzIGRlZmluZWQgbm93IGFuZCBub3QgbGF0ZXIgdGhlIGNvZGUgc2Vl
bXMgdG8gYmUgY29tcGxldGUgYW5kIG1vcmUgY2xlYW4NCg0KYXMgaXQgaXMgc2VlbiB3aGF0IGlz
IHRoaXMgY29uZmlndXJhdGlvbiBvcHRpb24gYW5kIGhvdyBpdCBpcyBlbmFibGVkIGFuZCB1c2Vk
IGluIHRoZQ0KDQpjb2RlLg0KDQpTbywgSSBzZWUgbm8gcHJvYmxlbSBpZiBpdCBpcyBkZWZpbmVk
IGluIHRoaXMgS2NvbmZpZyBhbmQgZXZhbHVhdGVzIHRvICJuIiBmb3IgeDg2DQoNCmFuZCBldmVu
dHVhbGx5IHdpbGwgYmUgInkiIGZvciBBcm0gd2hlbiBpdCBlbmFibGVzIEhBU19WUENJLg0KDQoN
Cj4NCj4gSmFuDQo+DQpUaGFuayB5b3UsDQoNCk9sZWtzYW5kcg0K


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:16:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:16:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199107.352971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRr-0004Ea-Lu; Wed, 29 Sep 2021 13:16:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199107.352971; Wed, 29 Sep 2021 13:16:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZRr-0004ET-IR; Wed, 29 Sep 2021 13:16:35 +0000
Received: by outflank-mailman (input) for mailman id 199107;
 Wed, 29 Sep 2021 13:16:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lrrL=OT=linux.ibm.com=ajd@srs-us1.protection.inumbo.net>)
 id 1mVZRp-0003pX-Mq
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:16:33 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.156.1])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1d2d0db6-194d-4892-9da5-1133d81ce5c6;
 Wed, 29 Sep 2021 13:16:30 +0000 (UTC)
Received: from pps.filterd (m0098410.ppops.net [127.0.0.1])
 by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TD0Tpj027476; 
 Wed, 29 Sep 2021 09:15:46 -0400
Received: from pps.reinject (localhost [127.0.0.1])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3bcrn9rbsx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 09:15:46 -0400
Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1])
 by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18TD1Rbo030294;
 Wed, 29 Sep 2021 09:15:45 -0400
Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com
 [159.122.73.70])
 by mx0a-001b2d01.pphosted.com with ESMTP id 3bcrn9rbrq-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 09:15:45 -0400
Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1])
 by ppma01fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18TDC3n0019787;
 Wed, 29 Sep 2021 13:15:42 GMT
Received: from b06cxnps4074.portsmouth.uk.ibm.com
 (d06relay11.portsmouth.uk.ibm.com [9.149.109.196])
 by ppma01fra.de.ibm.com with ESMTP id 3b9ud9xasx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 13:15:42 +0000
Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com
 [9.149.105.61])
 by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18TDFevi57213414
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 13:15:40 GMT
Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id E44C911C04C;
 Wed, 29 Sep 2021 13:15:39 +0000 (GMT)
Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 8BE3911C05E;
 Wed, 29 Sep 2021 13:15:39 +0000 (GMT)
Received: from ozlabs.au.ibm.com (unknown [9.192.253.14])
 by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP;
 Wed, 29 Sep 2021 13:15:39 +0000 (GMT)
Received: from [9.206.131.40] (unknown [9.206.131.40])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested)
 by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 7E5A9600F5;
 Wed, 29 Sep 2021 23:15:37 +1000 (AEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d2d0db6-194d-4892-9da5-1133d81ce5c6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc :
 references : from : message-id : date : mime-version : in-reply-to :
 content-type : content-transfer-encoding; s=pp1;
 bh=8WNH5WIjmi4VXGGXQVQqQGa9LxSWiuKObMKI3R3N3v0=;
 b=eTICHaTe5EWn0Iqkz8yeEUgAvudcmWSKeYOZL3tmsOVg2p/vuJCd3lLc7X4IpTJRsQvN
 N+h9f7ERDYgfvyZzbNdXaYE4xiu6SSMbi+oOoQnXmFqA0MDjRWByMV8QDZklznxhWnP8
 Zfu6Z65PrhA/glaMqoEOzOj4/+LPoWJLpdQ3z2MhMDq2F8jcszdMujO8CYHVNV9Cz7k0
 sstnbptCuAlwXwUd+5bwWpT1qWREPrnHxzZBLT02ytCWu6N5JyvTo1qOy/7Be3XDwwgp
 SBf9sCOqX6+7qHmwe/BbrU7S8zsBAly7zJlvoZTU2jK/RY4l2dgZ/h3Am2Pd3krxwQpW Gg== 
Subject: Re: [PATCH v5 10/11] PCI: Replace pci_dev::driver usage by
 pci_dev::dev.driver
To: =?UTF-8?Q?Uwe_Kleine-K=c3=b6nig?= <u.kleine-koenig@pengutronix.de>,
        Bjorn Helgaas <helgaas@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>, linux-pci@vger.kernel.org,
        kernel@pengutronix.de, Russell Currey <ruscur@russell.cc>,
        "Oliver O'Halloran" <oohall@gmail.com>,
        Michael Ellerman <mpe@ellerman.id.au>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        Paul Mackerras <paulus@samba.org>,
        Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>, Borislav Petkov <bp@alien8.de>,
        x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
        Frederic Barrat <fbarrat@linux.ibm.com>, Arnd Bergmann <arnd@arndb.de>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross
 <jgross@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
        Mathias Nyman <mathias.nyman@intel.com>, linuxppc-dev@lists.ozlabs.org,
        linux-perf-users@vger.kernel.org, xen-devel@lists.xenproject.org,
        linux-usb@vger.kernel.org
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
 <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
From: Andrew Donnellan <ajd@linux.ibm.com>
Message-ID: <75dd6d60-08b9-fa68-352c-3a0c5a04c0ab@linux.ibm.com>
Date: Wed, 29 Sep 2021 23:15:37 +1000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: hXtcU9LxcZFWJ3_AQvYkTcqzdGELwYzx
X-Proofpoint-GUID: JvUu1pq2Dv5jXGImGMo8koe6SjbY5_vk
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_05,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0
 malwarescore=0 clxscore=1011 lowpriorityscore=0 mlxlogscore=928
 spamscore=0 impostorscore=0 bulkscore=0 mlxscore=0 adultscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290081

On 29/9/21 6:53 pm, Uwe Kleine-König wrote:>   	 
list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
> -			if (afu_dev->driver && afu_dev->driver->err_handler &&
> -			    afu_dev->driver->err_handler->resume)
> -				afu_dev->driver->err_handler->resume(afu_dev);
> +			struct pci_driver *afu_drv;
> +			if (afu_dev->dev.driver &&
> +			    (afu_drv = to_pci_driver(afu_dev->dev.driver))->err_handler &&

I'm not a huge fan of assignments in if statements and if you send a v6 
I'd prefer you break it up.

Apart from that everything in the powerpc and cxl sections looks good to me.

-- 
Andrew Donnellan              OzLabs, ADL Canberra
ajd@linux.ibm.com             IBM Australia Limited


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:23:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199120.352981 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZYu-0006CU-Fb; Wed, 29 Sep 2021 13:23:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199120.352981; Wed, 29 Sep 2021 13:23:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZYu-0006CN-CK; Wed, 29 Sep 2021 13:23:52 +0000
Received: by outflank-mailman (input) for mailman id 199120;
 Wed, 29 Sep 2021 13:23:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZYs-0006CH-DW
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:23:50 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id da39c8c0-55b6-4ddc-a1dd-f1c5f49e0dd0;
 Wed, 29 Sep 2021 13:23:49 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2058.outbound.protection.outlook.com [104.47.12.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-40-W5CYHx9SN-e0rqNP3yJQpg-1; Wed, 29 Sep 2021 15:23:47 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0401MB2333.eurprd04.prod.outlook.com (2603:10a6:800:28::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:23:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:23:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P251CA0027.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:2f2::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:23:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da39c8c0-55b6-4ddc-a1dd-f1c5f49e0dd0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632921828;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LxxVHC5DI2Oe+uQaFcjF9bzlScAMMc57H+MBGVqKnZE=;
	b=eeXBb3VDYRpYkdGwvITpRKlRhJomTDWLaq9bdUWy2Oycv+rFvmpsJQ37A3hC9GExKSI1eL
	eIoaQ/XQ2EiO3fcTC85Ih8byij6D80AirVkSIe4D+NdI/nYUm/JVvye8MMYeRlWTq3CyyK
	1/Kj9QLxk1x7DZFzQZ44OnehX9lL070=
X-MC-Unique: W5CYHx9SN-e0rqNP3yJQpg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=b6LbTx+d33V/CrpqLr94URQHO1B0CsbY4MYVDCLGfwwRjaeuFqXvXSvAwW8slU+575xgldlQppkCuokmvy9P2bDNyz8puXcLFTOg1z6zrenjYMalW4LriT0S5HnPmmrDvZqEY62Qjwejm5rxo151QPSeYE1Y4MD/5LZo+89pSJIrLjbHFQSppOyu5s6e9YTTi+AqOVAISss7JnQJl3MmsPJ05V5Rsc2U3zwqMFWzKyyCueRoOwRLU4AdvCaA5+0k+WvceVYFz5u4bZRL6accXk4dtDSaF1DQUGfuxktLjwklHhDoFasVdX+RTn1d5/INp0kzbts5PjAHLJId5ZQnNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=7bWYQaG7TskQCma8sk6hdeKPJ8HJVY7LRSAjfWfTtLw=;
 b=eLDzijqvFqxr8iPR05EvVV0Hb+M/xPtz6TMKgxL0cs2B+P6VJME7tRXzDKfcfhD0c2HLlQl+D25U2NuGTI6v8c8wKmp19Y9VU93gPAkplchC4EXdQgXBHiAWYVYXqONPjQoohOBvy8FPiEkqw2loxIwtacYwCtjkID8l00AL35vHCarRpC0wSQlHvKEAEpE0/8qH9zpxheAtFs3eMVTJ2w5UUHPP48zG1h/ANGEfOwLCMMrzzFweabRR/Wim7lVZckHyHKQa6rKsvqgIiT+p03IFQzVIP4P12NfUe7GOm+o7EoemUZ7XRzG3vGWBKjlmBNY7zsmCpecwMdIeCX9kNw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Michal Orzel <michal.orzel@arm.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
 <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
 <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <548561a6-f878-cacc-cda2-30508a43072d@suse.com>
Date: Wed, 29 Sep 2021 15:23:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AS8P251CA0027.EURP251.PROD.OUTLOOK.COM
 (2603:10a6:20b:2f2::9) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6c863af8-d35a-4185-25d9-08d9834c5cf8
X-MS-TrafficTypeDiagnostic: VI1PR0401MB2333:
X-Microsoft-Antispam-PRVS:
	<VI1PR0401MB2333D02029FBED74A3DF69AFB3A99@VI1PR0401MB2333.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	GSyFRYE6v3HRTfEgtE3WaGC2lLUaXK8uoQHZ1G5Rx9EUjZTsbZu4ThUreemrG1qlYgaMUqusmnF6wl1eaLEJt1hIHbmHCLz1AYx4fQaGcvynpjmaMVfNrxuuL3ajz5fs9K3T3ghjWX7jDaasZM8+SJa9fMIbv4oD+IQ4HGGNE1/jHUI5ZWzwZiTqiQ2iZzhpFBM+mAJOR+LjNz1Ig0WWNZqETqk0oMYVIZg7+AEduiQacBOzqKWEIKFa2EaoR5cGgbphXsxF5jhiZjR39lKsotffUwXuu73AySY04Ii+OIoJrk9ZCNLHVyrQUwJT9Zjg3QeQf0vZpsqDF+wZeRyVmzG7vll4ivqD7ZwKSdYkU/JxbsqsDDuj9QifFkPKJ/uJhCrH7huq/l9DUxc9pIfvjW9DsOkIgrdIJUlMQAEXoWDqjYSAMS9qDhcrjx58FziRXXom8yXKR0hUB3L1wu5C/XIRuwjY3o8tixgZ+S4xG0rf6njZSVQh8xFVMVE85ryjG2nBJF/CZpGe/kS2B6qyWZdEqlgFOGwHp1exRccJCMkYkSvhfqF2kCm2kD/FnzCg4GtvVtfAWPq5ZAP4+Y1dans3UyA8b8spLuqKMY/ZnWwvNV75HWH1COdEaIDZx4uXF4RfdhxgVoWsrKlo36cV9WdwtPXZkS9Wgcdd15zjzRUVlHJiav/sfM9hBuUbn/EqnZULmZH6mBS2q68t3uQoFoCg7/vSYEyJSNsRJ4SUeck=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(26005)(508600001)(8676002)(83380400001)(38100700002)(53546011)(4326008)(956004)(2616005)(31686004)(5660300002)(54906003)(16576012)(66556008)(66476007)(66946007)(8936002)(86362001)(7416002)(6916009)(316002)(6486002)(31696002)(2906002)(36756003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?kgOLTnNfpVKXDsn+/JmdIBgo7zAev71sKG8S7f4v/mBtjkP9XnUKTopgJhLh?=
 =?us-ascii?Q?puqD+3x8cB95MYivVGCx2MUn6JTzH0ntK7mh3UHShYBpBUXURh7zhkjhGy3Q?=
 =?us-ascii?Q?ZaDnvrndlM1cciO9Pmlb8BFAuZhjb//OpnLGh+Bw8/AYyy9InCse5XkA12v2?=
 =?us-ascii?Q?4r9VWNhD38lPWeZjg5a5dxz7YPif2KZw9qZTsb0MNlTqAbrQkiabO0A7/DRt?=
 =?us-ascii?Q?n26oc9bY7nE4Uxm5yhtQ3puuiDjIE/ZsABXuoKKi03eUui2lo6uXNmD/Rof4?=
 =?us-ascii?Q?1SNwRYCECtm+Kxb/lpuwIqJ+b3eWAsfemLO5ubRDO/M4pijXazAoWSwcSO+M?=
 =?us-ascii?Q?dct8d/2M7SkTCiB54Gfwtare4qzezakez4H4K0jB1bhAxwifiMOJe8J3wkQk?=
 =?us-ascii?Q?ujj21+ZH+kAoevALE6DUmOUWRk8XGKPppcuLjKky0U5sFVidHvEpv1uNr/4P?=
 =?us-ascii?Q?/bIYYHkxKbzd5LTyhKeBs5y+UGfIU4PAIkywwVKHmIdSWJmGWHDi9UGmXz/I?=
 =?us-ascii?Q?+2lHhjS6l0xJWYoU0vBLg7nMQ/4VoUJoYYQtgHUMtPCa6DyHKJk/w79RAjTF?=
 =?us-ascii?Q?Akg0g3Jjx9+19leg1siA5U1uMGRPkJLpNXLuJzrzrXRaUj07O82LdoM8Dy1q?=
 =?us-ascii?Q?MmNFN9/+MlQ87DZRM+dcYBWK6opyaezds5QsRyUsX5SfrxESX4C9n1Lu1cjB?=
 =?us-ascii?Q?h1mN1heGpL4zHmKj9k7KI6oBN6VR0Tp2uyQ//zlw1CLWTTSii8ET3QgmfynM?=
 =?us-ascii?Q?z5Fq/uSVxuQmBa/p0ZZM0CBMR+sW4lYWC0ztzOasK+Z5I0V1DBIEuh1LxXmw?=
 =?us-ascii?Q?URWx0q2VbbZ6WUoeDgUgDQ89V/2ZgSKPIpEsJTLKSglohpAIV5znvot5bJLn?=
 =?us-ascii?Q?pagLWSEZGoI15TCwun/BEpZVnBc4bT828yQyYz7+9bP3YaET9Ywastx3o9bT?=
 =?us-ascii?Q?wUgzcpvZ1URnmIkCpuaBfs2S50vC5lGVBNkqhv/T4iJSrZ6YT5mSNSvIZ9dH?=
 =?us-ascii?Q?Tr7ywxd2dELs2gWrtHnEStZo+O+BSJoTkqWz+TAtHR8bwMYsRvSs+7WY+08k?=
 =?us-ascii?Q?fFjLpg3hbbKp1/7kFGkpGG0YGgi55FJgsGtwkuv9uwDpjtkZzaHFn1WFVVLg?=
 =?us-ascii?Q?eTzADsuNbeubkf2+vPDkYhK9S9HyDYfSvQg8oQjCV4F1tLhQ1Tcn3N+3Xc1y?=
 =?us-ascii?Q?2ThCaeooLpTKcXeAwcZrl78aCvHPg7Lu3wzbwx7EHH7jBcD3cDOze7MjgjGa?=
 =?us-ascii?Q?2kuQ6ymxR8H/J+PZ46oOKr3SH1D3SAmNXZymjc8bV3MopkTZH6PwMXMZbTcR?=
 =?us-ascii?Q?AqCCvAq2P4EuzUEHvH4SgBEA?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6c863af8-d35a-4185-25d9-08d9834c5cf8
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:23:45.1454
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: V4So5Z465Az18rgKRJDPwSkLyYHwTIpNPSIqQi7X+yUNnNCiIHpqTfVOwT1a4im41wzjjWaZvwC5xTSmkqGIoA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2333

On 29.09.2021 15:16, Oleksandr Andrushchenko wrote:
>=20
> On 29.09.21 15:54, Jan Beulich wrote:
>> On 29.09.2021 13:56, Oleksandr Andrushchenko wrote:
>>> On 29.09.21 12:09, Jan Beulich wrote:
>>>> On 29.09.2021 11:03, Oleksandr Andrushchenko wrote:
>>>>> Sorry for top posting, but this is a general question on this patch/f=
unctionality.
>>>>>
>>>>> Do you see we need to gate all this with CONFIG_HAS_VPCI_GUEST_SUPPOR=
T
>>>>> as this renders in somewhat dead code for x86 for now? I do think thi=
s still
>>>>> needs to be in the common code though.
>>>> I agree it wants to live in common code, but I'd still like the code t=
o
>>>> not bloat x86 binaries. Hence yes, I think there want to be
>>>> "if ( !IS_ENABLED() )" early bailout paths or, whenever this isn't
>>>> possible without breaking the build, respective #ifdef-s.
>>> Then it needs to be defined as (xen/drivers/Kconfig):
>>>
>>> config HAS_VPCI_GUEST_SUPPORT
>>>   =C2=A0=C2=A0=C2=A0 # vPCI guest support is only enabled for Arm now
>>>   =C2=A0=C2=A0=C2=A0 def_bool y if ARM
>>>   =C2=A0=C2=A0=C2=A0 depends on HAS_VPCI
>>>
>>> Because it needs to be defined as "y" for Arm with vPCI support.
>>>
>>> Otherwise it breaks the PCI passthrough feature, e.g. it compiles,
>>>
>>> but the resulting binary behaves wrong.
>>>
>>> Do you see this as an acceptable solution?
>> Like all (or at least the majority) of other HAS_*, it ought to be
>> "select"-ed (by arm/Kconfig). Is there a reason this isn't possible?
>> (I don't mind the "depends on", as long as it's clear that it exists
>> solely to allow kconfig to warn about bogus "select"s.)
>=20
> There is yet no Kconfig exists (even for Arm) that enables HAS_VPCI,
>=20
> thus I can't do that at the moment even if I want to. Yes, this can be de=
ferred
>=20
> to the later stage when we enable VPCI for Arm, bit this config option is=
 still
>=20
> considered as "common code" as the functionality being added
>=20
> to the common code is just gated with CONFIG_HAS_VPCI_GUEST_SUPPORT.
>=20
> With this defined now and not later the code seems to be complete and mor=
e clean
>=20
> as it is seen what is this configuration option and how it is enabled and=
 used in the
>=20
> code.
>=20
> So, I see no problem if it is defined in this Kconfig and evaluates to "n=
" for x86
>=20
> and eventually will be "y" for Arm when it enables HAS_VPCI.

I'm afraid I don't view this as a reply reflecting that you have
understood what I'm asking for. The primary request is for there to
not be "def_bool y" but just "bool" accompanied by a "select" in
Arm's Kconfig. If HAS_VPCI doesn't exist as an option yet, simply
omit the (questionable) "depends on".

Jan

PS: The more replies I get from you, the more annoying I find the
insertion of blank lines between every two lines of text. Blank
lines are usually used to separate paragraphs. If it is your mail
program which inserts these, can you please try to do something
about this? Thanks.



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:32:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:32:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199128.352992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZhA-0007pE-Cf; Wed, 29 Sep 2021 13:32:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199128.352992; Wed, 29 Sep 2021 13:32:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZhA-0007p7-9l; Wed, 29 Sep 2021 13:32:24 +0000
Received: by outflank-mailman (input) for mailman id 199128;
 Wed, 29 Sep 2021 13:32:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVZh9-0007p1-1Z
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:32:23 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0d132a86-2299-45bb-b07c-f0a890d0bd82;
 Wed, 29 Sep 2021 13:32:21 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-3-41nuvAm4MKqCPRVcxVXYeA-1; Wed, 29 Sep 2021 15:32:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2701.eurprd04.prod.outlook.com (2603:10a6:800:af::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 13:32:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 13:32:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0199.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:32:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d132a86-2299-45bb-b07c-f0a890d0bd82
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632922340;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oHMBgiVs+96mMllAD1yAUgzCmsNXT083ycPIKU3HohE=;
	b=Ck0Y+VBO00ejInwQEbqF136YHsb1Of8izY00YJtnP+ImNM5CG+QmRZlaorr+Yv9YiEJuDR
	ONP5L3FaV9dDsD9uDOOY0dSDXuxS/uth4lLs60VbN8NE8/i+wYiAWoevSQsrLbtudIq+Ct
	+LjMaKEX/8gYgEircrlYjOewEuvFcY0=
X-MC-Unique: 41nuvAm4MKqCPRVcxVXYeA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fPLkKJXAwuoLq1mmNASECZnzC1P+gKIG4zOs/a+5ofMQxjkfqrQvAsPcseG9Eef9h/FInEqW7V+XdU9gUz87iCLNKp3zOS1lW4tP95Um/wEvj7QjFqeWiQjxYPu6XGYeVx690uDj+3DuTwFh4S1U2/YXrsONdCnCA7P31LrZ3sK4S1xomGjjT+fqSHBTvE7z9yDN4aiR3CP0A5xN9r1f5qcifY4IkjAkY81eE4MU0Q7ZPwtc9PtM8xJG7yVHyTtPGM2NY3Au2PiGATEsEPslGEzk6144TbZgvWYm9dxp4w3GYegLr9dBAhXyTEMyzVM1gCjBRiJXXpOJKRbxh8lTQA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oHMBgiVs+96mMllAD1yAUgzCmsNXT083ycPIKU3HohE=;
 b=hq7jAioj2W/mwZVIUXq3qVvW4MMBz/rtAjVPVkKcroiXoYI4ZCVMvBQ8go9VgiOqGinWlvybTpCviVq+opu7A78d+fXmYIB9jVGzO/ezTgiMCU7br7+Ry+FsEnOiJmZ6+j0jpol24JB9wFf79GfC1Q0yQ7dJR1ka9tvz54jXzXCgCwVS4NX01NsSMnqtcpu/0drwov05p4K5FkD5oR9CUTUUFzlSmmt51mr0C/tt40XsIr/FduT5ZGy5sS9Hkq8qxDKMr5dcFg7wyrRNAq9E/g/nV7msVu23P2RYEyAqrdGW7r8jslMTRFLK+sKPJVSeWRwARacj7dyk4Wh/m3vikg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
 <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
 <YVD59QVbmdVwzYQI@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <033cc499-34de-d27a-7b1b-2a0a7ce38672@suse.com>
Date: Wed, 29 Sep 2021 15:32:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YVD59QVbmdVwzYQI@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0199.eurprd04.prod.outlook.com
 (2603:10a6:20b:2f3::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 7d0101ff-0565-4a77-4f47-08d9834d8e2d
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2701:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB27016E671221CAEE36D39747B3A99@VI1PR0402MB2701.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	F4JRCpxUgVpsI5owphRY2ylPW4md65DFAvE7THloGCKdxyGeHT8s1mN2/W2li3bbmmv5Xk1iNeWjiK1kHF0rklte2SBuCoXg2kh9yDoZS+EwlMtuN/1k/76OAYb9FIzAjMM4HryBaSf/742Kgv0jsxOFbUY04zyxQbWuU14Aa1KwtitOr0i+i0L/C425RIb7FUl1eNcGWdeP+PmdK/4T1b0bUFjqni5SWJhMxw2kJzSy1C0u1eJOUgMGqGQSBiBAJEcIakL4Qcn7cv4kfOPkNG9Z4MggvGZLTmRVW0RmflLyH95ghSndySzqypqCksXgkqJzk+EqS3Tfba5k8oe1JBA6G/4HdRmmlHRzTwRaYk9cXwuXWRgpIwzWFhAt6Wd2QxxyOGR+E/Y0cSvQOQtMn6v/nmDct2KNDa6/SxulVf790FpqfVfS8FS0cXw69vefK6jR84Xv10Fm66sL7MqiEKEKuPvueLb2curVfAFRpSLEkCyUhD2/ewpPRi+Sr/L3Y02lFDRy+agruKjcT2cZIngNLBIKLEZvKKL2wBhPa4jdpbin8x87yA4Qi5U3hxSrBbLMwMgha9zRRlZ+0UE83j6tlGJWgEjo8N6j30odEsC1v76M9qOSq8ggTgos34FJmHKFHTVxn8W+JND6RP3dGI8mwarPlHbXnJSnxcFPZHVG84QSgiF0qyveTD9SlZOcSI3ouZokYJu0kJAfZquAxUS704yfvKWonRyAEj07BjU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(16576012)(26005)(38100700002)(53546011)(31686004)(66556008)(8676002)(2616005)(316002)(6486002)(186003)(2906002)(3480700007)(66946007)(4326008)(86362001)(8936002)(956004)(66476007)(31696002)(5660300002)(508600001)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Yy9DY2hzSFE5QlgwOTl2WlIxQkRsWC9EVlRmRkVYSFZxdGlqNEVnNWQ3M3RY?=
 =?utf-8?B?eFZpYjhMRUxsR0RGaE9LYUlyNEtuUEt3RUxWRHdZVUd4YmZRWnoxTmpqcWxj?=
 =?utf-8?B?R2hYZGlsNExqa0dpYWtvYWErQ3MyaEFvb2hadTQ2UGVlTk14VzhrckI1OEJ2?=
 =?utf-8?B?UXpXVDZXSm4zeUFDdEYrR1FUdGVEOHdsb1o2M2VQeVdkM2dHOVZQLzNBNnhX?=
 =?utf-8?B?Y0F1R3MxUVlYaEdZN1JRUHJUd3FVV2VWZmlrT2FTVjJHMUpzQ2ZwdHU1dkFU?=
 =?utf-8?B?RFFZK0d3Q29LODNNeFRxTDl6dEkrSEpGRTBoa1NRTnR3M09PSmdNK1BVYXR1?=
 =?utf-8?B?bmFwZXNZQ3hMNnQxVTBHWmppRkxQTkE1WUJ1amhVVlNDMzd3N1YxNDRPRnds?=
 =?utf-8?B?L1JFalptSDVxYjNaNzlQUmt5Rll0bVlYNEE5MzZTSVFYKzF0LzAyemxKM1pp?=
 =?utf-8?B?UmFreSthVlNiUEFhb1RMd0IvVGxaQ3JxN0c4dGxwdENlbDdUMGpmc1RLTXlz?=
 =?utf-8?B?VE9mVDcyZXlrNjVvcVdFbDlsUGg5d2d2eklhNEJnUmZnalRhOHJIdHZBWkxp?=
 =?utf-8?B?a0cra2tTeTRSVHozVUlpTjNhZ3RaWHpvVy9paEhnRUpHekEzS2NWTXgxSE53?=
 =?utf-8?B?ODdRSnZWT1g1SHVpK21BT05LOEc4a3dHdVlvYXBHL040RkxVcjBWTW1Ia0pT?=
 =?utf-8?B?bm11N1Fac1dRYmlvKzdZbkdWU2ZmWkZXUE93TDFLTGhnSXptM0wxb0l1eVVE?=
 =?utf-8?B?eUJCSDRFa1I0WGd2dmpyZklVamN6SUhrcFI2eERhRUJuT1d5Z0FmaHRzNUxZ?=
 =?utf-8?B?a3MyVVRSRWw4ZDRXUmtZSEwrd2dMQ0pNYVRHVktjWkZab1lORDc1OFhJVVFK?=
 =?utf-8?B?amZBNHBLdTVLYzBwYmdiUHAyV2ltVURyK1FVQnNPczJOckpQdmsxZDlaaGRz?=
 =?utf-8?B?QmhiWERLYmpJeDhVdkgvbnV4TXVQWG1yTVQxdnZvY0FadXZCMGFUZ1JobnIy?=
 =?utf-8?B?NVpodHlWcXIrRTVMc0tZTDFEelBNR3BPeGNkTEY4ZHlQb3EvdEF6M2tnUXo4?=
 =?utf-8?B?eFJIVjIxSzNmR0I1c090RHFwbjNBR1pUNVAySUFVek13dktNOW1KaWsxekhD?=
 =?utf-8?B?OTBTWDFiNzVVQjVJYWM1UDdjbGZLSE43SGxBSDhjWndrZHAxT0tRRm50MUJq?=
 =?utf-8?B?WWlmcWRSeTNWNWkyOG5FL1dXN3lYdmpqdHJTbk8rZEdQbFFWK0hlRWJkT2VK?=
 =?utf-8?B?T1BlNFN0SHMyV21jYWJ5L3dodWFxdUhoYzBPQWVVNDAzWFg2N3JKc3J1bFJY?=
 =?utf-8?B?SlhjR2tsTGJOYlg1N3hzUkpXRzVBOXJPRmJmTDVNdWt5RzBnNHZxZmd2N25C?=
 =?utf-8?B?TVNVeTR1djZNYSthbTJCbXdYZGpxYm1hMkRhWi91ZTVodW9TMDB2QjZtWVBv?=
 =?utf-8?B?NzJScmlZcWIxM3F0dEVrN0ZKQ2pLWDdMeWRLVm5JVktVcTFEQ3dyaUpXRytZ?=
 =?utf-8?B?RFhmQjNFOEJ4dTdCZXVxNGZ2OFRxbU1wYTlieU5FVjZMR2NUdEZHSTFBMEtI?=
 =?utf-8?B?TjRjZkJ2MzAzNzU0c1hRQ0krS2c4SVBGUHY2Yy9Xd2R0ZDBJRWY3Sk1lRFJa?=
 =?utf-8?B?VXpjWmM0TFdhUThDemtLTzhTLzEreStPM0FSMi90Q1JlY3JYaFlMTkE2V1JS?=
 =?utf-8?B?Mk9FZm4vaGpkZnFBRGIwTnY4K3BzWDhXaEh2SGdKd0NmdnI0bUJYM0J1V0lq?=
 =?utf-8?Q?4oq+RXP7Ca3JFrRJ9uCB7nnYD228wTFmU/cqvxy?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d0101ff-0565-4a77-4f47-08d9834d8e2d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:32:17.3788
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UZwONhIVppji53T5njpa7ubqLv/gmJlWk70JtpC9pDRIh/dsaaTCkG8YgmiiHCLZQO4WmFtjmZH99DXl+Gafqg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2701

On 27.09.2021 00:53, Elliott Mitchell wrote:
> Getting everything right to recreate is rather inexact.  Having an
> equivalent of `sysctl` to turn on the serial console while running might
> be handy...
> 
> Luckily get things together and...

Thanks; finally got around to look at this in at least slightly more
detail.

> (XEN) mm locking order violation: 48 > 16
> (XEN) Xen BUG at mm-locks.h:82
> (XEN) ----[ Xen-4.14.3  x86_64  debug=n   Not tainted ]----
> (XEN) CPU:    2
> (XEN) RIP:    e008:[<ffff82d0402e8be0>] arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d1v0)
> (XEN) rax: ffff83080b2f106c   rbx: ffff83081da0f2d0   rcx: 0000000000000000
> (XEN) rdx: ffff83080b27ffff   rsi: 000000000000000a   rdi: ffff82d040469738
> (XEN) rbp: ffff82d040580688   rsp: ffff83080b27f8b0   r8:  0000000000000002
> (XEN) r9:  0000000000008000   r10: ffff82d04058f381   r11: ffff82d040375100
> (XEN) r12: ffff82d040580688   r13: ffff83080b27ffff   r14: ffff83081ddf6000
> (XEN) r15: 00000000004f8c00   cr0: 000000008005003b   cr4: 00000000000406e0
> (XEN) cr3: 000000081dee6000   cr2: 0000000000000000
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0010   gs: 0010   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d0402e8be0> (arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260):
> (XEN)  e3 0c 00 e8 30 7f f6 ff <0f> 0b 66 0f 1f 44 00 00 42 8b 34 20 48 8d 3d 8d
> (XEN) Xen stack trace from rsp=ffff83080b27f8b0:
> [...]
> (XEN) Xen call trace:
> (XEN)    [<ffff82d0402e8be0>] R arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
> (XEN)    [<ffff82d0402ec51c>] S p2m_flush_nestedp2m+0x1c/0x30
> (XEN)    [<ffff82d0402e0528>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x378/0x490

hap_write_p2m_entry() calling p2m_flush_nestedp2m() suggests that
nestedhvm_enabled() was true for the domain. While we will want to
fix this, nested virt is experimental (even in current staging),
and hence there at least is no security concern.

Can you confirm that by leaving nested off you don't run into this
(or a similar) issue?

Of course you not having done this with a debug build (and frame
pointers in particular) leaves a level of uncertainty, i.e. the
real call chain may have been different from what this call trace
suggests.

Jan

> (XEN)    [<ffff82d0402f009a>] S arch/x86/mm/p2m-pt.c#p2m_next_level.constprop.10+0x24a/0x2e0
> (XEN)    [<ffff82d0402f1097>] S arch/x86/mm/p2m-pt.c#p2m_pt_set_entry+0x3c7/0x7b0
> (XEN)    [<ffff82d0402ea0a6>] S p2m_set_entry+0xa6/0x130
> (XEN)    [<ffff82d0402f4ecd>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check+0x1cd/0x440
> (XEN)    [<ffff82d0402f023f>] S arch/x86/mm/p2m-pt.c#do_recalc+0x10f/0x470
> (XEN)    [<ffff82d0402f02ed>] S arch/x86/mm/p2m-pt.c#do_recalc+0x1bd/0x470
> (XEN)    [<ffff82d0402f00ed>] S arch/x86/mm/p2m-pt.c#p2m_next_level.constprop.10+0x29d/0x2e0
> (XEN)    [<ffff82d0402e03da>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x22a/0x490
> (XEN)    [<ffff82d0402f0fe2>] S arch/x86/mm/p2m-pt.c#p2m_pt_set_entry+0x312/0x7b0
> (XEN)    [<ffff82d0402f0c4e>] S arch/x86/mm/p2m-pt.c#p2m_pt_get_entry+0x3fe/0x480
> (XEN)    [<ffff82d0402f59aa>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check_superpage+0x17a/0x600
> (XEN)    [<ffff82d0402f5ba0>] S arch/x86/mm/p2m-pod.c#p2m_pod_zero_check_superpage+0x370/0x600
> (XEN)    [<ffff82d0402f7c78>] S p2m_pod_demand_populate+0x6b8/0xa90
> (XEN)    [<ffff82d0402f0aa6>] S arch/x86/mm/p2m-pt.c#p2m_pt_get_entry+0x256/0x480
> (XEN)    [<ffff82d0402e9a1f>] S __get_gfn_type_access+0x6f/0x130
> (XEN)    [<ffff82d0402ab12b>] S hvm_hap_nested_page_fault+0xeb/0x760
> (XEN)    [<ffff82d04028c87e>] S svm_asm_do_resume+0x12e/0x164
> (XEN)    [<ffff82d04028c87e>] S svm_asm_do_resume+0x12e/0x164
> 
> The stack trace goes further, but I suspect the rest would be overkill.
> That seems to readily qualify as "Xen bug".
> 
> 



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:36:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:36:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199135.353004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZl5-0008TN-UH; Wed, 29 Sep 2021 13:36:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199135.353004; Wed, 29 Sep 2021 13:36:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZl5-0008TG-R4; Wed, 29 Sep 2021 13:36:27 +0000
Received: by outflank-mailman (input) for mailman id 199135;
 Wed, 29 Sep 2021 13:36:26 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVZl4-0008TA-HA
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:36:26 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0e::61a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5a13a848-2b61-410b-9a02-3ef62f575330;
 Wed, 29 Sep 2021 13:36:24 +0000 (UTC)
Received: from AS9PR06CA0071.eurprd06.prod.outlook.com (2603:10a6:20b:464::22)
 by PAXPR08MB6880.eurprd08.prod.outlook.com (2603:10a6:102:13d::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 13:36:13 +0000
Received: from AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:464:cafe::d9) by AS9PR06CA0071.outlook.office365.com
 (2603:10a6:20b:464::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:36:13 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT021.mail.protection.outlook.com (10.152.16.105) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:36:13 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Wed, 29 Sep 2021 13:36:12 +0000
Received: from cbfb6b425cef.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8C09195E-60A6-4CF4-BBEE-A527731CA7BC.1; 
 Wed, 29 Sep 2021 13:36:02 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cbfb6b425cef.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 13:36:01 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6096.eurprd08.prod.outlook.com (2603:10a6:102:eb::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 13:36:00 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 13:36:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a13a848-2b61-410b-9a02-3ef62f575330
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oUdQTf4RQFZ+vYRUAEgBYR/Ty0P7fxukhQDG5LwOMZE=;
 b=xzfb0xCZl7FgBQTEkRw6vblaSpGP+9T3yiLIRQxz/5C/V61iarqXWnxV4Digr3rk15CBj8BRcP4GU3zak3pRO99McPT8qelUr888IZXGKx14Xz3pxjjnNgJTw1qWRWkSW9QAe8l79/nw4LlDOOI51XbQY1D+Vnx/4c+Yso4/7Ag=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: fbcf46bb206553d5
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f8Q74y7Ndje+FrdOzaHaco/aLzTXcU7yoOfgZBIMky71LDMr0z0Fsggza+X14/u5aSFoqU2HDJxhxpb6rDIWBFS+uiixKIPMVAWTvw0lBloVvEYyStJgxnxWn1mEeJGJiKGL6LOl+8lXdhM3WFRAcrRgyOyINFNEj4QIoQn/+2BkqJyK7SPwrKX2hsYLBP0FT4lUru+I/fP97hC+/cd40tcQxuqiN+Ej9sCTx5iZzVazH66DCP4nj1gmDyDghVCLjlZsf3VlA7J+WY1uI7kzaGhvUhOe4ipjAXFuHpwwlnXTBgDaauXn8jqKJxWvHqf7icuF+sFLrah/okTxi7sx2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oUdQTf4RQFZ+vYRUAEgBYR/Ty0P7fxukhQDG5LwOMZE=;
 b=HflqfEnTwDMRB5MWMaK59OCB2gfhmFUUpQ5mrcmhOFDiatvoEMOAeW83CfyCo2gqx17YLGOmYYdMz55ZsnKBI9gOriU60O8xPP8PHEwrWWjgEwTuLRXoOiqxmSsaesTECXyjtNR5M0awaxQihw3fkaoit0p23ir7hgpjwDYkZ04urplW+JwRGKYOJHu85UGWu6aKWsam2xnleem36hB2YhV5fsTCRESjyYA/+WY3ZEz0m9uNqAEPhNa3LQIS6GdTAnNsXMrhaGWj54ZLw+BnRcf/Ill9FN4GLrpHyjPIelgAM8wDUktPXToc96B3QAlFf+MroEjk3zI+F74/IfuhOg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oUdQTf4RQFZ+vYRUAEgBYR/Ty0P7fxukhQDG5LwOMZE=;
 b=xzfb0xCZl7FgBQTEkRw6vblaSpGP+9T3yiLIRQxz/5C/V61iarqXWnxV4Digr3rk15CBj8BRcP4GU3zak3pRO99McPT8qelUr888IZXGKx14Xz3pxjjnNgJTw1qWRWkSW9QAe8l79/nw4LlDOOI51XbQY1D+Vnx/4c+Yso4/7Ag=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 13/17] xen/arm: Implement pci access functions
Thread-Topic: [PATCH v3 13/17] xen/arm: Implement pci access functions
Thread-Index: AQHXtJXigQn0BAihyEq+PFitR4jFoqu7BLSA
Date: Wed, 29 Sep 2021 13:35:59 +0000
Message-ID: <92C54E23-582B-4DA6-87C3-DAC1041ABA7B@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <33cc6b7a133787700ea8ba4e54a03141d3aca1ea.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <33cc6b7a133787700ea8ba4e54a03141d3aca1ea.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 4535d10b-64cb-4c26-a34a-08d9834e1b03
x-ms-traffictypediagnostic: PA4PR08MB6096:|PAXPR08MB6880:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PAXPR08MB6880BE5CB7A1A517773952429DA99@PAXPR08MB6880.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 KQPi+Bj3SIx3qbGCyypgqf93GUQAyNha9mjRPJH/E2/jdUF1cBRAeYHbZw1qwfwIKOoLMQlK+aGdpd/tSm34grOeX26nxjh25B9GMohIE3mRYwhxQzDfItugxnYaXHN+Ic0eKhjdMbKHKV6ewV5UZiY4QjXjYt18ArVaAoy6uBmuAd4vz0n7IhgRVa481O8nCWr19Q4srDlE6Tz+Kdnbdr+Yj/OGHhr6rWY+jrP5UFrTnVyDawGjdiya5nyM0tZcUns8jjgvtOuYlCX+6cjDUkM9loBRB5B8AvM4mlOyo0P7eJ/olr0qIJZDZQ8mQwA8yxwzlbi0BUl+PH2lPRre64OFyqwlPXvGtyDoeU+2Mqxx17b7wOjkJuT0ryGxfEAOX4/7+gjbrVqZsOmHjtf9olLA57SaarYc3tqsPhXmR2QeZj9ebxchgfW8smnS0xv5RAXQG/wm/FWSZxleKR1JymcM5hLJKHHiGIM0oayikB0GpnngAjkqmOBhiTH9JEyLpC6ipm62Ru/ov4qhztPLjphw3NHUK7jZKcVkCrVwUFTa8zWk1V2UtKcW2Ex4iWm80fQ+VeiNHBXNx121mHTDQKJRi2v+EqlZpV69los7LVKowpHg1a0lPGAfmUpilbJmsubr86MXAMP1SaoP9X8+7+xqtZAcW5+Q2F4jjQtpVI8vJsRT5cx6xz/amsJ+cMVCmBFUMfGlOQzg1XxNnTY2OyHm8mkaju/M17bmZTOIQS3RLfXIsaxCiR+JzRvlVM74
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6636002)(54906003)(2906002)(66446008)(5660300002)(53546011)(6506007)(91956017)(64756008)(66946007)(66556008)(66476007)(26005)(76116006)(186003)(2616005)(71200400001)(38100700002)(37006003)(36756003)(6512007)(122000001)(316002)(8936002)(33656002)(4326008)(6862004)(86362001)(8676002)(508600001)(38070700005)(6486002)(83380400001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <9C4A06334DF3044697413927AF293B8A@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6096
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8b6e492d-1476-42e0-12ca-08d9834e1312
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ePTbsTcCR4F/tCftxqXc5f7ue3/VDyJYwVo2+5MOBOmJbvNYk2paQsvFCLOIHXb0vSSA6Ku36bi4rqSHLq5JDxs7xs5EIxrT1q6fQJfJsjlkiCKfut0D1rtS1Lsfsvd2yqQPTBBy+Gdc4AKm5wIM3fBisQ4B6E3ud0qOUcKIZ544ieM2ffhWrAYy9j0B3UER8CPtgiGpRxgMqroukve6XJSrqrB618Vys2IX4dT6eI4q1Cdw4VEOkxpFW8xTHkqZwkBKxq6h6TT3BT8lsfzH+I87f5rZaChNmz40LylOeK1cYOj8ptqR1ItsVBqQFSsHnv+M57v6JNiL2jWo8FU8OJBabp8zC6LZdSFThoKmASmwdWlJx/3cOIGiaU44m6oi4d/bPilwSIzWcVgJtE7ff45ugMctzT1p/npjpr90qs5cdGgf30Lu7Aje8mI9Ba3ZCtIMv7MTxhYMnOXA9wPt2m5SKbQOYj40etgvghXhPYdgC7kPthol3NHA7S9C0gSfGEOGBQZrvD9pjMexcFvbUAkcTilm3Aezc4Nf6R1j7VSGyjfEL0pq12JAHi6oekdOVmDzASvh8N1TBqQUNAZ0YifZ8xmf4dpvOwUg/GQLf9xbbEPmyYFrS+tnh5EaUiO+pJaY9tmWyAxrvTiS9Go0QFMdm1+G1bs8dKjDrUC72LeSKmY9+8qq4rkFa9kyYtpFdUF0jRxDkXrKFcmlwtP0/A==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(4326008)(356005)(107886003)(70586007)(186003)(6862004)(86362001)(33656002)(2906002)(82310400003)(26005)(36860700001)(70206006)(36756003)(6486002)(2616005)(316002)(6506007)(47076005)(81166007)(336012)(508600001)(37006003)(8676002)(53546011)(5660300002)(6636002)(6512007)(83380400001)(54906003)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:36:13.2737
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4535d10b-64cb-4c26-a34a-08d9834e1b03
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6880

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> Implement generic pci access functions to read/write the configuration
> space.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Remove PRI_pci as not used.
> - Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
> Change in v2: Fixed comments
> ---
> xen/arch/arm/pci/pci-access.c      | 57 ++++++++++++++++++++++++++++++
> xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
> xen/include/asm-arm/pci.h          |  1 +
> 3 files changed, 77 insertions(+)
>=20
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.=
c
> index 3cd14a4b87..9f9aac43d7 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -16,6 +16,7 @@
> #include <asm/io.h>
>=20
> #define INVALID_VALUE (~0U)
> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>=20
> int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sb=
df,
>                             uint32_t reg, uint32_t len, uint32_t *value)
> @@ -72,6 +73,62 @@ int pci_generic_config_write(struct pci_host_bridge *b=
ridge, pci_sbdf_t sbdf,
>     return 0;
> }
>=20
> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
> +                                unsigned int len)
> +{
> +    uint32_t val =3D PCI_ERR_VALUE(len);
> +    struct pci_host_bridge *bridge =3D pci_find_host_bridge(sbdf.seg, sb=
df.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return val;
> +
> +    if ( unlikely(!bridge->ops->read) )
> +        return val;
> +
> +    bridge->ops->read(bridge, sbdf, reg, len, &val);
> +
> +    return val;
> +}
> +
> +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
> +                             unsigned int len, uint32_t val)
> +{
> +    struct pci_host_bridge *bridge =3D pci_find_host_bridge(sbdf.seg, sb=
df.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return;
> +
> +    if ( unlikely(!bridge->ops->write) )
> +        return;
> +
> +    bridge->ops->write(bridge, sbdf, reg, len, val);
> +}
> +
> +/*
> + * Wrappers for all PCI configuration access functions.
> + */
> +
> +#define PCI_OP_WRITE(size, type)                            \
> +    void pci_conf_write##size(pci_sbdf_t sbdf,              \
> +                              unsigned int reg, type val)   \
> +{                                                           \
> +    pci_config_write(sbdf, reg, size / 8, val);             \
> +}
> +
> +#define PCI_OP_READ(size, type)                             \
> +    type pci_conf_read##size(pci_sbdf_t sbdf,               \
> +                              unsigned int reg)             \
> +{                                                           \
> +    return pci_config_read(sbdf, reg, size / 8);            \
> +}
> +
> +PCI_OP_READ(8, uint8_t)
> +PCI_OP_READ(16, uint16_t)
> +PCI_OP_READ(32, uint32_t)
> +PCI_OP_WRITE(8, uint8_t)
> +PCI_OP_WRITE(16, uint16_t)
> +PCI_OP_WRITE(32, uint32_t)
> +
> /*
>  * Local variables:
>  * mode: C
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-ho=
st-common.c
> index a08e06cea1..c5941b10e9 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -236,6 +236,25 @@ err_exit:
>     return err;
> }
>=20
> +/*
> + * This function will lookup an hostbridge based on the segment and bus
> + * number.
> + */
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t b=
us)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->segment !=3D segment )
> +            continue;
> +        if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn=
_end) )
> +            continue;
> +        return bridge;
> +    }
> +
> +    return NULL;
> +}
> /*
>  * Local variables:
>  * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index bb7eda6705..49c9622902 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -81,6 +81,7 @@ int pci_generic_config_write(struct pci_host_bridge *br=
idge, pci_sbdf_t sbdf,
>                              uint32_t reg, uint32_t len, uint32_t value);
> void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                pci_sbdf_t sbdf, uint32_t where);
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t b=
us);
>=20
> static always_inline bool is_pci_passthrough_enabled(void)
> {
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:39:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:39:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199141.353015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZna-0000rg-Do; Wed, 29 Sep 2021 13:39:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199141.353015; Wed, 29 Sep 2021 13:39:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZna-0000rZ-9s; Wed, 29 Sep 2021 13:39:02 +0000
Received: by outflank-mailman (input) for mailman id 199141;
 Wed, 29 Sep 2021 13:39:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVZnY-0000rN-Dp
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:39:00 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com (unknown
 [40.107.3.57]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 835dd1c3-6b61-440e-9754-013d0a52bd2c;
 Wed, 29 Sep 2021 13:38:58 +0000 (UTC)
Received: from AS8PR05CA0029.eurprd05.prod.outlook.com (2603:10a6:20b:311::34)
 by AM6PR08MB3510.eurprd08.prod.outlook.com (2603:10a6:20b:48::30)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 13:38:52 +0000
Received: from AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:311:cafe::7b) by AS8PR05CA0029.outlook.office365.com
 (2603:10a6:20b:311::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:38:52 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by AM5EUR03FT048.mail.protection.outlook.com (10.152.17.177) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:38:51 +0000
Received: ("Tessian outbound 1fd2edb24cb2:v103");
 Wed, 29 Sep 2021 13:38:49 +0000
Received: from 3149e8527a7e.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F7A06962-3672-4EF0-9B6D-A51A16166D38.1; 
 Wed, 29 Sep 2021 13:38:38 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3149e8527a7e.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 13:38:38 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6175.eurprd08.prod.outlook.com (2603:10a6:102:eb::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 13:38:37 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 13:38:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 835dd1c3-6b61-440e-9754-013d0a52bd2c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TO0Txn4BRsm8uZRPy1N5W+Z3LVYl95tktEvtCP7jog0=;
 b=8auhBPQ9sK4ghnaIiNz8/Std0pU4tEIs70DuZ6BUp3YZcLzd4j5s/zF0OSTAbWO4B8XoxKwCIeDclZalcS69bwrN8k5zlI0Q79iIRfy34UIJi4POUX+Zo1v3zfO008J5tVhCBGMmXFyWQBcojfDoPuVEasUQqxpJvLkPwmVlJMg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 2118c24ba1a05354
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gbhdshO9k9fLcbN2FWP+OiTWltjCcUwM59tT7fcbD1uTRD3DC8ML0X8fIAOlIYOSw44VXl/ejqS3iXfQHGbWD0jFeUSTdWr5WA8DL0BtJGE15Dpbzh7+GCXmzCBvRcQqYT6xIAFa5dXeuUqk4bmQ8/bIAxwD4jYIvygpyDCOXRALXx3bMZrS0hfHR9oLVX8wWeK1ttSmwLVj0jQ5CCzaP4TSFgIu/qMgRSo/Tt/BS5TIac3JhX1DO/PT1ckcCrxqsjbeYlzgv20Bu759UDuPyGI6bzvkNIDpA+Mr6G+OF3BElKwR+OAM2bxHmTR2lJQzuFMz/16BzZwOV7V2ZgExeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TO0Txn4BRsm8uZRPy1N5W+Z3LVYl95tktEvtCP7jog0=;
 b=I56pBUzOXMiqov1c4j6YD/uLyKS8ZI/MlsBKsaixlBvuMNO406i4H0tynDNjqYrGghEStvMFWXF1hgfW2+8Pknld1UxQDad9XvGf6P5NhGyAJ1h6/S3g3do2WKz/0Kj1K3S2cbaX0IywzDVgTdnKpozIu0s4QOKXqOXTx7MPeb53NSBBSpanMQ92H/9fbOSNdJSdhu1jXD1XEFJAmzOaC9y6LJl5HfZbusHt6quGW2YHiNqFOCv2MKAJyDJ+QPCQhaCv68ZPCZ3ms49xR/L8y+iNTYJqwoXXC673M+JU0IARDhLj4JbEko5OQDJPRri5s67NtTxfKT7miANpxbWYXg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TO0Txn4BRsm8uZRPy1N5W+Z3LVYl95tktEvtCP7jog0=;
 b=8auhBPQ9sK4ghnaIiNz8/Std0pU4tEIs70DuZ6BUp3YZcLzd4j5s/zF0OSTAbWO4B8XoxKwCIeDclZalcS69bwrN8k5zlI0Q79iIRfy34UIJi4POUX+Zo1v3zfO008J5tVhCBGMmXFyWQBcojfDoPuVEasUQqxpJvLkPwmVlJMg=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Oleksandr Andrushchenko
	<oleksandr_andrushchenko@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXtJXiCWYoQi1XqkmiV/9wfpLCyKu7BW8A
Date: Wed, 29 Sep 2021 13:38:37 +0000
Message-ID: <02F22926-2A4A-4301-A975-4BB2EFD5653F@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 93d5e7cf-388c-49b5-0c6c-08d9834e7945
x-ms-traffictypediagnostic: PA4PR08MB6175:|AM6PR08MB3510:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB3510AFB893553781A3C0D1FE9DA99@AM6PR08MB3510.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 jaok1tnNMo0F73AGiGOYhAlL8tSF2OllJQCgAdEaTz1aUDStsK0gYSACF5aCLvvUf7B2hTo9nXm5RvNqiXFTg8+JYGEMYyTJ0ho3DUOjY06cgyNCwT+2pG4sWW+ilpRTkfTThPt0gV6GFeLYMeje/ZxzBjLotmaQfap1NeOeO6OVjPohtRxlh/Z7SvMc58xvnv9k2sQtvMd4zodM5+tEHmuuV22hFzkOJ9cboKuqy6a0hqHlve6oKe6bmLQEfhnJP7HH+nrqzEE+ZZubBurvaGEW5d+aGXwwhLoDN5vfPu6h6+bJfUG7APnad3pr/L7qcOI3Y0zccmFteuqSgXWlDe+5+4oOTYq6KTz8JjPPwDQkQdSIm/heZ2qAvE890lFCSYJKkiQjDxVYrx9/hvONQfbT+NK5uew1lfFxaMaIahC/kzXZspQPte+svGRaHidlSYrHUzjg7vE9nIHuZyDQm9T6EiTXr8TF4BdcgMDtSMEWCztbEMcV9njB9D6wlx9xJa1XkSPro/WpZwXPSA+s2uvVJS6JHCENeFoYXYwNCCTh2ELu8whC2hlY2L4Y6WdA4qmoyTzlN3G/cRQ5+7tPGqr0HKJKYYp5QB3AFGanGRox3kCpfZuN7/+qjcFNPctmH6aiRl8goRKi1DmOZXYCCMv9KQ4rOvOTk5oNXBHamHPxKfjUQEBdk0XKlUCpPT9ouamBc0V4ePkE0AJQLpsEG10dO0hiC4x/ql7oaazlPCGGkRGapWZgL5v6hvIgH8avx5Ttks4Wn2q5QlwKIpxK7R387Hf2AYjtnO2AGZ0EIEfmfMt4WgYSvifqr9e9OQ9tLjrIP9mGC7AJS0MwVKfOxtN9jSb/3JQuAkeD3+jJTCo=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(76116006)(5660300002)(66476007)(54906003)(6506007)(6862004)(91956017)(2906002)(2616005)(6486002)(122000001)(66446008)(38100700002)(66556008)(64756008)(316002)(71200400001)(6636002)(37006003)(26005)(508600001)(53546011)(186003)(8676002)(33656002)(83380400001)(8936002)(36756003)(66946007)(38070700005)(6512007)(86362001)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2900C16C4B84BC43B5A6CDB0734622E3@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6175
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b761d401-08f2-4307-9c32-08d9834e70c7
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	E0fsYqN1u0cYBb9nx7hYt+tAhYFKLkTmLcbGBWk3vpQSCRnZRNl4JqvF8RFnHG5NDj9EF0tb5NG4xLsy4sQvBfBo8cW2zq1d/NJuwZMVmm2RK64fUfzflQvPZDsG4NASv1kOvYQTI9uPPR2YZXLWnAtfSgigIL37+HrOvMckKNUNfRpsEAonr3YfvHokG3qMOwhMsYf7Eo9G39Oej8A/U7RDMM3yE91zNhjnka/5fWLoaXtifq8bozokdhmC/FbNZZHuXH6IhAjBucq7eoCwPnVvx0Kpz7G2yFwand0f8kOkctKnhLOhmxHEct/Sc3pxPsSUcMsoBSpYJe9ZbU3JKz8I6oWorWvZgGWjaIACxzB2C7nkQCusKJ6nIijrCqdC/R7PeHrzLlwqGGEv7GqWdDvdY1wn0MEvDRqEA46GWt06ZM5teHfq5HVGs8RZbGc0NN7UCyWrMyHGfcn8EXkoIsRiVQiBrF1kI/m/+1PmM/7gj/xeomum+n1kjEmZo6gyja0kWON2cRnzYeWhwQGIRr6IKzXXqew+pXtHSA7aYRcpYQPJw8tYa/G1H7RNTjLck8+D4VhcoGhOI27cqSTSNN3df9tS/6m+Sl+dpmCWJPMxpwLAi/nyYpo0nyGcixn4WIKApo4TRpXZjg1OJMbZDHTjjF7WXPB7cw2kBpVpxl5yep4JSwYD3ESeYrZvK3G62+7Bt7AYJBPc5SOb/k8CKh7GK5fc3ZuW659wUoDSZLeuud19BzRcQEKJj0XULVU+NwMdps+CJcnmvmc5nA3Ye1RpDtY4K4btIk8Npyn75Km7nO+ZhOvxof/NZwDiQqxh
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(336012)(5660300002)(83380400001)(316002)(6636002)(6862004)(70586007)(70206006)(36860700001)(4326008)(36756003)(186003)(2616005)(508600001)(81166007)(356005)(8936002)(107886003)(82310400003)(33656002)(54906003)(26005)(6506007)(86362001)(47076005)(8676002)(53546011)(37006003)(2906002)(6486002)(6512007)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:38:51.4242
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 93d5e7cf-388c-49b5-0c6c-08d9834e7945
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT048.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3510

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>=20
> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
> space to the XEN memory.
>=20
> Patch helps to understand how the generic infrastructure for PCI
> host-bridge discovery will be used for future references.
>=20
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

As you are submitting this patch from Oleksandr:
- if you modified it you should add your signed-off
- if not you might as well add your reviewed-by

Anyway:
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - nwl_cfg_reg_index(..) as static function
> - Add support for pci_host_generic_probe()=20
> Change in v2:
> - Add more info in commit msg
> ---
> xen/arch/arm/pci/Makefile          |  1 +
> xen/arch/arm/pci/pci-host-zynqmp.c | 63 ++++++++++++++++++++++++++++++
> 2 files changed, 64 insertions(+)
> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>=20
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index 6f32fbbe67..1d045ade01 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -3,3 +3,4 @@ obj-y +=3D pci-access.o
> obj-y +=3D pci-host-generic.o
> obj-y +=3D pci-host-common.o
> obj-y +=3D ecam.o
> +obj-y +=3D pci-host-zynqmp.o
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-ho=
st-zynqmp.c
> new file mode 100644
> index 0000000000..6ccbfd15c9
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -0,0 +1,63 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static int nwl_cfg_reg_index(struct dt_device_node *np)
> +{
> +    return dt_property_match_string(np, "reg-names", "cfg");
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops nwl_pcie_ops =3D {
> +    .bus_shift  =3D 20,
> +    .cfg_reg_index =3D nwl_cfg_reg_index,
> +    .pci_ops    =3D {
> +        .map_bus                =3D pci_ecam_map_bus,
> +        .read                   =3D pci_generic_config_read,
> +        .write                  =3D pci_generic_config_write,
> +    }
> +};
> +
> +static const struct dt_device_match nwl_pcie_dt_match[] =3D {
> +    { .compatible =3D "xlnx,nwl-pcie-2.11" },
> +    { },
> +};
> +
> +static int pci_host_generic_probe(struct dt_device_node *dev,
> +                                  const void *data)
> +{
> +    return pci_host_common_probe(dev, &nwl_pcie_ops);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
> +.dt_match =3D nwl_pcie_dt_match,
> +.init =3D pci_host_generic_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> --=20
> 2.17.1
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:40:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:40:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199149.353026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZof-0002EM-T0; Wed, 29 Sep 2021 13:40:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199149.353026; Wed, 29 Sep 2021 13:40:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZof-0002EF-Pu; Wed, 29 Sep 2021 13:40:09 +0000
Received: by outflank-mailman (input) for mailman id 199149;
 Wed, 29 Sep 2021 13:40:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVZof-0002E7-Du
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:40:09 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0d::601])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e5c9c11-9585-4e9d-913d-0be6e05781ae;
 Wed, 29 Sep 2021 13:40:07 +0000 (UTC)
Received: from DB3PR08CA0028.eurprd08.prod.outlook.com (2603:10a6:8::41) by
 HE1PR0802MB2540.eurprd08.prod.outlook.com (2603:10a6:3:d4::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4544.20; Wed, 29 Sep 2021 13:39:59 +0000
Received: from DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:8:0:cafe::91) by DB3PR08CA0028.outlook.office365.com
 (2603:10a6:8::41) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Wed, 29 Sep 2021 13:39:59 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT058.mail.protection.outlook.com (10.152.20.255) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:39:59 +0000
Received: ("Tessian outbound 71ebfb754289:v103");
 Wed, 29 Sep 2021 13:39:58 +0000
Received: from c1d74d1e4404.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4FE5ABBB-8FE2-46B0-9E33-68F2E351CDD9.1; 
 Wed, 29 Sep 2021 13:39:45 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c1d74d1e4404.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 13:39:45 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PA4PR08MB6175.eurprd08.prod.outlook.com (2603:10a6:102:eb::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 13:39:44 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 13:39:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e5c9c11-9585-4e9d-913d-0be6e05781ae
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v4jf3x5fJ5th9pQGNKA3lrxyQ8fD6O68usEJPC0wdXM=;
 b=GDg2c9zgT0v8LoycjdVrv2c3dy2dBXB5p2IJ8ODuImZaU5LPUgNar0shR5pjO1adgpuHxyg5Gm4NSysmHi/lKv8Su0qaYCjT252KvmVT22jN5/wSIxQvd7ZbEpR777WMio2DIxcWkDJvQjxwwNzNVBfijRR4GW/8a+Gs6tIWax8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 35bcfc5f9b5600a6
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GFRiBg1MlsMkcGi+1kET5EvXoRmuYdwL+wUIRfWdt9+umzHS6n9P+PWuiPs2qoaILvXw3wdZXpfYE9UBJFBb63h1JNbW+ZS8UiqfgeTo/h/1/cGTE4Anae+XFrjPy2sBcCaGpu1tIb9bYL36Ku12CGbp9uIJorEMwMcFeh/ir8YzhISWU9XfPBvzvCEJqg3B1dD3mrm8VfynDpWTxtdfNJUdAudEizLwItV0uqOk0kd36A2HdfZEY3mTAtrIBW+0iO1SXFAFO0XA2x9zX9EG/QjUGyJtwtJTEUIJphz8j35D9ZWkpsYi/ke/81EoqRuCm6WifclziGmC3uT1c1tQdA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=v4jf3x5fJ5th9pQGNKA3lrxyQ8fD6O68usEJPC0wdXM=;
 b=nL4joMVbZgW6MhHvq5BNHUnMdendvqWY/UX/tgrMqp4VyGZIVdYFTNWzcqkxfHpHNr/6iMTCm3tSFs8WHE4jLAANkZB5DBIE4Ju6wMenHpW4TUnCZxZrX3uqiv1dZ9WXavoXWge+DG8Qr03WPJ/q/vhBjhUlmNqWV7UDUQsnGWW+sqYRwqiWjA7Na7aK4XXepxBBwq0WtcXIDz1fXmbB340PYxsmO2WsGQ/bzixwy+Cc+C4/uctqcxUe8iXB8To3cr/XrpwtzJvWXQ4gevDshpqY2SqEZgNR2Ts/avp1HLwnCC5L0VOh+zMRN8vYFxU6eVIufjG3SjgErW1pIagZPw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v4jf3x5fJ5th9pQGNKA3lrxyQ8fD6O68usEJPC0wdXM=;
 b=GDg2c9zgT0v8LoycjdVrv2c3dy2dBXB5p2IJ8ODuImZaU5LPUgNar0shR5pjO1adgpuHxyg5Gm4NSysmHi/lKv8Su0qaYCjT252KvmVT22jN5/wSIxQvd7ZbEpR777WMio2DIxcWkDJvQjxwwNzNVBfijRR4GW/8a+Gs6tIWax8=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, =?iso-8859-1?Q?Roger_Pau_Monn=E9?=
	<roger.pau@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 15/17] xen/arm: Transitional change to build HAS_VPCI
 on ARM.
Thread-Topic: [PATCH v3 15/17] xen/arm: Transitional change to build HAS_VPCI
 on ARM.
Thread-Index: AQHXtJYGCkZuB1/ry0i7sxShepvGpqu7BcAA
Date: Wed, 29 Sep 2021 13:39:44 +0000
Message-ID: <153BBA12-C98D-4A02-A8C6-E6AA5D117C9E@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <adabef3cd1774ca51f1b409bd97a2ae353eaac2b.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <adabef3cd1774ca51f1b409bd97a2ae353eaac2b.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 83d23661-6369-4173-8be3-08d9834ea1c6
x-ms-traffictypediagnostic: PA4PR08MB6175:|HE1PR0802MB2540:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR0802MB2540F0ADFF84F92FE99F20499DA99@HE1PR0802MB2540.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 BpZ1CC9MgAKMzU/awWR6vgM2+n0qmMgWjPRXOp2fx10Erm5bQiK+47Vp/ut3WvoaNzaRJR8eOqaqIrB0JUt6Wz/0Pd1Mfkh8uHWciqde/E2+o/Bi8Ek1OjtGvJbMholXfgAaNSZ5RapZHJx8HIUfiBvVb+GuIulOzY7J5qggNcFGkN37bUBv0Oe2LaldBQ37UWTlxVJLhAH/cPPyumjJACLcZAi3r7j6SNCo965WqtDTwYJetlWL6D/VvDDz/yMXHz82oF3UE0jw5QQKThlIp+DYEr/TGGDOFwqw7j4K7aug3J/gWBTfrqucqxH3tKEjg6wRULmkAjuvO8UuTxhqqy8kei4bLbrhIgyInR6X00Mowpyx0ILND8XPKjKXCZfJCs/vcjkuRd7fjORxgRh2rPEC3czYfcueUD0zaT8F/ny6q/3JyH20t396xQEBOO63dTXs9lE/UcjJJRqZndFczsH9ma+m/D7CWinbkTYEOLe/d/EE89e3Q/SRiDU8uI3NFqYspwfTzXd5D7XTddkJSwMAxzF4Xco1J0L+psvoLO597z4cqP7RC0rdmGdYnw3Ukf3gAFwq+xqpg93DkSJsTOiURbHXmZROuPwrrXbpP1VSFAkGfRtOgBlGaGqYFic/27mwLc72Uslo6NebzwA989nIlA2OWnNhIpFx8TlE4zT/J34j83x5fHiwNhmgMcq7D/io+dx4hC3zjqhAiWwTY9TVOgSnr6BPe/bIdgNYOQyiFSrzmFpT3czI+YXmeXrI
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(4326008)(76116006)(5660300002)(66476007)(54906003)(6506007)(6862004)(91956017)(2906002)(2616005)(6486002)(122000001)(66446008)(38100700002)(66556008)(64756008)(316002)(71200400001)(6636002)(37006003)(26005)(508600001)(53546011)(186003)(8676002)(33656002)(83380400001)(8936002)(36756003)(66946007)(38070700005)(6512007)(86362001)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <618757FC021F7D42B7C70811DA1CFAC2@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6175
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	76d14686-2469-4571-541f-08d9834e98ea
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Df0OCZqThHoGbKaOCBZeSvgPnS7rY3Jv1pSrAEYlYQ8mB4ka4hxQiZw4cVIjUtHHSBxZo8yI7I2z4fqr0gFpSTjb5Ljd9U0P6S2w9VNlhlsOKoEoe4tQNAXQrWLREKdiqYeMweCCum10z8anYFvRExeIeMp6TV+VZb4cbKlRYNQwkQ/KcB6vIcqSDSdBWB2CwlqnKH+vn1aqb0R3lkrc9IJAEcNhLyBXQz4AZ8r4/MqMSbiLGsfYwdRxSVfEzymBjdjSsOVhM720nYA1BGl5I3j4GD1xydni7Bn5q5VRIEhqYtEL0hNfdeG1RDq3ghdw8EMV60X9anQkO4cMEcWIKnI115y47cg9yKtYVnx3/cFefH1zcEU6+J4M3SqPKVpN9XpeuZr6uPQ4CSF65vI0ZZZCCPLtoi8dWjKtB6NklhVk8BJyiMkf10Y3zhJYzfzcY1LW6C84t4Y8pgyliIFA6RKqYAVKXd5v6ZidUxMrtgtVD6R9nW3d2Utckh4vhBle/ejdFjZG7k27iAwNT7bokbW5EsbMxtd4c71/yF3Y7jLn2akQuy/SdZ1E5JcpKvXPUqWOenmccG+ZV92I8ERQg5to+qh7YBLCwdhaHRAdfdu+EuuT/gxOrqglcEBw/yrqQHW4N5OJsSo3yZg3zI65uXBfYkCCXzlAdxTi84hIhOYDmPGuGaeklRTCfQSZnTpPnBQgA1Q11H1hclxWHJsI4A==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(2906002)(316002)(6862004)(336012)(83380400001)(186003)(6636002)(33656002)(107886003)(36756003)(26005)(8936002)(70206006)(6486002)(70586007)(356005)(47076005)(5660300002)(6512007)(81166007)(82310400003)(86362001)(53546011)(54906003)(8676002)(37006003)(508600001)(36860700001)(4326008)(6506007)(2616005);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:39:59.4387
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 83d23661-6369-4173-8be3-08d9834ea1c6
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT058.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2540

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> This patch will be reverted once we add support for VPCI MSI/MSIX
> support on ARM.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3: none
> Change in v2: Patch introduced in v2
> ---
> xen/drivers/vpci/Makefile | 3 ++-
> xen/drivers/vpci/header.c | 2 ++
> xen/include/asm-arm/pci.h | 8 ++++++++
> 3 files changed, 12 insertions(+), 1 deletion(-)
>=20
> diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
> index 55d1bdfda0..1a1413b93e 100644
> --- a/xen/drivers/vpci/Makefile
> +++ b/xen/drivers/vpci/Makefile
> @@ -1 +1,2 @@
> -obj-y +=3D vpci.o header.o msi.o msix.o
> +obj-y +=3D vpci.o header.o
> +obj-$(CONFIG_HAS_PCI_MSI) +=3D msi.o msix.o
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index ba9a036202..f8cd55e7c0 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev=
, uint16_t cmd,
>      * FIXME: punching holes after the p2m has been set up might be racy =
for
>      * DomU usage, needs to be revisited.
>      */
> +#ifdef CONFIG_HAS_PCI_MSI
>     if ( map && !rom_only && vpci_make_msix_hole(pdev) )
>         return;
> +#endif
>=20
>     for ( i =3D 0; i < ARRAY_SIZE(header->bars); i++ )
>     {
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 49c9622902..5532ce3977 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,14 @@ struct arch_pci_dev {
>     struct device dev;
> };
>=20
> +/* Arch-specific MSI data for vPCI. */
> +struct vpci_arch_msi {
> +};
> +
> +/* Arch-specific MSI-X entry data for vPCI. */
> +struct vpci_arch_msix_entry {
> +};
> +
> /*
>  * struct to hold the mappings of a config space window. This
>  * is expected to be used as sysdata for PCI controllers that
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:42:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:42:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199156.353037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZrF-0002s6-B8; Wed, 29 Sep 2021 13:42:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199156.353037; Wed, 29 Sep 2021 13:42:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZrF-0002ry-7f; Wed, 29 Sep 2021 13:42:49 +0000
Received: by outflank-mailman (input) for mailman id 199156;
 Wed, 29 Sep 2021 13:42:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVZrE-0002rs-Dw
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:42:48 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.15.77]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a65f1e55-6b0f-451c-8f5e-2ace51002102;
 Wed, 29 Sep 2021 13:42:47 +0000 (UTC)
Received: from DB7PR05CA0021.eurprd05.prod.outlook.com (2603:10a6:10:36::34)
 by AM0PR08MB3380.eurprd08.prod.outlook.com (2603:10a6:208:e4::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Wed, 29 Sep
 2021 13:42:31 +0000
Received: from DB5EUR03FT012.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:10:36:cafe::a9) by DB7PR05CA0021.outlook.office365.com
 (2603:10a6:10:36::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Wed, 29 Sep 2021 13:42:31 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT012.mail.protection.outlook.com (10.152.20.161) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:42:31 +0000
Received: ("Tessian outbound 0e48c0de19a3:v103");
 Wed, 29 Sep 2021 13:42:27 +0000
Received: from af24f523c9ec.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 89ED3D68-48CA-4649-8CD4-35BCF88909B3.1; 
 Wed, 29 Sep 2021 13:42:16 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id af24f523c9ec.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 13:42:16 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6496.eurprd08.prod.outlook.com (2603:10a6:102:df::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 13:42:12 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 13:42:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a65f1e55-6b0f-451c-8f5e-2ace51002102
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pgklEmJv5KWJcOjW60lRH5oWCx7Ng7cH+53tB11knY0=;
 b=XoEFt09EY4QR4g7ZWPLX/yElDo/qnTT5+pbb+cIPjR/wY1FGOHGpef2yMaSIC/hGQjmWgHSsMyN/PiV7BtmoMuWXngKSDkKm+Z7Ge12D/SRVv8bjqRapOjvRqVKtHsvNeA/ZvF6xT8i0vjfdZ9OdD/x51S9I4EhS3iu1K1yypS0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: f8d14c3f15e8d3ff
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EZBEo3A/mJ21iZscOheF4vvAyo4wWuqwWz591vRfpiDoyL0/mN9sBeuqNE+37GLCpDs5uQpNo/ph9xEhdzFshv0Q1Ni5X0D8levr5Ym+nTVj+/uH7FLIrhBEdHoM4Twt4l1AhPUR+HBlmXsgJHJ5Xbc07gJXSamCeyEZNLZs4I4sQkvmYyZQhYNiYbZ4qwYGVZYH4SvOoO3H1fdHj38vNn1EFrJQ5aoNUzNrJRl96Bn1tjyEhTDifNzPhQXyUGUdVpyi/0czO+oIN9aRLILP7CmFlEVReZC6g7lPfTZp4aUYMBvFbapPlPHj1ZlZDlHTJJOK7ikB839cCcrCyj++Ag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pgklEmJv5KWJcOjW60lRH5oWCx7Ng7cH+53tB11knY0=;
 b=i05aVKyIN2TFHqKX0X8bJ0ND5c1g3fW8ybKWyTjWDxRIirTmQHt0tGew8z8fGtv/pb9H6kWsKmsYyADZvShJgLkcFyUrCZHp2Iy7PUlE6IcGD+z9D6P3f9oZUNGHTpPaJh0nbeVOnXe0MNFp7cPqAxYBOx+O2DA5r+6Swg38H0zwJEx913iaR3xNP2nATu/PrgHl+g5CQjvxxB31D4pvhWGCcPg3zbYp2wt7hwodVtncc1+paUTM05p6RnblwC2Dd9EWMexVc7vFT2XR6lQDg6u2bhIpeiBklTnkpCmufmVVpfpCQ4ckQpPtiKd38bpLn8i8JHHg7WawhNs+qA5T6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pgklEmJv5KWJcOjW60lRH5oWCx7Ng7cH+53tB11knY0=;
 b=XoEFt09EY4QR4g7ZWPLX/yElDo/qnTT5+pbb+cIPjR/wY1FGOHGpef2yMaSIC/hGQjmWgHSsMyN/PiV7BtmoMuWXngKSDkKm+Z7Ge12D/SRVv8bjqRapOjvRqVKtHsvNeA/ZvF6xT8i0vjfdZ9OdD/x51S9I4EhS3iu1K1yypS0=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Ian Jackson <iwj@xenproject.org>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 16/17] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Topic: [PATCH v3 16/17] arm/libxl: Emulated PCI device tree node in
 libxl
Thread-Index: AQHXtJYZNsoakdSb20yaB3wcBYqJ26u7BnAA
Date: Wed, 29 Sep 2021 13:42:12 +0000
Message-ID: <21569545-B5FB-4C9E-88A6-C5C26CD40661@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <7f19d0802a9ac7544ddefe96f282ba7f97caefe9.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <7f19d0802a9ac7544ddefe96f282ba7f97caefe9.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 821e9264-680a-4d83-1f19-08d9834efc88
x-ms-traffictypediagnostic: PAXPR08MB6496:|AM0PR08MB3380:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM0PR08MB3380E9A29887C2EE2FE344ED9DA99@AM0PR08MB3380.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 +ZhUy0B9Iddru1x9uxEo/K7wmz3RL+915qupOT7/BK1pz3lEzi48Ipym9ugwuYOlD0eIhmqRc2qx1eRJwvLZXqOUab2Fs7YzIQHI2jqNZQVIVNxx91mQyV6ZTUpamoYrGSaaIsnVYF7ylk0tt6+fSI5KSqlvqU2cr92CkAlJcZb2ojA2IGcyetEfoPeKJiPEZj6M8O9Xl8mslNPSqeqI7NyI7rLYKptMnaPbUOpnhOWPqYOLbjfcz5n0sLuKm1G1iy0LoBMMbSWtbhbaYOWbbGZr9iHSG2dm2gIttaJyOxDeGnozR1KHhFjX27Yq1FmNnBTfm5ZGY479EDp9cjXYRnZ/39nNuSKZjFL1Q4LeMD7jW/UGu6ZzpKTTfruTE6fd8IdLm6NzeiCeVGqshLjjtQoQ4oFPF/clz6BPSILVf08UiIYWwgbK9bBTOIEPmTjoe294heuggrsX/MGT0cUd4nRbtNXXzBaZqeoXfmIQj8LCtuijhcsFnvUBQvc6aEWUL1jSaySnNH49OHPihBu3TpVBz6RirU0AeN8XPttRPNyZvQMLpCDutgZ/HkqdlFA8hpoSgSthz6lLd4b9+Sq9EL42Qgaskjvc/VJIyUEv/sBETOCmvkOukg+4gXAYRHEaHruuMTVqhZ2Adxk+WmfIcKGwIDu8tXVN5yH8xl2L3Z2zdEPCd4ziQM2o8vIxIWFw62u972KVfyeK0nZD+MwS2aPmae584IS53F4XlkvHoEI48LlX2wj+OYD4KH1hMGHE
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(36756003)(64756008)(186003)(8936002)(4326008)(6862004)(53546011)(316002)(26005)(91956017)(6506007)(66946007)(66556008)(76116006)(66446008)(66476007)(83380400001)(6636002)(6486002)(508600001)(54906003)(122000001)(37006003)(2906002)(86362001)(6512007)(38100700002)(2616005)(71200400001)(5660300002)(33656002)(8676002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8B43DE908182354FAA9345C5CD7FBD3E@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6496
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	d57f5ef0-7f69-4253-9654-08d9834ef142
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oX6Cb/cQwCFMjSu5QDHHbtKqVSA9OtTyQ9S7SRdnJR93g7Do5/UTx1S/+0gakvR9U+ZPB+j6r/PzUtek6n8U2i2CLOr9gSYQTvILBFK3tAlk5jxLdnhj3LzsNKPeMMLK5FOTis22CbSGjyGMqiw4CkrT9e4/gkk0IjmGTQOrRxoXo+QpJ1toKeg0N6sWzEytu64qVoS7+sGouBxt7Q1ANIdHAn8QrEu84HeH05xIospUqf+Ua5R6CYEZNguaGztQIDHhq3SbFCs3+/X4EpUQatwG3UMJ7/NH+cjYb1qwae4KUuSkOvBCP8+0WWPBTE5AdHqn+2U2OSlrwizZSSJ1c5c9tx/ZVSfm3Et9QrAia6/dzOY4JkRKmJTykX59lTwxfh0/xqAWXQLZ5I2jgHcgRh0zgtok9ArwML2U/pYvm3LpqamwYsGHLrMoXBKthuGc4UWtbuo0d+x5Kz4o7QJjJ6jH/GNb8bcAaa/Tk+89TzwtnKg7rH3KP4ztattp9sa0LeTemJdyFI3U9FK411ZlwvJ5i0x2KANxNaNU1GwYXz9peuxc+SWJQrLqySNB5F2/VqBXLKUorb1x36eTgZ8tYinIOIuevaG8KsOfQrYyRt2DbKE/Dnkr7kntzln/RlfrER1x8ob05nsjVpnUwQ5a+QJpbKD426dAUgA26JlpF5O+foVJc6N/GOnD/DLOROl8/JC2+WjzfFj+ftrDKL7kyQ==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(316002)(37006003)(53546011)(26005)(70586007)(86362001)(8936002)(8676002)(6506007)(70206006)(33656002)(6636002)(82310400003)(5660300002)(508600001)(186003)(2616005)(81166007)(4326008)(356005)(36756003)(6512007)(2906002)(336012)(6862004)(83380400001)(36860700001)(47076005)(54906003)(107886003)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:42:31.7012
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 821e9264-680a-4d83-1f19-08d9834efc88
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT012.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3380

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> libxl will create an emulated PCI device tree node in the device tree to
> enable the guest OS to discover the virtual PCI during guest boot.
> Emulated PCI device tree node will only be created when there is any
> device assigned to guest.
>=20
> A new area has been reserved in the arm guest physical map at
> which the VPCI bus is declared in the device tree (reg and ranges
> parameters of the node).
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Make GUEST_VPCI_MEM_ADDR address 2MB aligned
> Change in v2:
> - enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain.
> ---
> tools/include/libxl.h            |   6 ++
> tools/libs/light/libxl_arm.c     | 105 +++++++++++++++++++++++++++++++
> tools/libs/light/libxl_create.c  |   3 +
> tools/libs/light/libxl_types.idl |   1 +
> tools/xl/xl_parse.c              |   2 +
> xen/include/public/arch-arm.h    |  10 +++
> 6 files changed, 127 insertions(+)
>=20
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d698..3362073b21 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -358,6 +358,12 @@
>  */
> #define LIBXL_HAVE_BUILDINFO_ARM_VUART 1
>=20
> +/*
> + * LIBXL_HAVE_BUILDINFO_ARM_VPCI indicates that the toolstack supports v=
irtual
> + * PCI for ARM.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARM_VPCI 1
> +
> /*
>  * LIBXL_HAVE_BUILDINFO_GRANT_LIMITS indicates that libxl_domain_build_in=
fo
>  * has the max_grant_frames and max_maptrack_frames fields.
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e00..52f1ddce48 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fd=
t,
>     return fdt_property(fdt, "reg", regs, sizeof(regs));
> }
>=20
> +static int fdt_property_values(libxl__gc *gc, void *fdt,
> +        const char *name, unsigned num_cells, ...)
> +{
> +    uint32_t prop[num_cells];
> +    be32 *cells =3D &prop[0];
> +    int i;
> +    va_list ap;
> +    uint32_t arg;
> +
> +    va_start(ap, num_cells);
> +    for (i =3D 0 ; i < num_cells; i++) {
> +        arg =3D va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, name, prop, sizeof(prop));
> +}
> +
> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
> +                                    unsigned addr_cells,
> +                                    unsigned size_cells,
> +                                    unsigned num_regs, ...)
> +{
> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
> +    be32 *cells =3D &regs[0];
> +    int i;
> +    va_list ap;
> +    uint64_t arg;
> +
> +    va_start(ap, num_regs);
> +    for (i =3D 0 ; i < num_regs; i++) {
> +        /* Set the memory bit field */
> +        arg =3D va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +
> +        /* Set the vpci bus address */
> +        arg =3D addr_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, addr_cells , arg);
> +
> +        /* Set the cpu bus address where vpci address is mapped */
> +        set_cell(&cells, addr_cells, arg);
> +
> +        /* Set the vpci size requested */
> +        arg =3D size_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, size_cells, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
> +}
> +
> static int make_root_properties(libxl__gc *gc,
>                                 const libxl_version_info *vers,
>                                 void *fdt)
> @@ -668,6 +720,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void=
 *fdt,
>     return 0;
> }
>=20
> +static int make_vpci_node(libxl__gc *gc, void *fdt,
> +        const struct arch_info *ainfo,
> +        struct xc_dom_image *dom)
> +{
> +    int res;
> +    const uint64_t vpci_ecam_base =3D GUEST_VPCI_ECAM_BASE;
> +    const uint64_t vpci_ecam_size =3D GUEST_VPCI_ECAM_SIZE;
> +    const char *name =3D GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
> +
> +    res =3D fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res =3D fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
> +    if (res) return res;
> +
> +    res =3D fdt_property_string(fdt, "device_type", "pci");
> +    if (res) return res;
> +
> +    res =3D fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
> +    if (res) return res;
> +
> +    res =3D fdt_property_values(gc, fdt, "bus-range", 2, 0, 255);
> +    if (res) return res;
> +
> +    res =3D fdt_property_cell(fdt, "#address-cells", 3);
> +    if (res) return res;
> +
> +    res =3D fdt_property_cell(fdt, "#size-cells", 2);
> +    if (res) return res;
> +
> +    res =3D fdt_property_string(fdt, "status", "okay");
> +    if (res) return res;
> +
> +    res =3D fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +        GUEST_ROOT_SIZE_CELLS, 2,
> +        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SI=
ZE,
> +        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
> +        GUEST_VPCI_PREFETCH_MEM_SIZE);
> +    if (res) return res;
> +
> +    res =3D fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> static const struct arch_info *get_arch_info(libxl__gc *gc,
>                                              const struct xc_dom_image *d=
om)
> {
> @@ -971,6 +1070,9 @@ next_resize:
>         if (info->tee =3D=3D LIBXL_TEE_TYPE_OPTEE)
>             FDT( make_optee_node(gc, fdt) );
>=20
> +        if (libxl_defbool_val(info->arch_arm.vpci))
> +            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
> +
>         if (pfdt)
>             FDT( copy_partial_fdt(gc, fdt, pfdt) );
>=20
> @@ -1189,6 +1291,9 @@ void libxl__arch_domain_build_info_setdefault(libxl=
__gc *gc,
>     /* ACPI is disabled by default */
>     libxl_defbool_setdefault(&b_info->acpi, false);
>=20
> +    /* VPCI is disabled by default */
> +    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
> +
>     if (b_info->type !=3D LIBXL_DOMAIN_TYPE_PV)
>         return;
>=20
> diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_cre=
ate.c
> index e356b2106d..529e475489 100644
> --- a/tools/libs/light/libxl_create.c
> +++ b/tools/libs/light/libxl_create.c
> @@ -632,6 +632,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_co=
nfig *d_config,
>         if (info->passthrough =3D=3D LIBXL_PASSTHROUGH_SYNC_PT)
>             create.iommu_opts |=3D XEN_DOMCTL_IOMMU_no_sharept;
>=20
> +        if ( libxl_defbool_val(b_info->arch_arm.vpci) )
> +            create.flags |=3D XEN_DOMCTL_CDF_vpci;
> +
>         /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
>         libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
>=20
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_ty=
pes.idl
> index 3f9fff653a..78b1ddf0b8 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,7 @@ libxl_domain_build_info =3D Struct("domain_build_info=
",[
>=20
>     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                ("vuart", libxl_vuart_type),
> +                               ("vpci", libxl_defbool),
>                               ])),
>     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                               ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd5..ffafbeffb4 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>         }
>         if (d_config->num_pcidevs && c_info->type =3D=3D LIBXL_DOMAIN_TYP=
E_PV)
>             libxl_defbool_set(&b_info->u.pv.e820_host, true);
> +        if (d_config->num_pcidevs)
> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>     }
>=20
>     if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.=
h
> index 727541a321..acd97eb327 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -431,6 +431,11 @@ typedef uint64_t xen_callback_t;
> #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
> #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
>=20
> +/* Guest PCI-PCIe memory space where config space and BAR will be availa=
ble.*/
> +#define GUEST_VPCI_ADDR_TYPE_MEM            xen_mk_ullong(0x02000000)
> +#define GUEST_VPCI_MEM_ADDR                 xen_mk_ullong(0x23000000)
> +#define GUEST_VPCI_MEM_SIZE                 xen_mk_ullong(0x10000000)
> +
> /*
>  * 16MB =3D=3D 4096 pages reserved for guest to use as a region to map it=
s
>  * grant table in.
> @@ -446,6 +451,11 @@ typedef uint64_t xen_callback_t;
> #define GUEST_RAM0_BASE   xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1=
GB */
> #define GUEST_RAM0_SIZE   xen_mk_ullong(0xc0000000)
>=20
> +/* 4GB @ 4GB Prefetch Memory for VPCI */
> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM   xen_mk_ullong(0x42000000)
> +#define GUEST_VPCI_PREFETCH_MEM_ADDR        xen_mk_ullong(0x100000000)
> +#define GUEST_VPCI_PREFETCH_MEM_SIZE        xen_mk_ullong(0x100000000)
> +
> #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ =
8GB */
> #define GUEST_RAM1_SIZE   xen_mk_ullong(0xfe00000000)
>=20
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:42:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:42:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199157.353048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZrL-0003CL-Ml; Wed, 29 Sep 2021 13:42:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199157.353048; Wed, 29 Sep 2021 13:42:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZrL-0003CB-Jn; Wed, 29 Sep 2021 13:42:55 +0000
Received: by outflank-mailman (input) for mailman id 199157;
 Wed, 29 Sep 2021 13:42:54 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c0G+=OT=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVZrK-0003Bc-GW
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:42:54 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:7d00::628])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 61a74d2b-70a9-4362-8321-dad6a140112e;
 Wed, 29 Sep 2021 13:42:53 +0000 (UTC)
Received: from AM0PR03CA0088.eurprd03.prod.outlook.com (2603:10a6:208:69::29)
 by AS8PR08MB6487.eurprd08.prod.outlook.com (2603:10a6:20b:33e::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 13:42:51 +0000
Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:208:69:cafe::1) by AM0PR03CA0088.outlook.office365.com
 (2603:10a6:208:69::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend
 Transport; Wed, 29 Sep 2021 13:42:51 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 13:42:50 +0000
Received: ("Tessian outbound c9f4ff96a6ad:v103");
 Wed, 29 Sep 2021 13:42:49 +0000
Received: from 88a2a7138c70.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F82425CF-A882-4795-80B8-3168991BCFD1.1; 
 Wed, 29 Sep 2021 13:42:38 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 88a2a7138c70.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Wed, 29 Sep 2021 13:42:38 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5724.eurprd08.prod.outlook.com (2603:10a6:102:85::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Wed, 29 Sep
 2021 13:42:35 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4478.025; Wed, 29 Sep 2021
 13:42:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61a74d2b-70a9-4362-8321-dad6a140112e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GK6htvme9xg7hSsrL/AXnqo3AzT3yL+ksC2ZRu1U1qg=;
 b=0+K4YQneDNcEeHXLuq6FC6kiUCJsOlre8UvhbKZXdTbxwqZ/laxWaVrVTNx8cHIff2RC2xHiHkrLNWf0rxH16Ap2C2gmhqPOCuEbitDUgx+yAtBZe1Mw5sNx+VDxCdAuV8oU0+J21rBafzpD/vxO77vlgoRPOwIF0y3dbJNWEIA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 90eb376cf091c9cb
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kTo9e3USnb24yQS2GGeRJYqBEAt9FYuw07O+0KcFsuJRYCU2fdNm9dJOWP0a4RKXWPdHoISjiWODkCMYeF9Pep1wc0qIReCLim+wihSS042IcmLht7a+oXwOH1d43cpA83cpltNZf4LhbgW6DECJHPxVdOAIhLljjPG96uO4SmxE0N/Pf8/7UQ0iBVcU/3Ni+ypVcaO1u8jaN8/zquNCbGjP/qsQBZ7JT4OjY2w27vnMgzXe5EfClTKrbn1M6Z2/mX3ySLpdBhE5tqCDnYVejHrMQBK6dvghN3fT0VmspCjujvaxkcOnwV9WyRfZnjUxOvziWBZa063T3L0Dvyw+Qw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=GK6htvme9xg7hSsrL/AXnqo3AzT3yL+ksC2ZRu1U1qg=;
 b=FtGMC0aBXYDehbEir/MVTetpuX2CDW8PFfdMzOO8e1MZrhjVr+Ps7ifMRD4rbqwYV4c4bv8+Fn+gHFaFoDYg5VIjy6ebc89QwtuFDIeB/yHa+fUSGwZjNocZc8jG2fITy0EJ+kzEb2kq7PYyVT1/DDLNSFYFzsCwCOELhbxbbDQ81RFkPZr/pSbZZnHAbIWla+//L+yJ2Hh+GbWaQd5M+KOzg+0drStgSjDDcpJNm9ThD4WQkkAgqVK7YNp1k1dIZEAPDeeTAurbPxaCCRe1tCP4JgI19qM294wzsnU1Xhc/ItZnFpGh7YlOPjejxBeov4PPj6mqqwWVtd8PWUNOWQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GK6htvme9xg7hSsrL/AXnqo3AzT3yL+ksC2ZRu1U1qg=;
 b=0+K4YQneDNcEeHXLuq6FC6kiUCJsOlre8UvhbKZXdTbxwqZ/laxWaVrVTNx8cHIff2RC2xHiHkrLNWf0rxH16Ap2C2gmhqPOCuEbitDUgx+yAtBZe1Mw5sNx+VDxCdAuV8oU0+J21rBafzpD/vxO77vlgoRPOwIF0y3dbJNWEIA=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Rahul Singh <Rahul.Singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Andre Przywara
	<Andre.Przywara@arm.com>, Stefano Stabellini <sstabellini@kernel.org>, Julien
 Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 17/17] xen/arm: Add linux,pci-domain property for hwdom
 if not available.
Thread-Topic: [PATCH v3 17/17] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
Thread-Index: AQHXtJYkOb3La5Yps0eoFpxXuc4i76u7BooA
Date: Wed, 29 Sep 2021 13:42:35 +0000
Message-ID: <7EA77A3D-68A7-4738-A7B6-0C144BB741A1@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <f290d8e2f9b3780c5a956a8032aba6f35de83a90.1632847120.git.rahul.singh@arm.com>
In-Reply-To:
 <f290d8e2f9b3780c5a956a8032aba6f35de83a90.1632847120.git.rahul.singh@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: f8deb226-96c6-436a-59e4-08d9834f0802
x-ms-traffictypediagnostic: PR3PR08MB5724:|AS8PR08MB6487:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB6487A3940B9665D3405D9EE99DA99@AS8PR08MB6487.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Fo5GyJJWPQGZnNJXFjg2eP5jq1vU7ydatcyPjoLg/xAEI9WuxNzu4luVZoQPaEbdIl3wkBVvUvJFMDQvVhitS6/F5IX8BT7f3PgPz4jYcpGf6xl7TNY3N2HqCZ2Bxa4UPThQWW/JH8hMooZHdWxgHWi0z3dNXkNgIxpcFf9tORJ98p/1v5dcgGBXV/ecTkgGOi5LKFpJTXr4RmtdYwvlkpBBEGhcA2AfLIdxOvVsF4byWAgZN0waFnFNLEDV8Q1sB9Nex2pplWRX4czvHrh9hRhJsLf57YPncEV7blh4GSFsqZ4V/ztZtcePGrIMqvjXPjUwMDcBhyEeZrihHc0smtJ3+YUhvydHiYjyzPYCZAT6Eamq1AvHJS2HTa9uVGf0bQ/6ovGsj6pHlH2zGbrkaf/2iYiFCPrXG3zjb8PDkZEMhXNr/HHK0db4FrIO4b9ur/LbxRnuC1IV/hxDDo6UjCrL/eJjEV2ljIKxhCIo78J0LYQgg9exrmBNqEzmv2rzVECuEistgHwt8yEWe2oBKTgxB9xcTLhSEuIm8kBed0VDznbu/YlkpNg9vXXg74kC78XK/eoLM09Wq7YcFVLU5Ssob+ruIYIlnjmo0YHEqFd+mh5+HyblMcVph5mapdN569tZnYJ3fK/qEne07cNYr83+5E8hdrdUVnVxz+Xkcok5A2Bg2hHnUocsvrfNX9S05GYikT7fsrOs1i0q5oUbBZn3CF24Uno350N6XseBoHN/JhLCOJ7qRZLJUDUdcLRr
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38070700005)(53546011)(6506007)(2906002)(122000001)(71200400001)(6486002)(26005)(33656002)(37006003)(36756003)(6512007)(38100700002)(2616005)(8676002)(4326008)(54906003)(64756008)(6862004)(316002)(186003)(8936002)(508600001)(66476007)(66556008)(66446008)(66946007)(76116006)(86362001)(91956017)(5660300002)(6636002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <9348349101CA564B9CEAE45AE9C5CA27@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5724
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	41b3e227-560c-43b1-b2e4-08d9834efebe
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	QRZjsde9bFpXxQoC6ElTn8lJzoZoKwUDziyKzQNXfUEZhhAaSXHpNOOHKcYwyUFkGBB7xQDBVhoLp/0W4yL7bY2HLwbqsnCOUOJ0JyVQAPo1HvekNb/dCP/G+muwwh0G/i/gUweoKGRAoyp7qn9Tdol25/CkxZDcAUy2whAjBE0lj1Ye7PHuweAmjS2N+fB46Uprws3bMuz3yZP8tr+duls0ZklcnPQu0CUCir5ML0NjucEMD/JvSOpqSIJrOffKTdj/cDj6JY3c9P4h6j8ZkYnvKB3h5ycK65DJNUPjPd5i29bkpeHSariOSEfmm7pmeqJqSf1XJj7g4KiDp4DyC8dkNamlAFYfvdjZPDXJdk1ShVf+u+bUGDGxS6FWeqwudAHZBUNMyVrXaSEIWgilh7qopq/mSmYN/23u1vd6LlzcbienwmYPDCVcRFCp23NmlL7Duinbt/th/vYAi2YibiNVaQnoN5hD9SoxXQE77mHxi9riaoTHU0fYnl+pIQhdDLVIXsmKi90gCYKJBexYo7gtTEjOkvoRbINInhuSJjiUjBlmF6WYXBcJd5hQXRlLOc7+Lhx4r5la+qLlI5itJtCxGigcmwX497vmpmP0L5EQgrOOTFt5zwdcMws0YaG0hWA7Iw5BZyDnJn/q0m85UNXmIfb5ngC9Ga7oPMIh2HJNRoB6QLNFr15IyZBWgrn40vWUo3flFYanTl3Dj/I7nw==
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(6486002)(70586007)(70206006)(2616005)(356005)(8936002)(82310400003)(336012)(54906003)(6636002)(508600001)(37006003)(53546011)(6512007)(33656002)(81166007)(2906002)(36860700001)(107886003)(8676002)(316002)(36756003)(4326008)(5660300002)(186003)(47076005)(86362001)(6506007)(26005)(6862004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 13:42:50.8543
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f8deb226-96c6-436a-59e4-08d9834f0802
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6487

Hi Rahul,

> On 28 Sep 2021, at 19:18, Rahul Singh <rahul.singh@arm.com> wrote:
>=20
> If the property is not present in the device tree node for host bridge,
> XEN while creating the dtb for hwdom will create this property and
> assigns the already allocated segment to the host bridge
> so that XEN and linux will have the same segment for the host bridges.
>=20
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Change in v3:
> - Use is_pci_passthrough_enabled()
> Change in v2:
> - Add linux,pci-domain only when pci-passthrough command line option is e=
nabeld
> ---
> xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
> xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
> xen/include/asm-arm/pci.h          |  9 +++++++++
> 3 files changed, 46 insertions(+)
>=20
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 1731ae2028..026c9e5c6c 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d,=
 struct kernel_info *kinfo,
>             return res;
>     }
>=20
> +    if ( is_pci_passthrough_enabled() && dt_device_type_is_equal(node, "=
pci") )
> +    {
> +        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
> +        {
> +            uint16_t segment;
> +
> +            res =3D pci_get_host_bridge_segment(node, &segment);
> +            if ( res < 0 )
> +                return res;
> +
> +            res =3D fdt_property_cell(kinfo->fdt, "linux,pci-domain", se=
gment);
> +            if ( res )
> +                return res;
> +        }
> +    }
> +
>     /*
>      * Override the property "status" to disable the device when it's
>      * marked for passthrough.
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-ho=
st-common.c
> index c5941b10e9..593beeb48c 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -255,6 +255,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_=
t segment, uint8_t bus)
>=20
>     return NULL;
> }
> +
> +/*
> + * This function will lookup an hostbridge based on config space address=
.
> + */
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->dt_node !=3D node )
> +            continue;
> +
> +        *segment =3D bridge->segment;
> +        return 0;
> +    }
> +
> +    return -EINVAL;
> +}
> +
> /*
>  * Local variables:
>  * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 5532ce3977..7cb2e2f1ed 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -90,6 +90,8 @@ int pci_generic_config_write(struct pci_host_bridge *br=
idge, pci_sbdf_t sbdf,
> void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                pci_sbdf_t sbdf, uint32_t where);
> struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bu=
s);
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment);
>=20
> static always_inline bool is_pci_passthrough_enabled(void)
> {
> @@ -104,5 +106,12 @@ static always_inline bool is_pci_passthrough_enabled=
(void)
>     return false;
> }
>=20
> +static inline int pci_get_host_bridge_segment(const struct dt_device_nod=
e *node,
> +                                              uint16_t *segment)
> +{
> +    ASSERT_UNREACHABLE();
> +    return -EINVAL;
> +}
> +
> #endif  /*!CONFIG_HAS_PCI*/
> #endif /* __ARM_PCI_H__ */
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:44:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:44:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199170.353059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZsV-00048W-10; Wed, 29 Sep 2021 13:44:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199170.353059; Wed, 29 Sep 2021 13:44:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZsU-00048P-U0; Wed, 29 Sep 2021 13:44:06 +0000
Received: by outflank-mailman (input) for mailman id 199170;
 Wed, 29 Sep 2021 13:44:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CEC3=OT=pengutronix.de=ukl@srs-us1.protection.inumbo.net>)
 id 1mVZsT-00048B-MG
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:44:05 +0000
Received: from metis.ext.pengutronix.de (unknown
 [2001:67c:670:201:290:27ff:fe1d:cc33])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bdc11d22-62f2-481a-b433-5ae93a89744b;
 Wed, 29 Sep 2021 13:44:04 +0000 (UTC)
Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2])
 by metis.ext.pengutronix.de with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVZs4-0002ro-Tl; Wed, 29 Sep 2021 15:43:40 +0200
Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de)
 by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVZru-0005It-Cu; Wed, 29 Sep 2021 15:43:30 +0200
Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.92)
 (envelope-from <ukl@pengutronix.de>)
 id 1mVZru-0001Wi-AO; Wed, 29 Sep 2021 15:43:30 +0200
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bdc11d22-62f2-481a-b433-5ae93a89744b
Date: Wed, 29 Sep 2021 15:43:30 +0200
From: Uwe =?utf-8?Q?Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
To: Andrew Donnellan <ajd@linux.ibm.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Oliver O'Halloran <oohall@gmail.com>,
	Russell Currey <ruscur@russell.cc>, Jiri Olsa <jolsa@redhat.com>,
	Christoph Hellwig <hch@lst.de>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Mathias Nyman <mathias.nyman@intel.com>,
	Michael Ellerman <mpe@ellerman.id.au>, x86@kernel.org,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Ingo Molnar <mingo@redhat.com>, linux-pci@vger.kernel.org,
	xen-devel@lists.xenproject.org, Juergen Gross <jgross@suse.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Borislav Petkov <bp@alien8.de>, Bjorn Helgaas <bhelgaas@google.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	linux-usb@vger.kernel.org, linux-perf-users@vger.kernel.org,
	kernel@pengutronix.de, Frederic Barrat <fbarrat@linux.ibm.com>,
	Paul Mackerras <paulus@samba.org>, linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v5 10/11] PCI: Replace pci_dev::driver usage by
 pci_dev::dev.driver
Message-ID: <20210929134330.e5c57t7mtwu5iner@pengutronix.de>
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
 <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
 <75dd6d60-08b9-fa68-352c-3a0c5a04c0ab@linux.ibm.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="ycp367bsarc33tfb"
Content-Disposition: inline
In-Reply-To: <75dd6d60-08b9-fa68-352c-3a0c5a04c0ab@linux.ibm.com>
X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2
X-SA-Exim-Mail-From: ukl@pengutronix.de
X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false
X-PTX-Original-Recipient: xen-devel@lists.xenproject.org


--ycp367bsarc33tfb
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Sep 29, 2021 at 11:15:37PM +1000, Andrew Donnellan wrote:
> On 29/9/21 6:53 pm, Uwe Kleine-K=F6nig wrote:>   =09
> list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
> > -			if (afu_dev->driver && afu_dev->driver->err_handler &&
> > -			    afu_dev->driver->err_handler->resume)
> > -				afu_dev->driver->err_handler->resume(afu_dev);
> > +			struct pci_driver *afu_drv;
> > +			if (afu_dev->dev.driver &&
> > +			    (afu_drv =3D to_pci_driver(afu_dev->dev.driver))->err_handler &&
>=20
> I'm not a huge fan of assignments in if statements and if you send a v6 I=
'd
> prefer you break it up.

I'm not a huge fan either, I used it to keep the control flow as is and
without introducing several calls to to_pci_driver.

The whole code looks as follows:

	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
		struct pci_driver *afu_drv;
		if (afu_dev->dev.driver &&
		    (afu_drv =3D to_pci_driver(afu_dev->dev.driver))->err_handler &&
		    afu_drv->err_handler->resume)
			afu_drv->err_handler->resume(afu_dev);
	}

Without assignment in the if it could look as follows:

	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
		struct pci_driver *afu_drv;

		if (!afu_dev->dev.driver)
			continue;

		afu_drv =3D to_pci_driver(afu_dev->dev.driver));

		if (afu_drv->err_handler && afu_drv->err_handler->resume)
			afu_drv->err_handler->resume(afu_dev);
	}

Fine for me.

(Sidenote: What happens if the device is unbound directly after the
check for afu_dev->dev.driver? This is a problem the old code had, too
(assuming it is a real problem, didn't check deeply).)

> Apart from that everything in the powerpc and cxl sections looks good to =
me.

Thanks for checking.

Best regards
Uwe

--=20
Pengutronix e.K.                           | Uwe Kleine-K=F6nig            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

--ycp367bsarc33tfb
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEyBAABCgAdFiEEfnIqFpAYrP8+dKQLwfwUeK3K7AkFAmFUbX8ACgkQwfwUeK3K
7Am1vgf4oAZW8TWGX7Ao47AspECT0eq6axIwcj5PXeVpv5NC7BZPYClTPkycpcUZ
q8GsCEDyP5ratJ0gW3XFPyCkvgml0ndAP8y5Cs1ZeE1rU6xCJivD/HWMBoTJvaIH
uWxKIKzt/IiZWdt4Wo+fYplgR1/fak8IgUcV7jWOXA7pVVcZB3enVL/cB6xinq/H
+T9nGsYMVfSfdJdcD/EHVePI34oVlE9Eor3g1MFzRHk21B+r9J/TxUCsj84/JW60
P//CUO5CrpsjnqvhmShEm8POvpOjaTXLPxxTh3Oan2WFcgc/8mNx5zaKFdM1Rjmg
YeZXa3sMNdP7hIFZvqylIJDO3SbK
=MbZA
-----END PGP SIGNATURE-----

--ycp367bsarc33tfb--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 13:49:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 13:49:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199176.353070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZy2-0004zN-MK; Wed, 29 Sep 2021 13:49:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199176.353070; Wed, 29 Sep 2021 13:49:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVZy2-0004zG-IK; Wed, 29 Sep 2021 13:49:50 +0000
Received: by outflank-mailman (input) for mailman id 199176;
 Wed, 29 Sep 2021 13:49:49 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVZy1-0004zA-3f
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 13:49:49 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a42bd75a-15cc-4f6d-aed7-cf0c17291b1c;
 Wed, 29 Sep 2021 13:49:47 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TDJbs6005612; 
 Wed, 29 Sep 2021 13:49:43 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bcrx90448-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 13:49:43 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB4148.eurprd03.prod.outlook.com (2603:10a6:208:c7::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Wed, 29 Sep
 2021 13:49:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 13:49:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a42bd75a-15cc-4f6d-aed7-cf0c17291b1c
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EymcXtyUMFiWXOzfyVKXEB83wK9g+s0Sq//z3Eojrg5X87JH3pFoN2zsVcMZeGQu5PZZHLNnQ2OVPcYOtWwO1FAOpXGc2N9wLx+0FbT95d0Osso2522xQwDzZVBYHKNuyeo4BL+008PndPjtCe2JqZCx7w03PelAjptXE4Ll7Ry4EgxuIBnPYuGRlGUcpcMyiwUonOBd9XNaa7zWYLizSEr5T8ZA9KVXOSjm9z30x/QOol23RK+5b44CM/vwtgFJZ2IsYR5kpsU6IT4XtxUFKfjhMKVzQ6tyssw4c35SStBVwm8BPA+0vMnrkkewFlSoQOWnzEhRPsWc+HQrt9f2PQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=LDjzv/JHOi1ZoCTSe65PJ2JY2i/NlwXfIkaTelbeItA=;
 b=XHivATxkABGL7nJp3T8sMiC+yRlUSpSkTsYZpJKt4f0DNrzrdR0x+4G2lJ7cD8O81q8H3xr3PDfX9KX4ILF2GtbuKqub7kHDuGXmvY4XUivAhinbZw2yc9hwDwWD9v7NKnK1+0v5An+1JZFqZ4NbUOSFURSZDQThsarSsoTAf0P4/QpVOk6Xpk8T9PyQ4Ce4SnbG/0I1dQwxg4VAAOuu9coLwt5WDnAe6P74RICtKBXsKePLatnuJ9EXtwrS8LjzFDFwsoYAymkKD/a/znVQq4Sz7hSc84BabjPL7oTp3mTKoxZx8n33lWc7wM8aqiLS4iaUtoflJiwGS3MbM7YHvw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LDjzv/JHOi1ZoCTSe65PJ2JY2i/NlwXfIkaTelbeItA=;
 b=lbUBa+Xcp8YyefeKnjk8O+Q5rbxPhj6oaHvVCLV7k7AesnSiUlW1LAqtMh9vRsFAxX2CthKAa46yn0W6jxWRom3woL3RGAUR7B7N/TxfyGRTEg8iFwiFdFRikIozEvQ4FaTF+FEqBrfzOZlLLgA39oiNMat/0dVd2oZurQeKovOPtmQkE0r1Lx4pEzgCZuFhLopO2ozGugGDSg2Fs5Wfkti0kYaDn/KQPUi8BrCBXV5AyG5l21jvLpbRJHM9Eje5jKA6XtmsTbzHi2025e6GLykqD8UfNSEYkJhzChip8eaU07S97os8mAys/9PDHNoeb6zRW5mKkC4LK+FYRbze/g==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Michal
 Orzel <michal.orzel@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: 
 AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyAgAFQpACAAAGzgIAALsGAgAAQSwCAAAYBgIAAAhkAgAAHPoA=
Date: Wed, 29 Sep 2021 13:49:38 +0000
Message-ID: <5dad2004-f507-3610-cdec-cbf26f77cd5a@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
 <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
 <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
 <548561a6-f878-cacc-cda2-30508a43072d@suse.com>
In-Reply-To: <548561a6-f878-cacc-cda2-30508a43072d@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 18c614e2-8659-4ad1-e108-08d9834ffae2
x-ms-traffictypediagnostic: AM0PR03MB4148:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB4148BB2ABAF96F3EB5DF5817E7A99@AM0PR03MB4148.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 b8Udbz6UzngyhCwSYcW1rBB3GNPf2RkNKB/l2M549AIBPYtrl/Q1oy2RLQd1lzOMGud7Ud6r7E9zWEFrSvvxLSmyCYPKnpPwuvKHWTUu9YXzB898FvzyF/9q3CWZWf0FwlFer9uE2WwrcCjGn8CyCIwkDBf5S+Hv+4w3o6ZwwxhrOyuQKPmRKcoyigZyrzaJJi81Rx8FRhJdrMID7gb9wK4WqCO8E8chWGoqe6MGrNGg7vTPcil0wLFU266Xx5lg9JsHJo82sZ5XdDnLkZKXuf7UEIK4hzNuUfONVh0SZ2ib47eRqmkXc1HYu0a1KjoWcCLJgIvQOWUJq8rM5AK2SHKUl5mVIg9dpo8lBux8PsEOc7Cv1Dd91Bkh3zeLKxVTRtaxNfsk+FsLkPFBVvHBregQXcaa9PSxdpdRuo2Lbiv1zcqeHyYnve9t49CWUVPTCyhYgF8MN7nXDPINb/HiFHbV0ztuzzB/EkluezCR/QZT7LXtC8gzMYFsEb9MLP20x4U2/CdxVwaVHhY5I7Cp56NTyWJk0YwqiyByIuzwfxOALtypAvz9yS4EdtkvgoQrrwUUv7eDdMO4o71p2lJLN4g+h6kbtTg+T9R1uVos77+F9IEWeF9mSvcfje8zO16UiJ9R6dR6pG6vz4Tdax8U9NfTY7xyDWXz8jHMqBu425NS+iGPcjjM5asKImnmjltxaoCP08O//x/vVtBul87n2JP5TMbPuwJfZh4SQvm91ahtaEelPpBtsGZ+RcAcDdk4
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(83380400001)(31696002)(91956017)(2616005)(86362001)(36756003)(31686004)(54906003)(76116006)(38100700002)(122000001)(107886003)(6486002)(2906002)(4326008)(71200400001)(6916009)(26005)(55236004)(316002)(66946007)(186003)(66446008)(66556008)(66476007)(64756008)(8676002)(8936002)(6506007)(53546011)(6512007)(5660300002)(38070700005)(508600001)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?M2Y1UFB0QlJjR0czRE1xb0IydE8waDdMMmVjZm1HVGxDQVFNY1c5K1VwWmtz?=
 =?utf-8?B?ajBOdHFzMlZWQ1AxbmF3cWFBWUZNOGFSYXdiRyt5eGp1Tm9Bc0ZkQ21hVFpN?=
 =?utf-8?B?TlkyNERsMm1ucmUrOEFObUxMSUFKb2NyalQ1OThlbGE2L1pwOHAwOG9SRkNL?=
 =?utf-8?B?SHE1Q0l0S0pXTXBHN2FBS3FFQzhxVHlxYVNtSU5MSkhEMUhoZlBpVzMxSXhs?=
 =?utf-8?B?UEFBWHJLYUFVWklxeDRBa0Z2M1U5UlFlbzBBaW1hRitUM3ZGT3k5M092cmRz?=
 =?utf-8?B?WmJIS0dIdWZIS1hiUXVZWUJudUNXdXR6SjFTWVUwY1VMSkhUUjE5WXlwOFds?=
 =?utf-8?B?MFVZOHFiTnpnSnZpVVpSNVBRNlhIa1pzSHlOMmRoQitnbDVsU3RNS2g5WDUv?=
 =?utf-8?B?NXRRUzZSYXYvZDAvUjJLTklWcWVQcGtuVlRVMnhBSSsxb0IwVEZ4RG52c09i?=
 =?utf-8?B?a3cyaUhGdmRHb0NtSVpUdzRBamtnZUxZWnl2WXJERC9abFhkQWxqRlRzZGxB?=
 =?utf-8?B?TkhPVGppcStMLzczVDM5QlYvWi93N2FCd051QTExM1c2eWRHMWJoZFUxa2th?=
 =?utf-8?B?SVJmQ05OUmZIY1gxcElCNUpmdXAwUlBMT29aN3JRT3FqUHRLc3kvc3FZNnQx?=
 =?utf-8?B?NVdmSnZjT2U5eUR0NFp2MTdHY2sveW9CSktwOEs4THlOemxRa3J3c08xYjdW?=
 =?utf-8?B?azZ5OXNWSmpOSlJJZDNtN3dzQnNWay9lbEJFNjNCK24zZXhuUUQrY0VqSFlB?=
 =?utf-8?B?c1M4bnRKU3E4WGEwdkVWaitEbVNvK3paUVlPWEhyK3NNK05CRHBXM0RVS2Q3?=
 =?utf-8?B?cmZ3ZlN2WWpVdGc1Ni9CYk5aM3R6TVdMV3hzci9UbVVVclhzcEVpYkdVRHRD?=
 =?utf-8?B?eVcwSmlpZnpzejFmK29jSWRlRHQxYSt4VlFUUlFRMGQ1TVE3OTVQNmhYVGg2?=
 =?utf-8?B?MzRYYUc5MURwb1dxZXUvOHJHY1ZhVGptUW1UVzd4YzVUem05YjZlaFdGbXJB?=
 =?utf-8?B?YkI1dndlcTVTRUdtVnA5cUtpVXhHaGRrUSt4Z0o0Q0Z1Z2tNa1dDUzFMVWZ1?=
 =?utf-8?B?ZitCYkU2WG9yQmhrUFI4OE9zMGhCNUhlak05emFjNFJLMnVybDM1Sjl3ODFJ?=
 =?utf-8?B?WEI0eDhoMFA0SXJjU2oxZUpEZEhDNzJkYm55VlJka3k2TVFKU0NBVGg2UW9u?=
 =?utf-8?B?ZWNHNE55dzVwdHA5RW4wSFROdUd5ckc0Z0dmMFUrb29kMzNhWWlpVmZxR2ZT?=
 =?utf-8?B?Uis3RDVETW9Ra3B4M3lOWVhNVU5VSHVNWTdKK2FsSGhjZGp0eHdubUhYaXlC?=
 =?utf-8?B?dHNsbFhGbWlUcWovdUx4QlovS0VFU2VaUVQwdk45WlhHZnBBeCtGdDZISE95?=
 =?utf-8?B?K0kwM0lHcHFaRGQ0cWtKOUQxak5MSVdyOWhlNjUvR2J4bEZvRmlLNnEzNExu?=
 =?utf-8?B?ME8zdkRuUndPd3JFUmJXSmwwenB6VlFORXpjZktwRnJSZ2xPaUZXTTJWeks5?=
 =?utf-8?B?ajBXWS9PSW1SaHgzUEJ0NXdvSjFLV0VRWHhRRy9scGwrcTFCRlREbFVSL21h?=
 =?utf-8?B?cnJlSTd0RmhwbHZ6Y1VRc1dMbVNLUXZ4Sis0OUQzdDY1dkRTTEdvL2ZQT1J1?=
 =?utf-8?B?am9LekNuOTNPNG9rNlNUSHZPejgvRGI5cXNGTWtGMDd0RHFoU2xvdENFTTE2?=
 =?utf-8?B?eHlvQ01tTk9TODd4bC9KVkgxLzlMR2ZGUUN2a1VvM2JnZ2lrbEFsMjJoT0Rv?=
 =?utf-8?B?VmtzTys0V3FLdDc2UWVUTkladTlnV2I0eGNoL0tUYnNreGF1QUc1dGNaazVE?=
 =?utf-8?B?UXJZY3Rwd3JhWFJkVXFoQT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A86E25AAA90C134D9C74152CDF13E8C9@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 18c614e2-8659-4ad1-e108-08d9834ffae2
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 13:49:38.3089
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 40o+kPzM2P0BJkazHqil93kTxjOBeKZ4fHu9FOdI53L50vVdPlbCoMAsWcibOkA99LJd0oZjQ9yn3AuKHi9JXXedZ8peyKXTaMivrBmNVOd5pdeIqkulidj6GTaB5s8/
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB4148
X-Proofpoint-ORIG-GUID: CPToLJk17EHajEhHou77XrOGBCzg2zUP
X-Proofpoint-GUID: CPToLJk17EHajEhHou77XrOGBCzg2zUP
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_05,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0
 spamscore=0 mlxscore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0
 mlxlogscore=836 impostorscore=0 phishscore=0 suspectscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290086

DQoNCk9uIDI5LjA5LjIxIDE2OjIzLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMjkuMDkuMjAy
MSAxNToxNiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBPbiAyOS4wOS4yMSAx
NTo1NCwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4gT24gMjkuMDkuMjAyMSAxMzo1NiwgT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IE9uIDI5LjA5LjIxIDEyOjA5LCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+Pj4+IE9uIDI5LjA5LjIwMjEgMTE6MDMsIE9sZWtzYW5kciBBbmRydXNo
Y2hlbmtvIHdyb3RlOg0KPj4+Pj4+IFNvcnJ5IGZvciB0b3AgcG9zdGluZywgYnV0IHRoaXMgaXMg
YSBnZW5lcmFsIHF1ZXN0aW9uIG9uIHRoaXMgcGF0Y2gvZnVuY3Rpb25hbGl0eS4NCj4+Pj4+Pg0K
Pj4+Pj4+IERvIHlvdSBzZWUgd2UgbmVlZCB0byBnYXRlIGFsbCB0aGlzIHdpdGggQ09ORklHX0hB
U19WUENJX0dVRVNUX1NVUFBPUlQNCj4+Pj4+PiBhcyB0aGlzIHJlbmRlcnMgaW4gc29tZXdoYXQg
ZGVhZCBjb2RlIGZvciB4ODYgZm9yIG5vdz8gSSBkbyB0aGluayB0aGlzIHN0aWxsDQo+Pj4+Pj4g
bmVlZHMgdG8gYmUgaW4gdGhlIGNvbW1vbiBjb2RlIHRob3VnaC4NCj4+Pj4+IEkgYWdyZWUgaXQg
d2FudHMgdG8gbGl2ZSBpbiBjb21tb24gY29kZSwgYnV0IEknZCBzdGlsbCBsaWtlIHRoZSBjb2Rl
IHRvDQo+Pj4+PiBub3QgYmxvYXQgeDg2IGJpbmFyaWVzLiBIZW5jZSB5ZXMsIEkgdGhpbmsgdGhl
cmUgd2FudCB0byBiZQ0KPj4+Pj4gImlmICggIUlTX0VOQUJMRUQoKSApIiBlYXJseSBiYWlsb3V0
IHBhdGhzIG9yLCB3aGVuZXZlciB0aGlzIGlzbid0DQo+Pj4+PiBwb3NzaWJsZSB3aXRob3V0IGJy
ZWFraW5nIHRoZSBidWlsZCwgcmVzcGVjdGl2ZSAjaWZkZWYtcy4NCj4+Pj4gVGhlbiBpdCBuZWVk
cyB0byBiZSBkZWZpbmVkIGFzICh4ZW4vZHJpdmVycy9LY29uZmlnKToNCj4+Pj4NCj4+Pj4gY29u
ZmlnIEhBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+Pj4gICAgwqDCoMKgICMgdlBDSSBndWVzdCBz
dXBwb3J0IGlzIG9ubHkgZW5hYmxlZCBmb3IgQXJtIG5vdw0KPj4+PiAgICDCoMKgwqAgZGVmX2Jv
b2wgeSBpZiBBUk0NCj4+Pj4gICAgwqDCoMKgIGRlcGVuZHMgb24gSEFTX1ZQQ0kNCj4+Pj4NCj4+
Pj4gQmVjYXVzZSBpdCBuZWVkcyB0byBiZSBkZWZpbmVkIGFzICJ5IiBmb3IgQXJtIHdpdGggdlBD
SSBzdXBwb3J0Lg0KPj4+Pg0KPj4+PiBPdGhlcndpc2UgaXQgYnJlYWtzIHRoZSBQQ0kgcGFzc3Ro
cm91Z2ggZmVhdHVyZSwgZS5nLiBpdCBjb21waWxlcywNCj4+Pj4NCj4+Pj4gYnV0IHRoZSByZXN1
bHRpbmcgYmluYXJ5IGJlaGF2ZXMgd3JvbmcuDQo+Pj4+DQo+Pj4+IERvIHlvdSBzZWUgdGhpcyBh
cyBhbiBhY2NlcHRhYmxlIHNvbHV0aW9uPw0KPj4+IExpa2UgYWxsIChvciBhdCBsZWFzdCB0aGUg
bWFqb3JpdHkpIG9mIG90aGVyIEhBU18qLCBpdCBvdWdodCB0byBiZQ0KPj4+ICJzZWxlY3QiLWVk
IChieSBhcm0vS2NvbmZpZykuIElzIHRoZXJlIGEgcmVhc29uIHRoaXMgaXNuJ3QgcG9zc2libGU/
DQo+Pj4gKEkgZG9uJ3QgbWluZCB0aGUgImRlcGVuZHMgb24iLCBhcyBsb25nIGFzIGl0J3MgY2xl
YXIgdGhhdCBpdCBleGlzdHMNCj4+PiBzb2xlbHkgdG8gYWxsb3cga2NvbmZpZyB0byB3YXJuIGFi
b3V0IGJvZ3VzICJzZWxlY3Qicy4pDQo+PiBUaGVyZSBpcyB5ZXQgbm8gS2NvbmZpZyBleGlzdHMg
KGV2ZW4gZm9yIEFybSkgdGhhdCBlbmFibGVzIEhBU19WUENJLA0KPj4NCj4+IHRodXMgSSBjYW4n
dCBkbyB0aGF0IGF0IHRoZSBtb21lbnQgZXZlbiBpZiBJIHdhbnQgdG8uIFllcywgdGhpcyBjYW4g
YmUgZGVmZXJyZWQNCj4+DQo+PiB0byB0aGUgbGF0ZXIgc3RhZ2Ugd2hlbiB3ZSBlbmFibGUgVlBD
SSBmb3IgQXJtLCBiaXQgdGhpcyBjb25maWcgb3B0aW9uIGlzIHN0aWxsDQo+Pg0KPj4gY29uc2lk
ZXJlZCBhcyAiY29tbW9uIGNvZGUiIGFzIHRoZSBmdW5jdGlvbmFsaXR5IGJlaW5nIGFkZGVkDQo+
Pg0KPj4gdG8gdGhlIGNvbW1vbiBjb2RlIGlzIGp1c3QgZ2F0ZWQgd2l0aCBDT05GSUdfSEFTX1ZQ
Q0lfR1VFU1RfU1VQUE9SVC4NCj4+DQo+PiBXaXRoIHRoaXMgZGVmaW5lZCBub3cgYW5kIG5vdCBs
YXRlciB0aGUgY29kZSBzZWVtcyB0byBiZSBjb21wbGV0ZSBhbmQgbW9yZSBjbGVhbg0KPj4NCj4+
IGFzIGl0IGlzIHNlZW4gd2hhdCBpcyB0aGlzIGNvbmZpZ3VyYXRpb24gb3B0aW9uIGFuZCBob3cg
aXQgaXMgZW5hYmxlZCBhbmQgdXNlZCBpbiB0aGUNCj4+DQo+PiBjb2RlLg0KPj4NCj4+IFNvLCBJ
IHNlZSBubyBwcm9ibGVtIGlmIGl0IGlzIGRlZmluZWQgaW4gdGhpcyBLY29uZmlnIGFuZCBldmFs
dWF0ZXMgdG8gIm4iIGZvciB4ODYNCj4+DQo+PiBhbmQgZXZlbnR1YWxseSB3aWxsIGJlICJ5IiBm
b3IgQXJtIHdoZW4gaXQgZW5hYmxlcyBIQVNfVlBDSS4NCj4gSSdtIGFmcmFpZCBJIGRvbid0IHZp
ZXcgdGhpcyBhcyBhIHJlcGx5IHJlZmxlY3RpbmcgdGhhdCB5b3UgaGF2ZQ0KPiB1bmRlcnN0b29k
IHdoYXQgSSdtIGFza2luZyBmb3IuIFRoZSBwcmltYXJ5IHJlcXVlc3QgaXMgZm9yIHRoZXJlIHRv
DQo+IG5vdCBiZSAiZGVmX2Jvb2wgeSIgYnV0IGp1c3QgImJvb2wiIGFjY29tcGFuaWVkIGJ5IGEg
InNlbGVjdCIgaW4NCj4gQXJtJ3MgS2NvbmZpZy4gSWYgSEFTX1ZQQ0kgZG9lc24ndCBleGlzdCBh
cyBhbiBvcHRpb24geWV0LCBzaW1wbHkNCj4gb21pdCB0aGUgKHF1ZXN0aW9uYWJsZSkgImRlcGVu
ZHMgb24iLg0KSSB1bmRlcnN0b29kIHRoYXQsIGJ1dCB3YXMgdHJ5aW5nIHRvIG1ha2Ugc3VyZSB3
ZSBkb24ndCBtaXNzDQp0aGlzIG9wdGlvbiB3aGlsZSBlbmFibGluZyB2UENJIG9uIEFybSwgYnV0
IG9rLCBJJ2xsIGhhdmUgdGhlIGZvbGxvd2luZzoNCmNvbmZpZyBIQVNfVlBDSQ0KIMKgwqDCoCBi
b29sDQoNCmNvbmZpZyBIQVNfVlBDSV9HVUVTVF9TVVBQT1JUDQogwqDCoMKgIGJvb2wNCiDCoMKg
wqAgZGVwZW5kcyBvbiBIQVNfVlBDSQ0KYW5kIHNlbGVjdCBpdCBmb3IgQXJtIHhlbi9hcmNoL2Fy
bS9LY29uZmlnDQo+DQo+IEphbg0KDQo+IFBTOiBUaGUgbW9yZSByZXBsaWVzIEkgZ2V0IGZyb20g
eW91LCB0aGUgbW9yZSBhbm5veWluZyBJIGZpbmQgdGhlDQo+IGluc2VydGlvbiBvZiBibGFuayBs
aW5lcyBiZXR3ZWVuIGV2ZXJ5IHR3byBsaW5lcyBvZiB0ZXh0LiBCbGFuaw0KPiBsaW5lcyBhcmUg
dXN1YWxseSB1c2VkIHRvIHNlcGFyYXRlIHBhcmFncmFwaHMuIElmIGl0IGlzIHlvdXIgbWFpbA0K
PiBwcm9ncmFtIHdoaWNoIGluc2VydHMgdGhlc2UsIGNhbiB5b3UgcGxlYXNlIHRyeSB0byBkbyBz
b21ldGhpbmcNCj4gYWJvdXQgdGhpcz8gVGhhbmtzLg0KPg0KSSBmaXJzdCB0aG91Z2h0IHRoYXQg
dGhpcyBpcyBob3cgVGh1bmRlcmJpcmQgc3RhcnRlZCBzaG93aW5nDQpteSByZXBsaWVzIGFuZCB3
YXMgYWxzbyBjdXJpb3VzIGFib3V0IHRoZSBkaXN0YW5jZSBiZXR3ZWVuIHRoZSBsaW5lcw0Kd2hp
Y2ggc2VlbWVkIHRvIGJlIGFzIGRvdWJsZS1saW5lLCBidXQgSSBjb3VsZG4ndCBkZWxldGUgb3Ig
ZWRpdA0KdGhvc2UuIEkgdGhvdWdodCB0aGlzIGlzIG9ubHkgdmlzaWJsZSB0byBtZS4uLg0KSXQg
Y2FtZSBvdXQgdGhhdCB0aGlzIHdhcyBiZWNhdXNlIG9mIHNvbWUgVGh1bmRlcmJpcmQgc2V0dGlu
Z3Mgd2hpY2gNCm1hZGUgbXkgcmVwbGllcyB3aXRoIHRob3NlIGRvdWJsZS1saW5lcnMuIEhvcGUg
aXQgaXMgZml4ZWQgbm93Lg0KDQpJIGFtIHJlYWxseSBzb3JyeSBhYm91dCB0aGF0IGFuZCBkbyB1
bmRlcnN0YW5kIGhvdyBhbm5veWluZyBpdCB3YXMuDQoNCkJlc3QgcmVnYXJkcywNCk9sZWtzYW5k
cg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 14:08:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 14:08:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199185.353081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaFi-0007g8-Bv; Wed, 29 Sep 2021 14:08:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199185.353081; Wed, 29 Sep 2021 14:08:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaFi-0007g1-6o; Wed, 29 Sep 2021 14:08:06 +0000
Received: by outflank-mailman (input) for mailman id 199185;
 Wed, 29 Sep 2021 14:08:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lbjb=OT=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVaFh-0007fv-4p
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:08:05 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a8b3ae3a-212e-11ec-bd03-12813bfff9fa;
 Wed, 29 Sep 2021 14:08:03 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-4-P3TJ84vJOou3NOP7P4kGgQ-1; Wed, 29 Sep 2021 16:08:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3389.eurprd04.prod.outlook.com (2603:10a6:803:b::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 14:07:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Wed, 29 Sep 2021
 14:07:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P194CA0010.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:90::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Wed, 29 Sep 2021 14:07:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8b3ae3a-212e-11ec-bd03-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632924482;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ViM8jzRzCOTzWSnpTwG5LntelwydUcoBntssLsOKEvY=;
	b=LhX0yxXckvejhOJwDW3yCXa03Y7B1EkeQUamc3QwoUUYd1sfqoJnXXCasHkjSUAvw7t6Sn
	tEsWRl14aIdbo4YCWcIPO7RdVAWhOPaEFq8sACz2L7wPH0/6SG/okmyE0S2E0bPtMrgluh
	JgyJZFh9XtrIsKplR8afffYyZRGACJA=
X-MC-Unique: P3TJ84vJOou3NOP7P4kGgQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZumrToqLkUlg1Bmp3scytRGpSmGWkrsmRSSFsFvkFnf+8/Vawi4F1/SFMdJN7hTeaTOrh6TFCXkjU5fAgzHaMyz38USM1Q8JRx2vEbz7tTgNurKrvzI+wA4ylOY/lnfnY+Mh3DmUaw+5QfXPxixUlR7Ab0zunNvGHwErvqVUBhZ6DjOQ5xJwfxG6Vbhv4uHiHr6y6bdnGebmgEOyEVneXM+/ZeFOCeomYLfKH5Oyikk1RmjzZZPleYhi9FY0Bc3g4OYn1f054/lfFyYFB7TdUDe/Um3vWKRlDO5sSWFqKX6ABIN83ZyKQtz2YEUiK9+EU+s7GP+TiY32j2UO+eZgYA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=t0yf6sfhCUwuD0kmdu4FIfuByO8IeAI67Hnzb1aLonA=;
 b=mQifHXtb20WrSIt7aFvoJK5CM3LCWYYRVH2Uvw1gKPtiQOTHO5EUAk6oLHIzfkmQP3SO3wlqraY2UjGnkbnjCiKkZxfwuVLzdWT+LUGiLI0h89m2qc/IpKp1XVb1+l0o9bY4TKyD0N/xEpjyBlstMsNq2Ewep3xgE0Oq5nzy4i2+ePiAEa4ucz1AUHXOtyHi/isQJjQHlZ4eM+6qwNUGwPhnOx99SYtMC/p5D1ErGhhnWLkDFiAvJRPuUt8ibq5EXWRk6KtU43EdFNDnfpJFFDyRMAS2r2eaABwXt8rhrqPUAZzfjHVsOR1vSj2CUbvwLUHHTbpOkdRSFwTaGs0/Zw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
CC: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Michal Orzel <michal.orzel@arm.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
 <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
 <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
 <548561a6-f878-cacc-cda2-30508a43072d@suse.com>
 <5dad2004-f507-3610-cdec-cbf26f77cd5a@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3f9850d7-9d48-de59-56b7-fe3202f1764e@suse.com>
Date: Wed, 29 Sep 2021 16:07:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <5dad2004-f507-3610-cdec-cbf26f77cd5a@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM6P194CA0010.EURP194.PROD.OUTLOOK.COM
 (2603:10a6:209:90::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c2a77296-3a58-41d5-2b05-08d983528a67
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3389:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB33898FD7F0686861C405FDF6B3A99@VI1PR0402MB3389.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	l9R6Wq8E5fnmVSU8Z5BDOdkKG3xQRMUpPEsKyysqlu5tZCITOO6e2PKDkHDzp5WZUP8jAjeCS89k7FpkWVEB/4LHWa/zeHi3VtMbHRGYCHfwi3+v4Y+qKTCBz2wSmtmfsFnbn7q5TJhrCYX+4t24ZUp7JUj4f18xRfGvyO3nXs1sb5bDRmaRGvNzigVsUzhooH1U+9vs3yHlVeW7vlWLW9FUKlkvPAyD7X2JCOUB0ZNnMjR1BehPr9ohZoWHVNgNpniDrgzn53t9Q6ur6F8XvUF+t7tDLutBUM/xfQXQF4klZbhwTloLANMIZUNvhw/DnHekHWW5fIjcD57K4aPHJJdHGiaH9PxrI3zS5JjcqAGUyPdiEvMz6MvzYxceUNPDhaRr7vLHMLvRI/9+zSACs7h6wHTUov1fbVspqic7SMEdc7aPOMQh2ovD/WTuelycGf4h00sKjEXQBjgmZfoDsqp0hLqumIeKRGszpnQEHz2qqxN6+avX+iMEfdjKPwHAkb+CNKdbbNahDlPRh+3i0gsn7MmdxP8kng5g5Zb1HfZY6hbo3KVwwI5eOh3cbmWH1IyHYtJrCRyJ+8RrCJY3IOayPTA6xZbLWRvISdIClWwsCjQUFa57IfWUF9/hPAKWTlGGZSgev2OCIB7bpGt24wx2NOiurAqgYPnXg2zXVBhcIDxmWOyfMKeCb6H8iiRz6DLvh5uhbYZKclOXppbn7EGt3/5MqTB9v/jnl1693KI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(2906002)(36756003)(16576012)(31686004)(66556008)(956004)(86362001)(53546011)(4326008)(7416002)(2616005)(54906003)(508600001)(66946007)(6916009)(31696002)(66476007)(316002)(8936002)(5660300002)(38100700002)(8676002)(83380400001)(6486002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Q15VEFBOiKd3ag2LBI0epg92R8WGtfKkAcDRlPQ5WwhUBxBl+zB7UcFveZaX?=
 =?us-ascii?Q?gX8fJeIdCHZdfpUF2vPC+gs+dBoIBnylw2vyr0GIAoBnhzhv/PlKo2/LcH+V?=
 =?us-ascii?Q?TVmuj25iR+iwxM2fpwrIgr5gUluy2iga6+73++Sku6u2e8NQCv7fFfFm4AOg?=
 =?us-ascii?Q?bFOfhkck5xZDPS/x9hJ2WV5MuEVkoFwyaGrr0ogtc8AVGA9Bw/m+Av4efYN4?=
 =?us-ascii?Q?MMjmG1w+ITeCBjmB/mOP5Rt8UEuwmOiVQR9yUs5EPP4Gwu59SSK6cX8KF1Ws?=
 =?us-ascii?Q?mqCj/mbySuxAm/aG01D0UfGc3RJD+mpPExOkGomEAnc66zzl1tKA/WjY1iNj?=
 =?us-ascii?Q?WZO2cu9EmNLyNiq8j+2xV9+bujcHzCa+1o2S+DH7XCeB+O0XdjnrSDZC4K4i?=
 =?us-ascii?Q?2/DVzRpmEL6P0FS1ODrT4NL0sz6ytjnGcOmoo2eAJvbajlxIPL+hPJAO3BtO?=
 =?us-ascii?Q?dtJPWZy3l+fjRGWFPn1WxOv3iTJ2rF6/Li9zyp9MWITyyWprKuvyeHymc4+V?=
 =?us-ascii?Q?qyK73FZ3Pc0kkp50GkvnvUqdnOTgICcDZoC9Cd08a95qmoFCWR1ERX7W8kI0?=
 =?us-ascii?Q?bWgO/bPhaEOYN2G1gSC2UKPL29nLJxBdeR/fkcJNs9Yzd9IixldiaPE5tMJs?=
 =?us-ascii?Q?+E0AcrPxhaqOdvulnlwz6Kh0/IA25g7ByiqsHumENPtIWZn/nbRA/8EAeXuT?=
 =?us-ascii?Q?e3ycKBHnKasplPzGFafeyutF5HO3ef7qDLYxFD2wEbhdJeVxOaXKJxJ/IDe6?=
 =?us-ascii?Q?sT0c/x64123WnTTJb9ZVeBNaeesKWDyeg2OH8JM5l11DZ8wqmJkiAbJTDFUL?=
 =?us-ascii?Q?C8q/ZnI3JxQ+VgI6Exs9ef/u/9hv9D/9Dgtm3UgEqxGE0oT+HYpsZs2Us8N/?=
 =?us-ascii?Q?VDp6DPBpNlFYt5iJ2J9U5pAOoMQobIpXTCVlHq7UX78Yqcr3a5RIMXo8va/g?=
 =?us-ascii?Q?/1VywGRyvp0+rNzmNfmxSaeyeKibJ7Z7HFUmbCs/VJ8c1rK9mk4NaQ66AnYB?=
 =?us-ascii?Q?SrML8dQpUCwy16Yk+21FutnqaPEqqYrOKURFM79LQATDXGFclDaqRXQUQU/2?=
 =?us-ascii?Q?SBU4gWOOD8lPJBriaI6kCw0EewblU2GIDIHt4alOVjPckvB0v9SJ+SXWRGRg?=
 =?us-ascii?Q?aZiC27mSj1dkcVnEpusHEHhRlfS8kdB/o1XDHl3zFBrJhjXfv6WB9+/aDn8x?=
 =?us-ascii?Q?IiagHXXIFQYzc3VkbtWq7Rlhrj7Ky4u1t60JSMblDC3pWNyaH0cb3kYTrW1i?=
 =?us-ascii?Q?A8wjQKeWTfS8tmgVZvtZyQnv6Wng/14ac/CrnwumgXaPfr89sCt9RlQ+jZGE?=
 =?us-ascii?Q?DXx6ae4cO7Del0fwsgjAdsH0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c2a77296-3a58-41d5-2b05-08d983528a67
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 14:07:58.3279
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NJwTrk8a2W7X0x/ZBDUHyN/w/1BOcmNUcVIKDZSkUKVgVMf8gjt0vT3sdISF7BEGcLz85LGl4pBAOD4ZsYpPRw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3389

On 29.09.2021 15:49, Oleksandr Andrushchenko wrote:
>=20
>=20
> On 29.09.21 16:23, Jan Beulich wrote:
>> On 29.09.2021 15:16, Oleksandr Andrushchenko wrote:
>>> On 29.09.21 15:54, Jan Beulich wrote:
>>>> On 29.09.2021 13:56, Oleksandr Andrushchenko wrote:
>>>>> On 29.09.21 12:09, Jan Beulich wrote:
>>>>>> On 29.09.2021 11:03, Oleksandr Andrushchenko wrote:
>>>>>>> Sorry for top posting, but this is a general question on this patch=
/functionality.
>>>>>>>
>>>>>>> Do you see we need to gate all this with CONFIG_HAS_VPCI_GUEST_SUPP=
ORT
>>>>>>> as this renders in somewhat dead code for x86 for now? I do think t=
his still
>>>>>>> needs to be in the common code though.
>>>>>> I agree it wants to live in common code, but I'd still like the code=
 to
>>>>>> not bloat x86 binaries. Hence yes, I think there want to be
>>>>>> "if ( !IS_ENABLED() )" early bailout paths or, whenever this isn't
>>>>>> possible without breaking the build, respective #ifdef-s.
>>>>> Then it needs to be defined as (xen/drivers/Kconfig):
>>>>>
>>>>> config HAS_VPCI_GUEST_SUPPORT
>>>>>    =C2=A0=C2=A0=C2=A0 # vPCI guest support is only enabled for Arm no=
w
>>>>>    =C2=A0=C2=A0=C2=A0 def_bool y if ARM
>>>>>    =C2=A0=C2=A0=C2=A0 depends on HAS_VPCI
>>>>>
>>>>> Because it needs to be defined as "y" for Arm with vPCI support.
>>>>>
>>>>> Otherwise it breaks the PCI passthrough feature, e.g. it compiles,
>>>>>
>>>>> but the resulting binary behaves wrong.
>>>>>
>>>>> Do you see this as an acceptable solution?
>>>> Like all (or at least the majority) of other HAS_*, it ought to be
>>>> "select"-ed (by arm/Kconfig). Is there a reason this isn't possible?
>>>> (I don't mind the "depends on", as long as it's clear that it exists
>>>> solely to allow kconfig to warn about bogus "select"s.)
>>> There is yet no Kconfig exists (even for Arm) that enables HAS_VPCI,
>>>
>>> thus I can't do that at the moment even if I want to. Yes, this can be =
deferred
>>>
>>> to the later stage when we enable VPCI for Arm, bit this config option =
is still
>>>
>>> considered as "common code" as the functionality being added
>>>
>>> to the common code is just gated with CONFIG_HAS_VPCI_GUEST_SUPPORT.
>>>
>>> With this defined now and not later the code seems to be complete and m=
ore clean
>>>
>>> as it is seen what is this configuration option and how it is enabled a=
nd used in the
>>>
>>> code.
>>>
>>> So, I see no problem if it is defined in this Kconfig and evaluates to =
"n" for x86
>>>
>>> and eventually will be "y" for Arm when it enables HAS_VPCI.
>> I'm afraid I don't view this as a reply reflecting that you have
>> understood what I'm asking for. The primary request is for there to
>> not be "def_bool y" but just "bool" accompanied by a "select" in
>> Arm's Kconfig. If HAS_VPCI doesn't exist as an option yet, simply
>> omit the (questionable) "depends on".
> I understood that, but was trying to make sure we don't miss
> this option while enabling vPCI on Arm, but ok, I'll have the following:
> config HAS_VPCI
>  =C2=A0=C2=A0=C2=A0 bool
>=20
> config HAS_VPCI_GUEST_SUPPORT
>  =C2=A0=C2=A0=C2=A0 bool
>  =C2=A0=C2=A0=C2=A0 depends on HAS_VPCI
> and select it for Arm xen/arch/arm/Kconfig

Btw you could also have

config HAS_VPCI
     bool

config HAS_VPCI_GUEST_SUPPORT
     bool
     select HAS_VPCI

which would require arm/Kconfig to only select the latter, while
x86/Kconfig would only select the former.

>> PS: The more replies I get from you, the more annoying I find the
>> insertion of blank lines between every two lines of text. Blank
>> lines are usually used to separate paragraphs. If it is your mail
>> program which inserts these, can you please try to do something
>> about this? Thanks.
>>
> I first thought that this is how Thunderbird started showing
> my replies and was also curious about the distance between the lines
> which seemed to be as double-line, but I couldn't delete or edit
> those. I thought this is only visible to me...
> It came out that this was because of some Thunderbird settings which
> made my replies with those double-liners. Hope it is fixed now.

Indeed, thanks - I did not remove any blank lines from context above.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 14:16:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 14:16:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199191.353092 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaNa-0000ff-6d; Wed, 29 Sep 2021 14:16:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199191.353092; Wed, 29 Sep 2021 14:16:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaNa-0000fY-1Q; Wed, 29 Sep 2021 14:16:14 +0000
Received: by outflank-mailman (input) for mailman id 199191;
 Wed, 29 Sep 2021 14:16:12 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVaNY-0000fS-Gb
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:16:12 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id cc0aaf5e-212f-11ec-bd06-12813bfff9fa;
 Wed, 29 Sep 2021 14:16:11 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18TE2qAn015422;
 Wed, 29 Sep 2021 14:16:06 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bcqkg8kpv-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 14:16:06 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6881.eurprd03.prod.outlook.com (2603:10a6:20b:286::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 29 Sep
 2021 14:16:03 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 14:16:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc0aaf5e-212f-11ec-bd06-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=c9Jo1yfEUNgUGWpk6aXN3zRd3Pi3A24Y+phgANmvl7mTdBLckPGI0Mfa/4O4mkvngEyw7r7G7Wlq6FAO12lsW1m39gBp7qNqnkTxu26jXapFmIxD8U8vHpISDXkQI++qoZi0NEY6DixtFGpleDrV32OZq66/LT9ydm4EAnGYO7nQGqLAWxE7/2w/DpDTSOHKxTZQ3eIGhlMpa2z3BhHw8dfV4wLdNweciUCC8tNq0S0MLD3okQl6xpQ07X0K1fNejsxh6eQYpBSJwFgZqhKa2vkfXhOyB25WeHUKfA6109M9np575SrRXmBFBgtvr//64g3cvB29Q8327Bdv/D4i7Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+4u7yU/MZDWTYXQR8WvZ7C+daVkSYKblBrIWXy9kAhg=;
 b=k9TQRDTF2L/MF+5ZZ3Kd/L80cdoJ1Ao6NGYMbdz0nqzLm2px4eyz16fOwV4xy2iXvGnCsOOqGveJb0KTVKQ5za6QFoP0r9J1HU0QKFlY6jHhT2tgHFYcPRLOvPyvI3YB5a2OHMQm5KxadbJjyB6UaPPECQoP2ZBDQ+jfhUlmrgllJ99GjMKPiMSkASfdJl2gvtOkjYcbgIvedVuaFut9dqMHWHgoh8JH4hwBqPnhqzixVGGqlQBuLetMLeJYjBII2+j814oppFO04qc7bf+ZSmnhLdbUW5gXMr54iG+JLmmQDRhoRpBBDaHHZkAxA4DeEqt36i6AOBc8bp4njo0bvg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+4u7yU/MZDWTYXQR8WvZ7C+daVkSYKblBrIWXy9kAhg=;
 b=YF4hF3JOceYsRHd5r8eF9FqRrox+Hesqzr5iiLVkmkdmDecAsr2i2sO2ly9YsudPyq9vHR7C9sBfkke64rJgdCp9mQq+FhrKjAuxrFSuqQR8aGwl60Gco6O0x8+5UYSu89a2QwUOTfQ9FpDcWpjc0ofRX6sd/mKJf6e1iVcoMOQb8lVW8kpoFKMkbKCW8xNLV50T+yASEG98aW5pvZJhWExQSE5+OhPKxbSlkzijfHcCpV2w0+dGnKjWJXcawkzAnhpDRL67ohu1SkJnbde5JUxWF8hw2bW0ceGzWXb71jTYjR5w6bniZN8T4tPRt8SsrfUSK5DxlIHkkGlo6aNG4Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Michal
 Orzel <michal.orzel@arm.com>
Subject: Re: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v2 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: 
 AQHXsHpEaYKJY1uGNUmlwbBKeBrI6Ku5GYwAgAAC+oCAAAUNAIAATmyAgAFQpACAAAGzgIAALsGAgAAQSwCAAAYBgIAAAhkAgAAHPoCAAAUeAIAAAkQA
Date: Wed, 29 Sep 2021 14:16:03 +0000
Message-ID: <bcc2446a-8384-fa49-d628-63085082d765@epam.com>
References: <20210923125501.234252-1-andr2000@gmail.com>
 <20210923125501.234252-11-andr2000@gmail.com>
 <6a760126-ba6e-bdf7-97a3-205d75d3d99f@arm.com>
 <66cc2b43-eedf-5d58-bf93-5047ecf9ac7b@suse.com>
 <71d8eda6-49f8-691c-1e6b-711ce9f99d50@arm.com>
 <600cc603-5e6c-b430-def3-74a9b596daa0@epam.com>
 <8f5f9b93-2b13-30dc-372b-10326130b71d@epam.com>
 <4bde7d37-6b11-675e-79c1-e6e413fc97f9@suse.com>
 <7402aa24-e203-795b-0ca7-69e55c6b3f4b@epam.com>
 <38d64660-bf9f-98a5-9aef-5a3c39804f7f@suse.com>
 <26c228b1-eb4d-1984-4f24-30dcd24e1cb5@epam.com>
 <548561a6-f878-cacc-cda2-30508a43072d@suse.com>
 <5dad2004-f507-3610-cdec-cbf26f77cd5a@epam.com>
 <3f9850d7-9d48-de59-56b7-fe3202f1764e@suse.com>
In-Reply-To: <3f9850d7-9d48-de59-56b7-fe3202f1764e@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0e034e1b-6e6f-4dcf-51b0-08d98353ab6f
x-ms-traffictypediagnostic: AM9PR03MB6881:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB688104605C5E312E2FF455A5E7A99@AM9PR03MB6881.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 oAj+ymJCRA90DRNYuG+cJdZNa9z1OmJD5MKiUR8R2yzeIoqNtwXJaQzRxrH6jcjG7cBRfZKKXZo+Ep07BQnIT1zd8/gOuon1ApnpOKfLURApfJO9Vv9JwYYBUT68aKwFY3oTG0QMH4tdhfPNe18xulZUBgdhuPpKNP0Pjxkrq4Ys9Cpe1o/mhKo1ZdRK31dcNWOlwJtTx64121h2ZoPPy4BWBdEnrntPU9nLHDAQuC5sOI5oiIHChlUK/Y1J6L6wKIl+8ftgrk6ZBil9QXeaN6N2CCJTiXtpjn2KLNfEmfjCT3QCZCEp6cGhBIwxC2rXwDbVlOS3MGh2l84tr7LtkulnwLMQMtH3ujeRYIZK8hIknBYcLOoZMTH20vBZ6feEHILEa4MZsa59/C1QdG7mhRaaWIyOqukXaKskdXTe6b2xnzcPMb26Pant/SuOt1+Yglpsx7KalSI7ZTSNMo0p1JbeXQ9hcYX3WY6K9WnOuuvxSe79mDcorjZAVVekxLL52nUQzgA2oXPxu+j/mm8NuEXlcqQ8Jplo+z4EH4ODxUlcbA6f6MTPW55hZfaUbzcI/+xc4i28JEA0utzR0l0vuwaAkO3Dfe3EqLWR/9rgwU0DSOrLCu+sLmrA0A3gUDMOW6o58o5ETCJs3WtBPpeB01md8K9rK+aGs+cdCi8QNd+u6n48/grEkbNZhIcocv5XwM6JV/AvgeqaVsKSnjfQDHxBfSDJFfCO/kxXdXNAR+GIv01vREn/Vh6Hc6O5WchF
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(55236004)(6512007)(6916009)(8936002)(91956017)(38070700005)(8676002)(26005)(53546011)(6486002)(66446008)(66556008)(5660300002)(66476007)(66946007)(6506007)(86362001)(186003)(31696002)(71200400001)(64756008)(316002)(122000001)(4326008)(2906002)(36756003)(83380400001)(38100700002)(2616005)(76116006)(508600001)(31686004)(54906003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SWd3amZrMEJLWmIwc3cvQWZ3KzBULzI1QmVRbENodmM1dnF2b1JwaEcvdllr?=
 =?utf-8?B?NTBQQVhCa24yNnErU3JjZGpoNFRrdElZSE9wZ0crY0RCd3dlbzQ4TUlKNVBB?=
 =?utf-8?B?cEs5QWNoQm9URExiUTFxenZtZlBlUi9IUVAzMklGWGNqVitiVnorZHczdjBR?=
 =?utf-8?B?MzlTTVdYOWkrRTc5UUdWYk1MVjg0M3h3TDFxSXJjR1hkYTdjeVdJR3NyKzZH?=
 =?utf-8?B?ZUpyV0t0RkNacSt3L3lFNmNWakcwd3pNYVk2YXhHMldwTVRDSXVOSmRwWXlC?=
 =?utf-8?B?YXd3RWprSWZqTkZKTW1JTldTRG5rNzdNVkQzMXNBWmV2MDlkWW52SFVLUmc2?=
 =?utf-8?B?bTNEQVlQS0E1VUJ6MlRXN1VmbTZwYyt0MklqNWtqcloyclZha3JRdDExbEFM?=
 =?utf-8?B?YUw3R2JVYzFmZG8zUG8rYTExUE1CN1lka3ZmOTBaa0J2TU9obHkwaXQ5WFhJ?=
 =?utf-8?B?VExEbFhlVWxwMWZmVEtpTGU5SldlRFBEOGswUDdsU05CM0VxNDlOeC9GbGF0?=
 =?utf-8?B?aXpwaVFXeWNDZENnTEJsbHI5cS9ncjErbjcxdWlBdDRKUjJjTVFPMzJyTElC?=
 =?utf-8?B?Z3dGcXpWQVRqMGVzTUpxQUdpMFlDdjROblh4WTRVd3Z6ZlZDbnBST29ieWRn?=
 =?utf-8?B?L1VHaFdGV01yRUpHVitYZXhWSW9yK1pTVGo0bERFRWU3OWp0SnhNMkJ4VmlZ?=
 =?utf-8?B?V0VDQWM2ZWp1WTVhVzBHWGYyS1djcGxNaWRsVEZYcFpGUXkrT3JvTHA2T3lB?=
 =?utf-8?B?a29PM0JWdml5RE1jbVJjNlZ5R3BSVjg4ZUJuSjVyOXpFd1QyUmE1VHVRTmJ0?=
 =?utf-8?B?UTBURDY5VEJCYW02QTRzaW1RRm4xWGdvbkZtL0ROMVRvWkxYOFdxVk9RTGlP?=
 =?utf-8?B?QTUwL3RjZXZaR0FEYnJzV04zY0N4VU9QWFZoaU1sQVBrRVRoeC96dWZHN1N6?=
 =?utf-8?B?azNjM3FEdzNaNlg2LzNlcm9QcWhSMkUrWnNYM2MybENOMGlmL3p0bkN4MENW?=
 =?utf-8?B?Wk80Q1oxL1VzbTQ4ZWNHVEJoV0dMV3hOSlpPTHo3cGF5blBaT2VOMkFrdU04?=
 =?utf-8?B?aGMxY2FPb04ydzJMQTRMeVpST3RTT1lZV2pwbWtEQTRHbms4elloamVXN1pm?=
 =?utf-8?B?bXJMUzRsTXRsOEJodThLYko0dlQ0UXRuUzBvelIreGFWdjJQclpzeWxxTHJp?=
 =?utf-8?B?WVV4ZE9JL3BiTm9KOGNSWW1qMTlWNXpkLzkwM0ZpcUsvbDdFMEZWbEpCcytC?=
 =?utf-8?B?OVBiMnY4MmR5RlZSMjhEV2ZvSzh5cGRkRmZmMzlUMndNWDh2R0F5cU5mTTFk?=
 =?utf-8?B?ZkVtRlQvdTd1L25VOTkzRGRSQzU5MkZrT2c4SXVQT2RsMlFJS2QwQUhDdlhY?=
 =?utf-8?B?Mm5GRWRVb2VraGs0NUFzM2ZyR1lVbE5pdlB5eXZlL0w5Q2dUa0tveFA5NUd3?=
 =?utf-8?B?bkY5TDloa0xUTzFQZlJ2QUpDdHMyMUttbXNocmhZbDQ3Nm1GVlQrUUtyK1BW?=
 =?utf-8?B?bzlKMW9JNkpNek54dG5WZEJEZnpnUGJYZU5LZStMOHUyK2hmUzlqTmZiNFp3?=
 =?utf-8?B?YXhvYUtxUGhyaGF0SmlkRjRGbHZSQ3NSOXhHWGVENkhpSThNRWpudEI1cmVV?=
 =?utf-8?B?OWU2SmFuRzl6ZEcwbHpKMUVxRlFEUktqQXV4bXRvSUpLREJjRENVcG94Ujlq?=
 =?utf-8?B?cnNPcklySFJiUnZaK3diVDAxVC9KMzl6Uk5QakZJa0RoWEd0WmZGeWE0QXBp?=
 =?utf-8?B?U3pDK2FKUFdXejJCUkVhTXVBMzlCWmcvV25rZk1vZE9ZYkViOWxVWmJzN1Er?=
 =?utf-8?B?cjNaNzBDN0cxVC85WDk2UT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <77F4102B0E3A0948B184CB7E99708958@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e034e1b-6e6f-4dcf-51b0-08d98353ab6f
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 14:16:03.0134
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: mdDcPzqs1jfknWrqbFuIRJhiPYIrNzxHkq6Iq3etDaru1DL07RvXCo8gvysHNA3HF0ey2SDaBI3vLqJ4+oY+fCvKnDsFCC2O4CFw7gRcaQkDvyWbCbmNyxw5GIV21htu
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6881
X-Proofpoint-GUID: g8gUl5e4e60CbtPbB2TrDBRoSquPX8Fl
X-Proofpoint-ORIG-GUID: g8gUl5e4e60CbtPbB2TrDBRoSquPX8Fl
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_05,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0
 spamscore=0 mlxlogscore=748 adultscore=0 bulkscore=0 malwarescore=0
 clxscore=1015 mlxscore=0 priorityscore=1501 suspectscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290089

DQoNCk9uIDI5LjA5LjIxIDE3OjA3LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMjkuMDkuMjAy
MSAxNTo0OSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pg0KPj4gT24gMjkuMDku
MjEgMTY6MjMsIEphbiBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDI5LjA5LjIwMjEgMTU6MTYsIE9s
ZWtzYW5kciBBbmRydXNoY2hlbmtvIHdyb3RlOg0KPj4+PiBPbiAyOS4wOS4yMSAxNTo1NCwgSmFu
IEJldWxpY2ggd3JvdGU6DQo+Pj4+PiBPbiAyOS4wOS4yMDIxIDEzOjU2LCBPbGVrc2FuZHIgQW5k
cnVzaGNoZW5rbyB3cm90ZToNCj4+Pj4+PiBPbiAyOS4wOS4yMSAxMjowOSwgSmFuIEJldWxpY2gg
d3JvdGU6DQo+Pj4+Pj4+IE9uIDI5LjA5LjIwMjEgMTE6MDMsIE9sZWtzYW5kciBBbmRydXNoY2hl
bmtvIHdyb3RlOg0KPj4+Pj4+Pj4gU29ycnkgZm9yIHRvcCBwb3N0aW5nLCBidXQgdGhpcyBpcyBh
IGdlbmVyYWwgcXVlc3Rpb24gb24gdGhpcyBwYXRjaC9mdW5jdGlvbmFsaXR5Lg0KPj4+Pj4+Pj4N
Cj4+Pj4+Pj4+IERvIHlvdSBzZWUgd2UgbmVlZCB0byBnYXRlIGFsbCB0aGlzIHdpdGggQ09ORklH
X0hBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+Pj4+Pj4+IGFzIHRoaXMgcmVuZGVycyBpbiBzb21l
d2hhdCBkZWFkIGNvZGUgZm9yIHg4NiBmb3Igbm93PyBJIGRvIHRoaW5rIHRoaXMgc3RpbGwNCj4+
Pj4+Pj4+IG5lZWRzIHRvIGJlIGluIHRoZSBjb21tb24gY29kZSB0aG91Z2guDQo+Pj4+Pj4+IEkg
YWdyZWUgaXQgd2FudHMgdG8gbGl2ZSBpbiBjb21tb24gY29kZSwgYnV0IEknZCBzdGlsbCBsaWtl
IHRoZSBjb2RlIHRvDQo+Pj4+Pj4+IG5vdCBibG9hdCB4ODYgYmluYXJpZXMuIEhlbmNlIHllcywg
SSB0aGluayB0aGVyZSB3YW50IHRvIGJlDQo+Pj4+Pj4+ICJpZiAoICFJU19FTkFCTEVEKCkgKSIg
ZWFybHkgYmFpbG91dCBwYXRocyBvciwgd2hlbmV2ZXIgdGhpcyBpc24ndA0KPj4+Pj4+PiBwb3Nz
aWJsZSB3aXRob3V0IGJyZWFraW5nIHRoZSBidWlsZCwgcmVzcGVjdGl2ZSAjaWZkZWYtcy4NCj4+
Pj4+PiBUaGVuIGl0IG5lZWRzIHRvIGJlIGRlZmluZWQgYXMgKHhlbi9kcml2ZXJzL0tjb25maWcp
Og0KPj4+Pj4+DQo+Pj4+Pj4gY29uZmlnIEhBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+Pj4+PiAg
ICAgwqDCoMKgICMgdlBDSSBndWVzdCBzdXBwb3J0IGlzIG9ubHkgZW5hYmxlZCBmb3IgQXJtIG5v
dw0KPj4+Pj4+ICAgICDCoMKgwqAgZGVmX2Jvb2wgeSBpZiBBUk0NCj4+Pj4+PiAgICAgwqDCoMKg
IGRlcGVuZHMgb24gSEFTX1ZQQ0kNCj4+Pj4+Pg0KPj4+Pj4+IEJlY2F1c2UgaXQgbmVlZHMgdG8g
YmUgZGVmaW5lZCBhcyAieSIgZm9yIEFybSB3aXRoIHZQQ0kgc3VwcG9ydC4NCj4+Pj4+Pg0KPj4+
Pj4+IE90aGVyd2lzZSBpdCBicmVha3MgdGhlIFBDSSBwYXNzdGhyb3VnaCBmZWF0dXJlLCBlLmcu
IGl0IGNvbXBpbGVzLA0KPj4+Pj4+DQo+Pj4+Pj4gYnV0IHRoZSByZXN1bHRpbmcgYmluYXJ5IGJl
aGF2ZXMgd3JvbmcuDQo+Pj4+Pj4NCj4+Pj4+PiBEbyB5b3Ugc2VlIHRoaXMgYXMgYW4gYWNjZXB0
YWJsZSBzb2x1dGlvbj8NCj4+Pj4+IExpa2UgYWxsIChvciBhdCBsZWFzdCB0aGUgbWFqb3JpdHkp
IG9mIG90aGVyIEhBU18qLCBpdCBvdWdodCB0byBiZQ0KPj4+Pj4gInNlbGVjdCItZWQgKGJ5IGFy
bS9LY29uZmlnKS4gSXMgdGhlcmUgYSByZWFzb24gdGhpcyBpc24ndCBwb3NzaWJsZT8NCj4+Pj4+
IChJIGRvbid0IG1pbmQgdGhlICJkZXBlbmRzIG9uIiwgYXMgbG9uZyBhcyBpdCdzIGNsZWFyIHRo
YXQgaXQgZXhpc3RzDQo+Pj4+PiBzb2xlbHkgdG8gYWxsb3cga2NvbmZpZyB0byB3YXJuIGFib3V0
IGJvZ3VzICJzZWxlY3Qicy4pDQo+Pj4+IFRoZXJlIGlzIHlldCBubyBLY29uZmlnIGV4aXN0cyAo
ZXZlbiBmb3IgQXJtKSB0aGF0IGVuYWJsZXMgSEFTX1ZQQ0ksDQo+Pj4+DQo+Pj4+IHRodXMgSSBj
YW4ndCBkbyB0aGF0IGF0IHRoZSBtb21lbnQgZXZlbiBpZiBJIHdhbnQgdG8uIFllcywgdGhpcyBj
YW4gYmUgZGVmZXJyZWQNCj4+Pj4NCj4+Pj4gdG8gdGhlIGxhdGVyIHN0YWdlIHdoZW4gd2UgZW5h
YmxlIFZQQ0kgZm9yIEFybSwgYml0IHRoaXMgY29uZmlnIG9wdGlvbiBpcyBzdGlsbA0KPj4+Pg0K
Pj4+PiBjb25zaWRlcmVkIGFzICJjb21tb24gY29kZSIgYXMgdGhlIGZ1bmN0aW9uYWxpdHkgYmVp
bmcgYWRkZWQNCj4+Pj4NCj4+Pj4gdG8gdGhlIGNvbW1vbiBjb2RlIGlzIGp1c3QgZ2F0ZWQgd2l0
aCBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVC4NCj4+Pj4NCj4+Pj4gV2l0aCB0aGlzIGRl
ZmluZWQgbm93IGFuZCBub3QgbGF0ZXIgdGhlIGNvZGUgc2VlbXMgdG8gYmUgY29tcGxldGUgYW5k
IG1vcmUgY2xlYW4NCj4+Pj4NCj4+Pj4gYXMgaXQgaXMgc2VlbiB3aGF0IGlzIHRoaXMgY29uZmln
dXJhdGlvbiBvcHRpb24gYW5kIGhvdyBpdCBpcyBlbmFibGVkIGFuZCB1c2VkIGluIHRoZQ0KPj4+
Pg0KPj4+PiBjb2RlLg0KPj4+Pg0KPj4+PiBTbywgSSBzZWUgbm8gcHJvYmxlbSBpZiBpdCBpcyBk
ZWZpbmVkIGluIHRoaXMgS2NvbmZpZyBhbmQgZXZhbHVhdGVzIHRvICJuIiBmb3IgeDg2DQo+Pj4+
DQo+Pj4+IGFuZCBldmVudHVhbGx5IHdpbGwgYmUgInkiIGZvciBBcm0gd2hlbiBpdCBlbmFibGVz
IEhBU19WUENJLg0KPj4+IEknbSBhZnJhaWQgSSBkb24ndCB2aWV3IHRoaXMgYXMgYSByZXBseSBy
ZWZsZWN0aW5nIHRoYXQgeW91IGhhdmUNCj4+PiB1bmRlcnN0b29kIHdoYXQgSSdtIGFza2luZyBm
b3IuIFRoZSBwcmltYXJ5IHJlcXVlc3QgaXMgZm9yIHRoZXJlIHRvDQo+Pj4gbm90IGJlICJkZWZf
Ym9vbCB5IiBidXQganVzdCAiYm9vbCIgYWNjb21wYW5pZWQgYnkgYSAic2VsZWN0IiBpbg0KPj4+
IEFybSdzIEtjb25maWcuIElmIEhBU19WUENJIGRvZXNuJ3QgZXhpc3QgYXMgYW4gb3B0aW9uIHll
dCwgc2ltcGx5DQo+Pj4gb21pdCB0aGUgKHF1ZXN0aW9uYWJsZSkgImRlcGVuZHMgb24iLg0KPj4g
SSB1bmRlcnN0b29kIHRoYXQsIGJ1dCB3YXMgdHJ5aW5nIHRvIG1ha2Ugc3VyZSB3ZSBkb24ndCBt
aXNzDQo+PiB0aGlzIG9wdGlvbiB3aGlsZSBlbmFibGluZyB2UENJIG9uIEFybSwgYnV0IG9rLCBJ
J2xsIGhhdmUgdGhlIGZvbGxvd2luZzoNCj4+IGNvbmZpZyBIQVNfVlBDSQ0KPj4gICDCoMKgwqAg
Ym9vbA0KPj4NCj4+IGNvbmZpZyBIQVNfVlBDSV9HVUVTVF9TVVBQT1JUDQo+PiAgIMKgwqDCoCBi
b29sDQo+PiAgIMKgwqDCoCBkZXBlbmRzIG9uIEhBU19WUENJDQo+PiBhbmQgc2VsZWN0IGl0IGZv
ciBBcm0geGVuL2FyY2gvYXJtL0tjb25maWcNCj4gQnR3IHlvdSBjb3VsZCBhbHNvIGhhdmUNCj4N
Cj4gY29uZmlnIEhBU19WUENJDQo+ICAgICAgIGJvb2wNCj4NCj4gY29uZmlnIEhBU19WUENJX0dV
RVNUX1NVUFBPUlQNCj4gICAgICAgYm9vbA0KPiAgICAgICBzZWxlY3QgSEFTX1ZQQ0kNCj4NCj4g
d2hpY2ggd291bGQgcmVxdWlyZSBhcm0vS2NvbmZpZyB0byBvbmx5IHNlbGVjdCB0aGUgbGF0dGVy
LCB3aGlsZQ0KPiB4ODYvS2NvbmZpZyB3b3VsZCBvbmx5IHNlbGVjdCB0aGUgZm9ybWVyLg0KSSds
bCBwcm9iYWJseSBsZWF2ZSBpdCBhcyBJIHdyb3RlIGJlZm9yZSwgYmVjYXVzZSBpdCB0aGVuIGxv
b2tzIGxpa2UNCmEgc3ViLWZlYXR1cmUgZW5hYmxlcyB0aGUgcGFyZW50IGZlYXR1cmUgYW5kIGRv
ZXNuJ3Qgc2VlbSByaWdodA0KQWx0aG91Z2ggaXQgbWF5IHN0aWxsIGxvb2sgcmlnaHQuLi4NCj4N
Cj4+PiBQUzogVGhlIG1vcmUgcmVwbGllcyBJIGdldCBmcm9tIHlvdSwgdGhlIG1vcmUgYW5ub3lp
bmcgSSBmaW5kIHRoZQ0KPj4+IGluc2VydGlvbiBvZiBibGFuayBsaW5lcyBiZXR3ZWVuIGV2ZXJ5
IHR3byBsaW5lcyBvZiB0ZXh0LiBCbGFuaw0KPj4+IGxpbmVzIGFyZSB1c3VhbGx5IHVzZWQgdG8g
c2VwYXJhdGUgcGFyYWdyYXBocy4gSWYgaXQgaXMgeW91ciBtYWlsDQo+Pj4gcHJvZ3JhbSB3aGlj
aCBpbnNlcnRzIHRoZXNlLCBjYW4geW91IHBsZWFzZSB0cnkgdG8gZG8gc29tZXRoaW5nDQo+Pj4g
YWJvdXQgdGhpcz8gVGhhbmtzLg0KPj4+DQo+PiBJIGZpcnN0IHRob3VnaHQgdGhhdCB0aGlzIGlz
IGhvdyBUaHVuZGVyYmlyZCBzdGFydGVkIHNob3dpbmcNCj4+IG15IHJlcGxpZXMgYW5kIHdhcyBh
bHNvIGN1cmlvdXMgYWJvdXQgdGhlIGRpc3RhbmNlIGJldHdlZW4gdGhlIGxpbmVzDQo+PiB3aGlj
aCBzZWVtZWQgdG8gYmUgYXMgZG91YmxlLWxpbmUsIGJ1dCBJIGNvdWxkbid0IGRlbGV0ZSBvciBl
ZGl0DQo+PiB0aG9zZS4gSSB0aG91Z2h0IHRoaXMgaXMgb25seSB2aXNpYmxlIHRvIG1lLi4uDQo+
PiBJdCBjYW1lIG91dCB0aGF0IHRoaXMgd2FzIGJlY2F1c2Ugb2Ygc29tZSBUaHVuZGVyYmlyZCBz
ZXR0aW5ncyB3aGljaA0KPj4gbWFkZSBteSByZXBsaWVzIHdpdGggdGhvc2UgZG91YmxlLWxpbmVy
cy4gSG9wZSBpdCBpcyBmaXhlZCBub3cuDQo+IEluZGVlZCwgdGhhbmtzIC0gSSBkaWQgbm90IHJl
bW92ZSBhbnkgYmxhbmsgbGluZXMgZnJvbSBjb250ZXh0IGFib3ZlLg0KPg0KPiBKYW4NCj4NCj4N
ClRoYW5rIHlvdSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 14:17:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 14:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199201.353102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaOs-0001Tv-KE; Wed, 29 Sep 2021 14:17:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199201.353102; Wed, 29 Sep 2021 14:17:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaOs-0001To-HI; Wed, 29 Sep 2021 14:17:34 +0000
Received: by outflank-mailman (input) for mailman id 199201;
 Wed, 29 Sep 2021 14:17:33 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM9C=OT=epam.com=prvs=09065c5b4f=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVaOr-0001Tc-5V
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:17:33 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e1a0f609-ca1b-4009-9885-6d1d8517992f;
 Wed, 29 Sep 2021 14:17:32 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TE4EU1014541; 
 Wed, 29 Sep 2021 14:17:27 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2058.outbound.protection.outlook.com [104.47.0.58])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bcnh092qf-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 14:17:27 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6770.eurprd03.prod.outlook.com (2603:10a6:20b:283::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Wed, 29 Sep
 2021 14:17:23 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 14:17:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1a0f609-ca1b-4009-9885-6d1d8517992f
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nQ/vomCP6LnhCRDZOuHCha0zBCQo1/QjdeblUeHHjaji/mnFUsZYxkRWgqURIZ4XPXG/vIdZEcX+MOc9mQ48Bnw1FCBtCDgReSmBFoFzWDhRj7ITI2sLdh7eRw2DIk7mYaFdRLLk7bf0adhV+Ug43Yd8UAYxG2bucfOyHNFVZkKLINNdPfABV0B7MEid/osnJM5hchEHUIkepGVCJ4tJWzv6IRs5OlH9GY1iOaYqwvch6DcqoTQoWHQ0Nr+jPv8Uqi1fGR13MViobKO75YKAQaDfB820RbffJORjYNPJYS7IMAxtsL5NGJH4dBc8hboWXNAKU8pss4/r5F0cWjtldw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=0owqIExO+hT7G+jzgy+DubfnoFP+udZump4PClvN/xs=;
 b=QQVDLnmYQjn/yIeCyulg35iA4G8iA/OWI4d8Unz9s6tkhiNgBntx7Z2YBsQdSPsg2M91Rx5PYOGup90fz365K6AV9ul4MGjQIPaokiqe9kW1tR0hcx+F3ouxY5HP+NE8BlJylDEBDsT8HW/709UApBK2IGQqofXcwmd3hLFCcZ11o2u+UV0oP6T3xo9BdYAbMSPk9wZesmtCHIqTleMvHYkL528ZwMNlDlXqonUqE8JzRj3sjtgVhANXqDXEVPrr4el/154+SlxXeSetL1RcXVyZVcKKVp6E2Te+5XmeihdHtYnvBXcXmtZ2DQCJqjN5711r7XqFrBe5LbUu2nqdLw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=0owqIExO+hT7G+jzgy+DubfnoFP+udZump4PClvN/xs=;
 b=ZvhUdigFzf8Bthm+DjbT1oPgrlSXkNxC8rUCxBwVU2hfyiwFVB3syv6NcAQIFg07o1OIyUK9GaF+EU5df9ZcRZxVqJFz8Xm8vGio/b+L2cp7QCoFjkXwLOl6MUkJkWAi5QfRJ3YGhLGJq9laiJaUwI8rXuE7m/Ohl470+CAjt8/sgwALrGExTl2slGPMhNrA1gGmFHrZEkehPDVt9uUHUVu3Ic7t2gJdMOHYHxNDiCH4Kk835IVPni9nW63WSmkVZKO4Gg1pnekQWG0MhbMfpGv1Non29i3avAh5BobJcdVcawmVsgRRPoca947I7/pxldGIsO7oiQ91jSkxg/4WbQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Bertrand Marquis <bertrand.marquis@arm.com>,
        Rahul Singh
	<rahul.singh@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>,
        Andre Przywara
	<Andre.Przywara@arm.com>,
        Oleksandr Andrushchenko
	<Oleksandr_Andrushchenko@epam.com>,
        Stefano Stabellini
	<sstabellini@kernel.org>,
        Julien Grall <julien@xen.org>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXtJXcmX/LQUbr3kiBv6S+hyMHIKu7BXCAgAAK1YA=
Date: Wed, 29 Sep 2021 14:17:23 +0000
Message-ID: <47721cc7-985d-9b22-4ab5-bce258a34703@epam.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
 <02F22926-2A4A-4301-A975-4BB2EFD5653F@arm.com>
In-Reply-To: <02F22926-2A4A-4301-A975-4BB2EFD5653F@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0f50e100-4b53-4d2a-609c-08d98353db7a
x-ms-traffictypediagnostic: AM9PR03MB6770:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB677053DC8C484B29E9204D9AE7A99@AM9PR03MB6770.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 Av/qx6w3jFpoe2klk8YNuad6jsAI8dgZ0UXWRWbNScxIJDrYp4pPHEUsEPMcBsTS27Vl6UJtgq1UHiBNNR4kl7sSWkPJqmR2c1odll4vsbroar8AgW8I8MqIpLVYmWXBGnYOAX9izv+ke8Hb+UAmb3bgabsFopAcW+2cqtWsrHhEsXMlO7xcO3JMzk2LHmAPeMT9CAbzs1uTWUObFez5fikRRKAHEV1nTJDTify0ngZlf289Y1iQPmDVfVPF9j2Gs7EPc0Ze9eGTstnvDTA7OAYR5mcK2seEhHcLAF4vMMUEJ+uGIk1GkPHDWgyYvI5GAM+CEWX0A6r3VXf9Kl0X3TiFzV6Z0ZflEUaFlDow3I4Nf2LEEY/hWCqdoPVCsoh508TXquNmWuaqQ4kGsa6ZnJHX6ZYKINFBzZgCnEyRFaib4jqXDVwtg1D4M0qOlDO2aN1XaQZ4s512TywdfS6lhrShEg2ovTVxBuElIYUCiPU1tWb9C6sjH1qvO3/MxcM2SM7uKPWjjmuDFXYuoHLT1WJNhq84pN8WboGBbTrkBHpgtVZvCG8aRcL0LMCUjCnd20M/4/A3M7RvScQl6wfBuywxJZsxXZ9S8MjE3VBTTt2mq5yQywrmt9szhn5EorD949Ho3EjmGC/P4c4gsC2qCKmAq0h9RzbvCFNQjsEhSFBpe3ARr+37/6jdJWTu0JNzKMlE7mFheI6UcefaQvKIhuIf2RVlkgwrQMEZz+Usg20=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(110136005)(4326008)(66946007)(54906003)(66556008)(66476007)(4744005)(76116006)(91956017)(2616005)(316002)(122000001)(38100700002)(36756003)(64756008)(6486002)(66446008)(5660300002)(8936002)(31696002)(6506007)(2906002)(55236004)(186003)(86362001)(508600001)(26005)(8676002)(53546011)(38070700005)(71200400001)(107886003)(6512007)(31686004)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?M2o4RFBBK2E0eWZ5aG1YOWlhSWh3cWRKWkVKdGlrMHR1dExUeVZPbmpiWHhQ?=
 =?utf-8?B?ZGNib3h0R1dmMTJiRkZMczVHUlJZeTFlNGtrQzUyamZwSDdZU1BsY0lPU3p3?=
 =?utf-8?B?djQwWDhYaWVWdFdZVFRiRHROLy8xTU1pTzBGeUFvZEhMZHd0VnBXRWEzamo1?=
 =?utf-8?B?dk4vM0MxNTU4NFd6NFZrNXB6bzRuOFFCdklYb0hlYmMyRTZsTVo1S2xWUSs4?=
 =?utf-8?B?VTZWTW45cERGRFFDM2FpQkFqVW4wdUUzdmk5L0F3ZVI5UldaZTEyNzVwRXlv?=
 =?utf-8?B?WGJDcENpSjJ1UnNCdG8zajM4V2U5RWlQMmh6dmFXZWNDOEx1aG9CWllsOEw0?=
 =?utf-8?B?M3Uvak42VXJFWFR3TVUyRmY0bmhoYVNPT0ZjSCtlQ2x2ekYzMXJQYkNFRFcr?=
 =?utf-8?B?K3lYejB4VGRNSllyT2ZGSUl2aHhhQkNodkpnWHFmMjdTb1dybUVEalJ0amE0?=
 =?utf-8?B?cHBzd2ZoczlScWJMODA2eTI5UkNxdVJORE9TQ0VKMmRUeThkaE0rZlFvRUdS?=
 =?utf-8?B?cWVvMHBOMTg4MUhxd2dnaG1SZjlXSWNPbVRnd1dwVHdhM3c1UTZUN2dKZWc0?=
 =?utf-8?B?K2txdWJNNjFETHh5TXZuVHZFdEY4emNDSUgrL3Q3Qk5mOElsTDJiaVplZlRq?=
 =?utf-8?B?QVlJRk1RV0t2TDhXR1ZEcmE2YnpyWVZLOEcvTjVMNHhaencza3JLZmFxeDFV?=
 =?utf-8?B?WDNyd2pXQVoxYVVMcnpab0xqVzR4dkxLVGNOM3B4TGlFU3Jib01jZW5FeFBJ?=
 =?utf-8?B?VFBrNjVzdGtWaFpLMmdEK045QUxsVlQ2WG1jVWxoY0JPcXdkdFlnWk1GSzRC?=
 =?utf-8?B?aDBkRytST2d3K0VPcVFpZXpUUWdJVjNrRy93RTR5WUIwTktUa1lnMEVkeEtF?=
 =?utf-8?B?WTgzVDArT3hRam5qeXcwNTVJNTY1NlJhUEdNbVQ3a1R2d00vQ2o1aEZ5SUJP?=
 =?utf-8?B?VkpuaUR3M0RSa3VBRDJySzdENTBlb2VtSis1OFFiZm10Y3NvSUpvanhKc2ho?=
 =?utf-8?B?a2V1SGlxR0ZOWVM0a21CVW5zMERoTmFmd3BwUlFoWHREMEhicDVTZm1DUFBn?=
 =?utf-8?B?YSsvZWhITHB0Nmtkd01wYXlQTVZaUTUzd0wrNTFqN3lDc0ZnTXRhUllodExC?=
 =?utf-8?B?YW5aZWRPZmJ1OVl2T216eVg4RFVUMkJHSHdaUGY3NXYya3lrMDF5TUJoSlpk?=
 =?utf-8?B?MW8yNXNtVlNLa01jdkRBOTNrUXRzMmJnNG5VT2NnQ3pEUHFrcC9Zb3VIaGRE?=
 =?utf-8?B?RjdacjhsK2hCV2E2ekRpbGlWV3RYbFBpYm9wclJtQ3dHV243YmhQWFUwMFdP?=
 =?utf-8?B?TzVQQ0dUVlliUVA2L0JRZ1FVVHNsNmY0QVhYVHV5bVh6aFJoRUJrWXpXYko4?=
 =?utf-8?B?SFRYWk95Sy90a3BHRHNYVWMrWlM4TDN2dVhHdTB0TENmU05sSWRMRWprdVVw?=
 =?utf-8?B?YVlEQlgyakh0UUFsNG1DdUFaOTJmUUttM3Z1dnlGcklxVWJjaXg4MisyZGZO?=
 =?utf-8?B?U3oxL0QyZlJrcFdOVFUvaWlSWkFvS3FyZ3dIb04xcVZxa2dURTM4d3N1citT?=
 =?utf-8?B?S1ZYTm5BZyttOWxpY0V2RzRvWkQ2VkxUc25qK0FXcmhoQVZIa2VDUWJxVEt0?=
 =?utf-8?B?S0NaNWNKY20vRDJ6ck1jaUQ4VVlaWXlVVVZ0VGpyWVIzUnZvZmZuMmpDQmxR?=
 =?utf-8?B?dVFsVThNZjUrZHpLcmJsQm1GL2J2SU9vWnRzcnNsdTU4dWh6aGJJYU8vVy8w?=
 =?utf-8?B?KzI3NjMza0F3M1lIUjcxSW0zNGlCUDByZ3dMNzg1ZWNvZXJwTXpoSHQ5SnJV?=
 =?utf-8?B?bmJXdXNmem9aVHY1cnRQUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <6661A53446ADBE419EA05F4E98A2F600@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f50e100-4b53-4d2a-609c-08d98353db7a
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2021 14:17:23.6026
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: A+lIWA9B/U6nfB+gz3HxGLupi8v29oSwrOiX2bjXStR5laPqp9X6etMtJlsq+XFMwDuSbNllSgTyyQCKv52gZpfhM28wRVy9ni5fFvrB5g9+GhmUO5hymcwXwb0MSFEd
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6770
X-Proofpoint-GUID: CyS-Zmdt50qatZ5zLzqvaBLueHOWcr6Q
X-Proofpoint-ORIG-GUID: CyS-Zmdt50qatZ5zLzqvaBLueHOWcr6Q
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_05,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=530
 phishscore=0 priorityscore=1501 impostorscore=0 mlxscore=0 clxscore=1015
 lowpriorityscore=0 spamscore=0 malwarescore=0 suspectscore=0 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109290089

DQoNCk9uIDI5LjA5LjIxIDE2OjM4LCBCZXJ0cmFuZCBNYXJxdWlzIHdyb3RlOg0KPiBIaSBSYWh1
bCwNCj4NCj4+IE9uIDI4IFNlcCAyMDIxLCBhdCAxOToxOCwgUmFodWwgU2luZ2ggPHJhaHVsLnNp
bmdoQGFybS5jb20+IHdyb3RlOg0KPj4NCj4+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hlbmtv
IDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4+DQo+PiBBZGQgc3VwcG9ydCBm
b3IgWGlsaW54IFp5bnFNUCBQQ0kgaG9zdCBjb250cm9sbGVyIHRvIG1hcCB0aGUgUENJIGNvbmZp
Zw0KPj4gc3BhY2UgdG8gdGhlIFhFTiBtZW1vcnkuDQo+Pg0KPj4gUGF0Y2ggaGVscHMgdG8gdW5k
ZXJzdGFuZCBob3cgdGhlIGdlbmVyaWMgaW5mcmFzdHJ1Y3R1cmUgZm9yIFBDSQ0KPj4gaG9zdC1i
cmlkZ2UgZGlzY292ZXJ5IHdpbGwgYmUgdXNlZCBmb3IgZnV0dXJlIHJlZmVyZW5jZXMuDQo+Pg0K
Pj4gU2lnbmVkLW9mZi1ieTogT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gPG9sZWtzYW5kcl9hbmRy
dXNoY2hlbmtvQGVwYW0uY29tPg0KPiBBcyB5b3UgYXJlIHN1Ym1pdHRpbmcgdGhpcyBwYXRjaCBm
cm9tIE9sZWtzYW5kcjoNCj4gLSBpZiB5b3UgbW9kaWZpZWQgaXQgeW91IHNob3VsZCBhZGQgeW91
ciBzaWduZWQtb2ZmDQpZZXMsIEkgdGhpbmsgdGhhdCBSYWh1bCdzIFMtby1iIGlzIHJlcXVpcmVk
IGhlcmUNCg0KVGhhbmsgeW91LA0KT2xla3NhbmRy


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 14:29:14 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 14:29:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199208.353114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaa3-000393-N4; Wed, 29 Sep 2021 14:29:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199208.353114; Wed, 29 Sep 2021 14:29:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVaa3-00038w-KA; Wed, 29 Sep 2021 14:29:07 +0000
Received: by outflank-mailman (input) for mailman id 199208;
 Wed, 29 Sep 2021 14:29:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVaa2-00038q-VS
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:29:06 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVaa2-0001rQ-R7
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:29:06 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVaa2-0006bf-Q8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 14:29:06 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVaZz-0007Px-Gw; Wed, 29 Sep 2021 15:29:03 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:Cc:To:Date
	:Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=5pPaFIJaKwGfQmy5oTRo+z107oHebTGHXSB/MZIjSso=; b=o3pHgr3cYAd3fYtNTkLReMxw6K
	RoJAhTy+5lGdr0F5i+tK13ulcSi/4hml+/LHvbf+D4q0J+LIOfPcIwhtkMw0+YzWL+NEDfK0cfGpD
	XXCSe7rShL3quq9uXbTbMNAnG8YVAS4QurPKF0KKupdjkJVQfqvx91EnX9liCEv+NvCI=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24916.30767.245601.814518@mariner.uk.xensource.com>
Date: Wed, 29 Sep 2021 15:29:03 +0100
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: <xen-devel@lists.xenproject.org>,
    Jan Beulich <jbeulich@suse.com>,
    Andrew Cooper <andrew.cooper3@citrix.com>,
    Wei Liu <wl@xen.org>,
    George Dunlap <george.dunlap@citrix.com>,
    Julien Grall <julien@xen.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    Daniel De Graaf <dgdegra@tycho.nsa.gov>,
    "Daniel P.  Smith" <dpsmith@apertussolutions.com>
Subject: Re: [XEN PATCH v2] build: add --full to version.sh to guess $(XEN_FULLVERSION)
In-Reply-To: <20210909143306.466836-1-anthony.perard@citrix.com>
References: <20210909143306.466836-1-anthony.perard@citrix.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Anthony PERARD writes ("[XEN PATCH v2] build: add --full to version.sh to guess $(XEN_FULLVERSION)"):
> Running $(MAKE) like that in a $(shell ) while parsing the Makefile
> doesn't work reliably. In some case, make will complain with
> "jobserver unavailable: using -j1.  Add '+' to parent make rule.".
> Also, it isn't possible to distinguish between the output produced by
> the target "xenversion" and `make`'s own output.
> 
> Instead of running make, this patch "improve" `version.sh` to try to
> guess the output of `make xenversion`.
> 
> In order to have version.sh works in more scenario, it will use
> XEN_EXTRAVERSION and XEN_VENDORVERSION from the environment when
> present. As for the cases were those two variables are overridden by a
> make command line arguments, we export them when invoking version.sh
> via a new $(XEN_FULLVERSION) macro.
> 
> That should hopefully get us to having ./version.sh returning the same
> value that `make xenversion` would.
> 
> This fix GitLab CI's build job "debian-unstable-gcc-arm64".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Ian Jackson <iwj@xenproject.org>


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 15:01:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 15:01:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199218.353128 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVb4k-0007Ww-8P; Wed, 29 Sep 2021 15:00:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199218.353128; Wed, 29 Sep 2021 15:00:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVb4k-0007Wp-5L; Wed, 29 Sep 2021 15:00:50 +0000
Received: by outflank-mailman (input) for mailman id 199218;
 Wed, 29 Sep 2021 15:00:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YPmi=OT=oracle.com=boris.ostrovsky@srs-us1.protection.inumbo.net>)
 id 1mVb4j-0007Wi-5S
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 15:00:49 +0000
Received: from mx0b-00069f02.pphosted.com (unknown [205.220.177.32])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 06daa0b8-2136-11ec-bd0f-12813bfff9fa;
 Wed, 29 Sep 2021 15:00:47 +0000 (UTC)
Received: from pps.filterd (m0246630.ppops.net [127.0.0.1])
 by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TEeOlJ013625; 
 Wed, 29 Sep 2021 15:00:35 GMT
Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80])
 by mx0b-00069f02.pphosted.com with ESMTP id 3bcf6cuubc-3
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 15:00:35 +0000
Received: from pps.filterd (userp3030.oracle.com [127.0.0.1])
 by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18TECWx9088712;
 Wed, 29 Sep 2021 14:22:13 GMT
Received: from nam10-bn7-obe.outbound.protection.outlook.com
 (mail-bn7nam10lp2107.outbound.protection.outlook.com [104.47.70.107])
 by userp3030.oracle.com with ESMTP id 3bc3bk02wy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 14:22:13 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com (2603:10b6:208:321::10)
 by MN2PR10MB4256.namprd10.prod.outlook.com (2603:10b6:208:1db::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Wed, 29 Sep
 2021 14:22:11 +0000
Received: from BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::3c49:46aa:83e1:a329]) by BLAPR10MB5009.namprd10.prod.outlook.com
 ([fe80::3c49:46aa:83e1:a329%5]) with mapi id 15.20.4566.015; Wed, 29 Sep 2021
 14:22:11 +0000
Received: from [10.74.99.231] (160.34.89.231) by
 BLAPR03CA0135.namprd03.prod.outlook.com (2603:10b6:208:32e::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Wed, 29 Sep 2021 14:22:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06daa0b8-2136-11ec-bd0f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc :
 references : from : message-id : date : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=corp-2021-07-09;
 bh=cNa49YG+pdjys2nA8WWc5jfkEudTq1iUfeKyNbDTooA=;
 b=cXvBmSyj3W/jJmpWwvYYGVd3eFK3lprQnDCXVCKTZIyqZJXuGZl6cizj0X7lwX9gIO7G
 uWjqjrrhRfXkgvbNOjR3YymhskZXOsfpqI0L1YwuAyHMr1TNftuZnGbqsbMSi8vcZk3L
 H85+QVpdYCiTG832o8IIuazK8xDnmIbiTSyZ1+ZhzIWgo+weq8y7AQLfJV5wNstR8N7z
 +RnJNrjA84pHeF2GhS2LgMvbcHMO99TJ049qnCzlfBkEI0Dj6O5Vei3z4vSg7dMVUJu4
 opSfQRizy4esAqaOGK0tu5gxkDihguNN/POKhdBnH+ouTXOmPo+fSRx7JmVOu754TIIU Zg== 
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=j7sLU61+kNwi9sNIJFY0/QXNmz8DombIWW9jFiaMNA5Os5KPFWUVGKhnbqQewtpy4GXLrL7gGJItiDXfBrIx7/Jot22pdYgl2PrEOhasPZFo8gNiERQGusxkKcg4vA5AZ67oD8fTLf4/UvSt3tgkWmUrWurjzGkn4ua7L7nBLZ8gB907kOnv9rzSTxJem5nL9Ah866TaAsDojn7oqCocij28APZsopH0HIJ3R1AE6Vrs0BSYU8XZkwKf9QS2qZ0m+lH5+rhZ82ft8Y+73fOLHwcU7yzig4J+KrWsxCsAlL5dmyafoiTH72ZgnzZqxaig7aq/06RCfvIwldEfTLiZIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=cNa49YG+pdjys2nA8WWc5jfkEudTq1iUfeKyNbDTooA=;
 b=fmNe3Dki4C+OZI58WwwDszwu0c7DwdWod7h+d1VI8YztFtcwuOcFejDSgsmvDrX+Mz+Y92IqPXu3V079WWHQmoT3iwqbJvJXhNKDLlvVuDiiaSHvxG7eWpsQn6hoTAOqild3NUuopqWee3URYquoMqjTyTGYkJpPGFqFDU/cCfLsrWFwqFgnS3B6x+BA0dSuuqVwrgBoCe1MfjDmdbpAb0IuksVwsnX/A2y4l+eKAgyLVjaThltQz4sRURqadErO6tauqyuYYTNXyEzEE3SVe4ACIXS9xrPjm9GHcyn7mZZ6Ztp2SnxZl81PuPCsBrDUASVNjBcmcaVZVbBNofb1qw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com;
 dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cNa49YG+pdjys2nA8WWc5jfkEudTq1iUfeKyNbDTooA=;
 b=DBTLHDlrMvC3RoiGNYmbJxRlmQFT0n2qh1WLIz7chzntd7pK17MgfQNQ9tfwzAYxHTj/mVCgtCPKrYpS5Po9qqQQ7Xn4x9U9N2nCLIFHyahZlFtztOliaqTwNxfFwAn2voUL0Lz2f5iELBTP+FX2fR0B6uhiaRVUODZpD636nW8=
Authentication-Results: kvack.org; dkim=none (message not signed)
 header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com;
Subject: Re: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
        Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
        Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
        Juergen Gross <jgross@suse.com>,
        Stefano Stabellini
 <sstabellini@kernel.org>,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
        Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
        Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
        Mike Rapoport <rppt@kernel.org>,
        "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
        xen-devel@lists.xenproject.org,
        virtualization@lists.linux-foundation.org, kexec@lists.infradead.org,
        linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-3-david@redhat.com>
 <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
 <bfe72f46-9a0d-1a87-64bd-4b03999edd1e@redhat.com>
 <e9a230f9-85cb-d4c1-8027-508b7c344d94@redhat.com>
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <3b935aa0-6d85-0bcd-100e-15098add3c4c@oracle.com>
Date: Wed, 29 Sep 2021 10:22:07 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <e9a230f9-85cb-d4c1-8027-508b7c344d94@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-ClientProxiedBy: BLAPR03CA0135.namprd03.prod.outlook.com
 (2603:10b6:208:32e::20) To BLAPR10MB5009.namprd10.prod.outlook.com
 (2603:10b6:208:321::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c3bbbc54-c9df-4045-f8c6-08d983548687
X-MS-TrafficTypeDiagnostic: MN2PR10MB4256:
X-Microsoft-Antispam-PRVS: 
	<MN2PR10MB42568C672075B186D7D71A048AA99@MN2PR10MB4256.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 
	azXtEWuJiK0fdiSDxA7JVs8roo0D2WShIYFkYSvmFe0X4rt+0FOxoBhCpAD9CVgOGKKE+0PiLuq+9sUsxGPoXXyMPUYGlNY55e4vmORntr1hk+KDetSX8rfvu+6rALsCYRiT4xvsjysQvu2BFemm4NMy8sG1Ur51H/MQ220hJIcH8MSVEXm9MfMCC7kOmPpeOiX/FrTTH5SMck/rv3Pm9abliFJ+aUqVWir8/dY/RmZyo9Zc6ZXfRmqPpbs0J8s2Pr97DLgsjgbKhwm/oaVlaOFdmibGCKmCKaJiiFnSn68mtmQorJ1XpWdfqT4Gkc4EANuAPmaaJLdn6HgH/ZPenViafbzX9q4xQxotl8sqPwovdT97+mjr7uWwhwoUMfPSHxkP990lCyCiJDvKVJFc0AOsX1xeZ6s8KGmllLXKo+D7KLJeHbS4xl/ZtLL1DtaiJC4Z1OgskwpQM6g9SnPkp4DnsEZ6Q6wo8ou4ZFa/CAQnR2n0iD9kVitoemz2AeaqWNB9IL92eaNaMVn8UiPS9RNSuwZiebDeVdXRLe3PSI8YAZw/fuew/zYwS0iVXDsK0/wyI86UELdu7hAsvKVtiQ1SoHhDsotSimFIw2dDhZAtmiZc71LyLUxLP8CC0tZtLCOOZc0jjAC1LhJsEPmldYb2Dv40z0/KK/qWgwoZYRKyJ2WLAOS59uK9IV4jKcLT55XDT2s0IXkDI1oMPvsjTLdIAeaDOPHXlKWdcQ9Sk3Q=
X-Forefront-Antispam-Report: 
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(316002)(54906003)(7416002)(8936002)(86362001)(5660300002)(38100700002)(66476007)(16576012)(66556008)(36756003)(66946007)(31686004)(4326008)(6486002)(2616005)(8676002)(956004)(31696002)(26005)(186003)(44832011)(53546011)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?VkZGejU1YkRyZnVacitYRk5JR3BmMFJFZzV2bSt1cWJGTU1nektNK0V2ZDYx?=
 =?utf-8?B?TEIvWlR5Q1BBekMyUkRKai95N0RiUnhxNVZtcVdtR216eUIxS2gyYjREdS9o?=
 =?utf-8?B?TXBYVC83TGVwQnBUSmszU1RPcmtOZFVoMUZxRU4wQmZuT2xzNWtZMFpudGZs?=
 =?utf-8?B?V3hhY1lSU0QzeVU3dDFlMFZnUmhOM29qelJJOHo2VkJFbmlKYzBTSUliVXI5?=
 =?utf-8?B?SFFoOWU4M1pwUnNsNW1waDFKT2IzdEkwcVJjWHBLUk1NWlNrZnp3b29tdnl0?=
 =?utf-8?B?VDdWQ2NuL2Y1d3VOTy9laHZhNzhmQlkwMStqTXB6MjN1OXJqUFFGWUtuaDJI?=
 =?utf-8?B?ZktVR25kYXhtdGJlWXNpQkc0RWI1SUhLOE9TYTh2TzJZVWY4UzJhZk9XbnVJ?=
 =?utf-8?B?d2VJczZpQlZLMSs3cTV0NUdDbTJqYWxsb2dtWHJYRmo2T0RjMFJsdStUZVZ6?=
 =?utf-8?B?UmUxaWZPaGw0Y0Fza3JmZEZMbnVFaGNUdnJhSytMVE9nWHN2UDVvYUZJMzdD?=
 =?utf-8?B?TkttTGFFWFBUQjdoVy9iUWZDcXoyb1luSlVUTHJhcERoUjVoQ2pFZXFjT1l0?=
 =?utf-8?B?elNoNWZUKzN4ZllHVXVYeWMzbjZCQ0pVdFJ5R0FFc1hGVXJOdXNaZEIvcC9Z?=
 =?utf-8?B?citIOXhDWHpzclZscktBcjcwN2twMFFKbkpUOHpLb0Y0S2x5Tkw4S2ZNdnZU?=
 =?utf-8?B?ZGozZkF6M3Zady9VeERhQjF3dFZhSUs5YzVhM091blBwbklrYVFKcytjNlBt?=
 =?utf-8?B?WVRVTk1Pem1zNm43N1k0OW9yeHJPdi9BSVJlVE1kMUlIV3RNUnFsNlFsMUx6?=
 =?utf-8?B?U2JsM1JSaEI4dXZodW4wbjNyN212WXp6NUJ0OHpPWmsvRG9nV1NGK2c2ZGpS?=
 =?utf-8?B?Z0Iwc1ZQU3hjV1JpRkN2WlNFZFE2dXpzd1g0bUFJOTBOZnZxYnFrbU9Ib1hv?=
 =?utf-8?B?eDdaTDVIQ29nTjJyVk1DWUtDVGR1cnhhemNwMEVCNXVwVEZjNVZ0ODRCQUh4?=
 =?utf-8?B?cnMvWEpaVTlzbk9pNEVaVlNwNVVjWUxUM281dE4zcDVIVk45dnZFUjMyY3RD?=
 =?utf-8?B?RENqaXZQTU1BRXdBVy9heE5pb251MnN2SzhzVUE1ZG9BaEhnRkorSzd4ZGNV?=
 =?utf-8?B?TFp2djl3MjlnZGxFaFFVaUx0VVVkM1ZkaE5WVVlIeGg0cUNIYUtPckhqRDlM?=
 =?utf-8?B?SDcwVkROaXN6V3Y5WlY3MXFUYUNrdVhkamJXVU4zem9VMy9EUmhVMFNESzJT?=
 =?utf-8?B?R0NyUE9yUXlSOVg0djVFYWRXRGlGSnF3ME9wbFJ4K2JheGlsZUpaOHFoZzNB?=
 =?utf-8?B?Vnl1c0ZDQjJaa1pLcG91ZUw4bmI3blVNbXpkV0R6UXpMZFNGeDFoTkordkxP?=
 =?utf-8?B?aUxPK3N5UWE1SFRXNWJRbzlEblBwbXhZanlNekhoUFZDZFRQV2FaT2MyU2kz?=
 =?utf-8?B?dnduWmJianVpYXlaUi9rSG83MlFFYllGSEt0aWdpUkJEbGhoMGRmdElmR1c1?=
 =?utf-8?B?Z2sya1hSTnowRm41UmpDRitla3A5bTBEZTdkd3U4Q1VCY1Z5SmZYYllDYTRQ?=
 =?utf-8?B?L01XU25kQmlvNU83S0pDWEx1dUNRaXZNVWxSNlg1eUlRNi9vcUJ2dGZHMkNN?=
 =?utf-8?B?RURvUmE5VHQ0SVI3WWlDbEhTb2JSVWVRQlFZZndYajFwdU9mR05HRXRHU2Ux?=
 =?utf-8?B?K005WG5pYTUzWlV3c2lXNytoamp0NjcyeHpuUS9JNnBCdDR2cGpPV1FtMys1?=
 =?utf-8?Q?m78cTlBTE3d58Vjajl2WQl19AH0Znv9xj9BWUC6?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c3bbbc54-c9df-4045-f8c6-08d983548687
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2021 14:22:10.9515
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RMdriNBFwQVFfWL3dz6ApDwfdbfGxJEWfnKdCmTuP7hK5ZnmAKZ5whRtqfqQ356giLcxBDbI4x+W0yHFjGfuPO4TPuIuTI3olX5XhXFy+n4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4256
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0
 mlxscore=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290089
X-Proofpoint-GUID: PkBLbRGZ5d5dZzyjxXUaMDKbPdBPQLe0
X-Proofpoint-ORIG-GUID: PkBLbRGZ5d5dZzyjxXUaMDKbPdBPQLe0


On 9/29/21 5:03 AM, David Hildenbrand wrote:
> On 29.09.21 10:45, David Hildenbrand wrote:
>>>
>> Can we go one step further and do
>>
>>
>> @@ -20,24 +20,11 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
>>           struct xen_hvm_get_mem_type a = {
>>                   .domid = DOMID_SELF,
>>                   .pfn = pfn,
>> +               .mem_type = HVMMEM_ram_rw,
>>           };
>> -       int ram;
>>    -       if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
>> -               return -ENXIO;
>> -
>> -       switch (a.mem_type) {
>> -       case HVMMEM_mmio_dm:
>> -               ram = 0;
>> -               break;
>> -       case HVMMEM_ram_rw:
>> -       case HVMMEM_ram_ro:
>> -       default:
>> -               ram = 1;
>> -               break;
>> -       }
>> -
>> -       return ram;
>> +       HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a);
>> +       return a.mem_type != HVMMEM_mmio_dm;


I was actually thinking of asking you to add another patch with pr_warn_once() here (and print error code as well). This call failing is indication of something going quite wrong and it would be good to know about this.


>>    }
>>    #endif
>>
>>
>> Assuming that if HYPERVISOR_hvm_op() fails that
>> .mem_type is not set to HVMMEM_mmio_dm.


I don't think we can assume that argument described as OUT in the ABI will not be clobbered in case of error


>>
>
> Okay we can't, due to "__must_check" ...


so this is a good thing ;-)


-boris



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 15:07:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 15:07:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199224.353140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVbB1-0008Lm-VH; Wed, 29 Sep 2021 15:07:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199224.353140; Wed, 29 Sep 2021 15:07:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVbB1-0008Lf-SG; Wed, 29 Sep 2021 15:07:19 +0000
Received: by outflank-mailman (input) for mailman id 199224;
 Wed, 29 Sep 2021 15:07:18 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=NQqk=OT=redhat.com=david@srs-us1.protection.inumbo.net>)
 id 1mVbB0-0008LY-5p
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 15:07:18 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [170.10.133.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ef5b9445-2136-11ec-bd0f-12813bfff9fa;
 Wed, 29 Sep 2021 15:07:17 +0000 (UTC)
Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com
 [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-110-buY_zqtcOh2t29buqRu-Dw-1; Wed, 29 Sep 2021 11:07:14 -0400
Received: by mail-wm1-f72.google.com with SMTP id
 z137-20020a1c7e8f000000b0030cd1800d86so1361613wmc.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 08:07:13 -0700 (PDT)
Received: from [192.168.3.132] (p4ff23c3b.dip0.t-ipconnect.de. [79.242.60.59])
 by smtp.gmail.com with ESMTPSA id
 f8sm177044wrx.15.2021.09.29.08.07.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 29 Sep 2021 08:07:12 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef5b9445-2136-11ec-bd0f-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632928036;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gapF6Q7MxbKj+/Jpg0HNCHa+0ngI63VYNLFNHFDInM4=;
	b=KM8YB6ciT9CY51YYfZdK61z7iUAtHYXNHB1ZJ8fpmZyP0UrhOHXBGnTimY0jE0uxLJ+SXR
	yZZNjyqnG+w1t7LH71OrlenCXfMGSpg1RlNDY54gu1a3oo3DN4FaHbcVN0WHEAOaxM5wyJ
	rTLtvokMYRooahm2NYzm6Af+dcD4fxg=
X-MC-Unique: buY_zqtcOh2t29buqRu-Dw-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:subject:to:cc:references:from:organization
         :message-id:date:user-agent:mime-version:in-reply-to
         :content-language:content-transfer-encoding;
        bh=gapF6Q7MxbKj+/Jpg0HNCHa+0ngI63VYNLFNHFDInM4=;
        b=c/6kbwthcfUKWJyDKLKW/gjTjYKAFX6JekASPcHnKzeUNdRFJAoxyQCfecIygS5nrk
         Ta+GLoF3bPok5E6oZhzLd+NU1EwrkSj/wuAjbbsbPUQFtF3cbuTLoWsieN7qmxyDHTkQ
         cBD+iI8tldiXghHLa/EA1B2lyAKjO2BNsGfmWczTUgTXpreor27neBMvG3V0JIiSlHAE
         xE5iLOPojB8rY5V+g2ZKYiPQ8LGKYPxu9uDbqQGY5mQUJBjiANh05zzCRPGJy8KH+06p
         jGdy8Ud4Rh/TrYAOR4uKDCIudW2OqalHhBDLSdkNvB9Wbcj3LFNzCzRJlqV+hA8oTlLc
         yH+g==
X-Gm-Message-State: AOAM530wM+9tjsMX6NcY++54r167pCla21Ru0JjEtMwRA0KyaFhB8KWp
	d8xpac0HAYg5frYZWexGvQplGee3AUjqDdUUvZHElmzrx07cKcix8wAlD9bm/d4ABVIxOdSzKDy
	e8+Qrcea3qTGo3Fmldsbkv9O2xMs=
X-Received: by 2002:a5d:4e4e:: with SMTP id r14mr378694wrt.147.1632928032838;
        Wed, 29 Sep 2021 08:07:12 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxo/OiD1pi8Vj8uvbtvhFWxk5A000qCxnhsR/1CMHH5asTCVBPsn8x+8H3lxRFK4haIMhJq1A==
X-Received: by 2002:a5d:4e4e:: with SMTP id r14mr378630wrt.147.1632928032598;
        Wed, 29 Sep 2021 08:07:12 -0700 (PDT)
Subject: Re: [PATCH v1 2/8] x86/xen: simplify xen_oldmem_pfn_is_ram()
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>, linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
 Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
 Juergen Gross <jgross@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, "Michael S. Tsirkin" <mst@redhat.com>,
 Jason Wang <jasowang@redhat.com>, Dave Young <dyoung@redhat.com>,
 Baoquan He <bhe@redhat.com>, Vivek Goyal <vgoyal@redhat.com>,
 Michal Hocko <mhocko@suse.com>, Oscar Salvador <osalvador@suse.de>,
 Mike Rapoport <rppt@kernel.org>,
 "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, x86@kernel.org,
 xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org,
 kexec@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
References: <20210928182258.12451-1-david@redhat.com>
 <20210928182258.12451-3-david@redhat.com>
 <4ab2f8c2-c3d5-30b3-a670-a8b38e218b6e@oracle.com>
 <bfe72f46-9a0d-1a87-64bd-4b03999edd1e@redhat.com>
 <e9a230f9-85cb-d4c1-8027-508b7c344d94@redhat.com>
 <3b935aa0-6d85-0bcd-100e-15098add3c4c@oracle.com>
From: David Hildenbrand <david@redhat.com>
Organization: Red Hat
Message-ID: <e6ace8c8-8a2d-9bf7-e65b-91d0037c4d08@redhat.com>
Date: Wed, 29 Sep 2021 17:07:10 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <3b935aa0-6d85-0bcd-100e-15098add3c4c@oracle.com>
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit

On 29.09.21 16:22, Boris Ostrovsky wrote:
> 
> On 9/29/21 5:03 AM, David Hildenbrand wrote:
>> On 29.09.21 10:45, David Hildenbrand wrote:
>>>>
>>> Can we go one step further and do
>>>
>>>
>>> @@ -20,24 +20,11 @@ static int xen_oldmem_pfn_is_ram(unsigned long pfn)
>>>            struct xen_hvm_get_mem_type a = {
>>>                    .domid = DOMID_SELF,
>>>                    .pfn = pfn,
>>> +               .mem_type = HVMMEM_ram_rw,
>>>            };
>>> -       int ram;
>>>     -       if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
>>> -               return -ENXIO;
>>> -
>>> -       switch (a.mem_type) {
>>> -       case HVMMEM_mmio_dm:
>>> -               ram = 0;
>>> -               break;
>>> -       case HVMMEM_ram_rw:
>>> -       case HVMMEM_ram_ro:
>>> -       default:
>>> -               ram = 1;
>>> -               break;
>>> -       }
>>> -
>>> -       return ram;
>>> +       HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a);
>>> +       return a.mem_type != HVMMEM_mmio_dm;
> 
> 
> I was actually thinking of asking you to add another patch with pr_warn_once() here (and print error code as well). This call failing is indication of something going quite wrong and it would be good to know about this.

Will include a patch in v2, thanks!


-- 
Thanks,

David / dhildenb



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 15:31:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 15:31:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199232.353150 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVbYH-0003Kg-UH; Wed, 29 Sep 2021 15:31:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199232.353150; Wed, 29 Sep 2021 15:31:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVbYH-0003KZ-RP; Wed, 29 Sep 2021 15:31:21 +0000
Received: by outflank-mailman (input) for mailman id 199232;
 Wed, 29 Sep 2021 15:31:20 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=R9Bq=OT=m5p.com=ehem@srs-us1.protection.inumbo.net>)
 id 1mVbYG-0003KT-A8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 15:31:20 +0000
Received: from mailhost.m5p.com (unknown [74.104.188.4])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4ace6376-213a-11ec-bd11-12813bfff9fa;
 Wed, 29 Sep 2021 15:31:19 +0000 (UTC)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 18TFV9s9015040
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Wed, 29 Sep 2021 11:31:15 -0400 (EDT) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.16.1/8.15.2/Submit) id 18TFV8OJ015039;
 Wed, 29 Sep 2021 08:31:08 -0700 (PDT) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ace6376-213a-11ec-bd11-12813bfff9fa
Date: Wed, 29 Sep 2021 08:31:08 -0700
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: HVM/PVH Balloon crash
Message-ID: <YVSGvGZc67ZLMdKm@mattapan.m5p.com>
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
 <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
 <YVD59QVbmdVwzYQI@mattapan.m5p.com>
 <033cc499-34de-d27a-7b1b-2a0a7ce38672@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <033cc499-34de-d27a-7b1b-2a0a7ce38672@suse.com>
X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no
	autolearn_force=no version=3.4.5
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com

On Wed, Sep 29, 2021 at 03:32:15PM +0200, Jan Beulich wrote:
> On 27.09.2021 00:53, Elliott Mitchell wrote:
> > (XEN) Xen call trace:
> > (XEN)    [<ffff82d0402e8be0>] R arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
> > (XEN)    [<ffff82d0402ec51c>] S p2m_flush_nestedp2m+0x1c/0x30
> > (XEN)    [<ffff82d0402e0528>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x378/0x490
> 
> hap_write_p2m_entry() calling p2m_flush_nestedp2m() suggests that
> nestedhvm_enabled() was true for the domain. While we will want to
> fix this, nested virt is experimental (even in current staging),
> and hence there at least is no security concern.

Copy and paste from the xl.cfg man page:

       nestedhvm=BOOLEAN
           Enable or disables guest access to hardware virtualisation
           features, e.g. it allows a guest Operating System to also function
           as a hypervisor. You may want this option if you want to run
           another hypervisor (including another copy of Xen) within a Xen
           guest or to support a guest Operating System which uses hardware
           virtualisation extensions (e.g. Windows XP compatibility mode on
           more modern Windows OS).  This option is disabled by default.

"This option is disabled by default." doesn't mean "this is an
experimental feature with no security support and is likely to crash the
hypervisor".

More notably this is fully enabled in default builds of Xen.  Contrast
this with the stance of the ARM side with regards to ACPI.


> Can you confirm that by leaving nested off you don't run into this
> (or a similar) issue?

Hypervisor doesn't panic.  `xl dmesg` does end up with:

(XEN) p2m_pod_demand_populate: Dom72 out of PoD memory! (tot=524304 ents=28773031 dom72)
(XEN) domain_crash called from p2m-pod.c:1233

Which is problematic.  maxmem for this domain is set to allow for trading
memory around, so it is desireable for it to successfully load even when
its maximum isn't available.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Wed Sep 29 15:45:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 15:45:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199238.353162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVblz-0004xp-3k; Wed, 29 Sep 2021 15:45:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199238.353162; Wed, 29 Sep 2021 15:45:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVblz-0004xi-0K; Wed, 29 Sep 2021 15:45:31 +0000
Received: by outflank-mailman (input) for mailman id 199238;
 Wed, 29 Sep 2021 15:45:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lrrL=OT=linux.ibm.com=ajd@srs-us1.protection.inumbo.net>)
 id 1mVblx-0004xc-RH
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 15:45:29 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.158.5])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c2d46a91-f4d0-4a3a-9108-e444d6325ff6;
 Wed, 29 Sep 2021 15:45:29 +0000 (UTC)
Received: from pps.filterd (m0098420.ppops.net [127.0.0.1])
 by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18TFIXs5013454; 
 Wed, 29 Sep 2021 11:44:54 -0400
Received: from pps.reinject (localhost [127.0.0.1])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3bcs3bkpay-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 11:44:54 -0400
Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1])
 by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18TFKswJ027183;
 Wed, 29 Sep 2021 11:44:53 -0400
Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com
 [149.81.74.108])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3bcs3bkp9q-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 11:44:53 -0400
Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1])
 by ppma05fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18TFI54C008881;
 Wed, 29 Sep 2021 15:44:51 GMT
Received: from b06cxnps4076.portsmouth.uk.ibm.com
 (d06relay13.portsmouth.uk.ibm.com [9.149.109.198])
 by ppma05fra.de.ibm.com with ESMTP id 3bc11evrnx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Wed, 29 Sep 2021 15:44:50 +0000
Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com
 [9.149.105.61])
 by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18TFimIU46596452
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Wed, 29 Sep 2021 15:44:48 GMT
Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 1E5C211C050;
 Wed, 29 Sep 2021 15:44:48 +0000 (GMT)
Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id 70FD011C06E;
 Wed, 29 Sep 2021 15:44:47 +0000 (GMT)
Received: from ozlabs.au.ibm.com (unknown [9.192.253.14])
 by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP;
 Wed, 29 Sep 2021 15:44:47 +0000 (GMT)
Received: from [9.206.131.40] (unknown [9.206.131.40])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested)
 by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 5834F600F5;
 Thu, 30 Sep 2021 01:44:45 +1000 (AEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2d46a91-f4d0-4a3a-9108-e444d6325ff6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc :
 references : from : message-id : date : mime-version : in-reply-to :
 content-type : content-transfer-encoding; s=pp1;
 bh=2lZJdFw4WNk77XQ3gfuFQ0Wu4AQF8qNlzBIEQyCzgpc=;
 b=T/0vUcqJAa2fgLtyzY4tlQYtWLN8QVM0E25FTA6Zi7YulGd8Xo7QJfagdJ75M9VcA0n3
 e5TU1HSXPryyf0I8/UZHPWxNlno2R4cb5TY22SkLbPnBHwC8BMRJtj9V5AzF/UaxwtuY
 6ozdx9pMD3nIp0LpRP23GSJCB1cwz/V1u4oNvhdKizQ58LwiDQ55qqO0voze+t0jMrsx
 Q7Gn6g+aahJZ8K19odhcjNOxEOq1cqhn6zoUOeptzFurvTZWEtFGYe4xAywRRaUxG/4p
 QvzXrQ/vp5FNVPs9hO+wNi1UTQtCH93tezwbf6qv5yeWPPACnypEMeCrGi7e+1XPjByB Hg== 
Subject: Re: [PATCH v5 10/11] PCI: Replace pci_dev::driver usage by
 pci_dev::dev.driver
To: =?UTF-8?Q?Uwe_Kleine-K=c3=b6nig?= <u.kleine-koenig@pengutronix.de>
Cc: Bjorn Helgaas <helgaas@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        "Oliver O'Halloran" <oohall@gmail.com>,
        Russell Currey <ruscur@russell.cc>, Jiri Olsa <jolsa@redhat.com>,
        Christoph Hellwig <hch@lst.de>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Mathias Nyman <mathias.nyman@intel.com>,
        Michael Ellerman <mpe@ellerman.id.au>, x86@kernel.org,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Ingo Molnar <mingo@redhat.com>, linux-pci@vger.kernel.org,
        xen-devel@lists.xenproject.org, Juergen Gross <jgross@suse.com>,
        Arnd Bergmann <arnd@arndb.de>,
        Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Borislav Petkov <bp@alien8.de>, Bjorn Helgaas <bhelgaas@google.com>,
        Namhyung Kim <namhyung@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        linux-usb@vger.kernel.org, linux-perf-users@vger.kernel.org,
        kernel@pengutronix.de, Frederic Barrat <fbarrat@linux.ibm.com>,
        Paul Mackerras <paulus@samba.org>, linuxppc-dev@lists.ozlabs.org
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
 <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
 <75dd6d60-08b9-fa68-352c-3a0c5a04c0ab@linux.ibm.com>
 <20210929134330.e5c57t7mtwu5iner@pengutronix.de>
From: Andrew Donnellan <ajd@linux.ibm.com>
Message-ID: <1414b3c5-167c-c271-baed-d3d7f6cd0309@linux.ibm.com>
Date: Thu, 30 Sep 2021 01:44:44 +1000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <20210929134330.e5c57t7mtwu5iner@pengutronix.de>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: xoS4V0L0XnAZXIlSObNRrRHFrN-qRdXU
X-Proofpoint-GUID: aqWPGKvFeDrRCpepuY6Q_1i7K4Jcg0AY
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-29_06,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 phishscore=0
 impostorscore=0 suspectscore=0 clxscore=1015 mlxlogscore=874
 malwarescore=0 spamscore=0 priorityscore=1501 adultscore=0 bulkscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109290092

On 29/9/21 11:43 pm, Uwe Kleine-Knig wrote:> I'm not a huge fan either, 
I used it to keep the control flow as is and
> without introducing several calls to to_pci_driver.
> 
> The whole code looks as follows:
> 
> 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
> 		struct pci_driver *afu_drv;
> 		if (afu_dev->dev.driver &&
> 		    (afu_drv = to_pci_driver(afu_dev->dev.driver))->err_handler &&
> 		    afu_drv->err_handler->resume)
> 			afu_drv->err_handler->resume(afu_dev);
> 	}
> 
> Without assignment in the if it could look as follows:
> 
> 	list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
> 		struct pci_driver *afu_drv;
> 
> 		if (!afu_dev->dev.driver)
> 			continue;
> 
> 		afu_drv = to_pci_driver(afu_dev->dev.driver));
> 
> 		if (afu_drv->err_handler && afu_drv->err_handler->resume)
> 			afu_drv->err_handler->resume(afu_dev);
> 	}
> 
> Fine for me.

This looks fine.

As an aside while writing my email I discovered the existence of 
container_of_safe(), a version of container_of() that handles the null 
and err ptr cases... if to_pci_driver() used that, the null check in the 
caller could be moved until after the to_pci_driver() call which would 
be neater.

But then, grep tells me that container_of_safe() is used precisely zero 
times in the entire tree. Interesting.

> (Sidenote: What happens if the device is unbound directly after the
> check for afu_dev->dev.driver? This is a problem the old code had, too
> (assuming it is a real problem, didn't check deeply).)

Looking at any of the cxl PCI error handling paths brings back 
nightmares from a few years ago... Fred: I wonder if we need to add a 
lock here?

-- 
Andrew Donnellan              OzLabs, ADL Canberra
ajd@linux.ibm.com             IBM Australia Limited


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:17:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:17:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199244.353172 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcH8-0000nC-C7; Wed, 29 Sep 2021 16:17:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199244.353172; Wed, 29 Sep 2021 16:17:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcH8-0000n5-97; Wed, 29 Sep 2021 16:17:42 +0000
Received: by outflank-mailman (input) for mailman id 199244;
 Wed, 29 Sep 2021 16:17:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcH6-0000mz-Il
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:17:40 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b145f211-232a-4400-8ed4-805faa3d0a02;
 Wed, 29 Sep 2021 16:17:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 689B66128C;
 Wed, 29 Sep 2021 16:17:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b145f211-232a-4400-8ed4-805faa3d0a02
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632932258;
	bh=/FWzk9gz+WM+agqm0EVIgp4Xe6RX12pr52i65jxXhFE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=t1Y1Bpm7NRopopKAafJm+PksyW+Bt+X0JudkwBSbyQfNxPeGiiafg8bN2abCoFVgB
	 Bme0o8v+N0csTWDfieLKJZytt6Iz9tlduZPXVr4YTM/Hrebdk/9sWUxbK6UlAJBoSG
	 daWi+poGc4rUAxLbbb63eSN9OWjsPqEkvAiqpoU57kKYrhfJUhqXnetNcwtHHGTopv
	 91BoEv+Os2gDjoEoZNF+UpZRo07xkajn08/XPOYXEf+BPTPFItXO8uID+q3XDLwg8m
	 q8mPLF6lqeo8VHIAImAjVFbxxqH9hGEfq78/iH8uf0vE9hOzlm25G9A73i3guzUoOV
	 BjwGs/6O9b/6A==
Date: Wed, 29 Sep 2021 09:17:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, wei.chen@arm.com, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 1/3] arm/efi: Introduce uefi,cfg-load DT property
In-Reply-To: <46255EF5-17E6-41AC-A6D9-5D276056C0DA@arm.com>
Message-ID: <alpine.DEB.2.21.2109290916541.5022@sstabellini-ThinkPad-T480s>
References: <alpine.DEB.2.21.2109281459250.5022@sstabellini-ThinkPad-T480s> <AB8FEF63-EA1F-427F-AF2B-13C1E930F682@arm.com> <46255EF5-17E6-41AC-A6D9-5D276056C0DA@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 29 Sep 2021, Luca Fancellu wrote:
> > On Tue, 28 Sep 2021, Luca Fancellu wrote:
> >> Introduce the uefi,cfg-load DT property of /chosen
> >> node for ARM whose presence decide whether to force
> >> the load of the UEFI Xen configuration file.
> >> 
> >> The logic is that if any multiboot,module is found in
> >> the DT, then the uefi,cfg-load property is used to see
> >> if the UEFI Xen configuration file is needed.
> >> 
> >> Modify a comment in efi_arch_use_config_file, removing
> >> the part that states "dom0 required" because it's not
> >> true anymore with this commit.
> >> 
> >> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> > 
> > The patch looks good. Only one minor change: given that this is a Xen
> > parameter that we are introducing and not a parameter defined by UEFI
> > Forum, I think uefi,cfg-load should be called xen,uefi-cfg-load instead.
> > Because "xen," is our prefix, while "uefi," is not.
> > 
> > With that minor change:
> > 
> > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> Yes I will rename it.
> 
> > 
> > 
> > Note that the uefi,binary property is different because that property is
> > for xen,domain nodes, so we are already in a Xen specific namespace when
> > we are using it. Instead this property is for /chosen which is not a Xen
> > specific node.
> 
> Given that uefi,binary will be used also for multiboot,module directly under /chosen
> for Dom0, do you think I should rename also that to xen,uefi-binary?

Yes, maybe to stay on the safe side, it would be better to also rename
uefi,binary to xen,uefi-binary.



> >> ---
> >> v3 changes:
> >> - add documentation to misc/arm/device-tree/booting.txt
> >> - Modified variable name and logic from skip_cfg_file to
> >> load_cfg_file
> >> - Add in the commit message that I'm modifying a comment.
> >> v2 changes:
> >> - Introduced uefi,cfg-load property
> >> - Add documentation about the property
> >> ---
> >> docs/misc/arm/device-tree/booting.txt |  8 ++++++++
> >> docs/misc/efi.pandoc                  |  2 ++
> >> xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
> >> 3 files changed, 33 insertions(+), 5 deletions(-)
> >> 
> >> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> >> index 44cd9e1a9a..cf878b478e 100644
> >> --- a/docs/misc/arm/device-tree/booting.txt
> >> +++ b/docs/misc/arm/device-tree/booting.txt
> >> @@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
> >> for Dom0 and bootargs for native Linux.
> >> 
> >> 
> >> +UEFI boot and DT
> >> +================
> >> +
> >> +When Xen is booted using UEFI, it doesn't read the configuration file if any
> >> +multiboot module is specified. To force Xen to load the configuration file, the
> >> +boolean property uefi,cfg-load must be declared in the /chosen node.
> >> +
> >> +
> >> Creating Multiple Domains directly from Xen
> >> ===========================================
> >> 
> >> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> >> index ac3cd58cae..e289c5e7ba 100644
> >> --- a/docs/misc/efi.pandoc
> >> +++ b/docs/misc/efi.pandoc
> >> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
> >> bootloader provides a device tree containing modules then any configuration
> >> files are ignored, and the bootloader is responsible for populating all
> >> relevant device tree nodes.
> >> +The property "uefi,cfg-load" can be specified in the /chosen node to force Xen
> >> +to load the configuration file even if multiboot modules are found.
> >> 
> >> Once built, `make install-xen` will place the resulting binary directly into
> >> the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
> >> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> >> index cf9c37153f..4f1b01757d 100644
> >> --- a/xen/arch/arm/efi/efi-boot.h
> >> +++ b/xen/arch/arm/efi/efi-boot.h
> >> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
> >> 
> >> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
> >> {
> >> +    bool load_cfg_file = true;
> >>     /*
> >>      * For arm, we may get a device tree from GRUB (or other bootloader)
> >>      * that contains modules that have already been loaded into memory.  In
> >> -     * this case, we do not use a configuration file, and rely on the
> >> -     * bootloader to have loaded all required modules and appropriate
> >> -     * options.
> >> +     * this case, we search for the property uefi,cfg-load in the /chosen node
> >> +     * to decide whether to skip the UEFI Xen configuration file or not.
> >>      */
> >> 
> >>     fdt = lookup_fdt_config_table(SystemTable);
> >>     dtbfile.ptr = fdt;
> >>     dtbfile.need_to_free = false; /* Config table memory can't be freed. */
> >> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
> >> +
> >> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
> >> +    {
> >> +        /* Locate chosen node */
> >> +        int node = fdt_subnode_offset(fdt, 0, "chosen");
> >> +        const void *cfg_load_prop;
> >> +        int cfg_load_len;
> >> +
> >> +        if ( node > 0 )
> >> +        {
> >> +            /* Check if uefi,cfg-load property exists */
> >> +            cfg_load_prop = fdt_getprop(fdt, node, "uefi,cfg-load",
> >> +                                        &cfg_load_len);
> >> +            if ( !cfg_load_prop )
> >> +                load_cfg_file = false;
> >> +        }
> >> +    }
> >> +
> >> +    if ( !fdt || load_cfg_file )
> >>     {
> >>         /*
> >>          * We either have no FDT, or one without modules, so we must have a
> >> -         * Xen EFI configuration file to specify modules.  (dom0 required)
> >> +         * Xen EFI configuration file to specify modules.
> >>          */
> >>         return true;
> >>     }
> >> -- 
> >> 2.17.1
> >> 
> 
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:40:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:40:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199250.353184 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcdP-0004B4-86; Wed, 29 Sep 2021 16:40:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199250.353184; Wed, 29 Sep 2021 16:40:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcdP-0004Ax-58; Wed, 29 Sep 2021 16:40:43 +0000
Received: by outflank-mailman (input) for mailman id 199250;
 Wed, 29 Sep 2021 16:40:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcdN-0004Ar-HS
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:40:41 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 16bc2e00-1b6e-4a89-a008-f8f62a701523;
 Wed, 29 Sep 2021 16:40:39 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2EE5C61406;
 Wed, 29 Sep 2021 16:40:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16bc2e00-1b6e-4a89-a008-f8f62a701523
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632933638;
	bh=TeBOg9a6sh++KgUKToezH33AiyfneKXMQLQlkgkFsVw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=FEkbdP7ofXKL3ykCMTu9S+t1qdt696f3W/fan9hTiL1eCEYrmf7jPAzxY/3EgvNhy
	 6g/lt0iDzvkYXCAA7LCxkbg+Dclh8EWVy5Jcf5eYYyOy6XWIcyKBiFsJDff1SvDqo3
	 PrTm9z2aAQRAzb8L9yJtZktZzZztWAsAbtCf5jWIb5h2sZd+nrF92Bn5jxl14ShmD2
	 X9JCjShKD7fgxs9Zw0ansefKyu8K+q4gKkQKeLLdbch5qoIhc9V232Ru6OrHVOodK9
	 3Vg6a4r3ZP7Qv7BxWuiWDHYJSGV2bCbuGIbjOAW1YRaJ63jgmbSk7ZvZNUflwk6f+Z
	 8ZUCaOIDrVVoQ==
Date: Wed, 29 Sep 2021 09:40:38 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN
 on ARM
In-Reply-To: <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281636420.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1969758331-1632872232=:5022"
Content-ID: <alpine.DEB.2.21.2109281640220.5022@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1969758331-1632872232=:5022
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109281640221.5022@sstabellini-ThinkPad-T480s>

On Tue, 28 Sep 2021, Rahul Singh wrote:
> XEN during boot will read the PCI device tree node “reg” property
> and will map the PCI config space to the XEN memory.
> 
> As of now only "pci-host-ecam-generic" compatible board is supported.
> 
> "linux,pci-domain" device tree property assigns a fixed PCI domain
> number to a host bridge, otherwise an unstable (across boots) unique
> number will be assigned by Linux. XEN access the PCI devices based on
> Segment:Bus:Device:Function. A Segment number in the XEN is same as a
> domain number in Linux. Segment number and domain number has to be in
> sync to access the correct PCI devices.
> 
> XEN will read the “linux,pci-domain” property from the device tree node
> and configure the host bridge segment number accordingly. If this
> property is not available XEN will allocate the unique segment number
> to the host bridge.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

This is so much better! I only have one comment left below


> ---
> Change in v3:
> - Modify commit msg based on received comments.
> - Remove added struct match_table{} struct in struct device{}
> - Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
> - Remove bus_start,bus_end and void *sysdata from struct pci_host_bridge{}
> - Move "#include <asm/pci.h>" in "xen/pci.h" after pci_sbdf_t sbdf declaration
> - Add pci_host_generic_probe() function 
> Change in v2:
> - Add more info in commit msg
> - Add callback to parse register index.
> - Merge patch pci_ecam_operation into this patch to avoid confusion
> - Add new struct in struct device for match table
> ---
>  xen/arch/arm/pci/Makefile           |   4 +
>  xen/arch/arm/pci/ecam.c             |  61 +++++++
>  xen/arch/arm/pci/pci-access.c       |  83 ++++++++++
>  xen/arch/arm/pci/pci-host-common.c  | 247 ++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-generic.c |  46 ++++++
>  xen/include/asm-arm/pci.h           |  56 +++++++
>  xen/include/xen/pci.h               |   3 +-
>  7 files changed, 499 insertions(+), 1 deletion(-)
>  create mode 100644 xen/arch/arm/pci/ecam.c
>  create mode 100644 xen/arch/arm/pci/pci-access.c
>  create mode 100644 xen/arch/arm/pci/pci-host-common.c
>  create mode 100644 xen/arch/arm/pci/pci-host-generic.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index a98035df4c..6f32fbbe67 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -1 +1,5 @@
>  obj-y += pci.o
> +obj-y += pci-access.o
> +obj-y += pci-host-generic.o
> +obj-y += pci-host-common.o
> +obj-y += ecam.o
> diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
> new file mode 100644
> index 0000000000..602d00799c
> --- /dev/null
> +++ b/xen/arch/arm/pci/ecam.c
> @@ -0,0 +1,61 @@
> +/*
> + * Based on Linux drivers/pci/ecam.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/sched.h>
> +
> +/*
> + * Function to implement the pci_ops->map_bus method.
> + */
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               pci_sbdf_t sbdf, uint32_t where)
> +{
> +    const struct pci_config_window *cfg = bridge->cfg;
> +    struct pci_ecam_ops *ops =
> +        container_of(bridge->ops, struct pci_ecam_ops, pci_ops);
> +    unsigned int devfn_shift = ops->bus_shift - 8;
> +    void __iomem *base;
> +
> +    unsigned int busn = PCI_BUS(sbdf.bdf);
> +
> +    if ( busn < cfg->busn_start || busn > cfg->busn_end )
> +        return NULL;
> +
> +    busn -= cfg->busn_start;
> +    base = cfg->win + (busn << ops->bus_shift);
> +
> +    return base + (PCI_DEVFN2(sbdf.bdf) << devfn_shift) + where;
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops pci_generic_ecam_ops = {
> +    .bus_shift  = 20,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> new file mode 100644
> index 0000000000..3cd14a4b87
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -0,0 +1,83 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <asm/io.h>
> +
> +#define INVALID_VALUE (~0U)
> +
> +int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +
> +    if ( !addr )
> +    {
> +        *value = INVALID_VALUE;
> +        return -ENODEV;
> +    }
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        *value = readb(addr);
> +        break;
> +    case 2:
> +        *value = readw(addr);
> +        break;
> +    case 4:
> +        *value = readl(addr);
> +        break;
> +    default:
> +        ASSERT_UNREACHABLE();
> +    }
> +
> +    return 0;
> +}
> +
> +int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                             uint32_t reg, uint32_t len, uint32_t value)
> +{
> +    void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> +
> +    if ( !addr )
> +        return -ENODEV;
> +
> +    switch ( len )
> +    {
> +    case 1:
> +        writeb(value, addr);
> +        break;
> +    case 2:
> +        writew(value, addr);
> +        break;
> +    case 4:
> +        writel(value, addr);
> +        break;
> +    default:
> +        ASSERT_UNREACHABLE();
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> new file mode 100644
> index 0000000000..a08e06cea1
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -0,0 +1,247 @@
> +/*
> + * Based on Linux drivers/pci/ecam.c
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <xen/rwlock.h>
> +#include <xen/sched.h>
> +#include <xen/vmap.h>
> +
> +/*
> + * List for all the pci host bridges.
> + */
> +
> +static LIST_HEAD(pci_host_bridges);
> +
> +static atomic_t domain_nr = ATOMIC_INIT(-1);
> +
> +static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len)
> +{
> +    return ioremap_nocache(start, len);
> +}
> +
> +static void pci_ecam_free(struct pci_config_window *cfg)
> +{
> +    if ( cfg->win )
> +        iounmap(cfg->win);
> +
> +    xfree(cfg);
> +}
> +
> +static struct pci_config_window * __init
> +gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
> +{
> +    int err, cfg_reg_idx;
> +    u32 bus_range[2];
> +    paddr_t addr, size;
> +    struct pci_config_window *cfg;
> +
> +    cfg = xzalloc(struct pci_config_window);
> +    if ( !cfg )
> +        return NULL;
> +
> +    err = dt_property_read_u32_array(dev, "bus-range", bus_range,
> +                                     ARRAY_SIZE(bus_range));
> +    if ( err ) {
> +        cfg->busn_start = 0;
> +        cfg->busn_end = 0xff;
> +        printk(XENLOG_INFO "%s: No bus range found for pci controller\n",
> +               dt_node_full_name(dev));
> +    } else {
> +        cfg->busn_start = bus_range[0];
> +        cfg->busn_end = bus_range[1];
> +        if ( cfg->busn_end > cfg->busn_start + 0xff )
> +            cfg->busn_end = cfg->busn_start + 0xff;
> +    }
> +
> +    if ( ops->cfg_reg_index )
> +    {
> +        cfg_reg_idx = ops->cfg_reg_index(dev);
> +        if ( cfg_reg_idx < 0 )
> +            goto err_exit;
> +    }
> +    else
> +        cfg_reg_idx = 0;
> +
> +    /* Parse our PCI ecam register address */
> +    err = dt_device_get_address(dev, cfg_reg_idx, &addr, &size);
> +    if ( err )
> +        goto err_exit;
> +
> +    cfg->phys_addr = addr;
> +    cfg->size = size;
> +
> +    /*
> +     * On 64-bit systems, we do a single ioremap for the whole config space
> +     * since we have enough virtual address range available.  On 32-bit, we
> +     * ioremap the config space for each bus individually.
> +     * As of now only 64-bit is supported 32-bit is not supported.
> +     *
> +     * TODO: For 32-bit implement the ioremap/iounmap of config space
> +     * dynamically for each read/write call.
> +     */
> +    cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size);
> +    if ( !cfg->win )
> +    {
> +        printk(XENLOG_ERR "ECAM ioremap failed\n");
> +        goto err_exit;
> +    }
> +    printk("ECAM at [mem 0x%"PRIpaddr"-0x%"PRIpaddr"] for [bus %x-%x] \n",
> +            cfg->phys_addr, cfg->phys_addr + cfg->size - 1,
> +            cfg->busn_start, cfg->busn_end);
> +
> +    if ( ops->init )
> +    {
> +        err = ops->init(cfg);
> +        if ( err )
> +            goto err_exit;
> +    }
> +
> +    return cfg;
> +
> +err_exit:
> +    pci_ecam_free(cfg);
> +
> +    return NULL;
> +}
> +
> +struct pci_host_bridge *pci_alloc_host_bridge(void)
> +{
> +    struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge);
> +
> +    if ( !bridge )
> +        return NULL;
> +
> +    INIT_LIST_HEAD(&bridge->node);
> +
> +    return bridge;
> +}
> +
> +void pci_add_host_bridge(struct pci_host_bridge *bridge)
> +{
> +    list_add_tail(&bridge->node, &pci_host_bridges);
> +}
> +
> +static int pci_get_new_domain_nr(void)
> +{
> +    return atomic_inc_return(&domain_nr);
> +}
> +
> +static int pci_bus_find_domain_nr(struct dt_device_node *dev)
> +{
> +    static int use_dt_domains = -1;
> +    int domain;
> +
> +    domain = dt_get_pci_domain_nr(dev);
> +
> +    /*
> +     * Check DT domain and use_dt_domains values.
> +     *
> +     * If DT domain property is valid (domain >= 0) and
> +     * use_dt_domains != 0, the DT assignment is valid since this means
> +     * we have not previously allocated a domain number by using
> +     * pci_get_new_domain_nr(); we should also update use_dt_domains to
> +     * 1, to indicate that we have just assigned a domain number from
> +     * DT.
> +     *
> +     * If DT domain property value is not valid (ie domain < 0), and we
> +     * have not previously assigned a domain number from DT
> +     * (use_dt_domains != 1) we should assign a domain number by
> +     * using the:
> +     *
> +     * pci_get_new_domain_nr()
> +     *
> +     * API and update the use_dt_domains value to keep track of method we
> +     * are using to assign domain numbers (use_dt_domains = 0).
> +     *
> +     * All other combinations imply we have a platform that is trying
> +     * to mix domain numbers obtained from DT and pci_get_new_domain_nr(),
> +     * which is a recipe for domain mishandling and it is prevented by
> +     * invalidating the domain value (domain = -1) and printing a
> +     * corresponding error.
> +     */
> +    if ( domain >= 0 && use_dt_domains )
> +    {
> +        use_dt_domains = 1;
> +    }
> +    else if ( domain < 0 && use_dt_domains != 1 )
> +    {
> +        use_dt_domains = 0;
> +        domain = pci_get_new_domain_nr();
> +    }
> +    else
> +    {
> +        domain = -1;
> +    }
> +
> +    return domain;
> +}
> +
> +int pci_host_common_probe(struct dt_device_node *dev, const void *data)
> +{
> +    struct pci_host_bridge *bridge;
> +    struct pci_config_window *cfg;
> +    struct pci_ecam_ops *ops;
> +    int err;
> +
> +    if ( dt_device_for_passthrough(dev) )
> +        return 0;
> +
> +    ops = (struct pci_ecam_ops *) data;
> +
> +    bridge = pci_alloc_host_bridge();
> +    if ( !bridge )
> +        return -ENOMEM;
> +
> +    /* Parse and map our Configuration Space windows */
> +    cfg = gen_pci_init(dev, ops);
> +    if ( !cfg )
> +    {
> +        err = -ENOMEM;
> +        goto err_exit;
> +    }
> +
> +    bridge->dt_node = dev;
> +    bridge->cfg = cfg;
> +    bridge->ops = &ops->pci_ops;
> +
> +    bridge->segment = pci_bus_find_domain_nr(dev);
> +    if ( bridge->segment < 0 )
> +    {
> +        printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n");
> +        BUG();
> +    }
> +    pci_add_host_bridge(bridge);
> +
> +    return 0;
> +
> +err_exit:
> +    xfree(bridge);
> +
> +    return err;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> new file mode 100644
> index 0000000000..5e4f8f28a0
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -0,0 +1,46 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static const struct dt_device_match gen_pci_dt_match[] = {

This could be __initdata


> +    { .compatible = "pci-host-ecam-generic" },
> +    { },
> +};
> +
> +static int pci_host_generic_probe(struct dt_device_node *dev,
> +                                  const void *data)

and this could be __init


> +{
> +    return pci_host_common_probe(dev, &pci_generic_ecam_ops);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI)
> +.dt_match = gen_pci_dt_match,
> +.init = pci_host_generic_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 0cf849e26f..bb7eda6705 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,62 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/*
> + * struct to hold the mappings of a config space window. This
> + * is expected to be used as sysdata for PCI controllers that
> + * use ECAM.
> + */
> +struct pci_config_window {
> +    paddr_t         phys_addr;
> +    paddr_t         size;
> +    uint8_t         busn_start;
> +    uint8_t         busn_end;
> +    void __iomem    *win;
> +};
> +
> +/*
> + * struct to hold pci host bridge information
> + * for a PCI controller.
> + */
> +struct pci_host_bridge {
> +    struct dt_device_node *dt_node;  /* Pointer to the associated DT node */
> +    struct list_head node;           /* Node in list of host bridges */
> +    uint16_t segment;                /* Segment number */
> +    struct pci_config_window* cfg;   /* Pointer to the bridge config window */
> +    struct pci_ops *ops;
> +};
> +
> +struct pci_ops {
> +    void __iomem *(*map_bus)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                             uint32_t offset);
> +    int (*read)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                uint32_t reg, uint32_t len, uint32_t *value);
> +    int (*write)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                 uint32_t reg, uint32_t len, uint32_t value);
> +};
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_ecam_ops {
> +    unsigned int            bus_shift;
> +    struct pci_ops          pci_ops;
> +    int (*cfg_reg_index)(struct dt_device_node *dev);
> +    int (*init)(struct pci_config_window *);
> +};
> +
> +/* Default ECAM ops */
> +extern const struct pci_ecam_ops pci_generic_ecam_ops;
> +
> +int pci_host_common_probe(struct dt_device_node *dev, const void *data);
> +int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                            uint32_t reg, uint32_t len, uint32_t *value);
> +int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> +                             uint32_t reg, uint32_t len, uint32_t value);
> +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
> +                               pci_sbdf_t sbdf, uint32_t where);
> +
>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
>      return pci_passthrough_enabled;
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index 8e3d4d9454..70ac25345c 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -15,7 +15,6 @@
>  #include <xen/pfn.h>
>  #include <asm/device.h>
>  #include <asm/numa.h>
> -#include <asm/pci.h>
>  
>  /*
>   * The PCI interface treats multi-function devices as independent
> @@ -62,6 +61,8 @@ typedef union {
>      };
>  } pci_sbdf_t;
>  
> +#include <asm/pci.h>
> +
>  struct pci_dev_info {
>      /*
>       * VF's 'is_extfn' field is used to indicate whether its PF is an extended
--8323329-1969758331-1632872232=:5022--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:41:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:41:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199258.353195 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcea-0004ng-Lo; Wed, 29 Sep 2021 16:41:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199258.353195; Wed, 29 Sep 2021 16:41:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcea-0004nZ-Ie; Wed, 29 Sep 2021 16:41:56 +0000
Received: by outflank-mailman (input) for mailman id 199258;
 Wed, 29 Sep 2021 16:41:56 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcea-0004nT-1L
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:41:56 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2787b692-2144-11ec-bd13-12813bfff9fa;
 Wed, 29 Sep 2021 16:41:55 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 510BB61209;
 Wed, 29 Sep 2021 16:41:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2787b692-2144-11ec-bd13-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632933714;
	bh=z91rOq2RvvcZpmymQuLhQEnfMpw8VLa8tosZvwqeRNk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=cT1AQfmFbzByjwbmObJDSpep6Zllw+9olAJHGHNKLoYkg/FXQzFnl5cvjVzrpfhFd
	 Y0PrDjQGcbRyQPqbi2RMaf+IG0au0AvSf6XOIuL0tMQhQs+tLsTyPN1LIFoKbKYejz
	 s04hLeo1MAfEEFdpZxdmWxLJy1UuV3DlJQ+ZeOdyL9y0WHsuXTGmqZTx3sCZa2Lz8H
	 14WYL18MHHvqtKPuWc4zjQv6pDMY7yIbhHZVik+AmH3DtbmA4OZxN4gGGi0HxEFVxb
	 O63fKCMgqZQVbVXG+ug7uWevTWsO7VyB0bBLZ3iNItBuhwv2N3jsGeI5d76+IOu3yF
	 ur5JG7gl/1fYA==
Date: Wed, 29 Sep 2021 09:41:54 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, 
    Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
In-Reply-To: <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281638210.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Add support for Xilinx ZynqMP PCI host controller to map the PCI config
> space to the XEN memory.
> 
> Patch helps to understand how the generic infrastructure for PCI
> host-bridge discovery will be used for future references.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
> Change in v3:
> - nwl_cfg_reg_index(..) as static function
> - Add support for pci_host_generic_probe() 
> Change in v2:
> - Add more info in commit msg
> ---
>  xen/arch/arm/pci/Makefile          |  1 +
>  xen/arch/arm/pci/pci-host-zynqmp.c | 63 ++++++++++++++++++++++++++++++
>  2 files changed, 64 insertions(+)
>  create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
> 
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index 6f32fbbe67..1d045ade01 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -3,3 +3,4 @@ obj-y += pci-access.o
>  obj-y += pci-host-generic.o
>  obj-y += pci-host-common.o
>  obj-y += ecam.o
> +obj-y += pci-host-zynqmp.o
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> new file mode 100644
> index 0000000000..6ccbfd15c9
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -0,0 +1,63 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <asm/device.h>
> +#include <xen/pci.h>
> +#include <asm/pci.h>
> +
> +static int nwl_cfg_reg_index(struct dt_device_node *np)
> +{
> +    return dt_property_match_string(np, "reg-names", "cfg");
> +}

Can this be __init?


> +/* ECAM ops */
> +const struct pci_ecam_ops nwl_pcie_ops = {
> +    .bus_shift  = 20,
> +    .cfg_reg_index = nwl_cfg_reg_index,
> +    .pci_ops    = {
> +        .map_bus                = pci_ecam_map_bus,
> +        .read                   = pci_generic_config_read,
> +        .write                  = pci_generic_config_write,
> +    }
> +};
> +
> +static const struct dt_device_match nwl_pcie_dt_match[] = {

This should probably be __initconst


> +    { .compatible = "xlnx,nwl-pcie-2.11" },
> +    { },
> +};
> +
> +static int pci_host_generic_probe(struct dt_device_node *dev,
> +                                  const void *data)

and this could be __init


> +{
> +    return pci_host_common_probe(dev, &nwl_pcie_ops);
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI)
> +.dt_match = nwl_pcie_dt_match,
> +.init = pci_host_generic_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:42:50 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:42:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199264.353205 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcfR-0005OS-VB; Wed, 29 Sep 2021 16:42:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199264.353205; Wed, 29 Sep 2021 16:42:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcfR-0005OL-SF; Wed, 29 Sep 2021 16:42:49 +0000
Received: by outflank-mailman (input) for mailman id 199264;
 Wed, 29 Sep 2021 16:42:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcfQ-0005OA-Vw
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:42:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 47101cac-2144-11ec-bd13-12813bfff9fa;
 Wed, 29 Sep 2021 16:42:47 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 2AA9161262;
 Wed, 29 Sep 2021 16:42:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47101cac-2144-11ec-bd13-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632933767;
	bh=Ls0HHz7ufDWk8S7FpflBhQt8FKDysLwB294j6ZcCKvc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kkvO2JXodbreZ7ggI82UmVodYchQYAQ8VY46AI+BrAqy5LTmq4HNLAa1N9CfhrVMf
	 xCG9xzn4vX67PqBI6V2ZBi4ewmOmSL2fQGdHYm3gVCX3t+4vKcshCi0rSZQKNQHz7F
	 vzI8t4GGTYXWAX//CYXPyAkzGqt5pCcSSYciiHjTsAX3tqZEPopXAmAa5AxsrkvcBf
	 ti/HsS83+RwoswneR4Pn9ShXcNaIr0D2JGNDiu0HbVumiwg0JLwt80MiH28BCC0EQL
	 aebgs0KxhoN6Khxlz3WPQwNJTZ9QlHw83wq19gBg/Yp/MUMr4eO8f/MohWBtOMPeap
	 m8DF+T6r71CtQ==
Date: Wed, 29 Sep 2021 09:42:46 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 13/17] xen/arm: Implement pci access functions
In-Reply-To: <33cc6b7a133787700ea8ba4e54a03141d3aca1ea.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281642360.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <33cc6b7a133787700ea8ba4e54a03141d3aca1ea.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> Implement generic pci access functions to read/write the configuration
> space.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Remove PRI_pci as not used.
> - Replace uint32_t sbdf to pci_sbdf_t sbdf to avoid typecast
> Change in v2: Fixed comments
> ---
>  xen/arch/arm/pci/pci-access.c      | 57 ++++++++++++++++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++
>  xen/include/asm-arm/pci.h          |  1 +
>  3 files changed, 77 insertions(+)
> 
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index 3cd14a4b87..9f9aac43d7 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -16,6 +16,7 @@
>  #include <asm/io.h>
>  
>  #define INVALID_VALUE (~0U)
> +#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>  
>  int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
>                              uint32_t reg, uint32_t len, uint32_t *value)
> @@ -72,6 +73,62 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
>      return 0;
>  }
>  
> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
> +                                unsigned int len)
> +{
> +    uint32_t val = PCI_ERR_VALUE(len);
> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return val;
> +
> +    if ( unlikely(!bridge->ops->read) )
> +        return val;
> +
> +    bridge->ops->read(bridge, sbdf, reg, len, &val);
> +
> +    return val;
> +}
> +
> +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
> +                             unsigned int len, uint32_t val)
> +{
> +    struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> +
> +    if ( unlikely(!bridge) )
> +        return;
> +
> +    if ( unlikely(!bridge->ops->write) )
> +        return;
> +
> +    bridge->ops->write(bridge, sbdf, reg, len, val);
> +}
> +
> +/*
> + * Wrappers for all PCI configuration access functions.
> + */
> +
> +#define PCI_OP_WRITE(size, type)                            \
> +    void pci_conf_write##size(pci_sbdf_t sbdf,              \
> +                              unsigned int reg, type val)   \
> +{                                                           \
> +    pci_config_write(sbdf, reg, size / 8, val);             \
> +}
> +
> +#define PCI_OP_READ(size, type)                             \
> +    type pci_conf_read##size(pci_sbdf_t sbdf,               \
> +                              unsigned int reg)             \
> +{                                                           \
> +    return pci_config_read(sbdf, reg, size / 8);            \
> +}
> +
> +PCI_OP_READ(8, uint8_t)
> +PCI_OP_READ(16, uint16_t)
> +PCI_OP_READ(32, uint32_t)
> +PCI_OP_WRITE(8, uint8_t)
> +PCI_OP_WRITE(16, uint16_t)
> +PCI_OP_WRITE(32, uint32_t)
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index a08e06cea1..c5941b10e9 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -236,6 +236,25 @@ err_exit:
>      return err;
>  }
>  
> +/*
> + * This function will lookup an hostbridge based on the segment and bus
> + * number.
> + */
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->segment != segment )
> +            continue;
> +        if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn_end) )
> +            continue;
> +        return bridge;
> +    }
> +
> +    return NULL;
> +}
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index bb7eda6705..49c9622902 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -81,6 +81,7 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
>                               uint32_t reg, uint32_t len, uint32_t value);
>  void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                 pci_sbdf_t sbdf, uint32_t where);
> +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
>  
>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:45:49 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:45:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199270.353216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVciK-00065L-Ct; Wed, 29 Sep 2021 16:45:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199270.353216; Wed, 29 Sep 2021 16:45:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVciK-00065E-9m; Wed, 29 Sep 2021 16:45:48 +0000
Received: by outflank-mailman (input) for mailman id 199270;
 Wed, 29 Sep 2021 16:45:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVciJ-000658-2H
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:45:47 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 3d40f2b3-2190-4079-b82b-50ad3d1034b2;
 Wed, 29 Sep 2021 16:45:44 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id D00F161262;
 Wed, 29 Sep 2021 16:45:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d40f2b3-2190-4079-b82b-50ad3d1034b2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632933944;
	bh=NVcaL2f67A8Hj4/uKqwfBjf7qYHxOnwhj3b2Vh0tfcE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XNf1j6szoy0obwwTHkb7RYiCb4EGezgAIp73ZRJCi7qN/rmFfxWWrrhk8swNShd2z
	 j6Dd3MHt1JrTcui2lNBCjnCt0RsILKvGG3cf1mG1GX5PYkVM6M5ARq1At7sPTBM2TO
	 oD6aW8s3jv9n1i2ZJaUQSSSwkmBFDxjajuSpGRPZX58RwzLsPpK6z5n/y4TSOpKjl5
	 7QbBMboLpx2M7mCIfuWqxGQRmxxz1UPnKicg7S1kJXweKLYUwS6QLb9nfIxB6v5i3Q
	 nT1kQtfj8LlpDrCSufaNMKIrkcihVscM8l5LZ9QWm++hI/JXRI1c1WqE5DO6qMGpcu
	 SLmd3x23OvqOw==
Date: Wed, 29 Sep 2021 09:45:43 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Paul Durrant <paul@xen.org>
Subject: Re: [PATCH v3 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
In-Reply-To: <d7f62249e6aa7b33017ef707d6bc451b5e2c7047.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281645150.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <d7f62249e6aa7b33017ef707d6bc451b5e2c7047.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1855915649-1632872724=:5022"
Content-ID: <alpine.DEB.2.21.2109290942510.5022@sstabellini-ThinkPad-T480s>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1855915649-1632872724=:5022
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.21.2109290942511.5022@sstabellini-ThinkPad-T480s>

On Tue, 28 Sep 2021, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> “PHYSDEVOP_pci_device_add”, VPCI handler for the config space
> access is added to the Xen to emulate the PCI devices config space.
> 
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.
> 
> For Dom0less systems scan_pci_devices() would be used to discover the
> PCI device in XEN and VPCI handler will be added during XEN boots.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Use is_pci_passthrough_enabled() in place of pci_passthrough_enabled variable
> - Reject XEN_DOMCTL_CDF_vpci for x86 in arch_sanitise_domain_config()
> - Remove IS_ENABLED(CONFIG_HAS_VPCI) from has_vpci()
> Change in v2:
> - Add new XEN_DOMCTL_CDF_vpci flag
> - modify has_vpci() to include XEN_DOMCTL_CDF_vpci
> - enable vpci support when pci-passthough option is enabled.
> ---
>  xen/arch/arm/Makefile         |   1 +
>  xen/arch/arm/domain.c         |   8 ++-
>  xen/arch/arm/domain_build.c   |   3 +
>  xen/arch/arm/vpci.c           | 102 ++++++++++++++++++++++++++++++++++
>  xen/arch/arm/vpci.h           |  36 ++++++++++++
>  xen/arch/x86/domain.c         |   6 ++
>  xen/common/domain.c           |   2 +-
>  xen/drivers/passthrough/pci.c |  12 ++++
>  xen/include/asm-arm/domain.h  |   8 ++-
>  xen/include/asm-x86/pci.h     |   2 -
>  xen/include/public/arch-arm.h |   7 +++
>  xen/include/public/domctl.h   |   4 +-
>  xen/include/xen/pci.h         |   2 +
>  13 files changed, 186 insertions(+), 7 deletions(-)
>  create mode 100644 xen/arch/arm/vpci.c
>  create mode 100644 xen/arch/arm/vpci.h
> 
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 44d7cc81fa..fb9c976ea2 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
>  obj-y += platforms/
>  endif
>  obj-$(CONFIG_TEE) += tee/
> +obj-$(CONFIG_HAS_VPCI) += vpci.o
>  
>  obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>  obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..fbb52f78f1 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -39,6 +39,7 @@
>  #include <asm/vgic.h>
>  #include <asm/vtimer.h>
>  
> +#include "vpci.h"
>  #include "vuart.h"
>  
>  DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
> @@ -622,8 +623,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>  {
>      unsigned int max_vcpus;
>  
> -    /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
> +    /* HVM and HAP must be set. IOMMU and VPCI may or may not be */
> +    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu & ~XEN_DOMCTL_CDF_vpci) !=
>           (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>      {
>          dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
> @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d,
>      if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) )
>          goto fail;
>  
> +    if ( (rc = domain_vpci_init(d)) != 0 )
> +        goto fail;
> +
>      return 0;
>  
>  fail:
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d634c1..1731ae2028 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2773,6 +2773,9 @@ void __init create_dom0(void)
>      if ( iommu_enabled )
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
> +    if ( is_pci_passthrough_enabled() )
> +        dom0_cfg.flags |= XEN_DOMCTL_CDF_vpci;
> +
>      dom0 = domain_create(0, &dom0_cfg, true);
>      if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>          panic("Error creating domain 0\n");
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> new file mode 100644
> index 0000000000..76c12b9281
> --- /dev/null
> +++ b/xen/arch/arm/vpci.c
> @@ -0,0 +1,102 @@
> +/*
> + * xen/arch/arm/vpci.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +#include <xen/sched.h>
> +
> +#include <asm/mmio.h>
> +
> +#define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
> +
> +/* Do some sanity checks. */
> +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
> +{
> +    /* Check access size. */
> +    if ( len > 8 )
> +        return false;
> +
> +    /* Check that access is size aligned. */
> +    if ( (reg & (len - 1)) )
> +        return false;
> +
> +    return true;
> +}
> +
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                          register_t *r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = ~0UL;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    data = vpci_read(sbdf, reg, min(4u, size));
> +    if ( size == 8 )
> +        data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
> +
> +    *r = data;
> +
> +    return 1;
> +}
> +
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                           register_t r, void *p)
> +{
> +    unsigned int reg;
> +    pci_sbdf_t sbdf;
> +    unsigned long data = r;
> +    unsigned int size = 1U << info->dabt.size;
> +
> +    sbdf.sbdf = MMCFG_BDF(info->gpa);
> +    reg = REGISTER_OFFSET(info->gpa);
> +
> +    if ( !vpci_mmio_access_allowed(reg, size) )
> +        return 0;
> +
> +    vpci_write(sbdf, reg, min(4u, size), data);
> +    if ( size == 8 )
> +        vpci_write(sbdf, reg + 4, 4, data >> 32);
> +
> +    return 1;
> +}
> +
> +static const struct mmio_handler_ops vpci_mmio_handler = {
> +    .read  = vpci_mmio_read,
> +    .write = vpci_mmio_write,
> +};
> +
> +int domain_vpci_init(struct domain *d)
> +{
> +    if ( !has_vpci(d) )
> +        return 0;
> +
> +    register_mmio_handler(d, &vpci_mmio_handler,
> +                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
> +
> +    return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> +
> diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
> new file mode 100644
> index 0000000000..d8a7b0e3e8
> --- /dev/null
> +++ b/xen/arch/arm/vpci.h
> @@ -0,0 +1,36 @@
> +/*
> + * xen/arch/arm/vpci.h
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __ARCH_ARM_VPCI_H__
> +#define __ARCH_ARM_VPCI_H__
> +
> +#ifdef CONFIG_HAS_VPCI
> +int domain_vpci_init(struct domain *d);
> +#else
> +static inline int domain_vpci_init(struct domain *d)
> +{
> +    return 0;
> +}
> +#endif
> +
> +#endif /* __ARCH_ARM_VPCI_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index ef1812dc14..d7a48781c5 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -662,6 +662,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }
>  
> +    if ( config->flags & XEN_DOMCTL_CDF_vpci )
> +    {
> +        dprintk(XENLOG_INFO, "VPCI not supported\n");
> +        return -EINVAL;
> +    }
> +
>      if ( config->vmtrace_size )
>      {
>          unsigned int size = config->vmtrace_size;
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6ee5d033b0..40d67ec342 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>           ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>             XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>             XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> -           XEN_DOMCTL_CDF_nested_virt) )
> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpci) )
>      {
>          dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>          return -EINVAL;
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index d774a6154e..633e89ac13 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>      else
>          iommu_enable_device(pdev);
>  
> +#ifdef CONFIG_ARM
> +    /*
> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
> +     * Dom0 inform XEN to add the PCI devices in XEN.
> +     */
> +    ret = vpci_add_handlers(pdev);
> +    if ( ret ) {
> +        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);
> +        goto out;
> +    }
> +#endif
> +
>      pci_enable_acs(pdev);
>  
>  out:
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..5d9dd6b0d0 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -2,6 +2,7 @@
>  #define __ASM_DOMAIN_H__
>  
>  #include <xen/cache.h>
> +#include <xen/nospec.h>
>  #include <xen/timer.h>
>  #include <asm/page.h>
>  #include <asm/p2m.h>
> @@ -262,7 +263,12 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
>  
>  #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
>  
> -#define has_vpci(d)    ({ (void)(d); false; })
> +/*
> + * For X86 VPCI is enabled and tested for PVH DOM0 only but
> + * for ARM we enable support VPCI for guest domain also.
> + */
> +#define has_vpci(d) ((void)(d), \
> +        evaluate_nospec(d->options & XEN_DOMCTL_CDF_vpci))
>  
>  #endif /* __ASM_DOMAIN_H__ */
>  
> diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
> index 3f806ce7a8..61b940c91d 100644
> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -6,8 +6,6 @@
>  #define CF8_ADDR_HI(cf8) (  ((cf8) & 0x0f000000) >> 16)
>  #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000))
>  
> -#define MMCFG_BDF(addr)  ( ((addr) & 0x0ffff000) >> 12)
> -
>  #define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \
>                          || id == 0x01268086 || id == 0x01028086 \
>                          || id == 0x01128086 || id == 0x01228086 \
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 6b5a5f818a..727541a321 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -416,6 +416,13 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
>  #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
>  
> +/*
> + * 256 MB is reserved for VPCI configuration space based on calculation
> + * 256 buses × 32 devices × 8 functions × 4 KB = 256 MB
> + */
> +#define GUEST_VPCI_ECAM_BASE    xen_mk_ullong(0x10000000)
> +#define GUEST_VPCI_ECAM_SIZE    xen_mk_ullong(0x10000000)
> +
>  /* ACPI tables physical address */
>  #define GUEST_ACPI_BASE xen_mk_ullong(0x20000000)
>  #define GUEST_ACPI_SIZE xen_mk_ullong(0x02000000)
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842..4245da3f45 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -70,9 +70,11 @@ struct xen_domctl_createdomain {
>  #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
>  #define _XEN_DOMCTL_CDF_nested_virt   6
>  #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
> +#define _XEN_DOMCTL_CDF_vpci          7
> +#define XEN_DOMCTL_CDF_vpci           (1U << _XEN_DOMCTL_CDF_vpci)
>  
>  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_vpci
>  
>      uint32_t flags;
>  
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index 70ac25345c..43b8a08170 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -40,6 +40,8 @@
>  #define PCI_SBDF3(s,b,df) \
>      ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) })
>  
> +#define MMCFG_BDF(addr)  (((addr) & 0x0ffff000) >> 12)
> +
>  typedef union {
>      uint32_t sbdf;
>      struct {
> -- 
> 2.17.1
> 
--8323329-1855915649-1632872724=:5022--


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:46:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:46:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199277.353227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVciq-0006hP-PL; Wed, 29 Sep 2021 16:46:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199277.353227; Wed, 29 Sep 2021 16:46:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVciq-0006hI-ML; Wed, 29 Sep 2021 16:46:20 +0000
Received: by outflank-mailman (input) for mailman id 199277;
 Wed, 29 Sep 2021 16:46:19 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcip-0006fu-5A
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:46:19 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e4a5e4d0-fe18-43fa-ada7-7f6233ae9aa6;
 Wed, 29 Sep 2021 16:46:18 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C550161262;
 Wed, 29 Sep 2021 16:46:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e4a5e4d0-fe18-43fa-ada7-7f6233ae9aa6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632933977;
	bh=0C5ebCP15EqzFvJnPAZfSX9Cho3Km9f0yDxkWA9Uxyw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=TsaFJ6mJfp1vkC/LhIUZbuPItMgGyok1beGvqrrTn94cs1lBnIv3vg/7G6qFFqkSF
	 e+uQTp9se1lSJe6BGQd9mTgf5x495/tORdW1ZiWUKB+jpYbRtnjm2SGfuz1W88tgOm
	 520ZEGh1NPTw/YHd5mBMhnulVWvvl5hDVHa52a3FIH5CUcxlYX6ku/hPAuuNCDJCDh
	 j1GFT7505dyA0uuwexsNJZigjM4U4y1F6uC7MqpZp+DtC2TK235v7Z30iYHJE0REJs
	 gjHNTp6xNnD0cjXhpgyImujsm1NRpu/d/KkYleeb4JGQGm1dT7W001bbZ+zzrrLST1
	 QNDypwS4+lERQ==
Date: Wed, 29 Sep 2021 09:46:17 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 15/17] xen/arm: Transitional change to build HAS_VPCI
 on ARM.
In-Reply-To: <adabef3cd1774ca51f1b409bd97a2ae353eaac2b.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281645490.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <adabef3cd1774ca51f1b409bd97a2ae353eaac2b.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> This patch will be reverted once we add support for VPCI MSI/MSIX
> support on ARM.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3: none
> Change in v2: Patch introduced in v2
> ---
>  xen/drivers/vpci/Makefile | 3 ++-
>  xen/drivers/vpci/header.c | 2 ++
>  xen/include/asm-arm/pci.h | 8 ++++++++
>  3 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
> index 55d1bdfda0..1a1413b93e 100644
> --- a/xen/drivers/vpci/Makefile
> +++ b/xen/drivers/vpci/Makefile
> @@ -1 +1,2 @@
> -obj-y += vpci.o header.o msi.o msix.o
> +obj-y += vpci.o header.o
> +obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
> diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
> index ba9a036202..f8cd55e7c0 100644
> --- a/xen/drivers/vpci/header.c
> +++ b/xen/drivers/vpci/header.c
> @@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
>       * FIXME: punching holes after the p2m has been set up might be racy for
>       * DomU usage, needs to be revisited.
>       */
> +#ifdef CONFIG_HAS_PCI_MSI
>      if ( map && !rom_only && vpci_make_msix_hole(pdev) )
>          return;
> +#endif
>  
>      for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
>      {
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 49c9622902..5532ce3977 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -26,6 +26,14 @@ struct arch_pci_dev {
>      struct device dev;
>  };
>  
> +/* Arch-specific MSI data for vPCI. */
> +struct vpci_arch_msi {
> +};
> +
> +/* Arch-specific MSI-X entry data for vPCI. */
> +struct vpci_arch_msix_entry {
> +};
> +
>  /*
>   * struct to hold the mappings of a config space window. This
>   * is expected to be used as sysdata for PCI controllers that
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:47:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:47:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199285.353239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcju-0007U6-3N; Wed, 29 Sep 2021 16:47:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199285.353239; Wed, 29 Sep 2021 16:47:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcju-0007Tz-0Q; Wed, 29 Sep 2021 16:47:26 +0000
Received: by outflank-mailman (input) for mailman id 199285;
 Wed, 29 Sep 2021 16:47:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcjs-0007Tr-Qa
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:47:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id bbe2b518-59df-4d9a-8084-8f42d47091f4;
 Wed, 29 Sep 2021 16:47:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id A93F36142A;
 Wed, 29 Sep 2021 16:47:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbe2b518-59df-4d9a-8084-8f42d47091f4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632934042;
	bh=ivj9hFXx7OOIFh8x2VnRKOVQNI2v8LjxZccm6gngLrw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=AkWUbUNOEn4H+Vmk2KTfSPGQpN7Wn35lC3R4KsXHOf/5r74qyPoPmkodoXduJHTIT
	 CoXwq7EineTJR8+D7uduNw3jwcrU21hzMqLUufkXW4bobRv2hXSdDALl69jbBO5v5d
	 yKK7wffzPDBiPE2QdvoePgdm7Ho+S7DL3hKv/TQDTUC1xzKH8DdW7Sna9QoCjOX/wy
	 w/HTnWrJfA4ukCJLVPtDNQwQkxLS5hxv0Jt6mRis1k9cpuJywakvxDEmghdyUUedVD
	 qWVJKE5JW9p1I90uV8hhWLFv13humAuCYy8kpuW96+sRUapERVDLNPFb7pBgFHnz6J
	 fA8E5jui88Wtg==
Date: Wed, 29 Sep 2021 09:47:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Ian Jackson <iwj@xenproject.org>, 
    Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 16/17] arm/libxl: Emulated PCI device tree node in
 libxl
In-Reply-To: <7f19d0802a9ac7544ddefe96f282ba7f97caefe9.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281658320.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <7f19d0802a9ac7544ddefe96f282ba7f97caefe9.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> libxl will create an emulated PCI device tree node in the device tree to
> enable the guest OS to discover the virtual PCI during guest boot.
> Emulated PCI device tree node will only be created when there is any
> device assigned to guest.
> 
> A new area has been reserved in the arm guest physical map at
> which the VPCI bus is declared in the device tree (reg and ranges
> parameters of the node).
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Make GUEST_VPCI_MEM_ADDR address 2MB aligned
> Change in v2:
> - enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain.
> ---
>  tools/include/libxl.h            |   6 ++
>  tools/libs/light/libxl_arm.c     | 105 +++++++++++++++++++++++++++++++
>  tools/libs/light/libxl_create.c  |   3 +
>  tools/libs/light/libxl_types.idl |   1 +
>  tools/xl/xl_parse.c              |   2 +
>  xen/include/public/arch-arm.h    |  10 +++
>  6 files changed, 127 insertions(+)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d698..3362073b21 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -358,6 +358,12 @@
>   */
>  #define LIBXL_HAVE_BUILDINFO_ARM_VUART 1
>  
> +/*
> + * LIBXL_HAVE_BUILDINFO_ARM_VPCI indicates that the toolstack supports virtual
> + * PCI for ARM.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARM_VPCI 1
> +
>  /*
>   * LIBXL_HAVE_BUILDINFO_GRANT_LIMITS indicates that libxl_domain_build_info
>   * has the max_grant_frames and max_maptrack_frames fields.
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e00..52f1ddce48 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
>      return fdt_property(fdt, "reg", regs, sizeof(regs));
>  }
>  
> +static int fdt_property_values(libxl__gc *gc, void *fdt,
> +        const char *name, unsigned num_cells, ...)
> +{
> +    uint32_t prop[num_cells];
> +    be32 *cells = &prop[0];
> +    int i;
> +    va_list ap;
> +    uint32_t arg;
> +
> +    va_start(ap, num_cells);
> +    for (i = 0 ; i < num_cells; i++) {
> +        arg = va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, name, prop, sizeof(prop));
> +}
> +
> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
> +                                    unsigned addr_cells,
> +                                    unsigned size_cells,
> +                                    unsigned num_regs, ...)
> +{
> +    uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)];
> +    be32 *cells = &regs[0];
> +    int i;
> +    va_list ap;
> +    uint64_t arg;
> +
> +    va_start(ap, num_regs);
> +    for (i = 0 ; i < num_regs; i++) {
> +        /* Set the memory bit field */
> +        arg = va_arg(ap, uint32_t);
> +        set_cell(&cells, 1, arg);
> +
> +        /* Set the vpci bus address */
> +        arg = addr_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, addr_cells , arg);
> +
> +        /* Set the cpu bus address where vpci address is mapped */
> +        set_cell(&cells, addr_cells, arg);
> +
> +        /* Set the vpci size requested */
> +        arg = size_cells ? va_arg(ap, uint64_t) : 0;
> +        set_cell(&cells, size_cells, arg);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, "ranges", regs, sizeof(regs));
> +}
> +
>  static int make_root_properties(libxl__gc *gc,
>                                  const libxl_version_info *vers,
>                                  void *fdt)
> @@ -668,6 +720,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
>      return 0;
>  }
>  
> +static int make_vpci_node(libxl__gc *gc, void *fdt,
> +        const struct arch_info *ainfo,
> +        struct xc_dom_image *dom)
> +{
> +    int res;
> +    const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
> +    const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
> +    const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic");
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "device_type", "pci");
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +            GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size);
> +    if (res) return res;
> +
> +    res = fdt_property_values(gc, fdt, "bus-range", 2, 0, 255);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 3);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", 2);
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "status", "okay");
> +    if (res) return res;
> +
> +    res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +        GUEST_ROOT_SIZE_CELLS, 2,
> +        GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE,
> +        GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR,
> +        GUEST_VPCI_PREFETCH_MEM_SIZE);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
>  static const struct arch_info *get_arch_info(libxl__gc *gc,
>                                               const struct xc_dom_image *dom)
>  {
> @@ -971,6 +1070,9 @@ next_resize:
>          if (info->tee == LIBXL_TEE_TYPE_OPTEE)
>              FDT( make_optee_node(gc, fdt) );
>  
> +        if (libxl_defbool_val(info->arch_arm.vpci))
> +            FDT( make_vpci_node(gc, fdt, ainfo, dom) );
> +
>          if (pfdt)
>              FDT( copy_partial_fdt(gc, fdt, pfdt) );
>  
> @@ -1189,6 +1291,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
>      /* ACPI is disabled by default */
>      libxl_defbool_setdefault(&b_info->acpi, false);
>  
> +    /* VPCI is disabled by default */
> +    libxl_defbool_setdefault(&b_info->arch_arm.vpci, false);
> +
>      if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
>          return;
>  
> diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
> index e356b2106d..529e475489 100644
> --- a/tools/libs/light/libxl_create.c
> +++ b/tools/libs/light/libxl_create.c
> @@ -632,6 +632,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
>          if (info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
>              create.iommu_opts |= XEN_DOMCTL_IOMMU_no_sharept;
>  
> +        if ( libxl_defbool_val(b_info->arch_arm.vpci) )
> +            create.flags |= XEN_DOMCTL_CDF_vpci;
> +
>          /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
>          libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
>  
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 3f9fff653a..78b1ddf0b8 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>  
>      ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                 ("vuart", libxl_vuart_type),
> +                               ("vpci", libxl_defbool),
>                                ])),
>      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                                ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd5..ffafbeffb4 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source,
>          }
>          if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV)
>              libxl_defbool_set(&b_info->u.pv.e820_host, true);
> +        if (d_config->num_pcidevs)
> +            libxl_defbool_set(&b_info->arch_arm.vpci, true);
>      }
>  
>      if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 727541a321..acd97eb327 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -431,6 +431,11 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_PL011_BASE    xen_mk_ullong(0x22000000)
>  #define GUEST_PL011_SIZE    xen_mk_ullong(0x00001000)
>  
> +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/
> +#define GUEST_VPCI_ADDR_TYPE_MEM            xen_mk_ullong(0x02000000)
> +#define GUEST_VPCI_MEM_ADDR                 xen_mk_ullong(0x23000000)
> +#define GUEST_VPCI_MEM_SIZE                 xen_mk_ullong(0x10000000)
> +
>  /*
>   * 16MB == 4096 pages reserved for guest to use as a region to map its
>   * grant table in.
> @@ -446,6 +451,11 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_RAM0_BASE   xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1GB */
>  #define GUEST_RAM0_SIZE   xen_mk_ullong(0xc0000000)
>  
> +/* 4GB @ 4GB Prefetch Memory for VPCI */
> +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM   xen_mk_ullong(0x42000000)
> +#define GUEST_VPCI_PREFETCH_MEM_ADDR        xen_mk_ullong(0x100000000)
> +#define GUEST_VPCI_PREFETCH_MEM_SIZE        xen_mk_ullong(0x100000000)
> +
>  #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ 8GB */
>  #define GUEST_RAM1_SIZE   xen_mk_ullong(0xfe00000000)
>  
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:48:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:48:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199291.353250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVckr-000857-EJ; Wed, 29 Sep 2021 16:48:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199291.353250; Wed, 29 Sep 2021 16:48:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVckr-000850-BA; Wed, 29 Sep 2021 16:48:25 +0000
Received: by outflank-mailman (input) for mailman id 199291;
 Wed, 29 Sep 2021 16:48:24 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVckq-00084s-7Z
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:48:24 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a335ff98-c219-4189-8876-f98992f65417;
 Wed, 29 Sep 2021 16:48:23 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 90B0B6142A;
 Wed, 29 Sep 2021 16:48:22 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a335ff98-c219-4189-8876-f98992f65417
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632934102;
	bh=+ddRhEXcQ9gfIkluZQrMrifayFfgn4cW+gKLN1SNfpg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Fu0XWwFBXXODPjBi0vT5VZ0GTa6S5SdcmtdwGcNBTjuSTZ9osRb/XwAosH1NLMWhP
	 ka75RhD17c6/YAzF6OBuNQacSHeKvAkFWGebWCIplT17A45YlW1hZlBcqTjBoenAIr
	 aohaXbjDP1UXsRIntmfE+XXIfifvC/lj2XCoRjZyqbbo3Y9yZ15RXkEwnYAZjqB8qr
	 W03Q/DoUb+5Tdcd8SDlXRJ02WEOgyef30yCUpde2qf6pLxE45ixg5PZyk2MhAmKKx8
	 D1ak5nDLjQX8K7zAQ4KUPERBuXL8k3OzT7vuUSU8wBk8WHXJgi5yEmy83IYkbdqiTH
	 GULWAN870CTNQ==
Date: Wed, 29 Sep 2021 09:48:22 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Stefano Stabellini <sstabellini@kernel.org>, 
    Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 17/17] xen/arm: Add linux,pci-domain property for
 hwdom if not available.
In-Reply-To: <f290d8e2f9b3780c5a956a8032aba6f35de83a90.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109281659170.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <f290d8e2f9b3780c5a956a8032aba6f35de83a90.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> If the property is not present in the device tree node for host bridge,
> XEN while creating the dtb for hwdom will create this property and
> assigns the already allocated segment to the host bridge
> so that XEN and linux will have the same segment for the host bridges.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3:
> - Use is_pci_passthrough_enabled()
> Change in v2:
> - Add linux,pci-domain only when pci-passthrough command line option is enabeld
> ---
>  xen/arch/arm/domain_build.c        | 16 ++++++++++++++++
>  xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++
>  xen/include/asm-arm/pci.h          |  9 +++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 1731ae2028..026c9e5c6c 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -743,6 +743,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>              return res;
>      }
>  
> +    if ( is_pci_passthrough_enabled() && dt_device_type_is_equal(node, "pci") )
> +    {
> +        if ( !dt_find_property(node, "linux,pci-domain", NULL) )
> +        {
> +            uint16_t segment;
> +
> +            res = pci_get_host_bridge_segment(node, &segment);
> +            if ( res < 0 )
> +                return res;
> +
> +            res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment);
> +            if ( res )
> +                return res;
> +        }
> +    }
> +
>      /*
>       * Override the property "status" to disable the device when it's
>       * marked for passthrough.
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index c5941b10e9..593beeb48c 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -255,6 +255,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
>  
>      return NULL;
>  }
> +
> +/*
> + * This function will lookup an hostbridge based on config space address.
> + */
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment)
> +{
> +    struct pci_host_bridge *bridge;
> +
> +    list_for_each_entry( bridge, &pci_host_bridges, node )
> +    {
> +        if ( bridge->dt_node != node )
> +            continue;
> +
> +        *segment = bridge->segment;
> +        return 0;
> +    }
> +
> +    return -EINVAL;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
> index 5532ce3977..7cb2e2f1ed 100644
> --- a/xen/include/asm-arm/pci.h
> +++ b/xen/include/asm-arm/pci.h
> @@ -90,6 +90,8 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
>  void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
>                                 pci_sbdf_t sbdf, uint32_t where);
>  struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
> +int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                uint16_t *segment);
>  
>  static always_inline bool is_pci_passthrough_enabled(void)
>  {
> @@ -104,5 +106,12 @@ static always_inline bool is_pci_passthrough_enabled(void)
>      return false;
>  }
>  
> +static inline int pci_get_host_bridge_segment(const struct dt_device_node *node,
> +                                              uint16_t *segment)
> +{
> +    ASSERT_UNREACHABLE();
> +    return -EINVAL;
> +}
> +
>  #endif  /*!CONFIG_HAS_PCI*/
>  #endif /* __ARM_PCI_H__ */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 16:56:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 16:56:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199296.353261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcsi-00016V-9Y; Wed, 29 Sep 2021 16:56:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199296.353261; Wed, 29 Sep 2021 16:56:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVcsi-00016O-5w; Wed, 29 Sep 2021 16:56:32 +0000
Received: by outflank-mailman (input) for mailman id 199296;
 Wed, 29 Sep 2021 16:56:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=arYX=OT=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mVcsg-00016I-J1
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 16:56:30 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 100b2c7e-a5d7-4c3c-b1d2-e0d26321b7aa;
 Wed, 29 Sep 2021 16:56:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 4667961425;
 Wed, 29 Sep 2021 16:56:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 100b2c7e-a5d7-4c3c-b1d2-e0d26321b7aa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1632934588;
	bh=pRsdFcGDKsvRQA0D5Ma5OJ94jcX779GXwOP6Rb9P6K4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ux2v2u7avh9Awd+AtoEyQHlKnMcno/Bkb7vUuVhz+ARyMS3VNS7uHmFjw9xjqo9dY
	 u68akB2rrClrCx7f1kV30QhQRTu6OIPTXFK4eKoJAvtPQzbYBKRxkrQ9+yXufr/4ID
	 HA48X+WgQuVz7qSQ1rfTwGRJzMYqMXxbSzoC6WA+OKmYZthijTc03DPS2OmpSwWupw
	 mmNG5GCnq/uyBFZu5cOROS5w4pVKNoaSUFYFvcrG1DRDUfXP+Bdr/8BaN2ehPwH6Ks
	 pUsIMjQeYljgaX26rTB+KxtcbKTzfcVX4oUFk5z+FtXOBm6KEAjhk4L8hbefoPGSN5
	 +WB2JNta4hEtw==
Date: Wed, 29 Sep 2021 09:56:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Rahul Singh <rahul.singh@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, 
    Andre.Przywara@arm.com, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, 
    Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Paul Durrant <paul@xen.org>, 
    Daniel De Graaf <dgdegra@tycho.nsa.gov>, 
    "Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: Re: [PATCH v3 01/17] xen/pci: Refactor MSI code that implements MSI
 functionality within XEN
In-Reply-To: <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
Message-ID: <alpine.DEB.2.21.2109290956180.5022@sstabellini-ThinkPad-T480s>
References: <cover.1632847120.git.rahul.singh@arm.com> <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 28 Sep 2021, Rahul Singh wrote:
> On Arm, the initial plan is to only support GICv3 ITS which doesn't
> require us to manage the MSIs because the HW will protect against
> spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
> for ARM.
> 
> No functional change intended.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Change in v3: none 
> Change in v2: Fixed minor comments
> ---
>  xen/arch/x86/Kconfig             |  1 +
>  xen/drivers/passthrough/Makefile |  1 +
>  xen/drivers/passthrough/msi.c    | 83 ++++++++++++++++++++++++++++++++
>  xen/drivers/passthrough/pci.c    | 54 +++++----------------
>  xen/drivers/pci/Kconfig          |  4 ++
>  xen/include/xen/msi.h            | 43 +++++++++++++++++
>  xen/xsm/flask/hooks.c            |  8 +--
>  7 files changed, 149 insertions(+), 45 deletions(-)
>  create mode 100644 xen/drivers/passthrough/msi.c
>  create mode 100644 xen/include/xen/msi.h
> 
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 1f83518ee0..b4abfca46f 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -20,6 +20,7 @@ config X86
>  	select HAS_NS16550
>  	select HAS_PASSTHROUGH
>  	select HAS_PCI
> +	select HAS_PCI_MSI
>  	select HAS_PDX
>  	select HAS_SCHED_GRANULARITY
>  	select HAS_UBSAN
> diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile
> index 445690e3e5..a5efa22714 100644
> --- a/xen/drivers/passthrough/Makefile
> +++ b/xen/drivers/passthrough/Makefile
> @@ -7,3 +7,4 @@ obj-y += iommu.o
>  obj-$(CONFIG_HAS_PCI) += pci.o
>  obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
>  obj-$(CONFIG_HAS_PCI) += ats.o
> +obj-$(CONFIG_HAS_PCI_MSI) += msi.o
> diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.c
> new file mode 100644
> index 0000000000..ce1a450f6f
> --- /dev/null
> +++ b/xen/drivers/passthrough/msi.c
> @@ -0,0 +1,83 @@
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +#include <xen/msi.h>
> +#include <asm/hvm/io.h>
> +
> +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
> +{
> +    int rc;
> +
> +    if ( pdev->msix )
> +    {
> +        rc = pci_reset_msix_state(pdev);
> +        if ( rc )
> +            return rc;
> +        msixtbl_init(d);
> +    }
> +
> +    return 0;
> +}
> +
> +int pdev_msi_init(struct pci_dev *pdev)
> +{
> +    unsigned int pos;
> +
> +    INIT_LIST_HEAD(&pdev->msi_list);
> +
> +    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
> +                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSI);
> +    if ( pos )
> +    {
> +        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
> +
> +        pdev->msi_maxvec = multi_msi_capable(ctrl);
> +    }
> +
> +    pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn),
> +                              PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSIX);
> +    if ( pos )
> +    {
> +        struct arch_msix *msix = xzalloc(struct arch_msix);
> +        uint16_t ctrl;
> +
> +        if ( !msix )
> +            return -ENOMEM;
> +
> +        spin_lock_init(&msix->table_lock);
> +
> +        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
> +        msix->nr_entries = msix_table_size(ctrl);
> +
> +        pdev->msix = msix;
> +    }
> +
> +    return 0;
> +}
> +
> +void pdev_msi_deinit(struct pci_dev *pdev)
> +{
> +    XFREE(pdev->msix);
> +}
> +
> +void pdev_dump_msi(const struct pci_dev *pdev)
> +{
> +    const struct msi_desc *msi;
> +
> +    if ( list_empty(&pdev->msi_list) )
> +        return;
> +
> +    printk(" - MSIs < ");
> +    list_for_each_entry ( msi, &pdev->msi_list, list )
> +        printk("%d ", msi->irq);
> +    printk(">");
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
> index fc4fa2e5c3..8996403161 100644
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -32,8 +32,8 @@
>  #include <xen/softirq.h>
>  #include <xen/tasklet.h>
>  #include <xen/vpci.h>
> +#include <xen/msi.h>
>  #include <xsm/xsm.h>
> -#include <asm/msi.h>
>  #include "ats.h"
>  
>  struct pci_seg {
> @@ -314,6 +314,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>  {
>      struct pci_dev *pdev;
>      unsigned int pos;
> +    int rc;
>  
>      list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
>          if ( pdev->bus == bus && pdev->devfn == devfn )
> @@ -327,35 +328,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
>      *((u8*) &pdev->bus) = bus;
>      *((u8*) &pdev->devfn) = devfn;
>      pdev->domain = NULL;
> -    INIT_LIST_HEAD(&pdev->msi_list);
> -
> -    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> -                              PCI_CAP_ID_MSI);
> -    if ( pos )
> -    {
> -        uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos));
> -
> -        pdev->msi_maxvec = multi_msi_capable(ctrl);
> -    }
>  
> -    pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
> -                              PCI_CAP_ID_MSIX);
> -    if ( pos )
> +    rc = pdev_msi_init(pdev);
> +    if ( rc )
>      {
> -        struct arch_msix *msix = xzalloc(struct arch_msix);
> -        uint16_t ctrl;
> -
> -        if ( !msix )
> -        {
> -            xfree(pdev);
> -            return NULL;
> -        }
> -        spin_lock_init(&msix->table_lock);
> -
> -        ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos));
> -        msix->nr_entries = msix_table_size(ctrl);
> -
> -        pdev->msix = msix;
> +        xfree(pdev);
> +        return NULL;
>      }
>  
>      list_add(&pdev->alldevs_list, &pseg->alldevs_list);
> @@ -449,7 +427,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev)
>      }
>  
>      list_del(&pdev->alldevs_list);
> -    xfree(pdev->msix);
> +    pdev_msi_deinit(pdev);
>      xfree(pdev);
>  }
>  
> @@ -1271,18 +1249,16 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev)
>  static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
>  {
>      struct pci_dev *pdev;
> -    struct msi_desc *msi;
>  
>      printk("==== segment %04x ====\n", pseg->nr);
>  
>      list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
>      {
> -        printk("%pp - %pd - node %-3d - MSIs < ",
> +        printk("%pp - %pd - node %-3d",
>                 &pdev->sbdf, pdev->domain,
>                 (pdev->node != NUMA_NO_NODE) ? pdev->node : -1);
> -        list_for_each_entry ( msi, &pdev->msi_list, list )
> -               printk("%d ", msi->irq);
> -        printk(">\n");
> +        pdev_dump_msi(pdev);
> +        printk("\n");
>      }
>  
>      return 0;
> @@ -1422,13 +1398,9 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>      ASSERT(pdev && (pdev->domain == hardware_domain ||
>                      pdev->domain == dom_io));
>  
> -    if ( pdev->msix )
> -    {
> -        rc = pci_reset_msix_state(pdev);
> -        if ( rc )
> -            goto done;
> -        msixtbl_init(d);
> -    }
> +    rc = pdev_msix_assign(d, pdev);
> +    if ( rc )
> +        goto done;
>  
>      pdev->fault.count = 0;
>  
> diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
> index 7da03fa13b..c6a7bc8007 100644
> --- a/xen/drivers/pci/Kconfig
> +++ b/xen/drivers/pci/Kconfig
> @@ -1,3 +1,7 @@
>  
>  config HAS_PCI
>  	bool
> +
> +config HAS_PCI_MSI
> +	bool
> +	depends on HAS_PCI
> diff --git a/xen/include/xen/msi.h b/xen/include/xen/msi.h
> new file mode 100644
> index 0000000000..c903d0050c
> --- /dev/null
> +++ b/xen/include/xen/msi.h
> @@ -0,0 +1,43 @@
> +#ifndef __XEN_MSI_H_
> +#define __XEN_MSI_H_
> +
> +#include <xen/pci.h>
> +
> +#ifdef CONFIG_HAS_PCI_MSI
> +
> +#include <asm/msi.h>
> +
> +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev);
> +int pdev_msi_init(struct pci_dev *pdev);
> +void pdev_msi_deinit(struct pci_dev *pdev);
> +void pdev_dump_msi(const struct pci_dev *pdev);
> +
> +#else /* !CONFIG_HAS_PCI_MSI */
> +
> +static inline int pdev_msix_assign(struct domain *d, struct pci_dev *pdev)
> +{
> +    return 0;
> +}
> +
> +static inline int pdev_msi_init(struct pci_dev *pdev)
> +{
> +    return 0;
> +}
> +
> +static inline void pdev_msi_deinit(struct pci_dev *pdev) {}
> +static inline void pci_cleanup_msi(struct pci_dev *pdev) {}
> +static inline void pdev_dump_msi(const struct pci_dev *pdev) {}
> +
> +#endif /* CONFIG_HAS_PCI_MSI */
> +
> +#endif /* __XEN_MSI_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
> index 25e87180b4..ea9a12bd71 100644
> --- a/xen/xsm/flask/hooks.c
> +++ b/xen/xsm/flask/hooks.c
> @@ -21,7 +21,7 @@
>  #include <xen/guest_access.h>
>  #include <xen/xenoprof.h>
>  #include <xen/iommu.h>
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>  #include <asm/msi.h>
>  #endif
>  #include <public/xen.h>
> @@ -114,7 +114,7 @@ static int get_irq_sid(int irq, u32 *sid, struct avc_audit_data *ad)
>          }
>          return security_irq_sid(irq, sid);
>      }
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>      {
>          struct irq_desc *desc = irq_to_desc(irq);
>          if ( desc->msi_desc && desc->msi_desc->dev ) {
> @@ -874,7 +874,7 @@ static int flask_map_domain_pirq (struct domain *d)
>  static int flask_map_domain_msi (struct domain *d, int irq, const void *data,
>                                   u32 *sid, struct avc_audit_data *ad)
>  {
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>      const struct msi_info *msi = data;
>      u32 machine_bdf = (msi->seg << 16) | (msi->bus << 8) | msi->devfn;
>  
> @@ -940,7 +940,7 @@ static int flask_unmap_domain_pirq (struct domain *d)
>  static int flask_unmap_domain_msi (struct domain *d, int irq, const void *data,
>                                     u32 *sid, struct avc_audit_data *ad)
>  {
> -#ifdef CONFIG_HAS_PCI
> +#ifdef CONFIG_HAS_PCI_MSI
>      const struct pci_dev *pdev = data;
>      u32 machine_bdf = (pdev->seg << 16) | (pdev->bus << 8) | pdev->devfn;
>  
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 20:50:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 20:50:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199308.353275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVgWq-0000M5-Sz; Wed, 29 Sep 2021 20:50:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199308.353275; Wed, 29 Sep 2021 20:50:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVgWq-0000Ly-P2; Wed, 29 Sep 2021 20:50:12 +0000
Received: by outflank-mailman (input) for mailman id 199308;
 Wed, 29 Sep 2021 20:50:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVgWp-0000Ls-Jb
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 20:50:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVgWp-0000Vd-H8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 20:50:11 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVgWp-0001YO-G8
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 20:50:11 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVgWn-0000Gv-Qa; Wed, 29 Sep 2021 21:50:09 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:To:Date:
	Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=P035az/3F/SWq9Q9ubjL95zAELjd1lWm1l6ysAB2UEA=; b=i5et6UqVgcywc+ZWQhkh17/uj2
	AeLIcBDF912kgy7OfzxVbkGMw+02yakyaalWRyp6Pwv/mCL/O9TIJ0BGUIoBt9l5NBaA90j9TVfkb
	oYCCBBx2Fosea4cjxEZ8S4bdanLylZW47beJFcel68kLv73u1MjpCpjQRzfJtIh3mG2c=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24916.53633.593814.456485@mariner.uk.xensource.com>
Date: Wed, 29 Sep 2021 21:50:09 +0100
To: xen-devel@lists.xenproject.org,
    committers@xenproject.org
Subject: Re: osstest down, PDU failure
In-Reply-To: <24916.14069.358118.417330@mariner.uk.xensource.com>
References: <24916.14069.358118.417330@mariner.uk.xensource.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Ian Jackson writes ("osstest down, PDU failure"):
> Currently, osstest is not working.  We have lost one of our PDUs,
> meaning that about half a rack is out of action, including one of the
> VM hosts.
> 
> There has been quite a bit of outstanding maintenance which has been
> deferred due to the pandemic.  I am trying to see if we can get
> someone on-site to the colo, in Massachusetts, soon.  A complication
> is that the replacement PDU is in still New York.  Again, due to the
> pandemic.

I managed to get an on-site look by the staff of the colo facility.  A
breaker had tripped, depriving our PDU of power.  They reset the
breaker.  The VM host has come back fully operational.  I have
verified that all the test boxes connected to that PDU (apart from one
knonw-dead box) are powered and responsive enough.  Initial reports
from a smoke flight were encouraging, so I have re-enabled everything.

It may trip again of course.

A power trip in a colo is not a normal event, but we haven't
determined the root cause.  The colo facility are going to ask their
electrical supply technicians to investigate the trip.  I think the
breaker or associated equipment is probably "smart" and will have some
useful records.

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 21:36:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 21:36:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199314.353286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVhF9-00051c-8q; Wed, 29 Sep 2021 21:35:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199314.353286; Wed, 29 Sep 2021 21:35:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVhF9-00051V-5Y; Wed, 29 Sep 2021 21:35:59 +0000
Received: by outflank-mailman (input) for mailman id 199314;
 Wed, 29 Sep 2021 21:35:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVhF8-00051N-6X
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 21:35:58 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVhF8-0001Gi-45
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 21:35:58 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVhF8-0005kd-2z
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 21:35:58 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVhF3-0000V3-DA; Wed, 29 Sep 2021 22:35:53 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:To:Date:
	Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=2LgtL/bOBLKlY+f62iWCjTBNXfxnBCoS7kexrmF43FA=; b=LqkJhR2s/Y1OIzRItC9IVmDCmI
	rmy5MwE6rIK2ROtYrBbOUp7i8KKm/OqPss4HRyW0ssbs85nJULcAXLpdJ0u3b+qSMYnlK3MR2B3t3
	0O79Y6wScIkjm3/lchvb4Pe3Q8MYe5P8igyAdEcJCA0mKd2XWkKmbAylpule0r0dN2VA=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24916.56377.137361.34717@mariner.uk.xensource.com>
Date: Wed, 29 Sep 2021 22:35:53 +0100
To: Jan Beulich <jbeulich@suse.com>,
    Julien Grall <julien.grall.oss@gmail.com>,
    xen-devel <xen-devel@lists.xenproject.org>,
    Stefano Stabellini <sstabellini@kernel.org>,
    dpsmith@apertussolutions.com
Subject: Re: [xen-unstable test] 164996: regressions - FAIL
In-Reply-To: <24915.16323.60012.172236@mariner.uk.xensource.com>
References: <osstest-164996-mainreport@xen.org>
	<d049ba60-db81-aaa4-1769-54c6964cfd06@suse.com>
	<24904.44119.940679.241639@mariner.uk.xensource.com>
	<alpine.DEB.2.21.2109211631330.17979@sstabellini-ThinkPad-T480s>
	<19906cf8-365b-63c9-5c59-aef8dae41505@suse.com>
	<alpine.DEB.2.21.2109221807131.17979@sstabellini-ThinkPad-T480s>
	<CAJ=z9a1W9BqeOGFu3F2+qbXM6UNo8ZO=0y-zYsnK01MNSNdbkw@mail.gmail.com>
	<dda286dc-1edc-4505-a049-e82047b44590@suse.com>
	<24915.16323.60012.172236@mariner.uk.xensource.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

Ian Jackson writes ("Re: [xen-unstable test] 164996: regressions - FAIL"):
> Thanks.  The attachment didn't git-am but I managed to make a tree
> with it in (but a bogus commit message).
> 
> I have a repro of 165218 test-arm64-arm64-libvirt-raw (that's the last
> xen-unstable flight) running.  If all goes well it will rebuild Linux
> from my branch (new flight 165241) and then run the test using that
> kernel (new flight 165242).  I have told it to report to the people on
> this thread (and the list).
> 
> It will probably report in an hour or two (since it needs to rebuild a
> kernel and then negotiate to get a host to run the repro on).
> I didn't ask it to keep the host for me, but it ought to publish the
> logs and as I say, send an email report here.

Restarted as 165323 and 165324.  Maybe the thing won't catch fire this
time.  Unusual consequences for a small kernel patch :-).

Ian.


From xen-devel-bounces@lists.xenproject.org Wed Sep 29 21:45:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 21:45:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199319.353297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVhOe-0006bX-56; Wed, 29 Sep 2021 21:45:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199319.353297; Wed, 29 Sep 2021 21:45:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVhOe-0006bQ-1r; Wed, 29 Sep 2021 21:45:48 +0000
Received: by outflank-mailman (input) for mailman id 199319;
 Wed, 29 Sep 2021 21:45:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>) id 1mVhOc-0006bK-UN
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 21:45:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVhOc-0001RU-LU; Wed, 29 Sep 2021 21:45:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVhOc-0007gT-2T; Wed, 29 Sep 2021 21:45:46 +0000
Received: from iwj by osstest.test-lab.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVhOc-00083g-1q; Wed, 29 Sep 2021 21:45:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
To: xen-devel@lists.xenproject.org,jbeulich@suse.com,julien.grall.oss@gmail.com,iwj@xenproject.org,sstabellini@kernel.org,dpsmith@apertussolutions.com
Subject: [adhoc test] 165323: all pass
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Message-Id: <E1mVhOc-00083g-1q@osstest.test-lab.xenproject.org>
From: iwj@xenbits.xen.org
Date: Wed, 29 Sep 2021 21:45:46 +0000

[adhoc adhoc] <testing.git master /dev/pts/25>
harness 3a3089c9: mfi-common: Drop Linux dom0 i386 tests for newer Lin...
165323: all pass

flight 165323 xen-unstable adhoc [adhoc]
http://logs.test-lab.xenproject.org/osstest/logs/165323/

Perfect :-)
All tests in this flight passed as required

jobs:
 build-arm64-pvops                                            pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 22:52:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 22:52:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199328.353319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRO-0005up-7P; Wed, 29 Sep 2021 22:52:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199328.353319; Wed, 29 Sep 2021 22:52:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRO-0005ug-4F; Wed, 29 Sep 2021 22:52:42 +0000
Received: by outflank-mailman (input) for mailman id 199328;
 Wed, 29 Sep 2021 22:52:41 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XAMn=OT=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mViRN-0005eO-8A
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 22:52:41 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 39071c9b-55bf-45a7-acc5-da34a53ef60c;
 Wed, 29 Sep 2021 22:52:35 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id m3so16941328lfu.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 15:52:35 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id g26sm133178lja.16.2021.09.29.15.52.33
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 29 Sep 2021 15:52:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39071c9b-55bf-45a7-acc5-da34a53ef60c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=6iJhS8sphgwZjkcusuNiu4A9g9hD+Ik9xi5wqFSEXCA=;
        b=XfpD5CaoOHUa3T1eWlpV4iVp0qd4qArA6lMrXYi/2x+QtbZVzo2qdWz231Bd5KEOz2
         1FzHBouCfYWsTOTsVDRiTM0tPP+P0BW22fiNi14MLZKDDyMcAo6P8jykx8KW9/Uaillw
         gZKewKZ55Tom9y153hfyfvuqu89h85tHAEsg7LDkfYd6QXlj8R5j/u6Y/pPKHROtKlqP
         HU7RM+N0wFsB3xYN2Zc1ML4/1P5BGsEVatqCPUNvg285Jnz/kxN2pq9WWQG2hS+WTGxn
         l5H9CxpvbPWIjpIxDnVcZJkKJSw+1aClDjEIimKKQoQX5LsLg+JLmSNgyBfb7vPfySO8
         zmrA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=6iJhS8sphgwZjkcusuNiu4A9g9hD+Ik9xi5wqFSEXCA=;
        b=3QWJVBFBC2EIIMCb1cl4mwgO9pZPCyH9OxYogXT2jFj8yuuvk5uzfaA5EgqAOF+mIM
         9BejCxCGJsvB4NvkjjdZ4VomtZRlVfBvLypAzitOmMeBjduvLjbvR/Z/PWHdROej4x5C
         4eN/hdLZN9V8u4p7xdFWt0A/0tkuhAtajFTo5BzpvQJh7cIdgzXEP8KnHV9XEV5gm227
         ua/2v/kqjfwyxjJEUX07YyrTR8rHkuG47UE47e+FAj9OMAG8WFlFccmiBmzTBqsdmInd
         ew/R2Lp2m7oIkTE38Xc+I/0E9ttVN4jMddTGML9mkKDXH/jeA2gHxDU8BeV4UXW4pa9/
         2iow==
X-Gm-Message-State: AOAM533jybLkzBvycPpSPrgaShL8CI6BwxKd75JPvRZtvAumXJkb1kWX
	zm2kRC87bgo0UMrxIAUF4D8z5UKBcIhXqw==
X-Google-Smtp-Source: ABdhPJxHS/aqZSklH95g0ibf9cFZWdz2JQxvIIqc4Id9zhoH+ECAkzA68y4SqkB2/FDTSIw8AOuHKw==
X-Received: by 2002:a05:651c:12c6:: with SMTP id 6mr2570306lje.194.1632955954429;
        Wed, 29 Sep 2021 15:52:34 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH V4 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
Date: Thu, 30 Sep 2021 01:52:05 +0300
Message-Id: <1632955927-27911-2-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

We need to pass info about maximum supported guest address
space size to the toolstack on Arm in order to properly
calculate the base and size of the extended region (safe range)
for the guest. The extended region is unused address space which
could be safely used by domain for foreign/grant mappings on Arm.
The extended region itself will be handled by the subsequents
patch.

Use p2m_ipa_bits variable on Arm, the x86 equivalent is
hap_paddr_bits.

As we change the size of structure bump the interface version.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
Please note, that review comments for the RFC version [1] haven't been addressed yet.
It is not forgotten, some clarification is needed. It will be addressed for the next version.

[1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/

Changes RFC -> V2:
   - update patch subject/description
   - replace arch-specific sub-struct with common gpaddr_bits
     field and update code to reflect that

Changes V2 -> V3:
   - make the field uint8_t and add uint8_t pad[7] after
   - remove leading blanks in libxl.h

Changes V3 -> V4:
   - also print gpaddr_bits from output_physinfo()
   - add Michal's R-b
---
 tools/include/libxl.h            | 7 +++++++
 tools/libs/light/libxl.c         | 2 ++
 tools/libs/light/libxl_types.idl | 2 ++
 tools/xl/xl_info.c               | 2 ++
 xen/arch/arm/sysctl.c            | 2 ++
 xen/arch/x86/sysctl.c            | 2 ++
 xen/include/public/sysctl.h      | 4 +++-
 7 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d..63f9550 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -856,6 +856,13 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
 
 /*
+ * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
+ *
+ * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
+ */
+#define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
+
+/*
  * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1
  *
  * If this is defined, libxl_dominfo will contain a MemKB type field called
diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c
index 204eb0b..c86624f 100644
--- a/tools/libs/light/libxl.c
+++ b/tools/libs/light/libxl.c
@@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
     physinfo->cap_vmtrace =
         !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace);
 
+    physinfo->gpaddr_bits = xcphysinfo.gpaddr_bits;
+
     GC_FREE;
     return 0;
 }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff6..bf27fe6 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -1061,6 +1061,8 @@ libxl_physinfo = Struct("physinfo", [
     ("cap_shadow", bool),
     ("cap_iommu_hap_pt_share", bool),
     ("cap_vmtrace", bool),
+
+    ("gpaddr_bits", uint8),
     ], dir=DIR_OUT)
 
 libxl_connectorinfo = Struct("connectorinfo", [
diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
index 8383e4a..dfbbeaa 100644
--- a/tools/xl/xl_info.c
+++ b/tools/xl/xl_info.c
@@ -221,6 +221,8 @@ static void output_physinfo(void)
          info.cap_vmtrace ? " vmtrace" : ""
         );
 
+    maybe_printf("gpaddr_bits            : %d\n", info.gpaddr_bits);
+
     vinfo = libxl_get_version_info(ctx);
     if (vinfo) {
         i = (1 << 20) / vinfo->pagesize;
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index f87944e..91dca4f 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -15,6 +15,8 @@
 void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
 {
     pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
+
+    pi->gpaddr_bits = p2m_ipa_bits;
 }
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a1..7b14865 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
     if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow;
+
+    pi->gpaddr_bits = hap_paddr_bits;
 }
 
 long arch_do_sysctl(
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 039ccf8..0450a78 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -35,7 +35,7 @@
 #include "domctl.h"
 #include "physdev.h"
 
-#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014
 
 /*
  * Read console content from Xen buffer ring.
@@ -120,6 +120,8 @@ struct xen_sysctl_physinfo {
     uint64_aligned_t outstanding_pages;
     uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
     uint32_t hw_cap[8];
+    uint8_t gpaddr_bits;
+    uint8_t pad[7];
 };
 
 /*
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 22:52:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 22:52:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199330.353341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRY-0006a6-SC; Wed, 29 Sep 2021 22:52:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199330.353341; Wed, 29 Sep 2021 22:52:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRY-0006Zx-OQ; Wed, 29 Sep 2021 22:52:52 +0000
Received: by outflank-mailman (input) for mailman id 199330;
 Wed, 29 Sep 2021 22:52:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XAMn=OT=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mViRX-0005eO-8S
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 22:52:51 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c000e0a0-3405-4bf9-9bbb-3804c7cd0959;
 Wed, 29 Sep 2021 22:52:37 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id m3so16941616lfu.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 15:52:37 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id g26sm133178lja.16.2021.09.29.15.52.35
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 29 Sep 2021 15:52:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c000e0a0-3405-4bf9-9bbb-3804c7cd0959
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=/1JvRrp5W5uJkWjfb1ZqIFuL/StY5VJHuwwhfYh02cI=;
        b=fiDgjzxCvzMpU+cge8HG1YXhBeDd1ALRtgkbGFFTLjsGfFwVqKESbq9IB7CTe7YvWg
         c0wrNKIsfAJ1vfz/VOVEPBJNlj81Eado2i3ktjoCI4Cd9dzf+SpEIw+y9c7GPH0uAIzT
         J2XNG3IDV0nIazOUO+3TW9DFRglJXcUWJpfWpk4dIZVv7dsNRNXo1GpR7leJ5zLIySQB
         8ejfP3VByyW1G4srgR4Ao57FET1fXN3hnzwJBrCOEJlr5JoDWo5yQU09QngNYV1GyeYX
         TbyQ6ZOzvgCLj5wMsFJP2xLK9FS4Tr+CC0uHQxNntYQs2xDkIM/62qJPELr2URNSiihS
         clPg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=/1JvRrp5W5uJkWjfb1ZqIFuL/StY5VJHuwwhfYh02cI=;
        b=xJh/npI185PD0n3myoJ55OXveMLPNCFNm+I7kvUn//LQvxIQni2NVHqtCGFUK38XgP
         Rk5hIWh34nM8dBtMvOknrfl9pPjFNp0QS8Af0exhbYJl2HFl4kPRRXdOB2SyM5fMHx9u
         afAXXwCWvSoIRSgyuVigSdWbDRdTYvIPcewC4iqTrQamrmKNxyKeGASDOxtiMfb2y3xu
         JygE7f9axvgBvQSH3JatRKZJgYlBz2WvGlBsNH3nr4iAVmkDlwUfezdrtyMynbrX5Zjh
         nX+06o8qiAXUMzCCNeFHG7LXMRMvnym/WDxYm0Ld1FVe/NN8iB5yrshrC6Z4jq/KLiWt
         Lgig==
X-Gm-Message-State: AOAM530bzviR1kF+gtEuoFZcMUfaub+cYAiUwfQXhgWuiPDzlrC+Q9EG
	NJkok/M4Hvmm5RBG3UZ9pqOVFOg5TGnpLQ==
X-Google-Smtp-Source: ABdhPJzl6QAZiwF0YkTVqnJbdYmYGW4g0VGERnOjOyaoDOewIYtd11OLyFArNNk+PgEOqO6j7hBWKw==
X-Received: by 2002:a2e:a446:: with SMTP id v6mr2419279ljn.143.1632955956436;
        Wed, 29 Sep 2021 15:52:36 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH V4 3/3] libxl/arm: Add handling of extended regions for DomU
Date: Thu, 30 Sep 2021 01:52:07 +0300
Message-Id: <1632955927-27911-4-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain
  currently.
- The ACPI case is not covered.

***

The algorithm to choose extended regions for non-direct mapped
DomU is simpler in comparison with the algorithm for direct mapped
Dom0. As we have a lot of unused space above 4GB, provide single
2MB-aligned region from the second RAM bank taking into the account
the maximum supported guest address space size and the amount of
memory assigned to the guest. The maximum size of the region is 128GB.
The minimum size is 64MB.

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Ian Jackson <iwj@xenproject.org>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
Tested-by: Michal Orzel <michal.orzel@arm.com>
---
Changes RFC -> V2:
   - update patch description
   - drop uneeded "extended-region" DT property
   - clear reg array in finalise_ext_region() and add a TODO

Changes V2 -> V3:
   - update patch description, comments in code
   - only pick up regions with size >= 64MB
   - move the region calculation to make_hypervisor_node() and drop
     finalise_ext_region()
   - extend the list of arguments for make_hypervisor_node()
   - do not show warning for 32-bit domain
   - change the region alignment from 1GB to 2MB
   - move EXT_REGION_SIZE to public/arch-arm.h

Changes V3 -> V4:
   - add R-b, A-b and T-b
---
 tools/libs/light/libxl_arm.c  | 70 +++++++++++++++++++++++++++++++++++++++----
 xen/include/public/arch-arm.h |  3 ++
 2 files changed, 68 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6..a67b68e 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -598,9 +598,17 @@ static int make_timer_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+#define ALIGN_UP_TO_2MB(x)   (((x) + MB(2) - 1) & (~(MB(2) - 1)))
+
 static int make_hypervisor_node(libxl__gc *gc, void *fdt,
-                                const libxl_version_info *vers)
+                                const libxl_version_info *vers,
+                                const libxl_domain_build_info *b_info,
+                                const struct xc_dom_image *dom)
 {
+    uint64_t region_size = 0, region_base, ramsize, bank1size,
+        bank1end_align, bank1end_max;
+    uint8_t gpaddr_bits;
+    libxl_physinfo physinfo;
     int res;
     gic_interrupt intr;
 
@@ -615,9 +623,61 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
                               "xen,xen");
     if (res) return res;
 
-    /* reg 0 is grant table space */
-    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
-                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+    if (strcmp(dom->guest_type, "xen-3.0-aarch64")) {
+        LOG(DEBUG, "The extended regions are only supported for 64-bit guest currently");
+        goto out;
+    }
+
+    res = libxl_get_physinfo(CTX, &physinfo);
+    assert(!res);
+
+    gpaddr_bits = physinfo.gpaddr_bits;
+    assert(gpaddr_bits >= 32 && gpaddr_bits <= 48);
+
+    /*
+     * Try to allocate single 2MB-aligned extended region from the second RAM
+     * bank (above 4GB) taking into the account the maximum supported guest
+     * address space size and the amount of memory assigned to the guest.
+     * As the guest memory layout is not populated yet we cannot rely on
+     * dom->rambank_size[1], so calculate the actual size of the second bank
+     * using "max_memkb" value.
+     */
+    bank1end_max = min(1ULL << gpaddr_bits, GUEST_RAM1_BASE + GUEST_RAM1_SIZE);
+    ramsize = b_info->max_memkb * 1024;
+    if (ramsize <= GUEST_RAM0_SIZE)
+        bank1size = 0;
+    else
+        bank1size = ramsize - GUEST_RAM0_SIZE;
+    bank1end_align = GUEST_RAM1_BASE + ALIGN_UP_TO_2MB(bank1size);
+
+    if (bank1end_max <= bank1end_align) {
+        LOG(WARN, "The extended region cannot be allocated, not enough space");
+        goto out;
+    }
+
+    if (bank1end_max - bank1end_align > GUEST_EXT_REGION_MAX_SIZE) {
+        region_base = bank1end_max - GUEST_EXT_REGION_MAX_SIZE;
+        region_size = GUEST_EXT_REGION_MAX_SIZE;
+    } else {
+        region_base = bank1end_align;
+        region_size = bank1end_max - bank1end_align;
+    }
+
+out:
+    /*
+     * The region 0 for grant table space must be always present. If we managed
+     * to allocate the extended region then insert it as region 1.
+     */
+    if (region_size >= GUEST_EXT_REGION_MIN_SIZE) {
+        LOG(DEBUG, "Extended region: %#"PRIx64"->%#"PRIx64"\n",
+            region_base, region_base + region_size);
+
+        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                                2, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE,
+                                region_base, region_size);
+    } else
+        res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                                1, GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
     if (res) return res;
 
     /*
@@ -963,7 +1023,7 @@ next_resize:
         }
 
         FDT( make_timer_node(gc, fdt, ainfo, state->clock_frequency) );
-        FDT( make_hypervisor_node(gc, fdt, vers) );
+        FDT( make_hypervisor_node(gc, fdt, vers, info, dom) );
 
         if (info->arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART)
             FDT( make_vpl011_uart_node(gc, fdt, ainfo, dom) );
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 6b5a5f8..df59933 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -449,6 +449,9 @@ typedef uint64_t xen_callback_t;
 #define GUEST_RAM_BANK_BASES   { GUEST_RAM0_BASE, GUEST_RAM1_BASE }
 #define GUEST_RAM_BANK_SIZES   { GUEST_RAM0_SIZE, GUEST_RAM1_SIZE }
 
+#define GUEST_EXT_REGION_MAX_SIZE   xen_mk_ullong(0x2000000000) /* 128GB */
+#define GUEST_EXT_REGION_MIN_SIZE   xen_mk_ullong(0x0004000000) /* 64MB */
+
 /* Current supported guest VCPUs */
 #define GUEST_MAX_VCPUS 128
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 22:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 22:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199327.353307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRJ-0005eb-VI; Wed, 29 Sep 2021 22:52:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199327.353307; Wed, 29 Sep 2021 22:52:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRJ-0005eU-SP; Wed, 29 Sep 2021 22:52:37 +0000
Received: by outflank-mailman (input) for mailman id 199327;
 Wed, 29 Sep 2021 22:52:36 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XAMn=OT=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mViRI-0005eO-DP
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 22:52:36 +0000
Received: from mail-lf1-x130.google.com (unknown [2a00:1450:4864:20::130])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f45b94f-b844-4215-9cc7-eaf060905e3e;
 Wed, 29 Sep 2021 22:52:34 +0000 (UTC)
Received: by mail-lf1-x130.google.com with SMTP id z24so17049919lfu.13
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 15:52:34 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id g26sm133178lja.16.2021.09.29.15.52.32
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 29 Sep 2021 15:52:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f45b94f-b844-4215-9cc7-eaf060905e3e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id;
        bh=pXpW4hIKiLN3Bm2D+gLtF28MQzPdlZAEENqKnqgOkgw=;
        b=E6TbyQIW8pbfhjdtmu4cl3auKbaggZh/cL+3dOe4AkkFDdvaDzme25J1viTF0PGy0m
         oxJUFbJT5bKOecH0br1O0IS7OWL2JqMTBjOHNf2NxpwdNL/QuBrvql7Q089jMMaiRIKC
         UHeSda/qKec5ppPtLILPwPpYXn5tVV/pfz/JKHHiwcuArucLw0yxwAgWQwQCK7D3eSaD
         WzyrCaEM3sXK5gCWHrRm/ZRYy5mX9MjPzw9mUUbb7SnxBEOOmRpbiyTgZ94WrwMtpUHq
         eMWSzj0/Oq2hQqt5OuWZhBZ8PXbsWMV4Un4xn1LQNSk440/gW4lRtdjwugdaXAtlWNk2
         ygsQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id;
        bh=pXpW4hIKiLN3Bm2D+gLtF28MQzPdlZAEENqKnqgOkgw=;
        b=yb2jfej/XEoh8qox2nwKh4vqKbGm+FWLZkghvxZyqZeA+jAcOI4HQ9Q1QL7Xo9f6f1
         NK1MaPf5eZSsw/meNvH2ydUyA50MtqC4q3liLSVwou1Y8ejZcY8FlZuihJqYqijog/iv
         L7+Xvjv1YwUPDMCGYR9QZGsg4XFxUUEfpFaac/wra6Yn4Zeaz1O7rGgRGNlV5ZsseGxs
         RICbx/AySU2zL8KVrY3g/bUTyeFFJSm3RCoNtR8DceZzQV9hdjlAUGd7ZcvEfEI8yulv
         KDfBvG+ZwlAx6FPny9SCkGkxt/PafRir751eG7jJkTNj9Rq+XMHz+iSXnhboEIIF+RoZ
         c9Xg==
X-Gm-Message-State: AOAM532j5oP3JHcHA4JlTw3LDgbZbjjJqowkgLnJ9ovvO/SqPVsfulC2
	TRfO+ZjdzhC4KHM2Op4GKo1KTcTmLM2BaQ==
X-Google-Smtp-Source: ABdhPJxlvxenimr81Aw9teWgnwEQY7+GVOVZLk95wY6b0sb6ENv9VL9xNUucOGDuerfW8TltNju67Q==
X-Received: by 2002:a05:6512:3087:: with SMTP id z7mr2279728lfd.556.1632955953448;
        Wed, 29 Sep 2021 15:52:33 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Henry Wang <Henry.Wang@arm.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Wei Chen <Wei.Chen@arm.com>
Subject: [PATCH V4 0/3] Add handling of extended regions (safe ranges) on Arm (Was "xen/memory: Introduce a hypercall to provide unallocated space")
Date: Thu, 30 Sep 2021 01:52:04 +0300
Message-Id: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

You can find an initial discussion at [1]-[4].

The extended region (safe range) is a region of guest physical address space
which is unused and could be safely used to create grant/foreign mappings instead
of wasting real RAM pages from the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and advertised
to it via "reg" property under hypervisor node in the guest device-tree
(the indexes for extended regions are 1...N).
No device tree bindings update is needed, guest infers the presense of extended
regions from the number of regions in "reg" property.
New compatible/property will be needed (but only after this patch [5] or alternative
goes in) to indicate that "region 0 is safe to use". Until this patch is merged it is
not safe to use extended regions for the grant table space.

The extended regions are calculated differently for direct mapped Dom0 (with and without
IOMMU) and non-direct mapped DomUs.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain currently.
- The ACPI case is not covered.

Xen patch series is also available at [6]. The corresponding Linux patch series is at [7]
for now (last 4 patches).

Tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with updated virtio-disk backend [8]
running in Dom0 (256MB RAM) and DomD (2GB RAM). In both cases the backend pre-maps DomU memory
which is 3GB. All foreign memory gets mapped into extended regions (so the amount of RAM in
the backend domain is not reduced). No issues were observed.

[1] https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@gmail.com/
[2] https://lore.kernel.org/xen-devel/1631034578-12598-1-git-send-email-olekstysh@gmail.com/
[3] https://lore.kernel.org/xen-devel/1631297924-8658-1-git-send-email-olekstysh@gmail.com/
[4] https://lore.kernel.org/xen-devel/1632437334-12015-1-git-send-email-olekstysh@gmail.com/
[5] https://lore.kernel.org/xen-devel/1632425551-18910-1-git-send-email-olekstysh@gmail.com/
[6] https://github.com/otyshchenko1/xen/commits/map_opt_ml5
[7] https://github.com/otyshchenko1/linux/commits/map_opt_ml4
[8] https://github.com/otyshchenko1/virtio-disk/commits/map_opt_next

Oleksandr Tyshchenko (3):
  xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo
  xen/arm: Add handling of extended regions for Dom0
  libxl/arm: Add handling of extended regions for DomU

 docs/misc/xen-command-line.pandoc |  11 ++
 tools/include/libxl.h             |   7 +
 tools/libs/light/libxl.c          |   2 +
 tools/libs/light/libxl_arm.c      |  70 +++++++++-
 tools/libs/light/libxl_types.idl  |   2 +
 tools/xl/xl_info.c                |   2 +
 xen/arch/arm/domain_build.c       | 286 +++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/sysctl.c             |   2 +
 xen/arch/x86/sysctl.c             |   2 +
 xen/include/public/arch-arm.h     |   3 +
 xen/include/public/sysctl.h       |   4 +-
 11 files changed, 382 insertions(+), 9 deletions(-)

-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Wed Sep 29 22:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 29 Sep 2021 22:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199329.353330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRT-0006E6-Fd; Wed, 29 Sep 2021 22:52:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199329.353330; Wed, 29 Sep 2021 22:52:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mViRT-0006Du-CZ; Wed, 29 Sep 2021 22:52:47 +0000
Received: by outflank-mailman (input) for mailman id 199329;
 Wed, 29 Sep 2021 22:52:46 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XAMn=OT=gmail.com=olekstysh@srs-us1.protection.inumbo.net>)
 id 1mViRS-0005eO-8V
 for xen-devel@lists.xenproject.org; Wed, 29 Sep 2021 22:52:46 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ae92a60d-2dcd-47cc-a077-514b6a7ec018;
 Wed, 29 Sep 2021 22:52:36 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id e15so17151185lfr.10
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 15:52:36 -0700 (PDT)
Received: from otyshchenko.router ([212.22.223.21])
 by smtp.gmail.com with ESMTPSA id g26sm133178lja.16.2021.09.29.15.52.34
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 29 Sep 2021 15:52:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae92a60d-2dcd-47cc-a077-514b6a7ec018
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references;
        bh=2up+jpzvkiUGC0ZnAjQlIu5gOHMwSu/FSUWJVQDo7sg=;
        b=NaYnQ2utvVo47PVBmbrhDtMAasyXppuMpH4+xXi2Uc4sNoJCxJn9ztYmX1D6edSptW
         VS35YnR0lcDzHKU3dPHumY3TKszcEh1pwkzhiVnDhVQexejVStqGaoD7+50vrI+s/D1l
         Pw5rHt4eHduwVJNctlxjxpC8Zjw/wPAnI6ogkC4XgWg2pZPajeihCgizDOdYBwRN/Jsb
         dylbnmpSjXm8cASEKscw5snndzKjfDX1YjzhmPSqbStFoMIBLoibUfi5/XXYhzu7XXh1
         BkE4VZqaa9XnqlUHkeEghO/7DYzUxRgSdLZb5nGb9HdCMRDcG8d6SZ0BICiIjsQGypBJ
         Tg3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references;
        bh=2up+jpzvkiUGC0ZnAjQlIu5gOHMwSu/FSUWJVQDo7sg=;
        b=4FevMsXYcCBQXQiN5JyIhl1xwhvMGPAO5QEOY7Ux9uDdN2yKeD8yaJSJYN/r3av4OW
         wbesFCwbrsyVs0EqlYszMLdVGmTuLRnmAITLvOsGcbqptvHkvZlpBS+B6Cyurn4KgFW4
         xgBKYr5c/rGYoML9f0+K2QuBwbUVeFYpBU1CH131rEYbh0/wBjRoSpHOTd7NOt9znHnE
         IeU+HlDX6SfH1LKBYO7vhpM2wVXyzst0RXU2IAy0zCiWOLj3/wR6zofCj9SzgHS2JvTu
         uzSlHVKYyyvktbD+xQsL+NNeqJnj6c0fs56II7suA16OcdGUiZlYYQlHfekRVbFf/Ryk
         zaSg==
X-Gm-Message-State: AOAM533p+Vy9JfyHARiTqX1X5HdJA1VElbSx0WoixJXfdv930GPcYDY2
	YGbD57yKUOIoYXcfc3xlzWRn308aYDeo+Q==
X-Google-Smtp-Source: ABdhPJxZf5d54evx4goBRAO/op87y8XxoFBaz3jFucB1o1inkxXir4YOVSyaT/CgGrCzpXo3gaQLkA==
X-Received: by 2002:a05:6512:3341:: with SMTP id y1mr2511831lfd.424.1632955955505;
        Wed, 29 Sep 2021 15:52:35 -0700 (PDT)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH V4 2/3] xen/arm: Add handling of extended regions for Dom0
Date: Thu, 30 Sep 2021 01:52:06 +0300
Message-Id: <1632955927-27911-3-git-send-email-olekstysh@gmail.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen at the domain creation time and
advertised to it via "reg" property under hypervisor node in
the guest device-tree. As region 0 is reserved for grant table
space (always present), the indexes for extended regions are 1...N.
If extended regions could not be allocated for some reason,
Xen doesn't fail and behaves as usual, so only inserts region 0.

Please note the following limitations:
- The extended region feature is only supported for 64-bit domain
  currently.
- The ACPI case is not covered.

***

As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
the algorithm to choose extended regions for it is different
in comparison with the algorithm for non-direct mapped DomU.
What is more, that extended regions should be chosen differently
whether IOMMU is enabled or not.

Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
holes found in host device-tree if otherwise. Make sure that
extended regions are 2MB-aligned and located within maximum possible
addressable physical memory range. The minimum size of extended
region is 64MB. The maximum number of extended regions is 128,
which is an artificial limit to minimize code changes (we reuse
struct meminfo to describe extended regions, so there are an array
field for 128 elements).

It worth mentioning that unallocated memory solution (when the IOMMU
is disabled) will work safely until Dom0 is able to allocate memory
outside of the original range.

Also introduce command line option to be able to globally enable or
disable support for extended regions for Dom0 (enabled by default).

Suggested-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, we need to decide which approach to use in find_unallocated_memory(),
you can find details at:
https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/

Changes RFC -> V2:
   - update patch description
   - drop uneeded "extended-region" DT property

Changes V2 -> V3:
   - update patch description
   - add comment for "size" calculation in add_ext_regions()
   - clarify "end" calculation in find_unallocated_memory() and
     find_memory_holes()
   - only pick up regions with size >= 64MB
   - allocate reg dynamically instead of keeping on the stack in
     make_hypervisor_node()
   - do not show warning for 32-bit domain
   - drop Linux specific limits EXT_REGION_*
   - also cover "ranges" property in find_memory_holes()
   - add command line arg to enable/disable extended region support

Changes V3 -> V4:
  - update opt_ext_regions purpose and comment in code
  - reorganize make_hypervisor_node() to move allocations after initial
    checks, allocate only required amount of elements instead of maximum
    possible
---
 docs/misc/xen-command-line.pandoc |  11 ++
 xen/arch/arm/domain_build.c       | 286 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 294 insertions(+), 3 deletions(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 177e656..5cae4ad 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1081,6 +1081,17 @@ hardware domain is architecture dependent.
 Note that specifying zero as domU value means zero, while for dom0 it means
 to use the default.
 
+### ext_regions (Arm)
+> `= <boolean>`
+
+> Default : `true`
+
+Flag to enable or disable support for extended regions for Dom0.
+
+Extended regions are ranges of unused address space exposed to Dom0 as
+"safe to use" for special memory mappings. Disable if your board device
+tree is incomplete.
+
 ### flask
 > `= permissive | enforcing | late | disabled`
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d233d63..c5afbe2 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -34,6 +34,10 @@
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
 
+/* If true, the extended regions support is enabled for dom0 */
+static bool __initdata opt_ext_regions = true;
+boolean_param("ext_regions", opt_ext_regions);
+
 static u64 __initdata dom0_mem;
 static bool __initdata dom0_mem_set;
 
@@ -886,6 +890,232 @@ static int __init make_memory_node(const struct domain *d,
     return res;
 }
 
+static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
+{
+    struct meminfo *ext_regions = data;
+    paddr_t start, size;
+
+    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
+        return 0;
+
+    /* Both start and size of the extended region should be 2MB aligned */
+    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
+    if ( start > e )
+        return 0;
+
+    /*
+     * e is actually "end-1" because it is called by rangeset functions
+     * which are inclusive of the last address.
+     */
+    e += 1;
+    size = (e - start) & ~(SZ_2M - 1);
+    if ( size < MB(64) )
+        return 0;
+
+    ext_regions->bank[ext_regions->nr_banks].start = start;
+    ext_regions->bank[ext_regions->nr_banks].size = size;
+    ext_regions->nr_banks++;
+
+    return 0;
+}
+
+static int __init find_unallocated_memory(const struct kernel_info *kinfo,
+                                          struct meminfo *ext_regions)
+{
+    const struct meminfo *assign_mem = &kinfo->mem;
+    struct rangeset *unalloc_mem;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find unallocated memory for extended regions\n");
+
+    unalloc_mem = rangeset_new(NULL, NULL, 0);
+    if ( !unalloc_mem )
+        return -ENOMEM;
+
+    /* Start with all available RAM */
+    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
+    {
+        start = bootinfo.mem.bank[i].start;
+        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
+        res = rangeset_add_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove RAM assigned to Dom0 */
+    for ( i = 0; i < assign_mem->nr_banks; i++ )
+    {
+        start = assign_mem->bank[i].start;
+        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
+        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove reserved-memory regions */
+    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
+    {
+        start = bootinfo.reserved_mem.bank[i].start;
+        end = bootinfo.reserved_mem.bank[i].start +
+            bootinfo.reserved_mem.bank[i].size;
+        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                   start, end);
+            goto out;
+        }
+    }
+
+    /* Remove grant table region */
+    start = kinfo->gnttab_start;
+    end = kinfo->gnttab_start + kinfo->gnttab_size;
+    res = rangeset_remove_range(unalloc_mem, start, end - 1);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_report_ranges(unalloc_mem, start, end,
+                                 add_ext_regions, ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(unalloc_mem);
+
+    return res;
+}
+
+static int __init find_memory_holes(const struct kernel_info *kinfo,
+                                    struct meminfo *ext_regions)
+{
+    struct dt_device_node *np;
+    struct rangeset *mem_holes;
+    paddr_t start, end;
+    unsigned int i;
+    int res;
+
+    dt_dprintk("Find memory holes for extended regions\n");
+
+    mem_holes = rangeset_new(NULL, NULL, 0);
+    if ( !mem_holes )
+        return -ENOMEM;
+
+    /* Start with maximum possible addressable physical memory range */
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_add_range(mem_holes, start, end);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
+               start, end);
+        goto out;
+    }
+
+    /*
+     * Remove regions described by "reg" and "ranges" properties where
+     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
+     */
+    dt_for_each_device_node( dt_host, np )
+    {
+        unsigned int naddr;
+        u64 addr, size;
+
+        naddr = dt_number_of_address(np);
+
+        for ( i = 0; i < naddr; i++ )
+        {
+            res = dt_device_get_address(np, i, &addr, &size);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+                       i, dt_node_full_name(np));
+                goto out;
+            }
+
+            start = addr & PAGE_MASK;
+            end = PAGE_ALIGN(addr + size);
+            res = rangeset_remove_range(mem_holes, start, end - 1);
+            if ( res )
+            {
+                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                       start, end);
+                goto out;
+            }
+        }
+
+        if ( dt_device_type_is_equal(np, "pci" ) )
+        {
+            unsigned int range_size, nr_ranges;
+            int na, ns, pna;
+            const __be32 *ranges;
+            u32 len;
+
+            /*
+             * Looking for non-empty ranges property which in this context
+             * describes the PCI host bridge aperture.
+             */
+            ranges = dt_get_property(np, "ranges", &len);
+            if ( !ranges || !len )
+                continue;
+
+            pna = dt_n_addr_cells(np);
+            na = dt_child_n_addr_cells(np);
+            ns = dt_child_n_size_cells(np);
+            range_size = pna + na + ns;
+            nr_ranges = len / sizeof(__be32) / range_size;
+
+            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
+            {
+                /* Skip the child address and get the parent (CPU) address */
+                addr = dt_read_number(ranges + na, pna);
+                size = dt_read_number(ranges + na + pna, ns);
+
+                start = addr & PAGE_MASK;
+                end = PAGE_ALIGN(addr + size);
+                res = rangeset_remove_range(mem_holes, start, end - 1);
+                if ( res )
+                {
+                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
+                           start, end);
+                    goto out;
+                }
+            }
+        }
+    }
+
+    start = 0;
+    end = (1ULL << p2m_ipa_bits) - 1;
+    res = rangeset_report_ranges(mem_holes, start, end,
+                                 add_ext_regions,  ext_regions);
+    if ( res )
+        ext_regions->nr_banks = 0;
+    else if ( !ext_regions->nr_banks )
+        res = -ENOENT;
+
+out:
+    rangeset_destroy(mem_holes);
+
+    return res;
+}
+
 static int __init make_hypervisor_node(struct domain *d,
                                        const struct kernel_info *kinfo,
                                        int addrcells, int sizecells)
@@ -893,11 +1123,12 @@ static int __init make_hypervisor_node(struct domain *d,
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
         "xen,xen";
-    __be32 reg[4];
+    __be32 *reg, *cells;
     gic_interrupt_t intr;
-    __be32 *cells;
     int res;
     void *fdt = kinfo->fdt;
+    struct meminfo *ext_regions = NULL;
+    unsigned int i, nr_ext_regions;
 
     dt_dprintk("Create hypervisor node\n");
 
@@ -919,12 +1150,61 @@ static int __init make_hypervisor_node(struct domain *d,
     if ( res )
         return res;
 
+    if ( !opt_ext_regions )
+    {
+        printk(XENLOG_DEBUG "The extended regions support is disabled\n");
+        nr_ext_regions = 0;
+    }
+    else if ( is_32bit_domain(d) )
+    {
+        printk(XENLOG_DEBUG "The extended regions are only supported for 64-bit guest currently\n");
+        nr_ext_regions = 0;
+    }
+    else
+    {
+        ext_regions = xzalloc(struct meminfo);
+        if ( !ext_regions )
+            return -ENOMEM;
+
+        if ( !is_iommu_enabled(d) )
+            res = find_unallocated_memory(kinfo, ext_regions);
+        else
+            res = find_memory_holes(kinfo, ext_regions);
+
+        if ( res )
+            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
+        nr_ext_regions = ext_regions->nr_banks;
+    }
+
+    reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + sizecells));
+    if ( !reg )
+    {
+        xfree(ext_regions);
+        return -ENOMEM;
+    }
+
     /* reg 0 is grant table space */
     cells = &reg[0];
     dt_child_set_range(&cells, addrcells, sizecells,
                        kinfo->gnttab_start, kinfo->gnttab_size);
+    /* reg 1...N are extended regions */
+    for ( i = 0; i < nr_ext_regions; i++ )
+    {
+        u64 start = ext_regions->bank[i].start;
+        u64 size = ext_regions->bank[i].size;
+
+        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
+                   i, start, start + size);
+
+        dt_child_set_range(&cells, addrcells, sizecells, start, size);
+    }
+
     res = fdt_property(fdt, "reg", reg,
-                       dt_cells_to_size(addrcells + sizecells));
+                       dt_cells_to_size(addrcells + sizecells) *
+                       (nr_ext_regions + 1));
+    xfree(ext_regions);
+    xfree(reg);
+
     if ( res )
         return res;
 
-- 
2.7.4



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 00:00:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 00:00:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199355.353352 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVjUW-0005vd-Rq; Thu, 30 Sep 2021 00:00:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199355.353352; Thu, 30 Sep 2021 00:00:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVjUW-0005vW-Nr; Thu, 30 Sep 2021 00:00:00 +0000
Received: by outflank-mailman (input) for mailman id 199355;
 Wed, 29 Sep 2021 23:59:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVjUU-0005vF-Ox; Wed, 29 Sep 2021 23:59:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVjUU-0003dV-I5; Wed, 29 Sep 2021 23:59:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVjUU-0005PW-5m; Wed, 29 Sep 2021 23:59:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVjUU-0005P1-5G; Wed, 29 Sep 2021 23:59:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iwxLUdfieLDS7vzX6Z+GCqoTdmu+AFPGPDzfpD/HkJE=; b=bYa07OsIrs3q2S44M0hGAKTUmQ
	V5XY94AWh08m9EKZhfw7cCUC70iNLEDV2TuaPIaEJhxi55J4Px0wpYOyGif9XVdztryQL3m4NfAiE
	e4kMcsoL/OnI63dkwAbclpffGTFm7VaRs12B68zmSaQAVStAC1yNQRfwRw38EckIleZc=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165317-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 165317: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ab4a83023eda9f04ad864877c1956b087ec6fc4f
X-Osstest-Versions-That:
    xen=890ceb9453171c85e881103e65dbb5cdcf81659e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 29 Sep 2021 23:59:58 +0000

flight 165317 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165317/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ab4a83023eda9f04ad864877c1956b087ec6fc4f
baseline version:
 xen                  890ceb9453171c85e881103e65dbb5cdcf81659e

Last test of basis   165233  2021-09-28 12:00:26 Z    1 days
Failing since        165243  2021-09-28 17:00:26 Z    1 days    2 attempts
Testing same since   165317  2021-09-29 20:22:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Igor Druzhinin <igor.druzhinin@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   890ceb9453..ab4a83023e  ab4a83023eda9f04ad864877c1956b087ec6fc4f -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 04:17:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 04:17:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199364.353369 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVnVo-0000V5-MG; Thu, 30 Sep 2021 04:17:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199364.353369; Thu, 30 Sep 2021 04:17:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVnVo-0000Uy-HZ; Thu, 30 Sep 2021 04:17:36 +0000
Received: by outflank-mailman (input) for mailman id 199364;
 Thu, 30 Sep 2021 04:17:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9gkK=OU=gmail.com=ysaikiran1997@srs-us1.protection.inumbo.net>)
 id 1mVnVn-0000Us-Pq
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 04:17:35 +0000
Received: from mail-yb1-xb2f.google.com (unknown [2607:f8b0:4864:20::b2f])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 72d1f738-06bb-40e6-b7ee-dba019ffd8d8;
 Thu, 30 Sep 2021 04:17:34 +0000 (UTC)
Received: by mail-yb1-xb2f.google.com with SMTP id z5so10268765ybj.2
 for <xen-devel@lists.xenproject.org>; Wed, 29 Sep 2021 21:17:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72d1f738-06bb-40e6-b7ee-dba019ffd8d8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=o+zO69n1HhJGeraWaQe/BVUQnuHDbkPftPnw1VZX1PM=;
        b=fYxdCLSWVrvhT+KbMsxuT084xUJrK+di03FXzjvlg+xyF57tXiXmCR0OHovFCTNY/e
         9yOri9Q2LU+j0dSxdvsa8uv91n+jbIv0Po4LKlKqP8cRBZ/Z93QlLUb2603NeDhZj75r
         CwcU8gt/HdeTEF4FxHE1Ci7fzB1LbzqNnVSvwYpRUYWZvwPXwLrKZe2MgWkRjr5kEdQb
         xlPpxDVsoZaBFD4nJLD8LsinmirBweH5gLjhr/8uKm4NEkJci9ujpCYVqei7al0UFh9N
         LUzOqh78zrfwWA+hFerCOLsPVvdO5e5dqPQXxoszKkIyw6OIvibIQXkZdQApg3ZiwFFL
         bXIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=o+zO69n1HhJGeraWaQe/BVUQnuHDbkPftPnw1VZX1PM=;
        b=ZMgmzCQklgwZCcAFiS+aF/14KQ3j/GtYJtSEaBWxbDi6JQapNS3hN/4RiU5Dr85xtw
         mr+UawyAUeQCU7S7MPDXgdGanQaCQBUDeltkxfHVcMx+aCtXKL56MtO3OmB+5bZUGqCH
         ESRka7KgzZQIAy4Fqo8l0g6liLsA8pzyDgdy/SLxXoZ8DR4XZ3O0NEt2EmX5Xc0S64AV
         nKFf9XFcR6gri+GF4F1xD7j04yd5WRYVgrp7flVDuWz8fHfRBb6zP2aNpYVD6JzC/xFJ
         O5xEDieRnEMBo60Sc7PSh3MihQPoos6pnmrzZLMtbx9kVnCmulC4O5f7OdcXO/KnMHlW
         qMHw==
X-Gm-Message-State: AOAM533AmzJPUSwEzs0WlVQMkz2XqYJ7G6PgCFGg6NIOcch/ZZfKsSaD
	eMMXJnDSYJ2TC2QqaMJ+BoiurDVikwPMfPLOb0E=
X-Google-Smtp-Source: ABdhPJwPYswQekdO7vENLu+Xeo7PR6ILqsOzLfhbBqtLKIdjTT4utPzdoBr+/yQ360L1zXTWEJ5kKiKbZIvRdm8SJhc=
X-Received: by 2002:a25:ce51:: with SMTP id x78mr4457151ybe.138.1632975453694;
 Wed, 29 Sep 2021 21:17:33 -0700 (PDT)
MIME-Version: 1.0
References: <33C29853-D896-4E4E-91D3-4D2FA89A9B91@hxcore.ol>
 <eb5fd47b-6bc9-2eec-7f46-9ab9a42c9f1f@xen.org> <CAEsO4uzsCnsTtTmYHAT4NN6=girCY2NHHdAHbH6GG33D7jwe_A@mail.gmail.com>
 <alpine.DEB.2.21.2109131437030.10523@sstabellini-ThinkPad-T480s>
 <CAEsO4uwpiDO2QoqLRTOxpsYM9YSPAsbX0P=gagdEy21pk1VsPQ@mail.gmail.com> <alpine.DEB.2.21.2109141825330.21985@sstabellini-ThinkPad-T480s>
In-Reply-To: <alpine.DEB.2.21.2109141825330.21985@sstabellini-ThinkPad-T480s>
From: Sai Kiran Kumar Reddy Y <ysaikiran1997@gmail.com>
Date: Thu, 30 Sep 2021 09:47:22 +0530
Message-ID: <CAEsO4uyjNs97Cx8n2owDk4kKTVCycji377pm7aXHcPWzeszUTw@mail.gmail.com>
Subject: Re: Xen Booting Problem on ARM Machine
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
	Jan Beulich <jbeulich@suse.com>, Luca Fancellu <luca.fancellu@arm.com>, 
	xen-devel@lists.xenproject.org, kannan@cimware.in
Content-Type: multipart/alternative; boundary="000000000000d82f9a05cd2eb9cc"

--000000000000d82f9a05cd2eb9cc
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

Sorry about the delay. We have been trying to access the serial of the
machine. Tried with couple of JTAG connectors. There's still no debug
messages on the serial. Is there any other way of figuring this out?

On Wed, Sep 15, 2021, 7:02 AM Stefano Stabellini <sstabellini@kernel.org>
wrote:

> Something is off. When you enabled earlyprintk in Xen, you should see
> something like this at boot time:
> https://marc.info/?l=3Dxen-devel&m=3D158829968025334
>
> All the Xen logs starting with "(XEN)" on the serial. Do you have access
> to the serial of the machine? Without it, it is going to be hard to
> debug.
>
>
> On Tue, 14 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
> > In the folder "/var/log", there's a file called "xen", which is empty.
> As far as the boot logs are concerned, I don't see any debug
> > messages related to xen. I am attaching the log files, "kern.txt" and
> "boot.txt"
> >
> > On Tue, Sep 14, 2021 at 3:08 AM Stefano Stabellini <
> sstabellini@kernel.org> wrote:
> >       On Mon, 13 Sep 2021, Sai Kiran Kumar Reddy Y wrote:
> >       > On Fri, Sep 10, 2021 at 7:30 PM Julien Grall <julien@xen.org>
> wrote:
> >       >
> >       >
> >       >       On 08/09/2021 11:43, Sai Kiran wrote:
> >       >       > Hello,
> >       >
> >       >       Hi,
> >       >
> >       >       Thank you for the report. Moving the discussion back to
> xen-devel
> >       >       because this looks like a potential issue in the UEFI stu=
b
> in Xen.
> >       >
> >       >       > I have Xen-4.15.0 on an ARM Machine, with Debian 11
> installed on it.
> >       >
> >       >       Would you be able to give more details on the Arm machine
> you are using?
> >       >       Also, are you using ACPI or DT to boot?
> >       >
> >       >
> >       >    Sai >> DT . ACPI configuration is disabled in Boot settings
>
> >       >
> >       >       >  I
> >       >       > am able to do =E2=80=9Cmake world=E2=80=9D and =E2=80=
=9Cmake install=E2=80=9D, after
> =E2=80=9C./configure=E2=80=9D, as
> >       >       > specified in README file. When I reboot the system, I
> get the following
> >       >       > message:
> >       >       >
> >       >       > Warning: All 128 bootinfo mem banks exhausted.
> >       >       >
> >       >       > Warning: All 128 bootinfo mem banks exhausted.
> >       >
> >       >       Hmmm... This means that you have more than 128 memory
> regions described
> >       >       in the EFI memory map. That's quite a lot.
> >       >
> >       >       Although, this should be harmless as it means Xen will no=
t
> use the extra
> >       >       memory banks.
> >       >
> >       >       >
> >       >       > Cannot exit boot services: ErrCode: 0x8000000000000002
> >       >
> >       >       This means EFI_INVALID_PARAMETER. We have code to retry
> because AFAICT
> >       >       ExitBootServices() may sometime fails (I have CCed Jan ma=
y
> have more
> >       >       idea what's happening).
> >       >
> >       >       Would you be able to provide more details on the UEFI
> firmware you are
> >       >       using? Is it EDK2 or U-boot?
> >       >
> >       > Sai >>  EDK2
> >       >       Also, do you know if Linux is boot on the same system?
> >       >
> >       > Sai >> Yes
> >       >       However, AFAICT, the error message would not prevent Xen
> to continue
> >       >       booting. So you may get stuck later in the boot process.
> >       >
> >       >       My suggestion would be to enable earlyprintk for your
> platform. You can
> >       >       setup it up from the menuconfig in "Debugging Options".
> >       >
> >       > Sai >> Yes, I have enabled earlyprintk.
> >       > I tried changing NR_MEM_BANKS(in xen/include/asm-arm/setup.h)
> value to 256, from 128. The error message is no longer seen,
> >       but the device
> >       > is stuck in the boot process.
> >
> >       Could you please post the boot logs now that you enabled
> earlyprintk?
> >       Ideally not a camera picture but a textual copy/paste from the
> target
> >       serial?
> >
> >       Earlyprintk is pretty verbose, we should be able to figure out
> where it
> >       gets stuck.
> >
> >
> >

--000000000000d82f9a05cd2eb9cc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Hi,<div dir=3D"auto"><br></div><div dir=3D"auto">Sorry ab=
out the delay. We have been trying to access the serial of the machine. Tri=
ed with couple of JTAG connectors. There&#39;s still no debug messages on t=
he serial. Is there any other way of figuring this out?</div></div><br><div=
 class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Sep 15=
, 2021, 7:02 AM Stefano Stabellini &lt;<a href=3D"mailto:sstabellini@kernel=
.org">sstabellini@kernel.org</a>&gt; wrote:<br></div><blockquote class=3D"g=
mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l=
eft:1ex">Something is off. When you enabled earlyprintk in Xen, you should =
see<br>
something like this at boot time:<br>
<a href=3D"https://marc.info/?l=3Dxen-devel&amp;m=3D158829968025334" rel=3D=
"noreferrer noreferrer" target=3D"_blank">https://marc.info/?l=3Dxen-devel&=
amp;m=3D158829968025334</a><br>
<br>
All the Xen logs starting with &quot;(XEN)&quot; on the serial. Do you have=
 access<br>
to the serial of the machine? Without it, it is going to be hard to<br>
debug.<br>
<br>
<br>
On Tue, 14 Sep 2021, Sai Kiran Kumar Reddy Y wrote:<br>
&gt; In the folder &quot;/var/log&quot;, there&#39;s a file called &quot;xe=
n&quot;, which is empty. As far as the boot logs are=C2=A0concerned, I don&=
#39;t see any debug<br>
&gt; messages related to xen. I am attaching the log files, &quot;kern.txt&=
quot; and &quot;boot.txt&quot;<br>
&gt; <br>
&gt; On Tue, Sep 14, 2021 at 3:08 AM Stefano Stabellini &lt;<a href=3D"mail=
to:sstabellini@kernel.org" target=3D"_blank" rel=3D"noreferrer">sstabellini=
@kernel.org</a>&gt; wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0On Mon, 13 Sep 2021, Sai Kiran Kumar Reddy Y=
 wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; On Fri, Sep 10, 2021 at 7:30 PM Julien =
Grall &lt;<a href=3D"mailto:julien@xen.org" target=3D"_blank" rel=3D"norefe=
rrer">julien@xen.org</a>&gt; wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0On 08/09/2021=
 11:43, Sai Kiran wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Hello,<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Hi,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Thank you for=
 the report. Moving the discussion back to xen-devel<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0because this =
looks like a potential issue in the UEFI stub in Xen.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; I have X=
en-4.15.0 on an ARM Machine, with Debian 11 installed on it.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Would you be =
able to give more details on the Arm machine you are using?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Also, are you=
 using ACPI or DT to boot?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; =C2=A0 =C2=A0Sai &gt;&gt; DT . ACPI con=
figuration is disabled in Boot settings=C2=A0 =C2=A0=C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 I<=
br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; am able =
to do =E2=80=9Cmake world=E2=80=9D and =E2=80=9Cmake install=E2=80=9D, afte=
r =E2=80=9C./configure=E2=80=9D, as<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; specifie=
d in README file. When I reboot the system, I get the following<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; message:=
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Warning:=
 All 128 bootinfo mem banks exhausted.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Warning:=
 All 128 bootinfo mem banks exhausted.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Hmmm... This =
means that you have more than 128 memory regions described<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0in the EFI me=
mory map. That&#39;s quite a lot.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Although, thi=
s should be harmless as it means Xen will not use the extra<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0memory banks.=
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Cannot e=
xit boot services: ErrCode: 0x8000000000000002<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0This means EF=
I_INVALID_PARAMETER. We have code to retry because AFAICT<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0ExitBootServi=
ces() may sometime fails (I have CCed Jan may have more<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0idea what&#39=
;s happening).<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Would you be =
able to provide more details on the UEFI firmware you are<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0using? Is it =
EDK2 or U-boot?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Sai &gt;&gt;=C2=A0 EDK2=C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Also, do you =
know if Linux is boot on the same system?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Sai &gt;&gt; Yes=C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0However, AFAI=
CT, the error message would not prevent Xen to continue<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0booting. So y=
ou may get stuck later in the boot process.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0My suggestion=
 would be to enable earlyprintk for your platform. You can<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0setup it up f=
rom the menuconfig in &quot;Debugging Options&quot;.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; Sai &gt;&gt; Yes, I have enabled earlyp=
rintk.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; I tried changing NR_MEM_BANKS(in xen/in=
clude/asm-arm/setup.h) value to 256, from 128. The error message is no long=
er seen,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0but the device<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0&gt; is stuck in the boot process.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Could you please post the boot logs now that=
 you enabled earlyprintk?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Ideally not a camera picture but a textual c=
opy/paste from the target<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0serial?<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Earlyprintk is pretty verbose, we should be =
able to figure out where it<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0gets stuck.<br>
&gt; <br>
&gt; <br>
&gt; </blockquote></div>

--000000000000d82f9a05cd2eb9cc--


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 05:48:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 05:48:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199372.353380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVovl-0001uB-Bw; Thu, 30 Sep 2021 05:48:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199372.353380; Thu, 30 Sep 2021 05:48:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVovl-0001u4-8m; Thu, 30 Sep 2021 05:48:29 +0000
Received: by outflank-mailman (input) for mailman id 199372;
 Thu, 30 Sep 2021 05:48:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tdhe=OU=lst.de=hch@srs-us1.protection.inumbo.net>)
 id 1mVovk-0001ty-2i
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 05:48:28 +0000
Received: from verein.lst.de (unknown [213.95.11.211])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 19f006ae-ef9a-487a-97f0-2b1379dc5eff;
 Thu, 30 Sep 2021 05:48:26 +0000 (UTC)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 272C868AFE; Thu, 30 Sep 2021 07:48:22 +0200 (CEST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19f006ae-ef9a-487a-97f0-2b1379dc5eff
Date: Thu, 30 Sep 2021 07:48:21 +0200
From: Christoph Hellwig <hch@lst.de>
To: Tianyu Lan <ltykernel@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>,
	"iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	vkuznets <vkuznets@redhat.com>,
	"parri.andrea@gmail.com" <parri.andrea@gmail.com>,
	"dave.hansen@intel.com" <dave.hansen@intel.com>,
	Michael Kelley <mikelley@microsoft.com>,
	KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	"wei.liu@kernel.org" <wei.liu@kernel.org>,
	Dexuan Cui <decui@microsoft.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>,
	"boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
	"jgross@suse.com" <jgross@suse.com>,
	"sstabellini@kernel.org" <sstabellini@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"will@kernel.org" <will@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"jejb@linux.ibm.com" <jejb@linux.ibm.com>,
	"martin.petersen@oracle.com" <martin.petersen@oracle.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"m.szyprowski@samsung.com" <m.szyprowski@samsung.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"brijesh.singh@amd.com" <brijesh.singh@amd.com>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>,
	"thomas.lendacky@amd.com" <thomas.lendacky@amd.com>,
	"pgonda@google.com" <pgonda@google.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
	"rppt@kernel.org" <rppt@kernel.org>,
	"sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
	"aneesh.kumar@linux.ibm.com" <aneesh.kumar@linux.ibm.com>,
	"saravanand@fb.com" <saravanand@fb.com>,
	"krish.sadhukhan@oracle.com" <krish.sadhukhan@oracle.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"tj@kernel.org" <tj@kernel.org>,
	"rientjes@google.com" <rientjes@google.com>
Subject: Re: [PATCH V5 12/12] net: netvsc: Add Isolation VM support for
 netvsc driver
Message-ID: <20210930054821.GA22332@lst.de>
References: <20210914133916.1440931-1-ltykernel@gmail.com> <20210914133916.1440931-13-ltykernel@gmail.com> <MWHPR21MB15939A5D74CA1DF25EE816ADD7DB9@MWHPR21MB1593.namprd21.prod.outlook.com> <43e22b84-7273-4099-42ea-54b06f398650@gmail.com> <e379a60b-4d74-9167-983f-f70c96bb279e@gmail.com> <20210928053911.GA29208@lst.de> <de18b708-7252-642b-c60f-59e12ac27421@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <de18b708-7252-642b-c60f-59e12ac27421@gmail.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Sep 28, 2021 at 05:23:31PM +0800, Tianyu Lan wrote:
>>
>>   - the bare memremap usage in swiotlb looks strange and I'd
>>     definitively expect a well documented wrapper.
>
> OK. Should the wrapper in the DMA code? How about dma_map_decrypted() 
> introduced in the V4?

A mentioned then the name is a pretty bad choice as it touches the dma_map*
namespace that it is not related to.  I suspect just a little helper
in the swiotlb code that explains how it is used might be enogh for now.


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 05:54:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 05:54:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199378.353391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVp1O-0003Ij-1D; Thu, 30 Sep 2021 05:54:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199378.353391; Thu, 30 Sep 2021 05:54:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVp1N-0003Ia-S7; Thu, 30 Sep 2021 05:54:17 +0000
Received: by outflank-mailman (input) for mailman id 199378;
 Thu, 30 Sep 2021 05:54:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVp1M-0003IQ-QE; Thu, 30 Sep 2021 05:54:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVp1M-0000MI-Kj; Thu, 30 Sep 2021 05:54:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVp1M-0000aO-92; Thu, 30 Sep 2021 05:54:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVp1M-0002I2-88; Thu, 30 Sep 2021 05:54:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EXHJCQds1HmlIFAbUfPLo4eKhf3tcIOxJyTrbf73EVY=; b=7N+BsBinBlgsKomI8addlk6fD5
	LNzifxexdn3EY7JiVHBXPgdwl6//3+P8uSllS5Yr2XJW+okiDHGZR+WWNfP+6HBABV1QjVZmzFWBl
	2ysWSTzgbMnBoht/AD8IvsViM6xIBTm9uWX+w4dFrgwKFbRs7N+ootA52BCRaPdfzz8U=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165318-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 165318: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=890ceb9453171c85e881103e65dbb5cdcf81659e
X-Osstest-Versions-That:
    xen=c76cfada1cfad05aaf64ce3ad305c5467650e782
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 05:54:16 +0000

flight 165318 xen-unstable real [real]
flight 165326 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165318/
http://logs.test-lab.xenproject.org/osstest/logs/165326/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164945

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164945
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 164945
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164945
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164945
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164945
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164945
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164945
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  890ceb9453171c85e881103e65dbb5cdcf81659e
baseline version:
 xen                  c76cfada1cfad05aaf64ce3ad305c5467650e782

Last test of basis   164945  2021-09-10 21:23:48 Z   19 days
Failing since        164951  2021-09-12 00:14:36 Z   18 days   35 attempts
Testing same since   165318  2021-09-29 20:36:48 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Bertrand Marquis <bertrand.marquis@arm.com>
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Ian Jackson <iwj@xenproject.org>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Kevin Stefanov <kevin.stefanov@citrix.com>
  Nick Rosbrook <rosbrookn@ainfosec.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Penny Zheng <penny.zheng@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Stefano Stabellini <stefano.stabellini@xilinx.com>
  Tamas K Lengyel <tamas.lengyel@intel.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1030 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:08:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:08:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199385.353405 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqBP-0002lr-M1; Thu, 30 Sep 2021 07:08:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199385.353405; Thu, 30 Sep 2021 07:08:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqBP-0002lk-In; Thu, 30 Sep 2021 07:08:43 +0000
Received: by outflank-mailman (input) for mailman id 199385;
 Thu, 30 Sep 2021 07:08:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVqBO-0002le-3g
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:08:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3d36a45c-21bd-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 07:08:40 +0000 (UTC)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01lp2052.outbound.protection.outlook.com [104.47.2.52]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-r5bIzi4iMwmn4-s-3bYUBQ-1; Thu, 30 Sep 2021 09:08:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5469.eurprd04.prod.outlook.com (2603:10a6:803:cf::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 07:08:37 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 07:08:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0171.eurprd04.prod.outlook.com (2603:10a6:20b:331::26) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Thu, 30 Sep 2021 07:08:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d36a45c-21bd-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632985719;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FhB/8Qr3CA60wYZIDezcYv3ARVU27hfmmIJBlqDkLWY=;
	b=Xf8V88xgJjwklv8EBOfYsRw6G6B/+nah89PJ3Sl5dS33OIlcvYE64NZ+t8RxRjxlT6QKsE
	1gdoq5uvsGTCh0GHHfh2a1d4p6sLtV9suW5TtN2sKR8aGRuCnt5afnYp9pL7mHU8s+2z0i
	BCXeqlB5FySQbqAkrt7CjgNofI9Beiw=
X-MC-Unique: r5bIzi4iMwmn4-s-3bYUBQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WTNrSsMLlkze7zvkdGdRBs76fRrBagFu7W4fpVb5bES2QqHhE/LX3Su5qYymaIVlW8vcKRkSUwXlY1iQWML/udc5NJyXjxqtYtx7EFKZndMbDGuhXSW4hWTRxwrCg/V6H2NBU5yHGyj5ZTI/74Br0q3eS7JUZoKEmXy1C0SOMLdFB7/sjPMoZedvP8iB2ud4jGkrfebJKw/9zfF8mIwAe+LJUXTg785nvZDtjFD7txpEawpuRTSp8jjlojYQxaOrxZl6lL4dBdDfr6TMMCvvK934gB2DIf3iLqQHCLYja//+YQO4gG3cvddnJIFwupnnrHFzDSlPzl9ErwlTd0rR6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=FhB/8Qr3CA60wYZIDezcYv3ARVU27hfmmIJBlqDkLWY=;
 b=Mu+x2iJeOvQ6rlmkqJeNvz+1Ene+OkqGLINi63qHVtyqmmZbqqwIl5btkB+RPVhxlO8fA/8iyrbbDTDOF1hYP/1ffpZNzcOvsWRRHNeNgxrYlkaWG+G7HPlweXRlJ8/LjvGrLj4ss7GDgX+x0QifEmogodN0CQzan0G6pf2cmQ10bv/QtmAlXJV5ni7iRaqGfEh0FBIJH2fz8PUng1H/5Dre+Tex5mdJTwtY28E2jOILXSNjSZsIjEKGlbxq978xGTHAXaI3b27lDcgPL70RIpGQrBkMzMQ4MPYYPWYctKZXcS97siDWzu6sKGgMrP0GX8p6cTL/g8a+156rxEJqSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
 <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
 <YVD59QVbmdVwzYQI@mattapan.m5p.com>
 <033cc499-34de-d27a-7b1b-2a0a7ce38672@suse.com>
 <YVSGvGZc67ZLMdKm@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <03052a89-cb51-250d-4fcd-b9d8ceb93e39@suse.com>
Date: Thu, 30 Sep 2021 09:08:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YVSGvGZc67ZLMdKm@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0171.eurprd04.prod.outlook.com
 (2603:10a6:20b:331::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f7b687fd-8a82-453f-1cae-08d983e11f33
X-MS-TrafficTypeDiagnostic: VI1PR04MB5469:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5469679E0CE1DDCE6306ED9DB3AA9@VI1PR04MB5469.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dgsq/LgcDKFl88x47WrxCIgP73vIZSwpu9FUAbawBRdzQSke0eJa7o2gjYJ3zMHqBrkXpLxU5DR+vsmK77TG4ZorSAv88k/snBY5O+sx4jWpERVeoFsEfWfQ7qouHZuWJwe6wYS8PkPw7qx4k/xPEKHQc+q2kfSjEqdlT2pwOVgtrglhLint+17pj+NYrj9+h9hOnUZ7E5ftJhptjUaU+z9T7Z5RYS94APfIhbBpA+RyJcEDTeJf0IwgPa7/sgaiJ8OtM95w3S12OEbh/r8qdyHBzh7x8lX4CLMXmIE8YzHr1QXa+JtcJLFEY4/OwstBkWyKCjAjA2avuWIWVEPOnzRTysmRufIyuKQXIbxPn/ZgIY8MttuAEPHvz1GzUAScvRdaGZGjI+QBeRBKaGj9PUBwEW+GoYZh/mFGguaFAYcpX9O8c2ssdisDuInPs0KMlMxWRsO11UhXchRDG/VOAuJQbYvVhcZtyf3o0mLUgG7hT+WpPnqqL0Uc+smW5oUbTinZghnJocmFm9V/5AB/h34zBbrWe0KhbZmPnwjk2ADevYYnxB9XjFDa8/XVDdd2v8IRVbL1OhFcTxqE/dW+LxJ4XW6te0z2V+rmF5S0n0wtx8Y/Hxy0smk+gzq2Bs4cyA6KjNL3erHkT55fWdqWF4er3s5wqs3J9ZOu9IGckImiL3Spo9MnmRbl9jX06RdXcSFaONWyO639+Ifv7fE2YfnQ6fZB4SVQJZqCOKJ8KE8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66946007)(31686004)(26005)(31696002)(3480700007)(316002)(86362001)(8936002)(186003)(66476007)(66556008)(508600001)(6486002)(4326008)(2906002)(36756003)(16576012)(53546011)(5660300002)(83380400001)(8676002)(2616005)(38100700002)(956004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bzRqemdwaTVlK3pIOVNCSTVtL0czRnZOb05hdHdSWGpmTFBManhJL3dkMC90?=
 =?utf-8?B?VkQvOTROa1B0YXptODdXdTg3RW1ybTJYN1E2ZmcwZXpLVG4zOEZ0WEZId0FN?=
 =?utf-8?B?L3JPNjlOODdxdnNQNXo0dUFSZVIxNVdvRndnbFZUNFdrdGZKUkNLWHlDQ3hv?=
 =?utf-8?B?RkNnOWVCcll3bXROb0h2V1dsbHRWZVBxL3FRR0JlSWF4K2luTldYL1JheHJQ?=
 =?utf-8?B?dkpFSUJJSnZsWGM4SkhzQ2J0Zy9TQnd4NFFxRHlrWWFmNHVwajlXMHBzaXZo?=
 =?utf-8?B?NDBQNHpnNldmVnJLbTRjZzFZUEUzcTRNT0p6M25Wbnd6RFp6WXV4ZUE0Z3lx?=
 =?utf-8?B?VVExUWV5Ykd3TkZTRksvODdaU1Q2T0REYWFuZ3FBUUNmaDVOLzRjUVFMOG9D?=
 =?utf-8?B?YWE5QVU0R2xUY1hsMEhpRlZmZHRHeEJQUlZETExSQ1ZjK2VGcXBVUy9lTSt1?=
 =?utf-8?B?UVNPSm9HM0VCYTQrbTdtbzZGUWJNTjVuckU1dFZWRXZOQmhmRURqM0VKTm81?=
 =?utf-8?B?QXdaMS9FSGR3TW03NmdiZEl4KzlEWndMRk1KbHVjaUhaRTRpc2lSZDlqNGxu?=
 =?utf-8?B?cmNBSW9QVmtjRlRZREtwVHhJdkwvbDBEanNCNVo5Y0ZzcTZxbjFuclZ4a2hV?=
 =?utf-8?B?Sno5VjRBcVhobDRGQUE4VmZWMlhyd015T0xGbG5oOEVXMVN6dmtrUXpaalR2?=
 =?utf-8?B?cDRDb2Z0WUQvb0Qva3NJKzBiaW5SNytHQUdIeDFMYjZhdm44WkJ6VXY1ZzdY?=
 =?utf-8?B?Qkx3WUFzWGNVM3dIaXN0dUVxQUNNWmw3SGZ3RnZIRDYwaVM0TXN4L3ppaGJ3?=
 =?utf-8?B?cXp1bWd3MVRKZjhLbzV6SDlsN0dqNndkeTN3RmtyU2kvdllGOGFGQmtNeWQ4?=
 =?utf-8?B?bGQxNmRtUzlMeHhuMzAvYndXa3NSNndlaVRJWklmSE1XSk5mY3NpeEVUa2dF?=
 =?utf-8?B?anB3aFZveEJxajZ1SFFiQWZ4WE1nSG1jdEI5ZW1HWFJEdUFORDlmckt4YUxm?=
 =?utf-8?B?QkdQR3Bvd1gvNURDSXBwWG4yNVZTRlRSaU5EYlJtZjQyRCsvYWdONDQ1VHhQ?=
 =?utf-8?B?VEgzNURhQitCSUdRK0lxVWpobzdnSUNRRzRBNkpDaE9YVSt5cUtHby8zYWVu?=
 =?utf-8?B?WFpOTVplaElCWVhIcDRhbFoycitPaDNEQVdXWXpVcGZKcGN0MFlodnNJd1k0?=
 =?utf-8?B?MDBCZDJxQzR6OW1sbVVvWFgvTXpsYzZYUnlyMUpnQ3pyZHFOd1RBOFJTTElI?=
 =?utf-8?B?aWdTMlhObDBCWTg2MUduT2NIYVR2WHpqVVBZU0RQMUVkSUhMSkJkRWtrR0ZZ?=
 =?utf-8?B?THBKd0syNWI1SEZwSFRqYTJDR25IQ1FDb2dtb1R3Vncvanlkcmt6a1pDeU1E?=
 =?utf-8?B?Z1VabVNaTmIwN3JjUDhGS1M4Y1ZGMmdULy82RmZyVmQwUDQvK2lDNWpWTlda?=
 =?utf-8?B?R1h6VVdNS0RacGpLY25rY1ZQdGZ2OFVaVlA4R1IyQ08xNzJQQVF3clpxZk1z?=
 =?utf-8?B?TGRlcmcrQ2V1VUo0bnZIY0wzM0tTNlJyTkR6d3ltTWJJU0R2TmpVcFEvK294?=
 =?utf-8?B?V3had1ZuMnFoelpmMmtlV3VQTGxGdVdDdUNGdytnUllzQ2xlb1E2Y1dsRWZr?=
 =?utf-8?B?V3JmV2JCUmlRR1lpYjQzUzNNeVBsanRzeVg1WU95YVNveFRDWTdSZVdhUFV0?=
 =?utf-8?B?RDFQUHBGTEx6VEJUSXg0cUppVFZ4Y3VIa1VkemxNa3IxQldRLzJSQStGVXhy?=
 =?utf-8?Q?vnndi4TQ6TFATnW1xk4qqwynO7f82G1L2EIlRp0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f7b687fd-8a82-453f-1cae-08d983e11f33
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 07:08:36.8157
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1ncEj701BNODAAyF60ODzANrcrxaMOT8D+AhOOQJExBhxRc3i7nNmbwNMX7LfkCavAWxiEX8XC5+8NfJD5ayFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5469

On 29.09.2021 17:31, Elliott Mitchell wrote:
> On Wed, Sep 29, 2021 at 03:32:15PM +0200, Jan Beulich wrote:
>> On 27.09.2021 00:53, Elliott Mitchell wrote:
>>> (XEN) Xen call trace:
>>> (XEN)    [<ffff82d0402e8be0>] R arch/x86/mm/p2m.c#p2m_flush_table+0x240/0x260
>>> (XEN)    [<ffff82d0402ec51c>] S p2m_flush_nestedp2m+0x1c/0x30
>>> (XEN)    [<ffff82d0402e0528>] S arch/x86/mm/hap/hap.c#hap_write_p2m_entry+0x378/0x490
>>
>> hap_write_p2m_entry() calling p2m_flush_nestedp2m() suggests that
>> nestedhvm_enabled() was true for the domain. While we will want to
>> fix this, nested virt is experimental (even in current staging),
>> and hence there at least is no security concern.
> 
> Copy and paste from the xl.cfg man page:
> 
>        nestedhvm=BOOLEAN
>            Enable or disables guest access to hardware virtualisation
>            features, e.g. it allows a guest Operating System to also function
>            as a hypervisor. You may want this option if you want to run
>            another hypervisor (including another copy of Xen) within a Xen
>            guest or to support a guest Operating System which uses hardware
>            virtualisation extensions (e.g. Windows XP compatibility mode on
>            more modern Windows OS).  This option is disabled by default.
> 
> "This option is disabled by default." doesn't mean "this is an
> experimental feature with no security support and is likely to crash the
> hypervisor".

Correct, but this isn't the only place to look at. Quoting
SUPPORT.md:

"### x86/Nested HVM

 This means providing hardware virtulization support to guest VMs
 allowing, for instance, a nested Xen to support both PV and HVM guests.
 It also implies support for other hypervisors,
 such as KVM, Hyper-V, Bromium, and so on as guests.

    Status, x86 HVM: Experimental"

And with an experimental feature you have to expect crashes, no matter
that we'd prefer if you wouldn't hit any.

>> Can you confirm that by leaving nested off you don't run into this
>> (or a similar) issue?
> 
> Hypervisor doesn't panic.  `xl dmesg` does end up with:
> 
> (XEN) p2m_pod_demand_populate: Dom72 out of PoD memory! (tot=524304 ents=28773031 dom72)
> (XEN) domain_crash called from p2m-pod.c:1233
> 
> Which is problematic.  maxmem for this domain is set to allow for trading
> memory around, so it is desireable for it to successfully load even when
> its maximum isn't available.

Yet that's still a configuration error (of the guest), not a bug in
Xen.

Thanks for confirming that the issue is nested-hvm related. I'm in the
process of putting together a draft fix, but I'm afraid there's a
bigger underlying issue, so I'm not convinced we would want to go with
that fix even if you were to find that it helps in your case.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199393.353415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqG3-0004BG-CD; Thu, 30 Sep 2021 07:13:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199393.353415; Thu, 30 Sep 2021 07:13:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqG3-0004B9-9K; Thu, 30 Sep 2021 07:13:31 +0000
Received: by outflank-mailman (input) for mailman id 199393;
 Thu, 30 Sep 2021 07:13:30 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqG2-0004B3-AO
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:30 +0000
Received: from mail-lf1-x136.google.com (unknown [2a00:1450:4864:20::136])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b6ce97bb-51ee-4b26-b5eb-7974bd2019d0;
 Thu, 30 Sep 2021 07:13:29 +0000 (UTC)
Received: by mail-lf1-x136.google.com with SMTP id i19so18305371lfu.0
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:29 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6ce97bb-51ee-4b26-b5eb-7974bd2019d0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=JxEMJjjs3uh8Q+DakT3e0prA56g9kYxCO6qvYOCBJaA=;
        b=Q0yoOXzdg30oSNTyERMKnmfTSOJ6kir/Ngr17hbIHGVobzKKZO7yTEJ8hgAO2yB1f1
         JZnlGL2KhhbufQgdtm/wSAC27EdMTCvKFlC0R0fjHCH/vh/B8kKq8O9/QGXyD+zJ4f21
         tXjuZuS0XLigO3AwVldv9djiPMrvzewCOqXAiOojy3bcEA3C6MjVR5mDRVZaovlRtQEK
         IDsYtIrc/EfutqQOn89GqST/Emxerh8ZC1Z+xoQh2iRE4ALSDn1omFpsEHgkyUj8/crT
         2tryb2YurHWdPASbz+ahnrx5y12GZWrRvCMNL6khrDaaDBYieln/eN0ftVgmQAChG0Cj
         GCaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=JxEMJjjs3uh8Q+DakT3e0prA56g9kYxCO6qvYOCBJaA=;
        b=t/rg/Y9R+EepXG0LdA58Zm+6X5QWe7y9ZMcaUmXdyYbAejXa5GAEVj7ozd3QSCONtO
         ni/HuIO4r5rWmhU7H4FP6thxZmB6nTc25zuSuowG/xg7mpqO4R91Q7/ldjhw1BrmMppK
         cpjhdGm99cd7k8R3eSWGCn26F6Ot3cUEq7QNgBwkOGVgzAKa7cORs+qdvTG1gixlwOBL
         2+a2qYwnIbbgu2p8btEjFHJ2gvTppM73kYcS1hGiND28gHkZNP02WivEId24jJlKk/0K
         mV7ZSbhhEDO38tjEJ4Iu2XFXKFvIfVcVzpvc3p57tyZHEV5uNBM1wZLN099zKxW100N4
         zBUw==
X-Gm-Message-State: AOAM532HLauQCWMeY5r19MyzuYwLlhfJJ2hcgSiPusjCBzCoD+187GV3
	A+bCTE4YriBdtI/wJhY2NzSdwy2amGY=
X-Google-Smtp-Source: ABdhPJzwP93rdaCp9KnZbHtdQSZDvGJd95LI8oOVmP4BphHADrjZlyAILhNcxf60vYuUZwXEq3cDKA==
X-Received: by 2002:a05:6512:234a:: with SMTP id p10mr4428851lfu.558.1632986007881;
        Thu, 30 Sep 2021 00:13:27 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 00/11] PCI devices passthrough on Arm, part 2
Date: Thu, 30 Sep 2021 10:13:15 +0300
Message-Id: <20210930071326.857390-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This is an assorted series of patches which aim is to make some further
basis for PCI passthrough on Arm support. The series continues the work
published earlier by Arm [1] and adds new helpers and clears the way for
vPCI changes which will follow.

RFC is at [2], [3]. Design presentation can be found at [4].

I have dropped some r-b's from Stefano as I touched some patches after
the review: most of those are minor cleanups, but still.

Thank you,
Oleksandr

[1] https://patchwork.kernel.org/project/xen-devel/list/?series=554415
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
[3] https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
[4] https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf


Oleksandr Andrushchenko (10):
  xen/arm: Fix dev_is_dt macro definition
  xen/arm: Add new device type for PCI
  xen/arm: Introduce pci_find_host_bridge_node helper
  xen/device-tree: Make dt_find_node_by_phandle global
  xen/arm: Mark device as PCI while creating one
  libxl: Allow removing PCI devices for all types of domains
  libxl: Only map legacy PCI IRQs if they are supported
  xen/arm: Setup MMIO range trap handlers for hardware domain
  xen/arm: Do not map PCI ECAM and MMIO space to Domain-0's p2m
  xen/arm: Process pending vPCI map/unmap operations

Oleksandr Tyshchenko (1):
  xen/domain: Call pci_release_devices() when releasing domain resources

 tools/libs/light/Makefile          |  4 ++
 tools/libs/light/libxl_pci.c       | 15 ++++-
 xen/arch/arm/domain.c              | 11 +++-
 xen/arch/arm/domain_build.c        | 45 ++++++++-------
 xen/arch/arm/pci/ecam.c            | 14 +++++
 xen/arch/arm/pci/pci-host-common.c | 93 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
 xen/arch/arm/pci/pci.c             | 12 ++++
 xen/arch/arm/traps.c               | 13 +++++
 xen/arch/arm/vpci.c                | 33 +++++++++++
 xen/arch/arm/vpci.h                |  6 ++
 xen/arch/x86/hvm/hvm.c             |  6 ++
 xen/common/device_tree.c           |  2 +-
 xen/common/ioreq.c                 |  9 ---
 xen/drivers/passthrough/pci.c      |  2 +
 xen/include/asm-arm/device.h       |  6 +-
 xen/include/asm-arm/pci.h          | 28 +++++++++
 xen/include/asm-arm/setup.h        | 13 +++++
 xen/include/asm-x86/pci.h          |  2 +
 xen/include/xen/device_tree.h      |  2 +
 20 files changed, 280 insertions(+), 37 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199394.353427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqG8-0004Uk-KL; Thu, 30 Sep 2021 07:13:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199394.353427; Thu, 30 Sep 2021 07:13:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqG8-0004Ud-HA; Thu, 30 Sep 2021 07:13:36 +0000
Received: by outflank-mailman (input) for mailman id 199394;
 Thu, 30 Sep 2021 07:13:35 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqG7-0004B3-9N
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:35 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f515294a-3ed3-4353-bcc5-859044fbd7cf;
 Thu, 30 Sep 2021 07:13:30 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id y26so21268579lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:30 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f515294a-3ed3-4353-bcc5-859044fbd7cf
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=IBrpPALaHgEu2bRfUzTni2BaAuDBWtjiAy9s9dEtWuo=;
        b=Z9yyFXZsdmLsyD6irTOnBZJ4a2peO0DlcILrViPlOtCLbHHHPBQvfm+XXagPHBIesr
         /pMhLzjvAAIanKEVPa8Nq6293vtqZeg+RlKdHr7BiqpGn0SkK482Hgfb/0zRUBhEM4Aq
         Z/uWQEElIz3RBoFJ7n7ofrKD6c4WI6kQFEvukBKkEyhhvJxTadzdB1iprMtH4BC65DgT
         HyhURu3MK0YDM/lJBSpXuiD4mh1zxkztDBY4QCzblQdW+WCFuChcPQfIOq8sWMerf3oH
         vGX5MGSvIxz8vjzHuhIcTZJbKsrTlROnMMmB6uSVl7imLVoAZZMh8CzzJa9BVJG+JCbo
         qcoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=IBrpPALaHgEu2bRfUzTni2BaAuDBWtjiAy9s9dEtWuo=;
        b=zF/JAyoV7Bu9psGwPnOyO2q9IucMUm18En0TvcneoT2oaEn6Ujs8/xAlFmE6RMY85N
         r0ZYnswjcxR6cz/YgPR6wz6yrAk/cmRRr35aN78uRHChK6d5m8RRJTFkYH7+yCnBthNA
         UrjGj9mwtl1Xz3B2s9tfhXRO5WfWTJL0aIOlN1pMJx/XI5oov1jMz9fj6C4SkaFUbgHi
         9a6NEN2H11v7FjX1AFe2j+REgpYPlgLs0y/XJ2JbOsf7ap3SUiw2g3+S6QJ5mENy7NSs
         bnbKw/Ukrji8/ESwicbnz1eih4GjkBcQ00+F236KLuc24uTjaTtfKlYS+/xA3oJKEvZu
         wgGA==
X-Gm-Message-State: AOAM532lnOumxz99bkfRp102hznTPgEuAzcEQk8ia+uTJMjMboGgPaaS
	31jvtXDCfbChItrJLQqSxFNkhKvgDaY=
X-Google-Smtp-Source: ABdhPJxz3EzEY2bcM09sXyYuyw/lkXDJlm3JlawU2ox4wsarBkIa6FBNzfi8GwOhhMMM4VKX3TL9JA==
X-Received: by 2002:a2e:9591:: with SMTP id w17mr4250036ljh.40.1632986008850;
        Thu, 30 Sep 2021 00:13:28 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Michal Orzel <michal.orzel@arm.com>
Subject: [PATCH v3 01/11] xen/arm: Fix dev_is_dt macro definition
Date: Thu, 30 Sep 2021 10:13:16 +0300
Message-Id: <20210930071326.857390-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This macro is not currently used, but still has an error in it:
a missing parenthesis. Fix this, so the macro is properly defined.

Fixes: 6c5d3075d97e ("xen/arm: Introduce a generic way to describe device")

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
New in v2
---
 xen/include/asm-arm/device.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 5ecd5e7bd15e..ebe84ea853cd 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -27,7 +27,7 @@ typedef struct device device_t;
 
 /* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
 #define dev_is_pci(dev) ((void)(dev), 0)
-#define dev_is_dt(dev)  ((dev->type == DEV_DT)
+#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
 
 enum device_class
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199395.353438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGC-0004pG-VO; Thu, 30 Sep 2021 07:13:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199395.353438; Thu, 30 Sep 2021 07:13:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGC-0004p9-QA; Thu, 30 Sep 2021 07:13:40 +0000
Received: by outflank-mailman (input) for mailman id 199395;
 Thu, 30 Sep 2021 07:13:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGC-0004B3-9X
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:40 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 345376d7-90bf-4731-be1f-4e150d108e43;
 Thu, 30 Sep 2021 07:13:31 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id b15so21120329lfe.7
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:30 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 345376d7-90bf-4731-be1f-4e150d108e43
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=IQOqq0vw5ObAcax4b2zzMkwlZl3+eElA9Can/zWllgQ=;
        b=brzQZIxPuUAGA3eUAp2p9S981LqvoqpmB6dwDkdSo9Kf3r52uaiKUKcScK3jLExAy8
         xndRqxG+TGKHKvETguZ94WipFL+oPPSt6C2e7P0IaL/+/l4GUSRPt+wL7KnAYm9vhPxm
         aG4/3SkDXTINYSHVBLyPRqzBSnxaay4EEvfO/JlFttZlrB5g4H5I6Aa7c8tEPajhVdFD
         cTppFYUIGzf5uEjXhSlfRP3y71vRKIXyZO+KqjU0SeLdBhHguE5HPsjMdqkvzuUNj5Va
         lztysJxyr4t5Yb+UBMmlUDJbO/fY2EtKIo8aug7l8baAIPLnRfOfFkVHOTYckBIJZGyv
         KHdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=IQOqq0vw5ObAcax4b2zzMkwlZl3+eElA9Can/zWllgQ=;
        b=LArj/pXZv7Crm6lUmXPmSWJ+JurAQqkh/zVy4EZY0KoWgwKad8fkZ3V317obfDRi5q
         KRxT1VFyupGZtZFoA6T28PseDe93C322NgVAha7Mc6NGcubSk+4M+v+jNTKgwMCDFVpp
         /GWEATKOWJg8w3mnjkaZ80sn1W+ajtem43y/+DoZHZke87XbVJKyd7DPAKBgrVlL5/Za
         GbXKeYirTzpf1vHB4OWNVoXy+iTEyFgh77wFJ9gV+QHwFfGRjVYSndtkmSghftE8PtSu
         CYrCmpky5vRPvYB7uNDINc2k+PjdGi1iabD9iLRMHJcs4aMv37PxlWvcgkikIhu+3sKo
         IADg==
X-Gm-Message-State: AOAM532DXHY16qfx9rtBPYD4rpuaBdaec3Aadswe0MG3v4vupnAHS7xF
	ewfgW/F02aeNp62sYxr9+zw2WYb1KBc=
X-Google-Smtp-Source: ABdhPJzyPgBi1m/E6vkHHJqtg/a9OyNjPIlynE53PEkapio7Yjdxyz7UXZ78Qo64Pb94Pvl771CKrw==
X-Received: by 2002:a2e:5709:: with SMTP id l9mr4101790ljb.315.1632986009754;
        Thu, 30 Sep 2021 00:13:29 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 02/11] xen/arm: Add new device type for PCI
Date: Thu, 30 Sep 2021 10:13:17 +0300
Message-Id: <20210930071326.857390-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add new device type (DEV_PCI) to distinguish PCI devices from platform
DT devices, so some drivers, like IOMMU, can handle PCI devices
differently.

Also add a helper which is when given a struct device returns the
corresponding struct pci_dev which this device is a part of.

Because of the header cross-dependencies, e.g. we need both
struct pci_dev and struct arch_pci_dev at the same time, this cannot be
done with an inline.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Since v2:
 - !! dropped Stefano's r-b because of the changes
 - simplified dev_to_pci to use a single
   container_of(dev, struct pci_dev, arch.dev) (Jan)
Since v1:
 - Folded new device type (DEV_PCI) into this patch.
---
 xen/arch/arm/pci/pci.c       | 7 +++++++
 xen/include/asm-arm/device.h | 4 ++--
 xen/include/asm-arm/pci.h    | 7 +++++++
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index 84d8f0d63430..9fc522e566a9 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -27,6 +27,13 @@ int arch_pci_clean_pirqs(struct domain *d)
     return 0;
 }
 
+struct pci_dev *dev_to_pci(struct device *dev)
+{
+    ASSERT(dev->type == DEV_PCI);
+
+    return container_of(dev, struct pci_dev, arch.dev);
+}
+
 static int __init dt_pci_init(void)
 {
     struct dt_device_node *np;
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index ebe84ea853cd..7bf040560363 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -4,6 +4,7 @@
 enum device_type
 {
     DEV_DT,
+    DEV_PCI,
 };
 
 struct dev_archdata {
@@ -25,8 +26,7 @@ typedef struct device device_t;
 
 #include <xen/device_tree.h>
 
-/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
-#define dev_is_pci(dev) ((void)(dev), 0)
+#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
 #define dev_is_dt(dev)  ((dev)->type == DEV_DT)
 
 enum device_class
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 7cb2e2f1edaf..603a1fc072d1 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -26,6 +26,13 @@ struct arch_pci_dev {
     struct device dev;
 };
 
+/*
+ * Because of the header cross-dependencies, e.g. we need both
+ * struct pci_dev and struct arch_pci_dev at the same time, this cannot be
+ * done with an inline here. Macro can be implemented, but looks scary.
+ */
+struct pci_dev *dev_to_pci(struct device *dev);
+
 /* Arch-specific MSI data for vPCI. */
 struct vpci_arch_msi {
 };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:47 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199397.353449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGI-0005H5-99; Thu, 30 Sep 2021 07:13:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199397.353449; Thu, 30 Sep 2021 07:13:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGI-0005Gv-3e; Thu, 30 Sep 2021 07:13:46 +0000
Received: by outflank-mailman (input) for mailman id 199397;
 Thu, 30 Sep 2021 07:13:45 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGH-0004B3-9a
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:45 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id af20419a-75c2-4a4e-b952-e5462fa26285;
 Thu, 30 Sep 2021 07:13:31 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id z24so21166047lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:31 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af20419a-75c2-4a4e-b952-e5462fa26285
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=paCjRVnjfgtH4vjqWfPka1UDhVQcLz7hY/mZa1EMk18=;
        b=h7fp1+CvHJx6R/9JuM6y/TneeLsVMxymOjdZCMKjKOrN4f68MqO4HbcUGdUtFE++gN
         2XyAxRyUDPqM2WAsPBcSbovRYUUP3VfVyo42DO9MNjkFAX/Jcs22p3gVh9CLEDRY4qNl
         ettiWdKMaAcojj9OQ74LO6KTfqw/atujdDB9A3dy1Qq4e77bWYkD2tSehTsOP3csmw/t
         47NvVimcjUAslLm0SzErCZOOFj/TfvPWIy1EfVW+aXvOb+hyz0sQm56ATOLAvz1QTb5s
         oFbGiGwXtzSkQHssxqZpFA0jDvCU+mAKp7Gs9xIlHQSxuosfJl3Ro6b9l7IH9pqryUO3
         3DjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=paCjRVnjfgtH4vjqWfPka1UDhVQcLz7hY/mZa1EMk18=;
        b=Snb1M8+6DlhIf3niQ4tJP+0WDsde2vnWeKB1GeW9nmBYaAzjtjwL/qeGEpr7sVnyhI
         Np0j95dCvHfou07Wrk1QIst9BbT0ogowG9WrA89YcigImcl1pm7QL5KcBIAazaawvhFU
         KQ98TpKOjtz9+wznLWkYZYeHsN6QWIYfD/8ItY8szJkSYItdpSy1v3e7CnficlpGoj1A
         7SaXXFnfIhoakiPEscRNOTgsRPqTvRO2TduJ9olOFbk5VWXVlJcPJbkZ6Jel33CABfn9
         bOR6ZTH1aR9cj+MOqNtZMeDJpvoMY/3TDTX1KxRI6K0K0qp+axFju3c/nRO/H7ayjYQB
         VW+w==
X-Gm-Message-State: AOAM530rPsujvVe9lelfb8GZ9kTA0Ir9KZn1ASKkF9tBT43oyYgdnMI0
	YzpKl3eDvOLBECBFdZf6ZOQYNBBYWj0=
X-Google-Smtp-Source: ABdhPJzgK6zEPNoC1RoV6e4MX4dvw0DYh6rJCjA+Ht48/hAjTz1ybEuyJuIEVwZDhLE1L7NXtVbgsg==
X-Received: by 2002:a2e:5354:: with SMTP id t20mr4195057ljd.204.1632986010685;
        Thu, 30 Sep 2021 00:13:30 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 03/11] xen/arm: Introduce pci_find_host_bridge_node helper
Date: Thu, 30 Sep 2021 10:13:18 +0300
Message-Id: <20210930071326.857390-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Get host bridge node given a PCI device attached to it.

This helper will be re-used for adding PCI devices by the subsequent
patches.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

---
Since v2:
 - !! dropped Stefano's r-b because of the changes
 - s/PRI_pci/%pp after rebase onto Arm series
---
 xen/arch/arm/pci/pci-host-common.c | 16 ++++++++++++++++
 xen/include/asm-arm/pci.h          |  1 +
 2 files changed, 17 insertions(+)

diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 593beeb48ce4..592c01aae5bb 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -276,6 +276,22 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node,
     return -EINVAL;
 }
 
+/*
+ * Get host bridge node given a device attached to it.
+ */
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
+{
+    struct pci_host_bridge *bridge;
+    struct pci_dev *pdev = dev_to_pci(dev);
+
+    bridge = pci_find_host_bridge(pdev->seg, pdev->bus);
+    if ( unlikely(!bridge) )
+    {
+        printk(XENLOG_ERR "Unable to find PCI bridge for %pp\n", &pdev->sbdf);
+        return NULL;
+    }
+    return bridge->dt_node;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 603a1fc072d1..e6d4000e2ac8 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -99,6 +99,7 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
+struct dt_device_node *pci_find_host_bridge_node(struct device *dev);
 
 static always_inline bool is_pci_passthrough_enabled(void)
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199399.353460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGN-0005na-Gn; Thu, 30 Sep 2021 07:13:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199399.353460; Thu, 30 Sep 2021 07:13:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGN-0005nT-DG; Thu, 30 Sep 2021 07:13:51 +0000
Received: by outflank-mailman (input) for mailman id 199399;
 Thu, 30 Sep 2021 07:13:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGM-0004B3-9m
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:50 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fa0297e2-f1ca-4583-8ec1-5ceb5ab925c1;
 Thu, 30 Sep 2021 07:13:32 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id g41so21251914lfv.1
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:32 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa0297e2-f1ca-4583-8ec1-5ceb5ab925c1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=SW5l0COaWEZ5SQENULoKJ23OIO+t8l75C/PAXs3yYFM=;
        b=IbIUqXrhJTRPq6lhYGZqZrFXaKdmScTrbfzt7COuq+xkWXbioejqNA/Y81bZGni0mz
         98aJ1/kDEQYY1t9G1uY4Vn/Mpm4lKTGKEyd0B1IJsN0oJcZ0n5KFDQSo68GdWPoSxQ65
         L70hoLDq8j/gb+pmvkEe7e+xHxvf6sX/LsKV3cACJeJ/jjOQzJU+OoWYgn7xu8bLsB8N
         NjrMiJZ5t6oIU8CWOCFQkJIO0TbtO+dvxmZ3rVc7fX6m+1CZ5qdH8hVP01cZfQYvkld9
         6wiJ9zs61SYCV8jFTtBCKU3PZ8eTqxcrIabpWKfr8eAD914vLrDflYY4GrW952B8nvZA
         IegA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=SW5l0COaWEZ5SQENULoKJ23OIO+t8l75C/PAXs3yYFM=;
        b=tz+x2rEDvWaXtWaXfA/vjCpyBXHzULcRQNLM2K2qwj0sNIUZW0NQVSd5FyveREU75U
         EWhV5lXJkJ618tqG/S1A2vkYgLCW2evxp56HDB7AvUy7Tug9nml4k+bkxR7KqBnGiDKm
         SNJ8yfufL922bDQGLdC3XB8M8bAL02DnspgoV0dUfRSBL5M+WMFN+J/NlItIN1pm3JO7
         rPDs8b2zGzYtc7X81+jqgMM+ubrzK4w8iFVmisl3TWDAb91wqBdf30nPvW7KnUm5f2rh
         rANh1yDCag338Ko19ZY35+mWKZ2zcuPcy2rxITE8uxavL1NAGQBPH+yII/0lFe1yHG17
         rg8w==
X-Gm-Message-State: AOAM5326USpvQscraBt7p88QsFsv1LBjBv1zhpneSQ1R7c6DYIzHxdns
	IC5Nki+uaA2gDmyphM0NbNb0TiqRb6M=
X-Google-Smtp-Source: ABdhPJwauDS6x9R7+bWvRGCmKDDA9RWkLxAH63+3ofu4OaajWPa6y1TqRujWamY2ecr6eMk5JJHDuA==
X-Received: by 2002:a2e:974b:: with SMTP id f11mr4336756ljj.385.1632986011655;
        Thu, 30 Sep 2021 00:13:31 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 04/11] xen/device-tree: Make dt_find_node_by_phandle global
Date: Thu, 30 Sep 2021 10:13:19 +0300
Message-Id: <20210930071326.857390-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Make dt_find_node_by_phandle globally visible, so it can be re-used by
other frameworks.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 xen/common/device_tree.c      | 2 +-
 xen/include/xen/device_tree.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index ea93da1725f6..4aae281e89bf 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1047,7 +1047,7 @@ int dt_for_each_range(const struct dt_device_node *dev,
  *
  * Returns a node pointer.
  */
-static struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle)
 {
     struct dt_device_node *np;
 
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 2297c59ce66d..fd6cd00b433a 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -849,6 +849,8 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
  */
 int dt_get_pci_domain_nr(struct dt_device_node *node);
 
+struct dt_device_node *dt_find_node_by_phandle(dt_phandle handle);
+
 #ifdef CONFIG_DEVICE_TREE_DEBUG
 #define dt_dprintk(fmt, args...)  \
     printk(XENLOG_DEBUG fmt, ## args)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:13:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:13:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199403.353471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGS-0006Fa-Pg; Thu, 30 Sep 2021 07:13:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199403.353471; Thu, 30 Sep 2021 07:13:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGS-0006FJ-Lx; Thu, 30 Sep 2021 07:13:56 +0000
Received: by outflank-mailman (input) for mailman id 199403;
 Thu, 30 Sep 2021 07:13:55 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGR-0004B3-A1
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:13:55 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 217905a7-2aad-4d67-baac-a2ec09057c84;
 Thu, 30 Sep 2021 07:13:33 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id y26so21269232lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:33 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 217905a7-2aad-4d67-baac-a2ec09057c84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=hrW151Pn3PyP3za7zVLCbht2FcxzTewAzo6dwWGEqVU=;
        b=KyfuI8SonaSWtz6zuzp494aO7A2NeB2DFx5t5TBA7HLhBzmr0zEhamJF8rtXINz02Z
         g7xwVq2gx1xENguCWdaZgT02OwqEMKR2hAYJyR5m7BAD4TdoQyQ0IHW7QyeFrniw9C+L
         JgjGmpfvodDgYdTg4KrllQ5NlrOlx3qJmPCcyo0Kd1NEvdetdWK4lpllURpPP5K776wD
         vWo/Yf6W6UHHyTFrlXrB773n5JK+LZchpgueXLVT1EFMqt6C8zBntZDl/osxmcjNtbv4
         weFFl2qpX6aHOjjd2FZ7RIB8UblXwgA6GqOoCQxmm/AiwlMhgia0q9IEWQucM35t3ZgG
         WejA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=hrW151Pn3PyP3za7zVLCbht2FcxzTewAzo6dwWGEqVU=;
        b=JViZynxMC0BTguzu8kV5IGAWamnM+x1n6tDMZzziKKX0oZh54twgo/QMf47j2G7maJ
         f8D8jWXtV0mndVWn6crjMPZDnd9K9Q8YE2WZPpmHqyNY/TzmmUbXUM8Pyo1GywQBvVtY
         tFf9NjEsfkwqlN/wObcOeDCwRUHK70ONAui/afSBpVN10kSDh/v+b3FftOwttPonNwIp
         azPnrDRHuLUm6XMgfIcpT+XfFQq/fe4MikizVVScvgK/jraF4f3tSGCz7ZxlIkdvAf/r
         VPMNkncdMkMkvWEwJKFnxGyM7gjHPUcfLdIxvH0EzppxgzTOcXLd7WfpzC7XaTY1hTtk
         jgrg==
X-Gm-Message-State: AOAM531SqzYXEIdByPbVtxNE93/K5U9xgUJEWUAbNOdo2ieSBPGtyA1W
	pVgEOPhogo/01silz1c2+3RQoBpWRfg=
X-Google-Smtp-Source: ABdhPJzt9ZlZEBCnM5k9rLLU/Zi6Qrtzj23d7bzRA4B8Hmo0JcAyQKiyusR2vxgD//ruRm6yTB+myQ==
X-Received: by 2002:a2e:9410:: with SMTP id i16mr4390219ljh.134.1632986012562;
        Thu, 30 Sep 2021 00:13:32 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 05/11] xen/arm: Mark device as PCI while creating one
Date: Thu, 30 Sep 2021 10:13:20 +0300
Message-Id: <20210930071326.857390-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

While adding a PCI device mark it as such, so other frameworks
can distinguish it from DT devices.
For that introduce an architecture defined helper which may perform
additional initialization of the newly created PCI device.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Since v2:
 - !! dropped Stefano's r-b because of the changes
 - introduced arch_pci_init_pdev (Jan)
Since v1:
 - moved the assignment from iommu_add_device to alloc_pdev
---
 xen/arch/arm/pci/pci.c        | 5 +++++
 xen/drivers/passthrough/pci.c | 2 ++
 xen/include/asm-arm/pci.h     | 5 +++++
 xen/include/asm-x86/pci.h     | 2 ++
 4 files changed, 14 insertions(+)

diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c
index 9fc522e566a9..a8d10e869603 100644
--- a/xen/arch/arm/pci/pci.c
+++ b/xen/arch/arm/pci/pci.c
@@ -34,6 +34,11 @@ struct pci_dev *dev_to_pci(struct device *dev)
     return container_of(dev, struct pci_dev, arch.dev);
 }
 
+void arch_pci_init_pdev(struct pci_dev *pdev)
+{
+    pci_to_dev(pdev)->type = DEV_PCI;
+}
+
 static int __init dt_pci_init(void)
 {
     struct dt_device_node *np;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 38eb451448a3..9f804a50e780 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -329,6 +329,8 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn)
     *((u8*) &pdev->devfn) = devfn;
     pdev->domain = NULL;
 
+    arch_pci_init_pdev(pdev);
+
     rc = pdev_msi_init(pdev);
     if ( rc )
     {
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index e6d4000e2ac8..566a9436a18e 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -105,6 +105,9 @@ static always_inline bool is_pci_passthrough_enabled(void)
 {
     return pci_passthrough_enabled;
 }
+
+void arch_pci_init_pdev(struct pci_dev *pdev);
+
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
@@ -121,5 +124,7 @@ static inline int pci_get_host_bridge_segment(const struct dt_device_node *node,
     return -EINVAL;
 }
 
+static inline void arch_pci_init_pdev(struct pci_dev *pdev) {}
+
 #endif  /*!CONFIG_HAS_PCI*/
 #endif /* __ARM_PCI_H__ */
diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h
index 61b940c91d4a..e7f10beb4eb8 100644
--- a/xen/include/asm-x86/pci.h
+++ b/xen/include/asm-x86/pci.h
@@ -38,4 +38,6 @@ static always_inline bool is_pci_passthrough_enabled(void)
     return true;
 }
 
+static inline void arch_pci_init_pdev(struct pci_dev *pdev) {}
+
 #endif /* __X86_PCI_H__ */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:14:01 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:14:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199405.353481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGX-0006mR-9Z; Thu, 30 Sep 2021 07:14:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199405.353481; Thu, 30 Sep 2021 07:14:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGX-0006mD-61; Thu, 30 Sep 2021 07:14:01 +0000
Received: by outflank-mailman (input) for mailman id 199405;
 Thu, 30 Sep 2021 07:14:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGW-0004B3-A0
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:00 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fd2f18fa-4e3d-477f-b7ea-6800107a4ee7;
 Thu, 30 Sep 2021 07:13:34 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id b20so21448376lfv.3
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:34 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd2f18fa-4e3d-477f-b7ea-6800107a4ee7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=+Ihu8WIsBNrrd/XYg45D3f8fvmiiGUNqVLLkujQ3kSQ=;
        b=fE5/IjUDjo4l107VbJ4aqktSyFzd7iAjLsxvokbkRHyLo4sKvfQL4RQpibEt36GD28
         4C6KrzW4Q8H4zCYcaEyMQYWNKFUaB/2kZi9MAWp2Q3qzzg+UCq/5rWPHvhvpcQ9/etwK
         CkKOm1FYGwihWockE7SHYnX3bdl7BG2uN+Or25KDZzJcvSLaqOBRjGdm09nXqFIN0Gkn
         LOCPa24WrPp0twSwtoraG/DDudssRsvSI3T1XzByPAoff5lcqX25ey1HogZNmgZ53fAO
         neyW7FovDTbbLvFWucFJ74aD+gu1bdgd5/qTW3PMuIvnqzAormQOEIvOu4pbeYDAWOUf
         UvtQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=+Ihu8WIsBNrrd/XYg45D3f8fvmiiGUNqVLLkujQ3kSQ=;
        b=rWOf1NzpFS0eyKDFQ9QoaWlPFI2nwaOoU+iu7Lzo4YfX9GvnBnOx9KZ7YTvYKlaT2c
         M9Aafjm+kksGK2VGP+n++KP5Gdl4UvgBTYMyyEhLlati2GNcVCpThk4XUbKU9U6C5xkI
         7ZgRJnayZfHFIHz3KrPKo9+Wc4ThVrTGcLPZoHOobv/SpurE7uuUy9AWPQdZiwfXObyb
         tcCylXn8iIoSwhAP5dSwfJzWm7cmCoT1aEhEEVinbbbDHXBSxVbUYvy7wLRkgukvTupq
         JbMcnbmvbnU8FiX3didYjPxySRKbomhFneSmHKf/m88I368ZyJQLAnZGVd/S7mRfJEOL
         2RGg==
X-Gm-Message-State: AOAM530rwyOl/nUbhnNNnu60tiMWOL8Cldeh6G06J13z4EWzhWotOzEH
	1031yPR0AO4Es7EKjgpNAO4DCqsjBVo=
X-Google-Smtp-Source: ABdhPJz3/MVKmhM5NSLFMJHtFSPfgOVqhklIHuHK5sSHXtiDLFtawOUTyUxTLRQNQ4StGxtpEF7bfg==
X-Received: by 2002:ac2:50d1:: with SMTP id h17mr4320299lfm.212.1632986013540;
        Thu, 30 Sep 2021 00:13:33 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 06/11] xen/domain: Call pci_release_devices() when releasing domain resources
Date: Thu, 30 Sep 2021 10:13:21 +0300
Message-Id: <20210930071326.857390-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

This is the very same that we already do for DT devices. Moreover, x86
already calls pci_release_devices().

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Since v1:
 - re-wording in the commit message
---
 xen/arch/arm/domain.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index fbb52f78f1a6..79012bf77757 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -985,7 +985,8 @@ static int relinquish_memory(struct domain *d, struct page_list_head *list)
  * function which may return -ERESTART.
  */
 enum {
-    PROG_tee = 1,
+    PROG_pci = 1,
+    PROG_tee,
     PROG_xen,
     PROG_page,
     PROG_mapping,
@@ -1022,6 +1023,12 @@ int domain_relinquish_resources(struct domain *d)
 #ifdef CONFIG_IOREQ_SERVER
         ioreq_server_destroy_all(d);
 #endif
+#ifdef CONFIG_HAS_PCI
+    PROGRESS(pci):
+        ret = pci_release_devices(d);
+        if ( ret )
+            return ret;
+#endif
 
     PROGRESS(tee):
         ret = tee_relinquish_resources(d);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:14:06 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:14:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199409.353493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGc-0007FS-L9; Thu, 30 Sep 2021 07:14:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199409.353493; Thu, 30 Sep 2021 07:14:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGc-0007FC-HH; Thu, 30 Sep 2021 07:14:06 +0000
Received: by outflank-mailman (input) for mailman id 199409;
 Thu, 30 Sep 2021 07:14:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGb-0004B3-AJ
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:05 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d1ae476d-7a57-4cb3-97fd-038cc0246998;
 Thu, 30 Sep 2021 07:13:35 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id x27so21169465lfa.9
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:35 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:34 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1ae476d-7a57-4cb3-97fd-038cc0246998
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=FiQ0pvnKAQPmcXJwpGM/6RO4ozSTrcDZxYX1z5919gATo4P3P1EFGVZYDPBys8gdiE
         EWLY95k4AM5D6CpQHdaw0QMd+PUlp9+pj3TGfpPgcv3Fdo4y0lXHyg9DnV7qBTSW/x0f
         3CiMwVJdQzHtxQHHeK8jtEVxEgrj5/RXegApxZfc84UlAgVWWL0hMhQyChFGUV5kmMtv
         pHM8xSFrotLYTq6YLyeQZlhn/CQ4ZZYL7mT3fHXBG8zi7HXBMSFPTeG5fzzXgGjc7hWL
         +xZmGUDnB9Le0mvIndlJEF7UOLeu0JCciIxHox1KZcmELzt5L1xesbt5SvTT7viC259m
         1B8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=kD50gSwDk757eJ2CQu1b9/IhYMvfHenQQYtu99uRv54=;
        b=26TCNL0p5uhGx75DlxgKlk243S35sGayyy+AiUx74dgHWA19puozOPcsOVv/zdOoLZ
         /kby0Gaqc3gCMVINhc68gmHQNmWh3veGJjEekiWWSTaANooo+6a4oRJHCZVLkLgGAsDZ
         EZpSWKMWdAuybBZTrVjQ6yRFN28q+itNNEt5gX+eHnCMsjrBv8PhqD/NgN+QaQRB2htv
         aXfn9E532CpeWFmOqVIXARQzjlQJ2OlFhI+/DSoxjpBS2nbuTpS6WMBZ1UVqgh8KWRDH
         bH90mGVS7p3hMyZuJMXEMqqTCDmLj0E8COqyta5kMG4LIi8ZXyZ8XzCX7p/tvUthuXQ4
         9hjw==
X-Gm-Message-State: AOAM5333rBkZFn+TsgfoqJ4cv61IWBoDDlbsunaoXIFMchPqN6S1rGuA
	Ah0jHcH20FPSMVgf9dXlGfLhTCeueoE=
X-Google-Smtp-Source: ABdhPJxK2IG4yCGDAvt+yiuT3W0VTju+UvV6mAq2t5rLimiqArg9MYluOZtC5rhtEqDjKIwfSmFz+Q==
X-Received: by 2002:a2e:7311:: with SMTP id o17mr4344687ljc.488.1632986014658;
        Thu, 30 Sep 2021 00:13:34 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3 07/11] libxl: Allow removing PCI devices for all types of domains
Date: Thu, 30 Sep 2021 10:13:22 +0300
Message-Id: <20210930071326.857390-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

The PCI device remove path may now be used by PVH on ARM, so the
assert is no longer valid.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>
---
 tools/libs/light/libxl_pci.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 1a1c2630803b..59f3686fc85e 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1947,8 +1947,6 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
             goto out_fail;
         }
     } else {
-        assert(type == LIBXL_DOMAIN_TYPE_PV);
-
         char *sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", pci->domain,
                                      pci->bus, pci->dev, pci->func);
         FILE *f = fopen(sysfs_path, "r");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:14:17 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:14:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199415.353504 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGm-00086w-Vq; Thu, 30 Sep 2021 07:14:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199415.353504; Thu, 30 Sep 2021 07:14:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqGm-00086d-RW; Thu, 30 Sep 2021 07:14:16 +0000
Received: by outflank-mailman (input) for mailman id 199415;
 Thu, 30 Sep 2021 07:14:15 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGl-0004B3-Ag
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:15 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0e1c1189-8353-42d6-9b1f-c782ada7b6bb;
 Thu, 30 Sep 2021 07:13:37 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id m3so21062150lfu.2
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:37 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e1c1189-8353-42d6-9b1f-c782ada7b6bb
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=7YmzvU66v4oKyYcFR8Qcx6xXzeJU6lQheixQJ2KXwag=;
        b=DPS9owDYe0SrU04Tkqz8pVBaOYEPDHe/1M9uoRWQbJ4fzwucCP0lzxCsleZaVXMjy2
         WAonXBRY6B8YdoUTs5VRCU693cgFzaezS8q6briLup4Tj48PQI+K5skX5J/Sn1IsBGAX
         B00kUojW7vkP/GM6oOAnEKDYRBBHAdyhyExNEZMOc7yGGv5fpG+hhQo3kDVKFZ34xHj1
         zb1lghVJEQP4z6hcLdv2twbzp3p8kPrBaM21XVA0M9ssRK8Zx44U9CwQ3UnGokchlGoH
         2JSCe/e7pa2XJiVkco+vYBV4jDN6BEzfKTaBhoqkw7Kycb+jNR4b/Mw0US0jMKqTbIP1
         KZ5g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=7YmzvU66v4oKyYcFR8Qcx6xXzeJU6lQheixQJ2KXwag=;
        b=W+95KdK3XgJ+sOU8ZXvH/DfRHOpT82Tb80wyTTr3IQ2KY9AlKtRaVUCKj8RS7/3fK+
         2mqlXGFm1m2YPcBppJoORNAclg/MFcNjwiERMvYRf1TjZid1EF+DLEQKf3e4kT9OrnrY
         9DxAr0mMR56PkGstjjiKyFrlvZ2E9tp3S04EXXzyyNU1kpqSyywg/XUP9HfJ/u2MW3rb
         ep/fMd5VOilQFJfFAbt5FzfSxe5Iaf/mroBgmBcpHWlfIRJgGV1hL7kDgQLqKMyUBKiO
         /H8Wu6xZmDcZbFkoSx71Es3Wbv010lXBjV5ZAnybF7iZeCVvo5qAMdLG35uA9bf3Sp8x
         i/mg==
X-Gm-Message-State: AOAM531YneY5DMo3Dzpchm5MUC5wHjGi8oncqYEpirWTrhlS9Zp7LL27
	Jhs5Nu3m3ap3kjiH76HHhpls8U4dR8Y=
X-Google-Smtp-Source: ABdhPJzdNWHX6vLA3+VFvxgtFIgKFatKPLdNOnkIiZ8AUSDd7r1CIz1XOK8h44L41T5B42wSG7tgJQ==
X-Received: by 2002:ac2:59c7:: with SMTP id x7mr4391563lfn.662.1632986015644;
        Thu, 30 Sep 2021 00:13:35 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Ian Jackson <iwj@xenproject.org>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3 08/11] libxl: Only map legacy PCI IRQs if they are supported
Date: Thu, 30 Sep 2021 10:13:23 +0300
Message-Id: <20210930071326.857390-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Arm's PCI passthrough implementation doesn't support legacy interrupts,
but MSI/MSI-X. This can be the case for other platforms too.
For that reason introduce a new CONFIG_PCI_SUPP_LEGACY_IRQ and add
it to the CFLAGS and compile the relevant code in the toolstack only if
applicable.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Juergen Gross <jgross@suse.com>

---
Since v1:
 - Minimized #idefery by introducing pci_supp_legacy_irq function
   for relevant checks
---
 tools/libs/light/Makefile    |  4 ++++
 tools/libs/light/libxl_pci.c | 13 +++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 7d8c51d49242..bd3f6be2a183 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -46,6 +46,10 @@ CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
 	-Wno-declaration-after-statement -Wformat-nonliteral
 CFLAGS += -I.
 
+ifeq ($(CONFIG_X86),y)
+CFLAGS += -DCONFIG_PCI_SUPP_LEGACY_IRQ
+endif
+
 SRCS-$(CONFIG_X86) += libxl_cpuid.c
 SRCS-$(CONFIG_X86) += libxl_x86.c
 SRCS-$(CONFIG_X86) += libxl_psr.c
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 59f3686fc85e..4c2d7aeefbb2 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1364,6 +1364,15 @@ static void pci_add_timeout(libxl__egc *egc, libxl__ev_time *ev,
     pci_add_dm_done(egc, pas, rc);
 }
 
+static bool pci_supp_legacy_irq(void)
+{
+#ifdef CONFIG_PCI_SUPP_LEGACY_IRQ
+    return true;
+#else
+    return false;
+#endif
+}
+
 static void pci_add_dm_done(libxl__egc *egc,
                             pci_add_state *pas,
                             int rc)
@@ -1434,6 +1443,8 @@ static void pci_add_dm_done(libxl__egc *egc,
         }
     }
     fclose(f);
+    if (!pci_supp_legacy_irq())
+        goto out_no_irq;
     sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                 pci->bus, pci->dev, pci->func);
     f = fopen(sysfs_path, "r");
@@ -1983,6 +1994,8 @@ static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
         }
         fclose(f);
 skip1:
+        if (!pci_supp_legacy_irq())
+            goto skip_irq;
         sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
                                pci->bus, pci->dev, pci->func);
         f = fopen(sysfs_path, "r");
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:17:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:17:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199432.353515 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJX-0001F6-FV; Thu, 30 Sep 2021 07:17:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199432.353515; Thu, 30 Sep 2021 07:17:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJX-0001Ez-CA; Thu, 30 Sep 2021 07:17:07 +0000
Received: by outflank-mailman (input) for mailman id 199432;
 Thu, 30 Sep 2021 07:17:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGv-0004B3-B3
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:25 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f6020577-704f-4491-a844-1f63bf867f4d;
 Thu, 30 Sep 2021 07:13:39 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id z24so21167246lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:38 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:37 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6020577-704f-4491-a844-1f63bf867f4d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ouh1MyH5wsYBFAWUFQt0I31bU+DCu59CTYrgPlr2Ams=;
        b=VcKcE+jcaXiXp/2dpGdv+Mnjvwe6akCaqH0aUi5/stb0aVWw1j0V32hdsc4Kx0nxGv
         62DR1w79fvwDnHtk0JE47EYTeZq3g1h0q0HvJIZ0Ik9HK78uN2qZyUwY4ytD8heJ2YEx
         wP5xoJIJDhCFDJRmSx9Dcvy5yQEAlyyPP6q+23O/aUX1nAQ9qDf4eFoYLUcIT4Abub4H
         XZjoBb/uRUwKVX0jIau1PZYaspx3EuupvknEJmVMu+y8hunNrTMFPMgDCOqd/hhnzNPu
         bSn7/9QiwKhjV4IZCwv1hoBxOmiW1MXaVhT3cWW1+zLcn9nqqXje9qg5DGX2W00/YKG+
         Eymg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ouh1MyH5wsYBFAWUFQt0I31bU+DCu59CTYrgPlr2Ams=;
        b=fbABh9ZRGMV/ogsvHHt1Gky+sKx5cEEQXKVF9uakeMkPCcbEDdfy4TDhWhX1IiLMsp
         a4Rj5I2oSj0eRuBDyhlf75GlBLdRb5o8rUsLir6kxypftjvUfdjMdagAZAGxNFxUU1c8
         f8VuqxfRkYip9pDd82br0rGY+JQpLs2CWYhJyQkbCb1GB3gdtNZwTcC5U4lFOvsEhtuG
         /FMi0s+xQOixC7PIlrNOM8cBC/3QbPAWWZEkLTlY17CfSDC1mbRM3sdAoOvS/H+ah4B1
         KV1gjKRDehHMhfKALBmdVUHGWEElGYbXeD06s6p6I5GSWamOIzr3B5gojadrb1sKeYk4
         xzSQ==
X-Gm-Message-State: AOAM533zsSGCVqD0EOW5zzB2wZCbP4SmkYZI5iQBNQiYZw8XmPVN1xwy
	U+6s+EV+dCV0Mx0OzELaLZoS40bukrQ=
X-Google-Smtp-Source: ABdhPJzWPbyxltCvg1odF5xweKWtH4/vqqNiEOkxAEqP9mTxT1BkhSBqK9jNGwdY2GKSV3cBacklyw==
X-Received: by 2002:a05:6512:2036:: with SMTP id s22mr4417721lfs.317.1632986017656;
        Thu, 30 Sep 2021 00:13:37 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 10/11] xen/arm: Do not map PCI ECAM and MMIO space to Domain-0's p2m
Date: Thu, 30 Sep 2021 10:13:25 +0300
Message-Id: <20210930071326.857390-11-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

PCI host bridges are special devices in terms of implementing PCI
passthrough. According to [1] the current implementation depends on
Domain-0 to perform the initialization of the relevant PCI host
bridge hardware and perform PCI device enumeration. In order to
achieve that one of the required changes is to not map all the memory
ranges in map_range_to_domain as we traverse the device tree on startup
and perform some additional checks if the range needs to be mapped to
Domain-0.

The generic PCI host controller device tree binding says [2]:
- ranges: As described in IEEE Std 1275-1994, but must provide
          at least a definition of non-prefetchable memory. One
          or both of prefetchable Memory and IO Space may also
          be provided.

- reg   : The Configuration Space base address and size, as accessed
          from the parent bus.  The base address corresponds to
          the first bus in the "bus-range" property.  If no
          "bus-range" is specified, this will be bus 0 (the default).

>From the above none of the memory ranges from the "ranges" property
needs to be mapped to Domain-0 at startup as MMIO mapping is going to
be handled dynamically by vPCI as we assign PCI devices, e.g. each
device assigned to Domain-0/guest will have its MMIOs mapped/unmapped
as needed by Xen.

The "reg" property covers not only ECAM space, but may also have other
then the configuration memory ranges described, for example [3]:
- reg: Should contain rc_dbi, config registers location and length.
- reg-names: Must include the following entries:
   "rc_dbi": controller configuration registers;
   "config": PCIe configuration space registers.

This patch makes it possible to not map all the ranges from the
"ranges" property and also ECAM from the "reg". All the rest from the
"reg" property still needs to be mapped to Domain-0, so the PCI
host bridge remains functional in Domain-0.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00777.html
[2] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt
[3] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
 - removed check in map_range_to_domain for PCI_DEV
   and moved it to handle_device, so the code is
   simpler
 - s/map_pci_bridge/skip_mapping
 - extended comment in pci_host_bridge_mappings
 - minor code restructure in construct_dom0
 - s/.need_p2m_mapping/.need_p2m_hwdom_mapping and related
   callbacks
 - unsigned int i; in pci_host_bridge_mappings
Since v1:
 - Added better description of why and what needs to be mapped into
   Domain-0's p2m and what doesn't
 - Do not do any mappings for PCI devices while traversing the DT
 - Walk all the bridges and make required mappings in one go
---
 xen/arch/arm/domain_build.c        | 45 ++++++++++++++-------------
 xen/arch/arm/pci/ecam.c            | 14 +++++++++
 xen/arch/arm/pci/pci-host-common.c | 49 ++++++++++++++++++++++++++++++
 xen/arch/arm/pci/pci-host-zynqmp.c |  1 +
 xen/include/asm-arm/pci.h          | 10 ++++++
 xen/include/asm-arm/setup.h        | 13 ++++++++
 6 files changed, 111 insertions(+), 21 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 026c9e5c6cc4..ed46bcf8e7eb 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -10,7 +10,6 @@
 #include <asm/regs.h>
 #include <xen/errno.h>
 #include <xen/err.h>
-#include <xen/device_tree.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/guest_access.h>
 #include <xen/iocap.h>
@@ -47,12 +46,6 @@ static int __init parse_dom0_mem(const char *s)
 }
 custom_param("dom0_mem", parse_dom0_mem);
 
-struct map_range_data
-{
-    struct domain *d;
-    p2m_type_t p2mt;
-};
-
 /* Override macros from asm/page.h to make them work with mfn_t */
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
@@ -1388,13 +1381,13 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
     return 0;
 }
 
-static int __init map_range_to_domain(const struct dt_device_node *dev,
-                                      u64 addr, u64 len,
-                                      void *data)
+int __init map_range_to_domain(const struct dt_device_node *dev,
+                               u64 addr, u64 len, void *data)
 {
     struct map_range_data *mr_data = data;
     struct domain *d = mr_data->d;
-    bool need_mapping = !dt_device_for_passthrough(dev);
+    bool need_mapping = !dt_device_for_passthrough(dev) &&
+                        !mr_data->skip_mapping;
     int res;
 
     /*
@@ -1446,23 +1439,21 @@ static int __init map_range_to_domain(const struct dt_device_node *dev,
  * then we may need to perform additional mappings in order to make
  * the child resources available to domain 0.
  */
-static int __init map_device_children(struct domain *d,
-                                      const struct dt_device_node *dev,
-                                      p2m_type_t p2mt)
+static int __init map_device_children(const struct dt_device_node *dev,
+                                      struct map_range_data *mr_data)
 {
-    struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
-    int ret;
-
     if ( dt_device_type_is_equal(dev, "pci") )
     {
+        int ret;
+
         dt_dprintk("Mapping children of %s to guest\n",
                    dt_node_full_name(dev));
 
-        ret = dt_for_each_irq_map(dev, &map_dt_irq_to_domain, d);
+        ret = dt_for_each_irq_map(dev, &map_dt_irq_to_domain, mr_data->d);
         if ( ret < 0 )
             return ret;
 
-        ret = dt_for_each_range(dev, &map_range_to_domain, &mr_data);
+        ret = dt_for_each_range(dev, &map_range_to_domain, mr_data);
         if ( ret < 0 )
             return ret;
     }
@@ -1543,6 +1534,12 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     int res;
     u64 addr, size;
     bool need_mapping = !dt_device_for_passthrough(dev);
+    struct map_range_data mr_data = {
+        .d = d,
+        .p2mt = p2mt,
+        .skip_mapping = is_pci_passthrough_enabled() &&
+                        (device_get_class(dev) == DEVICE_PCI)
+    };
 
     naddr = dt_number_of_address(dev);
 
@@ -1582,7 +1579,6 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     /* Give permission and map MMIOs */
     for ( i = 0; i < naddr; i++ )
     {
-        struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
         res = dt_device_get_address(dev, i, &addr, &size);
         if ( res )
         {
@@ -1596,7 +1592,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
             return res;
     }
 
-    res = map_device_children(d, dev, p2mt);
+    res = map_device_children(dev, &mr_data);
     if ( res )
         return res;
 
@@ -2754,7 +2750,14 @@ static int __init construct_dom0(struct domain *d)
         return rc;
 
     if ( acpi_disabled )
+    {
         rc = prepare_dtb_hwdom(d, &kinfo);
+        if ( rc < 0 )
+            return rc;
+#ifdef CONFIG_HAS_PCI
+        rc = pci_host_bridge_mappings(d, p2m_mmio_direct_c);
+#endif
+    }
     else
         rc = prepare_acpi(d, &kinfo);
 
diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c
index 602d00799c8d..b81620074a91 100644
--- a/xen/arch/arm/pci/ecam.c
+++ b/xen/arch/arm/pci/ecam.c
@@ -40,6 +40,19 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
     return base + (PCI_DEVFN2(sbdf.bdf) << devfn_shift) + where;
 }
 
+bool pci_ecam_need_p2m_hwdom_mapping(struct domain *d,
+                                     struct pci_host_bridge *bridge,
+                                     uint64_t addr)
+{
+    struct pci_config_window *cfg = bridge->cfg;
+
+    /*
+     * We do not want ECAM address space to be mapped in Domain-0's p2m,
+     * so we can trap access to it.
+     */
+    return cfg->phys_addr != addr;
+}
+
 /* ECAM ops */
 const struct pci_ecam_ops pci_generic_ecam_ops = {
     .bus_shift  = 20,
@@ -47,6 +60,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = {
         .map_bus                = pci_ecam_map_bus,
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
+        .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping,
     }
 };
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 1eb4daa87365..085f08e23e0c 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -18,6 +18,7 @@
 
 #include <xen/init.h>
 #include <xen/pci.h>
+#include <asm/setup.h>
 #include <xen/rwlock.h>
 #include <xen/sched.h>
 #include <xen/vmap.h>
@@ -320,6 +321,54 @@ int pci_host_get_num_bridges(void)
     return count;
 }
 
+int __init pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt)
+{
+    struct pci_host_bridge *bridge;
+    struct map_range_data mr_data = {
+        .d = d,
+        .p2mt = p2mt,
+        .skip_mapping = false
+    };
+
+    /*
+     * For each PCI host bridge we need to only map those ranges
+     * which are used by Domain-0 to properly initialize the bridge,
+     * e.g. we do not want to map ECAM configuration space which lives in
+     * "reg" or "assigned-addresses" device tree property, but we want to
+     * map other regions of the host bridge. The PCI aperture defined by
+     * the "ranges" device tree property should also be skipped.
+     */
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        const struct dt_device_node *dev = bridge->dt_node;
+        unsigned int i;
+
+        for ( i = 0; i < dt_number_of_address(dev); i++ )
+        {
+            uint64_t addr, size;
+            int err;
+
+            err = dt_device_get_address(dev, i, &addr, &size);
+            if ( err )
+            {
+                printk(XENLOG_ERR
+                       "Unable to retrieve address range index=%u for %s\n",
+                       i, dt_node_full_name(dev));
+                return err;
+            }
+
+            if ( bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr) )
+            {
+                err = map_range_to_domain(dev, addr, size, &mr_data);
+                if ( err )
+                    return err;
+            }
+        }
+    }
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
index 6ccbfd15c968..a57bf0fbc1cc 100644
--- a/xen/arch/arm/pci/pci-host-zynqmp.c
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -33,6 +33,7 @@ const struct pci_ecam_ops nwl_pcie_ops = {
         .map_bus                = pci_ecam_map_bus,
         .read                   = pci_generic_config_read,
         .write                  = pci_generic_config_write,
+        .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping,
     }
 };
 
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 0d0266b13cba..1bfba3da8f51 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -17,6 +17,8 @@
 
 #ifdef CONFIG_HAS_PCI
 
+#include <asm/p2m.h>
+
 #define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
 
 extern bool_t pci_passthrough_enabled;
@@ -73,6 +75,9 @@ struct pci_ops {
                 uint32_t reg, uint32_t len, uint32_t *value);
     int (*write)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
                  uint32_t reg, uint32_t len, uint32_t value);
+    bool (*need_p2m_hwdom_mapping)(struct domain *d,
+                                   struct pci_host_bridge *bridge,
+                                   uint64_t addr);
 };
 
 /*
@@ -96,6 +101,9 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
                              uint32_t reg, uint32_t len, uint32_t value);
 void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge,
                                pci_sbdf_t sbdf, uint32_t where);
+bool pci_ecam_need_p2m_hwdom_mapping(struct domain *d,
+                                     struct pci_host_bridge *bridge,
+                                     uint64_t addr);
 struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus);
 int pci_get_host_bridge_segment(const struct dt_device_node *node,
                                 uint16_t *segment);
@@ -113,6 +121,8 @@ int pci_host_iterate_bridges(struct domain *d,
                                         struct pci_host_bridge *bridge));
 int pci_host_get_num_bridges(void);
 
+int pci_host_bridge_mappings(struct domain *d, p2m_type_t p2mt);
+
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 95da0b7ab9cd..5b30135fda38 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -2,6 +2,8 @@
 #define __ARM_SETUP_H_
 
 #include <public/version.h>
+#include <asm/p2m.h>
+#include <xen/device_tree.h>
 
 #define MIN_FDT_ALIGN 8
 #define MAX_FDT_SIZE SZ_2M
@@ -77,6 +79,14 @@ struct bootinfo {
 #endif
 };
 
+struct map_range_data
+{
+    struct domain *d;
+    p2m_type_t p2mt;
+    /* Set if mappings for PCI host bridges must not be skipped. */
+    bool skip_mapping;
+};
+
 extern struct bootinfo bootinfo;
 
 extern domid_t max_init_domid;
@@ -124,6 +134,9 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
 u32 device_tree_get_u32(const void *fdt, int node,
                         const char *prop_name, u32 dflt);
 
+int map_range_to_domain(const struct dt_device_node *dev,
+                        u64 addr, u64 len, void *data);
+
 #endif
 /*
  * Local variables:
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:17:11 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199438.353526 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJb-0001bA-Sf; Thu, 30 Sep 2021 07:17:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199438.353526; Thu, 30 Sep 2021 07:17:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJb-0001b1-OW; Thu, 30 Sep 2021 07:17:11 +0000
Received: by outflank-mailman (input) for mailman id 199438;
 Thu, 30 Sep 2021 07:17:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqH0-0004B3-BD
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:30 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 93ee8183-6f97-4d4d-abc4-45984e718a69;
 Thu, 30 Sep 2021 07:13:39 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id x27so21403290lfu.5
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:39 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:38 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93ee8183-6f97-4d4d-abc4-45984e718a69
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=kkjiAREPfgf5h7Wy+LaOiO2DkflhH8+fOSmHBO3xXoo=;
        b=n1p4J0xc5EcdU1E6lwrvdv3FPUzNKkurjTyMdCLcMChhvN3HWpV6H/wS+bys/5fzrD
         m8dZmv640qhzH1JBFUK/WGgaMIckyYxAD+g3zIu2xKQyrQ6pCqT6W8ih3CoiSrboZz0b
         BuiGlhmHlYc6MC2nXi8UUQAgUduMgeCQ7FjpbcBWUpb/pwp/tUl2PUg69rXnunj/9w42
         a6TNcmIiSyeRw/3L8y+Fh3+DLPAuybhQxQFk4HprAGlLnzluUf+IIZ834a80OyD2AljH
         w1zmf87nMP4V4oflrC0O5vhbSKyhkasge0M1y838mvE1dTQIL00//cZaMAdbPoRAiiDu
         tOEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=kkjiAREPfgf5h7Wy+LaOiO2DkflhH8+fOSmHBO3xXoo=;
        b=Nj7IImJ7D5XRQgjdxfW7tORkBzP9oDr+ruaPtC+orOn0T4LVLFRpU5BkD3drI8Sg9z
         +91rJC8DG4RcZIqc/F1v1F4P2CCamv6pKFpWJAccXJBgisJ043LthFrfIuiJaeq6qTDS
         YZF7mWxJi3IR7KiIr+mnqk2IgK0hvYTNSOr33RU9vj5Pbg35wW5qsUv69EQSnrQDune/
         ctbuhuX5JXurXRDwXYkZJHllxfrVnvrWpQV026ZzZC742ClZSay3HY5PHHsSR9Gr6xqq
         JlbHVVgS8i9J8ncWkLHn2UZJ3txfTweAjTHnbfeTSZldMMuwvBFJxhkTTdpK/k4sHNX6
         GyEg==
X-Gm-Message-State: AOAM532cerGvYL4CiOxxMSdXWz2TTt+2zd9eklDsPrWOjnJSNY5ZULnA
	5KEorH1gCMClr5B7je85vhoWjNTtkqo=
X-Google-Smtp-Source: ABdhPJyLQMp8dJHYSa+Pdww4/1nbcTrHpwgOpkBAqfeXXjEnwv9CdhpTtc4T4AAV0iwYluKZM/Jtlg==
X-Received: by 2002:a2e:8e90:: with SMTP id z16mr4165430ljk.513.1632986018612;
        Thu, 30 Sep 2021 00:13:38 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 11/11] xen/arm: Process pending vPCI map/unmap operations
Date: Thu, 30 Sep 2021 10:13:26 +0300
Message-Id: <20210930071326.857390-12-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

vPCI may map and unmap PCI device memory (BARs) being passed through which
may take a lot of time. For this those operations may be deferred to be
performed later, so that they can be safely preempted.

Currently this deferred processing is happening in common IOREQ code
which doesn't seem to be the right place for x86 and is even more
doubtful because IOREQ may not be enabled for Arm at all.
So, for Arm the pending vPCI work may have no chance to be executed
if the processing is left as is in the common IOREQ code only.
For that reason make vPCI processing happen in arch specific code.

Please be aware that there are a few outstanding TODOs affecting this
code path, see xen/drivers/vpci/header.c:map_range and
xen/drivers/vpci/header.c:vpci_process_pending.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
 - update commit message with more insight on x86, IOREQ and Arm
 - restored order of invocation for IOREQ and vPCI processing (Jan)
Since v1:
 - Moved the check for pending vpci work from the common IOREQ code
   to hvm_do_resume on x86
 - Re-worked the code for Arm to ensure we don't miss pending vPCI work
---
 xen/arch/arm/traps.c   | 13 +++++++++++++
 xen/arch/x86/hvm/hvm.c |  6 ++++++
 xen/common/ioreq.c     |  9 ---------
 3 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 219ab3c3fbde..b246f51086e3 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -34,6 +34,7 @@
 #include <xen/symbols.h>
 #include <xen/version.h>
 #include <xen/virtual_region.h>
+#include <xen/vpci.h>
 
 #include <public/sched.h>
 #include <public/xen.h>
@@ -2304,6 +2305,18 @@ static bool check_for_vcpu_work(void)
     }
 #endif
 
+    if ( has_vpci(v->domain) )
+    {
+        bool pending;
+
+        local_irq_enable();
+        pending = vpci_process_pending(v);
+        local_irq_disable();
+
+        if ( pending )
+            return true;
+    }
+
     if ( likely(!v->arch.need_flush_to_ram) )
         return false;
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7b48a1b925bb..38bf697c5b04 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -546,6 +546,12 @@ void hvm_do_resume(struct vcpu *v)
 
     pt_restore_timer(v);
 
+    if ( has_vpci(v->domain) && vpci_process_pending(v) )
+    {
+        raise_softirq(SCHEDULE_SOFTIRQ);
+        return;
+    }
+
     if ( !vcpu_ioreq_handle_completion(v) )
         return;
 
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index d732dc045df9..689d256544c8 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -25,9 +25,7 @@
 #include <xen/lib.h>
 #include <xen/paging.h>
 #include <xen/sched.h>
-#include <xen/softirq.h>
 #include <xen/trace.h>
-#include <xen/vpci.h>
 
 #include <asm/guest_atomics.h>
 #include <asm/ioreq.h>
@@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
 
 bool vcpu_ioreq_handle_completion(struct vcpu *v)
 {
-    struct domain *d = v->domain;
     struct vcpu_io *vio = &v->io;
     struct ioreq_server *s;
     struct ioreq_vcpu *sv;
     enum vio_completion completion;
     bool res = true;
 
-    if ( has_vpci(d) && vpci_process_pending(v) )
-    {
-        raise_softirq(SCHEDULE_SOFTIRQ);
-        return false;
-    }
-
     while ( (sv = get_pending_vcpu(v, &s)) != NULL )
         if ( !wait_for_io(sv, get_ioreq(s, v)) )
             return false;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:17:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:17:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199441.353537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJd-0001sK-4q; Thu, 30 Sep 2021 07:17:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199441.353537; Thu, 30 Sep 2021 07:17:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqJd-0001s7-1W; Thu, 30 Sep 2021 07:17:13 +0000
Received: by outflank-mailman (input) for mailman id 199441;
 Thu, 30 Sep 2021 07:17:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqGq-0004B3-Ak
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:14:20 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 68e97808-1059-4749-9c40-e1fa980222f6;
 Thu, 30 Sep 2021 07:13:37 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id z24so21167038lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:13:37 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id y3sm267835lfh.132.2021.09.30.00.13.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:13:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68e97808-1059-4749-9c40-e1fa980222f6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=ed0cb37zU/24QxEPfk3Zh0UOvwcZKTudBJfaTlZ2/uM=;
        b=HSe7JfAQdriC3ZOfjhvs1qHTJcCnMu5luam2DqethH9rst2ywFP4j5/KS/rE6tPfsC
         /9hXHbefKF40vX1HA9T5roPW8CTYykp093i//AZbnwYh75tzLf6iRhxI5DQQvyyb9VNY
         STaMAxJZlD8dtJUKCNPiG1zLg+SiQ9/bDO/6OCuDotfYs5kre+7ZP+9jeHKPqPJWb2EV
         ERcUbFEsA54oaWI9RPTSjugVcXnE6qIsNQXA+zBvYNvtBJa8qgqMSLsaNnBaRohugRdc
         wiS7jW66GBKPL0IBNav1MPBja4vX0/1X0PI99qEPzvWHRRlhrV8Kswzcig8IMnmGocoj
         9k6A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=ed0cb37zU/24QxEPfk3Zh0UOvwcZKTudBJfaTlZ2/uM=;
        b=MzpeClWebIA3NSHxtUcC4cTz6Noz70eB+9Ez0MAuF4qCx7lJGPJ5sa3mCsOd0sAeDs
         ME61teX5w91qTOkZY3Jb9wfDqlhjA/RB7WGackASzyPpBH5TgN4ytB8PYkUlgeVA778w
         b9J6UWz3KbHsC0un4olzOgmmCsEWpGJpdLUvcFh8hiRkHICUOrQ+ad7FxKCcPE9OicWO
         lO41HfU5xbcGkAYsxgTznazWwgHu71pEc/a+mHi3Exzly+F9kw5PfOrFGARF6Uv14H83
         HUuOJm0Ed97RMN03gfENJnnB+CfCagkSY4rbcqXj0zfoCO5k8qLC/XCJv8k3AvVFCEE7
         6IeQ==
X-Gm-Message-State: AOAM5312mryDk5ovubnxVuApDO4auDMysfHujui3+m+JAPka6RLu5VfF
	OHs9CDGBMcR76EshwH8HGBi13gNj5oY=
X-Google-Smtp-Source: ABdhPJwVw+tTaWEnG4vCJ2L8eQD0NQFZzHaVSmlxZ4kRzOcOMsOqFQ/APk7egIw+0tz53PGxMYTINw==
X-Received: by 2002:a19:8c0f:: with SMTP id o15mr4428022lfd.464.1632986016583;
        Thu, 30 Sep 2021 00:13:36 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 09/11] xen/arm: Setup MMIO range trap handlers for hardware domain
Date: Thu, 30 Sep 2021 10:13:24 +0300
Message-Id: <20210930071326.857390-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930071326.857390-1-andr2000@gmail.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

In order for vPCI to work it needs to maintain guest and hardware
domain's views of the configuration space. For example, BARs and
COMMAND registers require emulation for guests and the guest view
of the registers needs to be in sync with the real contents of the
relevant registers. For that ECAM address space needs to also be
trapped for the hardware domain, so we need to implement PCI host
bridge specific callbacks to properly setup MMIO handlers for those
ranges depending on particular host bridge implementation.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
- removed unneeded assignment (count = 0)
- removed unneeded header inclusion
- update commit message
Since v1:
 - Dynamically calculate the number of MMIO handlers required for vPCI
   and update the total number accordingly
 - s/clb/cb
 - Do not introduce a new callback for MMIO handler setup
---
 xen/arch/arm/domain.c              |  2 ++
 xen/arch/arm/pci/pci-host-common.c | 28 +++++++++++++++++++++++++
 xen/arch/arm/vpci.c                | 33 ++++++++++++++++++++++++++++++
 xen/arch/arm/vpci.h                |  6 ++++++
 xen/include/asm-arm/pci.h          |  5 +++++
 5 files changed, 74 insertions(+)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 79012bf77757..fa6fcc5e467c 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -733,6 +733,8 @@ int arch_domain_create(struct domain *d,
     if ( (rc = domain_vgic_register(d, &count)) != 0 )
         goto fail;
 
+    count += domain_vpci_get_num_mmio_handlers(d);
+
     if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 )
         goto fail;
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 592c01aae5bb..1eb4daa87365 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -292,6 +292,34 @@ struct dt_device_node *pci_find_host_bridge_node(struct device *dev)
     }
     return bridge->dt_node;
 }
+
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*cb)(struct domain *d,
+                                       struct pci_host_bridge *bridge))
+{
+    struct pci_host_bridge *bridge;
+    int err;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+    {
+        err = cb(d, bridge);
+        if ( err )
+            return err;
+    }
+    return 0;
+}
+
+int pci_host_get_num_bridges(void)
+{
+    struct pci_host_bridge *bridge;
+    int count = 0;
+
+    list_for_each_entry( bridge, &pci_host_bridges, node )
+        count++;
+
+    return count;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 76c12b92814f..5d6c29c8dcd9 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -80,17 +80,50 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
     .write = vpci_mmio_write,
 };
 
+static int vpci_setup_mmio_handler(struct domain *d,
+                                   struct pci_host_bridge *bridge)
+{
+    struct pci_config_window *cfg = bridge->cfg;
+
+    register_mmio_handler(d, &vpci_mmio_handler,
+                          cfg->phys_addr, cfg->size, NULL);
+    return 0;
+}
+
 int domain_vpci_init(struct domain *d)
 {
     if ( !has_vpci(d) )
         return 0;
 
+    if ( is_hardware_domain(d) )
+        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+
+    /* Guest domains use what is programmed in their device tree. */
     register_mmio_handler(d, &vpci_mmio_handler,
                           GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
 
     return 0;
 }
 
+int domain_vpci_get_num_mmio_handlers(struct domain *d)
+{
+    int count;
+
+    if ( is_hardware_domain(d) )
+        /* For each PCI host bridge's configuration space. */
+        count = pci_host_get_num_bridges();
+    else
+        /*
+         * VPCI_MSIX_MEM_NUM handlers for MSI-X tables per each PCI device
+         * being passed through. Maximum number of supported devices
+         * is 32 as virtual bus topology emulates the devices as embedded
+         * endpoints.
+         * +1 for a single emulated host bridge's configuration space. */
+        count = VPCI_MSIX_MEM_NUM * 32 + 1;
+
+    return count;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
index d8a7b0e3e802..27a2b069abd2 100644
--- a/xen/arch/arm/vpci.h
+++ b/xen/arch/arm/vpci.h
@@ -17,11 +17,17 @@
 
 #ifdef CONFIG_HAS_VPCI
 int domain_vpci_init(struct domain *d);
+int domain_vpci_get_num_mmio_handlers(struct domain *d);
 #else
 static inline int domain_vpci_init(struct domain *d)
 {
     return 0;
 }
+
+static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
+{
+    return 0;
+}
 #endif
 
 #endif /* __ARCH_ARM_VPCI_H__ */
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 566a9436a18e..0d0266b13cba 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -108,6 +108,11 @@ static always_inline bool is_pci_passthrough_enabled(void)
 
 void arch_pci_init_pdev(struct pci_dev *pdev);
 
+int pci_host_iterate_bridges(struct domain *d,
+                             int (*clb)(struct domain *d,
+                                        struct pci_host_bridge *bridge));
+int pci_host_get_num_bridges(void);
+
 #else   /*!CONFIG_HAS_PCI*/
 
 struct arch_pci_dev { };
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:30:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:30:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199469.353548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqWq-000515-Ek; Thu, 30 Sep 2021 07:30:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199469.353548; Thu, 30 Sep 2021 07:30:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqWq-00050y-9o; Thu, 30 Sep 2021 07:30:52 +0000
Received: by outflank-mailman (input) for mailman id 199469;
 Thu, 30 Sep 2021 07:30:50 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVqWo-00050s-9Q
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:30:50 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 54bf3852-21c0-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 07:30:48 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U7SXwc027172; 
 Thu, 30 Sep 2021 07:30:46 GMT
Received: from eur04-db3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bd76wgasx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 07:30:45 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR0302MB3427.eurprd03.prod.outlook.com (2603:10a6:208:4::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 07:30:39 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 07:30:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54bf3852-21c0-11ec-bd3d-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gtTrjFEHfAfQ96cceiw4NSYDi9CttEdgDZ3Iq4najymMWWgCAEkou5KJTWy9eq9ecRSRy1vAYpE2EV9BHSlgwdgusYb5c+j1iEEOIYyhznsbjBq2a+CtXiZ9H31h2HtNBC5AOWQg1/LlE4X8ZFK2so0/cgigdSCCl7RaV0aRKzs52+ZxGKqylDCYNsY3BoX7i1403SPQWwtW1kvBP2kUymLJIhdGETZflECK76e2fPm4kNfK0QSHfXluLjRh+7wN3zRijQx32Qe3iM4OWSjwP7W4ylsev+fKHF2QcOBOgfd07ZXPW3VW6EmLT7SJF7xfNXjMD6oOWHRXCzzBgtIv6w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ImI6GMwkmhyOCjOZ3l0uxg640UFEOsFxiiewcIAifCI=;
 b=V2yqF+wLXO31LCCNlhLRA1xh8Z/efSy8sFuFGUrp65G4f33ASzg+nhPEAHFvc+3zb2MVgK1TC9aEJKjTl9WrQz/In7Y6Q3PUUdA/r29yuIyqsdqKRAoASbAYOfG2tnmCKiL40wy/JwQ9dd2Zg2M6WrCu/Aaj4CEn+OAh9lS6j7L9DWyPl8yvEJEoCLNpG6otlZnKXbtQMiz0/5jFCXDEHciBK/gsM30EgIVG+NLgAE+Y1ZoNlhuhmCLxRQ3JpgHHsd0r+VS1cfy9yfMl3VtezadBF3Tg6sUd8iGZw4kul7EkfLctCak38JgsQ7+gd2dhWKQZn/EcrI0FKgax3NQTtA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ImI6GMwkmhyOCjOZ3l0uxg640UFEOsFxiiewcIAifCI=;
 b=RTTF7fsVmaQBIPoOKfR0/NlJRuzeb1AuwvmRlU7/zmGAG4yYkn7cm+q0op08bjtu97OYKrWEdFBoA1O/lDVXiDvXI5mO8Ms95K1UHBLnXL8ZiXfcF3ewqy/6ZLaBozaOIgMdTk4QVn9ajZzMJMfKJgzNwq9aZckr20lYzw4+EIgTo9X3WKaTI4JnaxrUC3hVeWRZvBPg3qagCAxAqaFIkoBsZX/i7XK9Jk0nAwKtngT/G2Yeahwh/Gg0wUox8EsZ7Zdsuz1Pb3zLONALOngOGvBRrUbDP2WmVMK8+8rErETFkUV4PBeGvGrSRLj8DFQi5XwdOzAFCUYUcBnsSNHraw==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: "jgross@suse.com" <jgross@suse.com>
CC: "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "julien@xen.org" <julien@xen.org>,
        "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        "jbeulich@suse.com" <jbeulich@suse.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        Anastasiia Lukianenko
	<Anastasiia_Lukianenko@epam.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v5] xen-pciback: allow compiling on other archs than x86
Thread-Topic: [PATCH v5] xen-pciback: allow compiling on other archs than x86
Thread-Index: AQHXtDtdCEW9atNFMk2qqaldf5A98Ku8MaoA
Date: Thu, 30 Sep 2021 07:30:39 +0000
Message-ID: <b7adb3d8-f965-6e9f-5701-48778b5e8beb@epam.com>
References: <20210928073501.433559-1-andr2000@gmail.com>
In-Reply-To: <20210928073501.433559-1-andr2000@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: oracle.com; dkim=none (message not signed)
 header.d=none;oracle.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 41c5ae73-8ef2-456e-e660-08d983e433e4
x-ms-traffictypediagnostic: AM0PR0302MB3427:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: 
 <AM0PR0302MB342770B38B207338E72B97FCE7AA9@AM0PR0302MB3427.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 bXgK8IYy8HQlBJp7g9Tq53L+gGUUzOsjUPVeK9c3HAtQgzKSTGNYgsOasoJ+MBbeay9JJE94QaPJ6DuOr/7OPfmbcuNbE7o7wHjlgNXa7qYKzMp419ZKVw8xdVW2Xz2szM9JmDaQsujYxo8X91xUk3YCOwrGg49/ZR+vBPhzVJnwpecQ1WlRUIwT0DmME7Nyr+9Db2KlOjUVL2jR92zC6PGGjNup6JRLxM3KqUCrGQ1gogNfkbte+PGQW7Ghz8YwhgxNy/x2TH7IrioQBZGMdAOxbaM4DDX+lVyGs53IRa9ElwyCvF72VipRMO8id/U9CxrbY5EDrtINA1kE05xVLPeZfS9CKLdH6+XFejj4/af7oVbd/oy5434mO92CJFcJB7CFcCiCUYvyFIFK2rtXBxR24GUwFWIUVOql8l6UIDA7gIGWwwwLU0uSv7sLPf3jqD8DHP4TFprORjPAhYZXIMDBOcfjaoCZWCg9w45DU+8kc4rkp15kLhOn/RT9NTtxvenahiTd1zbRX1OS1W/uIa36BitvoQSfBpKqThLs2kEGVVK2OGAir3Q+M+B4EMOwIvQdQW8OcCdcBPrBgMVZhb/d6SNvMCV1/k+2tg70ZguwTs6IRr3B9yE8xUvFL1Fbq4kK5kqM/HlqLcnZq0y+M5KdsElC8QCut3sLoomgxUKF9kvR2dlXBF7bIkPQjl0ZI7jtZJVugTREt5EYX1tEMRum1pkWdKkFk9PEv/Yhonm04cbMcBHyQ2kFmJJB77Yk
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(31686004)(5660300002)(6486002)(91956017)(30864003)(76116006)(31696002)(8936002)(66556008)(6506007)(508600001)(86362001)(38100700002)(4326008)(122000001)(26005)(316002)(2616005)(6916009)(36756003)(66476007)(8676002)(6512007)(71200400001)(83380400001)(64756008)(55236004)(66946007)(186003)(66446008)(53546011)(54906003)(38070700005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UVNjWk13dVp6NkxOMTluRCtBWG1OUjU1RHhmWFEzTjZGcWFydjRCUzFFTTR1?=
 =?utf-8?B?T2hHTHBFTTJvZEdGRktqV3E2RktNKzhOSmdCdEJva3MwQ2MvRS9KWTRDQzkz?=
 =?utf-8?B?bFZGNmZtZXVkS2s1ckEyY0FpTzdtOXlOY29kV0ZvYUptTGtyRG5BQUlqSlJX?=
 =?utf-8?B?QjZZT2g4b3VaYWVla2FTOVRFQnZqZjJpaEFOTm9KU0FMSlE5UFBzcXl4dlRW?=
 =?utf-8?B?TTBCdTRzOHFxeW8rY2F1Z0M1RGxYS2lYUEVwdU5YVlRreFkxMFpLZzVDMmJZ?=
 =?utf-8?B?eGo5ZXZ0VjV5MnFsc2VLY2dNcy9MQ1JscjRzSlBoRDU1YVl6dnVHR0JJQUlG?=
 =?utf-8?B?T2hiV013K3I0VWFsL1czaFdnVVpOckF5UUtIZ0RkUkdOTHZlVmZyODNqVE1i?=
 =?utf-8?B?dXFJZlo1ZTNaSm9ZSG9NSmwxYkRqQkVvT2Jkc3pXV1lrQnJ1WnN2ZXVZTWtz?=
 =?utf-8?B?UnhYcHVYbENoeVJkOWdsVUxvZEJGS0ZreE5ISEJQTXRDYjluVlBLM0hDRHNh?=
 =?utf-8?B?OGh4aXowa21UZUd6VmN6MjNSMG1Hb1JQUUxrL1ZDYkJLNHNPYy9ZbGVuVU5L?=
 =?utf-8?B?a0FCNFRUemhwOUdaRVRGRlpiVS96bDJ1NnBjbjYxQWtWbVlHS1RaMEJrV2VR?=
 =?utf-8?B?VWxSUkRuNDRKNzFNSVdaWnZ2OS9RWVFNVFdWc1pucVlzalB5b29yeUpnSlBJ?=
 =?utf-8?B?V3JSVzJQMHpUQXhnWEwvazRiRlpGQ0tWMFpxWHBqY2lCa0NGNFVlODc0b1E0?=
 =?utf-8?B?blFEOXZJcDI2UUg1NUNsYVZUZlVpK0t1bHJZNXo5TmZEUS9FYjF5VDV0Szlv?=
 =?utf-8?B?VWpqOU9MUWxRWUp3RkNLOGNaVVI4RkRSMzRWcGtaSDBuNHYraVdDOStFanhr?=
 =?utf-8?B?Zkh0aGxBd1V2WVBIQ25RMDloUk1JeFhIVUtndFRpWFd3QWxuVmovWkI4Zm5E?=
 =?utf-8?B?R0lKb2NjRkc4WUh6dUhHZzFyRnZ0eG03SlBnUi9OZVdiNi9oMklKcHQwc1lK?=
 =?utf-8?B?WGFJcW5DZEdWT29URDhPck1kdXpYNi9CQ1M5TmJPTW1EUnpETE03WWFaeXJW?=
 =?utf-8?B?R2U1bWJkT1RlL1M0cUNkQTRHdDVvK0lkZngyZ1E5dGdHa0QrREZvRVUxaWRG?=
 =?utf-8?B?YmViNnpKeG9ZUXE3NktzS2VBMDE4S1g4UWl4V2pBUEdpRWlRKzNPamJBS1Iy?=
 =?utf-8?B?NHpOSjNDREh1QXYxZVIvejBHTWpuTGdndHE2TUd4Z1I4YU9zRHovUzd6YW9t?=
 =?utf-8?B?U21VQlhqMmo2WC9LK2JTRHlLUVZLUC9WcWJGOTBVWHE0VGFjejdZSnY3aU9W?=
 =?utf-8?B?WXpqOVIxVGZCb3o3SUtiM09JdklvdFdFaHBNakc5N1Z6YnMrelVxRXVDS3FH?=
 =?utf-8?B?YlpPSUtNK1p1T2VqQ2FXUkJibUdhYkdxN0k3V0NVOFVkY0gwRG00RVVNSDY1?=
 =?utf-8?B?NWZrYXFPdzdpQ29RSTdpU0ZBakwxUjZ2N293OGh5bm50dEVyWEZadFRjUmJJ?=
 =?utf-8?B?bkxEbnA2aU44SkFUSVh5bkZwWnU2SVAydlIwSHlHRTFaSDdhT3EzWGVMUUg4?=
 =?utf-8?B?WU1TOVVPK0dQNHk5QWt5M3Zid1VOY2wzUUFwVVhPbFdMN2RybjhGN0tRV1VC?=
 =?utf-8?B?a3laMTB1RmY2T0pLN2ZYdU1keHVLOHlnZUZNQXpheERnaGg2ay9tQ2ZFaXl3?=
 =?utf-8?B?Zy9XZjNHdmtQQ1hDVFlRTDdwOFJ0M2xPWkhRMUFvM0VtcHlkZVZDbDBza2M0?=
 =?utf-8?B?WmM5TW5Ed044c1NzOHMzdGlpTXZBYVpia3RBeFNaSmZidE5CZHNCSy9sRXFj?=
 =?utf-8?B?Z2ROZE9MZWM3MU9xenZaUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D72E12AF83EB0048A3D2D80E2E829D5D@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 41c5ae73-8ef2-456e-e660-08d983e433e4
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 07:30:39.5141
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: MBv0uXAQTg0ytAiJETdhNfaw8OsUxhxM+8OTxmaR0AjRTQ1pqX1be0drD4R05cynvsbpyYd8SxlHzrf3T2M17QDrRuyUOkAv4bApW7zhWvZZMug1rYaZotJxYKk+fspf
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3427
X-Proofpoint-ORIG-GUID: aKHrYplwtYBFPdGK_wkhBC6eaThl9Tvj
X-Proofpoint-GUID: aKHrYplwtYBFPdGK_wkhBC6eaThl9Tvj
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_02,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 phishscore=0 bulkscore=0 adultscore=0 spamscore=0 clxscore=1015
 lowpriorityscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0
 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300044

SGksIEp1ZXJnZW4hDQoNCklzIGl0IGdvb2QgdG8gZ28gb3Igd2UgbmVlZCBzb21lIG90aGVyIGNo
YW5nZXM/DQoNClRoYW5rIHlvdSwNCk9sZWtzYW5kcg0KDQpPbiAyOC4wOS4yMSAxMDozNSwgT2xl
a3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+IEZyb206IE9sZWtzYW5kciBBbmRydXNoY2hl
bmtvIDxvbGVrc2FuZHJfYW5kcnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4NCj4gWGVuLXBjaWJhY2sg
ZHJpdmVyIHdhcyBkZXNpZ25lZCB0byBiZSBidWlsdCBmb3IgeDg2IG9ubHkuIEJ1dCBpdA0KPiBj
YW4gYWxzbyBiZSB1c2VkIGJ5IG90aGVyIGFyY2hpdGVjdHVyZXMsIGUuZy4gQXJtLg0KPg0KPiBD
dXJyZW50bHkgUENJIGJhY2tlbmQgaW1wbGVtZW50cyBtdWx0aXBsZSBmdW5jdGlvbmFsaXRpZXMg
YXQgYSB0aW1lLA0KPiBzdWNoIGFzOg0KPiAxLiBJdCBpcyB1c2VkIGFzIGEgZGF0YWJhc2UgZm9y
IGFzc2lnbmFibGUgUENJIGRldmljZXMsIGUuZy4geGwNCj4gICAgIHBjaS1hc3NpZ25hYmxlLXth
ZGR8cmVtb3ZlfGxpc3R9IG1hbmlwdWxhdGVzIHRoYXQgbGlzdC4gU28sIHdoZW5ldmVyDQo+ICAg
ICB0aGUgdG9vbHN0YWNrIG5lZWRzIHRvIGtub3cgd2hpY2ggUENJIGRldmljZXMgY2FuIGJlIHBh
c3NlZCB0aHJvdWdoDQo+ICAgICBpdCByZWFkcyB0aGF0IGZyb20gdGhlIHJlbGV2YW50IHN5c2Zz
IGVudHJpZXMgb2YgdGhlIHBjaWJhY2suDQo+IDIuIEl0IGlzIHVzZWQgdG8gaG9sZCB0aGUgdW5i
b3VuZCBQQ0kgZGV2aWNlcyBsaXN0LCBlLmcuIHdoZW4gcGFzc2luZw0KPiAgICAgdGhyb3VnaCBh
IFBDSSBkZXZpY2UgaXQgbmVlZHMgdG8gYmUgdW5ib3VuZCBmcm9tIHRoZSByZWxldmFudCBkZXZp
Y2UNCj4gICAgIGRyaXZlciBhbmQgYm91bmQgdG8gcGNpYmFjayAoc3RyaWN0bHkgc3BlYWtpbmcg
aXQgaXMgbm90IHJlcXVpcmVkDQo+ICAgICB0aGF0IHRoZSBkZXZpY2UgaXMgYm91bmQgdG8gcGNp
YmFjaywgYnV0IHBjaWJhY2sgaXMgYWdhaW4gdXNlZCBhcyBhDQo+ICAgICBkYXRhYmFzZSBvZiB0
aGUgcGFzc2VkIHRocm91Z2ggUENJIGRldmljZXMsIHNvIHdlIGNhbiByZS1iaW5kIHRoZQ0KPiAg
ICAgZGV2aWNlcyBiYWNrIHRvIHRoZWlyIG9yaWdpbmFsIGRyaXZlcnMgd2hlbiBndWVzdCBkb21h
aW4gc2h1dHMgZG93bikNCj4gMy4gRGV2aWNlIHJlc2V0IGZvciB0aGUgZGV2aWNlcyBiZWluZyBw
YXNzZWQgdGhyb3VnaA0KPiA0LiBQYXJhLXZpcnR1YWxpc2VkIHVzZS1jYXNlcyBzdXBwb3J0DQo+
DQo+IFRoZSBwYXJhLXZpcnR1YWxpc2VkIHBhcnQgb2YgdGhlIGRyaXZlciBpcyBub3QgYWx3YXlz
IG5lZWRlZCBhcyBzb21lDQo+IGFyY2hpdGVjdHVyZXMsIGUuZy4gQXJtIG9yIHg4NiBQVkggRG9t
MCwgYXJlIG5vdCB1c2luZyBiYWNrZW5kLWZyb250ZW5kDQo+IG1vZGVsIGZvciBQQ0kgZGV2aWNl
IHBhc3N0aHJvdWdoLg0KPg0KPiBGb3Igc3VjaCB1c2UtY2FzZXMgbWFrZSB0aGUgdmVyeSBmaXJz
dCBzdGVwIGluIHNwbGl0dGluZyB0aGUNCj4geGVuLXBjaWJhY2sgZHJpdmVyIGludG8gdHdvIHBh
cnRzOiBYZW4gUENJIHN0dWIgYW5kIFBDSSBQViBiYWNrZW5kDQo+IGRyaXZlcnMuDQo+DQo+IEZv
ciB0aGF0IGFkZCBuZXcgY29uZmlndXJhdGlvbiBvcHRpb25zIENPTkZJR19YRU5fUENJX1NUVUIg
YW5kDQo+IENPTkZJR19YRU5fUENJREVWX1NUVUIsIHNvIHRoZSBkcml2ZXIgY2FuIGJlIGxpbWl0
ZWQgaW4gaXRzDQo+IGZ1bmN0aW9uYWxpdHksIGUuZy4gbm8gc3VwcG9ydCBmb3IgcGFyYS12aXJ0
dWFsaXNlZCBzY2VuYXJpby4NCj4geDg2IHBsYXRmb3JtIHdpbGwgY29udGludWUgdXNpbmcgQ09O
RklHX1hFTl9QQ0lERVZfQkFDS0VORCBmb3IgdGhlDQo+IGZ1bGx5IGZlYXR1cmVkIGJhY2tlbmQg
ZHJpdmVyLg0KPg0KPiBQbGVhc2Ugbm90ZSwgdGhhdCBDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5E
IGFuZCBDT05GSUdfWEVOX1BDSURFVl9TVFVCDQo+IGFyZSBtdXR1YWxseSBleGNsdXNpdmUuDQo+
DQo+IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBBbmRydXNoY2hlbmtvIDxvbGVrc2FuZHJfYW5k
cnVzaGNoZW5rb0BlcGFtLmNvbT4NCj4gU2lnbmVkLW9mZi1ieTogQW5hc3Rhc2lpYSBMdWtpYW5l
bmtvIDxhbmFzdGFzaWlhX2x1a2lhbmVua29AZXBhbS5jb20+DQo+IFJldmlld2VkLWJ5OiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+IC0tLQ0KPiBDaGFuZ2Vz
IHNpbmNlIHY0Og0KPiAtIHNxdWFzaGVkIHR3byBwYXRjaGVzDQo+IC0gYWRkZWQgY29tbWVudHMg
dG8gTWFrZWZpbGUNCj4gQ2hhbmdlcyBzaW5jZSB2MzoNCj4gICAtIEZpeCA+PjMyIHNoaWZ0IGZv
ciAzMi1iaXQgYXJjaGl0ZWN0dXJlcw0KPiBDaGFuZ2VzIHNpbmNlIHYyOg0KPiAgIC0gc3dhcCB0
aGUgcGF0Y2ggb3JkZXINCj4gU2luY2UgdjE6DQo+ICAgLSBEbyBub3QgbW92ZSBwY2lfeGVuX2lu
aXRpYWxfZG9tYWluIGFzIGl0IGlzIHg4NiBzcGVjaWZpYw0KPiAtLS0NCj4gICBhcmNoL3g4Ni9p
bmNsdWRlL2FzbS94ZW4vcGNpLmggICAgICAgICAgICAgIHwgMTYgLS0tLS0NCj4gICBhcmNoL3g4
Ni9wY2kveGVuLmMgICAgICAgICAgICAgICAgICAgICAgICAgIHwgNzQgKy0tLS0tLS0tLS0tLS0t
LS0tLS0NCj4gICBkcml2ZXJzL3hlbi9LY29uZmlnICAgICAgICAgICAgICAgICAgICAgICAgIHwg
MjQgKysrKysrKw0KPiAgIGRyaXZlcnMveGVuL01ha2VmaWxlICAgICAgICAgICAgICAgICAgICAg
ICAgfCAgMiArLQ0KPiAgIGRyaXZlcnMveGVuL3BjaS5jICAgICAgICAgICAgICAgICAgICAgICAg
ICAgfCA3NSArKysrKysrKysrKysrKysrKysrKysNCj4gICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9NYWtlZmlsZSAgICAgICAgICAgIHwgIDcgKysNCj4gICBkcml2ZXJzL3hlbi94ZW4tcGNpYmFj
ay9jb25mX3NwYWNlX2hlYWRlci5jIHwgIDggKystDQo+ICAgZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2svcGNpX3N0dWIuYyAgICAgICAgICB8ICAzICstDQo+ICAgZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2svcGNpYmFjay5oICAgICAgICAgICB8ICA1ICsrDQo+ICAgZHJpdmVycy94ZW4veGVuLXBjaWJh
Y2sveGVuYnVzLmMgICAgICAgICAgICB8ICA4ICsrLQ0KPiAgIGluY2x1ZGUveGVuL3BjaS5oICAg
ICAgICAgICAgICAgICAgICAgICAgICAgfCAyOCArKysrKysrKw0KPiAgIDExIGZpbGVzIGNoYW5n
ZWQsIDE1NSBpbnNlcnRpb25zKCspLCA5NSBkZWxldGlvbnMoLSkNCj4gICBjcmVhdGUgbW9kZSAx
MDA2NDQgaW5jbHVkZS94ZW4vcGNpLmgNCj4NCj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L2luY2x1
ZGUvYXNtL3hlbi9wY2kuaCBiL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2kuaA0KPiBpbmRl
eCAzNTA2ZDhjNTk4YzEuLjRiMDhjMmI4MDllYSAxMDA2NDQNCj4gLS0tIGEvYXJjaC94ODYvaW5j
bHVkZS9hc20veGVuL3BjaS5oDQo+ICsrKyBiL2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9wY2ku
aA0KPiBAQCAtMTYsMjcgKzE2LDExIEBAIHN0YXRpYyBpbmxpbmUgaW50IHBjaV94ZW5faHZtX2lu
aXQodm9pZCkNCj4gICAjZW5kaWYNCj4gICAjaWYgZGVmaW5lZChDT05GSUdfWEVOX0RPTTApDQo+
ICAgaW50IF9faW5pdCBwY2lfeGVuX2luaXRpYWxfZG9tYWluKHZvaWQpOw0KPiAtaW50IHhlbl9m
aW5kX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldik7DQo+IC1pbnQgeGVu
X3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldiwgdWludDE2
X3QgZG9tYWluKTsNCj4gLWludCB4ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0
cnVjdCBwY2lfZGV2ICpkZXYpOw0KPiAgICNlbHNlDQo+ICAgc3RhdGljIGlubGluZSBpbnQgX19p
bml0IHBjaV94ZW5faW5pdGlhbF9kb21haW4odm9pZCkNCj4gICB7DQo+ICAgCXJldHVybiAtMTsN
Cj4gICB9DQo+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVy
KHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+IC17DQo+IC0JcmV0dXJuIC0xOw0KPiAtfQ0KPiAtc3Rh
dGljIGlubGluZSBpbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBj
aV9kZXYgKmRldiwNCj4gLQkJCQkJCSAgIHVpbnQxNl90IGRvbWFpbikNCj4gLXsNCj4gLQlyZXR1
cm4gLTE7DQo+IC19DQo+IC1zdGF0aWMgaW5saW5lIGludCB4ZW5fdW5yZWdpc3Rlcl9kZXZpY2Vf
ZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+IC17DQo+IC0JcmV0dXJuIC0xOw0K
PiAtfQ0KPiAgICNlbmRpZg0KPiAgIA0KPiAgICNpZiBkZWZpbmVkKENPTkZJR19QQ0lfTVNJKQ0K
PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYvcGNpL3hlbi5jIGIvYXJjaC94ODYvcGNpL3hlbi5jDQo+
IGluZGV4IDNkNDFhMDljMmMxNC4uNGE0NWIwYmY5YWU0IDEwMDY0NA0KPiAtLS0gYS9hcmNoL3g4
Ni9wY2kveGVuLmMNCj4gKysrIGIvYXJjaC94ODYvcGNpL3hlbi5jDQo+IEBAIC0yMyw2ICsyMyw3
IEBADQo+ICAgDQo+ICAgI2luY2x1ZGUgPHhlbi9mZWF0dXJlcy5oPg0KPiAgICNpbmNsdWRlIDx4
ZW4vZXZlbnRzLmg+DQo+ICsjaW5jbHVkZSA8eGVuL3BjaS5oPg0KPiAgICNpbmNsdWRlIDxhc20v
eGVuL3BjaS5oPg0KPiAgICNpbmNsdWRlIDxhc20veGVuL2NwdWlkLmg+DQo+ICAgI2luY2x1ZGUg
PGFzbS9hcGljLmg+DQo+IEBAIC01ODMsNzcgKzU4NCw0IEBAIGludCBfX2luaXQgcGNpX3hlbl9p
bml0aWFsX2RvbWFpbih2b2lkKQ0KPiAgIAl9DQo+ICAgCXJldHVybiAwOw0KPiAgIH0NCj4gLQ0K
PiAtc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyIHsNCj4gLQlkb21pZF90IGRvbWFpbjsN
Cj4gLQlzdHJ1Y3QgcGNpX2RldiAqZGV2Ow0KPiAtCXN0cnVjdCBsaXN0X2hlYWQgbGlzdDsNCj4g
LX07DQo+IC0NCj4gLXN0YXRpYyBERUZJTkVfU1BJTkxPQ0soZGV2X2RvbWFpbl9saXN0X3NwaW5s
b2NrKTsNCj4gLXN0YXRpYyBzdHJ1Y3QgbGlzdF9oZWFkIGRldl9kb21haW5fbGlzdCA9IExJU1Rf
SEVBRF9JTklUKGRldl9kb21haW5fbGlzdCk7DQo+IC0NCj4gLXN0YXRpYyBzdHJ1Y3QgeGVuX2Rl
dmljZV9kb21haW5fb3duZXIgKmZpbmRfZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+IC17
DQo+IC0Jc3RydWN0IHhlbl9kZXZpY2VfZG9tYWluX293bmVyICpvd25lcjsNCj4gLQ0KPiAtCWxp
c3RfZm9yX2VhY2hfZW50cnkob3duZXIsICZkZXZfZG9tYWluX2xpc3QsIGxpc3QpIHsNCj4gLQkJ
aWYgKG93bmVyLT5kZXYgPT0gZGV2KQ0KPiAtCQkJcmV0dXJuIG93bmVyOw0KPiAtCX0NCj4gLQly
ZXR1cm4gTlVMTDsNCj4gLX0NCj4gLQ0KPiAtaW50IHhlbl9maW5kX2RldmljZV9kb21haW5fb3du
ZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4gLXsNCj4gLQlzdHJ1Y3QgeGVuX2RldmljZV9kb21h
aW5fb3duZXIgKm93bmVyOw0KPiAtCWludCBkb21haW4gPSAtRU5PREVWOw0KPiAtDQo+IC0Jc3Bp
bl9sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPiAtCW93bmVyID0gZmluZF9kZXZp
Y2UoZGV2KTsNCj4gLQlpZiAob3duZXIpDQo+IC0JCWRvbWFpbiA9IG93bmVyLT5kb21haW47DQo+
IC0Jc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+IC0JcmV0dXJuIGRv
bWFpbjsNCj4gLX0NCj4gLUVYUE9SVF9TWU1CT0xfR1BMKHhlbl9maW5kX2RldmljZV9kb21haW5f
b3duZXIpOw0KPiAtDQo+IC1pbnQgeGVuX3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIoc3Ry
dWN0IHBjaV9kZXYgKmRldiwgdWludDE2X3QgZG9tYWluKQ0KPiAtew0KPiAtCXN0cnVjdCB4ZW5f
ZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+IC0NCj4gLQlvd25lciA9IGt6YWxsb2Moc2l6
ZW9mKHN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciksIEdGUF9LRVJORUwpOw0KPiAtCWlm
ICghb3duZXIpDQo+IC0JCXJldHVybiAtRU5PREVWOw0KPiAtDQo+IC0Jc3Bpbl9sb2NrKCZkZXZf
ZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPiAtCWlmIChmaW5kX2RldmljZShkZXYpKSB7DQo+IC0J
CXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPiAtCQlrZnJlZShvd25l
cik7DQo+IC0JCXJldHVybiAtRUVYSVNUOw0KPiAtCX0NCj4gLQlvd25lci0+ZG9tYWluID0gZG9t
YWluOw0KPiAtCW93bmVyLT5kZXYgPSBkZXY7DQo+IC0JbGlzdF9hZGRfdGFpbCgmb3duZXItPmxp
c3QsICZkZXZfZG9tYWluX2xpc3QpOw0KPiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rf
c3BpbmxvY2spOw0KPiAtCXJldHVybiAwOw0KPiAtfQ0KPiAtRVhQT1JUX1NZTUJPTF9HUEwoeGVu
X3JlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIpOw0KPiAtDQo+IC1pbnQgeGVuX3VucmVnaXN0
ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPiAtew0KPiAtCXN0
cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+IC0NCj4gLQlzcGluX2xvY2so
JmRldl9kb21haW5fbGlzdF9zcGlubG9jayk7DQo+IC0Jb3duZXIgPSBmaW5kX2RldmljZShkZXYp
Ow0KPiAtCWlmICghb3duZXIpIHsNCj4gLQkJc3Bpbl91bmxvY2soJmRldl9kb21haW5fbGlzdF9z
cGlubG9jayk7DQo+IC0JCXJldHVybiAtRU5PREVWOw0KPiAtCX0NCj4gLQlsaXN0X2RlbCgmb3du
ZXItPmxpc3QpOw0KPiAtCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0K
PiAtCWtmcmVlKG93bmVyKTsNCj4gLQlyZXR1cm4gMDsNCj4gLX0NCj4gLUVYUE9SVF9TWU1CT0xf
R1BMKHhlbl91bnJlZ2lzdGVyX2RldmljZV9kb21haW5fb3duZXIpOw0KPiAgICNlbmRpZg0KPiBk
aWZmIC0tZ2l0IGEvZHJpdmVycy94ZW4vS2NvbmZpZyBiL2RyaXZlcnMveGVuL0tjb25maWcNCj4g
aW5kZXggMjJmNWFmZjBjMTM2Li43NWI5MzMwZGM2ZmQgMTAwNjQ0DQo+IC0tLSBhL2RyaXZlcnMv
eGVuL0tjb25maWcNCj4gKysrIGIvZHJpdmVycy94ZW4vS2NvbmZpZw0KPiBAQCAtMTgxLDEwICsx
ODEsMzQgQEAgY29uZmlnIFNXSU9UTEJfWEVODQo+ICAgCXNlbGVjdCBETUFfT1BTDQo+ICAgCXNl
bGVjdCBTV0lPVExCDQo+ICAgDQo+ICtjb25maWcgWEVOX1BDSV9TVFVCDQo+ICsJYm9vbA0KPiAr
DQo+ICtjb25maWcgWEVOX1BDSURFVl9TVFVCDQo+ICsJdHJpc3RhdGUgIlhlbiBQQ0ktZGV2aWNl
IHN0dWIgZHJpdmVyIg0KPiArCWRlcGVuZHMgb24gUENJICYmICFYODYgJiYgWEVODQo+ICsJZGVw
ZW5kcyBvbiBYRU5fQkFDS0VORA0KPiArCXNlbGVjdCBYRU5fUENJX1NUVUINCj4gKwlkZWZhdWx0
IG0NCj4gKwloZWxwDQo+ICsJICBUaGUgUENJIGRldmljZSBzdHViIGRyaXZlciBwcm92aWRlcyBs
aW1pdGVkIHZlcnNpb24gb2YgdGhlIFBDSQ0KPiArCSAgZGV2aWNlIGJhY2tlbmQgZHJpdmVyIHdp
dGhvdXQgcGFyYS12aXJ0dWFsaXplZCBzdXBwb3J0IGZvciBndWVzdHMuDQo+ICsJICBJZiB5b3Ug
c2VsZWN0IHRoaXMgdG8gYmUgYSBtb2R1bGUsIHlvdSB3aWxsIG5lZWQgdG8gbWFrZSBzdXJlIG5v
DQo+ICsJICBvdGhlciBkcml2ZXIgaGFzIGJvdW5kIHRvIHRoZSBkZXZpY2UocykgeW91IHdhbnQg
dG8gbWFrZSB2aXNpYmxlIHRvDQo+ICsJICBvdGhlciBndWVzdHMuDQo+ICsNCj4gKwkgIFRoZSAi
aGlkZSIgcGFyYW1ldGVyIChvbmx5IGFwcGxpY2FibGUgaWYgYmFja2VuZCBkcml2ZXIgaXMgY29t
cGlsZWQNCj4gKwkgIGludG8gdGhlIGtlcm5lbCkgYWxsb3dzIHlvdSB0byBiaW5kIHRoZSBQQ0kg
ZGV2aWNlcyB0byB0aGlzIG1vZHVsZQ0KPiArCSAgZnJvbSB0aGUgZGVmYXVsdCBkZXZpY2UgZHJp
dmVycy4gVGhlIGFyZ3VtZW50IGlzIHRoZSBsaXN0IG9mIFBDSSBCREZzOg0KPiArCSAgeGVuLXBj
aWJhY2suaGlkZT0oMDM6MDAuMCkoMDQ6MDAuMCkNCj4gKw0KPiArCSAgSWYgaW4gZG91YnQsIHNh
eSBtLg0KPiArDQo+ICAgY29uZmlnIFhFTl9QQ0lERVZfQkFDS0VORA0KPiAgIAl0cmlzdGF0ZSAi
WGVuIFBDSS1kZXZpY2UgYmFja2VuZCBkcml2ZXIiDQo+ICAgCWRlcGVuZHMgb24gUENJICYmIFg4
NiAmJiBYRU4NCj4gICAJZGVwZW5kcyBvbiBYRU5fQkFDS0VORA0KPiArCXNlbGVjdCBYRU5fUENJ
X1NUVUINCj4gICAJZGVmYXVsdCBtDQo+ICAgCWhlbHANCj4gICAJICBUaGUgUENJIGRldmljZSBi
YWNrZW5kIGRyaXZlciBhbGxvd3MgdGhlIGtlcm5lbCB0byBleHBvcnQgYXJiaXRyYXJ5DQo+IGRp
ZmYgLS1naXQgYS9kcml2ZXJzL3hlbi9NYWtlZmlsZSBiL2RyaXZlcnMveGVuL01ha2VmaWxlDQo+
IGluZGV4IDM0MzQ1OTM0NTViMi4uNWFhZTY2ZTYzOGE3IDEwMDY0NA0KPiAtLS0gYS9kcml2ZXJz
L3hlbi9NYWtlZmlsZQ0KPiArKysgYi9kcml2ZXJzL3hlbi9NYWtlZmlsZQ0KPiBAQCAtMjQsNyAr
MjQsNyBAQCBvYmotJChDT05GSUdfWEVOX1NZU19IWVBFUlZJU09SKQkrPSBzeXMtaHlwZXJ2aXNv
ci5vDQo+ICAgb2JqLSQoQ09ORklHX1hFTl9QVkhWTV9HVUVTVCkJCSs9IHBsYXRmb3JtLXBjaS5v
DQo+ICAgb2JqLSQoQ09ORklHX1NXSU9UTEJfWEVOKQkJKz0gc3dpb3RsYi14ZW4ubw0KPiAgIG9i
ai0kKENPTkZJR19YRU5fTUNFX0xPRykJCSs9IG1jZWxvZy5vDQo+IC1vYmotJChDT05GSUdfWEVO
X1BDSURFVl9CQUNLRU5EKQkrPSB4ZW4tcGNpYmFjay8NCj4gK29iai0kKENPTkZJR19YRU5fUENJ
X1NUVUIpCSAgICAgICAgKz0geGVuLXBjaWJhY2svDQo+ICAgb2JqLSQoQ09ORklHX1hFTl9QUklW
Q01EKQkJKz0geGVuLXByaXZjbWQubw0KPiAgIG9iai0kKENPTkZJR19YRU5fQUNQSV9QUk9DRVNT
T1IpCSs9IHhlbi1hY3BpLXByb2Nlc3Nvci5vDQo+ICAgb2JqLSQoQ09ORklHX1hFTl9FRkkpCQkJ
Kz0gZWZpLm8NCj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL3BjaS5jIGIvZHJpdmVycy94ZW4v
cGNpLmMNCj4gaW5kZXggMjI0ZGYwM2NlNDJlLi5mYzhjMTI0OWQ0OWYgMTAwNjQ0DQo+IC0tLSBh
L2RyaXZlcnMveGVuL3BjaS5jDQo+ICsrKyBiL2RyaXZlcnMveGVuL3BjaS5jDQo+IEBAIC0yNTQs
MyArMjU0LDc4IEBAIHN0YXRpYyBpbnQgeGVuX21jZmdfbGF0ZSh2b2lkKQ0KPiAgIAlyZXR1cm4g
MDsNCj4gICB9DQo+ICAgI2VuZGlmDQo+ICsNCj4gKyNpZmRlZiBDT05GSUdfWEVOX0RPTTANCj4g
K3N0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciB7DQo+ICsJZG9taWRfdCBkb21haW47DQo+
ICsJc3RydWN0IHBjaV9kZXYgKmRldjsNCj4gKwlzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7DQo+ICt9
Ow0KPiArDQo+ICtzdGF0aWMgREVGSU5FX1NQSU5MT0NLKGRldl9kb21haW5fbGlzdF9zcGlubG9j
ayk7DQo+ICtzdGF0aWMgc3RydWN0IGxpc3RfaGVhZCBkZXZfZG9tYWluX2xpc3QgPSBMSVNUX0hF
QURfSU5JVChkZXZfZG9tYWluX2xpc3QpOw0KPiArDQo+ICtzdGF0aWMgc3RydWN0IHhlbl9kZXZp
Y2VfZG9tYWluX293bmVyICpmaW5kX2RldmljZShzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPiArew0K
PiArCXN0cnVjdCB4ZW5fZGV2aWNlX2RvbWFpbl9vd25lciAqb3duZXI7DQo+ICsNCj4gKwlsaXN0
X2Zvcl9lYWNoX2VudHJ5KG93bmVyLCAmZGV2X2RvbWFpbl9saXN0LCBsaXN0KSB7DQo+ICsJCWlm
IChvd25lci0+ZGV2ID09IGRldikNCj4gKwkJCXJldHVybiBvd25lcjsNCj4gKwl9DQo+ICsJcmV0
dXJuIE5VTEw7DQo+ICt9DQo+ICsNCj4gK2ludCB4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293bmVy
KHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+ICt7DQo+ICsJc3RydWN0IHhlbl9kZXZpY2VfZG9tYWlu
X293bmVyICpvd25lcjsNCj4gKwlpbnQgZG9tYWluID0gLUVOT0RFVjsNCj4gKw0KPiArCXNwaW5f
bG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4gKwlvd25lciA9IGZpbmRfZGV2aWNl
KGRldik7DQo+ICsJaWYgKG93bmVyKQ0KPiArCQlkb21haW4gPSBvd25lci0+ZG9tYWluOw0KPiAr
CXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPiArCXJldHVybiBkb21h
aW47DQo+ICt9DQo+ICtFWFBPUlRfU1lNQk9MX0dQTCh4ZW5fZmluZF9kZXZpY2VfZG9tYWluX293
bmVyKTsNCj4gKw0KPiAraW50IHhlbl9yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVj
dCBwY2lfZGV2ICpkZXYsIHVpbnQxNl90IGRvbWFpbikNCj4gK3sNCj4gKwlzdHJ1Y3QgeGVuX2Rl
dmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPiArDQo+ICsJb3duZXIgPSBremFsbG9jKHNpemVv
ZihzdHJ1Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIpLCBHRlBfS0VSTkVMKTsNCj4gKwlpZiAo
IW93bmVyKQ0KPiArCQlyZXR1cm4gLUVOT0RFVjsNCj4gKw0KPiArCXNwaW5fbG9jaygmZGV2X2Rv
bWFpbl9saXN0X3NwaW5sb2NrKTsNCj4gKwlpZiAoZmluZF9kZXZpY2UoZGV2KSkgew0KPiArCQlz
cGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4gKwkJa2ZyZWUob3duZXIp
Ow0KPiArCQlyZXR1cm4gLUVFWElTVDsNCj4gKwl9DQo+ICsJb3duZXItPmRvbWFpbiA9IGRvbWFp
bjsNCj4gKwlvd25lci0+ZGV2ID0gZGV2Ow0KPiArCWxpc3RfYWRkX3RhaWwoJm93bmVyLT5saXN0
LCAmZGV2X2RvbWFpbl9saXN0KTsNCj4gKwlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3Nw
aW5sb2NrKTsNCj4gKwlyZXR1cm4gMDsNCj4gK30NCj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhlbl9y
ZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4gKw0KPiAraW50IHhlbl91bnJlZ2lzdGVy
X2RldmljZV9kb21haW5fb3duZXIoc3RydWN0IHBjaV9kZXYgKmRldikNCj4gK3sNCj4gKwlzdHJ1
Y3QgeGVuX2RldmljZV9kb21haW5fb3duZXIgKm93bmVyOw0KPiArDQo+ICsJc3Bpbl9sb2NrKCZk
ZXZfZG9tYWluX2xpc3Rfc3BpbmxvY2spOw0KPiArCW93bmVyID0gZmluZF9kZXZpY2UoZGV2KTsN
Cj4gKwlpZiAoIW93bmVyKSB7DQo+ICsJCXNwaW5fdW5sb2NrKCZkZXZfZG9tYWluX2xpc3Rfc3Bp
bmxvY2spOw0KPiArCQlyZXR1cm4gLUVOT0RFVjsNCj4gKwl9DQo+ICsJbGlzdF9kZWwoJm93bmVy
LT5saXN0KTsNCj4gKwlzcGluX3VubG9jaygmZGV2X2RvbWFpbl9saXN0X3NwaW5sb2NrKTsNCj4g
KwlrZnJlZShvd25lcik7DQo+ICsJcmV0dXJuIDA7DQo+ICt9DQo+ICtFWFBPUlRfU1lNQk9MX0dQ
TCh4ZW5fdW5yZWdpc3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKTsNCj4gKyNlbmRpZg0KPiBkaWZm
IC0tZ2l0IGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUgYi9kcml2ZXJzL3hlbi94
ZW4tcGNpYmFjay9NYWtlZmlsZQ0KPiBpbmRleCBlOGQ5ODFkNDMyMzUuLmQ2M2RmMDlkZTgxYyAx
MDA2NDQNCj4gLS0tIGEvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4gKysrIGIv
ZHJpdmVycy94ZW4veGVuLXBjaWJhY2svTWFrZWZpbGUNCj4gQEAgLTEsNSArMSwxMiBAQA0KPiAg
ICMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjANCj4gKw0KPiArIyBOLkIuIFRoZSBi
ZWxvdyBjYW5ub3QgYmUgZXhwcmVzc2VkIHdpdGggYSBzaW5nbGUgbGluZSB1c2luZw0KPiArIyBD
T05GSUdfWEVOX1BDSV9TVFVCIGFzIGl0IGFsd2F5cyByZW1haW5zIGluICJ5IiBzdGF0ZSwNCj4g
KyMgdGh1cyBwcmV2ZW50aW5nIHRoZSBkcml2ZXIgdG8gYmUgYnVpbHQgYXMgYSBtb2R1bGUuDQo+
ICsjIFBsZWFzZSBub3RlLCB0aGF0IENPTkZJR19YRU5fUENJREVWX0JBQ0tFTkQgYW5kDQo+ICsj
IENPTkZJR19YRU5fUENJREVWX1NUVUIgYXJlIG11dHVhbGx5IGV4Y2x1c2l2ZS4NCj4gICBvYmot
JChDT05GSUdfWEVOX1BDSURFVl9CQUNLRU5EKSArPSB4ZW4tcGNpYmFjay5vDQo+ICtvYmotJChD
T05GSUdfWEVOX1BDSURFVl9TVFVCKSArPSB4ZW4tcGNpYmFjay5vDQo+ICAgDQo+ICAgeGVuLXBj
aWJhY2steSA6PSBwY2lfc3R1Yi5vIHBjaWJhY2tfb3BzLm8geGVuYnVzLm8NCj4gICB4ZW4tcGNp
YmFjay15ICs9IGNvbmZfc3BhY2UubyBjb25mX3NwYWNlX2hlYWRlci5vIFwNCj4gZGlmZiAtLWdp
dCBhL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL2NvbmZfc3BhY2VfaGVhZGVyLmMgYi9kcml2ZXJz
L3hlbi94ZW4tcGNpYmFjay9jb25mX3NwYWNlX2hlYWRlci5jDQo+IGluZGV4IGFjNDVjZGMzOGU4
NS4uOTgxNDM1MTAzYWYxIDEwMDY0NA0KPiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9j
b25mX3NwYWNlX2hlYWRlci5jDQo+ICsrKyBiL2RyaXZlcnMveGVuL3hlbi1wY2liYWNrL2NvbmZf
c3BhY2VfaGVhZGVyLmMNCj4gQEAgLTIzNiw4ICsyMzYsMTIgQEAgc3RhdGljIHZvaWQgKmJhcl9p
bml0KHN0cnVjdCBwY2lfZGV2ICpkZXYsIGludCBvZmZzZXQpDQo+ICAgCWVsc2Ugew0KPiAgIAkJ
cG9zID0gKG9mZnNldCAtIFBDSV9CQVNFX0FERFJFU1NfMCkgLyA0Ow0KPiAgIAkJaWYgKHBvcyAm
JiAocmVzW3BvcyAtIDFdLmZsYWdzICYgSU9SRVNPVVJDRV9NRU1fNjQpKSB7DQo+IC0JCQliYXIt
PnZhbCA9IHJlc1twb3MgLSAxXS5zdGFydCA+PiAzMjsNCj4gLQkJCWJhci0+bGVuX3ZhbCA9IC1y
ZXNvdXJjZV9zaXplKCZyZXNbcG9zIC0gMV0pID4+IDMyOw0KPiArCQkJLyoNCj4gKwkJCSAqIFVz
ZSAiPj4gMTYgPj4gMTYiIGluc3RlYWQgb2YgZGlyZWN0ICI+PiAzMiIgc2hpZnQNCj4gKwkJCSAq
IHRvIGF2b2lkIHdhcm5pbmdzIG9uIDMyLWJpdCBhcmNoaXRlY3R1cmVzLg0KPiArCQkJICovDQo+
ICsJCQliYXItPnZhbCA9IHJlc1twb3MgLSAxXS5zdGFydCA+PiAxNiA+PiAxNjsNCj4gKwkJCWJh
ci0+bGVuX3ZhbCA9IC1yZXNvdXJjZV9zaXplKCZyZXNbcG9zIC0gMV0pID4+IDE2ID4+IDE2Ow0K
PiAgIAkJCXJldHVybiBiYXI7DQo+ICAgCQl9DQo+ICAgCX0NCj4gZGlmZiAtLWdpdCBhL2RyaXZl
cnMveGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9w
Y2lfc3R1Yi5jDQo+IGluZGV4IGY4ZTRmYWE5NmFkNi4uYmJhNTI3NjIwNTA3IDEwMDY0NA0KPiAt
LS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5jDQo+ICsrKyBiL2RyaXZlcnMv
eGVuL3hlbi1wY2liYWNrL3BjaV9zdHViLmMNCj4gQEAgLTE5LDcgKzE5LDggQEANCj4gICAjaW5j
bHVkZSA8bGludXgvc2NoZWQuaD4NCj4gICAjaW5jbHVkZSA8bGludXgvYXRvbWljLmg+DQo+ICAg
I2luY2x1ZGUgPHhlbi9ldmVudHMuaD4NCj4gLSNpbmNsdWRlIDxhc20veGVuL3BjaS5oPg0KPiAr
I2luY2x1ZGUgPHhlbi9wY2kuaD4NCj4gKyNpbmNsdWRlIDx4ZW4veGVuLmg+DQo+ICAgI2luY2x1
ZGUgPGFzbS94ZW4vaHlwZXJ2aXNvci5oPg0KPiAgICNpbmNsdWRlIDx4ZW4vaW50ZXJmYWNlL3Bo
eXNkZXYuaD4NCj4gICAjaW5jbHVkZSAicGNpYmFjay5oIg0KPiBkaWZmIC0tZ2l0IGEvZHJpdmVy
cy94ZW4veGVuLXBjaWJhY2svcGNpYmFjay5oIGIvZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNp
YmFjay5oDQo+IGluZGV4IDk1ZTI4ZWU0OGQ1Mi4uOWE2NDE5NmU4MzFkIDEwMDY0NA0KPiAtLS0g
YS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2liYWNrLmgNCj4gKysrIGIvZHJpdmVycy94ZW4v
eGVuLXBjaWJhY2svcGNpYmFjay5oDQo+IEBAIC03MSw2ICs3MSwxMSBAQCBzdHJ1Y3QgcGNpX2Rl
diAqcGNpc3R1Yl9nZXRfcGNpX2RldihzdHJ1Y3QgeGVuX3BjaWJrX2RldmljZSAqcGRldiwNCj4g
ICAJCQkJICAgIHN0cnVjdCBwY2lfZGV2ICpkZXYpOw0KPiAgIHZvaWQgcGNpc3R1Yl9wdXRfcGNp
X2RldihzdHJ1Y3QgcGNpX2RldiAqZGV2KTsNCj4gICANCj4gK3N0YXRpYyBpbmxpbmUgYm9vbCB4
ZW5fcGNpYmtfcHZfc3VwcG9ydCh2b2lkKQ0KPiArew0KPiArCXJldHVybiBJU19FTkFCTEVEKENP
TkZJR19YRU5fUENJREVWX0JBQ0tFTkQpOw0KPiArfQ0KPiArDQo+ICAgLyogRW5zdXJlIGEgZGV2
aWNlIGlzIHR1cm5lZCBvZmYgb3IgcmVzZXQgKi8NCj4gICB2b2lkIHhlbl9wY2lia19yZXNldF9k
ZXZpY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYpOw0KPiAgIA0KPiBkaWZmIC0tZ2l0IGEvZHJpdmVy
cy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay94ZW5i
dXMuYw0KPiBpbmRleCBjMDljN2ViZDY5NjguLmJkZTYzZWY2NzdiOCAxMDA2NDQNCj4gLS0tIGEv
ZHJpdmVycy94ZW4veGVuLXBjaWJhY2sveGVuYnVzLmMNCj4gKysrIGIvZHJpdmVycy94ZW4veGVu
LXBjaWJhY2sveGVuYnVzLmMNCj4gQEAgLTE0LDcgKzE0LDcgQEANCj4gICAjaW5jbHVkZSA8bGlu
dXgvd29ya3F1ZXVlLmg+DQo+ICAgI2luY2x1ZGUgPHhlbi94ZW5idXMuaD4NCj4gICAjaW5jbHVk
ZSA8eGVuL2V2ZW50cy5oPg0KPiAtI2luY2x1ZGUgPGFzbS94ZW4vcGNpLmg+DQo+ICsjaW5jbHVk
ZSA8eGVuL3BjaS5oPg0KPiAgICNpbmNsdWRlICJwY2liYWNrLmgiDQo+ICAgDQo+ICAgI2RlZmlu
ZSBJTlZBTElEX0VWVENITl9JUlEgICgtMSkNCj4gQEAgLTc0Myw2ICs3NDMsOSBAQCBjb25zdCBz
dHJ1Y3QgeGVuX3BjaWJrX2JhY2tlbmQgKl9fcmVhZF9tb3N0bHkgeGVuX3BjaWJrX2JhY2tlbmQ7
DQo+ICAgDQo+ICAgaW50IF9faW5pdCB4ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+
ICAgew0KPiArCWlmICgheGVuX3BjaWJrX3B2X3N1cHBvcnQoKSkNCj4gKwkJcmV0dXJuIDA7DQo+
ICsNCj4gICAJeGVuX3BjaWJrX2JhY2tlbmQgPSAmeGVuX3BjaWJrX3ZwY2lfYmFja2VuZDsNCj4g
ICAJaWYgKHBhc3N0aHJvdWdoKQ0KPiAgIAkJeGVuX3BjaWJrX2JhY2tlbmQgPSAmeGVuX3BjaWJr
X3Bhc3N0aHJvdWdoX2JhY2tlbmQ7DQo+IEBAIC03NTIsNSArNzU1LDYgQEAgaW50IF9faW5pdCB4
ZW5fcGNpYmtfeGVuYnVzX3JlZ2lzdGVyKHZvaWQpDQo+ICAgDQo+ICAgdm9pZCBfX2V4aXQgeGVu
X3BjaWJrX3hlbmJ1c191bnJlZ2lzdGVyKHZvaWQpDQo+ICAgew0KPiAtCXhlbmJ1c191bnJlZ2lz
dGVyX2RyaXZlcigmeGVuX3BjaWJrX2RyaXZlcik7DQo+ICsJaWYgKHhlbl9wY2lia19wdl9zdXBw
b3J0KCkpDQo+ICsJCXhlbmJ1c191bnJlZ2lzdGVyX2RyaXZlcigmeGVuX3BjaWJrX2RyaXZlcik7
DQo+ICAgfQ0KPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS94ZW4vcGNpLmggYi9pbmNsdWRlL3hlbi9w
Y2kuaA0KPiBuZXcgZmlsZSBtb2RlIDEwMDY0NA0KPiBpbmRleCAwMDAwMDAwMDAwMDAuLmI4MzM3
Y2Y4NWZkMQ0KPiAtLS0gL2Rldi9udWxsDQo+ICsrKyBiL2luY2x1ZGUveGVuL3BjaS5oDQo+IEBA
IC0wLDAgKzEsMjggQEANCj4gKy8qIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wICov
DQo+ICsNCj4gKyNpZm5kZWYgX19YRU5fUENJX0hfXw0KPiArI2RlZmluZSBfX1hFTl9QQ0lfSF9f
DQo+ICsNCj4gKyNpZiBkZWZpbmVkKENPTkZJR19YRU5fRE9NMCkNCj4gK2ludCB4ZW5fZmluZF9k
ZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpOw0KPiAraW50IHhlbl9yZWdp
c3Rlcl9kZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYsIHVpbnQxNl90IGRv
bWFpbik7DQo+ICtpbnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3Qg
cGNpX2RldiAqZGV2KTsNCj4gKyNlbHNlDQo+ICtzdGF0aWMgaW5saW5lIGludCB4ZW5fZmluZF9k
ZXZpY2VfZG9tYWluX293bmVyKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+ICt7DQo+ICsJcmV0dXJu
IC0xOw0KPiArfQ0KPiArDQo+ICtzdGF0aWMgaW5saW5lIGludCB4ZW5fcmVnaXN0ZXJfZGV2aWNl
X2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAqZGV2LA0KPiArCQkJCQkJICAgdWludDE2X3Qg
ZG9tYWluKQ0KPiArew0KPiArCXJldHVybiAtMTsNCj4gK30NCj4gKw0KPiArc3RhdGljIGlubGlu
ZSBpbnQgeGVuX3VucmVnaXN0ZXJfZGV2aWNlX2RvbWFpbl9vd25lcihzdHJ1Y3QgcGNpX2RldiAq
ZGV2KQ0KPiArew0KPiArCXJldHVybiAtMTsNCj4gK30NCj4gKyNlbmRpZg0KPiArDQo+ICsjZW5k
aWYNCg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:43:22 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:43:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199478.353559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqiq-0006iN-NZ; Thu, 30 Sep 2021 07:43:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199478.353559; Thu, 30 Sep 2021 07:43:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqiq-0006iG-JQ; Thu, 30 Sep 2021 07:43:16 +0000
Received: by outflank-mailman (input) for mailman id 199478;
 Thu, 30 Sep 2021 07:43:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVqip-0006iA-CW
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:43:15 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 110c5ba6-21c2-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 07:43:14 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-Ad-g4B63Nv-5uhENxI1DJQ-1; Thu, 30 Sep 2021 09:43:12 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 07:43:10 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 07:43:09 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM7PR04CA0027.eurprd04.prod.outlook.com (2603:10a6:20b:110::37) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 07:43:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 110c5ba6-21c2-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632987793;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=SpucGsVZhrfevTC5K4PRHVCceXpffo7TMvvJJoU3tNg=;
	b=WyaK/KaUG+Vqnd7y2UsLabRwDF1pB9368rCGvA4mpld4TRwjM2I+ic0wzWGM1djLcMIEAD
	5Ws75B7db4t8yqbAVmtiFOpIAtrJTkYFpCUbmltJBUnwhAn8bi5ZxMsb4dVVpCAUEOn5/5
	zHwlL25sR1MrzSnpBP8ZKr9qP5tUNXc=
X-MC-Unique: Ad-g4B63Nv-5uhENxI1DJQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Vnl9iO/14sn5Zt2MsKCaFt9LiX9TcmuIWi+ACC+KKjmtlrcI0H94LV8AcJmzXys3ZktuQgNcxHl7zTBzc7qtV/GXjkpeWKedC4J1OgN+LDgBUnoZJRI9lCcZ65SrJbfJIrIUXggmY7WV72tozDWhzg/OE7on8nE+56iwmRpPvXIJ+BKR0G5Edw8uMZY8kZ14AW1xJsg9JHO0o2QR3wnnuOIFp93dB8XOIEWNB34Qtrhn0N8aw8ZEQYhqvogIHjX61iR/JH5gZ8v3PgyJAHLQcbxXau5p3Jhzx7EswNxTsQv+W50dkD6YFl0rE2t8C0nR4rBy3w8LbzRmq7+HQn4oHg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=SpucGsVZhrfevTC5K4PRHVCceXpffo7TMvvJJoU3tNg=;
 b=Y4EH+gK/2R4wwICbvw/tt2oC/zrGnjqFsw1O9wiizyIXNdp0c1lq4kCimS6P0gKK5KlsWJA83mzchRQkiqnf45O0D/Etfe0f3mmnZPDP1zzUsTM8ibWsyXhvmnvAwH4AjVdTs3sUNr8+6qKOpydUy1LIlRGbSkG7SpWmZU9K6Wza7CktykMRPf9k/z5xA0Q55K5jPI5ioj+t0p0LUvTuVx3frHRSgz44souAa6Z7OzuZPM7ElFwHu95qk5vQDfhdrGE9EvLs11ziVj8DkBy3Nx0+RXNfyw8NUGGW3TLUKb964gWxuFmJLTCkPVDpeY1M9XaDkT+NgKvKrTvhl5bW8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: HVM/PVH Balloon crash
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: xen-devel@lists.xenproject.org
References: <YTVAT0CQD9cPyyDF@mattapan.m5p.com>
 <84d9137e-a268-c3d8-57d2-76fb596e00d3@suse.com>
 <YTZ+XsnoKNnV4IOz@mattapan.m5p.com>
 <1b3d4cb1-ba61-0f61-5097-9978462a2401@suse.com>
 <YTd/SFtvuzejeiik@mattapan.m5p.com>
 <935dc03f-74f5-4b49-3a45-71148364fb5a@suse.com>
 <YUFdIPLo1AfM5BT2@mattapan.m5p.com>
 <3efe115b-1ff7-dcf1-8198-37bd7d7fb52f@suse.com>
 <YVD59QVbmdVwzYQI@mattapan.m5p.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <e5a0bbc6-30ae-58f4-0326-3c6fafa9be25@suse.com>
Date: Thu, 30 Sep 2021 09:43:07 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <YVD59QVbmdVwzYQI@mattapan.m5p.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM7PR04CA0027.eurprd04.prod.outlook.com
 (2603:10a6:20b:110::37) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 882470a9-afd9-446f-7198-08d983e5f300
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6479205C4DFE5E9759E0D460B3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	giJHaIKsSnH35IgGq/YldBTiCuAxA+znDDhAL0P2Xd2d6mz42Jme9+82NdPFVZiHgrTggnPINX5yVnKNAV10FCFGuCThbJBmphR9YJolIZNDEbdcfrXI5BZOKDvpYA96RU7asoVIEqrj1tKYvzq/dk+guO1NCjVcISadq9kzIZcD5L0C6XhK99p7J9wjNK9ZH543hJ6oaFq4HOjjMB5GgDQeggIdAGnG0lXfkaUKWhOFpxoNL0dWn2PWK0v+yafJSlp0JW/cHKL2K/PIfHjcyFFrZ32TxQC+WE67tsL4tm28WgUkE5MgvAR/WMazRxglIUCysh9vUp5IedjzDSH6GNiec6G1HHl3j1yM890ITZQLONbZ2P9MRO9ILnjVYhOW7ypBwmwpRbvchs4bw6Xb+XDUDPPMWkll63RlutJm6iwGkEDERcFlmkIGnVySDtmnoSPbERcOHlhBi3WPaKlqEdI2/TTrCOKyFOGTRgdl5fUCCbw/jyW57nDONOk0srD2+yesPWQB1lOjvQUItdHDjqkn+CCe2UUprltgIDtGswG/05pw19BLRWz7QnX57hd4gNAWiLU/HdG4kgalsFkr+woBoBzHwSwQ6vDXTbHAcsvf9eUh4M527tUt7WNGdSU1T64RVE1S1Ymq6LWs4Ne/S+pDt9fsnqGU6auP51QqM7zD0wJJnn2qlNl/1VLC6MKCtJ8HOwwUO4FO9UU/b3tISI7awjVENYWA3jEz4Zb6AkE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(8936002)(83380400001)(31686004)(38100700002)(6486002)(8676002)(66556008)(5660300002)(31696002)(66476007)(2616005)(66946007)(956004)(3480700007)(36756003)(2906002)(53546011)(186003)(16576012)(316002)(26005)(86362001)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eEp2a3dISU4vYUQ0bVJjdjF6eC9hUzNIRUM4bW0vYWRlREdFSG9IU0pEeDRZ?=
 =?utf-8?B?NEZVM2dmaXBzWjR4UGVNUU8vSldjN2d0NDY0UTliMmdJbDI0aEFZZ2RkZGFl?=
 =?utf-8?B?MmZDTzhZL2c4OExpV2ZVWG5nSjVSZXdhWldqcDVwL293QVphUXJYSGVKODFU?=
 =?utf-8?B?VUZlQXhkZ3RGU25QeHpNN1F1V2xLUk1Yd0tXTERWNlJFSUpFQ1psUzJVdHE1?=
 =?utf-8?B?SUU3NnFkQkpKbFBZYnZrK2JabC9nd3pTTzNjb1k0bDFSMHFRcG1EYnowZkZ0?=
 =?utf-8?B?U2V5UTBWRmxNWDZqeExQT1dQejNyM0lOMVI0dVZWZWViVnhvNDVYSXZnRzdG?=
 =?utf-8?B?dmc0YUhWb0dqeVVXWFlnU3RqMjNvcklzbWVNc09OY2VGbm9wd2M5dDdmRTRE?=
 =?utf-8?B?bDBRWVIwTUJTZGR1OHJSMG5WVmFyMGhQeFpMdHNPcnc3Q292Zk9PL0svcGhK?=
 =?utf-8?B?MWNpSWkzazRjb2pDMlNaem5vdlI4MXE0bG52YzlrZHhYNXdxYitUWUV6UFhi?=
 =?utf-8?B?R3VnZVdyODdvenc2QmE5ekxGeWZGTW1EejcwSitVL3ByNlRjenYvQWdJSysx?=
 =?utf-8?B?YjBwbnlNMk9ORkZPeTRPZWpqTXZwUXFIZlBLVG5rbWw5SlZJSE5XVDZCaUdU?=
 =?utf-8?B?WXJSeDJKWkw0MEdUanlDUkNXV0RIdzVnL0kxRkxIUWhTbUMveXpXUGkvaW05?=
 =?utf-8?B?RmNUbnU3LzAzdVBvbjYvZ0NrbzdmUUtORVM1TXRMaWtmaWdXcFBsKy9GQ1RQ?=
 =?utf-8?B?cEVVQlQ3eDdyM3h3ek40UDZ3Vy9zWmNaU20wNUV2Y2FWSURmZ05iWGF6Rnhp?=
 =?utf-8?B?QVplZkdOZE5sa2dLRDUzSE9iYldjNDgyKzlJM3gvWEU4Y3BuYUJ4djlNSTBW?=
 =?utf-8?B?SkJYRGF3NndVQU9YOWMyeVpuYmFLWFpaM0oxU2ZkandnRTkzcnVaUk5SZ3U4?=
 =?utf-8?B?NkdLbUcyeVVwNm1meDFKZUc0Rm9QcWY0L1R2MHJSdU5EYVJyeTNiaFVHZUpv?=
 =?utf-8?B?dVh6YkhOM3lvVjlwQ0ZDZWVOQTEveU1SZmUzOGtoQ083cHBTR1FGVWlWY2lR?=
 =?utf-8?B?dVBKVmwzVXUwSlhNbTNqd28xNzVKVG4zRllYZVNDZHRIVlZhSEU2cDY3aUtj?=
 =?utf-8?B?QmdVbUlkdjVha2NFR2NPSlpHL0NhMlNKTE8rd3F2MCtmbUlkam9OR1dBMzZa?=
 =?utf-8?B?dU8wRnd6UGJUTGhFa2tWa29xbmIxWnpNcmdTZkhCeENMTXdKbkNhZFN3azZk?=
 =?utf-8?B?anNBd2hhQUNlSUVheHI4c3pkMU9BTnkwSW1ybmRScU9UTFd0RGdyaUMySHpm?=
 =?utf-8?B?MnA5NzVhN1N0TEJoczNVVmZUckkrbG4xckRKSk9FV3ozU0g4RDlnOVhodXQw?=
 =?utf-8?B?RUUrbHR6RWtTeTUwbDAvNnJ4VmpaWFVGOUhrRjZvTjZSSUlqZkE1V2Fwczhu?=
 =?utf-8?B?VzMvMVJZR2FVZTdCczl0MExJbHYyNllDOXNzclUxcFdWd1hxY3UrU2tpcVZ4?=
 =?utf-8?B?Z1R2ZmJ5VjVla054SWlRNzBKWkwzc0xnQ0FyTG1RV1JlZStZaE4vQXMveU9O?=
 =?utf-8?B?TWxNZnNRVHZzTDZMeUFCZWNmaVRLajJEejUxT3JGd0JTekFOUGljYUNvWWZB?=
 =?utf-8?B?dzVmZnVDTS8rdXNlMGY0eW9hS054aE5GWUN2KytVU3ljeXVGMHRabFdZQzd5?=
 =?utf-8?B?TWhLTkJTTXk1QVloQ3hwMTM3U0hrcVhGVTVjSnpKMTNiNXpqK1Z0SFBpbTlX?=
 =?utf-8?Q?7vZnpbX5v/Hco6K48YXy4YsYyZx6H+tqMPWo6g0?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 882470a9-afd9-446f-7198-08d983e5f300
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 07:43:09.8357
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: U8aGpajH7av7GBE+eCr+jpVZ4BwYN404noWfcx38tqXT1Tq/iRuenofnjE364QYqB1IeCN1ZgxkUa4fqv9d6+Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

On 27.09.2021 00:53, Elliott Mitchell wrote:
> On Wed, Sep 15, 2021 at 08:05:05AM +0200, Jan Beulich wrote:
>> On 15.09.2021 04:40, Elliott Mitchell wrote:
>>> On Tue, Sep 07, 2021 at 05:57:10PM +0200, Jan Beulich wrote:
>>>> On 07.09.2021 17:03, Elliott Mitchell wrote:
>>>>>  Could be this system is in an
>>>>> intergenerational hole, and some spot in the PVH/HVM code makes an
>>>>> assumption of the presence of NPT guarantees presence of an operational
>>>>> IOMMU.  Otherwise if there was some copy and paste while writing IOMMU
>>>>> code, some portion of the IOMMU code might be checking for presence of
>>>>> NPT instead of presence of IOMMU.
>>>>
>>>> This is all very speculative; I consider what you suspect not very likely,
>>>> but also not entirely impossible. This is not the least because for a
>>>> long time we've been running without shared page tables on AMD.
>>>>
>>>> I'm afraid without technical data and without knowing how to repro, I
>>>> don't see a way forward here.
>>>
>>> Downtimes are very expensive even for lower-end servers.  Plus there is
>>> the issue the system wasn't meant for development and thus never had
>>> appropriate setup done.
>>>
>>> Experimentation with a system of similar age suggested another candidate.
>>> System has a conventional BIOS.  Might some dependancies on the presence
>>> of UEFI snuck into the NPT code?
>>
>> I can't think of any such, but as all of this is very nebulous I can't
>> really rule out anything.
> 
> Getting everything right to recreate is rather inexact.  Having an
> equivalent of `sysctl` to turn on the serial console while running might
> be handy...
> 
> Luckily get things together and...
> 
> (XEN) mm locking order violation: 48 > 16
> (XEN) Xen BUG at mm-locks.h:82

Would you give the patch below a try? While against current staging it
looks to apply fine to 4.14.3.

Jan

x86/PoD: defer nested P2M flushes

With NPT or shadow in use, the p2m_set_entry() -> p2m_pt_set_entry() ->
write_p2m_entry() -> p2m_flush_nestedp2m() call sequence triggers a lock
order violation when the PoD lock is held around it. Hence such flushing
needs to be deferred. Steal the approach from p2m_change_type_range().

Reported-by: Elliott Mitchell <ehem+xen@m5p.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/mm/p2m-pod.c
+++ b/xen/arch/x86/mm/p2m-pod.c
@@ -24,6 +24,7 @@
 #include <xen/mm.h>
 #include <xen/sched.h>
 #include <xen/trace.h>
+#include <asm/hvm/nestedhvm.h>
 #include <asm/page.h>
 #include <asm/paging.h>
 #include <asm/p2m.h>
@@ -494,6 +495,13 @@ p2m_pod_offline_or_broken_replace(struct
 static int
 p2m_pod_zero_check_superpage(struct p2m_domain *p2m, gfn_t gfn);
 
+static void pod_unlock_and_flush(struct p2m_domain *p2m)
+{
+    pod_unlock(p2m);
+    p2m->defer_nested_flush = false;
+    if ( nestedhvm_enabled(p2m->domain) )
+        p2m_flush_nestedp2m(p2m->domain);
+}
 
 /*
  * This function is needed for two reasons:
@@ -514,6 +522,7 @@ p2m_pod_decrease_reservation(struct doma
 
     gfn_lock(p2m, gfn, order);
     pod_lock(p2m);
+    p2m->defer_nested_flush = true;
 
     /*
      * If we don't have any outstanding PoD entries, let things take their
@@ -665,7 +674,7 @@ out_entry_check:
     }
 
 out_unlock:
-    pod_unlock(p2m);
+    pod_unlock_and_flush(p2m);
     gfn_unlock(p2m, gfn, order);
     return ret;
 }
@@ -1144,8 +1153,10 @@ p2m_pod_demand_populate(struct p2m_domai
      * won't start until we're done.
      */
     if ( unlikely(d->is_dying) )
-        goto out_fail;
-
+    {
+        pod_unlock(p2m);
+        return false;
+    }
 
     /*
      * Because PoD does not have cache list for 1GB pages, it has to remap
@@ -1167,6 +1178,8 @@ p2m_pod_demand_populate(struct p2m_domai
                               p2m_populate_on_demand, p2m->default_access);
     }
 
+    p2m->defer_nested_flush = true;
+
     /* Only reclaim if we're in actual need of more cache. */
     if ( p2m->pod.entry_count > p2m->pod.count )
         pod_eager_reclaim(p2m);
@@ -1229,8 +1242,9 @@ p2m_pod_demand_populate(struct p2m_domai
         __trace_var(TRC_MEM_POD_POPULATE, 0, sizeof(t), &t);
     }
 
-    pod_unlock(p2m);
+    pod_unlock_and_flush(p2m);
     return true;
+
 out_of_memory:
     pod_unlock(p2m);
 
@@ -1239,12 +1253,14 @@ out_of_memory:
            p2m->pod.entry_count, current->domain->domain_id);
     domain_crash(d);
     return false;
+
 out_fail:
-    pod_unlock(p2m);
+    pod_unlock_and_flush(p2m);
     return false;
+
 remap_and_retry:
     BUG_ON(order != PAGE_ORDER_2M);
-    pod_unlock(p2m);
+    pod_unlock_and_flush(p2m);
 
     /*
      * Remap this 2-meg region in singleton chunks. See the comment on the



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:47:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:47:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199484.353570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqn4-0007WK-7g; Thu, 30 Sep 2021 07:47:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199484.353570; Thu, 30 Sep 2021 07:47:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqn4-0007WD-3f; Thu, 30 Sep 2021 07:47:38 +0000
Received: by outflank-mailman (input) for mailman id 199484;
 Thu, 30 Sep 2021 07:47:36 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVqn2-0007W7-FY
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:47:36 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id acd36a3e-21c2-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 07:47:35 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-HnYUTLfINT6vVIUT86yEDg-1; Thu, 30 Sep 2021 09:47:33 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB3118.eurprd04.prod.outlook.com (2603:10a6:802:a::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 07:47:31 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 07:47:31 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0089.eurprd06.prod.outlook.com (2603:10a6:20b:464::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 07:47:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acd36a3e-21c2-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632988054;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rEi8YCGoQCMd3c0uBLGf8bg22g3wYRC/W5+8MykFD+o=;
	b=Xt+B+ofSbMjt6smFfe4Y2c8vScHPIs+MHTP+D4XlvZBIkbX6is30hGxz3XmpVE8xvGMTuf
	2ibTjQuIEzUwirwMYenwVTxzDA+xvdP+hRNqIfs6WFqIqYOS4sfyUa9gWI+W7OUeE+xbaZ
	CAgsqmGa4s6QSsSEHsLXa8rcXhr+fJE=
X-MC-Unique: HnYUTLfINT6vVIUT86yEDg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Qk7SdqfppdGZzjp+6m3i7iBD24pwR/eivlmK79uoxUzuCMOi1WDogT8kj6pIZf4Nfl7D39U1DkZe/UsHa64F7WvtDqqaZVROYwCeNcGIq0eOeksJ7TMSlYR7Cic90PCLYuojy2KpbZIV1asBHZz+2/fMi0fxU+erEQeKyS1x9TeOew1rWFZ/0eHO/XDc/SmQLJ9/rh5WLob3NYgRIjsBpCLDyjWYZ35Uoan5W0OGxF20CaZ5ezutBFCC+BN0+g2CDvszo+s8v+oBcPHVHh/MrPF6QG+iA+FhbBpCG5HM4Yp2IKKEqyRZ6I7RsiBDfQtLE6EWuXcehD13fCzSRGmWRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rEi8YCGoQCMd3c0uBLGf8bg22g3wYRC/W5+8MykFD+o=;
 b=Jtb5Fvg8EDXGr+/2WggMy1APWXiyFshtxoXSOmpBHazCv+aEClvVev8nvioo/oFxyWtDQ7tU6jOjIS9fy0vXyzpRoNMFCuPsdgycudA/LLihTBygNfMwYNqyZDNAF0Xq4xCVNLh7kbXw1gY2BNnedvwulMBHpUZecpQcXb3emRZNH2vrFRDdZqcOjEuPOrzwkrBgOFP7pOPk+5cddYVRZtLGsY3y2YoT2vS9RlUbt4ZSG4cyl+xBi2TP0/vGYrjUGlovZI7s31bA60uopBi4uUSb3L8mve2NO8nxQ82cwg8FgirZFY8BfQTQodaFMhwbVp4qn4sMRusEFL+WPWFbDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: xen.org; dkim=none (message not signed)
 header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 11/17] xen/arm: PCI host bridge discovery within XEN on
 ARM
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 Andre.Przywara@arm.com, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <a16e90a04ecb722e0f6c5fb8f9a9b0129b4fe96c.1632847120.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109281636420.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <adac8bc5-2af3-ccf0-0dbf-6e17d21f6ee1@suse.com>
Date: Thu, 30 Sep 2021 09:47:28 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109281636420.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0089.eurprd06.prod.outlook.com
 (2603:10a6:20b:464::14) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ac8ac913-1c06-4f2f-b22e-08d983e68e9b
X-MS-TrafficTypeDiagnostic: VI1PR04MB3118:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB31182D4AAC3DE4BEA7087208B3AA9@VI1PR04MB3118.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0bV15i/1X09UuKz/LYtGQ75C73J3VWHA3FyIDHPoTDC/WrjcsRuD2NF8fn9S19ebFBsqNR7J+WDXiq8x893TeNpEE699/hh9PIZLCceMw9nD4BKSSn8OBTwfHC4D46HjpLUlTzxgTwFrcEwK2F72FE9YIXskw0xCvRxwYwHPpnIghi/3ptWKcPP27F5d8fwbWtHeg6zSh+MCLM7FeOv9f3bP8O07fl68qm8t6G86qjUQYapw8mAHjrJefmWuzkUrXm2jBLHBKNxPaHLWoiAtdMLi5Rrn6sLiWVFqSEDXyXRztLJulva567fBzgRUEKiP5KaBUp2FZc7UfH6XYaBFoj2iobDKEjkMiBZBod6TH7wL3eRLQ44AYNejvXfVtZ2WynRyv1He6IVgJ8sWgooJzbzTbxR08M5vqdBrBGlyldNL3slJvGpLbYCnXCNyNBxS/Gu3d7YCQVKZpAsCmX1PUiCTrsDN8Wanig5prfKZEVMF0pxoT/ysg7ca1Zv98YsGn01xyboZtxZThQJbnG3grTTJsacb37iZTWiOaTZsEE358jI1ifgItOnZAsovPeifQaY+97O5AAAC/YnyHLsHtLs6JAEpKmzGXdbzDQiwPtRiUImec+g/06qsYjptDhpQ5Blo1xvd+kFYpDgcoGPhBxOAvayNJcmKtKjTC6X0k9NHqNMZ9X5V7ULkfCPj3oeDG6J7bKZvsd/XdgiTB2XJvAESDI+CL0FDDf4ZHKNluUNbrpyx4TnnNBLfSzx79oUTlJcqoURLYw09tvXfe1Uoi7JvGeL8Jx6p+9PmHz46R0kaNHUo9aa/Ce9BEaExo//qol+oRRypSoFaRG12nY5SSw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(316002)(31696002)(186003)(110136005)(66946007)(66476007)(66556008)(6486002)(86362001)(4326008)(54906003)(16576012)(2616005)(31686004)(36756003)(2906002)(956004)(508600001)(5660300002)(38100700002)(53546011)(8676002)(7416002)(8936002)(26005)(2004002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UHkxV25uS25nclloUzVhT01MSllCT1dQZlMxa3BiRGZLUUU5Z2tQWS9NVzB4?=
 =?utf-8?B?YmdXeFZJYUFYeEF2MVVsUGJBa3NrZnZKNmJnTHlVRk9pUGxldDR6ak5oeFA2?=
 =?utf-8?B?dHhKN3BSWVplc3ZlNVZtNnd2dngxVVVpbGljc1hoQzUrY1p0UXcwczhUMTZi?=
 =?utf-8?B?eU1NTG52aG5ZaXBsU1ROWmZLL2JHOUxpUVJBcmt6cDJyamRCRGJTbCt5eERF?=
 =?utf-8?B?L3A5eUphcGRhcmVXSzVkWXdzWDNVdnc4SWp1K09EdU9rT0dCR3lEV1FjcGsv?=
 =?utf-8?B?V1pUVUxPb0k2QUxReldaMVlIWkVWV0hsS3VmcS9CTytMcFE0Vk5MSHBYOVBM?=
 =?utf-8?B?TlNzRnhvRnZ6dGsrMmQzTWtXWEV3MkNWdHlOSWpkSWYrYSs1WHBpQ1FGNnJo?=
 =?utf-8?B?UlNZbmViNklwN01tVG9mcGUycFR5cE9aUTJkQzVkY0hwY3FtTm5zV2t2MjhT?=
 =?utf-8?B?NjFLTnFNYWRHK3QyMWgyMllQL2xiMUFJK2JjcHFuVnljeXBBY29jeTY3dTlV?=
 =?utf-8?B?R0puY0wxVU4walM2UVdOcTBXU294TGlxUEJxb3ZvYnM1cnh2NHUyNVZadlZz?=
 =?utf-8?B?NXNWRlIrWCszVk1EK2NtaVBxVDFGSGl1T3BFZGRWRVhnS0dncm9VN3JXOG1N?=
 =?utf-8?B?dENjK0ZwZ0lvdnZiUm9BbDRORnFkOHZMVUNiZGhOMzM5bHhvSDBvQXRZMmRu?=
 =?utf-8?B?MkFFcUR1K0ZkN25aaWVHbjE0K1UreTB3T2cySk0xaHhnVUdYdmxQSzdvbHV4?=
 =?utf-8?B?Vm9sWWFMTkduRXpwMkhGZlRCaWd3dlpNVWEvYi9DVTA5SnovR0I5eDM2WUU4?=
 =?utf-8?B?M0NyVUxQb1FiNHpYTHdqbkFkbFIrMjlROTFZUEROYVc4dU95c3cveldhZnU4?=
 =?utf-8?B?bndzSytza1N2UVpWRUNUNFNQNU5adFk5VHRLa2tMMnUzaGY4dGRPR29QWlpY?=
 =?utf-8?B?V3NOeXZMYzlLL3RzN2UrVU1DT01oZGRYazlvWHBMbkwrSVl0bjkzUTFSdzdo?=
 =?utf-8?B?TkxRK3RnVy91cUM5U0Q0QXpNUEswRkhhMUZqZno4KzRqelY2MzVLMHJoc0dP?=
 =?utf-8?B?VDhHMG9tUEZaRFZOK3l5YVJ4c0xYVi9ESGJIZWRJSjZXL0lUUXdxVzJ6N2Jm?=
 =?utf-8?B?R2NTNHN3cDdlVGFTSmVTSklFdkQ0YXFDS2JkODZSNUVtS0JFL2xOVk5iZ0pZ?=
 =?utf-8?B?YXh3emZIdXR0NkROUlZWTFN2aUFSa0UrNWNkanlFZ1crY2Z3S01HSDd5TWoz?=
 =?utf-8?B?elorS1kvOStpZnNnSEMvZUt3NFhGZ3JENnZaUmVodTA5UDN0eDlWQzNML2la?=
 =?utf-8?B?cjU5Y1JVcFZCWGxEdDE1MHRBcGwrLzZqanJWQ0Y4UDBRWlpCb2t6WitETHdv?=
 =?utf-8?B?a0RTNkx5VGt3K3RWa2NUL0R6L0dDQTRNQmU4VURhRUlkVTJVYkVVc0ZsLzJQ?=
 =?utf-8?B?SUNwbWVtbDFDdFhoY2R6emNUV080WVRZMlVyMzlwYUVpNHArTy9QNTkyNG9y?=
 =?utf-8?B?YklWdk1tdXl0YmJndUR4ZzNDdlJVZlF1cm9BOXpFUzdLaTc4ampEV0x0M0k5?=
 =?utf-8?B?aDdnVFdxWVA3VkNjbzNyMjJaWUFtdjRCZUg5STF2Q3VDZ1JkcFJ6SThHRjFB?=
 =?utf-8?B?bmIvYmFkaG9LYkdEWVJBelBaNkl5cnl3UXVhL1R6NWpQK1JYOEk3SE1zaW1J?=
 =?utf-8?B?TGZMdUJVLzNXRVJ2OGhHa01SbFZJc3FFZUViaWlZVU9kZzhWL2xYb1hhMVNP?=
 =?utf-8?Q?ThEw9fLiZDM8STcWCWfP+SJKw84Hl7/AiUSOFZj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ac8ac913-1c06-4f2f-b22e-08d983e68e9b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 07:47:30.9000
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1JsBZFwL3dwCcdR0q1i/K7QvAH3aSVQuA9kF5wjDsHhlGwmdq1fcfrGGpmzOKaJP6ugFt9HSDmkDqAhq6SLBQw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3118

On 29.09.2021 18:40, Stefano Stabellini wrote:
> On Tue, 28 Sep 2021, Rahul Singh wrote:
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/pci-host-generic.c
>> @@ -0,0 +1,46 @@
>> +/*
>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <asm/device.h>
>> +#include <xen/pci.h>
>> +#include <asm/pci.h>
>> +
>> +static const struct dt_device_match gen_pci_dt_match[] = {
> 
> This could be __initdata

__initconstrel then afaict.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:48:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:48:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199490.353581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqnm-00084M-Hm; Thu, 30 Sep 2021 07:48:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199490.353581; Thu, 30 Sep 2021 07:48:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqnm-00084F-E0; Thu, 30 Sep 2021 07:48:22 +0000
Received: by outflank-mailman (input) for mailman id 199490;
 Thu, 30 Sep 2021 07:48:20 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVqnk-00082o-N1
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:48:20 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1e79797c-0778-495c-bb0e-042f04840ab7;
 Thu, 30 Sep 2021 07:48:19 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2054.outbound.protection.outlook.com [104.47.14.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-xryZEqKIPumBwEgTV9hQxw-1;
 Thu, 30 Sep 2021 09:48:17 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5472.eurprd04.prod.outlook.com (2603:10a6:803:d3::28)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 07:48:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 07:48:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0064.eurprd06.prod.outlook.com (2603:10a6:20b:464::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 07:48:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e79797c-0778-495c-bb0e-042f04840ab7
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632988098;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=X7HAhWs+o1SPKsixdaXKwcMz49MgaVt+tkC+p8wyH7Y=;
	b=GgrVmYLd+2ujItqxv5P1rkFrDSYvw+6f5mKUbOdTOYOb2aPHWA06bH5u2XHNQTN9eW6mC6
	bCIs/gsV05jh9m5mnpLmvH827I+rPu/pfYd0Lp8T9zzqwSA4wvFtSPHRFGoD9sww6q1WAx
	Fj1nwP7yTCQhdlA+aCks+5LGOr3/MNg=
X-MC-Unique: xryZEqKIPumBwEgTV9hQxw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ohB+IxDjBSxZwH+rtV1DyBSYoSIL7r860l4RhOwawwm37JCRw+gyZsgBlApyHOm98rqIso3vUwUchHdlAuwgVBzA4Qa08jaRAOQq7HpHu99omEWv/oQ4bsFv0Pib3kyoPWkkk33ID49jnk5GjtiUggr7FYQ6uGI5Ucqii1vLgQ8o19Q1GXLG4iLCMEBiGQ20NIJUZIQwIik7UZlOSe3RUW6u8WjK+zP4F47vsBft9s9ujSiVZoTTj6gBaQGH9aruzS84lFvUI7twHikxZ+19gf0WSEHG5SPhas8b0gMet0dKRi86VobaHCqTyfrBxrWh1cjHswFOdT5thiNFJjNwjQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=X7HAhWs+o1SPKsixdaXKwcMz49MgaVt+tkC+p8wyH7Y=;
 b=dG9dwnOFgAkgj6voJQCY3AXnQbuBm3n5fEhDb7A0rbj/J1nK2ftjduHaP5jyVmhwkSszMK/f2O1+s2K9L+JfXpm8rxxhhwTKBPO+3AJUaAzLCmmH83qKO5KyXVqAIl3ER8VIhpyW6G6SFtcEZzkj88akTJGmqsmhh7zFsiNvlJcilE3ARu096tW1K+K7icc5YM949QEESEcE6rrwo+zhH34xGxok+Tfsx6Ye7j1uNr4WZsRzzUA4Kaov5L2RhHuMo+O9iC90MHgYotRCa/IjC3z6IAdfzA81+vAOZHpcPDCeWx7DC81va6feeBbjIlMnvB33pbvsQnGQe256J8VKxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: epam.com; dkim=none (message not signed)
 header.d=none;epam.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 Andre.Przywara@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Julien Grall <julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109281638210.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <3478d1b9-7043-3505-a125-700eb3761ed4@suse.com>
Date: Thu, 30 Sep 2021 09:48:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109281638210.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0064.eurprd06.prod.outlook.com
 (2603:10a6:20b:464::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 08afe28e-23b2-4d42-0992-08d983e6a913
X-MS-TrafficTypeDiagnostic: VI1PR04MB5472:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB54729BB76710526BC72878D8B3AA9@VI1PR04MB5472.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	T4iD3lfsZdWjF1cw/ED1torz3izpTiCDmN2Txz2pxSOAuiF6fvLXcHyRwRvpRnJmMj+O87AKzzeLCV69OSY3Wsl7Y0CPLkaMzh/YwcAcs3e54Q0+wFHGC8NF94z+hBPpKEFOGE1mDoPQfqYbncA9YL8gMlcK8fm/MIpvrmS5SaHB4gLXL9gpKRl6QIBxDozItgO4yaG5tMT2j1r/fxZyMzEKAEd4JTbAAELttgTtvkznAoUN9o21BQTHYIhP1ycFVdlcKWZvwcOP2ZrNQ3u/z91UjXCt5zw2gqG0C7Eov25n5q/kxdD27DGVArutv02EuRRv1UZfQCrqQ8wIPuE3/gwuo+es8KO57fPLKyM2vhxXGE+7OSTEeO0GK+PAKkCSD2p3Y6+/KFQG+Gq9JA9BYUkbkXnHcTIP/oPB/X0ISbl3EPwcQKQnf3vTGb8fsbkv5AFiGEuYSetDHMsLPUjtaq2q0VgkQpyXshenptVHRbd8gkttbyEHYH0nZZ7h2CahvkeP7fynnn8dYWMQtEmZTzORf4CHvZqvw6X+VpoEbXlIOxUNolvvTW4XRcdVJY7taypNqw8mN0v9eao+i+CkgfPLuqni5ubdBdSiR5C3uQ1WMr9/8t98HHQqusYxnM9R7JJV5QKePCbeFXSnJPomEB91hBjfgJmrY8fXBMxHSPmEmbZN7NQSkWREvII4nwzcoGtiJbPoKOaXqBk8Qr9065PedrLXxMgW8QLDm2xdntICXId4BhNvbv+nntYltfyGuHfy4etWWruJqD8YYaQWIORZyVqTi6ISnzFQpLfyhoGvAcAKD6zBvNBZUCMx09gjmsfSy/djkQmjfc1AVv9mCw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(54906003)(110136005)(4326008)(6486002)(38100700002)(16576012)(316002)(8936002)(66476007)(66556008)(2906002)(8676002)(66946007)(5660300002)(53546011)(956004)(2616005)(31686004)(86362001)(508600001)(36756003)(31696002)(186003)(2004002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TS9WSjh3VlN5MzN5dWFndllscVlxeTV4TzQ0dzN1MHg3ZGNWNXgxMVVoTm5G?=
 =?utf-8?B?RmJTZlZPWEl2YzBwNFU0UTlxUGE3TGp0TEthM1RyZTRPaWJ3M3FMSnEwZGdn?=
 =?utf-8?B?Zm1xUjNETGJ2ejBEcTQzK2p3S0xFTGhhKzFuWm5uYVNuQWRBQ2RLZXA5cG0y?=
 =?utf-8?B?WG5kZVFUdFpndXlqL1M3dE9LaWhNci9iZHpCa1FiQVp5M25LMDN4eUJ0NWgz?=
 =?utf-8?B?MXFKa3BEMUFSSkMrTHRDanIzT2dwSVB4TUVUUTFvVXF2R09TL0VnUmRsbjZ4?=
 =?utf-8?B?akRodEhHWHZINHBKSGhNUzNYSjdnSGgwdE1aRy9sb2UwK2FjSG9jSEtiYkIx?=
 =?utf-8?B?MFFtbWFSRHlLQTE0TUU1QmdrQkpVRDRLNHM3TzBORWtpdzNMVFNicm9DYm9E?=
 =?utf-8?B?Ty9CSSt3T1RuMGtBd3BTQ2pzTHVqWFBUOGF1T3JsWDBZUERBenFNaENmSUlr?=
 =?utf-8?B?Si9sQ3VoNFpiU1JZL2s1bllOZk5DUWhJYXpjQm1kTm9oMmxqSHJZRU05T1Iv?=
 =?utf-8?B?Y1Jadm9yWjBkeXBla2FLaG1GNXJvdjc1UmY4eGRGL1FZMVBIMjJvWFU2UEtT?=
 =?utf-8?B?NnVqbFFLWXJWY0pVVWhGekFBTVlNSS9UL2IyS0toUHQvdmZaamFUYXBIZ1Vz?=
 =?utf-8?B?Q0drTWdZUkIzYWF5S0VNeVZQNXpGWDk4S0w0cG9DMmR6T1ZLVGZrMGd5U3Fv?=
 =?utf-8?B?K252cXd5YkJZR0V0bDJWUTkwaHRsQlBGL1pWdEdDazY1cDBCa2pUTFM2MDNM?=
 =?utf-8?B?cWs3T2diQ0oyanZiRWpYRjZDaW4xQ3RsZHozeWkrZUZSMU1SQllldWl0YWYy?=
 =?utf-8?B?SjhvL3pKTUxQVmE4QlFBZklkTHU5UDlTdEVhRVNOU0I1VktObWk2M1dxaktO?=
 =?utf-8?B?aU1PWXpPRHdjQkNZRE96bHZIWk9yRU9FNVRnMG5ocEN2QVJmYTdFWFEwci9P?=
 =?utf-8?B?MVBpV3Q2c2M2UW0vZW5BYlVKaEpJejFZbFBreS9HT0ZENDQrZUlYSE5qYk9N?=
 =?utf-8?B?Mk0rc1JpekdCM2UrWGpDN25BdEJkcVQ4bjBnakI5TEs3cEZHY1hKaHlzSGpt?=
 =?utf-8?B?UkprY3VraHFta1pGVThxa050K1dlQTMxdG94a3ViR01YOWtxVkdXditadUc4?=
 =?utf-8?B?QisraUFtK1M0aDBCTTEycXphNnA2VWFiRURhYU8yZDFBMXhkMmdZWGQvdDlG?=
 =?utf-8?B?OE1tUDVrazdLVWJ4b2F5L2NIME9wUzZWUkgzT29pakQxZG5DWkR1NmRpZXJ1?=
 =?utf-8?B?dWxIa051RWwxL2FGVjZZRldyQ1VQdVIweDBMeXVmeXZXMVVKRDZERkFKL2Rp?=
 =?utf-8?B?SjE1T0dsNWZFdk1CRWc2RzFvTXMwYVFkVGdpUVlRaU1kc040bkhkTVhNYTFD?=
 =?utf-8?B?aHVZVmhVQ2kxQnJGZDlUZmtHcXBOcWhwRXZrL3hGQk44UFpVOGpOSWMrWnRa?=
 =?utf-8?B?SHJyS1VES1ZBWFJKZmhKcEhEU2p3dnJLd04rQmVTc3VVaCtMdGV2NUtHaytp?=
 =?utf-8?B?R21Lb0M1SzdwL3NjTlNwK282aDYwRXNIVTB0bzVuWnFFSSs1SGJ2WjRqbVUz?=
 =?utf-8?B?d1R1VmFKclNWT2RBeGkydDlmUGRabFBXRTE3MUhCZXBhNVdkRjgvZ0JEYUI3?=
 =?utf-8?B?bTFBSHcrMDlNejU0ZUJJMkQzVmI4U25IUHkzT1cycDZjVTloMGsxVEVhSytR?=
 =?utf-8?B?WXNWTy9HSG1hQ0ZaZldIVzJjc3JjVGtZQnN6SXJHQVdjYXBZM2hzSDRaVXM5?=
 =?utf-8?Q?9Q3dR2gr5El5GCmB1rwGUTlsl+lEHr6jUHdPBLS?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 08afe28e-23b2-4d42-0992-08d983e6a913
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 07:48:15.2939
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AjbpjsinJYxKTDoTxJSxumdr6T76zr3mfI4zC7HkMXQjrNPUPAFObcH0tcv4C6S6oeQNIDcadL9YXMcyC07u1w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5472

On 29.09.2021 18:41, Stefano Stabellini wrote:
> On Tue, 28 Sep 2021, Rahul Singh wrote:
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
>> @@ -0,0 +1,63 @@
>> +/*
>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>> + * Based on xen/arch/arm/pci/pci-host-generic.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <asm/device.h>
>> +#include <xen/pci.h>
>> +#include <asm/pci.h>
>> +
>> +static int nwl_cfg_reg_index(struct dt_device_node *np)
>> +{
>> +    return dt_property_match_string(np, "reg-names", "cfg");
>> +}
> 
> Can this be __init?
> 
> 
>> +/* ECAM ops */
>> +const struct pci_ecam_ops nwl_pcie_ops = {
>> +    .bus_shift  = 20,
>> +    .cfg_reg_index = nwl_cfg_reg_index,
>> +    .pci_ops    = {
>> +        .map_bus                = pci_ecam_map_bus,
>> +        .read                   = pci_generic_config_read,
>> +        .write                  = pci_generic_config_write,
>> +    }
>> +};
>> +
>> +static const struct dt_device_match nwl_pcie_dt_match[] = {
> 
> This should probably be __initconst

__initconstrel again as it seems.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199498.353591 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrk-00017s-4p; Thu, 30 Sep 2021 07:52:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199498.353591; Thu, 30 Sep 2021 07:52:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrk-00017l-1u; Thu, 30 Sep 2021 07:52:28 +0000
Received: by outflank-mailman (input) for mailman id 199498;
 Thu, 30 Sep 2021 07:52:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqrj-00017f-5u
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:27 +0000
Received: from mail-lf1-x12b.google.com (unknown [2a00:1450:4864:20::12b])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cfc4d3ba-e8ea-410a-b0d1-6b24c80b1834;
 Thu, 30 Sep 2021 07:52:26 +0000 (UTC)
Received: by mail-lf1-x12b.google.com with SMTP id j5so16802725lfg.8
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:26 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:24 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cfc4d3ba-e8ea-410a-b0d1-6b24c80b1834
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=f1lP+PWhUKzqxJrgjRUDJ6qHEEYJMmdke3MY78TaS4Q=;
        b=lpL7Q9gwdueyTmaOwbFQ/AG0v61hiI3nn0kvPZnOSWcI/gQEohI5AoM+x/intb990j
         vlA4edTT+NZbMuQBc1aIOqUklDTiTDoyQwFftukdjo1Q60vZZHAv1dh+nebIo7Hj1OZn
         E0AgbpKPplxnTcOD8RmpbZ9NizL9zbSgZsNMsh8Rmd2S8MNxe2CFr4C9Dhy8E6AQjjuZ
         ja7Odwqk8P+fkrs7OYhhEjUCVZ772IdQqlXLVjvDt6GCzC4WneeWUoeC/Xu8uv3iZB0k
         Dm+XNBkZwEH8DIZlHHdiIIwIoGAz11jqn8+8rNMY2Aue3jwRxr9PxO6GIZa05lMc1X5T
         Wgug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
         :content-transfer-encoding;
        bh=f1lP+PWhUKzqxJrgjRUDJ6qHEEYJMmdke3MY78TaS4Q=;
        b=JOv5/ceYXKlfEwcjnVXyQbVy9iIgPYSk+L8Nbo45u2Dqq4s/DgIU4/Sgr6yeBs6qWd
         4KNKfj6r9BIW7R7tYectPoSnugOVQtrHvw3ZLsrjBm3Jfh6XhsyYYGfRBMPjfkHrYZdb
         vVE+BuVnRB9kdcJx+wTG6L3qNH7nLgtI+4yNs3I7mER7zLzc4pywtfKeMblWOh58b02B
         HjzxF+nGzNBXIdPg9ea0s1tPgOBhegJRfSbtdeJokUPFsN7mVrQSQ/5xzLdIL46P8WTA
         eW6Pp0USEY6X4qk7m09bSMkb2rENBS2oJpHgdGV9rhy8AOoJev264AtahrbtzP4pBLqM
         wz9w==
X-Gm-Message-State: AOAM532CrAv1sH4kKG0183BuyaodngfOpOVgK7J7DjTfcl5qhAQ6txpc
	KjmxkqEKcsF3hucRwyVjBOfnIfVwPIE=
X-Google-Smtp-Source: ABdhPJy00kRD3QeitcjiG131mSv12tgDLeW24qU+z8p5OfcujYnsG7AADcTi51YcrW05Is1n8wKKig==
X-Received: by 2002:a05:6512:3da5:: with SMTP id k37mr4744888lfv.359.1632988344762;
        Thu, 30 Sep 2021 00:52:24 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 00/11] PCI devices passthrough on Arm, part 3
Date: Thu, 30 Sep 2021 10:52:12 +0300
Message-Id: <20210930075223.860329-1-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

This patch series is focusing on vPCI and adds support for non-identity
PCI BAR mappings which is required while passing through a PCI device to
a guest. The highlights are:

- Add relevant vpci register handlers when assigning PCI device to a domain
  and remove those when de-assigning. This allows having different
  handlers for different domains, e.g. hwdom and other guests.

- Emulate guest BAR register values based on physical BAR values.
  This allows creating a guest view of the registers and emulates
  size and properties probe as it is done during PCI device enumeration by
  the guest.

- Instead of handling a single range set, that contains all the memory
  regions of all the BARs and ROM, have them per BAR.

- Take into account guest's BAR view and program its p2m accordingly:
  gfn is guest's view of the BAR and mfn is the physical BAR value as set
  up by the host bridge in the hardware domain.
  This way hardware doamin sees physical BAR values and guest sees
  emulated ones.

The series also adds support for virtual PCI bus topology for guests:
 - We emulate a single host bridge for the guest, so segment is always 0.
 - The implementation is limited to 32 devices which are allowed on
   a single PCI bus.
 - The virtual bus number is set to 0, so virtual devices are seen
   as embedded endpoints behind the root complex.

The series was also tested on:
 - x86 PVH Dom0 and doesn't break it.
 - x86 HVM with PCI passthrough to DomU and doesn't break it.

Thank you,
Oleksandr

Oleksandr Andrushchenko (11):
  vpci: Make vpci registers removal a dedicated function
  vpci: Add hooks for PCI device assign/de-assign
  vpci/header: Move register assignments from init_bars
  vpci/header: Add and remove register handlers dynamically
  vpci/header: Implement guest BAR register handlers
  vpci/header: Handle p2m range sets per BAR
  vpci/header: program p2m with guest BAR view
  vpci/header: Emulate PCI_COMMAND register for guests
  vpci/header: Reset the command register when adding devices
  vpci: Add initial support for virtual PCI bus topology
  xen/arm: Translate virtual PCI bus topology for guests

 xen/arch/arm/domain.c         |   1 +
 xen/arch/arm/vpci.c           |  86 ++++++-
 xen/arch/arm/vpci.h           |   3 +
 xen/common/domain.c           |   3 +
 xen/drivers/Kconfig           |   4 +
 xen/drivers/passthrough/pci.c |  94 ++++++++
 xen/drivers/vpci/header.c     | 411 +++++++++++++++++++++++++++-------
 xen/drivers/vpci/vpci.c       |  42 +++-
 xen/include/asm-arm/pci.h     |   1 +
 xen/include/xen/pci.h         |  23 ++
 xen/include/xen/sched.h       |  10 +
 xen/include/xen/vpci.h        |  36 ++-
 12 files changed, 623 insertions(+), 91 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199499.353602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrp-0001Q2-DW; Thu, 30 Sep 2021 07:52:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199499.353602; Thu, 30 Sep 2021 07:52:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrp-0001Pt-A7; Thu, 30 Sep 2021 07:52:33 +0000
Received: by outflank-mailman (input) for mailman id 199499;
 Thu, 30 Sep 2021 07:52:32 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqro-00017f-4Y
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:32 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id c0cba9af-d264-4507-8780-0fae988df9d3;
 Thu, 30 Sep 2021 07:52:27 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id j5so16802840lfg.8
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:27 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:25 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0cba9af-d264-4507-8780-0fae988df9d3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=SYPt7pXJxe63RSclQtSSHNmQ1mlJJN2741FNOcAd+O0=;
        b=mmTcp6QmfMwKYK8Ecm2Edk6w8zGbwZLtEjMPv51QDEdZ+/3BWqGxipsHqJioA8AD+X
         G3ak1zkWZvnjKpRTiNDLAzXNhxPype90V/3cZJuA1MaMck62kHkq2dZ6lFDA3DSO3GJI
         md3ktNg1lOpedGpEu6B9dOaK5dCwN7RysoAJHmt2URAL24egrXeHkWlYV6xvbcYgCTeI
         Us9GIm8dupHRsyGaojXSXOPSJe5EF5K1tXmtRxBKrpz0l+kILm93Muf6DbGQiNb52WeI
         MhL8Yd2Knzn0mHaB2u3fD9RkBicAmppviwtWL2u/0tEVROx2igk4cy4ftabtb5zsQmVY
         mK7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=SYPt7pXJxe63RSclQtSSHNmQ1mlJJN2741FNOcAd+O0=;
        b=N0UzxToz2+Qu3odDCXTdVn1qf39LYhwBp2rtZF4zIZZEZ+PU3n4KnHQoVFStNO/S35
         Sh3+mksvwjVptWM2L5xa2ewyYuW42BfMU+riSVP2LAxiSBsB6LGuV5ovNeQi4f4dlqox
         hPXQpaz5ay/LnwIOJLaoPkOi7NY0CKn5DeJ4dNj8gaTyWkBuugagkUPDBNUB7j16v++2
         oyaAXxyDME+XboYjv/V+SmGXvDWq9zXkFSossU6hm/XefEoXE/PTInffbrD7yBFoLfkW
         RA9kZVem+OXwnGyovtUmzUrcGGkBG273vituH8r4ZZwvWddmRNn80K0CLYScwWp3hP0s
         Fqlw==
X-Gm-Message-State: AOAM532AGQCzyVBez6afEDQzG0f1ePyy6KwrBkNcYS2BdEiLGiRKbRWT
	BzdPLQiBqeIXMm/DBO/HFfi1HGknmR0=
X-Google-Smtp-Source: ABdhPJy0PnfpM6dmqJ2wGT69DJkwD/EOCR2NGBPp6wOubDrxAQpc5CiVAiUU5b60uCf9xGGlkW7xfw==
X-Received: by 2002:a05:6512:1691:: with SMTP id bu17mr4290276lfb.401.1632988345818;
        Thu, 30 Sep 2021 00:52:25 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Michal Orzel <michal.orzel@arm.com>
Subject: [PATCH v3 01/11] vpci: Make vpci registers removal a dedicated function
Date: Thu, 30 Sep 2021 10:52:13 +0300
Message-Id: <20210930075223.860329-2-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vpci register
handlers depending on the domain: hwdom or guest.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
Since v1:
 - constify struct pci_dev where possible
---
 xen/drivers/vpci/vpci.c | 7 ++++++-
 xen/include/xen/vpci.h  | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index cbd1bac7fc33..1666402d55b8 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -35,7 +35,7 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
-void vpci_remove_device(struct pci_dev *pdev)
+void vpci_remove_device_registers(const struct pci_dev *pdev)
 {
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
@@ -48,6 +48,11 @@ void vpci_remove_device(struct pci_dev *pdev)
         xfree(r);
     }
     spin_unlock(&pdev->vpci->lock);
+}
+
+void vpci_remove_device(struct pci_dev *pdev)
+{
+    vpci_remove_device_registers(pdev);
     xfree(pdev->vpci->msix);
     xfree(pdev->vpci->msi);
     xfree(pdev->vpci);
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 9f5b5d52e159..2e910d0b1f90 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -28,6 +28,8 @@ int __must_check vpci_add_handlers(struct pci_dev *dev);
 
 /* Remove all handlers and free vpci related structures. */
 void vpci_remove_device(struct pci_dev *pdev);
+/* Remove all handlers for the device given. */
+void vpci_remove_device_registers(const struct pci_dev *pdev);
 
 /* Add/remove a register handler. */
 int __must_check vpci_add_register(struct vpci *vpci,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199500.353614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrs-0001jy-Nz; Thu, 30 Sep 2021 07:52:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199500.353614; Thu, 30 Sep 2021 07:52:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrs-0001jk-JK; Thu, 30 Sep 2021 07:52:36 +0000
Received: by outflank-mailman (input) for mailman id 199500;
 Thu, 30 Sep 2021 07:52:35 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVqrr-0001ht-CV
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:35 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 5ecbac88-21c3-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 07:52:34 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-25-0eng-7IzNMKM_WUJ0eN_CQ-1; Thu, 30 Sep 2021 09:52:31 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4941.eurprd04.prod.outlook.com (2603:10a6:803:5a::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 07:52:29 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 07:52:29 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR04CA0017.eurprd04.prod.outlook.com (2603:10a6:206:1::30) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 07:52:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ecbac88-21c3-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632988353;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZHc27peg13663JzGUs4ZDuwHlmiLMP8fAVjD66xZyOk=;
	b=FOjx8cVVKLzrGncZjhDlR9XgdjbeGrRjW7EXyKiDRnpY+eeEcjjBlxdDVCqJ3Cx5B8T0P8
	CyRF54v6xMjQ6QcIXTbHABzqYdw4Tr37KOSB/7YQiSLQpVUbGH7AbnyDJ1TuyHBg2WoFNC
	RRbU8DZ61xfTlwrH/QFoMuthnhJIIHc=
X-MC-Unique: 0eng-7IzNMKM_WUJ0eN_CQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DVssgNz5JCm3SYPmFQGZ1bIQLaKB6r9lPN226d4GsD885C1eJQwNzwHs07gbmFHIT+rown/qkgfd3NfiLvr32Mjo4qf3ywzFg1Lle0qpu01uNV/Q1asR4CkVBUB1pGbmfOi9L0mgRe/MMRMS4+b9gfLGf46bS+LayYrFhP0POP6CpSdn751gK8U6uSi+Yvjt0HJAoCaYqUdBfwVPohOjV5fgeVTWXVeSsGSwQoQzuk74ytbgOdSSFUXJL5b9Qu/MmqUTJ7I7S3dC4s0eVpkoIx1v+ZBgsY/jH1GZzsl4n2bzv5jFJ4KeWbBXiAR4jtUYYCtqiB1bJ/kRDO2Fd3CaeA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZHc27peg13663JzGUs4ZDuwHlmiLMP8fAVjD66xZyOk=;
 b=oXs3B/hmX1P1tQ9pYSG+nWf1Q6yC5P/OImeT2UZHlE+3h/7mlq9zGvkIPZ9RPjUes/vhzeDABYoKaLPxCAHTb3XHfVIh3CozguPjGTiD3ePeohI0JK03eN6zabLQNnIbIaByDrLdW6BhYR+yxwckC5wf8EvKBiOaKO7cJKR8vA7LJCZMv1qIKVXflsBkD82Joo+JihSvNEP0qR7n7bCZ2gS4MtbXAWWUsZPmaPZGshIncxia9Gcal30cR6EkbHJnT0a9rsrmDH4h76rcoXw9DWF/dSAntfJjRJyV6pgk5+a1WbxVWi7qi5OHn/swzRcSdoZp0GVCDJBXekJbOJhBqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 05/11] xen/arm: Mark device as PCI while creating one
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210930071326.857390-1-andr2000@gmail.com>
 <20210930071326.857390-6-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ecc8024f-d3f1-ed47-fdcf-dd5aca40be97@suse.com>
Date: Thu, 30 Sep 2021 09:52:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930071326.857390-6-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR04CA0017.eurprd04.prod.outlook.com
 (2603:10a6:206:1::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4490a5a3-1f2d-4dd0-6f21-08d983e740c5
X-MS-TrafficTypeDiagnostic: VI1PR04MB4941:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB49411FEF26911C99B0FD0319B3AA9@VI1PR04MB4941.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1360;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rBWyFeW7itBS9WBqW0kxMSeACxpb20F1eWjs24wQdpqqJOrUbpx76q36shblbgeqHbQogdKWWyDjWpBE3EA9ZFB5WII/k22MB/Lh3Z4TSJkfRqH884AT+GtNN+HzKLO/+2YzebuIQ0dMsSkUqUGAfYXSu2PD9iO92LG0OvkrzEEA3VDq1zvi3eyba1G89TxrbSqZyl6c2iDRqJ/dQqsrsNJsQLucEeljfszMOI0bHGYft/EPRAMb8U/EFTCgcFCaQ3mGIyAQC2cmRqv7JkVjR/vwPSPq55l0sTUUs7kIi65M8KbSeqXjycaBIrVJ1Fj5IuYGEVXes+B4m6FGMIStOwkC6PJ7Ad36rjPizo8UtTMC/Uapk4bUGuE3nZtBQwjdjbOhC5s7WcIcnwnaIc6QNKLQopu3UyRxDJFjhx9B8iN7D5X+9hRZUI221tzUD1c0npK6kGzJ52A2xkrk2RDfzxTTHVts796O8DpSPTWYjaQtwdhXQeluSDKctUmZBRuIBotvkHRLemxAWLK924KvFRbeM3PnbrgzaRRoLGV11Ft2jKYdLuPYQ4OiO5BgQ/8i02qS5sSEDYVH9O4IeKe/yM3UQ9Qr3zAkxjH9rpcFCuMJt5avQpMJzSZs5BTQ+tvwOYl6QX5xzmwqw4L3Dlfn9VXz+dR5EqCPX83gDR29yzcwG0cRlaxbutl7QqBruz28qS8E5MqUpGpA70v1HtEHXN3yFU7NmfUStK7N216VLy2MdVoB1S50BDIOSMwyQ7rv
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(6916009)(956004)(5660300002)(66946007)(186003)(6486002)(31686004)(16576012)(38100700002)(4744005)(2616005)(4326008)(31696002)(508600001)(8936002)(8676002)(53546011)(36756003)(66476007)(66556008)(7416002)(2906002)(86362001)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Zy9qbVBBaEZMVTU5UEswMFJ0aDVwMGxhZWpQeWx2R3RBU2ZOZHJBU05aeHd6?=
 =?utf-8?B?NjdQT21wTGZGQzgrY3g5NnI4bGZ2azZLWWJHQmFseXdvTXZnWXFNNm9xODNh?=
 =?utf-8?B?WHBxNDEvWlBSaHRhZEhJZ2VDQmhlY2pvNnB1MC9wZUIyTGcxUFc5dGJPTHZl?=
 =?utf-8?B?RzVzV01TeU1oWHJEUFBpT0FyN29GVnU1V1I2L01kMzBCSmlNSHRpY1Nldmkx?=
 =?utf-8?B?SGtIdHNHaUIyaE5VS0I1TnlvTW51bkVzaElpbS9sc0ZpNUpQNnppR2hTOWxV?=
 =?utf-8?B?dFVzbXZQMHBxUGs4T2JBbjRQT3VIeldRZEdnbDZsc0FJcTB3SWVmTThYdjJP?=
 =?utf-8?B?RE5saTkrQm1WaW9JVU90Zk1selBrbGpKOXIyRmZudmJ3SVA2K043VXNFbzJD?=
 =?utf-8?B?R3N1bGFwZjdENlM5a0dncTB6RVRlUllpRk5QT0dvaWg4c1VVa3BsSlduQnhU?=
 =?utf-8?B?cG5Sa1R6ZE9OZmtudnhlSHhqSXlQcnJLeWVwQ3dvcTlEVFFVNXNOYjNGdm9t?=
 =?utf-8?B?ZUpvVzRVQ2kwSnhIR0NFOXZubDRkZkJ3ZHdWSG9Mc3B1WkFVME5PdWFKVUZJ?=
 =?utf-8?B?SjVESDhNSDl0UlEvVG9iMi9MUFlKVy9nKzNiNHkxOUhSaW5pM2pjU0p1WnM1?=
 =?utf-8?B?TmF2QWwwS2JKWFNFWk1GUEVZQ3NPU2N2aDBwRWR6NFYxM0MxdXR3UjR6MHRK?=
 =?utf-8?B?eUxWMHFNYjNmYVlpeFM5NXVpSW53S3V6VU1SN1hJelFqRTZJSmY1aGJOZnBG?=
 =?utf-8?B?ZEJscWJ5bjFyVkh3TFgvSnJ6aE5YZm5WbTJ2TmdWVnMrdVBjU3pRZUl3ak5H?=
 =?utf-8?B?YmpLSE5YV1hIeXUyeWJ3WXNzR1o2RUtVVTYwSk5xbTlDc3JKa2hlaU1BK0Fy?=
 =?utf-8?B?aXhyQ2dPdE1sZUwrbHBFajBRUHV5Umo2M241MXVkT3V2eTE3UDNYb0JJaVpD?=
 =?utf-8?B?bXJZWE5pMnlGdGNicHJCTi9ZdUh2MmNpSCtqenRKS25ES2IzYW5tKzZnQ3V2?=
 =?utf-8?B?ek4wMlUzQm9RTVZzZElsVnNWZmF1Qk92UUZmYUpoeUs1RnFoVFhhVEZiMmhV?=
 =?utf-8?B?VnNkK0wxYjUrbFpEWkZzcW1Lbm16Mk0wUFhBK2liYTJWbEZ0dHltUmJ3QjZx?=
 =?utf-8?B?amphb2ZQR3JaaGZFRGZ5L3lIU1hQTEgwd2lkajk5R2xlandxUExKQWRUUzc5?=
 =?utf-8?B?eHpSdHFaUHZhT2JFUW9XRWNIbVVsd3loalFWN21RemtBWWRFTE1yY3FkVEMx?=
 =?utf-8?B?MklEUGVKdG82bytuRUV6VFVzVnBjM09jTnJwQ3Nla1RDbVRsWVNKQkdxM0Rs?=
 =?utf-8?B?TDd4cGVZYnUxTjBkY1RFTVprdnVjV1VxcDhweENzN0Fta2pYSm9BVGNrZjVt?=
 =?utf-8?B?OTBWaDBrdmpOZ1Q4UWhERmUyUE5LbncvdkQ1OHg0Mml2MjkrMlV3MVRqdFUv?=
 =?utf-8?B?ZG1WaHFWS3FQYzZYN092T1JjamJtNUk4MFFSbW50MmZpMkNKSHRWUTlVR2dO?=
 =?utf-8?B?TWl2dWhqbmpCZVBJVFQ2aVRTRlVDaG1qZDQ0V1ZhVlEzV0ZBbmJaWTd0N0NH?=
 =?utf-8?B?ZGNlNDlpVXhIUCtpQUhudnNUTW1rY09DL1dGR2JNMlNNVlk3YWl4WDNybHdG?=
 =?utf-8?B?VEFUQ2MrczgzNWNnYmNoRzhqVTNoa3hQbE9QNGxOZ2NiT1JxY051dzFZNWcx?=
 =?utf-8?B?NU9KZGhHWG5rSVlid3FzdmU2YWxJTXVXeEZBSDEvNjZDRG1yOFBQZ1ZYVEhs?=
 =?utf-8?Q?kp+aES19w1db27xRe1U9zp1DLJjjkjipQYvg4Mi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4490a5a3-1f2d-4dd0-6f21-08d983e740c5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 07:52:29.8160
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6a/AsElm2Bm7GLv+KP0gcwEanJO6/dJyp1xRWU8sUQhWJ5ZnawHv0qXA7AtJKa6rHp86k8aVMNj1Yl2cfXsCdQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4941

On 30.09.2021 09:13, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> While adding a PCI device mark it as such, so other frameworks
> can distinguish it from DT devices.
> For that introduce an architecture defined helper which may perform
> additional initialization of the newly created PCI device.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Applicable parts
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:39 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199501.353625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrv-000233-1H; Thu, 30 Sep 2021 07:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199501.353625; Thu, 30 Sep 2021 07:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqru-00022r-TZ; Thu, 30 Sep 2021 07:52:38 +0000
Received: by outflank-mailman (input) for mailman id 199501;
 Thu, 30 Sep 2021 07:52:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqrt-00017f-4s
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:37 +0000
Received: from mail-lf1-x129.google.com (unknown [2a00:1450:4864:20::129])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 31dbb497-6e8f-4456-bbcc-c77af33a62b1;
 Thu, 30 Sep 2021 07:52:28 +0000 (UTC)
Received: by mail-lf1-x129.google.com with SMTP id u18so21640519lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:28 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:26 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31dbb497-6e8f-4456-bbcc-c77af33a62b1
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=7n1ziOkiQyIV9eus199iv91YK2oXQWN+eMmfapyz5UQ=;
        b=lxxe9SG314t15/TZPQBDOd4EQv7HddnV6SUX5ZEEwxxkkQT436vPQX0Xbltf0RLnvL
         ppnWhUbLWnELFmajNIbuSPJfPV6vK+41CSvAxQh9YrG+hmR2t8cKdbhnHfb71yauhfGq
         mqhx4oklREJFi9EDKdEcauypiX1t40XJ3lL7t8a9BPkB2Ub8HL2i3OIOp7vJFwNAO4G8
         Be+o5fbPa3/Z1E/saDKs+dxR4aHFUpPs1HXABqnszlcs4kb5a+LiDzyMQzP2rNjGHIxl
         3MSlHrlxYi3P3d49i2Wlz9Lk4cALL+4kmVo/3Fkj8FLVg2HKNFrzf/Jb9idlhCWVjwrv
         cgfA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=7n1ziOkiQyIV9eus199iv91YK2oXQWN+eMmfapyz5UQ=;
        b=qAIlXeF9ZnaOiG5PW8OZIR7WtPjRs8tOgFrrERI9u/LjmCwJdhEUWMgaLAsMcTlqXC
         Ottu/JD+kO7QGtEbnlusHy32Mp8wJIhHfOegn3n0LBLMQtLpE8qzG2xQ/JJCK+wBdwH8
         UC0SJP0sY6kWOleuklc8ct/qBAebRvZy/Zj03s+ZNBamyr7Z3q4uSUGWmSbbAz+N/U+3
         ifdA6PgiAPhpB1M8/Tl19mt8bLeJjJM1F+Yqg23ZmyEKBkknsZijnr6o8KXoaSG0EgZN
         jPgVzRn6FekDwSWOARglFHvZSZmtsUGpN5v6SoivImUi5OMa/lysNfT/Eg11IdnPEo3h
         /oWw==
X-Gm-Message-State: AOAM533l+l04VojrA0YtzSUYiUF2a1t/gckSr9yrQBTwPhHufxDxeTct
	89vS47y9cPKxMJBBsYqHdzghbIurKv4=
X-Google-Smtp-Source: ABdhPJwdvPHmCq0g0ujvcf3lB0FqKYqVUjUSP/uNbltsIKz6JKcCHEqRNXIjrl7bvplAx1UBUHQHIA==
X-Received: by 2002:ac2:4d10:: with SMTP id r16mr4307930lfi.309.1632988346764;
        Thu, 30 Sep 2021 00:52:26 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Date: Thu, 30 Sep 2021 10:52:14 +0300
Message-Id: <20210930075223.860329-3-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

When a PCI device gets assigned/de-assigned some work on vPCI side needs
to be done for that device. Introduce a pair of hooks so vPCI can handle
that.

Please note, that in the current design the error path is handled by
the toolstack via XEN_DOMCTL_assign_device/XEN_DOMCTL_deassign_device,
so this is why it is acceptable not to de-assign devices if vPCI's
assign fails, e.g. the roll back will be handled on deassign_device when
it is called by the toolstack.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
Since v2:
- define CONFIG_HAS_VPCI_GUEST_SUPPORT so dead code is not compiled
  for x86
Since v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - extended the commit message
---
 xen/drivers/Kconfig           |  4 ++++
 xen/drivers/passthrough/pci.c |  9 +++++++++
 xen/drivers/vpci/vpci.c       | 23 +++++++++++++++++++++++
 xen/include/xen/vpci.h        | 20 ++++++++++++++++++++
 4 files changed, 56 insertions(+)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..780490cf8e39 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	depends on HAS_VPCI
+
 endmenu
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 9f804a50e780..805ab86ed555 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -870,6 +870,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
     if ( ret )
         goto out;
 
+    ret = vpci_deassign_device(d, pdev);
+    if ( ret )
+        goto out;
+
     if ( pdev->domain == hardware_domain  )
         pdev->quarantine = false;
 
@@ -1429,6 +1433,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
         rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
     }
 
+    if ( rc )
+        goto done;
+
+    rc = vpci_assign_device(d, pdev);
+
  done:
     if ( rc )
         printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 1666402d55b8..0fe86cb30d23 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -86,6 +86,29 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
 
     return rc;
 }
+
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+/* Notify vPCI that device is assigned to guest. */
+int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
+{
+    /* It only makes sense to assign for hwdom or guest domain. */
+    if ( is_system_domain(d) || !has_vpci(d) )
+        return 0;
+
+    return 0;
+}
+
+/* Notify vPCI that device is de-assigned from guest. */
+int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
+{
+    /* It only makes sense to de-assign from hwdom or guest domain. */
+    if ( is_system_domain(d) || !has_vpci(d) )
+        return 0;
+
+    return 0;
+}
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 #endif /* __XEN__ */
 
 static int vpci_register_cmp(const struct vpci_register *r1,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 2e910d0b1f90..ecc08f2c0f65 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -242,6 +242,26 @@ static inline bool vpci_process_pending(struct vcpu *v)
 }
 #endif
 
+#if defined(CONFIG_HAS_VPCI) && defined(CONFIG_HAS_VPCI_GUEST_SUPPORT)
+/* Notify vPCI that device is assigned/de-assigned to/from guest. */
+int __must_check vpci_assign_device(struct domain *d,
+                                    const struct pci_dev *dev);
+int __must_check vpci_deassign_device(struct domain *d,
+                                      const struct pci_dev *dev);
+#else
+static inline int vpci_assign_device(struct domain *d,
+                                     const struct pci_dev *dev)
+{
+    return 0;
+};
+
+static inline int vpci_deassign_device(struct domain *d,
+                                       const struct pci_dev *dev)
+{
+    return 0;
+};
+#endif
+
 #endif
 
 /*
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199502.353636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrz-0002V9-Ei; Thu, 30 Sep 2021 07:52:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199502.353636; Thu, 30 Sep 2021 07:52:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqrz-0002Uq-9F; Thu, 30 Sep 2021 07:52:43 +0000
Received: by outflank-mailman (input) for mailman id 199502;
 Thu, 30 Sep 2021 07:52:42 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqry-00017f-5G
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:42 +0000
Received: from mail-lf1-x135.google.com (unknown [2a00:1450:4864:20::135])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a220379f-c5e5-450e-bfa6-05edd90ae1a3;
 Thu, 30 Sep 2021 07:52:29 +0000 (UTC)
Received: by mail-lf1-x135.google.com with SMTP id y26so21640122lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:29 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:27 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a220379f-c5e5-450e-bfa6-05edd90ae1a3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=TDiVVp/85Hu7qvLwikbqJ56SNZPH692g3qboO1XLKAU=;
        b=gq+EMwDmDw1J4eB8taY5f3+FOEntO+piewAuZeunn1C+ww+Gpe4FN6A4SMAuc96WSi
         WtQb6Bg6gpOBRKJNNFp/GdB3g/+CvNwEKOkYRPT+HN2l3zvRpuel0N2cCvaeJfo1T/k1
         YFpWFuOYmjRVr2Ee5iap9oNkIwElPkIaVMFIKflEslmtC+vym0qPnktCmae2FHOiacOr
         2AuMTN6SlkIBODhZSgOzQZfCJVkyKafhUqestf/egmoF+TfG5jrdIOJPPx7sqzz2VPiP
         TIb3PHHBsyEVScDfFPrGm0R+4TXfcCcm91GNOCLD58aT7vvr8Y1QsCDVd6eR5MDPNEwG
         x+sQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=TDiVVp/85Hu7qvLwikbqJ56SNZPH692g3qboO1XLKAU=;
        b=LR1rSyifsQztEPc0TgMjEqRgA4ybpWVlQyoWxTw0gG0jLcapWJE1pT2D/+ilkQRAYn
         Jcd2EPU8I6lehQ43VIxt76Ywc5pvN9y0nAnusB67styG/+2RiRT06dwX/MaL4ir0VcN3
         RWYK4FwtysE87GPlu4kpMolMl5aMIqyCsr2oMerDS8eBkWFWqxL3woc2PERtquOeEwuV
         RT3rZYRaVLLYtiZixhn3O1Sdm/tDIX11UXKa+mvP4OhnFjuf+/8jjlSw4jvIgvGFxXoD
         Lj+hoJ4LyT98C8vof2cUkx6TmJgCCuiHK+1kWtYhRhlssVdOKQpW1g9KuP4TJ0iI83M5
         +/WQ==
X-Gm-Message-State: AOAM532DC6hEznrqSCLl4bubWtZjT+3Ip5rRLViZOAugqWTvZqd5QXMn
	ckDHuWbzNtZ5yqMRbNuGFYm0qoC4yxQ=
X-Google-Smtp-Source: ABdhPJwpuWaTde9OK4X7N00bG/MnIC2YJvlxlY2pxfCMW92/HGzZzZm71JgEX/xw/8AO8tnK5RzPZQ==
X-Received: by 2002:a05:6512:31cb:: with SMTP id j11mr4500919lfe.299.1632988347903;
        Thu, 30 Sep 2021 00:52:27 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 03/11] vpci/header: Move register assignments from init_bars
Date: Thu, 30 Sep 2021 10:52:15 +0300
Message-Id: <20210930075223.860329-4-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

This is in preparation for dynamic assignment of the vPCI register
handlers depending on the domain: hwdom or guest.
The need for this step is that it is easier to have all related functionality
put at one place. When the subsequent patches add decisions on which
handlers to install, e.g. hwdom or guest handlers, then this is easily
achievable.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v1:
 - constify struct pci_dev where possible
 - extend patch description
---
 xen/drivers/vpci/header.c | 83 ++++++++++++++++++++++++++-------------
 1 file changed, 56 insertions(+), 27 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index f8cd55e7c024..3d571356397a 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -445,6 +445,55 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
+static int add_bar_handlers(const struct pci_dev *pdev)
+{
+    unsigned int i;
+    struct vpci_header *header = &pdev->vpci->header;
+    struct vpci_bar *bars = header->bars;
+    int rc;
+
+    /* Setup a handler for the command register. */
+    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
+                           2, header);
+    if ( rc )
+        return rc;
+
+    if ( pdev->ignore_bars )
+        return 0;
+
+    for ( i = 0; i < PCI_HEADER_NORMAL_NR_BARS + 1; i++ )
+    {
+        if ( (bars[i].type == VPCI_BAR_IO) || (bars[i].type == VPCI_BAR_EMPTY) )
+            continue;
+
+        if ( bars[i].type == VPCI_BAR_ROM )
+        {
+            unsigned int rom_reg;
+            uint8_t header_type = pci_conf_read8(pdev->sbdf,
+                                                 PCI_HEADER_TYPE) & 0x7f;
+            if ( header_type == PCI_HEADER_TYPE_NORMAL )
+                rom_reg = PCI_ROM_ADDRESS;
+            else
+                rom_reg = PCI_ROM_ADDRESS1;
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                   rom_reg, 4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+        else
+        {
+            uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
+
+            /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
+            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
+                                   4, &bars[i]);
+            if ( rc )
+                return rc;
+        }
+    }
+    return 0;
+}
+
 static int init_bars(struct pci_dev *pdev)
 {
     uint16_t cmd;
@@ -470,14 +519,8 @@ static int init_bars(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
-    if ( rc )
-        return rc;
-
     if ( pdev->ignore_bars )
-        return 0;
+        return add_bar_handlers(pdev);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -492,14 +535,6 @@ static int init_bars(struct pci_dev *pdev)
         if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO )
         {
             bars[i].type = VPCI_BAR_MEM64_HI;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
-            if ( rc )
-            {
-                pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-                return rc;
-            }
-
             continue;
         }
 
@@ -532,14 +567,6 @@ static int init_bars(struct pci_dev *pdev)
         bars[i].addr = addr;
         bars[i].size = size;
         bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH;
-
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4,
-                               &bars[i]);
-        if ( rc )
-        {
-            pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-            return rc;
-        }
     }
 
     /* Check expansion ROM. */
@@ -553,11 +580,13 @@ static int init_bars(struct pci_dev *pdev)
         rom->addr = addr;
         header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) &
                               PCI_ROM_ADDRESS_ENABLE;
+    }
 
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg,
-                               4, rom);
-        if ( rc )
-            rom->type = VPCI_BAR_EMPTY;
+    rc = add_bar_handlers(pdev);
+    if ( rc )
+    {
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+        return rc;
     }
 
     return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:48 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199505.353647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqs4-00032P-O3; Thu, 30 Sep 2021 07:52:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199505.353647; Thu, 30 Sep 2021 07:52:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqs4-00032G-KW; Thu, 30 Sep 2021 07:52:48 +0000
Received: by outflank-mailman (input) for mailman id 199505;
 Thu, 30 Sep 2021 07:52:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqs3-00017f-59
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:47 +0000
Received: from mail-lf1-x12e.google.com (unknown [2a00:1450:4864:20::12e])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 85314c14-274d-4084-803c-7360f29354b5;
 Thu, 30 Sep 2021 07:52:30 +0000 (UTC)
Received: by mail-lf1-x12e.google.com with SMTP id z24so21538335lfu.13
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:30 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:28 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85314c14-274d-4084-803c-7360f29354b5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=+lvSnXHbydzAtV18JqDZDz/Q4JuLo0GbEWE975rivV4=;
        b=RvBF4VUM1Fk+E+WSGa+N7HuxNUkCF1yYamFAn+WCByN3h1ck/uPr2ThjCz3Htwvszv
         JD7MB/R4pYL7vd/scgYTgNkJQNgCjwrlCskorlXeGcafl9saDZeKdSlCK5sx/iGgPWvY
         FkluY894cb3vxLxpgDS+LhXM5DGQFst3R2A2zGIicFMY2V7MJBsqu2XOTQxDLsd6H5pM
         t1WQYQPTn9FHiTj+oWpuU/ilwnQXWAgf3VsRffW0cmEJT83rP2+h6O3y50MoDxSQrDCI
         C7t/ulzm7EjOmIoaSP5sIFPxPdDCdYSQ0tw1XpQfjFH7PR5oAVZvTbzkejD+3t+qJxQp
         d5ow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=+lvSnXHbydzAtV18JqDZDz/Q4JuLo0GbEWE975rivV4=;
        b=bbEmUZqfG+Et0o4TqJfZoETZ3j1sfaPnfQtC8JPViwOC1MmlocMDkNl1ZlV7X/92n8
         0oVUpug0pvdlv85Yy2CvXwwtT7lCS7TMvRgN30XkBwZcSC/m/Aq2Z0HfJ2FdL63eoypu
         ipoomkjank1MhNl0n0RQijxB08e3Zb29ItFXpb4RM8X6KFBiJqcwuQyfEiNxOLmAHUcy
         tTfBn8z+KSV5L+uz3RMO7dbaflnCVI96QQIa3uxUA5itrnweDXLDAliYM4SNiTOjDG65
         CDSFsCLh8tfzMtTCyjg7yACmThSzqgwdktchlHHmeqI+daFESFPP5yqJBtu6H865HL1Q
         jIgg==
X-Gm-Message-State: AOAM530eR/hB3bG2uFbWDeQXOAPHE/Hn7DKEtKcHACX/RiRc1EVZ15gr
	s5OkVFC4/pFccrlmLyQcIsSPyQbkFGY=
X-Google-Smtp-Source: ABdhPJz0Z246UwdyliQVLaHOSo2T5zlzJH0n8U8wqH0wwto1FrUvKVB6rbV1sDwYhVfQlVl9ywO6xQ==
X-Received: by 2002:a05:6512:b8f:: with SMTP id b15mr4596943lfv.655.1632988348949;
        Thu, 30 Sep 2021 00:52:28 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 04/11] vpci/header: Add and remove register handlers dynamically
Date: Thu, 30 Sep 2021 10:52:16 +0300
Message-Id: <20210930075223.860329-5-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add relevant vpci register handlers when assigning PCI device to a domain
and remove those when de-assigning. This allows having different
handlers for different domains, e.g. hwdom and other guests.

Use stubs for guest domains for now.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
- remove unneeded ifdefs for CONFIG_HAS_VPCI_GUEST_SUPPORT as more code
  has been eliminated from being built on x86
Since v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - simplify some code3. simplify
 - use gdprintk + error code instead of gprintk
 - gate vpci_bar_{add|remove}_handlers with CONFIG_HAS_VPCI_GUEST_SUPPORT,
   so these do not get compiled for x86
 - removed unneeded is_system_domain check
---
 xen/drivers/vpci/header.c | 72 ++++++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c   |  4 +--
 xen/include/xen/vpci.h    |  8 +++++
 3 files changed, 74 insertions(+), 10 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 3d571356397a..1ce98795fcca 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -397,6 +397,17 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
     pci_conf_write32(pdev->sbdf, reg, val);
 }
 
+static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
                       uint32_t val, void *data)
 {
@@ -445,14 +456,25 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
-static int add_bar_handlers(const struct pci_dev *pdev)
+static void guest_rom_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t val, void *data)
+{
+}
+
+static uint32_t guest_rom_read(const struct pci_dev *pdev, unsigned int reg,
+                               void *data)
+{
+    return 0xffffffff;
+}
+
+static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
 {
     unsigned int i;
     struct vpci_header *header = &pdev->vpci->header;
     struct vpci_bar *bars = header->bars;
     int rc;
 
-    /* Setup a handler for the command register. */
+    /* Setup a handler for the command register: same for hwdom and guests. */
     rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
                            2, header);
     if ( rc )
@@ -475,8 +497,13 @@ static int add_bar_handlers(const struct pci_dev *pdev)
                 rom_reg = PCI_ROM_ADDRESS;
             else
                 rom_reg = PCI_ROM_ADDRESS1;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
-                                   rom_reg, 4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write,
+                                       rom_reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_rom_read, guest_rom_write,
+                                       rom_reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -485,8 +512,13 @@ static int add_bar_handlers(const struct pci_dev *pdev)
             uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
 
             /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
+            if ( is_hwdom )
+                rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write,
+                                       reg, 4, &bars[i]);
+            else
+                rc = vpci_add_register(pdev->vpci,
+                                       guest_bar_read, guest_bar_write,
+                                       reg, 4, &bars[i]);
             if ( rc )
                 return rc;
         }
@@ -520,7 +552,7 @@ static int init_bars(struct pci_dev *pdev)
     }
 
     if ( pdev->ignore_bars )
-        return add_bar_handlers(pdev);
+        return add_bar_handlers(pdev, true);
 
     /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
@@ -582,7 +614,7 @@ static int init_bars(struct pci_dev *pdev)
                               PCI_ROM_ADDRESS_ENABLE;
     }
 
-    rc = add_bar_handlers(pdev);
+    rc = add_bar_handlers(pdev, true);
     if ( rc )
     {
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
@@ -593,6 +625,30 @@ static int init_bars(struct pci_dev *pdev)
 }
 REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+int vpci_bar_add_handlers(const struct domain *d, const struct pci_dev *pdev)
+{
+    int rc;
+
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+
+    rc = add_bar_handlers(pdev, is_hardware_domain(d));
+    if ( rc )
+        gdprintk(XENLOG_ERR,
+                 "%pp: failed to add BAR handlers for dom%pd: %d\n",
+                 &pdev->sbdf, d, rc);
+    return rc;
+}
+
+int vpci_bar_remove_handlers(const struct domain *d, const struct pci_dev *pdev)
+{
+    /* Remove previously added registers. */
+    vpci_remove_device_registers(pdev);
+    return 0;
+}
+#endif
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 0fe86cb30d23..702f7b5d5dda 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -95,7 +95,7 @@ int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
-    return 0;
+    return vpci_bar_add_handlers(d, dev);
 }
 
 /* Notify vPCI that device is de-assigned from guest. */
@@ -105,7 +105,7 @@ int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
-    return 0;
+    return vpci_bar_remove_handlers(d, dev);
 }
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index ecc08f2c0f65..fd822c903af5 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -57,6 +57,14 @@ uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg,
  */
 bool __must_check vpci_process_pending(struct vcpu *v);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+/* Add/remove BAR handlers for a domain. */
+int vpci_bar_add_handlers(const struct domain *d,
+                          const struct pci_dev *pdev);
+int vpci_bar_remove_handlers(const struct domain *d,
+                             const struct pci_dev *pdev);
+#endif
+
 struct vpci {
     /* List of vPCI handlers for a device. */
     struct list_head handlers;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199509.353658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsA-0003aI-4H; Thu, 30 Sep 2021 07:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199509.353658; Thu, 30 Sep 2021 07:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqs9-0003a6-VA; Thu, 30 Sep 2021 07:52:53 +0000
Received: by outflank-mailman (input) for mailman id 199509;
 Thu, 30 Sep 2021 07:52:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqs8-00017f-5R
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:52 +0000
Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id fac7a3cc-26c2-4c47-b457-e24184136f4b;
 Thu, 30 Sep 2021 07:52:31 +0000 (UTC)
Received: by mail-lf1-x134.google.com with SMTP id i25so21624009lfg.6
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:31 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:29 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fac7a3cc-26c2-4c47-b457-e24184136f4b
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=rVtZpNhlXnSzmfbhCEL+IMfyjOh3bF01cvc201QQbmU=;
        b=hE2uDfRL1hRLRQVqes0DgETsiFrCAaEESlXXIrx14t2k2qtoniLHSCW3Vh+IJBx0sV
         jR0frYyDX02OCrbrTE0+xVaPoiG7IBmARxKBGlO5xYlEvM1AHpt0VP/5OC6FZ+KhNcMi
         gLauGCou1utr94NVGYrqj+DOhs2vnmXf1MkyUH27TcN8WpAPi2tgfH33I2CQAnD49Isu
         HsGrimm/gnCM8k41GqEEOr4htxMX71j4IjAfrqNaEroHyZf1hWcqdhMig7NbiSTdwNiI
         wCs6hRdLHt0TNGcleVYUxDr/4DzDcZ7eMqnNAytWzR8U8xqPWjt4FEM1wIY5Q7LmCJac
         EYPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=rVtZpNhlXnSzmfbhCEL+IMfyjOh3bF01cvc201QQbmU=;
        b=glbScFWEZDA+TYr+elkg1ROQANJyYANvl8qMqP8kY3kbIpJjiVnkM73aTq34QOGHMJ
         4PcyX7YNLn8xRBSlVTQCdReRF9MfiuuJUmlGQgqiTu+s9y8nfPtjhXNZwPP4Fe8a0Pjv
         RCdje38P/o5nwjNoTfIriQr1wVByAtKNGPo0K7Z6wdaBE8PXA5Cn+4tgcCgdjr9lHnYV
         aUW8ld7pwyeMJKUT+BjUXdHWpxg4K7O/cHwHhwtiiB+/YXjmB0A7pN+kbIWvwTuWg0xj
         gYO6y8YQqqoNaRBR6sxoi5I1ctqWlSNf/xjSDW039bImdMcXF5l/yB0XnuCH+Td/vj+f
         49KQ==
X-Gm-Message-State: AOAM531WkyQTPqB+nP4DVOAQis7qOhK2W1NAi9Q9/nXL1+/+k0XigMAR
	FUXMunkiHtfzmkdbLbsQR1qOVwkeu38=
X-Google-Smtp-Source: ABdhPJy3FaYBUB50uoi6zzCEs665+xGWg3WT1AJusJrIM7oB7ma6dHvZ/NwDWDbmw2T0DPxHmRZ1Kw==
X-Received: by 2002:a05:6512:12d6:: with SMTP id p22mr4595737lfg.42.1632988350031;
        Thu, 30 Sep 2021 00:52:30 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Michal Orzel <michal.orzel@arm.com>
Subject: [PATCH v3 05/11] vpci/header: Implement guest BAR register handlers
Date: Thu, 30 Sep 2021 10:52:17 +0300
Message-Id: <20210930075223.860329-6-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Emulate guest BAR register values: this allows creating a guest view
of the registers and emulates size and properties probe as it is done
during PCI device enumeration by the guest.

ROM BAR is only handled for the hardware domain and for guest domains
there is a stub: at the moment PCI expansion ROM is x86 only, so it
might not be used by other architectures without emulating x86. Other
use-cases may include using that expansion ROM before Xen boots, hence
no emulation is needed in Xen itself. Or when a guest wants to use the
ROM code which seems to be rare.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
Since v1:
 - re-work guest read/write to be much simpler and do more work on write
   than read which is expected to be called more frequently
 - removed one too obvious comment

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 30 +++++++++++++++++++++++++++++-
 xen/include/xen/vpci.h    |  3 +++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 1ce98795fcca..ec4d215f36ff 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -400,12 +400,38 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
 static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg,
                             uint32_t val, void *data)
 {
+    struct vpci_bar *bar = data;
+    bool hi = false;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+    {
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        hi = true;
+    }
+    else
+    {
+        val &= PCI_BASE_ADDRESS_MEM_MASK;
+        val |= bar->type == VPCI_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32
+                                           : PCI_BASE_ADDRESS_MEM_TYPE_64;
+        val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
+    }
+
+    bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0));
+    bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0);
+
+    bar->guest_addr &= ~(bar->size - 1) | ~PCI_BASE_ADDRESS_MEM_MASK;
 }
 
 static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg,
                                void *data)
 {
-    return 0xffffffff;
+    const struct vpci_bar *bar = data;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+        return bar->guest_addr >> 32;
+
+    return bar->guest_addr;
 }
 
 static void rom_write(const struct pci_dev *pdev, unsigned int reg,
@@ -522,6 +548,8 @@ static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
             if ( rc )
                 return rc;
         }
+
+        bars[i].guest_addr = 0;
     }
     return 0;
 }
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index fd822c903af5..a0320b22cb36 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -75,7 +75,10 @@ struct vpci {
     struct vpci_header {
         /* Information about the PCI BARs of this device. */
         struct vpci_bar {
+            /* Physical view of the BAR. */
             uint64_t addr;
+            /* Guest view of the BAR. */
+            uint64_t guest_addr;
             uint64_t size;
             enum {
                 VPCI_BAR_EMPTY,
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:52:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:52:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199512.353669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsF-00049J-Df; Thu, 30 Sep 2021 07:52:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199512.353669; Thu, 30 Sep 2021 07:52:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsF-000497-87; Thu, 30 Sep 2021 07:52:59 +0000
Received: by outflank-mailman (input) for mailman id 199512;
 Thu, 30 Sep 2021 07:52:57 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsD-00017f-5g
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:52:57 +0000
Received: from mail-lf1-x132.google.com (unknown [2a00:1450:4864:20::132])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id b232defa-7f1f-4f21-a814-d8a592319df9;
 Thu, 30 Sep 2021 07:52:32 +0000 (UTC)
Received: by mail-lf1-x132.google.com with SMTP id e15so21624994lfr.10
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:32 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:30 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b232defa-7f1f-4f21-a814-d8a592319df9
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=UYb/ANGHKxQ/isBZ8jDr5PIwHDVyjvEKH2Wdujmg3Lc=;
        b=WsHQwrDr4Uem1Ds+1pFynepoQjyxeZ3K137PcLJmgDm7Mrrm1d4dj6crZNwwcehEsM
         rGxflMhwppxr+6Em7Ci3Czif9QiZf01ignkNZku1A2kDqHxa5dAstqUNVHxYq4DrTdAe
         FA6fxmriL3u0ohz55idRcIQB7Hof+Ysk65j+f8vxzl9zHHj/ESrkw3INkTBsub6Nc2uN
         ZgEi2Q2Y+Uqdgi7FhVG5x/sfpr1/QFBKHfNzcXPy4itnubQpBs585W8SvHSQF0HeBWNK
         wRF+reCZw69rw/RCSITlGRJwLzXYfOEH4H0fLtextUzkDroqcXaPYTayGMsYeLMXoG5M
         tfiw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=UYb/ANGHKxQ/isBZ8jDr5PIwHDVyjvEKH2Wdujmg3Lc=;
        b=4zAgsL6XanX8+iKOPhsRXjsQlYX/hUouKk6oyNSNTbs6sb8Rl5/b1TFX+WGj6CQ04t
         6602IAcLo65bYVIWWmNAkBiSJwyw9Iw8o1cKQBDogATvh3dACclIxwFFCD/VyfBE2ZYI
         JQeBOLMvJd+T1BLFPvIA5T4FnyL1NrKbw7JRvMLA3Ng+rnSzgmNa39lgmsRuQkmm+urB
         3fA9SFbay3JoPM3ifX6n4GJqSNcajW098UjhYaofQhCaFCln6yFxbIDYJDKbhaUuTpNN
         B7ga0FmEYwq2qfVg6C8PhFxmjjfDIZODhirwWtdoM/ZmNgnG3/2FmgCargyb6G35vGsI
         /9wg==
X-Gm-Message-State: AOAM530O8wjJY05sKi9HCdsCe+GuNhpnXAA0V5SrKyb0WQIlTHiFx2Sf
	B7pducG1RTtofNuGpj3lWTJONTp0g68=
X-Google-Smtp-Source: ABdhPJxtPd0OLutL4StYyqSyghgjofxpr4rhGLOzhT+xWd9gDCurFN9nBS2/pZyCBlB8otMRHCex5g==
X-Received: by 2002:a05:6512:c9:: with SMTP id c9mr4363608lfp.142.1632988351070;
        Thu, 30 Sep 2021 00:52:31 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 06/11] vpci/header: Handle p2m range sets per BAR
Date: Thu, 30 Sep 2021 10:52:18 +0300
Message-Id: <20210930075223.860329-7-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Instead of handling a single range set, that contains all the memory
regions of all the BARs and ROM, have them per BAR.

This is in preparation of making non-identity mappings in p2m for the
MMIOs/ROM.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
---
 xen/drivers/vpci/header.c | 172 ++++++++++++++++++++++++++------------
 xen/include/xen/vpci.h    |   3 +-
 2 files changed, 122 insertions(+), 53 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index ec4d215f36ff..9c603d26d302 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -131,49 +131,75 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
 
 bool vpci_process_pending(struct vcpu *v)
 {
-    if ( v->vpci.mem )
+    if ( v->vpci.num_mem_ranges )
     {
         struct map_data data = {
             .d = v->domain,
             .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
         };
-        int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data);
+        struct pci_dev *pdev = v->vpci.pdev;
+        struct vpci_header *header = &pdev->vpci->header;
+        unsigned int i;
 
-        if ( rc == -ERESTART )
-            return true;
+        for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+        {
+            struct vpci_bar *bar = &header->bars[i];
+            int rc;
 
-        spin_lock(&v->vpci.pdev->vpci->lock);
-        /* Disable memory decoding unconditionally on failure. */
-        modify_decoding(v->vpci.pdev,
-                        rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
-                        !rc && v->vpci.rom_only);
-        spin_unlock(&v->vpci.pdev->vpci->lock);
+            if ( !bar->mem )
+                continue;
 
-        rangeset_destroy(v->vpci.mem);
-        v->vpci.mem = NULL;
-        if ( rc )
-            /*
-             * FIXME: in case of failure remove the device from the domain.
-             * Note that there might still be leftover mappings. While this is
-             * safe for Dom0, for DomUs the domain will likely need to be
-             * killed in order to avoid leaking stale p2m mappings on
-             * failure.
-             */
-            vpci_remove_device(v->vpci.pdev);
+            rc = rangeset_consume_ranges(bar->mem, map_range, &data);
+
+            if ( rc == -ERESTART )
+                return true;
+
+            spin_lock(&pdev->vpci->lock);
+            /* Disable memory decoding unconditionally on failure. */
+            modify_decoding(pdev,
+                            rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
+                            !rc && v->vpci.rom_only);
+            spin_unlock(&pdev->vpci->lock);
+
+            rangeset_destroy(bar->mem);
+            bar->mem = NULL;
+            v->vpci.num_mem_ranges--;
+            if ( rc )
+                /*
+                 * FIXME: in case of failure remove the device from the domain.
+                 * Note that there might still be leftover mappings. While this is
+                 * safe for Dom0, for DomUs the domain will likely need to be
+                 * killed in order to avoid leaking stale p2m mappings on
+                 * failure.
+                 */
+                vpci_remove_device(pdev);
+        }
     }
 
     return false;
 }
 
 static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
-                            struct rangeset *mem, uint16_t cmd)
+                            uint16_t cmd)
 {
     struct map_data data = { .d = d, .map = true };
-    int rc;
+    struct vpci_header *header = &pdev->vpci->header;
+    int rc = 0;
+    unsigned int i;
+
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
 
-    while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
-        process_pending_softirqs();
-    rangeset_destroy(mem);
+        if ( !bar->mem )
+            continue;
+
+        while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
+                                              &data)) == -ERESTART )
+            process_pending_softirqs();
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
     if ( !rc )
         modify_decoding(pdev, cmd, false);
 
@@ -181,7 +207,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
 }
 
 static void defer_map(struct domain *d, struct pci_dev *pdev,
-                      struct rangeset *mem, uint16_t cmd, bool rom_only)
+                      uint16_t cmd, bool rom_only, uint8_t num_mem_ranges)
 {
     struct vcpu *curr = current;
 
@@ -192,9 +218,9 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
      * started for the same device if the domain is not well-behaved.
      */
     curr->vpci.pdev = pdev;
-    curr->vpci.mem = mem;
     curr->vpci.cmd = cmd;
     curr->vpci.rom_only = rom_only;
+    curr->vpci.num_mem_ranges = num_mem_ranges;
     /*
      * Raise a scheduler softirq in order to prevent the guest from resuming
      * execution with pending mapping operations, to trigger the invocation
@@ -206,42 +232,47 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
 static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
 {
     struct vpci_header *header = &pdev->vpci->header;
-    struct rangeset *mem = rangeset_new(NULL, NULL, 0);
     struct pci_dev *tmp, *dev = NULL;
     const struct vpci_msix *msix = pdev->vpci->msix;
-    unsigned int i;
+    unsigned int i, j;
     int rc;
-
-    if ( !mem )
-        return -ENOMEM;
+    uint8_t num_mem_ranges;
 
     /*
-     * Create a rangeset that represents the current device BARs memory region
+     * Create a rangeset per BAR that represents the current device memory region
      * and compare it against all the currently active BAR memory regions. If
      * an overlap is found, subtract it from the region to be mapped/unmapped.
      *
-     * First fill the rangeset with all the BARs of this device or with the ROM
+     * First fill the rangesets with all the BARs of this device or with the ROM
      * BAR only, depending on whether the guest is toggling the memory decode
      * bit of the command register, or the enable bit of the ROM BAR register.
      */
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
-        const struct vpci_bar *bar = &header->bars[i];
+        struct vpci_bar *bar = &header->bars[i];
         unsigned long start = PFN_DOWN(bar->addr);
         unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
+        bar->mem = NULL;
+
         if ( !MAPPABLE_BAR(bar) ||
              (rom_only ? bar->type != VPCI_BAR_ROM
                        : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) )
             continue;
 
-        rc = rangeset_add_range(mem, start, end);
+        bar->mem = rangeset_new(NULL, NULL, 0);
+        if ( !bar->mem )
+        {
+            rc = -ENOMEM;
+            goto fail;
+        }
+
+        rc = rangeset_add_range(bar->mem, start, end);
         if ( rc )
         {
             printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n",
                    start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            goto fail;
         }
     }
 
@@ -252,14 +283,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
         unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) +
                                      vmsix_table_size(pdev->vpci, i) - 1);
 
-        rc = rangeset_remove_range(mem, start, end);
-        if ( rc )
+        for ( j = 0; j < ARRAY_SIZE(header->bars); j++ )
         {
-            printk(XENLOG_G_WARNING
-                   "Failed to remove MSIX table [%lx, %lx]: %d\n",
-                   start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            const struct vpci_bar *bar = &header->bars[j];
+
+            if ( !bar->mem )
+                continue;
+
+            rc = rangeset_remove_range(bar->mem, start, end);
+            if ( rc )
+            {
+                printk(XENLOG_G_WARNING
+                       "Failed to remove MSIX table [%lx, %lx]: %d\n",
+                       start, end, rc);
+                goto fail;
+            }
         }
     }
 
@@ -291,7 +329,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             unsigned long start = PFN_DOWN(bar->addr);
             unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
-            if ( !bar->enabled || !rangeset_overlaps_range(mem, start, end) ||
+            if ( !bar->enabled ||
+                 !rangeset_overlaps_range(bar->mem, start, end) ||
                  /*
                   * If only the ROM enable bit is toggled check against other
                   * BARs in the same device for overlaps, but not against the
@@ -300,13 +339,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
                  (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) )
                 continue;
 
-            rc = rangeset_remove_range(mem, start, end);
+            rc = rangeset_remove_range(bar->mem, start, end);
             if ( rc )
             {
                 printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n",
                        start, end, rc);
-                rangeset_destroy(mem);
-                return rc;
+                goto fail;
             }
         }
     }
@@ -324,12 +362,42 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
          * will always be to establish mappings and process all the BARs.
          */
         ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only);
-        return apply_map(pdev->domain, pdev, mem, cmd);
+        return apply_map(pdev->domain, pdev, cmd);
     }
 
-    defer_map(dev->domain, dev, mem, cmd, rom_only);
+    /* Find out how many memory ranges has left after MSI and overlaps. */
+    num_mem_ranges = 0;
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        if ( !rangeset_is_empty(bar->mem) )
+            num_mem_ranges++;
+    }
+
+    /*
+     * There are cases when PCI device, root port for example, has neither
+     * memory space nor IO. In this case PCI command register write is
+     * missed resulting in the underlying PCI device not functional, so:
+     *   - if there are no regions write the command register now
+     *   - if there are regions then defer work and write later on
+     */
+    if ( !num_mem_ranges )
+        pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+    else
+        defer_map(dev->domain, dev, cmd, rom_only, num_mem_ranges);
 
     return 0;
+
+fail:
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+
+        rangeset_destroy(bar->mem);
+        bar->mem = NULL;
+    }
+    return rc;
 }
 
 static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index a0320b22cb36..352e02d0106d 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -80,6 +80,7 @@ struct vpci {
             /* Guest view of the BAR. */
             uint64_t guest_addr;
             uint64_t size;
+            struct rangeset *mem;
             enum {
                 VPCI_BAR_EMPTY,
                 VPCI_BAR_IO,
@@ -154,9 +155,9 @@ struct vpci {
 
 struct vpci_vcpu {
     /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */
-    struct rangeset *mem;
     struct pci_dev *pdev;
     uint16_t cmd;
+    uint8_t num_mem_ranges;
     bool rom_only : 1;
 };
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:53:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:53:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199515.353680 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsJ-0004kS-Su; Thu, 30 Sep 2021 07:53:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199515.353680; Thu, 30 Sep 2021 07:53:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsJ-0004kB-PD; Thu, 30 Sep 2021 07:53:03 +0000
Received: by outflank-mailman (input) for mailman id 199515;
 Thu, 30 Sep 2021 07:53:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsI-00017f-5o
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:53:02 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e1af39ec-0bab-4c47-980f-847a81a9d1a4;
 Thu, 30 Sep 2021 07:52:33 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id u18so21641295lfd.12
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:33 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:31 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1af39ec-0bab-4c47-980f-847a81a9d1a4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=R0Ou982lM6u92fk8c4IDhUUqsj1bOg2PvsoDj71PTZg=;
        b=HsfB4Yy6HLz+R+wFcmkgwTUbBCHnW6SkBpWvFqaBK2rAHPBJrw+xeCSH9432+DJZo9
         5ve06kx9L3dX7UtDZKq9DA3nlIi+bE14CIY97SbUeqz6i03Z+ohPE3k/beNTmLdolK/A
         oPfJd7wVYViAm5aNhgGsL+P2IVLL4FnXl0D3CFUcwIdOU/c3OfZiSxwE4Qmq4Hw18HUC
         U1GbiFMFKmbGs5jrbj/BMKDR61jpPpfjdkPLZdwyU7R+bsxohUhkL+RQDuKKgMlhHzVz
         hY3NPrrGoFrgkWyUzztwtk2tTgCkuWp9zS/RmsNONtd1nRCdlfWBLfkk1sXdN3z2Ocjj
         AnMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=R0Ou982lM6u92fk8c4IDhUUqsj1bOg2PvsoDj71PTZg=;
        b=LHSMg9E1DeReWQlODAsw4mEQRzO7Fchdkl89r8vF45fbHB7q7xCFfl2yZW5EjyNbN/
         mUSOYdvK7tdqgT2VSY/T2OynokkR7/hFmItJUXWE/uFOq/hDL9kd82UkVtj0jE+tpDkQ
         EsHFaf5fuQaH70CjyWLyUeEwArhNQwgXfT+xxtn5VGmZ9pedIdyfulFa/gE+m+e8M4aS
         E3RWgqekfnkzpjtjaAy2s/GqvnhqgmCSjEpOLrx8JvF0qO3XUs1njEvh8/518hlpsDRA
         +ujBFrO80/76xcBSsfHAdlQrXzhOGDy5nPpqsBvz8B9MwnEfBtnBaIniSPmvjtROQDgJ
         alWg==
X-Gm-Message-State: AOAM531nOLfC8j/QX3xh8988aPlJc3GnU51fMVFqj8qutyyOUJyDmHJR
	5JOAfKKZmuycx4nqSbHmNi8y089ZvTs=
X-Google-Smtp-Source: ABdhPJwNTxH3rY7fVFK+T2n7C8LTqtOBuO5NPVO/Zedb0cpTLVSylMe3TUoK8Zrs6YggTeOiaDSe7Q==
X-Received: by 2002:ac2:4c81:: with SMTP id d1mr4566179lfl.340.1632988352180;
        Thu, 30 Sep 2021 00:52:32 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 07/11] vpci/header: program p2m with guest BAR view
Date: Thu, 30 Sep 2021 10:52:19 +0300
Message-Id: <20210930075223.860329-8-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Take into account guest's BAR view and program its p2m accordingly:
gfn is guest's view of the BAR and mfn is the physical BAR value as set
up by the host bridge in the hardware domain.
This way hardware doamin sees physical BAR values and guest sees
emulated ones.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
- improve readability for data.start_gfn and restructure ?: construct
Since v1:
 - s/MSI/MSI-X in comments
---
 xen/drivers/vpci/header.c | 34 ++++++++++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 9c603d26d302..f23c956cde6c 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -30,6 +30,10 @@
 
 struct map_data {
     struct domain *d;
+    /* Start address of the BAR as seen by the guest. */
+    gfn_t start_gfn;
+    /* Physical start address of the BAR. */
+    mfn_t start_mfn;
     bool map;
 };
 
@@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data,
                      unsigned long *c)
 {
     const struct map_data *map = data;
+    gfn_t start_gfn;
     int rc;
 
     for ( ; ; )
     {
         unsigned long size = e - s + 1;
 
+        /*
+         * Any BAR may have holes in its memory we want to map, e.g.
+         * we don't want to map MSI-X regions which may be a part of that BAR,
+         * e.g. when a single BAR is used for both MMIO and MSI-X.
+         * In this case MSI-X regions are subtracted from the mapping, but
+         * map->start_gfn still points to the very beginning of the BAR.
+         * So if there is a hole present then we need to adjust start_gfn
+         * to reflect the fact of that substraction.
+         */
+        start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn));
+
+        printk(XENLOG_G_DEBUG
+               "%smap [%lx, %lx] -> %#"PRI_gfn" for d%d\n",
+               map->map ? "" : "un", s, e, gfn_x(start_gfn),
+               map->d->domain_id);
         /*
          * ARM TODOs:
          * - On ARM whether the memory is prefetchable or not should be passed
@@ -52,8 +72,10 @@ static int map_range(unsigned long s, unsigned long e, void *data,
          * - {un}map_mmio_regions doesn't support preemption.
          */
 
-        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
-                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
+        rc = map->map ? map_mmio_regions(map->d, start_gfn,
+                                         size, _mfn(s))
+                      : unmap_mmio_regions(map->d, start_gfn,
+                                           size, _mfn(s));
         if ( rc == 0 )
         {
             *c += size;
@@ -69,6 +91,7 @@ static int map_range(unsigned long s, unsigned long e, void *data,
         ASSERT(rc < size);
         *c += rc;
         s += rc;
+        gfn_add(map->start_gfn, rc);
         if ( general_preempt_check() )
                 return -ERESTART;
     }
@@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v)
             if ( !bar->mem )
                 continue;
 
+            data.start_gfn =
+                 _gfn(PFN_DOWN(is_hardware_domain(v->vpci.pdev->domain)
+                               ? bar->addr : bar->guest_addr));
+            data.start_mfn = _mfn(PFN_DOWN(bar->addr));
             rc = rangeset_consume_ranges(bar->mem, map_range, &data);
 
             if ( rc == -ERESTART )
@@ -194,6 +221,9 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
         if ( !bar->mem )
             continue;
 
+        data.start_gfn = _gfn(PFN_DOWN(is_hardware_domain(d)
+                                       ? bar->addr : bar->guest_addr));
+        data.start_mfn = _mfn(PFN_DOWN(bar->addr));
         while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
                                               &data)) == -ERESTART )
             process_pending_softirqs();
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:53:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:53:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199521.353691 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsT-0005X2-7c; Thu, 30 Sep 2021 07:53:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199521.353691; Thu, 30 Sep 2021 07:53:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqsT-0005Wt-3j; Thu, 30 Sep 2021 07:53:13 +0000
Received: by outflank-mailman (input) for mailman id 199521;
 Thu, 30 Sep 2021 07:53:12 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsS-00017f-6C
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:53:12 +0000
Received: from mail-lf1-x12c.google.com (unknown [2a00:1450:4864:20::12c])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9b5aaab3-e57d-4343-9b5c-247ec58c3a4c;
 Thu, 30 Sep 2021 07:52:34 +0000 (UTC)
Received: by mail-lf1-x12c.google.com with SMTP id j5so16804013lfg.8
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:34 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:32 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b5aaab3-e57d-4343-9b5c-247ec58c3a4c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=OR+8PvgVFP8pLzhFaQ6RjiWWTesnzHH/8qQE78jWNGs=;
        b=h6yENr4JdtnEf1wnG93DzqvZpXcFLZuU/1n0d6/YtV6R/vfnAzlRk88b3OdwgWTGhv
         IInWBJMeO4gdxQHRrESr85YYKjh60ND9+7IeV680A5/04pP2uAvMtNtSXoR1CpWWVhBj
         udqAiXOK/yIn2GDG8DVJ9J4/NDo+zQU18k6LOP++b/03/Bk5tsVOK2LghgJBrqM4ktmS
         IVFAFFPSAhmWHpLrF/GPFBiIDvne2z26lgVMN542h3uSRpSvkELVtq/IuXBPn8ndYKm7
         /95BPB/BGwq574gch5vj3+XDhH22/QPCchQL1UWhBWjU+KjiACBU6bHJmWOWf/J7CsQK
         oEew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=OR+8PvgVFP8pLzhFaQ6RjiWWTesnzHH/8qQE78jWNGs=;
        b=ga4IY3PqtAiqUaJfqNGsZnT5frsVlxscfrZHnuSpXdNTRom8bx611UrSS6BpLsioLo
         SUhWCxCUwnGhYnyh59XhWEafdauQOLEtlbLJGfenF8Knz4ybAAB++i8F/pVHs2e46zHC
         WcvcWEGydpOOrSI2pKst27c0YGlApLo+OwIUg02IpY+0WgPuUkiuhuvtRibs2qVO4AAi
         BeugOlYemxY0bxWMmGDBxDkHuOjeEw5tqvOed450JW1FZkmAbgTVlGMty4CtCN/zSied
         f/78JbTDB70hH8lU1HccYsWEGcsM3Ik86xRidbMs1odc8iyEVsryy5RboCChsVG+xTsX
         Z4Vw==
X-Gm-Message-State: AOAM530LeGu8010FXSg24ogihVtuOwGKLpyWlmu4WPxG9Zg/4kXUuAWW
	WaoajmY1OeVDw3IIHn1crgkCliAvW6A=
X-Google-Smtp-Source: ABdhPJwJzt7joepxqpjEuultT1xkb7pTstL7l3EASTsUuwntiXFMaEoCuRVVohr5ZAIfI4WqJIiFfQ==
X-Received: by 2002:ac2:5c50:: with SMTP id s16mr99079lfp.605.1632988353306;
        Thu, 30 Sep 2021 00:52:33 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Michal Orzel <michal.orzel@arm.com>
Subject: [PATCH v3 08/11] vpci/header: Emulate PCI_COMMAND register for guests
Date: Thu, 30 Sep 2021 10:52:20 +0300
Message-Id: <20210930075223.860329-9-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add basic emulation support for guests. At the moment only emulate
PCI_COMMAND_INTX_DISABLE bit, the rest is not emulated yet and left
as TODO.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
New in v2
---
 xen/drivers/vpci/header.c | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index f23c956cde6c..754aeb5a584f 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -451,6 +451,32 @@ static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t cmd, void *data)
+{
+    /* TODO: Add proper emulation for all bits of the command register. */
+
+    if ( (cmd & PCI_COMMAND_INTX_DISABLE) == 0 )
+    {
+        /*
+         * Guest wants to enable INTx. It can't be enabled if:
+         *  - host has INTx disabled
+         *  - MSI/MSI-X enabled
+         */
+        if ( pdev->vpci->msi->enabled )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+        else
+        {
+            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, reg);
+
+            if ( current_cmd & PCI_COMMAND_INTX_DISABLE )
+                cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
+    }
+
+    cmd_write(pdev, reg, cmd, data);
+}
+
 static void bar_write(const struct pci_dev *pdev, unsigned int reg,
                       uint32_t val, void *data)
 {
@@ -598,9 +624,12 @@ static int add_bar_handlers(const struct pci_dev *pdev, bool is_hwdom)
     struct vpci_bar *bars = header->bars;
     int rc;
 
-    /* Setup a handler for the command register: same for hwdom and guests. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    if ( is_hwdom )
+        rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write,
+                               PCI_COMMAND, 2, header);
+    else
+        rc = vpci_add_register(pdev->vpci, vpci_hw_read16, guest_cmd_write,
+                               PCI_COMMAND, 2, header);
     if ( rc )
         return rc;
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:57:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:57:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199534.353702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwD-0007FK-T3; Thu, 30 Sep 2021 07:57:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199534.353702; Thu, 30 Sep 2021 07:57:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwD-0007FD-PV; Thu, 30 Sep 2021 07:57:05 +0000
Received: by outflank-mailman (input) for mailman id 199534;
 Thu, 30 Sep 2021 07:57:05 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsm-00017f-6u
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:53:32 +0000
Received: from mail-lf1-x131.google.com (unknown [2a00:1450:4864:20::131])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id dbd70223-3b19-4d81-ba12-e4167578e2fc;
 Thu, 30 Sep 2021 07:52:37 +0000 (UTC)
Received: by mail-lf1-x131.google.com with SMTP id y26so21641792lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:37 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:36 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbd70223-3b19-4d81-ba12-e4167578e2fc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=QXLr0ePaWCVb8hGG2KFBTiGRR67U58CE46v0AcmCx0Q=;
        b=oO3fIj/xL0UQThCgy+wTmcU6bDceKmLX8Qzv6Apktl1039A22n7iyzhM4ILHZQEk+b
         3U8aKwZeuanr3uJ16QoBVuY6pkBoOkibx83X/0uFmy9zb3E3MZIwJG6XgDJ4ZJVGV/Bz
         FSDqAYzRqJlZoEtw0kluwX3tuJZyRpGFEZi3GCuGqMYZHkmeZbKAsETl1r6hJ76UYDgo
         nUSBxwnxwpko//NSxwLtc5bp8fA2gnmOkLNmjRhBcwRQuFOzTZdSmUEMNYonZa4nPcSy
         VFRCpwji+OY4ubtBrAqGLp1523Q0c8VZ1KwIMe/C43Z/VNDuT3b06DpD86EOVul+VWeZ
         6GbQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=QXLr0ePaWCVb8hGG2KFBTiGRR67U58CE46v0AcmCx0Q=;
        b=0zNYfrgLZnZGtK+CWikRoPGaA3s9rK5JuQRgRKeJONfxrEgwyS+uBpp3GeoekkW7Sj
         JWYwcJJR/yMRftbhEPfxqTZdLckDoz4WdI/sCV/xy81nYLzuy4+x1m847NyeVLgvpchH
         mMQDqSt5z6dO9i8xY/yGT/gwDT+rcbT+hVt1fceiuwM6QYyHcShKBHy/5X8Z0h1k5U8b
         yJnsgAYOL96tGHL+OEgNdxzQYK9jsetBy0ZxY4UW8lzkQH2cw2nRwBSTVt7aw5E4VUJI
         43gd2LY4ZRr6uoARph13m/KOcBpzlKfgqCL2Vhl9G5Trhl6hA5qef24FEAFxdFt5IXDP
         xqIQ==
X-Gm-Message-State: AOAM531nbaOzMRThh4O5LLqt3CQJ40tvzxEgR9hNv87KWiuExsCP1000
	TWHXZQ9APQb3H+BeY4VL0S8VQgJPNRM=
X-Google-Smtp-Source: ABdhPJw6wtvHV/qjC/Fbro+8Unmd1W1nGr/gXYTIoQK09Mru1KFJP4dAtX44NeWGQfRPa6PCRGVP4Q==
X-Received: by 2002:ac2:4c42:: with SMTP id o2mr4317773lfk.504.1632988356657;
        Thu, 30 Sep 2021 00:52:36 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for guests
Date: Thu, 30 Sep 2021 10:52:23 +0300
Message-Id: <20210930075223.860329-12-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
 - pass struct domain instead of struct vcpu
 - constify arguments where possible
 - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/arch/arm/domain.c         |  1 +
 xen/arch/arm/vpci.c           | 86 +++++++++++++++++++++++++++++++----
 xen/arch/arm/vpci.h           |  3 ++
 xen/drivers/passthrough/pci.c | 25 ++++++++++
 xen/include/asm-arm/pci.h     |  1 +
 xen/include/xen/pci.h         |  1 +
 xen/include/xen/sched.h       |  2 +
 7 files changed, 111 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index fa6fcc5e467c..095671742ad8 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -797,6 +797,7 @@ void arch_domain_destroy(struct domain *d)
                        get_order_from_bytes(d->arch.efi_acpi_len));
 #endif
     domain_io_free(d);
+    domain_vpci_free(d);
 }
 
 void arch_domain_shutdown(struct domain *d)
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 5d6c29c8dcd9..26ec2fa7cf2d 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -17,6 +17,14 @@
 
 #define REGISTER_OFFSET(addr)  ( (addr) & 0x00000fff)
 
+struct vpci_mmio_priv {
+    /*
+     * Set to true if the MMIO handlers were set up for the emulated
+     * ECAM host PCI bridge.
+     */
+    bool is_virt_ecam;
+};
+
 /* Do some sanity checks. */
 static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len)
 {
@@ -38,6 +46,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
     pci_sbdf_t sbdf;
     unsigned long data = ~0UL;
     unsigned int size = 1U << info->dabt.size;
+    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
 
     sbdf.sbdf = MMCFG_BDF(info->gpa);
     reg = REGISTER_OFFSET(info->gpa);
@@ -45,6 +54,13 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
     if ( !vpci_mmio_access_allowed(reg, size) )
         return 0;
 
+    /*
+     * For the passed through devices we need to map their virtual SBDF
+     * to the physical PCI device being passed through.
+     */
+    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v->domain, &sbdf) )
+            return 1;
+
     data = vpci_read(sbdf, reg, min(4u, size));
     if ( size == 8 )
         data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32;
@@ -61,6 +77,7 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
     pci_sbdf_t sbdf;
     unsigned long data = r;
     unsigned int size = 1U << info->dabt.size;
+    struct vpci_mmio_priv *priv = (struct vpci_mmio_priv *)p;
 
     sbdf.sbdf = MMCFG_BDF(info->gpa);
     reg = REGISTER_OFFSET(info->gpa);
@@ -68,6 +85,13 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
     if ( !vpci_mmio_access_allowed(reg, size) )
         return 0;
 
+    /*
+     * For the passed through devices we need to map their virtual SBDF
+     * to the physical PCI device being passed through.
+     */
+    if ( priv->is_virt_ecam && !pci_translate_virtual_device(v->domain, &sbdf) )
+            return 1;
+
     vpci_write(sbdf, reg, min(4u, size), data);
     if ( size == 8 )
         vpci_write(sbdf, reg + 4, 4, data >> 32);
@@ -80,13 +104,48 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
     .write = vpci_mmio_write,
 };
 
+/*
+ * There are three  originators for the PCI configuration space access:
+ * 1. The domain that owns physical host bridge: MMIO handlers are
+ *    there so we can update vPCI register handlers with the values
+ *    written by the hardware domain, e.g. physical view of the registers/
+ *    configuration space.
+ * 2. Guest access to the passed through PCI devices: we need to properly
+ *    map virtual bus topology to the physical one, e.g. pass the configuration
+ *    space access to the corresponding physical devices.
+ * 3. Emulated host PCI bridge access. It doesn't exist in the physical
+ *    topology, e.g. it can't be mapped to some physical host bridge.
+ *    So, all access to the host bridge itself needs to be trapped and
+ *    emulated.
+ */
 static int vpci_setup_mmio_handler(struct domain *d,
                                    struct pci_host_bridge *bridge)
 {
-    struct pci_config_window *cfg = bridge->cfg;
+    struct vpci_mmio_priv *priv;
+
+    priv = xzalloc(struct vpci_mmio_priv);
+    if ( !priv )
+        return -ENOMEM;
+
+    priv->is_virt_ecam = !is_hardware_domain(d);
 
-    register_mmio_handler(d, &vpci_mmio_handler,
-                          cfg->phys_addr, cfg->size, NULL);
+    if ( is_hardware_domain(d) )
+    {
+        struct pci_config_window *cfg = bridge->cfg;
+
+        bridge->mmio_priv = priv;
+        register_mmio_handler(d, &vpci_mmio_handler,
+                              cfg->phys_addr, cfg->size,
+                              priv);
+    }
+    else
+    {
+        d->vpci_mmio_priv = priv;
+        /* Guest domains use what is programmed in their device tree. */
+        register_mmio_handler(d, &vpci_mmio_handler,
+                              GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE,
+                              priv);
+    }
     return 0;
 }
 
@@ -95,14 +154,25 @@ int domain_vpci_init(struct domain *d)
     if ( !has_vpci(d) )
         return 0;
 
+    return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+}
+
+static int domain_vpci_free_cb(struct domain *d,
+                               struct pci_host_bridge *bridge)
+{
     if ( is_hardware_domain(d) )
-        return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+        XFREE(bridge->mmio_priv);
+    else
+        XFREE(d->vpci_mmio_priv);
+    return 0;
+}
 
-    /* Guest domains use what is programmed in their device tree. */
-    register_mmio_handler(d, &vpci_mmio_handler,
-                          GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL);
+void domain_vpci_free(struct domain *d)
+{
+    if ( !has_vpci(d) )
+        return;
 
-    return 0;
+    pci_host_iterate_bridges(d, domain_vpci_free_cb);
 }
 
 int domain_vpci_get_num_mmio_handlers(struct domain *d)
diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h
index 27a2b069abd2..38e5a28c0d95 100644
--- a/xen/arch/arm/vpci.h
+++ b/xen/arch/arm/vpci.h
@@ -18,6 +18,7 @@
 #ifdef CONFIG_HAS_VPCI
 int domain_vpci_init(struct domain *d);
 int domain_vpci_get_num_mmio_handlers(struct domain *d);
+void domain_vpci_free(struct domain *d);
 #else
 static inline int domain_vpci_init(struct domain *d)
 {
@@ -28,6 +29,8 @@ static inline int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
     return 0;
 }
+
+static inline void domain_vpci_free(struct domain *d) { }
 #endif
 
 #endif /* __ARCH_ARM_VPCI_H__ */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 5b963d75d1ba..b7dffb769cfd 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -889,6 +889,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
     xfree(vdev);
     return 0;
 }
+
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool pci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
+{
+    struct vpci_dev *vdev;
+    bool found = false;
+
+    pcidevs_lock();
+    list_for_each_entry ( vdev, &d->vdev_list, list )
+    {
+        if ( vdev->sbdf.sbdf == sbdf->sbdf )
+        {
+            /* Replace virtual SBDF with the physical one. */
+            *sbdf = vdev->pdev->sbdf;
+            found = true;
+            break;
+        }
+    }
+    pcidevs_unlock();
+
+    return found;
+}
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
 /* Caller should hold the pcidevs_lock */
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index 1bfba3da8f51..12b4bf467ad2 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -66,6 +66,7 @@ struct pci_host_bridge {
     uint16_t segment;                /* Segment number */
     struct pci_config_window* cfg;   /* Pointer to the bridge config window */
     struct pci_ops *ops;
+    void *mmio_priv;                 /* MMIO handler's private data. */
 };
 
 struct pci_ops {
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 33033a3a8f8d..89cfc4853331 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -188,6 +188,7 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn);
 #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
 int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev);
 int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev);
+bool pci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
 #endif
 int pci_ro_device(int seg, int bus, int devfn);
 int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index ecdb04b4f7fc..858b4133482f 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -451,6 +451,8 @@ struct domain
      * to assign a unique SBDF to a passed through virtual PCI device.
      */
     int vpci_dev_next;
+    /* Virtual PCI MMIO handler's private data. */
+    void *vpci_mmio_priv;
 #endif
 #endif
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:57:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:57:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199537.353713 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwK-0007Xi-4L; Thu, 30 Sep 2021 07:57:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199537.353713; Thu, 30 Sep 2021 07:57:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwK-0007Xb-1D; Thu, 30 Sep 2021 07:57:12 +0000
Received: by outflank-mailman (input) for mailman id 199537;
 Thu, 30 Sep 2021 07:57:10 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsc-00017f-6R
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:53:22 +0000
Received: from mail-lf1-x12d.google.com (unknown [2a00:1450:4864:20::12d])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 15844ac1-baf8-4ff5-8fba-69a9900d443d;
 Thu, 30 Sep 2021 07:52:36 +0000 (UTC)
Received: by mail-lf1-x12d.google.com with SMTP id y26so21641542lfa.11
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:36 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:35 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15844ac1-baf8-4ff5-8fba-69a9900d443d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=IvOUvJEBvxetANJzBoBgBMczs7Ntkr0TVyDjFYQuZ/M=;
        b=VoB77MfqKEQnQtKp+lPWH1kg+gpqtdmADxdUy6EQRGw1QG6Bpabmz8yFGEsugn5XlN
         XAyQBFqu7TT4/qCP8RqsLPEGtUuyM53MQx37PL4X84E1UzWXD+zf3bN4anHofgQL+eoE
         4tL7ybx5OYRjtT8cztLSwMusaE9J+8JXxD0e0+UtX7O2lzWOnfAYbkHwWjtKM/FRSS22
         lwsQBCEKwhKyG8yWEEiT7JG4LMyllM/1Mr8/vmGhABarplV3lAQgwhOjSX+qMAM0VQtD
         NjO9u7+bBSv0EMamA/xDsvtBA63G2LIIM4iyBYWhKAnIairyEVCZTYZHOhRXt38yyapX
         TX2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=IvOUvJEBvxetANJzBoBgBMczs7Ntkr0TVyDjFYQuZ/M=;
        b=Tuynr6w9faHH7FgeePdyYoqCMHRZIYjSwVc/L2DeZPGRhjViB56NYBAQO957QEy8I+
         Qiboaxdm02HK7owR9z2n8empcW70+uI4W1tIuAt9LrQqD1H35yIMoDriN4D+oTj/QZsH
         3Yq157SF145aYjVCqxEQC/QZ68CM8FJ69k/P4/WZZXc9YVxWsa0fPjNs6zaqxP6ymOcc
         SYOZ8Uw2f6veqYdemmbVROZ0rxrNCr90iICM0jr9yHA1d324MXOy7xWdteHw5RxTprrR
         o5a51RIDO4OwW3/VPZmt1/bfC34sVpt2UKVSPvuhQmgqAvQBXMfFDoQz71PtD5u4idiQ
         xBYw==
X-Gm-Message-State: AOAM533fDSe/9837sIXbggKgQL0KGLdLttcJBEIGjaKupYd0/CFpARrL
	PwgRJd5UGzNT4vzX/XBSfrFdXIsVqg0=
X-Google-Smtp-Source: ABdhPJzHAYn0FeIUINVCTCrJ6p1OupRcFNPn66qoht+4Rh+LnDfq3NvAUqoEvvgCWqQcD7851GTPEw==
X-Received: by 2002:a19:f515:: with SMTP id j21mr4274651lfb.125.1632988355472;
        Thu, 30 Sep 2021 00:52:35 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Subject: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus topology
Date: Thu, 30 Sep 2021 10:52:22 +0300
Message-Id: <20210930075223.860329-11-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

---
Since v2:
 - remove casts that are (a) malformed and (b) unnecessary
 - add new line for better readability
 - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
    functions are now completely gated with this config
 - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/common/domain.c           |  3 ++
 xen/drivers/passthrough/pci.c | 60 +++++++++++++++++++++++++++++++++++
 xen/drivers/vpci/vpci.c       | 14 +++++++-
 xen/include/xen/pci.h         | 22 +++++++++++++
 xen/include/xen/sched.h       |  8 +++++
 5 files changed, 106 insertions(+), 1 deletion(-)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index 40d67ec34232..e0170087612d 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -601,6 +601,9 @@ struct domain *domain_create(domid_t domid,
 
 #ifdef CONFIG_HAS_PCI
     INIT_LIST_HEAD(&d->pdev_list);
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    INIT_LIST_HEAD(&d->vdev_list);
+#endif
 #endif
 
     /* All error paths can depend on the above setup. */
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 805ab86ed555..5b963d75d1ba 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -831,6 +831,66 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
     return ret;
 }
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
+                                                const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    list_for_each_entry ( vdev, &d->vdev_list, list )
+        if ( vdev->pdev == pdev )
+            return vdev;
+    return NULL;
+}
+
+int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    ASSERT(!pci_find_virtual_device(d, pdev));
+
+    /* Each PCI bus supports 32 devices/slots at max. */
+    if ( d->vpci_dev_next > 31 )
+        return -ENOSPC;
+
+    vdev = xzalloc(struct vpci_dev);
+    if ( !vdev )
+        return -ENOMEM;
+
+    /* We emulate a single host bridge for the guest, so segment is always 0. */
+    vdev->seg = 0;
+
+    /*
+     * The bus number is set to 0, so virtual devices are seen
+     * as embedded endpoints behind the root complex.
+     */
+    vdev->bus = 0;
+    vdev->devfn = PCI_DEVFN(d->vpci_dev_next++, 0);
+
+    vdev->pdev = pdev;
+    vdev->domain = d;
+
+    pcidevs_lock();
+    list_add_tail(&vdev->list, &d->vdev_list);
+    pcidevs_unlock();
+
+    return 0;
+}
+
+int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
+{
+    struct vpci_dev *vdev;
+
+    pcidevs_lock();
+    vdev = pci_find_virtual_device(d, pdev);
+    if ( vdev )
+        list_del(&vdev->list);
+    pcidevs_unlock();
+    xfree(vdev);
+    return 0;
+}
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 /* Caller should hold the pcidevs_lock */
 static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
                            uint8_t devfn)
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 702f7b5d5dda..d787f13e679e 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -91,20 +91,32 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
 /* Notify vPCI that device is assigned to guest. */
 int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
 {
+    int rc;
+
     /* It only makes sense to assign for hwdom or guest domain. */
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
-    return vpci_bar_add_handlers(d, dev);
+    rc = vpci_bar_add_handlers(d, dev);
+    if ( rc )
+        return rc;
+
+    return pci_add_virtual_device(d, dev);
 }
 
 /* Notify vPCI that device is de-assigned from guest. */
 int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
 {
+    int rc;
+
     /* It only makes sense to de-assign from hwdom or guest domain. */
     if ( is_system_domain(d) || !has_vpci(d) )
         return 0;
 
+    rc = pci_remove_virtual_device(d, dev);
+    if ( rc )
+        return rc;
+
     return vpci_bar_remove_handlers(d, dev);
 }
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 43b8a0817076..33033a3a8f8d 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -137,6 +137,24 @@ struct pci_dev {
     struct vpci *vpci;
 };
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+struct vpci_dev {
+    struct list_head list;
+    /* Physical PCI device this virtual device is connected to. */
+    const struct pci_dev *pdev;
+    /* Virtual SBDF of the device. */
+    union {
+        struct {
+            uint8_t devfn;
+            uint8_t bus;
+            uint16_t seg;
+        };
+        pci_sbdf_t sbdf;
+    };
+    struct domain *domain;
+};
+#endif
+
 #define for_each_pdev(domain, pdev) \
     list_for_each_entry(pdev, &(domain)->pdev_list, domain_list)
 
@@ -167,6 +185,10 @@ const unsigned long *pci_get_ro_map(u16 seg);
 int pci_add_device(u16 seg, u8 bus, u8 devfn,
                    const struct pci_dev_info *, nodeid_t node);
 int pci_remove_device(u16 seg, u8 bus, u8 devfn);
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev);
+int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev);
+#endif
 int pci_ro_device(int seg, int bus, int devfn);
 int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn);
 struct pci_dev *pci_get_pdev(int seg, int bus, int devfn);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 28146ee404e6..ecdb04b4f7fc 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -444,6 +444,14 @@ struct domain
 
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    struct list_head vdev_list;
+    /*
+     * Current device number used by the virtual PCI bus topology
+     * to assign a unique SBDF to a passed through virtual PCI device.
+     */
+    int vpci_dev_next;
+#endif
 #endif
 
 #ifdef CONFIG_HAS_PASSTHROUGH
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 07:57:32 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 07:57:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199561.353724 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwe-0008ML-Ij; Thu, 30 Sep 2021 07:57:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199561.353724; Thu, 30 Sep 2021 07:57:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVqwe-0008ME-Ee; Thu, 30 Sep 2021 07:57:32 +0000
Received: by outflank-mailman (input) for mailman id 199561;
 Thu, 30 Sep 2021 07:57:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OAwA=OU=gmail.com=andr2000@srs-us1.protection.inumbo.net>)
 id 1mVqsX-00017f-6I
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 07:53:17 +0000
Received: from mail-lf1-x12a.google.com (unknown [2a00:1450:4864:20::12a])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9ac7d20e-d2a5-489d-acb4-b13358f83a1a;
 Thu, 30 Sep 2021 07:52:35 +0000 (UTC)
Received: by mail-lf1-x12a.google.com with SMTP id j5so16804175lfg.8
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 00:52:35 -0700 (PDT)
Received: from localhost.localdomain (host-176-36-245-220.b024.la.net.ua.
 [176.36.245.220])
 by smtp.gmail.com with ESMTPSA id 13sm251099ljf.51.2021.09.30.00.52.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 30 Sep 2021 00:52:33 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ac7d20e-d2a5-489d-acb4-b13358f83a1a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=from:to:cc:subject:date:message-id:in-reply-to:references
         :mime-version:content-transfer-encoding;
        bh=kFKr9R5/eoTpWJugUvdw0hZSjF5zvjYKXnAMVRzF8Pk=;
        b=cWJGSCPP+epJ29KCHDh+8crG7u2S+kudHu40Amr21tBEaYnCRj4sLpj6yJlBz5lWjY
         bkH0E367iOZOHaRQVdwMKNzJb+JV4+Hdpvm9oZKzkFxD726NCHoDuDDfPzLQSBUpRL3L
         9gFvonYsfiA/cdTdKzmi+5Ei+EzFxrwYmM4UMvr7JF5bpk214Gygsk6sXxyi9OaJBOnD
         gUV2hlylrGcyAW4tN/mZrbZ2AeINau62FVNJQ+e/v/u+p9CChqku8UpSjyViK6YE5vD2
         xj/9U0yHM5Xhn43L9AK2JPGa9oCy5CEHeEiudzW4kf5ZRLmKNidiK4/f1BUA6xzxzKsv
         39FA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
         :references:mime-version:content-transfer-encoding;
        bh=kFKr9R5/eoTpWJugUvdw0hZSjF5zvjYKXnAMVRzF8Pk=;
        b=r4AWQ2Ooqfpm6p4JXzLGJkWw8tW67sIWlk0D+cPk2Wicb9rDHl8iWspPvc/WqNQnWI
         wgNI+WNDlNEEZzyGb5lL0uQx7jdewvKVc9tss0ImmQrqXqkP9hMjnvljrkolfzJgnYtw
         KFQtA6AJft7Axeq7Bo5selkocKYUuzAcIKFbDjKguIsV3regHtfp8a51xGpq23vp3g0Y
         U36o/fmNAVJjKZIC0iR3ZIhN4nmTieiLA0zTwVNJJk8AslesdPrA2BKi9VAstXhuXCG2
         ZV5c46+rcQD1rzDTFniUEmXi5TwVTUzWyDat9zCVZPuPr9ZX16AnxeVEwU+sw5pZ3VqC
         wOVQ==
X-Gm-Message-State: AOAM53095PPna5TjcukGTE0ypoywHseoZHeELUgGc6rfgQEN6MlDOPHf
	Zi781nuFbzJS0VPDs0+zDdDou3gG62w=
X-Google-Smtp-Source: ABdhPJzNrHs+SZ/Q+mD+Sr3QROyeTeMKrWMeDO5LFmSFYlTZ1RNOC9HAByTat96meLn6Ha/BcigpWQ==
X-Received: by 2002:ac2:4f02:: with SMTP id k2mr4255807lfr.378.1632988354376;
        Thu, 30 Sep 2021 00:52:34 -0700 (PDT)
From: Oleksandr Andrushchenko <andr2000@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: julien@xen.org,
	sstabellini@kernel.org,
	oleksandr_tyshchenko@epam.com,
	volodymyr_babchuk@epam.com,
	Artem_Mygaiev@epam.com,
	roger.pau@citrix.com,
	jbeulich@suse.com,
	bertrand.marquis@arm.com,
	rahul.singh@arm.com,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Michal Orzel <michal.orzel@arm.com>
Subject: [PATCH v3 09/11] vpci/header: Reset the command register when adding devices
Date: Thu, 30 Sep 2021 10:52:21 +0300
Message-Id: <20210930075223.860329-10-andr2000@gmail.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210930075223.860329-1-andr2000@gmail.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Reset the command register when passing through a PCI device:
it is possible that when passing through a PCI device its memory
decoding bits in the command register are already set. Thus, a
guest OS may not write to the command register to update memory
decoding, so guest mappings (guest's view of the BARs) are
left not updated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
---
Since v1:
 - do not write 0 to the command register, but respect host settings.
---
 xen/drivers/vpci/header.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 754aeb5a584f..70d911b147e1 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -451,8 +451,7 @@ static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
-static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
-                            uint32_t cmd, void *data)
+static uint32_t emulate_cmd_reg(const struct pci_dev *pdev, uint32_t cmd)
 {
     /* TODO: Add proper emulation for all bits of the command register. */
 
@@ -467,14 +466,20 @@ static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
             cmd |= PCI_COMMAND_INTX_DISABLE;
         else
         {
-            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, reg);
+            uint16_t current_cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
 
             if ( current_cmd & PCI_COMMAND_INTX_DISABLE )
                 cmd |= PCI_COMMAND_INTX_DISABLE;
         }
     }
 
-    cmd_write(pdev, reg, cmd, data);
+    return cmd;
+}
+
+static void guest_cmd_write(const struct pci_dev *pdev, unsigned int reg,
+                            uint32_t cmd, void *data)
+{
+    cmd_write(pdev, reg, emulate_cmd_reg(pdev, cmd), data);
 }
 
 static void bar_write(const struct pci_dev *pdev, unsigned int reg,
@@ -793,6 +798,10 @@ int vpci_bar_add_handlers(const struct domain *d, const struct pci_dev *pdev)
         gdprintk(XENLOG_ERR,
                  "%pp: failed to add BAR handlers for dom%pd: %d\n",
                  &pdev->sbdf, d, rc);
+
+    /* Reset the command register with respect to host settings. */
+    pci_conf_write16(pdev->sbdf, PCI_COMMAND, emulate_cmd_reg(pdev, 0));
+
     return rc;
 }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:04:40 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:04:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199585.353735 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVr3S-0002Df-Gj; Thu, 30 Sep 2021 08:04:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199585.353735; Thu, 30 Sep 2021 08:04:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVr3S-0002DY-Cr; Thu, 30 Sep 2021 08:04:34 +0000
Received: by outflank-mailman (input) for mailman id 199585;
 Thu, 30 Sep 2021 08:04:33 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVr3Q-0002DS-Tz
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:04:33 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 0a6799f2-21c5-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 08:04:31 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-4tEksrK5MRWJb4Qd7oxIQQ-1; Thu, 30 Sep 2021 10:04:29 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB2829.eurprd04.prod.outlook.com (2603:10a6:800:bb::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.20; Thu, 30 Sep
 2021 08:04:27 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 08:04:27 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P194CA0102.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:8f::43) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 08:04:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a6799f2-21c5-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632989070;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=K95pUxLUjw7UaDwmtJKGNRMvxbs8axOcXiHMYClJwCI=;
	b=hMuq/wNDan9fauboDXeelJHX0t/474iZuNTm2wu/+6q/RnfIsGRZAfBSSd9zokVnDxBUwl
	atwzlaURrG/G4Pq7nKVvPQ0A1wYMNHLSaQNSyrCcqgzPlHOrcC0RkkBxqyEulDZa3qZzlH
	a/zDPvzwxnQseGX+Rj7MOTEGeyBUu94=
X-MC-Unique: 4tEksrK5MRWJb4Qd7oxIQQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kaVM/CiapICcdS4VhwawbQkZ5Je8ncQ+TSLGhQv8OoeLWTH6OGVudy9GFik5vicINdonEgs/ckv225y/vAQmqq0YjupXFqHP2qOac6YqWPbIzW4/6cbVCxdZkj2JGsWA844IM+xMqDrPgj5c+Z8QysHYs+MxU0K/xxCrgduP/udAtYSKlxTPjgfN8lAvmG2jCn2rShPRVzdwE4FrT5uUwtKNyFlykusN1CurxJsP0RBxp3VIbNkcX++cq1NWHQEekAZn/NSn6DM7b+X542h6pYduWIVNUMO4nYb2iz2vkIMKrbZeBOqEfRBBXmXrzA3GCf92yGf33BJeONOt/Ors4A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=K95pUxLUjw7UaDwmtJKGNRMvxbs8axOcXiHMYClJwCI=;
 b=JdZ0uXRSaioMPQ9TxxcwGcs3gxLz3yr5XmEThIu0/L6i7DooimLjVIa0E09CajPMA9mIbYuFPKvZc2tRmFzeg2+YvTHBA5TtgxLB/T8/Hk3spNcRBII0ajw6EQ+GsHUTuKhW/r/a2JSnd97fgUMlhhOVKn1Rqc2xQo8V6F4UJHem2Da21Qe3mvRCE4dBrhXhw0L5I4WdLqZJZV0nU4yuNK9pWvkEFaoHL8gno+FbkNlR2U07JrToHDfqWQ1L/Kv3uCAqj3e9dcfu9LmlcyDUHtMKV8JJhS1vHutIIe7OJwYqn/gVb4+CXdQo6+ABgcB6gM7FwDbAcfNZdPb7hnuzgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: [PATCH v3 11/11] xen/arm: Process pending vPCI map/unmap
 operations
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <20210930071326.857390-1-andr2000@gmail.com>
 <20210930071326.857390-12-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <49166fbf-6436-1ecc-fccf-98be4ad400b8@suse.com>
Date: Thu, 30 Sep 2021 10:04:24 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930071326.857390-12-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P194CA0102.EURP194.PROD.OUTLOOK.COM
 (2603:10a6:209:8f::43) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3b6a8b38-9599-456c-e561-08d983e8ec46
X-MS-TrafficTypeDiagnostic: VI1PR0402MB2829:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB282978AF393F56E14E410583B3AA9@VI1PR0402MB2829.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:580;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pi1v8CXOEPyCUV/ph5kGrkrm9Xrm00VccSzzSRi2TCfYP8n33+LyrgBhW5NwP04mjCWD6O+2U4wpjOvq9W6NKgt3PYJ/dmiUDO/7bdhDw2VT0uoH8KiwqlpgvDyr66gZYPn9EWqdg1fK5pwrSM+rTX90COj3CsNFISUbDFoKruld1ljStRlEZ7bTxfWvPj6l/LXmANpwAKTAiG4A43kZuWE0QyR8Efr7fEXhmZNdr8GhUmq+qiAnZqXzqbaG1qCpW40LLJT+2Etvt0lrUErdRPty8NL+mrwjb+eG5LDJc8mU89nZJ4y94uEEud7ROBRPVFpSLOAoFt+axrLfmzexI1H3I+nklneZWNeyRNqY6NHbIUocZgWnuqUCY/OW/RDlCZBkxdIdphDfiD9AJ4seU9yt0voE9xRFyE/BfhzU++JX5ZyybKVR72be9rMcovpaSIBPl7q7dYkwOdbHUtnmALu1MtXjljGuDRG6Y90B8vhKTKXl0HceVNfKV5Ht8GVmHVxgnh+VkoF+paCY3Cl8N8poC1BFmhZKJZIL61hPIWR6qMBsa/Wa9T1QRfehnKsOzOPl1KtW19Bx7EzfjzYzYmmi9rzDdK67xtPKCsuwL31WqC6hQgzHt5vOTmrqVmWIm/4Hm066u3DTT7sGobV9ulL0q72r5dMbCh3ImayGxHhlR66X6pGNKCF+94RmOmlf8fE9WokTo3R4atzzLt6NBZyzgD65PsyuvMEiXLfMtVA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66476007)(4326008)(956004)(8676002)(8936002)(31686004)(83380400001)(6486002)(316002)(86362001)(16576012)(54906003)(38100700002)(7416002)(66556008)(2906002)(5660300002)(508600001)(186003)(2616005)(36756003)(66946007)(31696002)(53546011)(6916009)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NzlVM3ZBcGM2eXNRbkRsVFpvSEVVTG5VN0JqSE0zNGRSd1NubXVFVE9ScnRE?=
 =?utf-8?B?TE85MWtGa2g4aEx1bk82UXgxakx3NVBMV2Q5cXUvbXdocDdEeEE2VkdxV3Vo?=
 =?utf-8?B?b0hQWi8rc3hTdDg1dTFCMjl1TVN1TTZTcUtRQ0E0a3hPRUtaZkZHeStJOEl4?=
 =?utf-8?B?ck9TcGpicTZMelh3K1Q5Qlp5QjI4bHJmV3Z2Qlc3bkZnZHJMT0VMOTQ4azJk?=
 =?utf-8?B?ZUZPSW93V0dWTjZGZjNlRXpkM3dqekNSYWYrdHVZQzcyOWpEVWNyL2hURVRx?=
 =?utf-8?B?UkxyTXhUQVdmRitqVWl4UkdUQUZHYXArUUp6RHZmNjJKdDB1Wk5kSTIvU1Fm?=
 =?utf-8?B?SlBLRW5xb29qSlloN1lONkZRZjczVFVSc3hSNzNoMG5RdS9NVGJDbk5jMXg2?=
 =?utf-8?B?M0JyU3FtSkdqZzdWaElocjlCTldpb0dlTVJIUkYwSFFYc2V1a1J0ODZiRVU4?=
 =?utf-8?B?K3FtRnZERnI0ODZaNUx3WFBpUzM0NUd5WllmRUpuMWpER0svbUR6ajdpbU5r?=
 =?utf-8?B?WGlsSktIbmhwSGJ3enBwNVdicXBodS9HMzNtTTQzTVJxWElZeERiUEdiRnVP?=
 =?utf-8?B?c0dLTUdXME9va2pVRVpKK3l6ckxVUDR6OWFBNnBJV3ZBS2ZaWDNOckpCa0Ex?=
 =?utf-8?B?MWJEeGxBY25MMjRQUVhNRkVUWWpRK25UQ2hNSXA3cnlFNDFxUjVhZFhnSk5p?=
 =?utf-8?B?TjRpU0NtUGUrbHN2MWh1ckZoR0FTZEZvb0RMd3dVQzVsbEJsSmhjcTZYR0NS?=
 =?utf-8?B?MmJFaGdYK2ExanIxQ0hzSnpIck5leDE4VUhQblZqUGVqZnRrU2VWNWFQVkh1?=
 =?utf-8?B?RGhyYTRNek95WE5oTkkyR1BucFFMTkVmSVFOSkdHRTRwbGVkcGY0QWpLV2l3?=
 =?utf-8?B?Mml4VUhUOGtxNlE0SjBXLzArSS9Xc2V6KytCaXBKUmQ5dVU0Y1UxVFJ5U0Ny?=
 =?utf-8?B?VDZZQVhiSm5iY1V1ZDRyMjJ2MExQUGxNMXBWVTVncVNXbWZDTEhIOUZSMjNU?=
 =?utf-8?B?T2draDR0R2trWHBzcUZ0NXdOQTJBWmVRc1hyRUNYSksxNEdnQXV3UThrdUkv?=
 =?utf-8?B?bGxsYUprRW9HekFCSldod2lGNjZoNkFnSUw4NE9IYWJOb21mZCtKSkVtN203?=
 =?utf-8?B?STFQYVltczQycmdPTithbllEYmtLK0pCZkdzNmUrck9YMjI5YURqOXVGcjA3?=
 =?utf-8?B?OGsvNEJHZXVZajQ4VVBjYm5RbEdaaVk5Y2tFNWs0bFdscUFwVnZKaUJ2Y1E1?=
 =?utf-8?B?S0hIaXplMFJsak13K1Z1UWxjTGRjZXNSYVZZcGFydjBaNXRVWGRIV1IxbTdk?=
 =?utf-8?B?bnRBTTBYSHJQQUY4WTNGWDVXdUo4d2VzSjBKWEVFNmw5OFIvVDlJMWptaGxv?=
 =?utf-8?B?YzJmL2pNTjhacm15SVlCYXRuSG1LNnZCOTY0U1V5Q1kxaWF1QzVmdkVOeExT?=
 =?utf-8?B?RmYrQkdKMlBjU3NHc0txWHRLcnZKTEVaV0xFS2FsU1hHdUs0L3liSWtmRERU?=
 =?utf-8?B?WFBaY3A4bHprS2J6ZEhvbzF5TURZVzhQN3B5aU5VdEwvMzM0QWg0QThOaWpl?=
 =?utf-8?B?RFFTaGMrTjRQV1FOU0hpY0g5NkxhNkkzRjAzQWNtaE4zbmVRM3dXenV1cGFR?=
 =?utf-8?B?TjlCR2x3UTdsTlVIeEdELzEwRmZWYXRHMHk0QXJSSUFSWmJLS2U3ZW5iTDlH?=
 =?utf-8?B?cjRNekovbU9KZkpMOHdJRlMySWxZR05zL0JFSXJseE9XV3FpaUdYVkl0bkxT?=
 =?utf-8?Q?hgCL5bd0k1dEhTYFDZJC17l6m8W5RSz+v/CnwHa?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3b6a8b38-9599-456c-e561-08d983e8ec46
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 08:04:27.0385
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uaE2+qPM+sJ42CJxkl36Cpp6Y5oWNfP9SUmcEyXh8+Mn3XahyHAzfSNDUxc8XjFADEGdtpUvw64tNCZfV9sdhQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB2829

On 30.09.2021 09:13, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> vPCI may map and unmap PCI device memory (BARs) being passed through which
> may take a lot of time. For this those operations may be deferred to be
> performed later, so that they can be safely preempted.
> 
> Currently this deferred processing is happening in common IOREQ code
> which doesn't seem to be the right place for x86 and is even more
> doubtful because IOREQ may not be enabled for Arm at all.
> So, for Arm the pending vPCI work may have no chance to be executed
> if the processing is left as is in the common IOREQ code only.
> For that reason make vPCI processing happen in arch specific code.
> 
> Please be aware that there are a few outstanding TODOs affecting this
> code path, see xen/drivers/vpci/header.c:map_range and
> xen/drivers/vpci/header.c:vpci_process_pending.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> ---
> Since v2:
>  - update commit message with more insight on x86, IOREQ and Arm
>  - restored order of invocation for IOREQ and vPCI processing (Jan)
> Since v1:
>  - Moved the check for pending vpci work from the common IOREQ code
>    to hvm_do_resume on x86
>  - Re-worked the code for Arm to ensure we don't miss pending vPCI work
> ---
>  xen/arch/arm/traps.c   | 13 +++++++++++++
>  xen/arch/x86/hvm/hvm.c |  6 ++++++

This x86 change
Acked-by: Jan Beulich <jbeulich@suse.com>

>  xen/common/ioreq.c     |  9 ---------

Already on v2 I did indicate that you need to Cc Paul to get an ack for
this part of the change.

> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -546,6 +546,12 @@ void hvm_do_resume(struct vcpu *v)
>  
>      pt_restore_timer(v);
>  
> +    if ( has_vpci(v->domain) && vpci_process_pending(v) )
> +    {
> +        raise_softirq(SCHEDULE_SOFTIRQ);
> +        return;
> +    }
> +
>      if ( !vcpu_ioreq_handle_completion(v) )
>          return;

Seeing this movement I'm now wondering whether we shouldn't take the
opportunity and move this further up. I notice that besides Paul you
also failed to Cc the other x86 maintainers, whom I'm now adding.

Jan

> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -25,9 +25,7 @@
>  #include <xen/lib.h>
>  #include <xen/paging.h>
>  #include <xen/sched.h>
> -#include <xen/softirq.h>
>  #include <xen/trace.h>
> -#include <xen/vpci.h>
>  
>  #include <asm/guest_atomics.h>
>  #include <asm/ioreq.h>
> @@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
>  
>  bool vcpu_ioreq_handle_completion(struct vcpu *v)
>  {
> -    struct domain *d = v->domain;
>      struct vcpu_io *vio = &v->io;
>      struct ioreq_server *s;
>      struct ioreq_vcpu *sv;
>      enum vio_completion completion;
>      bool res = true;
>  
> -    if ( has_vpci(d) && vpci_process_pending(v) )
> -    {
> -        raise_softirq(SCHEDULE_SOFTIRQ);
> -        return false;
> -    }
> -
>      while ( (sv = get_pending_vcpu(v, &s)) != NULL )
>          if ( !wait_for_io(sv, get_ioreq(s, v)) )
>              return false;
> 



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:07:20 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:07:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199591.353746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVr67-0002yj-0j; Thu, 30 Sep 2021 08:07:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199591.353746; Thu, 30 Sep 2021 08:07:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVr66-0002yc-S8; Thu, 30 Sep 2021 08:07:18 +0000
Received: by outflank-mailman (input) for mailman id 199591;
 Thu, 30 Sep 2021 08:07:18 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lcZS=OU=suse.com=dfaggioli@srs-us1.protection.inumbo.net>)
 id 1mVr66-0002vo-BA
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:07:18 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1746c208-bbd2-499a-9eb5-c1e14ddf7d35;
 Thu, 30 Sep 2021 08:07:17 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2053.outbound.protection.outlook.com [104.47.13.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-lW3pAXosO3qoBtYF2jZn6g-1; Thu, 30 Sep 2021 10:07:14 +0200
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com (2603:10a6:208:134::22)
 by AM4PR0401MB2307.eurprd04.prod.outlook.com (2603:10a6:200:50::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Thu, 30 Sep
 2021 08:07:12 +0000
Received: from AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95]) by AM0PR04MB5826.eurprd04.prod.outlook.com
 ([fe80::7dc7:c:77b6:ed95%7]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021
 08:07:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1746c208-bbd2-499a-9eb5-c1e14ddf7d35
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632989236;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=kD/3FSSuo+0Qv8jCUpQXYHGDFD27mwPLJVLhJ2SwDvo=;
	b=W/LNsHRYeBhpynqX+WYHIvS2IlpE2sEsCKs+B3ybExgBNGOZ19+S0QpkL6numl/N/JEBV1
	PkaEWCkLnNA6M5EXKNUfaG6FaaJKqqT3MrsOz78N7M4M0glxEDOddF+QfgloYfOyjFKrI0
	DJB6RplwxPrfVM1hvB4S2JNtAUcd9XA=
X-MC-Unique: lW3pAXosO3qoBtYF2jZn6g-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hqHqtDjhbt02FQv3laidogZOGG96DBf1zOzjM/YeGVDH/7cNxOjIooPRI9WVDkfOAPkxFryeitKRDny6OvRh0esnotFmdBG9TNUoX9z5MVym/4to+unTcD7sYPK8Yo/YM+NT1rRdUHuU8JI8Au3HpKCoVgEkiQBA3mQV4sMtjsPQCQ6HlGEbtmr17zdcjpm8c0A1I7jvQqsyODq+1AOJVOON2AKHQlXChpbbzPy3KAaPiuSF2t7nmUR5ZMmpmRAnsKaun0+YqgiGZnRGj/y35tlENCS2kSq0sLMR3Z0XN9RVyST+q2eglAmO453EtG6v0gtKGHpVVHZS1r6D8jnySw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kD/3FSSuo+0Qv8jCUpQXYHGDFD27mwPLJVLhJ2SwDvo=;
 b=b9jA8ENMX1dKkvS1h03kyY4tvZlkjJFGA4y6X4aush3wQtqPJwkEn0tUmsCY+7RHhKk2S7z6Bq6ETN4ypDK7CH65Mlou+c3Fk6mCx/0MI9JlkOoeXR2bHRtsSUzs4BZE44jCOzmNOfHkfWh1CCKmXY8KOMBGmc/wc5dm81MBghLoKtM1UbEMVqyicZRZF6tRV/ltm/b24D3IbZ737bkY0Ldh+GZjtwEIyWi57CjFWqjFHDo3n7C7oEqWf1g5fYctutlPVDQnAnpVPcb7EQCYf/tVBdvAKMP05L4s6+ZFJdskIYsO4FFoLPr33Vnf6a3+sMhjNzJ87fc1feCF7RaSQA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
From: Dario Faggioli <dfaggioli@suse.com>
To: Jan Beulich <JBeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>, "wl@xen.org" <wl@xen.org>,
	"iwj@xenproject.org" <iwj@xenproject.org>, "sstabellini@kernel.org"
	<sstabellini@kernel.org>, "mengxu@cis.upenn.edu" <mengxu@cis.upenn.edu>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"George.Dunlap@eu.citrix.com" <George.Dunlap@eu.citrix.com>
Subject: Re: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Topic: [PATCH v2 01/12] xen/trace: Don't over-read trace objects
Thread-Index: AQHXrkSLbMGUvlBXkkK2FQ1vuFXyFauzSsKAgARBtICABLtYgA==
Date: Thu, 30 Sep 2021 08:07:12 +0000
Message-ID: <db0443d4fc414e17e0bba075043f8ae60b062860.camel@suse.com>
References: <20210920172529.24932-1-andrew.cooper3@citrix.com>
	 <20210920172529.24932-2-andrew.cooper3@citrix.com>
	 <da06dfe42adad13650755650518a232dd41ac46e.camel@suse.com>
	 <8e3f7318-2fb4-0011-f582-268816ebb59c@suse.com>
In-Reply-To: <8e3f7318-2fb4-0011-f582-268816ebb59c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
user-agent: Evolution 3.42.0 (by Flathub.org) 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=suse.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 87d9c2b1-4fcb-4723-d80a-08d983e94f09
x-ms-traffictypediagnostic: AM4PR0401MB2307:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs:
 <AM4PR0401MB2307A045B7993C83823D0F03C5AA9@AM4PR0401MB2307.eurprd04.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7219;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 6lH2Wb7BDUoSmy52XfQONPCDo+pHLzc818/XBnEzQZMA7onY+9FWyvijskESWO6aymTHzAzMpGmVnmQbVsUV3DI3IoA/SKzZGO3P7OfOKpfzrGGEtevEYnB+hmsSdcvxbfM/umo7c5AOUN1DuuyXRQ13bkmfM62hkwwJAM9OD6N9ZtFp7KIufqN8Q+Cxyfel3TNAGNc610dFLGiwIfGdfRt+/1s8S+4KZdm9vJzxCCilhXaKh0tdYiXEUdK759VUSBerqRv027Va9pb3wri1K37E3kpm0Sv/MRb2NXkhQmDxoXorfme2Q8CspWtcDXL1cetrjS5WU8b+KRq4a89wUXpyoYsok29iy/h9W/AmI8tSSujREPKt4hqlCOIE3dCPVJ93xjsEZ0AnO940j6hUSr+FizIXtM3vvYWu/Q558NflSshfGr8Sl46FiY+BCeVAULpAca8b/XG3fgQY8yPiReaASJePLJl5gOyLWcAnU6g0mdOuf+FbP2DwFRfrLFjrD5Ls7RbQ30mdLtrSQzFUIb35es86/vvtqBc0/bfK09miHlZoQOmo5D2SZf/8DGjBUpp8Mbf/iEAsiVcXsaUJEcn48pKdgkrP3BfazdccZ3Ax3BEblpmfDd59i+p7EVtmbTkkgyC3ZHp+us7q84vS2ISTPcM6b/8JVThSTa7YieMj0XGVMK6S3ktQAQVDDJursSEUI4MdAhAR7L2tCHvnrwSZTaPjXAazeGhmsaRHr8M45TDxqfgfVMfhDXazwT52lPmjv/ZnEm1+e92ArwnVC9zhTglaE05QesKvLqKBrmjqnjJLpl7fOnaEVkYyduuUGaaXzw0wcuczzUdGTuWBCA==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB5826.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6486002)(38070700005)(36756003)(966005)(2616005)(508600001)(83380400001)(99936003)(6636002)(71200400001)(5660300002)(66446008)(54906003)(2906002)(91956017)(76116006)(122000001)(8676002)(26005)(6862004)(8936002)(6512007)(53546011)(6506007)(64756008)(66556008)(66476007)(66616009)(86362001)(66946007)(4326008)(38100700002)(316002)(37006003)(186003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?ZThFNUFJUFE2QmZZRkNVQ0x4NnhIMmhyWld5QVorMmV3dDFTL0tIR1JMeW8y?=
 =?utf-8?B?TmU2cEpzeDlpK3M4SlY5RjBjU0xVc01WZzBtMHJ5Qi9oZ1liL0lEbFJIMm11?=
 =?utf-8?B?dUdjc3ZGTmRBdzNqMHVBZGRXSzNkOFRic3hXYitNcnJjMzNwd25ZSlBtV05N?=
 =?utf-8?B?VFZ5QmxsWmxsRTViYVJiWng1SkoxZUNzTHFLSWtTMTZtRFJOUjlXSzJ4dVd2?=
 =?utf-8?B?Q1JBWkZRcVBvemhBYTJMdjZObTlxZ2FjM0FhcHIyT0taZDZFcnpmTVNWR2dT?=
 =?utf-8?B?cWxTTTVMNFBra1p3aWZzYWR1dWhFVDZZYXNsQUdFVEIvOW5uS2t2TGhYeU9P?=
 =?utf-8?B?TjBycDNNeU5RNGpRSndqdFd1UU1RYmNmUjJJdm9vbVcvM3RPQ05SSmtGSFYx?=
 =?utf-8?B?Q2lOSVdpaTdBdHhlK0ZXSXJzOWdwQW9rZ2VYbkZ2ZGU4bXAvai9SOGV3YWl5?=
 =?utf-8?B?VXdKM3U5dzRHcFJXL01pdFkvdy9ERHZRMWFlUTRzRTJDaC81R2hIZFdMbkM2?=
 =?utf-8?B?M2dkR25UTHUrWEc2UUF5bktHaWxLU0VaODlRdlNUSTZ5K3V2YmR6WHJLY3hT?=
 =?utf-8?B?MXUwQW54MkZkV0NyYXY1bEVReUh0YzFwWFVlK2RjMjliUk5RTVpuTTFMaTNL?=
 =?utf-8?B?dXFtZTcxTE5URlo5cnFhYkJGTGpXV1FTMDJqZDI2QTJtMCt2bnJ3VWIyam45?=
 =?utf-8?B?UGZqa29Eems3RGlRUEd4dHJUaHNoNDBuL3JwenRrdGNxVWhXVjlic0kydXQ4?=
 =?utf-8?B?VG02TTl1OUVDYncvTWRsMjBseXhyNVFaYTdUUnk1TEJ3bnJwMjhiZmpNVlZS?=
 =?utf-8?B?NmlQSXZEMnpSYzV2RG4ydHIxSUp4OWE1eHp0eEN6ZkFPdm5Jd0ZRbTk2Tlly?=
 =?utf-8?B?WUtRbEhpYnZ0ZWl4RnlhaHdkcEdyd2FPdFU5Y0NoMCtoNzBUS3FIZXNPcldC?=
 =?utf-8?B?UkRMdVZ1MlJ3V3dmRGJ5dDdIUXB1bHd1OUZUVE51YlgxSmR3VmhlRTd4YWdI?=
 =?utf-8?B?UlhwczN5dFVveEJzTXhWZCtHTGh4L2tBREVQVjMydzFZM0g0QmFVT3lwc3BS?=
 =?utf-8?B?UWVTZ095SHNDQmFVVVp0eHJDRmwrUjVySWZGZE9ORUtHby9uUTlZQXlJWWQy?=
 =?utf-8?B?dkE2bHM2Smx3Mm55ZkR6R1J4aHFUdDdOaHRPelM2QVpkVmFIZXBSNC9MZXNk?=
 =?utf-8?B?UkVvY2NsbzBkK3A2R0JxWm1nZXczVTFQUE5oSk9PMzB1cFJYOGp1RUZkcVlR?=
 =?utf-8?B?cUZWVGJDQWs3eHVRSnZxWjN5TlhBRGhFUDV6R2t1bzdwNXBjMTZDM1RnMUpn?=
 =?utf-8?B?R200ck96T2J1MXlFZ1JPL3R5VVd3VXQ1cWJRWVJaNUxJekdEVXlUUEVoaklZ?=
 =?utf-8?B?S0NHS2UxbzRLWENyaEEyZGdibVZESjZJU1VlYVpHSHppS01tRWVINjVrbG1y?=
 =?utf-8?B?ejNpelZ0THRER2dwenh0c0Erd2lmREJrUUxDdGFjbklwdnNKZ1JPcDBvdGkw?=
 =?utf-8?B?b2JRc0xvTXN0RDJ0KzFmSlBWSUZoNjJScXFZdHNNcXhnTEtMejRON3ZMdTEr?=
 =?utf-8?B?Q0lhVzJUTmF0Q3U4bG5uOURKWWVmNVE1cFBGN0djKzJsZVhSR0JnOGxqQWxo?=
 =?utf-8?B?Wm81ZEcxekVwYldPK2VUYWVyNkFXdDJVRFhna3ZsczJEVlNwNExvM051SURF?=
 =?utf-8?B?bWdWQXdTcStRQks5MkM4bFFwc0JzZzEzUmhPWWFhaWFVKzBXQ1dNbkNDYktE?=
 =?utf-8?Q?yC9BqEZfhE0q+ln5cbxlQc8CLzpkTSIrM0MtGbf?=
Content-Type: multipart/signed; micalg="pgp-sha256";
	protocol="application/pgp-signature"; boundary="=-xGn5+hbfUoMJp9efajUb"
MIME-Version: 1.0
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB5826.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 87d9c2b1-4fcb-4723-d80a-08d983e94f09
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 08:07:12.5500
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: T9jii470oxAjM/ybhHjJi672gXTAjladPFOAV7r4Xs7M1cUDFsxQEJKxpC80IYm1YfmkQVLleKCAUdvEQyfgPg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0401MB2307

--=-xGn5+hbfUoMJp9efajUb
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2021-09-27 at 09:51 +0200, Jan Beulich wrote:
> On 24.09.2021 16:51, Dario Faggioli wrote:
> > On Mon, 2021-09-20 at 18:25 +0100, Andrew Cooper wrote:
> >=20
> > > There is one buggy race record, TRC_RTDS_BUDGET_BURN.=C2=A0 As it mus=
t
> > > remain
> > > __packed (as cur_budget is misaligned), change bool has_extratime
> > > to
> > > uint32_t
> > > to compensate.
> > >=20
> > Mmm... maybe my understanding of data alignment inside structs is a
> > bit
> > lacking, but what the actual issue here, and what would we need to
> > do
> > to fix it (where, by fix, I mean us being able to get rid of the
> > `__packed`)?
> >=20
> > If rearranging fields is not enough, we can think about making
> > priority_level and has_extratime smaller, or even combining them in
> > just one field and decode the information in xentrace.
>=20
> I guess Andrew has tried to avoid re-arranging field order so that
> the consumer side doesn't need to also change.
>=20
Right, but is it really worth it, in this case?

Aren't we (very very likely) in control, by having them here in the
tree, of all the consumers? And is is this a stable ABI?

Also, both xentrace_format and xenalyze are being modified in this
series anyway...

Maybe there's still something I'm missing, but if we've getting rid of
those ugly bitfields and __packed attributes, it seems to me that doing
it completely --i.e., including in this case-- is worth the small
change in the tools.

Regards
--=20
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

--=-xGn5+hbfUoMJp9efajUb
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEES5ssOj3Vhr0WPnOLFkJ4iaW4c+4FAmFVcC8ACgkQFkJ4iaW4
c+76cA/+JRV2UjRtn7u7diOwwzTvGcK2MueTlmEJ5eRUme77UK+fUORK8W70UGEf
T1Nbb6/7Hvsb5SyzFjptqplwLIKw1feu09tGOwEWT8uObZKALXy/LjH2PWdVYmlN
gmEOuX2snBohCEZpdzlUrEzsZ61I+hXgEyCD+dQjLUEe3rGdDjS3Bt54ZURWzoxA
4QQ2VDoZ219h7tP1ReIRnTMpTQ8qX+nY3vsTGWR/4jomrN2cAHPv+Geua3oXSmq3
W1QN6OgBbRXlLnKLbf0ElRegnUsWwfC3/xSP/B9dQOh5HtMK/qTw9DQJiPjy7Cxb
TVroR5abq451hRAaOIgj1iUeO1VW1FPyDlTqSeLZ7GcBUnM2B/M0i+FTDEnmPx1M
MjFmfkWajEonnGCWhy9YZ4DC/p+Rw/5Gmn61dZ78iL05Rlb7tbg4ZGlVXEXjVhGt
KfbAM9P8Ms+MVKqqtT8LH9K1XZSLkJqv+InhHpeb8Xcj68K0A5bml2CpFEytZpzy
buEbXNcRXrALJPJ1rtP45loMQKpv8O4avhsAiFbUW3uc7I+oWGIDBhTZ/1+cHyxV
ToI83k5eoWye3TuWz2lgR4QhZ8cCFYpsRScUtvnJsD41Y4f8+CT8Kr5KRtVf4IBU
ZN+hQfbCxm8dMfrFwq49qA/db86gj78pX27VRTTu/TKEa+bTZ0M=
=8dMA
-----END PGP SIGNATURE-----

--=-xGn5+hbfUoMJp9efajUb--



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:22:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:22:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199600.353756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrK3-0005QZ-Ah; Thu, 30 Sep 2021 08:21:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199600.353756; Thu, 30 Sep 2021 08:21:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrK3-0005QS-7m; Thu, 30 Sep 2021 08:21:43 +0000
Received: by outflank-mailman (input) for mailman id 199600;
 Thu, 30 Sep 2021 08:21:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVrK1-0005QM-LJ
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:21:41 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6fa925f5-21c7-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 08:21:40 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-30-HRyeKV5vMCKyNBtJ0cSiqA-1; Thu, 30 Sep 2021 10:21:38 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5326.eurprd04.prod.outlook.com (2603:10a6:803:57::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 08:21:36 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 08:21:36 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0073.eurprd04.prod.outlook.com (2603:10a6:20b:313::18) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 08:21:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fa925f5-21c7-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632990099;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=JMUnRCvzD2pwGQ80VOLu5qbV6OaD3jNYdij6ak0Egf0=;
	b=FiMtZYO8dHkaD11qYVZajFEDkroJX6Xh0EYV+WgIqCAwlgBbmVzFyorWVirRfWTgKQLL11
	+sBiyZ3pW0RnJ3qfleX2EQyENq3aR+B3w2Tr354FGjk81q682ryxoYpn80Lk6OsWOJMYjX
	02YoCAFtr3pVoAVeXQUV4uAcjR3hI7A=
X-MC-Unique: HRyeKV5vMCKyNBtJ0cSiqA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Z70nkyE8g8ZVYEvfx42U8bjbazRoTUSA7o6PrG0cq9g+m1lAzkOn7OPkWZ1vvjTJjrxotKrTGNygWYKWWQuweF8kklm3ECPH+jO138+o2qfxLs1Vldx6LpW5yxhhKbEWzHIRtn/OXlnO/cJlNn+J4inZpoXl3wINfzaDTm/XpbA5+n/PuCMKhVk54RLiQvhDPly33OWufTkcRJ3Rz1XAAB8IiDK8KNYHbRkgYjDFDshzxZCsDEPs/wdDhNNdjvsVvFHB1Jj3TSto9lsZY2QIIbujSmOFkPAAKEktA2PzTdQTUY2lR2WfICMVDqUN3DH6V4B21+vv2Fxm9iXLywk7ow==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=JMUnRCvzD2pwGQ80VOLu5qbV6OaD3jNYdij6ak0Egf0=;
 b=jglEKsWRqJHMoJ8w+6d/Q6f7l71cuzTvLaRHgTqtDXTRKZiNn4bhmuVagrTzsSnPjUmSrs1YnfnV1zXy0oS5Ew6eRURUHHt9d7+lsIMFQpSEL3XoEMgR/Vr4F/ggKaHfli7yNkP9AY2LU4g+0ZgKzsNgYFJOenQp2Og/BBeHO/ClE1IMe8Bb7KOQcsRYKKWAI4Lhn+i3fOI3U9A28SVvQIcY/oEAlUR38fqDabxaPoDgGgJ0zz/jcPJiMQLMEtjtxSDa/Z03ZnVpr1zKD6LdVAMk5hGEe/ZW0DDJvu23TCBvRL8W5gcyqSiN5vlVhIFxVg3k2wgY7r3IttfdGPmmBg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
Date: Thu, 30 Sep 2021 10:21:34 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930075223.860329-3-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0073.eurprd04.prod.outlook.com
 (2603:10a6:20b:313::18) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9cc0fe0f-a1d7-4d3f-3ffc-08d983eb51ee
X-MS-TrafficTypeDiagnostic: VI1PR04MB5326:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5326A8A5C34879206BF758C1B3AA9@VI1PR04MB5326.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2089;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wjhjvsqVwxgIoUtGMdGdqA/8Q6kuWJX1PKARUB312mFgXRXcSd1erLpww9IC3whEPS66eMsT5zko56JA4YSgyEVwAd2VayOPZCeY43awkryRzhjnoFFADVtVWiAA+4xl0+2t0FX4XE3nMZioMc0i6gsMDCgVC6NYK3xdoHZD/SzCddeYDpsbf8mlWSwwDQ4ahnNwmwhC9QFuEw7DJkgjayzEA6kkbkRx4Mm4V5nY1HPC1QmKmWXsd5Epktubi7Dvt48EMuEpDbpNrhNHwDjKsbkK92cDNLJp0jhYRS6QCU7SMlemyp/qsRlfBTWI8coQCs89P7cMBGXOZBosXPCs04+KReInnUBckYQfRON9Qwa08ucDuCaDWkVwHj0NZLhPxTOgMwPYq+TCqj5GWqosaGQ1KalIwytFU/AYcMGvRkcfLkI6NsXn7AMomivQozztBiZ3uxhUSFDRLcMAF5mZcIG3+FvFUebBj6MeY5iuBaJ59+ucy/tZ7QatTh8uFa8xUFU/6IIPgFdVqfw1UJT3e/xzoWd/htVzKkCHo1CqMLCOYouge+GJ7UGEHzreGugUIplrf+yyzP9hkmUd946FowzBnU7faXgENxhnph1QOiaf3k1I6xC34sLPUg9mu8RlOi/NTPALSSKKXlDp/QX/vEE85TpS8acZ6QFFZTHge4HxGUg09tTYeXKLJP2szrR0Ieu2DGal1zflWCISRRm1a5FsGy+zpFTg/7aGXqM511c=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(8676002)(26005)(5660300002)(186003)(8936002)(53546011)(36756003)(2906002)(83380400001)(7416002)(31696002)(316002)(31686004)(66946007)(6486002)(66556008)(16576012)(66476007)(2616005)(6916009)(508600001)(956004)(86362001)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RnVKcFVjN2RmMWtBNHhhZ21RL01KZjZoOEtBOXkxM2dLT24yWmVkQ2NaZzhI?=
 =?utf-8?B?dnl0clYwK3hkZVdhckNFTFJ2VjJhYnJJeGVJWS9tcmE4OEhmNW9LOGhWOUtD?=
 =?utf-8?B?SVgyb3VraUgvMWtWVCtCem9TYWlxSWxzQldhZzVaN1VTSzB1STdrMi9WalE0?=
 =?utf-8?B?SVg5ZDl3YWpxN2xFUUYycW92YmFtRE9uWUxKclJPZFMvQ25uQ2Izd2FIY2hM?=
 =?utf-8?B?eDNaZjY2Ty9IczQzZGN5REQyMGZHWXI2M0M0eXBiUkRrRXU3bXFNU1JZa2Vq?=
 =?utf-8?B?K2hQUEdoUE9RMks4VjUrc1RWdTFJTS9oeU42VVFVSXBLS2x0Ull0Rzh3RGxF?=
 =?utf-8?B?MXlrdWVXT3FwT3VGbTJGa3k3RnpjdE8ya1FGcnBBbTR4YnRhTjM4RjVWcHZl?=
 =?utf-8?B?WXdkQTZ2c0pTMENvM0JMdWNha3N6ckFqTnBraGxUemdzeXQrRzhpKzJzYlE5?=
 =?utf-8?B?OURRT1dwdmhuVUZEZFM4SWtOREZuT0Ryd0ZEN1ZBNExUejkwZlBDVkZZa29L?=
 =?utf-8?B?VXVlT3JhV2p2U1NiSzVmU0dOUG93Ujg4NmFmcFA3YU0zVmFmaVJUY0dad0J1?=
 =?utf-8?B?WGZqOVI4bjE4ZmlXN2JOMnI2N3JWUGtrRjU2VzhiNXRZMGl6ZEIxL2hra09C?=
 =?utf-8?B?RDQ1UmVoZGJCaEhqdFh1WSs1eURZRHVDUFJoQURIMzV2S29ycHBzSDNIUGRS?=
 =?utf-8?B?M1pjdnR3SW9OWlUreXBpYXcxZnFkWm44a2Rrb0VoUEhpWnI3SVdLbm52QVJw?=
 =?utf-8?B?eUZDays1NlB2SExST3YraFpsVjBmaWE4cHJROVFzQjIzMlR0dDJDeUtvT2cx?=
 =?utf-8?B?RXVVVy9QaEZlWFZqMDNTaEJNTW9EeFNsbGNxYzVWSnF2aGlkQ0VZWW9kbUl4?=
 =?utf-8?B?OHpOOFcwamVZMlFMak9WZnhjTGtpMWJSaEFxZFBqbXZwcFc3LzlQcHgvWGtW?=
 =?utf-8?B?SDU2cENoZ1J3UHZEcEdORGtYVzdQZW1NN1E4OVQ5Snh0QTI0TncwazIyb2hr?=
 =?utf-8?B?MUNLcVdKZWJURjU1dGlqcytWS3NvTkxjZHJ4Um50MW9GQlF3RTBFRjJOMG9Y?=
 =?utf-8?B?ZFVBd2RRdVA4eitxZ2Z6bTJnSFJXR1JMVXY5eDZSNW12M2xBOHhWODJlWndD?=
 =?utf-8?B?REFaVGtPZHNGdlVZRExXUzhoZEhsU1E1THFsd3RmY0ZCc2U5TzVMOERydDJP?=
 =?utf-8?B?ZTZzQUxsaUI3a2piaDJOVXpwWCtzWU5RUGZjd203Z0ROLzJWWFJsOVJnNDEr?=
 =?utf-8?B?eDRnOTRuSkZVaHUrWlJlNW9yV3I0Y0ZBbUFEcDNPVXlaTVMrTnUzeVdWSFI1?=
 =?utf-8?B?YlIvQWdIYjA5SjlTTkQvQmVFVjdndzJvMzJBNlVONG1ZWm1QUEVhays5eVZL?=
 =?utf-8?B?M3FyMzlFOGRvZENZL0pIbEd3dGxlS0QwZUE4OVIvaHlJaENPQURRL3NEczZy?=
 =?utf-8?B?dnJLR1dSQnRHbUdSdlk5WUJwK3F5MlpaMk8vNFFQbTliQ1dNdGk0QXZ5dC9m?=
 =?utf-8?B?RVltMmdjQ2xMUGkwTjdlZGlNKytVNmxDcjkxOEs1bzgyN0FFZWljYUdoOStE?=
 =?utf-8?B?SStNZ1NKMy8vWUlGQ1ZXQ2JFenZvME9QTDJyb0d4WmlTTU5HaDdkUjZ4ZHVW?=
 =?utf-8?B?WUNvQkxYekUwWnYydHNwOVZ0T1lieTFrZCt0QjV4RHByMUk0ZHZWc2sybVg0?=
 =?utf-8?B?VGNiUUUxZHhBOFlFSVpUanNhVktFc0dsMFprTVcvRnVncVBFTGkvQXdrWHkx?=
 =?utf-8?Q?e6yUa34FM82k+t2LcUjYYvebjbIw1igQOJplanv?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9cc0fe0f-a1d7-4d3f-3ffc-08d983eb51ee
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 08:21:36.5806
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Lbyo1GoHCVpEXco9OQaKubgiiIVf1ONK24v44R7/uqrn4px69YHMJ/9VdDk5Z8fU1u8qLSMuML/txCReRDRdOQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5326

On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> When a PCI device gets assigned/de-assigned some work on vPCI side needs
> to be done for that device. Introduce a pair of hooks so vPCI can handle
> that.
> 
> Please note, that in the current design the error path is handled by
> the toolstack via XEN_DOMCTL_assign_device/XEN_DOMCTL_deassign_device,
> so this is why it is acceptable not to de-assign devices if vPCI's
> assign fails, e.g. the roll back will be handled on deassign_device when
> it is called by the toolstack.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> ---
> Since v2:
> - define CONFIG_HAS_VPCI_GUEST_SUPPORT so dead code is not compiled
>   for x86
> Since v1:
>  - constify struct pci_dev where possible
>  - do not open code is_system_domain()
>  - extended the commit message
> ---
>  xen/drivers/Kconfig           |  4 ++++
>  xen/drivers/passthrough/pci.c |  9 +++++++++

The Cc list does not match these files getting modified. Please make
sure you Cc maintainers, so they have a chance of noticing that
changes are being made which they may have an opinion on.

> @@ -1429,6 +1433,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>          rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
>      }
>  
> +    if ( rc )
> +        goto done;

>From all I can tell this is dead code.

> +    rc = vpci_assign_device(d, pdev);
> +
>   done:
>      if ( rc )
>          printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n",
> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -86,6 +86,29 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>  
>      return rc;
>  }
> +
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +/* Notify vPCI that device is assigned to guest. */
> +int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
> +{
> +    /* It only makes sense to assign for hwdom or guest domain. */

Could you clarify for me in how far this code path is indeed intended
to be taken by hwdom? Because if it is, I'd like to further understand
the interaction with setup_hwdom_pci_devices().

> +    if ( is_system_domain(d) || !has_vpci(d) )
> +        return 0;
> +
> +    return 0;
> +}
> +
> +/* Notify vPCI that device is de-assigned from guest. */
> +int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
> +{
> +    /* It only makes sense to de-assign from hwdom or guest domain. */
> +    if ( is_system_domain(d) || !has_vpci(d) )
> +        return 0;
> +
> +    return 0;
> +}
> +#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */

At this point of the series #ifdef is the less preferable variant of
arranging for dead code to get compiled out. I expect later patches
will change that?

> --- a/xen/include/xen/vpci.h
> +++ b/xen/include/xen/vpci.h
> @@ -242,6 +242,26 @@ static inline bool vpci_process_pending(struct vcpu *v)
>  }
>  #endif
>  
> +#if defined(CONFIG_HAS_VPCI) && defined(CONFIG_HAS_VPCI_GUEST_SUPPORT)
> +/* Notify vPCI that device is assigned/de-assigned to/from guest. */
> +int __must_check vpci_assign_device(struct domain *d,
> +                                    const struct pci_dev *dev);
> +int __must_check vpci_deassign_device(struct domain *d,
> +                                      const struct pci_dev *dev);
> +#else
> +static inline int vpci_assign_device(struct domain *d,
> +                                     const struct pci_dev *dev)
> +{
> +    return 0;
> +};
> +
> +static inline int vpci_deassign_device(struct domain *d,
> +                                       const struct pci_dev *dev)
> +{
> +    return 0;
> +};
> +#endif

Please put the __must_check also on the stubs, or else people only
build-testing x86 may not notice that they might be introducing
build failures on Arm. Then again I'm not sure whether in this case
the attributes are necessary in the first place.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:43:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:43:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199606.353768 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrfE-00082n-3d; Thu, 30 Sep 2021 08:43:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199606.353768; Thu, 30 Sep 2021 08:43:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrfD-00082g-Vj; Thu, 30 Sep 2021 08:43:35 +0000
Received: by outflank-mailman (input) for mailman id 199606;
 Thu, 30 Sep 2021 08:43:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVrfC-00082W-HV; Thu, 30 Sep 2021 08:43:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVrfC-0003vh-Cf; Thu, 30 Sep 2021 08:43:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVrfC-0000PY-2s; Thu, 30 Sep 2021 08:43:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVrfC-0007MK-2K; Thu, 30 Sep 2021 08:43:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PEGxfrrPvn3wdZcd95ayAQkZDM5DjaDQxiVocv1F2+s=; b=0vxzyRXuDQJZlXHuznIyyiRUpJ
	P/BPMAWvADKMuEcKjG5fRvYLKfOpnJOFaQ/VwjwpB9x9jDk/d6WuMBT73PaPvakgfbNCki06oD9HA
	RkRu8FGbokTT/GC+yR8wr3vLsPuv99TduQe8PCFO1VuFYb5zgDxM7wgZJK5Gf3odNPDo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165321-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165321: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=442e46d3b6c1931b54111c92e0efb5a797bc622b
X-Osstest-Versions-That:
    ovmf=46b4606ba23498d3d0e66b53e498eb3d5d592586
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 08:43:34 +0000

flight 165321 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165321/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 442e46d3b6c1931b54111c92e0efb5a797bc622b
baseline version:
 ovmf                 46b4606ba23498d3d0e66b53e498eb3d5d592586

Last test of basis   165200  2021-09-26 03:10:18 Z    4 days
Testing same since   165321  2021-09-29 20:41:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ard Biesheuvel <ardb@kernel.org>
  Bob Feng <bob.c.feng@intel.com>
  Cheng-Chieh Huang <chengchieh@google.com>
  Guo Dong <guo.dong@intel.com>
  Leif Lindholm <leif@nuviainc.com>
  Liming Gao <gaoliming@byosoft.com.cn>
  Pierre Gondois <Pierre.Gondois@arm.com>
  Rebecca Cran <rebecca@bsdio.com>
  yi1 li <yi1.li@intel.com>
  Zhichao Gao <zhichao.gao@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   46b4606ba2..442e46d3b6  442e46d3b6c1931b54111c92e0efb5a797bc622b -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:47:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:47:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199613.353782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrj5-0000Oi-Lp; Thu, 30 Sep 2021 08:47:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199613.353782; Thu, 30 Sep 2021 08:47:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrj5-0000Ob-IO; Thu, 30 Sep 2021 08:47:35 +0000
Received: by outflank-mailman (input) for mailman id 199613;
 Thu, 30 Sep 2021 08:47:34 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVrj4-0000OV-64
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:47:34 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0f74e1c9-0d57-4488-ab17-4aeefc265fce;
 Thu, 30 Sep 2021 08:47:32 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U8dKGv016584; 
 Thu, 30 Sep 2021 08:47:30 GMT
Received: from eur03-am5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2059.outbound.protection.outlook.com [104.47.8.59])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bd9wmr3m3-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 08:47:30 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM4PR0302MB2660.eurprd03.prod.outlook.com (2603:10a6:200:93::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 08:45:38 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 08:45:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f74e1c9-0d57-4488-ab17-4aeefc265fce
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nKEY/RYjZb4OROWnDF6mY54uza0dZmHjo+xUdM3ZYZt1K1Sh0ebI13HE+v4SmN3N8eQTb9dUKiazpCI707QBFMVQmFrHYROLtaMnyIAVl5m3oh8cEXumy/Hd18bIFD41vg5GAb1MhiSNSSTjNMqpT2QcivRp7Xe5w1SCfQyjgw5bNyEIGSk3/q3Izc2nEYR3RIRSXEePxzveoDo8s/9IvjdIJp2cv0eHhoxKCscGcNxD2vYGMO6tGWFOV1uZTDmrks3o7ItIgYlcbXQtAM0DWqJeIJX/Pc99IfOXwhPpXtOdbJ0J8Fij9l9/Yt2H6YUu2tCvoLb4uvVVyZx1Q0IeLQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lyunuClZk8k0l4m5JL36m3Pi8UWDE4GqKGkuRzdRpgM=;
 b=ABe1GXdaLxRFcS5Joe4sSxn9Z8Ym51m7SfX/ayp1KSZTQ+Ohv/B8SkkFcJlyw01SUyTQ3XAAzcBLosQFjwrXsKS+BLp2ZofZ+WW0odSpAyIbPFbN/8RJiNwQP1umwf1HSCklyiSfg3pIfTg9J+3rSUbwGw6ASsCdjk7e0rlsjco4zu7CUHGGL/xf5XPGb8Fcc3GcVG30gi6aUI1blyEeAjYztthSIXlQVKiGrqLjE+naKnJqb1AuTKj6J/GdM71zlz1JQIBIt+1SYfxmAQ0+pFRtbvtrCSXLAoCPyaQso0wraIG6xk8E9jpAlWL1gSbuqy2agqJWHrXOr2t/ScClhw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lyunuClZk8k0l4m5JL36m3Pi8UWDE4GqKGkuRzdRpgM=;
 b=WkuRcIFMbwpYQ5D4VinOdJct7UuCQhj9pslcUPoRtEQW/o7HzSbVp7MYQM254zGJCGfZMlidfWRhieG5NvE9E9R519QZLaZGyGMXqufZq0ADHh9N1EclAzlT33wWhemJ64S3tZrPpnLmlTlkcuUKiACuFfgfcxRvIi24HHRj5U83N/i+MDn1t4mOzMJ6l8ANrZuXXgC1rPVaJZOpixDVLBW2hqE8PKU5d723TvWy6Rhk+z6eMXx8ofop6nriN3T/8DNcu/vSlomPZK0vLfKx3MoHZqRbB2v45JVSqxVWfmVYsCxbglB3DNRAy1BigzpyuowK9Qb/9Sginvk7p9ZGgg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Topic: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Index: AQHXtdAg2qzxZ1hIP02wGTFUbQtoPau8PLsAgAAGuIA=
Date: Thu, 30 Sep 2021 08:45:38 +0000
Message-ID: <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
 <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
In-Reply-To: <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 82f6ccf0-ec68-49e0-6a24-08d983eead4a
x-ms-traffictypediagnostic: AM4PR0302MB2660:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM4PR0302MB26605ED06EF9B770B618D46DE7AA9@AM4PR0302MB2660.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5797;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 uG8PYGPPJpGTGFV+grXzArpTq6YquiJuz9W9cOWH2ZshRtuRyHmC1nqsNnkRweHniDkN0jrplv9pOTwcI9uSWKnXGVFByOAPNrxqr6bhEIGwG1c4GLyBPU+Vjdu9xSvL0Xrcy3HO4MHy22ocLsohegNCTBpk6KkRk/GzVIWN0OUDJpb3+zdOH4AXXZ94+y8s/JHyadWWdgGr7cIVztRtyZTdMpJy1igBqJAg6IpkZAODDm0oCrHfcR8e+50cYAJu2TbL+XJ4pJMF3S8ZWxlAui4ZmptS8GVUsexCXJToX9gA9e9c9G/VBANp3PqsKG1HQwON32+DgnzylZZFgneJTX2MMI0fqmsAOO58FaskBnLuCQhNOeO1AKvzkuEFFAt1XPXyZsulFZX+7Y+Ol+5JU0fwTTJA4Zu1FR64m4pb1YFKQIcQfsKO9FTYWBxH/GTiBQfmtyYj2Dnlw/pl/WdNwJbEhy2QXhGkEhzWX6A86KzG5bfU/Rkl88cXMtmzxHy+q+K6qxLITN20fsV6pxFqhmJvY1rKPyCFKVWkp5Y8R7JFjgM2QPqgVg66g+/b1+10hlBUocw5D0ewEqcSck4Vp9WL1Ba3OzJYtyZX36ev+fIHDwEg+VS7VlYrhwsXkuKNVbK2XF7pnaPxsd5guiyFCht4MlKZLk2EqlQixKtMwbNCveW8l3Kh0+iFNsuf/qw5o7U/zON33ecjyDiwKALzL8/WjWJFs4GtTmKA1I3IeZ3F9U/HQj2Ln9YE1ihlECfv
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6916009)(66446008)(26005)(38070700005)(31686004)(66476007)(66556008)(64756008)(38100700002)(122000001)(2906002)(8676002)(83380400001)(4326008)(86362001)(55236004)(36756003)(91956017)(66946007)(76116006)(8936002)(6506007)(508600001)(31696002)(54906003)(6486002)(316002)(71200400001)(53546011)(186003)(6512007)(2616005)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?aUxSWTRIcFBLSlYrc1lHamxxWUE1TDlWMGVnYlpwWmY4emw0bStuNUtXRi8w?=
 =?utf-8?B?NnhpK3NmLzJ4aWZueGE1Zm53OW9ydGpCb2VISVd4V0V3azBMR0YzU0xLcFVm?=
 =?utf-8?B?VjdyMzBCc2wwbDJKYzk0SmJ4bExIdk92K2VYc2h2UkMvVGFvOEpKYVBFWFpn?=
 =?utf-8?B?R0g2eUlsUlZudHY4RmJtelJERE9OeEZOeDA3TlNxdlQ2UHg5TXlUU3F5VlNl?=
 =?utf-8?B?T2pNVlFhLy9CQ3JnSGFWMHcrWTdxZ2hQczkyNmEwR3NSZmhSRmhmSWdPMk12?=
 =?utf-8?B?cExVYmRSUDZKc3lIZURVYnF5SkpnaStRNHduOEh1V3BDYkQzR0drQUZ6c1FX?=
 =?utf-8?B?bVFRdkxxRWZid3hHVFY2d20wMTIxYWFtR2Uvdk1HYzhXd3hLelJFSEZCYlBF?=
 =?utf-8?B?M1VCRGJuMXRSazRHQ1U0WHI2djlCNzhWYjY1S09iVWVNN3piRXRkY2hwNVYx?=
 =?utf-8?B?bDhkMjNsd2R2YXBlOHB2V1VGaTZ3N0NiaS9wNUdDcHd1MkJBZHVUK0FwMGo0?=
 =?utf-8?B?RWluZmczWWtSVXo1V2p2WHFhZjhTWmVuOVZQUjNMS1BxWGQ4NWRNU0VqQ3lN?=
 =?utf-8?B?MVZjclVlaEFDUFdCY21LZ1pFSXROYU1TOVBDSXdkWW1KTVdsNzJVNmxxUEh2?=
 =?utf-8?B?dzFaN1lPVUpQMndVeUhtTnY0dTJoSWplVUdoM3RoT01zTmN6ZjlydVg4SmlT?=
 =?utf-8?B?VnJsOGwvSm9RQTZNMHFHVjRDNnRPaTExOXRzdVNjRmFreXJuOVgwdkxFMlZL?=
 =?utf-8?B?NUx3MVdGNGthZk5SNjhHUzNkTHdhWThWRU1hdE9hMXJrL1RCQTlSdlRaWnN2?=
 =?utf-8?B?TTVWSTN4N3NBdTlLREFwL2wrSTJzNHRCK3Q1aEo0SHYzQ1NReGNjZFM0RVV4?=
 =?utf-8?B?N2hyR0VaclpBaTcrRXBlSmpnYmdveHFHTHNLVkRQRlFSM254ZUU1UUl5TXhS?=
 =?utf-8?B?QW01N25jbVNON2cvblV0VTZ2SzRFd1dYdkdsaGxBTnBmUzluQVA5cDliYzVU?=
 =?utf-8?B?NW45WTZJblYyeFdrdWltZHR4bUNIZlhJaFZxb1dXTkxVYzlnTDNCcmZhZTl2?=
 =?utf-8?B?NnJkdEliTFVKWXpEd1RDWnI3Y204SUd1SWFVeWtsamYwM0h3OFdtc2V6VXJT?=
 =?utf-8?B?Z0g3Zk9LOEFDYkFaTGR6WnZ0Z3ZTTmp3eW0vWE5DL3YrWTIzMEdWNU0rS1Ja?=
 =?utf-8?B?NXZFMnFBbnJnM0cvVytKaFc2M2hrOUxVV0t0VzFJM0FOOVRrdm5BdGVJL1NO?=
 =?utf-8?B?SUg3anhwQWs2TDFnM3J3VnhjQjVUcW5IYUVoSFRvZTdBczUzcTRadHlhV0dh?=
 =?utf-8?B?TytDWGM2anZjVUlJa1gzaW9TSzhEWFNxNHdkYTFPVEtyeUFWTUF4bG5XN1lj?=
 =?utf-8?B?cUk5Y2JBYlNWMlFIek0xY3NoTTUrbGdWb0dIazBUL3pOMDRqM3YrTW1sbm4w?=
 =?utf-8?B?R1YyQUVyQnBFNzBLVVMzQTBLYjBSeE9XQ3Z2SWNqQU9ZTWl0MGZidDdxSkw4?=
 =?utf-8?B?WS9SOXlwOHdlSUE1eXVKd1hGVHM5NWd6RnhjZFBNMkRJN0VQK25ZS1ZucDJv?=
 =?utf-8?B?cjd2UDNRc21ZZUhPSm5TaW1URGJmYXBlZHBBVzhIdWJLVDBGb0gxTlFzVmJI?=
 =?utf-8?B?dUp6QkZFNTVwM3hLNGR0SDVFVEdkMVpYWk5wQitXOGIzeFBFK1pWaXMzNXNJ?=
 =?utf-8?B?cjYrcUs1ckZhYWVLVmtFZC9IWTR6c0NSZTQ5YzRRaVo3cmNhOGJFTjZEbXRO?=
 =?utf-8?B?ZTJuRFBoRTJ5NWhVMGJLMmJuTUdieldPQWtmbjNWRzNqUEptTDVLVEJkendo?=
 =?utf-8?B?c1NlRy9kVnpKRXlXN2lBdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <D8BCE7796DF9E843ADB37C71D2FEB20F@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 82f6ccf0-ec68-49e0-6a24-08d983eead4a
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 08:45:38.0534
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qehk+1uJl1I+qWdCvbGI/ZrVq1Jbu/qjViXai3ZGWIJaySkOzOHk6RrDzfsDDApcYu5hG8CzD9M7DJY45f/8bt6VEicJFEt/gcGW/wdf5Cl4v6tNDR0wrP3/YX/G77Qc
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0302MB2660
X-Proofpoint-ORIG-GUID: 0moApjGEPA1Hn9YA62cbriQPpcDAi_Y-
X-Proofpoint-GUID: 0moApjGEPA1Hn9YA62cbriQPpcDAi_Y-
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_02,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501
 clxscore=1015 impostorscore=0 bulkscore=0 spamscore=0 lowpriorityscore=0
 malwarescore=0 mlxlogscore=999 mlxscore=0 adultscore=0 phishscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300053

DQoNCk9uIDMwLjA5LjIxIDExOjIxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAwOTo1MiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBGcm9tOiBPbGVrc2Fu
ZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pg0K
Pj4gV2hlbiBhIFBDSSBkZXZpY2UgZ2V0cyBhc3NpZ25lZC9kZS1hc3NpZ25lZCBzb21lIHdvcmsg
b24gdlBDSSBzaWRlIG5lZWRzDQo+PiB0byBiZSBkb25lIGZvciB0aGF0IGRldmljZS4gSW50cm9k
dWNlIGEgcGFpciBvZiBob29rcyBzbyB2UENJIGNhbiBoYW5kbGUNCj4+IHRoYXQuDQo+Pg0KPj4g
UGxlYXNlIG5vdGUsIHRoYXQgaW4gdGhlIGN1cnJlbnQgZGVzaWduIHRoZSBlcnJvciBwYXRoIGlz
IGhhbmRsZWQgYnkNCj4+IHRoZSB0b29sc3RhY2sgdmlhIFhFTl9ET01DVExfYXNzaWduX2Rldmlj
ZS9YRU5fRE9NQ1RMX2RlYXNzaWduX2RldmljZSwNCj4+IHNvIHRoaXMgaXMgd2h5IGl0IGlzIGFj
Y2VwdGFibGUgbm90IHRvIGRlLWFzc2lnbiBkZXZpY2VzIGlmIHZQQ0kncw0KPj4gYXNzaWduIGZh
aWxzLCBlLmcuIHRoZSByb2xsIGJhY2sgd2lsbCBiZSBoYW5kbGVkIG9uIGRlYXNzaWduX2Rldmlj
ZSB3aGVuDQo+PiBpdCBpcyBjYWxsZWQgYnkgdGhlIHRvb2xzdGFjay4NCj4+DQo+PiBTaWduZWQt
b2ZmLWJ5OiBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29A
ZXBhbS5jb20+DQo+PiAtLS0NCj4+IFNpbmNlIHYyOg0KPj4gLSBkZWZpbmUgQ09ORklHX0hBU19W
UENJX0dVRVNUX1NVUFBPUlQgc28gZGVhZCBjb2RlIGlzIG5vdCBjb21waWxlZA0KPj4gICAgZm9y
IHg4Ng0KPj4gU2luY2UgdjE6DQo+PiAgIC0gY29uc3RpZnkgc3RydWN0IHBjaV9kZXYgd2hlcmUg
cG9zc2libGUNCj4+ICAgLSBkbyBub3Qgb3BlbiBjb2RlIGlzX3N5c3RlbV9kb21haW4oKQ0KPj4g
ICAtIGV4dGVuZGVkIHRoZSBjb21taXQgbWVzc2FnZQ0KPj4gLS0tDQo+PiAgIHhlbi9kcml2ZXJz
L0tjb25maWcgICAgICAgICAgIHwgIDQgKysrKw0KPj4gICB4ZW4vZHJpdmVycy9wYXNzdGhyb3Vn
aC9wY2kuYyB8ICA5ICsrKysrKysrKw0KPiBUaGUgQ2MgbGlzdCBkb2VzIG5vdCBtYXRjaCB0aGVz
ZSBmaWxlcyBnZXR0aW5nIG1vZGlmaWVkLiBQbGVhc2UgbWFrZQ0KPiBzdXJlIHlvdSBDYyBtYWlu
dGFpbmVycywgc28gdGhleSBoYXZlIGEgY2hhbmNlIG9mIG5vdGljaW5nIHRoYXQNCj4gY2hhbmdl
cyBhcmUgYmVpbmcgbWFkZSB3aGljaCB0aGV5IG1heSBoYXZlIGFuIG9waW5pb24gb24uDQpTdXJl
LCBJIHdpbGwgZ28gb3ZlciB0aGUgcGF0Y2hlcyBhbmQgcHV0IHJlcXVpcmVkIENjOiBuZXh0IHRp
bWUNCj4NCj4+IEBAIC0xNDI5LDYgKzE0MzMsMTEgQEAgc3RhdGljIGludCBhc3NpZ25fZGV2aWNl
KHN0cnVjdCBkb21haW4gKmQsIHUxNiBzZWcsIHU4IGJ1cywgdTggZGV2Zm4sIHUzMiBmbGFnKQ0K
Pj4gICAgICAgICAgIHJjID0gaGQtPnBsYXRmb3JtX29wcy0+YXNzaWduX2RldmljZShkLCBkZXZm
biwgcGNpX3RvX2RldihwZGV2KSwgZmxhZyk7DQo+PiAgICAgICB9DQo+PiAgIA0KPj4gKyAgICBp
ZiAoIHJjICkNCj4+ICsgICAgICAgIGdvdG8gZG9uZTsNCj4gIEZyb20gYWxsIEkgY2FuIHRlbGwg
dGhpcyBpcyBkZWFkIGNvZGUuDQpCZWZvcmUgdGhlIGNoYW5nZSByYyB3YXMgc2V0IGluIHRoZSBs
b29wLiBBbmQgdGhlbiB3ZSBmYWxsIHRocm91Z2gNCnRvIHRoZSAiZG9uZSIgbGFiZWwuIEkgZG8g
YWdyZWUgdGhhdCB0aGUgd2F5IHRoaXMgY29kZSBpcyBkb25lIHRoZQ0KdmFsdWUgb2YgdGhhdCBy
YyB3aWxsIG9ubHkgcmVmbGVjdCB0aGUgbGFzdCBhc3NpZ25tZW50IGRvbmUgaW4gdGhlIGxvb3As
DQpidXQgd2l0aCBteSBjaGFuZ2UgSSBkaWRuJ3Qgd2FudCB0byBjaGFuZ2UgdGhlIGV4aXN0aW5n
IGJlaGF2aW9yLA0KdGh1cyAiaWYgKCByYyINCj4NCj4+ICsgICAgcmMgPSB2cGNpX2Fzc2lnbl9k
ZXZpY2UoZCwgcGRldik7DQo+PiArDQo+PiAgICBkb25lOg0KPj4gICAgICAgaWYgKCByYyApDQo+
PiAgICAgICAgICAgcHJpbnRrKFhFTkxPR19HX1dBUk5JTkcgIiVwZDogYXNzaWduICglcHApIGZh
aWxlZCAoJWQpXG4iLA0KPj4gLS0tIGEveGVuL2RyaXZlcnMvdnBjaS92cGNpLmMNCj4+ICsrKyBi
L3hlbi9kcml2ZXJzL3ZwY2kvdnBjaS5jDQo+PiBAQCAtODYsNiArODYsMjkgQEAgaW50IF9faHdk
b21faW5pdCB2cGNpX2FkZF9oYW5kbGVycyhzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+ICAgDQo+
PiAgICAgICByZXR1cm4gcmM7DQo+PiAgIH0NCj4+ICsNCj4+ICsjaWZkZWYgQ09ORklHX0hBU19W
UENJX0dVRVNUX1NVUFBPUlQNCj4+ICsvKiBOb3RpZnkgdlBDSSB0aGF0IGRldmljZSBpcyBhc3Np
Z25lZCB0byBndWVzdC4gKi8NCj4+ICtpbnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21h
aW4gKmQsIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiArew0KPj4gKyAgICAvKiBJdCBv
bmx5IG1ha2VzIHNlbnNlIHRvIGFzc2lnbiBmb3IgaHdkb20gb3IgZ3Vlc3QgZG9tYWluLiAqLw0K
PiBDb3VsZCB5b3UgY2xhcmlmeSBmb3IgbWUgaW4gaG93IGZhciB0aGlzIGNvZGUgcGF0aCBpcyBp
bmRlZWQgaW50ZW5kZWQNCj4gdG8gYmUgdGFrZW4gYnkgaHdkb20/IEJlY2F1c2UgaWYgaXQgaXMs
IEknZCBsaWtlIHRvIGZ1cnRoZXIgdW5kZXJzdGFuZA0KPiB0aGUgaW50ZXJhY3Rpb24gd2l0aCBz
ZXR1cF9od2RvbV9wY2lfZGV2aWNlcygpLg0Kc2V0dXBfaHdkb21fcGNpX2RldmljZXMgaXMgbm90
IHVzZWQgb24gQXJtIGFzIHdlIGRvIHJlbHkgb24NCkRvbTAgdG8gcGVyZm9ybSBQQ0kgaG9zdCBp
bml0aWFsaXphdGlvbiBhbmQgUENJIGRldmljZSBlbnVtZXJhdGlvbi4NCg0KVGhpcyBpcyBiZWNh
dXNlIG9mIHRoZSBmYWN0IHRoYXQgb24gQXJtIGl0IGlzIG5vdCBhIHRyaXZpYWwgdGFzayB0bw0K
aW5pdGlhbGl6ZSBhIFBDSSBob3N0IGJyaWRnZSBpbiBYZW4sIGUuZy4geW91IG5lZWQgdG8gcHJv
cGVybHkgaW5pdGlhbGl6ZQ0KcG93ZXIgZG9tYWlucywgY2xvY2tzLCBxdWlya3MgZXRjLiBmb3Ig
ZGlmZmVyZW50IFNvQ3MuDQpBbGwgdGhlc2UgbWFrZSB0aGUgdGFzayB0b28gY29tcGxleCBhbmQg
aXQgd2FzIGRlY2lkZWQgdGhhdCBhdCB0aGUNCm1vbWVudCB3ZSBkbyBub3Qgd2FudCB0byBicmlu
ZyBQQ0kgZGV2aWNlIGRyaXZlcnMgaW4gWGVuIGZvciB0aGF0Lg0KSXQgd2FzIGFsc28gZGVjaWRl
ZCB0aGF0IHdlIGV4cGVjdCBEb20wIHRvIHRha2UgY2FyZSBvZiBpbml0aWFsaXphdGlvbg0KYW5k
IGVudW1lcmF0aW9uLg0KU29tZSBkYXksIHdoZW4gZmlybXdhcmUgY2FuIGRvIFBDSSBpbml0aWFs
aXphdGlvbiBmb3IgdXMgYW5kIHRoZW4gd2UNCmNhbiBlYXNpbHkgYWNjZXNzIEVDQU0sIHRoaXMg
d2lsbCBjaGFuZ2UuIFRoZW4gc2V0dXBfaHdkb21fcGNpX2RldmljZXMNCndpbGwgYmUgdXNlZCBv
biBBcm0gYXMgd2VsbC4NCg0KVGh1cywgd2UgbmVlZCB0byB0YWtlIGNhcmUgdGhhdCBYZW4ga25v
d3MgYWJvdXQgdGhlIGRpc2NvdmVyZWQNClBDSSBkZXZpY2VzIHZpYSBhc3NpZ25fZGV2aWNlIGV0
Yy4NCj4NCj4+ICsgICAgaWYgKCBpc19zeXN0ZW1fZG9tYWluKGQpIHx8ICFoYXNfdnBjaShkKSAp
DQo+PiArICAgICAgICByZXR1cm4gMDsNCj4+ICsNCj4+ICsgICAgcmV0dXJuIDA7DQo+PiArfQ0K
Pj4gKw0KPj4gKy8qIE5vdGlmeSB2UENJIHRoYXQgZGV2aWNlIGlzIGRlLWFzc2lnbmVkIGZyb20g
Z3Vlc3QuICovDQo+PiAraW50IHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQs
IGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiArew0KPj4gKyAgICAvKiBJdCBvbmx5IG1h
a2VzIHNlbnNlIHRvIGRlLWFzc2lnbiBmcm9tIGh3ZG9tIG9yIGd1ZXN0IGRvbWFpbi4gKi8NCj4+
ICsgICAgaWYgKCBpc19zeXN0ZW1fZG9tYWluKGQpIHx8ICFoYXNfdnBjaShkKSApDQo+PiArICAg
ICAgICByZXR1cm4gMDsNCj4+ICsNCj4+ICsgICAgcmV0dXJuIDA7DQo+PiArfQ0KPj4gKyNlbmRp
ZiAvKiBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVCAqLw0KPiBBdCB0aGlzIHBvaW50IG9m
IHRoZSBzZXJpZXMgI2lmZGVmIGlzIHRoZSBsZXNzIHByZWZlcmFibGUgdmFyaWFudCBvZg0KPiBh
cnJhbmdpbmcgZm9yIGRlYWQgY29kZSB0byBnZXQgY29tcGlsZWQgb3V0Lg0KV2hhdCBpcyB0aGF0
IG90aGVyIHByZWZlcmFibGUgd2F5IHRoZW4/DQo+ICAgSSBleHBlY3QgbGF0ZXIgcGF0Y2hlcw0K
PiB3aWxsIGNoYW5nZSB0aGF0Pw0KTm8sIGl0IGlzIGdvaW5nIHRvIGJlIHRoaXMgd2F5IGFsbCB0
aGUgdGltZQ0KPg0KPj4gLS0tIGEveGVuL2luY2x1ZGUveGVuL3ZwY2kuaA0KPj4gKysrIGIveGVu
L2luY2x1ZGUveGVuL3ZwY2kuaA0KPj4gQEAgLTI0Miw2ICsyNDIsMjYgQEAgc3RhdGljIGlubGlu
ZSBib29sIHZwY2lfcHJvY2Vzc19wZW5kaW5nKHN0cnVjdCB2Y3B1ICp2KQ0KPj4gICB9DQo+PiAg
ICNlbmRpZg0KPj4gICANCj4+ICsjaWYgZGVmaW5lZChDT05GSUdfSEFTX1ZQQ0kpICYmIGRlZmlu
ZWQoQ09ORklHX0hBU19WUENJX0dVRVNUX1NVUFBPUlQpDQo+PiArLyogTm90aWZ5IHZQQ0kgdGhh
dCBkZXZpY2UgaXMgYXNzaWduZWQvZGUtYXNzaWduZWQgdG8vZnJvbSBndWVzdC4gKi8NCj4+ICtp
bnQgX19tdXN0X2NoZWNrIHZwY2lfYXNzaWduX2RldmljZShzdHJ1Y3QgZG9tYWluICpkLA0KPj4g
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwY2lfZGV2
ICpkZXYpOw0KPj4gK2ludCBfX211c3RfY2hlY2sgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0
IGRvbWFpbiAqZCwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpOw0KPj4gKyNlbHNlDQo+PiArc3RhdGljIGlubGluZSBp
bnQgdnBjaV9hc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsDQo+PiArICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAr
ew0KPj4gKyAgICByZXR1cm4gMDsNCj4+ICt9Ow0KPj4gKw0KPj4gK3N0YXRpYyBpbmxpbmUgaW50
IHZwY2lfZGVhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsDQo+PiArICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBjaV9kZXYgKmRldikNCj4+
ICt7DQo+PiArICAgIHJldHVybiAwOw0KPj4gK307DQo+PiArI2VuZGlmDQo+IFBsZWFzZSBwdXQg
dGhlIF9fbXVzdF9jaGVjayBhbHNvIG9uIHRoZSBzdHVicywgb3IgZWxzZSBwZW9wbGUgb25seQ0K
PiBidWlsZC10ZXN0aW5nIHg4NiBtYXkgbm90IG5vdGljZSB0aGF0IHRoZXkgbWlnaHQgYmUgaW50
cm9kdWNpbmcNCj4gYnVpbGQgZmFpbHVyZXMgb24gQXJtLiBUaGVuIGFnYWluIEknbSBub3Qgc3Vy
ZSB3aGV0aGVyIGluIHRoaXMgY2FzZQ0KPiB0aGUgYXR0cmlidXRlcyBhcmUgbmVjZXNzYXJ5IGlu
IHRoZSBmaXJzdCBwbGFjZS4NCkkgd2lsbCByZW1vdmUgX19tdXN0X2NoZWNrDQo+DQo+IEphbg0K
Pg0K


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:51:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:51:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199622.353793 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrnE-0001pk-B0; Thu, 30 Sep 2021 08:51:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199622.353793; Thu, 30 Sep 2021 08:51:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrnE-0001pd-7V; Thu, 30 Sep 2021 08:51:52 +0000
Received: by outflank-mailman (input) for mailman id 199622;
 Thu, 30 Sep 2021 08:51:51 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVrnD-0001pX-Bp
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:51:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id a587f124-21cb-11ec-bd3d-12813bfff9fa;
 Thu, 30 Sep 2021 08:51:48 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-Nv0kNwjDPjCv_DZIVZ0xTw-1; Thu, 30 Sep 2021 10:51:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6381.eurprd04.prod.outlook.com (2603:10a6:803:119::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 08:51:44 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 08:51:44 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P250CA0007.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:330::12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 08:51:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a587f124-21cb-11ec-bd3d-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632991907;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=anyY2T1rcTg5NsX7B+m+J3daanHfu0TPvhigms+KrTQ=;
	b=A1UZIOLOAb9wpCGGCPnVtC68aSVw+cZcKfdeqC+kyr6ACEbeuiL+b0Zc1/XzyCMPfvtwvS
	IZxkbTVvYQGKTNszpaG2+mZo+susKRDYZXWR6aPpDAAwD+P65JJwNn2J6TWuUlOlltkf07
	VtuMtHEh4W2ldVjNXm8Z1C0VX6rLKRs=
X-MC-Unique: Nv0kNwjDPjCv_DZIVZ0xTw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=U/BTrIay8XC1fXcRA+mFyrbuXu2wgGiDGkle4KOU9ZVQCE0xeJ2Fp+c2geCKsuNW4kny+XKa3flHOo+4A/ZmwA5v2NeSY1V9tyl+p0ZjRHdqwnfLvGdSy2vf25FlpCH79uq/77/i6OR1s2dShaJpd20GNaC5Vtyu5ImgUdJAATh+n7p8YSJdgNnRuxuRTOA1oLHTxSIS7/zfGa26l2REW6qewp3aU01wRdEwdU18utFm6FcioAGhL89a2LdmnkojVuWTrm4RB2J1hXQj623JmAxHWxkQwDhAE54A+p1KgimU6TPVj7Rzi7QShTzmXVQfzsLUqVybjJqk12NfkgDLzA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=anyY2T1rcTg5NsX7B+m+J3daanHfu0TPvhigms+KrTQ=;
 b=VQkR5GgWRhGaWjgrCJEdNteY6eAPaeLwF5el4/zlIrcjw/KEuUtKVgo/c6VI/Sy84hG18+d2ZswuVNLi7AUjSKuC3bNMOk/7PclGtxu7Im9nLU9NeO7Tg6Yp2lrU/LbGRzWmBhPjygf851PzjusRmBqqgA4pX588GAG4XdJfTOArsiBTk/o9rjQjmvK7dczQZJkY8jzxC6WYsleegHEv5SOS+WzbGcQ45zQ6TcKBKgZ+4fDX0PpwT6EKKSlAKptT1Ss+XFplwHtElWG+Mi1O7AsvlkN79wtEL2x/3fXzQApg2D700zMj7eyw6r/jcsvJe4dFJYCQ62FmC0XlLPCUSQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-11-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <ef2c008f-7682-1549-8d27-712b04fbd84f@suse.com>
Date: Thu, 30 Sep 2021 10:51:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930075223.860329-11-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8P250CA0007.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:20b:330::12) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 9a6e0f46-c023-4d8a-45c9-08d983ef8750
X-MS-TrafficTypeDiagnostic: VE1PR04MB6381:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6381C5EB325F0226EF637365B3AA9@VE1PR04MB6381.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Wku6kni5IrHHd+LGd/lOc6CegPEL6yeb/bRc3KwSQ9xSe/mESmPOwSdyA9ZorYieQqcbA7ru8dFHQzwfRrkSaWa41D4KRiy6xFz1jkcU0zlsdrTNKmQmwKMZsgAQs1pAkAm5C/P+o/TvnAxgsQ+rnBapDMnuP4QFEbXqhC2zsKBf55GDgps0xAh2DlWBS278S5I9qWakhRLUDEN7rnyq80XkXQFp29rPxKLazZNYdtMMNqUl96hZx4myVIsA/D1Ipn/vGxEhKdCkjlPjXkLfZ50WT/cwV8mrKjZmPCdII07boaVcgjMnteORb72lVuXgcYfdpcfseETIxSH2m/U0BtEcWdLaiqs+RgPfZU9SVdCDD13fhz/G0WzLdEh2JSiavdu2Xa494Fp9PpbslJ0BfcWYPiVsrK6oXyZ1fG1VcgyOLamuA1EanFdSL0ji+Y7fQJFiqvATWWEcQrj1dOeh3+nlG9ngimEWpcNOCDYHXzGlOKHSCbAH5vT0t3iT3k/dT9rq/TFyMEv4fe+X6b9fv7X7/n36jgvii1iF2Mlh5IVhG4tMt7YCU0f0SPMaagI0p9MWvBAFrCcc3CSzBcnUuiN+weg5kiZAjJH0oG37O3bDnmTF56xG+tf0Od63nFs1S+ZGKQlFv3T0550Lbm20JkL54lqyqNawNrK0r91/XXMjOpoTYXYzlYvuwbgHRxotTpDX1T13PKeDoTH/IWccStsvHlR0s6dBN2a93+8gnACYLA8MAo1JuUxHTtFcEX9x
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(7416002)(4326008)(956004)(83380400001)(2616005)(2906002)(86362001)(5660300002)(36756003)(186003)(316002)(31686004)(6916009)(6486002)(8936002)(66556008)(66476007)(66946007)(16576012)(31696002)(8676002)(26005)(53546011)(508600001)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?ZExYR0JsTVFjVW9mNVNQZDZGZ0NpMEttTkp0dENGVXFmUVBXTlEwUHZDYVFF?=
 =?utf-8?B?dVFjUUZjc2h0Sy9FaE1PTERNbldDS2JIZDBHaVYvYS8rbjlEUGt1V2NPSWpy?=
 =?utf-8?B?L0FTZkFhMUIzdlJuaGxZSGwyaFcreCtIaVhqbjVhOWtPVnM5UTZwQ1hSdG1E?=
 =?utf-8?B?TXo4NHArZjcvQzJOZlNpcTFySk5OR0x5WUtKUC9EL3RIaXRtWmlEYzZ4YVc2?=
 =?utf-8?B?SVN6ZzNURHYrdmpjVW1tR0NwbDhqcUkrMWh5NGVxbUlZSzk4eXJoWVZPYzhR?=
 =?utf-8?B?a0wvT2hsUHVIUTRnaXEweGJHb3RBbFR6VTl0NkU0aUVDMEtPZHUrZHU3NmJx?=
 =?utf-8?B?eWZmVUduNHFOMGxCUzhVUDc3aVJlWDkzdXFPV2RCVnpqTEQ0Y2pOdEdUTkJI?=
 =?utf-8?B?emRnZlBRQjBZUkY2d0pJNytHQUdrV2FMMjhFeEVYWGt4UnJsdW12VHdHU1NH?=
 =?utf-8?B?L1p2cDdBVUU3Z3BSTGdYRkV3bzVNb1NsNjg1N0lxbEpZSWRlWm9QRVZJZGFZ?=
 =?utf-8?B?ckFBU2szS3FnMFVmeTZOV2dYSFlDek0rbFVpMytPcDY0YWQ5TlR3Z3IxWVpO?=
 =?utf-8?B?ditZUTFZakN6WDNWVWNBZkhzMzBoa3RqOTZ4M0o1azE2ei82ZHlwMkxmWDh2?=
 =?utf-8?B?ZEtMSkpiSllBbm5aeGIvelkwRUNVWHg2REFMVGlvYlIwMlViRWczR3l5STVG?=
 =?utf-8?B?Y3Vrekw0UDZ2Zzg3Q01rUXpUaGdTUzBYOEw1TWlZSVFvR2ZPSGhpV090MGVy?=
 =?utf-8?B?YjlhZDlsRUo5VCtjUkkyOSsyS1huYjJlVURtUGdtYjk4MmloMWdIbW5BTU5a?=
 =?utf-8?B?U0RualUvblUzNmx1bVRXdEp5b0NON2FaWTI5ZE5tbnRybjB3MGk2L2x2N3VK?=
 =?utf-8?B?QlQyaVBYdzdIbG9yOXB5bmJ4MktEbS8zUzRGL1h5eE5sVnhLcVlZTWZpb3RB?=
 =?utf-8?B?UlNySEFXTHhVeHp5V2ZIclNaYkpzVUo4akwzM1owQXlHUzFMa1lGOEI5a3RC?=
 =?utf-8?B?cHRhQ2NzQ1pWYVY5TGY0eUtlaXg4SUI0eWVjVmx0VlpSL3VTcE9mV3U0RTQ4?=
 =?utf-8?B?UEtDSm1DVUJOWEl0SDNELzRKN1c2NktwbElTYW1KTm1WOFhxZXlUMlMvMmdU?=
 =?utf-8?B?eWNLaEw2dEE0ZFhka3RxMXYzTVZ2RFNJbWFUcVNTaW56cDRvY3NOVy9nVjBr?=
 =?utf-8?B?NjUvUklqbzlNay9CS1hJUzNqaFJ2elN2TCt5SC9xa2RKbjM4Z2llSTJ1MXF0?=
 =?utf-8?B?cWhQWTQ4WTFDekp0VVVkMW83NEx2YmpzWDRKUlhsSEtHM2ZxNXluOEIvWjNZ?=
 =?utf-8?B?bTIrVjhYVnJlOWExNEZNVDlEeWRFTXFMQURUUlVFTUJPaWUreDRFZ2FWd0dw?=
 =?utf-8?B?d0FGdytzMzVobGYyVEI2TlBSaTlwWnM2VGgwMFBjTTM1WFBqTzlib3htbXFr?=
 =?utf-8?B?YXBxNUt0WUJKUWYwSFBzOHVsMmcwODFGWURmT0hCRnZKN0YxaWlUV2FwSFdC?=
 =?utf-8?B?N0o5VW9FRjdNVFl0SU9Ka0F6Q3dycW5QdzRyamFxeDN3WWpPT2h2Nmczc0NO?=
 =?utf-8?B?OFpCVnpScjE1SGxneHZqVE04RlNqMnNDZWxobHZQdCtUM1k1SThKaE81WEdG?=
 =?utf-8?B?V3dRSkhIZFZxSVNyTmR4L3dRN2FQV3MzeVdhZkpCNWt4WkdRRGdEMXhrOWpR?=
 =?utf-8?B?d1BSSmFwci9GVGdUQU85bkRyWUQ3dVFkWGtIVkRzNTAwTGN5Y0NOaWRuVm5K?=
 =?utf-8?Q?ETuJMNylBaowW/2LB5r/uGjZwyPJcKljGtrP6aD?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a6e0f46-c023-4d8a-45c9-08d983ef8750
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 08:51:44.1262
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NcHPz3gaShV22XGrG849wfVkiRQNS6cAeTD1zNddf0QP5ODEDl0Eb7DXPG8G8tmTc1t562GmD/FKEOPPKZdtZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6381

On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Assign SBDF to the PCI devices being passed through with bus 0.

This reads a little odd: If bus is already known (and I think you imply
segment to also be known), it's only DF which get assigned.

> The resulting topology is where PCIe devices reside on the bus 0 of the
> root complex itself (embedded endpoints).
> This implementation is limited to 32 devices which are allowed on
> a single PCI bus.

Or up to 256 when there are multi-function ones. Imo you at least want
to spell out how that case is intended to be handled (even if maybe
the code doesn't cover that case yet, in which case a respective code
comment would also want leaving).

> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -831,6 +831,66 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>      return ret;
>  }
>  
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT

May I ask why the code enclosed by this conditional has been put here
rather than under drivers/vpci/?

> +static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
> +                                                const struct pci_dev *pdev)
> +{
> +    struct vpci_dev *vdev;
> +
> +    list_for_each_entry ( vdev, &d->vdev_list, list )
> +        if ( vdev->pdev == pdev )
> +            return vdev;
> +    return NULL;
> +}

No locking here or ...

> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
> +{
> +    struct vpci_dev *vdev;
> +
> +    ASSERT(!pci_find_virtual_device(d, pdev));

... in this first caller that I've managed to spot? See also below as
to up the call tree the pcidevs-lock being held (which at the very
least you would then want to ASSERT() for here).

> +    /* Each PCI bus supports 32 devices/slots at max. */
> +    if ( d->vpci_dev_next > 31 )
> +        return -ENOSPC;

Please avoid open-coding literals when they can be suitably expressed.

> +    vdev = xzalloc(struct vpci_dev);
> +    if ( !vdev )
> +        return -ENOMEM;
> +
> +    /* We emulate a single host bridge for the guest, so segment is always 0. */
> +    vdev->seg = 0;
> +
> +    /*
> +     * The bus number is set to 0, so virtual devices are seen
> +     * as embedded endpoints behind the root complex.
> +     */
> +    vdev->bus = 0;

Strictly speaking both of these assignments are redundant with you
using xzalloc(). I'd prefer if there was just a comment, as the compiler
has no way recognizing this in order to eliminate these stores.

> +    vdev->devfn = PCI_DEVFN(d->vpci_dev_next++, 0);
> +
> +    vdev->pdev = pdev;
> +    vdev->domain = d;
> +
> +    pcidevs_lock();
> +    list_add_tail(&vdev->list, &d->vdev_list);
> +    pcidevs_unlock();

I don't support a global lock getting (ab)used for per-domain list
management.

Apart from that I don't understand why you acquire the lock here. Does
that mean the functions further were truly left without any locking,
by you not having noticed that this lock is already being held by the
sole caller?

> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -91,20 +91,32 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>  /* Notify vPCI that device is assigned to guest. */
>  int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
>  {
> +    int rc;
> +
>      /* It only makes sense to assign for hwdom or guest domain. */
>      if ( is_system_domain(d) || !has_vpci(d) )
>          return 0;
>  
> -    return vpci_bar_add_handlers(d, dev);
> +    rc = vpci_bar_add_handlers(d, dev);
> +    if ( rc )
> +        return rc;
> +
> +    return pci_add_virtual_device(d, dev);
>  }

I've peeked at the earlier patch, and both there and here I'm struggling to
see how undoing partially completed steps or fully completed earlier steps
is intended to work. I'm not convinced it is legitimate to leave handlers
in place until the tool stack manages to roll back the failed device
assignment.

>  /* Notify vPCI that device is de-assigned from guest. */
>  int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
>  {
> +    int rc;
> +
>      /* It only makes sense to de-assign from hwdom or guest domain. */
>      if ( is_system_domain(d) || !has_vpci(d) )
>          return 0;
>  
> +    rc = pci_remove_virtual_device(d, dev);
> +    if ( rc )
> +        return rc;
> +
>      return vpci_bar_remove_handlers(d, dev);
>  }

So what's the ultimate effect of a partially de-assigned device, where
one of the later steps failed? In a number of places we do best-effort
full cleanup, by recording errors but nevertheless continuing with
subsequent cleanup steps. I wonder whether that's a model to use here
as well.

> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -137,6 +137,24 @@ struct pci_dev {
>      struct vpci *vpci;
>  };
>  
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +struct vpci_dev {
> +    struct list_head list;
> +    /* Physical PCI device this virtual device is connected to. */
> +    const struct pci_dev *pdev;
> +    /* Virtual SBDF of the device. */
> +    union {
> +        struct {
> +            uint8_t devfn;
> +            uint8_t bus;
> +            uint16_t seg;
> +        };
> +        pci_sbdf_t sbdf;

Could you explain to me why pci_sbdf_t (a typedef of a union) isn't
providing all you need? By putting it in a union with a custom
struct you set yourself up for things going out of sync if anyone
chose to alter pci_sbdf_t's layout.

> @@ -167,6 +185,10 @@ const unsigned long *pci_get_ro_map(u16 seg);
>  int pci_add_device(u16 seg, u8 bus, u8 devfn,
>                     const struct pci_dev_info *, nodeid_t node);
>  int pci_remove_device(u16 seg, u8 bus, u8 devfn);
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev);
> +int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev);
> +#endif

Like for their definitions I question the placement of these
declarations.

> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -444,6 +444,14 @@ struct domain
>  
>  #ifdef CONFIG_HAS_PCI
>      struct list_head pdev_list;
> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
> +    struct list_head vdev_list;
> +    /*
> +     * Current device number used by the virtual PCI bus topology
> +     * to assign a unique SBDF to a passed through virtual PCI device.
> +     */
> +    int vpci_dev_next;

In how far can the number stored here be negative? If it can't be,
please use unsigned int.

As to the comment - "current" is ambiguous: Is it the number that
was used last, or the next one to be used?

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 08:53:46 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 08:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199628.353804 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrp2-0002S1-Ms; Thu, 30 Sep 2021 08:53:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199628.353804; Thu, 30 Sep 2021 08:53:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVrp2-0002Ru-Jf; Thu, 30 Sep 2021 08:53:44 +0000
Received: by outflank-mailman (input) for mailman id 199628;
 Thu, 30 Sep 2021 08:53:43 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVrp1-0002Rm-Fv
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 08:53:43 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id f2432c45-d303-48b4-865b-76e5d0e5ae86;
 Thu, 30 Sep 2021 08:53:42 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2056.outbound.protection.outlook.com [104.47.5.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-TdokWeptML2htBxx0mBZKg-1; Thu, 30 Sep 2021 10:53:40 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6381.eurprd04.prod.outlook.com (2603:10a6:803:119::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 08:53:38 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 08:53:38 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR0202CA0053.eurprd02.prod.outlook.com (2603:10a6:20b:3a::30) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 08:53:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2432c45-d303-48b4-865b-76e5d0e5ae86
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632992021;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=gyNj+9vlrP3omPgt6HOQRriQEpOzh8GdSwwftm6nJhU=;
	b=mooQVmHKSfu1KvUV6uy5kv05Wp0LpoiY3TEbNUdGRKUyG8gBdycTVKfvE4JvljbbV7YjgJ
	oGywom2xW9reDiZbKjCZ/AF0gNPUhUc1BzmzSrt1k/b08KHlBu7iQ1m9YzqqDxtCAW41rf
	v0EcUsHUE41zOKQrXMn3twutgfiky00=
X-MC-Unique: TdokWeptML2htBxx0mBZKg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fNlT0UQvroZjdVOpoJt19sNdL00DA+np9Xq3EKTa3wF4YoJWNZOSFQ+1wtNSfJyau/ncpHwfazJUhaU/GvhklxmzQwoIhxZQFmToGTQvJtPIIGKekFqT1zWKLDDfXO75zSfX0eNZlOQMbT2Zzj7CtrYH3KUVnktNOG9CGPmYAEKeTvDsuORFf/TYBgQn83o5pSoXwksCRqnkTb5GhEuEi/6yrqEACIUWZ127l/ZuRdA6Ft2LQ9g872p8xpnBonoXlnItDyq4mzv+xCz7S7m63BIV0+sxRdW31qP4RhSL4dELKrdo5Ph8tNDkYWF1xa75l/4SfvhipNxwYU6eO+ff1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gyNj+9vlrP3omPgt6HOQRriQEpOzh8GdSwwftm6nJhU=;
 b=RESUDXTrN/m068UYN7iVuDwSDQYfG62O1iCeU5PwDhH8YbdkBZRDP5rRwE4kmC98EPqCSX8EbgOopCFxk8AyMeQM7L28kI0wJPvq0IpAljPeBsfbExIi+Le2rw+U03YtblqsUeGvVYHa3Ys8YOA5OPzVsbJygW1GjtzQ3G4StlctB+XZ5rUnnaZjTS7/Zyww1EdD1ltOOWz0W1PRkfeh9a1dWjWMK5RFbVQKXelQdZO5GNtt8lhawoHbyQ3u7YuUZcoUBolurwjyYdSOQ/d9rVnriovM/QfSeLw9NPv3rkdBlAivQsMG62yMldVHSHb1AQgv58VkZhmAL22ZUwsGjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <andr2000@gmail.com>
Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, rahul.singh@arm.com,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 xen-devel@lists.xenproject.org
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-12-andr2000@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
Date: Thu, 30 Sep 2021 10:53:35 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930075223.860329-12-andr2000@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR0202CA0053.eurprd02.prod.outlook.com
 (2603:10a6:20b:3a::30) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: b1921f54-d526-47a1-4b7d-08d983efcb29
X-MS-TrafficTypeDiagnostic: VE1PR04MB6381:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6381A419ED6B7483994E5973B3AA9@VE1PR04MB6381.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	tOIn7zBtRwHdAHC2K+stQWueE+WovUYW03effl1eHG3wpZeGLS7rp2wXkPqbFg3qWIAQUTAcvgzu+lhwYJTFnGyvpHx5oU5rNXFnyVsXeKn44RQ1d45g6qRQHO2UwxZuj9F2ttvejS3K9DDcAQuOpCmpVekJ2HYPWNdA1+kwmsIseFKo6eGmKATyoLPT5OjPZvPn0zci6rukzibQlg4mvXUHQIxNBJID4hmmwDUZaH4JJjP3ThicdaMqC6+DCzFPM+lsik9NptoTpQCMwfjMZYsNg6m5ZCRuEQ13OcKKQSVmhe5HF2kp2OpjTyJ/9JlMQBQ1sgtG/8qt7zSom7Sg2Qp5G1joCt8yxJDt+PCJjLsTFLv825+H6N/f1wsy5b1LLUMsQiyFIFZcZc1fYsHbmm1r3avr5DOv/9oNWh6jZ7ecexrfAodW9sP3BTkGwT3aZmwQc8K8dWwPQPMEYutDUzEi66QCNCOd3FGk3gtZskCw2ZoiWSRECLmb5pDcKGi4g6Nk0g63AfPRBCyNWOa1lFvVp35ZBS0Jwc4glBBe8+kxgJitVmsfyaHguMIwqGABQPIR6ITIBr+EwMv3FXMqPW6osE2uy1n2vMKlytGNCWgzsKVKkdwM55rBSYumqo7Mxbi48oMGa7zsjIf6ydxDlzLf91HHSIXf5gBIeXFStxlaaGFMCerTONQC1qZaIfRwfGzNISEodcsiHn/dFbN4tYt6e3a5JOSWxpN0bBPVP3s=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(6486002)(8936002)(31686004)(53546011)(508600001)(26005)(16576012)(31696002)(8676002)(66556008)(66476007)(66946007)(316002)(7416002)(4326008)(956004)(2906002)(83380400001)(4744005)(2616005)(38100700002)(36756003)(186003)(5660300002)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?T0lkeTZNNE1QZ1lWaWdqYmRZM0tVOTdBbFNMK2pYaGFwZVdGMjRPY3lyS25C?=
 =?utf-8?B?SjZtb0d3M2UxWjZoZXhRVG16ZUhrYlpmdVd1elB2QXVNQ0FmWldzcE9DTkVu?=
 =?utf-8?B?MEtJOVFDUHg2dWNjQWpZMGJRaDVoL25QWVNQT05vSXFpUHFhbzJOMDlSR0F5?=
 =?utf-8?B?MlVLRTI3Rm45NHFBNnV4eEFBUERCVEtLUTIwZjZXSm1zTHd1NnZTdVhXUmhk?=
 =?utf-8?B?OHE0d2hoT3lTb3grN053YlJncHBkNytLbU5ZNWZFWmR0NWR5b0ZBenQxMlBJ?=
 =?utf-8?B?QzBkcUFCMmQ4UlQrc2NNd2RPbXZ1RDJmV0VabVRiMGJqNVBmNXB2QUJYcUlq?=
 =?utf-8?B?SlVaSjdRdlgya0VWNk1iV0RtQzBtY0hjZEtYQTJHcllQUGdpT1E0aXkyb2Nm?=
 =?utf-8?B?VnExekYxQ0syLzQ4UVpjR1hCUGN4c3ZZT3pxdUdWQXQ5blVUaURiY25XWDBO?=
 =?utf-8?B?b1ZnaTVHUVZVVnN4UERML3hKdVZXbTVBM1VjaWxTZEtXVWhYcWI4a1lmU2ZD?=
 =?utf-8?B?OGhBMU41OTJ1dXROYytjUkV4NzZ5U25neFRCR1dzMjhWNGdtWUNzR2RFd0hI?=
 =?utf-8?B?RXdDY0F1b0xNUG5TMk9NSTdaYVZISzkwc3N3elBMa1JDdkhOYlovNEJTZDVL?=
 =?utf-8?B?ZWZseGtGOGNNcGVFNlhlU055N2Z2VUJ5dXg2WWRoTE5BVWZnVXlpb2RGaG1O?=
 =?utf-8?B?Y3lwNEFCUlBvMXFNRytNSnhXNkZmTkp6RHVGQ1VYUHZDVWd2WHdsb0g1TU4z?=
 =?utf-8?B?M2crRUNtejQ2bjZ0SXJIZEFKcmlIeUZYU0V1OFJ1WmtZaGxNVzVWQWtSdkxJ?=
 =?utf-8?B?K1RndUhXMjBnSzFPV1JJM0gyVGM0TEVkMmpnZk85bGdhS2s2V1FvWFljWStr?=
 =?utf-8?B?NHRLVUMvWGF3bjZBR0ZSU2RwTmJPUFdaMkhzWWFTajkwQ0xVczJ1VnEzUWhZ?=
 =?utf-8?B?SXVjZDVXOW5XS04xbDBMbjZZNHZ5WFV6Q01LMHhpZW1zZ3dHUDVxWFNRK2Nu?=
 =?utf-8?B?OUsxdXM5ZENxcUg3TnNRbTRjcTZRWkltR0JQYnM4UFZRSE53NnAycy9QLzR3?=
 =?utf-8?B?M2tydGJxZE5lcGkyb0VzNldzVlAwVi9WYXJGNHdKTHh4QjZvYVZFa2s4OS9r?=
 =?utf-8?B?d0ZxdXFQNzV2RFlCdkk0TG5WWXlxSytHQks2ck9DZnFSejNSWE1zaTYxclBx?=
 =?utf-8?B?VGtoYXFnc3NTTXdUOFhwYXJheTdHYlpIVzVCS2c3YmhEcmJwUHBxR0xXZGJH?=
 =?utf-8?B?ZHNiOHRGK05vN2ZiTzZCeE5sMDBpSEtpcndKQ2VoTTNlQzFCa05MT01XZmhw?=
 =?utf-8?B?NlRyWlRkaDZxd3c0dUZZYStQUjdpV3BHR1VQNDlWRFludVRwQVhua1hTRW8r?=
 =?utf-8?B?bHZEaEM2SE13TjlSRjltMXhBTmEyd0w0L05RUGpTQlZFVXpmaEVoR1p2Kzdn?=
 =?utf-8?B?R3J5RTBhRTUveGg3Y0NWRDBZWkJ1U212RVl1clIzNzZTMHJtVzJHQ3JzYVE5?=
 =?utf-8?B?azdWV0NyZWM3NjZKc05PbkJYWExuMUc3aFpFa0w5dnI3aVh6Mm4wVldkcVdB?=
 =?utf-8?B?UWE1eFRKZjhNaXpXRm9qc2ExYnNkSlZ5SUFzS2NDbXRTWU1GdEdpbi8zS2sx?=
 =?utf-8?B?d2J5dUk5SjVDeDZUd21JSDI2dEZNQUFhbjZ5WnVmc3FBWUVLNEdFR2tmYk5E?=
 =?utf-8?B?NlVUT0NVMUp6Y1FrTTlieWVBWXUzMnIrTVZnTFBFc3g5NC9uNnpES0RIUzFC?=
 =?utf-8?Q?EoxhmybKhc9lI1cw8XWrV2bsPfIBgA7OZpyH7w2?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b1921f54-d526-47a1-4b7d-08d983efcb29
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 08:53:37.9708
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: m+eV9h1i6ed9ALzEk8CYLk00uDdZN2MTXrr1OnvMuAF9gUU8cHxO/RFks65TWeg4BzS/wzzYPO0ZtoLanTBGLg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6381

On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -889,6 +889,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>      xfree(vdev);
>      return 0;
>  }
> +
> +/*
> + * Find the physical device which is mapped to the virtual device
> + * and translate virtual SBDF to the physical one.
> + */
> +bool pci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
> +{
> +    struct vpci_dev *vdev;

const (afaict)

> +    bool found = false;
> +
> +    pcidevs_lock();
> +    list_for_each_entry ( vdev, &d->vdev_list, list )
> +    {
> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
> +        {
> +            /* Replace virtual SBDF with the physical one. */
> +            *sbdf = vdev->pdev->sbdf;
> +            found = true;
> +            break;
> +        }
> +    }
> +    pcidevs_unlock();

As per the comments on the earlier patch, locking as well as placement
may need reconsidering.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:06:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:06:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199635.353815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVs1H-00045z-Ne; Thu, 30 Sep 2021 09:06:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199635.353815; Thu, 30 Sep 2021 09:06:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVs1H-00045s-Ka; Thu, 30 Sep 2021 09:06:23 +0000
Received: by outflank-mailman (input) for mailman id 199635;
 Thu, 30 Sep 2021 09:06:22 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVs1G-00045m-1Q
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:06:22 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5f7e1691-c7ee-4932-bd59-a57bb9a194ed;
 Thu, 30 Sep 2021 09:06:20 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-26-0ydWM4VCPg2trlJHBS_pEA-1; Thu, 30 Sep 2021 11:06:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB7375.eurprd04.prod.outlook.com (2603:10a6:800:1a8::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 30 Sep
 2021 09:06:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 09:06:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR1001CA0057.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::34) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 09:06:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f7e1691-c7ee-4932-bd59-a57bb9a194ed
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632992779;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sYGcCumBlh/iqHb8Hi/PrcnD7eDeCZFRwKWtGjgsJFU=;
	b=RgXj0qQV78q+TnTH4tJ4bi9eVL0EQFoXeZjXGETyoNTmLpqy16JDd6GKVIBByPrcAg4nu1
	spK4ZIjxvqh7P/gj2HLMfj16icaDVhyFPYcTr9N3CDRi7lhJsb7OuiZjTXpPPDbSNt6Iuo
	pRNahSU57zMR+C95qC7DyVp8aLNyyBs=
X-MC-Unique: 0ydWM4VCPg2trlJHBS_pEA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=B4xDdPGUpzoJXsS222n/a1+/7rYzpbNNn53djj1wWXGsWDaadFybAR8ZDMMBi/4JLqH6EhqCFnjpgsHBomIW6VDSnI3PWMKjQRdv4Pv4Xo7NyCuDQiDmNfOpwb01D5y7aBvS890oKbDHVcjLP0QPEQxU/31b/+cNqrQZsT+hloM0GuuhwyIdV+vt62UDXVwo7paQsSZ9WZsyh3lCA1K/YcHOy0eBRhu1TDF33IfTr3A2xEK0wx7yujBdnFXWFBY6lQ3Ue136IlC0vZ4/MQYxcqJnaA4wQMeNrcgvXvOFsqIzbpQAv9eEnkVf0lhl5uGcLVRG6PAI+kV155cas0jujw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=sYGcCumBlh/iqHb8Hi/PrcnD7eDeCZFRwKWtGjgsJFU=;
 b=A6x3Y/J0S/9c/nNx8e1paxbJ/HbGnpuXVvv7op0USXXslcHyA5EVyK4jbbR5r/q7mGIFVwTE1dD0FpNYy8sz+Ho0mmU1+cdypg31AP15NwmygrqwSuuYyOAjnKFstfQjWAenNZFpCNd+QmvnGm3AYsDvtMLKZTAeKqCOffoXXfzCEDyzxShGOINVTFFluX4egDGMdacI3RV2Wmn70/R0PcKmzKCie5tJwkXtBVbKHrVf4mVTA3xrKRpFyMRwSnz3/jcul3t8mZJPnXsa2qIiEfcNZJKYdkqRxFuKyHc8CMKnXZMaJBc0gwcMwwcAGsB16xixOQ3Buz9IlJqNS9+Ayg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
 <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
 <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <7d035651-6e16-a6cb-6760-e5d077e1a85c@suse.com>
Date: Thu, 30 Sep 2021 11:06:14 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR1001CA0057.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:206:15::34) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d5808d2d-3f9d-47a2-0ffe-08d983f18f82
X-MS-TrafficTypeDiagnostic: VE1PR04MB7375:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB7375DCC50F2B39D9CC3C4857B3AA9@VE1PR04MB7375.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	U/3GyfP0tlTkiELl4jqVZ6LdYgI8JrFfTpzbkMnGXjSHqZkPD5s9JLDNQJILzaGw8D3kdaHifqyPOGJfcHqDfMihyBBoJPjnHT8yhZUvDSOvSqpFATorqNBlc7Sg3qBKAY5aHZpyEo7SX+RUTZHhXaA5KfcWOVieWAAcPFXwD23/s4Xodb3aG3EH7Yje5JO20TDDCirz6A7kYMmzhYuIrP4jzblh/ZVSzBG9L3khuR0NwwT8gw/blOfx2p8DGmyTTGHMHfGabcU/1vPiG2sCUusKu+k1zMZsUJgicVPylJiR8eU/SD7EDSvYvXBiFNYhTRDqObfRG35fbI2TCidnFnTL7sf/HDk7HqAdYFBEmj6tLuDPNoDZD+jzXKApMVcmelIknj8SnEup7kEm1xLyLi8kDsnus38jeICEULh9MoEmWIScmj9UCg8clbdRdgVMavFFEDtkTEc/JTmLAmwVN2aIV+5bfwX1tdL0gQZGHmWQ8BEl1wTC4g7e+iFWIkEbugiA34BeOHxrJso1JuBJCZmpSRX2QXbxM0xIeHIDZMbxccKAPspw6ASnBMBRQB3+Jkz38VgZI9fCoqfGNbrfm+wZUocdnDiK8iUxPEhaIQr9MM8qTyv/kckEFzTFBZRjK8ySE9IsL/WUxQxHtWyaYXPe11syQII09tYOsxH1Ee2tRz+CJV0yseq7z4hwB+7po0ORyqwE4g6q52cCUEOYa+ey7I0dPIWYd2rtN2j4BDmEqoEyu+ZjhJnaL/QqRM9G
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(186003)(26005)(53546011)(2906002)(66946007)(38100700002)(4326008)(6916009)(31696002)(8676002)(316002)(7416002)(6486002)(16576012)(31686004)(956004)(2616005)(54906003)(86362001)(66476007)(8936002)(5660300002)(36756003)(83380400001)(66556008)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	yn7ZjB7zn5inFgtVgKTciuvs/fhcLvjNZ2F+w/r215apqLrNrGVPLppXjhEhFusSh3UhLaKI/A9oH2ZLJjCgUoH+46W5lhmtxtFNeOGNPUj8vaVDpU69SctN8svUsfWiU+aoKb1/70RtSyFnsPX1PrXo5QZpiPbKl1xWUfWtWYE7CMRCRCRnOToh+1Uq8bhA/nxQMDDIOWkS0A6KVDLeW1ljuvS6h0IgGwGX9EigBNcbqxHYQpWqAsX9WHUCyoRpa4HFtXKGt5P5vTHyihlX7LzdM+xGMOA0gSEwP8FXdZyh4C/Sq08pD/cIwBjTqSBpba+tz+kM2dZ/mQrIL04C01fGwkYVlxfA29Vc156mgjX5YPSaJmju3meCEeYQCBjTtemPvlsj8/BOGZ+jfjoMVXEmzS9zwmdE1H6HvR/ixVofKRCXCNDo7DsisldnSEIIqDsc0CtbYfTNIhEy3LAJ2Yh0EBXYj8qTBMOgXqnc9+zZGcRgW/kdCPv8WRGueF0NtKpwlvnJkJR9njmfg4MqQT3NgjJk0mPqkBW9btJLKmrGNKXFk0vrCCAtI9cr62WwbX4VH2jhutWl1/CXQdQSgCzaRWrhcY47iq4mvUEpWFlI0IN7YqCKOLNeUGbwwWvYaDgoqVgZYVHmxIYDByeAt2EEb3XpJy2tApqn17gg5bpfBMiw6TQFClKh+C44OB0bv2vi7BToKyTRa8iWZ/+1Kmxv5IydHr/N1DqOcIje2IjMZ2rYeZsJKobOfXnvOKo4
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d5808d2d-3f9d-47a2-0ffe-08d983f18f82
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 09:06:16.8807
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jWxhOzLiRKdqRcPauWgkvHqcJ9qJMkxw4ywYQIXJ/hg+sxt8s7+CZeZf/qUqFIeGt6zek7F8K9iMHI+Ic9UalQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7375

On 30.09.2021 10:45, Oleksandr Andrushchenko wrote:
> On 30.09.21 11:21, Jan Beulich wrote:
>> On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
>>> @@ -1429,6 +1433,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>>>           rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
>>>       }
>>>   
>>> +    if ( rc )
>>> +        goto done;
>>  From all I can tell this is dead code.
> Before the change rc was set in the loop. And then we fall through
> to the "done" label. I do agree that the way this code is done the
> value of that rc will only reflect the last assignment done in the loop,
> but with my change I didn't want to change the existing behavior,
> thus "if ( rc"

rc is always 0 upon loop exit, afaict:

    for ( ; pdev->phantom_stride; rc = 0 )

Granted this is unusual and hence possibly unexpected.

>>> --- a/xen/drivers/vpci/vpci.c
>>> +++ b/xen/drivers/vpci/vpci.c
>>> @@ -86,6 +86,29 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>>>   
>>>       return rc;
>>>   }
>>> +
>>> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>>> +/* Notify vPCI that device is assigned to guest. */
>>> +int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
>>> +{
>>> +    /* It only makes sense to assign for hwdom or guest domain. */
>> Could you clarify for me in how far this code path is indeed intended
>> to be taken by hwdom? Because if it is, I'd like to further understand
>> the interaction with setup_hwdom_pci_devices().
> setup_hwdom_pci_devices is not used on Arm as we do rely on
> Dom0 to perform PCI host initialization and PCI device enumeration.
> 
> This is because of the fact that on Arm it is not a trivial task to
> initialize a PCI host bridge in Xen, e.g. you need to properly initialize
> power domains, clocks, quirks etc. for different SoCs.
> All these make the task too complex and it was decided that at the
> moment we do not want to bring PCI device drivers in Xen for that.
> It was also decided that we expect Dom0 to take care of initialization
> and enumeration.
> Some day, when firmware can do PCI initialization for us and then we
> can easily access ECAM, this will change. Then setup_hwdom_pci_devices
> will be used on Arm as well.
> 
> Thus, we need to take care that Xen knows about the discovered
> PCI devices via assign_device etc.

Fair enough, but since I've not spotted a patch expressing this (by
adding suitable conditionals), may I ask that you do so in yet another
patch (unless I've overlooked where this gets done)?

>>> +    if ( is_system_domain(d) || !has_vpci(d) )
>>> +        return 0;
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +/* Notify vPCI that device is de-assigned from guest. */
>>> +int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
>>> +{
>>> +    /* It only makes sense to de-assign from hwdom or guest domain. */
>>> +    if ( is_system_domain(d) || !has_vpci(d) )
>>> +        return 0;
>>> +
>>> +    return 0;
>>> +}
>>> +#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
>> At this point of the series #ifdef is the less preferable variant of
>> arranging for dead code to get compiled out.
> What is that other preferable way then?

"if ( !IS_ENABLED() )" as I did already point out to you yesterday in
reply to v2 of patch 10 of this very series.

>>   I expect later patches
>> will change that?
> No, it is going to be this way all the time

The question wasn't whether you switch away from the #ifdef-s, but
whether later patches leave that as the only choice (avoiding build
breakage).

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:19:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:19:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199643.353826 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsE0-0005wU-0d; Thu, 30 Sep 2021 09:19:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199643.353826; Thu, 30 Sep 2021 09:19:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsDz-0005wN-Si; Thu, 30 Sep 2021 09:19:31 +0000
Received: by outflank-mailman (input) for mailman id 199643;
 Thu, 30 Sep 2021 09:19:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsDy-0005wD-Ca; Thu, 30 Sep 2021 09:19:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsDy-0004Z0-4X; Thu, 30 Sep 2021 09:19:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsDx-0002Rx-PK; Thu, 30 Sep 2021 09:19:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsDx-0005Vl-Ol; Thu, 30 Sep 2021 09:19:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HqDMN1Ti8f3SHL+8d5d/q4Fcnt9ZtcszofCuNQKA84o=; b=Bz8ZUECSwsL5ch/wykCS2y/KjD
	8FFGZYM8A/zuqe4dgYvNEgVUSaikNrLSDxL9GqMMbSBbPXIT1bI9MD3wSBhYj677waPgxp/6LET9a
	uH0qZb/kAuSt6v+44aKF095/mWXy09Anl8oGyd8w68IRq6ZnRtzHzKvaIHeQ487JhWmw=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165319-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165319: regressions - FAIL
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    qemu-mainline:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    qemu-mainline:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=6b54a31bf7b403672a798b6443b1930ae6c74dea
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 09:19:29 +0000

flight 165319 qemu-mainline real [real]
flight 165328 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165319/
http://logs.test-lab.xenproject.org/osstest/logs/165328/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-raw 13 guest-start      fail in 165328 pass in 165319
 test-armhf-armhf-xl-multivcpu  8 xen-boot           fail pass in 165328-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 165328 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 165328 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 164950
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 164950
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 164950
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 164950
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 164950
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 164950
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 164950
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 qemuu                6b54a31bf7b403672a798b6443b1930ae6c74dea
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   18 days
Failing since        164967  2021-09-13 13:06:52 Z   16 days   31 attempts
Testing same since   165319  2021-09-29 20:37:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cleber Rosa <crosa@redhat.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dongli Zhang <dongli.zhang@oracle.com>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  G S Niteesh Babu <niteesh.gs@gmail.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Gupta <pankaj.gupta.linux@gmail.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Wainer dos Santos Moschetta <wainersm@redhat.com>
  Willian Rampazzo <willianr@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 6687 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:21:16 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:21:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199650.353840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsFg-0007Gr-Cl; Thu, 30 Sep 2021 09:21:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199650.353840; Thu, 30 Sep 2021 09:21:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsFg-0007Gk-9V; Thu, 30 Sep 2021 09:21:16 +0000
Received: by outflank-mailman (input) for mailman id 199650;
 Thu, 30 Sep 2021 09:21:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVsFf-0007Ga-M6
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:21:15 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c187d994-21cf-11ec-bd3f-12813bfff9fa;
 Thu, 30 Sep 2021 09:21:13 +0000 (UTC)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U9IuOY004245; 
 Thu, 30 Sep 2021 09:21:11 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bdaga809c-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 09:21:11 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB6243.eurprd03.prod.outlook.com (2603:10a6:20b:144::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 09:21:08 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 09:21:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c187d994-21cf-11ec-bd3f-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JWk5Gu1nOmUhzXAZy0M1eXhlHsyZ2RaRm7nlgtGhJgkqdd94fX9bc/1nTaV0LS4amUq3G6H2aSpX0PEhagmAjwoF2cBOkYSEpwS9O7MT3FnqaMz4ZYuWkRvf3+nMaj0o6dxpzH8j3ATn6IuPFDGwubCbOlyqDTDclh1rSjBaejrieVbP2Vi4YukOsZ3UmS9jRqSDn8X1/oifcgjcYPtlA9rnTZoE5qNU62OVxSiWRe0NmgptQLH0QxiJn8/2WKsoVjvQUit2cXnr6Su6+Ca7zSDGyyX1Hu21pij1q8QOaHbrnZI077QJ/MMQRA4b/YHBwFqM1vApfEdQI1xnqFGIhw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3mdNhlz9ee2At3vWD9/9dOjLTnENwnSEQLVZCu6rKRE=;
 b=isuotNQ9Xmwxra4CImz1M4KtGOUEHTw1B8hC4xPk0TXzvn4BUkPeQY/3kJAA6o38GuIfb1ETGVsNfem8XUy0YgaoAzxZJwPyOek1SpoM4kksdpPblYSfPsJFKm8iHw9HYovji/6lybii8w+92HlIjPuAOp8fousMfF4bpg2CucpS1LMVldebvuFdBLzh5xl3gygIQoDnVB0zgX+ix16J8AjZXKpU5+82k3LS1DQocX7HRbo61U5PAehK1kkCya4mQnLLVlTlh7+PoganlDHQvR200rMSA/rkPxHrERtfAKXofbO7ylB4pCtlV+ejqfNERRGCGP22a35WMoQWHWKSTg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3mdNhlz9ee2At3vWD9/9dOjLTnENwnSEQLVZCu6rKRE=;
 b=PNR7C6u/9lJYJJpT6zRkyXvkMRs8R4An3kB9kvM7Ciqw5/hglnyr7h1dI7HG1Hf8mpo5S21dCw5jEKsw49DU8XHc2olMtVxpAAG58NEEkZ0EmNvowEh4pntPsKSq/6RDQslzDiGfI35lb6OOtqed2nIH6KBl4Wv2CQ2IxrCLrQKhUxYBJsM7IWU5/x6Vh2pNWJUj9Fwani3W/E3JE7AEGhRT+WpZsL2/tuHdeXitJV4U7yRv3gC91L07MRCeAtEy7XB5jZx2G4y7fMP1iKyDA4MoAifOEOo8meDgfXPurhTryCsdE3xLwZ1ZFNYi+LrIvIEpj6MipgumNl/rnbMeHQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Topic: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Index: AQHXtdAg2qzxZ1hIP02wGTFUbQtoPau8PLsAgAAGuICAAAXDAIAABCmA
Date: Thu, 30 Sep 2021 09:21:08 +0000
Message-ID: <91fca69f-1a6f-198d-be27-c6b0ddd62206@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
 <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
 <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
 <7d035651-6e16-a6cb-6760-e5d077e1a85c@suse.com>
In-Reply-To: <7d035651-6e16-a6cb-6760-e5d077e1a85c@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7f04f02f-371c-4a26-6971-08d983f3a300
x-ms-traffictypediagnostic: AM0PR03MB6243:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB624340A776EB110058C8EC0AE7AA9@AM0PR03MB6243.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 z68vKnO0xOjx3CMYfFolhVw9qOovlNLV7pbyWgz5WY25uBcTgoAXIhvVhcpTY6L59uWREVMyGJiG2Znr1sf+Hgu3aoA7UuLZX/B/OvohsuESFjSwjfuTYuMhTc7HRuNK4kMl4VSk0d7vqqWj6gF8ntYzPi9IOAsYnkfC74kRE5h+LcqORwcATkQRkCqbtPDKy1x0Tbe1X3qJQVDeTxZ5ePamhwo1QCkKooRrZ5OQcuh6tkT/f9765aeKRNzRmxnzZ2s0lziwAEzqguoLfGKsmNMsbknX4ZtX+zpDQvLiSb5S6+ASd6CS2SR9Gy51UdsR0UXDCTPNgB0lnfcFBK4jsM/znsQXoweBP6B3v4XPB1ctSB3qyhYFih9V1fZkmPCNCe/o1aa9lp1UH9JBCsq5QpsWeCR/Rj8kk5OUkLg9j6jz/HbUFpF4dLGOGW/nz58HUYcqxZzm5lmEmVKgIn3/nmNvrZ0J6clpNJX7PBwKnYXEE7AHgJM6KbrSb9T6H2Ty5799LDtf7o++dDTbwJG7lg4KY5CQuBX+Z/IkVl5PCxBbbERv4uG7/CNp4J9DhH8aD04w/rMHfN5+8iOCLVmLW8SXTuzUBekvM9Wm+uY8rGWUpuGQFNHRzXOTrkGXI8oT/OwRIXWpmrI81+yM5OWK88J92AvZ7gmHSsdD8c3m+IhU29PUIv8Kdpgts8gKNqd7mn++hR4lVciyiR9cbaEcXJQmP4fwUNiDp78wXy8f9i0oq2BZtv2Pl5YkX2qIQJY9
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(55236004)(83380400001)(54906003)(107886003)(508600001)(91956017)(8676002)(2616005)(26005)(2906002)(4326008)(53546011)(6506007)(76116006)(186003)(38070700005)(66446008)(66556008)(64756008)(31696002)(66946007)(6512007)(66476007)(122000001)(38100700002)(71200400001)(86362001)(6486002)(36756003)(31686004)(8936002)(6916009)(5660300002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?MXRQMWc2bmE3RDJmNTFXWTRwYmU3ajFibGJGMlA3NHlpMFpPSzNTWXlnZFB0?=
 =?utf-8?B?a2U4Z1gzMEJFenhvQU5vSEFFT0tKZUhQYWJmdTkzdGFBdVJqbTVZSWNsUzJU?=
 =?utf-8?B?QVpZL1FIZ05jT21Pb3JNM1oxU1oxSnlRUWw1aGxxaGF0NlJLTlE0WDk5YzUw?=
 =?utf-8?B?RFBoRE9wajNWWm13Rk83clFNOTFXQVYwd3QzVkZjRHFmS21GTy8vZlZKRGEx?=
 =?utf-8?B?LzN4ZkhPdlFNYk1ldWl1WjdtakVGQkVkUkJQeER1TWp6bzdFVkc5UnllaTJR?=
 =?utf-8?B?dFpwcHpoV3N3M2cvbGpNTTZjM1AwVFlRQWFNbXhKdFNFSHk4M2ZxNU1CUHdv?=
 =?utf-8?B?NFQ1ODd6WTRSM1BQZkJra3RQNUF3a2I0RWNlS3ZjTnlzcTNkSEtTMlFuL0RS?=
 =?utf-8?B?bU9SWWUyYTZvU0I0aWszTlNnbi9tWllEd0VpNnQ3cjdONy9KeExucmx4Sit5?=
 =?utf-8?B?eGMrcjZOSiszL0hQbnlIdFlpdFhXQjdRTTBrdXh0a084aG5oNk5LUmtubVJB?=
 =?utf-8?B?ck0rSXVkV2Jld3ZCaVJjSmduNVVKTEk2SFdoWDlxQ1hnQmF3OW5HLzM3cTNF?=
 =?utf-8?B?OWR5Y3J0SmdyWFNDbE05ZzBLdHVySVI2VUt4U2hmNlhPZlVRaWpWZDhTekph?=
 =?utf-8?B?NHQ0R2g2V3E4NXN1VDBFR1Jtck5HWXUyc3FmK0dteXc4ZitXNThXNFlYNW9V?=
 =?utf-8?B?WmhaT2ROUzdwT2paeVNMRU9DWHBVSVpBSlZhZU9ibVhLRC9UYjJXcUQ5ZXlD?=
 =?utf-8?B?Q3NqV3ZjeWp5czZCNW5KU1g3VTRXZmFGbzh0UlBiZ3JPTXQ1RXUrT3d4bUxk?=
 =?utf-8?B?bkcxVmVQUzFNeEtiNmZWR1k1SXFlUy9YN3JVYmdIWElGa0FQR29WSW9qcE1i?=
 =?utf-8?B?SFdKRWZRdXVCVmltY2hpQlo4SWs4RXY4U2pTenh1dnQvd0pqSTFhN2F5YlQ1?=
 =?utf-8?B?dlNseVJLaEpyTGJMS3JqQzNBRys3QlQ5TjBVRVlsWjVQY29vUjFnL3M3eGN2?=
 =?utf-8?B?NkhGZmxoNVQvMVZ6OW9NY0R4dkYzS2pIdWNpb3hyb2p2WXh5Njh5TXkyMEhR?=
 =?utf-8?B?c1NUUjgyVDErKzJybXcxZTZNZmdXbE1wTU1yQlFHOVlKVkpubFRsNW5PUXpn?=
 =?utf-8?B?TXpaUEZhaTR1T1hGS2REeFpqakp3RisvVzBlbVpQcU45T3VTeHgzYTU1M1JR?=
 =?utf-8?B?ZG5zcG5QengzUk8rWTBtNDl2UFJKZGZVZnNYejdGSnE2OU5RTUtRVE1mc1BK?=
 =?utf-8?B?NHEvbFhkM2J4S090c1UxUlNZK3pUVmpOZDQwR3VDRy9STVRDclVVaVpaM0dN?=
 =?utf-8?B?UlNKcE9iaWRybU5BVkNiUEJRd2k0KzZvcHlWL1FSc0xiZ2VxNjRMUnZIUk1L?=
 =?utf-8?B?dWFJUDNvTzFoMHovd1NZVXRHZGlLOXY0cjVqL2JhYzN3ckc0dWs1dFJIUWNt?=
 =?utf-8?B?RGczVWoyTENvWm9vVlJHU00wTEMzOW5pWC9NT2pRU2RrK2ZURzRibXBrdGlB?=
 =?utf-8?B?T2NSQUtKUzFpWXkyc1N4OHlkbU14MWZLcHpZQjc2andxR1FXdnlBS1lydjFM?=
 =?utf-8?B?QmtVRjNWRExic3JJY2Z6YkU0Ukp0NjBMT3g2S0VwdlNabEZuVTF3TGlhMG9x?=
 =?utf-8?B?TFY3Z3ltSGtNWE5OQlJNRThkbEdETEZuamtXRkxZQkZDd2dXVXVIMzBLNy9z?=
 =?utf-8?B?bnRNUXpFTFByTWZEdWZFUjZMK1ZaWnFJSVRhWHY3S25Ic09WTUxBK0IxQlI3?=
 =?utf-8?B?ZStYUDYweHF3aW9pTXFVUkN6U3l1SHdLNVFNS3dUeTRlTHgzYnJ6eDd5OEVB?=
 =?utf-8?B?K2EwdlE1S2VEVGFKVlAydz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <A5E8C70EA105B0419C75BB6D7EA4662A@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f04f02f-371c-4a26-6971-08d983f3a300
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 09:21:08.3386
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: goSWT4q7j5aBVL+4dPO2S05lR5e6NA2pL7MK5Xft/VLcl7hU6HVFcTOBGvzGSCvbpEwqNPySbd9hJXSlQA04XlgAJ6f/9Kgc2Tx+XAD+Y/VTKlg91zUQTWvzPG/lJiVv
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB6243
X-Proofpoint-GUID: MjhJtAp7tR-othFylupieeP_PNLorz_c
X-Proofpoint-ORIG-GUID: MjhJtAp7tR-othFylupieeP_PNLorz_c
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_03,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0
 lowpriorityscore=0 suspectscore=0 phishscore=0 bulkscore=0 mlxlogscore=999
 mlxscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 spamscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300057

DQoNCk9uIDMwLjA5LjIxIDEyOjA2LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAxMDo0NSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBPbiAzMC4wOS4yMSAx
MToyMSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4gT24gMzAuMDkuMjAyMSAwOTo1MiwgT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IEBAIC0xNDI5LDYgKzE0MzMsMTEgQEAgc3Rh
dGljIGludCBhc3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsIHUxNiBzZWcsIHU4IGJ1cywg
dTggZGV2Zm4sIHUzMiBmbGFnKQ0KPj4+PiAgICAgICAgICAgIHJjID0gaGQtPnBsYXRmb3JtX29w
cy0+YXNzaWduX2RldmljZShkLCBkZXZmbiwgcGNpX3RvX2RldihwZGV2KSwgZmxhZyk7DQo+Pj4+
ICAgICAgICB9DQo+Pj4+ICAgIA0KPj4+PiArICAgIGlmICggcmMgKQ0KPj4+PiArICAgICAgICBn
b3RvIGRvbmU7DQo+Pj4gICBGcm9tIGFsbCBJIGNhbiB0ZWxsIHRoaXMgaXMgZGVhZCBjb2RlLg0K
Pj4gQmVmb3JlIHRoZSBjaGFuZ2UgcmMgd2FzIHNldCBpbiB0aGUgbG9vcC4gQW5kIHRoZW4gd2Ug
ZmFsbCB0aHJvdWdoDQo+PiB0byB0aGUgImRvbmUiIGxhYmVsLiBJIGRvIGFncmVlIHRoYXQgdGhl
IHdheSB0aGlzIGNvZGUgaXMgZG9uZSB0aGUNCj4+IHZhbHVlIG9mIHRoYXQgcmMgd2lsbCBvbmx5
IHJlZmxlY3QgdGhlIGxhc3QgYXNzaWdubWVudCBkb25lIGluIHRoZSBsb29wLA0KPj4gYnV0IHdp
dGggbXkgY2hhbmdlIEkgZGlkbid0IHdhbnQgdG8gY2hhbmdlIHRoZSBleGlzdGluZyBiZWhhdmlv
ciwNCj4+IHRodXMgImlmICggcmMiDQo+IHJjIGlzIGFsd2F5cyAwIHVwb24gbG9vcCBleGl0LCBh
ZmFpY3Q6DQo+DQo+ICAgICAgZm9yICggOyBwZGV2LT5waGFudG9tX3N0cmlkZTsgcmMgPSAwICkN
Cj4NCj4gR3JhbnRlZCB0aGlzIGlzIHVudXN1YWwgYW5kIGhlbmNlIHBvc3NpYmx5IHVuZXhwZWN0
ZWQuDQpJIHdpbGwgcmVtb3ZlIHRoYXQgY2hlY2sgdGhlbi4gRG8gd2Ugd2FudCBhIGNvbW1lbnQg
YWJvdXQgcmMgPT0gMCwNCnNvIGl0IGlzIHNlZW4gd2h5IHRoZXJlIGlzIG5vIGNoZWNrIGZvciBy
Yz8NCj4NCj4+Pj4gLS0tIGEveGVuL2RyaXZlcnMvdnBjaS92cGNpLmMNCj4+Pj4gKysrIGIveGVu
L2RyaXZlcnMvdnBjaS92cGNpLmMNCj4+Pj4gQEAgLTg2LDYgKzg2LDI5IEBAIGludCBfX2h3ZG9t
X2luaXQgdnBjaV9hZGRfaGFuZGxlcnMoc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+Pj4+ICAgIA0K
Pj4+PiAgICAgICAgcmV0dXJuIHJjOw0KPj4+PiAgICB9DQo+Pj4+ICsNCj4+Pj4gKyNpZmRlZiBD
T05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVA0KPj4+PiArLyogTm90aWZ5IHZQQ0kgdGhhdCBk
ZXZpY2UgaXMgYXNzaWduZWQgdG8gZ3Vlc3QuICovDQo+Pj4+ICtpbnQgdnBjaV9hc3NpZ25fZGV2
aWNlKHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+Pj4+ICt7
DQo+Pj4+ICsgICAgLyogSXQgb25seSBtYWtlcyBzZW5zZSB0byBhc3NpZ24gZm9yIGh3ZG9tIG9y
IGd1ZXN0IGRvbWFpbi4gKi8NCj4+PiBDb3VsZCB5b3UgY2xhcmlmeSBmb3IgbWUgaW4gaG93IGZh
ciB0aGlzIGNvZGUgcGF0aCBpcyBpbmRlZWQgaW50ZW5kZWQNCj4+PiB0byBiZSB0YWtlbiBieSBo
d2RvbT8gQmVjYXVzZSBpZiBpdCBpcywgSSdkIGxpa2UgdG8gZnVydGhlciB1bmRlcnN0YW5kDQo+
Pj4gdGhlIGludGVyYWN0aW9uIHdpdGggc2V0dXBfaHdkb21fcGNpX2RldmljZXMoKS4NCj4+IHNl
dHVwX2h3ZG9tX3BjaV9kZXZpY2VzIGlzIG5vdCB1c2VkIG9uIEFybSBhcyB3ZSBkbyByZWx5IG9u
DQo+PiBEb20wIHRvIHBlcmZvcm0gUENJIGhvc3QgaW5pdGlhbGl6YXRpb24gYW5kIFBDSSBkZXZp
Y2UgZW51bWVyYXRpb24uDQo+Pg0KPj4gVGhpcyBpcyBiZWNhdXNlIG9mIHRoZSBmYWN0IHRoYXQg
b24gQXJtIGl0IGlzIG5vdCBhIHRyaXZpYWwgdGFzayB0bw0KPj4gaW5pdGlhbGl6ZSBhIFBDSSBo
b3N0IGJyaWRnZSBpbiBYZW4sIGUuZy4geW91IG5lZWQgdG8gcHJvcGVybHkgaW5pdGlhbGl6ZQ0K
Pj4gcG93ZXIgZG9tYWlucywgY2xvY2tzLCBxdWlya3MgZXRjLiBmb3IgZGlmZmVyZW50IFNvQ3Mu
DQo+PiBBbGwgdGhlc2UgbWFrZSB0aGUgdGFzayB0b28gY29tcGxleCBhbmQgaXQgd2FzIGRlY2lk
ZWQgdGhhdCBhdCB0aGUNCj4+IG1vbWVudCB3ZSBkbyBub3Qgd2FudCB0byBicmluZyBQQ0kgZGV2
aWNlIGRyaXZlcnMgaW4gWGVuIGZvciB0aGF0Lg0KPj4gSXQgd2FzIGFsc28gZGVjaWRlZCB0aGF0
IHdlIGV4cGVjdCBEb20wIHRvIHRha2UgY2FyZSBvZiBpbml0aWFsaXphdGlvbg0KPj4gYW5kIGVu
dW1lcmF0aW9uLg0KPj4gU29tZSBkYXksIHdoZW4gZmlybXdhcmUgY2FuIGRvIFBDSSBpbml0aWFs
aXphdGlvbiBmb3IgdXMgYW5kIHRoZW4gd2UNCj4+IGNhbiBlYXNpbHkgYWNjZXNzIEVDQU0sIHRo
aXMgd2lsbCBjaGFuZ2UuIFRoZW4gc2V0dXBfaHdkb21fcGNpX2RldmljZXMNCj4+IHdpbGwgYmUg
dXNlZCBvbiBBcm0gYXMgd2VsbC4NCj4+DQo+PiBUaHVzLCB3ZSBuZWVkIHRvIHRha2UgY2FyZSB0
aGF0IFhlbiBrbm93cyBhYm91dCB0aGUgZGlzY292ZXJlZA0KPj4gUENJIGRldmljZXMgdmlhIGFz
c2lnbl9kZXZpY2UgZXRjLg0KPiBGYWlyIGVub3VnaCwgYnV0IHNpbmNlIEkndmUgbm90IHNwb3R0
ZWQgYSBwYXRjaCBleHByZXNzaW5nIHRoaXMNCldlbGwsIGl0IGlzIGFsbCBpbiB0aGUgUkZDIGZv
ciBQQ0kgcGFzc3Rocm91Z2ggb24gQXJtIHdoaWNoIGlzIG1lbnRpb25lZA0KaW4gc2VyaWVzIGZy
b20gQXJtIGFuZCBFUEFNIChwYXJ0IDIpLiBJIGRpZG4ndCBtZW50aW9uIHRoZSBSRkMgaW4gdGhl
DQpjb3ZlciBsZXR0ZXIgZm9yIHRoaXMgc2VyaWVzIHRob3VnaC4NCj4gICAoYnkNCj4gYWRkaW5n
IHN1aXRhYmxlIGNvbmRpdGlvbmFscyksIG1heSBJIGFzayB0aGF0IHlvdSBkbyBzbyBpbiB5ZXQg
YW5vdGhlcg0KPiBwYXRjaCAodW5sZXNzIEkndmUgb3Zlcmxvb2tlZCB3aGVyZSB0aGlzIGdldHMg
ZG9uZSk/DQpDb3VsZCB5b3UgcGxlYXNlIGVsYWJvcmF0ZSBtb3JlIG9uIHdoaWNoIGNvbmRpdGlv
bmFscyB5b3UgYXJlDQp0YWxraW5nIGFib3V0IGhlcmU/IEknbSBhZnJhaWQgSSBkaWRuJ3QgdW5k
ZXJzdGFuZCB0aGlzIHBhcnQuDQo+DQo+Pj4+ICsgICAgaWYgKCBpc19zeXN0ZW1fZG9tYWluKGQp
IHx8ICFoYXNfdnBjaShkKSApDQo+Pj4+ICsgICAgICAgIHJldHVybiAwOw0KPj4+PiArDQo+Pj4+
ICsgICAgcmV0dXJuIDA7DQo+Pj4+ICt9DQo+Pj4+ICsNCj4+Pj4gKy8qIE5vdGlmeSB2UENJIHRo
YXQgZGV2aWNlIGlzIGRlLWFzc2lnbmVkIGZyb20gZ3Vlc3QuICovDQo+Pj4+ICtpbnQgdnBjaV9k
ZWFzc2lnbl9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0IHBjaV9kZXYgKmRl
dikNCj4+Pj4gK3sNCj4+Pj4gKyAgICAvKiBJdCBvbmx5IG1ha2VzIHNlbnNlIHRvIGRlLWFzc2ln
biBmcm9tIGh3ZG9tIG9yIGd1ZXN0IGRvbWFpbi4gKi8NCj4+Pj4gKyAgICBpZiAoIGlzX3N5c3Rl
bV9kb21haW4oZCkgfHwgIWhhc192cGNpKGQpICkNCj4+Pj4gKyAgICAgICAgcmV0dXJuIDA7DQo+
Pj4+ICsNCj4+Pj4gKyAgICByZXR1cm4gMDsNCj4+Pj4gK30NCj4+Pj4gKyNlbmRpZiAvKiBDT05G
SUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVCAqLw0KPj4+IEF0IHRoaXMgcG9pbnQgb2YgdGhlIHNl
cmllcyAjaWZkZWYgaXMgdGhlIGxlc3MgcHJlZmVyYWJsZSB2YXJpYW50IG9mDQo+Pj4gYXJyYW5n
aW5nIGZvciBkZWFkIGNvZGUgdG8gZ2V0IGNvbXBpbGVkIG91dC4NCj4+IFdoYXQgaXMgdGhhdCBv
dGhlciBwcmVmZXJhYmxlIHdheSB0aGVuPw0KPiAiaWYgKCAhSVNfRU5BQkxFRCgpICkiIGFzIEkg
ZGlkIGFscmVhZHkgcG9pbnQgb3V0IHRvIHlvdSB5ZXN0ZXJkYXkgaW4NCj4gcmVwbHkgdG8gdjIg
b2YgcGF0Y2ggMTAgb2YgdGhpcyB2ZXJ5IHNlcmllcy4NClBsZWFzZSBzZWUgYmVsb3cNCj4NCj4+
PiAgICBJIGV4cGVjdCBsYXRlciBwYXRjaGVzDQo+Pj4gd2lsbCBjaGFuZ2UgdGhhdD8NCj4+IE5v
LCBpdCBpcyBnb2luZyB0byBiZSB0aGlzIHdheSBhbGwgdGhlIHRpbWUNCj4gVGhlIHF1ZXN0aW9u
IHdhc24ndCB3aGV0aGVyIHlvdSBzd2l0Y2ggYXdheSBmcm9tIHRoZSAjaWZkZWYtcywgYnV0DQo+
IHdoZXRoZXIgbGF0ZXIgcGF0Y2hlcyBsZWF2ZSB0aGF0IGFzIHRoZSBvbmx5IGNob2ljZSAoYXZv
aWRpbmcgYnVpbGQNCj4gYnJlYWthZ2UpLg0KWWVzLCB0aGUgY29kZSBpcyBnb2luZyB0byBhbHdh
eXMgcmVtYWluIGlmZGVmJ2VkLCBzbyB3ZSBkb24ndCBoYXZlDQpkZWFkIGNvZGUgZm9yIHg4NiAo
YXQgbGVhc3QpLg0KU28sIGRvZXMgdGhlIGFib3ZlIG1lYW4gdGhhdCB5b3UgYXJlIG9rIHdpdGgg
IiNpZmRlZiBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVCINCmFuZCB0aGVyZSBpcyBubyBu
ZWVkIGZvciAiaWYgKCAhSVNfRU5BQkxFRCgpICkiPw0KPg0KPiBKYW4NCj4NCj4NClRoYW5rIHlv
dSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:27:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:27:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199659.353851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsLN-0008Ah-8I; Thu, 30 Sep 2021 09:27:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199659.353851; Thu, 30 Sep 2021 09:27:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsLN-0008Aa-4A; Thu, 30 Sep 2021 09:27:09 +0000
Received: by outflank-mailman (input) for mailman id 199659;
 Thu, 30 Sep 2021 09:27:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mlQV=OU=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVsLL-0008AU-QO
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:27:07 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id b7f99fc4-1234-4d97-86ac-d9038ac64d39;
 Thu, 30 Sep 2021 09:27:05 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BE4E5D6E;
 Thu, 30 Sep 2021 02:27:04 -0700 (PDT)
Received: from e123311-lin.arm.com (unknown [10.57.19.250])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7D1DC3F793;
 Thu, 30 Sep 2021 02:27:01 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7f99fc4-1234-4d97-86ac-d9038ac64d39
From: Michal Orzel <michal.orzel@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>,
	Wei Liu <wl@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	bertrand.marquis@arm.com,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH] xen/arm: Expose the PMU to the guests
Date: Thu, 30 Sep 2021 11:26:51 +0200
Message-Id: <20210930092651.1350-1-michal.orzel@arm.com>
X-Mailer: git-send-email 2.29.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add parameter vpmu to xl domain configuration syntax
to enable the access to PMU registers by disabling
the PMU traps.

This change does not expose the full PMU to the guest.
Long term, we will want to at least expose the PMU
interrupts, device-tree binding. For more use cases
we will also need to save/restore the PMU context.

Signed-off-by: Michal Orzel <michal.orzel@arm.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
---
 docs/man/xl.cfg.5.pod.in         | 12 ++++++++++++
 tools/include/libxl.h            |  7 +++++++
 tools/libs/light/libxl_arm.c     |  6 +++++-
 tools/libs/light/libxl_types.idl |  1 +
 tools/xl/xl_parse.c              |  2 ++
 xen/arch/arm/domain.c            | 10 ++++++++--
 xen/common/domain.c              |  2 +-
 xen/include/asm-arm/domain.h     |  1 +
 xen/include/public/domctl.h      |  5 ++++-
 9 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 4b1e3028d2..4a75203b9f 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2843,6 +2843,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
 
 =back
 
+=item B<vpmu=BOOLEAN>
+
+Specifies whether to enable the access to PMU registers by disabling
+the PMU traps.
+
+This change does not expose the full PMU to the guest.
+Currently there is no support for virtualization, interrupts, etc.
+
+Enabling this option may result in potential security holes.
+
+If this option is not specified then it will default to B<false>.
+
 =head3 x86
 
 =over 4
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index b9ba16d698..c6694e977d 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -502,6 +502,13 @@
  */
 #define LIBXL_HAVE_X86_MSR_RELAXED 1
 
+/*
+ * LIBXL_HAVE_ARM_VPMU indicates the toolstack has support for enabling
+ * the access to PMU registers by disabling the PMU traps. This is done
+ * by setting the libxl_domain_build_info arch_arm.vpmu field.
+ */
+#define LIBXL_HAVE_ARM_VPMU 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index e3140a6e00..bc614dcb05 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -29,6 +29,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
     uint32_t vuart_irq;
     bool vuart_enabled = false;
 
+    if (libxl_defbool_val(d_config->b_info.arch_arm.vpmu))
+        config->flags |= XEN_DOMCTL_CDF_pmu;
+
     /*
      * If pl011 vuart is enabled then increment the nr_spis to allow allocation
      * of SPI VIRQ for pl011.
@@ -1186,8 +1189,9 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc,
 void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
                                               libxl_domain_build_info *b_info)
 {
-    /* ACPI is disabled by default */
+    /* ACPI and vPMU is disabled by default */
     libxl_defbool_setdefault(&b_info->acpi, false);
+    libxl_defbool_setdefault(&b_info->arch_arm.vpmu, false);
 
     if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
         return;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 3f9fff653a..3524629909 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 
     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
                                ("vuart", libxl_vuart_type),
+                               ("vpmu", libxl_defbool),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 17dddb4cd5..e15c2e64f5 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2750,6 +2750,8 @@ skip_usbdev:
                     "If it fixes an issue you are having please report to "
                     "xen-devel@lists.xenproject.org.\n");
 
+    xlu_cfg_get_defbool(config, "vpmu", &b_info->arch_arm.vpmu, 0);
+
     xlu_cfg_destroy(config);
 }
 
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 19c756ac3d..02120ff640 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -276,6 +276,8 @@ static void ctxt_switch_to(struct vcpu *n)
      * timer. The interrupt needs to be injected into the guest. */
     WRITE_SYSREG(n->arch.cntkctl, CNTKCTL_EL1);
     virt_timer_restore(n);
+
+    WRITE_SYSREG(n->arch.mdcr_el2, MDCR_EL2);
 }
 
 /* Update per-VCPU guest runstate shared memory area (if registered). */
@@ -586,6 +588,10 @@ int arch_vcpu_create(struct vcpu *v)
 
     v->arch.hcr_el2 = get_default_hcr_flags();
 
+    v->arch.mdcr_el2 = HDCR_TDRA | HDCR_TDOSA | HDCR_TDA;
+    if ( !(v->domain->options & XEN_DOMCTL_CDF_pmu) )
+        v->arch.mdcr_el2 |= HDCR_TPM | HDCR_TPMCR;
+
     if ( (rc = vcpu_vgic_init(v)) != 0 )
         goto fail;
 
@@ -622,8 +628,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
     unsigned int max_vcpus;
 
-    /* HVM and HAP must be set. IOMMU may or may not be */
-    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=
+    /* HVM and HAP must be set. IOMMU and PMU may or may not be */
+    if ( (config->flags & ~(XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_pmu)) !=
          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6ee5d033b0..63c4db8b9f 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
-           XEN_DOMCTL_CDF_nested_virt) )
+           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_pmu) )
     {
         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
         return -EINVAL;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c9277b5c6d..14e575288f 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -166,6 +166,7 @@ struct arch_vcpu
 
     /* HYP configuration */
     register_t hcr_el2;
+    register_t mdcr_el2;
 
     uint32_t teecr, teehbr; /* ThumbEE, 32-bit guests only */
 #ifdef CONFIG_ARM_32
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 96696e3842..a55ceb81db 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -70,9 +70,12 @@ struct xen_domctl_createdomain {
 #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
 #define _XEN_DOMCTL_CDF_nested_virt   6
 #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
+/* Should we expose the vPMU to the guest? */
+#define _XEN_DOMCTL_CDF_pmu           7
+#define XEN_DOMCTL_CDF_pmu            (1U << _XEN_DOMCTL_CDF_pmu)
 
 /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
-#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
+#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_pmu
 
     uint32_t flags;
 
-- 
2.29.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:35:12 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:35:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199665.353863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsT6-00019v-3S; Thu, 30 Sep 2021 09:35:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199665.353863; Thu, 30 Sep 2021 09:35:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsT5-00019o-UE; Thu, 30 Sep 2021 09:35:07 +0000
Received: by outflank-mailman (input) for mailman id 199665;
 Thu, 30 Sep 2021 09:35:06 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVsT4-00019i-Ak
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:35:06 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b058db1d-21d1-11ec-bd40-12813bfff9fa;
 Thu, 30 Sep 2021 09:35:04 +0000 (UTC)
Received: from pps.filterd (m0174679.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U9WguI028296; 
 Thu, 30 Sep 2021 09:35:03 GMT
Received: from eur05-am6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bdapw80bd-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 09:35:02 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6882.eurprd03.prod.outlook.com (2603:10a6:20b:283::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 09:34:59 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 09:34:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b058db1d-21d1-11ec-bd40-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bRF48iGRfLfIE37uKTLuuB+CBRAeTE3q9hogMule+3vrPjWjZTmxgmN6NusdEJpxD8NZPnAae87EWInbZBh7WbvSNV5HT9FwaFwGotx9k9A/GEqImVOmJ2DyAWkM426QJY90CqC95LC5RbuzkNGO7hTUDLFqEJiQrRwbWVoa2IZlw6JQI2c5lexHYXjqq3UNxj3iiLbl3/cy3pdOz/IwHYZZ5q99v6pLBjqrgDhy5pEBJDJXEvH8RLq7VLR/On3SpqVc3G1l8AhbS16BK6tQ60Juck1nNVC/Kgy0NABEywm/PrOGhi0hjNGDwMQXELi+5Ab5yXB2INX3wJJU4iOOTw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=n1XZHUwlSAfHhhLMFO4ZKfkLQSJEd/109NwVnP/VIWw=;
 b=Mb+DRDqXLla3NCEu+FE++bQlfEFLzcdh52M4EXpX8uwR2tAr56umrWrmSrquiSHHy5bV4pOVo4LlyRCcR3MnKsLs0Rzb+MxlGCqLigQypVU4IrPyE3H3SERDLAQ8Zib5uAJq23Be9h9vDEBDreXxAWqU8qqve3ow8FItQ7WHLDm1xZ3MjxlPlsVZMAXa2TWJpcWMM1SPlOtLHr6iz0fVc9zJWpLZZizmssHmX8/ujuzVvkqILM7eKld5VK0YS5WHlbez2qFkpivWd/skGll3HfMNj0XD+QC0ub9tgOU1mmMZI90xiQfhxAj0GWtTeOLClguHnOSz4Op+bnWGNnTFPw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=n1XZHUwlSAfHhhLMFO4ZKfkLQSJEd/109NwVnP/VIWw=;
 b=5hCjIkUHT1fUR0LYngCQiNmD8Px7aQ70eoi18ahIUjedunJuR3ESHLTrSgLBumwIubLtzMdYocaWjPHg3ACbOtDqF8InjaztU9xTL3ZOMjvqLvNAcXWj6+1xnGSTe4qi9pOGXwAc+EMpN5+Yl6ONXAm4OBTmtpmZuWNOQ0PT3WBoKrHVYGuXHmEfRgJBsK9ysLa6SDkFmi2i94hNrEkNyEh7jaRMQpkIRo3RSN/MqNfq5Kv875dBlRyjtIELJZVpw3xXOzRyQYOQAhU/DEHEdmFFEonK4Ww465qJfKpHvS8J0N4s9CgRE8bGt+AOauRwftcS8okz5sqxu/awAuIZ4Q==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: AQHXtdAlOCu2AnkIOUC4Prsocox3squ8RSWAgAAMGAA=
Date: Thu, 30 Sep 2021 09:34:59 +0000
Message-ID: <603e2cdf-748f-f18b-99f4-44ddf3c0b9c1@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-11-andr2000@gmail.com>
 <ef2c008f-7682-1549-8d27-712b04fbd84f@suse.com>
In-Reply-To: <ef2c008f-7682-1549-8d27-712b04fbd84f@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2f9dac90-6452-404a-a080-08d983f59286
x-ms-traffictypediagnostic: AM9PR03MB6882:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB68824DD13F4A43C296CD0E6CE7AA9@AM9PR03MB6882.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 9+A3/EfctKrFxT6hnk2m61n7r827erPDGKWa19h151u4atxsYhM6bnBcnDAPO2LtsfGsr8xGtfxGODrjnSvkM2tjOZCJPGinrDSUa4FQTOl2DLLuONqpPM5Ij88qwxJhpXyi31H3yUUP/HD050D1H452mdCPXjgnwMGt5CNdxZX0bZsKUF2szdtEU/4bPLj9Lrgb6jPuQ21sNUDXN4Re994dp6RgzDNl7AwVQoPDLG3eEMrXXlOZDXOLceHyLy1ulkOOf9nIw3LHNZw+6SvT6AErsExJPViUHbOckBI0JHRNt5/HDT4K0UQwQlvUJqD3OHMppgApJQHWyGk4AvYF3z0EkDZjnN5rRUBBUfyCL1vBhgYmhuVfgrm+X7Ar1B0/YWYT1MfkVGxV3b7fgBsrLF5ToBeQE5/owTjZOziQ9rvGY/MA7Px1KHgBgIUh4Nvm8vTL5/+1DXBt2Pso2DckbhHnR1no4S+dIqXfAW9mPPRS5WaufmjLupenPJ+5LgCu5sA6tBQnRNXWnwDXdQu8k6+l/u3oAcMSn6HHSRLQBKexEQdsM7cqtHj/I4KGhTIxqB/Bk65Hf/X06nJwW1KQgYl+aDqUPgrgv2vT9UTFr3JGsgptJ/GT/2N+sd8oI0NCMGPcHysVJji/XiY0RfBpW4VOPBoG4XH8Ssd7nGvPxbSNoM9SYNecSc1BQINNUmw8TWIYjpZ9Tr26+6fPz00ZGcu1LkIdLX0QWxLPfM+XVprrp0l5tVHceRzMF2EBKX/W
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6506007)(8676002)(26005)(53546011)(4326008)(186003)(2906002)(6916009)(83380400001)(5660300002)(86362001)(55236004)(31696002)(71200400001)(8936002)(38070700005)(91956017)(66946007)(76116006)(66446008)(2616005)(64756008)(31686004)(38100700002)(6512007)(122000001)(66476007)(66556008)(508600001)(6486002)(54906003)(316002)(36756003)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?alNEdkNBTS9BR0ZmdHVNTkdQak93S0JVMTBReUJWSzkvRERzUUtYUmFiYjl1?=
 =?utf-8?B?ZmZFRU1Zb0t4anoxckVKL2xWaWh6SGdJeFM2cWR6clEyd0daeENvcmtFeDY4?=
 =?utf-8?B?bTBjc2lBbWw0QTlNN0h3UUhUNStSRENmNVVZVG9GbENXOVg5S2Q1Njl1MzVz?=
 =?utf-8?B?TEdlMEYyOUZmeWo0SnNTWUc3QTJrN2Q2Rnh1MDlSTjdhdHh4SUU5clhVNkRY?=
 =?utf-8?B?NytwWHoyTlF1STFUeFNjM3BVTnhzK1FyY3YyVEpvR1BVdlZnVWVsWXhoVkFK?=
 =?utf-8?B?ZEdDVTlzdk9TUlJlUzA4YVlWRnN4UjI3Z2FaaENTaC9Tb25CS0ZqS0c3QzJt?=
 =?utf-8?B?NU5vdFFmbFAxcDNxdkFNVnd1MVFLcHplRGFRZHJ6K0wrT1BEOXpoOVlrMzFW?=
 =?utf-8?B?Ui85UTdSUFNWU2FjbkxvNDBjU2JjTzFyNjVsQUEvZVRzV3kxWXBCVmdFWWxZ?=
 =?utf-8?B?ZWVtd1VxQWZxOGJucnpLTkU5c1gzVjFIMXJ5aDhYOWlBbm1iRzA1ZFlDVUFn?=
 =?utf-8?B?K25CNHRnNHFGMFcxdHByYnZzcGwrcm5qMmk1Kys1eHVGR2RwWVhIUkVKWEJx?=
 =?utf-8?B?eDkyWWovaDFaWHpCM1dzMmhDeDZZcEhnYXNIMWswOXdjVzBrcEpTTW9hR2xt?=
 =?utf-8?B?QTZ6b3VVeFR2MWlzcmRURUExaVF0WnVZZm92ZTBHUkhLUUhJTmR5NFA5VjVt?=
 =?utf-8?B?N2JmVmkyVTNwV2xkTVFQNVFmVHlPbkdPVEZsQ0NGVjFkSm1lMTNlQlk1K2Mv?=
 =?utf-8?B?eEpQWmVoMDRCeVBaYzZ4b3gvM2dBTGU2MCs5WTB3TjRudmlQeHZJY3liV1M1?=
 =?utf-8?B?ZFVKb3oySGcxQnNzM1YwMUlpbml2cVk5VlpHMllNck5RbW1naEdNKzNIc0tu?=
 =?utf-8?B?YVJ5YjlSQ3R3eS9sWmRKc1UwOTBjUFA5aGY5Uzc0Z3FrZllDd1d2bEJYa216?=
 =?utf-8?B?NlpzTUZiZ1Y5YldQTkliZ1IvbFhGK1paSEhOM085NGU5RXFOSWFlSkpZMG4y?=
 =?utf-8?B?RVQvSTdRR0FFSThYZnhIdC9WR3JldXZaV00xRU9qNERoa2J6bk5HaXFrdnlF?=
 =?utf-8?B?ejdldDVzVWVNczNIMCtCMkhvN3c4b00zVnU4UE1VT3VzTklUdzc2NjZuYmFw?=
 =?utf-8?B?YjhDN3FoQU05VTNmT3FKcFY1MThwb3JjK25CN0dWWWMxZGdZa1pkclc5b1dG?=
 =?utf-8?B?U1M4SklRZlFGYlNiOHA2Si9uTGpqL1l4RFBsZmxSbTFPSUwycVNkemVoNklQ?=
 =?utf-8?B?T01mODVkMlF1dW9hUXRhK3pWRTJIbkFTUmtHRzNnVTFLMnZFZmk0enc0TlRZ?=
 =?utf-8?B?TmptUWkzTTVtQTRubWFySWNYZExpTFdoVjg4WnBxdkg5RXNSMkc4UlZiZGlh?=
 =?utf-8?B?dmNGbkMzeXF5ZlhHMFRJNFhzSzVkUmpNc0lQalIyMjdlQUxndVk5NXZlKzEz?=
 =?utf-8?B?dWNWRk8yWjhtbXlaQktBeXQzTTQ3U3hySE5YU1B5SVkrR2J6bm95anUzSXdG?=
 =?utf-8?B?RnNZeFVnLzVibklnWFNEWStkVUN5blZGSFpVWXNYMlJZR3RRSTFOVFBDZW1s?=
 =?utf-8?B?QVdvRUZMcjhFRXZJT3dEVFlXT1lpZkt5TWlhTGZDajdnbzRhZkFJbzNBVzZa?=
 =?utf-8?B?Vlp6UjZMSXZKbFBCRGx0VUkyVGltcGtDaUszRzZrTTRuUUVtS2orVHVxNTdl?=
 =?utf-8?B?U1JIYTFsdnY5MDZiT0FWa3hiREdHbllXdlN3K3VwS3p3RFpCb2xMZS85eWt1?=
 =?utf-8?B?WVgyY1FiN1YwZVA1KzlRdWdrZVdjRXJkc0FlMktpV3UrWDJEQnBhbmRhU1Mv?=
 =?utf-8?B?SnBUK0s2amZFbXREb3N1Zz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <8F13D52DA31080458DE3B6B4F58CA9D0@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f9dac90-6452-404a-a080-08d983f59286
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 09:34:59.6684
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: QmZJWMRHIR2y7M9DbCUWJ0nSb6WVJa6sEGiaalrM36tVDSlC/+B45rjDvZD0+pTeLNZTxg9jiKPT4JV/jNqvEk7kEz9LTHdp4ojo7JfTbsFWqAHzaUM8cssqe3OZNKe2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6882
X-Proofpoint-ORIG-GUID: GMY5DWjaduFxNQMjyJQSUvyukfhPDbHF
X-Proofpoint-GUID: GMY5DWjaduFxNQMjyJQSUvyukfhPDbHF
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_03,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0
 lowpriorityscore=0 priorityscore=1501 mlxscore=0 suspectscore=0
 mlxlogscore=999 bulkscore=0 phishscore=0 impostorscore=0 adultscore=0
 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300058

DQoNCk9uIDMwLjA5LjIxIDExOjUxLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAwOTo1MiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBGcm9tOiBPbGVrc2Fu
ZHIgQW5kcnVzaGNoZW5rbyA8b2xla3NhbmRyX2FuZHJ1c2hjaGVua29AZXBhbS5jb20+DQo+Pg0K
Pj4gQXNzaWduIFNCREYgdG8gdGhlIFBDSSBkZXZpY2VzIGJlaW5nIHBhc3NlZCB0aHJvdWdoIHdp
dGggYnVzIDAuDQo+IFRoaXMgcmVhZHMgYSBsaXR0bGUgb2RkOiBJZiBidXMgaXMgYWxyZWFkeSBr
bm93biAoYW5kIEkgdGhpbmsgeW91IGltcGx5DQo+IHNlZ21lbnQgdG8gYWxzbyBiZSBrbm93biks
IGl0J3Mgb25seSBERiB3aGljaCBnZXQgYXNzaWduZWQuDQpCdXQgYXQgdGhlIGVuZCBvZiB0aGUg
ZGF5IHdlIHNldCBhbGwgdGhlIHBhcnRzIG9mIHRoYXQgU0JERi4NCk90aGVyd2lzZSBJIHNob3Vs
ZCB3cml0ZSAiQXNzaWduIERGIGFzIHdlIGtub3cgdGhhdCBidXMgYW5kIHNlZ21lbnQNCmFyZSAw
Ig0KPg0KPj4gVGhlIHJlc3VsdGluZyB0b3BvbG9neSBpcyB3aGVyZSBQQ0llIGRldmljZXMgcmVz
aWRlIG9uIHRoZSBidXMgMCBvZiB0aGUNCj4+IHJvb3QgY29tcGxleCBpdHNlbGYgKGVtYmVkZGVk
IGVuZHBvaW50cykuDQo+PiBUaGlzIGltcGxlbWVudGF0aW9uIGlzIGxpbWl0ZWQgdG8gMzIgZGV2
aWNlcyB3aGljaCBhcmUgYWxsb3dlZCBvbg0KPj4gYSBzaW5nbGUgUENJIGJ1cy4NCj4gT3IgdXAg
dG8gMjU2IHdoZW4gdGhlcmUgYXJlIG11bHRpLWZ1bmN0aW9uIG9uZXMuIEltbyB5b3UgYXQgbGVh
c3Qgd2FudA0KPiB0byBzcGVsbCBvdXQgaG93IHRoYXQgY2FzZSBpcyBpbnRlbmRlZCB0byBiZSBo
YW5kbGVkIChldmVuIGlmIG1heWJlDQo+IHRoZSBjb2RlIGRvZXNuJ3QgY292ZXIgdGhhdCBjYXNl
IHlldCwgaW4gd2hpY2ggY2FzZSBhIHJlc3BlY3RpdmUgY29kZQ0KPiBjb21tZW50IHdvdWxkIGFs
c28gd2FudCBsZWF2aW5nKS4NCldlIGFyZSBub3Qgc3VwcG9ydGluZyBtdWx0aS1mdW5jdGlvbiB5
ZXQsIHNvIEknbGwgYWRkIGEgY29tbWVudC4NCj4NCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0
aHJvdWdoL3BjaS5jDQo+PiArKysgYi94ZW4vZHJpdmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4g
QEAgLTgzMSw2ICs4MzEsNjYgQEAgaW50IHBjaV9yZW1vdmVfZGV2aWNlKHUxNiBzZWcsIHU4IGJ1
cywgdTggZGV2Zm4pDQo+PiAgICAgICByZXR1cm4gcmV0Ow0KPj4gICB9DQo+PiAgIA0KPj4gKyNp
ZmRlZiBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVA0KPiBNYXkgSSBhc2sgd2h5IHRoZSBj
b2RlIGVuY2xvc2VkIGJ5IHRoaXMgY29uZGl0aW9uYWwgaGFzIGJlZW4gcHV0IGhlcmUNCj4gcmF0
aGVyIHRoYW4gdW5kZXIgZHJpdmVycy92cGNpLz8NCkluZGVlZCB0aGlzIGNhbiBiZSBtb3ZlZCB0
byB4ZW4vZHJpdmVycy92cGNpL3ZwY2kuYy4NCkknbGwgbW92ZSBhbmQgdXBkYXRlIGZ1bmN0aW9u
IG5hbWVzIGFjY29yZGluZ2x5Lg0KPg0KPj4gK3N0YXRpYyBzdHJ1Y3QgdnBjaV9kZXYgKnBjaV9m
aW5kX3ZpcnR1YWxfZGV2aWNlKGNvbnN0IHN0cnVjdCBkb21haW4gKmQsDQo+PiArICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IHBjaV9k
ZXYgKnBkZXYpDQo+PiArew0KPj4gKyAgICBzdHJ1Y3QgdnBjaV9kZXYgKnZkZXY7DQo+PiArDQo+
PiArICAgIGxpc3RfZm9yX2VhY2hfZW50cnkgKCB2ZGV2LCAmZC0+dmRldl9saXN0LCBsaXN0ICkN
Cj4+ICsgICAgICAgIGlmICggdmRldi0+cGRldiA9PSBwZGV2ICkNCj4+ICsgICAgICAgICAgICBy
ZXR1cm4gdmRldjsNCj4+ICsgICAgcmV0dXJuIE5VTEw7DQo+PiArfQ0KPiBObyBsb2NraW5nIGhl
cmUgb3IgLi4uDQo+DQo+PiAraW50IHBjaV9hZGRfdmlydHVhbF9kZXZpY2Uoc3RydWN0IGRvbWFp
biAqZCwgY29uc3Qgc3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+PiArew0KPj4gKyAgICBzdHJ1Y3Qg
dnBjaV9kZXYgKnZkZXY7DQo+PiArDQo+PiArICAgIEFTU0VSVCghcGNpX2ZpbmRfdmlydHVhbF9k
ZXZpY2UoZCwgcGRldikpOw0KPiAuLi4gaW4gdGhpcyBmaXJzdCBjYWxsZXIgdGhhdCBJJ3ZlIG1h
bmFnZWQgdG8gc3BvdD8gU2VlIGFsc28gYmVsb3cgYXMNCj4gdG8gdXAgdGhlIGNhbGwgdHJlZSB0
aGUgcGNpZGV2cy1sb2NrIGJlaW5nIGhlbGQgKHdoaWNoIGF0IHRoZSB2ZXJ5DQo+IGxlYXN0IHlv
dSB3b3VsZCB0aGVuIHdhbnQgdG8gQVNTRVJUKCkgZm9yIGhlcmUpLg0KSSB3aWxsIG1vdmUgdGhl
IGNvZGUgdG8gdnBjaSBhbmQgbWFrZSBzdXJlIHByb3BlciBsb2NraW5nIHRoZXJlDQo+DQo+PiAr
ICAgIC8qIEVhY2ggUENJIGJ1cyBzdXBwb3J0cyAzMiBkZXZpY2VzL3Nsb3RzIGF0IG1heC4gKi8N
Cj4+ICsgICAgaWYgKCBkLT52cGNpX2Rldl9uZXh0ID4gMzEgKQ0KPj4gKyAgICAgICAgcmV0dXJu
IC1FTk9TUEM7DQo+IFBsZWFzZSBhdm9pZCBvcGVuLWNvZGluZyBsaXRlcmFscyB3aGVuIHRoZXkg
Y2FuIGJlIHN1aXRhYmx5IGV4cHJlc3NlZC4NCkkgZmFpbGVkIHRvIGZpbmQgYSBzdWl0YWJsZSBj
b25zdGFudCBmb3IgdGhhdC4gQ291bGQgeW91IHBsZWFzZSBwb2ludA0KbWUgdG8gdGhlIG9uZSBJ
IGNhbiB1c2UgaGVyZT8NCj4NCj4+ICsgICAgdmRldiA9IHh6YWxsb2Moc3RydWN0IHZwY2lfZGV2
KTsNCj4+ICsgICAgaWYgKCAhdmRldiApDQo+PiArICAgICAgICByZXR1cm4gLUVOT01FTTsNCj4+
ICsNCj4+ICsgICAgLyogV2UgZW11bGF0ZSBhIHNpbmdsZSBob3N0IGJyaWRnZSBmb3IgdGhlIGd1
ZXN0LCBzbyBzZWdtZW50IGlzIGFsd2F5cyAwLiAqLw0KPj4gKyAgICB2ZGV2LT5zZWcgPSAwOw0K
Pj4gKw0KPj4gKyAgICAvKg0KPj4gKyAgICAgKiBUaGUgYnVzIG51bWJlciBpcyBzZXQgdG8gMCwg
c28gdmlydHVhbCBkZXZpY2VzIGFyZSBzZWVuDQo+PiArICAgICAqIGFzIGVtYmVkZGVkIGVuZHBv
aW50cyBiZWhpbmQgdGhlIHJvb3QgY29tcGxleC4NCj4+ICsgICAgICovDQo+PiArICAgIHZkZXYt
PmJ1cyA9IDA7DQo+IFN0cmljdGx5IHNwZWFraW5nIGJvdGggb2YgdGhlc2UgYXNzaWdubWVudHMg
YXJlIHJlZHVuZGFudCB3aXRoIHlvdQ0KPiB1c2luZyB4emFsbG9jKCkuIEknZCBwcmVmZXIgaWYg
dGhlcmUgd2FzIGp1c3QgYSBjb21tZW50LCBhcyB0aGUgY29tcGlsZXINCj4gaGFzIG5vIHdheSBy
ZWNvZ25pemluZyB0aGlzIGluIG9yZGVyIHRvIGVsaW1pbmF0ZSB0aGVzZSBzdG9yZXMuDQpZZXMs
IEkganVzdCBwdXQgdGhlIGFzc2lnbm1lbnRzIHRvIGJlIGV4cGxpY2l0bHkgc2VlbiBoZXJlIGFz
IGJlaW5nIDAuDQpJIHdpbGwgcmVtb3ZlIHRob3NlIGFuZCBwdXQgYSBjb21tZW50Lg0KPg0KPj4g
KyAgICB2ZGV2LT5kZXZmbiA9IFBDSV9ERVZGTihkLT52cGNpX2Rldl9uZXh0KyssIDApOw0KPj4g
Kw0KPj4gKyAgICB2ZGV2LT5wZGV2ID0gcGRldjsNCj4+ICsgICAgdmRldi0+ZG9tYWluID0gZDsN
Cj4+ICsNCj4+ICsgICAgcGNpZGV2c19sb2NrKCk7DQo+PiArICAgIGxpc3RfYWRkX3RhaWwoJnZk
ZXYtPmxpc3QsICZkLT52ZGV2X2xpc3QpOw0KPj4gKyAgICBwY2lkZXZzX3VubG9jaygpOw0KPiBJ
IGRvbid0IHN1cHBvcnQgYSBnbG9iYWwgbG9jayBnZXR0aW5nIChhYil1c2VkIGZvciBwZXItZG9t
YWluIGxpc3QNCj4gbWFuYWdlbWVudC4NCj4NCj4gQXBhcnQgZnJvbSB0aGF0IEkgZG9uJ3QgdW5k
ZXJzdGFuZCB3aHkgeW91IGFjcXVpcmUgdGhlIGxvY2sgaGVyZS4gRG9lcw0KPiB0aGF0IG1lYW4g
dGhlIGZ1bmN0aW9ucyBmdXJ0aGVyIHdlcmUgdHJ1bHkgbGVmdCB3aXRob3V0IGFueSBsb2NraW5n
LA0KPiBieSB5b3Ugbm90IGhhdmluZyBub3RpY2VkIHRoYXQgdGhpcyBsb2NrIGlzIGFscmVhZHkg
YmVpbmcgaGVsZCBieSB0aGUNCj4gc29sZSBjYWxsZXI/DQpJJ2xsIHJlLXdvcmsgbG9ja2luZyB3
aXRoIHJlc3BlY3QgdG8gdGhlIG5ldyBsb2NhdGlvbiBmb3IgdGhpcywgZS5nLiB2cGNpDQo+DQo+
PiAtLS0gYS94ZW4vZHJpdmVycy92cGNpL3ZwY2kuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvdnBj
aS92cGNpLmMNCj4+IEBAIC05MSwyMCArOTEsMzIgQEAgaW50IF9faHdkb21faW5pdCB2cGNpX2Fk
ZF9oYW5kbGVycyhzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+ICAgLyogTm90aWZ5IHZQQ0kgdGhh
dCBkZXZpY2UgaXMgYXNzaWduZWQgdG8gZ3Vlc3QuICovDQo+PiAgIGludCB2cGNpX2Fzc2lnbl9k
ZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0IHBjaV9kZXYgKmRldikNCj4+ICAg
ew0KPj4gKyAgICBpbnQgcmM7DQo+PiArDQo+PiAgICAgICAvKiBJdCBvbmx5IG1ha2VzIHNlbnNl
IHRvIGFzc2lnbiBmb3IgaHdkb20gb3IgZ3Vlc3QgZG9tYWluLiAqLw0KPj4gICAgICAgaWYgKCBp
c19zeXN0ZW1fZG9tYWluKGQpIHx8ICFoYXNfdnBjaShkKSApDQo+PiAgICAgICAgICAgcmV0dXJu
IDA7DQo+PiAgIA0KPj4gLSAgICByZXR1cm4gdnBjaV9iYXJfYWRkX2hhbmRsZXJzKGQsIGRldik7
DQo+PiArICAgIHJjID0gdnBjaV9iYXJfYWRkX2hhbmRsZXJzKGQsIGRldik7DQo+PiArICAgIGlm
ICggcmMgKQ0KPj4gKyAgICAgICAgcmV0dXJuIHJjOw0KPj4gKw0KPj4gKyAgICByZXR1cm4gcGNp
X2FkZF92aXJ0dWFsX2RldmljZShkLCBkZXYpOw0KPj4gICB9DQo+IEkndmUgcGVla2VkIGF0IHRo
ZSBlYXJsaWVyIHBhdGNoLCBhbmQgYm90aCB0aGVyZSBhbmQgaGVyZSBJJ20gc3RydWdnbGluZyB0
bw0KPiBzZWUgaG93IHVuZG9pbmcgcGFydGlhbGx5IGNvbXBsZXRlZCBzdGVwcyBvciBmdWxseSBj
b21wbGV0ZWQgZWFybGllciBzdGVwcw0KPiBpcyBpbnRlbmRlZCB0byB3b3JrLiBJJ20gbm90IGNv
bnZpbmNlZCBpdCBpcyBsZWdpdGltYXRlIHRvIGxlYXZlIGhhbmRsZXJzDQo+IGluIHBsYWNlIHVu
dGlsIHRoZSB0b29sIHN0YWNrIG1hbmFnZXMgdG8gcm9sbCBiYWNrIHRoZSBmYWlsZWQgZGV2aWNl
DQo+IGFzc2lnbm1lbnQuDQpJJ2xsIHNlZSB3aGF0IGFuZCBob3cgd2UgY2FuIHJvbGwgYmFjayBp
biBjYXNlIG9mIGVycm9yDQo+DQo+PiAgIC8qIE5vdGlmeSB2UENJIHRoYXQgZGV2aWNlIGlzIGRl
LWFzc2lnbmVkIGZyb20gZ3Vlc3QuICovDQo+PiAgIGludCB2cGNpX2RlYXNzaWduX2RldmljZShz
dHJ1Y3QgZG9tYWluICpkLCBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqZGV2KQ0KPj4gICB7DQo+PiAr
ICAgIGludCByYzsNCj4+ICsNCj4+ICAgICAgIC8qIEl0IG9ubHkgbWFrZXMgc2Vuc2UgdG8gZGUt
YXNzaWduIGZyb20gaHdkb20gb3IgZ3Vlc3QgZG9tYWluLiAqLw0KPj4gICAgICAgaWYgKCBpc19z
eXN0ZW1fZG9tYWluKGQpIHx8ICFoYXNfdnBjaShkKSApDQo+PiAgICAgICAgICAgcmV0dXJuIDA7
DQo+PiAgIA0KPj4gKyAgICByYyA9IHBjaV9yZW1vdmVfdmlydHVhbF9kZXZpY2UoZCwgZGV2KTsN
Cj4+ICsgICAgaWYgKCByYyApDQo+PiArICAgICAgICByZXR1cm4gcmM7DQo+PiArDQo+PiAgICAg
ICByZXR1cm4gdnBjaV9iYXJfcmVtb3ZlX2hhbmRsZXJzKGQsIGRldik7DQo+PiAgIH0NCj4gU28g
d2hhdCdzIHRoZSB1bHRpbWF0ZSBlZmZlY3Qgb2YgYSBwYXJ0aWFsbHkgZGUtYXNzaWduZWQgZGV2
aWNlLCB3aGVyZQ0KPiBvbmUgb2YgdGhlIGxhdGVyIHN0ZXBzIGZhaWxlZD8gSW4gYSBudW1iZXIg
b2YgcGxhY2VzIHdlIGRvIGJlc3QtZWZmb3J0DQo+IGZ1bGwgY2xlYW51cCwgYnkgcmVjb3JkaW5n
IGVycm9ycyBidXQgbmV2ZXJ0aGVsZXNzIGNvbnRpbnVpbmcgd2l0aA0KPiBzdWJzZXF1ZW50IGNs
ZWFudXAgc3RlcHMuIEkgd29uZGVyIHdoZXRoZXIgdGhhdCdzIGEgbW9kZWwgdG8gdXNlIGhlcmUN
Cj4gYXMgd2VsbC4NCj4NCj4+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9wY2kuaA0KPj4gKysrIGIv
eGVuL2luY2x1ZGUveGVuL3BjaS5oDQo+PiBAQCAtMTM3LDYgKzEzNywyNCBAQCBzdHJ1Y3QgcGNp
X2RldiB7DQo+PiAgICAgICBzdHJ1Y3QgdnBjaSAqdnBjaTsNCj4+ICAgfTsNCj4+ICAgDQo+PiAr
I2lmZGVmIENPTkZJR19IQVNfVlBDSV9HVUVTVF9TVVBQT1JUDQo+PiArc3RydWN0IHZwY2lfZGV2
IHsNCj4+ICsgICAgc3RydWN0IGxpc3RfaGVhZCBsaXN0Ow0KPj4gKyAgICAvKiBQaHlzaWNhbCBQ
Q0kgZGV2aWNlIHRoaXMgdmlydHVhbCBkZXZpY2UgaXMgY29ubmVjdGVkIHRvLiAqLw0KPj4gKyAg
ICBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldjsNCj4+ICsgICAgLyogVmlydHVhbCBTQkRGIG9m
IHRoZSBkZXZpY2UuICovDQo+PiArICAgIHVuaW9uIHsNCj4+ICsgICAgICAgIHN0cnVjdCB7DQo+
PiArICAgICAgICAgICAgdWludDhfdCBkZXZmbjsNCj4+ICsgICAgICAgICAgICB1aW50OF90IGJ1
czsNCj4+ICsgICAgICAgICAgICB1aW50MTZfdCBzZWc7DQo+PiArICAgICAgICB9Ow0KPj4gKyAg
ICAgICAgcGNpX3NiZGZfdCBzYmRmOw0KPiBDb3VsZCB5b3UgZXhwbGFpbiB0byBtZSB3aHkgcGNp
X3NiZGZfdCAoYSB0eXBlZGVmIG9mIGEgdW5pb24pIGlzbid0DQo+IHByb3ZpZGluZyBhbGwgeW91
IG5lZWQ/IEJ5IHB1dHRpbmcgaXQgaW4gYSB1bmlvbiB3aXRoIGEgY3VzdG9tDQo+IHN0cnVjdCB5
b3Ugc2V0IHlvdXJzZWxmIHVwIGZvciB0aGluZ3MgZ29pbmcgb3V0IG9mIHN5bmMgaWYgYW55b25l
DQo+IGNob3NlIHRvIGFsdGVyIHBjaV9zYmRmX3QncyBsYXlvdXQuDQpTdXJlLCBwY2lfc2JkZl90
IHNob3VsZCBiZSBlbm91Z2gNCj4NCj4+IEBAIC0xNjcsNiArMTg1LDEwIEBAIGNvbnN0IHVuc2ln
bmVkIGxvbmcgKnBjaV9nZXRfcm9fbWFwKHUxNiBzZWcpOw0KPj4gICBpbnQgcGNpX2FkZF9kZXZp
Y2UodTE2IHNlZywgdTggYnVzLCB1OCBkZXZmbiwNCj4+ICAgICAgICAgICAgICAgICAgICAgIGNv
bnN0IHN0cnVjdCBwY2lfZGV2X2luZm8gKiwgbm9kZWlkX3Qgbm9kZSk7DQo+PiAgIGludCBwY2lf
cmVtb3ZlX2RldmljZSh1MTYgc2VnLCB1OCBidXMsIHU4IGRldmZuKTsNCj4+ICsjaWZkZWYgQ09O
RklHX0hBU19WUENJX0dVRVNUX1NVUFBPUlQNCj4+ICtpbnQgcGNpX2FkZF92aXJ0dWFsX2Rldmlj
ZShzdHJ1Y3QgZG9tYWluICpkLCBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldik7DQo+PiAraW50
IHBjaV9yZW1vdmVfdmlydHVhbF9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0
IHBjaV9kZXYgKnBkZXYpOw0KPj4gKyNlbmRpZg0KPiBMaWtlIGZvciB0aGVpciBkZWZpbml0aW9u
cyBJIHF1ZXN0aW9uIHRoZSBwbGFjZW1lbnQgb2YgdGhlc2UNCj4gZGVjbGFyYXRpb25zLg0KV2ls
bCBtb3ZlIHRvIHZwY2kuaA0KPg0KPj4gLS0tIGEveGVuL2luY2x1ZGUveGVuL3NjaGVkLmgNCj4+
ICsrKyBiL3hlbi9pbmNsdWRlL3hlbi9zY2hlZC5oDQo+PiBAQCAtNDQ0LDYgKzQ0NCwxNCBAQCBz
dHJ1Y3QgZG9tYWluDQo+PiAgIA0KPj4gICAjaWZkZWYgQ09ORklHX0hBU19QQ0kNCj4+ICAgICAg
IHN0cnVjdCBsaXN0X2hlYWQgcGRldl9saXN0Ow0KPj4gKyNpZmRlZiBDT05GSUdfSEFTX1ZQQ0lf
R1VFU1RfU1VQUE9SVA0KPj4gKyAgICBzdHJ1Y3QgbGlzdF9oZWFkIHZkZXZfbGlzdDsNCj4+ICsg
ICAgLyoNCj4+ICsgICAgICogQ3VycmVudCBkZXZpY2UgbnVtYmVyIHVzZWQgYnkgdGhlIHZpcnR1
YWwgUENJIGJ1cyB0b3BvbG9neQ0KPj4gKyAgICAgKiB0byBhc3NpZ24gYSB1bmlxdWUgU0JERiB0
byBhIHBhc3NlZCB0aHJvdWdoIHZpcnR1YWwgUENJIGRldmljZS4NCj4+ICsgICAgICovDQo+PiAr
ICAgIGludCB2cGNpX2Rldl9uZXh0Ow0KPiBJbiBob3cgZmFyIGNhbiB0aGUgbnVtYmVyIHN0b3Jl
ZCBoZXJlIGJlIG5lZ2F0aXZlPyBJZiBpdCBjYW4ndCBiZSwNCj4gcGxlYXNlIHVzZSB1bnNpZ25l
ZCBpbnQuDQpXaWxsIHVzZSB1bnNpZ25lZA0KPg0KPiBBcyB0byB0aGUgY29tbWVudCAtICJjdXJy
ZW50IiBpcyBhbWJpZ3VvdXM6IElzIGl0IHRoZSBudW1iZXIgdGhhdA0KPiB3YXMgdXNlZCBsYXN0
LCBvciB0aGUgbmV4dCBvbmUgdG8gYmUgdXNlZD8NCkkgd2lsbCB1cGRhdGUgdGhlIGNvbW1lbnQg
dG8gcmVtb3ZlIGFtYmlndWl0eQ0KPiBKYW4NCj4NClRoYW5rIHlvdSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:35:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:35:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199673.353873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsTs-0001lc-EJ; Thu, 30 Sep 2021 09:35:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199673.353873; Thu, 30 Sep 2021 09:35:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsTs-0001lT-B9; Thu, 30 Sep 2021 09:35:56 +0000
Received: by outflank-mailman (input) for mailman id 199673;
 Thu, 30 Sep 2021 09:35:55 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVsTr-0001lL-AR
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:35:55 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id ceb93386-21d1-11ec-bd40-12813bfff9fa;
 Thu, 30 Sep 2021 09:35:54 +0000 (UTC)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U9S86m012929; 
 Thu, 30 Sep 2021 09:35:52 GMT
Received: from eur02-he1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bdabvr4j0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 09:35:51 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM0PR03MB3716.eurprd03.prod.outlook.com (2603:10a6:208:4a::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 09:35:49 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 09:35:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ceb93386-21d1-11ec-bd40-12813bfff9fa
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Oih6JFIp34hv5EaeL0o3//CggnhNRh9qYvnIbcHckTOVWS/X1ErcZbsNmKhmOTqYRJFiRvWLH15OlPVOgQgay8ZSEDoOWeEyL6DCGIfryxhlDCi4TUfDUgqTkDcVcuRw6/hSzgUOsv7U3Ke9SfLYCiDzMH+/F2wxPzviRLOkFhWdvYMDnaxP1J6ma2cwq3TQNJHXubFAapSxvOPzm0nbTclZxLXEp2aXQ1xCijLuxRc1RvH2jrPeSOhKJBpND5Qo28ddtrQBfLp7wl47A4jk6qdMO2n1o/42nwRbCSxIiZYhmObZE+4tUtc9/Jm6ZhoPjrYFuyJ6ATI3jpjKg7+NWg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=2uuNPe+aR9cXUgQhOz+22Lww5ecJCpAhQ1R6sKi28MM=;
 b=AJMylZU9Kk5FY/fw8/KPMBdx7fvmBzEFrUbzxU98+PiELg+jof7OfUGLDNW9O2Ry7/9eDg1WWnN/Y/x8ooh+cin/jt7mqpMBwVgx3kMnjAKsw7Wss8B40o2zW9ETh6WmotP9Lainiaj0RLjdBfw9NNIYAdveNYH/FpF6rpUr4EeInUpIimNeXjdjIBk5MvfGk7WvwvetpeLmdON/zg3njDRVq/Ou7GQJztfBIYCpg8+oM7tD5vli7vU4yNXuHfgmJ4UZQ2etUpKL3Hzrqs8M3meYYnghxgUKxFpaLdvKEAdr2OIFXOOHEKtRo7UnDoDjSDyndrNkK8PLt65zi2n/HQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2uuNPe+aR9cXUgQhOz+22Lww5ecJCpAhQ1R6sKi28MM=;
 b=4+tiiNEDP5xGc1kD0U2dI6cwt9nxoWxzzwpC+3crhubGcxMA16ZXllY+cyBRDb6LkDmb3XjFM1jOlUc8NIqyqOOsdwKOhH2MDi+XBllGvSZnc9606UPvRazkorC1hs+rVPOKTNGjzcoyACoW7VlPaV7NBKndV6K8Zoj3e0c4OM8z3k/PphnLw8WPspBMouYl23BlduJPHtVPHy2vnXKr/35nMQqpwl01AlKx8k8JeKgyNpLg+Ne51A9kWWeV3ZOBzdi/BR33eVlZgiGRq3OVcHVTKPBneQjJ9qftcZm4uEG/+JyfdG6TDEuay+Xrg94G36u0hjzVSs5WXcfc8ot+uQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: AQHXtdAlBwDHpaYUBUq6q0PlsxoTp6u8Ra2AgAALzYA=
Date: Thu, 30 Sep 2021 09:35:49 +0000
Message-ID: <fcbb082c-d8ee-b59c-64b9-129b20308778@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-12-andr2000@gmail.com>
 <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
In-Reply-To: <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 15755b13-46fe-4712-1c83-08d983f5b043
x-ms-traffictypediagnostic: AM0PR03MB3716:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM0PR03MB3716DD7C469CF2D58685A357E7AA9@AM0PR03MB3716.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 ghPjdqnj+BXU5YJ47PgqxjcFj0Y050wvwjZ4QW3ixmDaHpl1KTTkJRBn8FOZpsSd1NhD4QXabSmBDCNemDPhHZ42JkvJ3Z2F7/wcJ1LCMl2tHAXX8im6llaFUeGDFPT6esn4exd0qbA1Hy23FoN7xw07PYRDs8rqvRh8OBlGX+8NL/eLR9Ty5fZKqBcGooMz+kYGy9lhKjl20vpUT1NGQewz5mtwJ1jkPb9appl4hRb0YPfOJUKEi6NK8+r9/QSKhchpU+ZlrSpSoqB0/YYnxZcSPChPCAQKj3ESu6Kpv1zbVJaNDzmRUDeHUVBmAqr2yYvtOcNm8g7N7tOVAatKKBUgzzaijtMu4utUQreXeG/XTJFWi41eGyJgr/50HmQs+L+jP1L/EGUM06D+CCiPM3eHa0m3mrHjfeBd9QTrvycfTweGfwK2KmPzTXOaXJb8CGct4XQP6tC9Fss/uuwy98kPpdMe7bBDuyMDsU9A6jsSaDnlf6NbEIDdMeUh6oDirus2MFrfx/UEVmWb5vlAVNXjGS99Zd51uvfxlxuupUXGTR8iXNGJwmkbSvVkp4FJ8kCD2aD9+mLbU3rVgmlgbQkF4qzYPOgae4qRJ8YRxlJMxhpszCGgXSxqrnCI2/GgtaYxc+DmEBKmCBJjuZsC9umz64FUHTulJt2wlpRXmesUUU+nxfRk9oR1JJuyibZJHemktAMiT+lMC+jEi++Ep5hS/CLnf1Nb6IMbIrPXhcw=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66946007)(6916009)(38100700002)(91956017)(83380400001)(6486002)(76116006)(86362001)(2906002)(5660300002)(6512007)(122000001)(54906003)(66476007)(64756008)(66556008)(66446008)(31696002)(316002)(71200400001)(53546011)(38070700005)(186003)(26005)(8936002)(55236004)(4326008)(6506007)(36756003)(31686004)(2616005)(508600001)(8676002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?bU00YWlMampxU2toTzhsaUZUT0UvMWc0QVVmYmVRelpEVnVGVTJCNDJDMmxS?=
 =?utf-8?B?NGdQNUxtR1U0OXRDVzNGREs0akdHcmQvU3dSMm51bFZsSUFMeEpwOHAza05r?=
 =?utf-8?B?cHc3SXdPa2p6YlQvbTU0VjRhVWI5dzdkR2NTUUJCdjNJWitXb2p4VzVRUk0z?=
 =?utf-8?B?b0Y2UU10Rm01RWVreXVLY1Y4b3ZuSmI0SzlyN25YK2duVUw0SCtudGF4d0FK?=
 =?utf-8?B?V2RyMCtZOHFYNThwaDVWbnZUdWNlY1VLY0hueGxjYzRiN0IxaS9oRFFRMTFj?=
 =?utf-8?B?eHNYSFZlQm5iUnFLZ2Y5UCtrWVlnNENQTzZBem80c1VZUVJETExsTXJld0s4?=
 =?utf-8?B?eVhtUkFCUmxmWWsyMlJ4UlU4VTlrQ2pBbEt1Vk5LbG10ajMzV2dDbWxTN1ht?=
 =?utf-8?B?QmdhZVBSZTVSQXdXTVRPWnd5YnNKK09SWkw3UC9NcThlZ0gwZkJBb0JKdHVO?=
 =?utf-8?B?UkxsSGtvR2dVMGNCWjRvamtPQ1FaemVHUVA1UnAzV2swUjdESFVoVFUrcVNx?=
 =?utf-8?B?TUxkZ1F4OFFqUE9uNFd0SFZ5NzhnSWxWMGRPRzlyUW03K3RIMUlqSWIwV3g1?=
 =?utf-8?B?dGh1SzEwd09XSlFtVHdjSVJLaE1tc3FMMXVWcWltV2lvcHhHelVFeThKM0tn?=
 =?utf-8?B?U0VtS3BNbWZBbGtFZGtqUG1IdmErVEdUM0lKbDdwTStJV0tpVFMwZFdWQlJR?=
 =?utf-8?B?ZDVwOWxiajNVOHhpM2dHc0tNdVd4SFlidTFKRUJTRSt2OGNrSmpyQVhRbndz?=
 =?utf-8?B?bEU0cjh3elNDZlFJbUFqQ0pJa2FOOEJJQnRnWlhXajVPa1VMeVF4WGRFYkJK?=
 =?utf-8?B?Wm1ERGZ0UWRNbGRzMXEvMnNuTlQwVS8rMm53YUZOWC9MdmxBZi9xakhJZCtE?=
 =?utf-8?B?RW1ST2VsUXljMnY5SlNLci9LS1QyRXVmcE9jQmNNc0RxUEdpYnp0VDl4aWZU?=
 =?utf-8?B?QWV3SzU2MWdRSGFtWFVIQ1JsRlFYMitsL2xoSDR3eVY0OWRHc0hhcGtCN2ZC?=
 =?utf-8?B?cnMyTUJhL0pHY1ZZaE1wNDR1MXBuN2NvdEVjNmduNEJnK1VMSERQRjRqRUVo?=
 =?utf-8?B?MlpPNUdDZDAvNXNCUU91L2ZlWExDWWZTY0k5SzRIMDNaZmRkenZ2TDdwbzFP?=
 =?utf-8?B?cFZwWjZGamJ4SVFFRjAvT0hrWFdBRjhJQllHS2VRbFZvdkJzTDFtRGhXN0hX?=
 =?utf-8?B?YVdacTFSemgveEdJcEhWNldIRXJLb3EweCtqRm53dWNZTk5NZnJ2eTExUm5M?=
 =?utf-8?B?QXFkcTNpc3JweGRkWGNLdHpQK1piQWpIUTdFMVgvTk5ucXNYNXFKZHNNT2RP?=
 =?utf-8?B?aW96c1Nsb2IyUkU2NUlobDVMMWVIUVhnQ1ZjTktJaEh6bUdsWVppakI5YXB5?=
 =?utf-8?B?Q3MxZ3YzSkg1QWVRMG1mRk9rVEcxZlozd3pOZmp4RUsxZnEzZ2NYZ2pWbVRL?=
 =?utf-8?B?YlJaQUk4cHAvdDBkNTVwQkVFSVdEaUJTa0lvNEttVkF6RTVhUlZmZHBWWWp5?=
 =?utf-8?B?OVkybFpEMkJwOTdWcHA4dmYyZElTSkNNM2pzek0rUVJhclMxOUJwUUZsZ0tP?=
 =?utf-8?B?bkRpVEZFZFQ1V2pmRG5nUHpUbGZNbUNQSlVSSTIzN3l6d1prZFZ2UUtZWGxS?=
 =?utf-8?B?cy9EbTJmQ1MzY1ZXOGZWWGl0dzg3d2R0RDNHdjVtY1RGMXR1bGI4MVlrNWVn?=
 =?utf-8?B?NWJIenZINWNnaFptYTlIbE9ZZGdCRzFRc0QyM0VmalgxU2lMdy9FTHRXT0N4?=
 =?utf-8?B?dFVWeGV1TlJxcU9Gdll0SjAyc2NvMUVCSGdSS0JpUmRDUENXNVVlbXRJWFZy?=
 =?utf-8?B?aXRIQ1NLRTJLUWZTR3VSUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C19BEF0282801546844731074AA0C6A6@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 15755b13-46fe-4712-1c83-08d983f5b043
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 09:35:49.6263
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: op8GC7cCNZ4d64OWpc8cgT6J3wNHPUpK9bfbnw7gT8Q8Y4h7Z/OIXZpKd7AnfexmuJSQaByi9VVYgA5XfHqo7YMC9RGeyp0ARAcE6zAx5V1IfdTFTcGFmw9gM8B9SsOD
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR03MB3716
X-Proofpoint-GUID: JkhGZGzoD4hYlJgnDmudyQOimhuiypUU
X-Proofpoint-ORIG-GUID: JkhGZGzoD4hYlJgnDmudyQOimhuiypUU
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_03,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0
 mlxscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501
 suspectscore=0 bulkscore=0 clxscore=1015 phishscore=0 mlxlogscore=932
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300058

DQoNCk9uIDMwLjA5LjIxIDExOjUzLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAwOTo1MiwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiAtLS0gYS94ZW4vZHJp
dmVycy9wYXNzdGhyb3VnaC9wY2kuYw0KPj4gKysrIGIveGVuL2RyaXZlcnMvcGFzc3Rocm91Z2gv
cGNpLmMNCj4+IEBAIC04ODksNiArODg5LDMxIEBAIGludCBwY2lfcmVtb3ZlX3ZpcnR1YWxfZGV2
aWNlKHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4gICAg
ICAgeGZyZWUodmRldik7DQo+PiAgICAgICByZXR1cm4gMDsNCj4+ICAgfQ0KPj4gKw0KPj4gKy8q
DQo+PiArICogRmluZCB0aGUgcGh5c2ljYWwgZGV2aWNlIHdoaWNoIGlzIG1hcHBlZCB0byB0aGUg
dmlydHVhbCBkZXZpY2UNCj4+ICsgKiBhbmQgdHJhbnNsYXRlIHZpcnR1YWwgU0JERiB0byB0aGUg
cGh5c2ljYWwgb25lLg0KPj4gKyAqLw0KPj4gK2Jvb2wgcGNpX3RyYW5zbGF0ZV92aXJ0dWFsX2Rl
dmljZShjb25zdCBzdHJ1Y3QgZG9tYWluICpkLCBwY2lfc2JkZl90ICpzYmRmKQ0KPj4gK3sNCj4+
ICsgICAgc3RydWN0IHZwY2lfZGV2ICp2ZGV2Ow0KPiBjb25zdCAoYWZhaWN0KQ0KT2sNCj4NCj4+
ICsgICAgYm9vbCBmb3VuZCA9IGZhbHNlOw0KPj4gKw0KPj4gKyAgICBwY2lkZXZzX2xvY2soKTsN
Cj4+ICsgICAgbGlzdF9mb3JfZWFjaF9lbnRyeSAoIHZkZXYsICZkLT52ZGV2X2xpc3QsIGxpc3Qg
KQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBpZiAoIHZkZXYtPnNiZGYuc2JkZiA9PSBzYmRmLT5z
YmRmICkNCj4+ICsgICAgICAgIHsNCj4+ICsgICAgICAgICAgICAvKiBSZXBsYWNlIHZpcnR1YWwg
U0JERiB3aXRoIHRoZSBwaHlzaWNhbCBvbmUuICovDQo+PiArICAgICAgICAgICAgKnNiZGYgPSB2
ZGV2LT5wZGV2LT5zYmRmOw0KPj4gKyAgICAgICAgICAgIGZvdW5kID0gdHJ1ZTsNCj4+ICsgICAg
ICAgICAgICBicmVhazsNCj4+ICsgICAgICAgIH0NCj4+ICsgICAgfQ0KPj4gKyAgICBwY2lkZXZz
X3VubG9jaygpOw0KPiBBcyBwZXIgdGhlIGNvbW1lbnRzIG9uIHRoZSBlYXJsaWVyIHBhdGNoLCBs
b2NraW5nIGFzIHdlbGwgYXMgcGxhY2VtZW50DQo+IG1heSBuZWVkIHJlY29uc2lkZXJpbmcuDQpP
dGhlciB0aGVuIHRoYXQgZG8geW91IGhhdmUgb3RoZXIgY29tbWVudHMgb24gdGhpcz8NCj4NCj4g
SmFuDQo+DQoNClRoYW5rIHlvdSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:42:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:42:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199681.353883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsZj-0003Me-0s; Thu, 30 Sep 2021 09:41:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199681.353883; Thu, 30 Sep 2021 09:41:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsZi-0003MX-UD; Thu, 30 Sep 2021 09:41:58 +0000
Received: by outflank-mailman (input) for mailman id 199681;
 Thu, 30 Sep 2021 09:41:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsZi-0003MN-AM; Thu, 30 Sep 2021 09:41:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsZi-0004w9-3K; Thu, 30 Sep 2021 09:41:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsZh-00032A-O4; Thu, 30 Sep 2021 09:41:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVsZh-0002oJ-NZ; Thu, 30 Sep 2021 09:41:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=18vbxodbOI3R+hSOUZ+mbAegZRFD+FXgfyM0eqNXIeA=; b=nAzB4/pRwFE0PbOwgRYpy8lYiz
	aYWbS4urD10v6HZf5+b/1qBDnvUy+J0htmfK8V8bzbi40Ry8iqsZDYysAZ7PL748pEv3wLuzwd8Db
	iZkv/aShVPCpKPDkw8pC0svFaeXDIVI5DatnOMfMeD979+qFmOc2Cpk5eCWJABcYNj4k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165325-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 165325: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-armhf-libvirt:libvirt-build:fail:regression
    libvirt:build-amd64-libvirt:libvirt-build:fail:regression
    libvirt:build-i386-libvirt:libvirt-build:fail:regression
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:build-check(1):blocked:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-pair:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    libvirt=f0580a9301806edb23aca38c8f7b664cae184539
X-Osstest-Versions-That:
    libvirt=2c846fa6bcc11929c9fb857a22430fb9945654ad
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 09:41:57 +0000

flight 165325 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/165325/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-amd64-libvirt           6 libvirt-build            fail REGR. vs. 151777
 build-i386-libvirt            6 libvirt-build            fail REGR. vs. 151777
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt       1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)               blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt      1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)               blocked  n/a

version targeted for testing:
 libvirt              f0580a9301806edb23aca38c8f7b664cae184539
baseline version:
 libvirt              2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  447 days
Failing since        151818  2020-07-11 04:18:52 Z  446 days  436 attempts
Testing same since   165325  2021-09-30 04:20:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
    Adolfo Jayme Barrientos <fitoschido@gmail.com>
  Aleksandr Alekseev <alexander.alekseev@virtuozzo.com>
  Aleksei Zakharov <zaharov@selectel.ru>
  Andika Triwidada <andika@gmail.com>
  Andrea Bolognani <abologna@redhat.com>
  Balázs Meskó <meskobalazs@mailbox.org>
  Barrett Schonefeld <bschoney@utexas.edu>
  Bastian Germann <bastiangermann@fishpost.de>
  Bastien Orivel <bastien.orivel@diateam.net>
  BiaoXiang Ye <yebiaoxiang@huawei.com>
  Bihong Yu <yubihong@huawei.com>
  Binfeng Wu <wubinfeng@huawei.com>
  Bjoern Walk <bwalk@linux.ibm.com>
  Boris Fiuczynski <fiuczy@linux.ibm.com>
  Brian Turek <brian.turek@gmail.com>
  Bruno Haible <bruno@clisp.org>
  Chris Mayo <aklhfex@gmail.com>
  Christian Borntraeger <borntraeger@de.ibm.com>
  Christian Ehrhardt <christian.ehrhardt@canonical.com>
  Christian Kirbach <christian.kirbach@gmail.com>
  Christian Schoenebeck <qemu_oss@crudebyte.com>
  Cole Robinson <crobinso@redhat.com>
  Collin Walling <walling@linux.ibm.com>
  Cornelia Huck <cohuck@redhat.com>
  Cédric Bosdonnat <cbosdonnat@suse.com>
  Côme Borsoi <fedora@borsoi.fr>
  Daniel Henrique Barboza <danielhb413@gmail.com>
  Daniel Letai <dani@letai.org.il>
  Daniel P. Berrange <berrange@redhat.com>
  Daniel P. Berrangé <berrange@redhat.com>
  Didik Supriadi <didiksupriadi41@gmail.com>
  dinglimin <dinglimin@cmss.chinamobile.com>
  Dmytro Linkin <dlinkin@nvidia.com>
  Eiichi Tsukata <eiichi.tsukata@nutanix.com>
  Eric Farman <farman@linux.ibm.com>
  Erik Skultety <eskultet@redhat.com>
  Fabian Affolter <mail@fabian-affolter.ch>
  Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
  Fabiano Fidêncio <fabiano@fidencio.org>
  Fangge Jin <fjin@redhat.com>
  Farhan Ali <alifm@linux.ibm.com>
  Fedora Weblate Translation <i18n@lists.fedoraproject.org>
  gongwei <gongwei@smartx.com>
  Guoyi Tu<tu.guoyi@h3c.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Halil Pasic <pasic@linux.ibm.com>
  Han Han <hhan@redhat.com>
  Hao Wang <wanghao232@huawei.com>
  Hela Basa <r45xveza@pm.me>
  Helmut Grohne <helmut@subdivi.de>
  Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
  Ian Wienand <iwienand@redhat.com>
  Jakob Meng <jakobmeng@web.de>
  Jamie Strandboge <jamie@canonical.com>
  Jamie Strandboge <jamie@ubuntu.com>
  Jan Kuparinen <copper_fin@hotmail.com>
  jason lee <ppark5237@gmail.com>
  Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr>
  Jia Zhou <zhou.jia2@zte.com.cn>
  Jianan Gao <jgao@redhat.com>
  Jim Fehlig <jfehlig@suse.com>
  Jin Yan <jinyan12@huawei.com>
  Jinsheng Zhang <zhangjl02@inspur.com>
  Jiri Denemark <jdenemar@redhat.com>
  John Ferlan <jferlan@redhat.com>
  Jonathan Watt <jwatt@jwatt.org>
  Jonathon Jongsma <jjongsma@redhat.com>
  Julio Faracco <jcfaracco@gmail.com>
  Justin Gatzen <justin.gatzen@gmail.com>
  Ján Tomko <jtomko@redhat.com>
  Kashyap Chamarthy <kchamart@redhat.com>
  Kevin Locke <kevin@kevinlocke.name>
  Kristina Hanicova <khanicov@redhat.com>
  Laine Stump <laine@redhat.com>
  Laszlo Ersek <lersek@redhat.com>
  Lee Yarwood <lyarwood@redhat.com>
  Lei Yang <yanglei209@huawei.com>
  Liao Pingfang <liao.pingfang@zte.com.cn>
  Lin Ma <lma@suse.com>
  Lin Ma <lma@suse.de>
  Lin Ma <morecache@gmail.com>
  Liu Yiding <liuyd.fnst@fujitsu.com>
  Luke Yue <lukedyue@gmail.com>
  Luyao Zhong <luyao.zhong@intel.com>
  Marc Hartmayer <mhartmay@linux.ibm.com>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Markus Schade <markus.schade@hetzner.com>
  Martin Kletzander <mkletzan@redhat.com>
  Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
  Matej Cepl <mcepl@cepl.eu>
  Matt Coleman <matt@datto.com>
  Matt Coleman <mcoleman@datto.com>
  Mauro Matteo Cascella <mcascell@redhat.com>
  Meina Li <meili@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Michał Smyk <fedora@smyk.it>
  Milo Casagrande <milo@milo.name>
  Moshe Levi <moshele@nvidia.com>
  Muha Aliss <muhaaliss@gmail.com>
  Nathan <nathan95@live.it>
  Neal Gompa <ngompa13@gmail.com>
  Nick Chevsky <nchevsky@gmail.com>
  Nick Shyrokovskiy <nshyrokovskiy@gmail.com>
  Nickys Music Group <nickys.music.group@gmail.com>
  Nico Pache <npache@redhat.com>
  Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
  Olaf Hering <olaf@aepfle.de>
  Olesya Gerasimenko <gammaray@basealt.ru>
  Orion Poplawski <orion@nwra.com>
  Pany <geekpany@gmail.com>
  Patrick Magauran <patmagauran.j@gmail.com>
  Paulo de Rezende Pinatti <ppinatti@linux.ibm.com>
  Pavel Hrdina <phrdina@redhat.com>
  Peng Liang <liangpeng10@huawei.com>
  Peter Krempa <pkrempa@redhat.com>
  Pino Toscano <ptoscano@redhat.com>
  Pino Toscano <toscano.pino@tiscali.it>
  Piotr Drąg <piotrdrag@gmail.com>
  Prathamesh Chavan <pc44800@gmail.com>
  Richard W.M. Jones <rjones@redhat.com>
  Ricky Tigg <ricky.tigg@gmail.com>
  Robin Lee <cheeselee@fedoraproject.org>
  Roman Bogorodskiy <bogorodskiy@gmail.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Ryan Gahagan <rgahagan@cs.utexas.edu>
  Ryan Schmidt <git@ryandesign.com>
  Sam Hartman <hartmans@debian.org>
  Scott Shambarger <scott-libvirt@shambarger.net>
  Sebastian Mitterle <smitterl@redhat.com>
  SeongHyun Jo <caelus9536@gmail.com>
  Shalini Chellathurai Saroja <shalini@linux.ibm.com>
  Shaojun Yang <yangshaojun@phytium.com.cn>
  Shi Lei <shi_lei@massclouds.com>
  simmon <simmon@nplob.com>
  Simon Chopin <chopin.simon@gmail.com>
  Simon Gaiser <simon@invisiblethingslab.com>
  Simon Rowe <simon.rowe@nutanix.com>
  Stefan Bader <stefan.bader@canonical.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>
  Stefan Hajnoczi <stefanha@gmail.com>
  Stefan Hajnoczi <stefanha@redhat.com>
  Szymon Scholz <szymonscholz@gmail.com>
  Thomas Huth <thuth@redhat.com>
  Tim Wiederhake <twiederh@redhat.com>
  Tomáš Golembiovský <tgolembi@redhat.com>
  Tomáš Janoušek <tomi@nomi.cz>
  Tuguoyi <tu.guoyi@h3c.com>
  Victor Toso <victortoso@redhat.com>
  Ville Skyttä <ville.skytta@iki.fi>
  Vinayak Kale <vkale@nvidia.com>
  Wang Xin <wangxinxin.wang@huawei.com>
  WangJian <wangjian161@huawei.com>
  Weblate <noreply@weblate.org>
  Wei Liu <liuwe@microsoft.com>
  Wei Liu <wei.liu@kernel.org>
  William Douglas <william.douglas@intel.com>
  Yalei Li <274268859@qq.com>
  Yalei Li <liyl43@chinatelecom.cn>
  Yang Fei <yangfei85@huawei.com>
  Yang Hang <yanghang44@huawei.com>
  Yanqiu Zhang <yanqzhan@redhat.com>
  Yaroslav Kargin <ykargin@virtuozzo.com>
  Yi Li <yili@winhong.com>
  Yi Wang <wang.yi59@zte.com.cn>
  Yuri Chornoivan <yurchor@ukr.net>
  Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
  zhangjl02 <zhangjl02@inspur.com>
  Zheng Chuan <zhengchuan@huawei.com>
  zhenwei pi <pizhenwei@bytedance.com>
  Zhenyu Ye <yezhenyu2@huawei.com>
  Zhenyu Zheng <zheng.zhenyu@outlook.com>
  Zhenzhong Duan <zhenzhong.duan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          fail    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          fail    
 build-i386-libvirt                                           fail    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            blocked 
 test-amd64-amd64-libvirt-xsm                                 blocked 
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  blocked 
 test-amd64-amd64-libvirt                                     blocked 
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     blocked 
 test-amd64-i386-libvirt                                      blocked 
 test-amd64-amd64-libvirt-pair                                blocked 
 test-amd64-i386-libvirt-pair                                 blocked 
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               blocked 
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 blocked 
 test-amd64-i386-libvirt-raw                                  blocked 
 test-amd64-amd64-libvirt-vhd                                 blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 76261 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 09:43:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 09:43:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199688.353897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsbA-0003xH-DB; Thu, 30 Sep 2021 09:43:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199688.353897; Thu, 30 Sep 2021 09:43:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsbA-0003xA-AA; Thu, 30 Sep 2021 09:43:28 +0000
Received: by outflank-mailman (input) for mailman id 199688;
 Thu, 30 Sep 2021 09:43:27 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YInd=OU=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVsb9-0003wh-9h
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 09:43:27 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe0e::614])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 550c3535-fb4d-470f-95d9-d834d68aa1ba;
 Thu, 30 Sep 2021 09:43:25 +0000 (UTC)
Received: from AS8P250CA0006.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:330::11)
 by DB6PR0802MB2199.eurprd08.prod.outlook.com (2603:10a6:4:82::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 09:43:20 +0000
Received: from VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:330:cafe::bf) by AS8P250CA0006.outlook.office365.com
 (2603:10a6:20b:330::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 09:43:20 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT007.mail.protection.outlook.com (10.152.18.114) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 09:43:19 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 30 Sep 2021 09:43:19 +0000
Received: from 544f83ba2cb5.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 6F4DF077-0C78-4E01-AB13-5D50AB5A149C.1; 
 Thu, 30 Sep 2021 09:43:07 +0000
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 544f83ba2cb5.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 09:43:07 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PR3PR08MB5706.eurprd08.prod.outlook.com (2603:10a6:102:80::24)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 09:43:04 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4566.017; Thu, 30 Sep 2021
 09:43:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 550c3535-fb4d-470f-95d9-d834d68aa1ba
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+KOPRA8woA7GFpEEycwOKYeCiKXnsQNEbMEPlXO0pww=;
 b=Mhx7dB8aLPTrqT3jYKL1YbRmP+aY/cOwAVHjRMkMbYAOLvuZ4oj7IDlxVBictmkhaa+1pAXWkdspsZkQmuxQ8NpeXlYp8FXKDggBaIS5o+qAqWxLTQCiEfVjj++auaF0cPFSUyd4E2OR43Ddwd+vEO6EP9CuGS8T4CH/Nic6pog=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ad706858e98831c1
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=X3F4eW0gzz02pPthrk3IJF7VOpsW5zzJV4+R507WrpX2q64cqVRix7jUcVJlxKfMds1Yak72M5KfxorLqV38NzgeDpDYb/iAW/FSVlk2NGqvLPARUso3pobvPEXDpKEbT9gB7Be64crOQBdT3RLx8YM070adQZKqMq5/Z4wXmwqefNa7zrH6Qb62lDnW7xazwM0AVVp0AsHfREQ6OtMdYq3i9bu4qTkDr5AEy1qz9fAWissizk7CSISNs08vvAfxe/vtFr/9aX5b3QrnKxpgFWun/hd7nqjT3vunN3Q7gg+FupS9XOKJB5Gaj7LTarnUzrtF8b49Z0rB0mr3Inb69A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=+KOPRA8woA7GFpEEycwOKYeCiKXnsQNEbMEPlXO0pww=;
 b=RYWUzPk5k+KiUDhtXtEbW59WK2Yt0QrZlRDMBNWzO/qgJtyVlgtrDuCEWiQhl4apVUHTOB//hs0BVJalOrmeObRyNxHaCNctWB0wBxU1QpZsubwOthDA2jdaF4fXV0Lb2xx+7gZv9s3lDTOWWw9lETOl2RW9AIydjuNzVMCiPDnIxJBJgiKKDH0t0/UNl/LwmJi80GthLrnMJZzGi225iqJ4YVCDL73Xc6I+gnOfdZwR3PMz8pSaiDtLXLNXLc51agfGSp6ML8VI2MX5V1Q9Tf+ZMqlyXOrFztG5rsZzhPh8Yzu6APzRIkDYMJxRsaz/ef+YqyERmxUz+/oYtTNZWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=+KOPRA8woA7GFpEEycwOKYeCiKXnsQNEbMEPlXO0pww=;
 b=Mhx7dB8aLPTrqT3jYKL1YbRmP+aY/cOwAVHjRMkMbYAOLvuZ4oj7IDlxVBictmkhaa+1pAXWkdspsZkQmuxQ8NpeXlYp8FXKDggBaIS5o+qAqWxLTQCiEfVjj++auaF0cPFSUyd4E2OR43Ddwd+vEO6EP9CuGS8T4CH/Nic6pog=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Michal Orzel <Michal.Orzel@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson
	<iwj@xenproject.org>, Wei Liu <wl@xen.org>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH] xen/arm: Expose the PMU to the guests
Thread-Topic: [PATCH] xen/arm: Expose the PMU to the guests
Thread-Index: AQHXtd1dT9hAp9ZDCE6Sy/7vjqGAOqu8U2WA
Date: Thu, 30 Sep 2021 09:43:04 +0000
Message-ID: <50185DD3-12E0-419C-8B02-0486DB68F1C4@arm.com>
References: <20210930092651.1350-1-michal.orzel@arm.com>
In-Reply-To: <20210930092651.1350-1-michal.orzel@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 3fbf524c-418c-4cb6-24d3-08d983f6bc97
x-ms-traffictypediagnostic: PR3PR08MB5706:|DB6PR0802MB2199:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<DB6PR0802MB2199D2AD37740C7742A3AEC59DAA9@DB6PR0802MB2199.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 c8bDp/63TcvhRjMcTlVBZ8oRzcGF4b4XO7rbTFIy7uWOyo3TO/OzjlPa0pKwjmmqrYaUfqbWRuG3dDHZumYBjqxa4eephEqCZEHiLhM+05IeBYTmRkTTL+WV8JtVx6OH/B1z0jp53q1K07zpoG/G3zlcJ6pfjWtSxEqERXRwkD3AqLvvcef21V0xTHVCnZ8ZB/8r6UnA47znBcVM2ze2Y7sgE48NyIBRPFnT0pvxp5LkBmCXU9XIz5LPZXdGLAAu3wnXzMGG1F2ouwpiBilKeShSgIwgbg+GVZev0TANP+3kyA+Meipypi9+iXFgmdEvVo1xyQW3tElFC+YsDr3wim6NtLQEm8SFmyzWlj2Q1PvMvMtVUO/zvhJF9a3658cUsThZyM3oNUpaopd62UBJfld8whyoXimKRbe8Y2BJuH8oo5YpJX1PlfXVH9A4hryWR4BHACQGtH5sqBCghR+bCoguSeAp3mhjwVvdrrH7L76wt0bzCMry8qY4XFQdzMeEezjHWXGdh/7axYCfx9Pr108kI1/hpugF0sS5uB7bJnCU6QozaAkFXb6+XoLWrsqQ7MusBZ0T4wsHtakEFTsldQVH7VdiJ/RO/YZtQwW0IZhJL7v4kDUi1BdQi2+ScTLs1UhKAwDIezjSHLp/sdgYa4J7q0lIfwzQ3+A69eTXJDuMsqSDP944oSBWVHsZzhUqNmu4ySmml5hDrwfobab9OJwZrHB5nfwWaOzA08bk3ik4sjExYlORrmo/4aVRnjz+
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(66556008)(64756008)(66446008)(33656002)(66946007)(2906002)(91956017)(76116006)(71200400001)(83380400001)(38070700005)(86362001)(122000001)(38100700002)(2616005)(6506007)(186003)(6862004)(4326008)(6512007)(508600001)(37006003)(316002)(53546011)(26005)(54906003)(7416002)(36756003)(6636002)(5660300002)(8676002)(8936002)(6486002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <CE6797C851093B439673652C316818F4@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5706
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	8bff03ad-2531-4b3c-ddff-08d983f6b33f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7BvAGvCZXv/QA8mpJjWJ9/9CFcak97jCjY0dvQE6IWEm/zXSHwTUd6sUy72YStd6aR94J0e4RwKmP1aGkWkaCtibEjUvjNsYtQe6bnYmz3lYkGmoI1W0C1fvkqvtUk4frwta0vu692U3oFyaI+FKXUYGNyLsfkLIgLRIxURqxAGeno/W3H/w1xCq6KS83OgxJ9+wYBVSeLpGB0cu5W6ZF/T9IeLNL1FJiee1eUpm7IsxKZ59LUzAbZmc2IdimvZ3xV/bdc9ehjHIBidqLpO6MkLJ3ljZsg9NWKY1BSX+miaMKcFBpdYDsUG4YkbPoH8N5ldCMBrRHPPciPa5e7dE/2A2JV/tpSAcxay+G5cYDZjnCgZDjVvVyQOdCxpqQ+PntgyLhHsaiq5cIMY1c9ptaoohnmz292O2Zf9CY4q0FwFk+lEgvpCGFueNZLjGrieN2RuMF1jJvgvRvzCzhORrYIZ4a4XHk2rEDL+T4dQsqK4VppQnm5iXD1VDzxvPc0wMkJXQMwfHHbuASOAlY7XTVGE+ftoH4d31gzNAW9aH8HycqFQHuidvgD5OvFVbtcI+IcIU6Utfxidiaq9VNC93ZCNrp5XmNfYQYn9GvM8GVr0ThSWicKCocMCPPIVyc3+26Rn3g1xyeiQJ/BF/Fyp9ii6Zi129p3JTD7kewOZoKLdaQOKzAZxZ8xgOCxIUVSb4y7bBGZQfY4fP/RLFiMblZw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(356005)(81166007)(70206006)(8676002)(6486002)(26005)(70586007)(2906002)(2616005)(37006003)(6512007)(47076005)(8936002)(36860700001)(508600001)(6506007)(53546011)(6862004)(107886003)(83380400001)(36756003)(6636002)(86362001)(186003)(33656002)(5660300002)(82310400003)(336012)(4326008)(54906003)(316002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 09:43:19.7402
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3fbf524c-418c-4cb6-24d3-08d983f6bc97
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2199

Hi Michal,

> On 30 Sep 2021, at 10:26, Michal Orzel <Michal.Orzel@arm.com> wrote:
>=20
> Add parameter vpmu to xl domain configuration syntax
> to enable the access to PMU registers by disabling
> the PMU traps.
>=20
> This change does not expose the full PMU to the guest.
> Long term, we will want to at least expose the PMU
> interrupts, device-tree binding. For more use cases
> we will also need to save/restore the PMU context.
>=20
> Signed-off-by: Michal Orzel <michal.orzel@arm.com>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> docs/man/xl.cfg.5.pod.in         | 12 ++++++++++++
> tools/include/libxl.h            |  7 +++++++
> tools/libs/light/libxl_arm.c     |  6 +++++-
> tools/libs/light/libxl_types.idl |  1 +
> tools/xl/xl_parse.c              |  2 ++
> xen/arch/arm/domain.c            | 10 ++++++++--
> xen/common/domain.c              |  2 +-
> xen/include/asm-arm/domain.h     |  1 +
> xen/include/public/domctl.h      |  5 ++++-
> 9 files changed, 41 insertions(+), 5 deletions(-)
>=20
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 4b1e3028d2..4a75203b9f 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -2843,6 +2843,18 @@ Currently, only the "sbsa_uart" model is supported=
 for ARM.
>=20
> =3Dback
>=20
> +=3Ditem B<vpmu=3DBOOLEAN>
> +
> +Specifies whether to enable the access to PMU registers by disabling
> +the PMU traps.
> +
> +This change does not expose the full PMU to the guest.
> +Currently there is no support for virtualization, interrupts, etc.
> +
> +Enabling this option may result in potential security holes.
> +
> +If this option is not specified then it will default to B<false>.
> +
> =3Dhead3 x86
>=20
> =3Dover 4
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d698..c6694e977d 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -502,6 +502,13 @@
>  */
> #define LIBXL_HAVE_X86_MSR_RELAXED 1
>=20
> +/*
> + * LIBXL_HAVE_ARM_VPMU indicates the toolstack has support for enabling
> + * the access to PMU registers by disabling the PMU traps. This is done
> + * by setting the libxl_domain_build_info arch_arm.vpmu field.
> + */
> +#define LIBXL_HAVE_ARM_VPMU 1
> +
> /*
>  * libxl ABI compatibility
>  *
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index e3140a6e00..bc614dcb05 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -29,6 +29,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>     uint32_t vuart_irq;
>     bool vuart_enabled =3D false;
>=20
> +    if (libxl_defbool_val(d_config->b_info.arch_arm.vpmu))
> +        config->flags |=3D XEN_DOMCTL_CDF_pmu;
> +
>     /*
>      * If pl011 vuart is enabled then increment the nr_spis to allow allo=
cation
>      * of SPI VIRQ for pl011.
> @@ -1186,8 +1189,9 @@ void libxl__arch_domain_create_info_setdefault(libx=
l__gc *gc,
> void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
>                                               libxl_domain_build_info *b_=
info)
> {
> -    /* ACPI is disabled by default */
> +    /* ACPI and vPMU is disabled by default */
>     libxl_defbool_setdefault(&b_info->acpi, false);
> +    libxl_defbool_setdefault(&b_info->arch_arm.vpmu, false);
>=20
>     if (b_info->type !=3D LIBXL_DOMAIN_TYPE_PV)
>         return;
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_ty=
pes.idl
> index 3f9fff653a..3524629909 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -644,6 +644,7 @@ libxl_domain_build_info =3D Struct("domain_build_info=
",[
>=20
>     ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
>                                ("vuart", libxl_vuart_type),
> +                               ("vpmu", libxl_defbool),
>                               ])),
>     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                               ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 17dddb4cd5..e15c2e64f5 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -2750,6 +2750,8 @@ skip_usbdev:
>                     "If it fixes an issue you are having please report to=
 "
>                     "xen-devel@lists.xenproject.org.\n");
>=20
> +    xlu_cfg_get_defbool(config, "vpmu", &b_info->arch_arm.vpmu, 0);
> +
>     xlu_cfg_destroy(config);
> }
>=20
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 19c756ac3d..02120ff640 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -276,6 +276,8 @@ static void ctxt_switch_to(struct vcpu *n)
>      * timer. The interrupt needs to be injected into the guest. */
>     WRITE_SYSREG(n->arch.cntkctl, CNTKCTL_EL1);
>     virt_timer_restore(n);
> +
> +    WRITE_SYSREG(n->arch.mdcr_el2, MDCR_EL2);
> }
>=20
> /* Update per-VCPU guest runstate shared memory area (if registered). */
> @@ -586,6 +588,10 @@ int arch_vcpu_create(struct vcpu *v)
>=20
>     v->arch.hcr_el2 =3D get_default_hcr_flags();
>=20
> +    v->arch.mdcr_el2 =3D HDCR_TDRA | HDCR_TDOSA | HDCR_TDA;
> +    if ( !(v->domain->options & XEN_DOMCTL_CDF_pmu) )
> +        v->arch.mdcr_el2 |=3D HDCR_TPM | HDCR_TPMCR;
> +
>     if ( (rc =3D vcpu_vgic_init(v)) !=3D 0 )
>         goto fail;
>=20
> @@ -622,8 +628,8 @@ int arch_sanitise_domain_config(struct xen_domctl_cre=
atedomain *config)
> {
>     unsigned int max_vcpus;
>=20
> -    /* HVM and HAP must be set. IOMMU may or may not be */
> -    if ( (config->flags & ~XEN_DOMCTL_CDF_iommu) !=3D
> +    /* HVM and HAP must be set. IOMMU and PMU may or may not be */
> +    if ( (config->flags & ~(XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_pmu)) =
!=3D
>          (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
>     {
>         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n",
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6ee5d033b0..63c4db8b9f 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_c=
reatedomain *config)
>          ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> -           XEN_DOMCTL_CDF_nested_virt) )
> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_pmu) )
>     {
>         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>         return -EINVAL;
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c9277b5c6d..14e575288f 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -166,6 +166,7 @@ struct arch_vcpu
>=20
>     /* HYP configuration */
>     register_t hcr_el2;
> +    register_t mdcr_el2;
>=20
>     uint32_t teecr, teehbr; /* ThumbEE, 32-bit guests only */
> #ifdef CONFIG_ARM_32
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842..a55ceb81db 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -70,9 +70,12 @@ struct xen_domctl_createdomain {
> #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
> #define _XEN_DOMCTL_CDF_nested_virt   6
> #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
> +/* Should we expose the vPMU to the guest? */
> +#define _XEN_DOMCTL_CDF_pmu           7
> +#define XEN_DOMCTL_CDF_pmu            (1U << _XEN_DOMCTL_CDF_pmu)
>=20
> /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_pmu
>=20
>     uint32_t flags;
>=20
> --=20
> 2.29.0
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:02:33 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:02:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199696.353908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVstX-0006hI-4e; Thu, 30 Sep 2021 10:02:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199696.353908; Thu, 30 Sep 2021 10:02:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVstX-0006hB-1o; Thu, 30 Sep 2021 10:02:27 +0000
Received: by outflank-mailman (input) for mailman id 199696;
 Thu, 30 Sep 2021 10:02:25 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVstV-0006h5-1Z
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:02:25 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 69d11b6d-2b35-45d2-ae67-175a0499d76a;
 Thu, 30 Sep 2021 10:02:23 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-6-5KlF8yv8NpaL79WgfeLpHw-1; Thu, 30 Sep 2021 12:02:21 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7150.eurprd04.prod.outlook.com (2603:10a6:800:12a::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 30 Sep
 2021 10:02:20 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 10:02:20 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 FR2P281CA0013.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.10 via Frontend Transport; Thu, 30 Sep 2021 10:02:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69d11b6d-2b35-45d2-ae67-175a0499d76a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632996143;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=UjD7HxCfcb4pbywXa2JcJ2B7izfbj4CYQEkWazOF0kc=;
	b=d5eUPMut/mTQHVliV9thbJsKRiPrsE/AIN0je9eU2TqqzbmisM+yuMfeJToNCbM+auMBj/
	W0T/kuTHIhaaQonZc3xKsQlBz9mvU0RUQt39zMOtxYiSKLhP2Wny1gazUrMobsdn8MElxU
	N7sGFDnR9TN/lKwNjy5GVVIu+dxnVPs=
X-MC-Unique: 5KlF8yv8NpaL79WgfeLpHw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KTxULLFYfRsnAvrvBXSGQGZtJoBLb3JA+t/fvFjf/TgiM2ugxK1zTr3icFtufi/jNpcuKj3e42fAHljhJVvrL2W+Tr9JaEge36Os4VC0BUIGiM3QwH8TNDsTVmRS2kqwHhTPO/YlyYK5CbTyTaaQOnMBxElw3oNwZBx1EqtscNxLBOyIw1wUZe63+DmiLdEbZ9DThAqJcz7Bk9tsiAcKKo7apZm33UrlDO3/bac1+Hb7aECjxvbLrwg35dUnvsUkrGE61ChveB1VfJwbVKOhgl0TPyx/pufKkfoyiFfCcanSQMC0q4z2izU2J+UPiUWT29LipIlfhZCFJdtQtMsIlA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=UjD7HxCfcb4pbywXa2JcJ2B7izfbj4CYQEkWazOF0kc=;
 b=kJf9r6Vlaya/SlARVC3rk+ZNMY/sYlyhmFxqHUJ+V88HhH+2xe3T/l+HKFdiZciI+IlrAjsAohlo/4usbWZLsisMUA3oFNpazhk8QaIa+uNOo1cSw5EyLJr62Jji5pRzfD4QBuXnxpJSMPx2FfDuDcT8eEy4vvl4KPuYZeQ8xuAhB1YHEFOuiPEifmqR6Q3IzQopZ6YnvUS5ZDB7NgBHvBGXsJLa8K7H1inEAX2qPHsXAOkKCrlYwoPY0AChr9bII92c7nCLYsZcxwLJrYeBt9Pt9DiwStulZ2ad3oee1/oKeIUIsQpJ/JQGnnUslZyqxF1Or7dfU3cDCXtw8I7hcA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: x86/mm lock order enforcement
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <47986e72-5794-7679-1cd9-2e05cf71bbca@suse.com>
Date: Thu, 30 Sep 2021 12:02:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: FR2P281CA0013.DEUP281.PROD.OUTLOOK.COM
 (2603:10a6:d10:a::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 05ee0b57-b82d-419c-7ab9-08d983f9641d
X-MS-TrafficTypeDiagnostic: VI1PR04MB7150:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7150B5ACB6B073B291A738CFB3AA9@VI1PR04MB7150.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	vCq4jPscxa4YPS7ZcX5VeYme3Ts0R/BByUP23qFRX5qhIDOkenBbx1AcwpWi9GZAay/YAJuJ/AXaW4iVAMaXlaeAgHLD3XAfQjEEWDT/mO4mXmX/ktDmJ3JuqZxaXqDyB4gN09lp2a+58M9zzDm3vT8lDf3FPeB6xuq62OgImNBKFTzyaHkwaND7kidAiYzNmVyyUGkLVVPvWhrklLTWQXeiLZVLcWlecitB4k2MNLrLpU5emKsJ/70XES+sDWmKwGBdp1AbrXr2mJraC/GqokLcuOVt14+8jv35ZMBlaqO3XT0wWu732Bol4gQ1nvltSbDqOVlU6k1vsnIbE7pku4CKAsrQ/vUCOFf1SKIwcOPxsyRo6Vsvc/UwJEL4W3D9AvUcFmSzF3TGebOV9zUYcrMvGEDZGxEzo26Ey+OPn0N+EloHRumCFqi6BBmeGfE1mByncVH1X1NZ7snmRh3VVeSW+rJavFmPiym+hS5klJCDvaxRq4IuixOZPO5VVY0V1B7DVyc8UG5sEgNJR+sCp/nRXpKZhyVMNxKAakQP0N0GFTaD3JrZnJXI9XUQUTchaSZ73k2ukJMaCbus3DXI5pbDhJj0OB/ewVFM/pLQiBmpPffeiqEpXOfGSND/BFIwc/jKRBuykrcgcx26Swtb3znibHs1utLeHJYtTQ2g+9FxXGm1NtV4byQ5ewn9y5fIFA6JoYKn3SgwJXYmZj986nv5pT+P6wddpKaP+XPJZJAxq62hFogoNV2DUswT6knYtaB/fclObrZ+PfzBSQ3qoT/UWmQ2AU7jeMYXEE1mhl4Xr1T0FRViT5twlgsKviw+28SpNaWU4gRH21XCPviHORFl4x+W5oXz60Ishttp0Lc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(6916009)(2616005)(86362001)(508600001)(966005)(36756003)(31696002)(4326008)(956004)(8936002)(38100700002)(5660300002)(31686004)(16576012)(316002)(8676002)(83380400001)(2906002)(6486002)(66556008)(186003)(66946007)(26005)(54906003)(66476007)(87944003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	RG0RRLceMD2FFD94JzHY1l68ifAVXK2R/fkMAybSgO/MNk/rHyBUU6pGOvUQyPtqTJhXhDd4F0AshQ3t138f1wuE/1Zz8FuLWhG+NPt/nLjnbbcky5mvkmALR3LtPxnUmKkq1K/onNFE0liYq6tJ7ta+fSqaFvLpOwfl6tvUoq5a+bT/n60ER+IxFdpphy3IK3liIUFWeu3PcViHVIYN0eFq/YelxMoPupauooRpRruezgnBatXVk4WRy7qgXO8WIYpFBkSxO8gjr0A89qmXieJtzodtgY3sTgTAzLN3Be5TxkVAWuOMK51LFR+5i8xE3t1jLiYCu17L+G5K+gJEZBb16AAdgJDfaA3JWU5HM6CRWxzF9P2ET8FZGyHbWaGgZ2e/qy0fafbLI0zzSdDfhKkQc1xIUyn7JtoqcoawKu/PuJsd4sMTL/JkzSVs45wW/nGnqujKJZHjwPJW7hxglUllBuGoLHdZtw1Esyf5rQuwahWaxYcLm0AokEE7W/ugMuGruN3KSZ0wxCX+1CNCK60aKNai+8V72JQtwFKlMOQ3Upflske5bxdM1IBYd+w2zuBFj+Jb8TK/cd61/47fmMUbT3XJ+0RJucDCZ9dxbQBiv440MUUU3SHkcWpgNLDkl+6nkiyw121ekeih4p6/olTPS45ehmtmOc+8oNpjAQZOLxvLObCU5O9pDT3JeQ7dEqKv+FB2tMf/y1z5zM3TtgKcKxlYOXmDTT0K/Ef6XVbDkBdCgsogPGUO/sQX8jic
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 05ee0b57-b82d-419c-7ab9-08d983f9641d
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:02:20.0414
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rT6AoVnhAhYviApWluqQ2ISDJUNVH8sBFDp0reL+SQpBp9uTCJ8T0nMEbXlXYnws77eJKGKFvh6W0HZB1eedLA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7150

Hello,

the issue pointed out in [1] (and hence the proposed fix in a reply) is
actually pointing out a bigger problem, addressing of which may also
invalidate that proposed fix.

First, fundamentally any code path reaching p2m_pt_set_entry() (i.e. in
particular any caller of p2m_set_entry()) is liable to hit the same
lock order violation if it holds _any_ of the locks numbered higher than
the p2m one in mm-locks.h, due to write_p2m_entry()'s call to
p2m_flush_nestedp2m().

Similarly I think ept_set_entry() -> ept_sync_domain() ->
ept_sync_domain_prepare() -> p2m_flush_nestedp2m() is affected.

But what's imo worse - there's no enforcement (and afaics not even a
rule written down) when it comes to the many p2m locks associated with
a single domain: The host p2m has one, and every altp2m as well as every
nested p2m have one, too. _check_lock_level() only checks whether the
already held lock has a strictly lower number in the scheme, so all of
these p2m locks are only put in just two separate groups, the members of
which are then all equal right now. The two groups are altp2m (all the
altp2m locks) and the rest (host p2m and all nested p2ms). Within a
group the various locks can be freely acquired in any order without the
resulting issue getting pointed out.

A similar issue looks to have been recognized for mem-sharing's per-
page locking: These also all fall into a single group. share_pages()
has special logic to guarantee uniform order of acquires. (I don't
think the comment in mm-locks.h is quite right there, though: "We
enforce order here against the p2m lock, which is taken after the
page_lock to change the gfn's p2m entry." To me this contradicts both
the code in share_pages() and the numbering in the scheme.)

A first step here might be to have _check_lock_level() consider not
only the level, but also the involved lock. Since recursive acquires
are frequently intended, the two involved levels being equal would
only be permitted if it is the same lock that gets used.

Otoh it may be somehow implicit that all of the altp2m and similarly
all of the nested p2m locks may not nest with one another, in which
case it might be good enough to separate the nested p2m ones from the
host p2m one (just like the altp2m ones have their own group). If so,
I'm afraid I would see no way for myself to determine where in the
numbering scheme they should be placed - them currently sharing
position with the host p2m lock suggests that any lock numbered
higher may already get acquired somewhere with a nested p2m lock held.
Yet to address the original report they would need to go above of at
least the PoD lock.

Thoughts anyone?

Jan

[1] https://lists.xen.org/archives/html/xen-devel/2021-09/msg02231.html



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:05:03 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:05:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199702.353920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsw1-0007Kw-Id; Thu, 30 Sep 2021 10:05:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199702.353920; Thu, 30 Sep 2021 10:05:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVsw1-0007Kp-Fd; Thu, 30 Sep 2021 10:05:01 +0000
Received: by outflank-mailman (input) for mailman id 199702;
 Thu, 30 Sep 2021 10:05:00 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVsw0-0007Kj-En
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:05:00 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2a30a6f4-99ea-410f-8841-f2324c449a9c;
 Thu, 30 Sep 2021 10:04:59 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-32-zTB6YbDsM2yxp_3Zqz9zFw-1; Thu, 30 Sep 2021 12:04:57 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3535.eurprd04.prod.outlook.com (2603:10a6:803:11::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Thu, 30 Sep
 2021 10:04:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 10:04:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P194CA0086.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:8f::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 10:04:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a30a6f4-99ea-410f-8841-f2324c449a9c
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632996298;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z2azI78UXlA9qvJuwaLYSaRqihayGVHtBo94uTNyiMI=;
	b=ilZQ8iwNk2+MDdAb22LybO30qlatEDVnnvOdJo1LAOOUSmrzvROwKYqBWCjHmdTJBFvxIP
	9Ls26joPTcm8nCC3CEYYMdKfmLKqolzsddrxt3G2le7Kee7TbM+0Gb3vIgce/7l5v7KMq7
	bHCFBsk8TIg5C9yCmfphuHiJ1e/TW5w=
X-MC-Unique: zTB6YbDsM2yxp_3Zqz9zFw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NSgOz4yMaUoKI6Cy4DOmYzno8X8mu7VUljZ7x8WeDY4VrI/sRoHnwKwd9DYu3bUM3RnsvhBlUvz3c3uz9VxSkbQJp6+M9FFE0KeALqhP98ypa4eiAvVDAgN78TQWcmHul3JyyFo6xFIgJu/YSUMJ4aZohzqhZs4z/y5cf2lDjyFGw4UwYwD4XHwjtjwSLyxqlaKim3rDxTrolgMCJYTkmIA1r7iuMrWZNhlPuP+IdyQZYlhxYXVw6Q9zxANVkyJH6IOTYJa1JYSHFWZFEWoM/WDatAa4Eq2/Rrgz2wYvbcPQN+WyZ2RmzXbEnxDxfbUnSyE/wb4+3in8yFGZDFz5PA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Z2azI78UXlA9qvJuwaLYSaRqihayGVHtBo94uTNyiMI=;
 b=JDoxVNeVR7tyEYyt0AN0alSrAO50vM7l7C3QO0EG7dtjs2B4JINvNP7lKfytVDX8t5kp8gMD6goo+M7PqQU3NJ86AW3r8SfR5oBhutasihwG1UZgiZpuFZXKjYsXrbxO+x1ohe74LTIkiE6MA7dIQ4paVINALd3g0KH0yHrY2DpEBUOT+jlI/uoUzDWmYmltPaxhVAtIlcTgPGw1Ez3nTrF0biclkKc40ac8gQK3lWcAJOX2vWvgp81xDN0IvkPNWTeAbuYTQxhwCQ7lXEy6Hkkizc+JFehZlxD+XIiphb0mlxw0kzs3Rq60QH4Zqssl5dHNs/w4IgZxm7syCs+jmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH] xen/arm: Expose the PMU to the guests
To: Michal Orzel <michal.orzel@arm.com>
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20210930092651.1350-1-michal.orzel@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1e37a54c-3937-4400-b126-31115da00df6@suse.com>
Date: Thu, 30 Sep 2021 12:04:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930092651.1350-1-michal.orzel@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P194CA0086.EURP194.PROD.OUTLOOK.COM
 (2603:10a6:209:8f::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4f7aaf7c-e690-476f-5553-08d983f9bd70
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3535:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3535ACE7EBD58503C89F32CBB3AA9@VI1PR0402MB3535.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2331;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VXd86F34NWAsUlicjRYoBwMi7jBpKDYNHuFnO8OE/5sgwni4wev7eOzAFVHDD+nxPDD3cXQIRuhOKXb/qnR8WMRI5UAqifsNmlO/z+YDS8Ri1CNaVS2ZlzC3f2C46WM7tWGmDjc264E6gXivziLIXy6YSU3k5Xl1Mw+ESWVtQHlQ2Rg5KthzfJLcqkL2ko6iEHWwiTT4Lcklh44OeFscuMJDfvAWxLF6KzaIaIhIbTCWyn+T7ldSivfa4xZHxm3EcHgVV3DJR5gvWs/wy+eeJGhwR0lfmE0vFOriXR7G8Hk7k4w35oBYg37cubBgVu3sWPq+2I2Gvj4HdWkVnI8Zh5Rsb+zsJBZ+mHZHpNL8OcIWPqCWroKwYOIqN9+XqblbDPGRdNHDIY/3jHFxZ9VP+09WHF571R8BR4rjHvza254YJITK7TD1OzgqcvDfypKOthe1b0H8nxPIsRHhlkazEtoJkWPG8QqdQs7zOur9ENnDjxdrO2YKrC45bfv4mKlzh/b/8Xx5SodDDb3DfYD6RNisiq4V7flKV/uBGmrQL5kzyyPUZptf4YaBIrg2FEnbKJz8O2v2oYRoq4vDHetcXEjV3pT6CVJpc/HYBaqjJ4NYQMnWahlMMVlzeOeJs5ePbWbxMRMPQkRGYVFt7FtNITaIWcIRGf8vjdFHS/VWscnhDS+x1Zsg5rVgVQMxmOJ4mK9gUAP2RlxR1W7ClktB3Skudf5gagxic2jH3K97U1k=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(26005)(8676002)(508600001)(53546011)(66476007)(66946007)(6486002)(66556008)(4326008)(186003)(2906002)(38100700002)(31686004)(7416002)(54906003)(86362001)(36756003)(6916009)(8936002)(4744005)(956004)(31696002)(5660300002)(2616005)(16576012)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	h16bUuVCCNbjhik8sIgWgEEwVLtLZv97oJ/wd75XpO/RHy/RVrmYUYFzMQMrEs1nA7XPoQIXigNI21UAyMW0qOhnVcgPmBIUjIPW5exSSlvgraGn6JTXJYstzoFmf+/Z13x8FqOdFe+BBYjiFXx4uQqnvg+3Ep0gppzIztcB7NL9oVvU9+uCFYqS7pxSsVFEzlqbdurM3P0yh44FalePT8IPAQyT/uMu/Hol7VCBy4CqWrIPCy0qXBojuGIXuTX8F1wxjTneAlw4otZh/oUuBvaCJkVzMmBlZ7yTrC0/aaNPh8yTeuiHSGdo/5RxIImmELMcGctatgMhhydik7rS9BmD07TDRfwTJAHa6Prb6eAwYWjTFIMGqjr6beDbZAImAVi6O2n+TS8DIsbpyK8645SsWE11GFom9iq/EdhhxftluHXKfRqpanF9VAzg4EESYPoI58ZuyXtPnT/nGYhPH4/B3szdZjn3n4SjY2MMVXBQQPWfHhQ2fSzrTZdQqcbLVtfTsPYqN7Ch0pTLdlglnDOUqBxrjhMYEaB2ecWnBCH0Mm8OBLqM2mAnFs95GfDQ/4RDjmNUskN7MqP1hJFZk2MMi5T2LiSbh3WWmaltucIB2pkQ66JkUn4fsUyg9oQKbAA0B6hnHKo1vrF7hr3VkeW+Agg+khlAEGPGiB0AeSo54RUg5TnTGHFlP1neh1YUxXaxChoBgZyqOI4tvvOTMbui9qSUWXpBQNadRSJuEA3H8IJmzb2iphlLm331kQPG
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f7aaf7c-e690-476f-5553-08d983f9bd70
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:04:49.9016
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /eo6qeGfj7IPFAiWeqIybz6FDVcZEba7J1crAnoZPBbi+6dXispCY14fEzDfct8uF8ba0yi3Qenk6ZD3UNjVNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3535

On 30.09.2021 11:26, Michal Orzel wrote:
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -483,7 +483,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>           ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
>             XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>             XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
> -           XEN_DOMCTL_CDF_nested_virt) )
> +           XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_pmu) )

Alongside this I think you need to reject the flag in x86 code.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:14:56 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:14:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199709.353931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVt5Y-0000Vb-Gr; Thu, 30 Sep 2021 10:14:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199709.353931; Thu, 30 Sep 2021 10:14:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVt5Y-0000VU-Dc; Thu, 30 Sep 2021 10:14:52 +0000
Received: by outflank-mailman (input) for mailman id 199709;
 Thu, 30 Sep 2021 10:14:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVt5X-0000VO-SI
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:14:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 884d1abe-2b90-4b41-afef-e59f42db7ffc;
 Thu, 30 Sep 2021 10:14:50 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2059.outbound.protection.outlook.com [104.47.13.59]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-29--E0JaUw1P2uXIiAHILdtRQ-1; Thu, 30 Sep 2021 12:14:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB2960.eurprd04.prod.outlook.com (2603:10a6:802:9::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 10:14:46 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 10:14:46 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P195CA0010.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:81::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.16 via Frontend Transport; Thu, 30 Sep 2021 10:14:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 884d1abe-2b90-4b41-afef-e59f42db7ffc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632996889;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=oxW123egbkz55sKIIo7VKXycxJ4eEGt5zqs0Rc6FPqQ=;
	b=apvIhRlCDQQzrgUK4oPIt9f1MkOpuI7/6KreftvTaYjOYN6fP7KtBy0a47hGIx+QSFXsX/
	BeoPjA83kXVrFMBy5bxbL2PfHUIyqHNi5yKlgMnSQbzdsgCKCCtTMVL/wnBq4e/J+UwtmQ
	hF/pZ2oq2Qp4dqIEjBsy4JtCIody+Zo=
X-MC-Unique: -E0JaUw1P2uXIiAHILdtRQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DCjOjihQbz5NCR0p6FU3MT4MlBUSzOxz6oypjvy6MQ+5IRLG6xNx4OcVBbseVAbYZ/BlaGhXweK8vbtxxz43dO6GnH3qu5DuxX1jPcxkjhp1Uip+vd/ODZv1nGK9IwOCuMxB4gIrtaTWUKasEMQ5PdRbNv2kcxv3nbXLyALVU9W5Xpber+PCeIwmpxmTLKSSECHiUFInWZlfN3sEEmKVPrR1kc6jxt0OSeY4E8JH76F3wC9QAvHIBy+uV0FCcFdlDHYytiCkPh7yAZZFCmbfZJT4pRaVKAZstOfqs9nd87vXdQ4iJ2UHTXQNasPSJyGtnN3eFyNvSXJzTQNSkeZuhA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=oxW123egbkz55sKIIo7VKXycxJ4eEGt5zqs0Rc6FPqQ=;
 b=Qfm2SlLRGLH1EsN6wxuGYWgArK+hOc4RrQAqYTagCe571cgvl+sFpnU2FAc02b1t01uoMIUwyzTrDNwI+vMNdtJTk3nLz4/PT9z7ZUxRjtwvg1n+UaNUuYj2mGIsm08/Opv4j7m/8kmIv4CWSbic4DuuYJLOX9OpQ7i3cjtXboDA4GWCHzvUR1X7ECXKID4fvPKzxo6CoxwrQIRLkGwcSQE0RGzeEELVXzbGhNrFXq16ZXnEL/RM9p6rJ11FSqCG2bQVzjKlH4d0kQ/01LvL+IRqlRv3A0wfAqOSe4zxJ5zGb9giIuf2fCGzyFCfPvLD6s81/EnpbXbSysIbYy3tTw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
 <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
 <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
 <7d035651-6e16-a6cb-6760-e5d077e1a85c@suse.com>
 <91fca69f-1a6f-198d-be27-c6b0ddd62206@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <078e3a01-4498-7894-87dc-31798cfc474b@suse.com>
Date: Thu, 30 Sep 2021 12:14:44 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <91fca69f-1a6f-198d-be27-c6b0ddd62206@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P195CA0010.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:209:81::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 56c030d3-394f-4286-d6ec-08d983fb20f1
X-MS-TrafficTypeDiagnostic: VI1PR04MB2960:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB2960F9B6B03AD32094A43017B3AA9@VI1PR04MB2960.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0BqTZzXBKd4Tw2B3ujl3lSvdMqqf17wTUQ2HBgfrQhOShzK5kf75nV/07/6IIXtwE21FNEVBTFsn/lNMQz56bJ0aOnwpl30d0/c557LpFfhsoiWf1/MCJtNUCyj2/sSvtr6EI0oId5kZ3r3dYHrzP8Abgs9eVUG57htCY3hHMatu6u3yqUFL3p4gu/ybL5rKcU5vwhO0qmpMrYgcrCg5+7tYiFYZn+G/sRN1zUgk0azqO8cbJKZwUbqQ8fvphe/P9YD2z+4yPh89zJwuAWRohomZ7ALCaXMny60Rc8jE842jt0RTf1Hqtg4w3ncpV61VUhSwvtCrxQYNRYrq/8dIV59FCbFMpqQpoY88j0HIDYm6FBrL6xWNFvy4LvR+87Ec1bAyoSpczx3EW0TC29+pcxyzVa6mw3HaHSjAMSftxEcDLVB2Q9O19m3vLhDdLbGHGhQlYJYv6i1v7k6VmP/30HB3mOpQYCcGVhRXeCXSGLDjPviwlxc4Da+25joE/5HX/DPFUQ4N0ZvECpqHa1zpkV9ixc1qENslmbMzfQuAMg0m/5ZLSzbMI/AnB0cnGxjF1fx2hyzqireDriiTE37tRmLXY0AwBF4ZYRVhaktIaR04FWeFdogR+IMVhA1eSOwmqnPLWeeEJpdurNCPjUwn/Z6wFp90n+hNXHP65cwsSjlYYxyiFphva5VHNLVP88Ln8PfPlc2S6wBorFisjJ7X7ToqlaB/ZnKnJ/MTQ2RuAIeHFHUZ/BI48hLaELszLT2A
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(6486002)(54906003)(31696002)(26005)(4326008)(36756003)(53546011)(8936002)(31686004)(8676002)(2616005)(38100700002)(86362001)(66556008)(66476007)(5660300002)(956004)(2906002)(6916009)(186003)(508600001)(66946007)(16576012)(7416002)(316002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cGs0VjV6MTZRMnhKdk5DRitoLzNGT2J0SllGcVhNWWwyZXVDRkxrNml2ODJ0?=
 =?utf-8?B?SFh3YUp1Vk5zTjQyTEMyWG1zZWZQZ3k3ejB4TVIyWHI0MnczSFA5bWxnMzZU?=
 =?utf-8?B?MFJ1VVdYNFVRUEZJNnNDNVpkcDh1S0h0OXNBeUlSUmxjRVRDeTQwY2p6QXMw?=
 =?utf-8?B?QWNoaU1OaitoYVQ0dUhCYkJjZXBVQ3lRTy9XMVdLdTR2bWRIclVxYXcrL1Uv?=
 =?utf-8?B?TTZMMW4vVzBnVmMyNWliNitOempnUXNNVFpERTVrekViR0JHd0h4VFJFeUFX?=
 =?utf-8?B?d0xNQVNHaHZWQkpIdUdKbmtPc2Y3d1RObzk3d0NwTEEzSDlESXpOT0lWOHph?=
 =?utf-8?B?QjBmYzI5N0gwVVd0dUxzNzRIRklYK01jL3Y4eDUvOUJlZlpuY1ZJSUFack9P?=
 =?utf-8?B?SStWeEFlVkpiU2lMNjQ1R21iZ3R4bWg2RDhKUU9Nc2tua0puNWlTS1FJWkxO?=
 =?utf-8?B?b2ZUSzJIYkkwZXZYZVFKZjFxV0JxK3BqQmdFak1nSVJHVUhoSWRFZzFvTHVM?=
 =?utf-8?B?dVVQR29JcE9mMFh0TmE5N2xQOStXaElEazU5VmdxcUpUaGxYeUZDWTk1RWdv?=
 =?utf-8?B?dnJlQ1hTZTlUZEp6ZnVFR3p2VHFVUGtKOUYrdzRmQzRobFdidXhQMTN3OGtV?=
 =?utf-8?B?UjVERTBvTE96R01GR1VYL09MRlcydmxucUlNUU95djhFWEtXdWZWdGVmeUJP?=
 =?utf-8?B?K2dmWFpzbmk2T2phUmZOWkdiaGJFZGxQa1VMQ2xEU3ZXbnduK1RlR2dFbjdG?=
 =?utf-8?B?QnRKdXRLTVJ1MGhnSXdkRHVwa3lZN2Q3NmhZZVV5ZG9neEp4ZHRPQlJObUV0?=
 =?utf-8?B?VGRYNElUVXhnVnNpVVd1L1lKMlM2N21wcUhKNnRXenk4NmVoRDlrNmM4cWtt?=
 =?utf-8?B?UEFTV3h6Q1hoSzk1SlhRTmpTZFhPYkd0UURKOUpGVzJ2Uzg2eStpL2s2TGVC?=
 =?utf-8?B?enZVZ29YS3NuQkgvRUgzZWoxWXJqWjZpUC9zV1YwQ0xFNzhncStGYTk4MTZq?=
 =?utf-8?B?VHpVL28vQ2U2TVh1WHRMZk05dXRDanl4bVI1Qnp4ZWFIY2JkZHBrNTlvL1gz?=
 =?utf-8?B?ZDZkb1R5SnFOMkw3VEhFMkl4Zi9INitiSXY5S1M2YlczR2tCU2thWkg5NTIy?=
 =?utf-8?B?ejdkR2U2VXFrd2NOcExuK0I4L04xLzd0M2hrVVZ4VU1lR3k3RWl3Q3lwTEdV?=
 =?utf-8?B?eWp4YStPamphRHJ3ZXBrdmtoWEhCMWhwNVU4TTE1dWxUUVJaRzJwbDVKd3E4?=
 =?utf-8?B?R2kzVlFzc0hoUEJ5NFJpUmg2c2hRSThEaDVQenpYNmIwUGMyNk8xWmpjYTA1?=
 =?utf-8?B?UTNWWmJEVXovZzJmWWdaQ1NmaGhuaklDUzRjTlEwTnlEalZKdjNESkhYYThz?=
 =?utf-8?B?Z0VTZGozcGUyQWpoQ29Wak9tQnorVnVKSDlCRk01NlpMcE4zMk5LTmRKWVdn?=
 =?utf-8?B?eXQwQ004V3RHMG0wSXFxUjBhL01DUXpreGlVSy80TXJhMjZzZTVmOGxEQnVM?=
 =?utf-8?B?YkNtMy94dndKNWpjaytoWWQzOGM3R0Zsa3hlNGRRQThrWFZFMGNKVVhHOXda?=
 =?utf-8?B?eDlETVdXOEJuazduRkFLQStHZ21obHRZa0JBbXdmR01tUjNpbHlJRFRRaGk1?=
 =?utf-8?B?VC82STk3Z2xDUVV4dnpPaDdHM0xSSWM0VGtibXNrL0ZLRE1aTTR4NkwrcGp3?=
 =?utf-8?B?UlBScGZZR0hteHZrRTNON05mUFJtV0kzTnRYWVpjbFM0ZHNKYy93eTNwZi84?=
 =?utf-8?Q?u93ybVl9gX4w+kZ2ZaIrPwK7SnSZykoxZF2GIK1?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 56c030d3-394f-4286-d6ec-08d983fb20f1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:14:46.3505
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Ec0qsW9AYn6PDA0E0vkbwptd3IvCOxtF4pCwM/I+WD+SQTr0c/H5Rat+Zovj9xa6UOi0jIGwZhEed9mZRecA5Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2960

On 30.09.2021 11:21, Oleksandr Andrushchenko wrote:
> On 30.09.21 12:06, Jan Beulich wrote:
>> On 30.09.2021 10:45, Oleksandr Andrushchenko wrote:
>>> On 30.09.21 11:21, Jan Beulich wrote:
>>>> On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
>>>>> @@ -1429,6 +1433,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
>>>>>            rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag);
>>>>>        }
>>>>>    
>>>>> +    if ( rc )
>>>>> +        goto done;
>>>>   From all I can tell this is dead code.
>>> Before the change rc was set in the loop. And then we fall through
>>> to the "done" label. I do agree that the way this code is done the
>>> value of that rc will only reflect the last assignment done in the loop,
>>> but with my change I didn't want to change the existing behavior,
>>> thus "if ( rc"
>> rc is always 0 upon loop exit, afaict:
>>
>>      for ( ; pdev->phantom_stride; rc = 0 )
>>
>> Granted this is unusual and hence possibly unexpected.
> I will remove that check then. Do we want a comment about rc == 0,
> so it is seen why there is no check for rc?

So far we've been doing fine without such a comment, but I wouldn't
object to a well worded one getting added.

>>>>> --- a/xen/drivers/vpci/vpci.c
>>>>> +++ b/xen/drivers/vpci/vpci.c
>>>>> @@ -86,6 +86,29 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev)
>>>>>    
>>>>>        return rc;
>>>>>    }
>>>>> +
>>>>> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>>>>> +/* Notify vPCI that device is assigned to guest. */
>>>>> +int vpci_assign_device(struct domain *d, const struct pci_dev *dev)
>>>>> +{
>>>>> +    /* It only makes sense to assign for hwdom or guest domain. */
>>>> Could you clarify for me in how far this code path is indeed intended
>>>> to be taken by hwdom? Because if it is, I'd like to further understand
>>>> the interaction with setup_hwdom_pci_devices().
>>> setup_hwdom_pci_devices is not used on Arm as we do rely on
>>> Dom0 to perform PCI host initialization and PCI device enumeration.
>>>
>>> This is because of the fact that on Arm it is not a trivial task to
>>> initialize a PCI host bridge in Xen, e.g. you need to properly initialize
>>> power domains, clocks, quirks etc. for different SoCs.
>>> All these make the task too complex and it was decided that at the
>>> moment we do not want to bring PCI device drivers in Xen for that.
>>> It was also decided that we expect Dom0 to take care of initialization
>>> and enumeration.
>>> Some day, when firmware can do PCI initialization for us and then we
>>> can easily access ECAM, this will change. Then setup_hwdom_pci_devices
>>> will be used on Arm as well.
>>>
>>> Thus, we need to take care that Xen knows about the discovered
>>> PCI devices via assign_device etc.
>> Fair enough, but since I've not spotted a patch expressing this
> Well, it is all in the RFC for PCI passthrough on Arm which is mentioned
> in series from Arm and EPAM (part 2). I didn't mention the RFC in the
> cover letter for this series though.
>>   (by
>> adding suitable conditionals), may I ask that you do so in yet another
>> patch (unless I've overlooked where this gets done)?
> Could you please elaborate more on which conditionals you are
> talking about here? I'm afraid I didn't understand this part.

By putting it inside #if or adding "if ( !IS_ENABLED() )", you'd make
very obvious that the code in question isn't used, and hence no
interaction issues with vPCI exist.

>>>>> +    if ( is_system_domain(d) || !has_vpci(d) )
>>>>> +        return 0;
>>>>> +
>>>>> +    return 0;
>>>>> +}
>>>>> +
>>>>> +/* Notify vPCI that device is de-assigned from guest. */
>>>>> +int vpci_deassign_device(struct domain *d, const struct pci_dev *dev)
>>>>> +{
>>>>> +    /* It only makes sense to de-assign from hwdom or guest domain. */
>>>>> +    if ( is_system_domain(d) || !has_vpci(d) )
>>>>> +        return 0;
>>>>> +
>>>>> +    return 0;
>>>>> +}
>>>>> +#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
>>>> At this point of the series #ifdef is the less preferable variant of
>>>> arranging for dead code to get compiled out.
>>> What is that other preferable way then?
>> "if ( !IS_ENABLED() )" as I did already point out to you yesterday in
>> reply to v2 of patch 10 of this very series.
> Please see below
>>
>>>>    I expect later patches
>>>> will change that?
>>> No, it is going to be this way all the time
>> The question wasn't whether you switch away from the #ifdef-s, but
>> whether later patches leave that as the only choice (avoiding build
>> breakage).
> Yes, the code is going to always remain ifdef'ed, so we don't have
> dead code for x86 (at least).
> So, does the above mean that you are ok with "#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT"
> and there is no need for "if ( !IS_ENABLED() )"?

I'm afraid you still didn't understand: "if ( !IS_ENABLED() )" is
also a way to make sure there's (almost) no dead code. And this model
has the advantage that the compiler would still check all that code
even in x86 builds (throwing away most of it in one of its DCE passes),
reducing the risk for someone not routinely doing Arm builds to
introduce a build issue.

But as soon a code references struct members which sit inside an
#ifdef, that code can't use this preferred approach anymore. That's
what I suspect might be happening in subsequent patches, which would
then justify your choice of #ifdef.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:23:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:23:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199719.353942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtE2-00029j-Fe; Thu, 30 Sep 2021 10:23:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199719.353942; Thu, 30 Sep 2021 10:23:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtE2-00029c-Cf; Thu, 30 Sep 2021 10:23:38 +0000
Received: by outflank-mailman (input) for mailman id 199719;
 Thu, 30 Sep 2021 10:23:37 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVtE1-00029W-A2
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:23:37 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0654e257-3c8d-4cbb-9869-645bd790e82e;
 Thu, 30 Sep 2021 10:23:36 +0000 (UTC)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01lp2053.outbound.protection.outlook.com [104.47.1.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-8-uKjFh5p3MB2clVxN7cHFZg-1;
 Thu, 30 Sep 2021 12:23:34 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5471.eurprd04.prod.outlook.com (2603:10a6:803:d0::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 10:23:32 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 10:23:32 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR10CA0048.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:80::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 10:23:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0654e257-3c8d-4cbb-9869-645bd790e82e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632997415;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OH4c6v8MmIglWbh9xaH+Cj0cJtHeQRU7Omh855cbIUc=;
	b=NGLEJJWereAtlBFYPbV0SBFmZ9porZ1PKCPtmkMPocWurD32ENm9d1iGlAfXyNkNEZRN2k
	vQOxdRXoGHCHROHQ9EqfE3xvweg9yVlvxjihHOqhtSXm7xDKY7frsDcX4DoLIEiqQFNTSQ
	MdUkcaROn+MCqUo8vq99V1onUFJO7rQ=
X-MC-Unique: uKjFh5p3MB2clVxN7cHFZg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Lnz9ry1ZIkJRw7ExYyLaLy13LSZ+c7KnpQ/1JxWdurM4v1WCCg22RLFo54dAYWTXObxi8KJX+ad3gZtFr5kYjS8rrs7fGrcDBW5NMeydHl/XqQfeqlZhILL0MVtXcGkMqKlJW9KJ9GiVacZIBBuuh9UynBBie+ihcTZd20klBbGcEmquAqNUE9YPFDddVST2JkHpoKDu8otJUpXEgQyzOLwAwqN/kGUg0Ix4fnUWWRcf5xRzmGTzHmySlUwLjYrI9JFabdvYbCDArfkHwQcZXj8UfvsxD9jzwTBfDtBamirUkgm+spYOrBCKbxIrFHLMYEWZkgt2TbCL1PNtkQA6mA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=OH4c6v8MmIglWbh9xaH+Cj0cJtHeQRU7Omh855cbIUc=;
 b=mQFz/l8IrZEw9ojXpTo/aQJRqpmR2OF6SxUwjRmcYy4viZCVR3fZxfXGxVoYQs4W1yLkBHeGppQcMrxONlLN2PAhF1oGFQWBQKLhvMMNA2KdEPOPK0SoOQR5rKNxKFbJ4M1TxBQWE0Xsr3t4uCv8LWbZwPWyVl540kTzJKJc8LQtFkgdBVFgIvB0ixsxyau9Kwrp36jfd/+GtDL0fj5+GQAOCx/2wB+LwRvS9y0puzLrHrSNVCyo8JDIafUgfLtKFfMxHGkh63sVsdKu7XcI7LvphGW8nyMLB+9NYVtrVbSo0W2cGgRvHPNmAsHIPXk+7dOA0Va0scWbPNg3tDjJ4A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-11-andr2000@gmail.com>
 <ef2c008f-7682-1549-8d27-712b04fbd84f@suse.com>
 <603e2cdf-748f-f18b-99f4-44ddf3c0b9c1@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <84368ff1-64f5-418f-6a49-5098b7edf119@suse.com>
Date: Thu, 30 Sep 2021 12:23:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <603e2cdf-748f-f18b-99f4-44ddf3c0b9c1@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR10CA0048.EURPRD10.PROD.OUTLOOK.COM
 (2603:10a6:209:80::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 361e3912-8e16-434a-21fa-08d983fc5acf
X-MS-TrafficTypeDiagnostic: VI1PR04MB5471:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB5471CE46E7F2E1169941B710B3AA9@VI1PR04MB5471.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gHXGPVqcssuGnb3aG7Uq7jofEpRKJtdAbfxXQgKoifYxtinD26+V/BLXz9Mr9G6hpwqYJdXDMjfzqGVrhggnP/TyxSy1tG9LYeKIAFSZRz8FioZK1C9A9kWwivnBbcBNWvUF9ivc53hqCck/rNhVxZ+L30dxSwx5NZuTWEtNvVOg0jkTgttbzTWqnFXuK55MITADwKbHvI7Aj5gTMSLJY1r1NvSwTBzuw2fvSYb4tpTaFoxzwvayOEZdUJPYxqceWvRQoOuFtzj51bFGx0zh7pSq/a0oxRCldTh2pwGhhUiJhq19AkcMSriYRaszExx9omYnH7WKpPtT4kFSlQ2Ez9t4bNnmQvJxPieqa4Ytt5oXict1Wp9j/iuZqHbTwmo2QLdogGeOcGzXVGYGIbP7veoR5/ZSMQ/koO7UG67DMZ2PR1UbeiWLkPGx7fL+7m2wAE3DGnt1K3YV68nRuhz0J2U4SKfqHZfJIQ+xYyLoOk6nxpRm0y3Kfma4kj6O82nS2EB3yi+KA8PMZoPGcOB6ymYhB9bl8NWAolLVbDwMOMObdqpI8SMMKXnVxtWfCMP5UC9ecJ8j9x+0Hw7HYCy6uOmUExUOcCOffKMlGB5DceXqvCYKpclIuMWh4IVUUqMKTkLRSkke64sxL8BSK9he2a3V0g0QxAEUGGrknpEmUxSAAeB527Of84t/f0j/D609f0FulJ5W8fGCcyGucN7EPITt9JI72tm34p1yaxuS9gL/MZ+RIuoYZUYBmfVIgOvB
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(8936002)(7416002)(6486002)(6916009)(2906002)(31696002)(8676002)(4326008)(31686004)(83380400001)(66946007)(66476007)(66556008)(53546011)(26005)(36756003)(186003)(54906003)(5660300002)(316002)(956004)(38100700002)(508600001)(16576012)(2616005)(86362001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Q0FjeklRSmdralFMajNybGZzUkI4Mms0aTh1NnNHYTNKMjh3WGJsSW5qQWow?=
 =?utf-8?B?RzA4MUNDa3I0VldENnE4aVFsRHlVekUyUEtKWEpnMXlneDhjaXlMOVA5enU2?=
 =?utf-8?B?cStIYmZRcjB2ckVEdkRQUUNQelVHZG9XRG1ZTU85b1F0dWgzU1FqUE9IUjVY?=
 =?utf-8?B?MHVIWlM2ckIrWjhodVRNaXdvZGxGRUxPaGN1dkFwaWVyMjhpRVo0SDZONUJs?=
 =?utf-8?B?TEVPN2l0OUNDQkNOcHZDNEIwaTJERlhSQzBudVY2dTIwLzY1Z1ZLNHVJZGd3?=
 =?utf-8?B?b0lzMUhkRkJlTmJnaWRraSt2T2VkaWthdEJGNnljZkFwakw3YVBWMU4ySzVQ?=
 =?utf-8?B?c2FCNTVUS0M2cU5OWXNlaFhTajVKSVV1K1JVQUhrcWtZVGhlS3RkMjFUdXBp?=
 =?utf-8?B?VHFvMG5jckwwTzVGaUMzbU13b1hDckJLcklBckpxMW0vYTMwM25xdTl6WGhi?=
 =?utf-8?B?b2Z6Y0FkcTlRVFd5eVJhVjRJcERYYkNMc2xqbmJ6cmw4dWh6UWJmV09FaUha?=
 =?utf-8?B?Q0JpNTNEcVl5Si8xY0VDNjF1ZTZxTDRpeUZ6QjI0S1FLMy8rTk5veDhrMytX?=
 =?utf-8?B?UnJDaVBCZHY3bzVxRkZXSUFoV3o5RkhITFdiRGgvN2NKL3I2VzBpRjBTKzlB?=
 =?utf-8?B?MnJiMkZ2OWgxUlpocXpRTE1OWWlPZFFOUzdyMUtRSjlMUXN0Yjd2M2lSWjlH?=
 =?utf-8?B?Rk11SlJORTFvMzZEOERkR2YwR25JZDZMWEhURzFYdUFPRlR6TVB5Y1FKcUY0?=
 =?utf-8?B?bmxEV291NDY4VlFuVEVZRTBtSlVlSlRGREJkekJpU3paQ241YkREUkRoc2Iz?=
 =?utf-8?B?ckxQcm4xY1MzYVVKY0pWM1VVS25LQVUyNGlWeXQrRUxCbXA3Z21wZ2c5TFQv?=
 =?utf-8?B?MThZN1JVZmN6T3g4UXNhQlQyckNxa1FKcHZjbWovdnBkVE9qVDZNcmdWNVdp?=
 =?utf-8?B?a1puZnA0d1BWSk0rMGM4QUJoMTNPUW56dFRLcit6T2JtYXpwNVZtS0dtaFk2?=
 =?utf-8?B?Q1JjUDZ2VW5OcTM2NHRQVTVhN2JlSkcwcjZwQ25aTHdnVGdnSTRmREwzVzZC?=
 =?utf-8?B?bXMyV090VmVYWVROL3BoTm9ESG81eWZiUGtaSDhVQ212UjdGU2hGVkhvWWYz?=
 =?utf-8?B?RGQrVHZ4MHlWTGpMWndrMGMreFRrMVJVUmxkMmVYTU90d0kvcFlUZ3BqVHFy?=
 =?utf-8?B?ZGlwdTVnOFpIT2V4MlgxYkhkTk8rRmNmRVdHdEcrK1Y2R2VVdDFCQmpya2Rh?=
 =?utf-8?B?M1l4OTNhcVpjaWMvSkVFZ1N1MDRFM0hPTEtPeDZLdEttcnlyQlh1RGI0RDRX?=
 =?utf-8?B?cFk5dVhSbUF4TkhtNkVxd3dXOFE1a3VXT1VXeXI2OW5nSVhmRkZFOEQ3bnhQ?=
 =?utf-8?B?cVg3SnF3K0xHYytjL1ZLNFFIdXdjeTZxQ0h1a1ovYXd2QnZrdWh6RWd6OGNU?=
 =?utf-8?B?VnI0emovWG4xWFhCUmNIVHZvc0l5YzFyTWx3NitGV2doS3YxMkVNbGNZMm4w?=
 =?utf-8?B?aFUvZnptTTR5SDNLNkZEUnZRTzNUS1FZN1JjRTZWcFhwNzRvUjRobWMyaEdN?=
 =?utf-8?B?NVVzMkxVVzk1MFZPWmsvOWQ4NzV3a1MrRG9obG9Qa1IwUXIwVzJianFveHRX?=
 =?utf-8?B?RHRLbU9DeFVNZERZRURueU4xZTEwUXpCY1kyYWNWQ2VEY3dUM1hta0F3dmt4?=
 =?utf-8?B?ZXVubVRVdFYxQmJLNEVqMTM2OEdxODZabFRjemNSV1Vxc2h4aXVLT25oOTlw?=
 =?utf-8?Q?ND/xar3usER6u7HSrzAyarI12bIlbaqYaXIU8z7?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 361e3912-8e16-434a-21fa-08d983fc5acf
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:23:32.9096
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /oz1P7Co/YLgqODSF24uSgAeNwv2WubQ45+Fpk78c6FYohAthin9DvpkBVFH64bJVMXP81NQz98Z8qK+JeREmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5471

On 30.09.2021 11:34, Oleksandr Andrushchenko wrote:
> On 30.09.21 11:51, Jan Beulich wrote:
>> On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -831,6 +831,66 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>>>       return ret;
>>>   }
>>>   
>>> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>> May I ask why the code enclosed by this conditional has been put here
>> rather than under drivers/vpci/?
> Indeed this can be moved to xen/drivers/vpci/vpci.c.
> I'll move and update function names accordingly.
>>
>>> +static struct vpci_dev *pci_find_virtual_device(const struct domain *d,
>>> +                                                const struct pci_dev *pdev)
>>> +{
>>> +    struct vpci_dev *vdev;
>>> +
>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>> +        if ( vdev->pdev == pdev )
>>> +            return vdev;
>>> +    return NULL;
>>> +}
>> No locking here or ...
>>
>>> +int pci_add_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>> +{
>>> +    struct vpci_dev *vdev;
>>> +
>>> +    ASSERT(!pci_find_virtual_device(d, pdev));
>> ... in this first caller that I've managed to spot? See also below as
>> to up the call tree the pcidevs-lock being held (which at the very
>> least you would then want to ASSERT() for here).
> I will move the code to vpci and make sure proper locking there
>>
>>> +    /* Each PCI bus supports 32 devices/slots at max. */
>>> +    if ( d->vpci_dev_next > 31 )
>>> +        return -ENOSPC;
>> Please avoid open-coding literals when they can be suitably expressed.
> I failed to find a suitable constant for that. Could you please point
> me to the one I can use here?

I wasn't hinting at a constant, but at an expression. If you grep, you
will find e.g. at least one instance of PCI_FUNC(~0); I'd suggest to
use PCI_SLOT(~0) here. (My rule of thumb is: Before I write a literal
number anywhere outside of a #define, and not 0 or 1 or some such
starting a loop, I try to think hard how that number can instead be
expressed. Such expressions then often also serve as documentation for
what the number actually means, helping future readers.)

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:25:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:25:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199725.353952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtFo-0002kW-Qw; Thu, 30 Sep 2021 10:25:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199725.353952; Thu, 30 Sep 2021 10:25:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtFo-0002kP-Nt; Thu, 30 Sep 2021 10:25:28 +0000
Received: by outflank-mailman (input) for mailman id 199725;
 Thu, 30 Sep 2021 10:25:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVtFn-0002jv-0n
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:25:27 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b9209ae5-21d8-11ec-bd43-12813bfff9fa;
 Thu, 30 Sep 2021 10:25:26 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2051.outbound.protection.outlook.com [104.47.0.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33--h7--FhdNECAqOXXf8Pxtg-1; Thu, 30 Sep 2021 12:25:24 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7151.eurprd04.prod.outlook.com (2603:10a6:800:129::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 30 Sep
 2021 10:25:21 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 10:25:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0144.eurprd04.prod.outlook.com (2603:10a6:20b:127::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend
 Transport; Thu, 30 Sep 2021 10:25:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9209ae5-21d8-11ec-bd43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1632997525;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lcKUmlalB7FPEGLasEPn2v96X2SEhgV64/x47Ny5XaM=;
	b=iXYQgXyM/7dl/ZTA+S+JcVrVFNavpvPbOA6cnFImuZsjhY3kBtA9jQcVMbte9LlC9mPL/6
	UcvN78TPU57u9BIxodSQ4dTObmWkTKKfELYOSeupCt6VcfkF6MgVqUldA12a9MfX2KDFW7
	BDEBJQXscxlKRFonZXHtiSagGqRn884=
X-MC-Unique: -h7--FhdNECAqOXXf8Pxtg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HPQ5/4UnpE+eONtfczvsVjRYksIikTczeUnxxncADsPwfKxIsyUGoSTixOxbSBRFz6W/6iFAXfySPIT6k8q1fVCjLKLugEgjYB0dePOSFQjXDefdwHtQcHMpivTuJbVqi46BW0WLaBWuEFFi3bfU6A90XtGl9ypGKm3JG9G+ra9xdOsQ6NPBSQVLqLPo946oU19LpXZZhDmBGkGRZ2gOqonojoyaUYN20INN9TxX+w/TSnlXDuv85oI52bqqeXOedIzsjRecUrXM+UHlE1BPqES1mAb6hjN4u9kEfoAY5HyHTb+5aN10baS4rLG0iHSqTUZsmOBxWDvn760XxZ8idw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lcKUmlalB7FPEGLasEPn2v96X2SEhgV64/x47Ny5XaM=;
 b=PTp+96KP+8ehuRU0n4zwQdg1kbvdUEltu0lqoGgyR6TLn00sbgTNcY19GhZOu6wQ3sg31NYv8DvyprfBblF1pFkOKRYm/xHf0YjkzkIIITMSE9H2xyfCaYKl0EPNFNEGn/5DuX95RdPNzOr5sn9icMEugv1sgv7+1wS1gGpeE4w49HXFgP/n1UnJTQYjOiPTaHmSKq74j8F09Hf4KwrOrP03EUs/4FEQ4ZaLC71mc3Z3u/wuuN1sikfN0FX25YAD0y1wu45gzG7qlae4DEl9rd7zjhx0pxxPlTtQc2WhZEbTmy/jFB0FZgVJ0QzxrqFCc/YRVWk32NokUI6cHWCPWg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
 "sstabellini@kernel.org" <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Artem Mygaiev <Artem_Mygaiev@epam.com>,
 "roger.pau@citrix.com" <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Rahul Singh <rahul.singh@arm.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-12-andr2000@gmail.com>
 <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
 <fcbb082c-d8ee-b59c-64b9-129b20308778@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <1b2098f4-a462-41b2-8273-6ade04e737c1@suse.com>
Date: Thu, 30 Sep 2021 12:25:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <fcbb082c-d8ee-b59c-64b9-129b20308778@epam.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0144.eurprd04.prod.outlook.com
 (2603:10a6:20b:127::29) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e334a0a0-618e-4458-d4b6-08d983fc9ba9
X-MS-TrafficTypeDiagnostic: VI1PR04MB7151:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB71510B940309F79F4B80DCF9B3AA9@VI1PR04MB7151.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ycLyrQ6YeJR0Puc3zHYN0m7EHt5lZ1ALUaIPk+UFrMztO3k1ggBzt9/1rx1s6nfaewvwcMZNHC0KfErKXQUHT8Cr9/7IUXSXByufGC0F4y8ijxKF1uMR7xv3muEV1kvwLodKP/ygBHpRVsdMyTDMRguuYfBiueisVLAYU1EqnLPShgHKSFC8pIMAEFN81ZKDgWEb9RW2dUL43LMEFgU57RKqyfxLiH3yVfMOUv+AXjLJtmI5ZOcUhfnjyRTCttTLSlHlY1GkaEKwQAsls5AIsrGNTka6clFpSXo0b/+AeQfKCmiHFEpE6XBi5u6XUA1nzlKsqUMJ5D6PM564XCUHYEOv0vrPegdMvnVCWSehWOSrX1iKqx0zPQak8x0YYipvHNYi7hjrGr8YbBS9Ht0cUSMrz58yDggfRt+fMT0SakXsgh7DVNFOgSmR7DQxYA7GI5ECqDNFeKZbxdiTrPnDxhc5KNDfkSKwTCFy/LHTqluPuMt+oSEk8SnoOZh3bMIyM/fezod3mfNEPCGjdITbYMMZAtO9xuQE0jnrCJ5SdtUQsK2AswJo/XCIcGl4wzMfaz9YedQM6RLw+e4qYcQVrRScCJ5716oc+JTshTpfukGYe0zYfJbnkJh9y1U41ZhjHgvbZ29QHzZwrinqgje78KwluaOAeptYsWpKgSvuvOQIC+H2Uczga9W3NklYjGK8oWXVtl/iUH9OIjTaPmI1hUg1NZgqn2mXd9UyvExfbyI=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(53546011)(31686004)(6916009)(83380400001)(7416002)(16576012)(316002)(2906002)(38100700002)(86362001)(54906003)(186003)(26005)(5660300002)(8936002)(66476007)(66556008)(31696002)(508600001)(66946007)(2616005)(36756003)(8676002)(6486002)(956004)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VlE5eDRYWkFtZzNkM0toTVhVOFN1Qm9SRjc5aGk3aG5aa1UyNHhHMXlFWFdi?=
 =?utf-8?B?WFZ2VWtjT2VNaEhQeDdGZXZSOW1sM1JlZUxETHJZdXBEMm9XckxxSjl1OWFD?=
 =?utf-8?B?NEVVcHh4enJJWUVjWVh4a3ROdlovQ2Qvd2YwbDV1T2RmS2FKQnlNUWdBdkph?=
 =?utf-8?B?YVdtTmM0UTQ3UEpzZmhzVUdTNFpmT1c0S21JT0VycVJPSllZOWE0V3NoZm1Y?=
 =?utf-8?B?aytjUjBZbU51ckVCWm85dzhUUGdTMnVlSHowR0tONWV5MzZQSm1IL3NROXNG?=
 =?utf-8?B?bGJYaGIrRk0vbVVrdXhwWWRHS2Q4eno1aW92MCt5eS9YN0d1cHA0N2VKV2x6?=
 =?utf-8?B?Y0VRbXd4MEZMM0Q2WGJwZ0FpUy9oM2N2SjI2K3RPSVV5OTdNd2FQK0xKMXBs?=
 =?utf-8?B?dEt4U3ZUR1hKaDZjcU00OGNpOUdMeUJYRjNONkZ2cm5mYUt6LzJFK1FZUVFQ?=
 =?utf-8?B?L0RaZVVKaURKSjNCRWl0ek8xMEVGOUFZblJseEVlVXNEbktZYjVpcnhpdGN0?=
 =?utf-8?B?ZmQyNE0zaUNvU21ibk14MEJRdXBYcVdtTVptcTRNZ3lVS0t6QkI2enByM2J6?=
 =?utf-8?B?djBtOStCb0E3RWdMd0tzSDZyYVRuZHhHcGlvazZkdXdWZGNOam5FdHpLZU9r?=
 =?utf-8?B?NWhIUzhYNEhJZUZPbUE0cGMvNHd0eE16K3hBZDFZU3p2NUtBZG9EQnVqV0VG?=
 =?utf-8?B?OGJYRnlQSmFCSTEreE5tcm54Qm1MRDkzREVOdUIzRVFsbU92a05zM3psVm14?=
 =?utf-8?B?NVNMS2dhN0NmMU9RWjlNUGlSZGcvS2lFeElQM05Jbm43L2sxRmVDa2xjTE5s?=
 =?utf-8?B?cExaeVlNVk11T0wvdUZaZjB4dTRQYjVVSWh0MnErRTlYR0JMTGUwTEk5WDY0?=
 =?utf-8?B?azZ3TEpkdG9JZHlLdjR1SU1IWFBNaDBrTjlIalo2ajJOVG1SQkcreXVNZXcr?=
 =?utf-8?B?aUlzMVRaY1JzcG9Od1JYZ3MrL3FwWVVpZ2p4bXg4TDZrSm1FNXRqb2M2M3ps?=
 =?utf-8?B?bVBuWC9jYXdIT0RIWVJ5R1dBQkFLK0Y1Vy9xcG8wUnVaN0UzUWc1ZXRWeUh0?=
 =?utf-8?B?MEFNVndTemtocWlaWTdrVFdkWHBSQ2NYUURFL2hmR2cxWVE0NElCQ0lhQ3Vq?=
 =?utf-8?B?SGFCY0ZTbkIzK3N6aW1TbGdZemRMc2l4QlRZTVo3ZTh2NUFCNTdYNjZxMHRu?=
 =?utf-8?B?RzZ4UmVhcDQ1MEsySk1NeE9tbUxvYytTR21STUl6bkV3a3hJQ1NFcElkSjZF?=
 =?utf-8?B?dDJncHNYWllHZjlpRy8vVFBLWVRUN2s2ajVPRkJwQ1E0d3Vib0hJeEx2TW1t?=
 =?utf-8?B?Mkx3VjNhY2FvdGNpNHlySjhFOG1Hb01DNUVEWEFVdEkvODdaU2JsMzRXdWFr?=
 =?utf-8?B?c0NMZ0xjb20rNDU4VHNRSUVNek9xQ2hTWnREOVZ5ZFp6NU1XWHFINS94Qm5C?=
 =?utf-8?B?OTJGTEVvVG1SYkVibWhXaTZrcno5SnRzcU1QbUNrMVRTcHJQZkM5WFJYVHlG?=
 =?utf-8?B?Wk5WUWpnTnZ3Tm9STERyV3k5dllnZEtkaVdzZHRURHJXVTlhWnR3ZXV3U2Zl?=
 =?utf-8?B?OXlJMm9keEdobTNhT3RjM29BQ1lTMkN1bUhIRlRHelNHaWxHUUhwS2syKytY?=
 =?utf-8?B?czlyWXJzV0xKamxFK293bTNQWTBmYm1ya0liL1dRbHJ3d2IzQWFDdU90UUI2?=
 =?utf-8?B?VXpMSnN5U2M5NU5SaEpWNDFBcnRmMEV2aW93YXQvcmxId3hGZDYzMGhDaFNu?=
 =?utf-8?Q?1MEr1XJWQw8UNCLDDKvfM6Q9+waFmHAGgXBKksV?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e334a0a0-618e-4458-d4b6-08d983fc9ba9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:25:21.7320
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: x2XpHs/aqKbOAUEAWVRxT/+Uu2vRWpgz+ZTNkIT3Q8aORA5VDnNAHmMz1qkko+m/aKzaQjJbsZqa3bVOjls5cA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7151

On 30.09.2021 11:35, Oleksandr Andrushchenko wrote:
> On 30.09.21 11:53, Jan Beulich wrote:
>> On 30.09.2021 09:52, Oleksandr Andrushchenko wrote:
>>> --- a/xen/drivers/passthrough/pci.c
>>> +++ b/xen/drivers/passthrough/pci.c
>>> @@ -889,6 +889,31 @@ int pci_remove_virtual_device(struct domain *d, const struct pci_dev *pdev)
>>>       xfree(vdev);
>>>       return 0;
>>>   }
>>> +
>>> +/*
>>> + * Find the physical device which is mapped to the virtual device
>>> + * and translate virtual SBDF to the physical one.
>>> + */
>>> +bool pci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
>>> +{
>>> +    struct vpci_dev *vdev;
>> const (afaict)
> Ok
>>
>>> +    bool found = false;
>>> +
>>> +    pcidevs_lock();
>>> +    list_for_each_entry ( vdev, &d->vdev_list, list )
>>> +    {
>>> +        if ( vdev->sbdf.sbdf == sbdf->sbdf )
>>> +        {
>>> +            /* Replace virtual SBDF with the physical one. */
>>> +            *sbdf = vdev->pdev->sbdf;
>>> +            found = true;
>>> +            break;
>>> +        }
>>> +    }
>>> +    pcidevs_unlock();
>> As per the comments on the earlier patch, locking as well as placement
>> may need reconsidering.
> Other then that do you have other comments on this?

Iirc this was the only thing here. But I haven't got around to look
at patches 4-9 yet ...

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:26:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:26:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199731.353964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtGO-0003Iq-5S; Thu, 30 Sep 2021 10:26:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199731.353964; Thu, 30 Sep 2021 10:26:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtGO-0003Ij-1e; Thu, 30 Sep 2021 10:26:04 +0000
Received: by outflank-mailman (input) for mailman id 199731;
 Thu, 30 Sep 2021 10:26:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVtGL-0003IV-R3; Thu, 30 Sep 2021 10:26:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVtGL-0005lF-Jw; Thu, 30 Sep 2021 10:26:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVtGL-0004Zg-An; Thu, 30 Sep 2021 10:26:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVtGL-0005QY-AD; Thu, 30 Sep 2021 10:26:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TsDPQyW8vSNTJZVV/xO/TWYXGVeEERa6vk/yTcup7yw=; b=oqoKdk2dPyetehjEEbFSyTOSlP
	r4BXO+XX9FKpzeFsNcv57FDQCkMb7kCmaLQZeTkL6nHABJMzU7+0LQRFed4nAb8QKhmQ+32wOIbmC
	N3bjO9JNc+iPFuBPDWpNVGrhWrPzfmWjpqDzxqkSDYiTiaST3alyyA1PXnZuerlZKx1k=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165320-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165320: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=02d5e016800d082058b3d3b7c3ede136cdc6ddcb
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 10:26:01 +0000

flight 165320 linux-linus real [real]
flight 165332 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/165320/
http://logs.test-lab.xenproject.org/osstest/logs/165332/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-xsm 18 guest-start/debian.repeat fail pass in 165332-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 152332
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 152332
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                02d5e016800d082058b3d3b7c3ede136cdc6ddcb
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  425 days
Failing since        152366  2020-08-01 20:49:34 Z  424 days  741 attempts
Testing same since   165320  2021-09-29 20:41:33 Z    0 days    1 attempts

------------------------------------------------------------
7349 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 fail    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2274884 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:26:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:26:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199740.353978 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtGy-0003xV-KO; Thu, 30 Sep 2021 10:26:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199740.353978; Thu, 30 Sep 2021 10:26:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtGy-0003xO-H6; Thu, 30 Sep 2021 10:26:40 +0000
Received: by outflank-mailman (input) for mailman id 199740;
 Thu, 30 Sep 2021 10:26:39 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVtGx-0003xC-DO
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:26:39 +0000
Received: from mx0a-0039f301.pphosted.com (unknown [148.163.133.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 04f08620-6463-480c-aa20-9bcf9e789e86;
 Thu, 30 Sep 2021 10:26:38 +0000 (UTC)
Received: from pps.filterd (m0174678.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18UAEEXM006447; 
 Thu, 30 Sep 2021 10:26:36 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 by mx0a-0039f301.pphosted.com with ESMTP id 3bdba281ps-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 10:26:35 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB7425.eurprd03.prod.outlook.com (2603:10a6:20b:260::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Thu, 30 Sep
 2021 10:26:33 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 10:26:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04f08620-6463-480c-aa20-9bcf9e789e86
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E3OVLi6bkRblPVHNZbv4G6c9zAOyJyG/MGz4cdRNd3NxuR3kAOlYliSCM5UtaYbEXAUwzrtNilu55RGF0nzxQKkK2T1hVfIFERUf0TMlkQAgz74jqVXEb4fw+cqsRoXefdfOF9ut8TkQ4M4IBnSXnYrWNtm9avvxcp3xXOcrJYc7e1jb5S3dEX+n2zpwQFRk6uwZhUx1+w8fO4d1WNjz2+hEgrsN0d+H4B/WItNNye2N2JkiIsx5G2f4itj5/hE1EQbjV+GtyHd5WXce3Dq2zqFJTE+IUnnRZhBHth83cjSg7Fa5DoK8OaDV3fyIuR3hWsaf1Tn52YAUqn6q+VqBRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=gr6/izDEWd/J5qmLzgFi9em5E0EfdM6Ox5sG+11pNEU=;
 b=CkfEj2NT17lmKWpzlf7q5JtkUYvC97EIYXIfFQdA06kbAnqRLtVXXZZJ7mjd5HdCSgXMdEFxrCJuJ520mLPFiXV22coSBzFebQhL3EmjWpxyxqnDua6S6HH/tCfN+jSczBy5A1U2YghaW80k41o+LcP8cBGnGZWanWN9Hx6ogsyzowgaWU+f49Yy5+dzscSLyo0bIesqcC2uNpYUiNZUBMYIMuXpz5hyMR/DRqIIi5Q9dUTMNLkqLT3+DR7swXM1rYXlQfTngbNkqy/lfDmrqAYVDCfy677u2rS0i14sMvaDOVfRPz6G0K2l+NPyOMp6HRf9wAr0jg4pgQrZdGJhTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=gr6/izDEWd/J5qmLzgFi9em5E0EfdM6Ox5sG+11pNEU=;
 b=W6rqw4VDugOowYSa/eF5Ug3YLnhHuTvb8rKzPsANXzb91x0XHGe3AD6DE5q0PxCtZt+XVBoJkKhIaNK3Tarssc1CbNPG4qkn8KPiDlJr/Q203dewbDPYKzcnr/k+zYUeKY3epusDb0h+FIbOrPSsekPCmbVNMK9JDs8Ng0S0phDne4YPeAwJLvCY1Pu6siBod2GQGg9yQYg+gHICmrhulZF/2atcC4465mVLxRefhpVUy7zAe1VuzYF8p24/l3Wf4frNjaJbbbDCbBCx3Jogwgbp0FRZaibxhBaVcvxtPr8zj7fZQsBKQUSbe1dS8ZkM3SiXSh1r98DYMbOO3JwpaQ==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Topic: [PATCH v3 10/11] vpci: Add initial support for virtual PCI bus
 topology
Thread-Index: AQHXtdAlOCu2AnkIOUC4Prsocox3squ8RSWAgAAMGACAAA2QAIAAANkA
Date: Thu, 30 Sep 2021 10:26:33 +0000
Message-ID: <939859eb-8358-5363-337b-926b13324350@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-11-andr2000@gmail.com>
 <ef2c008f-7682-1549-8d27-712b04fbd84f@suse.com>
 <603e2cdf-748f-f18b-99f4-44ddf3c0b9c1@epam.com>
 <84368ff1-64f5-418f-6a49-5098b7edf119@suse.com>
In-Reply-To: <84368ff1-64f5-418f-6a49-5098b7edf119@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f97947b5-0227-4631-042e-08d983fcc675
x-ms-traffictypediagnostic: AM9PR03MB7425:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB7425DC8FB0491AEC5F799B1CE7AA9@AM9PR03MB7425.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 VMI9N0n8tk3CvtenElmRRK7+9B7xN4COtyGDy+RZeezl9dlHZ5iLO152JtggEJbodoCdRLs5g0/3hl9l3Sy82ryGMhz6mQg53cAVx/ktTymTOUFHTXNewlrJU0mT6nG2O8U8uJ8ZDilu/1BUf++Y/8rjQzG5IuoKNuVjBzJ+rUtPvoQzcFQ07qv+EFo0zdo4kP98H+9Ottx3GwhZ6vhj9JCxR/fYqXy0qMcYxoJXR3gUWPq7hkQaOcRfPuXYEdnVWRxdFHNVsxUbxfEC9GNh3fbu/pa+IkjjS1EEPGLL/SugyE7ZzUS4AZDrTQ4//rM5VhF4DvfVBjkttGEvtbcJk4zlVo/LmMWY8QYueQ52kZ/VmhQh4xgRwV3eOeH7a/nZUMYmfjmxjq7/CooQC7Re/YhwYrtHo6PfQQOUygVtJr+BqX01+feKnIgab7pJ5fcSHW3gcL6I1DSkg4AZhGtvJLvumIqk9y5e8Pin/RErQ6F06kohG1HbJXUpDZvZrlTV+QghK8RPHSXu2qTz8FlLRRjc6ILcHC7JmWKLcqBEUZg1EJs1rlU/Cl9IzFFwGkcD2zvaDnFZCue4vPBhY3Tew7aWEP7PLZsp5VrdunpB/NONaAkK86PPDL7lnhyCnKSCyAdome3srjm9d/8bxpcTEgBGNKpIZyUcN1uZtQBrs8m/tQzh4OwQJRaS197LQYEY5DKO/TzC4b0m+ttI6gbGz2MYyqKENPuS0dpGYE6hU6jIhHlxtOsn3rKK9uKh6pgk
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(38100700002)(316002)(8676002)(122000001)(26005)(4326008)(36756003)(53546011)(83380400001)(2906002)(54906003)(6506007)(71200400001)(6916009)(5660300002)(6486002)(64756008)(66946007)(186003)(107886003)(76116006)(55236004)(66556008)(38070700005)(91956017)(31696002)(31686004)(86362001)(2616005)(66476007)(6512007)(508600001)(66446008)(8936002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?SW1HWXFLQlozMVpWdWVmamtJSWFKK1V1SmJ2ci9PNlJMSHZ6TzRKd203clp0?=
 =?utf-8?B?QzZNbmlhTm15aHFuM0RocWs0YjZwR2xZdmVURWpUUnY2bS9WNUhEcCszY0Jp?=
 =?utf-8?B?T3hjeWZBb1EzWFJPczRQMVNvNVN3M0hUNFB3ZnNqL2dGeGs0MHpadUF3NzlC?=
 =?utf-8?B?SWUxZ1dSbnB1aGlUVzJEWm9adXZhOERNS1pjejV5TGlUSkxvUzFUM3YyQ3Zj?=
 =?utf-8?B?NDgxOFRMUkVQM1FteHAzNjRlTXhwQ3ZIWjRFU1E0d3oyTm92QnVGNlF4dTBD?=
 =?utf-8?B?bWNHNWhsVHVYNlpPRG91Y0pCdTRVSTFxeTIzb3ZmYXRUdXl6L3dBT3AvQlVV?=
 =?utf-8?B?S2lsTU1kN29abjBieitXNDdrd3hCcFlGd08vdEtqWlNpZmxCUmJkQU5DUmtr?=
 =?utf-8?B?R0cvT29sR2JURHg5dVlHczllNHowU2ZqVkVFdVVPeGN4T1JidDFDYzdwRkhT?=
 =?utf-8?B?cDNsUTE3d1poSFoza0lWTEQ1djJ2aFUyUzlaSHhwVTBFZUd5aW4xNEZkemJk?=
 =?utf-8?B?T3NTVXJoMTFxVFlWd1FJY0F5M0w5c3Z5c29tNEVWL2FTckMxdzNxYks3cnNS?=
 =?utf-8?B?SmZZS1hlUHZ1aEs3dENBUUJ2c1lpMHJMNzZBNlRUK0tpYjhCYzBnVGE0SCtm?=
 =?utf-8?B?ZVljamFXdjY0NkdzR0l2dE9PRWVKL3J5MVo1S0Nza0dmbkxndWJHY3c3eitm?=
 =?utf-8?B?RSswYk5GcDVRcGY2MnkxYy9uZUo5ajYxaTQ1K2EyaTJ0OEY4a3g4NU0zTGQ1?=
 =?utf-8?B?YXdUdlBmTloyME01QzBqYnlMZFhoaXlqNmx0WWxmaWFOTHZIYVVuSklQSyt5?=
 =?utf-8?B?Wm5xdElEMDRqSDJRYmpSK2dxWmM3aThreHZ4bDYzbERBQXFzZmtjWFFXcFB3?=
 =?utf-8?B?K0l4aDFXN3pBS0Jmc3pSeDNiMlZIc1Q3NjRQV2NjZU56STlvMHdpbzYwMEpv?=
 =?utf-8?B?cjA3YjJocTh1K2Jqb0RGc3Qrb3NYT1BTVCtUMkZsWjFLRkd0VDlsSTRYVEoz?=
 =?utf-8?B?dHZzZldsRFFKbmQzOVdHendMOVpNdC96ejhoZnI4N3k4c0JUeXdRbVVQWDdu?=
 =?utf-8?B?VFFFdXVjL3FCT1ZCazhMUWllVDArbmM3eUZiZmZYZ0F3VG14UFQ2bTFXZytn?=
 =?utf-8?B?RjVXYzVTTDdMcDc5cmtKT2lBdUdRc0UvSTR0LzQreFJ0U0FZc292R1JmSjdl?=
 =?utf-8?B?T0dUNXV5Mnh1RHBLL1RpL2VlUnFtVytFOEJIbzhyNmVTYVhueC9wZGdvVmQy?=
 =?utf-8?B?RDA2Z2NNL3dWMzFacHRFeHV4T0xrOHdaVDNQU3JVa2dlMk5kWEN1SWt3K1o5?=
 =?utf-8?B?d3MwU1pIanZmeVVBZStsamhVWHVpOFNKYkM3bTduMWdFM3JKMW05Y283V0s3?=
 =?utf-8?B?Rk11TGhpRURMdUJZWUd3MmVzeVhYbzY1YnU2SnJTZWZvQXo5U3BEdXdiS1FL?=
 =?utf-8?B?b3lCWDJ5ZCs0QWlFM1NURWtIUkphZlduZmt4anBtVGxiRkdSanFKT3RqWi94?=
 =?utf-8?B?SUJyR3p6OElQSGR3eGpFZmlHMkdGQk9kbjM0VFRuU1RDSVFOcTMwdmVmZ1Y0?=
 =?utf-8?B?YmtGMWQxRDc3dng1VHlqRGRvVk9aYkhBc2cxOHJMb3RpdUdoS1pucjdROWVh?=
 =?utf-8?B?ZDFiTjJLaVlCd25rVUltVFM4WkppYzJtR0dDRjUzSG5NRXJoVy9pVnU5emIx?=
 =?utf-8?B?UUZrVFJ1RUFUbENJV2dwNzFKNU1hS2FLZ3Y1WEt1OHdqeThuTW1DWjFhaW9L?=
 =?utf-8?B?ODRvd2VxWW5TNDJMYjZaUWZhaEFJTTRwVGt4N0VMeUwzYzhvVE9JUXlMb0dz?=
 =?utf-8?B?d0pWdk9MbnZVYmhqbmRlUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B6C715EEF6FE55498778AC8575B143DE@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f97947b5-0227-4631-042e-08d983fcc675
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 10:26:33.2717
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tFifY7LdZ015TtTmlm4xcuXChwvDRu10K5gjhxicTMy5Fkx+a7hvTx9TRAm/9N2KkTSsOXOr6E8L6RGg1yZsBu7BLN7huq/mVSYnQGt9G9PWgreUy3MOtY9LTBdNeI40
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7425
X-Proofpoint-ORIG-GUID: O36zuhWDuTfCoEl2WmBjYvKz-R1JW7Uv
X-Proofpoint-GUID: O36zuhWDuTfCoEl2WmBjYvKz-R1JW7Uv
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_03,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0
 impostorscore=0 suspectscore=0 lowpriorityscore=0 malwarescore=0
 adultscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 bulkscore=0
 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300064

DQoNCk9uIDMwLjA5LjIxIDEzOjIzLCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAxMTozNCwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBPbiAzMC4wOS4yMSAx
MTo1MSwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4gT24gMzAuMDkuMjAyMSAwOTo1MiwgT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IC0tLSBhL3hlbi9kcml2ZXJzL3Bhc3N0aHJv
dWdoL3BjaS5jDQo+Pj4+ICsrKyBiL3hlbi9kcml2ZXJzL3Bhc3N0aHJvdWdoL3BjaS5jDQo+Pj4+
IEBAIC04MzEsNiArODMxLDY2IEBAIGludCBwY2lfcmVtb3ZlX2RldmljZSh1MTYgc2VnLCB1OCBi
dXMsIHU4IGRldmZuKQ0KPj4+PiAgICAgICAgcmV0dXJuIHJldDsNCj4+Pj4gICAgfQ0KPj4+PiAg
ICANCj4+Pj4gKyNpZmRlZiBDT05GSUdfSEFTX1ZQQ0lfR1VFU1RfU1VQUE9SVA0KPj4+IE1heSBJ
IGFzayB3aHkgdGhlIGNvZGUgZW5jbG9zZWQgYnkgdGhpcyBjb25kaXRpb25hbCBoYXMgYmVlbiBw
dXQgaGVyZQ0KPj4+IHJhdGhlciB0aGFuIHVuZGVyIGRyaXZlcnMvdnBjaS8/DQo+PiBJbmRlZWQg
dGhpcyBjYW4gYmUgbW92ZWQgdG8geGVuL2RyaXZlcnMvdnBjaS92cGNpLmMuDQo+PiBJJ2xsIG1v
dmUgYW5kIHVwZGF0ZSBmdW5jdGlvbiBuYW1lcyBhY2NvcmRpbmdseS4NCj4+Pj4gK3N0YXRpYyBz
dHJ1Y3QgdnBjaV9kZXYgKnBjaV9maW5kX3ZpcnR1YWxfZGV2aWNlKGNvbnN0IHN0cnVjdCBkb21h
aW4gKmQsDQo+Pj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBjb25zdCBzdHJ1Y3QgcGNpX2RldiAqcGRldikNCj4+Pj4gK3sNCj4+Pj4gKyAgICBzdHJ1
Y3QgdnBjaV9kZXYgKnZkZXY7DQo+Pj4+ICsNCj4+Pj4gKyAgICBsaXN0X2Zvcl9lYWNoX2VudHJ5
ICggdmRldiwgJmQtPnZkZXZfbGlzdCwgbGlzdCApDQo+Pj4+ICsgICAgICAgIGlmICggdmRldi0+
cGRldiA9PSBwZGV2ICkNCj4+Pj4gKyAgICAgICAgICAgIHJldHVybiB2ZGV2Ow0KPj4+PiArICAg
IHJldHVybiBOVUxMOw0KPj4+PiArfQ0KPj4+IE5vIGxvY2tpbmcgaGVyZSBvciAuLi4NCj4+Pg0K
Pj4+PiAraW50IHBjaV9hZGRfdmlydHVhbF9kZXZpY2Uoc3RydWN0IGRvbWFpbiAqZCwgY29uc3Qg
c3RydWN0IHBjaV9kZXYgKnBkZXYpDQo+Pj4+ICt7DQo+Pj4+ICsgICAgc3RydWN0IHZwY2lfZGV2
ICp2ZGV2Ow0KPj4+PiArDQo+Pj4+ICsgICAgQVNTRVJUKCFwY2lfZmluZF92aXJ0dWFsX2Rldmlj
ZShkLCBwZGV2KSk7DQo+Pj4gLi4uIGluIHRoaXMgZmlyc3QgY2FsbGVyIHRoYXQgSSd2ZSBtYW5h
Z2VkIHRvIHNwb3Q/IFNlZSBhbHNvIGJlbG93IGFzDQo+Pj4gdG8gdXAgdGhlIGNhbGwgdHJlZSB0
aGUgcGNpZGV2cy1sb2NrIGJlaW5nIGhlbGQgKHdoaWNoIGF0IHRoZSB2ZXJ5DQo+Pj4gbGVhc3Qg
eW91IHdvdWxkIHRoZW4gd2FudCB0byBBU1NFUlQoKSBmb3IgaGVyZSkuDQo+PiBJIHdpbGwgbW92
ZSB0aGUgY29kZSB0byB2cGNpIGFuZCBtYWtlIHN1cmUgcHJvcGVyIGxvY2tpbmcgdGhlcmUNCj4+
Pj4gKyAgICAvKiBFYWNoIFBDSSBidXMgc3VwcG9ydHMgMzIgZGV2aWNlcy9zbG90cyBhdCBtYXgu
ICovDQo+Pj4+ICsgICAgaWYgKCBkLT52cGNpX2Rldl9uZXh0ID4gMzEgKQ0KPj4+PiArICAgICAg
ICByZXR1cm4gLUVOT1NQQzsNCj4+PiBQbGVhc2UgYXZvaWQgb3Blbi1jb2RpbmcgbGl0ZXJhbHMg
d2hlbiB0aGV5IGNhbiBiZSBzdWl0YWJseSBleHByZXNzZWQuDQo+PiBJIGZhaWxlZCB0byBmaW5k
IGEgc3VpdGFibGUgY29uc3RhbnQgZm9yIHRoYXQuIENvdWxkIHlvdSBwbGVhc2UgcG9pbnQNCj4+
IG1lIHRvIHRoZSBvbmUgSSBjYW4gdXNlIGhlcmU/DQo+IEkgd2Fzbid0IGhpbnRpbmcgYXQgYSBj
b25zdGFudCwgYnV0IGF0IGFuIGV4cHJlc3Npb24uIElmIHlvdSBncmVwLCB5b3UNCj4gd2lsbCBm
aW5kIGUuZy4gYXQgbGVhc3Qgb25lIGluc3RhbmNlIG9mIFBDSV9GVU5DKH4wKTsgSSdkIHN1Z2dl
c3QgdG8NCj4gdXNlIFBDSV9TTE9UKH4wKSBoZXJlLg0KR3JlYXQsIHdpbGwgdXNlIHRoaXMuIEl0
IGlzIGluZGVlZCBkb2VzIHRoZSBqb2IgaW4gYSBjbGVhciB3YXkuDQpUaGFuayB5b3UhIQ0KPiAg
IChNeSBydWxlIG9mIHRodW1iIGlzOiBCZWZvcmUgSSB3cml0ZSBhIGxpdGVyYWwNCj4gbnVtYmVy
IGFueXdoZXJlIG91dHNpZGUgb2YgYSAjZGVmaW5lLCBhbmQgbm90IDAgb3IgMSBvciBzb21lIHN1
Y2gNCj4gc3RhcnRpbmcgYSBsb29wLCBJIHRyeSB0byB0aGluayBoYXJkIGhvdyB0aGF0IG51bWJl
ciBjYW4gaW5zdGVhZCBiZQ0KPiBleHByZXNzZWQuIFN1Y2ggZXhwcmVzc2lvbnMgdGhlbiBvZnRl
biBhbHNvIHNlcnZlIGFzIGRvY3VtZW50YXRpb24gZm9yDQo+IHdoYXQgdGhlIG51bWJlciBhY3R1
YWxseSBtZWFucywgaGVscGluZyBmdXR1cmUgcmVhZGVycy4pDQpTb3VuZHMgZ29vZA0KPiBKYW4N
Cj4NCj4NClRoYW5rIHlvdSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:30:41 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:30:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199748.353989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtKi-0005YO-4H; Thu, 30 Sep 2021 10:30:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199748.353989; Thu, 30 Sep 2021 10:30:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtKi-0005YH-1I; Thu, 30 Sep 2021 10:30:32 +0000
Received: by outflank-mailman (input) for mailman id 199748;
 Thu, 30 Sep 2021 10:30:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVtKg-0005YB-Vi
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:30:31 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eafe07d5-1fc7-47a0-a471-7fee08cbcce2;
 Thu, 30 Sep 2021 10:30:28 +0000 (UTC)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18UAQ2iW007709;
 Thu, 30 Sep 2021 10:30:25 GMT
Received: from eur03-ve1-obe.outbound.protection.outlook.com
 (mail-ve1eur03lp2054.outbound.protection.outlook.com [104.47.9.54])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bdbfu00j6-2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 10:30:25 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6898.eurprd03.prod.outlook.com (2603:10a6:20b:2d6::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 10:30:21 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 10:30:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eafe07d5-1fc7-47a0-a471-7fee08cbcce2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QjZb5FsyB65Swg6TbSl2sDFaGvdHItqrCEsH2/bgCz+ELWmRHxsp5IOpCRWtgll2uaHzJdk8aydDJtRE1wjXfP3rLeqhHPCWYE3obvwlJQk2icPfBc6tXdBwijSgQb4uIrPAu009MOVmNAO+j3jaEpSKyjUXRwr6DL0kNcCrZhv+eXu7SmsXlP7aGh/36NFm7n6LW+T2Ii3pDDFY23VtI4Okx6q+eNBzKGlqkuM5Aek97MEVa8KCqDXDppmOgq8ngiupUh1vvUc7VEocdpyT8744jTJ+c5VbDRYF26C2Gs+u2qiRhoF3K8hQaWLlcocWtaYHaIbD91zaIQoJoIOfFw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rhNm1HMfp7Ye67xZqdgOJPZkerPQh/LQI45xxCaBtsc=;
 b=ZI96DnC+S8ggl4oLymCybN95SskYEWnLfxVvAELPnI6qlMYDeZ7yHnxw1ZJpBr3EsSd/ou6f1rZaL8v80EUt4LXSM9vvNQWpcIqxFsPQjl8YfjwAXDp2AtklzOcs7otErIP+meFJrQ4sknPxstrvzDhasnLkuXZpH4yIp5+mD1pA447rvGUAweWqQ5RV10uzYDyELxqWyKArkaocIGw7ZWZnGzbHM5Yh0auoyFDCkS3mf6qX7VrznL93GQxCgf3ooyrSRTv6ZFuFPxAo94zdmPrxwNuArCRtsV1BT0Hg73c6Axf5Sg/7N6LP11QOtiillFokBC+vo6UEs2m9RqpW+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=rhNm1HMfp7Ye67xZqdgOJPZkerPQh/LQI45xxCaBtsc=;
 b=c/sWCMLO0jMPxwVP04PwHNTAuwQwe+3tF4jTWDwjv3kc1EBAralgozLQ2NCYVv+RDrLX1XwZjK0VWBPDnWMsinKICVyJtPW/qgelleb5rDEj4lHVpHhlHJCrJ6Js9E3K4Eryq3oyDJVg2RH63z/nUWXOJnMrWCqmGKG38dBf+Aoc2Q8sCCy2qvBETu8oSqu+x4ICfzXNCMKJc+/TtLlYQ6z+Fwm0tW4M/ceHyS8zDoPTU16Ytx94jou+wGVqzP7NPrScgU35kapUfA7Bgd2ALmNFvij0JI/Nq4PhjRPkPIBxOCLg2fQCfv+dve8V9jVXd16Y9hvr2UG57tkbNGGIjg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Bertrand Marquis
	<bertrand.marquis@arm.com>,
        Rahul Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Oleksandr
 Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Re: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Topic: [PATCH v3 02/11] vpci: Add hooks for PCI device assign/de-assign
Thread-Index: 
 AQHXtdAg2qzxZ1hIP02wGTFUbQtoPau8PLsAgAAGuICAAAXDAIAABCmAgAAO+wCAAARbAA==
Date: Thu, 30 Sep 2021 10:30:21 +0000
Message-ID: <5d34be9d-8d70-b2f6-e7bd-c3998b72d8ee@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-3-andr2000@gmail.com>
 <79fa6e5f-ace7-c46a-489a-2e1ba52e53da@suse.com>
 <7e179a48-3127-93e4-cdaa-e5be82cfe770@epam.com>
 <7d035651-6e16-a6cb-6760-e5d077e1a85c@suse.com>
 <91fca69f-1a6f-198d-be27-c6b0ddd62206@epam.com>
 <078e3a01-4498-7894-87dc-31798cfc474b@suse.com>
In-Reply-To: <078e3a01-4498-7894-87dc-31798cfc474b@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8818ebd1-c347-4d9a-0623-08d983fd4e81
x-ms-traffictypediagnostic: AM9PR03MB6898:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB68983D72F722278AD7307B03E7AA9@AM9PR03MB6898.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 W5JITjh0ke1M6N5sqv2B2gztdATqj8nNQoiGoqm4fl+8rTh+AG5LIvXeud0/4jETmKQw1VAoxMNX4dLF/FJN4JN91XUR4Jp/AaewAxXP978S0eTBMC1bord1A8hNLKdoAUziBy/gu+XiafGqZFOFKLHwh4VQRZXOzsI2XdGN+u9K7+wpjNMrvX9dAvrD6pC3GnztKqEFEDNkR4cwjHJI+fXpMmNTISHFUfFklsE3rxlrktelE65ao/E3GKmxPMDiC1t8JSFGHhcpuEgmS4udDAPDSUd3v7qmiuYMP7cHEk8XtNqUiCbnEOSuwXrSuMPDIc2Q1DZpcHl1tqAXRCIYoYFMRFpfN3XvKLG5e7vXb9MEOjMHxZl4AgAF4raAu0far5UHTilM6a8kmZ+Du+drPYB1kCO52gz1bs2wPCFzkxbjw90aTQ/o9YMwDhRqHQR5puuqL67VKI6cQEsqu6scvpPqXWmX5VWXnw0ShK+dOKQ/DeibHYV9UsdhxGfZGY1pc0146JDwrWSivyHsKpZBQLGzhTVfrbxST7Lzp/sFaC/yWrWlILmCqU+t3T1Qf/ox8wUeZ4BRqPFpVWAv623h/W4UzgGbd119EfTva0AbiBeC6cHUsprhGoxj0/jn4atmVjWLL0c8jGbsAU7whGIqbwEMwSlLo96+x0ww8tk3COcVQ0ZePaesxcomLp+ob8d3wXhHOof5XKTy82G6fVI5owFkuCFHgMxdT4jqZJMTCr0C2VPFWJ//vltjnf02le9a
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66476007)(36756003)(508600001)(2616005)(26005)(66946007)(76116006)(38070700005)(66556008)(6486002)(91956017)(8676002)(186003)(6512007)(316002)(83380400001)(8936002)(66446008)(64756008)(54906003)(5660300002)(31686004)(107886003)(2906002)(38100700002)(53546011)(55236004)(6916009)(122000001)(71200400001)(4326008)(31696002)(86362001)(6506007)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?UENjK3NHTUxwZklxdkhxYXluV0U1K0ZMVmpJeE9iNEVOL0JJa242UmdaWDBu?=
 =?utf-8?B?c0ZlVU1XZ0RzbUFUeitmS284T1ZxQk83SWNBcklrL3R4aHFhUGZycllsL0VW?=
 =?utf-8?B?UlU1SzVYempRY2FHWVd6a1RneW5iRDdDd2tmUGl3dG1wRDhYbDhxbzFWalo3?=
 =?utf-8?B?TlFOcVBLWnl0K2oreW10YS9oR3JGSFk4d0xYaGoySGY5M21XSnNxVDM2ZGpa?=
 =?utf-8?B?Zkh1WmMvMUxrUmgyTzN2aWFuTGhXTmVtZGtqdzV4b3EyVXNxVlZJVkk5V2xZ?=
 =?utf-8?B?WXNqckRDTGZKYy9tRmlpeCtPK09Gb284RjhrUXVQaHAzYzQreXA1RFlBcWFz?=
 =?utf-8?B?RXR3ZnVSdSs4d29Nem91eVlIN2dDNWh3RnFhV1BkdWhsdEFWR201K00rRi9Q?=
 =?utf-8?B?R1AyOGUwNWQ2b0pmOTljNTBUVlVMUjBSaE1WV3NpdDRNTXVhNVZLcm85M1gw?=
 =?utf-8?B?dE9sQk0vN1Q5N1I2UlJNQ25uaEpoWVREU2svektvNXp6djlQV3ZVOEZnYU1h?=
 =?utf-8?B?UFhSc3Y1VVhpcWpWNnoxdE15dENDaVNRS2pVeksraEFuangyYUU2Y09MTklQ?=
 =?utf-8?B?N24rQnJXOVNkRThOTkdlU0V2dUhHamxqU0pUUjBGaFpvY0k3QTR6dFMzTC91?=
 =?utf-8?B?Ym1BZFVwMkFCSWtlZC9IVGxCRkZobDB5b25pZ0RVSDY1NEYwdUJVMGd5TGNn?=
 =?utf-8?B?WXJDYzYrZStlazFSZ050dE5Qb3lLbmpCUVRjS3hiSkhtR2M1YytUano5Q004?=
 =?utf-8?B?SUlaQU9jcnAyNzJnOUU2ekNrWFdiOExEVzh6T1pLdGFlcDZNNXdERXhFZm1W?=
 =?utf-8?B?K3R3UVFxUDI5NGhJYlgzQmptYmJYZTBFWUN2SXJHYWJhL1JKakRIYzdQY05r?=
 =?utf-8?B?VURudnNBOE1iZVUxUnFFRnpzTDAvcTZkdG8zekVLSXdrZVdRVFppZVZUaUk1?=
 =?utf-8?B?WUtuNlpMKzFKbDNva3pPdFRkRm4zOGdvTklkZDc1V0pxd0pWMUpqM285T3JQ?=
 =?utf-8?B?ZXpjZ2UvbGFRYmpJMnVManYxRHovYStLWXRtaFZTMGIwNm9wM3Brd1I1T1Vn?=
 =?utf-8?B?Vlcwc2FiaVBobFROQzVaOFdhb25Fa01GTHI0M3gwcmRzSFZKT3RSazFwNEV6?=
 =?utf-8?B?bkVBaDhteUZveStFMWF5d2hQK1pPSStYUVc2blhDNlRJVGtIZ0piVXBESit6?=
 =?utf-8?B?SEZ6SnZqUlhjOHROZkN1NS9NOXlqWm93cUxwZ1R0djJBYjVSM2NJNWltb0pm?=
 =?utf-8?B?U2ErM3pWRUNhOTc1YUZrcmdVS0dKcjJLL3hBSnFyYjZ6ZmM2TWpncVp2M2V2?=
 =?utf-8?B?RFBXb0ppWjUrQ3hOQ3VaR2dGNUVMWnNiVzExYzFENzcwbGtaSGNpb2ZVaCtC?=
 =?utf-8?B?RnMwMGI1QTNMbjlJREVLQnkrNWorWmlpU24zMG1xSXdaTkRMNjhNTlBXdlE4?=
 =?utf-8?B?SWlLcFovUTNVMnBYQU51ZnVsV1I3ZEIxT0k3OFJ4ZExTTGZOSE03SHkxS3FH?=
 =?utf-8?B?M3ZLM1FKbC9YSUlKMEVuTm15NlFBZGwzV2RPazJsYW1QM3lMT2k4amlpZXJS?=
 =?utf-8?B?eTBISXJBUzZBUVI4YllJc0FzY2VML20xNVhjZWlYdnFQcVA5K0dpNE1hb0th?=
 =?utf-8?B?MDVJK1lEYnZwelFNWTU3SzlWOHFuaWJ6OG8wVmZZZjF5QTdRbXNMcVN5QVBw?=
 =?utf-8?B?a1FiMi82OXhSaXNlald1MkI3NEJqaEJIM1VUOERncENXMmgzU0hLZjN1WEJq?=
 =?utf-8?B?Q29XYVhsSlJWK0J6TVVJZm1ZeHE1bzZrU3RlQjFReGZleFVaMnNjYjNwRFZm?=
 =?utf-8?B?RzNndWh0TGwvRUFZQU5Hdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <898DFA1D36149C42AFC9DC024A6D9780@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8818ebd1-c347-4d9a-0623-08d983fd4e81
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 10:30:21.4996
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: KnctWkSRuDbPZp2FtsJd02S9PU0bssADKs5hCxT6p43Gaw3PYQl0xxSPPK1ApBjiN+x/p8pdFUduaxm2wE7v7N+M6lWYaqi3HxDE4dWd8dak/zipE8xeXtQrf0ybEsG0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6898
X-Proofpoint-ORIG-GUID: 3yOCiiGoztPNMBlvcdQUXzZIpOfOy2oq
X-Proofpoint-GUID: 3yOCiiGoztPNMBlvcdQUXzZIpOfOy2oq
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_03,2021-09-29_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0
 suspectscore=0 clxscore=1015 bulkscore=0 malwarescore=0 mlxlogscore=999
 lowpriorityscore=0 priorityscore=1501 phishscore=0 adultscore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2109300065

DQoNCk9uIDMwLjA5LjIxIDEzOjE0LCBKYW4gQmV1bGljaCB3cm90ZToNCj4gT24gMzAuMDkuMjAy
MSAxMToyMSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+PiBPbiAzMC4wOS4yMSAx
MjowNiwgSmFuIEJldWxpY2ggd3JvdGU6DQo+Pj4gT24gMzAuMDkuMjAyMSAxMDo0NSwgT2xla3Nh
bmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6DQo+Pj4+IE9uIDMwLjA5LjIxIDExOjIxLCBKYW4gQmV1
bGljaCB3cm90ZToNCj4+Pj4+IE9uIDMwLjA5LjIwMjEgMDk6NTIsIE9sZWtzYW5kciBBbmRydXNo
Y2hlbmtvIHdyb3RlOg0KPj4+Pj4+IEBAIC0xNDI5LDYgKzE0MzMsMTEgQEAgc3RhdGljIGludCBh
c3NpZ25fZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsIHUxNiBzZWcsIHU4IGJ1cywgdTggZGV2Zm4s
IHUzMiBmbGFnKQ0KPj4+Pj4+ICAgICAgICAgICAgIHJjID0gaGQtPnBsYXRmb3JtX29wcy0+YXNz
aWduX2RldmljZShkLCBkZXZmbiwgcGNpX3RvX2RldihwZGV2KSwgZmxhZyk7DQo+Pj4+Pj4gICAg
ICAgICB9DQo+Pj4+Pj4gICAgIA0KPj4+Pj4+ICsgICAgaWYgKCByYyApDQo+Pj4+Pj4gKyAgICAg
ICAgZ290byBkb25lOw0KPj4+Pj4gICAgRnJvbSBhbGwgSSBjYW4gdGVsbCB0aGlzIGlzIGRlYWQg
Y29kZS4NCj4+Pj4gQmVmb3JlIHRoZSBjaGFuZ2UgcmMgd2FzIHNldCBpbiB0aGUgbG9vcC4gQW5k
IHRoZW4gd2UgZmFsbCB0aHJvdWdoDQo+Pj4+IHRvIHRoZSAiZG9uZSIgbGFiZWwuIEkgZG8gYWdy
ZWUgdGhhdCB0aGUgd2F5IHRoaXMgY29kZSBpcyBkb25lIHRoZQ0KPj4+PiB2YWx1ZSBvZiB0aGF0
IHJjIHdpbGwgb25seSByZWZsZWN0IHRoZSBsYXN0IGFzc2lnbm1lbnQgZG9uZSBpbiB0aGUgbG9v
cCwNCj4+Pj4gYnV0IHdpdGggbXkgY2hhbmdlIEkgZGlkbid0IHdhbnQgdG8gY2hhbmdlIHRoZSBl
eGlzdGluZyBiZWhhdmlvciwNCj4+Pj4gdGh1cyAiaWYgKCByYyINCj4+PiByYyBpcyBhbHdheXMg
MCB1cG9uIGxvb3AgZXhpdCwgYWZhaWN0Og0KPj4+DQo+Pj4gICAgICAgZm9yICggOyBwZGV2LT5w
aGFudG9tX3N0cmlkZTsgcmMgPSAwICkNCj4+Pg0KPj4+IEdyYW50ZWQgdGhpcyBpcyB1bnVzdWFs
IGFuZCBoZW5jZSBwb3NzaWJseSB1bmV4cGVjdGVkLg0KPj4gSSB3aWxsIHJlbW92ZSB0aGF0IGNo
ZWNrIHRoZW4uIERvIHdlIHdhbnQgYSBjb21tZW50IGFib3V0IHJjID09IDAsDQo+PiBzbyBpdCBp
cyBzZWVuIHdoeSB0aGVyZSBpcyBubyBjaGVjayBmb3IgcmM/DQo+IFNvIGZhciB3ZSd2ZSBiZWVu
IGRvaW5nIGZpbmUgd2l0aG91dCBzdWNoIGEgY29tbWVudCwgYnV0IEkgd291bGRuJ3QNCj4gb2Jq
ZWN0IHRvIGEgd2VsbCB3b3JkZWQgb25lIGdldHRpbmcgYWRkZWQuDQo+DQo+Pj4+Pj4gLS0tIGEv
eGVuL2RyaXZlcnMvdnBjaS92cGNpLmMNCj4+Pj4+PiArKysgYi94ZW4vZHJpdmVycy92cGNpL3Zw
Y2kuYw0KPj4+Pj4+IEBAIC04Niw2ICs4NiwyOSBAQCBpbnQgX19od2RvbV9pbml0IHZwY2lfYWRk
X2hhbmRsZXJzKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4+Pj4+ICAgICANCj4+Pj4+PiAgICAg
ICAgIHJldHVybiByYzsNCj4+Pj4+PiAgICAgfQ0KPj4+Pj4+ICsNCj4+Pj4+PiArI2lmZGVmIENP
TkZJR19IQVNfVlBDSV9HVUVTVF9TVVBQT1JUDQo+Pj4+Pj4gKy8qIE5vdGlmeSB2UENJIHRoYXQg
ZGV2aWNlIGlzIGFzc2lnbmVkIHRvIGd1ZXN0LiAqLw0KPj4+Pj4+ICtpbnQgdnBjaV9hc3NpZ25f
ZGV2aWNlKHN0cnVjdCBkb21haW4gKmQsIGNvbnN0IHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+Pj4+
Pj4gK3sNCj4+Pj4+PiArICAgIC8qIEl0IG9ubHkgbWFrZXMgc2Vuc2UgdG8gYXNzaWduIGZvciBo
d2RvbSBvciBndWVzdCBkb21haW4uICovDQo+Pj4+PiBDb3VsZCB5b3UgY2xhcmlmeSBmb3IgbWUg
aW4gaG93IGZhciB0aGlzIGNvZGUgcGF0aCBpcyBpbmRlZWQgaW50ZW5kZWQNCj4+Pj4+IHRvIGJl
IHRha2VuIGJ5IGh3ZG9tPyBCZWNhdXNlIGlmIGl0IGlzLCBJJ2QgbGlrZSB0byBmdXJ0aGVyIHVu
ZGVyc3RhbmQNCj4+Pj4+IHRoZSBpbnRlcmFjdGlvbiB3aXRoIHNldHVwX2h3ZG9tX3BjaV9kZXZp
Y2VzKCkuDQo+Pj4+IHNldHVwX2h3ZG9tX3BjaV9kZXZpY2VzIGlzIG5vdCB1c2VkIG9uIEFybSBh
cyB3ZSBkbyByZWx5IG9uDQo+Pj4+IERvbTAgdG8gcGVyZm9ybSBQQ0kgaG9zdCBpbml0aWFsaXph
dGlvbiBhbmQgUENJIGRldmljZSBlbnVtZXJhdGlvbi4NCj4+Pj4NCj4+Pj4gVGhpcyBpcyBiZWNh
dXNlIG9mIHRoZSBmYWN0IHRoYXQgb24gQXJtIGl0IGlzIG5vdCBhIHRyaXZpYWwgdGFzayB0bw0K
Pj4+PiBpbml0aWFsaXplIGEgUENJIGhvc3QgYnJpZGdlIGluIFhlbiwgZS5nLiB5b3UgbmVlZCB0
byBwcm9wZXJseSBpbml0aWFsaXplDQo+Pj4+IHBvd2VyIGRvbWFpbnMsIGNsb2NrcywgcXVpcmtz
IGV0Yy4gZm9yIGRpZmZlcmVudCBTb0NzLg0KPj4+PiBBbGwgdGhlc2UgbWFrZSB0aGUgdGFzayB0
b28gY29tcGxleCBhbmQgaXQgd2FzIGRlY2lkZWQgdGhhdCBhdCB0aGUNCj4+Pj4gbW9tZW50IHdl
IGRvIG5vdCB3YW50IHRvIGJyaW5nIFBDSSBkZXZpY2UgZHJpdmVycyBpbiBYZW4gZm9yIHRoYXQu
DQo+Pj4+IEl0IHdhcyBhbHNvIGRlY2lkZWQgdGhhdCB3ZSBleHBlY3QgRG9tMCB0byB0YWtlIGNh
cmUgb2YgaW5pdGlhbGl6YXRpb24NCj4+Pj4gYW5kIGVudW1lcmF0aW9uLg0KPj4+PiBTb21lIGRh
eSwgd2hlbiBmaXJtd2FyZSBjYW4gZG8gUENJIGluaXRpYWxpemF0aW9uIGZvciB1cyBhbmQgdGhl
biB3ZQ0KPj4+PiBjYW4gZWFzaWx5IGFjY2VzcyBFQ0FNLCB0aGlzIHdpbGwgY2hhbmdlLiBUaGVu
IHNldHVwX2h3ZG9tX3BjaV9kZXZpY2VzDQo+Pj4+IHdpbGwgYmUgdXNlZCBvbiBBcm0gYXMgd2Vs
bC4NCj4+Pj4NCj4+Pj4gVGh1cywgd2UgbmVlZCB0byB0YWtlIGNhcmUgdGhhdCBYZW4ga25vd3Mg
YWJvdXQgdGhlIGRpc2NvdmVyZWQNCj4+Pj4gUENJIGRldmljZXMgdmlhIGFzc2lnbl9kZXZpY2Ug
ZXRjLg0KPj4+IEZhaXIgZW5vdWdoLCBidXQgc2luY2UgSSd2ZSBub3Qgc3BvdHRlZCBhIHBhdGNo
IGV4cHJlc3NpbmcgdGhpcw0KPj4gV2VsbCwgaXQgaXMgYWxsIGluIHRoZSBSRkMgZm9yIFBDSSBw
YXNzdGhyb3VnaCBvbiBBcm0gd2hpY2ggaXMgbWVudGlvbmVkDQo+PiBpbiBzZXJpZXMgZnJvbSBB
cm0gYW5kIEVQQU0gKHBhcnQgMikuIEkgZGlkbid0IG1lbnRpb24gdGhlIFJGQyBpbiB0aGUNCj4+
IGNvdmVyIGxldHRlciBmb3IgdGhpcyBzZXJpZXMgdGhvdWdoLg0KPj4+ICAgIChieQ0KPj4+IGFk
ZGluZyBzdWl0YWJsZSBjb25kaXRpb25hbHMpLCBtYXkgSSBhc2sgdGhhdCB5b3UgZG8gc28gaW4g
eWV0IGFub3RoZXINCj4+PiBwYXRjaCAodW5sZXNzIEkndmUgb3Zlcmxvb2tlZCB3aGVyZSB0aGlz
IGdldHMgZG9uZSk/DQo+PiBDb3VsZCB5b3UgcGxlYXNlIGVsYWJvcmF0ZSBtb3JlIG9uIHdoaWNo
IGNvbmRpdGlvbmFscyB5b3UgYXJlDQo+PiB0YWxraW5nIGFib3V0IGhlcmU/IEknbSBhZnJhaWQg
SSBkaWRuJ3QgdW5kZXJzdGFuZCB0aGlzIHBhcnQuDQo+IEJ5IHB1dHRpbmcgaXQgaW5zaWRlICNp
ZiBvciBhZGRpbmcgImlmICggIUlTX0VOQUJMRUQoKSApIiwgeW91J2QgbWFrZQ0KPiB2ZXJ5IG9i
dmlvdXMgdGhhdCB0aGUgY29kZSBpbiBxdWVzdGlvbiBpc24ndCB1c2VkLCBhbmQgaGVuY2Ugbm8N
Cj4gaW50ZXJhY3Rpb24gaXNzdWVzIHdpdGggdlBDSSBleGlzdC4NCj4NCj4+Pj4+PiArICAgIGlm
ICggaXNfc3lzdGVtX2RvbWFpbihkKSB8fCAhaGFzX3ZwY2koZCkgKQ0KPj4+Pj4+ICsgICAgICAg
IHJldHVybiAwOw0KPj4+Pj4+ICsNCj4+Pj4+PiArICAgIHJldHVybiAwOw0KPj4+Pj4+ICt9DQo+
Pj4+Pj4gKw0KPj4+Pj4+ICsvKiBOb3RpZnkgdlBDSSB0aGF0IGRldmljZSBpcyBkZS1hc3NpZ25l
ZCBmcm9tIGd1ZXN0LiAqLw0KPj4+Pj4+ICtpbnQgdnBjaV9kZWFzc2lnbl9kZXZpY2Uoc3RydWN0
IGRvbWFpbiAqZCwgY29uc3Qgc3RydWN0IHBjaV9kZXYgKmRldikNCj4+Pj4+PiArew0KPj4+Pj4+
ICsgICAgLyogSXQgb25seSBtYWtlcyBzZW5zZSB0byBkZS1hc3NpZ24gZnJvbSBod2RvbSBvciBn
dWVzdCBkb21haW4uICovDQo+Pj4+Pj4gKyAgICBpZiAoIGlzX3N5c3RlbV9kb21haW4oZCkgfHwg
IWhhc192cGNpKGQpICkNCj4+Pj4+PiArICAgICAgICByZXR1cm4gMDsNCj4+Pj4+PiArDQo+Pj4+
Pj4gKyAgICByZXR1cm4gMDsNCj4+Pj4+PiArfQ0KPj4+Pj4+ICsjZW5kaWYgLyogQ09ORklHX0hB
U19WUENJX0dVRVNUX1NVUFBPUlQgKi8NCj4+Pj4+IEF0IHRoaXMgcG9pbnQgb2YgdGhlIHNlcmll
cyAjaWZkZWYgaXMgdGhlIGxlc3MgcHJlZmVyYWJsZSB2YXJpYW50IG9mDQo+Pj4+PiBhcnJhbmdp
bmcgZm9yIGRlYWQgY29kZSB0byBnZXQgY29tcGlsZWQgb3V0Lg0KPj4+PiBXaGF0IGlzIHRoYXQg
b3RoZXIgcHJlZmVyYWJsZSB3YXkgdGhlbj8NCj4+PiAiaWYgKCAhSVNfRU5BQkxFRCgpICkiIGFz
IEkgZGlkIGFscmVhZHkgcG9pbnQgb3V0IHRvIHlvdSB5ZXN0ZXJkYXkgaW4NCj4+PiByZXBseSB0
byB2MiBvZiBwYXRjaCAxMCBvZiB0aGlzIHZlcnkgc2VyaWVzLg0KPj4gUGxlYXNlIHNlZSBiZWxv
dw0KPj4+Pj4gICAgIEkgZXhwZWN0IGxhdGVyIHBhdGNoZXMNCj4+Pj4+IHdpbGwgY2hhbmdlIHRo
YXQ/DQo+Pj4+IE5vLCBpdCBpcyBnb2luZyB0byBiZSB0aGlzIHdheSBhbGwgdGhlIHRpbWUNCj4+
PiBUaGUgcXVlc3Rpb24gd2Fzbid0IHdoZXRoZXIgeW91IHN3aXRjaCBhd2F5IGZyb20gdGhlICNp
ZmRlZi1zLCBidXQNCj4+PiB3aGV0aGVyIGxhdGVyIHBhdGNoZXMgbGVhdmUgdGhhdCBhcyB0aGUg
b25seSBjaG9pY2UgKGF2b2lkaW5nIGJ1aWxkDQo+Pj4gYnJlYWthZ2UpLg0KPj4gWWVzLCB0aGUg
Y29kZSBpcyBnb2luZyB0byBhbHdheXMgcmVtYWluIGlmZGVmJ2VkLCBzbyB3ZSBkb24ndCBoYXZl
DQo+PiBkZWFkIGNvZGUgZm9yIHg4NiAoYXQgbGVhc3QpLg0KPj4gU28sIGRvZXMgdGhlIGFib3Zl
IG1lYW4gdGhhdCB5b3UgYXJlIG9rIHdpdGggIiNpZmRlZiBDT05GSUdfSEFTX1ZQQ0lfR1VFU1Rf
U1VQUE9SVCINCj4+IGFuZCB0aGVyZSBpcyBubyBuZWVkIGZvciAiaWYgKCAhSVNfRU5BQkxFRCgp
ICkiPw0KPiBJJ20gYWZyYWlkIHlvdSBzdGlsbCBkaWRuJ3QgdW5kZXJzdGFuZDogImlmICggIUlT
X0VOQUJMRUQoKSApIiBpcw0KPiBhbHNvIGEgd2F5IHRvIG1ha2Ugc3VyZSB0aGVyZSdzIChhbG1v
c3QpIG5vIGRlYWQgY29kZS4gQW5kIHRoaXMgbW9kZWwNCj4gaGFzIHRoZSBhZHZhbnRhZ2UgdGhh
dCB0aGUgY29tcGlsZXIgd291bGQgc3RpbGwgY2hlY2sgYWxsIHRoYXQgY29kZQ0KPiBldmVuIGlu
IHg4NiBidWlsZHMgKHRocm93aW5nIGF3YXkgbW9zdCBvZiBpdCBpbiBvbmUgb2YgaXRzIERDRSBw
YXNzZXMpLA0KPiByZWR1Y2luZyB0aGUgcmlzayBmb3Igc29tZW9uZSBub3Qgcm91dGluZWx5IGRv
aW5nIEFybSBidWlsZHMgdG8NCj4gaW50cm9kdWNlIGEgYnVpbGQgaXNzdWUuDQo+DQo+IEJ1dCBh
cyBzb29uIGEgY29kZSByZWZlcmVuY2VzIHN0cnVjdCBtZW1iZXJzIHdoaWNoIHNpdCBpbnNpZGUg
YW4NCj4gI2lmZGVmLCB0aGF0IGNvZGUgY2FuJ3QgdXNlIHRoaXMgcHJlZmVycmVkIGFwcHJvYWNo
IGFueW1vcmUuIFRoYXQncw0KPiB3aGF0IEkgc3VzcGVjdCBtaWdodCBiZSBoYXBwZW5pbmcgaW4g
c3Vic2VxdWVudCBwYXRjaGVzLCB3aGljaCB3b3VsZA0KPiB0aGVuIGp1c3RpZnkgeW91ciBjaG9p
Y2Ugb2YgI2lmZGVmLg0KVGhpcyBpcyB0aGUga2V5IHRvIG15IG5vdCB1bmRlcnN0YW5kaW5nOiBp
bmRlZWQsIHRoZXJlIGFyZQ0Kc3RydWN0dXJlIG1lbWJlcnMgd2hpY2ggYXJlIGlmZGVmJ2VkIHRo
dXMgcmVuZGVyaW5nIHRoZSBpZGVhIHdpdGgNCklTX0VOQUJMRUQgbm90IGFwcGxpY2FibGU6DQpA
QCAtNDQ0LDYgKzQ0NCwxNCBAQCBzdHJ1Y3QgZG9tYWluDQoNCisjaWZkZWYgQ09ORklHX0hBU19W
UENJX0dVRVNUX1NVUFBPUlQNCivCoMKgwqAgc3RydWN0IGxpc3RfaGVhZCB2ZGV2X2xpc3Q7DQor
wqDCoMKgIC8qDQorwqDCoMKgwqAgKiBDdXJyZW50IGRldmljZSBudW1iZXIgdXNlZCBieSB0aGUg
dmlydHVhbCBQQ0kgYnVzIHRvcG9sb2d5DQorwqDCoMKgwqAgKiB0byBhc3NpZ24gYSB1bmlxdWUg
U0JERiB0byBhIHBhc3NlZCB0aHJvdWdoIHZpcnR1YWwgUENJIGRldmljZS4NCivCoMKgwqDCoCAq
Lw0KK8KgwqDCoCBpbnQgdnBjaV9kZXZfbmV4dDsNCisjZW5kaWYNCg0KPg0KPiBKYW4NCj4NCj4N
ClRoYW5rIHlvdSwNCk9sZWtzYW5kcg==


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:31:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:31:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199755.354000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtLH-00067O-Id; Thu, 30 Sep 2021 10:31:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199755.354000; Thu, 30 Sep 2021 10:31:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtLH-00067H-Dg; Thu, 30 Sep 2021 10:31:07 +0000
Received: by outflank-mailman (input) for mailman id 199755;
 Thu, 30 Sep 2021 10:31:05 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tgyX=OU=redhat.com=berrange@srs-us1.protection.inumbo.net>)
 id 1mVtLF-00066v-ES
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:31:05 +0000
Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id 82e5deac-21d9-11ec-bd43-12813bfff9fa;
 Thu, 30 Sep 2021 10:31:02 +0000 (UTC)
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-201-OXBYxjN3NhejJMUuaXLn1w-1; Thu, 30 Sep 2021 06:30:51 -0400
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
 [10.5.11.11])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 738C01006AA4;
 Thu, 30 Sep 2021 10:30:49 +0000 (UTC)
Received: from redhat.com (unknown [10.39.195.104])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 715E760BD8;
 Thu, 30 Sep 2021 10:30:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82e5deac-21d9-11ec-bd43-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1632997862;
	h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1LdDo/aO4T4/u7SlfRZRwccYmDGejjKHpxqBAJClDwA=;
	b=WC3agA+blTMfyKXE3MxaLNLpbLAJjPknryrhFLwkXuBS0K4mRm/GpGUq3f+b2hER9Kg9NS
	SSyQvyE1xyhRYceMLiSajsIWJ5QpAi0ErfIQrIv49VEgwVdgHfXQqdZc/66GxyFm3lKXaT
	O6lTK9DpKvQ0RRdJbDXOelAsvlNlwKE=
X-MC-Unique: OXBYxjN3NhejJMUuaXLn1w-1
Date: Thu, 30 Sep 2021 11:30:27 +0100
From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
To: P J P <pjp@fedoraproject.org>
Cc: Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <philmd@redhat.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	Thomas Huth <thuth@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= <f4bug@amsat.org>,
	Eric Blake <eblake@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	"qemu-block@nongnu.org" <qemu-block@nongnu.org>,
	Peter Maydell <peter.maydell@linaro.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [RFC PATCH 00/10] security: Introduce
 qemu_security_policy_taint() API
Message-ID: <YVWRw0ZKq3G6P5YP@redhat.com>
Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
References: <20210908232024.2399215-1-philmd@redhat.com>
 <YTnhy/Wo5M1qVs1k@redhat.com>
 <798304472.4432617.1631626227208@mail.yahoo.com>
MIME-Version: 1.0
In-Reply-To: <798304472.4432617.1631626227208@mail.yahoo.com>
User-Agent: Mutt/2.0.7 (2021-05-04)
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
Authentication-Results: relay.mimecast.com;
	auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

On Tue, Sep 14, 2021 at 01:30:27PM +0000, P J P wrote:
> Hello Philippe, all
> 
> >On Thursday, 9 September, 2021, 03:58:40 pm IST, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >On Thu, Sep 09, 2021 at 01:20:14AM +0200, Philippe Mathieu-Daudé wrote:
> >> This series is experimental! The goal is to better limit the
> >> boundary of what code is considerated security critical, and
> >> what is less critical (but still important!).
> >>
> >> This approach was quickly discussed few months ago with Markus
> >> then Daniel. Instead of classifying the code on a file path
> >> basis (see [1]), we insert (runtime) hints into the code
> >> (which survive code movement). Offending unsafe code can
> >> taint the global security policy. By default this policy
> >> is 'none': the current behavior. It can be changed on the
> >> command line to 'warn' to display warnings, and to 'strict'
> >> to prohibit QEMU running with a tainted policy.
> >
> 
> * Thanks so much for restarting this thread. I've been at it intermittently last few
>   months, thinking about how could we annotate the source/module objects.
> 
>    -> [*] https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04642.html
> 
> * Last time we discussed about having both compile and run time options for users
>   to be able to select the qualified objects/backends/devices as desired.

Right, we have multiple different use cases here

 - People building QEMU who want to cut down what they ship to
   minimize the stuff they support, which is outside the security
   guarantee. This can be OS distros, but also any other consumer
   of QEMU
   
   eg. RHEL wants to cut out almost all non-virtualization stuff.
   There is a quirk here though, that RHEL still includes TCG
   which is considered outside the security guarantee. So a
   simple build time "--secure on|off" doesn't do the job on
   its own.

   We need something to let people understand the consequences
   of the build options they are enabling.

   NB, when I talk of build options, I include both configure/meson
   args, and also the CONFIG_* options set in configs/**/*.mak


 - Application developers want to check that they're not using
   stuff outside the security guarantee, even if the distro
   has enable it.  These need to be able to query whether the
   VM they've launched has undesirable configuration choices.


Some people fall into both groups, some people fall into neither
group.


> * To confirm: How/Where is the security policy defined? Is it
>   device/module specific OR QEMU project wide?

Currently our only definition is in the docs

  https://qemu-project.gitlab.io/qemu/system/security.html#security-requirements

Philippe's patch is proposing tagging against internal QEMU objects
of various types.  I further proposed that we expose this in QMP
so it is introspectable.

I think there's scope for doing stuff at build time with configure
args and *mak CONFIG_* options, but haven't thought what that might
look like.

> > IOW, the reporting via QAPI introspetion is much more important
> > for libvirt and mgmt apps, than any custom cli arg / printfs
> > at the QEMU level.
> >
> 
> * True, while it makes sense to have a solution that is conversant with
>   the management/libvirt layers, It'll be great if we could address qemu/cli
>   other use cases too.
> 
> >it feels like we need
> >  'secure': 'bool'
> 
> * Though we started the (above [*]) discussion with '--security' option in mind,
>   I wonder if 'secure' annotation is much specific. And if we could widen its scope.
> --- x ---
> 
> 
> Source annotations: I've been thinking over following approaches
> ===================
> 
> 1) Segregate the QEMU sources under
> 
>       ../staging/      <= devel/experimental, not for production usage
>       ../src/          <= good for production usage, hence security relevant
>       ../deprecated/   <= Bad for production usage, not security relevant 
> 
>    - This is similar to Linux staging drivers' tree.
>    - Staging drivers are not considered for production usage and hence CVE allocation.
>    - At build time by default we only build sources under ../src/ tree.
>    - But we can still have options to build /staging/ and /deprecated/ trees.  
>    - It's readily understandable to end users.

I don't think we should be working in terms of source files at all.
Some files contain multiple distinct bits of functionality that are
not easily separated, and will have distinct security levels. Also
IMHO it is unpleasant to be moving files around in git to when code
switches between levels.  Also there are distinct criteria here,
both security levels, and support levels - there can be stuff which
is fully supported but considered insecure, and stuff that is
deprecated but considered secure. 

> 2) pkgconfig(1) way:
>    - If we could define per device/backend a configuration (.pc) file which is then used
>      at build/run time to decide which sources are suitable for the build or usage.
> 
>    - I'm trying to experiment with this.

For build time configuration, we have a pretty clear set of
toggles between the configure/meson options, and the CONFIG_*
make options.  I don't think we need to complicate things by
trying to add pkg-config into the mix here.

> 3) We annotate QEMU devices/backends/modules with macros which define its status.
>    It can then be used at build/run times to decide if it's suitable for usage.

What is what Philippe's patches are doing in essence

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:40:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:40:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199763.354011 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtUe-0007o4-Es; Thu, 30 Sep 2021 10:40:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199763.354011; Thu, 30 Sep 2021 10:40:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtUe-0007nx-Bp; Thu, 30 Sep 2021 10:40:48 +0000
Received: by outflank-mailman (input) for mailman id 199763;
 Thu, 30 Sep 2021 10:40:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Bos=OU=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVtUd-0007nr-4j
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:40:47 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9e19d629-6f1a-43e1-bcca-2710543771b6;
 Thu, 30 Sep 2021 10:40:44 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e19d629-6f1a-43e1-bcca-2710543771b6
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1632998443;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=n5gJgrNXUS23H91tjeK37aLN1I3GAH4JsrvnGmyMhv4=;
  b=TiL7akgfLH+3u9eQsZzDGTfjA+Vz4JyE8AufUrVBykfw3K1oQaZjFPVy
   Zr59nGQzPXHzKVTjWHpEBl3A2WWyf6tkMLxt0UsFSV5IQYeVxhg6LBCDW
   zSOSk0Uog7e4JXNIJ8HUx6gOu+1bFEJUVNoGc8mOU5tGyrB4jRuovIsTh
   s=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: fWVKFkz/Yl7WtH62gTl9gDwVRx3WGXntpoVc5ChQL/O+fBeO5powsEqoT3IMG747U0CHRylZlg
 OthQLBZp1n5qam61R6r2dpN9bwAw/S7mF798P2A9DueHnwgIHSx5qaqqnSPbGUmyXpF7NIjFe5
 OkCzchmvq4sa2/zpvJPK198xq05ItNIb1oNUs5Dtx6r4drFSJpJbYKrmZ/dru4jbfKwIkWJz6T
 4u6KX4Pkwia9dr1Ws52lNIa51AgV+vd3JkXQJkFDYNcqmhfNj641tU2+sJOCKnkBVLqLiSQCMR
 M+QUH3upmP4xfD+CgFktX755
X-SBRS: 5.1
X-MesageID: 54418356
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:A658V6MFDLo1TFPvrR2ukcFynXyQoLVcMsEvi/4bfWQNrUom3mAOz
 2ofC22OM/3ZZmenf952OoSz/EwEu8PRy4VmTgto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdpJYz/uUGuCJQUNUjMlkfZKhTr6ZUsxNbVU8En552Ek7w7RRbrNA2rBVPSvc4
 bsenOWHULOV82Yc3rU8sv/rRLtH5ZweiRtA1rAMTakjUGz2zhH5OKk3N6CpR0YUd6EPdgKMq
 0Qv+5nilo/R109F5tpICd8XeGVSKlLZFVDmZna7x8FOK/WNz8A/+v9TCRYSVatYozDOlot8k
 45oibmxFCQUB+7FgcRGDRYNRkmSPYUekFPGCX22sMjVxEzaaXr8hf5pCSnaP6VBpLwxWzsXs
 6VFdnZdNXhvhMrvqF6/YsBqit4uM4/AO4QHt2s75TrYEewnUdbIRKCiCdpwgWts35AVQKi2i
 8wxSQBzfA+eQ0V0K1pKGZIenr+0h3DPbGgNwL6SjfVuuDWCpOBr65DyNPLFd9rMQt9a9m6ar
 G/b+2XyAjkBKceSjzGC9xqEh/DNtTP2XpoIE7+1/eIsh0ecrkQWCRYWXF/9puOrh0qWUshab
 UcT/0IGvaU0sUCmUNT5dxm5u2Kf+A4RXcJKFO834x3LzbDbiy6bG2wFQzhpeNEg8sgsSlQC3
 FKTg8ngAzAptbSPUG+c7Z+dtzb0Mi8QRUcAajUNSQ8t6tzqsoY1yBnIS75LLqmxidHkHCDq9
 BqDpiM+mrY7gNYC0uOw+lWvqym3upHDQwox5wPWdmGo9AV0YMiifYPAwVrU9/FbN66CU0KM+
 nMDnqCjAPsmVM/X0nbXGaNUQe/vt63t3CDgbUBHGt4orDn2oFufWIENwwxHZ05wEp1admq8C
 KPMgj956JhWNXqsSKZ4ZYOtFsgnpZTd+cTZuuP8NIQXPckoHOOT1GQ+PRfPgzCFfF0Ey/lXB
 HuNTSq74Z/244xJyyCqD8MUzLMm3Cw3wW67qXvTlE/8iev2iJJ4U9443LqyggIRsP7sTOb9q
 Y832y62J/N3C7aWjs7/q9N7ELzyBSJnba0aUuQOHgJ5HiJoGXs6F9jayq47dopuksx9z7mTp
 CDhAh4IlASu2hUrzDlmjFg4M9sDur4l8BoG0dEEZw70ixDPn671hEvgS3fHVeZ+r7EypRKFZ
 /IEZ9+BEpxypsfvoFwggW3GhNU6LnyD3FvWVwL8OWRXV8MwFmThp46/FiOypXZmM8ZCnZZny
 1FW/liAGsRrqsULJJu+Vc9DOHvr5iVCxLIuARGYSjSREW21mLVXx+XKpqZfC+kHKAnZxyvc0
 ACTABwCovLKrZNz+97M7Z1oZa/1ewenNkYFTWTd85isMizWojiqzYNaCb7adjHBTmLkvq6lY
 LwNnf37NfQGmndMspZ9TOk3nf5vuYO3qu8I1BlgEVXKc0+vVuFqLE6Z0JQdraZK3LJY51e7A
 xrd5tlANLyVE8r5C1pNdhE9Z+GO2KhMyDnf5Pg4Omvg4ypz8ObVWEleJUDU2idcMKF0IMUux
 uJ44Jwa7Am2ixwLNNeaj38LqzTQfyJYC6h+78MUGo7mjAYv22puW52EB3+k+oyLZvVNLlIuf
 m2eip3diukO3UHFaXcySyTAhLIPmZQUtRlW51YePFDVyMHdj/o60RANozQ6SgNZkkdO3+5pY
 zU5MkR0IeOF/ityhdgFVGepQlkTCBqc80336l0IiGyGEBX4Cj2TdDUwabSX4UQU02NAZTwKr
 riXxVHsXSvuYMysjDA5XlRoqqC7QNF8nuEYdBtLwyhR80EGXAfY
IronPort-HdrOrdr: A9a23:LyPz9qN6X5SBOMBcT0j155DYdb4zR+YMi2TDiHoedfUFSKOlfp
 6V8MjztSWVtN4QMEtQ/exoS5PwP080kqQFnrX5XI3SIDUO3VHIEGgM1/qY/9SNIVyZygcZ79
 YcT0EcMqyCMbEZt7eD3ODQKb9Jq7PrgcPY55at854ud3AMV0gJ1XYINu/xKDwOeOApP+tdKH
 PR3Ls8m9L2Ek5nH/hTS0N1ENTrlpnurtbLcBQGDxko5E2nii6p0qfzF1y90g0FWz1C7L8++S
 yd+jaJp5mLgrWe8FvxxmXT55NZlJ/IzcZCPtWFjowwJi/3ggilSYx9U/mpvSwzosuo9FE2+e
 O87SsIDoBW0Tf8b2u1qRzi103J1ysv0WbrzRuijX7qsaXCNXgHIvsEobgcXgrS6kImst05+r
 lMxXilu51eCg6FtDjh5vDTPisa1HackD4Hq6o+nnZfWYwRZPt6tooE5n5YF58GAWbT9J0nKu
 9zF8vRjcwmMm9yV0qp+lWH/ebcGUjaRny9Mw4/U42uonhrdUlCvg4lLJd1pAZYyHo/I6M0rN
 gsfJ4YzI2n46ctHNRA7dw6ML+K41r2MFrx2VKpUCHa/Z48SgXwQr7MkfgIDbKRCdA1JKVbou
 WJbLofjx9oR37T
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54418356"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kRSEeCqQh4GAwjdMCOfGPmvQUWtippdy+SDWecAqRSwtTMjEcEGyUTS1FFj1mQ/7qqlBtaCLy5C9IeDRGwmxTG9K/+JkpyeoGhmjEFWjrHLACiZ9QU4/+JkCehvqhhf72qemik2+h3X/SC+GUcRYxUQ5yCSOUcoHN9yp1ck1t9YMTiViELqstmZlMKh5jNLgfGg4URNLvDcJdtgqxWDFZdjP/SfWkTEL3EOcv7w2rZviOWCTCfOOkyp4M7lqEU0CLk1sCHWYf9XAs66KQUl8Bg6lDaeTRxxtBkZ1C6mVKjUy1XLtGX24k8E90w/O4ApfgmNOPNiBZ6wY1zi847IaCg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=1Xxz7R5B15hALx4NDFcBV82CATwyw1U5VEPiwOZ+KW4=;
 b=hi6oDab/OEjh8iUdtFQLpgozJHDz5oXy1r4bh8N+DqKTlq9uBK21JwuGTKUHnq0j0vJbKwxFgqZ5Tol6zodkUY0FgGfJqNoOeyMDqGFHKFoHTam+9chS7Uu6zlw0lAKjsyqSX9HU+m9FAYnH+QvkOkmHD0ezbtfe4toV2T1eYSuIFhJJVW1YbB/VXbFVufwascYDEbBCjvcHpX5EELjiGePHDYi+WuKLcGZkSs3spePMqOnXifQowD/A8V/uLoZdJZaZWdRb8mxfcwpX2WLO5IhrvyTgmtl3sjti8OviZqDeQOGUJmJGYhDaA3IIpvk8EOzzaLj+GxESH5hSLrJLyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1Xxz7R5B15hALx4NDFcBV82CATwyw1U5VEPiwOZ+KW4=;
 b=tUeDdb/VO0vBW8VB/1UNiMDg+gE9Tnxl820GbEariesMydwb6PUNb5eTpn7oHWpWA0bOyosy9YqvuEj8aRSAmH/MYBonne3ck1p/HwTwtId2SsdDhiNLQ5+0N/uP/CeOxZtmiJDV+7co8v/vh9zpDxphpW2AbRmrZanicyX4Zh0=
To: Michal Orzel <michal.orzel@arm.com>, <xen-devel@lists.xenproject.org>
CC: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>, "Julien
 Grall" <jgrall@amazon.com>
References: <20210930092651.1350-1-michal.orzel@arm.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] xen/arm: Expose the PMU to the guests
Message-ID: <17d393c9-b249-cccf-e360-054b66466143@citrix.com>
Date: Thu, 30 Sep 2021 11:40:18 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930092651.1350-1-michal.orzel@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0123.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:192::20) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0c50fea4-b10a-48a0-7167-08d983feb689
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5648:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5648676AF7859B75C1BA7C3BBAAA9@SJ0PR03MB5648.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: tTnnNRmclIte1vcTKauVh0Ib8ZsVWZzSRDRI2XXdSGJLKQKCQhjLU6zCA+qX682AegC/zZk+aJFJzL6mEwcR8pFSAF4in4idYQ/YcKueDNwA0WRwW/WM/oCd8b6oV5N73sSF6PlebLNSgPmNVsEpALV8zAb6ZR2yAW20aDtAeeBPodEyzMKqOGqlOpmOZgI+Pd3pmFgHQ/LmXjZFfUg51cF5C7FMwccoWN3sajBJOHEd5tPHjXdr/8WPYLOHXVk2PpbTcUfezFYFUuHNsujwhA1ZkBwXr45f8e7FJJFI20nVfC/x70V0PqLYZo2ynvtTCqlIM7K6B557mxFvexcnOljvQnRcnh+1ujTuFKBMRu4mTfvHZIIz4gzChXBDaizSWWsLK0pKygeSpmQKdDtPZAsbOiFaPAXAR9ywaO74KyI7Rs33qwkcVnYhHhiBITPohno9qyDSV1GJMCbjAetAACivMEowv2TLdMSXbuH9u6SKZw5X9b/qnLluKbMOavtzvNsyUTWiZon+ANFB5WaSnaoS7KNxkXlw+sh+jASmg63Ml6G+m0Yaz4cNRbmKerJfyxdX09q2tL99D4Oa8ChlLx+MKhu3nGeddgj+f7egicD08754DMO7Ycd5hUdc2w3WSoeH2JQWq6imNPm/E86Ffn1h9pbc2PGbsRzOu8RRJOplpNjCQbH292xWbyFBsHW0jz57HAPqs86+kBFShhXN0wIjrspG2LNoAlyrb55e/WM=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66556008)(6666004)(66946007)(86362001)(38100700002)(4326008)(36756003)(2906002)(31686004)(5660300002)(186003)(54906003)(316002)(6486002)(8676002)(26005)(83380400001)(2616005)(956004)(55236004)(53546011)(508600001)(66476007)(7416002)(16576012)(31696002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZXFQa1JCWU1tMzFIajZJRnVzZXdvNlBOY3VZNWEwRnV6VUswSm9LY29Ceko3?=
 =?utf-8?B?WmV1Myt6UG9DaDByMThxT0l0OUwwcC9WMmZOTHhZd1lzd05nLzJxdVc1TXBX?=
 =?utf-8?B?RDlqbXhNakV2YmZ2N0xoN1I2ZHJySFFwdTZFRWJLWlVuVkpmcUdsSW9qMG95?=
 =?utf-8?B?bUdqSnFuR3M1V2dqOVpYdmovZFYzbkFacVczVmc3ajkxY244c2tRNFhQaGxI?=
 =?utf-8?B?Nkcwc2xHVjlhWG9ybTdHWCtPMzVFK3E3YkRmTzNTcmZEUlQ3ajJiNGU3RDcw?=
 =?utf-8?B?NVZmVElxYWsyV2tPaDgwNFkzY3ArMzgyV09kNXJKUGFNQnBHOFNGbFIvSk9I?=
 =?utf-8?B?bFVCdFliUXJNdWZpZSs1TDdoaHFLZndhR2xNbkRIdWF6a1lHRU9YVHMrL2ZI?=
 =?utf-8?B?OFZZSnZGeXI0ZlVpOTBocWIycG1zVFp1Mk5GeVlqTDJHbWgxTFdtQm1renll?=
 =?utf-8?B?NkM2S2ljNDI5TVEyRTU3L0pSTGxhMkZnajF0YWlGMFpXWFVYVVFIbERVdDA0?=
 =?utf-8?B?a0huczM0VjdnT3lKMGhsWWJMRGlJaWZvTUFzKzdtWUhXYVgxNXJBb1dIaGIy?=
 =?utf-8?B?cURDRWI3aVpJUjVSbHpETHc2eHNOMDRuUFlTMndXQVR2T0NhN1laUzR2VVNU?=
 =?utf-8?B?U0lyT3hWT3J1QWVFaTY5YlVUVjBUb0J2cmZQK0JKUHJ4T1o5OWx3Y1RmenJi?=
 =?utf-8?B?YmlWZlI0eld2eG1CUFpLOVY2Z0EyRXEveGtvaGxWQk1TVTg0bDZjVytkMC82?=
 =?utf-8?B?andKeXdqYjhseDAwbkdIM2t0WWpWTVhuSTllOWQrdG8yVENoSlhkT09NL09F?=
 =?utf-8?B?QzE3bXYyOUM2dmN5U2c0cDNnTVlqUXhUeVBSb0lDYnpHL1lXcnNqWGNYeU85?=
 =?utf-8?B?bFkwc0lOamhxVmZxQVdGVFllckwxL0NFeGxPZTUxdEVsUjlCdElpVXdKMzhT?=
 =?utf-8?B?bWFiSlA5SUd0b29OUTlHMXFEcEZ5YzhvRzJoMVVvQXNpRW1UVkFWRS85ZDM5?=
 =?utf-8?B?Rlh2a1J0dDROa2VUZ1FVZklQS3pUQnlXWUREaEU3YmFUNnFnUXRnckxWZ21O?=
 =?utf-8?B?OUlEWHRubUpNWi92SFZieStrZVJSQVhGYVpROFJsWHhPaFJGTVV0S054bnBm?=
 =?utf-8?B?Mml3OXNlMWMxSnFMTG1FbUFEa0loblpVN3gxV3pHZUczTWZGTEFUN3lwK0JJ?=
 =?utf-8?B?VDBuOWkzR09IeFRCTC9QOXB2QkJwTnlTbkN5MVk3VzZ3ZUJhamlReHVQWU5V?=
 =?utf-8?B?R0drRGxyUjVGZkFyRXAvVFdITVhhYStNRlVVQkxXV3FnU1lmV2ZmeWczcVB6?=
 =?utf-8?B?ZHA3Y25JSU9IV3V1dlJySTY2WEQ0QkQ5UXppRGlWNEhlc3NVUTFrTHdzWDZ2?=
 =?utf-8?B?Nzh6ZytZK05ZU0hRSXNCVDFtUjlyaTRQNkV5WjdDSHR2bUd3blViU0dXMDBp?=
 =?utf-8?B?T21hOXMya2Q5L2FNMXlsbU5xV3lOMUpMVUN4a01EOTVsVlE0YXdyemd1TG9O?=
 =?utf-8?B?VGdscjhDZjI5OEk3cTY5Qnh3NEZoQVF2cTFZUXBhUUFqc05wQStIdTE2cVBY?=
 =?utf-8?B?eVVrRmtqVUwyaEYzQXdPb1dFbmpsaUh5ZHkzOCtvbW9UNFZUZmdPai9XVTVJ?=
 =?utf-8?B?aVZLeGhCTVd4cXkzZ3lDV3kvLy9IL2VwZXpMSlNneCtKa2trU2VOdit3RkF0?=
 =?utf-8?B?N0o0WVo2cjNuNm1tenNVTk9NQm5hdTY2MjR6clY5b3ZEZ1VDbTJaczZ4Nk9H?=
 =?utf-8?Q?BhmgV26UoI7H1R+NOKBdezMqS2zpQWqcp1h8KNh?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0c50fea4-b10a-48a0-7167-08d983feb689
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:40:25.8367
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Jl3LXuQgWFpaG1+4feYzKuFkAUE2QrBr/4uRc4dggwWvNYLzhTb2/cpeH/1C0HPzlMxIMKxCn2D6skp7LK+5Hgf9th4uQThSC/FwtvxqZbQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5648
X-OriginatorOrg: citrix.com

On 30/09/2021 10:26, Michal Orzel wrote:
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 4b1e3028d2..4a75203b9f 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -2843,6 +2843,18 @@ Currently, only the "sbsa_uart" model is supported=
 for ARM.
> =20
>  =3Dback
> =20
> +=3Ditem B<vpmu=3DBOOLEAN>
> +
> +Specifies whether to enable the access to PMU registers by disabling
> +the PMU traps.
> +
> +This change does not expose the full PMU to the guest.
> +Currently there is no support for virtualization, interrupts, etc.
> +
> +Enabling this option may result in potential security holes.
> +
> +If this option is not specified then it will default to B<false>.

There are rather better ways of phrasing this...

It isn't "maybe security holes".=C2=A0 There are two salient points.

1) vPMU, by design and purpose, exposes system level performance
information to the guest.=C2=A0 Only to be used by sufficiently privileged
domains (however the system admin cares to draw this particular line).

2) Feature is experimental, and thus might explode on you.=C2=A0 Bugfixes
welcome.

> +
>  =3Dhead3 x86
> =20
>  =3Dover 4
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d698..c6694e977d 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -502,6 +502,13 @@
>   */
>  #define LIBXL_HAVE_X86_MSR_RELAXED 1
> =20
> +/*
> + * LIBXL_HAVE_ARM_VPMU indicates the toolstack has support for enabling
> + * the access to PMU registers by disabling the PMU traps. This is done
> + * by setting the libxl_domain_build_info arch_arm.vpmu field.
> + */
> +#define LIBXL_HAVE_ARM_VPMU 1

Please make this generic, not ARM-specific.

The domctl flag is (correctly) common, and x86 could do with this too,
as well as other architectures.

Don't worry about plumbing the x86 side to work - that's a little more
involved, and can be done at a later date.


However, you do need Xen to report the availability of vPMU on the
hardware as a prerequisite.=C2=A0 The toolstack needs to be able to know
whether XEN_DOMCTL_CDF_pmu will be accepted so it can error out in a
useful way on hardware lacking the capabilities.

You probably want to follow the example in
a48066d25c652aeecafba5a3f33e77ad9a9c07f6

> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 96696e3842..a55ceb81db 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -70,9 +70,12 @@ struct xen_domctl_createdomain {
>  #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
>  #define _XEN_DOMCTL_CDF_nested_virt   6
>  #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt=
)
> +/* Should we expose the vPMU to the guest? */
> +#define _XEN_DOMCTL_CDF_pmu           7
> +#define XEN_DOMCTL_CDF_pmu            (1U << _XEN_DOMCTL_CDF_pmu)
> =20
>  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_pmu

Without an adjustment in the Ocaml bindings, the ABI check will fail.

~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 10:43:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 10:43:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199769.354022 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtWn-0008Oq-Sa; Thu, 30 Sep 2021 10:43:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199769.354022; Thu, 30 Sep 2021 10:43:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtWn-0008Oj-Oi; Thu, 30 Sep 2021 10:43:01 +0000
Received: by outflank-mailman (input) for mailman id 199769;
 Thu, 30 Sep 2021 10:43:00 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CD8Y=OU=arm.com=Rahul.Singh@srs-us1.protection.inumbo.net>)
 id 1mVtWm-0008Od-J0
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 10:43:00 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (unknown
 [40.107.4.49]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2d482dc2-21db-11ec-bd44-12813bfff9fa;
 Thu, 30 Sep 2021 10:42:58 +0000 (UTC)
Received: from AS8PR04CA0141.eurprd04.prod.outlook.com (2603:10a6:20b:127::26)
 by AM6PR08MB4040.eurprd08.prod.outlook.com (2603:10a6:20b:a2::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 10:42:56 +0000
Received: from AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:127:cafe::11) by AS8PR04CA0141.outlook.office365.com
 (2603:10a6:20b:127::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 10:42:56 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT030.mail.protection.outlook.com (10.152.16.117) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 10:42:55 +0000
Received: ("Tessian outbound 78bf72cc015a:v103");
 Thu, 30 Sep 2021 10:42:55 +0000
Received: from 3f9faf28cd54.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3E5D68E5-3E72-4A50-AF12-679A41BE5498.1; 
 Thu, 30 Sep 2021 10:42:44 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3f9faf28cd54.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 10:42:44 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com (2603:10a6:20b:39e::10)
 by AM5PR0802MB2561.eurprd08.prod.outlook.com (2603:10a6:203:9b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 10:42:42 +0000
Received: from AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0]) by AS8PR08MB6919.eurprd08.prod.outlook.com
 ([fe80::c8de:afad:1d5a:efd0%7]) with mapi id 15.20.4544.022; Thu, 30 Sep 2021
 10:42:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d482dc2-21db-11ec-bd44-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nSXoGgx7NxmWUe6x2sgiMmzI+SJ40pEEFvsUCgRO5vw=;
 b=ryvTkHXvWK6DI1DN5WyUn3E8YdkRC349R0X+2MMwnfO5Z3VC9b+EKZewnMMkscDDniGtDtpGynNcNucJEf2inxdGmUztTKYqBaq0kZtn4PJeRdCTNi6SP5CvLxUNUZUHFoB4P+CvUdeAhBXuCsOUPFjrnexlgTthNBiN2s4Etoo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ef73f219c3c0a208
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hotV08bMnUlwzIsbgCYuLunbxAvnQ8xE76gIYMMERFUZiDXP8cBC4dHLNQ/IQ5JUj+elnSCOhPOc91hJvxt7ho6Otzl8rxmOaMSk7mjV1p5++zPE6nh+FFZO6NgA0kaK5lKDjaDpuKBjxz8X768bCsc/eVUauxg5coUrPc9pAbBEU1xMwsRtzVN8+I8elSAbo9YGOy/RajqYbstTXNEwJmAPX3Dh27WQW2LMWD127VmOe+zqCGpoTjQCsykhHdgTD4MmXqmPfU5RdU2fDaXBlHZ4lWCbbg2pmZlC2AxV8hRK9xkX3U5sxn1oOYp9iulufXIjG+hMgiZbK3TrIgE39w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=nSXoGgx7NxmWUe6x2sgiMmzI+SJ40pEEFvsUCgRO5vw=;
 b=ezQAYB38W9eSEHTUvVfY/gJFp6yB6r0fzinm7p3b1hXgfgYYgg3NgLCLRrxiPEnPDfDx5mqmeLKfXEYQOC/jW0kgguY25/1k87j0B5uq7gKGUoD7eKKMchW4k8wJ/PGLxTgs98KHl0JNQ1iU3IoJMTYR7M2c48xUsSxFuJiFIzCfV1WAjM3cmFK+FO3GSJwesSHv4MT59vq9TZk8wmgZIZEQGHGP8tPE7aP9z78mmJ3jWb0piodw2GOWcUb5U3BqhWhph2t+F+noKBl77c+FT7y4NZDGjFd18AC63Wj2/RGMxxfUguLr7d+njxdRPgekqPQDsVGrrXA9T0LNpY151Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=nSXoGgx7NxmWUe6x2sgiMmzI+SJ40pEEFvsUCgRO5vw=;
 b=ryvTkHXvWK6DI1DN5WyUn3E8YdkRC349R0X+2MMwnfO5Z3VC9b+EKZewnMMkscDDniGtDtpGynNcNucJEf2inxdGmUztTKYqBaq0kZtn4PJeRdCTNi6SP5CvLxUNUZUHFoB4P+CvUdeAhBXuCsOUPFjrnexlgTthNBiN2s4Etoo=
From: Rahul Singh <Rahul.Singh@arm.com>
To: Jan Beulich <jbeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>, xen-devel
	<xen-devel@lists.xenproject.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Andre Przywara <Andre.Przywara@arm.com>,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Julien Grall
	<julien@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Topic: [PATCH v3 12/17] xen/arm: Add support for Xilinx ZynqMP PCI host
 controller
Thread-Index: AQHXtJXoF+vDzFdmiEeINAyFVRQ836u7OKUAgAD9OYCAADC+AA==
Date: Thu, 30 Sep 2021 10:42:41 +0000
Message-ID: <ACBA6FD3-DC58-4531-91BD-DB39E262325E@arm.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <c40a520d289799e52234fb1865e7f3ef1fd27431.1632847120.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109281638210.5022@sstabellini-ThinkPad-T480s>
 <3478d1b9-7043-3505-a125-700eb3761ed4@suse.com>
In-Reply-To: <3478d1b9-7043-3505-a125-700eb3761ed4@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 51177321-be65-4a13-306f-08d983ff1017
x-ms-traffictypediagnostic: AM5PR0802MB2561:|AM6PR08MB4040:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB404012847F15E82459015993FCAA9@AM6PR08MB4040.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 xHQIz/739fAMPlCR9iTUsb6MEybXPAVknXmGvvXFLLiMjqBIPQryKf0/1l+LorfOU4gkuSnSrZY9t0sHWlQdu9P8ATd7yIlXRUtU1vPunZuhRkx8uMqUX2o8YK6b0aXfzdeAVw5x8+cZ9KW7yb5B11R3w/aQQc7p4Ju2MyHJ2rV4kdokTl13JQ0eWhQZ4OYL8WIyK8Gi+XaihO/oleA5q+QJ/PBxmRZg3gTd4WsgpzzCTmwXt7vCL/3SEGY7wT5eRaMgJYSZwY06mg/+yiNO3LNPSDgsGmj6NsMnor84j5JW6v4nSCFYoyqxrhd6Y+0QxgWXHpzofgVn31tZiod29qXN9T2e/drMDb3mQu5GAxKSjT+IGafllA8tuMp/ohXFZYCDMZd3gCiFiX09qYVb9/8qewQWhToKXFy+2Xc7+Spbl6CDO0VObY3FrfJUEZbYHkr+DGpyMhPcqlCxlcW/lJ1r3L6dPfmf/Qc3aw2EmquyYigeFSegnodSYjk5gpAIjqqfLBoAmzginYgEfQEK/4Pt7cVbqXdzxQ90R/4eiC623zPHoWKTusnHLWh5kEbDOhRywN39/VPCI6VG38/LlWcS3XrQmAhhw23pSOp4adx4z4WsJRKOsSuTv2w/oiICH4U7h3KSyzza3JO89e/+Ypxtuc+RIq1Hq/STr4WQ4KEa29Yvw8aZDHb7Njntpp+FPzlbM4JKsJqLJfA+rP21V2XgfVYs91bJMQemHEvy2sN1McY0MWihNyOa1l3Uxl6SlRl1XRZDBfT507ylENb9dxM1dsCw2ygUDSH2biv+EI8ly1EO7SVCIt/6Ml1dBjQpLnLMx+CQCX6n238CEMOC8vpWNEttwxxflUS9dK4R3jw=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB6919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(76116006)(91956017)(316002)(53546011)(5660300002)(66446008)(6506007)(4326008)(122000001)(38100700002)(2906002)(86362001)(66476007)(66946007)(64756008)(6916009)(66556008)(71200400001)(54906003)(186003)(6512007)(8676002)(26005)(33656002)(38070700005)(508600001)(2616005)(8936002)(36756003)(6486002)(2004002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1A11552D1688F549B27DDD47D78810D6@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2561
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	aecfa92a-92c7-43a0-5d84-08d983ff077f
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	rZ/o9B1sZAioS/tEC5Wb5LrCiAvIc2Nqun3dp9/3u+vJVImE3dUtsW3ljTDE150F9GmFmieCi18ABYLtvfPVZttVuNVVAAgGm8QnjpqiflcSypNKthbqvfDie51MAaJo9N1TReLCYT5TKe+1T40SNMTI9xNEljfp1B+w0nKFjDy26tScM+MlIFp9Xnki6Mt0jrG6U0r15pb6vSXmfZCd8iqqFg4JXqjugFrcUiP8slCD7ZM610IM1m/LvWIdlJZSY3Y6Q+N0LezM5pZhrG2bD7VGsaZAKXUB9Kd3xEaC4NeXraExETxbmOhIQern5dXqmf649mwjtA34bvCensb3O1gWN+OmKRFlrIY49hdrjCPAiEAu+Fd8Y3bSYLQgrU97Y2kqzmUhkGPiNLiiRw3xsKk6PJyaYXSyl6FmNiVtBvnufyu9b8tZhaPZQVK/FUL+9vEzXmZHzhWoKPYeOcpZuL5QJEpa1F/v/RLDNJ71MjZW9ODuV7FcLBfyiAw4XvfsAH19g90Rnk412fm3uPdEXwixZZQouonF0sLfUhnEdEV0U0XJ5zkja1ZZ61ZT2iugiZfuBztK6duqAymCQNy1YZq+Y3GwN8XeBhX8yIrYuIq/8JMoUBwAg5PwmylSj6wkvCxvu7AzvWFkACAhZHAchXw44ALs3dvW1jbnED0+HppKTzU7uv0TT4V9rSP+ssSTVCdngKAQmD8hMTsNel1t7bVmjHdWioJ2zFqpp4XQQwskWaDcJK6lKnoRiDGDgFNXz2X3vmPnLDXCaw+BvnDfr6Eo07VMcLTCQu69vTP8DQLsb1iGIZIHmJ8FMfz6Baop
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70586007)(6862004)(336012)(4326008)(6486002)(6512007)(2616005)(508600001)(70206006)(107886003)(53546011)(82310400003)(36860700001)(54906003)(36756003)(81166007)(47076005)(5660300002)(356005)(6506007)(316002)(33656002)(186003)(2906002)(26005)(8936002)(86362001)(8676002)(2004002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 10:42:55.7501
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 51177321-be65-4a13-306f-08d983ff1017
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4040

Hi Jan

> On 30 Sep 2021, at 8:48 am, Jan Beulich <jbeulich@suse.com> wrote:
>=20
> On 29.09.2021 18:41, Stefano Stabellini wrote:
>> On Tue, 28 Sep 2021, Rahul Singh wrote:
>>> --- /dev/null
>>> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
>>> @@ -0,0 +1,63 @@
>>> +/*
>>> + * Based on Linux drivers/pci/controller/pci-host-common.c
>>> + * Based on Linux drivers/pci/controller/pci-host-generic.c
>>> + * Based on xen/arch/arm/pci/pci-host-generic.c
>>> + *
>>> + * This program is free software; you can redistribute it and/or modif=
y
>>> + * it under the terms of the GNU General Public License version 2 as
>>> + * published by the Free Software Foundation.
>>> + *
>>> + * This program is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * along with this program.  If not, see <http://www.gnu.org/licenses/=
>.
>>> + */
>>> +
>>> +#include <asm/device.h>
>>> +#include <xen/pci.h>
>>> +#include <asm/pci.h>
>>> +
>>> +static int nwl_cfg_reg_index(struct dt_device_node *np)
>>> +{
>>> +    return dt_property_match_string(np, "reg-names", "cfg");
>>> +}
>>=20
>> Can this be __init?
>>=20
>>=20
>>> +/* ECAM ops */
>>> +const struct pci_ecam_ops nwl_pcie_ops =3D {
>>> +    .bus_shift  =3D 20,
>>> +    .cfg_reg_index =3D nwl_cfg_reg_index,
>>> +    .pci_ops    =3D {
>>> +        .map_bus                =3D pci_ecam_map_bus,
>>> +        .read                   =3D pci_generic_config_read,
>>> +        .write                  =3D pci_generic_config_write,
>>> +    }
>>> +};
>>> +
>>> +static const struct dt_device_match nwl_pcie_dt_match[] =3D {
>>=20
>> This should probably be __initconst
>=20
> __initconstrel again as it seems.

Ack. I will use __initconstrel in next version.

Regards,
Rahul
>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 11:01:57 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 11:01:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199777.354033 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtp0-0002eE-Gk; Thu, 30 Sep 2021 11:01:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199777.354033; Thu, 30 Sep 2021 11:01:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVtp0-0002e7-D7; Thu, 30 Sep 2021 11:01:50 +0000
Received: by outflank-mailman (input) for mailman id 199777;
 Thu, 30 Sep 2021 11:01:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mlQV=OU=arm.com=michal.orzel@srs-us1.protection.inumbo.net>)
 id 1mVtoy-0002e1-MM
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 11:01:48 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id ce596c9c-21dd-11ec-bd45-12813bfff9fa;
 Thu, 30 Sep 2021 11:01:47 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38CD7106F;
 Thu, 30 Sep 2021 04:01:47 -0700 (PDT)
Received: from [10.57.19.250] (unknown [10.57.19.250])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ED71A3F793;
 Thu, 30 Sep 2021 04:01:44 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce596c9c-21dd-11ec-bd45-12813bfff9fa
Subject: Re: [PATCH] xen/arm: Expose the PMU to the guests
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, bertrand.marquis@arm.com,
 Julien Grall <jgrall@amazon.com>
References: <20210930092651.1350-1-michal.orzel@arm.com>
 <17d393c9-b249-cccf-e360-054b66466143@citrix.com>
From: Michal Orzel <michal.orzel@arm.com>
Message-ID: <6d9738c8-eadd-b85c-fd3c-1d48dc52ecf0@arm.com>
Date: Thu, 30 Sep 2021 13:01:35 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <17d393c9-b249-cccf-e360-054b66466143@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit

Hi Andrew,

On 30.09.2021 12:40, Andrew Cooper wrote:
> On 30/09/2021 10:26, Michal Orzel wrote:
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index 4b1e3028d2..4a75203b9f 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -2843,6 +2843,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
>>  
>>  =back
>>  
>> +=item B<vpmu=BOOLEAN>
>> +
>> +Specifies whether to enable the access to PMU registers by disabling
>> +the PMU traps.
>> +
>> +This change does not expose the full PMU to the guest.
>> +Currently there is no support for virtualization, interrupts, etc.
>> +
>> +Enabling this option may result in potential security holes.
>> +
>> +If this option is not specified then it will default to B<false>.
> 
> There are rather better ways of phrasing this...
> 
> It isn't "maybe security holes".  There are two salient points.
> 
> 1) vPMU, by design and purpose, exposes system level performance
> information to the guest.  Only to be used by sufficiently privileged
> domains (however the system admin cares to draw this particular line).
> 
> 2) Feature is experimental, and thus might explode on you.  Bugfixes
> welcome.
> 
Ok I will provide more description.
>> +
>>  =head3 x86
>>  
>>  =over 4
>> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
>> index b9ba16d698..c6694e977d 100644
>> --- a/tools/include/libxl.h
>> +++ b/tools/include/libxl.h
>> @@ -502,6 +502,13 @@
>>   */
>>  #define LIBXL_HAVE_X86_MSR_RELAXED 1
>>  
>> +/*
>> + * LIBXL_HAVE_ARM_VPMU indicates the toolstack has support for enabling
>> + * the access to PMU registers by disabling the PMU traps. This is done
>> + * by setting the libxl_domain_build_info arch_arm.vpmu field.
>> + */
>> +#define LIBXL_HAVE_ARM_VPMU 1
> 
> Please make this generic, not ARM-specific.
> 
Ok, I will.
> The domctl flag is (correctly) common, and x86 could do with this too,
> as well as other architectures.
> 
> Don't worry about plumbing the x86 side to work - that's a little more
> involved, and can be done at a later date.
> 
> 
> However, you do need Xen to report the availability of vPMU on the
> hardware as a prerequisite.  The toolstack needs to be able to know
> whether XEN_DOMCTL_CDF_pmu will be accepted so it can error out in a
> useful way on hardware lacking the capabilities.
> 
> You probably want to follow the example in
> a48066d25c652aeecafba5a3f33e77ad9a9c07f6
> 
>> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
>> index 96696e3842..a55ceb81db 100644
>> --- a/xen/include/public/domctl.h
>> +++ b/xen/include/public/domctl.h
>> @@ -70,9 +70,12 @@ struct xen_domctl_createdomain {
>>  #define XEN_DOMCTL_CDF_iommu          (1U<<_XEN_DOMCTL_CDF_iommu)
>>  #define _XEN_DOMCTL_CDF_nested_virt   6
>>  #define XEN_DOMCTL_CDF_nested_virt    (1U << _XEN_DOMCTL_CDF_nested_virt)
>> +/* Should we expose the vPMU to the guest? */
>> +#define _XEN_DOMCTL_CDF_pmu           7
>> +#define XEN_DOMCTL_CDF_pmu            (1U << _XEN_DOMCTL_CDF_pmu)
>>  
>>  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
>> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_nested_virt
>> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_pmu
> 
> Without an adjustment in the Ocaml bindings, the ABI check will fail.
> 
You're right. I forgot about that. 
> ~Andrew
> 

Cheers,
Michal


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:13:07 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:13:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199786.354044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuva-0001qL-Ti; Thu, 30 Sep 2021 12:12:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199786.354044; Thu, 30 Sep 2021 12:12:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuva-0001qE-Q7; Thu, 30 Sep 2021 12:12:42 +0000
Received: by outflank-mailman (input) for mailman id 199786;
 Thu, 30 Sep 2021 12:12:42 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVuva-0001q8-4u
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:12:42 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b4e88860-21e7-11ec-bd48-12813bfff9fa;
 Thu, 30 Sep 2021 12:12:40 +0000 (UTC)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur02lp2053.outbound.protection.outlook.com [104.47.5.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-JLtMH3sINxKkIHz7YCw7mQ-1; Thu, 30 Sep 2021 14:12:37 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6382.eurprd04.prod.outlook.com (2603:10a6:803:122::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 12:12:33 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:12:33 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P192CA0085.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:12:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4e88860-21e7-11ec-bd48-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633003959;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=QEISk7lW5cGpTfVXdr7Uq4IqbkSOrbZC3tjyIezCZzo=;
	b=BH1hnzscBByBrTDwN/CqIxeWoGG4aBSYJ63xuZ1ziKBTcg7dMjZ940Wicp8pvm+sygU8Rz
	iNE4GHBToPVfQcb7rYbFbHRG6I9ylzvYsNiMfH80ppmWujqNSnulfXKEgEdgMSd4x01kJd
	cZyeUaTicbk9D/ti4+nhjTciznOKf30=
X-MC-Unique: JLtMH3sINxKkIHz7YCw7mQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ODqMRHZ2QVJbk1/sYWquUYTxdgCqd1WDABi9PmEVHBu98VkJuXUHgFA+1PBzYonf2urHk3GYCestx74nqKd3jCIf0kIM+HNlnb5kNYdDD7YXCO4DVynXZITsJj2r9b1iWd9CEUGXTLGtPIsqmQWVWsDCrWtvJUShwD28aK3o8sGSpU6Vnh4+2AbLwx6We92oPDho9aeHCG3BKMFZKc9+lmVsPGGnvkngGSxcqb6OAJTS9OVjbXGSEzCFd1vDqDhCCJW7/jvNy8LShUYPnzsNWxS4nVnH6ngEO+kU3On7HLovF4OdTcW+sTpMtRdZK1tx80qSiqe/qsiiquDUBCWnvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=QEISk7lW5cGpTfVXdr7Uq4IqbkSOrbZC3tjyIezCZzo=;
 b=A7BT+0LbeiSNdnWpjdK/6zMLC950wGwt8Woe95yC61hgN+6rDb1ZKxw+/k+bs1tVCgQK8rq1ciNlRduxX1T7MnpZH0n+dvQ3jVRMY+2PGmH7FbsJP8H3nD9ByT/kgGuPoV5eT+w2UEHv+YxYw1RJ2/s1JJ949yDwWOvqST5TCPALosP6O4d7aMepix1J8Up6wSQcjl3iEB1VjYDXHCSDjJxE3RM+JIokfm1/9gZMb/ldT9YLOLox7WJgCI5QZrtVoh1VVGFG+dIrxjWXIaiTvT8uKnOW1J3uFB6QRzSGXbfeTkRFMBK3P6EtpARBLl7uzJkw1ZeUSKTk4sE8/H3T1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/9] xen/x86: PVH Dom0 fixes and fallout adjustments
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Date: Thu, 30 Sep 2021 14:12:31 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P192CA0085.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:209:8d::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae202350-3d4f-4be3-939b-08d9840b951a
X-MS-TrafficTypeDiagnostic: VE1PR04MB6382:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6382FEC671A6C5A22987A4EAB3AA9@VE1PR04MB6382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FR4T0bpe9/MF4EthAX96l8WilIl24HM8Q6f+Pohv8SCzqSwpZQEnSWQAM3rEiAIWNLRrmLC4bWmDx2N0KfDFG5ix9xYufS1+iVSRMdlad5D4oUo0YXFXM4AI+QvwirlNTI2/RhMbYXTd48atV1lnfmJqsk+G+JJVvV4/iD+TbvjShf47SM6hHedCM/DtGG0U6JH6HXNdL3TqqsaEw1A1juE2m8dCl3YPV4oFUEli0loCiFvCs3RUJ0qcQmqTqg3rQXJXJgakT3TNoN36Rhpfoivhy3oV52xbHR5KaTn7gyHT4ZTlOxXooRnLRY8/7cyQdvjchC2M5DvS6RuSkRQl4fCMvzM1LxkyEgZTHz9l+oU0kE8ycOBpiUwYS3k7HuebXiCzYwNJt6w2Xl+jOfRu8dIOHGOSlYHNQrO8dOXzrq0afCcBtovR2u06KSk8SxmVojWWK0SLnTU/NJO4Y3zEmiGZ6UMyQ2DwqWbHUMpfclTo7NJA1AAA/aUwtpvf7j/DbWGHa9L80hq74ymZxaOyI1p3p/nWVYT+kwqW7BvHPfIospsDFd/tcSqyrilIm1lgJhR+lFvs2Wqng+RyjDO48jTalvVfWt8rMR64DxQW6kzYttoJ56cuJx16hxnnPFRpbZdkv9smPYkKdR3RKbY5mBdzxRKWg7JZwE6Ddo/3UyPu4pdFu37uPo8t2cg98wgxJOncnVQwb4cNUQehY8M+UyWGihe4slag9uDlQ0wYkGs=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(86362001)(31696002)(186003)(8676002)(31686004)(83380400001)(26005)(66476007)(4326008)(508600001)(316002)(956004)(54906003)(6486002)(5660300002)(16576012)(8936002)(2616005)(36756003)(66556008)(110136005)(2906002)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TEVSVzFiWnFIVFdDS3RPdFhkMmxuKzdMam5sd1IyT1B3dzdyR3dIeWJPdzF2?=
 =?utf-8?B?dTNtbmpTOWlpelBYekl0eG90SXgzYSsyVWg3clllbG43WTNqR0dCRTFZdUNp?=
 =?utf-8?B?bFpKQ3dNTEE2MDlBSWRVWHcyTnhVdE10clVvd1VMZC85QXBTbEEzdTNwUVlP?=
 =?utf-8?B?N0lYRElFV0R1OFBQc2JxSS9YcTlKTEpFZjNqZ3RMajFOVUIrYmxvelVKamtO?=
 =?utf-8?B?QVlBLzFIM0VFMzZRbGVTNDMvSnpxNmdNMndRMGxsQXRVMWNyc0cwRjVkaDU3?=
 =?utf-8?B?SFhDTjRuVzFxTzNIcVorZUhDTmQ2L1hJdnZoZnBzUGoxS3dlMUsvYXZENUZp?=
 =?utf-8?B?eUwvdlh6U3o4NGlmajE4UURYMWdQNmdEOXlxOWVqSnkxWExnQlIrZjk4Vzlj?=
 =?utf-8?B?SlUyZFNCaVR5cWhRbmIwTkRCM3Ntc3JJbXNaWW9YMkJWZkpMTWM3RDU0OVVS?=
 =?utf-8?B?d2tFYmxqeFM5dVFBSXdMb1pPUmZJQ0t3aURrNTJ3TmU5Vjg3NlNoQVNURVJi?=
 =?utf-8?B?amtUUWZ1S0l0TzdPZS9NVlBBbWFVUEZGUFdFNGk0MVRlT2hBRnFxU1BTdGFG?=
 =?utf-8?B?emovSUZnNWRSZ29qYUVLUTNuVThzZHpxQ2tNeE5CL1V6SUhvRlJXbGlHa2Yy?=
 =?utf-8?B?d3VCcXgzVjZidlU0WDB2cmNTVys4ZTAyNytGanVZbEJtbHFUSXVzblBMSUhu?=
 =?utf-8?B?cjZTZDIyVFBWclVKQVhJa0hiUFptaUpXc0FkSVVvMG5QTW9BMitJWVFPMnFQ?=
 =?utf-8?B?TkpSaUFoZGJ3amVQVWtocjhqNXB0VE9PV0V0ejUxVUZweHlGNFI1c3lPTDJE?=
 =?utf-8?B?czZ0Ky9va0dPRDkxRFFoQ3NsR2N1YkVwSTZZajF0a2tneXIxWDFvMGN4bm1H?=
 =?utf-8?B?MitaU3pLQ2dRaEYweU93UHhlZmFlMnk5bTZOeUowUUI5YWhZaVJtS09NV0F4?=
 =?utf-8?B?Wlg3YThlcG9aVGU4UnR1ZHZZaGR2KzRBZjF1Rmw1WGIzUmRlKzM2Y1J3R2JX?=
 =?utf-8?B?TXJUR3A4TE8wRDM2VWU2clYwUVNZSllnK3VFWTZkVUpPZHBJVC9jQlo2VjJB?=
 =?utf-8?B?K21HWGtrUlhzWjZja3p5NlUvV0EzanJFK2dXVnVXVXRUY3lXN1cxbEpaclpa?=
 =?utf-8?B?bTVpakpWWW52NWN1UThEVlZYMXpNTTJlY0E0VkZudlA0eU9SdXhkZG1IaDJ6?=
 =?utf-8?B?UFFZbFRNb3FFa2ZNZDY1UHBubWhkWWI1a3FybXZ1M3RUb0VOVEg0Qzl2amVD?=
 =?utf-8?B?OG1YaDd2MVl5ays1TGV3cXZmcGJ6L0RHZEJwUUt0Y0sxcXpEMlhaSmVEckFD?=
 =?utf-8?B?dVFDZm1PR2VYVGRUSGtrd0c4MnhwdGt1YTlIaEtIL1Fzc0YwcmxXTWp2NHRX?=
 =?utf-8?B?bTZsRFNqNTlwWHF1TWtQK0p0dE12d3h1MzlxejlhdDk0RW9uLyt3dVNrUjN1?=
 =?utf-8?B?TTMyVTdGbHNCQXI0RmFUTUVpMlI3VmwvR0M1V214MUhTMnkvSmhVUXBHRElz?=
 =?utf-8?B?T3Z1N3BZUGFLMFZWNXYwNkRXUFhXN3BuTFRqNGczM2VDUTBPZ1Nzbk1QRmdP?=
 =?utf-8?B?bVZoRlYwcVNQdFM4NFloQktEVkFZaThJenVIOTE5b2Y3YTRudGdCWHJqRmtz?=
 =?utf-8?B?NjZtVWQrV3loS0t3dzY4VFY4SXB0eHBJNkdQUDhyOXdEdmRHUHl6NXdoejdN?=
 =?utf-8?B?TUUxcVYrWHJNK1RpNy9UaGo5RUlwRzlyaFFPL0V6eUNkNXpSdG8vNzNnekQ4?=
 =?utf-8?Q?aXsJHbJ6Ar011t4nWode5ixf+gCgZTOzXpZyDqp?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ae202350-3d4f-4be3-939b-08d9840b951a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:12:33.1758
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XxWRJUU7dRPfmjmjzvXaaWx/SV972jQGpT4qIg6Bh93Yd0iwDXHzbxo3mA3jZUkY+k29WtjCkt2yAiZZDOWmrw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6382

In order to try to debug hypervisor side breakage from XSA-378 I found
myself urged to finally give PVH Dom0 a try. Sadly things didn't work
quite as expected. In the course of investigating these issues I actually
spotted one piece of PV Dom0 breakage as well, a fix for which is also
included here.

There are one immediate remaining issues: Dom0, unlike in the PV case,
cannot access the screen (to use as a console) when in a non-default
mode (i.e. not 80x25 text), as the necessary information (in particular
about VESA-bases LFB modes) is not communicated. On the hypervisor side
this looks like deliberate behavior, but it is unclear to me what the
intentions were towards an alternative model. (X may be able to access
the screen depending on whether it has a suitable driver besides the
presently unusable /dev/fb<N> based one.)

v2 merely addresses small review comments in patches 7 and 9 (see there).

1: xen/x86: prevent PVH type from getting clobbered
2: xen/x86: allow PVH Dom0 without XEN_PV=y
3: xen/x86: make "earlyprintk=xen" work better for PVH Dom0
4: xen/x86: allow "earlyprintk=xen" to work for PV Dom0
5: xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
6: xen/x86: generalize preferred console model from PV to PVH Dom0
7: xen/x86: hook up xen_banner() also for PVH 
8: x86/PVH: adjust function/data placement
9: xen/x86: adjust data placement

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:16:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:16:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199792.354055 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuzC-0002UO-Cv; Thu, 30 Sep 2021 12:16:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199792.354055; Thu, 30 Sep 2021 12:16:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuzC-0002UH-9x; Thu, 30 Sep 2021 12:16:26 +0000
Received: by outflank-mailman (input) for mailman id 199792;
 Thu, 30 Sep 2021 12:16:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVuzA-0002U9-Q2
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:16:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3a12fa52-21e8-11ec-bd49-12813bfff9fa;
 Thu, 30 Sep 2021 12:16:24 +0000 (UTC)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur03lp2051.outbound.protection.outlook.com [104.47.10.51]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-4_WgSp7WOJitte5f-y4AzA-2;
 Thu, 30 Sep 2021 14:16:22 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6382.eurprd04.prod.outlook.com (2603:10a6:803:122::31)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 12:16:17 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:16:17 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P189CA0028.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:31f::13) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:16:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a12fa52-21e8-11ec-bd49-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004183;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/NqxYuSK5poj82wwlDFvGZG2lRpG9jB2q+2T7aAqVlo=;
	b=QI0AadEEFjtavmayZJOqoqlmZyOw5PKNo9G86tQZzhjoIZOtCi74f53j67IGJpuMAEK6ti
	2XsjoFKujz969g5T66jt2CK9tklYLbEnfICSDKZ2szNPOZSoMqERvhrO1A5BxY3LbDpLix
	SQY9c/Cvjlk6HaYLhNK4UTNnKeiTdNg=
X-MC-Unique: 4_WgSp7WOJitte5f-y4AzA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ifElNPL8ivCsdVMzAXc5ocymM357vWVNWj+2KoAVTKem4Z8uo3uOk3lsqXh/PFnGfrzAKvzXjYgKhzUTydzPihZi02FmMjcqp2jZ+8fEzfaPqJ9O4CQpaZNAzMjsYEFzkKXnnxWR24xNx6uZLP4Mm1tjVgJY1bBkeaT2vOedcaHAT5Eej3UDk+Yzh37ZDKjbFGo+jxHvrqUKR2TEKK3UzPMrk8QTbnHtRdcYEpxx76Fj4WoLJAfF6RYFRFbIMxKcOs/Yk7lb4WWBjpySkDJ1GxIa8GvaPx95nH4ZRDN+8c/+45scCf7PUjZgkS8sNYfr5MrLzVcUrZRfMzoSlwxv8w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=/NqxYuSK5poj82wwlDFvGZG2lRpG9jB2q+2T7aAqVlo=;
 b=FtPtM23utb3A1B3gG1FuTQdA4iP0x4YwzZyve2vTyJM+HS41jAA0UifkrFtFtFmzo7WhU1wZEMNZD7rL9mhCQfwTOwleSMURjUzzu7cfjiqK758XhEImBet6fDahTXmGWjch1fm6aoBExGjzzR3rhrXLiKV6PMZXbkn+0sUW/U3zB2TzODepRTpOPOcE5eyd6X+8Ndn7B+R6dSEjY4FodgqIpCHwbIjzifgKMp3RCUO54AS6nm/8b7RcvgkmuIfIlAN5qwx8M3inOzuiqh+NAhORg62WzrrCCSXA77s9fHi461TsiZc3+p9rRjZhJ6Q8oufPYCXrqC3B9IBDUO7N3w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 1/9] xen/x86: prevent PVH type from getting clobbered
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <d301677b-6f22-5ae6-bd36-458e1f323d0b@suse.com>
Date: Thu, 30 Sep 2021 14:16:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8P189CA0028.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:20b:31f::13) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 1da47a57-0b3c-46b1-7b3f-08d9840c1ae2
X-MS-TrafficTypeDiagnostic: VE1PR04MB6382:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6382FC63FF3E95DA554899F9B3AA9@VE1PR04MB6382.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7QR4Zmvtm49JMRlNKmeBq+9giJSpxwDZ4rvpDcKPgLI15B8zOsI2fXW7DG91NqTRQffiXwx4+vcLPIzOqrYjQW/+v31FN1Ixus8ndNgGXXDnduTwH3ra21Hgx++pN0KprC8gWBUTXkmzqjVNDh5Hcx7IYk3qsQuVQJLvaAsRczyW7VNrGtferUuIBUFsGbKMW9MZyEuv+MbfoB5UEa2/7G5E5HM7nJ2NH5fvsWH7AXeUc63XkTeUGs2LfrRFHEgTQ0o+90kC9xGb9wOiRL6+S0kjpQuRVOIp5D4djv2m2zvEruK2pdhanKzVDch3kzMZBqgLfY8MNLoSRum2j8LgfeKicFJhAC8YXjUU6xjRHg+tDzOQUApZgNH8UnY/uzJvPB+E5d6qXhh28JmEvhBIyeufhH0hnSwcI9gfWEsoRnFwUYW/vSv6b6dwcD79J2im1oJEBd6QD2izhtOpFsoIVdS2vWvBdczza2AhI26M9Gm7rYR3NM/Nf6NImZJZL4WXRrm+ldkvpBs8IRpG5VsR/WdIZ1oGT5cqh6Jnhtpb+f1d5nW65hUGkYIhCAL6X807cXDQ31RIk86WwxWi6QFTwiRYSTh3q9Qs5AcXi9vuq8nBwgR++KKTiEjp7X5vs6AhjihDgJTTAVVzZ/rVRVRa/eofqbJQm8ibKjWEPdgO9mShuHjMDPKdeY6Ml5NiOKmgtg+mXBZAvvyw3dpMGlvMzj8MACf9Ru+TFxumVTXxuoA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(38100700002)(86362001)(31696002)(186003)(8676002)(31686004)(26005)(66476007)(4326008)(508600001)(316002)(956004)(54906003)(6486002)(5660300002)(16576012)(8936002)(2616005)(36756003)(66556008)(110136005)(2906002)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VS9aNDQ1SENwbGU5YjVqdkY0VUtORitRNUZ5bUx6NTl2WUZhb3NNcXhoamg5?=
 =?utf-8?B?b25mMjdIdTU4dzZaN3JBUGZHK3hibHphRUl0OHJJVjZwalZlMXFmQWNxYW1t?=
 =?utf-8?B?eVVOK2YxTDhsdFdWSTV5dDZoS1I3TkZNVzN3aitPVGtBTVcvQTN1b2F4bk9C?=
 =?utf-8?B?bjJRVWNsdWhJY3ZCN0VxRGs5bzlvRlV3RElNcUFRYnY5MnB0WlZVSldLM2Z5?=
 =?utf-8?B?TWFESVg0aWhFdGNYT2NhZzVnUlVoN0xFYWlLdGhzUmFiKzA1L0VURGJEdWlT?=
 =?utf-8?B?dkphNmtPWWRZTDcrV21ab0ZTWGQ0enlPckx3RUFSQzhibUYzL1Fhd3E1TERI?=
 =?utf-8?B?MzlzZHg5UkJpaHV3bWs5Vk9zcVBScFJIdEVyS3hTOWZtYkltTVkvVDFhV3pn?=
 =?utf-8?B?L2NTai94QzlDZlpLUWxSZFVnM1VwcElKYWx6MDJuYlBqcmtsclNicmdoNFVm?=
 =?utf-8?B?WTFoTkRxOFpLWXBxeEZ3QTdkQ1ZUekppckhMZGNuTmczd2xrcXh6Um5iZVFT?=
 =?utf-8?B?UnFOaVlmcnc5ZVYySm1scnY4aHBMTktReXNLeFBhTmpXZStvQlQ2MFBXRGFW?=
 =?utf-8?B?dWh1V1F5L2JLKzFWVStyZ1NtYW8vdUkrWjF6OHhVQ1ltc1FvM2pBUlA3dzRN?=
 =?utf-8?B?MnBHcUtsdEhJZUJHUUk2bnltNVFhcHQvY2t0RVljRWFQZ1pkL0paSFBYQ1RJ?=
 =?utf-8?B?b2pTSFNBMFNiaGhyMlp6VUtlalBZRElnU3BlMFJ5Z2tOTWRVMitlOWlNbWlH?=
 =?utf-8?B?b1ZoYkVDUlVDTFZ1YTlIMlBEWEFiZDNDV0lROHViYmNsSDB5RUNqeTZwMURL?=
 =?utf-8?B?M3VzYWNsaDBRVjIwTmZ2K2JIcnZnejdZNHVZZFlITEFVdVRNZWZuUWsyNXYy?=
 =?utf-8?B?YXlPNUxYamswc2x0UkVITzhQbzVLbCtpdnhoME1iWVdpSzJKeXB2aFdESDRj?=
 =?utf-8?B?N1ptcnZ2dGNHOUhNT0ZiYU9XYklqbUZhV3NFakY1aWR6KzZRVGw2YXhZVkZE?=
 =?utf-8?B?bmpYWWYyV1BRQUFHUktaVlJVK3JCenFSekZoSDhvekpsbWdINDRLNnpGS0ty?=
 =?utf-8?B?eG9TNDdVb3RLRlQrQTlOVlpyeWhZb0hSMks3STVIZlo4UjdrMklNY0o0NytB?=
 =?utf-8?B?UEFnNUxVSGNEckpzM3c3YmU3UlV5VGZlVzhaenl6dGJMRnpKTWZ1TlEyTVRG?=
 =?utf-8?B?eEQ4YWR0UmFmZUttVVQ0aEV3Qy9MbEFxVzVZUEdYa1UzTTliVjVKWnlnVHQ1?=
 =?utf-8?B?RExYWjV4YXlOMUdaUm5VOGxhRGxWa2JlNjNxajhBdWpyaWo5RjFvMEVCVDla?=
 =?utf-8?B?M0YxRld2YmlXUGpkelZaVEIzWTZLS1dqUm83RDMvQXlDU3VmdFVJbHhmOVJ4?=
 =?utf-8?B?UlowQXlXUGtmeXZzYmF5Tnd6cVdWaElJQW9kbGREN0ovbEdPMk95aTM3Vlk3?=
 =?utf-8?B?R2ExMlJRdm1XcFY5OTRpQklnbWF3WnBlei9nS1F1TWQ1TlNBdkdqdnp4RXdE?=
 =?utf-8?B?TUdtMFNRZnZnSG1KVCtoYWxCVm5DUEl3K3VDRXZhb0JuVjNxaDk1TWVuL2Vm?=
 =?utf-8?B?YUQweU5TRHJ1OFhzd2JrbENQZ1ZiT3dOYUlwUUk2RWczcUhnRlB6ZHdyY1ox?=
 =?utf-8?B?VjhiNzRMMk5HOEJkQjhwRDlRRFJJWHBCKzFiRU9lazBlTmpCOGNwNGNycmE1?=
 =?utf-8?B?ZjA0N281TEpZL3ViRUVVeElBOC83R3paeDJYZmNLZ2dXRFE0VEtlRnRGWlBT?=
 =?utf-8?Q?ZL6VEO5J2BXu6nic87ZIgc8FtkGFhamJSH3xaIt?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1da47a57-0b3c-46b1-7b3f-08d9840c1ae2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:16:17.6349
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: t5NwW851KivOKa8DdKBD43rjsoLPincopUC+WfLp12/qLeT+T3xABojJXm3sgBVrIHWQMupYp99ajz8Bao9zkw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6382

Like xen_start_flags, xen_domain_type gets set before .bss gets cleared.
Hence this variable also needs to be prevented from getting put in .bss,
which is possible because XEN_NATIVE is an enumerator evaluating to
zero. Any use prior to init_hvm_pv_info() setting the variable again
would lead to wrong decisions; one such case is xenboot_console_setup()
when called as a result of "earlyprintk=xen".

Use __ro_after_init as more applicable than either __section(".data") or
__read_mostly.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -52,9 +52,6 @@ DEFINE_PER_CPU(struct vcpu_info, xen_vcp
 DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
 EXPORT_PER_CPU_SYMBOL(xen_vcpu_id);
 
-enum xen_domain_type xen_domain_type = XEN_NATIVE;
-EXPORT_SYMBOL_GPL(xen_domain_type);
-
 unsigned long *machine_to_phys_mapping = (void *)MACH2PHYS_VIRT_START;
 EXPORT_SYMBOL(machine_to_phys_mapping);
 unsigned long  machine_to_phys_nr;
@@ -69,9 +66,11 @@ __read_mostly int xen_have_vector_callba
 EXPORT_SYMBOL_GPL(xen_have_vector_callback);
 
 /*
- * NB: needs to live in .data because it's used by xen_prepare_pvh which runs
- * before clearing the bss.
+ * NB: These need to live in .data or alike because they're used by
+ * xen_prepare_pvh() which runs before clearing the bss.
  */
+enum xen_domain_type __ro_after_init xen_domain_type = XEN_NATIVE;
+EXPORT_SYMBOL_GPL(xen_domain_type);
 uint32_t xen_start_flags __section(".data") = 0;
 EXPORT_SYMBOL(xen_start_flags);
 



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:16:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:16:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199798.354066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuzj-000312-NB; Thu, 30 Sep 2021 12:16:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199798.354066; Thu, 30 Sep 2021 12:16:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVuzj-00030v-KC; Thu, 30 Sep 2021 12:16:59 +0000
Received: by outflank-mailman (input) for mailman id 199798;
 Thu, 30 Sep 2021 12:16:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVuzi-00030c-JQ
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:16:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 4b708440-21e8-11ec-bd49-12813bfff9fa;
 Thu, 30 Sep 2021 12:16:53 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2057.outbound.protection.outlook.com [104.47.4.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-2-plxmdjIENIahOSP6ObybZg-1;
 Thu, 30 Sep 2021 14:16:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 12:16:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:16:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P189CA0001.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:31f::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 12:16:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b708440-21e8-11ec-bd49-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004212;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZNbam7aHstMlEKiiTfzq0joC0jNz1ZNdGuw3sodQCjU=;
	b=lljORLsE7SYxXCzpMIugjP2GJsQGwymLM4DdyY9IgVZmygPXI9NgTADJmf0PqyVoVRUDOH
	VAXG2EXiHuSKPdAAkELwLt9Scm9Jx4aSLTuqua7FfIYY4dWOesIOf6M6Q5X5xvuTB8BDyK
	QEYp6cHZ7fmCQ/lOaE5wkKFydiNO48Q=
X-MC-Unique: plxmdjIENIahOSP6ObybZg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JQ/I+dp1Z9WJ3fIl4h6Spmx2nJRf676c6UcosbNQ8td/WaVXqVnBMhQmZ7QAn5hveqLhuTVthVxfq4IFBAD6URoXWF9+6Cz9gBvHvvJvcGe0rSbxNsssYkdHGbD7cWLCrUHqA9Q9v+BbzyXPXahz57z1GoEiyTKWS+ONmUiqpALMnYelEyXWo8YUPiih9p7jMBsIcGkmC+xBbgrXD0fvCv5MSI93m4XxVY7t42kNSc2I+N8+cxnxyxIdHduNMc43QN4dlJhra2zIc8V5uZpWgGyjXH4nGjjjIp387m6NoV+d79/N42qFl8qtjaYg7GUesiwm/dJLDmTjMf7SQi1nLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=ZNbam7aHstMlEKiiTfzq0joC0jNz1ZNdGuw3sodQCjU=;
 b=h3sA93cXpNudaLmvVd7hSEJgjwY8ozwrW2DXZsokppqb/aHK5yVHYtSaRvSi1iudNixFGknvYbZlf9btki3zdMqxb4avRLooyvvrh9X3fZlxHGrx1Q4iViTGi4FUxclzs+VS5DlvWW7E6LWkoU1rVOPftS1junoOIttcPYhom62v0idUGFNFLTTKtUFigyXi1ulu4gJ6p7K1Z60kNn7b1LgrwRaM82uQUamGNVm4fTvEJQr8W66VGDoN2yEs1HYbFjuhUmCzOv+DmOGGs6EjOZuD4C538WK0M+pQMUdL+86Kzvd8HIbEC7USH7riWgH+4I/mUys3pLG/PKpOoJNfHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 2/9] xen/x86: allow PVH Dom0 without XEN_PV=y
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <983bb72f-53df-b6af-14bd-5e088bd06a08@suse.com>
Date: Thu, 30 Sep 2021 14:16:47 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8P189CA0001.EURP189.PROD.OUTLOOK.COM
 (2603:10a6:20b:31f::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 8c708c28-f89e-4fab-6d84-08d9840c2dc1
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB64798659F13EA82CE2087A54B3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3826;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	a+j/3DJ6jGTOR2Y+9eiOGVDBHEbBm0BcTTqXZOWS7N46qGTXwC3++5VoDOMo9kGUbckiOKATOkbvhbVfjgAyz/WgftlpOMHRBIQiIhAvAxNrYJyH76V/GatH1I9Wolx461Ed4oIEE4oYpL19WOxLeEl3yej/HQEvE2/twbzHTLCVk3nxkyXN+OumzxGCXMTwhPQWpl4Oc0NWxYDsHHc7jdomWReqUct2+GfQU1hOpHCv5KpU+jxOIRJpPcSxonfdxkt1O69ibGZBgJbaDHREfGsFy84mGm6jifWAr1o2RRghyPWDCF4S/HLgIXFIgC0hGT4oay9Biv3ZRsWnj18QD/9KqhDr28hlRYAJ4m1HGfgc/FnM/qNJay+GckAV1mjLOx6IeYGm0VKTHnZYmEe+PQzo03I6vmRNAY0/l3IfC91+OAtvZkT8tvncXzvHU2qUekRITi9LxvrLqgQ1mKdpvXzQleLm0MMbSfarK0VvWm3/nl0yhM7TdFwW/vWmccFDgYobkZ4bhPZDNCsSrf+HLng/rnyhs4SMBhrwaI7hvn47iPEAmRWMp2KwSkdr2XSOMibjpDqBcoxYngaJFrH+zgx9msrDYtIdv52IeG9qNs0ompnLbhexbhGnsWk7iAZr2HXogOPGbtS4xgUgjoPUVvmkOiirIyZIRyq/9aQ0VruSgKIdoSfiJLj/KPa3eLSeeY/pO124NzMTSmUuCuJEUnKcTZw06WDhSkMJmOxWVz5+b6SPWR47jbWRjMwhzCv3
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(2906002)(86362001)(26005)(16576012)(508600001)(316002)(956004)(186003)(6486002)(8936002)(31686004)(54906003)(4326008)(83380400001)(110136005)(38100700002)(2616005)(66946007)(31696002)(66476007)(66556008)(5660300002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QitHQmZUSHgzSyswM0kySzAvcHJGWXZEOFFnSjlHWUlRM1F2ZFhRcnZ0SGhv?=
 =?utf-8?B?ajF5NU9rZllxemZxRUlnUVZJOTlUeEZQeDBhQVZPUjFVWHBRK3N2UzJ1ODdi?=
 =?utf-8?B?bk9IL28wOG1xa0dCVHorOWxzZ1JYUDBPamdkSUUzQW55ZmlYUzRtTkc4dHZ4?=
 =?utf-8?B?MUlCbElBUCtMSkdVcFlTN0oxNHhuVC9ON29MNnFvYVNJT2tPWUxyeWRRZHBu?=
 =?utf-8?B?ZFBoRW5zSTNRRlF1RXQ4OVlLaVhONWw5V3J3cjNxT1NWYnQyd0pMK3RjRWI4?=
 =?utf-8?B?RHgxRWF3cWJQc2ZtdlNPYlhLMElQbTRVSHc4cDF4ZGtETWMrR0FEdFZuM1Iw?=
 =?utf-8?B?YnVpV0toU1hHK1lHb3U4TjN2YkFEdmc3dzljNVE0RXZ6MjI3TGxkbGZqbzhW?=
 =?utf-8?B?WldFMUZlVVl3dEkxN0tVb0FUK25CTUM2SDhaQkpheW1NaVNzOThBQVhGbG9j?=
 =?utf-8?B?K0hJVHh3b04xaVNTVDhSZFVOL2NLZklsZmJWSmtiemtJZ1VjajdIMnA1dUxE?=
 =?utf-8?B?cHdlTHo0cnVZLzgrYlZoczRmMDdLdHE4c3pyd0xmWjdtSUxzZUR3WDRvSzBj?=
 =?utf-8?B?OG5taS9pdEoxVkVVUEg1Z09iRjRJU0Y1SUJFdDVqejk2c1M1eGhtdUNnYmpy?=
 =?utf-8?B?U3FEbDZudThlNm9nRVpFeXBPcWN5RWhSNWFHaDV4UWhQRi9ZRG1RN0tLcWt6?=
 =?utf-8?B?Tk5EN09IeWNDZHpBV0lTRWZwc2M2VWdtMmpjUmVVVmhhbkl6UlRNSUx0S0pT?=
 =?utf-8?B?NDlOalF2b2xmeStFYzZicEwyREhuckZMRXpWaGxjRnJ1Q2dDelh3aEl4UVBk?=
 =?utf-8?B?alZUazdXOWVEdGZZVVZ0SzMzN0hLdHVNaVAzWEVPQ09rdGdHTVRnd1BJZGgw?=
 =?utf-8?B?Q2xnMEtKcGdBb3dLZmNaSzYwWmFIK1kvQUQzUGJZaVQ5RzVTZ3JlNytNbUJx?=
 =?utf-8?B?VzgrS1hUS2NJL2MwTDFXYjh4cFBlL2UwTjI3NVhsaTRHbkROY3VHSndjSFNC?=
 =?utf-8?B?VXJnWWpUMFNGbE1YVWord1hNbFNGSHkzeHNiRWdvWTEwZGxkNWhqaThuY29P?=
 =?utf-8?B?eG03M29yRTF6d2NoMEJ4MVpGYzNCcVlKbHk5YitqaEsvMTlFNmZ6QUM5RDg3?=
 =?utf-8?B?VVVYZ01ubVpTYzdFSUgzb1Y5QkR4NUljcSsvVnFKVjhXbVJ4K293cGk1TVFn?=
 =?utf-8?B?SjY2eWZXemNWK0xkR1ZFcXhPUmpVY2llWll1QmtHb2VHZ3lCZEE1K3VxSVZI?=
 =?utf-8?B?K0djUVdtckJFelBqZ0JwZ204UkRETEg3ZzcxSjNwMWNXeGFvZk5lUnNlM0Iv?=
 =?utf-8?B?WElkdDRzcG9oYTM0Rk1tU3BLaytINjZtNW5nNDFuVElBb3VXZmlSeHlxRnpY?=
 =?utf-8?B?NDhOOURWeVAvZUdGbEtaZ1U0dTRDVGo0VzZIVGFQcndvdmdHM3oxSlZiRE00?=
 =?utf-8?B?aThqdFZoK0hXNmptUFBsZldwVk5YaXJtWGVLM2JDSWlxbG9GcTBzTGxTdmtF?=
 =?utf-8?B?NXp0ZitsalhHWjNtK2FqK3c5T3d6T0JnYVlocXVPL2Zpa2VtL1ROMHRqaW96?=
 =?utf-8?B?MG05TDE3UkI1ejRxL0s0ZzRyQWU4SnliSTJXZzJKc0k4am82NTdGdWR1eXd1?=
 =?utf-8?B?ejZySUhuUkxWSERmbzdISFlCdktmNTVLNytCL3ZKMWNkODU4VXV3NTFsL0tB?=
 =?utf-8?B?dzg3VTBXZ2JBWU5NaUhHc3dCT2tlajZJVmFaS3NwWTBzUG05STNRLzhyZU1G?=
 =?utf-8?Q?3bmiWeQ3X0KzGF568NFvPQWm3ttFWK2eK9K2G+U?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c708c28-f89e-4fab-6d84-08d9840c2dc1
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:16:49.2990
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eGcjr+3Rb8WDBmwm6nxsxx31sut4GUTuoMIelbKcNtMkFdd66TxVxTELLD1QJnhECL/7kJMFzTQobPpwoFF76g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

Decouple XEN_DOM0 from XEN_PV, converting some existing uses of XEN_DOM0
to a new XEN_PV_DOM0. (I'm not convinced all are really / should really
be PV-specific, but for starters I've tried to be conservative.)

For PVH Dom0 the hypervisor populates MADT with only x2APIC entries, so
without x2APIC support enabled in the kernel things aren't going to work
very well. (As opposed, DomU-s would only ever see LAPIC entries in MADT
as of now.) Note that this then requires PVH Dom0 to be 64-bit, as
X86_X2APIC depends on X86_64.

In the course of this xen_running_on_version_or_later() needs to be
available more broadly. Move it from a PV-specific to a generic file,
considering that what it does isn't really PV-specific at all anyway.

Note that xen/interface/version.h cannot be included on its own; in
enlighten.c, which uses SCHEDOP_* anyway, include xen/interface/sched.h
first to resolve the apparently sole missing type (xen_ulong_t).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -10,6 +10,8 @@
 
 #include <xen/xen.h>
 #include <xen/features.h>
+#include <xen/interface/sched.h>
+#include <xen/interface/version.h>
 #include <xen/page.h>
 
 #include <asm/xen/hypercall.h>
@@ -257,6 +259,21 @@ int xen_vcpu_setup(int cpu)
 	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
 }
 
+/* Check if running on Xen version (major, minor) or later */
+bool xen_running_on_version_or_later(unsigned int major, unsigned int minor)
+{
+	unsigned int version;
+
+	if (!xen_domain())
+		return false;
+
+	version = HYPERVISOR_xen_version(XENVER_version, NULL);
+	if ((((version >> 16) == major) && ((version & 0xffff) >= minor)) ||
+		((version >> 16) > major))
+		return true;
+	return false;
+}
+
 void xen_reboot(int reason)
 {
 	struct sched_shutdown r = { .reason = reason };
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -142,22 +142,6 @@ static void __init xen_pv_guest_late_ini
 #endif
 }
 
-/* Check if running on Xen version (major, minor) or later */
-bool
-xen_running_on_version_or_later(unsigned int major, unsigned int minor)
-{
-	unsigned int version;
-
-	if (!xen_domain())
-		return false;
-
-	version = HYPERVISOR_xen_version(XENVER_version, NULL);
-	if ((((version >> 16) == major) && ((version & 0xffff) >= minor)) ||
-		((version >> 16) > major))
-		return true;
-	return false;
-}
-
 static __read_mostly unsigned int cpuid_leaf5_ecx_val;
 static __read_mostly unsigned int cpuid_leaf5_edx_val;
 
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -14,16 +14,19 @@ static inline int pci_xen_hvm_init(void)
 	return -1;
 }
 #endif
-#if defined(CONFIG_XEN_DOM0)
+#ifdef CONFIG_XEN_PV_DOM0
 int __init pci_xen_initial_domain(void);
-int xen_find_device_domain_owner(struct pci_dev *dev);
-int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
-int xen_unregister_device_domain_owner(struct pci_dev *dev);
 #else
 static inline int __init pci_xen_initial_domain(void)
 {
 	return -1;
 }
+#endif
+#ifdef CONFIG_XEN_DOM0
+int xen_find_device_domain_owner(struct pci_dev *dev);
+int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
+int xen_unregister_device_domain_owner(struct pci_dev *dev);
+#else
 static inline int xen_find_device_domain_owner(struct pci_dev *dev)
 {
 	return -1;
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -113,7 +113,7 @@ static int acpi_register_gsi_xen_hvm(str
 				 false /* no mapping of GSI to PIRQ */);
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 static int xen_register_gsi(u32 gsi, int triggering, int polarity)
 {
 	int rc, irq;
@@ -261,7 +261,7 @@ error:
 	return irq;
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 static bool __read_mostly pci_seg_supported = true;
 
 static int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
@@ -375,10 +375,10 @@ static void xen_initdom_restore_msi_irqs
 		WARN(ret && ret != -ENOSYS, "restore_msi -> %d\n", ret);
 	}
 }
-#else /* CONFIG_XEN_DOM0 */
+#else /* CONFIG_XEN_PV_DOM0 */
 #define xen_initdom_setup_msi_irqs	NULL
 #define xen_initdom_restore_msi_irqs	NULL
-#endif /* !CONFIG_XEN_DOM0 */
+#endif /* !CONFIG_XEN_PV_DOM0 */
 
 static void xen_teardown_msi_irqs(struct pci_dev *dev)
 {
@@ -555,7 +555,7 @@ int __init pci_xen_hvm_init(void)
 	return 0;
 }
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 int __init pci_xen_initial_domain(void)
 {
 	int irq;
@@ -583,6 +583,9 @@ int __init pci_xen_initial_domain(void)
 	}
 	return 0;
 }
+#endif
+
+#ifdef CONFIG_XEN_DOM0
 
 struct xen_device_domain_owner {
 	domid_t domain;
@@ -656,4 +659,4 @@ int xen_unregister_device_domain_owner(s
 	return 0;
 }
 EXPORT_SYMBOL_GPL(xen_unregister_device_domain_owner);
-#endif
+#endif /* CONFIG_XEN_DOM0 */
--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -43,13 +43,9 @@ config XEN_PV_SMP
 	def_bool y
 	depends on XEN_PV && SMP
 
-config XEN_DOM0
-	bool "Xen PV Dom0 support"
-	default y
-	depends on XEN_PV && PCI_XEN && SWIOTLB_XEN
-	depends on X86_IO_APIC && ACPI && PCI
-	help
-	  Support running as a Xen PV Dom0 guest.
+config XEN_PV_DOM0
+	def_bool y
+	depends on XEN_PV && XEN_DOM0
 
 config XEN_PVHVM
 	def_bool y
@@ -86,3 +82,12 @@ config XEN_PVH
 	def_bool n
 	help
 	  Support for running as a Xen PVH guest.
+
+config XEN_DOM0
+	bool "Xen Dom0 support"
+	default XEN_PV
+	depends on (XEN_PV && SWIOTLB_XEN) || (XEN_PVH && X86_64)
+	depends on X86_IO_APIC && ACPI && PCI
+	select X86_X2APIC if XEN_PVH && X86_64
+	help
+	  Support running as a Xen Dom0 guest.
--- a/arch/x86/xen/Makefile
+++ b/arch/x86/xen/Makefile
@@ -45,7 +45,7 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinl
 
 obj-$(CONFIG_XEN_DEBUG_FS)	+= debugfs.o
 
-obj-$(CONFIG_XEN_DOM0)		+= vga.o
+obj-$(CONFIG_XEN_PV_DOM0)	+= vga.o
 
 obj-$(CONFIG_SWIOTLB_XEN)	+= pci-swiotlb-xen.o
 
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -109,7 +109,7 @@ static inline void xen_uninit_lock_cpu(i
 
 struct dom0_vga_console_info;
 
-#ifdef CONFIG_XEN_DOM0
+#ifdef CONFIG_XEN_PV_DOM0
 void __init xen_init_vga(const struct dom0_vga_console_info *, size_t size);
 #else
 static inline void __init xen_init_vga(const struct dom0_vga_console_info *info,
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -241,7 +241,7 @@ config XEN_PRIVCMD
 
 config XEN_ACPI_PROCESSOR
 	tristate "Xen ACPI processor"
-	depends on XEN && XEN_DOM0 && X86 && ACPI_PROCESSOR && CPU_FREQ
+	depends on XEN && XEN_PV_DOM0 && X86 && ACPI_PROCESSOR && CPU_FREQ
 	default m
 	help
 	  This ACPI processor uploads Power Management information to the Xen
@@ -259,7 +259,7 @@ config XEN_ACPI_PROCESSOR
 
 config XEN_MCE_LOG
 	bool "Xen platform mcelog"
-	depends on XEN_DOM0 && X86_MCE
+	depends on XEN_PV_DOM0 && X86_MCE
 	help
 	  Allow kernel fetching MCE error from Xen platform and
 	  converting it into Linux mcelog format for mcelog tools



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:17:52 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:17:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199807.354076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv0Y-0003qe-5E; Thu, 30 Sep 2021 12:17:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199807.354076; Thu, 30 Sep 2021 12:17:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv0Y-0003qX-28; Thu, 30 Sep 2021 12:17:50 +0000
Received: by outflank-mailman (input) for mailman id 199807;
 Thu, 30 Sep 2021 12:17:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv0X-0003qL-9c
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:17:49 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 6c70553a-21e8-11ec-bd4a-12813bfff9fa;
 Thu, 30 Sep 2021 12:17:48 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2054.outbound.protection.outlook.com [104.47.13.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-G1tPMFA8MW2IYg9YuyJBTQ-1; Thu, 30 Sep 2021 14:17:46 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep
 2021 12:17:43 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:17:43 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P195CA0046.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:87::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:17:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c70553a-21e8-11ec-bd4a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004267;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jj8pjeS0x7GY4bohbvuUA6qEPFJfyqs2go/BOwB/66o=;
	b=erQoTiFQQ7lZIT/mDXn89ISLmBA/09g2nITSZUK2yN6eZe+M7tmOnFobh+iQ2VAOeuRGmv
	off3+vTUAUdv3PDaYahHWU1PKD7KAHs4HNcEQ1ZwlW4/M+IDnp1bpSDCzWmJEhw1wVn7cu
	RRswFdn7BvG25WT07kUpwE0GPX9iPy8=
X-MC-Unique: G1tPMFA8MW2IYg9YuyJBTQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bbxZpEUsDAdXvRQ7u7cs0XjXCHpZLlYAstjkIWokH2Uw2HpmXwLCgre5MCiSHrYswACwVCyX3OaqImew6WzuG8932/ds/6QFbLIx31kYsP8n5UoZN4NKcQ4d1htk2Qb3GPDHxIxTFuqFrvUJ+JTnrfnGdi9s3l5xDEyJAOUtbUCQzKjtmwyJw3Cm1LeWlgmQgfJ3ifFDpyuO07+VJaWv0wJrZBm5P+2o+yNpzxSxYB6y5VinvZpP9GMJ1moARkpI45HLmksUMgU8bb5t4WMY6v/uqOHWj1GlyEFnLBTZxw510iaOyRqXsIzWO6btzmJ+teO/8xty2DtWGsphDG12jQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=jj8pjeS0x7GY4bohbvuUA6qEPFJfyqs2go/BOwB/66o=;
 b=GRNB4Ttas8nqohJVyWXNL7BwKR0m+O/QHHbuL19QldlXGAyxuAUUO98Q51VgRLfAtjwskKALqKZC6hd+LGVx11rYFqJjMVL9rTZlqmtvchYzpwGApibDt3UQaqBaAAU7uApEGD32s5EtnScE19/MC8j+OYh72N40OkR5QuRfsk6Efv53Y+YT6rfok9YW8dXsfdvVNYQ7emkcTSIUTSByBduR1pZ/6AHAg8g5THahGYGYSCv1FugGpTGy5yO4A2x68VuGzZc/zG6MOrxJMM1i/CAA1WLl5XE+Ps6ScTCxOE4sCPwNIJxEdmPz05cYME5tfideWtjHwi+0cjTIvk/MDA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.ozlabs.org; dkim=none (message not signed)
 header.d=none;lists.ozlabs.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 3/9] xen/x86: make "earlyprintk=xen" work better for PVH
 Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <4fd89dcb-cfc5-c740-2e94-bb271e432d3e@suse.com>
Date: Thu, 30 Sep 2021 14:17:41 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P195CA0046.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:209:87::23) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 470e41f3-34f5-4da3-c49f-08d9840c4df2
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038C0AA781435D480EBE6ADB3AA9@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	0Z41bVlnytrFRDAc2UAQ0vwr9ncaX4vNdRDM85qp6xJt1yeeUqLLhaPoIHBSahTy6W/d43yTef7GErIsnG/vdVT9ZemeJcYjQG9Ry9sTWsHIR8ntzhiokKjiQ1wdjpYN+eviPToDFfsgvtaWhzH00vNd+YlgQwXWz1ql5EbeTbGXFk1KhDQtrp/mUabjJqlZoo0DfHqXlsOUmxnHwcSPXP60Adlt2Lq4a+vUsexQrUf5f+WOnh7nxC1fNt+2WIC7tSSUTwp6zJr5wV3jJHRjEYvnQh8poZyCMekW3jKnOLoVCQipiDv4uF5SfsQbG4blPff+8zWtYEPRilZN38ArZ8wwIgrUsS036G7nsvjotY1ZQ9MhWeiSVo6Ldl5+w0/vF9xcu5KN7IswPZ7rnIA8xSbhchTh7s48+lxJQ8wb9ezs0QdsTLwH+aQfBBtR6wUbpX7t0zlyMtPKXEPySDZlfdIOn1Tp1glLjxMZ/j5UNMb2sGwnSnotGFDcWMQtSd3uK7cug3DeSTUzR30wPdpxa6sHuD8kY62WOoGMjHNK9XO23qXc4IdUjp9hwIFNV7yBGtjBMT1W0crGRVwFOOIciMRbkVqWMBlKVF82yhWr6GNgBE+WXlNvy4IbqNQ3R6ORwhmUQMqGwmUg9xkmV1+o6W9Yo53O8vlFYQdV5+dwIOxP7HUMglGEA2uF9bUfJmmOAhgMA7rY7ofm2tkDCk3v9pC7vCVgWgIa6Q6TSpwF4lcy59c1tNl+PQpYCkXKNp2o
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(956004)(8936002)(31686004)(110136005)(2616005)(316002)(186003)(38100700002)(26005)(5660300002)(6486002)(54906003)(4326008)(8676002)(31696002)(16576012)(66946007)(86362001)(508600001)(36756003)(66556008)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z2l5Ui9veDJHZjJsaU1YcXNKQ1NCUWl1b1hjRXFUUTU3SmxzWTZjZCtLTW85?=
 =?utf-8?B?bkhWMnB0MkdVUkIvNmN6QkVISmRqU2FES3VNWm1VaTcvd1JMVGpxRmtDb0tL?=
 =?utf-8?B?T09kNmgrYzJLaGlJNS9GMEFESVVqYmFrVFpucE1aTk9RRjlvOG5ra0xVeHQw?=
 =?utf-8?B?THp0V2lGZG1uTVo4UGNZYW1QMnBYN1BBWnl2KytEczVFbzErNTZiVUJzUVdY?=
 =?utf-8?B?d3FBMi9Rc0Zaa0JEN2l3ZUhuTVk4ZEl3WHR0b2xhalVHZXhXUFVWMUZVYWEx?=
 =?utf-8?B?M2NVcU9MRUZ2ZFo1MW5CcE1aRnNCSGFEaU5VbnhjeW1KNmY0emdZZWY0Q2xi?=
 =?utf-8?B?dUtRd2krL0dSUVUyV3NkNHBUU3pMMXNRNUtNbnM0ajM5TzkyekRBNDk3TDF5?=
 =?utf-8?B?VmdKVWMzWDNYdzhvcE8vdFVpQ2lHU1M1Rldsc2RrY21UT1lOb2xSdzNiZU8y?=
 =?utf-8?B?YWhlbzBjNFdDVTRJaHhqellIUDlOT3REeTBDMFlOWjZIOU0ra0RXVDhiN0RC?=
 =?utf-8?B?TCtDcHRIdFNMMkdjWG8vMG1XeEt2Z0FUQmtCb2daY3hxbG5BT1BiS1VXK2lm?=
 =?utf-8?B?ejJacTMxS2EzUTgvNUdYVXhUNHZaWXBpNVRBeFBxVHdsdGU4RjVhRm9KK0Ex?=
 =?utf-8?B?SkFzWlUrRERDOEoxSTZWdjdNNUdpYzlneTNLdlZyN0Y1MHF4N2h3VEZkMG5w?=
 =?utf-8?B?eEYzVXRvQ2ZqeEE2MGVKdzNvL1p6R1VwNldPbVdhTnVSTG5ydFNLNExIVGJN?=
 =?utf-8?B?ZGI4MWg5aG5tcWh6NXBEYjBMWDdtU0pFNmdjSkdlazlGdTlIVVBaSTNTcGJu?=
 =?utf-8?B?aGdnNGlXMVJ3UFlEd2tTVndTeTd5R2ZxcjlsaHNScHBjSkJoMzYreXNZZmxV?=
 =?utf-8?B?VW5lM0oxSUoxeGpoR010OFpEaVU5blhGQUxWYWErNHRtYnNWYjFXeFdhQXlI?=
 =?utf-8?B?aE1yVTE4Kyt6RTF4Mmd3cng5cDl5T3NVRVFCOE9vNEpzbk85RzFRdFlpK1BW?=
 =?utf-8?B?R29SN1BIZTYxcnE5a1dHR0xmZThIdFJXcjFsaS9hLzhyczd4aXZEVmduNmlz?=
 =?utf-8?B?Z2Uvcm5hdjJIQWwrZU1GTk0yVWN2VDl4YzVvOGxZRjVOaXpxQ0Y3QkVHeWMv?=
 =?utf-8?B?clhZd3VFRENWQlN2dk1qNU1NVTVDUllXV3JmTjVDbXlLZW9JOWNLL2hCbGlC?=
 =?utf-8?B?WHBoSVBiamdUVWhEZ0ZPYTcrZFFBcEJlODQ3NzRwUW51eGxyMU15cVFJR3dm?=
 =?utf-8?B?TkpVbm92NlFsTEJ1ckhvdi8zUGx0V25QK1RwSy9OT3pEa2tOZmJmblZLaEZD?=
 =?utf-8?B?ZnpvT2tORVNPRzF5bGk0L21tOFd5enpzOXc1REl2RmdUNVZPdmFYckVIQnBk?=
 =?utf-8?B?czNJTHJnUWRUSTNCaFlPeUxtd3ljQzZHVWRPWjZQZi81ZTZjdlNCb2JvWDEz?=
 =?utf-8?B?a0NXSE12WDhrSm9iOW9wendIZlRvV2ZWME8wMzVYZ3VSSlMxaVc3T0JPZEJ1?=
 =?utf-8?B?Q2Z0MWJ0K0tCTGlSSDBUZGpodWtZZGFUTVI4VVR3Y1kwbk52bE1MeWNWVTRq?=
 =?utf-8?B?OXhGKzNTaUhGT0gyWXVsRXl4cEIweDE3WnJRcmJJT1llVVVSNENvWmJpQVdZ?=
 =?utf-8?B?WFpaVVJtSXRtU3VrL0ZxdThEZTRGQ2FicFZmOWozMzVoVmlDeVFxcUUySTZZ?=
 =?utf-8?B?aktuVnBoK1VWRTJ5UWNiZ21zbjdTS3RhaEhPQ2oyNCs5bnRXZUF0bGE2b0pR?=
 =?utf-8?Q?oxCC5HmGvDaL6Bxq2SsRm47MlhXV05bdknJeo/5?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 470e41f3-34f5-4da3-c49f-08d9840c4df2
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:17:43.2894
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lFadeeFoNL/7Rb1Q+XP2lyTmtWaR1GHLFxRSjCxuQFlbBOBsfkCJp2t/4s7/Uv9e4J6Rb5RdE/atFnGkhO0OFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

The xen_hvm_early_write() path better wouldn't be taken in this case;
while port 0xE9 can be used, the hypercall path is quite a bit more
efficient. Put that first, as it may also work for DomU-s (see also
xen_raw_console_write()).

While there also bail from the function when the first
domU_write_console() failed - later ones aren't going to succeed.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>

--- a/drivers/tty/hvc/hvc_xen.c
+++ b/drivers/tty/hvc/hvc_xen.c
@@ -632,17 +632,16 @@ static void xenboot_write_console(struct
 	unsigned int linelen, off = 0;
 	const char *pos;
 
+	if (dom0_write_console(0, string, len) >= 0)
+		return;
+
 	if (!xen_pv_domain()) {
 		xen_hvm_early_write(0, string, len);
 		return;
 	}
 
-	dom0_write_console(0, string, len);
-
-	if (xen_initial_domain())
+	if (domU_write_console(0, "(early) ", 8) < 0)
 		return;
-
-	domU_write_console(0, "(early) ", 8);
 	while (off < len && NULL != (pos = strchr(string+off, '\n'))) {
 		linelen = pos-string+off;
 		if (off + linelen > len)



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:18:21 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:18:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199813.354087 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv12-0004R1-Da; Thu, 30 Sep 2021 12:18:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199813.354087; Thu, 30 Sep 2021 12:18:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv12-0004Qu-Ai; Thu, 30 Sep 2021 12:18:20 +0000
Received: by outflank-mailman (input) for mailman id 199813;
 Thu, 30 Sep 2021 12:18:19 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv11-0004Qm-QV
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:18:19 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 7ea89ec4-21e8-11ec-bd4a-12813bfff9fa;
 Thu, 30 Sep 2021 12:18:19 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2050.outbound.protection.outlook.com [104.47.4.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-35-DM9gnjjIPzqegUU0-iFPDQ-1; Thu, 30 Sep 2021 14:18:16 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 12:18:15 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:18:15 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM7PR03CA0011.eurprd03.prod.outlook.com (2603:10a6:20b:130::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 12:18:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ea89ec4-21e8-11ec-bd4a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004298;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=A3Nsesxu7Ouso1Ynp59Ie6SiuUu3Q+vNIHx5xr527BQ=;
	b=b7vT0lBPFd46VgJ4X+1jQKMszcFIlV1zJ8z4OSxxysGAkbm/pve3kShNtztCi58VJM8dtV
	eWgHf5MElWLxjFzlNbc3Vn0HEYSpnhP1z4zlUnP4xNAZ1H5kdgiK5xSJe0NxTpSliKUg+K
	CRuEDIy2TQRcDULRwOJ7vt+ty6bwgiE=
X-MC-Unique: DM9gnjjIPzqegUU0-iFPDQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=emqtrX2XYYweu4RAZ2JlfUU8hXYjS/+fjwksUN1LdgDie0U7zsqLPztPfslPqs+/kkNoOxFI9o+cbzvm5egeQSvXgPghLYVc+XvcE0VgHOjpQdEiG69QQyWbJPMq0Nmfk/jUSEPHc7UWpkQbtngCFI93usCpbbwV4/A0jqsQXTVB7nnn2bViBvnTFAhDUlu6mhWGHuj+Q+uAI3P/5eEyD6sNg3IrTuq723oYSy3CSp0psw/ALwkIOSUK7s93E+7icI4XQAg3nory0xgdmb0hGrnId4INNMNYqpoMDne9g5d/i2XlDHmbeq2IQ/ODTFh3l5efblWTeJ9Ad++B08GhLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=A3Nsesxu7Ouso1Ynp59Ie6SiuUu3Q+vNIHx5xr527BQ=;
 b=GLoVLo8QPW2jvkBquzDkXsxTG0jIbMQKhu4ZPLYcjZlbKJwQo3s8oUVND7LxAMD/2pnD3xRlvLN3Qpt+khTVBZcATAytNEYO5L5bI7BOoBReD+KvtDIBWW7MRl/IdZul4YJ0ytu9oL1iVvH/3Z4el1spYnUmea21tjhYjgFsFs3mXpZXqYFcilZNzxestkgpcPiS1MGZJrI/B9cEOpFuAqUz9Pb8LOmaH7HFbovpveIctIm+TkZlrs9ROKxC5WoVV3GFJFLbNsYbXRUIRFY1GMqjFFL24mQzYs474yxYZcPw9D5dPEtF2UBfwS4xy6xMzh14Sa8iUCOMtcmtclo/LQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 4/9] xen/x86: allow "earlyprintk=xen" to work for PV Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <d4a34540-a476-df2c-bca6-732d0d58c5f0@suse.com>
Date: Thu, 30 Sep 2021 14:18:12 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM7PR03CA0011.eurprd03.prod.outlook.com
 (2603:10a6:20b:130::21) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 97f795b8-3159-416b-3e3a-08d9840c6144
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB64792007107B6A798A7A0C73B3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6108;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3dxdqcnhuYwi/Z7EvOy6I2MDcmOkGcMLwme/dt/7fQ/Rz5Y9b0BcoxCojyfIVI/uUNr841INRG6F5M536DDl42y/SW91yz9Sd5+pzcuEZyA1qa4atGqae7Dd3NEml9VxkX2kbZu1mh9wcWCUlpzP7qjBTMCt7LwZ+/DSFwJ0oDpXICwzVhpXS6P4SQhn2WY5T+9jiSvVEYYhi2z7SGlNlPz6wR1ldaPjxBUyiNlrCZfPJDVL5io2Z6V3IlEpg9rIFjAukIN7MUtQVLHONW9ltivDGjR8csIP19mBPURhL2k666rN6nCw9DPPIl5XRV7F+o6iUlaMd32RMwDMvMA9Meo/XLdf00mQYYEMoa40Lxgvpgwqvq0pSpwxMiFa5UQXVoiECPwEucBdPuV2XRoXcPv1kSt4VCGbMdxBbH6I5Md+jZKHfj/VD2ce6xLm83wyhKRW3hBVRiWmzsoR6KN802dwjQ35FZJsUaK1qwMg7I0BxcbOmYo3eb/uConbKGi29mXCbJxXPaxPk5AU+YlzaduFN79EveAvN4bFvbSeVMlZQ/tdSs4zo+s/b/ip69j5mEtjVaQ+wnxHQYFi3V9HMAiI5YMLoNco4QDxPiS4Pz1mnyDsVxi0NCPalG59415Zo4fHemjV73gvQZLhiSOmDopDb7LQG97DuCh1R3BYs251q4vn8+SdVZSswGZWWPXDl5SbUBcjBBHuJPGNeJ1Xd8J3cLxpD65GeEv/j3OLYMo=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(2906002)(4744005)(86362001)(26005)(16576012)(508600001)(316002)(956004)(186003)(6486002)(8936002)(31686004)(54906003)(4326008)(110136005)(38100700002)(2616005)(66946007)(31696002)(66476007)(66556008)(5660300002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z2dFeTc0b0dsR0lVZzBIQ1RuUzd1VG9PSVBUcHJLVEQ1a2JWcmJOVDNGZ3VO?=
 =?utf-8?B?cFk3MFAvV0NwbWorKzJSVG1kajMwakNVcVpIb0hqY1RoK0tKWUUwSzJnN1NQ?=
 =?utf-8?B?bkFoWU5qNnQ0TFhQN1NDNDNvZm53UEhmSFBrdGRUcTJQNUE2Q1lTOU1QVWdU?=
 =?utf-8?B?WjVCSG1CZisyYytlcjJuQ0RLbnhIUkFWWGhmcDhkeXJMSTk0WWMwOFhiYWgx?=
 =?utf-8?B?T2xvVlBUdHcwQjdXZjIwZlVWcnBkdDlkZmtpRzZOOFREcHR4Ti9BdURMamVC?=
 =?utf-8?B?cGc1NHdLTjhhYXlUZjArTUZaSVd4UmE2YUtDWS9GcVozeUJDejcreTFBU05p?=
 =?utf-8?B?czB0bCs3bjBWYmt0RzNUUjdWQ3NJaUJuOXplQUM2VkVGaFEvdzNPbkx4cXlU?=
 =?utf-8?B?SXRyNzNPSnBuYXVjbUd1cW5ySGRzRHZlWmIvY2VVRUdpNWFiRkpKRzdoKzlJ?=
 =?utf-8?B?cWNvQ1lFcXd5eEJOU0t3cmFWa0lyYWRNc2s3OXhPQ09MMU5od0xYNUY5dkNo?=
 =?utf-8?B?OEhqbWNTVFE3UVdkc0RLem9rNjc0SVJoVU1mZ2JoMm1tb3VJbkQvZjVXV21C?=
 =?utf-8?B?ZW1jUDJORElod250dUZkcDdyK0FkOW1TQXNKRmpoNW9MTmN0ZlBTbDBTd1hF?=
 =?utf-8?B?ZnVudDJSTXA3cGhSVENTNUIzV002b0s0MkZVNUJsTDVjZWxLZy9ka2ZGMU5B?=
 =?utf-8?B?K2ZwTGxtK1FlRENhczlKYmJZdUpOWDNVa2cwUDdZL01HK3N2N0I0NWN4WmdU?=
 =?utf-8?B?VFZQWXBFakU1MERMaHBEOWsvNlM5SUhraHp3TURxd0F6YUNaOWlCcHM2Yloy?=
 =?utf-8?B?QTJPcXJoN3hydU9WRENKV1NVbWEzUzJURFBTRzNKMWp5ckZQVDZWMVN5Z05k?=
 =?utf-8?B?WjY5ZkpPYTBtMG5oQXdjeGxjaGt3dGhFOUUyU3JFdzcxOGptRkR5eG94TFFp?=
 =?utf-8?B?Y0JzZUdEUFdLM0hmbDE4WXFzbzRDLzBqSVdJQitBS2ZNbFBpM0ZFK3FVRXpZ?=
 =?utf-8?B?bEtQUEpIbldoMk9zQmJueGhkdnh4dit5R3VEdVBFTVZwRlBxY1o4NjJvSFR0?=
 =?utf-8?B?Nzhlam5oZmV5dlpCUTBGdlh2Y29PamJ2YlpJRUVpTzlqbFFmUjFPRkJYRUtB?=
 =?utf-8?B?Z0thbTZ2MnY5NkdEaWxGeGNxazdMQ0VXVXp0OFIxeVE0U3pZeUVEaE4veXRt?=
 =?utf-8?B?YzhyVHdZSDBaYzNLbks2a0hPMzZvc25jNTRNMktzdGt2bWE0M1hSMndpS0R5?=
 =?utf-8?B?Zk5qclpqZ2Z1V2JERTBHeTNrZkhPNE9Fb0FOZU0yTjY0ZFFiVU5rRHBleXpu?=
 =?utf-8?B?dFluZ290VmE3dXFYSWFtMmYwWnNrM1R2S0JKNVpVdDM4aWNsM0NhMVAyN1RO?=
 =?utf-8?B?MlgwY25lTkR3L0VyVTNmVGs1K0s2bFBMdjcrRUNnL2FIeEFLSDJsczRwZGN4?=
 =?utf-8?B?MHdzaEE1Y0NDTy9udU5KS2FKTE1BbG5zMXQ5b3JiejhpZjh5OFFUdTBjdmRY?=
 =?utf-8?B?cE1WSlU3eHQrMllpd2J5aGRGT3RZanU0ZzZXdWw1Y3VqajlHd0w2cGxhTFlI?=
 =?utf-8?B?a0JFRTZWQkh4NHhzQkRlWjR2REU2SE5USnByMGREMlZBM3hJRloyMFZ1enRt?=
 =?utf-8?B?OTBNY2lTNk1NVG9DNTN2NFF0Q0xCR1NKR1AxZDF6aEl5OHB0MGhnOEYzMVVk?=
 =?utf-8?B?Y0Vxc003c1h6RThGRnJaYk1NUURpVDRFc0o2U2Q2VFN0MmNlOFFCSEpyYW0r?=
 =?utf-8?Q?6tIrI0PS5qoWKsdkOQnxaEttlv2Tv6SmmjhzaWz?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 97f795b8-3159-416b-3e3a-08d9840c6144
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:18:15.7101
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4PhDoFle6mwMbklMriNVIDQv396oC4+UoGaUr3VQ3/AOcsWIkvnvJLgwhiUCkSQlvH55gtdQOYoZWvI2LfNauA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

With preferred consoles "tty" and "hvc" announced as preferred,
registering "xenboot" early won't result in use of the console: It also
needs to be registered as preferred. Generalize this from being DomU-
only so far.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
---
v2: Re-base.

--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1348,7 +1348,6 @@ asmlinkage __visible void __init xen_sta
 	boot_params.hdr.hardware_subarch = X86_SUBARCH_XEN;
 
 	if (!xen_initial_domain()) {
-		add_preferred_console("xenboot", 0, NULL);
 		if (pci_xen)
 			x86_init.pci.arch_init = pci_xen_init;
 		x86_platform.set_legacy_features =
@@ -1393,6 +1392,7 @@ asmlinkage __visible void __init xen_sta
 #endif
 	}
 
+	add_preferred_console("xenboot", 0, NULL);
 	if (!boot_params.screen_info.orig_video_isVGA)
 		add_preferred_console("tty", 0, NULL);
 	add_preferred_console("hvc", 0, NULL);



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:18:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:18:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199818.354098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv1Z-000537-OG; Thu, 30 Sep 2021 12:18:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199818.354098; Thu, 30 Sep 2021 12:18:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv1Z-000530-L6; Thu, 30 Sep 2021 12:18:53 +0000
Received: by outflank-mailman (input) for mailman id 199818;
 Thu, 30 Sep 2021 12:18:52 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv1X-00052a-WF
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:18:52 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e32195f9-903a-48e6-9000-7b281227afe0;
 Thu, 30 Sep 2021 12:18:50 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2058.outbound.protection.outlook.com [104.47.4.58]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-12-liUTvDdlOv-a8xFeYzXRmA-1; Thu, 30 Sep 2021 14:18:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 12:18:47 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:18:47 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P195CA0086.EURP195.PROD.OUTLOOK.COM (2603:10a6:209:86::27) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:18:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e32195f9-903a-48e6-9000-7b281227afe0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004329;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pFp9W7xkMIq9TQ4/Odb7qXzfO9m2/t23Esal7KykW2c=;
	b=KE0DKLHZoiPlgh8lz2oYwgwt5wdMYNb3Bh8I542iMcgUu6ZxChZ5ciJBmBETRarejm95bj
	6NG3RhtYz8mW0gS4wAJJ7gN/IYV/N9uc4sfcGlCFSF0SFfL86BNb3mQZD6n3fkQVQD0Q/f
	qoixc6p/0KKgWnFmRerxrAKxjkwORiY=
X-MC-Unique: liUTvDdlOv-a8xFeYzXRmA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CnWQDZXIyZAwX094LDOKe//dgZ79cM/1DWCNUHXz3xRvHz0Nimt5F+6hd38/7uVZpIRjyBI/Ilkf3c7gISrU37COKJPvRdAivsK/S7KcgMRjP0vP4AyQeptdey44vURUa7lXe1G5jQPNmHl7oisBu+7urJRehTtmgUYWWTm6q+7ZMVv2XHT3vPUly1fsG8Ma/0w3VlKxXyrMT2Cw+7FkO6OQ7Mx5vgh1IhndPTAj9UJOwWSvdRMMJPKgvk3yv0+v4DKYoYWaQ0KOopSUX12thU64FNhgqE1yvdxoMew7vfpZXd8h2NkHt7K2Ja2vNn/YkMdtOMWGjLWccBGRRjflVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=pFp9W7xkMIq9TQ4/Odb7qXzfO9m2/t23Esal7KykW2c=;
 b=nhoclBXVO2GNXKVr8A7s6Q3vjD0KoUIFA2WstyrXNlVZY+yvpPUDTixno7Prgs0zOY4MxGrF+juPakpC80QCoW2f0nfUHr+2+mmnmxZPtuCx1GNdzhOpsNBdKuoPFhblqI/BlaGximuBk8g+OB+OREZ1KQ4I7WMmf8iKaz86ndyknMVOLSgZaeeNk+sB//8Ktl0hh5oHoiaB5TAigIO1tbx8jtn4gYh1HwPZaBNfM3GEbrS8KBWZdfTEczo4InBONcBnYm09+HpTr7gq5pN8RiNnuOeQVNsXmzK7milAqtTvDutm+jVSxPt5scAZf/iIt0y6Pl4DrcYz+fDnhPiKyg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.ozlabs.org; dkim=none (message not signed)
 header.d=none;lists.ozlabs.org; dmarc=none action=none header.from=suse.com;
Subject: [PATCH v2 5/9] xen/x86: make "earlyprintk=xen" work for HVM/PVH DomU
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <3d212583-700e-8b2d-727a-845ef33ac265@suse.com>
Date: Thu, 30 Sep 2021 14:18:45 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P195CA0086.EURP195.PROD.OUTLOOK.COM
 (2603:10a6:209:86::27) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5f94770a-0387-4e6f-a28c-08d9840c7450
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6479F1AAA2E82534A51E6828B3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	lnl0PB5aWWgYLd8eDlWI1/6kXmPf3zO/JHvihmY1pyTPv7XBWBdHERXU2tYvVz05uxSbO36SGz7RlBgMyT0epVBMilxPvEc7Cpoq18fkHqcRTVb9Db9Ny8EVLbickedVCf9KhG/Aamv+wurfhUeQfiCf4mVPAam3UcJ6Wxkm84cZRLNudL1QG+MPSE+9rv6RqGtfNTBCZmVP5roOGWyGHcVytQQfmWZfNKCozU8N9+MnGJXjp4+pQF3ubHNvIXxe2bR/b3wVp2nfrmv6SkdlShpgM0pMeo83eahG5AAZbKF0JE66Zum+TXC8y9YsCdjFCvuSfA0L2zz8N0zYuWff72Qs7UPKfhILP0a/Ifs0fHUdYDVE6i9qNHRWsS3HGzvGb68DyYhk+2Z1UZztF6DUq1xrega8YE1xHt3fqy20B2HuYRJtZDg54fBhAK1N5fZGoKTIYXPXyXMYv7Xtfc1RbZYO2PEJPAz9Vr6hKM8fBxrJ9SgaB60fYK6S8ZwE4NgYEze4JKxxBxDpr1jXislKE/mjhbwFf+pVkuSVnbjovj9/ZMVn8jJBgWaVgOAMsKFF6b8jQpjtH3bUbcWMx2eCgN1jtaH5Mk5Och70uDMF1kOgncQE4VV8/ddSsfDyuvqO6iRO48iCgga8Xop4XQ796eKl2s9qey4g8QVXSzGZ11x3X0QhdkCdpBhr39v5+/XRElKmNDdegYcm3e7qI4ONpgQbvbviB6wp/1DFxTJfsPMOwkSLC8uIyYBKhtqTNRC2
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(2906002)(4744005)(86362001)(26005)(16576012)(508600001)(316002)(956004)(186003)(6486002)(8936002)(31686004)(54906003)(4326008)(83380400001)(110136005)(38100700002)(2616005)(66946007)(31696002)(66476007)(66556008)(5660300002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?MkEwQllJVlN3bXNTRm5hWElnYS8wVE9KMFYwNXR1UmtpczdSS2VsUTFWYVA4?=
 =?utf-8?B?di9PSG44OWlrK0gvbDNteHBXV0ozWmtzcXh0L2NGYThTUWpLOG0rOUxCYUNO?=
 =?utf-8?B?SlNjRFZiRkVNVUJHdGgxd3dwWG1LQklCYlgwQlRwNDgyRVlwOUNvcTBGTFcr?=
 =?utf-8?B?UkFxMEpVL3lrZ2srdHdZZ3h5cnVOZ0dTaFIyUmFMSDI3TUltU0I1bGw1Tmdt?=
 =?utf-8?B?NW9FYzk4WENOMWVLeXBucWhNcUxoZHlmbW9kdWFZY1pPTk4wSEFiOUJ3Wito?=
 =?utf-8?B?Q0VQdmx6eFVKK2t4bTNFL3pYNUlMQlpLYndsMGNIUU9QRFdmNC9tUDUyZUVC?=
 =?utf-8?B?emRJaENKSUlKZFVXN3kxR1NGSGNicE1IeVFtbDBVUXR5QWdYVTBBc0w2Tm80?=
 =?utf-8?B?eXZUSjFXQWEvWkZGWG1qaFcrSnlWbDhIdnJTbXJzV0NlZjdENlNSVE5SMnVk?=
 =?utf-8?B?czBTWGtUYnZhUEdkODR0MFpnWExJRDJYNWQ1bzRDOWM5YnN6bXBGWkpZL3FH?=
 =?utf-8?B?NGVWU3NJZ2c1OTJDbW1FWmtzbDdTZ1JRMzhxeHVuR1lULytIR0MxS1BYUUkw?=
 =?utf-8?B?OUk3NzFEeFNHMEtHMUpJVFRoaHcyVkRVRCtkRFZibGgrYXA0YzJrbW52MzlD?=
 =?utf-8?B?L1RzbElvZG5JbzhNZzlCZnp2NnlYaVZmdk01VVhqMmVYS2dHclo1cmJ4YUl3?=
 =?utf-8?B?aDVXdG53MGdwYWljWkg1bmxxa0UwTWljdXpqcHErdHl0TWQrcWJnUitTdGoz?=
 =?utf-8?B?K3RIZHVtNnp0MUV6K0pMU0JqRGRtSStDV0h2VDlYSWxTeUxxRVRyZk1IYlNq?=
 =?utf-8?B?b0U2S3RncG05a0R0N1NINXJqTVQyU0x3T1B0eTh3MWh6ZGU3em9YVTZwZyts?=
 =?utf-8?B?eTdWWStnM0c5N1A2eGxUMHZkMjlmYWRwOEJMVVBLVW1JRGZ6TEwwdlFmdlNw?=
 =?utf-8?B?OWQ1QWNXM3BLOFdxNDNjakVGOGN5aEZiUVFaY1FiTVhBeDZSekJMZHZvZXdC?=
 =?utf-8?B?ZlYwbGhiam0ya09KR2lRWXIySTFFMXRRaDZ6SDR2RWtJWnIvck1Ja0x0Zzh3?=
 =?utf-8?B?eVVkdFBkcVBZTnZndEpueXJlbE9VTFJueWw2YWdza1Ayai9DeFdpUGp3cW9v?=
 =?utf-8?B?QUpRZHFnOFN5elNta2ZnZENZV1czN0hpVE9zdlR6Y2p0Kzg2dVVLSXU5eDM3?=
 =?utf-8?B?UGYxYTZQOTBZdnB0NnVqOXR0dGdmdllhRzFhalJZdkwyZUNrK2VhNHhlMm5t?=
 =?utf-8?B?V1ZxQlg1SGYvWlp4WFV5dU5aZ3crYlFZdXNoMXY1ODNqZ0dmQlZlWTdOVTE4?=
 =?utf-8?B?L3BRdGpzVEFUanZ3NHcvU2c5SDJXVmtBNkt2N1BERnRnbjk1UTV2T2RqdGFJ?=
 =?utf-8?B?aEFzc25RSmlsZWpnVmNSR01lc3U3OUtQTXlNd3BsZXVTS21FQlJMaXdvNDl0?=
 =?utf-8?B?TVNBaWNZT0hKeElUeFJGaUlzWk5iaURYR1Qxakp6M2I0NjBYaTdkRDZlSmlX?=
 =?utf-8?B?b0pEajRTUVNVeUZGZGQ1YjR2S1J1VFAwMDl5Mk1ZMGJuYzlpSk1tVEdwbUgx?=
 =?utf-8?B?QytEUzVSUXRrOENsRmZNZWZ0RUxVTDNaQkdsNHAwOW9kcHgvZ2ZkZG9RSWk4?=
 =?utf-8?B?U0RrMkFIUjR6aEpnNzlxUER0RWt6emtIQTIwRjFZd1JxRkpzTDdiRUNieG9V?=
 =?utf-8?B?SU1FNnJ3M053Sk1RYlFZM2dsZk8zcFZKK2hEdW1ITHFpN243b2tsU0xsems5?=
 =?utf-8?Q?QYd7NQVN1nvB6JOROPhUhkBEtkeiqfh6cWNiy9I?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5f94770a-0387-4e6f-a28c-08d9840c7450
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:18:47.6540
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: g6W8VjTPWAIihlQeipm4QJQSRzAg+RJs0zq+VDcl2WuNwOoB7QZeExYb/RG7PEfM1EDse2x3ynSLLhbqEdwznA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

xenboot_write_console() is dealing with these quite fine so I don't see
why xenboot_console_setup() would return -ENOENT in this case.

Adjust documentation accordingly.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>

--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1266,7 +1266,7 @@
 			The VGA and EFI output is eventually overwritten by
 			the real console.
 
-			The xen output can only be used by Xen PV guests.
+			The xen option can only be used in Xen domains.
 
 			The sclp output can only be used on s390.
 
--- a/drivers/tty/hvc/hvc_xen.c
+++ b/drivers/tty/hvc/hvc_xen.c
@@ -618,10 +618,8 @@ static int __init xenboot_console_setup(
 {
 	static struct xencons_info xenboot;
 
-	if (xen_initial_domain())
+	if (xen_initial_domain() || !xen_pv_domain())
 		return 0;
-	if (!xen_pv_domain())
-		return -ENODEV;
 
 	return xencons_info_pv_init(&xenboot, 0);
 }



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:19:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:19:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199827.354110 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv25-0005iD-6o; Thu, 30 Sep 2021 12:19:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199827.354110; Thu, 30 Sep 2021 12:19:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv25-0005i4-3W; Thu, 30 Sep 2021 12:19:25 +0000
Received: by outflank-mailman (input) for mailman id 199827;
 Thu, 30 Sep 2021 12:19:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv23-0005hS-O3
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:19:23 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id a3aa67a2-b485-4dc3-9a45-2083fd2ca9bc;
 Thu, 30 Sep 2021 12:19:22 +0000 (UTC)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur02lp2050.outbound.protection.outlook.com [104.47.4.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-33-MhWaNnAmNSCRANtamFCC_Q-2; Thu, 30 Sep 2021 14:19:20 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 12:19:18 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:19:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM5PR04CA0032.eurprd04.prod.outlook.com (2603:10a6:206:1::45) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:19:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3aa67a2-b485-4dc3-9a45-2083fd2ca9bc
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004361;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=lYYLhi3kHcIFifWyH6tHSRW/l6Pd/aQQC0R+PsIihXg=;
	b=gMacaiA4D/NXTiybMuoD3R2LK3lkTkzUdFnkjPHrqCRORwB5pjqCZMluNzzawD7XVinrB5
	k07K3fAHMnohDDtRBObIKG4YBgrKeMEJrmj4jEEnkJG0Eg705GqD44UI3LDPe9fLN9bgga
	3KowmQ5w15r2hm/J2ac2wDP2TBUwSA4=
X-MC-Unique: MhWaNnAmNSCRANtamFCC_Q-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hQKEa8b2c9PcL+xz/rOcCZ+5r+xQTrEt27WZ3BYew9JbQ8JY/enQ2oeIEDSQAfnNSHcUY5c3ah9epCBcOnHxmI0i645tZ44cbSW2EtWfkp16pSsHhDp+9/bodomxC16wrp3UZb5kERASMSHZ4pd5D8mm9lOT93Ypqp0SvPaF3Z1n6vd1NTWTU71iAdrJiV2W2hmpezPkxPrehV8DCyu3XytLK4kidXOGKhzW1PheISL4MTmcMBsBvCb1/L/dwe1TFmUFKIw+Cx9Wa23XKqXYW2D2Mjx+HhFtGxLBXaZ2QYE0laHU5V+aIBJD+bLWT06JURHgChC2AnwcsVVykumt6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=lYYLhi3kHcIFifWyH6tHSRW/l6Pd/aQQC0R+PsIihXg=;
 b=jp6MvJL1OYx+3FNgfl1Q+cjuEhEVes4EGbHkRjlzyp2qmmvX4YHlTYljF+LGi0kqHgpB0fJNLqltmBEZVFJl0Z3q8SLJ9PQAB8xCVN5xVK86uy1W/QMYIGDwruFOPuDyC94RJjLOIH+VM1MELaEfF02ZMP3sjdm4tw+CB8XLM+sWcxAFYPXIFNSQBXRURq5RhIhiEkWpJwBDDXWwEkxgxsQZyaoNbLsujgFlJ97e+dcLKJdAz3gzaKQLXAwZYKb4hkPI3mlmveYfHPf6aZNYwaqHSgcvR6aS+gdVOP7y3N+eLYSifglUdE2ZyemlFKw6OCE+Yi8uTybH8bX6ffKD1g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 6/9] xen/x86: generalize preferred console model from PV to
 PVH Dom0
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <582328b6-c86c-37f3-d802-5539b7a86736@suse.com>
Date: Thu, 30 Sep 2021 14:19:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM5PR04CA0032.eurprd04.prod.outlook.com
 (2603:10a6:206:1::45) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 02da42aa-3468-4345-7ae0-08d9840c86af
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6479A1B8A557D70B9EF9E7FEB3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	UQAKGfoWrYfKpa+FBNEOEkPTr7Zsjf3Ial7V1xrRocM3R3wqkTyc6nGM+KMMJakrJGb/R+3UQES7nMHyo2TwWKiFOYaDXH1htJbs9M9aYGqcRBK+wO0dDAhSJ+VPrn2Sd/xvIBa8pHywwBRb7VS4BFxk1wTOqcXdqHFtDhzPskBPphvw96DH2QCCFO8AXRxval8EYvK1tw6iNR+MlaOQOPPgHJ2pe4tvn7d2irkq27Dl+vOgg6zTGD4No8Z98oJTFfEhsXzcbt7aEn2jWvR0Vrn/Mr+xRj/Bnb0j9PDkYVbOpKH5QZ45kc88dZ08Rq7Ld5P+tBtVe1E7Gn6bGxXADegorKJM1rK65tPIpYBmsb5XiVXIYkf2H2KP4YgM/ppWeHQRtb5HDR6AO/2KRDBopuuu1sKJ8hNOLHL9zKFHJdOh5EgAXLKRx5OvwuMxRi15sDLL4IcZE/OFQ/BamOAV5QFZO5+q/NzSgiND6y13IFu7vb+meT2VGZ7+5FrS7KcZ5hM9OkVmpzgGlQYK/tsoc14/q7YJhcL8HjVZzM1r0nDo2h43hsLNXG2QuI/mhQ1+Wyp/7VJ6lTK1dF/UwKAWVAJW5dWwvXPyQJNjex1qoxPrq0V0vpK3y3zX797z0wq4bJiEDBedBn7I1NhnVxlb9MR9ZKVDoplb5qDsEt5YLn3oUDV6+TZmdnqqSq3ljTsJ3Guk8CTzhGqXtIBHrlpbmGyDLc1d70X5XUaIgx39nA2ka4L3p/LiR6xBk2gFc53b
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(36756003)(2906002)(86362001)(26005)(16576012)(508600001)(316002)(956004)(186003)(6486002)(8936002)(31686004)(54906003)(4326008)(83380400001)(110136005)(38100700002)(2616005)(66946007)(31696002)(66476007)(66556008)(5660300002)(8676002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RDVXN0FvYmttK3FoYnhLdDVnZjEvcHljWkoyKzRCR3ZmeWQ5WU5qMjAxY0xU?=
 =?utf-8?B?bkRabFVlV2wyOTgwZmwvaUNETlloYk0rNUp4czlka205c1Rna1lBUFN5REU0?=
 =?utf-8?B?d3RLSEx4SHo0bTJHWVgwSzhEdWpFR2FjWVNFVFJ1cjhzaGxjVW5VTFNiZ1dJ?=
 =?utf-8?B?eG1TTCt0SWFxTmJPSFhrNjVJZWF5Y3J1eHY1TzZudDBxSHgrbDltL0lzWW5H?=
 =?utf-8?B?S012eW9CeitDVk10amRpdFVtMDVtQmZsaEMvTGJXNW4rWHFzbFYvTWpCcWZN?=
 =?utf-8?B?T0xwM1p4cXFBaXFHeWRyazZCMi9rS2VvaG91YWdTNUlXejRzMENjZlBVRnRS?=
 =?utf-8?B?VjF3MlEvYmxTVysyUFMxaEpDUEFTRUhNQ1hlY3JrZmFVZ0JyMlJzT0NLZHdY?=
 =?utf-8?B?RC9wWTFCSlNwSTgxYjROdTFxWHYzN0NTYVh0MjlPSHNnU2xTVUEzZjNxMnp4?=
 =?utf-8?B?YjFQNkdnNFU4NVBXOWFCcFpiaHNXY3Z1MHA1R2NDam9yQzd0WlhMeTE1aE1R?=
 =?utf-8?B?RG9BOUFHQXI1RXc4d2lCNytGNVJ5UU5WNDMyeHV0OXRPcUFqL2FjZFR2MURC?=
 =?utf-8?B?R01WdTE5bkNZeHowYXc2Q0FjbnJYU1hKc21ad2tzZTA5cGc4Z2tHblEvQWZr?=
 =?utf-8?B?aWYxUXZobldiVzRkWTFxY1lxNnk1UWJPaTI4d1BGdnQwa3I3WlBxKy9nZ0h4?=
 =?utf-8?B?M2Q0djk5VEVCSFpsT2RPTE5WczZ1d044eUlYN29kcG9CS0plYnpvVWpvenAz?=
 =?utf-8?B?UEtjckQySTN1K21GeEROSzVoZFN0NHFuR2lCS2pBdmRRMUFKRk1WczdBUnZ4?=
 =?utf-8?B?SktlSU9QR21WZ1FiSlJLQVU2L2trWWRCaDE0SktTWFlTaUw2aEVUT1ZLRnRt?=
 =?utf-8?B?OGRnbmQzUHQ0MW50Q09XaUtZaWZMYVRQdWtLdGNWcDBBOVB3cHplQkFSM1Zl?=
 =?utf-8?B?QmdrL3ZKd0NGaWNDTWRkcTFsOUU4VDNCYzBxSVQrNEdlNlE3YUlBaTV2aG15?=
 =?utf-8?B?R2lwNnNFZEVmQkxGQ05vSXFPSXRGR1N1Ykk2UTJiT3RUS09TeStmS0NHRnMz?=
 =?utf-8?B?aXBMM1RrUEtES1RycUt2NHpoMTluVDZkK29jK0NpYXkwRWdEM29SMDI0RUt6?=
 =?utf-8?B?Z204ajBsSjJPTUpPRVF3aWpuVmN1cnpvTjZXdisxSi9qSkNMMVgyMk1zNU5B?=
 =?utf-8?B?NEJOUXprWVl4TkpvaGQ5Rm9McHMzR0Y2NUxlRk1mUHVGaE9RVkpNYjBzQlVM?=
 =?utf-8?B?Uk9FVHdoMzNXNGVMNmZleHhMcUJQc2tXd3pNVzlCTU9scUhzZExFZWFob3Ni?=
 =?utf-8?B?WFg5YWM4a3JDT1F1U0JaM0pDeCtaSTlUQW9QUnFYSEdleVRKMnYvd3lkZCtk?=
 =?utf-8?B?cktnOXJpMEFWRWlOdGhERzYySjBDU1YxcjZJOVVBdU5BdFRJRjA1cGU3RTMv?=
 =?utf-8?B?bTZiT3U4N2xHNkFKZnhHUitqY2Rqa3lLUW00bWVpRVNlbW13cnZHd2t4K3JF?=
 =?utf-8?B?ZHB5anl0bzRxV0pidGYrRTRGODU5TlR2c0doTWpRUFg4dnJSYzA5a0xCSkpm?=
 =?utf-8?B?OTdseEVYcE9vR1IxOUovT0E2SDZ3SzdwVXpMVmlTQytURU03MEZRS1NvZVNq?=
 =?utf-8?B?VFlwY2V3N3RqSmV2NzBRbjlCRFhvaDdtcm1jbjVSeHkveE9ISDlxbXFJWHNr?=
 =?utf-8?B?VmdQaHA5YzZjT3liV01VQ3lYUmlUelJlM2dQdDFmU1Z1ZThmd0tnSk9mMWNk?=
 =?utf-8?Q?WnplLGizjskCuU8kEBAwvLH7omgqzT9JRbHTl+K?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 02da42aa-3468-4345-7ae0-08d9840c86af
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:19:18.5366
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3uiHMhrvh9IV9SsMOI2PpOt83c2MzEK6QEEyOxwjY3MhC52dEuLOvmdSm1mfolXaM4F39Y166Q3PEag7ZTjOVg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

Without announcing hvc0 as preferred it won't get used as long as tty0
gets registered earlier. This is particularly problematic with there not
being any screen output for PVH Dom0 when the screen is in graphics
mode, as the necessary information doesn't get conveyed yet from the
hypervisor.

Follow PV's model, but be conservative and do this for Dom0 only for
now.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
---
Prior to 418492ba40b2 ("x86/virt/xen: Use guest_late_init to detect Xen
PVH guest") x86_init.oem.arch_setup was already used by PVH, so I assume
the use of this hook is acceptable here.

Seeing that change, I wonder in how far setting xen_pvh to true only in
xen_hvm_guest_late_init() can really work: This hook, as its name says,
gets called pretty late; any decision taken earlier might have been
wrong. One such wrong decision is what gets added here - preferred
consoles won't be registered when taking that path. While adding a 2nd
call there might work, aiui they would better be registered prior to
parse_early_param(), i.e. before "earlyprintk=" gets evaluated.

I also consider tying "detecting" PVH mode to the no-VGA and no-CMOS-RTC
FADT flags as problematic looking forward: There may conceivably be
"legacy free" HVM guests down the road, yet they shouldn't be mistaken
for being PVH. Most of the XEN_X86_EMU_* controlled functionality would
seem unsuitable for the same reason; presence/absence of
XENFEAT_hvm_pirqs (tied to XEN_X86_EMU_USE_PIRQ) might be sufficiently
reliable an indicator. Question there is whether the separation
introduced by Xen commit b96b50004804 ("x86: remove XENFEAT_hvm_pirqs
for PVHv2 guests") came early enough in the process of enabling PVHv2.
Plus I'm not sure a HVM guest without pass-through enabled couldn't be
run with this off (i.e. by relaxing emulation_flags_ok() and having the
tool stack not request this emulation in such cases).

I think the approach here might be equally applicable for DomU, albeit
potentially pointless (i.e. dropping the conditional might make sense
even if simply benign there): A PVH DomU ought to never come with a VGA
console. Yet even then a dummy one may still get registered and would
take precedence over hvc?

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -3,6 +3,7 @@
 #ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG
 #include <linux/memblock.h>
 #endif
+#include <linux/console.h>
 #include <linux/cpu.h>
 #include <linux/kexec.h>
 #include <linux/slab.h>
@@ -18,6 +19,7 @@
 #include <asm/xen/hypervisor.h>
 #include <asm/cpu.h>
 #include <asm/e820/api.h> 
+#include <asm/setup.h>
 
 #include "xen-ops.h"
 #include "smp.h"
@@ -274,6 +276,16 @@ bool xen_running_on_version_or_later(uns
 	return false;
 }
 
+void __init xen_add_preferred_consoles(void)
+{
+	add_preferred_console("xenboot", 0, NULL);
+	if (!boot_params.screen_info.orig_video_isVGA)
+		add_preferred_console("tty", 0, NULL);
+	add_preferred_console("hvc", 0, NULL);
+	if (boot_params.screen_info.orig_video_isVGA)
+		add_preferred_console("tty", 0, NULL);
+}
+
 void xen_reboot(int reason)
 {
 	struct sched_shutdown r = { .reason = reason };
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -28,7 +28,6 @@
 #include <linux/mm.h>
 #include <linux/page-flags.h>
 #include <linux/highmem.h>
-#include <linux/console.h>
 #include <linux/pci.h>
 #include <linux/gfp.h>
 #include <linux/edd.h>
@@ -1392,12 +1391,7 @@ asmlinkage __visible void __init xen_sta
 #endif
 	}
 
-	add_preferred_console("xenboot", 0, NULL);
-	if (!boot_params.screen_info.orig_video_isVGA)
-		add_preferred_console("tty", 0, NULL);
-	add_preferred_console("hvc", 0, NULL);
-	if (boot_params.screen_info.orig_video_isVGA)
-		add_preferred_console("tty", 0, NULL);
+	xen_add_preferred_consoles();
 
 #ifdef CONFIG_PCI
 	/* PCI BIOS service won't work from a PV guest. */
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -36,6 +36,9 @@ void __init xen_pvh_init(struct boot_par
 	pfn = __pa(hypercall_page);
 	wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
 
+	if (xen_initial_domain())
+		x86_init.oem.arch_setup = xen_add_preferred_consoles;
+
 	xen_efi_init(boot_params);
 }
 
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -118,6 +118,8 @@ static inline void __init xen_init_vga(c
 }
 #endif
 
+void xen_add_preferred_consoles(void);
+
 void __init xen_init_apic(void);
 
 #ifdef CONFIG_XEN_EFI



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:19:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:19:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199833.354121 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv2c-0006Is-Fd; Thu, 30 Sep 2021 12:19:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199833.354121; Thu, 30 Sep 2021 12:19:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv2c-0006Il-CY; Thu, 30 Sep 2021 12:19:58 +0000
Received: by outflank-mailman (input) for mailman id 199833;
 Thu, 30 Sep 2021 12:19:58 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv2c-0006Hw-2k
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:19:58 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b711d58c-21e8-11ec-bd4a-12813bfff9fa;
 Thu, 30 Sep 2021 12:19:53 +0000 (UTC)
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 (mail-am5eur03lp2054.outbound.protection.outlook.com [104.47.8.54]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-22-onSn-rNINdyhkkG5Xz_7xQ-1; Thu, 30 Sep 2021 14:19:51 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep
 2021 12:19:50 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:19:50 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0243.eurprd06.prod.outlook.com (2603:10a6:20b:45f::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:19:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b711d58c-21e8-11ec-bd4a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004392;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=wR+FOs7yn/K7NS++9RwN2a7HTGpb8J25jreIb/7TIi0=;
	b=DC9K/CAy+eptAR9XbmxY2/oJOuIIhzZY2j5m4FqSKckUgZipGgvP+9mTzgpbUVOYqDdSiz
	spIk92CpxmdjahWCFc7qO2vErTtFh8RQ5FgDUbQpC3902Y3FpgSSwL+WI4SkfimwSssvQj
	nODf926IeswT4li+W6qsiF45SMz3RH0=
X-MC-Unique: onSn-rNINdyhkkG5Xz_7xQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g1jmC9y6tMl7E5M3XlCAwXIi6rts76/LxdBtWyVHIR9/itF2Y/U4j4FqjtWSa/hYIhiKtxnbUHgyFAlYberMgiuk9Id7Q/2enVJMwLRH1hlRWc84ueICSD4zo3ZjYkPu2VqYfe2Byrj1jnwBggu5SPOlBMGnxgROjk/gt0xbNRWqeyQVpxtl1O/hq7wOwnhlLFvVBNNgUWuGkSSnEXYOU1Fafuj/KJQSyiSqriPOBq0yVyxPAnoP9R4ISpmv2ChmkDJu3PzY1K4YQqVCrMYT+LF3jMjHmk5egVG/EQkvRTqiUA8BAANGTHvifZgLXKworVcDINi/gY1UA4HoyHZWcw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=wR+FOs7yn/K7NS++9RwN2a7HTGpb8J25jreIb/7TIi0=;
 b=f67+Bgc34Axk7/FlgYarH8n7KvIM047m27VJEwb2dEagF4iqSNwNzieEd7Z/yonQ9+SfeMOYObxnS10r4zjCjr35WLoWN1mNy3XT1jPi7kBpygzKNbcn8dE/1jl2I2A92Z6JzriN97gLr0xivYFRLZooar7ON2qPcsoPGSScSX2KVO3DkZ3UsoPuFZsmQNTiFNqWf6tDtHVuftdWmd/kSclu8fhY/4p9+2FAesdk0/EGW09kPUqMKXvriy+ay7gZzNiD052mf4UfF7dzLXtxEV6o1NbuH3OmXvARntS2QMLRrKCaCptFmgJsKx82MdYfmp+Gk4fUgBscLiMaG2IEyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 7/9] xen/x86: hook up xen_banner() also for PVH
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <de03054d-a20d-2114-bb86-eec28e17b3b8@suse.com>
Date: Thu, 30 Sep 2021 14:19:48 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0243.eurprd06.prod.outlook.com
 (2603:10a6:20b:45f::6) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3dbbd057-b871-4e24-a2bb-08d9840c99bc
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038D4E79662802BF3B80A03B3AA9@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1013;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WZvAPPnHdx1InNkFrXCgdFVfRHrScIMRa8RXaw3IWm6VYtXGFi7Xd4npoEReMHWmdCd47EQzRxbs9IlEx7sY+Op1XNpboD3eYzQmDsZU6x8S7QQuyFRiZW972svkFRKM6Mn95/UxoHYJ9ziDUiJeT75lA9omodJhHJMLPdK9w6FFygTURFQepiXTlGEahKMdMBG9yPXWM/5AXZGMlThi2YK05Q+Em3oWw7hB3aOfn8Hzu0UvdulXz1ZP44p8FBynHbwnn4EHeLRlx0qLz/zD1nnkbb5NWfBciRYH6i9243F7SI7gQOQj5jpmvXz1ow4sRBdVUwJ+FoyLCQKRtLyaHEyEJ+QzXPDv6uvmw/BeTmVu0fdIlqA6aC13MsF3bPmgnaIGWM98+atS7EwHo2XS0z58rK45w2usMyhYYc9iXbDZUcpi6L1Nx/i4ikYCSAv3giTeNmVvF7W1AJLgeYGgU+0cJIV8fxSREJnesDl/xSmuiW2ZFiVeBZiSmCMvAhSD4ABq6DNJawfzWCbI7Mv6O2Leo7hvO5/EzYWV09JroBRvduuZQPJpDSdSUhc6sF1xiecuy5qDqP9ceBj7Ztj4rdQtrKdE+YqZpGFNfNkpFDaEcN1+xM7HCd2jje/yApDbm3Z5f/AIayrnbNH4HXj/LrN3yom9x9VEICeXe31Y7UdIhU4zScnBLeioR71FcnlRu0kEv+m85tFxU5qT8vTJZQTrC3Ht46Qmao0PdJIm7t4uAJxn57lsRGCoUV9H97vV8BxnLIV9geqGfWK6L4/msKy1T3EMaW+Ds2nfak0nZU8=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(956004)(8936002)(31686004)(110136005)(2616005)(316002)(186003)(38100700002)(26005)(83380400001)(5660300002)(6486002)(54906003)(4326008)(8676002)(31696002)(16576012)(66946007)(86362001)(508600001)(36756003)(66556008)(66476007)(52103002)(158003001)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SDNHZTNYNlJYVmhkNXBmaEhRVGFDbTJrSGxlTTdGZWxScWJwaHNPeDVOMWkx?=
 =?utf-8?B?NERNdHZhMTljUlRaMVVveU9scC90VG5USkpkT3BRc3NDOTc0RitVVW5MK1cz?=
 =?utf-8?B?ZlZmakpFcndVZHdJQlYwY21YbnRkK0FCdXJKZGhDT3Z0aFF2WjdKYWJMTEJm?=
 =?utf-8?B?Q25xWWhtdWswZUZyT3VGYmtjZUZrOFd6RGQ4UVY4OS84SVNSV0k5RUVPNWxx?=
 =?utf-8?B?aWt3OS9yNzNEMHpSaUVpQk9IelRMdG9GYko2czVIenRscXo5MHQ4S0F5UDdH?=
 =?utf-8?B?Wml1bFVVRjhjVjZFSDhTQXZ6SFRuTjEwcWY0cHZZZVdnMGZBbi93K2pJWFdj?=
 =?utf-8?B?V3oyZUdNdVl6RHJNaU9wejJlaUlTTnp0QW81M3UyaGFnbW9XTjlXaHgvb1U1?=
 =?utf-8?B?QS91L3lvQ3ZycXNVamJ6OHBOL1VFSXpyeFMwNlJWc0JWTHcrN1V0NDltU1ZH?=
 =?utf-8?B?T2l4SVhHTHlPTnpWc2dEUFdlUFR3bmwzTytyRTRqeFlrb0h1K2xSRkg1ZHBr?=
 =?utf-8?B?MHFEY1VuS3RLbHU4V1ZpU0hvT0RBaHVleFZ3Rjh3L21WWmFIOWM5SFdtUWFP?=
 =?utf-8?B?TnI1WTE0RnlkVVpmdWhzRFcreGRJdGY5WE1VTk1jL2tOdEZkMy9EcWtrVmM4?=
 =?utf-8?B?OU1XQzZTOUtjS3FRdlR2WWZzM0lSSnFiR3d5MjUwZ3JtbkRMMnFRdEpGY0pO?=
 =?utf-8?B?VjV2NzlxTUp0ZjVyR3MyZk40dHBzWTlxM0lEMUtIREFZcXB4cDlDVm5vK2xq?=
 =?utf-8?B?OHJGeWF5N1FGelE3MjV6VWtSY0NYb1ZHK09iWG5hdkdVZ01YYkRlR3ZMUG5K?=
 =?utf-8?B?OHRKYytscUdaOWdiZTh0aGFhSEVlQlM4SGs0N3RrSEZnUjI4S1hRall3cmhn?=
 =?utf-8?B?TEpmcGEwNzRCODJmei81a3NDdjlaL3l2bmc5OW00S3FKWmVrZnNpNkZvMFNC?=
 =?utf-8?B?TXJaT1ZQM1N4MkpQK1h5b1NtVDI3dFhOcnVqTE0wcXYrT3AvS3pqQTl5Mmlo?=
 =?utf-8?B?Y1BUZDdQOHNObVo5cDRtSFhTOVpNMXFpOVUxUmVvL2VTY0dseVI3K3FUZmI2?=
 =?utf-8?B?Q2VBN0ZRUXQvWlZaSnpRYjJuTmF6YThpdUR0T250cVY1R3BLZ25VSmI1WEJI?=
 =?utf-8?B?eGcvMysvUUtDTzdjU210TFp1WWZKWm1temd6ODkxRjZYMlBrN3dXQlhtK1pi?=
 =?utf-8?B?bkx3czF0anRERldUUlluU0xFOVNqTFVqUnVSbThvQW4wakIxMWNJYms4WUtj?=
 =?utf-8?B?bFViQTRlbmFXRU5zQzNhRjVncVVNZ25kWGdDc3FweTVuZXd5UUZlSmFENFVr?=
 =?utf-8?B?S203anlIR3ZQZlhHU2RrMnpCaUZCTkswTm1hS0xLNlZQd0gvVFZGaml1SEg1?=
 =?utf-8?B?QVduUUthTjJHN3hsQ0VyaWtPRitNTjJRcWV0bHM1bzY0ODk3M0VLYlF6aXov?=
 =?utf-8?B?MG9JdnBrSnFqajF1VkNiVHRrMXdVaGxqSzhEYTVYajhMVU5hUy95enV5ZjAr?=
 =?utf-8?B?QmxHTnA1UUVOa2hBY2E5NTZkZjR1blVzanA2bVJUdC9FbFg4ZkdsM1RTTTFJ?=
 =?utf-8?B?Y1V2ZGpBYnY4dWV2QjZPaFNVa2pMVnpDSHd3TVhka21aSVZCMWZSRE9IL0pJ?=
 =?utf-8?B?aEUrRGVUVUY1Y1NBRkkwNUNLNTh6MmZBc0dscVl6SzBUcU1hbXZmdlA3S1FV?=
 =?utf-8?B?VEVUMGYyR0x2azB3Q2VOL21PVExXQ01UNmUyMmdUT0VYL2NXcm03ZG4vRG9m?=
 =?utf-8?Q?JVLLLFkauGlengPGmM/YvszFsLArITPCI3dM73n?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3dbbd057-b871-4e24-a2bb-08d9840c99bc
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:19:50.4325
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0Q+cofDn+VPWsB7ANXEanaMpMi0NbjJ5bkfM8TZ2lDohi2iaGWF7L3tGO5FYYopL5HMW006PPHQEmNZ3KPq3EA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

This was effectively lost while dropping PVHv1 code. Move the function
and arrange for it to be called the same way as done in PV mode. Clearly
this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
check that was recently removed, as that's a PV-only feature.

Since the string pointed at by pv_info.name describes the mode, drop
"paravirtualized" from the log message while moving the code.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Add blank line. Drop "paravirtualized" from log message. Wrap a long
    line.

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -261,6 +261,20 @@ int xen_vcpu_setup(int cpu)
 	return ((per_cpu(xen_vcpu, cpu) == NULL) ? -ENODEV : 0);
 }
 
+void __init xen_banner(void)
+{
+	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
+	struct xen_extraversion extra;
+
+	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
+
+	pr_info("Booting kernel on %s\n", pv_info.name);
+	pr_info("Xen version: %u.%u%s%s\n",
+		version >> 16, version & 0xffff, extra.extraversion,
+		xen_feature(XENFEAT_mmu_pt_update_preserve_ad)
+		? " (preserve-AD)" : "");
+}
+
 /* Check if running on Xen version (major, minor) or later */
 bool xen_running_on_version_or_later(unsigned int major, unsigned int minor)
 {
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -108,17 +108,6 @@ struct tls_descs {
  */
 static DEFINE_PER_CPU(struct tls_descs, shadow_tls_desc);
 
-static void __init xen_banner(void)
-{
-	unsigned version = HYPERVISOR_xen_version(XENVER_version, NULL);
-	struct xen_extraversion extra;
-	HYPERVISOR_xen_version(XENVER_extraversion, &extra);
-
-	pr_info("Booting paravirtualized kernel on %s\n", pv_info.name);
-	pr_info("Xen version: %d.%d%s (preserve-AD)\n",
-		version >> 16, version & 0xffff, extra.extraversion);
-}
-
 static void __init xen_pv_init_platform(void)
 {
 	populate_extra_pte(fix_to_virt(FIX_PARAVIRT_BOOTMAP));
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -38,6 +38,7 @@ void __init xen_pvh_init(struct boot_par
 
 	if (xen_initial_domain())
 		x86_init.oem.arch_setup = xen_add_preferred_consoles;
+	x86_init.oem.banner = xen_banner;
 
 	xen_efi_init(boot_params);
 }
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -51,6 +51,7 @@ void __init xen_remap_memory(void);
 phys_addr_t __init xen_find_free_area(phys_addr_t size);
 char * __init xen_memory_setup(void);
 void __init xen_arch_setup(void);
+void xen_banner(void);
 void xen_enable_sysenter(void);
 void xen_enable_syscall(void);
 void xen_vcpu_restore(void);



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:20:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:20:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199838.354131 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv3U-0007ft-Q8; Thu, 30 Sep 2021 12:20:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199838.354131; Thu, 30 Sep 2021 12:20:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv3U-0007fm-NE; Thu, 30 Sep 2021 12:20:52 +0000
Received: by outflank-mailman (input) for mailman id 199838;
 Thu, 30 Sep 2021 12:20:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv3T-0007fc-C2
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:20:51 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5ad345f7-f49a-4cbc-aafa-a682679f9172;
 Thu, 30 Sep 2021 12:20:50 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-11-5AYQGfvzPqC3WQV53YyTvA-1; Thu, 30 Sep 2021 14:20:48 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep
 2021 12:20:45 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:20:45 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0138.eurprd06.prod.outlook.com (2603:10a6:20b:467::24) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 12:20:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ad345f7-f49a-4cbc-aafa-a682679f9172
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004449;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rcnbfPfvjwvbRBDjXE1pl6H+OGafQ+4UuiRZ3VCNkN4=;
	b=Cp/nz0BNVd+JnG7JJncC3xJH5rnono2A6oRiIoYN6DQC5ba/22optT0SoXayOIPuIrUHQk
	gI7Xg6cfNe0PKqtDMaXVXkeD4JIfcCFs1a4vo0t4KUJ2HgmHRTuokHnuhgFCmFCTYhRmVW
	a59OhmkHi/VgvRmaPWmVwoZd3xrdWZE=
X-MC-Unique: 5AYQGfvzPqC3WQV53YyTvA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UrBJTr4FxA+8BvHSEZ3m+6PzZSDClJT8lUEHWNuyPzJRqWvdWndhdt3MKur0ZHdnEJDZMauyvi0X/p6HXQUmwyr6ESEesEPyhW34AzeLTjY8hnKOLNSX5jIHYCA7FxVp4rXtMHMvreBwkduk58OHk2flEPTNAwxgOPfw/hHB+Agz0Tn3j0RU5b+JJwJS+rxlCs/hXsmgnnikW1aTHs7SquU8e6CG+ahSTmL9M5t0jqrt4nxZoc9KR3u7fcd4Z7xACB6xDKeKeDbCqN+hdrHtQJEXF1hqOROKTHA2UWVjbrcpuf1dBLLPiip0p426whTH6Sl+719R84ZTzO3C30bVlA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=rcnbfPfvjwvbRBDjXE1pl6H+OGafQ+4UuiRZ3VCNkN4=;
 b=gTMbXi5NorRXIPD1/M6hvcSJ8QYJtdKB65jGRcsnJ9l+zRtZ2ViXG7Prs5ZDkncIkKXj9QMcHW6GH3aDxqIZZOoY2Dcj4OqNfHNRn7p04fzwUo8p4Wvxii/Ds6hO4UFKPspF2uCHX/OV0Pwq7GIJ6egCv/N/y9RPbOpTuYT/hr+L1X8vzgEWFKmfIc+KqJBI0lO36SV85AwcFxr+kUHHSDt6PehUafpGOEvaYsKX+YHzcGUitAvrHk7npAdeUcamcwa8XKZNzbeEAhk9cdfXEXX7JQUh+rOBm0vV7+EeefOOwGYoYX49bQPGdUwHmFJkCvJFECGwqIV8nCpzruZudg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 8/9] x86/PVH: adjust function/data placement
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <3b0bb22e-43f4-e459-c5cb-169f996b5669@suse.com>
Date: Thu, 30 Sep 2021 14:20:42 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0138.eurprd06.prod.outlook.com
 (2603:10a6:20b:467::24) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2182c689-5b0b-4c06-da13-08d9840cba4a
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038917799A871473AB602A3B3AA9@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7URPomHxvGjus8IZCVo+VjyB1HBy2tjx1Gu+BOnX/3gASZa74UFTy4bJrgH6bD/kzfJ8LWwJm3x9vGa5siHDDAPchRtASpUdop5gBUuMaI64aJKQuZfA0psKGycr5lBIpAGmqWVaBAPaxhgNIOsT+fEvyY9BAKFAkwv8resUFWJu6wVWUTT7dZLhS9YwsryV7EG+LFnIDqutKOj8NNY+1OB12laQt9r6RlWngkalJBkZoWmW8vGThOO2c1OMd12MT4Ww/LJmDbb1/z3TVAhXZX4eBV2Oicrfg8IFfO0fFIZ4ZqGJtCleytDXjzVmYwRPk25Wq3pfzI16IO7ROC0GcJxdU0liC0xAHMh0SKaaaIkj9kAiAmxCmurczYcziaSFmMLYme6TBqDkZQeq0boLnIxE+cvsBDkuqe0UzDDd7N3EOC7jQG0MqKPspJtBh6GUMw56OgsvRp4xM9gWOeJZb1WD4zKDZg14vyfFm8cEWQlzqFW4SZdaStW/Tmvq5PxIf7kvENcJk9+SwDdPyUkImWdJh3jhmLa84lcd5yJ9HeL4wexCbD6bIeKvGnpcELpK98Y+A9Ohbky+LLZl0s76cpKnB61YK5kWVojDxSpuMP9v4aIuWx30CBUnSyl1Luv2TZTMA+rpUZaiMbhu83fgr/lR+ZSpNsEDisQw1O4YTSM/bHllwaI7nGPfTFmhWT5v8ahp1fRmydK9dJeDiLnY7tw4leGOEmLeHm7G2oTNnUnLhwBvlvS9J+c/IIhhPhRK
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(956004)(8936002)(31686004)(110136005)(2616005)(316002)(186003)(38100700002)(26005)(5660300002)(6486002)(54906003)(4326008)(8676002)(31696002)(16576012)(66946007)(86362001)(508600001)(36756003)(66556008)(66476007)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?blB5NVg1blZyM3oyaVJpWXJ4WTdndklUUUFPSEIrUVVsc1FyV3lsdG4wREZ3?=
 =?utf-8?B?UDQxZENhVTI5RUJadmJLWU1tWlEvZlFEbmdDaFRJZlhDQnUxNE5ZaWgvRTVU?=
 =?utf-8?B?OGF3UFlQamIwM2E3eXNBMFlnUjRzNjFNb3ZoZEhpRnIybjMzR1VUdTZZdVB6?=
 =?utf-8?B?aEtGQUdkbUxNYjcyK1BIOCtReGFjOU9wVEdLaFdJdVRtVWFSTDZjeEZGQUhl?=
 =?utf-8?B?NUJEaGRxUDhLbjlEZXprU21BQzlZb3JMWWZ0VVBWNHE5WDdadDZReWhwTkJG?=
 =?utf-8?B?L2ZWcmtTWnVwZDk5Q0xlVU84eGp6N0YrVXlnRU52MElTR0JHUW15UzU1N0JI?=
 =?utf-8?B?cllHZ3lwNHhNRkk3b05jbGs0bVBqbVpJeVJOeXVjZXNuR3NmSThYaEVwaXNv?=
 =?utf-8?B?L2FRakpJbzhtMVNlbytFaFNidk45eldMdGVCRCtTQlMra3hjRnV1U3NOZFNJ?=
 =?utf-8?B?RW5xZFAvNWZqR2R4VjBLR0FSdVljU0JiNmROUkU5UGhnelo2WGR4eWZKQUVt?=
 =?utf-8?B?OFNFT1BzdWlvVVlEeHhDK1h3NjVXeW9GeWIwa1h0VngrN0tLSlFBYmNZeUFs?=
 =?utf-8?B?VTVKN05TY09UK2FsNjMyamhLVmtRTGtNTnJuOG5BU2NMNFFTWVl3YUpUbHRm?=
 =?utf-8?B?NFZmSHZaY3NVblF5RzZGQXh2WU1DaEhsVlJqNlcxQk9pS083SXppNFZiTFJP?=
 =?utf-8?B?cis4aXkrL0dNaC9ZNmVFd1oyK0Z1bmlYeHFkL1lKY1pWYVZjTkxrcWJweERv?=
 =?utf-8?B?QVpJdGFsdWFsd2xWSm9NSm5QazZTeXZ4Ylg4NUZkaS92NmV4MlAzQ203QUE1?=
 =?utf-8?B?SjliYjVzdlZsVGRoRDRVU0tTclZ2OUdlMENZOUJLZnJ4bUp6SDZuMU1vd09s?=
 =?utf-8?B?RzFHK3luZE41YVJYNTBKRG9Ga2xRSmlvdjQzSEh1YUNjdGNSQ3ZRYmo0SFgv?=
 =?utf-8?B?eHBzZisvZkxZeFk0OU41ZXR6ZlpLODd2M0pyM0tLeWl4c3ZsWStQbHhUd1o1?=
 =?utf-8?B?UGEzWm5ZVS9PYzVvckFjNGROeE5oaldVUWF1NVFZbjdDWnZ1VkY2eEFxbHYx?=
 =?utf-8?B?dzY1amNKaVBpQXg1NDJaS2hKb05Pa2tJZm93NnpPMWdHeHkwR3haOFJscWdx?=
 =?utf-8?B?a3o2Wkg3c1RNQm1TTHdDYUpXdUN2NVhvUnVlWkFOUWVPa3puUmZHKzRMdFlZ?=
 =?utf-8?B?N25BbWxaS1kvODdIemhFWjhSbEdleFJVbjRoQmJlcG5PTVNUZy9sc2NOWW5h?=
 =?utf-8?B?SThmTVlvdUNwcDhyK3loSHdOSVhIb2ZIalVrcEU2ZFhHVmpBTElscFgrUGh5?=
 =?utf-8?B?RlhRV3E5VHlhcVpGbE1DM1dwakR5WVh2UVJUeUdVMTZPQmR1SFRzTWJmRVpk?=
 =?utf-8?B?ODY2RGtDUWU2S2VURXRQcFNEZHVJeHExZmlVZzhVWDNQenBkdENFQzlHejJR?=
 =?utf-8?B?Rjl3cHdZWnVMNzVJcUY4eGN0MDlEdEljZVU4RHc1aGNiQjBEOGxYVm55STly?=
 =?utf-8?B?NzB1c0xjQVh6aUUyRWVtcCtjUDdYQ3JEVlF2NENXVkp3UktRbnNJTXlYS0Mx?=
 =?utf-8?B?RE1JVUZDMHlBTjlUMHFtN0JlQ2YrT2NpT0pPYmNSMjZuUzhMNzZlSGFLRmp4?=
 =?utf-8?B?VDBHdXVQSFdZRXIvc0RtcmhnQ003OEpLN1RSdXk3Wm5ERVV0MWp1Wm5zeHJh?=
 =?utf-8?B?ZFQ3M2ZKUDkvejdSWkw3QWtPa1I3aGRXcXRjcjZWMTRlaUwrVzhKWVlOcDV3?=
 =?utf-8?Q?uAwk3aZIqgE9qRXdNMwAnnsfLOyewp0qIGGwNBe?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2182c689-5b0b-4c06-da13-08d9840cba4a
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:20:45.0786
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ppjs6jifD+dQgrIqxFi9CzgyTJJcyBLRME/y1Xr5CHLUX6DNCQ4KPok4NA/IvKleABQxGHy7f4lZLrZJ2We+3A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

Two of the variables can live in .init.data, allowing the open-coded
placing in .data to go away. Another "variable" is used to communicate a
size value only to very early assembly code, which hence can be both
const and live in .init.*. Additionally two functions were lacking
__init annotations.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>

--- a/arch/x86/platform/pvh/enlighten.c
+++ b/arch/x86/platform/pvh/enlighten.c
@@ -16,15 +16,15 @@
 /*
  * PVH variables.
  *
- * pvh_bootparams and pvh_start_info need to live in the data segment since
+ * pvh_bootparams and pvh_start_info need to live in a data segment since
  * they are used after startup_{32|64}, which clear .bss, are invoked.
  */
-struct boot_params pvh_bootparams __section(".data");
-struct hvm_start_info pvh_start_info __section(".data");
+struct boot_params __initdata pvh_bootparams;
+struct hvm_start_info __initdata pvh_start_info;
 
-unsigned int pvh_start_info_sz = sizeof(pvh_start_info);
+const unsigned int __initconst pvh_start_info_sz = sizeof(pvh_start_info);
 
-static u64 pvh_get_root_pointer(void)
+static u64 __init pvh_get_root_pointer(void)
 {
 	return pvh_start_info.rsdp_paddr;
 }
@@ -107,7 +107,7 @@ void __init __weak xen_pvh_init(struct b
 	BUG();
 }
 
-static void hypervisor_specific_init(bool xen_guest)
+static void __init hypervisor_specific_init(bool xen_guest)
 {
 	if (xen_guest)
 		xen_pvh_init(&pvh_bootparams);



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:21:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:21:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199848.354142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv3z-0008K1-6Z; Thu, 30 Sep 2021 12:21:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199848.354142; Thu, 30 Sep 2021 12:21:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVv3z-0008Ju-3j; Thu, 30 Sep 2021 12:21:23 +0000
Received: by outflank-mailman (input) for mailman id 199848;
 Thu, 30 Sep 2021 12:21:21 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVv3x-0008Ji-M0
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:21:21 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id eb3c4875-2fc5-4628-bd66-b96927cc7f2a;
 Thu, 30 Sep 2021 12:21:20 +0000 (UTC)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-R1EdHtdwPb-ypfUNmqcC8A-2;
 Thu, 30 Sep 2021 14:21:18 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7038.eurprd04.prod.outlook.com (2603:10a6:800:12d::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep
 2021 12:21:16 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:21:16 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR08CA0044.eurprd08.prod.outlook.com (2603:10a6:20b:c0::32) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:21:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb3c4875-2fc5-4628-bd66-b96927cc7f2a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633004479;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9OR7SHokyJQPfRnNpqW3y2s0XYRmL/qDbA1LJNDMXcA=;
	b=gCbkq4xI+Hq+dWhq+bM4Zi2UdyK14TEW9bE4QBCgfFq3AuFnRDH1l+E4c1LSH9adohP1U5
	yjq84A5x5eUYMB0xce7jK/hiF+19+Zkg8eNa2MZ0laCdzEgwyMRZL9MtqD3h2WsNdo8sMG
	9g6dFZiKkSCJidlaMaCSdr/nJbSFb5M=
X-MC-Unique: R1EdHtdwPb-ypfUNmqcC8A-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UpUYlH/GwjqQxxAyph4HQ61KdVBWuzcE+U3pVdILhNNcYHrkzJx5DJlzsPSCVYJ3Kh7OkjJ44gJ9hv6n5GzNj22jtcQVn2l6trPzXEGmJ52U9AJNlGuvafswwfTbyDI3RwzWq9Ko42xFSvmtYduqIkOGlTa1jHOIzxMUx5hxkEQ0owBJ4Ykg/qTzq3u/EQwYE7dRCX2XOMXv5OC+r0kif3Gy/bJhFxW0rssnzhFOTU+qPe5hQd96bDPnZNMSZ/KIgCMoteGZ5wNKsA/x66xJhvB8VHLOR9rt9dkLKI9rnYNkEPo7ctZ207q2CIaZISstn9DhU5tMn/K84f2WYG44tw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Ditd68ADMvga99ZSd2S+JMyE109ILHuWeFBMIRmBXp8=;
 b=CV37N+srVIulJ63nHpN171ukDgxMSgk/z84ghwN0bFrTnsWhEF7Rm3SLwPGJNuzCoHCyoZUriWzTw1d0lwFm/ciVd5q5YfdW9Y/J1pC41vTCYic1SfuRHv0NodvVAvuYvTW5qlG6+aJVHz8nnAsWAlNf2z/i8uGjTtCSU2BfHFBzStmOfS5IjXCI0EJQdxKa2OqndNogxm8cfxWLLbusnR7Hcf8jkgfUZrqVWH8HCVyiZkYCdgCSonCkhSl8tc44owzkp3y4PF2ujmo846e9jeZUpTOJ29U8uH952N8F8s7b+v6oKdggM+koOL4y/o4fke6tp8UOl4nvLd+Rj9Zrmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH v2 9/9] xen/x86: adjust data placement
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Message-ID: <8155ed26-5a1d-c06f-42d8-596d26e75849@suse.com>
Date: Thu, 30 Sep 2021 14:21:13 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM6PR08CA0044.eurprd08.prod.outlook.com
 (2603:10a6:20b:c0::32) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4aa12f35-be2e-4f20-45a2-08d9840cccae
X-MS-TrafficTypeDiagnostic: VI1PR04MB7038:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7038253A0D7CCCBF98226E29B3AA9@VI1PR04MB7038.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	zsmmEdYh/SJGGSnvOAJE/uiDUt5nXS1I1q0ORdV8qO67IOcTSAbp7B7tWsiBgcfLiSJ+TuyYsUvoyzwk30aXUsLFkwgPBHsMP2VQ+Hyl307pNlP0jz0a4oRXK9wdGOTiB9mNVod5DcPjMvSImBrUSu3Wcmm2cfqiUfuJzofkpL5FnU6A2Qq4QVQqPAXdzb+rGcH8G84sn6K3jG9NdOB0FlML8Qma4fWVu1CWk4rxcyOWH/TT0liEnKdRkJfCfWArPdiPokFSQlW5UYQHa7LOUxGmv02dt4fm9veTX+pWbRdqx649VxzsfxSpLJgk44ioiTPOJe0tWMtReQyqhqRKUv6jHMmVTH9fW3iCTdUwpyLIPXjX3KWrHUSz2G82753x4A+qsYAX6W+PwGjfitu9mUa4dyivC3Wyyv1AnryPG955clJfKvhudhRjfC45a/9R3br/N6IKAomxQ6d3EfXJipGtNo1y4Fzc296Lr5YUwhVR9Mt3vbYJpGb3eIKY5lw9gUeRwe4rZzZ7ikP3cXRNudYFGKUjcQ7apMnF5l2xkRDFT7NLDx+DKfP3DjG75Kk0pNRtmYJtikREzHdndIYj2VpbAQQyrI1Bo0lZsofdct/3l3fX2RueJEOAphQTVn/pFqbH5RVdS7TTNcAu0OIVLVVo04zKZZpbYFl/tlbjkyXLslyfBEtp2aU/K0UFtVASsUho6ch41w8VRgDH8xpOTjRWeMV37uVjjt+nLWrgbpACYa6ldPna7LiTNhC6nTCm
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(6486002)(54906003)(66946007)(86362001)(16576012)(508600001)(36756003)(66556008)(66476007)(8676002)(31696002)(5660300002)(8936002)(31686004)(110136005)(2906002)(956004)(2616005)(26005)(38100700002)(316002)(186003)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?n3v2wC5ga7QHVRb8OQ9O8FqR4sL5QZNRpApXIrP6j1D8BQlDxdiAmJO2y8sv?=
 =?us-ascii?Q?Ovr9V8EKikUP2virTtcCHm0R/jI6CnrYPhbchgFd3AKjuLKBgjKlR1AkzxiG?=
 =?us-ascii?Q?zB0x80WYEfIOevUqv2ZKfyObzWFr+ZMsCc9CfXXQ8rsAHsEzaP9N6RFaCNIH?=
 =?us-ascii?Q?HVQKJqfB9YYREx/NoV0BL471Fvz5HPxRbDrO9Xd0Zo51a6XNfqEZKUCHCKig?=
 =?us-ascii?Q?fv1ESpAQXkCAv1daFO3V+dST2sodgCUng5U6Q9Hgn+h+qihw4EZ6PjPEa2RR?=
 =?us-ascii?Q?WL/KpNa9StH/zH8KCyvEirT8Q74ankMr60ur9OLSIhP/FA8aRaO7P+kun5oX?=
 =?us-ascii?Q?aqDgdA10YLG252aBZz0mfq+pwptnMWAgm1c3n+SwvAF61yVAKpuZ2D9vM37Z?=
 =?us-ascii?Q?9oiNIBIcm3M3zM6vz83byONCttCAk/u7FY29WUnpLhEwsfWuUt1pAQ1Seoqy?=
 =?us-ascii?Q?DGywVgIOsSTGCy76a4cpOS83ODyYzGM6pMJUuT3lIsWLJz2ucjap5LP44SWK?=
 =?us-ascii?Q?Y1ZYVS6006SsldVSaPuCMSi3qHDYV4aiJOzZ++Yg4q0/5Q+geSbjDExUZz5v?=
 =?us-ascii?Q?BDafk7fmHzzb6G9y06sj0pCSqx367vDyVBGsLTm/LQduELx/wfINUdruUz5s?=
 =?us-ascii?Q?7xeI0k9qM5TuacIxzRDVc4/eZk5G2lIba7j2IhbSfZbDmqMWBa+4uL+E0uJI?=
 =?us-ascii?Q?Eqot2JPVh848Qy3/ryufn3Gx+lGHtDTCBPTnp5NhdBA4dfw/HksAliw+FFLl?=
 =?us-ascii?Q?0bVEjvDc6yXoY8n98dhfqVqz9Wgx9SsGKkPKaN+DVxYDbwLzpE9xBN1AKuor?=
 =?us-ascii?Q?upOlJa66fwZRTZNzXWS5sONbYhwvJ8GgGAuVK3hBgP4iaBd6y1qEoXarQ5/7?=
 =?us-ascii?Q?pmSAlRZVL7Da0shrQsQt9515M/rgdscmjsgj/GQcL53T3IS21pu6vUWd86+T?=
 =?us-ascii?Q?GcLfvTgAy/KiCK6ox7ls7YyRpTsx6sEzDGoUzO8ulisBoTjMbkqEjf+3aoLl?=
 =?us-ascii?Q?XXHxA4Knd2rcZ5cNbSKy4zvxK/CVX/3yg0mvl0d5jwC6G23nqDP+Iv/QfaJ2?=
 =?us-ascii?Q?QCmKHF9njCVemqBnxI82LJnZtOt3MXlrzk7rNf2T4gBsEu7FvHZpdE3T+scb?=
 =?us-ascii?Q?9txd1aG10c8XElBbt8J+zE/yUl1UIvZuWm9XMyvI5gzpr4peQSLjukzMEH99?=
 =?us-ascii?Q?nJGjvRg719HQdxBVFc4Spgimj9tzI07BFEzRt683yv0hFBAU8n20mbXwUHmm?=
 =?us-ascii?Q?nHwy8Hl7av2rc7iqzOF7K6MkR4COiRF9mdLUwbMhkf/SFF8IPEH5xXV799xG?=
 =?us-ascii?Q?d+qZzfweJ4CSpxNZ7wVY4GAB?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4aa12f35-be2e-4f20-45a2-08d9840cccae
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:21:15.9612
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 954wTYkxX9m5rOjLoyvleCp5N6DGDbZG1BjyA40SVZduq+MykI2PgMgKYRFbVzCt5vOo7bS2PLhXCApvTzY0nQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7038

Both xen_pvh and xen_start_flags get written just once early during
init. Using the respective annotation then allows the open-coded placing
in .data to go away.

Additionally the former, like the latter, wants exporting, or else
xen_pvh_domain() can't be used from modules.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I have to admit that it is completely unclear to me which form of
exporting I should have used: xen_domain_type is GPL-only while
xen_start_flags is not, yet both are used in similar ways, extending to
xen_pvh. Picking the GPL version was suggested by J=C3=BCrgen.
---
v2: Use EXPORT_SYMBOL_GPL().

--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -75,7 +75,7 @@ EXPORT_SYMBOL_GPL(xen_have_vector_callba
  */
 enum xen_domain_type __ro_after_init xen_domain_type =3D XEN_NATIVE;
 EXPORT_SYMBOL_GPL(xen_domain_type);
-uint32_t xen_start_flags __section(".data") =3D 0;
+uint32_t __ro_after_init xen_start_flags;
 EXPORT_SYMBOL(xen_start_flags);
=20
 /*
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/acpi.h>
+#include <linux/export.h>
=20
 #include <xen/hvc-console.h>
=20
@@ -18,10 +19,11 @@
 /*
  * PVH variables.
  *
- * The variable xen_pvh needs to live in the data segment since it is used
+ * The variable xen_pvh needs to live in a data segment since it is used
  * after startup_{32|64} is invoked, which will clear the .bss segment.
  */
-bool xen_pvh __section(".data") =3D 0;
+bool __ro_after_init xen_pvh;
+EXPORT_SYMBOL_GPL(xen_pvh);
=20
 void __init xen_pvh_init(struct boot_params *boot_params)
 {



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:33:35 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:33:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199854.354153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvFi-0001b7-74; Thu, 30 Sep 2021 12:33:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199854.354153; Thu, 30 Sep 2021 12:33:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvFi-0001b0-45; Thu, 30 Sep 2021 12:33:30 +0000
Received: by outflank-mailman (input) for mailman id 199854;
 Thu, 30 Sep 2021 12:33:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvFg-0001ab-JK
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:33:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 371278c1-16b9-4cbf-a985-0a39ed8ed55a;
 Thu, 30 Sep 2021 12:33:27 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2057.outbound.protection.outlook.com [104.47.0.57]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-9-ctqt8nizO3K9nubEjvT3bA-1;
 Thu, 30 Sep 2021 14:33:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3933.eurprd04.prod.outlook.com (2603:10a6:803:24::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.19; Thu, 30 Sep
 2021 12:33:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:33:22 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P191CA0040.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::17) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 12:33:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 371278c1-16b9-4cbf-a985-0a39ed8ed55a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005206;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=dINmbKQuoSuwcjjwI59ZTgbgDNydobyH36DWyQ4IiJE=;
	b=WlAIKKVHRalHZB0DwbTf+PUIgvl/v0Lt8tHEYJDFPsTnArvD7m0/au2hFUpNib+kZpPLJh
	+p+ypZlOiQZqfDGaNZM97LurrLKY/pCAnVc/RldhaD7auRSdChA+0zwOO84r1tzw+vDhfv
	bRf/fhP6x9kXC0xgCGawuappabyK31w=
X-MC-Unique: ctqt8nizO3K9nubEjvT3bA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GzNZyUHBnVlMrwGfmd5Y4uqUB3rUrE0tAzJFk0dVmWutdzE+lekRPYgv3jDjdVA3+Wk9jgMax6x/cS35wdgAB+csU1XDYI95aANMyV+yJ5bGj0T2Oyw+3fiY8tmiv1LrGrfAEeTR8PGcHQq5zJRHHb0fqjjVWxgVaXXUXFbhHup0fveK9tOA68u51C1UDj8n/ilofh6E1IrC2mnUY/g0XbyD0s1RIclLH4z5ImwjbZmA5eyywt8Ac177KUFrs/7buZRQIaP8hOWdqVBT9YUbzdJ2COXvuLyMnzIfeHkLq206zs+Qw2xTjuCIskpheT4RhYG+CgfegbdYeKh9Z1uVsQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dINmbKQuoSuwcjjwI59ZTgbgDNydobyH36DWyQ4IiJE=;
 b=a9eKSJlRc1RmDIs8axrWEnaHFRrA/wXsVoVTxo2iyqr4mvwxOfkLLKgN+Bc0cYvsCDBfcHyvL4kgL5iC3cFLGagIKxyDkKcJmM/YDDyjjGxkAK+6l+YMcAvmY27inOiURPwEgSDPEdD7IVo0j1Rnjr/VWKpuLNHDjr0LwHOGl/wqYZzM2txSd5UXfNv1YbOIhPByyMhBDbT2i3m9W/Rf3BownjbLBsDp5eGTTIDZY2bdXWfQ+Ab/2YG3KZ6cWHG3jvAPoCyLLH0p5fswwuo7PEJMu1UeXgBheC9ywxtBu2jRzy3DXjt+bAudY5GaMFOBNz0kYHmx4NMIePka9b+ZbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/6] xen/x86: PV boot speedup
Message-ID: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Date: Thu, 30 Sep 2021 14:33:20 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P191CA0040.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:209:7f::17) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 145504e2-5acd-4064-9e41-08d9840e7de5
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3933:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB393332E315927CCE328E05E2B3AA9@VI1PR0402MB3933.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+gk4dW9Mi/6VrXtJN5I+qBA7rRec8OQONkT/feLYU6MPVrQPjeASSk5E+nPUPIk8KQchuwyvjIF0gH0/6bIUwln+dpVKXbsiT9/oBlsggKqtLB+vvu5MpWhB+EQf+X1IyE88j3/7KoDxDpFxX4iI8uDsx+m1duIExdllLwGnbFc6igv99NqX/O5stJIDscimoUkSeU3nN+nVMPnWJi25OGQAtvW+DDmigIUEexUHmpUtmAS09zM8+GS4LcEdh2rIUWygMblw9amIBkPATKRfpJuQEpcg3yuYb87ytvC88tPjzGoD1bKbPAw7onznskS65dl/DPNZ7O4YGj3ifH6WpuGhlSQHOtVe6mHG9RBckRMS76qBUZBsEubbvB4w7cOAcdJyWQHyyxwE5ucVpFqocUm/gtFjF70Hcy2tDilj8tqQzjazP/PM0AnBlJEnqlPLGnY7BWdd/Q/3Ff5VRd2ZHCSF0Q6OS3Cbq0QCCSF6/LLQclwSfQUGONSE0UQOu4r+w8rKo4yzFsw71SnCZ6Z0130/waBw+QHjj84KCf97cpkm+5akP4gnTqhcYyv4God4Hc6yzoYB6aAm6Oyv717Vjis8Fw0LW+eS6cUENaOz7Gh163KvMW2/ouwwqZP9WXMykhFVS3tbCCquk6lPTVJzlEsB1GM9S9nVsRvUuqB7hhxoyOiR9Ew63qHAZphQbHSHNIQdbh0UvqPRchGGNb0W6CNObMC628P7GIg4uDI3v7g=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(66556008)(66476007)(2616005)(31686004)(4744005)(36756003)(508600001)(26005)(956004)(2906002)(66946007)(86362001)(8676002)(6486002)(31696002)(16576012)(8936002)(38100700002)(316002)(110136005)(54906003)(83380400001)(5660300002)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c1hhbkNESDgzNUpoczZ2UHBQb3loeGdSYmc0RGs3WGlrSmJYMWVqMUJhWXZr?=
 =?utf-8?B?Y0dFaE1zWVRiaUg1YjN3L1F5b0QwUzhBVG1RbnBLN3BhZ0pwMG1FdGl5YWNz?=
 =?utf-8?B?RERHL2h1dlpJOHlyaEh0ZUQvNTcyOHMvR1lvU2dNVlZEWmJDd3VZSFJsczds?=
 =?utf-8?B?eWNRWlhTWHdnTFhnOVNpd0lrMU52RlY1aWNjdldmRGZWR2FXREs2ek5MeDRq?=
 =?utf-8?B?VDdiWWdxWTFSZWdrWXF2YTBTQnBzTEtOTFhBcjhOWGdtcGpvTjRVWWszR0Nn?=
 =?utf-8?B?dk1QRTc1YlR4bExEcTRFdXBFRyt4dk95ZG1kcG9HYUVNWXdsdzBkTHdrVTJG?=
 =?utf-8?B?YkVHcVU1Vjh3ZzRWSENPU2dveitwb3dRVFBKUDljc1A2SXBlMHBGZE83Q1VM?=
 =?utf-8?B?TkJ4bjZmRkxGR0pvTHdoZDkvcnM2alZsSkdpUlVJaGVaYXZ6bnRVNXlwMGlm?=
 =?utf-8?B?N2p2bEdIZzJqdWJWUXh5c1JqZVZpVWR0c2M5Ulp5VGRnUlhXMWJldW9RT3Jv?=
 =?utf-8?B?OURaVENVY3crak56TUtRdmgyWkt6TVM5azVoZ2NYV2d3akVOYk8yeG8xcUov?=
 =?utf-8?B?ZFhKWkRPUmRDNXlVVGpNM01QeUcvN3EvVUtPVU9lejljTXVWODVERC83UGcw?=
 =?utf-8?B?K2FTalhKUHEvYXZlQVVYbklmNW4zd0x4bmI4TXdKSCtJNzZaYXE0ZDByZ1lH?=
 =?utf-8?B?R0J2SSthYURtRERtcGNET3M1SWxQdGh1M0N6bmFTdEM1alBVYk1ubytsT1FV?=
 =?utf-8?B?UU1DSGpoTGhKU09Pa003dmdxRXdueG1PL3RzNEVpS1BZRUhBdk5KRXpRQkdv?=
 =?utf-8?B?dzFqTERtL05OVTBsaUV4WEFyREZDcGhybUUzT0JyTlI2cTJHWGViVFpSU1JD?=
 =?utf-8?B?ZXl5V2tjb05KeUsxQ2V4RDRiUC9kdDZuMHNXVUtCWnZuc2xUUnV3MDREWlhT?=
 =?utf-8?B?UTFKNW8reTV3R1dWVnNhYXBSSVpETHhiS2dxYi8zUzk5U21oN0h6WHdaQURG?=
 =?utf-8?B?MXZ5L0xISElPMEIvRUp4ZThJRU1UZkVtM2VqdGdkUTgvU0xWeFk1TmFWQUd6?=
 =?utf-8?B?MkVtbWdXdVBXS2lraDh3Rm9OZk5WeUYxUGo5b1VPalh3RkljaTFGYmNRVVhM?=
 =?utf-8?B?emcvMUtMdGs0S1VPd1o0Z0lCWk8yb1JXUU84ZlR1R0RnODNjdFJveFAzQ0VZ?=
 =?utf-8?B?TjM1YWdFZFVzRktpVGVtUnQ4Tm9tNkdwNTFvZ2dsM2xadWMvRzh2K05TSXln?=
 =?utf-8?B?MnBCNU40SkpCQWhHT29TaGhFSlBlVlBrRXBPdmxlZ1lQQ2pJNWx5d3dsb0Yv?=
 =?utf-8?B?cEptWUd5bXFCa25SUGNMUXVaRzJ1Y2tIb1VHWGpmNkhmNkNQamxIWGdGdWZW?=
 =?utf-8?B?Unc1NXR4MkNjZmpta1hKTlRzVWU3Mlo0d0p0VTR1NS94alFJajRXS3U2M2dy?=
 =?utf-8?B?aVVrTEI1aUM3YkJ0c0FoOUxsb0VpeGJaYWwzSi9qb1JTOVBYNWRxU2JlZUdn?=
 =?utf-8?B?K1N5cmU0UkFwR0NVaFl1TnJRVDlyUkltVDU5RTB3M2IzemlqR2NKZTFSODRq?=
 =?utf-8?B?ZFQxaC9Va2FGY2JSSGxKQ2JFTjJlZFcyWDhiZGhIM0ZhUUhEcDVlL3FlanJx?=
 =?utf-8?B?Z0JwdVpKOEpvT2hPM2pWWFRvdkJScWsyZnl5cUVVZUsyNjdJWnU1N1BGUkYx?=
 =?utf-8?B?RkZZS0cyckNXRFRoanZSaU84OUNnUG1USXVBQm1qNGEzQmc2MElidkdZTTFq?=
 =?utf-8?Q?/PEu0jqNBcW0wPB5oOkUgG7r9lRBGWQexFBtp7w?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 145504e2-5acd-4064-9e41-08d9840e7de5
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:33:22.7384
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YDNSFHOsNsp3WrDw566oCfQV4enP/5r9f3FhoSkz9jrndrNJsrXGblX/GtE/ZqLWM+onV+sr75/sAAYkbMQRJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3933

The observed (by the human eye) performance difference of early boot
between native and PV-on-Xen was just too large to not look into. As
it turns out, gaining performance back wasn't all that difficult.

While the series (re)introduces a small number of PTWR emulations on
the boot path (from phys_pte_init()), there has been a much larger
number of them post-boot. Hence I think if this was of concern, the
post-boot instances would want eliminating first.

Some of the later changes aren'r directly related to the main goal of
the series; these address aspects noticed while doing the investigation.

1: streamline set_pte_mfn()
2: restore (fix) xen_set_pte_init() behavior
3: adjust xen_set_fixmap()
4: adjust handling of the L3 user vsyscall special page table
5: there's no highmem anymore in PV mode
6: restrict PV Dom0 identity mapping

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:35:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:35:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199860.354165 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvHE-0002Bd-KI; Thu, 30 Sep 2021 12:35:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199860.354165; Thu, 30 Sep 2021 12:35:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvHE-0002BW-FV; Thu, 30 Sep 2021 12:35:04 +0000
Received: by outflank-mailman (input) for mailman id 199860;
 Thu, 30 Sep 2021 12:35:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvHD-0002BO-IT
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:35:03 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 2449351a-2bed-44de-bdc3-aea2482869e4;
 Thu, 30 Sep 2021 12:35:02 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-BQHqtiwQPkGOwPYhZzIz8Q-1; Thu, 30 Sep 2021 14:35:00 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:34:59 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:34:59 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0226.eurprd06.prod.outlook.com (2603:10a6:20b:45e::25) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 12:34:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2449351a-2bed-44de-bdc3-aea2482869e4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005301;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=PbKi1v7uP9b6x4lBqFwjoXN28PwSTCvFjDe33pv1YU8=;
	b=cWg/DoKMo8jMX2NCzV96eNxXPTBYNuYyo75/jRK5qa0a3nlGrin6ohDQixzyZ4pkRCSXzg
	aPyBE1qjVxWnocWu0DAUkZHpQla64MxFBp6tJFNFy2w6vXq8i9+X7CuoIRSAQ9DLPVKU0z
	kL91O/lFYjv80aW7nVPiB6/5PHLBgQY=
X-MC-Unique: BQHqtiwQPkGOwPYhZzIz8Q-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V34JzRotiHrrRxaj62lpYZ6LyQ3MVAns8brGxGNFisP46zu+4Bm2gQlHGLyEaNO/mYdsKcps2/uKh53FHs1miNRKwNrotDJi0GKE25MeMCdi6EdDq+7bEXzwWmfCC8Vb1kNB12Y/ENAhsgdOo9/kBGb8A3SQi6CyQmtFGR2Cj7HDwFFsBBeBXkYWlRb1DWj8KV4OEqb/WmB7cJ0Kbr7cLj9rN+GXK8hzgdFgAKWWZTooUZXfHKjsfuxcaQeeSPGwcuMI2ymxJ6ThJi+/Gzo6yBGAPiXZbvdo1QyV3qp4O/MdGt2kicoZhgFD0IKSxP0aioVPgijZXbGVHGJjKq6hDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=PbKi1v7uP9b6x4lBqFwjoXN28PwSTCvFjDe33pv1YU8=;
 b=LQaWmTodXJtHB8gox1R4O2husN39gb5WzDaABpRF4C7J/bmNGmaUtNjN3+3uYIZlrm0TEcfDGg5efuFYPH0DjLu30OXUp0HuoaKrvvEqJHui0ziurBKC5SFDY60KnXVXaBnCrRqLsSANtshNj84G6U2C1yb3EKEqQScH+xdTgNXFKrhbKHmFPVYHpJyM6BxRNuqtatM1BaxVEjPKKBCCzgnCI9D44vVMtbiSIMZzOKAdX1fhrg4Dmb2Caf7A7m/sVMaLxIofNos202jjOEFm4Vipd2BP67XaiIixaqoCxh77L9jN3GVCsU8COWchafe185LNCAzRBGzWMi2JgsCKtg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 1/6] xen/x86: streamline set_pte_mfn()
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <b39c08e8-4a53-8bca-e6e7-3684a6cab8d0@suse.com>
Date: Thu, 30 Sep 2021 14:34:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0226.eurprd06.prod.outlook.com
 (2603:10a6:20b:45e::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 6ceb9de9-bca4-4c85-34ae-08d9840eb73c
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB63839DD3C04C7A3272F8371BB3AA9@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TC1mRBq40fgQodWv/jlWln35z0/4GeyyPOSAzy0hKfoTPda6IsbItMlXTXqjk/zB/Feoal2zVIeqmTdFCH/Jyff8olq8/F49T61Lj67kzXz8lTrJCffUHgnAIf0Zq8Vyy/kM8HBIFgGGySYmxbxVw+Q0IxOvGFs1GlMwkoqbeUVMIgqcXQkIwUSWWTmXkmrdBb28JuYF0nSZVt1zVbseyf753N/gdjc0kjj1dcoAe1vwTZS1Vagiyv52uN+9p/yUbuX5+AiznQ6ow7yxu52Y0uP6abHkiLK5SID5DEoYcFb2+R9X7icu86PQ6KhSRPIiG6FwNW4uGj1W61FyL5ptn/hcYCW/hC92OwxhaX0FBMyEfhvlO0c1avZA6HjTqr9BGjzaVY8ICZCB6+nWokNhH50wVIi2bY9nn8NkIDCFZR3easZXxAiDg4rWNaZG0LcHKvam5b55uuW9C8idAYt8hC7dW8mn0qCWfSiwuAt39d7XyTXZuxWAJrv//idBCe2b0mEMp4elJPtsXfWRM+6ZThKaG+EwpaZcgbYuJHBwl8rwpxeMh2E0qRtoC0vksGK4mL0msprkka+OgNt2cC6l9Ozp8XW8lJcQdtJghpA9vxrteUXg6YWbzOTgVAum6yuPQsTN/zd+PiNpH8wvUUQ247WmeGdhEiLTmNqJzLcKm+hdDH03qSG6BkR8uOUWbHS1pyE7ntpSjw3DAS6TMAvTEloO7pvc/llKV5r+nE7BO2M=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(2616005)(186003)(956004)(508600001)(31686004)(66946007)(83380400001)(8936002)(86362001)(2906002)(31696002)(16576012)(316002)(38100700002)(110136005)(8676002)(36756003)(6486002)(66556008)(66476007)(54906003)(26005)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UFgzaXhGQkkrZHhQa2FhN1pXc0p6Yk8yRVBhNU1MaDRyVUJUUEsyWnZIdmxF?=
 =?utf-8?B?UkxYVnNwKythcGpMRld5b205U2hjd1JHc3JPZ25rdHhMZlJyT1dEOVl6ZzBr?=
 =?utf-8?B?MGVaNFprbDMxdFhBZmFkdzdrTjg1OXVjNHBaRjhUVklCTWRaejRDUll0UUlY?=
 =?utf-8?B?a3pqa2xraDE2TFBoUnhWRUpSdnVXcUNta1dKTzRkRllqN0t5TnZGanQ3Mlk5?=
 =?utf-8?B?Z1liTmp3NDhaRnRqNGlqZ0JEOElXRHVyelhISi9VLzBqY2VPUG9KRFJPK2ta?=
 =?utf-8?B?YU9KNnF1RFVKSXFGYlpSOTh3cVd5c3V2VWFoTFRGSHM0WUdiNWN5Ykw1Y21r?=
 =?utf-8?B?b3VKTTZtQUFMUzV4MmxuYlhZYjMxL3BqUDFpQ1ZmczB5M21wQ3Z4S1FyUzRR?=
 =?utf-8?B?K1BFV3hjL2Q3TVV5WUg5S1luQW9DQ2RDa3pOZnd6eWtzeEpaOXU4OWxXKzM4?=
 =?utf-8?B?U09peHgzM3FIZTVjejFsaFV3a0lOOFM2NUowcFdjZHpNMmhHaDlYaWNSWGpx?=
 =?utf-8?B?a1owemxEdnQ3Q01ML2VHRXMzUDBKRUFEUEpLY0cxWmdwKzJUMVg4cnRrUU9C?=
 =?utf-8?B?M24ycjQ4LytZQkR5MWV2b3lENjY2VGVSd0dzcnN5clZQYVJjYzZxVXRYMlg2?=
 =?utf-8?B?T1kwRmVreDB6TkNqRGhEMUtNQ1VaYTNlNE5TcGtOVGVRMmx3c2RiYXM0K25N?=
 =?utf-8?B?aEVzeTRRbWZVbFBLZHM0RVZSWlgxZTAyaUNRdEtqS0s4OWZ0Tm1HaWpRWUxl?=
 =?utf-8?B?ZHhoWFFzYjV3aUcxNklEa3VLaXNGUkhiM3c3c3R4ZmVaaWZwMWw0WkZVLy9W?=
 =?utf-8?B?M1haUmZJYnZMTHJleVhhd1ZmNFNYYlNLcXdCVVV0QUFBZXpjbnpZcWRvMHhL?=
 =?utf-8?B?WitxVlZrM2MrZjlWTXl2MXF6c21wZnlnVE0yMmUyd3FJOWhKc2JqSjNiNXU0?=
 =?utf-8?B?ZUs0STN5YWtlamxuSUhGcit4T0U5QjN1TnhGU1dmWTRoeTV3cDZ4QlpveGtm?=
 =?utf-8?B?MEdvNlI4MnprTlcxR0JONkZMQ2NhUEhTaElIaExoVUpZVTlEL0Jyb0JOY2lH?=
 =?utf-8?B?RUJPMXl3VlpqRmJzSkpBWVh0aklNU29DUkR6UTlMaTZTZ0ltNFlVTmlyWi9y?=
 =?utf-8?B?SXA0bldKdHIwTm8rYUlZUFo5bjR2RHVBU1FVbU1KY0RNYjlyRmQwSEdYWk93?=
 =?utf-8?B?ZnF3ekVkWk1UenU4Yk1heFQzUUpZbDZQbVBLaVd6ZHViNStwWkN1K1BOWkFI?=
 =?utf-8?B?aFJ6Ri9FV0dxZEh0ejNCRFRTWklnUytQS2tpc0phYjVnWEV2SGpMMmo1TGI5?=
 =?utf-8?B?NWVuamd4ZmJLNEl3cXRmdzRrMTFlaTVvQVUxczlqWE5CTlNGUDgvNzhXRTh5?=
 =?utf-8?B?UzZ3eEhwelQ3Q3RzOUdMeno0Z00vV2c4YzN5KzlDOXlBVnFrSkVtd2hPUi9a?=
 =?utf-8?B?QlZlczBodkVocVN6NldXWE01MkhwdUZFTC81SUY0RjkrOURKUzlMNWRKTUVB?=
 =?utf-8?B?ZVRsQ0txRHgrSzkwMjBoRHlvVnVDQis5NlpSTEtodW5YTXRPeGNRMWM4Nk14?=
 =?utf-8?B?QWM3UEpYWWZ3OVFoZXRUZWxyZmVXNDE5ZGlDQTJiY1dEVm91WFFYb0V1UVFu?=
 =?utf-8?B?N29nUm44eFYyZE1PSXNYRkt2QldrWlBac2ZNSllkemwzTTdhN3Y4NW45NUZp?=
 =?utf-8?B?QXZ4N2NUNDVlVERFcy9QTWd6TkFJWmxiKzF6TmhuZmRxcVZhY1U1emFMcUhy?=
 =?utf-8?Q?KYJhkcLgwFGmV7C9CzW/n+9xxUGCX9gQ6alpI6m?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ceb9de9-bca4-4c85-34ae-08d9840eb73c
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:34:58.9369
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MTCCM6GdL4RD5oKcxgFMrJJWni9iY3k6Cc66Fcm16BqN2uOyT8Q+giH0k25AdM9GSW7VfEOWDYcRVjjYFIxMLw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

In preparation for restoring xen_set_pte_init()'s original behavior of
avoiding hypercalls, make set_pte_mfn() no longer use the standard
set_pte() code path. That one is more complicated than the alternative
of simply using an available hypercall directly. This way we can avoid
introducing a fair number (2k on my test system) of cases where the
hypervisor would trap-and-emulate page table updates.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -241,9 +241,11 @@ static void xen_set_pmd(pmd_t *ptr, pmd_
  * Associate a virtual page frame with a given physical page frame
  * and protection flags for that frame.
  */
-void set_pte_mfn(unsigned long vaddr, unsigned long mfn, pgprot_t flags)
+void __init set_pte_mfn(unsigned long vaddr, unsigned long mfn, pgprot_t flags)
 {
-	set_pte_vaddr(vaddr, mfn_pte(mfn, flags));
+	if (HYPERVISOR_update_va_mapping(vaddr, mfn_pte(mfn, flags),
+					 UVMF_INVLPG))
+		BUG();
 }
 
 static bool xen_batched_set_pte(pte_t *ptep, pte_t pteval)



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:35:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:35:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199867.354176 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvHe-0002mq-0A; Thu, 30 Sep 2021 12:35:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199867.354176; Thu, 30 Sep 2021 12:35:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvHd-0002mj-TK; Thu, 30 Sep 2021 12:35:29 +0000
Received: by outflank-mailman (input) for mailman id 199867;
 Thu, 30 Sep 2021 12:35:29 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvHc-0002kX-VO
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:35:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7087557a-fe7b-48c5-8378-b1e43e0c7094;
 Thu, 30 Sep 2021 12:35:27 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2113.outbound.protection.outlook.com [104.47.17.113])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-20-aoc0CG_oOqS0JfWrqru6WQ-1; Thu, 30 Sep 2021 14:35:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:35:24 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:35:24 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0219.eurprd06.prod.outlook.com (2603:10a6:20b:45e::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 12:35:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7087557a-fe7b-48c5-8378-b1e43e0c7094
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005327;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=eWRvbp7ta3owYpcdKj7PO7T0r2HXZ7zs9WVfAD35Uvw=;
	b=UXXlauToOjHhrRu8X3wJHT8P7ehbax9MC1OeZ3RsHo86vGKJB0it7YUxco6xoxXbkgaw3S
	ztAjrHLkaA1JeyHZxaaQ5wHEoNMwZX1QGNvQN1IEDv9/H0rHnt0ku5QsTmpa9foc2KLImI
	MA1BJaalnudb+f5VWlA01VsvQSia5Sg=
X-MC-Unique: aoc0CG_oOqS0JfWrqru6WQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gHkFibb5LYshIVpl0Jp9XnQZEGUsmW7OWVWVt0BujZRFHmCG+Bc50cgQqtzeRjcbiYLLLyPp0v7NQqVfrtBiNOqdMjnNKS6XkdwG0hx0DPY9PKuUk97H+nWKvlabCeMYG0nO6Cv3dX40QvfHKH3weYlGrh0gofHGLkrCYCtRRdQpVpZyWSlxBSEjB/JHVZyboqadYFIGEYsDJIR/N9YdBetu/2osTnEew84x+VfLMq3lcMVRoDoEK+7uwwI/2+I3z7XD5GHGfSHJTOznViGOtsnJKSWNmsn7GF/fhTZiPrN19YwZsuiqmddGoZr/i4zUXs4sWaQ8xhoRPc6WEvFh8w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=eWRvbp7ta3owYpcdKj7PO7T0r2HXZ7zs9WVfAD35Uvw=;
 b=YsImSKohjx7Tx5XUvZCvBF0KUB/FrYqV17rGkQVV+9k75XwtAkVl8oe8qcsDUAD0lvstAt1+cYkDhiTXIoYMHE0DnpWyPbXLceIXXlEi0QICl3TcgypmmsBapKE59xmxL/anh/vyg+M0BjWzHKOrgvll6/aMGFQLIyEYnl7dw5tt0y/rd8xPZQI/GwpYH6WK2E7BLRFMCOnsMvRps7GY0/EMEmflRe7xvtpwfqbUb0Vc9MeFe+lAtlwTkSwR/4uxHmbgoZvgRIZjhb3tYtq9STA4tIjUjf9s+SJCOsrclV63PQ48ZXwH/Myqx2KVt1iZhJ0XmcF0nv6BARaYO4wCbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 2/6] xen/x86: restore (fix) xen_set_pte_init() behavior
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <57ce1289-0297-e96e-79e1-cedafb5d9bf6@suse.com>
Date: Thu, 30 Sep 2021 14:35:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0219.eurprd06.prod.outlook.com
 (2603:10a6:20b:45e::10) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 74700795-3594-4d3b-fd63-08d9840ec687
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB638392EFAC473C948BD49794B3AA9@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3173;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	w3I5bU96eYptQzelHHw9QkIo2SbqWLNurMYmpBlW2DK5UVBHPgPgtxQg9tSYuQ1lObrEXgzpRgOfhMx4S2WELbT6wKE1zE0/fE66OBbI4c2JDgnoAcsWUMN3f6Qlo4hAJ382EcUf2prMHo5u6YCXoNDZyv9NBzCZyZewBA1AdH29qlEZaez7fVEfR7Hu+p9HEs1+C9AWlDjsfCIJz9KVgIII8PNMR6pLwAcL7Ie5bUqxWgYza07SitTUTdY/P1Ca2ATtd/A6dyk7aCQ1LHPGL9SS56Iv4bL0ZiY2K+5TW59ElCM4kaHi+9Rf+JALzQebeV5WYeid90F9gDrQVdjMihv239FLVaMqfYu6/2G4S8oDfpoIZZVDLCiD24m7SR6/in71fd22+YeL7VfC30TPdEiuQGCkeUZNMmHHcirbqkqsWSI12lfu7wI1ycT0RpeXdwnRiMLpNmspBYyfzAkLOsMTdBLYjVqHKV8Uq6NdiuTeqHzTAcLP8OC0Fxy5JU1uz58MH6FBWGQfqln/WjRoidjfFoFigOVwXBjbzqPFKrgXm0zChU4PaZZ7qP3BwbKSoIj6/9UJAhAwj0p30VywGdGiiUNUzd8tn1+EO3lehmpuFrKu3PpmH+shMcfWeE8a1yz+7N7F/iF8YXYmIEqtS+wgayvOWBFLZbFq+LZJW2OSEZECzQ2FjhlKRORfS4JHGLxI/x95rUKr2pK36L+m69sRDzxNdXDqPgBH7L+dstoBoQzCJZEYIQLtmU+xnYIh
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(2616005)(186003)(956004)(508600001)(31686004)(66946007)(83380400001)(8936002)(86362001)(2906002)(31696002)(16576012)(316002)(38100700002)(110136005)(8676002)(36756003)(6486002)(66556008)(66476007)(54906003)(26005)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?eFRnVmthazE2TnZIdTNjRWltRXNxVFRDbDdINzFXWlMvUFV6QzFIRFBEN2sr?=
 =?utf-8?B?dFFMOThPbDZqdkptU1FGWkkvUTc4RERYTThkVDlVSUpGWFBsR1RnNXdOc3lm?=
 =?utf-8?B?Z0l6NTJna2FkSEhZQUVYdEhuZll6UU5sUEVQQ2ZwWHF0YVpScDBmUVNyRnBQ?=
 =?utf-8?B?aytWWnFNa01GU0RFR0M1S3lrd0dqQVc1R1RhK3l1MDRIY0kzTHZVdXc2VEN5?=
 =?utf-8?B?NjhqOEw0YUdlajluUFdZRjlvYTRFRUxoL1hicGw3WnNqbnlJcjJrZHdET2pt?=
 =?utf-8?B?ZUJUVS9xNHJHTVJHbEcrWDQvdG9EOUJoZjBVNTR3dGgxSnRKYXliMDJ5Zkc2?=
 =?utf-8?B?U3g0ZFl6NkFsUldwT2FocHN2MDN6MmFiVFl1djBTSm9Gck5SM2VobzR1TUk1?=
 =?utf-8?B?WGp5UEFnc2gvSzNnSGtSYlJRWGlDNFE1UXlFU1UxaGVBdFpvM0pCMVlBL1FD?=
 =?utf-8?B?TVZ2RU1Sdmo4ZS9aSzRSc0FJc3RCYU5hcUVnMGxGaFVtS0xBSTlZL1NaWm1Y?=
 =?utf-8?B?NW91WUZmckhMbVJHYy8vb1FaemkxQTQzWDVHTHh3R1JOVkhQZ0xRYjljdjg3?=
 =?utf-8?B?ZnRIZDVaRDQvazRFT2Y2RHpzUm40c29DanVYdUVNNFdRTkxUWnFEeDY0Zmx1?=
 =?utf-8?B?cGZObjVDZFlYNEc5em1vMzZCN0Z0SUg1aG8ycDFGMVpFbVVOWnFrREthUmww?=
 =?utf-8?B?N1QwUlprc3BHcVlEbFFmZzNqY2lTWlhYSGtOZjFMNHhzWVhVZU5IT3VlUUdW?=
 =?utf-8?B?UlBGd29RdFA4Nnk0UDREallOZ1N4bkFLRTRHN0Ezb1dQZTRxRTJRQUdpL05Y?=
 =?utf-8?B?RmpFYnoyaDlYTkFDOUsrU2JMT3JXMndoaUxjT1hiOUdsTU1IS2pJNU5uQnZX?=
 =?utf-8?B?UDJhbkJwK0QyN3ZZeEx0cWs4S2l5NTNWQzRKUWNlK05iaG16c0swdkxLQXJt?=
 =?utf-8?B?V0lWNTI4b2tpMm5xdUFzWlhCOXY4YnhtelkveUd5bU5HOVYybnp3OGJvaDJ0?=
 =?utf-8?B?WFNRWVFhVDg5Zmo0Q1d0eElHMGtTSTkrM25vZlRUT3lpNkJNZGZJTE41WnJC?=
 =?utf-8?B?bjE5MlVVTXVlMm8va1BNSHlRYk5OOFdkTE9ZcFRvNGIxdDVmNm9ua25keHB1?=
 =?utf-8?B?bCtIbXpKaFAvTEl5cHppSEFSdTVEeEpQdzRLN3JjT2paREw2QTU3bFJMcW5l?=
 =?utf-8?B?SWxNUG1aVThySDVlQXVobkIrZFNNcTJ4M2xFWVhFVm52NXlpZ3MxYnZkMm1J?=
 =?utf-8?B?M0xJTUN6V1NWdmFPQmVGVmhYZHNTQUJDTWlZYjJpRi9kRy8xRUVKZ3R4MndR?=
 =?utf-8?B?cCtJSkZjekRxUU04U2RpZUEvb2VMZkpWQ2Q2VlRtaTdBblI1Y3VUTlJjajhu?=
 =?utf-8?B?WnF2NFkvTko4dkVqY0JRdzdVbFRleGx1MkFPWU9YbnRZYU03b3FYWjZ0Slg3?=
 =?utf-8?B?WDFBZHN2Ync5OWNGQjJPcCtjTm5HR0FyRWE1aVdlRkJ2aE5WZ2lkbWxWYUNu?=
 =?utf-8?B?OXlESCtpb1hqblhXZlptaEk5QitYU3phY2g3cGd6a0lwZVYxMEhwMUlGM1pY?=
 =?utf-8?B?aUtzWFJuMmJiMVNMeCtHemh0dHNjcVN1Q2poeW9SQWIwWUpJY1BrSU5TaFFz?=
 =?utf-8?B?RSt0S3ZUd3lVa2JvMHo1NHlTV1lRdE1DTEgxYWY3MGREZnRxS24zNXFEUzBy?=
 =?utf-8?B?MTA4Sk1raGszWXljSVN4RU83MzlqZWlST1A0dkJFQUczU2QrNnhxcVNISEg2?=
 =?utf-8?Q?dlV7EnSH3Vo+EakO3a/jG/FhhROjRzqBJAxvm2V?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 74700795-3594-4d3b-fd63-08d9840ec687
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:35:24.5804
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Z+QpJ6hdD1dvtUjfh20b2SeBFtCK/5mrepLbPm7M8obAJPEYT8rkLVValbSZp1jWDTxr/UUqZCjnonlfRIpSDA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

Commit f7c90c2aa400 ("x86/xen: don't write ptes directly in 32-bit PV
guests") needlessly (and heavily) penalized 64-bit guests here: The
majority of the early page table updates is to writable pages (which get
converted to r/o only after all the writes are done), in particular
those involved in building the direct map (which consists of all 4k
mappings in PV). On my test system this accounts for almost 16 million
hypercalls when each could simply have been a plain memory write.

Switch back to using native_set_pte(), except for updates of early
ioremap tables (where a suitable accessor exists to recognize them).
With 32-bit PV support gone, this doesn't need to be further
conditionalized (albeit backports thereof may need adjustment).

To avoid a fair number (almost 256k on my test system) of trap-and-
emulate cases appearing as a result, switch the hook in
xen_pagetable_init().

Finally commit d6b186c1e2d8 ("x86/xen: avoid m2p lookup when setting
early page table entries") inserted a function ahead of
xen_set_pte_init(), separating it from its comment (which may have been
part of the reason why the performance regression wasn't anticipated /
recognized while codeing / reviewing the change mentioned further up).
Move the function up and adjust that comment to describe the new
behavior.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1194,6 +1194,13 @@ static void __init xen_pagetable_p2m_set
 
 static void __init xen_pagetable_init(void)
 {
+	/*
+	 * The majority of further PTE writes is to pagetables already
+	 * announced as such to Xen. Hence it is more efficient to use
+	 * hypercalls for these updates.
+	 */
+	pv_ops.mmu.set_pte = __xen_set_pte;
+
 	paging_init();
 	xen_post_allocator_init();
 
@@ -1422,10 +1429,18 @@ static void xen_pgd_free(struct mm_struc
  *
  * Many of these PTE updates are done on unpinned and writable pages
  * and doing a hypercall for these is unnecessary and expensive.  At
- * this point it is not possible to tell if a page is pinned or not,
- * so always write the PTE directly and rely on Xen trapping and
+ * this point it is rarely possible to tell if a page is pinned, so
+ * mostly write the PTE directly and rely on Xen trapping and
  * emulating any updates as necessary.
  */
+static void __init xen_set_pte_init(pte_t *ptep, pte_t pte)
+{
+	if (unlikely(is_early_ioremap_ptep(ptep)))
+		__xen_set_pte(ptep, pte);
+	else
+		native_set_pte(ptep, pte);
+}
+
 __visible pte_t xen_make_pte_init(pteval_t pte)
 {
 	unsigned long pfn;
@@ -1447,11 +1462,6 @@ __visible pte_t xen_make_pte_init(pteval
 }
 PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init);
 
-static void __init xen_set_pte_init(pte_t *ptep, pte_t pte)
-{
-	__xen_set_pte(ptep, pte);
-}
-
 /* Early in boot, while setting up the initial pagetable, assume
    everything is pinned. */
 static void __init xen_alloc_pte_init(struct mm_struct *mm, unsigned long pfn)



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:36:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:36:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199876.354186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvI8-0003OB-8w; Thu, 30 Sep 2021 12:36:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199876.354186; Thu, 30 Sep 2021 12:36:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvI8-0003O4-5w; Thu, 30 Sep 2021 12:36:00 +0000
Received: by outflank-mailman (input) for mailman id 199876;
 Thu, 30 Sep 2021 12:35:59 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvI7-0003Nq-3v
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:35:59 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 89c7f0a5-7e08-470c-8c32-c4657eb79df8;
 Thu, 30 Sep 2021 12:35:58 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2055.outbound.protection.outlook.com [104.47.6.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-23-4iLhtgzoOE-IdxjPXrGF9g-2; Thu, 30 Sep 2021 14:35:56 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:35:54 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:35:54 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR01CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:20b:e0::35) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend
 Transport; Thu, 30 Sep 2021 12:35:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89c7f0a5-7e08-470c-8c32-c4657eb79df8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005357;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=csvwBDBtFGc48jg4jEJIdJUsqEpEZpnnkIuWC7zw3YQ=;
	b=bqvNY1Rp3+97atd0vJfMyGRPQbRJVrhbbEJspkM+hBJK/l9rNiqm327UbdbUsu26poYojI
	6eE0EeYH/9K1ilvbI0v2ZnLSS/iTudsMNpVMv8Rp8yXGfLtepBukGQW8+mgYRu5tMNTOC/
	/bvsBvfjtfgdlUfeEjMMtSLDdQSdIJo=
X-MC-Unique: 4iLhtgzoOE-IdxjPXrGF9g-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FEdWrKHzXgjyp7ZEHrbXCFH6f7gIoRdJxTJRaM5qejyIyVP5t8YLM941Ss03r005KvIpTh+w66LzNQo8+4TcfIxapaGDiFInA7duYaZxTud6TPt7AqSsaq8Lx/oBw2SY7/MSWiL3oqcEZHxY+MfIr1S3u8ziwIK4FxZwW7f/YhW4TicFf3V8h4RTZc8NnV89jE2+hKmcBsAC8d+pMXBi1ftcmD8889QrL77QN2WuW2iuo5fpZeT47XZ49P/XJKNjk24ptdlvh1LHsLyQ+FIcqls0cj0zvGazmq6HzY+gfv/ZyF7HQxcOMBB6yG3Zhwm9HrxotvovuDubrOz2VW0MXw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=csvwBDBtFGc48jg4jEJIdJUsqEpEZpnnkIuWC7zw3YQ=;
 b=ZcvCV2pd5y9a99dHbgErSusWKYtrYYe1JVKdU+Fl6+9kfK/90vRoQVjEmE6q1n2811b5sAPf69SBxeBlVuYse7isclAJs956GlPonBVpu54oWs6/qOrGvzTspb3RTCrjLZUv29l2h3nNwrbGNmTQOsSRfxJtJ0kqmjKITl89pXZYdUhlG7TOB+wYPP4H9Xls4J8neNjCMOXvFPoxMmaxejYfnGmcadQxbUVmLXk9bWpYZ9CQig7KuNXy1POLCoqXJCWVuqZRm3XfZ9FAaKaki0fxxfrXcLzvJd40xyQPblcoonPQsQaNbg/468i4jNb7P3RrLy/nRP7E/sXv6GAdmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 3/6] xen/x86: adjust xen_set_fixmap()
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <11fcaea2-ec17-3edd-ecdf-4cdd2d472bd0@suse.com>
Date: Thu, 30 Sep 2021 14:35:52 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR01CA0058.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::35) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 733cb1a5-c54a-41c4-369a-08d9840ed853
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB6383C21A54C6482E4DE9AB88B3AA9@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:5516;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	myEIyzqFlWR0QrxdEpAZVC+kMgoa8VyQRPVsJvIIB/9mrT6ZOoOJ53sMJyck12R5nMTjIuRneA7fU3mZOtVPxF/IpUAJzxRcF4QrYKkWHyaCjM+GdrxkpwYjGkemq5LTUIFLuO9mq0vXkqk0Z1NgbPukI7GjnCaGoo4EZ7lTYPidPJzS4brADk8XIix+EYobpcg4xqbWE8DjJumuPHc2CNSUjThGyr95nBBwU1BwA7GkIOLycVw3De2qFb4AKl71yLsdAm+XFe89/K7vwFR/0zM1jKmWgOdbfxToPxb2a3Zk1zkqD9so+/Jp/cXyQYiIUXeSCywNWVfjWiznDpjAZNVQ7lHxNy4VBRQCwTnfnxUC9CZHtoqHf286Tb3SG7ZQEXQ8wVETXsrjNlOEkXeD0XN4RVGDyDhUzNijQinc09b1tkut+wv7i1yJcPysW9PX4K+rCy23Tjh+HJL8lizfMDAhtndXUi+Y0deQ5niOT87aK6fGOdnZKl3YR/L/t+NVdl7VrzJFlJt1+kHRD2EVW6XfxDB7F36dw0spF24JV/wHhGkJrlNJGtd9/CyoR5piWigIndnre1js/4hdlvGRBW4fKjSOpElHtaoEdOr71/3MgOaHy9LA7sqh4N/XCVOuqigSYFG4wGHK4yOzO5Csb6De8VJGJqf0Slog9wcfHLPertff7YBdUA02Sv+udubbmCI+UVR8TVPre9slX5rNfg==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(2616005)(186003)(956004)(508600001)(31686004)(66946007)(83380400001)(8936002)(86362001)(2906002)(31696002)(16576012)(316002)(38100700002)(110136005)(8676002)(36756003)(6486002)(66556008)(66476007)(54906003)(26005)(5660300002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UFB3UlR3YkVQSU5ic1M5TlFsT3BjMURKYjNCZEtTdjFDaDhMbjk1WFB4ZFIx?=
 =?utf-8?B?Q1FVNFNDVVRLQVdzNCtJRmduckJMcS9PSm9tUHBqNy9JMm5RZCsxQUFZM0ht?=
 =?utf-8?B?UkpHcUtUNGhtbi9uZHFteVpsTUovRHRqczZ2UzNuQ1pENlRwVms4TWZCaGtq?=
 =?utf-8?B?emxWOHQwZ1k0cHNmaHlqZ3F3cmRkdW5pUGs5MFd2NExlL01mbVdHNllxRmlE?=
 =?utf-8?B?ZjZsTzhtbUZyTGMzdzdOaFMyZ1ppWHV0N0c5d3ZXOWlSOGNWVjJGNjd0VWI1?=
 =?utf-8?B?OGU4b2hQOXhIbTNTZ211R3U5bTJ3OGo1NjhJdHJjY0QvQ1FaUHJlYlZkTkxW?=
 =?utf-8?B?cUVFMWhOVTl1MHVLaDQzaG51SGV0ck12akxSZDBtRGZrbHZER0RIWkFpdkhl?=
 =?utf-8?B?SEg1bDNzcnFRSU9uR3BKUzYyNy9YMTB1TE1vUmNWR2piTHJOblN3WnhZdEhI?=
 =?utf-8?B?b2c5eWkvRENQNEkxWllaanFUdkxjaktvcVgxY0oxSXdWVFF5M3BXa1V1eGhI?=
 =?utf-8?B?NUJIRXZtQkdMN1dXYnRzMXNmeG9ZQW5ua3ZsMThHd2tVMmlBb1YwOUNDTFNr?=
 =?utf-8?B?Tk1JQi93alF6VkIxL3RtTmtncWdwWVkvV1R1VWdmeGY1YzkzRDVheDE0c3M0?=
 =?utf-8?B?YnFaL0NBU2pNL0xwSTI2dXpFRktnNlo4L1AwNkh3c3lpVjFwUmFLVTB0MS81?=
 =?utf-8?B?ZG4ySTY3V3Y5djF3a0VkZzFUTWcvR2h4Ui85ajhWU2lPY1NLRlNOYVJ3K0wx?=
 =?utf-8?B?V21lSlptZk9mNmlLMTNTbXE3aHI2eVMzVkpyUWt2emxybXByNWdVTDRXSmpv?=
 =?utf-8?B?UlB6b1FKais1WkJNTlBKeml1RW1heW8xeWtxSWxrZHh6S1RGaHEyRTQ3Wm1F?=
 =?utf-8?B?dkxxc0dzV0xqZ0ZGTnBCWXlHWTRTWkJKUzhwNmgwcVAydDZWNldZL2FSclhG?=
 =?utf-8?B?VHhBRnZvSFdmQXk0MWNuMHozNEtQVUtFYWg5Vk9JSzBGSTNOWlA4TmpXMmRC?=
 =?utf-8?B?M1V5eStWS2VtL1Awa1QxQi80UTVCWE8yQlBPR2tCVEg1cG84cnl2WFhzUyt6?=
 =?utf-8?B?ODRaZG1ZdGd1aWxQSjIrUnFhYXViN29QRU5mMGZCWDM0UE9HZUwxeXRBemM1?=
 =?utf-8?B?ZnRFZ2JmREgyVlBRd0dwb2lDVWpIYjhZdVFaL1JDczJUZEsxUlNZL043WGtT?=
 =?utf-8?B?SmMrNS9YRDBSN1lYR3VvbWw1OUVGdXpwcXZuMERpRjRrNVBUN3ZKSlhZbzk4?=
 =?utf-8?B?RTdjeUR5cDhadWdsV3gySzR1VnlyTlR4RGpHVTlsZXNCQjFocFlrTStyVkNx?=
 =?utf-8?B?WEplR1FBYTVFc2hIUlhZS1hkU0I5ZGJLN09jMzFTbE9WQmE4VVRqQml2c1l1?=
 =?utf-8?B?Vmxid0UyVDAvN1NrbElQT2N1Nlk3dXlQdVdjMFFiS0dUeGx4LzY3a1l5U1VL?=
 =?utf-8?B?UkYyNWNxNjNKcXlyaEdWYXpnamRDTHNYc1lQY2N0T2o0aCtmUmwybVFDTUhl?=
 =?utf-8?B?OFZ3a0ZMZnFBTXJSK1NBWG44VFRHSnZMTEppaHhBczN4ek16eHQvdnZycWJZ?=
 =?utf-8?B?MzZKQ1NaMDBYSUprVmtvUGZ3b2M2VUcvNm5QdlhWeXIvSVprblc1QVp1SjYr?=
 =?utf-8?B?WjF0ZEJ3TnhLTnlaNWlndlNVNEZLa1ovdVYxN1VXVjB1MXcrdFlKbU9OQmJ2?=
 =?utf-8?B?NFRFNmg3ODlSdXkwUkp1d3planBYRUcvTXFPS0paaUQ1QW5OZzBxWWlscExm?=
 =?utf-8?Q?rX8YE4XHd/jpxBwtdja4KUYuj5kV+O7aAPytAvi?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 733cb1a5-c54a-41c4-369a-08d9840ed853
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:35:54.4645
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SgL0mzeJ4pJiRZIPT79Ewr88HsQdIU/G/WhWCllKZd+ErP5iAy3MdUnMGElgEPiueK0J3e7on0LHxLRtAnkq2w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

Using __native_set_fixmap() here means guaranteed trap-and-emulate
instances the hypervisor has to deal with. Since the virtual address
covered by the to be adjusted page table entry is easy to determine (and
actually already gets obtained in a special case), simply use an
available, easy to invoke hypercall instead.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -2010,6 +2010,7 @@ static unsigned char dummy_mapping[PAGE_
 static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot)
 {
 	pte_t pte;
+	unsigned long vaddr;
 
 	phys >>= PAGE_SHIFT;
 
@@ -2050,15 +2051,15 @@ static void xen_set_fixmap(unsigned idx,
 		break;
 	}
 
-	__native_set_fixmap(idx, pte);
+	vaddr = __fix_to_virt(idx);
+	if (HYPERVISOR_update_va_mapping(vaddr, pte, UVMF_INVLPG))
+		BUG();
 
 #ifdef CONFIG_X86_VSYSCALL_EMULATION
 	/* Replicate changes to map the vsyscall page into the user
 	   pagetable vsyscall mapping. */
-	if (idx == VSYSCALL_PAGE) {
-		unsigned long vaddr = __fix_to_virt(idx);
+	if (idx == VSYSCALL_PAGE)
 		set_pte_vaddr_pud(level3_user_vsyscall, vaddr, pte);
-	}
 #endif
 }
 



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:36:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:36:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199881.354198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvIY-0003xK-J1; Thu, 30 Sep 2021 12:36:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199881.354198; Thu, 30 Sep 2021 12:36:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvIY-0003xD-F9; Thu, 30 Sep 2021 12:36:26 +0000
Received: by outflank-mailman (input) for mailman id 199881;
 Thu, 30 Sep 2021 12:36:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvIW-0003qN-Gm
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:36:24 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 04402960-21eb-11ec-bd4c-12813bfff9fa;
 Thu, 30 Sep 2021 12:36:22 +0000 (UTC)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur02lp2053.outbound.protection.outlook.com [104.47.6.53]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-24-6eL9DEv_Pv2FjHTNjZfUMA-1; Thu, 30 Sep 2021 14:36:19 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:36:19 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:36:18 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P250CA0002.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:330::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:36:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04402960-21eb-11ec-bd4c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005381;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=o8zK1TzXB8klvX7+Pkf65I5+DzQn9KW4HMfyUQ+KytU=;
	b=VKRTLO8jAajIVo5wGOmhq2mOiqO36DXcIbBucVZF1lFsuk16nm/EKLu9dy28xKnFdfhEpA
	goRqEtecj/e4FeXVorWtHtjX7XDcKKuwu7n1GKIFQjiRvcy9DrF+nzEwjp08VtCqeRm2ir
	lNS3fC7eJMcsmofeghsixzjiN8eAh64=
X-MC-Unique: 6eL9DEv_Pv2FjHTNjZfUMA-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kMBiKcdZwFUlZCDUnusQJqhHEl01m6RqnYCQoiilh8RcBq/yikADO0kZd0xNg4SRVIKUfYrq3iKxYe2uIDcPYkXsF6DZ94GX96k3TiSRJF5DAWdBqeNHteF61SANsoROfAvZMzJ3kae/FwIlJArqXovkYearrU7oM6ninbUbuDN0vwNNtgQzrZAB/HOYaWGVypJMktV+Gs6kySnj9ggFTaLWU0orM1ppMs7qUG9k9H+BaQvRbOkVxB7qm9mkmeCylmCA6TdYsm4Z1NberxqPJztMOdgJ9CmbEGHzVg4Bvw+XdAIatVVtnW32e+eBo4d62Xf+16J+TIlrvn3c45fUqA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=o8zK1TzXB8klvX7+Pkf65I5+DzQn9KW4HMfyUQ+KytU=;
 b=Z3p//lghpU4IxqbBK0J98DcyYeBPPdwrJ+g9NIcykIvo1f4Xl8DNMQ0WxUm7bPn9w+vSSJ67JFv9GYVNpnBRL85w1XESnDYbTgTDEaMgTuVLbXYmvLUt+jll4pO9gU537gQ9powSL35qzc+jEZ7Aa4gCM65fm6Bka/YeGTzc9IgzYm95vVidqI9pXsKe7wQzZHm22EoA1Q2UlB2dijg7eFaVw9FhcddpiRrQSRrhUk/L08HNoFOm7hh9B+AGFIeGjeVosuX0AO+WPcaRP3uYlvCQty5JhWZeBh4pqxTYt5Tym6mVr45aa6BJO5YjD5egEeno1ll6xSxkARnUTKQ3Ow==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 4/6] xen/x86: adjust handling of the L3 user vsyscall special
 page table
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <1048f5b8-b726-dcc1-1216-9d5ac328ce82@suse.com>
Date: Thu, 30 Sep 2021 14:36:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8P250CA0002.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:20b:330::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2b717692-eea2-41fb-d7d2-08d9840ee6e7
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB638371F6C3E5AD73B331DBB7B3AA9@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	gdR4A6Abb/a+iWFzTqvPecxY4Eub29t75bTxiVjq2QZn305HQ4ddz4Bc1OOb+VFCCGL1cNjCrrFtwyUFu8RPc/a2VvspELkOFrkoFe5UHbH7s742kuTdaJkq2rIO21WFXEFMVFH6mXtLcqA6wUPm+BJTpZNqlJuJUGHnLB0r3wqOZBelOEe1ptaH461vAnjkoU7g6fj56UYk5uwye9lRJTh1MfD3DfRyrlRAyM2G37N7zKuVs1r99N3LWmqaDKc+nmyfSs0lPo/6C5WEE1eH+aHNBMIIXTZHwgT9UYpm+IPndP03GvQ77PnOgoGZ/0d26dFf2JUbTiIqzkvsgK0IUfo0sITIiOXLSautuuecisOIm3hJisPM2MjUKokGbbWSzqG6+1+Fdq2mnIr4Nzb9MH8Mf7hy1VyWqMfQ8vHzZxK1hfSseDVkTOdsehd50rLlEmHtE0BcjXdE3q5IsTqw/4LW65Aobf/dkBhcHd9HdLyQXU6WZhARRxPx64w+FPirP9NBPJ8DKwFLDaJZn8GiLTayTQemhdblCBWVFjm4zGo9LAwxpl6wMbTVkB3TNkieRUJcxECQWYmyu0uZPd1COwAurTTcqRwwjPes3uvSbJRQBmhCXN/OG5HhcEgBZ63uc1T5H90dlEE/0LLJV1dSlGgqJ7YBBlOuWm0ZWDqsxbdJKR+DeZR1cKm6wSQxZlE6TX+Na4ILH5aoalSx2sScWmmeDrf3gSpg8H4ubVDFgQHEAGUQQyo71vYynd9FrUAZ
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(2616005)(186003)(956004)(508600001)(31686004)(66946007)(83380400001)(8936002)(86362001)(2906002)(31696002)(16576012)(316002)(38100700002)(110136005)(8676002)(36756003)(6486002)(66556008)(66476007)(54906003)(26005)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bTAwNm9TS1RoekNsWmJkVWdpeW9PRkNqb2cxVmRFTnl5ZjZwUjJDWDJtNC9Y?=
 =?utf-8?B?bDkxdU1tczhwNHpmVXdCSDltUWg0S2xzaC9uUUcycG9yMjd5S2V5b20wWlRV?=
 =?utf-8?B?Vk9FZG5sUlc0dmRFUHZ5Q083NVgzTHlSMWxwOGNzZ0pwWVJvM3k5MWdmWlFt?=
 =?utf-8?B?SGQ0TjdVNHh5eUJyRXpGZFJ6eDVmbW5OcFpCSWc2ajlsK2p1SThaZDYwSWdo?=
 =?utf-8?B?NmdWNkcrTFVhUHk2UGxKRWFQN3VDOXQxREt6MTBjaE82c1dqazFHcEN1eUVz?=
 =?utf-8?B?QWt6eU1MK3BaMnoyRGRzSkxCWElpK0h2bEMrdDVrL1NDM0I4d3F6Y2UwVmpP?=
 =?utf-8?B?dDFraFhnVHlsanRQS1RjTDZMSTJONzFyTFZpc1RuV1ZCSmpyN0hhUko3TU5v?=
 =?utf-8?B?cWRoSVZZZWRocytRMHAwMUUrSUREZTlHcjRMMzlKTUM1ZStYYmQ2TU1yRThE?=
 =?utf-8?B?d21NZ1dSV2J4WE1WOEdBZlE1UjlUZTZXS1VZK2ZDelNzWVhBSVdVUWZDL2h4?=
 =?utf-8?B?RDdqb2NvNTVzM2s3dlVKS3RUOHhZTVVoUGQvQTFPandFWXZIRVA2Q09qNUZP?=
 =?utf-8?B?VnJWcGRuQnVycFJKUW1aN3JlK0k2M3VBWU9icXNrRGdSNTBYUUErNWx6clMx?=
 =?utf-8?B?NmxEK0dQemZNc2ZydTVkRnFudlFmdTJJYktrNnZneENIOUhiQ3pIckpoWTl6?=
 =?utf-8?B?R28ybTU4dTN2TGFxM2lKY0pDZEpVTzc2UkNlQ1p2VEp0UEN4UzVML1VRbFhY?=
 =?utf-8?B?VU0rck5TZEw4MTZ5Y05EUzNweW5FQkEwUzhFeDQrd3BxRFdaMVg1dUhmZTJz?=
 =?utf-8?B?bkQ4V29tQUk2T0MrZzVJWmptbVRLUCtUbXk2OWRMOTFPL2ZFdVNtNVE1SzJn?=
 =?utf-8?B?QmpuUTZxbTRpY3pKNGdwYk9ZMUtCT0kyWDBxTTB4K0plWTYvOEV0YUNQRmZa?=
 =?utf-8?B?d3V1NFJQTVRZVlFLQUpPQkFiUjhrMFVUMmJsMldvQjNaS0E4QWZnOW9uMlRn?=
 =?utf-8?B?Q0EyUnBMM1FESWkzVUkvTS9KQkpJc2tzQkYzMWdidGEyMFpnY2xQVTg4RjA5?=
 =?utf-8?B?c2N2NWhVeEc5TFRMbGVscC9DQm5nczA3TlJwMUFBTnpzQjAyU0hvRFBTeVdE?=
 =?utf-8?B?TDRDVk9tWTluSzNQK0Y1eGFjcDc4NUxZVjl0alFMZlNiM0QyamZ3SC81NWl0?=
 =?utf-8?B?Q3FtbmtSRktlRDJtOWVlOGNFeVhUOGcxOExNWVZ4ZVlkV282Tnh3cEljKzVG?=
 =?utf-8?B?WTNSMzRtS0FEeDJZMmw1V2MyWWtDQmU3alNjTGtvTTNPMHE3TmUwVjloOWFo?=
 =?utf-8?B?M3VISXVDOUJqSWQ2bEpCek4wSks4OWl6MWZPTXQvWDB4V1hET1BCTFJwY2Uw?=
 =?utf-8?B?QkJjUENPVm84eFV5amhlblFjMEJXREg3RlhwczU5cG15cHVtTHo5TWdLbmk3?=
 =?utf-8?B?QUlSMXdaY2RwTVpMM2tXMEhZdWJlb2tYZmVHRnBXSHpSTE9PMGhaZy9mYlNJ?=
 =?utf-8?B?VS9aWktHd2xpUW54L3BodDhoeDFZRGhEOW00OEpXcUNyS3MzV2htTk9BaDJr?=
 =?utf-8?B?SHdmeGVrai8xcnk3NTJlSGgyVXZxQjk3anNBN25hY2docnZRVVZHYnA3VllW?=
 =?utf-8?B?MGpsUlhsbERiYkZZb0NrYWpxME9yVlE4QklMQkFIODBOOFQyMlIwemlqYnpJ?=
 =?utf-8?B?VDlYUWoyQ0JtTzZsZGQyaXRUTysxYWxxRTcvczlJMDZiamV3Zmx2eHVQajVO?=
 =?utf-8?Q?c9peOGsyC02fuM45TWZ2tI0IDfAblxG1zFUz95G?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b717692-eea2-41fb-d7d2-08d9840ee6e7
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:36:18.9427
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: p/UVvcBHTex3EDann2EoFsoQf08BAYRzVOqe4nNdq8gQITWVXi39PUQVG29gmdW3u5gyJ6YEe5ipqfbkTqb8sg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

Marking the page tableas pinned without ever actually pinning is was
probably an oversight in the first place. The main reason for the change
is more subtle, though: The write of the one present entry each here and
in the subsequently allocated L2 table engage a code path in the
hypervisor which exists only for thought-to-be-broken guests: An mmu-
update operation to a page which is neither a page table nor marked
writable. The hypervisor merely assumes (or should I say "hopes") that
the fact that a writable reference to the page can be obtained means it
is okay to actually write to that page in response to such a hypercall.

While there make all involved code and data dependent upon
X86_VSYSCALL_EMULATION (some code was already).

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -86,8 +86,10 @@
 #include "mmu.h"
 #include "debugfs.h"
 
+#ifdef CONFIG_X86_VSYSCALL_EMULATION
 /* l3 pud for userspace vsyscall mapping */
 static pud_t level3_user_vsyscall[PTRS_PER_PUD] __page_aligned_bss;
+#endif
 
 /*
  * Protects atomic reservation decrease/increase against concurrent increases.
@@ -791,7 +793,9 @@ static void __init xen_mark_pinned(struc
 static void __init xen_after_bootmem(void)
 {
 	static_branch_enable(&xen_struct_pages_ready);
+#ifdef CONFIG_X86_VSYSCALL_EMULATION
 	SetPagePinned(virt_to_page(level3_user_vsyscall));
+#endif
 	xen_pgd_walk(&init_mm, xen_mark_pinned, FIXADDR_TOP);
 }
 
@@ -1761,7 +1765,6 @@ void __init xen_setup_kernel_pagetable(p
 	set_page_prot(init_top_pgt, PAGE_KERNEL_RO);
 	set_page_prot(level3_ident_pgt, PAGE_KERNEL_RO);
 	set_page_prot(level3_kernel_pgt, PAGE_KERNEL_RO);
-	set_page_prot(level3_user_vsyscall, PAGE_KERNEL_RO);
 	set_page_prot(level2_ident_pgt, PAGE_KERNEL_RO);
 	set_page_prot(level2_kernel_pgt, PAGE_KERNEL_RO);
 	set_page_prot(level2_fixmap_pgt, PAGE_KERNEL_RO);
@@ -1778,6 +1781,13 @@ void __init xen_setup_kernel_pagetable(p
 	/* Unpin Xen-provided one */
 	pin_pagetable_pfn(MMUEXT_UNPIN_TABLE, PFN_DOWN(__pa(pgd)));
 
+#ifdef CONFIG_X86_VSYSCALL_EMULATION
+	/* Pin user vsyscall L3 */
+	set_page_prot(level3_user_vsyscall, PAGE_KERNEL_RO);
+	pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE,
+			  PFN_DOWN(__pa_symbol(level3_user_vsyscall)));
+#endif
+
 	/*
 	 * At this stage there can be no user pgd, and no page structure to
 	 * attach it to, so make sure we just set kernel pgd.



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:37:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:37:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199891.354209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvJ9-0004j0-Ve; Thu, 30 Sep 2021 12:37:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199891.354209; Thu, 30 Sep 2021 12:37:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvJ9-0004it-Sh; Thu, 30 Sep 2021 12:37:03 +0000
Received: by outflank-mailman (input) for mailman id 199891;
 Thu, 30 Sep 2021 12:37:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvJ8-0004hO-J1
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:37:02 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 14e64bfa-0714-4185-8266-1e86bbc43d80;
 Thu, 30 Sep 2021 12:37:01 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-27-DpP6b-ieMKKti2mFjHTOAQ-1; Thu, 30 Sep 2021 14:36:59 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6383.eurprd04.prod.outlook.com (2603:10a6:803:11b::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:36:58 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:36:58 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0006.eurprd04.prod.outlook.com (2603:10a6:20b:310::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 12:36:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14e64bfa-0714-4185-8266-1e86bbc43d80
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005420;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kQLmBvAO/NkPEGuiu0F/4ldlzMcypIF4EQud5MBiHsY=;
	b=KPMhD+ubwGcbeJ0Z2FbrprXjSue7JAAg7cG2m0GyPJ+FF4jtAEloLuVzW74wOw1nzDRigx
	DrwceYg5vXeKkTYgP8kP9NtGOUyOqdn/xJ6gDJ+hp1RAwwdlQJxRzuWk/1pLLb69h/OX7d
	Ie6o1qBwoizJS8rH2L8wai7I7wK0/FA=
X-MC-Unique: DpP6b-ieMKKti2mFjHTOAQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NLgF1NznO7aBcsrf6Z1TOlaQBeHNNiUvPb2gb58YMwWGMdmcyr53JQqCdpcreZ7Mjb/r45nbJqKRFxjSMVd2QOxHKosrUQFuTwT+EVoUjNVyMS0PnkPMXSesUtCjVATBVJxx7gfyithJ2fp/wuINmACD9+GToLPPAIuYsWgu+3sPgOBFlVp4bumGgND9EYYPXXHC3Omys6Gff0HXtaCoCX+SKwbru80+fbcMq3mUvkBoQn9Nej8YmFuUW14Z/B0hnY52aQZHLj6eH/boVdwg1uuUFBMr0cpLDBtUQwMyzvER+c6XOE9unB1CzuUQ/3nz4fhX/TZCmjUB2N/1dAscrQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kQLmBvAO/NkPEGuiu0F/4ldlzMcypIF4EQud5MBiHsY=;
 b=WjwgnqVCBjNuQzXULQV5ichQfsGruTlopM3javsrTOXEu+Z9uY6ZPwGMPhlLJM5n2tqvRorzSHXGxZXHrF7Nmn25LCE5saLTdE2VldA3Ms6G2pzc8g5xc1HAQu4GnHedfbjHbzblLVVXm3e4zUrPBCsNTEoLjQhYwUTjqyGSIOnDInLJkqiUFfjSnBb1dxe8WxDbOw0iW2y77A4c/R7u8UMeAaKUG5jb66LenCx9nC0isclvQMvtEO7DU2bf0C3LkjqAQ8vaDIIBqAWjUgVLmtgu4B5JCtWGlwGSFt28vfOqvyc3vF7JkDMw8BKeyXO67qz7j/8a+sIDxs/mUnfVng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 5/6] xen/x86: there's no highmem anymore in PV mode
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <ba6e0779-18f4-ae64-b216-73205b4eec3c@suse.com>
Date: Thu, 30 Sep 2021 14:36:56 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0006.eurprd04.prod.outlook.com
 (2603:10a6:20b:310::11) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0612504c-decf-405b-a082-08d9840efe61
X-MS-TrafficTypeDiagnostic: VE1PR04MB6383:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB63832D0048136F8F301FF155B3AA9@VE1PR04MB6383.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:2449;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	R4UcBi46QwQHaVjkTAZQCfIupTQMqrpyBx9n7ju4hxbxhJQkdJVISVHTlCeCjp0PFqqT18cQXi5Br8YkyrctwK6p4eIjkHXb2VeOBBLWk+DVHVtVmUNLSC5t9giu8zkNZkkWUG6JQabWTd804QPntlKBrEba+0qEI9boaajzUrwVXWM+nvk1dYGj/60hq+hhxGN8XnUgbFGEicwvyedCnP+AjMnIiHywhqSBPSfK8HcH77XBaNQrMG/scSsBW7BbTLhr54CLGpqVDPB1khY1bkLtO28R/m8iVrRanDHjeTfGHh13sanFzW/cwvE2XCh+aQM+PqyqefjUBftXJmwQM3ri9hVlwqu4KOPDmQhtspyVl1iK+VB8QuyhsIEE9o91TU43LWh39uVYFaUdFTMg008Q/obFHIqvGpgCaOsJNhZWgCr+hhJAKzx7XlnRm1L/WXQANRPXGfR0s16utEEr95IPWhGB5sxlUOTPgGZ2xBybR3gjKsJ+B1QVciSRCpBZ8Opw+mgBZCltRvCAO3DPEWNjOAhQavCdMXkCdj3mepDdjeLFmwtZB19g5H9gfzKGG9AzEu0pGUOQ2i3nKkC0l19YiQmNDhTQ1BDQtHd4nD7gUBCM5mvmVbWQMFhz6fA/UKzA0KI25SJ76/kk9CxorPwfuHBng3FI5ix6dipLk1iW/OXP8D/gABOvQOINqK48t0OY1fqCKeYPEOo1S9fyQy1/FYGrM6iI7U2aAubCwoI2hx7HxILDmXIiRqZ6dIgy
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(4326008)(2616005)(186003)(956004)(508600001)(31686004)(66946007)(83380400001)(4744005)(8936002)(86362001)(2906002)(31696002)(16576012)(316002)(38100700002)(110136005)(8676002)(36756003)(6486002)(66556008)(66476007)(54906003)(26005)(5660300002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aVNIQVdJUFNvZEhPTE4zZ2Y2U2lwdnh4ampJcHA4ajB3UWtnR2JMR2hOa1FB?=
 =?utf-8?B?NXVJWDcwQ3pkTXliMU5jdDMyeWtmam8yV2QydHp1bURsWGhvT1cxUW96c2M2?=
 =?utf-8?B?dUpsKzRJb0lIRzNrSVAyVjYrcnRQSWpabnQ2WU9PMC93TVhnTkFLakdRT2lT?=
 =?utf-8?B?b3pEcXB5Zm9qN3E1TkV3dklqVnBLWE9QaWhORWh3MmtlbFpSMGhFcFExQTVa?=
 =?utf-8?B?TFVzOTcvWEJNY29NamRGRy92RXM3MzY3ejk2RWpTUzU0eWNzdC9yL1Ezanov?=
 =?utf-8?B?bkErZEhvQWR4VndmNlRrZEkySE5mdjZGRVNlTjZ1ZHpUTElIOWdabkRiL004?=
 =?utf-8?B?NnBUSGFHRlJnaUY2U2hKWDZoQ3FHTkpzUjI4RGxQaDhJWFV3anZtY0FVcm9C?=
 =?utf-8?B?aW9jdXQya2dWcGF6VEZGK2I2NDFaSEJhQ28vd0dQTWIzTE9QTDZOUUk0RlFj?=
 =?utf-8?B?S0ZZVjU2ZTRhUzV5b1FWOFQvUzUxc1R6SEdqSEhabVNDbksrNTM2MUlBck8v?=
 =?utf-8?B?dGtLNE9oa29kQ2UrWmNYU0N6T00raEtoaHEwWUg5V3JZNzg2SDQzMml0VCtY?=
 =?utf-8?B?U09zWDNEUGVjQ0NKbnM1STdtbTE1UDdlMjZLcEYvY1NVdXdkTUU1aUhzUFVo?=
 =?utf-8?B?bGJnUkpSZ1I1RjFXaXJEUVh1S2hDVEwvUk41Q3ZkNEFoR3hSK0k2VTdTT1Bq?=
 =?utf-8?B?UEJ4cllINXg0c01rT21kL0dCVHV3dG9PVVlBcHNGWmg4VVp4cFdkQjJTTy9j?=
 =?utf-8?B?U2t2Ri80bkp1RitYN3c5dzJ6ZG9paGFDeUh2eUdTL1hTTGo2QXZGV2krYUR2?=
 =?utf-8?B?dmZqblZ1ZTliTjdzZ2laOFMrTHlFb09tOFNpNnZIdCszMFVqWjFpU0xQampw?=
 =?utf-8?B?Z29XUitya3dmTC9nb0pqcXkwTjlxRDhrWFRMSEY1bUtuazZEdzJ1UHpTTy8r?=
 =?utf-8?B?b3E3T1FUd3lhSkRIR1dRSDdOWks2VEU2dlR5WFQzNFpON0VIaWRkWVJNd2c4?=
 =?utf-8?B?eld4aUxvY0dFMDUzdFN5ZEwxWEM3M213dGlGaHJqR056VkVua1BaWHhRT1ZQ?=
 =?utf-8?B?eHVHWld5VXY0dW9GMFVvS05DWkhldnJXck9tU0JwNkxLTVQrRUk5Zk9FQlNa?=
 =?utf-8?B?SzR3aC9MeDVwUHZEbWFwK1k0a2orcGxhVCtrbXBxTk1ObjZLd2VHc2VTYTRk?=
 =?utf-8?B?bFJPaDBxZm5nWjhjV1FQeUczNDIxbTlrSW8yOVlxMlpOM3BkTTdQcmFJQmQ1?=
 =?utf-8?B?b0E0MnVLbkVDRSsvV09oTUJVOWQwbFgxRXVUTzhwM1lHQ2xxcTMvQlhaOUdh?=
 =?utf-8?B?VkIyMkc5V0pjRUkzTkpxd05NUitnTTErTVlyUjVHeDBVZXg4UWZCWGpYRjR1?=
 =?utf-8?B?bC9oRXQwazZoWXZIOWpJZFRtNTJVK1F0T1djMjg0a2pKZVlzb2QwWDBSMDNJ?=
 =?utf-8?B?NEFUVmtucVBWTEhPRnJ6eDhhMlJJbDliOXFsZGk3NEh3VVpHcm54eC8vUWhQ?=
 =?utf-8?B?blNQZWxBWGhHdzZGWlNHb3dyeklhSjkvc1h3MXN2QnFvSEpPdkFrdVFZaFp2?=
 =?utf-8?B?WElKZzlNN2poT2NSYjUrTW5NdFErSzcvbEZDNjlrTDNqWjlwdkZqZkVkWnZ0?=
 =?utf-8?B?cWExVlpGS1dxQ2VJWjB1YXVhUUxzTWxxQXc3dEdEUld1cVU3Q01lRHZpZDJi?=
 =?utf-8?B?TThyV2JoZjlmOU90enhZTEROcTRkTVBTdFczUmsxNVduZ1M5eVNuTG8zN3Ux?=
 =?utf-8?Q?Jg+1OiP19zLbircV4luKpuR4I3O06jn9mVVG5eH?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0612504c-decf-405b-a082-08d9840efe61
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:36:58.2824
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lorDMFZEw33I+PbNj/8V+X2Uh+khFHY8f/LF/MBUkcYTQgNai/Ztpy/5HDhjm0hmyow0lZj1JBun0R3ePZGdVw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383

Considerations for it are a leftover from when 32-bit was still
supported.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -306,10 +306,6 @@ static void __init xen_update_mem_tables
 		BUG();
 	}
 
-	/* Update kernel mapping, but not for highmem. */
-	if (pfn >= PFN_UP(__pa(high_memory - 1)))
-		return;
-
 	if (HYPERVISOR_update_va_mapping((unsigned long)__va(pfn << PAGE_SHIFT),
 					 mfn_pte(mfn, PAGE_KERNEL), 0)) {
 		WARN(1, "Failed to update kernel mapping for mfn=%ld pfn=%ld\n",



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:37:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:37:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199896.354220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvJa-0005Q4-8B; Thu, 30 Sep 2021 12:37:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199896.354220; Thu, 30 Sep 2021 12:37:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvJa-0005Px-5A; Thu, 30 Sep 2021 12:37:30 +0000
Received: by outflank-mailman (input) for mailman id 199896;
 Thu, 30 Sep 2021 12:37:28 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvJY-0005OJ-JQ
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:37:28 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 1aa5581e-a0bd-4b30-9dfa-2e87ad04a3e8;
 Thu, 30 Sep 2021 12:37:27 +0000 (UTC)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2107.outbound.protection.outlook.com [104.47.17.107])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-18-xfIf_bZcPQyeHCkSQiq7nA-2; Thu, 30 Sep 2021 14:37:25 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13; Thu, 30 Sep
 2021 12:37:22 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:37:21 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8PR04CA0044.eurprd04.prod.outlook.com (2603:10a6:20b:312::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend
 Transport; Thu, 30 Sep 2021 12:37:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1aa5581e-a0bd-4b30-9dfa-2e87ad04a3e8
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633005446;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=kPPXxxwugPBH+beN9q4uizgqAq1OKQYB05wUlCmlLlM=;
	b=KE1jczkPjRiGYi1Af8INeMw6vNy/tRNOAmuF7RjGtwuJStzm7pFbWblctRGKbNnzGbFmHm
	6+5D/2lvjvsACUAoH5cR/ca3Ns670cFhFGo4MwCdyQg3lwSzraeqD7E4FtoIF6KQfG9DVK
	Lr9SWQhCsvs89C7bOEGCZex+b84Hbpo=
X-MC-Unique: xfIf_bZcPQyeHCkSQiq7nA-2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XilTaAxqeBZWIJGbAE5s+aGyJxwBTKSk8JJIcQrGqZ56NngxDJh9hPmPvxM+fRJQTOOB8UH7l9TvHBP2yjvJNk2u7t5ZIXVOBZZ2pgHd2OUxBbsavaHT65oINnSGheq2AgnETqceJNhs8rRDrLvf2Mwn2fwG+z8yPgwpQ0X5wvFKQ6800h8iOyqyeGAtYYPQKIKTfP8YRaq8U+JObZf6Zc87ncHz9vA/3KQPwS8BTst102uFq1izpVMFUAglboi19KA1Le00aggzRFcl4Kz2GhsUCqQ3RoeahTcOaD0p8VzszyCWvwdBYEYwT25OcSHBfoi6/AJnFOskedlbBRqHgQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=kPPXxxwugPBH+beN9q4uizgqAq1OKQYB05wUlCmlLlM=;
 b=L81yw8xq8kuaHu8deo1awIB9H4n/kFDii481Ozwug91sYMLO9KNforOxYqXeAE6KaYblcI6PkfRJkyuE0vNEbiKRemO9hHxneo51pjXTovhT0QgvlLRa3LoeEqVFyMTvLE8U0LwTwlzxbPVDsGpGfmycn48kUh1rwFCe/NG8SAZlhUocJtmcsqlJTfcuS8WQspEXM4XC2lZUIo8u27f6nEkyXYGPJwmU+p6/kqDEjKwZ0r8Z3BoblFLH0U9wXHh3Z7VCfMDJLFVg/+82qREaMopTNDrv7a8rQ3galWv6Dx1Fo5LuWX3RDraRs+KnxISelYZ3+zZEFD06EkxqCJ6VYQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: [PATCH 6/6] xen/x86: restrict PV Dom0 identity mapping
From: Jan Beulich <jbeulich@suse.com>
To: Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <038b8c02-3621-d66a-63ae-982ccf67ae88@suse.com>
Date: Thu, 30 Sep 2021 14:37:19 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8PR04CA0044.eurprd04.prod.outlook.com
 (2603:10a6:20b:312::19) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2cfb7984-2657-4317-0ae4-08d9840f0c60
X-MS-TrafficTypeDiagnostic: VI1PR04MB5600:
X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB56009F302F81D72F5CA19C83B3AA9@VI1PR04MB5600.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XvmdscEd9rEdQFfFcHzHE1yUzmCA78KefJFiaS7tFz7cTJRHRuJb5OVAubYJ/hJnDKcaueTs421LujF3q2Gq74K6EKMuda/8MmLMK15SeKvZ3iB+HEliVw3Vpkat8CpnlVaIWdyk8tBxSJs7E94elR/+7cS7aRfU3aJYV+fLwULbQSA471/n+h7e1H/Fq0AWl0TE/sVeGnAznw1QpzpaNU52beKmxKjL8DyTUSFpGt3U6ZaZ++2osWQekH0GlOyQ3VoPJXLiVnEPM+5E1k9uet3VlQdAY6j8zFRIM6B3FfkD+ojsMqME2UNz7+tFnjAQwfc/2BQafiOv6ElT7GUygYdOry4RH4o61ClJMhKGmCSsJI3mpsT+VMaYgxNOZvx1OwvTfH3K1CjFez0uYtKGWZQPegnhcRIB+co5wZB8Oac16+v19khbEap0d/Qi3CFgMLDotqVe5PRInBTTm2a0LxGvsgD84nD/a9igeJ9A7hjSiIPJ4+SUo4Uo1NMb/OLRohHufORSrIfFmfoPsCvx/DcEo8LAS9YbCSZX/nlBrkKnCONLsaGbmdnOWRGko1e7bfaBV/9UnPr+wsNs0Q/EPvzTFdbOtXAKIdfm4yH99c9aNpR7+25fXkiX2/JpwT2h2Pxjd5u92hxxQdMwtWRagqyiWt/0jC/YuBrtHUF89CQ5h+6NWmJ69+0oQYKrKgdDib07cB7P5+rAvc2CtRDwAyNDVfI2ThBQT8SVCYhhjaXbMBDOlMJ912X64JEspchd
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(508600001)(2906002)(316002)(5660300002)(83380400001)(38100700002)(66556008)(66476007)(66946007)(16576012)(110136005)(54906003)(8676002)(31696002)(8936002)(31686004)(4744005)(186003)(6486002)(956004)(4326008)(2616005)(36756003)(86362001)(26005)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?M2p2R1N1S3czTkRsYXdxQ2MwZjdXdlAzVlYrVkRFWmNXVzhveTlQWlo4cWpE?=
 =?utf-8?B?dE1tNmNZUEcrU244WGl0c2Q5eWhhMXQ2WHQ5ck9qVDlEb1VCZnNaS0RCT0Vt?=
 =?utf-8?B?K2dXWWtLMm1DYWF4cjd4NDA1MEEvWXlnaU9jY0NkME9iTzFVbk01c2o0b2lW?=
 =?utf-8?B?MlJrK3R5d2Z0Y0kvTW1yck1PWHhEM3lIeUVmdVFscjZnZXFPMHJUcVFDV3kw?=
 =?utf-8?B?cjM4K0V5dEQrajhZT2dWMzhPdWNUWVZsUU9OSTlQTzNNL2tENlhMcld3NWRF?=
 =?utf-8?B?VWRaVCtKdnBrdmo3NkE3SWRQeUhURy9tV0YwcjlLYmdxZkVGTm9hTGdCdkFn?=
 =?utf-8?B?WUwwQmFpZDE3RUlDck8rdW53QWRERzJuRm51YVI3QTk0WlNGTWZWMFE3VXJY?=
 =?utf-8?B?aWRTdnBwWE9oZFhvLy9aMitGRUxUZENJcVhSRnhscC9pZUM5aVRZdlRyaWZU?=
 =?utf-8?B?cnYyUFVCS2NIcmxnNm05L2diNUVhaTN4NnFUeng4OWlpSGpjem9hcjI0MXhL?=
 =?utf-8?B?ZHp3MWRMQXc2QmhwK21XUVBHTHAxS2dwZ2Znc1pEbWY5eHdkYm9icmJFa09I?=
 =?utf-8?B?aU01ZGF0Qm13K3VsQnJYUHlQRSt5bENUeDRHcGFyNDg3WEZvd0M2V05oTk1W?=
 =?utf-8?B?STd3T1c1ZkcvSjdrSmJxRjFkNVBqZm5vdjVvWTFWRlRpNXpaakJ4NnlJTFdJ?=
 =?utf-8?B?dTlsMjV1N1JTQ01NWllUZW0wbUJZZ3NucnBzYno4a2dpT2JneVRFTURaQnZm?=
 =?utf-8?B?Ym9pbXdwUURvL0VvcEkxNEVIRHJDQWRIbmIzbXZseDNIZURDZ1FtaTNDUUh6?=
 =?utf-8?B?RjlGUGJKK1RYLzZGN29TYSs2eUg3dHhYNkJwR2pPeTZjRDhNM0Jsc2FFUTVt?=
 =?utf-8?B?Um9kRUxGU0E5c0QrRDBadis0VkVoa1Vqc3VzK0plMFE1R2UzNGJxZlZWZjd6?=
 =?utf-8?B?UWVrTVY1SS9nQmtTYkF6N1hFUW5iSVIxK0xxWWZwYmMrbnlxNlZ6c3g4ajNm?=
 =?utf-8?B?dzJsWGlYemxKWVhTRjhnVWhqVEJ2M3B2QmJoR0toNXZxNy9mdUNZZlhmRzlG?=
 =?utf-8?B?cjB1NTFEaklGcU1rTWQrc3dod2pEanpybWVhTzI3WEZEY1phQU9vbW9vT3dL?=
 =?utf-8?B?UGhCSFNIWU1JcDRPbnhmV05OYkNRRFRjY1R1Z2xRVHFSdHBCNXM3RWRWV3pk?=
 =?utf-8?B?UXYwL2prWnJ2R2VxZ055bXp5cGxzZVlKaFFrZTZWUXpEeCsrbUxtN3NRNVN3?=
 =?utf-8?B?d1ZMbEk1NTIzL2ZFalJ4Tm5vVmZBV1RPWUlOQWR6NXlkNFg5OGFIN1FsVlE5?=
 =?utf-8?B?WklRckQvRVV4aEJXUUdjQjJhSDJlc00xVmpvUW9Ba2pwSS95dzVtL3JlNm82?=
 =?utf-8?B?S3NNL0kwVDA5azBYOXNZdlVld1RuY0dnN2I2ZDkvYmRUY01FWVN3OExYUHAr?=
 =?utf-8?B?TkJpbk1XdXZEZVVheXh6UW45VFdqQUJQT0VjczYyNlVsTmlCVmozd21FMzJB?=
 =?utf-8?B?cmtkYzBrRCttcXlOK0h4eEo0aFIzZldyajA0UWN1WkxiRXBiMkgwZXdHWG9G?=
 =?utf-8?B?RkpnZWh2QXhTdHJPRnlJbEhkM1o4ZnJKczVMYmNwd1Vldk5GYWJ3TExIZ2xh?=
 =?utf-8?B?SUx1aC94eXIvTGNlWEhlNERJeEpyYVRuN3c0eWtweU5XYjI5TW5ZL0pzemY4?=
 =?utf-8?B?WFBEYXlEZFVqRmFHclJQRndWNzBpUEU0NFM4bEpRM0xaVWVCQ3ZzMGgySXgx?=
 =?utf-8?Q?93ZKV8oWsM+vckc/XhkCzli11oCJHU6m/B8ieBu?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2cfb7984-2657-4317-0ae4-08d9840f0c60
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:37:21.8021
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: N+68ucEy+pC2zMfLvWa3YBirMpv4MDFLr5fQVKEKl3arg/Yr/kg/vljVeeZ6LHuxaQ1Uv2OjsEoU6X25wHf1UQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5600

When moving away RAM pages, there having been a mapping of those is not
a proper indication that instead MMIO should be mapped there. At the
point in time this effectively covers the low megabyte only. Mapping of
that is, however, the job of init_mem_mapping(). Comparing the two one
can also spot that we've been wrongly (or at least inconsistently) using
PAGE_KERNEL_IO here.

Simply zap any such mappings instead.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -425,13 +425,13 @@ static unsigned long __init xen_set_iden
 	}
 
 	/*
-	 * If the PFNs are currently mapped, the VA mapping also needs
-	 * to be updated to be 1:1.
+	 * If the PFNs are currently mapped, their VA mappings need to be
+	 * zapped.
 	 */
 	for (pfn = start_pfn; pfn <= max_pfn_mapped && pfn < end_pfn; pfn++)
 		(void)HYPERVISOR_update_va_mapping(
 			(unsigned long)__va(pfn << PAGE_SHIFT),
-			mfn_pte(pfn, PAGE_KERNEL_IO), 0);
+			native_make_pte(0), 0);
 
 	return remap_pfn;
 }



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:45:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:45:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199903.354231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvQz-0006w8-39; Thu, 30 Sep 2021 12:45:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199903.354231; Thu, 30 Sep 2021 12:45:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvQy-0006w1-UL; Thu, 30 Sep 2021 12:45:08 +0000
Received: by outflank-mailman (input) for mailman id 199903;
 Thu, 30 Sep 2021 12:45:07 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wsc4=OU=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVvQx-0006vv-GK
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:45:07 +0000
Received: from smtp-out1.suse.de (unknown [195.135.220.28])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d1afc5ab-d5c0-4aee-89aa-b84dbff2c331;
 Thu, 30 Sep 2021 12:45:06 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8DCC42261F;
 Thu, 30 Sep 2021 12:45:05 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5A44E13AF5;
 Thu, 30 Sep 2021 12:45:05 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id e+owFFGxVWFLRgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 30 Sep 2021 12:45:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1afc5ab-d5c0-4aee-89aa-b84dbff2c331
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1633005905; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=TWJTayddefnypqmsghCaK9+8qol/6a+/ZBHJ0MnmsD0=;
	b=JPF3ldj9m6CRX5hp3qpyIf9ZXMfLGy3NKSHBDW/BPTPW+CDpUkF+CqDBBt71lai2+spmU6
	NS/0o6OJDEGHl8fbQvifnDu7ocrFnJAhzXVxmyfwMkY6p+vA0OSImzMQ0Dy/rNy9SO+ZVx
	X4gnj8riJAKZOCmFWnFUvH8c+gui6xY=
Subject: Re: [PATCH v2 7/9] xen/x86: hook up xen_banner() also for PVH
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
 <de03054d-a20d-2114-bb86-eec28e17b3b8@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <c00627b8-b60c-ca55-5f82-64a6dbabea3f@suse.com>
Date: Thu, 30 Sep 2021 14:45:04 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <de03054d-a20d-2114-bb86-eec28e17b3b8@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="ZQrmQY3031osLcTGSOrv03T89tkhGwRki"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--ZQrmQY3031osLcTGSOrv03T89tkhGwRki
Content-Type: multipart/mixed; boundary="5K4eyEyxBCTTZPHg4X6e4lrm62lADzsE0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <c00627b8-b60c-ca55-5f82-64a6dbabea3f@suse.com>
Subject: Re: [PATCH v2 7/9] xen/x86: hook up xen_banner() also for PVH
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
 <de03054d-a20d-2114-bb86-eec28e17b3b8@suse.com>
In-Reply-To: <de03054d-a20d-2114-bb86-eec28e17b3b8@suse.com>

--5K4eyEyxBCTTZPHg4X6e4lrm62lADzsE0
Content-Type: multipart/mixed;
 boundary="------------37763B345289940F937E8A0E"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------37763B345289940F937E8A0E
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 30.09.21 14:19, Jan Beulich wrote:
> This was effectively lost while dropping PVHv1 code. Move the function
> and arrange for it to be called the same way as done in PV mode. Clearl=
y
> this then needs re-introducing the XENFEAT_mmu_pt_update_preserve_ad
> check that was recently removed, as that's a PV-only feature.
>=20
> Since the string pointed at by pv_info.name describes the mode, drop
> "paravirtualized" from the log message while moving the code.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------37763B345289940F937E8A0E
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------37763B345289940F937E8A0E--

--5K4eyEyxBCTTZPHg4X6e4lrm62lADzsE0--

--ZQrmQY3031osLcTGSOrv03T89tkhGwRki
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFVsVAFAwAAAAAACgkQsN6d1ii/Ey+Q
9Qf/QbC+GJ+9k3VKVTxwErhYevb/KI/p7WHXoGlDyIVrhQ4bVyAoKGSLbG4OD4MpHD8de+oCSIyM
kzco50I4Rjouf0cxM8lA4wFCEcrr58taOskVPnKnKSvjbeO/B6Q2iRMdYodf8AIYi50MPfEDbt5c
WYp6olZGssU0G98dqiz3pNcu0Ro9HGhOygBaj6eojwsYsH/qNYX1rD3NIU5HAiFee253ZNC5+yg/
qYw2svG4RZPW5k8teuVstMxvz3b5YL+BzY4/106ErqDNKXMA9CSdEXxLUgsnSeIwz7rowdUT2WhY
DVrKOc4lC+msBBmC0X3xnDQVujsDspv5baYtQJXXZg==
=mPYF
-----END PGP SIGNATURE-----

--ZQrmQY3031osLcTGSOrv03T89tkhGwRki--


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:45:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:45:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199912.354242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvRh-0007Y8-GF; Thu, 30 Sep 2021 12:45:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199912.354242; Thu, 30 Sep 2021 12:45:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvRh-0007Y1-CN; Thu, 30 Sep 2021 12:45:53 +0000
Received: by outflank-mailman (input) for mailman id 199912;
 Thu, 30 Sep 2021 12:45:52 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wsc4=OU=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVvRg-0007Xt-5D
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:45:52 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 579f4bbc-21ec-11ec-bd4c-12813bfff9fa;
 Thu, 30 Sep 2021 12:45:51 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 95486203AF;
 Thu, 30 Sep 2021 12:45:50 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6BB9C13AF5;
 Thu, 30 Sep 2021 12:45:50 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id IS0DGX6xVWHIRgAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 30 Sep 2021 12:45:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 579f4bbc-21ec-11ec-bd4c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1633005950; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=wu7GRg/evl4OfqbzgGaoYBTup4MJPjbtbsww+YH0qBY=;
	b=cNyKi2lkcRw2fpHvuDL0KaCPmPy4AnOJRCz/EsaStuMjuK72QrBHaI6Yx6Jkb+J+E/CTSv
	gIeVv/zGcedRyiLAts4r01nUbk+3YS1vISUd4mxIihogmNx9AOHyxVbMgOdAHrFn5wzzUP
	fpXmTJYplB48sLekmp9zX4iXzldPloc=
Subject: Re: [PATCH v2 9/9] xen/x86: adjust data placement
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
 <8155ed26-5a1d-c06f-42d8-596d26e75849@suse.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <adadbd76-c42e-d0f4-62ee-51f5fbfa7cb4@suse.com>
Date: Thu, 30 Sep 2021 14:45:49 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <8155ed26-5a1d-c06f-42d8-596d26e75849@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="3gqztLcLIbU9pqrinmrbuszvpNec8xuDe"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--3gqztLcLIbU9pqrinmrbuszvpNec8xuDe
Content-Type: multipart/mixed; boundary="dMhEmZyl3XCMNbhfkcBfMoFAIFH8OueZ9";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 lkml <linux-kernel@vger.kernel.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Message-ID: <adadbd76-c42e-d0f4-62ee-51f5fbfa7cb4@suse.com>
Subject: Re: [PATCH v2 9/9] xen/x86: adjust data placement
References: <9a26d4ff-80a1-e0c1-f528-31a8568d41f7@suse.com>
 <8155ed26-5a1d-c06f-42d8-596d26e75849@suse.com>
In-Reply-To: <8155ed26-5a1d-c06f-42d8-596d26e75849@suse.com>

--dMhEmZyl3XCMNbhfkcBfMoFAIFH8OueZ9
Content-Type: multipart/mixed;
 boundary="------------65D229AC2BF0E5E96187DE92"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------65D229AC2BF0E5E96187DE92
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 30.09.21 14:21, Jan Beulich wrote:
> Both xen_pvh and xen_start_flags get written just once early during
> init. Using the respective annotation then allows the open-coded placin=
g
> in .data to go away.
>=20
> Additionally the former, like the latter, wants exporting, or else
> xen_pvh_domain() can't be used from modules.
>=20
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

--------------65D229AC2BF0E5E96187DE92
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------65D229AC2BF0E5E96187DE92--

--dMhEmZyl3XCMNbhfkcBfMoFAIFH8OueZ9--

--3gqztLcLIbU9pqrinmrbuszvpNec8xuDe
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFVsX0FAwAAAAAACgkQsN6d1ii/Ey/M
7Qf9Gbt6CjO9VaYm8CBQ0zDBvORop0qUx51PV2MOJM7ku56eDJWodAh4rDDLqEtvtswwALkF6k4O
nRqafPGPY1CWjZAsNEQ2wYtYa9MbftqaFL4eJ0edn8S0yqqPvGPBUPoTB1lSYnM7+B4wRFsRJmYP
jew9435BtoGQ7/qToBb4E8hPgSmEXJ+vP8t+FlmCf737Ax28+sep0Tvkj6jrE1L12P8aR1cUsP4h
xiaLNK52Ww5w/F+NSP7/rOAcTp3h8fPhyuHaKr/PCPJgnavip/wxWD8cpCrld4J2WQbJag1fHtjk
/QyPv7bzWoafZ4McA9lTzV3qxhZiM3E4aYjYyUqmKA==
=w8AB
-----END PGP SIGNATURE-----

--3gqztLcLIbU9pqrinmrbuszvpNec8xuDe--


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:49:43 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:49:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199918.354253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvVN-0008Og-Vc; Thu, 30 Sep 2021 12:49:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199918.354253; Thu, 30 Sep 2021 12:49:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvVN-0008OZ-S6; Thu, 30 Sep 2021 12:49:41 +0000
Received: by outflank-mailman (input) for mailman id 199918;
 Thu, 30 Sep 2021 12:49:41 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wsc4=OU=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVvVN-0008OT-Av
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:49:41 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e000ec55-21ec-11ec-bd4c-12813bfff9fa;
 Thu, 30 Sep 2021 12:49:40 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 906DC203AF;
 Thu, 30 Sep 2021 12:49:39 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 59EAE13AF5;
 Thu, 30 Sep 2021 12:49:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id MS/JFGOyVWEaSQAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 30 Sep 2021 12:49:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e000ec55-21ec-11ec-bd4c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1633006179; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=eXXM4x1MRCauedXY2Jv1XTDU3b0ITJwPLNKV+xczrqQ=;
	b=KdHv4EqN+hMlURQc9YJDfqDVWCOsP3OyS6YoLepn8C+zm2YYySGPjs49H1ltg5GEMZ5rYg
	cWoozbu15cXsa93apgDsyQoab2QBp0mdv01rLokTxhdOxsG82fjkC1doBvkMktBHRxWOLd
	gqafWi9bhJnudnGR6/Mc/w2BXDFbqUk=
Subject: Re: [PATCH v5] xen-pciback: allow compiling on other archs than x86
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 jbeulich@suse.com, Oleksandr Andrushchenko
 <oleksandr_andrushchenko@epam.com>,
 Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
References: <20210928073501.433559-1-andr2000@gmail.com>
From: Juergen Gross <jgross@suse.com>
Message-ID: <3da6887b-6df4-a094-5bd1-50c2d411f83c@suse.com>
Date: Thu, 30 Sep 2021 14:49:38 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210928073501.433559-1-andr2000@gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="lUlzfy1G6XsbRc7JKvGWmrqpDhmvSe9Z9"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--lUlzfy1G6XsbRc7JKvGWmrqpDhmvSe9Z9
Content-Type: multipart/mixed; boundary="cXsIjVzo514qfTzLwto4Ph6nYxAeADC7t";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Oleksandr Andrushchenko <andr2000@gmail.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Cc: boris.ostrovsky@oracle.com, julien@xen.org, sstabellini@kernel.org,
 jbeulich@suse.com, Oleksandr Andrushchenko
 <oleksandr_andrushchenko@epam.com>,
 Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
Message-ID: <3da6887b-6df4-a094-5bd1-50c2d411f83c@suse.com>
Subject: Re: [PATCH v5] xen-pciback: allow compiling on other archs than x86
References: <20210928073501.433559-1-andr2000@gmail.com>
In-Reply-To: <20210928073501.433559-1-andr2000@gmail.com>

--cXsIjVzo514qfTzLwto4Ph6nYxAeADC7t
Content-Type: multipart/mixed;
 boundary="------------54D70514E388F36FE477B465"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------54D70514E388F36FE477B465
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 09:35, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>=20
> Xen-pciback driver was designed to be built for x86 only. But it
> can also be used by other architectures, e.g. Arm.
>=20
> Currently PCI backend implements multiple functionalities at a time,
> such as:
> 1. It is used as a database for assignable PCI devices, e.g. xl
>     pci-assignable-{add|remove|list} manipulates that list. So, wheneve=
r
>     the toolstack needs to know which PCI devices can be passed through=

>     it reads that from the relevant sysfs entries of the pciback.
> 2. It is used to hold the unbound PCI devices list, e.g. when passing
>     through a PCI device it needs to be unbound from the relevant devic=
e
>     driver and bound to pciback (strictly speaking it is not required
>     that the device is bound to pciback, but pciback is again used as a=

>     database of the passed through PCI devices, so we can re-bind the
>     devices back to their original drivers when guest domain shuts down=
)
> 3. Device reset for the devices being passed through
> 4. Para-virtualised use-cases support
>=20
> The para-virtualised part of the driver is not always needed as some
> architectures, e.g. Arm or x86 PVH Dom0, are not using backend-frontend=

> model for PCI device passthrough.
>=20
> For such use-cases make the very first step in splitting the
> xen-pciback driver into two parts: Xen PCI stub and PCI PV backend
> drivers.
>=20
> For that add new configuration options CONFIG_XEN_PCI_STUB and
> CONFIG_XEN_PCIDEV_STUB, so the driver can be limited in its
> functionality, e.g. no support for para-virtualised scenario.
> x86 platform will continue using CONFIG_XEN_PCIDEV_BACKEND for the
> fully featured backend driver.
>=20
> Please note, that CONFIG_XEN_PCIDEV_BACKEND and CONFIG_XEN_PCIDEV_STUB
> are mutually exclusive.
>=20
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.co=
m>
> Signed-off-by: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen


--------------54D70514E388F36FE477B465
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------54D70514E388F36FE477B465--

--cXsIjVzo514qfTzLwto4Ph6nYxAeADC7t--

--lUlzfy1G6XsbRc7JKvGWmrqpDhmvSe9Z9
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFVsmIFAwAAAAAACgkQsN6d1ii/Ey+S
pwgAlPBK8BIl52Q0Iox5w8WmBsuStuEPF/1DwtiVdnKExhIz39cO9Sn/dOA1hNLeLzXoSCWfXpkU
qbQi7zpt/vXePjFuBabqDAsJFBDABfoq7JzeygCI5ml93dCgvCk+XLC3dWQHZ87gGDN68+dnmFpx
hXzu2TtHYFI24jjwGcev3Y37AiUYp+IhkNuAblnfu52bhhmfFxRBtFHPfSeelcs5mXmSTcoh8E2J
W5rr/dnWY15iC141j7Egcc3sPvytKwSMbb3MinO6eILflVHGuvEMMJn2fhQg4VLn3+8EGiCyRCuo
fRCv/ejY3pQg8mGo+LX0iGVjkGfMvY9dWZIMKk6Deg==
=4kLO
-----END PGP SIGNATURE-----

--lUlzfy1G6XsbRc7JKvGWmrqpDhmvSe9Z9--


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 12:57:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 12:57:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199925.354263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvcZ-0001Vd-ON; Thu, 30 Sep 2021 12:57:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199925.354263; Thu, 30 Sep 2021 12:57:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvcZ-0001VW-Kw; Thu, 30 Sep 2021 12:57:07 +0000
Received: by outflank-mailman (input) for mailman id 199925;
 Thu, 30 Sep 2021 12:57:06 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVvcY-0001VQ-Jx
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 12:57:06 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id ff7a9e87-ad82-486d-b78a-62d26d647916;
 Thu, 30 Sep 2021 12:57:05 +0000 (UTC)
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-VQDCdMeePb2Ef9ThNPZJPw-1; Thu, 30 Sep 2021 14:57:03 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4349.eurprd04.prod.outlook.com (2603:10a6:803:40::27)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 12:57:00 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 12:57:00 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS9PR06CA0345.eurprd06.prod.outlook.com (2603:10a6:20b:466::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 12:57:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff7a9e87-ad82-486d-b78a-62d26d647916
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633006624;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IxCuf0pT2Zfq6oHeAx77qfUtNUjzCLWnMNKHXtce4V4=;
	b=IWHCmb4u+KxbAIFfyDBPogNld+5f6IepS9s3zxK261Y5M8f2CmVJbc42IAviM0WzjLdkhr
	nH8RRSdo4R89LhMOoLxoghoS8AVRhmQwjvEO55iHNocrY0p/8Eaa+J5FMWSEZTraQ8ETVa
	D0OLbk1qKMSSt1WsYvUJyuzQjYZI/44=
X-MC-Unique: VQDCdMeePb2Ef9ThNPZJPw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FWHQhGwzj4KoMT06lwzAh5XzsIM9TNlDFbAA8DhaB6un+/NRB/OJN4Dg3vMF/OjWW0pwHJVynlkvaPDyQ1aYELCkbhDxBi6YonIt3a2mvr5YNZl7LBQ2WmTU+cz7XTAHD+FXeQhBS/cIX/P4Im1LBfUF/0P0faOtdYbVthVLZGu0xr/cYWJKHezJ+cU0luCqkS/0Noj77OYF0hP4m/gaEKfTEUjzDXEOvzIx18iN8bUs3ADnTJb3NessQz2H/ceLiCbDvsGbapEcmGYSPO6ihwR51+2Ga1/a6l3s6JTr4wxGvhn2TTdbYHYnOflBoPRZEk/PAeYnOLG59SQLfdqVNw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=IxCuf0pT2Zfq6oHeAx77qfUtNUjzCLWnMNKHXtce4V4=;
 b=epGxvGpR1V4tuog3gXOwbHOzUb23FKjEXXCpvCYrWqiry3yODshrhkSY4hpTkmGEc9Nz5lJI9m16+EBJZa1s4cVnEeQ6g7WYx8s0UcTKa9DhuFSugKyUr57/q1v5Qynbz4Wdd8qK8G9+tpc9jNW0UauVfakwm8dyTJVbvcsTn0tpOpktnsChSVINlbdTmX2AqC/U827p79F9Uv1JUxgRYHvv5mYA7wGLEY/ecZu+d1r86PzyMpQsc+ALENMDWt/8VM5mrRfSYo2sUGykbn4tjJqTNY60bKdSfepyVktvUkZKdcgOyIyvzo04HtBdtWQbdKEMzQibLeAKyTXNfQ1Q+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: citrix.com; dkim=none (message not signed)
 header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
Subject: Re: Xen 4.16 development update; request for regression bug reports
To: iwj@xenproject.org
Cc: committers@xenproject.org, xen-devel@lists.xenproject.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>
References: <24913.54299.590779.955301@mariner.uk.xensource.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <70fe8be0-e3d6-c88a-bb32-d6289dd27ff3@suse.com>
Date: Thu, 30 Sep 2021 14:56:58 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <24913.54299.590779.955301@mariner.uk.xensource.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS9PR06CA0345.eurprd06.prod.outlook.com
 (2603:10a6:20b:466::7) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: a87e3247-1e3c-4bb8-5a1f-08d98411cb0b
X-MS-TrafficTypeDiagnostic: VI1PR04MB4349:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB43494CB79AD7F0CDBF9F4B4BB3AA9@VI1PR04MB4349.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Z9ulkqLGXHjDcDi+lCktnXUg3V/fJRA6tQq+h2NXIvzKbFs//FMEqdJKO24tA8kg4vX5/pZPGlRcl//NCqlXWmJfZMCxr2Iua/EQU5gi1Nt9ASNqJ1kh0MlFzcMtkG4bzRxu8w6tPRDlb9FNMT0Gca5/oujWa3HMdaqFopqXtK6qP3MvYzvMkaGctB58whgMenXUr0UfJbzNc/ivpNTk9lWZLObtBeILNReIDhmbvggtpFDhsnrC2KscJMU4+DKP1ZGgvF7LYH3S8yJnH+VbMXMwZbQvuA1bngAStC9GEeTxYmlMDvXeLEq+9clBY5PGd9h5k47MA6LCx+tJ1ke3+wTN7Dwcs4Ylmpeh+r28inW5PMFZoLPT5QUvBbaIaPBZ8s+WxeVVffmsTwTTn/qzf1T8sqRTQaOakmomQ5QH9JVWS1ld7RJfjDm6arCkuTYAOVJm04zlh9P3L7eoz56jKgEh4ebnx1lfynHzT5inYP/8dHOvFXJYxCbYoum0G29DA1T+7pDKMC0oyXQEe6nOdS1xWLa4eHjYrLNE7YbW+dUG9rzD4oqaow0aOm975yLXBnk6eHrN6QGwNfX+OT/nj/wzwYYTHFAP4uCW1I6psiutnRqr1ng9iJ6RJcnVgko/BJkS+K6nSqo3enZ8MsmAb48RHCsOVbWuppiT1iorrBi7Jy2igFgeWaLILyQtUVvQvtTuKtpD9d+5ZVRTO2Byt+CB8y/+jsKE+pEZIHrLnvE=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(186003)(4326008)(86362001)(31696002)(6916009)(53546011)(8936002)(31686004)(2616005)(15650500001)(36756003)(54906003)(508600001)(66556008)(8676002)(66476007)(6486002)(16576012)(26005)(83380400001)(66946007)(316002)(5660300002)(956004)(2906002)(38100700002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SFhDTUppOWduNVJiMGRzNUFzM3phVndKTjdYaWthY2ZjZU5pcTJOaDV2UXpD?=
 =?utf-8?B?bGREYkd6eWprMERvd0lUeFZIN1pnbnM1bDNKcHZIVVZSNVlnU0dxSHg2NVpF?=
 =?utf-8?B?Wmlib0JJS2tlYkVWMTR2R1B3SHk1eWorOWpFUHVEYi9CVzVXZGdpNkNMeGFz?=
 =?utf-8?B?eXVtUEFIU3NTeVV6akQvZDY5aFo5UzdBeVpGd3Z3UEtRcm5naUNtVzBJb1pm?=
 =?utf-8?B?WitHanl5QWswUmJWRnIyL1lWaWNZdFgrdmpaVElYR1RqRWV3QVpzWmhzTEM5?=
 =?utf-8?B?N3NsdTlGUnlPb0lzc3IrZWt1Z1FtWXZXM2pLb0RlcVNsa3ZNRXEzUzRjTWx3?=
 =?utf-8?B?bkNudWMwOWg3eHhDbDYvazMvR1VuR3dvN1hQS3FkT2tmUlozWEtYYVhiSmJx?=
 =?utf-8?B?V2dDZituTEhFd1h2d0puS1l3d0dIV1gxOGN0d2Vnck5malpyME5HMlhUZGNM?=
 =?utf-8?B?TitYOUZIS3Y1Y3ZndVRVYTNTei81OTlnYzlNemdJWjNBTFNiUiswMFp5NG5E?=
 =?utf-8?B?dDFqMUpYYksxZCtINE9uQlZTSEZ3Q2NrMHpoeWt1Y29MVlJzY3RpSnVDek14?=
 =?utf-8?B?b2Ivb016c0FrTVFHUGQzNUVYQkc0VVBCSDQwVDh1NnZuT3U3clRtOTJVbTI0?=
 =?utf-8?B?UDFKcHhseStCQ0UyVHp5aWk5UHVQSG5qSDRFMEEyNzlZaHh3M1FiVGpEL3pL?=
 =?utf-8?B?S1gwaWFkOWNXOEdyYkZTQU9QRk10TGwyS0YwQlMraytnbVZZQXlheXJMTW9z?=
 =?utf-8?B?eDBuOGpwSUxFMWtwNEtSRWR1Q3JBMEJmUERaNlJHWjBtL2ZteVlJVGd3Sk54?=
 =?utf-8?B?ekNDSVFXOHRPOGpQRUsxd3dySGhLNEQwT3JFQmhuSXRaTjFISFlheWZBb1dQ?=
 =?utf-8?B?UTlQOGtxVm5jZEhJN2dHMU8zVW5ka0w4UU9jc0FadHc1emhvZjkrZG03UWdC?=
 =?utf-8?B?dGw2TnFzS1hKcmg2REVpU0JFaTY0bzRYQmRnVjBQSDBuQUxFRlo0NGN3WWNl?=
 =?utf-8?B?ZnozaTVKYzVxTDlYZGxhUGdyK1dRQVFYQllOaHl0UjVvVFJuYXFXaGFheHU1?=
 =?utf-8?B?b2ZkQy9vL3pEWnRvNHEveWNyYnZTVUE5azVzNTJSTVF1cytjelAxVUJNVkNB?=
 =?utf-8?B?b1k5TGZ5ODk4U2F2WWk0ajlUY1c0Q3pPQzl6K0lCLzR5TThncjBoODlxUmxx?=
 =?utf-8?B?S2JuOU9CTjZ5MXJRMG4rdC9uTngvVUExUVpBdHRWYTAxSEpuUFl2cEVVRkFy?=
 =?utf-8?B?aXgyb1JvbEV4eUcwVDFrOFJEOGowZ1VsRmlUeWIwdVo3UHRoWndCdi9GWmcr?=
 =?utf-8?B?REt0QkU4ODNvM0lUUldNSEFITnhxRXBTOFJaZDlyQjBVNWllSVozVGEyZU5a?=
 =?utf-8?B?UjBhM1Y2Q1FwakdhWkFvNHBnM2lGTUcrdVFnd0NON3pGdTczazVoaDJNRnp4?=
 =?utf-8?B?b051c20wMzl2R1hKZTcweThXMmR4T3BqdjhDd3hrQS9RYXV0cWpRZ3Bjczc5?=
 =?utf-8?B?alp0RUxZOVB0TmxjZ0Fwc281YkE4Nk80T1ZRMDJDcFRYRVlvYkpKREZXRFZH?=
 =?utf-8?B?dlhiN3h5Mkw3anlFLzJISXBsSjBxam5vakQ2NGJVbU1wenJzUHNZa2tMd1U5?=
 =?utf-8?B?V1dSSGo0MURDeFpDZnVTaWVrSXZkNHhDd2owMXlzd0JHQ1RsVW1SbmpDZG5I?=
 =?utf-8?B?eUNEUWRsRENiWUVHNTd0NTBrbVlidUd1MUFiVGtsa3VraXJFMVd1NmxXeFRv?=
 =?utf-8?Q?ijj2qt6nPBlyyFGdy8fWZHhSOUTr0UPNVprHP7K?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a87e3247-1e3c-4bb8-5a1f-08d98411cb0b
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:57:00.7304
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: L4rrTa7vi7vpG+EZ88YYwFCCJDoHDu9jqCOIDIiu2UX0/GHh1AoP1dJ061+WC06H8Rx5eBx786Pmgcmn9IBZcw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4349

On 27.09.2021 16:24, Ian Jackson wrote:
> The mainline development branch of Xen is has taken the first step
> towards stabilisation for 4.16: new features not yet submitted, at
> least in draft form, will now be deferred to the following release.
> 
> We aim to make each Xen release as good as possible, and that includes
> high quality and particularly an absence of serious regressions.
> To help deliver this as Release Manager for Xen 4.16, I need the help
> of the Xen developer and user community
> 
> In particular, please would users and developers make sure that I know
> about:
> 
>  * Regressions in the current development branch (xen-unstable)
>    compared to Xen 4.15;
> 
>  * Major or important work which is targeted for 4.16, but at risk
>    of missing the deadlines.

Effectively all of the series that have been stuck for a more or less
long periods of time were originally intended for 4.16:

- xvmalloc() / x86 xstate area / x86 CPUID / AMX+XFD
- x86: memcpy() / memset() (non-)ERMS flavors plus fallout
- x86/mm: large parts of P2M code and struct p2m_domain are HVM-only
- x86emul: a few small steps towards disintegration
- x86: more or less log-dirty related improvements
- IOMMU: superpage support when not sharing pagetables
- AMD/IOMMU: further work split from XSA-378
- gnttab: further work from XSA-380 / -382 context
- x86/mwait-idle: updates from Linux
- x86/boot: (mostly) video mode handling adjustments

There are further individual patches, which I've omitted. The only one
that I think (or should I say hope) is on track is "x86/PVH: Dom0
building adjustments".

It would of course also be nice for Anthony's "xen: Build system
improvements, now with out-of-tree build!" to go in, but it's hard to
predict whether the remaining almost 40 patches can get reviewed in
time. I've been meaning to get through with this (and others), but
other stuff keeps popping up.

Additionally there is Marek's "x86/xstate: reset cached register values
on resume", which Andrew has been meaning to supply a superset patch for.
He has hence been objecting to this patch going in, but seeing that 1.5
months have passed now I think this wants reconsidering.

Similarly there is Roger's even older "x86/viridian: EOI MSR should
always happen in affected vCPU context".

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 13:07:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 13:07:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199936.354274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvmv-0003BY-Sa; Thu, 30 Sep 2021 13:07:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199936.354274; Thu, 30 Sep 2021 13:07:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVvmv-0003BR-Pp; Thu, 30 Sep 2021 13:07:49 +0000
Received: by outflank-mailman (input) for mailman id 199936;
 Thu, 30 Sep 2021 13:07:48 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wsc4=OU=suse.com=jgross@srs-us1.protection.inumbo.net>)
 id 1mVvmu-0003BL-2R
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 13:07:48 +0000
Received: from smtp-out2.suse.de (unknown [195.135.220.29])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 7b8af45d-1aad-4937-8842-7a374c6c342d;
 Thu, 30 Sep 2021 13:07:47 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6B654203BB;
 Thu, 30 Sep 2021 13:07:46 +0000 (UTC)
Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512)
 (No client certificate requested)
 by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DEA2613AF5;
 Thu, 30 Sep 2021 13:07:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([192.168.254.65])
 by imap2.suse-dmz.suse.de with ESMTPSA id DesOM6G2VWG1VAAAMHmgww
 (envelope-from <jgross@suse.com>); Thu, 30 Sep 2021 13:07:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b8af45d-1aad-4937-8842-7a374c6c342d
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1633007266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=i1SW8NPZGg1GriXI8CfQXVexUlt3KJF4BcMKC51Kwak=;
	b=LPlYtfdJpzZoWGEb9OpP3I+Ro64B6sDmaWTwNzu+WURecbGDciVR+wilIgJJT0FcNfWA0b
	QkqhidCcc22ktXKB6zBhOlcDjxvHZJaLkGBvb0gyZp/XHrLod0MDWsz/Pjenue+MJHQSlC
	6USfxJ0g+eShB767YAxj/m6iTzNMHzI=
Subject: Re: [PATCH v2 09/10] xen-blkfront: add error handling support for
 add_disk()
To: Luis Chamberlain <mcgrof@kernel.org>, axboe@kernel.dk, colyli@suse.de,
 kent.overstreet@gmail.com, kbusch@kernel.org, sagi@grimberg.me,
 vishal.l.verma@intel.com, dan.j.williams@intel.com, dave.jiang@intel.com,
 ira.weiny@intel.com, konrad.wilk@oracle.com, roger.pau@citrix.com,
 boris.ostrovsky@oracle.com, sstabellini@kernel.org, minchan@kernel.org,
 ngupta@vflare.org, senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org, nvdimm@lists.linux.dev,
 linux-nvme@lists.infradead.org, linux-bcache@vger.kernel.org,
 linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20210927220039.1064193-1-mcgrof@kernel.org>
 <20210927220039.1064193-10-mcgrof@kernel.org>
From: Juergen Gross <jgross@suse.com>
Message-ID: <50f5fcbe-fb34-1cb8-1965-dd3bfd7e1f12@suse.com>
Date: Thu, 30 Sep 2021 15:07:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <20210927220039.1064193-10-mcgrof@kernel.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="O5jaGfuKPjp2dmoP5IHIhPbF1qqBfXK7C"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--O5jaGfuKPjp2dmoP5IHIhPbF1qqBfXK7C
Content-Type: multipart/mixed; boundary="ZhXoHV0IZVGJD3tZd4HDqm50wpYE06BGa";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Luis Chamberlain <mcgrof@kernel.org>, axboe@kernel.dk, colyli@suse.de,
 kent.overstreet@gmail.com, kbusch@kernel.org, sagi@grimberg.me,
 vishal.l.verma@intel.com, dan.j.williams@intel.com, dave.jiang@intel.com,
 ira.weiny@intel.com, konrad.wilk@oracle.com, roger.pau@citrix.com,
 boris.ostrovsky@oracle.com, sstabellini@kernel.org, minchan@kernel.org,
 ngupta@vflare.org, senozhatsky@chromium.org
Cc: xen-devel@lists.xenproject.org, nvdimm@lists.linux.dev,
 linux-nvme@lists.infradead.org, linux-bcache@vger.kernel.org,
 linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
Message-ID: <50f5fcbe-fb34-1cb8-1965-dd3bfd7e1f12@suse.com>
Subject: Re: [PATCH v2 09/10] xen-blkfront: add error handling support for
 add_disk()
References: <20210927220039.1064193-1-mcgrof@kernel.org>
 <20210927220039.1064193-10-mcgrof@kernel.org>
In-Reply-To: <20210927220039.1064193-10-mcgrof@kernel.org>

--ZhXoHV0IZVGJD3tZd4HDqm50wpYE06BGa
Content-Type: multipart/mixed;
 boundary="------------657B867CC90834C60D5C497D"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------657B867CC90834C60D5C497D
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.09.21 00:00, Luis Chamberlain wrote:
> We never checked for errors on device_add_disk() as this function
> returned void. Now that this is fixed, use the shiny new error
> handling. The function xlvbd_alloc_gendisk() typically does the
> unwinding on error on allocating the disk and creating the tag,
> but since all that error handling was stuffed inside
> xlvbd_alloc_gendisk() we must repeat the tag free'ing as well.
>=20
> We set the info->rq to NULL to ensure blkif_free() doesn't crash
> on blk_mq_stop_hw_queues() on device_add_disk() error as the queue
> will be long gone by then.
>=20
> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen


--------------657B867CC90834C60D5C497D
Content-Type: application/pgp-keys;
 name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Transfer-Encoding: quoted-printable
Content-Description: OpenPGP public key
Content-Disposition: attachment;
 filename="OpenPGP_0xB0DE9DD628BF132F.asc"

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOBy=
cWx
w3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJvedYm8O=
f8Z
d621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y=
9bf
IhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xq=
G7/
377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR=
3Jv
c3MgPGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsEFgIDA=
QIe
AQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4FUGNQH2lvWAUy+dnyT=
hpw
dtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3TyevpB0CA3dbBQp0OW0fgCetToGIQrg0=
MbD
1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbv=
oPH
Z8SlM4KWm8rG+lIkGurqqu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v=
5QL
+qHI3EIPtyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVyZ=
2Vu
IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJCAcDAgEGFQgCC=
QoL
BBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4RF7HoZhPVPogNVbC4YA6lW7Dr=
Wf0
teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz78X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC=
/nu
AFVGy+67q2DH8As3KPu0344TBDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0Lh=
ITT
d9jLzdDad1pQSToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLm=
XBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkMnQfvUewRz=
80h
SnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMBAgAjBQJTjHDXAhsDBwsJC=
AcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJn=
FOX
gMLdBQgBlVPO3/D9R8LtF9DBAFPNhlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1=
jnD
kfJZr6jrbjgyoZHiw/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0=
N51
N5JfVRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwPOoE+l=
otu
fe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK/1xMI3/+8jbO0tsn1=
tqS
EUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuZGU+wsB5BBMBAgAjBQJTjHDrA=
hsD
BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3=
g3O
ZUEBmDHVVbqMtzwlmNC4k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5=
dM7
wRqzgJpJwK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu5=
D+j
LRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzBTNh30FVKK1Evm=
V2x
AKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37IoN1EblHI//x/e2AaIHpzK5h88N=
Eaw
QsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpW=
nHI
s98ndPUDpnoxWQugJ6MpMncr0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZR=
wgn
BC5mVM6JjQ5xDk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNV=
bVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mmwe0icXKLk=
pEd
IXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0Iv3OOImwTEe4co3c1mwARA=
QAB
wsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMvQ/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEw=
Tbe
8YFsw2V/Buv6Z4Mysln3nQK5ZadD534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1=
vJz
Q1fOU8lYFpZXTXIHb+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8=
VGi
wXvTyJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqcsuylW=
svi
uGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5BjR/i1DG86lem3iBDX=
zXs
ZDn8R38=3D
=3D2wuH
-----END PGP PUBLIC KEY BLOCK-----

--------------657B867CC90834C60D5C497D--

--ZhXoHV0IZVGJD3tZd4HDqm50wpYE06BGa--

--O5jaGfuKPjp2dmoP5IHIhPbF1qqBfXK7C
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmFVtqAFAwAAAAAACgkQsN6d1ii/Ey9/
Ggf9Fzrs35IrfdUjKBrgggetJDMgl6I0Cc/n6Jtov78ZI8feS38DFu/MBlJr7yJi7nDb927EgLkY
Y12gOa0cyy+IViB/FsDGnzneZefxCV8ddwXHxUCK087pTn/rAExcemxNE6AQuP4c+Sg8SCgYbLuq
VMaH5zOY5g4WAv7oC5wxvUNRJDun8GXGkc3nrpqjmmK1jlQ3eYGUIuHk4I7aD4Ms75pVOywkbpOg
C9JTmp+fz8MknREAfAwX9343sfEr0+yceuLOx508VcF7asOTuQe78cbszDRUqWjUKvBniise0zdL
XkmSEsvS4U6iJZ0ZvU8D1pH443giDVPA6cWx9XIbQg==
=r4+e
-----END PGP SIGNATURE-----

--O5jaGfuKPjp2dmoP5IHIhPbF1qqBfXK7C--


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 13:49:51 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 13:49:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199947.354286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVwRC-0007kX-3r; Thu, 30 Sep 2021 13:49:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199947.354286; Thu, 30 Sep 2021 13:49:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVwRC-0007kQ-0e; Thu, 30 Sep 2021 13:49:26 +0000
Received: by outflank-mailman (input) for mailman id 199947;
 Thu, 30 Sep 2021 13:49:24 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BYNK=OU=linux.ibm.com=fbarrat@srs-us1.protection.inumbo.net>)
 id 1mVwRA-0007kK-5e
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 13:49:24 +0000
Received: from mx0a-001b2d01.pphosted.com (unknown [148.163.158.5])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 37347a42-21f5-11ec-bd4c-12813bfff9fa;
 Thu, 30 Sep 2021 13:49:21 +0000 (UTC)
Received: from pps.filterd (m0098420.ppops.net [127.0.0.1])
 by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18UCBxYt019958; 
 Thu, 30 Sep 2021 09:48:44 -0400
Received: from pps.reinject (localhost [127.0.0.1])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3bdd18tfqq-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 09:48:44 -0400
Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1])
 by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18UDMbpq025093;
 Thu, 30 Sep 2021 09:48:43 -0400
Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com
 [169.51.49.102])
 by mx0b-001b2d01.pphosted.com with ESMTP id 3bdd18tfq0-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 09:48:43 -0400
Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1])
 by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18UDlI4r026910;
 Thu, 30 Sep 2021 13:48:41 GMT
Received: from b06cxnps3074.portsmouth.uk.ibm.com
 (d06relay09.portsmouth.uk.ibm.com [9.149.109.194])
 by ppma06ams.nl.ibm.com with ESMTP id 3b9u1k4bx8-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 13:48:41 +0000
Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com
 [9.149.105.59])
 by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
 18UDmc3x46334294
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
 Thu, 30 Sep 2021 13:48:38 GMT
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id C8532A4040;
 Thu, 30 Sep 2021 13:48:38 +0000 (GMT)
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
 by IMSVA (Postfix) with ESMTP id BA81EA405D;
 Thu, 30 Sep 2021 13:48:37 +0000 (GMT)
Received: from [9.145.153.93] (unknown [9.145.153.93])
 by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP;
 Thu, 30 Sep 2021 13:48:37 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 37347a42-21f5-11ec-bd4c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date :
 mime-version : subject : to : cc : references : from : in-reply-to :
 content-type : content-transfer-encoding; s=pp1;
 bh=Q+AzT0Ijwdw7ICtovDN1my42gJlkON+amsvj9v8ooiM=;
 b=lQgzyUqXVkfGYfIt0Ch8RaKtlmIO6gizazSCeTsQzOdqLwcuzmMl4mFsZAnyWbwIp4FP
 6DPoyR44kXJJZ7lmHh9ozPD83SUDnbwObYunomLIOYbRNofigHr7W9owQJVPIP2u5PDl
 scdMAbKBB/ZsfyiQfRjbBG3urvEypmfqvptUmDx86XfEU90JuutbQpCroOqtimmJ09Gh
 ViMuqNernMb0exN9vnb0YA6w6iPkNqWWei+kMGtd6YDcHT8Ydk3Yta5cyWRODcb4ZK7k
 RURHAnrkOzxAIeIYtMoDQcv/ZnZ+yEWpBxNEfXrPaFCZxPwzSjZH+em+PjODNqawGzU7 8w== 
Message-ID: <e53a8113-5037-db2a-7313-d1ef5e430682@linux.ibm.com>
Date: Thu, 30 Sep 2021 15:48:37 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.1.0
Subject: Re: [PATCH v5 10/11] PCI: Replace pci_dev::driver usage by
 pci_dev::dev.driver
Content-Language: en-US
To: Andrew Donnellan <ajd@linux.ibm.com>,
        =?UTF-8?Q?Uwe_Kleine-K=c3=b6nig?= <u.kleine-koenig@pengutronix.de>
Cc: Bjorn Helgaas <helgaas@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        "Oliver O'Halloran" <oohall@gmail.com>,
        Russell Currey <ruscur@russell.cc>, Jiri Olsa <jolsa@redhat.com>,
        Christoph Hellwig <hch@lst.de>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Mathias Nyman <mathias.nyman@intel.com>,
        Michael Ellerman <mpe@ellerman.id.au>, x86@kernel.org,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Ingo Molnar <mingo@redhat.com>, linux-pci@vger.kernel.org,
        xen-devel@lists.xenproject.org, Juergen Gross <jgross@suse.com>,
        Arnd Bergmann <arnd@arndb.de>,
        Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Borislav Petkov <bp@alien8.de>, Bjorn Helgaas <bhelgaas@google.com>,
        Namhyung Kim <namhyung@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        linux-usb@vger.kernel.org, linux-perf-users@vger.kernel.org,
        kernel@pengutronix.de, Paul Mackerras <paulus@samba.org>,
        linuxppc-dev@lists.ozlabs.org
References: <20210929085306.2203850-1-u.kleine-koenig@pengutronix.de>
 <20210929085306.2203850-11-u.kleine-koenig@pengutronix.de>
 <75dd6d60-08b9-fa68-352c-3a0c5a04c0ab@linux.ibm.com>
 <20210929134330.e5c57t7mtwu5iner@pengutronix.de>
 <1414b3c5-167c-c271-baed-d3d7f6cd0309@linux.ibm.com>
From: Frederic Barrat <fbarrat@linux.ibm.com>
In-Reply-To: <1414b3c5-167c-c271-baed-d3d7f6cd0309@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: ydDkSS-JwggQoWGJmx2KVo1ai4eV3eaN
X-Proofpoint-ORIG-GUID: _H8ui1Jc9Hf375ilmtd2tw6D1AxPAzql
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_04,2021-09-30_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0
 lowpriorityscore=0 bulkscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999
 priorityscore=1501 adultscore=0 clxscore=1011 spamscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109300085



On 29/09/2021 17:44, Andrew Donnellan wrote:
> On 29/9/21 11:43 pm, Uwe Kleine-König wrote:> I'm not a huge fan either, 
> I used it to keep the control flow as is and
>> without introducing several calls to to_pci_driver.
>>
>> The whole code looks as follows:
>>
>>     list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
>>         struct pci_driver *afu_drv;
>>         if (afu_dev->dev.driver &&
>>             (afu_drv = 
>> to_pci_driver(afu_dev->dev.driver))->err_handler &&
>>             afu_drv->err_handler->resume)
>>             afu_drv->err_handler->resume(afu_dev);
>>     }
>>
>> Without assignment in the if it could look as follows:
>>
>>     list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) {
>>         struct pci_driver *afu_drv;
>>
>>         if (!afu_dev->dev.driver)
>>             continue;
>>
>>         afu_drv = to_pci_driver(afu_dev->dev.driver));
>>
>>         if (afu_drv->err_handler && afu_drv->err_handler->resume)
>>             afu_drv->err_handler->resume(afu_dev);
>>     }
>>
>> Fine for me.
> 
> This looks fine.
> 
> As an aside while writing my email I discovered the existence of 
> container_of_safe(), a version of container_of() that handles the null 
> and err ptr cases... if to_pci_driver() used that, the null check in the 
> caller could be moved until after the to_pci_driver() call which would 
> be neater.
> 
> But then, grep tells me that container_of_safe() is used precisely zero 
> times in the entire tree. Interesting.
> 
>> (Sidenote: What happens if the device is unbound directly after the
>> check for afu_dev->dev.driver? This is a problem the old code had, too
>> (assuming it is a real problem, didn't check deeply).)
> 
> Looking at any of the cxl PCI error handling paths brings back 
> nightmares from a few years ago... Fred: I wonder if we need to add a 
> lock here?

Yes, it's indeed a potential issue, there's nothing to prevent the afu 
driver to unbind during that window. Sigh..

   Fred



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:23:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:23:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199959.354297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVwxa-0003pk-KQ; Thu, 30 Sep 2021 14:22:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199959.354297; Thu, 30 Sep 2021 14:22:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVwxa-0003pd-Gt; Thu, 30 Sep 2021 14:22:54 +0000
Received: by outflank-mailman (input) for mailman id 199959;
 Thu, 30 Sep 2021 14:22:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVwxa-0003pX-7Y
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:22:54 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id e582479d-21f9-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:22:53 +0000 (UTC)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-28-GY67x6N2OqCYqeVV4kb7Vg-1; Thu, 30 Sep 2021 16:22:50 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB4943.eurprd04.prod.outlook.com (2603:10a6:803:51::21)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 14:22:49 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 14:22:49 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P192CA0032.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::45) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 14:22:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e582479d-21f9-11ec-bd4e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633011772;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=J8nakvMkRLnc7FFQUjjQiIHAtqM9aF1YkbrbFKHyJKI=;
	b=iKXvGGz/OhIY/5nf3g+MdT4HYJD65dbe2i8z7bJcSMexHvlpKY90CuKCI5Wc9rmIRx/d9S
	Yltn4yvoKExiRKE27N8DywhiZ9X0ScSN8HgOF2SRGTKhNtg2TiRuE6zoXqQ29rvsRrnT7T
	EV4LxPXs4MtlRhiUlgqpDHb37DcyaA4=
X-MC-Unique: GY67x6N2OqCYqeVV4kb7Vg-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QvTDl67UkcmD2uGX2lzzT9eGyXRASCOimsNGw77Wu0iGILg5PwsM4fiNRl0TQ+IfVrqFuERIwq+VKw4WuAPmfnQpxnI0qgkxzZUTcag6qAKH2Js0kFhTtWRMjb538mnKMVRo1mnCDdQlY1F8/rCLp2T7MamSpQzpQ3IMd2h2FYv+Hb7F7GIcY7dc3MLeOUZU+4CSgQW9KauX7DUHRpWYdZV6tJ/nM1P2UF/u+G+A7jegU4dlpMW5A3dbCAcUysRriedpglzIU36R1WBJ+1RRhUg4xclPf4Lhg7vVeuv7QNpf7ppGQatemtDoqQ0Cg7gKywvz5LvFf2J7nglWXE+oxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=J8nakvMkRLnc7FFQUjjQiIHAtqM9aF1YkbrbFKHyJKI=;
 b=FVn8mKe6edXl4nwDGEd/cXZhsTqIgFBR++qx+UdOh1ry5W3u/LOm+wgezStMXdMD4pxK0Bk7z/b/g+M3mMlSd6NcrJKZLhkwq1DnuRjLGxVDFYezknd5lXCTWwPrLSiRrRrloS1rAxIPOguDMiS1+rspll3LQbdKt/vTseJbGO3aizNzLqrtP53aSi689jMRAYDRF1S4GP8j35iP7S+WiVYnSUQKggbyvmv2c6nij+LqCiHyhU+VkAFxjPOABm8SFORMCtJc4uikE76bnNsjDQT1e8Ng66mSFfDyEWHyCEsKjazD8lNFfWB1K+FSZVTQ2437vasWaocbjExzoVuizg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: apertussolutions.com; dkim=none (message not signed)
 header.d=none;apertussolutions.com; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 01/17] xen/pci: Refactor MSI code that implements MSI
 functionality within XEN
To: Stefano Stabellini <sstabellini@kernel.org>,
 Rahul Singh <rahul.singh@arm.com>
Cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com,
 Andre.Przywara@arm.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>, Julien Grall <julien@xen.org>,
 Paul Durrant <paul@xen.org>, Daniel De Graaf <dgdegra@tycho.nsa.gov>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <cover.1632847120.git.rahul.singh@arm.com>
 <db3af270e884838cbde7c29c366740bb3f628725.1632847120.git.rahul.singh@arm.com>
 <alpine.DEB.2.21.2109290956180.5022@sstabellini-ThinkPad-T480s>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <417ffe2a-0d79-99aa-eaaa-1d1715a3e0f3@suse.com>
Date: Thu, 30 Sep 2021 16:22:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <alpine.DEB.2.21.2109290956180.5022@sstabellini-ThinkPad-T480s>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6P192CA0032.EURP192.PROD.OUTLOOK.COM
 (2603:10a6:209:83::45) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: c7fa4e9f-f723-41bf-2ee1-08d9841dc7ca
X-MS-TrafficTypeDiagnostic: VI1PR04MB4943:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB49435427CE52ECE3C3C66418B3AA9@VI1PR04MB4943.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	VaFI9byU6RXECVqce7TN+P+xvC4qUVBy9lUg2zPtM0CgL+1OsAqNt2zNq/MiLmE4SBRLK6e73BGhUncSsSJXaj2mJCO+skHtWgzWrIFOtYCaDs8L3Z5oitrQUq74uvCD4tsllhXeQOkAfEiCvEP7o2deLrC3Q1cU2qUeYrJ46bl0fTZMvvjVxyoYLQRTR0Z7at9O1iEtO7dqnU68N+GkvvwWIFkEwPxyzYXz/4y9YAP6g4mTKApCOT54i4OypcoYrDoYqJdpBCreuHTGumQnFSfekgL0AX354EmxRKKahFkq/GJqLLhyO4Sweg1wdab2nVxZaxmZ6vdKKATHVF4Pt1pM2jpl818u1cxtL/aZ0Z0uKdE9815OTZ5A9hByOB9VSa0puo6qPcap7f1jdH4LtwswRClDlSPJub7G31IhYgSBND1l9IOrUs/FEFVxdUend+VYceLAQ+hL2wRHDxtaG7pwm8ubDxJvPQPoMY4aCmbxPpWjnSIoPVTeDSLrI8nblFNYs47YhiMspNDOCq9WSJWcbS61YQ3gWxa/+uB6UwA09PSma0NXVos++yTAMLIhCSPCKKF4bscZ3q+0bjQoVpOw+g4+R4VyMlLYa7BkKD6+XYnCTIosj6pvR2GPD9XorSfeHCZbjyhku1TpeIK+dHap9QeVHaWtCxaORz9gw0peEJDe4QNPX2W26NLkE5WovOkXlzvoRxdOGSIO6ogg4Qb6zzPrbdC8J3jKr2NeJdQ=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(66476007)(66556008)(66946007)(7416002)(2906002)(5660300002)(4744005)(6486002)(36756003)(38100700002)(508600001)(26005)(4326008)(956004)(2616005)(31686004)(86362001)(186003)(31696002)(53546011)(16576012)(54906003)(316002)(110136005)(8676002)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OUNBSktvMENIOUY1cjkvSTEwS1dHMlZZQklkRmZPUkNIS3RRVWl6Y2hWb0Ir?=
 =?utf-8?B?ektleXpCbEY1ellFMytpVEl3QUZWUGxndm5SRkdxREFSWWJrYnNqWlp4UWpW?=
 =?utf-8?B?djM4STdGcjNMUWgzZ0k5MmltRjJId0hhWHhOL21vQmJoZHB2TDd5SUNNUkJ1?=
 =?utf-8?B?SnJybHZkV1VsN2FKeGNkTGNVUS9MQ1lNVEx3aVl6R0RFekM5cXBWUkY4OVdo?=
 =?utf-8?B?WHpkR2cvMngvT1hhL3ozRzdPK3FGTXZqcVROS1dyTmxMV010d0xicGRnT0Y2?=
 =?utf-8?B?ZzdNdHJ6V3IrTzRNNXU2Z3VrK2Ruc0o1ZTM2bXN0RkF1Unp5blFxWkJvVGds?=
 =?utf-8?B?c1liQlEyeEQyVWRrcGNGRTIwSDZ0bWt0dXZTaXlzcUVFaGw1bXJkdXJGbW5o?=
 =?utf-8?B?VXd6eVQvNWVRVVBaeSthTHJjendFUFROZ2ZRUHd4RmlGUW43UHhid09vQ2VI?=
 =?utf-8?B?Q0JWME9pYnJsak5EanRvR1B4aVlwTG84SmRFdVcrSGxGeStScFMrZUxjU3kz?=
 =?utf-8?B?Nm1xU050bjJCWjRKWjRzcVNYb3ZjRHVxSHlwS2VoQllPdGJxQWZHek40U1c1?=
 =?utf-8?B?cGtPNXg5QWs0RkJtOTcrNWRKSEkwT25HeXFsTVdzUnFDa2NtMCtVYUYzdW15?=
 =?utf-8?B?emF0djFtWmg1U1phUEZxdzFxMHdXVTJoUVRyTjAzb3RCM3g2LzYwWWN6LzhJ?=
 =?utf-8?B?aE1CdVVUUGdFUHg2WlFrSnFWY2NCVmVUWnhzRDY3eFNOWkJ4NFRZRGtmaEpv?=
 =?utf-8?B?QTNKU1hvaG40a1pKRVVZNm9ULzg0V1JEWGFoMjh1Z1dNTlFGdjFJbDBuU2d6?=
 =?utf-8?B?Q0gxWUNsZ1owc0hWVllWeUhhMEtHc2pSd0R1S05QSDZMbHBiV1pIMEM3enBm?=
 =?utf-8?B?b09uUjhRVkNZbFo4YXdBeTJZQmVZM0IwT203S0RwbmZ4WlFpR21OemUveXVK?=
 =?utf-8?B?aUM0dkYyK0ZIblhUSVZqTG1pOEtwM1lPQ1JCNkFGaDk4SlBPWFZwZ1RSN2Yy?=
 =?utf-8?B?cU1nNzNMZlRJNVFuRmVuYUFBcFNWTlI5VHhCNjZ4TUpmcU91UjRlNlZ3TkFB?=
 =?utf-8?B?VzQ0VnlKWERzUzJiVVJPQmFzZ29JZVNOcHB0aGtVbDlMbjcvSnZiWW9ONXFm?=
 =?utf-8?B?UHY2cUVQTzg1VDNOZjIwakVXSWNTcndEbDMrUTBvUGZHQVpzWlZhU254WWN0?=
 =?utf-8?B?VTM3WVFyK24xekYvQk9QRmQwQko4bUhwUGhyUS9iMlRmSlVWUjFwQUJrT1lv?=
 =?utf-8?B?aTNjN3ZrZFE0K1BOUU0xa0kxbm9NaHNad2ZPRVBYTXZWNE1XWXNVVjdEeFpB?=
 =?utf-8?B?bVg0Wm9xbzJZOWNKYTM1RytwMU8rMitYTVNHUFZIRGg2UjVGeEQzbW9STmlz?=
 =?utf-8?B?NjZjc2dLMjVIQk5tNVNpSXl6N0V4eGt2eXFvRE9JYkxVdy93L2k2TDV6T2o0?=
 =?utf-8?B?RVl6am4wL1p3bjY1NkhUejAwd0EyRUppMnk1a1RTZm5yaTQzUGVUcEF1THZM?=
 =?utf-8?B?blhpY0NvcDh5VitLaFNXcUFZTGZCbkdyanRVTnlGNkRNS2hKc2ZQeFN4VHRt?=
 =?utf-8?B?UVFDVnBkNFBkakV1YjRTdXAwK01STGdGazZla0dKZ3JjTEJaZVAyM21ackRi?=
 =?utf-8?B?RGhPa1N5cEN4bXhPYXR5MDFGMVMxVFcraUV4MUNIZ1BzbUZUODRUNjc4cEJK?=
 =?utf-8?B?M3dvL2dYK1poSXdncmxISUlhVm5yU1dCLzdsUUU3TjVjVGVpRjFhYldDQUlG?=
 =?utf-8?Q?o5SdIboyufl8DFIWGCEOj6VJxHYXZtTYGugds1o?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c7fa4e9f-f723-41bf-2ee1-08d9841dc7ca
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:22:49.1829
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LWJsPLoUP7eDX7N5DkKxtdOps1ndEXaovy6RerNo9KTtVLAu586pl140pulbTAI1oIUfATguAK5P9ut5KHFI5w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4943

On 29.09.2021 18:56, Stefano Stabellini wrote:
> On Tue, 28 Sep 2021, Rahul Singh wrote:
>> On Arm, the initial plan is to only support GICv3 ITS which doesn't
>> require us to manage the MSIs because the HW will protect against
>> spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
>> for ARM.
>>
>> No functional change intended.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Acked-by: Jan Beulich <jbeulich@suse.com>

Nevertheless I'd like to express that ...

>> --- a/xen/drivers/pci/Kconfig
>> +++ b/xen/drivers/pci/Kconfig
>> @@ -1,3 +1,7 @@
>>  
>>  config HAS_PCI
>>  	bool
>> +
>> +config HAS_PCI_MSI
>> +	bool
>> +	depends on HAS_PCI

... personally I think this should be "select HAS_PCI", and the then
redundant one in x86 might then want dropping.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:29:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:29:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199966.354308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3T-0004fH-Ab; Thu, 30 Sep 2021 14:28:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199966.354308; Thu, 30 Sep 2021 14:28:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3T-0004fA-7L; Thu, 30 Sep 2021 14:28:59 +0000
Received: by outflank-mailman (input) for mailman id 199966;
 Thu, 30 Sep 2021 14:28:58 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVx3S-0004f4-5U
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:28:58 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id eafbac8d-1b5b-468a-86b5-7a6d6cd63ad6;
 Thu, 30 Sep 2021 14:28:56 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B8BFD101E;
 Thu, 30 Sep 2021 07:28:55 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.195.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4C9E33F70D;
 Thu, 30 Sep 2021 07:28:54 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eafbac8d-1b5b-468a-86b5-7a6d6cd63ad6
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 0/3] arm/efi: Add dom0less support to UEFI boot
Date: Thu, 30 Sep 2021 15:28:43 +0100
Message-Id: <20210930142846.13348-1-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1

This serie introduces a way to start a dom0less setup when Xen is booting as EFI
application.
Using the device tree it's now possible to fetch from the disk and load in
memory all the modules needed to start any domU defined in the DT.
Dom0less for now is supported only by the arm architecture.

Luca Fancellu (3):
  arm/efi: Introduce xen,uefi-cfg-load DT property
  arm/efi: Use dom0less configuration when using EFI boot
  arm/efi: load dom0 modules from DT using UEFI

 docs/misc/arm/device-tree/booting.txt |  37 +++
 docs/misc/efi.pandoc                  | 263 ++++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 372 +++++++++++++++++++++++++-
 xen/common/efi/boot.c                 |  62 +++--
 4 files changed, 709 insertions(+), 25 deletions(-)

-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:29:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:29:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199967.354319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3Y-0004xQ-MT; Thu, 30 Sep 2021 14:29:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199967.354319; Thu, 30 Sep 2021 14:29:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3Y-0004xF-Ig; Thu, 30 Sep 2021 14:29:04 +0000
Received: by outflank-mailman (input) for mailman id 199967;
 Thu, 30 Sep 2021 14:29:03 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVx3W-0004f4-Vb
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:29:03 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTP
 id 138cb30a-50fd-4426-a5a2-b52cb5b73a10;
 Thu, 30 Sep 2021 14:28:57 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 615E8113E;
 Thu, 30 Sep 2021 07:28:57 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.195.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EB7113F70D;
 Thu, 30 Sep 2021 07:28:55 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 138cb30a-50fd-4426-a5a2-b52cb5b73a10
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 1/3] arm/efi: Introduce xen,uefi-cfg-load DT property
Date: Thu, 30 Sep 2021 15:28:44 +0100
Message-Id: <20210930142846.13348-2-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210930142846.13348-1-luca.fancellu@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>

Introduce the xen,uefi-cfg-load DT property of /chosen
node for ARM whose presence decide whether to force
the load of the UEFI Xen configuration file.

The logic is that if any multiboot,module is found in
the DT, then the xen,uefi-cfg-load property is used to see
if the UEFI Xen configuration file is needed.

Modify a comment in efi_arch_use_config_file, removing
the part that states "dom0 required" because it's not
true anymore with this commit.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
v4 changes:
- modify property name to xen,uefi-cfg-load
v3 changes:
- add documentation to misc/arm/device-tree/booting.txt
- Modified variable name and logic from skip_cfg_file to
load_cfg_file
- Add in the commit message that I'm modifying a comment.
v2 changes:
- Introduced uefi,cfg-load property
- Add documentation about the property
---
 docs/misc/arm/device-tree/booting.txt |  8 ++++++++
 docs/misc/efi.pandoc                  |  2 ++
 xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
 3 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 44cd9e1a9a..352b0ec43a 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
 for Dom0 and bootargs for native Linux.
 
 
+UEFI boot and DT
+================
+
+When Xen is booted using UEFI, it doesn't read the configuration file if any
+multiboot module is specified. To force Xen to load the configuration file, the
+boolean property xen,uefi-cfg-load must be declared in the /chosen node.
+
+
 Creating Multiple Domains directly from Xen
 ===========================================
 
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ac3cd58cae..ed85351541 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -14,6 +14,8 @@ loaded the modules and describes them in the device tree provided to Xen.  If a
 bootloader provides a device tree containing modules then any configuration
 files are ignored, and the bootloader is responsible for populating all
 relevant device tree nodes.
+The property "xen,uefi-cfg-load" can be specified in the /chosen node to force
+Xen to load the configuration file even if multiboot modules are found.
 
 Once built, `make install-xen` will place the resulting binary directly into
 the EFI boot partition, provided `EFI_VENDOR` is set in the environment (and
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index cf9c37153f..a3e46453d4 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
 
 static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
 {
+    bool load_cfg_file = true;
     /*
      * For arm, we may get a device tree from GRUB (or other bootloader)
      * that contains modules that have already been loaded into memory.  In
-     * this case, we do not use a configuration file, and rely on the
-     * bootloader to have loaded all required modules and appropriate
-     * options.
+     * this case, we search for the property xen,uefi-cfg-load in the /chosen
+     * node to decide whether to skip the UEFI Xen configuration file or not.
      */
 
     fdt = lookup_fdt_config_table(SystemTable);
     dtbfile.ptr = fdt;
     dtbfile.need_to_free = false; /* Config table memory can't be freed. */
-    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") < 0 )
+
+    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
+    {
+        /* Locate chosen node */
+        int node = fdt_subnode_offset(fdt, 0, "chosen");
+        const void *cfg_load_prop;
+        int cfg_load_len;
+
+        if ( node > 0 )
+        {
+            /* Check if xen,uefi-cfg-load property exists */
+            cfg_load_prop = fdt_getprop(fdt, node, "xen,uefi-cfg-load",
+                                        &cfg_load_len);
+            if ( !cfg_load_prop )
+                load_cfg_file = false;
+        }
+    }
+
+    if ( !fdt || load_cfg_file )
     {
         /*
          * We either have no FDT, or one without modules, so we must have a
-         * Xen EFI configuration file to specify modules.  (dom0 required)
+         * Xen EFI configuration file to specify modules.
          */
         return true;
     }
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:29:05 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199968.354323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3Z-00050n-0K; Thu, 30 Sep 2021 14:29:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199968.354323; Thu, 30 Sep 2021 14:29:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3Y-00050J-Rl; Thu, 30 Sep 2021 14:29:04 +0000
Received: by outflank-mailman (input) for mailman id 199968;
 Thu, 30 Sep 2021 14:29:03 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVx3X-0004wW-5Z
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:29:03 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id c066a1be-21fa-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:28:59 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2E788101E;
 Thu, 30 Sep 2021 07:28:59 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.195.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 95AB93F70D;
 Thu, 30 Sep 2021 07:28:57 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c066a1be-21fa-11ec-bd4e-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 2/3] arm/efi: Use dom0less configuration when using EFI boot
Date: Thu, 30 Sep 2021 15:28:45 +0100
Message-Id: <20210930142846.13348-3-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210930142846.13348-1-luca.fancellu@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>

This patch introduces the support for dom0less configuration
when using UEFI boot on ARM, it permits the EFI boot to
continue if no dom0 kernel is specified but at least one domU
is found.

Introduce the new property "xen,uefi-binary" for device tree boot
module nodes that are subnode of "xen,domain" compatible nodes.
The property holds a string containing the file name of the
binary that shall be loaded by the uefi loader from the filesystem.

Update efi documentation about how to start a dom0less
setup using UEFI

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v4:
- update uefi,cfg-load to xen,uefi-cfg-load in documentation
- fixed comments and code style
- changed variable name from dt_module_found to dt_modules_found
in boot.c
- removed stub efi_arch_check_dt_boot from x86 code because the
architecture does not support DT, protected call with #ifdef
in the common code.
- add comment to explain the result from efi_arch_check_dt_boot
just looking the common code
- Add space before comment in boot.c
- renamed uefi,binary property to xen,uefi-binary
Changes in v3:
- fixed documentation
- fixed name len in strlcpy
- fixed some style issues
- closed filesystem handle before calling blexit
- passed runtime errors up to the stack instead
of calling blexit
- renamed names and function to make them more
general in prevision to load also Dom0 kernel
and ramdisk from DT
Changes in v2:
- remove array of struct file
- fixed some int types
- Made the code use filesystem even when configuration
file is skipped.
- add documentation of uefi,binary in booting.txt
- add documentation on how to boot all configuration
for Xen using UEFI in efi.pandoc
---
 docs/misc/arm/device-tree/booting.txt |  21 ++
 docs/misc/efi.pandoc                  | 203 +++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 305 +++++++++++++++++++++++++-
 xen/common/efi/boot.c                 |  46 ++--
 4 files changed, 560 insertions(+), 15 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 352b0ec43a..7258e7e1ec 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
 
     Command line parameters for the guest kernel.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 The ramdisk sub-node has the following properties:
 
 - compatible
@@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
     Specifies the physical address of the ramdisk in RAM and its
     length.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 
 Example
 =======
@@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
     Specifies the physical address of the device tree binary fragment
     RAM and its length.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 As an example:
 
         module@0xc000000 {
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ed85351541..876cd55005 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,3 +167,206 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+
+## UEFI boot and dom0less on ARM
+
+Dom0less feature is supported by ARM and it is possible to use it when Xen is
+started as an EFI application.
+The way to specify the domU domains is by Device Tree as specified in the
+[dom0less](dom0less.html) documentation page under the "Device Tree
+configuration" section, but instead of declaring the reg property in the boot
+module, the user must specify the "xen,uefi-binary" property containing the name
+of the binary file that has to be loaded in memory.
+The UEFI stub will load the binary in memory and it will add the reg property
+accordingly.
+
+An example here:
+
+domU1 {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	compatible = "xen,domain";
+	memory = <0 0x20000>;
+	cpus = <1>;
+	vpl011;
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "console=ttyAMA0";
+	};
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+	module@3 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "passthrough.dtb";
+	};
+};
+
+## How to boot different Xen setup using UEFI
+
+These are the different ways to boot a Xen system from UEFI:
+
+ - Boot Xen and Dom0 (minimum required)
+ - Boot Xen and DomU(s) (true dom0less, only on ARM)
+ - Boot Xen, Dom0 and DomU(s) (only on ARM)
+
+### Boot Xen and Dom0
+
+This configuration can be started using the Xen configuration file in the
+example above.
+
+### Boot Xen and DomU(s)
+
+This configuration needs the domU domain(s) specified in the /chosen node,
+examples of how to do that are provided by the documentation about dom0less
+and the example above shows how to use the "xen,uefi-binary" property to use the
+UEFI stub for module loading.
+When adding DomU modules to device tree, also add the property
+xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
+Otherwise, Xen will skip the config file and rely on device tree alone.
+
+Example 1 of how to boot a true dom0less configuration:
+
+Xen configuration file: skipped.
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "<Xen command line>"
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+Example 2 of how to boot a true dom0less configuration:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,uefi-cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+### Boot Xen, Dom0 and DomU(s)
+
+This configuration is a mix of the two configuration above, to boot this one
+the configuration file must be processed so the /chosen node must have the
+"xen,uefi-cfg-load" property.
+
+Here an example:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
+ramdisk=initrd-3.0.31-0.4-xen
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,uefi-cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index a3e46453d4..50b3829ae0 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -8,9 +8,49 @@
 #include <asm/setup.h>
 #include <asm/smp.h>
 
+typedef struct {
+    char *name;
+    unsigned int name_len;
+    EFI_PHYSICAL_ADDRESS addr;
+    UINTN size;
+} module_name;
+
+/*
+ * Binaries will be translated into bootmodules, the maximum number for them is
+ * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
+ */
+#define MAX_UEFI_MODULES (MAX_MODULES - 2)
+static struct file __initdata module_binary;
+static module_name __initdata modules[MAX_UEFI_MODULES];
+static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
+static unsigned int __initdata modules_idx;
+
+#define ERROR_BINARY_FILE_NOT_FOUND (-1)
+#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
+#define ERROR_ALLOC_MODULE_NAME     (-2)
+#define ERROR_MISSING_DT_PROPERTY   (-3)
+#define ERROR_RENAME_MODULE_NAME    (-4)
+#define ERROR_SET_REG_PROPERTY      (-5)
+#define ERROR_DT_MODULE_DOMU        (-1)
+#define ERROR_DT_CHOSEN_NODE        (-2)
+
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
 
+static int get_module_file_index(const char *name, unsigned int name_len);
+static void PrintMessage(const CHAR16 *s);
+static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                const char *name,
+                                unsigned int name_len);
+static int handle_module_node(EFI_FILE_HANDLE dir_handle,
+                              int module_node_offset,
+                              int reg_addr_cells,
+                              int reg_size_cells);
+static bool is_boot_module(int dt_module_offset);
+static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                       int domain_node);
+static int efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle);
+
 #define DEVICE_TREE_GUID \
 {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
 
@@ -552,8 +592,260 @@ static void __init efi_arch_handle_module(const struct file *file,
                          kernel.size) < 0 )
             blexit(L"Unable to set reg property.");
     }
-    else
+    else if ( file != &module_binary )
+        /*
+         * If file is not a dom0 module file and it's not a domU module,
+         * stop here.
+         */
         blexit(L"Unknown module type");
+
+    /*
+     * modules_available is decremented here because for each dom0 file added
+     * from the configuration file, there will be an additional bootmodule,
+     * so the number of available slots will be decremented because there is a
+     * maximum amount of bootmodules that can be loaded.
+     */
+    modules_available--;
+}
+
+/*
+ * This function checks for a binary previously loaded with a give name, it
+ * returns the index of the file in the modules array or a negative number if no
+ * file with that name is found.
+ */
+static int __init get_module_file_index(const char *name,
+                                        unsigned int name_len)
+{
+    unsigned int i;
+    int ret = ERROR_BINARY_FILE_NOT_FOUND;
+
+    for ( i = 0; i < modules_idx; i++ )
+    {
+        module_name *mod = &modules[i];
+        if ( (mod->name_len == name_len) &&
+             (strncmp(mod->name, name, name_len) == 0) )
+        {
+            ret = i;
+            break;
+        }
+    }
+    return ret;
+}
+
+static void __init PrintMessage(const CHAR16 *s)
+{
+    PrintStr(s);
+    PrintStr(newline);
+}
+
+/*
+ * This function allocates a binary and keeps track of its name, it returns the
+ * index of the file in the modules array or a negative number on error.
+ */
+static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                       const char *name,
+                                       unsigned int name_len)
+{
+    module_name *file_name;
+    union string module_name;
+    int ret;
+
+    /*
+     * Check if there is any space left for a module, the variable
+     * modules_available is updated each time we use read_file(...)
+     * successfully.
+     */
+    if ( !modules_available )
+    {
+        PrintMessage(L"No space left for modules");
+        return ERROR_ALLOC_MODULE_NO_SPACE;
+    }
+
+    module_name.cs = name;
+    ret = modules_idx;
+
+    /* Save at this index the name of this binary */
+    file_name = &modules[ret];
+
+    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
+                              (void**)&file_name->name) != EFI_SUCCESS )
+    {
+        PrintMessage(L"Error allocating memory for module binary name");
+        return ERROR_ALLOC_MODULE_NAME;
+    }
+
+    /* Save name and length of the binary in the data structure */
+    strlcpy(file_name->name, name, name_len + 1);
+    file_name->name_len = name_len;
+
+    /* Load the binary in memory */
+    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
+
+    /* Save address and size */
+    file_name->addr = module_binary.addr;
+    file_name->size = module_binary.size;
+
+    /* s2w(...) allocates some memory, free it */
+    efi_bs->FreePool(module_name.w);
+
+    modules_idx++;
+
+    return ret;
+}
+
+/*
+ * This function checks for the presence of the xen,uefi-binary property in the
+ * module, if found it loads the binary as module and sets the right address
+ * for the reg property into the module DT node.
+ */
+static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
+                                     int module_node_offset,
+                                     int reg_addr_cells,
+                                     int reg_size_cells)
+{
+    const void *uefi_name_prop;
+    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
+    int uefi_name_len, file_idx;
+    module_name *file;
+
+    /* Read xen,uefi-binary property to get the file name. */
+    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
+                                 &uefi_name_len);
+
+    if ( !uefi_name_prop )
+        /* Property not found */
+        return 0;
+
+    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
+    if ( file_idx < 0 )
+    {
+        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
+                                        uefi_name_len);
+        if ( file_idx < 0 )
+            return file_idx;
+    }
+
+    file = &modules[file_idx];
+
+    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
+
+    /* Rename the module to be module@{address} */
+    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
+    {
+        PrintMessage(L"Unable to modify module node name.");
+        return ERROR_RENAME_MODULE_NAME;
+    }
+
+    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
+                     file->addr, file->size) < 0 )
+    {
+        PrintMessage(L"Unable to set module reg property.");
+        return ERROR_SET_REG_PROPERTY;
+    }
+
+    return 0;
+}
+
+static bool __init is_boot_module(int dt_module_offset)
+{
+    if ( (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,kernel") == 0) ||
+         (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,ramdisk") == 0) ||
+         (fdt_node_check_compatible(fdt, dt_module_offset,
+                                    "multiboot,device-tree") == 0) )
+        return true;
+
+    return false;
+}
+
+/*
+ * This function checks for boot modules under the domU guest domain node
+ * in the DT.
+ * Returns 0 on success, negative number on error.
+ */
+static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                              int domain_node)
+{
+    int module_node, addr_cells, size_cells, len;
+    const struct fdt_property *prop;
+
+    /* Get #address-cells and #size-cells from domain node */
+    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#address-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#size-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    /*
+     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
+     * inside this node
+     */
+    for ( module_node = fdt_first_subnode(fdt, domain_node);
+          module_node > 0;
+          module_node = fdt_next_subnode(fdt, module_node) )
+        if ( is_boot_module(module_node) )
+        {
+            int ret = handle_module_node(dir_handle, module_node, addr_cells,
+                                         size_cells);
+            if ( ret < 0 )
+                return ret;
+        }
+
+    return 0;
+}
+
+/*
+ * This function checks for xen domain nodes under the /chosen node for possible
+ * domU guests to be loaded.
+ * Returns the number of modules loaded or a negative number for error.
+ */
+static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
+{
+    int chosen, node, addr_len, size_len;
+    unsigned int i = 0;
+
+    /* Check for the chosen node in the current DTB */
+    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
+    if ( chosen < 0 )
+    {
+        PrintMessage(L"Unable to setup chosen node");
+        return ERROR_DT_CHOSEN_NODE;
+    }
+
+    /* Check for nodes compatible with xen,domain under the chosen node */
+    for ( node = fdt_first_subnode(fdt, chosen);
+          node > 0;
+          node = fdt_next_subnode(fdt, node) )
+    {
+        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
+        {
+            /* Found a node with compatible xen,domain; handle this node. */
+            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
+                return ERROR_DT_MODULE_DOMU;
+        }
+    }
+
+    /* Free boot modules file names if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free boot modules binary names */
+        efi_bs->FreePool(modules[i].name);
+    }
+
+    return modules_idx;
 }
 
 static void __init efi_arch_cpu(void)
@@ -562,8 +854,19 @@ static void __init efi_arch_cpu(void)
 
 static void __init efi_arch_blexit(void)
 {
+    unsigned int i = 0;
+
     if ( dtbfile.need_to_free )
         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
+    /* Free boot modules file names if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free boot modules binary names */
+        efi_bs->FreePool(modules[i].name);
+        /* Free modules binaries */
+        efi_bs->FreePages(modules[i].addr,
+                          PFN_UP(modules[i].size));
+    }
     if ( memmap )
         efi_bs->FreePool(memmap);
 }
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 758f9d74d2..daf7c26099 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
     EFI_LOADED_IMAGE *loaded_image;
     EFI_STATUS status;
-    unsigned int i, argc;
-    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
+    unsigned int i, argc = 0;
+    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
     UINTN gop_mode = ~0;
     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
     union string section = { NULL }, name;
     bool base_video = false;
-    const char *option_str;
+    const char *option_str = NULL;
     bool use_cfg_file;
+    int dt_modules_found = 0;
+    EFI_FILE_HANDLE dir_handle;
 
     __set_bit(EFI_BOOT, &efi_flags);
     __set_bit(EFI_LOADER, &efi_flags);
@@ -1216,9 +1218,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
     efi_arch_relocate_image(0);
 
+    /* Get the file system interface. */
+    dir_handle = get_parent_handle(loaded_image, &file_name);
+
     if ( use_cfg_file )
     {
-        EFI_FILE_HANDLE dir_handle;
         UINTN depth, cols, rows, size;
 
         size = cols = rows = depth = 0;
@@ -1229,9 +1233,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
         gop = efi_get_gop();
 
-        /* Get the file system interface. */
-        dir_handle = get_parent_handle(loaded_image, &file_name);
-
         /* Read and parse the config file. */
         if ( read_section(loaded_image, L"config", &cfg, NULL) )
             PrintStr(L"Using builtin config file\r\n");
@@ -1285,14 +1286,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
             efi_bs->FreePool(name.w);
         }
 
-        if ( !name.s )
-            blexit(L"No Dom0 kernel image specified.");
-
         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
 
-        option_str = split_string(name.s);
+        if ( name.s )
+            option_str = split_string(name.s);
 
-        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
+        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
+             name.s )
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
@@ -1361,12 +1361,30 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
         cfg.addr = 0;
 
-        dir_handle->Close(dir_handle);
-
         if ( gop && !base_video )
             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
     }
 
+#ifdef CONFIG_HAS_DEVICE_TREE
+    /* Get the number of boot modules specified on the DT or an error (<0) */
+    dt_modules_found = efi_arch_check_dt_boot(dir_handle);
+#endif
+
+    dir_handle->Close(dir_handle);
+
+    if ( dt_modules_found < 0 )
+        /* efi_arch_check_dt_boot throws some error */
+        blexit(L"Error processing boot modules on DT.");
+
+    /*
+     * Check if a proper configuration is provided to start Xen:
+     *  - Dom0 specified (minimum required)
+     *  - Dom0 and DomU(s) specified
+     *  - DomU(s) specified
+     */
+    if ( !dt_modules_found && !kernel.addr )
+        blexit(L"No Dom0 kernel image specified.");
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:29:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:29:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199970.354341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3d-0005aw-GI; Thu, 30 Sep 2021 14:29:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199970.354341; Thu, 30 Sep 2021 14:29:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx3d-0005an-C3; Thu, 30 Sep 2021 14:29:09 +0000
Received: by outflank-mailman (input) for mailman id 199970;
 Thu, 30 Sep 2021 14:29:08 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=luca.fancellu@srs-us1.protection.inumbo.net>)
 id 1mVx3c-0004wW-1w
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:29:08 +0000
Received: from foss.arm.com (unknown [217.140.110.172])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTP
 id c09c3825-21fa-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:29:01 +0000 (UTC)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
 by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CE57C11FB;
 Thu, 30 Sep 2021 07:29:00 -0700 (PDT)
Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com
 [10.1.195.16])
 by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 62D263F70D;
 Thu, 30 Sep 2021 07:28:59 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c09c3825-21fa-11ec-bd4e-12813bfff9fa
From: Luca Fancellu <luca.fancellu@arm.com>
To: xen-devel@lists.xenproject.org
Cc: bertrand.marquis@arm.com,
	wei.chen@arm.com,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Ian Jackson <iwj@xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 3/3] arm/efi: load dom0 modules from DT using UEFI
Date: Thu, 30 Sep 2021 15:28:46 +0100
Message-Id: <20210930142846.13348-4-luca.fancellu@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210930142846.13348-1-luca.fancellu@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>

Add support to load Dom0 boot modules from
the device tree using the xen,uefi-binary property.

Update documentation about that.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v4:
- Add check to avoid double definition of Dom0 ramdisk
from cfg file and DT
- Fix if conditions indentation in boot.c
- Moved Dom0 kernel verification code after check for
presence for Dom0 or DomU(s)
- Changed uefi,binary property to xen,uefi-binary
Changes in v3:
- new patch
---
 docs/misc/arm/device-tree/booting.txt |  8 ++++
 docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
 xen/arch/arm/efi/efi-boot.h           | 47 ++++++++++++++++++--
 xen/common/efi/boot.c                 | 16 ++++---
 4 files changed, 123 insertions(+), 12 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 7258e7e1ec..8e0c327ba4 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -70,6 +70,14 @@ Each node contains the following properties:
 	priority of this field vs. other mechanisms of specifying the
 	bootargs for the kernel.
 
+- uefi,binary (UEFI boot only)
+
+	String property that specifies the file name to be loaded by the UEFI
+	boot for this module. If this is specified, there is no need to specify
+	the reg property because it will be created by the UEFI stub on boot.
+	This option is needed only when UEFI boot is used, the node needs to be
+	compatible with multiboot,kernel or multiboot,ramdisk.
+
 Examples
 ========
 
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index 876cd55005..4abbb5bb82 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,6 +167,28 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+## UEFI boot and Dom0 modules on ARM
+
+When booting using UEFI on ARM, it is possible to specify the Dom0 modules
+directly from the device tree without using the Xen configuration file, here an
+example:
+
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+}
 
 ## UEFI boot and dom0less on ARM
 
@@ -326,10 +348,10 @@ chosen {
 ### Boot Xen, Dom0 and DomU(s)
 
 This configuration is a mix of the two configuration above, to boot this one
-the configuration file must be processed so the /chosen node must have the
-"xen,uefi-cfg-load" property.
+the configuration file can be processed or the Dom0 modules can be read from
+the device tree.
 
-Here an example:
+Here the first example:
 
 Xen configuration file:
 
@@ -369,4 +391,40 @@ chosen {
 };
 ```
 
+Here the second example:
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
 
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 50b3829ae0..b122e2846a 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -31,8 +31,11 @@ static unsigned int __initdata modules_idx;
 #define ERROR_MISSING_DT_PROPERTY   (-3)
 #define ERROR_RENAME_MODULE_NAME    (-4)
 #define ERROR_SET_REG_PROPERTY      (-5)
+#define ERROR_DOM0_ALREADY_FOUND    (-6)
+#define ERROR_DOM0_RAMDISK_FOUND    (-7)
 #define ERROR_DT_MODULE_DOMU        (-1)
 #define ERROR_DT_CHOSEN_NODE        (-2)
+#define ERROR_DT_MODULE_DOM0        (-3)
 
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
@@ -45,7 +48,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int handle_module_node(EFI_FILE_HANDLE dir_handle,
                               int module_node_offset,
                               int reg_addr_cells,
-                              int reg_size_cells);
+                              int reg_size_cells,
+                              bool is_domu_module);
 static bool is_boot_module(int dt_module_offset);
 static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
                                        int domain_node);
@@ -701,7 +705,8 @@ static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
                                      int module_node_offset,
                                      int reg_addr_cells,
-                                     int reg_size_cells)
+                                     int reg_size_cells,
+                                     bool is_domu_module)
 {
     const void *uefi_name_prop;
     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
@@ -743,6 +748,34 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
         return ERROR_SET_REG_PROPERTY;
     }
 
+    if ( !is_domu_module )
+    {
+        if ( (fdt_node_check_compatible(fdt, module_node_offset,
+                                    "multiboot,kernel") == 0) )
+        {
+            /*
+            * This is the Dom0 kernel, wire it to the kernel variable because it
+            * will be verified by the shim lock protocol later in the common
+            * code.
+            */
+            if ( kernel.addr )
+            {
+                PrintMessage(L"Dom0 kernel already found in cfg file.");
+                return ERROR_DOM0_ALREADY_FOUND;
+            }
+            kernel.need_to_free = false; /* Freed using the module array */
+            kernel.addr = file->addr;
+            kernel.size = file->size;
+        }
+        else if ( ramdisk.addr &&
+                  (fdt_node_check_compatible(fdt, module_node_offset,
+                                             "multiboot,ramdisk") == 0) )
+        {
+            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
+            return ERROR_DOM0_RAMDISK_FOUND;
+        }
+    }
+
     return 0;
 }
 
@@ -799,7 +832,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
         if ( is_boot_module(module_node) )
         {
             int ret = handle_module_node(dir_handle, module_node, addr_cells,
-                                         size_cells);
+                                         size_cells, true);
             if ( ret < 0 )
                 return ret;
         }
@@ -809,7 +842,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
 
 /*
  * This function checks for xen domain nodes under the /chosen node for possible
- * domU guests to be loaded.
+ * dom0 and domU guests to be loaded.
  * Returns the number of modules loaded or a negative number for error.
  */
 static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
@@ -836,6 +869,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
                 return ERROR_DT_MODULE_DOMU;
         }
+        else if ( is_boot_module(node) )
+        {
+            if ( handle_module_node(dir_handle, node, addr_len, size_len,
+                                    false) < 0 )
+                return ERROR_DT_MODULE_DOM0;
+        }
     }
 
     /* Free boot modules file names if any */
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index daf7c26099..e4295dbe34 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
-
-            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
-                            (void **)&shim_lock)) &&
-                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
-                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
         }
 
         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
@@ -1385,6 +1380,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     if ( !dt_modules_found && !kernel.addr )
         blexit(L"No Dom0 kernel image specified.");
 
+    /*
+     * The Dom0 kernel can be loaded from the configuration file or by the
+     * device tree through the efi_arch_check_dt_boot function, in this stage
+     * verify it.
+     */
+    if ( kernel.addr &&
+         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
+                                           (void **)&shim_lock)) &&
+         (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
+        PrintErrMesg(L"Dom0 kernel image could not be verified", status);
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:34:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:34:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199992.354352 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx8V-0007yN-5F; Thu, 30 Sep 2021 14:34:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199992.354352; Thu, 30 Sep 2021 14:34:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx8V-0007yG-0M; Thu, 30 Sep 2021 14:34:11 +0000
Received: by outflank-mailman (input) for mailman id 199992;
 Thu, 30 Sep 2021 14:34:09 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YInd=OU=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVx8T-0007yA-75
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:34:09 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.13.49]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 657b1f26-6833-4f45-b3fc-56edc1a764c6;
 Thu, 30 Sep 2021 14:34:07 +0000 (UTC)
Received: from AM5PR0101CA0034.eurprd01.prod.exchangelabs.com
 (2603:10a6:206:16::47) by PA4PR08MB5917.eurprd08.prod.outlook.com
 (2603:10a6:102:f1::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 14:34:03 +0000
Received: from AM5EUR03FT025.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:206:16:cafe::be) by AM5PR0101CA0034.outlook.office365.com
 (2603:10a6:206:16::47) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.13 via Frontend
 Transport; Thu, 30 Sep 2021 14:34:03 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT025.mail.protection.outlook.com (10.152.16.157) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 14:34:03 +0000
Received: ("Tessian outbound 45760a30af4a:v103");
 Thu, 30 Sep 2021 14:33:59 +0000
Received: from e6945dac6937.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 0D7E9B09-5687-4E95-B2CE-5588CCEDEE84.1; 
 Thu, 30 Sep 2021 14:33:48 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e6945dac6937.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 14:33:48 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6365.eurprd08.prod.outlook.com (2603:10a6:102:156::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 14:33:46 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4566.017; Thu, 30 Sep 2021
 14:33:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 657b1f26-6833-4f45-b3fc-56edc1a764c6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5u7rtDI602zU9fTMa4HGEajXQzX2jKcqbMIWThX2wTg=;
 b=SJdoS36V7t1XrKN0eXJfcRbdb6om3iC2t0vXLZqHliTVLiUrkUN1uPc/RfiHJgCvm04tqSEapIrO1OS8Um04xv71B1CQTkSL4a5bZeaJOxdiFogA7xs585kb2Vqqk8KYmQsEF+bgQUmwllfU0oFhfB0e2209j4VoPrjhKt6QuyI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: ef96f29aec1b67bc
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fUzxlgqroOvAOe8ad8h2UqJX6hsZ1glvD2FREk2Yr1C6fYYE7NHrLg8BWY+dQT63tEpAUDMxR8jfqJA0xMX8fhbw3wFVJWhIFAXNUJNwl9JbdrIexufOpH8GxXZFJ0008dVqn5Qa4NweZJo6Zlev9QRbyRr7gTQ9URypksEpDUvgp3W86f6bjXxmCpp8EQsfCx/k3ibXdix0OVEXa0QN7w/mJBNK0n7Iayf2H5OM6JYuidKcFCOSe79Iro34IxGD7S5UTFxc4gQ7Gddf5Ee9LX47T/+A/WEtGN2RuNq6krpJjbLgx4l7bi/G1nAhO6vtcMQrgqIasiBrXg2B71YXVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=5u7rtDI602zU9fTMa4HGEajXQzX2jKcqbMIWThX2wTg=;
 b=JTlMUERo8ZrRxJQUME9UKqwjacYWd3HSj3xFjB3pvgWscXLn+aJmc9x091f7fDuJQOJv8P3acKBfwQb9T+briG55Ieflo84TxxZi0s2gP+7JjZ5FuHTG4K+nD7fq2g3lnpPqWdfMKvF+uAEOOZc7yV3UejmAJhbc7+F2vnYRePqGNjZpkXXCNxZSsSc1DrJw5z0aIv1TuEGANdD4dTVAF4f9FaLGtmMRJGfAx7Ild03W4tS1Uxp0yY5PGZu+XQW3P0spOVkVfOF3iWPBPuIZ+rWTPmHesPsdQ83LSD2XeZJ3IXVTFSuBNM9mnp9rE5pSI48iPMoE/UD4HmXb+9lkiQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5u7rtDI602zU9fTMa4HGEajXQzX2jKcqbMIWThX2wTg=;
 b=SJdoS36V7t1XrKN0eXJfcRbdb6om3iC2t0vXLZqHliTVLiUrkUN1uPc/RfiHJgCvm04tqSEapIrO1OS8Um04xv71B1CQTkSL4a5bZeaJOxdiFogA7xs585kb2Vqqk8KYmQsEF+bgQUmwllfU0oFhfB0e2209j4VoPrjhKt6QuyI=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Wei Chen <Wei.Chen@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH v4 2/3] arm/efi: Use dom0less configuration when using EFI
 boot
Thread-Topic: [PATCH v4 2/3] arm/efi: Use dom0less configuration when using
 EFI boot
Thread-Index: AQHXtgeNDP3zjkVx/ECJF6CJrFpyb6u8pEmA
Date: Thu, 30 Sep 2021 14:33:46 +0000
Message-ID: <5B036B07-2ADB-4132-BC26-633A3F80BFFC@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>
 <20210930142846.13348-3-luca.fancellu@arm.com>
In-Reply-To: <20210930142846.13348-3-luca.fancellu@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 92bf8fb2-68eb-4068-8396-08d9841f59b1
x-ms-traffictypediagnostic: PAXPR08MB6365:|PA4PR08MB5917:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PA4PR08MB5917ACF80BFC6870095620659DAA9@PA4PR08MB5917.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:1107;OLM:1107;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 HhAI4tno9wXdHDkXa3kqX2vCzfLEQWEIlj6G1Y2Obx9umv6a0z/1ezmYYIJDMIvNz2DTiAevFPsMgKcOjUzXhlTkmvyYZxwmkFhph5m+wy+QJr3Y3ehNnd0De5WyWiqBnSleisu6ocKUdsPJ8nBkyN/XUJj7m0f67WttSwa4fMzSS3lxvNQ6sJSSgODArouIIVzRtgq1l/aN7K0xnl6Sz7/WlzBBiPK2c3OTvRmSnN5Ip4YV3KsfaQe+3YmppLI8shMjDpiQyeQLkAiltAw1mXkJdFPIRlisUGkhXVmKCCUVMthowQ/LwMuRGtp6oEKj0EwoFVafrBfZQFRNtm2KximFphgXBMHshU17T2B5kaA89L2QIwzMJywb7in020Gc1Cb8TPXa3d9KrkWtCE1AEM/YDVLgGp8UTYq8+Uy4H6Hqr2wvICWMfs/+lOmyLaMdWLoBv2FEKFDDXM+BWmqhseh/eiwtXfgTRsDWWvRP9l5Ew2jYA7fjt+hc+doydYAfBAzyAhiLrHIbFl+xN1OMzS1fMWMic4ylKX3gtG+PdtCg98QYkqRbuaGyxwqKGz12sO8J9iKaer7SUMtRBx0K6aN+hTMT1vnfrZJuK/bjYARBQuTn498dPUd0g1taMW25LMUbKtPFO7Mj3qc8UfMedISq5/RGAx1HNbaNwWGeD/SGfSrMqMzwFY06n4xtOkJefvdNKcUMhdWAbqIcNTEjq4B1cOnWzlfXIkUJFEevzqD6ay3HqBqVcL0CTS+mTCCXpQAngTf+Qkr4hdh9GIXz7kU8EFJ5Eux3b95tzBIs9mA=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(6862004)(6486002)(6512007)(508600001)(71200400001)(53546011)(37006003)(54906003)(2906002)(186003)(316002)(4326008)(26005)(2616005)(6636002)(6506007)(8936002)(83380400001)(76116006)(91956017)(30864003)(64756008)(66946007)(66556008)(66476007)(66446008)(86362001)(33656002)(38070700005)(38100700002)(122000001)(5660300002)(36756003)(32563001)(21314003)(45980500001)(579004)(559001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <52B7FCAF6426504AACCBC79FD95CCC32@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6365
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT025.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	02810c79-2ca3-4916-9ed3-08d9841f4f87
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	PpKFcFy1ZWjwdLIJk+YsJLPcDl2ZLeQ95gWdhyrlITOilvU28LUU5LV+ypiWqRVfbU6+renarMDCEucgLmKTYKzaAZKNaqkTlIEzwnF3JVrGFhJ7AfvpuLwm1s/ZQqZoVLxTsGCv5iH/RYkOaBczMFYFD4qxFPS2Bvxj0Srp5g9SGdvXaZyLwWF5f8BYJxIl0AVVUxyAZ01cIi5iwFbZ7eCf1IYGkbIkTkKbIln4X/P90jYMX5Bqe9BWjU5HDuCBpONRMgJk4MklprR6YY0oU6ep2uzFlNoNS9ul7zem+KB1bYWwAhp21zSCyorWEUoA6WvwOuTZofSJvCD36vudob1OIMko79vPLDmLx45oAEhoQ6dtSF3CH0Icysdgl6FjimuSzP2heE5K8NS9Xjl7DY9q+7ckSu7D9I/V8ELU0Rg25Oj0ATs+lJnPj3gn0VOTd7oXdAfs8ZHj6JVkT7WUhjeHX+5uV66RDujGnYJ0JLYE2cX3fLj9WzwDxRK23DQqpI7KJu6J4rwUQCpOisgHXDFtBy63IlXahZaUAP9B6RzPfkb8Opo3vS0hmaKcAsdbmaZccKqUZOt/+q0rlg01dQExfy9DGQ4rBBFjUDFzUF7W6NuZtRZdqtuPe8AAe9bk94GNxpCkwOGU3GaYWWv7vDmn6JDErcXZjCuocvJc5V5wwbejQoTEXBrybU0QQtGVv9M4wttSHZ7UEfzIlEDdA23PG0PLU8olVlrJCVVzFGdFP+IYtUsfGizyxm4YhcYg
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(5660300002)(54906003)(70206006)(33656002)(8676002)(30864003)(4326008)(316002)(6636002)(70586007)(36756003)(26005)(86362001)(186003)(37006003)(336012)(82310400003)(6862004)(508600001)(6486002)(2616005)(36860700001)(6512007)(8936002)(81166007)(47076005)(6506007)(53546011)(2906002)(356005)(83380400001)(32563001)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:34:03.2425
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 92bf8fb2-68eb-4068-8396-08d9841f59b1
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT025.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5917

Hi Luca,

> On 30 Sep 2021, at 15:28, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
>=20
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
>=20
> Introduce the new property "xen,uefi-binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
>=20
> Update efi documentation about how to start a dom0less
> setup using UEFI
>=20
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Changes in v4:
> - update uefi,cfg-load to xen,uefi-cfg-load in documentation
> - fixed comments and code style
> - changed variable name from dt_module_found to dt_modules_found
> in boot.c
> - removed stub efi_arch_check_dt_boot from x86 code because the
> architecture does not support DT, protected call with #ifdef
> in the common code.
> - add comment to explain the result from efi_arch_check_dt_boot
> just looking the common code
> - Add space before comment in boot.c
> - renamed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - fixed documentation
> - fixed name len in strlcpy
> - fixed some style issues
> - closed filesystem handle before calling blexit
> - passed runtime errors up to the stack instead
> of calling blexit
> - renamed names and function to make them more
> general in prevision to load also Dom0 kernel
> and ramdisk from DT
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
> docs/misc/arm/device-tree/booting.txt |  21 ++
> docs/misc/efi.pandoc                  | 203 +++++++++++++++++
> xen/arch/arm/efi/efi-boot.h           | 305 +++++++++++++++++++++++++-
> xen/common/efi/boot.c                 |  46 ++--
> 4 files changed, 560 insertions(+), 15 deletions(-)
>=20
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device=
-tree/booting.txt
> index 352b0ec43a..7258e7e1ec 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
>=20
>     Command line parameters for the guest kernel.
>=20
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEF=
I boot
> +    for this module. If this is specified, there is no need to specify t=
he reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
> The ramdisk sub-node has the following properties:
>=20
> - compatible
> @@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
>     Specifies the physical address of the ramdisk in RAM and its
>     length.
>=20
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEF=
I boot
> +    for this module. If this is specified, there is no need to specify t=
he reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>=20
> Example
> =3D=3D=3D=3D=3D=3D=3D
> @@ -265,6 +279,13 @@ The dtb sub-node should have the following propertie=
s:
>     Specifies the physical address of the device tree binary fragment
>     RAM and its length.
>=20
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEF=
I boot
> +    for this module. If this is specified, there is no need to specify t=
he reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
> As an example:
>=20
>         module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ed85351541..876cd55005 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
> 	--output xen.signed.efi \
> 	xen.unified.efi
> ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when X=
en is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in th=
e
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the=
 boot
> +module, the user must specify the "xen,uefi-binary" property containing =
the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg pro=
perty
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells =3D <1>;
> +	#size-cells =3D <1>;
> +	compatible =3D "xen,domain";
> +	memory =3D <0 0x20000>;
> +	cpus =3D <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible =3D "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary =3D "vmlinuz-3.0.31-0.4-xen";
> +		bootargs =3D "console=3DttyAMA0";
> +	};
> +	module@2 {
> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary =3D "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary =3D "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +These are the different ways to boot a Xen system from UEFI:
> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in th=
e
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen nod=
e,
> +examples of how to do that are provided by the documentation about dom0l=
ess
> +and the example above shows how to use the "xen,uefi-binary" property to=
 use the
> +UEFI stub for module loading.
> +When adding DomU modules to device tree, also add the property
> +xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
> +Otherwise, Xen will skip the config file and rely on device tree alone.
> +
> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells =3D <0x1>;
> +	#address-cells =3D <0x1>;
> +	xen,xen-bootargs =3D "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu1.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu2.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=3Dxen
> +
> +[xen]
> +options=3D<Xen command line>
> +dtb=3D<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells =3D <0x1>;
> +	#address-cells =3D <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu1.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu2.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this=
 one
> +the configuration file must be processed so the /chosen node must have t=
he
> +"xen,uefi-cfg-load" property.
> +
> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=3Dxen
> +
> +[xen]
> +options=3D<Xen command line>
> +kernel=3Dvmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=3Dinitrd-3.0.31-0.4-xen
> +dtb=3D<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells =3D <0x1>;
> +	#address-cells =3D <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu1.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index a3e46453d4..50b3829ae0 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,49 @@
> #include <asm/setup.h>
> #include <asm/smp.h>
>=20
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for =
them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_UEFI_MODULES (MAX_MODULES - 2)
> +static struct file __initdata module_binary;
> +static module_name __initdata modules[MAX_UEFI_MODULES];
> +static unsigned int __initdata modules_available =3D MAX_UEFI_MODULES;
> +static unsigned int __initdata modules_idx;
> +
> +#define ERROR_BINARY_FILE_NOT_FOUND (-1)
> +#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
> +#define ERROR_ALLOC_MODULE_NAME     (-2)
> +#define ERROR_MISSING_DT_PROPERTY   (-3)
> +#define ERROR_RENAME_MODULE_NAME    (-4)
> +#define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DT_MODULE_DOMU        (-1)
> +#define ERROR_DT_CHOSEN_NODE        (-2)
> +
> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
> void __flush_dcache_area(const void *vaddr, unsigned long size);
>=20
> +static int get_module_file_index(const char *name, unsigned int name_len=
);
> +static void PrintMessage(const CHAR16 *s);
> +static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                const char *name,
> +                                unsigned int name_len);
> +static int handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                              int module_node_offset,
> +                              int reg_addr_cells,
> +                              int reg_size_cells);
> +static bool is_boot_module(int dt_module_offset);
> +static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                       int domain_node);
> +static int efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle);
> +
> #define DEVICE_TREE_GUID \
> {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0=
xe0}}
>=20
> @@ -552,8 +592,260 @@ static void __init efi_arch_handle_module(const str=
uct file *file,
>                          kernel.size) < 0 )
>             blexit(L"Unable to set reg property.");
>     }
> -    else
> +    else if ( file !=3D &module_binary )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>         blexit(L"Unknown module type");
> +
> +    /*
> +     * modules_available is decremented here because for each dom0 file =
added
> +     * from the configuration file, there will be an additional bootmodu=
le,
> +     * so the number of available slots will be decremented because ther=
e is a
> +     * maximum amount of bootmodules that can be loaded.
> +     */
> +    modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name,=
 it
> + * returns the index of the file in the modules array or a negative numb=
er if no
> + * file with that name is found.
> + */
> +static int __init get_module_file_index(const char *name,
> +                                        unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret =3D ERROR_BINARY_FILE_NOT_FOUND;
> +
> +    for ( i =3D 0; i < modules_idx; i++ )
> +    {
> +        module_name *mod =3D &modules[i];
> +        if ( (mod->name_len =3D=3D name_len) &&
> +             (strncmp(mod->name, name, name_len) =3D=3D 0) )
> +        {
> +            ret =3D i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +static void __init PrintMessage(const CHAR16 *s)
> +{
> +    PrintStr(s);
> +    PrintStr(newline);
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it retu=
rns the
> + * index of the file in the modules array or a negative number on error.
> + */
> +static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                       const char *name,
> +                                       unsigned int name_len)
> +{
> +    module_name *file_name;
> +    union string module_name;
> +    int ret;
> +
> +    /*
> +     * Check if there is any space left for a module, the variable
> +     * modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !modules_available )
> +    {
> +        PrintMessage(L"No space left for modules");
> +        return ERROR_ALLOC_MODULE_NO_SPACE;
> +    }
> +
> +    module_name.cs =3D name;
> +    ret =3D modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name =3D &modules[ret];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(cha=
r),
> +                              (void**)&file_name->name) !=3D EFI_SUCCESS=
 )
> +    {
> +        PrintMessage(L"Error allocating memory for module binary name");
> +        return ERROR_ALLOC_MODULE_NAME;
> +    }
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len + 1);
> +    file_name->name_len =3D name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr =3D module_binary.addr;
> +    file_name->size =3D module_binary.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    modules_idx++;
> +
> +    return ret;
> +}
> +
> +/*
> + * This function checks for the presence of the xen,uefi-binary property=
 in the
> + * module, if found it loads the binary as module and sets the right add=
ress
> + * for the reg property into the module DT node.
> + */
> +static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                                     int module_node_offset,
> +                                     int reg_addr_cells,
> +                                     int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + =
\0 */
> +    int uefi_name_len, file_idx;
> +    module_name *file;
> +
> +    /* Read xen,uefi-binary property to get the file name. */
> +    uefi_name_prop =3D fdt_getprop(fdt, module_node_offset, "xen,uefi-bi=
nary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return 0;
> +
> +    file_idx =3D get_module_file_index(uefi_name_prop, uefi_name_len);
> +    if ( file_idx < 0 )
> +    {
> +        file_idx =3D allocate_module_file(dir_handle, uefi_name_prop,
> +                                        uefi_name_len);
> +        if ( file_idx < 0 )
> +            return file_idx;
> +    }
> +
> +    file =3D &modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->add=
r);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +    {
> +        PrintMessage(L"Unable to modify module node name.");
> +        return ERROR_RENAME_MODULE_NAME;
> +    }
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_c=
ells,
> +                     file->addr, file->size) < 0 )
> +    {
> +        PrintMessage(L"Unable to set module reg property.");
> +        return ERROR_SET_REG_PROPERTY;
> +    }
> +
> +    return 0;
> +}
> +
> +static bool __init is_boot_module(int dt_module_offset)
> +{
> +    if ( (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,kernel") =3D=3D 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,ramdisk") =3D=3D 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,device-tree") =3D=3D 0) )
> +        return true;
> +
> +    return false;
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain nod=
e
> + * in the DT.
> + * Returns 0 on success, negative number on error.
> + */
> +static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle=
,
> +                                              int domain_node)
> +{
> +    int module_node, addr_cells, size_cells, len;
> +    const struct fdt_property *prop;
> +
> +    /* Get #address-cells and #size-cells from domain node */
> +    prop =3D fdt_get_property(fdt, domain_node, "#address-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#address-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    addr_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    prop =3D fdt_get_property(fdt, domain_node, "#size-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#size-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    size_cells =3D fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    /*
> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-=
tree}
> +     * inside this node
> +     */
> +    for ( module_node =3D fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node =3D fdt_next_subnode(fdt, module_node) )
> +        if ( is_boot_module(module_node) )
> +        {
> +            int ret =3D handle_module_node(dir_handle, module_node, addr=
_cells,
> +                                         size_cells);
> +            if ( ret < 0 )
> +                return ret;
> +        }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for =
possible
> + * domU guests to be loaded.
> + * Returns the number of modules loaded or a negative number for error.
> + */
> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen, node, addr_len, size_len;
> +    unsigned int i =3D 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen =3D setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +    {
> +        PrintMessage(L"Unable to setup chosen node");
> +        return ERROR_DT_CHOSEN_NODE;
> +    }
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node =
*/
> +    for ( node =3D fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node =3D fdt_next_subnode(fdt, node) )
> +    {
> +        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
> +        {
> +            /* Found a node with compatible xen,domain; handle this node=
. */
> +            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> +                return ERROR_DT_MODULE_DOMU;
> +        }
> +    }
> +
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +    }
> +
> +    return modules_idx;
> }
>=20
> static void __init efi_arch_cpu(void)
> @@ -562,8 +854,19 @@ static void __init efi_arch_cpu(void)
>=20
> static void __init efi_arch_blexit(void)
> {
> +    unsigned int i =3D 0;
> +
>     if ( dtbfile.need_to_free )
>         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +        /* Free modules binaries */
> +        efi_bs->FreePages(modules[i].addr,
> +                          PFN_UP(modules[i].size));
> +    }
>     if ( memmap )
>         efi_bs->FreePool(memmap);
> }
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..daf7c26099 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>     static EFI_GUID __initdata shim_lock_guid =3D SHIM_LOCK_PROTOCOL_GUID=
;
>     EFI_LOADED_IMAGE *loaded_image;
>     EFI_STATUS status;
> -    unsigned int i, argc;
> -    CHAR16 **argv, *file_name, *cfg_file_name =3D NULL, *options =3D NUL=
L;
> +    unsigned int i, argc =3D 0;
> +    CHAR16 **argv, *file_name =3D NULL, *cfg_file_name =3D NULL, *option=
s =3D NULL;
>     UINTN gop_mode =3D ~0;
>     EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop =3D NULL;
>     union string section =3D { NULL }, name;
>     bool base_video =3D false;
> -    const char *option_str;
> +    const char *option_str =3D NULL;
>     bool use_cfg_file;
> +    int dt_modules_found =3D 0;
> +    EFI_FILE_HANDLE dir_handle;
>=20
>     __set_bit(EFI_BOOT, &efi_flags);
>     __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1218,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>=20
>     efi_arch_relocate_image(0);
>=20
> +    /* Get the file system interface. */
> +    dir_handle =3D get_parent_handle(loaded_image, &file_name);
> +
>     if ( use_cfg_file )
>     {
> -        EFI_FILE_HANDLE dir_handle;
>         UINTN depth, cols, rows, size;
>=20
>         size =3D cols =3D rows =3D depth =3D 0;
> @@ -1229,9 +1233,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE =
*SystemTable)
>=20
>         gop =3D efi_get_gop();
>=20
> -        /* Get the file system interface. */
> -        dir_handle =3D get_parent_handle(loaded_image, &file_name);
> -
>         /* Read and parse the config file. */
>         if ( read_section(loaded_image, L"config", &cfg, NULL) )
>             PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1286,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>             efi_bs->FreePool(name.w);
>         }
>=20
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>=20
> -        option_str =3D split_string(name.s);
> +        if ( name.s )
> +            option_str =3D split_string(name.s);
>=20
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str)=
 )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str)=
 &&
> +             name.s )
>         {
>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>             efi_bs->FreePool(name.w);
> @@ -1361,12 +1361,30 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABL=
E *SystemTable)
>         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>         cfg.addr =3D 0;
>=20
> -        dir_handle->Close(dir_handle);
> -
>         if ( gop && !base_video )
>             gop_mode =3D efi_find_gop_mode(gop, cols, rows, depth);
>     }
>=20
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    /* Get the number of boot modules specified on the DT or an error (<=
0) */
> +    dt_modules_found =3D efi_arch_check_dt_boot(dir_handle);
> +#endif
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if ( dt_modules_found < 0 )
> +        /* efi_arch_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");
> +
> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !dt_modules_found && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");
> +
>     efi_arch_edd();
>=20
>     /* XXX Collect EDID info. */
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:34:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:34:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.199994.354363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx8m-0008Ly-Hc; Thu, 30 Sep 2021 14:34:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 199994.354363; Thu, 30 Sep 2021 14:34:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx8m-0008Lr-Dy; Thu, 30 Sep 2021 14:34:28 +0000
Received: by outflank-mailman (input) for mailman id 199994;
 Thu, 30 Sep 2021 14:34:26 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YInd=OU=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVx8k-0008Kr-ND
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:34:26 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.21.41]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 82835e69-21fb-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:34:25 +0000 (UTC)
Received: from DB6PR07CA0112.eurprd07.prod.outlook.com (2603:10a6:6:2c::26) by
 AS8PR08MB6696.eurprd08.prod.outlook.com (2603:10a6:20b:395::8) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15; Thu, 30 Sep 2021 14:34:23 +0000
Received: from DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:2c:cafe::75) by DB6PR07CA0112.outlook.office365.com
 (2603:10a6:6:2c::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.7 via Frontend
 Transport; Thu, 30 Sep 2021 14:34:23 +0000
Received: from 64aa7808-outbound-2.mta.getcheckrecipient.com (63.33.187.114)
 by DB5EUR03FT062.mail.protection.outlook.com (10.152.20.197) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 14:34:23 +0000
Received: ("Tessian outbound ac52c8afb262:v103");
 Thu, 30 Sep 2021 14:34:20 +0000
Received: from 56e10d3f41fb.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 19CCD413-BD0A-44CF-9BDA-0B86ECB9268D.1; 
 Thu, 30 Sep 2021 14:34:04 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 56e10d3f41fb.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 14:34:04 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB7033.eurprd08.prod.outlook.com (2603:10a6:102:205::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 14:34:03 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4566.017; Thu, 30 Sep 2021
 14:34:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82835e69-21fb-11ec-bd4e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=x/Go2rrV/CU19V32/sbDJzpTN+DxJv09PRmwy2pct5Q=;
 b=+IypazHw6IAwmOadB49aATZ6ZqCzFiN/atClH2prO33eQcPNNlIankkoKe/PjndX2vztVUXf234nnMDF+QVPdodwhzpQyfcl/pdfPE8zv9SgjYRh+Mh0rjNh/ButHZl0qrRVWShft9QhDq9HEaYom1F07M75v9JQDt/q7F9HT3k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.33.187.114)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.33.187.114 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.33.187.114; helo=64aa7808-outbound-2.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: d7116eb65465f56a
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BPPCpLsLnazcTOcrYuKpyd0yhy0Txi+nfGiddnXSDACyjJ5OVOVEAQUjrj/gbzAVm7WPjP2LUImYS2hEnQuoZ+43BVynzkpoX9P2PN+Sjz+xQB2MOWFx+uriHu70nqn8KkV8kSTC/i8Q9upAAISggipfP0DUlshx+xXqMv3YHeYnb/SO+7ktvSIbrEzq3j3dWHBHjSf7whFnsksZcdCewlkG76844VXm91IdVubd2qUnWpmVr3Y9mOqBMzUa+7dIdVHAR21gISYV61tZg00IAdaMndi6uSub5pteyw3nA4VW2IR3fPwvZwncIGp6prUrFHwjOY51/5cUQ8z6lXWIuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=x/Go2rrV/CU19V32/sbDJzpTN+DxJv09PRmwy2pct5Q=;
 b=YntlPN6DtYAxNMmtTP9F+2pHuy9iup/IBLnWtma+YDgdR4mgl5n8obeUnoqrObucnzd8Uc5ihYQTStbz6ZHEx4ZRp9GUyUnrGmA3SQVAclxZicYgxgXqGhLjghscsnjbYZZN2qJzqJQvBwosFHVRav1wya5YtNmaqm3u9+lg8IKJDB4Lh37EdM/rxXp+H9yMGHOWk0bTIvpTrpKRF8PlvHbzCac6XUJF4qwTYGIV04dL4q7KjFKqPMLgKzJ5+XtmJDmjAY/VeC0VIQVMk0lQ3MQDg8NAohGmf7cUtpdTIcUIC6nMqkv2bARt97ijbacPcyTXy+6E4iENXp7TVuXF+w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=x/Go2rrV/CU19V32/sbDJzpTN+DxJv09PRmwy2pct5Q=;
 b=+IypazHw6IAwmOadB49aATZ6ZqCzFiN/atClH2prO33eQcPNNlIankkoKe/PjndX2vztVUXf234nnMDF+QVPdodwhzpQyfcl/pdfPE8zv9SgjYRh+Mh0rjNh/ButHZl0qrRVWShft9QhDq9HEaYom1F07M75v9JQDt/q7F9HT3k=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Wei Chen <Wei.Chen@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH v4 3/3] arm/efi: load dom0 modules from DT using UEFI
Thread-Topic: [PATCH v4 3/3] arm/efi: load dom0 modules from DT using UEFI
Thread-Index: AQHXtgeNfczdAxKdc02vjouzHO+KY6u8pF6A
Date: Thu, 30 Sep 2021 14:34:03 +0000
Message-ID: <21DC7A33-876E-409C-9A8A-F5263D29CA2D@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>
 <20210930142846.13348-4-luca.fancellu@arm.com>
In-Reply-To: <20210930142846.13348-4-luca.fancellu@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: 85c5a054-5b77-4b51-f7a1-08d9841f659b
x-ms-traffictypediagnostic: PAXPR08MB7033:|AS8PR08MB6696:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<AS8PR08MB6696A8901C01693BEC5FFCCE9DAA9@AS8PR08MB6696.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:4941;OLM:4941;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 YDqow4uzVBtFqJA9mp8APe477GHGksIVHcC88EmMiSfemgRkL2nVn6TCmiDNibkLlUHSg11zglnsOHdIPToqwrRFA95GgHbI9dP5lqsHHPbac+U2HWR8dekw0Htec8pHm+fivS4t5PSvHsMw4m8CXw+QaENWTAtYTKc3+5v+T0DiP0VacT7Z/WFCcs/+SLZcz93NQgAUn3QpGrAyF+7d0nZ/Zq4A0qK45G5h3HD15+JG0IHn76WHd+vAhxlqMeorXJ6hlgSZu4yKeMtWlVEpKJecp616TkUrV0EpUNfldREQr/tE/ged8Sn+PiTmOTAayW+GcingukmEe59uxIjncPCq2NzeBBtvBwk39uPGFlNnvYflAUCkcJMZWDYHUvcfB+T2HVCEQ23Lc4ZwS5A51320UMMmB8rFNSxpdBfa/6a2NQ+PdcOTawXVGMbML9pmUiC5KpuD1irnSoeiYQna2JfF4UHEDfZr5uyvwnN84b9cvEt2r6uc0rNR3A8/2y9y1DFnyFNgBu1CgDcrQWG1NPDVjsqf7um7Gc+zPMkgI5Ui9ndpLrKobM8Bi8bjrfdGvGQksBOZpjeghYSlvuwvmCTOuEc18hiCxj8sqFvJ0RAGKf/WjtZgHSPQ6L3wdtTGq82E+Bcbo+H3tgH11jTOGITKkq/qX9ECFFbGJPG0ECqAEj6i/wj/KV2II3dZr8qbgCAX2jx+jvQfWxXQuAoJWJ1nABMs6N4rWvbpLO2Ohx7/42DIikEzFsGxpr1PBlyGth1qWtrioQE67uWJ3MM0rQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2616005)(83380400001)(508600001)(36756003)(186003)(6636002)(33656002)(86362001)(122000001)(5660300002)(4326008)(37006003)(38100700002)(6486002)(26005)(54906003)(6862004)(76116006)(71200400001)(53546011)(6512007)(38070700005)(66446008)(64756008)(66556008)(66476007)(91956017)(66946007)(8936002)(2906002)(6506007)(8676002)(316002)(21314003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <6D6D1CC7C0C0204BA293C6D3E44BAB3F@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7033
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2aa19fd4-aa03-4792-37ec-08d9841f59d0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3EGiq3lZvz9HoIV+crRxczqAd6u/qw6Ct6EU/GT9ILWpHDJ5cJO0Iy/3bZIWmmJBdS9CKsc4ZMbIJEjLHH+LG1EaPRjIFSQSY3TG0CSd7y5ixy3my3oqhlSR88HtjfTlv5GcAG4sKUgptmbjpE1Im4KJLwuo0izANoHpFitDCzRkKBvzm+W0pmacFIQxidUHuSwljoaxoVv20jtM9ZrLC9Y/5TfqusxQvhq5u2PVM487+VBf8dyCSgCIpzqICs87cVDwD1lOY7zfDP9n00jMwD1FjMxKSAF4xuF7slk0fj4KZXT4Vt8f1wcDkUeVYNBOhUgcCdDmsf4EBCvmOIXUr5t8iqtPZaeLjVgmlfjZS02LZ9l9I0KsGO27KDWQYvLhUk+01sFn1Qlbmi8LUp2mgg2XNr1f28AQWRSHSqW2N/EVCTGhU9rBkmFbed52p7dhFTV2gdi9duqAGzBOhh9zP8yI85G7DgnlNVoHj0OV3s6c4UjB8e870kBhM2l9Xro7zZvLSbTJ/uhMsbQtpPWhwwIKVNZsuZpGQFmj3YOENUEq1tSdVSjtnsUCvcY8ZyxFP5Ny0V6Jh0uyvOpG6xDjwzhR8GLv5MBq1z2bWeD3cNYlV9DtAToiE6ilrOtvXm2qRh08evba/tv2SjD07GOR78q7ixkHKXAE+cordZJEppkp+2DpKefovsUyk+1WIcxYgDZYSW2SmaMY2GUP4P9plHUBxVSHzBb2NJbhksUGb0M=
X-Forefront-Antispam-Report:
	CIP:63.33.187.114;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-2.mta.getcheckrecipient.com;PTR:ec2-63-33-187-114.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(86362001)(70206006)(8936002)(82310400003)(70586007)(6506007)(53546011)(6512007)(83380400001)(186003)(2616005)(336012)(26005)(4326008)(8676002)(33656002)(5660300002)(6862004)(36860700001)(6636002)(2906002)(356005)(36756003)(508600001)(54906003)(37006003)(316002)(47076005)(6486002)(81166007)(21314003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:34:23.3138
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 85c5a054-5b77-4b51-f7a1-08d9841f659b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.33.187.114];Helo=[64aa7808-outbound-2.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6696

Hi Luca,

> On 30 Sep 2021, at 15:28, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
>=20
> Add support to load Dom0 boot modules from
> the device tree using the xen,uefi-binary property.
>=20
> Update documentation about that.
>=20
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Changes in v4:
> - Add check to avoid double definition of Dom0 ramdisk
> from cfg file and DT
> - Fix if conditions indentation in boot.c
> - Moved Dom0 kernel verification code after check for
> presence for Dom0 or DomU(s)
> - Changed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - new patch
> ---
> docs/misc/arm/device-tree/booting.txt |  8 ++++
> docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
> xen/arch/arm/efi/efi-boot.h           | 47 ++++++++++++++++++--
> xen/common/efi/boot.c                 | 16 ++++---
> 4 files changed, 123 insertions(+), 12 deletions(-)
>=20
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device=
-tree/booting.txt
> index 7258e7e1ec..8e0c327ba4 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -70,6 +70,14 @@ Each node contains the following properties:
> 	priority of this field vs. other mechanisms of specifying the
> 	bootargs for the kernel.
>=20
> +- uefi,binary (UEFI boot only)
> +
> +	String property that specifies the file name to be loaded by the UEFI
> +	boot for this module. If this is specified, there is no need to specify
> +	the reg property because it will be created by the UEFI stub on boot.
> +	This option is needed only when UEFI boot is used, the node needs to be
> +	compatible with multiboot,kernel or multiboot,ramdisk.
> +
> Examples
> =3D=3D=3D=3D=3D=3D=3D=3D
>=20
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index 876cd55005..4abbb5bb82 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,6 +167,28 @@ sbsign \
> 	--output xen.signed.efi \
> 	xen.unified.efi
> ```
> +## UEFI boot and Dom0 modules on ARM
> +
> +When booting using UEFI on ARM, it is possible to specify the Dom0 modul=
es
> +directly from the device tree without using the Xen configuration file, =
here an
> +example:
> +
> +chosen {
> +	#size-cells =3D <0x1>;
> +	#address-cells =3D <0x1>;
> +	xen,xen-bootargs =3D "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible =3D "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary =3D "vmlinuz-3.0.31-0.4-xen";
> +		bootargs =3D "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary =3D "initrd-3.0.31-0.4-xen";
> +	};
> +}
>=20
> ## UEFI boot and dom0less on ARM
>=20
> @@ -326,10 +348,10 @@ chosen {
> ### Boot Xen, Dom0 and DomU(s)
>=20
> This configuration is a mix of the two configuration above, to boot this =
one
> -the configuration file must be processed so the /chosen node must have t=
he
> -"xen,uefi-cfg-load" property.
> +the configuration file can be processed or the Dom0 modules can be read =
from
> +the device tree.
>=20
> -Here an example:
> +Here the first example:
>=20
> Xen configuration file:
>=20
> @@ -369,4 +391,40 @@ chosen {
> };
> ```
>=20
> +Here the second example:
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells =3D <0x1>;
> +	#address-cells =3D <0x1>;
> +	xen,xen-bootargs =3D "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible =3D "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary =3D "vmlinuz-3.0.31-0.4-xen";
> +		bootargs =3D "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible =3D "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary =3D "initrd-3.0.31-0.4-xen";
> +	};
> +
> +	domU1 {
> +		#size-cells =3D <0x1>;
> +		#address-cells =3D <0x1>;
> +		compatible =3D "xen,domain";
> +		cpus =3D <0x1>;
> +		memory =3D <0x0 0xc0000>;
> +		vpl011;
>=20
> +		module@1 {
> +			compatible =3D "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary =3D "Image-domu1.bin";
> +			bootargs =3D "console=3DttyAMA0 root=3D/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 50b3829ae0..b122e2846a 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -31,8 +31,11 @@ static unsigned int __initdata modules_idx;
> #define ERROR_MISSING_DT_PROPERTY   (-3)
> #define ERROR_RENAME_MODULE_NAME    (-4)
> #define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DOM0_ALREADY_FOUND    (-6)
> +#define ERROR_DOM0_RAMDISK_FOUND    (-7)
> #define ERROR_DT_MODULE_DOMU        (-1)
> #define ERROR_DT_CHOSEN_NODE        (-2)
> +#define ERROR_DT_MODULE_DOM0        (-3)
>=20
> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
> void __flush_dcache_area(const void *vaddr, unsigned long size);
> @@ -45,7 +48,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_han=
dle,
> static int handle_module_node(EFI_FILE_HANDLE dir_handle,
>                               int module_node_offset,
>                               int reg_addr_cells,
> -                              int reg_size_cells);
> +                              int reg_size_cells,
> +                              bool is_domu_module);
> static bool is_boot_module(int dt_module_offset);
> static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>                                        int domain_node);
> @@ -701,7 +705,8 @@ static int __init allocate_module_file(EFI_FILE_HANDL=
E dir_handle,
> static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>                                      int module_node_offset,
>                                      int reg_addr_cells,
> -                                     int reg_size_cells)
> +                                     int reg_size_cells,
> +                                     bool is_domu_module)
> {
>     const void *uefi_name_prop;
>     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \=
0 */
> @@ -743,6 +748,34 @@ static int __init handle_module_node(EFI_FILE_HANDLE=
 dir_handle,
>         return ERROR_SET_REG_PROPERTY;
>     }
>=20
> +    if ( !is_domu_module )
> +    {
> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
> +                                    "multiboot,kernel") =3D=3D 0) )
> +        {
> +            /*
> +            * This is the Dom0 kernel, wire it to the kernel variable be=
cause it
> +            * will be verified by the shim lock protocol later in the co=
mmon
> +            * code.
> +            */
> +            if ( kernel.addr )
> +            {
> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
> +                return ERROR_DOM0_ALREADY_FOUND;
> +            }
> +            kernel.need_to_free =3D false; /* Freed using the module arr=
ay */
> +            kernel.addr =3D file->addr;
> +            kernel.size =3D file->size;
> +        }
> +        else if ( ramdisk.addr &&
> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> +                                             "multiboot,ramdisk") =3D=3D=
 0) )
> +        {
> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> +            return ERROR_DOM0_RAMDISK_FOUND;
> +        }
> +    }
> +
>     return 0;
> }
>=20
> @@ -799,7 +832,7 @@ static int __init handle_dom0less_domain_node(EFI_FIL=
E_HANDLE dir_handle,
>         if ( is_boot_module(module_node) )
>         {
>             int ret =3D handle_module_node(dir_handle, module_node, addr_=
cells,
> -                                         size_cells);
> +                                         size_cells, true);
>             if ( ret < 0 )
>                 return ret;
>         }
> @@ -809,7 +842,7 @@ static int __init handle_dom0less_domain_node(EFI_FIL=
E_HANDLE dir_handle,
>=20
> /*
>  * This function checks for xen domain nodes under the /chosen node for p=
ossible
> - * domU guests to be loaded.
> + * dom0 and domU guests to be loaded.
>  * Returns the number of modules loaded or a negative number for error.
>  */
> static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> @@ -836,6 +869,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_HA=
NDLE dir_handle)
>             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>                 return ERROR_DT_MODULE_DOMU;
>         }
> +        else if ( is_boot_module(node) )
> +        {
> +            if ( handle_module_node(dir_handle, node, addr_len, size_len=
,
> +                                    false) < 0 )
> +                return ERROR_DT_MODULE_DOM0;
> +        }
>     }
>=20
>     /* Free boot modules file names if any */
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index daf7c26099..e4295dbe34 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>         {
>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>             efi_bs->FreePool(name.w);
> -
> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL=
,
> -                            (void **)&shim_lock)) &&
> -                 (status =3D shim_lock->Verify(kernel.ptr, kernel.size))=
 !=3D EFI_SUCCESS )
> -                PrintErrMesg(L"Dom0 kernel image could not be verified",=
 status);
>         }
>=20
>         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
> @@ -1385,6 +1380,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE=
 *SystemTable)
>     if ( !dt_modules_found && !kernel.addr )
>         blexit(L"No Dom0 kernel image specified.");
>=20
> +    /*
> +     * The Dom0 kernel can be loaded from the configuration file or by t=
he
> +     * device tree through the efi_arch_check_dt_boot function, in this =
stage
> +     * verify it.
> +     */
> +    if ( kernel.addr &&
> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> +                                           (void **)&shim_lock)) &&
> +         (status =3D shim_lock->Verify(kernel.ptr, kernel.size)) !=3D EF=
I_SUCCESS )
> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status)=
;
> +
>     efi_arch_edd();
>=20
>     /* XXX Collect EDID info. */
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:34:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:34:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200005.354373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx9C-0000hF-R0; Thu, 30 Sep 2021 14:34:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200005.354373; Thu, 30 Sep 2021 14:34:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVx9C-0000h8-Nn; Thu, 30 Sep 2021 14:34:54 +0000
Received: by outflank-mailman (input) for mailman id 200005;
 Thu, 30 Sep 2021 14:34:54 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YInd=OU=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVx9C-0000e5-96
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:34:54 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown
 [40.107.8.43]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 9297bc4b-21fb-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:34:53 +0000 (UTC)
Received: from AS9PR06CA0213.eurprd06.prod.outlook.com (2603:10a6:20b:45e::12)
 by PA4PR08MB6032.eurprd08.prod.outlook.com (2603:10a6:102:e4::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep
 2021 14:34:51 +0000
Received: from AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:45e:cafe::2d) by AS9PR06CA0213.outlook.office365.com
 (2603:10a6:20b:45e::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14 via Frontend
 Transport; Thu, 30 Sep 2021 14:34:50 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT037.mail.protection.outlook.com (10.152.17.241) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 14:34:50 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 30 Sep 2021 14:33:46 +0000
Received: from 7c533c432ce7.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 813328F4-B15B-4199-8845-86E9136F7F88.1; 
 Thu, 30 Sep 2021 14:33:33 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7c533c432ce7.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 14:33:33 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6365.eurprd08.prod.outlook.com (2603:10a6:102:156::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 14:33:30 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4566.017; Thu, 30 Sep 2021
 14:33:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9297bc4b-21fb-11ec-bd4e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=jgYDGAC+1s0Q9PxnlZ0/PVMFZzM3gA2FxvfVwgbK2wLRPdoxy07UPgHYa9+ciqPnoT4NVK7fTtoteTJXjEN0a7TZOhnz3cNcE20AgIGsAaaoMVYGTiCdOlC9aPlTtzZcyBOQIglVaFPQYuwD2tl22QiXtT8qPCpKItP8j4RgfYo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e92ac0beda1af1e9
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HuXUygolcnfHsGvUGqvBBhXFHiZ5OCFVPbSfHEekKiSg4PtI5PDSEbwTM7so4F8aZCFUZopOiAau1FN+fGxlMesU5w56Inb/IFnMHt+hsTmCebmn/sz9i7Jm2Pq2HAEbOC+6y3wl7w7mLSo75dpCzgtzQ5kLYTsPXcRvK2Bm7FI5XHNT1RDSj8m1NANVT/KQQh3gDoMOuEOz20pugN8vvggU3Z0E9gGQd0I8j1lFVMwAaJLzrOTpgTx0HIvJczwzsiCgUnjxOROYn6oDiUxSIvIcA04xkykXOsxaeB6DhIK60Wru5Kdgm/zhKkT7kF6JdhVqSiE6HFK7H+92KQXasA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=WVcWXJ9stWvVbt/Rnt/PVKoStVU40XOxWQPfkICYvuBu+IyFJnZeuuQSLYNXEWL2HwWUmkpdzVL9v/i1qhGxlU5HBhOS8Hk390lTPdIwSsceia19M+RD7TwknfSdht3pH/AbQpE8OaBwRsChpuMoeiAjwh5b/J5tx8DL2IdynmQWcIv1XseoPKtNe2Eikgqgie+FoQPxiKloVRvnJXb0/7iLzTUcxMPmGYQXrUqw0XNZTwBWthOogQt8UAmU70mjLQfnvtapKlQ2xEriD8Myk0KHYQN2tL0O3ucLv/IK9KnQi+Aibww3P02pYFi4B4SE9Xo1Mf5KUphc1BNiu/wNOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=jgYDGAC+1s0Q9PxnlZ0/PVMFZzM3gA2FxvfVwgbK2wLRPdoxy07UPgHYa9+ciqPnoT4NVK7fTtoteTJXjEN0a7TZOhnz3cNcE20AgIGsAaaoMVYGTiCdOlC9aPlTtzZcyBOQIglVaFPQYuwD2tl22QiXtT8qPCpKItP8j4RgfYo=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Wei Chen <Wei.Chen@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH v4 1/3] arm/efi: Introduce xen,uefi-cfg-load DT property
Thread-Topic: [PATCH v4 1/3] arm/efi: Introduce xen,uefi-cfg-load DT property
Thread-Index: AQHXtgeKm5+W6XbeOUevr2j0oD4u3qu8pDcA
Date: Thu, 30 Sep 2021 14:33:30 +0000
Message-ID: <0A3942A2-C668-4BDD-A1A0-D6EFD2738560@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>
 <20210930142846.13348-2-luca.fancellu@arm.com>
In-Reply-To: <20210930142846.13348-2-luca.fancellu@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: bcb5bbce-548d-4c1e-9794-08d9841f75c7
x-ms-traffictypediagnostic: PAXPR08MB6365:|PA4PR08MB6032:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<PA4PR08MB6032EDE507FB49BE2BC9BC4B9DAA9@PA4PR08MB6032.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rj0dKr6twHPE9X6cOtOaMOz3qo2WFQTb655aha2g2rQGX/I/WUa+gtZJ9rMRpnLrgEPTfGNJnJb7FylvjJTZqYulJWI7NKnw5vAW90RiYF7m9mFxWayH3hXFR9NuLafFqQapHUe7UG5OeIm2tMUYRoNXkjnhFZR5e1Zzwj7QonVaI/akGHjdBYPn0pwXAkFKugsiaLORxoHFOfvS+VFtRoFtw5Ggvb0tso9qt7WNKpdYbAzTLauHZ4Ke663Z9a6udFPYY5M1vv49IP6yzBUDXrS7uI1UWWbsuy/xlr811PnW3foqPOlYBi6JNAr1PmKrXaTANYeZHKZxfiixi9m2/QsE0f/LOQ4vm+DIxeXJ1vEe+oczJFr/+HzUP5OPNHQii2fNNwuFJ5dZzuEbv4MG0QPzKdmWwMxFOKid3VuplyhRJfQFHdrtKjg99N84p4OZGIinm8SiRBoecA60vBEgBbdELHx72X7wyPClfofGcYLo9F/lrar4kR1SgggYeDkCCuQzfUg6XSI868MiB80QXcWJMMHomi48wR/7ipzgvVoe2rnwJObqMlKYDnRcZ6llbt+X3owGMcrA2V6yWCtv/2XM0ujFHJS1uWsu03WkLNipCrbaBhXJG7WmSVwCSY2bcuGKgSARWh8R/ZXGKmPHV305UEa/hWt2jZiw8cTjccqLPADMZNPSMbnO7zfyU24izZsuNSDG7oKD4xV/lZAb5eZxXlTeP9XjlqQvQ7ADwGhEICBSFwK5mjCnqJMA9FLO
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(6862004)(6486002)(6512007)(508600001)(71200400001)(53546011)(37006003)(54906003)(2906002)(186003)(316002)(4326008)(26005)(2616005)(6636002)(6506007)(8936002)(83380400001)(76116006)(91956017)(64756008)(66946007)(66556008)(66476007)(66446008)(86362001)(33656002)(38070700005)(38100700002)(122000001)(5660300002)(36756003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7D0F0FB8D4138F4CB582D24B708E264D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6365
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c39bdc37-e2f2-4613-3b50-08d9841f4658
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wYnGC8x5tBlRHWd8p+kijtSm5CpwKbFvQoovqCXE0j6p8GeY2+Ym32pMH+CssBOL9C6pFIek7gI61/Lew+tACTkevXpMOSiFmh+DG2L4wi0wOKowNN4watN/5DpdC++YRFn9AshJWJ/MSf4uPlpQkp5vNLlAPrLXJp4yFjVfjwVgVtNKoOWMQbxyo2+ql2eR2uZagJT8ac0u1MuFu1ZeykfYq0E+oScPSK0GTL+W489Vn/f1IBtFINDqU5RjKvhbgbDVPRdRWIeKNRTKkkdKkD/edaLCRPL/PLl9HvHGqqpq4+NcF5fEdyyQQds3ntW1BPJAdfrGK+cptl+YKSohj9bYJTPOFg6vPukxl4HRodE1hYR3qu1QNsaHRNoL5AH3YIVbgFCQqXdrmfONpi5VlEmD7SnCsFd7CsbY6IxXTfcS/21oQsXVI9V0WBeJud6K4qAwlVTrrreNJnBEekNraRovAcFHEc5u0n5xlY15hsJEtHk8HuoMQa2+XEPGiQx6IwAj1v5tQAHqpEdt154CYPSVjQX5Oc+W0xfqiKeGVgyF/7ugYeTM3Gb9gF9vMkFA8TgCQHpAcu7vbCDzagg872zKVEM6nKSnhYAX+Mc3cziMYWjvS3KS9bSGrdKB7u6iyfRlBOxVYX7RPGTokCoKtlmyDhnpo6RtLllBTKqTkGLAhScsiEN9ZMxoS3eQ4jumYeL2vjSY/AKnpv2csIhqdw==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(26005)(70586007)(36860700001)(6862004)(316002)(86362001)(81166007)(336012)(37006003)(508600001)(54906003)(2906002)(33656002)(2616005)(6512007)(8676002)(82310400003)(4326008)(186003)(53546011)(8936002)(6506007)(47076005)(83380400001)(5660300002)(36756003)(356005)(6636002)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:34:50.3943
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: bcb5bbce-548d-4c1e-9794-08d9841f75c7
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6032

Hi Luca,

> On 30 Sep 2021, at 15:28, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
>=20
> Introduce the xen,uefi-cfg-load DT property of /chosen
> node for ARM whose presence decide whether to force
> the load of the UEFI Xen configuration file.
>=20
> The logic is that if any multiboot,module is found in
> the DT, then the xen,uefi-cfg-load property is used to see
> if the UEFI Xen configuration file is needed.
>=20
> Modify a comment in efi_arch_use_config_file, removing
> the part that states "dom0 required" because it's not
> true anymore with this commit.
>=20
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> v4 changes:
> - modify property name to xen,uefi-cfg-load
> v3 changes:
> - add documentation to misc/arm/device-tree/booting.txt
> - Modified variable name and logic from skip_cfg_file to
> load_cfg_file
> - Add in the commit message that I'm modifying a comment.
> v2 changes:
> - Introduced uefi,cfg-load property
> - Add documentation about the property
> ---
> docs/misc/arm/device-tree/booting.txt |  8 ++++++++
> docs/misc/efi.pandoc                  |  2 ++
> xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
> 3 files changed, 33 insertions(+), 5 deletions(-)
>=20
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device=
-tree/booting.txt
> index 44cd9e1a9a..352b0ec43a 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs fo=
r Xen, xen,dom0-bootargs
> for Dom0 and bootargs for native Linux.
>=20
>=20
> +UEFI boot and DT
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> +
> +When Xen is booted using UEFI, it doesn't read the configuration file if=
 any
> +multiboot module is specified. To force Xen to load the configuration fi=
le, the
> +boolean property xen,uefi-cfg-load must be declared in the /chosen node.
> +
> +
> Creating Multiple Domains directly from Xen
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ac3cd58cae..ed85351541 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tre=
e provided to Xen.  If a
> bootloader provides a device tree containing modules then any configurati=
on
> files are ignored, and the bootloader is responsible for populating all
> relevant device tree nodes.
> +The property "xen,uefi-cfg-load" can be specified in the /chosen node to=
 force
> +Xen to load the configuration file even if multiboot modules are found.
>=20
> Once built, `make install-xen` will place the resulting binary directly i=
nto
> the EFI boot partition, provided `EFI_VENDOR` is set in the environment (=
and
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..a3e46453d4 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOA=
DED_IMAGE *loaded_image)
>=20
> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable=
)
> {
> +    bool load_cfg_file =3D true;
>     /*
>      * For arm, we may get a device tree from GRUB (or other bootloader)
>      * that contains modules that have already been loaded into memory.  =
In
> -     * this case, we do not use a configuration file, and rely on the
> -     * bootloader to have loaded all required modules and appropriate
> -     * options.
> +     * this case, we search for the property xen,uefi-cfg-load in the /c=
hosen
> +     * node to decide whether to skip the UEFI Xen configuration file or=
 not.
>      */
>=20
>     fdt =3D lookup_fdt_config_table(SystemTable);
>     dtbfile.ptr =3D fdt;
>     dtbfile.need_to_free =3D false; /* Config table memory can't be freed=
. */
> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module=
") < 0 )
> +
> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
> +    {
> +        /* Locate chosen node */
> +        int node =3D fdt_subnode_offset(fdt, 0, "chosen");
> +        const void *cfg_load_prop;
> +        int cfg_load_len;
> +
> +        if ( node > 0 )
> +        {
> +            /* Check if xen,uefi-cfg-load property exists */
> +            cfg_load_prop =3D fdt_getprop(fdt, node, "xen,uefi-cfg-load"=
,
> +                                        &cfg_load_len);
> +            if ( !cfg_load_prop )
> +                load_cfg_file =3D false;
> +        }
> +    }
> +
> +    if ( !fdt || load_cfg_file )
>     {
>         /*
>          * We either have no FDT, or one without modules, so we must have=
 a
> -         * Xen EFI configuration file to specify modules.  (dom0 require=
d)
> +         * Xen EFI configuration file to specify modules.
>          */
>         return true;
>     }
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:39:13 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:39:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200018.354384 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxDK-0001g4-Gb; Thu, 30 Sep 2021 14:39:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200018.354384; Thu, 30 Sep 2021 14:39:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxDK-0001fx-Db; Thu, 30 Sep 2021 14:39:10 +0000
Received: by outflank-mailman (input) for mailman id 200018;
 Thu, 30 Sep 2021 14:39:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YInd=OU=arm.com=Bertrand.Marquis@srs-us1.protection.inumbo.net>)
 id 1mVxDJ-0001fr-4x
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:39:09 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown
 [40.107.7.57]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 2ae94203-21fc-11ec-bd4e-12813bfff9fa;
 Thu, 30 Sep 2021 14:39:08 +0000 (UTC)
Received: from AM7PR02CA0007.eurprd02.prod.outlook.com (2603:10a6:20b:100::17)
 by HE1PR08MB2860.eurprd08.prod.outlook.com (2603:10a6:7:38::29) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Thu, 30 Sep
 2021 14:33:50 +0000
Received: from VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:100:cafe::be) by AM7PR02CA0007.outlook.office365.com
 (2603:10a6:20b:100::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend
 Transport; Thu, 30 Sep 2021 14:33:50 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 VE1EUR03FT015.mail.protection.outlook.com (10.152.18.176) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 14:33:49 +0000
Received: ("Tessian outbound a492f2284909:v103");
 Thu, 30 Sep 2021 14:33:46 +0000
Received: from 7c533c432ce7.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 813328F4-B15B-4199-8845-86E9136F7F88.1; 
 Thu, 30 Sep 2021 14:33:33 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7c533c432ce7.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 14:33:33 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com (2603:10a6:102:12d::10)
 by PAXPR08MB6365.eurprd08.prod.outlook.com (2603:10a6:102:156::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.14; Thu, 30 Sep
 2021 14:33:30 +0000
Received: from PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f]) by PAXPR08MB6446.eurprd08.prod.outlook.com
 ([fe80::c029:ed0:82e7:3c2f%6]) with mapi id 15.20.4566.017; Thu, 30 Sep 2021
 14:33:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ae94203-21fc-11ec-bd4e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=jgYDGAC+1s0Q9PxnlZ0/PVMFZzM3gA2FxvfVwgbK2wLRPdoxy07UPgHYa9+ciqPnoT4NVK7fTtoteTJXjEN0a7TZOhnz3cNcE20AgIGsAaaoMVYGTiCdOlC9aPlTtzZcyBOQIglVaFPQYuwD2tl22QiXtT8qPCpKItP8j4RgfYo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: e92ac0beda1af1e9
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HuXUygolcnfHsGvUGqvBBhXFHiZ5OCFVPbSfHEekKiSg4PtI5PDSEbwTM7so4F8aZCFUZopOiAau1FN+fGxlMesU5w56Inb/IFnMHt+hsTmCebmn/sz9i7Jm2Pq2HAEbOC+6y3wl7w7mLSo75dpCzgtzQ5kLYTsPXcRvK2Bm7FI5XHNT1RDSj8m1NANVT/KQQh3gDoMOuEOz20pugN8vvggU3Z0E9gGQd0I8j1lFVMwAaJLzrOTpgTx0HIvJczwzsiCgUnjxOROYn6oDiUxSIvIcA04xkykXOsxaeB6DhIK60Wru5Kdgm/zhKkT7kF6JdhVqSiE6HFK7H+92KQXasA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=WVcWXJ9stWvVbt/Rnt/PVKoStVU40XOxWQPfkICYvuBu+IyFJnZeuuQSLYNXEWL2HwWUmkpdzVL9v/i1qhGxlU5HBhOS8Hk390lTPdIwSsceia19M+RD7TwknfSdht3pH/AbQpE8OaBwRsChpuMoeiAjwh5b/J5tx8DL2IdynmQWcIv1XseoPKtNe2Eikgqgie+FoQPxiKloVRvnJXb0/7iLzTUcxMPmGYQXrUqw0XNZTwBWthOogQt8UAmU70mjLQfnvtapKlQ2xEriD8Myk0KHYQN2tL0O3ucLv/IK9KnQi+Aibww3P02pYFi4B4SE9Xo1Mf5KUphc1BNiu/wNOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=19xI88ys4R3Ukt/N9WTN1BTeGYle4keyVGvSxbzotoQ=;
 b=jgYDGAC+1s0Q9PxnlZ0/PVMFZzM3gA2FxvfVwgbK2wLRPdoxy07UPgHYa9+ciqPnoT4NVK7fTtoteTJXjEN0a7TZOhnz3cNcE20AgIGsAaaoMVYGTiCdOlC9aPlTtzZcyBOQIglVaFPQYuwD2tl22QiXtT8qPCpKItP8j4RgfYo=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Luca Fancellu <Luca.Fancellu@arm.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Wei Chen <Wei.Chen@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Ian
 Jackson <iwj@xenproject.org>, Jan Beulich <jbeulich@suse.com>, Wei Liu
	<wl@xen.org>
Subject: Re: [PATCH v4 1/3] arm/efi: Introduce xen,uefi-cfg-load DT property
Thread-Topic: [PATCH v4 1/3] arm/efi: Introduce xen,uefi-cfg-load DT property
Thread-Index: AQHXtgeKm5+W6XbeOUevr2j0oD4u3qu8pDcA
Date: Thu, 30 Sep 2021 14:33:30 +0000
Message-ID: <0A3942A2-C668-4BDD-A1A0-D6EFD2738560@arm.com>
References: <20210930142846.13348-1-luca.fancellu@arm.com>
 <20210930142846.13348-2-luca.fancellu@arm.com>
In-Reply-To: <20210930142846.13348-2-luca.fancellu@arm.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
Authentication-Results-Original: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: a5e59db4-39b5-4db4-130f-08d9841f51b4
x-ms-traffictypediagnostic: PAXPR08MB6365:|HE1PR08MB2860:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS:
	<HE1PR08MB2860D536932D2518E45299A29DAA9@HE1PR08MB2860.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rj0dKr6twHPE9X6cOtOaMOz3qo2WFQTb655aha2g2rQGX/I/WUa+gtZJ9rMRpnLrgEPTfGNJnJb7FylvjJTZqYulJWI7NKnw5vAW90RiYF7m9mFxWayH3hXFR9NuLafFqQapHUe7UG5OeIm2tMUYRoNXkjnhFZR5e1Zzwj7QonVaI/akGHjdBYPn0pwXAkFKugsiaLORxoHFOfvS+VFtRoFtw5Ggvb0tso9qt7WNKpdYbAzTLauHZ4Ke663Z9a6udFPYY5M1vv49IP6yzBUDXrS7uI1UWWbsuy/xlr811PnW3foqPOlYBi6JNAr1PmKrXaTANYeZHKZxfiixi9m2/QsE0f/LOQ4vm+DIxeXJ1vEe+oczJFr/+HzUP5OPNHQii2fNNwuFJ5dZzuEbv4MG0QPzKdmWwMxFOKid3VuplyhRJfQFHdrtKjg99N84p4OZGIinm8SiRBoecA60vBEgBbdELHx72X7wyPClfofGcYLo9F/lrar4kR1SgggYeDkCCuQzfUg6XSI868MiB80QXcWJMMHomi48wR/7ipzgvVoe2rnwJObqMlKYDnRcZ6llbt+X3owGMcrA2V6yWCtv/2XM0ujFHJS1uWsu03WkLNipCrbaBhXJG7WmSVwCSY2bcuGKgSARWh8R/ZXGKmPHV305UEa/hWt2jZiw8cTjccqLPADMZNPSMbnO7zfyU24izZsuNSDG7oKD4xV/lZAb5eZxXlTeP9XjlqQvQ7ADwGhEICBSFwK5mjCnqJMA9FLO
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6446.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(8676002)(6862004)(6486002)(6512007)(508600001)(71200400001)(53546011)(37006003)(54906003)(2906002)(186003)(316002)(4326008)(26005)(2616005)(6636002)(6506007)(8936002)(83380400001)(76116006)(91956017)(64756008)(66946007)(66556008)(66476007)(66446008)(86362001)(33656002)(38070700005)(38100700002)(122000001)(5660300002)(36756003)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7D0F0FB8D4138F4CB582D24B708E264D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6365
Original-Authentication-Results: arm.com; dkim=none (message not signed)
 header.d=none;arm.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	c39bdc37-e2f2-4613-3b50-08d9841f4658
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/Iem90vf9QdNeABahpCH+6NDW0q6/An48FJAWsbKSLk/tQN4HsYX4CLs4z2lISx51FRJTDA4WVyEczkblU50gYwlSdpCRNqwTIMQrpaqxjFFymOXInRVMIYLgRvEVcvGQrL3qTbI1qbt7lyJjYrJXb20ktoMqyNM4DxSB+cRtY1C6xP8LMo9xLA9bb3mNjkOyI0MBDuP2QFQY4L1LYqv0r7xk583U3gFyP8lf1p5jOSq05I8s8O8Xn0Ee2rDBXfUvxGLH0fyNHp2dhd7mWnDjyd1S7FSlxi0UD6MJOHJLUlVn5rMnSKbsM5SIEq8K8bbCTL+z0NPfnFuH5DZw3+GhBicMmvGgc1WyVTDXNvuZJ9aYA+r39XpejLGQSOw0hqyMD3wE8Vjxu1/f73qQiEcwW5l58lnmt2yDLCTs3lqBA5S/6VPgUiVryLAi4F5SKuzttbpvMwtKlwQaXTx0mI31rN8elBx+MEWuEBh8yHRUZDykRbx9RqUDX0nardwjoaygaYrNWNXGIfqUkZBg/aNWwt/80PKsaLoXkLTbZOYJLwO7VAVwvigkDeeVHc9uSMOqY9E9qVhj+w2Xm7AXshY9lzMq7LMJ1vs/xVFZk1c7oyaCk/c649xm9gqC6D1gcFlan/4V0ZCGf8n4GjBTBC6dRyt1/6zF6LiBufFuFMxB/Gv9oz9PfHQcICoUJgtGt6WmJ2HiuzW6fvdlyDLYu2Y5A==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(82310400003)(6862004)(81166007)(8936002)(83380400001)(2616005)(356005)(6636002)(316002)(508600001)(37006003)(54906003)(86362001)(26005)(53546011)(336012)(6512007)(6506007)(5660300002)(47076005)(36756003)(70206006)(70586007)(36860700001)(4326008)(8676002)(186003)(33656002)(2906002)(6486002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:33:49.8144
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a5e59db4-39b5-4db4-130f-08d9841f51b4
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	VE1EUR03FT015.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR08MB2860

Hi Luca,

> On 30 Sep 2021, at 15:28, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
>=20
> Introduce the xen,uefi-cfg-load DT property of /chosen
> node for ARM whose presence decide whether to force
> the load of the UEFI Xen configuration file.
>=20
> The logic is that if any multiboot,module is found in
> the DT, then the xen,uefi-cfg-load property is used to see
> if the UEFI Xen configuration file is needed.
>=20
> Modify a comment in efi_arch_use_config_file, removing
> the part that states "dom0 required" because it's not
> true anymore with this commit.
>=20
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> v4 changes:
> - modify property name to xen,uefi-cfg-load
> v3 changes:
> - add documentation to misc/arm/device-tree/booting.txt
> - Modified variable name and logic from skip_cfg_file to
> load_cfg_file
> - Add in the commit message that I'm modifying a comment.
> v2 changes:
> - Introduced uefi,cfg-load property
> - Add documentation about the property
> ---
> docs/misc/arm/device-tree/booting.txt |  8 ++++++++
> docs/misc/efi.pandoc                  |  2 ++
> xen/arch/arm/efi/efi-boot.h           | 28 ++++++++++++++++++++++-----
> 3 files changed, 33 insertions(+), 5 deletions(-)
>=20
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device=
-tree/booting.txt
> index 44cd9e1a9a..352b0ec43a 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -121,6 +121,14 @@ A Xen-aware bootloader would set xen,xen-bootargs fo=
r Xen, xen,dom0-bootargs
> for Dom0 and bootargs for native Linux.
>=20
>=20
> +UEFI boot and DT
> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> +
> +When Xen is booted using UEFI, it doesn't read the configuration file if=
 any
> +multiboot module is specified. To force Xen to load the configuration fi=
le, the
> +boolean property xen,uefi-cfg-load must be declared in the /chosen node.
> +
> +
> Creating Multiple Domains directly from Xen
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ac3cd58cae..ed85351541 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -14,6 +14,8 @@ loaded the modules and describes them in the device tre=
e provided to Xen.  If a
> bootloader provides a device tree containing modules then any configurati=
on
> files are ignored, and the bootloader is responsible for populating all
> relevant device tree nodes.
> +The property "xen,uefi-cfg-load" can be specified in the /chosen node to=
 force
> +Xen to load the configuration file even if multiboot modules are found.
>=20
> Once built, `make install-xen` will place the resulting binary directly i=
nto
> the EFI boot partition, provided `EFI_VENDOR` is set in the environment (=
and
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index cf9c37153f..a3e46453d4 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -581,22 +581,40 @@ static void __init efi_arch_load_addr_check(EFI_LOA=
DED_IMAGE *loaded_image)
>=20
> static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable=
)
> {
> +    bool load_cfg_file =3D true;
>     /*
>      * For arm, we may get a device tree from GRUB (or other bootloader)
>      * that contains modules that have already been loaded into memory.  =
In
> -     * this case, we do not use a configuration file, and rely on the
> -     * bootloader to have loaded all required modules and appropriate
> -     * options.
> +     * this case, we search for the property xen,uefi-cfg-load in the /c=
hosen
> +     * node to decide whether to skip the UEFI Xen configuration file or=
 not.
>      */
>=20
>     fdt =3D lookup_fdt_config_table(SystemTable);
>     dtbfile.ptr =3D fdt;
>     dtbfile.need_to_free =3D false; /* Config table memory can't be freed=
. */
> -    if ( !fdt || fdt_node_offset_by_compatible(fdt, 0, "multiboot,module=
") < 0 )
> +
> +    if ( fdt_node_offset_by_compatible(fdt, 0, "multiboot,module") > 0 )
> +    {
> +        /* Locate chosen node */
> +        int node =3D fdt_subnode_offset(fdt, 0, "chosen");
> +        const void *cfg_load_prop;
> +        int cfg_load_len;
> +
> +        if ( node > 0 )
> +        {
> +            /* Check if xen,uefi-cfg-load property exists */
> +            cfg_load_prop =3D fdt_getprop(fdt, node, "xen,uefi-cfg-load"=
,
> +                                        &cfg_load_len);
> +            if ( !cfg_load_prop )
> +                load_cfg_file =3D false;
> +        }
> +    }
> +
> +    if ( !fdt || load_cfg_file )
>     {
>         /*
>          * We either have no FDT, or one without modules, so we must have=
 a
> -         * Xen EFI configuration file to specify modules.  (dom0 require=
d)
> +         * Xen EFI configuration file to specify modules.
>          */
>         return true;
>     }
> --=20
> 2.17.1
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 14:51:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 14:51:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200025.354395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxOz-0003zj-Ke; Thu, 30 Sep 2021 14:51:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200025.354395; Thu, 30 Sep 2021 14:51:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxOz-0003zc-Hg; Thu, 30 Sep 2021 14:51:13 +0000
Received: by outflank-mailman (input) for mailman id 200025;
 Thu, 30 Sep 2021 14:51:11 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVxOx-0003zW-Sn
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 14:51:11 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id d95309f8-21fd-11ec-bd52-12813bfff9fa;
 Thu, 30 Sep 2021 14:51:10 +0000 (UTC)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2055.outbound.protection.outlook.com [104.47.14.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id de-mta-3-XvnH9s1tOgqgOhuJ6aE7tw-1;
 Thu, 30 Sep 2021 16:51:08 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VE1PR04MB6479.eurprd04.prod.outlook.com (2603:10a6:803:11c::33)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Thu, 30 Sep
 2021 14:51:07 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 14:51:07 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6PR01CA0065.eurprd01.prod.exchangelabs.com (2603:10a6:20b:e0::42) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 14:51:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d95309f8-21fd-11ec-bd52-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633013469;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=srFDW2pOcHBYQzELexsxJ7TSCRkMdj8CE4QVd8VrhBs=;
	b=GrY/mM5brBfOBY5uXFTA7GNN/DE+9IDgmCNjhCs9nbtYD+Ho0Vc0TJaRF1lG/SINI1rU4P
	dE44pJKVEfr4yj2/vdm97nrjTkUX+0rjua5Jg0Q+FAK/gl2TR1RoVzO+PACFGtIsCMViD0
	hIjZaYQtq0X/sNN/edmuHddrYyW7dzY=
X-MC-Unique: XvnH9s1tOgqgOhuJ6aE7tw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=R8HDDEWcyccI2B87WJUEQQNDnd3Zdp7deo1iSvhMdT09s8kAmUVSIvWKuUxdfkfvKyXrT3yqMjSVLepx9kt3EcNALL0W9u8pndzoaLcRWtnHsVa/k5qz0J+y6Qr+oK2DkwS8AyFZr876xlJyzsD2BGY4JX/CtaeOrms1lDO9G7JscYkRX4/Zs4dD+qwNvq7SExPPL0hYF28v20IH7fcMjbi/qKmKmzk7ezTpjLXyi8o5yHIl731it3bq8GyWPCjNVS/hcJ0vWSbM1qLX/Rm0Rr1yjZM0ci5qIw8MbffoM+Huy3FBdOffQbNGV70Pn9MhIV8M+XWdLwBjwqTfV1HeDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=srFDW2pOcHBYQzELexsxJ7TSCRkMdj8CE4QVd8VrhBs=;
 b=G/ySFCSu2DtePpKITow1zmyCXpkaf8fKy7HN6sSXBKx22pIBytXP9PKoaGkIqX6dlRvmXkjodPymHnkjGztSJ3DTofgEwLzF/ZVfXQsHUzWUgDGVKzvxEFKm76ScWOIlgJKI9xQcFz0Xsq1NrWAkCTeOXsTxezR46jaMQ1Ju5MSF9JPxpx5n1LG5rS00PV1HePV8P6LSvaBA4dWzl1OyA+u+RxBcxp4/MAnRPtvwlKtULenynl2uQZiKLIFqdq6Jt8SK3Lb5zSp4a/K/Ao3RoGpPsSZ+RFdmhONYRwjoE90pU8EYvgDzXcrxWAJsIIBOfcrtKpytWLY35tPcbhaY+Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 05/17] xen/arm: Add PHYSDEVOP_pci_device_* support for
 ARM
To: Rahul Singh <rahul.singh@arm.com>
Cc: bertrand.marquis@arm.com, Andre.Przywara@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <cover.1632847120.git.rahul.singh@arm.com>
 <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <f4ff4369-3be1-eed0-d3e3-6c9d93952526@suse.com>
Date: Thu, 30 Sep 2021 16:51:04 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <f8ac00f7d52f4853d276b4da24294fbeb3602245.1632847120.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AM6PR01CA0065.eurprd01.prod.exchangelabs.com
 (2603:10a6:20b:e0::42) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fc0b3b67-4c82-45cf-8896-08d98421bbc9
X-MS-TrafficTypeDiagnostic: VE1PR04MB6479:
X-Microsoft-Antispam-PRVS:
	<VE1PR04MB64798F427007DA2F5F506FD8B3AA9@VE1PR04MB6479.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ImUrXuWjtfeCjEbPy8ZxKzvxJp7jQ7RJPE7gqwt3+nk7dqmLTr92YmrS8YYZj6tk/Vn5Ui1jhX58ETZB/WnGRfc/1rbQ+mHexaftHms95wYCG8YhQ6bQjX43SYRmXN0l9AuAU6Y9E71RXFJuAFawOsSUqeBgfZe5Jyck8zsbYu6LnH3yOcYhe0gPW0RFBMWWDeYT+bNp1rVRNynHalrCe+5UuKXVZ8DgwPxSwvbwxpKUcVjCTpXmzoIBH/LMKwVFKprsRlR3xT/PIr1LEEPfr2WAflg3jNmqpIzpwn+opF9e5Sockc56THqaXhfdRz/1pVpwQ6aIbJVPW3zwVrCpb2qauAj0x+BUsQTOIkR1ii7CUeytS0asFcu49Rs586jVw0pms28vmNNYgeMDUor7+pRwgcElacu64TW6IXSVd64LcSxRnIdixjO2i1+YfpYb+iqGZHfVUQ6bagpMsipRPEXa+mzhP6YxoHk4u3DyGTpjzyF+X1c8C6WuD5/hIZv3g85oNuYtLHdSrWQDnJKfPtflPdbAmunXsht6sIqN4QXZAo5yVZgN1c5RbullOige8P/1Zjg3KHEQn6Y301ahjVdtXKv7MMYDFz5uSUr6yFWkXfHmkWmcBXqDGuBlPTTteFLFbdZOiSQJiHSJgIm3hl3rmyGR41P2rjI3w953MphCwtIKSYn0puMrw0ZX71UyH0NQk4vuT1atR6upauLzBw==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2906002)(508600001)(6916009)(36756003)(86362001)(26005)(16576012)(316002)(186003)(53546011)(956004)(6486002)(31686004)(54906003)(4326008)(83380400001)(38100700002)(8936002)(2616005)(66946007)(31696002)(66476007)(5660300002)(8676002)(66556008)(7416002)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?RDBqbUo3VC9mOUFWcldTNllzYnZTNXlWY25zMDV4a0dHU1VhZEdZWG43aE1m?=
 =?utf-8?B?SThHZkdiSzN0ZDN3a3I1dWd1ZEpGNDQxZ2dwK3JiR1ZJZ1IwblF0bnVNVjdP?=
 =?utf-8?B?VVdqaXBBM3MrSkhoNjgzN0lXU1BPaDByV01yWEpqNnFJOFBUSDVNdjY5eVlv?=
 =?utf-8?B?QnRwejJ1WHlUdDlSdVRndFJFVHY1VU4zWEZLVGRZaEFrSUFiU3Y1RTc3N2N2?=
 =?utf-8?B?N2lFb3drQ2lEZG1veFZCMFNtaS9ybHFzZVk4T2g5cnZJT1RWSmVNSCtnbDZP?=
 =?utf-8?B?Q1FMVklVeTN5OGMvdjBzVGdUYjI2TVdoSGN3dDVoZVBOSnJCVHU0aE51bTda?=
 =?utf-8?B?L2h4Q2s0UmFMZ0VJL0JZK1VzZ2E3YWZYTWhRajVPdXdiNTR5bU0yQnQ4aVla?=
 =?utf-8?B?Nk5rMVNsdFQ3T1pvd2JrVlI5V0VYZk94RjlGVUZIZTd4SGN6cE0yaGpLN3BZ?=
 =?utf-8?B?ZEp3ajhyV1p5djJVZVNHZ3lmbWlQWFg1dHBBZG0rTjZGNi9hdnpjcHZMOFRy?=
 =?utf-8?B?MjdLVjVRQzJ4eW5WSkhCSVE0S3JTOEpVK0xBZFpwZFB0L3VwUU4yRXBIVEUy?=
 =?utf-8?B?T3lxd3JKMU9rZ1htMDduVXNKSHpnS3dZRTJyVURILzdLUWVMQVBNSFBPbnUz?=
 =?utf-8?B?UUxQY1N6M092S21jcUdGNjdEUEIwY1NHY2hRdEVOd3l0WlpWbmdmeXgyRXB1?=
 =?utf-8?B?S3ZLazkvTlFOZENaWUQ4a3g0ait2UlBJRGloQVZMaGxIMGRObHNvbWVYWlVG?=
 =?utf-8?B?MHJJTGZsb2oweXMvUDFyMlZ4dEh2R3BlUS9qNWpyNTdnZzBkUzMvOXFLcjhw?=
 =?utf-8?B?RUsxNCtzS0p5KzVmZnZaS0FocU56a3JlZjJLMkhqYlZ6R2Y0dGtvLy9VNDFn?=
 =?utf-8?B?N3B0MFRoWHdYcVBTR0VkWlFCc2NpZmpMR3JxUDJ3OVQzYWtkSDNkUFlSNWkw?=
 =?utf-8?B?K05RZ1grOG9DNUR5ajhtTGovT3BUWXEyRFR1bGdVMDhlamJ2TWxoTXNxalFl?=
 =?utf-8?B?MU51SkFGZ0lJMGo2NS9yVkg1T0UwcTNQcjJiRTA5bXE5S2ozc2pIdDIxNVBX?=
 =?utf-8?B?bUx5NmQ4cFhubnM5c0JXVWZFbTNGVmIzelk1bWV3Rk90QWM2TmdlRGFVNm0z?=
 =?utf-8?B?UXBaNDkyRmx2a0NNZlVzYzdXaFRBd1JZZEc1aEFtanhhZDBNNG43U08vbEpy?=
 =?utf-8?B?bndhaEY3R1dyT3V6T3hYZmFhNkJOa1FtU3NEWU5MeHdhQmVFU1kxK2V1d3c2?=
 =?utf-8?B?RDdSRVJBVVZnLzFUekFKVkF0ZnhrVmJ1ZGw0MTZkT3Q1cUtPWEo3K21HTVRN?=
 =?utf-8?B?K2E2RUtIenNIVEVkNDFvL29xZUVCZmc2aEFFME1zSHBITXRlUVBYb0NHQVRU?=
 =?utf-8?B?ZjVQbnJzaEplNVA0SWt6dUVmbzk4eENaWVVKeVJ0RmEvMGVMRzd2MGdmRWZ4?=
 =?utf-8?B?SVNsemNvKzZ4K1ZEWHNkY1NhU0Vja3FsSEUxUkprWHlQRmlrSitkL1g1ZFh6?=
 =?utf-8?B?d2REQ3dzTjArSVU3Nk9aUmdWSXJ2MmFwbStzMURzOStMVXl2bjlxY3R3Tm9a?=
 =?utf-8?B?VXkwQXZYaWliK0hNL2RmQ3RFSC9jb2Z4U3VBSWdoSEhjcldZTnJLUkswTWZN?=
 =?utf-8?B?QWF2ZXVyV29KQjA3YUUwalpncE1LMC9qNkYxUGVqK1dmYXBQM1VRdXk3MGx2?=
 =?utf-8?B?bkx4VjhhK3RPSnMxaVRMdDE3M3NwSnBmUlc2S1hwV1lXSWJZdE9qazNRSkhM?=
 =?utf-8?Q?huxRxaYwYIbV3nDUbKIJ4Lv/tz/GHCvFIW73KMj?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fc0b3b67-4c82-45cf-8896-08d98421bbc9
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 14:51:06.9949
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3qumflTGaykpbWFThVAz2ZHOz4b18NOG1K5XpqsKb1O4lZMc7K3bMk3ea6TjNaHu3xQF2gI6ND1vJpgoahsTOw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6479

On 28.09.2021 20:18, Rahul Singh wrote:
> Hardware domain is in charge of doing the PCI enumeration and will
> discover the PCI devices and then will communicate to XEN via hyper
> call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
> 
> Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
> 
> As most of the code for PHYSDEVOP_pci_device_* is the same between x86
> and ARM, move the code to a common file to avoid duplication.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

On v1 Julien said:

"There are other PHYSDEVOP operations to add PCI devices. I think it is 
 fine to only implement the latest (CC Jan for some opinion and confirm 
 this is the latest). However, this ought to be explained in the commit 
 message."

> @@ -82,6 +83,7 @@ CHECK_physdev_pci_device
>  typedef int ret_t;
>  
>  #include "../physdev.c"
> +#include "../../../common/physdev.c"

Please don't unless entirely unavoidable: common/ has its own
common/compat/.

> --- /dev/null
> +++ b/xen/common/physdev.c
> @@ -0,0 +1,81 @@
> +
> +#include <xen/guest_access.h>
> +#include <xen/hypercall.h>
> +#include <xen/init.h>
> +
> +#ifndef COMPAT
> +typedef long ret_t;
> +#endif
> +
> +ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> +{
> +    ret_t ret;
> +
> +    switch ( cmd )
> +    {
> +#ifdef CONFIG_HAS_PCI

All of the enclosed code should really be under drivers/pci/ or in
drivers/passthrough/pci.c, e.g. in a pci_physdev_op() function
called from both Arm and x86. Unless, I will admit, this would pose
undue problems for x86'es compat handling. But I'd like to know
whether that route was at least explored. (I.e. I'm afraid Julien's
request to move this code to "common" was understood too much to the
word, sorry.)

> +    case PHYSDEVOP_pci_device_add: {
> +        struct physdev_pci_device_add add;
> +        struct pci_dev_info pdev_info;
> +        nodeid_t node;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&add, arg, 1) != 0 )
> +            break;
> +
> +        pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);

While I'm not going to insist (as you're merely moving this code), it
would be nice if the !!() was dropped here, ...

> +        if ( add.flags & XEN_PCI_DEV_VIRTFN )
> +        {
> +            pdev_info.is_virtfn = 1;

... "true" was used here, and ...

> +            pdev_info.physfn.bus = add.physfn.bus;
> +            pdev_info.physfn.devfn = add.physfn.devfn;
> +        }
> +        else
> +            pdev_info.is_virtfn = 0;

... "false" here while moving, as both fields are bool.

> +        if ( add.flags & XEN_PCI_DEV_PXM )
> +        {
> +            uint32_t pxm;
> +            size_t optarr_off = offsetof(struct physdev_pci_device_add, optarr) /
> +                                sizeof(add.optarr[0]);
> +
> +            if ( copy_from_guest_offset(&pxm, arg, optarr_off, 1) )
> +                break;
> +
> +            node = pxm_to_node(pxm);
> +        }
> +        else

I think this code should become CONFIG_NUMA dependent, now that it
gets moved to common code. This would save you from (oddly; see
below) implementing pxm_to_node() on Arm.

> +            node = NUMA_NO_NODE;
> +
> +        ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node);
> +        break;
> +    }
> +
> +    case PHYSDEVOP_pci_device_remove: {
> +        struct physdev_pci_device dev;
> +
> +        ret = -EFAULT;
> +        if ( copy_from_guest(&dev, arg, 1) != 0 )
> +            break;
> +
> +        ret = pci_remove_device(dev.seg, dev.bus, dev.devfn);
> +        break;
> +    }
> +#endif
> +    default:

Blank line between non-fall-through case blocks please.

> --- a/xen/include/asm-arm/numa.h
> +++ b/xen/include/asm-arm/numa.h
> @@ -25,6 +25,11 @@ extern mfn_t first_valid_mfn;
>  #define node_start_pfn(nid) (mfn_x(first_valid_mfn))
>  #define __node_distance(a, b) (20)
>  
> +static inline nodeid_t pxm_to_node(unsigned pxm)
> +{
> +    return 0xff;

If you can use NUMA_NO_NODE in do_physdev_op(), why not also here?
(Assuming this function is going to survive in this series in the
first place, as per the earlier comment.)

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 15:00:23 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 15:00:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200034.354406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxXn-0005ep-Mx; Thu, 30 Sep 2021 15:00:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200034.354406; Thu, 30 Sep 2021 15:00:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxXn-0005ei-Jz; Thu, 30 Sep 2021 15:00:19 +0000
Received: by outflank-mailman (input) for mailman id 200034;
 Thu, 30 Sep 2021 15:00:17 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVxXl-0005ec-Pe
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 15:00:17 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.111.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 1edfcf3e-21ff-11ec-bd52-12813bfff9fa;
 Thu, 30 Sep 2021 15:00:16 +0000 (UTC)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2055.outbound.protection.outlook.com [104.47.0.55]) (Using
 TLS) by relay.mimecast.com with ESMTP id
 de-mta-36-t-qMvoq3PmGQuy9JxWWLKw-1; Thu, 30 Sep 2021 17:00:14 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR0402MB3535.eurprd04.prod.outlook.com (2603:10a6:803:11::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.21; Thu, 30 Sep
 2021 15:00:11 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 15:00:11 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AS8P250CA0020.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:330::25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 15:00:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1edfcf3e-21ff-11ec-bd52-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633014015;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Daebu7f1lPHNMtBuyqaM6U+dABWfR56uw/7u819YTzI=;
	b=Vg4FpHZMN2lSWOzviGpNB4FB51bxu8afVV+K3zwvWLVugT33jhox2B9XqoBKFo8j6aqtRA
	7o/YkhDMh35SE3A1dbVcl150qYnnv8sthkx+kYIoN2NMFYXPTslkaXZcPUZ0dNAFdghi94
	Z1j9gV27gZ3b/OTYzbW6VkCi8B2UVw8=
X-MC-Unique: t-qMvoq3PmGQuy9JxWWLKw-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Q1C191YB2oUUzayy8aN88Ww275t/hl+vldKdITU0K3+8q8fYaNql6Rb5/u5MZALZg7qvSO8hJpfYmPoHzirj/yYhP61ntJf/z6bU0/LAi9xUXAaRQpUipKr3ioCxraUkDsvB18QNftkkB1cGkAFJosY+F+2IagPIoLs47dKfN6mvwNKSpqtezXoB6NTLfnz6yzU/HasGwLgT7ffHQf0+ZQkyeySaDDEHPpofsmZOyptKx87HOnl1469RlYeAmQabWL7D36XLPiejWfmtv9tRUISq7OwQAQoq+aScrIp008xfmVXdLX4MQhVnDMiR0+Ywo+Qg5Jrs1i0l1QMHp2Lizw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Daebu7f1lPHNMtBuyqaM6U+dABWfR56uw/7u819YTzI=;
 b=XVh2xZLiGYAr7pttPJXQ01ASAHzHMsKi2qngs8yxQ84ZKLBV40DEHm76xTR2XoyolJEegiSMxkjlOzok+JZwl8oAMBiu1k7X1QxLyxzgc7SoeRiO4Pil/UrVMXCeG2WNC34KIODe5+qA459BYcq54lPcVCR3B926cE3/mWC6Z9ws1nlcjQxmn3Zz+/K/AGIY+QxaIKJWu7FptJoDAYLgzvM02OGJAywsQDH+0FuI3oiBcaftp6ZyvZ+wY/WScxgvQGAg+ZWQWCOdXK7oAiXZmwgpLAiV6Zq81wrFE5CdsEyMwFFGgdmv2S7s7XYeNxTQrrl8Rx9MhbzL9CVg+e/Ffg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 10/17] xen/arm: Add cmdline boot option
 "pci-passthrough = <boolean>"
To: Rahul Singh <rahul.singh@arm.com>
Cc: bertrand.marquis@arm.com, Andre.Przywara@arm.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org
References: <cover.1632847120.git.rahul.singh@arm.com>
 <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <8b9dc108-c20a-bce3-42cf-18a44e369776@suse.com>
Date: Thu, 30 Sep 2021 17:00:08 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <edd6689cc977292fa874059861474993e5819c14.1632847120.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: AS8P250CA0020.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:20b:330::25) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 13af75ac-ed47-43e9-57d6-08d984230031
X-MS-TrafficTypeDiagnostic: VI1PR0402MB3535:
X-Microsoft-Antispam-PRVS:
	<VI1PR0402MB3535B0ED91584D25B3245D93B3AA9@VI1PR0402MB3535.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6430;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KaOm52H7PHdPm9uELuix4Xbajge9Fk8fNAT/CLGiv4G3IypSAHJ/d4BaDP5vsk1hL91Xn30bdEIuLkwhpdcc2Od1TDwyNWzJXKe7o9vDaPEnfwP1yNJZivvJsMllmbaE/MXzfZvgP+/WG9E8jL7epnfBKDYhF4qGrIyqHydHHaLuqJSe2GLtSmFsmpaOTR2C3IpkHBUUmRbgGImO6uo1qTfs97eh1J3KJyqRFhCkp3OfWqeeaQZRZe9cXRnoGrHlXOSFeZcGCperbsXCu3m0D4+SsFDzyhKqF/MXljW3jcFy4s1PJ6qufNKshrIamGZ+Q+eq/pVnw335G+SXW/pKpiVwRuk2Z43sJTdERbatClQY2FXJ5hfnno7nlBxcmHlh/ppo/i/0H5SIgg9KodcLLE/xOlRDUlHyRJlTLjfqI1Vd9X5B8uZBFXCrIbe8d9PYY1f010qoMXeOh75dFXp4C3aoCJ1wM3nPVW/AtonMr7kQJ4zCN3zduNdFP/hD/pgENh1ABtIXC5EIQPtWlUfVzDh5jfs8UeWG19GRT6juBnfGFPjJlrWlLaPwRPprlpRt1bVndGVrkphE/BZVJhgNGZxtxJu+qs2cH7lyqhHPNLQKTdPJQqwCs2zYO4YenWAlzwz/3SVbBgXo2zKFdO6kgTP0IAjHKIPQQLBaPvJRZLtbjup9cpMeZ0kc4wWmb/8smfyMb0l56VawIkA3UEnej/ApZUhUOohQHDSk/elC5Mxh6W6tglNTmi9ZhsNF+SSp
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(83380400001)(26005)(8676002)(66476007)(66946007)(6486002)(66556008)(4326008)(186003)(508600001)(53546011)(6916009)(2906002)(16576012)(31686004)(7416002)(54906003)(86362001)(36756003)(8936002)(38100700002)(31696002)(5660300002)(2616005)(316002)(956004)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YkJzS1EwbEF5a0FpL3YxTlIwQ1kvL2xvODNIWVRYanA4Q0NlLy9RK0UvWjFU?=
 =?utf-8?B?emJNYnc5azZ4bmtZVmRtd0FHZGVSa1VtVkYxRjdPVGVQTGJqVGtFRlQ3ZE92?=
 =?utf-8?B?TjNkWTVGZStOcWZiYzB6UzFhUW5jNnpMbE82Z0doMmNhc0NTeGpUNjZKZHFD?=
 =?utf-8?B?MHdScElEdjJvZGwrZGM4cVd0dHNub3U3bXFiM0dvSEZMbWZySVREMXd6bTdD?=
 =?utf-8?B?Nzg5MXB5alpENGpDcnRuVUkrdDhPWUF5ZHBCYUFqU0JKaUd5OFkybkdETHRM?=
 =?utf-8?B?ZTY1YiswaHVRa0lIUEFHNU9IZDh1SFBNVG5QdEdvVmlweTVSMmlST1BjV25D?=
 =?utf-8?B?b1lJZjRkSzhSOHZmaGYrelhJS1Jvb1JvalJMMmdhRktwOVBnOWZ3N3JDZDl3?=
 =?utf-8?B?WWhLUlpqclNBMTlQSnRCSC9kOFFMcmluaHdmcnkvU0gwbnlTR0kyaGRpNi9Y?=
 =?utf-8?B?L0hQSnIxK1kzdDlBU2VrVFNYNW1GWTZDbDFORWJMUTFTNmhodm5oMnE3YTgw?=
 =?utf-8?B?VER6Q1prK21MMmhSSndDZ0UyWS92UFRDam5ZZVM0T0NrNk5aa1dyK2JRR1Ni?=
 =?utf-8?B?VDJId2N5SDg0eUFaejNGK1Yvc0tuUTBQVGlMUTRTRkNydkVOQmFLT0JyY1ZB?=
 =?utf-8?B?b0JVWU1XVHVCN1R6MDU1d2ZVVEJhOTRrV0R0MWNZUmRHakJvRzJqb2dWUFMr?=
 =?utf-8?B?b2xCSnFVN0dEdGQ2aUF0Q2hSSllzOXh3RGhnd2hhN3Y3MWZEUXhpUHR1U0xw?=
 =?utf-8?B?eXpYSFhFNGZXbWFpbnFvczRjdWVwRHJrcXMwTkxOaVRJSXNkU01qRUNLdUxL?=
 =?utf-8?B?ejVvM3c3MUN3aytVU2hPdE5EajJLRnYrTlVUZVpNcDV5SElyUytkN0U5VEp0?=
 =?utf-8?B?eGhOMVo1bW90Y1N6UkVVTHpoTEQvT0JUSmtwZGlxZWwySUJrL1pxay85YWE2?=
 =?utf-8?B?d051QmE2TC9XTHR5WkhIMzRld09nSE1XQVBOK25WY1ZQVmExdlA0S3cwN3F5?=
 =?utf-8?B?SHBWVndMMVFTa2dLN1hXUmwxRDM5bmN6SHFhZTNLb3RnNTFDVE1XUUp2QjZH?=
 =?utf-8?B?d1dYMTc4VHRmUEJjVHF5SnYxMDJPWUNMMnpXU1FiUzkwNW5LNFNjWUtzTUx6?=
 =?utf-8?B?ZVhqdEQza2JVLzJGSUhjaUlFUlNQUGs3YzFMSWdDU1M4L3AyT2dSSHZiUXJB?=
 =?utf-8?B?eUtVczVSczBXUVVsMGZMNjRMMkY0Qzh0a014ZDM1V2pGZHJ1L1F5dmtvQmg2?=
 =?utf-8?B?cVkzTi9XWGI4MmNrZFhreWIwMW16VTZSN2Y0K2hUTjFMak1NaTBIVjE0WjRZ?=
 =?utf-8?B?c1d3M2hxUWIxVGhSeVFaR3c3MVRGL2FPd2VuWkNYaGZ0TU9NZTM2d3hZc3Uw?=
 =?utf-8?B?aDE2TzVQUDR4WUFGR2ZPSi9jQUUrZW1GbDYrSHk1S3JsdlRHbGRYajk5MmNQ?=
 =?utf-8?B?NS8wT3lYWWx5dlY1NVFpWVFqQTM5ZFJPeDJlYzdMcW1nMmFJWkpGaEp1eFBh?=
 =?utf-8?B?QVFnRHgyZkhlcElDWk5ndm1KVDFtbEhoRjFqNiszVXFIZE9BQ1gybEZLS2Fq?=
 =?utf-8?B?Qm9OWXZSMzhBNTJqbXd4djBGZGVUSkZMMjlZTmZpOURaejNCWnVUeGViTTFW?=
 =?utf-8?B?cWVDbHdKdHhrWTJocmdVYW9xYTlWSzlNeENKNmpNSVcvSUExVk5VbEZkell6?=
 =?utf-8?B?RGpjN0hPNk91SnFQQW5CbXlyTEh1dE8yZFQ2c0xrMHd4Q1pUNnViTjZCaDRz?=
 =?utf-8?Q?W7J1u9t0v/a01QT6PCXHArzISJbOBmoTdlAPwxr?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 13af75ac-ed47-43e9-57d6-08d984230031
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 15:00:11.2520
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JY85OKEw7xxOAGDyrMwihQNG1TiVMbFSG0V/ZyLs/gHsLH0oZyMbY4oWmIADTE/QUNYvEh0JMAcISkA8ruYlDA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3535

On 28.09.2021 20:18, Rahul Singh wrote:
> @@ -62,8 +63,21 @@ static int __init acpi_pci_init(void)
>  }
>  #endif
>  
> +/*
> + * By default pci passthrough is disabled
> + */

Nit: This is a single line comment.

> +bool __read_mostly pci_passthrough_enabled = false;

Nit: Unnecessary initializer.

> --- a/xen/common/physdev.c
> +++ b/xen/common/physdev.c
> @@ -19,6 +19,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          struct pci_dev_info pdev_info;
>          nodeid_t node;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;

Here and ...

> @@ -54,6 +57,9 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_pci_device_remove: {
>          struct physdev_pci_device dev;
>  
> +        if ( !is_pci_passthrough_enabled() )
> +            return -ENOSYS;

... here -EOPNOTSUPP (or any other suitable value, which -ENOSYS is not
from all I can tell) please.

> --- a/xen/include/asm-x86/pci.h
> +++ b/xen/include/asm-x86/pci.h
> @@ -32,4 +32,12 @@ bool_t pci_ro_mmcfg_decode(unsigned long mfn, unsigned int *seg,
>  extern int pci_mmcfg_config_num;
>  extern struct acpi_mcfg_allocation *pci_mmcfg_config;
>  
> +/*
> + * Unlike ARM, PCI passthrough is always enabled for x86.
> + */

Nit: This again is a single line comment.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 15:19:24 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 15:19:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200043.354418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxq3-0007W2-9U; Thu, 30 Sep 2021 15:19:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200043.354418; Thu, 30 Sep 2021 15:19:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVxq3-0007Vv-68; Thu, 30 Sep 2021 15:19:11 +0000
Received: by outflank-mailman (input) for mailman id 200043;
 Thu, 30 Sep 2021 15:19:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=lHSZ=OU=suse.com=jbeulich@srs-us1.protection.inumbo.net>)
 id 1mVxq1-0007Vp-Rl
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 15:19:09 +0000
Received: from de-smtp-delivery-102.mimecast.com (unknown [194.104.109.102])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c11f9c77-2201-11ec-bd54-12813bfff9fa;
 Thu, 30 Sep 2021 15:19:08 +0000 (UTC)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105])
 (Using TLS) by relay.mimecast.com with ESMTP id
 de-mta-31-QHld5u4RPiq49rp9TzMvJQ-1; Thu, 30 Sep 2021 17:19:05 +0200
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com (2603:10a6:803:e7::16)
 by VI1PR04MB7149.eurprd04.prod.outlook.com (2603:10a6:800:12e::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 30 Sep
 2021 15:19:04 +0000
Received: from VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b]) by VI1PR04MB5600.eurprd04.prod.outlook.com
 ([fe80::4d37:ec64:4e90:b16b%7]) with mapi id 15.20.4566.014; Thu, 30 Sep 2021
 15:19:04 +0000
Received: from [10.156.60.236] (37.24.206.209) by
 AM6P191CA0049.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.16 via Frontend Transport; Thu, 30 Sep 2021 15:19:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c11f9c77-2201-11ec-bd54-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619;
	t=1633015147;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=bgDlE3U3NKHONbHyYc0K96n7YPWEMknxgbSho7XY7Y4=;
	b=EzrTuAPiAq1pC/zGorw0NXQf0duE/JnxrI9wOG6TzBvw9pEGy1ifRRJS0wZjE1dJUHZ4BW
	mqKY89IWyvVRhhqATboE7q4fvx3uNuvO0OIhZEBdRqt0oJEToIbUMkORWCrzukTMDRz0HY
	hzZbAfXvuqchKn6re1+3wdV0gWNNT6A=
X-MC-Unique: QHld5u4RPiq49rp9TzMvJQ-1
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iMFro7XkR6oxrOGIS6oFjd5FexPO4IqCYTdpZ1LT2lHwaRBFFEUaXPzQ/ENMCRA/p4yXDj3VXMrwWFEQczaJxJkjb496/8Hod0IckgEcGNBE/l68vjRQetCxT7vx5yszMQPMdO1pDCA637zOK9eLNvqJq/xGyjxojT9Q+gX9kNOYc5FY+ou3mOKVa/l5jfi//3vzGcDwSDKpCtuB4OwCjovLjEEcUrXZjhDJfno5dW5AqsFuOURpfg3lWucLLgw5rkuIj6I7vBqj99LAK+pv3TOPJQNtVXRpIxeN4/2Qvp9iEHjPejDrYDjz6xmoIZV5AUBhq63FS58x/s0PBHxrKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=3BAhEpSNUGTXpf37neCD/9++svB9G8eXQxhjuiXz/nk=;
 b=hCKLxGhDnim676lHqbEIhee8vQfFIwNTV6taXNldh7WusB9DwlsevNGIZs+Oyn0V0RRHjMPxcGECZxRzVylUg/2Cd+536bul2DTrlazN/5BADpyiN7Noq5+dzUHB0ZMlvFBiqFlAH4sAz9tlOKQO1gcihUtqreReiVJy66wCEC7mIhD8Ej9wwzWW3BkZSPyOMtcMBNM3vU3SfJlLDL3EXSgB82ka1YbUAeIhrYb3xHwUqWmQuCbK8aaCD9nbIVJulUCg8LWdKTGVj814XJLt7ZPYYNk6W4sl698MIyP3HBKv+0/RzPLnyvS0hJFOBLroDns5hckCZp6SmFt87+fmQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com;
 dkim=pass header.d=suse.com; arc=none
Authentication-Results: lists.xenproject.org; dkim=none (message not signed)
 header.d=none;lists.xenproject.org; dmarc=none action=none
 header.from=suse.com;
Subject: Re: [PATCH v3 14/17] xen/arm: Enable the existing x86 virtual PCI
 support for ARM.
To: Rahul Singh <rahul.singh@arm.com>
CC: bertrand.marquis@arm.com, Andre.Przywara@arm.com,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>,
 Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=c3=a9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1632847120.git.rahul.singh@arm.com>
 <d7f62249e6aa7b33017ef707d6bc451b5e2c7047.1632847120.git.rahul.singh@arm.com>
From: Jan Beulich <jbeulich@suse.com>
Message-ID: <491a49c0-d609-c122-f64a-999b16ef3a83@suse.com>
Date: Thu, 30 Sep 2021 17:19:01 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <d7f62249e6aa7b33017ef707d6bc451b5e2c7047.1632847120.git.rahul.singh@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-ClientProxiedBy: AM6P191CA0049.EURP191.PROD.OUTLOOK.COM
 (2603:10a6:209:7f::26) To VI1PR04MB5600.eurprd04.prod.outlook.com
 (2603:10a6:803:e7::16)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: af7807d2-5055-4ec9-0849-08d98425a378
X-MS-TrafficTypeDiagnostic: VI1PR04MB7149:
X-Microsoft-Antispam-PRVS:
	<VI1PR04MB7149636C3F774877EF9C8E4CB3AA9@VI1PR04MB7149.eurprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xcDEG15fIgmZytgs/PzvOJWAALl/JZK7v1fmi7lOVjtIh3DuaZU20dXb1cAt/wJ7mI+x+xP+G/JvvZ6J55GjrrH1X4GSOxnEr2StChYhjP2vXhhhlKBsG7NSpn2+Etk2W82wfQRwtIzRxW2pO8otZUklZkGqby2z9KqsWyni4WiiTSYbqFev+uzqbg9JpV33DH/9FK4GVF8i5X0lUSpFukpuZYl9zeDPUT0R5gk7ezGQ6WwS+5VUndhYaxQNHaEhFJYbFsPAFReALOGfO7dEsdSX7XQPMsAXB8tGrwF5BJKr6FVoS7GiaaZrEhVjgqktEgUdirP1DX9JBKjYCX91AVsk15zIijIr7YdbTjojIbCpNyYI0ffANXOy5IB7ImSOzR4nRnA2CflDzSOIDpX30ALQhvgaWKNqtkrS/VM+UF61cGJr5XqqgM5gdFTTKzDl9M6/g4in3KPSuQ9ih4Q3Pkr0/1y96XRfE9oPyyA/3U7fXX/uNHPesCF5p/QJQYK9iFRtqT3ze3R9sFMsBefqePKxYDUtt8M9ZCQcxmiKQ6PdbmqINZWouUV5/rKS9E3aR/UGzANvq0AWV9JOfG9AoSJqJuZfRfj/wWADlfD/vFHrBff4rja0bnGVilfb/8/Mry8e1x4i7zDxzzI0AlDfujVRiZ7cLXgqZMpUwtHrccKn3CjMh6lmIff04Asl7bUQCAntu9elbU5G128PO8wZw47Ozox/gfnBflDyQkXGdXz66Nnw8EbGnRSKHQ0s9hO5
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR04MB5600.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(31686004)(53546011)(26005)(186003)(2616005)(6916009)(508600001)(2906002)(956004)(5660300002)(7416002)(6486002)(66946007)(36756003)(54906003)(86362001)(4326008)(8936002)(66476007)(8676002)(83380400001)(38100700002)(31696002)(316002)(16576012)(66556008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?fuM6PBukR3OoA0+NNcEr6zEvmmHF1RQW09fYY7ENLvTJIqCw+vBlqHB6J8K0?=
 =?us-ascii?Q?csRNOUKVURveUBPwJrjquarlSHEm1RdPQzITw1kUA9bTz1fCkf+ReZrV7NxC?=
 =?us-ascii?Q?Z4CkWthq4JKf9L66ToM3DXvqXfyqfnNZUbnD1Sy0inj2nTe3DmaKRkIG5K5r?=
 =?us-ascii?Q?TmhJc82l+UKYtLK28f7wSypyWcMge3w0T50lF0RRQCGyidBPhJRsWajv97ai?=
 =?us-ascii?Q?QOPDa74UxTme5pjYsvnwGNSykGrr5iWZSKJ6EBKtyxcKA+herTLTZHiy+R61?=
 =?us-ascii?Q?/KVtCxbR0mC256PFtBrwQfrJ0JhlC61TRwylGLB5vyG3yTRWquVXj6dct7WD?=
 =?us-ascii?Q?O29Tb8F0v8TRwumrkcCWB65kkOnFJzhM2mIDMo8hxzBXiMVG0Ri2e9kTyu4/?=
 =?us-ascii?Q?vFYeCP1/aixT11P1AgOTDbP3sCY2fEvZueEYa22YaLksWpUtPtHR5ej8VZFQ?=
 =?us-ascii?Q?huu8gbCLqd2lIIVGdxV31TVLqyPCT6DGlzTQnMNwQ97IF7jBVC9Gt2/gVXWT?=
 =?us-ascii?Q?SDCufKKy6C2G6bu55QE5KXDAZa93AVHMgJoieMkW89UjSpInsueEo7IVuAlA?=
 =?us-ascii?Q?AoI2IEFCg/Mzy+izXX7bPTxmrh9DeTa1nZ4nM3+gsXws9gSu99RDURXpy+pZ?=
 =?us-ascii?Q?SAzaWc9mkm18qDWy6oiG5ayFOZTrsxIge7N3DulKzRi5bYzXoxGgK3vVQotP?=
 =?us-ascii?Q?AlfyLQf9B+hdqkhx1RWEARVey5Hdd4VYE8bGqAV8zkN7OYjb2TM+DBl0nd15?=
 =?us-ascii?Q?dlEeAgkCRoDQIfuS04QEwX6/WPW3LuJo4lhO3aIduJvyzioPyOj2pEiFrSSa?=
 =?us-ascii?Q?sw2MZUY/jeZYtDntSAhsO4kBKeUvFkMHeW+6fFqrD+3cGtW2zQcawVvD1IK8?=
 =?us-ascii?Q?qZECo3Pd29XHRitYEq7IxE4jvPa6Uoe3KzjxGAFhwkhaWH7OpKWdUat5HHnT?=
 =?us-ascii?Q?xqgf6DNIB+gmYm3Tvp5NAQDzfvCEl/WgGoHHMSjk7B5stU5rXA5a7OvfIprm?=
 =?us-ascii?Q?sw3j9XZaEyirwuvRInHtfr1iAb+1qRiA3ZNEA6eKbgkzp1CHlg8scFH2uMQM?=
 =?us-ascii?Q?1xA4kc5D6bJ23QuRqu8XRS/8+NF7zcVZnPFIu7qTJX4INjptnxolqlUKZ2Pk?=
 =?us-ascii?Q?hy9RZFXzFNEsPjH99c28DBxNaUeGaCIFiWdfw28LYxNfrXv7u5HlbP/pt/H0?=
 =?us-ascii?Q?lCtCytRsGlHzVnKK+fcMDWADV/iBo8MN+griCpkYUZcR7JNFa6eCz4DHD6G7?=
 =?us-ascii?Q?rIV8eFAfzI29uBHR1Mg+PmY37FJUN/3awMmMeMTBxdrAjbC3PLN4IdCXReB+?=
 =?us-ascii?Q?l9UWsmHo5JFviwRCgLxAtOrG?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: af7807d2-5055-4ec9-0849-08d98425a378
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 15:19:04.2374
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xCkbLTrrQhVAqTrR91f3yYPOffUjko/d/fJWt1G9BwzTsJ9aaGVuE572QGmr/Hvswq7KavV8conYWA383WuZPQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB7149

On 28.09.2021 20:18, Rahul Singh wrote:
> The existing VPCI support available for X86 is adapted for Arm.
> When the device is added to XEN via the hyper call
> =E2=80=9CPHYSDEVOP_pci_device_add=E2=80=9D, VPCI handler for the config s=
pace
> access is added to the Xen to emulate the PCI devices config space.
>=20
> A MMIO trap handler for the PCI ECAM space is registered in XEN
> so that when guest is trying to access the PCI config space,XEN
> will trap the access and emulate read/write using the VPCI and
> not the real PCI hardware.

All of this is just for Dom0, I understand? Could you say so, perhaps
already in the title?

> For Dom0less systems scan_pci_devices() would be used to discover the
> PCI device in XEN and VPCI handler will be added during XEN boots.

So "would be" here means at some point in the future, rather than
before or in this patch? This could do with making unambiguous.

> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -662,6 +662,12 @@ int arch_sanitise_domain_config(struct xen_domctl_cr=
eatedomain *config)
>          return -EINVAL;
>      }
> =20
> +    if ( config->flags & XEN_DOMCTL_CDF_vpci )
> +    {
> +        dprintk(XENLOG_INFO, "VPCI not supported\n");

This is a misleading message, at least if for some reason it was to
trigger for Dom0. But down the road perhaps also for DomU, as I could
imagine vPCI to get enabled alongside passthrough rather than via a
separate control.

> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -767,6 +767,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>      else
>          iommu_enable_device(pdev);
> =20
> +#ifdef CONFIG_ARM
> +    /*
> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handl=
er when
> +     * Dom0 inform XEN to add the PCI devices in XEN.
> +     */
> +    ret =3D vpci_add_handlers(pdev);
> +    if ( ret ) {

Nit: Style.

> +        printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret);

Nit: Style again.

> +        goto out;

No other error handling (cleanup)?

> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -262,7 +263,12 @@ static inline void arch_vcpu_block(struct vcpu *v) {=
}
> =20
>  #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update=
_flag)
> =20
> -#define has_vpci(d)    ({ (void)(d); false; })
> +/*
> + * For X86 VPCI is enabled and tested for PVH DOM0 only but
> + * for ARM we enable support VPCI for guest domain also.
> + */
> +#define has_vpci(d) ((void)(d), \
> +        evaluate_nospec(d->options & XEN_DOMCTL_CDF_vpci))

Why the (void)(d)? Instead you want to parenthesize the other use of d.

Jan



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 15:37:00 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 15:37:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200051.354429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVy7A-0001W6-Qy; Thu, 30 Sep 2021 15:36:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200051.354429; Thu, 30 Sep 2021 15:36:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVy7A-0001Vz-Nf; Thu, 30 Sep 2021 15:36:52 +0000
Received: by outflank-mailman (input) for mailman id 200051;
 Thu, 30 Sep 2021 15:36:51 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVy79-0001Vt-Km
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 15:36:51 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.83]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 0ca52ed7-5219-47c7-8323-646980ee4403;
 Thu, 30 Sep 2021 15:36:48 +0000 (UTC)
Received: from AM7PR03CA0008.eurprd03.prod.outlook.com (2603:10a6:20b:130::18)
 by DB6PR08MB2869.eurprd08.prod.outlook.com (2603:10a6:6:22::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 15:36:46 +0000
Received: from AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:20b:130:cafe::d1) by AM7PR03CA0008.outlook.office365.com
 (2603:10a6:20b:130::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 15:36:46 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM5EUR03FT029.mail.protection.outlook.com (10.152.16.150) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 15:36:46 +0000
Received: ("Tessian outbound a77cafe56b47:v103");
 Thu, 30 Sep 2021 15:36:42 +0000
Received: from 398c540e35e2.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 9462DF97-B7F6-4C44-9D1C-ABAAEBE104FF.1; 
 Thu, 30 Sep 2021 15:36:26 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 398c540e35e2.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 15:36:26 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB5965.eurprd08.prod.outlook.com (2603:10a6:102:f3::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 30 Sep
 2021 15:36:25 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 15:36:25 +0000
Received: from smtpclient.apple (217.140.106.55) by
 LO4P123CA0077.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:190::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 15:36:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ca52ed7-5219-47c7-8323-646980ee4403
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XbSXGrRM3PmYmiIcZVj8Bzx7c4oZSm6muisaMMkEUAA=;
 b=ZRCglczKFt7Fn5yRyp116JcSrxQyKX1FTUrp0HtHqYciVc0TIrU7EtBAOan57xeIdLB+/WI9meqonkAfk1vgFu2NxnCGki5fDoXVoDXHL7rcQB/7BRs9ipRXjjPXHj50sBvhBGmHPmBvbkfawUhdKzDYJcqRArCq5BmMVd0pKBI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6d57b9af516319f8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=V+eQqDiSVYO6e1uHq6YyuGgNaX2pq9LjkxFfk8EAsxhierwoYcOBoP/MbwVLw/m1+JL5Jw1VjDqCRLorb8LUlpZcXzntkZpXQr5zZ/SBGy1KFTV1T39f+A6TV6evzLA5+j28CFgn2r9k2qqSeOIISJYeqtei282iOPXR/kgLKkWTYVoma7AupZ1FgR8AC62gnInz0yaCQYALO8ZGQSaVqWAPmJIvLaX/1sLmOKGxxCG73YCg2vCZ2tTprTkg90daImRj0Y0d4PdCOolPKimwv15at0aPXWlLb0KsywH6BakydUenRPMJ0GDhi0jdzT0B03ZA9sRL34nebJpReQdn3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=XbSXGrRM3PmYmiIcZVj8Bzx7c4oZSm6muisaMMkEUAA=;
 b=mD5IN4wPpjuP/QSk/f9kABdqQ6KaG9G6MdHsYW1OCyZwKPYEKrY8pJIYMHBRgCBI1baLmr45WnfSR/gl5t8zPhP0PSzk7F7SVDOwg9UDCvgcavb4qVzW3W6tH70IgRqXXFvUqJ3fxJP2l519Ym3+GO3hFeV7RK8mZxKJp0CFtMnYpt/Mq9I8tANxhvKhnTUzQD2BOSPVQdi09fWhosLXOs7nYD9FecFsfUq5zOpTDkiwAb6TdaycbOjlAwfgYN5HMpzxn4k9lIDKZUtOBgH+a6zbSSgZs6LnhZABPW6r9trmqov/Kc0TDbD+eWEoAlbdQNzx84LysIAAEVO/EuUoQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=XbSXGrRM3PmYmiIcZVj8Bzx7c4oZSm6muisaMMkEUAA=;
 b=ZRCglczKFt7Fn5yRyp116JcSrxQyKX1FTUrp0HtHqYciVc0TIrU7EtBAOan57xeIdLB+/WI9meqonkAfk1vgFu2NxnCGki5fDoXVoDXHL7rcQB/7BRs9ipRXjjPXHj50sBvhBGmHPmBvbkfawUhdKzDYJcqRArCq5BmMVd0pKBI=
Authentication-Results-Original: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH V4 2/3] xen/arm: Add handling of extended regions for Dom0
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <1632955927-27911-3-git-send-email-olekstysh@gmail.com>
Date: Thu, 30 Sep 2021 16:36:22 +0100
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Ian Jackson <iwj@xenproject.org>,
 Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <F7174E14-CBD9-47E2-86BC-4EE70D388EEB@arm.com>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com>
 <1632955927-27911-3-git-send-email-olekstysh@gmail.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0077.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:190::10) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4bd425f2-5346-4cb2-b1c2-08d984281c8b
X-MS-TrafficTypeDiagnostic: PA4PR08MB5965:|DB6PR08MB2869:
X-Microsoft-Antispam-PRVS:
	<DB6PR08MB28691D5B36E42F19E5A877F7E4AA9@DB6PR08MB2869.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:597;OLM:597;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 mZgnuy3XUFDYzAhxr11pY0GpbyzofxJtnDLX4bolGU0Gv0BHyS+/2q+jmV3ns1y5V6sBLOhJSTttjSNaFrVFWltyOXH6YzNzLPv8lHAXnIM0PYQqZ8TVXnU/YeEFTJ5h/sHWdUurC/2zvVTkvKDzZbgNAs1xBxjshYB0+L2cLBSZ6IG/20Ob7GnyGIyfYukVWvBlTQ0fQ6oQ2WLzsI97kGb4/idQx7QT617EHdubBbqOKV+KzYIqJaoa4w4x99ZY3+GareVAEtZOGy0M10mpufxj+0YO3t2h8GbIIvLsjSSyuRDQFD23ApztdhTLaXdATnUL9qRTLtfsqj7mcSlFmHZSzEhS/a7QWVuq17LRFcY7huZWQNt1KK8c2aP8SzXR4mZ18sqtZ6q9Op6tWzCl8iQVFTR2Q32CbR6wyhzZ9b2quwOL9FKQCHfuro/HnfQHsA3frjM7mpvHYFX4tZrbSmaT6UsxQVm1iIGBDSnlPUVr+2xgvXg9FllJ2arvX/qoGV4ZSdW/mMJnQBbT5Dp1JGIvv9mR+PSowH/bBFM/KK/bUDwK33pcFyLdYONLnf2916DAjnodBFZ77ZtxYCNN4MBEtMhlAgaVrvC1y3bJBUhJxKNvUGbG30IIXwZ8kOSCxRQDI7EopW/ae7jR31mkStZFUBgIQub+L07m4DJugK0prdS6ZTtx7FXK6c6ajVxRMAPhkYJFuEC692eQaQnvqSWxiF/8hG74+wlvyZdTP1EeFwemTVkVJf0+6aNxbisc70DUT5lMF7+/sIrw9l5EGrMtPKKVlvpjgJQJZCCrlyHqesIxa78laUR8SGDtwp37
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(54906003)(6512007)(956004)(86362001)(5660300002)(6916009)(966005)(2616005)(33656002)(38350700002)(38100700002)(2906002)(30864003)(6666004)(44832011)(8676002)(36756003)(6486002)(4326008)(508600001)(6506007)(66476007)(66946007)(8936002)(66556008)(83380400001)(186003)(52116002)(316002)(53546011)(7416002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5965
Original-Authentication-Results: gmail.com; dkim=none (message not signed)
 header.d=none;gmail.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5fffccbe-cc8c-467c-dc4d-08d984281012
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Rhk9hb1llXyydc5Rwej27Ild41Wy/8gs3dmhBitKucztpwpKzeDSsc3gvPcfPYniBrecGpiHect3X9jyMNDNy1yNBgz8uqP7n3VblVrv3L7YI15wH9Rdg8FUCDa2J16JEKkFLYyLMRIdOQ9/nwQ+drAwdtQTQUXM6tfcQ8IJnnHaW1pF5oiTN0xwqjZlJzojc64BUM6OwrEAl1nOdzJApKAAYif99PCuvHOIvJhx8aGfd3eoV1j5U6SqV4oXmFLeCuzR0stt75uSvptayBlwOi56/QXXDou55Kp98vItpdJrhQhm39PxSTsSmmgOKVx5p3OO+p0myrNjbqedAw2V6I1wPC3Hsph04tTgT+XscJQEi8Cb77T/Wu0KjT/bub+v3KHmebkt+wmyeR0YyTCS+GA1Rs8GmjxKQaRcv6KF5i1yfFmgOkKcpx3LidtwLTjLMhTbf6sPSfiM3jgw6l2QakeX19BghKPXkFK1O+yYHu7xBOQt0bj/aPb/3sa0+qjRK5ujUqn2AWXfzoN9eDGAPqSFmAodu6J/GobTk1cvFwRbsENPnqkU3W2QHfy3Dv4iWIPx097KyamZbA+BaoDHjQ2FZxveooCT0qvToXff0mnAw4eVPoU29d+HHh7QP461GTSmGDC8+rVom58iw1qDC1mI8rflyXVHkZJoBmlm1uZkPtc+krFdeTky2FiUF6t8OH3rUIJnJHhS4FcdyhIP6qbUqp1h6tcflTqPhkjZaCwhubc1QMroBJ/woef3WHxcCtUe9jeewGYjc9craM6TYApIf39ynFi+QiRIzVlE8F4=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36756003)(508600001)(316002)(4326008)(44832011)(956004)(356005)(26005)(47076005)(2616005)(86362001)(966005)(186003)(107886003)(8676002)(6486002)(81166007)(336012)(6512007)(54906003)(33656002)(30864003)(8936002)(53546011)(70206006)(36860700001)(6666004)(2906002)(70586007)(6862004)(5660300002)(83380400001)(82310400003)(6506007);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 15:36:46.1351
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4bd425f2-5346-4cb2-b1c2-08d984281c8b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM5EUR03FT029.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2869



> On 29 Sep 2021, at 23:52, Oleksandr Tyshchenko <olekstysh@gmail.com> wrot=
e:
>=20
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>=20
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
>=20
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
>=20
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain
>  currently.
> - The ACPI case is not covered.
>=20
> ***
>=20
> As Dom0 is direct mapped domain on Arm (e.g. MFN =3D=3D GFN)
> the algorithm to choose extended regions for it is different
> in comparison with the algorithm for non-direct mapped DomU.
> What is more, that extended regions should be chosen differently
> whether IOMMU is enabled or not.
>=20
> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> holes found in host device-tree if otherwise. Make sure that
> extended regions are 2MB-aligned and located within maximum possible
> addressable physical memory range. The minimum size of extended
> region is 64MB. The maximum number of extended regions is 128,
> which is an artificial limit to minimize code changes (we reuse
> struct meminfo to describe extended regions, so there are an array
> field for 128 elements).
>=20
> It worth mentioning that unallocated memory solution (when the IOMMU
> is disabled) will work safely until Dom0 is able to allocate memory
> outside of the original range.
>=20
> Also introduce command line option to be able to globally enable or
> disable support for extended regions for Dom0 (enabled by default).
>=20
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

> ---
> Please note, we need to decide which approach to use in find_unallocated_=
memory(),
> you can find details at:
> https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gm=
ail.com/
>=20
> Changes RFC -> V2:
>   - update patch description
>   - drop uneeded "extended-region" DT property
>=20
> Changes V2 -> V3:
>   - update patch description
>   - add comment for "size" calculation in add_ext_regions()
>   - clarify "end" calculation in find_unallocated_memory() and
>     find_memory_holes()
>   - only pick up regions with size >=3D 64MB
>   - allocate reg dynamically instead of keeping on the stack in
>     make_hypervisor_node()
>   - do not show warning for 32-bit domain
>   - drop Linux specific limits EXT_REGION_*
>   - also cover "ranges" property in find_memory_holes()
>   - add command line arg to enable/disable extended region support
>=20
> Changes V3 -> V4:
>  - update opt_ext_regions purpose and comment in code
>  - reorganize make_hypervisor_node() to move allocations after initial
>    checks, allocate only required amount of elements instead of maximum
>    possible
> ---
> docs/misc/xen-command-line.pandoc |  11 ++
> xen/arch/arm/domain_build.c       | 286 +++++++++++++++++++++++++++++++++=
++++-
> 2 files changed, 294 insertions(+), 3 deletions(-)
>=20
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-li=
ne.pandoc
> index 177e656..5cae4ad 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1081,6 +1081,17 @@ hardware domain is architecture dependent.
> Note that specifying zero as domU value means zero, while for dom0 it mea=
ns
> to use the default.
>=20
> +### ext_regions (Arm)
> +> `=3D <boolean>`
> +
> +> Default : `true`
> +
> +Flag to enable or disable support for extended regions for Dom0.
> +
> +Extended regions are ranges of unused address space exposed to Dom0 as
> +"safe to use" for special memory mappings. Disable if your board device
> +tree is incomplete.
> +
> ### flask
>> `=3D permissive | enforcing | late | disabled`
>=20
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d63..c5afbe2 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -34,6 +34,10 @@
> static unsigned int __initdata opt_dom0_max_vcpus;
> integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>=20
> +/* If true, the extended regions support is enabled for dom0 */
> +static bool __initdata opt_ext_regions =3D true;
> +boolean_param("ext_regions", opt_ext_regions);
> +
> static u64 __initdata dom0_mem;
> static bool __initdata dom0_mem_set;
>=20
> @@ -886,6 +890,232 @@ static int __init make_memory_node(const struct dom=
ain *d,
>     return res;
> }
>=20
> +static int __init add_ext_regions(unsigned long s, unsigned long e, void=
 *data)
> +{
> +    struct meminfo *ext_regions =3D data;
> +    paddr_t start, size;
> +
> +    if ( ext_regions->nr_banks >=3D ARRAY_SIZE(ext_regions->bank) )
> +        return 0;
> +
> +    /* Both start and size of the extended region should be 2MB aligned =
*/
> +    start =3D (s + SZ_2M - 1) & ~(SZ_2M - 1);
> +    if ( start > e )
> +        return 0;
> +
> +    /*
> +     * e is actually "end-1" because it is called by rangeset functions
> +     * which are inclusive of the last address.
> +     */
> +    e +=3D 1;
> +    size =3D (e - start) & ~(SZ_2M - 1);
> +    if ( size < MB(64) )
> +        return 0;
> +
> +    ext_regions->bank[ext_regions->nr_banks].start =3D start;
> +    ext_regions->bank[ext_regions->nr_banks].size =3D size;
> +    ext_regions->nr_banks++;
> +
> +    return 0;
> +}
> +
> +static int __init find_unallocated_memory(const struct kernel_info *kinf=
o,
> +                                          struct meminfo *ext_regions)
> +{
> +    const struct meminfo *assign_mem =3D &kinfo->mem;
> +    struct rangeset *unalloc_mem;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find unallocated memory for extended regions\n");
> +
> +    unalloc_mem =3D rangeset_new(NULL, NULL, 0);
> +    if ( !unalloc_mem )
> +        return -ENOMEM;
> +
> +    /* Start with all available RAM */
> +    for ( i =3D 0; i < bootinfo.mem.nr_banks; i++ )
> +    {
> +        start =3D bootinfo.mem.bank[i].start;
> +        end =3D bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> +        res =3D rangeset_add_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove RAM assigned to Dom0 */
> +    for ( i =3D 0; i < assign_mem->nr_banks; i++ )
> +    {
> +        start =3D assign_mem->bank[i].start;
> +        end =3D assign_mem->bank[i].start + assign_mem->bank[i].size;
> +        res =3D rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\=
n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove reserved-memory regions */
> +    for ( i =3D 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> +    {
> +        start =3D bootinfo.reserved_mem.bank[i].start;
> +        end =3D bootinfo.reserved_mem.bank[i].start +
> +            bootinfo.reserved_mem.bank[i].size;
> +        res =3D rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\=
n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove grant table region */
> +    start =3D kinfo->gnttab_start;
> +    end =3D kinfo->gnttab_start + kinfo->gnttab_size;
> +    res =3D rangeset_remove_range(unalloc_mem, start, end - 1);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    start =3D 0;
> +    end =3D (1ULL << p2m_ipa_bits) - 1;
> +    res =3D rangeset_report_ranges(unalloc_mem, start, end,
> +                                 add_ext_regions, ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks =3D 0;
> +    else if ( !ext_regions->nr_banks )
> +        res =3D -ENOENT;
> +
> +out:
> +    rangeset_destroy(unalloc_mem);
> +
> +    return res;
> +}
> +
> +static int __init find_memory_holes(const struct kernel_info *kinfo,
> +                                    struct meminfo *ext_regions)
> +{
> +    struct dt_device_node *np;
> +    struct rangeset *mem_holes;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find memory holes for extended regions\n");
> +
> +    mem_holes =3D rangeset_new(NULL, NULL, 0);
> +    if ( !mem_holes )
> +        return -ENOMEM;
> +
> +    /* Start with maximum possible addressable physical memory range */
> +    start =3D 0;
> +    end =3D (1ULL << p2m_ipa_bits) - 1;
> +    res =3D rangeset_add_range(mem_holes, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    /*
> +     * Remove regions described by "reg" and "ranges" properties where
> +     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
> +     */
> +    dt_for_each_device_node( dt_host, np )
> +    {
> +        unsigned int naddr;
> +        u64 addr, size;
> +
> +        naddr =3D dt_number_of_address(np);
> +
> +        for ( i =3D 0; i < naddr; i++ )
> +        {
> +            res =3D dt_device_get_address(np, i, &addr, &size);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\=
n",
> +                       i, dt_node_full_name(np));
> +                goto out;
> +            }
> +
> +            start =3D addr & PAGE_MASK;
> +            end =3D PAGE_ALIGN(addr + size);
> +            res =3D rangeset_remove_range(mem_holes, start, end - 1);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx=
64"\n",
> +                       start, end);
> +                goto out;
> +            }
> +        }
> +
> +        if ( dt_device_type_is_equal(np, "pci" ) )
> +        {
> +            unsigned int range_size, nr_ranges;
> +            int na, ns, pna;
> +            const __be32 *ranges;
> +            u32 len;
> +
> +            /*
> +             * Looking for non-empty ranges property which in this conte=
xt
> +             * describes the PCI host bridge aperture.
> +             */
> +            ranges =3D dt_get_property(np, "ranges", &len);
> +            if ( !ranges || !len )
> +                continue;
> +
> +            pna =3D dt_n_addr_cells(np);
> +            na =3D dt_child_n_addr_cells(np);
> +            ns =3D dt_child_n_size_cells(np);
> +            range_size =3D pna + na + ns;
> +            nr_ranges =3D len / sizeof(__be32) / range_size;
> +
> +            for ( i =3D 0; i < nr_ranges; i++, ranges +=3D range_size )
> +            {
> +                /* Skip the child address and get the parent (CPU) addre=
ss */
> +                addr =3D dt_read_number(ranges + na, pna);
> +                size =3D dt_read_number(ranges + na + pna, ns);
> +
> +                start =3D addr & PAGE_MASK;
> +                end =3D PAGE_ALIGN(addr + size);
> +                res =3D rangeset_remove_range(mem_holes, start, end - 1)=
;
> +                if ( res )
> +                {
> +                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"=
PRIx64"\n",
> +                           start, end);
> +                    goto out;
> +                }
> +            }
> +        }
> +    }
> +
> +    start =3D 0;
> +    end =3D (1ULL << p2m_ipa_bits) - 1;
> +    res =3D rangeset_report_ranges(mem_holes, start, end,
> +                                 add_ext_regions,  ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks =3D 0;
> +    else if ( !ext_regions->nr_banks )
> +        res =3D -ENOENT;
> +
> +out:
> +    rangeset_destroy(mem_holes);
> +
> +    return res;
> +}
> +
> static int __init make_hypervisor_node(struct domain *d,
>                                        const struct kernel_info *kinfo,
>                                        int addrcells, int sizecells)
> @@ -893,11 +1123,12 @@ static int __init make_hypervisor_node(struct doma=
in *d,
>     const char compat[] =3D
>         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"=
\0"
>         "xen,xen";
> -    __be32 reg[4];
> +    __be32 *reg, *cells;
>     gic_interrupt_t intr;
> -    __be32 *cells;
>     int res;
>     void *fdt =3D kinfo->fdt;
> +    struct meminfo *ext_regions =3D NULL;
> +    unsigned int i, nr_ext_regions;
>=20
>     dt_dprintk("Create hypervisor node\n");
>=20
> @@ -919,12 +1150,61 @@ static int __init make_hypervisor_node(struct doma=
in *d,
>     if ( res )
>         return res;
>=20
> +    if ( !opt_ext_regions )
> +    {
> +        printk(XENLOG_DEBUG "The extended regions support is disabled\n"=
);
> +        nr_ext_regions =3D 0;
> +    }
> +    else if ( is_32bit_domain(d) )
> +    {
> +        printk(XENLOG_DEBUG "The extended regions are only supported for=
 64-bit guest currently\n");
> +        nr_ext_regions =3D 0;
> +    }
> +    else
> +    {
> +        ext_regions =3D xzalloc(struct meminfo);
> +        if ( !ext_regions )
> +            return -ENOMEM;
> +
> +        if ( !is_iommu_enabled(d) )
> +            res =3D find_unallocated_memory(kinfo, ext_regions);
> +        else
> +            res =3D find_memory_holes(kinfo, ext_regions);
> +
> +        if ( res )
> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n=
");
> +        nr_ext_regions =3D ext_regions->nr_banks;
> +    }
> +
> +    reg =3D xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + si=
zecells));
> +    if ( !reg )
> +    {
> +        xfree(ext_regions);
> +        return -ENOMEM;
> +    }
> +
>     /* reg 0 is grant table space */
>     cells =3D &reg[0];
>     dt_child_set_range(&cells, addrcells, sizecells,
>                        kinfo->gnttab_start, kinfo->gnttab_size);
> +    /* reg 1...N are extended regions */
> +    for ( i =3D 0; i < nr_ext_regions; i++ )
> +    {
> +        u64 start =3D ext_regions->bank[i].start;
> +        u64 size =3D ext_regions->bank[i].size;
> +
> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
> +                   i, start, start + size);
> +
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> +    }
> +
>     res =3D fdt_property(fdt, "reg", reg,
> -                       dt_cells_to_size(addrcells + sizecells));
> +                       dt_cells_to_size(addrcells + sizecells) *
> +                       (nr_ext_regions + 1));
> +    xfree(ext_regions);
> +    xfree(reg);
> +
>     if ( res )
>         return res;
>=20
> --=20
> 2.7.4
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 15:37:25 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 15:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200058.354440 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVy7h-0002E5-7d; Thu, 30 Sep 2021 15:37:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200058.354440; Thu, 30 Sep 2021 15:37:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVy7h-0002Dy-4N; Thu, 30 Sep 2021 15:37:25 +0000
Received: by outflank-mailman (input) for mailman id 200058;
 Thu, 30 Sep 2021 15:37:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>) id 1mVy7f-0002Dg-K9
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 15:37:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVy7f-0002f1-9U; Thu, 30 Sep 2021 15:37:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVy7e-0005xE-Uy; Thu, 30 Sep 2021 15:37:23 +0000
Received: from iwj by osstest.test-lab.xenproject.org with local (Exim 4.92)
 (envelope-from <iwj@xenbits.xen.org>)
 id 1mVy7e-00013E-U3; Thu, 30 Sep 2021 15:37:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
To: xen-devel@lists.xenproject.org,jbeulich@suse.com,julien.grall.oss@gmail.com,iwj@xenproject.org,sstabellini@kernel.org,dpsmith@apertussolutions.com
Subject: [adhoc test] 165337: all pass
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Message-Id: <E1mVy7e-00013E-U3@osstest.test-lab.xenproject.org>
From: iwj@xenbits.xen.org
Date: Thu, 30 Sep 2021 15:37:22 +0000

[adhoc adhoc] <testing.git master /dev/pts/25>
harness 3a3089c9: mfi-common: Drop Linux dom0 i386 tests for newer Lin...
165337: all pass

flight 165337 xen-unstable adhoc [adhoc]
http://logs.test-lab.xenproject.org/osstest/logs/165337/

Perfect :-)
All tests in this flight passed as required

jobs:
 build-arm64-pvops                                            pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:05:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:05:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200069.354451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyYa-0006Hw-Eo; Thu, 30 Sep 2021 16:05:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200069.354451; Thu, 30 Sep 2021 16:05:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyYa-0006Hp-BE; Thu, 30 Sep 2021 16:05:12 +0000
Received: by outflank-mailman (input) for mailman id 200069;
 Thu, 30 Sep 2021 16:05:11 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVyYZ-0006Hj-65
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:05:11 +0000
Received: from EUR03-VE1-obe.outbound.protection.outlook.com (unknown
 [2a01:111:f400:fe09::605])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id d5a9a6b1-e0f4-49c2-b13b-6f82865f972e;
 Thu, 30 Sep 2021 16:05:07 +0000 (UTC)
Received: from DB6P193CA0003.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::13) by
 DBBPR08MB4475.eurprd08.prod.outlook.com (2603:10a6:10:cf::10) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14; Thu, 30 Sep 2021 16:05:03 +0000
Received: from DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:6:29:cafe::78) by DB6P193CA0003.outlook.office365.com
 (2603:10a6:6:29::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend
 Transport; Thu, 30 Sep 2021 16:05:03 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT023.mail.protection.outlook.com (10.152.20.68) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 16:05:03 +0000
Received: ("Tessian outbound 010023020eef:v103");
 Thu, 30 Sep 2021 16:05:03 +0000
Received: from 2308aaa785f6.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 011FA557-478E-4E8C-9ADF-B43CB2ACA342.1; 
 Thu, 30 Sep 2021 16:04:52 +0000
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2308aaa785f6.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 16:04:52 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PA4PR08MB6143.eurprd08.prod.outlook.com (2603:10a6:102:ed::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 16:04:50 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 16:04:50 +0000
Received: from smtpclient.apple (217.140.106.55) by
 LO2P265CA0280.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a1::28) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 16:04:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5a9a6b1-e0f4-49c2-b13b-6f82865f972e
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NJIU8W+o1kGXMDCjmeTdGa8w2xMckZxA/lHljKNhmXQ=;
 b=bMRLrm89eg533XRjb0TXTSYCxpcOhJJpxrYy4XQ/eK3uc5ZEFO3QW1Ot+qH5jU5LBIiw4KjMApCcWqJyTTmq7i/Q6HlcZOjzBEfVUjNnxG5k8Xs51WPsuVgx5Abj68WSyZPc1UhMBmAaHTdXeq1U+mv27gZ7Fiw/459vtCqxUgc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: 477f27175c3969a3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bFzVZFE7FED456gUuiByr8UtfFrk2Tig1qhFGnhWh/o0Nh96M/8Oaw6rilwRlf4LndG0hC6OIeLRPXgtm7XYZk190vV4IFAexs89cr3X/Pl1lSMtavKAdn4AtRKnlbt3PPgodoK13MXJqi99TJw+bhuTWsBo62uC1gCmpj/dVqoyy/+i9fa44Gh/BOUpmkTWDN4zL+ODqSXhc0aY2+GbMrm1bev6eOXNtsfGNHKt6mo5OYDvHwrN9R4i7LiPoBldKDmRa3x9YCiJNrDW4kyWuqenIwxZVMrL/E3JJ/2ED08a5TvTCG9l90yp3M6MGMt4W+Zlez+PK7XVci4pZV87zw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=NJIU8W+o1kGXMDCjmeTdGa8w2xMckZxA/lHljKNhmXQ=;
 b=lpK5jnwh0rJf1wOlTN7JsDP1vKOFPcTvkPv/kiT/oa2pMykvLp6C7cuf1Ypt2W73bB3VQNSuwU+rQ3BBHqDtEgrvdRAMvmCYJ35vAomW63BBxOaU+NJyuS7HnAfujD3s6wcu+sxhdp6ZRQrT+JFp440IWFw7zaGwxfwIdvhVNQxHLRQZpW6ylGKWjCvdnasrPBIGJDM/N073dfRPry3ZDnimpW2Qt34fv5NUcKuzvjagw/s1laxNpAXvFZNIpEsh1XqsWlzR/lqWE5fdPSZ21zJ7VGocQjkJu4j7zteRF/igNzT65u/aY2oqXgj6iD24yBhCVMJIhD4CpaJh5TmxyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NJIU8W+o1kGXMDCjmeTdGa8w2xMckZxA/lHljKNhmXQ=;
 b=bMRLrm89eg533XRjb0TXTSYCxpcOhJJpxrYy4XQ/eK3uc5ZEFO3QW1Ot+qH5jU5LBIiw4KjMApCcWqJyTTmq7i/Q6HlcZOjzBEfVUjNnxG5k8Xs51WPsuVgx5Abj68WSyZPc1UhMBmAaHTdXeq1U+mv27gZ7Fiw/459vtCqxUgc=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v2 2/3] include/public: add better interface description
 to usbif.h
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <20210929074620.28495-3-jgross@suse.com>
Date: Thu, 30 Sep 2021 17:04:47 +0100
Cc: xen-devel@lists.xenproject.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <121F36C1-243F-4546-8B27-8315EE580D5C@arm.com>
References: <20210929074620.28495-1-jgross@suse.com>
 <20210929074620.28495-3-jgross@suse.com>
To: Juergen Gross <jgross@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO2P265CA0280.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a1::28) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: abe1b250-d704-4bdc-f5a3-08d9842c1003
X-MS-TrafficTypeDiagnostic: PA4PR08MB6143:|DBBPR08MB4475:
X-Microsoft-Antispam-PRVS:
	<DBBPR08MB447597CE86B1A5FCBFC4A10FE4AA9@DBBPR08MB4475.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 6ulF3fe4Ni8l6dR0Xg/UwuVZ7SXHrOq7JjOzQ9FjhByOv1ROaPdu8wyvQ3XRtIPUDyhxYI5wgvhNl1mLOdB2blVp/xoder2uHmFz7T36i9ps9s0oUT924Ozos584PSUwAECVxxU45KaSo9xrZT3C/FhZ3MrSFsNd0XloLB3z1KuG+ZpxNtMP3yAFh0k8XPv+ElUeYbGPfId1NwvVC3fQay0zw8wBie7KvWiL+Hfwfofi+L4YfFDs2+NVKRfcaabC6BBRL/PwMUqOOqXjip3YBsCm98LptPNk4wxTfCyw+pxhpJvE+RVzgBPOnKS7raMrhl1VAjFiquZr5Olzl00SMsANq2lZcstbnFNupEVTw0rgz4llApLJumDKii6zn+grngn/PyCPvfIj+QI78asGsCMalrq3xh8X9eOkGn9qsFwxvSGMpaK5++U3w7u2VQAsZ5yxiYbg/MGeAhPRfggn6TP1HtwS++zL1zLQNYdrfXaq2UXWbAVYweZ3h2S4hH8zGNGrRsTZ5QcR6Pupc1VlUqHOUbadyP22gSCOqLV9Eb5tpHRDAktKLKIpg7pQlWeXRSrvSZiNPxgx0gAzkmTw5vT70EUmV3Y6gF6NSCmrB5c0PZZqiqyJhGYb0DI39sfwlwYGz5qoUoZfxuvYBOxOXBW3f4GDv2/QRoXEW3hOsjWQqqWxgLhNnsp1gOenj56/Tlxf8u2HAE/e73QXHZufXY/uLBfz/Uu6vY4JZkXUv9A=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(6486002)(44832011)(4326008)(6512007)(33656002)(508600001)(8936002)(5660300002)(316002)(8676002)(38350700002)(2906002)(38100700002)(86362001)(6666004)(26005)(186003)(2616005)(956004)(36756003)(6506007)(53546011)(83380400001)(52116002)(66946007)(66476007)(66556008)(6916009)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6143
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	3903dc97-f474-44ef-fc0d-08d9842c0839
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZtpLbesjXIq7+3gmYYZeshWB/yikYaF33yu2kUnkYnWWwBGZizeYD2m8shyJwPUSG2lZ1xmUBtKuJ8LCCGxwdulnDRQwnLvEi0XN+znHIGR1Cpd8PJmr+J4vJm/9H5OeQJTQ1XfZ0IkMoQTyn2dobr5dLo4fZLGumryH1+W8p+gJcGpMbfAbXRLvQ6eZplm2gz2MybqhFQOUXgfGWTf5mmJpbz7Yf/FJaAxo57H9t6LCB+4eVGsE0DWQOCfN5Jc/fZtPpELsfb4dhGeS3TE5gbArhHhEIQniHFOYPSsiTNKBhV/w9OonsgLBvfSiauPy5eIGYdFK6/E3GJnSPpYWbzcpVBUvP5QIeX4zUG3wpXQepKpPxRS5Wh/aN0gMxU5byugRW0KqzKPSZtJb9wkINzlLI00DhmGxdRnUyzhO7vl83TkqNsyo712S2mnvOPDPYhdiXM1we5n8irCHsGbEAV3bM67HHoRzeRkP3mw+JQYSAgNBlqhan6olEDJqo4A3XeYnMrlELURdlAm5mrK5QCjLbVhUIlB/LrNo2F8gtN3uGS6IRVJaw/l4YA4ZAcPfO5TJ/DQFztDv2qwYRiNoQ0S1f59cTJfJEjKazvGMD1GvCih5JjQfzvtteyjl+3sQu4C3ar0iISnQSfTCWEJaGzR6y1Q67nNIrpdQlAQf0cLkRonVszEMQrR8VfE1KGnNY0pWSGcneJ9GSxr+PBgxwQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(70206006)(36756003)(26005)(53546011)(47076005)(4326008)(70586007)(356005)(83380400001)(81166007)(36860700001)(5660300002)(6486002)(86362001)(6506007)(336012)(956004)(316002)(2616005)(8676002)(33656002)(6666004)(6512007)(82310400003)(186003)(6862004)(2906002)(508600001)(44832011)(8936002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 16:05:03.1678
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: abe1b250-d704-4bdc-f5a3-08d9842c1003
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT023.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4475



> On 29 Sep 2021, at 08:46, Juergen Gross <jgross@suse.com> wrote:
>=20
> The PV USB protocol is poorly described. Add a more detailed
> description to the usbif.h header file.
>=20
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

> ---
> xen/include/public/io/usbif.h | 164 ++++++++++++++++++++++++++++++++++
> 1 file changed, 164 insertions(+)
>=20
> diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.=
h
> index 96b9fb661d..dd378bcba5 100644
> --- a/xen/include/public/io/usbif.h
> +++ b/xen/include/public/io/usbif.h
> @@ -32,6 +32,34 @@
> #include "../grant_table.h"
>=20
> /*
> + * Detailed Interface Description
> + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> + * The pvUSB interface is using a split driver design: a frontend driver=
 in
> + * the guest and a backend driver in a driver domain (normally dom0) hav=
ing
> + * access to the physical USB device(s) being passed to the guest.
> + *
> + * The frontend and backend drivers use XenStore to initiate the connect=
ion
> + * between them, the I/O activity is handled via two shared ring pages a=
nd an
> + * event channel. As the interface between frontend and backend is at th=
e USB
> + * host connector level, multiple (up to 31) physical USB devices can be
> + * handled by a single connection.
> + *
> + * The Xen pvUSB device name is "qusb", so the frontend's XenStore entri=
es are
> + * to be found under "device/qusb", while the backend's XenStore entries=
 are
> + * under "backend/<guest-dom-id>/qusb".
> + *
> + * When a new pvUSB connection is established, the frontend needs to set=
up the
> + * two shared ring pages for communication and the event channel. The ri=
ng
> + * pages need to be made available to the backend via the grant table
> + * interface.
> + *
> + * One of the shared ring pages is used by the backend to inform the fro=
ntend
> + * about USB device plug events (device to be added or removed). This is=
 the
> + * "conn-ring".
> + *
> + * The other ring page is used for USB I/O communication (requests and
> + * responses). This is the "urb-ring".
> + *
>  * Feature and Parameter Negotiation
>  * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>  * The two halves of a Xen pvUSB driver utilize nodes within the XenStore=
 to
> @@ -99,6 +127,142 @@
>  *      The machine ABI rules governing the format of all ring request an=
d
>  *      response structures.
>  *
> + * Protocol Description
> + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> + *
> + *-------------------------- USB device plug events --------------------=
------
> + *
> + * USB device plug events are send via the "conn-ring" shared page. As o=
nly
> + * events are being sent, the respective requests from the frontend to t=
he
> + * backend are just dummy ones.
> + * The events sent to the frontend have the following layout:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |               id                |    portnum     |     speed      |=
 4
> + * +----------------+----------------+----------------+----------------+
> + *   id - uint16_t, event id (taken from the actual frontend dummy reque=
st)
> + *   portnum - uint8_t, port number (1 ... 31)
> + *   speed - uint8_t, device USBIF_SPEED_*, USBIF_SPEED_NONE =3D=3D unpl=
ug
> + *
> + * The dummy request:
> + *         0                1        octet
> + * +----------------+----------------+
> + * |               id                | 2
> + * +----------------+----------------+
> + *   id - uint16_t, guest supplied value (no need for being unique)
> + *
> + *-------------------------- USB I/O request ---------------------------=
------
> + *
> + * A single USB I/O request on the "urb-ring" has the following layout:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |               id                |         nr_buffer_segs          |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |                               pipe                                |=
 8
> + * +----------------+----------------+----------------+----------------+
> + * |         transfer_flags          |          buffer_length          |=
 12
> + * +----------------+----------------+----------------+----------------+
> + * |                       request type specific                       |=
 16
> + * |                               data                                |=
 20
> + * +----------------+----------------+----------------+----------------+
> + * |                              seg[0]                               |=
 24
> + * |                               data                                |=
 28
> + * +----------------+----------------+----------------+----------------+
> + * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
> + * +----------------+----------------+----------------+----------------+
> + * |             seg[USBIF_MAX_SEGMENTS_PER_REQUEST - 1]               |=
 144
> + * |                               data                                |=
 148
> + * +----------------+----------------+----------------+----------------+
> + * Bit field bit number 0 is always least significant bit, undefined bit=
s must
> + * be zero.
> + *   id - uint16_t, guest supplied value
> + *   nr_buffer_segs - uint16_t, number of segment entries in seg[] array
> + *   pipe - uint32_t, bit field with multiple information:
> + *     bits 0-4: port request to send to
> + *     bit 5: unlink request with specified id (cancel I/O) if set (see =
below)
> + *     bit 7: direction (1 =3D read from device)
> + *     bits 8-14: device number on port
> + *     bits 15-18: endpoint of device
> + *     bits 30-31: request type: 00 =3D isochronous, 01 =3D interrupt,
> + *                               10 =3D control, 11 =3D bulk
> + *   transfer_flags - uint16_t, bit field with processing flags:
> + *     bit 0: less data than specified allowed
> + *   buffer_length - uint16_t, total length of data
> + *   request type specific data - 8 bytes, see below
> + *   seg[] - array with 8 byte elements, see below
> + *
> + * Request type specific data for isochronous request:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |            interval             |           start_frame           |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |       number_of_packets         |       nr_frame_desc_segs        |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *   interval - uint16_t, time interval in msecs between frames
> + *   start_frame - uint16_t, start frame number
> + *   number_of_packets - uint16_t, number of packets to transfer
> + *   nr_frame_desc_segs - uint16_t number of seg[] frame descriptors ele=
ments
> + *
> + * Request type specific data for interrupt request:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |            interval             |                0                |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |                                 0                                 |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *   interval - uint16_t, time in msecs until interruption
> + *
> + * Request type specific data for control request:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |                      data of setup packet                         |=
 4
> + * |                                                                   |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *
> + * Request type specific data for bulk request:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |                                 0                                 |=
 4
> + * |                                 0                                 |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *
> + * Request type specific data for unlink request:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |           unlink_id             |                0                |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |                                 0                                 |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *   unlink_id - uint16_t, request id of request to terminate
> + *
> + * seg[] array element layout:
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |                               gref                                |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |             offset              |             length              |=
 8
> + * +----------------+----------------+----------------+----------------+
> + *   gref - uint32_t, grant reference of buffer page
> + *   offset - uint16_t, offset of buffer start in page
> + *   length - uint16_t, length of buffer in page
> + *
> + *-------------------------- USB I/O response --------------------------=
------
> + *
> + *         0                1                 2               3        o=
ctet
> + * +----------------+----------------+----------------+----------------+
> + * |               id                |          start_frame            |=
 4
> + * +----------------+----------------+----------------+----------------+
> + * |                              status                               |=
 8
> + * +----------------+----------------+----------------+----------------+
> + * |                          actual_length                            |=
 12
> + * +----------------+----------------+----------------+----------------+
> + * |                           error_count                             |=
 16
> + * +----------------+----------------+----------------+----------------+
> + *   id - uint16_t, id of the request this response belongs to
> + *   start_frame - uint16_t, start_frame this response (iso requests onl=
y)
> + *   status - int32_t, USBIF_STATUS_* (non-iso requests)
> + *   actual_length - uint32_t, actual size of data transferred
> + *   error_count - uint32_t, number of errors (iso requests)
>  */
>=20
> enum usb_spec_version {
> --=20
> 2.26.2
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:09:30 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:09:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200076.354462 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVycj-000763-Vf; Thu, 30 Sep 2021 16:09:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200076.354462; Thu, 30 Sep 2021 16:09:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVycj-00075w-SI; Thu, 30 Sep 2021 16:09:29 +0000
Received: by outflank-mailman (input) for mailman id 200076;
 Thu, 30 Sep 2021 16:09:29 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=v5L4=OU=arm.com=Luca.Fancellu@srs-us1.protection.inumbo.net>)
 id 1mVycj-00075q-2Z
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:09:29 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (unknown
 [40.107.6.79]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id c93859c9-2208-11ec-bd59-12813bfff9fa;
 Thu, 30 Sep 2021 16:09:27 +0000 (UTC)
Received: from DB6PR0301CA0052.eurprd03.prod.outlook.com (2603:10a6:4:54::20)
 by AM6PR08MB4182.eurprd08.prod.outlook.com (2603:10a6:20b:ae::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 16:09:20 +0000
Received: from DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
 (2603:10a6:4:54:cafe::ab) by DB6PR0301CA0052.outlook.office365.com
 (2603:10a6:4:54::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend
 Transport; Thu, 30 Sep 2021 16:09:20 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DB5EUR03FT003.mail.protection.outlook.com (10.152.20.157) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 16:09:19 +0000
Received: ("Tessian outbound c21c48fbc857:v103");
 Thu, 30 Sep 2021 16:09:19 +0000
Received: from 8ecf0c33fb25.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 3F5306B8-6B91-4CB9-82D8-DD5E0486C22A.1; 
 Thu, 30 Sep 2021 16:09:07 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8ecf0c33fb25.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 30 Sep 2021 16:09:07 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com (2603:10a6:102:130::10)
 by PR2PR08MB4745.eurprd08.prod.outlook.com (2603:10a6:101:1a::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep
 2021 16:09:05 +0000
Received: from PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697]) by PAXPR08MB6816.eurprd08.prod.outlook.com
 ([fe80::c1b4:db1c:376f:b697%9]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 16:09:05 +0000
Received: from smtpclient.apple (217.140.106.55) by
 LO4P123CA0058.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:153::9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 16:09:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c93859c9-2208-11ec-bd59-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DfvFlHQbUs9B4O1IsuZ2sz6yZd7cM0C64X4xZDlnGJM=;
 b=VCsK7eA12xqUuTUsEIKE1ROYdqI0xzNKFhAmWVGcItdT3OzSThoUtL1dLthJpT+FZSa6rcovawnvtVVCmPbeb2vosqIYsOjaFCzxARzQreVFuJoTqNWMdDHLsi2/b1T4uZ1ytuXniAr8ir/tTbjl3jnHLXdFpW6dTJIpA8q451I=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; lists.xenproject.org; dkim=pass (signature was
 verified) header.d=armh.onmicrosoft.com;lists.xenproject.org; dmarc=pass
 action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
X-CheckRecipientChecked: true
X-CR-MTA-CID: c61d9c83a9056149
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=facK5Jhf0mhGT7N+saymLPcHeo4Nn84HnE4eTaZXpTXKt9Zjs3crEHAZnmczwgTMVAHk+VDEKSAo90S80MMyIm10mZczdae/BuEpMBBVjoKFnVuWTB6haECMItp5VoUSuX1wCNWwR3Wo0t/hEnWJ0giHe00TZ5+F28XFnL1Aahm+UOlRsSh4HYyEoLznfFIv3YbVsFjdzgbCncnVklcqPhksxgs/7hdPylxHaXCwsaU6/fEmTaKyJ6c3KnX7fxZmLor4c4HH+NG40uKoeZ2fhBsPOaqqyl9WCUjt1USVcfH8g4vCtZl+JflDninh0URU7HJZ77wQYsSE5+5abOpWhQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=DfvFlHQbUs9B4O1IsuZ2sz6yZd7cM0C64X4xZDlnGJM=;
 b=PjUJFpeK9HG0aagrvYz5rY2/2G2s7ggLn8hToAbKLVGw/XaricYQh2rBg+27w7iqhQbb1nNwS9Zml5q0h74QepgBVWtyzjQmqilPkaLpze0g1FKyPg4BZoiMzP3RQQXSA38h8JvWwkSKFQLOwElR33iKESBCZe/0xT8PzZ3C0o2HeTM6kiPSXQ0db+qLnFyKHGanmV8tgw48HEkGsPMXgKZPNVY2/OUEQrk8G/zervzx9sv7T15OWogV3ekfXziRTDP5XvYx9/lVLQ8+nWwFpHrASigNC1yWHOrDEngyxEUxDPWzyP3BppuFMki/2K+EX6ofXbU+r4faPsu9DSwrBQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DfvFlHQbUs9B4O1IsuZ2sz6yZd7cM0C64X4xZDlnGJM=;
 b=VCsK7eA12xqUuTUsEIKE1ROYdqI0xzNKFhAmWVGcItdT3OzSThoUtL1dLthJpT+FZSa6rcovawnvtVVCmPbeb2vosqIYsOjaFCzxARzQreVFuJoTqNWMdDHLsi2/b1T4uZ1ytuXniAr8ir/tTbjl3jnHLXdFpW6dTJIpA8q451I=
Authentication-Results-Original: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
Content-Type: text/plain;
	charset=us-ascii
Subject: Re: [PATCH v2 1/3] include/public: add possible status values to
 usbif.h
From: Luca Fancellu <luca.fancellu@arm.com>
In-Reply-To: <20210929074620.28495-2-jgross@suse.com>
Date: Thu, 30 Sep 2021 17:09:02 +0100
Cc: xen-devel@lists.xenproject.org
Content-Transfer-Encoding: 7bit
Message-Id: <4EB52D64-766F-485F-9B24-D213E8B8BC49@arm.com>
References: <20210929074620.28495-1-jgross@suse.com>
 <20210929074620.28495-2-jgross@suse.com>
To: Juergen Gross <jgross@suse.com>
X-Mailer: Apple Mail (2.3654.120.0.1.13)
X-ClientProxiedBy: LO4P123CA0058.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:153::9) To PAXPR08MB6816.eurprd08.prod.outlook.com
 (2603:10a6:102:130::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f9afb4cf-5e67-4c3d-a2d2-08d9842ca90c
X-MS-TrafficTypeDiagnostic: PR2PR08MB4745:|AM6PR08MB4182:
X-Microsoft-Antispam-PRVS:
	<AM6PR08MB4182FE5C06A32BEB54F11254E4AA9@AM6PR08MB4182.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
NoDisclaimer: true
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 AGmPcCaodY5m3XE8qBdv8cprNUQnhLH74qwypERvZKNuLQ3055+VG933N3YOju3soTActWDt3EWJkMCW2d0Xx3eDrbqYqgRaPyZs8nbZadKCYkbKbnFxk5dIGzE6Np4ew1Eqz5sM8m8WkGCCKxO3+fkmh5yIn+YmX+79W5GQk/1kOOY1bp5kZp47zQqH3G0zHWooP53kgvBt+FIC79wGHuyoeHkgxIRQRbWH4N8fKQe9I0auCJYO4KqvjlVsRW5m3XHKPsbzr52NBQch25ILXTwQ0OsJ0PD2KQ5hz2ZLTt+m3tUdVU9mYld11sncxArWorqir9h1JBVv0xgxn5Y6ml0oi2xeyVSrKeN4EIijT19aaQMW6E8mW9WdrgZX5p1fNd4BJ9MZhMsMD4hd4WGQtteQHzG1WOJzwwiO0ZfSdJjYyEmXYQGIFsgRUbyS2dHFiJBDRxwuXazXaWWysypX5GtAtpNDzigPlqncuomweUC8PvWViOLiHrJoHXjtCoaj6eJ6aZauHXl+NEE6WeVKji2AyI+SGvfdLMT4e8WrvyYYxlp3ZHDoULzcal+l2SjQJP/QgLwzVy0rwRTuyPuEVPaWWoZ7xUw9ag7DanZUTW72iMuxj5U8HVryiu8Trtqb1KG3sX+NWQcHsiN7z+tKAyKrt6IXs1MbRD1s1Ouog60Z6Chyxj0wqnxKKJqwWlMfN7eK+wIrCY0OI4MNC4x0+6UayYLxCGatJL0dJRV15JU=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR08MB6816.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(8676002)(6916009)(26005)(5660300002)(2906002)(66946007)(8936002)(44832011)(33656002)(186003)(6512007)(956004)(38350700002)(66556008)(316002)(66476007)(6506007)(52116002)(53546011)(2616005)(83380400001)(86362001)(36756003)(4326008)(6666004)(6486002)(38100700002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4745
Original-Authentication-Results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6f05700d-7826-4559-a8c1-08d9842ca065
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YQxO9CuvrGbVWWtGM9TMbNpLb+tENYgZ80d5dX+186TvXUKY7IXl4iVPXxfXoq7lcuTxyJl9mqmQzsynB1tCPSQIoXRArlL1Z8HJorfGzEiHEm8ny2TZt0aUjuRsH9whjIuIRsf4b7SacOgpUfK6u38ovHaEVgMRD5e2FMKQ+YHXuenp27GScfmozam/jl/sL6GptstnZ60QpXFtKJ8wBHZTfSg+Vm3PDHEJhHT6CY/aAVQIQAu9xTwBaXOa7KBLUmoeYpBB/nrG//1wgU/tncx7Cdz+tGVqR4V5Qdthn+vg1WVv94ledrIx4DkEz3im3wLIGNGlYWWN4fhp2QxJDP58RcJSg/9WRILjyGlrjjJd9R8XDfkl3OOOP9jGrY8pwxzFuGsqZ9LpS4LBdhqa3OmHNxvU4xiKiznMrbVAg9KAuVXKaVvkUFoe7uBadwKW1B0n1YAO3l3qDdsXVr3CuQo+Qvmrkxn4zohrIpK9eDnRH8a8JcRweQI44gTyA+yDRRz53EbirJRg9oE08O2EfVDWGoam+3EGq9GCe/xI9vrB8kQuK2HIXUbRUi50YNlFB0B4ybbOaaYq9//B56Sb2cGfZ25yECyLjdp5DS6HAUso2izWakDZwaK6dawtxHogYKWXkqZjPGZ1kkbn5iZniix5mZHd7kadMhd3vg5UIE6nayelg2vaAkInxYjYQREspL/jwcPZF7H+W6H1Ze4nbg==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(6486002)(44832011)(4326008)(6512007)(33656002)(36860700001)(6862004)(508600001)(8936002)(5660300002)(47076005)(316002)(8676002)(356005)(81166007)(2906002)(82310400003)(86362001)(6666004)(26005)(186003)(2616005)(956004)(36756003)(336012)(6506007)(53546011)(83380400001)(70586007)(70206006);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 16:09:19.9180
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f9afb4cf-5e67-4c3d-a2d2-08d9842ca90c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DB5EUR03FT003.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4182



> On 29 Sep 2021, at 08:46, Juergen Gross <jgross@suse.com> wrote:
> 
> The interface definition of PV USB devices is lacking the specification
> of possible values of the status field in a response. Those are
> negative errno values as used in Linux, so they might differ in other
> OS's. Specify them via appropriate defines.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

> ---
> V2:
> - add parentheses to macro values (Jan Beulich)
> ---
> xen/include/public/io/usbif.h | 7 +++++++
> 1 file changed, 7 insertions(+)
> 
> diff --git a/xen/include/public/io/usbif.h b/xen/include/public/io/usbif.h
> index c6a58639d6..96b9fb661d 100644
> --- a/xen/include/public/io/usbif.h
> +++ b/xen/include/public/io/usbif.h
> @@ -221,6 +221,13 @@ struct usbif_urb_response {
> 	uint16_t id; /* request id */
> 	uint16_t start_frame;  /* start frame (ISO) */
> 	int32_t status; /* status (non-ISO) */
> +#define USBIF_STATUS_OK		0
> +#define USBIF_STATUS_NODEV	(-19)
> +#define USBIF_STATUS_INVAL	(-22)
> +#define USBIF_STATUS_STALL	(-32)
> +#define USBIF_STATUS_IOERROR	(-71)
> +#define USBIF_STATUS_BABBLE	(-75)
> +#define USBIF_STATUS_SHUTDOWN	(-108)
> 	int32_t actual_length; /* actual transfer length */
> 	int32_t error_count; /* number of ISO errors */
> };
> -- 
> 2.26.2
> 
> 



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:17:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:17:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200087.354473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVykW-0000Gu-Tb; Thu, 30 Sep 2021 16:17:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200087.354473; Thu, 30 Sep 2021 16:17:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVykW-0000Gn-QH; Thu, 30 Sep 2021 16:17:32 +0000
Received: by outflank-mailman (input) for mailman id 200087;
 Thu, 30 Sep 2021 16:17:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5o/Z=OU=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVykV-0000GN-Dr
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:17:31 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id cc82e4a3-6dcd-4676-a9d2-523f2aa18138;
 Thu, 30 Sep 2021 16:17:30 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc82e4a3-6dcd-4676-a9d2-523f2aa18138
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633018650;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=SbncEBKHNrhIHO54GucIwwh3ENu4Nen3Olbl5JVgzeY=;
  b=hICE/2GUxPLgX04X1fK8bauyBoj+4KhUhfLG8TNw+ejisEQ+sQKojLlQ
   hkOCmBcaVy5vsGv0/LFf3iPLc1RKFo12BYDCYDvo3uVlpZxyzPe11Xum3
   6sbhNK3YJsp7eHT79dpE5RSr2Rx/HJNBk+3nagW4gf0U2MknBUcj978LU
   8=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: Rvo5U9IbjyivvlhtvlaEWHHxFxHghdkEqZyiPaYfXTaGCoXUprceQ1l4kCo6RiEryAO2LxM1kw
 XbC1CFZVtLXBIz9veIm8h0SKbKk0EEjeotqWNceY5PGUbvjiZ8fd5Pjve7EH8iDL1Mwh/WfeOx
 P9Z1XRMybn/edMOSGDCTvdUVf0NRUZIbStsuV5rlqEjP2Oom4H71JXe3qGYpw5ho7Lgl+31T93
 gLicUbTtOePLQzRNq/Jnt8MA1+Cnm+M+xOeHHSMRTz4vRW3r6C2NH1jhENVt03fgUYDodPsZ8L
 /8YusxZxzBR+q9jESNsNhiFJ
X-SBRS: 5.1
X-MesageID: 54035940
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:0K1Nb6BLK5ppnxVW//Xkw5YqxClBgxIJ4kV8jS/XYbTApGwrgzIDm
 2UeXmGDOP+NMzCkctF+bY22oxgE75bVyoAxQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLOs1hxZH1c+EX9w0ko7wYbVv6Yz6TSHK1LV0
 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5
 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/oGXWosxrm
 c90mY2taEQHYpX0w+IzTEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr
 KFGbmBWBvyAr7veLLaTQ+9whsMlPY/zMZkWoH1IxjDFF/c2B5vERs0m4PcFgG9r25ESRZ4yY
 eI6djhFV0XBTiFkP0gVFoIjg+T5pnrGJmgwRFW9+vNsvjm7IBZK+LPkKtbSd/SBTN9ZmUver
 WXDl0zwBA0aM9qOjyKI6Hu2isfAmCr6XMQZE7jQ3u5nhhify3IeDDUSVECnur+ph0imQdVdJ
 kcIvC00osAPGFeDF4enGUfi+Tjd40BaC4E4//AGBB+l4fqL3hqDV3I9Yi9cSZ8I6PcOHjoEy
 Qrc9z/2PgBHvLqQQHOb076bqzKuJCQYRVM/iT84oRgtuYe6/9Bq5v7bZpMzSPfq0o2d9STYm
 WjikcQou1kEYSfnPY2V+krbyxalu5TEVAI84gi/somNtVgiPN/Ni2BF8zHmARd8wGSxEgTpU
 JsswZH2AAUy4Xelz3flrAIlRuzB2hp9GGeA6WOD5rF4n9hXx1atfJpL/BZ1L1pzP8APdFfBO
 RGI41INvM4DYCX7Msebhr5d7exxlsAM8vy/C5jpgidmOMAtJGdrAgk0DaJv44wduBd1yvxuU
 XtqWc2tEWwbGcxaIMmeHI8gPUsQ7nlmnwv7HMmjpzz+iOb2TCPFGN8tbQrVBshkvfzsnekg2
 4sGXyd8404EC7OWj+i+2dN7EG3m2lBhXs2p95wLKLTTSuekcUl4Y8LsLXoaU9QNt8xoei3go
 hlRg2dUlwjyg2PpMwKPZiwxYb/jR88n/3k6ITYtLRCj3H16OdSj66IWdp0We7g79bM8ka4oH
 qddI8jQUO5STjnn+igGacWvpoJVaxn21xmFODCoYWZjcsc4FRDJ4NLtYiDm6DIKUnisrcI7r
 rD5jlHbTJMPSh5MFsHTbP7znVq9sWJEwLB5XlfSI8kVc0LpqdA4Jyv0h/4xAscNNRScmWfKi
 1fIWU8V/LCfrZU0/d/FgbG/g72oS+YuTFBHG2T77KqtMXWI9GSU3oIdAv2DeirQVT2o9fz6N
 /lV1fz1LNYOgE1O79hnC79uwK8zu4nvqrtdwlg2FXnHdQ32WLZpI33A1shTrKxdgLRevFLuC
 E6I/9BbP5SPOd/kTwFNdFZ0MLzb2KFGgCTW4NQ0PF7+tX1+87ewWElPOwWB1X5GJ7xvPYJ5m
 eostab6MeBkZsbG5jpesh1pyg==
IronPort-HdrOrdr: A9a23:ebugV6w0nH+G4UexYmIqKrPw1r1zdoMgy1knxilNoHxuH/BwWf
 rPoB17726RtN91YhsdcL+7V5VoLUmzyXcX2/h1AV7BZniEhILAFugLgbcKqweKJ8SUzJ8+6U
 4PSclD4N2bNykGsS75ijPIb+rJFrO8gd+VbeS19QYScelzAZsQiDuQkmygYzZLrA8tP+teKL
 OsovBpihCHYnotYsGyFhA+LpL+T42iruOeXfYebSRXkDWzsQ==
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54035940"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Anthony PERARD
	<anthony.perard@gmail.com>, Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH 2/2] automation: Add qemu to debian:stretch container for smoke test
Date: Thu, 30 Sep 2021 17:17:20 +0100
Message-ID: <20210930161720.825098-3-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210930161720.825098-1-anthony.perard@citrix.com>
References: <20210930161720.825098-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Anthony PERARD <anthony.perard@gmail.com>

We can add qemu into the container so that there's no need to install
it everytime we run a test.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Also, smoke tests stopped working as of today due to outdated
root certificate, so container needs to be updated anyway.
    fatal: unable to access 'https://xenbits.xen.org/git-http/xtf.git/': server certificate verification failed.

I haven't push the container yet, I've only pushed it as
debian:tmp-stretch for testing it.
---
 automation/build/debian/stretch.dockerfile | 2 ++
 automation/scripts/qemu-smoke-x86-64.sh    | 8 +++++---
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/automation/build/debian/stretch.dockerfile b/automation/build/debian/stretch.dockerfile
index e2ee14e2017c..da6aa874dd70 100644
--- a/automation/build/debian/stretch.dockerfile
+++ b/automation/build/debian/stretch.dockerfile
@@ -47,6 +47,8 @@ RUN apt-get update && \
         nasm \
         gnupg \
         apt-transport-https \
+        # for test phase, qemu-smoke-* jobs
+        qemu-system-x86 \
         && \
         apt-get autoremove -y && \
         apt-get clean && \
diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
index 4b176c508dec..8ac065491c75 100755
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -6,9 +6,11 @@ set -ex
 variant=$1
 
 # Install QEMU
-export DEBIAN_FRONTENT=noninteractive
-apt-get -qy update
-apt-get -qy install qemu-system-x86
+if ! type qemu-system-x86_64; then
+    export DEBIAN_FRONTENT=noninteractive
+    apt-get -qy update
+    apt-get -qy install qemu-system-x86
+fi
 
 # Clone and build XTF
 git clone https://xenbits.xen.org/git-http/xtf.git
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:17:42 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:17:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200088.354484 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVykg-0000ar-4y; Thu, 30 Sep 2021 16:17:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200088.354484; Thu, 30 Sep 2021 16:17:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVykg-0000ai-22; Thu, 30 Sep 2021 16:17:42 +0000
Received: by outflank-mailman (input) for mailman id 200088;
 Thu, 30 Sep 2021 16:17:40 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5o/Z=OU=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVyke-0000ZS-BF
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:17:40 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 42b9bd05-d711-476c-8568-50f406b3dec7;
 Thu, 30 Sep 2021 16:17:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42b9bd05-d711-476c-8568-50f406b3dec7
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633018659;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=eutcveOxLfIyiRm0KCSSaQX2O4MlKKNJAO3ZP0okZ28=;
  b=NqoCNh4qYnef+e9X5665iNazUfaL1aeXpUj+FKzwJypCSknw9SCJoVD3
   /IaoOjstZClLhk3aWe6Y7gpzRp2X30BoOjbpskDyZO+y2T2h6JcY3WeAw
   xvGBsCQL70+biPlVcfxwoQz4qHcPk1gz4kEwMJgXPohWJ7i9SuWyZI/XH
   w=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: X+cj+Z3bTgHRg7kDsy/+WpD2LrTvAvHDWe4FteYjwFgBL3gP0NYOY4OL9BEXD5OBiqSmCGmqLy
 d3a5BFIm//WXMRHZPeLTj3tVQN75LqPlXuW3JYluMO/FcLnjptqmKIRhp49aGMTZpA3a5W3dzU
 Uz1xAxhVfW7yUgDemqICZjgruRUf3J68qho91LO3cLN5TzZhEdYzr0mk7QYgqVspjYm8mkp2Sf
 XDC7BR08PV92dipxzQOgmch7NU/aVnwWae/IxSdIWRm1M+P5zIwux4/B5ZlkSb4zToHKdpyc+z
 qc+bfA/4oZRGdy8wJ4YpcfRT
X-SBRS: 5.1
X-MesageID: 54035975
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:7LviV60iT+wqH/jMdvbD5Q92kn2cJEfYwER7XKvMYLTBsI5bpzRVz
 jceDGDTP/qJNGb2foxzPNjl9R5UsZ7UyIMxTlNrpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCan0ZqTNMEn970EoywLZh2OaEvPDia++zk
 YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq
 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhn8947
 IxmuISJWR4YMLD1vfsTYShcHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr
 6VecW1TKEDY7w616OvTpu1Ej8I/LM7tLcUHt2tp1z3xBvc6W5HTBa7N4Le02R9t2pgVQKeBO
 qL1bxJkcx/JSRdXOG0ZVsoMo+eypUnzLwNx/Qf9Sa0fvDGIkV0ZPKLWGNHcYNCNSO1el1yUo
 W+A9GP8ajkTMsCfzjyUtGqhnObVkQv0XYsTEPuz8fsCqE2ewCkfBQMbUXO/oOKlkQiuVtRHM
 UsW9yEy668o+ySWosLVBkPi5iTe51hFBoQWQ7ZSBByxJrT84xmQIjEvQyZ4eMUXutIPYCMk5
 G6NkIa8bdBwi4G9RXWY/7aSiDq9PykJMGMPDRM5oRs5D8rL+t9q0UOWJjp3OOvl1oSqRWGYL
 yWi9nBWulkFsSIcO0xXF3jphCiw7rzAUwI4/AneWm/NAuhRP9X+P93ABbQ29599wGelorup4
 CVsdyu2trlm4XSxeMqlG75l8FaBvavtDdEkqQQzd6TNDhz0k5JZQahe4StlOGBiOdsedDnib
 Sf74F0Kvs4PZCr6MfEmPOpd7vjGK4C7TrwJsdiOM7JzjmVZLlfbrEmCm2bKt4wSrKTcuf5mY
 srKGSpdJX0bFb5m3FKLqxQ1itcWKtQF7TqLH/jTlk3/uZLHPSL9YepVYTOmM7FihIvZ8Vq9z
 jqqH5bTo/mpeLalOXe/HE96BQ1iEEXX8ris8JEIKr7ffFA2cIzjYteIqY4cl0Vet/w9vo/1E
 ruVBie0EXLz2i/KLxukcHdmZO+9VJpztytjbyctIUypyz4oZoP2tPUTcJ4+fL8G8u1/zKErE
 6lZKpvYWvkfGC7a/zk9bIXmqNAwfhqcmg/TbTGuZyIyfsA8SlWRqMPkZAbm6AIHEjGz6Zklu
 7Sl2w6CGcgDSg1uAdz4cvWqy1/t73ERlPgrBxnDI8VJeVWq+49vcnSjgvgyKsAKCBPC2jrFi
 FrGXUZG/bHA+tZn/sPIiKaIq5aSP9F/RkcKTXPG6buWNDXB+jbxy4F3T+vVLyvWU3n5+fv+a
 LwNne38KvAOgH1Dr5F4T+Rw1as76tbi++1awwBjECmZZlinEOo9cHyP3M0JvaxR3L5J/wCxX
 xvXqNVdPLyIPuLjEUIQe1V5PrjSi6lMl2mA9+kxLWX7+DRzreiOXkhlNhWRjDBQceluO4Q/z
 OZ94MMb5mRTUPbx3gpqWsyMy1mxEw==
IronPort-HdrOrdr: A9a23:TsnFiaEZqYqCwiE/pLqE7seALOsnbusQ8zAXP0AYc3Nom6uj5q
 eTdZUgpGbJYVkqOU3I9ersBEDEewK/yXcX2/h0AV7BZmnbUQKTRekIh7cKgQeQfhEWntQts5
 uIGJIRNDSfNzRHZL7BkWqFL+o=
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54035975"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Anthony PERARD
	<anthony.perard@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "Doug
 Goldstein" <cardoe@cardoe.com>
Subject: [XEN PATCH 1/2] automation: switch GitLab x86 smoke test to use PV 64bit binary
Date: Thu, 30 Sep 2021 17:17:19 +0100
Message-ID: <20210930161720.825098-2-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
In-Reply-To: <20210930161720.825098-1-anthony.perard@citrix.com>
References: <20210930161720.825098-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Anthony PERARD <anthony.perard@gmail.com>

Xen is now built without CONFIG_PV32 by default and thus test jobs
"qemu-smoke-x86-64-gcc" and "qemu-smoke-x86-64-clang" fails because
they are using XTF's "test-pv32pae-example" which is an hello word
32bit PV guest.

As we are looking for whether Xen boot or not with a quick smoke test,
we will use "test-pv64-example" instead, which is a hello word 64bit
PV guest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 automation/scripts/qemu-smoke-x86-64.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
index 09152e3e9ca1..4b176c508dec 100755
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -16,7 +16,7 @@ cd xtf && make -j$(nproc) && cd -
 
 case $variant in
     pvh) k=test-hvm32pae-example extra="dom0-iommu=none dom0=pvh" ;;
-    *)   k=test-pv32pae-example  extra= ;;
+    *)   k=test-pv64-example     extra= ;;
 esac
 
 rm -f smoke.serial
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:18:09 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:18:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200095.354495 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyl7-0001M4-Dt; Thu, 30 Sep 2021 16:18:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200095.354495; Thu, 30 Sep 2021 16:18:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyl7-0001Lx-An; Thu, 30 Sep 2021 16:18:09 +0000
Received: by outflank-mailman (input) for mailman id 200095;
 Thu, 30 Sep 2021 16:18:07 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5o/Z=OU=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVyl5-0001Ka-AV
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:18:07 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id fe0dcf38-2209-11ec-bd5a-12813bfff9fa;
 Thu, 30 Sep 2021 16:18:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe0dcf38-2209-11ec-bd5a-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633018686;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=8LIacdvWaCNVkUc54HHZZzLmortJQrIHn5Qqs0IThkQ=;
  b=dR9+WF+LIKb/iR+MiELWbnkyJP13Dm8xwHXiJYmsVBgLEJWAhUJGguOR
   N7/QubHVmXdDF/RYbRaFMM55QHKlJ9iLBTu8KxZhyCk1DKurPKsByixJ4
   c+AoiJBxE5W1UYmp5tNFFuP7Ac/0/Ul2RiLlu+1lXHUmcmxosEMkOjroA
   M=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: WDieJmFIR/tRTU4DIaM/Qe/Xye0TWdharfQVEmJzCTBgVd+J1lc52f2UvazEVYNjD7iLbtGjfT
 QoAppD0AOwxXl12hdKWVxB/3qL/Xiussw4RJp1JpG+grqoPjyOIUsYRz7EzCxctFzJTW/qMT+4
 r3U4aOG4zTd6VOqmbATJSPVO9DCX+HC1MtWMIOY59l3dIRZY/OOaf9L5CC5fiBLjPh++LGB94S
 sI/XUWWtj4ZFY2UEDWj+N7ZIYI8AEwtpnXsYNlDAXfLutKvuC6XHxczZ2xMB6S6NutWasJitVk
 SKhWGuVU9581drLCTD/LuWDL
X-SBRS: 5.1
X-MesageID: 54455432
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:8x9ryKxywEU4qcxiMFR6t+fywSrEfRIJ4+MujC+fZmUNrF6WrkVRz
 2EYDD/QafrYa2vyeop1bYSw8k8AsMDdnIM1TwI+/iAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrVh2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt8hN0
 9lH8qGMcg4KLLH0yOIeCSVZMAgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVors0lMMnsOpJZonx6xCvVJf0nXYrCU+PB4towMDIY25wSQKuCP
 JBxhTxHXCrsOyBIBHcuS49us/qKv3nxLRNbgQfAzUYwyzeKl1EguFT3C/Lec8aGQsh9lU+Cq
 mXLuW/+B3kyJNGZjDaI7H+oruvOhj/gHpIfEqWi8fxni0HVwXYcYCD6TnPi/6P/0BTnHYsCd
 QpEoULCsJTe6mSzcsLxVVqFnEWNnQwAY4pWELcG6SKkn/+8DxmiOkAISTtIadoDvcAwRCA32
 lLho+4FFQCDo5XPFyvCp+78QSeafHhNdDBeNUfoWCNfu7HeTJcPYgUjpzqJOICylMG9PTjtz
 z2QoCE6it3/ZuZQjP7jrDgrb9+2z6UlrzLZBC2LAwpJDSsjPeZJgrBED3CAtp6sy67DEzG8U
 IAswZT20Qz3JcjleNaxaOsMBqq1wP2OLSfRh1Vid7F4qW/3qiL7LNAMvmAhTKuMDiriUWSyC
 KM0kVkMjKK/wVPwNfMnC25PI55CIVfc+STNCamPM4smjmlZfw6b5iB+DXN8LEi2+HXAZZoXY
 M/BGe71VC5yIf0+kFKeGrdMuZd2l3tW7T6CGvjGI+GPjOP2iIi9EuxebjNjr4kRscu5neki2
 4wEapTWm0kBCr2Wj+u+2dd7EG3m5EMTXfjew/G7vMbfeGKKwUktVK3cx60PYYtgk/gHn+vE5
 CjlCERZ1ED+lTvMLgDTMiJvb7bmXJBeq3MnPHNzYQb0iiZ7OYv/vr0Cc5YXfKU88LAxx/BDU
 PRYKd6LBe5CS2qb9m1FP4X9toFraD+imRmKY3i+eDE6cpM5H17J99bocxHB7i4LCibr58Iyr
 6f5jlHQQIYZRhQkB8HTMar9w1S0tHkbueRzQ0qXfYUDJBSyqNBncnWjgOU2LscALQT46gGbj
 wvGUw0FoeTtopMu9IWbj66zsIr0QfB1GVBXHjeH4O/uZzXa5Geq3aRJTP2MIWLGTGrx9aivO
 bdVwvX7PKFVlVpGqdMhQbNizKZ47Nrzvb5KiA9jGSyTPVisD7phJFiA3NVO6fIRluMI51PuV
 xLd4MReNJWIJNjhQQwYKwcSZ+ie0e0Zx2vJ5vMvLUSmvCJ68dJri6mJ08VgXMCFEIZIDQ==
IronPort-HdrOrdr: A9a23:tbtmKqs80QIxfyYFnj70Ynps7skDetV00zEX/kB9WHVpmszxra
 6TdZMgpHvJYVcqKRcdcL+7WJVoLUmxyXcX2/h1AV7BZniEhILAFugLhuGO/9SJIVyYygc378
 ZdmsZFZ+EYdWIK7/rH3A==
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54455432"
From: Anthony PERARD <anthony.perard@citrix.com>
To: <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Doug Goldstein
	<cardoe@cardoe.com>
Subject: [XEN PATCH 0/2] Fixing gitlab CI tests
Date: Thu, 30 Sep 2021 17:17:18 +0100
Message-ID: <20210930161720.825098-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.gitlab-pv32-test-fix-v1

Fixing tests due to CONFIG_PV32 missing.

debian:stretch container needs updating due to certificate issue, so also
install qemu in the container.

Anthony PERARD (2):
  automation: switch GitLab x86 smoke test to use PV 64bit binary
  automation: Add qemu to debian:stretch container for smoke test

 automation/build/debian/stretch.dockerfile |  2 ++
 automation/scripts/qemu-smoke-x86-64.sh    | 10 ++++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:20:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:20:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200104.354506 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVynp-0002pJ-S2; Thu, 30 Sep 2021 16:20:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200104.354506; Thu, 30 Sep 2021 16:20:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVynp-0002pC-Op; Thu, 30 Sep 2021 16:20:57 +0000
Received: by outflank-mailman (input) for mailman id 200104;
 Thu, 30 Sep 2021 16:20:56 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Bos=OU=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVyno-0002p6-Qb
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:20:56 +0000
Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 5e3e2316-4a53-4201-b5f5-638a3bf8789b;
 Thu, 30 Sep 2021 16:20:55 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e3e2316-4a53-4201-b5f5-638a3bf8789b
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633018855;
  h=subject:to:cc:references:from:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=/67HFbhx/Yu00e4CdcAnZrFPeyBkC4QhOFQgInI4wUU=;
  b=dlQwrE/ilQiCTAL+HJdmcrIFH+H4473K9pVoaJquEzJGkyj2Q6egONRv
   V8fWcusKHiGYAKpZHALmN5Pi4dwFtS8OTrxVWJR3IlKY2xWcNcns7iddp
   GK30huyfEgk3b+nyR4HPyV6Vl7z+brMp2MXmZxb7PFGD4otL+RWcmQxh+
   k=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: JJ28ox8VsgiJgeLfna5WGQ34U3GklxUlNAP2uLPw4cg2exC49Sy8cAARp5JltEenjTxSzTzU6B
 xqqGDGfn5LIa4A9sEadYGk/HXDFxN71T9qTTifet3sinPiSxc652LRovln4+kyGU9Ve35Nxna1
 u3d2TmFuP0MtZKDNH2WjT63AoGv/3tutdKzP+aIDfMQ5SOz5nHhUZExeAGhq1jXlrDK7GqIw/L
 QAgldSLBTEuznmZyFA69nf01xOSIMsrpnVI34WgrpuErOKxLfTAVhLb7JArOUrkvM7Cm/Gql2Z
 GwG+d3h2T4EcTJXiswA+7P9A
X-SBRS: 5.1
X-MesageID: 54455875
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:eCs2uKuoo/rTSrfkoWNjiDSkK+fnVKNZMUV32f8akzHdYApBsoF/q
 tZmKT3VM6zfZzH8L9siPI+wpE4O7JWEzd9nHFRo+CkxFnkR+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZQP0VOZigHtIQMsadUsxKbVIiGHpJZS5LwbZj29Y52oDhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Nplu4eXSRYnYpP1w9tacSBgTjpUILR7weqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DFYUToHx/ixreCu4rW8vrSKTW/95Imjw3g6iiGN6DO
 5ZENGowMXwsZTV+OQ9LIYofnNymn1/2NANRomnM+5Eetj27IAtZj+G2bYu9lsaxbdVYmAOUq
 3zL+0z9AwoGL5qPxDyd6HWui+TT2yThV+o6Hbuk9vlskRuLy3YaEhE+Wl6yoP3/gUm7M/pUI
 lYY/ScGpqEo+EumCNL6WnWFTGWs50BGHYAKSqtjtV/LmvG8Dxul6nYsQCVmaM0o6PEPHgNt9
 GaxwsjDGCVSiejAIZ6CzYu8oTS3MCkTCGYNYy4YUAcIi+XeTJEPYgHnFIg6T//s5jHhMXSgm
 2nV8HhWa6A715Zj6kmtwbzQb9tATLDnSRQpriHeQ26o9A9wYIPNi2eAsgOAtacowGp0VDC8U
 Jk4dyq2sL1m4XKlznXlrAAx8FeBvK3t3Nr02wMHInXZ327xk0NPhKgJiN2EGG9nM9wfZRjia
 1LJtAVa6fd7ZSXxN/4rO9PhV5p7ksAM8OgJsNiPNbKihbArKWe6ENxGPxbMjwgBbmB2+U3AB
 XtrWZn1VitLYUiW5DG3W/0cwdcWKtMWngvuqWTA503/i9K2PSfNIZ9caQfmRr1pvcus/VSOm
 /4CZpTi9vmqeLCnCsUh2dVIdg5iwLlSLc2elvG7gcbZeVc5RDF9VaWBqV7jEqQ895loei7z1
 ijVcmdTyUblhG2BLgOPa3t5b6joU4o5pnU+VRHA9370s5T6SYrwvqoZabUterwrqL5qwfJuF
 qFXcMScGPVfDD/A/m1FP5X6qYVjcjWthB6PYHX5MGRuIcY4Slyb4MLgcyvu6DIKUni9u/whr
 uDyzQjcW5cCGVhvVZ6EdPK1wlqtlnEBg+YuDVDQK9xedRy0oohnIiD8lNEtJMQIJUmRzzeWz
 V/OUxwZufPMs8k+99yQ3fKIqIKgEu1fGEtGHjaEsebqZHeCpmf6mN1OSueFezzZRVjYwqT6a
 LUH1ez4Pd0GgE1O79h2HYF0wP9s/NDovbJbkFhpRS2Zc1SxB7p8CXCaxs0T5LZVz7pUtAbqC
 EKC/t5WZeeANM//SQNDIQMkaqKI1O0OmymU5vMweR2o6Chy9buBcENTIxjT13ANcOoraNsok
 bU7pcobyw2jkR57YN+Jgxdd+3mIMnFdAb4ssYsXAdOzhwcmor2YjUcw1sMiDEmzVuhx
IronPort-HdrOrdr: A9a23:xCym+aoZENoMoQCHeN11tisaV5oveYIsimQD101hICG9Ffbo8P
 xG/c5rsSMc7Qx7ZJhOo7y90cW7Lk80lqQU3WByB9mftWDd0QPDQb2KhrGC/xTQXwH46+5Bxe
 NBXsFFebjN5IFB/KXHCd+DYrQd/OU=
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54455875"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fc1tTxLkB1Qa6/6OUfdJAV43JrLUbR3iPMcr+eS3zXxMYFJGrK6o/3EQATV+5iTSNLaS9mCBJPg8LB/yt9L8hZO0MPckjEOEd1W8mO4o02EEN2jnN8Pl5wKiEBErY/gIvFPSKkUh4nYXBK4iXk9U/5v7RFuhfikdWUyOLBRGx9m+jl9fXyeYh72IPIJvnoC9GAjnH8c8E6fGPjYV9vBKxcjwgQ9faXy31LpHLYQeyACI4nxeDrTW38Ypo+riZzboqr3bUlaACZ4dxPpX5YvNWLpGlKnXFiXM+62rLbcea9I8+Xmmcd9fnozvMQ8xHL61H9X1dbkIhpjnzJG9pFtwtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=CqOOlg3EQfoiMvUZHKxb4QSuuT+aojwlTuDKISbUfpw=;
 b=VGL4auWIPa1XfwbzqvE2QZVWgGrKZ/Vfze7MK8N6RSHuMLtT1yqLdFNPXdSlMevdrRb/yP0k/8o2eYBEmxlZFdOt+QBGzLSVhWcGpfgKcjyv3/K6zyniXd5yfPHOQUt5ruNjnSfUiQzgOn4A8bWJ+v8pysaEJMtYyHsPzC/kMYn8KXvbzZSYQIqAnuzO04OGyqnoTdsHT3KzbGd1YdFlnI+fRx0WHFeOGBYwdtxN0Q+GGh6bg+mUeAkwKHGlfdvNMc7TlpZYo0Ns8UeJxum6U1yGwFC9M1sLJjTHQiUgp5Ov/Sf0l8eHGhv59ogIwoiNFxIHMWPcbesQCwu0S7t5bA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=CqOOlg3EQfoiMvUZHKxb4QSuuT+aojwlTuDKISbUfpw=;
 b=mQWSeW1yCF6ITRwWeKh4Ra17F2oATg3XonGCZESy2Cf4SK6V04VZPKVPohk69GeGtFPHMQhkzmO7S1CnbQICgHTDN6MSZvMZlq95RfaxgyHJXPP4+FqL3frT4ftWZC+NDSy2FySAARFRP+1++615S/0CrdJqw/sdCzTg7xswM7c=
Subject: Re: [XEN PATCH 1/2] automation: switch GitLab x86 smoke test to use
 PV 64bit binary
To: Anthony PERARD <anthony.perard@citrix.com>,
	<xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@gmail.com>, Doug Goldstein
	<cardoe@cardoe.com>
References: <20210930161720.825098-1-anthony.perard@citrix.com>
 <20210930161720.825098-2-anthony.perard@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Message-ID: <a01d951f-1078-2048-7f86-ed834bdcf76b@citrix.com>
Date: Thu, 30 Sep 2021 17:20:31 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930161720.825098-2-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-ClientProxiedBy: LO4P123CA0394.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:18f::21) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 3773e175-381d-4cea-c6f9-08d9842e3cc2
X-MS-TrafficTypeDiagnostic: BYAPR03MB4166:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR03MB4166181A300B2E2C3E1399DABAAA9@BYAPR03MB4166.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:352;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: shDZwz2nAIVJ2EaomsD8sxpNc1nOXhi+1AcLATuLt24da9UpAcsYbD2+43AdCf+uzM5dEiKLKX6rOuI4DpgXSWpzyAdxBSuxC38vbs+dajeECbbkI2MbELIATaRzuX6dpbmPeeA58FiJnYKqZ5ngmADpB0NI5VVzoZJKTTSCT/ScUmLfT3PTmn23UZ5iYpiOpCatAEg78H85XqjYgOSPl+s5RSbJEUiYLJvC3nJrUtNKQFMghMJVFcrEgo+HzuHkBYm3u/Y+1VJ+FSHxUXPfZQZgeadElsbg8I/HTHBquu0p8pwexrOJSFQv/CRQ5iBL21rTqab83d1oQjdQhY+l2OF38j8OF9BNsVuKd24Ar4Q9Y26rUKrcPc3OyXoS55fWB9pgjOAJzUD/UnCFgQ4qFFy7IikfRObJDm6s774FkpDEd0KrDt1yla9A1hnB8xkaDWulspYaPDKEMdUWALazEZweF2GL//niZy5RMR6JyeC0VjsVgNxT0J4TqL/dJa/xyKr8s/prgV63dIxGHZ8SMGLJQ2xCZQn1VtRKZTtJnfc3m8gJUEwB0sDrPmd2nZi4zi0daUjwnWbjsUKfXeKm8bWt2QwKhqLnd/FMaHxYvo3QM0zKAhmI83VDKhQFn6wd1wJXlJeaWTZ6UD908alxL2OY2VTMho0aUBHvLCYNmDqfZTxUJ06OTIejTZsw+gkxtMbk+bsuyLwEpazBuM8pAuqmc2DFAH1kJLcQew+ph+M=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(36756003)(31686004)(5660300002)(16576012)(8936002)(316002)(8676002)(186003)(83380400001)(4326008)(6666004)(54906003)(31696002)(508600001)(956004)(66946007)(38100700002)(55236004)(2616005)(86362001)(6486002)(53546011)(66476007)(66556008)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U0FYM0NMMzNwZGp5d1RPa0hQSjRrMytJckcwOTNmYmxFeHQyUmtYMnVVWHNp?=
 =?utf-8?B?ckhBQmhmYThrN1hBZGdOc0hxM0xROG1lemhLQzFBOVZqNVhFMGtVRjlXejU5?=
 =?utf-8?B?QWFlTmVwVnRHOTFBNGZrc1lPbDAwcjlPM1NFUUFVZzNpd2EycUFscVV4Uk1B?=
 =?utf-8?B?Z1BGS25KcGxrYUcrSVB3WUF0RWtqbnhOVG9tMFVLcTg3Y3JtWmhtZHdwYjYx?=
 =?utf-8?B?M29OdFYxczZFeFBKN3dIeFlnVjJiQnRhbGVYb0tKOU5ldUR0dzVwdk44Tm1t?=
 =?utf-8?B?V0pnR1dSUTlSVHhLb3ptNkpRaks5Mm96dThMdVRPOVVwbzljdzBwRElVQVZj?=
 =?utf-8?B?bEtGd21tWUUyWVpWbjlKVjExZVE2RHNvWmpza044SmRmVE9XS2VMdE5MbG9a?=
 =?utf-8?B?YUVyaDc4Rm5KU3UyblN0OTV1cXllbzBJQ29HQXRndXo1ZUNjbWl1eWNlMXRV?=
 =?utf-8?B?VmNGRFFMRHc2TDU2eS9ubjJlSmZtM2h2Z0swRTZZYjNhTG1EQlpHY1dKVURE?=
 =?utf-8?B?aUhIQ2VROHZDVTFqVUo3QjZ3S0lqRmJXNWhNNFN0aTByVmZOdFR3ZDRWSVlk?=
 =?utf-8?B?K0tZMVcwZHY1Q25mQWsrM3FlL1R6cVNYUUY1ajlKYmVLTVhNU0UvY2RDQVRk?=
 =?utf-8?B?TEVHSUtOdHVIaFo3TStibkVYb2l3dUQ1eUk2Zk9xR0h3K1gwY2ViK1RaYXJw?=
 =?utf-8?B?TU1Hd21QamM0bWhtZ3AyM0loQ3pOSkpNMEN2V0NXM0ErUWJWc3hrYW4wd0w2?=
 =?utf-8?B?eFJncDJEKzRrc0NheVBSMEovOGR5VWhUSGUxdlBKd29mSU5hN3BIcmQzTUVQ?=
 =?utf-8?B?b3NZdytHS3BOdmM3TE5RSGYrVmpxSTBEa3BGMmREN2djYmJUMk14bnZ6ZytW?=
 =?utf-8?B?ZEs1c250TW1ObUZlQWpSaC9ORXlHa2lJREh3eXV6TitNM2hFT0ZQcHBkVkJU?=
 =?utf-8?B?UDdaTDZ0Ky9rQ3hMT0RkRklZZ2dXM2YzL0NSYWd0TUxZTFkxRUFKK00vZk1u?=
 =?utf-8?B?ODBab1c0dXgwQnBad2VCVU9YVXplZlkzMTVyMmp5U1VWYStjVkFjSThYQUhz?=
 =?utf-8?B?UzBzZTAvMldxSnkrYlZCeXBPUjQzMkZudmFCckZsOXRET1FxSWMrTWpDS3hX?=
 =?utf-8?B?cFZmNDRhQ0hVdk91cnJOKzhEZDJlZTFINXdmMWs5Z1d2RFF4WTNsZ3pObHZF?=
 =?utf-8?B?N2FkS0hSRkQ4bGQ3akdtR0src00ybE5XNGphUnFFYnFQbHM2a1NmV2ljbFVF?=
 =?utf-8?B?QWN0bFJHSUJ3UUdXZE02b0k5dDJsY0tQUDZwMFlIZ0dmK3hFUTNraml3OUor?=
 =?utf-8?B?aGtvWmVIMjM0WEZBTUF3MkZQOGZ5a2c2a3Nldm04OXE2S3JNOVBnTk1KOHNv?=
 =?utf-8?B?NW1IcnU5U0w2d2JHTTYwMEF4b1llNDhraklQMUJiOER4YWYyL1FKbFZrL3U0?=
 =?utf-8?B?NWFxbTBIYU5jZmllL0JBeCs5QU9HNEowTGNpM2FNbERUcWJ6NWRNaERzUUhn?=
 =?utf-8?B?MFNlVUVDeTNaakcyRG9lQWlqbjBpMHl4WXppejFNNEY1d3pyd052RGJjT0tC?=
 =?utf-8?B?WjZ1NWl0TFh6WnNWVzMrNHNSQm1kN0hlUnZNRmE4K1IyRnNsV3dyVU8zMG5O?=
 =?utf-8?B?RHJHQVFLaVhSQ1JSREJveml2NnlOalc5U3hrRk5MN2tXT1A0Y2dLZWFzVklP?=
 =?utf-8?B?cXZZU2NuN2Roek9DMkdlNzhzcklNVThTOWovTlZPMyt6Mi9sd3BCTG5tUitJ?=
 =?utf-8?Q?Nzkkf44rVE0OCbAJlVcKtaLHuTpWTrq9CweXXUb?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3773e175-381d-4cea-c6f9-08d9842e3cc2
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 16:20:37.4974
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LD7deeQLIv1ESw8yheeVpu2RDC4OWNm6/eR09J2bPj18J2awYEk3xKgJ+gRHnOrGvCxglsvioru7yauONDKXTUvY6uxrqxnlDvnbrTWnoOQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4166
X-OriginatorOrg: citrix.com

On 30/09/2021 17:17, Anthony PERARD wrote:
> From: Anthony PERARD <anthony.perard@gmail.com>
>
> Xen is now built without CONFIG_PV32 by default and thus test jobs
> "qemu-smoke-x86-64-gcc" and "qemu-smoke-x86-64-clang" fails because
> they are using XTF's "test-pv32pae-example" which is an hello word
> 32bit PV guest.
>
> As we are looking for whether Xen boot or not with a quick smoke test,
> we will use "test-pv64-example" instead, which is a hello word 64bit
> PV guest.
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

> ---
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  automation/scripts/qemu-smoke-x86-64.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> index 09152e3e9ca1..4b176c508dec 100755
> --- a/automation/scripts/qemu-smoke-x86-64.sh
> +++ b/automation/scripts/qemu-smoke-x86-64.sh
> @@ -16,7 +16,7 @@ cd xtf && make -j$(nproc) && cd -
>  
>  case $variant in
>      pvh) k=test-hvm32pae-example extra="dom0-iommu=none dom0=pvh" ;;

TBH, I'd be tempted to change to hvm64 here too for consistency.  I can
fix on commit if you're happy.

~Andrew

> -    *)   k=test-pv32pae-example  extra= ;;
> +    *)   k=test-pv64-example     extra= ;;
>  esac
>  
>  rm -f smoke.serial



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:30:29 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:30:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200114.354516 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyww-0004TS-Sj; Thu, 30 Sep 2021 16:30:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200114.354516; Thu, 30 Sep 2021 16:30:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVyww-0004TL-Pp; Thu, 30 Sep 2021 16:30:22 +0000
Received: by outflank-mailman (input) for mailman id 200114;
 Thu, 30 Sep 2021 16:30:21 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5o/Z=OU=citrix.com=anthony.perard@srs-us1.protection.inumbo.net>)
 id 1mVywv-0004TF-7v
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:30:21 +0000
Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b3c186ac-220b-11ec-bd5c-12813bfff9fa;
 Thu, 30 Sep 2021 16:30:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3c186ac-220b-11ec-bd5c-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633019420;
  h=date:from:to:cc:subject:message-id:references:
   mime-version:content-transfer-encoding:in-reply-to;
  bh=KEZxX95J3dB7sfFKTORKgeH8btgqGBbEZFfrMGcday4=;
  b=Ay+w3SfF8vqSL482ATTNm+FUoVV++vLtClme+as8yKRsZYi4qy8j+wib
   XrWKkNxi7JWt0z8CyITWEvxJB6oAL3NGj/tUPY2LehDdmOp1ypbC0KJO8
   j3foG8JbkNQ2xkUd/8z20AUrwcxxeJwN8z5lOgZXmZDx8yGAIAgph+6Or
   0=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
IronPort-SDR: aP2tL6hpJ3EfStr6v1Zz5pEEPzZikmRkF3xCpfzbHLqphqDiHBIZhYm5m46hBDza6BUfVQgpUs
 wANzvZT55yXtIZwILprL2GUBmESFf+Tp1BG30Xg8Nj78rUoG9+FjIDaDNmFmuqiAKl/AXdhuUA
 tCMWIv/b8orpnYXUq7YjL9mimw40mubxgRiNkDbBLKweOymgDHDReB3VEUkM1Z5r0tIY+TDZTF
 /6NvxRgnG19ell/iuT2j1utJ9uIA+KyVP4T8gi7BRTR/zoeYWEgusDoukAuAMrwT/YjaArSk4W
 7+fXgt/Xj+PPY/jfPLS83d+p
X-SBRS: 5.1
X-MesageID: 54029962
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QuN3Ua5wIf3SIyipYkjjAAxRtOTAchMFZxGqfqrLsTDasY5as4F+v
 jMaCDqCb/2KZTb3LYgnbYS+pBlV7J7QzdE3TVBsqX0yHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuV3zyIQUBUjclkfJKlYAL/En03FVIMpBsJ00o5wrZo2dYw27BVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z7
 Mlrvoy+dCcSMbT1gdRMCkBKHSA5FPgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALJc/3PIQZqzd4wCvQF/oOSpHfWaTao9Rf2V/cg+gTQa6EN
 ptHM1KDajzOQwRlKFlMEqk9v9z5vFXjdwMD9k2a8P9fD2/7k1UqjemF3MDuUteDX8lSk26Tr
 3jK+Gm/CRYfXPSWzyaF9XS9wPfChyrhVKodEbS58rhhh1j77nweDlgaWEW2pdG9i1WiQJRPJ
 koM4C0soKMuskuxQbHAswaQ+SDe+ERGApwJTrN8uFrlJrfoDxixInQiRS5PVfwdu9YPXAx7j
 wabkdKuLGk62FGKck61+rCRpDK0HCEaK24eeCMJJTc4D8nfTJIb1EyWEYc6eEKhppilQWivn
 2DiQD0W3e17sCId60msEbkraRqCoYLVBio8+wnaRG6s6g4RiGWNPNfzsQmzARqtNu+kori9U
 Josx5X2AAMmV8jleMmxrAMlR+rBCxGtamC0vLKXN8N9nwlBAlb6FWyq3BlwJV1yLuEPciLzb
 UnYtGt5vcEIZir2Nf4nP9zgVKzGKJQM8/y+DZg4ifIUP/BMmPKvpnkyNSZ8IUi3+KTTrU3PE
 cjCKpv9ZZrrIa9m0CC3V48gPUwDnUgDKZfobcmjlXyPiOPGDFbMEOttGAbeP4gRsfLfyC2Io
 ok3Cid/40gGOAEISnKMqtB7wJFjBSVTOK0aXOQNLbPcflo6RjtxYxITqJt4E7FYc21uvr+g1
 hmAtoVwkzITXFXLdleHbG5NcrTqUcotpH43J3V0b12px2IiccCk66JGL8k7erwu9epCy/9oT
 qZaJ5XcU6oXEjmXqS4AaZTdrZB5cEj5jwy5ICf4MiM0eIRtRlKV94a8LBfv7iQHEgG+qdA6/
 ++7zgreTJdaH1ZiAc/aZeiB1VS0uXRByut+U1GReotYeVn28ZgsICv016dlL8YJIBTF5z2by
 wfJXktI+biT+9c4qYCbi7qFooGlF/pFMnBbR2SLv6yrMST6/3a4xdMSWui/Yj2ABnj//7+vZ
 LsJwqikYuEHhltDr6F1D61vkfAl/9LqqrJXklZkEXHMYwj5A79sOCDbj8xGt6kLzb5FowqmH
 EmI/4ACa7mOPcrkFn8XJRYkMbvfha1FxGGK4KRnOlj+6Q924KGDABdbMBS7gSBAKKd4bdE+y
 uA7tc9KswGyh3LG6DpdYvy4I4hUEkE9bg==
IronPort-HdrOrdr: A9a23:7FGXJq7UCRqEIlzOBgPXwM7XdLJyesId70hD6qhwISY6TiW9rb
 HLoB19726StN9xYgBEpTnuAsS9qB/nmaKdpLNhW4tKPzOW2ldATrsD0WKK+VSJcEfDH6xmpM
 RdmsBFeaTN5DNB7PoSjjPWL+od
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54029962"
Date: Thu, 30 Sep 2021 17:29:54 +0100
From: Anthony PERARD <anthony.perard@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: <xen-devel@lists.xenproject.org>, Anthony PERARD
	<anthony.perard@gmail.com>, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [XEN PATCH 1/2] automation: switch GitLab x86 smoke test to use
 PV 64bit binary
Message-ID: <YVXmArrCuK8oYrHr@perard>
References: <20210930161720.825098-1-anthony.perard@citrix.com>
 <20210930161720.825098-2-anthony.perard@citrix.com>
 <a01d951f-1078-2048-7f86-ed834bdcf76b@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a01d951f-1078-2048-7f86-ed834bdcf76b@citrix.com>

On Thu, Sep 30, 2021 at 05:20:31PM +0100, Andrew Cooper wrote:
> On 30/09/2021 17:17, Anthony PERARD wrote:
> > From: Anthony PERARD <anthony.perard@gmail.com>
> >
> > Xen is now built without CONFIG_PV32 by default and thus test jobs
> > "qemu-smoke-x86-64-gcc" and "qemu-smoke-x86-64-clang" fails because
> > they are using XTF's "test-pv32pae-example" which is an hello word
> > 32bit PV guest.
> >
> > As we are looking for whether Xen boot or not with a quick smoke test,
> > we will use "test-pv64-example" instead, which is a hello word 64bit
> > PV guest.
> >
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> > ---
> > Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> > ---
> >  automation/scripts/qemu-smoke-x86-64.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> > index 09152e3e9ca1..4b176c508dec 100755
> > --- a/automation/scripts/qemu-smoke-x86-64.sh
> > +++ b/automation/scripts/qemu-smoke-x86-64.sh
> > @@ -16,7 +16,7 @@ cd xtf && make -j$(nproc) && cd -
> >  
> >  case $variant in
> >      pvh) k=test-hvm32pae-example extra="dom0-iommu=none dom0=pvh" ;;
> 
> TBH, I'd be tempted to change to hvm64 here too for consistency. I can
> fix on commit if you're happy.

Sound good to me, I haven't tested it, but I guess it should work.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 16:57:55 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 16:57:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200123.354528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzNV-0007Jt-3F; Thu, 30 Sep 2021 16:57:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200123.354528; Thu, 30 Sep 2021 16:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzNV-0007Jm-0I; Thu, 30 Sep 2021 16:57:49 +0000
Received: by outflank-mailman (input) for mailman id 200123;
 Thu, 30 Sep 2021 16:57:47 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y4Rr=OU=epam.com=prvs=0907a041b9=oleksandr_andrushchenko@srs-us1.protection.inumbo.net>)
 id 1mVzNT-0007Jg-E5
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 16:57:47 +0000
Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 600dd42f-f30a-410c-ba18-8189be2f1fbb;
 Thu, 30 Sep 2021 16:57:46 +0000 (UTC)
Received: from pps.filterd (m0174682.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18UGt4aW026779; 
 Thu, 30 Sep 2021 16:57:43 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169])
 by mx0b-0039f301.pphosted.com with ESMTP id 3bdh63r0fs-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 30 Sep 2021 16:57:42 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com (2603:10a6:20b:153::17)
 by AM9PR03MB6851.eurprd03.prod.outlook.com (2603:10a6:20b:2d9::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.17; Thu, 30 Sep
 2021 16:57:39 +0000
Received: from AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd]) by AM0PR03MB6324.eurprd03.prod.outlook.com
 ([fe80::e966:6290:5736:b7bd%7]) with mapi id 15.20.4566.015; Thu, 30 Sep 2021
 16:57:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 600dd42f-f30a-410c-ba18-8189be2f1fbb
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LqFP0/pJp8jOQv9eMC8HRaXrZ6AtMaICPkm7Vxw/xHxWXzJywZZO+3iUMnKWJL1WumZy1FK0Y2/cv7UG9y+/JJG9Oc/+ew1ylPve3xJ5oOgNzONBfwVhTDcmzUFlmub+8YwFcEuNTOG0IsZCNFhOqO+5rYNlhqQjZoOJM/pEy810Dyc2y64U23JAGTphpKInafoDER7YJQJmoN2voPmPvw2GsKp2Cx9J0mxzNkKFnl8PVIekTvLBBUrWUpAAZLJsEPoDp25I03/GSZljsGlUgGi+U+DkSzUi+UgZLVKkBZWCOiK2Ip7Zp5iWuCSlvPMRbOICzqrMKMU7xmEIK1Ek1Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=uDoYUOXlLzB4VCICsvsP6JEGGAbdMCw2O3Y2VNCsizk=;
 b=FJeWVhgTfb0OTP5PU3xlBGtAkU7kzoTmiC1prjH1n5AniGTzRks3Vh1ydDXIlH+Z1VZrZhOZ6EoDN6ZXCN42tOtmlSz9i8wMAnYAkYvyOP0QSCRzUJK/hf7EeSI+syVsEqXht5pYc8Dy2JZLyjPDM0o/Hq5r0tY8ti0TNZ8bG0uH+vThVO267tJnEN0tymCf3vr3RfIuYWBS6DGo0qAcjBn8eErK/9ZWs8RhdSjC84qIq37lZl8FmU/gzL73ZENMajthTWtvVpslynAf5/wXy/9k2BbgSwilZJreFLnwDt2xY7xXWArDVfK/56UvACsVyOvMD5vn0J41PBl1r4bBdg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uDoYUOXlLzB4VCICsvsP6JEGGAbdMCw2O3Y2VNCsizk=;
 b=he8Ti7QNhvH2mIeyGGQzMaYppaf5Ex+f1nQ5v5SVEE7cCVgi1D27RRTasvJfYbzX4rcIoacSKvX++L9y5VLT6hZHFsTQe6mKUoLVyBxECDLeJGeGAkEdsBPBsyqHz4VM2qemSrPXxXqu/cXxuJzJD+Ll6zlpXtbycD9O9zTEM+lQ+8n2DULgD4Zct9HzoJa7UkKymWlE5OBCE9DBdqJCTvuAWmFDqmxbEyG+WvYV6g+aeDgqm5P5zwkbTWqzcZ26eH598Otm9jev+GAO4eROKzE6xP9g3hzh7RNgZXDHzEeBj4D08LfF9bWPiHfAKo18XFPH17Zsiv8J2iiviSNyeg==
From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
To: Jan Beulich <jbeulich@suse.com>
CC: "julien@xen.org" <julien@xen.org>,
        "sstabellini@kernel.org"
	<sstabellini@kernel.org>,
        Oleksandr Tyshchenko
	<Oleksandr_Tyshchenko@epam.com>,
        Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>,
        Artem Mygaiev <Artem_Mygaiev@epam.com>,
        "roger.pau@citrix.com" <roger.pau@citrix.com>,
        Oleksandr Andrushchenko
	<andr2000@gmail.com>,
        Bertrand Marquis <bertrand.marquis@arm.com>,
        Rahul
 Singh <rahul.singh@arm.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Topic: [PATCH v3 11/11] xen/arm: Translate virtual PCI bus topology for
 guests
Thread-Index: AQHXtdAlBwDHpaYUBUq6q0PlsxoTp6u8Ra2AgACHPgA=
Date: Thu, 30 Sep 2021 16:57:39 +0000
Message-ID: <68a608f5-4159-5734-4c08-5a70ae9d3ff4@epam.com>
References: <20210930075223.860329-1-andr2000@gmail.com>
 <20210930075223.860329-12-andr2000@gmail.com>
 <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
In-Reply-To: <2bbf1de4-75cf-4f67-a96b-52762332d220@suse.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: suse.com; dkim=none (message not signed)
 header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ac2e953a-c141-4923-0a1c-08d98433694a
x-ms-traffictypediagnostic: AM9PR03MB6851:
x-ms-exchange-transport-forked: True
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-microsoft-antispam-prvs: 
 <AM9PR03MB685123FF346B9FA271B909F9E7AA9@AM9PR03MB6851.eurprd03.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 1/H9gWDTrFo6Kwb50QLUmQXg96FKmhlkKYbm/e9fv0LQZXyOZxbWkyI3Pp3KuKEHOL9CgQO/Zs/3NNhN7md667jXsdsA4konJqN0f3hOPy3duQPWRiizjU6oZqPNriRHbaQ/mkaPBEhYqZftdYY7wxqi9x5dsnII/cnoGrbUobqqKwXpTOp3Z3DlbYIzvGDOAqG7kgIIi8e8URZv5YHwyZwDyjTLxje2mHttsZwYNofx8O82l1brL/1h8L8f+fYlYdamLLI6Zzsj1JFwa7iwr85QWvh0dut3bGFMp+vrCaNtx0PM+SSuUBXkbs4Uj5kAiwvh6oNzijlPKeZiiV3TkCYN92amgVXXsv69cjflSQgbdbcp/UzC+Z1inrDOiYkgeoyk5kMVsTozdOpL9O2Cj/sW+X06w9QHRvFA7VHXI37/Im0npQZFQdeDYXt3Y5oakVTpCKwJP6ArKMoirvM8XsDgs0RTu20URwmiLhc3CvInRYRatqc2ZQBxm5aMTl7GZcaJ7oHoqe6bk0F+fIOtPkuQ0aGgXeB/yiAJh5WbgpFXsDf6r9jb8Oc88lvioQoFj63CiYNmoaTVSUgUiHNcDIbukCAjFtcrsvsb3C86PZLdJUHIuSM+K84Dwvh4hcV/BvpP6qv4EkUxxrTkLPsXLMf+lFkrM5HEBiI3FHUC6wwuYhzwGZybARidW52jTtWHJz7xHHgTtZvbIA2V6wlI62rQwSIOkDtjcb8WnwpyErbbx4PHxHnBnG1XB3WdDDN2
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB6324.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(26005)(186003)(66446008)(38070700005)(5660300002)(31696002)(6512007)(86362001)(31686004)(66476007)(71200400001)(64756008)(66556008)(6486002)(66946007)(91956017)(76116006)(2616005)(83380400001)(8676002)(122000001)(38100700002)(55236004)(6916009)(8936002)(6506007)(508600001)(4326008)(54906003)(316002)(36756003)(2906002)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?Y01ySHQwUnZJLzdMYiswN1NIWFhpK05sdkR5dUlLWTVBUXdtNWFLenhKWnhM?=
 =?utf-8?B?S3N0NWwycThETlpKVkJTVm02MkNmRnVuVHRVYm5YM0ZEV3AwZ05XSlNnUi9T?=
 =?utf-8?B?OGwvWklVemhZbzIza2JXbUNHMTdVOW83QnpJWjh1bHFVYVlDQWU3RlRxRDNu?=
 =?utf-8?B?M01QbHhLZUx4Y1dkZ1NITXgvNThteVpRMjgrUHVNRzZKYVArSkRCRnJIN3ha?=
 =?utf-8?B?YU5FSzdpcXcvMDFKYUNHNVVOV0VqTlJsVUhyMmZJOHlsK1cyQ2F0VDFBdnY0?=
 =?utf-8?B?V2E4RFNkbnk1cUxybTFHY09GNjd6TkZDaFdRbHNGMWJnOXk3eCtvZUd0azZ0?=
 =?utf-8?B?QXJZT3F4VnZ5ajBhZDFpNktPTEFsbTRwWmJFNnhZV2hjMVliYm9idVhlNWF4?=
 =?utf-8?B?RjVEN3JNanZpTjlqMFVhZXJPWVhpdWxOdXVEVUdiVmR2c0gwQVFEWUVzYkhi?=
 =?utf-8?B?UEd2dkxqQTV3VnR5RnJOYTZ5bWF1WHlXUlN3dlErSFRCZnZqeWpXQ0duWGox?=
 =?utf-8?B?UUtXKzVOMTlsZjFpcmFzMDVuWTRKTGhqTHVsNDRKSXNZVGJMbHIwdnNhYUNF?=
 =?utf-8?B?L2JkdWEzRk1QK0ZVRU1EWnh0eW1EWkZPVnVrOGVNdW9vQzlJRjlIQ3ptb0xm?=
 =?utf-8?B?YXpGSU4zYkdoOWVlSXJ5emswZTIvLzdLZzVuOWJqT1ZmOFhIT0ptck5lVXA3?=
 =?utf-8?B?amppMmNLTS9KRW1zdHNzMklXMmt5SHNjSTl5aVBIaklUNTBNbkR0T3J0R1Jt?=
 =?utf-8?B?ckgxdVVpU29ZOExMQ2VCMFZ4cVl4dlpTM29pc3ljaS90UDBxMktCQWdxUk80?=
 =?utf-8?B?OXk2RXZtRTN4RlB4dzJ2YzNXeHpweERtMGRCMHpRd3RpMnU4Wm1rOHJwTXpv?=
 =?utf-8?B?SGFQK0lVK0s2amRjc3VuYVdKYlBSQjhzUzJLYlA3V2VIdEVRWXMyV00xcGY3?=
 =?utf-8?B?dVVuSDVBN1l4S0RzWmhJeTVrMFgzWWs0YmtBZzVHeE5mckJ4R1RPZ291cWQ5?=
 =?utf-8?B?YXFuME5YNTY4TlpTZ1Z1cXRLZGp0UHR2N2YxR0M3NHlNZXpQbUhCcittVTZR?=
 =?utf-8?B?VkdHNWVJU1NYaTRkWXdxdGllRXlXc2tsVVdNTEkzK3JzRnFNcDFmaW9Fb0ZO?=
 =?utf-8?B?VUlwbURINmxqSGUxNFd3U2NZQmMzRnRobHNwUmdZZSs4ZEROZVVtZXlUWCtT?=
 =?utf-8?B?OTVFN2Zqb1dKU0VXNXF2TUZzeFZJam5hZU1GeVYwdUlNcXRyM1FjZkxhOVhn?=
 =?utf-8?B?RGl5Q29JZDAyeW83SmtLVjIra2c4N09KeTlhbWpQVlpTKzk1YzRTQ2xlQXpZ?=
 =?utf-8?B?cEdGa3VoM0RxdUlTRkdTMUJoSWcxenJDdmhIZW9HaEZVdSs5M3hDQ1RIMWph?=
 =?utf-8?B?UTkwdjFEUUtOOU9rYkxKOWd6bFlIcHg0ZjVnYUs4dldnTXhINWQ0aEVFSnVB?=
 =?utf-8?B?SjlLRHZqaVhuS2x1Z1hMcjlhQXl6NzdaR2tQMzNKTTJVRUc5MnY4VmxINkhB?=
 =?utf-8?B?T3FoZVdoclozY1ZWcktWckZES0xUMGhCUkFXOW9hRnRWWWkvalRQWmcxd3Fx?=
 =?utf-8?B?WUpka3pHMnFTL0s3WlpLbndyUDJaY0s4Tmp2MXp2ZENkTUp5cmFvcGRRK0RT?=
 =?utf-8?B?YjJueE1UWEZ5VUZPOG56NEhUS0I3amVnb3pkQTRqaFhaUjBTNkdOemtSMzJG?=
 =?utf-8?B?dUlnQ1VDRHFkZkJOZlJKK2psMGkwSnk2VDFzeHFpU1RHdTNuVUV4TCtncFlP?=
 =?utf-8?B?NzM3NVkxVVByK2phUklmVVV6NlNtMEhiNDJZb0FkM3FyWGxReUIrNzRmczBq?=
 =?utf-8?B?VHJkQ0FXRk0wbE1KTTZrUT09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C0A76F4966E959469E07943732D30058@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB6324.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ac2e953a-c141-4923-0a1c-08d98433694a
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Sep 2021 16:57:39.3228
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sL9PIF5Udp5/fMuDm/PKA5FdKqeFnZrs5RLoXF2MHGyhbsYxVGiELnR47rxRt6DS3yWU7gM1LTcm8Xjv/AYe5Sxy1RTDs/5g5xdWjv0UND7+F9moyOMKdEVI0mU9gPL3
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB6851
X-Proofpoint-ORIG-GUID: pQ3K3oxrssbiVQEjkBB8jswJS4sz_lvM
X-Proofpoint-GUID: pQ3K3oxrssbiVQEjkBB8jswJS4sz_lvM
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475
 definitions=2021-09-30_06,2021-09-30_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0
 malwarescore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0
 clxscore=1015 phishscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2109300103

W3NuaXBdDQoNCj4+ICsgICAgYm9vbCBmb3VuZCA9IGZhbHNlOw0KPj4gKw0KPj4gKyAgICBwY2lk
ZXZzX2xvY2soKTsNCj4+ICsgICAgbGlzdF9mb3JfZWFjaF9lbnRyeSAoIHZkZXYsICZkLT52ZGV2
X2xpc3QsIGxpc3QgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBpZiAoIHZkZXYtPnNiZGYuc2Jk
ZiA9PSBzYmRmLT5zYmRmICkNCj4+ICsgICAgICAgIHsNCj4+ICsgICAgICAgICAgICAvKiBSZXBs
YWNlIHZpcnR1YWwgU0JERiB3aXRoIHRoZSBwaHlzaWNhbCBvbmUuICovDQo+PiArICAgICAgICAg
ICAgKnNiZGYgPSB2ZGV2LT5wZGV2LT5zYmRmOw0KPj4gKyAgICAgICAgICAgIGZvdW5kID0gdHJ1
ZTsNCj4+ICsgICAgICAgICAgICBicmVhazsNCj4+ICsgICAgICAgIH0NCj4+ICsgICAgfQ0KPj4g
KyAgICBwY2lkZXZzX3VubG9jaygpOw0KPiBBcyBwZXIgdGhlIGNvbW1lbnRzIG9uIHRoZSBlYXJs
aWVyIHBhdGNoLCBsb2NraW5nIGFzIHdlbGwgYXMgcGxhY2VtZW50DQo+IG1heSBuZWVkIHJlY29u
c2lkZXJpbmcuDQpJIHdhcyB0aGlua2luZyBhYm91dCB0aGUgbG9ja2luZyBoYXBwZW5pbmcgaGVy
ZS4NClNvLCB0aGVyZSBhcmUgNCBzb3VyY2VzIHdoZXJlIHdlIG5lZWQgdG8gbWFuaXB1bGF0ZSBk
LT52ZGV2X2xpc3Q6DQoxLiBYRU5fRE9NQ1RMX2Fzc2lnbl9kZXZpY2UNCjIuIFhFTl9ET01DVExf
dGVzdF9hc3NpZ25fZGV2aWNlDQozLiBYRU5fRE9NQ1RMX2RlYXNzaWduX2RldmljZQ0KNC4gTU1J
TyBoYW5kbGVycw0KNS4gRG8gSSBtaXNzIG90aGVycz8NCg0KVGhlIGZpcnN0IHRocmVlIGFscmVh
ZHkgdXNlIHBjaWRldnNfe2xvY2t8dW5sb2NrfSBhbmQgdGhlcmUgaXQgc2VlbXMNCnRvIGJlIG9r
IGFzIHRob3NlIGdldCBjYWxsZWQgd2hlbiBQQ0kgZGV2aWNlcyBhcmUgZGlzY292ZXJlZCBieSBE
b20wDQphbmQgZHVyaW5nIGd1ZXN0IGRvbWFpbiBjcmVhdGlvbi4gU28sIHRoaXMgaXMgYXNzdW1l
ZCBub3QgdG8gaGFwcGVuDQpmcmVxdWVudGx5IGFuZCBjYW4gYmUgYWNjZXB0ZWQgd3J0IGdsb2Jh
bCBsb2NraW5nLg0KDQpXaGF0IGlzIG1vcmUgaW1wb3J0YW50IGlzIHRoZSBmb3VydGggY2FzZSwg
d2hlcmUgaW4gb3JkZXIgdG8gcmVkaXJlY3QNCmNvbmZpZ3VyYXRpb24gc3BhY2UgYWNjZXNzIGZy
b20gdmlydHVhbCBTQkRGIHRvIHBoeXNpY2FsIFNCREYgd2UgbmVlZA0KdG8gdHJhdmVyc2UgdGhl
IGQtPnZkZXZfbGlzdCBlYWNoIHRpbWUgdGhlIGd1ZXN0IGFjY2Vzc2VzIFBDSSBjb25maWd1cmF0
aW9uDQpzcGFjZS4gVGhpcyBtZWFucyB0aGF0IHdpdGggZWFjaCBzdWNoIGFjY2VzcyB3ZSB0YWtl
IGEgQklHIFBDSSBsb2NrLi4uDQoNClRoYXQgYmVpbmcgc2FpZCwgSSB0aGluayB0aGF0IHdlIG1h
eSB3YW50IGhhdmluZyBhIGRlZGljYXRlZCBwZXItZG9tYWluDQpsb2NrIGZvciBkLT52ZGV2X2xp
c3QgaGFuZGxpbmcsIGUuZy4gZC0+dmRldl9sb2NrLg0KQXQgdGhlIHNhbWUgdGltZSB3ZSBtYXkg
YWxzbyBjb25zaWRlciB0aGF0IGV2ZW4gZm9yIGd1ZXN0cyBpdCBpcyBhY2NlcHRhYmxlDQp0byB1
c2UgcGNpZGV2c197bG9ja3x1bmxvY2t9IGFzIHRoaXMgd2lsbCBub3QgYWZmZWN0IFBDSSBtZW1v
cnkgc3BhY2UgYWNjZXNzDQphbmQgb25seSBoYXMgaW5mbHVlbmNlIGR1cmluZyBkZXZpY2Ugc2V0
dXAuDQoNCkkgd291bGQgbG92ZSB0byBoZWFyIHlvdXIgb3BpbmlvbiBvbiB0aGlzDQo+IEphbg0K
Pg0KVGhhbmsgeW91LA0KT2xla3NhbmRy


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:06:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:06:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200133.354539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzVg-0000LB-2s; Thu, 30 Sep 2021 17:06:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200133.354539; Thu, 30 Sep 2021 17:06:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzVf-0000L4-TG; Thu, 30 Sep 2021 17:06:15 +0000
Received: by outflank-mailman (input) for mailman id 200133;
 Thu, 30 Sep 2021 17:06:15 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Bos=OU=citrix.com=Andrew.Cooper3@srs-us1.protection.inumbo.net>)
 id 1mVzVf-0000Kx-HF
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 17:06:15 +0000
Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id b7c17668-2210-11ec-bd5e-12813bfff9fa;
 Thu, 30 Sep 2021 17:06:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7c17668-2210-11ec-bd5e-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1633021574;
  h=to:cc:references:from:subject:message-id:date:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=+yBG1dYgnROXjWD7I/OH3lcY6t51yc1yWq26CtjcYTI=;
  b=hsn1zZd3vgaNfNou6eGTr5VZNPDYchYOQqpOHmKuEVraHNa6mhga5sBO
   CCgI1Jxsohy7l5eHwFMa7h/XT3xpH0rfxxE7+AFmPrkV0QAI3LEIazkUK
   pl7l7TcuTtCRaRGUb/xWHWX4lgChB20QLkiFuqQIIyT+ziRZvxPRa4JSy
   s=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
IronPort-SDR: mpr9c8JZGpMpJkG2E2hV5Zv1IcML4QW7Qjy2IXalcjoCaWyneFQpcZTx/Qvzv70UIcieTE6IzQ
 +nu5PFm3jcqSlhxldqEgDUpxKe5DFkzFa+RU3SYICzY5N5IhypUZPOAgTJtEtDdyF2iWd44ohe
 zPzYSBXa+LN5T1P97ONnVQDTz+iP4YjTdKxxMSsq2mNJVBWSOzN/jCKmna7D8bZbH+9wso2URy
 s2an3sWgtafNLjbe7DjNqjxhzC/tkM9lOd1vfZ4p75UwT0tfeanVdKeZIyRh7yllDXRmDFLpOx
 zxK3PzJzjG5RjKQinJwjOkNd
X-SBRS: 5.1
X-MesageID: 54041134
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:5Wsy36xSWmkHPsQRVA96t+fCwSrEfRIJ4+MujC+fZmUNrF6WrkUCx
 jAcUGDXa/veYGf3L41zPNzk8UgO7JHdydI1T1ZsrCAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnvopW1TYhSEUOZugH9IQM8aZfHAsLeNYYH1500s7yrVh2tQAbeWRWGthh
 /uj+6UzB3f9s9JEGjp8B3Wr8U4HUFza4Vv0j3RmDRx5lAa2e0o9VfrzEZqZPXrgKrS4K8bhL
 wr1IBNVyUuCl/slIovNfr8W6STmSJaKVeSFoiI+t6RPHnGuD8H9u0o2HKN0VKtZt9mGt8B/7
 o1Pq52dcgYWOvz3vs4sbEMfPwgraMWq+JefSZS+mcmazkmAeHrw2fR+SkoxOOX0+M4uXzsIr
 6ZBbmlQMFbT3Ipaw5riIgVoru0lINPmI8U0vXZ4wCuCJf0nXYrCU+PB4towMDIY25wWQa+HP
 JZxhTxHNgiYfyNePw8rT6lhuuCj1l7kb2VDpwfAzUYwyzeKl1EguFT3C/LFd9rPSchLk0Kwo
 mPd43+/EhwcLMaYyzeO7jSrnOCntSr9RIIVEqz+7vNwiUef7mMWARwSE1C8pJGRgUekV9VSA
 0cd4CYpoO4580nDczXmd0Tm+jje5EdaAocOVb1hgO2Q9kbKyxaXIDAGUD9sU90ZsJ4NYxYb6
 QXOs/q8UFSDr4apYX6a876Vqxa7Ni4UMXIOaEc4cOcV3zXwiNtt102eF76PBIbw3oWsRm2gq
 9yfhHJm74j/m/LnwElSEbrvrTW2utDtRxM56x/bV2aohu+STNX+PNT5gbQ3AOwpEWp4crVjl
 CRZ8yR9xLpXZX1oqMBraL5WdF1Oz6zYWAAweXY1Q/EcG82FohZPh7x47jBkP1tOOc0ZYzLva
 0K7kVoPv8UCYCP7PfMpMt3Z5yEWIU7ITo+Nuhf8NIcmX3SMXFXfoHEGibC4hQgBb3TAYYlgY
 MzGIK5A/F4RCLh9zSreegvu+eRD+8zK/kuKHcqT503+idK2PSfJIZ9YYArmRr1ot8us/VSKm
 +uzwuPXkn2zpsWlOXKJmWPSRHhXRUUG6Wfe8ZcKKbLTflQ8SAnMyZb5mNscRmCspIwM/s/g9
 XChQE5Ijl35gHzMMwKRbX5/LrjoWP5CQbgTbETA5H6khCouZ5iB9qAae8dldLUr7rU7n/V1U
 +MEa4OLBfEWEmbL/DEUbJ/cqo1+dUv02VLSbnT9ODVvLYR9QwHp+8P/ele9/ic5ESfq59A1p
 Ket112HTMNbFRhiFsvfdNmm00i14SoGgOt3UkaReotTdUzg/ZJEMSv0ivNrccgAJQ+anmmR1
 hqMAAderu7I+tdn/N7MjKGCjoGoD+ohQRYKQziFte67bHCI8HCizIlMVPezUQrcDG6kqr+/Y
 eh1zu3nNKFVllh9rIchQa1gyrgz5oWzquYCnBhkBnjCc3+iFqhkfiudxcBKu6BAmu1ZtA+xV
 h7d89VWI+zUasbsEVpXLws5dOWTk/oTn2CKv/gyJUz74g5x/aaGDhoOb0Xd1nQFIesnKp4hz
 McgpNUSul62hRcdO9qbijxZqjaXJXsaXqR77pwXDecHUObwJo2utXAENhLL3Q==
IronPort-HdrOrdr: A9a23:1tj8Dqwdu8rnOEfixq8UKrPwIr1zdoMgy1knxilNoH1uHvBw8v
 rEoB1173DJYVoqNk3I++rhBEDwexLhHPdOiOF6UItKNzOW21dAQrsSiLfK8nnNHDD/6/4Y9Y
 oISdkbNDQoNykZsfrH
X-IronPort-AV: E=Sophos;i="5.85,336,1624334400"; 
   d="scan'208";a="54041134"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=l5kDTGUJDrWhdruXxm7PH8q5TKKwZSEqautSwi+axyRiTK5K75NhqUuQ++QeVa3xKNPrqL76Qh+WtPP4hCdy+236XFpZqGv5TiOfc+7UG0VMe29KB2GTXShPvVFHMZ1k8rfgDHedtpJvxzinv3wiNcPqYTYF8hJ3cBm7wWuEBBmRGb1pAaXI5lODvob4lW7+/9IDt7NqM1B3Bxg6Que+L/DQ5uXOHYzP4o/iCZdB+7AIZQv9/DIUuZWAfMoQAjOpm0jwXgj0O65S/BFK+PbA4bleXkBOnOQsIUOZJEdDAlJ9OOVeUm54+dM0NdCm3PZjn46Z7RgKwhcJKKNS2mUvuA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dc5vJBu+ChwdTu9pieDDIJmA6hfIZVGwJLJVkIwSfP8=;
 b=ltJXTZSyPXT2r/32vSA6XJ805I83BylQnBzL32KWhRYrJeU0qhNfcLb8YAvVSoF5Vcv14A9c+nQy0QVYpLrteZAqnCZCIOx9K2Q8qBXCAVjGEN8JxAOzmvZYZHWUxabqCoTrCyMGeWdhBbhB+VUpu3BHdqPIrLfx/IlePIqTir6N6dR1mYN3ZUSyJe4p8UgeTahB3VTSusfCYJEry9e0tgo3N8C9KtmSZ01EMaeB5LCl7ohdQmFGsuZrKW6v4sTUxkqKFpWHn3rYjWJCqEbpPn/UDL35kKgnEPDF+g02QKkraMZxXWHjvh/ucGHXgJASgoTseiTQxji+rOrtcfd4dA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=dc5vJBu+ChwdTu9pieDDIJmA6hfIZVGwJLJVkIwSfP8=;
 b=FOcjfFBCiPW8ooKrgwAQ5HfGp1aCfrKhSFaU0j4+/53OLNt7dEu5mWlsOU0mWPjNGcHytThVNJvDlbJ2Fot+hkHwK+ErRa+Z1Ul4X83/44Xhq8yM6HfhvnkcS8/UKoFOVETHowCCcJntItr6uW5RJtmNlHXZnQ3a+6MN2EH4I3o=
To: Anthony PERARD <anthony.perard@citrix.com>,
	<xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@gmail.com>, Doug Goldstein
	<cardoe@cardoe.com>
References: <20210930161720.825098-1-anthony.perard@citrix.com>
 <20210930161720.825098-3-anthony.perard@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH 2/2] automation: Add qemu to debian:stretch container
 for smoke test
Message-ID: <c242d263-25e1-1540-419b-11c74b2719d5@citrix.com>
Date: Thu, 30 Sep 2021 18:05:44 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20210930161720.825098-3-anthony.perard@citrix.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-GB
X-ClientProxiedBy: LO2P265CA0363.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:a3::15) To BYAPR03MB3623.namprd03.prod.outlook.com
 (2603:10b6:a02:aa::12)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5ed3ee7c-8fbb-434e-7d6a-08d984348db5
X-MS-TrafficTypeDiagnostic: SJ0PR03MB5887:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <SJ0PR03MB5887EED3EBA846D123CE40B5BAAA9@SJ0PR03MB5887.namprd03.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:1468;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 1MpRYu1ucL51becZvbQYVgIPsZd6QoURDmt1Fbx3l1OvvOoJy8eXOqimDLvCv6ZaiPuwQWBDyQeBgdxqM+JkmOEnGm86c10yNzKa53I+3569Lw4911MkvU5dfyxDYsuNZQebOJ6keYtbsB3zXG5ML4o1nPoYQW1x5c2x8yipMlPWGtgz7I5fs82g8NTlPshEJRIqn0NbAwzJ2hGjWS/sr2S75QRoPlBgNGnT9nlGAan/96cObRaHhxu49xxFfxkDWVFowvm5/Wc+s4VIuqxm1aw/Xnk+gXDzQWqlExIXKzkM1L7MAT5aF0N+w8Ys5K8UywqUGYpdlzXU7JbxKacleGgIW/H4nE6lDKgLGx8cIa7LLU4wzQQ6to6q9jyZuZ7vB8csLZ7TTNaqdSvqkY/DC+p9j/ZMaJnWglthq+HLnPIXVR6FHEevWLUOx1m7YTUCHbS7ZNUBE5k0nv/rOzgXGVlMsCf714cKdKyu0Di325YM1yya1FPH7S8TV6MjmsMGoXfuiPZ828eF80xIYxtTWOyONNaBFhY76l5BuYuL1ixBKLU09bDlTwquDhCPXsxUF/ZU5B3F5Z72XxJ91XOwveRRnTtuQwNr6/iUFrj9r66YdybRN5MW+Bke72QpFRhbg8yBraH+mGQ9odTv8mbgE4VwLvGOwU1QA/AQXYmi4GTaeydrRIPEgO1E19IDe5oz9AJbfSqYpiQG7bqs0f242K0iC8OfQD65kv30UIgQL2eePf6iGvJshTGlpqpMVyUWS3twUV0x3X4w+9FrsECGhreoSGjRATJ+qG9GubqXvEysSVK4bwbiB6YewM7oYJ3n
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR03MB3623.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(316002)(4326008)(8936002)(66946007)(2616005)(31696002)(83380400001)(36756003)(38100700002)(5660300002)(16576012)(31686004)(53546011)(54906003)(186003)(55236004)(956004)(26005)(508600001)(86362001)(966005)(6486002)(66556008)(66476007)(2906002)(8676002)(6666004)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZGRENkQ1YVVGZk51dUlISkNYTDZYNm9ycmNvY3NQeDIwWVErUVJNekszY2Ru?=
 =?utf-8?B?Nk1LUHk1SEx2WUZzQWpOTklReFRockVKaHBDVHF1VE5IQWhGT0lZeUo5L25L?=
 =?utf-8?B?aGZsQUFPU0h3Ymw3Sis1SjRlRUdoNkZhYmp3NGlsZ1pHMzA0eFJxYWdJVDlX?=
 =?utf-8?B?dWJrMmpvVVVYMUxhcGptL3lSV3pRM29ha2hBQm5oOUVqZngxVHZoZVNtcDdB?=
 =?utf-8?B?Y2Z2emY0aldmMDVzeHZMd2pSUTJkRmtRZkpJd21uTDZzckIvVElSSnhEUTN1?=
 =?utf-8?B?RjNuSFZiUVNxb1pUL2pUQWQ0ZHJvK2s3QklJZEd0R0hxRkRBRGllSmExcloz?=
 =?utf-8?B?NzkrenpYS1JpdGJjTDhPZTRGYXlnUXcraFVsZmQ3M05aSllUTHY3ZGJvdHJV?=
 =?utf-8?B?Q2E0bk51a0d3NFdXQTFOTXNxSGZYTVNpQnNESFU3bHJJRUtpMmIrMHlGWS9W?=
 =?utf-8?B?czJzYkY4WXNZVTlweFFnUVVZNlBvNTdSWXNFQWtBOU9leTNxWUdLeGREdWU2?=
 =?utf-8?B?NEZuTFpMeDB0TVA5empSK3E2K1BxanQ1VmgwOXRrZFpMQXYwTEVOZElBVTA3?=
 =?utf-8?B?SjhmSmo3bmRKQ3JGaSsxSTY5WDZQNk1zWnppUDVPMGo1eEFJREhWMjlZM0pR?=
 =?utf-8?B?dkxFRk53a1lyTWtVWEx3aWhMdlhESjB2T1VadUs2Um81a3QwYXhmTytaR2hV?=
 =?utf-8?B?ZTR6MCtWcUZpQzVtN3ZiU0w0SDVOOGQ4OWlsM3FTdEV0TUxMNnROVjE2MXhw?=
 =?utf-8?B?WGhIK2ViM0pMSlA3czdiUnIwV1hKeGZwdDdua2ZmL2tXSUJXb2h1UmlUUnc3?=
 =?utf-8?B?QVRyUWprZGdJNmxKWmw5UTVGSzJaY0Erd3dWTU9KZ1IrUHpuc3B3Z2laV1dJ?=
 =?utf-8?B?dVA0cXZWOXdjZzhXVzc3TXVkYkF3NVIyYjRNZ0M1T2VOVHhJeHI1Y04wYlJh?=
 =?utf-8?B?VVIrZ3Z5RVY4NjEwd1hxdjRoczl5Y21SQlNoenFWVHJpakR1NXhjazloZ3V1?=
 =?utf-8?B?VDZUdzRPcG5yY1ArMG5DeXlXdkFWVEtlK1NaL2NlZVVDb0hXSHpxMkovdC9p?=
 =?utf-8?B?dXBjUEY2QVkvVzVqVjlGRHFaL3J1OHI5VVlWRHBlV1hYdTI3V0NEekRzVnhG?=
 =?utf-8?B?bi95bUN2S3I5MzFLdFlPQjFySmNjNE5hTnhvSm1pbGxiajBSakNOMEVmVmt5?=
 =?utf-8?B?YVFST2d3YW5Vbkd5RnRoQzhaOW1iUnpXUExKcGpVN296MThRVTVvMzNJWDRY?=
 =?utf-8?B?R25UNmh1UWRXdTJ0dllqRldJMDllQUYyRWVuL2w4a0lLVUFuM21DMWp6SmMz?=
 =?utf-8?B?bm5OWXRCaHFCSnJUby92K2NrRUJkdHlMd2sxTGZxU29Uc09QKzdZdXBXTS9m?=
 =?utf-8?B?aXNkczJ3NEtVSDJsM2c2djFwck5GS201TTA4eFlxSnVKdG5qM1lXa3hsQWFJ?=
 =?utf-8?B?dXpmSWRBQjBjVlI3T0U4RVZUZ0pDdjlNN2FiNU9PMk9vb1lrd2F5VkROZE5X?=
 =?utf-8?B?c3A4cWJ0ZDkyWm1oZHJVQTk3VitPUjhuMW1NUlBrZ3d3ZnFjQTJ6dGQvR1VQ?=
 =?utf-8?B?Q3BORk92OUorRzVPK1hqbWpNUGdRUEhMK0ZIcE9icG52V3B6NnZCMkN0Rncz?=
 =?utf-8?B?bmNDbXdxV0ZJSnJnZkxLRERPb0U4a2xaekJaR3FWTU83aHpwb2VreWx4WDJr?=
 =?utf-8?B?cmcyb3JLYWZOcG9INzUvQmc5aSs2MWFkY09CVmUvdFhZdzFrRkxSS0FwTXJ0?=
 =?utf-8?Q?37xkEY8H28WqboXbWK7ny2+zAXInrrkLvNLzbAL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5ed3ee7c-8fbb-434e-7d6a-08d984348db5
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 17:05:50.3311
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XGWVwPVnw2wQxYyOWyBkNaIJy+lBYfIWgRF446VoJfeXfVMJEZM+kvNvmdeeqNUm1KEPNR6tBk9pj1d8yUCbezU2ektx3VWjzY9v4JT3DLQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5887
X-OriginatorOrg: citrix.com

On 30/09/2021 17:17, Anthony PERARD wrote:
> From: Anthony PERARD <anthony.perard@gmail.com>
>
> We can add qemu into the container so that there's no need to install
> it everytime we run a test.
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>
> Also, smoke tests stopped working as of today due to outdated
> root certificate, so container needs to be updated anyway.
>     fatal: unable to access 'https://xenbits.xen.org/git-http/xtf.git/': =
server certificate verification failed.
>
> I haven't push the container yet, I've only pushed it as
> debian:tmp-stretch for testing it.
> ---
>  automation/build/debian/stretch.dockerfile | 2 ++
>  automation/scripts/qemu-smoke-x86-64.sh    | 8 +++++---
>  2 files changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/automation/build/debian/stretch.dockerfile b/automation/buil=
d/debian/stretch.dockerfile
> index e2ee14e2017c..da6aa874dd70 100644
> --- a/automation/build/debian/stretch.dockerfile
> +++ b/automation/build/debian/stretch.dockerfile
> @@ -47,6 +47,8 @@ RUN apt-get update && \
>          nasm \
>          gnupg \
>          apt-transport-https \
> +        # for test phase, qemu-smoke-* jobs
> +        qemu-system-x86 \
>          && \
>          apt-get autoremove -y && \
>          apt-get clean && \
> diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts=
/qemu-smoke-x86-64.sh
> index 4b176c508dec..8ac065491c75 100755
> --- a/automation/scripts/qemu-smoke-x86-64.sh
> +++ b/automation/scripts/qemu-smoke-x86-64.sh
> @@ -6,9 +6,11 @@ set -ex
>  variant=3D$1
> =20
>  # Install QEMU
> -export DEBIAN_FRONTENT=3Dnoninteractive
> -apt-get -qy update
> -apt-get -qy install qemu-system-x86
> +if ! type qemu-system-x86_64; then
> +    export DEBIAN_FRONTENT=3Dnoninteractive
> +    apt-get -qy update
> +    apt-get -qy install qemu-system-x86
> +fi

I'd just delete this all.=C2=A0 It's wrong for running smoke tests in other
containers anyway.

Can fix commit too.

~Andrew

> =20
>  # Clone and build XTF
>  git clone https://xenbits.xen.org/git-http/xtf.git




From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:14:59 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:14:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200142.354550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVze1-0001z7-0m; Thu, 30 Sep 2021 17:14:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200142.354550; Thu, 30 Sep 2021 17:14:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVze0-0001z0-Sx; Thu, 30 Sep 2021 17:14:52 +0000
Received: by outflank-mailman (input) for mailman id 200142;
 Thu, 30 Sep 2021 17:14:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVzdz-0001yu-HL
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 17:14:51 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVzdz-0004vB-Bu
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 17:14:51 +0000
Received: from iwj (helo=mariner.uk.xensource.com)
 by xenbits.xenproject.org with local-bsmtp (Exim 4.92)
 (envelope-from <iwj@xenproject.org>) id 1mVzdz-0005Ow-Aw
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 17:14:51 +0000
Received: from iwj by mariner.uk.xensource.com with local (Exim 4.89)
 (envelope-from <iwj@xenproject.org>)
 id 1mVzdx-00032o-Hz; Thu, 30 Sep 2021 18:14:49 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=References:In-Reply-To:Subject:To:Date:
	Message-ID:Content-Transfer-Encoding:Content-Type:MIME-Version:From;
	bh=f8Z/iv8ikt50TQ2giy3eqC2D+ZjICP3qy3UB9SpVY1E=; b=ZpWq96sG3Aom96HX9i2cKRTBZk
	aTK2vx09sELiAgqm5tn24vhdcJzpUUrYMH3gen5wrCWRvNlUJs6kQnRQog/XjORjYVTo3I5kuhc8u
	7Npb7Z/Q2dKus/k5frvgL28bmDqb1OIlp4n7UXLU9a/sCboDH5xsXMhIEGp92+YrX3hU=;
From: Ian Jackson <iwj@xenproject.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <24917.61577.291999.406078@mariner.uk.xensource.com>
Date: Thu, 30 Sep 2021 18:14:49 +0100
To: xen-devel@lists.xenproject.org,
    committers@xenproject.org
Subject: Re: osstest down, PDU failure
In-Reply-To: <24916.53633.593814.456485@mariner.uk.xensource.com>
References: <24916.14069.358118.417330@mariner.uk.xensource.com>
	<24916.53633.593814.456485@mariner.uk.xensource.com>
X-Mailer: VM 8.2.0b under 24.5.1 (i686-pc-linux-gnu)

We have staff on site and are going to replace some PDUs.  There will
be some incomplete flight reports and then an outage.  I'm not sure
when service will be restored...

Ian.


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:16:27 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:16:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200147.354561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzfW-0002Z9-BZ; Thu, 30 Sep 2021 17:16:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200147.354561; Thu, 30 Sep 2021 17:16:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzfW-0002Z2-8Z; Thu, 30 Sep 2021 17:16:26 +0000
Received: by outflank-mailman (input) for mailman id 200147;
 Thu, 30 Sep 2021 17:16:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzfU-0002Ys-Ot; Thu, 30 Sep 2021 17:16:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzfU-0004xD-JJ; Thu, 30 Sep 2021 17:16:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzfU-0003Je-Al; Thu, 30 Sep 2021 17:16:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzfU-0003ys-AC; Thu, 30 Sep 2021 17:16:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BgvdHEVHLCHA4AJdKtDvvSxMG5wLfv7eKG8P6lBt9/0=; b=Jsa0LIo97lXTK5KLrGO5gX8jHI
	4G9427AIsGrFOZVnn+6CpryxApIshKLnsxsWJD4iUMlhPD9s1+7hCvqPNloG+EqLKH2KG3G00QuWT
	JGfRVdcJ3ksymAEkhDlrGUU5XOl4vSPyLJhIUXPtsU6bRJyk0zlmGHViuTleT25wpOJ4=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165334-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 165334: trouble: pass/preparing
X-Osstest-Failures:
    xtf:test-xtf-amd64-amd64-1:hosts-allocate:running:regression
    xtf:test-xtf-amd64-amd64-2:hosts-allocate:running:regression
    xtf:test-xtf-amd64-amd64-3:hosts-allocate:running:regression
    xtf:test-xtf-amd64-amd64-4:hosts-allocate:running:regression
    xtf:test-xtf-amd64-amd64-5:hosts-allocate:running:regression
X-Osstest-Versions-This:
    xtf=61e6f40b07d256bd62ae7b231a3eeecd49d0b15b
X-Osstest-Versions-That:
    xtf=91d215a4ed1463ab14d1f68e497117ac1255e05e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 17:16:24 +0000

flight 165334 xtf running [real]
http://logs.test-lab.xenproject.org/osstest/logs/165334/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-xtf-amd64-amd64-1        3 hosts-allocate               running
 test-xtf-amd64-amd64-2        3 hosts-allocate               running
 test-xtf-amd64-amd64-3        3 hosts-allocate               running
 test-xtf-amd64-amd64-4        3 hosts-allocate               running
 test-xtf-amd64-amd64-5        3 hosts-allocate               running

version targeted for testing:
 xtf                  61e6f40b07d256bd62ae7b231a3eeecd49d0b15b
baseline version:
 xtf                  91d215a4ed1463ab14d1f68e497117ac1255e05e

Last test of basis   164867  2021-09-07 00:10:12 Z   23 days
Testing same since                          (not found)         0 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Michal Orzel <michal.orzel@arm.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       preparing
 test-xtf-amd64-amd64-2                                       preparing
 test-xtf-amd64-amd64-3                                       preparing
 test-xtf-amd64-amd64-4                                       preparing
 test-xtf-amd64-amd64-5                                       preparing


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 61e6f40b07d256bd62ae7b231a3eeecd49d0b15b
Author: Michal Orzel <michal.orzel@arm.com>
Date:   Thu Sep 30 08:52:30 2021 +0200

    xsa-227: Fix link to XSA-227 web page
    
    Current link is invalid and gives the following error:
    "The requested URL was not found on this server."
    Fix it.
    
    Signed-off-by: Michal Orzel <michal.orzel@arm.com>
    Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
    
    XSA-227 was first written before c/s 487bce87eda33 "Docs: Fix the generated
    advisory links" but upstreamed later, hence the missing of the tree-wide
    cleanup.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

commit 9dadad475cf2018b1feac7fa440b22131b6a6ab4
Author: Michal Orzel <michal.orzel@arm.com>
Date:   Thu Sep 30 09:26:43 2021 +0200

    xsa-265: Fix link to a web page
    
    Currently, link points to XSA-264 and not XSA-265.
    Fix it.
    
    Signed-off-by: Michal Orzel <michal.orzel@arm.com>
    Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:17:53 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:17:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200154.354575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzgu-0003Ks-Mm; Thu, 30 Sep 2021 17:17:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200154.354575; Thu, 30 Sep 2021 17:17:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzgu-0003Kl-Jk; Thu, 30 Sep 2021 17:17:52 +0000
Received: by outflank-mailman (input) for mailman id 200154;
 Thu, 30 Sep 2021 17:17:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzgt-0003KV-5W; Thu, 30 Sep 2021 17:17:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzgt-0004yA-0g; Thu, 30 Sep 2021 17:17:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzgs-0003LH-Hy; Thu, 30 Sep 2021 17:17:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzgs-000401-HS; Thu, 30 Sep 2021 17:17:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Ur5w1u7RdbZNl8nO8qKiGkqZyGT++tliDsmPcBQZtx8=; b=a63orfcnVYfPe36f6mTiuDt/ui
	anfkaRXM+4mEmv6nl5sUIagnMid2pQ/K/3nUX1kyaFPsvon7w5NUtrNMa3RXnqyJAudjqdI+rdDP9
	weSwKAh26vUGm20D3CexfL080/QfAjmtOKheFbWBEV8ojOu6yEoTDEeULKuwVlMpSA10=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165331-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [qemu-mainline test] 165331: regressions - trouble: fail/pass/preparing/running
X-Osstest-Failures:
    qemu-mainline:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:regression
    qemu-mainline:test-armhf-armhf-libvirt-raw:hosts-allocate:running:regression
    qemu-mainline:test-armhf-armhf-libvirt:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-libvirt-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:hosts-allocate:running:regression
    qemu-mainline:test-armhf-armhf-xl-vhd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-libvirt:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-libvirt-pair:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-libvirt-raw:hosts-allocate:running:regression
    qemu-mainline:test-armhf-armhf-xl-multivcpu:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-libvirt-pair:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-libvirt:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-pygrub:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-libvirt-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-dom0pvh-xl-amd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-pvhv2-amd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-win7-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-debianhvm-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-pvshim:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-qemuu-freebsd11-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-coresched-amd64-xl:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-multivcpu:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-shadow:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-pair:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-freebsd10-i386:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-qemuu-nested-amd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-coresched-i386-xl:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-ws16-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qcow2:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-pvhv2-intel:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-qemuu-freebsd12-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-qemuu-nested-intel:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-ovmf-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-vhd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-shadow:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-win7-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-credit1:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-ws16-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-freebsd10-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-ovmf-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-rtds:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-amd64:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-pair:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-qemuu-rhel6hvm-intel:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-xl-pvshim:hosts-allocate:running:regression
    qemu-mainline:test-amd64-i386-qemuu-rhel6hvm-amd:hosts-allocate:running:regression
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:host-install(5):running:regression
    qemu-mainline:test-amd64-amd64-xl-credit2:debian-install:running:regression
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:host-install(5):running:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:host-ping-check-native:running:regression
    qemu-mainline:test-amd64-amd64-xl-credit2:syslog-server:running:regression
    qemu-mainline:test-armhf-armhf-xl-cubietruck:debian-install:running:regression
    qemu-mainline:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:syslog-server:running:regression
    qemu-mainline:test-armhf-armhf-xl-credit2:guest-start/debian.repeat:running:regression
    qemu-mainline:test-armhf-armhf-libvirt-qcow2:syslog-server:running:regression
    qemu-mainline:test-armhf-armhf-xl-credit2:syslog-server:running:regression
    qemu-mainline:test-armhf-armhf-xl-rtds:syslog-server:running:regression
    qemu-mainline:test-armhf-armhf-xl-cubietruck:syslog-server:running:regression
    qemu-mainline:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    qemu-mainline:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    qemu-mainline:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    qemuu=6b54a31bf7b403672a798b6443b1930ae6c74dea
X-Osstest-Versions-That:
    qemuu=99c44988d5ba1866a411450c877ed818b1b70081
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 17:17:50 +0000

flight 165331 qemu-mainline running [real]
http://logs.test-lab.xenproject.org/osstest/logs/165331/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 164950
 test-armhf-armhf-libvirt-raw  3 hosts-allocate               running
 test-armhf-armhf-libvirt      3 hosts-allocate               running
 test-amd64-i386-libvirt-xsm   3 hosts-allocate               running
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm  3 hosts-allocate    running
 test-armhf-armhf-xl-vhd       3 hosts-allocate               running
 test-amd64-i386-libvirt       3 hosts-allocate               running
 test-amd64-i386-libvirt-pair  4 hosts-allocate               running
 test-amd64-i386-libvirt-raw   3 hosts-allocate               running
 test-armhf-armhf-xl-multivcpu  3 hosts-allocate               running
 test-amd64-amd64-libvirt-pair  4 hosts-allocate               running
 test-amd64-amd64-libvirt      3 hosts-allocate               running
 test-amd64-amd64-pygrub       3 hosts-allocate               running
 test-amd64-amd64-libvirt-xsm  3 hosts-allocate               running
 test-amd64-amd64-dom0pvh-xl-amd  3 hosts-allocate               running
 test-amd64-amd64-xl-pvhv2-amd  3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-win7-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 3 hosts-allocate running
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  3 hosts-allocate            running
 test-amd64-amd64-xl-pvshim    3 hosts-allocate               running
 test-amd64-amd64-qemuu-freebsd11-amd64  3 hosts-allocate               running
 test-amd64-coresched-amd64-xl  3 hosts-allocate               running
 test-amd64-amd64-xl-multivcpu  3 hosts-allocate               running
 test-amd64-i386-xl-shadow     3 hosts-allocate               running
 test-amd64-amd64-pair         4 hosts-allocate               running
 test-amd64-i386-freebsd10-i386  3 hosts-allocate               running
 test-amd64-amd64-qemuu-nested-amd  3 hosts-allocate               running
 test-amd64-coresched-i386-xl  3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-ws16-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-qcow2     3 hosts-allocate               running
 test-amd64-amd64-xl-pvhv2-intel  3 hosts-allocate               running
 test-amd64-amd64-qemuu-freebsd12-amd64  3 hosts-allocate               running
 test-amd64-amd64-qemuu-nested-intel  3 hosts-allocate               running
 test-amd64-amd64-xl           3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-ovmf-amd64  3 hosts-allocate               running
 test-amd64-i386-xl-vhd        3 hosts-allocate               running
 test-amd64-amd64-xl-shadow    3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-win7-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  3 hosts-allocate     running
 test-amd64-amd64-xl-credit1   3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-ws16-amd64  3 hosts-allocate               running
 test-amd64-i386-freebsd10-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-xsm       3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-ovmf-amd64  3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  3 hosts-allocate          running
 test-amd64-amd64-xl-rtds      3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-debianhvm-amd64  3 hosts-allocate             running
 test-amd64-i386-xl-xsm        3 hosts-allocate               running
 test-amd64-i386-pair          4 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  3 hosts-allocate         running
 test-amd64-i386-qemuu-rhel6hvm-intel  3 hosts-allocate               running
 test-amd64-i386-xl            3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  3 hosts-allocate      running
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict  3 hosts-allocate running
 test-amd64-i386-xl-pvshim     3 hosts-allocate               running
 test-amd64-i386-qemuu-rhel6hvm-amd  3 hosts-allocate               running
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm  5 host-install(5)  running
 test-amd64-amd64-xl-credit2  12 debian-install               running
 test-armhf-armhf-libvirt-qcow2  5 host-install(5)              running
 test-armhf-armhf-xl-rtds      6 host-ping-check-native       running
 test-amd64-amd64-xl-credit2   4 syslog-server                running
 test-armhf-armhf-xl-cubietruck 12 debian-install               running
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm  4 syslog-server    running
 test-armhf-armhf-xl-credit2  18 guest-start/debian.repeat    running
 test-armhf-armhf-libvirt-qcow2  4 syslog-server                running
 test-armhf-armhf-xl-credit2   4 syslog-server                running
 test-armhf-armhf-xl-rtds      4 syslog-server                running
 test-armhf-armhf-xl-cubietruck  4 syslog-server                running

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 qemuu                6b54a31bf7b403672a798b6443b1930ae6c74dea
baseline version:
 qemuu                99c44988d5ba1866a411450c877ed818b1b70081

Last test of basis   164950  2021-09-11 18:57:22 Z   18 days
Failing since        164967  2021-09-13 13:06:52 Z   17 days   31 attempts
Testing same since   165319  2021-09-29 20:37:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Chen <alex.chen@huawei.com>
  Alexander Graf <agraf@csgraf.de>
  Alexandra Clifford <aclifford@draper.com>
  AlexChen <alex.chen@huawei.com>
  Alistair Francis <alistair.francis@wdc.com>
  Amanda Strnad <astrnad@draper.com>
  Andrew Jeffery <andrew@aj.id.au>
  Anup Patel <anup.patel@wdc.com>
  Bin Meng <bmeng.cn@gmail.com>
  Cai Huoqing <caihuoqing@baidu.com>
  Chris Rauer <crauer@google.com>
  Cleber Rosa <crosa@redhat.com>
  Cédric Le Goater <clg@kaod.org>
  Daniel P. Berrangé <berrange@redhat.com>
  David Gibson <david@gibson.dropbear.id.au>
  Dongli Zhang <dongli.zhang@oracle.com>
  Dr. David Alan Gilbert <dgilbert@redhat.com>
  Eric Blake <eblake@redhat.com>
  Frank Chang <frank.chang@sifive.com>
  G S Niteesh Babu <niteesh.gs@gmail.com>
  Gerd Hoffmann <kraxel@redhat.com>
  Green Wan <green.wan@sifive.com>
  Guenter Roeck <linux@roeck-us.net>
  Hanna Reitz <hreitz@redhat.com>
  Igor Mammedov <imammedo@redhat.com>
  Ilya Leoshkevich <iii@linux.ibm.com>
  Jason Wang <jasowang@redhat.com>
  Joel Stanley <joel@jms.id.au>
  John Arbuckle <programmingkidx@gmail.com>
  John Snow <jsnow@redhat.com>
  Jose R. Ziviani <jziviani@suse.de>
  Justin Restivo <jrestivo@draper.com>
  Kirill Tkhai <ktkhai@virtuozzo.com>
  Klaus Jensen <k.jensen@samsung.com>
  Lara Lazier <laramglazier@gmail.com>
  Laurent Vivier <laurent@vivier.eu>
  LIU Zhiwei <zhiwei_liu@c-sky.com>
  Luc Michel <lmichel@kalray.eu>
  Marc Zyngier <maz@kernel.org>
  Marc-André Lureau <marcandre.lureau@redhat.com>
  Markus Armbruster <armbru@redhat.com>
  Max Hsu <max.hsu@sifive.com>
  Max Reitz <mreitz@redhat.com>
  Michael Tokarev <mjt@tls.msk.ru>
  Michal Privoznik <mprivozn@redhat.com>
  Naveen Nagar <naveen.n1@samsung.com>
  Neil Armstrong <narmstrong@baylibre.com>
  Pankaj Gupta <pankaj.gupta.linux@gmail.com>
  Pankaj Raghav <p.raghav@samsung.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
  Peter Collingbourne <pcc@google.com>
  Peter Delevoryas <pdel@fb.com>
  Peter Maydell <peter.maydell@linaro.org>
  Peter Xu <peterx@redhat.com>
  Philippe Mathieu-Daudé <f4bug@amsat.org>
  Philippe Mathieu-Daudé <philmd@redhat.com>
  Reinoud Zandijk <Reinoud@NetBSD.org>
  Richard Henderson <richard.henderson@linaro.org>
  Richard W.M. Jones <rjones@redhat.com>
  Roman Bolshakov <r.bolshakov@yadro.com>
  Roman Bolshakov <r.bolshakov@yadro.com> (x86 only)
  Sergio Lopez <slp@redhat.com>
  Shashi Mallela <shashi.mallela@linaro.org>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Weil <sw@weilnetz.de>
  Stefano Garzarella <sgarzare@redhat.com>
  Thomas Huth <thuth@redhat.com>
  Viktor Prutyanov <viktor.prutyanov@phystech.edu>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Wainer dos Santos Moschetta <wainersm@redhat.com>
  Willian Rampazzo <willianr@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          preparing
 test-amd64-coresched-amd64-xl                                preparing
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           preparing
 test-amd64-coresched-i386-xl                                 preparing
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           running 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            preparing
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 preparing
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  preparing
 test-amd64-amd64-libvirt-xsm                                 preparing
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  preparing
 test-amd64-amd64-xl-xsm                                      preparing
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       preparing
 test-amd64-amd64-qemuu-nested-amd                            preparing
 test-amd64-amd64-xl-pvhv2-amd                                preparing
 test-amd64-i386-qemuu-rhel6hvm-amd                           preparing
 test-amd64-amd64-dom0pvh-xl-amd                              preparing
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    preparing
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     preparing
 test-amd64-i386-freebsd10-amd64                              preparing
 test-amd64-amd64-qemuu-freebsd11-amd64                       preparing
 test-amd64-amd64-qemuu-freebsd12-amd64                       preparing
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         preparing
 test-amd64-i386-xl-qemuu-ovmf-amd64                          preparing
 test-amd64-amd64-xl-qemuu-win7-amd64                         preparing
 test-amd64-i386-xl-qemuu-win7-amd64                          preparing
 test-amd64-amd64-xl-qemuu-ws16-amd64                         preparing
 test-amd64-i386-xl-qemuu-ws16-amd64                          preparing
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  preparing
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  running 
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  running 
 test-armhf-armhf-xl-cubietruck                               running 
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        preparing
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         preparing
 test-amd64-i386-freebsd10-i386                               preparing
 test-amd64-amd64-qemuu-nested-intel                          preparing
 test-amd64-amd64-xl-pvhv2-intel                              preparing
 test-amd64-i386-qemuu-rhel6hvm-intel                         preparing
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     preparing
 test-armhf-armhf-libvirt                                     preparing
 test-amd64-i386-libvirt                                      preparing
 test-amd64-amd64-xl-multivcpu                                preparing
 test-armhf-armhf-xl-multivcpu                                preparing
 test-amd64-amd64-pair                                        preparing
 test-amd64-i386-pair                                         preparing
 test-amd64-amd64-libvirt-pair                                preparing
 test-amd64-i386-libvirt-pair                                 preparing
 test-amd64-amd64-xl-pvshim                                   preparing
 test-amd64-i386-xl-pvshim                                    preparing
 test-amd64-amd64-pygrub                                      preparing
 test-armhf-armhf-libvirt-qcow2                               running 
 test-amd64-amd64-xl-qcow2                                    preparing
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 preparing
 test-amd64-i386-libvirt-raw                                  preparing
 test-amd64-amd64-xl-rtds                                     preparing
 test-armhf-armhf-xl-rtds                                     running 
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             preparing
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              preparing
 test-amd64-amd64-xl-shadow                                   preparing
 test-amd64-i386-xl-shadow                                    preparing
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      preparing
 test-amd64-i386-xl-vhd                                       preparing


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 6687 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:19:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:19:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200163.354589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVziF-00041R-8K; Thu, 30 Sep 2021 17:19:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200163.354589; Thu, 30 Sep 2021 17:19:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVziF-00041K-4q; Thu, 30 Sep 2021 17:19:15 +0000
Received: by outflank-mailman (input) for mailman id 200163;
 Thu, 30 Sep 2021 17:19:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVziE-000418-4v; Thu, 30 Sep 2021 17:19:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVziE-00051L-2W; Thu, 30 Sep 2021 17:19:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVziD-0003OJ-Qs; Thu, 30 Sep 2021 17:19:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVziD-00042P-QM; Thu, 30 Sep 2021 17:19:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sJLl+6jDJtjtD7VgeBBZFyA9mkDEjbKP0zmNo3+/rN4=; b=aMvqt3VKPoNFAUmdXbiiwKAFw8
	B5Kfnvq9qHEGNJdop8DgzvPx/qW91zDEVB9MlxrIJ4/ra0KDjBoQKNdiDrAz86Kszyfed/Y1GNx7q
	DOcHl3mYg3Qx9KpUq3pJukie2UwSsvzl0YmP543jlq/oKTWJV6toFgOPXztNYftN/NFM=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165333-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 165333: trouble: fail/pass/preparing/queued/running
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:<none executed>:queued:regression
    linux-linus:test-armhf-armhf-libvirt-qcow2:<none executed>:queued:regression
    linux-linus:test-armhf-armhf-libvirt-raw:<none executed>:queued:regression
    linux-linus:test-armhf-armhf-examine:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-examine:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-rtds:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-arndale:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-credit1:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-cubietruck:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-vhd:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-intel:hosts-allocate:running:regression
    linux-linus:test-armhf-armhf-xl-credit2:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-freebsd11-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-freebsd12-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt-raw:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt-pair:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-amd:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt-qcow2:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-libvirt-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-qemuu-nested-amd:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-pygrub:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-credit1:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-vhd:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-ovmf-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-pair:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-multivcpu:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-qemuu-nested-intel:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-pvshim:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-rtds:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-shadow:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-credit2:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-coresched-amd64-xl:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64:hosts-allocate:running:regression
    linux-linus:test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict:hosts-allocate:running:regression
    linux-linus:build-armhf-libvirt:syslog-server:running:regression
    linux-linus:build-armhf-libvirt:libvirt-build:running:regression
    linux-linus:test-arm64-arm64-xl-vhd:guest-start/debian.repeat:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=02d5e016800d082058b3d3b7c3ede136cdc6ddcb
X-Osstest-Versions-That:
    linux=deacdb3e3979979016fcd0ffd518c320a62ad166
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 17:19:13 +0000

flight 165333 linux-linus running [real]
http://logs.test-lab.xenproject.org/osstest/logs/165333/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt        <none executed>              queued
 test-armhf-armhf-libvirt-qcow2    <none executed>              queued
 test-armhf-armhf-libvirt-raw    <none executed>              queued
 test-armhf-armhf-examine      2 hosts-allocate               running
 test-amd64-amd64-examine      2 hosts-allocate               running
 test-armhf-armhf-xl           3 hosts-allocate               running
 test-armhf-armhf-xl-multivcpu  3 hosts-allocate               running
 test-armhf-armhf-xl-rtds      3 hosts-allocate               running
 test-armhf-armhf-xl-arndale   3 hosts-allocate               running
 test-armhf-armhf-xl-credit1   3 hosts-allocate               running
 test-armhf-armhf-xl-cubietruck  3 hosts-allocate               running
 test-armhf-armhf-xl-vhd       3 hosts-allocate               running
 test-amd64-amd64-dom0pvh-xl-intel  3 hosts-allocate               running
 test-armhf-armhf-xl-credit2   3 hosts-allocate               running
 test-amd64-amd64-freebsd11-amd64  3 hosts-allocate               running
 test-amd64-amd64-freebsd12-amd64  3 hosts-allocate               running
 test-amd64-amd64-libvirt      3 hosts-allocate               running
 test-amd64-amd64-libvirt-raw  3 hosts-allocate               running
 test-amd64-amd64-dom0pvh-xl-amd  3 hosts-allocate               running
 test-amd64-amd64-libvirt-pair  4 hosts-allocate               running
 test-amd64-amd64-xl-xsm       3 hosts-allocate               running
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm  3 hosts-allocate   running
 test-amd64-amd64-xl-pvhv2-amd  3 hosts-allocate               running
 test-amd64-amd64-libvirt-qcow2  3 hosts-allocate               running
 test-amd64-amd64-libvirt-xsm  3 hosts-allocate               running
 test-amd64-amd64-qemuu-nested-amd  3 hosts-allocate               running
 test-amd64-amd64-xl           3 hosts-allocate               running
 test-amd64-amd64-pygrub       3 hosts-allocate               running
 test-amd64-amd64-xl-credit1   3 hosts-allocate               running
 test-amd64-amd64-xl-vhd       3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-ovmf-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-pvhv2-intel  3 hosts-allocate               running
 test-amd64-amd64-pair         4 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  3 hosts-allocate         running
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 3 hosts-allocate running
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  3 hosts-allocate         running
 test-amd64-amd64-xl-multivcpu  3 hosts-allocate               running
 test-amd64-amd64-qemuu-nested-intel  3 hosts-allocate               running
 test-amd64-amd64-xl-qemut-ws16-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-qemut-win7-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-pvshim    3 hosts-allocate               running
 test-amd64-amd64-xl-rtds      3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-win7-amd64  3 hosts-allocate               running
 test-amd64-amd64-xl-shadow    3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  3 hosts-allocate     running
 test-amd64-amd64-xl-credit2   3 hosts-allocate               running
 test-amd64-amd64-xl-qemuu-ws16-amd64  3 hosts-allocate               running
 test-amd64-coresched-amd64-xl  3 hosts-allocate               running
 test-amd64-amd64-xl-qemut-debianhvm-amd64  3 hosts-allocate            running
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  3 hosts-allocate            running
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 3 hosts-allocate running
 build-armhf-libvirt           3 syslog-server                running
 build-armhf-libvirt           6 libvirt-build                running

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-vhd    17 guest-start/debian.repeat fail baseline untested
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                02d5e016800d082058b3d3b7c3ede136cdc6ddcb
baseline version:
 linux                deacdb3e3979979016fcd0ffd518c320a62ad166

Last test of basis   152332  2020-07-31 19:41:23 Z  425 days
Failing since        152366  2020-08-01 20:49:34 Z  424 days  741 attempts
Testing same since   165320  2021-09-29 20:41:33 Z    0 days    1 attempts

------------------------------------------------------------
7349 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          running 
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          preparing
 test-amd64-coresched-amd64-xl                                preparing
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          preparing
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           preparing
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        preparing
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 preparing
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 preparing
 test-amd64-amd64-libvirt-xsm                                 preparing
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      preparing
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            preparing
 test-amd64-amd64-xl-pvhv2-amd                                preparing
 test-amd64-amd64-dom0pvh-xl-amd                              preparing
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    preparing
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    preparing
 test-amd64-amd64-freebsd11-amd64                             preparing
 test-amd64-amd64-freebsd12-amd64                             preparing
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         preparing
 test-amd64-amd64-xl-qemut-win7-amd64                         preparing
 test-amd64-amd64-xl-qemuu-win7-amd64                         preparing
 test-amd64-amd64-xl-qemut-ws16-amd64                         preparing
 test-amd64-amd64-xl-qemuu-ws16-amd64                         preparing
 test-armhf-armhf-xl-arndale                                  preparing
 test-amd64-amd64-xl-credit1                                  preparing
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  preparing
 test-amd64-amd64-xl-credit2                                  preparing
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  preparing
 test-armhf-armhf-xl-cubietruck                               preparing
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        preparing
 test-amd64-amd64-examine                                     running 
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     running 
 test-amd64-amd64-qemuu-nested-intel                          preparing
 test-amd64-amd64-xl-pvhv2-intel                              preparing
 test-amd64-amd64-dom0pvh-xl-intel                            preparing
 test-amd64-amd64-libvirt                                     preparing
 test-armhf-armhf-libvirt                                     queued  
 test-amd64-amd64-xl-multivcpu                                preparing
 test-armhf-armhf-xl-multivcpu                                preparing
 test-amd64-amd64-pair                                        preparing
 test-amd64-amd64-libvirt-pair                                preparing
 test-amd64-amd64-xl-pvshim                                   preparing
 test-amd64-amd64-pygrub                                      preparing
 test-amd64-amd64-libvirt-qcow2                               preparing
 test-armhf-armhf-libvirt-qcow2                               queued  
 test-amd64-amd64-libvirt-raw                                 preparing
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 queued  
 test-amd64-amd64-xl-rtds                                     preparing
 test-armhf-armhf-xl-rtds                                     preparing
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             preparing
 test-amd64-amd64-xl-shadow                                   preparing
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-xl-vhd                                      preparing
 test-arm64-arm64-xl-vhd                                      fail    
 test-armhf-armhf-xl-vhd                                      preparing


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-libvirt queued
broken-job test-armhf-armhf-libvirt-qcow2 queued
broken-job test-armhf-armhf-libvirt-raw queued

Not pushing.

(No revision log; it would be 2274884 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:20:04 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:20:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200170.354603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVziz-0004ay-Km; Thu, 30 Sep 2021 17:20:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200170.354603; Thu, 30 Sep 2021 17:20:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVziz-0004ar-Fw; Thu, 30 Sep 2021 17:20:01 +0000
Received: by outflank-mailman (input) for mailman id 200170;
 Thu, 30 Sep 2021 17:20:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzix-0004ad-V8; Thu, 30 Sep 2021 17:19:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzix-00051x-Oo; Thu, 30 Sep 2021 17:19:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzix-0003P9-9B; Thu, 30 Sep 2021 17:19:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzix-00044l-8g; Thu, 30 Sep 2021 17:19:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KFASEe5cphzgTA/addmm0dOEy5F8IoO0ljH2HFrqo4Q=; b=hhRqKDEOVmDqUj5PqlyZ9bwQ+l
	KW+HBdxdUwe7XboiZAdwzdI9NTcqOX1DrlHP834H2f4FX8QEoigAcHbZjOeV/Q6Lfl+WlsQwSWvM0
	C2TIVKf0ysMjQ76kqe0lelqy2mpq/4RvNxcVHsXzWOB896xp3rXLEP/xJyRg82R5rMF8=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165330-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 165330: trouble: pass/preparing
X-Osstest-Failures:
    ovmf:test-amd64-amd64-xl-qemuu-ovmf-amd64:hosts-allocate:running:regression
    ovmf:test-amd64-i386-xl-qemuu-ovmf-amd64:hosts-allocate:running:regression
X-Osstest-Versions-This:
    ovmf=9e950cda6ad9dab6bb31868d341e3f5d8cd296e3
X-Osstest-Versions-That:
    ovmf=442e46d3b6c1931b54111c92e0efb5a797bc622b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 17:19:59 +0000

flight 165330 ovmf running [real]
http://logs.test-lab.xenproject.org/osstest/logs/165330/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-ovmf-amd64  3 hosts-allocate               running

version targeted for testing:
 ovmf                 9e950cda6ad9dab6bb31868d341e3f5d8cd296e3
baseline version:
 ovmf                 442e46d3b6c1931b54111c92e0efb5a797bc622b

Last test of basis   165321  2021-09-29 20:41:38 Z    0 days
Testing same since                          (not found)         0 attempts

------------------------------------------------------------
People who touched revisions under test:
  Kun Qin <kuqin12@gmail.com>
  Stefan Berger <stefanb@linux.ibm.com>
  Stefan Berger <stefanb@linux.vnet.ibm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         preparing
 test-amd64-i386-xl-qemuu-ovmf-amd64                          preparing


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 9e950cda6ad9dab6bb31868d341e3f5d8cd296e3
Author: Kun Qin <kuqin12@gmail.com>
Date:   Thu Sep 16 08:14:20 2021 +0800

    MdeModulePkg: CI YAML: Added new GUID to ignore duplicate list
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3629
    
    SMM Communication PPI GUID from MdeModulePkg is defined the same as MM
    Communication PPI GUID from MdePkg, according to PI Spec v1.5 and onward.
    
    After introduction of MM Communication PPI definitions, an update in the
    ignore duplicate list is needed to avoid breaking CI build.
    
    Cc: Jian J Wang <jian.j.wang@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    
    Signed-off-by: Kun Qin <kuqin12@gmail.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit 8b4bb94f6484f84d48a3dc77fa31e954fc75c4b5
Author: Kun Qin <kuqin12@gmail.com>
Date:   Thu Sep 16 08:14:19 2021 +0800

    MdePkg: CI YAML: Added new GUID to ignore duplicate list
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3629
    
    SMM Communication PPI GUID from MdeModulePkg is defined the same as MM
    Communication PPI GUID from MdePkg, according to PI Spec v1.5 and onward.
    
    After introduction of MM Communication PPI definitions, an update in the
    ignore duplicate list is needed to avoid breaking CI build.
    
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Zhiguang Liu <zhiguang.liu@intel.com>
    
    Signed-off-by: Kun Qin <kuqin12@gmail.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit f86de75862c20077aa9e7dbe71c9c9182d783d5f
Author: Kun Qin <kuqin12@gmail.com>
Date:   Thu Sep 16 08:14:18 2021 +0800

    MdePkg: MmCommunication: Added definition of MM Communication PPI
    
    REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3629
    
    MM Communication PPI was defined in PI Specification since v1.5. This
    change added definition of such PPI and related GUIDs into MdePkg.
    
    Cc: Michael D Kinney <michael.d.kinney@intel.com>
    Cc: Liming Gao <gaoliming@byosoft.com.cn>
    Cc: Zhiguang Liu <zhiguang.liu@intel.com>
    Cc: Sean Brogan <sean.brogan@microsoft.com>
    
    Signed-off-by: Kun Qin <kuqin12@gmail.com>
    Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

commit bd298d75934a7a0cf6b9e2863a5e4975e9714285
Author: Stefan Berger <stefanb@linux.vnet.ibm.com>
Date:   Wed Sep 15 09:25:06 2021 +0800

    OvmfPkg: Reference new Tcg2PlatformPei in the build system
    
    Compile the Tcg2PlatformPei related code now to support TPM 2 platform
    hierachy disablement if the TPM state cannot be resumed upon S3 resume.
    
    Cc: Rebecca Cran <rebecca@bsdio.com>
    Cc: Peter Grehan <grehan@freebsd.org>
    Cc: Brijesh Singh <brijesh.singh@amd.com>
    Cc: Erdem Aktas <erdemaktas@google.com>
    Cc: James Bottomley <jejb@linux.ibm.com>
    Cc: Jiewen Yao <jiewen.yao@intel.com>
    Cc: Min Xu <min.m.xu@intel.com>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
    Cc: Jordan Justen <jordan.l.justen@intel.com>
    Cc: Gerd Hoffmann <kraxel@redhat.com>
    Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

commit 8ab8fbc016e789bb45b2c8fb7a6887a28cc32353
Author: Stefan Berger <stefanb@linux.vnet.ibm.com>
Date:   Wed Sep 15 09:25:05 2021 +0800

    OvmfPkg: Reference new Tcg2PlatformDxe in the build system for compilation
    
    Compile the Tcg2PlatformDxe related code now.
    
    Cc: Rebecca Cran <rebecca@bsdio.com>
    Cc: Peter Grehan <grehan@freebsd.org>
    Cc: Brijesh Singh <brijesh.singh@amd.com>
    Cc: Erdem Aktas <erdemaktas@google.com>
    Cc: James Bottomley <jejb@linux.ibm.com>
    Cc: Jiewen Yao <jiewen.yao@intel.com>
    Cc: Min Xu <min.m.xu@intel.com>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
    Cc: Jordan Justen <jordan.l.justen@intel.com>
    Cc: Gerd Hoffmann <kraxel@redhat.com>
    Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

commit b8675deaa819631db2667df63f89799fe65fc906
Author: Stefan Berger <stefanb@linux.vnet.ibm.com>
Date:   Wed Sep 15 09:25:04 2021 +0800

    OvmfPkg: Handle TPM 2 physical presence opcodes much earlier
    
    Handle the TPM 2 physical presence interface (PPI) opcodes in
    PlatformBootManagerBeforeConsole() before the TPM 2 platform hierarchy
    is disabled. Since the handling of the PPI opcodes may require inter-
    action with the user, initialize the keyboard before handling PPI codes.
    
    Cc: Rebecca Cran <rebecca@bsdio.com>
    Cc: Peter Grehan <grehan@freebsd.org>
    Cc: Brijesh Singh <brijesh.singh@amd.com>
    Cc: Erdem Aktas <erdemaktas@google.com>
    Cc: James Bottomley <jejb@linux.ibm.com>
    Cc: Jiewen Yao <jiewen.yao@intel.com>
    Cc: Min Xu <min.m.xu@intel.com>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
    Cc: Jordan Justen <jordan.l.justen@intel.com>
    Cc: Gerd Hoffmann <kraxel@redhat.com>
    Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

commit 499c4608b13df0738213329722cdcb0ac838f3ad
Author: Stefan Berger <stefanb@linux.vnet.ibm.com>
Date:   Wed Sep 15 09:25:03 2021 +0800

    OvmfPkg/TPM PPI: Connect default consoles for user interaction
    
    Activate the default console when user interaction is required for
    the processing of TPM 2 physical presence interface opcodes.
    
    Background:
    TPM 2 physical presence interface (PPI) opcodes need to be handled before
    the TPM 2 platform hierarchy is disabled. Due to this requirement we will
    move the function call to handle the PPI opcodes into
    PlatformBootManagerBeforeConsole() which runs before the initialization
    of the consoles. However, since for interaction with the user we need
    the console to be available, activate it now before displaying any message
    to the user.
    
    Cc: Rebecca Cran <rebecca@bsdio.com>
    Cc: Peter Grehan <grehan@freebsd.org>
    Cc: Brijesh Singh <brijesh.singh@amd.com>
    Cc: Erdem Aktas <erdemaktas@google.com>
    Cc: James Bottomley <jejb@linux.ibm.com>
    Cc: Jiewen Yao <jiewen.yao@intel.com>
    Cc: Min Xu <min.m.xu@intel.com>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
    Cc: Jordan Justen <jordan.l.justen@intel.com>
    Cc: Gerd Hoffmann <kraxel@redhat.com>
    Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
    Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 17:22:08 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 17:22:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200180.354617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzl2-00063e-7U; Thu, 30 Sep 2021 17:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200180.354617; Thu, 30 Sep 2021 17:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mVzl2-00063X-4D; Thu, 30 Sep 2021 17:22:08 +0000
Received: by outflank-mailman (input) for mailman id 200180;
 Thu, 30 Sep 2021 17:22:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzl0-00063L-Jx; Thu, 30 Sep 2021 17:22:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzl0-000545-D2; Thu, 30 Sep 2021 17:22:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzl0-0003Rk-5r; Thu, 30 Sep 2021 17:22:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1mVzl0-0004lX-5I; Thu, 30 Sep 2021 17:22:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=siwnnwy3UaYdeVgDKglqP7hmeR5rej2qyPoDrDaxjpg=; b=OEhxtnOTofUbhGSrfEwgDhiwrv
	AojMmkESzlo9hLXEuTa18tMR0YxkT4LiZSdtGZEmlQVTjyqVV6qpi8RlL3DXc/qAJP/eV7p+7YJih
	tux3viVeUK9iQZxW79UloTxkeFG06RsqBni3xm8EPwuD1+oAnhQpKkF8qe/G/tyjM7wo=;
To: xen-devel@lists.xenproject.org,
    osstest-admin@xenproject.org
Message-ID: <osstest-165329-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 165329: regressions - trouble: fail/pass/preparing/running
X-Osstest-Failures:
    linux-5.4:test-armhf-armhf-xl-credit1:guest-start:fail:regression
    linux-5.4:test-amd64-coresched-i386-xl:hosts-allocate:running:regression
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:hosts-allocate:running:regression
    linux-5.4:test-amd64-i386-qemut-rhel6hvm-amd:hosts-allocate:running:regression
    linux-5.4:test-amd64-coresched-amd64-xl:hosts-allocate:running:regression
    linux-5.4:test-amd64-i386-qemuu-rhel6hvm-amd:hosts-allocate:running:regression
    linux-5.4:test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow:hosts-allocate:running:regression
    linux-5.4:test-amd64-i386-libvirt-xsm:host-install(5):running:regression
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:host-install(5):running:regression
    linux-5.4:test-amd64-i386-libvirt:debian-install:running:regression
    linux-5.4:test-amd64-i386-xl-qemuu-debianhvm-amd64:debian-hvm-install:running:regression
    linux-5.4:test-armhf-armhf-libvirt-raw:host-install(5):running:regression
    linux-5.4:test-amd64-i386-xl:guest-start/debian.repeat:running:regression
    linux-5.4:test-amd64-i386-xl:syslog-server:running:regression
    linux-5.4:test-amd64-i386-libvirt-xsm:syslog-server:running:regression
    linux-5.4:test-amd64-amd64-xl-pvhv2-amd:capture-logs(26):running:regression
    linux-5.4:test-amd64-amd64-xl-pvhv2-amd:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:running:regression
    linux-5.4:test-amd64-i386-xl-shadow:guest-start/debian.repeat:running:regression
    linux-5.4:test-amd64-amd64-qemuu-freebsd11-amd64:syslog-server:running:regression
    linux-5.4:test-amd64-amd64-qemuu-freebsd11-amd64:capture-logs(25):running:regression
    linux-5.4:test-amd64-i386-xl-shadow:syslog-server:running:regression
    linux-5.4:test-amd64-i386-libvirt:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:running:regression
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:windows-install:running:regression
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemuu-debianhvm-amd64:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:syslog-server:running:regression
    linux-5.4:test-armhf-armhf-libvirt-raw:syslog-server:running:regression
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:syslog-server:running:regression
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:guest-start/debian.repeat:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-seattle:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-cubietruck:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=3a7dc5b4cfbdfd8da37cb0e54f439d84cf5a5123
X-Osstest-Versions-That:
    linux=e74e2950a0d6f800858e54860d7124c86e494f62
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 30 Sep 2021 17:22:06 +0000

flight 165329 linux-5.4 running [real]
http://logs.test-lab.xenproject.org/osstest/logs/165329/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1  14 guest-start              fail REGR. vs. 165206
 test-amd64-coresched-i386-xl  3 hosts-allocate               running
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm  3 hosts-allocate    running
 test-amd64-i386-qemut-rhel6hvm-amd  3 hosts-allocate               running
 test-amd64-coresched-amd64-xl  3 hosts-allocate               running
 test-amd64-i386-qemuu-rhel6hvm-amd  3 hosts-allocate               running
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  3 hosts-allocate      running
 test-amd64-i386-libvirt-xsm   5 host-install(5)              running
 test-amd64-amd64-qemuu-nested-amd  5 host-install(5)              running
 test-amd64-i386-libvirt      12 debian-install               running
 test-amd64-i386-xl-qemuu-debianhvm-amd64 12 debian-hvm-install         running
 test-armhf-armhf-libvirt-raw  5 host-install(5)              running
 test-amd64-i386-xl           22 guest-start/debian.repeat    running
 test-amd64-i386-xl            4 syslog-server                running
 test-amd64-i386-libvirt-xsm   4 syslog-server                running
 test-amd64-amd64-xl-pvhv2-amd 26 capture-logs(26)             running
 test-amd64-amd64-xl-pvhv2-amd  4 syslog-server                running
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install      running
 test-amd64-i386-xl-shadow    22 guest-start/debian.repeat    running
 test-amd64-amd64-qemuu-freebsd11-amd64  4 syslog-server                running
 test-amd64-amd64-qemuu-freebsd11-amd64 25 capture-logs(25)             running
 test-amd64-i386-xl-shadow     4 syslog-server                running
 test-amd64-i386-libvirt       4 syslog-server                running
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop                   running
 test-amd64-i386-xl-qemut-win7-amd64  4 syslog-server                running
 test-amd64-i386-xl-qemuu-ws16-amd64 12 windows-install              running
 test-amd64-amd64-qemuu-nested-amd  4 syslog-server                running
 test-amd64-i386-xl-qemuu-debianhvm-amd64  4 syslog-server              running
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  4 syslog-server           running
 test-armhf-armhf-libvirt-raw  4 syslog-server                running
 test-amd64-i386-xl-qemuu-ws16-amd64  4 syslog-server                running

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 165206
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 165206
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat    fail  like 165206
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 165206
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 165206
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 165206
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 165206
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 165206
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 165206
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-check        fail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-check    fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                3a7dc5b4cfbdfd8da37cb0e54f439d84cf5a5123
baseline version:
 linux                e74e2950a0d6f800858e54860d7124c86e494f62

Last test of basis   165206  2021-09-26 12:42:13 Z    4 days
Testing same since                          (not found)         0 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alan Stern <stern@rowland.harvard.edu>
  Alexei Starovoitov <ast@kernel.org>
  Andreas Larsson <andreas@gaisler.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Antoine Tenart <atenart@kernel.org>
  Anton Eidelman <anton.eidelman@gmail.com>
  Anton Eidelman <anton@lightbitslabs.com>
  Ariel Elior <aelior@marvell.com>
  Arnd Bergmann <arnd@arndb.de>
  Aya Levin <ayal@nvidia.com>
  Baokun Li <libaokun1@huawei.com>
  Bartosz Golaszewski <brgl@bgdev.pl>
  Bixuan Cui <cuibixuan@huawei.com>
  Borislav Petkov <bp@suse.de>
  Carlo Lobrano <c.lobrano@gmail.com>
  Catalin Marinas <catalin.marinas@arm.com>
  Chris Chiu <chris.chiu@canonical.com>
  Christian Brauner <christian.brauner@ubuntu.com>
  Christoph Hellwig <hch@lst.de>
  Claudiu Manoil <claudiu.manoil@nxp.com>
  Dan Carpenter <dan.carpenter@oracle.com>
  Dan Li <ashimida@linux.alibaba.com>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Sterba <dsterba@suse.com>
  Dmitry Bogdanov <d.bogdanov@yadro.com>
  Felipe Balbi <balbi@kernel.org>
  Florian Fainelli <f.fainelli@gmail.com>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Geert Uytterhoeven <geert@linux-m68k.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Gregory CLEMENT <gregory.clement@bootlin.com>
  Guenter Roeck <linux@roeck-us.net>
  Hans de Goede <hdegoede@redhat.com>
  Helge Deller <deller@gmx.de>
  Hulk Robot <hulkrobot@huawei.com>
  Jan Beulich <jbeulich@suse.com>
  Jens Axboe <axboe@kernel.dk>
  Jesper Nilsson <jesper.nilsson@axis.com>
  Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
  Jiri Slaby <jslaby@suse.cz>
  Johan Hovold <johan@kernel.org>
  Johannes Thumshirn <jth@kernel.org>
  Jon Hunter <jonathanh@nvidia.com>
  Juergen Gross <jgross@suse.com>
  Julian Sikorski <belegdol+github@gmail.com>
  Julian Sikorski <belegdol@gmail.com>
  Kaige Fu <kaige.fu@linux.alibaba.com>
  Karsten Graul <kgraul@linux.ibm.com>
  Kishon Vijay Abraham I <kishon@ti.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
  Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
  Li Jinlin <lijinlin3@huawei.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Marc Zyngier <maz@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Markus Suvanto <markus.suvanto@gmail.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Michael Chan <michael.chan@broadcom.com>
  Michael Chan <michael.chan@broadocm.com>
  Michal Kalderon <mkalderon@marvell.com>
  Minas Harutyunyan <Minas.Harutyunyan@synopsys.com>
  Moritz Fischer <mdf@kernel.org>
  Nicolas Ferre <nicolas.ferre@microchip.com>
  Oliver Neukum <oneukum@suse.com>
  Ondrej Zary <linux@zary.sk>
  Pali Rohár <pali@kernel.org>
  Qu Wenruo <wqu@suse.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ronnie Sahlberg <lsahlber@redhat.com>
  Sai Krishna Potthuri <lakshmi.sai.krishna.potthuri@xilinx.com>
  Sasha Levin <sashal@kernel.org>
  Shai Malin <smalin@marvell.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Shubhrajyoti Datta <shubhrajyoti.datta@xilinx.com>
  Slark Xiao <slark_xiao@163.com>
  Song Liu <songliubraving@fb.com>
  Srinivas Pandruvada <srinivas.pI andruvada@linux.intel.com>
  Steve French <stfrench@microsoft.com>
  Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
  syzbot+fadc0aaf497e6a493b9f@syzkaller.appspotmail.com
  Tariq Toukan <tariqt@nvidia.com>
  Tejun Heo <tj@kernel.org>
  Todd Kjos <tkjos@google.com>
  Tom Rix <trix@redhat.com>
  Tong Zhang <ztong0001@gmail.com>
  Uwe Brandt <uwe.brandt@gmail.com>
  Wengang Wang <wen.gang.wang@oracle.com>
  Yonghong Song <yhs@fb.com>
  zhang kai <zhangkaiheb@126.com>
  Zhihao Cheng <chengzhihao1@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                preparing
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           running 
 test-amd64-coresched-i386-xl                                 preparing
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            preparing
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  running 
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  running 
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            running 
 test-amd64-amd64-xl-pvhv2-amd                                running 
 test-amd64-i386-qemut-rhel6hvm-amd                           preparing
 test-amd64-i386-qemuu-rhel6hvm-amd                           preparing
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     running 
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       running 
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          running 
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          running 
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-armhf-armhf-xl-cubietruck                               pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      running 
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 fail    
 test-armhf-armhf-libvirt-raw                                 running 
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-arm64-arm64-xl-seattle                                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              preparing
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    running 
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1974 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:02 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200194.354631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW18o-0007B5-LK; Thu, 30 Sep 2021 18:50:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200194.354631; Thu, 30 Sep 2021 18:50:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW18o-0007Ay-I9; Thu, 30 Sep 2021 18:50:46 +0000
Received: by outflank-mailman (input) for mailman id 200194;
 Thu, 30 Sep 2021 18:50:44 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW18l-0007As-Pm
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:50:44 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id e19b9a62-fa6b-4503-8147-c0d59cd72030;
 Thu, 30 Sep 2021 18:50:42 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8244C61216;
 Thu, 30 Sep 2021 18:50:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e19b9a62-fa6b-4503-8147-c0d59cd72030
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027842;
	bh=sHuklxRhVNrOUIGACjRjjFQjjWujtmMEebA4SOO7XVI=;
	h=From:To:Cc:Subject:Date:From;
	b=uHnQO7KnOuN+F6Kg/VYMCkYNZKP/FQuRQKsHlQ02RSvOU+Lsr6a/Ejf3NJuypnQ0r
	 wgcHaC0oqgL3wh/+Bjpr7enKokEpGkilkhAcBEHhE9uLRLhpb0YgHbs+X+j4bqDpV7
	 0AkfKl3RUdJBxJT+ECt+6aVhpjwQ315nbX0A7Nb4a9ZuqQ7VSnVENanCiGErOWceKw
	 TlOjeEhAK0IhpTcaKezOnHkaHC7hBR/4sk1qUNqqo7JqlIbR9UE+BEQUtaXKIMytSU
	 2kYuIhxuTpWdfCtUUQdGYdGcDoSXL5vM5e4zMX86OIlLp0nscaVxPuacUtXLN3LJw1
	 +kT+1vHV0j00g==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 0/6] memblock: cleanup memblock_free interface
Date: Thu, 30 Sep 2021 21:50:25 +0300
Message-Id: <20210930185031.18648-1-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Hi,

Following the discussion on [1] this is the fix for memblock freeing APIs
mismatch. 

The first patch is a cleanup of numa_distance allocation in arch_numa I've
spotted during the conversion.
The second patch is a fix for Xen memory freeing on some of the error
paths.

I agree with Christophe that doing step by step makes the thing easier to
review, so the patches 3-6 do the actual cleanup step by step.

This time I used stricter coccinelle scripts so that only straightforward
uses would get converted.

There still a couple of (void *) castings for the cases when a virtual
address has unsigned long type rather than a pointer type, like e.g
initrd_start.

Since scripts/get_maintainer.pl returned more than 100 addresses I've
trimmed the distribution list only to the relevant lists.

Juergen and Shahab, I didn't keep your Reviewed-by because the patches are
a bit different this time.

v2:
* split changes into several patches
* use stricter coccinelle scripts 

[1] https://lore.kernel.org/all/CAHk-=wj9k4LZTz+svCxLYs5Y1=+yKrbAUArH1+ghyG3OLd8VVg@mail.gmail.com

Mike Rapoport (6):
  arch_numa: simplify numa_distance allocation
  xen/x86: free_p2m_page: use memblock_free_ptr() to free a virtual pointer
  memblock: drop memblock_free_early_nid() and memblock_free_early()
  memblock: stop aliasing __memblock_free_late with memblock_free_late
  memblock: rename memblock_free to memblock_phys_free
  memblock: use memblock_free for freeing virtual pointers

 arch/alpha/kernel/core_irongate.c         |  2 +-
 arch/arc/mm/init.c                        |  2 +-
 arch/arm/mach-hisi/platmcpm.c             |  2 +-
 arch/arm/mm/init.c                        |  2 +-
 arch/arm64/mm/mmu.c                       |  4 ++--
 arch/mips/mm/init.c                       |  2 +-
 arch/mips/sgi-ip30/ip30-setup.c           |  6 +++---
 arch/powerpc/kernel/dt_cpu_ftrs.c         |  4 ++--
 arch/powerpc/kernel/paca.c                |  8 ++++----
 arch/powerpc/kernel/setup-common.c        |  2 +-
 arch/powerpc/kernel/setup_64.c            |  2 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |  2 +-
 arch/powerpc/platforms/pseries/svm.c      |  3 +--
 arch/riscv/kernel/setup.c                 |  4 ++--
 arch/s390/kernel/setup.c                  |  8 ++++----
 arch/s390/kernel/smp.c                    |  4 ++--
 arch/s390/kernel/uv.c                     |  2 +-
 arch/s390/mm/kasan_init.c                 |  2 +-
 arch/sh/boards/mach-ap325rxa/setup.c      |  2 +-
 arch/sh/boards/mach-ecovec24/setup.c      |  4 ++--
 arch/sh/boards/mach-kfr2r09/setup.c       |  2 +-
 arch/sh/boards/mach-migor/setup.c         |  2 +-
 arch/sh/boards/mach-se/7724/setup.c       |  4 ++--
 arch/sparc/kernel/smp_64.c                |  2 +-
 arch/um/kernel/mem.c                      |  2 +-
 arch/x86/kernel/setup.c                   |  4 ++--
 arch/x86/kernel/setup_percpu.c            |  2 +-
 arch/x86/mm/init.c                        |  2 +-
 arch/x86/mm/kasan_init_64.c               |  4 ++--
 arch/x86/mm/numa.c                        |  2 +-
 arch/x86/mm/numa_emulation.c              |  2 +-
 arch/x86/xen/mmu_pv.c                     |  6 +++---
 arch/x86/xen/p2m.c                        |  2 +-
 arch/x86/xen/setup.c                      |  6 +++---
 drivers/base/arch_numa.c                  | 10 ++++------
 drivers/firmware/efi/memmap.c             |  2 +-
 drivers/macintosh/smu.c                   |  2 +-
 drivers/of/kexec.c                        |  3 +--
 drivers/of/of_reserved_mem.c              |  5 +++--
 drivers/s390/char/sclp_early.c            |  2 +-
 drivers/usb/early/xhci-dbc.c              | 10 +++++-----
 drivers/xen/swiotlb-xen.c                 |  2 +-
 include/linux/memblock.h                  | 23 +++--------------------
 init/initramfs.c                          |  2 +-
 init/main.c                               |  2 +-
 kernel/dma/swiotlb.c                      |  2 +-
 kernel/printk/printk.c                    |  4 ++--
 lib/bootconfig.c                          |  2 +-
 lib/cpumask.c                             |  2 +-
 mm/cma.c                                  |  2 +-
 mm/memblock.c                             | 22 +++++++++++-----------
 mm/memory_hotplug.c                       |  2 +-
 mm/percpu.c                               |  8 ++++----
 mm/sparse.c                               |  2 +-
 54 files changed, 99 insertions(+), 119 deletions(-)


base-commit: 5816b3e6577eaa676ceb00a848f0fd65fe2adc29
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200196.354653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW191-0007od-9y; Thu, 30 Sep 2021 18:50:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200196.354653; Thu, 30 Sep 2021 18:50:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW191-0007oR-61; Thu, 30 Sep 2021 18:50:59 +0000
Received: by outflank-mailman (input) for mailman id 200196;
 Thu, 30 Sep 2021 18:50:57 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW18z-0007mo-EK
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:50:57 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 584b46dc-221f-11ec-bd6b-12813bfff9fa;
 Thu, 30 Sep 2021 18:50:56 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8841861209;
 Thu, 30 Sep 2021 18:50:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 584b46dc-221f-11ec-bd6b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027856;
	bh=UhBCHbOj9Jl5ZEx32c+R0M1PnzpsDKyF6CtjxvFScPo=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=c2tPiDJenxyQCvMzIY4Em8nsDs+rtaeSgB5mPhO6rdWaZxL8dx3plalQIb93rr+g7
	 VQ8cGTHdyEbFp89BNk1xmo9SM+kleGvnBNuMMGt+h3NC2iEs3mimxx37p8Rh1Q49I1
	 dICy9rhz40FtM07iwqUokDQc+WAsE9WW38PuHUYN59W32pQcFuBqF49AW2YbxvUmMe
	 aml+AvUr5+hgysic8DOhS2c2vDqm1VwkFyfbPUhqrlF8CS9WEcfvcWhLGpQ9MtlfVo
	 xwgzIBOzSTgMKLsybKBHHglY8ZfOkRoMe48nKFLqWW35isec2qKzGOkfCDbTRn1Osu
	 LXpN7CWI+EqJw==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 2/6] xen/x86: free_p2m_page: use memblock_free_ptr() to free a virtual pointer
Date: Thu, 30 Sep 2021 21:50:27 +0300
Message-Id: <20210930185031.18648-3-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

free_p2m_page() wrongly passes a virtual pointer to memblock_free() that
treats it as a physical address.

Call memblock_free_ptr() instead that gets a virtual address to free the
memory.

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/p2m.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index 5e6e236977c7..141bb9dbd2fb 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -197,7 +197,7 @@ static void * __ref alloc_p2m_page(void)
 static void __ref free_p2m_page(void *p)
 {
 	if (unlikely(!slab_is_available())) {
-		memblock_free((unsigned long)p, PAGE_SIZE);
+		memblock_free_ptr(p, PAGE_SIZE);
 		return;
 	}
 
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:10 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200195.354642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW18t-0007Rp-1z; Thu, 30 Sep 2021 18:50:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200195.354642; Thu, 30 Sep 2021 18:50:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW18s-0007Rg-Uk; Thu, 30 Sep 2021 18:50:50 +0000
Received: by outflank-mailman (input) for mailman id 200195;
 Thu, 30 Sep 2021 18:50:50 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW18s-0007RJ-7B
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:50:50 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8a372d29-5879-407e-8d92-235082b553ce;
 Thu, 30 Sep 2021 18:50:49 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8637261350;
 Thu, 30 Sep 2021 18:50:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a372d29-5879-407e-8d92-235082b553ce
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027849;
	bh=ISUQEAeabD8TFuzFwCipkjqJm558AV8oII+raBjMcR4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=s/nujFn9CMUdpbr325ijsunQ25k2M/i9/fhbz5LMb3fbAE+MyzkyTivG+OFV7pGA0
	 0mMCr9cL+6NMUzw1OueI7eDhpkmWrqSBHnTh+7Nay6CoC0XZhtahIjyxjuWdGb1TMY
	 1fRZO+c8HfzVCLDgD/FPrUdTCnminsC4OmaCiVgNI2Caxtuy3ub58zJW0FfPkLYG/4
	 R7PbZYxgMJggZ9+WvzrbboZwD/xf9+ZJDL5nQhjHdCwwqiZtN4l9t+qSPzTUbjwwk1
	 i6P2OXjm/YcqSpDPT9ALm+2hHHTDISEFV6snwAfjb4Ntj5CMY+VJ95ZP8rE0c/vFKI
	 9qAPHZmD3sE5A==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 1/6] arch_numa: simplify numa_distance allocation
Date: Thu, 30 Sep 2021 21:50:26 +0300
Message-Id: <20210930185031.18648-2-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Memory allocation of numa_distance uses memblock_phys_alloc_range() without
actual range limits, converts the returned physical address to virtual and
then only uses the virtual address for further initialization.

Simplify this by replacing memblock_phys_alloc_range() with
memblock_alloc().

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 drivers/base/arch_numa.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index 00fb4120a5b3..f6d0efd01188 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -275,15 +275,13 @@ void __init numa_free_distance(void)
 static int __init numa_alloc_distance(void)
 {
 	size_t size;
-	u64 phys;
 	int i, j;
 
 	size = nr_node_ids * nr_node_ids * sizeof(numa_distance[0]);
-	phys = memblock_phys_alloc_range(size, PAGE_SIZE, 0, PFN_PHYS(max_pfn));
-	if (WARN_ON(!phys))
+	numa_distance = memblock_alloc(size, PAGE_SIZE);
+	if (WARN_ON(!numa_distance))
 		return -ENOMEM;
 
-	numa_distance = __va(phys);
 	numa_distance_cnt = nr_node_ids;
 
 	/* fill with the default distances */
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:15 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200207.354663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19H-0000FK-Ip; Thu, 30 Sep 2021 18:51:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200207.354663; Thu, 30 Sep 2021 18:51:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19H-0000FB-Ft; Thu, 30 Sep 2021 18:51:15 +0000
Received: by outflank-mailman (input) for mailman id 200207;
 Thu, 30 Sep 2021 18:51:13 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW19F-0007ms-NW
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:51:13 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9916289a-fc09-4d5f-83b4-c2f22fb11a93;
 Thu, 30 Sep 2021 18:51:03 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 89FA66135D;
 Thu, 30 Sep 2021 18:50:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9916289a-fc09-4d5f-83b4-c2f22fb11a93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027863;
	bh=tivO3RxhX536I7wMbKedMKNTS477CjzFA9f2OsEnazU=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=rhARG/usd0caltO5GtB0juCqz5hY7xnPMr7IDQAFGz+nbzYYJBkUHPwMW10JpfG1U
	 GOjmrnB4JBMWYoWBbarrDNmo5G26FRM82TfHgLvs9xJG9DYeZ9xwwtdXHJOFYM1+zd
	 broKbyPnESumKB7LYEtfnlnXQTK9RJC7LFGyyQAFNET0qbO8Ui7UyAFT3z68DsrzPC
	 aSlcrogUj2KG46BRCUUxTr+2wozXgwfNdMvU4+UXXJEv8m/I8QC+L5fDpiVv8eqvrz
	 KJRENgprcqiCeomJ3lhZWrrHNbhnG7JAjUXbVddnNblNFrgbfFY2trb2oFu2cP7GBo
	 lH1ruowZhTR2g==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 3/6] memblock: drop memblock_free_early_nid() and memblock_free_early()
Date: Thu, 30 Sep 2021 21:50:28 +0300
Message-Id: <20210930185031.18648-4-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

memblock_free_early_nid() is unused and memblock_free_early() is an alias
for memblock_free().

Replace calls to memblock_free_early() with calls to memblock_free() and
remove memblock_free_early() and memblock_free_early_nid().

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 arch/mips/mm/init.c                  |  2 +-
 arch/powerpc/platforms/pseries/svm.c |  3 +--
 arch/s390/kernel/smp.c               |  2 +-
 drivers/base/arch_numa.c             |  2 +-
 drivers/s390/char/sclp_early.c       |  2 +-
 include/linux/memblock.h             | 12 ------------
 kernel/dma/swiotlb.c                 |  2 +-
 lib/cpumask.c                        |  2 +-
 mm/percpu.c                          |  8 ++++----
 mm/sparse.c                          |  2 +-
 10 files changed, 12 insertions(+), 25 deletions(-)

diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 19347dc6bbf8..21a5a7ac0037 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -529,7 +529,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index 87f001b4c4e4..f12229ce7301 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -56,8 +56,7 @@ void __init svm_swiotlb_init(void)
 		return;
 
 
-	memblock_free_early(__pa(vstart),
-			    PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+	memblock_free(__pa(vstart), PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
 	panic("SVM: Cannot allocate SWIOTLB buffer");
 }
 
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 1a04e5bdf655..066efd6d9345 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
 
 	/* Add CPUs present at boot */
 	__smp_rescan_cpus(info, true);
-	memblock_free_early((unsigned long)info, sizeof(*info));
+	memblock_free((unsigned long)info, sizeof(*info));
 }
 
 /*
diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index f6d0efd01188..e28d9dfe3c20 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -165,7 +165,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/drivers/s390/char/sclp_early.c b/drivers/s390/char/sclp_early.c
index f3d5c7f4c13d..f01d942e1c1d 100644
--- a/drivers/s390/char/sclp_early.c
+++ b/drivers/s390/char/sclp_early.c
@@ -139,7 +139,7 @@ int __init sclp_early_get_core_info(struct sclp_core_info *info)
 	}
 	sclp_fill_core_info(info, sccb);
 out:
-	memblock_free_early((unsigned long)sccb, length);
+	memblock_free((unsigned long)sccb, length);
 	return rc;
 }
 
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 34de69b3b8ba..fc8183be340c 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -441,18 +441,6 @@ static inline void *memblock_alloc_node(phys_addr_t size,
 				      MEMBLOCK_ALLOC_ACCESSIBLE, nid);
 }
 
-static inline void memblock_free_early(phys_addr_t base,
-					      phys_addr_t size)
-{
-	memblock_free(base, size);
-}
-
-static inline void memblock_free_early_nid(phys_addr_t base,
-						  phys_addr_t size, int nid)
-{
-	memblock_free(base, size);
-}
-
 static inline void memblock_free_late(phys_addr_t base, phys_addr_t size)
 {
 	__memblock_free_late(base, size);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 87c40517e822..430d2f78d540 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -247,7 +247,7 @@ swiotlb_init(int verbose)
 	return;
 
 fail_free_mem:
-	memblock_free_early(__pa(tlb), bytes);
+	memblock_free(__pa(tlb), bytes);
 fail:
 	pr_warn("Cannot allocate buffer");
 }
diff --git a/lib/cpumask.c b/lib/cpumask.c
index c3c76b833384..045779446a18 100644
--- a/lib/cpumask.c
+++ b/lib/cpumask.c
@@ -188,7 +188,7 @@ EXPORT_SYMBOL(free_cpumask_var);
  */
 void __init free_bootmem_cpumask_var(cpumask_var_t mask)
 {
-	memblock_free_early(__pa(mask), cpumask_size());
+	memblock_free(__pa(mask), cpumask_size());
 }
 #endif
 
diff --git a/mm/percpu.c b/mm/percpu.c
index e0a986818903..f58318cb04c0 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -2472,7 +2472,7 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
  */
 void __init pcpu_free_alloc_info(struct pcpu_alloc_info *ai)
 {
-	memblock_free_early(__pa(ai), ai->__ai_size);
+	memblock_free(__pa(ai), ai->__ai_size);
 }
 
 /**
@@ -3134,7 +3134,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 out_free:
 	pcpu_free_alloc_info(ai);
 	if (areas)
-		memblock_free_early(__pa(areas), areas_size);
+		memblock_free(__pa(areas), areas_size);
 	return rc;
 }
 #endif /* BUILD_EMBED_FIRST_CHUNK */
@@ -3256,7 +3256,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size,
 		free_fn(page_address(pages[j]), PAGE_SIZE);
 	rc = -ENOMEM;
 out_free_ar:
-	memblock_free_early(__pa(pages), pages_size);
+	memblock_free(__pa(pages), pages_size);
 	pcpu_free_alloc_info(ai);
 	return rc;
 }
@@ -3286,7 +3286,7 @@ static void * __init pcpu_dfl_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_dfl_fc_free(void *ptr, size_t size)
 {
-	memblock_free_early(__pa(ptr), size);
+	memblock_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/mm/sparse.c b/mm/sparse.c
index 120bc8ea5293..55fea0c2f927 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -451,7 +451,7 @@ static void *sparsemap_buf_end __meminitdata;
 static inline void __meminit sparse_buffer_free(unsigned long size)
 {
 	WARN_ON(!sparsemap_buf || size == 0);
-	memblock_free_early(__pa(sparsemap_buf), size);
+	memblock_free(__pa(sparsemap_buf), size);
 }
 
 static void __init sparse_buffer_init(unsigned long size, int nid)
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:26 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200211.354675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19R-0000kD-S8; Thu, 30 Sep 2021 18:51:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200211.354675; Thu, 30 Sep 2021 18:51:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19R-0000k2-PA; Thu, 30 Sep 2021 18:51:25 +0000
Received: by outflank-mailman (input) for mailman id 200211;
 Thu, 30 Sep 2021 18:51:23 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW19P-0007ms-Nj
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:51:23 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id acc106fe-a6e2-4e44-bcbd-ae935183d227;
 Thu, 30 Sep 2021 18:51:10 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8A2FA613CD;
 Thu, 30 Sep 2021 18:51:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acc106fe-a6e2-4e44-bcbd-ae935183d227
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027870;
	bh=t26przUqxCSGoBx41IMst/8wXJXoFwnOeTOXHswXAgQ=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=dXxprhJQZdZ6sk4rSn/rJZ0y8BsCGoLVOFNDP79gSG9bfzMuQ6Jj2iGJcEg84pkyp
	 M6Ch/waR2QQadlfi7FqEJZBx/jculiE7ratgds8uDW/Hl+kI8vNSqGY+Bnfe1jSZJa
	 bk55N5LaCDn5b7P2i2GzxYuHtQK/V6Lj6nQVhHYY3gwcoF1r53ooG6I5iCe7Q+3Fze
	 qmq4xeJ2HVjQ8xQvxiVjVvdkL5t/CKmwoGH0d06XFqf35691Wg6/f2+5qy96xPVOHK
	 nEt3pQk9SsCPQ7ksYP1gKGmSLs18lrKf5M2VN3dItzxzsbwE1uoM/EEg4NKM+HRtLx
	 2Csut9P5pc//w==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 4/6] memblock: stop aliasing __memblock_free_late with memblock_free_late
Date: Thu, 30 Sep 2021 21:50:29 +0300
Message-Id: <20210930185031.18648-5-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

memblock_free_late() is a NOP wrapper for __memblock_free_late(), there is
no point to keep this indirection.

Drop the wrapper and rename __memblock_free_late() to memblock_free_late().

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 include/linux/memblock.h | 7 +------
 mm/memblock.c            | 8 ++++----
 2 files changed, 5 insertions(+), 10 deletions(-)

diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index fc8183be340c..e25f964fdd60 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -133,7 +133,7 @@ void __next_mem_range_rev(u64 *idx, int nid, enum memblock_flags flags,
 			  struct memblock_type *type_b, phys_addr_t *out_start,
 			  phys_addr_t *out_end, int *out_nid);
 
-void __memblock_free_late(phys_addr_t base, phys_addr_t size);
+void memblock_free_late(phys_addr_t base, phys_addr_t size);
 
 #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
 static inline void __next_physmem_range(u64 *idx, struct memblock_type *type,
@@ -441,11 +441,6 @@ static inline void *memblock_alloc_node(phys_addr_t size,
 				      MEMBLOCK_ALLOC_ACCESSIBLE, nid);
 }
 
-static inline void memblock_free_late(phys_addr_t base, phys_addr_t size)
-{
-	__memblock_free_late(base, size);
-}
-
 /*
  * Set the allocation direction to bottom-up or top-down.
  */
diff --git a/mm/memblock.c b/mm/memblock.c
index 184dcd2e5d99..603f4a02be9b 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -366,14 +366,14 @@ void __init memblock_discard(void)
 		addr = __pa(memblock.reserved.regions);
 		size = PAGE_ALIGN(sizeof(struct memblock_region) *
 				  memblock.reserved.max);
-		__memblock_free_late(addr, size);
+		memblock_free_late(addr, size);
 	}
 
 	if (memblock.memory.regions != memblock_memory_init_regions) {
 		addr = __pa(memblock.memory.regions);
 		size = PAGE_ALIGN(sizeof(struct memblock_region) *
 				  memblock.memory.max);
-		__memblock_free_late(addr, size);
+		memblock_free_late(addr, size);
 	}
 
 	memblock_memory = NULL;
@@ -1586,7 +1586,7 @@ void * __init memblock_alloc_try_nid(
 }
 
 /**
- * __memblock_free_late - free pages directly to buddy allocator
+ * memblock_free_late - free pages directly to buddy allocator
  * @base: phys starting address of the  boot memory block
  * @size: size of the boot memory block in bytes
  *
@@ -1594,7 +1594,7 @@ void * __init memblock_alloc_try_nid(
  * down, but we are still initializing the system.  Pages are released directly
  * to the buddy allocator.
  */
-void __init __memblock_free_late(phys_addr_t base, phys_addr_t size)
+void __init memblock_free_late(phys_addr_t base, phys_addr_t size)
 {
 	phys_addr_t cursor, end;
 
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:51:28 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:51:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200212.354686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19U-00014X-6C; Thu, 30 Sep 2021 18:51:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200212.354686; Thu, 30 Sep 2021 18:51:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW19U-00014M-2G; Thu, 30 Sep 2021 18:51:28 +0000
Received: by outflank-mailman (input) for mailman id 200212;
 Thu, 30 Sep 2021 18:51:27 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW19T-00010t-09
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:51:27 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 69214d8a-221f-11ec-bd6b-12813bfff9fa;
 Thu, 30 Sep 2021 18:51:24 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id C9D6861209;
 Thu, 30 Sep 2021 18:51:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69214d8a-221f-11ec-bd6b-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027884;
	bh=JK9jLNwOn83tFpdzsz5SWNGjwtk9QqdCPfSb8ZeVHJ8=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=Bqq6vS5DF/5QPQwMSg/5caCykDp4qGu0WNz6OZuOntVpri/vaYk4DPD7qZBstqQpc
	 yBKMwZVZLHzYCS4qoZ2CHcBvuvSU82Rzb1xMSbQUdJ9phkjqqi5REj6uQASji/ETA/
	 lhvo4thwbHBCzfvMgpHCbChiFuLrNUKBRpS2P3wLxZhQ4PiADHmQtC5G+QBcZOAZWh
	 J0qJaKhs5xHST0xaxZtmkoJ11M7yzqr9KTdWJk8m11M5+3oDPbXQGANhne9SWTYDa/
	 znUyN54iw3g3seQ/KCoAhzChw/r8H2PuZS71dZ8b/Lc6wXAQ2y5oMw/SZez4Rcqqg9
	 PGa760fcOAZMg==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 6/6] memblock: use memblock_free for freeing virtual pointers
Date: Thu, 30 Sep 2021 21:50:31 +0300
Message-Id: <20210930185031.18648-7-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Rename memblock_free_ptr() to memblock_free() and use memblock_free()
when freeing a virtual pointer so that memblock_free() will be a
counterpart of memblock_alloc()

The callers are updated with the below semantic patch and manual addition
of (void *) casting to pointers that are represented by unsigned long
variables.

@@
identifier vaddr;
expression size;
@@
(
- memblock_phys_free(__pa(vaddr), size);
+ memblock_free(vaddr, size);
|
- memblock_free_ptr(vaddr, size);
+ memblock_free(vaddr, size);
)

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 arch/alpha/kernel/core_irongate.c         | 3 +--
 arch/mips/mm/init.c                       | 2 +-
 arch/powerpc/kernel/dt_cpu_ftrs.c         | 4 ++--
 arch/powerpc/kernel/setup-common.c        | 2 +-
 arch/powerpc/kernel/setup_64.c            | 2 +-
 arch/powerpc/platforms/powernv/pci-ioda.c | 2 +-
 arch/powerpc/platforms/pseries/svm.c      | 3 +--
 arch/riscv/kernel/setup.c                 | 5 ++---
 arch/sparc/kernel/smp_64.c                | 2 +-
 arch/um/kernel/mem.c                      | 2 +-
 arch/x86/kernel/setup_percpu.c            | 2 +-
 arch/x86/mm/kasan_init_64.c               | 4 ++--
 arch/x86/mm/numa.c                        | 2 +-
 arch/x86/mm/numa_emulation.c              | 2 +-
 arch/x86/xen/mmu_pv.c                     | 2 +-
 arch/x86/xen/p2m.c                        | 2 +-
 drivers/base/arch_numa.c                  | 4 ++--
 drivers/macintosh/smu.c                   | 2 +-
 drivers/xen/swiotlb-xen.c                 | 2 +-
 include/linux/memblock.h                  | 2 +-
 init/initramfs.c                          | 2 +-
 init/main.c                               | 2 +-
 kernel/dma/swiotlb.c                      | 2 +-
 kernel/printk/printk.c                    | 4 ++--
 lib/bootconfig.c                          | 2 +-
 lib/cpumask.c                             | 2 +-
 mm/memblock.c                             | 6 +++---
 mm/percpu.c                               | 8 ++++----
 mm/sparse.c                               | 2 +-
 29 files changed, 39 insertions(+), 42 deletions(-)

diff --git a/arch/alpha/kernel/core_irongate.c b/arch/alpha/kernel/core_irongate.c
index ee26dcc49418..6b8ed12936b6 100644
--- a/arch/alpha/kernel/core_irongate.c
+++ b/arch/alpha/kernel/core_irongate.c
@@ -233,8 +233,7 @@ albacore_init_arch(void)
 			unsigned long size;
 
 			size = initrd_end - initrd_start;
-			memblock_phys_free(__pa(initrd_start),
-					   PAGE_ALIGN(size));
+			memblock_free((void *)initrd_start, PAGE_ALIGN(size));
 			if (!move_initrd(pci_mem))
 				printk("irongate_init_arch: initrd too big "
 				       "(%ldK)\ndisabling initrd\n",
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 3be1c29084fa..325e1552cbea 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -529,7 +529,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_phys_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
index 42839d6bd486..ba527fb52993 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -1095,8 +1095,8 @@ static int __init dt_cpu_ftrs_scan_callback(unsigned long node, const char
 
 	cpufeatures_setup_finished();
 
-	memblock_phys_free(__pa(dt_cpu_features),
-			   sizeof(struct dt_cpu_feature) * nr_dt_cpu_features);
+	memblock_free(dt_cpu_features,
+		      sizeof(struct dt_cpu_feature) * nr_dt_cpu_features);
 
 	return 0;
 }
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 5af8993a8e6d..6b1338db8779 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -825,7 +825,7 @@ static void __init smp_setup_pacas(void)
 		set_hard_smp_processor_id(cpu, cpu_to_phys_id[cpu]);
 	}
 
-	memblock_phys_free(__pa(cpu_to_phys_id), nr_cpu_ids * sizeof(u32));
+	memblock_free(cpu_to_phys_id, nr_cpu_ids * sizeof(u32));
 	cpu_to_phys_id = NULL;
 }
 #endif
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index 75bc294ac40d..1777e992b20b 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -812,7 +812,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_phys_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 static int pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index b5a9d343b720..004cd6a96c8a 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2981,7 +2981,7 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 	if (!phb->hose) {
 		pr_err("  Can't allocate PCI controller for %pOF\n",
 		       np);
-		memblock_phys_free(__pa(phb), sizeof(struct pnv_phb));
+		memblock_free(phb, sizeof(struct pnv_phb));
 		return;
 	}
 
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index b7c017bb40f7..6332365d2891 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -56,8 +56,7 @@ void __init svm_swiotlb_init(void)
 		return;
 
 
-	memblock_phys_free(__pa(vstart),
-			   PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+	memblock_free(vstart, PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
 	panic("SVM: Cannot allocate SWIOTLB buffer");
 }
 
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 6ea7c53b82cd..b42bfdc67482 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -230,14 +230,13 @@ static void __init init_resources(void)
 
 	/* Clean-up any unused pre-allocated resources */
 	if (res_idx >= 0)
-		memblock_phys_free(__pa(mem_res),
-				   (res_idx + 1) * sizeof(*mem_res));
+		memblock_free(mem_res, (res_idx + 1) * sizeof(*mem_res));
 	return;
 
  error:
 	/* Better an empty resource tree than an inconsistent one */
 	release_child_resources(&iomem_resource);
-	memblock_phys_free(__pa(mem_res), mem_res_sz);
+	memblock_free(mem_res, mem_res_sz);
 }
 
 
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 2507549538df..b98a7bbe6728 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -1567,7 +1567,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_phys_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 static int __init pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c
index d1710ebb44f4..0039771eb01c 100644
--- a/arch/um/kernel/mem.c
+++ b/arch/um/kernel/mem.c
@@ -47,7 +47,7 @@ void __init mem_init(void)
 	 */
 	brk_end = (unsigned long) UML_ROUND_UP(sbrk(0));
 	map_memory(brk_end, __pa(brk_end), uml_reserved - brk_end, 1, 1, 0);
-	memblock_phys_free(__pa(brk_end), uml_reserved - brk_end);
+	memblock_free((void *)brk_end, uml_reserved - brk_end);
 	uml_reserved = brk_end;
 
 	/* this will put all low memory onto the freelists */
diff --git a/arch/x86/kernel/setup_percpu.c b/arch/x86/kernel/setup_percpu.c
index 5afd98559193..7b65275544b2 100644
--- a/arch/x86/kernel/setup_percpu.c
+++ b/arch/x86/kernel/setup_percpu.c
@@ -135,7 +135,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size, size_t align)
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free_ptr(ptr, size);
+	memblock_free(ptr, size);
 }
 
 static int __init pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index ef885370719a..e7b9b464a82f 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -49,7 +49,7 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr,
 			p = early_alloc(PMD_SIZE, nid, false);
 			if (p && pmd_set_huge(pmd, __pa(p), PAGE_KERNEL))
 				return;
-			memblock_free_ptr(p, PMD_SIZE);
+			memblock_free(p, PMD_SIZE);
 		}
 
 		p = early_alloc(PAGE_SIZE, nid, true);
@@ -85,7 +85,7 @@ static void __init kasan_populate_pud(pud_t *pud, unsigned long addr,
 			p = early_alloc(PUD_SIZE, nid, false);
 			if (p && pud_set_huge(pud, __pa(p), PAGE_KERNEL))
 				return;
-			memblock_free_ptr(p, PUD_SIZE);
+			memblock_free(p, PUD_SIZE);
 		}
 
 		p = early_alloc(PAGE_SIZE, nid, true);
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index 1e9b93b088db..c6b1213086d6 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -355,7 +355,7 @@ void __init numa_reset_distance(void)
 
 	/* numa_distance could be 1LU marking allocation failure, test cnt */
 	if (numa_distance_cnt)
-		memblock_free_ptr(numa_distance, size);
+		memblock_free(numa_distance, size);
 	numa_distance_cnt = 0;
 	numa_distance = NULL;	/* enable table creation */
 }
diff --git a/arch/x86/mm/numa_emulation.c b/arch/x86/mm/numa_emulation.c
index e801e30089c4..1a02b791d273 100644
--- a/arch/x86/mm/numa_emulation.c
+++ b/arch/x86/mm/numa_emulation.c
@@ -517,7 +517,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 	}
 
 	/* free the copied physical distance table */
-	memblock_free_ptr(phys_dist, phys_size);
+	memblock_free(phys_dist, phys_size);
 	return;
 
 no_emu:
diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 3500b22ff087..0c831ad78933 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1151,7 +1151,7 @@ static void __init xen_pagetable_p2m_free(void)
 		xen_cleanhighmap(addr, addr + size);
 		size = PAGE_ALIGN(xen_start_info->nr_pages *
 				  sizeof(unsigned long));
-		memblock_phys_free(__pa(addr), size);
+		memblock_free((void *)addr, size);
 	} else {
 		xen_cleanmfnmap(addr);
 	}
diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index 141bb9dbd2fb..58db86f7b384 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -197,7 +197,7 @@ static void * __ref alloc_p2m_page(void)
 static void __ref free_p2m_page(void *p)
 {
 	if (unlikely(!slab_is_available())) {
-		memblock_free_ptr(p, PAGE_SIZE);
+		memblock_free(p, PAGE_SIZE);
 		return;
 	}
 
diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index 7974913af09c..cad34cec3acc 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -165,7 +165,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_phys_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
@@ -264,7 +264,7 @@ void __init numa_free_distance(void)
 	size = numa_distance_cnt * numa_distance_cnt *
 		sizeof(numa_distance[0]);
 
-	memblock_free_ptr(numa_distance, size);
+	memblock_free(numa_distance, size);
 	numa_distance_cnt = 0;
 	numa_distance = NULL;
 }
diff --git a/drivers/macintosh/smu.c b/drivers/macintosh/smu.c
index fe63d5ee201b..f62152111236 100644
--- a/drivers/macintosh/smu.c
+++ b/drivers/macintosh/smu.c
@@ -570,7 +570,7 @@ int __init smu_init (void)
 fail_db_node:
 	of_node_put(smu->db_node);
 fail_bootmem:
-	memblock_free_ptr(smu, sizeof(struct smu_device));
+	memblock_free(smu, sizeof(struct smu_device));
 	smu = NULL;
 fail_np:
 	of_node_put(np);
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 4b671cc0a7ea..f083194e2634 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -241,7 +241,7 @@ void __init xen_swiotlb_init_early(void)
 	 */
 	rc = xen_swiotlb_fixup(start, nslabs);
 	if (rc) {
-		memblock_phys_free(__pa(start), PAGE_ALIGN(bytes));
+		memblock_free(start, PAGE_ALIGN(bytes));
 		if (nslabs > 1024 && repeat--) {
 			/* Min is 2MB */
 			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index d32d41709513..484650681bee 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -118,7 +118,7 @@ int memblock_mark_nomap(phys_addr_t base, phys_addr_t size);
 int memblock_clear_nomap(phys_addr_t base, phys_addr_t size);
 
 void memblock_free_all(void);
-void memblock_free_ptr(void *ptr, size_t size);
+void memblock_free(void *ptr, size_t size);
 void reset_node_managed_pages(pg_data_t *pgdat);
 void reset_all_zones_managed_pages(void);
 
diff --git a/init/initramfs.c b/init/initramfs.c
index 1a971f070dd4..2f3d96dc3db6 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -607,7 +607,7 @@ void __weak __init free_initrd_mem(unsigned long start, unsigned long end)
 	unsigned long aligned_start = ALIGN_DOWN(start, PAGE_SIZE);
 	unsigned long aligned_end = ALIGN(end, PAGE_SIZE);
 
-	memblock_phys_free(__pa(aligned_start), aligned_end - aligned_start);
+	memblock_free((void *)aligned_start, aligned_end - aligned_start);
 #endif
 
 	free_reserved_area((void *)start, (void *)end, POISON_FREE_INITMEM,
diff --git a/init/main.c b/init/main.c
index 81a79a77db46..2bfcf9054f8e 100644
--- a/init/main.c
+++ b/init/main.c
@@ -924,7 +924,7 @@ static void __init print_unknown_bootoptions(void)
 		end += sprintf(end, " %s", *p);
 
 	pr_notice("Unknown command line parameters:%s\n", unknown_options);
-	memblock_free_ptr(unknown_options, len);
+	memblock_free(unknown_options, len);
 }
 
 asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index b9fa173e5e56..02656d7ccbfd 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -247,7 +247,7 @@ swiotlb_init(int verbose)
 	return;
 
 fail_free_mem:
-	memblock_phys_free(__pa(tlb), bytes);
+	memblock_free(tlb, bytes);
 fail:
 	pr_warn("Cannot allocate buffer");
 }
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index a8d0a58deebc..2cae1bfa6be7 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1166,9 +1166,9 @@ void __init setup_log_buf(int early)
 	return;
 
 err_free_descs:
-	memblock_free_ptr(new_descs, new_descs_size);
+	memblock_free(new_descs, new_descs_size);
 err_free_log_buf:
-	memblock_free_ptr(new_log_buf, new_log_buf_len);
+	memblock_free(new_log_buf, new_log_buf_len);
 }
 
 static bool __read_mostly ignore_loglevel;
diff --git a/lib/bootconfig.c b/lib/bootconfig.c
index 5ae248b29373..547558d80e64 100644
--- a/lib/bootconfig.c
+++ b/lib/bootconfig.c
@@ -792,7 +792,7 @@ void __init xbc_destroy_all(void)
 	xbc_data = NULL;
 	xbc_data_size = 0;
 	xbc_node_num = 0;
-	memblock_free_ptr(xbc_nodes, sizeof(struct xbc_node) * XBC_NODE_MAX);
+	memblock_free(xbc_nodes, sizeof(struct xbc_node) * XBC_NODE_MAX);
 	xbc_nodes = NULL;
 	brace_index = 0;
 }
diff --git a/lib/cpumask.c b/lib/cpumask.c
index a90786b77c1c..a971a82d2f43 100644
--- a/lib/cpumask.c
+++ b/lib/cpumask.c
@@ -188,7 +188,7 @@ EXPORT_SYMBOL(free_cpumask_var);
  */
 void __init free_bootmem_cpumask_var(cpumask_var_t mask)
 {
-	memblock_phys_free(__pa(mask), cpumask_size());
+	memblock_free(mask, cpumask_size());
 }
 #endif
 
diff --git a/mm/memblock.c b/mm/memblock.c
index a23baa482f3f..3b0def459f97 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -472,7 +472,7 @@ static int __init_memblock memblock_double_array(struct memblock_type *type,
 		kfree(old_array);
 	else if (old_array != memblock_memory_init_regions &&
 		 old_array != memblock_reserved_init_regions)
-		memblock_free_ptr(old_array, old_alloc_size);
+		memblock_free(old_array, old_alloc_size);
 
 	/*
 	 * Reserve the new array if that comes from the memblock.  Otherwise, we
@@ -796,14 +796,14 @@ int __init_memblock memblock_remove(phys_addr_t base, phys_addr_t size)
 }
 
 /**
- * memblock_free_ptr - free boot memory allocation
+ * memblock_free - free boot memory allocation
  * @ptr: starting address of the  boot memory allocation
  * @size: size of the boot memory block in bytes
  *
  * Free boot memory block previously allocated by memblock_alloc_xx() API.
  * The freeing memory will not be released to the buddy allocator.
  */
-void __init_memblock memblock_free_ptr(void *ptr, size_t size)
+void __init_memblock memblock_free(void *ptr, size_t size)
 {
 	if (ptr)
 		memblock_phys_free(__pa(ptr), size);
diff --git a/mm/percpu.c b/mm/percpu.c
index d65ddf6f2a35..f5b2c2ea5a54 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -2472,7 +2472,7 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
  */
 void __init pcpu_free_alloc_info(struct pcpu_alloc_info *ai)
 {
-	memblock_phys_free(__pa(ai), ai->__ai_size);
+	memblock_free(ai, ai->__ai_size);
 }
 
 /**
@@ -3134,7 +3134,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 out_free:
 	pcpu_free_alloc_info(ai);
 	if (areas)
-		memblock_phys_free(__pa(areas), areas_size);
+		memblock_free(areas, areas_size);
 	return rc;
 }
 #endif /* BUILD_EMBED_FIRST_CHUNK */
@@ -3256,7 +3256,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size,
 		free_fn(page_address(pages[j]), PAGE_SIZE);
 	rc = -ENOMEM;
 out_free_ar:
-	memblock_phys_free(__pa(pages), pages_size);
+	memblock_free(pages, pages_size);
 	pcpu_free_alloc_info(ai);
 	return rc;
 }
@@ -3286,7 +3286,7 @@ static void * __init pcpu_dfl_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_dfl_fc_free(void *ptr, size_t size)
 {
-	memblock_phys_free(__pa(ptr), size);
+	memblock_free(ptr, size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/mm/sparse.c b/mm/sparse.c
index fc3ab8d3b6bc..e5c84b0cf0c9 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -451,7 +451,7 @@ static void *sparsemap_buf_end __meminitdata;
 static inline void __meminit sparse_buffer_free(unsigned long size)
 {
 	WARN_ON(!sparsemap_buf || size == 0);
-	memblock_phys_free(__pa(sparsemap_buf), size);
+	memblock_free(sparsemap_buf, size);
 }
 
 static void __init sparse_buffer_init(unsigned long size, int nid)
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 18:57:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 18:57:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200236.354696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW1FN-0002gZ-1R; Thu, 30 Sep 2021 18:57:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200236.354696; Thu, 30 Sep 2021 18:57:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW1FM-0002gS-Uf; Thu, 30 Sep 2021 18:57:32 +0000
Received: by outflank-mailman (input) for mailman id 200236;
 Thu, 30 Sep 2021 18:57:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW19o-0007ms-OF
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 18:51:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 9df0f022-b9fc-4b3b-8865-b058f1577be3;
 Thu, 30 Sep 2021 18:51:17 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 8F2C1613CE;
 Thu, 30 Sep 2021 18:51:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9df0f022-b9fc-4b3b-8865-b058f1577be3
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633027877;
	bh=vvw/CX97uJRYq4lw4+eUMmguzmclRo38E917kdICwSA=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=CqsrMZIj1zkE+0A9Ivwfz/cRiqIG1V7tATijS22rkmV6/NeZl7eVMUO2JvPZO7tex
	 kgV2n0FgnpGGuyEmDPWjDVT1XdopqBt/42N0NbSVzKv9hj5RDfZMVZNhbAVQST7+oB
	 3+hRfHC1RZWtlQn9nTBMXWNFg1vQNd2BoPzrbIZpUdZXXBTxrLQ1JkeWLJstltMXAs
	 dCnr8bwBvWL/vJLjYcVcwORnAqedFR90VvQPxGnwzpD67NyULUJbfjmvKfcCQo54eB
	 gJsCrX0oPS8F5J+BYjZi9RrczVTMx9rjSEGzFA45vmX8uYaD2/vz4OXp0YzzlUnrx2
	 GyiE8thi4nLMA==
From: Mike Rapoport <rppt@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Mike Rapoport <rppt@kernel.org>,
	Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree@vger.kernel.org,
	iommu@lists.linux-foundation.org,
	kasan-dev@googlegroups.com,
	kvm@vger.kernel.org,
	linux-alpha@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-efi@vger.kernel.org,
	linux-mips@vger.kernel.org,
	linux-mm@kvack.org,
	linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org,
	linux-sh@vger.kernel.org,
	linux-snps-arc@lists.infradead.org,
	linux-um@lists.infradead.org,
	linux-usb@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org,
	sparclinux@vger.kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH v2 5/6] memblock: rename memblock_free to memblock_phys_free
Date: Thu, 30 Sep 2021 21:50:30 +0300
Message-Id: <20210930185031.18648-6-rppt@kernel.org>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Mike Rapoport <rppt@linux.ibm.com>

Since memblock_free() operates on a physical range, make its name reflect
it and rename it to memblock_phys_free(), so it will be a logical
counterpart to memblock_phys_alloc().

The callers are updated with the below semantic patch:

@@
expression addr;
expression size;
@@
- memblock_free(addr, size);
+ memblock_phys_free(addr, size);

Signed-off-by: Mike Rapoport <rppt@linux.ibm.com>
---
 arch/alpha/kernel/core_irongate.c         |  3 ++-
 arch/arc/mm/init.c                        |  2 +-
 arch/arm/mach-hisi/platmcpm.c             |  2 +-
 arch/arm/mm/init.c                        |  2 +-
 arch/arm64/mm/mmu.c                       |  4 ++--
 arch/mips/mm/init.c                       |  2 +-
 arch/mips/sgi-ip30/ip30-setup.c           |  6 +++---
 arch/powerpc/kernel/dt_cpu_ftrs.c         |  4 ++--
 arch/powerpc/kernel/paca.c                |  8 ++++----
 arch/powerpc/kernel/setup-common.c        |  2 +-
 arch/powerpc/kernel/setup_64.c            |  2 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |  2 +-
 arch/powerpc/platforms/pseries/svm.c      |  3 ++-
 arch/riscv/kernel/setup.c                 |  5 +++--
 arch/s390/kernel/setup.c                  |  8 ++++----
 arch/s390/kernel/smp.c                    |  4 ++--
 arch/s390/kernel/uv.c                     |  2 +-
 arch/s390/mm/kasan_init.c                 |  2 +-
 arch/sh/boards/mach-ap325rxa/setup.c      |  2 +-
 arch/sh/boards/mach-ecovec24/setup.c      |  4 ++--
 arch/sh/boards/mach-kfr2r09/setup.c       |  2 +-
 arch/sh/boards/mach-migor/setup.c         |  2 +-
 arch/sh/boards/mach-se/7724/setup.c       |  4 ++--
 arch/sparc/kernel/smp_64.c                |  2 +-
 arch/um/kernel/mem.c                      |  2 +-
 arch/x86/kernel/setup.c                   |  4 ++--
 arch/x86/mm/init.c                        |  2 +-
 arch/x86/xen/mmu_pv.c                     |  6 +++---
 arch/x86/xen/setup.c                      |  6 +++---
 drivers/base/arch_numa.c                  |  2 +-
 drivers/firmware/efi/memmap.c             |  2 +-
 drivers/of/kexec.c                        |  3 +--
 drivers/of/of_reserved_mem.c              |  5 +++--
 drivers/s390/char/sclp_early.c            |  2 +-
 drivers/usb/early/xhci-dbc.c              | 10 +++++-----
 drivers/xen/swiotlb-xen.c                 |  2 +-
 include/linux/memblock.h                  |  2 +-
 init/initramfs.c                          |  2 +-
 kernel/dma/swiotlb.c                      |  2 +-
 lib/cpumask.c                             |  2 +-
 mm/cma.c                                  |  2 +-
 mm/memblock.c                             |  8 ++++----
 mm/memory_hotplug.c                       |  2 +-
 mm/percpu.c                               |  8 ++++----
 mm/sparse.c                               |  2 +-
 45 files changed, 79 insertions(+), 76 deletions(-)

diff --git a/arch/alpha/kernel/core_irongate.c b/arch/alpha/kernel/core_irongate.c
index 72af1e72d833..ee26dcc49418 100644
--- a/arch/alpha/kernel/core_irongate.c
+++ b/arch/alpha/kernel/core_irongate.c
@@ -233,7 +233,8 @@ albacore_init_arch(void)
 			unsigned long size;
 
 			size = initrd_end - initrd_start;
-			memblock_free(__pa(initrd_start), PAGE_ALIGN(size));
+			memblock_phys_free(__pa(initrd_start),
+					   PAGE_ALIGN(size));
 			if (!move_initrd(pci_mem))
 				printk("irongate_init_arch: initrd too big "
 				       "(%ldK)\ndisabling initrd\n",
diff --git a/arch/arc/mm/init.c b/arch/arc/mm/init.c
index 699ecf119641..59408f6a02d4 100644
--- a/arch/arc/mm/init.c
+++ b/arch/arc/mm/init.c
@@ -173,7 +173,7 @@ static void __init highmem_init(void)
 #ifdef CONFIG_HIGHMEM
 	unsigned long tmp;
 
-	memblock_free(high_mem_start, high_mem_sz);
+	memblock_phys_free(high_mem_start, high_mem_sz);
 	for (tmp = min_high_pfn; tmp < max_high_pfn; tmp++)
 		free_highmem_page(pfn_to_page(tmp));
 #endif
diff --git a/arch/arm/mach-hisi/platmcpm.c b/arch/arm/mach-hisi/platmcpm.c
index 96a484095194..258586e31333 100644
--- a/arch/arm/mach-hisi/platmcpm.c
+++ b/arch/arm/mach-hisi/platmcpm.c
@@ -339,7 +339,7 @@ static int __init hip04_smp_init(void)
 err_sysctrl:
 	iounmap(relocation);
 err_reloc:
-	memblock_free(hip04_boot_method[0], hip04_boot_method[1]);
+	memblock_phys_free(hip04_boot_method[0], hip04_boot_method[1]);
 err:
 	return ret;
 }
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 6162a070a410..6d0cb0f7bc54 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -158,7 +158,7 @@ phys_addr_t __init arm_memblock_steal(phys_addr_t size, phys_addr_t align)
 		panic("Failed to steal %pa bytes at %pS\n",
 		      &size, (void *)_RET_IP_);
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	return phys;
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index cfd9deb347c3..f68c2d953617 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -738,8 +738,8 @@ void __init paging_init(void)
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
 	init_mm.pgd = swapper_pg_dir;
 
-	memblock_free(__pa_symbol(init_pg_dir),
-		      __pa_symbol(init_pg_end) - __pa_symbol(init_pg_dir));
+	memblock_phys_free(__pa_symbol(init_pg_dir),
+			   __pa_symbol(init_pg_end) - __pa_symbol(init_pg_dir));
 
 	memblock_allow_resize();
 }
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 21a5a7ac0037..3be1c29084fa 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -529,7 +529,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_phys_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/arch/mips/sgi-ip30/ip30-setup.c b/arch/mips/sgi-ip30/ip30-setup.c
index 44b1607e964d..75a34684e704 100644
--- a/arch/mips/sgi-ip30/ip30-setup.c
+++ b/arch/mips/sgi-ip30/ip30-setup.c
@@ -69,10 +69,10 @@ static void __init ip30_mem_init(void)
 		total_mem += size;
 
 		if (addr >= IP30_REAL_MEMORY_START)
-			memblock_free(addr, size);
+			memblock_phys_free(addr, size);
 		else if ((addr + size) > IP30_REAL_MEMORY_START)
-			memblock_free(IP30_REAL_MEMORY_START,
-				     size - IP30_MAX_PROM_MEMORY);
+			memblock_phys_free(IP30_REAL_MEMORY_START,
+					   size - IP30_MAX_PROM_MEMORY);
 	}
 	pr_info("Detected %luMB of physical memory.\n", MEM_SHIFT(total_mem));
 }
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
index 358aee7c2d79..42839d6bd486 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -1095,8 +1095,8 @@ static int __init dt_cpu_ftrs_scan_callback(unsigned long node, const char
 
 	cpufeatures_setup_finished();
 
-	memblock_free(__pa(dt_cpu_features),
-			sizeof(struct dt_cpu_feature)*nr_dt_cpu_features);
+	memblock_phys_free(__pa(dt_cpu_features),
+			   sizeof(struct dt_cpu_feature) * nr_dt_cpu_features);
 
 	return 0;
 }
diff --git a/arch/powerpc/kernel/paca.c b/arch/powerpc/kernel/paca.c
index 9bd30cac852b..4208b4044d12 100644
--- a/arch/powerpc/kernel/paca.c
+++ b/arch/powerpc/kernel/paca.c
@@ -322,8 +322,8 @@ void __init free_unused_pacas(void)
 
 	new_ptrs_size = sizeof(struct paca_struct *) * nr_cpu_ids;
 	if (new_ptrs_size < paca_ptrs_size)
-		memblock_free(__pa(paca_ptrs) + new_ptrs_size,
-					paca_ptrs_size - new_ptrs_size);
+		memblock_phys_free(__pa(paca_ptrs) + new_ptrs_size,
+				   paca_ptrs_size - new_ptrs_size);
 
 	paca_nr_cpu_ids = nr_cpu_ids;
 	paca_ptrs_size = new_ptrs_size;
@@ -331,8 +331,8 @@ void __init free_unused_pacas(void)
 #ifdef CONFIG_PPC_BOOK3S_64
 	if (early_radix_enabled()) {
 		/* Ugly fixup, see new_slb_shadow() */
-		memblock_free(__pa(paca_ptrs[boot_cpuid]->slb_shadow_ptr),
-				sizeof(struct slb_shadow));
+		memblock_phys_free(__pa(paca_ptrs[boot_cpuid]->slb_shadow_ptr),
+				   sizeof(struct slb_shadow));
 		paca_ptrs[boot_cpuid]->slb_shadow_ptr = NULL;
 	}
 #endif
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index b1e43b69a559..5af8993a8e6d 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -825,7 +825,7 @@ static void __init smp_setup_pacas(void)
 		set_hard_smp_processor_id(cpu, cpu_to_phys_id[cpu]);
 	}
 
-	memblock_free(__pa(cpu_to_phys_id), nr_cpu_ids * sizeof(u32));
+	memblock_phys_free(__pa(cpu_to_phys_id), nr_cpu_ids * sizeof(u32));
 	cpu_to_phys_id = NULL;
 }
 #endif
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index eaa79a0996d1..75bc294ac40d 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -812,7 +812,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_phys_free(__pa(ptr), size);
 }
 
 static int pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 3dd35c327d1c..b5a9d343b720 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2981,7 +2981,7 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 	if (!phb->hose) {
 		pr_err("  Can't allocate PCI controller for %pOF\n",
 		       np);
-		memblock_free(__pa(phb), sizeof(struct pnv_phb));
+		memblock_phys_free(__pa(phb), sizeof(struct pnv_phb));
 		return;
 	}
 
diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/pseries/svm.c
index f12229ce7301..b7c017bb40f7 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -56,7 +56,8 @@ void __init svm_swiotlb_init(void)
 		return;
 
 
-	memblock_free(__pa(vstart), PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+	memblock_phys_free(__pa(vstart),
+			   PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
 	panic("SVM: Cannot allocate SWIOTLB buffer");
 }
 
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index b9620e5f00ba..6ea7c53b82cd 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -230,13 +230,14 @@ static void __init init_resources(void)
 
 	/* Clean-up any unused pre-allocated resources */
 	if (res_idx >= 0)
-		memblock_free(__pa(mem_res), (res_idx + 1) * sizeof(*mem_res));
+		memblock_phys_free(__pa(mem_res),
+				   (res_idx + 1) * sizeof(*mem_res));
 	return;
 
  error:
 	/* Better an empty resource tree than an inconsistent one */
 	release_child_resources(&iomem_resource);
-	memblock_free(__pa(mem_res), mem_res_sz);
+	memblock_phys_free(__pa(mem_res), mem_res_sz);
 }
 
 
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 67e5fff96ee0..7fc836e9e194 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -693,7 +693,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	if (register_memory_notifier(&kdump_mem_nb)) {
-		memblock_free(crash_base, crash_size);
+		memblock_phys_free(crash_base, crash_size);
 		return;
 	}
 
@@ -748,7 +748,7 @@ static void __init free_mem_detect_info(void)
 
 	get_mem_detect_reserved(&start, &size);
 	if (size)
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 }
 
 static const char * __init get_mem_info_source(void)
@@ -793,7 +793,7 @@ static void __init check_initrd(void)
 	if (initrd_data.start && initrd_data.size &&
 	    !memblock_is_region_memory(initrd_data.start, initrd_data.size)) {
 		pr_err("The initial RAM disk does not fit into the memory\n");
-		memblock_free(initrd_data.start, initrd_data.size);
+		memblock_phys_free(initrd_data.start, initrd_data.size);
 		initrd_start = initrd_end = 0;
 	}
 #endif
@@ -890,7 +890,7 @@ static void __init setup_randomness(void)
 
 	if (stsi(vmms, 3, 2, 2) == 0 && vmms->count)
 		add_device_randomness(&vmms->vm, sizeof(vmms->vm[0]) * vmms->count);
-	memblock_free((unsigned long) vmms, PAGE_SIZE);
+	memblock_phys_free((unsigned long)vmms, PAGE_SIZE);
 }
 
 /*
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 066efd6d9345..78a8ea6fd582 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -723,7 +723,7 @@ void __init smp_save_dump_cpus(void)
 			/* Get the CPU registers */
 			smp_save_cpu_regs(sa, addr, is_boot_cpu, page);
 	}
-	memblock_free(page, PAGE_SIZE);
+	memblock_phys_free(page, PAGE_SIZE);
 	diag_amode31_ops.diag308_reset();
 	pcpu_set_smt(0);
 }
@@ -880,7 +880,7 @@ void __init smp_detect_cpus(void)
 
 	/* Add CPUs present at boot */
 	__smp_rescan_cpus(info, true);
-	memblock_free((unsigned long)info, sizeof(*info));
+	memblock_phys_free((unsigned long)info, sizeof(*info));
 }
 
 /*
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 5a656c7b7a67..d57457b16fe5 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -64,7 +64,7 @@ void __init setup_uv(void)
 	}
 
 	if (uv_init(uv_stor_base, uv_info.uv_base_stor_len)) {
-		memblock_free(uv_stor_base, uv_info.uv_base_stor_len);
+		memblock_phys_free(uv_stor_base, uv_info.uv_base_stor_len);
 		goto fail;
 	}
 
diff --git a/arch/s390/mm/kasan_init.c b/arch/s390/mm/kasan_init.c
index 3e4735168019..483b9dbe0970 100644
--- a/arch/s390/mm/kasan_init.c
+++ b/arch/s390/mm/kasan_init.c
@@ -399,5 +399,5 @@ void __init kasan_copy_shadow_mapping(void)
 
 void __init kasan_free_early_identity(void)
 {
-	memblock_free(pgalloc_pos, pgalloc_freeable - pgalloc_pos);
+	memblock_phys_free(pgalloc_pos, pgalloc_freeable - pgalloc_pos);
 }
diff --git a/arch/sh/boards/mach-ap325rxa/setup.c b/arch/sh/boards/mach-ap325rxa/setup.c
index bac8a058ebd7..c77b5f00a66a 100644
--- a/arch/sh/boards/mach-ap325rxa/setup.c
+++ b/arch/sh/boards/mach-ap325rxa/setup.c
@@ -560,7 +560,7 @@ static void __init ap325rxa_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-ecovec24/setup.c b/arch/sh/boards/mach-ecovec24/setup.c
index bab91a99124e..2b22ce792147 100644
--- a/arch/sh/boards/mach-ecovec24/setup.c
+++ b/arch/sh/boards/mach-ecovec24/setup.c
@@ -1502,7 +1502,7 @@ static void __init ecovec_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU0 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu0_dma_membase = phys;
 
@@ -1510,7 +1510,7 @@ static void __init ecovec_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU1 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu1_dma_membase = phys;
 }
diff --git a/arch/sh/boards/mach-kfr2r09/setup.c b/arch/sh/boards/mach-kfr2r09/setup.c
index eeb5ce341efd..20f4db778ed6 100644
--- a/arch/sh/boards/mach-kfr2r09/setup.c
+++ b/arch/sh/boards/mach-kfr2r09/setup.c
@@ -633,7 +633,7 @@ static void __init kfr2r09_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-migor/setup.c b/arch/sh/boards/mach-migor/setup.c
index 6703a2122c0d..f60061283c48 100644
--- a/arch/sh/boards/mach-migor/setup.c
+++ b/arch/sh/boards/mach-migor/setup.c
@@ -633,7 +633,7 @@ static void __init migor_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 
 	ceu_dma_membase = phys;
diff --git a/arch/sh/boards/mach-se/7724/setup.c b/arch/sh/boards/mach-se/7724/setup.c
index 8d6541ba0186..8bbf5a6aa423 100644
--- a/arch/sh/boards/mach-se/7724/setup.c
+++ b/arch/sh/boards/mach-se/7724/setup.c
@@ -966,7 +966,7 @@ static void __init ms7724se_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU0 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu0_dma_membase = phys;
 
@@ -974,7 +974,7 @@ static void __init ms7724se_mv_mem_reserve(void)
 	if (!phys)
 		panic("Failed to allocate CEU1 memory\n");
 
-	memblock_free(phys, size);
+	memblock_phys_free(phys, size);
 	memblock_remove(phys, size);
 	ceu1_dma_membase = phys;
 }
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 0224d8f19ed6..2507549538df 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -1567,7 +1567,7 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, size_t size,
 
 static void __init pcpu_free_bootmem(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_phys_free(__pa(ptr), size);
 }
 
 static int __init pcpu_cpu_distance(unsigned int from, unsigned int to)
diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c
index 8e636ce02949..d1710ebb44f4 100644
--- a/arch/um/kernel/mem.c
+++ b/arch/um/kernel/mem.c
@@ -47,7 +47,7 @@ void __init mem_init(void)
 	 */
 	brk_end = (unsigned long) UML_ROUND_UP(sbrk(0));
 	map_memory(brk_end, __pa(brk_end), uml_reserved - brk_end, 1, 1, 0);
-	memblock_free(__pa(brk_end), uml_reserved - brk_end);
+	memblock_phys_free(__pa(brk_end), uml_reserved - brk_end);
 	uml_reserved = brk_end;
 
 	/* this will put all low memory onto the freelists */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 40ed44ead063..49b596db5631 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -322,7 +322,7 @@ static void __init reserve_initrd(void)
 
 	relocate_initrd();
 
-	memblock_free(ramdisk_image, ramdisk_end - ramdisk_image);
+	memblock_phys_free(ramdisk_image, ramdisk_end - ramdisk_image);
 }
 
 #else
@@ -521,7 +521,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
-		memblock_free(crash_base, crash_size);
+		memblock_phys_free(crash_base, crash_size);
 		return;
 	}
 
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 23a14d82e783..1895986842b9 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -618,7 +618,7 @@ static void __init memory_map_top_down(unsigned long map_start,
 	 */
 	addr = memblock_phys_alloc_range(PMD_SIZE, PMD_SIZE, map_start,
 					 map_end);
-	memblock_free(addr, PMD_SIZE);
+	memblock_phys_free(addr, PMD_SIZE);
 	real_end = addr + PMD_SIZE;
 
 	/* step_size need to be small so pgt_buf from BRK could cover it */
diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 8d751939c6f3..3500b22ff087 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1025,7 +1025,7 @@ static void __init xen_free_ro_pages(unsigned long paddr, unsigned long size)
 	for (; vaddr < vaddr_end; vaddr += PAGE_SIZE)
 		make_lowmem_page_readwrite(vaddr);
 
-	memblock_free(paddr, size);
+	memblock_phys_free(paddr, size);
 }
 
 static void __init xen_cleanmfnmap_free_pgtbl(void *pgtbl, bool unpin)
@@ -1151,7 +1151,7 @@ static void __init xen_pagetable_p2m_free(void)
 		xen_cleanhighmap(addr, addr + size);
 		size = PAGE_ALIGN(xen_start_info->nr_pages *
 				  sizeof(unsigned long));
-		memblock_free(__pa(addr), size);
+		memblock_phys_free(__pa(addr), size);
 	} else {
 		xen_cleanmfnmap(addr);
 	}
@@ -1955,7 +1955,7 @@ void __init xen_relocate_p2m(void)
 		pfn_end = p2m_pfn_end;
 	}
 
-	memblock_free(PFN_PHYS(pfn), PAGE_SIZE * (pfn_end - pfn));
+	memblock_phys_free(PFN_PHYS(pfn), PAGE_SIZE * (pfn_end - pfn));
 	while (pfn < pfn_end) {
 		if (pfn == p2m_pfn) {
 			pfn = p2m_pfn_end;
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 8bfc10330107..f387fc7e5250 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -153,7 +153,7 @@ static void __init xen_del_extra_mem(unsigned long start_pfn,
 			break;
 		}
 	}
-	memblock_free(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
+	memblock_phys_free(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
 }
 
 /*
@@ -719,7 +719,7 @@ static void __init xen_reserve_xen_mfnlist(void)
 		return;
 
 	xen_relocate_p2m();
-	memblock_free(start, size);
+	memblock_phys_free(start, size);
 }
 
 /**
@@ -885,7 +885,7 @@ char * __init xen_memory_setup(void)
 		xen_phys_memcpy(new_area, start, size);
 		pr_info("initrd moved from [mem %#010llx-%#010llx] to [mem %#010llx-%#010llx]\n",
 			start, start + size, new_area, new_area + size);
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 		boot_params.hdr.ramdisk_image = new_area;
 		boot_params.ext_ramdisk_image = new_area >> 32;
 	}
diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index e28d9dfe3c20..7974913af09c 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -165,7 +165,7 @@ static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_fc_free(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_phys_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/drivers/firmware/efi/memmap.c b/drivers/firmware/efi/memmap.c
index 2ff1883dc788..4df55a55da84 100644
--- a/drivers/firmware/efi/memmap.c
+++ b/drivers/firmware/efi/memmap.c
@@ -35,7 +35,7 @@ void __init __efi_memmap_free(u64 phys, unsigned long size, unsigned long flags)
 		if (slab_is_available())
 			memblock_free_late(phys, size);
 		else
-			memblock_free(phys, size);
+			memblock_phys_free(phys, size);
 	} else if (flags & EFI_MEMMAP_SLAB) {
 		struct page *p = pfn_to_page(PHYS_PFN(phys));
 		unsigned int order = get_order(size);
diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
index 761fd870d1db..57ad67c04dfe 100644
--- a/drivers/of/kexec.c
+++ b/drivers/of/kexec.c
@@ -170,8 +170,7 @@ int ima_free_kexec_buffer(void)
 	if (ret)
 		return ret;
 
-	return memblock_free(addr, size);
-
+	return memblock_phys_free(addr, size);
 }
 
 /**
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index 59c1390cdf42..161c71e1e390 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -45,7 +45,7 @@ static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size,
 	if (nomap) {
 		err = memblock_mark_nomap(base, size);
 		if (err)
-			memblock_free(base, size);
+			memblock_phys_free(base, size);
 	}
 
 	return err;
@@ -282,7 +282,8 @@ void __init fdt_init_reserved_mem(void)
 				if (nomap)
 					memblock_clear_nomap(rmem->base, rmem->size);
 				else
-					memblock_free(rmem->base, rmem->size);
+					memblock_phys_free(rmem->base,
+							   rmem->size);
 			}
 		}
 	}
diff --git a/drivers/s390/char/sclp_early.c b/drivers/s390/char/sclp_early.c
index f01d942e1c1d..c0052655fc4f 100644
--- a/drivers/s390/char/sclp_early.c
+++ b/drivers/s390/char/sclp_early.c
@@ -139,7 +139,7 @@ int __init sclp_early_get_core_info(struct sclp_core_info *info)
 	}
 	sclp_fill_core_info(info, sccb);
 out:
-	memblock_free((unsigned long)sccb, length);
+	memblock_phys_free((unsigned long)sccb, length);
 	return rc;
 }
 
diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index be4ecbabdd58..933d77ad0a64 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -185,7 +185,7 @@ static void __init xdbc_free_ring(struct xdbc_ring *ring)
 	if (!seg)
 		return;
 
-	memblock_free(seg->dma, PAGE_SIZE);
+	memblock_phys_free(seg->dma, PAGE_SIZE);
 	ring->segment = NULL;
 }
 
@@ -665,10 +665,10 @@ int __init early_xdbc_setup_hardware(void)
 		xdbc_free_ring(&xdbc.in_ring);
 
 		if (xdbc.table_dma)
-			memblock_free(xdbc.table_dma, PAGE_SIZE);
+			memblock_phys_free(xdbc.table_dma, PAGE_SIZE);
 
 		if (xdbc.out_dma)
-			memblock_free(xdbc.out_dma, PAGE_SIZE);
+			memblock_phys_free(xdbc.out_dma, PAGE_SIZE);
 
 		xdbc.table_base = NULL;
 		xdbc.out_buf = NULL;
@@ -987,8 +987,8 @@ static int __init xdbc_init(void)
 	xdbc_free_ring(&xdbc.evt_ring);
 	xdbc_free_ring(&xdbc.out_ring);
 	xdbc_free_ring(&xdbc.in_ring);
-	memblock_free(xdbc.table_dma, PAGE_SIZE);
-	memblock_free(xdbc.out_dma, PAGE_SIZE);
+	memblock_phys_free(xdbc.table_dma, PAGE_SIZE);
+	memblock_phys_free(xdbc.out_dma, PAGE_SIZE);
 	writel(0, &xdbc.xdbc_reg->control);
 	early_iounmap(xdbc.xhci_base, xdbc.xhci_length);
 
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index e56a5faac395..4b671cc0a7ea 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -241,7 +241,7 @@ void __init xen_swiotlb_init_early(void)
 	 */
 	rc = xen_swiotlb_fixup(start, nslabs);
 	if (rc) {
-		memblock_free(__pa(start), PAGE_ALIGN(bytes));
+		memblock_phys_free(__pa(start), PAGE_ALIGN(bytes));
 		if (nslabs > 1024 && repeat--) {
 			/* Min is 2MB */
 			nslabs = max(1024UL, ALIGN(nslabs >> 1, IO_TLB_SEGSIZE));
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index e25f964fdd60..d32d41709513 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -103,7 +103,7 @@ void memblock_allow_resize(void);
 int memblock_add_node(phys_addr_t base, phys_addr_t size, int nid);
 int memblock_add(phys_addr_t base, phys_addr_t size);
 int memblock_remove(phys_addr_t base, phys_addr_t size);
-int memblock_free(phys_addr_t base, phys_addr_t size);
+int memblock_phys_free(phys_addr_t base, phys_addr_t size);
 int memblock_reserve(phys_addr_t base, phys_addr_t size);
 #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
 int memblock_physmem_add(phys_addr_t base, phys_addr_t size);
diff --git a/init/initramfs.c b/init/initramfs.c
index a842c0544745..1a971f070dd4 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -607,7 +607,7 @@ void __weak __init free_initrd_mem(unsigned long start, unsigned long end)
 	unsigned long aligned_start = ALIGN_DOWN(start, PAGE_SIZE);
 	unsigned long aligned_end = ALIGN(end, PAGE_SIZE);
 
-	memblock_free(__pa(aligned_start), aligned_end - aligned_start);
+	memblock_phys_free(__pa(aligned_start), aligned_end - aligned_start);
 #endif
 
 	free_reserved_area((void *)start, (void *)end, POISON_FREE_INITMEM,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 430d2f78d540..b9fa173e5e56 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -247,7 +247,7 @@ swiotlb_init(int verbose)
 	return;
 
 fail_free_mem:
-	memblock_free(__pa(tlb), bytes);
+	memblock_phys_free(__pa(tlb), bytes);
 fail:
 	pr_warn("Cannot allocate buffer");
 }
diff --git a/lib/cpumask.c b/lib/cpumask.c
index 045779446a18..a90786b77c1c 100644
--- a/lib/cpumask.c
+++ b/lib/cpumask.c
@@ -188,7 +188,7 @@ EXPORT_SYMBOL(free_cpumask_var);
  */
 void __init free_bootmem_cpumask_var(cpumask_var_t mask)
 {
-	memblock_free(__pa(mask), cpumask_size());
+	memblock_phys_free(__pa(mask), cpumask_size());
 }
 #endif
 
diff --git a/mm/cma.c b/mm/cma.c
index 995e15480937..f9a53bc708f2 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -378,7 +378,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base,
 	return 0;
 
 free_mem:
-	memblock_free(base, size);
+	memblock_phys_free(base, size);
 err:
 	pr_err("Failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
 	return ret;
diff --git a/mm/memblock.c b/mm/memblock.c
index 603f4a02be9b..a23baa482f3f 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -806,18 +806,18 @@ int __init_memblock memblock_remove(phys_addr_t base, phys_addr_t size)
 void __init_memblock memblock_free_ptr(void *ptr, size_t size)
 {
 	if (ptr)
-		memblock_free(__pa(ptr), size);
+		memblock_phys_free(__pa(ptr), size);
 }
 
 /**
- * memblock_free - free boot memory block
+ * memblock_phys_free - free boot memory block
  * @base: phys starting address of the  boot memory block
  * @size: size of the boot memory block in bytes
  *
  * Free boot memory block previously allocated by memblock_alloc_xx() API.
  * The freeing memory will not be released to the buddy allocator.
  */
-int __init_memblock memblock_free(phys_addr_t base, phys_addr_t size)
+int __init_memblock memblock_phys_free(phys_addr_t base, phys_addr_t size)
 {
 	phys_addr_t end = base + size - 1;
 
@@ -1934,7 +1934,7 @@ static void __init free_memmap(unsigned long start_pfn, unsigned long end_pfn)
 	 * memmap array.
 	 */
 	if (pg < pgend)
-		memblock_free(pg, pgend - pg);
+		memblock_phys_free(pg, pgend - pg);
 }
 
 /*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 9fd0be32a281..feffaa9423fe 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -2204,7 +2204,7 @@ static int __ref try_remove_memory(u64 start, u64 size)
 	arch_remove_memory(start, size, altmap);
 
 	if (IS_ENABLED(CONFIG_ARCH_KEEP_MEMBLOCK)) {
-		memblock_free(start, size);
+		memblock_phys_free(start, size);
 		memblock_remove(start, size);
 	}
 
diff --git a/mm/percpu.c b/mm/percpu.c
index f58318cb04c0..d65ddf6f2a35 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -2472,7 +2472,7 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
  */
 void __init pcpu_free_alloc_info(struct pcpu_alloc_info *ai)
 {
-	memblock_free(__pa(ai), ai->__ai_size);
+	memblock_phys_free(__pa(ai), ai->__ai_size);
 }
 
 /**
@@ -3134,7 +3134,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 out_free:
 	pcpu_free_alloc_info(ai);
 	if (areas)
-		memblock_free(__pa(areas), areas_size);
+		memblock_phys_free(__pa(areas), areas_size);
 	return rc;
 }
 #endif /* BUILD_EMBED_FIRST_CHUNK */
@@ -3256,7 +3256,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size,
 		free_fn(page_address(pages[j]), PAGE_SIZE);
 	rc = -ENOMEM;
 out_free_ar:
-	memblock_free(__pa(pages), pages_size);
+	memblock_phys_free(__pa(pages), pages_size);
 	pcpu_free_alloc_info(ai);
 	return rc;
 }
@@ -3286,7 +3286,7 @@ static void * __init pcpu_dfl_fc_alloc(unsigned int cpu, size_t size,
 
 static void __init pcpu_dfl_fc_free(void *ptr, size_t size)
 {
-	memblock_free(__pa(ptr), size);
+	memblock_phys_free(__pa(ptr), size);
 }
 
 void __init setup_per_cpu_areas(void)
diff --git a/mm/sparse.c b/mm/sparse.c
index 55fea0c2f927..fc3ab8d3b6bc 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -451,7 +451,7 @@ static void *sparsemap_buf_end __meminitdata;
 static inline void __meminit sparse_buffer_free(unsigned long size)
 {
 	WARN_ON(!sparsemap_buf || size == 0);
-	memblock_free(__pa(sparsemap_buf), size);
+	memblock_phys_free(__pa(sparsemap_buf), size);
 }
 
 static void __init sparse_buffer_init(unsigned long size, int nid)
-- 
2.28.0



From xen-devel-bounces@lists.xenproject.org Thu Sep 30 21:21:18 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 21:21:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200250.354708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW3UG-0001I8-BP; Thu, 30 Sep 2021 21:21:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200250.354708; Thu, 30 Sep 2021 21:21:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW3UG-0001I1-87; Thu, 30 Sep 2021 21:21:04 +0000
Received: by outflank-mailman (input) for mailman id 200250;
 Thu, 30 Sep 2021 21:21:02 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fx8M=OU=linuxfoundation.org=torvalds@srs-us1.protection.inumbo.net>)
 id 1mW3UE-0001Hv-Lf
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 21:21:02 +0000
Received: from mail-qk1-x735.google.com (unknown [2607:f8b0:4864:20::735])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 8362652c-3305-4b41-9f6e-8539cafeafcd;
 Thu, 30 Sep 2021 21:21:01 +0000 (UTC)
Received: by mail-qk1-x735.google.com with SMTP id f130so7249987qke.6
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 14:21:01 -0700 (PDT)
Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com.
 [209.85.219.173])
 by smtp.gmail.com with ESMTPSA id p15sm2222453qti.70.2021.09.30.14.21.00
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 30 Sep 2021 14:21:00 -0700 (PDT)
Received: by mail-yb1-f173.google.com with SMTP id u32so16180503ybd.9
 for <xen-devel@lists.xenproject.org>; Thu, 30 Sep 2021 14:21:00 -0700 (PDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8362652c-3305-4b41-9f6e-8539cafeafcd
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linux-foundation.org; s=google;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to
         :cc;
        bh=h/TBKn03dF9IntMFcuDE9WlvX4rnNMcxTnYaxfj4f3M=;
        b=BlxMYaM/OWq2QOg73mmSIuZReJqS9WDVJwYpygBxYlCdpjQvMPkG8JUUJuWMEfe7VW
         Kc4xJOkMskHk8O+awxj6C0zYig/iRuEgT85zngr4j5VHau1O2THU+xT3x/AlUo4/mD7V
         wX52wLkXcsoyoK8aKbnCd70Ljj6pEqXvKGjjk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=h/TBKn03dF9IntMFcuDE9WlvX4rnNMcxTnYaxfj4f3M=;
        b=NSzdVm7uVYffpeMxX5Q6SQbAxUsJV8Cqyknf2kk19DHRw8rjrlVUkftbjsE1Lpb2yM
         jaOrk5c5BrtuLlquAWiChFDNSC3HcfA3syx6plvXUWFvgJ6X2zlF2eR/c9xn478TCQEh
         HG4w/MUeVLdWG4ZgHygRits0XDicxSUnIG8U+pXu2NuT9VpJCTv70LiKqE5M8fxC3im1
         Lm0M9y8Ycvbn0Ax3Awoc3uH5hjvG+KOJn2KS4oZHYUVLoUkmtTnaHQOUzbW/i7FEK+T3
         /ABhrx+6PZm/8kXEdmAVlmZAHUSCRY844AQykF2ne1/DGD2Tg3u1PgKqOLWNzt2HXzA6
         SVGQ==
X-Gm-Message-State: AOAM530ThJ9fr9CgROGx3D+r/ilhRQRPgPp89sBWBqg5GIUarzFtR8bl
	WbqsL8265rll0SF451M25dhqBqsf+ZWuq71ocLo=
X-Google-Smtp-Source: ABdhPJwyxpSfA9iqeH8zYycgVMlOoOwRkpt4nbQEQtjCKAM5tcUXs7m0lM6wT9ofWATcp6mXeUWEdg==
X-Received: by 2002:a37:dcc7:: with SMTP id v190mr7152414qki.35.1633036861318;
        Thu, 30 Sep 2021 14:21:01 -0700 (PDT)
X-Received: by 2002:a25:df06:: with SMTP id w6mr1562849ybg.459.1633036849801;
 Thu, 30 Sep 2021 14:20:49 -0700 (PDT)
MIME-Version: 1.0
References: <20210930185031.18648-1-rppt@kernel.org>
In-Reply-To: <20210930185031.18648-1-rppt@kernel.org>
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Thu, 30 Sep 2021 14:20:33 -0700
X-Gmail-Original-Message-ID: <CAHk-=wjS76My8aJLWJAHd-5GnMEVC1D+kV7DgtV9GjcbtqZdig@mail.gmail.com>
Message-ID: <CAHk-=wjS76My8aJLWJAHd-5GnMEVC1D+kV7DgtV9GjcbtqZdig@mail.gmail.com>
Subject: Re: [PATCH v2 0/6] memblock: cleanup memblock_free interface
To: Mike Rapoport <rppt@kernel.org>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Andrew Morton <akpm@linux-foundation.org>, 
	Christophe Leroy <christophe.leroy@csgroup.eu>, Juergen Gross <jgross@suse.com>, 
	Mike Rapoport <rppt@linux.ibm.com>, Shahab Vahedi <Shahab.Vahedi@synopsys.com>, 
	devicetree <devicetree@vger.kernel.org>, iommu <iommu@lists.linux-foundation.org>, 
	kasan-dev <kasan-dev@googlegroups.com>, KVM list <kvm@vger.kernel.org>, 
	alpha <linux-alpha@vger.kernel.org>, 
	Linux ARM <linux-arm-kernel@lists.infradead.org>, 
	linux-efi <linux-efi@vger.kernel.org>, 
	"open list:BROADCOM NVRAM DRIVER" <linux-mips@vger.kernel.org>, Linux-MM <linux-mm@kvack.org>, 
	linux-riscv <linux-riscv@lists.infradead.org>, linux-s390 <linux-s390@vger.kernel.org>, 
	Linux-sh list <linux-sh@vger.kernel.org>, 
	"open list:SYNOPSYS ARC ARCHITECTURE" <linux-snps-arc@lists.infradead.org>, 
	linux-um <linux-um@lists.infradead.org>, linux-usb@vger.kernel.org, 
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, linux-sparc <sparclinux@vger.kernel.org>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"

On Thu, Sep 30, 2021 at 11:50 AM Mike Rapoport <rppt@kernel.org> wrote:
>
> The first patch is a cleanup of numa_distance allocation in arch_numa I've
> spotted during the conversion.
> The second patch is a fix for Xen memory freeing on some of the error
> paths.

Well, at least patch 2 looks like something that should go into 5.15
and be marked for stable.

Patch 1 looks like a trivial local cleanup, and could go in
immediately. Patch 4 might be in that same category.

The rest look like "next merge window" to me, since they are spread
out and neither bugfixes nor tiny localized cleanups (iow renaming
functions, global resulting search-and-replace things).

So my gut feel is that two (maybe three) of these patches should go in
asap, with three (maybe four) be left for 5.16.

IOW, not trat this as a single series.

Hmm?

             Linus


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 21:48:19 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 21:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200257.354719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW3uU-00043z-H8; Thu, 30 Sep 2021 21:48:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200257.354719; Thu, 30 Sep 2021 21:48:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW3uU-00043s-E6; Thu, 30 Sep 2021 21:48:10 +0000
Received: by outflank-mailman (input) for mailman id 200257;
 Thu, 30 Sep 2021 21:48:09 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2g57=OU=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mW3uT-00043m-0c
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 21:48:09 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 17ef42de-2238-11ec-bd73-12813bfff9fa;
 Thu, 30 Sep 2021 21:48:06 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id DEBF9619F5;
 Thu, 30 Sep 2021 21:48:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17ef42de-2238-11ec-bd73-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633038485;
	bh=najERrnlPvINVEoZ1wuhvqLvU4foW7Xj1i7Z/vC4NL8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XUz74xjBZK3I9WCyUNIidA2ln5tTe3MqnPwz1VzxqIz76o7qZt9aN2n5us384k8XJ
	 G4Bv9TOYO6z9ZpZFWAxX6EZb0Vr1rYQDxjfhOoZ/J3AfN6ar+1AIlTfC8j91UZrzEi
	 3amSwUKHxLpWfmKUIZDRm1/E7EyYrKco3dc8YRCFrG3xHkL7hDMQggrgnzUoPxnJkr
	 FVfJENOxZUwA8vfY8nergE4jhQma5eS0UyxZCUV7G+M1uHZ7lcZ3Affru+T7DSWtP9
	 /NdfUUPtqxVxnWljTNn4KBSAg33reXxtjbpFshXMp76R5BgBhsTIf/rWO/odu6SqnX
	 +WuogV0V50IFQ==
Date: Thu, 30 Sep 2021 14:47:26 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v4 2/3] arm/efi: Use dom0less configuration when using
 EFI boot
In-Reply-To: <20210930142846.13348-3-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109301447010.3209@sstabellini-ThinkPad-T480s>
References: <20210930142846.13348-1-luca.fancellu@arm.com> <20210930142846.13348-3-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 30 Sep 2021, Luca Fancellu wrote:
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "xen,uefi-binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v4:
> - update uefi,cfg-load to xen,uefi-cfg-load in documentation
> - fixed comments and code style
> - changed variable name from dt_module_found to dt_modules_found
> in boot.c
> - removed stub efi_arch_check_dt_boot from x86 code because the
> architecture does not support DT, protected call with #ifdef
> in the common code.
> - add comment to explain the result from efi_arch_check_dt_boot
> just looking the common code
> - Add space before comment in boot.c
> - renamed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - fixed documentation
> - fixed name len in strlcpy
> - fixed some style issues
> - closed filesystem handle before calling blexit
> - passed runtime errors up to the stack instead
> of calling blexit
> - renamed names and function to make them more
> general in prevision to load also Dom0 kernel
> and ramdisk from DT
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
>  docs/misc/arm/device-tree/booting.txt |  21 ++
>  docs/misc/efi.pandoc                  | 203 +++++++++++++++++
>  xen/arch/arm/efi/efi-boot.h           | 305 +++++++++++++++++++++++++-
>  xen/common/efi/boot.c                 |  46 ++--
>  4 files changed, 560 insertions(+), 15 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 352b0ec43a..7258e7e1ec 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
>  
>      Command line parameters for the guest kernel.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  The ramdisk sub-node has the following properties:
>  
>  - compatible
> @@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
>      Specifies the physical address of the ramdisk in RAM and its
>      length.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  
>  Example
>  =======
> @@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
>      Specifies the physical address of the device tree binary fragment
>      RAM and its length.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  As an example:
>  
>          module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ed85351541..876cd55005 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "xen,uefi-binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	compatible = "xen,domain";
> +	memory = <0 0x20000>;
> +	cpus = <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "console=ttyAMA0";
> +	};
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +These are the different ways to boot a Xen system from UEFI:
> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in the
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen node,
> +examples of how to do that are provided by the documentation about dom0less
> +and the example above shows how to use the "xen,uefi-binary" property to use the
> +UEFI stub for module loading.
> +When adding DomU modules to device tree, also add the property
> +xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
> +Otherwise, Xen will skip the config file and rely on device tree alone.
> +
> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this one
> +the configuration file must be processed so the /chosen node must have the
> +"xen,uefi-cfg-load" property.
> +
> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index a3e46453d4..50b3829ae0 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,49 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_UEFI_MODULES (MAX_MODULES - 2)
> +static struct file __initdata module_binary;
> +static module_name __initdata modules[MAX_UEFI_MODULES];
> +static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
> +static unsigned int __initdata modules_idx;
> +
> +#define ERROR_BINARY_FILE_NOT_FOUND (-1)
> +#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
> +#define ERROR_ALLOC_MODULE_NAME     (-2)
> +#define ERROR_MISSING_DT_PROPERTY   (-3)
> +#define ERROR_RENAME_MODULE_NAME    (-4)
> +#define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DT_MODULE_DOMU        (-1)
> +#define ERROR_DT_CHOSEN_NODE        (-2)
> +
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int get_module_file_index(const char *name, unsigned int name_len);
> +static void PrintMessage(const CHAR16 *s);
> +static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                const char *name,
> +                                unsigned int name_len);
> +static int handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                              int module_node_offset,
> +                              int reg_addr_cells,
> +                              int reg_size_cells);
> +static bool is_boot_module(int dt_module_offset);
> +static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                       int domain_node);
> +static int efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle);
> +
>  #define DEVICE_TREE_GUID \
>  {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
>  
> @@ -552,8 +592,260 @@ static void __init efi_arch_handle_module(const struct file *file,
>                           kernel.size) < 0 )
>              blexit(L"Unable to set reg property.");
>      }
> -    else
> +    else if ( file != &module_binary )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>          blexit(L"Unknown module type");
> +
> +    /*
> +     * modules_available is decremented here because for each dom0 file added
> +     * from the configuration file, there will be an additional bootmodule,
> +     * so the number of available slots will be decremented because there is a
> +     * maximum amount of bootmodules that can be loaded.
> +     */
> +    modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the modules array or a negative number if no
> + * file with that name is found.
> + */
> +static int __init get_module_file_index(const char *name,
> +                                        unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret = ERROR_BINARY_FILE_NOT_FOUND;
> +
> +    for ( i = 0; i < modules_idx; i++ )
> +    {
> +        module_name *mod = &modules[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +static void __init PrintMessage(const CHAR16 *s)
> +{
> +    PrintStr(s);
> +    PrintStr(newline);
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it returns the
> + * index of the file in the modules array or a negative number on error.
> + */
> +static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                       const char *name,
> +                                       unsigned int name_len)
> +{
> +    module_name *file_name;
> +    union string module_name;
> +    int ret;
> +
> +    /*
> +     * Check if there is any space left for a module, the variable
> +     * modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !modules_available )
> +    {
> +        PrintMessage(L"No space left for modules");
> +        return ERROR_ALLOC_MODULE_NO_SPACE;
> +    }
> +
> +    module_name.cs = name;
> +    ret = modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &modules[ret];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +    {
> +        PrintMessage(L"Error allocating memory for module binary name");
> +        return ERROR_ALLOC_MODULE_NAME;
> +    }
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len + 1);
> +    file_name->name_len = name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr = module_binary.addr;
> +    file_name->size = module_binary.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    modules_idx++;
> +
> +    return ret;
> +}
> +
> +/*
> + * This function checks for the presence of the xen,uefi-binary property in the
> + * module, if found it loads the binary as module and sets the right address
> + * for the reg property into the module DT node.
> + */
> +static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                                     int module_node_offset,
> +                                     int reg_addr_cells,
> +                                     int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx;
> +    module_name *file;
> +
> +    /* Read xen,uefi-binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return 0;
> +
> +    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
> +    if ( file_idx < 0 )
> +    {
> +        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
> +                                        uefi_name_len);
> +        if ( file_idx < 0 )
> +            return file_idx;
> +    }
> +
> +    file = &modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +    {
> +        PrintMessage(L"Unable to modify module node name.");
> +        return ERROR_RENAME_MODULE_NAME;
> +    }
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +    {
> +        PrintMessage(L"Unable to set module reg property.");
> +        return ERROR_SET_REG_PROPERTY;
> +    }
> +
> +    return 0;
> +}
> +
> +static bool __init is_boot_module(int dt_module_offset)
> +{
> +    if ( (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,kernel") == 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,ramdisk") == 0) ||
> +         (fdt_node_check_compatible(fdt, dt_module_offset,
> +                                    "multiboot,device-tree") == 0) )
> +        return true;
> +
> +    return false;
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + * Returns 0 on success, negative number on error.
> + */
> +static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                              int domain_node)
> +{
> +    int module_node, addr_cells, size_cells, len;
> +    const struct fdt_property *prop;
> +
> +    /* Get #address-cells and #size-cells from domain node */
> +    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#address-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#size-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    /*
> +     * Check for nodes compatible with multiboot,{kernel,ramdisk,device-tree}
> +     * inside this node
> +     */
> +    for ( module_node = fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +        if ( is_boot_module(module_node) )
> +        {
> +            int ret = handle_module_node(dir_handle, module_node, addr_cells,
> +                                         size_cells);
> +            if ( ret < 0 )
> +                return ret;
> +        }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + * Returns the number of modules loaded or a negative number for error.
> + */
> +static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen, node, addr_len, size_len;
> +    unsigned int i = 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +    {
> +        PrintMessage(L"Unable to setup chosen node");
> +        return ERROR_DT_CHOSEN_NODE;
> +    }
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
> +        {
> +            /* Found a node with compatible xen,domain; handle this node. */
> +            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> +                return ERROR_DT_MODULE_DOMU;
> +        }
> +    }
> +
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +    }
> +
> +    return modules_idx;
>  }
>  
>  static void __init efi_arch_cpu(void)
> @@ -562,8 +854,19 @@ static void __init efi_arch_cpu(void)
>  
>  static void __init efi_arch_blexit(void)
>  {
> +    unsigned int i = 0;
> +
>      if ( dtbfile.need_to_free )
>          efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +        /* Free modules binaries */
> +        efi_bs->FreePages(modules[i].addr,
> +                          PFN_UP(modules[i].size));
> +    }
>      if ( memmap )
>          efi_bs->FreePool(memmap);
>  }
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..daf7c26099 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1127,15 +1127,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      static EFI_GUID __initdata shim_lock_guid = SHIM_LOCK_PROTOCOL_GUID;
>      EFI_LOADED_IMAGE *loaded_image;
>      EFI_STATUS status;
> -    unsigned int i, argc;
> -    CHAR16 **argv, *file_name, *cfg_file_name = NULL, *options = NULL;
> +    unsigned int i, argc = 0;
> +    CHAR16 **argv, *file_name = NULL, *cfg_file_name = NULL, *options = NULL;
>      UINTN gop_mode = ~0;
>      EFI_SHIM_LOCK_PROTOCOL *shim_lock;
>      EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>      union string section = { NULL }, name;
>      bool base_video = false;
> -    const char *option_str;
> +    const char *option_str = NULL;
>      bool use_cfg_file;
> +    int dt_modules_found = 0;
> +    EFI_FILE_HANDLE dir_handle;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1218,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>      efi_arch_relocate_image(0);
>  
> +    /* Get the file system interface. */
> +    dir_handle = get_parent_handle(loaded_image, &file_name);
> +
>      if ( use_cfg_file )
>      {
> -        EFI_FILE_HANDLE dir_handle;
>          UINTN depth, cols, rows, size;
>  
>          size = cols = rows = depth = 0;
> @@ -1229,9 +1233,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>          gop = efi_get_gop();
>  
> -        /* Get the file system interface. */
> -        dir_handle = get_parent_handle(loaded_image, &file_name);
> -
>          /* Read and parse the config file. */
>          if ( read_section(loaded_image, L"config", &cfg, NULL) )
>              PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1286,13 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +        if ( name.s )
> +            option_str = split_string(name.s);
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
> +             name.s )
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> @@ -1361,12 +1361,30 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    /* Get the number of boot modules specified on the DT or an error (<0) */
> +    dt_modules_found = efi_arch_check_dt_boot(dir_handle);
> +#endif
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if ( dt_modules_found < 0 )
> +        /* efi_arch_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");
> +
> +    /*
> +     * Check if a proper configuration is provided to start Xen:
> +     *  - Dom0 specified (minimum required)
> +     *  - Dom0 and DomU(s) specified
> +     *  - DomU(s) specified
> +     */
> +    if ( !dt_modules_found && !kernel.addr )
> +        blexit(L"No Dom0 kernel image specified.");
> +
>      efi_arch_edd();
>  
>      /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 21:58:54 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 21:58:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200267.354730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW44n-0005jP-L4; Thu, 30 Sep 2021 21:58:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200267.354730; Thu, 30 Sep 2021 21:58:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW44n-0005jI-HO; Thu, 30 Sep 2021 21:58:49 +0000
Received: by outflank-mailman (input) for mailman id 200267;
 Thu, 30 Sep 2021 21:58:48 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2g57=OU=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mW44m-0005jC-Jh
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 21:58:48 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 95ed8e90-2239-11ec-bd73-12813bfff9fa;
 Thu, 30 Sep 2021 21:58:46 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id AF31B61A4F;
 Thu, 30 Sep 2021 21:58:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95ed8e90-2239-11ec-bd73-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633039126;
	bh=0oTU0moiXXod14kv3+0BdJyKbDmZGbCqITPCjWwoORQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=t4ha3YjVDLe3kvPNXhIqjttQ7VABkEGIJnnXH/mm4tg2JynI2ZTN8wCXbYRKBPRxJ
	 rPLlO/EcwvZ8vtx7Eew53B7C4oFwFCJpkbZQ8DVCzgjnOMx9qMTefkRjrSJV4wZP/p
	 cT5oMqfKxQSqcK67nqrzAToaBM6P985+sgWHyA/XRO/0xpuGs8etNmuiQyP3EglJzM
	 eQN1UEYDcCWU7xkD5bg7yEe7SU7ShLQoccfoE0gM4Dh37POMn8hadzWkXLxEI6k6yU
	 gaSmXrbmK3WLLLtaR2FVpAJf/1+KDHvbVXOWbELtZ5uqMiXFDwdGHjk1++GtIym2oX
	 igxegChJx+7Lg==
Date: Thu, 30 Sep 2021 14:58:45 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, bertrand.marquis@arm.com, wei.chen@arm.com, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v4 3/3] arm/efi: load dom0 modules from DT using UEFI
In-Reply-To: <20210930142846.13348-4-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.21.2109301450120.3209@sstabellini-ThinkPad-T480s>
References: <20210930142846.13348-1-luca.fancellu@arm.com> <20210930142846.13348-4-luca.fancellu@arm.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 30 Sep 2021, Luca Fancellu wrote:
> Add support to load Dom0 boot modules from
> the device tree using the xen,uefi-binary property.
> 
> Update documentation about that.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> Changes in v4:
> - Add check to avoid double definition of Dom0 ramdisk
> from cfg file and DT
> - Fix if conditions indentation in boot.c
> - Moved Dom0 kernel verification code after check for
> presence for Dom0 or DomU(s)
> - Changed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - new patch
> ---
>  docs/misc/arm/device-tree/booting.txt |  8 ++++
>  docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
>  xen/arch/arm/efi/efi-boot.h           | 47 ++++++++++++++++++--
>  xen/common/efi/boot.c                 | 16 ++++---
>  4 files changed, 123 insertions(+), 12 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 7258e7e1ec..8e0c327ba4 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -70,6 +70,14 @@ Each node contains the following properties:
>  	priority of this field vs. other mechanisms of specifying the
>  	bootargs for the kernel.
>  
> +- uefi,binary (UEFI boot only)

Needs to be renamed, but it could be done on commit:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



> +	String property that specifies the file name to be loaded by the UEFI
> +	boot for this module. If this is specified, there is no need to specify
> +	the reg property because it will be created by the UEFI stub on boot.
> +	This option is needed only when UEFI boot is used, the node needs to be
> +	compatible with multiboot,kernel or multiboot,ramdisk.
> +
>  Examples
>  ========
>  
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index 876cd55005..4abbb5bb82 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,6 +167,28 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +## UEFI boot and Dom0 modules on ARM
> +
> +When booting using UEFI on ARM, it is possible to specify the Dom0 modules
> +directly from the device tree without using the Xen configuration file, here an
> +example:
> +
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
> +	};
> +}
>  
>  ## UEFI boot and dom0less on ARM
>  
> @@ -326,10 +348,10 @@ chosen {
>  ### Boot Xen, Dom0 and DomU(s)
>  
>  This configuration is a mix of the two configuration above, to boot this one
> -the configuration file must be processed so the /chosen node must have the
> -"xen,uefi-cfg-load" property.
> +the configuration file can be processed or the Dom0 modules can be read from
> +the device tree.
>  
> -Here an example:
> +Here the first example:
>  
>  Xen configuration file:
>  
> @@ -369,4 +391,40 @@ chosen {
>  };
>  ```
>  
> +Here the second example:
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "[Xen boot arguments]"
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "[domain 0 command line options]";
> +	};
> +
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
> +	};
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
>  
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 50b3829ae0..b122e2846a 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -31,8 +31,11 @@ static unsigned int __initdata modules_idx;
>  #define ERROR_MISSING_DT_PROPERTY   (-3)
>  #define ERROR_RENAME_MODULE_NAME    (-4)
>  #define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_DOM0_ALREADY_FOUND    (-6)
> +#define ERROR_DOM0_RAMDISK_FOUND    (-7)
>  #define ERROR_DT_MODULE_DOMU        (-1)
>  #define ERROR_DT_CHOSEN_NODE        (-2)
> +#define ERROR_DT_MODULE_DOM0        (-3)
>  
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
> @@ -45,7 +48,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
>  static int handle_module_node(EFI_FILE_HANDLE dir_handle,
>                                int module_node_offset,
>                                int reg_addr_cells,
> -                              int reg_size_cells);
> +                              int reg_size_cells,
> +                              bool is_domu_module);
>  static bool is_boot_module(int dt_module_offset);
>  static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>                                         int domain_node);
> @@ -701,7 +705,8 @@ static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
>  static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>                                       int module_node_offset,
>                                       int reg_addr_cells,
> -                                     int reg_size_cells)
> +                                     int reg_size_cells,
> +                                     bool is_domu_module)
>  {
>      const void *uefi_name_prop;
>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> @@ -743,6 +748,34 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>          return ERROR_SET_REG_PROPERTY;
>      }
>  
> +    if ( !is_domu_module )
> +    {
> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
> +                                    "multiboot,kernel") == 0) )
> +        {
> +            /*
> +            * This is the Dom0 kernel, wire it to the kernel variable because it
> +            * will be verified by the shim lock protocol later in the common
> +            * code.
> +            */
> +            if ( kernel.addr )
> +            {
> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
> +                return ERROR_DOM0_ALREADY_FOUND;
> +            }
> +            kernel.need_to_free = false; /* Freed using the module array */
> +            kernel.addr = file->addr;
> +            kernel.size = file->size;
> +        }
> +        else if ( ramdisk.addr &&
> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> +                                             "multiboot,ramdisk") == 0) )
> +        {
> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> +            return ERROR_DOM0_RAMDISK_FOUND;
> +        }
> +    }
> +
>      return 0;
>  }
>  
> @@ -799,7 +832,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>          if ( is_boot_module(module_node) )
>          {
>              int ret = handle_module_node(dir_handle, module_node, addr_cells,
> -                                         size_cells);
> +                                         size_cells, true);
>              if ( ret < 0 )
>                  return ret;
>          }
> @@ -809,7 +842,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>  
>  /*
>   * This function checks for xen domain nodes under the /chosen node for possible
> - * domU guests to be loaded.
> + * dom0 and domU guests to be loaded.
>   * Returns the number of modules loaded or a negative number for error.
>   */
>  static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> @@ -836,6 +869,12 @@ static int __init efi_arch_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>              if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>                  return ERROR_DT_MODULE_DOMU;
>          }
> +        else if ( is_boot_module(node) )
> +        {
> +            if ( handle_module_node(dir_handle, node, addr_len, size_len,
> +                                    false) < 0 )
> +                return ERROR_DT_MODULE_DOM0;
> +        }
>      }
>  
>      /* Free boot modules file names if any */
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index daf7c26099..e4295dbe34 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -1296,11 +1296,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> -
> -            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> -                            (void **)&shim_lock)) &&
> -                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> -                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
>          }
>  
>          if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
> @@ -1385,6 +1380,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      if ( !dt_modules_found && !kernel.addr )
>          blexit(L"No Dom0 kernel image specified.");
>  
> +    /*
> +     * The Dom0 kernel can be loaded from the configuration file or by the
> +     * device tree through the efi_arch_check_dt_boot function, in this stage
> +     * verify it.
> +     */
> +    if ( kernel.addr &&
> +         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
> +                                           (void **)&shim_lock)) &&
> +         (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
> +        PrintErrMesg(L"Dom0 kernel image could not be verified", status);
> +
>      efi_arch_edd();
>  
>      /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 22:10:58 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 22:10:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200274.354740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW4GQ-00086q-Mh; Thu, 30 Sep 2021 22:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200274.354740; Thu, 30 Sep 2021 22:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW4GQ-00086j-Jk; Thu, 30 Sep 2021 22:10:50 +0000
Received: by outflank-mailman (input) for mailman id 200274;
 Thu, 30 Sep 2021 22:10:49 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=zaCp=OU=kernel.org=rppt@srs-us1.protection.inumbo.net>)
 id 1mW4GP-00086c-9n
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 22:10:49 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 44067cc0-223b-11ec-bd73-12813bfff9fa;
 Thu, 30 Sep 2021 22:10:48 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 965D361390;
 Thu, 30 Sep 2021 22:10:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44067cc0-223b-11ec-bd73-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633039847;
	bh=IwzBemW+hy1M1ewr7atFO0cSMjKx7nF1iOUj3EQWXE8=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=VMFkm4a+fgdg4zYaCkaWnSOTeOZ8uAbKkitUxM0+m4BBfXT2b5oLMM7rDTLawyLAE
	 NhhBOznSRQz5yRbiTNsvbV6E0IU3T5qz407KpFRCdUtW5hyDqK4d2pDAOMg4JRadt6
	 KWrjYIXEm0KrNBVtP+JPzhB2VrG3+5vrqbM5hiSfUBy+07gwCIoejO0uW/sH7DYI6V
	 pgwC0QydyQR/hM9nqnvQyR6gIiVBVoszYMQd2xY8yoPecdNAhtt2AC8GtVh+C0O4VI
	 bPLArmwwdAbQbiLreQFVTZM6xuVy7lVHSRkS66W+8O7Twsm9hTlzsCUC2ioP79rEse
	 MlAOzYU2R7N3A==
Date: Thu, 30 Sep 2021 15:10:46 -0700
From: Mike Rapoport <rppt@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	Juergen Gross <jgross@suse.com>, Mike Rapoport <rppt@linux.ibm.com>,
	Shahab Vahedi <Shahab.Vahedi@synopsys.com>,
	devicetree <devicetree@vger.kernel.org>,
	iommu <iommu@lists.linux-foundation.org>,
	kasan-dev <kasan-dev@googlegroups.com>,
	KVM list <kvm@vger.kernel.org>, alpha <linux-alpha@vger.kernel.org>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	linux-efi <linux-efi@vger.kernel.org>,
	"open list:BROADCOM NVRAM DRIVER" <linux-mips@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>,
	linux-riscv <linux-riscv@lists.infradead.org>,
	linux-s390 <linux-s390@vger.kernel.org>,
	Linux-sh list <linux-sh@vger.kernel.org>,
	"open list:SYNOPSYS ARC ARCHITECTURE" <linux-snps-arc@lists.infradead.org>,
	linux-um <linux-um@lists.infradead.org>, linux-usb@vger.kernel.org,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	linux-sparc <sparclinux@vger.kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2 0/6] memblock: cleanup memblock_free interface
Message-ID: <YVY15nd56j8x8udh@kernel.org>
References: <20210930185031.18648-1-rppt@kernel.org>
 <CAHk-=wjS76My8aJLWJAHd-5GnMEVC1D+kV7DgtV9GjcbtqZdig@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAHk-=wjS76My8aJLWJAHd-5GnMEVC1D+kV7DgtV9GjcbtqZdig@mail.gmail.com>

On Thu, Sep 30, 2021 at 02:20:33PM -0700, Linus Torvalds wrote:
> On Thu, Sep 30, 2021 at 11:50 AM Mike Rapoport <rppt@kernel.org> wrote:
> >
> > The first patch is a cleanup of numa_distance allocation in arch_numa I've
> > spotted during the conversion.
> > The second patch is a fix for Xen memory freeing on some of the error
> > paths.
> 
> Well, at least patch 2 looks like something that should go into 5.15
> and be marked for stable.
> 
> Patch 1 looks like a trivial local cleanup, and could go in
> immediately. Patch 4 might be in that same category.
> 
> The rest look like "next merge window" to me, since they are spread
> out and neither bugfixes nor tiny localized cleanups (iow renaming
> functions, global resulting search-and-replace things).
> 
> So my gut feel is that two (maybe three) of these patches should go in
> asap, with three (maybe four) be left for 5.16.
> 
> IOW, not trat this as a single series.
> 
> Hmm?

Yes, why not :)
I'd keep patch 4 for the next merge window, does not look urgent to me.

Andrew, can you please take care of this or you'd prefer me resending
everything separately?
 
>              Linus

-- 
Sincerely yours,
Mike.


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 22:53:37 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 22:53:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200281.354752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW4vX-0004Cq-0v; Thu, 30 Sep 2021 22:53:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200281.354752; Thu, 30 Sep 2021 22:53:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW4vW-0004Cj-Sa; Thu, 30 Sep 2021 22:53:18 +0000
Received: by outflank-mailman (input) for mailman id 200281;
 Thu, 30 Sep 2021 22:53:16 +0000
Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]
 helo=us1-amaz-eas2.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2g57=OU=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mW4vU-0004Cd-RM
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 22:53:16 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS
 id 3167b240-2241-11ec-bd77-12813bfff9fa;
 Thu, 30 Sep 2021 22:53:14 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id 288AF61882;
 Thu, 30 Sep 2021 22:53:13 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3167b240-2241-11ec-bd77-12813bfff9fa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633042393;
	bh=slGvynQStahnHFFIDcDwS6mX5WxI52rBvpjspSzeIsQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hr1YFEYRb5ATAX9bjAkST6ll28jY2oc+GCnS9jOvCYKC624v3mCiUY9k6ZkEhRyR+
	 yKXWWYuZJShgc4YhD3J2KnvyIyF+1X3GK3L88oJLaXhBLm3Y8d72Qc+bo4JSuw+N3R
	 /H5ka6KrY++TnGcAJNfnyRq8Og1g5Z0Eb8x0kf+69nwUuLZKDs/UvxySbzr/jByMRx
	 yK6bNLY8U9AMNfn+ouHeWreU2tQeZUaA0DvaWSKXIdlY5TQOZYgVF1eoR/kpBjWnqz
	 bCJCL6pMG0Sk1orHmrSys6Aq5+yRwapmDdmjah2FfBFkbqsAmMZC9bOwR4naG58dwn
	 GJgeY/3oL1Iow==
Date: Thu, 30 Sep 2021 15:53:12 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Ian Jackson <iwj@xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH V4 2/3] xen/arm: Add handling of extended regions for
 Dom0
In-Reply-To: <1632955927-27911-3-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109301553032.3209@sstabellini-ThinkPad-T480s>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com> <1632955927-27911-3-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 30 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> The extended region (safe range) is a region of guest physical
> address space which is unused and could be safely used to create
> grant/foreign mappings instead of wasting real RAM pages from
> the domain memory for establishing these mappings.
> 
> The extended regions are chosen at the domain creation time and
> advertised to it via "reg" property under hypervisor node in
> the guest device-tree. As region 0 is reserved for grant table
> space (always present), the indexes for extended regions are 1...N.
> If extended regions could not be allocated for some reason,
> Xen doesn't fail and behaves as usual, so only inserts region 0.
> 
> Please note the following limitations:
> - The extended region feature is only supported for 64-bit domain
>   currently.
> - The ACPI case is not covered.
> 
> ***
> 
> As Dom0 is direct mapped domain on Arm (e.g. MFN == GFN)
> the algorithm to choose extended regions for it is different
> in comparison with the algorithm for non-direct mapped DomU.
> What is more, that extended regions should be chosen differently
> whether IOMMU is enabled or not.
> 
> Provide RAM not assigned to Dom0 if IOMMU is disabled or memory
> holes found in host device-tree if otherwise. Make sure that
> extended regions are 2MB-aligned and located within maximum possible
> addressable physical memory range. The minimum size of extended
> region is 64MB. The maximum number of extended regions is 128,
> which is an artificial limit to minimize code changes (we reuse
> struct meminfo to describe extended regions, so there are an array
> field for 128 elements).
> 
> It worth mentioning that unallocated memory solution (when the IOMMU
> is disabled) will work safely until Dom0 is able to allocate memory
> outside of the original range.
> 
> Also introduce command line option to be able to globally enable or
> disable support for extended regions for Dom0 (enabled by default).
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Please note, we need to decide which approach to use in find_unallocated_memory(),
> you can find details at:
> https://lore.kernel.org/xen-devel/28503e09-44c3-f623-bb8d-8778bb94225f@gmail.com/
> 
> Changes RFC -> V2:
>    - update patch description
>    - drop uneeded "extended-region" DT property
> 
> Changes V2 -> V3:
>    - update patch description
>    - add comment for "size" calculation in add_ext_regions()
>    - clarify "end" calculation in find_unallocated_memory() and
>      find_memory_holes()
>    - only pick up regions with size >= 64MB
>    - allocate reg dynamically instead of keeping on the stack in
>      make_hypervisor_node()
>    - do not show warning for 32-bit domain
>    - drop Linux specific limits EXT_REGION_*
>    - also cover "ranges" property in find_memory_holes()
>    - add command line arg to enable/disable extended region support
> 
> Changes V3 -> V4:
>   - update opt_ext_regions purpose and comment in code
>   - reorganize make_hypervisor_node() to move allocations after initial
>     checks, allocate only required amount of elements instead of maximum
>     possible
> ---
>  docs/misc/xen-command-line.pandoc |  11 ++
>  xen/arch/arm/domain_build.c       | 286 +++++++++++++++++++++++++++++++++++++-
>  2 files changed, 294 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 177e656..5cae4ad 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1081,6 +1081,17 @@ hardware domain is architecture dependent.
>  Note that specifying zero as domU value means zero, while for dom0 it means
>  to use the default.
>  
> +### ext_regions (Arm)
> +> `= <boolean>`
> +
> +> Default : `true`
> +
> +Flag to enable or disable support for extended regions for Dom0.
> +
> +Extended regions are ranges of unused address space exposed to Dom0 as
> +"safe to use" for special memory mappings. Disable if your board device
> +tree is incomplete.
> +
>  ### flask
>  > `= permissive | enforcing | late | disabled`
>  
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d233d63..c5afbe2 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -34,6 +34,10 @@
>  static unsigned int __initdata opt_dom0_max_vcpus;
>  integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>  
> +/* If true, the extended regions support is enabled for dom0 */
> +static bool __initdata opt_ext_regions = true;
> +boolean_param("ext_regions", opt_ext_regions);
> +
>  static u64 __initdata dom0_mem;
>  static bool __initdata dom0_mem_set;
>  
> @@ -886,6 +890,232 @@ static int __init make_memory_node(const struct domain *d,
>      return res;
>  }
>  
> +static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
> +{
> +    struct meminfo *ext_regions = data;
> +    paddr_t start, size;
> +
> +    if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
> +        return 0;
> +
> +    /* Both start and size of the extended region should be 2MB aligned */
> +    start = (s + SZ_2M - 1) & ~(SZ_2M - 1);
> +    if ( start > e )
> +        return 0;
> +
> +    /*
> +     * e is actually "end-1" because it is called by rangeset functions
> +     * which are inclusive of the last address.
> +     */
> +    e += 1;
> +    size = (e - start) & ~(SZ_2M - 1);
> +    if ( size < MB(64) )
> +        return 0;
> +
> +    ext_regions->bank[ext_regions->nr_banks].start = start;
> +    ext_regions->bank[ext_regions->nr_banks].size = size;
> +    ext_regions->nr_banks++;
> +
> +    return 0;
> +}
> +
> +static int __init find_unallocated_memory(const struct kernel_info *kinfo,
> +                                          struct meminfo *ext_regions)
> +{
> +    const struct meminfo *assign_mem = &kinfo->mem;
> +    struct rangeset *unalloc_mem;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find unallocated memory for extended regions\n");
> +
> +    unalloc_mem = rangeset_new(NULL, NULL, 0);
> +    if ( !unalloc_mem )
> +        return -ENOMEM;
> +
> +    /* Start with all available RAM */
> +    for ( i = 0; i < bootinfo.mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.mem.bank[i].start;
> +        end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
> +        res = rangeset_add_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove RAM assigned to Dom0 */
> +    for ( i = 0; i < assign_mem->nr_banks; i++ )
> +    {
> +        start = assign_mem->bank[i].start;
> +        end = assign_mem->bank[i].start + assign_mem->bank[i].size;
> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove reserved-memory regions */
> +    for ( i = 0; i < bootinfo.reserved_mem.nr_banks; i++ )
> +    {
> +        start = bootinfo.reserved_mem.bank[i].start;
> +        end = bootinfo.reserved_mem.bank[i].start +
> +            bootinfo.reserved_mem.bank[i].size;
> +        res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                   start, end);
> +            goto out;
> +        }
> +    }
> +
> +    /* Remove grant table region */
> +    start = kinfo->gnttab_start;
> +    end = kinfo->gnttab_start + kinfo->gnttab_size;
> +    res = rangeset_remove_range(unalloc_mem, start, end - 1);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_report_ranges(unalloc_mem, start, end,
> +                                 add_ext_regions, ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(unalloc_mem);
> +
> +    return res;
> +}
> +
> +static int __init find_memory_holes(const struct kernel_info *kinfo,
> +                                    struct meminfo *ext_regions)
> +{
> +    struct dt_device_node *np;
> +    struct rangeset *mem_holes;
> +    paddr_t start, end;
> +    unsigned int i;
> +    int res;
> +
> +    dt_dprintk("Find memory holes for extended regions\n");
> +
> +    mem_holes = rangeset_new(NULL, NULL, 0);
> +    if ( !mem_holes )
> +        return -ENOMEM;
> +
> +    /* Start with maximum possible addressable physical memory range */
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_add_range(mem_holes, start, end);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Failed to add: %#"PRIx64"->%#"PRIx64"\n",
> +               start, end);
> +        goto out;
> +    }
> +
> +    /*
> +     * Remove regions described by "reg" and "ranges" properties where
> +     * the memory is addressable (MMIO, RAM, PCI BAR, etc).
> +     */
> +    dt_for_each_device_node( dt_host, np )
> +    {
> +        unsigned int naddr;
> +        u64 addr, size;
> +
> +        naddr = dt_number_of_address(np);
> +
> +        for ( i = 0; i < naddr; i++ )
> +        {
> +            res = dt_device_get_address(np, i, &addr, &size);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> +                       i, dt_node_full_name(np));
> +                goto out;
> +            }
> +
> +            start = addr & PAGE_MASK;
> +            end = PAGE_ALIGN(addr + size);
> +            res = rangeset_remove_range(mem_holes, start, end - 1);
> +            if ( res )
> +            {
> +                printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                       start, end);
> +                goto out;
> +            }
> +        }
> +
> +        if ( dt_device_type_is_equal(np, "pci" ) )
> +        {
> +            unsigned int range_size, nr_ranges;
> +            int na, ns, pna;
> +            const __be32 *ranges;
> +            u32 len;
> +
> +            /*
> +             * Looking for non-empty ranges property which in this context
> +             * describes the PCI host bridge aperture.
> +             */
> +            ranges = dt_get_property(np, "ranges", &len);
> +            if ( !ranges || !len )
> +                continue;
> +
> +            pna = dt_n_addr_cells(np);
> +            na = dt_child_n_addr_cells(np);
> +            ns = dt_child_n_size_cells(np);
> +            range_size = pna + na + ns;
> +            nr_ranges = len / sizeof(__be32) / range_size;
> +
> +            for ( i = 0; i < nr_ranges; i++, ranges += range_size )
> +            {
> +                /* Skip the child address and get the parent (CPU) address */
> +                addr = dt_read_number(ranges + na, pna);
> +                size = dt_read_number(ranges + na + pna, ns);
> +
> +                start = addr & PAGE_MASK;
> +                end = PAGE_ALIGN(addr + size);
> +                res = rangeset_remove_range(mem_holes, start, end - 1);
> +                if ( res )
> +                {
> +                    printk(XENLOG_ERR "Failed to remove: %#"PRIx64"->%#"PRIx64"\n",
> +                           start, end);
> +                    goto out;
> +                }
> +            }
> +        }
> +    }
> +
> +    start = 0;
> +    end = (1ULL << p2m_ipa_bits) - 1;
> +    res = rangeset_report_ranges(mem_holes, start, end,
> +                                 add_ext_regions,  ext_regions);
> +    if ( res )
> +        ext_regions->nr_banks = 0;
> +    else if ( !ext_regions->nr_banks )
> +        res = -ENOENT;
> +
> +out:
> +    rangeset_destroy(mem_holes);
> +
> +    return res;
> +}
> +
>  static int __init make_hypervisor_node(struct domain *d,
>                                         const struct kernel_info *kinfo,
>                                         int addrcells, int sizecells)
> @@ -893,11 +1123,12 @@ static int __init make_hypervisor_node(struct domain *d,
>      const char compat[] =
>          "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
>          "xen,xen";
> -    __be32 reg[4];
> +    __be32 *reg, *cells;
>      gic_interrupt_t intr;
> -    __be32 *cells;
>      int res;
>      void *fdt = kinfo->fdt;
> +    struct meminfo *ext_regions = NULL;
> +    unsigned int i, nr_ext_regions;
>  
>      dt_dprintk("Create hypervisor node\n");
>  
> @@ -919,12 +1150,61 @@ static int __init make_hypervisor_node(struct domain *d,
>      if ( res )
>          return res;
>  
> +    if ( !opt_ext_regions )
> +    {
> +        printk(XENLOG_DEBUG "The extended regions support is disabled\n");
> +        nr_ext_regions = 0;
> +    }
> +    else if ( is_32bit_domain(d) )
> +    {
> +        printk(XENLOG_DEBUG "The extended regions are only supported for 64-bit guest currently\n");
> +        nr_ext_regions = 0;
> +    }
> +    else
> +    {
> +        ext_regions = xzalloc(struct meminfo);
> +        if ( !ext_regions )
> +            return -ENOMEM;
> +
> +        if ( !is_iommu_enabled(d) )
> +            res = find_unallocated_memory(kinfo, ext_regions);
> +        else
> +            res = find_memory_holes(kinfo, ext_regions);
> +
> +        if ( res )
> +            printk(XENLOG_WARNING "Failed to allocate extended regions\n");
> +        nr_ext_regions = ext_regions->nr_banks;
> +    }
> +
> +    reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + sizecells));
> +    if ( !reg )
> +    {
> +        xfree(ext_regions);
> +        return -ENOMEM;
> +    }
> +
>      /* reg 0 is grant table space */
>      cells = &reg[0];
>      dt_child_set_range(&cells, addrcells, sizecells,
>                         kinfo->gnttab_start, kinfo->gnttab_size);
> +    /* reg 1...N are extended regions */
> +    for ( i = 0; i < nr_ext_regions; i++ )
> +    {
> +        u64 start = ext_regions->bank[i].start;
> +        u64 size = ext_regions->bank[i].size;
> +
> +        dt_dprintk("Extended region %d: %#"PRIx64"->%#"PRIx64"\n",
> +                   i, start, start + size);
> +
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> +    }
> +
>      res = fdt_property(fdt, "reg", reg,
> -                       dt_cells_to_size(addrcells + sizecells));
> +                       dt_cells_to_size(addrcells + sizecells) *
> +                       (nr_ext_regions + 1));
> +    xfree(ext_regions);
> +    xfree(reg);
> +
>      if ( res )
>          return res;
>  
> -- 
> 2.7.4
> 


From xen-devel-bounces@lists.xenproject.org Thu Sep 30 23:00:36 2021
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 30 Sep 2021 23:00:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.200291.354762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW52W-0005rG-Pt; Thu, 30 Sep 2021 23:00:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 200291.354762; Thu, 30 Sep 2021 23:00:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1mW52W-0005r9-Mq; Thu, 30 Sep 2021 23:00:32 +0000
Received: by outflank-mailman (input) for mailman id 200291;
 Thu, 30 Sep 2021 23:00:31 +0000
Received: from us1-rack-iad1.inumbo.com ([172.99.69.81])
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=2g57=OU=kernel.org=sstabellini@srs-us1.protection.inumbo.net>)
 id 1mW52V-0005r3-NQ
 for xen-devel@lists.xenproject.org; Thu, 30 Sep 2021 23:00:31 +0000
Received: from mail.kernel.org (unknown [198.145.29.99])
 by us1-rack-iad1.inumbo.com (Halon) with ESMTPS
 id 45095e8e-9b47-42ad-9d34-1e32da525be5;
 Thu, 30 Sep 2021 23:00:29 +0000 (UTC)
Received: by mail.kernel.org (Postfix) with ESMTPSA id BFEEE61A54;
 Thu, 30 Sep 2021 23:00:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45095e8e-9b47-42ad-9d34-1e32da525be5
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1633042829;
	bh=92bS3ks99WIDwuP8LmBEJKsSjScFwmh1i0p2cuLpP4Q=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UMesJHxS0qQuToJVR/oT7qcwXlJm4ucSb/7sj44hvjDmBvBspr9b+xjcSGIAtFp9t
	 cJ7D5bUADoSfPTFhRIkJbtCH4YUXsWPUvaWgtbuwH5Qifw3J6IQy2HDmEh9h/awuly
	 DyeBP+r415H/peHrxK4S+ipAHiZmTUn83M1jdfD7tHCyBqsWeFAhESnrOor5G/bOC9
	 g3uyfEKzM8ycJTmLygmxtJGjoC78hVpNVx6kfTmXjDiVwDed0x7KznxIl7lDa7NGUU
	 6NA99boXEhItuCwm00dZ2ZXmk11/oBTAA5FXVUyZnWKVGrXq+4o6WFpBkUGGoYUbpc
	 O1DPpLxew1ntw==
Date: Thu, 30 Sep 2021 16:00:28 -0700 (PDT)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
cc: xen-devel@lists.xenproject.org, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Ian Jackson <iwj@xenproject.org>, Wei Liu <wl@xen.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Juergen Gross <jgross@suse.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Subject: Re: [PATCH V4 1/3] xen: Introduce "gpaddr_bits" field to
 XEN_SYSCTL_physinfo
In-Reply-To: <1632955927-27911-2-git-send-email-olekstysh@gmail.com>
Message-ID: <alpine.DEB.2.21.2109301600060.3209@sstabellini-ThinkPad-T480s>
References: <1632955927-27911-1-git-send-email-olekstysh@gmail.com> <1632955927-27911-2-git-send-email-olekstysh@gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 30 Sep 2021, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> We need to pass info about maximum supported guest address
> space size to the toolstack on Arm in order to properly
> calculate the base and size of the extended region (safe range)
> for the guest. The extended region is unused address space which
> could be safely used by domain for foreign/grant mappings on Arm.
> The extended region itself will be handled by the subsequents
> patch.
> 
> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
> hap_paddr_bits.
> 
> As we change the size of structure bump the interface version.
> 
> Suggested-by: Julien Grall <jgrall@amazon.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Reviewed-by: Michal Orzel <michal.orzel@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Please note, that review comments for the RFC version [1] haven't been addressed yet.
> It is not forgotten, some clarification is needed. It will be addressed for the next version.
> 
> [1] https://lore.kernel.org/xen-devel/973f5344-aa10-3ad6-ff02-ad5f358ad279@citrix.com/
> 
> Changes RFC -> V2:
>    - update patch subject/description
>    - replace arch-specific sub-struct with common gpaddr_bits
>      field and update code to reflect that
> 
> Changes V2 -> V3:
>    - make the field uint8_t and add uint8_t pad[7] after
>    - remove leading blanks in libxl.h
> 
> Changes V3 -> V4:
>    - also print gpaddr_bits from output_physinfo()
>    - add Michal's R-b
> ---
>  tools/include/libxl.h            | 7 +++++++
>  tools/libs/light/libxl.c         | 2 ++
>  tools/libs/light/libxl_types.idl | 2 ++
>  tools/xl/xl_info.c               | 2 ++
>  xen/arch/arm/sysctl.c            | 2 ++
>  xen/arch/x86/sysctl.c            | 2 ++
>  xen/include/public/sysctl.h      | 4 +++-
>  7 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index b9ba16d..63f9550 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -856,6 +856,13 @@ typedef struct libxl__ctx libxl_ctx;
>  #define LIBXL_HAVE_PHYSINFO_MAX_POSSIBLE_MFN 1
>  
>  /*
> + * LIBXL_HAVE_PHYSINFO_GPADDR_BITS
> + *
> + * If this is defined, libxl_physinfo has a "gpaddr_bits" field.
> + */
> +#define LIBXL_HAVE_PHYSINFO_GPADDR_BITS 1
> +
> +/*
>   * LIBXL_HAVE_DOMINFO_OUTSTANDING_MEMKB 1
>   *
>   * If this is defined, libxl_dominfo will contain a MemKB type field called
> diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c
> index 204eb0b..c86624f 100644
> --- a/tools/libs/light/libxl.c
> +++ b/tools/libs/light/libxl.c
> @@ -405,6 +405,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
>      physinfo->cap_vmtrace =
>          !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_vmtrace);
>  
> +    physinfo->gpaddr_bits = xcphysinfo.gpaddr_bits;
> +
>      GC_FREE;
>      return 0;
>  }
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 3f9fff6..bf27fe6 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -1061,6 +1061,8 @@ libxl_physinfo = Struct("physinfo", [
>      ("cap_shadow", bool),
>      ("cap_iommu_hap_pt_share", bool),
>      ("cap_vmtrace", bool),
> +
> +    ("gpaddr_bits", uint8),
>      ], dir=DIR_OUT)
>  
>  libxl_connectorinfo = Struct("connectorinfo", [
> diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
> index 8383e4a..dfbbeaa 100644
> --- a/tools/xl/xl_info.c
> +++ b/tools/xl/xl_info.c
> @@ -221,6 +221,8 @@ static void output_physinfo(void)
>           info.cap_vmtrace ? " vmtrace" : ""
>          );
>  
> +    maybe_printf("gpaddr_bits            : %d\n", info.gpaddr_bits);
> +
>      vinfo = libxl_get_version_info(ctx);
>      if (vinfo) {
>          i = (1 << 20) / vinfo->pagesize;
> diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
> index f87944e..91dca4f 100644
> --- a/xen/arch/arm/sysctl.c
> +++ b/xen/arch/arm/sysctl.c
> @@ -15,6 +15,8 @@
>  void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
>  {
>      pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
> +
> +    pi->gpaddr_bits = p2m_ipa_bits;
>  }
>  
>  long arch_do_sysctl(struct xen_sysctl *sysctl,
> diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
> index aff52a1..7b14865 100644
> --- a/xen/arch/x86/sysctl.c
> +++ b/xen/arch/x86/sysctl.c
> @@ -135,6 +135,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
>          pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
>      if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
>          pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow;
> +
> +    pi->gpaddr_bits = hap_paddr_bits;
>  }
>  
>  long arch_do_sysctl(
> diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
> index 039ccf8..0450a78 100644
> --- a/xen/include/public/sysctl.h
> +++ b/xen/include/public/sysctl.h
> @@ -35,7 +35,7 @@
>  #include "domctl.h"
>  #include "physdev.h"
>  
> -#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013
> +#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014
>  
>  /*
>   * Read console content from Xen buffer ring.
> @@ -120,6 +120,8 @@ struct xen_sysctl_physinfo {
>      uint64_aligned_t outstanding_pages;
>      uint64_aligned_t max_mfn; /* Largest possible MFN on this host */
>      uint32_t hw_cap[8];
> +    uint8_t gpaddr_bits;
> +    uint8_t pad[7];
>  };
>  
>  /*
> -- 
> 2.7.4
> 


